2024-09-16T09:08:27.143 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-16T09:08:27.160 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-16T09:08:27.236 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/gabrioux-2024-09-16_07:44:22-orch:cephadm-wip-guits-main-2024-09-13-1248-distro-default-smithi/7907025 branch: wip-guits-main-2024-09-13-1248 description: orch:cephadm/thrash/{0-distro/centos_9.stream_runc 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async root} email: gabrioux@ibm.com first_in_suite: false job_id: '7907025' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: gabrioux-2024-09-16_07:44:22-orch:cephadm-wip-guits-main-2024-09-13-1248-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-guits-main-2024-09-13-1248 ceph: conf: client.0: debug ms: 1 debug objecter: 20 debug rados: 20 global: ms type: async mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon min osdmap epochs: 50 mon osdmap full prune interval: 2 mon osdmap full prune min: 15 mon osdmap full prune txsize: 2 paxos service trim min: 10 osd: debug ms: 1 debug osd: 20 osd debug reject backfill probability: 0.3 osd delete sleep: 1 osd max backfills: 3 osd scrub max interval: 120 osd scrub min interval: 60 osd shutdown pgref assert: true osd snap trim sleep: 2 flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - CEPHADM_STRAY_DAEMON - CEPHADM_FAILED_DAEMON - but it is still running - objects unfound and apparently lost - \(POOL_APP_NOT_ENABLED\) log-only-match: - CEPHADM_ sha1: 8293d73f8690540e843a81caec373f9cc29cf705 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} cephadm: cephadm_mode: root install: ceph: flavor: default sha1: 8293d73f8690540e843a81caec373f9cc29cf705 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 workunit: branch: wip-guits-main-2024-09-13-1248 sha1: 8293d73f8690540e843a81caec373f9cc29cf705 owner: scheduled_gabrioux@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - ceph.rgw.foo.a - node-exporter.a - alertmanager.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - grafana.a - node-exporter.b - ceph.iscsi.iscsi.a seed: 6714 sha1: 8293d73f8690540e843a81caec373f9cc29cf705 sleep_before_teardown: 0 subset: 1/8 suite: orch:cephadm suite_branch: wip-guits-main-2024-09-13-1248 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_8293d73f8690540e843a81caec373f9cc29cf705/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 8293d73f8690540e843a81caec373f9cc29cf705 targets: smithi070.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAJnhDxwoaKCZL4OPm9UHbpvDDd7cb48N6zBFOAfQujiJO9ENpSJDp39SN6IsdVKb5htpw38HP1aDosyCs5qAs4= smithi084.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHFn8Wbj8T7BY9DlUQNcaJNrdR5urJG+XnVeCUTZWKnB/wACleXNq5e2zGJ6k1u9dayPDvN7pD3N1VnGV/6LCcg= tasks: - pexec: all: - sudo dnf remove nvme-cli -y - sudo dnf install runc nvmetcli nvme-cli -y - sudo sed -i 's/^#runtime = "crun"/runtime = "runc"/g' /usr/share/containers/containers.conf - sudo sed -i 's/runtime = "crun"/#runtime = "crun"/g' /usr/share/containers/containers.conf - install: null - cephadm: conf: mgr: debug mgr: 20 debug ms: 1 - thrashosds: chance_pgnum_grow: 1 chance_pgnum_shrink: 1 chance_pgpnum_fix: 1 timeout: 1200 - full_sequential: - radosbench: clients: - client.0 time: 90 - radosbench: clients: - client.0 time: 90 - radosbench: clients: - client.0 time: 90 - radosbench: clients: - client.0 time: 90 - radosbench: clients: - client.0 time: 90 teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-16_07:44:22 tube: smithi user: gabrioux verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-16T09:08:27.236 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_8293d73f8690540e843a81caec373f9cc29cf705/qa; will attempt to use it 2024-09-16T09:08:27.237 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_8293d73f8690540e843a81caec373f9cc29cf705/qa/tasks 2024-09-16T09:08:27.237 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-16T09:08:27.238 INFO:teuthology.task.internal:Checking packages... 2024-09-16T09:08:27.260 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '8293d73f8690540e843a81caec373f9cc29cf705' 2024-09-16T09:08:27.260 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-16T09:08:27.260 INFO:teuthology.packaging:ref: None 2024-09-16T09:08:27.260 INFO:teuthology.packaging:tag: None 2024-09-16T09:08:27.260 INFO:teuthology.packaging:branch: wip-guits-main-2024-09-13-1248 2024-09-16T09:08:27.261 INFO:teuthology.packaging:sha1: 8293d73f8690540e843a81caec373f9cc29cf705 2024-09-16T09:08:27.261 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-guits-main-2024-09-13-1248 2024-09-16T09:08:27.405 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-4917.g8293d73f 2024-09-16T09:08:27.407 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-16T09:08:27.413 INFO:teuthology.task.internal:no buildpackages task found 2024-09-16T09:08:27.413 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-16T09:08:27.461 INFO:teuthology.task.internal:Saving configuration 2024-09-16T09:08:27.474 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-16T09:08:27.482 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-16T09:08:27.508 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi070.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-09-16_07:44:22-orch:cephadm-wip-guits-main-2024-09-13-1248-distro-default-smithi/7907025', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-16 09:03:34.454144', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAJnhDxwoaKCZL4OPm9UHbpvDDd7cb48N6zBFOAfQujiJO9ENpSJDp39SN6IsdVKb5htpw38HP1aDosyCs5qAs4='} 2024-09-16T09:08:27.535 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi084.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-09-16_07:44:22-orch:cephadm-wip-guits-main-2024-09-13-1248-distro-default-smithi/7907025', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-16 09:03:34.455381', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHFn8Wbj8T7BY9DlUQNcaJNrdR5urJG+XnVeCUTZWKnB/wACleXNq5e2zGJ6k1u9dayPDvN7pD3N1VnGV/6LCcg='} 2024-09-16T09:08:27.535 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-16T09:08:27.541 INFO:teuthology.task.internal:roles: ubuntu@smithi070.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'ceph.rgw.foo.a', 'node-exporter.a', 'alertmanager.a'] 2024-09-16T09:08:27.542 INFO:teuthology.task.internal:roles: ubuntu@smithi084.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b', 'ceph.iscsi.iscsi.a'] 2024-09-16T09:08:27.542 INFO:teuthology.run_tasks:Running task console_log... 2024-09-16T09:08:27.625 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fedf8277be0>, signals=[15]) 2024-09-16T09:08:27.625 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-16T09:08:27.632 INFO:teuthology.task.internal:Opening connections... 2024-09-16T09:08:27.632 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi070.front.sepia.ceph.com 2024-09-16T09:08:27.633 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T09:08:27.708 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi084.front.sepia.ceph.com 2024-09-16T09:08:27.709 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi084.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T09:08:27.785 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-16T09:08:27.791 DEBUG:teuthology.orchestra.run.smithi070:> uname -m 2024-09-16T09:08:27.809 INFO:teuthology.orchestra.run.smithi070.stdout:x86_64 2024-09-16T09:08:27.809 DEBUG:teuthology.orchestra.run.smithi070:> cat /etc/os-release 2024-09-16T09:08:27.867 INFO:teuthology.orchestra.run.smithi070.stdout:NAME="CentOS Stream" 2024-09-16T09:08:27.867 INFO:teuthology.orchestra.run.smithi070.stdout:VERSION="9" 2024-09-16T09:08:27.867 INFO:teuthology.orchestra.run.smithi070.stdout:ID="centos" 2024-09-16T09:08:27.867 INFO:teuthology.orchestra.run.smithi070.stdout:ID_LIKE="rhel fedora" 2024-09-16T09:08:27.867 INFO:teuthology.orchestra.run.smithi070.stdout:VERSION_ID="9" 2024-09-16T09:08:27.867 INFO:teuthology.orchestra.run.smithi070.stdout:PLATFORM_ID="platform:el9" 2024-09-16T09:08:27.867 INFO:teuthology.orchestra.run.smithi070.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-16T09:08:27.867 INFO:teuthology.orchestra.run.smithi070.stdout:ANSI_COLOR="0;31" 2024-09-16T09:08:27.867 INFO:teuthology.orchestra.run.smithi070.stdout:LOGO="fedora-logo-icon" 2024-09-16T09:08:27.867 INFO:teuthology.orchestra.run.smithi070.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-16T09:08:27.867 INFO:teuthology.orchestra.run.smithi070.stdout:HOME_URL="https://centos.org/" 2024-09-16T09:08:27.868 INFO:teuthology.orchestra.run.smithi070.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-16T09:08:27.868 INFO:teuthology.orchestra.run.smithi070.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-16T09:08:27.868 INFO:teuthology.orchestra.run.smithi070.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-16T09:08:27.868 INFO:teuthology.lock.ops:Updating smithi070.front.sepia.ceph.com on lock server 2024-09-16T09:08:27.889 DEBUG:teuthology.orchestra.run.smithi084:> uname -m 2024-09-16T09:08:27.907 INFO:teuthology.orchestra.run.smithi084.stdout:x86_64 2024-09-16T09:08:27.908 DEBUG:teuthology.orchestra.run.smithi084:> cat /etc/os-release 2024-09-16T09:08:27.966 INFO:teuthology.orchestra.run.smithi084.stdout:NAME="CentOS Stream" 2024-09-16T09:08:27.966 INFO:teuthology.orchestra.run.smithi084.stdout:VERSION="9" 2024-09-16T09:08:27.966 INFO:teuthology.orchestra.run.smithi084.stdout:ID="centos" 2024-09-16T09:08:27.966 INFO:teuthology.orchestra.run.smithi084.stdout:ID_LIKE="rhel fedora" 2024-09-16T09:08:27.966 INFO:teuthology.orchestra.run.smithi084.stdout:VERSION_ID="9" 2024-09-16T09:08:27.966 INFO:teuthology.orchestra.run.smithi084.stdout:PLATFORM_ID="platform:el9" 2024-09-16T09:08:27.966 INFO:teuthology.orchestra.run.smithi084.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-16T09:08:27.966 INFO:teuthology.orchestra.run.smithi084.stdout:ANSI_COLOR="0;31" 2024-09-16T09:08:27.967 INFO:teuthology.orchestra.run.smithi084.stdout:LOGO="fedora-logo-icon" 2024-09-16T09:08:27.967 INFO:teuthology.orchestra.run.smithi084.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-16T09:08:27.967 INFO:teuthology.orchestra.run.smithi084.stdout:HOME_URL="https://centos.org/" 2024-09-16T09:08:27.967 INFO:teuthology.orchestra.run.smithi084.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-16T09:08:27.967 INFO:teuthology.orchestra.run.smithi084.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-16T09:08:27.967 INFO:teuthology.orchestra.run.smithi084.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-16T09:08:27.967 INFO:teuthology.lock.ops:Updating smithi084.front.sepia.ceph.com on lock server 2024-09-16T09:08:27.988 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-16T09:08:27.997 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-16T09:08:28.008 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-16T09:08:28.008 DEBUG:teuthology.orchestra.run.smithi070:> test '!' -e /home/ubuntu/cephtest 2024-09-16T09:08:28.011 DEBUG:teuthology.orchestra.run.smithi084:> test '!' -e /home/ubuntu/cephtest 2024-09-16T09:08:28.028 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-16T09:08:28.035 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-16T09:08:28.035 DEBUG:teuthology.orchestra.run.smithi070:> test -z $(ls -A /var/lib/ceph) 2024-09-16T09:08:28.068 DEBUG:teuthology.orchestra.run.smithi084:> test -z $(ls -A /var/lib/ceph) 2024-09-16T09:08:28.148 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-16T09:08:28.209 INFO:teuthology.run_tasks:Running task kernel... 2024-09-16T09:08:28.225 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-16T09:08:28.225 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-16T09:08:28.226 DEBUG:teuthology.orchestra.run.smithi070:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-16T09:08:28.226 DEBUG:teuthology.orchestra.run.smithi084:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-16T09:08:28.242 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:08:28.243 DEBUG:teuthology.orchestra.run.smithi070:> uname -r 2024-09-16T09:08:28.244 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:08:28.244 DEBUG:teuthology.orchestra.run.smithi084:> uname -r 2024-09-16T09:08:28.301 INFO:teuthology.orchestra.run.smithi070.stdout:5.14.0-503.el9.x86_64 2024-09-16T09:08:28.301 INFO:teuthology.task.kernel:Running kernel on smithi070: 5.14.0-503.el9.x86_64 2024-09-16T09:08:28.301 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum install -y kernel 2024-09-16T09:08:28.303 INFO:teuthology.orchestra.run.smithi084.stdout:5.14.0-503.el9.x86_64 2024-09-16T09:08:28.303 INFO:teuthology.task.kernel:Running kernel on smithi084: 5.14.0-503.el9.x86_64 2024-09-16T09:08:28.303 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum install -y kernel 2024-09-16T09:08:31.787 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - BaseOS 4.8 MB/s | 8.2 MB 00:01 2024-09-16T09:08:31.868 INFO:teuthology.orchestra.run.smithi084.stdout:CentOS Stream 9 - BaseOS 4.4 MB/s | 8.2 MB 00:01 2024-09-16T09:08:33.819 INFO:teuthology.orchestra.run.smithi084.stdout:CentOS Stream 9 - AppStream 23 MB/s | 20 MB 00:00 2024-09-16T09:08:36.404 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - AppStream 5.8 MB/s | 20 MB 00:03 2024-09-16T09:08:38.961 INFO:teuthology.orchestra.run.smithi084.stdout:CentOS Stream 9 - CRB 4.7 MB/s | 6.5 MB 00:01 2024-09-16T09:08:40.412 INFO:teuthology.orchestra.run.smithi084.stdout:CentOS Stream 9 - Extras packages 71 kB/s | 19 kB 00:00 2024-09-16T09:08:41.021 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - CRB 7.9 MB/s | 6.5 MB 00:00 2024-09-16T09:08:41.241 INFO:teuthology.orchestra.run.smithi084.stdout:Extra Packages for Enterprise Linux 32 MB/s | 23 MB 00:00 2024-09-16T09:08:42.580 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - Extras packages 70 kB/s | 19 kB 00:00 2024-09-16T09:08:43.426 INFO:teuthology.orchestra.run.smithi070.stdout:Extra Packages for Enterprise Linux 31 MB/s | 23 MB 00:00 2024-09-16T09:08:46.679 INFO:teuthology.orchestra.run.smithi084.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-16T09:08:48.677 INFO:teuthology.orchestra.run.smithi084.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-16T09:08:48.677 INFO:teuthology.orchestra.run.smithi084.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-16T09:08:48.766 INFO:teuthology.orchestra.run.smithi084.stdout:Dependencies resolved. 2024-09-16T09:08:48.773 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-09-16T09:08:48.773 INFO:teuthology.orchestra.run.smithi084.stdout: Package Architecture Version Repository Size 2024-09-16T09:08:48.774 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-09-16T09:08:48.774 INFO:teuthology.orchestra.run.smithi084.stdout:Installing: 2024-09-16T09:08:48.774 INFO:teuthology.orchestra.run.smithi084.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-16T09:08:48.774 INFO:teuthology.orchestra.run.smithi084.stdout:Installing dependencies: 2024-09-16T09:08:48.774 INFO:teuthology.orchestra.run.smithi084.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-16T09:08:48.774 INFO:teuthology.orchestra.run.smithi084.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-16T09:08:48.774 INFO:teuthology.orchestra.run.smithi084.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-16T09:08:48.774 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-16T09:08:48.774 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction Summary 2024-09-16T09:08:48.774 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-09-16T09:08:48.774 INFO:teuthology.orchestra.run.smithi084.stdout:Install 4 Packages 2024-09-16T09:08:48.774 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-16T09:08:48.775 INFO:teuthology.orchestra.run.smithi084.stdout:Total download size: 79 M 2024-09-16T09:08:48.775 INFO:teuthology.orchestra.run.smithi084.stdout:Installed size: 126 M 2024-09-16T09:08:48.775 INFO:teuthology.orchestra.run.smithi084.stdout:Downloading Packages: 2024-09-16T09:08:48.864 INFO:teuthology.orchestra.run.smithi070.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-09-16T09:08:48.906 INFO:teuthology.orchestra.run.smithi084.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 267 kB/s | 23 kB 00:00 2024-09-16T09:08:49.247 INFO:teuthology.orchestra.run.smithi084.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 37 MB/s | 16 MB 00:00 2024-09-16T09:08:49.632 INFO:teuthology.orchestra.run.smithi084.stdout:(3/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 43 MB/s | 35 MB 00:00 2024-09-16T09:08:50.090 INFO:teuthology.orchestra.run.smithi084.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 24 MB/s | 29 MB 00:01 2024-09-16T09:08:50.090 INFO:teuthology.orchestra.run.smithi084.stdout:-------------------------------------------------------------------------------- 2024-09-16T09:08:50.090 INFO:teuthology.orchestra.run.smithi084.stdout:Total 60 MB/s | 79 MB 00:01 2024-09-16T09:08:50.766 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction check 2024-09-16T09:08:50.896 INFO:teuthology.orchestra.run.smithi070.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-16T09:08:50.896 INFO:teuthology.orchestra.run.smithi070.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-16T09:08:50.987 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-16T09:08:50.994 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-16T09:08:50.994 INFO:teuthology.orchestra.run.smithi070.stdout: Package Architecture Version Repository Size 2024-09-16T09:08:50.995 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-16T09:08:50.995 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-09-16T09:08:50.995 INFO:teuthology.orchestra.run.smithi070.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-16T09:08:50.995 INFO:teuthology.orchestra.run.smithi070.stdout:Installing dependencies: 2024-09-16T09:08:50.995 INFO:teuthology.orchestra.run.smithi070.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-16T09:08:50.995 INFO:teuthology.orchestra.run.smithi070.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-16T09:08:50.995 INFO:teuthology.orchestra.run.smithi070.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-16T09:08:50.995 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:08:50.995 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-09-16T09:08:50.995 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-16T09:08:50.995 INFO:teuthology.orchestra.run.smithi070.stdout:Install 4 Packages 2024-09-16T09:08:50.995 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:08:50.996 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 79 M 2024-09-16T09:08:50.996 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 126 M 2024-09-16T09:08:50.996 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-09-16T09:08:51.109 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction check succeeded. 2024-09-16T09:08:51.109 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction test 2024-09-16T09:08:51.348 INFO:teuthology.orchestra.run.smithi070.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 74 kB/s | 23 kB 00:00 2024-09-16T09:08:52.046 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction test succeeded. 2024-09-16T09:08:52.046 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction 2024-09-16T09:08:54.475 INFO:teuthology.orchestra.run.smithi084.stdout: Preparing : 1/1 2024-09-16T09:08:55.123 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-16T09:08:55.225 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-16T09:08:58.026 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-16T09:08:58.152 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-16T09:09:02.981 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-16T09:09:03.094 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-16T09:09:07.088 INFO:teuthology.orchestra.run.smithi070.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 999 kB/s | 16 MB 00:16 2024-09-16T09:09:07.934 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-16T09:09:25.377 INFO:teuthology.orchestra.run.smithi070.stdout:(3/4): kernel-modules-core-5.14.0-505.el9.x86_6 859 kB/s | 29 MB 00:34 2024-09-16T09:09:33.267 INFO:teuthology.orchestra.run.smithi070.stdout:(4/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 838 kB/s | 35 MB 00:42 2024-09-16T09:09:33.267 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-09-16T09:09:33.267 INFO:teuthology.orchestra.run.smithi070.stdout:Total 1.9 MB/s | 79 MB 00:42 2024-09-16T09:09:33.953 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-09-16T09:09:34.322 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-09-16T09:09:34.322 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-09-16T09:09:35.293 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-09-16T09:09:35.294 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-09-16T09:09:37.751 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-09-16T09:09:38.461 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-16T09:09:38.542 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-16T09:09:41.321 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-16T09:09:41.454 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-16T09:09:46.250 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-16T09:09:46.286 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-16T09:09:46.288 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-16T09:09:46.403 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-16T09:09:46.626 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-16T09:09:46.626 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-16T09:09:46.626 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-16T09:09:46.626 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-16T09:09:47.510 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-16T09:09:47.510 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-16T09:09:47.510 INFO:teuthology.orchestra.run.smithi084.stdout:Installed: 2024-09-16T09:09:47.510 INFO:teuthology.orchestra.run.smithi084.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-16T09:09:47.511 INFO:teuthology.orchestra.run.smithi084.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-16T09:09:47.511 INFO:teuthology.orchestra.run.smithi084.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-16T09:09:47.511 INFO:teuthology.orchestra.run.smithi084.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-16T09:09:47.511 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-16T09:09:47.511 INFO:teuthology.orchestra.run.smithi084.stdout:Complete! 2024-09-16T09:09:47.874 DEBUG:teuthology.orchestra.run.smithi084:> rpm -q kernel | sort -rV | head -n 1 2024-09-16T09:09:47.968 INFO:teuthology.orchestra.run.smithi084.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-16T09:09:47.968 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-16T09:09:47.969 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-16T09:09:47.969 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-16T09:09:47.969 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-16T09:09:47.969 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-16T09:09:47.969 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi084.front.sepia.ceph.com, path=None, version=distro) 2024-09-16T09:09:47.969 DEBUG:teuthology.orchestra.run.smithi084:> rpm -q kernel | sort -rV | head -n 1 2024-09-16T09:09:48.028 INFO:teuthology.orchestra.run.smithi084.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-16T09:09:48.029 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-16T09:09:48.029 DEBUG:teuthology.orchestra.run.smithi084:> sudo rpm -qi grub2-tools 2024-09-16T09:09:48.060 INFO:teuthology.orchestra.run.smithi084.stdout:Name : grub2-tools 2024-09-16T09:09:48.060 INFO:teuthology.orchestra.run.smithi084.stdout:Epoch : 1 2024-09-16T09:09:48.061 INFO:teuthology.orchestra.run.smithi084.stdout:Version : 2.06 2024-09-16T09:09:48.061 INFO:teuthology.orchestra.run.smithi084.stdout:Release : 82.el9 2024-09-16T09:09:48.061 INFO:teuthology.orchestra.run.smithi084.stdout:Architecture: x86_64 2024-09-16T09:09:48.061 INFO:teuthology.orchestra.run.smithi084.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-16T09:09:48.061 INFO:teuthology.orchestra.run.smithi084.stdout:Group : Unspecified 2024-09-16T09:09:48.061 INFO:teuthology.orchestra.run.smithi084.stdout:Size : 8274670 2024-09-16T09:09:48.061 INFO:teuthology.orchestra.run.smithi084.stdout:License : GPLv3+ 2024-09-16T09:09:48.061 INFO:teuthology.orchestra.run.smithi084.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-16T09:09:48.061 INFO:teuthology.orchestra.run.smithi084.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-16T09:09:48.061 INFO:teuthology.orchestra.run.smithi084.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-16T09:09:48.061 INFO:teuthology.orchestra.run.smithi084.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-16T09:09:48.061 INFO:teuthology.orchestra.run.smithi084.stdout:Packager : builder@centos.org 2024-09-16T09:09:48.061 INFO:teuthology.orchestra.run.smithi084.stdout:Vendor : CentOS 2024-09-16T09:09:48.062 INFO:teuthology.orchestra.run.smithi084.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-16T09:09:48.062 INFO:teuthology.orchestra.run.smithi084.stdout:Summary : Support tools for GRUB. 2024-09-16T09:09:48.062 INFO:teuthology.orchestra.run.smithi084.stdout:Description : 2024-09-16T09:09:48.062 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-16T09:09:48.062 INFO:teuthology.orchestra.run.smithi084.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-16T09:09:48.062 INFO:teuthology.orchestra.run.smithi084.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-16T09:09:48.062 INFO:teuthology.orchestra.run.smithi084.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-16T09:09:48.062 INFO:teuthology.orchestra.run.smithi084.stdout:hardware devices. 2024-09-16T09:09:48.062 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-16T09:09:48.062 INFO:teuthology.orchestra.run.smithi084.stdout:This subpackage provides tools for support of all platforms. 2024-09-16T09:09:48.064 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-16T09:09:48.064 INFO:teuthology.task.kernel:Updating grub on smithi084 to boot 5.14.0-505.el9.x86_64 2024-09-16T09:09:48.064 DEBUG:teuthology.orchestra.run.smithi084:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-16T09:09:48.694 INFO:teuthology.orchestra.run.smithi084.stderr:Generating grub configuration file ... 2024-09-16T09:09:50.045 INFO:teuthology.orchestra.run.smithi084.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-16T09:09:50.082 INFO:teuthology.orchestra.run.smithi084.stderr:done 2024-09-16T09:09:50.084 DEBUG:teuthology.orchestra.run.smithi084:> mktemp 2024-09-16T09:09:50.100 INFO:teuthology.orchestra.run.smithi084.stdout:/tmp/tmp.9dZKg29XTs 2024-09-16T09:09:50.100 DEBUG:teuthology.orchestra.run.smithi084:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.9dZKg29XTs 2024-09-16T09:09:50.170 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod 0666 /tmp/tmp.9dZKg29XTs 2024-09-16T09:09:50.302 DEBUG:teuthology.orchestra.remote:smithi084:/tmp/tmp.9dZKg29XTs is 6KB 2024-09-16T09:09:50.352 DEBUG:teuthology.orchestra.run.smithi084:> rm -fr /tmp/tmp.9dZKg29XTs 2024-09-16T09:09:50.367 DEBUG:teuthology.orchestra.run.smithi084:> sudo /bin/ls /boot/loader/entries || true 2024-09-16T09:09:50.430 INFO:teuthology.orchestra.run.smithi084.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-16T09:09:50.430 INFO:teuthology.orchestra.run.smithi084.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-16T09:09:50.430 INFO:teuthology.orchestra.run.smithi084.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-16T09:09:50.430 INFO:teuthology.orchestra.run.smithi084.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-16T09:09:50.431 DEBUG:teuthology.orchestra.run.smithi084:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-16T09:09:50.583 DEBUG:teuthology.orchestra.run.smithi084:> sudo shutdown -r now 2024-09-16T09:09:51.228 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-16T09:10:20.586 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-16T09:10:20.587 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi084.front.sepia.ceph.com' 2024-09-16T09:10:20.589 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi084.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T09:10:29.124 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-16T09:10:29.163 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-16T09:10:29.506 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-16T09:10:29.506 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-16T09:10:29.506 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-16T09:10:29.506 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-16T09:10:30.360 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-16T09:10:30.360 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:10:30.360 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-09-16T09:10:30.361 INFO:teuthology.orchestra.run.smithi070.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-16T09:10:30.361 INFO:teuthology.orchestra.run.smithi070.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-16T09:10:30.361 INFO:teuthology.orchestra.run.smithi070.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-16T09:10:30.361 INFO:teuthology.orchestra.run.smithi070.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-16T09:10:30.361 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:10:30.361 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-16T09:10:30.732 DEBUG:teuthology.orchestra.run.smithi070:> rpm -q kernel | sort -rV | head -n 1 2024-09-16T09:10:30.827 INFO:teuthology.orchestra.run.smithi070.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-16T09:10:30.827 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-16T09:10:30.827 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-16T09:10:30.827 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-16T09:10:30.827 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-16T09:10:30.827 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-16T09:10:30.828 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi070.front.sepia.ceph.com, path=None, version=distro) 2024-09-16T09:10:30.828 DEBUG:teuthology.orchestra.run.smithi070:> rpm -q kernel | sort -rV | head -n 1 2024-09-16T09:10:30.886 INFO:teuthology.orchestra.run.smithi070.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-16T09:10:30.887 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-16T09:10:30.887 DEBUG:teuthology.orchestra.run.smithi070:> sudo rpm -qi grub2-tools 2024-09-16T09:10:30.919 INFO:teuthology.orchestra.run.smithi070.stdout:Name : grub2-tools 2024-09-16T09:10:30.920 INFO:teuthology.orchestra.run.smithi070.stdout:Epoch : 1 2024-09-16T09:10:30.920 INFO:teuthology.orchestra.run.smithi070.stdout:Version : 2.06 2024-09-16T09:10:30.920 INFO:teuthology.orchestra.run.smithi070.stdout:Release : 82.el9 2024-09-16T09:10:30.920 INFO:teuthology.orchestra.run.smithi070.stdout:Architecture: x86_64 2024-09-16T09:10:30.920 INFO:teuthology.orchestra.run.smithi070.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-16T09:10:30.920 INFO:teuthology.orchestra.run.smithi070.stdout:Group : Unspecified 2024-09-16T09:10:30.920 INFO:teuthology.orchestra.run.smithi070.stdout:Size : 8274670 2024-09-16T09:10:30.920 INFO:teuthology.orchestra.run.smithi070.stdout:License : GPLv3+ 2024-09-16T09:10:30.920 INFO:teuthology.orchestra.run.smithi070.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-16T09:10:30.920 INFO:teuthology.orchestra.run.smithi070.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-16T09:10:30.921 INFO:teuthology.orchestra.run.smithi070.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-16T09:10:30.921 INFO:teuthology.orchestra.run.smithi070.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-16T09:10:30.921 INFO:teuthology.orchestra.run.smithi070.stdout:Packager : builder@centos.org 2024-09-16T09:10:30.921 INFO:teuthology.orchestra.run.smithi070.stdout:Vendor : CentOS 2024-09-16T09:10:30.921 INFO:teuthology.orchestra.run.smithi070.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-16T09:10:30.921 INFO:teuthology.orchestra.run.smithi070.stdout:Summary : Support tools for GRUB. 2024-09-16T09:10:30.921 INFO:teuthology.orchestra.run.smithi070.stdout:Description : 2024-09-16T09:10:30.921 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:10:30.921 INFO:teuthology.orchestra.run.smithi070.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-16T09:10:30.921 INFO:teuthology.orchestra.run.smithi070.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-16T09:10:30.921 INFO:teuthology.orchestra.run.smithi070.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-16T09:10:30.922 INFO:teuthology.orchestra.run.smithi070.stdout:hardware devices. 2024-09-16T09:10:30.922 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:10:30.922 INFO:teuthology.orchestra.run.smithi070.stdout:This subpackage provides tools for support of all platforms. 2024-09-16T09:10:30.924 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-16T09:10:30.924 INFO:teuthology.task.kernel:Updating grub on smithi070 to boot 5.14.0-505.el9.x86_64 2024-09-16T09:10:30.924 DEBUG:teuthology.orchestra.run.smithi070:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-16T09:10:31.604 INFO:teuthology.orchestra.run.smithi070.stderr:Generating grub configuration file ... 2024-09-16T09:10:32.965 INFO:teuthology.orchestra.run.smithi070.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-16T09:10:32.993 INFO:teuthology.orchestra.run.smithi070.stderr:done 2024-09-16T09:10:32.995 DEBUG:teuthology.orchestra.run.smithi070:> mktemp 2024-09-16T09:10:33.013 INFO:teuthology.orchestra.run.smithi070.stdout:/tmp/tmp.a23fC4CJpG 2024-09-16T09:10:33.013 DEBUG:teuthology.orchestra.run.smithi070:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.a23fC4CJpG 2024-09-16T09:10:33.081 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 0666 /tmp/tmp.a23fC4CJpG 2024-09-16T09:10:33.223 DEBUG:teuthology.orchestra.remote:smithi070:/tmp/tmp.a23fC4CJpG is 6KB 2024-09-16T09:10:33.272 DEBUG:teuthology.orchestra.run.smithi070:> rm -fr /tmp/tmp.a23fC4CJpG 2024-09-16T09:10:33.287 DEBUG:teuthology.orchestra.run.smithi070:> sudo /bin/ls /boot/loader/entries || true 2024-09-16T09:10:33.360 INFO:teuthology.orchestra.run.smithi070.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-16T09:10:33.360 INFO:teuthology.orchestra.run.smithi070.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-16T09:10:33.360 INFO:teuthology.orchestra.run.smithi070.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-16T09:10:33.360 INFO:teuthology.orchestra.run.smithi070.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-16T09:10:33.361 DEBUG:teuthology.orchestra.run.smithi070:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-16T09:10:33.524 DEBUG:teuthology.orchestra.run.smithi070:> sudo shutdown -r now 2024-09-16T09:10:55.041 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.84 2024-09-16T09:11:03.533 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-16T09:11:03.533 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi070.front.sepia.ceph.com' 2024-09-16T09:11:03.534 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T09:11:04.042 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi084.front.sepia.ceph.com' 2024-09-16T09:11:04.043 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi084.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T09:11:22.433 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.84 2024-09-16T09:11:34.445 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi084.front.sepia.ceph.com' 2024-09-16T09:11:34.446 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi084.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T09:11:34.832 DEBUG:teuthology.orchestra.run.smithi084:> true 2024-09-16T09:11:35.277 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi084.front.sepia.ceph.com' 2024-09-16T09:11:35.277 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-505.el9.x86_64"... 2024-09-16T09:11:35.277 DEBUG:teuthology.orchestra.run.smithi084:> uname -r 2024-09-16T09:11:35.292 INFO:teuthology.orchestra.run.smithi084.stdout:5.14.0-505.el9.x86_64 2024-09-16T09:11:35.292 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-16T09:11:35.292 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-16T09:11:35.292 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-16T09:11:36.294 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-16T09:11:36.294 DEBUG:teuthology.orchestra.run.smithi084:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-16T09:11:36.389 INFO:teuthology.orchestra.run.smithi084.stdout:ttyS1 2024-09-16T09:11:36.411 DEBUG:teuthology.parallel:result is None 2024-09-16T09:12:03.536 DEBUG:teuthology.orchestra.remote:timed out 2024-09-16T09:12:12.537 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi070.front.sepia.ceph.com' 2024-09-16T09:12:12.538 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T09:12:13.034 DEBUG:teuthology.orchestra.run.smithi070:> true 2024-09-16T09:12:13.569 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi070.front.sepia.ceph.com' 2024-09-16T09:12:13.569 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-505.el9.x86_64"... 2024-09-16T09:12:13.570 DEBUG:teuthology.orchestra.run.smithi070:> uname -r 2024-09-16T09:12:13.584 INFO:teuthology.orchestra.run.smithi070.stdout:5.14.0-505.el9.x86_64 2024-09-16T09:12:13.584 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-16T09:12:13.584 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-16T09:12:13.585 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-16T09:12:14.585 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-16T09:12:14.585 DEBUG:teuthology.orchestra.run.smithi070:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-16T09:12:14.670 INFO:teuthology.orchestra.run.smithi070.stdout:ttyS1 2024-09-16T09:12:14.713 DEBUG:teuthology.parallel:result is None 2024-09-16T09:12:14.714 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-16T09:12:14.724 INFO:teuthology.task.internal:Creating test directory... 2024-09-16T09:12:14.724 DEBUG:teuthology.orchestra.run.smithi070:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-16T09:12:14.727 DEBUG:teuthology.orchestra.run.smithi084:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-16T09:12:14.747 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-16T09:12:14.795 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-16T09:12:14.802 INFO:teuthology.task.internal:Creating archive directory... 2024-09-16T09:12:14.803 DEBUG:teuthology.orchestra.run.smithi070:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-16T09:12:14.805 DEBUG:teuthology.orchestra.run.smithi084:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-16T09:12:14.840 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-16T09:12:14.846 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-16T09:12:14.846 DEBUG:teuthology.orchestra.run.smithi070:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-16T09:12:14.862 DEBUG:teuthology.orchestra.run.smithi084:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-16T09:12:14.893 INFO:teuthology.orchestra.run.smithi070.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T09:12:14.904 INFO:teuthology.orchestra.run.smithi070.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T09:12:14.914 INFO:teuthology.orchestra.run.smithi084.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T09:12:14.925 INFO:teuthology.orchestra.run.smithi084.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T09:12:14.927 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-16T09:12:14.934 INFO:teuthology.task.internal:Configuring sudo... 2024-09-16T09:12:14.934 DEBUG:teuthology.orchestra.run.smithi070:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-16T09:12:14.949 DEBUG:teuthology.orchestra.run.smithi084:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-16T09:12:14.995 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-16T09:12:15.005 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-16T09:12:15.005 DEBUG:teuthology.orchestra.run.smithi070:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-16T09:12:15.017 DEBUG:teuthology.orchestra.run.smithi084:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-16T09:12:15.051 DEBUG:teuthology.orchestra.run.smithi070:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T09:12:15.117 DEBUG:teuthology.orchestra.run.smithi070:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T09:12:15.163 DEBUG:teuthology.orchestra.run.smithi070:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T09:12:15.250 DEBUG:teuthology.orchestra.run.smithi070:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T09:12:15.293 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-16T09:12:15.293 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-16T09:12:15.362 DEBUG:teuthology.orchestra.run.smithi084:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T09:12:15.409 DEBUG:teuthology.orchestra.run.smithi084:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T09:12:15.458 DEBUG:teuthology.orchestra.run.smithi084:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T09:12:15.545 DEBUG:teuthology.orchestra.run.smithi084:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T09:12:15.587 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-09-16T09:12:15.587 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-16T09:12:15.654 DEBUG:teuthology.orchestra.run.smithi070:> sudo service rsyslog restart 2024-09-16T09:12:15.657 DEBUG:teuthology.orchestra.run.smithi084:> sudo service rsyslog restart 2024-09-16T09:12:15.702 INFO:teuthology.orchestra.run.smithi070.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T09:12:15.750 INFO:teuthology.orchestra.run.smithi084.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T09:12:16.192 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-16T09:12:16.200 INFO:teuthology.task.internal:Starting timer... 2024-09-16T09:12:16.200 INFO:teuthology.run_tasks:Running task pcp... 2024-09-16T09:12:16.210 INFO:teuthology.run_tasks:Running task selinux... 2024-09-16T09:12:16.218 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-09-16T09:12:16.218 DEBUG:teuthology.orchestra.run.smithi070:> sudo service auditd rotate 2024-09-16T09:12:16.262 INFO:teuthology.orchestra.run.smithi070.stdout:Rotating logs: 2024-09-16T09:12:16.264 DEBUG:teuthology.orchestra.run.smithi084:> sudo service auditd rotate 2024-09-16T09:12:16.321 INFO:teuthology.orchestra.run.smithi084.stdout:Rotating logs: 2024-09-16T09:12:16.325 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-16T09:12:16.325 DEBUG:teuthology.orchestra.run.smithi070:> /usr/sbin/getenforce 2024-09-16T09:12:16.350 INFO:teuthology.orchestra.run.smithi070.stdout:Permissive 2024-09-16T09:12:16.350 DEBUG:teuthology.orchestra.run.smithi084:> /usr/sbin/getenforce 2024-09-16T09:12:16.401 INFO:teuthology.orchestra.run.smithi084.stdout:Permissive 2024-09-16T09:12:16.401 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi070.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi084.front.sepia.ceph.com': 'permissive'} 2024-09-16T09:12:16.401 DEBUG:teuthology.orchestra.run.smithi070:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-16T09:12:16.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:12:16.428 DEBUG:teuthology.orchestra.run.smithi084:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-16T09:12:16.455 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:12:16.456 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-16T09:12:16.456 DEBUG:teuthology.orchestra.run.smithi070:> sudo /usr/sbin/setenforce permissive 2024-09-16T09:12:16.512 DEBUG:teuthology.orchestra.run.smithi084:> sudo /usr/sbin/setenforce permissive 2024-09-16T09:12:16.544 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-16T09:12:16.553 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-16T09:12:16.565 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-16T09:12:16.565 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi070.front.sepia.ceph.com,smithi084.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-16T09:16:01.193 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi070.front.sepia.ceph.com'), Remote(name='ubuntu@smithi084.front.sepia.ceph.com')] 2024-09-16T09:16:01.195 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi070.front.sepia.ceph.com' 2024-09-16T09:16:01.196 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi070.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T09:16:01.293 DEBUG:teuthology.orchestra.run.smithi070:> true 2024-09-16T09:16:01.372 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi070.front.sepia.ceph.com' 2024-09-16T09:16:01.372 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi084.front.sepia.ceph.com' 2024-09-16T09:16:01.372 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi084.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T09:16:01.447 DEBUG:teuthology.orchestra.run.smithi084:> true 2024-09-16T09:16:01.488 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi084.front.sepia.ceph.com' 2024-09-16T09:16:01.488 INFO:teuthology.run_tasks:Running task clock... 2024-09-16T09:16:01.498 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-16T09:16:01.499 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-16T09:16:01.499 DEBUG:teuthology.orchestra.run.smithi070:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T09:16:01.501 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-16T09:16:01.501 DEBUG:teuthology.orchestra.run.smithi084:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T09:16:01.528 INFO:teuthology.orchestra.run.smithi070.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-16T09:16:01.542 INFO:teuthology.orchestra.run.smithi070.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-16T09:16:01.557 INFO:teuthology.orchestra.run.smithi084.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-16T09:16:01.566 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: ntpd: command not found 2024-09-16T09:16:01.571 INFO:teuthology.orchestra.run.smithi084.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-16T09:16:01.577 INFO:teuthology.orchestra.run.smithi070.stdout:506 Cannot talk to daemon 2024-09-16T09:16:01.591 INFO:teuthology.orchestra.run.smithi070.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-16T09:16:01.596 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: ntpd: command not found 2024-09-16T09:16:01.605 INFO:teuthology.orchestra.run.smithi070.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-16T09:16:01.607 INFO:teuthology.orchestra.run.smithi084.stdout:506 Cannot talk to daemon 2024-09-16T09:16:01.621 INFO:teuthology.orchestra.run.smithi084.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-16T09:16:01.635 INFO:teuthology.orchestra.run.smithi084.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-16T09:16:01.660 INFO:teuthology.orchestra.run.smithi070.stderr:bash: line 1: ntpq: command not found 2024-09-16T09:16:01.664 INFO:teuthology.orchestra.run.smithi070.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T09:16:01.664 INFO:teuthology.orchestra.run.smithi070.stdout:=============================================================================== 2024-09-16T09:16:01.664 INFO:teuthology.orchestra.run.smithi070.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T09:16:01.664 INFO:teuthology.orchestra.run.smithi070.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T09:16:01.664 INFO:teuthology.orchestra.run.smithi070.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T09:16:01.664 INFO:teuthology.orchestra.run.smithi070.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T09:16:01.700 INFO:teuthology.orchestra.run.smithi084.stderr:bash: line 1: ntpq: command not found 2024-09-16T09:16:01.703 INFO:teuthology.orchestra.run.smithi084.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T09:16:01.704 INFO:teuthology.orchestra.run.smithi084.stdout:=============================================================================== 2024-09-16T09:16:01.704 INFO:teuthology.orchestra.run.smithi084.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T09:16:01.704 INFO:teuthology.orchestra.run.smithi084.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T09:16:01.704 INFO:teuthology.orchestra.run.smithi084.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T09:16:01.704 INFO:teuthology.orchestra.run.smithi084.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T09:16:01.705 INFO:teuthology.run_tasks:Running task pexec... 2024-09-16T09:16:01.716 INFO:teuthology.task.pexec:Executing custom commands... 2024-09-16T09:16:01.716 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi070.front.sepia.ceph.com 2024-09-16T09:16:01.717 DEBUG:teuthology.orchestra.run.smithi070:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-09-16T09:16:01.717 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi084.front.sepia.ceph.com 2024-09-16T09:16:01.717 DEBUG:teuthology.orchestra.run.smithi084:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-09-16T09:16:02.065 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-16T09:16:02.081 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-16T09:16:02.081 INFO:teuthology.orchestra.run.smithi070.stdout: Package Architecture Version Repository Size 2024-09-16T09:16:02.081 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-16T09:16:02.081 INFO:teuthology.orchestra.run.smithi070.stdout:Removing: 2024-09-16T09:16:02.081 INFO:teuthology.orchestra.run.smithi070.stdout: nvme-cli x86_64 2.9.1-6.el9 @baseos 5.3 M 2024-09-16T09:16:02.081 INFO:teuthology.orchestra.run.smithi070.stdout:Removing unused dependencies: 2024-09-16T09:16:02.082 INFO:teuthology.orchestra.run.smithi070.stdout: libnvme x86_64 1.9-3.el9 @baseos 273 k 2024-09-16T09:16:02.082 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:16:02.082 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-09-16T09:16:02.082 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-16T09:16:02.082 INFO:teuthology.orchestra.run.smithi070.stdout:Remove 2 Packages 2024-09-16T09:16:02.082 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:16:02.082 INFO:teuthology.orchestra.run.smithi070.stdout:Freed space: 5.5 M 2024-09-16T09:16:02.082 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-09-16T09:16:02.085 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-09-16T09:16:02.085 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-09-16T09:16:02.093 INFO:teuthology.orchestra.run.smithi084.stdout:Dependencies resolved. 2024-09-16T09:16:02.095 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-09-16T09:16:02.095 INFO:teuthology.orchestra.run.smithi084.stdout: Package Architecture Version Repository Size 2024-09-16T09:16:02.095 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-09-16T09:16:02.096 INFO:teuthology.orchestra.run.smithi084.stdout:Removing: 2024-09-16T09:16:02.096 INFO:teuthology.orchestra.run.smithi084.stdout: nvme-cli x86_64 2.9.1-6.el9 @baseos 5.3 M 2024-09-16T09:16:02.096 INFO:teuthology.orchestra.run.smithi084.stdout:Removing unused dependencies: 2024-09-16T09:16:02.096 INFO:teuthology.orchestra.run.smithi084.stdout: libnvme x86_64 1.9-3.el9 @baseos 273 k 2024-09-16T09:16:02.096 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-16T09:16:02.096 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction Summary 2024-09-16T09:16:02.096 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-09-16T09:16:02.096 INFO:teuthology.orchestra.run.smithi084.stdout:Remove 2 Packages 2024-09-16T09:16:02.096 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-16T09:16:02.097 INFO:teuthology.orchestra.run.smithi084.stdout:Freed space: 5.5 M 2024-09-16T09:16:02.097 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction check 2024-09-16T09:16:02.100 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction check succeeded. 2024-09-16T09:16:02.100 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction test 2024-09-16T09:16:02.139 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-09-16T09:16:02.139 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-09-16T09:16:02.159 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction test succeeded. 2024-09-16T09:16:02.159 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction 2024-09-16T09:16:02.281 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-09-16T09:16:02.299 INFO:teuthology.orchestra.run.smithi084.stdout: Preparing : 1/1 2024-09-16T09:16:02.357 INFO:teuthology.orchestra.run.smithi070.stdout: Erasing : nvme-cli-2.9.1-6.el9.x86_64 1/2 2024-09-16T09:16:02.372 INFO:teuthology.orchestra.run.smithi084.stdout: Erasing : nvme-cli-2.9.1-6.el9.x86_64 1/2 2024-09-16T09:16:02.424 INFO:teuthology.orchestra.run.smithi070.stdout: Erasing : libnvme-1.9-3.el9.x86_64 2/2 2024-09-16T09:16:02.430 INFO:teuthology.orchestra.run.smithi084.stdout: Erasing : libnvme-1.9-3.el9.x86_64 2/2 2024-09-16T09:16:02.926 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: libnvme-1.9-3.el9.x86_64 2/2 2024-09-16T09:16:02.926 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/2 2024-09-16T09:16:02.929 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: libnvme-1.9-3.el9.x86_64 2/2 2024-09-16T09:16:02.929 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/2 2024-09-16T09:16:03.167 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/2 2024-09-16T09:16:03.168 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:16:03.168 INFO:teuthology.orchestra.run.smithi070.stdout:Removed: 2024-09-16T09:16:03.168 INFO:teuthology.orchestra.run.smithi070.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-16T09:16:03.168 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:16:03.168 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-16T09:16:03.174 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/2 2024-09-16T09:16:03.174 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-16T09:16:03.174 INFO:teuthology.orchestra.run.smithi084.stdout:Removed: 2024-09-16T09:16:03.174 INFO:teuthology.orchestra.run.smithi084.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-16T09:16:03.174 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-16T09:16:03.174 INFO:teuthology.orchestra.run.smithi084.stdout:Complete! 2024-09-16T09:16:04.000 INFO:teuthology.orchestra.run.smithi084.stdout:Last metadata expiration check: 0:00:47 ago on Mon 16 Sep 2024 09:15:16 AM UTC. 2024-09-16T09:16:04.005 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:00:47 ago on Mon 16 Sep 2024 09:15:16 AM UTC. 2024-09-16T09:16:04.137 INFO:teuthology.orchestra.run.smithi084.stdout:Dependencies resolved. 2024-09-16T09:16:04.138 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-09-16T09:16:04.138 INFO:teuthology.orchestra.run.smithi084.stdout: Package Arch Version Repository Size 2024-09-16T09:16:04.138 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-09-16T09:16:04.138 INFO:teuthology.orchestra.run.smithi084.stdout:Installing: 2024-09-16T09:16:04.139 INFO:teuthology.orchestra.run.smithi084.stdout: nvme-cli x86_64 2.9.1-6.el9 baseos 906 k 2024-09-16T09:16:04.139 INFO:teuthology.orchestra.run.smithi084.stdout: nvmetcli noarch 0.7-3.el9 baseos 44 k 2024-09-16T09:16:04.139 INFO:teuthology.orchestra.run.smithi084.stdout: runc x86_64 4:1.1.13-4.el9 appstream 3.2 M 2024-09-16T09:16:04.139 INFO:teuthology.orchestra.run.smithi084.stdout:Installing dependencies: 2024-09-16T09:16:04.139 INFO:teuthology.orchestra.run.smithi084.stdout: libnvme x86_64 1.9-3.el9 baseos 106 k 2024-09-16T09:16:04.139 INFO:teuthology.orchestra.run.smithi084.stdout: python3-configshell noarch 1:1.1.30-1.el9 baseos 72 k 2024-09-16T09:16:04.139 INFO:teuthology.orchestra.run.smithi084.stdout: python3-kmod x86_64 0.9-32.el9 baseos 84 k 2024-09-16T09:16:04.139 INFO:teuthology.orchestra.run.smithi084.stdout: python3-urwid x86_64 2.1.2-4.el9 baseos 837 k 2024-09-16T09:16:04.139 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-16T09:16:04.139 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction Summary 2024-09-16T09:16:04.139 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-09-16T09:16:04.139 INFO:teuthology.orchestra.run.smithi084.stdout:Install 7 Packages 2024-09-16T09:16:04.139 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-16T09:16:04.144 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-16T09:16:04.145 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-16T09:16:04.145 INFO:teuthology.orchestra.run.smithi070.stdout: Package Arch Version Repository Size 2024-09-16T09:16:04.145 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-16T09:16:04.145 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-09-16T09:16:04.145 INFO:teuthology.orchestra.run.smithi070.stdout: nvme-cli x86_64 2.9.1-6.el9 baseos 906 k 2024-09-16T09:16:04.145 INFO:teuthology.orchestra.run.smithi070.stdout: nvmetcli noarch 0.7-3.el9 baseos 44 k 2024-09-16T09:16:04.145 INFO:teuthology.orchestra.run.smithi070.stdout: runc x86_64 4:1.1.13-4.el9 appstream 3.2 M 2024-09-16T09:16:04.145 INFO:teuthology.orchestra.run.smithi070.stdout:Installing dependencies: 2024-09-16T09:16:04.145 INFO:teuthology.orchestra.run.smithi070.stdout: libnvme x86_64 1.9-3.el9 baseos 106 k 2024-09-16T09:16:04.145 INFO:teuthology.orchestra.run.smithi070.stdout: python3-configshell noarch 1:1.1.30-1.el9 baseos 72 k 2024-09-16T09:16:04.145 INFO:teuthology.orchestra.run.smithi070.stdout: python3-kmod x86_64 0.9-32.el9 baseos 84 k 2024-09-16T09:16:04.146 INFO:teuthology.orchestra.run.smithi070.stdout: python3-urwid x86_64 2.1.2-4.el9 baseos 837 k 2024-09-16T09:16:04.146 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:16:04.146 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-09-16T09:16:04.146 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-16T09:16:04.146 INFO:teuthology.orchestra.run.smithi070.stdout:Install 7 Packages 2024-09-16T09:16:04.146 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:16:04.148 INFO:teuthology.orchestra.run.smithi084.stdout:Total download size: 5.2 M 2024-09-16T09:16:04.149 INFO:teuthology.orchestra.run.smithi084.stdout:Installed size: 19 M 2024-09-16T09:16:04.149 INFO:teuthology.orchestra.run.smithi084.stdout:Downloading Packages: 2024-09-16T09:16:04.170 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 5.2 M 2024-09-16T09:16:04.170 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 19 M 2024-09-16T09:16:04.170 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-09-16T09:16:04.502 INFO:teuthology.orchestra.run.smithi084.stdout:(1/7): nvmetcli-0.7-3.el9.noarch.rpm 404 kB/s | 44 kB 00:00 2024-09-16T09:16:04.527 INFO:teuthology.orchestra.run.smithi084.stdout:(2/7): libnvme-1.9-3.el9.x86_64.rpm 784 kB/s | 106 kB 00:00 2024-09-16T09:16:04.577 INFO:teuthology.orchestra.run.smithi084.stdout:(3/7): python3-kmod-0.9-32.el9.x86_64.rpm 1.6 MB/s | 84 kB 00:00 2024-09-16T09:16:04.603 INFO:teuthology.orchestra.run.smithi084.stdout:(4/7): python3-configshell-1.1.30-1.el9.noarch. 713 kB/s | 72 kB 00:00 2024-09-16T09:16:04.669 INFO:teuthology.orchestra.run.smithi084.stdout:(5/7): nvme-cli-2.9.1-6.el9.x86_64.rpm 3.2 MB/s | 906 kB 00:00 2024-09-16T09:16:04.753 INFO:teuthology.orchestra.run.smithi084.stdout:(6/7): python3-urwid-2.1.2-4.el9.x86_64.rpm 4.7 MB/s | 837 kB 00:00 2024-09-16T09:16:05.280 INFO:teuthology.orchestra.run.smithi070.stdout:(1/7): nvmetcli-0.7-3.el9.noarch.rpm 77 kB/s | 44 kB 00:00 2024-09-16T09:16:05.441 INFO:teuthology.orchestra.run.smithi070.stdout:(2/7): libnvme-1.9-3.el9.x86_64.rpm 144 kB/s | 106 kB 00:00 2024-09-16T09:16:05.600 INFO:teuthology.orchestra.run.smithi070.stdout:(3/7): nvme-cli-2.9.1-6.el9.x86_64.rpm 1.0 MB/s | 906 kB 00:00 2024-09-16T09:16:05.653 INFO:teuthology.orchestra.run.smithi084.stdout:(7/7): runc-1.1.13-4.el9.x86_64.rpm 3.0 MB/s | 3.2 MB 00:01 2024-09-16T09:16:05.653 INFO:teuthology.orchestra.run.smithi084.stdout:-------------------------------------------------------------------------------- 2024-09-16T09:16:05.654 INFO:teuthology.orchestra.run.smithi084.stdout:Total 3.4 MB/s | 5.2 MB 00:01 2024-09-16T09:16:05.762 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction check 2024-09-16T09:16:05.777 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction check succeeded. 2024-09-16T09:16:05.777 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction test 2024-09-16T09:16:05.933 INFO:teuthology.orchestra.run.smithi070.stdout:(4/7): python3-configshell-1.1.30-1.el9.noarch. 110 kB/s | 72 kB 00:00 2024-09-16T09:16:06.097 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction test succeeded. 2024-09-16T09:16:06.097 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction 2024-09-16T09:16:06.100 INFO:teuthology.orchestra.run.smithi070.stdout:(5/7): python3-kmod-0.9-32.el9.x86_64.rpm 128 kB/s | 84 kB 00:00 2024-09-16T09:16:06.500 INFO:teuthology.orchestra.run.smithi070.stdout:(6/7): python3-urwid-2.1.2-4.el9.x86_64.rpm 931 kB/s | 837 kB 00:00 2024-09-16T09:16:06.565 INFO:teuthology.orchestra.run.smithi084.stdout: Preparing : 1/1 2024-09-16T09:16:06.655 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-urwid-2.1.2-4.el9.x86_64 1/7 2024-09-16T09:16:06.708 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-configshell-1:1.1.30-1.el9.noarch 2/7 2024-09-16T09:16:06.736 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-kmod-0.9-32.el9.x86_64 3/7 2024-09-16T09:16:06.950 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libnvme-1.9-3.el9.x86_64 4/7 2024-09-16T09:16:06.983 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : nvme-cli-2.9.1-6.el9.x86_64 5/7 2024-09-16T09:16:07.067 INFO:teuthology.orchestra.run.smithi070.stdout:(7/7): runc-1.1.13-4.el9.x86_64.rpm 2.8 MB/s | 3.2 MB 00:01 2024-09-16T09:16:07.068 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-09-16T09:16:07.068 INFO:teuthology.orchestra.run.smithi070.stdout:Total 1.8 MB/s | 5.2 MB 00:02 2024-09-16T09:16:07.185 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-09-16T09:16:07.201 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-09-16T09:16:07.201 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-09-16T09:16:07.423 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: nvme-cli-2.9.1-6.el9.x86_64 5/7 2024-09-16T09:16:07.452 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : nvmetcli-0.7-3.el9.noarch 6/7 2024-09-16T09:16:07.520 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-09-16T09:16:07.520 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-09-16T09:16:07.567 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: nvmetcli-0.7-3.el9.noarch 6/7 2024-09-16T09:16:07.631 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-16T09:16:07.994 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-09-16T09:16:08.095 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-urwid-2.1.2-4.el9.x86_64 1/7 2024-09-16T09:16:08.129 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-configshell-1:1.1.30-1.el9.noarch 2/7 2024-09-16T09:16:08.157 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-kmod-0.9-32.el9.x86_64 3/7 2024-09-16T09:16:08.363 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libnvme-1.9-3.el9.x86_64 4/7 2024-09-16T09:16:08.397 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : nvme-cli-2.9.1-6.el9.x86_64 5/7 2024-09-16T09:16:08.641 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-16T09:16:08.641 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/7 2024-09-16T09:16:08.642 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/7 2024-09-16T09:16:08.642 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : nvmetcli-0.7-3.el9.noarch 3/7 2024-09-16T09:16:08.642 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-configshell-1:1.1.30-1.el9.noarch 4/7 2024-09-16T09:16:08.642 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-kmod-0.9-32.el9.x86_64 5/7 2024-09-16T09:16:08.642 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-urwid-2.1.2-4.el9.x86_64 6/7 2024-09-16T09:16:08.832 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: nvme-cli-2.9.1-6.el9.x86_64 5/7 2024-09-16T09:16:08.858 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : nvmetcli-0.7-3.el9.noarch 6/7 2024-09-16T09:16:08.971 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: nvmetcli-0.7-3.el9.noarch 6/7 2024-09-16T09:16:09.028 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-16T09:16:09.029 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-16T09:16:09.029 INFO:teuthology.orchestra.run.smithi084.stdout:Installed: 2024-09-16T09:16:09.029 INFO:teuthology.orchestra.run.smithi084.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-16T09:16:09.029 INFO:teuthology.orchestra.run.smithi084.stdout: nvmetcli-0.7-3.el9.noarch python3-configshell-1:1.1.30-1.el9.noarch 2024-09-16T09:16:09.029 INFO:teuthology.orchestra.run.smithi084.stdout: python3-kmod-0.9-32.el9.x86_64 python3-urwid-2.1.2-4.el9.x86_64 2024-09-16T09:16:09.029 INFO:teuthology.orchestra.run.smithi084.stdout: runc-4:1.1.13-4.el9.x86_64 2024-09-16T09:16:09.029 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-16T09:16:09.029 INFO:teuthology.orchestra.run.smithi084.stdout:Complete! 2024-09-16T09:16:09.036 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-16T09:16:09.416 DEBUG:teuthology.parallel:result is None 2024-09-16T09:16:10.134 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-16T09:16:10.134 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/7 2024-09-16T09:16:10.134 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/7 2024-09-16T09:16:10.134 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : nvmetcli-0.7-3.el9.noarch 3/7 2024-09-16T09:16:10.134 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-configshell-1:1.1.30-1.el9.noarch 4/7 2024-09-16T09:16:10.134 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-kmod-0.9-32.el9.x86_64 5/7 2024-09-16T09:16:10.134 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-urwid-2.1.2-4.el9.x86_64 6/7 2024-09-16T09:16:10.495 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-16T09:16:10.495 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:16:10.495 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-09-16T09:16:10.495 INFO:teuthology.orchestra.run.smithi070.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-16T09:16:10.496 INFO:teuthology.orchestra.run.smithi070.stdout: nvmetcli-0.7-3.el9.noarch python3-configshell-1:1.1.30-1.el9.noarch 2024-09-16T09:16:10.496 INFO:teuthology.orchestra.run.smithi070.stdout: python3-kmod-0.9-32.el9.x86_64 python3-urwid-2.1.2-4.el9.x86_64 2024-09-16T09:16:10.496 INFO:teuthology.orchestra.run.smithi070.stdout: runc-4:1.1.13-4.el9.x86_64 2024-09-16T09:16:10.496 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:16:10.496 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-16T09:16:10.908 DEBUG:teuthology.parallel:result is None 2024-09-16T09:16:10.908 INFO:teuthology.run_tasks:Running task install... 2024-09-16T09:16:10.916 DEBUG:teuthology.task.install:project ceph 2024-09-16T09:16:10.916 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '8293d73f8690540e843a81caec373f9cc29cf705'}} 2024-09-16T09:16:10.916 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '8293d73f8690540e843a81caec373f9cc29cf705'} 2024-09-16T09:16:10.916 INFO:teuthology.task.install:Using flavor: default 2024-09-16T09:16:10.922 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-16T09:16:10.922 INFO:teuthology.task.install:extra packages: [] 2024-09-16T09:16:10.922 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8293d73f8690540e843a81caec373f9cc29cf705', 'tag': None, 'wait_for_package': False} 2024-09-16T09:16:10.923 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8293d73f8690540e843a81caec373f9cc29cf705 2024-09-16T09:16:10.925 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8293d73f8690540e843a81caec373f9cc29cf705', 'tag': None, 'wait_for_package': False} 2024-09-16T09:16:10.925 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8293d73f8690540e843a81caec373f9cc29cf705 2024-09-16T09:16:11.073 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-guits-main-2024-09-13-1248/8293d73f8690540e843a81caec373f9cc29cf705/centos/9/flavors/default/ 2024-09-16T09:16:11.073 INFO:teuthology.task.install.rpm:Package version is 19.3.0-4917.g8293d73f 2024-09-16T09:16:11.083 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-guits-main-2024-09-13-1248/8293d73f8690540e843a81caec373f9cc29cf705/centos/9/flavors/default/ 2024-09-16T09:16:11.083 INFO:teuthology.task.install.rpm:Package version is 19.3.0-4917.g8293d73f 2024-09-16T09:16:11.187 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-guits-main-2024-09-13-1248/8293d73f8690540e843a81caec373f9cc29cf705/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-guits-main-2024-09-13-1248/8293d73f8690540e843a81caec373f9cc29cf705/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-guits-main-2024-09-13-1248/8293d73f8690540e843a81caec373f9cc29cf705/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-16T09:16:11.187 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-16T09:16:11.187 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-16T09:16:11.192 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-guits-main-2024-09-13-1248/8293d73f8690540e843a81caec373f9cc29cf705/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-guits-main-2024-09-13-1248/8293d73f8690540e843a81caec373f9cc29cf705/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-guits-main-2024-09-13-1248/8293d73f8690540e843a81caec373f9cc29cf705/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-16T09:16:11.192 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-09-16T09:16:11.193 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-16T09:16:11.213 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-16T09:16:11.213 DEBUG:teuthology.orchestra.run.smithi070:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8293d73f8690540e843a81caec373f9cc29cf705/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-16T09:16:11.224 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-16T09:16:11.224 DEBUG:teuthology.orchestra.run.smithi084:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8293d73f8690540e843a81caec373f9cc29cf705/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-16T09:16:11.279 DEBUG:teuthology.orchestra.run.smithi070:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-16T09:16:11.293 DEBUG:teuthology.orchestra.run.smithi084:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-16T09:16:11.365 DEBUG:teuthology.orchestra.run.smithi070:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-16T09:16:11.373 DEBUG:teuthology.orchestra.run.smithi084:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-16T09:16:11.388 INFO:teuthology.orchestra.run.smithi070.stdout:check_obsoletes = 1 2024-09-16T09:16:11.390 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum clean all 2024-09-16T09:16:11.439 INFO:teuthology.orchestra.run.smithi084.stdout:check_obsoletes = 1 2024-09-16T09:16:11.441 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum clean all 2024-09-16T09:16:11.625 INFO:teuthology.orchestra.run.smithi070.stdout:45 files removed 2024-09-16T09:16:11.654 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install ceph-radosgw 2024-09-16T09:16:11.677 INFO:teuthology.orchestra.run.smithi084.stdout:45 files removed 2024-09-16T09:16:11.704 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum -y install ceph-radosgw 2024-09-16T09:16:12.200 INFO:teuthology.orchestra.run.smithi070.stdout:ceph packages for x86_64 326 kB/s | 84 kB 00:00 2024-09-16T09:16:12.241 INFO:teuthology.orchestra.run.smithi084.stdout:ceph packages for x86_64 336 kB/s | 84 kB 00:00 2024-09-16T09:16:12.476 INFO:teuthology.orchestra.run.smithi070.stdout:ceph noarch packages 104 kB/s | 24 kB 00:00 2024-09-16T09:16:12.525 INFO:teuthology.orchestra.run.smithi084.stdout:ceph noarch packages 100 kB/s | 24 kB 00:00 2024-09-16T09:16:12.752 INFO:teuthology.orchestra.run.smithi070.stdout:ceph source packages 8.1 kB/s | 1.9 kB 00:00 2024-09-16T09:16:12.801 INFO:teuthology.orchestra.run.smithi084.stdout:ceph source packages 8.1 kB/s | 1.9 kB 00:00 2024-09-16T09:16:13.343 INFO:teuthology.orchestra.run.smithi084.stdout:CentOS Stream 9 - BaseOS 16 MB/s | 8.2 MB 00:00 2024-09-16T09:16:14.476 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - BaseOS 4.9 MB/s | 8.2 MB 00:01 2024-09-16T09:16:15.094 INFO:teuthology.orchestra.run.smithi084.stdout:CentOS Stream 9 - AppStream 33 MB/s | 20 MB 00:00 2024-09-16T09:16:20.127 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - AppStream 4.5 MB/s | 20 MB 00:04 2024-09-16T09:16:20.869 INFO:teuthology.orchestra.run.smithi084.stdout:CentOS Stream 9 - CRB 3.5 MB/s | 6.5 MB 00:01 2024-09-16T09:16:23.253 INFO:teuthology.orchestra.run.smithi084.stdout:CentOS Stream 9 - Extras packages 18 kB/s | 19 kB 00:01 2024-09-16T09:16:24.065 INFO:teuthology.orchestra.run.smithi084.stdout:Extra Packages for Enterprise Linux 33 MB/s | 23 MB 00:00 2024-09-16T09:16:24.860 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - CRB 7.4 MB/s | 6.5 MB 00:00 2024-09-16T09:16:26.603 INFO:teuthology.orchestra.run.smithi070.stdout:CentOS Stream 9 - Extras packages 37 kB/s | 19 kB 00:00 2024-09-16T09:16:27.366 INFO:teuthology.orchestra.run.smithi070.stdout:Extra Packages for Enterprise Linux 36 MB/s | 23 MB 00:00 2024-09-16T09:16:29.596 INFO:teuthology.orchestra.run.smithi084.stdout:lab-extras 17 kB/s | 1.7 kB 00:00 2024-09-16T09:16:31.980 INFO:teuthology.orchestra.run.smithi084.stdout:Dependencies resolved. 2024-09-16T09:16:31.982 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-09-16T09:16:31.982 INFO:teuthology.orchestra.run.smithi084.stdout: Package Arch Version Repository Size 2024-09-16T09:16:31.983 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-09-16T09:16:31.983 INFO:teuthology.orchestra.run.smithi084.stdout:Installing: 2024-09-16T09:16:31.983 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-radosgw x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 11 M 2024-09-16T09:16:31.983 INFO:teuthology.orchestra.run.smithi084.stdout:Upgrading: 2024-09-16T09:16:31.983 INFO:teuthology.orchestra.run.smithi084.stdout: librados2 x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 3.5 M 2024-09-16T09:16:31.983 INFO:teuthology.orchestra.run.smithi084.stdout: librbd1 x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 3.2 M 2024-09-16T09:16:31.983 INFO:teuthology.orchestra.run.smithi084.stdout:Installing dependencies: 2024-09-16T09:16:31.983 INFO:teuthology.orchestra.run.smithi084.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-16T09:16:31.983 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-base x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 5.5 M 2024-09-16T09:16:31.983 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-common x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 22 M 2024-09-16T09:16:31.983 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-selinux x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 25 k 2024-09-16T09:16:31.983 INFO:teuthology.orchestra.run.smithi084.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-16T09:16:31.984 INFO:teuthology.orchestra.run.smithi084.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-16T09:16:31.984 INFO:teuthology.orchestra.run.smithi084.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-16T09:16:31.984 INFO:teuthology.orchestra.run.smithi084.stdout: libcephfs2 x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 755 k 2024-09-16T09:16:31.984 INFO:teuthology.orchestra.run.smithi084.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-16T09:16:31.984 INFO:teuthology.orchestra.run.smithi084.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-16T09:16:31.984 INFO:teuthology.orchestra.run.smithi084.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-16T09:16:31.984 INFO:teuthology.orchestra.run.smithi084.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-16T09:16:31.984 INFO:teuthology.orchestra.run.smithi084.stdout: libradosstriper1 x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 508 k 2024-09-16T09:16:31.984 INFO:teuthology.orchestra.run.smithi084.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-16T09:16:31.984 INFO:teuthology.orchestra.run.smithi084.stdout: librgw2 x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 5.5 M 2024-09-16T09:16:31.984 INFO:teuthology.orchestra.run.smithi084.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-16T09:16:31.984 INFO:teuthology.orchestra.run.smithi084.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-16T09:16:31.985 INFO:teuthology.orchestra.run.smithi084.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-16T09:16:31.985 INFO:teuthology.orchestra.run.smithi084.stdout: python3-ceph-argparse x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 45 k 2024-09-16T09:16:31.985 INFO:teuthology.orchestra.run.smithi084.stdout: python3-ceph-common x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 143 k 2024-09-16T09:16:31.985 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cephfs x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 161 k 2024-09-16T09:16:31.985 INFO:teuthology.orchestra.run.smithi084.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-16T09:16:31.985 INFO:teuthology.orchestra.run.smithi084.stdout: python3-rados x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 320 k 2024-09-16T09:16:31.985 INFO:teuthology.orchestra.run.smithi084.stdout: python3-rbd x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 306 k 2024-09-16T09:16:31.985 INFO:teuthology.orchestra.run.smithi084.stdout: python3-rgw x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 99 k 2024-09-16T09:16:31.985 INFO:teuthology.orchestra.run.smithi084.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-16T09:16:31.985 INFO:teuthology.orchestra.run.smithi084.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-16T09:16:31.985 INFO:teuthology.orchestra.run.smithi084.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-16T09:16:31.985 INFO:teuthology.orchestra.run.smithi084.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-16T09:16:31.985 INFO:teuthology.orchestra.run.smithi084.stdout:Installing weak dependencies: 2024-09-16T09:16:31.986 INFO:teuthology.orchestra.run.smithi084.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-16T09:16:31.986 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-16T09:16:31.986 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction Summary 2024-09-16T09:16:31.986 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-09-16T09:16:31.986 INFO:teuthology.orchestra.run.smithi084.stdout:Install 31 Packages 2024-09-16T09:16:31.986 INFO:teuthology.orchestra.run.smithi084.stdout:Upgrade 2 Packages 2024-09-16T09:16:31.986 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-16T09:16:31.986 INFO:teuthology.orchestra.run.smithi084.stdout:Total download size: 62 M 2024-09-16T09:16:31.987 INFO:teuthology.orchestra.run.smithi084.stdout:Downloading Packages: 2024-09-16T09:16:32.788 INFO:teuthology.orchestra.run.smithi070.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-16T09:16:32.913 INFO:teuthology.orchestra.run.smithi084.stdout:(1/33): ceph-radosgw-19.3.0-4917.g8293d73f.el9. 14 MB/s | 11 MB 00:00 2024-09-16T09:16:32.988 INFO:teuthology.orchestra.run.smithi084.stdout:(2/33): ceph-base-19.3.0-4917.g8293d73f.el9.x86 6.5 MB/s | 5.5 MB 00:00 2024-09-16T09:16:33.013 INFO:teuthology.orchestra.run.smithi084.stdout:(3/33): ceph-selinux-19.3.0-4917.g8293d73f.el9. 251 kB/s | 25 kB 00:00 2024-09-16T09:16:33.047 INFO:teuthology.orchestra.run.smithi084.stdout:(4/33): libcephfs2-19.3.0-4917.g8293d73f.el9.x8 13 MB/s | 755 kB 00:00 2024-09-16T09:16:33.072 INFO:teuthology.orchestra.run.smithi084.stdout:(5/33): libradosstriper1-19.3.0-4917.g8293d73f. 8.5 MB/s | 508 kB 00:00 2024-09-16T09:16:33.131 INFO:teuthology.orchestra.run.smithi084.stdout:(6/33): python3-ceph-argparse-19.3.0-4917.g8293 772 kB/s | 45 kB 00:00 2024-09-16T09:16:33.215 INFO:teuthology.orchestra.run.smithi084.stdout:(7/33): python3-ceph-common-19.3.0-4917.g8293d7 1.7 MB/s | 143 kB 00:00 2024-09-16T09:16:33.290 INFO:teuthology.orchestra.run.smithi084.stdout:(8/33): librgw2-19.3.0-4917.g8293d73f.el9.x86_6 23 MB/s | 5.5 MB 00:00 2024-09-16T09:16:33.315 INFO:teuthology.orchestra.run.smithi084.stdout:(9/33): python3-cephfs-19.3.0-4917.g8293d73f.el 1.6 MB/s | 161 kB 00:00 2024-09-16T09:16:33.349 INFO:teuthology.orchestra.run.smithi084.stdout:(10/33): python3-rados-19.3.0-4917.g8293d73f.el 5.3 MB/s | 320 kB 00:00 2024-09-16T09:16:33.382 INFO:teuthology.orchestra.run.smithi084.stdout:(11/33): python3-rbd-19.3.0-4917.g8293d73f.el9. 4.5 MB/s | 306 kB 00:00 2024-09-16T09:16:33.408 INFO:teuthology.orchestra.run.smithi084.stdout:(12/33): python3-rgw-19.3.0-4917.g8293d73f.el9. 1.7 MB/s | 99 kB 00:00 2024-09-16T09:16:33.616 INFO:teuthology.orchestra.run.smithi084.stdout:(13/33): ceph-common-19.3.0-4917.g8293d73f.el9. 15 MB/s | 22 MB 00:01 2024-09-16T09:16:33.992 INFO:teuthology.orchestra.run.smithi084.stdout:(14/33): boost-program-options-1.75.0-8.el9.x86 175 kB/s | 107 kB 00:00 2024-09-16T09:16:34.075 INFO:teuthology.orchestra.run.smithi084.stdout:(15/33): libnbd-1.20.2-2.el9.x86_64.rpm 256 kB/s | 171 kB 00:00 2024-09-16T09:16:34.109 INFO:teuthology.orchestra.run.smithi084.stdout:(16/33): librabbitmq-0.11.0-7.el9.x86_64.rpm 386 kB/s | 45 kB 00:00 2024-09-16T09:16:34.159 INFO:teuthology.orchestra.run.smithi084.stdout:(17/33): libpmemobj-1.12.1-1.el9.x86_64.rpm 295 kB/s | 160 kB 00:00 2024-09-16T09:16:34.276 INFO:teuthology.orchestra.run.smithi084.stdout:(18/33): python3-prettytable-0.7.2-27.el9.noarc 362 kB/s | 42 kB 00:00 2024-09-16T09:16:34.402 INFO:teuthology.orchestra.run.smithi084.stdout:(19/33): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.0 MB/s | 292 kB 00:00 2024-09-16T09:16:34.496 INFO:teuthology.orchestra.run.smithi084.stdout:(20/33): librdkafka-1.6.1-102.el9.x86_64.rpm 1.5 MB/s | 662 kB 00:00 2024-09-16T09:16:34.544 INFO:teuthology.orchestra.run.smithi084.stdout:(21/33): qatlib-service-24.02.0-1.el9.x86_64.rp 253 kB/s | 36 kB 00:00 2024-09-16T09:16:34.594 INFO:teuthology.orchestra.run.smithi084.stdout:(22/33): gperftools-libs-2.9.1-3.el9.x86_64.rpm 6.0 MB/s | 308 kB 00:00 2024-09-16T09:16:34.627 INFO:teuthology.orchestra.run.smithi084.stdout:(23/33): qatlib-24.02.0-1.el9.x86_64.rpm 629 kB/s | 221 kB 00:00 2024-09-16T09:16:34.653 INFO:teuthology.orchestra.run.smithi084.stdout:(24/33): qatzip-libs-1.2.0-1.el9.x86_64.rpm 304 kB/s | 47 kB 00:00 2024-09-16T09:16:34.695 INFO:teuthology.orchestra.run.smithi084.stdout:(25/33): libarrow-doc-9.0.0-11.el9.noarch.rpm 389 kB/s | 26 kB 00:00 2024-09-16T09:16:34.745 INFO:teuthology.orchestra.run.smithi084.stdout:(26/33): libunwind-1.6.2-1.el9.x86_64.rpm 1.4 MB/s | 67 kB 00:00 2024-09-16T09:16:34.762 INFO:teuthology.orchestra.run.smithi084.stdout:(27/33): liboath-2.6.7-2.el9.x86_64.rpm 479 kB/s | 49 kB 00:00 2024-09-16T09:16:34.795 INFO:teuthology.orchestra.run.smithi084.stdout:(28/33): re2-20211101-20.el9.x86_64.rpm 5.6 MB/s | 191 kB 00:00 2024-09-16T09:16:34.837 INFO:teuthology.orchestra.run.smithi084.stdout:(29/33): parquet-libs-9.0.0-11.el9.x86_64.rpm 8.9 MB/s | 839 kB 00:00 2024-09-16T09:16:34.938 INFO:teuthology.orchestra.run.smithi084.stdout:(30/33): libarrow-9.0.0-11.el9.x86_64.rpm 13 MB/s | 4.4 MB 00:00 2024-09-16T09:16:34.980 INFO:teuthology.orchestra.run.smithi084.stdout:(31/33): thrift-0.15.0-2.el9.x86_64.rpm 8.6 MB/s | 1.6 MB 00:00 2024-09-16T09:16:35.130 INFO:teuthology.orchestra.run.smithi084.stdout:(32/33): librados2-19.3.0-4917.g8293d73f.el9.x8 12 MB/s | 3.5 MB 00:00 2024-09-16T09:16:35.183 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-16T09:16:35.185 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-16T09:16:35.185 INFO:teuthology.orchestra.run.smithi070.stdout: Package Arch Version Repository Size 2024-09-16T09:16:35.185 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-16T09:16:35.185 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-09-16T09:16:35.185 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-radosgw x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 11 M 2024-09-16T09:16:35.185 INFO:teuthology.orchestra.run.smithi070.stdout:Upgrading: 2024-09-16T09:16:35.185 INFO:teuthology.orchestra.run.smithi070.stdout: librados2 x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 3.5 M 2024-09-16T09:16:35.185 INFO:teuthology.orchestra.run.smithi070.stdout: librbd1 x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 3.2 M 2024-09-16T09:16:35.185 INFO:teuthology.orchestra.run.smithi070.stdout:Installing dependencies: 2024-09-16T09:16:35.185 INFO:teuthology.orchestra.run.smithi070.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-16T09:16:35.185 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-base x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 5.5 M 2024-09-16T09:16:35.186 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-common x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 22 M 2024-09-16T09:16:35.186 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-selinux x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 25 k 2024-09-16T09:16:35.186 INFO:teuthology.orchestra.run.smithi070.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-16T09:16:35.186 INFO:teuthology.orchestra.run.smithi070.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-16T09:16:35.186 INFO:teuthology.orchestra.run.smithi070.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-16T09:16:35.186 INFO:teuthology.orchestra.run.smithi070.stdout: libcephfs2 x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 755 k 2024-09-16T09:16:35.186 INFO:teuthology.orchestra.run.smithi070.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-16T09:16:35.186 INFO:teuthology.orchestra.run.smithi070.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-16T09:16:35.186 INFO:teuthology.orchestra.run.smithi070.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-16T09:16:35.186 INFO:teuthology.orchestra.run.smithi070.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-16T09:16:35.186 INFO:teuthology.orchestra.run.smithi070.stdout: libradosstriper1 x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 508 k 2024-09-16T09:16:35.186 INFO:teuthology.orchestra.run.smithi070.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-16T09:16:35.186 INFO:teuthology.orchestra.run.smithi070.stdout: librgw2 x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 5.5 M 2024-09-16T09:16:35.186 INFO:teuthology.orchestra.run.smithi070.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-16T09:16:35.187 INFO:teuthology.orchestra.run.smithi070.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-16T09:16:35.187 INFO:teuthology.orchestra.run.smithi070.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-16T09:16:35.187 INFO:teuthology.orchestra.run.smithi070.stdout: python3-ceph-argparse x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 45 k 2024-09-16T09:16:35.187 INFO:teuthology.orchestra.run.smithi070.stdout: python3-ceph-common x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 143 k 2024-09-16T09:16:35.187 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cephfs x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 161 k 2024-09-16T09:16:35.187 INFO:teuthology.orchestra.run.smithi070.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-16T09:16:35.187 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rados x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 320 k 2024-09-16T09:16:35.187 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rbd x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 306 k 2024-09-16T09:16:35.187 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rgw x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 99 k 2024-09-16T09:16:35.187 INFO:teuthology.orchestra.run.smithi070.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-16T09:16:35.187 INFO:teuthology.orchestra.run.smithi070.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-16T09:16:35.187 INFO:teuthology.orchestra.run.smithi070.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-16T09:16:35.187 INFO:teuthology.orchestra.run.smithi070.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-16T09:16:35.187 INFO:teuthology.orchestra.run.smithi070.stdout:Installing weak dependencies: 2024-09-16T09:16:35.188 INFO:teuthology.orchestra.run.smithi070.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-16T09:16:35.188 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:16:35.188 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-09-16T09:16:35.188 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-16T09:16:35.188 INFO:teuthology.orchestra.run.smithi070.stdout:Install 31 Packages 2024-09-16T09:16:35.188 INFO:teuthology.orchestra.run.smithi070.stdout:Upgrade 2 Packages 2024-09-16T09:16:35.188 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:16:35.188 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 62 M 2024-09-16T09:16:35.189 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-09-16T09:16:35.222 INFO:teuthology.orchestra.run.smithi084.stdout:(33/33): librbd1-19.3.0-4917.g8293d73f.el9.x86_ 11 MB/s | 3.2 MB 00:00 2024-09-16T09:16:35.227 INFO:teuthology.orchestra.run.smithi084.stdout:-------------------------------------------------------------------------------- 2024-09-16T09:16:35.228 INFO:teuthology.orchestra.run.smithi084.stdout:Total 19 MB/s | 62 MB 00:03 2024-09-16T09:16:35.345 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction check 2024-09-16T09:16:35.382 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction check succeeded. 2024-09-16T09:16:35.382 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction test 2024-09-16T09:16:35.893 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction test succeeded. 2024-09-16T09:16:35.893 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction 2024-09-16T09:16:35.905 INFO:teuthology.orchestra.run.smithi070.stdout:(1/33): ceph-base-19.3.0-4917.g8293d73f.el9.x86 11 MB/s | 5.5 MB 00:00 2024-09-16T09:16:35.964 INFO:teuthology.orchestra.run.smithi070.stdout:(2/33): ceph-selinux-19.3.0-4917.g8293d73f.el9. 431 kB/s | 25 kB 00:00 2024-09-16T09:16:36.047 INFO:teuthology.orchestra.run.smithi070.stdout:(3/33): libcephfs2-19.3.0-4917.g8293d73f.el9.x8 8.9 MB/s | 755 kB 00:00 2024-09-16T09:16:36.206 INFO:teuthology.orchestra.run.smithi070.stdout:(4/33): ceph-radosgw-19.3.0-4917.g8293d73f.el9. 13 MB/s | 11 MB 00:00 2024-09-16T09:16:36.231 INFO:teuthology.orchestra.run.smithi070.stdout:(5/33): libradosstriper1-19.3.0-4917.g8293d73f. 2.7 MB/s | 508 kB 00:00 2024-09-16T09:16:36.290 INFO:teuthology.orchestra.run.smithi070.stdout:(6/33): python3-ceph-argparse-19.3.0-4917.g8293 773 kB/s | 45 kB 00:00 2024-09-16T09:16:36.374 INFO:teuthology.orchestra.run.smithi070.stdout:(7/33): python3-ceph-common-19.3.0-4917.g8293d7 1.7 MB/s | 143 kB 00:00 2024-09-16T09:16:36.474 INFO:teuthology.orchestra.run.smithi070.stdout:(8/33): librgw2-19.3.0-4917.g8293d73f.el9.x86_6 21 MB/s | 5.5 MB 00:00 2024-09-16T09:16:36.499 INFO:teuthology.orchestra.run.smithi070.stdout:(9/33): python3-cephfs-19.3.0-4917.g8293d73f.el 1.3 MB/s | 161 kB 00:00 2024-09-16T09:16:36.524 INFO:teuthology.orchestra.run.smithi070.stdout:(10/33): python3-rados-19.3.0-4917.g8293d73f.el 6.2 MB/s | 320 kB 00:00 2024-09-16T09:16:36.550 INFO:teuthology.orchestra.run.smithi070.stdout:(11/33): python3-rbd-19.3.0-4917.g8293d73f.el9. 5.9 MB/s | 306 kB 00:00 2024-09-16T09:16:36.759 INFO:teuthology.orchestra.run.smithi070.stdout:(12/33): ceph-common-19.3.0-4917.g8293d73f.el9. 16 MB/s | 22 MB 00:01 2024-09-16T09:16:36.784 INFO:teuthology.orchestra.run.smithi070.stdout:(13/33): python3-rgw-19.3.0-4917.g8293d73f.el9. 383 kB/s | 99 kB 00:00 2024-09-16T09:16:36.926 INFO:teuthology.orchestra.run.smithi070.stdout:(14/33): boost-program-options-1.75.0-8.el9.x86 285 kB/s | 107 kB 00:00 2024-09-16T09:16:36.951 INFO:teuthology.orchestra.run.smithi070.stdout:(15/33): libnbd-1.20.2-2.el9.x86_64.rpm 890 kB/s | 171 kB 00:00 2024-09-16T09:16:36.976 INFO:teuthology.orchestra.run.smithi070.stdout:(16/33): libpmemobj-1.12.1-1.el9.x86_64.rpm 832 kB/s | 160 kB 00:00 2024-09-16T09:16:36.994 INFO:teuthology.orchestra.run.smithi070.stdout:(17/33): librabbitmq-0.11.0-7.el9.x86_64.rpm 670 kB/s | 45 kB 00:00 2024-09-16T09:16:37.035 INFO:teuthology.orchestra.run.smithi070.stdout:(18/33): python3-prettytable-0.7.2-27.el9.noarc 1.0 MB/s | 42 kB 00:00 2024-09-16T09:16:37.061 INFO:teuthology.orchestra.run.smithi070.stdout:(19/33): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.4 MB/s | 292 kB 00:00 2024-09-16T09:16:37.086 INFO:teuthology.orchestra.run.smithi070.stdout:(20/33): librdkafka-1.6.1-102.el9.x86_64.rpm 4.8 MB/s | 662 kB 00:00 2024-09-16T09:16:37.111 INFO:teuthology.orchestra.run.smithi070.stdout:(21/33): qatlib-service-24.02.0-1.el9.x86_64.rp 715 kB/s | 36 kB 00:00 2024-09-16T09:16:37.145 INFO:teuthology.orchestra.run.smithi070.stdout:(22/33): qatzip-libs-1.2.0-1.el9.x86_64.rpm 811 kB/s | 47 kB 00:00 2024-09-16T09:16:37.178 INFO:teuthology.orchestra.run.smithi070.stdout:(23/33): qatlib-24.02.0-1.el9.x86_64.rpm 1.5 MB/s | 221 kB 00:00 2024-09-16T09:16:37.229 INFO:teuthology.orchestra.run.smithi070.stdout:(24/33): gperftools-libs-2.9.1-3.el9.x86_64.rpm 2.6 MB/s | 308 kB 00:00 2024-09-16T09:16:37.262 INFO:teuthology.orchestra.run.smithi070.stdout:(25/33): liboath-2.6.7-2.el9.x86_64.rpm 1.5 MB/s | 49 kB 00:00 2024-09-16T09:16:37.288 INFO:teuthology.orchestra.run.smithi070.stdout:(26/33): libarrow-doc-9.0.0-11.el9.noarch.rpm 243 kB/s | 26 kB 00:00 2024-09-16T09:16:37.316 INFO:teuthology.orchestra.run.smithi070.stdout:(27/33): libunwind-1.6.2-1.el9.x86_64.rpm 1.2 MB/s | 67 kB 00:00 2024-09-16T09:16:37.349 INFO:teuthology.orchestra.run.smithi070.stdout:(28/33): re2-20211101-20.el9.x86_64.rpm 5.6 MB/s | 191 kB 00:00 2024-09-16T09:16:37.357 INFO:teuthology.orchestra.run.smithi084.stdout: Preparing : 1/1 2024-09-16T09:16:37.408 INFO:teuthology.orchestra.run.smithi070.stdout:(29/33): parquet-libs-9.0.0-11.el9.x86_64.rpm 6.8 MB/s | 839 kB 00:00 2024-09-16T09:16:37.444 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/35 2024-09-16T09:16:37.475 INFO:teuthology.orchestra.run.smithi070.stdout:(30/33): libarrow-9.0.0-11.el9.x86_64.rpm 13 MB/s | 4.4 MB 00:00 2024-09-16T09:16:37.485 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/35 2024-09-16T09:16:37.510 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/35 2024-09-16T09:16:37.517 INFO:teuthology.orchestra.run.smithi070.stdout:(31/33): thrift-0.15.0-2.el9.x86_64.rpm 9.5 MB/s | 1.6 MB 00:00 2024-09-16T09:16:37.532 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/35 2024-09-16T09:16:37.557 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/35 2024-09-16T09:16:37.659 INFO:teuthology.orchestra.run.smithi070.stdout:(32/33): librados2-19.3.0-4917.g8293d73f.el9.x8 14 MB/s | 3.5 MB 00:00 2024-09-16T09:16:37.717 INFO:teuthology.orchestra.run.smithi070.stdout:(33/33): librbd1-19.3.0-4917.g8293d73f.el9.x86_ 13 MB/s | 3.2 MB 00:00 2024-09-16T09:16:37.722 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-09-16T09:16:37.722 INFO:teuthology.orchestra.run.smithi070.stdout:Total 24 MB/s | 62 MB 00:02 2024-09-16T09:16:37.751 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/35 2024-09-16T09:16:37.822 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/35 2024-09-16T09:16:37.828 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-09-16T09:16:37.866 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-09-16T09:16:37.866 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-09-16T09:16:38.181 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/35 2024-09-16T09:16:38.293 INFO:teuthology.orchestra.run.smithi084.stdout: Upgrading : librados2-2:19.3.0-4917.g8293d73f.el9.x86_64 7/35 2024-09-16T09:16:38.368 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-09-16T09:16:38.368 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-09-16T09:16:38.387 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: librados2-2:19.3.0-4917.g8293d73f.el9.x86_64 7/35 2024-09-16T09:16:38.493 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-rados-2:19.3.0-4917.g8293d73f.el9.x86_64 8/35 2024-09-16T09:16:38.524 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libcephfs2-2:19.3.0-4917.g8293d73f.el9.x86_64 9/35 2024-09-16T09:16:38.578 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: libcephfs2-2:19.3.0-4917.g8293d73f.el9.x86_64 9/35 2024-09-16T09:16:38.625 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/35 2024-09-16T09:16:38.674 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-ceph-argparse-2:19.3.0-4917.g8293d73f.el9. 11/35 2024-09-16T09:16:38.755 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-cephfs-2:19.3.0-4917.g8293d73f.el9.x86_64 12/35 2024-09-16T09:16:38.781 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libradosstriper1-2:19.3.0-4917.g8293d73f.el9.x86_6 13/35 2024-09-16T09:16:38.840 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: libradosstriper1-2:19.3.0-4917.g8293d73f.el9.x86_6 13/35 2024-09-16T09:16:38.874 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/35 2024-09-16T09:16:38.990 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/35 2024-09-16T09:16:39.022 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/35 2024-09-16T09:16:39.076 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/35 2024-09-16T09:16:39.104 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/35 2024-09-16T09:16:39.133 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/35 2024-09-16T09:16:39.166 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/35 2024-09-16T09:16:39.203 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/35 2024-09-16T09:16:39.544 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 22/35 2024-09-16T09:16:39.613 INFO:teuthology.orchestra.run.smithi084.stdout: Upgrading : librbd1-2:19.3.0-4917.g8293d73f.el9.x86_64 23/35 2024-09-16T09:16:39.704 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: librbd1-2:19.3.0-4917.g8293d73f.el9.x86_64 23/35 2024-09-16T09:16:39.734 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-rbd-2:19.3.0-4917.g8293d73f.el9.x86_64 24/35 2024-09-16T09:16:39.778 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 25/35 2024-09-16T09:16:39.857 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-09-16T09:16:39.943 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/35 2024-09-16T09:16:39.981 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/35 2024-09-16T09:16:40.006 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/35 2024-09-16T09:16:40.029 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/35 2024-09-16T09:16:40.052 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/35 2024-09-16T09:16:40.247 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/35 2024-09-16T09:16:40.282 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 26/35 2024-09-16T09:16:40.300 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/35 2024-09-16T09:16:40.307 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : librgw2-2:19.3.0-4917.g8293d73f.el9.x86_64 27/35 2024-09-16T09:16:40.384 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: librgw2-2:19.3.0-4917.g8293d73f.el9.x86_64 27/35 2024-09-16T09:16:40.456 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-rgw-2:19.3.0-4917.g8293d73f.el9.x86_64 28/35 2024-09-16T09:16:40.484 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-ceph-common-2:19.3.0-4917.g8293d73f.el9.x8 29/35 2024-09-16T09:16:40.630 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/35 2024-09-16T09:16:40.749 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : librados2-2:19.3.0-4917.g8293d73f.el9.x86_64 7/35 2024-09-16T09:16:40.825 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: librados2-2:19.3.0-4917.g8293d73f.el9.x86_64 7/35 2024-09-16T09:16:40.915 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-rados-2:19.3.0-4917.g8293d73f.el9.x86_64 8/35 2024-09-16T09:16:40.943 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libcephfs2-2:19.3.0-4917.g8293d73f.el9.x86_64 9/35 2024-09-16T09:16:41.016 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: libcephfs2-2:19.3.0-4917.g8293d73f.el9.x86_64 9/35 2024-09-16T09:16:41.051 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/35 2024-09-16T09:16:41.096 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-ceph-argparse-2:19.3.0-4917.g8293d73f.el9. 11/35 2024-09-16T09:16:41.174 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-cephfs-2:19.3.0-4917.g8293d73f.el9.x86_64 12/35 2024-09-16T09:16:41.204 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libradosstriper1-2:19.3.0-4917.g8293d73f.el9.x86_6 13/35 2024-09-16T09:16:41.268 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: libradosstriper1-2:19.3.0-4917.g8293d73f.el9.x86_6 13/35 2024-09-16T09:16:41.311 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/35 2024-09-16T09:16:41.440 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/35 2024-09-16T09:16:41.483 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/35 2024-09-16T09:16:41.530 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/35 2024-09-16T09:16:41.560 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/35 2024-09-16T09:16:41.589 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/35 2024-09-16T09:16:41.622 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/35 2024-09-16T09:16:41.659 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/35 2024-09-16T09:16:42.000 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 22/35 2024-09-16T09:16:42.053 INFO:teuthology.orchestra.run.smithi070.stdout: Upgrading : librbd1-2:19.3.0-4917.g8293d73f.el9.x86_64 23/35 2024-09-16T09:16:42.145 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: librbd1-2:19.3.0-4917.g8293d73f.el9.x86_64 23/35 2024-09-16T09:16:42.175 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-rbd-2:19.3.0-4917.g8293d73f.el9.x86_64 24/35 2024-09-16T09:16:42.219 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 25/35 2024-09-16T09:16:42.725 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 26/35 2024-09-16T09:16:42.764 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : librgw2-2:19.3.0-4917.g8293d73f.el9.x86_64 27/35 2024-09-16T09:16:42.818 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-common-2:19.3.0-4917.g8293d73f.el9.x86_64 30/35 2024-09-16T09:16:42.846 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: librgw2-2:19.3.0-4917.g8293d73f.el9.x86_64 27/35 2024-09-16T09:16:42.855 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-common-2:19.3.0-4917.g8293d73f.el9.x86_64 30/35 2024-09-16T09:16:42.920 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-rgw-2:19.3.0-4917.g8293d73f.el9.x86_64 28/35 2024-09-16T09:16:42.950 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-ceph-common-2:19.3.0-4917.g8293d73f.el9.x8 29/35 2024-09-16T09:16:43.438 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-common-2:19.3.0-4917.g8293d73f.el9.x86_64 30/35 2024-09-16T09:16:43.483 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-base-2:19.3.0-4917.g8293d73f.el9.x86_64 31/35 2024-09-16T09:16:43.578 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-base-2:19.3.0-4917.g8293d73f.el9.x86_64 31/35 2024-09-16T09:16:43.578 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-16T09:16:43.578 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-16T09:16:43.578 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-16T09:16:43.609 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-selinux-2:19.3.0-4917.g8293d73f.el9.x86_64 32/35 2024-09-16T09:16:45.280 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-common-2:19.3.0-4917.g8293d73f.el9.x86_64 30/35 2024-09-16T09:16:45.313 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-common-2:19.3.0-4917.g8293d73f.el9.x86_64 30/35 2024-09-16T09:16:45.900 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-common-2:19.3.0-4917.g8293d73f.el9.x86_64 30/35 2024-09-16T09:16:45.933 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-base-2:19.3.0-4917.g8293d73f.el9.x86_64 31/35 2024-09-16T09:16:46.011 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-base-2:19.3.0-4917.g8293d73f.el9.x86_64 31/35 2024-09-16T09:16:46.011 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-16T09:16:46.011 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-16T09:16:46.011 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:16:46.059 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-selinux-2:19.3.0-4917.g8293d73f.el9.x86_64 32/35 2024-09-16T09:16:53.625 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-selinux-2:19.3.0-4917.g8293d73f.el9.x86_64 32/35 2024-09-16T09:16:53.626 INFO:teuthology.orchestra.run.smithi084.stdout:skipping the directory /sys 2024-09-16T09:16:53.626 INFO:teuthology.orchestra.run.smithi084.stdout:skipping the directory /proc 2024-09-16T09:16:53.626 INFO:teuthology.orchestra.run.smithi084.stdout:skipping the directory /mnt 2024-09-16T09:16:53.626 INFO:teuthology.orchestra.run.smithi084.stdout:skipping the directory /var/tmp 2024-09-16T09:16:53.626 INFO:teuthology.orchestra.run.smithi084.stdout:skipping the directory /home 2024-09-16T09:16:53.626 INFO:teuthology.orchestra.run.smithi084.stdout:skipping the directory /root 2024-09-16T09:16:53.626 INFO:teuthology.orchestra.run.smithi084.stdout:skipping the directory /tmp 2024-09-16T09:16:53.626 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-16T09:16:54.649 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-radosgw-2:19.3.0-4917.g8293d73f.el9.x86_64 33/35 2024-09-16T09:16:54.675 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-4917.g8293d73f.el9.x86_64 33/35 2024-09-16T09:16:54.675 INFO:teuthology.orchestra.run.smithi084.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T09:16:54.675 INFO:teuthology.orchestra.run.smithi084.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-16T09:16:54.675 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T09:16:54.675 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T09:16:54.675 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-16T09:16:54.677 INFO:teuthology.orchestra.run.smithi084.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 34/35 2024-09-16T09:16:54.760 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 34/35 2024-09-16T09:16:54.760 INFO:teuthology.orchestra.run.smithi084.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 35/35 2024-09-16T09:16:55.964 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 35/35 2024-09-16T09:16:55.965 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-base-2:19.3.0-4917.g8293d73f.el9.x86_64 1/35 2024-09-16T09:16:55.965 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-common-2:19.3.0-4917.g8293d73f.el9.x86_64 2/35 2024-09-16T09:16:55.965 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-radosgw-2:19.3.0-4917.g8293d73f.el9.x86_64 3/35 2024-09-16T09:16:55.965 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-selinux-2:19.3.0-4917.g8293d73f.el9.x86_64 4/35 2024-09-16T09:16:55.965 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libcephfs2-2:19.3.0-4917.g8293d73f.el9.x86_64 5/35 2024-09-16T09:16:55.965 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libradosstriper1-2:19.3.0-4917.g8293d73f.el9.x86_6 6/35 2024-09-16T09:16:55.965 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : librgw2-2:19.3.0-4917.g8293d73f.el9.x86_64 7/35 2024-09-16T09:16:55.965 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-ceph-argparse-2:19.3.0-4917.g8293d73f.el9. 8/35 2024-09-16T09:16:55.965 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-ceph-common-2:19.3.0-4917.g8293d73f.el9.x8 9/35 2024-09-16T09:16:55.965 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-cephfs-2:19.3.0-4917.g8293d73f.el9.x86_64 10/35 2024-09-16T09:16:55.965 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-rados-2:19.3.0-4917.g8293d73f.el9.x86_64 11/35 2024-09-16T09:16:55.965 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-rbd-2:19.3.0-4917.g8293d73f.el9.x86_64 12/35 2024-09-16T09:16:55.966 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-rgw-2:19.3.0-4917.g8293d73f.el9.x86_64 13/35 2024-09-16T09:16:55.966 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/35 2024-09-16T09:16:55.966 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 15/35 2024-09-16T09:16:55.966 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 16/35 2024-09-16T09:16:55.966 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 17/35 2024-09-16T09:16:55.966 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 18/35 2024-09-16T09:16:55.966 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 19/35 2024-09-16T09:16:55.966 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 20/35 2024-09-16T09:16:55.966 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 21/35 2024-09-16T09:16:55.966 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 22/35 2024-09-16T09:16:55.966 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 23/35 2024-09-16T09:16:55.966 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 24/35 2024-09-16T09:16:55.967 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 25/35 2024-09-16T09:16:55.967 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 26/35 2024-09-16T09:16:55.967 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 27/35 2024-09-16T09:16:55.967 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 28/35 2024-09-16T09:16:55.967 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 29/35 2024-09-16T09:16:55.967 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : re2-1:20211101-20.el9.x86_64 30/35 2024-09-16T09:16:55.967 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 31/35 2024-09-16T09:16:55.967 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : librados2-2:19.3.0-4917.g8293d73f.el9.x86_64 32/35 2024-09-16T09:16:55.967 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 33/35 2024-09-16T09:16:55.967 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : librbd1-2:19.3.0-4917.g8293d73f.el9.x86_64 34/35 2024-09-16T09:16:55.974 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-selinux-2:19.3.0-4917.g8293d73f.el9.x86_64 32/35 2024-09-16T09:16:55.974 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /sys 2024-09-16T09:16:55.974 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /proc 2024-09-16T09:16:55.974 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /mnt 2024-09-16T09:16:55.974 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /var/tmp 2024-09-16T09:16:55.974 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /home 2024-09-16T09:16:55.974 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /root 2024-09-16T09:16:55.974 INFO:teuthology.orchestra.run.smithi070.stdout:skipping the directory /tmp 2024-09-16T09:16:55.975 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:16:56.881 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 35/35 2024-09-16T09:16:56.882 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-16T09:16:56.882 INFO:teuthology.orchestra.run.smithi084.stdout:Upgraded: 2024-09-16T09:16:56.882 INFO:teuthology.orchestra.run.smithi084.stdout: librados2-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T09:16:56.882 INFO:teuthology.orchestra.run.smithi084.stdout: librbd1-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T09:16:56.882 INFO:teuthology.orchestra.run.smithi084.stdout:Installed: 2024-09-16T09:16:56.882 INFO:teuthology.orchestra.run.smithi084.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-16T09:16:56.882 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-base-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T09:16:56.882 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-common-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T09:16:56.882 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-radosgw-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T09:16:56.882 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-selinux-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T09:16:56.882 INFO:teuthology.orchestra.run.smithi084.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-16T09:16:56.882 INFO:teuthology.orchestra.run.smithi084.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-16T09:16:56.883 INFO:teuthology.orchestra.run.smithi084.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-16T09:16:56.883 INFO:teuthology.orchestra.run.smithi084.stdout: libcephfs2-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T09:16:56.883 INFO:teuthology.orchestra.run.smithi084.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-16T09:16:56.883 INFO:teuthology.orchestra.run.smithi084.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-16T09:16:56.883 INFO:teuthology.orchestra.run.smithi084.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-16T09:16:56.883 INFO:teuthology.orchestra.run.smithi084.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-16T09:16:56.883 INFO:teuthology.orchestra.run.smithi084.stdout: libradosstriper1-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T09:16:56.883 INFO:teuthology.orchestra.run.smithi084.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-16T09:16:56.883 INFO:teuthology.orchestra.run.smithi084.stdout: librgw2-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T09:16:56.883 INFO:teuthology.orchestra.run.smithi084.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-16T09:16:56.883 INFO:teuthology.orchestra.run.smithi084.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-16T09:16:56.883 INFO:teuthology.orchestra.run.smithi084.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-16T09:16:56.883 INFO:teuthology.orchestra.run.smithi084.stdout: python3-ceph-argparse-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T09:16:56.883 INFO:teuthology.orchestra.run.smithi084.stdout: python3-ceph-common-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T09:16:56.884 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cephfs-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T09:16:56.884 INFO:teuthology.orchestra.run.smithi084.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-16T09:16:56.884 INFO:teuthology.orchestra.run.smithi084.stdout: python3-rados-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T09:16:56.884 INFO:teuthology.orchestra.run.smithi084.stdout: python3-rbd-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T09:16:56.884 INFO:teuthology.orchestra.run.smithi084.stdout: python3-rgw-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T09:16:56.884 INFO:teuthology.orchestra.run.smithi084.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-16T09:16:56.884 INFO:teuthology.orchestra.run.smithi084.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-16T09:16:56.884 INFO:teuthology.orchestra.run.smithi084.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-16T09:16:56.884 INFO:teuthology.orchestra.run.smithi084.stdout: re2-1:20211101-20.el9.x86_64 2024-09-16T09:16:56.884 INFO:teuthology.orchestra.run.smithi084.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-16T09:16:56.884 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-16T09:16:56.884 INFO:teuthology.orchestra.run.smithi084.stdout:Complete! 2024-09-16T09:16:57.020 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-radosgw-2:19.3.0-4917.g8293d73f.el9.x86_64 33/35 2024-09-16T09:16:57.049 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-4917.g8293d73f.el9.x86_64 33/35 2024-09-16T09:16:57.050 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T09:16:57.050 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-16T09:16:57.050 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T09:16:57.050 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T09:16:57.050 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:16:57.051 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 34/35 2024-09-16T09:16:57.141 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 34/35 2024-09-16T09:16:57.141 INFO:teuthology.orchestra.run.smithi070.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 35/35 2024-09-16T09:16:57.211 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum -y install ceph-test 2024-09-16T09:16:57.783 INFO:teuthology.orchestra.run.smithi084.stdout:Last metadata expiration check: 0:00:28 ago on Mon 16 Sep 2024 09:16:29 AM UTC. 2024-09-16T09:16:57.921 INFO:teuthology.orchestra.run.smithi084.stdout:Dependencies resolved. 2024-09-16T09:16:57.922 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-09-16T09:16:57.923 INFO:teuthology.orchestra.run.smithi084.stdout: Package Arch Version Repository Size 2024-09-16T09:16:57.923 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-09-16T09:16:57.923 INFO:teuthology.orchestra.run.smithi084.stdout:Installing: 2024-09-16T09:16:57.923 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-test x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 50 M 2024-09-16T09:16:57.923 INFO:teuthology.orchestra.run.smithi084.stdout:Installing dependencies: 2024-09-16T09:16:57.923 INFO:teuthology.orchestra.run.smithi084.stdout: libcephsqlite x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 161 k 2024-09-16T09:16:57.923 INFO:teuthology.orchestra.run.smithi084.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-16T09:16:57.923 INFO:teuthology.orchestra.run.smithi084.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-16T09:16:57.923 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-16T09:16:57.923 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction Summary 2024-09-16T09:16:57.924 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-09-16T09:16:57.924 INFO:teuthology.orchestra.run.smithi084.stdout:Install 4 Packages 2024-09-16T09:16:57.924 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-16T09:16:57.924 INFO:teuthology.orchestra.run.smithi084.stdout:Total download size: 51 M 2024-09-16T09:16:57.924 INFO:teuthology.orchestra.run.smithi084.stdout:Installed size: 217 M 2024-09-16T09:16:57.924 INFO:teuthology.orchestra.run.smithi084.stdout:Downloading Packages: 2024-09-16T09:16:58.108 INFO:teuthology.orchestra.run.smithi084.stdout:(1/4): socat-1.7.4.1-6.el9.x86_64.rpm 2.2 MB/s | 304 kB 00:00 2024-09-16T09:16:58.150 INFO:teuthology.orchestra.run.smithi084.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2024-09-16T09:16:58.191 INFO:teuthology.orchestra.run.smithi084.stdout:(3/4): libcephsqlite-19.3.0-4917.g8293d73f.el9. 728 kB/s | 161 kB 00:00 2024-09-16T09:16:58.314 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 35/35 2024-09-16T09:16:58.314 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-base-2:19.3.0-4917.g8293d73f.el9.x86_64 1/35 2024-09-16T09:16:58.315 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-common-2:19.3.0-4917.g8293d73f.el9.x86_64 2/35 2024-09-16T09:16:58.315 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-radosgw-2:19.3.0-4917.g8293d73f.el9.x86_64 3/35 2024-09-16T09:16:58.315 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-selinux-2:19.3.0-4917.g8293d73f.el9.x86_64 4/35 2024-09-16T09:16:58.315 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libcephfs2-2:19.3.0-4917.g8293d73f.el9.x86_64 5/35 2024-09-16T09:16:58.315 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libradosstriper1-2:19.3.0-4917.g8293d73f.el9.x86_6 6/35 2024-09-16T09:16:58.316 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librgw2-2:19.3.0-4917.g8293d73f.el9.x86_64 7/35 2024-09-16T09:16:58.316 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-ceph-argparse-2:19.3.0-4917.g8293d73f.el9. 8/35 2024-09-16T09:16:58.316 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-ceph-common-2:19.3.0-4917.g8293d73f.el9.x8 9/35 2024-09-16T09:16:58.316 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-cephfs-2:19.3.0-4917.g8293d73f.el9.x86_64 10/35 2024-09-16T09:16:58.316 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-rados-2:19.3.0-4917.g8293d73f.el9.x86_64 11/35 2024-09-16T09:16:58.317 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-rbd-2:19.3.0-4917.g8293d73f.el9.x86_64 12/35 2024-09-16T09:16:58.317 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-rgw-2:19.3.0-4917.g8293d73f.el9.x86_64 13/35 2024-09-16T09:16:58.317 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/35 2024-09-16T09:16:58.317 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 15/35 2024-09-16T09:16:58.317 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 16/35 2024-09-16T09:16:58.318 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 17/35 2024-09-16T09:16:58.318 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 18/35 2024-09-16T09:16:58.318 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 19/35 2024-09-16T09:16:58.318 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 20/35 2024-09-16T09:16:58.318 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 21/35 2024-09-16T09:16:58.318 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 22/35 2024-09-16T09:16:58.319 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 23/35 2024-09-16T09:16:58.319 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 24/35 2024-09-16T09:16:58.319 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 25/35 2024-09-16T09:16:58.319 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 26/35 2024-09-16T09:16:58.319 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 27/35 2024-09-16T09:16:58.320 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 28/35 2024-09-16T09:16:58.320 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 29/35 2024-09-16T09:16:58.320 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : re2-1:20211101-20.el9.x86_64 30/35 2024-09-16T09:16:58.320 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 31/35 2024-09-16T09:16:58.320 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librados2-2:19.3.0-4917.g8293d73f.el9.x86_64 32/35 2024-09-16T09:16:58.321 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 33/35 2024-09-16T09:16:58.321 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librbd1-2:19.3.0-4917.g8293d73f.el9.x86_64 34/35 2024-09-16T09:16:59.174 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 35/35 2024-09-16T09:16:59.174 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:16:59.174 INFO:teuthology.orchestra.run.smithi070.stdout:Upgraded: 2024-09-16T09:16:59.174 INFO:teuthology.orchestra.run.smithi070.stdout: librados2-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T09:16:59.175 INFO:teuthology.orchestra.run.smithi070.stdout: librbd1-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T09:16:59.175 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-09-16T09:16:59.175 INFO:teuthology.orchestra.run.smithi070.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-16T09:16:59.175 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-base-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T09:16:59.175 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-common-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T09:16:59.175 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-radosgw-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T09:16:59.175 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-selinux-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T09:16:59.175 INFO:teuthology.orchestra.run.smithi070.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-16T09:16:59.175 INFO:teuthology.orchestra.run.smithi070.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-16T09:16:59.175 INFO:teuthology.orchestra.run.smithi070.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-16T09:16:59.175 INFO:teuthology.orchestra.run.smithi070.stdout: libcephfs2-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T09:16:59.176 INFO:teuthology.orchestra.run.smithi070.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-16T09:16:59.176 INFO:teuthology.orchestra.run.smithi070.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-16T09:16:59.176 INFO:teuthology.orchestra.run.smithi070.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-16T09:16:59.176 INFO:teuthology.orchestra.run.smithi070.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-16T09:16:59.176 INFO:teuthology.orchestra.run.smithi070.stdout: libradosstriper1-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T09:16:59.176 INFO:teuthology.orchestra.run.smithi070.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-16T09:16:59.176 INFO:teuthology.orchestra.run.smithi070.stdout: librgw2-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T09:16:59.176 INFO:teuthology.orchestra.run.smithi070.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-16T09:16:59.176 INFO:teuthology.orchestra.run.smithi070.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-16T09:16:59.176 INFO:teuthology.orchestra.run.smithi070.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-16T09:16:59.177 INFO:teuthology.orchestra.run.smithi070.stdout: python3-ceph-argparse-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T09:16:59.177 INFO:teuthology.orchestra.run.smithi070.stdout: python3-ceph-common-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T09:16:59.177 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cephfs-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T09:16:59.177 INFO:teuthology.orchestra.run.smithi070.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-16T09:16:59.177 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rados-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T09:16:59.177 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rbd-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T09:16:59.177 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rgw-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T09:16:59.177 INFO:teuthology.orchestra.run.smithi070.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-16T09:16:59.177 INFO:teuthology.orchestra.run.smithi070.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-16T09:16:59.177 INFO:teuthology.orchestra.run.smithi070.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-16T09:16:59.178 INFO:teuthology.orchestra.run.smithi070.stdout: re2-1:20211101-20.el9.x86_64 2024-09-16T09:16:59.178 INFO:teuthology.orchestra.run.smithi070.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-16T09:16:59.178 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:16:59.178 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-16T09:16:59.564 INFO:teuthology.orchestra.run.smithi084.stdout:(4/4): ceph-test-19.3.0-4917.g8293d73f.el9.x86_ 32 MB/s | 50 MB 00:01 2024-09-16T09:16:59.566 INFO:teuthology.orchestra.run.smithi084.stdout:-------------------------------------------------------------------------------- 2024-09-16T09:16:59.566 INFO:teuthology.orchestra.run.smithi084.stdout:Total 31 MB/s | 51 MB 00:01 2024-09-16T09:16:59.569 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install ceph-test 2024-09-16T09:16:59.593 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction check 2024-09-16T09:16:59.613 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction check succeeded. 2024-09-16T09:16:59.613 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction test 2024-09-16T09:17:00.016 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction test succeeded. 2024-09-16T09:17:00.016 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction 2024-09-16T09:17:00.141 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:00:28 ago on Mon 16 Sep 2024 09:16:32 AM UTC. 2024-09-16T09:17:00.282 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-16T09:17:00.283 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-16T09:17:00.283 INFO:teuthology.orchestra.run.smithi070.stdout: Package Arch Version Repository Size 2024-09-16T09:17:00.283 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-16T09:17:00.283 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-09-16T09:17:00.283 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-test x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 50 M 2024-09-16T09:17:00.283 INFO:teuthology.orchestra.run.smithi070.stdout:Installing dependencies: 2024-09-16T09:17:00.283 INFO:teuthology.orchestra.run.smithi070.stdout: libcephsqlite x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 161 k 2024-09-16T09:17:00.283 INFO:teuthology.orchestra.run.smithi070.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-16T09:17:00.283 INFO:teuthology.orchestra.run.smithi070.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-16T09:17:00.283 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:17:00.284 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-09-16T09:17:00.284 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-16T09:17:00.284 INFO:teuthology.orchestra.run.smithi070.stdout:Install 4 Packages 2024-09-16T09:17:00.284 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:17:00.284 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 51 M 2024-09-16T09:17:00.284 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 217 M 2024-09-16T09:17:00.284 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-09-16T09:17:00.588 INFO:teuthology.orchestra.run.smithi084.stdout: Preparing : 1/1 2024-09-16T09:17:00.613 INFO:teuthology.orchestra.run.smithi070.stdout:(1/4): socat-1.7.4.1-6.el9.x86_64.rpm 1.5 MB/s | 304 kB 00:00 2024-09-16T09:17:00.638 INFO:teuthology.orchestra.run.smithi070.stdout:(2/4): libcephsqlite-19.3.0-4917.g8293d73f.el9. 714 kB/s | 161 kB 00:00 2024-09-16T09:17:00.663 INFO:teuthology.orchestra.run.smithi070.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-16T09:17:00.678 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-16T09:17:00.739 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-16T09:17:00.768 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libcephsqlite-2:19.3.0-4917.g8293d73f.el9.x86_64 3/4 2024-09-16T09:17:01.947 INFO:teuthology.orchestra.run.smithi070.stdout:(4/4): ceph-test-19.3.0-4917.g8293d73f.el9.x86_ 33 MB/s | 50 MB 00:01 2024-09-16T09:17:01.949 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-09-16T09:17:01.949 INFO:teuthology.orchestra.run.smithi070.stdout:Total 30 MB/s | 51 MB 00:01 2024-09-16T09:17:01.977 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-09-16T09:17:01.997 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-09-16T09:17:01.997 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-09-16T09:17:02.405 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-09-16T09:17:02.405 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-09-16T09:17:02.987 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-09-16T09:17:03.074 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-16T09:17:03.129 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-16T09:17:03.157 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libcephsqlite-2:19.3.0-4917.g8293d73f.el9.x86_64 3/4 2024-09-16T09:17:05.524 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: libcephsqlite-2:19.3.0-4917.g8293d73f.el9.x86_64 3/4 2024-09-16T09:17:05.607 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-test-2:19.3.0-4917.g8293d73f.el9.x86_64 4/4 2024-09-16T09:17:07.903 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: libcephsqlite-2:19.3.0-4917.g8293d73f.el9.x86_64 3/4 2024-09-16T09:17:07.991 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-test-2:19.3.0-4917.g8293d73f.el9.x86_64 4/4 2024-09-16T09:17:08.008 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-test-2:19.3.0-4917.g8293d73f.el9.x86_64 4/4 2024-09-16T09:17:08.008 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-test-2:19.3.0-4917.g8293d73f.el9.x86_64 1/4 2024-09-16T09:17:08.009 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libcephsqlite-2:19.3.0-4917.g8293d73f.el9.x86_64 2/4 2024-09-16T09:17:08.009 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-16T09:17:08.315 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-16T09:17:08.316 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-16T09:17:08.316 INFO:teuthology.orchestra.run.smithi084.stdout:Installed: 2024-09-16T09:17:08.316 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-test-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T09:17:08.316 INFO:teuthology.orchestra.run.smithi084.stdout: libcephsqlite-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T09:17:08.316 INFO:teuthology.orchestra.run.smithi084.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-16T09:17:08.316 INFO:teuthology.orchestra.run.smithi084.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-16T09:17:08.316 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-16T09:17:08.316 INFO:teuthology.orchestra.run.smithi084.stdout:Complete! 2024-09-16T09:17:08.582 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum -y install ceph 2024-09-16T09:17:09.161 INFO:teuthology.orchestra.run.smithi084.stdout:Last metadata expiration check: 0:00:40 ago on Mon 16 Sep 2024 09:16:29 AM UTC. 2024-09-16T09:17:09.306 INFO:teuthology.orchestra.run.smithi084.stdout:Dependencies resolved. 2024-09-16T09:17:09.309 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-09-16T09:17:09.309 INFO:teuthology.orchestra.run.smithi084.stdout: Package Arch Version Repository Size 2024-09-16T09:17:09.309 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-09-16T09:17:09.309 INFO:teuthology.orchestra.run.smithi084.stdout:Installing: 2024-09-16T09:17:09.309 INFO:teuthology.orchestra.run.smithi084.stdout: ceph x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 6.6 k 2024-09-16T09:17:09.309 INFO:teuthology.orchestra.run.smithi084.stdout:Installing dependencies: 2024-09-16T09:17:09.309 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mds x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 2.4 M 2024-09-16T09:17:09.309 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 1.5 M 2024-09-16T09:17:09.309 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-modules-core noarch 2:19.3.0-4917.g8293d73f.el9 ceph-noarch 291 k 2024-09-16T09:17:09.309 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mon x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 4.8 M 2024-09-16T09:17:09.310 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-osd x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 17 M 2024-09-16T09:17:09.310 INFO:teuthology.orchestra.run.smithi084.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-16T09:17:09.310 INFO:teuthology.orchestra.run.smithi084.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-16T09:17:09.310 INFO:teuthology.orchestra.run.smithi084.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-16T09:17:09.310 INFO:teuthology.orchestra.run.smithi084.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-16T09:17:09.310 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-16T09:17:09.310 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-16T09:17:09.310 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-16T09:17:09.310 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-16T09:17:09.310 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-16T09:17:09.310 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-16T09:17:09.310 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco-collections 2024-09-16T09:17:09.310 INFO:teuthology.orchestra.run.smithi084.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-09-16T09:17:09.311 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-16T09:17:09.311 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-16T09:17:09.311 INFO:teuthology.orchestra.run.smithi084.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-16T09:17:09.311 INFO:teuthology.orchestra.run.smithi084.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-16T09:17:09.311 INFO:teuthology.orchestra.run.smithi084.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-16T09:17:09.311 INFO:teuthology.orchestra.run.smithi084.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-16T09:17:09.311 INFO:teuthology.orchestra.run.smithi084.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-16T09:17:09.311 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-16T09:17:09.311 INFO:teuthology.orchestra.run.smithi084.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-16T09:17:09.311 INFO:teuthology.orchestra.run.smithi084.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-16T09:17:09.311 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-16T09:17:09.311 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-16T09:17:09.311 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-16T09:17:09.311 INFO:teuthology.orchestra.run.smithi084.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-16T09:17:09.312 INFO:teuthology.orchestra.run.smithi084.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-16T09:17:09.312 INFO:teuthology.orchestra.run.smithi084.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-16T09:17:09.312 INFO:teuthology.orchestra.run.smithi084.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-16T09:17:09.312 INFO:teuthology.orchestra.run.smithi084.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-16T09:17:09.312 INFO:teuthology.orchestra.run.smithi084.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-16T09:17:09.312 INFO:teuthology.orchestra.run.smithi084.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-16T09:17:09.312 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-16T09:17:09.312 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction Summary 2024-09-16T09:17:09.312 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-09-16T09:17:09.312 INFO:teuthology.orchestra.run.smithi084.stdout:Install 37 Packages 2024-09-16T09:17:09.312 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-16T09:17:09.313 INFO:teuthology.orchestra.run.smithi084.stdout:Total download size: 31 M 2024-09-16T09:17:09.313 INFO:teuthology.orchestra.run.smithi084.stdout:Installed size: 109 M 2024-09-16T09:17:09.313 INFO:teuthology.orchestra.run.smithi084.stdout:Downloading Packages: 2024-09-16T09:17:09.885 INFO:teuthology.orchestra.run.smithi084.stdout:(1/37): ceph-19.3.0-4917.g8293d73f.el9.x86_64.r 52 kB/s | 6.6 kB 00:00 2024-09-16T09:17:10.093 INFO:teuthology.orchestra.run.smithi084.stdout:(2/37): ceph-mds-19.3.0-4917.g8293d73f.el9.x86_ 7.1 MB/s | 2.4 MB 00:00 2024-09-16T09:17:10.135 INFO:teuthology.orchestra.run.smithi084.stdout:(3/37): ceph-mgr-19.3.0-4917.g8293d73f.el9.x86_ 4.0 MB/s | 1.5 MB 00:00 2024-09-16T09:17:10.249 INFO:teuthology.orchestra.run.smithi084.stdout:(4/37): ceph-mgr-modules-core-19.3.0-4917.g8293 2.5 MB/s | 291 kB 00:00 2024-09-16T09:17:10.319 INFO:teuthology.orchestra.run.smithi084.stdout:(5/37): ceph-mon-19.3.0-4917.g8293d73f.el9.x86_ 11 MB/s | 4.8 MB 00:00 2024-09-16T09:17:10.432 INFO:teuthology.orchestra.run.smithi084.stdout:(6/37): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.4 MB/s | 253 kB 00:00 2024-09-16T09:17:10.458 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-test-2:19.3.0-4917.g8293d73f.el9.x86_64 4/4 2024-09-16T09:17:10.458 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-test-2:19.3.0-4917.g8293d73f.el9.x86_64 1/4 2024-09-16T09:17:10.458 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libcephsqlite-2:19.3.0-4917.g8293d73f.el9.x86_64 2/4 2024-09-16T09:17:10.458 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-16T09:17:10.478 INFO:teuthology.orchestra.run.smithi084.stdout:(7/37): python3-ply-3.11-14.el9.noarch.rpm 2.3 MB/s | 106 kB 00:00 2024-09-16T09:17:10.662 INFO:teuthology.orchestra.run.smithi084.stdout:(8/37): ceph-osd-19.3.0-4917.g8293d73f.el9.x86_ 30 MB/s | 17 MB 00:00 2024-09-16T09:17:10.704 INFO:teuthology.orchestra.run.smithi084.stdout:(9/37): python3-pycparser-2.20-6.el9.noarch.rpm 600 kB/s | 135 kB 00:00 2024-09-16T09:17:10.737 INFO:teuthology.orchestra.run.smithi084.stdout:(10/37): python3-cryptography-36.0.1-4.el9.x86_ 2.9 MB/s | 1.2 MB 00:00 2024-09-16T09:17:10.762 INFO:teuthology.orchestra.run.smithi084.stdout:(11/37): python3-urllib3-1.26.5-6.el9.noarch.rp 3.6 MB/s | 215 kB 00:00 2024-09-16T09:17:10.792 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-16T09:17:10.792 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:17:10.792 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-09-16T09:17:10.792 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-test-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T09:17:10.792 INFO:teuthology.orchestra.run.smithi070.stdout: libcephsqlite-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T09:17:10.792 INFO:teuthology.orchestra.run.smithi070.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-16T09:17:10.792 INFO:teuthology.orchestra.run.smithi070.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-16T09:17:10.792 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:17:10.792 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-16T09:17:10.804 INFO:teuthology.orchestra.run.smithi084.stdout:(12/37): python3-requests-2.25.1-8.el9.noarch.r 879 kB/s | 125 kB 00:00 2024-09-16T09:17:10.904 INFO:teuthology.orchestra.run.smithi084.stdout:(13/37): lua-5.4.4-4.el9.x86_64.rpm 1.1 MB/s | 188 kB 00:00 2024-09-16T09:17:10.930 INFO:teuthology.orchestra.run.smithi084.stdout:(14/37): python3-markupsafe-1.1.1-12.el9.x86_64 277 kB/s | 35 kB 00:00 2024-09-16T09:17:10.955 INFO:teuthology.orchestra.run.smithi084.stdout:(15/37): python3-mako-1.1.4-6.el9.noarch.rpm 896 kB/s | 172 kB 00:00 2024-09-16T09:17:10.980 INFO:teuthology.orchestra.run.smithi084.stdout:(16/37): python3-packaging-20.9-5.el9.noarch.rp 1.0 MB/s | 77 kB 00:00 2024-09-16T09:17:11.005 INFO:teuthology.orchestra.run.smithi084.stdout:(17/37): python3-toml-0.10.2-6.el9.noarch.rpm 827 kB/s | 42 kB 00:00 2024-09-16T09:17:11.031 INFO:teuthology.orchestra.run.smithi084.stdout:(18/37): lua-devel-5.4.4-4.el9.x86_64.rpm 443 kB/s | 22 kB 00:00 2024-09-16T09:17:11.035 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install ceph 2024-09-16T09:17:11.056 INFO:teuthology.orchestra.run.smithi084.stdout:(19/37): python3-pytz-2021.1-5.el9.noarch.rpm 404 kB/s | 51 kB 00:00 2024-09-16T09:17:11.515 INFO:teuthology.orchestra.run.smithi084.stdout:(20/37): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 90 kB/s | 43 kB 00:00 2024-09-16T09:17:11.540 INFO:teuthology.orchestra.run.smithi084.stdout:(21/37): luarocks-3.9.2-5.el9.noarch.rpm 283 kB/s | 151 kB 00:00 2024-09-16T09:17:11.565 INFO:teuthology.orchestra.run.smithi084.stdout:(22/37): python3-cheroot-8.6.0-4.el9.noarch.rpm 337 kB/s | 172 kB 00:00 2024-09-16T09:17:11.590 INFO:teuthology.orchestra.run.smithi084.stdout:(23/37): python3-jaraco-8.2.1-3.el9.noarch.rpm 213 kB/s | 11 kB 00:00 2024-09-16T09:17:11.597 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:00:39 ago on Mon 16 Sep 2024 09:16:32 AM UTC. 2024-09-16T09:17:11.616 INFO:teuthology.orchestra.run.smithi084.stdout:(24/37): python3-jaraco-classes-3.2.1-5.el9.noa 352 kB/s | 18 kB 00:00 2024-09-16T09:17:11.641 INFO:teuthology.orchestra.run.smithi084.stdout:(25/37): python3-jaraco-collections-3.0.0-8.el9 460 kB/s | 23 kB 00:00 2024-09-16T09:17:11.675 INFO:teuthology.orchestra.run.smithi084.stdout:(26/37): python3-jaraco-functools-3.5.0-2.el9.n 330 kB/s | 19 kB 00:00 2024-09-16T09:17:11.700 INFO:teuthology.orchestra.run.smithi084.stdout:(27/37): python3-jaraco-text-3.2.0-6.el9.noarch 335 kB/s | 20 kB 00:00 2024-09-16T09:17:11.725 INFO:teuthology.orchestra.run.smithi084.stdout:(28/37): python3-logutils-0.3.5-21.el9.noarch.r 924 kB/s | 46 kB 00:00 2024-09-16T09:17:11.739 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-16T09:17:11.742 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-16T09:17:11.742 INFO:teuthology.orchestra.run.smithi070.stdout: Package Arch Version Repository Size 2024-09-16T09:17:11.742 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-16T09:17:11.742 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-09-16T09:17:11.742 INFO:teuthology.orchestra.run.smithi070.stdout: ceph x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 6.6 k 2024-09-16T09:17:11.743 INFO:teuthology.orchestra.run.smithi070.stdout:Installing dependencies: 2024-09-16T09:17:11.743 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mds x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 2.4 M 2024-09-16T09:17:11.743 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 1.5 M 2024-09-16T09:17:11.743 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-modules-core noarch 2:19.3.0-4917.g8293d73f.el9 ceph-noarch 291 k 2024-09-16T09:17:11.743 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mon x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 4.8 M 2024-09-16T09:17:11.743 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-osd x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 17 M 2024-09-16T09:17:11.743 INFO:teuthology.orchestra.run.smithi070.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-16T09:17:11.743 INFO:teuthology.orchestra.run.smithi070.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-16T09:17:11.743 INFO:teuthology.orchestra.run.smithi070.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-16T09:17:11.743 INFO:teuthology.orchestra.run.smithi070.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-16T09:17:11.743 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-16T09:17:11.743 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-16T09:17:11.743 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-16T09:17:11.744 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-16T09:17:11.744 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-16T09:17:11.744 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-16T09:17:11.744 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-collections 2024-09-16T09:17:11.744 INFO:teuthology.orchestra.run.smithi070.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-09-16T09:17:11.744 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-16T09:17:11.744 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-16T09:17:11.744 INFO:teuthology.orchestra.run.smithi070.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-16T09:17:11.744 INFO:teuthology.orchestra.run.smithi070.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-16T09:17:11.744 INFO:teuthology.orchestra.run.smithi070.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-16T09:17:11.744 INFO:teuthology.orchestra.run.smithi070.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-16T09:17:11.744 INFO:teuthology.orchestra.run.smithi070.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-16T09:17:11.744 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-16T09:17:11.744 INFO:teuthology.orchestra.run.smithi070.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-16T09:17:11.745 INFO:teuthology.orchestra.run.smithi070.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-16T09:17:11.745 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-16T09:17:11.745 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-16T09:17:11.745 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-16T09:17:11.745 INFO:teuthology.orchestra.run.smithi070.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-16T09:17:11.745 INFO:teuthology.orchestra.run.smithi070.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-16T09:17:11.745 INFO:teuthology.orchestra.run.smithi070.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-16T09:17:11.745 INFO:teuthology.orchestra.run.smithi070.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-16T09:17:11.745 INFO:teuthology.orchestra.run.smithi070.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-16T09:17:11.745 INFO:teuthology.orchestra.run.smithi070.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-16T09:17:11.745 INFO:teuthology.orchestra.run.smithi070.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-16T09:17:11.745 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:17:11.745 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-09-16T09:17:11.745 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-16T09:17:11.746 INFO:teuthology.orchestra.run.smithi070.stdout:Install 37 Packages 2024-09-16T09:17:11.746 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:17:11.746 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 31 M 2024-09-16T09:17:11.746 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 109 M 2024-09-16T09:17:11.746 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-09-16T09:17:11.750 INFO:teuthology.orchestra.run.smithi084.stdout:(29/37): python3-more-itertools-8.12.0-2.el9.no 1.5 MB/s | 79 kB 00:00 2024-09-16T09:17:11.776 INFO:teuthology.orchestra.run.smithi084.stdout:(30/37): python3-cherrypy-18.6.1-2.el9.noarch.r 1.3 MB/s | 358 kB 00:00 2024-09-16T09:17:11.801 INFO:teuthology.orchestra.run.smithi084.stdout:(31/37): python3-portend-3.1.0-2.el9.noarch.rpm 326 kB/s | 16 kB 00:00 2024-09-16T09:17:11.826 INFO:teuthology.orchestra.run.smithi084.stdout:(32/37): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.8 MB/s | 90 kB 00:00 2024-09-16T09:17:11.852 INFO:teuthology.orchestra.run.smithi084.stdout:(33/37): python3-tempora-5.0.0-2.el9.noarch.rpm 713 kB/s | 36 kB 00:00 2024-09-16T09:17:11.885 INFO:teuthology.orchestra.run.smithi084.stdout:(34/37): python3-pecan-1.4.2-3.el9.noarch.rpm 1.7 MB/s | 272 kB 00:00 2024-09-16T09:17:11.910 INFO:teuthology.orchestra.run.smithi084.stdout:(35/37): python3-zc-lockfile-2.0-10.el9.noarch. 804 kB/s | 20 kB 00:00 2024-09-16T09:17:11.936 INFO:teuthology.orchestra.run.smithi084.stdout:(36/37): python3-webob-1.8.8-2.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-09-16T09:17:11.977 INFO:teuthology.orchestra.run.smithi084.stdout:(37/37): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.3 MB/s | 427 kB 00:00 2024-09-16T09:17:11.982 INFO:teuthology.orchestra.run.smithi084.stdout:-------------------------------------------------------------------------------- 2024-09-16T09:17:11.982 INFO:teuthology.orchestra.run.smithi084.stdout:Total 12 MB/s | 31 MB 00:02 2024-09-16T09:17:12.135 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction check 2024-09-16T09:17:12.176 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction check succeeded. 2024-09-16T09:17:12.176 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction test 2024-09-16T09:17:12.326 INFO:teuthology.orchestra.run.smithi070.stdout:(1/37): ceph-19.3.0-4917.g8293d73f.el9.x86_64.r 46 kB/s | 6.6 kB 00:00 2024-09-16T09:17:12.485 INFO:teuthology.orchestra.run.smithi070.stdout:(2/37): ceph-mgr-19.3.0-4917.g8293d73f.el9.x86_ 5.1 MB/s | 1.5 MB 00:00 2024-09-16T09:17:12.535 INFO:teuthology.orchestra.run.smithi070.stdout:(3/37): ceph-mds-19.3.0-4917.g8293d73f.el9.x86_ 6.8 MB/s | 2.4 MB 00:00 2024-09-16T09:17:12.611 INFO:teuthology.orchestra.run.smithi070.stdout:(4/37): ceph-mgr-modules-core-19.3.0-4917.g8293 3.8 MB/s | 291 kB 00:00 2024-09-16T09:17:12.639 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction test succeeded. 2024-09-16T09:17:12.639 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction 2024-09-16T09:17:12.811 INFO:teuthology.orchestra.run.smithi070.stdout:(5/37): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.2 MB/s | 253 kB 00:00 2024-09-16T09:17:13.003 INFO:teuthology.orchestra.run.smithi070.stdout:(6/37): ceph-osd-19.3.0-4917.g8293d73f.el9.x86_ 33 MB/s | 17 MB 00:00 2024-09-16T09:17:13.128 INFO:teuthology.orchestra.run.smithi070.stdout:(7/37): python3-cryptography-36.0.1-4.el9.x86_6 3.9 MB/s | 1.2 MB 00:00 2024-09-16T09:17:13.170 INFO:teuthology.orchestra.run.smithi070.stdout:(8/37): python3-ply-3.11-14.el9.noarch.rpm 639 kB/s | 106 kB 00:00 2024-09-16T09:17:13.195 INFO:teuthology.orchestra.run.smithi070.stdout:(9/37): python3-pycparser-2.20-6.el9.noarch.rpm 2.0 MB/s | 135 kB 00:00 2024-09-16T09:17:13.254 INFO:teuthology.orchestra.run.smithi070.stdout:(10/37): python3-urllib3-1.26.5-6.el9.noarch.rp 3.6 MB/s | 215 kB 00:00 2024-09-16T09:17:13.279 INFO:teuthology.orchestra.run.smithi070.stdout:(11/37): python3-requests-2.25.1-8.el9.noarch.r 1.1 MB/s | 125 kB 00:00 2024-09-16T09:17:13.363 INFO:teuthology.orchestra.run.smithi070.stdout:(12/37): ceph-mon-19.3.0-4917.g8293d73f.el9.x86 4.6 MB/s | 4.8 MB 00:01 2024-09-16T09:17:13.430 INFO:teuthology.orchestra.run.smithi070.stdout:(13/37): python3-mako-1.1.4-6.el9.noarch.rpm 1.1 MB/s | 172 kB 00:00 2024-09-16T09:17:13.455 INFO:teuthology.orchestra.run.smithi070.stdout:(14/37): lua-5.4.4-4.el9.x86_64.rpm 939 kB/s | 188 kB 00:00 2024-09-16T09:17:13.480 INFO:teuthology.orchestra.run.smithi070.stdout:(15/37): python3-markupsafe-1.1.1-12.el9.x86_64 296 kB/s | 35 kB 00:00 2024-09-16T09:17:13.497 INFO:teuthology.orchestra.run.smithi070.stdout:(16/37): python3-packaging-20.9-5.el9.noarch.rp 1.1 MB/s | 77 kB 00:00 2024-09-16T09:17:13.523 INFO:teuthology.orchestra.run.smithi070.stdout:(17/37): python3-pytz-2021.1-5.el9.noarch.rpm 1.2 MB/s | 51 kB 00:00 2024-09-16T09:17:13.548 INFO:teuthology.orchestra.run.smithi070.stdout:(18/37): python3-toml-0.10.2-6.el9.noarch.rpm 623 kB/s | 42 kB 00:00 2024-09-16T09:17:13.590 INFO:teuthology.orchestra.run.smithi070.stdout:(19/37): luarocks-3.9.2-5.el9.noarch.rpm 2.2 MB/s | 151 kB 00:00 2024-09-16T09:17:13.615 INFO:teuthology.orchestra.run.smithi070.stdout:(20/37): python3-cheroot-8.6.0-4.el9.noarch.rpm 7.2 MB/s | 172 kB 00:00 2024-09-16T09:17:13.640 INFO:teuthology.orchestra.run.smithi070.stdout:(21/37): lua-devel-5.4.4-4.el9.x86_64.rpm 156 kB/s | 22 kB 00:00 2024-09-16T09:17:13.690 INFO:teuthology.orchestra.run.smithi070.stdout:(22/37): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 305 kB/s | 43 kB 00:00 2024-09-16T09:17:13.716 INFO:teuthology.orchestra.run.smithi070.stdout:(23/37): python3-jaraco-8.2.1-3.el9.noarch.rpm 430 kB/s | 11 kB 00:00 2024-09-16T09:17:13.749 INFO:teuthology.orchestra.run.smithi070.stdout:(24/37): python3-cherrypy-18.6.1-2.el9.noarch.r 2.6 MB/s | 358 kB 00:00 2024-09-16T09:17:13.775 INFO:teuthology.orchestra.run.smithi070.stdout:(25/37): python3-jaraco-collections-3.0.0-8.el9 448 kB/s | 23 kB 00:00 2024-09-16T09:17:13.800 INFO:teuthology.orchestra.run.smithi070.stdout:(26/37): python3-jaraco-classes-3.2.1-5.el9.noa 162 kB/s | 18 kB 00:00 2024-09-16T09:17:13.825 INFO:teuthology.orchestra.run.smithi070.stdout:(27/37): python3-jaraco-functools-3.5.0-2.el9.n 263 kB/s | 19 kB 00:00 2024-09-16T09:17:13.850 INFO:teuthology.orchestra.run.smithi070.stdout:(28/37): python3-jaraco-text-3.2.0-6.el9.noarch 260 kB/s | 20 kB 00:00 2024-09-16T09:17:13.876 INFO:teuthology.orchestra.run.smithi070.stdout:(29/37): python3-logutils-0.3.5-21.el9.noarch.r 926 kB/s | 46 kB 00:00 2024-09-16T09:17:13.901 INFO:teuthology.orchestra.run.smithi070.stdout:(30/37): python3-more-itertools-8.12.0-2.el9.no 1.0 MB/s | 79 kB 00:00 2024-09-16T09:17:13.926 INFO:teuthology.orchestra.run.smithi070.stdout:(31/37): python3-portend-3.1.0-2.el9.noarch.rpm 327 kB/s | 16 kB 00:00 2024-09-16T09:17:13.927 INFO:teuthology.orchestra.run.smithi084.stdout: Preparing : 1/1 2024-09-16T09:17:13.951 INFO:teuthology.orchestra.run.smithi070.stdout:(32/37): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.7 MB/s | 90 kB 00:00 2024-09-16T09:17:13.977 INFO:teuthology.orchestra.run.smithi070.stdout:(33/37): python3-tempora-5.0.0-2.el9.noarch.rpm 711 kB/s | 36 kB 00:00 2024-09-16T09:17:13.988 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/37 2024-09-16T09:17:14.002 INFO:teuthology.orchestra.run.smithi070.stdout:(34/37): python3-pecan-1.4.2-3.el9.noarch.rpm 1.8 MB/s | 272 kB 00:00 2024-09-16T09:17:14.020 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/37 2024-09-16T09:17:14.027 INFO:teuthology.orchestra.run.smithi070.stdout:(35/37): python3-webob-1.8.8-2.el9.noarch.rpm 4.5 MB/s | 230 kB 00:00 2024-09-16T09:17:14.049 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/37 2024-09-16T09:17:14.052 INFO:teuthology.orchestra.run.smithi070.stdout:(36/37): python3-zc-lockfile-2.0-10.el9.noarch. 397 kB/s | 20 kB 00:00 2024-09-16T09:17:14.078 INFO:teuthology.orchestra.run.smithi070.stdout:(37/37): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.2 MB/s | 427 kB 00:00 2024-09-16T09:17:14.082 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-09-16T09:17:14.082 INFO:teuthology.orchestra.run.smithi070.stdout:Total 13 MB/s | 31 MB 00:02 2024-09-16T09:17:14.120 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/37 2024-09-16T09:17:14.173 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/37 2024-09-16T09:17:14.214 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/37 2024-09-16T09:17:14.227 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-09-16T09:17:14.268 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-09-16T09:17:14.268 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-09-16T09:17:14.304 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/37 2024-09-16T09:17:14.378 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/37 2024-09-16T09:17:14.433 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 9/37 2024-09-16T09:17:14.468 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/37 2024-09-16T09:17:14.498 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/37 2024-09-16T09:17:14.533 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/37 2024-09-16T09:17:14.566 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/37 2024-09-16T09:17:14.615 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/37 2024-09-16T09:17:14.650 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/37 2024-09-16T09:17:14.684 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/37 2024-09-16T09:17:14.733 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/37 2024-09-16T09:17:14.763 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-09-16T09:17:14.763 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-09-16T09:17:14.764 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-packaging-20.9-5.el9.noarch 18/37 2024-09-16T09:17:14.836 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 19/37 2024-09-16T09:17:14.974 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 20/37 2024-09-16T09:17:15.055 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 21/37 2024-09-16T09:17:15.108 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 22/37 2024-09-16T09:17:15.147 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 23/37 2024-09-16T09:17:15.195 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/37 2024-09-16T09:17:15.250 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/37 2024-09-16T09:17:15.443 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/37 2024-09-16T09:17:15.492 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 27/37 2024-09-16T09:17:15.562 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/37 2024-09-16T09:17:15.653 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 29/37 2024-09-16T09:17:15.688 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/37 2024-09-16T09:17:15.829 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/37 2024-09-16T09:17:15.997 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-4917.g8293d73f.el9. 32/37 2024-09-16T09:17:16.027 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-mgr-2:19.3.0-4917.g8293d73f.el9.x86_64 33/37 2024-09-16T09:17:16.053 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-mgr-2:19.3.0-4917.g8293d73f.el9.x86_64 33/37 2024-09-16T09:17:16.054 INFO:teuthology.orchestra.run.smithi084.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T09:17:16.054 INFO:teuthology.orchestra.run.smithi084.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-16T09:17:16.054 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T09:17:16.054 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T09:17:16.054 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-16T09:17:16.076 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-09-16T09:17:16.147 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/37 2024-09-16T09:17:16.179 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/37 2024-09-16T09:17:16.207 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/37 2024-09-16T09:17:16.278 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/37 2024-09-16T09:17:16.341 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/37 2024-09-16T09:17:16.381 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/37 2024-09-16T09:17:16.472 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/37 2024-09-16T09:17:16.551 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/37 2024-09-16T09:17:16.592 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 9/37 2024-09-16T09:17:16.627 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/37 2024-09-16T09:17:16.657 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/37 2024-09-16T09:17:16.691 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/37 2024-09-16T09:17:16.725 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/37 2024-09-16T09:17:16.765 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/37 2024-09-16T09:17:16.799 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/37 2024-09-16T09:17:16.846 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/37 2024-09-16T09:17:16.891 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/37 2024-09-16T09:17:16.931 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-packaging-20.9-5.el9.noarch 18/37 2024-09-16T09:17:17.004 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 19/37 2024-09-16T09:17:17.144 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 20/37 2024-09-16T09:17:17.225 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 21/37 2024-09-16T09:17:17.277 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 22/37 2024-09-16T09:17:17.317 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 23/37 2024-09-16T09:17:17.365 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/37 2024-09-16T09:17:17.421 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/37 2024-09-16T09:17:17.511 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-osd-2:19.3.0-4917.g8293d73f.el9.x86_64 34/37 2024-09-16T09:17:17.539 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-osd-2:19.3.0-4917.g8293d73f.el9.x86_64 34/37 2024-09-16T09:17:17.539 INFO:teuthology.orchestra.run.smithi084.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T09:17:17.539 INFO:teuthology.orchestra.run.smithi084.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-16T09:17:17.539 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T09:17:17.539 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T09:17:17.539 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-16T09:17:17.614 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/37 2024-09-16T09:17:17.662 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 27/37 2024-09-16T09:17:17.731 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/37 2024-09-16T09:17:17.819 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 29/37 2024-09-16T09:17:17.859 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/37 2024-09-16T09:17:17.987 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-mon-2:19.3.0-4917.g8293d73f.el9.x86_64 35/37 2024-09-16T09:17:17.994 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/37 2024-09-16T09:17:18.010 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-mon-2:19.3.0-4917.g8293d73f.el9.x86_64 35/37 2024-09-16T09:17:18.010 INFO:teuthology.orchestra.run.smithi084.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T09:17:18.011 INFO:teuthology.orchestra.run.smithi084.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-16T09:17:18.011 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T09:17:18.011 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T09:17:18.011 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-16T09:17:18.158 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-4917.g8293d73f.el9. 32/37 2024-09-16T09:17:18.189 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mgr-2:19.3.0-4917.g8293d73f.el9.x86_64 33/37 2024-09-16T09:17:18.214 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mgr-2:19.3.0-4917.g8293d73f.el9.x86_64 33/37 2024-09-16T09:17:18.214 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T09:17:18.214 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-16T09:17:18.215 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T09:17:18.215 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T09:17:18.215 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:17:18.237 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-mds-2:19.3.0-4917.g8293d73f.el9.x86_64 36/37 2024-09-16T09:17:18.261 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-mds-2:19.3.0-4917.g8293d73f.el9.x86_64 36/37 2024-09-16T09:17:18.261 INFO:teuthology.orchestra.run.smithi084.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T09:17:18.261 INFO:teuthology.orchestra.run.smithi084.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-16T09:17:18.261 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T09:17:18.261 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T09:17:18.261 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-16T09:17:18.325 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-2:19.3.0-4917.g8293d73f.el9.x86_64 37/37 2024-09-16T09:17:19.673 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-osd-2:19.3.0-4917.g8293d73f.el9.x86_64 34/37 2024-09-16T09:17:19.700 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-osd-2:19.3.0-4917.g8293d73f.el9.x86_64 34/37 2024-09-16T09:17:19.700 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T09:17:19.700 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-16T09:17:19.700 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T09:17:19.700 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T09:17:19.700 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:17:20.112 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-2:19.3.0-4917.g8293d73f.el9.x86_64 37/37 2024-09-16T09:17:20.112 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-2:19.3.0-4917.g8293d73f.el9.x86_64 1/37 2024-09-16T09:17:20.112 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-mds-2:19.3.0-4917.g8293d73f.el9.x86_64 2/37 2024-09-16T09:17:20.112 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-mgr-2:19.3.0-4917.g8293d73f.el9.x86_64 3/37 2024-09-16T09:17:20.112 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-mon-2:19.3.0-4917.g8293d73f.el9.x86_64 4/37 2024-09-16T09:17:20.113 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-osd-2:19.3.0-4917.g8293d73f.el9.x86_64 5/37 2024-09-16T09:17:20.113 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-4917.g8293d73f.el9. 6/37 2024-09-16T09:17:20.113 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/37 2024-09-16T09:17:20.113 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/37 2024-09-16T09:17:20.113 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/37 2024-09-16T09:17:20.113 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/37 2024-09-16T09:17:20.113 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/37 2024-09-16T09:17:20.113 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/37 2024-09-16T09:17:20.113 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/37 2024-09-16T09:17:20.113 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/37 2024-09-16T09:17:20.113 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/37 2024-09-16T09:17:20.113 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 16/37 2024-09-16T09:17:20.113 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 17/37 2024-09-16T09:17:20.114 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 18/37 2024-09-16T09:17:20.114 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 19/37 2024-09-16T09:17:20.114 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 20/37 2024-09-16T09:17:20.114 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 21/37 2024-09-16T09:17:20.114 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 22/37 2024-09-16T09:17:20.114 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 23/37 2024-09-16T09:17:20.114 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 24/37 2024-09-16T09:17:20.114 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 25/37 2024-09-16T09:17:20.114 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 26/37 2024-09-16T09:17:20.114 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 27/37 2024-09-16T09:17:20.115 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 28/37 2024-09-16T09:17:20.115 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 29/37 2024-09-16T09:17:20.116 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 30/37 2024-09-16T09:17:20.116 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 31/37 2024-09-16T09:17:20.116 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 32/37 2024-09-16T09:17:20.116 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 33/37 2024-09-16T09:17:20.116 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 34/37 2024-09-16T09:17:20.116 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 35/37 2024-09-16T09:17:20.116 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 36/37 2024-09-16T09:17:20.148 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mon-2:19.3.0-4917.g8293d73f.el9.x86_64 35/37 2024-09-16T09:17:20.171 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mon-2:19.3.0-4917.g8293d73f.el9.x86_64 35/37 2024-09-16T09:17:20.172 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T09:17:20.172 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-16T09:17:20.172 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T09:17:20.172 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T09:17:20.172 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:17:20.407 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mds-2:19.3.0-4917.g8293d73f.el9.x86_64 36/37 2024-09-16T09:17:20.431 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mds-2:19.3.0-4917.g8293d73f.el9.x86_64 36/37 2024-09-16T09:17:20.431 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T09:17:20.431 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-16T09:17:20.431 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T09:17:20.431 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T09:17:20.431 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:17:20.495 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-2:19.3.0-4917.g8293d73f.el9.x86_64 37/37 2024-09-16T09:17:21.247 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 37/37 2024-09-16T09:17:21.247 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-16T09:17:21.247 INFO:teuthology.orchestra.run.smithi084.stdout:Installed: 2024-09-16T09:17:21.247 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T09:17:21.247 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mds-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T09:17:21.247 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T09:17:21.247 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-modules-core-2:19.3.0-4917.g8293d73f.el9.noarch 2024-09-16T09:17:21.247 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mon-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T09:17:21.247 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-osd-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T09:17:21.247 INFO:teuthology.orchestra.run.smithi084.stdout: lua-5.4.4-4.el9.x86_64 2024-09-16T09:17:21.248 INFO:teuthology.orchestra.run.smithi084.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-16T09:17:21.248 INFO:teuthology.orchestra.run.smithi084.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-16T09:17:21.248 INFO:teuthology.orchestra.run.smithi084.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-16T09:17:21.248 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-16T09:17:21.248 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-16T09:17:21.248 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-16T09:17:21.248 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-16T09:17:21.248 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-16T09:17:21.248 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-16T09:17:21.248 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-16T09:17:21.248 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-16T09:17:21.248 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-16T09:17:21.248 INFO:teuthology.orchestra.run.smithi084.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-16T09:17:21.249 INFO:teuthology.orchestra.run.smithi084.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-16T09:17:21.249 INFO:teuthology.orchestra.run.smithi084.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-16T09:17:21.249 INFO:teuthology.orchestra.run.smithi084.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-16T09:17:21.249 INFO:teuthology.orchestra.run.smithi084.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-16T09:17:21.249 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-16T09:17:21.249 INFO:teuthology.orchestra.run.smithi084.stdout: python3-ply-3.11-14.el9.noarch 2024-09-16T09:17:21.249 INFO:teuthology.orchestra.run.smithi084.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-16T09:17:21.249 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-16T09:17:21.249 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-16T09:17:21.249 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-16T09:17:21.249 INFO:teuthology.orchestra.run.smithi084.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-16T09:17:21.249 INFO:teuthology.orchestra.run.smithi084.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-16T09:17:21.249 INFO:teuthology.orchestra.run.smithi084.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-16T09:17:21.249 INFO:teuthology.orchestra.run.smithi084.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-16T09:17:21.250 INFO:teuthology.orchestra.run.smithi084.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-16T09:17:21.250 INFO:teuthology.orchestra.run.smithi084.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-16T09:17:21.250 INFO:teuthology.orchestra.run.smithi084.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-16T09:17:21.250 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-16T09:17:21.250 INFO:teuthology.orchestra.run.smithi084.stdout:Complete! 2024-09-16T09:17:21.606 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum -y install ceph-base 2024-09-16T09:17:22.169 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-2:19.3.0-4917.g8293d73f.el9.x86_64 37/37 2024-09-16T09:17:22.169 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-2:19.3.0-4917.g8293d73f.el9.x86_64 1/37 2024-09-16T09:17:22.169 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mds-2:19.3.0-4917.g8293d73f.el9.x86_64 2/37 2024-09-16T09:17:22.169 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mgr-2:19.3.0-4917.g8293d73f.el9.x86_64 3/37 2024-09-16T09:17:22.169 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mon-2:19.3.0-4917.g8293d73f.el9.x86_64 4/37 2024-09-16T09:17:22.169 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-osd-2:19.3.0-4917.g8293d73f.el9.x86_64 5/37 2024-09-16T09:17:22.169 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-4917.g8293d73f.el9. 6/37 2024-09-16T09:17:22.170 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/37 2024-09-16T09:17:22.170 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/37 2024-09-16T09:17:22.170 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/37 2024-09-16T09:17:22.170 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/37 2024-09-16T09:17:22.170 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/37 2024-09-16T09:17:22.170 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/37 2024-09-16T09:17:22.170 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/37 2024-09-16T09:17:22.170 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/37 2024-09-16T09:17:22.170 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/37 2024-09-16T09:17:22.170 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 16/37 2024-09-16T09:17:22.170 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 17/37 2024-09-16T09:17:22.170 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 18/37 2024-09-16T09:17:22.170 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 19/37 2024-09-16T09:17:22.171 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 20/37 2024-09-16T09:17:22.171 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 21/37 2024-09-16T09:17:22.171 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 22/37 2024-09-16T09:17:22.171 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 23/37 2024-09-16T09:17:22.171 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 24/37 2024-09-16T09:17:22.171 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 25/37 2024-09-16T09:17:22.171 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 26/37 2024-09-16T09:17:22.171 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 27/37 2024-09-16T09:17:22.171 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 28/37 2024-09-16T09:17:22.171 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 29/37 2024-09-16T09:17:22.171 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 30/37 2024-09-16T09:17:22.171 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 31/37 2024-09-16T09:17:22.171 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 32/37 2024-09-16T09:17:22.172 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 33/37 2024-09-16T09:17:22.172 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 34/37 2024-09-16T09:17:22.172 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 35/37 2024-09-16T09:17:22.172 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 36/37 2024-09-16T09:17:22.179 INFO:teuthology.orchestra.run.smithi084.stdout:Last metadata expiration check: 0:00:53 ago on Mon 16 Sep 2024 09:16:29 AM UTC. 2024-09-16T09:17:22.270 INFO:teuthology.orchestra.run.smithi084.stdout:Package ceph-base-2:19.3.0-4917.g8293d73f.el9.x86_64 is already installed. 2024-09-16T09:17:22.314 INFO:teuthology.orchestra.run.smithi084.stdout:Dependencies resolved. 2024-09-16T09:17:22.315 INFO:teuthology.orchestra.run.smithi084.stdout:Nothing to do. 2024-09-16T09:17:22.315 INFO:teuthology.orchestra.run.smithi084.stdout:Complete! 2024-09-16T09:17:22.374 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum -y install cephadm 2024-09-16T09:17:22.936 INFO:teuthology.orchestra.run.smithi084.stdout:Last metadata expiration check: 0:00:53 ago on Mon 16 Sep 2024 09:16:29 AM UTC. 2024-09-16T09:17:23.069 INFO:teuthology.orchestra.run.smithi084.stdout:Dependencies resolved. 2024-09-16T09:17:23.070 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-09-16T09:17:23.070 INFO:teuthology.orchestra.run.smithi084.stdout: Package Arch Version Repository Size 2024-09-16T09:17:23.070 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-09-16T09:17:23.070 INFO:teuthology.orchestra.run.smithi084.stdout:Installing: 2024-09-16T09:17:23.071 INFO:teuthology.orchestra.run.smithi084.stdout: cephadm noarch 2:19.3.0-4917.g8293d73f.el9 ceph-noarch 784 k 2024-09-16T09:17:23.071 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-16T09:17:23.071 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction Summary 2024-09-16T09:17:23.071 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-09-16T09:17:23.071 INFO:teuthology.orchestra.run.smithi084.stdout:Install 1 Package 2024-09-16T09:17:23.071 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-16T09:17:23.071 INFO:teuthology.orchestra.run.smithi084.stdout:Total download size: 784 k 2024-09-16T09:17:23.071 INFO:teuthology.orchestra.run.smithi084.stdout:Installed size: 791 k 2024-09-16T09:17:23.071 INFO:teuthology.orchestra.run.smithi084.stdout:Downloading Packages: 2024-09-16T09:17:23.094 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 37/37 2024-09-16T09:17:23.094 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:17:23.094 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-09-16T09:17:23.094 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T09:17:23.094 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mds-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T09:17:23.094 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T09:17:23.094 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-modules-core-2:19.3.0-4917.g8293d73f.el9.noarch 2024-09-16T09:17:23.094 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mon-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T09:17:23.094 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-osd-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T09:17:23.094 INFO:teuthology.orchestra.run.smithi070.stdout: lua-5.4.4-4.el9.x86_64 2024-09-16T09:17:23.094 INFO:teuthology.orchestra.run.smithi070.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-16T09:17:23.094 INFO:teuthology.orchestra.run.smithi070.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-16T09:17:23.095 INFO:teuthology.orchestra.run.smithi070.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-16T09:17:23.095 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-16T09:17:23.095 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-16T09:17:23.095 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-16T09:17:23.095 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-16T09:17:23.095 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-16T09:17:23.095 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-16T09:17:23.095 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-16T09:17:23.096 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-16T09:17:23.096 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-16T09:17:23.096 INFO:teuthology.orchestra.run.smithi070.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-16T09:17:23.096 INFO:teuthology.orchestra.run.smithi070.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-16T09:17:23.096 INFO:teuthology.orchestra.run.smithi070.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-16T09:17:23.096 INFO:teuthology.orchestra.run.smithi070.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-16T09:17:23.096 INFO:teuthology.orchestra.run.smithi070.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-16T09:17:23.096 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-16T09:17:23.096 INFO:teuthology.orchestra.run.smithi070.stdout: python3-ply-3.11-14.el9.noarch 2024-09-16T09:17:23.096 INFO:teuthology.orchestra.run.smithi070.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-16T09:17:23.096 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-16T09:17:23.096 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-16T09:17:23.096 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-16T09:17:23.097 INFO:teuthology.orchestra.run.smithi070.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-16T09:17:23.097 INFO:teuthology.orchestra.run.smithi070.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-16T09:17:23.097 INFO:teuthology.orchestra.run.smithi070.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-16T09:17:23.097 INFO:teuthology.orchestra.run.smithi070.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-16T09:17:23.097 INFO:teuthology.orchestra.run.smithi070.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-16T09:17:23.097 INFO:teuthology.orchestra.run.smithi070.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-16T09:17:23.097 INFO:teuthology.orchestra.run.smithi070.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-16T09:17:23.097 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:17:23.097 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-16T09:17:23.319 INFO:teuthology.orchestra.run.smithi084.stdout:cephadm-19.3.0-4917.g8293d73f.el9.noarch.rpm 3.1 MB/s | 784 kB 00:00 2024-09-16T09:17:23.320 INFO:teuthology.orchestra.run.smithi084.stdout:-------------------------------------------------------------------------------- 2024-09-16T09:17:23.320 INFO:teuthology.orchestra.run.smithi084.stdout:Total 3.1 MB/s | 784 kB 00:00 2024-09-16T09:17:23.320 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction check 2024-09-16T09:17:23.325 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction check succeeded. 2024-09-16T09:17:23.325 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction test 2024-09-16T09:17:23.331 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction test succeeded. 2024-09-16T09:17:23.332 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction 2024-09-16T09:17:23.401 INFO:teuthology.orchestra.run.smithi084.stdout: Preparing : 1/1 2024-09-16T09:17:23.509 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install ceph-base 2024-09-16T09:17:23.872 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: cephadm-2:19.3.0-4917.g8293d73f.el9.noarch 1/1 2024-09-16T09:17:23.965 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : cephadm-2:19.3.0-4917.g8293d73f.el9.noarch 1/1 2024-09-16T09:17:24.074 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:00:52 ago on Mon 16 Sep 2024 09:16:32 AM UTC. 2024-09-16T09:17:24.164 INFO:teuthology.orchestra.run.smithi070.stdout:Package ceph-base-2:19.3.0-4917.g8293d73f.el9.x86_64 is already installed. 2024-09-16T09:17:24.206 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-16T09:17:24.207 INFO:teuthology.orchestra.run.smithi070.stdout:Nothing to do. 2024-09-16T09:17:24.207 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-16T09:17:24.277 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install cephadm 2024-09-16T09:17:24.313 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: cephadm-2:19.3.0-4917.g8293d73f.el9.noarch 1/1 2024-09-16T09:17:24.570 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : cephadm-2:19.3.0-4917.g8293d73f.el9.noarch 1/1 2024-09-16T09:17:24.570 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-16T09:17:24.571 INFO:teuthology.orchestra.run.smithi084.stdout:Installed: 2024-09-16T09:17:24.571 INFO:teuthology.orchestra.run.smithi084.stdout: cephadm-2:19.3.0-4917.g8293d73f.el9.noarch 2024-09-16T09:17:24.571 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-16T09:17:24.571 INFO:teuthology.orchestra.run.smithi084.stdout:Complete! 2024-09-16T09:17:24.771 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum -y install ceph-immutable-object-cache 2024-09-16T09:17:24.835 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:00:52 ago on Mon 16 Sep 2024 09:16:32 AM UTC. 2024-09-16T09:17:24.966 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-16T09:17:24.967 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-16T09:17:24.967 INFO:teuthology.orchestra.run.smithi070.stdout: Package Arch Version Repository Size 2024-09-16T09:17:24.967 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-16T09:17:24.967 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-09-16T09:17:24.967 INFO:teuthology.orchestra.run.smithi070.stdout: cephadm noarch 2:19.3.0-4917.g8293d73f.el9 ceph-noarch 784 k 2024-09-16T09:17:24.968 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:17:24.968 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-09-16T09:17:24.968 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-16T09:17:24.968 INFO:teuthology.orchestra.run.smithi070.stdout:Install 1 Package 2024-09-16T09:17:24.968 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:17:24.968 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 784 k 2024-09-16T09:17:24.968 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 791 k 2024-09-16T09:17:24.968 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-09-16T09:17:25.207 INFO:teuthology.orchestra.run.smithi070.stdout:cephadm-19.3.0-4917.g8293d73f.el9.noarch.rpm 3.2 MB/s | 784 kB 00:00 2024-09-16T09:17:25.208 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-09-16T09:17:25.208 INFO:teuthology.orchestra.run.smithi070.stdout:Total 3.2 MB/s | 784 kB 00:00 2024-09-16T09:17:25.208 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-09-16T09:17:25.213 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-09-16T09:17:25.213 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-09-16T09:17:25.219 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-09-16T09:17:25.219 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-09-16T09:17:25.294 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-09-16T09:17:25.344 INFO:teuthology.orchestra.run.smithi084.stdout:Last metadata expiration check: 0:00:56 ago on Mon 16 Sep 2024 09:16:29 AM UTC. 2024-09-16T09:17:25.478 INFO:teuthology.orchestra.run.smithi084.stdout:Dependencies resolved. 2024-09-16T09:17:25.478 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-09-16T09:17:25.479 INFO:teuthology.orchestra.run.smithi084.stdout: Package Arch Version Repo Size 2024-09-16T09:17:25.479 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-09-16T09:17:25.479 INFO:teuthology.orchestra.run.smithi084.stdout:Installing: 2024-09-16T09:17:25.479 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 145 k 2024-09-16T09:17:25.479 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-16T09:17:25.479 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction Summary 2024-09-16T09:17:25.479 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-09-16T09:17:25.479 INFO:teuthology.orchestra.run.smithi084.stdout:Install 1 Package 2024-09-16T09:17:25.479 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-16T09:17:25.480 INFO:teuthology.orchestra.run.smithi084.stdout:Total download size: 145 k 2024-09-16T09:17:25.480 INFO:teuthology.orchestra.run.smithi084.stdout:Installed size: 432 k 2024-09-16T09:17:25.480 INFO:teuthology.orchestra.run.smithi084.stdout:Downloading Packages: 2024-09-16T09:17:25.658 INFO:teuthology.orchestra.run.smithi084.stdout:ceph-immutable-object-cache-19.3.0-4917.g8293d7 816 kB/s | 145 kB 00:00 2024-09-16T09:17:25.659 INFO:teuthology.orchestra.run.smithi084.stdout:-------------------------------------------------------------------------------- 2024-09-16T09:17:25.659 INFO:teuthology.orchestra.run.smithi084.stdout:Total 808 kB/s | 145 kB 00:00 2024-09-16T09:17:25.659 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction check 2024-09-16T09:17:25.667 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction check succeeded. 2024-09-16T09:17:25.667 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction test 2024-09-16T09:17:25.704 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: cephadm-2:19.3.0-4917.g8293d73f.el9.noarch 1/1 2024-09-16T09:17:25.725 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction test succeeded. 2024-09-16T09:17:25.725 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction 2024-09-16T09:17:25.802 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : cephadm-2:19.3.0-4917.g8293d73f.el9.noarch 1/1 2024-09-16T09:17:25.882 INFO:teuthology.orchestra.run.smithi084.stdout: Preparing : 1/1 2024-09-16T09:17:25.959 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-4917.g8293d73f. 1/1 2024-09-16T09:17:25.984 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-4917.g8293d73f. 1/1 2024-09-16T09:17:25.984 INFO:teuthology.orchestra.run.smithi084.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T09:17:25.984 INFO:teuthology.orchestra.run.smithi084.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-16T09:17:25.984 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-16T09:17:26.168 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: cephadm-2:19.3.0-4917.g8293d73f.el9.noarch 1/1 2024-09-16T09:17:26.423 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : cephadm-2:19.3.0-4917.g8293d73f.el9.noarch 1/1 2024-09-16T09:17:26.423 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:17:26.423 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-09-16T09:17:26.423 INFO:teuthology.orchestra.run.smithi070.stdout: cephadm-2:19.3.0-4917.g8293d73f.el9.noarch 2024-09-16T09:17:26.423 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:17:26.423 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-16T09:17:26.642 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install ceph-immutable-object-cache 2024-09-16T09:17:26.894 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-4917.g8293d73f. 1/1 2024-09-16T09:17:26.894 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-16T09:17:26.894 INFO:teuthology.orchestra.run.smithi084.stdout:Installed: 2024-09-16T09:17:26.894 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-immutable-object-cache-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T09:17:26.894 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-16T09:17:26.894 INFO:teuthology.orchestra.run.smithi084.stdout:Complete! 2024-09-16T09:17:27.094 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum -y install ceph-mgr 2024-09-16T09:17:27.222 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:00:55 ago on Mon 16 Sep 2024 09:16:32 AM UTC. 2024-09-16T09:17:27.352 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-16T09:17:27.353 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-16T09:17:27.353 INFO:teuthology.orchestra.run.smithi070.stdout: Package Arch Version Repo Size 2024-09-16T09:17:27.353 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-16T09:17:27.353 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-09-16T09:17:27.353 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 145 k 2024-09-16T09:17:27.353 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:17:27.353 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-09-16T09:17:27.353 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-16T09:17:27.353 INFO:teuthology.orchestra.run.smithi070.stdout:Install 1 Package 2024-09-16T09:17:27.353 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:17:27.353 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 145 k 2024-09-16T09:17:27.354 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 432 k 2024-09-16T09:17:27.354 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-09-16T09:17:27.572 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-immutable-object-cache-19.3.0-4917.g8293d7 667 kB/s | 145 kB 00:00 2024-09-16T09:17:27.572 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-09-16T09:17:27.573 INFO:teuthology.orchestra.run.smithi070.stdout:Total 662 kB/s | 145 kB 00:00 2024-09-16T09:17:27.573 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-09-16T09:17:27.580 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-09-16T09:17:27.580 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-09-16T09:17:27.638 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-09-16T09:17:27.638 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-09-16T09:17:27.672 INFO:teuthology.orchestra.run.smithi084.stdout:Last metadata expiration check: 0:00:58 ago on Mon 16 Sep 2024 09:16:29 AM UTC. 2024-09-16T09:17:27.763 INFO:teuthology.orchestra.run.smithi084.stdout:Package ceph-mgr-2:19.3.0-4917.g8293d73f.el9.x86_64 is already installed. 2024-09-16T09:17:27.795 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-09-16T09:17:27.805 INFO:teuthology.orchestra.run.smithi084.stdout:Dependencies resolved. 2024-09-16T09:17:27.806 INFO:teuthology.orchestra.run.smithi084.stdout:Nothing to do. 2024-09-16T09:17:27.806 INFO:teuthology.orchestra.run.smithi084.stdout:Complete! 2024-09-16T09:17:27.856 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-4917.g8293d73f. 1/1 2024-09-16T09:17:27.866 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum -y install ceph-mgr-dashboard 2024-09-16T09:17:27.882 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-4917.g8293d73f. 1/1 2024-09-16T09:17:27.882 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T09:17:27.883 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-16T09:17:27.883 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:17:28.422 INFO:teuthology.orchestra.run.smithi084.stdout:Last metadata expiration check: 0:00:59 ago on Mon 16 Sep 2024 09:16:29 AM UTC. 2024-09-16T09:17:28.556 INFO:teuthology.orchestra.run.smithi084.stdout:Dependencies resolved. 2024-09-16T09:17:28.557 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-09-16T09:17:28.557 INFO:teuthology.orchestra.run.smithi084.stdout: Package Arch Version Repository Size 2024-09-16T09:17:28.557 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-09-16T09:17:28.557 INFO:teuthology.orchestra.run.smithi084.stdout:Installing: 2024-09-16T09:17:28.557 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-dashboard noarch 2:19.3.0-4917.g8293d73f.el9 ceph-noarch 14 M 2024-09-16T09:17:28.557 INFO:teuthology.orchestra.run.smithi084.stdout:Installing dependencies: 2024-09-16T09:17:28.557 INFO:teuthology.orchestra.run.smithi084.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-16T09:17:28.557 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-grafana-dashboards noarch 2:19.3.0-4917.g8293d73f.el9 ceph-noarch 34 k 2024-09-16T09:17:28.557 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-prometheus-alerts noarch 2:19.3.0-4917.g8293d73f.el9 ceph-noarch 16 k 2024-09-16T09:17:28.558 INFO:teuthology.orchestra.run.smithi084.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-16T09:17:28.558 INFO:teuthology.orchestra.run.smithi084.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-16T09:17:28.558 INFO:teuthology.orchestra.run.smithi084.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-16T09:17:28.558 INFO:teuthology.orchestra.run.smithi084.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-16T09:17:28.558 INFO:teuthology.orchestra.run.smithi084.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-16T09:17:28.558 INFO:teuthology.orchestra.run.smithi084.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-16T09:17:28.558 INFO:teuthology.orchestra.run.smithi084.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-16T09:17:28.558 INFO:teuthology.orchestra.run.smithi084.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-16T09:17:28.559 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-16T09:17:28.559 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction Summary 2024-09-16T09:17:28.559 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-09-16T09:17:28.559 INFO:teuthology.orchestra.run.smithi084.stdout:Install 12 Packages 2024-09-16T09:17:28.559 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-16T09:17:28.560 INFO:teuthology.orchestra.run.smithi084.stdout:Total download size: 19 M 2024-09-16T09:17:28.560 INFO:teuthology.orchestra.run.smithi084.stdout:Installed size: 159 M 2024-09-16T09:17:28.560 INFO:teuthology.orchestra.run.smithi084.stdout:Downloading Packages: 2024-09-16T09:17:28.796 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-4917.g8293d73f. 1/1 2024-09-16T09:17:28.796 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:17:28.796 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-09-16T09:17:28.796 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-immutable-object-cache-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T09:17:28.796 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:17:28.796 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-16T09:17:28.903 INFO:teuthology.orchestra.run.smithi084.stdout:(1/12): ceph-grafana-dashboards-19.3.0-4917.g82 216 kB/s | 34 kB 00:00 2024-09-16T09:17:28.928 INFO:teuthology.orchestra.run.smithi084.stdout:(2/12): ceph-prometheus-alerts-19.3.0-4917.g829 90 kB/s | 16 kB 00:00 2024-09-16T09:17:29.020 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install ceph-mgr 2024-09-16T09:17:29.053 INFO:teuthology.orchestra.run.smithi084.stdout:(3/12): python3-protobuf-3.14.0-13.el9.noarch.r 2.1 MB/s | 269 kB 00:00 2024-09-16T09:17:29.095 INFO:teuthology.orchestra.run.smithi084.stdout:(4/12): protobuf-3.14.0-13.el9.x86_64.rpm 5.3 MB/s | 1.0 MB 00:00 2024-09-16T09:17:29.162 INFO:teuthology.orchestra.run.smithi084.stdout:(5/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 8.0 MB/s | 548 kB 00:00 2024-09-16T09:17:29.187 INFO:teuthology.orchestra.run.smithi084.stdout:(6/12): grpc-data-1.46.7-10.el9.noarch.rpm 782 kB/s | 19 kB 00:00 2024-09-16T09:17:29.254 INFO:teuthology.orchestra.run.smithi084.stdout:(7/12): python3-grpcio-1.46.7-10.el9.x86_64.rpm 31 MB/s | 2.0 MB 00:00 2024-09-16T09:17:29.280 INFO:teuthology.orchestra.run.smithi084.stdout:(8/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 5.7 MB/s | 144 kB 00:00 2024-09-16T09:17:29.305 INFO:teuthology.orchestra.run.smithi084.stdout:(9/12): python3-repoze-lru-0.7-16.el9.noarch.rp 1.2 MB/s | 31 kB 00:00 2024-09-16T09:17:29.338 INFO:teuthology.orchestra.run.smithi084.stdout:(10/12): python3-routes-2.5.1-5.el9.noarch.rpm 5.5 MB/s | 188 kB 00:00 2024-09-16T09:17:29.397 INFO:teuthology.orchestra.run.smithi084.stdout:(11/12): protobuf-compiler-3.14.0-13.el9.x86_64 2.5 MB/s | 863 kB 00:00 2024-09-16T09:17:29.547 INFO:teuthology.orchestra.run.smithi084.stdout:(12/12): ceph-mgr-dashboard-19.3.0-4917.g8293d7 18 MB/s | 14 MB 00:00 2024-09-16T09:17:29.550 INFO:teuthology.orchestra.run.smithi084.stdout:-------------------------------------------------------------------------------- 2024-09-16T09:17:29.550 INFO:teuthology.orchestra.run.smithi084.stdout:Total 19 MB/s | 19 MB 00:00 2024-09-16T09:17:29.589 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:00:57 ago on Mon 16 Sep 2024 09:16:32 AM UTC. 2024-09-16T09:17:29.600 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction check 2024-09-16T09:17:29.618 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction check succeeded. 2024-09-16T09:17:29.618 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction test 2024-09-16T09:17:29.679 INFO:teuthology.orchestra.run.smithi070.stdout:Package ceph-mgr-2:19.3.0-4917.g8293d73f.el9.x86_64 is already installed. 2024-09-16T09:17:29.723 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-16T09:17:29.724 INFO:teuthology.orchestra.run.smithi070.stdout:Nothing to do. 2024-09-16T09:17:29.724 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-16T09:17:29.796 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install ceph-mgr-dashboard 2024-09-16T09:17:29.893 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction test succeeded. 2024-09-16T09:17:29.893 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction 2024-09-16T09:17:30.355 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:00:58 ago on Mon 16 Sep 2024 09:16:32 AM UTC. 2024-09-16T09:17:30.486 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-16T09:17:30.487 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-16T09:17:30.487 INFO:teuthology.orchestra.run.smithi070.stdout: Package Arch Version Repository Size 2024-09-16T09:17:30.488 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-16T09:17:30.488 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-09-16T09:17:30.488 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-dashboard noarch 2:19.3.0-4917.g8293d73f.el9 ceph-noarch 14 M 2024-09-16T09:17:30.488 INFO:teuthology.orchestra.run.smithi070.stdout:Installing dependencies: 2024-09-16T09:17:30.488 INFO:teuthology.orchestra.run.smithi070.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-16T09:17:30.488 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-grafana-dashboards noarch 2:19.3.0-4917.g8293d73f.el9 ceph-noarch 34 k 2024-09-16T09:17:30.488 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-prometheus-alerts noarch 2:19.3.0-4917.g8293d73f.el9 ceph-noarch 16 k 2024-09-16T09:17:30.488 INFO:teuthology.orchestra.run.smithi070.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-16T09:17:30.488 INFO:teuthology.orchestra.run.smithi070.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-16T09:17:30.488 INFO:teuthology.orchestra.run.smithi070.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-16T09:17:30.488 INFO:teuthology.orchestra.run.smithi070.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-16T09:17:30.488 INFO:teuthology.orchestra.run.smithi070.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-16T09:17:30.488 INFO:teuthology.orchestra.run.smithi070.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-16T09:17:30.488 INFO:teuthology.orchestra.run.smithi070.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-16T09:17:30.489 INFO:teuthology.orchestra.run.smithi070.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-16T09:17:30.489 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:17:30.489 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-09-16T09:17:30.489 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-16T09:17:30.489 INFO:teuthology.orchestra.run.smithi070.stdout:Install 12 Packages 2024-09-16T09:17:30.489 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:17:30.489 INFO:teuthology.orchestra.run.smithi084.stdout: Preparing : 1/1 2024-09-16T09:17:30.490 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 19 M 2024-09-16T09:17:30.490 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 159 M 2024-09-16T09:17:30.490 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-09-16T09:17:30.565 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-09-16T09:17:30.609 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-09-16T09:17:30.655 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-09-16T09:17:30.694 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-09-16T09:17:30.774 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-09-16T09:17:30.903 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-09-16T09:17:30.997 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-09-16T09:17:31.048 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-09-16T09:17:31.076 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-09-16T09:17:31.109 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-4917.g8293d73f.el9 10/12 2024-09-16T09:17:31.176 INFO:teuthology.orchestra.run.smithi070.stdout:(1/12): ceph-prometheus-alerts-19.3.0-4917.g829 110 kB/s | 16 kB 00:00 2024-09-16T09:17:31.201 INFO:teuthology.orchestra.run.smithi070.stdout:(2/12): ceph-grafana-dashboards-19.3.0-4917.g82 196 kB/s | 34 kB 00:00 2024-09-16T09:17:31.376 INFO:teuthology.orchestra.run.smithi070.stdout:(3/12): python3-protobuf-3.14.0-13.el9.noarch.r 1.5 MB/s | 269 kB 00:00 2024-09-16T09:17:31.435 INFO:teuthology.orchestra.run.smithi070.stdout:(4/12): protobuf-3.14.0-13.el9.x86_64.rpm 3.9 MB/s | 1.0 MB 00:00 2024-09-16T09:17:31.502 INFO:teuthology.orchestra.run.smithi070.stdout:(5/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 8.1 MB/s | 548 kB 00:00 2024-09-16T09:17:31.527 INFO:teuthology.orchestra.run.smithi070.stdout:(6/12): grpc-data-1.46.7-10.el9.noarch.rpm 778 kB/s | 19 kB 00:00 2024-09-16T09:17:31.686 INFO:teuthology.orchestra.run.smithi070.stdout:(7/12): ceph-mgr-dashboard-19.3.0-4917.g8293d73 21 MB/s | 14 MB 00:00 2024-09-16T09:17:31.761 INFO:teuthology.orchestra.run.smithi070.stdout:(8/12): python3-grpcio-1.46.7-10.el9.x86_64.rpm 8.7 MB/s | 2.0 MB 00:00 2024-09-16T09:17:31.786 INFO:teuthology.orchestra.run.smithi070.stdout:(9/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 1.4 MB/s | 144 kB 00:00 2024-09-16T09:17:31.812 INFO:teuthology.orchestra.run.smithi070.stdout:(10/12): python3-repoze-lru-0.7-16.el9.noarch.r 613 kB/s | 31 kB 00:00 2024-09-16T09:17:31.837 INFO:teuthology.orchestra.run.smithi070.stdout:(11/12): python3-routes-2.5.1-5.el9.noarch.rpm 3.7 MB/s | 188 kB 00:00 2024-09-16T09:17:32.320 INFO:teuthology.orchestra.run.smithi070.stdout:(12/12): protobuf-compiler-3.14.0-13.el9.x86_64 915 kB/s | 863 kB 00:00 2024-09-16T09:17:32.324 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-09-16T09:17:32.324 INFO:teuthology.orchestra.run.smithi070.stdout:Total 11 MB/s | 19 MB 00:01 2024-09-16T09:17:32.371 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-09-16T09:17:32.390 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-09-16T09:17:32.390 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-09-16T09:17:32.648 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-09-16T09:17:32.649 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-09-16T09:17:33.249 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-09-16T09:17:33.313 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-09-16T09:17:33.356 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-09-16T09:17:33.411 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-09-16T09:17:33.450 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-09-16T09:17:33.538 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-09-16T09:17:33.666 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-09-16T09:17:33.752 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-09-16T09:17:33.796 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-09-16T09:17:33.825 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-09-16T09:17:33.858 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-4917.g8293d73f.el9 10/12 2024-09-16T09:17:34.288 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-4917.g8293d73f.el 11/12 2024-09-16T09:17:34.346 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-4917.g8293d73f.el9.noa 12/12 2024-09-16T09:17:36.294 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-4917.g8293d73f.el9.noa 12/12 2024-09-16T09:17:36.294 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-4917.g8293d73f.el 1/12 2024-09-16T09:17:36.294 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-4917.g8293d73f.el9.noa 2/12 2024-09-16T09:17:36.294 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-4917.g8293d73f.el9 3/12 2024-09-16T09:17:36.295 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-09-16T09:17:36.295 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-09-16T09:17:36.295 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-09-16T09:17:36.295 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-09-16T09:17:36.295 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-09-16T09:17:36.295 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-09-16T09:17:36.295 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-09-16T09:17:36.295 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-09-16T09:17:36.611 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-09-16T09:17:36.611 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-16T09:17:36.612 INFO:teuthology.orchestra.run.smithi084.stdout:Installed: 2024-09-16T09:17:36.612 INFO:teuthology.orchestra.run.smithi084.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-16T09:17:36.612 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-grafana-dashboards-2:19.3.0-4917.g8293d73f.el9.noarch 2024-09-16T09:17:36.612 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-dashboard-2:19.3.0-4917.g8293d73f.el9.noarch 2024-09-16T09:17:36.612 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-prometheus-alerts-2:19.3.0-4917.g8293d73f.el9.noarch 2024-09-16T09:17:36.612 INFO:teuthology.orchestra.run.smithi084.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-16T09:17:36.612 INFO:teuthology.orchestra.run.smithi084.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-16T09:17:36.612 INFO:teuthology.orchestra.run.smithi084.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-16T09:17:36.612 INFO:teuthology.orchestra.run.smithi084.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-16T09:17:36.612 INFO:teuthology.orchestra.run.smithi084.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-16T09:17:36.613 INFO:teuthology.orchestra.run.smithi084.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-16T09:17:36.613 INFO:teuthology.orchestra.run.smithi084.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-16T09:17:36.613 INFO:teuthology.orchestra.run.smithi084.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-16T09:17:36.613 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-16T09:17:36.613 INFO:teuthology.orchestra.run.smithi084.stdout:Complete! 2024-09-16T09:17:36.872 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-16T09:17:37.032 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-4917.g8293d73f.el 11/12 2024-09-16T09:17:37.087 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-4917.g8293d73f.el9.noa 12/12 2024-09-16T09:17:37.437 INFO:teuthology.orchestra.run.smithi084.stdout:Last metadata expiration check: 0:01:08 ago on Mon 16 Sep 2024 09:16:29 AM UTC. 2024-09-16T09:17:37.571 INFO:teuthology.orchestra.run.smithi084.stdout:Dependencies resolved. 2024-09-16T09:17:37.572 INFO:teuthology.orchestra.run.smithi084.stdout:======================================================================================= 2024-09-16T09:17:37.572 INFO:teuthology.orchestra.run.smithi084.stdout: Package Arch Version Repository Size 2024-09-16T09:17:37.573 INFO:teuthology.orchestra.run.smithi084.stdout:======================================================================================= 2024-09-16T09:17:37.573 INFO:teuthology.orchestra.run.smithi084.stdout:Installing: 2024-09-16T09:17:37.573 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-4917.g8293d73f.el9 ceph-noarch 7.4 M 2024-09-16T09:17:37.573 INFO:teuthology.orchestra.run.smithi084.stdout:Installing dependencies: 2024-09-16T09:17:37.573 INFO:teuthology.orchestra.run.smithi084.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-16T09:17:37.573 INFO:teuthology.orchestra.run.smithi084.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-16T09:17:37.573 INFO:teuthology.orchestra.run.smithi084.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-16T09:17:37.573 INFO:teuthology.orchestra.run.smithi084.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-16T09:17:37.573 INFO:teuthology.orchestra.run.smithi084.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-16T09:17:37.573 INFO:teuthology.orchestra.run.smithi084.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-16T09:17:37.573 INFO:teuthology.orchestra.run.smithi084.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-16T09:17:37.573 INFO:teuthology.orchestra.run.smithi084.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-16T09:17:37.573 INFO:teuthology.orchestra.run.smithi084.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-16T09:17:37.574 INFO:teuthology.orchestra.run.smithi084.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-16T09:17:37.574 INFO:teuthology.orchestra.run.smithi084.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-16T09:17:37.574 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-16T09:17:37.574 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction Summary 2024-09-16T09:17:37.574 INFO:teuthology.orchestra.run.smithi084.stdout:======================================================================================= 2024-09-16T09:17:37.574 INFO:teuthology.orchestra.run.smithi084.stdout:Install 12 Packages 2024-09-16T09:17:37.574 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-16T09:17:37.574 INFO:teuthology.orchestra.run.smithi084.stdout:Total download size: 38 M 2024-09-16T09:17:37.575 INFO:teuthology.orchestra.run.smithi084.stdout:Installed size: 212 M 2024-09-16T09:17:37.575 INFO:teuthology.orchestra.run.smithi084.stdout:Downloading Packages: 2024-09-16T09:17:38.427 INFO:teuthology.orchestra.run.smithi084.stdout:(1/12): ceph-mgr-diskprediction-local-19.3.0-49 17 MB/s | 7.4 MB 00:00 2024-09-16T09:17:38.452 INFO:teuthology.orchestra.run.smithi084.stdout:(2/12): libquadmath-11.5.0-2.el9.x86_64.rpm 405 kB/s | 191 kB 00:00 2024-09-16T09:17:38.603 INFO:teuthology.orchestra.run.smithi084.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 187 kB/s | 33 kB 00:00 2024-09-16T09:17:38.678 INFO:teuthology.orchestra.run.smithi084.stdout:(4/12): libgfortran-11.5.0-2.el9.x86_64.rpm 1.1 MB/s | 801 kB 00:00 2024-09-16T09:17:38.703 INFO:teuthology.orchestra.run.smithi084.stdout:(5/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 182 kB/s | 18 kB 00:00 2024-09-16T09:17:38.879 INFO:teuthology.orchestra.run.smithi084.stdout:(6/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 7.0 MB/s | 3.0 MB 00:00 2024-09-16T09:17:38.937 INFO:teuthology.orchestra.run.smithi084.stdout:(7/12): python3-devel-3.9.19-8.el9.x86_64.rpm 4.1 MB/s | 245 kB 00:00 2024-09-16T09:17:39.065 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-4917.g8293d73f.el9.noa 12/12 2024-09-16T09:17:39.065 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-4917.g8293d73f.el 1/12 2024-09-16T09:17:39.065 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-4917.g8293d73f.el9.noa 2/12 2024-09-16T09:17:39.065 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-4917.g8293d73f.el9 3/12 2024-09-16T09:17:39.065 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-09-16T09:17:39.066 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-09-16T09:17:39.066 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-09-16T09:17:39.066 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-09-16T09:17:39.066 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-09-16T09:17:39.066 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-09-16T09:17:39.066 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-09-16T09:17:39.066 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-09-16T09:17:39.079 INFO:teuthology.orchestra.run.smithi084.stdout:(8/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 36 MB/s | 5.1 MB 00:00 2024-09-16T09:17:39.146 INFO:teuthology.orchestra.run.smithi084.stdout:(9/12): openblas-openmp-0.3.26-2.el9.x86_64.rpm 11 MB/s | 4.9 MB 00:00 2024-09-16T09:17:39.171 INFO:teuthology.orchestra.run.smithi084.stdout:(10/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 2.7 MB/s | 250 kB 00:00 2024-09-16T09:17:39.305 INFO:teuthology.orchestra.run.smithi084.stdout:(11/12): openblas-0.3.26-2.el9.x86_64.rpm 61 kB/s | 38 kB 00:00 2024-09-16T09:17:39.376 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-09-16T09:17:39.376 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:17:39.376 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-09-16T09:17:39.376 INFO:teuthology.orchestra.run.smithi070.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-16T09:17:39.376 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-grafana-dashboards-2:19.3.0-4917.g8293d73f.el9.noarch 2024-09-16T09:17:39.376 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-dashboard-2:19.3.0-4917.g8293d73f.el9.noarch 2024-09-16T09:17:39.376 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-prometheus-alerts-2:19.3.0-4917.g8293d73f.el9.noarch 2024-09-16T09:17:39.376 INFO:teuthology.orchestra.run.smithi070.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-16T09:17:39.376 INFO:teuthology.orchestra.run.smithi070.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-16T09:17:39.377 INFO:teuthology.orchestra.run.smithi070.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-16T09:17:39.377 INFO:teuthology.orchestra.run.smithi070.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-16T09:17:39.377 INFO:teuthology.orchestra.run.smithi070.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-16T09:17:39.377 INFO:teuthology.orchestra.run.smithi070.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-16T09:17:39.377 INFO:teuthology.orchestra.run.smithi070.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-16T09:17:39.377 INFO:teuthology.orchestra.run.smithi070.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-16T09:17:39.377 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:17:39.377 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-16T09:17:39.531 INFO:teuthology.orchestra.run.smithi084.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 42 MB/s | 16 MB 00:00 2024-09-16T09:17:39.531 INFO:teuthology.orchestra.run.smithi084.stdout:-------------------------------------------------------------------------------- 2024-09-16T09:17:39.531 INFO:teuthology.orchestra.run.smithi084.stdout:Total 20 MB/s | 38 MB 00:01 2024-09-16T09:17:39.633 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-16T09:17:39.891 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction check 2024-09-16T09:17:39.917 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction check succeeded. 2024-09-16T09:17:39.918 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction test 2024-09-16T09:17:40.196 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:08 ago on Mon 16 Sep 2024 09:16:32 AM UTC. 2024-09-16T09:17:40.329 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-16T09:17:40.330 INFO:teuthology.orchestra.run.smithi070.stdout:======================================================================================= 2024-09-16T09:17:40.330 INFO:teuthology.orchestra.run.smithi070.stdout: Package Arch Version Repository Size 2024-09-16T09:17:40.330 INFO:teuthology.orchestra.run.smithi070.stdout:======================================================================================= 2024-09-16T09:17:40.330 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-09-16T09:17:40.330 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-4917.g8293d73f.el9 ceph-noarch 7.4 M 2024-09-16T09:17:40.330 INFO:teuthology.orchestra.run.smithi070.stdout:Installing dependencies: 2024-09-16T09:17:40.331 INFO:teuthology.orchestra.run.smithi070.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-16T09:17:40.331 INFO:teuthology.orchestra.run.smithi070.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-16T09:17:40.331 INFO:teuthology.orchestra.run.smithi070.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-16T09:17:40.331 INFO:teuthology.orchestra.run.smithi070.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-16T09:17:40.331 INFO:teuthology.orchestra.run.smithi070.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-16T09:17:40.331 INFO:teuthology.orchestra.run.smithi070.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-16T09:17:40.331 INFO:teuthology.orchestra.run.smithi070.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-16T09:17:40.331 INFO:teuthology.orchestra.run.smithi070.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-16T09:17:40.331 INFO:teuthology.orchestra.run.smithi070.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-16T09:17:40.331 INFO:teuthology.orchestra.run.smithi070.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-16T09:17:40.331 INFO:teuthology.orchestra.run.smithi070.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-16T09:17:40.331 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:17:40.331 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-09-16T09:17:40.331 INFO:teuthology.orchestra.run.smithi070.stdout:======================================================================================= 2024-09-16T09:17:40.332 INFO:teuthology.orchestra.run.smithi070.stdout:Install 12 Packages 2024-09-16T09:17:40.332 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:17:40.332 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 38 M 2024-09-16T09:17:40.332 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 212 M 2024-09-16T09:17:40.332 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-09-16T09:17:40.359 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction test succeeded. 2024-09-16T09:17:40.360 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction 2024-09-16T09:17:40.736 INFO:teuthology.orchestra.run.smithi070.stdout:(1/12): libgfortran-11.5.0-2.el9.x86_64.rpm 4.7 MB/s | 801 kB 00:00 2024-09-16T09:17:40.761 INFO:teuthology.orchestra.run.smithi070.stdout:(2/12): libquadmath-11.5.0-2.el9.x86_64.rpm 993 kB/s | 191 kB 00:00 2024-09-16T09:17:41.036 INFO:teuthology.orchestra.run.smithi070.stdout:(3/12): ceph-mgr-diskprediction-local-19.3.0-49 16 MB/s | 7.4 MB 00:00 2024-09-16T09:17:41.061 INFO:teuthology.orchestra.run.smithi070.stdout:(4/12): flexiblas-3.0.4-8.el9.x86_64.rpm 100 kB/s | 33 kB 00:00 2024-09-16T09:17:41.127 INFO:teuthology.orchestra.run.smithi084.stdout: Preparing : 1/1 2024-09-16T09:17:41.199 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-16T09:17:41.239 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-16T09:17:41.262 INFO:teuthology.orchestra.run.smithi070.stdout:(5/12): openblas-0.3.26-2.el9.x86_64.rpm 192 kB/s | 38 kB 00:00 2024-09-16T09:17:41.320 INFO:teuthology.orchestra.run.smithi070.stdout:(6/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 64 kB/s | 18 kB 00:00 2024-09-16T09:17:41.343 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-16T09:17:41.379 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-16T09:17:41.565 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-16T09:17:41.595 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-16T09:17:41.677 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-16T09:17:41.787 INFO:teuthology.orchestra.run.smithi070.stdout:(7/12): python3-devel-3.9.19-8.el9.x86_64.rpm 525 kB/s | 245 kB 00:00 2024-09-16T09:17:41.896 INFO:teuthology.orchestra.run.smithi070.stdout:(8/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 2.6 MB/s | 3.0 MB 00:01 2024-09-16T09:17:42.013 INFO:teuthology.orchestra.run.smithi070.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.1 MB/s | 250 kB 00:00 2024-09-16T09:17:42.288 INFO:teuthology.orchestra.run.smithi070.stdout:(10/12): openblas-openmp-0.3.26-2.el9.x86_64.rp 4.8 MB/s | 4.9 MB 00:01 2024-09-16T09:17:42.331 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-16T09:17:42.449 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-16T09:17:42.647 INFO:teuthology.orchestra.run.smithi070.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 5.9 MB/s | 5.1 MB 00:00 2024-09-16T09:17:43.699 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-16T09:17:44.246 INFO:teuthology.orchestra.run.smithi070.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 7.3 MB/s | 16 MB 00:02 2024-09-16T09:17:44.247 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-09-16T09:17:44.247 INFO:teuthology.orchestra.run.smithi070.stdout:Total 9.8 MB/s | 38 MB 00:03 2024-09-16T09:17:44.604 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-09-16T09:17:44.630 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-09-16T09:17:44.630 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-09-16T09:17:44.740 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-16T09:17:44.768 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-4917.g8293d 12/12 2024-09-16T09:17:45.100 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-09-16T09:17:45.100 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-09-16T09:17:45.861 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-09-16T09:17:45.942 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-16T09:17:45.992 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-16T09:17:46.094 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-16T09:17:46.129 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-16T09:17:46.313 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-16T09:17:46.345 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-16T09:17:46.424 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-16T09:17:47.083 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-16T09:17:47.215 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-16T09:17:47.426 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-4917.g8293d 12/12 2024-09-16T09:17:47.427 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-4917.g8293d 1/12 2024-09-16T09:17:47.427 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-16T09:17:47.427 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-16T09:17:47.427 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-16T09:17:47.427 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-16T09:17:47.427 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-16T09:17:47.427 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-16T09:17:47.427 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-16T09:17:47.427 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-16T09:17:47.427 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-16T09:17:47.427 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-16T09:17:47.841 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-16T09:17:47.842 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-16T09:17:47.842 INFO:teuthology.orchestra.run.smithi084.stdout:Installed: 2024-09-16T09:17:47.842 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-diskprediction-local-2:19.3.0-4917.g8293d73f.el9.noarch 2024-09-16T09:17:47.842 INFO:teuthology.orchestra.run.smithi084.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-16T09:17:47.842 INFO:teuthology.orchestra.run.smithi084.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-16T09:17:47.842 INFO:teuthology.orchestra.run.smithi084.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-16T09:17:47.842 INFO:teuthology.orchestra.run.smithi084.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-16T09:17:47.842 INFO:teuthology.orchestra.run.smithi084.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-16T09:17:47.842 INFO:teuthology.orchestra.run.smithi084.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-16T09:17:47.842 INFO:teuthology.orchestra.run.smithi084.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-16T09:17:47.843 INFO:teuthology.orchestra.run.smithi084.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-16T09:17:47.843 INFO:teuthology.orchestra.run.smithi084.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-16T09:17:47.843 INFO:teuthology.orchestra.run.smithi084.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-16T09:17:47.843 INFO:teuthology.orchestra.run.smithi084.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-16T09:17:47.843 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-16T09:17:47.843 INFO:teuthology.orchestra.run.smithi084.stdout:Complete! 2024-09-16T09:17:48.211 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum -y install ceph-mgr-rook 2024-09-16T09:17:48.438 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-16T09:17:48.791 INFO:teuthology.orchestra.run.smithi084.stdout:Last metadata expiration check: 0:01:19 ago on Mon 16 Sep 2024 09:16:29 AM UTC. 2024-09-16T09:17:48.927 INFO:teuthology.orchestra.run.smithi084.stdout:Dependencies resolved. 2024-09-16T09:17:48.928 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-09-16T09:17:48.928 INFO:teuthology.orchestra.run.smithi084.stdout: Package Arch Version Repository Size 2024-09-16T09:17:48.929 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-09-16T09:17:48.929 INFO:teuthology.orchestra.run.smithi084.stdout:Installing: 2024-09-16T09:17:48.929 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-rook noarch 2:19.3.0-4917.g8293d73f.el9 ceph-noarch 49 k 2024-09-16T09:17:48.929 INFO:teuthology.orchestra.run.smithi084.stdout:Installing dependencies: 2024-09-16T09:17:48.929 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-16T09:17:48.929 INFO:teuthology.orchestra.run.smithi084.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-16T09:17:48.929 INFO:teuthology.orchestra.run.smithi084.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-16T09:17:48.929 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-16T09:17:48.929 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-16T09:17:48.929 INFO:teuthology.orchestra.run.smithi084.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-16T09:17:48.930 INFO:teuthology.orchestra.run.smithi084.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-16T09:17:48.930 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-16T09:17:48.930 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-16T09:17:48.930 INFO:teuthology.orchestra.run.smithi084.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-16T09:17:48.930 INFO:teuthology.orchestra.run.smithi084.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-16T09:17:48.930 INFO:teuthology.orchestra.run.smithi084.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-16T09:17:48.930 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-16T09:17:48.930 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction Summary 2024-09-16T09:17:48.930 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-09-16T09:17:48.930 INFO:teuthology.orchestra.run.smithi084.stdout:Install 13 Packages 2024-09-16T09:17:48.930 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-16T09:17:48.931 INFO:teuthology.orchestra.run.smithi084.stdout:Total download size: 2.2 M 2024-09-16T09:17:48.931 INFO:teuthology.orchestra.run.smithi084.stdout:Installed size: 26 M 2024-09-16T09:17:48.931 INFO:teuthology.orchestra.run.smithi084.stdout:Downloading Packages: 2024-09-16T09:17:49.101 INFO:teuthology.orchestra.run.smithi084.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 308 kB/s | 26 kB 00:00 2024-09-16T09:17:49.126 INFO:teuthology.orchestra.run.smithi084.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 177 kB/s | 19 kB 00:00 2024-09-16T09:17:49.201 INFO:teuthology.orchestra.run.smithi084.stdout:(3/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.1 MB/s | 159 kB 00:00 2024-09-16T09:17:49.235 INFO:teuthology.orchestra.run.smithi084.stdout:(4/13): ceph-mgr-rook-19.3.0-4917.g8293d73f.el9 227 kB/s | 49 kB 00:00 2024-09-16T09:17:49.283 INFO:teuthology.orchestra.run.smithi084.stdout:(5/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.4 MB/s | 222 kB 00:00 2024-09-16T09:17:49.311 INFO:teuthology.orchestra.run.smithi084.stdout:(6/13): python3-pyasn1-modules-0.4.8-6.el9.noar 2.5 MB/s | 279 kB 00:00 2024-09-16T09:17:49.353 INFO:teuthology.orchestra.run.smithi084.stdout:(7/13): python3-requests-oauthlib-1.3.0-12.el9. 458 kB/s | 54 kB 00:00 2024-09-16T09:17:49.378 INFO:teuthology.orchestra.run.smithi084.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 338 kB/s | 32 kB 00:00 2024-09-16T09:17:49.411 INFO:teuthology.orchestra.run.smithi084.stdout:(9/13): python3-certifi-2023.05.07-4.el9.noarch 140 kB/s | 14 kB 00:00 2024-09-16T09:17:49.445 INFO:teuthology.orchestra.run.smithi084.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 1.8 MB/s | 59 kB 00:00 2024-09-16T09:17:49.471 INFO:teuthology.orchestra.run.smithi084.stdout:(11/13): python3-websocket-client-1.2.3-2.el9.n 3.5 MB/s | 90 kB 00:00 2024-09-16T09:17:49.472 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-16T09:17:49.503 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-4917.g8293d 12/12 2024-09-16T09:17:49.504 INFO:teuthology.orchestra.run.smithi084.stdout:(12/13): python3-google-auth-2.34.0-1.el9.noarc 1.4 MB/s | 220 kB 00:00 2024-09-16T09:17:49.538 INFO:teuthology.orchestra.run.smithi084.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 6.7 MB/s | 1.0 MB 00:00 2024-09-16T09:17:49.540 INFO:teuthology.orchestra.run.smithi084.stdout:-------------------------------------------------------------------------------- 2024-09-16T09:17:49.540 INFO:teuthology.orchestra.run.smithi084.stdout:Total 3.6 MB/s | 2.2 MB 00:00 2024-09-16T09:17:49.603 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction check 2024-09-16T09:17:49.618 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction check succeeded. 2024-09-16T09:17:49.618 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction test 2024-09-16T09:17:49.706 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction test succeeded. 2024-09-16T09:17:49.707 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction 2024-09-16T09:17:50.218 INFO:teuthology.orchestra.run.smithi084.stdout: Preparing : 1/1 2024-09-16T09:17:50.385 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-16T09:17:50.448 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-16T09:17:50.502 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-16T09:17:50.539 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-16T09:17:50.581 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-16T09:17:50.682 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-16T09:17:50.793 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-16T09:17:50.840 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-16T09:17:51.566 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-16T09:17:51.613 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-16T09:17:51.647 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-16T09:17:51.695 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-16T09:17:51.725 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-mgr-rook-2:19.3.0-4917.g8293d73f.el9.noarch 13/13 2024-09-16T09:17:52.235 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-4917.g8293d 12/12 2024-09-16T09:17:52.235 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-4917.g8293d 1/12 2024-09-16T09:17:52.235 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-16T09:17:52.236 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-16T09:17:52.236 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-16T09:17:52.236 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-16T09:17:52.236 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-16T09:17:52.236 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-16T09:17:52.236 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-16T09:17:52.236 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-16T09:17:52.236 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-16T09:17:52.236 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-16T09:17:52.527 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-4917.g8293d73f.el9.noarch 13/13 2024-09-16T09:17:52.528 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-mgr-rook-2:19.3.0-4917.g8293d73f.el9.noarch 1/13 2024-09-16T09:17:52.528 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-16T09:17:52.528 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-16T09:17:52.528 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-16T09:17:52.528 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-16T09:17:52.528 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-16T09:17:52.528 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-16T09:17:52.528 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-16T09:17:52.528 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-16T09:17:52.528 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-16T09:17:52.528 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-16T09:17:52.528 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-16T09:17:52.679 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-16T09:17:52.679 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:17:52.679 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-09-16T09:17:52.679 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-diskprediction-local-2:19.3.0-4917.g8293d73f.el9.noarch 2024-09-16T09:17:52.679 INFO:teuthology.orchestra.run.smithi070.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-16T09:17:52.679 INFO:teuthology.orchestra.run.smithi070.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-16T09:17:52.679 INFO:teuthology.orchestra.run.smithi070.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-16T09:17:52.680 INFO:teuthology.orchestra.run.smithi070.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-16T09:17:52.680 INFO:teuthology.orchestra.run.smithi070.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-16T09:17:52.680 INFO:teuthology.orchestra.run.smithi070.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-16T09:17:52.680 INFO:teuthology.orchestra.run.smithi070.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-16T09:17:52.680 INFO:teuthology.orchestra.run.smithi070.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-16T09:17:52.680 INFO:teuthology.orchestra.run.smithi070.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-16T09:17:52.680 INFO:teuthology.orchestra.run.smithi070.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-16T09:17:52.680 INFO:teuthology.orchestra.run.smithi070.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-16T09:17:52.680 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:17:52.680 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-16T09:17:52.898 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-16T09:17:52.899 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-16T09:17:52.899 INFO:teuthology.orchestra.run.smithi084.stdout:Installed: 2024-09-16T09:17:52.899 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-rook-2:19.3.0-4917.g8293d73f.el9.noarch 2024-09-16T09:17:52.899 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-16T09:17:52.899 INFO:teuthology.orchestra.run.smithi084.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-16T09:17:52.899 INFO:teuthology.orchestra.run.smithi084.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-16T09:17:52.899 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-16T09:17:52.899 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-16T09:17:52.899 INFO:teuthology.orchestra.run.smithi084.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-16T09:17:52.899 INFO:teuthology.orchestra.run.smithi084.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-16T09:17:52.899 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-16T09:17:52.899 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-16T09:17:52.899 INFO:teuthology.orchestra.run.smithi084.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-16T09:17:52.899 INFO:teuthology.orchestra.run.smithi084.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-16T09:17:52.900 INFO:teuthology.orchestra.run.smithi084.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-16T09:17:52.900 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-16T09:17:52.900 INFO:teuthology.orchestra.run.smithi084.stdout:Complete! 2024-09-16T09:17:53.088 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install ceph-mgr-rook 2024-09-16T09:17:53.168 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum -y install ceph-mgr-cephadm 2024-09-16T09:17:53.664 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:21 ago on Mon 16 Sep 2024 09:16:32 AM UTC. 2024-09-16T09:17:53.738 INFO:teuthology.orchestra.run.smithi084.stdout:Last metadata expiration check: 0:01:24 ago on Mon 16 Sep 2024 09:16:29 AM UTC. 2024-09-16T09:17:53.798 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-16T09:17:53.799 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-16T09:17:53.799 INFO:teuthology.orchestra.run.smithi070.stdout: Package Arch Version Repository Size 2024-09-16T09:17:53.799 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-16T09:17:53.800 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-09-16T09:17:53.800 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-rook noarch 2:19.3.0-4917.g8293d73f.el9 ceph-noarch 49 k 2024-09-16T09:17:53.800 INFO:teuthology.orchestra.run.smithi070.stdout:Installing dependencies: 2024-09-16T09:17:53.800 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-16T09:17:53.800 INFO:teuthology.orchestra.run.smithi070.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-16T09:17:53.800 INFO:teuthology.orchestra.run.smithi070.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-16T09:17:53.800 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-16T09:17:53.800 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-16T09:17:53.800 INFO:teuthology.orchestra.run.smithi070.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-16T09:17:53.800 INFO:teuthology.orchestra.run.smithi070.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-16T09:17:53.801 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-16T09:17:53.801 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-16T09:17:53.801 INFO:teuthology.orchestra.run.smithi070.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-16T09:17:53.801 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-16T09:17:53.801 INFO:teuthology.orchestra.run.smithi070.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-16T09:17:53.801 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:17:53.801 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-09-16T09:17:53.801 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-16T09:17:53.802 INFO:teuthology.orchestra.run.smithi070.stdout:Install 13 Packages 2024-09-16T09:17:53.802 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:17:53.802 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 2.2 M 2024-09-16T09:17:53.802 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 26 M 2024-09-16T09:17:53.802 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-09-16T09:17:53.874 INFO:teuthology.orchestra.run.smithi084.stdout:Dependencies resolved. 2024-09-16T09:17:53.875 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-09-16T09:17:53.875 INFO:teuthology.orchestra.run.smithi084.stdout: Package Arch Version Repository Size 2024-09-16T09:17:53.875 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-09-16T09:17:53.875 INFO:teuthology.orchestra.run.smithi084.stdout:Installing: 2024-09-16T09:17:53.875 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-cephadm noarch 2:19.3.0-4917.g8293d73f.el9 ceph-noarch 154 k 2024-09-16T09:17:53.875 INFO:teuthology.orchestra.run.smithi084.stdout:Installing dependencies: 2024-09-16T09:17:53.875 INFO:teuthology.orchestra.run.smithi084.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-16T09:17:53.875 INFO:teuthology.orchestra.run.smithi084.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-16T09:17:53.876 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-16T09:17:53.876 INFO:teuthology.orchestra.run.smithi084.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-16T09:17:53.876 INFO:teuthology.orchestra.run.smithi084.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-16T09:17:53.876 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-16T09:17:53.876 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction Summary 2024-09-16T09:17:53.876 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-09-16T09:17:53.876 INFO:teuthology.orchestra.run.smithi084.stdout:Install 6 Packages 2024-09-16T09:17:53.876 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-16T09:17:53.877 INFO:teuthology.orchestra.run.smithi084.stdout:Total download size: 7.0 M 2024-09-16T09:17:53.877 INFO:teuthology.orchestra.run.smithi084.stdout:Installed size: 33 M 2024-09-16T09:17:53.877 INFO:teuthology.orchestra.run.smithi084.stdout:Downloading Packages: 2024-09-16T09:17:54.106 INFO:teuthology.orchestra.run.smithi070.stdout:(1/13): python3-jsonpointer-2.0-4.el9.noarch.rp 176 kB/s | 19 kB 00:00 2024-09-16T09:17:54.131 INFO:teuthology.orchestra.run.smithi070.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 191 kB/s | 26 kB 00:00 2024-09-16T09:17:54.157 INFO:teuthology.orchestra.run.smithi070.stdout:(3/13): ceph-mgr-rook-19.3.0-4917.g8293d73f.el9 309 kB/s | 49 kB 00:00 2024-09-16T09:17:54.215 INFO:teuthology.orchestra.run.smithi070.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.0 MB/s | 222 kB 00:00 2024-09-16T09:17:54.239 INFO:teuthology.orchestra.run.smithi084.stdout:(1/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.7 MB/s | 248 kB 00:00 2024-09-16T09:17:54.249 INFO:teuthology.orchestra.run.smithi070.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.3 MB/s | 159 kB 00:00 2024-09-16T09:17:54.274 INFO:teuthology.orchestra.run.smithi070.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 912 kB/s | 54 kB 00:00 2024-09-16T09:17:54.297 INFO:teuthology.orchestra.run.smithi084.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 9.2 MB/s | 548 kB 00:00 2024-09-16T09:17:54.316 INFO:teuthology.orchestra.run.smithi070.stdout:(7/13): python3-cachetools-4.2.4-1.el9.noarch.r 481 kB/s | 32 kB 00:00 2024-09-16T09:17:54.322 INFO:teuthology.orchestra.run.smithi084.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 2.3 MB/s | 58 kB 00:00 2024-09-16T09:17:54.341 INFO:teuthology.orchestra.run.smithi070.stdout:(8/13): python3-certifi-2023.05.07-4.el9.noarch 211 kB/s | 14 kB 00:00 2024-09-16T09:17:54.348 INFO:teuthology.orchestra.run.smithi084.stdout:(4/6): ceph-mgr-cephadm-19.3.0-4917.g8293d73f.e 611 kB/s | 154 kB 00:00 2024-09-16T09:17:54.367 INFO:teuthology.orchestra.run.smithi070.stdout:(9/13): python3-google-auth-2.34.0-1.el9.noarch 4.4 MB/s | 220 kB 00:00 2024-09-16T09:17:54.373 INFO:teuthology.orchestra.run.smithi084.stdout:(5/6): python3-typing-extensions-4.12.2-2.el9.n 1.4 MB/s | 74 kB 00:00 2024-09-16T09:17:54.392 INFO:teuthology.orchestra.run.smithi070.stdout:(10/13): python3-pyasn1-modules-0.4.8-6.el9.noa 1.2 MB/s | 279 kB 00:00 2024-09-16T09:17:54.417 INFO:teuthology.orchestra.run.smithi070.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 1.1 MB/s | 59 kB 00:00 2024-09-16T09:17:54.459 INFO:teuthology.orchestra.run.smithi070.stdout:(12/13): python3-kubernetes-26.1.0-2.el9.noarch 8.7 MB/s | 1.0 MB 00:00 2024-09-16T09:17:54.484 INFO:teuthology.orchestra.run.smithi070.stdout:(13/13): python3-websocket-client-1.2.3-2.el9.n 974 kB/s | 90 kB 00:00 2024-09-16T09:17:54.487 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-09-16T09:17:54.487 INFO:teuthology.orchestra.run.smithi070.stdout:Total 3.2 MB/s | 2.2 MB 00:00 2024-09-16T09:17:54.550 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-09-16T09:17:54.565 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-09-16T09:17:54.565 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-09-16T09:17:54.652 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-09-16T09:17:54.652 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-09-16T09:17:55.169 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-09-16T09:17:55.321 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-16T09:17:55.380 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-16T09:17:55.432 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-16T09:17:55.470 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-16T09:17:55.511 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-16T09:17:55.606 INFO:teuthology.orchestra.run.smithi084.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 3.9 MB/s | 6.0 MB 00:01 2024-09-16T09:17:55.608 INFO:teuthology.orchestra.run.smithi084.stdout:-------------------------------------------------------------------------------- 2024-09-16T09:17:55.608 INFO:teuthology.orchestra.run.smithi084.stdout:Total 4.1 MB/s | 7.0 MB 00:01 2024-09-16T09:17:55.613 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-16T09:17:55.679 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction check 2024-09-16T09:17:55.690 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction check succeeded. 2024-09-16T09:17:55.690 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction test 2024-09-16T09:17:55.721 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-16T09:17:55.771 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-16T09:17:55.789 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction test succeeded. 2024-09-16T09:17:55.790 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction 2024-09-16T09:17:56.114 INFO:teuthology.orchestra.run.smithi084.stdout: Preparing : 1/1 2024-09-16T09:17:56.289 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-16T09:17:56.350 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-16T09:17:56.477 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-16T09:17:56.527 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-16T09:17:56.569 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-16T09:17:56.618 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-16T09:17:56.648 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mgr-rook-2:19.3.0-4917.g8293d73f.el9.noarch 13/13 2024-09-16T09:17:56.880 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-16T09:17:56.969 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-16T09:17:57.037 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-16T09:17:57.062 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-4917.g8293d73f.el9.noarch 6/6 2024-09-16T09:17:57.472 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-4917.g8293d73f.el9.noarch 13/13 2024-09-16T09:17:57.472 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mgr-rook-2:19.3.0-4917.g8293d73f.el9.noarch 1/13 2024-09-16T09:17:57.472 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-16T09:17:57.472 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-16T09:17:57.472 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-16T09:17:57.472 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-16T09:17:57.473 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-16T09:17:57.473 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-16T09:17:57.473 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-16T09:17:57.473 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-16T09:17:57.473 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-16T09:17:57.473 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-16T09:17:57.473 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-16T09:17:57.815 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-16T09:17:57.815 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:17:57.815 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-09-16T09:17:57.815 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-rook-2:19.3.0-4917.g8293d73f.el9.noarch 2024-09-16T09:17:57.815 INFO:teuthology.orchestra.run.smithi070.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-16T09:17:57.815 INFO:teuthology.orchestra.run.smithi070.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-16T09:17:57.815 INFO:teuthology.orchestra.run.smithi070.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-16T09:17:57.815 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-16T09:17:57.815 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-16T09:17:57.815 INFO:teuthology.orchestra.run.smithi070.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-16T09:17:57.815 INFO:teuthology.orchestra.run.smithi070.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-16T09:17:57.816 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-16T09:17:57.816 INFO:teuthology.orchestra.run.smithi070.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-16T09:17:57.816 INFO:teuthology.orchestra.run.smithi070.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-16T09:17:57.816 INFO:teuthology.orchestra.run.smithi070.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-16T09:17:57.816 INFO:teuthology.orchestra.run.smithi070.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-16T09:17:57.816 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:17:57.816 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-16T09:17:57.894 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-4917.g8293d73f.el9.noarch 6/6 2024-09-16T09:17:57.894 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-4917.g8293d73f.el9.noarch 1/6 2024-09-16T09:17:57.894 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-16T09:17:57.894 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-16T09:17:57.894 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-16T09:17:57.894 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-16T09:17:58.092 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install ceph-mgr-cephadm 2024-09-16T09:17:58.171 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-16T09:17:58.171 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-16T09:17:58.171 INFO:teuthology.orchestra.run.smithi084.stdout:Installed: 2024-09-16T09:17:58.171 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-cephadm-2:19.3.0-4917.g8293d73f.el9.noarch 2024-09-16T09:17:58.171 INFO:teuthology.orchestra.run.smithi084.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-16T09:17:58.171 INFO:teuthology.orchestra.run.smithi084.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-16T09:17:58.171 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-16T09:17:58.171 INFO:teuthology.orchestra.run.smithi084.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-16T09:17:58.172 INFO:teuthology.orchestra.run.smithi084.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-16T09:17:58.172 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-16T09:17:58.172 INFO:teuthology.orchestra.run.smithi084.stdout:Complete! 2024-09-16T09:17:58.405 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum -y install ceph-fuse 2024-09-16T09:17:58.654 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:26 ago on Mon 16 Sep 2024 09:16:32 AM UTC. 2024-09-16T09:17:58.789 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-16T09:17:58.790 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-16T09:17:58.790 INFO:teuthology.orchestra.run.smithi070.stdout: Package Arch Version Repository Size 2024-09-16T09:17:58.790 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-16T09:17:58.790 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-09-16T09:17:58.790 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-cephadm noarch 2:19.3.0-4917.g8293d73f.el9 ceph-noarch 154 k 2024-09-16T09:17:58.790 INFO:teuthology.orchestra.run.smithi070.stdout:Installing dependencies: 2024-09-16T09:17:58.790 INFO:teuthology.orchestra.run.smithi070.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-16T09:17:58.790 INFO:teuthology.orchestra.run.smithi070.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-16T09:17:58.791 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-16T09:17:58.791 INFO:teuthology.orchestra.run.smithi070.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-16T09:17:58.791 INFO:teuthology.orchestra.run.smithi070.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-16T09:17:58.791 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:17:58.791 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-09-16T09:17:58.791 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-16T09:17:58.791 INFO:teuthology.orchestra.run.smithi070.stdout:Install 6 Packages 2024-09-16T09:17:58.791 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:17:58.792 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 7.0 M 2024-09-16T09:17:58.792 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 33 M 2024-09-16T09:17:58.792 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-09-16T09:17:58.971 INFO:teuthology.orchestra.run.smithi084.stdout:Last metadata expiration check: 0:01:29 ago on Mon 16 Sep 2024 09:16:29 AM UTC. 2024-09-16T09:17:59.105 INFO:teuthology.orchestra.run.smithi084.stdout:Dependencies resolved. 2024-09-16T09:17:59.105 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-09-16T09:17:59.106 INFO:teuthology.orchestra.run.smithi084.stdout: Package Arch Version Repository Size 2024-09-16T09:17:59.106 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-09-16T09:17:59.106 INFO:teuthology.orchestra.run.smithi084.stdout:Installing: 2024-09-16T09:17:59.106 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-fuse x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 900 k 2024-09-16T09:17:59.106 INFO:teuthology.orchestra.run.smithi084.stdout:Installing dependencies: 2024-09-16T09:17:59.106 INFO:teuthology.orchestra.run.smithi084.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-16T09:17:59.106 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-16T09:17:59.106 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction Summary 2024-09-16T09:17:59.106 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-09-16T09:17:59.106 INFO:teuthology.orchestra.run.smithi084.stdout:Install 2 Packages 2024-09-16T09:17:59.107 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-16T09:17:59.107 INFO:teuthology.orchestra.run.smithi084.stdout:Total download size: 979 k 2024-09-16T09:17:59.107 INFO:teuthology.orchestra.run.smithi084.stdout:Installed size: 2.9 M 2024-09-16T09:17:59.107 INFO:teuthology.orchestra.run.smithi084.stdout:Downloading Packages: 2024-09-16T09:17:59.491 INFO:teuthology.orchestra.run.smithi084.stdout:(1/2): ceph-fuse-19.3.0-4917.g8293d73f.el9.x86_ 3.1 MB/s | 900 kB 00:00 2024-09-16T09:17:59.537 INFO:teuthology.orchestra.run.smithi070.stdout:(1/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.6 MB/s | 248 kB 00:00 2024-09-16T09:17:59.637 INFO:teuthology.orchestra.run.smithi070.stdout:(2/6): python3-babel-2.9.1-2.el9.noarch.rpm 24 MB/s | 6.0 MB 00:00 2024-09-16T09:17:59.663 INFO:teuthology.orchestra.run.smithi070.stdout:(3/6): ceph-mgr-cephadm-19.3.0-4917.g8293d73f.e 565 kB/s | 154 kB 00:00 2024-09-16T09:17:59.666 INFO:teuthology.orchestra.run.smithi084.stdout:(2/2): fuse-2.9.9-16.el9.x86_64.rpm 173 kB/s | 80 kB 00:00 2024-09-16T09:17:59.667 INFO:teuthology.orchestra.run.smithi084.stdout:-------------------------------------------------------------------------------- 2024-09-16T09:17:59.667 INFO:teuthology.orchestra.run.smithi084.stdout:Total 1.7 MB/s | 979 kB 00:00 2024-09-16T09:17:59.679 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction check 2024-09-16T09:17:59.701 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction check succeeded. 2024-09-16T09:17:59.701 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction test 2024-09-16T09:17:59.713 INFO:teuthology.orchestra.run.smithi070.stdout:(4/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 3.0 MB/s | 548 kB 00:00 2024-09-16T09:17:59.747 INFO:teuthology.orchestra.run.smithi070.stdout:(5/6): python3-natsort-7.1.1-5.el9.noarch.rpm 529 kB/s | 58 kB 00:00 2024-09-16T09:17:59.780 INFO:teuthology.orchestra.run.smithi070.stdout:(6/6): python3-typing-extensions-4.12.2-2.el9.n 673 kB/s | 74 kB 00:00 2024-09-16T09:17:59.783 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-09-16T09:17:59.783 INFO:teuthology.orchestra.run.smithi070.stdout:Total 7.1 MB/s | 7.0 MB 00:00 2024-09-16T09:17:59.783 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction test succeeded. 2024-09-16T09:17:59.783 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction 2024-09-16T09:17:59.853 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-09-16T09:17:59.865 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-09-16T09:17:59.865 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-09-16T09:17:59.964 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-09-16T09:17:59.964 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-09-16T09:17:59.996 INFO:teuthology.orchestra.run.smithi084.stdout: Preparing : 1/1 2024-09-16T09:18:00.135 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-16T09:18:00.207 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-fuse-2:19.3.0-4917.g8293d73f.el9.x86_64 2/2 2024-09-16T09:18:00.273 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-09-16T09:18:00.437 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-16T09:18:00.490 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-16T09:18:00.910 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-fuse-2:19.3.0-4917.g8293d73f.el9.x86_64 2/2 2024-09-16T09:18:00.910 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-fuse-2:19.3.0-4917.g8293d73f.el9.x86_64 1/2 2024-09-16T09:18:01.027 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-16T09:18:01.137 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-16T09:18:01.210 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-16T09:18:01.210 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-16T09:18:01.210 INFO:teuthology.orchestra.run.smithi084.stdout:Installed: 2024-09-16T09:18:01.210 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-fuse-2:19.3.0-4917.g8293d73f.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-16T09:18:01.210 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-16T09:18:01.210 INFO:teuthology.orchestra.run.smithi084.stdout:Complete! 2024-09-16T09:18:01.213 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-16T09:18:01.240 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-4917.g8293d73f.el9.noarch 6/6 2024-09-16T09:18:01.474 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum -y install ceph-volume 2024-09-16T09:18:02.053 INFO:teuthology.orchestra.run.smithi084.stdout:Last metadata expiration check: 0:01:33 ago on Mon 16 Sep 2024 09:16:29 AM UTC. 2024-09-16T09:18:02.068 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-4917.g8293d73f.el9.noarch 6/6 2024-09-16T09:18:02.069 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-4917.g8293d73f.el9.noarch 1/6 2024-09-16T09:18:02.069 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-16T09:18:02.069 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-16T09:18:02.069 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-16T09:18:02.069 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-16T09:18:02.192 INFO:teuthology.orchestra.run.smithi084.stdout:Dependencies resolved. 2024-09-16T09:18:02.193 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-09-16T09:18:02.193 INFO:teuthology.orchestra.run.smithi084.stdout: Package Arch Version Repository Size 2024-09-16T09:18:02.193 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-09-16T09:18:02.193 INFO:teuthology.orchestra.run.smithi084.stdout:Installing: 2024-09-16T09:18:02.193 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-volume noarch 2:19.3.0-4917.g8293d73f.el9 ceph-noarch 285 k 2024-09-16T09:18:02.193 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-16T09:18:02.193 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction Summary 2024-09-16T09:18:02.193 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-09-16T09:18:02.193 INFO:teuthology.orchestra.run.smithi084.stdout:Install 1 Package 2024-09-16T09:18:02.193 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-16T09:18:02.194 INFO:teuthology.orchestra.run.smithi084.stdout:Total download size: 285 k 2024-09-16T09:18:02.194 INFO:teuthology.orchestra.run.smithi084.stdout:Installed size: 1.3 M 2024-09-16T09:18:02.194 INFO:teuthology.orchestra.run.smithi084.stdout:Downloading Packages: 2024-09-16T09:18:02.347 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-16T09:18:02.347 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:18:02.347 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-09-16T09:18:02.347 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-mgr-cephadm-2:19.3.0-4917.g8293d73f.el9.noarch 2024-09-16T09:18:02.347 INFO:teuthology.orchestra.run.smithi070.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-16T09:18:02.348 INFO:teuthology.orchestra.run.smithi070.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-16T09:18:02.348 INFO:teuthology.orchestra.run.smithi070.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-16T09:18:02.348 INFO:teuthology.orchestra.run.smithi070.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-16T09:18:02.348 INFO:teuthology.orchestra.run.smithi070.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-16T09:18:02.348 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:18:02.348 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-16T09:18:02.404 INFO:teuthology.orchestra.run.smithi084.stdout:ceph-volume-19.3.0-4917.g8293d73f.el9.noarch.rp 1.3 MB/s | 285 kB 00:00 2024-09-16T09:18:02.404 INFO:teuthology.orchestra.run.smithi084.stdout:-------------------------------------------------------------------------------- 2024-09-16T09:18:02.405 INFO:teuthology.orchestra.run.smithi084.stdout:Total 1.3 MB/s | 285 kB 00:00 2024-09-16T09:18:02.405 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction check 2024-09-16T09:18:02.413 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction check succeeded. 2024-09-16T09:18:02.414 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction test 2024-09-16T09:18:02.440 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction test succeeded. 2024-09-16T09:18:02.441 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction 2024-09-16T09:18:02.634 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install ceph-fuse 2024-09-16T09:18:02.654 INFO:teuthology.orchestra.run.smithi084.stdout: Preparing : 1/1 2024-09-16T09:18:02.723 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-volume-2:19.3.0-4917.g8293d73f.el9.noarch 1/1 2024-09-16T09:18:02.739 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-volume-2:19.3.0-4917.g8293d73f.el9.noarch 1/1 2024-09-16T09:18:02.739 INFO:teuthology.orchestra.run.smithi084.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T09:18:02.739 INFO:teuthology.orchestra.run.smithi084.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-16T09:18:02.739 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-16T09:18:03.197 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:31 ago on Mon 16 Sep 2024 09:16:32 AM UTC. 2024-09-16T09:18:03.332 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-16T09:18:03.332 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-16T09:18:03.332 INFO:teuthology.orchestra.run.smithi070.stdout: Package Arch Version Repository Size 2024-09-16T09:18:03.332 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-16T09:18:03.332 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-09-16T09:18:03.332 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-fuse x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 900 k 2024-09-16T09:18:03.333 INFO:teuthology.orchestra.run.smithi070.stdout:Installing dependencies: 2024-09-16T09:18:03.333 INFO:teuthology.orchestra.run.smithi070.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-16T09:18:03.333 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:18:03.333 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-09-16T09:18:03.333 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-16T09:18:03.333 INFO:teuthology.orchestra.run.smithi070.stdout:Install 2 Packages 2024-09-16T09:18:03.333 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:18:03.334 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 979 k 2024-09-16T09:18:03.334 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 2.9 M 2024-09-16T09:18:03.334 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-09-16T09:18:03.583 INFO:teuthology.orchestra.run.smithi070.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 550 kB/s | 80 kB 00:00 2024-09-16T09:18:03.716 INFO:teuthology.orchestra.run.smithi070.stdout:(2/2): ceph-fuse-19.3.0-4917.g8293d73f.el9.x86_ 3.2 MB/s | 900 kB 00:00 2024-09-16T09:18:03.717 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-09-16T09:18:03.717 INFO:teuthology.orchestra.run.smithi070.stdout:Total 2.5 MB/s | 979 kB 00:00 2024-09-16T09:18:03.728 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-09-16T09:18:03.749 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-09-16T09:18:03.749 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-09-16T09:18:03.771 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-volume-2:19.3.0-4917.g8293d73f.el9.noarch 1/1 2024-09-16T09:18:03.771 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-16T09:18:03.771 INFO:teuthology.orchestra.run.smithi084.stdout:Installed: 2024-09-16T09:18:03.771 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-volume-2:19.3.0-4917.g8293d73f.el9.noarch 2024-09-16T09:18:03.771 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-16T09:18:03.771 INFO:teuthology.orchestra.run.smithi084.stdout:Complete! 2024-09-16T09:18:03.828 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-09-16T09:18:03.829 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-09-16T09:18:04.025 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-09-16T09:18:04.047 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum -y install librados-devel 2024-09-16T09:18:04.168 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-16T09:18:04.240 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-fuse-2:19.3.0-4917.g8293d73f.el9.x86_64 2/2 2024-09-16T09:18:04.639 INFO:teuthology.orchestra.run.smithi084.stdout:Last metadata expiration check: 0:01:35 ago on Mon 16 Sep 2024 09:16:29 AM UTC. 2024-09-16T09:18:04.773 INFO:teuthology.orchestra.run.smithi084.stdout:Dependencies resolved. 2024-09-16T09:18:04.774 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-09-16T09:18:04.774 INFO:teuthology.orchestra.run.smithi084.stdout: Package Arch Version Repo Size 2024-09-16T09:18:04.774 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-09-16T09:18:04.774 INFO:teuthology.orchestra.run.smithi084.stdout:Installing: 2024-09-16T09:18:04.774 INFO:teuthology.orchestra.run.smithi084.stdout: librados-devel x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 126 k 2024-09-16T09:18:04.774 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-16T09:18:04.774 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction Summary 2024-09-16T09:18:04.774 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-09-16T09:18:04.774 INFO:teuthology.orchestra.run.smithi084.stdout:Install 1 Package 2024-09-16T09:18:04.775 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-16T09:18:04.775 INFO:teuthology.orchestra.run.smithi084.stdout:Total download size: 126 k 2024-09-16T09:18:04.775 INFO:teuthology.orchestra.run.smithi084.stdout:Installed size: 456 k 2024-09-16T09:18:04.775 INFO:teuthology.orchestra.run.smithi084.stdout:Downloading Packages: 2024-09-16T09:18:04.933 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-fuse-2:19.3.0-4917.g8293d73f.el9.x86_64 2/2 2024-09-16T09:18:04.934 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-fuse-2:19.3.0-4917.g8293d73f.el9.x86_64 1/2 2024-09-16T09:18:04.966 INFO:teuthology.orchestra.run.smithi084.stdout:librados-devel-19.3.0-4917.g8293d73f.el9.x86_64 662 kB/s | 126 kB 00:00 2024-09-16T09:18:04.967 INFO:teuthology.orchestra.run.smithi084.stdout:-------------------------------------------------------------------------------- 2024-09-16T09:18:04.967 INFO:teuthology.orchestra.run.smithi084.stdout:Total 657 kB/s | 126 kB 00:00 2024-09-16T09:18:04.967 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction check 2024-09-16T09:18:04.973 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction check succeeded. 2024-09-16T09:18:04.973 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction test 2024-09-16T09:18:05.037 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction test succeeded. 2024-09-16T09:18:05.038 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction 2024-09-16T09:18:05.189 INFO:teuthology.orchestra.run.smithi084.stdout: Preparing : 1/1 2024-09-16T09:18:05.205 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-16T09:18:05.206 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:18:05.206 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-09-16T09:18:05.206 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-fuse-2:19.3.0-4917.g8293d73f.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-16T09:18:05.206 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:18:05.206 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-16T09:18:05.299 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : librados-devel-2:19.3.0-4917.g8293d73f.el9.x86_64 1/1 2024-09-16T09:18:05.497 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install ceph-volume 2024-09-16T09:18:05.766 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: librados-devel-2:19.3.0-4917.g8293d73f.el9.x86_64 1/1 2024-09-16T09:18:06.039 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : librados-devel-2:19.3.0-4917.g8293d73f.el9.x86_64 1/1 2024-09-16T09:18:06.039 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-16T09:18:06.040 INFO:teuthology.orchestra.run.smithi084.stdout:Installed: 2024-09-16T09:18:06.040 INFO:teuthology.orchestra.run.smithi084.stdout: librados-devel-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T09:18:06.040 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-16T09:18:06.040 INFO:teuthology.orchestra.run.smithi084.stdout:Complete! 2024-09-16T09:18:06.079 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:34 ago on Mon 16 Sep 2024 09:16:32 AM UTC. 2024-09-16T09:18:06.219 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-16T09:18:06.219 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-16T09:18:06.219 INFO:teuthology.orchestra.run.smithi070.stdout: Package Arch Version Repository Size 2024-09-16T09:18:06.220 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-16T09:18:06.220 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-09-16T09:18:06.220 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-volume noarch 2:19.3.0-4917.g8293d73f.el9 ceph-noarch 285 k 2024-09-16T09:18:06.220 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:18:06.220 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-09-16T09:18:06.220 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-16T09:18:06.220 INFO:teuthology.orchestra.run.smithi070.stdout:Install 1 Package 2024-09-16T09:18:06.220 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:18:06.220 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 285 k 2024-09-16T09:18:06.220 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 1.3 M 2024-09-16T09:18:06.220 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-09-16T09:18:06.275 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum -y install libcephfs2 2024-09-16T09:18:06.463 INFO:teuthology.orchestra.run.smithi070.stdout:ceph-volume-19.3.0-4917.g8293d73f.el9.noarch.rp 1.2 MB/s | 285 kB 00:00 2024-09-16T09:18:06.463 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-09-16T09:18:06.464 INFO:teuthology.orchestra.run.smithi070.stdout:Total 1.1 MB/s | 285 kB 00:00 2024-09-16T09:18:06.464 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-09-16T09:18:06.473 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-09-16T09:18:06.473 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-09-16T09:18:06.504 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-09-16T09:18:06.505 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-09-16T09:18:06.725 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-09-16T09:18:06.790 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : ceph-volume-2:19.3.0-4917.g8293d73f.el9.noarch 1/1 2024-09-16T09:18:06.807 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: ceph-volume-2:19.3.0-4917.g8293d73f.el9.noarch 1/1 2024-09-16T09:18:06.807 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T09:18:06.807 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-16T09:18:06.807 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:18:06.860 INFO:teuthology.orchestra.run.smithi084.stdout:Last metadata expiration check: 0:01:37 ago on Mon 16 Sep 2024 09:16:29 AM UTC. 2024-09-16T09:18:06.953 INFO:teuthology.orchestra.run.smithi084.stdout:Package libcephfs2-2:19.3.0-4917.g8293d73f.el9.x86_64 is already installed. 2024-09-16T09:18:06.999 INFO:teuthology.orchestra.run.smithi084.stdout:Dependencies resolved. 2024-09-16T09:18:07.000 INFO:teuthology.orchestra.run.smithi084.stdout:Nothing to do. 2024-09-16T09:18:07.000 INFO:teuthology.orchestra.run.smithi084.stdout:Complete! 2024-09-16T09:18:07.068 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum -y install libcephfs-devel 2024-09-16T09:18:07.637 INFO:teuthology.orchestra.run.smithi084.stdout:Last metadata expiration check: 0:01:38 ago on Mon 16 Sep 2024 09:16:29 AM UTC. 2024-09-16T09:18:07.770 INFO:teuthology.orchestra.run.smithi084.stdout:Dependencies resolved. 2024-09-16T09:18:07.771 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-09-16T09:18:07.771 INFO:teuthology.orchestra.run.smithi084.stdout: Package Arch Version Repo Size 2024-09-16T09:18:07.771 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-09-16T09:18:07.771 INFO:teuthology.orchestra.run.smithi084.stdout:Installing: 2024-09-16T09:18:07.771 INFO:teuthology.orchestra.run.smithi084.stdout: libcephfs-devel x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 32 k 2024-09-16T09:18:07.771 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-16T09:18:07.771 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction Summary 2024-09-16T09:18:07.771 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-09-16T09:18:07.771 INFO:teuthology.orchestra.run.smithi084.stdout:Install 1 Package 2024-09-16T09:18:07.772 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-16T09:18:07.772 INFO:teuthology.orchestra.run.smithi084.stdout:Total download size: 32 k 2024-09-16T09:18:07.772 INFO:teuthology.orchestra.run.smithi084.stdout:Installed size: 142 k 2024-09-16T09:18:07.772 INFO:teuthology.orchestra.run.smithi084.stdout:Downloading Packages: 2024-09-16T09:18:07.854 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : ceph-volume-2:19.3.0-4917.g8293d73f.el9.noarch 1/1 2024-09-16T09:18:07.854 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:18:07.854 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-09-16T09:18:07.854 INFO:teuthology.orchestra.run.smithi070.stdout: ceph-volume-2:19.3.0-4917.g8293d73f.el9.noarch 2024-09-16T09:18:07.854 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:18:07.854 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-16T09:18:07.936 INFO:teuthology.orchestra.run.smithi084.stdout:libcephfs-devel-19.3.0-4917.g8293d73f.el9.x86_6 193 kB/s | 32 kB 00:00 2024-09-16T09:18:07.937 INFO:teuthology.orchestra.run.smithi084.stdout:-------------------------------------------------------------------------------- 2024-09-16T09:18:07.937 INFO:teuthology.orchestra.run.smithi084.stdout:Total 192 kB/s | 32 kB 00:00 2024-09-16T09:18:07.937 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction check 2024-09-16T09:18:07.940 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction check succeeded. 2024-09-16T09:18:07.941 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction test 2024-09-16T09:18:07.967 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction test succeeded. 2024-09-16T09:18:07.967 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction 2024-09-16T09:18:08.059 INFO:teuthology.orchestra.run.smithi084.stdout: Preparing : 1/1 2024-09-16T09:18:08.160 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install librados-devel 2024-09-16T09:18:08.175 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libcephfs-devel-2:19.3.0-4917.g8293d73f.el9.x86_64 1/1 2024-09-16T09:18:08.550 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: libcephfs-devel-2:19.3.0-4917.g8293d73f.el9.x86_64 1/1 2024-09-16T09:18:08.743 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:36 ago on Mon 16 Sep 2024 09:16:32 AM UTC. 2024-09-16T09:18:08.786 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libcephfs-devel-2:19.3.0-4917.g8293d73f.el9.x86_64 1/1 2024-09-16T09:18:08.786 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-16T09:18:08.786 INFO:teuthology.orchestra.run.smithi084.stdout:Installed: 2024-09-16T09:18:08.786 INFO:teuthology.orchestra.run.smithi084.stdout: libcephfs-devel-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T09:18:08.786 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-16T09:18:08.786 INFO:teuthology.orchestra.run.smithi084.stdout:Complete! 2024-09-16T09:18:08.879 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-16T09:18:08.880 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-16T09:18:08.880 INFO:teuthology.orchestra.run.smithi070.stdout: Package Arch Version Repo Size 2024-09-16T09:18:08.880 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-16T09:18:08.880 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-09-16T09:18:08.880 INFO:teuthology.orchestra.run.smithi070.stdout: librados-devel x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 126 k 2024-09-16T09:18:08.880 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:18:08.880 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-09-16T09:18:08.880 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-16T09:18:08.880 INFO:teuthology.orchestra.run.smithi070.stdout:Install 1 Package 2024-09-16T09:18:08.880 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:18:08.881 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 126 k 2024-09-16T09:18:08.881 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 456 k 2024-09-16T09:18:08.881 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-09-16T09:18:08.973 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum -y install librados2 2024-09-16T09:18:09.102 INFO:teuthology.orchestra.run.smithi070.stdout:librados-devel-19.3.0-4917.g8293d73f.el9.x86_64 574 kB/s | 126 kB 00:00 2024-09-16T09:18:09.102 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-09-16T09:18:09.102 INFO:teuthology.orchestra.run.smithi070.stdout:Total 570 kB/s | 126 kB 00:00 2024-09-16T09:18:09.103 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-09-16T09:18:09.108 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-09-16T09:18:09.108 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-09-16T09:18:09.172 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-09-16T09:18:09.172 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-09-16T09:18:09.322 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-09-16T09:18:09.432 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : librados-devel-2:19.3.0-4917.g8293d73f.el9.x86_64 1/1 2024-09-16T09:18:09.531 INFO:teuthology.orchestra.run.smithi084.stdout:Last metadata expiration check: 0:01:40 ago on Mon 16 Sep 2024 09:16:29 AM UTC. 2024-09-16T09:18:09.621 INFO:teuthology.orchestra.run.smithi084.stdout:Package librados2-2:19.3.0-4917.g8293d73f.el9.x86_64 is already installed. 2024-09-16T09:18:09.664 INFO:teuthology.orchestra.run.smithi084.stdout:Dependencies resolved. 2024-09-16T09:18:09.665 INFO:teuthology.orchestra.run.smithi084.stdout:Nothing to do. 2024-09-16T09:18:09.665 INFO:teuthology.orchestra.run.smithi084.stdout:Complete! 2024-09-16T09:18:09.728 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum -y install librbd1 2024-09-16T09:18:09.870 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: librados-devel-2:19.3.0-4917.g8293d73f.el9.x86_64 1/1 2024-09-16T09:18:10.145 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : librados-devel-2:19.3.0-4917.g8293d73f.el9.x86_64 1/1 2024-09-16T09:18:10.145 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:18:10.145 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-09-16T09:18:10.145 INFO:teuthology.orchestra.run.smithi070.stdout: librados-devel-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T09:18:10.145 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:18:10.145 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-16T09:18:10.292 INFO:teuthology.orchestra.run.smithi084.stdout:Last metadata expiration check: 0:01:41 ago on Mon 16 Sep 2024 09:16:29 AM UTC. 2024-09-16T09:18:10.382 INFO:teuthology.orchestra.run.smithi084.stdout:Package librbd1-2:19.3.0-4917.g8293d73f.el9.x86_64 is already installed. 2024-09-16T09:18:10.425 INFO:teuthology.orchestra.run.smithi084.stdout:Dependencies resolved. 2024-09-16T09:18:10.426 INFO:teuthology.orchestra.run.smithi084.stdout:Nothing to do. 2024-09-16T09:18:10.426 INFO:teuthology.orchestra.run.smithi084.stdout:Complete! 2024-09-16T09:18:10.454 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install libcephfs2 2024-09-16T09:18:10.499 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum -y install python3-rados 2024-09-16T09:18:11.035 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:39 ago on Mon 16 Sep 2024 09:16:32 AM UTC. 2024-09-16T09:18:11.067 INFO:teuthology.orchestra.run.smithi084.stdout:Last metadata expiration check: 0:01:42 ago on Mon 16 Sep 2024 09:16:29 AM UTC. 2024-09-16T09:18:11.128 INFO:teuthology.orchestra.run.smithi070.stdout:Package libcephfs2-2:19.3.0-4917.g8293d73f.el9.x86_64 is already installed. 2024-09-16T09:18:11.159 INFO:teuthology.orchestra.run.smithi084.stdout:Package python3-rados-2:19.3.0-4917.g8293d73f.el9.x86_64 is already installed. 2024-09-16T09:18:11.172 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-16T09:18:11.173 INFO:teuthology.orchestra.run.smithi070.stdout:Nothing to do. 2024-09-16T09:18:11.174 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-16T09:18:11.202 INFO:teuthology.orchestra.run.smithi084.stdout:Dependencies resolved. 2024-09-16T09:18:11.203 INFO:teuthology.orchestra.run.smithi084.stdout:Nothing to do. 2024-09-16T09:18:11.203 INFO:teuthology.orchestra.run.smithi084.stdout:Complete! 2024-09-16T09:18:11.261 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install libcephfs-devel 2024-09-16T09:18:11.266 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum -y install python3-rgw 2024-09-16T09:18:11.827 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:39 ago on Mon 16 Sep 2024 09:16:32 AM UTC. 2024-09-16T09:18:11.829 INFO:teuthology.orchestra.run.smithi084.stdout:Last metadata expiration check: 0:01:42 ago on Mon 16 Sep 2024 09:16:29 AM UTC. 2024-09-16T09:18:11.921 INFO:teuthology.orchestra.run.smithi084.stdout:Package python3-rgw-2:19.3.0-4917.g8293d73f.el9.x86_64 is already installed. 2024-09-16T09:18:11.961 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-16T09:18:11.962 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-16T09:18:11.962 INFO:teuthology.orchestra.run.smithi070.stdout: Package Arch Version Repo Size 2024-09-16T09:18:11.962 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-16T09:18:11.962 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-09-16T09:18:11.962 INFO:teuthology.orchestra.run.smithi070.stdout: libcephfs-devel x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 32 k 2024-09-16T09:18:11.962 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:18:11.962 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-09-16T09:18:11.962 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-16T09:18:11.962 INFO:teuthology.orchestra.run.smithi070.stdout:Install 1 Package 2024-09-16T09:18:11.962 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:18:11.963 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 32 k 2024-09-16T09:18:11.963 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 142 k 2024-09-16T09:18:11.963 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-09-16T09:18:11.964 INFO:teuthology.orchestra.run.smithi084.stdout:Dependencies resolved. 2024-09-16T09:18:11.965 INFO:teuthology.orchestra.run.smithi084.stdout:Nothing to do. 2024-09-16T09:18:11.965 INFO:teuthology.orchestra.run.smithi084.stdout:Complete! 2024-09-16T09:18:12.033 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum -y install python3-cephfs 2024-09-16T09:18:12.122 INFO:teuthology.orchestra.run.smithi070.stdout:libcephfs-devel-19.3.0-4917.g8293d73f.el9.x86_6 198 kB/s | 32 kB 00:00 2024-09-16T09:18:12.123 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-09-16T09:18:12.123 INFO:teuthology.orchestra.run.smithi070.stdout:Total 197 kB/s | 32 kB 00:00 2024-09-16T09:18:12.123 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-09-16T09:18:12.127 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-09-16T09:18:12.127 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-09-16T09:18:12.144 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-09-16T09:18:12.145 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-09-16T09:18:12.236 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-09-16T09:18:12.352 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : libcephfs-devel-2:19.3.0-4917.g8293d73f.el9.x86_64 1/1 2024-09-16T09:18:12.600 INFO:teuthology.orchestra.run.smithi084.stdout:Last metadata expiration check: 0:01:43 ago on Mon 16 Sep 2024 09:16:29 AM UTC. 2024-09-16T09:18:12.690 INFO:teuthology.orchestra.run.smithi084.stdout:Package python3-cephfs-2:19.3.0-4917.g8293d73f.el9.x86_64 is already installed. 2024-09-16T09:18:12.726 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: libcephfs-devel-2:19.3.0-4917.g8293d73f.el9.x86_64 1/1 2024-09-16T09:18:12.733 INFO:teuthology.orchestra.run.smithi084.stdout:Dependencies resolved. 2024-09-16T09:18:12.734 INFO:teuthology.orchestra.run.smithi084.stdout:Nothing to do. 2024-09-16T09:18:12.734 INFO:teuthology.orchestra.run.smithi084.stdout:Complete! 2024-09-16T09:18:12.795 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum -y install python3-rbd 2024-09-16T09:18:12.959 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : libcephfs-devel-2:19.3.0-4917.g8293d73f.el9.x86_64 1/1 2024-09-16T09:18:12.960 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:18:12.960 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-09-16T09:18:12.960 INFO:teuthology.orchestra.run.smithi070.stdout: libcephfs-devel-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T09:18:12.960 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:18:12.960 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-16T09:18:13.178 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install librados2 2024-09-16T09:18:13.365 INFO:teuthology.orchestra.run.smithi084.stdout:Last metadata expiration check: 0:01:44 ago on Mon 16 Sep 2024 09:16:29 AM UTC. 2024-09-16T09:18:13.455 INFO:teuthology.orchestra.run.smithi084.stdout:Package python3-rbd-2:19.3.0-4917.g8293d73f.el9.x86_64 is already installed. 2024-09-16T09:18:13.498 INFO:teuthology.orchestra.run.smithi084.stdout:Dependencies resolved. 2024-09-16T09:18:13.499 INFO:teuthology.orchestra.run.smithi084.stdout:Nothing to do. 2024-09-16T09:18:13.500 INFO:teuthology.orchestra.run.smithi084.stdout:Complete! 2024-09-16T09:18:13.565 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum -y install rbd-fuse 2024-09-16T09:18:13.738 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:41 ago on Mon 16 Sep 2024 09:16:32 AM UTC. 2024-09-16T09:18:13.827 INFO:teuthology.orchestra.run.smithi070.stdout:Package librados2-2:19.3.0-4917.g8293d73f.el9.x86_64 is already installed. 2024-09-16T09:18:13.871 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-16T09:18:13.872 INFO:teuthology.orchestra.run.smithi070.stdout:Nothing to do. 2024-09-16T09:18:13.872 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-16T09:18:13.955 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install librbd1 2024-09-16T09:18:14.135 INFO:teuthology.orchestra.run.smithi084.stdout:Last metadata expiration check: 0:01:45 ago on Mon 16 Sep 2024 09:16:29 AM UTC. 2024-09-16T09:18:14.271 INFO:teuthology.orchestra.run.smithi084.stdout:Dependencies resolved. 2024-09-16T09:18:14.272 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-09-16T09:18:14.272 INFO:teuthology.orchestra.run.smithi084.stdout: Package Architecture Version Repository Size 2024-09-16T09:18:14.272 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-09-16T09:18:14.272 INFO:teuthology.orchestra.run.smithi084.stdout:Installing: 2024-09-16T09:18:14.272 INFO:teuthology.orchestra.run.smithi084.stdout: rbd-fuse x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 85 k 2024-09-16T09:18:14.272 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-16T09:18:14.273 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction Summary 2024-09-16T09:18:14.273 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-09-16T09:18:14.273 INFO:teuthology.orchestra.run.smithi084.stdout:Install 1 Package 2024-09-16T09:18:14.273 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-16T09:18:14.273 INFO:teuthology.orchestra.run.smithi084.stdout:Total download size: 85 k 2024-09-16T09:18:14.273 INFO:teuthology.orchestra.run.smithi084.stdout:Installed size: 231 k 2024-09-16T09:18:14.273 INFO:teuthology.orchestra.run.smithi084.stdout:Downloading Packages: 2024-09-16T09:18:14.464 INFO:teuthology.orchestra.run.smithi084.stdout:rbd-fuse-19.3.0-4917.g8293d73f.el9.x86_64.rpm 446 kB/s | 85 kB 00:00 2024-09-16T09:18:14.465 INFO:teuthology.orchestra.run.smithi084.stdout:-------------------------------------------------------------------------------- 2024-09-16T09:18:14.465 INFO:teuthology.orchestra.run.smithi084.stdout:Total 443 kB/s | 85 kB 00:00 2024-09-16T09:18:14.465 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction check 2024-09-16T09:18:14.472 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction check succeeded. 2024-09-16T09:18:14.472 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction test 2024-09-16T09:18:14.524 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:42 ago on Mon 16 Sep 2024 09:16:32 AM UTC. 2024-09-16T09:18:14.536 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction test succeeded. 2024-09-16T09:18:14.536 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction 2024-09-16T09:18:14.614 INFO:teuthology.orchestra.run.smithi070.stdout:Package librbd1-2:19.3.0-4917.g8293d73f.el9.x86_64 is already installed. 2024-09-16T09:18:14.658 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-16T09:18:14.659 INFO:teuthology.orchestra.run.smithi070.stdout:Nothing to do. 2024-09-16T09:18:14.659 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-16T09:18:14.705 INFO:teuthology.orchestra.run.smithi084.stdout: Preparing : 1/1 2024-09-16T09:18:14.742 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install python3-rados 2024-09-16T09:18:14.803 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : rbd-fuse-2:19.3.0-4917.g8293d73f.el9.x86_64 1/1 2024-09-16T09:18:15.206 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: rbd-fuse-2:19.3.0-4917.g8293d73f.el9.x86_64 1/1 2024-09-16T09:18:15.305 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:43 ago on Mon 16 Sep 2024 09:16:32 AM UTC. 2024-09-16T09:18:15.396 INFO:teuthology.orchestra.run.smithi070.stdout:Package python3-rados-2:19.3.0-4917.g8293d73f.el9.x86_64 is already installed. 2024-09-16T09:18:15.438 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-16T09:18:15.440 INFO:teuthology.orchestra.run.smithi070.stdout:Nothing to do. 2024-09-16T09:18:15.440 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-16T09:18:15.477 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : rbd-fuse-2:19.3.0-4917.g8293d73f.el9.x86_64 1/1 2024-09-16T09:18:15.477 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-16T09:18:15.477 INFO:teuthology.orchestra.run.smithi084.stdout:Installed: 2024-09-16T09:18:15.477 INFO:teuthology.orchestra.run.smithi084.stdout: rbd-fuse-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T09:18:15.477 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-16T09:18:15.478 INFO:teuthology.orchestra.run.smithi084.stdout:Complete! 2024-09-16T09:18:15.514 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install python3-rgw 2024-09-16T09:18:15.728 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum -y install rbd-mirror 2024-09-16T09:18:16.075 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:44 ago on Mon 16 Sep 2024 09:16:32 AM UTC. 2024-09-16T09:18:16.164 INFO:teuthology.orchestra.run.smithi070.stdout:Package python3-rgw-2:19.3.0-4917.g8293d73f.el9.x86_64 is already installed. 2024-09-16T09:18:16.207 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-16T09:18:16.208 INFO:teuthology.orchestra.run.smithi070.stdout:Nothing to do. 2024-09-16T09:18:16.208 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-16T09:18:16.297 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install python3-cephfs 2024-09-16T09:18:16.324 INFO:teuthology.orchestra.run.smithi084.stdout:Last metadata expiration check: 0:01:47 ago on Mon 16 Sep 2024 09:16:29 AM UTC. 2024-09-16T09:18:16.474 INFO:teuthology.orchestra.run.smithi084.stdout:Dependencies resolved. 2024-09-16T09:18:16.475 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-09-16T09:18:16.475 INFO:teuthology.orchestra.run.smithi084.stdout: Package Arch Version Repo Size 2024-09-16T09:18:16.475 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-09-16T09:18:16.475 INFO:teuthology.orchestra.run.smithi084.stdout:Installing: 2024-09-16T09:18:16.475 INFO:teuthology.orchestra.run.smithi084.stdout: rbd-mirror x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 3.2 M 2024-09-16T09:18:16.475 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-16T09:18:16.475 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction Summary 2024-09-16T09:18:16.476 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-09-16T09:18:16.476 INFO:teuthology.orchestra.run.smithi084.stdout:Install 1 Package 2024-09-16T09:18:16.476 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-16T09:18:16.476 INFO:teuthology.orchestra.run.smithi084.stdout:Total download size: 3.2 M 2024-09-16T09:18:16.476 INFO:teuthology.orchestra.run.smithi084.stdout:Installed size: 14 M 2024-09-16T09:18:16.476 INFO:teuthology.orchestra.run.smithi084.stdout:Downloading Packages: 2024-09-16T09:18:16.809 INFO:teuthology.orchestra.run.smithi084.stdout:rbd-mirror-19.3.0-4917.g8293d73f.el9.x86_64.rpm 9.6 MB/s | 3.2 MB 00:00 2024-09-16T09:18:16.810 INFO:teuthology.orchestra.run.smithi084.stdout:-------------------------------------------------------------------------------- 2024-09-16T09:18:16.810 INFO:teuthology.orchestra.run.smithi084.stdout:Total 9.5 MB/s | 3.2 MB 00:00 2024-09-16T09:18:16.810 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction check 2024-09-16T09:18:16.819 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction check succeeded. 2024-09-16T09:18:16.819 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction test 2024-09-16T09:18:16.862 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:44 ago on Mon 16 Sep 2024 09:16:32 AM UTC. 2024-09-16T09:18:16.902 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction test succeeded. 2024-09-16T09:18:16.902 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction 2024-09-16T09:18:16.953 INFO:teuthology.orchestra.run.smithi070.stdout:Package python3-cephfs-2:19.3.0-4917.g8293d73f.el9.x86_64 is already installed. 2024-09-16T09:18:16.996 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-16T09:18:16.997 INFO:teuthology.orchestra.run.smithi070.stdout:Nothing to do. 2024-09-16T09:18:16.997 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-16T09:18:17.075 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install python3-rbd 2024-09-16T09:18:17.348 INFO:teuthology.orchestra.run.smithi084.stdout: Preparing : 1/1 2024-09-16T09:18:17.410 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : rbd-mirror-2:19.3.0-4917.g8293d73f.el9.x86_64 1/1 2024-09-16T09:18:17.436 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: rbd-mirror-2:19.3.0-4917.g8293d73f.el9.x86_64 1/1 2024-09-16T09:18:17.436 INFO:teuthology.orchestra.run.smithi084.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T09:18:17.436 INFO:teuthology.orchestra.run.smithi084.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-16T09:18:17.437 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T09:18:17.437 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T09:18:17.437 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-16T09:18:17.645 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:45 ago on Mon 16 Sep 2024 09:16:32 AM UTC. 2024-09-16T09:18:17.738 INFO:teuthology.orchestra.run.smithi070.stdout:Package python3-rbd-2:19.3.0-4917.g8293d73f.el9.x86_64 is already installed. 2024-09-16T09:18:17.781 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-16T09:18:17.782 INFO:teuthology.orchestra.run.smithi070.stdout:Nothing to do. 2024-09-16T09:18:17.782 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-16T09:18:17.865 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install rbd-fuse 2024-09-16T09:18:18.424 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:46 ago on Mon 16 Sep 2024 09:16:32 AM UTC. 2024-09-16T09:18:18.559 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-16T09:18:18.559 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-16T09:18:18.560 INFO:teuthology.orchestra.run.smithi070.stdout: Package Architecture Version Repository Size 2024-09-16T09:18:18.560 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-16T09:18:18.560 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-09-16T09:18:18.560 INFO:teuthology.orchestra.run.smithi070.stdout: rbd-fuse x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 85 k 2024-09-16T09:18:18.560 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:18:18.560 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-09-16T09:18:18.560 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-16T09:18:18.560 INFO:teuthology.orchestra.run.smithi070.stdout:Install 1 Package 2024-09-16T09:18:18.560 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:18:18.560 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 85 k 2024-09-16T09:18:18.560 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 231 k 2024-09-16T09:18:18.560 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-09-16T09:18:18.567 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : rbd-mirror-2:19.3.0-4917.g8293d73f.el9.x86_64 1/1 2024-09-16T09:18:18.567 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-16T09:18:18.567 INFO:teuthology.orchestra.run.smithi084.stdout:Installed: 2024-09-16T09:18:18.567 INFO:teuthology.orchestra.run.smithi084.stdout: rbd-mirror-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T09:18:18.567 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-16T09:18:18.568 INFO:teuthology.orchestra.run.smithi084.stdout:Complete! 2024-09-16T09:18:18.758 INFO:teuthology.orchestra.run.smithi070.stdout:rbd-fuse-19.3.0-4917.g8293d73f.el9.x86_64.rpm 431 kB/s | 85 kB 00:00 2024-09-16T09:18:18.759 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-09-16T09:18:18.759 INFO:teuthology.orchestra.run.smithi070.stdout:Total 428 kB/s | 85 kB 00:00 2024-09-16T09:18:18.759 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-09-16T09:18:18.766 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-09-16T09:18:18.766 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-09-16T09:18:18.777 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum -y install rbd-nbd 2024-09-16T09:18:18.831 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-09-16T09:18:18.831 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-09-16T09:18:19.034 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-09-16T09:18:19.139 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : rbd-fuse-2:19.3.0-4917.g8293d73f.el9.x86_64 1/1 2024-09-16T09:18:19.360 INFO:teuthology.orchestra.run.smithi084.stdout:Last metadata expiration check: 0:01:50 ago on Mon 16 Sep 2024 09:16:29 AM UTC. 2024-09-16T09:18:19.492 INFO:teuthology.orchestra.run.smithi084.stdout:Dependencies resolved. 2024-09-16T09:18:19.493 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-09-16T09:18:19.493 INFO:teuthology.orchestra.run.smithi084.stdout: Package Architecture Version Repository Size 2024-09-16T09:18:19.493 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-09-16T09:18:19.493 INFO:teuthology.orchestra.run.smithi084.stdout:Installing: 2024-09-16T09:18:19.493 INFO:teuthology.orchestra.run.smithi084.stdout: rbd-nbd x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 171 k 2024-09-16T09:18:19.493 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-16T09:18:19.493 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction Summary 2024-09-16T09:18:19.494 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-09-16T09:18:19.494 INFO:teuthology.orchestra.run.smithi084.stdout:Install 1 Package 2024-09-16T09:18:19.494 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-16T09:18:19.494 INFO:teuthology.orchestra.run.smithi084.stdout:Total download size: 171 k 2024-09-16T09:18:19.494 INFO:teuthology.orchestra.run.smithi084.stdout:Installed size: 490 k 2024-09-16T09:18:19.494 INFO:teuthology.orchestra.run.smithi084.stdout:Downloading Packages: 2024-09-16T09:18:19.503 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: rbd-fuse-2:19.3.0-4917.g8293d73f.el9.x86_64 1/1 2024-09-16T09:18:19.704 INFO:teuthology.orchestra.run.smithi084.stdout:rbd-nbd-19.3.0-4917.g8293d73f.el9.x86_64.rpm 811 kB/s | 171 kB 00:00 2024-09-16T09:18:19.705 INFO:teuthology.orchestra.run.smithi084.stdout:-------------------------------------------------------------------------------- 2024-09-16T09:18:19.705 INFO:teuthology.orchestra.run.smithi084.stdout:Total 806 kB/s | 171 kB 00:00 2024-09-16T09:18:19.705 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction check 2024-09-16T09:18:19.712 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction check succeeded. 2024-09-16T09:18:19.712 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction test 2024-09-16T09:18:19.769 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : rbd-fuse-2:19.3.0-4917.g8293d73f.el9.x86_64 1/1 2024-09-16T09:18:19.770 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:18:19.770 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-09-16T09:18:19.770 INFO:teuthology.orchestra.run.smithi070.stdout: rbd-fuse-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T09:18:19.770 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:18:19.770 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-16T09:18:19.778 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction test succeeded. 2024-09-16T09:18:19.778 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction 2024-09-16T09:18:19.936 INFO:teuthology.orchestra.run.smithi084.stdout: Preparing : 1/1 2024-09-16T09:18:20.020 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install rbd-mirror 2024-09-16T09:18:20.044 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : rbd-nbd-2:19.3.0-4917.g8293d73f.el9.x86_64 1/1 2024-09-16T09:18:20.514 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: rbd-nbd-2:19.3.0-4917.g8293d73f.el9.x86_64 1/1 2024-09-16T09:18:20.616 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:48 ago on Mon 16 Sep 2024 09:16:32 AM UTC. 2024-09-16T09:18:20.755 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-16T09:18:20.755 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-16T09:18:20.755 INFO:teuthology.orchestra.run.smithi070.stdout: Package Arch Version Repo Size 2024-09-16T09:18:20.756 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-16T09:18:20.756 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-09-16T09:18:20.756 INFO:teuthology.orchestra.run.smithi070.stdout: rbd-mirror x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 3.2 M 2024-09-16T09:18:20.756 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:18:20.756 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-09-16T09:18:20.756 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-16T09:18:20.756 INFO:teuthology.orchestra.run.smithi070.stdout:Install 1 Package 2024-09-16T09:18:20.756 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:18:20.756 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 3.2 M 2024-09-16T09:18:20.756 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 14 M 2024-09-16T09:18:20.756 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-09-16T09:18:20.778 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : rbd-nbd-2:19.3.0-4917.g8293d73f.el9.x86_64 1/1 2024-09-16T09:18:20.779 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-16T09:18:20.779 INFO:teuthology.orchestra.run.smithi084.stdout:Installed: 2024-09-16T09:18:20.779 INFO:teuthology.orchestra.run.smithi084.stdout: rbd-nbd-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T09:18:20.779 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-16T09:18:20.779 INFO:teuthology.orchestra.run.smithi084.stdout:Complete! 2024-09-16T09:18:20.979 DEBUG:teuthology.parallel:result is None 2024-09-16T09:18:21.105 INFO:teuthology.orchestra.run.smithi070.stdout:rbd-mirror-19.3.0-4917.g8293d73f.el9.x86_64.rpm 9.1 MB/s | 3.2 MB 00:00 2024-09-16T09:18:21.106 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-09-16T09:18:21.106 INFO:teuthology.orchestra.run.smithi070.stdout:Total 9.1 MB/s | 3.2 MB 00:00 2024-09-16T09:18:21.106 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-09-16T09:18:21.116 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-09-16T09:18:21.116 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-09-16T09:18:21.199 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-09-16T09:18:21.199 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-09-16T09:18:21.641 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-09-16T09:18:21.706 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : rbd-mirror-2:19.3.0-4917.g8293d73f.el9.x86_64 1/1 2024-09-16T09:18:21.732 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: rbd-mirror-2:19.3.0-4917.g8293d73f.el9.x86_64 1/1 2024-09-16T09:18:21.732 INFO:teuthology.orchestra.run.smithi070.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T09:18:21.732 INFO:teuthology.orchestra.run.smithi070.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-16T09:18:21.732 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T09:18:21.733 INFO:teuthology.orchestra.run.smithi070.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T09:18:21.733 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:18:22.852 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : rbd-mirror-2:19.3.0-4917.g8293d73f.el9.x86_64 1/1 2024-09-16T09:18:22.852 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:18:22.852 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-09-16T09:18:22.852 INFO:teuthology.orchestra.run.smithi070.stdout: rbd-mirror-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T09:18:22.852 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:18:22.852 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-16T09:18:23.083 DEBUG:teuthology.orchestra.run.smithi070:> sudo yum -y install rbd-nbd 2024-09-16T09:18:23.669 INFO:teuthology.orchestra.run.smithi070.stdout:Last metadata expiration check: 0:01:51 ago on Mon 16 Sep 2024 09:16:32 AM UTC. 2024-09-16T09:18:23.803 INFO:teuthology.orchestra.run.smithi070.stdout:Dependencies resolved. 2024-09-16T09:18:23.804 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-16T09:18:23.804 INFO:teuthology.orchestra.run.smithi070.stdout: Package Architecture Version Repository Size 2024-09-16T09:18:23.804 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-16T09:18:23.804 INFO:teuthology.orchestra.run.smithi070.stdout:Installing: 2024-09-16T09:18:23.804 INFO:teuthology.orchestra.run.smithi070.stdout: rbd-nbd x86_64 2:19.3.0-4917.g8293d73f.el9 ceph 171 k 2024-09-16T09:18:23.804 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:18:23.804 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction Summary 2024-09-16T09:18:23.805 INFO:teuthology.orchestra.run.smithi070.stdout:================================================================================ 2024-09-16T09:18:23.805 INFO:teuthology.orchestra.run.smithi070.stdout:Install 1 Package 2024-09-16T09:18:23.805 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:18:23.805 INFO:teuthology.orchestra.run.smithi070.stdout:Total download size: 171 k 2024-09-16T09:18:23.805 INFO:teuthology.orchestra.run.smithi070.stdout:Installed size: 490 k 2024-09-16T09:18:23.805 INFO:teuthology.orchestra.run.smithi070.stdout:Downloading Packages: 2024-09-16T09:18:24.018 INFO:teuthology.orchestra.run.smithi070.stdout:rbd-nbd-19.3.0-4917.g8293d73f.el9.x86_64.rpm 803 kB/s | 171 kB 00:00 2024-09-16T09:18:24.018 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-09-16T09:18:24.019 INFO:teuthology.orchestra.run.smithi070.stdout:Total 797 kB/s | 171 kB 00:00 2024-09-16T09:18:24.019 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction check 2024-09-16T09:18:24.026 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction check succeeded. 2024-09-16T09:18:24.026 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction test 2024-09-16T09:18:24.090 INFO:teuthology.orchestra.run.smithi070.stdout:Transaction test succeeded. 2024-09-16T09:18:24.091 INFO:teuthology.orchestra.run.smithi070.stdout:Running transaction 2024-09-16T09:18:24.242 INFO:teuthology.orchestra.run.smithi070.stdout: Preparing : 1/1 2024-09-16T09:18:24.349 INFO:teuthology.orchestra.run.smithi070.stdout: Installing : rbd-nbd-2:19.3.0-4917.g8293d73f.el9.x86_64 1/1 2024-09-16T09:18:24.798 INFO:teuthology.orchestra.run.smithi070.stdout: Running scriptlet: rbd-nbd-2:19.3.0-4917.g8293d73f.el9.x86_64 1/1 2024-09-16T09:18:25.091 INFO:teuthology.orchestra.run.smithi070.stdout: Verifying : rbd-nbd-2:19.3.0-4917.g8293d73f.el9.x86_64 1/1 2024-09-16T09:18:25.091 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:18:25.092 INFO:teuthology.orchestra.run.smithi070.stdout:Installed: 2024-09-16T09:18:25.092 INFO:teuthology.orchestra.run.smithi070.stdout: rbd-nbd-2:19.3.0-4917.g8293d73f.el9.x86_64 2024-09-16T09:18:25.092 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:18:25.092 INFO:teuthology.orchestra.run.smithi070.stdout:Complete! 2024-09-16T09:18:25.396 DEBUG:teuthology.parallel:result is None 2024-09-16T09:18:25.397 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8293d73f8690540e843a81caec373f9cc29cf705 2024-09-16T09:18:25.547 DEBUG:teuthology.orchestra.run.smithi070:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-16T09:18:25.568 INFO:teuthology.orchestra.run.smithi070.stdout:19.3.0-4917.g8293d73f.el9 2024-09-16T09:18:25.569 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4917.g8293d73f.el9 2024-09-16T09:18:25.569 INFO:teuthology.task.install:The correct ceph version 19.3.0-4917.g8293d73f is installed. 2024-09-16T09:18:25.571 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8293d73f8690540e843a81caec373f9cc29cf705 2024-09-16T09:18:25.714 DEBUG:teuthology.orchestra.run.smithi084:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-16T09:18:25.739 INFO:teuthology.orchestra.run.smithi084.stdout:19.3.0-4917.g8293d73f.el9 2024-09-16T09:18:25.739 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4917.g8293d73f.el9 2024-09-16T09:18:25.740 INFO:teuthology.task.install:The correct ceph version 19.3.0-4917.g8293d73f is installed. 2024-09-16T09:18:25.741 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-16T09:18:25.742 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-16T09:18:25.742 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-16T09:18:25.766 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-09-16T09:18:25.767 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-16T09:18:25.809 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-16T09:18:25.810 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-16T09:18:25.810 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/usr/bin/daemon-helper 2024-09-16T09:18:25.834 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-16T09:18:25.899 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-09-16T09:18:25.899 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/usr/bin/daemon-helper 2024-09-16T09:18:25.927 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-16T09:18:25.993 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-16T09:18:25.994 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-16T09:18:25.994 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-16T09:18:26.020 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-16T09:18:26.085 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-09-16T09:18:26.085 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-16T09:18:26.112 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-16T09:18:26.178 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-16T09:18:26.178 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-16T09:18:26.178 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/usr/bin/stdin-killer 2024-09-16T09:18:26.203 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-16T09:18:26.269 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-09-16T09:18:26.269 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/usr/bin/stdin-killer 2024-09-16T09:18:26.294 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-16T09:18:26.360 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-16T09:18:26.461 INFO:tasks.cephadm:Config: {'conf': {'mgr': {'debug mgr': 20, 'debug ms': 1}, 'client.0': {'debug ms': 1, 'debug objecter': 20, 'debug rados': 20}, 'global': {'ms type': 'async'}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'paxos service trim min': 10}, 'osd': {'debug ms': 1, 'debug osd': 20, 'osd debug reject backfill probability': 0.3, 'osd delete sleep': 1, 'osd max backfills': 3, 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 2}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'CEPHADM_STRAY_DAEMON', 'CEPHADM_FAILED_DAEMON', 'but it is still running', 'objects unfound and apparently lost', '\\(POOL_APP_NOT_ENABLED\\)'], 'log-only-match': ['CEPHADM_'], 'sha1': '8293d73f8690540e843a81caec373f9cc29cf705', 'cephadm_mode': 'root'} 2024-09-16T09:18:26.461 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 2024-09-16T09:18:26.462 INFO:tasks.cephadm:Cluster fsid is a1321834-740c-11ef-bceb-c7b262605968 2024-09-16T09:18:26.462 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-16T09:18:26.462 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.70', 'mon.c': '[v2:172.21.15.70:3301,v1:172.21.15.70:6790]', 'mon.b': '172.21.15.84'} 2024-09-16T09:18:26.462 INFO:tasks.cephadm:First mon is mon.a on smithi070 2024-09-16T09:18:26.462 INFO:tasks.cephadm:First mgr is y 2024-09-16T09:18:26.462 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-16T09:18:26.462 DEBUG:teuthology.orchestra.run.smithi070:> sudo hostname $(hostname -s) 2024-09-16T09:18:26.488 DEBUG:teuthology.orchestra.run.smithi084:> sudo hostname $(hostname -s) 2024-09-16T09:18:26.513 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-16T09:18:26.513 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8293d73f8690540e843a81caec373f9cc29cf705 2024-09-16T09:18:26.654 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '8293d73f8690540e843a81caec373f9cc29cf705', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/83046/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.3.0-4917-g8293d73f', 'node_name': '172.21.5.39+adami09', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.3.0-4917.g8293d73f'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-guits-main-2024-09-13-1248/8293d73f8690540e843a81caec373f9cc29cf705/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-13 15:55:14.480698', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-guits-main-2024-09-13-1248', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-guits-main-2024-09-13-1248/8293d73f8690540e843a81caec373f9cc29cf705/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-09-16T09:18:26.791 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref wip-guits-main-2024-09-13-1248, sha1 8293d73f8690540e843a81caec373f9cc29cf705 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=8293d73f8690540e843a81caec373f9cc29cf705 2024-09-16T09:18:26.792 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/wip-guits-main-2024-09-13-1248/8293d73f8690540e843a81caec373f9cc29cf705/centos/9/x86_64/flavors/default/cephadm 2024-09-16T09:18:26.793 INFO:tasks.cephadm:Downloading cephadm from url: https://3.chacra.ceph.com/binaries/ceph/wip-guits-main-2024-09-13-1248/8293d73f8690540e843a81caec373f9cc29cf705/centos/9/x86_64/flavors/default/cephadm 2024-09-16T09:18:26.793 DEBUG:teuthology.orchestra.run.smithi070:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-guits-main-2024-09-13-1248/8293d73f8690540e843a81caec373f9cc29cf705/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-16T09:18:27.171 INFO:teuthology.orchestra.run.smithi070.stdout:-rw-r--r--. 1 ubuntu ubuntu 804182 Sep 16 09:18 /home/ubuntu/cephtest/cephadm 2024-09-16T09:18:27.171 DEBUG:teuthology.orchestra.run.smithi084:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-guits-main-2024-09-13-1248/8293d73f8690540e843a81caec373f9cc29cf705/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-16T09:18:27.549 INFO:teuthology.orchestra.run.smithi084.stdout:-rw-r--r--. 1 ubuntu ubuntu 804182 Sep 16 09:18 /home/ubuntu/cephtest/cephadm 2024-09-16T09:18:27.549 DEBUG:teuthology.orchestra.run.smithi070:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-16T09:18:27.569 DEBUG:teuthology.orchestra.run.smithi084:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-16T09:18:27.600 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 on all hosts... 2024-09-16T09:18:27.600 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 pull 2024-09-16T09:18:27.612 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 pull 2024-09-16T09:18:29.258 INFO:teuthology.orchestra.run.smithi070.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705... 2024-09-16T09:18:29.302 INFO:teuthology.orchestra.run.smithi084.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705... 2024-09-16T09:18:58.493 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-09-16T09:18:58.493 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_version": "ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev)", 2024-09-16T09:18:58.493 INFO:teuthology.orchestra.run.smithi084.stdout: "image_id": "425e316ba8ebf74f04aa3640f7deb4fa3f293a78d0c8d097cb679eaf86ddca6c", 2024-09-16T09:18:58.493 INFO:teuthology.orchestra.run.smithi084.stdout: "repo_digests": [ 2024-09-16T09:18:58.493 INFO:teuthology.orchestra.run.smithi084.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce" 2024-09-16T09:18:58.493 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:18:58.493 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-09-16T09:19:04.765 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T09:19:04.765 INFO:teuthology.orchestra.run.smithi070.stdout: "ceph_version": "ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev)", 2024-09-16T09:19:04.765 INFO:teuthology.orchestra.run.smithi070.stdout: "image_id": "425e316ba8ebf74f04aa3640f7deb4fa3f293a78d0c8d097cb679eaf86ddca6c", 2024-09-16T09:19:04.765 INFO:teuthology.orchestra.run.smithi070.stdout: "repo_digests": [ 2024-09-16T09:19:04.765 INFO:teuthology.orchestra.run.smithi070.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce" 2024-09-16T09:19:04.766 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:19:04.766 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T09:19:04.791 DEBUG:teuthology.orchestra.run.smithi070:> sudo mkdir -p /etc/ceph 2024-09-16T09:19:04.833 DEBUG:teuthology.orchestra.run.smithi084:> sudo mkdir -p /etc/ceph 2024-09-16T09:19:04.867 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 777 /etc/ceph 2024-09-16T09:19:04.897 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod 777 /etc/ceph 2024-09-16T09:19:04.935 INFO:tasks.cephadm:Writing seed config... 2024-09-16T09:19:04.937 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-16T09:19:04.937 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-16T09:19:04.937 INFO:tasks.cephadm: override: [client.0] debug ms = 1 2024-09-16T09:19:04.937 INFO:tasks.cephadm: override: [client.0] debug objecter = 20 2024-09-16T09:19:04.937 INFO:tasks.cephadm: override: [client.0] debug rados = 20 2024-09-16T09:19:04.937 INFO:tasks.cephadm: override: [global] ms type = async 2024-09-16T09:19:04.937 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-16T09:19:04.937 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-16T09:19:04.937 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-16T09:19:04.938 INFO:tasks.cephadm: override: [mon] mon min osdmap epochs = 50 2024-09-16T09:19:04.938 INFO:tasks.cephadm: override: [mon] mon osdmap full prune interval = 2 2024-09-16T09:19:04.938 INFO:tasks.cephadm: override: [mon] mon osdmap full prune min = 15 2024-09-16T09:19:04.938 INFO:tasks.cephadm: override: [mon] mon osdmap full prune txsize = 2 2024-09-16T09:19:04.938 INFO:tasks.cephadm: override: [mon] paxos service trim min = 10 2024-09-16T09:19:04.938 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-16T09:19:04.938 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-16T09:19:04.938 INFO:tasks.cephadm: override: [osd] osd debug reject backfill probability = 0.3 2024-09-16T09:19:04.938 INFO:tasks.cephadm: override: [osd] osd delete sleep = 1 2024-09-16T09:19:04.939 INFO:tasks.cephadm: override: [osd] osd max backfills = 3 2024-09-16T09:19:04.939 INFO:tasks.cephadm: override: [osd] osd scrub max interval = 120 2024-09-16T09:19:04.939 INFO:tasks.cephadm: override: [osd] osd scrub min interval = 60 2024-09-16T09:19:04.939 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-16T09:19:04.939 INFO:tasks.cephadm: override: [osd] osd snap trim sleep = 2 2024-09-16T09:19:04.940 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-16T09:19:04.940 DEBUG:teuthology.orchestra.run.smithi070:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-16T09:19:04.955 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = a1321834-740c-11ef-bceb-c7b262605968 ms type = async [osd] osd scrub load threshold = 5.0 osd scrub max interval = 120 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true debug ms = 1 debug osd = 20 osd debug reject backfill probability = 0.3 osd delete sleep = 1 osd max backfills = 3 osd scrub min interval = 60 osd snap trim sleep = 2 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon min osdmap epochs = 50 mon osdmap full prune interval = 2 mon osdmap full prune min = 15 mon osdmap full prune txsize = 2 paxos service trim min = 10 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client.0] debug ms = 1 debug objecter = 20 debug rados = 20 2024-09-16T09:19:04.955 DEBUG:teuthology.orchestra.run.smithi070:mon.a> sudo journalctl -f -n 0 -u ceph-a1321834-740c-11ef-bceb-c7b262605968@mon.a.service 2024-09-16T09:19:04.998 DEBUG:teuthology.orchestra.run.smithi070:mgr.y> sudo journalctl -f -n 0 -u ceph-a1321834-740c-11ef-bceb-c7b262605968@mgr.y.service 2024-09-16T09:19:05.040 INFO:tasks.cephadm:Bootstrapping... 2024-09-16T09:19:05.041 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 -v bootstrap --fsid a1321834-740c-11ef-bceb-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id y --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.70 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-16T09:19:05.247 INFO:teuthology.orchestra.run.smithi070.stdout:-------------------------------------------------------------------------------- 2024-09-16T09:19:05.247 INFO:teuthology.orchestra.run.smithi070.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705', '-v', 'bootstrap', '--fsid', 'a1321834-740c-11ef-bceb-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'y', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.70', '--skip-admin-label'] 2024-09-16T09:19:05.248 INFO:teuthology.orchestra.run.smithi070.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-16T09:19:05.248 INFO:teuthology.orchestra.run.smithi070.stdout:Verifying podman|docker is present... 2024-09-16T09:19:05.272 INFO:teuthology.orchestra.run.smithi070.stdout:/bin/podman: stdout 5.2.2 2024-09-16T09:19:05.272 INFO:teuthology.orchestra.run.smithi070.stdout:Verifying lvm2 is present... 2024-09-16T09:19:05.272 INFO:teuthology.orchestra.run.smithi070.stdout:Verifying time synchronization is in place... 2024-09-16T09:19:05.280 INFO:teuthology.orchestra.run.smithi070.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-16T09:19:05.280 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-16T09:19:05.286 INFO:teuthology.orchestra.run.smithi070.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-16T09:19:05.286 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stdout inactive 2024-09-16T09:19:05.293 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stdout enabled 2024-09-16T09:19:05.299 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stdout active 2024-09-16T09:19:05.299 INFO:teuthology.orchestra.run.smithi070.stdout:Unit chronyd.service is enabled and running 2024-09-16T09:19:05.299 INFO:teuthology.orchestra.run.smithi070.stdout:Repeating the final host check... 2024-09-16T09:19:05.321 INFO:teuthology.orchestra.run.smithi070.stdout:/bin/podman: stdout 5.2.2 2024-09-16T09:19:05.321 INFO:teuthology.orchestra.run.smithi070.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-16T09:19:05.321 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl is present 2024-09-16T09:19:05.321 INFO:teuthology.orchestra.run.smithi070.stdout:lvcreate is present 2024-09-16T09:19:05.328 INFO:teuthology.orchestra.run.smithi070.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-16T09:19:05.328 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-16T09:19:05.334 INFO:teuthology.orchestra.run.smithi070.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-16T09:19:05.334 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stdout inactive 2024-09-16T09:19:05.340 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stdout enabled 2024-09-16T09:19:05.347 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stdout active 2024-09-16T09:19:05.347 INFO:teuthology.orchestra.run.smithi070.stdout:Unit chronyd.service is enabled and running 2024-09-16T09:19:05.347 INFO:teuthology.orchestra.run.smithi070.stdout:Host looks OK 2024-09-16T09:19:05.347 INFO:teuthology.orchestra.run.smithi070.stdout:Cluster fsid: a1321834-740c-11ef-bceb-c7b262605968 2024-09-16T09:19:05.347 INFO:teuthology.orchestra.run.smithi070.stdout:Acquiring lock 140336618215792 on /run/cephadm/a1321834-740c-11ef-bceb-c7b262605968.lock 2024-09-16T09:19:05.347 INFO:teuthology.orchestra.run.smithi070.stdout:Lock 140336618215792 acquired on /run/cephadm/a1321834-740c-11ef-bceb-c7b262605968.lock 2024-09-16T09:19:05.348 INFO:teuthology.orchestra.run.smithi070.stdout:Verifying IP 172.21.15.70 port 3300 ... 2024-09-16T09:19:05.348 INFO:teuthology.orchestra.run.smithi070.stdout:Verifying IP 172.21.15.70 port 6789 ... 2024-09-16T09:19:05.348 INFO:teuthology.orchestra.run.smithi070.stdout:Base mon IP(s) is [172.21.15.70:3300, 172.21.15.70:6789], mon addrv is [v2:172.21.15.70:3300,v1:172.21.15.70:6789] 2024-09-16T09:19:05.352 INFO:teuthology.orchestra.run.smithi070.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.70 metric 100 2024-09-16T09:19:05.352 INFO:teuthology.orchestra.run.smithi070.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.70 metric 100 2024-09-16T09:19:05.354 INFO:teuthology.orchestra.run.smithi070.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-16T09:19:05.354 INFO:teuthology.orchestra.run.smithi070.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-09-16T09:19:05.354 INFO:teuthology.orchestra.run.smithi070.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-09-16T09:19:05.356 INFO:teuthology.orchestra.run.smithi070.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-16T09:19:05.357 INFO:teuthology.orchestra.run.smithi070.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-16T09:19:05.357 INFO:teuthology.orchestra.run.smithi070.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-16T09:19:05.357 INFO:teuthology.orchestra.run.smithi070.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-09-16T09:19:05.357 INFO:teuthology.orchestra.run.smithi070.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:d3e7/64 scope link noprefixroute 2024-09-16T09:19:05.357 INFO:teuthology.orchestra.run.smithi070.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-16T09:19:05.357 INFO:teuthology.orchestra.run.smithi070.stdout:Mon IP `172.21.15.70` is in CIDR network `172.21.0.0/20` 2024-09-16T09:19:05.357 INFO:teuthology.orchestra.run.smithi070.stdout:Mon IP `172.21.15.70` is in CIDR network `172.21.0.0/20` 2024-09-16T09:19:05.358 INFO:teuthology.orchestra.run.smithi070.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-16T09:19:05.358 INFO:teuthology.orchestra.run.smithi070.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-16T09:19:05.358 INFO:teuthology.orchestra.run.smithi070.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705... 2024-09-16T09:19:05.843 INFO:teuthology.orchestra.run.smithi070.stdout:/bin/podman: stdout 425e316ba8ebf74f04aa3640f7deb4fa3f293a78d0c8d097cb679eaf86ddca6c 2024-09-16T09:19:05.843 INFO:teuthology.orchestra.run.smithi070.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705... 2024-09-16T09:19:05.843 INFO:teuthology.orchestra.run.smithi070.stdout:/bin/podman: stderr Getting image source signatures 2024-09-16T09:19:05.843 INFO:teuthology.orchestra.run.smithi070.stdout:/bin/podman: stderr Copying blob sha256:7b1291ae133cf24c065f94db08f06624ab089b121831fa7f7f15d46063d39776 2024-09-16T09:19:05.843 INFO:teuthology.orchestra.run.smithi070.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-16T09:19:05.843 INFO:teuthology.orchestra.run.smithi070.stdout:/bin/podman: stderr Copying config sha256:425e316ba8ebf74f04aa3640f7deb4fa3f293a78d0c8d097cb679eaf86ddca6c 2024-09-16T09:19:05.843 INFO:teuthology.orchestra.run.smithi070.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-16T09:19:06.805 INFO:teuthology.orchestra.run.smithi070.stdout:ceph: stdout ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev) 2024-09-16T09:19:06.806 INFO:teuthology.orchestra.run.smithi070.stdout:Ceph version: ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev) 2024-09-16T09:19:06.806 INFO:teuthology.orchestra.run.smithi070.stdout:Extracting ceph user uid/gid from container image... 2024-09-16T09:19:07.827 INFO:teuthology.orchestra.run.smithi070.stdout:stat: stdout 167 167 2024-09-16T09:19:07.827 INFO:teuthology.orchestra.run.smithi070.stdout:Creating initial keys... 2024-09-16T09:19:08.807 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph-authtool: stdout AQAM+OdmtFaSCxAANB3V/uGe53qGfceXdCiKbQ== 2024-09-16T09:19:09.737 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph-authtool: stdout AQAN+OdmaCFuChAAndHb9SCk0oAZyb2frXCPsw== 2024-09-16T09:19:10.610 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph-authtool: stdout AQAO+OdmEc65BRAAimrf2GAVdb0NDJsm/kp8wQ== 2024-09-16T09:19:10.610 INFO:teuthology.orchestra.run.smithi070.stdout:Creating initial monmap... 2024-09-16T09:19:11.699 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-16T09:19:11.700 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-16T09:19:11.700 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to a1321834-740c-11ef-bceb-c7b262605968 2024-09-16T09:19:11.700 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-16T09:19:11.700 INFO:teuthology.orchestra.run.smithi070.stdout:monmaptool for a [v2:172.21.15.70:3300,v1:172.21.15.70:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-16T09:19:11.700 INFO:teuthology.orchestra.run.smithi070.stdout:setting min_mon_release = quincy 2024-09-16T09:19:11.700 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/monmaptool: set fsid to a1321834-740c-11ef-bceb-c7b262605968 2024-09-16T09:19:11.700 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-16T09:19:11.700 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:19:11.700 INFO:teuthology.orchestra.run.smithi070.stdout:Creating mon... 2024-09-16T09:19:12.654 INFO:teuthology.orchestra.run.smithi070.stdout:create mon.a on 2024-09-16T09:19:12.873 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-16T09:19:13.025 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-16T09:19:13.224 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-a1321834-740c-11ef-bceb-c7b262605968.target → /etc/systemd/system/ceph-a1321834-740c-11ef-bceb-c7b262605968.target. 2024-09-16T09:19:13.224 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-a1321834-740c-11ef-bceb-c7b262605968.target → /etc/systemd/system/ceph-a1321834-740c-11ef-bceb-c7b262605968.target. 2024-09-16T09:19:13.463 INFO:teuthology.orchestra.run.smithi070.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-a1321834-740c-11ef-bceb-c7b262605968@mon.a 2024-09-16T09:19:13.464 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stderr Failed to reset failed state of unit ceph-a1321834-740c-11ef-bceb-c7b262605968@mon.a.service: Unit ceph-a1321834-740c-11ef-bceb-c7b262605968@mon.a.service not loaded. 2024-09-16T09:19:13.637 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-a1321834-740c-11ef-bceb-c7b262605968.target.wants/ceph-a1321834-740c-11ef-bceb-c7b262605968@mon.a.service → /etc/systemd/system/ceph-a1321834-740c-11ef-bceb-c7b262605968@.service. 2024-09-16T09:19:14.081 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:13 smithi070 podman[29882]: 2024-09-16 09:19:13.77623797 +0000 UTC m=+0.015715802 image pull 425e316ba8ebf74f04aa3640f7deb4fa3f293a78d0c8d097cb679eaf86ddca6c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 2024-09-16T09:19:14.081 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:14 smithi070 podman[29882]: 2024-09-16 09:19:14.080210075 +0000 UTC m=+0.319687889 container init dbf92196867ca1555d73c19b20f92760682079f8e93ea9c5267064a3ea3e631f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-mon-a, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T09:19:14.203 INFO:teuthology.orchestra.run.smithi070.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-16T09:19:14.203 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stdout disabled 2024-09-16T09:19:14.210 INFO:teuthology.orchestra.run.smithi070.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-16T09:19:14.210 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stdout inactive 2024-09-16T09:19:14.210 INFO:teuthology.orchestra.run.smithi070.stdout:firewalld.service is not enabled 2024-09-16T09:19:14.210 INFO:teuthology.orchestra.run.smithi070.stdout:Not possible to enable service . firewalld.service is not available 2024-09-16T09:19:14.210 INFO:teuthology.orchestra.run.smithi070.stdout:Waiting for mon to start... 2024-09-16T09:19:14.210 INFO:teuthology.orchestra.run.smithi070.stdout:Waiting for mon... 2024-09-16T09:19:14.332 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:14 smithi070 podman[29882]: 2024-09-16 09:19:14.084921461 +0000 UTC m=+0.324399278 container start dbf92196867ca1555d73c19b20f92760682079f8e93ea9c5267064a3ea3e631f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-mon-a, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-16T09:19:14.333 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:14 smithi070 ceph-mon[29920]: mkfs a1321834-740c-11ef-bceb-c7b262605968 2024-09-16T09:19:14.333 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:14 smithi070 ceph-mon[29920]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-16T09:19:14.333 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:14 smithi070 bash[29882]: dbf92196867ca1555d73c19b20f92760682079f8e93ea9c5267064a3ea3e631f 2024-09-16T09:19:14.333 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:14 smithi070 systemd[1]: Started Ceph mon.a for a1321834-740c-11ef-bceb-c7b262605968. 2024-09-16T09:19:15.262 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:15 smithi070 ceph-mon[29920]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-16T09:19:15.263 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:15 smithi070 ceph-mon[29920]: monmap epoch 1 2024-09-16T09:19:15.263 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:15 smithi070 ceph-mon[29920]: fsid a1321834-740c-11ef-bceb-c7b262605968 2024-09-16T09:19:15.263 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:15 smithi070 ceph-mon[29920]: last_changed 2024-09-16T09:19:10.966128+0000 2024-09-16T09:19:15.263 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:15 smithi070 ceph-mon[29920]: created 2024-09-16T09:19:10.966128+0000 2024-09-16T09:19:15.263 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:15 smithi070 ceph-mon[29920]: min_mon_release 19 (squid) 2024-09-16T09:19:15.263 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:15 smithi070 ceph-mon[29920]: election_strategy: 1 2024-09-16T09:19:15.263 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:15 smithi070 ceph-mon[29920]: 0: [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] mon.a 2024-09-16T09:19:15.263 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:15 smithi070 ceph-mon[29920]: fsmap 2024-09-16T09:19:15.263 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:15 smithi070 ceph-mon[29920]: osdmap e1: 0 total, 0 up, 0 in 2024-09-16T09:19:15.263 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:15 smithi070 ceph-mon[29920]: mgrmap e1: no daemons active 2024-09-16T09:19:15.263 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:15 smithi070 ceph-mon[29920]: from='client.? 172.21.15.70:0/840355611' entity='client.admin' cmd={"prefix": "status"} : dispatch 2024-09-16T09:19:15.300 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout cluster: 2024-09-16T09:19:15.300 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout id: a1321834-740c-11ef-bceb-c7b262605968 2024-09-16T09:19:15.300 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-16T09:19:15.300 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout 2024-09-16T09:19:15.301 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout services: 2024-09-16T09:19:15.301 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.553475s) 2024-09-16T09:19:15.301 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-16T09:19:15.301 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-16T09:19:15.301 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout 2024-09-16T09:19:15.301 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout data: 2024-09-16T09:19:15.301 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-16T09:19:15.301 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-16T09:19:15.301 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-16T09:19:15.302 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout pgs: 2024-09-16T09:19:15.302 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout 2024-09-16T09:19:15.302 INFO:teuthology.orchestra.run.smithi070.stdout:mon is available 2024-09-16T09:19:15.302 INFO:teuthology.orchestra.run.smithi070.stdout:Assimilating anything we can from ceph.conf... 2024-09-16T09:19:16.364 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout 2024-09-16T09:19:16.364 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout [global] 2024-09-16T09:19:16.364 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout fsid = a1321834-740c-11ef-bceb-c7b262605968 2024-09-16T09:19:16.364 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-16T09:19:16.364 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.70:3300,v1:172.21.15.70:6789] 2024-09-16T09:19:16.364 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-16T09:19:16.364 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-16T09:19:16.364 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-16T09:19:16.365 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-16T09:19:16.365 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout 2024-09-16T09:19:16.365 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-16T09:19:16.365 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-16T09:19:16.365 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout 2024-09-16T09:19:16.365 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout [osd] 2024-09-16T09:19:16.365 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-16T09:19:16.365 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-16T09:19:16.365 INFO:teuthology.orchestra.run.smithi070.stdout:Generating new minimal ceph.conf... 2024-09-16T09:19:16.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:16 smithi070 ceph-mon[29920]: from='client.? 172.21.15.70:0/3871811564' entity='client.admin' cmd={"prefix": "config assimilate-conf"} : dispatch 2024-09-16T09:19:16.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:16 smithi070 ceph-mon[29920]: from='client.? 172.21.15.70:0/3871811564' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-16T09:19:17.390 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:17 smithi070 ceph-mon[29920]: from='client.? 172.21.15.70:0/64904241' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:19:17.394 INFO:teuthology.orchestra.run.smithi070.stdout:Restarting the monitor... 2024-09-16T09:19:17.655 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:17 smithi070 systemd[1]: Stopping Ceph mon.a for a1321834-740c-11ef-bceb-c7b262605968... 2024-09-16T09:19:17.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:17 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mon-a[29897]: 2024-09-16T09:19:17.653+0000 7f0a835d8640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T09:19:17.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:17 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mon-a[29897]: 2024-09-16T09:19:17.653+0000 7f0a835d8640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-16T09:19:17.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:17 smithi070 podman[30237]: 2024-09-16 09:19:17.81707681 +0000 UTC m=+0.256628214 container died dbf92196867ca1555d73c19b20f92760682079f8e93ea9c5267064a3ea3e631f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-mon-a, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T09:19:18.386 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:18 smithi070 podman[30237]: 2024-09-16 09:19:18.159239683 +0000 UTC m=+0.598791079 container cleanup dbf92196867ca1555d73c19b20f92760682079f8e93ea9c5267064a3ea3e631f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-mon-a, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T09:19:18.386 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:18 smithi070 bash[30237]: ceph-a1321834-740c-11ef-bceb-c7b262605968-mon-a 2024-09-16T09:19:18.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:18 smithi070 podman[30254]: 2024-09-16 09:19:18.385149152 +0000 UTC m=+0.568624210 container remove dbf92196867ca1555d73c19b20f92760682079f8e93ea9c5267064a3ea3e631f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-mon-a, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T09:19:18.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:18 smithi070 systemd[1]: ceph-a1321834-740c-11ef-bceb-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-16T09:19:18.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:18 smithi070 systemd[1]: Stopped Ceph mon.a for a1321834-740c-11ef-bceb-c7b262605968. 2024-09-16T09:19:18.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:18 smithi070 systemd[1]: Starting Ceph mon.a for a1321834-740c-11ef-bceb-c7b262605968... 2024-09-16T09:19:18.999 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:18 smithi070 podman[30350]: 2024-09-16 09:19:18.641415117 +0000 UTC m=+0.015756666 image pull 425e316ba8ebf74f04aa3640f7deb4fa3f293a78d0c8d097cb679eaf86ddca6c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 2024-09-16T09:19:18.999 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:18 smithi070 podman[30350]: 2024-09-16 09:19:18.777694017 +0000 UTC m=+0.152035601 container create 9e96378a12f59f065acba2ed441430f12044d8f6b219d5b9300ceedd0570921e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-mon-a, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T09:19:18.999 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:18 smithi070 podman[30350]: 2024-09-16 09:19:18.971278908 +0000 UTC m=+0.345620447 container init 9e96378a12f59f065acba2ed441430f12044d8f6b219d5b9300ceedd0570921e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-mon-a, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-16T09:19:18.999 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:18 smithi070 podman[30350]: 2024-09-16 09:19:18.976440137 +0000 UTC m=+0.350781673 container start 9e96378a12f59f065acba2ed441430f12044d8f6b219d5b9300ceedd0570921e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-mon-a, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-16T09:19:19.085 INFO:teuthology.orchestra.run.smithi070.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-16T09:19:19.305 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:18 smithi070 ceph-mon[30388]: set uid:gid to 167:167 (ceph:ceph) 2024-09-16T09:19:19.305 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:18 smithi070 ceph-mon[30388]: ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev), process ceph-mon, pid 7 2024-09-16T09:19:19.306 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:18 smithi070 ceph-mon[30388]: pidfile_write: ignore empty --pid-file 2024-09-16T09:19:19.306 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: load: jerasure load: lrc 2024-09-16T09:19:19.306 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: RocksDB version: 7.9.2 2024-09-16T09:19:19.306 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Git sha 0 2024-09-16T09:19:19.306 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Compile date 2024-09-13 14:50:41 2024-09-16T09:19:19.306 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: DB SUMMARY 2024-09-16T09:19:19.306 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: DB Session ID: WCTG3ERE005X44THGEDL 2024-09-16T09:19:19.306 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: CURRENT file: CURRENT 2024-09-16T09:19:19.306 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: IDENTITY file: IDENTITY 2024-09-16T09:19:19.306 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-16T09:19:19.306 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-16T09:19:19.306 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 94994 ; 2024-09-16T09:19:19.306 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.error_if_exists: 0 2024-09-16T09:19:19.307 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.create_if_missing: 0 2024-09-16T09:19:19.307 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.paranoid_checks: 1 2024-09-16T09:19:19.307 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-16T09:19:19.307 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-16T09:19:19.307 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-16T09:19:19.307 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.env: 0x55f56cce8120 2024-09-16T09:19:19.307 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.fs: PosixFileSystem 2024-09-16T09:19:19.307 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.info_log: 0x55f56f30d700 2024-09-16T09:19:19.307 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.max_file_opening_threads: 16 2024-09-16T09:19:19.307 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.statistics: (nil) 2024-09-16T09:19:19.307 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.use_fsync: 0 2024-09-16T09:19:19.307 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.max_log_file_size: 0 2024-09-16T09:19:19.307 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-16T09:19:19.307 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-16T09:19:19.308 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.keep_log_file_num: 1000 2024-09-16T09:19:19.308 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.recycle_log_file_num: 0 2024-09-16T09:19:19.308 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.allow_fallocate: 1 2024-09-16T09:19:19.308 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.allow_mmap_reads: 0 2024-09-16T09:19:19.308 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.allow_mmap_writes: 0 2024-09-16T09:19:19.308 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.use_direct_reads: 0 2024-09-16T09:19:19.308 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-16T09:19:19.308 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.create_missing_column_families: 0 2024-09-16T09:19:19.308 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.db_log_dir: 2024-09-16T09:19:19.308 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.wal_dir: 2024-09-16T09:19:19.308 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-16T09:19:19.310 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-16T09:19:19.310 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-16T09:19:19.310 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-16T09:19:19.310 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-16T09:19:19.310 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-16T09:19:19.310 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.advise_random_on_open: 1 2024-09-16T09:19:19.310 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.db_write_buffer_size: 0 2024-09-16T09:19:19.310 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.write_buffer_manager: 0x55f56f311b80 2024-09-16T09:19:19.310 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-16T09:19:19.310 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-16T09:19:19.310 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-16T09:19:19.311 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.rate_limiter: (nil) 2024-09-16T09:19:19.311 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-16T09:19:19.311 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.wal_recovery_mode: 2 2024-09-16T09:19:19.311 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.enable_thread_tracking: 0 2024-09-16T09:19:19.311 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.enable_pipelined_write: 0 2024-09-16T09:19:19.311 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.unordered_write: 0 2024-09-16T09:19:19.311 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-16T09:19:19.311 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-16T09:19:19.311 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-16T09:19:19.311 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-16T09:19:19.311 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.row_cache: None 2024-09-16T09:19:19.311 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.wal_filter: None 2024-09-16T09:19:19.311 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-16T09:19:19.312 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.allow_ingest_behind: 0 2024-09-16T09:19:19.312 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.two_write_queues: 0 2024-09-16T09:19:19.312 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.manual_wal_flush: 0 2024-09-16T09:19:19.312 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.wal_compression: 0 2024-09-16T09:19:19.312 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.atomic_flush: 0 2024-09-16T09:19:19.312 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-16T09:19:19.312 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-16T09:19:19.312 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-16T09:19:19.312 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.log_readahead_size: 0 2024-09-16T09:19:19.312 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-16T09:19:19.312 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.best_efforts_recovery: 0 2024-09-16T09:19:19.313 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-16T09:19:19.313 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-16T09:19:19.313 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.allow_data_in_errors: 0 2024-09-16T09:19:19.313 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.db_host_id: __hostname__ 2024-09-16T09:19:19.313 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-16T09:19:19.313 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.max_background_jobs: 2 2024-09-16T09:19:19.313 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.max_background_compactions: -1 2024-09-16T09:19:19.313 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.max_subcompactions: 1 2024-09-16T09:19:19.313 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-16T09:19:19.313 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-16T09:19:19.313 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-16T09:19:19.314 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.max_total_wal_size: 0 2024-09-16T09:19:19.314 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-16T09:19:19.314 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-16T09:19:19.314 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-16T09:19:19.314 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-16T09:19:19.314 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.max_open_files: -1 2024-09-16T09:19:19.314 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.bytes_per_sync: 0 2024-09-16T09:19:19.314 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-16T09:19:19.314 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-16T09:19:19.314 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.compaction_readahead_size: 0 2024-09-16T09:19:19.314 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.max_background_flushes: -1 2024-09-16T09:19:19.314 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Compression algorithms supported: 2024-09-16T09:19:19.314 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: kZSTD supported: 0 2024-09-16T09:19:19.314 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: kXpressCompression supported: 0 2024-09-16T09:19:19.314 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: kBZip2Compression supported: 0 2024-09-16T09:19:19.315 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-16T09:19:19.315 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: kLZ4Compression supported: 1 2024-09-16T09:19:19.315 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: kZlibCompression supported: 1 2024-09-16T09:19:19.315 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: kLZ4HCCompression supported: 1 2024-09-16T09:19:19.315 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: kSnappyCompression supported: 1 2024-09-16T09:19:19.315 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-16T09:19:19.315 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-16T09:19:19.315 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-16T09:19:19.315 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-16T09:19:19.315 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-16T09:19:19.315 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.merge_operator: 2024-09-16T09:19:19.315 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.compaction_filter: None 2024-09-16T09:19:19.315 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.compaction_filter_factory: None 2024-09-16T09:19:19.316 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.sst_partitioner_factory: None 2024-09-16T09:19:19.316 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-16T09:19:19.316 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-16T09:19:19.316 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55f56f30d320) 2024-09-16T09:19:19.316 INFO:journalctl@ceph.mon.a.smithi070.stdout: cache_index_and_filter_blocks: 1 2024-09-16T09:19:19.316 INFO:journalctl@ceph.mon.a.smithi070.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-16T09:19:19.316 INFO:journalctl@ceph.mon.a.smithi070.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-16T09:19:19.316 INFO:journalctl@ceph.mon.a.smithi070.stdout: pin_top_level_index_and_filter: 1 2024-09-16T09:19:19.317 INFO:journalctl@ceph.mon.a.smithi070.stdout: index_type: 0 2024-09-16T09:19:19.317 INFO:journalctl@ceph.mon.a.smithi070.stdout: data_block_index_type: 0 2024-09-16T09:19:19.317 INFO:journalctl@ceph.mon.a.smithi070.stdout: index_shortening: 1 2024-09-16T09:19:19.317 INFO:journalctl@ceph.mon.a.smithi070.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-16T09:19:19.317 INFO:journalctl@ceph.mon.a.smithi070.stdout: checksum: 4 2024-09-16T09:19:19.317 INFO:journalctl@ceph.mon.a.smithi070.stdout: no_block_cache: 0 2024-09-16T09:19:19.317 INFO:journalctl@ceph.mon.a.smithi070.stdout: block_cache: 0x55f56f301610 2024-09-16T09:19:19.317 INFO:journalctl@ceph.mon.a.smithi070.stdout: block_cache_name: BinnedLRUCache 2024-09-16T09:19:19.317 INFO:journalctl@ceph.mon.a.smithi070.stdout: block_cache_options: 2024-09-16T09:19:19.318 INFO:journalctl@ceph.mon.a.smithi070.stdout: capacity : 536870912 2024-09-16T09:19:19.318 INFO:journalctl@ceph.mon.a.smithi070.stdout: num_shard_bits : 4 2024-09-16T09:19:19.318 INFO:journalctl@ceph.mon.a.smithi070.stdout: strict_capacity_limit : 0 2024-09-16T09:19:19.318 INFO:journalctl@ceph.mon.a.smithi070.stdout: high_pri_pool_ratio: 0.000 2024-09-16T09:19:19.318 INFO:journalctl@ceph.mon.a.smithi070.stdout: block_cache_compressed: (nil) 2024-09-16T09:19:19.318 INFO:journalctl@ceph.mon.a.smithi070.stdout: persistent_cache: (nil) 2024-09-16T09:19:19.318 INFO:journalctl@ceph.mon.a.smithi070.stdout: block_size: 4096 2024-09-16T09:19:19.318 INFO:journalctl@ceph.mon.a.smithi070.stdout: block_size_deviation: 10 2024-09-16T09:19:19.318 INFO:journalctl@ceph.mon.a.smithi070.stdout: block_restart_interval: 16 2024-09-16T09:19:19.318 INFO:journalctl@ceph.mon.a.smithi070.stdout: index_block_restart_interval: 1 2024-09-16T09:19:19.318 INFO:journalctl@ceph.mon.a.smithi070.stdout: metadata_block_size: 4096 2024-09-16T09:19:19.318 INFO:journalctl@ceph.mon.a.smithi070.stdout: partition_filters: 0 2024-09-16T09:19:19.318 INFO:journalctl@ceph.mon.a.smithi070.stdout: use_delta_encoding: 1 2024-09-16T09:19:19.318 INFO:journalctl@ceph.mon.a.smithi070.stdout: filter_policy: bloomfilter 2024-09-16T09:19:19.318 INFO:journalctl@ceph.mon.a.smithi070.stdout: whole_key_filtering: 1 2024-09-16T09:19:19.319 INFO:journalctl@ceph.mon.a.smithi070.stdout: verify_compression: 0 2024-09-16T09:19:19.319 INFO:journalctl@ceph.mon.a.smithi070.stdout: read_amp_bytes_per_bit: 0 2024-09-16T09:19:19.319 INFO:journalctl@ceph.mon.a.smithi070.stdout: format_version: 5 2024-09-16T09:19:19.319 INFO:journalctl@ceph.mon.a.smithi070.stdout: enable_index_compression: 1 2024-09-16T09:19:19.319 INFO:journalctl@ceph.mon.a.smithi070.stdout: block_align: 0 2024-09-16T09:19:19.319 INFO:journalctl@ceph.mon.a.smithi070.stdout: max_auto_readahead_size: 262144 2024-09-16T09:19:19.319 INFO:journalctl@ceph.mon.a.smithi070.stdout: prepopulate_block_cache: 0 2024-09-16T09:19:19.319 INFO:journalctl@ceph.mon.a.smithi070.stdout: initial_auto_readahead_size: 8192 2024-09-16T09:19:19.319 INFO:journalctl@ceph.mon.a.smithi070.stdout: num_file_reads_for_auto_readahead: 2 2024-09-16T09:19:19.319 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.write_buffer_size: 33554432 2024-09-16T09:19:19.319 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.max_write_buffer_number: 2 2024-09-16T09:19:19.319 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.compression: NoCompression 2024-09-16T09:19:19.319 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.bottommost_compression: Disabled 2024-09-16T09:19:19.319 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.prefix_extractor: nullptr 2024-09-16T09:19:19.320 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-16T09:19:19.320 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.num_levels: 7 2024-09-16T09:19:19.320 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-16T09:19:19.320 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-16T09:19:19.320 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-16T09:19:19.320 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-16T09:19:19.320 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-16T09:19:19.320 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-16T09:19:19.320 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-16T09:19:19.320 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-16T09:19:19.320 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-16T09:19:19.320 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-16T09:19:19.320 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-16T09:19:19.321 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-16T09:19:19.321 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-16T09:19:19.321 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.compression_opts.level: 32767 2024-09-16T09:19:19.321 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.compression_opts.strategy: 0 2024-09-16T09:19:19.321 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-16T09:19:19.321 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-16T09:19:19.321 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-16T09:19:19.321 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-16T09:19:19.321 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.compression_opts.enabled: false 2024-09-16T09:19:19.321 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-16T09:19:19.321 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-16T09:19:19.321 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-16T09:19:19.321 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-16T09:19:19.321 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.target_file_size_base: 67108864 2024-09-16T09:19:19.322 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-16T09:19:19.322 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-16T09:19:19.322 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-16T09:19:19.322 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-16T09:19:19.322 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-16T09:19:19.322 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-16T09:19:19.322 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-16T09:19:19.322 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-16T09:19:19.322 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-16T09:19:19.322 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-16T09:19:19.322 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-16T09:19:19.322 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-16T09:19:19.322 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-16T09:19:19.323 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-16T09:19:19.323 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.arena_block_size: 1048576 2024-09-16T09:19:19.323 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-16T09:19:19.323 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-16T09:19:19.323 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.disable_auto_compactions: 0 2024-09-16T09:19:19.323 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-16T09:19:19.323 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-16T09:19:19.323 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-16T09:19:19.323 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-16T09:19:19.323 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-16T09:19:19.324 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-16T09:19:19.324 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-16T09:19:19.324 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-16T09:19:19.324 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-16T09:19:19.324 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-16T09:19:19.324 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-16T09:19:19.324 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.inplace_update_support: 0 2024-09-16T09:19:19.324 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-16T09:19:19.324 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-16T09:19:19.324 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-16T09:19:19.324 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-16T09:19:19.324 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.bloom_locality: 0 2024-09-16T09:19:19.325 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.max_successive_merges: 0 2024-09-16T09:19:19.325 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-16T09:19:19.325 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.paranoid_file_checks: 0 2024-09-16T09:19:19.325 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.force_consistency_checks: 1 2024-09-16T09:19:19.325 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.report_bg_io_stats: 0 2024-09-16T09:19:19.325 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.ttl: 2592000 2024-09-16T09:19:19.325 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-16T09:19:19.325 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-16T09:19:19.325 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-16T09:19:19.325 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.enable_blob_files: false 2024-09-16T09:19:19.325 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.min_blob_size: 0 2024-09-16T09:19:19.325 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.blob_file_size: 268435456 2024-09-16T09:19:19.326 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-16T09:19:19.326 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-16T09:19:19.326 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-16T09:19:19.326 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-16T09:19:19.326 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-16T09:19:19.326 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.blob_file_starting_level: 0 2024-09-16T09:19:19.326 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-16T09:19:19.326 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-16T09:19:19.326 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-16T09:19:19.326 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 80baec04-e9b5-49e5-b72c-d53fc8a6c0b3 2024-09-16T09:19:19.326 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726478359004270, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-16T09:19:19.326 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-16T09:19:19.326 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726478359005671, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 91014, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 285, "table_properties": {"data_size": 89077, "index_size": 257, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 645, "raw_key_size": 11698, "raw_average_key_size": 47, "raw_value_size": 82603, "raw_average_value_size": 338, "num_data_blocks": 11, "num_entries": 244, "num_filter_entries": 244, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726478359, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "80baec04-e9b5-49e5-b72c-d53fc8a6c0b3", "db_session_id": "WCTG3ERE005X44THGEDL", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-16T09:19:19.327 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726478359005819, "job": 1, "event": "recovery_finished"} 2024-09-16T09:19:19.327 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-16T09:19:19.327 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-16T09:19:19.327 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55f56f330e00 2024-09-16T09:19:19.327 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: DB pointer 0x55f56f432000 2024-09-16T09:19:19.327 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-16T09:19:19.327 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-16T09:19:19.327 INFO:journalctl@ceph.mon.a.smithi070.stdout: ** DB Stats ** 2024-09-16T09:19:19.327 INFO:journalctl@ceph.mon.a.smithi070.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-16T09:19:19.327 INFO:journalctl@ceph.mon.a.smithi070.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-16T09:19:19.327 INFO:journalctl@ceph.mon.a.smithi070.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-16T09:19:19.327 INFO:journalctl@ceph.mon.a.smithi070.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-16T09:19:19.327 INFO:journalctl@ceph.mon.a.smithi070.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-16T09:19:19.327 INFO:journalctl@ceph.mon.a.smithi070.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-16T09:19:19.327 INFO:journalctl@ceph.mon.a.smithi070.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-16T09:19:19.328 INFO:journalctl@ceph.mon.a.smithi070.stdout: 2024-09-16T09:19:19.328 INFO:journalctl@ceph.mon.a.smithi070.stdout: ** Compaction Stats [default] ** 2024-09-16T09:19:19.328 INFO:journalctl@ceph.mon.a.smithi070.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-16T09:19:19.328 INFO:journalctl@ceph.mon.a.smithi070.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-16T09:19:19.328 INFO:journalctl@ceph.mon.a.smithi070.stdout: L0 2/0 90.74 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 96.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-16T09:19:19.328 INFO:journalctl@ceph.mon.a.smithi070.stdout: Sum 2/0 90.74 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 96.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-16T09:19:19.328 INFO:journalctl@ceph.mon.a.smithi070.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 96.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-16T09:19:19.328 INFO:journalctl@ceph.mon.a.smithi070.stdout: 2024-09-16T09:19:19.328 INFO:journalctl@ceph.mon.a.smithi070.stdout: ** Compaction Stats [default] ** 2024-09-16T09:19:19.328 INFO:journalctl@ceph.mon.a.smithi070.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-16T09:19:19.328 INFO:journalctl@ceph.mon.a.smithi070.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-16T09:19:19.329 INFO:journalctl@ceph.mon.a.smithi070.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 96.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-16T09:19:19.329 INFO:journalctl@ceph.mon.a.smithi070.stdout: 2024-09-16T09:19:19.329 INFO:journalctl@ceph.mon.a.smithi070.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-16T09:19:19.329 INFO:journalctl@ceph.mon.a.smithi070.stdout: 2024-09-16T09:19:19.329 INFO:journalctl@ceph.mon.a.smithi070.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-16T09:19:19.329 INFO:journalctl@ceph.mon.a.smithi070.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-16T09:19:19.329 INFO:journalctl@ceph.mon.a.smithi070.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-16T09:19:19.329 INFO:journalctl@ceph.mon.a.smithi070.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-16T09:19:19.329 INFO:journalctl@ceph.mon.a.smithi070.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-16T09:19:19.329 INFO:journalctl@ceph.mon.a.smithi070.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-16T09:19:19.329 INFO:journalctl@ceph.mon.a.smithi070.stdout: Cumulative compaction: 0.00 GB write, 19.25 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-16T09:19:19.329 INFO:journalctl@ceph.mon.a.smithi070.stdout: Interval compaction: 0.00 GB write, 19.25 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-16T09:19:19.329 INFO:journalctl@ceph.mon.a.smithi070.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-16T09:19:19.330 INFO:journalctl@ceph.mon.a.smithi070.stdout: Block cache BinnedLRUCache@0x55f56f301610#7 capacity: 512.00 MB usage: 1.30 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-16T09:19:19.330 INFO:journalctl@ceph.mon.a.smithi070.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.83 KB,0.000157952%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2024-09-16T09:19:19.330 INFO:journalctl@ceph.mon.a.smithi070.stdout: 2024-09-16T09:19:19.330 INFO:journalctl@ceph.mon.a.smithi070.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-16T09:19:19.330 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: starting mon.a rank 0 at public addrs [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] at bind addrs [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid a1321834-740c-11ef-bceb-c7b262605968 2024-09-16T09:19:19.330 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: mon.a@-1(???) e1 preinit fsid a1321834-740c-11ef-bceb-c7b262605968 2024-09-16T09:19:19.330 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: mon.a@-1(???).mds e1 new map 2024-09-16T09:19:19.330 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: mon.a@-1(???).mds e1 print_map 2024-09-16T09:19:19.330 INFO:journalctl@ceph.mon.a.smithi070.stdout: e1 2024-09-16T09:19:19.330 INFO:journalctl@ceph.mon.a.smithi070.stdout: btime 2024-09-16T09:19:14:121324+0000 2024-09-16T09:19:19.330 INFO:journalctl@ceph.mon.a.smithi070.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T09:19:19.330 INFO:journalctl@ceph.mon.a.smithi070.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-16T09:19:19.330 INFO:journalctl@ceph.mon.a.smithi070.stdout: legacy client fscid: -1 2024-09-16T09:19:19.330 INFO:journalctl@ceph.mon.a.smithi070.stdout: 2024-09-16T09:19:19.330 INFO:journalctl@ceph.mon.a.smithi070.stdout: No filesystems configured 2024-09-16T09:19:19.331 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-16T09:19:19.331 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-16T09:19:19.331 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-16T09:19:19.331 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-16T09:19:19.331 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-16T09:19:19.331 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-16T09:19:19.331 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: monmap epoch 1 2024-09-16T09:19:19.331 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: fsid a1321834-740c-11ef-bceb-c7b262605968 2024-09-16T09:19:19.331 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: last_changed 2024-09-16T09:19:10.966128+0000 2024-09-16T09:19:19.331 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: created 2024-09-16T09:19:10.966128+0000 2024-09-16T09:19:19.331 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: min_mon_release 19 (squid) 2024-09-16T09:19:19.331 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: election_strategy: 1 2024-09-16T09:19:19.331 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: 0: [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] mon.a 2024-09-16T09:19:19.331 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: fsmap 2024-09-16T09:19:19.332 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: osdmap e1: 0 total, 0 up, 0 in 2024-09-16T09:19:19.332 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 ceph-mon[30388]: mgrmap e1: no daemons active 2024-09-16T09:19:19.332 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 bash[30350]: 9e96378a12f59f065acba2ed441430f12044d8f6b219d5b9300ceedd0570921e 2024-09-16T09:19:19.332 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:19 smithi070 systemd[1]: Started Ceph mon.a for a1321834-740c-11ef-bceb-c7b262605968. 2024-09-16T09:19:20.164 INFO:teuthology.orchestra.run.smithi070.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-16T09:19:20.186 INFO:teuthology.orchestra.run.smithi070.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-16T09:19:20.186 INFO:teuthology.orchestra.run.smithi070.stdout:Creating mgr... 2024-09-16T09:19:20.186 INFO:teuthology.orchestra.run.smithi070.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-16T09:19:20.187 INFO:teuthology.orchestra.run.smithi070.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-16T09:19:20.456 INFO:teuthology.orchestra.run.smithi070.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-a1321834-740c-11ef-bceb-c7b262605968@mgr.y 2024-09-16T09:19:20.456 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stderr Failed to reset failed state of unit ceph-a1321834-740c-11ef-bceb-c7b262605968@mgr.y.service: Unit ceph-a1321834-740c-11ef-bceb-c7b262605968@mgr.y.service not loaded. 2024-09-16T09:19:20.614 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-a1321834-740c-11ef-bceb-c7b262605968.target.wants/ceph-a1321834-740c-11ef-bceb-c7b262605968@mgr.y.service → /etc/systemd/system/ceph-a1321834-740c-11ef-bceb-c7b262605968@.service. 2024-09-16T09:19:20.850 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:20 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3198257805' entity='client.admin' 2024-09-16T09:19:20.850 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:19:20 smithi070 systemd[1]: Starting Ceph mgr.y for a1321834-740c-11ef-bceb-c7b262605968... 2024-09-16T09:19:21.132 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:19:20 smithi070 podman[30622]: 2024-09-16 09:19:20.751888691 +0000 UTC m=+0.015635969 image pull 425e316ba8ebf74f04aa3640f7deb4fa3f293a78d0c8d097cb679eaf86ddca6c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 2024-09-16T09:19:21.132 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:19:20 smithi070 podman[30622]: 2024-09-16 09:19:20.854305625 +0000 UTC m=+0.118052934 container create f4b4a3b21ded4963b9941bfff0cc662c9ab8f330dfe0b6ac3574c9851783d9c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2) 2024-09-16T09:19:21.132 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:19:21 smithi070 podman[30622]: 2024-09-16 09:19:21.056469332 +0000 UTC m=+0.320216600 container init f4b4a3b21ded4963b9941bfff0cc662c9ab8f330dfe0b6ac3574c9851783d9c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-16T09:19:21.132 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:19:21 smithi070 podman[30622]: 2024-09-16 09:19:21.061005328 +0000 UTC m=+0.324752596 container start f4b4a3b21ded4963b9941bfff0cc662c9ab8f330dfe0b6ac3574c9851783d9c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True) 2024-09-16T09:19:21.133 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:19:21 smithi070 bash[30622]: f4b4a3b21ded4963b9941bfff0cc662c9ab8f330dfe0b6ac3574c9851783d9c0 2024-09-16T09:19:21.148 INFO:teuthology.orchestra.run.smithi070.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-16T09:19:21.148 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stdout disabled 2024-09-16T09:19:21.156 INFO:teuthology.orchestra.run.smithi070.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-16T09:19:21.156 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stdout inactive 2024-09-16T09:19:21.156 INFO:teuthology.orchestra.run.smithi070.stdout:firewalld.service is not enabled 2024-09-16T09:19:21.157 INFO:teuthology.orchestra.run.smithi070.stdout:Not possible to enable service . firewalld.service is not available 2024-09-16T09:19:21.167 INFO:teuthology.orchestra.run.smithi070.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-16T09:19:21.167 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stdout disabled 2024-09-16T09:19:21.175 INFO:teuthology.orchestra.run.smithi070.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-16T09:19:21.175 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stdout inactive 2024-09-16T09:19:21.175 INFO:teuthology.orchestra.run.smithi070.stdout:firewalld.service is not enabled 2024-09-16T09:19:21.176 INFO:teuthology.orchestra.run.smithi070.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-16T09:19:21.176 INFO:teuthology.orchestra.run.smithi070.stdout:Waiting for mgr to start... 2024-09-16T09:19:21.176 INFO:teuthology.orchestra.run.smithi070.stdout:Waiting for mgr... 2024-09-16T09:19:21.391 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:19:21 smithi070 systemd[1]: Started Ceph mgr.y for a1321834-740c-11ef-bceb-c7b262605968. 2024-09-16T09:19:21.987 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:21 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1774095326' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-16T09:19:22.293 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout 2024-09-16T09:19:22.293 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout { 2024-09-16T09:19:22.294 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "fsid": "a1321834-740c-11ef-bceb-c7b262605968", 2024-09-16T09:19:22.294 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "health": { 2024-09-16T09:19:22.294 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-16T09:19:22.294 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-16T09:19:22.294 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-16T09:19:22.294 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2024-09-16T09:19:22.294 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-16T09:19:22.294 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-16T09:19:22.294 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout 0 2024-09-16T09:19:22.294 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout ], 2024-09-16T09:19:22.294 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-16T09:19:22.294 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "a" 2024-09-16T09:19:22.295 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout ], 2024-09-16T09:19:22.295 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-09-16T09:19:22.295 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-16T09:19:22.295 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T09:19:22.295 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-16T09:19:22.295 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-16T09:19:22.295 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2024-09-16T09:19:22.295 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-16T09:19:22.295 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T09:19:22.295 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-16T09:19:22.295 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-16T09:19:22.295 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-16T09:19:22.295 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-16T09:19:22.295 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-16T09:19:22.296 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-16T09:19:22.299 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2024-09-16T09:19:22.299 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-16T09:19:22.299 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-16T09:19:22.299 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-16T09:19:22.299 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-16T09:19:22.299 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-16T09:19:22.299 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-16T09:19:22.300 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-16T09:19:22.300 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-16T09:19:22.300 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-16T09:19:22.300 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2024-09-16T09:19:22.300 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-16T09:19:22.300 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T09:19:22.300 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "btime": "2024-09-16T09:19:14:121324+0000", 2024-09-16T09:19:22.300 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-16T09:19:22.300 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-16T09:19:22.300 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2024-09-16T09:19:22.300 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-16T09:19:22.300 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-16T09:19:22.300 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-16T09:19:22.300 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-16T09:19:22.301 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-16T09:19:22.301 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-16T09:19:22.301 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "restful" 2024-09-16T09:19:22.301 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout ], 2024-09-16T09:19:22.301 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-16T09:19:22.301 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2024-09-16T09:19:22.301 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-16T09:19:22.301 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T09:19:22.301 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "modified": "2024-09-16T09:19:14.121954+0000", 2024-09-16T09:19:22.301 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-16T09:19:22.301 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2024-09-16T09:19:22.301 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-16T09:19:22.301 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout } 2024-09-16T09:19:22.302 INFO:teuthology.orchestra.run.smithi070.stdout:mgr not available, waiting (1/15)... 2024-09-16T09:19:26.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:26 smithi070 ceph-mon[30388]: Activating manager daemon y 2024-09-16T09:19:26.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:26 smithi070 ceph-mon[30388]: mgrmap e2: y(active, starting, since 0.00335932s) 2024-09-16T09:19:26.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:26 smithi070 ceph-mon[30388]: from='mgr.14100 172.21.15.70:0/3492426294' entity='mgr.y' cmd={"prefix": "mds metadata"} : dispatch 2024-09-16T09:19:26.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:26 smithi070 ceph-mon[30388]: from='mgr.14100 172.21.15.70:0/3492426294' entity='mgr.y' cmd={"prefix": "osd metadata"} : dispatch 2024-09-16T09:19:26.422 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:26 smithi070 ceph-mon[30388]: from='mgr.14100 172.21.15.70:0/3492426294' entity='mgr.y' cmd={"prefix": "mon metadata"} : dispatch 2024-09-16T09:19:26.422 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:26 smithi070 ceph-mon[30388]: from='mgr.14100 172.21.15.70:0/3492426294' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-16T09:19:26.422 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:26 smithi070 ceph-mon[30388]: from='mgr.14100 172.21.15.70:0/3492426294' entity='mgr.y' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-16T09:19:26.422 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:26 smithi070 ceph-mon[30388]: Manager daemon y is now available 2024-09-16T09:19:26.422 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:26 smithi070 ceph-mon[30388]: from='mgr.14100 172.21.15.70:0/3492426294' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"} : dispatch 2024-09-16T09:19:26.422 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:26 smithi070 ceph-mon[30388]: from='mgr.14100 172.21.15.70:0/3492426294' entity='mgr.y' 2024-09-16T09:19:26.422 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:26 smithi070 ceph-mon[30388]: from='mgr.14100 172.21.15.70:0/3492426294' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"} : dispatch 2024-09-16T09:19:26.422 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:26 smithi070 ceph-mon[30388]: from='mgr.14100 172.21.15.70:0/3492426294' entity='mgr.y' 2024-09-16T09:19:26.422 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:26 smithi070 ceph-mon[30388]: from='mgr.14100 172.21.15.70:0/3492426294' entity='mgr.y' 2024-09-16T09:19:28.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:28 smithi070 ceph-mon[30388]: mgrmap e3: y(active, since 1.00788s) 2024-09-16T09:19:28.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:28 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3998526416' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-16T09:19:29.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:29 smithi070 ceph-mon[30388]: mgrmap e4: y(active, since 2s) 2024-09-16T09:19:34.473 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout 2024-09-16T09:19:34.474 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout { 2024-09-16T09:19:34.474 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "fsid": "a1321834-740c-11ef-bceb-c7b262605968", 2024-09-16T09:19:34.474 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "health": { 2024-09-16T09:19:34.474 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-16T09:19:34.474 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-16T09:19:34.474 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-16T09:19:34.474 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2024-09-16T09:19:34.474 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-16T09:19:34.474 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-16T09:19:34.474 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout 0 2024-09-16T09:19:34.474 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout ], 2024-09-16T09:19:34.475 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-16T09:19:34.475 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "a" 2024-09-16T09:19:34.475 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout ], 2024-09-16T09:19:34.475 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-09-16T09:19:34.475 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-16T09:19:34.475 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T09:19:34.475 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-16T09:19:34.475 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-16T09:19:34.475 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2024-09-16T09:19:34.475 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-16T09:19:34.475 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T09:19:34.475 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-16T09:19:34.475 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-16T09:19:34.475 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-16T09:19:34.479 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-16T09:19:34.479 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-16T09:19:34.479 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-16T09:19:34.479 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2024-09-16T09:19:34.479 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-16T09:19:34.479 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-16T09:19:34.479 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-16T09:19:34.479 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-16T09:19:34.480 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-16T09:19:34.480 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-16T09:19:34.480 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-16T09:19:34.480 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-16T09:19:34.480 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-16T09:19:34.480 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2024-09-16T09:19:34.480 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-16T09:19:34.480 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T09:19:34.480 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "btime": "2024-09-16T09:19:14:121324+0000", 2024-09-16T09:19:34.480 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-16T09:19:34.480 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-16T09:19:34.480 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2024-09-16T09:19:34.480 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-16T09:19:34.480 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-16T09:19:34.481 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-16T09:19:34.481 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-16T09:19:34.481 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-16T09:19:34.481 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-16T09:19:34.481 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "restful" 2024-09-16T09:19:34.481 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout ], 2024-09-16T09:19:34.481 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-16T09:19:34.481 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2024-09-16T09:19:34.481 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-16T09:19:34.481 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T09:19:34.481 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "modified": "2024-09-16T09:19:14.121954+0000", 2024-09-16T09:19:34.481 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-16T09:19:34.481 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout }, 2024-09-16T09:19:34.481 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-16T09:19:34.482 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout } 2024-09-16T09:19:34.482 INFO:teuthology.orchestra.run.smithi070.stdout:mgr is available 2024-09-16T09:19:37.170 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:36 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1519275735' entity='client.admin' cmd={"prefix": "config assimilate-conf"} : dispatch 2024-09-16T09:19:37.447 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout 2024-09-16T09:19:37.448 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout [global] 2024-09-16T09:19:37.448 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout fsid = a1321834-740c-11ef-bceb-c7b262605968 2024-09-16T09:19:37.448 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-16T09:19:37.448 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.70:3300,v1:172.21.15.70:6789] 2024-09-16T09:19:37.448 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-16T09:19:37.448 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-16T09:19:37.448 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-16T09:19:37.448 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-16T09:19:37.448 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout 2024-09-16T09:19:37.448 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-16T09:19:37.448 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-16T09:19:37.448 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout 2024-09-16T09:19:37.449 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout [osd] 2024-09-16T09:19:37.449 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-16T09:19:37.449 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-16T09:19:37.449 INFO:teuthology.orchestra.run.smithi070.stdout:Enabling cephadm module... 2024-09-16T09:19:38.458 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:38 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2113173452' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "cephadm"} : dispatch 2024-09-16T09:19:38.458 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:19:38 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ignoring --setuser ceph since I am not root 2024-09-16T09:19:38.458 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:19:38 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ignoring --setgroup ceph since I am not root 2024-09-16T09:19:39.416 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:39 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2113173452' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-16T09:19:39.416 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:39 smithi070 ceph-mon[30388]: mgrmap e5: y(active, since 12s) 2024-09-16T09:19:40.017 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout { 2024-09-16T09:19:40.017 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-16T09:19:40.017 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-16T09:19:40.018 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-09-16T09:19:40.018 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-16T09:19:40.018 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout } 2024-09-16T09:19:40.018 INFO:teuthology.orchestra.run.smithi070.stdout:Waiting for the mgr to restart... 2024-09-16T09:19:40.018 INFO:teuthology.orchestra.run.smithi070.stdout:Waiting for mgr epoch 5... 2024-09-16T09:19:40.279 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:40 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2903954234' entity='client.admin' cmd={"prefix": "mgr stat"} : dispatch 2024-09-16T09:19:43.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:43 smithi070 ceph-mon[30388]: Active manager daemon y restarted 2024-09-16T09:19:43.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:43 smithi070 ceph-mon[30388]: Activating manager daemon y 2024-09-16T09:19:43.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:43 smithi070 ceph-mon[30388]: osdmap e2: 0 total, 0 up, 0 in 2024-09-16T09:19:43.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:43 smithi070 ceph-mon[30388]: mgrmap e6: y(active, starting, since 0.00380421s) 2024-09-16T09:19:43.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:43 smithi070 ceph-mon[30388]: from='mgr.14116 172.21.15.70:0/3172615040' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-16T09:19:43.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:43 smithi070 ceph-mon[30388]: from='mgr.14116 172.21.15.70:0/3172615040' entity='mgr.y' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-16T09:19:43.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:43 smithi070 ceph-mon[30388]: from='mgr.14116 172.21.15.70:0/3172615040' entity='mgr.y' cmd={"prefix": "mds metadata"} : dispatch 2024-09-16T09:19:43.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:43 smithi070 ceph-mon[30388]: from='mgr.14116 172.21.15.70:0/3172615040' entity='mgr.y' cmd={"prefix": "osd metadata"} : dispatch 2024-09-16T09:19:43.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:43 smithi070 ceph-mon[30388]: from='mgr.14116 172.21.15.70:0/3172615040' entity='mgr.y' cmd={"prefix": "mon metadata"} : dispatch 2024-09-16T09:19:43.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:43 smithi070 ceph-mon[30388]: Manager daemon y is now available 2024-09-16T09:19:44.851 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:44 smithi070 ceph-mon[30388]: from='mgr.14116 172.21.15.70:0/3172615040' entity='mgr.y' 2024-09-16T09:19:44.851 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:44 smithi070 ceph-mon[30388]: from='mgr.14116 172.21.15.70:0/3172615040' entity='mgr.y' 2024-09-16T09:19:44.851 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:44 smithi070 ceph-mon[30388]: Found migration_current of "None". Setting to last migration. 2024-09-16T09:19:44.851 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:44 smithi070 ceph-mon[30388]: from='mgr.14116 172.21.15.70:0/3172615040' entity='mgr.y' 2024-09-16T09:19:44.851 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:44 smithi070 ceph-mon[30388]: from='mgr.14116 172.21.15.70:0/3172615040' entity='mgr.y' 2024-09-16T09:19:44.851 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:44 smithi070 ceph-mon[30388]: from='mgr.14116 172.21.15.70:0/3172615040' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:19:44.851 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:44 smithi070 ceph-mon[30388]: from='mgr.14116 172.21.15.70:0/3172615040' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:19:44.851 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:44 smithi070 ceph-mon[30388]: from='mgr.14116 172.21.15.70:0/3172615040' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"} : dispatch 2024-09-16T09:19:44.851 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:44 smithi070 ceph-mon[30388]: from='mgr.14116 172.21.15.70:0/3172615040' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"} : dispatch 2024-09-16T09:19:44.852 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:44 smithi070 ceph-mon[30388]: from='client.14120 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-16T09:19:44.852 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:44 smithi070 ceph-mon[30388]: mgrmap e7: y(active, since 1.00571s) 2024-09-16T09:19:44.852 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:44 smithi070 ceph-mon[30388]: from='client.14120 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-16T09:19:44.852 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:44 smithi070 ceph-mon[30388]: from='mgr.14116 172.21.15.70:0/3172615040' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:19:44.852 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:19:44 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: 2024-09-16T09:19:44.639+0000 7fe5c0031640 -1 log_channel(cephadm) log [ERR] : [16/Sep/2024:09:19:44] ENGINE Error in HTTPServer.serve 2024-09-16T09:19:44.852 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:19:44 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: Traceback (most recent call last): 2024-09-16T09:19:44.852 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:19:44 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T09:19:44.852 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:19:44 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: self._connections.run(self.expiration_interval) 2024-09-16T09:19:44.852 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:19:44 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T09:19:44.852 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:19:44 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: self._run(expiration_interval) 2024-09-16T09:19:44.853 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:19:44 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T09:19:44.853 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:19:44 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T09:19:44.853 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:19:44 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T09:19:44.853 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:19:44 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T09:19:44.853 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:19:44 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T09:19:44.853 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:19:44 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: s = self.context.wrap_socket( 2024-09-16T09:19:44.853 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:19:44 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T09:19:44.853 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:19:44 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: return self.sslsocket_class._create( 2024-09-16T09:19:44.853 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:19:44 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T09:19:44.853 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:19:44 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: self.do_handshake() 2024-09-16T09:19:44.853 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:19:44 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T09:19:44.854 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:19:44 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: self._sslobj.do_handshake() 2024-09-16T09:19:44.854 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:19:44 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T09:19:44.854 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:19:44 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: 2024-09-16T09:19:44.854 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout { 2024-09-16T09:19:44.855 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-16T09:19:44.855 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-16T09:19:44.855 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout } 2024-09-16T09:19:44.855 INFO:teuthology.orchestra.run.smithi070.stdout:mgr epoch 5 is available 2024-09-16T09:19:44.855 INFO:teuthology.orchestra.run.smithi070.stdout:Verifying orchestrator module is enabled... 2024-09-16T09:19:45.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:45 smithi070 ceph-mon[30388]: [16/Sep/2024:09:19:44] ENGINE Bus STARTING 2024-09-16T09:19:45.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:45 smithi070 ceph-mon[30388]: [16/Sep/2024:09:19:44] ENGINE Serving on https://172.21.15.70:7150 2024-09-16T09:19:45.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:45 smithi070 ceph-mon[30388]: [16/Sep/2024:09:19:44] ENGINE Error in HTTPServer.serve 2024-09-16T09:19:45.921 INFO:journalctl@ceph.mon.a.smithi070.stdout: Traceback (most recent call last): 2024-09-16T09:19:45.921 INFO:journalctl@ceph.mon.a.smithi070.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T09:19:45.921 INFO:journalctl@ceph.mon.a.smithi070.stdout: self._connections.run(self.expiration_interval) 2024-09-16T09:19:45.921 INFO:journalctl@ceph.mon.a.smithi070.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T09:19:45.921 INFO:journalctl@ceph.mon.a.smithi070.stdout: self._run(expiration_interval) 2024-09-16T09:19:45.922 INFO:journalctl@ceph.mon.a.smithi070.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T09:19:45.922 INFO:journalctl@ceph.mon.a.smithi070.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T09:19:45.922 INFO:journalctl@ceph.mon.a.smithi070.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T09:19:45.922 INFO:journalctl@ceph.mon.a.smithi070.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T09:19:45.922 INFO:journalctl@ceph.mon.a.smithi070.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T09:19:45.922 INFO:journalctl@ceph.mon.a.smithi070.stdout: s = self.context.wrap_socket( 2024-09-16T09:19:45.922 INFO:journalctl@ceph.mon.a.smithi070.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T09:19:45.922 INFO:journalctl@ceph.mon.a.smithi070.stdout: return self.sslsocket_class._create( 2024-09-16T09:19:45.922 INFO:journalctl@ceph.mon.a.smithi070.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T09:19:45.922 INFO:journalctl@ceph.mon.a.smithi070.stdout: self.do_handshake() 2024-09-16T09:19:45.923 INFO:journalctl@ceph.mon.a.smithi070.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T09:19:45.923 INFO:journalctl@ceph.mon.a.smithi070.stdout: self._sslobj.do_handshake() 2024-09-16T09:19:45.923 INFO:journalctl@ceph.mon.a.smithi070.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T09:19:45.923 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:45 smithi070 ceph-mon[30388]: [16/Sep/2024:09:19:44] ENGINE Serving on http://172.21.15.70:8765 2024-09-16T09:19:45.923 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:45 smithi070 ceph-mon[30388]: [16/Sep/2024:09:19:44] ENGINE Bus STARTED 2024-09-16T09:19:45.923 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:45 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2625398309' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "orchestrator"} : dispatch 2024-09-16T09:19:46.334 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr module 'orchestrator' is already enabled (always-on) 2024-09-16T09:19:46.334 INFO:teuthology.orchestra.run.smithi070.stdout:Setting orchestrator backend to cephadm... 2024-09-16T09:19:46.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:46 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2625398309' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "orchestrator"}]': finished 2024-09-16T09:19:46.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:46 smithi070 ceph-mon[30388]: mgrmap e8: y(active, since 2s) 2024-09-16T09:19:48.147 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:47 smithi070 ceph-mon[30388]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:19:48.147 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:47 smithi070 ceph-mon[30388]: from='mgr.14116 172.21.15.70:0/3172615040' entity='mgr.y' 2024-09-16T09:19:48.147 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:47 smithi070 ceph-mon[30388]: from='mgr.14116 172.21.15.70:0/3172615040' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:19:48.671 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-16T09:19:48.671 INFO:teuthology.orchestra.run.smithi070.stdout:Generating ssh key... 2024-09-16T09:19:49.170 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:48 smithi070 ceph-mon[30388]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:19:50.090 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:19:49 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: Generating public/private rsa key pair. 2024-09-16T09:19:50.091 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:19:49 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: Your identification has been saved in /tmp/tmpkwbbww4s/key 2024-09-16T09:19:50.091 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:19:49 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: Your public key has been saved in /tmp/tmpkwbbww4s/key.pub 2024-09-16T09:19:50.091 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:19:49 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: The key fingerprint is: 2024-09-16T09:19:50.091 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:19:49 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: SHA256:nT2TfbcB4VdeWyK8YqTPg6XnMOSRypCsZiGL1I4BAx0 ceph-a1321834-740c-11ef-bceb-c7b262605968 2024-09-16T09:19:50.091 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:19:49 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: The key's randomart image is: 2024-09-16T09:19:50.091 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:19:49 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: +---[RSA 3072]----+ 2024-09-16T09:19:50.091 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:19:49 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: |o.E. .... +| 2024-09-16T09:19:50.091 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:19:49 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: |o . . o..o=| 2024-09-16T09:19:50.091 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:19:49 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: |.... . + + o.| 2024-09-16T09:19:50.091 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:19:49 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: |.o..+ =.+o.oo | 2024-09-16T09:19:50.091 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:19:49 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: |oo+o o +SOo.= ..o| 2024-09-16T09:19:50.092 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:19:49 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: |o.+. o * = o .+| 2024-09-16T09:19:50.092 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:19:49 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: | o = . . | 2024-09-16T09:19:50.092 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:19:49 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: | . | 2024-09-16T09:19:50.092 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:19:49 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: | | 2024-09-16T09:19:50.092 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:19:49 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: +----[SHA256]-----+ 2024-09-16T09:19:50.955 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:50 smithi070 ceph-mon[30388]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:19:50.955 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:50 smithi070 ceph-mon[30388]: Generating ssh key... 2024-09-16T09:19:50.955 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:50 smithi070 ceph-mon[30388]: from='mgr.14116 172.21.15.70:0/3172615040' entity='mgr.y' 2024-09-16T09:19:50.955 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:50 smithi070 ceph-mon[30388]: from='mgr.14116 172.21.15.70:0/3172615040' entity='mgr.y' 2024-09-16T09:19:51.525 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCfdEJqBY6vtJyXSh4835ChIdxQUVfxOFEvxFjRO9kIz+kYOrdLEDX4GUi4AeADh9ozk8N1XphbfE3N9DSxZlsieEhoyUdjaubTQHXxJBoZajXIJExwfGRmG+u18zUCMbaXIESHQ2F1Z+/1Vo2iAuZPGiEckCUBJDmj/jp7cLv54MTrcfdfb64s5FY6yadBGDqZGFGXD/wsPDcGibRbLcHnnHLa1AohyrYgcjrqlqgl16aNJoc1ihXwKNf7SgvEUXMJ57FJRKR+QNZCy1fyFyo3uzMIaatXloF0MM7p94fn6funxeUiq91iFZNqxMN5QkrfWeCyh18khTceaSykkpkv7krQtzSI/d1W7ERZueB9gsVlLj7ag8jF6/stAzM2ss267gVyr1OXo9OWYRGisam6/vtEgR8BlURQcGdt4hCnj8WTxRuXNr65ZOV4RubCdcm0Game+/2T6Xh5MFtXh5Tvg5cVyy/N5nn45AchMHZUjgEb3wmY1p1mqljdfI1KPKk= ceph-a1321834-740c-11ef-bceb-c7b262605968 2024-09-16T09:19:51.525 INFO:teuthology.orchestra.run.smithi070.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-16T09:19:51.525 INFO:teuthology.orchestra.run.smithi070.stdout:Adding key to root@localhost authorized_keys... 2024-09-16T09:19:51.534 INFO:teuthology.orchestra.run.smithi070.stdout:Adding host smithi070... 2024-09-16T09:19:52.170 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:51 smithi070 ceph-mon[30388]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:19:52.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:52 smithi070 ceph-mon[30388]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi070", "addr": "172.21.15.70", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:19:53.842 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:53 smithi070 ceph-mon[30388]: Deploying cephadm binary to smithi070 2024-09-16T09:19:56.007 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout Added host 'smithi070' with addr '172.21.15.70' 2024-09-16T09:19:56.008 INFO:teuthology.orchestra.run.smithi070.stdout:Deploying unmanaged mon service... 2024-09-16T09:19:56.586 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:56 smithi070 ceph-mon[30388]: from='mgr.14116 172.21.15.70:0/3172615040' entity='mgr.y' 2024-09-16T09:19:56.586 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:56 smithi070 ceph-mon[30388]: Added host smithi070 2024-09-16T09:19:56.586 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:56 smithi070 ceph-mon[30388]: from='mgr.14116 172.21.15.70:0/3172615040' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:19:58.141 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:57 smithi070 ceph-mon[30388]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:19:58.141 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:57 smithi070 ceph-mon[30388]: Saving service mon spec with placement count:5 2024-09-16T09:19:58.141 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:57 smithi070 ceph-mon[30388]: from='mgr.14116 172.21.15.70:0/3172615040' entity='mgr.y' 2024-09-16T09:19:58.141 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:57 smithi070 ceph-mon[30388]: from='mgr.14116 172.21.15.70:0/3172615040' entity='mgr.y' 2024-09-16T09:19:58.143 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-16T09:19:58.143 INFO:teuthology.orchestra.run.smithi070.stdout:Deploying unmanaged mgr service... 2024-09-16T09:19:59.633 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:59 smithi070 ceph-mon[30388]: from='mgr.14116 172.21.15.70:0/3172615040' entity='mgr.y' 2024-09-16T09:19:59.641 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:19:59 smithi070 ceph-mon[30388]: from='mgr.14116 172.21.15.70:0/3172615040' entity='mgr.y' 2024-09-16T09:19:59.641 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-16T09:20:00.474 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:00 smithi070 ceph-mon[30388]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:20:00.475 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:00 smithi070 ceph-mon[30388]: Saving service mgr spec with placement count:2 2024-09-16T09:20:01.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:01 smithi070 ceph-mon[30388]: from='mgr.14116 172.21.15.70:0/3172615040' entity='mgr.y' 2024-09-16T09:20:01.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:01 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/4148102934' entity='client.admin' 2024-09-16T09:20:01.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:01 smithi070 ceph-mon[30388]: from='mgr.14116 172.21.15.70:0/3172615040' entity='mgr.y' 2024-09-16T09:20:02.614 INFO:teuthology.orchestra.run.smithi070.stdout:Enabling the dashboard module... 2024-09-16T09:20:03.255 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:02 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2132999684' entity='client.admin' 2024-09-16T09:20:04.529 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:04 smithi070 ceph-mon[30388]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T09:20:04.529 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:04 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/271282899' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "dashboard"} : dispatch 2024-09-16T09:20:04.530 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:20:04 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ignoring --setuser ceph since I am not root 2024-09-16T09:20:04.530 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:20:04 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ignoring --setgroup ceph since I am not root 2024-09-16T09:20:05.369 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:05 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/271282899' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-16T09:20:05.370 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:05 smithi070 ceph-mon[30388]: mgrmap e9: y(active, since 21s) 2024-09-16T09:20:06.182 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout { 2024-09-16T09:20:06.182 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-16T09:20:06.183 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-16T09:20:06.183 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-09-16T09:20:06.183 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-16T09:20:06.183 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout } 2024-09-16T09:20:06.183 INFO:teuthology.orchestra.run.smithi070.stdout:Waiting for the mgr to restart... 2024-09-16T09:20:06.183 INFO:teuthology.orchestra.run.smithi070.stdout:Waiting for mgr epoch 9... 2024-09-16T09:20:06.438 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:06 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1111802317' entity='client.admin' cmd={"prefix": "mgr stat"} : dispatch 2024-09-16T09:20:09.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:09 smithi070 ceph-mon[30388]: Active manager daemon y restarted 2024-09-16T09:20:09.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:09 smithi070 ceph-mon[30388]: Activating manager daemon y 2024-09-16T09:20:09.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:09 smithi070 ceph-mon[30388]: osdmap e3: 0 total, 0 up, 0 in 2024-09-16T09:20:09.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:09 smithi070 ceph-mon[30388]: mgrmap e10: y(active, starting, since 0.00320513s) 2024-09-16T09:20:09.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:09 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-16T09:20:09.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:09 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-16T09:20:09.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:09 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "mds metadata"} : dispatch 2024-09-16T09:20:09.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:09 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata"} : dispatch 2024-09-16T09:20:09.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:09 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "mon metadata"} : dispatch 2024-09-16T09:20:09.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:09 smithi070 ceph-mon[30388]: Manager daemon y is now available 2024-09-16T09:20:10.644 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:10 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:20:10.644 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:10 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"} : dispatch 2024-09-16T09:20:10.644 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:10 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"} : dispatch 2024-09-16T09:20:10.644 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:10 smithi070 ceph-mon[30388]: mgrmap e11: y(active, since 1.006s) 2024-09-16T09:20:10.930 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout { 2024-09-16T09:20:10.931 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-16T09:20:10.931 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-16T09:20:10.931 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout } 2024-09-16T09:20:10.931 INFO:teuthology.orchestra.run.smithi070.stdout:mgr epoch 9 is available 2024-09-16T09:20:10.931 INFO:teuthology.orchestra.run.smithi070.stdout:Using certmgr to generate dashboard self-signed certificate... 2024-09-16T09:20:11.633 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:11 smithi070 ceph-mon[30388]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-16T09:20:11.633 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:11 smithi070 ceph-mon[30388]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-16T09:20:12.349 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:20:11 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: 2024-09-16T09:20:11.976+0000 7f980c4aa640 -1 log_channel(cephadm) log [ERR] : [16/Sep/2024:09:20:11] ENGINE Error in HTTPServer.serve 2024-09-16T09:20:12.349 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:20:11 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: Traceback (most recent call last): 2024-09-16T09:20:12.349 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:20:11 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T09:20:12.349 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:20:11 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: self._connections.run(self.expiration_interval) 2024-09-16T09:20:12.350 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:20:11 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T09:20:12.350 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:20:11 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: self._run(expiration_interval) 2024-09-16T09:20:12.350 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:20:11 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T09:20:12.350 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:20:11 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T09:20:12.350 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:20:11 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T09:20:12.350 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:20:11 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T09:20:12.350 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:20:11 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T09:20:12.350 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:20:11 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: s = self.context.wrap_socket( 2024-09-16T09:20:12.350 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:20:11 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T09:20:12.350 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:20:11 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: return self.sslsocket_class._create( 2024-09-16T09:20:12.350 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:20:11 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T09:20:12.350 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:20:11 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: self.do_handshake() 2024-09-16T09:20:12.350 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:20:11 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T09:20:12.350 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:20:11 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: self._sslobj.do_handshake() 2024-09-16T09:20:12.351 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:20:11 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T09:20:12.351 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:20:11 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: 2024-09-16T09:20:12.652 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:12 smithi070 ceph-mon[30388]: mgrmap e12: y(active, since 2s) 2024-09-16T09:20:12.652 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:12 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:12.652 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:12 smithi070 ceph-mon[30388]: [16/Sep/2024:09:20:11] ENGINE Bus STARTING 2024-09-16T09:20:12.652 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:12 smithi070 ceph-mon[30388]: [16/Sep/2024:09:20:11] ENGINE Serving on http://172.21.15.70:8765 2024-09-16T09:20:12.653 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:12 smithi070 ceph-mon[30388]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "orch certmgr generate-certificates", "module_name": "dashboard", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:20:12.653 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:12 smithi070 ceph-mon[30388]: [16/Sep/2024:09:20:11] ENGINE Serving on https://172.21.15.70:7150 2024-09-16T09:20:12.653 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:12 smithi070 ceph-mon[30388]: [16/Sep/2024:09:20:11] ENGINE Bus STARTED 2024-09-16T09:20:12.653 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:12 smithi070 ceph-mon[30388]: [16/Sep/2024:09:20:11] ENGINE Error in HTTPServer.serve 2024-09-16T09:20:12.653 INFO:journalctl@ceph.mon.a.smithi070.stdout: Traceback (most recent call last): 2024-09-16T09:20:12.653 INFO:journalctl@ceph.mon.a.smithi070.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T09:20:12.653 INFO:journalctl@ceph.mon.a.smithi070.stdout: self._connections.run(self.expiration_interval) 2024-09-16T09:20:12.653 INFO:journalctl@ceph.mon.a.smithi070.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T09:20:12.653 INFO:journalctl@ceph.mon.a.smithi070.stdout: self._run(expiration_interval) 2024-09-16T09:20:12.653 INFO:journalctl@ceph.mon.a.smithi070.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T09:20:12.653 INFO:journalctl@ceph.mon.a.smithi070.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T09:20:12.653 INFO:journalctl@ceph.mon.a.smithi070.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T09:20:12.654 INFO:journalctl@ceph.mon.a.smithi070.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T09:20:12.654 INFO:journalctl@ceph.mon.a.smithi070.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T09:20:12.654 INFO:journalctl@ceph.mon.a.smithi070.stdout: s = self.context.wrap_socket( 2024-09-16T09:20:12.654 INFO:journalctl@ceph.mon.a.smithi070.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T09:20:12.654 INFO:journalctl@ceph.mon.a.smithi070.stdout: return self.sslsocket_class._create( 2024-09-16T09:20:12.654 INFO:journalctl@ceph.mon.a.smithi070.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T09:20:12.654 INFO:journalctl@ceph.mon.a.smithi070.stdout: self.do_handshake() 2024-09-16T09:20:12.654 INFO:journalctl@ceph.mon.a.smithi070.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T09:20:12.654 INFO:journalctl@ceph.mon.a.smithi070.stdout: self._sslobj.do_handshake() 2024-09-16T09:20:12.654 INFO:journalctl@ceph.mon.a.smithi070.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T09:20:12.654 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:12 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:13.057 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout {"cert": "-----BEGIN CERTIFICATE-----\nMIIFEDCCAvigAwIBAgIUfhnuAxzh4AHatdAgYYYyN9LXiDcwDQYJKoZIhvcNAQEL\nBQAwFzEVMBMGA1UEAwwMY2VwaGFkbS1yb290MB4XDTI0MDkxNjA5MjAxMloXDTM0\nMDkxNzA5MjAxMlowFzEVMBMGA1UEAwwMMTcyLjIxLjE1LjcwMIICIjANBgkqhkiG\n9w0BAQEFAAOCAg8AMIICCgKCAgEAzQR6wkyE0NtTX3/ZVgBjZGmilwyI3a4LLtes\nyr3rWWYaCttOqQZz27MBI92wEepS4Mx3iaQwSW0VdBFgvGbAOT+fpCRutbDsxfeU\nOlgLpfayQCRQBmg5RVV3F7BECVPdeHRDrB4NuGa8bTpH5mDeeyAw82SjFFXYF5TT\nxBkS6503II2F3N8cw6LJUk0eJPIpZ64EVW68jFEqdgTFgvyxvIXEtOwUTVE8IXIK\noqORK7MUnAGIm2H730Li4mzVaibx7NjEyDHxUFGE+o8sied/yEPwUqYmdXR81zGN\ngovTnmu/lm/yAzaRahYbcH3jH+UC42Tw3NDKIF/5TILmellFNAsYRxZGPgHPeQOt\nmuvTLLWY1y9G9nBsOq+tBxohKjEgsiZlhAKQkDu+LsY1NjeDtWF01hGuhny3G3sd\nnw5Ab8EE4MCrSTmVByAOt38Z3B4Rfy9yPdX6bcQEPJ9Y+kzsiZveidd7M50KgoYD\nIiyNqQ4nJ7RPb8fp+KQfiXcNQD8N1vk1umygW8PW3VASuPChAP21eIcHORt48DgW\nW+MSWZiwOOWiTGIu7NKQ7vW7TETRSS9I1fn6oOsE5fw2ER4Ak/4mTobxCa3GLg86\nUoe0wLMFBsOiWYnpDwDkaEHjm/Z9T9uUAKlzlrqgo4Zx7boU5XaD9VT06DjU47XR\n7b5f8bUCAwEAAaNUMFIwQgYDVR0RBDswOYIec21pdGhpMDcwLmZyb250LnNlcGlh\nLmNlcGguY29tghFkYXNoYm9hcmRfc2VydmVyc4cErBUPRjAMBgNVHRMBAf8EAjAA\nMA0GCSqGSIb3DQEBCwUAA4ICAQCOvdn1/elAcQq3P5ascsiot0e1PNnkOO059LWJ\nELMHsWlOAlWyFrB/dXTXZ35VQfC+1IXN0miwGkAkK/rnND1tRk/lbChO0eYAMNNx\nIbx+M7VwqauVmrn7sczNcOOFGm1hEHTrDQzfWYfbgEIYwCsQAB+R5o4IUzMXmgrQ\nZWdlOgpiVvZe70s9AUbk42rx9le5UOKFGexr4V7e/Dew4LeAqcLaJfv3ucLUr91B\nT7sbWGwdhwWT8al2DjYvzD8F4DQauQ7WN2TPfNOCAbPMhKHoBSwtZYjaFsXQ8VYc\n4uChC8nSa67rAplJG/0aRLT40aL8L9yQF1uKzpfIRmj8R7A9O3MEfjQdB3saPkqL\nN3L39keQFWCco1wAz2qEryTbrWThSrsuCvyN6iM9Nnr5zI3orZ9IxQX2mzkkFFjF\nHk5gkIzOuafBuCqtdwe7v4b536QFWCWsFvtPO8SQXMLtC0uaDOxe0dwiNbuUYt4f\nNSr76chdv4xLEZgdhHwn9qI/admyuc1UoALSD+G4bDiyLY3N0AjE/7YCf0N8u9+i\niOpvHujE1U//tHjWoOGUO8OHIMaR/PU039+tx6JR6XSGD/HWHOUFsqJ9xHI/5eiO\n1HZRIkrbKVfHj04t6BgGdcfjZeT0GcOOBA0gVam0LZ8sxp/+NeFycnWiRQl2uq6b\nsbL6/w==\n-----END CERTIFICATE-----\n", "key": "-----BEGIN RSA PRIVATE KEY-----\nMIIJJwIBAAKCAgEAzQR6wkyE0NtTX3/ZVgBjZGmilwyI3a4LLtesyr3rWWYaCttO\nqQZz27MBI92wEepS4Mx3iaQwSW0VdBFgvGbAOT+fpCRutbDsxfeUOlgLpfayQCRQ\nBmg5RVV3F7BECVPdeHRDrB4NuGa8bTpH5mDeeyAw82SjFFXYF5TTxBkS6503II2F\n3N8cw6LJUk0eJPIpZ64EVW68jFEqdgTFgvyxvIXEtOwUTVE8IXIKoqORK7MUnAGI\nm2H730Li4mzVaibx7NjEyDHxUFGE+o8sied/yEPwUqYmdXR81zGNgovTnmu/lm/y\nAzaRahYbcH3jH+UC42Tw3NDKIF/5TILmellFNAsYRxZGPgHPeQOtmuvTLLWY1y9G\n9nBsOq+tBxohKjEgsiZlhAKQkDu+LsY1NjeDtWF01hGuhny3G3sdnw5Ab8EE4MCr\nSTmVByAOt38Z3B4Rfy9yPdX6bcQEPJ9Y+kzsiZveidd7M50KgoYDIiyNqQ4nJ7RP\nb8fp+KQfiXcNQD8N1vk1umygW8PW3VASuPChAP21eIcHORt48DgWW+MSWZiwOOWi\nTGIu7NKQ7vW7TETRSS9I1fn6oOsE5fw2ER4Ak/4mTobxCa3GLg86Uoe0wLMFBsOi\nWYnpDwDkaEHjm/Z9T9uUAKlzlrqgo4Zx7boU5XaD9VT06DjU47XR7b5f8bUCAwEA\nAQKCAgAvkojGwbk2+pGx0QmWMHBfyjhJawXuRJbBN5gXknLEU5Ia71ElHQNw2Sxc\nTXJv8crp8WASjwgKXbbS8D5dcU1M63PIg5Cx9dw3QrABStFnISOzCQoNto0FhuOj\n2kmrTWvQadATJgUigypZgSLSdSti8DETpUhHLXU/YZAzCmtuw1QZqGVFv3rd9Y3Y\nHF9wjMWpUoKcbV3whFwBNzG6p5a5i2O95KMAXueATkVkAGbs+DjBy4ROo/GtWIir\nqDvsTh2oixM8bgRUEhp0hpj75f0rdAJVt0qoyg824v92aapA8jkqeA71Cmo7LuaO\nl5OGoN4m9FMEtikU0phXVe+Tt9e75ZIZJX+QNVvi5cvqyYMiJLGs/AccVUn7RSLX\n2bhSyPTGxUjq270aC2EEI6QHMWfdtuz5wbbyVXgwDIUeshk2O7sk6xi8XlBPMQYq\n0QJvpaJ1hhuiXD41UWozLL0Wp8VWj8UZZKxoX83HoePOyeh1artTMopHYr0xR8zg\nNptcROoZgEatYu5yEiWoPSOR6AlIPqWmZXIV5/SIJo4FgxmLdh/lpcy239LdyO0J\nemoJNVLfQ4rErEwRaOWAowvRw2XFY182YRz4ZaEdlb0KkuyvHtrd4nDusacfrlXv\n6hz2UtIjE6TptYC+oTEWbmPirY25ZJBwXVyvOj8EQBdy+lQZQQKCAQEA8oJetuX3\nZL3qPj/rd7tAvB2gwTKFIRkvnEx4T7rJQlFpAfMjYd3O+ofqPYO0j1dg053G8vj+\nQXD2iyZA3+qsVrjrTYKw1YG4Rtq2bxMTtQjsgFUTKw1e3G2t/Za0c+8wRbciYmnv\npexAw1k2rODfKInEJ/EDZMoqv9xXhzm6Q10ueAt+Nst6ffdB+6qGCcnILQZIl427\n4pMZygzwPpk7W5dT3qcpix75PPwgDzG/K/dK54HWjaTIK7bhHWFypkvWjiv2C5K8\nsAoDykboP9cSxxa8ogj4CpXdz6tS5WBhxSZr/h+tvPk4rWOsgbAZYCPStJQQq24B\nGEmZtpZ8dzVpdQKCAQEA2GwuSGkuibUyiuqUhjzaZgFVydhScj7e70zC2VQtPTag\nVEgnudYfa7EWvQk/nAsQat00YyyQqaEh1QqilBCcnis0oNbmbKhmHsJ8eokgYCSv\n+4z0MdZcU6IUHU/2fFj+BtVQcMYeDBUau2E7SM78XuZ2atkv6AHEilpvLN6danCA\nMov9/V1Ot9cswAa6kNUN1J79/LTaUXbG+HGUh8heIj2o7jZEXmRAIeEk2QTxYDW3\nazVUMKif7+TCWp3PewalRQp4//bo7pNimlKehYqpUAfZHElwENmnJjp0ljFDZW3N\nkVsq+UOmc7HxXe4qQ9tmehvhwNZNr4l70MzqzX8fQQKCAQBmCXK+tB4W1k3UKWLy\n+ufu/uxfvdkEJyvAMPDCY05BSC3Nszbk+8YV5dAqV9tGILbN82n9W8TnVhd2Tv8m\n9//RcXcYIvAytaUKswXl0HoPRxTznlg1WifayY9Wzu1YhoCiJL8j93ztLG3HhCcc\nMHXVAbp2L9OEugeouJZfgGud4nwl6HgPqCf/DmVF9eb2AV/niehLVEiEue409KK1\nRBYUoKSaX218aujlvxPiMMVGQ4PRyvCyom3S7tzjVtg9nlVFdrvM/M3FqEEWACXC\njsUI3ck6yFlNYud0dJqphpc8s8DmqP+aLqapVfIEk0rqB8ilUFN/PgbZ6EXz7Hes\nVN3tAoIBAGg4cHFedZLLN3O+U5fxZhsJOY7iJZHuxNzoUkzmRILd84ei176eZrMt\nrI2MDp3OljFgJQI9S8zkU66MWI6FEWb513Y9Bun8jnDxp5A6yOSDxluVHWOCnyOH\nQbh78W8gS/RudTZZsc6PuzfRMwN06gOpaA08Wd2YPFWEB1XMPnIrixDMaZvBe078\nyAgRNXbZordpeVZgoWHpmRe7WpOo4ewLVXahm3r5PAkUqvH3d2WleJagL9Le4WZc\n+1s389qC8MO2r15UQQfi4fD/KGeWxsJLhToiA12Z8uxt6gcJ4jLOFdiR1rgC4/2D\nC4D4WFIlLcxP8P80NGT8CCwdAlXDigECggEAQLX4IAx2453sG3j9fuMQAhXLqeaU\nqKkmrc8/ciJI8vs3QTT8dN57Oylaa8SviosPbaBm4CXmU+t6VAlN7yqZE7g1eYyJ\nB1meqkdufrC9Le1gbeLw0qMjGgqHJtgFrG5SNmsLiKP4tj9iDSaZ6oM91EA0Or8V\nA7b4OHwGE8rr4cbL263ByWqAP6IAP4rwZpnsEMv2QcXMhGy4BcQajr5geMWmmkGr\nfEM5dWI/QAUCavpZ37Xm7BL2B2MMym2OAT/RlYSPHKkkkCqwiDDWYu3zhvjcu8a4\nUh+vVzGBx98+fG0RYHZZQUWAYIB+EmhMAfb2QhrGBgMsonmEx8C4gi+UCA==\n-----END RSA PRIVATE KEY-----\n"} 2024-09-16T09:20:15.031 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout SSL certificate updated 2024-09-16T09:20:15.278 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:15 smithi070 ceph-mon[30388]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard set-ssl-certificate", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:20:15.278 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:15 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:16.325 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout SSL certificate key updated 2024-09-16T09:20:16.325 INFO:teuthology.orchestra.run.smithi070.stdout:Creating initial admin user... 2024-09-16T09:20:16.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:16 smithi070 ceph-mon[30388]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard set-ssl-certificate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:20:16.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:16 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:17.807 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$/Uq4qTifRpp82x6WhBXDb.2Kmf/mWQPjW.GQnpR7lEI4knWvPLR2m", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726478417, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-16T09:20:17.807 INFO:teuthology.orchestra.run.smithi070.stdout:Fetching dashboard port number... 2024-09-16T09:20:18.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:18 smithi070 ceph-mon[30388]: from='client.14168 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:20:18.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:18 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:19.071 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stdout 8443 2024-09-16T09:20:19.081 INFO:teuthology.orchestra.run.smithi070.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-16T09:20:19.081 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stdout disabled 2024-09-16T09:20:19.088 INFO:teuthology.orchestra.run.smithi070.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-16T09:20:19.088 INFO:teuthology.orchestra.run.smithi070.stdout:systemctl: stdout inactive 2024-09-16T09:20:19.088 INFO:teuthology.orchestra.run.smithi070.stdout:firewalld.service is not enabled 2024-09-16T09:20:19.088 INFO:teuthology.orchestra.run.smithi070.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-16T09:20:19.092 INFO:teuthology.orchestra.run.smithi070.stdout:Ceph Dashboard is now available at: 2024-09-16T09:20:19.092 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:20:19.092 INFO:teuthology.orchestra.run.smithi070.stdout: URL: https://smithi070.front.sepia.ceph.com:8443/ 2024-09-16T09:20:19.092 INFO:teuthology.orchestra.run.smithi070.stdout: User: admin 2024-09-16T09:20:19.092 INFO:teuthology.orchestra.run.smithi070.stdout: Password: 649h1e87zl 2024-09-16T09:20:19.093 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:20:19.093 INFO:teuthology.orchestra.run.smithi070.stdout:Saving cluster configuration to /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/config directory 2024-09-16T09:20:19.324 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:19 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/883592728' entity='client.admin' cmd={"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} : dispatch 2024-09-16T09:20:20.447 INFO:teuthology.orchestra.run.smithi070.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-16T09:20:20.447 INFO:teuthology.orchestra.run.smithi070.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-16T09:20:20.447 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:20:20.447 INFO:teuthology.orchestra.run.smithi070.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-16T09:20:20.447 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:20:20.448 INFO:teuthology.orchestra.run.smithi070.stdout:Or, if you are only running a single cluster on this host: 2024-09-16T09:20:20.448 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:20:20.448 INFO:teuthology.orchestra.run.smithi070.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-16T09:20:20.448 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:20:20.448 INFO:teuthology.orchestra.run.smithi070.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-16T09:20:20.448 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:20:20.448 INFO:teuthology.orchestra.run.smithi070.stdout: ceph telemetry on 2024-09-16T09:20:20.448 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:20:20.448 INFO:teuthology.orchestra.run.smithi070.stdout:For more information see: 2024-09-16T09:20:20.448 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:20:20.448 INFO:teuthology.orchestra.run.smithi070.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-16T09:20:20.448 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:20:20.448 INFO:teuthology.orchestra.run.smithi070.stdout:Bootstrap complete. 2024-09-16T09:20:20.484 INFO:tasks.cephadm:Fetching config... 2024-09-16T09:20:20.485 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-16T09:20:20.485 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-16T09:20:20.501 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-16T09:20:20.501 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-16T09:20:20.501 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-16T09:20:20.557 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-16T09:20:20.557 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-16T09:20:20.557 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-16T09:20:20.623 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-16T09:20:20.624 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-16T09:20:20.624 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-16T09:20:20.679 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-16T09:20:20.679 DEBUG:teuthology.orchestra.run.smithi070:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCfdEJqBY6vtJyXSh4835ChIdxQUVfxOFEvxFjRO9kIz+kYOrdLEDX4GUi4AeADh9ozk8N1XphbfE3N9DSxZlsieEhoyUdjaubTQHXxJBoZajXIJExwfGRmG+u18zUCMbaXIESHQ2F1Z+/1Vo2iAuZPGiEckCUBJDmj/jp7cLv54MTrcfdfb64s5FY6yadBGDqZGFGXD/wsPDcGibRbLcHnnHLa1AohyrYgcjrqlqgl16aNJoc1ihXwKNf7SgvEUXMJ57FJRKR+QNZCy1fyFyo3uzMIaatXloF0MM7p94fn6funxeUiq91iFZNqxMN5QkrfWeCyh18khTceaSykkpkv7krQtzSI/d1W7ERZueB9gsVlLj7ag8jF6/stAzM2ss267gVyr1OXo9OWYRGisam6/vtEgR8BlURQcGdt4hCnj8WTxRuXNr65ZOV4RubCdcm0Game+/2T6Xh5MFtXh5Tvg5cVyy/N5nn45AchMHZUjgEb3wmY1p1mqljdfI1KPKk= ceph-a1321834-740c-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-16T09:20:20.765 INFO:teuthology.orchestra.run.smithi070.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCfdEJqBY6vtJyXSh4835ChIdxQUVfxOFEvxFjRO9kIz+kYOrdLEDX4GUi4AeADh9ozk8N1XphbfE3N9DSxZlsieEhoyUdjaubTQHXxJBoZajXIJExwfGRmG+u18zUCMbaXIESHQ2F1Z+/1Vo2iAuZPGiEckCUBJDmj/jp7cLv54MTrcfdfb64s5FY6yadBGDqZGFGXD/wsPDcGibRbLcHnnHLa1AohyrYgcjrqlqgl16aNJoc1ihXwKNf7SgvEUXMJ57FJRKR+QNZCy1fyFyo3uzMIaatXloF0MM7p94fn6funxeUiq91iFZNqxMN5QkrfWeCyh18khTceaSykkpkv7krQtzSI/d1W7ERZueB9gsVlLj7ag8jF6/stAzM2ss267gVyr1OXo9OWYRGisam6/vtEgR8BlURQcGdt4hCnj8WTxRuXNr65ZOV4RubCdcm0Game+/2T6Xh5MFtXh5Tvg5cVyy/N5nn45AchMHZUjgEb3wmY1p1mqljdfI1KPKk= ceph-a1321834-740c-11ef-bceb-c7b262605968 2024-09-16T09:20:20.777 DEBUG:teuthology.orchestra.run.smithi084:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCfdEJqBY6vtJyXSh4835ChIdxQUVfxOFEvxFjRO9kIz+kYOrdLEDX4GUi4AeADh9ozk8N1XphbfE3N9DSxZlsieEhoyUdjaubTQHXxJBoZajXIJExwfGRmG+u18zUCMbaXIESHQ2F1Z+/1Vo2iAuZPGiEckCUBJDmj/jp7cLv54MTrcfdfb64s5FY6yadBGDqZGFGXD/wsPDcGibRbLcHnnHLa1AohyrYgcjrqlqgl16aNJoc1ihXwKNf7SgvEUXMJ57FJRKR+QNZCy1fyFyo3uzMIaatXloF0MM7p94fn6funxeUiq91iFZNqxMN5QkrfWeCyh18khTceaSykkpkv7krQtzSI/d1W7ERZueB9gsVlLj7ag8jF6/stAzM2ss267gVyr1OXo9OWYRGisam6/vtEgR8BlURQcGdt4hCnj8WTxRuXNr65ZOV4RubCdcm0Game+/2T6Xh5MFtXh5Tvg5cVyy/N5nn45AchMHZUjgEb3wmY1p1mqljdfI1KPKk= ceph-a1321834-740c-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-16T09:20:20.824 INFO:teuthology.orchestra.run.smithi084.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCfdEJqBY6vtJyXSh4835ChIdxQUVfxOFEvxFjRO9kIz+kYOrdLEDX4GUi4AeADh9ozk8N1XphbfE3N9DSxZlsieEhoyUdjaubTQHXxJBoZajXIJExwfGRmG+u18zUCMbaXIESHQ2F1Z+/1Vo2iAuZPGiEckCUBJDmj/jp7cLv54MTrcfdfb64s5FY6yadBGDqZGFGXD/wsPDcGibRbLcHnnHLa1AohyrYgcjrqlqgl16aNJoc1ihXwKNf7SgvEUXMJ57FJRKR+QNZCy1fyFyo3uzMIaatXloF0MM7p94fn6funxeUiq91iFZNqxMN5QkrfWeCyh18khTceaSykkpkv7krQtzSI/d1W7ERZueB9gsVlLj7ag8jF6/stAzM2ss267gVyr1OXo9OWYRGisam6/vtEgR8BlURQcGdt4hCnj8WTxRuXNr65ZOV4RubCdcm0Game+/2T6Xh5MFtXh5Tvg5cVyy/N5nn45AchMHZUjgEb3wmY1p1mqljdfI1KPKk= ceph-a1321834-740c-11ef-bceb-c7b262605968 2024-09-16T09:20:20.837 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-16T09:20:21.083 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:20:21.170 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:20 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2171685956' entity='client.admin' 2024-09-16T09:20:23.170 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:22 smithi070 ceph-mon[30388]: mgrmap e13: y(active, since 12s) 2024-09-16T09:20:23.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:22 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/709774682' entity='client.admin' 2024-09-16T09:20:23.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:22 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:23.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:22 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:23.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:22 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd/host:smithi070", "name": "osd_memory_target"} : dispatch 2024-09-16T09:20:23.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:22 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:23.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:22 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:20:23.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:22 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:23.246 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-16T09:20:23.246 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-16T09:20:24.006 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:20:25.340 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi084 2024-09-16T09:20:25.341 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-09-16T09:20:25.341 DEBUG:teuthology.orchestra.run.smithi084:> dd of=/etc/ceph/ceph.conf 2024-09-16T09:20:25.360 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-09-16T09:20:25.360 DEBUG:teuthology.orchestra.run.smithi084:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-16T09:20:25.418 INFO:tasks.cephadm:Adding host smithi084 to orchestrator... 2024-09-16T09:20:25.419 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph orch host add smithi084 2024-09-16T09:20:25.658 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:20:25.908 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:25 smithi070 ceph-mon[30388]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:20:25.908 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:25 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:25.908 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:25 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:20:25.908 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:25 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:20:25.908 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:25 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:20:25.908 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:25 smithi070 ceph-mon[30388]: Updating smithi070:/etc/ceph/ceph.conf 2024-09-16T09:20:25.908 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:25 smithi070 ceph-mon[30388]: Updating smithi070:/var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T09:20:25.908 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:25 smithi070 ceph-mon[30388]: Updating smithi070:/etc/ceph/ceph.client.admin.keyring 2024-09-16T09:20:25.909 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:25 smithi070 ceph-mon[30388]: Updating smithi070:/var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T09:20:25.909 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:25 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:25.909 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:25 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:25.909 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:25 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:27.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:27 smithi070 ceph-mon[30388]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi084", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:20:27.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:27 smithi070 ceph-mon[30388]: Deploying cephadm binary to smithi084 2024-09-16T09:20:29.357 INFO:teuthology.orchestra.run.smithi070.stdout:Added host 'smithi084' with addr '172.21.15.84' 2024-09-16T09:20:29.995 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph orch host ls --format=json 2024-09-16T09:20:30.236 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:20:30.643 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:30 smithi070 ceph-mon[30388]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T09:20:30.643 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:30 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:30.643 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:30 smithi070 ceph-mon[30388]: Added host smithi084 2024-09-16T09:20:30.643 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:30 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:20:30.643 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:30 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:30.643 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:30 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:31.038 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:20:31.038 INFO:teuthology.orchestra.run.smithi070.stdout:[{"addr": "172.21.15.70", "hostname": "smithi070", "labels": [], "status": ""}, {"addr": "172.21.15.84", "hostname": "smithi084", "labels": [], "status": ""}] 2024-09-16T09:20:31.666 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-16T09:20:31.667 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd crush tunables default 2024-09-16T09:20:31.907 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:20:31.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:31 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:31.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:31 smithi070 ceph-mon[30388]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:20:32.664 INFO:teuthology.orchestra.run.smithi070.stderr:adjusted tunables profile to default 2024-09-16T09:20:32.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:32 smithi070 ceph-mon[30388]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T09:20:32.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:32 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1174040230' entity='client.admin' cmd={"prefix": "osd crush tunables", "profile": "default"} : dispatch 2024-09-16T09:20:33.247 INFO:tasks.cephadm:Adding mon.a on smithi070 2024-09-16T09:20:33.247 INFO:tasks.cephadm:Adding mon.c on smithi070 2024-09-16T09:20:33.247 INFO:tasks.cephadm:Adding mon.b on smithi084 2024-09-16T09:20:33.247 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph orch apply mon '3;smithi070:172.21.15.70=a;smithi070:[v2:172.21.15.70:3301,v1:172.21.15.70:6790]=c;smithi084:172.21.15.84=b' 2024-09-16T09:20:33.519 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-16T09:20:33.566 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-16T09:20:33.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:33 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1174040230' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-16T09:20:33.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:33 smithi070 ceph-mon[30388]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T09:20:33.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:33 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:33.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:33 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:33.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:33 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:33.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:33 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:33.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:33 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd/host:smithi084", "name": "osd_memory_target"} : dispatch 2024-09-16T09:20:33.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:33 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:20:33.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:33 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:20:34.306 INFO:teuthology.orchestra.run.smithi084.stdout:Scheduled mon update... 2024-09-16T09:20:34.921 DEBUG:teuthology.orchestra.run.smithi070:mon.c> sudo journalctl -f -n 0 -u ceph-a1321834-740c-11ef-bceb-c7b262605968@mon.c.service 2024-09-16T09:20:34.924 DEBUG:teuthology.orchestra.run.smithi084:mon.b> sudo journalctl -f -n 0 -u ceph-a1321834-740c-11ef-bceb-c7b262605968@mon.b.service 2024-09-16T09:20:34.927 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-16T09:20:34.927 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-16T09:20:35.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:34 smithi070 ceph-mon[30388]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T09:20:35.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:34 smithi070 ceph-mon[30388]: Updating smithi084:/etc/ceph/ceph.conf 2024-09-16T09:20:35.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:34 smithi070 ceph-mon[30388]: Updating smithi084:/var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T09:20:35.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:34 smithi070 ceph-mon[30388]: Updating smithi084:/etc/ceph/ceph.client.admin.keyring 2024-09-16T09:20:35.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:34 smithi070 ceph-mon[30388]: Updating smithi084:/var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T09:20:35.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:34 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:35.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:34 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:35.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:34 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:35.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:34 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:35.422 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:34 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:20:35.422 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:34 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:20:35.422 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:34 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:20:35.422 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:34 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:35.422 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:34 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-16T09:20:35.422 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:34 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:20:35.681 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T09:20:36.412 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-16T09:20:36.413 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":1,"fsid":"a1321834-740c-11ef-bceb-c7b262605968","modified":"2024-09-16T09:19:10.966128Z","created":"2024-09-16T09:19:10.966128Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:3300","nonce":0},{"type":"v1","addr":"172.21.15.70:6789","nonce":0}]},"addr":"172.21.15.70:6789/0","public_addr":"172.21.15.70:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-16T09:20:36.413 INFO:teuthology.orchestra.run.smithi084.stderr:dumped monmap epoch 1 2024-09-16T09:20:36.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:35 smithi070 ceph-mon[30388]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi070:172.21.15.70=a;smithi070:[v2:172.21.15.70:3301,v1:172.21.15.70:6790]=c;smithi084:172.21.15.84=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:20:36.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:35 smithi070 ceph-mon[30388]: Saving service mon spec with placement smithi070:172.21.15.70=a;smithi070:[v2:172.21.15.70:3301,v1:172.21.15.70:6790]=c;smithi084:172.21.15.84=b;count:3 2024-09-16T09:20:36.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:35 smithi070 ceph-mon[30388]: Deploying daemon mon.b on smithi084 2024-09-16T09:20:37.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:36 smithi070 ceph-mon[30388]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T09:20:37.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:36 smithi070 ceph-mon[30388]: from='client.? 172.21.15.84:0/30804907' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-16T09:20:39.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 podman[32593]: 2024-09-16 09:20:39.038008139 +0000 UTC m=+0.015713874 image pull 425e316ba8ebf74f04aa3640f7deb4fa3f293a78d0c8d097cb679eaf86ddca6c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce 2024-09-16T09:20:39.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:38 smithi070 ceph-mon[30388]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T09:20:39.532 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-16T09:20:39.533 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-16T09:20:39.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 podman[32593]: 2024-09-16 09:20:39.32567591 +0000 UTC m=+0.303381644 container init 5808987e05042ecd2bd0ac3b17c2e85717fe56970b3144b07736437ba2d0a02f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-mon-b, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-16T09:20:39.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 podman[32593]: 2024-09-16 09:20:39.330218764 +0000 UTC m=+0.307924499 container start 5808987e05042ecd2bd0ac3b17c2e85717fe56970b3144b07736437ba2d0a02f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-mon-b, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T09:20:39.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: set uid:gid to 167:167 (ceph:ceph) 2024-09-16T09:20:39.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev), process ceph-mon, pid 7 2024-09-16T09:20:39.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: pidfile_write: ignore empty --pid-file 2024-09-16T09:20:39.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: load: jerasure load: lrc 2024-09-16T09:20:39.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: RocksDB version: 7.9.2 2024-09-16T09:20:39.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Git sha 0 2024-09-16T09:20:39.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Compile date 2024-09-13 14:50:41 2024-09-16T09:20:39.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: DB SUMMARY 2024-09-16T09:20:39.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: DB Session ID: 2YECXZG0I1V1MA00HNTU 2024-09-16T09:20:39.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: CURRENT file: CURRENT 2024-09-16T09:20:39.662 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: IDENTITY file: IDENTITY 2024-09-16T09:20:39.662 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-16T09:20:39.662 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-09-16T09:20:39.662 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 511 ; 2024-09-16T09:20:39.662 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.error_if_exists: 0 2024-09-16T09:20:39.662 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.create_if_missing: 0 2024-09-16T09:20:39.662 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.paranoid_checks: 1 2024-09-16T09:20:39.662 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-16T09:20:39.662 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-16T09:20:39.662 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-16T09:20:39.662 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.env: 0x55c793b3c120 2024-09-16T09:20:39.662 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.fs: PosixFileSystem 2024-09-16T09:20:39.662 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.info_log: 0x55c7954cd700 2024-09-16T09:20:39.663 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.max_file_opening_threads: 16 2024-09-16T09:20:39.664 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.statistics: (nil) 2024-09-16T09:20:39.664 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.use_fsync: 0 2024-09-16T09:20:39.664 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.max_log_file_size: 0 2024-09-16T09:20:39.664 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-16T09:20:39.664 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-16T09:20:39.664 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.keep_log_file_num: 1000 2024-09-16T09:20:39.665 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.recycle_log_file_num: 0 2024-09-16T09:20:39.665 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.allow_fallocate: 1 2024-09-16T09:20:39.665 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.allow_mmap_reads: 0 2024-09-16T09:20:39.665 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.allow_mmap_writes: 0 2024-09-16T09:20:39.665 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.use_direct_reads: 0 2024-09-16T09:20:39.665 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-16T09:20:39.665 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.create_missing_column_families: 0 2024-09-16T09:20:39.665 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.db_log_dir: 2024-09-16T09:20:39.665 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.wal_dir: 2024-09-16T09:20:39.665 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-16T09:20:39.666 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-16T09:20:39.666 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-16T09:20:39.666 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-16T09:20:39.666 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-16T09:20:39.666 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-16T09:20:39.666 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.advise_random_on_open: 1 2024-09-16T09:20:39.666 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.db_write_buffer_size: 0 2024-09-16T09:20:39.666 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.write_buffer_manager: 0x55c7954d1b80 2024-09-16T09:20:39.666 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-16T09:20:39.666 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-16T09:20:39.666 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-16T09:20:39.666 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.rate_limiter: (nil) 2024-09-16T09:20:39.667 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-16T09:20:39.667 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.wal_recovery_mode: 2 2024-09-16T09:20:39.667 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.enable_thread_tracking: 0 2024-09-16T09:20:39.667 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.enable_pipelined_write: 0 2024-09-16T09:20:39.667 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.unordered_write: 0 2024-09-16T09:20:39.667 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-16T09:20:39.667 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-16T09:20:39.667 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-16T09:20:39.667 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-16T09:20:39.667 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.row_cache: None 2024-09-16T09:20:39.667 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.wal_filter: None 2024-09-16T09:20:39.667 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-16T09:20:39.667 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.allow_ingest_behind: 0 2024-09-16T09:20:39.667 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.two_write_queues: 0 2024-09-16T09:20:39.668 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.manual_wal_flush: 0 2024-09-16T09:20:39.668 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.wal_compression: 0 2024-09-16T09:20:39.668 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.atomic_flush: 0 2024-09-16T09:20:39.668 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-16T09:20:39.668 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-16T09:20:39.668 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-16T09:20:39.668 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.log_readahead_size: 0 2024-09-16T09:20:39.668 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-16T09:20:39.668 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.best_efforts_recovery: 0 2024-09-16T09:20:39.668 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-16T09:20:39.668 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-16T09:20:39.668 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.allow_data_in_errors: 0 2024-09-16T09:20:39.668 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.db_host_id: __hostname__ 2024-09-16T09:20:39.668 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-16T09:20:39.669 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.max_background_jobs: 2 2024-09-16T09:20:39.669 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.max_background_compactions: -1 2024-09-16T09:20:39.669 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.max_subcompactions: 1 2024-09-16T09:20:39.669 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-16T09:20:39.669 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-16T09:20:39.669 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-16T09:20:39.669 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.max_total_wal_size: 0 2024-09-16T09:20:39.669 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-16T09:20:39.669 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-16T09:20:39.669 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-16T09:20:39.669 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-16T09:20:39.669 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.max_open_files: -1 2024-09-16T09:20:39.670 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.bytes_per_sync: 0 2024-09-16T09:20:39.670 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-16T09:20:39.670 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-16T09:20:39.670 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.compaction_readahead_size: 0 2024-09-16T09:20:39.670 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.max_background_flushes: -1 2024-09-16T09:20:39.670 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Compression algorithms supported: 2024-09-16T09:20:39.670 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: kZSTD supported: 0 2024-09-16T09:20:39.670 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: kXpressCompression supported: 0 2024-09-16T09:20:39.670 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: kBZip2Compression supported: 0 2024-09-16T09:20:39.670 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-16T09:20:39.670 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: kLZ4Compression supported: 1 2024-09-16T09:20:39.670 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: kZlibCompression supported: 1 2024-09-16T09:20:39.670 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: kLZ4HCCompression supported: 1 2024-09-16T09:20:39.670 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: kSnappyCompression supported: 1 2024-09-16T09:20:39.671 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-16T09:20:39.671 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-16T09:20:39.671 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-09-16T09:20:39.671 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-16T09:20:39.671 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-16T09:20:39.671 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.merge_operator: 2024-09-16T09:20:39.671 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.compaction_filter: None 2024-09-16T09:20:39.671 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.compaction_filter_factory: None 2024-09-16T09:20:39.671 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.sst_partitioner_factory: None 2024-09-16T09:20:39.671 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-16T09:20:39.671 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-16T09:20:39.671 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55c7954cd320) 2024-09-16T09:20:39.671 INFO:journalctl@ceph.mon.b.smithi084.stdout: cache_index_and_filter_blocks: 1 2024-09-16T09:20:39.671 INFO:journalctl@ceph.mon.b.smithi084.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-16T09:20:39.672 INFO:journalctl@ceph.mon.b.smithi084.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-16T09:20:39.672 INFO:journalctl@ceph.mon.b.smithi084.stdout: pin_top_level_index_and_filter: 1 2024-09-16T09:20:39.672 INFO:journalctl@ceph.mon.b.smithi084.stdout: index_type: 0 2024-09-16T09:20:39.672 INFO:journalctl@ceph.mon.b.smithi084.stdout: data_block_index_type: 0 2024-09-16T09:20:39.672 INFO:journalctl@ceph.mon.b.smithi084.stdout: index_shortening: 1 2024-09-16T09:20:39.672 INFO:journalctl@ceph.mon.b.smithi084.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-16T09:20:39.672 INFO:journalctl@ceph.mon.b.smithi084.stdout: checksum: 4 2024-09-16T09:20:39.672 INFO:journalctl@ceph.mon.b.smithi084.stdout: no_block_cache: 0 2024-09-16T09:20:39.672 INFO:journalctl@ceph.mon.b.smithi084.stdout: block_cache: 0x55c7954c1610 2024-09-16T09:20:39.672 INFO:journalctl@ceph.mon.b.smithi084.stdout: block_cache_name: BinnedLRUCache 2024-09-16T09:20:39.672 INFO:journalctl@ceph.mon.b.smithi084.stdout: block_cache_options: 2024-09-16T09:20:39.672 INFO:journalctl@ceph.mon.b.smithi084.stdout: capacity : 536870912 2024-09-16T09:20:39.672 INFO:journalctl@ceph.mon.b.smithi084.stdout: num_shard_bits : 4 2024-09-16T09:20:39.673 INFO:journalctl@ceph.mon.b.smithi084.stdout: strict_capacity_limit : 0 2024-09-16T09:20:39.673 INFO:journalctl@ceph.mon.b.smithi084.stdout: high_pri_pool_ratio: 0.000 2024-09-16T09:20:39.673 INFO:journalctl@ceph.mon.b.smithi084.stdout: block_cache_compressed: (nil) 2024-09-16T09:20:39.673 INFO:journalctl@ceph.mon.b.smithi084.stdout: persistent_cache: (nil) 2024-09-16T09:20:39.673 INFO:journalctl@ceph.mon.b.smithi084.stdout: block_size: 4096 2024-09-16T09:20:39.673 INFO:journalctl@ceph.mon.b.smithi084.stdout: block_size_deviation: 10 2024-09-16T09:20:39.673 INFO:journalctl@ceph.mon.b.smithi084.stdout: block_restart_interval: 16 2024-09-16T09:20:39.674 INFO:journalctl@ceph.mon.b.smithi084.stdout: index_block_restart_interval: 1 2024-09-16T09:20:39.674 INFO:journalctl@ceph.mon.b.smithi084.stdout: metadata_block_size: 4096 2024-09-16T09:20:39.674 INFO:journalctl@ceph.mon.b.smithi084.stdout: partition_filters: 0 2024-09-16T09:20:39.674 INFO:journalctl@ceph.mon.b.smithi084.stdout: use_delta_encoding: 1 2024-09-16T09:20:39.674 INFO:journalctl@ceph.mon.b.smithi084.stdout: filter_policy: bloomfilter 2024-09-16T09:20:39.674 INFO:journalctl@ceph.mon.b.smithi084.stdout: whole_key_filtering: 1 2024-09-16T09:20:39.674 INFO:journalctl@ceph.mon.b.smithi084.stdout: verify_compression: 0 2024-09-16T09:20:39.674 INFO:journalctl@ceph.mon.b.smithi084.stdout: read_amp_bytes_per_bit: 0 2024-09-16T09:20:39.675 INFO:journalctl@ceph.mon.b.smithi084.stdout: format_version: 5 2024-09-16T09:20:39.675 INFO:journalctl@ceph.mon.b.smithi084.stdout: enable_index_compression: 1 2024-09-16T09:20:39.675 INFO:journalctl@ceph.mon.b.smithi084.stdout: block_align: 0 2024-09-16T09:20:39.675 INFO:journalctl@ceph.mon.b.smithi084.stdout: max_auto_readahead_size: 262144 2024-09-16T09:20:39.675 INFO:journalctl@ceph.mon.b.smithi084.stdout: prepopulate_block_cache: 0 2024-09-16T09:20:39.675 INFO:journalctl@ceph.mon.b.smithi084.stdout: initial_auto_readahead_size: 8192 2024-09-16T09:20:39.676 INFO:journalctl@ceph.mon.b.smithi084.stdout: num_file_reads_for_auto_readahead: 2 2024-09-16T09:20:39.676 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.write_buffer_size: 33554432 2024-09-16T09:20:39.676 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.max_write_buffer_number: 2 2024-09-16T09:20:39.676 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.compression: NoCompression 2024-09-16T09:20:39.676 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.bottommost_compression: Disabled 2024-09-16T09:20:39.676 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.prefix_extractor: nullptr 2024-09-16T09:20:39.676 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-16T09:20:39.676 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.num_levels: 7 2024-09-16T09:20:39.676 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-16T09:20:39.676 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-16T09:20:39.676 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-16T09:20:39.677 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-16T09:20:39.677 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-16T09:20:39.677 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-16T09:20:39.677 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-16T09:20:39.677 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-16T09:20:39.677 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-16T09:20:39.677 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-16T09:20:39.677 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-16T09:20:39.677 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-16T09:20:39.677 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-16T09:20:39.677 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.compression_opts.level: 32767 2024-09-16T09:20:39.677 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.compression_opts.strategy: 0 2024-09-16T09:20:39.678 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-16T09:20:39.678 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-16T09:20:39.678 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-16T09:20:39.678 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-16T09:20:39.678 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.compression_opts.enabled: false 2024-09-16T09:20:39.678 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-16T09:20:39.678 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-16T09:20:39.678 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-16T09:20:39.678 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-16T09:20:39.678 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.target_file_size_base: 67108864 2024-09-16T09:20:39.678 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-16T09:20:39.678 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-16T09:20:39.678 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-16T09:20:39.679 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-16T09:20:39.679 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-16T09:20:39.679 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-16T09:20:39.679 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-16T09:20:39.679 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-16T09:20:39.679 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-16T09:20:39.679 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-16T09:20:39.679 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-16T09:20:39.679 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-16T09:20:39.679 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-16T09:20:39.679 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-16T09:20:39.679 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.arena_block_size: 1048576 2024-09-16T09:20:39.679 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-16T09:20:39.679 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-16T09:20:39.680 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.disable_auto_compactions: 0 2024-09-16T09:20:39.680 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-16T09:20:39.680 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-16T09:20:39.680 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-16T09:20:39.680 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-16T09:20:39.680 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-16T09:20:39.680 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-16T09:20:39.680 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-16T09:20:39.680 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-16T09:20:39.680 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-16T09:20:39.681 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-16T09:20:39.681 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-16T09:20:39.681 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.inplace_update_support: 0 2024-09-16T09:20:39.681 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-16T09:20:39.681 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-16T09:20:39.681 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-16T09:20:39.681 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-16T09:20:39.681 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.bloom_locality: 0 2024-09-16T09:20:39.681 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.max_successive_merges: 0 2024-09-16T09:20:39.681 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-16T09:20:39.681 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.paranoid_file_checks: 0 2024-09-16T09:20:39.681 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.force_consistency_checks: 1 2024-09-16T09:20:39.682 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.report_bg_io_stats: 0 2024-09-16T09:20:39.682 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.ttl: 2592000 2024-09-16T09:20:39.682 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-16T09:20:39.682 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-16T09:20:39.682 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-16T09:20:39.682 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.enable_blob_files: false 2024-09-16T09:20:39.682 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.min_blob_size: 0 2024-09-16T09:20:39.682 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.blob_file_size: 268435456 2024-09-16T09:20:39.682 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-16T09:20:39.682 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-16T09:20:39.683 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-16T09:20:39.683 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-16T09:20:39.683 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-16T09:20:39.683 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.blob_file_starting_level: 0 2024-09-16T09:20:39.683 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-16T09:20:39.683 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-16T09:20:39.683 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-16T09:20:39.683 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 22a97202-3bc3-4c2c-a7d8-5acb1fd7649d 2024-09-16T09:20:39.683 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726478439357395, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-16T09:20:39.683 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-16T09:20:39.683 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726478439357781, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726478439, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "22a97202-3bc3-4c2c-a7d8-5acb1fd7649d", "db_session_id": "2YECXZG0I1V1MA00HNTU", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-16T09:20:39.684 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726478439357875, "job": 1, "event": "recovery_finished"} 2024-09-16T09:20:39.684 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-16T09:20:39.684 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-16T09:20:39.684 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55c7954f0e00 2024-09-16T09:20:39.684 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: DB pointer 0x55c7955f2000 2024-09-16T09:20:39.684 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-16T09:20:39.684 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-16T09:20:39.684 INFO:journalctl@ceph.mon.b.smithi084.stdout: ** DB Stats ** 2024-09-16T09:20:39.684 INFO:journalctl@ceph.mon.b.smithi084.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-16T09:20:39.684 INFO:journalctl@ceph.mon.b.smithi084.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-16T09:20:39.684 INFO:journalctl@ceph.mon.b.smithi084.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-16T09:20:39.684 INFO:journalctl@ceph.mon.b.smithi084.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-16T09:20:39.685 INFO:journalctl@ceph.mon.b.smithi084.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-16T09:20:39.685 INFO:journalctl@ceph.mon.b.smithi084.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-16T09:20:39.685 INFO:journalctl@ceph.mon.b.smithi084.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-16T09:20:39.685 INFO:journalctl@ceph.mon.b.smithi084.stdout: 2024-09-16T09:20:39.685 INFO:journalctl@ceph.mon.b.smithi084.stdout: ** Compaction Stats [default] ** 2024-09-16T09:20:39.685 INFO:journalctl@ceph.mon.b.smithi084.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-16T09:20:39.685 INFO:journalctl@ceph.mon.b.smithi084.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-16T09:20:39.685 INFO:journalctl@ceph.mon.b.smithi084.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-16T09:20:39.685 INFO:journalctl@ceph.mon.b.smithi084.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-16T09:20:39.685 INFO:journalctl@ceph.mon.b.smithi084.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-16T09:20:39.685 INFO:journalctl@ceph.mon.b.smithi084.stdout: 2024-09-16T09:20:39.685 INFO:journalctl@ceph.mon.b.smithi084.stdout: ** Compaction Stats [default] ** 2024-09-16T09:20:39.685 INFO:journalctl@ceph.mon.b.smithi084.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-16T09:20:39.686 INFO:journalctl@ceph.mon.b.smithi084.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-16T09:20:39.686 INFO:journalctl@ceph.mon.b.smithi084.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.2 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-16T09:20:39.686 INFO:journalctl@ceph.mon.b.smithi084.stdout: 2024-09-16T09:20:39.686 INFO:journalctl@ceph.mon.b.smithi084.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-16T09:20:39.686 INFO:journalctl@ceph.mon.b.smithi084.stdout: 2024-09-16T09:20:39.686 INFO:journalctl@ceph.mon.b.smithi084.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-16T09:20:39.686 INFO:journalctl@ceph.mon.b.smithi084.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-16T09:20:39.686 INFO:journalctl@ceph.mon.b.smithi084.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-16T09:20:39.687 INFO:journalctl@ceph.mon.b.smithi084.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-16T09:20:39.687 INFO:journalctl@ceph.mon.b.smithi084.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-16T09:20:39.687 INFO:journalctl@ceph.mon.b.smithi084.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-16T09:20:39.687 INFO:journalctl@ceph.mon.b.smithi084.stdout: Cumulative compaction: 0.00 GB write, 0.62 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-16T09:20:39.687 INFO:journalctl@ceph.mon.b.smithi084.stdout: Interval compaction: 0.00 GB write, 0.62 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-16T09:20:39.687 INFO:journalctl@ceph.mon.b.smithi084.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-16T09:20:39.687 INFO:journalctl@ceph.mon.b.smithi084.stdout: Block cache BinnedLRUCache@0x55c7954c1610#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-16T09:20:39.687 INFO:journalctl@ceph.mon.b.smithi084.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-16T09:20:39.687 INFO:journalctl@ceph.mon.b.smithi084.stdout: 2024-09-16T09:20:39.687 INFO:journalctl@ceph.mon.b.smithi084.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-16T09:20:39.687 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-09-16T09:20:39.687 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: using public_addr v2:172.21.15.84:0/0 -> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] 2024-09-16T09:20:39.687 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: starting mon.b rank -1 at public addrs [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] at bind addrs [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid a1321834-740c-11ef-bceb-c7b262605968 2024-09-16T09:20:39.688 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: mon.b@-1(???) e0 preinit fsid a1321834-740c-11ef-bceb-c7b262605968 2024-09-16T09:20:39.688 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: mon.b@-1(synchronizing).mds e1 new map 2024-09-16T09:20:39.688 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: mon.b@-1(synchronizing).mds e1 print_map 2024-09-16T09:20:39.688 INFO:journalctl@ceph.mon.b.smithi084.stdout: e1 2024-09-16T09:20:39.688 INFO:journalctl@ceph.mon.b.smithi084.stdout: btime 2024-09-16T09:19:14:121324+0000 2024-09-16T09:20:39.688 INFO:journalctl@ceph.mon.b.smithi084.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T09:20:39.688 INFO:journalctl@ceph.mon.b.smithi084.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-16T09:20:39.688 INFO:journalctl@ceph.mon.b.smithi084.stdout: legacy client fscid: -1 2024-09-16T09:20:39.688 INFO:journalctl@ceph.mon.b.smithi084.stdout: 2024-09-16T09:20:39.688 INFO:journalctl@ceph.mon.b.smithi084.stdout: No filesystems configured 2024-09-16T09:20:39.688 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-16T09:20:39.688 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-16T09:20:39.689 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-16T09:20:39.689 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-16T09:20:39.689 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-16T09:20:39.689 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-16T09:20:39.689 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-16T09:20:39.689 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-16T09:20:39.689 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-16T09:20:39.689 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-16T09:20:39.689 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi070:172.21.15.70=a;smithi070:[v2:172.21.15.70:3301,v1:172.21.15.70:6790]=c;smithi084:172.21.15.84=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:20:39.689 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: Saving service mon spec with placement smithi070:172.21.15.70=a;smithi070:[v2:172.21.15.70:3301,v1:172.21.15.70:6790]=c;smithi084:172.21.15.84=b;count:3 2024-09-16T09:20:39.689 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: Deploying daemon mon.b on smithi084 2024-09-16T09:20:39.689 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T09:20:39.689 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: from='client.? 172.21.15.84:0/30804907' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-16T09:20:39.689 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T09:20:39.690 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 ceph-mon[32629]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-16T09:20:39.690 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 bash[32593]: 5808987e05042ecd2bd0ac3b17c2e85717fe56970b3144b07736437ba2d0a02f 2024-09-16T09:20:39.690 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:39 smithi084 systemd[1]: Started Ceph mon.b for a1321834-740c-11ef-bceb-c7b262605968. 2024-09-16T09:20:39.792 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:20:44.801 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:44 smithi070 ceph-mon[30388]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T09:20:44.801 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:44 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-16T09:20:44.801 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:44 smithi070 ceph-mon[30388]: mon.a calling monitor election 2024-09-16T09:20:44.801 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:44 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-16T09:20:44.802 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:44 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-16T09:20:44.802 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:44 smithi070 ceph-mon[30388]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T09:20:44.802 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:44 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-16T09:20:44.802 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:44 smithi070 ceph-mon[30388]: mon.b calling monitor election 2024-09-16T09:20:44.802 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:44 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-16T09:20:44.802 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:44 smithi070 ceph-mon[30388]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T09:20:44.802 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:44 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-16T09:20:44.802 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:44 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-16T09:20:44.802 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:44 smithi070 ceph-mon[30388]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-16T09:20:44.802 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:44 smithi070 ceph-mon[30388]: monmap epoch 2 2024-09-16T09:20:44.802 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:44 smithi070 ceph-mon[30388]: fsid a1321834-740c-11ef-bceb-c7b262605968 2024-09-16T09:20:44.802 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:44 smithi070 ceph-mon[30388]: last_changed 2024-09-16T09:20:39.381677+0000 2024-09-16T09:20:44.802 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:44 smithi070 ceph-mon[30388]: created 2024-09-16T09:19:10.966128+0000 2024-09-16T09:20:44.803 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:44 smithi070 ceph-mon[30388]: min_mon_release 19 (squid) 2024-09-16T09:20:44.803 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:44 smithi070 ceph-mon[30388]: election_strategy: 1 2024-09-16T09:20:44.803 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:44 smithi070 ceph-mon[30388]: 0: [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] mon.a 2024-09-16T09:20:44.803 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:44 smithi070 ceph-mon[30388]: 1: [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] mon.b 2024-09-16T09:20:44.803 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:44 smithi070 ceph-mon[30388]: fsmap 2024-09-16T09:20:44.803 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:44 smithi070 ceph-mon[30388]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T09:20:44.803 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:44 smithi070 ceph-mon[30388]: mgrmap e13: y(active, since 35s) 2024-09-16T09:20:44.803 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:44 smithi070 ceph-mon[30388]: overall HEALTH_OK 2024-09-16T09:20:44.803 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:44 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:44.803 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:44 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:44.803 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:44 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:44.803 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:44 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-16T09:20:44.803 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:44 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:20:44.879 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-16T09:20:44.880 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":2,"fsid":"a1321834-740c-11ef-bceb-c7b262605968","modified":"2024-09-16T09:20:39.381677Z","created":"2024-09-16T09:19:10.966128Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:3300","nonce":0},{"type":"v1","addr":"172.21.15.70:6789","nonce":0}]},"addr":"172.21.15.70:6789/0","public_addr":"172.21.15.70:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:3300","nonce":0},{"type":"v1","addr":"172.21.15.84:6789","nonce":0}]},"addr":"172.21.15.84:6789/0","public_addr":"172.21.15.84:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-16T09:20:44.880 INFO:teuthology.orchestra.run.smithi084.stderr:dumped monmap epoch 2 2024-09-16T09:20:44.880 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:44 smithi084 ceph-mon[32629]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T09:20:44.880 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:44 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-16T09:20:44.880 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:44 smithi084 ceph-mon[32629]: mon.a calling monitor election 2024-09-16T09:20:44.881 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:44 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-16T09:20:44.881 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:44 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-16T09:20:44.881 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:44 smithi084 ceph-mon[32629]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T09:20:44.881 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:44 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-16T09:20:44.881 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:44 smithi084 ceph-mon[32629]: mon.b calling monitor election 2024-09-16T09:20:44.881 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:44 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-16T09:20:44.881 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:44 smithi084 ceph-mon[32629]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T09:20:44.881 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:44 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-16T09:20:44.881 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:44 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-16T09:20:44.881 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:44 smithi084 ceph-mon[32629]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-16T09:20:44.881 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:44 smithi084 ceph-mon[32629]: monmap epoch 2 2024-09-16T09:20:44.881 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:44 smithi084 ceph-mon[32629]: fsid a1321834-740c-11ef-bceb-c7b262605968 2024-09-16T09:20:44.882 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:44 smithi084 ceph-mon[32629]: last_changed 2024-09-16T09:20:39.381677+0000 2024-09-16T09:20:44.882 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:44 smithi084 ceph-mon[32629]: created 2024-09-16T09:19:10.966128+0000 2024-09-16T09:20:44.882 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:44 smithi084 ceph-mon[32629]: min_mon_release 19 (squid) 2024-09-16T09:20:44.882 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:44 smithi084 ceph-mon[32629]: election_strategy: 1 2024-09-16T09:20:44.882 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:44 smithi084 ceph-mon[32629]: 0: [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] mon.a 2024-09-16T09:20:44.882 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:44 smithi084 ceph-mon[32629]: 1: [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] mon.b 2024-09-16T09:20:44.882 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:44 smithi084 ceph-mon[32629]: fsmap 2024-09-16T09:20:44.882 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:44 smithi084 ceph-mon[32629]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T09:20:44.882 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:44 smithi084 ceph-mon[32629]: mgrmap e13: y(active, since 35s) 2024-09-16T09:20:44.882 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:44 smithi084 ceph-mon[32629]: overall HEALTH_OK 2024-09-16T09:20:44.882 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:44 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:44.882 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:44 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:44.882 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:44 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:44.882 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:44 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-16T09:20:44.883 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:44 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:20:45.765 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:45 smithi070 ceph-mon[30388]: Deploying daemon mon.c on smithi070 2024-09-16T09:20:45.765 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:45 smithi070 ceph-mon[30388]: from='client.? 172.21.15.84:0/3669468361' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-16T09:20:45.765 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:45 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-16T09:20:45.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:45 smithi084 ceph-mon[32629]: Deploying daemon mon.c on smithi070 2024-09-16T09:20:45.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:45 smithi084 ceph-mon[32629]: from='client.? 172.21.15.84:0/3669468361' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-16T09:20:45.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:45 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-16T09:20:46.513 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-16T09:20:46.513 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-16T09:20:46.702 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:46 smithi070 ceph-mon[30388]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T09:20:46.752 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:20:46.873 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:46 smithi084 ceph-mon[32629]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T09:20:47.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 systemd[1]: Starting Ceph mon.c for a1321834-740c-11ef-bceb-c7b262605968... 2024-09-16T09:20:47.455 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-16T09:20:47.456 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":2,"fsid":"a1321834-740c-11ef-bceb-c7b262605968","modified":"2024-09-16T09:20:39.381677Z","created":"2024-09-16T09:19:10.966128Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:3300","nonce":0},{"type":"v1","addr":"172.21.15.70:6789","nonce":0}]},"addr":"172.21.15.70:6789/0","public_addr":"172.21.15.70:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:3300","nonce":0},{"type":"v1","addr":"172.21.15.84:6789","nonce":0}]},"addr":"172.21.15.84:6789/0","public_addr":"172.21.15.84:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-16T09:20:47.456 INFO:teuthology.orchestra.run.smithi084.stderr:dumped monmap epoch 2 2024-09-16T09:20:47.703 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 podman[40653]: 2024-09-16 09:20:47.342136295 +0000 UTC m=+0.015602968 image pull 425e316ba8ebf74f04aa3640f7deb4fa3f293a78d0c8d097cb679eaf86ddca6c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce 2024-09-16T09:20:47.703 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 podman[40653]: 2024-09-16 09:20:47.448020938 +0000 UTC m=+0.121487603 container create a5f3710eb38e9621e1b045feda0b4e0d907adddffa69804b05c48e9b8a4f6075 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-mon-c, GIT_CLEAN=True, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2) 2024-09-16T09:20:47.703 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 podman[40653]: 2024-09-16 09:20:47.675475464 +0000 UTC m=+0.348942140 container init a5f3710eb38e9621e1b045feda0b4e0d907adddffa69804b05c48e9b8a4f6075 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T09:20:47.703 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 podman[40653]: 2024-09-16 09:20:47.680584531 +0000 UTC m=+0.354051197 container start a5f3710eb38e9621e1b045feda0b4e0d907adddffa69804b05c48e9b8a4f6075 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-mon-c, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=wip-guits-main-2024-09-13-1248-8293d73) 2024-09-16T09:20:47.703 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: set uid:gid to 167:167 (ceph:ceph) 2024-09-16T09:20:47.703 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev), process ceph-mon, pid 7 2024-09-16T09:20:47.703 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: pidfile_write: ignore empty --pid-file 2024-09-16T09:20:48.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: load: jerasure load: lrc 2024-09-16T09:20:48.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: RocksDB version: 7.9.2 2024-09-16T09:20:48.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Git sha 0 2024-09-16T09:20:48.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Compile date 2024-09-13 14:50:41 2024-09-16T09:20:48.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: DB SUMMARY 2024-09-16T09:20:48.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: DB Session ID: 4SEX3HD0HZYSUKRBH6II 2024-09-16T09:20:48.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: CURRENT file: CURRENT 2024-09-16T09:20:48.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: IDENTITY file: IDENTITY 2024-09-16T09:20:48.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-16T09:20:48.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-09-16T09:20:48.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 636 ; 2024-09-16T09:20:48.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.error_if_exists: 0 2024-09-16T09:20:48.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.create_if_missing: 0 2024-09-16T09:20:48.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.paranoid_checks: 1 2024-09-16T09:20:48.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-16T09:20:48.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-16T09:20:48.173 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-16T09:20:48.173 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.env: 0x55f44aa3f120 2024-09-16T09:20:48.173 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.fs: PosixFileSystem 2024-09-16T09:20:48.173 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.info_log: 0x55f44c049700 2024-09-16T09:20:48.173 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.max_file_opening_threads: 16 2024-09-16T09:20:48.173 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.statistics: (nil) 2024-09-16T09:20:48.173 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.use_fsync: 0 2024-09-16T09:20:48.173 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.max_log_file_size: 0 2024-09-16T09:20:48.173 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-16T09:20:48.173 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-16T09:20:48.173 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.keep_log_file_num: 1000 2024-09-16T09:20:48.173 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.recycle_log_file_num: 0 2024-09-16T09:20:48.173 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.allow_fallocate: 1 2024-09-16T09:20:48.173 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.allow_mmap_reads: 0 2024-09-16T09:20:48.174 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.allow_mmap_writes: 0 2024-09-16T09:20:48.174 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.use_direct_reads: 0 2024-09-16T09:20:48.174 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-16T09:20:48.174 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.create_missing_column_families: 0 2024-09-16T09:20:48.174 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.db_log_dir: 2024-09-16T09:20:48.174 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.wal_dir: 2024-09-16T09:20:48.174 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-16T09:20:48.174 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-16T09:20:48.174 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-16T09:20:48.174 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-16T09:20:48.174 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-16T09:20:48.174 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-16T09:20:48.174 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.advise_random_on_open: 1 2024-09-16T09:20:48.174 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.db_write_buffer_size: 0 2024-09-16T09:20:48.175 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.write_buffer_manager: 0x55f44c04db80 2024-09-16T09:20:48.175 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-16T09:20:48.175 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-16T09:20:48.175 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-16T09:20:48.175 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.rate_limiter: (nil) 2024-09-16T09:20:48.175 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-16T09:20:48.175 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.wal_recovery_mode: 2 2024-09-16T09:20:48.175 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.enable_thread_tracking: 0 2024-09-16T09:20:48.175 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.enable_pipelined_write: 0 2024-09-16T09:20:48.175 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.unordered_write: 0 2024-09-16T09:20:48.175 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-16T09:20:48.175 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-16T09:20:48.175 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-16T09:20:48.175 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-16T09:20:48.175 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.row_cache: None 2024-09-16T09:20:48.175 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.wal_filter: None 2024-09-16T09:20:48.176 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-16T09:20:48.176 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.allow_ingest_behind: 0 2024-09-16T09:20:48.176 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.two_write_queues: 0 2024-09-16T09:20:48.176 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.manual_wal_flush: 0 2024-09-16T09:20:48.176 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.wal_compression: 0 2024-09-16T09:20:48.176 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.atomic_flush: 0 2024-09-16T09:20:48.176 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-16T09:20:48.176 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-16T09:20:48.176 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-16T09:20:48.176 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.log_readahead_size: 0 2024-09-16T09:20:48.176 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-16T09:20:48.176 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.best_efforts_recovery: 0 2024-09-16T09:20:48.176 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-16T09:20:48.176 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-16T09:20:48.176 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.allow_data_in_errors: 0 2024-09-16T09:20:48.177 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.db_host_id: __hostname__ 2024-09-16T09:20:48.177 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-16T09:20:48.177 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.max_background_jobs: 2 2024-09-16T09:20:48.177 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.max_background_compactions: -1 2024-09-16T09:20:48.177 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.max_subcompactions: 1 2024-09-16T09:20:48.178 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-16T09:20:48.178 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-16T09:20:48.178 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-16T09:20:48.178 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.max_total_wal_size: 0 2024-09-16T09:20:48.178 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-16T09:20:48.179 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-16T09:20:48.179 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-16T09:20:48.179 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-16T09:20:48.179 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.max_open_files: -1 2024-09-16T09:20:48.179 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.bytes_per_sync: 0 2024-09-16T09:20:48.179 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-16T09:20:48.179 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-16T09:20:48.179 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.compaction_readahead_size: 0 2024-09-16T09:20:48.179 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.max_background_flushes: -1 2024-09-16T09:20:48.179 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Compression algorithms supported: 2024-09-16T09:20:48.179 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: kZSTD supported: 0 2024-09-16T09:20:48.179 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: kXpressCompression supported: 0 2024-09-16T09:20:48.179 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: kBZip2Compression supported: 0 2024-09-16T09:20:48.179 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-16T09:20:48.180 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: kLZ4Compression supported: 1 2024-09-16T09:20:48.180 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: kZlibCompression supported: 1 2024-09-16T09:20:48.180 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: kLZ4HCCompression supported: 1 2024-09-16T09:20:48.180 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: kSnappyCompression supported: 1 2024-09-16T09:20:48.180 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-16T09:20:48.180 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-16T09:20:48.180 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-09-16T09:20:48.180 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-16T09:20:48.180 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-16T09:20:48.180 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.merge_operator: 2024-09-16T09:20:48.180 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.compaction_filter: None 2024-09-16T09:20:48.180 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.compaction_filter_factory: None 2024-09-16T09:20:48.181 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.sst_partitioner_factory: None 2024-09-16T09:20:48.181 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-16T09:20:48.181 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-16T09:20:48.181 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55f44c049320) 2024-09-16T09:20:48.181 INFO:journalctl@ceph.mon.c.smithi070.stdout: cache_index_and_filter_blocks: 1 2024-09-16T09:20:48.181 INFO:journalctl@ceph.mon.c.smithi070.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-16T09:20:48.181 INFO:journalctl@ceph.mon.c.smithi070.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-16T09:20:48.181 INFO:journalctl@ceph.mon.c.smithi070.stdout: pin_top_level_index_and_filter: 1 2024-09-16T09:20:48.181 INFO:journalctl@ceph.mon.c.smithi070.stdout: index_type: 0 2024-09-16T09:20:48.181 INFO:journalctl@ceph.mon.c.smithi070.stdout: data_block_index_type: 0 2024-09-16T09:20:48.181 INFO:journalctl@ceph.mon.c.smithi070.stdout: index_shortening: 1 2024-09-16T09:20:48.181 INFO:journalctl@ceph.mon.c.smithi070.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-16T09:20:48.181 INFO:journalctl@ceph.mon.c.smithi070.stdout: checksum: 4 2024-09-16T09:20:48.182 INFO:journalctl@ceph.mon.c.smithi070.stdout: no_block_cache: 0 2024-09-16T09:20:48.182 INFO:journalctl@ceph.mon.c.smithi070.stdout: block_cache: 0x55f44c03d610 2024-09-16T09:20:48.182 INFO:journalctl@ceph.mon.c.smithi070.stdout: block_cache_name: BinnedLRUCache 2024-09-16T09:20:48.182 INFO:journalctl@ceph.mon.c.smithi070.stdout: block_cache_options: 2024-09-16T09:20:48.182 INFO:journalctl@ceph.mon.c.smithi070.stdout: capacity : 536870912 2024-09-16T09:20:48.182 INFO:journalctl@ceph.mon.c.smithi070.stdout: num_shard_bits : 4 2024-09-16T09:20:48.182 INFO:journalctl@ceph.mon.c.smithi070.stdout: strict_capacity_limit : 0 2024-09-16T09:20:48.182 INFO:journalctl@ceph.mon.c.smithi070.stdout: high_pri_pool_ratio: 0.000 2024-09-16T09:20:48.182 INFO:journalctl@ceph.mon.c.smithi070.stdout: block_cache_compressed: (nil) 2024-09-16T09:20:48.182 INFO:journalctl@ceph.mon.c.smithi070.stdout: persistent_cache: (nil) 2024-09-16T09:20:48.182 INFO:journalctl@ceph.mon.c.smithi070.stdout: block_size: 4096 2024-09-16T09:20:48.182 INFO:journalctl@ceph.mon.c.smithi070.stdout: block_size_deviation: 10 2024-09-16T09:20:48.182 INFO:journalctl@ceph.mon.c.smithi070.stdout: block_restart_interval: 16 2024-09-16T09:20:48.182 INFO:journalctl@ceph.mon.c.smithi070.stdout: index_block_restart_interval: 1 2024-09-16T09:20:48.183 INFO:journalctl@ceph.mon.c.smithi070.stdout: metadata_block_size: 4096 2024-09-16T09:20:48.183 INFO:journalctl@ceph.mon.c.smithi070.stdout: partition_filters: 0 2024-09-16T09:20:48.183 INFO:journalctl@ceph.mon.c.smithi070.stdout: use_delta_encoding: 1 2024-09-16T09:20:48.183 INFO:journalctl@ceph.mon.c.smithi070.stdout: filter_policy: bloomfilter 2024-09-16T09:20:48.183 INFO:journalctl@ceph.mon.c.smithi070.stdout: whole_key_filtering: 1 2024-09-16T09:20:48.183 INFO:journalctl@ceph.mon.c.smithi070.stdout: verify_compression: 0 2024-09-16T09:20:48.183 INFO:journalctl@ceph.mon.c.smithi070.stdout: read_amp_bytes_per_bit: 0 2024-09-16T09:20:48.183 INFO:journalctl@ceph.mon.c.smithi070.stdout: format_version: 5 2024-09-16T09:20:48.183 INFO:journalctl@ceph.mon.c.smithi070.stdout: enable_index_compression: 1 2024-09-16T09:20:48.183 INFO:journalctl@ceph.mon.c.smithi070.stdout: block_align: 0 2024-09-16T09:20:48.183 INFO:journalctl@ceph.mon.c.smithi070.stdout: max_auto_readahead_size: 262144 2024-09-16T09:20:48.183 INFO:journalctl@ceph.mon.c.smithi070.stdout: prepopulate_block_cache: 0 2024-09-16T09:20:48.183 INFO:journalctl@ceph.mon.c.smithi070.stdout: initial_auto_readahead_size: 8192 2024-09-16T09:20:48.183 INFO:journalctl@ceph.mon.c.smithi070.stdout: num_file_reads_for_auto_readahead: 2 2024-09-16T09:20:48.184 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.write_buffer_size: 33554432 2024-09-16T09:20:48.184 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.max_write_buffer_number: 2 2024-09-16T09:20:48.184 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.compression: NoCompression 2024-09-16T09:20:48.184 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.bottommost_compression: Disabled 2024-09-16T09:20:48.184 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.prefix_extractor: nullptr 2024-09-16T09:20:48.184 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-16T09:20:48.184 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.num_levels: 7 2024-09-16T09:20:48.184 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-16T09:20:48.184 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-16T09:20:48.184 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-16T09:20:48.184 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-16T09:20:48.184 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-16T09:20:48.184 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-16T09:20:48.184 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-16T09:20:48.185 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-16T09:20:48.185 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-16T09:20:48.185 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-16T09:20:48.185 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-16T09:20:48.185 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-16T09:20:48.185 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-16T09:20:48.185 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.compression_opts.level: 32767 2024-09-16T09:20:48.185 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.compression_opts.strategy: 0 2024-09-16T09:20:48.185 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-16T09:20:48.185 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-16T09:20:48.185 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-16T09:20:48.185 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-16T09:20:48.185 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.compression_opts.enabled: false 2024-09-16T09:20:48.185 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-16T09:20:48.186 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-16T09:20:48.186 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-16T09:20:48.186 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-16T09:20:48.186 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.target_file_size_base: 67108864 2024-09-16T09:20:48.186 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-16T09:20:48.186 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-16T09:20:48.186 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-16T09:20:48.186 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-16T09:20:48.186 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-16T09:20:48.186 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-16T09:20:48.186 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-16T09:20:48.186 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-16T09:20:48.186 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-16T09:20:48.186 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-16T09:20:48.187 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-16T09:20:48.187 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-16T09:20:48.187 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-16T09:20:48.187 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-16T09:20:48.187 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.arena_block_size: 1048576 2024-09-16T09:20:48.187 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-16T09:20:48.187 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-16T09:20:48.187 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.disable_auto_compactions: 0 2024-09-16T09:20:48.187 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-16T09:20:48.187 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-16T09:20:48.187 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-16T09:20:48.187 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-16T09:20:48.187 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-16T09:20:48.187 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-16T09:20:48.188 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-16T09:20:48.188 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-16T09:20:48.188 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-16T09:20:48.188 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-16T09:20:48.188 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-16T09:20:48.188 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.inplace_update_support: 0 2024-09-16T09:20:48.188 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-16T09:20:48.188 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-16T09:20:48.188 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-16T09:20:48.188 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-16T09:20:48.188 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.bloom_locality: 0 2024-09-16T09:20:48.188 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.max_successive_merges: 0 2024-09-16T09:20:48.188 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-16T09:20:48.188 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.paranoid_file_checks: 0 2024-09-16T09:20:48.189 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.force_consistency_checks: 1 2024-09-16T09:20:48.189 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.report_bg_io_stats: 0 2024-09-16T09:20:48.189 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.ttl: 2592000 2024-09-16T09:20:48.189 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-16T09:20:48.189 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-16T09:20:48.189 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-16T09:20:48.189 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.enable_blob_files: false 2024-09-16T09:20:48.189 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.min_blob_size: 0 2024-09-16T09:20:48.189 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.blob_file_size: 268435456 2024-09-16T09:20:48.189 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-16T09:20:48.189 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-16T09:20:48.189 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-16T09:20:48.189 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-16T09:20:48.189 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-16T09:20:48.190 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.blob_file_starting_level: 0 2024-09-16T09:20:48.190 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-16T09:20:48.190 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-16T09:20:48.190 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-16T09:20:48.190 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 905e3a00-01d3-4607-b22d-886aed0b336b 2024-09-16T09:20:48.190 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726478447707243, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-16T09:20:48.190 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-16T09:20:48.190 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726478447707627, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726478447, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "905e3a00-01d3-4607-b22d-886aed0b336b", "db_session_id": "4SEX3HD0HZYSUKRBH6II", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-16T09:20:48.190 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726478447707719, "job": 1, "event": "recovery_finished"} 2024-09-16T09:20:48.190 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-16T09:20:48.190 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-16T09:20:48.190 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55f44c06ce00 2024-09-16T09:20:48.190 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: DB pointer 0x55f44c16e000 2024-09-16T09:20:48.190 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-16T09:20:48.190 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-16T09:20:48.190 INFO:journalctl@ceph.mon.c.smithi070.stdout: ** DB Stats ** 2024-09-16T09:20:48.191 INFO:journalctl@ceph.mon.c.smithi070.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-16T09:20:48.191 INFO:journalctl@ceph.mon.c.smithi070.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-16T09:20:48.191 INFO:journalctl@ceph.mon.c.smithi070.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-16T09:20:48.191 INFO:journalctl@ceph.mon.c.smithi070.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-16T09:20:48.191 INFO:journalctl@ceph.mon.c.smithi070.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-16T09:20:48.191 INFO:journalctl@ceph.mon.c.smithi070.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-16T09:20:48.191 INFO:journalctl@ceph.mon.c.smithi070.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-16T09:20:48.191 INFO:journalctl@ceph.mon.c.smithi070.stdout: 2024-09-16T09:20:48.191 INFO:journalctl@ceph.mon.c.smithi070.stdout: ** Compaction Stats [default] ** 2024-09-16T09:20:48.191 INFO:journalctl@ceph.mon.c.smithi070.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-16T09:20:48.191 INFO:journalctl@ceph.mon.c.smithi070.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-16T09:20:48.191 INFO:journalctl@ceph.mon.c.smithi070.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.6 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-16T09:20:48.191 INFO:journalctl@ceph.mon.c.smithi070.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.6 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-16T09:20:48.191 INFO:journalctl@ceph.mon.c.smithi070.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.6 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-16T09:20:48.191 INFO:journalctl@ceph.mon.c.smithi070.stdout: 2024-09-16T09:20:48.192 INFO:journalctl@ceph.mon.c.smithi070.stdout: ** Compaction Stats [default] ** 2024-09-16T09:20:48.192 INFO:journalctl@ceph.mon.c.smithi070.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-16T09:20:48.192 INFO:journalctl@ceph.mon.c.smithi070.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-16T09:20:48.192 INFO:journalctl@ceph.mon.c.smithi070.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.6 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-16T09:20:48.192 INFO:journalctl@ceph.mon.c.smithi070.stdout: 2024-09-16T09:20:48.192 INFO:journalctl@ceph.mon.c.smithi070.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-16T09:20:48.192 INFO:journalctl@ceph.mon.c.smithi070.stdout: 2024-09-16T09:20:48.192 INFO:journalctl@ceph.mon.c.smithi070.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-16T09:20:48.192 INFO:journalctl@ceph.mon.c.smithi070.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-16T09:20:48.192 INFO:journalctl@ceph.mon.c.smithi070.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-16T09:20:48.192 INFO:journalctl@ceph.mon.c.smithi070.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-16T09:20:48.192 INFO:journalctl@ceph.mon.c.smithi070.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-16T09:20:48.192 INFO:journalctl@ceph.mon.c.smithi070.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-16T09:20:48.192 INFO:journalctl@ceph.mon.c.smithi070.stdout: Cumulative compaction: 0.00 GB write, 0.67 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-16T09:20:48.192 INFO:journalctl@ceph.mon.c.smithi070.stdout: Interval compaction: 0.00 GB write, 0.67 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-16T09:20:48.193 INFO:journalctl@ceph.mon.c.smithi070.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-16T09:20:48.193 INFO:journalctl@ceph.mon.c.smithi070.stdout: Block cache BinnedLRUCache@0x55f44c03d610#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-16T09:20:48.193 INFO:journalctl@ceph.mon.c.smithi070.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-16T09:20:48.193 INFO:journalctl@ceph.mon.c.smithi070.stdout: 2024-09-16T09:20:48.193 INFO:journalctl@ceph.mon.c.smithi070.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-16T09:20:48.193 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-09-16T09:20:48.193 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: using public_addrv [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] 2024-09-16T09:20:48.193 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: starting mon.c rank -1 at public addrs [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] at bind addrs [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] mon_data /var/lib/ceph/mon/ceph-c fsid a1321834-740c-11ef-bceb-c7b262605968 2024-09-16T09:20:48.193 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: mon.c@-1(???) e0 preinit fsid a1321834-740c-11ef-bceb-c7b262605968 2024-09-16T09:20:48.193 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: mon.c@-1(synchronizing).mds e1 new map 2024-09-16T09:20:48.193 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: mon.c@-1(synchronizing).mds e1 print_map 2024-09-16T09:20:48.193 INFO:journalctl@ceph.mon.c.smithi070.stdout: e1 2024-09-16T09:20:48.193 INFO:journalctl@ceph.mon.c.smithi070.stdout: btime 2024-09-16T09:19:14:121324+0000 2024-09-16T09:20:48.193 INFO:journalctl@ceph.mon.c.smithi070.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T09:20:48.193 INFO:journalctl@ceph.mon.c.smithi070.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-16T09:20:48.194 INFO:journalctl@ceph.mon.c.smithi070.stdout: legacy client fscid: -1 2024-09-16T09:20:48.194 INFO:journalctl@ceph.mon.c.smithi070.stdout: 2024-09-16T09:20:48.194 INFO:journalctl@ceph.mon.c.smithi070.stdout: No filesystems configured 2024-09-16T09:20:48.194 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-16T09:20:48.194 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-16T09:20:48.194 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-16T09:20:48.194 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-16T09:20:48.194 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-16T09:20:48.194 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-16T09:20:48.194 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-16T09:20:48.194 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-16T09:20:48.194 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-16T09:20:48.194 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-16T09:20:48.194 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi070:172.21.15.70=a;smithi070:[v2:172.21.15.70:3301,v1:172.21.15.70:6790]=c;smithi084:172.21.15.84=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:20:48.194 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: Saving service mon spec with placement smithi070:172.21.15.70=a;smithi070:[v2:172.21.15.70:3301,v1:172.21.15.70:6790]=c;smithi084:172.21.15.84=b;count:3 2024-09-16T09:20:48.195 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: Deploying daemon mon.b on smithi084 2024-09-16T09:20:48.195 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T09:20:48.195 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: from='client.? 172.21.15.84:0/30804907' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-16T09:20:48.195 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T09:20:48.195 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T09:20:48.195 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-16T09:20:48.195 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: mon.a calling monitor election 2024-09-16T09:20:48.195 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-16T09:20:48.195 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-16T09:20:48.195 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T09:20:48.195 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-16T09:20:48.195 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: mon.b calling monitor election 2024-09-16T09:20:48.195 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-16T09:20:48.195 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T09:20:48.195 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-16T09:20:48.196 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-16T09:20:48.196 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-16T09:20:48.196 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: monmap epoch 2 2024-09-16T09:20:48.196 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: fsid a1321834-740c-11ef-bceb-c7b262605968 2024-09-16T09:20:48.196 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: last_changed 2024-09-16T09:20:39.381677+0000 2024-09-16T09:20:48.196 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: created 2024-09-16T09:19:10.966128+0000 2024-09-16T09:20:48.196 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: min_mon_release 19 (squid) 2024-09-16T09:20:48.196 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: election_strategy: 1 2024-09-16T09:20:48.196 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: 0: [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] mon.a 2024-09-16T09:20:48.196 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: 1: [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] mon.b 2024-09-16T09:20:48.196 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: fsmap 2024-09-16T09:20:48.196 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T09:20:48.196 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: mgrmap e13: y(active, since 35s) 2024-09-16T09:20:48.196 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: overall HEALTH_OK 2024-09-16T09:20:48.196 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:48.197 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:48.197 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:48.197 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-16T09:20:48.197 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:20:48.197 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: Deploying daemon mon.c on smithi070 2024-09-16T09:20:48.197 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: from='client.? 172.21.15.84:0/3669468361' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-16T09:20:48.197 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-16T09:20:48.197 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T09:20:48.197 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 ceph-mon[40691]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-16T09:20:48.197 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 bash[40653]: a5f3710eb38e9621e1b045feda0b4e0d907adddffa69804b05c48e9b8a4f6075 2024-09-16T09:20:48.197 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:47 smithi070 systemd[1]: Started Ceph mon.c for a1321834-740c-11ef-bceb-c7b262605968. 2024-09-16T09:20:49.100 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-16T09:20:49.100 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-16T09:20:49.351 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:20:53.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:52 smithi084 ceph-mon[32629]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T09:20:53.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:52 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-16T09:20:53.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:52 smithi084 ceph-mon[32629]: mon.a calling monitor election 2024-09-16T09:20:53.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:52 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-16T09:20:53.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:52 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-16T09:20:53.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:52 smithi084 ceph-mon[32629]: mon.b calling monitor election 2024-09-16T09:20:53.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:52 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-16T09:20:53.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:52 smithi084 ceph-mon[32629]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T09:20:53.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:52 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-16T09:20:53.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:52 smithi084 ceph-mon[32629]: mon.c calling monitor election 2024-09-16T09:20:53.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:52 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-16T09:20:53.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:52 smithi084 ceph-mon[32629]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T09:20:53.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:52 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-16T09:20:53.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:52 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-16T09:20:53.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:52 smithi084 ceph-mon[32629]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-16T09:20:53.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:52 smithi084 ceph-mon[32629]: monmap epoch 3 2024-09-16T09:20:53.162 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:52 smithi084 ceph-mon[32629]: fsid a1321834-740c-11ef-bceb-c7b262605968 2024-09-16T09:20:53.162 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:52 smithi084 ceph-mon[32629]: last_changed 2024-09-16T09:20:47.730044+0000 2024-09-16T09:20:53.162 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:52 smithi084 ceph-mon[32629]: created 2024-09-16T09:19:10.966128+0000 2024-09-16T09:20:53.162 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:52 smithi084 ceph-mon[32629]: min_mon_release 19 (squid) 2024-09-16T09:20:53.162 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:52 smithi084 ceph-mon[32629]: election_strategy: 1 2024-09-16T09:20:53.162 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:52 smithi084 ceph-mon[32629]: 0: [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] mon.a 2024-09-16T09:20:53.162 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:52 smithi084 ceph-mon[32629]: 1: [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] mon.b 2024-09-16T09:20:53.162 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:52 smithi084 ceph-mon[32629]: 2: [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] mon.c 2024-09-16T09:20:53.162 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:52 smithi084 ceph-mon[32629]: fsmap 2024-09-16T09:20:53.162 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:52 smithi084 ceph-mon[32629]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T09:20:53.162 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:52 smithi084 ceph-mon[32629]: mgrmap e13: y(active, since 43s) 2024-09-16T09:20:53.162 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:52 smithi084 ceph-mon[32629]: overall HEALTH_OK 2024-09-16T09:20:53.162 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:52 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:53.162 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:52 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:53.163 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:52 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:53.163 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:52 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:53.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:52 smithi070 ceph-mon[30388]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T09:20:53.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:52 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-16T09:20:53.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:52 smithi070 ceph-mon[30388]: mon.a calling monitor election 2024-09-16T09:20:53.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:52 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-16T09:20:53.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:52 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-16T09:20:53.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:52 smithi070 ceph-mon[30388]: mon.b calling monitor election 2024-09-16T09:20:53.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:52 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-16T09:20:53.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:52 smithi070 ceph-mon[30388]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T09:20:53.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:52 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-16T09:20:53.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:52 smithi070 ceph-mon[30388]: mon.c calling monitor election 2024-09-16T09:20:53.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:52 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-16T09:20:53.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:52 smithi070 ceph-mon[30388]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T09:20:53.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:52 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-16T09:20:53.172 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:52 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-16T09:20:53.172 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:52 smithi070 ceph-mon[30388]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-16T09:20:53.172 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:52 smithi070 ceph-mon[30388]: monmap epoch 3 2024-09-16T09:20:53.172 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:52 smithi070 ceph-mon[30388]: fsid a1321834-740c-11ef-bceb-c7b262605968 2024-09-16T09:20:53.172 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:52 smithi070 ceph-mon[30388]: last_changed 2024-09-16T09:20:47.730044+0000 2024-09-16T09:20:53.172 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:52 smithi070 ceph-mon[30388]: created 2024-09-16T09:19:10.966128+0000 2024-09-16T09:20:53.172 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:52 smithi070 ceph-mon[30388]: min_mon_release 19 (squid) 2024-09-16T09:20:53.172 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:52 smithi070 ceph-mon[30388]: election_strategy: 1 2024-09-16T09:20:53.172 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:52 smithi070 ceph-mon[30388]: 0: [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] mon.a 2024-09-16T09:20:53.172 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:52 smithi070 ceph-mon[30388]: 1: [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] mon.b 2024-09-16T09:20:53.172 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:52 smithi070 ceph-mon[30388]: 2: [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] mon.c 2024-09-16T09:20:53.172 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:52 smithi070 ceph-mon[30388]: fsmap 2024-09-16T09:20:53.172 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:52 smithi070 ceph-mon[30388]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T09:20:53.172 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:52 smithi070 ceph-mon[30388]: mgrmap e13: y(active, since 43s) 2024-09-16T09:20:53.173 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:52 smithi070 ceph-mon[30388]: overall HEALTH_OK 2024-09-16T09:20:53.173 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:52 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:53.173 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:52 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:53.173 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:52 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:53.173 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:52 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:53.173 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:52 smithi070 ceph-mon[40691]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T09:20:53.173 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:52 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-16T09:20:53.173 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:52 smithi070 ceph-mon[40691]: mon.a calling monitor election 2024-09-16T09:20:53.173 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:52 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-16T09:20:53.173 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:52 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-16T09:20:53.173 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:52 smithi070 ceph-mon[40691]: mon.b calling monitor election 2024-09-16T09:20:53.173 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:52 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-16T09:20:53.174 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:52 smithi070 ceph-mon[40691]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T09:20:53.174 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:52 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-16T09:20:53.174 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:52 smithi070 ceph-mon[40691]: mon.c calling monitor election 2024-09-16T09:20:53.174 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:52 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-16T09:20:53.174 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:52 smithi070 ceph-mon[40691]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T09:20:53.174 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:52 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-16T09:20:53.174 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:52 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-16T09:20:53.174 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:52 smithi070 ceph-mon[40691]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-16T09:20:53.174 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:52 smithi070 ceph-mon[40691]: monmap epoch 3 2024-09-16T09:20:53.174 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:52 smithi070 ceph-mon[40691]: fsid a1321834-740c-11ef-bceb-c7b262605968 2024-09-16T09:20:53.174 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:52 smithi070 ceph-mon[40691]: last_changed 2024-09-16T09:20:47.730044+0000 2024-09-16T09:20:53.174 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:52 smithi070 ceph-mon[40691]: created 2024-09-16T09:19:10.966128+0000 2024-09-16T09:20:53.174 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:52 smithi070 ceph-mon[40691]: min_mon_release 19 (squid) 2024-09-16T09:20:53.174 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:52 smithi070 ceph-mon[40691]: election_strategy: 1 2024-09-16T09:20:53.174 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:52 smithi070 ceph-mon[40691]: 0: [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] mon.a 2024-09-16T09:20:53.175 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:52 smithi070 ceph-mon[40691]: 1: [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] mon.b 2024-09-16T09:20:53.175 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:52 smithi070 ceph-mon[40691]: 2: [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] mon.c 2024-09-16T09:20:53.175 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:52 smithi070 ceph-mon[40691]: fsmap 2024-09-16T09:20:53.175 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:52 smithi070 ceph-mon[40691]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T09:20:53.175 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:52 smithi070 ceph-mon[40691]: mgrmap e13: y(active, since 43s) 2024-09-16T09:20:53.175 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:52 smithi070 ceph-mon[40691]: overall HEALTH_OK 2024-09-16T09:20:53.175 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:52 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:53.175 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:52 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:53.175 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:52 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:53.175 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:52 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:53.188 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-16T09:20:53.188 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":3,"fsid":"a1321834-740c-11ef-bceb-c7b262605968","modified":"2024-09-16T09:20:47.730044Z","created":"2024-09-16T09:19:10.966128Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:3300","nonce":0},{"type":"v1","addr":"172.21.15.70:6789","nonce":0}]},"addr":"172.21.15.70:6789/0","public_addr":"172.21.15.70:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:3300","nonce":0},{"type":"v1","addr":"172.21.15.84:6789","nonce":0}]},"addr":"172.21.15.84:6789/0","public_addr":"172.21.15.84:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:3301","nonce":0},{"type":"v1","addr":"172.21.15.70:6790","nonce":0}]},"addr":"172.21.15.70:6790/0","public_addr":"172.21.15.70:6790/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-09-16T09:20:53.188 INFO:teuthology.orchestra.run.smithi084.stderr:dumped monmap epoch 3 2024-09-16T09:20:53.978 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:53 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:20:53.978 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:53 smithi084 ceph-mon[32629]: from='client.? 172.21.15.84:0/2594131312' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-16T09:20:53.978 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:53 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-16T09:20:54.006 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-16T09:20:54.006 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph config generate-minimal-conf 2024-09-16T09:20:54.170 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:53 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:20:54.170 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:53 smithi070 ceph-mon[30388]: from='client.? 172.21.15.84:0/2594131312' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-16T09:20:54.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:53 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-16T09:20:54.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:53 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:20:54.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:53 smithi070 ceph-mon[40691]: from='client.? 172.21.15.84:0/2594131312' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-16T09:20:54.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:53 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-16T09:20:54.245 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:20:54.951 INFO:teuthology.orchestra.run.smithi070.stdout:# minimal ceph.conf for a1321834-740c-11ef-bceb-c7b262605968 2024-09-16T09:20:54.951 INFO:teuthology.orchestra.run.smithi070.stdout:[global] 2024-09-16T09:20:54.951 INFO:teuthology.orchestra.run.smithi070.stdout: fsid = a1321834-740c-11ef-bceb-c7b262605968 2024-09-16T09:20:54.951 INFO:teuthology.orchestra.run.smithi070.stdout: mon_host = [v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0] [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] [v2:172.21.15.70:3301/0,v1:172.21.15.70:6790/0] 2024-09-16T09:20:55.254 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:54 smithi070 ceph-mon[40691]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T09:20:55.254 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:54 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:55.254 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:54 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:55.254 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:54 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:55.254 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:54 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:55.254 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:54 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:20:55.254 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:54 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:20:55.254 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:54 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:55.254 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:54 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:55.254 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:54 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:55.255 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:54 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:55.255 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:54 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:55.255 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:54 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:55.255 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:54 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:55.255 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:54 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:55.255 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:54 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:55.255 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:54 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:55.255 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:54 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-16T09:20:55.255 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:54 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-16T09:20:55.255 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:54 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:20:55.255 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:54 smithi070 ceph-mon[30388]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T09:20:55.255 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:54 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:55.256 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:54 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:55.256 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:54 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:55.256 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:54 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:55.256 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:54 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:20:55.256 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:54 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:20:55.256 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:54 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:55.256 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:54 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:55.256 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:54 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:55.256 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:54 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:55.256 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:54 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:55.256 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:54 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:55.256 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:54 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:55.256 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:54 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:55.257 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:54 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:55.257 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:54 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:55.257 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:54 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-16T09:20:55.257 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:54 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-16T09:20:55.257 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:54 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:20:55.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:54 smithi084 ceph-mon[32629]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T09:20:55.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:54 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:55.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:54 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:55.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:54 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:55.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:54 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:55.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:54 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:20:55.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:54 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:20:55.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:54 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:55.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:54 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:55.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:54 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:55.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:54 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:55.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:54 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:55.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:54 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:55.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:54 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:55.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:54 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:55.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:54 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:55.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:54 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:55.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:54 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-16T09:20:55.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:54 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-16T09:20:55.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:54 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:20:55.609 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-16T09:20:55.609 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-16T09:20:55.609 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/etc/ceph/ceph.conf 2024-09-16T09:20:55.637 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-16T09:20:55.637 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-16T09:20:55.703 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-09-16T09:20:55.704 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/etc/ceph/ceph.conf 2024-09-16T09:20:55.732 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-09-16T09:20:55.732 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-16T09:20:55.798 INFO:tasks.cephadm:Adding mgr.y on smithi070 2024-09-16T09:20:55.799 INFO:tasks.cephadm:Adding mgr.x on smithi084 2024-09-16T09:20:55.799 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph orch apply mgr '2;smithi070=y;smithi084=x' 2024-09-16T09:20:56.077 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:20:56.141 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:55 smithi070 ceph-mon[40691]: Updating smithi070:/etc/ceph/ceph.conf 2024-09-16T09:20:56.141 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:55 smithi070 ceph-mon[40691]: Updating smithi084:/etc/ceph/ceph.conf 2024-09-16T09:20:56.141 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:55 smithi070 ceph-mon[40691]: Updating smithi084:/var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T09:20:56.141 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:55 smithi070 ceph-mon[40691]: Updating smithi070:/var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T09:20:56.141 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:55 smithi070 ceph-mon[40691]: Reconfiguring mon.a (unknown last config time)... 2024-09-16T09:20:56.142 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:55 smithi070 ceph-mon[40691]: Reconfiguring daemon mon.a on smithi070 2024-09-16T09:20:56.142 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:55 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1277878565' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:20:56.142 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:55 smithi070 ceph-mon[30388]: Updating smithi070:/etc/ceph/ceph.conf 2024-09-16T09:20:56.142 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:55 smithi070 ceph-mon[30388]: Updating smithi084:/etc/ceph/ceph.conf 2024-09-16T09:20:56.142 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:55 smithi070 ceph-mon[30388]: Updating smithi084:/var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T09:20:56.142 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:55 smithi070 ceph-mon[30388]: Updating smithi070:/var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T09:20:56.142 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:55 smithi070 ceph-mon[30388]: Reconfiguring mon.a (unknown last config time)... 2024-09-16T09:20:56.142 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:55 smithi070 ceph-mon[30388]: Reconfiguring daemon mon.a on smithi070 2024-09-16T09:20:56.142 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:55 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1277878565' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:20:56.198 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:55 smithi084 ceph-mon[32629]: Updating smithi070:/etc/ceph/ceph.conf 2024-09-16T09:20:56.198 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:55 smithi084 ceph-mon[32629]: Updating smithi084:/etc/ceph/ceph.conf 2024-09-16T09:20:56.198 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:55 smithi084 ceph-mon[32629]: Updating smithi084:/var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T09:20:56.199 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:55 smithi084 ceph-mon[32629]: Updating smithi070:/var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T09:20:56.199 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:55 smithi084 ceph-mon[32629]: Reconfiguring mon.a (unknown last config time)... 2024-09-16T09:20:56.199 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:55 smithi084 ceph-mon[32629]: Reconfiguring daemon mon.a on smithi070 2024-09-16T09:20:56.199 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:55 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1277878565' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:20:56.698 INFO:teuthology.orchestra.run.smithi084.stdout:Scheduled mgr update... 2024-09-16T09:20:57.166 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:56 smithi070 ceph-mon[30388]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T09:20:57.166 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:56 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:57.167 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:56 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:57.167 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:56 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:57.167 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:56 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-16T09:20:57.167 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:56 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "mgr services"} : dispatch 2024-09-16T09:20:57.167 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:56 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:20:57.167 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:56 smithi070 ceph-mon[40691]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T09:20:57.167 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:56 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:57.167 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:56 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:57.167 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:56 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:57.167 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:56 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-16T09:20:57.167 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:56 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "mgr services"} : dispatch 2024-09-16T09:20:57.167 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:56 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:20:57.253 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:56 smithi084 ceph-mon[32629]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T09:20:57.253 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:56 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:57.253 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:56 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:57.253 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:56 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:57.253 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:56 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-16T09:20:57.253 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:56 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "mgr services"} : dispatch 2024-09-16T09:20:57.254 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:56 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:20:57.280 DEBUG:teuthology.orchestra.run.smithi084:mgr.x> sudo journalctl -f -n 0 -u ceph-a1321834-740c-11ef-bceb-c7b262605968@mgr.x.service 2024-09-16T09:20:57.282 DEBUG:tasks.cephadm:set 0 configs 2024-09-16T09:20:57.283 INFO:tasks.cephadm:Deploying OSDs... 2024-09-16T09:20:57.283 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-16T09:20:57.283 DEBUG:teuthology.orchestra.run.smithi070:> dd if=/scratch_devs of=/dev/stdout 2024-09-16T09:20:57.300 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-16T09:20:57.300 DEBUG:teuthology.orchestra.run.smithi070:> stat /dev/vg_nvme/lv_1 2024-09-16T09:20:57.356 INFO:teuthology.orchestra.run.smithi070.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-16T09:20:57.356 INFO:teuthology.orchestra.run.smithi070.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T09:20:57.356 INFO:teuthology.orchestra.run.smithi070.stdout:Device: 5h/5d Inode: 968 Links: 1 2024-09-16T09:20:57.356 INFO:teuthology.orchestra.run.smithi070.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T09:20:57.356 INFO:teuthology.orchestra.run.smithi070.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T09:20:57.357 INFO:teuthology.orchestra.run.smithi070.stdout:Access: 2024-09-16 09:20:47.168637537 +0000 2024-09-16T09:20:57.357 INFO:teuthology.orchestra.run.smithi070.stdout:Modify: 2024-09-16 09:16:08.941012291 +0000 2024-09-16T09:20:57.357 INFO:teuthology.orchestra.run.smithi070.stdout:Change: 2024-09-16 09:16:08.941012291 +0000 2024-09-16T09:20:57.357 INFO:teuthology.orchestra.run.smithi070.stdout: Birth: 2024-09-16 09:16:08.941012291 +0000 2024-09-16T09:20:57.357 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-16T09:20:57.420 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records in 2024-09-16T09:20:57.420 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records out 2024-09-16T09:20:57.420 INFO:teuthology.orchestra.run.smithi070.stderr:512 bytes copied, 0.000114468 s, 4.5 MB/s 2024-09-16T09:20:57.421 DEBUG:teuthology.orchestra.run.smithi070:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-16T09:20:57.478 DEBUG:teuthology.orchestra.run.smithi070:> stat /dev/vg_nvme/lv_2 2024-09-16T09:20:57.534 INFO:teuthology.orchestra.run.smithi070.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-16T09:20:57.534 INFO:teuthology.orchestra.run.smithi070.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T09:20:57.534 INFO:teuthology.orchestra.run.smithi070.stdout:Device: 5h/5d Inode: 961 Links: 1 2024-09-16T09:20:57.534 INFO:teuthology.orchestra.run.smithi070.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T09:20:57.534 INFO:teuthology.orchestra.run.smithi070.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T09:20:57.534 INFO:teuthology.orchestra.run.smithi070.stdout:Access: 2024-09-16 09:20:47.168637537 +0000 2024-09-16T09:20:57.534 INFO:teuthology.orchestra.run.smithi070.stdout:Modify: 2024-09-16 09:16:08.941012291 +0000 2024-09-16T09:20:57.534 INFO:teuthology.orchestra.run.smithi070.stdout:Change: 2024-09-16 09:16:08.941012291 +0000 2024-09-16T09:20:57.534 INFO:teuthology.orchestra.run.smithi070.stdout: Birth: 2024-09-16 09:16:08.941012291 +0000 2024-09-16T09:20:57.535 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-16T09:20:57.598 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records in 2024-09-16T09:20:57.598 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records out 2024-09-16T09:20:57.598 INFO:teuthology.orchestra.run.smithi070.stderr:512 bytes copied, 0.000111339 s, 4.6 MB/s 2024-09-16T09:20:57.599 DEBUG:teuthology.orchestra.run.smithi070:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-16T09:20:57.656 DEBUG:teuthology.orchestra.run.smithi070:> stat /dev/vg_nvme/lv_3 2024-09-16T09:20:57.710 INFO:teuthology.orchestra.run.smithi070.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-16T09:20:57.711 INFO:teuthology.orchestra.run.smithi070.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T09:20:57.711 INFO:teuthology.orchestra.run.smithi070.stdout:Device: 5h/5d Inode: 970 Links: 1 2024-09-16T09:20:57.711 INFO:teuthology.orchestra.run.smithi070.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T09:20:57.711 INFO:teuthology.orchestra.run.smithi070.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T09:20:57.711 INFO:teuthology.orchestra.run.smithi070.stdout:Access: 2024-09-16 09:20:47.169637507 +0000 2024-09-16T09:20:57.711 INFO:teuthology.orchestra.run.smithi070.stdout:Modify: 2024-09-16 09:16:08.941012291 +0000 2024-09-16T09:20:57.711 INFO:teuthology.orchestra.run.smithi070.stdout:Change: 2024-09-16 09:16:08.941012291 +0000 2024-09-16T09:20:57.711 INFO:teuthology.orchestra.run.smithi070.stdout: Birth: 2024-09-16 09:16:08.941012291 +0000 2024-09-16T09:20:57.711 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-16T09:20:57.774 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records in 2024-09-16T09:20:57.774 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records out 2024-09-16T09:20:57.774 INFO:teuthology.orchestra.run.smithi070.stderr:512 bytes copied, 9.1101e-05 s, 5.6 MB/s 2024-09-16T09:20:57.776 DEBUG:teuthology.orchestra.run.smithi070:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-16T09:20:57.832 DEBUG:teuthology.orchestra.run.smithi070:> stat /dev/vg_nvme/lv_4 2024-09-16T09:20:57.887 INFO:teuthology.orchestra.run.smithi070.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-16T09:20:57.888 INFO:teuthology.orchestra.run.smithi070.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T09:20:57.888 INFO:teuthology.orchestra.run.smithi070.stdout:Device: 5h/5d Inode: 977 Links: 1 2024-09-16T09:20:57.888 INFO:teuthology.orchestra.run.smithi070.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T09:20:57.888 INFO:teuthology.orchestra.run.smithi070.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T09:20:57.888 INFO:teuthology.orchestra.run.smithi070.stdout:Access: 2024-09-16 09:20:47.169637507 +0000 2024-09-16T09:20:57.888 INFO:teuthology.orchestra.run.smithi070.stdout:Modify: 2024-09-16 09:16:08.941012291 +0000 2024-09-16T09:20:57.888 INFO:teuthology.orchestra.run.smithi070.stdout:Change: 2024-09-16 09:16:08.941012291 +0000 2024-09-16T09:20:57.888 INFO:teuthology.orchestra.run.smithi070.stdout: Birth: 2024-09-16 09:16:08.941012291 +0000 2024-09-16T09:20:57.888 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-16T09:20:57.952 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records in 2024-09-16T09:20:57.952 INFO:teuthology.orchestra.run.smithi070.stderr:1+0 records out 2024-09-16T09:20:57.952 INFO:teuthology.orchestra.run.smithi070.stderr:512 bytes copied, 8.6012e-05 s, 6.0 MB/s 2024-09-16T09:20:57.953 DEBUG:teuthology.orchestra.run.smithi070:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-16T09:20:58.009 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-09-16T09:20:58.010 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/scratch_devs of=/dev/stdout 2024-09-16T09:20:58.026 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-16T09:20:58.026 DEBUG:teuthology.orchestra.run.smithi084:> stat /dev/vg_nvme/lv_1 2024-09-16T09:20:58.081 INFO:teuthology.orchestra.run.smithi084.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-16T09:20:58.082 INFO:teuthology.orchestra.run.smithi084.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T09:20:58.082 INFO:teuthology.orchestra.run.smithi084.stdout:Device: 5h/5d Inode: 971 Links: 1 2024-09-16T09:20:58.082 INFO:teuthology.orchestra.run.smithi084.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T09:20:58.082 INFO:teuthology.orchestra.run.smithi084.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T09:20:58.082 INFO:teuthology.orchestra.run.smithi084.stdout:Access: 2024-09-16 09:20:38.834817827 +0000 2024-09-16T09:20:58.082 INFO:teuthology.orchestra.run.smithi084.stdout:Modify: 2024-09-16 09:16:07.537441145 +0000 2024-09-16T09:20:58.082 INFO:teuthology.orchestra.run.smithi084.stdout:Change: 2024-09-16 09:16:07.538441128 +0000 2024-09-16T09:20:58.082 INFO:teuthology.orchestra.run.smithi084.stdout: Birth: 2024-09-16 09:16:07.537441145 +0000 2024-09-16T09:20:58.082 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-16T09:20:58.146 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records in 2024-09-16T09:20:58.146 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records out 2024-09-16T09:20:58.147 INFO:teuthology.orchestra.run.smithi084.stderr:512 bytes copied, 0.000219937 s, 2.3 MB/s 2024-09-16T09:20:58.147 DEBUG:teuthology.orchestra.run.smithi084:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-16T09:20:58.170 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:57 smithi070 ceph-mon[30388]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi070=y;smithi084=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:20:58.170 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:57 smithi070 ceph-mon[30388]: Saving service mgr spec with placement smithi070=y;smithi084=x;count:2 2024-09-16T09:20:58.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:57 smithi070 ceph-mon[30388]: Reconfiguring mgr.y (unknown last config time)... 2024-09-16T09:20:58.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:57 smithi070 ceph-mon[30388]: Reconfiguring daemon mgr.y on smithi070 2024-09-16T09:20:58.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:57 smithi070 ceph-mon[40691]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi070=y;smithi084=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:20:58.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:57 smithi070 ceph-mon[40691]: Saving service mgr spec with placement smithi070=y;smithi084=x;count:2 2024-09-16T09:20:58.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:57 smithi070 ceph-mon[40691]: Reconfiguring mgr.y (unknown last config time)... 2024-09-16T09:20:58.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:57 smithi070 ceph-mon[40691]: Reconfiguring daemon mgr.y on smithi070 2024-09-16T09:20:58.204 DEBUG:teuthology.orchestra.run.smithi084:> stat /dev/vg_nvme/lv_2 2024-09-16T09:20:58.261 INFO:teuthology.orchestra.run.smithi084.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-16T09:20:58.261 INFO:teuthology.orchestra.run.smithi084.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T09:20:58.261 INFO:teuthology.orchestra.run.smithi084.stdout:Device: 5h/5d Inode: 982 Links: 1 2024-09-16T09:20:58.261 INFO:teuthology.orchestra.run.smithi084.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T09:20:58.261 INFO:teuthology.orchestra.run.smithi084.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T09:20:58.261 INFO:teuthology.orchestra.run.smithi084.stdout:Access: 2024-09-16 09:20:38.834817827 +0000 2024-09-16T09:20:58.261 INFO:teuthology.orchestra.run.smithi084.stdout:Modify: 2024-09-16 09:16:07.539441111 +0000 2024-09-16T09:20:58.261 INFO:teuthology.orchestra.run.smithi084.stdout:Change: 2024-09-16 09:16:07.539441111 +0000 2024-09-16T09:20:58.262 INFO:teuthology.orchestra.run.smithi084.stdout: Birth: 2024-09-16 09:16:07.539441111 +0000 2024-09-16T09:20:58.262 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-16T09:20:58.322 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:57 smithi084 ceph-mon[32629]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi070=y;smithi084=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:20:58.322 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:57 smithi084 ceph-mon[32629]: Saving service mgr spec with placement smithi070=y;smithi084=x;count:2 2024-09-16T09:20:58.322 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:57 smithi084 ceph-mon[32629]: Reconfiguring mgr.y (unknown last config time)... 2024-09-16T09:20:58.323 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:57 smithi084 ceph-mon[32629]: Reconfiguring daemon mgr.y on smithi070 2024-09-16T09:20:58.325 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records in 2024-09-16T09:20:58.325 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records out 2024-09-16T09:20:58.325 INFO:teuthology.orchestra.run.smithi084.stderr:512 bytes copied, 0.000243973 s, 2.1 MB/s 2024-09-16T09:20:58.326 DEBUG:teuthology.orchestra.run.smithi084:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-16T09:20:58.382 DEBUG:teuthology.orchestra.run.smithi084:> stat /dev/vg_nvme/lv_3 2024-09-16T09:20:58.438 INFO:teuthology.orchestra.run.smithi084.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-16T09:20:58.438 INFO:teuthology.orchestra.run.smithi084.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T09:20:58.438 INFO:teuthology.orchestra.run.smithi084.stdout:Device: 5h/5d Inode: 978 Links: 1 2024-09-16T09:20:58.438 INFO:teuthology.orchestra.run.smithi084.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T09:20:58.438 INFO:teuthology.orchestra.run.smithi084.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T09:20:58.438 INFO:teuthology.orchestra.run.smithi084.stdout:Access: 2024-09-16 09:20:38.835817810 +0000 2024-09-16T09:20:58.438 INFO:teuthology.orchestra.run.smithi084.stdout:Modify: 2024-09-16 09:16:07.538441128 +0000 2024-09-16T09:20:58.438 INFO:teuthology.orchestra.run.smithi084.stdout:Change: 2024-09-16 09:16:07.538441128 +0000 2024-09-16T09:20:58.438 INFO:teuthology.orchestra.run.smithi084.stdout: Birth: 2024-09-16 09:16:07.538441128 +0000 2024-09-16T09:20:58.439 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-16T09:20:58.501 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records in 2024-09-16T09:20:58.501 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records out 2024-09-16T09:20:58.501 INFO:teuthology.orchestra.run.smithi084.stderr:512 bytes copied, 0.000229949 s, 2.2 MB/s 2024-09-16T09:20:58.502 DEBUG:teuthology.orchestra.run.smithi084:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-16T09:20:58.559 DEBUG:teuthology.orchestra.run.smithi084:> stat /dev/vg_nvme/lv_4 2024-09-16T09:20:58.615 INFO:teuthology.orchestra.run.smithi084.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-16T09:20:58.615 INFO:teuthology.orchestra.run.smithi084.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T09:20:58.616 INFO:teuthology.orchestra.run.smithi084.stdout:Device: 5h/5d Inode: 973 Links: 1 2024-09-16T09:20:58.616 INFO:teuthology.orchestra.run.smithi084.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T09:20:58.616 INFO:teuthology.orchestra.run.smithi084.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T09:20:58.616 INFO:teuthology.orchestra.run.smithi084.stdout:Access: 2024-09-16 09:20:38.835817810 +0000 2024-09-16T09:20:58.616 INFO:teuthology.orchestra.run.smithi084.stdout:Modify: 2024-09-16 09:16:07.538441128 +0000 2024-09-16T09:20:58.616 INFO:teuthology.orchestra.run.smithi084.stdout:Change: 2024-09-16 09:16:07.538441128 +0000 2024-09-16T09:20:58.616 INFO:teuthology.orchestra.run.smithi084.stdout: Birth: 2024-09-16 09:16:07.538441128 +0000 2024-09-16T09:20:58.616 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-16T09:20:58.679 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records in 2024-09-16T09:20:58.679 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records out 2024-09-16T09:20:58.680 INFO:teuthology.orchestra.run.smithi084.stderr:512 bytes copied, 0.000293358 s, 1.7 MB/s 2024-09-16T09:20:58.681 DEBUG:teuthology.orchestra.run.smithi084:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-16T09:20:58.737 INFO:tasks.cephadm:Deploying osd.0 on smithi070 with /dev/vg_nvme/lv_4... 2024-09-16T09:20:58.737 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a1321834-740c-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-16T09:20:59.029 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:58 smithi070 ceph-mon[30388]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T09:20:59.029 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:58 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:59.029 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:58 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:59.029 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:58 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-16T09:20:59.029 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:58 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-16T09:20:59.029 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:20:58 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:20:59.030 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:58 smithi070 ceph-mon[40691]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T09:20:59.030 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:58 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:59.030 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:58 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:59.030 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:58 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-16T09:20:59.030 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:58 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-16T09:20:59.030 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:20:58 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:20:59.412 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:58 smithi084 ceph-mon[32629]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T09:20:59.412 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:58 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:59.412 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:58 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:20:59.412 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:58 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-16T09:20:59.413 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:58 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-16T09:20:59.413 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:20:58 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:20:59.729 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:21:00.244 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:00 smithi070 ceph-mon[30388]: Reconfiguring mon.c (monmap changed)... 2024-09-16T09:21:00.244 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:00 smithi070 ceph-mon[30388]: Reconfiguring daemon mon.c on smithi070 2024-09-16T09:21:00.244 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:00 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:21:00.244 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:00 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:21:00.245 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:00 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-16T09:21:00.245 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:00 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-16T09:21:00.245 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:00 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:21:00.245 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:00 smithi070 ceph-mon[40691]: Reconfiguring mon.c (monmap changed)... 2024-09-16T09:21:00.245 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:00 smithi070 ceph-mon[40691]: Reconfiguring daemon mon.c on smithi070 2024-09-16T09:21:00.245 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:00 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:21:00.245 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:00 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:21:00.245 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:00 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-16T09:21:00.245 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:00 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-16T09:21:00.245 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:00 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:21:00.397 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:00 smithi084 ceph-mon[32629]: Reconfiguring mon.c (monmap changed)... 2024-09-16T09:21:00.397 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:00 smithi084 ceph-mon[32629]: Reconfiguring daemon mon.c on smithi070 2024-09-16T09:21:00.397 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:00 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:21:00.397 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:00 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:21:00.397 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:00 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-16T09:21:00.397 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:00 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-16T09:21:00.398 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:00 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:21:01.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:01 smithi084 ceph-mon[32629]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T09:21:01.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:01 smithi084 ceph-mon[32629]: Reconfiguring mon.b (monmap changed)... 2024-09-16T09:21:01.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:01 smithi084 ceph-mon[32629]: Reconfiguring daemon mon.b on smithi084 2024-09-16T09:21:01.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:01 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:21:01.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:01 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:21:01.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:01 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:21:01.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:01 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:21:01.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:01 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:21:01.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:01 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:21:01.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:01 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-16T09:21:01.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:01 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-16T09:21:01.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:01 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "mgr services"} : dispatch 2024-09-16T09:21:01.409 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:01 smithi070 ceph-mon[40691]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T09:21:01.410 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:01 smithi070 ceph-mon[40691]: Reconfiguring mon.b (monmap changed)... 2024-09-16T09:21:01.410 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:01 smithi070 ceph-mon[40691]: Reconfiguring daemon mon.b on smithi084 2024-09-16T09:21:01.410 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:01 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:21:01.410 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:01 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:21:01.410 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:01 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:21:01.410 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:01 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:21:01.410 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:01 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:21:01.410 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:01 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:21:01.410 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:01 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-16T09:21:01.410 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:01 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-16T09:21:01.410 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:01 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "mgr services"} : dispatch 2024-09-16T09:21:01.411 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:01 smithi070 ceph-mon[30388]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T09:21:01.411 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:01 smithi070 ceph-mon[30388]: Reconfiguring mon.b (monmap changed)... 2024-09-16T09:21:01.411 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:01 smithi070 ceph-mon[30388]: Reconfiguring daemon mon.b on smithi084 2024-09-16T09:21:01.411 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:01 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:21:01.411 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:01 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:21:01.411 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:01 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:21:01.411 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:01 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:21:01.411 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:01 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:21:01.411 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:01 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:21:01.411 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:01 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-16T09:21:01.411 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:01 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-16T09:21:01.411 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:01 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "mgr services"} : dispatch 2024-09-16T09:21:02.250 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:02 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:21:02.250 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:02 smithi084 ceph-mon[32629]: Deploying daemon mgr.x on smithi084 2024-09-16T09:21:02.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:02 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:21:02.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:02 smithi070 ceph-mon[30388]: Deploying daemon mgr.x on smithi084 2024-09-16T09:21:02.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:02 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:21:02.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:02 smithi070 ceph-mon[40691]: Deploying daemon mgr.x on smithi084 2024-09-16T09:21:02.424 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:21:02.451 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi070:vg_nvme/lv_4 2024-09-16T09:21:02.693 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:21:02.894 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Sep 16 09:21:02 smithi084 systemd[1]: Starting Ceph mgr.x for a1321834-740c-11ef-bceb-c7b262605968... 2024-09-16T09:21:03.082 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:03 smithi070 ceph-mon[30388]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T09:21:03.082 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:03 smithi070 ceph-mon[40691]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T09:21:03.148 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:03 smithi084 ceph-mon[32629]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T09:21:03.148 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Sep 16 09:21:02 smithi084 podman[34271]: 2024-09-16 09:21:02.795718019 +0000 UTC m=+0.015807217 image pull 425e316ba8ebf74f04aa3640f7deb4fa3f293a78d0c8d097cb679eaf86ddca6c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce 2024-09-16T09:21:03.149 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Sep 16 09:21:02 smithi084 podman[34271]: 2024-09-16 09:21:02.915816366 +0000 UTC m=+0.135905609 container create 6cafed972ec4b1f5f3c274d276f91e52cdaff6ea400b74e50d3eda27c031ab51 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-guits-main-2024-09-13-1248-8293d73, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-16T09:21:03.149 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Sep 16 09:21:03 smithi084 podman[34271]: 2024-09-16 09:21:03.143159619 +0000 UTC m=+0.363248817 container init 6cafed972ec4b1f5f3c274d276f91e52cdaff6ea400b74e50d3eda27c031ab51 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-x, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-16T09:21:03.149 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Sep 16 09:21:03 smithi084 podman[34271]: 2024-09-16 09:21:03.147870615 +0000 UTC m=+0.367959813 container start 6cafed972ec4b1f5f3c274d276f91e52cdaff6ea400b74e50d3eda27c031ab51 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-x, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-09-16T09:21:03.409 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Sep 16 09:21:03 smithi084 ceph-mgr[34309]: -- 172.21.15.84:0/778486949 <== mon.1 v2:172.21.15.84:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55be7acc30e0 con 0x55be7aca1000 2024-09-16T09:21:03.410 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Sep 16 09:21:03 smithi084 bash[34271]: 6cafed972ec4b1f5f3c274d276f91e52cdaff6ea400b74e50d3eda27c031ab51 2024-09-16T09:21:03.410 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Sep 16 09:21:03 smithi084 systemd[1]: Started Ceph mgr.x for a1321834-740c-11ef-bceb-c7b262605968. 2024-09-16T09:21:04.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:04 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:21:04.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:04 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:21:04.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:04 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:21:04.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:04 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:21:04.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:04 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:21:04.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:04 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-16T09:21:04.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:04 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-16T09:21:04.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:04 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:21:04.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:04 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:21:04.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:04 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:21:04.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:04 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:21:04.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:04 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:21:04.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:04 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:21:04.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:04 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-16T09:21:04.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:04 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-16T09:21:04.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:04 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:21:04.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:04 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:21:04.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:04 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:21:04.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:04 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:21:04.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:04 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:21:04.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:04 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:21:04.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:04 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-16T09:21:04.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:04 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-16T09:21:04.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:04 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:21:05.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:05 smithi084 ceph-mon[32629]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T09:21:05.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:05 smithi084 ceph-mon[32629]: from='client.14217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:21:05.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:05 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:21:05.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:05 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:21:05.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:05 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:21:05.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:05 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:21:05.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:05 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:21:05.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:05 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:21:05.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:05 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:21:05.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:05 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:21:05.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:05 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:21:05.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:05 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:21:05.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:05 smithi070 ceph-mon[30388]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T09:21:05.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:05 smithi070 ceph-mon[30388]: from='client.14217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:21:05.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:05 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:21:05.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:05 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:21:05.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:05 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:21:05.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:05 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:21:05.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:05 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:21:05.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:05 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:21:05.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:05 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:21:05.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:05 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:21:05.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:05 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:21:05.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:05 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:21:05.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:05 smithi070 ceph-mon[40691]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T09:21:05.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:05 smithi070 ceph-mon[40691]: from='client.14217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:21:05.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:05 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:21:05.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:05 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:21:05.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:05 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:21:05.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:05 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:21:05.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:05 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:21:05.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:05 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:21:05.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:05 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:21:05.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:05 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:21:05.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:05 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:21:05.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:05 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:21:06.530 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:06 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2607518181' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "7ef288c8-3bf0-4814-8e86-f70a7de041ce"} : dispatch 2024-09-16T09:21:06.531 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:06 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2607518181' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7ef288c8-3bf0-4814-8e86-f70a7de041ce"}]': finished 2024-09-16T09:21:06.531 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:06 smithi070 ceph-mon[30388]: osdmap e5: 1 total, 0 up, 1 in 2024-09-16T09:21:06.531 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:06 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-16T09:21:06.531 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:06 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/342523643' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-16T09:21:06.531 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:06 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/2607518181' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "7ef288c8-3bf0-4814-8e86-f70a7de041ce"} : dispatch 2024-09-16T09:21:06.531 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:06 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/2607518181' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7ef288c8-3bf0-4814-8e86-f70a7de041ce"}]': finished 2024-09-16T09:21:06.531 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:06 smithi070 ceph-mon[40691]: osdmap e5: 1 total, 0 up, 1 in 2024-09-16T09:21:06.531 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:06 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-16T09:21:06.532 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:06 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/342523643' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-16T09:21:06.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:06 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/2607518181' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "7ef288c8-3bf0-4814-8e86-f70a7de041ce"} : dispatch 2024-09-16T09:21:06.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:06 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/2607518181' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7ef288c8-3bf0-4814-8e86-f70a7de041ce"}]': finished 2024-09-16T09:21:06.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:06 smithi084 ceph-mon[32629]: osdmap e5: 1 total, 0 up, 1 in 2024-09-16T09:21:06.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:06 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-16T09:21:06.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:06 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/342523643' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-16T09:21:07.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:07 smithi084 ceph-mon[32629]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T09:21:07.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:07 smithi070 ceph-mon[30388]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T09:21:07.670 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:07 smithi070 ceph-mon[40691]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T09:21:08.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:08 smithi084 ceph-mon[32629]: Standby manager daemon x started 2024-09-16T09:21:08.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:08 smithi084 ceph-mon[32629]: from='mgr.? 172.21.15.84:0/1108793244' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/x/crt"} : dispatch 2024-09-16T09:21:08.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:08 smithi084 ceph-mon[32629]: from='mgr.? 172.21.15.84:0/1108793244' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-16T09:21:08.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:08 smithi084 ceph-mon[32629]: from='mgr.? 172.21.15.84:0/1108793244' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/x/key"} : dispatch 2024-09-16T09:21:08.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:08 smithi084 ceph-mon[32629]: from='mgr.? 172.21.15.84:0/1108793244' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-16T09:21:08.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:08 smithi070 ceph-mon[30388]: Standby manager daemon x started 2024-09-16T09:21:08.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:08 smithi070 ceph-mon[30388]: from='mgr.? 172.21.15.84:0/1108793244' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/x/crt"} : dispatch 2024-09-16T09:21:08.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:08 smithi070 ceph-mon[30388]: from='mgr.? 172.21.15.84:0/1108793244' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-16T09:21:08.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:08 smithi070 ceph-mon[30388]: from='mgr.? 172.21.15.84:0/1108793244' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/x/key"} : dispatch 2024-09-16T09:21:08.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:08 smithi070 ceph-mon[30388]: from='mgr.? 172.21.15.84:0/1108793244' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-16T09:21:08.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:08 smithi070 ceph-mon[40691]: Standby manager daemon x started 2024-09-16T09:21:08.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:08 smithi070 ceph-mon[40691]: from='mgr.? 172.21.15.84:0/1108793244' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/x/crt"} : dispatch 2024-09-16T09:21:08.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:08 smithi070 ceph-mon[40691]: from='mgr.? 172.21.15.84:0/1108793244' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-16T09:21:08.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:08 smithi070 ceph-mon[40691]: from='mgr.? 172.21.15.84:0/1108793244' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/x/key"} : dispatch 2024-09-16T09:21:08.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:08 smithi070 ceph-mon[40691]: from='mgr.? 172.21.15.84:0/1108793244' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-16T09:21:09.518 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:09 smithi070 ceph-mon[30388]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T09:21:09.519 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:09 smithi070 ceph-mon[30388]: mgrmap e14: y(active, since 58s), standbys: x 2024-09-16T09:21:09.519 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:09 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-16T09:21:09.519 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:09 smithi070 ceph-mon[40691]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T09:21:09.519 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:09 smithi070 ceph-mon[40691]: mgrmap e14: y(active, since 58s), standbys: x 2024-09-16T09:21:09.519 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:09 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-16T09:21:09.659 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:09 smithi084 ceph-mon[32629]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T09:21:09.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:09 smithi084 ceph-mon[32629]: mgrmap e14: y(active, since 58s), standbys: x 2024-09-16T09:21:09.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:09 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-16T09:21:10.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:10 smithi070 ceph-mon[30388]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T09:21:10.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:10 smithi070 ceph-mon[40691]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T09:21:10.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:10 smithi084 ceph-mon[32629]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T09:21:12.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:12 smithi070 ceph-mon[30388]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T09:21:12.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:12 smithi070 ceph-mon[40691]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T09:21:12.659 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:12 smithi084 ceph-mon[32629]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T09:21:13.412 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:13 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-16T09:21:13.413 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:13 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:21:13.413 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:13 smithi070 ceph-mon[30388]: Deploying daemon osd.0 on smithi070 2024-09-16T09:21:13.413 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:13 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-16T09:21:13.413 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:13 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:21:13.413 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:13 smithi070 ceph-mon[40691]: Deploying daemon osd.0 on smithi070 2024-09-16T09:21:13.659 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:13 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-16T09:21:13.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:13 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:21:13.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:13 smithi084 ceph-mon[32629]: Deploying daemon osd.0 on smithi070 2024-09-16T09:21:14.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:14 smithi070 ceph-mon[30388]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T09:21:14.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:14 smithi070 ceph-mon[40691]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T09:21:14.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:14 smithi084 ceph-mon[32629]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T09:21:16.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:16 smithi084 ceph-mon[32629]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T09:21:16.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:16 smithi070 ceph-mon[30388]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T09:21:16.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:16 smithi070 ceph-mon[40691]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T09:21:18.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:18 smithi084 ceph-mon[32629]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T09:21:18.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:18 smithi070 ceph-mon[30388]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T09:21:18.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:18 smithi070 ceph-mon[40691]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T09:21:20.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:20 smithi070 ceph-mon[30388]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T09:21:20.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:20 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:21:20.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:20 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:21:20.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:20 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:21:20.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:20 smithi070 ceph-mon[40691]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T09:21:20.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:20 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:21:20.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:20 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:21:20.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:20 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:21:20.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:20 smithi084 ceph-mon[32629]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T09:21:20.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:20 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:21:20.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:20 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:21:20.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:20 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:21:22.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:22 smithi084 ceph-mon[32629]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T09:21:22.670 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:22 smithi070 ceph-mon[40691]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T09:21:22.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:22 smithi070 ceph-mon[30388]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T09:21:24.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:23 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:21:24.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:23 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:21:24.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:23 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:21:24.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:23 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:21:24.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:23 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:21:24.170 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:23 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:21:24.170 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:23 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:21:24.170 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:23 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:21:24.170 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:23 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:21:24.170 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:23 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:21:24.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:23 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:21:24.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:23 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:21:24.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:23 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:21:24.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:23 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:21:24.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:23 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:21:25.296 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:25 smithi084 ceph-mon[32629]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T09:21:25.296 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:25 smithi084 ceph-mon[32629]: from='osd.0 [v2:172.21.15.70:6802/2013075781,v1:172.21.15.70:6803/2013075781]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-16T09:21:25.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:25 smithi070 ceph-mon[30388]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T09:21:25.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:25 smithi070 ceph-mon[30388]: from='osd.0 [v2:172.21.15.70:6802/2013075781,v1:172.21.15.70:6803/2013075781]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-16T09:21:25.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:25 smithi070 ceph-mon[40691]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T09:21:25.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:25 smithi070 ceph-mon[40691]: from='osd.0 [v2:172.21.15.70:6802/2013075781,v1:172.21.15.70:6803/2013075781]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-16T09:21:25.929 INFO:teuthology.orchestra.run.smithi070.stdout:Created osd(s) 0 on host 'smithi070' 2024-09-16T09:21:26.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:26 smithi084 ceph-mon[32629]: from='osd.0 [v2:172.21.15.70:6802/2013075781,v1:172.21.15.70:6803/2013075781]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-16T09:21:26.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:26 smithi084 ceph-mon[32629]: osdmap e6: 1 total, 0 up, 1 in 2024-09-16T09:21:26.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:26 smithi084 ceph-mon[32629]: from='osd.0 [v2:172.21.15.70:6802/2013075781,v1:172.21.15.70:6803/2013075781]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi070", "root=default"]} : dispatch 2024-09-16T09:21:26.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:26 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-16T09:21:26.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:26 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:21:26.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:26 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:21:26.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:26 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:21:26.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:26 smithi084 ceph-mon[32629]: from='osd.0 [v2:172.21.15.70:6802/2013075781,v1:172.21.15.70:6803/2013075781]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]': finished 2024-09-16T09:21:26.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:26 smithi084 ceph-mon[32629]: osdmap e7: 1 total, 0 up, 1 in 2024-09-16T09:21:26.170 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:26 smithi070 ceph-mon[30388]: from='osd.0 [v2:172.21.15.70:6802/2013075781,v1:172.21.15.70:6803/2013075781]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-16T09:21:26.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:26 smithi070 ceph-mon[30388]: osdmap e6: 1 total, 0 up, 1 in 2024-09-16T09:21:26.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:26 smithi070 ceph-mon[30388]: from='osd.0 [v2:172.21.15.70:6802/2013075781,v1:172.21.15.70:6803/2013075781]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi070", "root=default"]} : dispatch 2024-09-16T09:21:26.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:26 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-16T09:21:26.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:26 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:21:26.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:26 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:21:26.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:26 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:21:26.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:26 smithi070 ceph-mon[30388]: from='osd.0 [v2:172.21.15.70:6802/2013075781,v1:172.21.15.70:6803/2013075781]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]': finished 2024-09-16T09:21:26.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:26 smithi070 ceph-mon[30388]: osdmap e7: 1 total, 0 up, 1 in 2024-09-16T09:21:26.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:26 smithi070 ceph-mon[40691]: from='osd.0 [v2:172.21.15.70:6802/2013075781,v1:172.21.15.70:6803/2013075781]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-16T09:21:26.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:26 smithi070 ceph-mon[40691]: osdmap e6: 1 total, 0 up, 1 in 2024-09-16T09:21:26.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:26 smithi070 ceph-mon[40691]: from='osd.0 [v2:172.21.15.70:6802/2013075781,v1:172.21.15.70:6803/2013075781]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi070", "root=default"]} : dispatch 2024-09-16T09:21:26.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:26 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-16T09:21:26.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:26 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:21:26.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:26 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:21:26.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:26 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:21:26.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:26 smithi070 ceph-mon[40691]: from='osd.0 [v2:172.21.15.70:6802/2013075781,v1:172.21.15.70:6803/2013075781]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]': finished 2024-09-16T09:21:26.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:26 smithi070 ceph-mon[40691]: osdmap e7: 1 total, 0 up, 1 in 2024-09-16T09:21:26.555 DEBUG:teuthology.orchestra.run.smithi070:osd.0> sudo journalctl -f -n 0 -u ceph-a1321834-740c-11ef-bceb-c7b262605968@osd.0.service 2024-09-16T09:21:26.557 INFO:tasks.cephadm:Deploying osd.1 on smithi070 with /dev/vg_nvme/lv_3... 2024-09-16T09:21:26.557 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a1321834-740c-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-16T09:21:27.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:27 smithi070 ceph-mon[30388]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T09:21:27.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:27 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-16T09:21:27.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:27 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-16T09:21:27.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:27 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-16T09:21:27.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:27 smithi070 ceph-mon[40691]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T09:21:27.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:27 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-16T09:21:27.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:27 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-16T09:21:27.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:27 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-16T09:21:27.338 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:21:27.409 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:27 smithi084 ceph-mon[32629]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T09:21:27.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:27 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-16T09:21:27.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:27 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-16T09:21:27.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:27 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-16T09:21:28.279 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:28 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:21:28.279 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:28 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:21:28.279 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:28 smithi070 ceph-mon[30388]: osd.0 [v2:172.21.15.70:6802/2013075781,v1:172.21.15.70:6803/2013075781] boot 2024-09-16T09:21:28.279 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:28 smithi070 ceph-mon[30388]: osdmap e8: 1 total, 1 up, 1 in 2024-09-16T09:21:28.279 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:28 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-16T09:21:28.279 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:28 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:21:28.280 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:28 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:21:28.280 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:28 smithi070 ceph-mon[40691]: osd.0 [v2:172.21.15.70:6802/2013075781,v1:172.21.15.70:6803/2013075781] boot 2024-09-16T09:21:28.280 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:28 smithi070 ceph-mon[40691]: osdmap e8: 1 total, 1 up, 1 in 2024-09-16T09:21:28.280 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:28 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-16T09:21:28.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:28 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:21:28.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:28 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:21:28.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:28 smithi084 ceph-mon[32629]: osd.0 [v2:172.21.15.70:6802/2013075781,v1:172.21.15.70:6803/2013075781] boot 2024-09-16T09:21:28.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:28 smithi084 ceph-mon[32629]: osdmap e8: 1 total, 1 up, 1 in 2024-09-16T09:21:28.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:28 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-16T09:21:29.287 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:29 smithi070 ceph-mon[30388]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T09:21:29.287 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:29 smithi070 ceph-mon[30388]: osdmap e9: 1 total, 1 up, 1 in 2024-09-16T09:21:29.287 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:29 smithi070 ceph-mon[40691]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T09:21:29.287 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:29 smithi070 ceph-mon[40691]: osdmap e9: 1 total, 1 up, 1 in 2024-09-16T09:21:29.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:29 smithi084 ceph-mon[32629]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T09:21:29.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:29 smithi084 ceph-mon[32629]: osdmap e9: 1 total, 1 up, 1 in 2024-09-16T09:21:31.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:31 smithi084 ceph-mon[32629]: pgmap v39: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T09:21:31.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:31 smithi084 ceph-mon[32629]: Detected new or changed devices on smithi070 2024-09-16T09:21:31.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:31 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:21:31.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:31 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:21:31.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:31 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-16T09:21:31.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:31 smithi084 ceph-mon[32629]: Adjusting osd_memory_target on smithi070 to 16064M 2024-09-16T09:21:31.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:31 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:21:31.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:31 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:21:31.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:31 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:21:31.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:31 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:21:31.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:31 smithi070 ceph-mon[30388]: pgmap v39: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T09:21:31.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:31 smithi070 ceph-mon[30388]: Detected new or changed devices on smithi070 2024-09-16T09:21:31.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:31 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:21:31.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:31 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:21:31.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:31 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-16T09:21:31.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:31 smithi070 ceph-mon[30388]: Adjusting osd_memory_target on smithi070 to 16064M 2024-09-16T09:21:31.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:31 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:21:31.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:31 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:21:31.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:31 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:21:31.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:31 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:21:31.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:31 smithi070 ceph-mon[40691]: pgmap v39: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T09:21:31.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:31 smithi070 ceph-mon[40691]: Detected new or changed devices on smithi070 2024-09-16T09:21:31.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:31 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:21:31.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:31 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:21:31.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:31 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-16T09:21:31.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:31 smithi070 ceph-mon[40691]: Adjusting osd_memory_target on smithi070 to 16064M 2024-09-16T09:21:31.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:31 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:21:31.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:31 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:21:31.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:31 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:21:31.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:31 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:21:31.716 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:21:31.742 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi070:vg_nvme/lv_3 2024-09-16T09:21:31.984 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:21:33.277 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:33 smithi070 ceph-mon[30388]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T09:21:33.277 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:33 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-16T09:21:33.277 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:33 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-16T09:21:33.277 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:33 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:21:33.277 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:33 smithi070 ceph-mon[40691]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T09:21:33.277 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:33 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-16T09:21:33.278 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:33 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-16T09:21:33.278 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:33 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:21:33.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:33 smithi084 ceph-mon[32629]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T09:21:33.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:33 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-16T09:21:33.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:33 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-16T09:21:33.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:33 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:21:34.292 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:34 smithi070 ceph-mon[30388]: from='client.14250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:21:34.292 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:34 smithi070 ceph-mon[40691]: from='client.14250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:21:34.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:34 smithi084 ceph-mon[32629]: from='client.14250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:21:35.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:35 smithi084 ceph-mon[32629]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T09:21:35.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:35 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1701405327' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "474d41d9-295a-450c-8f27-ce487a14678b"} : dispatch 2024-09-16T09:21:35.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:35 smithi084 ceph-mon[32629]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "474d41d9-295a-450c-8f27-ce487a14678b"} : dispatch 2024-09-16T09:21:35.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:35 smithi084 ceph-mon[32629]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "474d41d9-295a-450c-8f27-ce487a14678b"}]': finished 2024-09-16T09:21:35.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:35 smithi084 ceph-mon[32629]: osdmap e10: 2 total, 1 up, 2 in 2024-09-16T09:21:35.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:35 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-16T09:21:35.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:35 smithi070 ceph-mon[30388]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T09:21:35.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:35 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1701405327' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "474d41d9-295a-450c-8f27-ce487a14678b"} : dispatch 2024-09-16T09:21:35.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:35 smithi070 ceph-mon[30388]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "474d41d9-295a-450c-8f27-ce487a14678b"} : dispatch 2024-09-16T09:21:35.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:35 smithi070 ceph-mon[30388]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "474d41d9-295a-450c-8f27-ce487a14678b"}]': finished 2024-09-16T09:21:35.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:35 smithi070 ceph-mon[30388]: osdmap e10: 2 total, 1 up, 2 in 2024-09-16T09:21:35.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:35 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-16T09:21:35.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:35 smithi070 ceph-mon[40691]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T09:21:35.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:35 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1701405327' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "474d41d9-295a-450c-8f27-ce487a14678b"} : dispatch 2024-09-16T09:21:35.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:35 smithi070 ceph-mon[40691]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "474d41d9-295a-450c-8f27-ce487a14678b"} : dispatch 2024-09-16T09:21:35.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:35 smithi070 ceph-mon[40691]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "474d41d9-295a-450c-8f27-ce487a14678b"}]': finished 2024-09-16T09:21:35.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:35 smithi070 ceph-mon[40691]: osdmap e10: 2 total, 1 up, 2 in 2024-09-16T09:21:35.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:35 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-16T09:21:36.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:36 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3264461775' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-16T09:21:36.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:36 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3264461775' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-16T09:21:36.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:36 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3264461775' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-16T09:21:37.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:37 smithi070 ceph-mon[30388]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T09:21:37.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:37 smithi070 ceph-mon[40691]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T09:21:37.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:37 smithi084 ceph-mon[32629]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T09:21:39.409 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:39 smithi084 ceph-mon[32629]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T09:21:39.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:39 smithi070 ceph-mon[30388]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T09:21:39.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:39 smithi070 ceph-mon[40691]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T09:21:41.378 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:41 smithi070 ceph-mon[30388]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T09:21:41.378 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:41 smithi070 ceph-mon[40691]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T09:21:41.409 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:41 smithi084 ceph-mon[32629]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T09:21:43.238 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:43 smithi070 ceph-mon[30388]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T09:21:43.238 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:43 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-16T09:21:43.238 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:43 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:21:43.238 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:43 smithi070 ceph-mon[40691]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T09:21:43.239 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:43 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-16T09:21:43.239 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:43 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:21:43.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:43 smithi084 ceph-mon[32629]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T09:21:43.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:43 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-16T09:21:43.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:43 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:21:44.170 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:44 smithi070 ceph-mon[30388]: Deploying daemon osd.1 on smithi070 2024-09-16T09:21:44.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:44 smithi070 ceph-mon[40691]: Deploying daemon osd.1 on smithi070 2024-09-16T09:21:44.311 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:44 smithi084 ceph-mon[32629]: Deploying daemon osd.1 on smithi070 2024-09-16T09:21:45.368 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:45 smithi070 ceph-mon[40691]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T09:21:45.369 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:45 smithi070 ceph-mon[30388]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T09:21:45.409 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:45 smithi084 ceph-mon[32629]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T09:21:47.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:47 smithi084 ceph-mon[32629]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T09:21:47.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:47 smithi070 ceph-mon[30388]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T09:21:47.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:47 smithi070 ceph-mon[40691]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T09:21:49.371 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:49 smithi070 ceph-mon[30388]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T09:21:49.371 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:49 smithi070 ceph-mon[40691]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T09:21:49.409 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:49 smithi084 ceph-mon[32629]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T09:21:50.170 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:50 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:21:50.170 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:50 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:21:50.170 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:50 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:21:50.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:50 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:21:50.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:50 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:21:50.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:50 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:21:50.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:50 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:21:50.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:50 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:21:50.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:50 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:21:51.409 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:51 smithi084 ceph-mon[32629]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T09:21:51.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:51 smithi070 ceph-mon[30388]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T09:21:51.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:51 smithi070 ceph-mon[40691]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T09:21:53.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:53 smithi084 ceph-mon[32629]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T09:21:53.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:53 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:21:53.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:53 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:21:53.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:53 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:21:53.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:53 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:21:53.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:53 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:21:53.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:53 smithi070 ceph-mon[30388]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T09:21:53.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:53 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:21:53.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:53 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:21:53.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:53 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:21:53.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:53 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:21:53.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:53 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:21:53.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:53 smithi070 ceph-mon[40691]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T09:21:53.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:53 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:21:53.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:53 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:21:53.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:53 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:21:53.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:53 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:21:53.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:53 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:21:55.217 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:55 smithi070 ceph-mon[30388]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T09:21:55.217 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:55 smithi070 ceph-mon[30388]: from='osd.1 [v2:172.21.15.70:6810/1670012115,v1:172.21.15.70:6811/1670012115]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-16T09:21:55.217 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:55 smithi070 ceph-mon[30388]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-16T09:21:55.218 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:55 smithi070 ceph-mon[40691]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T09:21:55.218 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:55 smithi070 ceph-mon[40691]: from='osd.1 [v2:172.21.15.70:6810/1670012115,v1:172.21.15.70:6811/1670012115]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-16T09:21:55.218 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:55 smithi070 ceph-mon[40691]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-16T09:21:55.257 INFO:teuthology.orchestra.run.smithi070.stdout:Created osd(s) 1 on host 'smithi070' 2024-09-16T09:21:55.409 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:55 smithi084 ceph-mon[32629]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T09:21:55.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:55 smithi084 ceph-mon[32629]: from='osd.1 [v2:172.21.15.70:6810/1670012115,v1:172.21.15.70:6811/1670012115]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-16T09:21:55.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:55 smithi084 ceph-mon[32629]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-16T09:21:55.922 DEBUG:teuthology.orchestra.run.smithi070:osd.1> sudo journalctl -f -n 0 -u ceph-a1321834-740c-11ef-bceb-c7b262605968@osd.1.service 2024-09-16T09:21:55.924 INFO:tasks.cephadm:Deploying osd.2 on smithi070 with /dev/vg_nvme/lv_2... 2024-09-16T09:21:55.924 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a1321834-740c-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-16T09:21:56.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:56 smithi070 ceph-mon[30388]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-16T09:21:56.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:56 smithi070 ceph-mon[30388]: osdmap e11: 2 total, 1 up, 2 in 2024-09-16T09:21:56.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:56 smithi070 ceph-mon[30388]: from='osd.1 [v2:172.21.15.70:6810/1670012115,v1:172.21.15.70:6811/1670012115]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi070", "root=default"]} : dispatch 2024-09-16T09:21:56.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:56 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-16T09:21:56.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:56 smithi070 ceph-mon[30388]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi070", "root=default"]} : dispatch 2024-09-16T09:21:56.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:56 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:21:56.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:56 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:21:56.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:56 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:21:56.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:56 smithi070 ceph-mon[30388]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]': finished 2024-09-16T09:21:56.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:56 smithi070 ceph-mon[30388]: osdmap e12: 2 total, 1 up, 2 in 2024-09-16T09:21:56.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:56 smithi070 ceph-mon[40691]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-16T09:21:56.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:56 smithi070 ceph-mon[40691]: osdmap e11: 2 total, 1 up, 2 in 2024-09-16T09:21:56.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:56 smithi070 ceph-mon[40691]: from='osd.1 [v2:172.21.15.70:6810/1670012115,v1:172.21.15.70:6811/1670012115]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi070", "root=default"]} : dispatch 2024-09-16T09:21:56.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:56 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-16T09:21:56.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:56 smithi070 ceph-mon[40691]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi070", "root=default"]} : dispatch 2024-09-16T09:21:56.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:56 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:21:56.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:56 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:21:56.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:56 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:21:56.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:56 smithi070 ceph-mon[40691]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]': finished 2024-09-16T09:21:56.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:56 smithi070 ceph-mon[40691]: osdmap e12: 2 total, 1 up, 2 in 2024-09-16T09:21:56.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:56 smithi084 ceph-mon[32629]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-16T09:21:56.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:56 smithi084 ceph-mon[32629]: osdmap e11: 2 total, 1 up, 2 in 2024-09-16T09:21:56.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:56 smithi084 ceph-mon[32629]: from='osd.1 [v2:172.21.15.70:6810/1670012115,v1:172.21.15.70:6811/1670012115]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi070", "root=default"]} : dispatch 2024-09-16T09:21:56.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:56 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-16T09:21:56.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:56 smithi084 ceph-mon[32629]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi070", "root=default"]} : dispatch 2024-09-16T09:21:56.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:56 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:21:56.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:56 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:21:56.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:56 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:21:56.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:56 smithi084 ceph-mon[32629]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]': finished 2024-09-16T09:21:56.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:56 smithi084 ceph-mon[32629]: osdmap e12: 2 total, 1 up, 2 in 2024-09-16T09:21:56.457 INFO:journalctl@ceph.osd.1.smithi070.stdout:Sep 16 09:21:56 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-1[50557]: 2024-09-16T09:21:56.316+0000 7fab6e7b8640 -1 osd.1 0 waiting for initial osdmap 2024-09-16T09:21:56.457 INFO:journalctl@ceph.osd.1.smithi070.stdout:Sep 16 09:21:56 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-1[50557]: 2024-09-16T09:21:56.318+0000 7fab65dc3640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T09:21:56.665 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:21:57.409 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:57 smithi084 ceph-mon[32629]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T09:21:57.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:57 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-16T09:21:57.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:57 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-16T09:21:57.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:57 smithi084 ceph-mon[32629]: from='osd.1 ' entity='osd.1' 2024-09-16T09:21:57.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:57 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-16T09:21:57.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:57 smithi070 ceph-mon[30388]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T09:21:57.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:57 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-16T09:21:57.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:57 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-16T09:21:57.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:57 smithi070 ceph-mon[30388]: from='osd.1 ' entity='osd.1' 2024-09-16T09:21:57.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:57 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-16T09:21:57.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:57 smithi070 ceph-mon[40691]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T09:21:57.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:57 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-16T09:21:57.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:57 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-16T09:21:57.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:57 smithi070 ceph-mon[40691]: from='osd.1 ' entity='osd.1' 2024-09-16T09:21:57.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:57 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-16T09:21:58.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:58 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:21:58.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:58 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:21:58.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:58 smithi070 ceph-mon[30388]: osd.1 [v2:172.21.15.70:6810/1670012115,v1:172.21.15.70:6811/1670012115] boot 2024-09-16T09:21:58.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:58 smithi070 ceph-mon[30388]: osdmap e13: 2 total, 2 up, 2 in 2024-09-16T09:21:58.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:58 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-16T09:21:58.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:58 smithi070 ceph-mon[30388]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T09:21:58.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:58 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:21:58.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:58 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:21:58.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:58 smithi070 ceph-mon[40691]: osd.1 [v2:172.21.15.70:6810/1670012115,v1:172.21.15.70:6811/1670012115] boot 2024-09-16T09:21:58.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:58 smithi070 ceph-mon[40691]: osdmap e13: 2 total, 2 up, 2 in 2024-09-16T09:21:58.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:58 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-16T09:21:58.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:58 smithi070 ceph-mon[40691]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T09:21:58.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:58 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:21:58.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:58 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:21:58.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:58 smithi084 ceph-mon[32629]: osd.1 [v2:172.21.15.70:6810/1670012115,v1:172.21.15.70:6811/1670012115] boot 2024-09-16T09:21:58.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:58 smithi084 ceph-mon[32629]: osdmap e13: 2 total, 2 up, 2 in 2024-09-16T09:21:58.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:58 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-16T09:21:58.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:58 smithi084 ceph-mon[32629]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-16T09:21:59.420 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:21:59 smithi070 ceph-mon[40691]: osdmap e14: 2 total, 2 up, 2 in 2024-09-16T09:21:59.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:21:59 smithi070 ceph-mon[30388]: osdmap e14: 2 total, 2 up, 2 in 2024-09-16T09:21:59.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:21:59 smithi084 ceph-mon[32629]: osdmap e14: 2 total, 2 up, 2 in 2024-09-16T09:21:59.792 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:21:59.820 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi070:vg_nvme/lv_2 2024-09-16T09:22:00.334 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:22:00.635 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:00 smithi070 ceph-mon[30388]: pgmap v59: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T09:22:00.635 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:00 smithi070 ceph-mon[40691]: pgmap v59: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T09:22:00.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:00 smithi084 ceph-mon[32629]: pgmap v59: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T09:22:01.508 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:01 smithi070 ceph-mon[30388]: from='client.14277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:22:01.508 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:01 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-16T09:22:01.508 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:01 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-16T09:22:01.508 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:01 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:22:01.508 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:01 smithi070 ceph-mon[40691]: from='client.14277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:22:01.508 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:01 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-16T09:22:01.509 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:01 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-16T09:22:01.509 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:01 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:22:01.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:01 smithi084 ceph-mon[32629]: from='client.14277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:22:01.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:01 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-16T09:22:01.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:01 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-16T09:22:01.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:01 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:22:02.881 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:02 smithi070 ceph-mon[30388]: pgmap v60: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T09:22:02.881 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:02 smithi070 ceph-mon[30388]: Detected new or changed devices on smithi070 2024-09-16T09:22:02.882 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:02 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:22:02.882 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:02 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:22:02.882 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:02 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-16T09:22:02.882 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:02 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-16T09:22:02.882 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:02 smithi070 ceph-mon[30388]: Adjusting osd_memory_target on smithi070 to 8032M 2024-09-16T09:22:02.882 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:02 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:22:02.882 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:02 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:22:02.882 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:02 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:22:02.882 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:02 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:22:02.882 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:02 smithi070 ceph-mon[40691]: pgmap v60: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T09:22:02.882 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:02 smithi070 ceph-mon[40691]: Detected new or changed devices on smithi070 2024-09-16T09:22:02.882 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:02 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:22:02.883 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:02 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:22:02.883 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:02 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-16T09:22:02.883 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:02 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-16T09:22:02.883 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:02 smithi070 ceph-mon[40691]: Adjusting osd_memory_target on smithi070 to 8032M 2024-09-16T09:22:02.883 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:02 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:22:02.883 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:02 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:22:02.883 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:02 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:22:02.883 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:02 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:22:02.909 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:02 smithi084 ceph-mon[32629]: pgmap v60: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T09:22:02.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:02 smithi084 ceph-mon[32629]: Detected new or changed devices on smithi070 2024-09-16T09:22:02.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:02 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:22:02.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:02 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:22:02.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:02 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-16T09:22:02.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:02 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-16T09:22:02.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:02 smithi084 ceph-mon[32629]: Adjusting osd_memory_target on smithi070 to 8032M 2024-09-16T09:22:02.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:02 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:22:02.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:02 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:22:02.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:02 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:22:02.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:02 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:22:04.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:04 smithi084 ceph-mon[32629]: pgmap v61: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T09:22:04.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:04 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/2383337674' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5b4ca4e8-cecc-4163-b291-c9209b02c4b7"} : dispatch 2024-09-16T09:22:04.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:04 smithi084 ceph-mon[32629]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5b4ca4e8-cecc-4163-b291-c9209b02c4b7"} : dispatch 2024-09-16T09:22:04.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:04 smithi084 ceph-mon[32629]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5b4ca4e8-cecc-4163-b291-c9209b02c4b7"}]': finished 2024-09-16T09:22:04.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:04 smithi084 ceph-mon[32629]: osdmap e15: 3 total, 2 up, 3 in 2024-09-16T09:22:04.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:04 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-16T09:22:04.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:04 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3907850276' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-16T09:22:04.920 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:04 smithi070 ceph-mon[40691]: pgmap v61: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T09:22:04.920 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:04 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/2383337674' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5b4ca4e8-cecc-4163-b291-c9209b02c4b7"} : dispatch 2024-09-16T09:22:04.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:04 smithi070 ceph-mon[40691]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5b4ca4e8-cecc-4163-b291-c9209b02c4b7"} : dispatch 2024-09-16T09:22:04.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:04 smithi070 ceph-mon[40691]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5b4ca4e8-cecc-4163-b291-c9209b02c4b7"}]': finished 2024-09-16T09:22:04.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:04 smithi070 ceph-mon[40691]: osdmap e15: 3 total, 2 up, 3 in 2024-09-16T09:22:04.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:04 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-16T09:22:04.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:04 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3907850276' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-16T09:22:04.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:04 smithi070 ceph-mon[30388]: pgmap v61: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T09:22:04.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:04 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2383337674' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5b4ca4e8-cecc-4163-b291-c9209b02c4b7"} : dispatch 2024-09-16T09:22:04.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:04 smithi070 ceph-mon[30388]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5b4ca4e8-cecc-4163-b291-c9209b02c4b7"} : dispatch 2024-09-16T09:22:04.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:04 smithi070 ceph-mon[30388]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5b4ca4e8-cecc-4163-b291-c9209b02c4b7"}]': finished 2024-09-16T09:22:04.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:04 smithi070 ceph-mon[30388]: osdmap e15: 3 total, 2 up, 3 in 2024-09-16T09:22:04.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:04 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-16T09:22:04.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:04 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3907850276' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-16T09:22:06.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:06 smithi084 ceph-mon[32629]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T09:22:06.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:06 smithi070 ceph-mon[30388]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T09:22:06.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:06 smithi070 ceph-mon[40691]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T09:22:08.807 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:08 smithi070 ceph-mon[30388]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T09:22:08.807 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:08 smithi070 ceph-mon[40691]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T09:22:08.913 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:08 smithi084 ceph-mon[32629]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T09:22:10.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:10 smithi084 ceph-mon[32629]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T09:22:10.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:10 smithi070 ceph-mon[30388]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T09:22:10.920 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:10 smithi070 ceph-mon[40691]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T09:22:11.711 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:11 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-16T09:22:11.711 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:11 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:22:11.711 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:11 smithi070 ceph-mon[30388]: Deploying daemon osd.2 on smithi070 2024-09-16T09:22:11.712 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:11 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-16T09:22:11.712 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:11 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:22:11.712 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:11 smithi070 ceph-mon[40691]: Deploying daemon osd.2 on smithi070 2024-09-16T09:22:11.909 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:11 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-16T09:22:11.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:11 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:22:11.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:11 smithi084 ceph-mon[32629]: Deploying daemon osd.2 on smithi070 2024-09-16T09:22:12.786 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:12 smithi070 ceph-mon[30388]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T09:22:12.786 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:12 smithi070 ceph-mon[40691]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T09:22:12.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:12 smithi084 ceph-mon[32629]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T09:22:14.758 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:14 smithi070 ceph-mon[30388]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T09:22:14.759 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:14 smithi070 ceph-mon[40691]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T09:22:14.909 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:14 smithi084 ceph-mon[32629]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T09:22:16.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:16 smithi084 ceph-mon[32629]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T09:22:16.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:16 smithi070 ceph-mon[30388]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T09:22:16.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:16 smithi070 ceph-mon[40691]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T09:22:18.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:18 smithi084 ceph-mon[32629]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T09:22:18.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:18 smithi070 ceph-mon[30388]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T09:22:18.920 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:18 smithi070 ceph-mon[40691]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T09:22:19.904 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:19 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:22:19.904 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:19 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:22:19.905 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:19 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:22:19.905 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:19 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:22:19.905 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:19 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:22:19.905 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:19 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:22:19.909 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:19 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:22:19.909 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:19 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:22:19.909 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:19 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:22:20.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:20 smithi084 ceph-mon[32629]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T09:22:20.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:20 smithi070 ceph-mon[30388]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T09:22:20.920 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:20 smithi070 ceph-mon[40691]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T09:22:22.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:22 smithi084 ceph-mon[32629]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T09:22:22.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:22 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:22:22.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:22 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:22:22.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:22 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:22:22.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:22 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:22:22.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:22 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:22:22.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:22 smithi070 ceph-mon[30388]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T09:22:22.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:22 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:22:22.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:22 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:22:22.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:22 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:22:22.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:22 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:22:22.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:22 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:22:22.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:22 smithi070 ceph-mon[40691]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T09:22:22.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:22 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:22:22.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:22 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:22:22.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:22 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:22:22.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:22 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:22:22.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:22 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:22:24.199 INFO:teuthology.orchestra.run.smithi070.stdout:Created osd(s) 2 on host 'smithi070' 2024-09-16T09:22:24.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:24 smithi070 ceph-mon[30388]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T09:22:24.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:24 smithi070 ceph-mon[30388]: from='osd.2 [v2:172.21.15.70:6818/3601682336,v1:172.21.15.70:6819/3601682336]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-16T09:22:24.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:24 smithi070 ceph-mon[30388]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-16T09:22:24.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:24 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:22:24.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:24 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:22:24.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:24 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:22:24.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:24 smithi070 ceph-mon[40691]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T09:22:24.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:24 smithi070 ceph-mon[40691]: from='osd.2 [v2:172.21.15.70:6818/3601682336,v1:172.21.15.70:6819/3601682336]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-16T09:22:24.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:24 smithi070 ceph-mon[40691]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-16T09:22:24.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:24 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:22:24.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:24 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:22:24.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:24 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:22:24.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:24 smithi084 ceph-mon[32629]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T09:22:24.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:24 smithi084 ceph-mon[32629]: from='osd.2 [v2:172.21.15.70:6818/3601682336,v1:172.21.15.70:6819/3601682336]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-16T09:22:24.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:24 smithi084 ceph-mon[32629]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-16T09:22:24.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:24 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:22:24.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:24 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:22:24.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:24 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:22:24.809 DEBUG:teuthology.orchestra.run.smithi070:osd.2> sudo journalctl -f -n 0 -u ceph-a1321834-740c-11ef-bceb-c7b262605968@osd.2.service 2024-09-16T09:22:24.811 INFO:tasks.cephadm:Deploying osd.3 on smithi070 with /dev/vg_nvme/lv_1... 2024-09-16T09:22:24.811 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a1321834-740c-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-16T09:22:25.565 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:22:25.649 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:25 smithi070 ceph-mon[30388]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-16T09:22:25.649 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:25 smithi070 ceph-mon[30388]: osdmap e16: 3 total, 2 up, 3 in 2024-09-16T09:22:25.649 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:25 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-16T09:22:25.649 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:25 smithi070 ceph-mon[30388]: from='osd.2 [v2:172.21.15.70:6818/3601682336,v1:172.21.15.70:6819/3601682336]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi070", "root=default"]} : dispatch 2024-09-16T09:22:25.649 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:25 smithi070 ceph-mon[30388]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi070", "root=default"]} : dispatch 2024-09-16T09:22:25.650 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:25 smithi070 ceph-mon[40691]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-16T09:22:25.650 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:25 smithi070 ceph-mon[40691]: osdmap e16: 3 total, 2 up, 3 in 2024-09-16T09:22:25.650 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:25 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-16T09:22:25.650 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:25 smithi070 ceph-mon[40691]: from='osd.2 [v2:172.21.15.70:6818/3601682336,v1:172.21.15.70:6819/3601682336]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi070", "root=default"]} : dispatch 2024-09-16T09:22:25.650 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:25 smithi070 ceph-mon[40691]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi070", "root=default"]} : dispatch 2024-09-16T09:22:25.659 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:25 smithi084 ceph-mon[32629]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-16T09:22:25.659 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:25 smithi084 ceph-mon[32629]: osdmap e16: 3 total, 2 up, 3 in 2024-09-16T09:22:25.659 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:25 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-16T09:22:25.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:25 smithi084 ceph-mon[32629]: from='osd.2 [v2:172.21.15.70:6818/3601682336,v1:172.21.15.70:6819/3601682336]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi070", "root=default"]} : dispatch 2024-09-16T09:22:25.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:25 smithi084 ceph-mon[32629]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi070", "root=default"]} : dispatch 2024-09-16T09:22:25.921 INFO:journalctl@ceph.osd.2.smithi070.stdout:Sep 16 09:22:25 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-2[56773]: 2024-09-16T09:22:25.646+0000 7f6d68f3a640 -1 osd.2 0 waiting for initial osdmap 2024-09-16T09:22:25.921 INFO:journalctl@ceph.osd.2.smithi070.stdout:Sep 16 09:22:25 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-2[56773]: 2024-09-16T09:22:25.649+0000 7f6d60d59640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T09:22:26.659 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:26 smithi084 ceph-mon[32629]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T09:22:26.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:26 smithi084 ceph-mon[32629]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]': finished 2024-09-16T09:22:26.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:26 smithi084 ceph-mon[32629]: osdmap e17: 3 total, 2 up, 3 in 2024-09-16T09:22:26.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:26 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-16T09:22:26.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:26 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-16T09:22:26.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:26 smithi084 ceph-mon[32629]: from='osd.2 ' entity='osd.2' 2024-09-16T09:22:26.662 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:26 smithi070 ceph-mon[30388]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T09:22:26.662 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:26 smithi070 ceph-mon[30388]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]': finished 2024-09-16T09:22:26.662 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:26 smithi070 ceph-mon[30388]: osdmap e17: 3 total, 2 up, 3 in 2024-09-16T09:22:26.662 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:26 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-16T09:22:26.662 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:26 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-16T09:22:26.662 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:26 smithi070 ceph-mon[30388]: from='osd.2 ' entity='osd.2' 2024-09-16T09:22:26.662 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:26 smithi070 ceph-mon[40691]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T09:22:26.662 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:26 smithi070 ceph-mon[40691]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]': finished 2024-09-16T09:22:26.662 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:26 smithi070 ceph-mon[40691]: osdmap e17: 3 total, 2 up, 3 in 2024-09-16T09:22:26.662 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:26 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-16T09:22:26.663 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:26 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-16T09:22:26.663 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:26 smithi070 ceph-mon[40691]: from='osd.2 ' entity='osd.2' 2024-09-16T09:22:27.659 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:27 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:22:27.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:27 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:22:27.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:27 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-16T09:22:27.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:27 smithi084 ceph-mon[32629]: osd.2 [v2:172.21.15.70:6818/3601682336,v1:172.21.15.70:6819/3601682336] boot 2024-09-16T09:22:27.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:27 smithi084 ceph-mon[32629]: osdmap e18: 3 total, 3 up, 3 in 2024-09-16T09:22:27.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:27 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-16T09:22:27.670 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:27 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:22:27.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:27 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:22:27.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:27 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-16T09:22:27.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:27 smithi070 ceph-mon[40691]: osd.2 [v2:172.21.15.70:6818/3601682336,v1:172.21.15.70:6819/3601682336] boot 2024-09-16T09:22:27.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:27 smithi070 ceph-mon[40691]: osdmap e18: 3 total, 3 up, 3 in 2024-09-16T09:22:27.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:27 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-16T09:22:27.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:27 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:22:27.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:27 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:22:27.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:27 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-16T09:22:27.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:27 smithi070 ceph-mon[30388]: osd.2 [v2:172.21.15.70:6818/3601682336,v1:172.21.15.70:6819/3601682336] boot 2024-09-16T09:22:27.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:27 smithi070 ceph-mon[30388]: osdmap e18: 3 total, 3 up, 3 in 2024-09-16T09:22:27.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:27 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-16T09:22:28.659 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:28 smithi084 ceph-mon[32629]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T09:22:28.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:28 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-16T09:22:28.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:28 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-16T09:22:28.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:28 smithi084 ceph-mon[32629]: osdmap e19: 3 total, 3 up, 3 in 2024-09-16T09:22:28.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:28 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-16T09:22:28.670 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:28 smithi070 ceph-mon[40691]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T09:22:28.670 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:28 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-16T09:22:28.670 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:28 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-16T09:22:28.670 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:28 smithi070 ceph-mon[40691]: osdmap e19: 3 total, 3 up, 3 in 2024-09-16T09:22:28.670 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:28 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-16T09:22:28.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:28 smithi070 ceph-mon[30388]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T09:22:28.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:28 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-16T09:22:28.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:28 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-16T09:22:28.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:28 smithi070 ceph-mon[30388]: osdmap e19: 3 total, 3 up, 3 in 2024-09-16T09:22:28.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:28 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-16T09:22:28.958 INFO:journalctl@ceph.osd.2.smithi070.stdout:Sep 16 09:22:28 smithi070 sudo[60642]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-16T09:22:28.958 INFO:journalctl@ceph.osd.2.smithi070.stdout:Sep 16 09:22:28 smithi070 sudo[60642]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T09:22:28.958 INFO:journalctl@ceph.osd.2.smithi070.stdout:Sep 16 09:22:28 smithi070 sudo[60642]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T09:22:28.958 INFO:journalctl@ceph.osd.2.smithi070.stdout:Sep 16 09:22:28 smithi070 sudo[60642]: pam_unix(sudo:session): session closed for user root 2024-09-16T09:22:28.959 INFO:journalctl@ceph.osd.2.smithi070.stdout:Sep 16 09:22:28 smithi070 sudo[60677]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-16T09:22:28.959 INFO:journalctl@ceph.osd.2.smithi070.stdout:Sep 16 09:22:28 smithi070 sudo[60677]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T09:22:28.959 INFO:journalctl@ceph.osd.2.smithi070.stdout:Sep 16 09:22:28 smithi070 sudo[60677]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T09:22:28.959 INFO:journalctl@ceph.osd.2.smithi070.stdout:Sep 16 09:22:28 smithi070 sudo[60677]: pam_unix(sudo:session): session closed for user root 2024-09-16T09:22:28.959 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:28 smithi070 sudo[60683]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-16T09:22:28.959 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:28 smithi070 sudo[60683]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T09:22:28.959 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:28 smithi070 sudo[60683]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T09:22:28.959 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:28 smithi070 sudo[60683]: pam_unix(sudo:session): session closed for user root 2024-09-16T09:22:28.959 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:28 smithi070 sudo[60686]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-16T09:22:28.959 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:28 smithi070 sudo[60686]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T09:22:28.959 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:28 smithi070 sudo[60686]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T09:22:28.960 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:28 smithi070 sudo[60686]: pam_unix(sudo:session): session closed for user root 2024-09-16T09:22:28.960 INFO:journalctl@ceph.osd.1.smithi070.stdout:Sep 16 09:22:28 smithi070 sudo[60634]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-16T09:22:28.960 INFO:journalctl@ceph.osd.1.smithi070.stdout:Sep 16 09:22:28 smithi070 sudo[60634]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T09:22:28.960 INFO:journalctl@ceph.osd.1.smithi070.stdout:Sep 16 09:22:28 smithi070 sudo[60634]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T09:22:28.960 INFO:journalctl@ceph.osd.1.smithi070.stdout:Sep 16 09:22:28 smithi070 sudo[60634]: pam_unix(sudo:session): session closed for user root 2024-09-16T09:22:28.960 INFO:journalctl@ceph.osd.1.smithi070.stdout:Sep 16 09:22:28 smithi070 sudo[60639]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-16T09:22:28.960 INFO:journalctl@ceph.osd.1.smithi070.stdout:Sep 16 09:22:28 smithi070 sudo[60639]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T09:22:28.960 INFO:journalctl@ceph.osd.1.smithi070.stdout:Sep 16 09:22:28 smithi070 sudo[60639]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T09:22:28.960 INFO:journalctl@ceph.osd.1.smithi070.stdout:Sep 16 09:22:28 smithi070 sudo[60639]: pam_unix(sudo:session): session closed for user root 2024-09-16T09:22:28.961 INFO:journalctl@ceph.osd.0.smithi070.stdout:Sep 16 09:22:28 smithi070 sudo[60628]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-16T09:22:28.961 INFO:journalctl@ceph.osd.0.smithi070.stdout:Sep 16 09:22:28 smithi070 sudo[60628]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T09:22:28.961 INFO:journalctl@ceph.osd.0.smithi070.stdout:Sep 16 09:22:28 smithi070 sudo[60628]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T09:22:28.961 INFO:journalctl@ceph.osd.0.smithi070.stdout:Sep 16 09:22:28 smithi070 sudo[60628]: pam_unix(sudo:session): session closed for user root 2024-09-16T09:22:28.961 INFO:journalctl@ceph.osd.0.smithi070.stdout:Sep 16 09:22:28 smithi070 sudo[60631]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-16T09:22:28.961 INFO:journalctl@ceph.osd.0.smithi070.stdout:Sep 16 09:22:28 smithi070 sudo[60631]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T09:22:28.961 INFO:journalctl@ceph.osd.0.smithi070.stdout:Sep 16 09:22:28 smithi070 sudo[60631]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T09:22:28.961 INFO:journalctl@ceph.osd.0.smithi070.stdout:Sep 16 09:22:28 smithi070 sudo[60631]: pam_unix(sudo:session): session closed for user root 2024-09-16T09:22:29.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:28 smithi084 sudo[34761]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-16T09:22:29.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:28 smithi084 sudo[34761]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T09:22:29.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:28 smithi084 sudo[34761]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T09:22:29.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:28 smithi084 sudo[34761]: pam_unix(sudo:session): session closed for user root 2024-09-16T09:22:29.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:28 smithi084 sudo[34764]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-16T09:22:29.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:28 smithi084 sudo[34764]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T09:22:29.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:28 smithi084 sudo[34764]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T09:22:29.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:28 smithi084 sudo[34764]: pam_unix(sudo:session): session closed for user root 2024-09-16T09:22:29.258 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:28 smithi070 sudo[60689]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-16T09:22:29.259 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:28 smithi070 sudo[60689]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T09:22:29.259 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:28 smithi070 sudo[60689]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T09:22:29.259 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:28 smithi070 sudo[60689]: pam_unix(sudo:session): session closed for user root 2024-09-16T09:22:29.259 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:28 smithi070 sudo[60692]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-16T09:22:29.259 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:28 smithi070 sudo[60692]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T09:22:29.259 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:28 smithi070 sudo[60692]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T09:22:29.259 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:28 smithi070 sudo[60692]: pam_unix(sudo:session): session closed for user root 2024-09-16T09:22:29.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:29 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-16T09:22:29.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:29 smithi070 ceph-mon[30388]: osdmap e20: 3 total, 3 up, 3 in 2024-09-16T09:22:29.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:29 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:22:29.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:29 smithi070 ceph-mon[30388]: Detected new or changed devices on smithi070 2024-09-16T09:22:29.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:29 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:22:29.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:29 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:22:29.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:29 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-16T09:22:29.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:29 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-16T09:22:29.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:29 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-16T09:22:29.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:29 smithi070 ceph-mon[30388]: Adjusting osd_memory_target on smithi070 to 5354M 2024-09-16T09:22:29.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:29 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:22:29.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:29 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:22:29.923 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:29 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:22:29.923 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:29 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:22:29.923 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:29 smithi070 ceph-mon[30388]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T09:22:29.923 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:29 smithi070 ceph-mon[30388]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T09:22:29.923 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:29 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-16T09:22:29.923 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:29 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-16T09:22:29.923 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:29 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-16T09:22:29.923 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:29 smithi070 ceph-mon[30388]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T09:22:29.923 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:29 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-16T09:22:29.923 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:29 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-16T09:22:29.923 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:29 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-16T09:22:29.923 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:29 smithi070 ceph-mon[30388]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T09:22:29.923 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:29 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:22:29.923 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:29 smithi070 ceph-mon[30388]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T09:22:29.924 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:29 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-16T09:22:29.924 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:29 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-16T09:22:29.924 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:29 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-16T09:22:29.924 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:29 smithi070 ceph-mon[30388]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T09:22:29.924 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:29 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-16T09:22:29.924 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:29 smithi070 ceph-mon[40691]: osdmap e20: 3 total, 3 up, 3 in 2024-09-16T09:22:29.924 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:29 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:22:29.924 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:29 smithi070 ceph-mon[40691]: Detected new or changed devices on smithi070 2024-09-16T09:22:29.924 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:29 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:22:29.924 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:29 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:22:29.924 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:29 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-16T09:22:29.925 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:29 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-16T09:22:29.925 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:29 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-16T09:22:29.925 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:29 smithi070 ceph-mon[40691]: Adjusting osd_memory_target on smithi070 to 5354M 2024-09-16T09:22:29.925 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:29 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:22:29.925 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:29 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:22:29.925 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:29 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:22:29.925 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:29 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:22:29.925 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:29 smithi070 ceph-mon[40691]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T09:22:29.925 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:29 smithi070 ceph-mon[40691]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T09:22:29.925 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:29 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-16T09:22:29.925 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:29 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-16T09:22:29.925 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:29 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-16T09:22:29.925 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:29 smithi070 ceph-mon[40691]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T09:22:29.926 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:29 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-16T09:22:29.926 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:29 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-16T09:22:29.926 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:29 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-16T09:22:29.926 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:29 smithi070 ceph-mon[40691]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T09:22:29.926 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:29 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:22:29.926 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:29 smithi070 ceph-mon[40691]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T09:22:29.926 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:29 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-16T09:22:29.926 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:29 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-16T09:22:29.926 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:29 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-16T09:22:29.926 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:29 smithi070 ceph-mon[40691]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T09:22:30.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:29 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-16T09:22:30.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:29 smithi084 ceph-mon[32629]: osdmap e20: 3 total, 3 up, 3 in 2024-09-16T09:22:30.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:29 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:22:30.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:29 smithi084 ceph-mon[32629]: Detected new or changed devices on smithi070 2024-09-16T09:22:30.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:29 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:22:30.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:29 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:22:30.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:29 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-16T09:22:30.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:29 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-16T09:22:30.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:29 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-16T09:22:30.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:29 smithi084 ceph-mon[32629]: Adjusting osd_memory_target on smithi070 to 5354M 2024-09-16T09:22:30.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:29 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:22:30.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:29 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:22:30.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:29 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:22:30.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:29 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:22:30.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:29 smithi084 ceph-mon[32629]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T09:22:30.162 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:29 smithi084 ceph-mon[32629]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T09:22:30.162 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:29 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-16T09:22:30.162 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:29 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-16T09:22:30.162 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:29 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-16T09:22:30.162 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:29 smithi084 ceph-mon[32629]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T09:22:30.162 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:29 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-16T09:22:30.162 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:29 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-16T09:22:30.162 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:29 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-16T09:22:30.162 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:29 smithi084 ceph-mon[32629]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T09:22:30.162 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:29 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:22:30.162 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:29 smithi084 ceph-mon[32629]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T09:22:30.162 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:29 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-16T09:22:30.162 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:29 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-16T09:22:30.163 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:29 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-16T09:22:30.163 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:29 smithi084 ceph-mon[32629]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T09:22:30.509 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:22:30.535 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi070:vg_nvme/lv_1 2024-09-16T09:22:30.783 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:22:30.903 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:30 smithi070 ceph-mon[30388]: pgmap v80: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-16T09:22:30.903 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:30 smithi070 ceph-mon[30388]: osdmap e21: 3 total, 3 up, 3 in 2024-09-16T09:22:30.903 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:30 smithi070 ceph-mon[30388]: mgrmap e15: y(active, since 2m), standbys: x 2024-09-16T09:22:30.903 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:30 smithi070 ceph-mon[40691]: pgmap v80: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-16T09:22:30.904 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:30 smithi070 ceph-mon[40691]: osdmap e21: 3 total, 3 up, 3 in 2024-09-16T09:22:30.904 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:30 smithi070 ceph-mon[40691]: mgrmap e15: y(active, since 2m), standbys: x 2024-09-16T09:22:31.159 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:30 smithi084 ceph-mon[32629]: pgmap v80: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-16T09:22:31.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:30 smithi084 ceph-mon[32629]: osdmap e21: 3 total, 3 up, 3 in 2024-09-16T09:22:31.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:30 smithi084 ceph-mon[32629]: mgrmap e15: y(active, since 2m), standbys: x 2024-09-16T09:22:31.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:31 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-16T09:22:31.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:31 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-16T09:22:31.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:31 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:22:31.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:31 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-16T09:22:31.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:31 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-16T09:22:31.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:31 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:22:32.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:31 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-16T09:22:32.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:31 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-16T09:22:32.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:31 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:22:32.910 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:32 smithi070 ceph-mon[30388]: pgmap v82: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-16T09:22:32.911 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:32 smithi070 ceph-mon[30388]: from='client.14313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:22:32.911 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:32 smithi070 ceph-mon[40691]: pgmap v82: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-16T09:22:32.911 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:32 smithi070 ceph-mon[40691]: from='client.14313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:22:33.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:32 smithi084 ceph-mon[32629]: pgmap v82: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-16T09:22:33.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:32 smithi084 ceph-mon[32629]: from='client.14313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi070:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:22:35.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:34 smithi084 ceph-mon[32629]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-16T09:22:35.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:34 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3245845996' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ffb8d527-1b58-4ecd-808c-47d4ba1bf387"} : dispatch 2024-09-16T09:22:35.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:34 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3245845996' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ffb8d527-1b58-4ecd-808c-47d4ba1bf387"}]': finished 2024-09-16T09:22:35.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:34 smithi084 ceph-mon[32629]: osdmap e22: 4 total, 3 up, 4 in 2024-09-16T09:22:35.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:34 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-16T09:22:35.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:34 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1625199609' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-16T09:22:35.173 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:34 smithi070 ceph-mon[30388]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-16T09:22:35.173 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:34 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3245845996' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ffb8d527-1b58-4ecd-808c-47d4ba1bf387"} : dispatch 2024-09-16T09:22:35.173 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:34 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3245845996' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ffb8d527-1b58-4ecd-808c-47d4ba1bf387"}]': finished 2024-09-16T09:22:35.173 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:34 smithi070 ceph-mon[30388]: osdmap e22: 4 total, 3 up, 4 in 2024-09-16T09:22:35.173 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:34 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-16T09:22:35.173 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:34 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1625199609' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-16T09:22:35.173 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:34 smithi070 ceph-mon[40691]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-16T09:22:35.174 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:34 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3245845996' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ffb8d527-1b58-4ecd-808c-47d4ba1bf387"} : dispatch 2024-09-16T09:22:35.174 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:34 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3245845996' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ffb8d527-1b58-4ecd-808c-47d4ba1bf387"}]': finished 2024-09-16T09:22:35.174 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:34 smithi070 ceph-mon[40691]: osdmap e22: 4 total, 3 up, 4 in 2024-09-16T09:22:35.174 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:34 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-16T09:22:35.174 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:34 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1625199609' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-16T09:22:36.973 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:36 smithi070 ceph-mon[30388]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-16T09:22:36.974 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:36 smithi070 ceph-mon[40691]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-16T09:22:37.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:36 smithi084 ceph-mon[32629]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-16T09:22:38.850 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:38 smithi070 ceph-mon[30388]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-16T09:22:38.850 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:38 smithi070 ceph-mon[40691]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-16T09:22:39.159 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:38 smithi084 ceph-mon[32629]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-16T09:22:40.885 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:40 smithi070 ceph-mon[30388]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-16T09:22:40.885 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:40 smithi070 ceph-mon[40691]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-16T09:22:41.159 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:40 smithi084 ceph-mon[32629]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-16T09:22:41.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:41 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-16T09:22:41.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:41 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:22:41.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:41 smithi070 ceph-mon[30388]: Deploying daemon osd.3 on smithi070 2024-09-16T09:22:41.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:41 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-16T09:22:41.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:41 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:22:41.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:41 smithi070 ceph-mon[40691]: Deploying daemon osd.3 on smithi070 2024-09-16T09:22:42.159 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:41 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-16T09:22:42.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:41 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:22:42.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:41 smithi084 ceph-mon[32629]: Deploying daemon osd.3 on smithi070 2024-09-16T09:22:43.146 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:42 smithi070 ceph-mon[30388]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-16T09:22:43.147 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:42 smithi070 ceph-mon[40691]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-16T09:22:43.159 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:42 smithi084 ceph-mon[32629]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-16T09:22:45.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:45 smithi084 ceph-mon[32629]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-16T09:22:45.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:45 smithi070 ceph-mon[30388]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-16T09:22:45.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:45 smithi070 ceph-mon[40691]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-16T09:22:47.220 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:47 smithi070 ceph-mon[30388]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-16T09:22:47.221 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:47 smithi070 ceph-mon[40691]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-16T09:22:47.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:47 smithi084 ceph-mon[32629]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-16T09:22:49.170 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:49 smithi070 ceph-mon[30388]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-16T09:22:49.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:49 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:22:49.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:49 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:22:49.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:49 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:22:49.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:49 smithi070 ceph-mon[40691]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-16T09:22:49.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:49 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:22:49.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:49 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:22:49.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:49 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:22:49.409 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:49 smithi084 ceph-mon[32629]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-16T09:22:49.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:49 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:22:49.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:49 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:22:49.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:49 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:22:51.409 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:51 smithi084 ceph-mon[32629]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-16T09:22:51.420 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:51 smithi070 ceph-mon[40691]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-16T09:22:51.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:51 smithi070 ceph-mon[30388]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-16T09:22:52.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:52 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:22:52.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:52 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:22:52.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:52 smithi084 ceph-mon[32629]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-16T09:22:52.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:52 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:22:52.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:52 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:22:52.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:52 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:22:52.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:52 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:22:52.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:52 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:22:52.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:52 smithi084 ceph-mon[32629]: osdmap e23: 4 total, 3 up, 4 in 2024-09-16T09:22:52.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:52 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-16T09:22:52.670 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:52 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:22:52.670 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:52 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:22:52.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:52 smithi070 ceph-mon[40691]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-16T09:22:52.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:52 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:22:52.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:52 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:22:52.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:52 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:22:52.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:52 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:22:52.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:52 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:22:52.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:52 smithi070 ceph-mon[40691]: osdmap e23: 4 total, 3 up, 4 in 2024-09-16T09:22:52.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:52 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-16T09:22:52.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:52 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:22:52.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:52 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:22:52.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:52 smithi070 ceph-mon[30388]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-16T09:22:52.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:52 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:22:52.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:52 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:22:52.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:52 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:22:52.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:52 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:22:52.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:52 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:22:52.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:52 smithi070 ceph-mon[30388]: osdmap e23: 4 total, 3 up, 4 in 2024-09-16T09:22:52.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:52 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-16T09:22:54.104 INFO:teuthology.orchestra.run.smithi070.stdout:Created osd(s) 3 on host 'smithi070' 2024-09-16T09:22:54.645 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:54 smithi070 ceph-mon[30388]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-16T09:22:54.645 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:54 smithi070 ceph-mon[30388]: from='osd.3 [v2:172.21.15.70:6826/2038083007,v1:172.21.15.70:6827/2038083007]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-16T09:22:54.645 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:54 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:22:54.645 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:54 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:22:54.645 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:54 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:22:54.646 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:54 smithi070 ceph-mon[40691]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-16T09:22:54.646 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:54 smithi070 ceph-mon[40691]: from='osd.3 [v2:172.21.15.70:6826/2038083007,v1:172.21.15.70:6827/2038083007]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-16T09:22:54.646 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:54 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:22:54.646 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:54 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:22:54.646 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:54 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:22:54.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:54 smithi084 ceph-mon[32629]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-16T09:22:54.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:54 smithi084 ceph-mon[32629]: from='osd.3 [v2:172.21.15.70:6826/2038083007,v1:172.21.15.70:6827/2038083007]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-16T09:22:54.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:54 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:22:54.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:54 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:22:54.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:54 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:22:55.100 DEBUG:teuthology.orchestra.run.smithi070:osd.3> sudo journalctl -f -n 0 -u ceph-a1321834-740c-11ef-bceb-c7b262605968@osd.3.service 2024-09-16T09:22:55.102 INFO:tasks.cephadm:Deploying osd.4 on smithi084 with /dev/vg_nvme/lv_4... 2024-09-16T09:22:55.103 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a1321834-740c-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-16T09:22:55.348 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:22:55.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:55 smithi084 ceph-mon[32629]: from='osd.3 [v2:172.21.15.70:6826/2038083007,v1:172.21.15.70:6827/2038083007]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-16T09:22:55.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:55 smithi084 ceph-mon[32629]: osdmap e24: 4 total, 3 up, 4 in 2024-09-16T09:22:55.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:55 smithi084 ceph-mon[32629]: from='osd.3 [v2:172.21.15.70:6826/2038083007,v1:172.21.15.70:6827/2038083007]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi070", "root=default"]} : dispatch 2024-09-16T09:22:55.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:55 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-16T09:22:55.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:55 smithi070 ceph-mon[30388]: from='osd.3 [v2:172.21.15.70:6826/2038083007,v1:172.21.15.70:6827/2038083007]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-16T09:22:55.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:55 smithi070 ceph-mon[30388]: osdmap e24: 4 total, 3 up, 4 in 2024-09-16T09:22:55.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:55 smithi070 ceph-mon[30388]: from='osd.3 [v2:172.21.15.70:6826/2038083007,v1:172.21.15.70:6827/2038083007]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi070", "root=default"]} : dispatch 2024-09-16T09:22:55.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:55 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-16T09:22:55.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:55 smithi070 ceph-mon[40691]: from='osd.3 [v2:172.21.15.70:6826/2038083007,v1:172.21.15.70:6827/2038083007]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-16T09:22:55.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:55 smithi070 ceph-mon[40691]: osdmap e24: 4 total, 3 up, 4 in 2024-09-16T09:22:55.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:55 smithi070 ceph-mon[40691]: from='osd.3 [v2:172.21.15.70:6826/2038083007,v1:172.21.15.70:6827/2038083007]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi070", "root=default"]} : dispatch 2024-09-16T09:22:55.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:55 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-16T09:22:55.673 INFO:journalctl@ceph.osd.3.smithi070.stdout:Sep 16 09:22:55 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-3[62944]: 2024-09-16T09:22:55.613+0000 7f1dc070b640 -1 osd.3 0 waiting for initial osdmap 2024-09-16T09:22:55.673 INFO:journalctl@ceph.osd.3.smithi070.stdout:Sep 16 09:22:55 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-3[62944]: 2024-09-16T09:22:55.617+0000 7f1db7d16640 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T09:22:56.544 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:56 smithi084 ceph-mon[32629]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-16T09:22:56.544 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:56 smithi084 ceph-mon[32629]: from='osd.3 [v2:172.21.15.70:6826/2038083007,v1:172.21.15.70:6827/2038083007]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]': finished 2024-09-16T09:22:56.544 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:56 smithi084 ceph-mon[32629]: osdmap e25: 4 total, 3 up, 4 in 2024-09-16T09:22:56.544 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:56 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-16T09:22:56.544 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:56 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-16T09:22:56.544 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:56 smithi084 ceph-mon[32629]: from='osd.3 [v2:172.21.15.70:6826/2038083007,v1:172.21.15.70:6827/2038083007]' entity='osd.3' 2024-09-16T09:22:56.545 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:56 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-16T09:22:56.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:56 smithi070 ceph-mon[30388]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-16T09:22:56.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:56 smithi070 ceph-mon[30388]: from='osd.3 [v2:172.21.15.70:6826/2038083007,v1:172.21.15.70:6827/2038083007]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]': finished 2024-09-16T09:22:56.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:56 smithi070 ceph-mon[30388]: osdmap e25: 4 total, 3 up, 4 in 2024-09-16T09:22:56.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:56 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-16T09:22:56.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:56 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-16T09:22:56.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:56 smithi070 ceph-mon[30388]: from='osd.3 [v2:172.21.15.70:6826/2038083007,v1:172.21.15.70:6827/2038083007]' entity='osd.3' 2024-09-16T09:22:56.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:56 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-16T09:22:56.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:56 smithi070 ceph-mon[40691]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-16T09:22:56.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:56 smithi070 ceph-mon[40691]: from='osd.3 [v2:172.21.15.70:6826/2038083007,v1:172.21.15.70:6827/2038083007]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi070", "root=default"]}]': finished 2024-09-16T09:22:56.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:56 smithi070 ceph-mon[40691]: osdmap e25: 4 total, 3 up, 4 in 2024-09-16T09:22:56.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:56 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-16T09:22:56.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:56 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-16T09:22:56.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:56 smithi070 ceph-mon[40691]: from='osd.3 [v2:172.21.15.70:6826/2038083007,v1:172.21.15.70:6827/2038083007]' entity='osd.3' 2024-09-16T09:22:56.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:56 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-16T09:22:57.482 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:57 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:22:57.483 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:57 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:22:57.483 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:57 smithi070 ceph-mon[30388]: osd.3 [v2:172.21.15.70:6826/2038083007,v1:172.21.15.70:6827/2038083007] boot 2024-09-16T09:22:57.483 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:57 smithi070 ceph-mon[30388]: osdmap e26: 4 total, 4 up, 4 in 2024-09-16T09:22:57.483 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:57 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-16T09:22:57.483 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:57 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:22:57.483 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:57 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:22:57.483 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:57 smithi070 ceph-mon[40691]: osd.3 [v2:172.21.15.70:6826/2038083007,v1:172.21.15.70:6827/2038083007] boot 2024-09-16T09:22:57.483 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:57 smithi070 ceph-mon[40691]: osdmap e26: 4 total, 4 up, 4 in 2024-09-16T09:22:57.483 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:57 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-16T09:22:57.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:57 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:22:57.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:57 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:22:57.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:57 smithi084 ceph-mon[32629]: osd.3 [v2:172.21.15.70:6826/2038083007,v1:172.21.15.70:6827/2038083007] boot 2024-09-16T09:22:57.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:57 smithi084 ceph-mon[32629]: osdmap e26: 4 total, 4 up, 4 in 2024-09-16T09:22:57.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:57 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-16T09:22:58.003 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-16T09:22:58.028 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi084:vg_nvme/lv_4 2024-09-16T09:22:58.267 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:22:58.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:58 smithi084 ceph-mon[32629]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-16T09:22:58.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:58 smithi084 ceph-mon[32629]: Detected new or changed devices on smithi070 2024-09-16T09:22:58.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:58 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:22:58.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:58 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:22:58.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:58 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-16T09:22:58.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:58 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-16T09:22:58.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:58 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-16T09:22:58.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:58 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-16T09:22:58.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:58 smithi084 ceph-mon[32629]: Adjusting osd_memory_target on smithi070 to 4016M 2024-09-16T09:22:58.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:58 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:22:58.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:58 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:22:58.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:58 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:22:58.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:58 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:22:58.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:58 smithi084 ceph-mon[32629]: osdmap e27: 4 total, 4 up, 4 in 2024-09-16T09:22:58.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:58 smithi070 ceph-mon[30388]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-16T09:22:58.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:58 smithi070 ceph-mon[30388]: Detected new or changed devices on smithi070 2024-09-16T09:22:58.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:58 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:22:58.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:58 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:22:58.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:58 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-16T09:22:58.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:58 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-16T09:22:58.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:58 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-16T09:22:58.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:58 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-16T09:22:58.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:58 smithi070 ceph-mon[30388]: Adjusting osd_memory_target on smithi070 to 4016M 2024-09-16T09:22:58.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:58 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:22:58.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:58 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:22:58.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:58 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:22:58.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:58 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:22:58.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:58 smithi070 ceph-mon[30388]: osdmap e27: 4 total, 4 up, 4 in 2024-09-16T09:22:58.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:58 smithi070 ceph-mon[40691]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-16T09:22:58.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:58 smithi070 ceph-mon[40691]: Detected new or changed devices on smithi070 2024-09-16T09:22:58.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:58 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:22:58.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:58 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:22:58.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:58 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-16T09:22:58.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:58 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-16T09:22:58.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:58 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-16T09:22:58.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:58 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-16T09:22:58.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:58 smithi070 ceph-mon[40691]: Adjusting osd_memory_target on smithi070 to 4016M 2024-09-16T09:22:58.923 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:58 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:22:58.923 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:58 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:22:58.923 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:58 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:22:58.923 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:58 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:22:58.923 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:58 smithi070 ceph-mon[40691]: osdmap e27: 4 total, 4 up, 4 in 2024-09-16T09:22:59.538 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:59 smithi084 ceph-mon[32629]: from='client.24190 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi084:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:22:59.538 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:59 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-16T09:22:59.538 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:59 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-16T09:22:59.539 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:22:59 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:22:59.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:59 smithi070 ceph-mon[30388]: from='client.24190 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi084:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:22:59.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:59 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-16T09:22:59.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:59 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-16T09:22:59.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:22:59 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:22:59.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:59 smithi070 ceph-mon[40691]: from='client.24190 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi084:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:22:59.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:59 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-16T09:22:59.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:59 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-16T09:22:59.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:22:59 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:23:00.781 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:00 smithi084 ceph-mon[32629]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-16T09:23:00.781 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:00 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:23:00.782 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:00 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:23:00.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:00 smithi070 ceph-mon[30388]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-16T09:23:00.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:00 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:23:00.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:00 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:23:00.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:00 smithi070 ceph-mon[40691]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-16T09:23:00.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:00 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:23:00.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:00 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:23:01.779 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:01 smithi084 ceph-mon[32629]: osdmap e28: 4 total, 4 up, 4 in 2024-09-16T09:23:01.779 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:01 smithi084 ceph-mon[32629]: from='client.? 172.21.15.84:0/3089792484' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a946760b-4d91-4a4b-8e9b-a07a50589211"} : dispatch 2024-09-16T09:23:01.779 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:01 smithi084 ceph-mon[32629]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a946760b-4d91-4a4b-8e9b-a07a50589211"} : dispatch 2024-09-16T09:23:01.779 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:01 smithi084 ceph-mon[32629]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a946760b-4d91-4a4b-8e9b-a07a50589211"}]': finished 2024-09-16T09:23:01.779 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:01 smithi084 ceph-mon[32629]: osdmap e29: 5 total, 4 up, 5 in 2024-09-16T09:23:01.779 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:01 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-16T09:23:01.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:01 smithi070 ceph-mon[30388]: osdmap e28: 4 total, 4 up, 4 in 2024-09-16T09:23:01.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:01 smithi070 ceph-mon[30388]: from='client.? 172.21.15.84:0/3089792484' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a946760b-4d91-4a4b-8e9b-a07a50589211"} : dispatch 2024-09-16T09:23:01.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:01 smithi070 ceph-mon[30388]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a946760b-4d91-4a4b-8e9b-a07a50589211"} : dispatch 2024-09-16T09:23:01.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:01 smithi070 ceph-mon[30388]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a946760b-4d91-4a4b-8e9b-a07a50589211"}]': finished 2024-09-16T09:23:01.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:01 smithi070 ceph-mon[30388]: osdmap e29: 5 total, 4 up, 5 in 2024-09-16T09:23:01.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:01 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-16T09:23:01.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:01 smithi070 ceph-mon[40691]: osdmap e28: 4 total, 4 up, 4 in 2024-09-16T09:23:01.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:01 smithi070 ceph-mon[40691]: from='client.? 172.21.15.84:0/3089792484' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a946760b-4d91-4a4b-8e9b-a07a50589211"} : dispatch 2024-09-16T09:23:01.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:01 smithi070 ceph-mon[40691]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a946760b-4d91-4a4b-8e9b-a07a50589211"} : dispatch 2024-09-16T09:23:01.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:01 smithi070 ceph-mon[40691]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a946760b-4d91-4a4b-8e9b-a07a50589211"}]': finished 2024-09-16T09:23:01.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:01 smithi070 ceph-mon[40691]: osdmap e29: 5 total, 4 up, 5 in 2024-09-16T09:23:01.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:01 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-16T09:23:02.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:02 smithi084 ceph-mon[32629]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-16T09:23:02.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:02 smithi084 ceph-mon[32629]: from='client.? 172.21.15.84:0/1105459173' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-16T09:23:02.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:02 smithi070 ceph-mon[30388]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-16T09:23:02.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:02 smithi070 ceph-mon[30388]: from='client.? 172.21.15.84:0/1105459173' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-16T09:23:02.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:02 smithi070 ceph-mon[40691]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-16T09:23:02.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:02 smithi070 ceph-mon[40691]: from='client.? 172.21.15.84:0/1105459173' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-16T09:23:05.307 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:05 smithi084 ceph-mon[32629]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-16T09:23:05.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:05 smithi070 ceph-mon[30388]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-16T09:23:05.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:05 smithi070 ceph-mon[40691]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-16T09:23:07.230 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:07 smithi084 ceph-mon[32629]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-16T09:23:07.420 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:07 smithi070 ceph-mon[40691]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-16T09:23:07.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:07 smithi070 ceph-mon[30388]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-16T09:23:09.078 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:09 smithi084 ceph-mon[32629]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-16T09:23:09.078 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:09 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-16T09:23:09.078 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:09 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:23:09.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:09 smithi070 ceph-mon[30388]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-16T09:23:09.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:09 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-16T09:23:09.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:09 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:23:09.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:09 smithi070 ceph-mon[40691]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-16T09:23:09.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:09 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-16T09:23:09.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:09 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:23:10.260 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:10 smithi084 ceph-mon[32629]: Deploying daemon osd.4 on smithi084 2024-09-16T09:23:10.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:10 smithi070 ceph-mon[30388]: Deploying daemon osd.4 on smithi084 2024-09-16T09:23:10.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:10 smithi070 ceph-mon[40691]: Deploying daemon osd.4 on smithi084 2024-09-16T09:23:11.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:11 smithi084 ceph-mon[32629]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-16T09:23:11.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:11 smithi070 ceph-mon[30388]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-16T09:23:11.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:11 smithi070 ceph-mon[40691]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-16T09:23:13.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:13 smithi084 ceph-mon[32629]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-16T09:23:13.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:13 smithi070 ceph-mon[30388]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-16T09:23:13.420 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:13 smithi070 ceph-mon[40691]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-16T09:23:15.245 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:15 smithi084 ceph-mon[32629]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-16T09:23:15.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:15 smithi070 ceph-mon[30388]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-16T09:23:15.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:15 smithi070 ceph-mon[40691]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-16T09:23:16.325 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:16 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:23:16.325 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:16 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:23:16.325 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:16 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:23:16.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:16 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:23:16.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:16 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:23:16.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:16 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:23:16.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:16 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:23:16.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:16 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:23:16.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:16 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:23:17.229 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:17 smithi084 ceph-mon[32629]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-16T09:23:17.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:17 smithi070 ceph-mon[30388]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-16T09:23:17.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:17 smithi070 ceph-mon[40691]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-16T09:23:19.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:19 smithi084 ceph-mon[32629]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-16T09:23:19.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:19 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:23:19.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:19 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:23:19.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:19 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:23:19.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:19 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:23:19.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:19 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:23:19.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:19 smithi070 ceph-mon[30388]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-16T09:23:19.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:19 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:23:19.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:19 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:23:19.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:19 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:23:19.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:19 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:23:19.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:19 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:23:19.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:19 smithi070 ceph-mon[40691]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-16T09:23:19.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:19 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:23:19.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:19 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:23:19.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:19 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:23:19.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:19 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:23:19.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:19 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:23:20.903 INFO:teuthology.orchestra.run.smithi084.stdout:Created osd(s) 4 on host 'smithi084' 2024-09-16T09:23:21.159 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:21 smithi084 ceph-mon[32629]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-16T09:23:21.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:21 smithi084 ceph-mon[32629]: from='osd.4 [v2:172.21.15.84:6800/2250892710,v1:172.21.15.84:6801/2250892710]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-16T09:23:21.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:21 smithi084 ceph-mon[32629]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-16T09:23:21.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:21 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:23:21.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:21 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:23:21.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:21 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:23:21.420 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:21 smithi070 ceph-mon[40691]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-16T09:23:21.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:21 smithi070 ceph-mon[40691]: from='osd.4 [v2:172.21.15.84:6800/2250892710,v1:172.21.15.84:6801/2250892710]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-16T09:23:21.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:21 smithi070 ceph-mon[40691]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-16T09:23:21.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:21 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:23:21.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:21 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:23:21.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:21 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:23:21.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:21 smithi070 ceph-mon[30388]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-16T09:23:21.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:21 smithi070 ceph-mon[30388]: from='osd.4 [v2:172.21.15.84:6800/2250892710,v1:172.21.15.84:6801/2250892710]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-16T09:23:21.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:21 smithi070 ceph-mon[30388]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-16T09:23:21.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:21 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:23:21.422 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:21 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:23:21.422 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:21 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:23:21.570 DEBUG:teuthology.orchestra.run.smithi084:osd.4> sudo journalctl -f -n 0 -u ceph-a1321834-740c-11ef-bceb-c7b262605968@osd.4.service 2024-09-16T09:23:21.573 INFO:tasks.cephadm:Deploying osd.5 on smithi084 with /dev/vg_nvme/lv_3... 2024-09-16T09:23:21.573 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a1321834-740c-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-16T09:23:22.101 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:22 smithi084 ceph-mon[32629]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-16T09:23:22.101 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:22 smithi084 ceph-mon[32629]: osdmap e30: 5 total, 4 up, 5 in 2024-09-16T09:23:22.101 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:22 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-16T09:23:22.101 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:22 smithi084 ceph-mon[32629]: from='osd.4 [v2:172.21.15.84:6800/2250892710,v1:172.21.15.84:6801/2250892710]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi084", "root=default"]} : dispatch 2024-09-16T09:23:22.101 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:22 smithi084 ceph-mon[32629]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi084", "root=default"]} : dispatch 2024-09-16T09:23:22.101 INFO:journalctl@ceph.osd.4.smithi084.stdout:Sep 16 09:23:22 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-4[37218]: 2024-09-16T09:23:22.089+0000 7f140d096640 -1 osd.4 0 waiting for initial osdmap 2024-09-16T09:23:22.101 INFO:journalctl@ceph.osd.4.smithi084.stdout:Sep 16 09:23:22 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-4[37218]: 2024-09-16T09:23:22.099+0000 7f1408ebd640 -1 osd.4 31 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T09:23:22.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:22 smithi070 ceph-mon[30388]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-16T09:23:22.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:22 smithi070 ceph-mon[30388]: osdmap e30: 5 total, 4 up, 5 in 2024-09-16T09:23:22.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:22 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-16T09:23:22.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:22 smithi070 ceph-mon[30388]: from='osd.4 [v2:172.21.15.84:6800/2250892710,v1:172.21.15.84:6801/2250892710]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi084", "root=default"]} : dispatch 2024-09-16T09:23:22.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:22 smithi070 ceph-mon[30388]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi084", "root=default"]} : dispatch 2024-09-16T09:23:22.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:22 smithi070 ceph-mon[40691]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-16T09:23:22.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:22 smithi070 ceph-mon[40691]: osdmap e30: 5 total, 4 up, 5 in 2024-09-16T09:23:22.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:22 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-16T09:23:22.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:22 smithi070 ceph-mon[40691]: from='osd.4 [v2:172.21.15.84:6800/2250892710,v1:172.21.15.84:6801/2250892710]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi084", "root=default"]} : dispatch 2024-09-16T09:23:22.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:22 smithi070 ceph-mon[40691]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi084", "root=default"]} : dispatch 2024-09-16T09:23:22.895 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:23:23.133 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:23 smithi084 ceph-mon[32629]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-16T09:23:23.133 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:23 smithi084 ceph-mon[32629]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]': finished 2024-09-16T09:23:23.133 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:23 smithi084 ceph-mon[32629]: osdmap e31: 5 total, 4 up, 5 in 2024-09-16T09:23:23.133 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:23 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-16T09:23:23.133 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:23 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-16T09:23:23.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:23 smithi070 ceph-mon[30388]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-16T09:23:23.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:23 smithi070 ceph-mon[30388]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]': finished 2024-09-16T09:23:23.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:23 smithi070 ceph-mon[30388]: osdmap e31: 5 total, 4 up, 5 in 2024-09-16T09:23:23.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:23 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-16T09:23:23.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:23 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-16T09:23:23.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:23 smithi070 ceph-mon[40691]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-09-16T09:23:23.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:23 smithi070 ceph-mon[40691]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]': finished 2024-09-16T09:23:23.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:23 smithi070 ceph-mon[40691]: osdmap e31: 5 total, 4 up, 5 in 2024-09-16T09:23:23.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:23 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-16T09:23:23.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:23 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-16T09:23:24.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:24 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:23:24.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:24 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:23:24.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:24 smithi084 ceph-mon[32629]: osd.4 [v2:172.21.15.84:6800/2250892710,v1:172.21.15.84:6801/2250892710] boot 2024-09-16T09:23:24.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:24 smithi084 ceph-mon[32629]: osdmap e32: 5 total, 5 up, 5 in 2024-09-16T09:23:24.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:24 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-16T09:23:24.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:24 smithi084 ceph-mon[32629]: osdmap e33: 5 total, 5 up, 5 in 2024-09-16T09:23:24.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:24 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:23:24.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:24 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:23:24.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:24 smithi070 ceph-mon[30388]: osd.4 [v2:172.21.15.84:6800/2250892710,v1:172.21.15.84:6801/2250892710] boot 2024-09-16T09:23:24.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:24 smithi070 ceph-mon[30388]: osdmap e32: 5 total, 5 up, 5 in 2024-09-16T09:23:24.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:24 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-16T09:23:24.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:24 smithi070 ceph-mon[30388]: osdmap e33: 5 total, 5 up, 5 in 2024-09-16T09:23:24.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:24 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:23:24.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:24 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:23:24.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:24 smithi070 ceph-mon[40691]: osd.4 [v2:172.21.15.84:6800/2250892710,v1:172.21.15.84:6801/2250892710] boot 2024-09-16T09:23:24.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:24 smithi070 ceph-mon[40691]: osdmap e32: 5 total, 5 up, 5 in 2024-09-16T09:23:24.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:24 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-16T09:23:24.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:24 smithi070 ceph-mon[40691]: osdmap e33: 5 total, 5 up, 5 in 2024-09-16T09:23:25.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:25 smithi084 ceph-mon[32629]: pgmap v119: 1 pgs: 1 peering; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T09:23:25.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:25 smithi084 ceph-mon[32629]: osdmap e34: 5 total, 5 up, 5 in 2024-09-16T09:23:25.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:25 smithi070 ceph-mon[30388]: pgmap v119: 1 pgs: 1 peering; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T09:23:25.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:25 smithi070 ceph-mon[30388]: osdmap e34: 5 total, 5 up, 5 in 2024-09-16T09:23:25.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:25 smithi070 ceph-mon[40691]: pgmap v119: 1 pgs: 1 peering; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T09:23:25.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:25 smithi070 ceph-mon[40691]: osdmap e34: 5 total, 5 up, 5 in 2024-09-16T09:23:26.601 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:26 smithi084 ceph-mon[32629]: Detected new or changed devices on smithi084 2024-09-16T09:23:26.601 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:26 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:23:26.601 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:26 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:23:26.602 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:26 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-16T09:23:26.602 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:26 smithi084 ceph-mon[32629]: Adjusting osd_memory_target on smithi084 to 17088M 2024-09-16T09:23:26.602 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:26 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:23:26.602 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:26 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:23:26.602 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:26 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:23:26.602 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:26 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:23:26.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:26 smithi070 ceph-mon[40691]: Detected new or changed devices on smithi084 2024-09-16T09:23:26.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:26 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:23:26.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:26 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:23:26.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:26 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-16T09:23:26.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:26 smithi070 ceph-mon[40691]: Adjusting osd_memory_target on smithi084 to 17088M 2024-09-16T09:23:26.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:26 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:23:26.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:26 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:23:26.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:26 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:23:26.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:26 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:23:26.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:26 smithi070 ceph-mon[30388]: Detected new or changed devices on smithi084 2024-09-16T09:23:26.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:26 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:23:26.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:26 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:23:26.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:26 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-16T09:23:26.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:26 smithi070 ceph-mon[30388]: Adjusting osd_memory_target on smithi084 to 17088M 2024-09-16T09:23:26.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:26 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:23:26.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:26 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:23:26.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:26 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:23:26.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:26 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:23:26.888 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-16T09:23:26.917 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi084:vg_nvme/lv_3 2024-09-16T09:23:27.166 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:23:27.538 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:27 smithi084 ceph-mon[32629]: pgmap v122: 1 pgs: 1 peering; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T09:23:27.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:27 smithi070 ceph-mon[40691]: pgmap v122: 1 pgs: 1 peering; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T09:23:27.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:27 smithi070 ceph-mon[30388]: pgmap v122: 1 pgs: 1 peering; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T09:23:28.462 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:28 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-16T09:23:28.463 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:28 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-16T09:23:28.463 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:28 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:23:28.670 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:28 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-16T09:23:28.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:28 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-16T09:23:28.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:28 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:23:28.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:28 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-16T09:23:28.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:28 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-16T09:23:28.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:28 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:23:29.409 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:29 smithi084 ceph-mon[32629]: pgmap v123: 1 pgs: 1 peering; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T09:23:29.409 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:29 smithi084 ceph-mon[32629]: from='client.24217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi084:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:23:29.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:29 smithi070 ceph-mon[30388]: pgmap v123: 1 pgs: 1 peering; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T09:23:29.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:29 smithi070 ceph-mon[30388]: from='client.24217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi084:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:23:29.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:29 smithi070 ceph-mon[40691]: pgmap v123: 1 pgs: 1 peering; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T09:23:29.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:29 smithi070 ceph-mon[40691]: from='client.24217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi084:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:23:30.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:30 smithi084 ceph-mon[32629]: from='client.? 172.21.15.84:0/1134402752' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "67681760-75a9-4846-946e-a935d42c5e05"} : dispatch 2024-09-16T09:23:30.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:30 smithi084 ceph-mon[32629]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "67681760-75a9-4846-946e-a935d42c5e05"} : dispatch 2024-09-16T09:23:30.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:30 smithi084 ceph-mon[32629]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "67681760-75a9-4846-946e-a935d42c5e05"}]': finished 2024-09-16T09:23:30.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:30 smithi084 ceph-mon[32629]: osdmap e35: 6 total, 5 up, 6 in 2024-09-16T09:23:30.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:30 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-16T09:23:30.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:30 smithi070 ceph-mon[30388]: from='client.? 172.21.15.84:0/1134402752' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "67681760-75a9-4846-946e-a935d42c5e05"} : dispatch 2024-09-16T09:23:30.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:30 smithi070 ceph-mon[30388]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "67681760-75a9-4846-946e-a935d42c5e05"} : dispatch 2024-09-16T09:23:30.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:30 smithi070 ceph-mon[30388]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "67681760-75a9-4846-946e-a935d42c5e05"}]': finished 2024-09-16T09:23:30.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:30 smithi070 ceph-mon[30388]: osdmap e35: 6 total, 5 up, 6 in 2024-09-16T09:23:30.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:30 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-16T09:23:30.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:30 smithi070 ceph-mon[40691]: from='client.? 172.21.15.84:0/1134402752' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "67681760-75a9-4846-946e-a935d42c5e05"} : dispatch 2024-09-16T09:23:30.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:30 smithi070 ceph-mon[40691]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "67681760-75a9-4846-946e-a935d42c5e05"} : dispatch 2024-09-16T09:23:30.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:30 smithi070 ceph-mon[40691]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "67681760-75a9-4846-946e-a935d42c5e05"}]': finished 2024-09-16T09:23:30.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:30 smithi070 ceph-mon[40691]: osdmap e35: 6 total, 5 up, 6 in 2024-09-16T09:23:30.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:30 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-16T09:23:31.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:31 smithi084 ceph-mon[32629]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-09-16T09:23:31.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:31 smithi084 ceph-mon[32629]: from='client.? 172.21.15.84:0/2247631035' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-16T09:23:31.673 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:31 smithi070 ceph-mon[30388]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-09-16T09:23:31.673 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:31 smithi070 ceph-mon[30388]: from='client.? 172.21.15.84:0/2247631035' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-16T09:23:31.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:31 smithi070 ceph-mon[40691]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-09-16T09:23:31.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:31 smithi070 ceph-mon[40691]: from='client.? 172.21.15.84:0/2247631035' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-16T09:23:33.620 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:33 smithi084 ceph-mon[32629]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-09-16T09:23:33.620 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:33 smithi084 ceph-mon[32629]: 1.0 scrub starts 2024-09-16T09:23:33.620 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:33 smithi084 ceph-mon[32629]: 1.0 scrub ok 2024-09-16T09:23:33.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:33 smithi070 ceph-mon[30388]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-09-16T09:23:33.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:33 smithi070 ceph-mon[30388]: 1.0 scrub starts 2024-09-16T09:23:33.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:33 smithi070 ceph-mon[30388]: 1.0 scrub ok 2024-09-16T09:23:33.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:33 smithi070 ceph-mon[40691]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-09-16T09:23:33.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:33 smithi070 ceph-mon[40691]: 1.0 scrub starts 2024-09-16T09:23:33.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:33 smithi070 ceph-mon[40691]: 1.0 scrub ok 2024-09-16T09:23:35.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:35 smithi084 ceph-mon[32629]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-09-16T09:23:35.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:35 smithi070 ceph-mon[30388]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-09-16T09:23:35.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:35 smithi070 ceph-mon[40691]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-09-16T09:23:37.590 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:37 smithi084 ceph-mon[32629]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-16T09:23:37.590 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:37 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-16T09:23:37.590 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:37 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:23:37.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:37 smithi070 ceph-mon[30388]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-16T09:23:37.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:37 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-16T09:23:37.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:37 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:23:37.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:37 smithi070 ceph-mon[40691]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-16T09:23:37.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:37 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-16T09:23:37.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:37 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:23:38.535 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:38 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:23:38.535 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:38 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:23:38.535 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:38 smithi084 ceph-mon[32629]: Deploying daemon osd.5 on smithi084 2024-09-16T09:23:38.535 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:38 smithi084 ceph-mon[32629]: osdmap e36: 6 total, 5 up, 6 in 2024-09-16T09:23:38.535 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:38 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-16T09:23:38.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:38 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:23:38.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:38 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:23:38.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:38 smithi070 ceph-mon[30388]: Deploying daemon osd.5 on smithi084 2024-09-16T09:23:38.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:38 smithi070 ceph-mon[30388]: osdmap e36: 6 total, 5 up, 6 in 2024-09-16T09:23:38.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:38 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-16T09:23:38.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:38 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:23:38.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:38 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:23:38.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:38 smithi070 ceph-mon[40691]: Deploying daemon osd.5 on smithi084 2024-09-16T09:23:38.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:38 smithi070 ceph-mon[40691]: osdmap e36: 6 total, 5 up, 6 in 2024-09-16T09:23:38.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:38 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-16T09:23:39.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:39 smithi084 ceph-mon[32629]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T09:23:39.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:39 smithi070 ceph-mon[30388]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T09:23:39.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:39 smithi070 ceph-mon[40691]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T09:23:41.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:41 smithi084 ceph-mon[32629]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T09:23:41.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:41 smithi070 ceph-mon[30388]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T09:23:41.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:41 smithi070 ceph-mon[40691]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T09:23:43.479 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:43 smithi084 ceph-mon[32629]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T09:23:43.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:43 smithi070 ceph-mon[30388]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T09:23:43.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:43 smithi070 ceph-mon[40691]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T09:23:45.393 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:45 smithi084 ceph-mon[32629]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T09:23:45.393 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:45 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:23:45.393 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:45 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:23:45.393 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:45 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:23:45.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:45 smithi070 ceph-mon[30388]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T09:23:45.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:45 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:23:45.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:45 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:23:45.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:45 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:23:45.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:45 smithi070 ceph-mon[40691]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T09:23:45.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:45 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:23:45.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:45 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:23:45.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:45 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:23:47.620 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:47 smithi084 ceph-mon[32629]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T09:23:47.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:47 smithi070 ceph-mon[30388]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T09:23:47.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:47 smithi070 ceph-mon[40691]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T09:23:48.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:48 smithi084 ceph-mon[32629]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T09:23:48.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:48 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:23:48.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:48 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:23:48.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:48 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:23:48.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:48 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:23:48.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:48 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:23:48.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:48 smithi070 ceph-mon[30388]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T09:23:48.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:48 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:23:48.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:48 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:23:48.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:48 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:23:48.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:48 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:23:48.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:48 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:23:48.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:48 smithi070 ceph-mon[40691]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T09:23:48.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:48 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:23:48.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:48 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:23:48.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:48 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:23:48.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:48 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:23:48.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:48 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:23:49.949 INFO:teuthology.orchestra.run.smithi084.stdout:Created osd(s) 5 on host 'smithi084' 2024-09-16T09:23:50.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:50 smithi084 ceph-mon[32629]: from='osd.5 [v2:172.21.15.84:6808/1803945348,v1:172.21.15.84:6809/1803945348]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-16T09:23:50.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:50 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:23:50.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:50 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:23:50.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:50 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:23:50.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:50 smithi070 ceph-mon[30388]: from='osd.5 [v2:172.21.15.84:6808/1803945348,v1:172.21.15.84:6809/1803945348]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-16T09:23:50.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:50 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:23:50.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:50 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:23:50.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:50 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:23:50.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:50 smithi070 ceph-mon[40691]: from='osd.5 [v2:172.21.15.84:6808/1803945348,v1:172.21.15.84:6809/1803945348]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-16T09:23:50.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:50 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:23:50.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:50 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:23:50.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:50 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:23:50.621 DEBUG:teuthology.orchestra.run.smithi084:osd.5> sudo journalctl -f -n 0 -u ceph-a1321834-740c-11ef-bceb-c7b262605968@osd.5.service 2024-09-16T09:23:50.624 INFO:tasks.cephadm:Deploying osd.6 on smithi084 with /dev/vg_nvme/lv_2... 2024-09-16T09:23:50.624 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a1321834-740c-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-16T09:23:51.107 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:51 smithi084 ceph-mon[32629]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T09:23:51.107 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:51 smithi084 ceph-mon[32629]: from='osd.5 [v2:172.21.15.84:6808/1803945348,v1:172.21.15.84:6809/1803945348]' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-16T09:23:51.107 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:51 smithi084 ceph-mon[32629]: osdmap e37: 6 total, 5 up, 6 in 2024-09-16T09:23:51.107 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:51 smithi084 ceph-mon[32629]: from='osd.5 [v2:172.21.15.84:6808/1803945348,v1:172.21.15.84:6809/1803945348]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi084", "root=default"]} : dispatch 2024-09-16T09:23:51.107 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:51 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-16T09:23:51.108 INFO:journalctl@ceph.osd.5.smithi084.stdout:Sep 16 09:23:51 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-5[43256]: 2024-09-16T09:23:51.070+0000 7f394a037640 -1 osd.5 0 waiting for initial osdmap 2024-09-16T09:23:51.108 INFO:journalctl@ceph.osd.5.smithi084.stdout:Sep 16 09:23:51 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-5[43256]: 2024-09-16T09:23:51.075+0000 7f3945e5e640 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T09:23:51.314 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:23:51.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:51 smithi070 ceph-mon[30388]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T09:23:51.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:51 smithi070 ceph-mon[30388]: from='osd.5 [v2:172.21.15.84:6808/1803945348,v1:172.21.15.84:6809/1803945348]' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-16T09:23:51.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:51 smithi070 ceph-mon[30388]: osdmap e37: 6 total, 5 up, 6 in 2024-09-16T09:23:51.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:51 smithi070 ceph-mon[30388]: from='osd.5 [v2:172.21.15.84:6808/1803945348,v1:172.21.15.84:6809/1803945348]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi084", "root=default"]} : dispatch 2024-09-16T09:23:51.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:51 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-16T09:23:51.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:51 smithi070 ceph-mon[40691]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T09:23:51.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:51 smithi070 ceph-mon[40691]: from='osd.5 [v2:172.21.15.84:6808/1803945348,v1:172.21.15.84:6809/1803945348]' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-16T09:23:51.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:51 smithi070 ceph-mon[40691]: osdmap e37: 6 total, 5 up, 6 in 2024-09-16T09:23:51.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:51 smithi070 ceph-mon[40691]: from='osd.5 [v2:172.21.15.84:6808/1803945348,v1:172.21.15.84:6809/1803945348]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi084", "root=default"]} : dispatch 2024-09-16T09:23:51.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:51 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-16T09:23:52.400 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:52 smithi084 ceph-mon[32629]: from='osd.5 [v2:172.21.15.84:6808/1803945348,v1:172.21.15.84:6809/1803945348]' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]': finished 2024-09-16T09:23:52.400 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:52 smithi084 ceph-mon[32629]: osdmap e38: 6 total, 5 up, 6 in 2024-09-16T09:23:52.400 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:52 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-16T09:23:52.400 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:52 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-16T09:23:52.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:52 smithi070 ceph-mon[30388]: from='osd.5 [v2:172.21.15.84:6808/1803945348,v1:172.21.15.84:6809/1803945348]' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]': finished 2024-09-16T09:23:52.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:52 smithi070 ceph-mon[30388]: osdmap e38: 6 total, 5 up, 6 in 2024-09-16T09:23:52.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:52 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-16T09:23:52.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:52 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-16T09:23:52.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:52 smithi070 ceph-mon[40691]: from='osd.5 [v2:172.21.15.84:6808/1803945348,v1:172.21.15.84:6809/1803945348]' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]': finished 2024-09-16T09:23:52.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:52 smithi070 ceph-mon[40691]: osdmap e38: 6 total, 5 up, 6 in 2024-09-16T09:23:52.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:52 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-16T09:23:52.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:52 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-16T09:23:53.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:53 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:23:53.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:53 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:23:53.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:53 smithi084 ceph-mon[32629]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T09:23:53.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:53 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-16T09:23:53.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:53 smithi084 ceph-mon[32629]: osd.5 [v2:172.21.15.84:6808/1803945348,v1:172.21.15.84:6809/1803945348] boot 2024-09-16T09:23:53.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:53 smithi084 ceph-mon[32629]: osdmap e39: 6 total, 6 up, 6 in 2024-09-16T09:23:53.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:53 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-16T09:23:53.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:53 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:23:53.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:53 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:23:53.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:53 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:23:53.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:53 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:23:53.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:53 smithi070 ceph-mon[30388]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T09:23:53.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:53 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-16T09:23:53.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:53 smithi070 ceph-mon[30388]: osd.5 [v2:172.21.15.84:6808/1803945348,v1:172.21.15.84:6809/1803945348] boot 2024-09-16T09:23:53.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:53 smithi070 ceph-mon[30388]: osdmap e39: 6 total, 6 up, 6 in 2024-09-16T09:23:53.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:53 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-16T09:23:53.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:53 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:23:53.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:53 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:23:53.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:53 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:23:53.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:53 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:23:53.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:53 smithi070 ceph-mon[40691]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T09:23:53.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:53 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-16T09:23:53.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:53 smithi070 ceph-mon[40691]: osd.5 [v2:172.21.15.84:6808/1803945348,v1:172.21.15.84:6809/1803945348] boot 2024-09-16T09:23:53.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:53 smithi070 ceph-mon[40691]: osdmap e39: 6 total, 6 up, 6 in 2024-09-16T09:23:53.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:53 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-16T09:23:53.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:53 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:23:53.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:53 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:23:54.308 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:54 smithi084 ceph-mon[32629]: osdmap e40: 6 total, 6 up, 6 in 2024-09-16T09:23:54.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:54 smithi070 ceph-mon[30388]: osdmap e40: 6 total, 6 up, 6 in 2024-09-16T09:23:54.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:54 smithi070 ceph-mon[40691]: osdmap e40: 6 total, 6 up, 6 in 2024-09-16T09:23:55.383 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:55 smithi084 ceph-mon[32629]: pgmap v142: 1 pgs: 1 remapped; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:23:55.384 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:55 smithi084 ceph-mon[32629]: osdmap e41: 6 total, 6 up, 6 in 2024-09-16T09:23:55.384 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:55 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:23:55.384 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:55 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:23:55.384 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:55 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-16T09:23:55.384 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:55 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-16T09:23:55.384 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:55 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:23:55.384 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:55 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:23:55.384 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:55 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:23:55.384 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:55 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:23:55.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:55 smithi070 ceph-mon[30388]: pgmap v142: 1 pgs: 1 remapped; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:23:55.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:55 smithi070 ceph-mon[30388]: osdmap e41: 6 total, 6 up, 6 in 2024-09-16T09:23:55.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:55 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:23:55.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:55 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:23:55.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:55 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-16T09:23:55.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:55 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-16T09:23:55.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:55 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:23:55.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:55 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:23:55.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:55 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:23:55.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:55 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:23:55.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:55 smithi070 ceph-mon[40691]: pgmap v142: 1 pgs: 1 remapped; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:23:55.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:55 smithi070 ceph-mon[40691]: osdmap e41: 6 total, 6 up, 6 in 2024-09-16T09:23:55.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:55 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:23:55.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:55 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:23:55.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:55 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-16T09:23:55.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:55 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-16T09:23:55.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:55 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:23:55.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:55 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:23:55.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:55 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:23:55.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:55 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:23:56.018 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-16T09:23:56.044 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi084:vg_nvme/lv_2 2024-09-16T09:23:56.288 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:23:56.408 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:56 smithi084 ceph-mon[32629]: Detected new or changed devices on smithi084 2024-09-16T09:23:56.408 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:56 smithi084 ceph-mon[32629]: Adjusting osd_memory_target on smithi084 to 8544M 2024-09-16T09:23:56.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:56 smithi070 ceph-mon[30388]: Detected new or changed devices on smithi084 2024-09-16T09:23:56.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:56 smithi070 ceph-mon[30388]: Adjusting osd_memory_target on smithi084 to 8544M 2024-09-16T09:23:56.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:56 smithi070 ceph-mon[40691]: Detected new or changed devices on smithi084 2024-09-16T09:23:56.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:56 smithi070 ceph-mon[40691]: Adjusting osd_memory_target on smithi084 to 8544M 2024-09-16T09:23:57.408 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:57 smithi084 ceph-mon[32629]: pgmap v144: 1 pgs: 1 remapped; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:23:57.408 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:57 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-16T09:23:57.408 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:57 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-16T09:23:57.409 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:57 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:23:57.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:57 smithi070 ceph-mon[30388]: pgmap v144: 1 pgs: 1 remapped; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:23:57.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:57 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-16T09:23:57.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:57 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-16T09:23:57.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:57 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:23:57.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:57 smithi070 ceph-mon[40691]: pgmap v144: 1 pgs: 1 remapped; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:23:57.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:57 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-16T09:23:57.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:57 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-16T09:23:57.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:57 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:23:58.366 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:58 smithi084 ceph-mon[32629]: from='client.24244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi084:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:23:58.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:58 smithi070 ceph-mon[30388]: from='client.24244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi084:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:23:58.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:58 smithi070 ceph-mon[40691]: from='client.24244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi084:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:23:59.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:23:59 smithi084 ceph-mon[32629]: pgmap v145: 1 pgs: 1 remapped; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:23:59.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:23:59 smithi070 ceph-mon[30388]: pgmap v145: 1 pgs: 1 remapped; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:23:59.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:23:59 smithi070 ceph-mon[40691]: pgmap v145: 1 pgs: 1 remapped; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:24:00.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:00 smithi084 ceph-mon[32629]: from='client.? 172.21.15.84:0/2587589897' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "7c8f0152-f539-4b7e-8113-91dd7529d3a1"} : dispatch 2024-09-16T09:24:00.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:00 smithi084 ceph-mon[32629]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "7c8f0152-f539-4b7e-8113-91dd7529d3a1"} : dispatch 2024-09-16T09:24:00.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:00 smithi084 ceph-mon[32629]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7c8f0152-f539-4b7e-8113-91dd7529d3a1"}]': finished 2024-09-16T09:24:00.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:00 smithi084 ceph-mon[32629]: osdmap e42: 7 total, 6 up, 7 in 2024-09-16T09:24:00.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:00 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-16T09:24:00.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:00 smithi084 ceph-mon[32629]: from='client.? 172.21.15.84:0/2997750606' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-16T09:24:00.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:00 smithi070 ceph-mon[30388]: from='client.? 172.21.15.84:0/2587589897' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "7c8f0152-f539-4b7e-8113-91dd7529d3a1"} : dispatch 2024-09-16T09:24:00.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:00 smithi070 ceph-mon[30388]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "7c8f0152-f539-4b7e-8113-91dd7529d3a1"} : dispatch 2024-09-16T09:24:00.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:00 smithi070 ceph-mon[30388]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7c8f0152-f539-4b7e-8113-91dd7529d3a1"}]': finished 2024-09-16T09:24:00.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:00 smithi070 ceph-mon[30388]: osdmap e42: 7 total, 6 up, 7 in 2024-09-16T09:24:00.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:00 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-16T09:24:00.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:00 smithi070 ceph-mon[30388]: from='client.? 172.21.15.84:0/2997750606' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-16T09:24:00.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:00 smithi070 ceph-mon[40691]: from='client.? 172.21.15.84:0/2587589897' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "7c8f0152-f539-4b7e-8113-91dd7529d3a1"} : dispatch 2024-09-16T09:24:00.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:00 smithi070 ceph-mon[40691]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "7c8f0152-f539-4b7e-8113-91dd7529d3a1"} : dispatch 2024-09-16T09:24:00.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:00 smithi070 ceph-mon[40691]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7c8f0152-f539-4b7e-8113-91dd7529d3a1"}]': finished 2024-09-16T09:24:00.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:00 smithi070 ceph-mon[40691]: osdmap e42: 7 total, 6 up, 7 in 2024-09-16T09:24:00.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:00 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-16T09:24:00.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:00 smithi070 ceph-mon[40691]: from='client.? 172.21.15.84:0/2997750606' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-16T09:24:01.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:01 smithi084 ceph-mon[32629]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-09-16T09:24:01.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:01 smithi070 ceph-mon[30388]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-09-16T09:24:01.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:01 smithi070 ceph-mon[40691]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-09-16T09:24:02.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:02 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:24:02.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:02 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:24:02.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:02 smithi084 ceph-mon[32629]: osdmap e43: 7 total, 6 up, 7 in 2024-09-16T09:24:02.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:02 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-16T09:24:02.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:02 smithi084 ceph-mon[32629]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-09-16T09:24:02.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:02 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:24:02.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:02 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:24:02.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:02 smithi070 ceph-mon[30388]: osdmap e43: 7 total, 6 up, 7 in 2024-09-16T09:24:02.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:02 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-16T09:24:02.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:02 smithi070 ceph-mon[30388]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-09-16T09:24:02.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:02 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:24:02.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:02 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:24:02.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:02 smithi070 ceph-mon[40691]: osdmap e43: 7 total, 6 up, 7 in 2024-09-16T09:24:02.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:02 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-16T09:24:02.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:02 smithi070 ceph-mon[40691]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-09-16T09:24:04.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:04 smithi084 ceph-mon[32629]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-16T09:24:04.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:04 smithi070 ceph-mon[30388]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-16T09:24:04.670 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:04 smithi070 ceph-mon[40691]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-16T09:24:06.619 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:06 smithi084 ceph-mon[32629]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-16T09:24:06.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:06 smithi070 ceph-mon[30388]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-16T09:24:06.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:06 smithi070 ceph-mon[40691]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-16T09:24:07.638 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:07 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-16T09:24:07.638 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:07 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:24:07.638 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:07 smithi084 ceph-mon[32629]: Deploying daemon osd.6 on smithi084 2024-09-16T09:24:07.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:07 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-16T09:24:07.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:07 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:24:07.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:07 smithi070 ceph-mon[30388]: Deploying daemon osd.6 on smithi084 2024-09-16T09:24:07.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:07 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-16T09:24:07.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:07 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:24:07.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:07 smithi070 ceph-mon[40691]: Deploying daemon osd.6 on smithi084 2024-09-16T09:24:08.550 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:08 smithi084 ceph-mon[32629]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:24:08.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:08 smithi070 ceph-mon[30388]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:24:08.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:08 smithi070 ceph-mon[40691]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:24:11.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:11 smithi084 ceph-mon[32629]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:24:11.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:11 smithi070 ceph-mon[30388]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:24:11.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:11 smithi070 ceph-mon[40691]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:24:13.141 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:13 smithi084 ceph-mon[32629]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:24:13.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:13 smithi070 ceph-mon[30388]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:24:13.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:13 smithi070 ceph-mon[40691]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:24:15.269 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:15 smithi084 ceph-mon[32629]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:24:15.270 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:15 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:24:15.270 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:15 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:24:15.270 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:15 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:24:15.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:15 smithi070 ceph-mon[30388]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:24:15.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:15 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:24:15.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:15 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:24:15.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:15 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:24:15.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:15 smithi070 ceph-mon[40691]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:24:15.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:15 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:24:15.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:15 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:24:15.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:15 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:24:17.409 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:17 smithi084 ceph-mon[32629]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:24:17.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:17 smithi070 ceph-mon[30388]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:24:17.420 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:17 smithi070 ceph-mon[40691]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:24:18.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:18 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:24:18.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:18 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:24:18.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:18 smithi084 ceph-mon[32629]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:24:18.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:18 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:24:18.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:18 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:24:18.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:18 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:24:18.670 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:18 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:24:18.670 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:18 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:24:18.670 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:18 smithi070 ceph-mon[40691]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:24:18.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:18 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:24:18.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:18 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:24:18.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:18 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:24:18.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:18 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:24:18.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:18 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:24:18.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:18 smithi070 ceph-mon[30388]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:24:18.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:18 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:24:18.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:18 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:24:18.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:18 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:24:19.383 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:19 smithi084 ceph-mon[32629]: from='osd.6 [v2:172.21.15.84:6816/610727189,v1:172.21.15.84:6817/610727189]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-16T09:24:19.384 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:19 smithi084 ceph-mon[32629]: from='osd.6 ' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-16T09:24:19.422 INFO:teuthology.orchestra.run.smithi084.stdout:Created osd(s) 6 on host 'smithi084' 2024-09-16T09:24:19.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:19 smithi070 ceph-mon[30388]: from='osd.6 [v2:172.21.15.84:6816/610727189,v1:172.21.15.84:6817/610727189]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-16T09:24:19.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:19 smithi070 ceph-mon[30388]: from='osd.6 ' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-16T09:24:19.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:19 smithi070 ceph-mon[40691]: from='osd.6 [v2:172.21.15.84:6816/610727189,v1:172.21.15.84:6817/610727189]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-16T09:24:19.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:19 smithi070 ceph-mon[40691]: from='osd.6 ' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-16T09:24:20.067 DEBUG:teuthology.orchestra.run.smithi084:osd.6> sudo journalctl -f -n 0 -u ceph-a1321834-740c-11ef-bceb-c7b262605968@osd.6.service 2024-09-16T09:24:20.070 INFO:tasks.cephadm:Deploying osd.7 on smithi084 with /dev/vg_nvme/lv_1... 2024-09-16T09:24:20.070 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a1321834-740c-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-16T09:24:20.340 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:20 smithi084 ceph-mon[32629]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-16T09:24:20.340 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:20 smithi084 ceph-mon[32629]: osdmap e44: 7 total, 6 up, 7 in 2024-09-16T09:24:20.340 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:20 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-16T09:24:20.341 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:20 smithi084 ceph-mon[32629]: from='osd.6 [v2:172.21.15.84:6816/610727189,v1:172.21.15.84:6817/610727189]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi084", "root=default"]} : dispatch 2024-09-16T09:24:20.341 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:20 smithi084 ceph-mon[32629]: from='osd.6 ' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi084", "root=default"]} : dispatch 2024-09-16T09:24:20.341 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:20 smithi084 ceph-mon[32629]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:24:20.341 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:20 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:24:20.341 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:20 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:24:20.341 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:20 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:24:20.598 INFO:journalctl@ceph.osd.6.smithi084.stdout:Sep 16 09:24:20 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-6[49407]: 2024-09-16T09:24:20.341+0000 7f9154aa8640 -1 osd.6 0 waiting for initial osdmap 2024-09-16T09:24:20.598 INFO:journalctl@ceph.osd.6.smithi084.stdout:Sep 16 09:24:20 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-6[49407]: 2024-09-16T09:24:20.347+0000 7f91508cf640 -1 osd.6 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T09:24:20.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:20 smithi070 ceph-mon[40691]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-16T09:24:20.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:20 smithi070 ceph-mon[40691]: osdmap e44: 7 total, 6 up, 7 in 2024-09-16T09:24:20.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:20 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-16T09:24:20.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:20 smithi070 ceph-mon[40691]: from='osd.6 [v2:172.21.15.84:6816/610727189,v1:172.21.15.84:6817/610727189]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi084", "root=default"]} : dispatch 2024-09-16T09:24:20.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:20 smithi070 ceph-mon[40691]: from='osd.6 ' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi084", "root=default"]} : dispatch 2024-09-16T09:24:20.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:20 smithi070 ceph-mon[40691]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:24:20.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:20 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:24:20.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:20 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:24:20.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:20 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:24:20.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:20 smithi070 ceph-mon[30388]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-16T09:24:20.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:20 smithi070 ceph-mon[30388]: osdmap e44: 7 total, 6 up, 7 in 2024-09-16T09:24:20.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:20 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-16T09:24:20.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:20 smithi070 ceph-mon[30388]: from='osd.6 [v2:172.21.15.84:6816/610727189,v1:172.21.15.84:6817/610727189]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi084", "root=default"]} : dispatch 2024-09-16T09:24:20.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:20 smithi070 ceph-mon[30388]: from='osd.6 ' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi084", "root=default"]} : dispatch 2024-09-16T09:24:20.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:20 smithi070 ceph-mon[30388]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T09:24:20.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:20 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:24:20.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:20 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:24:20.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:20 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:24:21.202 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:24:21.407 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:21 smithi084 ceph-mon[32629]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]': finished 2024-09-16T09:24:21.407 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:21 smithi084 ceph-mon[32629]: osdmap e45: 7 total, 6 up, 7 in 2024-09-16T09:24:21.408 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:21 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-16T09:24:21.408 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:21 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-16T09:24:21.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:21 smithi070 ceph-mon[30388]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]': finished 2024-09-16T09:24:21.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:21 smithi070 ceph-mon[30388]: osdmap e45: 7 total, 6 up, 7 in 2024-09-16T09:24:21.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:21 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-16T09:24:21.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:21 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-16T09:24:21.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:21 smithi070 ceph-mon[40691]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]': finished 2024-09-16T09:24:21.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:21 smithi070 ceph-mon[40691]: osdmap e45: 7 total, 6 up, 7 in 2024-09-16T09:24:21.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:21 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-16T09:24:21.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:21 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-16T09:24:22.627 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:22 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:24:22.627 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:22 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:24:22.627 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:22 smithi084 ceph-mon[32629]: osd.6 [v2:172.21.15.84:6816/610727189,v1:172.21.15.84:6817/610727189] boot 2024-09-16T09:24:22.627 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:22 smithi084 ceph-mon[32629]: osdmap e46: 7 total, 7 up, 7 in 2024-09-16T09:24:22.627 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:22 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-16T09:24:22.627 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:22 smithi084 ceph-mon[32629]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-16T09:24:22.670 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:22 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:24:22.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:22 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:24:22.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:22 smithi070 ceph-mon[40691]: osd.6 [v2:172.21.15.84:6816/610727189,v1:172.21.15.84:6817/610727189] boot 2024-09-16T09:24:22.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:22 smithi070 ceph-mon[40691]: osdmap e46: 7 total, 7 up, 7 in 2024-09-16T09:24:22.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:22 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-16T09:24:22.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:22 smithi070 ceph-mon[40691]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-16T09:24:22.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:22 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:24:22.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:22 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:24:22.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:22 smithi070 ceph-mon[30388]: osd.6 [v2:172.21.15.84:6816/610727189,v1:172.21.15.84:6817/610727189] boot 2024-09-16T09:24:22.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:22 smithi070 ceph-mon[30388]: osdmap e46: 7 total, 7 up, 7 in 2024-09-16T09:24:22.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:22 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-16T09:24:22.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:22 smithi070 ceph-mon[30388]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 201 MiB used, 626 GiB / 626 GiB avail 2024-09-16T09:24:23.563 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:23 smithi084 ceph-mon[32629]: osdmap e47: 7 total, 7 up, 7 in 2024-09-16T09:24:23.563 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:23 smithi084 ceph-mon[32629]: Detected new or changed devices on smithi084 2024-09-16T09:24:23.563 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:23 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:24:23.563 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:23 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:24:23.563 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:23 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-16T09:24:23.563 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:23 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-16T09:24:23.563 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:23 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-16T09:24:23.563 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:23 smithi084 ceph-mon[32629]: Adjusting osd_memory_target on smithi084 to 5696M 2024-09-16T09:24:23.563 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:23 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:24:23.564 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:23 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:24:23.564 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:23 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:24:23.564 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:23 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:24:23.564 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:23 smithi084 ceph-mon[32629]: osdmap e48: 7 total, 7 up, 7 in 2024-09-16T09:24:23.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:23 smithi070 ceph-mon[30388]: osdmap e47: 7 total, 7 up, 7 in 2024-09-16T09:24:23.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:23 smithi070 ceph-mon[30388]: Detected new or changed devices on smithi084 2024-09-16T09:24:23.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:23 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:24:23.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:23 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:24:23.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:23 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-16T09:24:23.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:23 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-16T09:24:23.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:23 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-16T09:24:23.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:23 smithi070 ceph-mon[30388]: Adjusting osd_memory_target on smithi084 to 5696M 2024-09-16T09:24:23.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:23 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:24:23.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:23 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:24:23.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:23 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:24:23.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:23 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:24:23.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:23 smithi070 ceph-mon[30388]: osdmap e48: 7 total, 7 up, 7 in 2024-09-16T09:24:23.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:23 smithi070 ceph-mon[40691]: osdmap e47: 7 total, 7 up, 7 in 2024-09-16T09:24:23.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:23 smithi070 ceph-mon[40691]: Detected new or changed devices on smithi084 2024-09-16T09:24:23.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:23 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:24:23.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:23 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:24:23.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:23 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-16T09:24:23.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:23 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-16T09:24:23.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:23 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-16T09:24:23.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:23 smithi070 ceph-mon[40691]: Adjusting osd_memory_target on smithi084 to 5696M 2024-09-16T09:24:23.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:23 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:24:23.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:23 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:24:23.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:23 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:24:23.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:23 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:24:23.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:23 smithi070 ceph-mon[40691]: osdmap e48: 7 total, 7 up, 7 in 2024-09-16T09:24:25.033 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-09-16T09:24:25.061 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi084:vg_nvme/lv_1 2024-09-16T09:24:25.307 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:24:25.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:25 smithi084 ceph-mon[32629]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-16T09:24:25.420 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:25 smithi070 ceph-mon[40691]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-16T09:24:25.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:25 smithi070 ceph-mon[30388]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-16T09:24:26.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:26 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-16T09:24:26.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:26 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-16T09:24:26.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:26 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:24:26.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:26 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-16T09:24:26.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:26 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-16T09:24:26.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:26 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:24:26.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:26 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-16T09:24:26.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:26 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-16T09:24:26.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:26 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:24:27.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:27 smithi084 ceph-mon[32629]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-16T09:24:27.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:27 smithi084 ceph-mon[32629]: from='client.24271 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi084:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:24:27.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:27 smithi070 ceph-mon[30388]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-16T09:24:27.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:27 smithi070 ceph-mon[30388]: from='client.24271 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi084:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:24:27.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:27 smithi070 ceph-mon[40691]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-16T09:24:27.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:27 smithi070 ceph-mon[40691]: from='client.24271 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi084:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:24:29.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:29 smithi084 ceph-mon[32629]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-16T09:24:29.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:29 smithi084 ceph-mon[32629]: from='client.? 172.21.15.84:0/2899906537' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "945efaba-225e-49af-aefd-b4d58ca8093b"} : dispatch 2024-09-16T09:24:29.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:29 smithi084 ceph-mon[32629]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "945efaba-225e-49af-aefd-b4d58ca8093b"} : dispatch 2024-09-16T09:24:29.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:29 smithi084 ceph-mon[32629]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "945efaba-225e-49af-aefd-b4d58ca8093b"}]': finished 2024-09-16T09:24:29.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:29 smithi084 ceph-mon[32629]: osdmap e49: 8 total, 7 up, 8 in 2024-09-16T09:24:29.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:29 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-16T09:24:29.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:29 smithi084 ceph-mon[32629]: from='client.? 172.21.15.84:0/530259651' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-16T09:24:29.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:29 smithi070 ceph-mon[30388]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-16T09:24:29.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:29 smithi070 ceph-mon[30388]: from='client.? 172.21.15.84:0/2899906537' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "945efaba-225e-49af-aefd-b4d58ca8093b"} : dispatch 2024-09-16T09:24:29.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:29 smithi070 ceph-mon[30388]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "945efaba-225e-49af-aefd-b4d58ca8093b"} : dispatch 2024-09-16T09:24:29.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:29 smithi070 ceph-mon[30388]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "945efaba-225e-49af-aefd-b4d58ca8093b"}]': finished 2024-09-16T09:24:29.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:29 smithi070 ceph-mon[30388]: osdmap e49: 8 total, 7 up, 8 in 2024-09-16T09:24:29.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:29 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-16T09:24:29.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:29 smithi070 ceph-mon[30388]: from='client.? 172.21.15.84:0/530259651' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-16T09:24:29.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:29 smithi070 ceph-mon[40691]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-16T09:24:29.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:29 smithi070 ceph-mon[40691]: from='client.? 172.21.15.84:0/2899906537' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "945efaba-225e-49af-aefd-b4d58ca8093b"} : dispatch 2024-09-16T09:24:29.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:29 smithi070 ceph-mon[40691]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "945efaba-225e-49af-aefd-b4d58ca8093b"} : dispatch 2024-09-16T09:24:29.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:29 smithi070 ceph-mon[40691]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "945efaba-225e-49af-aefd-b4d58ca8093b"}]': finished 2024-09-16T09:24:29.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:29 smithi070 ceph-mon[40691]: osdmap e49: 8 total, 7 up, 8 in 2024-09-16T09:24:29.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:29 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-16T09:24:29.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:29 smithi070 ceph-mon[40691]: from='client.? 172.21.15.84:0/530259651' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-16T09:24:31.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:31 smithi084 ceph-mon[32629]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-16T09:24:31.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:31 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:24:31.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:31 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:24:31.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:31 smithi070 ceph-mon[30388]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-16T09:24:31.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:31 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:24:31.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:31 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:24:31.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:31 smithi070 ceph-mon[40691]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-16T09:24:31.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:31 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:24:31.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:31 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:24:32.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:32 smithi084 ceph-mon[32629]: osdmap e50: 8 total, 7 up, 8 in 2024-09-16T09:24:32.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:32 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-16T09:24:32.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:32 smithi070 ceph-mon[30388]: osdmap e50: 8 total, 7 up, 8 in 2024-09-16T09:24:32.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:32 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-16T09:24:32.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:32 smithi070 ceph-mon[40691]: osdmap e50: 8 total, 7 up, 8 in 2024-09-16T09:24:32.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:32 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-16T09:24:33.320 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:33 smithi084 ceph-mon[32629]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-16T09:24:33.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:33 smithi070 ceph-mon[30388]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-16T09:24:33.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:33 smithi070 ceph-mon[40691]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-16T09:24:35.350 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:35 smithi084 ceph-mon[32629]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-16T09:24:35.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:35 smithi070 ceph-mon[30388]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-16T09:24:35.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:35 smithi070 ceph-mon[40691]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-16T09:24:36.341 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:36 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-16T09:24:36.342 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:36 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:24:36.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:36 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-16T09:24:36.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:36 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:24:36.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:36 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-16T09:24:36.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:36 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:24:37.338 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:37 smithi084 ceph-mon[32629]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-16T09:24:37.338 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:37 smithi084 ceph-mon[32629]: Deploying daemon osd.7 on smithi084 2024-09-16T09:24:37.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:37 smithi070 ceph-mon[40691]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-16T09:24:37.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:37 smithi070 ceph-mon[40691]: Deploying daemon osd.7 on smithi084 2024-09-16T09:24:37.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:37 smithi070 ceph-mon[30388]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-16T09:24:37.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:37 smithi070 ceph-mon[30388]: Deploying daemon osd.7 on smithi084 2024-09-16T09:24:39.362 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:39 smithi084 ceph-mon[32629]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-16T09:24:39.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:39 smithi070 ceph-mon[30388]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-16T09:24:39.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:39 smithi070 ceph-mon[40691]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-16T09:24:41.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:41 smithi084 ceph-mon[32629]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-16T09:24:41.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:41 smithi070 ceph-mon[30388]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-16T09:24:41.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:41 smithi070 ceph-mon[40691]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-16T09:24:43.297 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:43 smithi084 ceph-mon[32629]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-16T09:24:43.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:43 smithi070 ceph-mon[30388]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-16T09:24:43.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:43 smithi070 ceph-mon[40691]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-16T09:24:44.332 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:44 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:24:44.332 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:44 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:24:44.332 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:44 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:24:44.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:44 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:24:44.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:44 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:24:44.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:44 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:24:44.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:44 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:24:44.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:44 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:24:44.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:44 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:24:45.225 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:45 smithi084 ceph-mon[32629]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-16T09:24:45.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:45 smithi070 ceph-mon[30388]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-16T09:24:45.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:45 smithi070 ceph-mon[40691]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-16T09:24:46.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:46 smithi084 ceph-mon[32629]: 1.0 scrub starts 2024-09-16T09:24:46.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:46 smithi084 ceph-mon[32629]: 1.0 scrub ok 2024-09-16T09:24:46.420 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:46 smithi070 ceph-mon[40691]: 1.0 scrub starts 2024-09-16T09:24:46.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:46 smithi070 ceph-mon[40691]: 1.0 scrub ok 2024-09-16T09:24:46.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:46 smithi070 ceph-mon[30388]: 1.0 scrub starts 2024-09-16T09:24:46.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:46 smithi070 ceph-mon[30388]: 1.0 scrub ok 2024-09-16T09:24:47.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:47 smithi084 ceph-mon[32629]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-16T09:24:47.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:47 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:24:47.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:47 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:24:47.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:47 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:24:47.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:47 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:24:47.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:47 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:24:47.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:47 smithi070 ceph-mon[30388]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-16T09:24:47.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:47 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:24:47.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:47 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:24:47.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:47 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:24:47.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:47 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:24:47.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:47 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:24:47.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:47 smithi070 ceph-mon[40691]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-16T09:24:47.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:47 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:24:47.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:47 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:24:47.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:47 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:24:47.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:47 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:24:47.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:47 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:24:48.723 INFO:teuthology.orchestra.run.smithi084.stdout:Created osd(s) 7 on host 'smithi084' 2024-09-16T09:24:49.265 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:49 smithi084 ceph-mon[32629]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-16T09:24:49.265 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:49 smithi084 ceph-mon[32629]: from='osd.7 [v2:172.21.15.84:6824/1002252531,v1:172.21.15.84:6825/1002252531]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-16T09:24:49.265 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:49 smithi084 ceph-mon[32629]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-16T09:24:49.265 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:49 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:24:49.266 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:49 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:24:49.266 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:49 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:24:49.291 DEBUG:teuthology.orchestra.run.smithi084:osd.7> sudo journalctl -f -n 0 -u ceph-a1321834-740c-11ef-bceb-c7b262605968@osd.7.service 2024-09-16T09:24:49.294 INFO:tasks.cephadm:Waiting for 8 OSDs to come up... 2024-09-16T09:24:49.294 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-16T09:24:49.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:49 smithi070 ceph-mon[30388]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-16T09:24:49.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:49 smithi070 ceph-mon[30388]: from='osd.7 [v2:172.21.15.84:6824/1002252531,v1:172.21.15.84:6825/1002252531]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-16T09:24:49.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:49 smithi070 ceph-mon[30388]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-16T09:24:49.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:49 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:24:49.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:49 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:24:49.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:49 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:24:49.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:49 smithi070 ceph-mon[40691]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-16T09:24:49.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:49 smithi070 ceph-mon[40691]: from='osd.7 [v2:172.21.15.84:6824/1002252531,v1:172.21.15.84:6825/1002252531]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-16T09:24:49.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:49 smithi070 ceph-mon[40691]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-16T09:24:49.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:49 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:24:49.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:49 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:24:49.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:49 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:24:49.539 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:24:50.286 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:24:50.286 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:50 smithi070 ceph-mon[40691]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-16T09:24:50.286 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:50 smithi070 ceph-mon[40691]: osdmap e51: 8 total, 7 up, 8 in 2024-09-16T09:24:50.286 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:50 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-16T09:24:50.286 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:50 smithi070 ceph-mon[40691]: from='osd.7 [v2:172.21.15.84:6824/1002252531,v1:172.21.15.84:6825/1002252531]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi084", "root=default"]} : dispatch 2024-09-16T09:24:50.286 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:50 smithi070 ceph-mon[40691]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi084", "root=default"]} : dispatch 2024-09-16T09:24:50.287 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:50 smithi070 ceph-mon[30388]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-16T09:24:50.287 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:50 smithi070 ceph-mon[30388]: osdmap e51: 8 total, 7 up, 8 in 2024-09-16T09:24:50.287 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:50 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-16T09:24:50.287 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:50 smithi070 ceph-mon[30388]: from='osd.7 [v2:172.21.15.84:6824/1002252531,v1:172.21.15.84:6825/1002252531]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi084", "root=default"]} : dispatch 2024-09-16T09:24:50.287 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:50 smithi070 ceph-mon[30388]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi084", "root=default"]} : dispatch 2024-09-16T09:24:50.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:50 smithi084 ceph-mon[32629]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-16T09:24:50.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:50 smithi084 ceph-mon[32629]: osdmap e51: 8 total, 7 up, 8 in 2024-09-16T09:24:50.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:50 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-16T09:24:50.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:50 smithi084 ceph-mon[32629]: from='osd.7 [v2:172.21.15.84:6824/1002252531,v1:172.21.15.84:6825/1002252531]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi084", "root=default"]} : dispatch 2024-09-16T09:24:50.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:50 smithi084 ceph-mon[32629]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi084", "root=default"]} : dispatch 2024-09-16T09:24:50.824 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:24:50 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:24:50.557+0000 7fa951851640 -1 osd.7 0 waiting for initial osdmap 2024-09-16T09:24:50.825 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:24:50 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:24:50.564+0000 7fa948e5c640 -1 osd.7 52 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T09:24:51.005 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":52,"num_osds":8,"num_up_osds":7,"osd_up_since":1726478661,"num_in_osds":8,"osd_in_since":1726478668,"num_remapped_pgs":0} 2024-09-16T09:24:51.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:51 smithi084 ceph-mon[32629]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-16T09:24:51.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:51 smithi084 ceph-mon[32629]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]': finished 2024-09-16T09:24:51.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:51 smithi084 ceph-mon[32629]: osdmap e52: 8 total, 7 up, 8 in 2024-09-16T09:24:51.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:51 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-16T09:24:51.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:51 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-16T09:24:51.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:51 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1514259182' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-16T09:24:51.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:51 smithi084 ceph-mon[32629]: from='osd.7 ' entity='osd.7' 2024-09-16T09:24:51.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:51 smithi070 ceph-mon[30388]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-16T09:24:51.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:51 smithi070 ceph-mon[30388]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]': finished 2024-09-16T09:24:51.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:51 smithi070 ceph-mon[30388]: osdmap e52: 8 total, 7 up, 8 in 2024-09-16T09:24:51.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:51 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-16T09:24:51.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:51 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-16T09:24:51.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:51 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1514259182' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-16T09:24:51.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:51 smithi070 ceph-mon[30388]: from='osd.7 ' entity='osd.7' 2024-09-16T09:24:51.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:51 smithi070 ceph-mon[40691]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 626 GiB / 626 GiB avail 2024-09-16T09:24:51.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:51 smithi070 ceph-mon[40691]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]': finished 2024-09-16T09:24:51.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:51 smithi070 ceph-mon[40691]: osdmap e52: 8 total, 7 up, 8 in 2024-09-16T09:24:51.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:51 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-16T09:24:51.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:51 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-16T09:24:51.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:51 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1514259182' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-16T09:24:51.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:51 smithi070 ceph-mon[40691]: from='osd.7 ' entity='osd.7' 2024-09-16T09:24:52.007 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-16T09:24:52.270 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:24:52.391 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:52 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:24:52.391 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:52 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:24:52.391 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:52 smithi070 ceph-mon[30388]: osd.7 [v2:172.21.15.84:6824/1002252531,v1:172.21.15.84:6825/1002252531] boot 2024-09-16T09:24:52.391 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:52 smithi070 ceph-mon[30388]: osdmap e53: 8 total, 8 up, 8 in 2024-09-16T09:24:52.391 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:52 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-16T09:24:52.391 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:52 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:24:52.391 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:52 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:24:52.391 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:52 smithi070 ceph-mon[30388]: osdmap e54: 8 total, 8 up, 8 in 2024-09-16T09:24:52.391 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:52 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:24:52.392 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:52 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:24:52.392 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:52 smithi070 ceph-mon[40691]: osd.7 [v2:172.21.15.84:6824/1002252531,v1:172.21.15.84:6825/1002252531] boot 2024-09-16T09:24:52.392 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:52 smithi070 ceph-mon[40691]: osdmap e53: 8 total, 8 up, 8 in 2024-09-16T09:24:52.392 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:52 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-16T09:24:52.392 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:52 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:24:52.392 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:52 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:24:52.392 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:52 smithi070 ceph-mon[40691]: osdmap e54: 8 total, 8 up, 8 in 2024-09-16T09:24:52.409 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:52 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:24:52.409 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:52 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:24:52.409 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:52 smithi084 ceph-mon[32629]: osd.7 [v2:172.21.15.84:6824/1002252531,v1:172.21.15.84:6825/1002252531] boot 2024-09-16T09:24:52.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:52 smithi084 ceph-mon[32629]: osdmap e53: 8 total, 8 up, 8 in 2024-09-16T09:24:52.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:52 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-16T09:24:52.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:52 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:24:52.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:52 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:24:52.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:52 smithi084 ceph-mon[32629]: osdmap e54: 8 total, 8 up, 8 in 2024-09-16T09:24:52.913 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:24:53.170 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:53 smithi070 ceph-mon[30388]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-16T09:24:53.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:53 smithi070 ceph-mon[30388]: Detected new or changed devices on smithi084 2024-09-16T09:24:53.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:53 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:24:53.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:53 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:24:53.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:53 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-16T09:24:53.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:53 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-16T09:24:53.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:53 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-16T09:24:53.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:53 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-16T09:24:53.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:53 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:24:53.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:53 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:24:53.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:53 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:24:53.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:53 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:24:53.172 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:53 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2605463657' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-16T09:24:53.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:53 smithi070 ceph-mon[40691]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-16T09:24:53.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:53 smithi070 ceph-mon[40691]: Detected new or changed devices on smithi084 2024-09-16T09:24:53.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:53 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:24:53.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:53 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:24:53.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:53 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-16T09:24:53.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:53 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-16T09:24:53.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:53 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-16T09:24:53.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:53 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-16T09:24:53.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:53 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:24:53.173 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:53 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:24:53.173 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:53 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:24:53.173 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:53 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:24:53.173 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:53 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/2605463657' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-16T09:24:53.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:53 smithi084 ceph-mon[32629]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-16T09:24:53.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:53 smithi084 ceph-mon[32629]: Detected new or changed devices on smithi084 2024-09-16T09:24:53.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:53 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:24:53.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:53 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:24:53.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:53 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-16T09:24:53.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:53 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-16T09:24:53.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:53 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-16T09:24:53.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:53 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-16T09:24:53.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:53 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:24:53.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:53 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:24:53.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:53 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:24:53.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:53 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:24:53.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:53 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/2605463657' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-16T09:24:53.545 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":54,"num_osds":8,"num_up_osds":8,"osd_up_since":1726478691,"num_in_osds":8,"osd_in_since":1726478668,"num_remapped_pgs":0} 2024-09-16T09:24:53.546 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T09:24:53.787 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:24:54.170 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:54 smithi070 ceph-mon[30388]: Adjusting osd_memory_target on smithi084 to 4272M 2024-09-16T09:24:54.170 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:54 smithi070 ceph-mon[30388]: osdmap e55: 8 total, 8 up, 8 in 2024-09-16T09:24:54.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:54 smithi070 ceph-mon[40691]: Adjusting osd_memory_target on smithi084 to 4272M 2024-09-16T09:24:54.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:54 smithi070 ceph-mon[40691]: osdmap e55: 8 total, 8 up, 8 in 2024-09-16T09:24:54.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:54 smithi084 ceph-mon[32629]: Adjusting osd_memory_target on smithi084 to 4272M 2024-09-16T09:24:54.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:54 smithi084 ceph-mon[32629]: osdmap e55: 8 total, 8 up, 8 in 2024-09-16T09:24:54.478 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:24:54.478 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":55,"fsid":"a1321834-740c-11ef-bceb-c7b262605968","created":"2024-09-16T09:19:14.121498+0000","modified":"2024-09-16T09:24:53.130540+0000","last_up_change":"2024-09-16T09:24:51.124496+0000","last_in_change":"2024-09-16T09:24:28.432920+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T09:22:27.617694+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7ef288c8-3bf0-4814-8e86-f70a7de041ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":2013075781},{"type":"v1","addr":"172.21.15.70:6803","nonce":2013075781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":2013075781},{"type":"v1","addr":"172.21.15.70:6805","nonce":2013075781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":2013075781},{"type":"v1","addr":"172.21.15.70:6809","nonce":2013075781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":2013075781},{"type":"v1","addr":"172.21.15.70:6807","nonce":2013075781}]},"public_addr":"172.21.15.70:6803/2013075781","cluster_addr":"172.21.15.70:6805/2013075781","heartbeat_back_addr":"172.21.15.70:6809/2013075781","heartbeat_front_addr":"172.21.15.70:6807/2013075781","state":["exists","up"]},{"osd":1,"uuid":"474d41d9-295a-450c-8f27-ce487a14678b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":1670012115},{"type":"v1","addr":"172.21.15.70:6811","nonce":1670012115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":1670012115},{"type":"v1","addr":"172.21.15.70:6813","nonce":1670012115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1670012115},{"type":"v1","addr":"172.21.15.70:6817","nonce":1670012115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":1670012115},{"type":"v1","addr":"172.21.15.70:6815","nonce":1670012115}]},"public_addr":"172.21.15.70:6811/1670012115","cluster_addr":"172.21.15.70:6813/1670012115","heartbeat_back_addr":"172.21.15.70:6817/1670012115","heartbeat_front_addr":"172.21.15.70:6815/1670012115","state":["exists","up"]},{"osd":2,"uuid":"5b4ca4e8-cecc-4163-b291-c9209b02c4b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3601682336},{"type":"v1","addr":"172.21.15.70:6819","nonce":3601682336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":3601682336},{"type":"v1","addr":"172.21.15.70:6821","nonce":3601682336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":3601682336},{"type":"v1","addr":"172.21.15.70:6825","nonce":3601682336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3601682336},{"type":"v1","addr":"172.21.15.70:6823","nonce":3601682336}]},"public_addr":"172.21.15.70:6819/3601682336","cluster_addr":"172.21.15.70:6821/3601682336","heartbeat_back_addr":"172.21.15.70:6825/3601682336","heartbeat_front_addr":"172.21.15.70:6823/3601682336","state":["exists","up"]},{"osd":3,"uuid":"ffb8d527-1b58-4ecd-808c-47d4ba1bf387","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2038083007},{"type":"v1","addr":"172.21.15.70:6827","nonce":2038083007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2038083007},{"type":"v1","addr":"172.21.15.70:6829","nonce":2038083007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":2038083007},{"type":"v1","addr":"172.21.15.70:6833","nonce":2038083007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2038083007},{"type":"v1","addr":"172.21.15.70:6831","nonce":2038083007}]},"public_addr":"172.21.15.70:6827/2038083007","cluster_addr":"172.21.15.70:6829/2038083007","heartbeat_back_addr":"172.21.15.70:6833/2038083007","heartbeat_front_addr":"172.21.15.70:6831/2038083007","state":["exists","up"]},{"osd":4,"uuid":"a946760b-4d91-4a4b-8e9b-a07a50589211","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2250892710},{"type":"v1","addr":"172.21.15.84:6801","nonce":2250892710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2250892710},{"type":"v1","addr":"172.21.15.84:6803","nonce":2250892710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2250892710},{"type":"v1","addr":"172.21.15.84:6807","nonce":2250892710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2250892710},{"type":"v1","addr":"172.21.15.84:6805","nonce":2250892710}]},"public_addr":"172.21.15.84:6801/2250892710","cluster_addr":"172.21.15.84:6803/2250892710","heartbeat_back_addr":"172.21.15.84:6807/2250892710","heartbeat_front_addr":"172.21.15.84:6805/2250892710","state":["exists","up"]},{"osd":5,"uuid":"67681760-75a9-4846-946e-a935d42c5e05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":1803945348},{"type":"v1","addr":"172.21.15.84:6809","nonce":1803945348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":1803945348},{"type":"v1","addr":"172.21.15.84:6811","nonce":1803945348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":1803945348},{"type":"v1","addr":"172.21.15.84:6815","nonce":1803945348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":1803945348},{"type":"v1","addr":"172.21.15.84:6813","nonce":1803945348}]},"public_addr":"172.21.15.84:6809/1803945348","cluster_addr":"172.21.15.84:6811/1803945348","heartbeat_back_addr":"172.21.15.84:6815/1803945348","heartbeat_front_addr":"172.21.15.84:6813/1803945348","state":["exists","up"]},{"osd":6,"uuid":"7c8f0152-f539-4b7e-8113-91dd7529d3a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":610727189},{"type":"v1","addr":"172.21.15.84:6817","nonce":610727189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":610727189},{"type":"v1","addr":"172.21.15.84:6819","nonce":610727189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":610727189},{"type":"v1","addr":"172.21.15.84:6823","nonce":610727189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":610727189},{"type":"v1","addr":"172.21.15.84:6821","nonce":610727189}]},"public_addr":"172.21.15.84:6817/610727189","cluster_addr":"172.21.15.84:6819/610727189","heartbeat_back_addr":"172.21.15.84:6823/610727189","heartbeat_front_addr":"172.21.15.84:6821/610727189","state":["exists","up"]},{"osd":7,"uuid":"945efaba-225e-49af-aefd-b4d58ca8093b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":1002252531},{"type":"v1","addr":"172.21.15.84:6825","nonce":1002252531}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1002252531},{"type":"v1","addr":"172.21.15.84:6827","nonce":1002252531}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":1002252531},{"type":"v1","addr":"172.21.15.84:6831","nonce":1002252531}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":1002252531},{"type":"v1","addr":"172.21.15.84:6829","nonce":1002252531}]},"public_addr":"172.21.15.84:6825/1002252531","cluster_addr":"172.21.15.84:6827/1002252531","heartbeat_back_addr":"172.21.15.84:6831/1002252531","heartbeat_front_addr":"172.21.15.84:6829/1002252531","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T09:23:52.227757+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T09:24:30.453059+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T09:23:36.945994+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T09:24:00.739156+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T09:24:51.788424+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T09:23:50.745474+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T09:24:19.886645+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T09:24:49.178172+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.70:0/1403001486":"2024-09-17T09:20:09.306330+0000","172.21.15.70:6801/3871553358":"2024-09-17T09:20:09.306330+0000","172.21.15.70:6800/3871553358":"2024-09-17T09:20:09.306330+0000","172.21.15.70:0/3351688485":"2024-09-17T09:20:09.306330+0000","172.21.15.70:6801/837338094":"2024-09-17T09:19:43.205989+0000","172.21.15.70:6800/837338094":"2024-09-17T09:19:43.205989+0000","172.21.15.70:0/1812373760":"2024-09-17T09:20:09.306330+0000","172.21.15.70:0/2213852865":"2024-09-17T09:19:43.205989+0000","172.21.15.70:0/186565817":"2024-09-17T09:19:43.205989+0000","172.21.15.70:0/3354284852":"2024-09-17T09:19:43.205989+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T09:24:55.139 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T09:22:27.617694+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 7.889999866485596, 'score_stable': 7.889999866485596, 'optimal_score': 0.3799999952316284, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-16T09:24:55.140 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-16T09:24:55.385 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:24:55.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:55 smithi084 ceph-mon[32629]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-16T09:24:55.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:55 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/439001110' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-16T09:24:55.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:55 smithi070 ceph-mon[30388]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-16T09:24:55.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:55 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/439001110' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-16T09:24:55.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:55 smithi070 ceph-mon[40691]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-16T09:24:55.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:55 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/439001110' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-16T09:24:56.046 INFO:teuthology.orchestra.run.smithi070.stdout:pg_num: 1 2024-09-16T09:24:56.365 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:56 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3132783139' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-16T09:24:56.366 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:56 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3132783139' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-16T09:24:56.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:56 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3132783139' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-16T09:24:56.752 INFO:tasks.cephadm:Adding ceph.rgw.foo.a on smithi070 2024-09-16T09:24:56.752 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph orch apply rgw foo.a --placement '1;smithi070=foo.a' 2024-09-16T09:24:57.016 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:24:57.381 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:57 smithi084 ceph-mon[32629]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-16T09:24:57.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:57 smithi070 ceph-mon[30388]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-16T09:24:57.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:57 smithi070 ceph-mon[40691]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-16T09:24:57.672 INFO:teuthology.orchestra.run.smithi084.stdout:Scheduled rgw.foo.a update... 2024-09-16T09:24:58.288 DEBUG:teuthology.orchestra.run.smithi070:rgw.foo.a> sudo journalctl -f -n 0 -u ceph-a1321834-740c-11ef-bceb-c7b262605968@rgw.foo.a.service 2024-09-16T09:24:58.290 INFO:tasks.cephadm:Adding ceph.iscsi.iscsi.a on smithi084 2024-09-16T09:24:58.291 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd pool create datapool 3 3 replicated 2024-09-16T09:24:58.535 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:24:58.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:58 smithi084 ceph-mon[32629]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-09-16T09:24:58.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:58 smithi084 ceph-mon[32629]: from='client.24310 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi070=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:24:58.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:58 smithi084 ceph-mon[32629]: Saving service rgw.foo.a spec with placement smithi070=foo.a;count:1 2024-09-16T09:24:58.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:58 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:24:58.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:58 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:24:58.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:58 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:24:58.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:58 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:24:58.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:58 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:24:58.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:58 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]} : dispatch 2024-09-16T09:24:58.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:58 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-16T09:24:58.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:58 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:24:58.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:58 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:24:58.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:58 smithi084 ceph-mon[32629]: Deploying daemon rgw.foo.a on smithi070 2024-09-16T09:24:58.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:58 smithi070 ceph-mon[30388]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-09-16T09:24:58.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:58 smithi070 ceph-mon[30388]: from='client.24310 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi070=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:24:58.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:58 smithi070 ceph-mon[30388]: Saving service rgw.foo.a spec with placement smithi070=foo.a;count:1 2024-09-16T09:24:58.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:58 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:24:58.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:58 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:24:58.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:58 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:24:58.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:58 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:24:58.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:58 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:24:58.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:58 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]} : dispatch 2024-09-16T09:24:58.923 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:58 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-16T09:24:58.923 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:58 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:24:58.923 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:58 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:24:58.923 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:58 smithi070 ceph-mon[30388]: Deploying daemon rgw.foo.a on smithi070 2024-09-16T09:24:58.923 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:58 smithi070 ceph-mon[40691]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 230 MiB used, 715 GiB / 715 GiB avail 2024-09-16T09:24:58.923 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:58 smithi070 ceph-mon[40691]: from='client.24310 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi070=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:24:58.924 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:58 smithi070 ceph-mon[40691]: Saving service rgw.foo.a spec with placement smithi070=foo.a;count:1 2024-09-16T09:24:58.924 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:58 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:24:58.924 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:58 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:24:58.924 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:58 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:24:58.924 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:58 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:24:58.924 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:58 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:24:58.924 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:58 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]} : dispatch 2024-09-16T09:24:58.924 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:58 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-16T09:24:58.924 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:58 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:24:58.924 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:58 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:24:58.925 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:58 smithi070 ceph-mon[40691]: Deploying daemon rgw.foo.a on smithi070 2024-09-16T09:24:59.670 INFO:journalctl@ceph.rgw.foo.a.smithi070.stdout:Sep 16 09:24:59 smithi070 systemd[1]: Starting Ceph rgw.foo.a for a1321834-740c-11ef-bceb-c7b262605968... 2024-09-16T09:24:59.693 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'datapool' created 2024-09-16T09:24:59.990 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:59 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:24:59.990 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:59 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:24:59.990 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:59 smithi084 ceph-mon[32629]: osdmap e56: 8 total, 8 up, 8 in 2024-09-16T09:24:59.990 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:59 smithi084 ceph-mon[32629]: from='client.? 172.21.15.84:0/3818627894' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"} : dispatch 2024-09-16T09:24:59.991 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:24:59 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"} : dispatch 2024-09-16T09:25:00.012 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:59 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:25:00.012 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:59 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:25:00.012 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:59 smithi070 ceph-mon[30388]: osdmap e56: 8 total, 8 up, 8 in 2024-09-16T09:25:00.012 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:59 smithi070 ceph-mon[30388]: from='client.? 172.21.15.84:0/3818627894' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"} : dispatch 2024-09-16T09:25:00.012 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:24:59 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"} : dispatch 2024-09-16T09:25:00.012 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:59 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:25:00.012 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:59 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:25:00.012 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:59 smithi070 ceph-mon[40691]: osdmap e56: 8 total, 8 up, 8 in 2024-09-16T09:25:00.012 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:59 smithi070 ceph-mon[40691]: from='client.? 172.21.15.84:0/3818627894' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"} : dispatch 2024-09-16T09:25:00.013 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:24:59 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"} : dispatch 2024-09-16T09:25:00.013 INFO:journalctl@ceph.rgw.foo.a.smithi070.stdout:Sep 16 09:24:59 smithi070 podman[67780]: 2024-09-16 09:24:59.673028771 +0000 UTC m=+0.016009781 image pull 425e316ba8ebf74f04aa3640f7deb4fa3f293a78d0c8d097cb679eaf86ddca6c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce 2024-09-16T09:25:00.013 INFO:journalctl@ceph.rgw.foo.a.smithi070.stdout:Sep 16 09:24:59 smithi070 podman[67780]: 2024-09-16 09:24:59.80074267 +0000 UTC m=+0.143723676 container create d83e321be456e6f9d886c9de9ed3d4f8c506699a72c937460dd4457f3a6a76f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-rgw-foo-a, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-16T09:25:00.013 INFO:journalctl@ceph.rgw.foo.a.smithi070.stdout:Sep 16 09:25:00 smithi070 podman[67780]: 2024-09-16 09:25:00.011327377 +0000 UTC m=+0.354308382 container init d83e321be456e6f9d886c9de9ed3d4f8c506699a72c937460dd4457f3a6a76f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-rgw-foo-a, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-16T09:25:00.243 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a1321834-740c-11ef-bceb-c7b262605968 -- rbd pool init datapool 2024-09-16T09:25:00.420 INFO:journalctl@ceph.rgw.foo.a.smithi070.stdout:Sep 16 09:25:00 smithi070 podman[67780]: 2024-09-16 09:25:00.016560862 +0000 UTC m=+0.359541867 container start d83e321be456e6f9d886c9de9ed3d4f8c506699a72c937460dd4457f3a6a76f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-rgw-foo-a, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2) 2024-09-16T09:25:00.421 INFO:journalctl@ceph.rgw.foo.a.smithi070.stdout:Sep 16 09:25:00 smithi070 bash[67780]: d83e321be456e6f9d886c9de9ed3d4f8c506699a72c937460dd4457f3a6a76f7 2024-09-16T09:25:00.421 INFO:journalctl@ceph.rgw.foo.a.smithi070.stdout:Sep 16 09:25:00 smithi070 systemd[1]: Started Ceph rgw.foo.a for a1321834-740c-11ef-bceb-c7b262605968. 2024-09-16T09:25:00.485 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:25:00.860 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:00 smithi084 ceph-mon[32629]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-16T09:25:00.860 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:00 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-09-16T09:25:00.860 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:00 smithi084 ceph-mon[32629]: osdmap e57: 8 total, 8 up, 8 in 2024-09-16T09:25:00.860 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:00 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:25:00.860 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:00 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:25:00.860 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:00 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:25:00.860 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:00 smithi084 ceph-mon[32629]: Saving service rgw.foo.a spec with placement smithi070=foo.a;count:1 2024-09-16T09:25:00.860 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:00 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:25:00.860 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:00 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:25:00.861 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:00 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:25:00.957 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:00 smithi070 ceph-mon[30388]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-16T09:25:00.957 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:00 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-09-16T09:25:00.957 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:00 smithi070 ceph-mon[30388]: osdmap e57: 8 total, 8 up, 8 in 2024-09-16T09:25:00.957 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:00 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:25:00.957 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:00 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:25:00.957 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:00 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:25:00.957 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:00 smithi070 ceph-mon[30388]: Saving service rgw.foo.a spec with placement smithi070=foo.a;count:1 2024-09-16T09:25:00.957 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:00 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:25:00.957 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:00 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:25:00.957 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:00 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:25:00.958 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:00 smithi070 ceph-mon[40691]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-16T09:25:00.958 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:00 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-09-16T09:25:00.958 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:00 smithi070 ceph-mon[40691]: osdmap e57: 8 total, 8 up, 8 in 2024-09-16T09:25:00.958 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:00 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:25:00.958 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:00 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:25:00.958 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:00 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:25:00.958 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:00 smithi070 ceph-mon[40691]: Saving service rgw.foo.a spec with placement smithi070=foo.a;count:1 2024-09-16T09:25:00.958 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:00 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:25:00.958 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:00 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:25:00.958 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:00 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:25:02.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:01 smithi084 ceph-mon[32629]: osdmap e58: 8 total, 8 up, 8 in 2024-09-16T09:25:02.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:01 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1271375122' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"} : dispatch 2024-09-16T09:25:02.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:01 smithi084 ceph-mon[32629]: from='client.? 172.21.15.84:0/3692274150' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "datapool","app": "rbd"} : dispatch 2024-09-16T09:25:02.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:01 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "datapool","app": "rbd"} : dispatch 2024-09-16T09:25:02.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:01 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:25:02.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:01 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:25:02.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:01 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:25:02.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:01 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:25:02.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:01 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1271375122' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-16T09:25:02.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:01 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-09-16T09:25:02.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:01 smithi084 ceph-mon[32629]: osdmap e59: 8 total, 8 up, 8 in 2024-09-16T09:25:02.170 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:01 smithi070 ceph-mon[30388]: osdmap e58: 8 total, 8 up, 8 in 2024-09-16T09:25:02.170 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:01 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1271375122' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"} : dispatch 2024-09-16T09:25:02.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:01 smithi070 ceph-mon[30388]: from='client.? 172.21.15.84:0/3692274150' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "datapool","app": "rbd"} : dispatch 2024-09-16T09:25:02.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:01 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "datapool","app": "rbd"} : dispatch 2024-09-16T09:25:02.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:01 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:25:02.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:01 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:25:02.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:01 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:25:02.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:01 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:25:02.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:01 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1271375122' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-16T09:25:02.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:01 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-09-16T09:25:02.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:01 smithi070 ceph-mon[30388]: osdmap e59: 8 total, 8 up, 8 in 2024-09-16T09:25:02.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:01 smithi070 ceph-mon[40691]: osdmap e58: 8 total, 8 up, 8 in 2024-09-16T09:25:02.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:01 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1271375122' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"} : dispatch 2024-09-16T09:25:02.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:01 smithi070 ceph-mon[40691]: from='client.? 172.21.15.84:0/3692274150' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "datapool","app": "rbd"} : dispatch 2024-09-16T09:25:02.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:01 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "datapool","app": "rbd"} : dispatch 2024-09-16T09:25:02.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:01 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:25:02.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:01 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:25:02.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:01 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:25:02.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:01 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:25:02.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:01 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1271375122' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-16T09:25:02.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:01 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-09-16T09:25:02.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:01 smithi070 ceph-mon[40691]: osdmap e59: 8 total, 8 up, 8 in 2024-09-16T09:25:03.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:02 smithi084 ceph-mon[32629]: pgmap v194: 36 pgs: 8 creating+peering, 2 active+clean, 26 unknown; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-16T09:25:03.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:02 smithi084 ceph-mon[32629]: Checking dashboard <-> RGW credentials 2024-09-16T09:25:03.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:02 smithi084 ceph-mon[32629]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T09:25:03.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:02 smithi084 ceph-mon[32629]: osdmap e60: 8 total, 8 up, 8 in 2024-09-16T09:25:03.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:02 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1123534357' entity='mgr.y' cmd={"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"} : dispatch 2024-09-16T09:25:03.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:02 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1049757559' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"} : dispatch 2024-09-16T09:25:03.170 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:02 smithi070 ceph-mon[30388]: pgmap v194: 36 pgs: 8 creating+peering, 2 active+clean, 26 unknown; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-16T09:25:03.170 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:02 smithi070 ceph-mon[30388]: Checking dashboard <-> RGW credentials 2024-09-16T09:25:03.170 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:02 smithi070 ceph-mon[30388]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T09:25:03.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:02 smithi070 ceph-mon[30388]: osdmap e60: 8 total, 8 up, 8 in 2024-09-16T09:25:03.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:02 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1123534357' entity='mgr.y' cmd={"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"} : dispatch 2024-09-16T09:25:03.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:02 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1049757559' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"} : dispatch 2024-09-16T09:25:03.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:02 smithi070 ceph-mon[40691]: pgmap v194: 36 pgs: 8 creating+peering, 2 active+clean, 26 unknown; 577 KiB data, 229 MiB used, 715 GiB / 715 GiB avail 2024-09-16T09:25:03.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:02 smithi070 ceph-mon[40691]: Checking dashboard <-> RGW credentials 2024-09-16T09:25:03.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:02 smithi070 ceph-mon[40691]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T09:25:03.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:02 smithi070 ceph-mon[40691]: osdmap e60: 8 total, 8 up, 8 in 2024-09-16T09:25:03.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:02 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1123534357' entity='mgr.y' cmd={"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"} : dispatch 2024-09-16T09:25:03.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:02 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1049757559' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"} : dispatch 2024-09-16T09:25:04.037 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:03 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:25:04.037 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:03 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:25:04.037 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:03 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1123534357' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-16T09:25:04.037 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:03 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1049757559' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-16T09:25:04.038 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:03 smithi084 ceph-mon[32629]: osdmap e61: 8 total, 8 up, 8 in 2024-09-16T09:25:04.170 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:03 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:25:04.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:03 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:25:04.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:03 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1123534357' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-16T09:25:04.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:03 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1049757559' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-16T09:25:04.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:03 smithi070 ceph-mon[30388]: osdmap e61: 8 total, 8 up, 8 in 2024-09-16T09:25:04.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:03 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:25:04.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:03 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:25:04.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:03 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1123534357' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-16T09:25:04.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:03 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1049757559' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-16T09:25:04.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:03 smithi070 ceph-mon[40691]: osdmap e61: 8 total, 8 up, 8 in 2024-09-16T09:25:04.371 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph orch apply iscsi datapool admin admin --trusted_ip_list 172.21.15.84 --placement '1;smithi084=iscsi.a' 2024-09-16T09:25:04.619 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:25:05.303 INFO:teuthology.orchestra.run.smithi084.stdout:Scheduled iscsi.datapool update... 2024-09-16T09:25:05.304 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:05 smithi084 ceph-mon[32629]: pgmap v197: 68 pgs: 29 creating+peering, 28 active+clean, 11 unknown; 578 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 1023 B/s wr, 2 op/s 2024-09-16T09:25:05.304 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:05 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:25:05.304 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:05 smithi084 ceph-mon[32629]: osdmap e62: 8 total, 8 up, 8 in 2024-09-16T09:25:05.304 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:05 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1123534357' entity='mgr.y' cmd={"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"} : dispatch 2024-09-16T09:25:05.304 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:05 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1049757559' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"} : dispatch 2024-09-16T09:25:05.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:05 smithi070 ceph-mon[30388]: pgmap v197: 68 pgs: 29 creating+peering, 28 active+clean, 11 unknown; 578 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 1023 B/s wr, 2 op/s 2024-09-16T09:25:05.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:05 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:25:05.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:05 smithi070 ceph-mon[30388]: osdmap e62: 8 total, 8 up, 8 in 2024-09-16T09:25:05.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:05 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1123534357' entity='mgr.y' cmd={"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"} : dispatch 2024-09-16T09:25:05.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:05 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1049757559' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"} : dispatch 2024-09-16T09:25:05.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:05 smithi070 ceph-mon[40691]: pgmap v197: 68 pgs: 29 creating+peering, 28 active+clean, 11 unknown; 578 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 1023 B/s wr, 2 op/s 2024-09-16T09:25:05.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:05 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:25:05.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:05 smithi070 ceph-mon[40691]: osdmap e62: 8 total, 8 up, 8 in 2024-09-16T09:25:05.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:05 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1123534357' entity='mgr.y' cmd={"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"} : dispatch 2024-09-16T09:25:05.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:05 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1049757559' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"} : dispatch 2024-09-16T09:25:05.946 INFO:tasks.cephadm:Distributing iscsi-gateway.cfg... 2024-09-16T09:25:05.946 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-16T09:25:05.946 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/etc/ceph/iscsi-gateway.cfg 2024-09-16T09:25:05.976 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-09-16T09:25:05.976 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/etc/ceph/iscsi-gateway.cfg 2024-09-16T09:25:06.002 DEBUG:teuthology.orchestra.run.smithi084:iscsi.iscsi.a> sudo journalctl -f -n 0 -u ceph-a1321834-740c-11ef-bceb-c7b262605968@iscsi.iscsi.a.service 2024-09-16T09:25:06.045 INFO:tasks.cephadm:Adding prometheus.a on smithi084 2024-09-16T09:25:06.045 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph orch apply prometheus '1;smithi084=a' 2024-09-16T09:25:06.326 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:25:06.559 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:06 smithi084 ceph-mon[32629]: from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.84", "placement": "1;smithi084=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:25:06.559 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:06 smithi084 ceph-mon[32629]: Saving service iscsi.datapool spec with placement smithi084=iscsi.a;count:1 2024-09-16T09:25:06.559 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:06 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:25:06.559 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:06 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1123534357' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-16T09:25:06.559 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:06 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1049757559' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-16T09:25:06.559 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:06 smithi084 ceph-mon[32629]: osdmap e63: 8 total, 8 up, 8 in 2024-09-16T09:25:06.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:06 smithi070 ceph-mon[30388]: from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.84", "placement": "1;smithi084=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:25:06.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:06 smithi070 ceph-mon[30388]: Saving service iscsi.datapool spec with placement smithi084=iscsi.a;count:1 2024-09-16T09:25:06.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:06 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:25:06.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:06 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1123534357' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-16T09:25:06.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:06 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1049757559' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-16T09:25:06.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:06 smithi070 ceph-mon[30388]: osdmap e63: 8 total, 8 up, 8 in 2024-09-16T09:25:06.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:06 smithi070 ceph-mon[40691]: from='client.24370 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.84", "placement": "1;smithi084=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:25:06.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:06 smithi070 ceph-mon[40691]: Saving service iscsi.datapool spec with placement smithi084=iscsi.a;count:1 2024-09-16T09:25:06.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:06 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:25:06.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:06 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1123534357' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-16T09:25:06.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:06 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1049757559' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-16T09:25:06.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:06 smithi070 ceph-mon[40691]: osdmap e63: 8 total, 8 up, 8 in 2024-09-16T09:25:06.935 INFO:teuthology.orchestra.run.smithi084.stdout:Scheduled prometheus update... 2024-09-16T09:25:07.573 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:07 smithi070 ceph-mon[30388]: pgmap v200: 100 pgs: 29 creating+peering, 28 active+clean, 43 unknown; 578 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 1023 B/s wr, 2 op/s 2024-09-16T09:25:07.573 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:07 smithi070 ceph-mon[30388]: osdmap e64: 8 total, 8 up, 8 in 2024-09-16T09:25:07.573 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:07 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1123534357' entity='mgr.y' cmd={"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"} : dispatch 2024-09-16T09:25:07.573 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:07 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1049757559' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"} : dispatch 2024-09-16T09:25:07.573 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:07 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:25:07.573 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:07 smithi070 ceph-mon[40691]: pgmap v200: 100 pgs: 29 creating+peering, 28 active+clean, 43 unknown; 578 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 1023 B/s wr, 2 op/s 2024-09-16T09:25:07.574 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:07 smithi070 ceph-mon[40691]: osdmap e64: 8 total, 8 up, 8 in 2024-09-16T09:25:07.574 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:07 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1123534357' entity='mgr.y' cmd={"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"} : dispatch 2024-09-16T09:25:07.574 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:07 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1049757559' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"} : dispatch 2024-09-16T09:25:07.574 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:07 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:25:07.619 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:07 smithi084 ceph-mon[32629]: pgmap v200: 100 pgs: 29 creating+peering, 28 active+clean, 43 unknown; 578 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 1023 B/s wr, 2 op/s 2024-09-16T09:25:07.619 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:07 smithi084 ceph-mon[32629]: osdmap e64: 8 total, 8 up, 8 in 2024-09-16T09:25:07.619 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:07 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1123534357' entity='mgr.y' cmd={"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"} : dispatch 2024-09-16T09:25:07.619 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:07 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1049757559' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"} : dispatch 2024-09-16T09:25:07.620 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:07 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:25:07.648 DEBUG:teuthology.orchestra.run.smithi084:prometheus.a> sudo journalctl -f -n 0 -u ceph-a1321834-740c-11ef-bceb-c7b262605968@prometheus.a.service 2024-09-16T09:25:07.651 INFO:tasks.cephadm:Adding node-exporter.a on smithi070 2024-09-16T09:25:07.651 INFO:tasks.cephadm:Adding node-exporter.b on smithi084 2024-09-16T09:25:07.652 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph orch apply node-exporter '2;smithi070=a;smithi084=b' 2024-09-16T09:25:07.942 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:25:08.588 INFO:teuthology.orchestra.run.smithi084.stdout:Scheduled node-exporter update... 2024-09-16T09:25:08.682 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:08 smithi084 ceph-mon[32629]: from='client.24376 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi084=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:25:08.682 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:08 smithi084 ceph-mon[32629]: Saving service prometheus spec with placement smithi084=a;count:1 2024-09-16T09:25:08.682 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:08 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1123534357' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-16T09:25:08.683 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:08 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1049757559' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-16T09:25:08.683 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:08 smithi084 ceph-mon[32629]: osdmap e65: 8 total, 8 up, 8 in 2024-09-16T09:25:08.683 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:08 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1049757559' entity='client.rgw.foo.a' cmd={"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"} : dispatch 2024-09-16T09:25:08.683 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:08 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1123534357' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"} : dispatch 2024-09-16T09:25:08.683 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:08 smithi070 ceph-mon[30388]: from='client.24376 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi084=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:25:08.683 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:08 smithi070 ceph-mon[30388]: Saving service prometheus spec with placement smithi084=a;count:1 2024-09-16T09:25:08.683 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:08 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1123534357' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-16T09:25:08.683 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:08 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1049757559' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-16T09:25:08.684 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:08 smithi070 ceph-mon[30388]: osdmap e65: 8 total, 8 up, 8 in 2024-09-16T09:25:08.684 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:08 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1049757559' entity='client.rgw.foo.a' cmd={"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"} : dispatch 2024-09-16T09:25:08.684 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:08 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1123534357' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"} : dispatch 2024-09-16T09:25:08.684 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:08 smithi070 ceph-mon[40691]: from='client.24376 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi084=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:25:08.684 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:08 smithi070 ceph-mon[40691]: Saving service prometheus spec with placement smithi084=a;count:1 2024-09-16T09:25:08.684 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:08 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1123534357' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-16T09:25:08.684 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:08 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1049757559' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-16T09:25:08.684 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:08 smithi070 ceph-mon[40691]: osdmap e65: 8 total, 8 up, 8 in 2024-09-16T09:25:08.684 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:08 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1049757559' entity='client.rgw.foo.a' cmd={"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"} : dispatch 2024-09-16T09:25:08.684 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:08 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1123534357' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"} : dispatch 2024-09-16T09:25:09.232 DEBUG:teuthology.orchestra.run.smithi070:node-exporter.a> sudo journalctl -f -n 0 -u ceph-a1321834-740c-11ef-bceb-c7b262605968@node-exporter.a.service 2024-09-16T09:25:09.235 DEBUG:teuthology.orchestra.run.smithi084:node-exporter.b> sudo journalctl -f -n 0 -u ceph-a1321834-740c-11ef-bceb-c7b262605968@node-exporter.b.service 2024-09-16T09:25:09.237 INFO:tasks.cephadm:Adding alertmanager.a on smithi070 2024-09-16T09:25:09.238 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph orch apply alertmanager '1;smithi070=a' 2024-09-16T09:25:09.535 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:09 smithi084 ceph-mon[32629]: pgmap v203: 132 pgs: 21 creating+peering, 46 active+clean, 65 unknown; 578 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 3.2 KiB/s rd, 1023 B/s wr, 5 op/s 2024-09-16T09:25:09.536 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:09 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:25:09.536 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:09 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1049757559' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-16T09:25:09.536 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:09 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1123534357' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-16T09:25:09.536 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:09 smithi084 ceph-mon[32629]: osdmap e66: 8 total, 8 up, 8 in 2024-09-16T09:25:09.536 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:09 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:25:09.536 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:09 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:25:09.536 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:09 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:25:09.536 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:09 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]} : dispatch 2024-09-16T09:25:09.536 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:09 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-16T09:25:09.536 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:09 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:25:09.536 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:09 smithi084 ceph-mon[32629]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T09:25:09.536 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:09 smithi084 ceph-mon[32629]: osdmap e67: 8 total, 8 up, 8 in 2024-09-16T09:25:09.670 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:09 smithi070 ceph-mon[40691]: pgmap v203: 132 pgs: 21 creating+peering, 46 active+clean, 65 unknown; 578 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 3.2 KiB/s rd, 1023 B/s wr, 5 op/s 2024-09-16T09:25:09.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:09 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:25:09.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:09 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1049757559' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-16T09:25:09.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:09 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1123534357' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-16T09:25:09.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:09 smithi070 ceph-mon[40691]: osdmap e66: 8 total, 8 up, 8 in 2024-09-16T09:25:09.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:09 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:25:09.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:09 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:25:09.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:09 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:25:09.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:09 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]} : dispatch 2024-09-16T09:25:09.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:09 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-16T09:25:09.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:09 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:25:09.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:09 smithi070 ceph-mon[40691]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T09:25:09.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:09 smithi070 ceph-mon[40691]: osdmap e67: 8 total, 8 up, 8 in 2024-09-16T09:25:09.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:09 smithi070 ceph-mon[30388]: pgmap v203: 132 pgs: 21 creating+peering, 46 active+clean, 65 unknown; 578 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 3.2 KiB/s rd, 1023 B/s wr, 5 op/s 2024-09-16T09:25:09.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:09 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:25:09.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:09 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1049757559' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-16T09:25:09.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:09 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1123534357' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-16T09:25:09.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:09 smithi070 ceph-mon[30388]: osdmap e66: 8 total, 8 up, 8 in 2024-09-16T09:25:09.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:09 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:25:09.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:09 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:25:09.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:09 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:25:09.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:09 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]} : dispatch 2024-09-16T09:25:09.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:09 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-16T09:25:09.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:09 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:25:09.673 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:09 smithi070 ceph-mon[30388]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T09:25:09.673 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:09 smithi070 ceph-mon[30388]: osdmap e67: 8 total, 8 up, 8 in 2024-09-16T09:25:09.988 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:25:10.378 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:10 smithi084 ceph-mon[32629]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi070=a;smithi084=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:25:10.378 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:10 smithi084 ceph-mon[32629]: Saving service node-exporter spec with placement smithi070=a;smithi084=b;count:2 2024-09-16T09:25:10.378 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:10 smithi084 ceph-mon[32629]: Deploying daemon iscsi.iscsi.a on smithi084 2024-09-16T09:25:10.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:10 smithi070 ceph-mon[30388]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi070=a;smithi084=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:25:10.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:10 smithi070 ceph-mon[30388]: Saving service node-exporter spec with placement smithi070=a;smithi084=b;count:2 2024-09-16T09:25:10.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:10 smithi070 ceph-mon[30388]: Deploying daemon iscsi.iscsi.a on smithi084 2024-09-16T09:25:10.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:10 smithi070 ceph-mon[40691]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi070=a;smithi084=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:25:10.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:10 smithi070 ceph-mon[40691]: Saving service node-exporter spec with placement smithi070=a;smithi084=b;count:2 2024-09-16T09:25:10.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:10 smithi070 ceph-mon[40691]: Deploying daemon iscsi.iscsi.a on smithi084 2024-09-16T09:25:11.069 INFO:teuthology.orchestra.run.smithi084.stdout:Scheduled alertmanager update... 2024-09-16T09:25:11.318 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:25:11 smithi084 systemd[1]: Starting Ceph iscsi.iscsi.a for a1321834-740c-11ef-bceb-c7b262605968... 2024-09-16T09:25:11.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:11 smithi084 ceph-mon[32629]: pgmap v207: 132 pgs: 9 creating+peering, 117 active+clean, 6 unknown; 578 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 4.4 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-09-16T09:25:11.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:11 smithi084 ceph-mon[32629]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-16T09:25:11.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:11 smithi084 ceph-mon[32629]: Cluster is now healthy 2024-09-16T09:25:11.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:11 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:25:11.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:11 smithi070 ceph-mon[30388]: pgmap v207: 132 pgs: 9 creating+peering, 117 active+clean, 6 unknown; 578 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 4.4 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-09-16T09:25:11.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:11 smithi070 ceph-mon[30388]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-16T09:25:11.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:11 smithi070 ceph-mon[30388]: Cluster is now healthy 2024-09-16T09:25:11.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:11 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:25:11.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:11 smithi070 ceph-mon[40691]: pgmap v207: 132 pgs: 9 creating+peering, 117 active+clean, 6 unknown; 578 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 4.4 KiB/s rd, 1.6 KiB/s wr, 8 op/s 2024-09-16T09:25:11.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:11 smithi070 ceph-mon[40691]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-16T09:25:11.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:11 smithi070 ceph-mon[40691]: Cluster is now healthy 2024-09-16T09:25:11.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:11 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:25:11.738 DEBUG:teuthology.orchestra.run.smithi070:alertmanager.a> sudo journalctl -f -n 0 -u ceph-a1321834-740c-11ef-bceb-c7b262605968@alertmanager.a.service 2024-09-16T09:25:11.741 INFO:tasks.cephadm:Adding grafana.a on smithi084 2024-09-16T09:25:11.742 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph orch apply grafana '1;smithi084=a' 2024-09-16T09:25:12.027 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:25:11 smithi084 podman[60356]: 2024-09-16 09:25:11.800193392 +0000 UTC m=+0.016792530 image pull 425e316ba8ebf74f04aa3640f7deb4fa3f293a78d0c8d097cb679eaf86ddca6c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce 2024-09-16T09:25:12.028 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:25:11 smithi084 podman[60356]: 2024-09-16 09:25:11.900358575 +0000 UTC m=+0.116957701 container create f962308caf1e439632d6ad1a2225c66c82203aef0b464aac695e82699d5cc957 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, GIT_CLEAN=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-16T09:25:12.268 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:25:12.394 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:12 smithi084 ceph-mon[32629]: from='client.24403 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi070=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:25:12.394 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:12 smithi084 ceph-mon[32629]: Saving service alertmanager spec with placement smithi070=a;count:1 2024-09-16T09:25:12.394 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:12 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:25:12.394 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:12 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:25:12.394 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:12 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:25:12.394 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:12 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:25:12.394 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:25:12 smithi084 podman[60356]: 2024-09-16 09:25:12.127542578 +0000 UTC m=+0.344141703 container init f962308caf1e439632d6ad1a2225c66c82203aef0b464aac695e82699d5cc957 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2) 2024-09-16T09:25:12.395 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:25:12 smithi084 podman[60356]: 2024-09-16 09:25:12.131970827 +0000 UTC m=+0.348569953 container start f962308caf1e439632d6ad1a2225c66c82203aef0b464aac695e82699d5cc957 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-guits-main-2024-09-13-1248-8293d73, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T09:25:12.395 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:25:12 smithi084 bash[60356]: f962308caf1e439632d6ad1a2225c66c82203aef0b464aac695e82699d5cc957 2024-09-16T09:25:12.395 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:25:12 smithi084 systemd[1]: Started Ceph iscsi.iscsi.a for a1321834-740c-11ef-bceb-c7b262605968. 2024-09-16T09:25:12.660 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:25:12 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug Started the configuration object watcher 2024-09-16T09:25:12.660 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:25:12 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug Checking for config object changes every 1s 2024-09-16T09:25:12.660 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:25:12 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug Processing osd blocklist entries for this node 2024-09-16T09:25:12.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:12 smithi070 ceph-mon[30388]: from='client.24403 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi070=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:25:12.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:12 smithi070 ceph-mon[30388]: Saving service alertmanager spec with placement smithi070=a;count:1 2024-09-16T09:25:12.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:12 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:25:12.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:12 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:25:12.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:12 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:25:12.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:12 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:25:12.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:12 smithi070 ceph-mon[40691]: from='client.24403 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi070=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:25:12.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:12 smithi070 ceph-mon[40691]: Saving service alertmanager spec with placement smithi070=a;count:1 2024-09-16T09:25:12.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:12 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:25:12.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:12 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:25:12.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:12 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:25:12.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:12 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:25:13.028 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:25:12 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug Reading the configuration object to update local LIO configuration 2024-09-16T09:25:13.028 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:25:12 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug Configuration does not have an entry for this host(smithi084.front.sepia.ceph.com) - nothing to define to LIO 2024-09-16T09:25:13.028 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:25:12 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: * Serving Flask app 'rbd-target-api' (lazy loading) 2024-09-16T09:25:13.028 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:25:12 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: * Environment: production 2024-09-16T09:25:13.028 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:25:12 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: WARNING: This is a development server. Do not use it in a production deployment. 2024-09-16T09:25:13.028 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:25:12 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: Use a production WSGI server instead. 2024-09-16T09:25:13.029 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:25:12 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: * Debug mode: off 2024-09-16T09:25:13.029 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:25:12 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug * Running on all addresses. 2024-09-16T09:25:13.029 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:25:12 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: WARNING: This is a development server. Do not use it in a production deployment. 2024-09-16T09:25:13.029 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:25:12 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: * Running on all addresses. 2024-09-16T09:25:13.029 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:25:12 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: WARNING: This is a development server. Do not use it in a production deployment. 2024-09-16T09:25:13.029 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:25:12 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug * Running on http://[fe80::ec4:7aff:fe8f:d751]:5000/ (Press CTRL+C to quit) 2024-09-16T09:25:13.029 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:25:12 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: * Running on http://[fe80::ec4:7aff:fe8f:d751]:5000/ (Press CTRL+C to quit) 2024-09-16T09:25:13.075 INFO:teuthology.orchestra.run.smithi084.stdout:Scheduled grafana update... 2024-09-16T09:25:13.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:13 smithi084 ceph-mon[32629]: pgmap v208: 132 pgs: 9 creating+peering, 123 active+clean; 579 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 12 KiB/s rd, 1.1 KiB/s wr, 27 op/s 2024-09-16T09:25:13.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:13 smithi084 ceph-mon[32629]: Checking pool "datapool" exists for service iscsi.datapool 2024-09-16T09:25:13.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:13 smithi084 ceph-mon[32629]: Deploying daemon prometheus.a on smithi084 2024-09-16T09:25:13.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:13 smithi084 ceph-mon[32629]: from='client.? 172.21.15.84:0/216166411' entity='client.iscsi.iscsi.a' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-16T09:25:13.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:13 smithi084 ceph-mon[32629]: from='client.24439 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi084=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:25:13.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:13 smithi084 ceph-mon[32629]: Saving service grafana spec with placement smithi084=a;count:1 2024-09-16T09:25:13.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:13 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:25:13.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:13 smithi070 ceph-mon[30388]: pgmap v208: 132 pgs: 9 creating+peering, 123 active+clean; 579 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 12 KiB/s rd, 1.1 KiB/s wr, 27 op/s 2024-09-16T09:25:13.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:13 smithi070 ceph-mon[30388]: Checking pool "datapool" exists for service iscsi.datapool 2024-09-16T09:25:13.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:13 smithi070 ceph-mon[30388]: Deploying daemon prometheus.a on smithi084 2024-09-16T09:25:13.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:13 smithi070 ceph-mon[30388]: from='client.? 172.21.15.84:0/216166411' entity='client.iscsi.iscsi.a' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-16T09:25:13.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:13 smithi070 ceph-mon[30388]: from='client.24439 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi084=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:25:13.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:13 smithi070 ceph-mon[30388]: Saving service grafana spec with placement smithi084=a;count:1 2024-09-16T09:25:13.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:13 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:25:13.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:13 smithi070 ceph-mon[40691]: pgmap v208: 132 pgs: 9 creating+peering, 123 active+clean; 579 KiB data, 234 MiB used, 715 GiB / 715 GiB avail; 12 KiB/s rd, 1.1 KiB/s wr, 27 op/s 2024-09-16T09:25:13.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:13 smithi070 ceph-mon[40691]: Checking pool "datapool" exists for service iscsi.datapool 2024-09-16T09:25:13.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:13 smithi070 ceph-mon[40691]: Deploying daemon prometheus.a on smithi084 2024-09-16T09:25:13.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:13 smithi070 ceph-mon[40691]: from='client.? 172.21.15.84:0/216166411' entity='client.iscsi.iscsi.a' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-16T09:25:13.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:13 smithi070 ceph-mon[40691]: from='client.24439 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi084=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:25:13.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:13 smithi070 ceph-mon[40691]: Saving service grafana spec with placement smithi084=a;count:1 2024-09-16T09:25:13.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:13 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:25:13.957 DEBUG:teuthology.orchestra.run.smithi084:grafana.a> sudo journalctl -f -n 0 -u ceph-a1321834-740c-11ef-bceb-c7b262605968@grafana.a.service 2024-09-16T09:25:13.960 INFO:tasks.cephadm:Setting up client nodes... 2024-09-16T09:25:13.960 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-16T09:25:14.226 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:25:14.907 INFO:teuthology.orchestra.run.smithi070.stdout:[client.0] 2024-09-16T09:25:14.907 INFO:teuthology.orchestra.run.smithi070.stdout: key = AQB6+edmqNPrNRAAzafj2M45bKrP3STy1snZzA== 2024-09-16T09:25:15.170 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:15 smithi070 ceph-mon[30388]: pgmap v209: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 77 KiB/s rd, 6.0 KiB/s wr, 186 op/s 2024-09-16T09:25:15.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:15 smithi070 ceph-mon[30388]: osdmap e68: 8 total, 8 up, 8 in 2024-09-16T09:25:15.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:15 smithi070 ceph-mon[30388]: mgrmap e16: y(active, since 5m), standbys: x 2024-09-16T09:25:15.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:15 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:25:15.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:15 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/628204953' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-16T09:25:15.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:15 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/628204953' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-16T09:25:15.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:15 smithi070 ceph-mon[40691]: pgmap v209: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 77 KiB/s rd, 6.0 KiB/s wr, 186 op/s 2024-09-16T09:25:15.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:15 smithi070 ceph-mon[40691]: osdmap e68: 8 total, 8 up, 8 in 2024-09-16T09:25:15.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:15 smithi070 ceph-mon[40691]: mgrmap e16: y(active, since 5m), standbys: x 2024-09-16T09:25:15.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:15 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:25:15.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:15 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/628204953' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-16T09:25:15.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:15 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/628204953' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-16T09:25:15.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:15 smithi084 ceph-mon[32629]: pgmap v209: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 77 KiB/s rd, 6.0 KiB/s wr, 186 op/s 2024-09-16T09:25:15.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:15 smithi084 ceph-mon[32629]: osdmap e68: 8 total, 8 up, 8 in 2024-09-16T09:25:15.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:15 smithi084 ceph-mon[32629]: mgrmap e16: y(active, since 5m), standbys: x 2024-09-16T09:25:15.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:15 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:25:15.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:15 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/628204953' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-16T09:25:15.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:15 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/628204953' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-16T09:25:15.542 DEBUG:teuthology.orchestra.run.smithi070:> set -ex 2024-09-16T09:25:15.542 DEBUG:teuthology.orchestra.run.smithi070:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-16T09:25:15.542 DEBUG:teuthology.orchestra.run.smithi070:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-16T09:25:15.580 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-16T09:25:16.024 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:25:16.926 INFO:teuthology.orchestra.run.smithi084.stdout:[client.1] 2024-09-16T09:25:16.926 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQB8+edmjsfxNhAAf43DmZHfyWoGYBK0659TTw== 2024-09-16T09:25:17.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:17 smithi084 ceph-mon[32629]: pgmap v211: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 69 KiB/s rd, 5.1 KiB/s wr, 167 op/s 2024-09-16T09:25:17.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:17 smithi084 ceph-mon[32629]: from='client.? 172.21.15.84:0/808276612' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-16T09:25:17.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:17 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-16T09:25:17.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:17 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-16T09:25:17.420 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:17 smithi070 ceph-mon[40691]: pgmap v211: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 69 KiB/s rd, 5.1 KiB/s wr, 167 op/s 2024-09-16T09:25:17.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:17 smithi070 ceph-mon[40691]: from='client.? 172.21.15.84:0/808276612' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-16T09:25:17.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:17 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-16T09:25:17.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:17 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-16T09:25:17.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:17 smithi070 ceph-mon[30388]: pgmap v211: 132 pgs: 132 active+clean; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 69 KiB/s rd, 5.1 KiB/s wr, 167 op/s 2024-09-16T09:25:17.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:17 smithi070 ceph-mon[30388]: from='client.? 172.21.15.84:0/808276612' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-16T09:25:17.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:17 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-16T09:25:17.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:17 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-16T09:25:17.895 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-09-16T09:25:17.895 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-09-16T09:25:17.895 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-16T09:25:17.939 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph config log 1 --format=json 2024-09-16T09:25:18.159 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:18 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:25:18.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:18 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:25:18.198 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:25:18.313 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:18 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:25:18.314 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:18 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:25:18.314 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:18 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:25:18.314 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:18 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:25:18.810 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:25:19.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:19 smithi084 ceph-mon[32629]: pgmap v212: 132 pgs: 132 active+clean; 583 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 56 KiB/s rd, 4.2 KiB/s wr, 136 op/s 2024-09-16T09:25:19.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:19 smithi084 ceph-mon[32629]: osdmap e69: 8 total, 8 up, 8 in 2024-09-16T09:25:19.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:19 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1213288531' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-16T09:25:19.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:19 smithi084 ceph-mon[32629]: osdmap e70: 8 total, 8 up, 8 in 2024-09-16T09:25:19.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:19 smithi070 ceph-mon[30388]: pgmap v212: 132 pgs: 132 active+clean; 583 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 56 KiB/s rd, 4.2 KiB/s wr, 136 op/s 2024-09-16T09:25:19.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:19 smithi070 ceph-mon[30388]: osdmap e69: 8 total, 8 up, 8 in 2024-09-16T09:25:19.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:19 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1213288531' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-16T09:25:19.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:19 smithi070 ceph-mon[30388]: osdmap e70: 8 total, 8 up, 8 in 2024-09-16T09:25:19.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:19 smithi070 ceph-mon[40691]: pgmap v212: 132 pgs: 132 active+clean; 583 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 56 KiB/s rd, 4.2 KiB/s wr, 136 op/s 2024-09-16T09:25:19.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:19 smithi070 ceph-mon[40691]: osdmap e69: 8 total, 8 up, 8 in 2024-09-16T09:25:19.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:19 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1213288531' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-16T09:25:19.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:19 smithi070 ceph-mon[40691]: osdmap e70: 8 total, 8 up, 8 in 2024-09-16T09:25:19.626 INFO:teuthology.orchestra.run.smithi070.stdout:[{"version":24,"timestamp":"2024-09-16T09:25:08.916013+0000","name":"","changes":[{"name":"mgr/mgr/dashboard/RGW_API_SECRET_KEY","new_value":"WjzZwgfrOhHm02mBDADRdvEXI5I2IobZbrFe8AJM"}]}] 2024-09-16T09:25:19.627 INFO:tasks.ceph_manager:config epoch is 24 2024-09-16T09:25:19.627 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-16T09:25:19.627 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-16T09:25:19.627 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph mgr dump --format=json 2024-09-16T09:25:19.875 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:25:20.574 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:25:20.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:20 smithi084 ceph-mon[32629]: pgmap v215: 132 pgs: 132 active+clean; 583 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 682 B/s rd, 170 B/s wr, 2 op/s 2024-09-16T09:25:20.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:20 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:25:20.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:20 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/2700018719' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-16T09:25:20.911 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:20 smithi070 ceph-mon[30388]: pgmap v215: 132 pgs: 132 active+clean; 583 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 682 B/s rd, 170 B/s wr, 2 op/s 2024-09-16T09:25:20.912 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:20 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:25:20.912 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:20 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2700018719' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-16T09:25:20.912 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:20 smithi070 ceph-mon[40691]: pgmap v215: 132 pgs: 132 active+clean; 583 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 682 B/s rd, 170 B/s wr, 2 op/s 2024-09-16T09:25:20.912 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:20 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:25:20.912 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:20 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/2700018719' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-16T09:25:21.293 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":16,"flags":0,"active_gid":14150,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6800","nonce":1876661229},{"type":"v1","addr":"172.21.15.70:6801","nonce":1876661229}]},"active_addr":"172.21.15.70:6801/1876661229","active_change":"2024-09-16T09:20:09.306441+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[{"gid":24115,"name":"x","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.70:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"cephadm","addrvec":[{"type":"v2","addr":"172.21.15.70:0","nonce":3284625547}]},{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.70:0","nonce":1723643739}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.70:0","nonce":125491368}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.70:0","nonce":3456417396}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.70:0","nonce":1230482871}]}]} 2024-09-16T09:25:21.299 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-16T09:25:21.299 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-16T09:25:21.299 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T09:25:21.546 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:25:22.190 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:25:22.190 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":70,"fsid":"a1321834-740c-11ef-bceb-c7b262605968","created":"2024-09-16T09:19:14.121498+0000","modified":"2024-09-16T09:25:19.074889+0000","last_up_change":"2024-09-16T09:24:51.124496+0000","last_in_change":"2024-09-16T09:24:28.432920+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T09:22:27.617694+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"datapool","create_time":"2024-09-16T09:24:59.169244+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":61,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.6500000953674316,"score_stable":2.6500000953674316,"optimal_score":0.87999999523162842,"raw_score_acting":2.3299999237060547,"raw_score_stable":2.3299999237060547,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":".rgw.root","create_time":"2024-09-16T09:25:00.056119+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-09-16T09:25:01.770016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-09-16T09:25:03.725555+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-09-16T09:25:05.773530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7ef288c8-3bf0-4814-8e86-f70a7de041ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":2013075781},{"type":"v1","addr":"172.21.15.70:6803","nonce":2013075781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":2013075781},{"type":"v1","addr":"172.21.15.70:6805","nonce":2013075781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":2013075781},{"type":"v1","addr":"172.21.15.70:6809","nonce":2013075781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":2013075781},{"type":"v1","addr":"172.21.15.70:6807","nonce":2013075781}]},"public_addr":"172.21.15.70:6803/2013075781","cluster_addr":"172.21.15.70:6805/2013075781","heartbeat_back_addr":"172.21.15.70:6809/2013075781","heartbeat_front_addr":"172.21.15.70:6807/2013075781","state":["exists","up"]},{"osd":1,"uuid":"474d41d9-295a-450c-8f27-ce487a14678b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":1670012115},{"type":"v1","addr":"172.21.15.70:6811","nonce":1670012115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":1670012115},{"type":"v1","addr":"172.21.15.70:6813","nonce":1670012115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1670012115},{"type":"v1","addr":"172.21.15.70:6817","nonce":1670012115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":1670012115},{"type":"v1","addr":"172.21.15.70:6815","nonce":1670012115}]},"public_addr":"172.21.15.70:6811/1670012115","cluster_addr":"172.21.15.70:6813/1670012115","heartbeat_back_addr":"172.21.15.70:6817/1670012115","heartbeat_front_addr":"172.21.15.70:6815/1670012115","state":["exists","up"]},{"osd":2,"uuid":"5b4ca4e8-cecc-4163-b291-c9209b02c4b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3601682336},{"type":"v1","addr":"172.21.15.70:6819","nonce":3601682336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":3601682336},{"type":"v1","addr":"172.21.15.70:6821","nonce":3601682336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":3601682336},{"type":"v1","addr":"172.21.15.70:6825","nonce":3601682336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3601682336},{"type":"v1","addr":"172.21.15.70:6823","nonce":3601682336}]},"public_addr":"172.21.15.70:6819/3601682336","cluster_addr":"172.21.15.70:6821/3601682336","heartbeat_back_addr":"172.21.15.70:6825/3601682336","heartbeat_front_addr":"172.21.15.70:6823/3601682336","state":["exists","up"]},{"osd":3,"uuid":"ffb8d527-1b58-4ecd-808c-47d4ba1bf387","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2038083007},{"type":"v1","addr":"172.21.15.70:6827","nonce":2038083007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2038083007},{"type":"v1","addr":"172.21.15.70:6829","nonce":2038083007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":2038083007},{"type":"v1","addr":"172.21.15.70:6833","nonce":2038083007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2038083007},{"type":"v1","addr":"172.21.15.70:6831","nonce":2038083007}]},"public_addr":"172.21.15.70:6827/2038083007","cluster_addr":"172.21.15.70:6829/2038083007","heartbeat_back_addr":"172.21.15.70:6833/2038083007","heartbeat_front_addr":"172.21.15.70:6831/2038083007","state":["exists","up"]},{"osd":4,"uuid":"a946760b-4d91-4a4b-8e9b-a07a50589211","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2250892710},{"type":"v1","addr":"172.21.15.84:6801","nonce":2250892710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2250892710},{"type":"v1","addr":"172.21.15.84:6803","nonce":2250892710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2250892710},{"type":"v1","addr":"172.21.15.84:6807","nonce":2250892710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2250892710},{"type":"v1","addr":"172.21.15.84:6805","nonce":2250892710}]},"public_addr":"172.21.15.84:6801/2250892710","cluster_addr":"172.21.15.84:6803/2250892710","heartbeat_back_addr":"172.21.15.84:6807/2250892710","heartbeat_front_addr":"172.21.15.84:6805/2250892710","state":["exists","up"]},{"osd":5,"uuid":"67681760-75a9-4846-946e-a935d42c5e05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":1803945348},{"type":"v1","addr":"172.21.15.84:6809","nonce":1803945348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":1803945348},{"type":"v1","addr":"172.21.15.84:6811","nonce":1803945348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":1803945348},{"type":"v1","addr":"172.21.15.84:6815","nonce":1803945348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":1803945348},{"type":"v1","addr":"172.21.15.84:6813","nonce":1803945348}]},"public_addr":"172.21.15.84:6809/1803945348","cluster_addr":"172.21.15.84:6811/1803945348","heartbeat_back_addr":"172.21.15.84:6815/1803945348","heartbeat_front_addr":"172.21.15.84:6813/1803945348","state":["exists","up"]},{"osd":6,"uuid":"7c8f0152-f539-4b7e-8113-91dd7529d3a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":610727189},{"type":"v1","addr":"172.21.15.84:6817","nonce":610727189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":610727189},{"type":"v1","addr":"172.21.15.84:6819","nonce":610727189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":610727189},{"type":"v1","addr":"172.21.15.84:6823","nonce":610727189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":610727189},{"type":"v1","addr":"172.21.15.84:6821","nonce":610727189}]},"public_addr":"172.21.15.84:6817/610727189","cluster_addr":"172.21.15.84:6819/610727189","heartbeat_back_addr":"172.21.15.84:6823/610727189","heartbeat_front_addr":"172.21.15.84:6821/610727189","state":["exists","up"]},{"osd":7,"uuid":"945efaba-225e-49af-aefd-b4d58ca8093b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":1002252531},{"type":"v1","addr":"172.21.15.84:6825","nonce":1002252531}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1002252531},{"type":"v1","addr":"172.21.15.84:6827","nonce":1002252531}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":1002252531},{"type":"v1","addr":"172.21.15.84:6831","nonce":1002252531}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":1002252531},{"type":"v1","addr":"172.21.15.84:6829","nonce":1002252531}]},"public_addr":"172.21.15.84:6825/1002252531","cluster_addr":"172.21.15.84:6827/1002252531","heartbeat_back_addr":"172.21.15.84:6831/1002252531","heartbeat_front_addr":"172.21.15.84:6829/1002252531","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T09:25:17.586798+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T09:24:30.453059+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T09:24:58.067595+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T09:24:00.739156+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T09:24:51.788424+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T09:25:02.667150+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T09:24:19.886645+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T09:24:49.178172+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.70:0/1403001486":"2024-09-17T09:20:09.306330+0000","172.21.15.70:6801/3871553358":"2024-09-17T09:20:09.306330+0000","172.21.15.70:6800/3871553358":"2024-09-17T09:20:09.306330+0000","172.21.15.70:0/3351688485":"2024-09-17T09:20:09.306330+0000","172.21.15.70:6801/837338094":"2024-09-17T09:19:43.205989+0000","172.21.15.70:6800/837338094":"2024-09-17T09:19:43.205989+0000","172.21.15.70:0/1812373760":"2024-09-17T09:20:09.306330+0000","172.21.15.70:0/2213852865":"2024-09-17T09:19:43.205989+0000","172.21.15.70:0/186565817":"2024-09-17T09:19:43.205989+0000","172.21.15.70:0/3354284852":"2024-09-17T09:19:43.205989+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T09:25:22.660 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:25:22 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:25:22.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:22 smithi084 ceph-mon[32629]: pgmap v216: 132 pgs: 132 active+clean; 583 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 140 B/s wr, 2 op/s 2024-09-16T09:25:22.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:22 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/2921207031' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-16T09:25:22.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:22 smithi070 ceph-mon[30388]: pgmap v216: 132 pgs: 132 active+clean; 583 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 140 B/s wr, 2 op/s 2024-09-16T09:25:22.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:22 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2921207031' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-16T09:25:22.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:22 smithi070 ceph-mon[40691]: pgmap v216: 132 pgs: 132 active+clean; 583 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 140 B/s wr, 2 op/s 2024-09-16T09:25:22.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:22 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/2921207031' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-16T09:25:23.008 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-16T09:25:23.009 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T09:25:23.253 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:25:23.901 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:25:23.902 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:23 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:25:23.902 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":70,"fsid":"a1321834-740c-11ef-bceb-c7b262605968","created":"2024-09-16T09:19:14.121498+0000","modified":"2024-09-16T09:25:19.074889+0000","last_up_change":"2024-09-16T09:24:51.124496+0000","last_in_change":"2024-09-16T09:24:28.432920+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T09:22:27.617694+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"datapool","create_time":"2024-09-16T09:24:59.169244+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":61,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.6500000953674316,"score_stable":2.6500000953674316,"optimal_score":0.87999999523162842,"raw_score_acting":2.3299999237060547,"raw_score_stable":2.3299999237060547,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":".rgw.root","create_time":"2024-09-16T09:25:00.056119+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-09-16T09:25:01.770016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-09-16T09:25:03.725555+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-09-16T09:25:05.773530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7ef288c8-3bf0-4814-8e86-f70a7de041ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":2013075781},{"type":"v1","addr":"172.21.15.70:6803","nonce":2013075781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":2013075781},{"type":"v1","addr":"172.21.15.70:6805","nonce":2013075781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":2013075781},{"type":"v1","addr":"172.21.15.70:6809","nonce":2013075781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":2013075781},{"type":"v1","addr":"172.21.15.70:6807","nonce":2013075781}]},"public_addr":"172.21.15.70:6803/2013075781","cluster_addr":"172.21.15.70:6805/2013075781","heartbeat_back_addr":"172.21.15.70:6809/2013075781","heartbeat_front_addr":"172.21.15.70:6807/2013075781","state":["exists","up"]},{"osd":1,"uuid":"474d41d9-295a-450c-8f27-ce487a14678b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":1670012115},{"type":"v1","addr":"172.21.15.70:6811","nonce":1670012115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":1670012115},{"type":"v1","addr":"172.21.15.70:6813","nonce":1670012115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1670012115},{"type":"v1","addr":"172.21.15.70:6817","nonce":1670012115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":1670012115},{"type":"v1","addr":"172.21.15.70:6815","nonce":1670012115}]},"public_addr":"172.21.15.70:6811/1670012115","cluster_addr":"172.21.15.70:6813/1670012115","heartbeat_back_addr":"172.21.15.70:6817/1670012115","heartbeat_front_addr":"172.21.15.70:6815/1670012115","state":["exists","up"]},{"osd":2,"uuid":"5b4ca4e8-cecc-4163-b291-c9209b02c4b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3601682336},{"type":"v1","addr":"172.21.15.70:6819","nonce":3601682336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":3601682336},{"type":"v1","addr":"172.21.15.70:6821","nonce":3601682336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":3601682336},{"type":"v1","addr":"172.21.15.70:6825","nonce":3601682336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3601682336},{"type":"v1","addr":"172.21.15.70:6823","nonce":3601682336}]},"public_addr":"172.21.15.70:6819/3601682336","cluster_addr":"172.21.15.70:6821/3601682336","heartbeat_back_addr":"172.21.15.70:6825/3601682336","heartbeat_front_addr":"172.21.15.70:6823/3601682336","state":["exists","up"]},{"osd":3,"uuid":"ffb8d527-1b58-4ecd-808c-47d4ba1bf387","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2038083007},{"type":"v1","addr":"172.21.15.70:6827","nonce":2038083007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2038083007},{"type":"v1","addr":"172.21.15.70:6829","nonce":2038083007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":2038083007},{"type":"v1","addr":"172.21.15.70:6833","nonce":2038083007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2038083007},{"type":"v1","addr":"172.21.15.70:6831","nonce":2038083007}]},"public_addr":"172.21.15.70:6827/2038083007","cluster_addr":"172.21.15.70:6829/2038083007","heartbeat_back_addr":"172.21.15.70:6833/2038083007","heartbeat_front_addr":"172.21.15.70:6831/2038083007","state":["exists","up"]},{"osd":4,"uuid":"a946760b-4d91-4a4b-8e9b-a07a50589211","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2250892710},{"type":"v1","addr":"172.21.15.84:6801","nonce":2250892710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2250892710},{"type":"v1","addr":"172.21.15.84:6803","nonce":2250892710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2250892710},{"type":"v1","addr":"172.21.15.84:6807","nonce":2250892710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2250892710},{"type":"v1","addr":"172.21.15.84:6805","nonce":2250892710}]},"public_addr":"172.21.15.84:6801/2250892710","cluster_addr":"172.21.15.84:6803/2250892710","heartbeat_back_addr":"172.21.15.84:6807/2250892710","heartbeat_front_addr":"172.21.15.84:6805/2250892710","state":["exists","up"]},{"osd":5,"uuid":"67681760-75a9-4846-946e-a935d42c5e05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":1803945348},{"type":"v1","addr":"172.21.15.84:6809","nonce":1803945348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":1803945348},{"type":"v1","addr":"172.21.15.84:6811","nonce":1803945348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":1803945348},{"type":"v1","addr":"172.21.15.84:6815","nonce":1803945348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":1803945348},{"type":"v1","addr":"172.21.15.84:6813","nonce":1803945348}]},"public_addr":"172.21.15.84:6809/1803945348","cluster_addr":"172.21.15.84:6811/1803945348","heartbeat_back_addr":"172.21.15.84:6815/1803945348","heartbeat_front_addr":"172.21.15.84:6813/1803945348","state":["exists","up"]},{"osd":6,"uuid":"7c8f0152-f539-4b7e-8113-91dd7529d3a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":610727189},{"type":"v1","addr":"172.21.15.84:6817","nonce":610727189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":610727189},{"type":"v1","addr":"172.21.15.84:6819","nonce":610727189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":610727189},{"type":"v1","addr":"172.21.15.84:6823","nonce":610727189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":610727189},{"type":"v1","addr":"172.21.15.84:6821","nonce":610727189}]},"public_addr":"172.21.15.84:6817/610727189","cluster_addr":"172.21.15.84:6819/610727189","heartbeat_back_addr":"172.21.15.84:6823/610727189","heartbeat_front_addr":"172.21.15.84:6821/610727189","state":["exists","up"]},{"osd":7,"uuid":"945efaba-225e-49af-aefd-b4d58ca8093b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":1002252531},{"type":"v1","addr":"172.21.15.84:6825","nonce":1002252531}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1002252531},{"type":"v1","addr":"172.21.15.84:6827","nonce":1002252531}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":1002252531},{"type":"v1","addr":"172.21.15.84:6831","nonce":1002252531}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":1002252531},{"type":"v1","addr":"172.21.15.84:6829","nonce":1002252531}]},"public_addr":"172.21.15.84:6825/1002252531","cluster_addr":"172.21.15.84:6827/1002252531","heartbeat_back_addr":"172.21.15.84:6831/1002252531","heartbeat_front_addr":"172.21.15.84:6829/1002252531","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T09:25:17.586798+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T09:24:30.453059+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T09:24:58.067595+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T09:24:00.739156+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T09:24:51.788424+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T09:25:02.667150+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T09:24:19.886645+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T09:24:49.178172+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.70:0/1403001486":"2024-09-17T09:20:09.306330+0000","172.21.15.70:6801/3871553358":"2024-09-17T09:20:09.306330+0000","172.21.15.70:6800/3871553358":"2024-09-17T09:20:09.306330+0000","172.21.15.70:0/3351688485":"2024-09-17T09:20:09.306330+0000","172.21.15.70:6801/837338094":"2024-09-17T09:19:43.205989+0000","172.21.15.70:6800/837338094":"2024-09-17T09:19:43.205989+0000","172.21.15.70:0/1812373760":"2024-09-17T09:20:09.306330+0000","172.21.15.70:0/2213852865":"2024-09-17T09:19:43.205989+0000","172.21.15.70:0/186565817":"2024-09-17T09:19:43.205989+0000","172.21.15.70:0/3354284852":"2024-09-17T09:19:43.205989+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T09:25:23.903 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:23 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:25:23.909 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:23 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:25:24.590 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-16T09:25:24.591 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-16T09:25:24.591 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-16T09:25:24.592 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-16T09:25:24.592 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-16T09:25:24.592 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-16T09:25:24.593 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-16T09:25:24.593 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-16T09:25:24.827 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:24 smithi084 ceph-mon[32629]: pgmap v217: 132 pgs: 132 active+clean; 583 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 127 B/s wr, 2 op/s 2024-09-16T09:25:24.827 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:24 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/2760616102' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-16T09:25:24.828 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:24 smithi084 ceph-mon[32629]: osdmap e71: 8 total, 8 up, 8 in 2024-09-16T09:25:24.920 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:24 smithi070 ceph-mon[40691]: pgmap v217: 132 pgs: 132 active+clean; 583 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 127 B/s wr, 2 op/s 2024-09-16T09:25:24.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:24 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/2760616102' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-16T09:25:24.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:24 smithi070 ceph-mon[40691]: osdmap e71: 8 total, 8 up, 8 in 2024-09-16T09:25:24.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:24 smithi070 ceph-mon[30388]: pgmap v217: 132 pgs: 132 active+clean; 583 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 127 B/s wr, 2 op/s 2024-09-16T09:25:24.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:24 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2760616102' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-16T09:25:24.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:24 smithi070 ceph-mon[30388]: osdmap e71: 8 total, 8 up, 8 in 2024-09-16T09:25:25.064 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:25:25.086 INFO:journalctl@ceph.prometheus.a.smithi084.stdout:Sep 16 09:25:25 smithi084 podman[61392]: 2024-09-16 09:25:25.066315122 +0000 UTC m=+0.347220691 container start 4034d5ab5c22e3e89ae9028061761a10bdd8e8f976d568ff403087a1f3e2bbc6 (image=quay.io/prometheus/prometheus:v2.51.0, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-16T09:25:25.410 INFO:journalctl@ceph.prometheus.a.smithi084.stdout:Sep 16 09:25:25 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-prometheus-a[61406]: ts=2024-09-16T09:25:25.085Z caller=main.go:617 level=info msg="Starting Prometheus Server" mode=server version="(version=2.51.0, branch=HEAD, revision=c05c15512acb675e3f6cd662a6727854e93fc024)" 2024-09-16T09:25:25.410 INFO:journalctl@ceph.prometheus.a.smithi084.stdout:Sep 16 09:25:25 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-prometheus-a[61406]: ts=2024-09-16T09:25:25.085Z caller=main.go:622 level=info build_context="(go=go1.22.1, platform=linux/amd64, user=root@b5723e458358, date=20240319-10:54:45, tags=netgo,builtinassets,stringlabels)" 2024-09-16T09:25:25.411 INFO:journalctl@ceph.prometheus.a.smithi084.stdout:Sep 16 09:25:25 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-prometheus-a[61406]: ts=2024-09-16T09:25:25.085Z caller=main.go:623 level=info host_details="(Linux 5.14.0-505.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Sep 5 07:54:07 UTC 2024 x86_64 smithi084 (none))" 2024-09-16T09:25:25.411 INFO:journalctl@ceph.prometheus.a.smithi084.stdout:Sep 16 09:25:25 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-prometheus-a[61406]: ts=2024-09-16T09:25:25.085Z caller=main.go:624 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-09-16T09:25:25.411 INFO:journalctl@ceph.prometheus.a.smithi084.stdout:Sep 16 09:25:25 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-prometheus-a[61406]: ts=2024-09-16T09:25:25.085Z caller=main.go:625 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-09-16T09:25:25.411 INFO:journalctl@ceph.prometheus.a.smithi084.stdout:Sep 16 09:25:25 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-prometheus-a[61406]: ts=2024-09-16T09:25:25.087Z caller=web.go:568 level=info component=web msg="Start listening for connections" address=:9095 2024-09-16T09:25:25.411 INFO:journalctl@ceph.prometheus.a.smithi084.stdout:Sep 16 09:25:25 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-prometheus-a[61406]: ts=2024-09-16T09:25:25.087Z caller=main.go:1129 level=info msg="Starting TSDB ..." 2024-09-16T09:25:25.411 INFO:journalctl@ceph.prometheus.a.smithi084.stdout:Sep 16 09:25:25 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-prometheus-a[61406]: ts=2024-09-16T09:25:25.089Z caller=tls_config.go:313 level=info component=web msg="Listening on" address=[::]:9095 2024-09-16T09:25:25.411 INFO:journalctl@ceph.prometheus.a.smithi084.stdout:Sep 16 09:25:25 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-prometheus-a[61406]: ts=2024-09-16T09:25:25.089Z caller=tls_config.go:316 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-09-16T09:25:25.411 INFO:journalctl@ceph.prometheus.a.smithi084.stdout:Sep 16 09:25:25 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-prometheus-a[61406]: ts=2024-09-16T09:25:25.091Z caller=head.go:616 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-09-16T09:25:25.411 INFO:journalctl@ceph.prometheus.a.smithi084.stdout:Sep 16 09:25:25 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-prometheus-a[61406]: ts=2024-09-16T09:25:25.091Z caller=head.go:698 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=1.574µs 2024-09-16T09:25:25.411 INFO:journalctl@ceph.prometheus.a.smithi084.stdout:Sep 16 09:25:25 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-prometheus-a[61406]: ts=2024-09-16T09:25:25.091Z caller=head.go:706 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-09-16T09:25:25.411 INFO:journalctl@ceph.prometheus.a.smithi084.stdout:Sep 16 09:25:25 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-prometheus-a[61406]: ts=2024-09-16T09:25:25.091Z caller=head.go:778 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=0 2024-09-16T09:25:25.411 INFO:journalctl@ceph.prometheus.a.smithi084.stdout:Sep 16 09:25:25 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-prometheus-a[61406]: ts=2024-09-16T09:25:25.091Z caller=head.go:815 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=19.852µs wal_replay_duration=153.065µs wbl_replay_duration=114ns total_replay_duration=190.548µs 2024-09-16T09:25:25.412 INFO:journalctl@ceph.prometheus.a.smithi084.stdout:Sep 16 09:25:25 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-prometheus-a[61406]: ts=2024-09-16T09:25:25.092Z caller=main.go:1150 level=info fs_type=XFS_SUPER_MAGIC 2024-09-16T09:25:25.412 INFO:journalctl@ceph.prometheus.a.smithi084.stdout:Sep 16 09:25:25 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-prometheus-a[61406]: ts=2024-09-16T09:25:25.092Z caller=main.go:1153 level=info msg="TSDB started" 2024-09-16T09:25:25.412 INFO:journalctl@ceph.prometheus.a.smithi084.stdout:Sep 16 09:25:25 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-prometheus-a[61406]: ts=2024-09-16T09:25:25.092Z caller=main.go:1335 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-09-16T09:25:25.412 INFO:journalctl@ceph.prometheus.a.smithi084.stdout:Sep 16 09:25:25 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-prometheus-a[61406]: ts=2024-09-16T09:25:25.108Z caller=main.go:1372 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=15.350781ms db_storage=724ns remote_storage=1.04µs web_handler=373ns query_engine=580ns scrape=208.632µs scrape_sd=92.13µs notify=853ns notify_sd=778ns rules=14.741774ms tracing=5.72µs 2024-09-16T09:25:25.412 INFO:journalctl@ceph.prometheus.a.smithi084.stdout:Sep 16 09:25:25 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-prometheus-a[61406]: ts=2024-09-16T09:25:25.108Z caller=main.go:1114 level=info msg="Server is ready to receive web requests." 2024-09-16T09:25:25.412 INFO:journalctl@ceph.prometheus.a.smithi084.stdout:Sep 16 09:25:25 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-prometheus-a[61406]: ts=2024-09-16T09:25:25.108Z caller=manager.go:163 level=info component="rule manager" msg="Starting rule manager..." 2024-09-16T09:25:25.412 INFO:journalctl@ceph.prometheus.a.smithi084.stdout:Sep 16 09:25:25 smithi084 bash[61392]: 4034d5ab5c22e3e89ae9028061761a10bdd8e8f976d568ff403087a1f3e2bbc6 2024-09-16T09:25:25.412 INFO:journalctl@ceph.prometheus.a.smithi084.stdout:Sep 16 09:25:25 smithi084 systemd[1]: Started Ceph prometheus.a for a1321834-740c-11ef-bceb-c7b262605968. 2024-09-16T09:25:25.859 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:25:25.860 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:25:25.860 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:25:25.861 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:25:25.862 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:25:25.862 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:25:25.862 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:25:26.250 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:26 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:25:26.250 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:26 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:25:26.250 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:26 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:25:26.250 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:26 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "mgr module enable", "module": "prometheus"} : dispatch 2024-09-16T09:25:26.250 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:26 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:25:26.250 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:26 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:25:26.250 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:26 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:25:26.250 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:26 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "mgr module enable", "module": "prometheus"} : dispatch 2024-09-16T09:25:26.307 INFO:teuthology.orchestra.run.smithi070.stdout:197568495631 2024-09-16T09:25:26.307 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-16T09:25:26.607 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:25:26 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ignoring --setuser ceph since I am not root 2024-09-16T09:25:26.607 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:25:26 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ignoring --setgroup ceph since I am not root 2024-09-16T09:25:26.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:26 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:25:26.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:26 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:25:26.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:26 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' 2024-09-16T09:25:26.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:26 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd={"prefix": "mgr module enable", "module": "prometheus"} : dispatch 2024-09-16T09:25:26.660 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Sep 16 09:25:26 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-x[34286]: ignoring --setuser ceph since I am not root 2024-09-16T09:25:26.660 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Sep 16 09:25:26 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-x[34286]: ignoring --setgroup ceph since I am not root 2024-09-16T09:25:26.661 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Sep 16 09:25:26 smithi084 ceph-mgr[34309]: -- 172.21.15.84:0/2995178610 <== mon.1 v2:172.21.15.84:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5590e6b230e0 con 0x5590e6b01000 2024-09-16T09:25:27.470 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:27 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:25:27.470 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:27 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:25:27.470 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:27 smithi070 ceph-mon[30388]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-16T09:25:27.470 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:27 smithi070 ceph-mon[30388]: mgrmap e17: y(active, since 5m), standbys: x 2024-09-16T09:25:27.471 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:27 smithi070 ceph-mon[30388]: osdmap e72: 8 total, 8 up, 8 in 2024-09-16T09:25:27.471 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:27 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:25:27.471 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:27 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:25:27.471 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:27 smithi070 ceph-mon[40691]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-16T09:25:27.471 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:27 smithi070 ceph-mon[40691]: mgrmap e17: y(active, since 5m), standbys: x 2024-09-16T09:25:27.471 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:27 smithi070 ceph-mon[40691]: osdmap e72: 8 total, 8 up, 8 in 2024-09-16T09:25:27.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:27 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:25:27.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:27 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:25:27.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:27 smithi084 ceph-mon[32629]: from='mgr.14150 172.21.15.70:0/4092270513' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-16T09:25:27.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:27 smithi084 ceph-mon[32629]: mgrmap e17: y(active, since 5m), standbys: x 2024-09-16T09:25:27.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:27 smithi084 ceph-mon[32629]: osdmap e72: 8 total, 8 up, 8 in 2024-09-16T09:25:30.095 INFO:teuthology.orchestra.run.smithi070.stdout:55834574893 2024-09-16T09:25:30.095 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-16T09:25:31.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:31 smithi070 ceph-mon[30388]: Standby manager daemon x restarted 2024-09-16T09:25:31.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:31 smithi070 ceph-mon[30388]: Standby manager daemon x started 2024-09-16T09:25:31.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:31 smithi070 ceph-mon[30388]: from='mgr.? 172.21.15.84:0/658508551' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/x/crt"} : dispatch 2024-09-16T09:25:31.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:31 smithi070 ceph-mon[30388]: from='mgr.? 172.21.15.84:0/658508551' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-16T09:25:31.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:31 smithi070 ceph-mon[30388]: from='mgr.? 172.21.15.84:0/658508551' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/x/key"} : dispatch 2024-09-16T09:25:31.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:31 smithi070 ceph-mon[30388]: from='mgr.? 172.21.15.84:0/658508551' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-16T09:25:31.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:31 smithi070 ceph-mon[40691]: Standby manager daemon x restarted 2024-09-16T09:25:31.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:31 smithi070 ceph-mon[40691]: Standby manager daemon x started 2024-09-16T09:25:31.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:31 smithi070 ceph-mon[40691]: from='mgr.? 172.21.15.84:0/658508551' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/x/crt"} : dispatch 2024-09-16T09:25:31.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:31 smithi070 ceph-mon[40691]: from='mgr.? 172.21.15.84:0/658508551' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-16T09:25:31.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:31 smithi070 ceph-mon[40691]: from='mgr.? 172.21.15.84:0/658508551' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/x/key"} : dispatch 2024-09-16T09:25:31.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:31 smithi070 ceph-mon[40691]: from='mgr.? 172.21.15.84:0/658508551' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-16T09:25:31.518 INFO:teuthology.orchestra.run.smithi070.stdout:111669149729 2024-09-16T09:25:31.519 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-16T09:25:31.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:31 smithi084 ceph-mon[32629]: Standby manager daemon x restarted 2024-09-16T09:25:31.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:31 smithi084 ceph-mon[32629]: Standby manager daemon x started 2024-09-16T09:25:31.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:31 smithi084 ceph-mon[32629]: from='mgr.? 172.21.15.84:0/658508551' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/x/crt"} : dispatch 2024-09-16T09:25:31.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:31 smithi084 ceph-mon[32629]: from='mgr.? 172.21.15.84:0/658508551' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-16T09:25:31.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:31 smithi084 ceph-mon[32629]: from='mgr.? 172.21.15.84:0/658508551' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/x/key"} : dispatch 2024-09-16T09:25:31.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:31 smithi084 ceph-mon[32629]: from='mgr.? 172.21.15.84:0/658508551' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-16T09:25:31.661 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Sep 16 09:25:31 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-x[34286]: [16/Sep/2024:09:25:31] ENGINE Bus STARTING 2024-09-16T09:25:31.661 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Sep 16 09:25:31 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-x[34286]: CherryPy Checker: 2024-09-16T09:25:31.661 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Sep 16 09:25:31 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-x[34286]: The Application mounted at '' has an empty config. 2024-09-16T09:25:31.661 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Sep 16 09:25:31 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-x[34286]: 2024-09-16T09:25:31.661 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Sep 16 09:25:31 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-x[34286]: [16/Sep/2024:09:25:31] ENGINE Serving on http://:::9283 2024-09-16T09:25:31.661 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Sep 16 09:25:31 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-x[34286]: [16/Sep/2024:09:25:31] ENGINE Bus STARTED 2024-09-16T09:25:31.841 INFO:teuthology.orchestra.run.smithi070.stdout:167503724566 2024-09-16T09:25:31.841 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-16T09:25:32.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:32 smithi070 ceph-mon[30388]: mgrmap e18: y(active, since 5m), standbys: x 2024-09-16T09:25:32.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:32 smithi070 ceph-mon[30388]: Active manager daemon y restarted 2024-09-16T09:25:32.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:32 smithi070 ceph-mon[30388]: Activating manager daemon y 2024-09-16T09:25:32.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:32 smithi070 ceph-mon[30388]: osdmap e73: 8 total, 8 up, 8 in 2024-09-16T09:25:32.422 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:32 smithi070 ceph-mon[30388]: mgrmap e19: y(active, starting, since 0.00914142s), standbys: x 2024-09-16T09:25:32.422 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:32 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-16T09:25:32.422 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:32 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-16T09:25:32.422 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:32 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-16T09:25:32.422 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:32 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-16T09:25:32.422 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:32 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-16T09:25:32.422 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:32 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-16T09:25:32.422 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:32 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-16T09:25:32.422 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:32 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-16T09:25:32.422 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:32 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-16T09:25:32.423 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:32 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-16T09:25:32.423 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:32 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-16T09:25:32.423 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:32 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-16T09:25:32.423 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:32 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-16T09:25:32.423 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:32 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "mds metadata"} : dispatch 2024-09-16T09:25:32.423 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:32 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd metadata"} : dispatch 2024-09-16T09:25:32.423 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:32 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "mon metadata"} : dispatch 2024-09-16T09:25:32.423 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:32 smithi070 ceph-mon[30388]: Manager daemon y is now available 2024-09-16T09:25:32.423 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:32 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:25:32.423 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:32 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:25:32.423 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:32 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-16T09:25:32.423 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:32 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:25:32.424 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:32 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"} : dispatch 2024-09-16T09:25:32.424 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:32 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"} : dispatch 2024-09-16T09:25:32.424 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:32 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"} : dispatch 2024-09-16T09:25:32.424 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:32 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"} : dispatch 2024-09-16T09:25:32.424 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:32 smithi070 ceph-mon[40691]: mgrmap e18: y(active, since 5m), standbys: x 2024-09-16T09:25:32.424 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:32 smithi070 ceph-mon[40691]: Active manager daemon y restarted 2024-09-16T09:25:32.424 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:32 smithi070 ceph-mon[40691]: Activating manager daemon y 2024-09-16T09:25:32.424 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:32 smithi070 ceph-mon[40691]: osdmap e73: 8 total, 8 up, 8 in 2024-09-16T09:25:32.424 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:32 smithi070 ceph-mon[40691]: mgrmap e19: y(active, starting, since 0.00914142s), standbys: x 2024-09-16T09:25:32.425 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:32 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-16T09:25:32.425 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:32 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-16T09:25:32.426 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:32 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-16T09:25:32.426 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:32 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-16T09:25:32.426 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:32 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-16T09:25:32.426 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:32 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-16T09:25:32.426 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:32 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-16T09:25:32.426 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:32 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-16T09:25:32.426 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:32 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-16T09:25:32.426 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:32 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-16T09:25:32.426 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:32 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-16T09:25:32.426 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:32 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-16T09:25:32.427 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:32 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-16T09:25:32.427 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:32 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "mds metadata"} : dispatch 2024-09-16T09:25:32.427 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:32 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd metadata"} : dispatch 2024-09-16T09:25:32.427 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:32 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "mon metadata"} : dispatch 2024-09-16T09:25:32.427 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:32 smithi070 ceph-mon[40691]: Manager daemon y is now available 2024-09-16T09:25:32.427 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:32 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:25:32.427 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:32 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:25:32.427 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:32 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-16T09:25:32.427 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:32 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:25:32.427 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:32 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"} : dispatch 2024-09-16T09:25:32.427 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:32 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"} : dispatch 2024-09-16T09:25:32.427 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:32 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"} : dispatch 2024-09-16T09:25:32.428 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:32 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"} : dispatch 2024-09-16T09:25:32.660 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:25:32 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:25:32.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:32 smithi084 ceph-mon[32629]: mgrmap e18: y(active, since 5m), standbys: x 2024-09-16T09:25:32.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:32 smithi084 ceph-mon[32629]: Active manager daemon y restarted 2024-09-16T09:25:32.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:32 smithi084 ceph-mon[32629]: Activating manager daemon y 2024-09-16T09:25:32.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:32 smithi084 ceph-mon[32629]: osdmap e73: 8 total, 8 up, 8 in 2024-09-16T09:25:32.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:32 smithi084 ceph-mon[32629]: mgrmap e19: y(active, starting, since 0.00914142s), standbys: x 2024-09-16T09:25:32.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:32 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-16T09:25:32.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:32 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-16T09:25:32.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:32 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-16T09:25:32.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:32 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-16T09:25:32.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:32 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-16T09:25:32.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:32 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-16T09:25:32.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:32 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-16T09:25:32.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:32 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-16T09:25:32.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:32 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-16T09:25:32.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:32 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-16T09:25:32.662 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:32 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-16T09:25:32.662 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:32 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-16T09:25:32.662 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:32 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-16T09:25:32.662 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:32 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "mds metadata"} : dispatch 2024-09-16T09:25:32.662 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:32 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd metadata"} : dispatch 2024-09-16T09:25:32.662 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:32 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "mon metadata"} : dispatch 2024-09-16T09:25:32.662 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:32 smithi084 ceph-mon[32629]: Manager daemon y is now available 2024-09-16T09:25:32.662 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:32 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:25:32.662 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:32 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:25:32.662 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:32 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-16T09:25:32.662 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:32 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:25:32.662 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:32 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"} : dispatch 2024-09-16T09:25:32.662 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:32 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"} : dispatch 2024-09-16T09:25:32.662 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:32 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"} : dispatch 2024-09-16T09:25:32.663 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:32 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"} : dispatch 2024-09-16T09:25:32.791 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:25:32 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: [16/Sep/2024:09:25:32] ENGINE Bus STARTING 2024-09-16T09:25:32.791 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:25:32 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: CherryPy Checker: 2024-09-16T09:25:32.791 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:25:32 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: The Application mounted at '' has an empty config. 2024-09-16T09:25:32.791 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:25:32 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: 2024-09-16T09:25:32.791 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:25:32 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: [16/Sep/2024:09:25:32] ENGINE Serving on http://:::9283 2024-09-16T09:25:32.791 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:25:32 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: [16/Sep/2024:09:25:32] ENGINE Bus STARTED 2024-09-16T09:25:32.817 INFO:teuthology.orchestra.run.smithi070.stdout:137438953499 2024-09-16T09:25:32.817 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-16T09:25:33.477 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:25:33 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: 2024-09-16T09:25:33.257+0000 7fc9b70d0640 -1 log_channel(cephadm) log [ERR] : [16/Sep/2024:09:25:33] ENGINE Error in HTTPServer.serve 2024-09-16T09:25:33.478 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:25:33 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: Traceback (most recent call last): 2024-09-16T09:25:33.478 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:25:33 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T09:25:33.478 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:25:33 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: self._connections.run(self.expiration_interval) 2024-09-16T09:25:33.478 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:25:33 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T09:25:33.478 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:25:33 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: self._run(expiration_interval) 2024-09-16T09:25:33.478 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:25:33 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T09:25:33.478 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:25:33 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T09:25:33.478 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:25:33 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T09:25:33.478 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:25:33 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T09:25:33.478 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:25:33 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T09:25:33.478 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:25:33 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: s = self.context.wrap_socket( 2024-09-16T09:25:33.478 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:25:33 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T09:25:33.478 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:25:33 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: return self.sslsocket_class._create( 2024-09-16T09:25:33.478 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:25:33 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T09:25:33.478 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:25:33 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: self.do_handshake() 2024-09-16T09:25:33.479 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:25:33 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T09:25:33.479 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:25:33 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: self._sslobj.do_handshake() 2024-09-16T09:25:33.479 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:25:33 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T09:25:33.479 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:25:33 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: 2024-09-16T09:25:33.529 INFO:teuthology.orchestra.run.smithi070.stdout:77309411366 2024-09-16T09:25:33.530 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-16T09:25:33.806 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:33 smithi070 ceph-mon[30388]: mgrmap e20: y(active, since 1.01385s), standbys: x 2024-09-16T09:25:33.806 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:33 smithi070 ceph-mon[30388]: [16/Sep/2024:09:25:33] ENGINE Bus STARTING 2024-09-16T09:25:33.806 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:33 smithi070 ceph-mon[30388]: [16/Sep/2024:09:25:33] ENGINE Serving on https://172.21.15.70:7150 2024-09-16T09:25:33.806 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:33 smithi070 ceph-mon[30388]: [16/Sep/2024:09:25:33] ENGINE Error in HTTPServer.serve 2024-09-16T09:25:33.806 INFO:journalctl@ceph.mon.a.smithi070.stdout: Traceback (most recent call last): 2024-09-16T09:25:33.806 INFO:journalctl@ceph.mon.a.smithi070.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T09:25:33.806 INFO:journalctl@ceph.mon.a.smithi070.stdout: self._connections.run(self.expiration_interval) 2024-09-16T09:25:33.806 INFO:journalctl@ceph.mon.a.smithi070.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T09:25:33.806 INFO:journalctl@ceph.mon.a.smithi070.stdout: self._run(expiration_interval) 2024-09-16T09:25:33.806 INFO:journalctl@ceph.mon.a.smithi070.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T09:25:33.806 INFO:journalctl@ceph.mon.a.smithi070.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T09:25:33.807 INFO:journalctl@ceph.mon.a.smithi070.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T09:25:33.807 INFO:journalctl@ceph.mon.a.smithi070.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T09:25:33.807 INFO:journalctl@ceph.mon.a.smithi070.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T09:25:33.807 INFO:journalctl@ceph.mon.a.smithi070.stdout: s = self.context.wrap_socket( 2024-09-16T09:25:33.807 INFO:journalctl@ceph.mon.a.smithi070.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T09:25:33.807 INFO:journalctl@ceph.mon.a.smithi070.stdout: return self.sslsocket_class._create( 2024-09-16T09:25:33.807 INFO:journalctl@ceph.mon.a.smithi070.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T09:25:33.807 INFO:journalctl@ceph.mon.a.smithi070.stdout: self.do_handshake() 2024-09-16T09:25:33.807 INFO:journalctl@ceph.mon.a.smithi070.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T09:25:33.807 INFO:journalctl@ceph.mon.a.smithi070.stdout: self._sslobj.do_handshake() 2024-09-16T09:25:33.807 INFO:journalctl@ceph.mon.a.smithi070.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T09:25:33.807 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:33 smithi070 ceph-mon[40691]: mgrmap e20: y(active, since 1.01385s), standbys: x 2024-09-16T09:25:33.808 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:33 smithi070 ceph-mon[40691]: [16/Sep/2024:09:25:33] ENGINE Bus STARTING 2024-09-16T09:25:33.808 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:33 smithi070 ceph-mon[40691]: [16/Sep/2024:09:25:33] ENGINE Serving on https://172.21.15.70:7150 2024-09-16T09:25:33.808 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:33 smithi070 ceph-mon[40691]: [16/Sep/2024:09:25:33] ENGINE Error in HTTPServer.serve 2024-09-16T09:25:33.808 INFO:journalctl@ceph.mon.c.smithi070.stdout: Traceback (most recent call last): 2024-09-16T09:25:33.808 INFO:journalctl@ceph.mon.c.smithi070.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T09:25:33.808 INFO:journalctl@ceph.mon.c.smithi070.stdout: self._connections.run(self.expiration_interval) 2024-09-16T09:25:33.808 INFO:journalctl@ceph.mon.c.smithi070.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T09:25:33.808 INFO:journalctl@ceph.mon.c.smithi070.stdout: self._run(expiration_interval) 2024-09-16T09:25:33.808 INFO:journalctl@ceph.mon.c.smithi070.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T09:25:33.808 INFO:journalctl@ceph.mon.c.smithi070.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T09:25:33.808 INFO:journalctl@ceph.mon.c.smithi070.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T09:25:33.808 INFO:journalctl@ceph.mon.c.smithi070.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T09:25:33.808 INFO:journalctl@ceph.mon.c.smithi070.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T09:25:33.809 INFO:journalctl@ceph.mon.c.smithi070.stdout: s = self.context.wrap_socket( 2024-09-16T09:25:33.809 INFO:journalctl@ceph.mon.c.smithi070.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T09:25:33.809 INFO:journalctl@ceph.mon.c.smithi070.stdout: return self.sslsocket_class._create( 2024-09-16T09:25:33.809 INFO:journalctl@ceph.mon.c.smithi070.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T09:25:33.809 INFO:journalctl@ceph.mon.c.smithi070.stdout: self.do_handshake() 2024-09-16T09:25:33.809 INFO:journalctl@ceph.mon.c.smithi070.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T09:25:33.809 INFO:journalctl@ceph.mon.c.smithi070.stdout: self._sslobj.do_handshake() 2024-09-16T09:25:33.809 INFO:journalctl@ceph.mon.c.smithi070.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T09:25:33.832 INFO:teuthology.orchestra.run.smithi070.stdout:34359738419 2024-09-16T09:25:33.833 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-16T09:25:33.893 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:25:33.893 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:33 smithi084 ceph-mon[32629]: mgrmap e20: y(active, since 1.01385s), standbys: x 2024-09-16T09:25:33.893 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:33 smithi084 ceph-mon[32629]: [16/Sep/2024:09:25:33] ENGINE Bus STARTING 2024-09-16T09:25:33.894 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:33 smithi084 ceph-mon[32629]: [16/Sep/2024:09:25:33] ENGINE Serving on https://172.21.15.70:7150 2024-09-16T09:25:33.894 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:33 smithi084 ceph-mon[32629]: [16/Sep/2024:09:25:33] ENGINE Error in HTTPServer.serve 2024-09-16T09:25:33.894 INFO:journalctl@ceph.mon.b.smithi084.stdout: Traceback (most recent call last): 2024-09-16T09:25:33.894 INFO:journalctl@ceph.mon.b.smithi084.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T09:25:33.894 INFO:journalctl@ceph.mon.b.smithi084.stdout: self._connections.run(self.expiration_interval) 2024-09-16T09:25:33.894 INFO:journalctl@ceph.mon.b.smithi084.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T09:25:33.894 INFO:journalctl@ceph.mon.b.smithi084.stdout: self._run(expiration_interval) 2024-09-16T09:25:33.894 INFO:journalctl@ceph.mon.b.smithi084.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T09:25:33.894 INFO:journalctl@ceph.mon.b.smithi084.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T09:25:33.894 INFO:journalctl@ceph.mon.b.smithi084.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T09:25:33.894 INFO:journalctl@ceph.mon.b.smithi084.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T09:25:33.894 INFO:journalctl@ceph.mon.b.smithi084.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T09:25:33.894 INFO:journalctl@ceph.mon.b.smithi084.stdout: s = self.context.wrap_socket( 2024-09-16T09:25:33.894 INFO:journalctl@ceph.mon.b.smithi084.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T09:25:33.895 INFO:journalctl@ceph.mon.b.smithi084.stdout: return self.sslsocket_class._create( 2024-09-16T09:25:33.895 INFO:journalctl@ceph.mon.b.smithi084.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T09:25:33.895 INFO:journalctl@ceph.mon.b.smithi084.stdout: self.do_handshake() 2024-09-16T09:25:33.895 INFO:journalctl@ceph.mon.b.smithi084.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T09:25:33.895 INFO:journalctl@ceph.mon.b.smithi084.stdout: self._sslobj.do_handshake() 2024-09-16T09:25:33.895 INFO:journalctl@ceph.mon.b.smithi084.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T09:25:34.318 INFO:teuthology.orchestra.run.smithi070.stdout:227633266698 2024-09-16T09:25:34.319 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-16T09:25:34.377 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:25:34.610 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:25:34.614 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:34 smithi070 ceph-mon[30388]: [16/Sep/2024:09:25:33] ENGINE Serving on http://172.21.15.70:8765 2024-09-16T09:25:34.614 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:34 smithi070 ceph-mon[30388]: [16/Sep/2024:09:25:33] ENGINE Bus STARTED 2024-09-16T09:25:34.614 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:34 smithi070 ceph-mon[30388]: pgmap v4: 132 pgs: 132 active+clean; 583 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-09-16T09:25:34.614 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:34 smithi070 ceph-mon[30388]: mgrmap e21: y(active, since 2s), standbys: x 2024-09-16T09:25:34.614 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:34 smithi070 ceph-mon[30388]: osdmap e74: 8 total, 8 up, 8 in 2024-09-16T09:25:34.614 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:34 smithi070 ceph-mon[40691]: [16/Sep/2024:09:25:33] ENGINE Serving on http://172.21.15.70:8765 2024-09-16T09:25:34.614 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:34 smithi070 ceph-mon[40691]: [16/Sep/2024:09:25:33] ENGINE Bus STARTED 2024-09-16T09:25:34.615 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:34 smithi070 ceph-mon[40691]: pgmap v4: 132 pgs: 132 active+clean; 583 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-09-16T09:25:34.615 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:34 smithi070 ceph-mon[40691]: mgrmap e21: y(active, since 2s), standbys: x 2024-09-16T09:25:34.615 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:34 smithi070 ceph-mon[40691]: osdmap e74: 8 total, 8 up, 8 in 2024-09-16T09:25:34.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:34 smithi084 ceph-mon[32629]: [16/Sep/2024:09:25:33] ENGINE Serving on http://172.21.15.70:8765 2024-09-16T09:25:34.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:34 smithi084 ceph-mon[32629]: [16/Sep/2024:09:25:33] ENGINE Bus STARTED 2024-09-16T09:25:34.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:34 smithi084 ceph-mon[32629]: pgmap v4: 132 pgs: 132 active+clean; 583 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-09-16T09:25:34.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:34 smithi084 ceph-mon[32629]: mgrmap e21: y(active, since 2s), standbys: x 2024-09-16T09:25:34.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:34 smithi084 ceph-mon[32629]: osdmap e74: 8 total, 8 up, 8 in 2024-09-16T09:25:35.200 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:25:35.200 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:25:35.258 INFO:teuthology.orchestra.run.smithi070.stdout:197568495632 2024-09-16T09:25:35.655 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:35 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:25:35.655 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:35 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:25:35.655 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:35 smithi084 ceph-mon[32629]: osdmap e75: 8 total, 8 up, 8 in 2024-09-16T09:25:35.655 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:35 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/2242376630' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-16T09:25:35.655 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:35 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3479292328' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-16T09:25:35.920 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:35 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:25:35.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:35 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:25:35.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:35 smithi070 ceph-mon[40691]: osdmap e75: 8 total, 8 up, 8 in 2024-09-16T09:25:35.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:35 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/2242376630' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-16T09:25:35.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:35 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3479292328' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-16T09:25:35.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:35 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:25:35.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:35 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:25:35.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:35 smithi070 ceph-mon[30388]: osdmap e75: 8 total, 8 up, 8 in 2024-09-16T09:25:35.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:35 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2242376630' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-16T09:25:35.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:35 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3479292328' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-16T09:25:35.991 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495631 got 197568495632 for osd.6 2024-09-16T09:25:35.991 DEBUG:teuthology.parallel:result is None 2024-09-16T09:25:36.315 INFO:teuthology.orchestra.run.smithi070.stdout:77309411367 2024-09-16T09:25:36.987 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411366 got 77309411367 for osd.2 2024-09-16T09:25:36.987 DEBUG:teuthology.parallel:result is None 2024-09-16T09:25:37.283 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:37 smithi070 ceph-mon[30388]: pgmap v7: 132 pgs: 132 active+clean; 583 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-09-16T09:25:37.283 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:37 smithi070 ceph-mon[30388]: mgrmap e22: y(active, since 4s), standbys: x 2024-09-16T09:25:37.283 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:37 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:25:37.283 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:37 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:25:37.283 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:37 smithi070 ceph-mon[40691]: pgmap v7: 132 pgs: 132 active+clean; 583 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-09-16T09:25:37.283 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:37 smithi070 ceph-mon[40691]: mgrmap e22: y(active, since 4s), standbys: x 2024-09-16T09:25:37.284 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:37 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:25:37.284 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:37 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:25:37.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:37 smithi084 ceph-mon[32629]: pgmap v7: 132 pgs: 132 active+clean; 583 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-09-16T09:25:37.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:37 smithi084 ceph-mon[32629]: mgrmap e22: y(active, since 4s), standbys: x 2024-09-16T09:25:37.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:37 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:25:37.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:37 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:25:37.433 INFO:teuthology.orchestra.run.smithi070.stdout:111669149730 2024-09-16T09:25:37.884 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:25:38.422 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149729 got 111669149730 for osd.3 2024-09-16T09:25:38.423 DEBUG:teuthology.parallel:result is None 2024-09-16T09:25:38.545 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:38 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:25:38.545 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:38 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:25:38.545 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:38 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-16T09:25:38.545 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:38 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-16T09:25:38.546 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:38 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-16T09:25:38.546 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:38 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-16T09:25:38.546 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:38 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-16T09:25:38.546 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:38 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-16T09:25:38.546 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:38 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-16T09:25:38.546 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:38 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-16T09:25:38.546 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:38 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:25:38.546 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:38 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1608105247' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-16T09:25:38.546 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:38 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:25:38.546 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:38 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:25:38.546 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:38 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-16T09:25:38.546 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:38 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-16T09:25:38.547 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:38 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-16T09:25:38.547 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:38 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-16T09:25:38.547 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:38 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-16T09:25:38.547 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:38 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-16T09:25:38.547 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:38 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-16T09:25:38.547 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:38 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-16T09:25:38.547 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:38 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:25:38.547 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:38 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1608105247' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-16T09:25:38.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:38 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:25:38.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:38 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:25:38.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:38 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-16T09:25:38.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:38 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-16T09:25:38.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:38 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-16T09:25:38.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:38 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-16T09:25:38.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:38 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-16T09:25:38.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:38 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-16T09:25:38.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:38 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-16T09:25:38.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:38 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-16T09:25:38.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:38 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:25:38.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:38 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1608105247' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-16T09:25:39.207 INFO:teuthology.orchestra.run.smithi070.stdout:167503724567 2024-09-16T09:25:39.382 INFO:teuthology.orchestra.run.smithi070.stdout:55834574894 2024-09-16T09:25:39.382 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:39 smithi070 ceph-mon[30388]: Adjusting osd_memory_target on smithi084 to 3760M 2024-09-16T09:25:39.382 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:39 smithi070 ceph-mon[30388]: pgmap v8: 132 pgs: 132 active+clean; 583 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-09-16T09:25:39.382 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:39 smithi070 ceph-mon[30388]: osdmap e76: 8 total, 8 up, 8 in 2024-09-16T09:25:39.383 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:39 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3284617824' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-16T09:25:39.383 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:39 smithi070 ceph-mon[40691]: Adjusting osd_memory_target on smithi084 to 3760M 2024-09-16T09:25:39.383 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:39 smithi070 ceph-mon[40691]: pgmap v8: 132 pgs: 132 active+clean; 583 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-09-16T09:25:39.383 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:39 smithi070 ceph-mon[40691]: osdmap e76: 8 total, 8 up, 8 in 2024-09-16T09:25:39.383 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:39 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3284617824' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-16T09:25:39.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:39 smithi084 ceph-mon[32629]: Adjusting osd_memory_target on smithi084 to 3760M 2024-09-16T09:25:39.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:39 smithi084 ceph-mon[32629]: pgmap v8: 132 pgs: 132 active+clean; 583 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-09-16T09:25:39.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:39 smithi084 ceph-mon[32629]: osdmap e76: 8 total, 8 up, 8 in 2024-09-16T09:25:39.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:39 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3284617824' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-16T09:25:39.974 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574893 got 55834574894 for osd.1 2024-09-16T09:25:39.975 DEBUG:teuthology.parallel:result is None 2024-09-16T09:25:40.004 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:25:40.418 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724566 got 167503724567 for osd.5 2024-09-16T09:25:40.419 DEBUG:teuthology.parallel:result is None 2024-09-16T09:25:40.454 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:25:40.554 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:40 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/428323904' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-16T09:25:40.555 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:40 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/428323904' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-16T09:25:40.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:40 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/428323904' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-16T09:25:41.374 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:41 smithi070 ceph-mon[30388]: pgmap v10: 132 pgs: 132 active+clean; 583 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 12 op/s 2024-09-16T09:25:41.375 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:41 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:25:41.375 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:41 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:25:41.375 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:41 smithi070 ceph-mon[40691]: pgmap v10: 132 pgs: 132 active+clean; 583 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 12 op/s 2024-09-16T09:25:41.375 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:41 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:25:41.375 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:41 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:25:41.625 INFO:teuthology.orchestra.run.smithi070.stdout:137438953501 2024-09-16T09:25:41.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:41 smithi084 ceph-mon[32629]: pgmap v10: 132 pgs: 132 active+clean; 583 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 12 op/s 2024-09-16T09:25:41.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:41 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:25:41.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:41 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:25:41.661 INFO:teuthology.orchestra.run.smithi070.stdout:227633266700 2024-09-16T09:25:42.289 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:25:41 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:25:41] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:25:42.447 INFO:tasks.cephadm.ceph_manager.ceph:need seq 227633266698 got 227633266700 for osd.7 2024-09-16T09:25:42.447 DEBUG:teuthology.parallel:result is None 2024-09-16T09:25:42.576 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:42 smithi070 ceph-mon[30388]: osdmap e77: 8 total, 8 up, 8 in 2024-09-16T09:25:42.576 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:42 smithi070 ceph-mon[30388]: pgmap v12: 132 pgs: 132 active+clean; 583 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 28 KiB/s rd, 0 B/s wr, 12 op/s 2024-09-16T09:25:42.576 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:42 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:25:42.576 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:42 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:25:42.576 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:42 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2318025165' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-16T09:25:42.576 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:42 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2385061260' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-16T09:25:42.576 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:42 smithi070 ceph-mon[40691]: osdmap e77: 8 total, 8 up, 8 in 2024-09-16T09:25:42.576 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:42 smithi070 ceph-mon[40691]: pgmap v12: 132 pgs: 132 active+clean; 583 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 28 KiB/s rd, 0 B/s wr, 12 op/s 2024-09-16T09:25:42.577 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:42 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:25:42.577 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:42 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:25:42.577 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:42 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/2318025165' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-16T09:25:42.577 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:42 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/2385061260' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-16T09:25:42.660 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:25:42 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:25:42.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:42 smithi084 ceph-mon[32629]: osdmap e77: 8 total, 8 up, 8 in 2024-09-16T09:25:42.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:42 smithi084 ceph-mon[32629]: pgmap v12: 132 pgs: 132 active+clean; 583 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 28 KiB/s rd, 0 B/s wr, 12 op/s 2024-09-16T09:25:42.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:42 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:25:42.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:42 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:25:42.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:42 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/2318025165' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-16T09:25:42.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:42 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/2385061260' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-16T09:25:42.853 INFO:teuthology.orchestra.run.smithi070.stdout:34359738421 2024-09-16T09:25:43.033 INFO:tasks.cephadm.ceph_manager.ceph:need seq 137438953499 got 137438953501 for osd.4 2024-09-16T09:25:43.034 DEBUG:teuthology.parallel:result is None 2024-09-16T09:25:43.658 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:43 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:25:43.658 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:43 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/4256995052' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-16T09:25:43.658 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:43 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:25:43.658 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:43 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/4256995052' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-16T09:25:43.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:43 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:25:43.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:43 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/4256995052' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-16T09:25:43.687 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738419 got 34359738421 for osd.0 2024-09-16T09:25:43.687 DEBUG:teuthology.parallel:result is None 2024-09-16T09:25:43.687 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-16T09:25:43.688 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-16T09:25:43.978 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:25:44.714 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:25:44.716 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2024-09-16T09:25:45.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:45 smithi084 ceph-mon[32629]: pgmap v13: 132 pgs: 132 active+clean; 583 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-16T09:25:45.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:45 smithi084 ceph-mon[32629]: osdmap e78: 8 total, 8 up, 8 in 2024-09-16T09:25:45.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:45 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:25:45.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:45 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:25:45.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:45 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-16T09:25:45.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:45 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-16T09:25:45.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:45 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-16T09:25:45.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:45 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-16T09:25:45.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:45 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-16T09:25:45.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:45 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-16T09:25:45.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:45 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-16T09:25:45.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:45 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-16T09:25:45.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:45 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:25:45.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:45 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:25:45.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:45 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:25:45.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:45 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:25:45.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:45 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:25:45.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:45 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:25:45.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:45 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:25:45.412 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:45 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:25:45.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:45 smithi070 ceph-mon[30388]: pgmap v13: 132 pgs: 132 active+clean; 583 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-16T09:25:45.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:45 smithi070 ceph-mon[30388]: osdmap e78: 8 total, 8 up, 8 in 2024-09-16T09:25:45.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:45 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:25:45.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:45 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:25:45.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:45 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-16T09:25:45.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:45 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-16T09:25:45.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:45 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-16T09:25:45.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:45 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-16T09:25:45.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:45 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-16T09:25:45.422 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:45 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-16T09:25:45.422 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:45 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-16T09:25:45.422 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:45 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-16T09:25:45.422 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:45 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:25:45.422 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:45 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:25:45.422 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:45 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:25:45.422 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:45 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:25:45.422 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:45 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:25:45.422 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:45 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:25:45.422 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:45 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:25:45.422 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:45 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:25:45.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:45 smithi070 ceph-mon[40691]: pgmap v13: 132 pgs: 132 active+clean; 583 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-16T09:25:45.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:45 smithi070 ceph-mon[40691]: osdmap e78: 8 total, 8 up, 8 in 2024-09-16T09:25:45.423 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:45 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:25:45.423 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:45 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:25:45.423 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:45 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-16T09:25:45.423 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:45 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-16T09:25:45.423 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:45 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-16T09:25:45.423 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:45 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-16T09:25:45.423 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:45 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-16T09:25:45.423 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:45 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-16T09:25:45.423 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:45 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-16T09:25:45.423 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:45 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-16T09:25:45.423 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:45 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:25:45.423 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:45 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:25:45.423 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:45 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:25:45.423 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:45 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:25:45.424 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:45 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:25:45.424 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:45 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:25:45.424 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:45 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:25:45.424 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:45 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:25:45.579 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":14,"stamp":"2024-09-16T09:25:44.083639+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596507,"num_objects":199,"num_object_clones":0,"num_object_copies":597,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":199,"num_whiteouts":0,"num_read":883,"num_read_kb":740,"num_write":549,"num_write_kb":1673,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":552,"ondisk_log_size":552,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":241416,"kb_used_data":8540,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":749719800,"statfs":{"total":767960285184,"available":767713075200,"internally_reserved":0,"allocated":8744960,"data_stored":4801721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":238276146},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":70,"num_read_kb":186,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.604200"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":48,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.451430+0000","last_change":"2024-09-16T09:25:01.709392+0000","last_active":"2024-09-16T09:25:41.451430+0000","last_peered":"2024-09-16T09:25:41.451430+0000","last_clean":"2024-09-16T09:25:41.451430+0000","last_became_active":"2024-09-16T09:25:01.709108+0000","last_became_peered":"2024-09-16T09:25:01.709108+0000","last_unstale":"2024-09-16T09:25:41.451430+0000","last_undegraded":"2024-09-16T09:25:41.451430+0000","last_fullsized":"2024-09-16T09:25:41.451430+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:05.830525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"66'9","reported_seq":60,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.288293+0000","last_change":"2024-09-16T09:25:03.715139+0000","last_active":"2024-09-16T09:25:41.288293+0000","last_peered":"2024-09-16T09:25:41.288293+0000","last_clean":"2024-09-16T09:25:41.288293+0000","last_became_active":"2024-09-16T09:25:03.714635+0000","last_became_peered":"2024-09-16T09:25:03.714635+0000","last_unstale":"2024-09-16T09:25:41.288293+0000","last_undegraded":"2024-09-16T09:25:41.288293+0000","last_fullsized":"2024-09-16T09:25:41.288293+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:19.723608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":40,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.285710+0000","last_change":"2024-09-16T09:25:05.716139+0000","last_active":"2024-09-16T09:25:41.285710+0000","last_peered":"2024-09-16T09:25:41.285710+0000","last_clean":"2024-09-16T09:25:41.285710+0000","last_became_active":"2024-09-16T09:25:05.715645+0000","last_became_peered":"2024-09-16T09:25:05.715645+0000","last_unstale":"2024-09-16T09:25:41.285710+0000","last_undegraded":"2024-09-16T09:25:41.285710+0000","last_fullsized":"2024-09-16T09:25:41.285710+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:31.956608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.288428+0000","last_change":"2024-09-16T09:25:07.715923+0000","last_active":"2024-09-16T09:25:41.288428+0000","last_peered":"2024-09-16T09:25:41.288428+0000","last_clean":"2024-09-16T09:25:41.288428+0000","last_became_active":"2024-09-16T09:25:07.715442+0000","last_became_peered":"2024-09-16T09:25:07.715442+0000","last_unstale":"2024-09-16T09:25:41.288428+0000","last_undegraded":"2024-09-16T09:25:41.288428+0000","last_fullsized":"2024-09-16T09:25:41.288428+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:35.494958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.450629+0000","last_change":"2024-09-16T09:25:07.718687+0000","last_active":"2024-09-16T09:25:41.450629+0000","last_peered":"2024-09-16T09:25:41.450629+0000","last_clean":"2024-09-16T09:25:41.450629+0000","last_became_active":"2024-09-16T09:25:07.718126+0000","last_became_peered":"2024-09-16T09:25:07.718126+0000","last_unstale":"2024-09-16T09:25:41.450629+0000","last_undegraded":"2024-09-16T09:25:41.450629+0000","last_fullsized":"2024-09-16T09:25:41.450629+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:23.444563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":48,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.452007+0000","last_change":"2024-09-16T09:25:01.708505+0000","last_active":"2024-09-16T09:25:41.452007+0000","last_peered":"2024-09-16T09:25:41.452007+0000","last_clean":"2024-09-16T09:25:41.452007+0000","last_became_active":"2024-09-16T09:25:01.708179+0000","last_became_peered":"2024-09-16T09:25:01.708179+0000","last_unstale":"2024-09-16T09:25:41.452007+0000","last_undegraded":"2024-09-16T09:25:41.452007+0000","last_fullsized":"2024-09-16T09:25:41.452007+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:22.060114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"66'15","reported_seq":69,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.450943+0000","last_change":"2024-09-16T09:25:03.709970+0000","last_active":"2024-09-16T09:25:41.450943+0000","last_peered":"2024-09-16T09:25:41.450943+0000","last_clean":"2024-09-16T09:25:41.450943+0000","last_became_active":"2024-09-16T09:25:03.709499+0000","last_became_peered":"2024-09-16T09:25:03.709499+0000","last_unstale":"2024-09-16T09:25:41.450943+0000","last_undegraded":"2024-09-16T09:25:41.450943+0000","last_fullsized":"2024-09-16T09:25:41.450943+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:05.973210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":40,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.288218+0000","last_change":"2024-09-16T09:25:05.717524+0000","last_active":"2024-09-16T09:25:41.288218+0000","last_peered":"2024-09-16T09:25:41.288218+0000","last_clean":"2024-09-16T09:25:41.288218+0000","last_became_active":"2024-09-16T09:25:05.717236+0000","last_became_peered":"2024-09-16T09:25:05.717236+0000","last_unstale":"2024-09-16T09:25:41.288218+0000","last_undegraded":"2024-09-16T09:25:41.288218+0000","last_fullsized":"2024-09-16T09:25:41.288218+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:14.774263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":47,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.287761+0000","last_change":"2024-09-16T09:25:01.705552+0000","last_active":"2024-09-16T09:25:41.287761+0000","last_peered":"2024-09-16T09:25:41.287761+0000","last_clean":"2024-09-16T09:25:41.287761+0000","last_became_active":"2024-09-16T09:25:01.705269+0000","last_became_peered":"2024-09-16T09:25:01.705269+0000","last_unstale":"2024-09-16T09:25:41.287761+0000","last_undegraded":"2024-09-16T09:25:41.287761+0000","last_fullsized":"2024-09-16T09:25:41.287761+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:18.445329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"66'9","reported_seq":60,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.288675+0000","last_change":"2024-09-16T09:25:03.709441+0000","last_active":"2024-09-16T09:25:41.288675+0000","last_peered":"2024-09-16T09:25:41.288675+0000","last_clean":"2024-09-16T09:25:41.288675+0000","last_became_active":"2024-09-16T09:25:03.709147+0000","last_became_peered":"2024-09-16T09:25:03.709147+0000","last_unstale":"2024-09-16T09:25:41.288675+0000","last_undegraded":"2024-09-16T09:25:41.288675+0000","last_fullsized":"2024-09-16T09:25:41.288675+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:21.309548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":39,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.287291+0000","last_change":"2024-09-16T09:25:05.718510+0000","last_active":"2024-09-16T09:25:41.287291+0000","last_peered":"2024-09-16T09:25:41.287291+0000","last_clean":"2024-09-16T09:25:41.287291+0000","last_became_active":"2024-09-16T09:25:05.718134+0000","last_became_peered":"2024-09-16T09:25:05.718134+0000","last_unstale":"2024-09-16T09:25:41.287291+0000","last_undegraded":"2024-09-16T09:25:41.287291+0000","last_fullsized":"2024-09-16T09:25:41.287291+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:30.593901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"66'1","reported_seq":37,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.450466+0000","last_change":"2024-09-16T09:25:07.716869+0000","last_active":"2024-09-16T09:25:41.450466+0000","last_peered":"2024-09-16T09:25:41.450466+0000","last_clean":"2024-09-16T09:25:41.450466+0000","last_became_active":"2024-09-16T09:25:07.716558+0000","last_became_peered":"2024-09-16T09:25:07.716558+0000","last_unstale":"2024-09-16T09:25:41.450466+0000","last_undegraded":"2024-09-16T09:25:41.450466+0000","last_fullsized":"2024-09-16T09:25:41.450466+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:17.035822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":47,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.287563+0000","last_change":"2024-09-16T09:25:01.697306+0000","last_active":"2024-09-16T09:25:41.287563+0000","last_peered":"2024-09-16T09:25:41.287563+0000","last_clean":"2024-09-16T09:25:41.287563+0000","last_became_active":"2024-09-16T09:25:01.696939+0000","last_became_peered":"2024-09-16T09:25:01.696939+0000","last_unstale":"2024-09-16T09:25:41.287563+0000","last_undegraded":"2024-09-16T09:25:41.287563+0000","last_fullsized":"2024-09-16T09:25:41.287563+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:26.024926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"66'5","reported_seq":54,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.288575+0000","last_change":"2024-09-16T09:25:03.716402+0000","last_active":"2024-09-16T09:25:41.288575+0000","last_peered":"2024-09-16T09:25:41.288575+0000","last_clean":"2024-09-16T09:25:41.288575+0000","last_became_active":"2024-09-16T09:25:03.716136+0000","last_became_peered":"2024-09-16T09:25:03.716136+0000","last_unstale":"2024-09-16T09:25:41.288575+0000","last_undegraded":"2024-09-16T09:25:41.288575+0000","last_fullsized":"2024-09-16T09:25:41.288575+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:32.360413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":39,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.450089+0000","last_change":"2024-09-16T09:25:05.718538+0000","last_active":"2024-09-16T09:25:41.450089+0000","last_peered":"2024-09-16T09:25:41.450089+0000","last_clean":"2024-09-16T09:25:41.450089+0000","last_became_active":"2024-09-16T09:25:05.718281+0000","last_became_peered":"2024-09-16T09:25:05.718281+0000","last_unstale":"2024-09-16T09:25:41.450089+0000","last_undegraded":"2024-09-16T09:25:41.450089+0000","last_fullsized":"2024-09-16T09:25:41.450089+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:29.527824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":35,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.287032+0000","last_change":"2024-09-16T09:25:07.717644+0000","last_active":"2024-09-16T09:25:41.287032+0000","last_peered":"2024-09-16T09:25:41.287032+0000","last_clean":"2024-09-16T09:25:41.287032+0000","last_became_active":"2024-09-16T09:25:07.716724+0000","last_became_peered":"2024-09-16T09:25:07.716724+0000","last_unstale":"2024-09-16T09:25:41.287032+0000","last_undegraded":"2024-09-16T09:25:41.287032+0000","last_fullsized":"2024-09-16T09:25:41.287032+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:32.145892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.289092+0000","last_change":"2024-09-16T09:25:07.721088+0000","last_active":"2024-09-16T09:25:41.289092+0000","last_peered":"2024-09-16T09:25:41.289092+0000","last_clean":"2024-09-16T09:25:41.289092+0000","last_became_active":"2024-09-16T09:25:07.720793+0000","last_became_peered":"2024-09-16T09:25:07.720793+0000","last_unstale":"2024-09-16T09:25:41.289092+0000","last_undegraded":"2024-09-16T09:25:41.289092+0000","last_fullsized":"2024-09-16T09:25:41.289092+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:16.293244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":48,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.450819+0000","last_change":"2024-09-16T09:25:01.706762+0000","last_active":"2024-09-16T09:25:41.450819+0000","last_peered":"2024-09-16T09:25:41.450819+0000","last_clean":"2024-09-16T09:25:41.450819+0000","last_became_active":"2024-09-16T09:25:01.706172+0000","last_became_peered":"2024-09-16T09:25:01.706172+0000","last_unstale":"2024-09-16T09:25:41.450819+0000","last_undegraded":"2024-09-16T09:25:41.450819+0000","last_fullsized":"2024-09-16T09:25:41.450819+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:07.135883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"66'15","reported_seq":69,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.451311+0000","last_change":"2024-09-16T09:25:03.714261+0000","last_active":"2024-09-16T09:25:41.451311+0000","last_peered":"2024-09-16T09:25:41.451311+0000","last_clean":"2024-09-16T09:25:41.451311+0000","last_became_active":"2024-09-16T09:25:03.713780+0000","last_became_peered":"2024-09-16T09:25:03.713780+0000","last_unstale":"2024-09-16T09:25:41.451311+0000","last_undegraded":"2024-09-16T09:25:41.451311+0000","last_fullsized":"2024-09-16T09:25:41.451311+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:29.826960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":40,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.286690+0000","last_change":"2024-09-16T09:25:05.714209+0000","last_active":"2024-09-16T09:25:41.286690+0000","last_peered":"2024-09-16T09:25:41.286690+0000","last_clean":"2024-09-16T09:25:41.286690+0000","last_became_active":"2024-09-16T09:25:05.713929+0000","last_became_peered":"2024-09-16T09:25:05.713929+0000","last_unstale":"2024-09-16T09:25:41.286690+0000","last_undegraded":"2024-09-16T09:25:41.286690+0000","last_fullsized":"2024-09-16T09:25:41.286690+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:05.941022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.451826+0000","last_change":"2024-09-16T09:25:07.719645+0000","last_active":"2024-09-16T09:25:41.451826+0000","last_peered":"2024-09-16T09:25:41.451826+0000","last_clean":"2024-09-16T09:25:41.451826+0000","last_became_active":"2024-09-16T09:25:07.719067+0000","last_became_peered":"2024-09-16T09:25:07.719067+0000","last_unstale":"2024-09-16T09:25:41.451826+0000","last_undegraded":"2024-09-16T09:25:41.451826+0000","last_fullsized":"2024-09-16T09:25:41.451826+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:19.447150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":48,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.288858+0000","last_change":"2024-09-16T09:25:01.706828+0000","last_active":"2024-09-16T09:25:41.288858+0000","last_peered":"2024-09-16T09:25:41.288858+0000","last_clean":"2024-09-16T09:25:41.288858+0000","last_became_active":"2024-09-16T09:25:01.706491+0000","last_became_peered":"2024-09-16T09:25:01.706491+0000","last_unstale":"2024-09-16T09:25:41.288858+0000","last_undegraded":"2024-09-16T09:25:41.288858+0000","last_fullsized":"2024-09-16T09:25:41.288858+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:16.321414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"66'12","reported_seq":67,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.451236+0000","last_change":"2024-09-16T09:25:03.709914+0000","last_active":"2024-09-16T09:25:41.451236+0000","last_peered":"2024-09-16T09:25:41.451236+0000","last_clean":"2024-09-16T09:25:41.451236+0000","last_became_active":"2024-09-16T09:25:03.709610+0000","last_became_peered":"2024-09-16T09:25:03.709610+0000","last_unstale":"2024-09-16T09:25:41.451236+0000","last_undegraded":"2024-09-16T09:25:41.451236+0000","last_fullsized":"2024-09-16T09:25:41.451236+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:26.267096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":40,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.288654+0000","last_change":"2024-09-16T09:25:05.710379+0000","last_active":"2024-09-16T09:25:41.288654+0000","last_peered":"2024-09-16T09:25:41.288654+0000","last_clean":"2024-09-16T09:25:41.288654+0000","last_became_active":"2024-09-16T09:25:05.710093+0000","last_became_peered":"2024-09-16T09:25:05.710093+0000","last_unstale":"2024-09-16T09:25:41.288654+0000","last_undegraded":"2024-09-16T09:25:41.288654+0000","last_fullsized":"2024-09-16T09:25:41.288654+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:28.802108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.450171+0000","last_change":"2024-09-16T09:25:07.719655+0000","last_active":"2024-09-16T09:25:41.450171+0000","last_peered":"2024-09-16T09:25:41.450171+0000","last_clean":"2024-09-16T09:25:41.450171+0000","last_became_active":"2024-09-16T09:25:07.719380+0000","last_became_peered":"2024-09-16T09:25:07.719380+0000","last_unstale":"2024-09-16T09:25:41.450171+0000","last_undegraded":"2024-09-16T09:25:41.450171+0000","last_fullsized":"2024-09-16T09:25:41.450171+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:36.059179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":49,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.285864+0000","last_change":"2024-09-16T09:25:01.700263+0000","last_active":"2024-09-16T09:25:41.285864+0000","last_peered":"2024-09-16T09:25:41.285864+0000","last_clean":"2024-09-16T09:25:41.285864+0000","last_became_active":"2024-09-16T09:25:01.699545+0000","last_became_peered":"2024-09-16T09:25:01.699545+0000","last_unstale":"2024-09-16T09:25:41.285864+0000","last_undegraded":"2024-09-16T09:25:41.285864+0000","last_fullsized":"2024-09-16T09:25:41.285864+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:04.529119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"66'10","reported_seq":59,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.451722+0000","last_change":"2024-09-16T09:25:03.720638+0000","last_active":"2024-09-16T09:25:41.451722+0000","last_peered":"2024-09-16T09:25:41.451722+0000","last_clean":"2024-09-16T09:25:41.451722+0000","last_became_active":"2024-09-16T09:25:03.720245+0000","last_became_peered":"2024-09-16T09:25:03.720245+0000","last_unstale":"2024-09-16T09:25:41.451722+0000","last_undegraded":"2024-09-16T09:25:41.451722+0000","last_fullsized":"2024-09-16T09:25:41.451722+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:21.451538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"66'8","reported_seq":60,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.451113+0000","last_change":"2024-09-16T09:25:05.713310+0000","last_active":"2024-09-16T09:25:41.451113+0000","last_peered":"2024-09-16T09:25:41.451113+0000","last_clean":"2024-09-16T09:25:41.451113+0000","last_became_active":"2024-09-16T09:25:05.712516+0000","last_became_peered":"2024-09-16T09:25:05.712516+0000","last_unstale":"2024-09-16T09:25:41.451113+0000","last_undegraded":"2024-09-16T09:25:41.451113+0000","last_fullsized":"2024-09-16T09:25:41.451113+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:24.505283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.f","version":"66'15","reported_seq":69,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.286028+0000","last_change":"2024-09-16T09:25:03.719134+0000","last_active":"2024-09-16T09:25:41.286028+0000","last_peered":"2024-09-16T09:25:41.286028+0000","last_clean":"2024-09-16T09:25:41.286028+0000","last_became_active":"2024-09-16T09:25:03.718849+0000","last_became_peered":"2024-09-16T09:25:03.718849+0000","last_unstale":"2024-09-16T09:25:41.286028+0000","last_undegraded":"2024-09-16T09:25:41.286028+0000","last_fullsized":"2024-09-16T09:25:41.286028+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:13.173718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":48,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.450185+0000","last_change":"2024-09-16T09:25:01.702936+0000","last_active":"2024-09-16T09:25:41.450185+0000","last_peered":"2024-09-16T09:25:41.450185+0000","last_clean":"2024-09-16T09:25:41.450185+0000","last_became_active":"2024-09-16T09:25:01.702629+0000","last_became_peered":"2024-09-16T09:25:01.702629+0000","last_unstale":"2024-09-16T09:25:41.450185+0000","last_undegraded":"2024-09-16T09:25:41.450185+0000","last_fullsized":"2024-09-16T09:25:41.450185+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:20.061024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"66'8","reported_seq":57,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.289293+0000","last_change":"2024-09-16T09:25:05.714128+0000","last_active":"2024-09-16T09:25:41.289293+0000","last_peered":"2024-09-16T09:25:41.289293+0000","last_clean":"2024-09-16T09:25:41.289293+0000","last_became_active":"2024-09-16T09:25:05.713813+0000","last_became_peered":"2024-09-16T09:25:05.713813+0000","last_unstale":"2024-09-16T09:25:41.289293+0000","last_undegraded":"2024-09-16T09:25:41.289293+0000","last_fullsized":"2024-09-16T09:25:41.289293+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:26.013786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":35,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.287434+0000","last_change":"2024-09-16T09:25:07.717639+0000","last_active":"2024-09-16T09:25:41.287434+0000","last_peered":"2024-09-16T09:25:41.287434+0000","last_clean":"2024-09-16T09:25:41.287434+0000","last_became_active":"2024-09-16T09:25:07.717344+0000","last_became_peered":"2024-09-16T09:25:07.717344+0000","last_unstale":"2024-09-16T09:25:41.287434+0000","last_undegraded":"2024-09-16T09:25:41.287434+0000","last_fullsized":"2024-09-16T09:25:41.287434+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:24.054256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.0","version":"66'18","reported_seq":76,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.450068+0000","last_change":"2024-09-16T09:25:03.714508+0000","last_active":"2024-09-16T09:25:41.450068+0000","last_peered":"2024-09-16T09:25:41.450068+0000","last_clean":"2024-09-16T09:25:41.450068+0000","last_became_active":"2024-09-16T09:25:03.714208+0000","last_became_peered":"2024-09-16T09:25:03.714208+0000","last_unstale":"2024-09-16T09:25:41.450068+0000","last_undegraded":"2024-09-16T09:25:41.450068+0000","last_fullsized":"2024-09-16T09:25:41.450068+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:03.200598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":48,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.451912+0000","last_change":"2024-09-16T09:25:01.705362+0000","last_active":"2024-09-16T09:25:41.451912+0000","last_peered":"2024-09-16T09:25:41.451912+0000","last_clean":"2024-09-16T09:25:41.451912+0000","last_became_active":"2024-09-16T09:25:01.704994+0000","last_became_peered":"2024-09-16T09:25:01.704994+0000","last_unstale":"2024-09-16T09:25:41.451912+0000","last_undegraded":"2024-09-16T09:25:41.451912+0000","last_fullsized":"2024-09-16T09:25:41.451912+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:17.466780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":40,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.288317+0000","last_change":"2024-09-16T09:25:05.713687+0000","last_active":"2024-09-16T09:25:41.288317+0000","last_peered":"2024-09-16T09:25:41.288317+0000","last_clean":"2024-09-16T09:25:41.288317+0000","last_became_active":"2024-09-16T09:25:05.713360+0000","last_became_peered":"2024-09-16T09:25:05.713360+0000","last_unstale":"2024-09-16T09:25:41.288317+0000","last_undegraded":"2024-09-16T09:25:41.288317+0000","last_fullsized":"2024-09-16T09:25:41.288317+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:16.400241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.288739+0000","last_change":"2024-09-16T09:25:07.719412+0000","last_active":"2024-09-16T09:25:41.288739+0000","last_peered":"2024-09-16T09:25:41.288739+0000","last_clean":"2024-09-16T09:25:41.288739+0000","last_became_active":"2024-09-16T09:25:07.719159+0000","last_became_peered":"2024-09-16T09:25:07.719159+0000","last_unstale":"2024-09-16T09:25:41.288739+0000","last_undegraded":"2024-09-16T09:25:41.288739+0000","last_fullsized":"2024-09-16T09:25:41.288739+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:36.545361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"66'14","reported_seq":65,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.288323+0000","last_change":"2024-09-16T09:25:03.710438+0000","last_active":"2024-09-16T09:25:41.288323+0000","last_peered":"2024-09-16T09:25:41.288323+0000","last_clean":"2024-09-16T09:25:41.288323+0000","last_became_active":"2024-09-16T09:25:03.708830+0000","last_became_peered":"2024-09-16T09:25:03.708830+0000","last_unstale":"2024-09-16T09:25:41.288323+0000","last_undegraded":"2024-09-16T09:25:41.288323+0000","last_fullsized":"2024-09-16T09:25:41.288323+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:17.190767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":49,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.451907+0000","last_change":"2024-09-16T09:25:01.703245+0000","last_active":"2024-09-16T09:25:41.451907+0000","last_peered":"2024-09-16T09:25:41.451907+0000","last_clean":"2024-09-16T09:25:41.451907+0000","last_became_active":"2024-09-16T09:25:01.702992+0000","last_became_peered":"2024-09-16T09:25:01.702992+0000","last_unstale":"2024-09-16T09:25:41.451907+0000","last_undegraded":"2024-09-16T09:25:41.451907+0000","last_fullsized":"2024-09-16T09:25:41.451907+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:00.901034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"66'8","reported_seq":57,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.449989+0000","last_change":"2024-09-16T09:25:05.715927+0000","last_active":"2024-09-16T09:25:41.449989+0000","last_peered":"2024-09-16T09:25:41.449989+0000","last_clean":"2024-09-16T09:25:41.449989+0000","last_became_active":"2024-09-16T09:25:05.715606+0000","last_became_peered":"2024-09-16T09:25:05.715606+0000","last_unstale":"2024-09-16T09:25:41.449989+0000","last_undegraded":"2024-09-16T09:25:41.449989+0000","last_fullsized":"2024-09-16T09:25:41.449989+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:23.752404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":35,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.450175+0000","last_change":"2024-09-16T09:25:07.722287+0000","last_active":"2024-09-16T09:25:41.450175+0000","last_peered":"2024-09-16T09:25:41.450175+0000","last_clean":"2024-09-16T09:25:41.450175+0000","last_became_active":"2024-09-16T09:25:07.722017+0000","last_became_peered":"2024-09-16T09:25:07.722017+0000","last_unstale":"2024-09-16T09:25:41.450175+0000","last_undegraded":"2024-09-16T09:25:41.450175+0000","last_fullsized":"2024-09-16T09:25:41.450175+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:27.828108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2","version":"66'10","reported_seq":59,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.286027+0000","last_change":"2024-09-16T09:25:03.713368+0000","last_active":"2024-09-16T09:25:41.286027+0000","last_peered":"2024-09-16T09:25:41.286027+0000","last_clean":"2024-09-16T09:25:41.286027+0000","last_became_active":"2024-09-16T09:25:03.712985+0000","last_became_peered":"2024-09-16T09:25:03.712985+0000","last_unstale":"2024-09-16T09:25:41.286027+0000","last_undegraded":"2024-09-16T09:25:41.286027+0000","last_fullsized":"2024-09-16T09:25:41.286027+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:21.169460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":47,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.287870+0000","last_change":"2024-09-16T09:25:01.706774+0000","last_active":"2024-09-16T09:25:41.287870+0000","last_peered":"2024-09-16T09:25:41.287870+0000","last_clean":"2024-09-16T09:25:41.287870+0000","last_became_active":"2024-09-16T09:25:01.706440+0000","last_became_peered":"2024-09-16T09:25:01.706440+0000","last_unstale":"2024-09-16T09:25:41.287870+0000","last_undegraded":"2024-09-16T09:25:41.287870+0000","last_fullsized":"2024-09-16T09:25:41.287870+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:18.805280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"66'8","reported_seq":57,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.450426+0000","last_change":"2024-09-16T09:25:05.714825+0000","last_active":"2024-09-16T09:25:41.450426+0000","last_peered":"2024-09-16T09:25:41.450426+0000","last_clean":"2024-09-16T09:25:41.450426+0000","last_became_active":"2024-09-16T09:25:05.714507+0000","last_became_peered":"2024-09-16T09:25:05.714507+0000","last_unstale":"2024-09-16T09:25:41.450426+0000","last_undegraded":"2024-09-16T09:25:41.450426+0000","last_fullsized":"2024-09-16T09:25:41.450426+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:13.181647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.450619+0000","last_change":"2024-09-16T09:25:07.719876+0000","last_active":"2024-09-16T09:25:41.450619+0000","last_peered":"2024-09-16T09:25:41.450619+0000","last_clean":"2024-09-16T09:25:41.450619+0000","last_became_active":"2024-09-16T09:25:07.719604+0000","last_became_peered":"2024-09-16T09:25:07.719604+0000","last_unstale":"2024-09-16T09:25:41.450619+0000","last_undegraded":"2024-09-16T09:25:41.450619+0000","last_fullsized":"2024-09-16T09:25:41.450619+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:20.929075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"66'19","reported_seq":80,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.450624+0000","last_change":"2024-09-16T09:25:03.715086+0000","last_active":"2024-09-16T09:25:41.450624+0000","last_peered":"2024-09-16T09:25:41.450624+0000","last_clean":"2024-09-16T09:25:41.450624+0000","last_became_active":"2024-09-16T09:25:03.714797+0000","last_became_peered":"2024-09-16T09:25:03.714797+0000","last_unstale":"2024-09-16T09:25:41.450624+0000","last_undegraded":"2024-09-16T09:25:41.450624+0000","last_fullsized":"2024-09-16T09:25:41.450624+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:23.683038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":48,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.286819+0000","last_change":"2024-09-16T09:25:01.707093+0000","last_active":"2024-09-16T09:25:41.286819+0000","last_peered":"2024-09-16T09:25:41.286819+0000","last_clean":"2024-09-16T09:25:41.286819+0000","last_became_active":"2024-09-16T09:25:01.706806+0000","last_became_peered":"2024-09-16T09:25:01.706806+0000","last_unstale":"2024-09-16T09:25:41.286819+0000","last_undegraded":"2024-09-16T09:25:41.286819+0000","last_fullsized":"2024-09-16T09:25:41.286819+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:01.655926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":40,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.450009+0000","last_change":"2024-09-16T09:25:05.708949+0000","last_active":"2024-09-16T09:25:41.450009+0000","last_peered":"2024-09-16T09:25:41.450009+0000","last_clean":"2024-09-16T09:25:41.450009+0000","last_became_active":"2024-09-16T09:25:05.708663+0000","last_became_peered":"2024-09-16T09:25:05.708663+0000","last_unstale":"2024-09-16T09:25:41.450009+0000","last_undegraded":"2024-09-16T09:25:41.450009+0000","last_fullsized":"2024-09-16T09:25:41.450009+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:06.107203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.285607+0000","last_change":"2024-09-16T09:25:07.719568+0000","last_active":"2024-09-16T09:25:41.285607+0000","last_peered":"2024-09-16T09:25:41.285607+0000","last_clean":"2024-09-16T09:25:41.285607+0000","last_became_active":"2024-09-16T09:25:07.719267+0000","last_became_peered":"2024-09-16T09:25:07.719267+0000","last_unstale":"2024-09-16T09:25:41.285607+0000","last_undegraded":"2024-09-16T09:25:41.285607+0000","last_fullsized":"2024-09-16T09:25:41.285607+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:31.531856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"66'28","reported_seq":101,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.286729+0000","last_change":"2024-09-16T09:25:03.705300+0000","last_active":"2024-09-16T09:25:41.286729+0000","last_peered":"2024-09-16T09:25:41.286729+0000","last_clean":"2024-09-16T09:25:41.286729+0000","last_became_active":"2024-09-16T09:25:03.704878+0000","last_became_peered":"2024-09-16T09:25:03.704878+0000","last_unstale":"2024-09-16T09:25:41.286729+0000","last_undegraded":"2024-09-16T09:25:41.286729+0000","last_fullsized":"2024-09-16T09:25:41.286729+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:32.612703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":48,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.288933+0000","last_change":"2024-09-16T09:25:01.703394+0000","last_active":"2024-09-16T09:25:41.288933+0000","last_peered":"2024-09-16T09:25:41.288933+0000","last_clean":"2024-09-16T09:25:41.288933+0000","last_became_active":"2024-09-16T09:25:01.703076+0000","last_became_peered":"2024-09-16T09:25:01.703076+0000","last_unstale":"2024-09-16T09:25:41.288933+0000","last_undegraded":"2024-09-16T09:25:41.288933+0000","last_fullsized":"2024-09-16T09:25:41.288933+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:21.922041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":57,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.287228+0000","last_change":"2024-09-16T09:25:03.706980+0000","last_active":"2024-09-16T09:25:41.287228+0000","last_peered":"2024-09-16T09:25:41.287228+0000","last_clean":"2024-09-16T09:25:41.287228+0000","last_became_active":"2024-09-16T09:25:00.696237+0000","last_became_peered":"2024-09-16T09:25:00.696237+0000","last_unstale":"2024-09-16T09:25:41.287228+0000","last_undegraded":"2024-09-16T09:25:41.287228+0000","last_fullsized":"2024-09-16T09:25:41.287228+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:24:59.684950+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:24.893358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025137430000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":40,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.451030+0000","last_change":"2024-09-16T09:25:05.718319+0000","last_active":"2024-09-16T09:25:41.451030+0000","last_peered":"2024-09-16T09:25:41.451030+0000","last_clean":"2024-09-16T09:25:41.451030+0000","last_became_active":"2024-09-16T09:25:05.717591+0000","last_became_peered":"2024-09-16T09:25:05.717591+0000","last_unstale":"2024-09-16T09:25:41.451030+0000","last_undegraded":"2024-09-16T09:25:41.451030+0000","last_fullsized":"2024-09-16T09:25:41.451030+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:32.732085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.451542+0000","last_change":"2024-09-16T09:25:07.717720+0000","last_active":"2024-09-16T09:25:41.451542+0000","last_peered":"2024-09-16T09:25:41.451542+0000","last_clean":"2024-09-16T09:25:41.451542+0000","last_became_active":"2024-09-16T09:25:07.717436+0000","last_became_peered":"2024-09-16T09:25:07.717436+0000","last_unstale":"2024-09-16T09:25:41.451542+0000","last_undegraded":"2024-09-16T09:25:41.451542+0000","last_fullsized":"2024-09-16T09:25:41.451542+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:16.230936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"66'13","reported_seq":71,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.286120+0000","last_change":"2024-09-16T09:25:03.713631+0000","last_active":"2024-09-16T09:25:41.286120+0000","last_peered":"2024-09-16T09:25:41.286120+0000","last_clean":"2024-09-16T09:25:41.286120+0000","last_became_active":"2024-09-16T09:25:03.713241+0000","last_became_peered":"2024-09-16T09:25:03.713241+0000","last_unstale":"2024-09-16T09:25:41.286120+0000","last_undegraded":"2024-09-16T09:25:41.286120+0000","last_fullsized":"2024-09-16T09:25:41.286120+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:19.940601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":48,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.285789+0000","last_change":"2024-09-16T09:25:01.707526+0000","last_active":"2024-09-16T09:25:41.285789+0000","last_peered":"2024-09-16T09:25:41.285789+0000","last_clean":"2024-09-16T09:25:41.285789+0000","last_became_active":"2024-09-16T09:25:01.706828+0000","last_became_peered":"2024-09-16T09:25:01.706828+0000","last_unstale":"2024-09-16T09:25:41.285789+0000","last_undegraded":"2024-09-16T09:25:41.285789+0000","last_fullsized":"2024-09-16T09:25:41.285789+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:17.040873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":57,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.450361+0000","last_change":"2024-09-16T09:25:03.702772+0000","last_active":"2024-09-16T09:25:41.450361+0000","last_peered":"2024-09-16T09:25:41.450361+0000","last_clean":"2024-09-16T09:25:41.450361+0000","last_became_active":"2024-09-16T09:25:00.697822+0000","last_became_peered":"2024-09-16T09:25:00.697822+0000","last_unstale":"2024-09-16T09:25:41.450361+0000","last_undegraded":"2024-09-16T09:25:41.450361+0000","last_fullsized":"2024-09-16T09:25:41.450361+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:24:59.684950+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:05.731438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055022299999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":40,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.450983+0000","last_change":"2024-09-16T09:25:05.711840+0000","last_active":"2024-09-16T09:25:41.450983+0000","last_peered":"2024-09-16T09:25:41.450983+0000","last_clean":"2024-09-16T09:25:41.450983+0000","last_became_active":"2024-09-16T09:25:05.711539+0000","last_became_peered":"2024-09-16T09:25:05.711539+0000","last_unstale":"2024-09-16T09:25:41.450983+0000","last_undegraded":"2024-09-16T09:25:41.450983+0000","last_fullsized":"2024-09-16T09:25:41.450983+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:28.306667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":35,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.450427+0000","last_change":"2024-09-16T09:25:07.723554+0000","last_active":"2024-09-16T09:25:41.450427+0000","last_peered":"2024-09-16T09:25:41.450427+0000","last_clean":"2024-09-16T09:25:41.450427+0000","last_became_active":"2024-09-16T09:25:07.723282+0000","last_became_peered":"2024-09-16T09:25:07.723282+0000","last_unstale":"2024-09-16T09:25:41.450427+0000","last_undegraded":"2024-09-16T09:25:41.450427+0000","last_fullsized":"2024-09-16T09:25:41.450427+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:28.111790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"66'12","reported_seq":62,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.451823+0000","last_change":"2024-09-16T09:25:03.715612+0000","last_active":"2024-09-16T09:25:41.451823+0000","last_peered":"2024-09-16T09:25:41.451823+0000","last_clean":"2024-09-16T09:25:41.451823+0000","last_became_active":"2024-09-16T09:25:03.715315+0000","last_became_peered":"2024-09-16T09:25:03.715315+0000","last_unstale":"2024-09-16T09:25:41.451823+0000","last_undegraded":"2024-09-16T09:25:41.451823+0000","last_fullsized":"2024-09-16T09:25:41.451823+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:31.110979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"59'1","reported_seq":54,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.450107+0000","last_change":"2024-09-16T09:25:01.701885+0000","last_active":"2024-09-16T09:25:41.450107+0000","last_peered":"2024-09-16T09:25:41.450107+0000","last_clean":"2024-09-16T09:25:41.450107+0000","last_became_active":"2024-09-16T09:25:01.701599+0000","last_became_peered":"2024-09-16T09:25:01.701599+0000","last_unstale":"2024-09-16T09:25:41.450107+0000","last_undegraded":"2024-09-16T09:25:41.450107+0000","last_fullsized":"2024-09-16T09:25:41.450107+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:03.521969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"67'5","reported_seq":93,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.449893+0000","last_change":"2024-09-16T09:25:03.711738+0000","last_active":"2024-09-16T09:25:41.449893+0000","last_peered":"2024-09-16T09:25:41.449893+0000","last_clean":"2024-09-16T09:25:41.449893+0000","last_became_active":"2024-09-16T09:25:00.696546+0000","last_became_peered":"2024-09-16T09:25:00.696546+0000","last_unstale":"2024-09-16T09:25:41.449893+0000","last_undegraded":"2024-09-16T09:25:41.449893+0000","last_fullsized":"2024-09-16T09:25:41.449893+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:24:59.684950+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:17.241966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0059770420000000001,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":35,"num_read_kb":30,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":39,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.288042+0000","last_change":"2024-09-16T09:25:05.714994+0000","last_active":"2024-09-16T09:25:41.288042+0000","last_peered":"2024-09-16T09:25:41.288042+0000","last_clean":"2024-09-16T09:25:41.288042+0000","last_became_active":"2024-09-16T09:25:05.714704+0000","last_became_peered":"2024-09-16T09:25:05.714704+0000","last_unstale":"2024-09-16T09:25:41.288042+0000","last_undegraded":"2024-09-16T09:25:41.288042+0000","last_fullsized":"2024-09-16T09:25:41.288042+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:32.353177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.286415+0000","last_change":"2024-09-16T09:25:07.714719+0000","last_active":"2024-09-16T09:25:41.286415+0000","last_peered":"2024-09-16T09:25:41.286415+0000","last_clean":"2024-09-16T09:25:41.286415+0000","last_became_active":"2024-09-16T09:25:07.714158+0000","last_became_peered":"2024-09-16T09:25:07.714158+0000","last_unstale":"2024-09-16T09:25:41.286415+0000","last_undegraded":"2024-09-16T09:25:41.286415+0000","last_fullsized":"2024-09-16T09:25:41.286415+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:13.991008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"66'16","reported_seq":75,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.450962+0000","last_change":"2024-09-16T09:25:03.714405+0000","last_active":"2024-09-16T09:25:41.450962+0000","last_peered":"2024-09-16T09:25:41.450962+0000","last_clean":"2024-09-16T09:25:41.450962+0000","last_became_active":"2024-09-16T09:25:03.714118+0000","last_became_peered":"2024-09-16T09:25:03.714118+0000","last_unstale":"2024-09-16T09:25:41.450962+0000","last_undegraded":"2024-09-16T09:25:41.450962+0000","last_fullsized":"2024-09-16T09:25:41.450962+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:12.074949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":48,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.450535+0000","last_change":"2024-09-16T09:25:01.704246+0000","last_active":"2024-09-16T09:25:41.450535+0000","last_peered":"2024-09-16T09:25:41.450535+0000","last_clean":"2024-09-16T09:25:41.450535+0000","last_became_active":"2024-09-16T09:25:01.703730+0000","last_became_peered":"2024-09-16T09:25:01.703730+0000","last_unstale":"2024-09-16T09:25:41.450535+0000","last_undegraded":"2024-09-16T09:25:41.450535+0000","last_fullsized":"2024-09-16T09:25:41.450535+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:20.523303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"73'79","reported_seq":85,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.450216+0000","last_change":"2024-09-16T09:24:53.142015+0000","last_active":"2024-09-16T09:25:41.450216+0000","last_peered":"2024-09-16T09:25:41.450216+0000","last_clean":"2024-09-16T09:25:41.450216+0000","last_became_active":"2024-09-16T09:24:53.135890+0000","last_became_peered":"2024-09-16T09:24:53.135890+0000","last_unstale":"2024-09-16T09:25:41.450216+0000","last_undegraded":"2024-09-16T09:25:41.450216+0000","last_fullsized":"2024-09-16T09:25:41.450216+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"20'72","last_scrub_stamp":"2024-09-16T09:24:45.089989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:24:45.089989+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:10.885850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":39,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.450390+0000","last_change":"2024-09-16T09:25:05.714741+0000","last_active":"2024-09-16T09:25:41.450390+0000","last_peered":"2024-09-16T09:25:41.450390+0000","last_clean":"2024-09-16T09:25:41.450390+0000","last_became_active":"2024-09-16T09:25:05.714341+0000","last_became_peered":"2024-09-16T09:25:05.714341+0000","last_unstale":"2024-09-16T09:25:41.450390+0000","last_undegraded":"2024-09-16T09:25:41.450390+0000","last_fullsized":"2024-09-16T09:25:41.450390+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:21.685216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":35,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.287929+0000","last_change":"2024-09-16T09:25:07.714544+0000","last_active":"2024-09-16T09:25:41.287929+0000","last_peered":"2024-09-16T09:25:41.287929+0000","last_clean":"2024-09-16T09:25:41.287929+0000","last_became_active":"2024-09-16T09:25:07.714281+0000","last_became_peered":"2024-09-16T09:25:07.714281+0000","last_unstale":"2024-09-16T09:25:41.287929+0000","last_undegraded":"2024-09-16T09:25:41.287929+0000","last_fullsized":"2024-09-16T09:25:41.287929+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:13.491593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"66'11","reported_seq":63,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.289255+0000","last_change":"2024-09-16T09:25:03.715654+0000","last_active":"2024-09-16T09:25:41.289255+0000","last_peered":"2024-09-16T09:25:41.289255+0000","last_clean":"2024-09-16T09:25:41.289255+0000","last_became_active":"2024-09-16T09:25:03.715373+0000","last_became_peered":"2024-09-16T09:25:03.715373+0000","last_unstale":"2024-09-16T09:25:41.289255+0000","last_undegraded":"2024-09-16T09:25:41.289255+0000","last_fullsized":"2024-09-16T09:25:41.289255+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:11.446460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":48,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.288636+0000","last_change":"2024-09-16T09:25:01.708568+0000","last_active":"2024-09-16T09:25:41.288636+0000","last_peered":"2024-09-16T09:25:41.288636+0000","last_clean":"2024-09-16T09:25:41.288636+0000","last_became_active":"2024-09-16T09:25:01.708270+0000","last_became_peered":"2024-09-16T09:25:01.708270+0000","last_unstale":"2024-09-16T09:25:41.288636+0000","last_undegraded":"2024-09-16T09:25:41.288636+0000","last_fullsized":"2024-09-16T09:25:41.288636+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:24.212331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":39,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.287847+0000","last_change":"2024-09-16T09:25:05.712303+0000","last_active":"2024-09-16T09:25:41.287847+0000","last_peered":"2024-09-16T09:25:41.287847+0000","last_clean":"2024-09-16T09:25:41.287847+0000","last_became_active":"2024-09-16T09:25:05.711964+0000","last_became_peered":"2024-09-16T09:25:05.711964+0000","last_unstale":"2024-09-16T09:25:41.287847+0000","last_undegraded":"2024-09-16T09:25:41.287847+0000","last_fullsized":"2024-09-16T09:25:41.287847+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:23.392105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.451234+0000","last_change":"2024-09-16T09:25:07.716780+0000","last_active":"2024-09-16T09:25:41.451234+0000","last_peered":"2024-09-16T09:25:41.451234+0000","last_clean":"2024-09-16T09:25:41.451234+0000","last_became_active":"2024-09-16T09:25:07.716514+0000","last_became_peered":"2024-09-16T09:25:07.716514+0000","last_unstale":"2024-09-16T09:25:41.451234+0000","last_undegraded":"2024-09-16T09:25:41.451234+0000","last_fullsized":"2024-09-16T09:25:41.451234+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:07.166770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"66'17","reported_seq":72,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.289156+0000","last_change":"2024-09-16T09:25:03.713691+0000","last_active":"2024-09-16T09:25:41.289156+0000","last_peered":"2024-09-16T09:25:41.289156+0000","last_clean":"2024-09-16T09:25:41.289156+0000","last_became_active":"2024-09-16T09:25:03.713358+0000","last_became_peered":"2024-09-16T09:25:03.713358+0000","last_unstale":"2024-09-16T09:25:41.289156+0000","last_undegraded":"2024-09-16T09:25:41.289156+0000","last_fullsized":"2024-09-16T09:25:41.289156+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:11.619625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":48,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.450493+0000","last_change":"2024-09-16T09:25:01.703939+0000","last_active":"2024-09-16T09:25:41.450493+0000","last_peered":"2024-09-16T09:25:41.450493+0000","last_clean":"2024-09-16T09:25:41.450493+0000","last_became_active":"2024-09-16T09:25:01.703355+0000","last_became_peered":"2024-09-16T09:25:01.703355+0000","last_unstale":"2024-09-16T09:25:41.450493+0000","last_undegraded":"2024-09-16T09:25:41.450493+0000","last_fullsized":"2024-09-16T09:25:41.450493+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:03.217835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":40,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.286501+0000","last_change":"2024-09-16T09:25:05.717224+0000","last_active":"2024-09-16T09:25:41.286501+0000","last_peered":"2024-09-16T09:25:41.286501+0000","last_clean":"2024-09-16T09:25:41.286501+0000","last_became_active":"2024-09-16T09:25:05.716900+0000","last_became_peered":"2024-09-16T09:25:05.716900+0000","last_unstale":"2024-09-16T09:25:41.286501+0000","last_undegraded":"2024-09-16T09:25:41.286501+0000","last_fullsized":"2024-09-16T09:25:41.286501+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:30.026527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.450528+0000","last_change":"2024-09-16T09:25:07.721657+0000","last_active":"2024-09-16T09:25:41.450528+0000","last_peered":"2024-09-16T09:25:41.450528+0000","last_clean":"2024-09-16T09:25:41.450528+0000","last_became_active":"2024-09-16T09:25:07.721347+0000","last_became_peered":"2024-09-16T09:25:07.721347+0000","last_unstale":"2024-09-16T09:25:41.450528+0000","last_undegraded":"2024-09-16T09:25:41.450528+0000","last_fullsized":"2024-09-16T09:25:41.450528+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:15.094644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"66'10","reported_seq":59,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.288800+0000","last_change":"2024-09-16T09:25:03.708144+0000","last_active":"2024-09-16T09:25:41.288800+0000","last_peered":"2024-09-16T09:25:41.288800+0000","last_clean":"2024-09-16T09:25:41.288800+0000","last_became_active":"2024-09-16T09:25:03.707723+0000","last_became_peered":"2024-09-16T09:25:03.707723+0000","last_unstale":"2024-09-16T09:25:41.288800+0000","last_undegraded":"2024-09-16T09:25:41.288800+0000","last_fullsized":"2024-09-16T09:25:41.288800+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:24.890527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":48,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.451046+0000","last_change":"2024-09-16T09:25:01.703388+0000","last_active":"2024-09-16T09:25:41.451046+0000","last_peered":"2024-09-16T09:25:41.451046+0000","last_clean":"2024-09-16T09:25:41.451046+0000","last_became_active":"2024-09-16T09:25:01.703094+0000","last_became_peered":"2024-09-16T09:25:01.703094+0000","last_unstale":"2024-09-16T09:25:41.451046+0000","last_undegraded":"2024-09-16T09:25:41.451046+0000","last_fullsized":"2024-09-16T09:25:41.451046+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:07.049694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":60,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.450650+0000","last_change":"2024-09-16T09:25:05.711891+0000","last_active":"2024-09-16T09:25:41.450650+0000","last_peered":"2024-09-16T09:25:41.450650+0000","last_clean":"2024-09-16T09:25:41.450650+0000","last_became_active":"2024-09-16T09:25:05.711590+0000","last_became_peered":"2024-09-16T09:25:05.711590+0000","last_unstale":"2024-09-16T09:25:41.450650+0000","last_undegraded":"2024-09-16T09:25:41.450650+0000","last_fullsized":"2024-09-16T09:25:41.450650+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:17.630914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.289092+0000","last_change":"2024-09-16T09:25:07.719496+0000","last_active":"2024-09-16T09:25:41.289092+0000","last_peered":"2024-09-16T09:25:41.289092+0000","last_clean":"2024-09-16T09:25:41.289092+0000","last_became_active":"2024-09-16T09:25:07.718958+0000","last_became_peered":"2024-09-16T09:25:07.718958+0000","last_unstale":"2024-09-16T09:25:41.289092+0000","last_undegraded":"2024-09-16T09:25:41.289092+0000","last_fullsized":"2024-09-16T09:25:41.289092+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:15.393455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.b","version":"66'9","reported_seq":60,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.450529+0000","last_change":"2024-09-16T09:25:03.712553+0000","last_active":"2024-09-16T09:25:41.450529+0000","last_peered":"2024-09-16T09:25:41.450529+0000","last_clean":"2024-09-16T09:25:41.450529+0000","last_became_active":"2024-09-16T09:25:03.712230+0000","last_became_peered":"2024-09-16T09:25:03.712230+0000","last_unstale":"2024-09-16T09:25:41.450529+0000","last_undegraded":"2024-09-16T09:25:41.450529+0000","last_fullsized":"2024-09-16T09:25:41.450529+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:18.262732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":47,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.287757+0000","last_change":"2024-09-16T09:25:01.702943+0000","last_active":"2024-09-16T09:25:41.287757+0000","last_peered":"2024-09-16T09:25:41.287757+0000","last_clean":"2024-09-16T09:25:41.287757+0000","last_became_active":"2024-09-16T09:25:01.702629+0000","last_became_peered":"2024-09-16T09:25:01.702629+0000","last_unstale":"2024-09-16T09:25:41.287757+0000","last_undegraded":"2024-09-16T09:25:41.287757+0000","last_fullsized":"2024-09-16T09:25:41.287757+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:19.763325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":40,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.450175+0000","last_change":"2024-09-16T09:25:05.712932+0000","last_active":"2024-09-16T09:25:41.450175+0000","last_peered":"2024-09-16T09:25:41.450175+0000","last_clean":"2024-09-16T09:25:41.450175+0000","last_became_active":"2024-09-16T09:25:05.712573+0000","last_became_peered":"2024-09-16T09:25:05.712573+0000","last_unstale":"2024-09-16T09:25:41.450175+0000","last_undegraded":"2024-09-16T09:25:41.450175+0000","last_fullsized":"2024-09-16T09:25:41.450175+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:16.863815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.450014+0000","last_change":"2024-09-16T09:25:07.719073+0000","last_active":"2024-09-16T09:25:41.450014+0000","last_peered":"2024-09-16T09:25:41.450014+0000","last_clean":"2024-09-16T09:25:41.450014+0000","last_became_active":"2024-09-16T09:25:07.718778+0000","last_became_peered":"2024-09-16T09:25:07.718778+0000","last_unstale":"2024-09-16T09:25:41.450014+0000","last_undegraded":"2024-09-16T09:25:41.450014+0000","last_fullsized":"2024-09-16T09:25:41.450014+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:21.733588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"66'19","reported_seq":75,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.450244+0000","last_change":"2024-09-16T09:25:03.713182+0000","last_active":"2024-09-16T09:25:41.450244+0000","last_peered":"2024-09-16T09:25:41.450244+0000","last_clean":"2024-09-16T09:25:41.450244+0000","last_became_active":"2024-09-16T09:25:03.712783+0000","last_became_peered":"2024-09-16T09:25:03.712783+0000","last_unstale":"2024-09-16T09:25:41.450244+0000","last_undegraded":"2024-09-16T09:25:41.450244+0000","last_fullsized":"2024-09-16T09:25:41.450244+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:24.025685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"59'1","reported_seq":53,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.450756+0000","last_change":"2024-09-16T09:25:01.708571+0000","last_active":"2024-09-16T09:25:41.450756+0000","last_peered":"2024-09-16T09:25:41.450756+0000","last_clean":"2024-09-16T09:25:41.450756+0000","last_became_active":"2024-09-16T09:25:01.708069+0000","last_became_peered":"2024-09-16T09:25:01.708069+0000","last_unstale":"2024-09-16T09:25:41.450756+0000","last_undegraded":"2024-09-16T09:25:41.450756+0000","last_fullsized":"2024-09-16T09:25:41.450756+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:06.099334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":40,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.450446+0000","last_change":"2024-09-16T09:25:05.709549+0000","last_active":"2024-09-16T09:25:41.450446+0000","last_peered":"2024-09-16T09:25:41.450446+0000","last_clean":"2024-09-16T09:25:41.450446+0000","last_became_active":"2024-09-16T09:25:05.709264+0000","last_became_peered":"2024-09-16T09:25:05.709264+0000","last_unstale":"2024-09-16T09:25:41.450446+0000","last_undegraded":"2024-09-16T09:25:41.450446+0000","last_fullsized":"2024-09-16T09:25:41.450446+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:10.529230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":35,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.449962+0000","last_change":"2024-09-16T09:25:07.719009+0000","last_active":"2024-09-16T09:25:41.449962+0000","last_peered":"2024-09-16T09:25:41.449962+0000","last_clean":"2024-09-16T09:25:41.449962+0000","last_became_active":"2024-09-16T09:25:07.718750+0000","last_became_peered":"2024-09-16T09:25:07.718750+0000","last_unstale":"2024-09-16T09:25:41.449962+0000","last_undegraded":"2024-09-16T09:25:41.449962+0000","last_fullsized":"2024-09-16T09:25:41.449962+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:07.594257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"66'12","reported_seq":67,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.288515+0000","last_change":"2024-09-16T09:25:03.714309+0000","last_active":"2024-09-16T09:25:41.288515+0000","last_peered":"2024-09-16T09:25:41.288515+0000","last_clean":"2024-09-16T09:25:41.288515+0000","last_became_active":"2024-09-16T09:25:03.714027+0000","last_became_peered":"2024-09-16T09:25:03.714027+0000","last_unstale":"2024-09-16T09:25:41.288515+0000","last_undegraded":"2024-09-16T09:25:41.288515+0000","last_fullsized":"2024-09-16T09:25:41.288515+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:31.556984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":47,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.451019+0000","last_change":"2024-09-16T09:25:01.707905+0000","last_active":"2024-09-16T09:25:41.451019+0000","last_peered":"2024-09-16T09:25:41.451019+0000","last_clean":"2024-09-16T09:25:41.451019+0000","last_became_active":"2024-09-16T09:25:01.707528+0000","last_became_peered":"2024-09-16T09:25:01.707528+0000","last_unstale":"2024-09-16T09:25:41.451019+0000","last_undegraded":"2024-09-16T09:25:41.451019+0000","last_fullsized":"2024-09-16T09:25:41.451019+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:13.046169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":40,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.449892+0000","last_change":"2024-09-16T09:25:05.715584+0000","last_active":"2024-09-16T09:25:41.449892+0000","last_peered":"2024-09-16T09:25:41.449892+0000","last_clean":"2024-09-16T09:25:41.449892+0000","last_became_active":"2024-09-16T09:25:05.715253+0000","last_became_peered":"2024-09-16T09:25:05.715253+0000","last_unstale":"2024-09-16T09:25:41.449892+0000","last_undegraded":"2024-09-16T09:25:41.449892+0000","last_fullsized":"2024-09-16T09:25:41.449892+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:23.326757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.450714+0000","last_change":"2024-09-16T09:25:07.719338+0000","last_active":"2024-09-16T09:25:41.450714+0000","last_peered":"2024-09-16T09:25:41.450714+0000","last_clean":"2024-09-16T09:25:41.450714+0000","last_became_active":"2024-09-16T09:25:07.718899+0000","last_became_peered":"2024-09-16T09:25:07.718899+0000","last_unstale":"2024-09-16T09:25:41.450714+0000","last_undegraded":"2024-09-16T09:25:41.450714+0000","last_fullsized":"2024-09-16T09:25:41.450714+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:23.435495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.8","version":"66'15","reported_seq":68,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.286857+0000","last_change":"2024-09-16T09:25:03.716075+0000","last_active":"2024-09-16T09:25:41.286857+0000","last_peered":"2024-09-16T09:25:41.286857+0000","last_clean":"2024-09-16T09:25:41.286857+0000","last_became_active":"2024-09-16T09:25:03.715733+0000","last_became_peered":"2024-09-16T09:25:03.715733+0000","last_unstale":"2024-09-16T09:25:41.286857+0000","last_undegraded":"2024-09-16T09:25:41.286857+0000","last_fullsized":"2024-09-16T09:25:41.286857+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:20.577367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":59,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.451095+0000","last_change":"2024-09-16T09:25:01.707886+0000","last_active":"2024-09-16T09:25:41.451095+0000","last_peered":"2024-09-16T09:25:41.451095+0000","last_clean":"2024-09-16T09:25:41.451095+0000","last_became_active":"2024-09-16T09:25:01.707372+0000","last_became_peered":"2024-09-16T09:25:01.707372+0000","last_unstale":"2024-09-16T09:25:41.451095+0000","last_undegraded":"2024-09-16T09:25:41.451095+0000","last_fullsized":"2024-09-16T09:25:41.451095+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:14.328776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"66'8","reported_seq":56,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.450308+0000","last_change":"2024-09-16T09:25:05.716778+0000","last_active":"2024-09-16T09:25:41.450308+0000","last_peered":"2024-09-16T09:25:41.450308+0000","last_clean":"2024-09-16T09:25:41.450308+0000","last_became_active":"2024-09-16T09:25:05.716189+0000","last_became_peered":"2024-09-16T09:25:05.716189+0000","last_unstale":"2024-09-16T09:25:41.450308+0000","last_undegraded":"2024-09-16T09:25:41.450308+0000","last_fullsized":"2024-09-16T09:25:41.450308+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:31.128760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":35,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.287019+0000","last_change":"2024-09-16T09:25:07.717505+0000","last_active":"2024-09-16T09:25:41.287019+0000","last_peered":"2024-09-16T09:25:41.287019+0000","last_clean":"2024-09-16T09:25:41.287019+0000","last_became_active":"2024-09-16T09:25:07.717213+0000","last_became_peered":"2024-09-16T09:25:07.717213+0000","last_unstale":"2024-09-16T09:25:41.287019+0000","last_undegraded":"2024-09-16T09:25:41.287019+0000","last_fullsized":"2024-09-16T09:25:41.287019+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:09.355594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":48,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.450125+0000","last_change":"2024-09-16T09:25:01.703489+0000","last_active":"2024-09-16T09:25:41.450125+0000","last_peered":"2024-09-16T09:25:41.450125+0000","last_clean":"2024-09-16T09:25:41.450125+0000","last_became_active":"2024-09-16T09:25:01.703202+0000","last_became_peered":"2024-09-16T09:25:01.703202+0000","last_unstale":"2024-09-16T09:25:41.450125+0000","last_undegraded":"2024-09-16T09:25:41.450125+0000","last_fullsized":"2024-09-16T09:25:41.450125+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:30.581098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"66'6","reported_seq":53,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.451828+0000","last_change":"2024-09-16T09:25:03.713606+0000","last_active":"2024-09-16T09:25:41.451828+0000","last_peered":"2024-09-16T09:25:41.451828+0000","last_clean":"2024-09-16T09:25:41.451828+0000","last_became_active":"2024-09-16T09:25:03.713245+0000","last_became_peered":"2024-09-16T09:25:03.713245+0000","last_unstale":"2024-09-16T09:25:41.451828+0000","last_undegraded":"2024-09-16T09:25:41.451828+0000","last_fullsized":"2024-09-16T09:25:41.451828+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:15.800169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":39,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.287638+0000","last_change":"2024-09-16T09:25:05.715938+0000","last_active":"2024-09-16T09:25:41.287638+0000","last_peered":"2024-09-16T09:25:41.287638+0000","last_clean":"2024-09-16T09:25:41.287638+0000","last_became_active":"2024-09-16T09:25:05.715616+0000","last_became_peered":"2024-09-16T09:25:05.715616+0000","last_unstale":"2024-09-16T09:25:41.287638+0000","last_undegraded":"2024-09-16T09:25:41.287638+0000","last_fullsized":"2024-09-16T09:25:41.287638+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:25.224184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":35,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.450428+0000","last_change":"2024-09-16T09:25:07.721375+0000","last_active":"2024-09-16T09:25:41.450428+0000","last_peered":"2024-09-16T09:25:41.450428+0000","last_clean":"2024-09-16T09:25:41.450428+0000","last_became_active":"2024-09-16T09:25:07.721059+0000","last_became_peered":"2024-09-16T09:25:07.721059+0000","last_unstale":"2024-09-16T09:25:41.450428+0000","last_undegraded":"2024-09-16T09:25:41.450428+0000","last_fullsized":"2024-09-16T09:25:41.450428+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:16.275710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"66'9","reported_seq":60,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.451855+0000","last_change":"2024-09-16T09:25:03.719919+0000","last_active":"2024-09-16T09:25:41.451855+0000","last_peered":"2024-09-16T09:25:41.451855+0000","last_clean":"2024-09-16T09:25:41.451855+0000","last_became_active":"2024-09-16T09:25:03.719573+0000","last_became_peered":"2024-09-16T09:25:03.719573+0000","last_unstale":"2024-09-16T09:25:41.451855+0000","last_undegraded":"2024-09-16T09:25:41.451855+0000","last_fullsized":"2024-09-16T09:25:41.451855+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:23.239324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":47,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.450167+0000","last_change":"2024-09-16T09:25:01.707139+0000","last_active":"2024-09-16T09:25:41.450167+0000","last_peered":"2024-09-16T09:25:41.450167+0000","last_clean":"2024-09-16T09:25:41.450167+0000","last_became_active":"2024-09-16T09:25:01.706491+0000","last_became_peered":"2024-09-16T09:25:01.706491+0000","last_unstale":"2024-09-16T09:25:41.450167+0000","last_undegraded":"2024-09-16T09:25:41.450167+0000","last_fullsized":"2024-09-16T09:25:41.450167+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:07.438370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":40,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.451698+0000","last_change":"2024-09-16T09:25:05.715083+0000","last_active":"2024-09-16T09:25:41.451698+0000","last_peered":"2024-09-16T09:25:41.451698+0000","last_clean":"2024-09-16T09:25:41.451698+0000","last_became_active":"2024-09-16T09:25:05.714801+0000","last_became_peered":"2024-09-16T09:25:05.714801+0000","last_unstale":"2024-09-16T09:25:41.451698+0000","last_undegraded":"2024-09-16T09:25:41.451698+0000","last_fullsized":"2024-09-16T09:25:41.451698+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:15.084815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.451022+0000","last_change":"2024-09-16T09:25:07.722356+0000","last_active":"2024-09-16T09:25:41.451022+0000","last_peered":"2024-09-16T09:25:41.451022+0000","last_clean":"2024-09-16T09:25:41.451022+0000","last_became_active":"2024-09-16T09:25:07.722050+0000","last_became_peered":"2024-09-16T09:25:07.722050+0000","last_unstale":"2024-09-16T09:25:41.451022+0000","last_undegraded":"2024-09-16T09:25:41.451022+0000","last_fullsized":"2024-09-16T09:25:41.451022+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:35.793652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"66'9","reported_seq":59,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.287829+0000","last_change":"2024-09-16T09:25:03.717609+0000","last_active":"2024-09-16T09:25:41.287829+0000","last_peered":"2024-09-16T09:25:41.287829+0000","last_clean":"2024-09-16T09:25:41.287829+0000","last_became_active":"2024-09-16T09:25:03.717310+0000","last_became_peered":"2024-09-16T09:25:03.717310+0000","last_unstale":"2024-09-16T09:25:41.287829+0000","last_undegraded":"2024-09-16T09:25:41.287829+0000","last_fullsized":"2024-09-16T09:25:41.287829+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:27.281102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":48,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.449846+0000","last_change":"2024-09-16T09:25:01.702732+0000","last_active":"2024-09-16T09:25:41.449846+0000","last_peered":"2024-09-16T09:25:41.449846+0000","last_clean":"2024-09-16T09:25:41.449846+0000","last_became_active":"2024-09-16T09:25:01.702419+0000","last_became_peered":"2024-09-16T09:25:01.702419+0000","last_unstale":"2024-09-16T09:25:41.449846+0000","last_undegraded":"2024-09-16T09:25:41.449846+0000","last_fullsized":"2024-09-16T09:25:41.449846+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:11.390955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"66'8","reported_seq":57,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.451416+0000","last_change":"2024-09-16T09:25:05.711929+0000","last_active":"2024-09-16T09:25:41.451416+0000","last_peered":"2024-09-16T09:25:41.451416+0000","last_clean":"2024-09-16T09:25:41.451416+0000","last_became_active":"2024-09-16T09:25:05.711642+0000","last_became_peered":"2024-09-16T09:25:05.711642+0000","last_unstale":"2024-09-16T09:25:41.451416+0000","last_undegraded":"2024-09-16T09:25:41.451416+0000","last_fullsized":"2024-09-16T09:25:41.451416+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:29.307023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.289223+0000","last_change":"2024-09-16T09:25:07.719123+0000","last_active":"2024-09-16T09:25:41.289223+0000","last_peered":"2024-09-16T09:25:41.289223+0000","last_clean":"2024-09-16T09:25:41.289223+0000","last_became_active":"2024-09-16T09:25:07.718810+0000","last_became_peered":"2024-09-16T09:25:07.718810+0000","last_unstale":"2024-09-16T09:25:41.289223+0000","last_undegraded":"2024-09-16T09:25:41.289223+0000","last_fullsized":"2024-09-16T09:25:41.289223+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:16.185390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"66'10","reported_seq":59,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.451529+0000","last_change":"2024-09-16T09:25:03.712705+0000","last_active":"2024-09-16T09:25:41.451529+0000","last_peered":"2024-09-16T09:25:41.451529+0000","last_clean":"2024-09-16T09:25:41.451529+0000","last_became_active":"2024-09-16T09:25:03.712387+0000","last_became_peered":"2024-09-16T09:25:03.712387+0000","last_unstale":"2024-09-16T09:25:41.451529+0000","last_undegraded":"2024-09-16T09:25:41.451529+0000","last_fullsized":"2024-09-16T09:25:41.451529+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:26.552452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":47,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.450501+0000","last_change":"2024-09-16T09:25:01.708461+0000","last_active":"2024-09-16T09:25:41.450501+0000","last_peered":"2024-09-16T09:25:41.450501+0000","last_clean":"2024-09-16T09:25:41.450501+0000","last_became_active":"2024-09-16T09:25:01.707513+0000","last_became_peered":"2024-09-16T09:25:01.707513+0000","last_unstale":"2024-09-16T09:25:41.450501+0000","last_undegraded":"2024-09-16T09:25:41.450501+0000","last_fullsized":"2024-09-16T09:25:41.450501+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:10.338638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"66'8","reported_seq":56,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.287661+0000","last_change":"2024-09-16T09:25:05.714214+0000","last_active":"2024-09-16T09:25:41.287661+0000","last_peered":"2024-09-16T09:25:41.287661+0000","last_clean":"2024-09-16T09:25:41.287661+0000","last_became_active":"2024-09-16T09:25:05.713833+0000","last_became_peered":"2024-09-16T09:25:05.713833+0000","last_unstale":"2024-09-16T09:25:41.287661+0000","last_undegraded":"2024-09-16T09:25:41.287661+0000","last_fullsized":"2024-09-16T09:25:41.287661+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:09.052011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.451034+0000","last_change":"2024-09-16T09:25:07.717629+0000","last_active":"2024-09-16T09:25:41.451034+0000","last_peered":"2024-09-16T09:25:41.451034+0000","last_clean":"2024-09-16T09:25:41.451034+0000","last_became_active":"2024-09-16T09:25:07.717357+0000","last_became_peered":"2024-09-16T09:25:07.717357+0000","last_unstale":"2024-09-16T09:25:41.451034+0000","last_undegraded":"2024-09-16T09:25:41.451034+0000","last_fullsized":"2024-09-16T09:25:41.451034+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:28.965425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"66'11","reported_seq":63,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.288606+0000","last_change":"2024-09-16T09:25:03.715601+0000","last_active":"2024-09-16T09:25:41.288606+0000","last_peered":"2024-09-16T09:25:41.288606+0000","last_clean":"2024-09-16T09:25:41.288606+0000","last_became_active":"2024-09-16T09:25:03.715333+0000","last_became_peered":"2024-09-16T09:25:03.715333+0000","last_unstale":"2024-09-16T09:25:41.288606+0000","last_undegraded":"2024-09-16T09:25:41.288606+0000","last_fullsized":"2024-09-16T09:25:41.288606+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:05.154887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":48,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.288975+0000","last_change":"2024-09-16T09:25:01.705459+0000","last_active":"2024-09-16T09:25:41.288975+0000","last_peered":"2024-09-16T09:25:41.288975+0000","last_clean":"2024-09-16T09:25:41.288975+0000","last_became_active":"2024-09-16T09:25:01.705158+0000","last_became_peered":"2024-09-16T09:25:01.705158+0000","last_unstale":"2024-09-16T09:25:41.288975+0000","last_undegraded":"2024-09-16T09:25:41.288975+0000","last_fullsized":"2024-09-16T09:25:41.288975+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:17.236788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":40,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.285899+0000","last_change":"2024-09-16T09:25:05.713087+0000","last_active":"2024-09-16T09:25:41.285899+0000","last_peered":"2024-09-16T09:25:41.285899+0000","last_clean":"2024-09-16T09:25:41.285899+0000","last_became_active":"2024-09-16T09:25:05.712816+0000","last_became_peered":"2024-09-16T09:25:05.712816+0000","last_unstale":"2024-09-16T09:25:41.285899+0000","last_undegraded":"2024-09-16T09:25:41.285899+0000","last_fullsized":"2024-09-16T09:25:41.285899+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:29.267320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.450551+0000","last_change":"2024-09-16T09:25:07.713617+0000","last_active":"2024-09-16T09:25:41.450551+0000","last_peered":"2024-09-16T09:25:41.450551+0000","last_clean":"2024-09-16T09:25:41.450551+0000","last_became_active":"2024-09-16T09:25:07.713109+0000","last_became_peered":"2024-09-16T09:25:07.713109+0000","last_unstale":"2024-09-16T09:25:41.450551+0000","last_undegraded":"2024-09-16T09:25:41.450551+0000","last_fullsized":"2024-09-16T09:25:41.450551+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:17.988989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"66'9","reported_seq":60,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.286104+0000","last_change":"2024-09-16T09:25:03.706126+0000","last_active":"2024-09-16T09:25:41.286104+0000","last_peered":"2024-09-16T09:25:41.286104+0000","last_clean":"2024-09-16T09:25:41.286104+0000","last_became_active":"2024-09-16T09:25:03.705807+0000","last_became_peered":"2024-09-16T09:25:03.705807+0000","last_unstale":"2024-09-16T09:25:41.286104+0000","last_undegraded":"2024-09-16T09:25:41.286104+0000","last_fullsized":"2024-09-16T09:25:41.286104+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:25.655887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":47,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.450730+0000","last_change":"2024-09-16T09:25:01.707075+0000","last_active":"2024-09-16T09:25:41.450730+0000","last_peered":"2024-09-16T09:25:41.450730+0000","last_clean":"2024-09-16T09:25:41.450730+0000","last_became_active":"2024-09-16T09:25:01.706762+0000","last_became_peered":"2024-09-16T09:25:01.706762+0000","last_unstale":"2024-09-16T09:25:41.450730+0000","last_undegraded":"2024-09-16T09:25:41.450730+0000","last_fullsized":"2024-09-16T09:25:41.450730+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:13.882359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":40,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.450797+0000","last_change":"2024-09-16T09:25:05.714142+0000","last_active":"2024-09-16T09:25:41.450797+0000","last_peered":"2024-09-16T09:25:41.450797+0000","last_clean":"2024-09-16T09:25:41.450797+0000","last_became_active":"2024-09-16T09:25:05.713791+0000","last_became_peered":"2024-09-16T09:25:05.713791+0000","last_unstale":"2024-09-16T09:25:41.450797+0000","last_undegraded":"2024-09-16T09:25:41.450797+0000","last_fullsized":"2024-09-16T09:25:41.450797+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:20.280445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.450317+0000","last_change":"2024-09-16T09:25:07.715572+0000","last_active":"2024-09-16T09:25:41.450317+0000","last_peered":"2024-09-16T09:25:41.450317+0000","last_clean":"2024-09-16T09:25:41.450317+0000","last_became_active":"2024-09-16T09:25:07.715301+0000","last_became_peered":"2024-09-16T09:25:07.715301+0000","last_unstale":"2024-09-16T09:25:41.450317+0000","last_undegraded":"2024-09-16T09:25:41.450317+0000","last_fullsized":"2024-09-16T09:25:41.450317+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:15.062673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"66'11","reported_seq":63,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.450687+0000","last_change":"2024-09-16T09:25:03.712637+0000","last_active":"2024-09-16T09:25:41.450687+0000","last_peered":"2024-09-16T09:25:41.450687+0000","last_clean":"2024-09-16T09:25:41.450687+0000","last_became_active":"2024-09-16T09:25:03.712333+0000","last_became_peered":"2024-09-16T09:25:03.712333+0000","last_unstale":"2024-09-16T09:25:41.450687+0000","last_undegraded":"2024-09-16T09:25:41.450687+0000","last_fullsized":"2024-09-16T09:25:41.450687+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:06.593958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":47,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.287760+0000","last_change":"2024-09-16T09:25:01.701295+0000","last_active":"2024-09-16T09:25:41.287760+0000","last_peered":"2024-09-16T09:25:41.287760+0000","last_clean":"2024-09-16T09:25:41.287760+0000","last_became_active":"2024-09-16T09:25:01.700733+0000","last_became_peered":"2024-09-16T09:25:01.700733+0000","last_unstale":"2024-09-16T09:25:41.287760+0000","last_undegraded":"2024-09-16T09:25:41.287760+0000","last_fullsized":"2024-09-16T09:25:41.287760+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:14.332112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":39,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.449631+0000","last_change":"2024-09-16T09:25:05.713760+0000","last_active":"2024-09-16T09:25:41.449631+0000","last_peered":"2024-09-16T09:25:41.449631+0000","last_clean":"2024-09-16T09:25:41.449631+0000","last_became_active":"2024-09-16T09:25:05.713479+0000","last_became_peered":"2024-09-16T09:25:05.713479+0000","last_unstale":"2024-09-16T09:25:41.449631+0000","last_undegraded":"2024-09-16T09:25:41.449631+0000","last_fullsized":"2024-09-16T09:25:41.449631+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:31.316771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.450942+0000","last_change":"2024-09-16T09:25:07.715946+0000","last_active":"2024-09-16T09:25:41.450942+0000","last_peered":"2024-09-16T09:25:41.450942+0000","last_clean":"2024-09-16T09:25:41.450942+0000","last_became_active":"2024-09-16T09:25:07.715582+0000","last_became_peered":"2024-09-16T09:25:07.715582+0000","last_unstale":"2024-09-16T09:25:41.450942+0000","last_undegraded":"2024-09-16T09:25:41.450942+0000","last_fullsized":"2024-09-16T09:25:41.450942+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:18.541254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"66'4","reported_seq":50,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.450044+0000","last_change":"2024-09-16T09:25:03.705167+0000","last_active":"2024-09-16T09:25:41.450044+0000","last_peered":"2024-09-16T09:25:41.450044+0000","last_clean":"2024-09-16T09:25:41.450044+0000","last_became_active":"2024-09-16T09:25:03.704869+0000","last_became_peered":"2024-09-16T09:25:03.704869+0000","last_unstale":"2024-09-16T09:25:41.450044+0000","last_undegraded":"2024-09-16T09:25:41.450044+0000","last_fullsized":"2024-09-16T09:25:41.450044+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:17.043143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":48,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.451506+0000","last_change":"2024-09-16T09:25:01.705965+0000","last_active":"2024-09-16T09:25:41.451506+0000","last_peered":"2024-09-16T09:25:41.451506+0000","last_clean":"2024-09-16T09:25:41.451506+0000","last_became_active":"2024-09-16T09:25:01.705686+0000","last_became_peered":"2024-09-16T09:25:01.705686+0000","last_unstale":"2024-09-16T09:25:41.451506+0000","last_undegraded":"2024-09-16T09:25:41.451506+0000","last_fullsized":"2024-09-16T09:25:41.451506+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:16.022170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":40,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.450011+0000","last_change":"2024-09-16T09:25:05.712816+0000","last_active":"2024-09-16T09:25:41.450011+0000","last_peered":"2024-09-16T09:25:41.450011+0000","last_clean":"2024-09-16T09:25:41.450011+0000","last_became_active":"2024-09-16T09:25:05.712545+0000","last_became_peered":"2024-09-16T09:25:05.712545+0000","last_unstale":"2024-09-16T09:25:41.450011+0000","last_undegraded":"2024-09-16T09:25:41.450011+0000","last_fullsized":"2024-09-16T09:25:41.450011+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:08.170532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":35,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.449875+0000","last_change":"2024-09-16T09:25:07.720368+0000","last_active":"2024-09-16T09:25:41.449875+0000","last_peered":"2024-09-16T09:25:41.449875+0000","last_clean":"2024-09-16T09:25:41.449875+0000","last_became_active":"2024-09-16T09:25:07.720117+0000","last_became_peered":"2024-09-16T09:25:07.720117+0000","last_unstale":"2024-09-16T09:25:41.449875+0000","last_undegraded":"2024-09-16T09:25:41.449875+0000","last_fullsized":"2024-09-16T09:25:41.449875+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:31.504626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.286461+0000","last_change":"2024-09-16T09:25:07.712880+0000","last_active":"2024-09-16T09:25:41.286461+0000","last_peered":"2024-09-16T09:25:41.286461+0000","last_clean":"2024-09-16T09:25:41.286461+0000","last_became_active":"2024-09-16T09:25:07.712594+0000","last_became_peered":"2024-09-16T09:25:07.712594+0000","last_unstale":"2024-09-16T09:25:41.286461+0000","last_undegraded":"2024-09-16T09:25:41.286461+0000","last_fullsized":"2024-09-16T09:25:41.286461+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:33.865572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":48,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.450157+0000","last_change":"2024-09-16T09:25:01.704711+0000","last_active":"2024-09-16T09:25:41.450157+0000","last_peered":"2024-09-16T09:25:41.450157+0000","last_clean":"2024-09-16T09:25:41.450157+0000","last_became_active":"2024-09-16T09:25:01.703736+0000","last_became_peered":"2024-09-16T09:25:01.703736+0000","last_unstale":"2024-09-16T09:25:41.450157+0000","last_undegraded":"2024-09-16T09:25:41.450157+0000","last_fullsized":"2024-09-16T09:25:41.450157+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:09.913011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"66'11","reported_seq":63,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.451122+0000","last_change":"2024-09-16T09:25:03.708095+0000","last_active":"2024-09-16T09:25:41.451122+0000","last_peered":"2024-09-16T09:25:41.451122+0000","last_clean":"2024-09-16T09:25:41.451122+0000","last_became_active":"2024-09-16T09:25:03.707769+0000","last_became_peered":"2024-09-16T09:25:03.707769+0000","last_unstale":"2024-09-16T09:25:41.451122+0000","last_undegraded":"2024-09-16T09:25:41.451122+0000","last_fullsized":"2024-09-16T09:25:41.451122+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:21.606932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":40,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.451646+0000","last_change":"2024-09-16T09:25:05.712247+0000","last_active":"2024-09-16T09:25:41.451646+0000","last_peered":"2024-09-16T09:25:41.451646+0000","last_clean":"2024-09-16T09:25:41.451646+0000","last_became_active":"2024-09-16T09:25:05.711952+0000","last_became_peered":"2024-09-16T09:25:05.711952+0000","last_unstale":"2024-09-16T09:25:41.451646+0000","last_undegraded":"2024-09-16T09:25:41.451646+0000","last_fullsized":"2024-09-16T09:25:41.451646+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:30.229560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":393,"ondisk_log_size":393,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":35,"num_read_kb":30,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266701,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32624,"kb_used_data":1432,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712528,"statfs":{"total":95995035648,"available":95961628672,"internally_reserved":0,"allocated":1466368,"data_stored":969480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495634,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28436,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716716,"statfs":{"total":95995035648,"available":95965917184,"internally_reserved":0,"allocated":1441792,"data_stored":968382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724569,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27860,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717292,"statfs":{"total":95995035648,"available":95966507008,"internally_reserved":0,"allocated":851968,"data_stored":378373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953502,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":876544,"data_stored":379176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149732,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32056,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713096,"statfs":{"total":95995035648,"available":95962210304,"internally_reserved":0,"allocated":884736,"data_stored":379462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411369,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32008,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713144,"statfs":{"total":95995035648,"available":95962259456,"internally_reserved":0,"allocated":839680,"data_stored":377726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574896,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":913408,"data_stored":379376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738422,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28468,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716684,"statfs":{"total":95995035648,"available":95965884416,"internally_reserved":0,"allocated":1470464,"data_stored":969746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T09:25:45.585 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-16T09:25:45.848 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:25:46.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:46 smithi084 ceph-mon[32629]: Adjusting osd_memory_target on smithi070 to 3760M 2024-09-16T09:25:46.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:46 smithi084 ceph-mon[32629]: Updating smithi070:/etc/ceph/ceph.conf 2024-09-16T09:25:46.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:46 smithi084 ceph-mon[32629]: Updating smithi084:/etc/ceph/ceph.conf 2024-09-16T09:25:46.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:46 smithi084 ceph-mon[32629]: Updating smithi070:/var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T09:25:46.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:46 smithi084 ceph-mon[32629]: Updating smithi084:/var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T09:25:46.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:46 smithi084 ceph-mon[32629]: Updating smithi070:/etc/ceph/ceph.client.admin.keyring 2024-09-16T09:25:46.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:46 smithi084 ceph-mon[32629]: Updating smithi084:/etc/ceph/ceph.client.admin.keyring 2024-09-16T09:25:46.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:46 smithi084 ceph-mon[32629]: Updating smithi070:/var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T09:25:46.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:46 smithi084 ceph-mon[32629]: Updating smithi084:/var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T09:25:46.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:46 smithi084 ceph-mon[32629]: from='client.14727 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:25:46.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:46 smithi084 ceph-mon[32629]: Deploying daemon alertmanager.a on smithi070 2024-09-16T09:25:46.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:46 smithi070 ceph-mon[30388]: Adjusting osd_memory_target on smithi070 to 3760M 2024-09-16T09:25:46.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:46 smithi070 ceph-mon[30388]: Updating smithi070:/etc/ceph/ceph.conf 2024-09-16T09:25:46.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:46 smithi070 ceph-mon[30388]: Updating smithi084:/etc/ceph/ceph.conf 2024-09-16T09:25:46.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:46 smithi070 ceph-mon[30388]: Updating smithi070:/var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T09:25:46.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:46 smithi070 ceph-mon[30388]: Updating smithi084:/var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T09:25:46.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:46 smithi070 ceph-mon[30388]: Updating smithi070:/etc/ceph/ceph.client.admin.keyring 2024-09-16T09:25:46.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:46 smithi070 ceph-mon[30388]: Updating smithi084:/etc/ceph/ceph.client.admin.keyring 2024-09-16T09:25:46.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:46 smithi070 ceph-mon[30388]: Updating smithi070:/var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T09:25:46.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:46 smithi070 ceph-mon[30388]: Updating smithi084:/var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T09:25:46.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:46 smithi070 ceph-mon[30388]: from='client.14727 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:25:46.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:46 smithi070 ceph-mon[30388]: Deploying daemon alertmanager.a on smithi070 2024-09-16T09:25:46.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:46 smithi070 ceph-mon[40691]: Adjusting osd_memory_target on smithi070 to 3760M 2024-09-16T09:25:46.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:46 smithi070 ceph-mon[40691]: Updating smithi070:/etc/ceph/ceph.conf 2024-09-16T09:25:46.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:46 smithi070 ceph-mon[40691]: Updating smithi084:/etc/ceph/ceph.conf 2024-09-16T09:25:46.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:46 smithi070 ceph-mon[40691]: Updating smithi070:/var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T09:25:46.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:46 smithi070 ceph-mon[40691]: Updating smithi084:/var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T09:25:46.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:46 smithi070 ceph-mon[40691]: Updating smithi070:/etc/ceph/ceph.client.admin.keyring 2024-09-16T09:25:46.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:46 smithi070 ceph-mon[40691]: Updating smithi084:/etc/ceph/ceph.client.admin.keyring 2024-09-16T09:25:46.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:46 smithi070 ceph-mon[40691]: Updating smithi070:/var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T09:25:46.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:46 smithi070 ceph-mon[40691]: Updating smithi084:/var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T09:25:46.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:46 smithi070 ceph-mon[40691]: from='client.14727 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:25:46.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:46 smithi070 ceph-mon[40691]: Deploying daemon alertmanager.a on smithi070 2024-09-16T09:25:46.470 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:25:46.472 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2024-09-16T09:25:47.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:47 smithi084 ceph-mon[32629]: pgmap v15: 132 pgs: 132 active+clean; 583 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 800 B/s rd, 0 op/s 2024-09-16T09:25:47.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:47 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:25:47.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:47 smithi070 ceph-mon[30388]: pgmap v15: 132 pgs: 132 active+clean; 583 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 800 B/s rd, 0 op/s 2024-09-16T09:25:47.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:47 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:25:47.420 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:47 smithi070 ceph-mon[40691]: pgmap v15: 132 pgs: 132 active+clean; 583 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 800 B/s rd, 0 op/s 2024-09-16T09:25:47.420 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:47 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:25:47.680 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":15,"stamp":"2024-09-16T09:25:45.481448+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596507,"num_objects":199,"num_object_clones":0,"num_object_copies":597,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":199,"num_whiteouts":0,"num_read":883,"num_read_kb":740,"num_write":549,"num_write_kb":1673,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":552,"ondisk_log_size":552,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":241416,"kb_used_data":8540,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":749719800,"statfs":{"total":767960285184,"available":767713075200,"internally_reserved":0,"allocated":8744960,"data_stored":4801721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":238276146},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.399364"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":48,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.451430+0000","last_change":"2024-09-16T09:25:01.709392+0000","last_active":"2024-09-16T09:25:41.451430+0000","last_peered":"2024-09-16T09:25:41.451430+0000","last_clean":"2024-09-16T09:25:41.451430+0000","last_became_active":"2024-09-16T09:25:01.709108+0000","last_became_peered":"2024-09-16T09:25:01.709108+0000","last_unstale":"2024-09-16T09:25:41.451430+0000","last_undegraded":"2024-09-16T09:25:41.451430+0000","last_fullsized":"2024-09-16T09:25:41.451430+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:05.830525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"66'9","reported_seq":60,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.288293+0000","last_change":"2024-09-16T09:25:03.715139+0000","last_active":"2024-09-16T09:25:41.288293+0000","last_peered":"2024-09-16T09:25:41.288293+0000","last_clean":"2024-09-16T09:25:41.288293+0000","last_became_active":"2024-09-16T09:25:03.714635+0000","last_became_peered":"2024-09-16T09:25:03.714635+0000","last_unstale":"2024-09-16T09:25:41.288293+0000","last_undegraded":"2024-09-16T09:25:41.288293+0000","last_fullsized":"2024-09-16T09:25:41.288293+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:19.723608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":40,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.285710+0000","last_change":"2024-09-16T09:25:05.716139+0000","last_active":"2024-09-16T09:25:41.285710+0000","last_peered":"2024-09-16T09:25:41.285710+0000","last_clean":"2024-09-16T09:25:41.285710+0000","last_became_active":"2024-09-16T09:25:05.715645+0000","last_became_peered":"2024-09-16T09:25:05.715645+0000","last_unstale":"2024-09-16T09:25:41.285710+0000","last_undegraded":"2024-09-16T09:25:41.285710+0000","last_fullsized":"2024-09-16T09:25:41.285710+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:31.956608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.288428+0000","last_change":"2024-09-16T09:25:07.715923+0000","last_active":"2024-09-16T09:25:41.288428+0000","last_peered":"2024-09-16T09:25:41.288428+0000","last_clean":"2024-09-16T09:25:41.288428+0000","last_became_active":"2024-09-16T09:25:07.715442+0000","last_became_peered":"2024-09-16T09:25:07.715442+0000","last_unstale":"2024-09-16T09:25:41.288428+0000","last_undegraded":"2024-09-16T09:25:41.288428+0000","last_fullsized":"2024-09-16T09:25:41.288428+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:35.494958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.450629+0000","last_change":"2024-09-16T09:25:07.718687+0000","last_active":"2024-09-16T09:25:41.450629+0000","last_peered":"2024-09-16T09:25:41.450629+0000","last_clean":"2024-09-16T09:25:41.450629+0000","last_became_active":"2024-09-16T09:25:07.718126+0000","last_became_peered":"2024-09-16T09:25:07.718126+0000","last_unstale":"2024-09-16T09:25:41.450629+0000","last_undegraded":"2024-09-16T09:25:41.450629+0000","last_fullsized":"2024-09-16T09:25:41.450629+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:23.444563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":48,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.452007+0000","last_change":"2024-09-16T09:25:01.708505+0000","last_active":"2024-09-16T09:25:41.452007+0000","last_peered":"2024-09-16T09:25:41.452007+0000","last_clean":"2024-09-16T09:25:41.452007+0000","last_became_active":"2024-09-16T09:25:01.708179+0000","last_became_peered":"2024-09-16T09:25:01.708179+0000","last_unstale":"2024-09-16T09:25:41.452007+0000","last_undegraded":"2024-09-16T09:25:41.452007+0000","last_fullsized":"2024-09-16T09:25:41.452007+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:22.060114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"66'15","reported_seq":69,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.450943+0000","last_change":"2024-09-16T09:25:03.709970+0000","last_active":"2024-09-16T09:25:41.450943+0000","last_peered":"2024-09-16T09:25:41.450943+0000","last_clean":"2024-09-16T09:25:41.450943+0000","last_became_active":"2024-09-16T09:25:03.709499+0000","last_became_peered":"2024-09-16T09:25:03.709499+0000","last_unstale":"2024-09-16T09:25:41.450943+0000","last_undegraded":"2024-09-16T09:25:41.450943+0000","last_fullsized":"2024-09-16T09:25:41.450943+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:05.973210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":40,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.288218+0000","last_change":"2024-09-16T09:25:05.717524+0000","last_active":"2024-09-16T09:25:41.288218+0000","last_peered":"2024-09-16T09:25:41.288218+0000","last_clean":"2024-09-16T09:25:41.288218+0000","last_became_active":"2024-09-16T09:25:05.717236+0000","last_became_peered":"2024-09-16T09:25:05.717236+0000","last_unstale":"2024-09-16T09:25:41.288218+0000","last_undegraded":"2024-09-16T09:25:41.288218+0000","last_fullsized":"2024-09-16T09:25:41.288218+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:14.774263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":47,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.287761+0000","last_change":"2024-09-16T09:25:01.705552+0000","last_active":"2024-09-16T09:25:41.287761+0000","last_peered":"2024-09-16T09:25:41.287761+0000","last_clean":"2024-09-16T09:25:41.287761+0000","last_became_active":"2024-09-16T09:25:01.705269+0000","last_became_peered":"2024-09-16T09:25:01.705269+0000","last_unstale":"2024-09-16T09:25:41.287761+0000","last_undegraded":"2024-09-16T09:25:41.287761+0000","last_fullsized":"2024-09-16T09:25:41.287761+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:18.445329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"66'9","reported_seq":60,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.288675+0000","last_change":"2024-09-16T09:25:03.709441+0000","last_active":"2024-09-16T09:25:41.288675+0000","last_peered":"2024-09-16T09:25:41.288675+0000","last_clean":"2024-09-16T09:25:41.288675+0000","last_became_active":"2024-09-16T09:25:03.709147+0000","last_became_peered":"2024-09-16T09:25:03.709147+0000","last_unstale":"2024-09-16T09:25:41.288675+0000","last_undegraded":"2024-09-16T09:25:41.288675+0000","last_fullsized":"2024-09-16T09:25:41.288675+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:21.309548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":39,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.287291+0000","last_change":"2024-09-16T09:25:05.718510+0000","last_active":"2024-09-16T09:25:41.287291+0000","last_peered":"2024-09-16T09:25:41.287291+0000","last_clean":"2024-09-16T09:25:41.287291+0000","last_became_active":"2024-09-16T09:25:05.718134+0000","last_became_peered":"2024-09-16T09:25:05.718134+0000","last_unstale":"2024-09-16T09:25:41.287291+0000","last_undegraded":"2024-09-16T09:25:41.287291+0000","last_fullsized":"2024-09-16T09:25:41.287291+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:30.593901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"66'1","reported_seq":37,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.450466+0000","last_change":"2024-09-16T09:25:07.716869+0000","last_active":"2024-09-16T09:25:41.450466+0000","last_peered":"2024-09-16T09:25:41.450466+0000","last_clean":"2024-09-16T09:25:41.450466+0000","last_became_active":"2024-09-16T09:25:07.716558+0000","last_became_peered":"2024-09-16T09:25:07.716558+0000","last_unstale":"2024-09-16T09:25:41.450466+0000","last_undegraded":"2024-09-16T09:25:41.450466+0000","last_fullsized":"2024-09-16T09:25:41.450466+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:17.035822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":47,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.287563+0000","last_change":"2024-09-16T09:25:01.697306+0000","last_active":"2024-09-16T09:25:41.287563+0000","last_peered":"2024-09-16T09:25:41.287563+0000","last_clean":"2024-09-16T09:25:41.287563+0000","last_became_active":"2024-09-16T09:25:01.696939+0000","last_became_peered":"2024-09-16T09:25:01.696939+0000","last_unstale":"2024-09-16T09:25:41.287563+0000","last_undegraded":"2024-09-16T09:25:41.287563+0000","last_fullsized":"2024-09-16T09:25:41.287563+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:26.024926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"66'5","reported_seq":54,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.288575+0000","last_change":"2024-09-16T09:25:03.716402+0000","last_active":"2024-09-16T09:25:41.288575+0000","last_peered":"2024-09-16T09:25:41.288575+0000","last_clean":"2024-09-16T09:25:41.288575+0000","last_became_active":"2024-09-16T09:25:03.716136+0000","last_became_peered":"2024-09-16T09:25:03.716136+0000","last_unstale":"2024-09-16T09:25:41.288575+0000","last_undegraded":"2024-09-16T09:25:41.288575+0000","last_fullsized":"2024-09-16T09:25:41.288575+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:32.360413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":39,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.450089+0000","last_change":"2024-09-16T09:25:05.718538+0000","last_active":"2024-09-16T09:25:41.450089+0000","last_peered":"2024-09-16T09:25:41.450089+0000","last_clean":"2024-09-16T09:25:41.450089+0000","last_became_active":"2024-09-16T09:25:05.718281+0000","last_became_peered":"2024-09-16T09:25:05.718281+0000","last_unstale":"2024-09-16T09:25:41.450089+0000","last_undegraded":"2024-09-16T09:25:41.450089+0000","last_fullsized":"2024-09-16T09:25:41.450089+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:29.527824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":35,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.287032+0000","last_change":"2024-09-16T09:25:07.717644+0000","last_active":"2024-09-16T09:25:41.287032+0000","last_peered":"2024-09-16T09:25:41.287032+0000","last_clean":"2024-09-16T09:25:41.287032+0000","last_became_active":"2024-09-16T09:25:07.716724+0000","last_became_peered":"2024-09-16T09:25:07.716724+0000","last_unstale":"2024-09-16T09:25:41.287032+0000","last_undegraded":"2024-09-16T09:25:41.287032+0000","last_fullsized":"2024-09-16T09:25:41.287032+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:32.145892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.289092+0000","last_change":"2024-09-16T09:25:07.721088+0000","last_active":"2024-09-16T09:25:41.289092+0000","last_peered":"2024-09-16T09:25:41.289092+0000","last_clean":"2024-09-16T09:25:41.289092+0000","last_became_active":"2024-09-16T09:25:07.720793+0000","last_became_peered":"2024-09-16T09:25:07.720793+0000","last_unstale":"2024-09-16T09:25:41.289092+0000","last_undegraded":"2024-09-16T09:25:41.289092+0000","last_fullsized":"2024-09-16T09:25:41.289092+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:16.293244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":48,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.450819+0000","last_change":"2024-09-16T09:25:01.706762+0000","last_active":"2024-09-16T09:25:41.450819+0000","last_peered":"2024-09-16T09:25:41.450819+0000","last_clean":"2024-09-16T09:25:41.450819+0000","last_became_active":"2024-09-16T09:25:01.706172+0000","last_became_peered":"2024-09-16T09:25:01.706172+0000","last_unstale":"2024-09-16T09:25:41.450819+0000","last_undegraded":"2024-09-16T09:25:41.450819+0000","last_fullsized":"2024-09-16T09:25:41.450819+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:07.135883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"66'15","reported_seq":69,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.451311+0000","last_change":"2024-09-16T09:25:03.714261+0000","last_active":"2024-09-16T09:25:41.451311+0000","last_peered":"2024-09-16T09:25:41.451311+0000","last_clean":"2024-09-16T09:25:41.451311+0000","last_became_active":"2024-09-16T09:25:03.713780+0000","last_became_peered":"2024-09-16T09:25:03.713780+0000","last_unstale":"2024-09-16T09:25:41.451311+0000","last_undegraded":"2024-09-16T09:25:41.451311+0000","last_fullsized":"2024-09-16T09:25:41.451311+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:29.826960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":40,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.286690+0000","last_change":"2024-09-16T09:25:05.714209+0000","last_active":"2024-09-16T09:25:41.286690+0000","last_peered":"2024-09-16T09:25:41.286690+0000","last_clean":"2024-09-16T09:25:41.286690+0000","last_became_active":"2024-09-16T09:25:05.713929+0000","last_became_peered":"2024-09-16T09:25:05.713929+0000","last_unstale":"2024-09-16T09:25:41.286690+0000","last_undegraded":"2024-09-16T09:25:41.286690+0000","last_fullsized":"2024-09-16T09:25:41.286690+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:05.941022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.451826+0000","last_change":"2024-09-16T09:25:07.719645+0000","last_active":"2024-09-16T09:25:41.451826+0000","last_peered":"2024-09-16T09:25:41.451826+0000","last_clean":"2024-09-16T09:25:41.451826+0000","last_became_active":"2024-09-16T09:25:07.719067+0000","last_became_peered":"2024-09-16T09:25:07.719067+0000","last_unstale":"2024-09-16T09:25:41.451826+0000","last_undegraded":"2024-09-16T09:25:41.451826+0000","last_fullsized":"2024-09-16T09:25:41.451826+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:19.447150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":48,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.288858+0000","last_change":"2024-09-16T09:25:01.706828+0000","last_active":"2024-09-16T09:25:41.288858+0000","last_peered":"2024-09-16T09:25:41.288858+0000","last_clean":"2024-09-16T09:25:41.288858+0000","last_became_active":"2024-09-16T09:25:01.706491+0000","last_became_peered":"2024-09-16T09:25:01.706491+0000","last_unstale":"2024-09-16T09:25:41.288858+0000","last_undegraded":"2024-09-16T09:25:41.288858+0000","last_fullsized":"2024-09-16T09:25:41.288858+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:16.321414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"66'12","reported_seq":67,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.451236+0000","last_change":"2024-09-16T09:25:03.709914+0000","last_active":"2024-09-16T09:25:41.451236+0000","last_peered":"2024-09-16T09:25:41.451236+0000","last_clean":"2024-09-16T09:25:41.451236+0000","last_became_active":"2024-09-16T09:25:03.709610+0000","last_became_peered":"2024-09-16T09:25:03.709610+0000","last_unstale":"2024-09-16T09:25:41.451236+0000","last_undegraded":"2024-09-16T09:25:41.451236+0000","last_fullsized":"2024-09-16T09:25:41.451236+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:26.267096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":40,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.288654+0000","last_change":"2024-09-16T09:25:05.710379+0000","last_active":"2024-09-16T09:25:41.288654+0000","last_peered":"2024-09-16T09:25:41.288654+0000","last_clean":"2024-09-16T09:25:41.288654+0000","last_became_active":"2024-09-16T09:25:05.710093+0000","last_became_peered":"2024-09-16T09:25:05.710093+0000","last_unstale":"2024-09-16T09:25:41.288654+0000","last_undegraded":"2024-09-16T09:25:41.288654+0000","last_fullsized":"2024-09-16T09:25:41.288654+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:28.802108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.450171+0000","last_change":"2024-09-16T09:25:07.719655+0000","last_active":"2024-09-16T09:25:41.450171+0000","last_peered":"2024-09-16T09:25:41.450171+0000","last_clean":"2024-09-16T09:25:41.450171+0000","last_became_active":"2024-09-16T09:25:07.719380+0000","last_became_peered":"2024-09-16T09:25:07.719380+0000","last_unstale":"2024-09-16T09:25:41.450171+0000","last_undegraded":"2024-09-16T09:25:41.450171+0000","last_fullsized":"2024-09-16T09:25:41.450171+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:36.059179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":49,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.285864+0000","last_change":"2024-09-16T09:25:01.700263+0000","last_active":"2024-09-16T09:25:41.285864+0000","last_peered":"2024-09-16T09:25:41.285864+0000","last_clean":"2024-09-16T09:25:41.285864+0000","last_became_active":"2024-09-16T09:25:01.699545+0000","last_became_peered":"2024-09-16T09:25:01.699545+0000","last_unstale":"2024-09-16T09:25:41.285864+0000","last_undegraded":"2024-09-16T09:25:41.285864+0000","last_fullsized":"2024-09-16T09:25:41.285864+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:04.529119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"66'10","reported_seq":59,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.451722+0000","last_change":"2024-09-16T09:25:03.720638+0000","last_active":"2024-09-16T09:25:41.451722+0000","last_peered":"2024-09-16T09:25:41.451722+0000","last_clean":"2024-09-16T09:25:41.451722+0000","last_became_active":"2024-09-16T09:25:03.720245+0000","last_became_peered":"2024-09-16T09:25:03.720245+0000","last_unstale":"2024-09-16T09:25:41.451722+0000","last_undegraded":"2024-09-16T09:25:41.451722+0000","last_fullsized":"2024-09-16T09:25:41.451722+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:21.451538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"66'8","reported_seq":60,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.451113+0000","last_change":"2024-09-16T09:25:05.713310+0000","last_active":"2024-09-16T09:25:41.451113+0000","last_peered":"2024-09-16T09:25:41.451113+0000","last_clean":"2024-09-16T09:25:41.451113+0000","last_became_active":"2024-09-16T09:25:05.712516+0000","last_became_peered":"2024-09-16T09:25:05.712516+0000","last_unstale":"2024-09-16T09:25:41.451113+0000","last_undegraded":"2024-09-16T09:25:41.451113+0000","last_fullsized":"2024-09-16T09:25:41.451113+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:24.505283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.f","version":"66'15","reported_seq":69,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.286028+0000","last_change":"2024-09-16T09:25:03.719134+0000","last_active":"2024-09-16T09:25:41.286028+0000","last_peered":"2024-09-16T09:25:41.286028+0000","last_clean":"2024-09-16T09:25:41.286028+0000","last_became_active":"2024-09-16T09:25:03.718849+0000","last_became_peered":"2024-09-16T09:25:03.718849+0000","last_unstale":"2024-09-16T09:25:41.286028+0000","last_undegraded":"2024-09-16T09:25:41.286028+0000","last_fullsized":"2024-09-16T09:25:41.286028+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:13.173718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":48,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.450185+0000","last_change":"2024-09-16T09:25:01.702936+0000","last_active":"2024-09-16T09:25:41.450185+0000","last_peered":"2024-09-16T09:25:41.450185+0000","last_clean":"2024-09-16T09:25:41.450185+0000","last_became_active":"2024-09-16T09:25:01.702629+0000","last_became_peered":"2024-09-16T09:25:01.702629+0000","last_unstale":"2024-09-16T09:25:41.450185+0000","last_undegraded":"2024-09-16T09:25:41.450185+0000","last_fullsized":"2024-09-16T09:25:41.450185+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:20.061024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"66'8","reported_seq":57,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.289293+0000","last_change":"2024-09-16T09:25:05.714128+0000","last_active":"2024-09-16T09:25:41.289293+0000","last_peered":"2024-09-16T09:25:41.289293+0000","last_clean":"2024-09-16T09:25:41.289293+0000","last_became_active":"2024-09-16T09:25:05.713813+0000","last_became_peered":"2024-09-16T09:25:05.713813+0000","last_unstale":"2024-09-16T09:25:41.289293+0000","last_undegraded":"2024-09-16T09:25:41.289293+0000","last_fullsized":"2024-09-16T09:25:41.289293+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:26.013786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":35,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.287434+0000","last_change":"2024-09-16T09:25:07.717639+0000","last_active":"2024-09-16T09:25:41.287434+0000","last_peered":"2024-09-16T09:25:41.287434+0000","last_clean":"2024-09-16T09:25:41.287434+0000","last_became_active":"2024-09-16T09:25:07.717344+0000","last_became_peered":"2024-09-16T09:25:07.717344+0000","last_unstale":"2024-09-16T09:25:41.287434+0000","last_undegraded":"2024-09-16T09:25:41.287434+0000","last_fullsized":"2024-09-16T09:25:41.287434+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:24.054256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.0","version":"66'18","reported_seq":76,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.450068+0000","last_change":"2024-09-16T09:25:03.714508+0000","last_active":"2024-09-16T09:25:41.450068+0000","last_peered":"2024-09-16T09:25:41.450068+0000","last_clean":"2024-09-16T09:25:41.450068+0000","last_became_active":"2024-09-16T09:25:03.714208+0000","last_became_peered":"2024-09-16T09:25:03.714208+0000","last_unstale":"2024-09-16T09:25:41.450068+0000","last_undegraded":"2024-09-16T09:25:41.450068+0000","last_fullsized":"2024-09-16T09:25:41.450068+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:03.200598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":48,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.451912+0000","last_change":"2024-09-16T09:25:01.705362+0000","last_active":"2024-09-16T09:25:41.451912+0000","last_peered":"2024-09-16T09:25:41.451912+0000","last_clean":"2024-09-16T09:25:41.451912+0000","last_became_active":"2024-09-16T09:25:01.704994+0000","last_became_peered":"2024-09-16T09:25:01.704994+0000","last_unstale":"2024-09-16T09:25:41.451912+0000","last_undegraded":"2024-09-16T09:25:41.451912+0000","last_fullsized":"2024-09-16T09:25:41.451912+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:17.466780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":40,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.288317+0000","last_change":"2024-09-16T09:25:05.713687+0000","last_active":"2024-09-16T09:25:41.288317+0000","last_peered":"2024-09-16T09:25:41.288317+0000","last_clean":"2024-09-16T09:25:41.288317+0000","last_became_active":"2024-09-16T09:25:05.713360+0000","last_became_peered":"2024-09-16T09:25:05.713360+0000","last_unstale":"2024-09-16T09:25:41.288317+0000","last_undegraded":"2024-09-16T09:25:41.288317+0000","last_fullsized":"2024-09-16T09:25:41.288317+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:16.400241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.288739+0000","last_change":"2024-09-16T09:25:07.719412+0000","last_active":"2024-09-16T09:25:41.288739+0000","last_peered":"2024-09-16T09:25:41.288739+0000","last_clean":"2024-09-16T09:25:41.288739+0000","last_became_active":"2024-09-16T09:25:07.719159+0000","last_became_peered":"2024-09-16T09:25:07.719159+0000","last_unstale":"2024-09-16T09:25:41.288739+0000","last_undegraded":"2024-09-16T09:25:41.288739+0000","last_fullsized":"2024-09-16T09:25:41.288739+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:36.545361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"66'14","reported_seq":65,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.288323+0000","last_change":"2024-09-16T09:25:03.710438+0000","last_active":"2024-09-16T09:25:41.288323+0000","last_peered":"2024-09-16T09:25:41.288323+0000","last_clean":"2024-09-16T09:25:41.288323+0000","last_became_active":"2024-09-16T09:25:03.708830+0000","last_became_peered":"2024-09-16T09:25:03.708830+0000","last_unstale":"2024-09-16T09:25:41.288323+0000","last_undegraded":"2024-09-16T09:25:41.288323+0000","last_fullsized":"2024-09-16T09:25:41.288323+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:17.190767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":49,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.451907+0000","last_change":"2024-09-16T09:25:01.703245+0000","last_active":"2024-09-16T09:25:41.451907+0000","last_peered":"2024-09-16T09:25:41.451907+0000","last_clean":"2024-09-16T09:25:41.451907+0000","last_became_active":"2024-09-16T09:25:01.702992+0000","last_became_peered":"2024-09-16T09:25:01.702992+0000","last_unstale":"2024-09-16T09:25:41.451907+0000","last_undegraded":"2024-09-16T09:25:41.451907+0000","last_fullsized":"2024-09-16T09:25:41.451907+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:00.901034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"66'8","reported_seq":57,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.449989+0000","last_change":"2024-09-16T09:25:05.715927+0000","last_active":"2024-09-16T09:25:41.449989+0000","last_peered":"2024-09-16T09:25:41.449989+0000","last_clean":"2024-09-16T09:25:41.449989+0000","last_became_active":"2024-09-16T09:25:05.715606+0000","last_became_peered":"2024-09-16T09:25:05.715606+0000","last_unstale":"2024-09-16T09:25:41.449989+0000","last_undegraded":"2024-09-16T09:25:41.449989+0000","last_fullsized":"2024-09-16T09:25:41.449989+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:23.752404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":35,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.450175+0000","last_change":"2024-09-16T09:25:07.722287+0000","last_active":"2024-09-16T09:25:41.450175+0000","last_peered":"2024-09-16T09:25:41.450175+0000","last_clean":"2024-09-16T09:25:41.450175+0000","last_became_active":"2024-09-16T09:25:07.722017+0000","last_became_peered":"2024-09-16T09:25:07.722017+0000","last_unstale":"2024-09-16T09:25:41.450175+0000","last_undegraded":"2024-09-16T09:25:41.450175+0000","last_fullsized":"2024-09-16T09:25:41.450175+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:27.828108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2","version":"66'10","reported_seq":59,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.286027+0000","last_change":"2024-09-16T09:25:03.713368+0000","last_active":"2024-09-16T09:25:41.286027+0000","last_peered":"2024-09-16T09:25:41.286027+0000","last_clean":"2024-09-16T09:25:41.286027+0000","last_became_active":"2024-09-16T09:25:03.712985+0000","last_became_peered":"2024-09-16T09:25:03.712985+0000","last_unstale":"2024-09-16T09:25:41.286027+0000","last_undegraded":"2024-09-16T09:25:41.286027+0000","last_fullsized":"2024-09-16T09:25:41.286027+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:21.169460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":47,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.287870+0000","last_change":"2024-09-16T09:25:01.706774+0000","last_active":"2024-09-16T09:25:41.287870+0000","last_peered":"2024-09-16T09:25:41.287870+0000","last_clean":"2024-09-16T09:25:41.287870+0000","last_became_active":"2024-09-16T09:25:01.706440+0000","last_became_peered":"2024-09-16T09:25:01.706440+0000","last_unstale":"2024-09-16T09:25:41.287870+0000","last_undegraded":"2024-09-16T09:25:41.287870+0000","last_fullsized":"2024-09-16T09:25:41.287870+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:18.805280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"66'8","reported_seq":57,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.450426+0000","last_change":"2024-09-16T09:25:05.714825+0000","last_active":"2024-09-16T09:25:41.450426+0000","last_peered":"2024-09-16T09:25:41.450426+0000","last_clean":"2024-09-16T09:25:41.450426+0000","last_became_active":"2024-09-16T09:25:05.714507+0000","last_became_peered":"2024-09-16T09:25:05.714507+0000","last_unstale":"2024-09-16T09:25:41.450426+0000","last_undegraded":"2024-09-16T09:25:41.450426+0000","last_fullsized":"2024-09-16T09:25:41.450426+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:13.181647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.450619+0000","last_change":"2024-09-16T09:25:07.719876+0000","last_active":"2024-09-16T09:25:41.450619+0000","last_peered":"2024-09-16T09:25:41.450619+0000","last_clean":"2024-09-16T09:25:41.450619+0000","last_became_active":"2024-09-16T09:25:07.719604+0000","last_became_peered":"2024-09-16T09:25:07.719604+0000","last_unstale":"2024-09-16T09:25:41.450619+0000","last_undegraded":"2024-09-16T09:25:41.450619+0000","last_fullsized":"2024-09-16T09:25:41.450619+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:20.929075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"66'19","reported_seq":80,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.450624+0000","last_change":"2024-09-16T09:25:03.715086+0000","last_active":"2024-09-16T09:25:41.450624+0000","last_peered":"2024-09-16T09:25:41.450624+0000","last_clean":"2024-09-16T09:25:41.450624+0000","last_became_active":"2024-09-16T09:25:03.714797+0000","last_became_peered":"2024-09-16T09:25:03.714797+0000","last_unstale":"2024-09-16T09:25:41.450624+0000","last_undegraded":"2024-09-16T09:25:41.450624+0000","last_fullsized":"2024-09-16T09:25:41.450624+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:23.683038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":48,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.286819+0000","last_change":"2024-09-16T09:25:01.707093+0000","last_active":"2024-09-16T09:25:41.286819+0000","last_peered":"2024-09-16T09:25:41.286819+0000","last_clean":"2024-09-16T09:25:41.286819+0000","last_became_active":"2024-09-16T09:25:01.706806+0000","last_became_peered":"2024-09-16T09:25:01.706806+0000","last_unstale":"2024-09-16T09:25:41.286819+0000","last_undegraded":"2024-09-16T09:25:41.286819+0000","last_fullsized":"2024-09-16T09:25:41.286819+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:01.655926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":40,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.450009+0000","last_change":"2024-09-16T09:25:05.708949+0000","last_active":"2024-09-16T09:25:41.450009+0000","last_peered":"2024-09-16T09:25:41.450009+0000","last_clean":"2024-09-16T09:25:41.450009+0000","last_became_active":"2024-09-16T09:25:05.708663+0000","last_became_peered":"2024-09-16T09:25:05.708663+0000","last_unstale":"2024-09-16T09:25:41.450009+0000","last_undegraded":"2024-09-16T09:25:41.450009+0000","last_fullsized":"2024-09-16T09:25:41.450009+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:06.107203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.285607+0000","last_change":"2024-09-16T09:25:07.719568+0000","last_active":"2024-09-16T09:25:41.285607+0000","last_peered":"2024-09-16T09:25:41.285607+0000","last_clean":"2024-09-16T09:25:41.285607+0000","last_became_active":"2024-09-16T09:25:07.719267+0000","last_became_peered":"2024-09-16T09:25:07.719267+0000","last_unstale":"2024-09-16T09:25:41.285607+0000","last_undegraded":"2024-09-16T09:25:41.285607+0000","last_fullsized":"2024-09-16T09:25:41.285607+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:31.531856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"66'28","reported_seq":101,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.286729+0000","last_change":"2024-09-16T09:25:03.705300+0000","last_active":"2024-09-16T09:25:41.286729+0000","last_peered":"2024-09-16T09:25:41.286729+0000","last_clean":"2024-09-16T09:25:41.286729+0000","last_became_active":"2024-09-16T09:25:03.704878+0000","last_became_peered":"2024-09-16T09:25:03.704878+0000","last_unstale":"2024-09-16T09:25:41.286729+0000","last_undegraded":"2024-09-16T09:25:41.286729+0000","last_fullsized":"2024-09-16T09:25:41.286729+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:32.612703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":48,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.288933+0000","last_change":"2024-09-16T09:25:01.703394+0000","last_active":"2024-09-16T09:25:41.288933+0000","last_peered":"2024-09-16T09:25:41.288933+0000","last_clean":"2024-09-16T09:25:41.288933+0000","last_became_active":"2024-09-16T09:25:01.703076+0000","last_became_peered":"2024-09-16T09:25:01.703076+0000","last_unstale":"2024-09-16T09:25:41.288933+0000","last_undegraded":"2024-09-16T09:25:41.288933+0000","last_fullsized":"2024-09-16T09:25:41.288933+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:21.922041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":57,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.287228+0000","last_change":"2024-09-16T09:25:03.706980+0000","last_active":"2024-09-16T09:25:41.287228+0000","last_peered":"2024-09-16T09:25:41.287228+0000","last_clean":"2024-09-16T09:25:41.287228+0000","last_became_active":"2024-09-16T09:25:00.696237+0000","last_became_peered":"2024-09-16T09:25:00.696237+0000","last_unstale":"2024-09-16T09:25:41.287228+0000","last_undegraded":"2024-09-16T09:25:41.287228+0000","last_fullsized":"2024-09-16T09:25:41.287228+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:24:59.684950+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:24.893358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025137430000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":40,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.451030+0000","last_change":"2024-09-16T09:25:05.718319+0000","last_active":"2024-09-16T09:25:41.451030+0000","last_peered":"2024-09-16T09:25:41.451030+0000","last_clean":"2024-09-16T09:25:41.451030+0000","last_became_active":"2024-09-16T09:25:05.717591+0000","last_became_peered":"2024-09-16T09:25:05.717591+0000","last_unstale":"2024-09-16T09:25:41.451030+0000","last_undegraded":"2024-09-16T09:25:41.451030+0000","last_fullsized":"2024-09-16T09:25:41.451030+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:32.732085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.451542+0000","last_change":"2024-09-16T09:25:07.717720+0000","last_active":"2024-09-16T09:25:41.451542+0000","last_peered":"2024-09-16T09:25:41.451542+0000","last_clean":"2024-09-16T09:25:41.451542+0000","last_became_active":"2024-09-16T09:25:07.717436+0000","last_became_peered":"2024-09-16T09:25:07.717436+0000","last_unstale":"2024-09-16T09:25:41.451542+0000","last_undegraded":"2024-09-16T09:25:41.451542+0000","last_fullsized":"2024-09-16T09:25:41.451542+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:16.230936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"66'13","reported_seq":71,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.286120+0000","last_change":"2024-09-16T09:25:03.713631+0000","last_active":"2024-09-16T09:25:41.286120+0000","last_peered":"2024-09-16T09:25:41.286120+0000","last_clean":"2024-09-16T09:25:41.286120+0000","last_became_active":"2024-09-16T09:25:03.713241+0000","last_became_peered":"2024-09-16T09:25:03.713241+0000","last_unstale":"2024-09-16T09:25:41.286120+0000","last_undegraded":"2024-09-16T09:25:41.286120+0000","last_fullsized":"2024-09-16T09:25:41.286120+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:19.940601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":48,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.285789+0000","last_change":"2024-09-16T09:25:01.707526+0000","last_active":"2024-09-16T09:25:41.285789+0000","last_peered":"2024-09-16T09:25:41.285789+0000","last_clean":"2024-09-16T09:25:41.285789+0000","last_became_active":"2024-09-16T09:25:01.706828+0000","last_became_peered":"2024-09-16T09:25:01.706828+0000","last_unstale":"2024-09-16T09:25:41.285789+0000","last_undegraded":"2024-09-16T09:25:41.285789+0000","last_fullsized":"2024-09-16T09:25:41.285789+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:17.040873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":57,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.450361+0000","last_change":"2024-09-16T09:25:03.702772+0000","last_active":"2024-09-16T09:25:41.450361+0000","last_peered":"2024-09-16T09:25:41.450361+0000","last_clean":"2024-09-16T09:25:41.450361+0000","last_became_active":"2024-09-16T09:25:00.697822+0000","last_became_peered":"2024-09-16T09:25:00.697822+0000","last_unstale":"2024-09-16T09:25:41.450361+0000","last_undegraded":"2024-09-16T09:25:41.450361+0000","last_fullsized":"2024-09-16T09:25:41.450361+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:24:59.684950+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:05.731438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055022299999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":40,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.450983+0000","last_change":"2024-09-16T09:25:05.711840+0000","last_active":"2024-09-16T09:25:41.450983+0000","last_peered":"2024-09-16T09:25:41.450983+0000","last_clean":"2024-09-16T09:25:41.450983+0000","last_became_active":"2024-09-16T09:25:05.711539+0000","last_became_peered":"2024-09-16T09:25:05.711539+0000","last_unstale":"2024-09-16T09:25:41.450983+0000","last_undegraded":"2024-09-16T09:25:41.450983+0000","last_fullsized":"2024-09-16T09:25:41.450983+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:28.306667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":35,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.450427+0000","last_change":"2024-09-16T09:25:07.723554+0000","last_active":"2024-09-16T09:25:41.450427+0000","last_peered":"2024-09-16T09:25:41.450427+0000","last_clean":"2024-09-16T09:25:41.450427+0000","last_became_active":"2024-09-16T09:25:07.723282+0000","last_became_peered":"2024-09-16T09:25:07.723282+0000","last_unstale":"2024-09-16T09:25:41.450427+0000","last_undegraded":"2024-09-16T09:25:41.450427+0000","last_fullsized":"2024-09-16T09:25:41.450427+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:28.111790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"66'12","reported_seq":62,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.451823+0000","last_change":"2024-09-16T09:25:03.715612+0000","last_active":"2024-09-16T09:25:41.451823+0000","last_peered":"2024-09-16T09:25:41.451823+0000","last_clean":"2024-09-16T09:25:41.451823+0000","last_became_active":"2024-09-16T09:25:03.715315+0000","last_became_peered":"2024-09-16T09:25:03.715315+0000","last_unstale":"2024-09-16T09:25:41.451823+0000","last_undegraded":"2024-09-16T09:25:41.451823+0000","last_fullsized":"2024-09-16T09:25:41.451823+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:31.110979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"59'1","reported_seq":54,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.450107+0000","last_change":"2024-09-16T09:25:01.701885+0000","last_active":"2024-09-16T09:25:41.450107+0000","last_peered":"2024-09-16T09:25:41.450107+0000","last_clean":"2024-09-16T09:25:41.450107+0000","last_became_active":"2024-09-16T09:25:01.701599+0000","last_became_peered":"2024-09-16T09:25:01.701599+0000","last_unstale":"2024-09-16T09:25:41.450107+0000","last_undegraded":"2024-09-16T09:25:41.450107+0000","last_fullsized":"2024-09-16T09:25:41.450107+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:03.521969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"67'5","reported_seq":93,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.449893+0000","last_change":"2024-09-16T09:25:03.711738+0000","last_active":"2024-09-16T09:25:41.449893+0000","last_peered":"2024-09-16T09:25:41.449893+0000","last_clean":"2024-09-16T09:25:41.449893+0000","last_became_active":"2024-09-16T09:25:00.696546+0000","last_became_peered":"2024-09-16T09:25:00.696546+0000","last_unstale":"2024-09-16T09:25:41.449893+0000","last_undegraded":"2024-09-16T09:25:41.449893+0000","last_fullsized":"2024-09-16T09:25:41.449893+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:24:59.684950+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:17.241966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0059770420000000001,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":35,"num_read_kb":30,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":39,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.288042+0000","last_change":"2024-09-16T09:25:05.714994+0000","last_active":"2024-09-16T09:25:41.288042+0000","last_peered":"2024-09-16T09:25:41.288042+0000","last_clean":"2024-09-16T09:25:41.288042+0000","last_became_active":"2024-09-16T09:25:05.714704+0000","last_became_peered":"2024-09-16T09:25:05.714704+0000","last_unstale":"2024-09-16T09:25:41.288042+0000","last_undegraded":"2024-09-16T09:25:41.288042+0000","last_fullsized":"2024-09-16T09:25:41.288042+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:32.353177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.286415+0000","last_change":"2024-09-16T09:25:07.714719+0000","last_active":"2024-09-16T09:25:41.286415+0000","last_peered":"2024-09-16T09:25:41.286415+0000","last_clean":"2024-09-16T09:25:41.286415+0000","last_became_active":"2024-09-16T09:25:07.714158+0000","last_became_peered":"2024-09-16T09:25:07.714158+0000","last_unstale":"2024-09-16T09:25:41.286415+0000","last_undegraded":"2024-09-16T09:25:41.286415+0000","last_fullsized":"2024-09-16T09:25:41.286415+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:13.991008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"66'16","reported_seq":75,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.450962+0000","last_change":"2024-09-16T09:25:03.714405+0000","last_active":"2024-09-16T09:25:41.450962+0000","last_peered":"2024-09-16T09:25:41.450962+0000","last_clean":"2024-09-16T09:25:41.450962+0000","last_became_active":"2024-09-16T09:25:03.714118+0000","last_became_peered":"2024-09-16T09:25:03.714118+0000","last_unstale":"2024-09-16T09:25:41.450962+0000","last_undegraded":"2024-09-16T09:25:41.450962+0000","last_fullsized":"2024-09-16T09:25:41.450962+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:12.074949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":48,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.450535+0000","last_change":"2024-09-16T09:25:01.704246+0000","last_active":"2024-09-16T09:25:41.450535+0000","last_peered":"2024-09-16T09:25:41.450535+0000","last_clean":"2024-09-16T09:25:41.450535+0000","last_became_active":"2024-09-16T09:25:01.703730+0000","last_became_peered":"2024-09-16T09:25:01.703730+0000","last_unstale":"2024-09-16T09:25:41.450535+0000","last_undegraded":"2024-09-16T09:25:41.450535+0000","last_fullsized":"2024-09-16T09:25:41.450535+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:20.523303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"73'79","reported_seq":85,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.450216+0000","last_change":"2024-09-16T09:24:53.142015+0000","last_active":"2024-09-16T09:25:41.450216+0000","last_peered":"2024-09-16T09:25:41.450216+0000","last_clean":"2024-09-16T09:25:41.450216+0000","last_became_active":"2024-09-16T09:24:53.135890+0000","last_became_peered":"2024-09-16T09:24:53.135890+0000","last_unstale":"2024-09-16T09:25:41.450216+0000","last_undegraded":"2024-09-16T09:25:41.450216+0000","last_fullsized":"2024-09-16T09:25:41.450216+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"20'72","last_scrub_stamp":"2024-09-16T09:24:45.089989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:24:45.089989+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:10.885850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":39,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.450390+0000","last_change":"2024-09-16T09:25:05.714741+0000","last_active":"2024-09-16T09:25:41.450390+0000","last_peered":"2024-09-16T09:25:41.450390+0000","last_clean":"2024-09-16T09:25:41.450390+0000","last_became_active":"2024-09-16T09:25:05.714341+0000","last_became_peered":"2024-09-16T09:25:05.714341+0000","last_unstale":"2024-09-16T09:25:41.450390+0000","last_undegraded":"2024-09-16T09:25:41.450390+0000","last_fullsized":"2024-09-16T09:25:41.450390+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:21.685216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":35,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.287929+0000","last_change":"2024-09-16T09:25:07.714544+0000","last_active":"2024-09-16T09:25:41.287929+0000","last_peered":"2024-09-16T09:25:41.287929+0000","last_clean":"2024-09-16T09:25:41.287929+0000","last_became_active":"2024-09-16T09:25:07.714281+0000","last_became_peered":"2024-09-16T09:25:07.714281+0000","last_unstale":"2024-09-16T09:25:41.287929+0000","last_undegraded":"2024-09-16T09:25:41.287929+0000","last_fullsized":"2024-09-16T09:25:41.287929+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:13.491593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"66'11","reported_seq":63,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.289255+0000","last_change":"2024-09-16T09:25:03.715654+0000","last_active":"2024-09-16T09:25:41.289255+0000","last_peered":"2024-09-16T09:25:41.289255+0000","last_clean":"2024-09-16T09:25:41.289255+0000","last_became_active":"2024-09-16T09:25:03.715373+0000","last_became_peered":"2024-09-16T09:25:03.715373+0000","last_unstale":"2024-09-16T09:25:41.289255+0000","last_undegraded":"2024-09-16T09:25:41.289255+0000","last_fullsized":"2024-09-16T09:25:41.289255+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:11.446460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":48,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.288636+0000","last_change":"2024-09-16T09:25:01.708568+0000","last_active":"2024-09-16T09:25:41.288636+0000","last_peered":"2024-09-16T09:25:41.288636+0000","last_clean":"2024-09-16T09:25:41.288636+0000","last_became_active":"2024-09-16T09:25:01.708270+0000","last_became_peered":"2024-09-16T09:25:01.708270+0000","last_unstale":"2024-09-16T09:25:41.288636+0000","last_undegraded":"2024-09-16T09:25:41.288636+0000","last_fullsized":"2024-09-16T09:25:41.288636+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:24.212331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":39,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.287847+0000","last_change":"2024-09-16T09:25:05.712303+0000","last_active":"2024-09-16T09:25:41.287847+0000","last_peered":"2024-09-16T09:25:41.287847+0000","last_clean":"2024-09-16T09:25:41.287847+0000","last_became_active":"2024-09-16T09:25:05.711964+0000","last_became_peered":"2024-09-16T09:25:05.711964+0000","last_unstale":"2024-09-16T09:25:41.287847+0000","last_undegraded":"2024-09-16T09:25:41.287847+0000","last_fullsized":"2024-09-16T09:25:41.287847+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:23.392105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.451234+0000","last_change":"2024-09-16T09:25:07.716780+0000","last_active":"2024-09-16T09:25:41.451234+0000","last_peered":"2024-09-16T09:25:41.451234+0000","last_clean":"2024-09-16T09:25:41.451234+0000","last_became_active":"2024-09-16T09:25:07.716514+0000","last_became_peered":"2024-09-16T09:25:07.716514+0000","last_unstale":"2024-09-16T09:25:41.451234+0000","last_undegraded":"2024-09-16T09:25:41.451234+0000","last_fullsized":"2024-09-16T09:25:41.451234+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:07.166770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"66'17","reported_seq":72,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.289156+0000","last_change":"2024-09-16T09:25:03.713691+0000","last_active":"2024-09-16T09:25:41.289156+0000","last_peered":"2024-09-16T09:25:41.289156+0000","last_clean":"2024-09-16T09:25:41.289156+0000","last_became_active":"2024-09-16T09:25:03.713358+0000","last_became_peered":"2024-09-16T09:25:03.713358+0000","last_unstale":"2024-09-16T09:25:41.289156+0000","last_undegraded":"2024-09-16T09:25:41.289156+0000","last_fullsized":"2024-09-16T09:25:41.289156+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:11.619625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":48,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.450493+0000","last_change":"2024-09-16T09:25:01.703939+0000","last_active":"2024-09-16T09:25:41.450493+0000","last_peered":"2024-09-16T09:25:41.450493+0000","last_clean":"2024-09-16T09:25:41.450493+0000","last_became_active":"2024-09-16T09:25:01.703355+0000","last_became_peered":"2024-09-16T09:25:01.703355+0000","last_unstale":"2024-09-16T09:25:41.450493+0000","last_undegraded":"2024-09-16T09:25:41.450493+0000","last_fullsized":"2024-09-16T09:25:41.450493+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:03.217835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":40,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.286501+0000","last_change":"2024-09-16T09:25:05.717224+0000","last_active":"2024-09-16T09:25:41.286501+0000","last_peered":"2024-09-16T09:25:41.286501+0000","last_clean":"2024-09-16T09:25:41.286501+0000","last_became_active":"2024-09-16T09:25:05.716900+0000","last_became_peered":"2024-09-16T09:25:05.716900+0000","last_unstale":"2024-09-16T09:25:41.286501+0000","last_undegraded":"2024-09-16T09:25:41.286501+0000","last_fullsized":"2024-09-16T09:25:41.286501+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:30.026527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.450528+0000","last_change":"2024-09-16T09:25:07.721657+0000","last_active":"2024-09-16T09:25:41.450528+0000","last_peered":"2024-09-16T09:25:41.450528+0000","last_clean":"2024-09-16T09:25:41.450528+0000","last_became_active":"2024-09-16T09:25:07.721347+0000","last_became_peered":"2024-09-16T09:25:07.721347+0000","last_unstale":"2024-09-16T09:25:41.450528+0000","last_undegraded":"2024-09-16T09:25:41.450528+0000","last_fullsized":"2024-09-16T09:25:41.450528+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:15.094644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"66'10","reported_seq":59,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.288800+0000","last_change":"2024-09-16T09:25:03.708144+0000","last_active":"2024-09-16T09:25:41.288800+0000","last_peered":"2024-09-16T09:25:41.288800+0000","last_clean":"2024-09-16T09:25:41.288800+0000","last_became_active":"2024-09-16T09:25:03.707723+0000","last_became_peered":"2024-09-16T09:25:03.707723+0000","last_unstale":"2024-09-16T09:25:41.288800+0000","last_undegraded":"2024-09-16T09:25:41.288800+0000","last_fullsized":"2024-09-16T09:25:41.288800+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:24.890527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":48,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.451046+0000","last_change":"2024-09-16T09:25:01.703388+0000","last_active":"2024-09-16T09:25:41.451046+0000","last_peered":"2024-09-16T09:25:41.451046+0000","last_clean":"2024-09-16T09:25:41.451046+0000","last_became_active":"2024-09-16T09:25:01.703094+0000","last_became_peered":"2024-09-16T09:25:01.703094+0000","last_unstale":"2024-09-16T09:25:41.451046+0000","last_undegraded":"2024-09-16T09:25:41.451046+0000","last_fullsized":"2024-09-16T09:25:41.451046+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:07.049694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":60,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.450650+0000","last_change":"2024-09-16T09:25:05.711891+0000","last_active":"2024-09-16T09:25:41.450650+0000","last_peered":"2024-09-16T09:25:41.450650+0000","last_clean":"2024-09-16T09:25:41.450650+0000","last_became_active":"2024-09-16T09:25:05.711590+0000","last_became_peered":"2024-09-16T09:25:05.711590+0000","last_unstale":"2024-09-16T09:25:41.450650+0000","last_undegraded":"2024-09-16T09:25:41.450650+0000","last_fullsized":"2024-09-16T09:25:41.450650+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:17.630914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.289092+0000","last_change":"2024-09-16T09:25:07.719496+0000","last_active":"2024-09-16T09:25:41.289092+0000","last_peered":"2024-09-16T09:25:41.289092+0000","last_clean":"2024-09-16T09:25:41.289092+0000","last_became_active":"2024-09-16T09:25:07.718958+0000","last_became_peered":"2024-09-16T09:25:07.718958+0000","last_unstale":"2024-09-16T09:25:41.289092+0000","last_undegraded":"2024-09-16T09:25:41.289092+0000","last_fullsized":"2024-09-16T09:25:41.289092+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:15.393455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.b","version":"66'9","reported_seq":60,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.450529+0000","last_change":"2024-09-16T09:25:03.712553+0000","last_active":"2024-09-16T09:25:41.450529+0000","last_peered":"2024-09-16T09:25:41.450529+0000","last_clean":"2024-09-16T09:25:41.450529+0000","last_became_active":"2024-09-16T09:25:03.712230+0000","last_became_peered":"2024-09-16T09:25:03.712230+0000","last_unstale":"2024-09-16T09:25:41.450529+0000","last_undegraded":"2024-09-16T09:25:41.450529+0000","last_fullsized":"2024-09-16T09:25:41.450529+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:18.262732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":47,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.287757+0000","last_change":"2024-09-16T09:25:01.702943+0000","last_active":"2024-09-16T09:25:41.287757+0000","last_peered":"2024-09-16T09:25:41.287757+0000","last_clean":"2024-09-16T09:25:41.287757+0000","last_became_active":"2024-09-16T09:25:01.702629+0000","last_became_peered":"2024-09-16T09:25:01.702629+0000","last_unstale":"2024-09-16T09:25:41.287757+0000","last_undegraded":"2024-09-16T09:25:41.287757+0000","last_fullsized":"2024-09-16T09:25:41.287757+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:19.763325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":40,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.450175+0000","last_change":"2024-09-16T09:25:05.712932+0000","last_active":"2024-09-16T09:25:41.450175+0000","last_peered":"2024-09-16T09:25:41.450175+0000","last_clean":"2024-09-16T09:25:41.450175+0000","last_became_active":"2024-09-16T09:25:05.712573+0000","last_became_peered":"2024-09-16T09:25:05.712573+0000","last_unstale":"2024-09-16T09:25:41.450175+0000","last_undegraded":"2024-09-16T09:25:41.450175+0000","last_fullsized":"2024-09-16T09:25:41.450175+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:16.863815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.450014+0000","last_change":"2024-09-16T09:25:07.719073+0000","last_active":"2024-09-16T09:25:41.450014+0000","last_peered":"2024-09-16T09:25:41.450014+0000","last_clean":"2024-09-16T09:25:41.450014+0000","last_became_active":"2024-09-16T09:25:07.718778+0000","last_became_peered":"2024-09-16T09:25:07.718778+0000","last_unstale":"2024-09-16T09:25:41.450014+0000","last_undegraded":"2024-09-16T09:25:41.450014+0000","last_fullsized":"2024-09-16T09:25:41.450014+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:21.733588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"66'19","reported_seq":75,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.450244+0000","last_change":"2024-09-16T09:25:03.713182+0000","last_active":"2024-09-16T09:25:41.450244+0000","last_peered":"2024-09-16T09:25:41.450244+0000","last_clean":"2024-09-16T09:25:41.450244+0000","last_became_active":"2024-09-16T09:25:03.712783+0000","last_became_peered":"2024-09-16T09:25:03.712783+0000","last_unstale":"2024-09-16T09:25:41.450244+0000","last_undegraded":"2024-09-16T09:25:41.450244+0000","last_fullsized":"2024-09-16T09:25:41.450244+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:24.025685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"59'1","reported_seq":53,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.450756+0000","last_change":"2024-09-16T09:25:01.708571+0000","last_active":"2024-09-16T09:25:41.450756+0000","last_peered":"2024-09-16T09:25:41.450756+0000","last_clean":"2024-09-16T09:25:41.450756+0000","last_became_active":"2024-09-16T09:25:01.708069+0000","last_became_peered":"2024-09-16T09:25:01.708069+0000","last_unstale":"2024-09-16T09:25:41.450756+0000","last_undegraded":"2024-09-16T09:25:41.450756+0000","last_fullsized":"2024-09-16T09:25:41.450756+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:06.099334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":40,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.450446+0000","last_change":"2024-09-16T09:25:05.709549+0000","last_active":"2024-09-16T09:25:41.450446+0000","last_peered":"2024-09-16T09:25:41.450446+0000","last_clean":"2024-09-16T09:25:41.450446+0000","last_became_active":"2024-09-16T09:25:05.709264+0000","last_became_peered":"2024-09-16T09:25:05.709264+0000","last_unstale":"2024-09-16T09:25:41.450446+0000","last_undegraded":"2024-09-16T09:25:41.450446+0000","last_fullsized":"2024-09-16T09:25:41.450446+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:10.529230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":35,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.449962+0000","last_change":"2024-09-16T09:25:07.719009+0000","last_active":"2024-09-16T09:25:41.449962+0000","last_peered":"2024-09-16T09:25:41.449962+0000","last_clean":"2024-09-16T09:25:41.449962+0000","last_became_active":"2024-09-16T09:25:07.718750+0000","last_became_peered":"2024-09-16T09:25:07.718750+0000","last_unstale":"2024-09-16T09:25:41.449962+0000","last_undegraded":"2024-09-16T09:25:41.449962+0000","last_fullsized":"2024-09-16T09:25:41.449962+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:07.594257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"66'12","reported_seq":67,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.288515+0000","last_change":"2024-09-16T09:25:03.714309+0000","last_active":"2024-09-16T09:25:41.288515+0000","last_peered":"2024-09-16T09:25:41.288515+0000","last_clean":"2024-09-16T09:25:41.288515+0000","last_became_active":"2024-09-16T09:25:03.714027+0000","last_became_peered":"2024-09-16T09:25:03.714027+0000","last_unstale":"2024-09-16T09:25:41.288515+0000","last_undegraded":"2024-09-16T09:25:41.288515+0000","last_fullsized":"2024-09-16T09:25:41.288515+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:31.556984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":47,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.451019+0000","last_change":"2024-09-16T09:25:01.707905+0000","last_active":"2024-09-16T09:25:41.451019+0000","last_peered":"2024-09-16T09:25:41.451019+0000","last_clean":"2024-09-16T09:25:41.451019+0000","last_became_active":"2024-09-16T09:25:01.707528+0000","last_became_peered":"2024-09-16T09:25:01.707528+0000","last_unstale":"2024-09-16T09:25:41.451019+0000","last_undegraded":"2024-09-16T09:25:41.451019+0000","last_fullsized":"2024-09-16T09:25:41.451019+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:13.046169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":40,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.449892+0000","last_change":"2024-09-16T09:25:05.715584+0000","last_active":"2024-09-16T09:25:41.449892+0000","last_peered":"2024-09-16T09:25:41.449892+0000","last_clean":"2024-09-16T09:25:41.449892+0000","last_became_active":"2024-09-16T09:25:05.715253+0000","last_became_peered":"2024-09-16T09:25:05.715253+0000","last_unstale":"2024-09-16T09:25:41.449892+0000","last_undegraded":"2024-09-16T09:25:41.449892+0000","last_fullsized":"2024-09-16T09:25:41.449892+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:23.326757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.450714+0000","last_change":"2024-09-16T09:25:07.719338+0000","last_active":"2024-09-16T09:25:41.450714+0000","last_peered":"2024-09-16T09:25:41.450714+0000","last_clean":"2024-09-16T09:25:41.450714+0000","last_became_active":"2024-09-16T09:25:07.718899+0000","last_became_peered":"2024-09-16T09:25:07.718899+0000","last_unstale":"2024-09-16T09:25:41.450714+0000","last_undegraded":"2024-09-16T09:25:41.450714+0000","last_fullsized":"2024-09-16T09:25:41.450714+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:23.435495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.8","version":"66'15","reported_seq":68,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.286857+0000","last_change":"2024-09-16T09:25:03.716075+0000","last_active":"2024-09-16T09:25:41.286857+0000","last_peered":"2024-09-16T09:25:41.286857+0000","last_clean":"2024-09-16T09:25:41.286857+0000","last_became_active":"2024-09-16T09:25:03.715733+0000","last_became_peered":"2024-09-16T09:25:03.715733+0000","last_unstale":"2024-09-16T09:25:41.286857+0000","last_undegraded":"2024-09-16T09:25:41.286857+0000","last_fullsized":"2024-09-16T09:25:41.286857+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:20.577367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":59,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.451095+0000","last_change":"2024-09-16T09:25:01.707886+0000","last_active":"2024-09-16T09:25:41.451095+0000","last_peered":"2024-09-16T09:25:41.451095+0000","last_clean":"2024-09-16T09:25:41.451095+0000","last_became_active":"2024-09-16T09:25:01.707372+0000","last_became_peered":"2024-09-16T09:25:01.707372+0000","last_unstale":"2024-09-16T09:25:41.451095+0000","last_undegraded":"2024-09-16T09:25:41.451095+0000","last_fullsized":"2024-09-16T09:25:41.451095+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:14.328776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"66'8","reported_seq":56,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.450308+0000","last_change":"2024-09-16T09:25:05.716778+0000","last_active":"2024-09-16T09:25:41.450308+0000","last_peered":"2024-09-16T09:25:41.450308+0000","last_clean":"2024-09-16T09:25:41.450308+0000","last_became_active":"2024-09-16T09:25:05.716189+0000","last_became_peered":"2024-09-16T09:25:05.716189+0000","last_unstale":"2024-09-16T09:25:41.450308+0000","last_undegraded":"2024-09-16T09:25:41.450308+0000","last_fullsized":"2024-09-16T09:25:41.450308+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:31.128760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":35,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.287019+0000","last_change":"2024-09-16T09:25:07.717505+0000","last_active":"2024-09-16T09:25:41.287019+0000","last_peered":"2024-09-16T09:25:41.287019+0000","last_clean":"2024-09-16T09:25:41.287019+0000","last_became_active":"2024-09-16T09:25:07.717213+0000","last_became_peered":"2024-09-16T09:25:07.717213+0000","last_unstale":"2024-09-16T09:25:41.287019+0000","last_undegraded":"2024-09-16T09:25:41.287019+0000","last_fullsized":"2024-09-16T09:25:41.287019+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:09.355594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":48,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.450125+0000","last_change":"2024-09-16T09:25:01.703489+0000","last_active":"2024-09-16T09:25:41.450125+0000","last_peered":"2024-09-16T09:25:41.450125+0000","last_clean":"2024-09-16T09:25:41.450125+0000","last_became_active":"2024-09-16T09:25:01.703202+0000","last_became_peered":"2024-09-16T09:25:01.703202+0000","last_unstale":"2024-09-16T09:25:41.450125+0000","last_undegraded":"2024-09-16T09:25:41.450125+0000","last_fullsized":"2024-09-16T09:25:41.450125+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:30.581098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"66'6","reported_seq":53,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.451828+0000","last_change":"2024-09-16T09:25:03.713606+0000","last_active":"2024-09-16T09:25:41.451828+0000","last_peered":"2024-09-16T09:25:41.451828+0000","last_clean":"2024-09-16T09:25:41.451828+0000","last_became_active":"2024-09-16T09:25:03.713245+0000","last_became_peered":"2024-09-16T09:25:03.713245+0000","last_unstale":"2024-09-16T09:25:41.451828+0000","last_undegraded":"2024-09-16T09:25:41.451828+0000","last_fullsized":"2024-09-16T09:25:41.451828+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:15.800169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":39,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.287638+0000","last_change":"2024-09-16T09:25:05.715938+0000","last_active":"2024-09-16T09:25:41.287638+0000","last_peered":"2024-09-16T09:25:41.287638+0000","last_clean":"2024-09-16T09:25:41.287638+0000","last_became_active":"2024-09-16T09:25:05.715616+0000","last_became_peered":"2024-09-16T09:25:05.715616+0000","last_unstale":"2024-09-16T09:25:41.287638+0000","last_undegraded":"2024-09-16T09:25:41.287638+0000","last_fullsized":"2024-09-16T09:25:41.287638+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:25.224184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":35,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.450428+0000","last_change":"2024-09-16T09:25:07.721375+0000","last_active":"2024-09-16T09:25:41.450428+0000","last_peered":"2024-09-16T09:25:41.450428+0000","last_clean":"2024-09-16T09:25:41.450428+0000","last_became_active":"2024-09-16T09:25:07.721059+0000","last_became_peered":"2024-09-16T09:25:07.721059+0000","last_unstale":"2024-09-16T09:25:41.450428+0000","last_undegraded":"2024-09-16T09:25:41.450428+0000","last_fullsized":"2024-09-16T09:25:41.450428+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:16.275710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"66'9","reported_seq":60,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.451855+0000","last_change":"2024-09-16T09:25:03.719919+0000","last_active":"2024-09-16T09:25:41.451855+0000","last_peered":"2024-09-16T09:25:41.451855+0000","last_clean":"2024-09-16T09:25:41.451855+0000","last_became_active":"2024-09-16T09:25:03.719573+0000","last_became_peered":"2024-09-16T09:25:03.719573+0000","last_unstale":"2024-09-16T09:25:41.451855+0000","last_undegraded":"2024-09-16T09:25:41.451855+0000","last_fullsized":"2024-09-16T09:25:41.451855+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:23.239324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":47,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.450167+0000","last_change":"2024-09-16T09:25:01.707139+0000","last_active":"2024-09-16T09:25:41.450167+0000","last_peered":"2024-09-16T09:25:41.450167+0000","last_clean":"2024-09-16T09:25:41.450167+0000","last_became_active":"2024-09-16T09:25:01.706491+0000","last_became_peered":"2024-09-16T09:25:01.706491+0000","last_unstale":"2024-09-16T09:25:41.450167+0000","last_undegraded":"2024-09-16T09:25:41.450167+0000","last_fullsized":"2024-09-16T09:25:41.450167+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:07.438370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":40,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.451698+0000","last_change":"2024-09-16T09:25:05.715083+0000","last_active":"2024-09-16T09:25:41.451698+0000","last_peered":"2024-09-16T09:25:41.451698+0000","last_clean":"2024-09-16T09:25:41.451698+0000","last_became_active":"2024-09-16T09:25:05.714801+0000","last_became_peered":"2024-09-16T09:25:05.714801+0000","last_unstale":"2024-09-16T09:25:41.451698+0000","last_undegraded":"2024-09-16T09:25:41.451698+0000","last_fullsized":"2024-09-16T09:25:41.451698+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:15.084815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.451022+0000","last_change":"2024-09-16T09:25:07.722356+0000","last_active":"2024-09-16T09:25:41.451022+0000","last_peered":"2024-09-16T09:25:41.451022+0000","last_clean":"2024-09-16T09:25:41.451022+0000","last_became_active":"2024-09-16T09:25:07.722050+0000","last_became_peered":"2024-09-16T09:25:07.722050+0000","last_unstale":"2024-09-16T09:25:41.451022+0000","last_undegraded":"2024-09-16T09:25:41.451022+0000","last_fullsized":"2024-09-16T09:25:41.451022+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:35.793652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"66'9","reported_seq":59,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.287829+0000","last_change":"2024-09-16T09:25:03.717609+0000","last_active":"2024-09-16T09:25:41.287829+0000","last_peered":"2024-09-16T09:25:41.287829+0000","last_clean":"2024-09-16T09:25:41.287829+0000","last_became_active":"2024-09-16T09:25:03.717310+0000","last_became_peered":"2024-09-16T09:25:03.717310+0000","last_unstale":"2024-09-16T09:25:41.287829+0000","last_undegraded":"2024-09-16T09:25:41.287829+0000","last_fullsized":"2024-09-16T09:25:41.287829+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:27.281102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":48,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.449846+0000","last_change":"2024-09-16T09:25:01.702732+0000","last_active":"2024-09-16T09:25:41.449846+0000","last_peered":"2024-09-16T09:25:41.449846+0000","last_clean":"2024-09-16T09:25:41.449846+0000","last_became_active":"2024-09-16T09:25:01.702419+0000","last_became_peered":"2024-09-16T09:25:01.702419+0000","last_unstale":"2024-09-16T09:25:41.449846+0000","last_undegraded":"2024-09-16T09:25:41.449846+0000","last_fullsized":"2024-09-16T09:25:41.449846+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:11.390955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"66'8","reported_seq":57,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.451416+0000","last_change":"2024-09-16T09:25:05.711929+0000","last_active":"2024-09-16T09:25:41.451416+0000","last_peered":"2024-09-16T09:25:41.451416+0000","last_clean":"2024-09-16T09:25:41.451416+0000","last_became_active":"2024-09-16T09:25:05.711642+0000","last_became_peered":"2024-09-16T09:25:05.711642+0000","last_unstale":"2024-09-16T09:25:41.451416+0000","last_undegraded":"2024-09-16T09:25:41.451416+0000","last_fullsized":"2024-09-16T09:25:41.451416+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:29.307023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.289223+0000","last_change":"2024-09-16T09:25:07.719123+0000","last_active":"2024-09-16T09:25:41.289223+0000","last_peered":"2024-09-16T09:25:41.289223+0000","last_clean":"2024-09-16T09:25:41.289223+0000","last_became_active":"2024-09-16T09:25:07.718810+0000","last_became_peered":"2024-09-16T09:25:07.718810+0000","last_unstale":"2024-09-16T09:25:41.289223+0000","last_undegraded":"2024-09-16T09:25:41.289223+0000","last_fullsized":"2024-09-16T09:25:41.289223+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:16.185390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"66'10","reported_seq":59,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.451529+0000","last_change":"2024-09-16T09:25:03.712705+0000","last_active":"2024-09-16T09:25:41.451529+0000","last_peered":"2024-09-16T09:25:41.451529+0000","last_clean":"2024-09-16T09:25:41.451529+0000","last_became_active":"2024-09-16T09:25:03.712387+0000","last_became_peered":"2024-09-16T09:25:03.712387+0000","last_unstale":"2024-09-16T09:25:41.451529+0000","last_undegraded":"2024-09-16T09:25:41.451529+0000","last_fullsized":"2024-09-16T09:25:41.451529+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:26.552452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":47,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.450501+0000","last_change":"2024-09-16T09:25:01.708461+0000","last_active":"2024-09-16T09:25:41.450501+0000","last_peered":"2024-09-16T09:25:41.450501+0000","last_clean":"2024-09-16T09:25:41.450501+0000","last_became_active":"2024-09-16T09:25:01.707513+0000","last_became_peered":"2024-09-16T09:25:01.707513+0000","last_unstale":"2024-09-16T09:25:41.450501+0000","last_undegraded":"2024-09-16T09:25:41.450501+0000","last_fullsized":"2024-09-16T09:25:41.450501+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:10.338638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"66'8","reported_seq":56,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.287661+0000","last_change":"2024-09-16T09:25:05.714214+0000","last_active":"2024-09-16T09:25:41.287661+0000","last_peered":"2024-09-16T09:25:41.287661+0000","last_clean":"2024-09-16T09:25:41.287661+0000","last_became_active":"2024-09-16T09:25:05.713833+0000","last_became_peered":"2024-09-16T09:25:05.713833+0000","last_unstale":"2024-09-16T09:25:41.287661+0000","last_undegraded":"2024-09-16T09:25:41.287661+0000","last_fullsized":"2024-09-16T09:25:41.287661+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:09.052011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.451034+0000","last_change":"2024-09-16T09:25:07.717629+0000","last_active":"2024-09-16T09:25:41.451034+0000","last_peered":"2024-09-16T09:25:41.451034+0000","last_clean":"2024-09-16T09:25:41.451034+0000","last_became_active":"2024-09-16T09:25:07.717357+0000","last_became_peered":"2024-09-16T09:25:07.717357+0000","last_unstale":"2024-09-16T09:25:41.451034+0000","last_undegraded":"2024-09-16T09:25:41.451034+0000","last_fullsized":"2024-09-16T09:25:41.451034+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:28.965425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"66'11","reported_seq":63,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.288606+0000","last_change":"2024-09-16T09:25:03.715601+0000","last_active":"2024-09-16T09:25:41.288606+0000","last_peered":"2024-09-16T09:25:41.288606+0000","last_clean":"2024-09-16T09:25:41.288606+0000","last_became_active":"2024-09-16T09:25:03.715333+0000","last_became_peered":"2024-09-16T09:25:03.715333+0000","last_unstale":"2024-09-16T09:25:41.288606+0000","last_undegraded":"2024-09-16T09:25:41.288606+0000","last_fullsized":"2024-09-16T09:25:41.288606+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:05.154887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":48,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.288975+0000","last_change":"2024-09-16T09:25:01.705459+0000","last_active":"2024-09-16T09:25:41.288975+0000","last_peered":"2024-09-16T09:25:41.288975+0000","last_clean":"2024-09-16T09:25:41.288975+0000","last_became_active":"2024-09-16T09:25:01.705158+0000","last_became_peered":"2024-09-16T09:25:01.705158+0000","last_unstale":"2024-09-16T09:25:41.288975+0000","last_undegraded":"2024-09-16T09:25:41.288975+0000","last_fullsized":"2024-09-16T09:25:41.288975+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:17.236788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":40,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.285899+0000","last_change":"2024-09-16T09:25:05.713087+0000","last_active":"2024-09-16T09:25:41.285899+0000","last_peered":"2024-09-16T09:25:41.285899+0000","last_clean":"2024-09-16T09:25:41.285899+0000","last_became_active":"2024-09-16T09:25:05.712816+0000","last_became_peered":"2024-09-16T09:25:05.712816+0000","last_unstale":"2024-09-16T09:25:41.285899+0000","last_undegraded":"2024-09-16T09:25:41.285899+0000","last_fullsized":"2024-09-16T09:25:41.285899+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:29.267320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.450551+0000","last_change":"2024-09-16T09:25:07.713617+0000","last_active":"2024-09-16T09:25:41.450551+0000","last_peered":"2024-09-16T09:25:41.450551+0000","last_clean":"2024-09-16T09:25:41.450551+0000","last_became_active":"2024-09-16T09:25:07.713109+0000","last_became_peered":"2024-09-16T09:25:07.713109+0000","last_unstale":"2024-09-16T09:25:41.450551+0000","last_undegraded":"2024-09-16T09:25:41.450551+0000","last_fullsized":"2024-09-16T09:25:41.450551+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:17.988989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"66'9","reported_seq":60,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.286104+0000","last_change":"2024-09-16T09:25:03.706126+0000","last_active":"2024-09-16T09:25:41.286104+0000","last_peered":"2024-09-16T09:25:41.286104+0000","last_clean":"2024-09-16T09:25:41.286104+0000","last_became_active":"2024-09-16T09:25:03.705807+0000","last_became_peered":"2024-09-16T09:25:03.705807+0000","last_unstale":"2024-09-16T09:25:41.286104+0000","last_undegraded":"2024-09-16T09:25:41.286104+0000","last_fullsized":"2024-09-16T09:25:41.286104+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:25.655887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":47,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.450730+0000","last_change":"2024-09-16T09:25:01.707075+0000","last_active":"2024-09-16T09:25:41.450730+0000","last_peered":"2024-09-16T09:25:41.450730+0000","last_clean":"2024-09-16T09:25:41.450730+0000","last_became_active":"2024-09-16T09:25:01.706762+0000","last_became_peered":"2024-09-16T09:25:01.706762+0000","last_unstale":"2024-09-16T09:25:41.450730+0000","last_undegraded":"2024-09-16T09:25:41.450730+0000","last_fullsized":"2024-09-16T09:25:41.450730+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:13.882359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":40,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.450797+0000","last_change":"2024-09-16T09:25:05.714142+0000","last_active":"2024-09-16T09:25:41.450797+0000","last_peered":"2024-09-16T09:25:41.450797+0000","last_clean":"2024-09-16T09:25:41.450797+0000","last_became_active":"2024-09-16T09:25:05.713791+0000","last_became_peered":"2024-09-16T09:25:05.713791+0000","last_unstale":"2024-09-16T09:25:41.450797+0000","last_undegraded":"2024-09-16T09:25:41.450797+0000","last_fullsized":"2024-09-16T09:25:41.450797+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:20.280445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.450317+0000","last_change":"2024-09-16T09:25:07.715572+0000","last_active":"2024-09-16T09:25:41.450317+0000","last_peered":"2024-09-16T09:25:41.450317+0000","last_clean":"2024-09-16T09:25:41.450317+0000","last_became_active":"2024-09-16T09:25:07.715301+0000","last_became_peered":"2024-09-16T09:25:07.715301+0000","last_unstale":"2024-09-16T09:25:41.450317+0000","last_undegraded":"2024-09-16T09:25:41.450317+0000","last_fullsized":"2024-09-16T09:25:41.450317+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:15.062673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"66'11","reported_seq":63,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.450687+0000","last_change":"2024-09-16T09:25:03.712637+0000","last_active":"2024-09-16T09:25:41.450687+0000","last_peered":"2024-09-16T09:25:41.450687+0000","last_clean":"2024-09-16T09:25:41.450687+0000","last_became_active":"2024-09-16T09:25:03.712333+0000","last_became_peered":"2024-09-16T09:25:03.712333+0000","last_unstale":"2024-09-16T09:25:41.450687+0000","last_undegraded":"2024-09-16T09:25:41.450687+0000","last_fullsized":"2024-09-16T09:25:41.450687+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:06.593958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":47,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.287760+0000","last_change":"2024-09-16T09:25:01.701295+0000","last_active":"2024-09-16T09:25:41.287760+0000","last_peered":"2024-09-16T09:25:41.287760+0000","last_clean":"2024-09-16T09:25:41.287760+0000","last_became_active":"2024-09-16T09:25:01.700733+0000","last_became_peered":"2024-09-16T09:25:01.700733+0000","last_unstale":"2024-09-16T09:25:41.287760+0000","last_undegraded":"2024-09-16T09:25:41.287760+0000","last_fullsized":"2024-09-16T09:25:41.287760+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:14.332112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":39,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.449631+0000","last_change":"2024-09-16T09:25:05.713760+0000","last_active":"2024-09-16T09:25:41.449631+0000","last_peered":"2024-09-16T09:25:41.449631+0000","last_clean":"2024-09-16T09:25:41.449631+0000","last_became_active":"2024-09-16T09:25:05.713479+0000","last_became_peered":"2024-09-16T09:25:05.713479+0000","last_unstale":"2024-09-16T09:25:41.449631+0000","last_undegraded":"2024-09-16T09:25:41.449631+0000","last_fullsized":"2024-09-16T09:25:41.449631+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:31.316771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.450942+0000","last_change":"2024-09-16T09:25:07.715946+0000","last_active":"2024-09-16T09:25:41.450942+0000","last_peered":"2024-09-16T09:25:41.450942+0000","last_clean":"2024-09-16T09:25:41.450942+0000","last_became_active":"2024-09-16T09:25:07.715582+0000","last_became_peered":"2024-09-16T09:25:07.715582+0000","last_unstale":"2024-09-16T09:25:41.450942+0000","last_undegraded":"2024-09-16T09:25:41.450942+0000","last_fullsized":"2024-09-16T09:25:41.450942+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:18.541254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"66'4","reported_seq":50,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.450044+0000","last_change":"2024-09-16T09:25:03.705167+0000","last_active":"2024-09-16T09:25:41.450044+0000","last_peered":"2024-09-16T09:25:41.450044+0000","last_clean":"2024-09-16T09:25:41.450044+0000","last_became_active":"2024-09-16T09:25:03.704869+0000","last_became_peered":"2024-09-16T09:25:03.704869+0000","last_unstale":"2024-09-16T09:25:41.450044+0000","last_undegraded":"2024-09-16T09:25:41.450044+0000","last_fullsized":"2024-09-16T09:25:41.450044+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:17.043143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":48,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.451506+0000","last_change":"2024-09-16T09:25:01.705965+0000","last_active":"2024-09-16T09:25:41.451506+0000","last_peered":"2024-09-16T09:25:41.451506+0000","last_clean":"2024-09-16T09:25:41.451506+0000","last_became_active":"2024-09-16T09:25:01.705686+0000","last_became_peered":"2024-09-16T09:25:01.705686+0000","last_unstale":"2024-09-16T09:25:41.451506+0000","last_undegraded":"2024-09-16T09:25:41.451506+0000","last_fullsized":"2024-09-16T09:25:41.451506+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:16.022170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":40,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.450011+0000","last_change":"2024-09-16T09:25:05.712816+0000","last_active":"2024-09-16T09:25:41.450011+0000","last_peered":"2024-09-16T09:25:41.450011+0000","last_clean":"2024-09-16T09:25:41.450011+0000","last_became_active":"2024-09-16T09:25:05.712545+0000","last_became_peered":"2024-09-16T09:25:05.712545+0000","last_unstale":"2024-09-16T09:25:41.450011+0000","last_undegraded":"2024-09-16T09:25:41.450011+0000","last_fullsized":"2024-09-16T09:25:41.450011+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:08.170532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":35,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.449875+0000","last_change":"2024-09-16T09:25:07.720368+0000","last_active":"2024-09-16T09:25:41.449875+0000","last_peered":"2024-09-16T09:25:41.449875+0000","last_clean":"2024-09-16T09:25:41.449875+0000","last_became_active":"2024-09-16T09:25:07.720117+0000","last_became_peered":"2024-09-16T09:25:07.720117+0000","last_unstale":"2024-09-16T09:25:41.449875+0000","last_undegraded":"2024-09-16T09:25:41.449875+0000","last_fullsized":"2024-09-16T09:25:41.449875+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:31.504626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.286461+0000","last_change":"2024-09-16T09:25:07.712880+0000","last_active":"2024-09-16T09:25:41.286461+0000","last_peered":"2024-09-16T09:25:41.286461+0000","last_clean":"2024-09-16T09:25:41.286461+0000","last_became_active":"2024-09-16T09:25:07.712594+0000","last_became_peered":"2024-09-16T09:25:07.712594+0000","last_unstale":"2024-09-16T09:25:41.286461+0000","last_undegraded":"2024-09-16T09:25:41.286461+0000","last_fullsized":"2024-09-16T09:25:41.286461+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:33.865572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":48,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.450157+0000","last_change":"2024-09-16T09:25:01.704711+0000","last_active":"2024-09-16T09:25:41.450157+0000","last_peered":"2024-09-16T09:25:41.450157+0000","last_clean":"2024-09-16T09:25:41.450157+0000","last_became_active":"2024-09-16T09:25:01.703736+0000","last_became_peered":"2024-09-16T09:25:01.703736+0000","last_unstale":"2024-09-16T09:25:41.450157+0000","last_undegraded":"2024-09-16T09:25:41.450157+0000","last_fullsized":"2024-09-16T09:25:41.450157+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:09.913011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"66'11","reported_seq":63,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.451122+0000","last_change":"2024-09-16T09:25:03.708095+0000","last_active":"2024-09-16T09:25:41.451122+0000","last_peered":"2024-09-16T09:25:41.451122+0000","last_clean":"2024-09-16T09:25:41.451122+0000","last_became_active":"2024-09-16T09:25:03.707769+0000","last_became_peered":"2024-09-16T09:25:03.707769+0000","last_unstale":"2024-09-16T09:25:41.451122+0000","last_undegraded":"2024-09-16T09:25:41.451122+0000","last_fullsized":"2024-09-16T09:25:41.451122+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:21.606932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":40,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-16T09:25:41.451646+0000","last_change":"2024-09-16T09:25:05.712247+0000","last_active":"2024-09-16T09:25:41.451646+0000","last_peered":"2024-09-16T09:25:41.451646+0000","last_clean":"2024-09-16T09:25:41.451646+0000","last_became_active":"2024-09-16T09:25:05.711952+0000","last_became_peered":"2024-09-16T09:25:05.711952+0000","last_unstale":"2024-09-16T09:25:41.451646+0000","last_undegraded":"2024-09-16T09:25:41.451646+0000","last_fullsized":"2024-09-16T09:25:41.451646+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:30.229560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":393,"ondisk_log_size":393,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":35,"num_read_kb":30,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266701,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32624,"kb_used_data":1432,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712528,"statfs":{"total":95995035648,"available":95961628672,"internally_reserved":0,"allocated":1466368,"data_stored":969480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495634,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28436,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716716,"statfs":{"total":95995035648,"available":95965917184,"internally_reserved":0,"allocated":1441792,"data_stored":968382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724569,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27860,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717292,"statfs":{"total":95995035648,"available":95966507008,"internally_reserved":0,"allocated":851968,"data_stored":378373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953502,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":876544,"data_stored":379176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149732,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32056,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713096,"statfs":{"total":95995035648,"available":95962210304,"internally_reserved":0,"allocated":884736,"data_stored":379462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411369,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32008,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713144,"statfs":{"total":95995035648,"available":95962259456,"internally_reserved":0,"allocated":839680,"data_stored":377726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574896,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":913408,"data_stored":379376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738422,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28468,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716684,"statfs":{"total":95995035648,"available":95965884416,"internally_reserved":0,"allocated":1470464,"data_stored":969746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T09:25:47.685 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-16T09:25:47.685 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-16T09:25:47.685 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-16T09:25:47.686 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph health --format=json 2024-09-16T09:25:47.943 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:25:48.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:48 smithi084 ceph-mon[32629]: from='client.14733 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:25:48.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:48 smithi070 ceph-mon[30388]: from='client.14733 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:25:48.420 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:48 smithi070 ceph-mon[40691]: from='client.14733 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:25:48.771 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:25:48.771 INFO:teuthology.orchestra.run.smithi070.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-16T09:25:49.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:49 smithi084 ceph-mon[32629]: pgmap v16: 132 pgs: 132 active+clean; 583 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:25:49.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:49 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/446945331' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-16T09:25:49.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:49 smithi070 ceph-mon[30388]: pgmap v16: 132 pgs: 132 active+clean; 583 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:25:49.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:49 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/446945331' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-16T09:25:49.420 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:49 smithi070 ceph-mon[40691]: pgmap v16: 132 pgs: 132 active+clean; 583 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:25:49.420 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:49 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/446945331' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-16T09:25:50.168 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-16T09:25:50.168 INFO:tasks.cephadm:Setup complete, yielding 2024-09-16T09:25:50.168 INFO:teuthology.run_tasks:Running task thrashosds... 2024-09-16T09:25:50.181 INFO:tasks.thrashosds:config is {'chance_pgnum_grow': 1, 'chance_pgnum_shrink': 1, 'chance_pgpnum_fix': 1, 'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-09-16T09:25:50.182 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'wip-guits-main-2024-09-13-1248'}, 'ceph': {'conf': {'client.0': {'debug ms': 1, 'debug objecter': 20, 'debug rados': 20}, 'global': {'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'paxos service trim min': 10}, 'osd': {'debug ms': 1, 'debug osd': 20, 'osd debug reject backfill probability': 0.3, 'osd delete sleep': 1, 'osd max backfills': 3, 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 2}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'CEPHADM_STRAY_DAEMON', 'CEPHADM_FAILED_DAEMON', 'but it is still running', 'objects unfound and apparently lost', '\\(POOL_APP_NOT_ENABLED\\)'], 'log-only-match': ['CEPHADM_'], 'sha1': '8293d73f8690540e843a81caec373f9cc29cf705'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'cephadm': {'cephadm_mode': 'root'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '8293d73f8690540e843a81caec373f9cc29cf705'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'workunit': {'branch': 'wip-guits-main-2024-09-13-1248', 'sha1': '8293d73f8690540e843a81caec373f9cc29cf705'}} 2024-09-16T09:25:50.182 INFO:tasks.thrashosds:config is {'chance_pgnum_grow': 1, 'chance_pgnum_shrink': 1, 'chance_pgpnum_fix': 1, 'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-09-16T09:25:50.182 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2024-09-16T09:25:50.503 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:25:51.328 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:51 smithi070 ceph-mon[30388]: pgmap v17: 132 pgs: 132 active+clean; 583 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:25:51.328 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:51 smithi070 ceph-mon[40691]: pgmap v17: 132 pgs: 132 active+clean; 583 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:25:51.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:51 smithi084 ceph-mon[32629]: pgmap v17: 132 pgs: 132 active+clean; 583 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:25:51.607 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:25:51 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: [16/Sep/2024:09:25:51] ENGINE Bus STOPPING 2024-09-16T09:25:51.608 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:25:51 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: [16/Sep/2024:09:25:51] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-16T09:25:51.608 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:25:51 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: [16/Sep/2024:09:25:51] ENGINE Bus STOPPED 2024-09-16T09:25:51.608 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:25:51 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: [16/Sep/2024:09:25:51] ENGINE Bus STARTING 2024-09-16T09:25:51.608 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:25:51 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: [16/Sep/2024:09:25:51] ENGINE Serving on http://:::9283 2024-09-16T09:25:51.608 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:25:51 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: [16/Sep/2024:09:25:51] ENGINE Bus STARTED 2024-09-16T09:25:52.170 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:25:51 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:25:51] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:25:52.660 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:25:52 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:25:52.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:52 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1173617643' entity='client.admin' 2024-09-16T09:25:52.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:52 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-16T09:25:52.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:52 smithi084 ceph-mon[32629]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-16T09:25:52.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:52 smithi084 ceph-mon[32629]: pgmap v18: 132 pgs: 132 active+clean; 583 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-16T09:25:52.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:52 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1173617643' entity='client.admin' 2024-09-16T09:25:52.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:52 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-16T09:25:52.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:52 smithi070 ceph-mon[30388]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-16T09:25:52.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:52 smithi070 ceph-mon[30388]: pgmap v18: 132 pgs: 132 active+clean; 583 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-16T09:25:52.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:52 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1173617643' entity='client.admin' 2024-09-16T09:25:52.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:52 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-16T09:25:52.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:52 smithi070 ceph-mon[40691]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-16T09:25:52.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:52 smithi070 ceph-mon[40691]: pgmap v18: 132 pgs: 132 active+clean; 583 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-16T09:25:52.738 INFO:tasks.thrashosds:Beginning thrashosds... 2024-09-16T09:25:52.738 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-16T09:25:52.738 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-16T09:25:53.525 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:53 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:25:53.526 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:53 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:25:53.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:53 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:25:54.125 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:25:54.760 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:25:54.762 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2024-09-16T09:25:55.351 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:55 smithi070 ceph-mon[30388]: pgmap v19: 132 pgs: 132 active+clean; 583 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-16T09:25:55.351 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:55 smithi070 ceph-mon[40691]: pgmap v19: 132 pgs: 132 active+clean; 583 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-16T09:25:55.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:55 smithi084 ceph-mon[32629]: pgmap v19: 132 pgs: 132 active+clean; 583 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-16T09:25:56.093 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Sep 16 09:25:55 smithi070 systemd[1]: Starting Ceph alertmanager.a for a1321834-740c-11ef-bceb-c7b262605968... 2024-09-16T09:25:56.208 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2024-09-16T09:25:53.483679+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596507,"num_objects":199,"num_object_clones":0,"num_object_copies":597,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":199,"num_whiteouts":0,"num_read":893,"num_read_kb":750,"num_write":549,"num_write_kb":1673,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":552,"ondisk_log_size":552,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":241500,"kb_used_data":8636,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":749719716,"statfs":{"total":767960285184,"available":767712989184,"internally_reserved":0,"allocated":8843264,"data_stored":4862689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":238276146},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002888"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":50,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.085013+0000","last_change":"2024-09-16T09:25:01.709392+0000","last_active":"2024-09-16T09:25:44.085013+0000","last_peered":"2024-09-16T09:25:44.085013+0000","last_clean":"2024-09-16T09:25:44.085013+0000","last_became_active":"2024-09-16T09:25:01.709108+0000","last_became_peered":"2024-09-16T09:25:01.709108+0000","last_unstale":"2024-09-16T09:25:44.085013+0000","last_undegraded":"2024-09-16T09:25:44.085013+0000","last_fullsized":"2024-09-16T09:25:44.085013+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:05.830525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"66'9","reported_seq":62,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.084866+0000","last_change":"2024-09-16T09:25:03.715139+0000","last_active":"2024-09-16T09:25:44.084866+0000","last_peered":"2024-09-16T09:25:44.084866+0000","last_clean":"2024-09-16T09:25:44.084866+0000","last_became_active":"2024-09-16T09:25:03.714635+0000","last_became_peered":"2024-09-16T09:25:03.714635+0000","last_unstale":"2024-09-16T09:25:44.084866+0000","last_undegraded":"2024-09-16T09:25:44.084866+0000","last_fullsized":"2024-09-16T09:25:44.084866+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:19.723608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":42,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.084994+0000","last_change":"2024-09-16T09:25:05.716139+0000","last_active":"2024-09-16T09:25:44.084994+0000","last_peered":"2024-09-16T09:25:44.084994+0000","last_clean":"2024-09-16T09:25:44.084994+0000","last_became_active":"2024-09-16T09:25:05.715645+0000","last_became_peered":"2024-09-16T09:25:05.715645+0000","last_unstale":"2024-09-16T09:25:44.084994+0000","last_undegraded":"2024-09-16T09:25:44.084994+0000","last_fullsized":"2024-09-16T09:25:44.084994+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:31.956608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":38,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.085058+0000","last_change":"2024-09-16T09:25:07.715923+0000","last_active":"2024-09-16T09:25:44.085058+0000","last_peered":"2024-09-16T09:25:44.085058+0000","last_clean":"2024-09-16T09:25:44.085058+0000","last_became_active":"2024-09-16T09:25:07.715442+0000","last_became_peered":"2024-09-16T09:25:07.715442+0000","last_unstale":"2024-09-16T09:25:44.085058+0000","last_undegraded":"2024-09-16T09:25:44.085058+0000","last_fullsized":"2024-09-16T09:25:44.085058+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:35.494958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":38,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.114483+0000","last_change":"2024-09-16T09:25:07.718687+0000","last_active":"2024-09-16T09:25:44.114483+0000","last_peered":"2024-09-16T09:25:44.114483+0000","last_clean":"2024-09-16T09:25:44.114483+0000","last_became_active":"2024-09-16T09:25:07.718126+0000","last_became_peered":"2024-09-16T09:25:07.718126+0000","last_unstale":"2024-09-16T09:25:44.114483+0000","last_undegraded":"2024-09-16T09:25:44.114483+0000","last_fullsized":"2024-09-16T09:25:44.114483+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:23.444563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":50,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.115035+0000","last_change":"2024-09-16T09:25:01.708505+0000","last_active":"2024-09-16T09:25:44.115035+0000","last_peered":"2024-09-16T09:25:44.115035+0000","last_clean":"2024-09-16T09:25:44.115035+0000","last_became_active":"2024-09-16T09:25:01.708179+0000","last_became_peered":"2024-09-16T09:25:01.708179+0000","last_unstale":"2024-09-16T09:25:44.115035+0000","last_undegraded":"2024-09-16T09:25:44.115035+0000","last_fullsized":"2024-09-16T09:25:44.115035+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:22.060114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"66'15","reported_seq":71,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.114635+0000","last_change":"2024-09-16T09:25:03.709970+0000","last_active":"2024-09-16T09:25:44.114635+0000","last_peered":"2024-09-16T09:25:44.114635+0000","last_clean":"2024-09-16T09:25:44.114635+0000","last_became_active":"2024-09-16T09:25:03.709499+0000","last_became_peered":"2024-09-16T09:25:03.709499+0000","last_unstale":"2024-09-16T09:25:44.114635+0000","last_undegraded":"2024-09-16T09:25:44.114635+0000","last_fullsized":"2024-09-16T09:25:44.114635+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:05.973210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":42,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.084860+0000","last_change":"2024-09-16T09:25:05.717524+0000","last_active":"2024-09-16T09:25:44.084860+0000","last_peered":"2024-09-16T09:25:44.084860+0000","last_clean":"2024-09-16T09:25:44.084860+0000","last_became_active":"2024-09-16T09:25:05.717236+0000","last_became_peered":"2024-09-16T09:25:05.717236+0000","last_unstale":"2024-09-16T09:25:44.084860+0000","last_undegraded":"2024-09-16T09:25:44.084860+0000","last_fullsized":"2024-09-16T09:25:44.084860+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:14.774263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":49,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.605268+0000","last_change":"2024-09-16T09:25:01.705552+0000","last_active":"2024-09-16T09:25:44.605268+0000","last_peered":"2024-09-16T09:25:44.605268+0000","last_clean":"2024-09-16T09:25:44.605268+0000","last_became_active":"2024-09-16T09:25:01.705269+0000","last_became_peered":"2024-09-16T09:25:01.705269+0000","last_unstale":"2024-09-16T09:25:44.605268+0000","last_undegraded":"2024-09-16T09:25:44.605268+0000","last_fullsized":"2024-09-16T09:25:44.605268+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:18.445329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"66'9","reported_seq":62,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.085428+0000","last_change":"2024-09-16T09:25:03.709441+0000","last_active":"2024-09-16T09:25:44.085428+0000","last_peered":"2024-09-16T09:25:44.085428+0000","last_clean":"2024-09-16T09:25:44.085428+0000","last_became_active":"2024-09-16T09:25:03.709147+0000","last_became_peered":"2024-09-16T09:25:03.709147+0000","last_unstale":"2024-09-16T09:25:44.085428+0000","last_undegraded":"2024-09-16T09:25:44.085428+0000","last_fullsized":"2024-09-16T09:25:44.085428+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:21.309548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":41,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.604538+0000","last_change":"2024-09-16T09:25:05.718510+0000","last_active":"2024-09-16T09:25:44.604538+0000","last_peered":"2024-09-16T09:25:44.604538+0000","last_clean":"2024-09-16T09:25:44.604538+0000","last_became_active":"2024-09-16T09:25:05.718134+0000","last_became_peered":"2024-09-16T09:25:05.718134+0000","last_unstale":"2024-09-16T09:25:44.604538+0000","last_undegraded":"2024-09-16T09:25:44.604538+0000","last_fullsized":"2024-09-16T09:25:44.604538+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:30.593901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"66'1","reported_seq":39,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.084246+0000","last_change":"2024-09-16T09:25:07.716869+0000","last_active":"2024-09-16T09:25:44.084246+0000","last_peered":"2024-09-16T09:25:44.084246+0000","last_clean":"2024-09-16T09:25:44.084246+0000","last_became_active":"2024-09-16T09:25:07.716558+0000","last_became_peered":"2024-09-16T09:25:07.716558+0000","last_unstale":"2024-09-16T09:25:44.084246+0000","last_undegraded":"2024-09-16T09:25:44.084246+0000","last_fullsized":"2024-09-16T09:25:44.084246+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:17.035822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":49,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.604930+0000","last_change":"2024-09-16T09:25:01.697306+0000","last_active":"2024-09-16T09:25:44.604930+0000","last_peered":"2024-09-16T09:25:44.604930+0000","last_clean":"2024-09-16T09:25:44.604930+0000","last_became_active":"2024-09-16T09:25:01.696939+0000","last_became_peered":"2024-09-16T09:25:01.696939+0000","last_unstale":"2024-09-16T09:25:44.604930+0000","last_undegraded":"2024-09-16T09:25:44.604930+0000","last_fullsized":"2024-09-16T09:25:44.604930+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:26.024926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"66'5","reported_seq":56,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.085109+0000","last_change":"2024-09-16T09:25:03.716402+0000","last_active":"2024-09-16T09:25:44.085109+0000","last_peered":"2024-09-16T09:25:44.085109+0000","last_clean":"2024-09-16T09:25:44.085109+0000","last_became_active":"2024-09-16T09:25:03.716136+0000","last_became_peered":"2024-09-16T09:25:03.716136+0000","last_unstale":"2024-09-16T09:25:44.085109+0000","last_undegraded":"2024-09-16T09:25:44.085109+0000","last_fullsized":"2024-09-16T09:25:44.085109+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:32.360413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":41,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.605054+0000","last_change":"2024-09-16T09:25:05.718538+0000","last_active":"2024-09-16T09:25:44.605054+0000","last_peered":"2024-09-16T09:25:44.605054+0000","last_clean":"2024-09-16T09:25:44.605054+0000","last_became_active":"2024-09-16T09:25:05.718281+0000","last_became_peered":"2024-09-16T09:25:05.718281+0000","last_unstale":"2024-09-16T09:25:44.605054+0000","last_undegraded":"2024-09-16T09:25:44.605054+0000","last_fullsized":"2024-09-16T09:25:44.605054+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:29.527824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":37,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.604126+0000","last_change":"2024-09-16T09:25:07.717644+0000","last_active":"2024-09-16T09:25:44.604126+0000","last_peered":"2024-09-16T09:25:44.604126+0000","last_clean":"2024-09-16T09:25:44.604126+0000","last_became_active":"2024-09-16T09:25:07.716724+0000","last_became_peered":"2024-09-16T09:25:07.716724+0000","last_unstale":"2024-09-16T09:25:44.604126+0000","last_undegraded":"2024-09-16T09:25:44.604126+0000","last_fullsized":"2024-09-16T09:25:44.604126+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:32.145892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":38,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.085350+0000","last_change":"2024-09-16T09:25:07.721088+0000","last_active":"2024-09-16T09:25:44.085350+0000","last_peered":"2024-09-16T09:25:44.085350+0000","last_clean":"2024-09-16T09:25:44.085350+0000","last_became_active":"2024-09-16T09:25:07.720793+0000","last_became_peered":"2024-09-16T09:25:07.720793+0000","last_unstale":"2024-09-16T09:25:44.085350+0000","last_undegraded":"2024-09-16T09:25:44.085350+0000","last_fullsized":"2024-09-16T09:25:44.085350+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:16.293244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":50,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.085187+0000","last_change":"2024-09-16T09:25:01.706762+0000","last_active":"2024-09-16T09:25:44.085187+0000","last_peered":"2024-09-16T09:25:44.085187+0000","last_clean":"2024-09-16T09:25:44.085187+0000","last_became_active":"2024-09-16T09:25:01.706172+0000","last_became_peered":"2024-09-16T09:25:01.706172+0000","last_unstale":"2024-09-16T09:25:44.085187+0000","last_undegraded":"2024-09-16T09:25:44.085187+0000","last_fullsized":"2024-09-16T09:25:44.085187+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:07.135883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"66'15","reported_seq":71,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.604477+0000","last_change":"2024-09-16T09:25:03.714261+0000","last_active":"2024-09-16T09:25:44.604477+0000","last_peered":"2024-09-16T09:25:44.604477+0000","last_clean":"2024-09-16T09:25:44.604477+0000","last_became_active":"2024-09-16T09:25:03.713780+0000","last_became_peered":"2024-09-16T09:25:03.713780+0000","last_unstale":"2024-09-16T09:25:44.604477+0000","last_undegraded":"2024-09-16T09:25:44.604477+0000","last_fullsized":"2024-09-16T09:25:44.604477+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:29.826960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":42,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.085112+0000","last_change":"2024-09-16T09:25:05.714209+0000","last_active":"2024-09-16T09:25:44.085112+0000","last_peered":"2024-09-16T09:25:44.085112+0000","last_clean":"2024-09-16T09:25:44.085112+0000","last_became_active":"2024-09-16T09:25:05.713929+0000","last_became_peered":"2024-09-16T09:25:05.713929+0000","last_unstale":"2024-09-16T09:25:44.085112+0000","last_undegraded":"2024-09-16T09:25:44.085112+0000","last_fullsized":"2024-09-16T09:25:44.085112+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:05.941022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":38,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.115048+0000","last_change":"2024-09-16T09:25:07.719645+0000","last_active":"2024-09-16T09:25:44.115048+0000","last_peered":"2024-09-16T09:25:44.115048+0000","last_clean":"2024-09-16T09:25:44.115048+0000","last_became_active":"2024-09-16T09:25:07.719067+0000","last_became_peered":"2024-09-16T09:25:07.719067+0000","last_unstale":"2024-09-16T09:25:44.115048+0000","last_undegraded":"2024-09-16T09:25:44.115048+0000","last_fullsized":"2024-09-16T09:25:44.115048+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:19.447150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":50,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.085738+0000","last_change":"2024-09-16T09:25:01.706828+0000","last_active":"2024-09-16T09:25:44.085738+0000","last_peered":"2024-09-16T09:25:44.085738+0000","last_clean":"2024-09-16T09:25:44.085738+0000","last_became_active":"2024-09-16T09:25:01.706491+0000","last_became_peered":"2024-09-16T09:25:01.706491+0000","last_unstale":"2024-09-16T09:25:44.085738+0000","last_undegraded":"2024-09-16T09:25:44.085738+0000","last_fullsized":"2024-09-16T09:25:44.085738+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:16.321414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"66'12","reported_seq":69,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.114882+0000","last_change":"2024-09-16T09:25:03.709914+0000","last_active":"2024-09-16T09:25:44.114882+0000","last_peered":"2024-09-16T09:25:44.114882+0000","last_clean":"2024-09-16T09:25:44.114882+0000","last_became_active":"2024-09-16T09:25:03.709610+0000","last_became_peered":"2024-09-16T09:25:03.709610+0000","last_unstale":"2024-09-16T09:25:44.114882+0000","last_undegraded":"2024-09-16T09:25:44.114882+0000","last_fullsized":"2024-09-16T09:25:44.114882+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:26.267096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":42,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.085216+0000","last_change":"2024-09-16T09:25:05.710379+0000","last_active":"2024-09-16T09:25:44.085216+0000","last_peered":"2024-09-16T09:25:44.085216+0000","last_clean":"2024-09-16T09:25:44.085216+0000","last_became_active":"2024-09-16T09:25:05.710093+0000","last_became_peered":"2024-09-16T09:25:05.710093+0000","last_unstale":"2024-09-16T09:25:44.085216+0000","last_undegraded":"2024-09-16T09:25:44.085216+0000","last_fullsized":"2024-09-16T09:25:44.085216+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:28.802108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":38,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.605509+0000","last_change":"2024-09-16T09:25:07.719655+0000","last_active":"2024-09-16T09:25:44.605509+0000","last_peered":"2024-09-16T09:25:44.605509+0000","last_clean":"2024-09-16T09:25:44.605509+0000","last_became_active":"2024-09-16T09:25:07.719380+0000","last_became_peered":"2024-09-16T09:25:07.719380+0000","last_unstale":"2024-09-16T09:25:44.605509+0000","last_undegraded":"2024-09-16T09:25:44.605509+0000","last_fullsized":"2024-09-16T09:25:44.605509+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:36.059179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":51,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.085126+0000","last_change":"2024-09-16T09:25:01.700263+0000","last_active":"2024-09-16T09:25:44.085126+0000","last_peered":"2024-09-16T09:25:44.085126+0000","last_clean":"2024-09-16T09:25:44.085126+0000","last_became_active":"2024-09-16T09:25:01.699545+0000","last_became_peered":"2024-09-16T09:25:01.699545+0000","last_unstale":"2024-09-16T09:25:44.085126+0000","last_undegraded":"2024-09-16T09:25:44.085126+0000","last_fullsized":"2024-09-16T09:25:44.085126+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:04.529119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"66'10","reported_seq":61,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.085454+0000","last_change":"2024-09-16T09:25:03.720638+0000","last_active":"2024-09-16T09:25:44.085454+0000","last_peered":"2024-09-16T09:25:44.085454+0000","last_clean":"2024-09-16T09:25:44.085454+0000","last_became_active":"2024-09-16T09:25:03.720245+0000","last_became_peered":"2024-09-16T09:25:03.720245+0000","last_unstale":"2024-09-16T09:25:44.085454+0000","last_undegraded":"2024-09-16T09:25:44.085454+0000","last_fullsized":"2024-09-16T09:25:44.085454+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:21.451538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"66'8","reported_seq":64,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:46.757941+0000","last_change":"2024-09-16T09:25:05.713310+0000","last_active":"2024-09-16T09:25:46.757941+0000","last_peered":"2024-09-16T09:25:46.757941+0000","last_clean":"2024-09-16T09:25:46.757941+0000","last_became_active":"2024-09-16T09:25:05.712516+0000","last_became_peered":"2024-09-16T09:25:05.712516+0000","last_unstale":"2024-09-16T09:25:46.757941+0000","last_undegraded":"2024-09-16T09:25:46.757941+0000","last_fullsized":"2024-09-16T09:25:46.757941+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:24.505283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.f","version":"66'15","reported_seq":71,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.086235+0000","last_change":"2024-09-16T09:25:03.719134+0000","last_active":"2024-09-16T09:25:44.086235+0000","last_peered":"2024-09-16T09:25:44.086235+0000","last_clean":"2024-09-16T09:25:44.086235+0000","last_became_active":"2024-09-16T09:25:03.718849+0000","last_became_peered":"2024-09-16T09:25:03.718849+0000","last_unstale":"2024-09-16T09:25:44.086235+0000","last_undegraded":"2024-09-16T09:25:44.086235+0000","last_fullsized":"2024-09-16T09:25:44.086235+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:13.173718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":50,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.114539+0000","last_change":"2024-09-16T09:25:01.702936+0000","last_active":"2024-09-16T09:25:44.114539+0000","last_peered":"2024-09-16T09:25:44.114539+0000","last_clean":"2024-09-16T09:25:44.114539+0000","last_became_active":"2024-09-16T09:25:01.702629+0000","last_became_peered":"2024-09-16T09:25:01.702629+0000","last_unstale":"2024-09-16T09:25:44.114539+0000","last_undegraded":"2024-09-16T09:25:44.114539+0000","last_fullsized":"2024-09-16T09:25:44.114539+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:20.061024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"66'8","reported_seq":61,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:46.757971+0000","last_change":"2024-09-16T09:25:05.714128+0000","last_active":"2024-09-16T09:25:46.757971+0000","last_peered":"2024-09-16T09:25:46.757971+0000","last_clean":"2024-09-16T09:25:46.757971+0000","last_became_active":"2024-09-16T09:25:05.713813+0000","last_became_peered":"2024-09-16T09:25:05.713813+0000","last_unstale":"2024-09-16T09:25:46.757971+0000","last_undegraded":"2024-09-16T09:25:46.757971+0000","last_fullsized":"2024-09-16T09:25:46.757971+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:26.013786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":37,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.604736+0000","last_change":"2024-09-16T09:25:07.717639+0000","last_active":"2024-09-16T09:25:44.604736+0000","last_peered":"2024-09-16T09:25:44.604736+0000","last_clean":"2024-09-16T09:25:44.604736+0000","last_became_active":"2024-09-16T09:25:07.717344+0000","last_became_peered":"2024-09-16T09:25:07.717344+0000","last_unstale":"2024-09-16T09:25:44.604736+0000","last_undegraded":"2024-09-16T09:25:44.604736+0000","last_fullsized":"2024-09-16T09:25:44.604736+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:24.054256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.0","version":"66'18","reported_seq":78,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.114403+0000","last_change":"2024-09-16T09:25:03.714508+0000","last_active":"2024-09-16T09:25:44.114403+0000","last_peered":"2024-09-16T09:25:44.114403+0000","last_clean":"2024-09-16T09:25:44.114403+0000","last_became_active":"2024-09-16T09:25:03.714208+0000","last_became_peered":"2024-09-16T09:25:03.714208+0000","last_unstale":"2024-09-16T09:25:44.114403+0000","last_undegraded":"2024-09-16T09:25:44.114403+0000","last_fullsized":"2024-09-16T09:25:44.114403+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:03.200598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":50,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.115220+0000","last_change":"2024-09-16T09:25:01.705362+0000","last_active":"2024-09-16T09:25:44.115220+0000","last_peered":"2024-09-16T09:25:44.115220+0000","last_clean":"2024-09-16T09:25:44.115220+0000","last_became_active":"2024-09-16T09:25:01.704994+0000","last_became_peered":"2024-09-16T09:25:01.704994+0000","last_unstale":"2024-09-16T09:25:44.115220+0000","last_undegraded":"2024-09-16T09:25:44.115220+0000","last_fullsized":"2024-09-16T09:25:44.115220+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:17.466780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":42,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.084923+0000","last_change":"2024-09-16T09:25:05.713687+0000","last_active":"2024-09-16T09:25:44.084923+0000","last_peered":"2024-09-16T09:25:44.084923+0000","last_clean":"2024-09-16T09:25:44.084923+0000","last_became_active":"2024-09-16T09:25:05.713360+0000","last_became_peered":"2024-09-16T09:25:05.713360+0000","last_unstale":"2024-09-16T09:25:44.084923+0000","last_undegraded":"2024-09-16T09:25:44.084923+0000","last_fullsized":"2024-09-16T09:25:44.084923+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:16.400241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":38,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.085226+0000","last_change":"2024-09-16T09:25:07.719412+0000","last_active":"2024-09-16T09:25:44.085226+0000","last_peered":"2024-09-16T09:25:44.085226+0000","last_clean":"2024-09-16T09:25:44.085226+0000","last_became_active":"2024-09-16T09:25:07.719159+0000","last_became_peered":"2024-09-16T09:25:07.719159+0000","last_unstale":"2024-09-16T09:25:44.085226+0000","last_undegraded":"2024-09-16T09:25:44.085226+0000","last_fullsized":"2024-09-16T09:25:44.085226+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:36.545361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"66'14","reported_seq":67,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.084923+0000","last_change":"2024-09-16T09:25:03.710438+0000","last_active":"2024-09-16T09:25:44.084923+0000","last_peered":"2024-09-16T09:25:44.084923+0000","last_clean":"2024-09-16T09:25:44.084923+0000","last_became_active":"2024-09-16T09:25:03.708830+0000","last_became_peered":"2024-09-16T09:25:03.708830+0000","last_unstale":"2024-09-16T09:25:44.084923+0000","last_undegraded":"2024-09-16T09:25:44.084923+0000","last_fullsized":"2024-09-16T09:25:44.084923+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:17.190767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":51,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.085717+0000","last_change":"2024-09-16T09:25:01.703245+0000","last_active":"2024-09-16T09:25:44.085717+0000","last_peered":"2024-09-16T09:25:44.085717+0000","last_clean":"2024-09-16T09:25:44.085717+0000","last_became_active":"2024-09-16T09:25:01.702992+0000","last_became_peered":"2024-09-16T09:25:01.702992+0000","last_unstale":"2024-09-16T09:25:44.085717+0000","last_undegraded":"2024-09-16T09:25:44.085717+0000","last_fullsized":"2024-09-16T09:25:44.085717+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:00.901034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"66'8","reported_seq":61,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:46.757849+0000","last_change":"2024-09-16T09:25:05.715927+0000","last_active":"2024-09-16T09:25:46.757849+0000","last_peered":"2024-09-16T09:25:46.757849+0000","last_clean":"2024-09-16T09:25:46.757849+0000","last_became_active":"2024-09-16T09:25:05.715606+0000","last_became_peered":"2024-09-16T09:25:05.715606+0000","last_unstale":"2024-09-16T09:25:46.757849+0000","last_undegraded":"2024-09-16T09:25:46.757849+0000","last_fullsized":"2024-09-16T09:25:46.757849+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:23.752404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":37,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.605258+0000","last_change":"2024-09-16T09:25:07.722287+0000","last_active":"2024-09-16T09:25:44.605258+0000","last_peered":"2024-09-16T09:25:44.605258+0000","last_clean":"2024-09-16T09:25:44.605258+0000","last_became_active":"2024-09-16T09:25:07.722017+0000","last_became_peered":"2024-09-16T09:25:07.722017+0000","last_unstale":"2024-09-16T09:25:44.605258+0000","last_undegraded":"2024-09-16T09:25:44.605258+0000","last_fullsized":"2024-09-16T09:25:44.605258+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:27.828108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2","version":"66'10","reported_seq":61,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.086135+0000","last_change":"2024-09-16T09:25:03.713368+0000","last_active":"2024-09-16T09:25:44.086135+0000","last_peered":"2024-09-16T09:25:44.086135+0000","last_clean":"2024-09-16T09:25:44.086135+0000","last_became_active":"2024-09-16T09:25:03.712985+0000","last_became_peered":"2024-09-16T09:25:03.712985+0000","last_unstale":"2024-09-16T09:25:44.086135+0000","last_undegraded":"2024-09-16T09:25:44.086135+0000","last_fullsized":"2024-09-16T09:25:44.086135+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:21.169460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":49,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.605335+0000","last_change":"2024-09-16T09:25:01.706774+0000","last_active":"2024-09-16T09:25:44.605335+0000","last_peered":"2024-09-16T09:25:44.605335+0000","last_clean":"2024-09-16T09:25:44.605335+0000","last_became_active":"2024-09-16T09:25:01.706440+0000","last_became_peered":"2024-09-16T09:25:01.706440+0000","last_unstale":"2024-09-16T09:25:44.605335+0000","last_undegraded":"2024-09-16T09:25:44.605335+0000","last_fullsized":"2024-09-16T09:25:44.605335+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:18.805280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"66'8","reported_seq":61,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:46.757821+0000","last_change":"2024-09-16T09:25:05.714825+0000","last_active":"2024-09-16T09:25:46.757821+0000","last_peered":"2024-09-16T09:25:46.757821+0000","last_clean":"2024-09-16T09:25:46.757821+0000","last_became_active":"2024-09-16T09:25:05.714507+0000","last_became_peered":"2024-09-16T09:25:05.714507+0000","last_unstale":"2024-09-16T09:25:46.757821+0000","last_undegraded":"2024-09-16T09:25:46.757821+0000","last_fullsized":"2024-09-16T09:25:46.757821+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:13.181647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":38,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.084322+0000","last_change":"2024-09-16T09:25:07.719876+0000","last_active":"2024-09-16T09:25:44.084322+0000","last_peered":"2024-09-16T09:25:44.084322+0000","last_clean":"2024-09-16T09:25:44.084322+0000","last_became_active":"2024-09-16T09:25:07.719604+0000","last_became_peered":"2024-09-16T09:25:07.719604+0000","last_unstale":"2024-09-16T09:25:44.084322+0000","last_undegraded":"2024-09-16T09:25:44.084322+0000","last_fullsized":"2024-09-16T09:25:44.084322+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:20.929075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"66'19","reported_seq":82,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.085024+0000","last_change":"2024-09-16T09:25:03.715086+0000","last_active":"2024-09-16T09:25:44.085024+0000","last_peered":"2024-09-16T09:25:44.085024+0000","last_clean":"2024-09-16T09:25:44.085024+0000","last_became_active":"2024-09-16T09:25:03.714797+0000","last_became_peered":"2024-09-16T09:25:03.714797+0000","last_unstale":"2024-09-16T09:25:44.085024+0000","last_undegraded":"2024-09-16T09:25:44.085024+0000","last_fullsized":"2024-09-16T09:25:44.085024+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:23.683038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":50,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.085873+0000","last_change":"2024-09-16T09:25:01.707093+0000","last_active":"2024-09-16T09:25:44.085873+0000","last_peered":"2024-09-16T09:25:44.085873+0000","last_clean":"2024-09-16T09:25:44.085873+0000","last_became_active":"2024-09-16T09:25:01.706806+0000","last_became_peered":"2024-09-16T09:25:01.706806+0000","last_unstale":"2024-09-16T09:25:44.085873+0000","last_undegraded":"2024-09-16T09:25:44.085873+0000","last_fullsized":"2024-09-16T09:25:44.085873+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:01.655926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":42,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.302460+0000","last_change":"2024-09-16T09:25:05.708949+0000","last_active":"2024-09-16T09:25:44.302460+0000","last_peered":"2024-09-16T09:25:44.302460+0000","last_clean":"2024-09-16T09:25:44.302460+0000","last_became_active":"2024-09-16T09:25:05.708663+0000","last_became_peered":"2024-09-16T09:25:05.708663+0000","last_unstale":"2024-09-16T09:25:44.302460+0000","last_undegraded":"2024-09-16T09:25:44.302460+0000","last_fullsized":"2024-09-16T09:25:44.302460+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:06.107203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":38,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.084912+0000","last_change":"2024-09-16T09:25:07.719568+0000","last_active":"2024-09-16T09:25:44.084912+0000","last_peered":"2024-09-16T09:25:44.084912+0000","last_clean":"2024-09-16T09:25:44.084912+0000","last_became_active":"2024-09-16T09:25:07.719267+0000","last_became_peered":"2024-09-16T09:25:07.719267+0000","last_unstale":"2024-09-16T09:25:44.084912+0000","last_undegraded":"2024-09-16T09:25:44.084912+0000","last_fullsized":"2024-09-16T09:25:44.084912+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:31.531856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"66'28","reported_seq":105,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:46.757815+0000","last_change":"2024-09-16T09:25:03.705300+0000","last_active":"2024-09-16T09:25:46.757815+0000","last_peered":"2024-09-16T09:25:46.757815+0000","last_clean":"2024-09-16T09:25:46.757815+0000","last_became_active":"2024-09-16T09:25:03.704878+0000","last_became_peered":"2024-09-16T09:25:03.704878+0000","last_unstale":"2024-09-16T09:25:46.757815+0000","last_undegraded":"2024-09-16T09:25:46.757815+0000","last_fullsized":"2024-09-16T09:25:46.757815+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:32.612703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":50,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.085856+0000","last_change":"2024-09-16T09:25:01.703394+0000","last_active":"2024-09-16T09:25:44.085856+0000","last_peered":"2024-09-16T09:25:44.085856+0000","last_clean":"2024-09-16T09:25:44.085856+0000","last_became_active":"2024-09-16T09:25:01.703076+0000","last_became_peered":"2024-09-16T09:25:01.703076+0000","last_unstale":"2024-09-16T09:25:44.085856+0000","last_undegraded":"2024-09-16T09:25:44.085856+0000","last_fullsized":"2024-09-16T09:25:44.085856+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:21.922041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":59,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.604566+0000","last_change":"2024-09-16T09:25:03.706980+0000","last_active":"2024-09-16T09:25:44.604566+0000","last_peered":"2024-09-16T09:25:44.604566+0000","last_clean":"2024-09-16T09:25:44.604566+0000","last_became_active":"2024-09-16T09:25:00.696237+0000","last_became_peered":"2024-09-16T09:25:00.696237+0000","last_unstale":"2024-09-16T09:25:44.604566+0000","last_undegraded":"2024-09-16T09:25:44.604566+0000","last_fullsized":"2024-09-16T09:25:44.604566+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:24:59.684950+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:24.893358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025137430000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":42,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.084795+0000","last_change":"2024-09-16T09:25:05.718319+0000","last_active":"2024-09-16T09:25:44.084795+0000","last_peered":"2024-09-16T09:25:44.084795+0000","last_clean":"2024-09-16T09:25:44.084795+0000","last_became_active":"2024-09-16T09:25:05.717591+0000","last_became_peered":"2024-09-16T09:25:05.717591+0000","last_unstale":"2024-09-16T09:25:44.084795+0000","last_undegraded":"2024-09-16T09:25:44.084795+0000","last_fullsized":"2024-09-16T09:25:44.084795+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:32.732085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":38,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.114938+0000","last_change":"2024-09-16T09:25:07.717720+0000","last_active":"2024-09-16T09:25:44.114938+0000","last_peered":"2024-09-16T09:25:44.114938+0000","last_clean":"2024-09-16T09:25:44.114938+0000","last_became_active":"2024-09-16T09:25:07.717436+0000","last_became_peered":"2024-09-16T09:25:07.717436+0000","last_unstale":"2024-09-16T09:25:44.114938+0000","last_undegraded":"2024-09-16T09:25:44.114938+0000","last_fullsized":"2024-09-16T09:25:44.114938+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:16.230936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"66'13","reported_seq":73,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.086289+0000","last_change":"2024-09-16T09:25:03.713631+0000","last_active":"2024-09-16T09:25:44.086289+0000","last_peered":"2024-09-16T09:25:44.086289+0000","last_clean":"2024-09-16T09:25:44.086289+0000","last_became_active":"2024-09-16T09:25:03.713241+0000","last_became_peered":"2024-09-16T09:25:03.713241+0000","last_unstale":"2024-09-16T09:25:44.086289+0000","last_undegraded":"2024-09-16T09:25:44.086289+0000","last_fullsized":"2024-09-16T09:25:44.086289+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:19.940601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":50,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.085322+0000","last_change":"2024-09-16T09:25:01.707526+0000","last_active":"2024-09-16T09:25:44.085322+0000","last_peered":"2024-09-16T09:25:44.085322+0000","last_clean":"2024-09-16T09:25:44.085322+0000","last_became_active":"2024-09-16T09:25:01.706828+0000","last_became_peered":"2024-09-16T09:25:01.706828+0000","last_unstale":"2024-09-16T09:25:44.085322+0000","last_undegraded":"2024-09-16T09:25:44.085322+0000","last_fullsized":"2024-09-16T09:25:44.085322+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:17.040873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":59,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.604082+0000","last_change":"2024-09-16T09:25:03.702772+0000","last_active":"2024-09-16T09:25:44.604082+0000","last_peered":"2024-09-16T09:25:44.604082+0000","last_clean":"2024-09-16T09:25:44.604082+0000","last_became_active":"2024-09-16T09:25:00.697822+0000","last_became_peered":"2024-09-16T09:25:00.697822+0000","last_unstale":"2024-09-16T09:25:44.604082+0000","last_undegraded":"2024-09-16T09:25:44.604082+0000","last_fullsized":"2024-09-16T09:25:44.604082+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:24:59.684950+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:05.731438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055022299999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":42,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.604640+0000","last_change":"2024-09-16T09:25:05.711840+0000","last_active":"2024-09-16T09:25:44.604640+0000","last_peered":"2024-09-16T09:25:44.604640+0000","last_clean":"2024-09-16T09:25:44.604640+0000","last_became_active":"2024-09-16T09:25:05.711539+0000","last_became_peered":"2024-09-16T09:25:05.711539+0000","last_unstale":"2024-09-16T09:25:44.604640+0000","last_undegraded":"2024-09-16T09:25:44.604640+0000","last_fullsized":"2024-09-16T09:25:44.604640+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:28.306667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":37,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.605681+0000","last_change":"2024-09-16T09:25:07.723554+0000","last_active":"2024-09-16T09:25:44.605681+0000","last_peered":"2024-09-16T09:25:44.605681+0000","last_clean":"2024-09-16T09:25:44.605681+0000","last_became_active":"2024-09-16T09:25:07.723282+0000","last_became_peered":"2024-09-16T09:25:07.723282+0000","last_unstale":"2024-09-16T09:25:44.605681+0000","last_undegraded":"2024-09-16T09:25:44.605681+0000","last_fullsized":"2024-09-16T09:25:44.605681+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:28.111790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"66'12","reported_seq":64,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.085547+0000","last_change":"2024-09-16T09:25:03.715612+0000","last_active":"2024-09-16T09:25:44.085547+0000","last_peered":"2024-09-16T09:25:44.085547+0000","last_clean":"2024-09-16T09:25:44.085547+0000","last_became_active":"2024-09-16T09:25:03.715315+0000","last_became_peered":"2024-09-16T09:25:03.715315+0000","last_unstale":"2024-09-16T09:25:44.085547+0000","last_undegraded":"2024-09-16T09:25:44.085547+0000","last_fullsized":"2024-09-16T09:25:44.085547+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:31.110979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"59'1","reported_seq":56,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.084569+0000","last_change":"2024-09-16T09:25:01.701885+0000","last_active":"2024-09-16T09:25:44.084569+0000","last_peered":"2024-09-16T09:25:44.084569+0000","last_clean":"2024-09-16T09:25:44.084569+0000","last_became_active":"2024-09-16T09:25:01.701599+0000","last_became_peered":"2024-09-16T09:25:01.701599+0000","last_unstale":"2024-09-16T09:25:44.084569+0000","last_undegraded":"2024-09-16T09:25:44.084569+0000","last_fullsized":"2024-09-16T09:25:44.084569+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:03.521969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"67'5","reported_seq":105,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:50.515226+0000","last_change":"2024-09-16T09:25:03.711738+0000","last_active":"2024-09-16T09:25:50.515226+0000","last_peered":"2024-09-16T09:25:50.515226+0000","last_clean":"2024-09-16T09:25:50.515226+0000","last_became_active":"2024-09-16T09:25:00.696546+0000","last_became_peered":"2024-09-16T09:25:00.696546+0000","last_unstale":"2024-09-16T09:25:50.515226+0000","last_undegraded":"2024-09-16T09:25:50.515226+0000","last_fullsized":"2024-09-16T09:25:50.515226+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:24:59.684950+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:17.241966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0059770420000000001,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":45,"num_read_kb":40,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":41,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.605166+0000","last_change":"2024-09-16T09:25:05.714994+0000","last_active":"2024-09-16T09:25:44.605166+0000","last_peered":"2024-09-16T09:25:44.605166+0000","last_clean":"2024-09-16T09:25:44.605166+0000","last_became_active":"2024-09-16T09:25:05.714704+0000","last_became_peered":"2024-09-16T09:25:05.714704+0000","last_unstale":"2024-09-16T09:25:44.605166+0000","last_undegraded":"2024-09-16T09:25:44.605166+0000","last_fullsized":"2024-09-16T09:25:44.605166+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:32.353177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":38,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.085459+0000","last_change":"2024-09-16T09:25:07.714719+0000","last_active":"2024-09-16T09:25:44.085459+0000","last_peered":"2024-09-16T09:25:44.085459+0000","last_clean":"2024-09-16T09:25:44.085459+0000","last_became_active":"2024-09-16T09:25:07.714158+0000","last_became_peered":"2024-09-16T09:25:07.714158+0000","last_unstale":"2024-09-16T09:25:44.085459+0000","last_undegraded":"2024-09-16T09:25:44.085459+0000","last_fullsized":"2024-09-16T09:25:44.085459+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:13.991008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"66'16","reported_seq":79,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:46.757937+0000","last_change":"2024-09-16T09:25:03.714405+0000","last_active":"2024-09-16T09:25:46.757937+0000","last_peered":"2024-09-16T09:25:46.757937+0000","last_clean":"2024-09-16T09:25:46.757937+0000","last_became_active":"2024-09-16T09:25:03.714118+0000","last_became_peered":"2024-09-16T09:25:03.714118+0000","last_unstale":"2024-09-16T09:25:46.757937+0000","last_undegraded":"2024-09-16T09:25:46.757937+0000","last_fullsized":"2024-09-16T09:25:46.757937+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:12.074949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":50,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.114649+0000","last_change":"2024-09-16T09:25:01.704246+0000","last_active":"2024-09-16T09:25:44.114649+0000","last_peered":"2024-09-16T09:25:44.114649+0000","last_clean":"2024-09-16T09:25:44.114649+0000","last_became_active":"2024-09-16T09:25:01.703730+0000","last_became_peered":"2024-09-16T09:25:01.703730+0000","last_unstale":"2024-09-16T09:25:44.114649+0000","last_undegraded":"2024-09-16T09:25:44.114649+0000","last_fullsized":"2024-09-16T09:25:44.114649+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:20.523303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"73'79","reported_seq":87,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.604453+0000","last_change":"2024-09-16T09:24:53.142015+0000","last_active":"2024-09-16T09:25:44.604453+0000","last_peered":"2024-09-16T09:25:44.604453+0000","last_clean":"2024-09-16T09:25:44.604453+0000","last_became_active":"2024-09-16T09:24:53.135890+0000","last_became_peered":"2024-09-16T09:24:53.135890+0000","last_unstale":"2024-09-16T09:25:44.604453+0000","last_undegraded":"2024-09-16T09:25:44.604453+0000","last_fullsized":"2024-09-16T09:25:44.604453+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"20'72","last_scrub_stamp":"2024-09-16T09:24:45.089989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:24:45.089989+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:10.885850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":41,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.605511+0000","last_change":"2024-09-16T09:25:05.714741+0000","last_active":"2024-09-16T09:25:44.605511+0000","last_peered":"2024-09-16T09:25:44.605511+0000","last_clean":"2024-09-16T09:25:44.605511+0000","last_became_active":"2024-09-16T09:25:05.714341+0000","last_became_peered":"2024-09-16T09:25:05.714341+0000","last_unstale":"2024-09-16T09:25:44.605511+0000","last_undegraded":"2024-09-16T09:25:44.605511+0000","last_fullsized":"2024-09-16T09:25:44.605511+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:21.685216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":37,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.605054+0000","last_change":"2024-09-16T09:25:07.714544+0000","last_active":"2024-09-16T09:25:44.605054+0000","last_peered":"2024-09-16T09:25:44.605054+0000","last_clean":"2024-09-16T09:25:44.605054+0000","last_became_active":"2024-09-16T09:25:07.714281+0000","last_became_peered":"2024-09-16T09:25:07.714281+0000","last_unstale":"2024-09-16T09:25:44.605054+0000","last_undegraded":"2024-09-16T09:25:44.605054+0000","last_fullsized":"2024-09-16T09:25:44.605054+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:13.491593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"66'11","reported_seq":65,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.085593+0000","last_change":"2024-09-16T09:25:03.715654+0000","last_active":"2024-09-16T09:25:44.085593+0000","last_peered":"2024-09-16T09:25:44.085593+0000","last_clean":"2024-09-16T09:25:44.085593+0000","last_became_active":"2024-09-16T09:25:03.715373+0000","last_became_peered":"2024-09-16T09:25:03.715373+0000","last_unstale":"2024-09-16T09:25:44.085593+0000","last_undegraded":"2024-09-16T09:25:44.085593+0000","last_fullsized":"2024-09-16T09:25:44.085593+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:11.446460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":50,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.085327+0000","last_change":"2024-09-16T09:25:01.708568+0000","last_active":"2024-09-16T09:25:44.085327+0000","last_peered":"2024-09-16T09:25:44.085327+0000","last_clean":"2024-09-16T09:25:44.085327+0000","last_became_active":"2024-09-16T09:25:01.708270+0000","last_became_peered":"2024-09-16T09:25:01.708270+0000","last_unstale":"2024-09-16T09:25:44.085327+0000","last_undegraded":"2024-09-16T09:25:44.085327+0000","last_fullsized":"2024-09-16T09:25:44.085327+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:24.212331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":41,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.605023+0000","last_change":"2024-09-16T09:25:05.712303+0000","last_active":"2024-09-16T09:25:44.605023+0000","last_peered":"2024-09-16T09:25:44.605023+0000","last_clean":"2024-09-16T09:25:44.605023+0000","last_became_active":"2024-09-16T09:25:05.711964+0000","last_became_peered":"2024-09-16T09:25:05.711964+0000","last_unstale":"2024-09-16T09:25:44.605023+0000","last_undegraded":"2024-09-16T09:25:44.605023+0000","last_fullsized":"2024-09-16T09:25:44.605023+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:23.392105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":38,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.114673+0000","last_change":"2024-09-16T09:25:07.716780+0000","last_active":"2024-09-16T09:25:44.114673+0000","last_peered":"2024-09-16T09:25:44.114673+0000","last_clean":"2024-09-16T09:25:44.114673+0000","last_became_active":"2024-09-16T09:25:07.716514+0000","last_became_peered":"2024-09-16T09:25:07.716514+0000","last_unstale":"2024-09-16T09:25:44.114673+0000","last_undegraded":"2024-09-16T09:25:44.114673+0000","last_fullsized":"2024-09-16T09:25:44.114673+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:07.166770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"66'17","reported_seq":74,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.085933+0000","last_change":"2024-09-16T09:25:03.713691+0000","last_active":"2024-09-16T09:25:44.085933+0000","last_peered":"2024-09-16T09:25:44.085933+0000","last_clean":"2024-09-16T09:25:44.085933+0000","last_became_active":"2024-09-16T09:25:03.713358+0000","last_became_peered":"2024-09-16T09:25:03.713358+0000","last_unstale":"2024-09-16T09:25:44.085933+0000","last_undegraded":"2024-09-16T09:25:44.085933+0000","last_fullsized":"2024-09-16T09:25:44.085933+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:11.619625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":50,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.302717+0000","last_change":"2024-09-16T09:25:01.703939+0000","last_active":"2024-09-16T09:25:44.302717+0000","last_peered":"2024-09-16T09:25:44.302717+0000","last_clean":"2024-09-16T09:25:44.302717+0000","last_became_active":"2024-09-16T09:25:01.703355+0000","last_became_peered":"2024-09-16T09:25:01.703355+0000","last_unstale":"2024-09-16T09:25:44.302717+0000","last_undegraded":"2024-09-16T09:25:44.302717+0000","last_fullsized":"2024-09-16T09:25:44.302717+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:03.217835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":42,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.085545+0000","last_change":"2024-09-16T09:25:05.717224+0000","last_active":"2024-09-16T09:25:44.085545+0000","last_peered":"2024-09-16T09:25:44.085545+0000","last_clean":"2024-09-16T09:25:44.085545+0000","last_became_active":"2024-09-16T09:25:05.716900+0000","last_became_peered":"2024-09-16T09:25:05.716900+0000","last_unstale":"2024-09-16T09:25:44.085545+0000","last_undegraded":"2024-09-16T09:25:44.085545+0000","last_fullsized":"2024-09-16T09:25:44.085545+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:30.026527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":38,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.604786+0000","last_change":"2024-09-16T09:25:07.721657+0000","last_active":"2024-09-16T09:25:44.604786+0000","last_peered":"2024-09-16T09:25:44.604786+0000","last_clean":"2024-09-16T09:25:44.604786+0000","last_became_active":"2024-09-16T09:25:07.721347+0000","last_became_peered":"2024-09-16T09:25:07.721347+0000","last_unstale":"2024-09-16T09:25:44.604786+0000","last_undegraded":"2024-09-16T09:25:44.604786+0000","last_fullsized":"2024-09-16T09:25:44.604786+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:15.094644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"66'10","reported_seq":61,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.085245+0000","last_change":"2024-09-16T09:25:03.708144+0000","last_active":"2024-09-16T09:25:44.085245+0000","last_peered":"2024-09-16T09:25:44.085245+0000","last_clean":"2024-09-16T09:25:44.085245+0000","last_became_active":"2024-09-16T09:25:03.707723+0000","last_became_peered":"2024-09-16T09:25:03.707723+0000","last_unstale":"2024-09-16T09:25:44.085245+0000","last_undegraded":"2024-09-16T09:25:44.085245+0000","last_fullsized":"2024-09-16T09:25:44.085245+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:24.890527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":50,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.114760+0000","last_change":"2024-09-16T09:25:01.703388+0000","last_active":"2024-09-16T09:25:44.114760+0000","last_peered":"2024-09-16T09:25:44.114760+0000","last_clean":"2024-09-16T09:25:44.114760+0000","last_became_active":"2024-09-16T09:25:01.703094+0000","last_became_peered":"2024-09-16T09:25:01.703094+0000","last_unstale":"2024-09-16T09:25:44.114760+0000","last_undegraded":"2024-09-16T09:25:44.114760+0000","last_fullsized":"2024-09-16T09:25:44.114760+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:07.049694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":64,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:46.757815+0000","last_change":"2024-09-16T09:25:05.711891+0000","last_active":"2024-09-16T09:25:46.757815+0000","last_peered":"2024-09-16T09:25:46.757815+0000","last_clean":"2024-09-16T09:25:46.757815+0000","last_became_active":"2024-09-16T09:25:05.711590+0000","last_became_peered":"2024-09-16T09:25:05.711590+0000","last_unstale":"2024-09-16T09:25:46.757815+0000","last_undegraded":"2024-09-16T09:25:46.757815+0000","last_fullsized":"2024-09-16T09:25:46.757815+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:17.630914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":38,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.085382+0000","last_change":"2024-09-16T09:25:07.719496+0000","last_active":"2024-09-16T09:25:44.085382+0000","last_peered":"2024-09-16T09:25:44.085382+0000","last_clean":"2024-09-16T09:25:44.085382+0000","last_became_active":"2024-09-16T09:25:07.718958+0000","last_became_peered":"2024-09-16T09:25:07.718958+0000","last_unstale":"2024-09-16T09:25:44.085382+0000","last_undegraded":"2024-09-16T09:25:44.085382+0000","last_fullsized":"2024-09-16T09:25:44.085382+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:15.393455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.b","version":"66'9","reported_seq":62,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.084946+0000","last_change":"2024-09-16T09:25:03.712553+0000","last_active":"2024-09-16T09:25:44.084946+0000","last_peered":"2024-09-16T09:25:44.084946+0000","last_clean":"2024-09-16T09:25:44.084946+0000","last_became_active":"2024-09-16T09:25:03.712230+0000","last_became_peered":"2024-09-16T09:25:03.712230+0000","last_unstale":"2024-09-16T09:25:44.084946+0000","last_undegraded":"2024-09-16T09:25:44.084946+0000","last_fullsized":"2024-09-16T09:25:44.084946+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:18.262732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":49,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.604847+0000","last_change":"2024-09-16T09:25:01.702943+0000","last_active":"2024-09-16T09:25:44.604847+0000","last_peered":"2024-09-16T09:25:44.604847+0000","last_clean":"2024-09-16T09:25:44.604847+0000","last_became_active":"2024-09-16T09:25:01.702629+0000","last_became_peered":"2024-09-16T09:25:01.702629+0000","last_unstale":"2024-09-16T09:25:44.604847+0000","last_undegraded":"2024-09-16T09:25:44.604847+0000","last_fullsized":"2024-09-16T09:25:44.604847+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:19.763325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":42,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.604193+0000","last_change":"2024-09-16T09:25:05.712932+0000","last_active":"2024-09-16T09:25:44.604193+0000","last_peered":"2024-09-16T09:25:44.604193+0000","last_clean":"2024-09-16T09:25:44.604193+0000","last_became_active":"2024-09-16T09:25:05.712573+0000","last_became_peered":"2024-09-16T09:25:05.712573+0000","last_unstale":"2024-09-16T09:25:44.604193+0000","last_undegraded":"2024-09-16T09:25:44.604193+0000","last_fullsized":"2024-09-16T09:25:44.604193+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:16.863815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":38,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.084536+0000","last_change":"2024-09-16T09:25:07.719073+0000","last_active":"2024-09-16T09:25:44.084536+0000","last_peered":"2024-09-16T09:25:44.084536+0000","last_clean":"2024-09-16T09:25:44.084536+0000","last_became_active":"2024-09-16T09:25:07.718778+0000","last_became_peered":"2024-09-16T09:25:07.718778+0000","last_unstale":"2024-09-16T09:25:44.084536+0000","last_undegraded":"2024-09-16T09:25:44.084536+0000","last_fullsized":"2024-09-16T09:25:44.084536+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:21.733588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"66'19","reported_seq":77,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.302519+0000","last_change":"2024-09-16T09:25:03.713182+0000","last_active":"2024-09-16T09:25:44.302519+0000","last_peered":"2024-09-16T09:25:44.302519+0000","last_clean":"2024-09-16T09:25:44.302519+0000","last_became_active":"2024-09-16T09:25:03.712783+0000","last_became_peered":"2024-09-16T09:25:03.712783+0000","last_unstale":"2024-09-16T09:25:44.302519+0000","last_undegraded":"2024-09-16T09:25:44.302519+0000","last_fullsized":"2024-09-16T09:25:44.302519+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:24.025685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"59'1","reported_seq":55,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.605881+0000","last_change":"2024-09-16T09:25:01.708571+0000","last_active":"2024-09-16T09:25:44.605881+0000","last_peered":"2024-09-16T09:25:44.605881+0000","last_clean":"2024-09-16T09:25:44.605881+0000","last_became_active":"2024-09-16T09:25:01.708069+0000","last_became_peered":"2024-09-16T09:25:01.708069+0000","last_unstale":"2024-09-16T09:25:44.605881+0000","last_undegraded":"2024-09-16T09:25:44.605881+0000","last_fullsized":"2024-09-16T09:25:44.605881+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:06.099334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":42,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.604167+0000","last_change":"2024-09-16T09:25:05.709549+0000","last_active":"2024-09-16T09:25:44.604167+0000","last_peered":"2024-09-16T09:25:44.604167+0000","last_clean":"2024-09-16T09:25:44.604167+0000","last_became_active":"2024-09-16T09:25:05.709264+0000","last_became_peered":"2024-09-16T09:25:05.709264+0000","last_unstale":"2024-09-16T09:25:44.604167+0000","last_undegraded":"2024-09-16T09:25:44.604167+0000","last_fullsized":"2024-09-16T09:25:44.604167+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:10.529230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":37,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.604361+0000","last_change":"2024-09-16T09:25:07.719009+0000","last_active":"2024-09-16T09:25:44.604361+0000","last_peered":"2024-09-16T09:25:44.604361+0000","last_clean":"2024-09-16T09:25:44.604361+0000","last_became_active":"2024-09-16T09:25:07.718750+0000","last_became_peered":"2024-09-16T09:25:07.718750+0000","last_unstale":"2024-09-16T09:25:44.604361+0000","last_undegraded":"2024-09-16T09:25:44.604361+0000","last_fullsized":"2024-09-16T09:25:44.604361+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:07.594257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"66'12","reported_seq":69,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.085147+0000","last_change":"2024-09-16T09:25:03.714309+0000","last_active":"2024-09-16T09:25:44.085147+0000","last_peered":"2024-09-16T09:25:44.085147+0000","last_clean":"2024-09-16T09:25:44.085147+0000","last_became_active":"2024-09-16T09:25:03.714027+0000","last_became_peered":"2024-09-16T09:25:03.714027+0000","last_unstale":"2024-09-16T09:25:44.085147+0000","last_undegraded":"2024-09-16T09:25:44.085147+0000","last_fullsized":"2024-09-16T09:25:44.085147+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:31.556984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":49,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.605893+0000","last_change":"2024-09-16T09:25:01.707905+0000","last_active":"2024-09-16T09:25:44.605893+0000","last_peered":"2024-09-16T09:25:44.605893+0000","last_clean":"2024-09-16T09:25:44.605893+0000","last_became_active":"2024-09-16T09:25:01.707528+0000","last_became_peered":"2024-09-16T09:25:01.707528+0000","last_unstale":"2024-09-16T09:25:44.605893+0000","last_undegraded":"2024-09-16T09:25:44.605893+0000","last_fullsized":"2024-09-16T09:25:44.605893+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:13.046169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":42,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.603975+0000","last_change":"2024-09-16T09:25:05.715584+0000","last_active":"2024-09-16T09:25:44.603975+0000","last_peered":"2024-09-16T09:25:44.603975+0000","last_clean":"2024-09-16T09:25:44.603975+0000","last_became_active":"2024-09-16T09:25:05.715253+0000","last_became_peered":"2024-09-16T09:25:05.715253+0000","last_unstale":"2024-09-16T09:25:44.603975+0000","last_undegraded":"2024-09-16T09:25:44.603975+0000","last_fullsized":"2024-09-16T09:25:44.603975+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:23.326757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":38,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.114558+0000","last_change":"2024-09-16T09:25:07.719338+0000","last_active":"2024-09-16T09:25:44.114558+0000","last_peered":"2024-09-16T09:25:44.114558+0000","last_clean":"2024-09-16T09:25:44.114558+0000","last_became_active":"2024-09-16T09:25:07.718899+0000","last_became_peered":"2024-09-16T09:25:07.718899+0000","last_unstale":"2024-09-16T09:25:44.114558+0000","last_undegraded":"2024-09-16T09:25:44.114558+0000","last_fullsized":"2024-09-16T09:25:44.114558+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:23.435495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.8","version":"66'15","reported_seq":70,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.603974+0000","last_change":"2024-09-16T09:25:03.716075+0000","last_active":"2024-09-16T09:25:44.603974+0000","last_peered":"2024-09-16T09:25:44.603974+0000","last_clean":"2024-09-16T09:25:44.603974+0000","last_became_active":"2024-09-16T09:25:03.715733+0000","last_became_peered":"2024-09-16T09:25:03.715733+0000","last_unstale":"2024-09-16T09:25:44.603974+0000","last_undegraded":"2024-09-16T09:25:44.603974+0000","last_fullsized":"2024-09-16T09:25:44.603974+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:20.577367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":61,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.605970+0000","last_change":"2024-09-16T09:25:01.707886+0000","last_active":"2024-09-16T09:25:44.605970+0000","last_peered":"2024-09-16T09:25:44.605970+0000","last_clean":"2024-09-16T09:25:44.605970+0000","last_became_active":"2024-09-16T09:25:01.707372+0000","last_became_peered":"2024-09-16T09:25:01.707372+0000","last_unstale":"2024-09-16T09:25:44.605970+0000","last_undegraded":"2024-09-16T09:25:44.605970+0000","last_fullsized":"2024-09-16T09:25:44.605970+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:14.328776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"66'8","reported_seq":60,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:46.757960+0000","last_change":"2024-09-16T09:25:05.716778+0000","last_active":"2024-09-16T09:25:46.757960+0000","last_peered":"2024-09-16T09:25:46.757960+0000","last_clean":"2024-09-16T09:25:46.757960+0000","last_became_active":"2024-09-16T09:25:05.716189+0000","last_became_peered":"2024-09-16T09:25:05.716189+0000","last_unstale":"2024-09-16T09:25:46.757960+0000","last_undegraded":"2024-09-16T09:25:46.757960+0000","last_fullsized":"2024-09-16T09:25:46.757960+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:31.128760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":37,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.604311+0000","last_change":"2024-09-16T09:25:07.717505+0000","last_active":"2024-09-16T09:25:44.604311+0000","last_peered":"2024-09-16T09:25:44.604311+0000","last_clean":"2024-09-16T09:25:44.604311+0000","last_became_active":"2024-09-16T09:25:07.717213+0000","last_became_peered":"2024-09-16T09:25:07.717213+0000","last_unstale":"2024-09-16T09:25:44.604311+0000","last_undegraded":"2024-09-16T09:25:44.604311+0000","last_fullsized":"2024-09-16T09:25:44.604311+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:09.355594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":50,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.302519+0000","last_change":"2024-09-16T09:25:01.703489+0000","last_active":"2024-09-16T09:25:44.302519+0000","last_peered":"2024-09-16T09:25:44.302519+0000","last_clean":"2024-09-16T09:25:44.302519+0000","last_became_active":"2024-09-16T09:25:01.703202+0000","last_became_peered":"2024-09-16T09:25:01.703202+0000","last_unstale":"2024-09-16T09:25:44.302519+0000","last_undegraded":"2024-09-16T09:25:44.302519+0000","last_fullsized":"2024-09-16T09:25:44.302519+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:30.581098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"66'6","reported_seq":55,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.115141+0000","last_change":"2024-09-16T09:25:03.713606+0000","last_active":"2024-09-16T09:25:44.115141+0000","last_peered":"2024-09-16T09:25:44.115141+0000","last_clean":"2024-09-16T09:25:44.115141+0000","last_became_active":"2024-09-16T09:25:03.713245+0000","last_became_peered":"2024-09-16T09:25:03.713245+0000","last_unstale":"2024-09-16T09:25:44.115141+0000","last_undegraded":"2024-09-16T09:25:44.115141+0000","last_fullsized":"2024-09-16T09:25:44.115141+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:15.800169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":41,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.604861+0000","last_change":"2024-09-16T09:25:05.715938+0000","last_active":"2024-09-16T09:25:44.604861+0000","last_peered":"2024-09-16T09:25:44.604861+0000","last_clean":"2024-09-16T09:25:44.604861+0000","last_became_active":"2024-09-16T09:25:05.715616+0000","last_became_peered":"2024-09-16T09:25:05.715616+0000","last_unstale":"2024-09-16T09:25:44.604861+0000","last_undegraded":"2024-09-16T09:25:44.604861+0000","last_fullsized":"2024-09-16T09:25:44.604861+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:25.224184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":37,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.605446+0000","last_change":"2024-09-16T09:25:07.721375+0000","last_active":"2024-09-16T09:25:44.605446+0000","last_peered":"2024-09-16T09:25:44.605446+0000","last_clean":"2024-09-16T09:25:44.605446+0000","last_became_active":"2024-09-16T09:25:07.721059+0000","last_became_peered":"2024-09-16T09:25:07.721059+0000","last_unstale":"2024-09-16T09:25:44.605446+0000","last_undegraded":"2024-09-16T09:25:44.605446+0000","last_fullsized":"2024-09-16T09:25:44.605446+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:16.275710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"66'9","reported_seq":62,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.085632+0000","last_change":"2024-09-16T09:25:03.719919+0000","last_active":"2024-09-16T09:25:44.085632+0000","last_peered":"2024-09-16T09:25:44.085632+0000","last_clean":"2024-09-16T09:25:44.085632+0000","last_became_active":"2024-09-16T09:25:03.719573+0000","last_became_peered":"2024-09-16T09:25:03.719573+0000","last_unstale":"2024-09-16T09:25:44.085632+0000","last_undegraded":"2024-09-16T09:25:44.085632+0000","last_fullsized":"2024-09-16T09:25:44.085632+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:23.239324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":49,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.605148+0000","last_change":"2024-09-16T09:25:01.707139+0000","last_active":"2024-09-16T09:25:44.605148+0000","last_peered":"2024-09-16T09:25:44.605148+0000","last_clean":"2024-09-16T09:25:44.605148+0000","last_became_active":"2024-09-16T09:25:01.706491+0000","last_became_peered":"2024-09-16T09:25:01.706491+0000","last_unstale":"2024-09-16T09:25:44.605148+0000","last_undegraded":"2024-09-16T09:25:44.605148+0000","last_fullsized":"2024-09-16T09:25:44.605148+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:07.438370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":42,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.115051+0000","last_change":"2024-09-16T09:25:05.715083+0000","last_active":"2024-09-16T09:25:44.115051+0000","last_peered":"2024-09-16T09:25:44.115051+0000","last_clean":"2024-09-16T09:25:44.115051+0000","last_became_active":"2024-09-16T09:25:05.714801+0000","last_became_peered":"2024-09-16T09:25:05.714801+0000","last_unstale":"2024-09-16T09:25:44.115051+0000","last_undegraded":"2024-09-16T09:25:44.115051+0000","last_fullsized":"2024-09-16T09:25:44.115051+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:15.084815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":38,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.604634+0000","last_change":"2024-09-16T09:25:07.722356+0000","last_active":"2024-09-16T09:25:44.604634+0000","last_peered":"2024-09-16T09:25:44.604634+0000","last_clean":"2024-09-16T09:25:44.604634+0000","last_became_active":"2024-09-16T09:25:07.722050+0000","last_became_peered":"2024-09-16T09:25:07.722050+0000","last_unstale":"2024-09-16T09:25:44.604634+0000","last_undegraded":"2024-09-16T09:25:44.604634+0000","last_fullsized":"2024-09-16T09:25:44.604634+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:35.793652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"66'9","reported_seq":61,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.605167+0000","last_change":"2024-09-16T09:25:03.717609+0000","last_active":"2024-09-16T09:25:44.605167+0000","last_peered":"2024-09-16T09:25:44.605167+0000","last_clean":"2024-09-16T09:25:44.605167+0000","last_became_active":"2024-09-16T09:25:03.717310+0000","last_became_peered":"2024-09-16T09:25:03.717310+0000","last_unstale":"2024-09-16T09:25:44.605167+0000","last_undegraded":"2024-09-16T09:25:44.605167+0000","last_fullsized":"2024-09-16T09:25:44.605167+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:27.281102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":50,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.084737+0000","last_change":"2024-09-16T09:25:01.702732+0000","last_active":"2024-09-16T09:25:44.084737+0000","last_peered":"2024-09-16T09:25:44.084737+0000","last_clean":"2024-09-16T09:25:44.084737+0000","last_became_active":"2024-09-16T09:25:01.702419+0000","last_became_peered":"2024-09-16T09:25:01.702419+0000","last_unstale":"2024-09-16T09:25:44.084737+0000","last_undegraded":"2024-09-16T09:25:44.084737+0000","last_fullsized":"2024-09-16T09:25:44.084737+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:11.390955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"66'8","reported_seq":61,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:46.758117+0000","last_change":"2024-09-16T09:25:05.711929+0000","last_active":"2024-09-16T09:25:46.758117+0000","last_peered":"2024-09-16T09:25:46.758117+0000","last_clean":"2024-09-16T09:25:46.758117+0000","last_became_active":"2024-09-16T09:25:05.711642+0000","last_became_peered":"2024-09-16T09:25:05.711642+0000","last_unstale":"2024-09-16T09:25:46.758117+0000","last_undegraded":"2024-09-16T09:25:46.758117+0000","last_fullsized":"2024-09-16T09:25:46.758117+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:29.307023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":38,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.085592+0000","last_change":"2024-09-16T09:25:07.719123+0000","last_active":"2024-09-16T09:25:44.085592+0000","last_peered":"2024-09-16T09:25:44.085592+0000","last_clean":"2024-09-16T09:25:44.085592+0000","last_became_active":"2024-09-16T09:25:07.718810+0000","last_became_peered":"2024-09-16T09:25:07.718810+0000","last_unstale":"2024-09-16T09:25:44.085592+0000","last_undegraded":"2024-09-16T09:25:44.085592+0000","last_fullsized":"2024-09-16T09:25:44.085592+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:16.185390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"66'10","reported_seq":61,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.114879+0000","last_change":"2024-09-16T09:25:03.712705+0000","last_active":"2024-09-16T09:25:44.114879+0000","last_peered":"2024-09-16T09:25:44.114879+0000","last_clean":"2024-09-16T09:25:44.114879+0000","last_became_active":"2024-09-16T09:25:03.712387+0000","last_became_peered":"2024-09-16T09:25:03.712387+0000","last_unstale":"2024-09-16T09:25:44.114879+0000","last_undegraded":"2024-09-16T09:25:44.114879+0000","last_fullsized":"2024-09-16T09:25:44.114879+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:26.552452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":49,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.605441+0000","last_change":"2024-09-16T09:25:01.708461+0000","last_active":"2024-09-16T09:25:44.605441+0000","last_peered":"2024-09-16T09:25:44.605441+0000","last_clean":"2024-09-16T09:25:44.605441+0000","last_became_active":"2024-09-16T09:25:01.707513+0000","last_became_peered":"2024-09-16T09:25:01.707513+0000","last_unstale":"2024-09-16T09:25:44.605441+0000","last_undegraded":"2024-09-16T09:25:44.605441+0000","last_fullsized":"2024-09-16T09:25:44.605441+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:10.338638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"66'8","reported_seq":60,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:46.758166+0000","last_change":"2024-09-16T09:25:05.714214+0000","last_active":"2024-09-16T09:25:46.758166+0000","last_peered":"2024-09-16T09:25:46.758166+0000","last_clean":"2024-09-16T09:25:46.758166+0000","last_became_active":"2024-09-16T09:25:05.713833+0000","last_became_peered":"2024-09-16T09:25:05.713833+0000","last_unstale":"2024-09-16T09:25:46.758166+0000","last_undegraded":"2024-09-16T09:25:46.758166+0000","last_fullsized":"2024-09-16T09:25:46.758166+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:09.052011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":38,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.085233+0000","last_change":"2024-09-16T09:25:07.717629+0000","last_active":"2024-09-16T09:25:44.085233+0000","last_peered":"2024-09-16T09:25:44.085233+0000","last_clean":"2024-09-16T09:25:44.085233+0000","last_became_active":"2024-09-16T09:25:07.717357+0000","last_became_peered":"2024-09-16T09:25:07.717357+0000","last_unstale":"2024-09-16T09:25:44.085233+0000","last_undegraded":"2024-09-16T09:25:44.085233+0000","last_fullsized":"2024-09-16T09:25:44.085233+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:28.965425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"66'11","reported_seq":65,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.085362+0000","last_change":"2024-09-16T09:25:03.715601+0000","last_active":"2024-09-16T09:25:44.085362+0000","last_peered":"2024-09-16T09:25:44.085362+0000","last_clean":"2024-09-16T09:25:44.085362+0000","last_became_active":"2024-09-16T09:25:03.715333+0000","last_became_peered":"2024-09-16T09:25:03.715333+0000","last_unstale":"2024-09-16T09:25:44.085362+0000","last_undegraded":"2024-09-16T09:25:44.085362+0000","last_fullsized":"2024-09-16T09:25:44.085362+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:05.154887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":50,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.085723+0000","last_change":"2024-09-16T09:25:01.705459+0000","last_active":"2024-09-16T09:25:44.085723+0000","last_peered":"2024-09-16T09:25:44.085723+0000","last_clean":"2024-09-16T09:25:44.085723+0000","last_became_active":"2024-09-16T09:25:01.705158+0000","last_became_peered":"2024-09-16T09:25:01.705158+0000","last_unstale":"2024-09-16T09:25:44.085723+0000","last_undegraded":"2024-09-16T09:25:44.085723+0000","last_fullsized":"2024-09-16T09:25:44.085723+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:17.236788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":42,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.085997+0000","last_change":"2024-09-16T09:25:05.713087+0000","last_active":"2024-09-16T09:25:44.085997+0000","last_peered":"2024-09-16T09:25:44.085997+0000","last_clean":"2024-09-16T09:25:44.085997+0000","last_became_active":"2024-09-16T09:25:05.712816+0000","last_became_peered":"2024-09-16T09:25:05.712816+0000","last_unstale":"2024-09-16T09:25:44.085997+0000","last_undegraded":"2024-09-16T09:25:44.085997+0000","last_fullsized":"2024-09-16T09:25:44.085997+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:29.267320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":38,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.114352+0000","last_change":"2024-09-16T09:25:07.713617+0000","last_active":"2024-09-16T09:25:44.114352+0000","last_peered":"2024-09-16T09:25:44.114352+0000","last_clean":"2024-09-16T09:25:44.114352+0000","last_became_active":"2024-09-16T09:25:07.713109+0000","last_became_peered":"2024-09-16T09:25:07.713109+0000","last_unstale":"2024-09-16T09:25:44.114352+0000","last_undegraded":"2024-09-16T09:25:44.114352+0000","last_fullsized":"2024-09-16T09:25:44.114352+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:17.988989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"66'9","reported_seq":62,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.086227+0000","last_change":"2024-09-16T09:25:03.706126+0000","last_active":"2024-09-16T09:25:44.086227+0000","last_peered":"2024-09-16T09:25:44.086227+0000","last_clean":"2024-09-16T09:25:44.086227+0000","last_became_active":"2024-09-16T09:25:03.705807+0000","last_became_peered":"2024-09-16T09:25:03.705807+0000","last_unstale":"2024-09-16T09:25:44.086227+0000","last_undegraded":"2024-09-16T09:25:44.086227+0000","last_fullsized":"2024-09-16T09:25:44.086227+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:25.655887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":49,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.605797+0000","last_change":"2024-09-16T09:25:01.707075+0000","last_active":"2024-09-16T09:25:44.605797+0000","last_peered":"2024-09-16T09:25:44.605797+0000","last_clean":"2024-09-16T09:25:44.605797+0000","last_became_active":"2024-09-16T09:25:01.706762+0000","last_became_peered":"2024-09-16T09:25:01.706762+0000","last_unstale":"2024-09-16T09:25:44.605797+0000","last_undegraded":"2024-09-16T09:25:44.605797+0000","last_fullsized":"2024-09-16T09:25:44.605797+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:13.882359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":42,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.114580+0000","last_change":"2024-09-16T09:25:05.714142+0000","last_active":"2024-09-16T09:25:44.114580+0000","last_peered":"2024-09-16T09:25:44.114580+0000","last_clean":"2024-09-16T09:25:44.114580+0000","last_became_active":"2024-09-16T09:25:05.713791+0000","last_became_peered":"2024-09-16T09:25:05.713791+0000","last_unstale":"2024-09-16T09:25:44.114580+0000","last_undegraded":"2024-09-16T09:25:44.114580+0000","last_fullsized":"2024-09-16T09:25:44.114580+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:20.280445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":38,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.084170+0000","last_change":"2024-09-16T09:25:07.715572+0000","last_active":"2024-09-16T09:25:44.084170+0000","last_peered":"2024-09-16T09:25:44.084170+0000","last_clean":"2024-09-16T09:25:44.084170+0000","last_became_active":"2024-09-16T09:25:07.715301+0000","last_became_peered":"2024-09-16T09:25:07.715301+0000","last_unstale":"2024-09-16T09:25:44.084170+0000","last_undegraded":"2024-09-16T09:25:44.084170+0000","last_fullsized":"2024-09-16T09:25:44.084170+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:15.062673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"66'11","reported_seq":65,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.114498+0000","last_change":"2024-09-16T09:25:03.712637+0000","last_active":"2024-09-16T09:25:44.114498+0000","last_peered":"2024-09-16T09:25:44.114498+0000","last_clean":"2024-09-16T09:25:44.114498+0000","last_became_active":"2024-09-16T09:25:03.712333+0000","last_became_peered":"2024-09-16T09:25:03.712333+0000","last_unstale":"2024-09-16T09:25:44.114498+0000","last_undegraded":"2024-09-16T09:25:44.114498+0000","last_fullsized":"2024-09-16T09:25:44.114498+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:06.593958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":49,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.604955+0000","last_change":"2024-09-16T09:25:01.701295+0000","last_active":"2024-09-16T09:25:44.604955+0000","last_peered":"2024-09-16T09:25:44.604955+0000","last_clean":"2024-09-16T09:25:44.604955+0000","last_became_active":"2024-09-16T09:25:01.700733+0000","last_became_peered":"2024-09-16T09:25:01.700733+0000","last_unstale":"2024-09-16T09:25:44.604955+0000","last_undegraded":"2024-09-16T09:25:44.604955+0000","last_fullsized":"2024-09-16T09:25:44.604955+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:14.332112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":41,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.604177+0000","last_change":"2024-09-16T09:25:05.713760+0000","last_active":"2024-09-16T09:25:44.604177+0000","last_peered":"2024-09-16T09:25:44.604177+0000","last_clean":"2024-09-16T09:25:44.604177+0000","last_became_active":"2024-09-16T09:25:05.713479+0000","last_became_peered":"2024-09-16T09:25:05.713479+0000","last_unstale":"2024-09-16T09:25:44.604177+0000","last_undegraded":"2024-09-16T09:25:44.604177+0000","last_fullsized":"2024-09-16T09:25:44.604177+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:31.316771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":38,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.114700+0000","last_change":"2024-09-16T09:25:07.715946+0000","last_active":"2024-09-16T09:25:44.114700+0000","last_peered":"2024-09-16T09:25:44.114700+0000","last_clean":"2024-09-16T09:25:44.114700+0000","last_became_active":"2024-09-16T09:25:07.715582+0000","last_became_peered":"2024-09-16T09:25:07.715582+0000","last_unstale":"2024-09-16T09:25:44.114700+0000","last_undegraded":"2024-09-16T09:25:44.114700+0000","last_fullsized":"2024-09-16T09:25:44.114700+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:18.541254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"66'4","reported_seq":52,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.114329+0000","last_change":"2024-09-16T09:25:03.705167+0000","last_active":"2024-09-16T09:25:44.114329+0000","last_peered":"2024-09-16T09:25:44.114329+0000","last_clean":"2024-09-16T09:25:44.114329+0000","last_became_active":"2024-09-16T09:25:03.704869+0000","last_became_peered":"2024-09-16T09:25:03.704869+0000","last_unstale":"2024-09-16T09:25:44.114329+0000","last_undegraded":"2024-09-16T09:25:44.114329+0000","last_fullsized":"2024-09-16T09:25:44.114329+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:17.043143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":50,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.085135+0000","last_change":"2024-09-16T09:25:01.705965+0000","last_active":"2024-09-16T09:25:44.085135+0000","last_peered":"2024-09-16T09:25:44.085135+0000","last_clean":"2024-09-16T09:25:44.085135+0000","last_became_active":"2024-09-16T09:25:01.705686+0000","last_became_peered":"2024-09-16T09:25:01.705686+0000","last_unstale":"2024-09-16T09:25:44.085135+0000","last_undegraded":"2024-09-16T09:25:44.085135+0000","last_fullsized":"2024-09-16T09:25:44.085135+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:16.022170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":42,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.302482+0000","last_change":"2024-09-16T09:25:05.712816+0000","last_active":"2024-09-16T09:25:44.302482+0000","last_peered":"2024-09-16T09:25:44.302482+0000","last_clean":"2024-09-16T09:25:44.302482+0000","last_became_active":"2024-09-16T09:25:05.712545+0000","last_became_peered":"2024-09-16T09:25:05.712545+0000","last_unstale":"2024-09-16T09:25:44.302482+0000","last_undegraded":"2024-09-16T09:25:44.302482+0000","last_fullsized":"2024-09-16T09:25:44.302482+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:08.170532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":37,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.605000+0000","last_change":"2024-09-16T09:25:07.720368+0000","last_active":"2024-09-16T09:25:44.605000+0000","last_peered":"2024-09-16T09:25:44.605000+0000","last_clean":"2024-09-16T09:25:44.605000+0000","last_became_active":"2024-09-16T09:25:07.720117+0000","last_became_peered":"2024-09-16T09:25:07.720117+0000","last_unstale":"2024-09-16T09:25:44.605000+0000","last_undegraded":"2024-09-16T09:25:44.605000+0000","last_fullsized":"2024-09-16T09:25:44.605000+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:31.504626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":38,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.084855+0000","last_change":"2024-09-16T09:25:07.712880+0000","last_active":"2024-09-16T09:25:44.084855+0000","last_peered":"2024-09-16T09:25:44.084855+0000","last_clean":"2024-09-16T09:25:44.084855+0000","last_became_active":"2024-09-16T09:25:07.712594+0000","last_became_peered":"2024-09-16T09:25:07.712594+0000","last_unstale":"2024-09-16T09:25:44.084855+0000","last_undegraded":"2024-09-16T09:25:44.084855+0000","last_fullsized":"2024-09-16T09:25:44.084855+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:33.865572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":50,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.114495+0000","last_change":"2024-09-16T09:25:01.704711+0000","last_active":"2024-09-16T09:25:44.114495+0000","last_peered":"2024-09-16T09:25:44.114495+0000","last_clean":"2024-09-16T09:25:44.114495+0000","last_became_active":"2024-09-16T09:25:01.703736+0000","last_became_peered":"2024-09-16T09:25:01.703736+0000","last_unstale":"2024-09-16T09:25:44.114495+0000","last_undegraded":"2024-09-16T09:25:44.114495+0000","last_fullsized":"2024-09-16T09:25:44.114495+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:09.913011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"66'11","reported_seq":65,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.303094+0000","last_change":"2024-09-16T09:25:03.708095+0000","last_active":"2024-09-16T09:25:44.303094+0000","last_peered":"2024-09-16T09:25:44.303094+0000","last_clean":"2024-09-16T09:25:44.303094+0000","last_became_active":"2024-09-16T09:25:03.707769+0000","last_became_peered":"2024-09-16T09:25:03.707769+0000","last_unstale":"2024-09-16T09:25:44.303094+0000","last_undegraded":"2024-09-16T09:25:44.303094+0000","last_fullsized":"2024-09-16T09:25:44.303094+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:21.606932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":42,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.085370+0000","last_change":"2024-09-16T09:25:05.712247+0000","last_active":"2024-09-16T09:25:44.085370+0000","last_peered":"2024-09-16T09:25:44.085370+0000","last_clean":"2024-09-16T09:25:44.085370+0000","last_became_active":"2024-09-16T09:25:05.711952+0000","last_became_peered":"2024-09-16T09:25:05.711952+0000","last_unstale":"2024-09-16T09:25:44.085370+0000","last_undegraded":"2024-09-16T09:25:44.085370+0000","last_fullsized":"2024-09-16T09:25:44.085370+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:30.229560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":393,"ondisk_log_size":393,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":45,"num_read_kb":40,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266703,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32632,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712520,"statfs":{"total":95995035648,"available":95961620480,"internally_reserved":0,"allocated":1478656,"data_stored":977101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495636,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28448,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716704,"statfs":{"total":95995035648,"available":95965904896,"internally_reserved":0,"allocated":1454080,"data_stored":976003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724571,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":864256,"data_stored":385994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953504,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":888832,"data_stored":386797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149734,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32064,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713088,"statfs":{"total":95995035648,"available":95962202112,"internally_reserved":0,"allocated":897024,"data_stored":387083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411371,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32020,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713132,"statfs":{"total":95995035648,"available":95962247168,"internally_reserved":0,"allocated":851968,"data_stored":385347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574898,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":925696,"data_stored":386997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738424,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28476,"kb_used_data":1448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716676,"statfs":{"total":95995035648,"available":95965876224,"internally_reserved":0,"allocated":1482752,"data_stored":977367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T09:25:56.214 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-16T09:25:56.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:56 smithi084 ceph-mon[32629]: from='client.24629 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:25:56.415 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:56 smithi070 ceph-mon[30388]: from='client.24629 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:25:56.415 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:56 smithi070 ceph-mon[40691]: from='client.24629 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:25:56.670 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Sep 16 09:25:56 smithi070 podman[75679]: 2024-09-16 09:25:56.317054335 +0000 UTC m=+0.016213265 image pull 11f11916f8cdfb17c1da74b20deb2075d778357b4d60474d82453303f8d461f5 quay.io/prometheus/alertmanager:v0.27.0 2024-09-16T09:25:56.671 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Sep 16 09:25:56 smithi070 podman[75679]: 2024-09-16 09:25:56.448016232 +0000 UTC m=+0.147175199 volume create 84d461f721208ab0f64ce78b1afdfc29060f206216c1c29a66dd70cb1a177c2c 2024-09-16T09:25:56.671 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Sep 16 09:25:56 smithi070 podman[75679]: 2024-09-16 09:25:56.556743284 +0000 UTC m=+0.255902204 container create f7b0530f3dbf8d8acb21cde05413a70b8de3412f7ad11a4fcff8fdf456b8593f (image=quay.io/prometheus/alertmanager:v0.27.0, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-16T09:25:57.055 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:25:57.170 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:57 smithi070 ceph-mon[30388]: pgmap v20: 132 pgs: 132 active+clean; 583 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 898 B/s rd, 0 op/s 2024-09-16T09:25:57.170 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:57 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:25:57.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:57 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:25:57.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:57 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:25:57.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:57 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:25:57.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:57 smithi070 ceph-mon[40691]: pgmap v20: 132 pgs: 132 active+clean; 583 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 898 B/s rd, 0 op/s 2024-09-16T09:25:57.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:57 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:25:57.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:57 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:25:57.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:57 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:25:57.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:57 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:25:57.172 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Sep 16 09:25:56 smithi070 podman[75679]: 2024-09-16 09:25:56.899465428 +0000 UTC m=+0.598624364 container init f7b0530f3dbf8d8acb21cde05413a70b8de3412f7ad11a4fcff8fdf456b8593f (image=quay.io/prometheus/alertmanager:v0.27.0, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-16T09:25:57.172 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Sep 16 09:25:56 smithi070 podman[75679]: 2024-09-16 09:25:56.904433372 +0000 UTC m=+0.603592294 container start f7b0530f3dbf8d8acb21cde05413a70b8de3412f7ad11a4fcff8fdf456b8593f (image=quay.io/prometheus/alertmanager:v0.27.0, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-16T09:25:57.172 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Sep 16 09:25:56 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-alertmanager-a[75718]: ts=2024-09-16T09:25:56.913Z caller=main.go:181 level=info msg="Starting Alertmanager" version="(version=0.27.0, branch=HEAD, revision=0aa3c2aad14cff039931923ab16b26b7481783b5)" 2024-09-16T09:25:57.172 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Sep 16 09:25:56 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-alertmanager-a[75718]: ts=2024-09-16T09:25:56.913Z caller=main.go:182 level=info build_context="(go=go1.21.7, platform=linux/amd64, user=root@22cd11f671e9, date=20240228-11:51:20, tags=netgo)" 2024-09-16T09:25:57.172 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Sep 16 09:25:56 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-alertmanager-a[75718]: ts=2024-09-16T09:25:56.914Z caller=cluster.go:186 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.70 port=9094 2024-09-16T09:25:57.172 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Sep 16 09:25:56 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-alertmanager-a[75718]: ts=2024-09-16T09:25:56.915Z caller=cluster.go:683 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-09-16T09:25:57.172 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Sep 16 09:25:56 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-alertmanager-a[75718]: ts=2024-09-16T09:25:56.937Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-16T09:25:57.172 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Sep 16 09:25:56 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-alertmanager-a[75718]: ts=2024-09-16T09:25:56.937Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-16T09:25:57.172 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Sep 16 09:25:56 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-alertmanager-a[75718]: ts=2024-09-16T09:25:56.939Z caller=tls_config.go:313 level=info msg="Listening on" address=[::]:9093 2024-09-16T09:25:57.172 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Sep 16 09:25:56 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-alertmanager-a[75718]: ts=2024-09-16T09:25:56.939Z caller=tls_config.go:316 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-09-16T09:25:57.172 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Sep 16 09:25:56 smithi070 bash[75679]: f7b0530f3dbf8d8acb21cde05413a70b8de3412f7ad11a4fcff8fdf456b8593f 2024-09-16T09:25:57.172 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Sep 16 09:25:57 smithi070 systemd[1]: Started Ceph alertmanager.a for a1321834-740c-11ef-bceb-c7b262605968. 2024-09-16T09:25:57.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:57 smithi084 ceph-mon[32629]: pgmap v20: 132 pgs: 132 active+clean; 583 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 898 B/s rd, 0 op/s 2024-09-16T09:25:57.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:57 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:25:57.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:57 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:25:57.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:57 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:25:57.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:57 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:25:57.760 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:25:57.761 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:25:57 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: [16/Sep/2024:09:25:57] ENGINE Bus STOPPING 2024-09-16T09:25:57.765 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2024-09-16T09:25:58.049 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:25:57 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: [16/Sep/2024:09:25:57] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-16T09:25:58.049 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:25:57 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: [16/Sep/2024:09:25:57] ENGINE Bus STOPPED 2024-09-16T09:25:58.049 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:25:57 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: [16/Sep/2024:09:25:57] ENGINE Bus STARTING 2024-09-16T09:25:58.049 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:25:58 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: [16/Sep/2024:09:25:58] ENGINE Serving on http://:::9283 2024-09-16T09:25:58.049 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:25:58 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: [16/Sep/2024:09:25:58] ENGINE Bus STARTED 2024-09-16T09:25:58.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:58 smithi084 ceph-mon[32629]: Regenerating cephadm self-signed grafana TLS certificates 2024-09-16T09:25:58.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:58 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:25:58.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:58 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:25:58.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:58 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"} : dispatch 2024-09-16T09:25:58.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:58 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:25:58.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:58 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-16T09:25:58.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:58 smithi070 ceph-mon[30388]: Regenerating cephadm self-signed grafana TLS certificates 2024-09-16T09:25:58.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:58 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:25:58.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:58 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:25:58.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:58 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"} : dispatch 2024-09-16T09:25:58.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:58 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:25:58.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:58 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-16T09:25:58.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:58 smithi070 ceph-mon[40691]: Regenerating cephadm self-signed grafana TLS certificates 2024-09-16T09:25:58.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:58 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:25:58.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:58 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:25:58.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:58 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"} : dispatch 2024-09-16T09:25:58.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:58 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:25:58.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:58 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-16T09:25:58.498 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-09-16T09:25:57.485066+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596507,"num_objects":199,"num_object_clones":0,"num_object_copies":597,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":199,"num_whiteouts":0,"num_read":898,"num_read_kb":755,"num_write":549,"num_write_kb":1673,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":552,"ondisk_log_size":552,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":241500,"kb_used_data":8636,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":749719716,"statfs":{"total":767960285184,"available":767712989184,"internally_reserved":0,"allocated":8843264,"data_stored":4862689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":238276146},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003618"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":50,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.085013+0000","last_change":"2024-09-16T09:25:01.709392+0000","last_active":"2024-09-16T09:25:44.085013+0000","last_peered":"2024-09-16T09:25:44.085013+0000","last_clean":"2024-09-16T09:25:44.085013+0000","last_became_active":"2024-09-16T09:25:01.709108+0000","last_became_peered":"2024-09-16T09:25:01.709108+0000","last_unstale":"2024-09-16T09:25:44.085013+0000","last_undegraded":"2024-09-16T09:25:44.085013+0000","last_fullsized":"2024-09-16T09:25:44.085013+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:05.830525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"66'9","reported_seq":62,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.084866+0000","last_change":"2024-09-16T09:25:03.715139+0000","last_active":"2024-09-16T09:25:44.084866+0000","last_peered":"2024-09-16T09:25:44.084866+0000","last_clean":"2024-09-16T09:25:44.084866+0000","last_became_active":"2024-09-16T09:25:03.714635+0000","last_became_peered":"2024-09-16T09:25:03.714635+0000","last_unstale":"2024-09-16T09:25:44.084866+0000","last_undegraded":"2024-09-16T09:25:44.084866+0000","last_fullsized":"2024-09-16T09:25:44.084866+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:19.723608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":42,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.084994+0000","last_change":"2024-09-16T09:25:05.716139+0000","last_active":"2024-09-16T09:25:44.084994+0000","last_peered":"2024-09-16T09:25:44.084994+0000","last_clean":"2024-09-16T09:25:44.084994+0000","last_became_active":"2024-09-16T09:25:05.715645+0000","last_became_peered":"2024-09-16T09:25:05.715645+0000","last_unstale":"2024-09-16T09:25:44.084994+0000","last_undegraded":"2024-09-16T09:25:44.084994+0000","last_fullsized":"2024-09-16T09:25:44.084994+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:31.956608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":38,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.085058+0000","last_change":"2024-09-16T09:25:07.715923+0000","last_active":"2024-09-16T09:25:44.085058+0000","last_peered":"2024-09-16T09:25:44.085058+0000","last_clean":"2024-09-16T09:25:44.085058+0000","last_became_active":"2024-09-16T09:25:07.715442+0000","last_became_peered":"2024-09-16T09:25:07.715442+0000","last_unstale":"2024-09-16T09:25:44.085058+0000","last_undegraded":"2024-09-16T09:25:44.085058+0000","last_fullsized":"2024-09-16T09:25:44.085058+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:35.494958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":38,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.114483+0000","last_change":"2024-09-16T09:25:07.718687+0000","last_active":"2024-09-16T09:25:44.114483+0000","last_peered":"2024-09-16T09:25:44.114483+0000","last_clean":"2024-09-16T09:25:44.114483+0000","last_became_active":"2024-09-16T09:25:07.718126+0000","last_became_peered":"2024-09-16T09:25:07.718126+0000","last_unstale":"2024-09-16T09:25:44.114483+0000","last_undegraded":"2024-09-16T09:25:44.114483+0000","last_fullsized":"2024-09-16T09:25:44.114483+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:23.444563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":50,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.115035+0000","last_change":"2024-09-16T09:25:01.708505+0000","last_active":"2024-09-16T09:25:44.115035+0000","last_peered":"2024-09-16T09:25:44.115035+0000","last_clean":"2024-09-16T09:25:44.115035+0000","last_became_active":"2024-09-16T09:25:01.708179+0000","last_became_peered":"2024-09-16T09:25:01.708179+0000","last_unstale":"2024-09-16T09:25:44.115035+0000","last_undegraded":"2024-09-16T09:25:44.115035+0000","last_fullsized":"2024-09-16T09:25:44.115035+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:22.060114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"66'15","reported_seq":71,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.114635+0000","last_change":"2024-09-16T09:25:03.709970+0000","last_active":"2024-09-16T09:25:44.114635+0000","last_peered":"2024-09-16T09:25:44.114635+0000","last_clean":"2024-09-16T09:25:44.114635+0000","last_became_active":"2024-09-16T09:25:03.709499+0000","last_became_peered":"2024-09-16T09:25:03.709499+0000","last_unstale":"2024-09-16T09:25:44.114635+0000","last_undegraded":"2024-09-16T09:25:44.114635+0000","last_fullsized":"2024-09-16T09:25:44.114635+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:05.973210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":42,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.084860+0000","last_change":"2024-09-16T09:25:05.717524+0000","last_active":"2024-09-16T09:25:44.084860+0000","last_peered":"2024-09-16T09:25:44.084860+0000","last_clean":"2024-09-16T09:25:44.084860+0000","last_became_active":"2024-09-16T09:25:05.717236+0000","last_became_peered":"2024-09-16T09:25:05.717236+0000","last_unstale":"2024-09-16T09:25:44.084860+0000","last_undegraded":"2024-09-16T09:25:44.084860+0000","last_fullsized":"2024-09-16T09:25:44.084860+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:14.774263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":49,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.605268+0000","last_change":"2024-09-16T09:25:01.705552+0000","last_active":"2024-09-16T09:25:44.605268+0000","last_peered":"2024-09-16T09:25:44.605268+0000","last_clean":"2024-09-16T09:25:44.605268+0000","last_became_active":"2024-09-16T09:25:01.705269+0000","last_became_peered":"2024-09-16T09:25:01.705269+0000","last_unstale":"2024-09-16T09:25:44.605268+0000","last_undegraded":"2024-09-16T09:25:44.605268+0000","last_fullsized":"2024-09-16T09:25:44.605268+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:18.445329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"66'9","reported_seq":62,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.085428+0000","last_change":"2024-09-16T09:25:03.709441+0000","last_active":"2024-09-16T09:25:44.085428+0000","last_peered":"2024-09-16T09:25:44.085428+0000","last_clean":"2024-09-16T09:25:44.085428+0000","last_became_active":"2024-09-16T09:25:03.709147+0000","last_became_peered":"2024-09-16T09:25:03.709147+0000","last_unstale":"2024-09-16T09:25:44.085428+0000","last_undegraded":"2024-09-16T09:25:44.085428+0000","last_fullsized":"2024-09-16T09:25:44.085428+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:21.309548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":41,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.604538+0000","last_change":"2024-09-16T09:25:05.718510+0000","last_active":"2024-09-16T09:25:44.604538+0000","last_peered":"2024-09-16T09:25:44.604538+0000","last_clean":"2024-09-16T09:25:44.604538+0000","last_became_active":"2024-09-16T09:25:05.718134+0000","last_became_peered":"2024-09-16T09:25:05.718134+0000","last_unstale":"2024-09-16T09:25:44.604538+0000","last_undegraded":"2024-09-16T09:25:44.604538+0000","last_fullsized":"2024-09-16T09:25:44.604538+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:30.593901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"66'1","reported_seq":39,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.084246+0000","last_change":"2024-09-16T09:25:07.716869+0000","last_active":"2024-09-16T09:25:44.084246+0000","last_peered":"2024-09-16T09:25:44.084246+0000","last_clean":"2024-09-16T09:25:44.084246+0000","last_became_active":"2024-09-16T09:25:07.716558+0000","last_became_peered":"2024-09-16T09:25:07.716558+0000","last_unstale":"2024-09-16T09:25:44.084246+0000","last_undegraded":"2024-09-16T09:25:44.084246+0000","last_fullsized":"2024-09-16T09:25:44.084246+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:17.035822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":49,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.604930+0000","last_change":"2024-09-16T09:25:01.697306+0000","last_active":"2024-09-16T09:25:44.604930+0000","last_peered":"2024-09-16T09:25:44.604930+0000","last_clean":"2024-09-16T09:25:44.604930+0000","last_became_active":"2024-09-16T09:25:01.696939+0000","last_became_peered":"2024-09-16T09:25:01.696939+0000","last_unstale":"2024-09-16T09:25:44.604930+0000","last_undegraded":"2024-09-16T09:25:44.604930+0000","last_fullsized":"2024-09-16T09:25:44.604930+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:26.024926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"66'5","reported_seq":56,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.085109+0000","last_change":"2024-09-16T09:25:03.716402+0000","last_active":"2024-09-16T09:25:44.085109+0000","last_peered":"2024-09-16T09:25:44.085109+0000","last_clean":"2024-09-16T09:25:44.085109+0000","last_became_active":"2024-09-16T09:25:03.716136+0000","last_became_peered":"2024-09-16T09:25:03.716136+0000","last_unstale":"2024-09-16T09:25:44.085109+0000","last_undegraded":"2024-09-16T09:25:44.085109+0000","last_fullsized":"2024-09-16T09:25:44.085109+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:32.360413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":41,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.605054+0000","last_change":"2024-09-16T09:25:05.718538+0000","last_active":"2024-09-16T09:25:44.605054+0000","last_peered":"2024-09-16T09:25:44.605054+0000","last_clean":"2024-09-16T09:25:44.605054+0000","last_became_active":"2024-09-16T09:25:05.718281+0000","last_became_peered":"2024-09-16T09:25:05.718281+0000","last_unstale":"2024-09-16T09:25:44.605054+0000","last_undegraded":"2024-09-16T09:25:44.605054+0000","last_fullsized":"2024-09-16T09:25:44.605054+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:29.527824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":37,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.604126+0000","last_change":"2024-09-16T09:25:07.717644+0000","last_active":"2024-09-16T09:25:44.604126+0000","last_peered":"2024-09-16T09:25:44.604126+0000","last_clean":"2024-09-16T09:25:44.604126+0000","last_became_active":"2024-09-16T09:25:07.716724+0000","last_became_peered":"2024-09-16T09:25:07.716724+0000","last_unstale":"2024-09-16T09:25:44.604126+0000","last_undegraded":"2024-09-16T09:25:44.604126+0000","last_fullsized":"2024-09-16T09:25:44.604126+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:32.145892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":38,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.085350+0000","last_change":"2024-09-16T09:25:07.721088+0000","last_active":"2024-09-16T09:25:44.085350+0000","last_peered":"2024-09-16T09:25:44.085350+0000","last_clean":"2024-09-16T09:25:44.085350+0000","last_became_active":"2024-09-16T09:25:07.720793+0000","last_became_peered":"2024-09-16T09:25:07.720793+0000","last_unstale":"2024-09-16T09:25:44.085350+0000","last_undegraded":"2024-09-16T09:25:44.085350+0000","last_fullsized":"2024-09-16T09:25:44.085350+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:16.293244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":50,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.085187+0000","last_change":"2024-09-16T09:25:01.706762+0000","last_active":"2024-09-16T09:25:44.085187+0000","last_peered":"2024-09-16T09:25:44.085187+0000","last_clean":"2024-09-16T09:25:44.085187+0000","last_became_active":"2024-09-16T09:25:01.706172+0000","last_became_peered":"2024-09-16T09:25:01.706172+0000","last_unstale":"2024-09-16T09:25:44.085187+0000","last_undegraded":"2024-09-16T09:25:44.085187+0000","last_fullsized":"2024-09-16T09:25:44.085187+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:07.135883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"66'15","reported_seq":71,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.604477+0000","last_change":"2024-09-16T09:25:03.714261+0000","last_active":"2024-09-16T09:25:44.604477+0000","last_peered":"2024-09-16T09:25:44.604477+0000","last_clean":"2024-09-16T09:25:44.604477+0000","last_became_active":"2024-09-16T09:25:03.713780+0000","last_became_peered":"2024-09-16T09:25:03.713780+0000","last_unstale":"2024-09-16T09:25:44.604477+0000","last_undegraded":"2024-09-16T09:25:44.604477+0000","last_fullsized":"2024-09-16T09:25:44.604477+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:29.826960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":42,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.085112+0000","last_change":"2024-09-16T09:25:05.714209+0000","last_active":"2024-09-16T09:25:44.085112+0000","last_peered":"2024-09-16T09:25:44.085112+0000","last_clean":"2024-09-16T09:25:44.085112+0000","last_became_active":"2024-09-16T09:25:05.713929+0000","last_became_peered":"2024-09-16T09:25:05.713929+0000","last_unstale":"2024-09-16T09:25:44.085112+0000","last_undegraded":"2024-09-16T09:25:44.085112+0000","last_fullsized":"2024-09-16T09:25:44.085112+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:05.941022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":38,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.115048+0000","last_change":"2024-09-16T09:25:07.719645+0000","last_active":"2024-09-16T09:25:44.115048+0000","last_peered":"2024-09-16T09:25:44.115048+0000","last_clean":"2024-09-16T09:25:44.115048+0000","last_became_active":"2024-09-16T09:25:07.719067+0000","last_became_peered":"2024-09-16T09:25:07.719067+0000","last_unstale":"2024-09-16T09:25:44.115048+0000","last_undegraded":"2024-09-16T09:25:44.115048+0000","last_fullsized":"2024-09-16T09:25:44.115048+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:19.447150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":50,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.085738+0000","last_change":"2024-09-16T09:25:01.706828+0000","last_active":"2024-09-16T09:25:44.085738+0000","last_peered":"2024-09-16T09:25:44.085738+0000","last_clean":"2024-09-16T09:25:44.085738+0000","last_became_active":"2024-09-16T09:25:01.706491+0000","last_became_peered":"2024-09-16T09:25:01.706491+0000","last_unstale":"2024-09-16T09:25:44.085738+0000","last_undegraded":"2024-09-16T09:25:44.085738+0000","last_fullsized":"2024-09-16T09:25:44.085738+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:16.321414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"66'12","reported_seq":69,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.114882+0000","last_change":"2024-09-16T09:25:03.709914+0000","last_active":"2024-09-16T09:25:44.114882+0000","last_peered":"2024-09-16T09:25:44.114882+0000","last_clean":"2024-09-16T09:25:44.114882+0000","last_became_active":"2024-09-16T09:25:03.709610+0000","last_became_peered":"2024-09-16T09:25:03.709610+0000","last_unstale":"2024-09-16T09:25:44.114882+0000","last_undegraded":"2024-09-16T09:25:44.114882+0000","last_fullsized":"2024-09-16T09:25:44.114882+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:26.267096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":42,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.085216+0000","last_change":"2024-09-16T09:25:05.710379+0000","last_active":"2024-09-16T09:25:44.085216+0000","last_peered":"2024-09-16T09:25:44.085216+0000","last_clean":"2024-09-16T09:25:44.085216+0000","last_became_active":"2024-09-16T09:25:05.710093+0000","last_became_peered":"2024-09-16T09:25:05.710093+0000","last_unstale":"2024-09-16T09:25:44.085216+0000","last_undegraded":"2024-09-16T09:25:44.085216+0000","last_fullsized":"2024-09-16T09:25:44.085216+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:28.802108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":38,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.605509+0000","last_change":"2024-09-16T09:25:07.719655+0000","last_active":"2024-09-16T09:25:44.605509+0000","last_peered":"2024-09-16T09:25:44.605509+0000","last_clean":"2024-09-16T09:25:44.605509+0000","last_became_active":"2024-09-16T09:25:07.719380+0000","last_became_peered":"2024-09-16T09:25:07.719380+0000","last_unstale":"2024-09-16T09:25:44.605509+0000","last_undegraded":"2024-09-16T09:25:44.605509+0000","last_fullsized":"2024-09-16T09:25:44.605509+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:36.059179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":51,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.085126+0000","last_change":"2024-09-16T09:25:01.700263+0000","last_active":"2024-09-16T09:25:44.085126+0000","last_peered":"2024-09-16T09:25:44.085126+0000","last_clean":"2024-09-16T09:25:44.085126+0000","last_became_active":"2024-09-16T09:25:01.699545+0000","last_became_peered":"2024-09-16T09:25:01.699545+0000","last_unstale":"2024-09-16T09:25:44.085126+0000","last_undegraded":"2024-09-16T09:25:44.085126+0000","last_fullsized":"2024-09-16T09:25:44.085126+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:04.529119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"66'10","reported_seq":61,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.085454+0000","last_change":"2024-09-16T09:25:03.720638+0000","last_active":"2024-09-16T09:25:44.085454+0000","last_peered":"2024-09-16T09:25:44.085454+0000","last_clean":"2024-09-16T09:25:44.085454+0000","last_became_active":"2024-09-16T09:25:03.720245+0000","last_became_peered":"2024-09-16T09:25:03.720245+0000","last_unstale":"2024-09-16T09:25:44.085454+0000","last_undegraded":"2024-09-16T09:25:44.085454+0000","last_fullsized":"2024-09-16T09:25:44.085454+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:21.451538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"66'8","reported_seq":64,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:46.757941+0000","last_change":"2024-09-16T09:25:05.713310+0000","last_active":"2024-09-16T09:25:46.757941+0000","last_peered":"2024-09-16T09:25:46.757941+0000","last_clean":"2024-09-16T09:25:46.757941+0000","last_became_active":"2024-09-16T09:25:05.712516+0000","last_became_peered":"2024-09-16T09:25:05.712516+0000","last_unstale":"2024-09-16T09:25:46.757941+0000","last_undegraded":"2024-09-16T09:25:46.757941+0000","last_fullsized":"2024-09-16T09:25:46.757941+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:24.505283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.f","version":"66'15","reported_seq":71,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.086235+0000","last_change":"2024-09-16T09:25:03.719134+0000","last_active":"2024-09-16T09:25:44.086235+0000","last_peered":"2024-09-16T09:25:44.086235+0000","last_clean":"2024-09-16T09:25:44.086235+0000","last_became_active":"2024-09-16T09:25:03.718849+0000","last_became_peered":"2024-09-16T09:25:03.718849+0000","last_unstale":"2024-09-16T09:25:44.086235+0000","last_undegraded":"2024-09-16T09:25:44.086235+0000","last_fullsized":"2024-09-16T09:25:44.086235+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:13.173718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":50,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.114539+0000","last_change":"2024-09-16T09:25:01.702936+0000","last_active":"2024-09-16T09:25:44.114539+0000","last_peered":"2024-09-16T09:25:44.114539+0000","last_clean":"2024-09-16T09:25:44.114539+0000","last_became_active":"2024-09-16T09:25:01.702629+0000","last_became_peered":"2024-09-16T09:25:01.702629+0000","last_unstale":"2024-09-16T09:25:44.114539+0000","last_undegraded":"2024-09-16T09:25:44.114539+0000","last_fullsized":"2024-09-16T09:25:44.114539+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:20.061024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"66'8","reported_seq":61,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:46.757971+0000","last_change":"2024-09-16T09:25:05.714128+0000","last_active":"2024-09-16T09:25:46.757971+0000","last_peered":"2024-09-16T09:25:46.757971+0000","last_clean":"2024-09-16T09:25:46.757971+0000","last_became_active":"2024-09-16T09:25:05.713813+0000","last_became_peered":"2024-09-16T09:25:05.713813+0000","last_unstale":"2024-09-16T09:25:46.757971+0000","last_undegraded":"2024-09-16T09:25:46.757971+0000","last_fullsized":"2024-09-16T09:25:46.757971+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:26.013786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":37,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.604736+0000","last_change":"2024-09-16T09:25:07.717639+0000","last_active":"2024-09-16T09:25:44.604736+0000","last_peered":"2024-09-16T09:25:44.604736+0000","last_clean":"2024-09-16T09:25:44.604736+0000","last_became_active":"2024-09-16T09:25:07.717344+0000","last_became_peered":"2024-09-16T09:25:07.717344+0000","last_unstale":"2024-09-16T09:25:44.604736+0000","last_undegraded":"2024-09-16T09:25:44.604736+0000","last_fullsized":"2024-09-16T09:25:44.604736+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:24.054256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.0","version":"66'18","reported_seq":78,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.114403+0000","last_change":"2024-09-16T09:25:03.714508+0000","last_active":"2024-09-16T09:25:44.114403+0000","last_peered":"2024-09-16T09:25:44.114403+0000","last_clean":"2024-09-16T09:25:44.114403+0000","last_became_active":"2024-09-16T09:25:03.714208+0000","last_became_peered":"2024-09-16T09:25:03.714208+0000","last_unstale":"2024-09-16T09:25:44.114403+0000","last_undegraded":"2024-09-16T09:25:44.114403+0000","last_fullsized":"2024-09-16T09:25:44.114403+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:03.200598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":50,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.115220+0000","last_change":"2024-09-16T09:25:01.705362+0000","last_active":"2024-09-16T09:25:44.115220+0000","last_peered":"2024-09-16T09:25:44.115220+0000","last_clean":"2024-09-16T09:25:44.115220+0000","last_became_active":"2024-09-16T09:25:01.704994+0000","last_became_peered":"2024-09-16T09:25:01.704994+0000","last_unstale":"2024-09-16T09:25:44.115220+0000","last_undegraded":"2024-09-16T09:25:44.115220+0000","last_fullsized":"2024-09-16T09:25:44.115220+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:17.466780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":42,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.084923+0000","last_change":"2024-09-16T09:25:05.713687+0000","last_active":"2024-09-16T09:25:44.084923+0000","last_peered":"2024-09-16T09:25:44.084923+0000","last_clean":"2024-09-16T09:25:44.084923+0000","last_became_active":"2024-09-16T09:25:05.713360+0000","last_became_peered":"2024-09-16T09:25:05.713360+0000","last_unstale":"2024-09-16T09:25:44.084923+0000","last_undegraded":"2024-09-16T09:25:44.084923+0000","last_fullsized":"2024-09-16T09:25:44.084923+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:16.400241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":38,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.085226+0000","last_change":"2024-09-16T09:25:07.719412+0000","last_active":"2024-09-16T09:25:44.085226+0000","last_peered":"2024-09-16T09:25:44.085226+0000","last_clean":"2024-09-16T09:25:44.085226+0000","last_became_active":"2024-09-16T09:25:07.719159+0000","last_became_peered":"2024-09-16T09:25:07.719159+0000","last_unstale":"2024-09-16T09:25:44.085226+0000","last_undegraded":"2024-09-16T09:25:44.085226+0000","last_fullsized":"2024-09-16T09:25:44.085226+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:36.545361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"66'14","reported_seq":67,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.084923+0000","last_change":"2024-09-16T09:25:03.710438+0000","last_active":"2024-09-16T09:25:44.084923+0000","last_peered":"2024-09-16T09:25:44.084923+0000","last_clean":"2024-09-16T09:25:44.084923+0000","last_became_active":"2024-09-16T09:25:03.708830+0000","last_became_peered":"2024-09-16T09:25:03.708830+0000","last_unstale":"2024-09-16T09:25:44.084923+0000","last_undegraded":"2024-09-16T09:25:44.084923+0000","last_fullsized":"2024-09-16T09:25:44.084923+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:17.190767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":51,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.085717+0000","last_change":"2024-09-16T09:25:01.703245+0000","last_active":"2024-09-16T09:25:44.085717+0000","last_peered":"2024-09-16T09:25:44.085717+0000","last_clean":"2024-09-16T09:25:44.085717+0000","last_became_active":"2024-09-16T09:25:01.702992+0000","last_became_peered":"2024-09-16T09:25:01.702992+0000","last_unstale":"2024-09-16T09:25:44.085717+0000","last_undegraded":"2024-09-16T09:25:44.085717+0000","last_fullsized":"2024-09-16T09:25:44.085717+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:00.901034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"66'8","reported_seq":62,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:51.759061+0000","last_change":"2024-09-16T09:25:05.715927+0000","last_active":"2024-09-16T09:25:51.759061+0000","last_peered":"2024-09-16T09:25:51.759061+0000","last_clean":"2024-09-16T09:25:51.759061+0000","last_became_active":"2024-09-16T09:25:05.715606+0000","last_became_peered":"2024-09-16T09:25:05.715606+0000","last_unstale":"2024-09-16T09:25:51.759061+0000","last_undegraded":"2024-09-16T09:25:51.759061+0000","last_fullsized":"2024-09-16T09:25:51.759061+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:23.752404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":37,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.605258+0000","last_change":"2024-09-16T09:25:07.722287+0000","last_active":"2024-09-16T09:25:44.605258+0000","last_peered":"2024-09-16T09:25:44.605258+0000","last_clean":"2024-09-16T09:25:44.605258+0000","last_became_active":"2024-09-16T09:25:07.722017+0000","last_became_peered":"2024-09-16T09:25:07.722017+0000","last_unstale":"2024-09-16T09:25:44.605258+0000","last_undegraded":"2024-09-16T09:25:44.605258+0000","last_fullsized":"2024-09-16T09:25:44.605258+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:27.828108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2","version":"66'10","reported_seq":61,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.086135+0000","last_change":"2024-09-16T09:25:03.713368+0000","last_active":"2024-09-16T09:25:44.086135+0000","last_peered":"2024-09-16T09:25:44.086135+0000","last_clean":"2024-09-16T09:25:44.086135+0000","last_became_active":"2024-09-16T09:25:03.712985+0000","last_became_peered":"2024-09-16T09:25:03.712985+0000","last_unstale":"2024-09-16T09:25:44.086135+0000","last_undegraded":"2024-09-16T09:25:44.086135+0000","last_fullsized":"2024-09-16T09:25:44.086135+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:21.169460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":49,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.605335+0000","last_change":"2024-09-16T09:25:01.706774+0000","last_active":"2024-09-16T09:25:44.605335+0000","last_peered":"2024-09-16T09:25:44.605335+0000","last_clean":"2024-09-16T09:25:44.605335+0000","last_became_active":"2024-09-16T09:25:01.706440+0000","last_became_peered":"2024-09-16T09:25:01.706440+0000","last_unstale":"2024-09-16T09:25:44.605335+0000","last_undegraded":"2024-09-16T09:25:44.605335+0000","last_fullsized":"2024-09-16T09:25:44.605335+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:18.805280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"66'8","reported_seq":61,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:46.757821+0000","last_change":"2024-09-16T09:25:05.714825+0000","last_active":"2024-09-16T09:25:46.757821+0000","last_peered":"2024-09-16T09:25:46.757821+0000","last_clean":"2024-09-16T09:25:46.757821+0000","last_became_active":"2024-09-16T09:25:05.714507+0000","last_became_peered":"2024-09-16T09:25:05.714507+0000","last_unstale":"2024-09-16T09:25:46.757821+0000","last_undegraded":"2024-09-16T09:25:46.757821+0000","last_fullsized":"2024-09-16T09:25:46.757821+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:13.181647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":38,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.084322+0000","last_change":"2024-09-16T09:25:07.719876+0000","last_active":"2024-09-16T09:25:44.084322+0000","last_peered":"2024-09-16T09:25:44.084322+0000","last_clean":"2024-09-16T09:25:44.084322+0000","last_became_active":"2024-09-16T09:25:07.719604+0000","last_became_peered":"2024-09-16T09:25:07.719604+0000","last_unstale":"2024-09-16T09:25:44.084322+0000","last_undegraded":"2024-09-16T09:25:44.084322+0000","last_fullsized":"2024-09-16T09:25:44.084322+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:20.929075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"66'19","reported_seq":82,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.085024+0000","last_change":"2024-09-16T09:25:03.715086+0000","last_active":"2024-09-16T09:25:44.085024+0000","last_peered":"2024-09-16T09:25:44.085024+0000","last_clean":"2024-09-16T09:25:44.085024+0000","last_became_active":"2024-09-16T09:25:03.714797+0000","last_became_peered":"2024-09-16T09:25:03.714797+0000","last_unstale":"2024-09-16T09:25:44.085024+0000","last_undegraded":"2024-09-16T09:25:44.085024+0000","last_fullsized":"2024-09-16T09:25:44.085024+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:23.683038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":50,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.085873+0000","last_change":"2024-09-16T09:25:01.707093+0000","last_active":"2024-09-16T09:25:44.085873+0000","last_peered":"2024-09-16T09:25:44.085873+0000","last_clean":"2024-09-16T09:25:44.085873+0000","last_became_active":"2024-09-16T09:25:01.706806+0000","last_became_peered":"2024-09-16T09:25:01.706806+0000","last_unstale":"2024-09-16T09:25:44.085873+0000","last_undegraded":"2024-09-16T09:25:44.085873+0000","last_fullsized":"2024-09-16T09:25:44.085873+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:01.655926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":42,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.302460+0000","last_change":"2024-09-16T09:25:05.708949+0000","last_active":"2024-09-16T09:25:44.302460+0000","last_peered":"2024-09-16T09:25:44.302460+0000","last_clean":"2024-09-16T09:25:44.302460+0000","last_became_active":"2024-09-16T09:25:05.708663+0000","last_became_peered":"2024-09-16T09:25:05.708663+0000","last_unstale":"2024-09-16T09:25:44.302460+0000","last_undegraded":"2024-09-16T09:25:44.302460+0000","last_fullsized":"2024-09-16T09:25:44.302460+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:06.107203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":38,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.084912+0000","last_change":"2024-09-16T09:25:07.719568+0000","last_active":"2024-09-16T09:25:44.084912+0000","last_peered":"2024-09-16T09:25:44.084912+0000","last_clean":"2024-09-16T09:25:44.084912+0000","last_became_active":"2024-09-16T09:25:07.719267+0000","last_became_peered":"2024-09-16T09:25:07.719267+0000","last_unstale":"2024-09-16T09:25:44.084912+0000","last_undegraded":"2024-09-16T09:25:44.084912+0000","last_fullsized":"2024-09-16T09:25:44.084912+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:31.531856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"66'28","reported_seq":106,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:51.759039+0000","last_change":"2024-09-16T09:25:03.705300+0000","last_active":"2024-09-16T09:25:51.759039+0000","last_peered":"2024-09-16T09:25:51.759039+0000","last_clean":"2024-09-16T09:25:51.759039+0000","last_became_active":"2024-09-16T09:25:03.704878+0000","last_became_peered":"2024-09-16T09:25:03.704878+0000","last_unstale":"2024-09-16T09:25:51.759039+0000","last_undegraded":"2024-09-16T09:25:51.759039+0000","last_fullsized":"2024-09-16T09:25:51.759039+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:32.612703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":50,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.085856+0000","last_change":"2024-09-16T09:25:01.703394+0000","last_active":"2024-09-16T09:25:44.085856+0000","last_peered":"2024-09-16T09:25:44.085856+0000","last_clean":"2024-09-16T09:25:44.085856+0000","last_became_active":"2024-09-16T09:25:01.703076+0000","last_became_peered":"2024-09-16T09:25:01.703076+0000","last_unstale":"2024-09-16T09:25:44.085856+0000","last_undegraded":"2024-09-16T09:25:44.085856+0000","last_fullsized":"2024-09-16T09:25:44.085856+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:21.922041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":59,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.604566+0000","last_change":"2024-09-16T09:25:03.706980+0000","last_active":"2024-09-16T09:25:44.604566+0000","last_peered":"2024-09-16T09:25:44.604566+0000","last_clean":"2024-09-16T09:25:44.604566+0000","last_became_active":"2024-09-16T09:25:00.696237+0000","last_became_peered":"2024-09-16T09:25:00.696237+0000","last_unstale":"2024-09-16T09:25:44.604566+0000","last_undegraded":"2024-09-16T09:25:44.604566+0000","last_fullsized":"2024-09-16T09:25:44.604566+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:24:59.684950+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:24.893358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025137430000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":42,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.084795+0000","last_change":"2024-09-16T09:25:05.718319+0000","last_active":"2024-09-16T09:25:44.084795+0000","last_peered":"2024-09-16T09:25:44.084795+0000","last_clean":"2024-09-16T09:25:44.084795+0000","last_became_active":"2024-09-16T09:25:05.717591+0000","last_became_peered":"2024-09-16T09:25:05.717591+0000","last_unstale":"2024-09-16T09:25:44.084795+0000","last_undegraded":"2024-09-16T09:25:44.084795+0000","last_fullsized":"2024-09-16T09:25:44.084795+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:32.732085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":38,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.114938+0000","last_change":"2024-09-16T09:25:07.717720+0000","last_active":"2024-09-16T09:25:44.114938+0000","last_peered":"2024-09-16T09:25:44.114938+0000","last_clean":"2024-09-16T09:25:44.114938+0000","last_became_active":"2024-09-16T09:25:07.717436+0000","last_became_peered":"2024-09-16T09:25:07.717436+0000","last_unstale":"2024-09-16T09:25:44.114938+0000","last_undegraded":"2024-09-16T09:25:44.114938+0000","last_fullsized":"2024-09-16T09:25:44.114938+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:16.230936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"66'13","reported_seq":73,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.086289+0000","last_change":"2024-09-16T09:25:03.713631+0000","last_active":"2024-09-16T09:25:44.086289+0000","last_peered":"2024-09-16T09:25:44.086289+0000","last_clean":"2024-09-16T09:25:44.086289+0000","last_became_active":"2024-09-16T09:25:03.713241+0000","last_became_peered":"2024-09-16T09:25:03.713241+0000","last_unstale":"2024-09-16T09:25:44.086289+0000","last_undegraded":"2024-09-16T09:25:44.086289+0000","last_fullsized":"2024-09-16T09:25:44.086289+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:19.940601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":50,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.085322+0000","last_change":"2024-09-16T09:25:01.707526+0000","last_active":"2024-09-16T09:25:44.085322+0000","last_peered":"2024-09-16T09:25:44.085322+0000","last_clean":"2024-09-16T09:25:44.085322+0000","last_became_active":"2024-09-16T09:25:01.706828+0000","last_became_peered":"2024-09-16T09:25:01.706828+0000","last_unstale":"2024-09-16T09:25:44.085322+0000","last_undegraded":"2024-09-16T09:25:44.085322+0000","last_fullsized":"2024-09-16T09:25:44.085322+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:17.040873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":59,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.604082+0000","last_change":"2024-09-16T09:25:03.702772+0000","last_active":"2024-09-16T09:25:44.604082+0000","last_peered":"2024-09-16T09:25:44.604082+0000","last_clean":"2024-09-16T09:25:44.604082+0000","last_became_active":"2024-09-16T09:25:00.697822+0000","last_became_peered":"2024-09-16T09:25:00.697822+0000","last_unstale":"2024-09-16T09:25:44.604082+0000","last_undegraded":"2024-09-16T09:25:44.604082+0000","last_fullsized":"2024-09-16T09:25:44.604082+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:24:59.684950+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:05.731438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055022299999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":42,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.604640+0000","last_change":"2024-09-16T09:25:05.711840+0000","last_active":"2024-09-16T09:25:44.604640+0000","last_peered":"2024-09-16T09:25:44.604640+0000","last_clean":"2024-09-16T09:25:44.604640+0000","last_became_active":"2024-09-16T09:25:05.711539+0000","last_became_peered":"2024-09-16T09:25:05.711539+0000","last_unstale":"2024-09-16T09:25:44.604640+0000","last_undegraded":"2024-09-16T09:25:44.604640+0000","last_fullsized":"2024-09-16T09:25:44.604640+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:28.306667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":37,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.605681+0000","last_change":"2024-09-16T09:25:07.723554+0000","last_active":"2024-09-16T09:25:44.605681+0000","last_peered":"2024-09-16T09:25:44.605681+0000","last_clean":"2024-09-16T09:25:44.605681+0000","last_became_active":"2024-09-16T09:25:07.723282+0000","last_became_peered":"2024-09-16T09:25:07.723282+0000","last_unstale":"2024-09-16T09:25:44.605681+0000","last_undegraded":"2024-09-16T09:25:44.605681+0000","last_fullsized":"2024-09-16T09:25:44.605681+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:28.111790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"66'12","reported_seq":64,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.085547+0000","last_change":"2024-09-16T09:25:03.715612+0000","last_active":"2024-09-16T09:25:44.085547+0000","last_peered":"2024-09-16T09:25:44.085547+0000","last_clean":"2024-09-16T09:25:44.085547+0000","last_became_active":"2024-09-16T09:25:03.715315+0000","last_became_peered":"2024-09-16T09:25:03.715315+0000","last_unstale":"2024-09-16T09:25:44.085547+0000","last_undegraded":"2024-09-16T09:25:44.085547+0000","last_fullsized":"2024-09-16T09:25:44.085547+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:31.110979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"59'1","reported_seq":56,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.084569+0000","last_change":"2024-09-16T09:25:01.701885+0000","last_active":"2024-09-16T09:25:44.084569+0000","last_peered":"2024-09-16T09:25:44.084569+0000","last_clean":"2024-09-16T09:25:44.084569+0000","last_became_active":"2024-09-16T09:25:01.701599+0000","last_became_peered":"2024-09-16T09:25:01.701599+0000","last_unstale":"2024-09-16T09:25:44.084569+0000","last_undegraded":"2024-09-16T09:25:44.084569+0000","last_fullsized":"2024-09-16T09:25:44.084569+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:03.521969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"67'5","reported_seq":110,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:55.523855+0000","last_change":"2024-09-16T09:25:03.711738+0000","last_active":"2024-09-16T09:25:55.523855+0000","last_peered":"2024-09-16T09:25:55.523855+0000","last_clean":"2024-09-16T09:25:55.523855+0000","last_became_active":"2024-09-16T09:25:00.696546+0000","last_became_peered":"2024-09-16T09:25:00.696546+0000","last_unstale":"2024-09-16T09:25:55.523855+0000","last_undegraded":"2024-09-16T09:25:55.523855+0000","last_fullsized":"2024-09-16T09:25:55.523855+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:24:59.684950+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:17.241966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0059770420000000001,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":50,"num_read_kb":45,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":41,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.605166+0000","last_change":"2024-09-16T09:25:05.714994+0000","last_active":"2024-09-16T09:25:44.605166+0000","last_peered":"2024-09-16T09:25:44.605166+0000","last_clean":"2024-09-16T09:25:44.605166+0000","last_became_active":"2024-09-16T09:25:05.714704+0000","last_became_peered":"2024-09-16T09:25:05.714704+0000","last_unstale":"2024-09-16T09:25:44.605166+0000","last_undegraded":"2024-09-16T09:25:44.605166+0000","last_fullsized":"2024-09-16T09:25:44.605166+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:32.353177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":38,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.085459+0000","last_change":"2024-09-16T09:25:07.714719+0000","last_active":"2024-09-16T09:25:44.085459+0000","last_peered":"2024-09-16T09:25:44.085459+0000","last_clean":"2024-09-16T09:25:44.085459+0000","last_became_active":"2024-09-16T09:25:07.714158+0000","last_became_peered":"2024-09-16T09:25:07.714158+0000","last_unstale":"2024-09-16T09:25:44.085459+0000","last_undegraded":"2024-09-16T09:25:44.085459+0000","last_fullsized":"2024-09-16T09:25:44.085459+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:13.991008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"66'16","reported_seq":79,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:46.757937+0000","last_change":"2024-09-16T09:25:03.714405+0000","last_active":"2024-09-16T09:25:46.757937+0000","last_peered":"2024-09-16T09:25:46.757937+0000","last_clean":"2024-09-16T09:25:46.757937+0000","last_became_active":"2024-09-16T09:25:03.714118+0000","last_became_peered":"2024-09-16T09:25:03.714118+0000","last_unstale":"2024-09-16T09:25:46.757937+0000","last_undegraded":"2024-09-16T09:25:46.757937+0000","last_fullsized":"2024-09-16T09:25:46.757937+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:12.074949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":50,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.114649+0000","last_change":"2024-09-16T09:25:01.704246+0000","last_active":"2024-09-16T09:25:44.114649+0000","last_peered":"2024-09-16T09:25:44.114649+0000","last_clean":"2024-09-16T09:25:44.114649+0000","last_became_active":"2024-09-16T09:25:01.703730+0000","last_became_peered":"2024-09-16T09:25:01.703730+0000","last_unstale":"2024-09-16T09:25:44.114649+0000","last_undegraded":"2024-09-16T09:25:44.114649+0000","last_fullsized":"2024-09-16T09:25:44.114649+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:20.523303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"73'79","reported_seq":87,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.604453+0000","last_change":"2024-09-16T09:24:53.142015+0000","last_active":"2024-09-16T09:25:44.604453+0000","last_peered":"2024-09-16T09:25:44.604453+0000","last_clean":"2024-09-16T09:25:44.604453+0000","last_became_active":"2024-09-16T09:24:53.135890+0000","last_became_peered":"2024-09-16T09:24:53.135890+0000","last_unstale":"2024-09-16T09:25:44.604453+0000","last_undegraded":"2024-09-16T09:25:44.604453+0000","last_fullsized":"2024-09-16T09:25:44.604453+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"20'72","last_scrub_stamp":"2024-09-16T09:24:45.089989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:24:45.089989+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:10.885850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":41,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.605511+0000","last_change":"2024-09-16T09:25:05.714741+0000","last_active":"2024-09-16T09:25:44.605511+0000","last_peered":"2024-09-16T09:25:44.605511+0000","last_clean":"2024-09-16T09:25:44.605511+0000","last_became_active":"2024-09-16T09:25:05.714341+0000","last_became_peered":"2024-09-16T09:25:05.714341+0000","last_unstale":"2024-09-16T09:25:44.605511+0000","last_undegraded":"2024-09-16T09:25:44.605511+0000","last_fullsized":"2024-09-16T09:25:44.605511+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:21.685216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":37,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.605054+0000","last_change":"2024-09-16T09:25:07.714544+0000","last_active":"2024-09-16T09:25:44.605054+0000","last_peered":"2024-09-16T09:25:44.605054+0000","last_clean":"2024-09-16T09:25:44.605054+0000","last_became_active":"2024-09-16T09:25:07.714281+0000","last_became_peered":"2024-09-16T09:25:07.714281+0000","last_unstale":"2024-09-16T09:25:44.605054+0000","last_undegraded":"2024-09-16T09:25:44.605054+0000","last_fullsized":"2024-09-16T09:25:44.605054+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:13.491593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"66'11","reported_seq":65,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.085593+0000","last_change":"2024-09-16T09:25:03.715654+0000","last_active":"2024-09-16T09:25:44.085593+0000","last_peered":"2024-09-16T09:25:44.085593+0000","last_clean":"2024-09-16T09:25:44.085593+0000","last_became_active":"2024-09-16T09:25:03.715373+0000","last_became_peered":"2024-09-16T09:25:03.715373+0000","last_unstale":"2024-09-16T09:25:44.085593+0000","last_undegraded":"2024-09-16T09:25:44.085593+0000","last_fullsized":"2024-09-16T09:25:44.085593+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:11.446460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":50,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.085327+0000","last_change":"2024-09-16T09:25:01.708568+0000","last_active":"2024-09-16T09:25:44.085327+0000","last_peered":"2024-09-16T09:25:44.085327+0000","last_clean":"2024-09-16T09:25:44.085327+0000","last_became_active":"2024-09-16T09:25:01.708270+0000","last_became_peered":"2024-09-16T09:25:01.708270+0000","last_unstale":"2024-09-16T09:25:44.085327+0000","last_undegraded":"2024-09-16T09:25:44.085327+0000","last_fullsized":"2024-09-16T09:25:44.085327+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:24.212331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":41,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.605023+0000","last_change":"2024-09-16T09:25:05.712303+0000","last_active":"2024-09-16T09:25:44.605023+0000","last_peered":"2024-09-16T09:25:44.605023+0000","last_clean":"2024-09-16T09:25:44.605023+0000","last_became_active":"2024-09-16T09:25:05.711964+0000","last_became_peered":"2024-09-16T09:25:05.711964+0000","last_unstale":"2024-09-16T09:25:44.605023+0000","last_undegraded":"2024-09-16T09:25:44.605023+0000","last_fullsized":"2024-09-16T09:25:44.605023+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:23.392105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":38,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.114673+0000","last_change":"2024-09-16T09:25:07.716780+0000","last_active":"2024-09-16T09:25:44.114673+0000","last_peered":"2024-09-16T09:25:44.114673+0000","last_clean":"2024-09-16T09:25:44.114673+0000","last_became_active":"2024-09-16T09:25:07.716514+0000","last_became_peered":"2024-09-16T09:25:07.716514+0000","last_unstale":"2024-09-16T09:25:44.114673+0000","last_undegraded":"2024-09-16T09:25:44.114673+0000","last_fullsized":"2024-09-16T09:25:44.114673+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:07.166770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"66'17","reported_seq":74,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.085933+0000","last_change":"2024-09-16T09:25:03.713691+0000","last_active":"2024-09-16T09:25:44.085933+0000","last_peered":"2024-09-16T09:25:44.085933+0000","last_clean":"2024-09-16T09:25:44.085933+0000","last_became_active":"2024-09-16T09:25:03.713358+0000","last_became_peered":"2024-09-16T09:25:03.713358+0000","last_unstale":"2024-09-16T09:25:44.085933+0000","last_undegraded":"2024-09-16T09:25:44.085933+0000","last_fullsized":"2024-09-16T09:25:44.085933+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:11.619625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":50,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.302717+0000","last_change":"2024-09-16T09:25:01.703939+0000","last_active":"2024-09-16T09:25:44.302717+0000","last_peered":"2024-09-16T09:25:44.302717+0000","last_clean":"2024-09-16T09:25:44.302717+0000","last_became_active":"2024-09-16T09:25:01.703355+0000","last_became_peered":"2024-09-16T09:25:01.703355+0000","last_unstale":"2024-09-16T09:25:44.302717+0000","last_undegraded":"2024-09-16T09:25:44.302717+0000","last_fullsized":"2024-09-16T09:25:44.302717+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:03.217835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":42,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.085545+0000","last_change":"2024-09-16T09:25:05.717224+0000","last_active":"2024-09-16T09:25:44.085545+0000","last_peered":"2024-09-16T09:25:44.085545+0000","last_clean":"2024-09-16T09:25:44.085545+0000","last_became_active":"2024-09-16T09:25:05.716900+0000","last_became_peered":"2024-09-16T09:25:05.716900+0000","last_unstale":"2024-09-16T09:25:44.085545+0000","last_undegraded":"2024-09-16T09:25:44.085545+0000","last_fullsized":"2024-09-16T09:25:44.085545+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:30.026527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":38,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.604786+0000","last_change":"2024-09-16T09:25:07.721657+0000","last_active":"2024-09-16T09:25:44.604786+0000","last_peered":"2024-09-16T09:25:44.604786+0000","last_clean":"2024-09-16T09:25:44.604786+0000","last_became_active":"2024-09-16T09:25:07.721347+0000","last_became_peered":"2024-09-16T09:25:07.721347+0000","last_unstale":"2024-09-16T09:25:44.604786+0000","last_undegraded":"2024-09-16T09:25:44.604786+0000","last_fullsized":"2024-09-16T09:25:44.604786+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:15.094644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"66'10","reported_seq":61,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.085245+0000","last_change":"2024-09-16T09:25:03.708144+0000","last_active":"2024-09-16T09:25:44.085245+0000","last_peered":"2024-09-16T09:25:44.085245+0000","last_clean":"2024-09-16T09:25:44.085245+0000","last_became_active":"2024-09-16T09:25:03.707723+0000","last_became_peered":"2024-09-16T09:25:03.707723+0000","last_unstale":"2024-09-16T09:25:44.085245+0000","last_undegraded":"2024-09-16T09:25:44.085245+0000","last_fullsized":"2024-09-16T09:25:44.085245+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:24.890527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":50,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.114760+0000","last_change":"2024-09-16T09:25:01.703388+0000","last_active":"2024-09-16T09:25:44.114760+0000","last_peered":"2024-09-16T09:25:44.114760+0000","last_clean":"2024-09-16T09:25:44.114760+0000","last_became_active":"2024-09-16T09:25:01.703094+0000","last_became_peered":"2024-09-16T09:25:01.703094+0000","last_unstale":"2024-09-16T09:25:44.114760+0000","last_undegraded":"2024-09-16T09:25:44.114760+0000","last_fullsized":"2024-09-16T09:25:44.114760+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:07.049694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":65,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:51.759038+0000","last_change":"2024-09-16T09:25:05.711891+0000","last_active":"2024-09-16T09:25:51.759038+0000","last_peered":"2024-09-16T09:25:51.759038+0000","last_clean":"2024-09-16T09:25:51.759038+0000","last_became_active":"2024-09-16T09:25:05.711590+0000","last_became_peered":"2024-09-16T09:25:05.711590+0000","last_unstale":"2024-09-16T09:25:51.759038+0000","last_undegraded":"2024-09-16T09:25:51.759038+0000","last_fullsized":"2024-09-16T09:25:51.759038+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:17.630914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":38,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.085382+0000","last_change":"2024-09-16T09:25:07.719496+0000","last_active":"2024-09-16T09:25:44.085382+0000","last_peered":"2024-09-16T09:25:44.085382+0000","last_clean":"2024-09-16T09:25:44.085382+0000","last_became_active":"2024-09-16T09:25:07.718958+0000","last_became_peered":"2024-09-16T09:25:07.718958+0000","last_unstale":"2024-09-16T09:25:44.085382+0000","last_undegraded":"2024-09-16T09:25:44.085382+0000","last_fullsized":"2024-09-16T09:25:44.085382+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:15.393455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.b","version":"66'9","reported_seq":62,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.084946+0000","last_change":"2024-09-16T09:25:03.712553+0000","last_active":"2024-09-16T09:25:44.084946+0000","last_peered":"2024-09-16T09:25:44.084946+0000","last_clean":"2024-09-16T09:25:44.084946+0000","last_became_active":"2024-09-16T09:25:03.712230+0000","last_became_peered":"2024-09-16T09:25:03.712230+0000","last_unstale":"2024-09-16T09:25:44.084946+0000","last_undegraded":"2024-09-16T09:25:44.084946+0000","last_fullsized":"2024-09-16T09:25:44.084946+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:18.262732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":49,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.604847+0000","last_change":"2024-09-16T09:25:01.702943+0000","last_active":"2024-09-16T09:25:44.604847+0000","last_peered":"2024-09-16T09:25:44.604847+0000","last_clean":"2024-09-16T09:25:44.604847+0000","last_became_active":"2024-09-16T09:25:01.702629+0000","last_became_peered":"2024-09-16T09:25:01.702629+0000","last_unstale":"2024-09-16T09:25:44.604847+0000","last_undegraded":"2024-09-16T09:25:44.604847+0000","last_fullsized":"2024-09-16T09:25:44.604847+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:19.763325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":42,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.604193+0000","last_change":"2024-09-16T09:25:05.712932+0000","last_active":"2024-09-16T09:25:44.604193+0000","last_peered":"2024-09-16T09:25:44.604193+0000","last_clean":"2024-09-16T09:25:44.604193+0000","last_became_active":"2024-09-16T09:25:05.712573+0000","last_became_peered":"2024-09-16T09:25:05.712573+0000","last_unstale":"2024-09-16T09:25:44.604193+0000","last_undegraded":"2024-09-16T09:25:44.604193+0000","last_fullsized":"2024-09-16T09:25:44.604193+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:16.863815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":38,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.084536+0000","last_change":"2024-09-16T09:25:07.719073+0000","last_active":"2024-09-16T09:25:44.084536+0000","last_peered":"2024-09-16T09:25:44.084536+0000","last_clean":"2024-09-16T09:25:44.084536+0000","last_became_active":"2024-09-16T09:25:07.718778+0000","last_became_peered":"2024-09-16T09:25:07.718778+0000","last_unstale":"2024-09-16T09:25:44.084536+0000","last_undegraded":"2024-09-16T09:25:44.084536+0000","last_fullsized":"2024-09-16T09:25:44.084536+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:21.733588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"66'19","reported_seq":77,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.302519+0000","last_change":"2024-09-16T09:25:03.713182+0000","last_active":"2024-09-16T09:25:44.302519+0000","last_peered":"2024-09-16T09:25:44.302519+0000","last_clean":"2024-09-16T09:25:44.302519+0000","last_became_active":"2024-09-16T09:25:03.712783+0000","last_became_peered":"2024-09-16T09:25:03.712783+0000","last_unstale":"2024-09-16T09:25:44.302519+0000","last_undegraded":"2024-09-16T09:25:44.302519+0000","last_fullsized":"2024-09-16T09:25:44.302519+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:24.025685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"59'1","reported_seq":55,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.605881+0000","last_change":"2024-09-16T09:25:01.708571+0000","last_active":"2024-09-16T09:25:44.605881+0000","last_peered":"2024-09-16T09:25:44.605881+0000","last_clean":"2024-09-16T09:25:44.605881+0000","last_became_active":"2024-09-16T09:25:01.708069+0000","last_became_peered":"2024-09-16T09:25:01.708069+0000","last_unstale":"2024-09-16T09:25:44.605881+0000","last_undegraded":"2024-09-16T09:25:44.605881+0000","last_fullsized":"2024-09-16T09:25:44.605881+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:06.099334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":42,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.604167+0000","last_change":"2024-09-16T09:25:05.709549+0000","last_active":"2024-09-16T09:25:44.604167+0000","last_peered":"2024-09-16T09:25:44.604167+0000","last_clean":"2024-09-16T09:25:44.604167+0000","last_became_active":"2024-09-16T09:25:05.709264+0000","last_became_peered":"2024-09-16T09:25:05.709264+0000","last_unstale":"2024-09-16T09:25:44.604167+0000","last_undegraded":"2024-09-16T09:25:44.604167+0000","last_fullsized":"2024-09-16T09:25:44.604167+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:10.529230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":37,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.604361+0000","last_change":"2024-09-16T09:25:07.719009+0000","last_active":"2024-09-16T09:25:44.604361+0000","last_peered":"2024-09-16T09:25:44.604361+0000","last_clean":"2024-09-16T09:25:44.604361+0000","last_became_active":"2024-09-16T09:25:07.718750+0000","last_became_peered":"2024-09-16T09:25:07.718750+0000","last_unstale":"2024-09-16T09:25:44.604361+0000","last_undegraded":"2024-09-16T09:25:44.604361+0000","last_fullsized":"2024-09-16T09:25:44.604361+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:07.594257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"66'12","reported_seq":69,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.085147+0000","last_change":"2024-09-16T09:25:03.714309+0000","last_active":"2024-09-16T09:25:44.085147+0000","last_peered":"2024-09-16T09:25:44.085147+0000","last_clean":"2024-09-16T09:25:44.085147+0000","last_became_active":"2024-09-16T09:25:03.714027+0000","last_became_peered":"2024-09-16T09:25:03.714027+0000","last_unstale":"2024-09-16T09:25:44.085147+0000","last_undegraded":"2024-09-16T09:25:44.085147+0000","last_fullsized":"2024-09-16T09:25:44.085147+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:31.556984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":49,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.605893+0000","last_change":"2024-09-16T09:25:01.707905+0000","last_active":"2024-09-16T09:25:44.605893+0000","last_peered":"2024-09-16T09:25:44.605893+0000","last_clean":"2024-09-16T09:25:44.605893+0000","last_became_active":"2024-09-16T09:25:01.707528+0000","last_became_peered":"2024-09-16T09:25:01.707528+0000","last_unstale":"2024-09-16T09:25:44.605893+0000","last_undegraded":"2024-09-16T09:25:44.605893+0000","last_fullsized":"2024-09-16T09:25:44.605893+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:13.046169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":42,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.603975+0000","last_change":"2024-09-16T09:25:05.715584+0000","last_active":"2024-09-16T09:25:44.603975+0000","last_peered":"2024-09-16T09:25:44.603975+0000","last_clean":"2024-09-16T09:25:44.603975+0000","last_became_active":"2024-09-16T09:25:05.715253+0000","last_became_peered":"2024-09-16T09:25:05.715253+0000","last_unstale":"2024-09-16T09:25:44.603975+0000","last_undegraded":"2024-09-16T09:25:44.603975+0000","last_fullsized":"2024-09-16T09:25:44.603975+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:23.326757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":38,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.114558+0000","last_change":"2024-09-16T09:25:07.719338+0000","last_active":"2024-09-16T09:25:44.114558+0000","last_peered":"2024-09-16T09:25:44.114558+0000","last_clean":"2024-09-16T09:25:44.114558+0000","last_became_active":"2024-09-16T09:25:07.718899+0000","last_became_peered":"2024-09-16T09:25:07.718899+0000","last_unstale":"2024-09-16T09:25:44.114558+0000","last_undegraded":"2024-09-16T09:25:44.114558+0000","last_fullsized":"2024-09-16T09:25:44.114558+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:23.435495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.8","version":"66'15","reported_seq":70,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.603974+0000","last_change":"2024-09-16T09:25:03.716075+0000","last_active":"2024-09-16T09:25:44.603974+0000","last_peered":"2024-09-16T09:25:44.603974+0000","last_clean":"2024-09-16T09:25:44.603974+0000","last_became_active":"2024-09-16T09:25:03.715733+0000","last_became_peered":"2024-09-16T09:25:03.715733+0000","last_unstale":"2024-09-16T09:25:44.603974+0000","last_undegraded":"2024-09-16T09:25:44.603974+0000","last_fullsized":"2024-09-16T09:25:44.603974+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:20.577367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":61,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.605970+0000","last_change":"2024-09-16T09:25:01.707886+0000","last_active":"2024-09-16T09:25:44.605970+0000","last_peered":"2024-09-16T09:25:44.605970+0000","last_clean":"2024-09-16T09:25:44.605970+0000","last_became_active":"2024-09-16T09:25:01.707372+0000","last_became_peered":"2024-09-16T09:25:01.707372+0000","last_unstale":"2024-09-16T09:25:44.605970+0000","last_undegraded":"2024-09-16T09:25:44.605970+0000","last_fullsized":"2024-09-16T09:25:44.605970+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:14.328776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"66'8","reported_seq":61,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:51.759430+0000","last_change":"2024-09-16T09:25:05.716778+0000","last_active":"2024-09-16T09:25:51.759430+0000","last_peered":"2024-09-16T09:25:51.759430+0000","last_clean":"2024-09-16T09:25:51.759430+0000","last_became_active":"2024-09-16T09:25:05.716189+0000","last_became_peered":"2024-09-16T09:25:05.716189+0000","last_unstale":"2024-09-16T09:25:51.759430+0000","last_undegraded":"2024-09-16T09:25:51.759430+0000","last_fullsized":"2024-09-16T09:25:51.759430+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:31.128760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":37,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.604311+0000","last_change":"2024-09-16T09:25:07.717505+0000","last_active":"2024-09-16T09:25:44.604311+0000","last_peered":"2024-09-16T09:25:44.604311+0000","last_clean":"2024-09-16T09:25:44.604311+0000","last_became_active":"2024-09-16T09:25:07.717213+0000","last_became_peered":"2024-09-16T09:25:07.717213+0000","last_unstale":"2024-09-16T09:25:44.604311+0000","last_undegraded":"2024-09-16T09:25:44.604311+0000","last_fullsized":"2024-09-16T09:25:44.604311+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:09.355594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":50,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.302519+0000","last_change":"2024-09-16T09:25:01.703489+0000","last_active":"2024-09-16T09:25:44.302519+0000","last_peered":"2024-09-16T09:25:44.302519+0000","last_clean":"2024-09-16T09:25:44.302519+0000","last_became_active":"2024-09-16T09:25:01.703202+0000","last_became_peered":"2024-09-16T09:25:01.703202+0000","last_unstale":"2024-09-16T09:25:44.302519+0000","last_undegraded":"2024-09-16T09:25:44.302519+0000","last_fullsized":"2024-09-16T09:25:44.302519+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:30.581098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"66'6","reported_seq":55,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.115141+0000","last_change":"2024-09-16T09:25:03.713606+0000","last_active":"2024-09-16T09:25:44.115141+0000","last_peered":"2024-09-16T09:25:44.115141+0000","last_clean":"2024-09-16T09:25:44.115141+0000","last_became_active":"2024-09-16T09:25:03.713245+0000","last_became_peered":"2024-09-16T09:25:03.713245+0000","last_unstale":"2024-09-16T09:25:44.115141+0000","last_undegraded":"2024-09-16T09:25:44.115141+0000","last_fullsized":"2024-09-16T09:25:44.115141+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:15.800169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":41,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.604861+0000","last_change":"2024-09-16T09:25:05.715938+0000","last_active":"2024-09-16T09:25:44.604861+0000","last_peered":"2024-09-16T09:25:44.604861+0000","last_clean":"2024-09-16T09:25:44.604861+0000","last_became_active":"2024-09-16T09:25:05.715616+0000","last_became_peered":"2024-09-16T09:25:05.715616+0000","last_unstale":"2024-09-16T09:25:44.604861+0000","last_undegraded":"2024-09-16T09:25:44.604861+0000","last_fullsized":"2024-09-16T09:25:44.604861+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:25.224184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":37,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.605446+0000","last_change":"2024-09-16T09:25:07.721375+0000","last_active":"2024-09-16T09:25:44.605446+0000","last_peered":"2024-09-16T09:25:44.605446+0000","last_clean":"2024-09-16T09:25:44.605446+0000","last_became_active":"2024-09-16T09:25:07.721059+0000","last_became_peered":"2024-09-16T09:25:07.721059+0000","last_unstale":"2024-09-16T09:25:44.605446+0000","last_undegraded":"2024-09-16T09:25:44.605446+0000","last_fullsized":"2024-09-16T09:25:44.605446+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:16.275710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"66'9","reported_seq":62,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.085632+0000","last_change":"2024-09-16T09:25:03.719919+0000","last_active":"2024-09-16T09:25:44.085632+0000","last_peered":"2024-09-16T09:25:44.085632+0000","last_clean":"2024-09-16T09:25:44.085632+0000","last_became_active":"2024-09-16T09:25:03.719573+0000","last_became_peered":"2024-09-16T09:25:03.719573+0000","last_unstale":"2024-09-16T09:25:44.085632+0000","last_undegraded":"2024-09-16T09:25:44.085632+0000","last_fullsized":"2024-09-16T09:25:44.085632+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:23.239324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":49,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.605148+0000","last_change":"2024-09-16T09:25:01.707139+0000","last_active":"2024-09-16T09:25:44.605148+0000","last_peered":"2024-09-16T09:25:44.605148+0000","last_clean":"2024-09-16T09:25:44.605148+0000","last_became_active":"2024-09-16T09:25:01.706491+0000","last_became_peered":"2024-09-16T09:25:01.706491+0000","last_unstale":"2024-09-16T09:25:44.605148+0000","last_undegraded":"2024-09-16T09:25:44.605148+0000","last_fullsized":"2024-09-16T09:25:44.605148+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:07.438370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":42,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.115051+0000","last_change":"2024-09-16T09:25:05.715083+0000","last_active":"2024-09-16T09:25:44.115051+0000","last_peered":"2024-09-16T09:25:44.115051+0000","last_clean":"2024-09-16T09:25:44.115051+0000","last_became_active":"2024-09-16T09:25:05.714801+0000","last_became_peered":"2024-09-16T09:25:05.714801+0000","last_unstale":"2024-09-16T09:25:44.115051+0000","last_undegraded":"2024-09-16T09:25:44.115051+0000","last_fullsized":"2024-09-16T09:25:44.115051+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:15.084815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":38,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.604634+0000","last_change":"2024-09-16T09:25:07.722356+0000","last_active":"2024-09-16T09:25:44.604634+0000","last_peered":"2024-09-16T09:25:44.604634+0000","last_clean":"2024-09-16T09:25:44.604634+0000","last_became_active":"2024-09-16T09:25:07.722050+0000","last_became_peered":"2024-09-16T09:25:07.722050+0000","last_unstale":"2024-09-16T09:25:44.604634+0000","last_undegraded":"2024-09-16T09:25:44.604634+0000","last_fullsized":"2024-09-16T09:25:44.604634+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:35.793652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"66'9","reported_seq":61,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.605167+0000","last_change":"2024-09-16T09:25:03.717609+0000","last_active":"2024-09-16T09:25:44.605167+0000","last_peered":"2024-09-16T09:25:44.605167+0000","last_clean":"2024-09-16T09:25:44.605167+0000","last_became_active":"2024-09-16T09:25:03.717310+0000","last_became_peered":"2024-09-16T09:25:03.717310+0000","last_unstale":"2024-09-16T09:25:44.605167+0000","last_undegraded":"2024-09-16T09:25:44.605167+0000","last_fullsized":"2024-09-16T09:25:44.605167+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:27.281102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":50,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.084737+0000","last_change":"2024-09-16T09:25:01.702732+0000","last_active":"2024-09-16T09:25:44.084737+0000","last_peered":"2024-09-16T09:25:44.084737+0000","last_clean":"2024-09-16T09:25:44.084737+0000","last_became_active":"2024-09-16T09:25:01.702419+0000","last_became_peered":"2024-09-16T09:25:01.702419+0000","last_unstale":"2024-09-16T09:25:44.084737+0000","last_undegraded":"2024-09-16T09:25:44.084737+0000","last_fullsized":"2024-09-16T09:25:44.084737+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:11.390955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"66'8","reported_seq":62,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:51.759130+0000","last_change":"2024-09-16T09:25:05.711929+0000","last_active":"2024-09-16T09:25:51.759130+0000","last_peered":"2024-09-16T09:25:51.759130+0000","last_clean":"2024-09-16T09:25:51.759130+0000","last_became_active":"2024-09-16T09:25:05.711642+0000","last_became_peered":"2024-09-16T09:25:05.711642+0000","last_unstale":"2024-09-16T09:25:51.759130+0000","last_undegraded":"2024-09-16T09:25:51.759130+0000","last_fullsized":"2024-09-16T09:25:51.759130+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:29.307023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":38,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.085592+0000","last_change":"2024-09-16T09:25:07.719123+0000","last_active":"2024-09-16T09:25:44.085592+0000","last_peered":"2024-09-16T09:25:44.085592+0000","last_clean":"2024-09-16T09:25:44.085592+0000","last_became_active":"2024-09-16T09:25:07.718810+0000","last_became_peered":"2024-09-16T09:25:07.718810+0000","last_unstale":"2024-09-16T09:25:44.085592+0000","last_undegraded":"2024-09-16T09:25:44.085592+0000","last_fullsized":"2024-09-16T09:25:44.085592+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:16.185390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"66'10","reported_seq":61,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.114879+0000","last_change":"2024-09-16T09:25:03.712705+0000","last_active":"2024-09-16T09:25:44.114879+0000","last_peered":"2024-09-16T09:25:44.114879+0000","last_clean":"2024-09-16T09:25:44.114879+0000","last_became_active":"2024-09-16T09:25:03.712387+0000","last_became_peered":"2024-09-16T09:25:03.712387+0000","last_unstale":"2024-09-16T09:25:44.114879+0000","last_undegraded":"2024-09-16T09:25:44.114879+0000","last_fullsized":"2024-09-16T09:25:44.114879+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:26.552452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":49,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.605441+0000","last_change":"2024-09-16T09:25:01.708461+0000","last_active":"2024-09-16T09:25:44.605441+0000","last_peered":"2024-09-16T09:25:44.605441+0000","last_clean":"2024-09-16T09:25:44.605441+0000","last_became_active":"2024-09-16T09:25:01.707513+0000","last_became_peered":"2024-09-16T09:25:01.707513+0000","last_unstale":"2024-09-16T09:25:44.605441+0000","last_undegraded":"2024-09-16T09:25:44.605441+0000","last_fullsized":"2024-09-16T09:25:44.605441+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:10.338638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"66'8","reported_seq":60,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:46.758166+0000","last_change":"2024-09-16T09:25:05.714214+0000","last_active":"2024-09-16T09:25:46.758166+0000","last_peered":"2024-09-16T09:25:46.758166+0000","last_clean":"2024-09-16T09:25:46.758166+0000","last_became_active":"2024-09-16T09:25:05.713833+0000","last_became_peered":"2024-09-16T09:25:05.713833+0000","last_unstale":"2024-09-16T09:25:46.758166+0000","last_undegraded":"2024-09-16T09:25:46.758166+0000","last_fullsized":"2024-09-16T09:25:46.758166+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:09.052011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":38,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.085233+0000","last_change":"2024-09-16T09:25:07.717629+0000","last_active":"2024-09-16T09:25:44.085233+0000","last_peered":"2024-09-16T09:25:44.085233+0000","last_clean":"2024-09-16T09:25:44.085233+0000","last_became_active":"2024-09-16T09:25:07.717357+0000","last_became_peered":"2024-09-16T09:25:07.717357+0000","last_unstale":"2024-09-16T09:25:44.085233+0000","last_undegraded":"2024-09-16T09:25:44.085233+0000","last_fullsized":"2024-09-16T09:25:44.085233+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:28.965425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"66'11","reported_seq":65,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.085362+0000","last_change":"2024-09-16T09:25:03.715601+0000","last_active":"2024-09-16T09:25:44.085362+0000","last_peered":"2024-09-16T09:25:44.085362+0000","last_clean":"2024-09-16T09:25:44.085362+0000","last_became_active":"2024-09-16T09:25:03.715333+0000","last_became_peered":"2024-09-16T09:25:03.715333+0000","last_unstale":"2024-09-16T09:25:44.085362+0000","last_undegraded":"2024-09-16T09:25:44.085362+0000","last_fullsized":"2024-09-16T09:25:44.085362+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:05.154887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":50,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.085723+0000","last_change":"2024-09-16T09:25:01.705459+0000","last_active":"2024-09-16T09:25:44.085723+0000","last_peered":"2024-09-16T09:25:44.085723+0000","last_clean":"2024-09-16T09:25:44.085723+0000","last_became_active":"2024-09-16T09:25:01.705158+0000","last_became_peered":"2024-09-16T09:25:01.705158+0000","last_unstale":"2024-09-16T09:25:44.085723+0000","last_undegraded":"2024-09-16T09:25:44.085723+0000","last_fullsized":"2024-09-16T09:25:44.085723+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:17.236788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":42,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.085997+0000","last_change":"2024-09-16T09:25:05.713087+0000","last_active":"2024-09-16T09:25:44.085997+0000","last_peered":"2024-09-16T09:25:44.085997+0000","last_clean":"2024-09-16T09:25:44.085997+0000","last_became_active":"2024-09-16T09:25:05.712816+0000","last_became_peered":"2024-09-16T09:25:05.712816+0000","last_unstale":"2024-09-16T09:25:44.085997+0000","last_undegraded":"2024-09-16T09:25:44.085997+0000","last_fullsized":"2024-09-16T09:25:44.085997+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:29.267320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":38,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.114352+0000","last_change":"2024-09-16T09:25:07.713617+0000","last_active":"2024-09-16T09:25:44.114352+0000","last_peered":"2024-09-16T09:25:44.114352+0000","last_clean":"2024-09-16T09:25:44.114352+0000","last_became_active":"2024-09-16T09:25:07.713109+0000","last_became_peered":"2024-09-16T09:25:07.713109+0000","last_unstale":"2024-09-16T09:25:44.114352+0000","last_undegraded":"2024-09-16T09:25:44.114352+0000","last_fullsized":"2024-09-16T09:25:44.114352+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:17.988989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"66'9","reported_seq":62,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.086227+0000","last_change":"2024-09-16T09:25:03.706126+0000","last_active":"2024-09-16T09:25:44.086227+0000","last_peered":"2024-09-16T09:25:44.086227+0000","last_clean":"2024-09-16T09:25:44.086227+0000","last_became_active":"2024-09-16T09:25:03.705807+0000","last_became_peered":"2024-09-16T09:25:03.705807+0000","last_unstale":"2024-09-16T09:25:44.086227+0000","last_undegraded":"2024-09-16T09:25:44.086227+0000","last_fullsized":"2024-09-16T09:25:44.086227+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:25.655887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":49,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.605797+0000","last_change":"2024-09-16T09:25:01.707075+0000","last_active":"2024-09-16T09:25:44.605797+0000","last_peered":"2024-09-16T09:25:44.605797+0000","last_clean":"2024-09-16T09:25:44.605797+0000","last_became_active":"2024-09-16T09:25:01.706762+0000","last_became_peered":"2024-09-16T09:25:01.706762+0000","last_unstale":"2024-09-16T09:25:44.605797+0000","last_undegraded":"2024-09-16T09:25:44.605797+0000","last_fullsized":"2024-09-16T09:25:44.605797+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:13.882359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":42,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.114580+0000","last_change":"2024-09-16T09:25:05.714142+0000","last_active":"2024-09-16T09:25:44.114580+0000","last_peered":"2024-09-16T09:25:44.114580+0000","last_clean":"2024-09-16T09:25:44.114580+0000","last_became_active":"2024-09-16T09:25:05.713791+0000","last_became_peered":"2024-09-16T09:25:05.713791+0000","last_unstale":"2024-09-16T09:25:44.114580+0000","last_undegraded":"2024-09-16T09:25:44.114580+0000","last_fullsized":"2024-09-16T09:25:44.114580+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:20.280445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":38,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.084170+0000","last_change":"2024-09-16T09:25:07.715572+0000","last_active":"2024-09-16T09:25:44.084170+0000","last_peered":"2024-09-16T09:25:44.084170+0000","last_clean":"2024-09-16T09:25:44.084170+0000","last_became_active":"2024-09-16T09:25:07.715301+0000","last_became_peered":"2024-09-16T09:25:07.715301+0000","last_unstale":"2024-09-16T09:25:44.084170+0000","last_undegraded":"2024-09-16T09:25:44.084170+0000","last_fullsized":"2024-09-16T09:25:44.084170+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:15.062673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"66'11","reported_seq":65,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.114498+0000","last_change":"2024-09-16T09:25:03.712637+0000","last_active":"2024-09-16T09:25:44.114498+0000","last_peered":"2024-09-16T09:25:44.114498+0000","last_clean":"2024-09-16T09:25:44.114498+0000","last_became_active":"2024-09-16T09:25:03.712333+0000","last_became_peered":"2024-09-16T09:25:03.712333+0000","last_unstale":"2024-09-16T09:25:44.114498+0000","last_undegraded":"2024-09-16T09:25:44.114498+0000","last_fullsized":"2024-09-16T09:25:44.114498+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:06.593958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":49,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.604955+0000","last_change":"2024-09-16T09:25:01.701295+0000","last_active":"2024-09-16T09:25:44.604955+0000","last_peered":"2024-09-16T09:25:44.604955+0000","last_clean":"2024-09-16T09:25:44.604955+0000","last_became_active":"2024-09-16T09:25:01.700733+0000","last_became_peered":"2024-09-16T09:25:01.700733+0000","last_unstale":"2024-09-16T09:25:44.604955+0000","last_undegraded":"2024-09-16T09:25:44.604955+0000","last_fullsized":"2024-09-16T09:25:44.604955+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:14.332112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":41,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.604177+0000","last_change":"2024-09-16T09:25:05.713760+0000","last_active":"2024-09-16T09:25:44.604177+0000","last_peered":"2024-09-16T09:25:44.604177+0000","last_clean":"2024-09-16T09:25:44.604177+0000","last_became_active":"2024-09-16T09:25:05.713479+0000","last_became_peered":"2024-09-16T09:25:05.713479+0000","last_unstale":"2024-09-16T09:25:44.604177+0000","last_undegraded":"2024-09-16T09:25:44.604177+0000","last_fullsized":"2024-09-16T09:25:44.604177+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:31.316771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":38,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.114700+0000","last_change":"2024-09-16T09:25:07.715946+0000","last_active":"2024-09-16T09:25:44.114700+0000","last_peered":"2024-09-16T09:25:44.114700+0000","last_clean":"2024-09-16T09:25:44.114700+0000","last_became_active":"2024-09-16T09:25:07.715582+0000","last_became_peered":"2024-09-16T09:25:07.715582+0000","last_unstale":"2024-09-16T09:25:44.114700+0000","last_undegraded":"2024-09-16T09:25:44.114700+0000","last_fullsized":"2024-09-16T09:25:44.114700+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:18.541254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"66'4","reported_seq":52,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.114329+0000","last_change":"2024-09-16T09:25:03.705167+0000","last_active":"2024-09-16T09:25:44.114329+0000","last_peered":"2024-09-16T09:25:44.114329+0000","last_clean":"2024-09-16T09:25:44.114329+0000","last_became_active":"2024-09-16T09:25:03.704869+0000","last_became_peered":"2024-09-16T09:25:03.704869+0000","last_unstale":"2024-09-16T09:25:44.114329+0000","last_undegraded":"2024-09-16T09:25:44.114329+0000","last_fullsized":"2024-09-16T09:25:44.114329+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:17.043143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":50,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.085135+0000","last_change":"2024-09-16T09:25:01.705965+0000","last_active":"2024-09-16T09:25:44.085135+0000","last_peered":"2024-09-16T09:25:44.085135+0000","last_clean":"2024-09-16T09:25:44.085135+0000","last_became_active":"2024-09-16T09:25:01.705686+0000","last_became_peered":"2024-09-16T09:25:01.705686+0000","last_unstale":"2024-09-16T09:25:44.085135+0000","last_undegraded":"2024-09-16T09:25:44.085135+0000","last_fullsized":"2024-09-16T09:25:44.085135+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:16.022170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":42,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.302482+0000","last_change":"2024-09-16T09:25:05.712816+0000","last_active":"2024-09-16T09:25:44.302482+0000","last_peered":"2024-09-16T09:25:44.302482+0000","last_clean":"2024-09-16T09:25:44.302482+0000","last_became_active":"2024-09-16T09:25:05.712545+0000","last_became_peered":"2024-09-16T09:25:05.712545+0000","last_unstale":"2024-09-16T09:25:44.302482+0000","last_undegraded":"2024-09-16T09:25:44.302482+0000","last_fullsized":"2024-09-16T09:25:44.302482+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:08.170532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":37,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.605000+0000","last_change":"2024-09-16T09:25:07.720368+0000","last_active":"2024-09-16T09:25:44.605000+0000","last_peered":"2024-09-16T09:25:44.605000+0000","last_clean":"2024-09-16T09:25:44.605000+0000","last_became_active":"2024-09-16T09:25:07.720117+0000","last_became_peered":"2024-09-16T09:25:07.720117+0000","last_unstale":"2024-09-16T09:25:44.605000+0000","last_undegraded":"2024-09-16T09:25:44.605000+0000","last_fullsized":"2024-09-16T09:25:44.605000+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:31.504626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":38,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.084855+0000","last_change":"2024-09-16T09:25:07.712880+0000","last_active":"2024-09-16T09:25:44.084855+0000","last_peered":"2024-09-16T09:25:44.084855+0000","last_clean":"2024-09-16T09:25:44.084855+0000","last_became_active":"2024-09-16T09:25:07.712594+0000","last_became_peered":"2024-09-16T09:25:07.712594+0000","last_unstale":"2024-09-16T09:25:44.084855+0000","last_undegraded":"2024-09-16T09:25:44.084855+0000","last_fullsized":"2024-09-16T09:25:44.084855+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:33.865572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":50,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.114495+0000","last_change":"2024-09-16T09:25:01.704711+0000","last_active":"2024-09-16T09:25:44.114495+0000","last_peered":"2024-09-16T09:25:44.114495+0000","last_clean":"2024-09-16T09:25:44.114495+0000","last_became_active":"2024-09-16T09:25:01.703736+0000","last_became_peered":"2024-09-16T09:25:01.703736+0000","last_unstale":"2024-09-16T09:25:44.114495+0000","last_undegraded":"2024-09-16T09:25:44.114495+0000","last_fullsized":"2024-09-16T09:25:44.114495+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:09.913011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"66'11","reported_seq":65,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.303094+0000","last_change":"2024-09-16T09:25:03.708095+0000","last_active":"2024-09-16T09:25:44.303094+0000","last_peered":"2024-09-16T09:25:44.303094+0000","last_clean":"2024-09-16T09:25:44.303094+0000","last_became_active":"2024-09-16T09:25:03.707769+0000","last_became_peered":"2024-09-16T09:25:03.707769+0000","last_unstale":"2024-09-16T09:25:44.303094+0000","last_undegraded":"2024-09-16T09:25:44.303094+0000","last_fullsized":"2024-09-16T09:25:44.303094+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:21.606932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":42,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-16T09:25:44.085370+0000","last_change":"2024-09-16T09:25:05.712247+0000","last_active":"2024-09-16T09:25:44.085370+0000","last_peered":"2024-09-16T09:25:44.085370+0000","last_clean":"2024-09-16T09:25:44.085370+0000","last_became_active":"2024-09-16T09:25:05.711952+0000","last_became_peered":"2024-09-16T09:25:05.711952+0000","last_unstale":"2024-09-16T09:25:44.085370+0000","last_undegraded":"2024-09-16T09:25:44.085370+0000","last_fullsized":"2024-09-16T09:25:44.085370+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:30.229560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":393,"ondisk_log_size":393,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":50,"num_read_kb":45,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266704,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32632,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712520,"statfs":{"total":95995035648,"available":95961620480,"internally_reserved":0,"allocated":1478656,"data_stored":977101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495636,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28448,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716704,"statfs":{"total":95995035648,"available":95965904896,"internally_reserved":0,"allocated":1454080,"data_stored":976003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724571,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":864256,"data_stored":385994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953504,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":888832,"data_stored":386797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149735,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32064,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713088,"statfs":{"total":95995035648,"available":95962202112,"internally_reserved":0,"allocated":897024,"data_stored":387083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411372,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32020,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713132,"statfs":{"total":95995035648,"available":95962247168,"internally_reserved":0,"allocated":851968,"data_stored":385347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574899,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32092,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713060,"statfs":{"total":95995035648,"available":95962173440,"internally_reserved":0,"allocated":925696,"data_stored":386997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738424,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28476,"kb_used_data":1448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716676,"statfs":{"total":95995035648,"available":95965876224,"internally_reserved":0,"allocated":1482752,"data_stored":977367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T09:25:58.505 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-16T09:25:58.505 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd dump 2024-09-16T09:25:58.754 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:25:59.170 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Sep 16 09:25:58 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-alertmanager-a[75718]: ts=2024-09-16T09:25:58.916Z caller=cluster.go:708 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000267268s 2024-09-16T09:25:59.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:59 smithi070 ceph-mon[30388]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-16T09:25:59.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:59 smithi070 ceph-mon[30388]: pgmap v21: 132 pgs: 132 active+clean; 583 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:25:59.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:59 smithi070 ceph-mon[30388]: Deploying daemon grafana.a on smithi084 2024-09-16T09:25:59.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:59 smithi070 ceph-mon[30388]: from='client.24635 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:25:59.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:25:59 smithi070 ceph-mon[30388]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-16T09:25:59.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:59 smithi070 ceph-mon[40691]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-16T09:25:59.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:59 smithi070 ceph-mon[40691]: pgmap v21: 132 pgs: 132 active+clean; 583 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:25:59.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:59 smithi070 ceph-mon[40691]: Deploying daemon grafana.a on smithi084 2024-09-16T09:25:59.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:59 smithi070 ceph-mon[40691]: from='client.24635 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:25:59.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:25:59 smithi070 ceph-mon[40691]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-16T09:25:59.371 INFO:teuthology.orchestra.run.smithi070.stdout:epoch 78 2024-09-16T09:25:59.371 INFO:teuthology.orchestra.run.smithi070.stdout:fsid a1321834-740c-11ef-bceb-c7b262605968 2024-09-16T09:25:59.372 INFO:teuthology.orchestra.run.smithi070.stdout:created 2024-09-16T09:19:14.121498+0000 2024-09-16T09:25:59.372 INFO:teuthology.orchestra.run.smithi070.stdout:modified 2024-09-16T09:25:44.081691+0000 2024-09-16T09:25:59.372 INFO:teuthology.orchestra.run.smithi070.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-16T09:25:59.372 INFO:teuthology.orchestra.run.smithi070.stdout:crush_version 18 2024-09-16T09:25:59.372 INFO:teuthology.orchestra.run.smithi070.stdout:full_ratio 0.95 2024-09-16T09:25:59.372 INFO:teuthology.orchestra.run.smithi070.stdout:backfillfull_ratio 0.9 2024-09-16T09:25:59.372 INFO:teuthology.orchestra.run.smithi070.stdout:nearfull_ratio 0.85 2024-09-16T09:25:59.372 INFO:teuthology.orchestra.run.smithi070.stdout:require_min_compat_client luminous 2024-09-16T09:25:59.372 INFO:teuthology.orchestra.run.smithi070.stdout:min_compat_client jewel 2024-09-16T09:25:59.372 INFO:teuthology.orchestra.run.smithi070.stdout:require_osd_release squid 2024-09-16T09:25:59.372 INFO:teuthology.orchestra.run.smithi070.stdout:stretch_mode_enabled false 2024-09-16T09:25:59.372 INFO:teuthology.orchestra.run.smithi070.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 7.89 2024-09-16T09:25:59.373 INFO:teuthology.orchestra.run.smithi070.stdout:pool 2 'datapool' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 3 pgp_num 3 autoscale_mode off last_change 61 flags hashpspool,selfmanaged_snaps stripe_width 0 application rbd read_balance_score 2.65 2024-09-16T09:25:59.373 INFO:teuthology.orchestra.run.smithi070.stdout:pool 3 '.rgw.root' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 60 flags hashpspool stripe_width 0 application rgw read_balance_score 1.50 2024-09-16T09:25:59.373 INFO:teuthology.orchestra.run.smithi070.stdout:pool 4 'default.rgw.log' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 62 flags hashpspool stripe_width 0 application rgw read_balance_score 2.25 2024-09-16T09:25:59.373 INFO:teuthology.orchestra.run.smithi070.stdout:pool 5 'default.rgw.control' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 64 flags hashpspool stripe_width 0 application rgw read_balance_score 1.25 2024-09-16T09:25:59.373 INFO:teuthology.orchestra.run.smithi070.stdout:pool 6 'default.rgw.meta' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 66 flags hashpspool stripe_width 0 pg_autoscale_bias 4 application rgw read_balance_score 1.75 2024-09-16T09:25:59.373 INFO:teuthology.orchestra.run.smithi070.stdout:max_osd 8 2024-09-16T09:25:59.373 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0 up in weight 1 up_from 8 up_thru 64 down_at 0 last_clean_interval [0,0) [v2:172.21.15.70:6802/2013075781,v1:172.21.15.70:6803/2013075781] [v2:172.21.15.70:6804/2013075781,v1:172.21.15.70:6805/2013075781] exists,up 7ef288c8-3bf0-4814-8e86-f70a7de041ce 2024-09-16T09:25:59.373 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1 up in weight 1 up_from 13 up_thru 64 down_at 0 last_clean_interval [0,0) [v2:172.21.15.70:6810/1670012115,v1:172.21.15.70:6811/1670012115] [v2:172.21.15.70:6812/1670012115,v1:172.21.15.70:6813/1670012115] exists,up 474d41d9-295a-450c-8f27-ce487a14678b 2024-09-16T09:25:59.373 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2 up in weight 1 up_from 18 up_thru 64 down_at 0 last_clean_interval [0,0) [v2:172.21.15.70:6818/3601682336,v1:172.21.15.70:6819/3601682336] [v2:172.21.15.70:6820/3601682336,v1:172.21.15.70:6821/3601682336] exists,up 5b4ca4e8-cecc-4163-b291-c9209b02c4b7 2024-09-16T09:25:59.373 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3 up in weight 1 up_from 26 up_thru 64 down_at 0 last_clean_interval [0,0) [v2:172.21.15.70:6826/2038083007,v1:172.21.15.70:6827/2038083007] [v2:172.21.15.70:6828/2038083007,v1:172.21.15.70:6829/2038083007] exists,up ffb8d527-1b58-4ecd-808c-47d4ba1bf387 2024-09-16T09:25:59.373 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4 up in weight 1 up_from 32 up_thru 64 down_at 0 last_clean_interval [0,0) [v2:172.21.15.84:6800/2250892710,v1:172.21.15.84:6801/2250892710] [v2:172.21.15.84:6802/2250892710,v1:172.21.15.84:6803/2250892710] exists,up a946760b-4d91-4a4b-8e9b-a07a50589211 2024-09-16T09:25:59.374 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5 up in weight 1 up_from 39 up_thru 64 down_at 0 last_clean_interval [0,0) [v2:172.21.15.84:6808/1803945348,v1:172.21.15.84:6809/1803945348] [v2:172.21.15.84:6810/1803945348,v1:172.21.15.84:6811/1803945348] exists,up 67681760-75a9-4846-946e-a935d42c5e05 2024-09-16T09:25:59.374 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6 up in weight 1 up_from 46 up_thru 62 down_at 0 last_clean_interval [0,0) [v2:172.21.15.84:6816/610727189,v1:172.21.15.84:6817/610727189] [v2:172.21.15.84:6818/610727189,v1:172.21.15.84:6819/610727189] exists,up 7c8f0152-f539-4b7e-8113-91dd7529d3a1 2024-09-16T09:25:59.374 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7 up in weight 1 up_from 53 up_thru 64 down_at 0 last_clean_interval [0,0) [v2:172.21.15.84:6824/1002252531,v1:172.21.15.84:6825/1002252531] [v2:172.21.15.84:6826/1002252531,v1:172.21.15.84:6827/1002252531] exists,up 945efaba-225e-49af-aefd-b4d58ca8093b 2024-09-16T09:25:59.374 INFO:teuthology.orchestra.run.smithi070.stdout:blocklist 172.21.15.70:0/125491368 expires 2024-09-17T09:25:31.465167+0000 2024-09-16T09:25:59.374 INFO:teuthology.orchestra.run.smithi070.stdout:blocklist 172.21.15.70:0/3354284852 expires 2024-09-17T09:19:43.205989+0000 2024-09-16T09:25:59.374 INFO:teuthology.orchestra.run.smithi070.stdout:blocklist 172.21.15.70:0/3284625547 expires 2024-09-17T09:25:31.465167+0000 2024-09-16T09:25:59.374 INFO:teuthology.orchestra.run.smithi070.stdout:blocklist 172.21.15.70:0/186565817 expires 2024-09-17T09:19:43.205989+0000 2024-09-16T09:25:59.374 INFO:teuthology.orchestra.run.smithi070.stdout:blocklist 172.21.15.70:0/2213852865 expires 2024-09-17T09:19:43.205989+0000 2024-09-16T09:25:59.374 INFO:teuthology.orchestra.run.smithi070.stdout:blocklist 172.21.15.70:0/1812373760 expires 2024-09-17T09:20:09.306330+0000 2024-09-16T09:25:59.374 INFO:teuthology.orchestra.run.smithi070.stdout:blocklist 172.21.15.70:6800/837338094 expires 2024-09-17T09:19:43.205989+0000 2024-09-16T09:25:59.374 INFO:teuthology.orchestra.run.smithi070.stdout:blocklist 172.21.15.70:6800/1876661229 expires 2024-09-17T09:25:31.465167+0000 2024-09-16T09:25:59.374 INFO:teuthology.orchestra.run.smithi070.stdout:blocklist 172.21.15.70:0/3351688485 expires 2024-09-17T09:20:09.306330+0000 2024-09-16T09:25:59.374 INFO:teuthology.orchestra.run.smithi070.stdout:blocklist 172.21.15.70:6800/3871553358 expires 2024-09-17T09:20:09.306330+0000 2024-09-16T09:25:59.375 INFO:teuthology.orchestra.run.smithi070.stdout:blocklist 172.21.15.70:0/1230482871 expires 2024-09-17T09:25:31.465167+0000 2024-09-16T09:25:59.375 INFO:teuthology.orchestra.run.smithi070.stdout:blocklist 172.21.15.70:6801/837338094 expires 2024-09-17T09:19:43.205989+0000 2024-09-16T09:25:59.375 INFO:teuthology.orchestra.run.smithi070.stdout:blocklist 172.21.15.70:6801/3871553358 expires 2024-09-17T09:20:09.306330+0000 2024-09-16T09:25:59.375 INFO:teuthology.orchestra.run.smithi070.stdout:blocklist 172.21.15.70:0/1403001486 expires 2024-09-17T09:20:09.306330+0000 2024-09-16T09:25:59.375 INFO:teuthology.orchestra.run.smithi070.stdout:blocklist 172.21.15.70:0/1723643739 expires 2024-09-17T09:25:31.465167+0000 2024-09-16T09:25:59.375 INFO:teuthology.orchestra.run.smithi070.stdout:blocklist 172.21.15.70:6801/1876661229 expires 2024-09-17T09:25:31.465167+0000 2024-09-16T09:25:59.375 INFO:teuthology.orchestra.run.smithi070.stdout:blocklist 172.21.15.70:0/3456417396 expires 2024-09-17T09:25:31.465167+0000 2024-09-16T09:25:59.409 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:59 smithi084 ceph-mon[32629]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-16T09:25:59.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:59 smithi084 ceph-mon[32629]: pgmap v21: 132 pgs: 132 active+clean; 583 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:25:59.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:59 smithi084 ceph-mon[32629]: Deploying daemon grafana.a on smithi084 2024-09-16T09:25:59.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:59 smithi084 ceph-mon[32629]: from='client.24635 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:25:59.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:25:59 smithi084 ceph-mon[32629]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-16T09:26:00.037 INFO:tasks.cephadm.ceph_manager.ceph:['osd.0 up in weight 1 up_from 8 up_thru 64 down_at 0 last_clean_interval [0,0) [v2:172.21.15.70:6802/2013075781,v1:172.21.15.70:6803/2013075781] [v2:172.21.15.70:6804/2013075781,v1:172.21.15.70:6805/2013075781] exists,up 7ef288c8-3bf0-4814-8e86-f70a7de041ce', 'osd.1 up in weight 1 up_from 13 up_thru 64 down_at 0 last_clean_interval [0,0) [v2:172.21.15.70:6810/1670012115,v1:172.21.15.70:6811/1670012115] [v2:172.21.15.70:6812/1670012115,v1:172.21.15.70:6813/1670012115] exists,up 474d41d9-295a-450c-8f27-ce487a14678b', 'osd.2 up in weight 1 up_from 18 up_thru 64 down_at 0 last_clean_interval [0,0) [v2:172.21.15.70:6818/3601682336,v1:172.21.15.70:6819/3601682336] [v2:172.21.15.70:6820/3601682336,v1:172.21.15.70:6821/3601682336] exists,up 5b4ca4e8-cecc-4163-b291-c9209b02c4b7', 'osd.3 up in weight 1 up_from 26 up_thru 64 down_at 0 last_clean_interval [0,0) [v2:172.21.15.70:6826/2038083007,v1:172.21.15.70:6827/2038083007] [v2:172.21.15.70:6828/2038083007,v1:172.21.15.70:6829/2038083007] exists,up ffb8d527-1b58-4ecd-808c-47d4ba1bf387', 'osd.4 up in weight 1 up_from 32 up_thru 64 down_at 0 last_clean_interval [0,0) [v2:172.21.15.84:6800/2250892710,v1:172.21.15.84:6801/2250892710] [v2:172.21.15.84:6802/2250892710,v1:172.21.15.84:6803/2250892710] exists,up a946760b-4d91-4a4b-8e9b-a07a50589211', 'osd.5 up in weight 1 up_from 39 up_thru 64 down_at 0 last_clean_interval [0,0) [v2:172.21.15.84:6808/1803945348,v1:172.21.15.84:6809/1803945348] [v2:172.21.15.84:6810/1803945348,v1:172.21.15.84:6811/1803945348] exists,up 67681760-75a9-4846-946e-a935d42c5e05', 'osd.6 up in weight 1 up_from 46 up_thru 62 down_at 0 last_clean_interval [0,0) [v2:172.21.15.84:6816/610727189,v1:172.21.15.84:6817/610727189] [v2:172.21.15.84:6818/610727189,v1:172.21.15.84:6819/610727189] exists,up 7c8f0152-f539-4b7e-8113-91dd7529d3a1', 'osd.7 up in weight 1 up_from 53 up_thru 64 down_at 0 last_clean_interval [0,0) [v2:172.21.15.84:6824/1002252531,v1:172.21.15.84:6825/1002252531] [v2:172.21.15.84:6826/1002252531,v1:172.21.15.84:6827/1002252531] exists,up 945efaba-225e-49af-aefd-b4d58ca8093b'] 2024-09-16T09:26:00.038 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon mon.a config show 2024-09-16T09:26:00.287 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:26:00.407 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:00 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/184277433' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-16T09:26:00.408 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:00 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/184277433' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-16T09:26:00.409 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:00 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/184277433' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-16T09:26:00.790 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T09:26:00.790 INFO:teuthology.orchestra.run.smithi070.stdout: "name": "mon.a", 2024-09-16T09:26:00.790 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster": "ceph", 2024-09-16T09:26:00.790 INFO:teuthology.orchestra.run.smithi070.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2024-09-16T09:26:00.790 INFO:teuthology.orchestra.run.smithi070.stdout: "admin_socket_mode": "", 2024-09-16T09:26:00.790 INFO:teuthology.orchestra.run.smithi070.stdout: "auth_allow_insecure_global_id_reclaim": "false", 2024-09-16T09:26:00.791 INFO:teuthology.orchestra.run.smithi070.stdout: "auth_client_required": "cephx, none", 2024-09-16T09:26:00.791 INFO:teuthology.orchestra.run.smithi070.stdout: "auth_cluster_required": "cephx", 2024-09-16T09:26:00.791 INFO:teuthology.orchestra.run.smithi070.stdout: "auth_debug": "true", 2024-09-16T09:26:00.791 INFO:teuthology.orchestra.run.smithi070.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2024-09-16T09:26:00.791 INFO:teuthology.orchestra.run.smithi070.stdout: "auth_mon_ticket_ttl": "660.000000", 2024-09-16T09:26:00.791 INFO:teuthology.orchestra.run.smithi070.stdout: "auth_service_required": "cephx", 2024-09-16T09:26:00.791 INFO:teuthology.orchestra.run.smithi070.stdout: "auth_service_ticket_ttl": "240.000000", 2024-09-16T09:26:00.791 INFO:teuthology.orchestra.run.smithi070.stdout: "auth_supported": "", 2024-09-16T09:26:00.791 INFO:teuthology.orchestra.run.smithi070.stdout: "bdev_aio": "true", 2024-09-16T09:26:00.791 INFO:teuthology.orchestra.run.smithi070.stdout: "bdev_aio_max_queue_depth": "1024", 2024-09-16T09:26:00.791 INFO:teuthology.orchestra.run.smithi070.stdout: "bdev_aio_poll_ms": "250", 2024-09-16T09:26:00.791 INFO:teuthology.orchestra.run.smithi070.stdout: "bdev_aio_reap_max": "16", 2024-09-16T09:26:00.791 INFO:teuthology.orchestra.run.smithi070.stdout: "bdev_async_discard_threads": "0", 2024-09-16T09:26:00.792 INFO:teuthology.orchestra.run.smithi070.stdout: "bdev_block_size": "4096", 2024-09-16T09:26:00.792 INFO:teuthology.orchestra.run.smithi070.stdout: "bdev_debug_aio": "false", 2024-09-16T09:26:00.792 INFO:teuthology.orchestra.run.smithi070.stdout: "bdev_debug_aio_log_age": "5.000000", 2024-09-16T09:26:00.792 INFO:teuthology.orchestra.run.smithi070.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2024-09-16T09:26:00.792 INFO:teuthology.orchestra.run.smithi070.stdout: "bdev_debug_inflight_ios": "false", 2024-09-16T09:26:00.792 INFO:teuthology.orchestra.run.smithi070.stdout: "bdev_enable_discard": "false", 2024-09-16T09:26:00.792 INFO:teuthology.orchestra.run.smithi070.stdout: "bdev_flock_retry": "3", 2024-09-16T09:26:00.792 INFO:teuthology.orchestra.run.smithi070.stdout: "bdev_flock_retry_interval": "0.100000", 2024-09-16T09:26:00.792 INFO:teuthology.orchestra.run.smithi070.stdout: "bdev_inject_crash": "0", 2024-09-16T09:26:00.792 INFO:teuthology.orchestra.run.smithi070.stdout: "bdev_inject_crash_flush_delay": "2", 2024-09-16T09:26:00.792 INFO:teuthology.orchestra.run.smithi070.stdout: "bdev_ioring": "false", 2024-09-16T09:26:00.792 INFO:teuthology.orchestra.run.smithi070.stdout: "bdev_ioring_hipri": "false", 2024-09-16T09:26:00.792 INFO:teuthology.orchestra.run.smithi070.stdout: "bdev_ioring_sqthread_poll": "false", 2024-09-16T09:26:00.793 INFO:teuthology.orchestra.run.smithi070.stdout: "bdev_nvme_unbind_from_kernel": "false", 2024-09-16T09:26:00.793 INFO:teuthology.orchestra.run.smithi070.stdout: "bdev_read_buffer_alignment": "4096", 2024-09-16T09:26:00.793 INFO:teuthology.orchestra.run.smithi070.stdout: "bdev_read_preallocated_huge_buffers": "", 2024-09-16T09:26:00.793 INFO:teuthology.orchestra.run.smithi070.stdout: "bdev_stalled_read_warn_lifetime": "86400", 2024-09-16T09:26:00.793 INFO:teuthology.orchestra.run.smithi070.stdout: "bdev_stalled_read_warn_threshold": "1", 2024-09-16T09:26:00.793 INFO:teuthology.orchestra.run.smithi070.stdout: "bdev_type": "", 2024-09-16T09:26:00.793 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_alloc_size": "1048576", 2024-09-16T09:26:00.793 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_allocator": "hybrid", 2024-09-16T09:26:00.793 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_buffered_io": "true", 2024-09-16T09:26:00.793 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_check_for_zeros": "false", 2024-09-16T09:26:00.793 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_check_volume_selector_often": "false", 2024-09-16T09:26:00.793 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_check_volume_selector_on_umount": "false", 2024-09-16T09:26:00.794 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_compact_log_sync": "false", 2024-09-16T09:26:00.794 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2024-09-16T09:26:00.794 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2024-09-16T09:26:00.794 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_log_compact_min_size": "16777216", 2024-09-16T09:26:00.794 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_log_replay_check_allocations": "true", 2024-09-16T09:26:00.794 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_max_log_runway": "4194304", 2024-09-16T09:26:00.794 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_max_prefetch": "1048576", 2024-09-16T09:26:00.794 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_min_flush_size": "524288", 2024-09-16T09:26:00.795 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_min_log_runway": "1048576", 2024-09-16T09:26:00.795 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_replay_recovery": "false", 2024-09-16T09:26:00.795 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_replay_recovery_disable_compact": "false", 2024-09-16T09:26:00.795 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_shared_alloc_size": "65536", 2024-09-16T09:26:00.795 INFO:teuthology.orchestra.run.smithi070.stdout: "bluefs_sync_write": "false", 2024-09-16T09:26:00.795 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2024-09-16T09:26:00.795 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2024-09-16T09:26:00.795 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2024-09-16T09:26:00.795 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_allocation_from_file": "true", 2024-09-16T09:26:00.795 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_allocator": "hybrid", 2024-09-16T09:26:00.795 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_async_db_compaction": "true", 2024-09-16T09:26:00.795 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2024-09-16T09:26:00.795 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2024-09-16T09:26:00.795 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2024-09-16T09:26:00.796 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2024-09-16T09:26:00.796 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_bdev_label_multi": "true", 2024-09-16T09:26:00.796 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_bdev_label_multi_upgrade": "false", 2024-09-16T09:26:00.796 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_bdev_label_require_all": "true", 2024-09-16T09:26:00.796 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2024-09-16T09:26:00.796 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_bitmapallocator_span_size": "1024", 2024-09-16T09:26:00.796 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_blobid_prealloc": "10240", 2024-09-16T09:26:00.796 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_block_create": "true", 2024-09-16T09:26:00.796 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_block_db_create": "false", 2024-09-16T09:26:00.796 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_block_db_path": "", 2024-09-16T09:26:00.796 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_block_db_size": "0", 2024-09-16T09:26:00.796 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_block_path": "", 2024-09-16T09:26:00.796 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_block_preallocate_file": "false", 2024-09-16T09:26:00.797 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_block_size": "107374182400", 2024-09-16T09:26:00.797 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_block_wal_create": "false", 2024-09-16T09:26:00.797 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_block_wal_path": "", 2024-09-16T09:26:00.797 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_block_wal_size": "100663296", 2024-09-16T09:26:00.797 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_bluefs": "true", 2024-09-16T09:26:00.797 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2024-09-16T09:26:00.797 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_bluefs_env_mirror": "false", 2024-09-16T09:26:00.797 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_bluefs_max_free": "10737418240", 2024-09-16T09:26:00.797 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_btree2_alloc_weight_factor": "2.000000", 2024-09-16T09:26:00.797 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2024-09-16T09:26:00.797 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2024-09-16T09:26:00.798 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2024-09-16T09:26:00.798 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2024-09-16T09:26:00.798 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2024-09-16T09:26:00.798 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_autotune": "true", 2024-09-16T09:26:00.798 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_autotune_interval": "5.000000", 2024-09-16T09:26:00.798 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2024-09-16T09:26:00.798 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_kv_ratio": "0.450000", 2024-09-16T09:26:00.798 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_meta_ratio": "0.450000", 2024-09-16T09:26:00.798 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_size": "0", 2024-09-16T09:26:00.798 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_size_hdd": "1073741824", 2024-09-16T09:26:00.798 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_size_ssd": "3221225472", 2024-09-16T09:26:00.798 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_trim_interval": "0.050000", 2024-09-16T09:26:00.798 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2024-09-16T09:26:00.799 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cache_type": "2q", 2024-09-16T09:26:00.799 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2024-09-16T09:26:00.799 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_clone_cow": "true", 2024-09-16T09:26:00.799 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_compression_algorithm": "snappy", 2024-09-16T09:26:00.799 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_compression_max_blob_size": "0", 2024-09-16T09:26:00.799 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2024-09-16T09:26:00.799 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2024-09-16T09:26:00.799 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_compression_min_blob_size": "0", 2024-09-16T09:26:00.799 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2024-09-16T09:26:00.799 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2024-09-16T09:26:00.800 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_compression_mode": "none", 2024-09-16T09:26:00.800 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_compression_required_ratio": "0.875000", 2024-09-16T09:26:00.800 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_csum_type": "crc32c", 2024-09-16T09:26:00.800 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_debug_enforce_min_alloc_size": "0", 2024-09-16T09:26:00.800 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_debug_enforce_settings": "default", 2024-09-16T09:26:00.800 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_debug_freelist": "false", 2024-09-16T09:26:00.800 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_debug_fsck_abort": "false", 2024-09-16T09:26:00.800 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2024-09-16T09:26:00.800 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2024-09-16T09:26:00.800 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_debug_inject_read_err": "false", 2024-09-16T09:26:00.800 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_debug_legacy_omap": "false", 2024-09-16T09:26:00.800 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_debug_no_reuse_blocks": "false", 2024-09-16T09:26:00.801 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_debug_omit_block_device_write": "false", 2024-09-16T09:26:00.801 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_debug_omit_kv_commit": "false", 2024-09-16T09:26:00.801 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2024-09-16T09:26:00.801 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_debug_prefragment_max": "1048576", 2024-09-16T09:26:00.801 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_debug_random_read_err": "0.000000", 2024-09-16T09:26:00.801 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2024-09-16T09:26:00.801 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_debug_small_allocations": "0", 2024-09-16T09:26:00.801 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2024-09-16T09:26:00.801 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_default_buffered_read": "true", 2024-09-16T09:26:00.801 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_default_buffered_write": "false", 2024-09-16T09:26:00.801 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_deferred_batch_ops": "0", 2024-09-16T09:26:00.801 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2024-09-16T09:26:00.801 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2024-09-16T09:26:00.801 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_elastic_shared_blobs": "true", 2024-09-16T09:26:00.802 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2024-09-16T09:26:00.802 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_extent_map_shard_max_size": "1200", 2024-09-16T09:26:00.802 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_extent_map_shard_min_size": "150", 2024-09-16T09:26:00.802 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_extent_map_shard_target_size": "500", 2024-09-16T09:26:00.802 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2024-09-16T09:26:00.802 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_fail_eio": "false", 2024-09-16T09:26:00.802 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_freelist_blocks_per_key": "128", 2024-09-16T09:26:00.802 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2024-09-16T09:26:00.802 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2024-09-16T09:26:00.802 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2024-09-16T09:26:00.802 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_fsck_on_mkfs": "true", 2024-09-16T09:26:00.802 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2024-09-16T09:26:00.802 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_fsck_on_mount": "false", 2024-09-16T09:26:00.802 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_fsck_on_mount_deep": "false", 2024-09-16T09:26:00.802 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_fsck_on_umount": "false", 2024-09-16T09:26:00.803 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_fsck_on_umount_deep": "false", 2024-09-16T09:26:00.803 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2024-09-16T09:26:00.803 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_fsck_quick_fix_threads": "2", 2024-09-16T09:26:00.803 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2024-09-16T09:26:00.803 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2024-09-16T09:26:00.803 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_gc_enable_blob_threshold": "0", 2024-09-16T09:26:00.803 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_gc_enable_total_threshold": "0", 2024-09-16T09:26:00.803 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2024-09-16T09:26:00.803 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_ignore_data_csum": "false", 2024-09-16T09:26:00.803 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2024-09-16T09:26:00.803 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_kvbackend": "rocksdb", 2024-09-16T09:26:00.803 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_log_collection_list_age": "60.000000", 2024-09-16T09:26:00.803 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2024-09-16T09:26:00.803 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_log_op_age": "5.000000", 2024-09-16T09:26:00.804 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_max_alloc_size": "0", 2024-09-16T09:26:00.804 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_max_blob_size": "0", 2024-09-16T09:26:00.804 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_max_blob_size_hdd": "65536", 2024-09-16T09:26:00.804 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_max_blob_size_ssd": "65536", 2024-09-16T09:26:00.804 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_max_defer_interval": "3.000000", 2024-09-16T09:26:00.804 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_max_deferred_txc": "32", 2024-09-16T09:26:00.804 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_min_alloc_size": "0", 2024-09-16T09:26:00.804 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_min_alloc_size_hdd": "4096", 2024-09-16T09:26:00.804 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_min_alloc_size_ssd": "4096", 2024-09-16T09:26:00.804 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_nid_prealloc": "1024", 2024-09-16T09:26:00.804 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_prefer_deferred_size": "0", 2024-09-16T09:26:00.804 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2024-09-16T09:26:00.804 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2024-09-16T09:26:00.804 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_qfsck_on_mount": "true", 2024-09-16T09:26:00.805 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_retry_disk_reads": "3", 2024-09-16T09:26:00.805 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_rocksdb_cf": "true", 2024-09-16T09:26:00.805 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2024-09-16T09:26:00.805 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_rocksdb_options": "compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0", 2024-09-16T09:26:00.805 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_rocksdb_options_annex": "", 2024-09-16T09:26:00.805 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_slow_ops_warn_lifetime": "86400", 2024-09-16T09:26:00.805 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_slow_ops_warn_threshold": "1", 2024-09-16T09:26:00.805 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_spdk_coremask": "0x1", 2024-09-16T09:26:00.805 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_spdk_io_sleep": "5", 2024-09-16T09:26:00.805 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_spdk_max_io_completion": "0", 2024-09-16T09:26:00.806 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_spdk_mem": "512", 2024-09-16T09:26:00.806 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_sync_submit_transaction": "false", 2024-09-16T09:26:00.806 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_throttle_bytes": "67108864", 2024-09-16T09:26:00.806 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_throttle_cost_per_io": "0", 2024-09-16T09:26:00.806 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2024-09-16T09:26:00.806 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2024-09-16T09:26:00.806 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2024-09-16T09:26:00.806 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_throttle_trace_rate": "0.000000", 2024-09-16T09:26:00.806 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_tracing": "false", 2024-09-16T09:26:00.806 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2024-09-16T09:26:00.806 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2024-09-16T09:26:00.806 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_volume_selection_reserved": "0", 2024-09-16T09:26:00.806 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2024-09-16T09:26:00.807 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2024-09-16T09:26:00.807 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_warn_on_legacy_statfs": "true", 2024-09-16T09:26:00.807 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2024-09-16T09:26:00.807 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2024-09-16T09:26:00.807 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2024-09-16T09:26:00.807 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_write_v2": "false", 2024-09-16T09:26:00.807 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_write_v2_random": "false", 2024-09-16T09:26:00.807 INFO:teuthology.orchestra.run.smithi070.stdout: "bluestore_zero_block_detection": "false", 2024-09-16T09:26:00.807 INFO:teuthology.orchestra.run.smithi070.stdout: "cephadm_path": "/usr/sbin/cephadm", 2024-09-16T09:26:00.807 INFO:teuthology.orchestra.run.smithi070.stdout: "cephfs_mirror_action_update_interval": "2", 2024-09-16T09:26:00.807 INFO:teuthology.orchestra.run.smithi070.stdout: "cephfs_mirror_directory_scan_interval": "10", 2024-09-16T09:26:00.807 INFO:teuthology.orchestra.run.smithi070.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2024-09-16T09:26:00.807 INFO:teuthology.orchestra.run.smithi070.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2024-09-16T09:26:00.807 INFO:teuthology.orchestra.run.smithi070.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2024-09-16T09:26:00.808 INFO:teuthology.orchestra.run.smithi070.stdout: "cephfs_mirror_mount_timeout": "10", 2024-09-16T09:26:00.808 INFO:teuthology.orchestra.run.smithi070.stdout: "cephfs_mirror_perf_stats_prio": "5", 2024-09-16T09:26:00.808 INFO:teuthology.orchestra.run.smithi070.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2024-09-16T09:26:00.808 INFO:teuthology.orchestra.run.smithi070.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2024-09-16T09:26:00.808 INFO:teuthology.orchestra.run.smithi070.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2024-09-16T09:26:00.808 INFO:teuthology.orchestra.run.smithi070.stdout: "cephsqlite_blocklist_dead_locker": "true", 2024-09-16T09:26:00.808 INFO:teuthology.orchestra.run.smithi070.stdout: "cephsqlite_lock_renewal_interval": "2000", 2024-09-16T09:26:00.808 INFO:teuthology.orchestra.run.smithi070.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2024-09-16T09:26:00.808 INFO:teuthology.orchestra.run.smithi070.stdout: "cephx_cluster_require_signatures": "false", 2024-09-16T09:26:00.808 INFO:teuthology.orchestra.run.smithi070.stdout: "cephx_cluster_require_version": "2", 2024-09-16T09:26:00.808 INFO:teuthology.orchestra.run.smithi070.stdout: "cephx_require_signatures": "false", 2024-09-16T09:26:00.808 INFO:teuthology.orchestra.run.smithi070.stdout: "cephx_require_version": "2", 2024-09-16T09:26:00.808 INFO:teuthology.orchestra.run.smithi070.stdout: "cephx_service_require_signatures": "false", 2024-09-16T09:26:00.808 INFO:teuthology.orchestra.run.smithi070.stdout: "cephx_service_require_version": "2", 2024-09-16T09:26:00.809 INFO:teuthology.orchestra.run.smithi070.stdout: "cephx_sign_messages": "true", 2024-09-16T09:26:00.809 INFO:teuthology.orchestra.run.smithi070.stdout: "chdir": "", 2024-09-16T09:26:00.809 INFO:teuthology.orchestra.run.smithi070.stdout: "client_acl_type": "", 2024-09-16T09:26:00.809 INFO:teuthology.orchestra.run.smithi070.stdout: "client_asio_thread_count": "2", 2024-09-16T09:26:00.809 INFO:teuthology.orchestra.run.smithi070.stdout: "client_cache_mid": "0.750000", 2024-09-16T09:26:00.809 INFO:teuthology.orchestra.run.smithi070.stdout: "client_cache_size": "16384", 2024-09-16T09:26:00.809 INFO:teuthology.orchestra.run.smithi070.stdout: "client_caps_release_delay": "5", 2024-09-16T09:26:00.809 INFO:teuthology.orchestra.run.smithi070.stdout: "client_check_pool_perm": "true", 2024-09-16T09:26:00.809 INFO:teuthology.orchestra.run.smithi070.stdout: "client_collect_and_send_global_metrics": "false", 2024-09-16T09:26:00.809 INFO:teuthology.orchestra.run.smithi070.stdout: "client_debug_force_sync_read": "false", 2024-09-16T09:26:00.809 INFO:teuthology.orchestra.run.smithi070.stdout: "client_debug_getattr_caps": "false", 2024-09-16T09:26:00.809 INFO:teuthology.orchestra.run.smithi070.stdout: "client_debug_inject_features": "", 2024-09-16T09:26:00.809 INFO:teuthology.orchestra.run.smithi070.stdout: "client_debug_inject_tick_delay": "0", 2024-09-16T09:26:00.809 INFO:teuthology.orchestra.run.smithi070.stdout: "client_die_on_failed_dentry_invalidate": "true", 2024-09-16T09:26:00.810 INFO:teuthology.orchestra.run.smithi070.stdout: "client_die_on_failed_remount": "false", 2024-09-16T09:26:00.810 INFO:teuthology.orchestra.run.smithi070.stdout: "client_dirsize_rbytes": "true", 2024-09-16T09:26:00.810 INFO:teuthology.orchestra.run.smithi070.stdout: "client_force_lazyio": "false", 2024-09-16T09:26:00.810 INFO:teuthology.orchestra.run.smithi070.stdout: "client_fs": "", 2024-09-16T09:26:00.810 INFO:teuthology.orchestra.run.smithi070.stdout: "client_inject_fixed_oldest_tid": "false", 2024-09-16T09:26:00.810 INFO:teuthology.orchestra.run.smithi070.stdout: "client_inject_release_failure": "false", 2024-09-16T09:26:00.810 INFO:teuthology.orchestra.run.smithi070.stdout: "client_max_inline_size": "4096", 2024-09-16T09:26:00.810 INFO:teuthology.orchestra.run.smithi070.stdout: "client_max_retries_on_remount_failure": "5", 2024-09-16T09:26:00.810 INFO:teuthology.orchestra.run.smithi070.stdout: "client_mds_namespace": "", 2024-09-16T09:26:00.810 INFO:teuthology.orchestra.run.smithi070.stdout: "client_metadata": "", 2024-09-16T09:26:00.810 INFO:teuthology.orchestra.run.smithi070.stdout: "client_mount_gid": "-1", 2024-09-16T09:26:00.810 INFO:teuthology.orchestra.run.smithi070.stdout: "client_mount_timeout": "300", 2024-09-16T09:26:00.810 INFO:teuthology.orchestra.run.smithi070.stdout: "client_mount_uid": "-1", 2024-09-16T09:26:00.810 INFO:teuthology.orchestra.run.smithi070.stdout: "client_mountpoint": "/", 2024-09-16T09:26:00.811 INFO:teuthology.orchestra.run.smithi070.stdout: "client_notify_timeout": "10", 2024-09-16T09:26:00.811 INFO:teuthology.orchestra.run.smithi070.stdout: "client_oc": "true", 2024-09-16T09:26:00.811 INFO:teuthology.orchestra.run.smithi070.stdout: "client_oc_max_dirty": "104857600", 2024-09-16T09:26:00.811 INFO:teuthology.orchestra.run.smithi070.stdout: "client_oc_max_dirty_age": "5.000000", 2024-09-16T09:26:00.811 INFO:teuthology.orchestra.run.smithi070.stdout: "client_oc_max_objects": "1000", 2024-09-16T09:26:00.811 INFO:teuthology.orchestra.run.smithi070.stdout: "client_oc_size": "209715200", 2024-09-16T09:26:00.811 INFO:teuthology.orchestra.run.smithi070.stdout: "client_oc_target_dirty": "8388608", 2024-09-16T09:26:00.811 INFO:teuthology.orchestra.run.smithi070.stdout: "client_permissions": "true", 2024-09-16T09:26:00.811 INFO:teuthology.orchestra.run.smithi070.stdout: "client_quota": "true", 2024-09-16T09:26:00.811 INFO:teuthology.orchestra.run.smithi070.stdout: "client_quota_df": "true", 2024-09-16T09:26:00.811 INFO:teuthology.orchestra.run.smithi070.stdout: "client_readahead_max_bytes": "0", 2024-09-16T09:26:00.811 INFO:teuthology.orchestra.run.smithi070.stdout: "client_readahead_max_periods": "4", 2024-09-16T09:26:00.811 INFO:teuthology.orchestra.run.smithi070.stdout: "client_readahead_min": "131072", 2024-09-16T09:26:00.811 INFO:teuthology.orchestra.run.smithi070.stdout: "client_reconnect_stale": "false", 2024-09-16T09:26:00.812 INFO:teuthology.orchestra.run.smithi070.stdout: "client_shutdown_timeout": "30", 2024-09-16T09:26:00.812 INFO:teuthology.orchestra.run.smithi070.stdout: "client_snapdir": ".snap", 2024-09-16T09:26:00.812 INFO:teuthology.orchestra.run.smithi070.stdout: "client_tick_interval": "1", 2024-09-16T09:26:00.812 INFO:teuthology.orchestra.run.smithi070.stdout: "client_trace": "", 2024-09-16T09:26:00.812 INFO:teuthology.orchestra.run.smithi070.stdout: "client_try_dentry_invalidate": "false", 2024-09-16T09:26:00.812 INFO:teuthology.orchestra.run.smithi070.stdout: "client_use_faked_inos": "false", 2024-09-16T09:26:00.812 INFO:teuthology.orchestra.run.smithi070.stdout: "client_use_random_mds": "false", 2024-09-16T09:26:00.812 INFO:teuthology.orchestra.run.smithi070.stdout: "clog_to_graylog": "false", 2024-09-16T09:26:00.812 INFO:teuthology.orchestra.run.smithi070.stdout: "clog_to_graylog_host": "127.0.0.1", 2024-09-16T09:26:00.812 INFO:teuthology.orchestra.run.smithi070.stdout: "clog_to_graylog_port": "12201", 2024-09-16T09:26:00.812 INFO:teuthology.orchestra.run.smithi070.stdout: "clog_to_monitors": "default=true", 2024-09-16T09:26:00.812 INFO:teuthology.orchestra.run.smithi070.stdout: "clog_to_syslog": "false", 2024-09-16T09:26:00.812 INFO:teuthology.orchestra.run.smithi070.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2024-09-16T09:26:00.812 INFO:teuthology.orchestra.run.smithi070.stdout: "clog_to_syslog_level": "info", 2024-09-16T09:26:00.813 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "-", 2024-09-16T09:26:00.813 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_network": "", 2024-09-16T09:26:00.813 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_network_interface": "", 2024-09-16T09:26:00.813 INFO:teuthology.orchestra.run.smithi070.stdout: "compressor_zlib_isal": "false", 2024-09-16T09:26:00.813 INFO:teuthology.orchestra.run.smithi070.stdout: "compressor_zlib_level": "5", 2024-09-16T09:26:00.813 INFO:teuthology.orchestra.run.smithi070.stdout: "compressor_zlib_winsize": "-15", 2024-09-16T09:26:00.813 INFO:teuthology.orchestra.run.smithi070.stdout: "compressor_zstd_level": "1", 2024-09-16T09:26:00.813 INFO:teuthology.orchestra.run.smithi070.stdout: "container_image": "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce", 2024-09-16T09:26:00.813 INFO:teuthology.orchestra.run.smithi070.stdout: "crash_dir": "/var/lib/ceph/crash", 2024-09-16T09:26:00.813 INFO:teuthology.orchestra.run.smithi070.stdout: "crimson_alien_op_num_threads": "6", 2024-09-16T09:26:00.813 INFO:teuthology.orchestra.run.smithi070.stdout: "crimson_alien_thread_cpu_cores": "", 2024-09-16T09:26:00.813 INFO:teuthology.orchestra.run.smithi070.stdout: "crimson_osd_obc_lru_size": "512", 2024-09-16T09:26:00.813 INFO:teuthology.orchestra.run.smithi070.stdout: "crimson_osd_scheduler_concurrency": "0", 2024-09-16T09:26:00.813 INFO:teuthology.orchestra.run.smithi070.stdout: "crimson_osd_stat_interval": "0", 2024-09-16T09:26:00.813 INFO:teuthology.orchestra.run.smithi070.stdout: "crimson_seastar_cpu_cores": "", 2024-09-16T09:26:00.814 INFO:teuthology.orchestra.run.smithi070.stdout: "crimson_seastar_num_threads": "0", 2024-09-16T09:26:00.814 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_location": "", 2024-09-16T09:26:00.814 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_location_hook": "", 2024-09-16T09:26:00.814 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_location_hook_timeout": "10", 2024-09-16T09:26:00.814 INFO:teuthology.orchestra.run.smithi070.stdout: "daemonize": "false", 2024-09-16T09:26:00.814 INFO:teuthology.orchestra.run.smithi070.stdout: "daos_pool": "tank", 2024-09-16T09:26:00.814 INFO:teuthology.orchestra.run.smithi070.stdout: "dbstore_config_uri": "file:/var/lib/ceph/radosgw/dbstore-config.db", 2024-09-16T09:26:00.814 INFO:teuthology.orchestra.run.smithi070.stdout: "dbstore_db_dir": "/var/lib/ceph/radosgw", 2024-09-16T09:26:00.814 INFO:teuthology.orchestra.run.smithi070.stdout: "dbstore_db_name_prefix": "dbstore", 2024-09-16T09:26:00.814 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_alienstore": "0/5", 2024-09-16T09:26:00.814 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_allow_any_pool_priority": "false", 2024-09-16T09:26:00.814 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_asok": "1/5", 2024-09-16T09:26:00.814 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_asok_assert_abort": "false", 2024-09-16T09:26:00.814 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_asserts_on_shutdown": "true", 2024-09-16T09:26:00.814 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_auth": "1/5", 2024-09-16T09:26:00.815 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_bdev": "1/3", 2024-09-16T09:26:00.815 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_bluefs": "1/5", 2024-09-16T09:26:00.815 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_bluestore": "1/5", 2024-09-16T09:26:00.815 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_buffer": "0/1", 2024-09-16T09:26:00.815 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_ceph_dedup": "0/5", 2024-09-16T09:26:00.815 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_ceph_exporter": "1/5", 2024-09-16T09:26:00.815 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_cephfs_mirror": "0/5", 2024-09-16T09:26:00.815 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_cephsqlite": "0/5", 2024-09-16T09:26:00.815 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_client": "0/5", 2024-09-16T09:26:00.815 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_compressor": "1/5", 2024-09-16T09:26:00.815 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_context": "0/1", 2024-09-16T09:26:00.815 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_crimson_interrupt": "0/5", 2024-09-16T09:26:00.815 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_crush": "1/1", 2024-09-16T09:26:00.815 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_crypto": "1/5", 2024-09-16T09:26:00.816 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_cyanstore": "0/5", 2024-09-16T09:26:00.816 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_deliberately_leak_memory": "false", 2024-09-16T09:26:00.816 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_disable_randomized_ping": "false", 2024-09-16T09:26:00.816 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_dpdk": "1/5", 2024-09-16T09:26:00.816 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_eventtrace": "1/5", 2024-09-16T09:26:00.816 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_filer": "0/1", 2024-09-16T09:26:00.816 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_filestore": "1/3", 2024-09-16T09:26:00.816 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_finisher": "1/1", 2024-09-16T09:26:00.816 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_fuse": "1/5", 2024-09-16T09:26:00.816 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_heartbeat_testing_span": "0", 2024-09-16T09:26:00.816 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_heartbeatmap": "1/5", 2024-09-16T09:26:00.816 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_immutable_obj_cache": "0/5", 2024-09-16T09:26:00.816 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_javaclient": "1/5", 2024-09-16T09:26:00.816 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_journal": "1/3", 2024-09-16T09:26:00.816 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_journaler": "0/5", 2024-09-16T09:26:00.817 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_kstore": "1/5", 2024-09-16T09:26:00.817 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_lockdep": "0/1", 2024-09-16T09:26:00.817 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_mclock": "1/5", 2024-09-16T09:26:00.817 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_mds": "1/5", 2024-09-16T09:26:00.817 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_mds_balancer": "1/5", 2024-09-16T09:26:00.817 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_mds_locker": "1/5", 2024-09-16T09:26:00.817 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_mds_log": "1/5", 2024-09-16T09:26:00.817 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_mds_log_expire": "1/5", 2024-09-16T09:26:00.817 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_mds_migrator": "1/5", 2024-09-16T09:26:00.817 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_mds_quiesce": "3/5", 2024-09-16T09:26:00.817 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_memstore": "1/5", 2024-09-16T09:26:00.818 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_mgr": "2/5", 2024-09-16T09:26:00.818 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_mgrc": "1/5", 2024-09-16T09:26:00.818 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_mon": "20/20", 2024-09-16T09:26:00.818 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_monc": "0/10", 2024-09-16T09:26:00.818 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_ms": "1/1", 2024-09-16T09:26:00.818 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_none": "0/5", 2024-09-16T09:26:00.818 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_objclass": "0/5", 2024-09-16T09:26:00.818 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_objectcacher": "0/5", 2024-09-16T09:26:00.818 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_objecter": "0/1", 2024-09-16T09:26:00.818 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_optracker": "0/5", 2024-09-16T09:26:00.818 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_osd": "1/5", 2024-09-16T09:26:00.818 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_paxos": "20/20", 2024-09-16T09:26:00.818 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_perfcounter": "1/5", 2024-09-16T09:26:00.818 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_prioritycache": "1/5", 2024-09-16T09:26:00.818 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_rados": "0/5", 2024-09-16T09:26:00.819 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_rbd": "0/5", 2024-09-16T09:26:00.819 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_rbd_mirror": "0/5", 2024-09-16T09:26:00.819 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_rbd_pwl": "0/5", 2024-09-16T09:26:00.819 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_rbd_replay": "0/5", 2024-09-16T09:26:00.819 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_refs": "0/0", 2024-09-16T09:26:00.819 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_reserver": "1/1", 2024-09-16T09:26:00.819 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_rgw": "1/5", 2024-09-16T09:26:00.819 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_rgw_access": "1/5", 2024-09-16T09:26:00.819 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_rgw_datacache": "1/5", 2024-09-16T09:26:00.819 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_rgw_dbstore": "1/5", 2024-09-16T09:26:00.819 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_rgw_flight": "1/5", 2024-09-16T09:26:00.820 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_rgw_lifecycle": "1/5", 2024-09-16T09:26:00.821 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_rgw_notification": "1/5", 2024-09-16T09:26:00.821 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_rgw_sync": "1/5", 2024-09-16T09:26:00.821 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_rocksdb": "4/5", 2024-09-16T09:26:00.821 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_seastore": "0/5", 2024-09-16T09:26:00.821 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_seastore_backref": "0/5", 2024-09-16T09:26:00.821 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_seastore_cache": "0/5", 2024-09-16T09:26:00.821 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_seastore_cleaner": "0/5", 2024-09-16T09:26:00.821 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_seastore_device": "0/5", 2024-09-16T09:26:00.821 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_seastore_epm": "0/5", 2024-09-16T09:26:00.821 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_seastore_fixedkv_tree": "0/5", 2024-09-16T09:26:00.821 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_seastore_journal": "0/5", 2024-09-16T09:26:00.822 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_seastore_lba": "0/5", 2024-09-16T09:26:00.822 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_seastore_odata": "0/5", 2024-09-16T09:26:00.822 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_seastore_omap": "0/5", 2024-09-16T09:26:00.822 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_seastore_onode": "0/5", 2024-09-16T09:26:00.822 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_seastore_t": "0/5", 2024-09-16T09:26:00.822 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_seastore_tm": "0/5", 2024-09-16T09:26:00.822 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_striper": "0/1", 2024-09-16T09:26:00.822 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_test": "0/5", 2024-09-16T09:26:00.822 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_throttle": "1/1", 2024-09-16T09:26:00.822 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_timer": "0/1", 2024-09-16T09:26:00.822 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_tp": "0/5", 2024-09-16T09:26:00.823 INFO:teuthology.orchestra.run.smithi070.stdout: "debug_trace": "1/5", 2024-09-16T09:26:00.823 INFO:teuthology.orchestra.run.smithi070.stdout: "defer_client_eviction_on_laggy_osds": "false", 2024-09-16T09:26:00.823 INFO:teuthology.orchestra.run.smithi070.stdout: "device_failure_prediction_mode": "none", 2024-09-16T09:26:00.823 INFO:teuthology.orchestra.run.smithi070.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2024-09-16T09:26:00.823 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_dir": "/usr/lib64/ceph/erasure-code", 2024-09-16T09:26:00.823 INFO:teuthology.orchestra.run.smithi070.stdout: "err_to_graylog": "false", 2024-09-16T09:26:00.823 INFO:teuthology.orchestra.run.smithi070.stdout: "err_to_journald": "false", 2024-09-16T09:26:00.823 INFO:teuthology.orchestra.run.smithi070.stdout: "err_to_stderr": "true", 2024-09-16T09:26:00.823 INFO:teuthology.orchestra.run.smithi070.stdout: "err_to_syslog": "false", 2024-09-16T09:26:00.823 INFO:teuthology.orchestra.run.smithi070.stdout: "event_tracing": "false", 2024-09-16T09:26:00.823 INFO:teuthology.orchestra.run.smithi070.stdout: "exporter_addr": "0.0.0.0", 2024-09-16T09:26:00.823 INFO:teuthology.orchestra.run.smithi070.stdout: "exporter_cert_file": "", 2024-09-16T09:26:00.823 INFO:teuthology.orchestra.run.smithi070.stdout: "exporter_http_port": "9926", 2024-09-16T09:26:00.823 INFO:teuthology.orchestra.run.smithi070.stdout: "exporter_key_file": "", 2024-09-16T09:26:00.823 INFO:teuthology.orchestra.run.smithi070.stdout: "exporter_prio_limit": "5", 2024-09-16T09:26:00.824 INFO:teuthology.orchestra.run.smithi070.stdout: "exporter_sock_dir": "/var/run/ceph/", 2024-09-16T09:26:00.824 INFO:teuthology.orchestra.run.smithi070.stdout: "exporter_sort_metrics": "true", 2024-09-16T09:26:00.824 INFO:teuthology.orchestra.run.smithi070.stdout: "exporter_stats_period": "5", 2024-09-16T09:26:00.824 INFO:teuthology.orchestra.run.smithi070.stdout: "fake_statfs_for_testing": "0", 2024-09-16T09:26:00.824 INFO:teuthology.orchestra.run.smithi070.stdout: "fatal_signal_handlers": "true", 2024-09-16T09:26:00.824 INFO:teuthology.orchestra.run.smithi070.stdout: "filer_max_purge_ops": "10", 2024-09-16T09:26:00.824 INFO:teuthology.orchestra.run.smithi070.stdout: "filer_max_truncate_ops": "128", 2024-09-16T09:26:00.824 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_apply_finisher_threads": "1", 2024-09-16T09:26:00.825 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_blackhole": "false", 2024-09-16T09:26:00.825 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_btrfs_clone_range": "true", 2024-09-16T09:26:00.825 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_btrfs_snap": "true", 2024-09-16T09:26:00.825 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_caller_concurrency": "10", 2024-09-16T09:26:00.825 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_collect_device_partition_information": "true", 2024-09-16T09:26:00.825 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_commit_timeout": "600.000000", 2024-09-16T09:26:00.825 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_debug_inject_read_err": "false", 2024-09-16T09:26:00.825 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_debug_omap_check": "false", 2024-09-16T09:26:00.825 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_debug_verify_split": "false", 2024-09-16T09:26:00.825 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_dump_file": "", 2024-09-16T09:26:00.825 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2024-09-16T09:26:00.825 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_expected_throughput_ops": "200.000000", 2024-09-16T09:26:00.826 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_fadvise": "true", 2024-09-16T09:26:00.826 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_fail_eio": "true", 2024-09-16T09:26:00.826 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_fd_cache_shards": "16", 2024-09-16T09:26:00.826 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_fd_cache_size": "128", 2024-09-16T09:26:00.826 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_fiemap": "false", 2024-09-16T09:26:00.826 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_fiemap_threshold": "4096", 2024-09-16T09:26:00.826 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_fsync_flushes_journal_data": "false", 2024-09-16T09:26:00.826 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_index_retry_probability": "0.000000", 2024-09-16T09:26:00.826 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_inject_stall": "0", 2024-09-16T09:26:00.826 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_journal_parallel": "false", 2024-09-16T09:26:00.827 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_journal_trailing": "false", 2024-09-16T09:26:00.827 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_journal_writeahead": "false", 2024-09-16T09:26:00.827 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_kill_at": "0", 2024-09-16T09:26:00.827 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_max_alloc_hint_size": "1048576", 2024-09-16T09:26:00.827 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_max_inline_xattr_size": "0", 2024-09-16T09:26:00.827 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2024-09-16T09:26:00.827 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_max_inline_xattr_size_other": "512", 2024-09-16T09:26:00.827 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2024-09-16T09:26:00.827 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_max_inline_xattrs": "0", 2024-09-16T09:26:00.827 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2024-09-16T09:26:00.827 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_max_inline_xattrs_other": "2", 2024-09-16T09:26:00.827 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_max_inline_xattrs_xfs": "10", 2024-09-16T09:26:00.827 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_max_sync_interval": "5.000000", 2024-09-16T09:26:00.827 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_max_xattr_value_size": "0", 2024-09-16T09:26:00.828 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2024-09-16T09:26:00.828 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_max_xattr_value_size_other": "1024", 2024-09-16T09:26:00.828 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2024-09-16T09:26:00.828 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_merge_threshold": "-10", 2024-09-16T09:26:00.828 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_min_sync_interval": "0.010000", 2024-09-16T09:26:00.828 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_odsync_write": "false", 2024-09-16T09:26:00.828 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_omap_backend": "rocksdb", 2024-09-16T09:26:00.828 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_omap_backend_path": "", 2024-09-16T09:26:00.828 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_omap_header_cache_size": "1024", 2024-09-16T09:26:00.828 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_ondisk_finisher_threads": "1", 2024-09-16T09:26:00.828 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_op_thread_suicide_timeout": "180", 2024-09-16T09:26:00.828 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_op_thread_timeout": "60", 2024-09-16T09:26:00.828 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_op_threads": "2", 2024-09-16T09:26:00.828 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_punch_hole": "false", 2024-09-16T09:26:00.829 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2024-09-16T09:26:00.829 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2024-09-16T09:26:00.829 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2024-09-16T09:26:00.829 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_queue_high_threshhold": "0.900000", 2024-09-16T09:26:00.829 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_queue_low_threshhold": "0.300000", 2024-09-16T09:26:00.829 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_queue_max_bytes": "104857600", 2024-09-16T09:26:00.829 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2024-09-16T09:26:00.829 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2024-09-16T09:26:00.829 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2024-09-16T09:26:00.829 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_queue_max_ops": "50", 2024-09-16T09:26:00.829 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2024-09-16T09:26:00.829 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_seek_data_hole": "false", 2024-09-16T09:26:00.829 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_sloppy_crc": "false", 2024-09-16T09:26:00.830 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_sloppy_crc_block_size": "65536", 2024-09-16T09:26:00.830 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_splice": "false", 2024-09-16T09:26:00.830 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_split_multiple": "2", 2024-09-16T09:26:00.830 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_split_rand_factor": "20", 2024-09-16T09:26:00.830 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_update_to": "1000", 2024-09-16T09:26:00.830 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2024-09-16T09:26:00.830 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2024-09-16T09:26:00.830 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2024-09-16T09:26:00.830 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2024-09-16T09:26:00.831 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2024-09-16T09:26:00.831 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2024-09-16T09:26:00.831 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_wbthrottle_enable": "true", 2024-09-16T09:26:00.831 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2024-09-16T09:26:00.831 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2024-09-16T09:26:00.831 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2024-09-16T09:26:00.831 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2024-09-16T09:26:00.831 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2024-09-16T09:26:00.831 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2024-09-16T09:26:00.831 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_xfs_extsize": "false", 2024-09-16T09:26:00.832 INFO:teuthology.orchestra.run.smithi070.stdout: "filestore_zfs_snap": "false", 2024-09-16T09:26:00.832 INFO:teuthology.orchestra.run.smithi070.stdout: "fio_dir": "/tmp/fio", 2024-09-16T09:26:00.832 INFO:teuthology.orchestra.run.smithi070.stdout: "fsid": "a1321834-740c-11ef-bceb-c7b262605968", 2024-09-16T09:26:00.832 INFO:teuthology.orchestra.run.smithi070.stdout: "fuse_allow_other": "true", 2024-09-16T09:26:00.832 INFO:teuthology.orchestra.run.smithi070.stdout: "fuse_atomic_o_trunc": "true", 2024-09-16T09:26:00.832 INFO:teuthology.orchestra.run.smithi070.stdout: "fuse_big_writes": "true", 2024-09-16T09:26:00.832 INFO:teuthology.orchestra.run.smithi070.stdout: "fuse_debug": "false", 2024-09-16T09:26:00.832 INFO:teuthology.orchestra.run.smithi070.stdout: "fuse_default_permissions": "false", 2024-09-16T09:26:00.832 INFO:teuthology.orchestra.run.smithi070.stdout: "fuse_disable_pagecache": "false", 2024-09-16T09:26:00.832 INFO:teuthology.orchestra.run.smithi070.stdout: "fuse_max_write": "0", 2024-09-16T09:26:00.833 INFO:teuthology.orchestra.run.smithi070.stdout: "fuse_multithreaded": "true", 2024-09-16T09:26:00.833 INFO:teuthology.orchestra.run.smithi070.stdout: "fuse_require_active_mds": "true", 2024-09-16T09:26:00.833 INFO:teuthology.orchestra.run.smithi070.stdout: "fuse_set_user_groups": "true", 2024-09-16T09:26:00.833 INFO:teuthology.orchestra.run.smithi070.stdout: "fuse_splice_move": "true", 2024-09-16T09:26:00.833 INFO:teuthology.orchestra.run.smithi070.stdout: "fuse_splice_read": "true", 2024-09-16T09:26:00.833 INFO:teuthology.orchestra.run.smithi070.stdout: "fuse_splice_write": "true", 2024-09-16T09:26:00.833 INFO:teuthology.orchestra.run.smithi070.stdout: "fuse_syncfs_on_mksnap": "true", 2024-09-16T09:26:00.833 INFO:teuthology.orchestra.run.smithi070.stdout: "fuse_use_invalidate_cb": "true", 2024-09-16T09:26:00.833 INFO:teuthology.orchestra.run.smithi070.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2024-09-16T09:26:00.833 INFO:teuthology.orchestra.run.smithi070.stdout: "gss_target_name": "ceph", 2024-09-16T09:26:00.833 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_file": "", 2024-09-16T09:26:00.833 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_inject_failure": "0", 2024-09-16T09:26:00.834 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_interval": "5", 2024-09-16T09:26:00.834 INFO:teuthology.orchestra.run.smithi070.stdout: "host": "smithi070", 2024-09-16T09:26:00.834 INFO:teuthology.orchestra.run.smithi070.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2024-09-16T09:26:00.834 INFO:teuthology.orchestra.run.smithi070.stdout: "immutable_object_cache_max_inflight_ops": "128", 2024-09-16T09:26:00.834 INFO:teuthology.orchestra.run.smithi070.stdout: "immutable_object_cache_max_size": "1073741824", 2024-09-16T09:26:00.834 INFO:teuthology.orchestra.run.smithi070.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2024-09-16T09:26:00.834 INFO:teuthology.orchestra.run.smithi070.stdout: "immutable_object_cache_qos_bps_burst": "0", 2024-09-16T09:26:00.834 INFO:teuthology.orchestra.run.smithi070.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2024-09-16T09:26:00.834 INFO:teuthology.orchestra.run.smithi070.stdout: "immutable_object_cache_qos_bps_limit": "0", 2024-09-16T09:26:00.834 INFO:teuthology.orchestra.run.smithi070.stdout: "immutable_object_cache_qos_iops_burst": "0", 2024-09-16T09:26:00.835 INFO:teuthology.orchestra.run.smithi070.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2024-09-16T09:26:00.835 INFO:teuthology.orchestra.run.smithi070.stdout: "immutable_object_cache_qos_iops_limit": "0", 2024-09-16T09:26:00.835 INFO:teuthology.orchestra.run.smithi070.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2024-09-16T09:26:00.835 INFO:teuthology.orchestra.run.smithi070.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2024-09-16T09:26:00.835 INFO:teuthology.orchestra.run.smithi070.stdout: "immutable_object_cache_watermark": "0.900000", 2024-09-16T09:26:00.835 INFO:teuthology.orchestra.run.smithi070.stdout: "inject_early_sigterm": "false", 2024-09-16T09:26:00.835 INFO:teuthology.orchestra.run.smithi070.stdout: "jaeger_agent_port": "6799", 2024-09-16T09:26:00.835 INFO:teuthology.orchestra.run.smithi070.stdout: "jaeger_tracing_enable": "false", 2024-09-16T09:26:00.835 INFO:teuthology.orchestra.run.smithi070.stdout: "journal_aio": "true", 2024-09-16T09:26:00.836 INFO:teuthology.orchestra.run.smithi070.stdout: "journal_align_min_size": "65536", 2024-09-16T09:26:00.836 INFO:teuthology.orchestra.run.smithi070.stdout: "journal_block_align": "true", 2024-09-16T09:26:00.836 INFO:teuthology.orchestra.run.smithi070.stdout: "journal_block_size": "4096", 2024-09-16T09:26:00.836 INFO:teuthology.orchestra.run.smithi070.stdout: "journal_dio": "true", 2024-09-16T09:26:00.836 INFO:teuthology.orchestra.run.smithi070.stdout: "journal_discard": "false", 2024-09-16T09:26:00.836 INFO:teuthology.orchestra.run.smithi070.stdout: "journal_force_aio": "false", 2024-09-16T09:26:00.836 INFO:teuthology.orchestra.run.smithi070.stdout: "journal_ignore_corruption": "false", 2024-09-16T09:26:00.837 INFO:teuthology.orchestra.run.smithi070.stdout: "journal_max_write_bytes": "10485760", 2024-09-16T09:26:00.837 INFO:teuthology.orchestra.run.smithi070.stdout: "journal_max_write_entries": "100", 2024-09-16T09:26:00.837 INFO:teuthology.orchestra.run.smithi070.stdout: "journal_replay_from": "0", 2024-09-16T09:26:00.837 INFO:teuthology.orchestra.run.smithi070.stdout: "journal_throttle_high_multiple": "0.000000", 2024-09-16T09:26:00.837 INFO:teuthology.orchestra.run.smithi070.stdout: "journal_throttle_high_threshhold": "0.900000", 2024-09-16T09:26:00.837 INFO:teuthology.orchestra.run.smithi070.stdout: "journal_throttle_low_threshhold": "0.600000", 2024-09-16T09:26:00.838 INFO:teuthology.orchestra.run.smithi070.stdout: "journal_throttle_max_multiple": "0.000000", 2024-09-16T09:26:00.838 INFO:teuthology.orchestra.run.smithi070.stdout: "journal_write_header_frequency": "0", 2024-09-16T09:26:00.838 INFO:teuthology.orchestra.run.smithi070.stdout: "journal_zero_on_create": "false", 2024-09-16T09:26:00.838 INFO:teuthology.orchestra.run.smithi070.stdout: "journaler_prefetch_periods": "10", 2024-09-16T09:26:00.838 INFO:teuthology.orchestra.run.smithi070.stdout: "journaler_prezero_periods": "5", 2024-09-16T09:26:00.838 INFO:teuthology.orchestra.run.smithi070.stdout: "journaler_write_head_interval": "15", 2024-09-16T09:26:00.838 INFO:teuthology.orchestra.run.smithi070.stdout: "key": "", 2024-09-16T09:26:00.839 INFO:teuthology.orchestra.run.smithi070.stdout: "keyfile": "", 2024-09-16T09:26:00.839 INFO:teuthology.orchestra.run.smithi070.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2024-09-16T09:26:00.839 INFO:teuthology.orchestra.run.smithi070.stdout: "kstore_backend": "rocksdb", 2024-09-16T09:26:00.839 INFO:teuthology.orchestra.run.smithi070.stdout: "kstore_default_stripe_size": "65536", 2024-09-16T09:26:00.839 INFO:teuthology.orchestra.run.smithi070.stdout: "kstore_fsck_on_mount": "false", 2024-09-16T09:26:00.839 INFO:teuthology.orchestra.run.smithi070.stdout: "kstore_fsck_on_mount_deep": "true", 2024-09-16T09:26:00.840 INFO:teuthology.orchestra.run.smithi070.stdout: "kstore_max_bytes": "67108864", 2024-09-16T09:26:00.840 INFO:teuthology.orchestra.run.smithi070.stdout: "kstore_max_ops": "512", 2024-09-16T09:26:00.840 INFO:teuthology.orchestra.run.smithi070.stdout: "kstore_nid_prealloc": "1024", 2024-09-16T09:26:00.840 INFO:teuthology.orchestra.run.smithi070.stdout: "kstore_onode_map_size": "1024", 2024-09-16T09:26:00.840 INFO:teuthology.orchestra.run.smithi070.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2024-09-16T09:26:00.840 INFO:teuthology.orchestra.run.smithi070.stdout: "kstore_sync_submit_transaction": "false", 2024-09-16T09:26:00.841 INFO:teuthology.orchestra.run.smithi070.stdout: "kstore_sync_transaction": "false", 2024-09-16T09:26:00.841 INFO:teuthology.orchestra.run.smithi070.stdout: "librados_thread_count": "2", 2024-09-16T09:26:00.841 INFO:teuthology.orchestra.run.smithi070.stdout: "lockdep": "false", 2024-09-16T09:26:00.841 INFO:teuthology.orchestra.run.smithi070.stdout: "lockdep_force_backtrace": "false", 2024-09-16T09:26:00.841 INFO:teuthology.orchestra.run.smithi070.stdout: "log_coarse_timestamps": "true", 2024-09-16T09:26:00.841 INFO:teuthology.orchestra.run.smithi070.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2024-09-16T09:26:00.842 INFO:teuthology.orchestra.run.smithi070.stdout: "log_flush_on_exit": "false", 2024-09-16T09:26:00.842 INFO:teuthology.orchestra.run.smithi070.stdout: "log_graylog_host": "127.0.0.1", 2024-09-16T09:26:00.842 INFO:teuthology.orchestra.run.smithi070.stdout: "log_graylog_port": "12201", 2024-09-16T09:26:00.842 INFO:teuthology.orchestra.run.smithi070.stdout: "log_max_new": "1000", 2024-09-16T09:26:00.842 INFO:teuthology.orchestra.run.smithi070.stdout: "log_max_recent": "10000", 2024-09-16T09:26:00.842 INFO:teuthology.orchestra.run.smithi070.stdout: "log_stderr_prefix": "", 2024-09-16T09:26:00.843 INFO:teuthology.orchestra.run.smithi070.stdout: "log_stop_at_utilization": "0.970000", 2024-09-16T09:26:00.843 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_file": "true", 2024-09-16T09:26:00.843 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_graylog": "false", 2024-09-16T09:26:00.843 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_journald": "false", 2024-09-16T09:26:00.843 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_stderr": "false", 2024-09-16T09:26:00.843 INFO:teuthology.orchestra.run.smithi070.stdout: "log_to_syslog": "false", 2024-09-16T09:26:00.844 INFO:teuthology.orchestra.run.smithi070.stdout: "max_rotating_auth_attempts": "10", 2024-09-16T09:26:00.844 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2024-09-16T09:26:00.844 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_action_on_write_error": "1", 2024-09-16T09:26:00.844 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_alternate_name_max": "8192", 2024-09-16T09:26:00.844 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_asio_thread_count": "2", 2024-09-16T09:26:00.845 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_bal_export_pin": "true", 2024-09-16T09:26:00.845 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_bal_fragment_dirs": "true", 2024-09-16T09:26:00.845 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2024-09-16T09:26:00.845 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_bal_fragment_interval": "5", 2024-09-16T09:26:00.845 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_bal_fragment_size_max": "100000", 2024-09-16T09:26:00.845 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_bal_idle_threshold": "0.000000", 2024-09-16T09:26:00.846 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_bal_interval": "10", 2024-09-16T09:26:00.846 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_bal_max": "-1", 2024-09-16T09:26:00.846 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_bal_max_until": "-1", 2024-09-16T09:26:00.846 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_bal_merge_size": "50", 2024-09-16T09:26:00.846 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_bal_midchunk": "0.300000", 2024-09-16T09:26:00.846 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_bal_min_rebalance": "0.100000", 2024-09-16T09:26:00.847 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_bal_min_start": "0.200000", 2024-09-16T09:26:00.847 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_bal_minchunk": "0.001000", 2024-09-16T09:26:00.847 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_bal_mode": "0", 2024-09-16T09:26:00.847 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_bal_need_max": "1.200000", 2024-09-16T09:26:00.847 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_bal_need_min": "0.800000", 2024-09-16T09:26:00.847 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_bal_overload_epochs": "2", 2024-09-16T09:26:00.848 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_bal_replicate_threshold": "8000.000000", 2024-09-16T09:26:00.848 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_bal_sample_interval": "3.000000", 2024-09-16T09:26:00.848 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_bal_split_bits": "3", 2024-09-16T09:26:00.848 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_bal_split_rd": "25000.000000", 2024-09-16T09:26:00.848 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_bal_split_size": "10000", 2024-09-16T09:26:00.848 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_bal_split_wr": "10000.000000", 2024-09-16T09:26:00.849 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_bal_target_decay": "10.000000", 2024-09-16T09:26:00.849 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2024-09-16T09:26:00.849 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_beacon_grace": "15.000000", 2024-09-16T09:26:00.849 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_beacon_interval": "4.000000", 2024-09-16T09:26:00.849 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_beacon_mon_down_grace": "60", 2024-09-16T09:26:00.849 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_cache_memory_limit": "4294967296", 2024-09-16T09:26:00.850 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_cache_mid": "0.700000", 2024-09-16T09:26:00.850 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_cache_quiesce_decay_rate": "1.000000", 2024-09-16T09:26:00.850 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_cache_quiesce_delay": "0", 2024-09-16T09:26:00.850 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_cache_quiesce_sleep": "200", 2024-09-16T09:26:00.850 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_cache_quiesce_splitauth": "true", 2024-09-16T09:26:00.850 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_cache_quiesce_threshold": "524288", 2024-09-16T09:26:00.851 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_cache_release_free_interval": "10", 2024-09-16T09:26:00.851 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_cache_reservation": "0.050000", 2024-09-16T09:26:00.851 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_cache_trim_decay_rate": "1.000000", 2024-09-16T09:26:00.851 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_cache_trim_interval": "1", 2024-09-16T09:26:00.851 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_cache_trim_threshold": "262144", 2024-09-16T09:26:00.851 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2024-09-16T09:26:00.852 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2024-09-16T09:26:00.852 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_client_delegate_inos_pct": "50", 2024-09-16T09:26:00.852 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_client_prealloc_inos": "1000", 2024-09-16T09:26:00.852 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2024-09-16T09:26:00.852 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_connect_bootstrapping": "false", 2024-09-16T09:26:00.853 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_damage_table_max_entries": "10000", 2024-09-16T09:26:00.853 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2024-09-16T09:26:00.853 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_debug_auth_pins": "false", 2024-09-16T09:26:00.853 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_debug_frag": "false", 2024-09-16T09:26:00.853 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_debug_scatterstat": "false", 2024-09-16T09:26:00.853 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_debug_subtrees": "false", 2024-09-16T09:26:00.854 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_decay_halflife": "5.000000", 2024-09-16T09:26:00.854 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_default_dir_hash": "2", 2024-09-16T09:26:00.854 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_defer_session_stale": "true", 2024-09-16T09:26:00.854 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_deny_all_reconnect": "false", 2024-09-16T09:26:00.854 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_dir_keys_per_op": "16384", 2024-09-16T09:26:00.854 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_dir_max_commit_size": "10", 2024-09-16T09:26:00.855 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_dir_max_entries": "0", 2024-09-16T09:26:00.855 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_dir_prefetch": "true", 2024-09-16T09:26:00.855 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_dirstat_min_interval": "1.000000", 2024-09-16T09:26:00.855 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_dump_cache_after_rejoin": "false", 2024-09-16T09:26:00.855 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_dump_cache_on_map": "false", 2024-09-16T09:26:00.855 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_dump_cache_threshold_file": "0", 2024-09-16T09:26:00.856 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2024-09-16T09:26:00.856 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_early_reply": "true", 2024-09-16T09:26:00.856 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_enable_op_tracker": "true", 2024-09-16T09:26:00.856 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_enforce_unique_name": "true", 2024-09-16T09:26:00.856 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_export_ephemeral_distributed": "true", 2024-09-16T09:26:00.856 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2024-09-16T09:26:00.857 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_export_ephemeral_random": "true", 2024-09-16T09:26:00.857 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_export_ephemeral_random_max": "0.010000", 2024-09-16T09:26:00.857 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_extraordinary_events_dump_interval": "0", 2024-09-16T09:26:00.857 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_forward_all_requests_to_auth": "false", 2024-09-16T09:26:00.857 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_freeze_tree_timeout": "30.000000", 2024-09-16T09:26:00.857 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_fscrypt_last_block_max_size": "4096", 2024-09-16T09:26:00.858 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_go_bad_corrupt_dentry": "true", 2024-09-16T09:26:00.858 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2024-09-16T09:26:00.858 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_health_cache_threshold": "1.500000", 2024-09-16T09:26:00.858 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_health_summarize_threshold": "10", 2024-09-16T09:26:00.858 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_heartbeat_grace": "15.000000", 2024-09-16T09:26:00.858 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_heartbeat_reset_grace": "1000", 2024-09-16T09:26:00.859 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_inject_health_dummy": "false", 2024-09-16T09:26:00.859 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2024-09-16T09:26:00.859 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_inject_migrator_session_race": "false", 2024-09-16T09:26:00.859 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2024-09-16T09:26:00.859 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_inject_skip_replaying_inotable": "false", 2024-09-16T09:26:00.859 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2024-09-16T09:26:00.860 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_join_fs": "", 2024-09-16T09:26:00.860 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_journal_format": "1", 2024-09-16T09:26:00.860 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_kill_after_journal_logs_flushed": "false", 2024-09-16T09:26:00.860 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_kill_create_at": "0", 2024-09-16T09:26:00.860 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_kill_dirfrag_at": "0", 2024-09-16T09:26:00.861 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_kill_export_at": "0", 2024-09-16T09:26:00.861 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_kill_import_at": "0", 2024-09-16T09:26:00.861 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_kill_journal_at": "0", 2024-09-16T09:26:00.861 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_kill_journal_expire_at": "0", 2024-09-16T09:26:00.861 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_kill_journal_replay_at": "0", 2024-09-16T09:26:00.861 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_kill_link_at": "0", 2024-09-16T09:26:00.862 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_kill_mdstable_at": "0", 2024-09-16T09:26:00.862 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_kill_openc_at": "0", 2024-09-16T09:26:00.862 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_kill_rename_at": "0", 2024-09-16T09:26:00.862 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_kill_shutdown_at": "0", 2024-09-16T09:26:00.862 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_log_event_large_threshold": "524288", 2024-09-16T09:26:00.862 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_log_events_per_segment": "1024", 2024-09-16T09:26:00.863 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_log_major_segment_event_ratio": "12", 2024-09-16T09:26:00.863 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_log_max_events": "-1", 2024-09-16T09:26:00.863 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_log_max_segments": "128", 2024-09-16T09:26:00.863 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_log_pause": "false", 2024-09-16T09:26:00.863 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_log_segment_size": "0", 2024-09-16T09:26:00.863 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_log_skip_corrupt_events": "false", 2024-09-16T09:26:00.863 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_log_skip_unbounded_events": "false", 2024-09-16T09:26:00.864 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_log_trim_decay_rate": "1.000000", 2024-09-16T09:26:00.864 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_log_trim_threshold": "128", 2024-09-16T09:26:00.864 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_log_trim_upkeep_interval": "1000", 2024-09-16T09:26:00.864 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_log_warn_factor": "2.000000", 2024-09-16T09:26:00.864 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_max_caps_per_client": "1048576", 2024-09-16T09:26:00.864 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_max_completed_flushes": "100000", 2024-09-16T09:26:00.864 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_max_completed_requests": "100000", 2024-09-16T09:26:00.864 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_max_export_size": "20971520", 2024-09-16T09:26:00.865 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_max_file_recover": "32", 2024-09-16T09:26:00.865 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_max_purge_files": "64", 2024-09-16T09:26:00.865 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_max_purge_ops": "8192", 2024-09-16T09:26:00.865 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2024-09-16T09:26:00.865 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_max_scrub_ops_in_progress": "5", 2024-09-16T09:26:00.865 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_max_snaps_per_dir": "100", 2024-09-16T09:26:00.865 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_metrics_update_interval": "2", 2024-09-16T09:26:00.865 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_min_caps_per_client": "100", 2024-09-16T09:26:00.865 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_min_caps_working_set": "10000", 2024-09-16T09:26:00.865 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_mon_shutdown_timeout": "5.000000", 2024-09-16T09:26:00.865 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_numa_node": "-1", 2024-09-16T09:26:00.865 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_oft_prefetch_dirfrags": "false", 2024-09-16T09:26:00.866 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_op_complaint_time": "30.000000", 2024-09-16T09:26:00.866 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_op_history_duration": "600", 2024-09-16T09:26:00.866 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_op_history_size": "20", 2024-09-16T09:26:00.866 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_op_history_slow_op_size": "20", 2024-09-16T09:26:00.866 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_op_history_slow_op_threshold": "10", 2024-09-16T09:26:00.866 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_op_log_threshold": "5", 2024-09-16T09:26:00.866 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_ping_grace": "15", 2024-09-16T09:26:00.866 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_ping_interval": "5", 2024-09-16T09:26:00.866 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2024-09-16T09:26:00.866 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_recall_global_max_decay_threshold": "131072", 2024-09-16T09:26:00.866 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_recall_max_caps": "30000", 2024-09-16T09:26:00.866 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_recall_max_decay_rate": "1.500000", 2024-09-16T09:26:00.867 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_recall_max_decay_threshold": "131072", 2024-09-16T09:26:00.867 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_recall_warning_decay_rate": "60.000000", 2024-09-16T09:26:00.867 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_recall_warning_threshold": "262144", 2024-09-16T09:26:00.867 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_reconnect_timeout": "45.000000", 2024-09-16T09:26:00.867 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_replay_interval": "1.000000", 2024-09-16T09:26:00.867 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_replay_unsafe_with_closed_session": "false", 2024-09-16T09:26:00.867 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_request_load_average_decay_rate": "60.000000", 2024-09-16T09:26:00.867 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_root_ino_gid": "0", 2024-09-16T09:26:00.867 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_root_ino_uid": "0", 2024-09-16T09:26:00.867 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_scatter_nudge_interval": "5.000000", 2024-09-16T09:26:00.867 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_scrub_stats_review_period": "1", 2024-09-16T09:26:00.867 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_server_dispatch_client_request_delay": "0", 2024-09-16T09:26:00.867 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_server_dispatch_killpoint_random": "0.000000", 2024-09-16T09:26:00.868 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_session_blocklist_on_evict": "true", 2024-09-16T09:26:00.868 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_session_blocklist_on_timeout": "true", 2024-09-16T09:26:00.868 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2024-09-16T09:26:00.868 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_session_cache_liveness_magnitude": "10", 2024-09-16T09:26:00.868 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2024-09-16T09:26:00.868 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_session_cap_acquisition_throttle": "100000", 2024-09-16T09:26:00.868 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2024-09-16T09:26:00.868 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_session_metadata_threshold": "16777216", 2024-09-16T09:26:00.868 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_sessionmap_keys_per_op": "1024", 2024-09-16T09:26:00.868 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_shutdown_check": "0", 2024-09-16T09:26:00.868 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_skip_ino": "0", 2024-09-16T09:26:00.868 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_sleep_rank_change": "0.000000", 2024-09-16T09:26:00.868 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_snap_max_uid": "4294967294", 2024-09-16T09:26:00.868 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_snap_min_uid": "0", 2024-09-16T09:26:00.869 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_snap_rstat": "false", 2024-09-16T09:26:00.869 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_standby_replay_damaged": "false", 2024-09-16T09:26:00.869 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_symlink_recovery": "true", 2024-09-16T09:26:00.869 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_task_status_update_interval": "2.000000", 2024-09-16T09:26:00.869 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_thrash_exports": "0", 2024-09-16T09:26:00.869 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_thrash_fragments": "0", 2024-09-16T09:26:00.869 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_tick_interval": "5.000000", 2024-09-16T09:26:00.869 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_valgrind_exit": "false", 2024-09-16T09:26:00.869 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_verify_backtrace": "1", 2024-09-16T09:26:00.869 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_verify_scatter": "false", 2024-09-16T09:26:00.869 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_wipe_ino_prealloc": "false", 2024-09-16T09:26:00.869 INFO:teuthology.orchestra.run.smithi070.stdout: "mds_wipe_sessions": "false", 2024-09-16T09:26:00.869 INFO:teuthology.orchestra.run.smithi070.stdout: "mempool_debug": "false", 2024-09-16T09:26:00.870 INFO:teuthology.orchestra.run.smithi070.stdout: "memstore_debug_omit_block_device_write": "false", 2024-09-16T09:26:00.870 INFO:teuthology.orchestra.run.smithi070.stdout: "memstore_device_bytes": "1073741824", 2024-09-16T09:26:00.870 INFO:teuthology.orchestra.run.smithi070.stdout: "memstore_page_set": "false", 2024-09-16T09:26:00.870 INFO:teuthology.orchestra.run.smithi070.stdout: "memstore_page_size": "65536", 2024-09-16T09:26:00.870 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr_client_bytes": "134217728", 2024-09-16T09:26:00.870 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr_client_messages": "512", 2024-09-16T09:26:00.870 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2024-09-16T09:26:00.870 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr_connect_retry_interval": "1.000000", 2024-09-16T09:26:00.870 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2024-09-16T09:26:00.870 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2024-09-16T09:26:00.870 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr_disabled_modules": "", 2024-09-16T09:26:00.870 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr_enable_op_tracker": "true", 2024-09-16T09:26:00.870 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr_initial_modules": "restful iostat nfs", 2024-09-16T09:26:00.871 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr_max_pg_creating": "1024", 2024-09-16T09:26:00.871 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr_max_pg_num_change": "128", 2024-09-16T09:26:00.871 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr_mds_bytes": "134217728", 2024-09-16T09:26:00.871 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr_mds_messages": "128", 2024-09-16T09:26:00.871 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2024-09-16T09:26:00.871 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr_mon_bytes": "134217728", 2024-09-16T09:26:00.871 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr_mon_messages": "128", 2024-09-16T09:26:00.871 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr_num_op_tracker_shard": "32", 2024-09-16T09:26:00.871 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr_op_complaint_time": "30.000000", 2024-09-16T09:26:00.871 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr_op_history_duration": "600", 2024-09-16T09:26:00.871 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr_op_history_size": "20", 2024-09-16T09:26:00.871 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr_op_history_slow_op_size": "20", 2024-09-16T09:26:00.871 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr_op_history_slow_op_threshold": "10.000000", 2024-09-16T09:26:00.871 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr_op_log_threshold": "5", 2024-09-16T09:26:00.872 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr_osd_bytes": "536870912", 2024-09-16T09:26:00.872 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr_osd_messages": "8192", 2024-09-16T09:26:00.872 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr_pool": "true", 2024-09-16T09:26:00.872 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr_service_beacon_grace": "60.000000", 2024-09-16T09:26:00.872 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr_standby_modules": "true", 2024-09-16T09:26:00.872 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr_stats_period": "5", 2024-09-16T09:26:00.872 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr_stats_threshold": "5", 2024-09-16T09:26:00.872 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr_tick_period": "2", 2024-09-16T09:26:00.872 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr_ttl_cache_expire_seconds": "0", 2024-09-16T09:26:00.872 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_accept_timeout_factor": "2.000000", 2024-09-16T09:26:00.873 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_allow_pool_delete": "true", 2024-09-16T09:26:00.873 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_allow_pool_size_one": "false", 2024-09-16T09:26:00.873 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_auth_validate_all_caps": "true", 2024-09-16T09:26:00.873 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2024-09-16T09:26:00.873 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2024-09-16T09:26:00.873 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_client_bytes": "104857600", 2024-09-16T09:26:00.873 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_client_directed_command_retry": "2", 2024-09-16T09:26:00.873 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_client_hunt_interval": "3.000000", 2024-09-16T09:26:00.873 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2024-09-16T09:26:00.873 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2024-09-16T09:26:00.873 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2024-09-16T09:26:00.873 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_client_hunt_parallel": "3", 2024-09-16T09:26:00.873 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_client_log_interval": "1.000000", 2024-09-16T09:26:00.874 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_client_max_log_entries_per_message": "1000", 2024-09-16T09:26:00.874 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_client_ping_interval": "10.000000", 2024-09-16T09:26:00.874 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_client_ping_timeout": "30.000000", 2024-09-16T09:26:00.874 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_client_target_rank": "-1", 2024-09-16T09:26:00.874 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_clock_drift_allowed": "1.000000", 2024-09-16T09:26:00.874 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2024-09-16T09:26:00.874 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2024-09-16T09:26:00.874 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_cluster_log_level": "debug", 2024-09-16T09:26:00.874 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_cluster_log_to_file": "true", 2024-09-16T09:26:00.874 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_cluster_log_to_graylog": "false", 2024-09-16T09:26:00.875 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2024-09-16T09:26:00.875 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_cluster_log_to_graylog_port": "12201", 2024-09-16T09:26:00.875 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_cluster_log_to_journald": "true", 2024-09-16T09:26:00.875 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_cluster_log_to_stderr": "false", 2024-09-16T09:26:00.875 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_cluster_log_to_syslog": "default=false", 2024-09-16T09:26:00.875 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2024-09-16T09:26:00.875 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_compact_on_bootstrap": "false", 2024-09-16T09:26:00.875 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_compact_on_start": "false", 2024-09-16T09:26:00.875 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_compact_on_trim": "true", 2024-09-16T09:26:00.875 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_con_tracker_persist_interval": "10", 2024-09-16T09:26:00.875 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_con_tracker_score_halflife": "43200", 2024-09-16T09:26:00.875 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_config_key_max_entry_size": "65536", 2024-09-16T09:26:00.875 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_cpu_threads": "4", 2024-09-16T09:26:00.876 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_crush_min_required_version": "hammer", 2024-09-16T09:26:00.876 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_daemon_bytes": "419430400", 2024-09-16T09:26:00.876 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2024-09-16T09:26:00.876 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_data_avail_crit": "5", 2024-09-16T09:26:00.876 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_data_avail_warn": "5", 2024-09-16T09:26:00.876 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_data_size_warn": "16106127360", 2024-09-16T09:26:00.876 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_debug_block_osdmap_trim": "false", 2024-09-16T09:26:00.876 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_debug_deprecated_as_obsolete": "false", 2024-09-16T09:26:00.876 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_debug_dump_json": "false", 2024-09-16T09:26:00.876 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2024-09-16T09:26:00.876 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_debug_dump_transactions": "false", 2024-09-16T09:26:00.876 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_debug_extra_checks": "false", 2024-09-16T09:26:00.876 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_debug_no_initial_persistent_features": "false", 2024-09-16T09:26:00.877 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2024-09-16T09:26:00.877 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_debug_no_require_reef": "false", 2024-09-16T09:26:00.877 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_debug_no_require_squid": "false", 2024-09-16T09:26:00.877 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2024-09-16T09:26:00.877 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_delta_reset_interval": "10.000000", 2024-09-16T09:26:00.877 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_dns_srv_name": "ceph-mon", 2024-09-16T09:26:00.877 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_down_mkfs_grace": "60", 2024-09-16T09:26:00.877 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_down_uptime_grace": "60", 2024-09-16T09:26:00.877 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_election_default_strategy": "1", 2024-09-16T09:26:00.877 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_election_timeout": "5.000000", 2024-09-16T09:26:00.877 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2024-09-16T09:26:00.878 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_elector_ping_divisor": "2", 2024-09-16T09:26:00.878 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_elector_ping_timeout": "2.000000", 2024-09-16T09:26:00.878 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_enable_op_tracker": "true", 2024-09-16T09:26:00.878 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_fake_pool_delete": "false", 2024-09-16T09:26:00.878 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_force_quorum_join": "false", 2024-09-16T09:26:00.878 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_fsmap_prune_threshold": "300", 2024-09-16T09:26:00.878 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_globalid_prealloc": "10000", 2024-09-16T09:26:00.878 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_health_detail_to_clog": "true", 2024-09-16T09:26:00.878 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_health_log_update_period": "5", 2024-09-16T09:26:00.879 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_health_max_detail": "50", 2024-09-16T09:26:00.879 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_health_to_clog": "true", 2024-09-16T09:26:00.879 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_health_to_clog_interval": "600", 2024-09-16T09:26:00.879 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2024-09-16T09:26:00.879 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_host": "[v2:172.21.15.70:3300/0,v1:172.21.15.70:6789/0]", 2024-09-16T09:26:00.879 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_host_override": "", 2024-09-16T09:26:00.879 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_initial_members": "", 2024-09-16T09:26:00.879 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2024-09-16T09:26:00.879 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2024-09-16T09:26:00.879 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_inject_transaction_delay_max": "10.000000", 2024-09-16T09:26:00.879 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2024-09-16T09:26:00.880 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_keyvaluedb": "rocksdb", 2024-09-16T09:26:00.880 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_lease": "5.000000", 2024-09-16T09:26:00.880 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2024-09-16T09:26:00.880 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_lease_renew_interval_factor": "0.600000", 2024-09-16T09:26:00.880 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_log_full_interval": "50", 2024-09-16T09:26:00.880 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_log_max": "10000", 2024-09-16T09:26:00.880 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_log_max_summary": "50", 2024-09-16T09:26:00.880 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_max_log_entries_per_event": "4096", 2024-09-16T09:26:00.880 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_max_log_epochs": "500", 2024-09-16T09:26:00.880 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_max_mdsmap_epochs": "500", 2024-09-16T09:26:00.880 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_max_mgrmap_epochs": "500", 2024-09-16T09:26:00.880 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_max_nvmeof_epochs": "500", 2024-09-16T09:26:00.880 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_max_osd": "10000", 2024-09-16T09:26:00.881 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_max_pg_per_osd": "10000", 2024-09-16T09:26:00.881 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_max_pool_pg_num": "65536", 2024-09-16T09:26:00.881 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_max_snap_prune_per_epoch": "100", 2024-09-16T09:26:00.881 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_mds_blocklist_interval": "86400.000000", 2024-09-16T09:26:00.881 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_mds_force_trim_to": "0", 2024-09-16T09:26:00.881 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_mds_skip_sanity": "false", 2024-09-16T09:26:00.881 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_memory_autotune": "true", 2024-09-16T09:26:00.881 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_memory_target": "2147483648", 2024-09-16T09:26:00.881 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_mgr_beacon_grace": "30", 2024-09-16T09:26:00.881 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2024-09-16T09:26:00.881 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_mgr_digest_period": "5", 2024-09-16T09:26:00.881 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_mgr_inactive_grace": "60", 2024-09-16T09:26:00.881 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_mgr_mkfs_grace": "240", 2024-09-16T09:26:00.881 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2024-09-16T09:26:00.882 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_min_osdmap_epochs": "50", 2024-09-16T09:26:00.882 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_nvmeofgw_beacon_grace": "10", 2024-09-16T09:26:00.882 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_nvmeofgw_set_group_id_retry": "1000", 2024-09-16T09:26:00.882 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_op_complaint_time": "30", 2024-09-16T09:26:00.882 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_op_history_duration": "600", 2024-09-16T09:26:00.882 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_op_history_size": "20", 2024-09-16T09:26:00.882 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_op_history_slow_op_size": "20", 2024-09-16T09:26:00.882 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_op_history_slow_op_threshold": "10", 2024-09-16T09:26:00.882 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_op_log_threshold": "5", 2024-09-16T09:26:00.882 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_osd_adjust_down_out_interval": "true", 2024-09-16T09:26:00.882 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2024-09-16T09:26:00.882 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2024-09-16T09:26:00.882 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_osd_auto_mark_in": "false", 2024-09-16T09:26:00.882 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_osd_auto_mark_new_in": "true", 2024-09-16T09:26:00.883 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2024-09-16T09:26:00.883 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2024-09-16T09:26:00.883 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_osd_cache_size": "500", 2024-09-16T09:26:00.883 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_osd_cache_size_min": "134217728", 2024-09-16T09:26:00.883 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_osd_crush_smoke_test": "true", 2024-09-16T09:26:00.883 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_osd_destroyed_out_interval": "600", 2024-09-16T09:26:00.883 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_osd_down_out_interval": "600", 2024-09-16T09:26:00.883 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_osd_down_out_subtree_limit": "rack", 2024-09-16T09:26:00.884 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2024-09-16T09:26:00.884 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_osd_force_trim_to": "0", 2024-09-16T09:26:00.884 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_osd_full_ratio": "0.950000", 2024-09-16T09:26:00.884 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2024-09-16T09:26:00.884 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_osd_laggy_halflife": "3600", 2024-09-16T09:26:00.884 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_osd_laggy_max_interval": "300", 2024-09-16T09:26:00.884 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_osd_laggy_weight": "0.300000", 2024-09-16T09:26:00.884 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2024-09-16T09:26:00.884 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_osd_max_initial_pgs": "1024", 2024-09-16T09:26:00.884 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_osd_min_down_reporters": "2", 2024-09-16T09:26:00.884 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_osd_min_in_ratio": "0.750000", 2024-09-16T09:26:00.884 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_osd_min_up_ratio": "0.300000", 2024-09-16T09:26:00.884 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_osd_nearfull_ratio": "0.850000", 2024-09-16T09:26:00.885 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_osd_prime_pg_temp": "true", 2024-09-16T09:26:00.885 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2024-09-16T09:26:00.885 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2024-09-16T09:26:00.885 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_osd_report_timeout": "900", 2024-09-16T09:26:00.885 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_osd_reporter_subtree_level": "osd", 2024-09-16T09:26:00.885 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2024-09-16T09:26:00.885 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_osd_warn_num_repaired": "10", 2024-09-16T09:26:00.885 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_osd_warn_op_age": "32.000000", 2024-09-16T09:26:00.885 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_osdmap_full_prune_enabled": "true", 2024-09-16T09:26:00.885 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_osdmap_full_prune_interval": "2", 2024-09-16T09:26:00.885 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_osdmap_full_prune_min": "15", 2024-09-16T09:26:00.885 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_osdmap_full_prune_txsize": "2", 2024-09-16T09:26:00.886 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2024-09-16T09:26:00.886 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_pg_stuck_threshold": "60", 2024-09-16T09:26:00.886 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2024-09-16T09:26:00.886 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_pg_warn_min_objects": "10000", 2024-09-16T09:26:00.886 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_pg_warn_min_per_osd": "0", 2024-09-16T09:26:00.886 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_pg_warn_min_pool_objects": "1000", 2024-09-16T09:26:00.886 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_pool_quota_crit_threshold": "0", 2024-09-16T09:26:00.886 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_pool_quota_warn_threshold": "0", 2024-09-16T09:26:00.886 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_probe_timeout": "2.000000", 2024-09-16T09:26:00.886 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_reweight_max_change": "0.050000", 2024-09-16T09:26:00.886 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_reweight_max_osds": "4", 2024-09-16T09:26:00.886 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_reweight_min_bytes_per_osd": "10", 2024-09-16T09:26:00.886 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_reweight_min_pgs_per_osd": "4", 2024-09-16T09:26:00.886 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2024-09-16T09:26:00.887 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2024-09-16T09:26:00.887 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2024-09-16T09:26:00.887 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_scrub_interval": "86400", 2024-09-16T09:26:00.887 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_scrub_max_keys": "100", 2024-09-16T09:26:00.887 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_scrub_timeout": "300", 2024-09-16T09:26:00.887 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_session_timeout": "300", 2024-09-16T09:26:00.887 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_smart_report_timeout": "5", 2024-09-16T09:26:00.887 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_stat_smooth_intervals": "6", 2024-09-16T09:26:00.887 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2024-09-16T09:26:00.887 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_stretch_pool_min_size": "2", 2024-09-16T09:26:00.887 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_stretch_pool_size": "4", 2024-09-16T09:26:00.887 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2024-09-16T09:26:00.888 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_subscribe_interval": "86400.000000", 2024-09-16T09:26:00.888 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_sync_debug": "false", 2024-09-16T09:26:00.888 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_sync_max_payload_keys": "2000", 2024-09-16T09:26:00.888 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_sync_max_payload_size": "1048576", 2024-09-16T09:26:00.888 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_sync_provider_kill_at": "0", 2024-09-16T09:26:00.888 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_sync_requester_kill_at": "0", 2024-09-16T09:26:00.888 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_sync_timeout": "60.000000", 2024-09-16T09:26:00.888 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_target_pg_per_osd": "100", 2024-09-16T09:26:00.888 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_tick_interval": "5", 2024-09-16T09:26:00.888 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_timecheck_interval": "300.000000", 2024-09-16T09:26:00.888 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_timecheck_skew_interval": "30.000000", 2024-09-16T09:26:00.888 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_use_min_delay_socket": "false", 2024-09-16T09:26:00.888 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_warn_older_version_delay": "604800", 2024-09-16T09:26:00.889 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2024-09-16T09:26:00.889 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2024-09-16T09:26:00.889 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2024-09-16T09:26:00.889 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_warn_on_filestore_osds": "true", 2024-09-16T09:26:00.889 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2024-09-16T09:26:00.889 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2024-09-16T09:26:00.889 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2024-09-16T09:26:00.889 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_warn_on_misplaced": "false", 2024-09-16T09:26:00.889 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2024-09-16T09:26:00.889 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_warn_on_older_version": "true", 2024-09-16T09:26:00.889 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2024-09-16T09:26:00.889 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_warn_on_pool_no_app": "true", 2024-09-16T09:26:00.889 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_warn_on_pool_no_redundancy": "true", 2024-09-16T09:26:00.890 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2024-09-16T09:26:00.890 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2024-09-16T09:26:00.890 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2024-09-16T09:26:00.890 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_warn_on_too_few_osds": "false", 2024-09-16T09:26:00.890 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2024-09-16T09:26:00.890 INFO:teuthology.orchestra.run.smithi070.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2024-09-16T09:26:00.890 INFO:teuthology.orchestra.run.smithi070.stdout: "monmap": "", 2024-09-16T09:26:00.890 INFO:teuthology.orchestra.run.smithi070.stdout: "motr_admin_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-16T09:26:00.890 INFO:teuthology.orchestra.run.smithi070.stdout: "motr_admin_fid": "0x7200000000000001:0x0", 2024-09-16T09:26:00.891 INFO:teuthology.orchestra.run.smithi070.stdout: "motr_ha_endpoint": "192.168.180.182@tcp:12345:1:1", 2024-09-16T09:26:00.891 INFO:teuthology.orchestra.run.smithi070.stdout: "motr_my_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-16T09:26:00.891 INFO:teuthology.orchestra.run.smithi070.stdout: "motr_my_fid": "0x7200000000000001:0x0", 2024-09-16T09:26:00.891 INFO:teuthology.orchestra.run.smithi070.stdout: "motr_profile_fid": "0x7000000000000001:0x0", 2024-09-16T09:26:00.891 INFO:teuthology.orchestra.run.smithi070.stdout: "motr_tracing_enabled": "false", 2024-09-16T09:26:00.891 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_async_op_threads": "3", 2024-09-16T09:26:00.892 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_async_rdma_buffer_size": "131072", 2024-09-16T09:26:00.892 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_async_rdma_cm": "false", 2024-09-16T09:26:00.892 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_async_rdma_device_name": "", 2024-09-16T09:26:00.892 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_async_rdma_dscp": "96", 2024-09-16T09:26:00.892 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_async_rdma_enable_hugepage": "false", 2024-09-16T09:26:00.892 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_async_rdma_gid_idx": "0", 2024-09-16T09:26:00.892 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_async_rdma_local_gid": "", 2024-09-16T09:26:00.892 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_async_rdma_polling_us": "1000", 2024-09-16T09:26:00.892 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_async_rdma_port_num": "1", 2024-09-16T09:26:00.892 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_async_rdma_receive_buffers": "32768", 2024-09-16T09:26:00.892 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_async_rdma_receive_queue_len": "4096", 2024-09-16T09:26:00.893 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_async_rdma_roce_ver": "1", 2024-09-16T09:26:00.893 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_async_rdma_send_buffers": "1024", 2024-09-16T09:26:00.893 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_async_rdma_sl": "3", 2024-09-16T09:26:00.893 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_async_rdma_support_srq": "true", 2024-09-16T09:26:00.893 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_async_rdma_type": "ib", 2024-09-16T09:26:00.893 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_async_reap_threshold": "5", 2024-09-16T09:26:00.893 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_bind_before_connect": "false", 2024-09-16T09:26:00.893 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_bind_ipv4": "true", 2024-09-16T09:26:00.893 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_bind_ipv6": "false", 2024-09-16T09:26:00.893 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_bind_msgr1": "true", 2024-09-16T09:26:00.894 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_bind_msgr2": "true", 2024-09-16T09:26:00.894 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_bind_port_max": "7568", 2024-09-16T09:26:00.894 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_bind_port_min": "6800", 2024-09-16T09:26:00.894 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_bind_prefer_ipv4": "false", 2024-09-16T09:26:00.894 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_bind_retry_count": "3", 2024-09-16T09:26:00.894 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_bind_retry_delay": "5", 2024-09-16T09:26:00.894 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_blackhole_client": "false", 2024-09-16T09:26:00.894 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_blackhole_mds": "false", 2024-09-16T09:26:00.894 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_blackhole_mgr": "false", 2024-09-16T09:26:00.894 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_blackhole_mon": "false", 2024-09-16T09:26:00.894 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_blackhole_osd": "false", 2024-09-16T09:26:00.894 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_client_mode": "crc secure", 2024-09-16T09:26:00.895 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_client_throttle_retry_time_interval": "5000", 2024-09-16T09:26:00.895 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_cluster_mode": "crc secure", 2024-09-16T09:26:00.895 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_cluster_type": "", 2024-09-16T09:26:00.895 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_compress_secure": "false", 2024-09-16T09:26:00.895 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_connection_idle_timeout": "900", 2024-09-16T09:26:00.895 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_connection_ready_timeout": "10", 2024-09-16T09:26:00.895 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_crc_data": "true", 2024-09-16T09:26:00.895 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_crc_header": "true", 2024-09-16T09:26:00.895 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_die_on_bad_msg": "false", 2024-09-16T09:26:00.895 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_die_on_bug": "true", 2024-09-16T09:26:00.895 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_die_on_old_message": "true", 2024-09-16T09:26:00.895 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_die_on_skipped_message": "false", 2024-09-16T09:26:00.896 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_die_on_unhandled_msg": "false", 2024-09-16T09:26:00.896 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_dispatch_throttle_bytes": "104857600", 2024-09-16T09:26:00.896 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_dpdk_coremask": "0xF", 2024-09-16T09:26:00.896 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_dpdk_debug_allow_loopback": "false", 2024-09-16T09:26:00.896 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_dpdk_devs_allowlist": "", 2024-09-16T09:26:00.896 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_dpdk_enable_tso": "true", 2024-09-16T09:26:00.896 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2024-09-16T09:26:00.897 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_dpdk_host_ipv4_addr": "", 2024-09-16T09:26:00.897 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_dpdk_hugepages": "", 2024-09-16T09:26:00.897 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_dpdk_hw_flow_control": "true", 2024-09-16T09:26:00.897 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2024-09-16T09:26:00.897 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_dpdk_lro": "true", 2024-09-16T09:26:00.897 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_dpdk_memory_channel": "4", 2024-09-16T09:26:00.897 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2024-09-16T09:26:00.897 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_dpdk_pmd": "", 2024-09-16T09:26:00.897 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_dpdk_port_id": "0", 2024-09-16T09:26:00.897 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2024-09-16T09:26:00.897 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_dump_corrupt_message_level": "1", 2024-09-16T09:26:00.898 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_dump_on_send": "false", 2024-09-16T09:26:00.898 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_initial_backoff": "0.200000", 2024-09-16T09:26:00.898 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_inject_delay_max": "1.000000", 2024-09-16T09:26:00.898 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_inject_delay_probability": "0.000000", 2024-09-16T09:26:00.898 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_inject_delay_type": "", 2024-09-16T09:26:00.898 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_inject_internal_delays": "0.000000", 2024-09-16T09:26:00.898 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_inject_network_congestion": "0", 2024-09-16T09:26:00.898 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_inject_socket_failures": "0", 2024-09-16T09:26:00.898 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_learn_addr_from_peer": "true", 2024-09-16T09:26:00.898 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_max_accept_failures": "4", 2024-09-16T09:26:00.898 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_max_backoff": "15.000000", 2024-09-16T09:26:00.898 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_mon_client_mode": "secure crc", 2024-09-16T09:26:00.899 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_mon_cluster_mode": "secure crc", 2024-09-16T09:26:00.899 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_mon_service_mode": "secure crc", 2024-09-16T09:26:00.899 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_osd_compress_min_size": "1024", 2024-09-16T09:26:00.899 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_osd_compress_mode": "none", 2024-09-16T09:26:00.899 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_osd_compression_algorithm": "snappy", 2024-09-16T09:26:00.899 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2024-09-16T09:26:00.899 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_pq_min_cost": "65536", 2024-09-16T09:26:00.899 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_public_type": "", 2024-09-16T09:26:00.899 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_service_mode": "crc secure", 2024-09-16T09:26:00.899 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_tcp_listen_backlog": "512", 2024-09-16T09:26:00.899 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_tcp_nodelay": "true", 2024-09-16T09:26:00.899 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_tcp_prefetch_max_size": "4096", 2024-09-16T09:26:00.900 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_tcp_rcvbuf": "0", 2024-09-16T09:26:00.900 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_time_events_min_wait_interval": "1000", 2024-09-16T09:26:00.900 INFO:teuthology.orchestra.run.smithi070.stdout: "ms_type": "async+posix", 2024-09-16T09:26:00.900 INFO:teuthology.orchestra.run.smithi070.stdout: "no_config_file": "false", 2024-09-16T09:26:00.900 INFO:teuthology.orchestra.run.smithi070.stdout: "nvmeof_mon_client_disconnect_panic": "100", 2024-09-16T09:26:00.900 INFO:teuthology.orchestra.run.smithi070.stdout: "nvmeof_mon_client_tick_period": "2", 2024-09-16T09:26:00.900 INFO:teuthology.orchestra.run.smithi070.stdout: "objecter_completion_locks_per_session": "32", 2024-09-16T09:26:00.900 INFO:teuthology.orchestra.run.smithi070.stdout: "objecter_debug_inject_relock_delay": "false", 2024-09-16T09:26:00.900 INFO:teuthology.orchestra.run.smithi070.stdout: "objecter_inflight_op_bytes": "104857600", 2024-09-16T09:26:00.900 INFO:teuthology.orchestra.run.smithi070.stdout: "objecter_inflight_ops": "1024", 2024-09-16T09:26:00.900 INFO:teuthology.orchestra.run.smithi070.stdout: "objecter_inject_no_watch_ping": "false", 2024-09-16T09:26:00.900 INFO:teuthology.orchestra.run.smithi070.stdout: "objecter_retry_writes_after_first_reply": "false", 2024-09-16T09:26:00.901 INFO:teuthology.orchestra.run.smithi070.stdout: "objecter_tick_interval": "5.000000", 2024-09-16T09:26:00.901 INFO:teuthology.orchestra.run.smithi070.stdout: "objecter_timeout": "10.000000", 2024-09-16T09:26:00.901 INFO:teuthology.orchestra.run.smithi070.stdout: "objectstore_blackhole": "false", 2024-09-16T09:26:00.901 INFO:teuthology.orchestra.run.smithi070.stdout: "objectstore_debug_throw_on_failed_txc": "false", 2024-09-16T09:26:00.901 INFO:teuthology.orchestra.run.smithi070.stdout: "openssl_engine_opts": "", 2024-09-16T09:26:00.901 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_agent_delay_time": "5.000000", 2024-09-16T09:26:00.901 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_agent_hist_halflife": "1000", 2024-09-16T09:26:00.901 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_agent_max_low_ops": "2", 2024-09-16T09:26:00.901 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_agent_max_ops": "4", 2024-09-16T09:26:00.901 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_agent_min_evict_effort": "0.100000", 2024-09-16T09:26:00.901 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_agent_quantize_effort": "0.100000", 2024-09-16T09:26:00.901 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_agent_slop": "0.020000", 2024-09-16T09:26:00.902 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_aggregated_slow_ops_logging": "true", 2024-09-16T09:26:00.902 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_allow_recovery_below_min_size": "true", 2024-09-16T09:26:00.902 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_asio_thread_count": "2", 2024-09-16T09:26:00.902 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_async_recovery_min_cost": "100", 2024-09-16T09:26:00.902 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_auto_mark_unfound_lost": "false", 2024-09-16T09:26:00.902 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_backfill_retry_interval": "30.000000", 2024-09-16T09:26:00.902 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_backfill_scan_max": "512", 2024-09-16T09:26:00.902 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_backfill_scan_min": "64", 2024-09-16T09:26:00.902 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_backoff_on_degraded": "false", 2024-09-16T09:26:00.902 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_backoff_on_peering": "false", 2024-09-16T09:26:00.902 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_backoff_on_unfound": "true", 2024-09-16T09:26:00.902 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_beacon_report_interval": "300", 2024-09-16T09:26:00.903 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_bench_duration": "30", 2024-09-16T09:26:00.903 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_bench_large_size_max_throughput": "104857600", 2024-09-16T09:26:00.903 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_bench_max_block_size": "67108864", 2024-09-16T09:26:00.903 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_bench_small_size_max_iops": "100", 2024-09-16T09:26:00.903 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_blkin_trace_all": "false", 2024-09-16T09:26:00.903 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_blocked_scrub_grace_period": "120", 2024-09-16T09:26:00.903 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2024-09-16T09:26:00.905 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_calc_pg_upmaps_aggressively_fast": "true", 2024-09-16T09:26:00.905 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2024-09-16T09:26:00.906 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_check_for_log_corruption": "false", 2024-09-16T09:26:00.906 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_check_max_object_name_len_on_startup": "true", 2024-09-16T09:26:00.906 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-16T09:26:00.906 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_class_dir": "/usr/lib64/rados-classes", 2024-09-16T09:26:00.906 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-16T09:26:00.906 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_class_update_on_start": "true", 2024-09-16T09:26:00.907 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_client_message_cap": "256", 2024-09-16T09:26:00.907 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_client_message_size_cap": "524288000", 2024-09-16T09:26:00.907 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_client_op_priority": "63", 2024-09-16T09:26:00.907 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_client_watch_timeout": "30", 2024-09-16T09:26:00.907 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_command_max_records": "256", 2024-09-16T09:26:00.907 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_compact_on_start": "false", 2024-09-16T09:26:00.908 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_copyfrom_max_chunk": "8388608", 2024-09-16T09:26:00.908 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_crush_chooseleaf_type": "1", 2024-09-16T09:26:00.908 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_crush_initial_weight": "-1.000000", 2024-09-16T09:26:00.908 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_crush_update_on_start": "true", 2024-09-16T09:26:00.908 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_crush_update_weight_set": "true", 2024-09-16T09:26:00.908 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2024-09-16T09:26:00.909 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2024-09-16T09:26:00.909 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2024-09-16T09:26:00.909 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_debug_drop_ping_duration": "0", 2024-09-16T09:26:00.909 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_debug_drop_ping_probability": "0.000000", 2024-09-16T09:26:00.909 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_debug_feed_pullee": "-1", 2024-09-16T09:26:00.910 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_debug_inject_copyfrom_error": "false", 2024-09-16T09:26:00.910 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2024-09-16T09:26:00.910 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2024-09-16T09:26:00.910 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_debug_misdirected_ops": "false", 2024-09-16T09:26:00.910 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_debug_no_acting_change": "false", 2024-09-16T09:26:00.910 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_debug_no_purge_strays": "false", 2024-09-16T09:26:00.911 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_debug_op_order": "false", 2024-09-16T09:26:00.911 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_debug_pg_log_writeout": "false", 2024-09-16T09:26:00.911 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_debug_pretend_recovery_active": "false", 2024-09-16T09:26:00.911 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_debug_random_push_read_error": "0.000000", 2024-09-16T09:26:00.911 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2024-09-16T09:26:00.911 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_debug_shutdown": "false", 2024-09-16T09:26:00.912 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2024-09-16T09:26:00.912 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2024-09-16T09:26:00.912 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_debug_trim_objects": "false", 2024-09-16T09:26:00.912 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_debug_verify_cached_snaps": "false", 2024-09-16T09:26:00.912 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_debug_verify_missing_on_start": "false", 2024-09-16T09:26:00.913 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_debug_verify_snaps": "false", 2024-09-16T09:26:00.913 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_debug_verify_stray_on_activate": "false", 2024-09-16T09:26:00.913 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_deep_scrub_interval": "604800.000000", 2024-09-16T09:26:00.913 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_deep_scrub_keys": "1024", 2024-09-16T09:26:00.913 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2024-09-16T09:26:00.913 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2024-09-16T09:26:00.914 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2024-09-16T09:26:00.914 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_deep_scrub_stride": "524288", 2024-09-16T09:26:00.914 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2024-09-16T09:26:00.914 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_default_data_pool_replay_window": "45", 2024-09-16T09:26:00.914 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_default_notify_timeout": "30", 2024-09-16T09:26:00.914 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_delete_sleep": "0.000000", 2024-09-16T09:26:00.915 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_delete_sleep_hdd": "5.000000", 2024-09-16T09:26:00.915 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_delete_sleep_hybrid": "1.000000", 2024-09-16T09:26:00.915 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_delete_sleep_ssd": "1.000000", 2024-09-16T09:26:00.915 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_discard_disconnected_ops": "true", 2024-09-16T09:26:00.915 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_ec_partial_reads": "true", 2024-09-16T09:26:00.916 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_enable_op_tracker": "true", 2024-09-16T09:26:00.916 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2024-09-16T09:26:00.916 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_extblkdev_plugins": "vdo", 2024-09-16T09:26:00.916 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_failsafe_full_ratio": "0.970000", 2024-09-16T09:26:00.916 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_fast_fail_on_connection_refused": "true", 2024-09-16T09:26:00.916 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_fast_info": "true", 2024-09-16T09:26:00.917 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_fast_shutdown": "true", 2024-09-16T09:26:00.917 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_fast_shutdown_notify_mon": "true", 2024-09-16T09:26:00.917 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_fast_shutdown_timeout": "15", 2024-09-16T09:26:00.917 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_find_best_info_ignore_history_les": "false", 2024-09-16T09:26:00.917 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_force_auth_primary_missing_objects": "100", 2024-09-16T09:26:00.918 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2024-09-16T09:26:00.918 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_function_tracing": "false", 2024-09-16T09:26:00.918 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_heartbeat_grace": "20", 2024-09-16T09:26:00.918 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_heartbeat_interval": "6", 2024-09-16T09:26:00.918 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2024-09-16T09:26:00.918 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_heartbeat_min_peers": "10", 2024-09-16T09:26:00.919 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_heartbeat_min_size": "2000", 2024-09-16T09:26:00.919 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_heartbeat_stale": "600", 2024-09-16T09:26:00.919 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2024-09-16T09:26:00.919 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_hit_set_max_size": "100000", 2024-09-16T09:26:00.919 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_hit_set_min_size": "1000", 2024-09-16T09:26:00.919 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_hit_set_namespace": ".ceph-internal", 2024-09-16T09:26:00.920 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_ignore_stale_divergent_priors": "false", 2024-09-16T09:26:00.920 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2024-09-16T09:26:00.920 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_inject_failure_on_pg_removal": "false", 2024-09-16T09:26:00.920 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2024-09-16T09:26:00.920 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_journal_flush_on_shutdown": "true", 2024-09-16T09:26:00.920 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_journal_size": "5120", 2024-09-16T09:26:00.921 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_kill_backfill_at": "0", 2024-09-16T09:26:00.921 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_loop_before_reset_tphandle": "64", 2024-09-16T09:26:00.921 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_map_cache_size": "50", 2024-09-16T09:26:00.921 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_map_dedup": "true", 2024-09-16T09:26:00.921 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_map_message_max": "40", 2024-09-16T09:26:00.921 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_map_message_max_bytes": "10485760", 2024-09-16T09:26:00.922 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_map_share_max_epochs": "40", 2024-09-16T09:26:00.922 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_max_attr_name_len": "100", 2024-09-16T09:26:00.922 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_max_attr_size": "0", 2024-09-16T09:26:00.922 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_max_backfills": "1", 2024-09-16T09:26:00.922 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_max_markdown_count": "5", 2024-09-16T09:26:00.922 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_max_markdown_period": "600", 2024-09-16T09:26:00.923 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_max_object_name_len": "2048", 2024-09-16T09:26:00.923 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_max_object_namespace_len": "256", 2024-09-16T09:26:00.923 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_max_object_size": "134217728", 2024-09-16T09:26:00.923 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2024-09-16T09:26:00.923 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_max_omap_entries_per_request": "1024", 2024-09-16T09:26:00.923 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_max_pg_blocked_by": "16", 2024-09-16T09:26:00.924 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_max_pg_log_entries": "10000", 2024-09-16T09:26:00.924 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2024-09-16T09:26:00.924 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_max_pgls": "1024", 2024-09-16T09:26:00.924 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_max_push_cost": "8388608", 2024-09-16T09:26:00.924 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_max_push_objects": "10", 2024-09-16T09:26:00.924 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_max_scrubs": "3", 2024-09-16T09:26:00.925 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2024-09-16T09:26:00.925 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_max_trimming_pgs": "2", 2024-09-16T09:26:00.925 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_max_write_op_reply_len": "64", 2024-09-16T09:26:00.925 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_max_write_size": "90", 2024-09-16T09:26:00.925 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2024-09-16T09:26:00.925 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2024-09-16T09:26:00.926 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2024-09-16T09:26:00.926 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2024-09-16T09:26:00.926 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2024-09-16T09:26:00.926 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2024-09-16T09:26:00.926 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2024-09-16T09:26:00.926 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_mclock_override_recovery_settings": "false", 2024-09-16T09:26:00.927 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_mclock_profile": "balanced", 2024-09-16T09:26:00.927 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2024-09-16T09:26:00.927 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2024-09-16T09:26:00.927 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2024-09-16T09:26:00.927 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2024-09-16T09:26:00.927 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2024-09-16T09:26:00.928 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2024-09-16T09:26:00.928 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2024-09-16T09:26:00.928 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2024-09-16T09:26:00.928 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2024-09-16T09:26:00.928 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_mclock_scheduler_client_wgt": "1", 2024-09-16T09:26:00.929 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_mclock_skip_benchmark": "false", 2024-09-16T09:26:00.929 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_memory_base": "805306368", 2024-09-16T09:26:00.929 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_memory_cache_min": "134217728", 2024-09-16T09:26:00.929 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_memory_cache_resize_interval": "1.000000", 2024-09-16T09:26:00.929 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_memory_expected_fragmentation": "0.150000", 2024-09-16T09:26:00.929 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_memory_target": "4294967296", 2024-09-16T09:26:00.930 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_memory_target_autotune": "false", 2024-09-16T09:26:00.930 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2024-09-16T09:26:00.930 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_min_pg_log_entries": "250", 2024-09-16T09:26:00.930 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_min_recovery_priority": "0", 2024-09-16T09:26:00.930 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_mon_heartbeat_interval": "30", 2024-09-16T09:26:00.930 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2024-09-16T09:26:00.930 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_mon_report_interval": "5", 2024-09-16T09:26:00.930 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_mon_report_max_in_flight": "2", 2024-09-16T09:26:00.931 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_mon_shutdown_timeout": "5.000000", 2024-09-16T09:26:00.931 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_num_cache_shards": "32", 2024-09-16T09:26:00.931 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_num_op_tracker_shard": "32", 2024-09-16T09:26:00.931 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_numa_auto_affinity": "true", 2024-09-16T09:26:00.931 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_numa_node": "-1", 2024-09-16T09:26:00.931 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_numa_prefer_iface": "true", 2024-09-16T09:26:00.932 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_object_clean_region_max_num_intervals": "10", 2024-09-16T09:26:00.932 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_objecter_finishers": "1", 2024-09-16T09:26:00.932 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_objectstore": "bluestore", 2024-09-16T09:26:00.932 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_objectstore_fuse": "false", 2024-09-16T09:26:00.932 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_objectstore_ideal_list_max": "64", 2024-09-16T09:26:00.932 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_objectstore_tracing": "false", 2024-09-16T09:26:00.932 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_op_complaint_time": "30.000000", 2024-09-16T09:26:00.932 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_op_history_duration": "600", 2024-09-16T09:26:00.932 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_op_history_size": "20", 2024-09-16T09:26:00.933 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_op_history_slow_op_size": "20", 2024-09-16T09:26:00.933 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2024-09-16T09:26:00.933 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_op_log_threshold": "5", 2024-09-16T09:26:00.933 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_op_num_shards": "0", 2024-09-16T09:26:00.933 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_op_num_shards_hdd": "5", 2024-09-16T09:26:00.933 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_op_num_shards_ssd": "8", 2024-09-16T09:26:00.933 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_op_num_threads_per_shard": "0", 2024-09-16T09:26:00.933 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2024-09-16T09:26:00.933 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2024-09-16T09:26:00.933 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2024-09-16T09:26:00.933 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_op_pq_min_cost": "65536", 2024-09-16T09:26:00.933 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_op_queue": "mclock_scheduler", 2024-09-16T09:26:00.933 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_op_queue_cut_off": "high", 2024-09-16T09:26:00.933 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_op_thread_suicide_timeout": "150", 2024-09-16T09:26:00.933 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_op_thread_timeout": "15", 2024-09-16T09:26:00.934 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_open_classes_on_start": "true", 2024-09-16T09:26:00.934 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_os_flags": "0", 2024-09-16T09:26:00.934 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_peering_op_priority": "255", 2024-09-16T09:26:00.934 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_pg_delete_cost": "1048576", 2024-09-16T09:26:00.934 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_pg_delete_priority": "5", 2024-09-16T09:26:00.934 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2024-09-16T09:26:00.934 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2024-09-16T09:26:00.934 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_pg_log_dups_tracked": "3000", 2024-09-16T09:26:00.934 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_pg_log_trim_max": "10000", 2024-09-16T09:26:00.934 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_pg_log_trim_min": "100", 2024-09-16T09:26:00.934 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2024-09-16T09:26:00.934 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_pg_object_context_cache_count": "64", 2024-09-16T09:26:00.934 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_pg_stat_report_interval_max_epochs": "500", 2024-09-16T09:26:00.934 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_pg_stat_report_interval_max_seconds": "5", 2024-09-16T09:26:00.935 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2024-09-16T09:26:00.935 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_pool_default_cache_min_evict_age": "0", 2024-09-16T09:26:00.935 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_pool_default_cache_min_flush_age": "0", 2024-09-16T09:26:00.935 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2024-09-16T09:26:00.935 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2024-09-16T09:26:00.935 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2024-09-16T09:26:00.935 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_pool_default_crimson": "false", 2024-09-16T09:26:00.935 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_pool_default_crush_rule": "-1", 2024-09-16T09:26:00.935 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_pool_default_ec_fast_read": "false", 2024-09-16T09:26:00.935 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2024-09-16T09:26:00.935 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_pool_default_flag_bulk": "false", 2024-09-16T09:26:00.935 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_pool_default_flag_hashpspool": "true", 2024-09-16T09:26:00.935 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_pool_default_flag_nodelete": "false", 2024-09-16T09:26:00.935 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_pool_default_flag_nopgchange": "false", 2024-09-16T09:26:00.936 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_pool_default_flag_nosizechange": "false", 2024-09-16T09:26:00.936 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_pool_default_flags": "0", 2024-09-16T09:26:00.936 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2024-09-16T09:26:00.936 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_pool_default_min_size": "0", 2024-09-16T09:26:00.936 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2024-09-16T09:26:00.936 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_pool_default_pg_num": "32", 2024-09-16T09:26:00.936 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_pool_default_pgp_num": "0", 2024-09-16T09:26:00.936 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2024-09-16T09:26:00.936 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_pool_default_read_ratio": "70", 2024-09-16T09:26:00.936 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_pool_default_size": "3", 2024-09-16T09:26:00.936 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_pool_default_type": "replicated", 2024-09-16T09:26:00.936 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2024-09-16T09:26:00.936 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_pool_use_gmt_hitset": "true", 2024-09-16T09:26:00.937 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_push_per_object_cost": "1000", 2024-09-16T09:26:00.937 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_read_ec_check_for_errors": "false", 2024-09-16T09:26:00.937 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_recover_clone_overlap": "true", 2024-09-16T09:26:00.937 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_recover_clone_overlap_limit": "10", 2024-09-16T09:26:00.937 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_recovery_cost": "20971520", 2024-09-16T09:26:00.937 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_recovery_delay_start": "0.000000", 2024-09-16T09:26:00.937 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_recovery_max_active": "0", 2024-09-16T09:26:00.937 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_recovery_max_active_hdd": "3", 2024-09-16T09:26:00.937 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_recovery_max_active_ssd": "10", 2024-09-16T09:26:00.937 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_recovery_max_chunk": "8388608", 2024-09-16T09:26:00.937 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2024-09-16T09:26:00.937 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_recovery_max_single_start": "1", 2024-09-16T09:26:00.937 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_recovery_op_priority": "3", 2024-09-16T09:26:00.938 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_recovery_op_warn_multiple": "16", 2024-09-16T09:26:00.938 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_recovery_priority": "5", 2024-09-16T09:26:00.938 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_recovery_retry_interval": "30.000000", 2024-09-16T09:26:00.938 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_recovery_sleep": "0.000000", 2024-09-16T09:26:00.938 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_recovery_sleep_hdd": "0.100000", 2024-09-16T09:26:00.938 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2024-09-16T09:26:00.938 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_recovery_sleep_ssd": "0.000000", 2024-09-16T09:26:00.938 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_repair_during_recovery": "false", 2024-09-16T09:26:00.938 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_requested_scrub_priority": "120", 2024-09-16T09:26:00.938 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2024-09-16T09:26:00.938 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_rollback_to_cluster_snap": "", 2024-09-16T09:26:00.938 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_scrub_auto_repair": "false", 2024-09-16T09:26:00.938 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_scrub_auto_repair_num_errors": "5", 2024-09-16T09:26:00.938 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_scrub_backoff_ratio": "0.660000", 2024-09-16T09:26:00.938 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_scrub_begin_hour": "0", 2024-09-16T09:26:00.939 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_scrub_begin_week_day": "0", 2024-09-16T09:26:00.939 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_scrub_chunk_max": "25", 2024-09-16T09:26:00.939 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_scrub_chunk_min": "5", 2024-09-16T09:26:00.939 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_scrub_cost": "52428800", 2024-09-16T09:26:00.939 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_scrub_disable_reservation_queuing": "false", 2024-09-16T09:26:00.939 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_scrub_during_recovery": "false", 2024-09-16T09:26:00.939 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_scrub_end_hour": "0", 2024-09-16T09:26:00.939 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_scrub_end_week_day": "0", 2024-09-16T09:26:00.939 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_scrub_event_cost": "4096", 2024-09-16T09:26:00.939 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_scrub_extended_sleep": "0.000000", 2024-09-16T09:26:00.939 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2024-09-16T09:26:00.939 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_scrub_invalid_stats": "true", 2024-09-16T09:26:00.939 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_scrub_load_threshold": "0.500000", 2024-09-16T09:26:00.939 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_scrub_max_interval": "604800.000000", 2024-09-16T09:26:00.940 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_scrub_max_preemptions": "5", 2024-09-16T09:26:00.940 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_scrub_min_interval": "86400.000000", 2024-09-16T09:26:00.940 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_scrub_priority": "5", 2024-09-16T09:26:00.940 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_scrub_retry_after_noscrub": "60", 2024-09-16T09:26:00.940 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_scrub_retry_delay": "30", 2024-09-16T09:26:00.940 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_scrub_retry_new_interval": "10", 2024-09-16T09:26:00.940 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_scrub_retry_pg_state": "60", 2024-09-16T09:26:00.940 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_scrub_retry_trimming": "10", 2024-09-16T09:26:00.940 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_scrub_sleep": "0.000000", 2024-09-16T09:26:00.940 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_shallow_scrub_chunk_max": "100", 2024-09-16T09:26:00.940 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_shallow_scrub_chunk_min": "50", 2024-09-16T09:26:00.940 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_shutdown_pgref_assert": "false", 2024-09-16T09:26:00.941 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_skip_check_past_interval_bounds": "false", 2024-09-16T09:26:00.941 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_skip_data_digest": "false", 2024-09-16T09:26:00.941 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_smart_report_timeout": "5", 2024-09-16T09:26:00.941 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_snap_trim_cost": "1048576", 2024-09-16T09:26:00.941 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_snap_trim_priority": "5", 2024-09-16T09:26:00.941 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_snap_trim_sleep": "0.000000", 2024-09-16T09:26:00.941 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2024-09-16T09:26:00.941 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2024-09-16T09:26:00.941 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2024-09-16T09:26:00.941 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_stats_update_period_not_scrubbing": "120", 2024-09-16T09:26:00.941 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_stats_update_period_scrubbing": "15", 2024-09-16T09:26:00.941 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2024-09-16T09:26:00.941 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_target_transaction_size": "30", 2024-09-16T09:26:00.942 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_tier_default_cache_hit_set_count": "4", 2024-09-16T09:26:00.942 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2024-09-16T09:26:00.942 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2024-09-16T09:26:00.942 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2024-09-16T09:26:00.942 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2024-09-16T09:26:00.942 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2024-09-16T09:26:00.942 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2024-09-16T09:26:00.942 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_tier_default_cache_mode": "writeback", 2024-09-16T09:26:00.942 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2024-09-16T09:26:00.942 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_tier_promote_max_objects_sec": "25", 2024-09-16T09:26:00.942 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_tracing": "false", 2024-09-16T09:26:00.942 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_use_stale_snap": "false", 2024-09-16T09:26:00.942 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2024-09-16T09:26:00.943 INFO:teuthology.orchestra.run.smithi070.stdout: "osdc_blkin_trace_all": "false", 2024-09-16T09:26:00.943 INFO:teuthology.orchestra.run.smithi070.stdout: "paxos_kill_at": "0", 2024-09-16T09:26:00.943 INFO:teuthology.orchestra.run.smithi070.stdout: "paxos_max_join_drift": "10", 2024-09-16T09:26:00.943 INFO:teuthology.orchestra.run.smithi070.stdout: "paxos_min": "500", 2024-09-16T09:26:00.943 INFO:teuthology.orchestra.run.smithi070.stdout: "paxos_min_wait": "0.050000", 2024-09-16T09:26:00.943 INFO:teuthology.orchestra.run.smithi070.stdout: "paxos_propose_interval": "1.000000", 2024-09-16T09:26:00.943 INFO:teuthology.orchestra.run.smithi070.stdout: "paxos_service_trim_max": "500", 2024-09-16T09:26:00.943 INFO:teuthology.orchestra.run.smithi070.stdout: "paxos_service_trim_max_multiplier": "20", 2024-09-16T09:26:00.943 INFO:teuthology.orchestra.run.smithi070.stdout: "paxos_service_trim_min": "10", 2024-09-16T09:26:00.943 INFO:teuthology.orchestra.run.smithi070.stdout: "paxos_stash_full_interval": "25", 2024-09-16T09:26:00.943 INFO:teuthology.orchestra.run.smithi070.stdout: "paxos_trim_max": "500", 2024-09-16T09:26:00.943 INFO:teuthology.orchestra.run.smithi070.stdout: "paxos_trim_min": "250", 2024-09-16T09:26:00.943 INFO:teuthology.orchestra.run.smithi070.stdout: "perf": "true", 2024-09-16T09:26:00.943 INFO:teuthology.orchestra.run.smithi070.stdout: "pid_file": "", 2024-09-16T09:26:00.944 INFO:teuthology.orchestra.run.smithi070.stdout: "plugin_crypto_accelerator": "crypto_isal", 2024-09-16T09:26:00.944 INFO:teuthology.orchestra.run.smithi070.stdout: "plugin_dir": "/usr/lib64/ceph", 2024-09-16T09:26:00.944 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "-", 2024-09-16T09:26:00.944 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrv": "", 2024-09-16T09:26:00.944 INFO:teuthology.orchestra.run.smithi070.stdout: "public_bind_addr": "-", 2024-09-16T09:26:00.944 INFO:teuthology.orchestra.run.smithi070.stdout: "public_network": "", 2024-09-16T09:26:00.944 INFO:teuthology.orchestra.run.smithi070.stdout: "public_network_interface": "", 2024-09-16T09:26:00.944 INFO:teuthology.orchestra.run.smithi070.stdout: "qat_compressor_busy_polling": "false", 2024-09-16T09:26:00.944 INFO:teuthology.orchestra.run.smithi070.stdout: "qat_compressor_enabled": "false", 2024-09-16T09:26:00.944 INFO:teuthology.orchestra.run.smithi070.stdout: "qat_compressor_session_max_number": "256", 2024-09-16T09:26:00.944 INFO:teuthology.orchestra.run.smithi070.stdout: "rados_mon_op_timeout": "0", 2024-09-16T09:26:00.944 INFO:teuthology.orchestra.run.smithi070.stdout: "rados_osd_op_timeout": "0", 2024-09-16T09:26:00.944 INFO:teuthology.orchestra.run.smithi070.stdout: "rados_tracing": "false", 2024-09-16T09:26:00.945 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_atime_update_interval": "60", 2024-09-16T09:26:00.945 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2024-09-16T09:26:00.945 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_balance_parent_reads": "false", 2024-09-16T09:26:00.945 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_balance_snap_reads": "false", 2024-09-16T09:26:00.945 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_blkin_trace_all": "false", 2024-09-16T09:26:00.945 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_blocklist_expire_seconds": "0", 2024-09-16T09:26:00.945 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_blocklist_on_break_lock": "true", 2024-09-16T09:26:00.945 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_cache": "true", 2024-09-16T09:26:00.945 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_cache_block_writes_upfront": "false", 2024-09-16T09:26:00.945 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_cache_max_dirty": "25165824", 2024-09-16T09:26:00.945 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_cache_max_dirty_age": "1.000000", 2024-09-16T09:26:00.945 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_cache_max_dirty_object": "0", 2024-09-16T09:26:00.945 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_cache_policy": "writearound", 2024-09-16T09:26:00.945 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_cache_size": "33554432", 2024-09-16T09:26:00.946 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_cache_target_dirty": "16777216", 2024-09-16T09:26:00.946 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_cache_writethrough_until_flush": "true", 2024-09-16T09:26:00.946 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_clone_copy_on_read": "false", 2024-09-16T09:26:00.946 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_compression_hint": "none", 2024-09-16T09:26:00.946 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_concurrent_management_ops": "10", 2024-09-16T09:26:00.946 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_config_pool_override_update_timestamp": "0", 2024-09-16T09:26:00.946 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_default_clone_format": "auto", 2024-09-16T09:26:00.946 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_default_data_pool": "", 2024-09-16T09:26:00.946 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_default_features": "61", 2024-09-16T09:26:00.946 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_default_format": "2", 2024-09-16T09:26:00.947 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_default_map_options": "", 2024-09-16T09:26:00.947 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_default_order": "22", 2024-09-16T09:26:00.947 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_default_pool": "rbd", 2024-09-16T09:26:00.947 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2024-09-16T09:26:00.947 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_default_stripe_count": "0", 2024-09-16T09:26:00.947 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_default_stripe_unit": "0", 2024-09-16T09:26:00.947 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_disable_zero_copy_writes": "true", 2024-09-16T09:26:00.947 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_discard_granularity_bytes": "65536", 2024-09-16T09:26:00.947 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_discard_on_zeroed_write_same": "true", 2024-09-16T09:26:00.947 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_enable_alloc_hint": "true", 2024-09-16T09:26:00.947 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2024-09-16T09:26:00.947 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_io_scheduler": "simple", 2024-09-16T09:26:00.947 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2024-09-16T09:26:00.947 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_journal_commit_age": "5.000000", 2024-09-16T09:26:00.948 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2024-09-16T09:26:00.948 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_journal_max_payload_bytes": "16384", 2024-09-16T09:26:00.948 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_journal_object_flush_age": "0.000000", 2024-09-16T09:26:00.948 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_journal_object_flush_bytes": "1048576", 2024-09-16T09:26:00.948 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_journal_object_flush_interval": "0", 2024-09-16T09:26:00.948 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2024-09-16T09:26:00.948 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2024-09-16T09:26:00.948 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_journal_order": "24", 2024-09-16T09:26:00.948 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_journal_pool": "", 2024-09-16T09:26:00.948 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_journal_splay_width": "4", 2024-09-16T09:26:00.948 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_localize_parent_reads": "false", 2024-09-16T09:26:00.948 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_localize_snap_reads": "false", 2024-09-16T09:26:00.948 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2024-09-16T09:26:00.949 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2024-09-16T09:26:00.949 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2024-09-16T09:26:00.949 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2024-09-16T09:26:00.949 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2024-09-16T09:26:00.949 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2024-09-16T09:26:00.949 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_mirror_image_policy_type": "simple", 2024-09-16T09:26:00.949 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2024-09-16T09:26:00.949 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_mirror_image_state_check_interval": "30", 2024-09-16T09:26:00.949 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2024-09-16T09:26:00.949 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2024-09-16T09:26:00.949 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2024-09-16T09:26:00.949 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2024-09-16T09:26:00.949 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2024-09-16T09:26:00.949 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_mirror_memory_autotune": "true", 2024-09-16T09:26:00.950 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_mirror_memory_base": "805306368", 2024-09-16T09:26:00.950 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2024-09-16T09:26:00.950 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_mirror_memory_cache_min": "134217728", 2024-09-16T09:26:00.950 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2024-09-16T09:26:00.950 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2024-09-16T09:26:00.950 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_mirror_memory_target": "4294967296", 2024-09-16T09:26:00.950 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_mirror_perf_stats_prio": "5", 2024-09-16T09:26:00.950 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2024-09-16T09:26:00.950 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2024-09-16T09:26:00.950 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_mirroring_delete_delay": "0", 2024-09-16T09:26:00.950 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2024-09-16T09:26:00.950 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_mirroring_replay_delay": "0", 2024-09-16T09:26:00.950 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2024-09-16T09:26:00.950 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2024-09-16T09:26:00.951 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_move_to_trash_on_remove": "false", 2024-09-16T09:26:00.951 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2024-09-16T09:26:00.951 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_mtime_update_interval": "60", 2024-09-16T09:26:00.951 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_non_blocking_aio": "true", 2024-09-16T09:26:00.951 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_op_thread_timeout": "60", 2024-09-16T09:26:00.951 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_op_threads": "1", 2024-09-16T09:26:00.951 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_parent_cache_enabled": "false", 2024-09-16T09:26:00.951 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_persistent_cache_mode": "disabled", 2024-09-16T09:26:00.951 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_persistent_cache_path": "/tmp", 2024-09-16T09:26:00.951 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_persistent_cache_size": "1073741824", 2024-09-16T09:26:00.951 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_plugins": "", 2024-09-16T09:26:00.951 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_qos_bps_burst": "0", 2024-09-16T09:26:00.951 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_qos_bps_burst_seconds": "1", 2024-09-16T09:26:00.951 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_qos_bps_limit": "0", 2024-09-16T09:26:00.952 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_qos_exclude_ops": "0", 2024-09-16T09:26:00.952 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_qos_iops_burst": "0", 2024-09-16T09:26:00.952 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_qos_iops_burst_seconds": "1", 2024-09-16T09:26:00.952 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_qos_iops_limit": "0", 2024-09-16T09:26:00.952 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_qos_read_bps_burst": "0", 2024-09-16T09:26:00.952 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2024-09-16T09:26:00.952 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_qos_read_bps_limit": "0", 2024-09-16T09:26:00.952 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_qos_read_iops_burst": "0", 2024-09-16T09:26:00.952 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2024-09-16T09:26:00.952 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_qos_read_iops_limit": "0", 2024-09-16T09:26:00.952 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_qos_schedule_tick_min": "50", 2024-09-16T09:26:00.952 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_qos_write_bps_burst": "0", 2024-09-16T09:26:00.952 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2024-09-16T09:26:00.953 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_qos_write_bps_limit": "0", 2024-09-16T09:26:00.953 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_qos_write_iops_burst": "0", 2024-09-16T09:26:00.953 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2024-09-16T09:26:00.953 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_qos_write_iops_limit": "0", 2024-09-16T09:26:00.953 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_quiesce_notification_attempts": "10", 2024-09-16T09:26:00.953 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_read_from_replica_policy": "default", 2024-09-16T09:26:00.953 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2024-09-16T09:26:00.953 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_readahead_max_bytes": "524288", 2024-09-16T09:26:00.953 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_readahead_trigger_requests": "10", 2024-09-16T09:26:00.953 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_request_timed_out_seconds": "30", 2024-09-16T09:26:00.953 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_skip_partial_discard": "true", 2024-09-16T09:26:00.953 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2024-09-16T09:26:00.953 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_tracing": "false", 2024-09-16T09:26:00.954 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_validate_names": "true", 2024-09-16T09:26:00.954 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd_validate_pool": "true", 2024-09-16T09:26:00.954 INFO:teuthology.orchestra.run.smithi070.stdout: "restapi_base_url": "", 2024-09-16T09:26:00.954 INFO:teuthology.orchestra.run.smithi070.stdout: "restapi_log_level": "", 2024-09-16T09:26:00.954 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_account_default_quota_max_objects": "-1", 2024-09-16T09:26:00.954 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_account_default_quota_max_size": "-1", 2024-09-16T09:26:00.954 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_acl_grants_max_num": "100", 2024-09-16T09:26:00.954 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_admin_entry": "admin", 2024-09-16T09:26:00.954 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_allow_notification_secrets_in_cleartext": "false", 2024-09-16T09:26:00.954 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_asio_assert_yielding": "false", 2024-09-16T09:26:00.954 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_backend_store": "rados", 2024-09-16T09:26:00.954 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_barbican_url": "", 2024-09-16T09:26:00.954 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_beast_enable_async": "true", 2024-09-16T09:26:00.954 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_bucket_counters_cache": "false", 2024-09-16T09:26:00.955 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_bucket_counters_cache_size": "10000", 2024-09-16T09:26:00.955 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2024-09-16T09:26:00.955 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_bucket_default_quota_max_size": "-1", 2024-09-16T09:26:00.955 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_bucket_index_max_aio": "128", 2024-09-16T09:26:00.955 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_bucket_index_transaction_instrumentation": "false", 2024-09-16T09:26:00.955 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_bucket_quota_cache_size": "10000", 2024-09-16T09:26:00.955 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_bucket_quota_ttl": "600", 2024-09-16T09:26:00.955 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_bucket_sync_spawn_window": "20", 2024-09-16T09:26:00.955 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_cache_enabled": "true", 2024-09-16T09:26:00.955 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_cache_expiry_interval": "900", 2024-09-16T09:26:00.955 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_cache_lru_size": "25000", 2024-09-16T09:26:00.955 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_config_store": "rados", 2024-09-16T09:26:00.955 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_content_length_compat": "false", 2024-09-16T09:26:00.955 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_copy_obj_progress": "true", 2024-09-16T09:26:00.956 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2024-09-16T09:26:00.956 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_cors_rules_max_num": "100", 2024-09-16T09:26:00.956 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_cross_domain_policy": "", 2024-09-16T09:26:00.956 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_crypt_default_encryption_key": "", 2024-09-16T09:26:00.956 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_crypt_kmip_addr": "", 2024-09-16T09:26:00.956 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_crypt_kmip_ca_path": "", 2024-09-16T09:26:00.956 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_crypt_kmip_client_cert": "", 2024-09-16T09:26:00.956 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_crypt_kmip_client_key": "", 2024-09-16T09:26:00.956 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_crypt_kmip_kms_key_template": "", 2024-09-16T09:26:00.956 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_crypt_kmip_password": "", 2024-09-16T09:26:00.956 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2024-09-16T09:26:00.956 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_crypt_kmip_username": "", 2024-09-16T09:26:00.956 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_crypt_require_ssl": "true", 2024-09-16T09:26:00.957 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2024-09-16T09:26:00.957 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2024-09-16T09:26:00.957 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_crypt_sse_s3_backend": "vault", 2024-09-16T09:26:00.957 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2024-09-16T09:26:00.957 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2024-09-16T09:26:00.957 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2024-09-16T09:26:00.957 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2024-09-16T09:26:00.957 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2024-09-16T09:26:00.957 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2024-09-16T09:26:00.957 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2024-09-16T09:26:00.957 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2024-09-16T09:26:00.957 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2024-09-16T09:26:00.957 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2024-09-16T09:26:00.958 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2024-09-16T09:26:00.958 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_crypt_suppress_logs": "true", 2024-09-16T09:26:00.958 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_crypt_vault_addr": "", 2024-09-16T09:26:00.958 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_crypt_vault_auth": "token", 2024-09-16T09:26:00.958 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_crypt_vault_namespace": "", 2024-09-16T09:26:00.958 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_crypt_vault_prefix": "", 2024-09-16T09:26:00.958 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_crypt_vault_secret_engine": "transit", 2024-09-16T09:26:00.958 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_crypt_vault_ssl_cacert": "", 2024-09-16T09:26:00.958 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2024-09-16T09:26:00.958 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2024-09-16T09:26:00.958 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_crypt_vault_token_file": "", 2024-09-16T09:26:00.958 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_crypt_vault_verify_ssl": "true", 2024-09-16T09:26:00.958 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_curl_buffersize": "524288", 2024-09-16T09:26:00.958 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_curl_low_speed_limit": "1024", 2024-09-16T09:26:00.959 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_curl_low_speed_time": "300", 2024-09-16T09:26:00.959 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_curl_tcp_keepalive": "0", 2024-09-16T09:26:00.959 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_curl_wait_timeout_ms": "1000", 2024-09-16T09:26:00.959 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2024-09-16T09:26:00.959 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2024-09-16T09:26:00.959 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2024-09-16T09:26:00.959 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2024-09-16T09:26:00.959 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_d3n_l1_fadvise": "4", 2024-09-16T09:26:00.959 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2024-09-16T09:26:00.959 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_d3n_libaio_aio_num": "64", 2024-09-16T09:26:00.959 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_d3n_libaio_aio_threads": "20", 2024-09-16T09:26:00.959 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_d4n_address": "127.0.0.1:6379", 2024-09-16T09:26:00.959 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_d4n_l1_datacache_address": "127.0.0.1:6379", 2024-09-16T09:26:00.959 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_d4n_l1_datacache_persistent_path": "/tmp/rgw_d4n_datacache/", 2024-09-16T09:26:00.959 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_d4n_l1_datacache_size": "1073741824", 2024-09-16T09:26:00.960 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_d4n_l1_evict_cache_on_start": "true", 2024-09-16T09:26:00.960 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_d4n_l1_fadvise": "4", 2024-09-16T09:26:00.960 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_d4n_libaio_aio_num": "64", 2024-09-16T09:26:00.960 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_d4n_libaio_aio_threads": "20", 2024-09-16T09:26:00.960 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2024-09-16T09:26:00.960 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_data_log_changes_size": "1000", 2024-09-16T09:26:00.960 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_data_log_num_shards": "128", 2024-09-16T09:26:00.960 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_data_log_obj_prefix": "data_log", 2024-09-16T09:26:00.960 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_data_log_window": "30", 2024-09-16T09:26:00.960 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_data_notify_interval_msec": "0", 2024-09-16T09:26:00.960 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_data_sync_poll_interval": "20", 2024-09-16T09:26:00.960 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_data_sync_spawn_window": "20", 2024-09-16T09:26:00.960 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_debug_inject_latency_bi_unlink": "0", 2024-09-16T09:26:00.960 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2024-09-16T09:26:00.961 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_debug_inject_set_olh_err": "0", 2024-09-16T09:26:00.961 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_default_data_log_backing": "fifo", 2024-09-16T09:26:00.961 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_default_realm_info_oid": "default.realm", 2024-09-16T09:26:00.961 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_default_region_info_oid": "default.region", 2024-09-16T09:26:00.961 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_default_zone_info_oid": "default.zone", 2024-09-16T09:26:00.961 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2024-09-16T09:26:00.961 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_defer_to_bucket_acls": "", 2024-09-16T09:26:00.961 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_delete_multi_obj_max_num": "1000", 2024-09-16T09:26:00.961 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_disable_s3select": "false", 2024-09-16T09:26:00.961 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_dmclock_admin_lim": "0.000000", 2024-09-16T09:26:00.961 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_dmclock_admin_res": "100.000000", 2024-09-16T09:26:00.961 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2024-09-16T09:26:00.961 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_dmclock_auth_lim": "0.000000", 2024-09-16T09:26:00.961 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_dmclock_auth_res": "200.000000", 2024-09-16T09:26:00.962 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2024-09-16T09:26:00.962 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_dmclock_data_lim": "0.000000", 2024-09-16T09:26:00.962 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_dmclock_data_res": "500.000000", 2024-09-16T09:26:00.962 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_dmclock_data_wgt": "500.000000", 2024-09-16T09:26:00.962 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2024-09-16T09:26:00.962 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_dmclock_metadata_res": "500.000000", 2024-09-16T09:26:00.962 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2024-09-16T09:26:00.962 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_dns_name": "", 2024-09-16T09:26:00.962 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_dns_s3website_name": "", 2024-09-16T09:26:00.962 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_dynamic_resharding": "true", 2024-09-16T09:26:00.962 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_dynamic_resharding_may_reduce": "true", 2024-09-16T09:26:00.963 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_dynamic_resharding_reduction_wait": "120", 2024-09-16T09:26:00.963 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2024-09-16T09:26:00.963 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_enable_gc_threads": "true", 2024-09-16T09:26:00.963 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_enable_lc_threads": "true", 2024-09-16T09:26:00.963 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_enable_mdsearch": "true", 2024-09-16T09:26:00.963 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_enable_ops_log": "false", 2024-09-16T09:26:00.963 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_enable_quota_threads": "true", 2024-09-16T09:26:00.963 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_enable_static_website": "false", 2024-09-16T09:26:00.963 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_enable_usage_log": "false", 2024-09-16T09:26:00.963 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_enforce_swift_acls": "true", 2024-09-16T09:26:00.963 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_exit_timeout_secs": "120", 2024-09-16T09:26:00.963 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_expose_bucket": "false", 2024-09-16T09:26:00.963 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_extended_http_attrs": "", 2024-09-16T09:26:00.964 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_filter": "none", 2024-09-16T09:26:00.964 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2024-09-16T09:26:00.964 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_frontends": "beast port=7480", 2024-09-16T09:26:00.964 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_gc_max_concurrent_io": "10", 2024-09-16T09:26:00.964 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_gc_max_deferred": "50", 2024-09-16T09:26:00.964 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2024-09-16T09:26:00.964 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_gc_max_objs": "32", 2024-09-16T09:26:00.964 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_gc_max_queue_size": "134213632", 2024-09-16T09:26:00.964 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_gc_max_trim_chunk": "16", 2024-09-16T09:26:00.964 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_gc_obj_min_wait": "7200", 2024-09-16T09:26:00.964 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_gc_processor_max_time": "3600", 2024-09-16T09:26:00.964 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_gc_processor_period": "3600", 2024-09-16T09:26:00.965 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_get_obj_max_req_size": "4194304", 2024-09-16T09:26:00.965 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_get_obj_window_size": "16777216", 2024-09-16T09:26:00.965 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_healthcheck_disabling_path": "", 2024-09-16T09:26:00.965 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_ignore_get_invalid_range": "false", 2024-09-16T09:26:00.965 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_init_timeout": "300", 2024-09-16T09:26:00.965 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2024-09-16T09:26:00.965 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_json_config": "/var/lib/ceph/radosgw/config.json", 2024-09-16T09:26:00.965 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_kafka_connection_idle": "300", 2024-09-16T09:26:00.965 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_kafka_message_timeout": "5000", 2024-09-16T09:26:00.965 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_kafka_sleep_timeout": "10", 2024-09-16T09:26:00.965 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_keystone_accepted_admin_roles": "", 2024-09-16T09:26:00.965 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_keystone_accepted_reader_roles": "", 2024-09-16T09:26:00.965 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2024-09-16T09:26:00.966 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_keystone_admin_domain": "", 2024-09-16T09:26:00.966 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_keystone_admin_password": "", 2024-09-16T09:26:00.966 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_keystone_admin_password_path": "", 2024-09-16T09:26:00.966 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_keystone_admin_project": "", 2024-09-16T09:26:00.966 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_keystone_admin_tenant": "", 2024-09-16T09:26:00.966 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_keystone_admin_token": "", 2024-09-16T09:26:00.966 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_keystone_admin_token_path": "", 2024-09-16T09:26:00.966 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_keystone_admin_user": "", 2024-09-16T09:26:00.966 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_keystone_barbican_domain": "", 2024-09-16T09:26:00.966 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_keystone_barbican_password": "", 2024-09-16T09:26:00.966 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_keystone_barbican_project": "", 2024-09-16T09:26:00.966 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_keystone_barbican_tenant": "", 2024-09-16T09:26:00.966 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_keystone_barbican_user": "", 2024-09-16T09:26:00.966 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2024-09-16T09:26:00.967 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_keystone_implicit_tenants": "false", 2024-09-16T09:26:00.967 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2024-09-16T09:26:00.967 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_keystone_service_token_enabled": "false", 2024-09-16T09:26:00.967 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_keystone_token_cache_size": "10000", 2024-09-16T09:26:00.967 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_keystone_url": "", 2024-09-16T09:26:00.967 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_keystone_verify_ssl": "true", 2024-09-16T09:26:00.967 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_lc_debug_interval": "-1", 2024-09-16T09:26:00.967 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_lc_lock_max_time": "90", 2024-09-16T09:26:00.967 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_lc_max_objs": "32", 2024-09-16T09:26:00.967 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_lc_max_rules": "1000", 2024-09-16T09:26:00.967 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_lc_max_worker": "3", 2024-09-16T09:26:00.967 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_lc_max_wp_worker": "3", 2024-09-16T09:26:00.968 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_lc_thread_delay": "0", 2024-09-16T09:26:00.968 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2024-09-16T09:26:00.968 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_ldap_dnattr": "uid", 2024-09-16T09:26:00.968 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2024-09-16T09:26:00.968 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_ldap_searchfilter": "", 2024-09-16T09:26:00.968 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2024-09-16T09:26:00.968 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_ldap_uri": "", 2024-09-16T09:26:00.969 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_lfuda_sync_frequency": "60", 2024-09-16T09:26:00.969 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2024-09-16T09:26:00.969 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_list_bucket_min_readahead": "1000", 2024-09-16T09:26:00.969 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_list_buckets_max_chunk": "1000", 2024-09-16T09:26:00.969 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_log_http_headers": "", 2024-09-16T09:26:00.969 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_log_nonexistent_bucket": "false", 2024-09-16T09:26:00.969 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2024-09-16T09:26:00.969 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_log_object_name_utc": "false", 2024-09-16T09:26:00.969 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_lua_max_memory_per_state": "128000", 2024-09-16T09:26:00.969 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_luarocks_location": "/tmp/rgw_luarocks/mon.a", 2024-09-16T09:26:00.969 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_max_attr_name_len": "0", 2024-09-16T09:26:00.969 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_max_attr_size": "0", 2024-09-16T09:26:00.969 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_max_attrs_num_in_req": "0", 2024-09-16T09:26:00.970 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_max_chunk_size": "4194304", 2024-09-16T09:26:00.970 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_max_concurrent_requests": "1024", 2024-09-16T09:26:00.970 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_max_copy_obj_concurrent_io": "10", 2024-09-16T09:26:00.970 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_max_dynamic_shards": "1999", 2024-09-16T09:26:00.970 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_max_listing_results": "5000", 2024-09-16T09:26:00.970 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_max_notify_retries": "10", 2024-09-16T09:26:00.970 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_max_objs_per_shard": "100000", 2024-09-16T09:26:00.970 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_max_put_param_size": "1048576", 2024-09-16T09:26:00.970 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_max_put_size": "5368709120", 2024-09-16T09:26:00.970 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_max_slo_entries": "1000", 2024-09-16T09:26:00.970 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_md_log_max_shards": "64", 2024-09-16T09:26:00.970 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_md_notify_interval_msec": "200", 2024-09-16T09:26:00.970 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_meta_sync_poll_interval": "20", 2024-09-16T09:26:00.971 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_meta_sync_spawn_window": "20", 2024-09-16T09:26:00.971 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_mime_types_file": "/etc/mime.types", 2024-09-16T09:26:00.971 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_mp_lock_max_time": "600", 2024-09-16T09:26:00.971 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_multi_obj_del_max_aio": "16", 2024-09-16T09:26:00.971 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_multipart_min_part_size": "5242880", 2024-09-16T09:26:00.971 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_multipart_part_upload_limit": "10000", 2024-09-16T09:26:00.971 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_nfs_fhcache_partitions": "3", 2024-09-16T09:26:00.971 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_nfs_fhcache_size": "2017", 2024-09-16T09:26:00.971 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_nfs_frontends": "rgw-nfs", 2024-09-16T09:26:00.971 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2024-09-16T09:26:00.971 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_nfs_lru_lanes": "5", 2024-09-16T09:26:00.971 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_nfs_max_gc": "300", 2024-09-16T09:26:00.971 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_nfs_namespace_expire_secs": "300", 2024-09-16T09:26:00.971 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_nfs_run_gc_threads": "false", 2024-09-16T09:26:00.972 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_nfs_run_lc_threads": "false", 2024-09-16T09:26:00.972 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_nfs_run_quota_threads": "false", 2024-09-16T09:26:00.972 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_nfs_run_sync_thread": "false", 2024-09-16T09:26:00.972 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_nfs_s3_fast_attrs": "false", 2024-09-16T09:26:00.972 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_nfs_write_completion_interval_s": "10", 2024-09-16T09:26:00.972 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_num_async_rados_threads": "32", 2024-09-16T09:26:00.972 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_num_control_oids": "8", 2024-09-16T09:26:00.972 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_numa_node": "-1", 2024-09-16T09:26:00.972 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_obj_stripe_size": "4194304", 2024-09-16T09:26:00.972 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_obj_tombstone_cache_size": "1000", 2024-09-16T09:26:00.972 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_objexp_chunk_size": "100", 2024-09-16T09:26:00.972 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_objexp_gc_interval": "600", 2024-09-16T09:26:00.972 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_objexp_hints_num_shards": "127", 2024-09-16T09:26:00.972 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_olh_pending_timeout_sec": "3600", 2024-09-16T09:26:00.973 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_op_thread_suicide_timeout": "0", 2024-09-16T09:26:00.973 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_op_thread_timeout": "600", 2024-09-16T09:26:00.973 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_op_tracing": "false", 2024-09-16T09:26:00.973 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_opa_token": "", 2024-09-16T09:26:00.973 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_opa_url": "", 2024-09-16T09:26:00.973 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_opa_verify_ssl": "true", 2024-09-16T09:26:00.973 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_ops_log_data_backlog": "5242880", 2024-09-16T09:26:00.973 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_ops_log_file_path": "/var/log/ceph/ops-log-ceph-mon.a.log", 2024-09-16T09:26:00.973 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_ops_log_rados": "false", 2024-09-16T09:26:00.973 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_ops_log_socket_path": "", 2024-09-16T09:26:00.973 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_override_bucket_index_max_shards": "0", 2024-09-16T09:26:00.974 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_pending_bucket_index_op_expiration": "120", 2024-09-16T09:26:00.974 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2024-09-16T09:26:00.974 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_period_push_interval": "2.000000", 2024-09-16T09:26:00.974 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_period_push_interval_max": "30.000000", 2024-09-16T09:26:00.974 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_period_root_pool": ".rgw.root", 2024-09-16T09:26:00.974 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_policy_reject_invalid_principals": "true", 2024-09-16T09:26:00.974 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_posix_base_path": "/tmp/rgw_posix_driver", 2024-09-16T09:26:00.974 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_posix_cache_lanes": "3", 2024-09-16T09:26:00.974 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_posix_cache_lmdb_count": "3", 2024-09-16T09:26:00.974 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_posix_cache_max_buckets": "100", 2024-09-16T09:26:00.975 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_posix_cache_partitions": "3", 2024-09-16T09:26:00.975 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_posix_database_root": "/var/lib/ceph/radosgw", 2024-09-16T09:26:00.975 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_print_continue": "true", 2024-09-16T09:26:00.975 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_print_prohibited_content_length": "false", 2024-09-16T09:26:00.975 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_put_obj_max_window_size": "67108864", 2024-09-16T09:26:00.975 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_put_obj_min_window_size": "16777216", 2024-09-16T09:26:00.975 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2024-09-16T09:26:00.975 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_rados_pool_recovery_priority": "5", 2024-09-16T09:26:00.975 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_rados_tracing": "false", 2024-09-16T09:26:00.975 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_realm": "", 2024-09-16T09:26:00.975 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_realm_id": "", 2024-09-16T09:26:00.975 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_realm_root_pool": ".rgw.root", 2024-09-16T09:26:00.975 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_region": "", 2024-09-16T09:26:00.975 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_region_root_pool": ".rgw.root", 2024-09-16T09:26:00.975 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_relaxed_region_enforcement": "false", 2024-09-16T09:26:00.976 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_relaxed_s3_bucket_names": "false", 2024-09-16T09:26:00.976 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2024-09-16T09:26:00.976 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_request_uri": "", 2024-09-16T09:26:00.976 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_reshard_batch_size": "64", 2024-09-16T09:26:00.976 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_reshard_bucket_lock_duration": "360", 2024-09-16T09:26:00.976 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_reshard_debug_interval": "-1", 2024-09-16T09:26:00.976 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_reshard_max_aio": "128", 2024-09-16T09:26:00.976 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_reshard_num_logs": "16", 2024-09-16T09:26:00.976 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_reshard_progress_judge_interval": "120", 2024-09-16T09:26:00.976 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_reshard_progress_judge_ratio": "0.500000", 2024-09-16T09:26:00.976 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_reshard_thread_interval": "600", 2024-09-16T09:26:00.976 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_reshardlog_threshold": "30000", 2024-09-16T09:26:00.976 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_resolve_cname": "false", 2024-09-16T09:26:00.976 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_rest_getusage_op_compat": "false", 2024-09-16T09:26:00.977 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_run_sync_thread": "true", 2024-09-16T09:26:00.977 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_s3_auth_disable_signature_url": "false", 2024-09-16T09:26:00.977 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_s3_auth_order": "sts, external, local", 2024-09-16T09:26:00.977 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_s3_auth_use_keystone": "false", 2024-09-16T09:26:00.977 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_s3_auth_use_ldap": "false", 2024-09-16T09:26:00.977 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_s3_auth_use_rados": "true", 2024-09-16T09:26:00.977 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_s3_auth_use_sts": "false", 2024-09-16T09:26:00.977 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_s3_client_max_sig_ver": "-1", 2024-09-16T09:26:00.977 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_s3_success_create_obj_status": "0", 2024-09-16T09:26:00.977 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_safe_max_objects_per_shard": "102400", 2024-09-16T09:26:00.977 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_scheduler_type": "throttler", 2024-09-16T09:26:00.977 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_script_uri": "", 2024-09-16T09:26:00.977 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_service_provider_name": "", 2024-09-16T09:26:00.978 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_shard_warning_threshold": "90.000000", 2024-09-16T09:26:00.978 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_sts_client_id": "", 2024-09-16T09:26:00.978 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_sts_client_secret": "", 2024-09-16T09:26:00.978 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_sts_entry": "sts", 2024-09-16T09:26:00.978 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_sts_key": "sts", 2024-09-16T09:26:00.978 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_sts_max_session_duration": "43200", 2024-09-16T09:26:00.978 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_sts_min_session_duration": "900", 2024-09-16T09:26:00.978 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_sts_token_introspection_url": "", 2024-09-16T09:26:00.978 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_swift_account_in_url": "false", 2024-09-16T09:26:00.978 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_swift_auth_entry": "auth", 2024-09-16T09:26:00.978 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_swift_auth_url": "", 2024-09-16T09:26:00.978 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_swift_custom_header": "", 2024-09-16T09:26:00.978 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_swift_enforce_content_length": "false", 2024-09-16T09:26:00.978 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_swift_need_stats": "true", 2024-09-16T09:26:00.979 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_swift_tenant_name": "", 2024-09-16T09:26:00.979 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_swift_token_expiration": "86400", 2024-09-16T09:26:00.979 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_swift_url": "", 2024-09-16T09:26:00.979 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_swift_url_prefix": "swift", 2024-09-16T09:26:00.979 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_swift_versioning_enabled": "false", 2024-09-16T09:26:00.979 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_sync_data_full_inject_err_probability": "0.000000", 2024-09-16T09:26:00.979 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2024-09-16T09:26:00.979 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_sync_lease_period": "120", 2024-09-16T09:26:00.979 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2024-09-16T09:26:00.979 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_sync_log_trim_interval": "1200", 2024-09-16T09:26:00.979 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_sync_log_trim_max_buckets": "16", 2024-09-16T09:26:00.980 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2024-09-16T09:26:00.980 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2024-09-16T09:26:00.980 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_sync_obj_etag_verify": "false", 2024-09-16T09:26:00.980 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_sync_trace_history_size": "4096", 2024-09-16T09:26:00.980 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_sync_trace_per_node_log_size": "32", 2024-09-16T09:26:00.980 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2024-09-16T09:26:00.980 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_thread_pool_size": "512", 2024-09-16T09:26:00.980 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_topic_persistency_max_retries": "0", 2024-09-16T09:26:00.980 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_topic_persistency_sleep_duration": "0", 2024-09-16T09:26:00.980 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_topic_persistency_time_to_live": "0", 2024-09-16T09:26:00.981 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_topic_require_publish_policy": "false", 2024-09-16T09:26:00.981 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_torrent_comment": "", 2024-09-16T09:26:00.981 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_torrent_createby": "", 2024-09-16T09:26:00.981 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_torrent_encoding": "", 2024-09-16T09:26:00.981 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_torrent_flag": "false", 2024-09-16T09:26:00.981 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_torrent_max_size": "5368709120", 2024-09-16T09:26:00.981 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_torrent_origin": "", 2024-09-16T09:26:00.981 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_torrent_sha_unit": "524288", 2024-09-16T09:26:00.981 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_torrent_tracker": "", 2024-09-16T09:26:00.981 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_trust_forwarded_https": "false", 2024-09-16T09:26:00.981 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_usage_log_flush_threshold": "1024", 2024-09-16T09:26:00.982 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_usage_log_tick_interval": "30", 2024-09-16T09:26:00.982 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_usage_max_shards": "32", 2024-09-16T09:26:00.982 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_usage_max_user_shards": "1", 2024-09-16T09:26:00.982 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_use_opa_authz": "false", 2024-09-16T09:26:00.982 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_user_counters_cache": "false", 2024-09-16T09:26:00.982 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_user_counters_cache_size": "10000", 2024-09-16T09:26:00.982 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_user_default_quota_max_objects": "-1", 2024-09-16T09:26:00.982 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_user_default_quota_max_size": "-1", 2024-09-16T09:26:00.982 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_user_max_buckets": "1000", 2024-09-16T09:26:00.982 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_user_policies_max_num": "100", 2024-09-16T09:26:00.982 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2024-09-16T09:26:00.982 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_user_quota_sync_idle_users": "false", 2024-09-16T09:26:00.982 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_user_quota_sync_interval": "86400", 2024-09-16T09:26:00.983 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_user_quota_sync_wait_time": "86400", 2024-09-16T09:26:00.983 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_user_unique_email": "true", 2024-09-16T09:26:00.983 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_verify_ssl": "true", 2024-09-16T09:26:00.983 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_website_routing_rules_max_num": "50", 2024-09-16T09:26:00.983 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_zone": "", 2024-09-16T09:26:00.983 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_zone_id": "", 2024-09-16T09:26:00.983 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_zone_root_pool": ".rgw.root", 2024-09-16T09:26:00.983 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_zonegroup": "", 2024-09-16T09:26:00.983 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_zonegroup_id": "", 2024-09-16T09:26:00.983 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2024-09-16T09:26:00.983 INFO:teuthology.orchestra.run.smithi070.stdout: "rgwlc_auto_session_clear": "true", 2024-09-16T09:26:00.983 INFO:teuthology.orchestra.run.smithi070.stdout: "rgwlc_skip_bucket_step": "false", 2024-09-16T09:26:00.983 INFO:teuthology.orchestra.run.smithi070.stdout: "rocksdb_block_size": "4096", 2024-09-16T09:26:00.984 INFO:teuthology.orchestra.run.smithi070.stdout: "rocksdb_bloom_bits_per_key": "20", 2024-09-16T09:26:00.984 INFO:teuthology.orchestra.run.smithi070.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2024-09-16T09:26:00.984 INFO:teuthology.orchestra.run.smithi070.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2024-09-16T09:26:00.984 INFO:teuthology.orchestra.run.smithi070.stdout: "rocksdb_cache_row_ratio": "0.000000", 2024-09-16T09:26:00.984 INFO:teuthology.orchestra.run.smithi070.stdout: "rocksdb_cache_shard_bits": "4", 2024-09-16T09:26:00.984 INFO:teuthology.orchestra.run.smithi070.stdout: "rocksdb_cache_size": "536870912", 2024-09-16T09:26:00.984 INFO:teuthology.orchestra.run.smithi070.stdout: "rocksdb_cache_type": "binned_lru", 2024-09-16T09:26:00.984 INFO:teuthology.orchestra.run.smithi070.stdout: "rocksdb_cf_compact_on_deletion": "true", 2024-09-16T09:26:00.984 INFO:teuthology.orchestra.run.smithi070.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2024-09-16T09:26:00.984 INFO:teuthology.orchestra.run.smithi070.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2024-09-16T09:26:00.984 INFO:teuthology.orchestra.run.smithi070.stdout: "rocksdb_collect_compaction_stats": "false", 2024-09-16T09:26:00.984 INFO:teuthology.orchestra.run.smithi070.stdout: "rocksdb_collect_extended_stats": "false", 2024-09-16T09:26:00.984 INFO:teuthology.orchestra.run.smithi070.stdout: "rocksdb_collect_memory_stats": "false", 2024-09-16T09:26:00.985 INFO:teuthology.orchestra.run.smithi070.stdout: "rocksdb_delete_range_threshold": "1048576", 2024-09-16T09:26:00.985 INFO:teuthology.orchestra.run.smithi070.stdout: "rocksdb_index_type": "binary_search", 2024-09-16T09:26:00.985 INFO:teuthology.orchestra.run.smithi070.stdout: "rocksdb_log_to_ceph_log": "true", 2024-09-16T09:26:00.985 INFO:teuthology.orchestra.run.smithi070.stdout: "rocksdb_metadata_block_size": "4096", 2024-09-16T09:26:00.985 INFO:teuthology.orchestra.run.smithi070.stdout: "rocksdb_partition_filters": "false", 2024-09-16T09:26:00.985 INFO:teuthology.orchestra.run.smithi070.stdout: "rocksdb_perf": "false", 2024-09-16T09:26:00.985 INFO:teuthology.orchestra.run.smithi070.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2024-09-16T09:26:00.985 INFO:teuthology.orchestra.run.smithi070.stdout: "rotating_keys_bootstrap_timeout": "30", 2024-09-16T09:26:00.985 INFO:teuthology.orchestra.run.smithi070.stdout: "rotating_keys_renewal_timeout": "10", 2024-09-16T09:26:00.985 INFO:teuthology.orchestra.run.smithi070.stdout: "run_dir": "/var/run/ceph", 2024-09-16T09:26:00.985 INFO:teuthology.orchestra.run.smithi070.stdout: "seastore_block_create": "true", 2024-09-16T09:26:00.985 INFO:teuthology.orchestra.run.smithi070.stdout: "seastore_cache_lru_size": "67108864", 2024-09-16T09:26:00.985 INFO:teuthology.orchestra.run.smithi070.stdout: "seastore_cbjournal_size": "5368709120", 2024-09-16T09:26:00.985 INFO:teuthology.orchestra.run.smithi070.stdout: "seastore_data_delta_based_overwrite": "0", 2024-09-16T09:26:00.986 INFO:teuthology.orchestra.run.smithi070.stdout: "seastore_default_max_object_size": "16777216", 2024-09-16T09:26:00.986 INFO:teuthology.orchestra.run.smithi070.stdout: "seastore_default_object_metadata_reservation": "16777216", 2024-09-16T09:26:00.986 INFO:teuthology.orchestra.run.smithi070.stdout: "seastore_device_size": "53687091200", 2024-09-16T09:26:00.986 INFO:teuthology.orchestra.run.smithi070.stdout: "seastore_disable_end_to_end_data_protection": "true", 2024-09-16T09:26:00.986 INFO:teuthology.orchestra.run.smithi070.stdout: "seastore_full_integrity_check": "false", 2024-09-16T09:26:00.986 INFO:teuthology.orchestra.run.smithi070.stdout: "seastore_journal_batch_capacity": "16", 2024-09-16T09:26:00.986 INFO:teuthology.orchestra.run.smithi070.stdout: "seastore_journal_batch_flush_size": "16777216", 2024-09-16T09:26:00.986 INFO:teuthology.orchestra.run.smithi070.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2024-09-16T09:26:00.986 INFO:teuthology.orchestra.run.smithi070.stdout: "seastore_journal_iodepth_limit": "5", 2024-09-16T09:26:00.986 INFO:teuthology.orchestra.run.smithi070.stdout: "seastore_main_device_type": "SSD", 2024-09-16T09:26:00.986 INFO:teuthology.orchestra.run.smithi070.stdout: "seastore_max_concurrent_transactions": "8", 2024-09-16T09:26:00.986 INFO:teuthology.orchestra.run.smithi070.stdout: "seastore_max_data_allocation_size": "32768", 2024-09-16T09:26:00.986 INFO:teuthology.orchestra.run.smithi070.stdout: "seastore_multiple_tiers_default_evict_ratio": "0.600000", 2024-09-16T09:26:00.986 INFO:teuthology.orchestra.run.smithi070.stdout: "seastore_multiple_tiers_fast_evict_ratio": "0.700000", 2024-09-16T09:26:00.986 INFO:teuthology.orchestra.run.smithi070.stdout: "seastore_multiple_tiers_stop_evict_ratio": "0.500000", 2024-09-16T09:26:00.987 INFO:teuthology.orchestra.run.smithi070.stdout: "seastore_obj_data_write_amplification": "1.250000", 2024-09-16T09:26:00.987 INFO:teuthology.orchestra.run.smithi070.stdout: "seastore_segment_size": "67108864", 2024-09-16T09:26:00.987 INFO:teuthology.orchestra.run.smithi070.stdout: "set_keepcaps": "false", 2024-09-16T09:26:00.987 INFO:teuthology.orchestra.run.smithi070.stdout: "setgroup": "ceph", 2024-09-16T09:26:00.987 INFO:teuthology.orchestra.run.smithi070.stdout: "setuser": "ceph", 2024-09-16T09:26:00.987 INFO:teuthology.orchestra.run.smithi070.stdout: "setuser_match_path": "", 2024-09-16T09:26:00.987 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_misplaced_ratio": "0.050000", 2024-09-16T09:26:00.987 INFO:teuthology.orchestra.run.smithi070.stdout: "thp": "false", 2024-09-16T09:26:00.987 INFO:teuthology.orchestra.run.smithi070.stdout: "threadpool_default_timeout": "60", 2024-09-16T09:26:00.987 INFO:teuthology.orchestra.run.smithi070.stdout: "threadpool_empty_queue_max_wait": "2", 2024-09-16T09:26:00.987 INFO:teuthology.orchestra.run.smithi070.stdout: "throttler_perf_counter": "true", 2024-09-16T09:26:00.987 INFO:teuthology.orchestra.run.smithi070.stdout: "tmp_dir": "/tmp", 2024-09-16T09:26:00.987 INFO:teuthology.orchestra.run.smithi070.stdout: "tmp_file_template": "/tmp/ceph-mon.a.XXXXXX", 2024-09-16T09:26:00.987 INFO:teuthology.orchestra.run.smithi070.stdout: "uadk_compressor_enabled": "false", 2024-09-16T09:26:00.987 INFO:teuthology.orchestra.run.smithi070.stdout: "uadk_wd_sync_ctx_num": "2" 2024-09-16T09:26:00.988 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T09:26:01.361 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:01 smithi070 ceph-mon[30388]: pgmap v22: 132 pgs: 132 active+clean; 583 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:26:01.361 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:01 smithi070 ceph-mon[40691]: pgmap v22: 132 pgs: 132 active+clean; 583 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:26:01.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:01 smithi084 ceph-mon[32629]: pgmap v22: 132 pgs: 132 active+clean; 583 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:26:01.908 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2024-09-16T09:26:02.157 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:26:02.170 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:26:01 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:26:01] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:26:02.609 INFO:teuthology.orchestra.run.smithi070.stdout:mon.a: {} 2024-09-16T09:26:02.609 INFO:teuthology.orchestra.run.smithi070.stdout:mon.b: {} 2024-09-16T09:26:02.610 INFO:teuthology.orchestra.run.smithi070.stdout:mon.c: {} 2024-09-16T09:26:02.610 INFO:teuthology.orchestra.run.smithi070.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-16T09:26:02.610 INFO:teuthology.orchestra.run.smithi070.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-16T09:26:02.610 INFO:teuthology.orchestra.run.smithi070.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-16T09:26:02.775 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:26:02 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:26:02.920 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:02 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:26:02.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:02 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:26:02.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:02 smithi070 ceph-mon[40691]: pgmap v23: 132 pgs: 132 active+clean; 583 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:26:02.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:02 smithi070 ceph-mon[40691]: 3.6 scrub starts 2024-09-16T09:26:02.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:02 smithi070 ceph-mon[40691]: 3.6 scrub ok 2024-09-16T09:26:02.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:02 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:26:02.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:02 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:26:02.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:02 smithi070 ceph-mon[40691]: osdmap e79: 8 total, 8 up, 8 in 2024-09-16T09:26:02.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:02 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3927790574' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-16T09:26:02.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:02 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:26:02.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:02 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:26:02.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:02 smithi070 ceph-mon[30388]: pgmap v23: 132 pgs: 132 active+clean; 583 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:26:02.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:02 smithi070 ceph-mon[30388]: 3.6 scrub starts 2024-09-16T09:26:02.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:02 smithi070 ceph-mon[30388]: 3.6 scrub ok 2024-09-16T09:26:02.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:02 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:26:02.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:02 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:26:02.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:02 smithi070 ceph-mon[30388]: osdmap e79: 8 total, 8 up, 8 in 2024-09-16T09:26:02.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:02 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3927790574' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-16T09:26:03.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:02 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:26:03.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:02 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:26:03.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:02 smithi084 ceph-mon[32629]: pgmap v23: 132 pgs: 132 active+clean; 583 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:26:03.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:02 smithi084 ceph-mon[32629]: 3.6 scrub starts 2024-09-16T09:26:03.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:02 smithi084 ceph-mon[32629]: 3.6 scrub ok 2024-09-16T09:26:03.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:02 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:26:03.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:02 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:26:03.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:02 smithi084 ceph-mon[32629]: osdmap e79: 8 total, 8 up, 8 in 2024-09-16T09:26:03.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:02 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3927790574' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-16T09:26:03.233 INFO:teuthology.run_tasks:Running task full_sequential... 2024-09-16T09:26:03.243 INFO:teuthology.task.full_sequential:In full_sequential, running task radosbench... 2024-09-16T09:26:03.248 INFO:tasks.radosbench:Beginning radosbench... 2024-09-16T09:26:03.248 INFO:tasks.cephadm.ceph_manager.ceph:creating pool_name unique_pool_0 2024-09-16T09:26:03.248 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd pool create unique_pool_0 16 2024-09-16T09:26:03.249 INFO:tasks.thrashosds.thrasher:starting do_thrash 2024-09-16T09:26:03.249 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T09:26:03.249 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T09:26:03.249 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2024-09-16T09:26:03.250 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:26:03.250 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2024-09-16T09:26:03.250 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T09:26:03.250 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2024-09-16T09:26:03.251 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.0 dump_ops_in_flight 2024-09-16T09:26:03.251 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2024-09-16T09:26:03.251 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd set noscrub 2024-09-16T09:26:03.277 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:03.278 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:03.279 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:26:03.379 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:26:03.404 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:03.405 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:03.405 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:26:03.506 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:26:03.536 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:03.537 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:03.538 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:26:03.538 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:26:03.546 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:26:03.549 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:26:03.554 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:26:03.638 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:26:03.659 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:03.660 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:03.660 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:26:03.761 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:26:03.783 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:03.784 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:03.784 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:26:03.885 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:26:03.911 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:03.912 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:03.912 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:26:03.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:03 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:26:03.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:03 smithi070 ceph-mon[30388]: 3.4 scrub starts 2024-09-16T09:26:03.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:03 smithi070 ceph-mon[30388]: 3.4 scrub ok 2024-09-16T09:26:03.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:03 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:26:03.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:03 smithi070 ceph-mon[40691]: 3.4 scrub starts 2024-09-16T09:26:03.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:03 smithi070 ceph-mon[40691]: 3.4 scrub ok 2024-09-16T09:26:04.013 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:26:04.034 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:04.034 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:04.035 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:26:04.135 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:26:04.157 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:04.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:04.158 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:26:04.159 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:03 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:26:04.159 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:03 smithi084 ceph-mon[32629]: 3.4 scrub starts 2024-09-16T09:26:04.159 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:03 smithi084 ceph-mon[32629]: 3.4 scrub ok 2024-09-16T09:26:04.259 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:26:04.280 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:04.280 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:04.281 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:26:04.382 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:26:04.404 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:04.405 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:04.405 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:26:04.497 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:26:04.498 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:26:04.506 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:26:04.514 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:26:04.514 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:26:04.527 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:04.528 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:04.528 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:26:04.530 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:26:04.531 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:26:04.546 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:26:04.546 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:26:04.563 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:26:04.563 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:26:04.580 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:26:04.581 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:26:04.597 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:26:04.597 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:26:04.613 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:26:04.613 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:26:04.629 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:26:04.650 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:04.651 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:04.651 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:26:04.752 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:26:04.775 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:04.776 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:04.776 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:26:04.877 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:26:04.898 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:04.899 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:04.899 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:26:05.000 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:26:05.022 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:05.022 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:05.023 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:26:05.093 INFO:teuthology.orchestra.run.smithi070.stderr:noscrub is set 2024-09-16T09:26:05.124 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:26:05.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:05.146 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:05.146 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:26:05.247 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:26:05.269 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:05.269 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:05.270 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:26:05.368 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:05 smithi084 ceph-mon[32629]: pgmap v25: 132 pgs: 132 active+clean; 583 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-16T09:26:05.368 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:05 smithi084 ceph-mon[32629]: osdmap e80: 8 total, 8 up, 8 in 2024-09-16T09:26:05.368 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:05 smithi084 ceph-mon[32629]: 3.a scrub starts 2024-09-16T09:26:05.368 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:05 smithi084 ceph-mon[32629]: 3.a scrub ok 2024-09-16T09:26:05.368 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:05 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/2008151810' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:26:05.368 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:05 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/214341259' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:26:05.370 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:26:05.392 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:05.392 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:05.393 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:26:05.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:05 smithi070 ceph-mon[30388]: pgmap v25: 132 pgs: 132 active+clean; 583 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-16T09:26:05.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:05 smithi070 ceph-mon[30388]: osdmap e80: 8 total, 8 up, 8 in 2024-09-16T09:26:05.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:05 smithi070 ceph-mon[30388]: 3.a scrub starts 2024-09-16T09:26:05.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:05 smithi070 ceph-mon[30388]: 3.a scrub ok 2024-09-16T09:26:05.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:05 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2008151810' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:26:05.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:05 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/214341259' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:26:05.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:05 smithi070 ceph-mon[40691]: pgmap v25: 132 pgs: 132 active+clean; 583 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-16T09:26:05.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:05 smithi070 ceph-mon[40691]: osdmap e80: 8 total, 8 up, 8 in 2024-09-16T09:26:05.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:05 smithi070 ceph-mon[40691]: 3.a scrub starts 2024-09-16T09:26:05.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:05 smithi070 ceph-mon[40691]: 3.a scrub ok 2024-09-16T09:26:05.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:05 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/2008151810' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:26:05.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:05 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/214341259' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:26:05.494 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:26:05.517 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:05.518 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:05.518 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:26:05.621 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:26:05.643 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:05.643 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:05.644 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:26:05.744 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:26:05.765 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:05.766 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:05.766 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:26:05.867 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:26:05.888 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:05.889 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:05.889 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:26:05.990 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:26:06.012 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:06.013 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:06.013 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:26:06.114 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:26:06.135 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:06.135 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:06.136 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:26:06.237 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:26:06.259 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:06 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:26:06.259 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:06 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:26:06.259 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:06 smithi070 ceph-mon[30388]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:26:06.259 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:06 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2008151810' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-16T09:26:06.259 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:06 smithi070 ceph-mon[30388]: osdmap e81: 8 total, 8 up, 8 in 2024-09-16T09:26:06.259 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:06 smithi070 ceph-mon[30388]: 3.1 scrub starts 2024-09-16T09:26:06.259 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:06 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:26:06.260 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:06 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:26:06.260 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:06 smithi070 ceph-mon[40691]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:26:06.260 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:06 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/2008151810' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-16T09:26:06.260 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:06 smithi070 ceph-mon[40691]: osdmap e81: 8 total, 8 up, 8 in 2024-09-16T09:26:06.260 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:06 smithi070 ceph-mon[40691]: 3.1 scrub starts 2024-09-16T09:26:06.260 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:06.261 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:06.261 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:26:06.288 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:06 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:26:06.289 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:06 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:26:06.289 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:06 smithi084 ceph-mon[32629]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:26:06.289 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:06 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/2008151810' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-16T09:26:06.289 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:06 smithi084 ceph-mon[32629]: osdmap e81: 8 total, 8 up, 8 in 2024-09-16T09:26:06.289 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:06 smithi084 ceph-mon[32629]: 3.1 scrub starts 2024-09-16T09:26:06.362 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:26:06.385 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:06.386 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:06.386 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:26:06.487 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:26:06.509 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:06.509 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:06.509 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:26:06.610 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:26:06.632 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:06.633 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:06.634 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:26:06.734 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:26:06.755 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:06.756 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:06.756 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:26:06.857 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:26:06.878 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:06.879 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:06.879 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:26:06.963 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T09:26:06.964 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:26:06.980 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:26:07.001 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:07.002 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:07.002 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:26:07.103 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:26:07.108 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'unique_pool_0' created 2024-09-16T09:26:07.129 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:07.130 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:07.130 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:26:07.170 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Sep 16 09:26:06 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-alertmanager-a[75718]: ts=2024-09-16T09:26:06.918Z caller=cluster.go:700 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.00246855s 2024-09-16T09:26:07.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:07 smithi070 ceph-mon[30388]: pgmap v28: 132 pgs: 132 active+clean; 583 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:26:07.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:07 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/621088453' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16} : dispatch 2024-09-16T09:26:07.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:07 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16} : dispatch 2024-09-16T09:26:07.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:07 smithi070 ceph-mon[40691]: pgmap v28: 132 pgs: 132 active+clean; 583 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:26:07.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:07 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/621088453' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16} : dispatch 2024-09-16T09:26:07.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:07 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16} : dispatch 2024-09-16T09:26:07.231 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:26:07.252 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:07.252 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:07.253 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:26:07.354 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:26:07.376 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:07.377 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:07.377 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:26:07.403 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:07 smithi084 ceph-mon[32629]: pgmap v28: 132 pgs: 132 active+clean; 583 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:26:07.403 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:07 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/621088453' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16} : dispatch 2024-09-16T09:26:07.403 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:07 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16} : dispatch 2024-09-16T09:26:07.478 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:26:07.500 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:07.501 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:07.501 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:26:07.523 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T09:26:07.601 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:26:07.623 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:07.623 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:07.624 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:26:07.724 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:26:07.747 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:07.748 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:07.748 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:26:07.779 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd pool application enable unique_pool_0 rados --yes-i-really-mean-it || true 2024-09-16T09:26:07.812 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:26:07.849 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:26:07.870 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:07.870 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:07.871 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:26:07.972 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:26:07.992 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:07.993 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:07.993 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:26:08.091 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:07 smithi084 systemd[1]: Starting Ceph grafana.a for a1321834-740c-11ef-bceb-c7b262605968... 2024-09-16T09:26:08.093 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:26:08.108 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:08 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16}]': finished 2024-09-16T09:26:08.109 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:08 smithi070 ceph-mon[30388]: osdmap e82: 8 total, 8 up, 8 in 2024-09-16T09:26:08.121 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:08.122 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:08.122 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:26:08.223 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:26:08.232 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.0 dump_blocked_ops 2024-09-16T09:26:08.246 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:08.246 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:08.246 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:26:08.272 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:26:08.346 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16}]': finished 2024-09-16T09:26:08.346 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-mon[32629]: osdmap e82: 8 total, 8 up, 8 in 2024-09-16T09:26:08.346 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 podman[63276]: 2024-09-16 09:26:07.993511786 +0000 UTC m=+0.016622930 image pull c8b91775d855b99270fc5d22f3c6737e8cca01ef4c25c8b0362295e0746fa39b quay.io/ceph/grafana:10.4.0 2024-09-16T09:26:08.346 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 podman[63276]: 2024-09-16 09:26:08.096547962 +0000 UTC m=+0.119659146 container create da0e1a0d709a9817205c71da9b13d267f3712c3633e0fcf84450716a6e824aa4 (image=quay.io/ceph/grafana:10.4.0, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a, maintainer=Grafana Labs ) 2024-09-16T09:26:08.346 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 podman[63276]: 2024-09-16 09:26:08.340458851 +0000 UTC m=+0.363569989 container init da0e1a0d709a9817205c71da9b13d267f3712c3633e0fcf84450716a6e824aa4 (image=quay.io/ceph/grafana:10.4.0, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a, maintainer=Grafana Labs ) 2024-09-16T09:26:08.347 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 podman[63276]: 2024-09-16 09:26:08.345213044 +0000 UTC m=+0.368324186 container start da0e1a0d709a9817205c71da9b13d267f3712c3633e0fcf84450716a6e824aa4 (image=quay.io/ceph/grafana:10.4.0, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a, maintainer=Grafana Labs ) 2024-09-16T09:26:08.347 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:26:08.358 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:08 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16}]': finished 2024-09-16T09:26:08.358 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:08 smithi070 ceph-mon[40691]: osdmap e82: 8 total, 8 up, 8 in 2024-09-16T09:26:08.370 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:08.370 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:08.371 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:26:08.471 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:26:08.496 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:08.496 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:08.500 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:26:08.597 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=settings t=2024-09-16T09:26:08.467635332Z level=info msg="Starting Grafana" version=10.4.0 commit=03f502a94d17f7dc4e6c34acdf8428aedd986e4c branch=HEAD compiled=2024-09-16T09:26:08Z 2024-09-16T09:26:08.597 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=settings t=2024-09-16T09:26:08.46785216Z level=info msg="Config loaded from" file=/usr/share/grafana/conf/defaults.ini 2024-09-16T09:26:08.597 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=settings t=2024-09-16T09:26:08.467860016Z level=info msg="Config loaded from" file=/etc/grafana/grafana.ini 2024-09-16T09:26:08.598 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=settings t=2024-09-16T09:26:08.467863095Z level=info msg="Config overridden from command line" arg="default.paths.data=/var/lib/grafana" 2024-09-16T09:26:08.598 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=settings t=2024-09-16T09:26:08.46786608Z level=info msg="Config overridden from command line" arg="default.paths.logs=/var/log/grafana" 2024-09-16T09:26:08.598 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=settings t=2024-09-16T09:26:08.467868898Z level=info msg="Config overridden from command line" arg="default.paths.plugins=/var/lib/grafana/plugins" 2024-09-16T09:26:08.598 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=settings t=2024-09-16T09:26:08.467871447Z level=info msg="Config overridden from command line" arg="default.paths.provisioning=/etc/grafana/provisioning" 2024-09-16T09:26:08.598 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=settings t=2024-09-16T09:26:08.467873937Z level=info msg="Config overridden from command line" arg="default.log.mode=console" 2024-09-16T09:26:08.598 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=settings t=2024-09-16T09:26:08.467876899Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_DATA=/var/lib/grafana" 2024-09-16T09:26:08.598 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=settings t=2024-09-16T09:26:08.467879676Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_LOGS=/var/log/grafana" 2024-09-16T09:26:08.598 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=settings t=2024-09-16T09:26:08.467882133Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PLUGINS=/var/lib/grafana/plugins" 2024-09-16T09:26:08.598 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=settings t=2024-09-16T09:26:08.467885546Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PROVISIONING=/etc/grafana/provisioning" 2024-09-16T09:26:08.598 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=settings t=2024-09-16T09:26:08.46788821Z level=info msg=Target target=[all] 2024-09-16T09:26:08.598 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=settings t=2024-09-16T09:26:08.467893377Z level=info msg="Path Home" path=/usr/share/grafana 2024-09-16T09:26:08.598 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=settings t=2024-09-16T09:26:08.467896106Z level=info msg="Path Data" path=/var/lib/grafana 2024-09-16T09:26:08.598 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=settings t=2024-09-16T09:26:08.46790398Z level=info msg="Path Logs" path=/var/log/grafana 2024-09-16T09:26:08.599 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=settings t=2024-09-16T09:26:08.467906865Z level=info msg="Path Plugins" path=/var/lib/grafana/plugins 2024-09-16T09:26:08.599 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=settings t=2024-09-16T09:26:08.46790982Z level=info msg="Path Provisioning" path=/etc/grafana/provisioning 2024-09-16T09:26:08.599 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=settings t=2024-09-16T09:26:08.467912255Z level=info msg="App mode production" 2024-09-16T09:26:08.599 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=sqlstore t=2024-09-16T09:26:08.468150485Z level=info msg="Connecting to DB" dbtype=sqlite3 2024-09-16T09:26:08.599 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=sqlstore t=2024-09-16T09:26:08.468163155Z level=warn msg="SQLite database file has broader permissions than it should" path=/var/lib/grafana/grafana.db mode=-rw-r--r-- expected=-rw-r----- 2024-09-16T09:26:08.599 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.468627114Z level=info msg="Starting DB migrations" 2024-09-16T09:26:08.599 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.469465819Z level=info msg="Executing migration" id="create migration_log table" 2024-09-16T09:26:08.599 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.470214837Z level=info msg="Migration successfully executed" id="create migration_log table" duration=748.428µs 2024-09-16T09:26:08.599 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.470668916Z level=info msg="Executing migration" id="create user table" 2024-09-16T09:26:08.599 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.47119983Z level=info msg="Migration successfully executed" id="create user table" duration=531.212µs 2024-09-16T09:26:08.599 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.471582711Z level=info msg="Executing migration" id="add unique index user.login" 2024-09-16T09:26:08.599 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.472045963Z level=info msg="Migration successfully executed" id="add unique index user.login" duration=463.677µs 2024-09-16T09:26:08.599 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.472413319Z level=info msg="Executing migration" id="add unique index user.email" 2024-09-16T09:26:08.600 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.472870924Z level=info msg="Migration successfully executed" id="add unique index user.email" duration=457.508µs 2024-09-16T09:26:08.600 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.473259294Z level=info msg="Executing migration" id="drop index UQE_user_login - v1" 2024-09-16T09:26:08.600 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.473685264Z level=info msg="Migration successfully executed" id="drop index UQE_user_login - v1" duration=425.757µs 2024-09-16T09:26:08.600 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.474107855Z level=info msg="Executing migration" id="drop index UQE_user_email - v1" 2024-09-16T09:26:08.600 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.47452766Z level=info msg="Migration successfully executed" id="drop index UQE_user_email - v1" duration=419.747µs 2024-09-16T09:26:08.600 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.474867417Z level=info msg="Executing migration" id="Rename table user to user_v1 - v1" 2024-09-16T09:26:08.600 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.476406419Z level=info msg="Migration successfully executed" id="Rename table user to user_v1 - v1" duration=1.556564ms 2024-09-16T09:26:08.600 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.476788384Z level=info msg="Executing migration" id="create user table v2" 2024-09-16T09:26:08.601 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:26:08.603 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.477305036Z level=info msg="Migration successfully executed" id="create user table v2" duration=516.605µs 2024-09-16T09:26:08.603 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.47764674Z level=info msg="Executing migration" id="create index UQE_user_login - v2" 2024-09-16T09:26:08.603 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.478081422Z level=info msg="Migration successfully executed" id="create index UQE_user_login - v2" duration=434.454µs 2024-09-16T09:26:08.603 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.478397215Z level=info msg="Executing migration" id="create index UQE_user_email - v2" 2024-09-16T09:26:08.603 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.478834105Z level=info msg="Migration successfully executed" id="create index UQE_user_email - v2" duration=435.721µs 2024-09-16T09:26:08.603 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.479210119Z level=info msg="Executing migration" id="copy data_source v1 to v2" 2024-09-16T09:26:08.604 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.479465743Z level=info msg="Migration successfully executed" id="copy data_source v1 to v2" duration=255.403µs 2024-09-16T09:26:08.604 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.479777129Z level=info msg="Executing migration" id="Drop old table user_v1" 2024-09-16T09:26:08.604 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.480133902Z level=info msg="Migration successfully executed" id="Drop old table user_v1" duration=355.61µs 2024-09-16T09:26:08.604 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.480452951Z level=info msg="Executing migration" id="Add column help_flags1 to user table" 2024-09-16T09:26:08.604 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.481148898Z level=info msg="Migration successfully executed" id="Add column help_flags1 to user table" duration=695.571µs 2024-09-16T09:26:08.604 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.481517726Z level=info msg="Executing migration" id="Update user table charset" 2024-09-16T09:26:08.604 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.481533647Z level=info msg="Migration successfully executed" id="Update user table charset" duration=16.675µs 2024-09-16T09:26:08.604 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.481933346Z level=info msg="Executing migration" id="Add last_seen_at column to user" 2024-09-16T09:26:08.604 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 bash[63276]: da0e1a0d709a9817205c71da9b13d267f3712c3633e0fcf84450716a6e824aa4 2024-09-16T09:26:08.604 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.482629758Z level=info msg="Migration successfully executed" id="Add last_seen_at column to user" duration=695.952µs 2024-09-16T09:26:08.604 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.482973654Z level=info msg="Executing migration" id="Add missing user data" 2024-09-16T09:26:08.604 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.483105263Z level=info msg="Migration successfully executed" id="Add missing user data" duration=131.673µs 2024-09-16T09:26:08.605 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.483490258Z level=info msg="Executing migration" id="Add is_disabled column to user" 2024-09-16T09:26:08.605 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.484196936Z level=info msg="Migration successfully executed" id="Add is_disabled column to user" duration=706.103µs 2024-09-16T09:26:08.605 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.484537417Z level=info msg="Executing migration" id="Add index user.login/user.email" 2024-09-16T09:26:08.605 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 systemd[1]: Started Ceph grafana.a for a1321834-740c-11ef-bceb-c7b262605968. 2024-09-16T09:26:08.605 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.484987649Z level=info msg="Migration successfully executed" id="Add index user.login/user.email" duration=450µs 2024-09-16T09:26:08.605 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.485325432Z level=info msg="Executing migration" id="Add is_service_account column to user" 2024-09-16T09:26:08.605 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.4860525Z level=info msg="Migration successfully executed" id="Add is_service_account column to user" duration=726.674µs 2024-09-16T09:26:08.605 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.486424742Z level=info msg="Executing migration" id="Update is_service_account column to nullable" 2024-09-16T09:26:08.605 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.491620005Z level=info msg="Migration successfully executed" id="Update is_service_account column to nullable" duration=5.192557ms 2024-09-16T09:26:08.605 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.492110334Z level=info msg="Executing migration" id="Add uid column to user" 2024-09-16T09:26:08.605 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.492910514Z level=info msg="Migration successfully executed" id="Add uid column to user" duration=789.176µs 2024-09-16T09:26:08.605 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.493277678Z level=info msg="Executing migration" id="Update uid column values for users" 2024-09-16T09:26:08.605 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.493429695Z level=info msg="Migration successfully executed" id="Update uid column values for users" duration=152.002µs 2024-09-16T09:26:08.606 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.493828804Z level=info msg="Executing migration" id="Add unique index user_uid" 2024-09-16T09:26:08.606 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.494307563Z level=info msg="Migration successfully executed" id="Add unique index user_uid" duration=478.656µs 2024-09-16T09:26:08.606 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.494739481Z level=info msg="Executing migration" id="create temp user table v1-7" 2024-09-16T09:26:08.606 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.49526106Z level=info msg="Migration successfully executed" id="create temp user table v1-7" duration=521.454µs 2024-09-16T09:26:08.606 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.495654055Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v1-7" 2024-09-16T09:26:08.606 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.496094334Z level=info msg="Migration successfully executed" id="create index IDX_temp_user_email - v1-7" duration=439.96µs 2024-09-16T09:26:08.606 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.496509612Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v1-7" 2024-09-16T09:26:08.606 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.497133834Z level=info msg="Migration successfully executed" id="create index IDX_temp_user_org_id - v1-7" duration=623.899µs 2024-09-16T09:26:08.606 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.497589437Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v1-7" 2024-09-16T09:26:08.606 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.498183443Z level=info msg="Migration successfully executed" id="create index IDX_temp_user_code - v1-7" duration=593.915µs 2024-09-16T09:26:08.606 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.498626612Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v1-7" 2024-09-16T09:26:08.606 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.499156401Z level=info msg="Migration successfully executed" id="create index IDX_temp_user_status - v1-7" duration=529.6µs 2024-09-16T09:26:08.607 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.499597502Z level=info msg="Executing migration" id="Update temp_user table charset" 2024-09-16T09:26:08.607 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.499620187Z level=info msg="Migration successfully executed" id="Update temp_user table charset" duration=23.705µs 2024-09-16T09:26:08.607 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.500075194Z level=info msg="Executing migration" id="drop index IDX_temp_user_email - v1" 2024-09-16T09:26:08.607 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.50053191Z level=info msg="Migration successfully executed" id="drop index IDX_temp_user_email - v1" duration=458.506µs 2024-09-16T09:26:08.607 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.500858673Z level=info msg="Executing migration" id="drop index IDX_temp_user_org_id - v1" 2024-09-16T09:26:08.607 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.501312912Z level=info msg="Migration successfully executed" id="drop index IDX_temp_user_org_id - v1" duration=454.221µs 2024-09-16T09:26:08.607 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.501687018Z level=info msg="Executing migration" id="drop index IDX_temp_user_code - v1" 2024-09-16T09:26:08.607 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.502131851Z level=info msg="Migration successfully executed" id="drop index IDX_temp_user_code - v1" duration=444.432µs 2024-09-16T09:26:08.607 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.502455307Z level=info msg="Executing migration" id="drop index IDX_temp_user_status - v1" 2024-09-16T09:26:08.607 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.502867949Z level=info msg="Migration successfully executed" id="drop index IDX_temp_user_status - v1" duration=412.523µs 2024-09-16T09:26:08.607 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.50320433Z level=info msg="Executing migration" id="Rename table temp_user to temp_user_tmp_qwerty - v1" 2024-09-16T09:26:08.607 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.505908436Z level=info msg="Migration successfully executed" id="Rename table temp_user to temp_user_tmp_qwerty - v1" duration=2.695074ms 2024-09-16T09:26:08.607 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.50638844Z level=info msg="Executing migration" id="create temp_user v2" 2024-09-16T09:26:08.607 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.507157517Z level=info msg="Migration successfully executed" id="create temp_user v2" duration=768.766µs 2024-09-16T09:26:08.608 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.507571587Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v2" 2024-09-16T09:26:08.608 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.508226747Z level=info msg="Migration successfully executed" id="create index IDX_temp_user_email - v2" duration=654.742µs 2024-09-16T09:26:08.608 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.508640458Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v2" 2024-09-16T09:26:08.608 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.509278476Z level=info msg="Migration successfully executed" id="create index IDX_temp_user_org_id - v2" duration=638.045µs 2024-09-16T09:26:08.608 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.509714911Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v2" 2024-09-16T09:26:08.608 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.51033725Z level=info msg="Migration successfully executed" id="create index IDX_temp_user_code - v2" duration=621.002µs 2024-09-16T09:26:08.608 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.510749631Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v2" 2024-09-16T09:26:08.608 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.511364371Z level=info msg="Migration successfully executed" id="create index IDX_temp_user_status - v2" duration=614.541µs 2024-09-16T09:26:08.608 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.511815375Z level=info msg="Executing migration" id="copy temp_user v1 to v2" 2024-09-16T09:26:08.608 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.512112623Z level=info msg="Migration successfully executed" id="copy temp_user v1 to v2" duration=297.384µs 2024-09-16T09:26:08.608 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.512443313Z level=info msg="Executing migration" id="drop temp_user_tmp_qwerty" 2024-09-16T09:26:08.609 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.512814837Z level=info msg="Migration successfully executed" id="drop temp_user_tmp_qwerty" duration=371.148µs 2024-09-16T09:26:08.609 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.513196871Z level=info msg="Executing migration" id="Set created for temp users that will otherwise prematurely expire" 2024-09-16T09:26:08.609 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.513446077Z level=info msg="Migration successfully executed" id="Set created for temp users that will otherwise prematurely expire" duration=249.319µs 2024-09-16T09:26:08.609 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.513816244Z level=info msg="Executing migration" id="create star table" 2024-09-16T09:26:08.609 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.514252857Z level=info msg="Migration successfully executed" id="create star table" duration=436.355µs 2024-09-16T09:26:08.609 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.514585733Z level=info msg="Executing migration" id="add unique index star.user_id_dashboard_id" 2024-09-16T09:26:08.609 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.515075617Z level=info msg="Migration successfully executed" id="add unique index star.user_id_dashboard_id" duration=489.652µs 2024-09-16T09:26:08.609 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.515436262Z level=info msg="Executing migration" id="create org table v1" 2024-09-16T09:26:08.609 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.515926304Z level=info msg="Migration successfully executed" id="create org table v1" duration=489.93µs 2024-09-16T09:26:08.609 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.516249153Z level=info msg="Executing migration" id="create index UQE_org_name - v1" 2024-09-16T09:26:08.609 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.516706825Z level=info msg="Migration successfully executed" id="create index UQE_org_name - v1" duration=457.481µs 2024-09-16T09:26:08.609 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.517041218Z level=info msg="Executing migration" id="create org_user table v1" 2024-09-16T09:26:08.610 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.517473337Z level=info msg="Migration successfully executed" id="create org_user table v1" duration=431.951µs 2024-09-16T09:26:08.610 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.517953237Z level=info msg="Executing migration" id="create index IDX_org_user_org_id - v1" 2024-09-16T09:26:08.610 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.518668495Z level=info msg="Migration successfully executed" id="create index IDX_org_user_org_id - v1" duration=715.031µs 2024-09-16T09:26:08.610 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.519135423Z level=info msg="Executing migration" id="create index UQE_org_user_org_id_user_id - v1" 2024-09-16T09:26:08.610 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.519798172Z level=info msg="Migration successfully executed" id="create index UQE_org_user_org_id_user_id - v1" duration=662.416µs 2024-09-16T09:26:08.610 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.520268058Z level=info msg="Executing migration" id="create index IDX_org_user_user_id - v1" 2024-09-16T09:26:08.610 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.520919517Z level=info msg="Migration successfully executed" id="create index IDX_org_user_user_id - v1" duration=651.117µs 2024-09-16T09:26:08.610 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.521358755Z level=info msg="Executing migration" id="Update org table charset" 2024-09-16T09:26:08.610 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.521381075Z level=info msg="Migration successfully executed" id="Update org table charset" duration=23.088µs 2024-09-16T09:26:08.610 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.522137746Z level=info msg="Executing migration" id="Update org_user table charset" 2024-09-16T09:26:08.610 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.522163832Z level=info msg="Migration successfully executed" id="Update org_user table charset" duration=26.827µs 2024-09-16T09:26:08.610 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.522696381Z level=info msg="Executing migration" id="Migrate all Read Only Viewers to Viewers" 2024-09-16T09:26:08.610 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.522844437Z level=info msg="Migration successfully executed" id="Migrate all Read Only Viewers to Viewers" duration=148.199µs 2024-09-16T09:26:08.611 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.523529977Z level=info msg="Executing migration" id="create dashboard table" 2024-09-16T09:26:08.611 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.524331742Z level=info msg="Migration successfully executed" id="create dashboard table" duration=801.102µs 2024-09-16T09:26:08.611 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.524817207Z level=info msg="Executing migration" id="add index dashboard.account_id" 2024-09-16T09:26:08.611 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.525608759Z level=info msg="Migration successfully executed" id="add index dashboard.account_id" duration=792.373µs 2024-09-16T09:26:08.611 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.526078773Z level=info msg="Executing migration" id="add unique index dashboard_account_id_slug" 2024-09-16T09:26:08.611 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.526791441Z level=info msg="Migration successfully executed" id="add unique index dashboard_account_id_slug" duration=712.165µs 2024-09-16T09:26:08.611 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.527261155Z level=info msg="Executing migration" id="create dashboard_tag table" 2024-09-16T09:26:08.611 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.528007508Z level=info msg="Migration successfully executed" id="create dashboard_tag table" duration=745.863µs 2024-09-16T09:26:08.611 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.529975085Z level=info msg="Executing migration" id="add unique index dashboard_tag.dasboard_id_term" 2024-09-16T09:26:08.611 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.530679612Z level=info msg="Migration successfully executed" id="add unique index dashboard_tag.dasboard_id_term" duration=704.169µs 2024-09-16T09:26:08.611 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.532650472Z level=info msg="Executing migration" id="drop index UQE_dashboard_tag_dashboard_id_term - v1" 2024-09-16T09:26:08.611 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.533349692Z level=info msg="Migration successfully executed" id="drop index UQE_dashboard_tag_dashboard_id_term - v1" duration=700.863µs 2024-09-16T09:26:08.611 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.535314877Z level=info msg="Executing migration" id="Rename table dashboard to dashboard_v1 - v1" 2024-09-16T09:26:08.612 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.539098956Z level=info msg="Migration successfully executed" id="Rename table dashboard to dashboard_v1 - v1" duration=3.784328ms 2024-09-16T09:26:08.612 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.539495518Z level=info msg="Executing migration" id="create dashboard v2" 2024-09-16T09:26:08.612 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.540003371Z level=info msg="Migration successfully executed" id="create dashboard v2" duration=507.633µs 2024-09-16T09:26:08.612 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.542469535Z level=info msg="Executing migration" id="create index IDX_dashboard_org_id - v2" 2024-09-16T09:26:08.612 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.542959663Z level=info msg="Migration successfully executed" id="create index IDX_dashboard_org_id - v2" duration=490.624µs 2024-09-16T09:26:08.612 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.545194783Z level=info msg="Executing migration" id="create index UQE_dashboard_org_id_slug - v2" 2024-09-16T09:26:08.612 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.545701472Z level=info msg="Migration successfully executed" id="create index UQE_dashboard_org_id_slug - v2" duration=506.437µs 2024-09-16T09:26:08.612 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.547749288Z level=info msg="Executing migration" id="copy dashboard v1 to v2" 2024-09-16T09:26:08.612 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.548031471Z level=info msg="Migration successfully executed" id="copy dashboard v1 to v2" duration=279.868µs 2024-09-16T09:26:08.612 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.550636754Z level=info msg="Executing migration" id="drop table dashboard_v1" 2024-09-16T09:26:08.612 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.551273793Z level=info msg="Migration successfully executed" id="drop table dashboard_v1" duration=636.76µs 2024-09-16T09:26:08.612 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.553103982Z level=info msg="Executing migration" id="alter dashboard.data to mediumtext v1" 2024-09-16T09:26:08.613 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.553152325Z level=info msg="Migration successfully executed" id="alter dashboard.data to mediumtext v1" duration=48.44µs 2024-09-16T09:26:08.613 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.553587068Z level=info msg="Executing migration" id="Add column updated_by in dashboard - v2" 2024-09-16T09:26:08.613 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.554728342Z level=info msg="Migration successfully executed" id="Add column updated_by in dashboard - v2" duration=1.140691ms 2024-09-16T09:26:08.613 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.556394799Z level=info msg="Executing migration" id="Add column created_by in dashboard - v2" 2024-09-16T09:26:08.613 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.557557963Z level=info msg="Migration successfully executed" id="Add column created_by in dashboard - v2" duration=1.162782ms 2024-09-16T09:26:08.613 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.559155875Z level=info msg="Executing migration" id="Add column gnetId in dashboard" 2024-09-16T09:26:08.613 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.560295584Z level=info msg="Migration successfully executed" id="Add column gnetId in dashboard" duration=1.141276ms 2024-09-16T09:26:08.613 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.562026486Z level=info msg="Executing migration" id="Add index for gnetId in dashboard" 2024-09-16T09:26:08.613 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.562546744Z level=info msg="Migration successfully executed" id="Add index for gnetId in dashboard" duration=519.937µs 2024-09-16T09:26:08.613 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.564991308Z level=info msg="Executing migration" id="Add column plugin_id in dashboard" 2024-09-16T09:26:08.613 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.566087057Z level=info msg="Migration successfully executed" id="Add column plugin_id in dashboard" duration=1.095263ms 2024-09-16T09:26:08.614 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.56780773Z level=info msg="Executing migration" id="Add index for plugin_id in dashboard" 2024-09-16T09:26:08.614 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.568354522Z level=info msg="Migration successfully executed" id="Add index for plugin_id in dashboard" duration=546.517µs 2024-09-16T09:26:08.614 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.570521217Z level=info msg="Executing migration" id="Add index for dashboard_id in dashboard_tag" 2024-09-16T09:26:08.614 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.571301287Z level=info msg="Migration successfully executed" id="Add index for dashboard_id in dashboard_tag" duration=784.919µs 2024-09-16T09:26:08.614 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.573077814Z level=info msg="Executing migration" id="Update dashboard table charset" 2024-09-16T09:26:08.614 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.573100445Z level=info msg="Migration successfully executed" id="Update dashboard table charset" duration=23.457µs 2024-09-16T09:26:08.614 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.573548167Z level=info msg="Executing migration" id="Update dashboard_tag table charset" 2024-09-16T09:26:08.614 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.573564064Z level=info msg="Migration successfully executed" id="Update dashboard_tag table charset" duration=17.853µs 2024-09-16T09:26:08.614 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.57559702Z level=info msg="Executing migration" id="Add column folder_id in dashboard" 2024-09-16T09:26:08.614 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.576800516Z level=info msg="Migration successfully executed" id="Add column folder_id in dashboard" duration=1.203116ms 2024-09-16T09:26:08.614 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.577226546Z level=info msg="Executing migration" id="Add column isFolder in dashboard" 2024-09-16T09:26:08.614 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.578434891Z level=info msg="Migration successfully executed" id="Add column isFolder in dashboard" duration=1.207994ms 2024-09-16T09:26:08.614 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.578812188Z level=info msg="Executing migration" id="Add column has_acl in dashboard" 2024-09-16T09:26:08.615 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.579999547Z level=info msg="Migration successfully executed" id="Add column has_acl in dashboard" duration=1.185513ms 2024-09-16T09:26:08.615 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.581597612Z level=info msg="Executing migration" id="Add column uid in dashboard" 2024-09-16T09:26:08.615 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.582769419Z level=info msg="Migration successfully executed" id="Add column uid in dashboard" duration=1.17146ms 2024-09-16T09:26:08.615 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.583930176Z level=info msg="Executing migration" id="Update uid column values in dashboard" 2024-09-16T09:26:08.615 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.584096365Z level=info msg="Migration successfully executed" id="Update uid column values in dashboard" duration=196.061µs 2024-09-16T09:26:08.615 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.586564494Z level=info msg="Executing migration" id="Add unique index dashboard_org_id_uid" 2024-09-16T09:26:08.615 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.587256241Z level=info msg="Migration successfully executed" id="Add unique index dashboard_org_id_uid" duration=693.888µs 2024-09-16T09:26:08.615 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.589128564Z level=info msg="Executing migration" id="Remove unique index org_id_slug" 2024-09-16T09:26:08.615 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.589628971Z level=info msg="Migration successfully executed" id="Remove unique index org_id_slug" duration=500.61µs 2024-09-16T09:26:08.615 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.591421263Z level=info msg="Executing migration" id="Update dashboard title length" 2024-09-16T09:26:08.615 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.591436938Z level=info msg="Migration successfully executed" id="Update dashboard title length" duration=16.37µs 2024-09-16T09:26:08.615 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.594103197Z level=info msg="Executing migration" id="Add unique index for dashboard_org_id_title_folder_id" 2024-09-16T09:26:08.615 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.594701986Z level=info msg="Migration successfully executed" id="Add unique index for dashboard_org_id_title_folder_id" duration=600.789µs 2024-09-16T09:26:08.615 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.596491298Z level=info msg="Executing migration" id="create dashboard_provisioning" 2024-09-16T09:26:08.616 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.597011101Z level=info msg="Migration successfully executed" id="create dashboard_provisioning" duration=520.451µs 2024-09-16T09:26:08.616 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.598970872Z level=info msg="Executing migration" id="Rename table dashboard_provisioning to dashboard_provisioning_tmp_qwerty - v1" 2024-09-16T09:26:08.623 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:08.624 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:08.624 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:26:08.666 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-16T09:26:08.725 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:26:08.747 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:08.748 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:08.749 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:26:08.849 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:26:08.851 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.602321159Z level=info msg="Migration successfully executed" id="Rename table dashboard_provisioning to dashboard_provisioning_tmp_qwerty - v1" duration=3.349849ms 2024-09-16T09:26:08.851 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.60269894Z level=info msg="Executing migration" id="create dashboard_provisioning v2" 2024-09-16T09:26:08.851 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.603179732Z level=info msg="Migration successfully executed" id="create dashboard_provisioning v2" duration=481.291µs 2024-09-16T09:26:08.852 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.605315506Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id - v2" 2024-09-16T09:26:08.852 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.605824773Z level=info msg="Migration successfully executed" id="create index IDX_dashboard_provisioning_dashboard_id - v2" duration=508.879µs 2024-09-16T09:26:08.852 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.608070735Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id_name - v2" 2024-09-16T09:26:08.852 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.608542771Z level=info msg="Migration successfully executed" id="create index IDX_dashboard_provisioning_dashboard_id_name - v2" duration=471.716µs 2024-09-16T09:26:08.852 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.610583388Z level=info msg="Executing migration" id="copy dashboard_provisioning v1 to v2" 2024-09-16T09:26:08.852 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.610810511Z level=info msg="Migration successfully executed" id="copy dashboard_provisioning v1 to v2" duration=227.153µs 2024-09-16T09:26:08.852 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.613153679Z level=info msg="Executing migration" id="drop dashboard_provisioning_tmp_qwerty" 2024-09-16T09:26:08.852 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.613529729Z level=info msg="Migration successfully executed" id="drop dashboard_provisioning_tmp_qwerty" duration=375.755µs 2024-09-16T09:26:08.852 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.615742754Z level=info msg="Executing migration" id="Add check_sum column" 2024-09-16T09:26:08.852 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.616966242Z level=info msg="Migration successfully executed" id="Add check_sum column" duration=1.223051ms 2024-09-16T09:26:08.852 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.618375344Z level=info msg="Executing migration" id="Add index for dashboard_title" 2024-09-16T09:26:08.853 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.618964605Z level=info msg="Migration successfully executed" id="Add index for dashboard_title" duration=590.194µs 2024-09-16T09:26:08.853 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.619289108Z level=info msg="Executing migration" id="delete tags for deleted dashboards" 2024-09-16T09:26:08.853 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.619392988Z level=info msg="Migration successfully executed" id="delete tags for deleted dashboards" duration=103.994µs 2024-09-16T09:26:08.853 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.62007305Z level=info msg="Executing migration" id="delete stars for deleted dashboards" 2024-09-16T09:26:08.853 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.620176673Z level=info msg="Migration successfully executed" id="delete stars for deleted dashboards" duration=103.553µs 2024-09-16T09:26:08.853 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.621101553Z level=info msg="Executing migration" id="Add index for dashboard_is_folder" 2024-09-16T09:26:08.853 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.621589552Z level=info msg="Migration successfully executed" id="Add index for dashboard_is_folder" duration=489.995µs 2024-09-16T09:26:08.853 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.621950567Z level=info msg="Executing migration" id="Add isPublic for dashboard" 2024-09-16T09:26:08.853 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.623221614Z level=info msg="Migration successfully executed" id="Add isPublic for dashboard" duration=1.270719ms 2024-09-16T09:26:08.853 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.623617209Z level=info msg="Executing migration" id="create data_source table" 2024-09-16T09:26:08.853 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.624229735Z level=info msg="Migration successfully executed" id="create data_source table" duration=612.302µs 2024-09-16T09:26:08.854 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.624633416Z level=info msg="Executing migration" id="add index data_source.account_id" 2024-09-16T09:26:08.854 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.625109958Z level=info msg="Migration successfully executed" id="add index data_source.account_id" duration=476.272µs 2024-09-16T09:26:08.854 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.62546246Z level=info msg="Executing migration" id="add unique index data_source.account_id_name" 2024-09-16T09:26:08.854 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.625942005Z level=info msg="Migration successfully executed" id="add unique index data_source.account_id_name" duration=479.174µs 2024-09-16T09:26:08.854 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.628141649Z level=info msg="Executing migration" id="drop index IDX_data_source_account_id - v1" 2024-09-16T09:26:08.854 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.628621519Z level=info msg="Migration successfully executed" id="drop index IDX_data_source_account_id - v1" duration=479.722µs 2024-09-16T09:26:08.854 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.630931036Z level=info msg="Executing migration" id="drop index UQE_data_source_account_id_name - v1" 2024-09-16T09:26:08.854 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.63142585Z level=info msg="Migration successfully executed" id="drop index UQE_data_source_account_id_name - v1" duration=478.82µs 2024-09-16T09:26:08.854 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.631785193Z level=info msg="Executing migration" id="Rename table data_source to data_source_v1 - v1" 2024-09-16T09:26:08.854 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.635214147Z level=info msg="Migration successfully executed" id="Rename table data_source to data_source_v1 - v1" duration=3.428425ms 2024-09-16T09:26:08.854 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.635584128Z level=info msg="Executing migration" id="create data_source table v2" 2024-09-16T09:26:08.854 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.636161934Z level=info msg="Migration successfully executed" id="create data_source table v2" duration=577.576µs 2024-09-16T09:26:08.854 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.636542704Z level=info msg="Executing migration" id="create index IDX_data_source_org_id - v2" 2024-09-16T09:26:08.855 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.637050645Z level=info msg="Migration successfully executed" id="create index IDX_data_source_org_id - v2" duration=507.688µs 2024-09-16T09:26:08.855 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.637393663Z level=info msg="Executing migration" id="create index UQE_data_source_org_id_name - v2" 2024-09-16T09:26:08.855 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.637896451Z level=info msg="Migration successfully executed" id="create index UQE_data_source_org_id_name - v2" duration=503.507µs 2024-09-16T09:26:08.855 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.638277847Z level=info msg="Executing migration" id="Drop old table data_source_v1 #2" 2024-09-16T09:26:08.855 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.638718759Z level=info msg="Migration successfully executed" id="Drop old table data_source_v1 #2" duration=441.047µs 2024-09-16T09:26:08.855 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.639127059Z level=info msg="Executing migration" id="Add column with_credentials" 2024-09-16T09:26:08.855 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.640564792Z level=info msg="Migration successfully executed" id="Add column with_credentials" duration=1.437396ms 2024-09-16T09:26:08.855 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.64096529Z level=info msg="Executing migration" id="Add secure json data column" 2024-09-16T09:26:08.855 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.642402402Z level=info msg="Migration successfully executed" id="Add secure json data column" duration=1.436842ms 2024-09-16T09:26:08.855 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.642792949Z level=info msg="Executing migration" id="Update data_source table charset" 2024-09-16T09:26:08.855 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.642808244Z level=info msg="Migration successfully executed" id="Update data_source table charset" duration=16.087µs 2024-09-16T09:26:08.855 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.643659232Z level=info msg="Executing migration" id="Update initial version to 1" 2024-09-16T09:26:08.855 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.643819522Z level=info msg="Migration successfully executed" id="Update initial version to 1" duration=160.1µs 2024-09-16T09:26:08.855 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.644636553Z level=info msg="Executing migration" id="Add read_only data column" 2024-09-16T09:26:08.856 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.646170723Z level=info msg="Migration successfully executed" id="Add read_only data column" duration=1.536205ms 2024-09-16T09:26:08.856 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.646602784Z level=info msg="Executing migration" id="Migrate logging ds to loki ds" 2024-09-16T09:26:08.856 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.646750815Z level=info msg="Migration successfully executed" id="Migrate logging ds to loki ds" duration=147.894µs 2024-09-16T09:26:08.856 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.647462783Z level=info msg="Executing migration" id="Update json_data with nulls" 2024-09-16T09:26:08.856 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.647617589Z level=info msg="Migration successfully executed" id="Update json_data with nulls" duration=154.769µs 2024-09-16T09:26:08.856 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.64850898Z level=info msg="Executing migration" id="Add uid column" 2024-09-16T09:26:08.856 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.650001559Z level=info msg="Migration successfully executed" id="Add uid column" duration=1.494337ms 2024-09-16T09:26:08.856 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.650374296Z level=info msg="Executing migration" id="Update uid value" 2024-09-16T09:26:08.856 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.650519105Z level=info msg="Migration successfully executed" id="Update uid value" duration=144.829µs 2024-09-16T09:26:08.856 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.651198649Z level=info msg="Executing migration" id="Add unique index datasource_org_id_uid" 2024-09-16T09:26:08.856 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.651742312Z level=info msg="Migration successfully executed" id="Add unique index datasource_org_id_uid" duration=543.259µs 2024-09-16T09:26:08.856 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.652178122Z level=info msg="Executing migration" id="add unique index datasource_org_id_is_default" 2024-09-16T09:26:08.856 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.652688158Z level=info msg="Migration successfully executed" id="add unique index datasource_org_id_is_default" duration=524.371µs 2024-09-16T09:26:08.857 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.653072466Z level=info msg="Executing migration" id="create api_key table" 2024-09-16T09:26:08.857 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.653619336Z level=info msg="Migration successfully executed" id="create api_key table" duration=546.225µs 2024-09-16T09:26:08.857 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.65399659Z level=info msg="Executing migration" id="add index api_key.account_id" 2024-09-16T09:26:08.857 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.654507836Z level=info msg="Migration successfully executed" id="add index api_key.account_id" duration=508.974µs 2024-09-16T09:26:08.857 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.654883042Z level=info msg="Executing migration" id="add index api_key.key" 2024-09-16T09:26:08.857 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.655392546Z level=info msg="Migration successfully executed" id="add index api_key.key" duration=509.202µs 2024-09-16T09:26:08.857 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.655781279Z level=info msg="Executing migration" id="add index api_key.account_id_name" 2024-09-16T09:26:08.857 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.656337599Z level=info msg="Migration successfully executed" id="add index api_key.account_id_name" duration=555.993µs 2024-09-16T09:26:08.857 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.656716376Z level=info msg="Executing migration" id="drop index IDX_api_key_account_id - v1" 2024-09-16T09:26:08.857 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.657222939Z level=info msg="Migration successfully executed" id="drop index IDX_api_key_account_id - v1" duration=506.579µs 2024-09-16T09:26:08.857 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.657571437Z level=info msg="Executing migration" id="drop index UQE_api_key_key - v1" 2024-09-16T09:26:08.857 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.658099078Z level=info msg="Migration successfully executed" id="drop index UQE_api_key_key - v1" duration=527.524µs 2024-09-16T09:26:08.858 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.658457122Z level=info msg="Executing migration" id="drop index UQE_api_key_account_id_name - v1" 2024-09-16T09:26:08.858 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.658961207Z level=info msg="Migration successfully executed" id="drop index UQE_api_key_account_id_name - v1" duration=504.003µs 2024-09-16T09:26:08.858 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.659317713Z level=info msg="Executing migration" id="Rename table api_key to api_key_v1 - v1" 2024-09-16T09:26:08.858 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.6629823Z level=info msg="Migration successfully executed" id="Rename table api_key to api_key_v1 - v1" duration=3.664162ms 2024-09-16T09:26:08.858 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.663353465Z level=info msg="Executing migration" id="create api_key table v2" 2024-09-16T09:26:08.858 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.663869995Z level=info msg="Migration successfully executed" id="create api_key table v2" duration=517.133µs 2024-09-16T09:26:08.858 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.6642555Z level=info msg="Executing migration" id="create index IDX_api_key_org_id - v2" 2024-09-16T09:26:08.858 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.664777865Z level=info msg="Migration successfully executed" id="create index IDX_api_key_org_id - v2" duration=522.171µs 2024-09-16T09:26:08.858 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.665159062Z level=info msg="Executing migration" id="create index UQE_api_key_key - v2" 2024-09-16T09:26:08.858 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.665655751Z level=info msg="Migration successfully executed" id="create index UQE_api_key_key - v2" duration=496.676µs 2024-09-16T09:26:08.858 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.666009377Z level=info msg="Executing migration" id="create index UQE_api_key_org_id_name - v2" 2024-09-16T09:26:08.858 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.666527752Z level=info msg="Migration successfully executed" id="create index UQE_api_key_org_id_name - v2" duration=518.227µs 2024-09-16T09:26:08.861 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.66691512Z level=info msg="Executing migration" id="copy api_key v1 to v2" 2024-09-16T09:26:08.861 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.6671746Z level=info msg="Migration successfully executed" id="copy api_key v1 to v2" duration=259.386µs 2024-09-16T09:26:08.861 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.667555099Z level=info msg="Executing migration" id="Drop old table api_key_v1" 2024-09-16T09:26:08.861 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.667950286Z level=info msg="Migration successfully executed" id="Drop old table api_key_v1" duration=395.053µs 2024-09-16T09:26:08.861 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.66841024Z level=info msg="Executing migration" id="Update api_key table charset" 2024-09-16T09:26:08.861 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.668443139Z level=info msg="Migration successfully executed" id="Update api_key table charset" duration=33.492µs 2024-09-16T09:26:08.861 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.669290027Z level=info msg="Executing migration" id="Add expires to api_key table" 2024-09-16T09:26:08.861 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.670797101Z level=info msg="Migration successfully executed" id="Add expires to api_key table" duration=1.506798ms 2024-09-16T09:26:08.861 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.671171507Z level=info msg="Executing migration" id="Add service account foreign key" 2024-09-16T09:26:08.861 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.672670563Z level=info msg="Migration successfully executed" id="Add service account foreign key" duration=1.498808ms 2024-09-16T09:26:08.862 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.673042557Z level=info msg="Executing migration" id="set service account foreign key to nil if 0" 2024-09-16T09:26:08.862 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.673164551Z level=info msg="Migration successfully executed" id="set service account foreign key to nil if 0" duration=122.005µs 2024-09-16T09:26:08.862 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.673918158Z level=info msg="Executing migration" id="Add last_used_at to api_key table" 2024-09-16T09:26:08.862 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.675419913Z level=info msg="Migration successfully executed" id="Add last_used_at to api_key table" duration=1.501463ms 2024-09-16T09:26:08.862 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.676595607Z level=info msg="Executing migration" id="Add is_revoked column to api_key table" 2024-09-16T09:26:08.862 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.678175651Z level=info msg="Migration successfully executed" id="Add is_revoked column to api_key table" duration=1.579771ms 2024-09-16T09:26:08.862 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.67948193Z level=info msg="Executing migration" id="create dashboard_snapshot table v4" 2024-09-16T09:26:08.862 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.680103149Z level=info msg="Migration successfully executed" id="create dashboard_snapshot table v4" duration=623.492µs 2024-09-16T09:26:08.862 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.680494243Z level=info msg="Executing migration" id="drop table dashboard_snapshot_v4 #1" 2024-09-16T09:26:08.862 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.680886681Z level=info msg="Migration successfully executed" id="drop table dashboard_snapshot_v4 #1" duration=392.402µs 2024-09-16T09:26:08.862 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.683401644Z level=info msg="Executing migration" id="create dashboard_snapshot table v5 #2" 2024-09-16T09:26:08.862 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.6840919Z level=info msg="Migration successfully executed" id="create dashboard_snapshot table v5 #2" duration=690.09µs 2024-09-16T09:26:08.862 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.686111957Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_key - v5" 2024-09-16T09:26:08.863 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.686680011Z level=info msg="Migration successfully executed" id="create index UQE_dashboard_snapshot_key - v5" duration=568.813µs 2024-09-16T09:26:08.863 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.689066087Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_delete_key - v5" 2024-09-16T09:26:08.863 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.689621974Z level=info msg="Migration successfully executed" id="create index UQE_dashboard_snapshot_delete_key - v5" duration=555.633µs 2024-09-16T09:26:08.863 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.689994084Z level=info msg="Executing migration" id="create index IDX_dashboard_snapshot_user_id - v5" 2024-09-16T09:26:08.863 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.690518279Z level=info msg="Migration successfully executed" id="create index IDX_dashboard_snapshot_user_id - v5" duration=523.906µs 2024-09-16T09:26:08.863 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.692863741Z level=info msg="Executing migration" id="alter dashboard_snapshot to mediumtext v2" 2024-09-16T09:26:08.863 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.692908131Z level=info msg="Migration successfully executed" id="alter dashboard_snapshot to mediumtext v2" duration=44.677µs 2024-09-16T09:26:08.863 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.695716724Z level=info msg="Executing migration" id="Update dashboard_snapshot table charset" 2024-09-16T09:26:08.863 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.695732838Z level=info msg="Migration successfully executed" id="Update dashboard_snapshot table charset" duration=16.838µs 2024-09-16T09:26:08.863 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.698416635Z level=info msg="Executing migration" id="Add column external_delete_url to dashboard_snapshots table" 2024-09-16T09:26:08.863 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.700066293Z level=info msg="Migration successfully executed" id="Add column external_delete_url to dashboard_snapshots table" duration=1.649392ms 2024-09-16T09:26:08.863 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.700463795Z level=info msg="Executing migration" id="Add encrypted dashboard json column" 2024-09-16T09:26:08.863 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.702092462Z level=info msg="Migration successfully executed" id="Add encrypted dashboard json column" duration=1.628129ms 2024-09-16T09:26:08.864 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.703530305Z level=info msg="Executing migration" id="Change dashboard_encrypted column to MEDIUMBLOB" 2024-09-16T09:26:08.864 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.703576616Z level=info msg="Migration successfully executed" id="Change dashboard_encrypted column to MEDIUMBLOB" duration=47.111µs 2024-09-16T09:26:08.864 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.706441275Z level=info msg="Executing migration" id="create quota table v1" 2024-09-16T09:26:08.864 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.706942504Z level=info msg="Migration successfully executed" id="create quota table v1" duration=500.974µs 2024-09-16T09:26:08.864 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.709146484Z level=info msg="Executing migration" id="create index UQE_quota_org_id_user_id_target - v1" 2024-09-16T09:26:08.864 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.709703679Z level=info msg="Migration successfully executed" id="create index UQE_quota_org_id_user_id_target - v1" duration=556.991µs 2024-09-16T09:26:08.864 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.710084517Z level=info msg="Executing migration" id="Update quota table charset" 2024-09-16T09:26:08.864 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.710100659Z level=info msg="Migration successfully executed" id="Update quota table charset" duration=17.737µs 2024-09-16T09:26:08.864 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.713041509Z level=info msg="Executing migration" id="create plugin_setting table" 2024-09-16T09:26:08.864 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.713585054Z level=info msg="Migration successfully executed" id="create plugin_setting table" duration=543.09µs 2024-09-16T09:26:08.864 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.716091416Z level=info msg="Executing migration" id="create index UQE_plugin_setting_org_id_plugin_id - v1" 2024-09-16T09:26:08.864 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.716823937Z level=info msg="Migration successfully executed" id="create index UQE_plugin_setting_org_id_plugin_id - v1" duration=734.638µs 2024-09-16T09:26:08.864 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.718744191Z level=info msg="Executing migration" id="Add column plugin_version to plugin_settings" 2024-09-16T09:26:08.865 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.72043816Z level=info msg="Migration successfully executed" id="Add column plugin_version to plugin_settings" duration=1.692649ms 2024-09-16T09:26:08.865 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.720788274Z level=info msg="Executing migration" id="Update plugin_setting table charset" 2024-09-16T09:26:08.865 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.720803181Z level=info msg="Migration successfully executed" id="Update plugin_setting table charset" duration=15.579µs 2024-09-16T09:26:08.865 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.723658881Z level=info msg="Executing migration" id="create session table" 2024-09-16T09:26:08.865 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.724325408Z level=info msg="Migration successfully executed" id="create session table" duration=668.619µs 2024-09-16T09:26:08.865 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.726338228Z level=info msg="Executing migration" id="Drop old table playlist table" 2024-09-16T09:26:08.865 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.726410483Z level=info msg="Migration successfully executed" id="Drop old table playlist table" duration=72.41µs 2024-09-16T09:26:08.865 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.729009328Z level=info msg="Executing migration" id="Drop old table playlist_item table" 2024-09-16T09:26:08.865 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.729081162Z level=info msg="Migration successfully executed" id="Drop old table playlist_item table" duration=71.851µs 2024-09-16T09:26:08.865 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.729461154Z level=info msg="Executing migration" id="create playlist table v2" 2024-09-16T09:26:08.865 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.729954585Z level=info msg="Migration successfully executed" id="create playlist table v2" duration=493.197µs 2024-09-16T09:26:08.865 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.732141855Z level=info msg="Executing migration" id="create playlist item table v2" 2024-09-16T09:26:08.866 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.732629336Z level=info msg="Migration successfully executed" id="create playlist item table v2" duration=487.424µs 2024-09-16T09:26:08.866 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.734802183Z level=info msg="Executing migration" id="Update playlist table charset" 2024-09-16T09:26:08.866 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.734816421Z level=info msg="Migration successfully executed" id="Update playlist table charset" duration=14.846µs 2024-09-16T09:26:08.866 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.737629701Z level=info msg="Executing migration" id="Update playlist_item table charset" 2024-09-16T09:26:08.866 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.73764425Z level=info msg="Migration successfully executed" id="Update playlist_item table charset" duration=14.72µs 2024-09-16T09:26:08.866 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.738057885Z level=info msg="Executing migration" id="Add playlist column created_at" 2024-09-16T09:26:08.866 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.739834183Z level=info msg="Migration successfully executed" id="Add playlist column created_at" duration=1.775902ms 2024-09-16T09:26:08.866 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.740838442Z level=info msg="Executing migration" id="Add playlist column updated_at" 2024-09-16T09:26:08.866 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.742640148Z level=info msg="Migration successfully executed" id="Add playlist column updated_at" duration=1.802041ms 2024-09-16T09:26:08.866 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.743536592Z level=info msg="Executing migration" id="drop preferences table v2" 2024-09-16T09:26:08.866 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.743652837Z level=info msg="Migration successfully executed" id="drop preferences table v2" duration=118.81µs 2024-09-16T09:26:08.866 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.746163581Z level=info msg="Executing migration" id="drop preferences table v3" 2024-09-16T09:26:08.866 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.746218075Z level=info msg="Migration successfully executed" id="drop preferences table v3" duration=54.492µs 2024-09-16T09:26:08.867 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.746587328Z level=info msg="Executing migration" id="create preferences table v3" 2024-09-16T09:26:08.867 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.747088766Z level=info msg="Migration successfully executed" id="create preferences table v3" duration=501.238µs 2024-09-16T09:26:08.867 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.749258198Z level=info msg="Executing migration" id="Update preferences table charset" 2024-09-16T09:26:08.867 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.749273226Z level=info msg="Migration successfully executed" id="Update preferences table charset" duration=15.162µs 2024-09-16T09:26:08.867 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.752040637Z level=info msg="Executing migration" id="Add column team_id in preferences" 2024-09-16T09:26:08.867 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.753891038Z level=info msg="Migration successfully executed" id="Add column team_id in preferences" duration=1.8524ms 2024-09-16T09:26:08.867 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.754515382Z level=info msg="Executing migration" id="Update team_id column values in preferences" 2024-09-16T09:26:08.867 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.754639816Z level=info msg="Migration successfully executed" id="Update team_id column values in preferences" duration=109.377µs 2024-09-16T09:26:08.867 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.755020507Z level=info msg="Executing migration" id="Add column week_start in preferences" 2024-09-16T09:26:08.867 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.756777945Z level=info msg="Migration successfully executed" id="Add column week_start in preferences" duration=1.7569ms 2024-09-16T09:26:08.867 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.75713973Z level=info msg="Executing migration" id="Add column preferences.json_data" 2024-09-16T09:26:08.867 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.75894829Z level=info msg="Migration successfully executed" id="Add column preferences.json_data" duration=1.808312ms 2024-09-16T09:26:08.867 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.759290475Z level=info msg="Executing migration" id="alter preferences.json_data to mediumtext v1" 2024-09-16T09:26:08.868 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.759328106Z level=info msg="Migration successfully executed" id="alter preferences.json_data to mediumtext v1" duration=38.011µs 2024-09-16T09:26:08.868 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.76021731Z level=info msg="Executing migration" id="Add preferences index org_id" 2024-09-16T09:26:08.868 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.760860013Z level=info msg="Migration successfully executed" id="Add preferences index org_id" duration=643.128µs 2024-09-16T09:26:08.868 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.761227694Z level=info msg="Executing migration" id="Add preferences index user_id" 2024-09-16T09:26:08.868 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.761778728Z level=info msg="Migration successfully executed" id="Add preferences index user_id" duration=551.958µs 2024-09-16T09:26:08.868 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.762136594Z level=info msg="Executing migration" id="create alert table v1" 2024-09-16T09:26:08.868 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.762853985Z level=info msg="Migration successfully executed" id="create alert table v1" duration=717.215µs 2024-09-16T09:26:08.868 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.763215444Z level=info msg="Executing migration" id="add index alert org_id & id " 2024-09-16T09:26:08.868 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.763824082Z level=info msg="Migration successfully executed" id="add index alert org_id & id " duration=608.24µs 2024-09-16T09:26:08.868 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.76418657Z level=info msg="Executing migration" id="add index alert state" 2024-09-16T09:26:08.868 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.764695456Z level=info msg="Migration successfully executed" id="add index alert state" duration=508.707µs 2024-09-16T09:26:08.868 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.765021025Z level=info msg="Executing migration" id="add index alert dashboard_id" 2024-09-16T09:26:08.868 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.765535538Z level=info msg="Migration successfully executed" id="add index alert dashboard_id" duration=514.201µs 2024-09-16T09:26:08.868 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.765869316Z level=info msg="Executing migration" id="Create alert_rule_tag table v1" 2024-09-16T09:26:08.869 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.766313623Z level=info msg="Migration successfully executed" id="Create alert_rule_tag table v1" duration=443.996µs 2024-09-16T09:26:08.869 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.766660742Z level=info msg="Executing migration" id="Add unique index alert_rule_tag.alert_id_tag_id" 2024-09-16T09:26:08.869 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.767219003Z level=info msg="Migration successfully executed" id="Add unique index alert_rule_tag.alert_id_tag_id" duration=559.133µs 2024-09-16T09:26:08.869 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.767563097Z level=info msg="Executing migration" id="drop index UQE_alert_rule_tag_alert_id_tag_id - v1" 2024-09-16T09:26:08.869 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.768071172Z level=info msg="Migration successfully executed" id="drop index UQE_alert_rule_tag_alert_id_tag_id - v1" duration=507.917µs 2024-09-16T09:26:08.869 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.768393145Z level=info msg="Executing migration" id="Rename table alert_rule_tag to alert_rule_tag_v1 - v1" 2024-09-16T09:26:08.869 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.774087495Z level=info msg="Migration successfully executed" id="Rename table alert_rule_tag to alert_rule_tag_v1 - v1" duration=5.69399ms 2024-09-16T09:26:08.869 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.774420862Z level=info msg="Executing migration" id="Create alert_rule_tag table v2" 2024-09-16T09:26:08.869 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.774864064Z level=info msg="Migration successfully executed" id="Create alert_rule_tag table v2" duration=441.473µs 2024-09-16T09:26:08.869 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.77520409Z level=info msg="Executing migration" id="create index UQE_alert_rule_tag_alert_id_tag_id - Add unique index alert_rule_tag.alert_id_tag_id V2" 2024-09-16T09:26:08.869 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.775713767Z level=info msg="Migration successfully executed" id="create index UQE_alert_rule_tag_alert_id_tag_id - Add unique index alert_rule_tag.alert_id_tag_id V2" duration=509.299µs 2024-09-16T09:26:08.870 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.776039294Z level=info msg="Executing migration" id="copy alert_rule_tag v1 to v2" 2024-09-16T09:26:08.870 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.776225103Z level=info msg="Migration successfully executed" id="copy alert_rule_tag v1 to v2" duration=185.601µs 2024-09-16T09:26:08.870 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.776533167Z level=info msg="Executing migration" id="drop table alert_rule_tag_v1" 2024-09-16T09:26:08.870 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.776869652Z level=info msg="Migration successfully executed" id="drop table alert_rule_tag_v1" duration=336.224µs 2024-09-16T09:26:08.870 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.777373142Z level=info msg="Executing migration" id="create alert_notification table v1" 2024-09-16T09:26:08.870 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.777862777Z level=info msg="Migration successfully executed" id="create alert_notification table v1" duration=489.166µs 2024-09-16T09:26:08.870 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.778226595Z level=info msg="Executing migration" id="Add column is_default" 2024-09-16T09:26:08.870 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.780270316Z level=info msg="Migration successfully executed" id="Add column is_default" duration=2.043454ms 2024-09-16T09:26:08.870 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.780613011Z level=info msg="Executing migration" id="Add column frequency" 2024-09-16T09:26:08.870 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.782788246Z level=info msg="Migration successfully executed" id="Add column frequency" duration=2.175859ms 2024-09-16T09:26:08.870 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.783163848Z level=info msg="Executing migration" id="Add column send_reminder" 2024-09-16T09:26:08.871 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.785377006Z level=info msg="Migration successfully executed" id="Add column send_reminder" duration=2.212967ms 2024-09-16T09:26:08.871 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.785719744Z level=info msg="Executing migration" id="Add column disable_resolve_message" 2024-09-16T09:26:08.871 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.787720444Z level=info msg="Migration successfully executed" id="Add column disable_resolve_message" duration=2.000407ms 2024-09-16T09:26:08.871 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.788067315Z level=info msg="Executing migration" id="add index alert_notification org_id & name" 2024-09-16T09:26:08.871 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.788579814Z level=info msg="Migration successfully executed" id="add index alert_notification org_id & name" duration=512.36µs 2024-09-16T09:26:08.871 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.788931376Z level=info msg="Executing migration" id="Update alert table charset" 2024-09-16T09:26:08.871 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.788946438Z level=info msg="Migration successfully executed" id="Update alert table charset" duration=15.632µs 2024-09-16T09:26:08.871 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.78931984Z level=info msg="Executing migration" id="Update alert_notification table charset" 2024-09-16T09:26:08.871 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.789334489Z level=info msg="Migration successfully executed" id="Update alert_notification table charset" duration=15.194µs 2024-09-16T09:26:08.871 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.790195418Z level=info msg="Executing migration" id="create notification_journal table v1" 2024-09-16T09:26:08.871 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.790681549Z level=info msg="Migration successfully executed" id="create notification_journal table v1" duration=485.899µs 2024-09-16T09:26:08.871 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.791066401Z level=info msg="Executing migration" id="add index notification_journal org_id & alert_id & notifier_id" 2024-09-16T09:26:08.871 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.791641781Z level=info msg="Migration successfully executed" id="add index notification_journal org_id & alert_id & notifier_id" duration=574.112µs 2024-09-16T09:26:08.872 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.792014975Z level=info msg="Executing migration" id="drop alert_notification_journal" 2024-09-16T09:26:08.872 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.792559603Z level=info msg="Migration successfully executed" id="drop alert_notification_journal" duration=543.193µs 2024-09-16T09:26:08.872 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.792948132Z level=info msg="Executing migration" id="create alert_notification_state table v1" 2024-09-16T09:26:08.872 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.79348462Z level=info msg="Migration successfully executed" id="create alert_notification_state table v1" duration=536.188µs 2024-09-16T09:26:08.872 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.793849801Z level=info msg="Executing migration" id="add index alert_notification_state org_id & alert_id & notifier_id" 2024-09-16T09:26:08.872 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.794399865Z level=info msg="Migration successfully executed" id="add index alert_notification_state org_id & alert_id & notifier_id" duration=549.739µs 2024-09-16T09:26:08.872 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.794755578Z level=info msg="Executing migration" id="Add for to alert table" 2024-09-16T09:26:08.872 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.796938039Z level=info msg="Migration successfully executed" id="Add for to alert table" duration=2.181889ms 2024-09-16T09:26:08.873 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.797300803Z level=info msg="Executing migration" id="Add column uid in alert_notification" 2024-09-16T09:26:08.873 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.799498158Z level=info msg="Migration successfully executed" id="Add column uid in alert_notification" duration=2.197848ms 2024-09-16T09:26:08.873 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.799870227Z level=info msg="Executing migration" id="Update uid column values in alert_notification" 2024-09-16T09:26:08.873 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.800015493Z level=info msg="Migration successfully executed" id="Update uid column values in alert_notification" duration=145.288µs 2024-09-16T09:26:08.873 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.800448158Z level=info msg="Executing migration" id="Add unique index alert_notification_org_id_uid" 2024-09-16T09:26:08.873 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.800985357Z level=info msg="Migration successfully executed" id="Add unique index alert_notification_org_id_uid" duration=535.508µs 2024-09-16T09:26:08.873 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.801353258Z level=info msg="Executing migration" id="Remove unique index org_id_name" 2024-09-16T09:26:08.873 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.801896745Z level=info msg="Migration successfully executed" id="Remove unique index org_id_name" duration=543.525µs 2024-09-16T09:26:08.873 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.802281834Z level=info msg="Executing migration" id="Add column secure_settings in alert_notification" 2024-09-16T09:26:08.873 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.804445733Z level=info msg="Migration successfully executed" id="Add column secure_settings in alert_notification" duration=2.16348ms 2024-09-16T09:26:08.873 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.805005492Z level=info msg="Executing migration" id="alter alert.settings to mediumtext" 2024-09-16T09:26:08.873 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.805050634Z level=info msg="Migration successfully executed" id="alter alert.settings to mediumtext" duration=44.904µs 2024-09-16T09:26:08.873 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.805494717Z level=info msg="Executing migration" id="Add non-unique index alert_notification_state_alert_id" 2024-09-16T09:26:08.874 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.806025372Z level=info msg="Migration successfully executed" id="Add non-unique index alert_notification_state_alert_id" duration=530.378µs 2024-09-16T09:26:08.874 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.808395277Z level=info msg="Executing migration" id="Add non-unique index alert_rule_tag_alert_id" 2024-09-16T09:26:08.874 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.809036566Z level=info msg="Migration successfully executed" id="Add non-unique index alert_rule_tag_alert_id" duration=641.059µs 2024-09-16T09:26:08.874 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.81125725Z level=info msg="Executing migration" id="Drop old annotation table v4" 2024-09-16T09:26:08.874 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.811326269Z level=info msg="Migration successfully executed" id="Drop old annotation table v4" duration=68.323µs 2024-09-16T09:26:08.874 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.813991082Z level=info msg="Executing migration" id="create annotation table v5" 2024-09-16T09:26:08.874 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.814597323Z level=info msg="Migration successfully executed" id="create annotation table v5" duration=606.01µs 2024-09-16T09:26:08.874 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.815081511Z level=info msg="Executing migration" id="add index annotation 0 v3" 2024-09-16T09:26:08.874 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.81569881Z level=info msg="Migration successfully executed" id="add index annotation 0 v3" duration=617.623µs 2024-09-16T09:26:08.874 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.817599749Z level=info msg="Executing migration" id="add index annotation 1 v3" 2024-09-16T09:26:08.874 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.818121121Z level=info msg="Migration successfully executed" id="add index annotation 1 v3" duration=521.123µs 2024-09-16T09:26:08.874 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.820615234Z level=info msg="Executing migration" id="add index annotation 2 v3" 2024-09-16T09:26:08.874 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.821227284Z level=info msg="Migration successfully executed" id="add index annotation 2 v3" duration=613.206µs 2024-09-16T09:26:08.875 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.823416479Z level=info msg="Executing migration" id="add index annotation 3 v3" 2024-09-16T09:26:08.875 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.824015634Z level=info msg="Migration successfully executed" id="add index annotation 3 v3" duration=598.807µs 2024-09-16T09:26:08.875 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.824366855Z level=info msg="Executing migration" id="add index annotation 4 v3" 2024-09-16T09:26:08.875 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.824956043Z level=info msg="Migration successfully executed" id="add index annotation 4 v3" duration=588.9µs 2024-09-16T09:26:08.875 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.827073034Z level=info msg="Executing migration" id="Update annotation table charset" 2024-09-16T09:26:08.875 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.827088026Z level=info msg="Migration successfully executed" id="Update annotation table charset" duration=15.632µs 2024-09-16T09:26:08.875 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.830007231Z level=info msg="Executing migration" id="Add column region_id to annotation table" 2024-09-16T09:26:08.875 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.832425943Z level=info msg="Migration successfully executed" id="Add column region_id to annotation table" duration=2.418351ms 2024-09-16T09:26:08.875 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.832926429Z level=info msg="Executing migration" id="Drop category_id index" 2024-09-16T09:26:08.875 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.833448256Z level=info msg="Migration successfully executed" id="Drop category_id index" duration=521.811µs 2024-09-16T09:26:08.875 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.833790264Z level=info msg="Executing migration" id="Add column tags to annotation table" 2024-09-16T09:26:08.875 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.835983738Z level=info msg="Migration successfully executed" id="Add column tags to annotation table" duration=2.193041ms 2024-09-16T09:26:08.875 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.836480639Z level=info msg="Executing migration" id="Create annotation_tag table v2" 2024-09-16T09:26:08.875 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.836946344Z level=info msg="Migration successfully executed" id="Create annotation_tag table v2" duration=465.47µs 2024-09-16T09:26:08.876 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.839258795Z level=info msg="Executing migration" id="Add unique index annotation_tag.annotation_id_tag_id" 2024-09-16T09:26:08.876 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.839810569Z level=info msg="Migration successfully executed" id="Add unique index annotation_tag.annotation_id_tag_id" duration=551.556µs 2024-09-16T09:26:08.876 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.842077821Z level=info msg="Executing migration" id="drop index UQE_annotation_tag_annotation_id_tag_id - v2" 2024-09-16T09:26:08.876 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.842618553Z level=info msg="Migration successfully executed" id="drop index UQE_annotation_tag_annotation_id_tag_id - v2" duration=542.719µs 2024-09-16T09:26:08.876 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.842963596Z level=info msg="Executing migration" id="Rename table annotation_tag to annotation_tag_v2 - v2" 2024-09-16T09:26:08.876 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.849351009Z level=info msg="Migration successfully executed" id="Rename table annotation_tag to annotation_tag_v2 - v2" duration=6.387188ms 2024-09-16T09:26:08.876 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.849746927Z level=info msg="Executing migration" id="Create annotation_tag table v3" 2024-09-16T09:26:08.876 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.850288207Z level=info msg="Migration successfully executed" id="Create annotation_tag table v3" duration=540.944µs 2024-09-16T09:26:08.876 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.852865425Z level=info msg="Executing migration" id="create index UQE_annotation_tag_annotation_id_tag_id - Add unique index annotation_tag.annotation_id_tag_id V3" 2024-09-16T09:26:08.876 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.853683191Z level=info msg="Migration successfully executed" id="create index UQE_annotation_tag_annotation_id_tag_id - Add unique index annotation_tag.annotation_id_tag_id V3" duration=816.623µs 2024-09-16T09:26:08.876 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.855099097Z level=info msg="Executing migration" id="copy annotation_tag v2 to v3" 2024-09-16T09:26:08.876 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.855296199Z level=info msg="Migration successfully executed" id="copy annotation_tag v2 to v3" duration=197.015µs 2024-09-16T09:26:08.876 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.855620365Z level=info msg="Executing migration" id="drop table annotation_tag_v2" 2024-09-16T09:26:08.877 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.856003148Z level=info msg="Migration successfully executed" id="drop table annotation_tag_v2" duration=382.539µs 2024-09-16T09:26:08.877 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:08.878 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:08.878 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:26:08.906 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:26:08.906 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:26:08.907 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:26:08.907 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:26:08.923 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:26:08.923 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:26:08.945 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:26:08.945 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:26:08.962 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:26:08.962 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:26:08.977 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:26:08.978 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:26:08.978 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:26:08.994 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:26:08.994 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:26:08.999 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:08.999 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:09.000 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:26:09.013 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:26:09.013 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:26:09.101 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:26:09.104 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.85816451Z level=info msg="Executing migration" id="Update alert annotations and set TEXT to empty" 2024-09-16T09:26:09.104 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.858315548Z level=info msg="Migration successfully executed" id="Update alert annotations and set TEXT to empty" duration=151.12µs 2024-09-16T09:26:09.105 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.860552614Z level=info msg="Executing migration" id="Add created time to annotation table" 2024-09-16T09:26:09.105 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.862946602Z level=info msg="Migration successfully executed" id="Add created time to annotation table" duration=2.391874ms 2024-09-16T09:26:09.105 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.863302103Z level=info msg="Executing migration" id="Add updated time to annotation table" 2024-09-16T09:26:09.105 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.865611507Z level=info msg="Migration successfully executed" id="Add updated time to annotation table" duration=2.309109ms 2024-09-16T09:26:09.105 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.866019665Z level=info msg="Executing migration" id="Add index for created in annotation table" 2024-09-16T09:26:09.105 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.866573786Z level=info msg="Migration successfully executed" id="Add index for created in annotation table" duration=553.844µs 2024-09-16T09:26:09.105 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.868584474Z level=info msg="Executing migration" id="Add index for updated in annotation table" 2024-09-16T09:26:09.105 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.869141381Z level=info msg="Migration successfully executed" id="Add index for updated in annotation table" duration=556.607µs 2024-09-16T09:26:09.105 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.871134065Z level=info msg="Executing migration" id="Convert existing annotations from seconds to milliseconds" 2024-09-16T09:26:09.106 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.871304156Z level=info msg="Migration successfully executed" id="Convert existing annotations from seconds to milliseconds" duration=169.379µs 2024-09-16T09:26:09.106 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.873564816Z level=info msg="Executing migration" id="Add epoch_end column" 2024-09-16T09:26:09.106 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.875999123Z level=info msg="Migration successfully executed" id="Add epoch_end column" duration=2.434014ms 2024-09-16T09:26:09.106 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.876370792Z level=info msg="Executing migration" id="Add index for epoch_end" 2024-09-16T09:26:09.106 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.876933764Z level=info msg="Migration successfully executed" id="Add index for epoch_end" duration=562.645µs 2024-09-16T09:26:09.106 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.877312899Z level=info msg="Executing migration" id="Make epoch_end the same as epoch" 2024-09-16T09:26:09.106 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.877453174Z level=info msg="Migration successfully executed" id="Make epoch_end the same as epoch" duration=140.387µs 2024-09-16T09:26:09.106 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.878081675Z level=info msg="Executing migration" id="Move region to single row" 2024-09-16T09:26:09.106 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.878376238Z level=info msg="Migration successfully executed" id="Move region to single row" duration=294.379µs 2024-09-16T09:26:09.106 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.878869472Z level=info msg="Executing migration" id="Remove index org_id_epoch from annotation table" 2024-09-16T09:26:09.106 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.879445774Z level=info msg="Migration successfully executed" id="Remove index org_id_epoch from annotation table" duration=578.265µs 2024-09-16T09:26:09.107 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.879773483Z level=info msg="Executing migration" id="Remove index org_id_dashboard_id_panel_id_epoch from annotation table" 2024-09-16T09:26:09.107 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.880294978Z level=info msg="Migration successfully executed" id="Remove index org_id_dashboard_id_panel_id_epoch from annotation table" duration=519.468µs 2024-09-16T09:26:09.107 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.880629163Z level=info msg="Executing migration" id="Add index for org_id_dashboard_id_epoch_end_epoch on annotation table" 2024-09-16T09:26:09.107 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.881165065Z level=info msg="Migration successfully executed" id="Add index for org_id_dashboard_id_epoch_end_epoch on annotation table" duration=535.768µs 2024-09-16T09:26:09.107 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.881492113Z level=info msg="Executing migration" id="Add index for org_id_epoch_end_epoch on annotation table" 2024-09-16T09:26:09.107 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.882023837Z level=info msg="Migration successfully executed" id="Add index for org_id_epoch_end_epoch on annotation table" duration=531.357µs 2024-09-16T09:26:09.108 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.882350748Z level=info msg="Executing migration" id="Remove index org_id_epoch_epoch_end from annotation table" 2024-09-16T09:26:09.109 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.882867551Z level=info msg="Migration successfully executed" id="Remove index org_id_epoch_epoch_end from annotation table" duration=515.441µs 2024-09-16T09:26:09.109 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.88320991Z level=info msg="Executing migration" id="Add index for alert_id on annotation table" 2024-09-16T09:26:09.109 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.883719471Z level=info msg="Migration successfully executed" id="Add index for alert_id on annotation table" duration=507.481µs 2024-09-16T09:26:09.109 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.884043746Z level=info msg="Executing migration" id="Increase tags column to length 4096" 2024-09-16T09:26:09.109 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.884081594Z level=info msg="Migration successfully executed" id="Increase tags column to length 4096" duration=38.09µs 2024-09-16T09:26:09.109 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.884762711Z level=info msg="Executing migration" id="create test_data table" 2024-09-16T09:26:09.109 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.885296211Z level=info msg="Migration successfully executed" id="create test_data table" duration=533.262µs 2024-09-16T09:26:09.109 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.885638175Z level=info msg="Executing migration" id="create dashboard_version table v1" 2024-09-16T09:26:09.109 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.886159176Z level=info msg="Migration successfully executed" id="create dashboard_version table v1" duration=520.869µs 2024-09-16T09:26:09.109 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.886502176Z level=info msg="Executing migration" id="add index dashboard_version.dashboard_id" 2024-09-16T09:26:09.110 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.88702454Z level=info msg="Migration successfully executed" id="add index dashboard_version.dashboard_id" duration=522.237µs 2024-09-16T09:26:09.110 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.887365513Z level=info msg="Executing migration" id="add unique index dashboard_version.dashboard_id and dashboard_version.version" 2024-09-16T09:26:09.110 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.887924333Z level=info msg="Migration successfully executed" id="add unique index dashboard_version.dashboard_id and dashboard_version.version" duration=558.519µs 2024-09-16T09:26:09.110 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.888264197Z level=info msg="Executing migration" id="Set dashboard version to 1 where 0" 2024-09-16T09:26:09.110 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.888373754Z level=info msg="Migration successfully executed" id="Set dashboard version to 1 where 0" duration=109.535µs 2024-09-16T09:26:09.110 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.88897437Z level=info msg="Executing migration" id="save existing dashboard data in dashboard_version table v1" 2024-09-16T09:26:09.110 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.889202669Z level=info msg="Migration successfully executed" id="save existing dashboard data in dashboard_version table v1" duration=228.204µs 2024-09-16T09:26:09.110 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.889521436Z level=info msg="Executing migration" id="alter dashboard_version.data to mediumtext v1" 2024-09-16T09:26:09.110 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.889558896Z level=info msg="Migration successfully executed" id="alter dashboard_version.data to mediumtext v1" duration=38.509µs 2024-09-16T09:26:09.110 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.890386615Z level=info msg="Executing migration" id="create team table" 2024-09-16T09:26:09.110 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.891042477Z level=info msg="Migration successfully executed" id="create team table" duration=657.191µs 2024-09-16T09:26:09.110 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.891401056Z level=info msg="Executing migration" id="add index team.org_id" 2024-09-16T09:26:09.110 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.89201519Z level=info msg="Migration successfully executed" id="add index team.org_id" duration=613.917µs 2024-09-16T09:26:09.112 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.892369178Z level=info msg="Executing migration" id="add unique index team_org_id_name" 2024-09-16T09:26:09.112 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.892918718Z level=info msg="Migration successfully executed" id="add unique index team_org_id_name" duration=549.33µs 2024-09-16T09:26:09.112 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.893273162Z level=info msg="Executing migration" id="Add column uid in team" 2024-09-16T09:26:09.112 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.896135315Z level=info msg="Migration successfully executed" id="Add column uid in team" duration=2.862345ms 2024-09-16T09:26:09.112 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.896481939Z level=info msg="Executing migration" id="Update uid column values in team" 2024-09-16T09:26:09.112 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.896595927Z level=info msg="Migration successfully executed" id="Update uid column values in team" duration=114.139µs 2024-09-16T09:26:09.113 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.897261509Z level=info msg="Executing migration" id="Add unique index team_org_id_uid" 2024-09-16T09:26:09.113 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.897819351Z level=info msg="Migration successfully executed" id="Add unique index team_org_id_uid" duration=557.554µs 2024-09-16T09:26:09.113 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.898177904Z level=info msg="Executing migration" id="create team member table" 2024-09-16T09:26:09.113 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.89866704Z level=info msg="Migration successfully executed" id="create team member table" duration=488.89µs 2024-09-16T09:26:09.113 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.89900794Z level=info msg="Executing migration" id="add index team_member.org_id" 2024-09-16T09:26:09.113 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.899540444Z level=info msg="Migration successfully executed" id="add index team_member.org_id" duration=532.16µs 2024-09-16T09:26:09.113 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.89987839Z level=info msg="Executing migration" id="add unique index team_member_org_id_team_id_user_id" 2024-09-16T09:26:09.113 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.900442437Z level=info msg="Migration successfully executed" id="add unique index team_member_org_id_team_id_user_id" duration=563.038µs 2024-09-16T09:26:09.113 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.900786078Z level=info msg="Executing migration" id="add index team_member.team_id" 2024-09-16T09:26:09.113 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.901323193Z level=info msg="Migration successfully executed" id="add index team_member.team_id" duration=536.966µs 2024-09-16T09:26:09.114 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.901667067Z level=info msg="Executing migration" id="Add column email to team table" 2024-09-16T09:26:09.114 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.904468646Z level=info msg="Migration successfully executed" id="Add column email to team table" duration=2.801198ms 2024-09-16T09:26:09.114 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.90480293Z level=info msg="Executing migration" id="Add column external to team_member table" 2024-09-16T09:26:09.114 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.907600701Z level=info msg="Migration successfully executed" id="Add column external to team_member table" duration=2.797645ms 2024-09-16T09:26:09.114 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.907948488Z level=info msg="Executing migration" id="Add column permission to team_member table" 2024-09-16T09:26:09.114 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.910577104Z level=info msg="Migration successfully executed" id="Add column permission to team_member table" duration=2.628389ms 2024-09-16T09:26:09.114 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.910944674Z level=info msg="Executing migration" id="create dashboard acl table" 2024-09-16T09:26:09.114 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.911544073Z level=info msg="Migration successfully executed" id="create dashboard acl table" duration=599.34µs 2024-09-16T09:26:09.114 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.911927849Z level=info msg="Executing migration" id="add index dashboard_acl_dashboard_id" 2024-09-16T09:26:09.114 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.912482978Z level=info msg="Migration successfully executed" id="add index dashboard_acl_dashboard_id" duration=556.02µs 2024-09-16T09:26:09.114 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.91281325Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_user_id" 2024-09-16T09:26:09.114 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.913436759Z level=info msg="Migration successfully executed" id="add unique index dashboard_acl_dashboard_id_user_id" duration=623.281µs 2024-09-16T09:26:09.114 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.913817597Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_team_id" 2024-09-16T09:26:09.114 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.914391735Z level=info msg="Migration successfully executed" id="add unique index dashboard_acl_dashboard_id_team_id" duration=573.898µs 2024-09-16T09:26:09.115 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.914718128Z level=info msg="Executing migration" id="add index dashboard_acl_user_id" 2024-09-16T09:26:09.115 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.915252238Z level=info msg="Migration successfully executed" id="add index dashboard_acl_user_id" duration=533.854µs 2024-09-16T09:26:09.115 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.915587573Z level=info msg="Executing migration" id="add index dashboard_acl_team_id" 2024-09-16T09:26:09.115 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.916139801Z level=info msg="Migration successfully executed" id="add index dashboard_acl_team_id" duration=551.983µs 2024-09-16T09:26:09.115 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.91647266Z level=info msg="Executing migration" id="add index dashboard_acl_org_id_role" 2024-09-16T09:26:09.115 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.917044055Z level=info msg="Migration successfully executed" id="add index dashboard_acl_org_id_role" duration=571.416µs 2024-09-16T09:26:09.115 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.919443776Z level=info msg="Executing migration" id="add index dashboard_permission" 2024-09-16T09:26:09.115 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.920196802Z level=info msg="Migration successfully executed" id="add index dashboard_permission" duration=755.82µs 2024-09-16T09:26:09.115 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.921797646Z level=info msg="Executing migration" id="save default acl rules in dashboard_acl table" 2024-09-16T09:26:09.115 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.922148147Z level=info msg="Migration successfully executed" id="save default acl rules in dashboard_acl table" duration=350.312µs 2024-09-16T09:26:09.115 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.92461015Z level=info msg="Executing migration" id="delete acl rules for deleted dashboards and folders" 2024-09-16T09:26:09.116 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.924798602Z level=info msg="Migration successfully executed" id="delete acl rules for deleted dashboards and folders" duration=192.306µs 2024-09-16T09:26:09.116 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.927311719Z level=info msg="Executing migration" id="create tag table" 2024-09-16T09:26:09.116 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.927813912Z level=info msg="Migration successfully executed" id="create tag table" duration=501.979µs 2024-09-16T09:26:09.116 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.929777572Z level=info msg="Executing migration" id="add index tag.key_value" 2024-09-16T09:26:09.116 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.9303704Z level=info msg="Migration successfully executed" id="add index tag.key_value" duration=591.829µs 2024-09-16T09:26:09.116 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.932637688Z level=info msg="Executing migration" id="create login attempt table" 2024-09-16T09:26:09.116 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.933129667Z level=info msg="Migration successfully executed" id="create login attempt table" duration=491.564µs 2024-09-16T09:26:09.116 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.935223136Z level=info msg="Executing migration" id="add index login_attempt.username" 2024-09-16T09:26:09.116 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.93580785Z level=info msg="Migration successfully executed" id="add index login_attempt.username" duration=584.471µs 2024-09-16T09:26:09.116 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.938032469Z level=info msg="Executing migration" id="drop index IDX_login_attempt_username - v1" 2024-09-16T09:26:09.116 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.93861074Z level=info msg="Migration successfully executed" id="drop index IDX_login_attempt_username - v1" duration=579.28µs 2024-09-16T09:26:09.116 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.940782334Z level=info msg="Executing migration" id="Rename table login_attempt to login_attempt_tmp_qwerty - v1" 2024-09-16T09:26:09.117 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.948838834Z level=info msg="Migration successfully executed" id="Rename table login_attempt to login_attempt_tmp_qwerty - v1" duration=8.056125ms 2024-09-16T09:26:09.117 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.949217547Z level=info msg="Executing migration" id="create login_attempt v2" 2024-09-16T09:26:09.117 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.94969802Z level=info msg="Migration successfully executed" id="create login_attempt v2" duration=480.536µs 2024-09-16T09:26:09.117 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.952140283Z level=info msg="Executing migration" id="create index IDX_login_attempt_username - v2" 2024-09-16T09:26:09.117 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.952697734Z level=info msg="Migration successfully executed" id="create index IDX_login_attempt_username - v2" duration=557.613µs 2024-09-16T09:26:09.117 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.954808578Z level=info msg="Executing migration" id="copy login_attempt v1 to v2" 2024-09-16T09:26:09.117 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.955043013Z level=info msg="Migration successfully executed" id="copy login_attempt v1 to v2" duration=235.579µs 2024-09-16T09:26:09.117 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.957678227Z level=info msg="Executing migration" id="drop login_attempt_tmp_qwerty" 2024-09-16T09:26:09.117 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.95809444Z level=info msg="Migration successfully executed" id="drop login_attempt_tmp_qwerty" duration=414.537µs 2024-09-16T09:26:09.117 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.960465939Z level=info msg="Executing migration" id="create user auth table" 2024-09-16T09:26:09.120 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.960945237Z level=info msg="Migration successfully executed" id="create user auth table" duration=479.037µs 2024-09-16T09:26:09.120 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.963062959Z level=info msg="Executing migration" id="create index IDX_user_auth_auth_module_auth_id - v1" 2024-09-16T09:26:09.120 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.963646277Z level=info msg="Migration successfully executed" id="create index IDX_user_auth_auth_module_auth_id - v1" duration=583.102µs 2024-09-16T09:26:09.120 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.965789617Z level=info msg="Executing migration" id="alter user_auth.auth_id to length 190" 2024-09-16T09:26:09.121 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.965854338Z level=info msg="Migration successfully executed" id="alter user_auth.auth_id to length 190" duration=64.852µs 2024-09-16T09:26:09.121 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.968695853Z level=info msg="Executing migration" id="Add OAuth access token to user_auth" 2024-09-16T09:26:09.121 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.971712718Z level=info msg="Migration successfully executed" id="Add OAuth access token to user_auth" duration=3.016708ms 2024-09-16T09:26:09.121 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.972071999Z level=info msg="Executing migration" id="Add OAuth refresh token to user_auth" 2024-09-16T09:26:09.122 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.974998077Z level=info msg="Migration successfully executed" id="Add OAuth refresh token to user_auth" duration=2.925679ms 2024-09-16T09:26:09.122 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.975348463Z level=info msg="Executing migration" id="Add OAuth token type to user_auth" 2024-09-16T09:26:09.122 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.978327898Z level=info msg="Migration successfully executed" id="Add OAuth token type to user_auth" duration=2.978929ms 2024-09-16T09:26:09.122 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.978659958Z level=info msg="Executing migration" id="Add OAuth expiry to user_auth" 2024-09-16T09:26:09.122 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.981619255Z level=info msg="Migration successfully executed" id="Add OAuth expiry to user_auth" duration=2.958869ms 2024-09-16T09:26:09.122 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.981962165Z level=info msg="Executing migration" id="Add index to user_id column in user_auth" 2024-09-16T09:26:09.122 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.982508097Z level=info msg="Migration successfully executed" id="Add index to user_id column in user_auth" duration=545.649µs 2024-09-16T09:26:09.122 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.982838243Z level=info msg="Executing migration" id="Add OAuth ID token to user_auth" 2024-09-16T09:26:09.122 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.985759976Z level=info msg="Migration successfully executed" id="Add OAuth ID token to user_auth" duration=2.922533ms 2024-09-16T09:26:09.122 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.986114766Z level=info msg="Executing migration" id="create server_lock table" 2024-09-16T09:26:09.122 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.986616689Z level=info msg="Migration successfully executed" id="create server_lock table" duration=501.643µs 2024-09-16T09:26:09.122 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.986972631Z level=info msg="Executing migration" id="add index server_lock.operation_uid" 2024-09-16T09:26:09.123 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.987530022Z level=info msg="Migration successfully executed" id="add index server_lock.operation_uid" duration=557.189µs 2024-09-16T09:26:09.123 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.987876298Z level=info msg="Executing migration" id="create user auth token table" 2024-09-16T09:26:09.123 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.988433099Z level=info msg="Migration successfully executed" id="create user auth token table" duration=558.12µs 2024-09-16T09:26:09.123 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.988775813Z level=info msg="Executing migration" id="add unique index user_auth_token.auth_token" 2024-09-16T09:26:09.123 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.989359962Z level=info msg="Migration successfully executed" id="add unique index user_auth_token.auth_token" duration=583.854µs 2024-09-16T09:26:09.123 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.989772888Z level=info msg="Executing migration" id="add unique index user_auth_token.prev_auth_token" 2024-09-16T09:26:09.123 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.990338963Z level=info msg="Migration successfully executed" id="add unique index user_auth_token.prev_auth_token" duration=565.797µs 2024-09-16T09:26:09.123 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.990665145Z level=info msg="Executing migration" id="add index user_auth_token.user_id" 2024-09-16T09:26:09.123 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.991299681Z level=info msg="Migration successfully executed" id="add index user_auth_token.user_id" duration=632.464µs 2024-09-16T09:26:09.123 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.991653248Z level=info msg="Executing migration" id="Add revoked_at to the user auth token" 2024-09-16T09:26:09.123 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.994825002Z level=info msg="Migration successfully executed" id="Add revoked_at to the user auth token" duration=3.17137ms 2024-09-16T09:26:09.123 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.995218327Z level=info msg="Executing migration" id="add index user_auth_token.revoked_at" 2024-09-16T09:26:09.123 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.996024482Z level=info msg="Migration successfully executed" id="add index user_auth_token.revoked_at" duration=806.633µs 2024-09-16T09:26:09.124 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.996364604Z level=info msg="Executing migration" id="create cache_data table" 2024-09-16T09:26:09.124 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.996917099Z level=info msg="Migration successfully executed" id="create cache_data table" duration=552.181µs 2024-09-16T09:26:09.124 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.997255023Z level=info msg="Executing migration" id="add unique index cache_data.cache_key" 2024-09-16T09:26:09.124 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.997814411Z level=info msg="Migration successfully executed" id="add unique index cache_data.cache_key" duration=559.206µs 2024-09-16T09:26:09.124 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.998163658Z level=info msg="Executing migration" id="create short_url table v1" 2024-09-16T09:26:09.124 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.998713198Z level=info msg="Migration successfully executed" id="create short_url table v1" duration=549.188µs 2024-09-16T09:26:09.124 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.999053197Z level=info msg="Executing migration" id="add index short_url.org_id-uid" 2024-09-16T09:26:09.124 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:08.999640339Z level=info msg="Migration successfully executed" id="add index short_url.org_id-uid" duration=586.831µs 2024-09-16T09:26:09.124 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.00157124Z level=info msg="Executing migration" id="alter table short_url alter column created_by type to bigint" 2024-09-16T09:26:09.124 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.001635154Z level=info msg="Migration successfully executed" id="alter table short_url alter column created_by type to bigint" duration=64.394µs 2024-09-16T09:26:09.124 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.002013967Z level=info msg="Executing migration" id="delete alert_definition table" 2024-09-16T09:26:09.124 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.002074485Z level=info msg="Migration successfully executed" id="delete alert_definition table" duration=60.479µs 2024-09-16T09:26:09.124 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.002455294Z level=info msg="Executing migration" id="recreate alert_definition table" 2024-09-16T09:26:09.125 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.003043468Z level=info msg="Migration successfully executed" id="recreate alert_definition table" duration=587.842µs 2024-09-16T09:26:09.125 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.003391446Z level=info msg="Executing migration" id="add index in alert_definition on org_id and title columns" 2024-09-16T09:26:09.125 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.004015207Z level=info msg="Migration successfully executed" id="add index in alert_definition on org_id and title columns" duration=623.469µs 2024-09-16T09:26:09.125 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.004387097Z level=info msg="Executing migration" id="add index in alert_definition on org_id and uid columns" 2024-09-16T09:26:09.125 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.005019799Z level=info msg="Migration successfully executed" id="add index in alert_definition on org_id and uid columns" duration=632.43µs 2024-09-16T09:26:09.125 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.005349118Z level=info msg="Executing migration" id="alter alert_definition table data column to mediumtext in mysql" 2024-09-16T09:26:09.125 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.005389853Z level=info msg="Migration successfully executed" id="alter alert_definition table data column to mediumtext in mysql" duration=40.94µs 2024-09-16T09:26:09.125 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.005771323Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and title columns" 2024-09-16T09:26:09.125 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.006359454Z level=info msg="Migration successfully executed" id="drop index in alert_definition on org_id and title columns" duration=587.9µs 2024-09-16T09:26:09.125 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.006694044Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and uid columns" 2024-09-16T09:26:09.125 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.007277268Z level=info msg="Migration successfully executed" id="drop index in alert_definition on org_id and uid columns" duration=583.169µs 2024-09-16T09:26:09.125 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.007616919Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and title columns" 2024-09-16T09:26:09.126 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.008233932Z level=info msg="Migration successfully executed" id="add unique index in alert_definition on org_id and title columns" duration=615.97µs 2024-09-16T09:26:09.126 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.010129806Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and uid columns" 2024-09-16T09:26:09.126 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.010736122Z level=info msg="Migration successfully executed" id="add unique index in alert_definition on org_id and uid columns" duration=606.164µs 2024-09-16T09:26:09.126 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.012686313Z level=info msg="Executing migration" id="Add column paused in alert_definition" 2024-09-16T09:26:09.126 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.016131786Z level=info msg="Migration successfully executed" id="Add column paused in alert_definition" duration=3.445637ms 2024-09-16T09:26:09.126 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.01646057Z level=info msg="Executing migration" id="drop alert_definition table" 2024-09-16T09:26:09.126 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.01709994Z level=info msg="Migration successfully executed" id="drop alert_definition table" duration=639.077µs 2024-09-16T09:26:09.126 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.017438041Z level=info msg="Executing migration" id="delete alert_definition_version table" 2024-09-16T09:26:09.126 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.017496022Z level=info msg="Migration successfully executed" id="delete alert_definition_version table" duration=58.005µs 2024-09-16T09:26:09.126 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.020046793Z level=info msg="Executing migration" id="recreate alert_definition_version table" 2024-09-16T09:26:09.126 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.020607655Z level=info msg="Migration successfully executed" id="recreate alert_definition_version table" duration=560.657µs 2024-09-16T09:26:09.126 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.020961902Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_id and version columns" 2024-09-16T09:26:09.126 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.021579812Z level=info msg="Migration successfully executed" id="add index in alert_definition_version table on alert_definition_id and version columns" duration=617.784µs 2024-09-16T09:26:09.126 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.023598024Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_uid and version columns" 2024-09-16T09:26:09.127 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.024240934Z level=info msg="Migration successfully executed" id="add index in alert_definition_version table on alert_definition_uid and version columns" duration=641.317µs 2024-09-16T09:26:09.127 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.026022233Z level=info msg="Executing migration" id="alter alert_definition_version table data column to mediumtext in mysql" 2024-09-16T09:26:09.127 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.026059769Z level=info msg="Migration successfully executed" id="alter alert_definition_version table data column to mediumtext in mysql" duration=38.018µs 2024-09-16T09:26:09.127 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.026457504Z level=info msg="Executing migration" id="drop alert_definition_version table" 2024-09-16T09:26:09.127 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.027137501Z level=info msg="Migration successfully executed" id="drop alert_definition_version table" duration=678.926µs 2024-09-16T09:26:09.127 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.02854121Z level=info msg="Executing migration" id="create alert_instance table" 2024-09-16T09:26:09.127 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.029238025Z level=info msg="Migration successfully executed" id="create alert_instance table" duration=696.462µs 2024-09-16T09:26:09.127 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.031060215Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, def_uid and current_state columns" 2024-09-16T09:26:09.127 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.031686079Z level=info msg="Migration successfully executed" id="add index in alert_instance table on def_org_id, def_uid and current_state columns" duration=625.584µs 2024-09-16T09:26:09.127 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.033765549Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, current_state columns" 2024-09-16T09:26:09.127 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.034395109Z level=info msg="Migration successfully executed" id="add index in alert_instance table on def_org_id, current_state columns" duration=629.246µs 2024-09-16T09:26:09.127 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.034730648Z level=info msg="Executing migration" id="add column current_state_end to alert_instance" 2024-09-16T09:26:09.127 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.038129994Z level=info msg="Migration successfully executed" id="add column current_state_end to alert_instance" duration=3.398941ms 2024-09-16T09:26:09.128 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.038456652Z level=info msg="Executing migration" id="remove index def_org_id, def_uid, current_state on alert_instance" 2024-09-16T09:26:09.128 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.039030064Z level=info msg="Migration successfully executed" id="remove index def_org_id, def_uid, current_state on alert_instance" duration=571.744µs 2024-09-16T09:26:09.128 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.039353426Z level=info msg="Executing migration" id="remove index def_org_id, current_state on alert_instance" 2024-09-16T09:26:09.128 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.040002448Z level=info msg="Migration successfully executed" id="remove index def_org_id, current_state on alert_instance" duration=648.756µs 2024-09-16T09:26:09.128 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.041955633Z level=info msg="Executing migration" id="rename def_org_id to rule_org_id in alert_instance" 2024-09-16T09:26:09.128 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.059465762Z level=info msg="Migration successfully executed" id="rename def_org_id to rule_org_id in alert_instance" duration=17.509849ms 2024-09-16T09:26:09.128 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.059851999Z level=info msg="Executing migration" id="rename def_uid to rule_uid in alert_instance" 2024-09-16T09:26:09.128 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.076524593Z level=info msg="Migration successfully executed" id="rename def_uid to rule_uid in alert_instance" duration=16.672891ms 2024-09-16T09:26:09.128 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.076893543Z level=info msg="Executing migration" id="add index rule_org_id, rule_uid, current_state on alert_instance" 2024-09-16T09:26:09.128 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.077524986Z level=info msg="Migration successfully executed" id="add index rule_org_id, rule_uid, current_state on alert_instance" duration=631.21µs 2024-09-16T09:26:09.128 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.07945256Z level=info msg="Executing migration" id="add index rule_org_id, current_state on alert_instance" 2024-09-16T09:26:09.128 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.080068153Z level=info msg="Migration successfully executed" id="add index rule_org_id, current_state on alert_instance" duration=615.323µs 2024-09-16T09:26:09.128 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.082087322Z level=info msg="Executing migration" id="add current_reason column related to current_state" 2024-09-16T09:26:09.129 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.085429124Z level=info msg="Migration successfully executed" id="add current_reason column related to current_state" duration=3.342283ms 2024-09-16T09:26:09.129 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.085795519Z level=info msg="Executing migration" id="add result_fingerprint column to alert_instance" 2024-09-16T09:26:09.129 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.089076023Z level=info msg="Migration successfully executed" id="add result_fingerprint column to alert_instance" duration=3.280122ms 2024-09-16T09:26:09.129 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.089489466Z level=info msg="Executing migration" id="create alert_rule table" 2024-09-16T09:26:09.129 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.090157965Z level=info msg="Migration successfully executed" id="create alert_rule table" duration=668.52µs 2024-09-16T09:26:09.129 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.090556514Z level=info msg="Executing migration" id="add index in alert_rule on org_id and title columns" 2024-09-16T09:26:09.129 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.091463103Z level=info msg="Migration successfully executed" id="add index in alert_rule on org_id and title columns" duration=905.402µs 2024-09-16T09:26:09.129 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.091895566Z level=info msg="Executing migration" id="add index in alert_rule on org_id and uid columns" 2024-09-16T09:26:09.129 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.092717516Z level=info msg="Migration successfully executed" id="add index in alert_rule on org_id and uid columns" duration=820.841µs 2024-09-16T09:26:09.129 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.093306901Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespace_uid, group_uid columns" 2024-09-16T09:26:09.129 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.09407245Z level=info msg="Migration successfully executed" id="add index in alert_rule on org_id, namespace_uid, group_uid columns" duration=765.783µs 2024-09-16T09:26:09.129 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.094433981Z level=info msg="Executing migration" id="alter alert_rule table data column to mediumtext in mysql" 2024-09-16T09:26:09.129 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.094474316Z level=info msg="Migration successfully executed" id="alter alert_rule table data column to mediumtext in mysql" duration=38.861µs 2024-09-16T09:26:09.129 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.095217733Z level=info msg="Executing migration" id="add column for to alert_rule" 2024-09-16T09:26:09.130 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.098777205Z level=info msg="Migration successfully executed" id="add column for to alert_rule" duration=3.555017ms 2024-09-16T09:26:09.130 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.099180819Z level=info msg="Executing migration" id="add column annotations to alert_rule" 2024-09-16T09:26:09.130 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.102713822Z level=info msg="Migration successfully executed" id="add column annotations to alert_rule" duration=3.53174ms 2024-09-16T09:26:09.130 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.103318238Z level=info msg="Executing migration" id="add column labels to alert_rule" 2024-09-16T09:26:09.131 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:09.131 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:09.131 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:26:09.232 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:26:09.253 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:09.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:09.254 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:26:09.282 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:09 smithi070 ceph-mon[30388]: pgmap v30: 148 pgs: 16 creating+peering, 132 active+clean; 583 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 1 op/s 2024-09-16T09:26:09.282 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:09 smithi070 ceph-mon[30388]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T09:26:09.282 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:09 smithi070 ceph-mon[30388]: osdmap e83: 8 total, 8 up, 8 in 2024-09-16T09:26:09.282 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:09 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:26:09.282 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:09 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:26:09.283 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:09 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:26:09.283 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:09 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:26:09.283 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:09 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/10048478' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:26:09.283 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:09 smithi070 ceph-mon[30388]: osdmap e84: 8 total, 8 up, 8 in 2024-09-16T09:26:09.283 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:09 smithi070 ceph-mon[40691]: pgmap v30: 148 pgs: 16 creating+peering, 132 active+clean; 583 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 1 op/s 2024-09-16T09:26:09.283 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:09 smithi070 ceph-mon[40691]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T09:26:09.283 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:09 smithi070 ceph-mon[40691]: osdmap e83: 8 total, 8 up, 8 in 2024-09-16T09:26:09.283 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:09 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:26:09.283 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:09 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:26:09.283 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:09 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:26:09.283 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:09 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:26:09.284 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:09 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/10048478' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:26:09.284 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:09 smithi070 ceph-mon[40691]: osdmap e84: 8 total, 8 up, 8 in 2024-09-16T09:26:09.353 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-mon[32629]: pgmap v30: 148 pgs: 16 creating+peering, 132 active+clean; 583 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 1 op/s 2024-09-16T09:26:09.354 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-mon[32629]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T09:26:09.354 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-mon[32629]: osdmap e83: 8 total, 8 up, 8 in 2024-09-16T09:26:09.354 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:26:09.354 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:26:09.354 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:26:09.354 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:26:09.354 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/10048478' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:26:09.354 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-mon[32629]: osdmap e84: 8 total, 8 up, 8 in 2024-09-16T09:26:09.355 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.109020637Z level=info msg="Migration successfully executed" id="add column labels to alert_rule" duration=5.694791ms 2024-09-16T09:26:09.355 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.112349756Z level=info msg="Executing migration" id="remove unique index from alert_rule on org_id, title columns" 2024-09-16T09:26:09.355 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.113346109Z level=info msg="Migration successfully executed" id="remove unique index from alert_rule on org_id, title columns" duration=997.529µs 2024-09-16T09:26:09.355 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.113837591Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespase_uid and title columns" 2024-09-16T09:26:09.355 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.114838727Z level=info msg="Migration successfully executed" id="add index in alert_rule on org_id, namespase_uid and title columns" duration=1.000374ms 2024-09-16T09:26:09.355 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.115317324Z level=info msg="Executing migration" id="add dashboard_uid column to alert_rule" 2024-09-16T09:26:09.355 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.119881336Z level=info msg="Migration successfully executed" id="add dashboard_uid column to alert_rule" duration=4.562301ms 2024-09-16T09:26:09.355 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.120323981Z level=info msg="Executing migration" id="add panel_id column to alert_rule" 2024-09-16T09:26:09.355 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.12376819Z level=info msg="Migration successfully executed" id="add panel_id column to alert_rule" duration=3.439681ms 2024-09-16T09:26:09.355 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.124165051Z level=info msg="Executing migration" id="add index in alert_rule on org_id, dashboard_uid and panel_id columns" 2024-09-16T09:26:09.355 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.124789369Z level=info msg="Migration successfully executed" id="add index in alert_rule on org_id, dashboard_uid and panel_id columns" duration=624.04µs 2024-09-16T09:26:09.355 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.125150092Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule" 2024-09-16T09:26:09.355 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.128529546Z level=info msg="Migration successfully executed" id="add rule_group_idx column to alert_rule" duration=3.379169ms 2024-09-16T09:26:09.356 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.12889114Z level=info msg="Executing migration" id="add is_paused column to alert_rule table" 2024-09-16T09:26:09.356 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.132297943Z level=info msg="Migration successfully executed" id="add is_paused column to alert_rule table" duration=3.406421ms 2024-09-16T09:26:09.356 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.132644457Z level=info msg="Executing migration" id="fix is_paused column for alert_rule table" 2024-09-16T09:26:09.356 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.132682452Z level=info msg="Migration successfully executed" id="fix is_paused column for alert_rule table" duration=38.335µs 2024-09-16T09:26:09.356 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.13367688Z level=info msg="Executing migration" id="create alert_rule_version table" 2024-09-16T09:26:09.356 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.134539607Z level=info msg="Migration successfully executed" id="create alert_rule_version table" duration=863.067µs 2024-09-16T09:26:09.356 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.134917848Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_uid and version columns" 2024-09-16T09:26:09.356 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.135568133Z level=info msg="Migration successfully executed" id="add index in alert_rule_version table on rule_org_id, rule_uid and version columns" duration=650.138µs 2024-09-16T09:26:09.356 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.135926394Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_namespace_uid and rule_group columns" 2024-09-16T09:26:09.356 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.136603882Z level=info msg="Migration successfully executed" id="add index in alert_rule_version table on rule_org_id, rule_namespace_uid and rule_group columns" duration=677.184µs 2024-09-16T09:26:09.356 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.136960119Z level=info msg="Executing migration" id="alter alert_rule_version table data column to mediumtext in mysql" 2024-09-16T09:26:09.357 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.136998086Z level=info msg="Migration successfully executed" id="alter alert_rule_version table data column to mediumtext in mysql" duration=38.525µs 2024-09-16T09:26:09.357 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.137922493Z level=info msg="Executing migration" id="add column for to alert_rule_version" 2024-09-16T09:26:09.357 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.141655678Z level=info msg="Migration successfully executed" id="add column for to alert_rule_version" duration=3.757172ms 2024-09-16T09:26:09.357 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.142021342Z level=info msg="Executing migration" id="add column annotations to alert_rule_version" 2024-09-16T09:26:09.357 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.145578754Z level=info msg="Migration successfully executed" id="add column annotations to alert_rule_version" duration=3.557117ms 2024-09-16T09:26:09.357 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.1459333Z level=info msg="Executing migration" id="add column labels to alert_rule_version" 2024-09-16T09:26:09.357 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.149429696Z level=info msg="Migration successfully executed" id="add column labels to alert_rule_version" duration=3.496109ms 2024-09-16T09:26:09.357 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.149780976Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule_version" 2024-09-16T09:26:09.357 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.153348399Z level=info msg="Migration successfully executed" id="add rule_group_idx column to alert_rule_version" duration=3.567073ms 2024-09-16T09:26:09.357 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:26:09.359 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.153715021Z level=info msg="Executing migration" id="add is_paused column to alert_rule_versions table" 2024-09-16T09:26:09.359 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.157207585Z level=info msg="Migration successfully executed" id="add is_paused column to alert_rule_versions table" duration=3.492298ms 2024-09-16T09:26:09.359 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.157539659Z level=info msg="Executing migration" id="fix is_paused column for alert_rule_version table" 2024-09-16T09:26:09.359 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.157578712Z level=info msg="Migration successfully executed" id="fix is_paused column for alert_rule_version table" duration=37.891µs 2024-09-16T09:26:09.359 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.158516226Z level=info msg="Executing migration" id=create_alert_configuration_table 2024-09-16T09:26:09.359 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.159013238Z level=info msg="Migration successfully executed" id=create_alert_configuration_table duration=496.795µs 2024-09-16T09:26:09.359 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.159355831Z level=info msg="Executing migration" id="Add column default in alert_configuration" 2024-09-16T09:26:09.359 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.162947488Z level=info msg="Migration successfully executed" id="Add column default in alert_configuration" duration=3.591149ms 2024-09-16T09:26:09.359 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.163277569Z level=info msg="Executing migration" id="alert alert_configuration alertmanager_configuration column from TEXT to MEDIUMTEXT if mysql" 2024-09-16T09:26:09.359 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.163314479Z level=info msg="Migration successfully executed" id="alert alert_configuration alertmanager_configuration column from TEXT to MEDIUMTEXT if mysql" duration=37.548µs 2024-09-16T09:26:09.359 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.16627441Z level=info msg="Executing migration" id="add column org_id in alert_configuration" 2024-09-16T09:26:09.359 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.169935936Z level=info msg="Migration successfully executed" id="add column org_id in alert_configuration" duration=3.662257ms 2024-09-16T09:26:09.360 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.170275357Z level=info msg="Executing migration" id="add index in alert_configuration table on org_id column" 2024-09-16T09:26:09.360 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.170862132Z level=info msg="Migration successfully executed" id="add index in alert_configuration table on org_id column" duration=586.593µs 2024-09-16T09:26:09.360 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.173142822Z level=info msg="Executing migration" id="add configuration_hash column to alert_configuration" 2024-09-16T09:26:09.360 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.176707351Z level=info msg="Migration successfully executed" id="add configuration_hash column to alert_configuration" duration=3.564119ms 2024-09-16T09:26:09.360 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.177063774Z level=info msg="Executing migration" id=create_ngalert_configuration_table 2024-09-16T09:26:09.360 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.177551222Z level=info msg="Migration successfully executed" id=create_ngalert_configuration_table duration=487.388µs 2024-09-16T09:26:09.360 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.179843936Z level=info msg="Executing migration" id="add index in ngalert_configuration on org_id column" 2024-09-16T09:26:09.360 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.180467629Z level=info msg="Migration successfully executed" id="add index in ngalert_configuration on org_id column" duration=623.473µs 2024-09-16T09:26:09.360 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.182566607Z level=info msg="Executing migration" id="add column send_alerts_to in ngalert_configuration" 2024-09-16T09:26:09.360 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.18616346Z level=info msg="Migration successfully executed" id="add column send_alerts_to in ngalert_configuration" duration=3.596447ms 2024-09-16T09:26:09.360 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.186540798Z level=info msg="Executing migration" id="create provenance_type table" 2024-09-16T09:26:09.360 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.187031919Z level=info msg="Migration successfully executed" id="create provenance_type table" duration=490.832µs 2024-09-16T09:26:09.360 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.189364913Z level=info msg="Executing migration" id="add index to uniquify (record_key, record_type, org_id) columns" 2024-09-16T09:26:09.361 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.190010217Z level=info msg="Migration successfully executed" id="add index to uniquify (record_key, record_type, org_id) columns" duration=644.916µs 2024-09-16T09:26:09.361 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.192052742Z level=info msg="Executing migration" id="create alert_image table" 2024-09-16T09:26:09.361 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.192559695Z level=info msg="Migration successfully executed" id="create alert_image table" duration=508.209µs 2024-09-16T09:26:09.361 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.194919388Z level=info msg="Executing migration" id="add unique index on token to alert_image table" 2024-09-16T09:26:09.361 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.19564577Z level=info msg="Migration successfully executed" id="add unique index on token to alert_image table" duration=727.226µs 2024-09-16T09:26:09.361 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.197822077Z level=info msg="Executing migration" id="support longer URLs in alert_image table" 2024-09-16T09:26:09.361 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.19786115Z level=info msg="Migration successfully executed" id="support longer URLs in alert_image table" duration=38.735µs 2024-09-16T09:26:09.361 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.200639142Z level=info msg="Executing migration" id=create_alert_configuration_history_table 2024-09-16T09:26:09.361 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.201249866Z level=info msg="Migration successfully executed" id=create_alert_configuration_history_table duration=610.58µs 2024-09-16T09:26:09.361 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.203379322Z level=info msg="Executing migration" id="drop non-unique orgID index on alert_configuration" 2024-09-16T09:26:09.361 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.203986383Z level=info msg="Migration successfully executed" id="drop non-unique orgID index on alert_configuration" duration=606.737µs 2024-09-16T09:26:09.361 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.206128153Z level=info msg="Executing migration" id="drop unique orgID index on alert_configuration if exists" 2024-09-16T09:26:09.361 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.206386723Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop unique orgID index on alert_configuration if exists" 2024-09-16T09:26:09.362 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.209142637Z level=info msg="Executing migration" id="extract alertmanager configuration history to separate table" 2024-09-16T09:26:09.362 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.209546224Z level=info msg="Migration successfully executed" id="extract alertmanager configuration history to separate table" duration=406.21µs 2024-09-16T09:26:09.362 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.212073982Z level=info msg="Executing migration" id="add unique index on orgID to alert_configuration" 2024-09-16T09:26:09.362 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.212751103Z level=info msg="Migration successfully executed" id="add unique index on orgID to alert_configuration" duration=679.105µs 2024-09-16T09:26:09.362 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.214889196Z level=info msg="Executing migration" id="add last_applied column to alert_configuration_history" 2024-09-16T09:26:09.362 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.218560257Z level=info msg="Migration successfully executed" id="add last_applied column to alert_configuration_history" duration=3.673169ms 2024-09-16T09:26:09.362 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.218933014Z level=info msg="Executing migration" id="create library_element table v1" 2024-09-16T09:26:09.362 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.219613728Z level=info msg="Migration successfully executed" id="create library_element table v1" duration=680.459µs 2024-09-16T09:26:09.362 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.221637067Z level=info msg="Executing migration" id="add index library_element org_id-folder_id-name-kind" 2024-09-16T09:26:09.362 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.222312527Z level=info msg="Migration successfully executed" id="add index library_element org_id-folder_id-name-kind" duration=675.19µs 2024-09-16T09:26:09.362 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.224199563Z level=info msg="Executing migration" id="create library_element_connection table v1" 2024-09-16T09:26:09.362 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.224726195Z level=info msg="Migration successfully executed" id="create library_element_connection table v1" duration=528.837µs 2024-09-16T09:26:09.363 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.227159773Z level=info msg="Executing migration" id="add index library_element_connection element_id-kind-connection_id" 2024-09-16T09:26:09.363 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.227953352Z level=info msg="Migration successfully executed" id="add index library_element_connection element_id-kind-connection_id" duration=795.552µs 2024-09-16T09:26:09.363 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.229643175Z level=info msg="Executing migration" id="add unique index library_element org_id_uid" 2024-09-16T09:26:09.363 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.230276469Z level=info msg="Migration successfully executed" id="add unique index library_element org_id_uid" duration=632.931µs 2024-09-16T09:26:09.363 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.232349858Z level=info msg="Executing migration" id="increase max description length to 2048" 2024-09-16T09:26:09.363 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.232364512Z level=info msg="Migration successfully executed" id="increase max description length to 2048" duration=17.797µs 2024-09-16T09:26:09.363 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.235000335Z level=info msg="Executing migration" id="alter library_element model to mediumtext" 2024-09-16T09:26:09.363 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.235038241Z level=info msg="Migration successfully executed" id="alter library_element model to mediumtext" duration=37.15µs 2024-09-16T09:26:09.363 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.237753373Z level=info msg="Executing migration" id="clone move dashboard alerts to unified alerting" 2024-09-16T09:26:09.363 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.237941234Z level=info msg="Migration successfully executed" id="clone move dashboard alerts to unified alerting" duration=187.721µs 2024-09-16T09:26:09.363 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.240350528Z level=info msg="Executing migration" id="create data_keys table" 2024-09-16T09:26:09.363 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.240999347Z level=info msg="Migration successfully executed" id="create data_keys table" duration=648.488µs 2024-09-16T09:26:09.363 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.243057086Z level=info msg="Executing migration" id="create secrets table" 2024-09-16T09:26:09.364 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.243668235Z level=info msg="Migration successfully executed" id="create secrets table" duration=613.413µs 2024-09-16T09:26:09.364 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.245490977Z level=info msg="Executing migration" id="rename data_keys name column to id" 2024-09-16T09:26:09.364 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.265804926Z level=info msg="Migration successfully executed" id="rename data_keys name column to id" duration=20.313616ms 2024-09-16T09:26:09.364 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.266291758Z level=info msg="Executing migration" id="add name column into data_keys" 2024-09-16T09:26:09.364 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.270408797Z level=info msg="Migration successfully executed" id="add name column into data_keys" duration=4.116822ms 2024-09-16T09:26:09.364 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.270757332Z level=info msg="Executing migration" id="copy data_keys id column values into name" 2024-09-16T09:26:09.364 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.27084803Z level=info msg="Migration successfully executed" id="copy data_keys id column values into name" duration=90.811µs 2024-09-16T09:26:09.364 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.273313508Z level=info msg="Executing migration" id="rename data_keys name column to label" 2024-09-16T09:26:09.364 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.294010108Z level=info msg="Migration successfully executed" id="rename data_keys name column to label" duration=20.696223ms 2024-09-16T09:26:09.364 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.29438307Z level=info msg="Executing migration" id="rename data_keys id column back to name" 2024-09-16T09:26:09.364 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.315542035Z level=info msg="Migration successfully executed" id="rename data_keys id column back to name" duration=21.158651ms 2024-09-16T09:26:09.365 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.315927207Z level=info msg="Executing migration" id="create kv_store table v1" 2024-09-16T09:26:09.365 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.316467711Z level=info msg="Migration successfully executed" id="create kv_store table v1" duration=538.835µs 2024-09-16T09:26:09.365 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.318767267Z level=info msg="Executing migration" id="add index kv_store.org_id-namespace-key" 2024-09-16T09:26:09.365 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.319433167Z level=info msg="Migration successfully executed" id="add index kv_store.org_id-namespace-key" duration=665.61µs 2024-09-16T09:26:09.365 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.319784924Z level=info msg="Executing migration" id="update dashboard_uid and panel_id from existing annotations" 2024-09-16T09:26:09.365 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.319916707Z level=info msg="Migration successfully executed" id="update dashboard_uid and panel_id from existing annotations" duration=131.802µs 2024-09-16T09:26:09.365 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.320641994Z level=info msg="Executing migration" id="create permission table" 2024-09-16T09:26:09.365 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.321235024Z level=info msg="Migration successfully executed" id="create permission table" duration=592.92µs 2024-09-16T09:26:09.365 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.321590509Z level=info msg="Executing migration" id="add unique index permission.role_id" 2024-09-16T09:26:09.365 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.322203676Z level=info msg="Migration successfully executed" id="add unique index permission.role_id" duration=612.933µs 2024-09-16T09:26:09.365 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.322583308Z level=info msg="Executing migration" id="add unique index role_id_action_scope" 2024-09-16T09:26:09.365 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.323235644Z level=info msg="Migration successfully executed" id="add unique index role_id_action_scope" duration=652.018µs 2024-09-16T09:26:09.366 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.323595333Z level=info msg="Executing migration" id="create role table" 2024-09-16T09:26:09.366 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.324165067Z level=info msg="Migration successfully executed" id="create role table" duration=569.55µs 2024-09-16T09:26:09.366 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.324521915Z level=info msg="Executing migration" id="add column display_name" 2024-09-16T09:26:09.366 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.328787028Z level=info msg="Migration successfully executed" id="add column display_name" duration=4.264752ms 2024-09-16T09:26:09.366 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.329165881Z level=info msg="Executing migration" id="add column group_name" 2024-09-16T09:26:09.366 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.333317383Z level=info msg="Migration successfully executed" id="add column group_name" duration=4.151293ms 2024-09-16T09:26:09.366 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.333669027Z level=info msg="Executing migration" id="add index role.org_id" 2024-09-16T09:26:09.366 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.334305908Z level=info msg="Migration successfully executed" id="add index role.org_id" duration=635.559µs 2024-09-16T09:26:09.366 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.334657027Z level=info msg="Executing migration" id="add unique index role_org_id_name" 2024-09-16T09:26:09.366 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.335325835Z level=info msg="Migration successfully executed" id="add unique index role_org_id_name" duration=668.532µs 2024-09-16T09:26:09.366 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.335676852Z level=info msg="Executing migration" id="add index role_org_id_uid" 2024-09-16T09:26:09.366 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.33633409Z level=info msg="Migration successfully executed" id="add index role_org_id_uid" duration=657.064µs 2024-09-16T09:26:09.366 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.336693131Z level=info msg="Executing migration" id="create team role table" 2024-09-16T09:26:09.367 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.337240502Z level=info msg="Migration successfully executed" id="create team role table" duration=547.205µs 2024-09-16T09:26:09.367 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.337592448Z level=info msg="Executing migration" id="add index team_role.org_id" 2024-09-16T09:26:09.367 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.338266518Z level=info msg="Migration successfully executed" id="add index team_role.org_id" duration=673.908µs 2024-09-16T09:26:09.367 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.338628767Z level=info msg="Executing migration" id="add unique index team_role_org_id_team_id_role_id" 2024-09-16T09:26:09.367 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.339334183Z level=info msg="Migration successfully executed" id="add unique index team_role_org_id_team_id_role_id" duration=705.027µs 2024-09-16T09:26:09.367 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.339691808Z level=info msg="Executing migration" id="add index team_role.team_id" 2024-09-16T09:26:09.368 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.340338091Z level=info msg="Migration successfully executed" id="add index team_role.team_id" duration=647.565µs 2024-09-16T09:26:09.368 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.34069332Z level=info msg="Executing migration" id="create user role table" 2024-09-16T09:26:09.368 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.341242728Z level=info msg="Migration successfully executed" id="create user role table" duration=533.194µs 2024-09-16T09:26:09.368 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.341606896Z level=info msg="Executing migration" id="add index user_role.org_id" 2024-09-16T09:26:09.368 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.342255032Z level=info msg="Migration successfully executed" id="add index user_role.org_id" duration=647.966µs 2024-09-16T09:26:09.368 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.342603027Z level=info msg="Executing migration" id="add unique index user_role_org_id_user_id_role_id" 2024-09-16T09:26:09.368 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.343235442Z level=info msg="Migration successfully executed" id="add unique index user_role_org_id_user_id_role_id" duration=631.961µs 2024-09-16T09:26:09.368 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.343562065Z level=info msg="Executing migration" id="add index user_role.user_id" 2024-09-16T09:26:09.368 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.344180995Z level=info msg="Migration successfully executed" id="add index user_role.user_id" duration=618.752µs 2024-09-16T09:26:09.369 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.344509888Z level=info msg="Executing migration" id="create builtin role table" 2024-09-16T09:26:09.369 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.345058178Z level=info msg="Migration successfully executed" id="create builtin role table" duration=548.014µs 2024-09-16T09:26:09.369 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.345394659Z level=info msg="Executing migration" id="add index builtin_role.role_id" 2024-09-16T09:26:09.369 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.346037918Z level=info msg="Migration successfully executed" id="add index builtin_role.role_id" duration=642.941µs 2024-09-16T09:26:09.369 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.346368276Z level=info msg="Executing migration" id="add index builtin_role.name" 2024-09-16T09:26:09.369 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.347018625Z level=info msg="Migration successfully executed" id="add index builtin_role.name" duration=650.05µs 2024-09-16T09:26:09.369 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.347345077Z level=info msg="Executing migration" id="Add column org_id to builtin_role table" 2024-09-16T09:26:09.369 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.351856197Z level=info msg="Migration successfully executed" id="Add column org_id to builtin_role table" duration=4.511542ms 2024-09-16T09:26:09.369 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.352209434Z level=info msg="Executing migration" id="add index builtin_role.org_id" 2024-09-16T09:26:09.369 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.352826945Z level=info msg="Migration successfully executed" id="add index builtin_role.org_id" duration=617.29µs 2024-09-16T09:26:09.369 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.353192539Z level=info msg="Executing migration" id="add unique index builtin_role_org_id_role_id_role" 2024-09-16T09:26:09.369 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.353855856Z level=info msg="Migration successfully executed" id="add unique index builtin_role_org_id_role_id_role" duration=662.982µs 2024-09-16T09:26:09.370 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.354211191Z level=info msg="Executing migration" id="Remove unique index role_org_id_uid" 2024-09-16T09:26:09.370 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.354843298Z level=info msg="Migration successfully executed" id="Remove unique index role_org_id_uid" duration=631.8µs 2024-09-16T09:26:09.370 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.355174738Z level=info msg="Executing migration" id="add unique index role.uid" 2024-09-16T09:26:09.370 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.355802178Z level=info msg="Migration successfully executed" id="add unique index role.uid" duration=627.174µs 2024-09-16T09:26:09.370 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.356136371Z level=info msg="Executing migration" id="create seed assignment table" 2024-09-16T09:26:09.370 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.356616277Z level=info msg="Migration successfully executed" id="create seed assignment table" duration=481.062µs 2024-09-16T09:26:09.370 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.356966132Z level=info msg="Executing migration" id="add unique index builtin_role_role_name" 2024-09-16T09:26:09.370 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.357679816Z level=info msg="Migration successfully executed" id="add unique index builtin_role_role_name" duration=713.298µs 2024-09-16T09:26:09.370 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.358016061Z level=info msg="Executing migration" id="add column hidden to role table" 2024-09-16T09:26:09.388 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:09.389 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:09.389 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:26:09.490 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:26:09.515 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:09.516 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:09.516 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:26:09.604 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.362693393Z level=info msg="Migration successfully executed" id="add column hidden to role table" duration=4.676955ms 2024-09-16T09:26:09.604 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.363019141Z level=info msg="Executing migration" id="permission kind migration" 2024-09-16T09:26:09.604 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.367598252Z level=info msg="Migration successfully executed" id="permission kind migration" duration=4.578829ms 2024-09-16T09:26:09.605 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.367949267Z level=info msg="Executing migration" id="permission attribute migration" 2024-09-16T09:26:09.605 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.372451609Z level=info msg="Migration successfully executed" id="permission attribute migration" duration=4.502213ms 2024-09-16T09:26:09.605 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.372787965Z level=info msg="Executing migration" id="permission identifier migration" 2024-09-16T09:26:09.605 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.377300708Z level=info msg="Migration successfully executed" id="permission identifier migration" duration=4.512403ms 2024-09-16T09:26:09.605 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.377646398Z level=info msg="Executing migration" id="add permission identifier index" 2024-09-16T09:26:09.605 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.378303536Z level=info msg="Migration successfully executed" id="add permission identifier index" duration=656.86µs 2024-09-16T09:26:09.605 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.378635503Z level=info msg="Executing migration" id="add permission action scope role_id index" 2024-09-16T09:26:09.605 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.379346918Z level=info msg="Migration successfully executed" id="add permission action scope role_id index" duration=711.972µs 2024-09-16T09:26:09.605 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.379680808Z level=info msg="Executing migration" id="remove permission role_id action scope index" 2024-09-16T09:26:09.605 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.380319637Z level=info msg="Migration successfully executed" id="remove permission role_id action scope index" duration=638.685µs 2024-09-16T09:26:09.605 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.38264637Z level=info msg="Executing migration" id="create query_history table v1" 2024-09-16T09:26:09.605 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.383240292Z level=info msg="Migration successfully executed" id="create query_history table v1" duration=596.27µs 2024-09-16T09:26:09.605 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.38549346Z level=info msg="Executing migration" id="add index query_history.org_id-created_by-datasource_uid" 2024-09-16T09:26:09.606 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.386258Z level=info msg="Migration successfully executed" id="add index query_history.org_id-created_by-datasource_uid" duration=766.287µs 2024-09-16T09:26:09.607 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.388322404Z level=info msg="Executing migration" id="alter table query_history alter column created_by type to bigint" 2024-09-16T09:26:09.607 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.388418952Z level=info msg="Migration successfully executed" id="alter table query_history alter column created_by type to bigint" duration=99.724µs 2024-09-16T09:26:09.607 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.391054444Z level=info msg="Executing migration" id="rbac disabled migrator" 2024-09-16T09:26:09.607 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.391100056Z level=info msg="Migration successfully executed" id="rbac disabled migrator" duration=44.947µs 2024-09-16T09:26:09.607 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.393866131Z level=info msg="Executing migration" id="teams permissions migration" 2024-09-16T09:26:09.607 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.394303502Z level=info msg="Migration successfully executed" id="teams permissions migration" duration=440.632µs 2024-09-16T09:26:09.607 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.396749455Z level=info msg="Executing migration" id="dashboard permissions" 2024-09-16T09:26:09.607 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.397203258Z level=info msg="Migration successfully executed" id="dashboard permissions" duration=456.838µs 2024-09-16T09:26:09.607 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.399600528Z level=info msg="Executing migration" id="dashboard permissions uid scopes" 2024-09-16T09:26:09.607 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.40006344Z level=info msg="Migration successfully executed" id="dashboard permissions uid scopes" duration=465.419µs 2024-09-16T09:26:09.607 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.402458687Z level=info msg="Executing migration" id="drop managed folder create actions" 2024-09-16T09:26:09.607 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.402637545Z level=info msg="Migration successfully executed" id="drop managed folder create actions" duration=178.756µs 2024-09-16T09:26:09.608 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.405112973Z level=info msg="Executing migration" id="alerting notification permissions" 2024-09-16T09:26:09.608 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.405445123Z level=info msg="Migration successfully executed" id="alerting notification permissions" duration=332.012µs 2024-09-16T09:26:09.608 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.407946813Z level=info msg="Executing migration" id="create query_history_star table v1" 2024-09-16T09:26:09.608 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.408568158Z level=info msg="Migration successfully executed" id="create query_history_star table v1" duration=623.42µs 2024-09-16T09:26:09.608 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.410660954Z level=info msg="Executing migration" id="add index query_history.user_id-query_uid" 2024-09-16T09:26:09.608 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.411368661Z level=info msg="Migration successfully executed" id="add index query_history.user_id-query_uid" duration=707.426µs 2024-09-16T09:26:09.608 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.413438506Z level=info msg="Executing migration" id="add column org_id in query_history_star" 2024-09-16T09:26:09.608 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.418116343Z level=info msg="Migration successfully executed" id="add column org_id in query_history_star" duration=4.677693ms 2024-09-16T09:26:09.608 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.41849685Z level=info msg="Executing migration" id="alter table query_history_star_mig column user_id type to bigint" 2024-09-16T09:26:09.608 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.418541327Z level=info msg="Migration successfully executed" id="alter table query_history_star_mig column user_id type to bigint" duration=44.076µs 2024-09-16T09:26:09.608 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.421246441Z level=info msg="Executing migration" id="create correlation table v1" 2024-09-16T09:26:09.608 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.42196613Z level=info msg="Migration successfully executed" id="create correlation table v1" duration=719.436µs 2024-09-16T09:26:09.608 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.422351219Z level=info msg="Executing migration" id="add index correlations.uid" 2024-09-16T09:26:09.609 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.423024891Z level=info msg="Migration successfully executed" id="add index correlations.uid" duration=674.591µs 2024-09-16T09:26:09.609 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.425270472Z level=info msg="Executing migration" id="add index correlations.source_uid" 2024-09-16T09:26:09.609 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.42598332Z level=info msg="Migration successfully executed" id="add index correlations.source_uid" duration=712.577µs 2024-09-16T09:26:09.609 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.4282617Z level=info msg="Executing migration" id="add correlation config column" 2024-09-16T09:26:09.609 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.43313836Z level=info msg="Migration successfully executed" id="add correlation config column" duration=4.874336ms 2024-09-16T09:26:09.609 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.433538985Z level=info msg="Executing migration" id="drop index IDX_correlation_uid - v1" 2024-09-16T09:26:09.609 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.434217184Z level=info msg="Migration successfully executed" id="drop index IDX_correlation_uid - v1" duration=695.088µs 2024-09-16T09:26:09.609 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.43458795Z level=info msg="Executing migration" id="drop index IDX_correlation_source_uid - v1" 2024-09-16T09:26:09.609 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.435259987Z level=info msg="Migration successfully executed" id="drop index IDX_correlation_source_uid - v1" duration=672.006µs 2024-09-16T09:26:09.609 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.43715702Z level=info msg="Executing migration" id="Rename table correlation to correlation_tmp_qwerty - v1" 2024-09-16T09:26:09.609 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.450433487Z level=info msg="Migration successfully executed" id="Rename table correlation to correlation_tmp_qwerty - v1" duration=13.27624ms 2024-09-16T09:26:09.609 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.450810633Z level=info msg="Executing migration" id="create correlation v2" 2024-09-16T09:26:09.609 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.451551825Z level=info msg="Migration successfully executed" id="create correlation v2" duration=740.786µs 2024-09-16T09:26:09.610 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.451940296Z level=info msg="Executing migration" id="create index IDX_correlation_uid - v2" 2024-09-16T09:26:09.610 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.452607464Z level=info msg="Migration successfully executed" id="create index IDX_correlation_uid - v2" duration=667.122µs 2024-09-16T09:26:09.610 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.452982858Z level=info msg="Executing migration" id="create index IDX_correlation_source_uid - v2" 2024-09-16T09:26:09.610 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.453700319Z level=info msg="Migration successfully executed" id="create index IDX_correlation_source_uid - v2" duration=716.506µs 2024-09-16T09:26:09.610 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.454062247Z level=info msg="Executing migration" id="create index IDX_correlation_org_id - v2" 2024-09-16T09:26:09.610 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.454747179Z level=info msg="Migration successfully executed" id="create index IDX_correlation_org_id - v2" duration=684.707µs 2024-09-16T09:26:09.610 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.455134898Z level=info msg="Executing migration" id="copy correlation v1 to v2" 2024-09-16T09:26:09.610 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.455316281Z level=info msg="Migration successfully executed" id="copy correlation v1 to v2" duration=181.255µs 2024-09-16T09:26:09.610 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.455705469Z level=info msg="Executing migration" id="drop correlation_tmp_qwerty" 2024-09-16T09:26:09.610 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.456267514Z level=info msg="Migration successfully executed" id="drop correlation_tmp_qwerty" duration=561.851µs 2024-09-16T09:26:09.610 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.457779964Z level=info msg="Executing migration" id="add provisioning column" 2024-09-16T09:26:09.610 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.462439353Z level=info msg="Migration successfully executed" id="add provisioning column" duration=4.659128ms 2024-09-16T09:26:09.610 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.462811145Z level=info msg="Executing migration" id="create entity_events table" 2024-09-16T09:26:09.610 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.463359549Z level=info msg="Migration successfully executed" id="create entity_events table" duration=548.322µs 2024-09-16T09:26:09.611 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.465382946Z level=info msg="Executing migration" id="create dashboard public config v1" 2024-09-16T09:26:09.611 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.466225461Z level=info msg="Migration successfully executed" id="create dashboard public config v1" duration=845.018µs 2024-09-16T09:26:09.611 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.467818026Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v1" 2024-09-16T09:26:09.611 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.468178401Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index UQE_dashboard_public_config_uid - v1" 2024-09-16T09:26:09.611 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.468691526Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-16T09:26:09.611 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.468996201Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-16T09:26:09.611 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.469393376Z level=info msg="Executing migration" id="Drop old dashboard public config table" 2024-09-16T09:26:09.611 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.470013314Z level=info msg="Migration successfully executed" id="Drop old dashboard public config table" duration=618.051µs 2024-09-16T09:26:09.611 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.470381024Z level=info msg="Executing migration" id="recreate dashboard public config v1" 2024-09-16T09:26:09.611 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.471085746Z level=info msg="Migration successfully executed" id="recreate dashboard public config v1" duration=704.442µs 2024-09-16T09:26:09.611 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.473061767Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v1" 2024-09-16T09:26:09.611 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.473741022Z level=info msg="Migration successfully executed" id="create index UQE_dashboard_public_config_uid - v1" duration=679.323µs 2024-09-16T09:26:09.611 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.474131954Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-16T09:26:09.611 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.474836306Z level=info msg="Migration successfully executed" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v1" duration=705.861µs 2024-09-16T09:26:09.612 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.476884439Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v2" 2024-09-16T09:26:09.612 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.47770585Z level=info msg="Migration successfully executed" id="drop index UQE_dashboard_public_config_uid - v2" duration=808.077µs 2024-09-16T09:26:09.612 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.478086239Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-09-16T09:26:09.612 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.478760797Z level=info msg="Migration successfully executed" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v2" duration=674.555µs 2024-09-16T09:26:09.612 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.479148832Z level=info msg="Executing migration" id="Drop public config table" 2024-09-16T09:26:09.612 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.479699965Z level=info msg="Migration successfully executed" id="Drop public config table" duration=550.838µs 2024-09-16T09:26:09.612 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.480078447Z level=info msg="Executing migration" id="Recreate dashboard public config v2" 2024-09-16T09:26:09.612 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.480786795Z level=info msg="Migration successfully executed" id="Recreate dashboard public config v2" duration=708.27µs 2024-09-16T09:26:09.612 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.481159005Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v2" 2024-09-16T09:26:09.612 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.481850809Z level=info msg="Migration successfully executed" id="create index UQE_dashboard_public_config_uid - v2" duration=690.442µs 2024-09-16T09:26:09.612 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.482256722Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-09-16T09:26:09.612 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.482969014Z level=info msg="Migration successfully executed" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v2" duration=711.956µs 2024-09-16T09:26:09.612 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.483326397Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_access_token - v2" 2024-09-16T09:26:09.613 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.484030641Z level=info msg="Migration successfully executed" id="create index UQE_dashboard_public_config_access_token - v2" duration=704.034µs 2024-09-16T09:26:09.613 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.4844313Z level=info msg="Executing migration" id="Rename table dashboard_public_config to dashboard_public - v2" 2024-09-16T09:26:09.613 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.500047507Z level=info msg="Migration successfully executed" id="Rename table dashboard_public_config to dashboard_public - v2" duration=15.615829ms 2024-09-16T09:26:09.613 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.500462304Z level=info msg="Executing migration" id="add annotations_enabled column" 2024-09-16T09:26:09.613 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.505411998Z level=info msg="Migration successfully executed" id="add annotations_enabled column" duration=4.949556ms 2024-09-16T09:26:09.613 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.505802577Z level=info msg="Executing migration" id="add time_selection_enabled column" 2024-09-16T09:26:09.613 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.510663539Z level=info msg="Migration successfully executed" id="add time_selection_enabled column" duration=4.860728ms 2024-09-16T09:26:09.613 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.511114497Z level=info msg="Executing migration" id="delete orphaned public dashboards" 2024-09-16T09:26:09.613 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.51127715Z level=info msg="Migration successfully executed" id="delete orphaned public dashboards" duration=162.642µs 2024-09-16T09:26:09.614 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.511699026Z level=info msg="Executing migration" id="add share column" 2024-09-16T09:26:09.614 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.516496836Z level=info msg="Migration successfully executed" id="add share column" duration=4.797367ms 2024-09-16T09:26:09.614 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.516867949Z level=info msg="Executing migration" id="backfill empty share column fields with default of public" 2024-09-16T09:26:09.614 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.517025494Z level=info msg="Migration successfully executed" id="backfill empty share column fields with default of public" duration=157.69µs 2024-09-16T09:26:09.614 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.517634725Z level=info msg="Executing migration" id="create file table" 2024-09-16T09:26:09.614 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.518272021Z level=info msg="Migration successfully executed" id="create file table" duration=636.959µs 2024-09-16T09:26:09.614 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.518653347Z level=info msg="Executing migration" id="file table idx: path natural pk" 2024-09-16T09:26:09.614 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.51934173Z level=info msg="Migration successfully executed" id="file table idx: path natural pk" duration=689.696µs 2024-09-16T09:26:09.614 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.519719227Z level=info msg="Executing migration" id="file table idx: parent_folder_path_hash fast folder retrieval" 2024-09-16T09:26:09.615 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.520420074Z level=info msg="Migration successfully executed" id="file table idx: parent_folder_path_hash fast folder retrieval" duration=700.559µs 2024-09-16T09:26:09.615 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.520844939Z level=info msg="Executing migration" id="create file_meta table" 2024-09-16T09:26:09.615 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.521396258Z level=info msg="Migration successfully executed" id="create file_meta table" duration=551.466µs 2024-09-16T09:26:09.615 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.52177314Z level=info msg="Executing migration" id="file table idx: path key" 2024-09-16T09:26:09.615 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.522459927Z level=info msg="Migration successfully executed" id="file table idx: path key" duration=686.548µs 2024-09-16T09:26:09.615 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.522839515Z level=info msg="Executing migration" id="set path collation in file table" 2024-09-16T09:26:09.615 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.522884701Z level=info msg="Migration successfully executed" id="set path collation in file table" duration=45.045µs 2024-09-16T09:26:09.615 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.523315311Z level=info msg="Executing migration" id="migrate contents column to mediumblob for MySQL" 2024-09-16T09:26:09.615 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.523360857Z level=info msg="Migration successfully executed" id="migrate contents column to mediumblob for MySQL" duration=45.457µs 2024-09-16T09:26:09.615 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.524095579Z level=info msg="Executing migration" id="managed permissions migration" 2024-09-16T09:26:09.615 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.524424936Z level=info msg="Migration successfully executed" id="managed permissions migration" duration=327.082µs 2024-09-16T09:26:09.615 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.524829894Z level=info msg="Executing migration" id="managed folder permissions alert actions migration" 2024-09-16T09:26:09.615 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.524988478Z level=info msg="Migration successfully executed" id="managed folder permissions alert actions migration" duration=158.494µs 2024-09-16T09:26:09.616 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.525384568Z level=info msg="Executing migration" id="RBAC action name migrator" 2024-09-16T09:26:09.616 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.526201311Z level=info msg="Migration successfully executed" id="RBAC action name migrator" duration=816.353µs 2024-09-16T09:26:09.616 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.526605493Z level=info msg="Executing migration" id="Add UID column to playlist" 2024-09-16T09:26:09.616 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.531450536Z level=info msg="Migration successfully executed" id="Add UID column to playlist" duration=4.844656ms 2024-09-16T09:26:09.616 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.531822814Z level=info msg="Executing migration" id="Update uid column values in playlist" 2024-09-16T09:26:09.616 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.531952856Z level=info msg="Migration successfully executed" id="Update uid column values in playlist" duration=130.089µs 2024-09-16T09:26:09.616 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.532632073Z level=info msg="Executing migration" id="Add index for uid in playlist" 2024-09-16T09:26:09.616 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.533396357Z level=info msg="Migration successfully executed" id="Add index for uid in playlist" duration=764.162µs 2024-09-16T09:26:09.616 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.53378255Z level=info msg="Executing migration" id="update group index for alert rules" 2024-09-16T09:26:09.616 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.534088549Z level=info msg="Migration successfully executed" id="update group index for alert rules" duration=308.102µs 2024-09-16T09:26:09.616 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.534546135Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated migration" 2024-09-16T09:26:09.616 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.534719456Z level=info msg="Migration successfully executed" id="managed folder permissions alert actions repeated migration" duration=173.016µs 2024-09-16T09:26:09.616 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.535129758Z level=info msg="Executing migration" id="admin only folder/dashboard permission" 2024-09-16T09:26:09.617 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.535432173Z level=info msg="Migration successfully executed" id="admin only folder/dashboard permission" duration=302.345µs 2024-09-16T09:26:09.617 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.535959135Z level=info msg="Executing migration" id="add action column to seed_assignment" 2024-09-16T09:26:09.617 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.540986199Z level=info msg="Migration successfully executed" id="add action column to seed_assignment" duration=5.026635ms 2024-09-16T09:26:09.617 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.541353018Z level=info msg="Executing migration" id="add scope column to seed_assignment" 2024-09-16T09:26:09.617 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.546379855Z level=info msg="Migration successfully executed" id="add scope column to seed_assignment" duration=5.026301ms 2024-09-16T09:26:09.617 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.546749195Z level=info msg="Executing migration" id="remove unique index builtin_role_role_name before nullable update" 2024-09-16T09:26:09.617 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.547453878Z level=info msg="Migration successfully executed" id="remove unique index builtin_role_role_name before nullable update" duration=704.704µs 2024-09-16T09:26:09.617 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.547859564Z level=info msg="Executing migration" id="update seed_assignment role_name column to nullable" 2024-09-16T09:26:09.617 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.603010267Z level=info msg="Migration successfully executed" id="update seed_assignment role_name column to nullable" duration=55.150368ms 2024-09-16T09:26:09.617 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.603380931Z level=info msg="Executing migration" id="add unique index builtin_role_name back" 2024-09-16T09:26:09.617 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.604173733Z level=info msg="Migration successfully executed" id="add unique index builtin_role_name back" duration=792.631µs 2024-09-16T09:26:09.617 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.604555042Z level=info msg="Executing migration" id="add unique index builtin_role_action_scope" 2024-09-16T09:26:09.617 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.605284589Z level=info msg="Migration successfully executed" id="add unique index builtin_role_action_scope" duration=729.256µs 2024-09-16T09:26:09.617 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.605657142Z level=info msg="Executing migration" id="add primary key to seed_assigment" 2024-09-16T09:26:09.618 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:26:09.638 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:09.639 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:09.639 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:26:09.740 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:26:09.762 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:09.762 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:09.763 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:26:09.860 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.622269595Z level=info msg="Migration successfully executed" id="add primary key to seed_assigment" duration=16.611357ms 2024-09-16T09:26:09.860 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.62270304Z level=info msg="Executing migration" id="add origin column to seed_assignment" 2024-09-16T09:26:09.860 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.627884527Z level=info msg="Migration successfully executed" id="add origin column to seed_assignment" duration=5.18102ms 2024-09-16T09:26:09.860 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.628278245Z level=info msg="Executing migration" id="add origin to plugin seed_assignment" 2024-09-16T09:26:09.860 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.628492832Z level=info msg="Migration successfully executed" id="add origin to plugin seed_assignment" duration=214.989µs 2024-09-16T09:26:09.860 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.630964299Z level=info msg="Executing migration" id="prevent seeding OnCall access" 2024-09-16T09:26:09.860 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.63109462Z level=info msg="Migration successfully executed" id="prevent seeding OnCall access" duration=130.309µs 2024-09-16T09:26:09.861 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.633644564Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated fixed migration" 2024-09-16T09:26:09.861 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.63380495Z level=info msg="Migration successfully executed" id="managed folder permissions alert actions repeated fixed migration" duration=159.65µs 2024-09-16T09:26:09.861 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.634202373Z level=info msg="Executing migration" id="managed folder permissions library panel actions migration" 2024-09-16T09:26:09.861 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.634330018Z level=info msg="Migration successfully executed" id="managed folder permissions library panel actions migration" duration=124.629µs 2024-09-16T09:26:09.861 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.63687823Z level=info msg="Executing migration" id="migrate external alertmanagers to datsourcse" 2024-09-16T09:26:09.861 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.637047743Z level=info msg="Migration successfully executed" id="migrate external alertmanagers to datsourcse" duration=170.005µs 2024-09-16T09:26:09.861 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.639664441Z level=info msg="Executing migration" id="create folder table" 2024-09-16T09:26:09.861 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.640314991Z level=info msg="Migration successfully executed" id="create folder table" duration=650.469µs 2024-09-16T09:26:09.861 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.642366334Z level=info msg="Executing migration" id="Add index for parent_uid" 2024-09-16T09:26:09.861 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.643196539Z level=info msg="Migration successfully executed" id="Add index for parent_uid" duration=829.939µs 2024-09-16T09:26:09.862 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.643576359Z level=info msg="Executing migration" id="Add unique index for folder.uid and folder.org_id" 2024-09-16T09:26:09.862 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.644308332Z level=info msg="Migration successfully executed" id="Add unique index for folder.uid and folder.org_id" duration=731.753µs 2024-09-16T09:26:09.862 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.646518723Z level=info msg="Executing migration" id="Update folder title length" 2024-09-16T09:26:09.862 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.646564112Z level=info msg="Migration successfully executed" id="Update folder title length" duration=45.382µs 2024-09-16T09:26:09.862 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.648863012Z level=info msg="Executing migration" id="Add unique index for folder.title and folder.parent_uid" 2024-09-16T09:26:09.862 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.649619299Z level=info msg="Migration successfully executed" id="Add unique index for folder.title and folder.parent_uid" duration=754.663µs 2024-09-16T09:26:09.862 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.65185058Z level=info msg="Executing migration" id="Remove unique index for folder.title and folder.parent_uid" 2024-09-16T09:26:09.863 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.652589573Z level=info msg="Migration successfully executed" id="Remove unique index for folder.title and folder.parent_uid" duration=738.855µs 2024-09-16T09:26:09.863 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.65294929Z level=info msg="Executing migration" id="Add unique index for title, parent_uid, and org_id" 2024-09-16T09:26:09.863 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.653683157Z level=info msg="Migration successfully executed" id="Add unique index for title, parent_uid, and org_id" duration=734.115µs 2024-09-16T09:26:09.863 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.655772636Z level=info msg="Executing migration" id="Sync dashboard and folder table" 2024-09-16T09:26:09.863 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.65607802Z level=info msg="Migration successfully executed" id="Sync dashboard and folder table" duration=305.16µs 2024-09-16T09:26:09.863 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.658833502Z level=info msg="Executing migration" id="Remove ghost folders from the folder table" 2024-09-16T09:26:09.863 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.659120161Z level=info msg="Migration successfully executed" id="Remove ghost folders from the folder table" duration=288.962µs 2024-09-16T09:26:09.863 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.661341138Z level=info msg="Executing migration" id="Remove unique index UQE_folder_uid_org_id" 2024-09-16T09:26:09.863 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.662035036Z level=info msg="Migration successfully executed" id="Remove unique index UQE_folder_uid_org_id" duration=693.775µs 2024-09-16T09:26:09.863 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.662393283Z level=info msg="Executing migration" id="Add unique index UQE_folder_org_id_uid" 2024-09-16T09:26:09.863 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.663096223Z level=info msg="Migration successfully executed" id="Add unique index UQE_folder_org_id_uid" duration=702.675µs 2024-09-16T09:26:09.863 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.665224334Z level=info msg="Executing migration" id="Remove unique index UQE_folder_title_parent_uid_org_id" 2024-09-16T09:26:09.863 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.665921363Z level=info msg="Migration successfully executed" id="Remove unique index UQE_folder_title_parent_uid_org_id" duration=697.867µs 2024-09-16T09:26:09.864 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.667847501Z level=info msg="Executing migration" id="Add unique index UQE_folder_org_id_parent_uid_title" 2024-09-16T09:26:09.864 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.668566626Z level=info msg="Migration successfully executed" id="Add unique index UQE_folder_org_id_parent_uid_title" duration=718.914µs 2024-09-16T09:26:09.864 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.670540432Z level=info msg="Executing migration" id="Remove index IDX_folder_parent_uid_org_id" 2024-09-16T09:26:09.864 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.671274559Z level=info msg="Migration successfully executed" id="Remove index IDX_folder_parent_uid_org_id" duration=736.427µs 2024-09-16T09:26:09.864 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.67164579Z level=info msg="Executing migration" id="create anon_device table" 2024-09-16T09:26:09.864 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.672211361Z level=info msg="Migration successfully executed" id="create anon_device table" duration=565.442µs 2024-09-16T09:26:09.864 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.674313729Z level=info msg="Executing migration" id="add unique index anon_device.device_id" 2024-09-16T09:26:09.864 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.67513564Z level=info msg="Migration successfully executed" id="add unique index anon_device.device_id" duration=824.087µs 2024-09-16T09:26:09.864 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.676659348Z level=info msg="Executing migration" id="add index anon_device.updated_at" 2024-09-16T09:26:09.864 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.677374464Z level=info msg="Migration successfully executed" id="add index anon_device.updated_at" duration=714.726µs 2024-09-16T09:26:09.864 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.679037913Z level=info msg="Executing migration" id="create signing_key table" 2024-09-16T09:26:09.864 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.679710851Z level=info msg="Migration successfully executed" id="create signing_key table" duration=671.197µs 2024-09-16T09:26:09.864 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.681701304Z level=info msg="Executing migration" id="add unique index signing_key.key_id" 2024-09-16T09:26:09.865 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.68251559Z level=info msg="Migration successfully executed" id="add unique index signing_key.key_id" duration=816.911µs 2024-09-16T09:26:09.865 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.684154117Z level=info msg="Executing migration" id="set legacy alert migration status in kvstore" 2024-09-16T09:26:09.865 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.684928423Z level=info msg="Migration successfully executed" id="set legacy alert migration status in kvstore" duration=774.236µs 2024-09-16T09:26:09.865 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.686545995Z level=info msg="Executing migration" id="migrate record of created folders during legacy migration to kvstore" 2024-09-16T09:26:09.865 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.6867522Z level=info msg="Migration successfully executed" id="migrate record of created folders during legacy migration to kvstore" duration=206.708µs 2024-09-16T09:26:09.865 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.689255258Z level=info msg="Executing migration" id="Add folder_uid for dashboard" 2024-09-16T09:26:09.865 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.694396484Z level=info msg="Migration successfully executed" id="Add folder_uid for dashboard" duration=5.140792ms 2024-09-16T09:26:09.865 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.694793368Z level=info msg="Executing migration" id="Populate dashboard folder_uid column" 2024-09-16T09:26:09.865 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.695288284Z level=info msg="Migration successfully executed" id="Populate dashboard folder_uid column" duration=495.631µs 2024-09-16T09:26:09.865 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.697464426Z level=info msg="Executing migration" id="Add unique index for dashboard_org_id_folder_uid_title" 2024-09-16T09:26:09.865 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.698257473Z level=info msg="Migration successfully executed" id="Add unique index for dashboard_org_id_folder_uid_title" duration=795.235µs 2024-09-16T09:26:09.865 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.699721801Z level=info msg="Executing migration" id="Delete unique index for dashboard_org_id_folder_id_title" 2024-09-16T09:26:09.865 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.700447462Z level=info msg="Migration successfully executed" id="Delete unique index for dashboard_org_id_folder_id_title" duration=725.575µs 2024-09-16T09:26:09.866 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.702265748Z level=info msg="Executing migration" id="Delete unique index for dashboard_org_id_folder_uid_title" 2024-09-16T09:26:09.866 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.702960665Z level=info msg="Migration successfully executed" id="Delete unique index for dashboard_org_id_folder_uid_title" duration=694.912µs 2024-09-16T09:26:09.866 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.70335013Z level=info msg="Executing migration" id="Add unique index for dashboard_org_id_folder_uid_title_is_folder" 2024-09-16T09:26:09.866 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.704093335Z level=info msg="Migration successfully executed" id="Add unique index for dashboard_org_id_folder_uid_title_is_folder" duration=742.917µs 2024-09-16T09:26:09.866 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.705862276Z level=info msg="Executing migration" id="Restore index for dashboard_org_id_folder_id_title" 2024-09-16T09:26:09.866 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.706860449Z level=info msg="Migration successfully executed" id="Restore index for dashboard_org_id_folder_id_title" duration=997.849µs 2024-09-16T09:26:09.866 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:26:09.867 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.708416249Z level=info msg="Executing migration" id="create sso_setting table" 2024-09-16T09:26:09.867 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.709152926Z level=info msg="Migration successfully executed" id="create sso_setting table" duration=736.366µs 2024-09-16T09:26:09.868 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.710861287Z level=info msg="Executing migration" id="copy kvstore migration status to each org" 2024-09-16T09:26:09.868 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.711396268Z level=info msg="Migration successfully executed" id="copy kvstore migration status to each org" duration=535.928µs 2024-09-16T09:26:09.868 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.711762704Z level=info msg="Executing migration" id="add back entry for orgid=0 migrated status" 2024-09-16T09:26:09.868 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.711959188Z level=info msg="Migration successfully executed" id="add back entry for orgid=0 migrated status" duration=195.667µs 2024-09-16T09:26:09.868 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.71252045Z level=info msg="Executing migration" id="alter kv_store.value to longtext" 2024-09-16T09:26:09.868 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.71256371Z level=info msg="Migration successfully executed" id="alter kv_store.value to longtext" duration=43.652µs 2024-09-16T09:26:09.868 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.713016165Z level=info msg="Executing migration" id="add notification_settings column to alert_rule table" 2024-09-16T09:26:09.868 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.719486357Z level=info msg="Migration successfully executed" id="add notification_settings column to alert_rule table" duration=6.469223ms 2024-09-16T09:26:09.868 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.719865443Z level=info msg="Executing migration" id="add notification_settings column to alert_rule_version table" 2024-09-16T09:26:09.868 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.72521735Z level=info msg="Migration successfully executed" id="add notification_settings column to alert_rule_version table" duration=5.351508ms 2024-09-16T09:26:09.868 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.725560307Z level=info msg="Executing migration" id="removing scope from alert.instances:read action migration" 2024-09-16T09:26:09.868 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.725768478Z level=info msg="Migration successfully executed" id="removing scope from alert.instances:read action migration" duration=208.147µs 2024-09-16T09:26:09.869 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=migrator t=2024-09-16T09:26:09.726130234Z level=info msg="migrations completed" performed=547 skipped=0 duration=1.256698772s 2024-09-16T09:26:09.869 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=sqlstore t=2024-09-16T09:26:09.727016741Z level=info msg="Created default organization" 2024-09-16T09:26:09.869 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=secrets t=2024-09-16T09:26:09.727527298Z level=info msg="Envelope encryption state" enabled=true currentprovider=secretKey.v1 2024-09-16T09:26:09.869 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=plugin.store t=2024-09-16T09:26:09.745110833Z level=info msg="Loading plugins..." 2024-09-16T09:26:09.869 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=local.finder t=2024-09-16T09:26:09.784643272Z level=warn msg="Skipping finding plugins as directory does not exist" path=/usr/share/grafana/plugins-bundled 2024-09-16T09:26:09.869 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=plugin.store t=2024-09-16T09:26:09.784660245Z level=info msg="Plugins loaded" count=55 duration=39.550132ms 2024-09-16T09:26:09.869 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=query_data t=2024-09-16T09:26:09.786734334Z level=info msg="Query Service initialization" 2024-09-16T09:26:09.869 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=live.push_http t=2024-09-16T09:26:09.789401833Z level=info msg="Live Push Gateway initialization" 2024-09-16T09:26:09.869 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=ngalert.migration t=2024-09-16T09:26:09.790511168Z level=info msg=Starting 2024-09-16T09:26:09.869 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=ngalert.migration t=2024-09-16T09:26:09.790793427Z level=info msg="Applying transition" currentType=Legacy desiredType=UnifiedAlerting cleanOnDowngrade=false cleanOnUpgrade=false 2024-09-16T09:26:09.869 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=ngalert.migration orgID=1 t=2024-09-16T09:26:09.791098691Z level=info msg="Migrating alerts for organisation" 2024-09-16T09:26:09.869 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=ngalert.migration orgID=1 t=2024-09-16T09:26:09.791547893Z level=info msg="Alerts found to migrate" alerts=0 2024-09-16T09:26:09.869 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=ngalert.migration t=2024-09-16T09:26:09.792750663Z level=info msg="Completed alerting migration" 2024-09-16T09:26:09.870 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=ngalert.state.manager t=2024-09-16T09:26:09.803295682Z level=info msg="Running in alternative execution of Error/NoData mode" 2024-09-16T09:26:09.870 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=infra.usagestats.collector t=2024-09-16T09:26:09.804693144Z level=info msg="registering usage stat providers" usageStatsProvidersLen=2 2024-09-16T09:26:09.870 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=provisioning.datasources t=2024-09-16T09:26:09.806365108Z level=info msg="inserting datasource from configuration" name=Dashboard1 uid=P43CA22E17D0F9596 2024-09-16T09:26:09.870 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=provisioning.datasources t=2024-09-16T09:26:09.818533937Z level=info msg="inserting datasource from configuration" name=Loki uid=P8E80F9AEF21F6940 2024-09-16T09:26:09.870 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=provisioning.alerting t=2024-09-16T09:26:09.831363199Z level=info msg="starting to provision alerting" 2024-09-16T09:26:09.870 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=provisioning.alerting t=2024-09-16T09:26:09.83137325Z level=info msg="finished to provision alerting" 2024-09-16T09:26:09.870 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=grafanaStorageLogger t=2024-09-16T09:26:09.831537318Z level=info msg="Storage starting" 2024-09-16T09:26:09.870 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=ngalert.state.manager t=2024-09-16T09:26:09.832171865Z level=info msg="Warming state cache for startup" 2024-09-16T09:26:09.870 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=http.server t=2024-09-16T09:26:09.833982928Z level=info msg="HTTP Server TLS settings" MinTLSVersion=TLS1.2 configuredciphers=TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA,TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA,TLS_RSA_WITH_AES_128_GCM_SHA256,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_128_CBC_SHA,TLS_RSA_WITH_AES_256_CBC_SHA 2024-09-16T09:26:09.870 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=http.server t=2024-09-16T09:26:09.834224524Z level=info msg="HTTP Server Listen" address=[::]:3000 protocol=https subUrl= socket= 2024-09-16T09:26:09.870 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=ngalert.multiorg.alertmanager t=2024-09-16T09:26:09.834544457Z level=info msg="Starting MultiOrg Alertmanager" 2024-09-16T09:26:09.870 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=ngalert.state.manager t=2024-09-16T09:26:09.848350899Z level=info msg="State cache has been initialized" states=0 duration=16.177927ms 2024-09-16T09:26:09.870 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=ngalert.scheduler t=2024-09-16T09:26:09.848371524Z level=info msg="Starting scheduler" tickInterval=10s maxAttempts=1 2024-09-16T09:26:09.871 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=ticker t=2024-09-16T09:26:09.848400703Z level=info msg=starting first_tick=2024-09-16T09:26:10Z 2024-09-16T09:26:09.871 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=provisioning.dashboard t=2024-09-16T09:26:09.85164879Z level=info msg="starting to provision dashboards" 2024-09-16T09:26:09.887 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:09.887 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:09.888 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:26:09.988 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:26:10.010 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:10.011 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:10.011 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:26:10.043 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:26:10.044 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:26:10.112 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:26:10.119 INFO:teuthology.orchestra.run.smithi070.stderr:enabled application 'rados' on pool 'unique_pool_0' 2024-09-16T09:26:10.120 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=sqlstore.transactions t=2024-09-16T09:26:09.858886312Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-16T09:26:10.120 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=sqlstore.transactions t=2024-09-16T09:26:09.872606084Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=1 code="database is locked" 2024-09-16T09:26:10.120 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=sqlstore.transactions t=2024-09-16T09:26:09.88313661Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=2 code="database is locked" 2024-09-16T09:26:10.120 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=plugins.update.checker t=2024-09-16T09:26:09.912420904Z level=info msg="Update check succeeded" duration=80.856659ms 2024-09-16T09:26:10.120 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=grafana.update.checker t=2024-09-16T09:26:09.920904802Z level=info msg="Update check succeeded" duration=89.357608ms 2024-09-16T09:26:10.120 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=sqlstore.transactions t=2024-09-16T09:26:09.936134797Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-16T09:26:10.120 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:10 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=provisioning.dashboard t=2024-09-16T09:26:10.023787491Z level=info msg="finished to provision dashboards" 2024-09-16T09:26:10.134 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:10.135 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:10.135 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:26:10.236 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:26:10.248 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:10 smithi070 ceph-mon[30388]: Deploying daemon node-exporter.a on smithi070 2024-09-16T09:26:10.248 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:10 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3414681429' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true} : dispatch 2024-09-16T09:26:10.249 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:10 smithi070 ceph-mon[40691]: Deploying daemon node-exporter.a on smithi070 2024-09-16T09:26:10.249 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:10 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3414681429' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true} : dispatch 2024-09-16T09:26:10.260 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:10.260 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:10.261 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:26:10.361 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:26:10.383 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:10.384 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:10.384 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:26:10.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:10 smithi084 ceph-mon[32629]: Deploying daemon node-exporter.a on smithi070 2024-09-16T09:26:10.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:10 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3414681429' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true} : dispatch 2024-09-16T09:26:10.410 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:10 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=grafana-apiserver t=2024-09-16T09:26:10.278898426Z level=info msg="Adding GroupVersion playlist.grafana.app v0alpha1 to ResourceManager" 2024-09-16T09:26:10.410 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:10 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=grafana-apiserver t=2024-09-16T09:26:10.279337987Z level=info msg="Adding GroupVersion featuretoggle.grafana.app v0alpha1 to ResourceManager" 2024-09-16T09:26:10.485 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:26:10.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:10.510 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:10.511 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:26:10.611 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:26:10.633 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:10.634 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:10.634 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:26:10.735 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:26:10.750 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T09:26:10.750 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T09:26:10.750 INFO:teuthology.orchestra.run.smithi070.stdout: "complaint_time": 30, 2024-09-16T09:26:10.750 INFO:teuthology.orchestra.run.smithi070.stdout: "num_blocked_ops": 0 2024-09-16T09:26:10.751 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T09:26:10.757 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:10.758 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:10.758 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:26:10.859 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:26:10.881 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:10.882 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:10.882 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:26:10.983 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:26:11.005 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:11.007 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:11.007 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:26:11.108 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:26:11.129 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:11.130 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:11.131 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:26:11.231 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:26:11.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:11.255 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:11.255 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:26:11.356 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:26:11.379 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:11.380 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:11.380 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:26:11.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:11 smithi084 ceph-mon[32629]: pgmap v33: 148 pgs: 16 creating+peering, 132 active+clean; 583 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-16T09:26:11.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:11 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3414681429' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-16T09:26:11.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:11 smithi084 ceph-mon[32629]: osdmap e85: 8 total, 8 up, 8 in 2024-09-16T09:26:11.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:11 smithi070 ceph-mon[30388]: pgmap v33: 148 pgs: 16 creating+peering, 132 active+clean; 583 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-16T09:26:11.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:11 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3414681429' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-16T09:26:11.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:11 smithi070 ceph-mon[30388]: osdmap e85: 8 total, 8 up, 8 in 2024-09-16T09:26:11.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:11 smithi070 ceph-mon[40691]: pgmap v33: 148 pgs: 16 creating+peering, 132 active+clean; 583 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-16T09:26:11.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:11 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3414681429' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-16T09:26:11.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:11 smithi070 ceph-mon[40691]: osdmap e85: 8 total, 8 up, 8 in 2024-09-16T09:26:11.482 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:26:11.504 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:11.505 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:11.505 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:26:11.571 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.0 dump_historic_ops 2024-09-16T09:26:11.606 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:26:11.640 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:11.640 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:11.641 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:26:11.742 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:26:11.765 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:11.765 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:11.766 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:26:11.866 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:26:11.889 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:11.889 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:11.890 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:26:11.990 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:26:12.001 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T09:26:12.012 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:12.013 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:12.013 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:26:12.114 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:26:12.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:12.139 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:12.139 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:26:12.168 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:26:11 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:26:11] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:26:12.239 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:26:12.262 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:12.262 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:12.263 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:26:12.363 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:26:12.391 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:12.391 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:12.392 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:26:12.452 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:26:12.455 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:26:12.493 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:26:12.519 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:12.519 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:12.520 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:26:12.621 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:26:12.645 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:12.646 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:12.646 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:26:12.747 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:26:12.772 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:12.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:12.774 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:26:12.782 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:26:12 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:26:12.875 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:26:12.900 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:12.901 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:12.901 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:26:12.924 DEBUG:teuthology.orchestra.run.smithi070:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.0 -b 65536 --object-size 65536 -p unique_pool_0 bench 90 write' 2024-09-16T09:26:12.925 INFO:tasks.thrashosds.thrasher:Growing pool .mgr 2024-09-16T09:26:12.925 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-16T09:26:12.927 INFO:tasks.radosbench:joining radosbench (timing out after 3000s) 2024-09-16T09:26:12.927 INFO:teuthology.orchestra.run:waiting for 3000 2024-09-16T09:26:13.002 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:hints = 1 2024-09-16T09:26:13.002 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Maintaining 16 concurrent writes of 65536 bytes to objects of size 65536 for up to 90 seconds or 0 objects 2024-09-16T09:26:13.002 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Object prefix: benchmark_data_smithi070_78438 2024-09-16T09:26:13.003 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-16T09:26:13.003 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 0 0 0 0 0 0 - 0 2024-09-16T09:26:13.003 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:26:13.040 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:13.041 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:13.041 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:26:13.126 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:12 smithi070 ceph-mon[30388]: pgmap v35: 148 pgs: 16 creating+peering, 132 active+clean; 583 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-16T09:26:13.126 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:12 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:26:13.126 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:12 smithi070 ceph-mon[30388]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-16T09:26:13.126 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:12 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/441486071' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:26:13.126 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:12 smithi070 ceph-mon[40691]: pgmap v35: 148 pgs: 16 creating+peering, 132 active+clean; 583 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-16T09:26:13.126 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:12 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:26:13.126 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:12 smithi070 ceph-mon[40691]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-16T09:26:13.127 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:12 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/441486071' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:26:13.127 INFO:teuthology.orchestra.run.smithi070.stderr:nodeep-scrub is set 2024-09-16T09:26:13.142 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:26:13.159 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:12 smithi084 ceph-mon[32629]: pgmap v35: 148 pgs: 16 creating+peering, 132 active+clean; 583 KiB data, 236 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-16T09:26:13.159 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:12 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:26:13.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:12 smithi084 ceph-mon[32629]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-16T09:26:13.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:12 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/441486071' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:26:13.198 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:13.199 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:13.199 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:26:13.300 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:26:13.344 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:13.345 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:13.345 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:26:13.446 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:26:13.476 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:13.477 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:13.477 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:26:13.578 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:26:13.610 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:13.611 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:13.611 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:26:13.648 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:26:13.648 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:26:13.648 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:26:13.648 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:26:13.648 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:26:13.648 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:26:13.649 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:26:13.649 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:26:13.649 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:26:13.649 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:26:13.649 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:26:13.649 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:26:13.649 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:26:13.649 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:26:13.650 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:26:13.650 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:26:13.712 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:26:13.747 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:13.748 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:13.748 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:26:13.849 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:26:13.889 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:13.890 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:13.890 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:26:13.991 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:26:14.005 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 1 16 3122 3106 193.659 194.125 0.0036176 0.00513574 2024-09-16T09:26:14.033 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:14.034 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:14.034 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:26:14.135 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:26:14.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:13 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:26:14.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:13 smithi084 ceph-mon[32629]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:26:14.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:13 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/441486071' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-16T09:26:14.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:13 smithi084 ceph-mon[32629]: osdmap e86: 8 total, 8 up, 8 in 2024-09-16T09:26:14.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:13 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/656557078' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:26:14.170 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:13 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:26:14.170 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:13 smithi070 ceph-mon[30388]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:26:14.170 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:13 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/441486071' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-16T09:26:14.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:13 smithi070 ceph-mon[30388]: osdmap e86: 8 total, 8 up, 8 in 2024-09-16T09:26:14.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:13 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/656557078' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:26:14.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:13 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:26:14.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:13 smithi070 ceph-mon[40691]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:26:14.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:13 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/441486071' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-16T09:26:14.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:13 smithi070 ceph-mon[40691]: osdmap e86: 8 total, 8 up, 8 in 2024-09-16T09:26:14.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:13 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/656557078' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:26:14.171 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:14.172 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:14.172 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:26:14.273 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:26:14.313 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:14.313 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:14.314 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:26:14.415 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:26:14.457 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:14.458 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:14.458 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:26:14.559 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:26:14.587 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:14.588 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:14.588 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:26:14.635 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T09:26:14.636 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:26:14.689 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:26:14.721 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:14.721 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:14.722 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:26:14.823 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:26:14.853 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:14.854 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:14.855 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:26:14.956 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:26:14.984 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:14.984 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:14.985 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:26:15.005 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 2 16 6192 6176 192.759 191.875 0.00463917 0.00517876 2024-09-16T09:26:15.085 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:26:15.116 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:15.235 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:15.235 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:26:15.335 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:26:15.371 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:15.372 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:15.372 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:26:15.409 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:15 smithi084 ceph-mon[32629]: pgmap v37: 148 pgs: 148 active+clean; 583 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 951 B/s rd, 0 op/s 2024-09-16T09:26:15.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:15 smithi070 ceph-mon[30388]: pgmap v37: 148 pgs: 148 active+clean; 583 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 951 B/s rd, 0 op/s 2024-09-16T09:26:15.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:15 smithi070 ceph-mon[40691]: pgmap v37: 148 pgs: 148 active+clean; 583 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 951 B/s rd, 0 op/s 2024-09-16T09:26:15.421 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Sep 16 09:26:15 smithi070 bash[78349]: Copying config sha256:72c9c208898624938c9e4183d6686ea4a5fd3f912bc29bc3f00147924c521a3e 2024-09-16T09:26:15.474 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:26:15.512 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:15.512 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:15.513 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:26:15.613 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:26:15.647 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:15.647 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:15.648 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:26:15.749 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:26:15.787 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:15.788 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:15.788 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:26:15.889 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:26:15.920 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Sep 16 09:26:15 smithi070 bash[78349]: Writing manifest to image destination 2024-09-16T09:26:15.923 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:15.924 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:15.924 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:26:16.005 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 3 16 9434 9418 196.039 202.625 0.00277903 0.00509265 2024-09-16T09:26:16.025 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:26:16.063 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:16.064 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:16.064 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:26:16.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:16 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:26:16.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:16 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:26:16.165 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:26:16.196 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:16.196 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:16.197 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:26:16.225 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:16 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:26:16.225 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:16 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:26:16.225 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:16 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:26:16.226 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:16 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:26:16.297 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:26:16.339 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:16.339 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:16.340 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:26:16.441 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:26:16.470 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:16.471 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:16.471 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:26:16.572 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:26:16.608 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:16.609 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:16.609 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:26:16.670 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Sep 16 09:26:16 smithi070 podman[78349]: 2024-09-16 09:26:16.124839844 +0000 UTC m=+3.727800039 image pull 72c9c208898624938c9e4183d6686ea4a5fd3f912bc29bc3f00147924c521a3e quay.io/prometheus/node-exporter:v1.7.0 2024-09-16T09:26:16.671 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Sep 16 09:26:16 smithi070 podman[78349]: 2024-09-16 09:26:16.455657357 +0000 UTC m=+4.058617542 container create 6857b9d39120ce21a2664f78ee4619e48110658158fe29999f581a467133512b (image=quay.io/prometheus/node-exporter:v1.7.0, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-16T09:26:16.712 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:26:16.745 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:16.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:16.746 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:26:16.847 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:26:16.886 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:16.887 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:16.887 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:26:16.988 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:26:17.005 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 4 16 12436 12420 193.933 187.625 0.00555439 0.00514881 2024-09-16T09:26:17.029 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:17.030 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:17.031 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:26:17.132 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:26:17.164 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:17.165 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:17.166 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:26:17.266 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:26:17.297 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:17.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:17.298 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:26:17.399 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:26:17.409 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:17 smithi084 ceph-mon[32629]: pgmap v38: 148 pgs: 148 active+clean; 583 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 800 B/s rd, 0 op/s 2024-09-16T09:26:17.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:17 smithi084 ceph-mon[32629]: osdmap e87: 8 total, 8 up, 8 in 2024-09-16T09:26:17.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:17 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:26:17.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:17 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:26:17.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:17 smithi070 ceph-mon[30388]: pgmap v38: 148 pgs: 148 active+clean; 583 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 800 B/s rd, 0 op/s 2024-09-16T09:26:17.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:17 smithi070 ceph-mon[30388]: osdmap e87: 8 total, 8 up, 8 in 2024-09-16T09:26:17.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:17 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:26:17.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:17 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:26:17.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:17 smithi070 ceph-mon[40691]: pgmap v38: 148 pgs: 148 active+clean; 583 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 800 B/s rd, 0 op/s 2024-09-16T09:26:17.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:17 smithi070 ceph-mon[40691]: osdmap e87: 8 total, 8 up, 8 in 2024-09-16T09:26:17.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:17 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:26:17.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:17 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:26:17.436 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:17.437 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:17.437 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:26:17.538 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:26:17.577 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:17.578 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:17.578 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:26:17.679 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:26:17.715 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:17.716 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:17.716 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:26:17.817 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:26:17.854 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:17.855 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:17.856 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:26:17.956 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:26:17.978 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T09:26:17.979 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:17.979 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:17.979 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:26:18.005 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 5 16 15079 15063 188.184 165.188 0.00139776 0.00525627 2024-09-16T09:26:18.081 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:26:18.126 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:18.126 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:18.126 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:26:18.227 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:26:18.253 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:18.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:18.254 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:26:18.355 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:26:18.375 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd unset noscrub 2024-09-16T09:26:18.390 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:18.391 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:18.391 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:26:18.493 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:26:18.533 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:18.534 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:18.534 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:26:18.635 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:26:18.674 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:18.675 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:18.675 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:26:18.776 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:26:18.820 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:18.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:18.821 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:26:18.922 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:26:18.970 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:18.971 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:18.972 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:26:19.005 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 6 16 17448 17432 181.497 148.062 0.0071492 0.00550327 2024-09-16T09:26:19.072 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:26:19.106 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:19.107 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:19.107 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:26:19.147 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Sep 16 09:26:18 smithi070 podman[78349]: 2024-09-16 09:26:18.804832347 +0000 UTC m=+6.407792542 container init 6857b9d39120ce21a2664f78ee4619e48110658158fe29999f581a467133512b (image=quay.io/prometheus/node-exporter:v1.7.0, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-16T09:26:19.147 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Sep 16 09:26:18 smithi070 podman[78349]: 2024-09-16 09:26:18.818275336 +0000 UTC m=+6.421235523 container start 6857b9d39120ce21a2664f78ee4619e48110658158fe29999f581a467133512b (image=quay.io/prometheus/node-exporter:v1.7.0, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-16T09:26:19.147 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Sep 16 09:26:18 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-a[78886]: ts=2024-09-16T09:26:18.825Z caller=node_exporter.go:192 level=info msg="Starting node_exporter" version="(version=1.7.0, branch=HEAD, revision=7333465abf9efba81876303bb57e6fadb946041b)" 2024-09-16T09:26:19.147 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Sep 16 09:26:18 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-a[78886]: ts=2024-09-16T09:26:18.825Z caller=node_exporter.go:193 level=info msg="Build context" build_context="(go=go1.21.4, platform=linux/amd64, user=root@35918982f6d8, date=20231112-23:53:35, tags=netgo osusergo static_build)" 2024-09-16T09:26:19.147 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Sep 16 09:26:18 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-a[78886]: ts=2024-09-16T09:26:18.828Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-09-16T09:26:19.147 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Sep 16 09:26:18 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-a[78886]: ts=2024-09-16T09:26:18.828Z caller=diskstats_linux.go:265 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-09-16T09:26:19.147 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Sep 16 09:26:18 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-a[78886]: ts=2024-09-16T09:26:18.829Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-09-16T09:26:19.148 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Sep 16 09:26:18 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-a[78886]: ts=2024-09-16T09:26:18.829Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-09-16T09:26:19.148 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Sep 16 09:26:18 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-a[78886]: ts=2024-09-16T09:26:18.829Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-09-16T09:26:19.148 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Sep 16 09:26:18 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-a[78886]: ts=2024-09-16T09:26:18.829Z caller=node_exporter.go:117 level=info collector=arp 2024-09-16T09:26:19.148 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Sep 16 09:26:18 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-a[78886]: ts=2024-09-16T09:26:18.829Z caller=node_exporter.go:117 level=info collector=bcache 2024-09-16T09:26:19.148 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Sep 16 09:26:18 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-a[78886]: ts=2024-09-16T09:26:18.829Z caller=node_exporter.go:117 level=info collector=bonding 2024-09-16T09:26:19.148 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Sep 16 09:26:18 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-a[78886]: ts=2024-09-16T09:26:18.829Z caller=node_exporter.go:117 level=info collector=btrfs 2024-09-16T09:26:19.148 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Sep 16 09:26:18 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-a[78886]: ts=2024-09-16T09:26:18.829Z caller=node_exporter.go:117 level=info collector=conntrack 2024-09-16T09:26:19.148 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Sep 16 09:26:18 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-a[78886]: ts=2024-09-16T09:26:18.829Z caller=node_exporter.go:117 level=info collector=cpu 2024-09-16T09:26:19.148 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Sep 16 09:26:18 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-a[78886]: ts=2024-09-16T09:26:18.829Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-09-16T09:26:19.148 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Sep 16 09:26:18 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-a[78886]: ts=2024-09-16T09:26:18.829Z caller=node_exporter.go:117 level=info collector=diskstats 2024-09-16T09:26:19.148 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Sep 16 09:26:18 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-a[78886]: ts=2024-09-16T09:26:18.829Z caller=node_exporter.go:117 level=info collector=dmi 2024-09-16T09:26:19.148 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Sep 16 09:26:18 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-a[78886]: ts=2024-09-16T09:26:18.829Z caller=node_exporter.go:117 level=info collector=edac 2024-09-16T09:26:19.149 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Sep 16 09:26:18 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-a[78886]: ts=2024-09-16T09:26:18.830Z caller=node_exporter.go:117 level=info collector=entropy 2024-09-16T09:26:19.149 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Sep 16 09:26:18 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-a[78886]: ts=2024-09-16T09:26:18.830Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-09-16T09:26:19.149 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Sep 16 09:26:18 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-a[78886]: ts=2024-09-16T09:26:18.830Z caller=node_exporter.go:117 level=info collector=filefd 2024-09-16T09:26:19.149 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Sep 16 09:26:18 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-a[78886]: ts=2024-09-16T09:26:18.830Z caller=node_exporter.go:117 level=info collector=filesystem 2024-09-16T09:26:19.149 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Sep 16 09:26:18 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-a[78886]: ts=2024-09-16T09:26:18.830Z caller=node_exporter.go:117 level=info collector=hwmon 2024-09-16T09:26:19.149 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Sep 16 09:26:18 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-a[78886]: ts=2024-09-16T09:26:18.830Z caller=node_exporter.go:117 level=info collector=infiniband 2024-09-16T09:26:19.149 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Sep 16 09:26:18 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-a[78886]: ts=2024-09-16T09:26:18.830Z caller=node_exporter.go:117 level=info collector=ipvs 2024-09-16T09:26:19.149 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Sep 16 09:26:18 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-a[78886]: ts=2024-09-16T09:26:18.830Z caller=node_exporter.go:117 level=info collector=loadavg 2024-09-16T09:26:19.149 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Sep 16 09:26:18 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-a[78886]: ts=2024-09-16T09:26:18.830Z caller=node_exporter.go:117 level=info collector=mdadm 2024-09-16T09:26:19.149 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Sep 16 09:26:18 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-a[78886]: ts=2024-09-16T09:26:18.830Z caller=node_exporter.go:117 level=info collector=meminfo 2024-09-16T09:26:19.149 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Sep 16 09:26:18 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-a[78886]: ts=2024-09-16T09:26:18.830Z caller=node_exporter.go:117 level=info collector=netclass 2024-09-16T09:26:19.149 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Sep 16 09:26:18 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-a[78886]: ts=2024-09-16T09:26:18.830Z caller=node_exporter.go:117 level=info collector=netdev 2024-09-16T09:26:19.149 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Sep 16 09:26:18 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-a[78886]: ts=2024-09-16T09:26:18.830Z caller=node_exporter.go:117 level=info collector=netstat 2024-09-16T09:26:19.150 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Sep 16 09:26:18 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-a[78886]: ts=2024-09-16T09:26:18.830Z caller=node_exporter.go:117 level=info collector=nfs 2024-09-16T09:26:19.150 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Sep 16 09:26:18 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-a[78886]: ts=2024-09-16T09:26:18.830Z caller=node_exporter.go:117 level=info collector=nfsd 2024-09-16T09:26:19.150 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Sep 16 09:26:18 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-a[78886]: ts=2024-09-16T09:26:18.830Z caller=node_exporter.go:117 level=info collector=nvme 2024-09-16T09:26:19.150 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Sep 16 09:26:18 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-a[78886]: ts=2024-09-16T09:26:18.830Z caller=node_exporter.go:117 level=info collector=os 2024-09-16T09:26:19.150 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Sep 16 09:26:18 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-a[78886]: ts=2024-09-16T09:26:18.830Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-09-16T09:26:19.150 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Sep 16 09:26:18 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-a[78886]: ts=2024-09-16T09:26:18.830Z caller=node_exporter.go:117 level=info collector=pressure 2024-09-16T09:26:19.150 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Sep 16 09:26:18 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-a[78886]: ts=2024-09-16T09:26:18.830Z caller=node_exporter.go:117 level=info collector=rapl 2024-09-16T09:26:19.150 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Sep 16 09:26:18 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-a[78886]: ts=2024-09-16T09:26:18.830Z caller=node_exporter.go:117 level=info collector=schedstat 2024-09-16T09:26:19.150 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Sep 16 09:26:18 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-a[78886]: ts=2024-09-16T09:26:18.830Z caller=node_exporter.go:117 level=info collector=selinux 2024-09-16T09:26:19.150 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Sep 16 09:26:18 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-a[78886]: ts=2024-09-16T09:26:18.830Z caller=node_exporter.go:117 level=info collector=sockstat 2024-09-16T09:26:19.150 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Sep 16 09:26:18 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-a[78886]: ts=2024-09-16T09:26:18.830Z caller=node_exporter.go:117 level=info collector=softnet 2024-09-16T09:26:19.150 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Sep 16 09:26:18 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-a[78886]: ts=2024-09-16T09:26:18.830Z caller=node_exporter.go:117 level=info collector=stat 2024-09-16T09:26:19.150 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Sep 16 09:26:18 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-a[78886]: ts=2024-09-16T09:26:18.830Z caller=node_exporter.go:117 level=info collector=tapestats 2024-09-16T09:26:19.151 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Sep 16 09:26:18 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-a[78886]: ts=2024-09-16T09:26:18.830Z caller=node_exporter.go:117 level=info collector=textfile 2024-09-16T09:26:19.151 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Sep 16 09:26:18 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-a[78886]: ts=2024-09-16T09:26:18.830Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-09-16T09:26:19.151 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Sep 16 09:26:18 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-a[78886]: ts=2024-09-16T09:26:18.830Z caller=node_exporter.go:117 level=info collector=time 2024-09-16T09:26:19.151 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Sep 16 09:26:18 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-a[78886]: ts=2024-09-16T09:26:18.830Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-09-16T09:26:19.151 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Sep 16 09:26:18 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-a[78886]: ts=2024-09-16T09:26:18.830Z caller=node_exporter.go:117 level=info collector=uname 2024-09-16T09:26:19.151 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Sep 16 09:26:18 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-a[78886]: ts=2024-09-16T09:26:18.830Z caller=node_exporter.go:117 level=info collector=vmstat 2024-09-16T09:26:19.151 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Sep 16 09:26:18 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-a[78886]: ts=2024-09-16T09:26:18.830Z caller=node_exporter.go:117 level=info collector=xfs 2024-09-16T09:26:19.152 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Sep 16 09:26:18 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-a[78886]: ts=2024-09-16T09:26:18.830Z caller=node_exporter.go:117 level=info collector=zfs 2024-09-16T09:26:19.152 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Sep 16 09:26:18 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-a[78886]: ts=2024-09-16T09:26:18.831Z caller=tls_config.go:274 level=info msg="Listening on" address=[::]:9100 2024-09-16T09:26:19.152 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Sep 16 09:26:18 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-a[78886]: ts=2024-09-16T09:26:18.831Z caller=tls_config.go:277 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-09-16T09:26:19.152 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Sep 16 09:26:18 smithi070 bash[78349]: 6857b9d39120ce21a2664f78ee4619e48110658158fe29999f581a467133512b 2024-09-16T09:26:19.152 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Sep 16 09:26:18 smithi070 systemd[1]: Started Ceph node-exporter.a for a1321834-740c-11ef-bceb-c7b262605968. 2024-09-16T09:26:19.208 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:26:19.244 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:19.245 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:19.245 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:26:19.346 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:26:19.397 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:19.398 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:19.399 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:26:19.399 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:19 smithi070 ceph-mon[40691]: pgmap v40: 148 pgs: 148 active+clean; 649 MiB data, 2.1 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 88 MiB/s wr, 1.41k op/s 2024-09-16T09:26:19.399 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:19 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:26:19.399 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:19 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:26:19.399 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:19 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:26:19.400 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:19 smithi070 ceph-mon[40691]: osdmap e88: 8 total, 8 up, 8 in 2024-09-16T09:26:19.400 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:19 smithi070 ceph-mon[30388]: pgmap v40: 148 pgs: 148 active+clean; 649 MiB data, 2.1 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 88 MiB/s wr, 1.41k op/s 2024-09-16T09:26:19.400 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:19 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:26:19.400 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:19 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:26:19.401 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:19 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:26:19.401 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:19 smithi070 ceph-mon[30388]: osdmap e88: 8 total, 8 up, 8 in 2024-09-16T09:26:19.402 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.1 dump_ops_in_flight 2024-09-16T09:26:19.402 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:26:19.409 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:19 smithi084 ceph-mon[32629]: pgmap v40: 148 pgs: 148 active+clean; 649 MiB data, 2.1 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 88 MiB/s wr, 1.41k op/s 2024-09-16T09:26:19.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:19 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:26:19.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:19 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:26:19.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:19 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:26:19.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:19 smithi084 ceph-mon[32629]: osdmap e88: 8 total, 8 up, 8 in 2024-09-16T09:26:19.416 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:26:19.499 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:26:19.553 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:19.553 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:19.554 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:26:19.654 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:26:19.684 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:19.685 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:19.685 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:26:19.786 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:26:19.814 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:19.815 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:19.815 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:26:19.916 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:26:19.947 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:19.947 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:19.948 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:26:20.005 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 7 16 19764 19748 176.247 144.75 0.00416377 0.0056673 2024-09-16T09:26:20.049 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:26:20.087 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:20.088 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:20.089 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:26:20.190 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:26:20.216 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:20.216 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:20.217 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:26:20.317 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:26:20.353 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:20.353 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:20.354 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:26:20.369 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:20 smithi070 ceph-mon[30388]: Deploying daemon node-exporter.b on smithi084 2024-09-16T09:26:20.369 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:20 smithi070 ceph-mon[40691]: Deploying daemon node-exporter.b on smithi084 2024-09-16T09:26:20.409 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:20 smithi084 ceph-mon[32629]: Deploying daemon node-exporter.b on smithi084 2024-09-16T09:26:20.455 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:26:20.494 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:20.495 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:20.495 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:26:20.501 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:26:20.596 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:26:20.622 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:20.622 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:20.623 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:26:20.724 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:26:20.754 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:20.755 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:20.755 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:26:20.856 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:26:20.884 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:20.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:20.886 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:26:20.901 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:26:20.904 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2024-09-16T09:26:20.987 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:26:21.005 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 8 16 21783 21767 169.99 126.188 0.00560901 0.00587727 2024-09-16T09:26:21.025 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:21.025 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:21.026 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:26:21.126 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:26:21.148 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:21.148 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:21.148 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:26:21.212 INFO:teuthology.orchestra.run.smithi070.stderr:noscrub is unset 2024-09-16T09:26:21.249 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:26:21.290 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:21.292 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:21.292 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:26:21.334 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:26:21.334 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:26:21.334 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:26:21.334 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:26:21.334 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:26:21.335 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:26:21.335 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:26:21.335 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:26:21.335 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:26:21.335 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:26:21.358 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:26:21.359 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:26:21.389 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:21 smithi084 ceph-mon[32629]: pgmap v42: 148 pgs: 148 active+clean; 865 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 804 B/s rd, 136 MiB/s wr, 2.17k op/s 2024-09-16T09:26:21.389 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:21 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:26:21.389 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:21 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:26:21.389 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:21 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1433979641' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:26:21.389 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:21 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:26:21.389 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:21 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/2164183279' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:26:21.389 INFO:journalctl@ceph.node-exporter.b.smithi084.stdout:Sep 16 09:26:21 smithi084 systemd[1]: Starting Ceph node-exporter.b for a1321834-740c-11ef-bceb-c7b262605968... 2024-09-16T09:26:21.393 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:26:21.396 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:26:21.397 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:26:21.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:21 smithi070 ceph-mon[30388]: pgmap v42: 148 pgs: 148 active+clean; 865 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 804 B/s rd, 136 MiB/s wr, 2.17k op/s 2024-09-16T09:26:21.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:21 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:26:21.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:21 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:26:21.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:21 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1433979641' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:26:21.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:21 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:26:21.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:21 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2164183279' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:26:21.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:21 smithi070 ceph-mon[40691]: pgmap v42: 148 pgs: 148 active+clean; 865 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 804 B/s rd, 136 MiB/s wr, 2.17k op/s 2024-09-16T09:26:21.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:21 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:26:21.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:21 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:26:21.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:21 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1433979641' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:26:21.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:21 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:26:21.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:21 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/2164183279' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:26:21.430 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:26:21.430 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:26:21.435 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:21.436 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:21.436 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:26:21.537 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:26:21.567 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:21.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:21.568 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:26:21.669 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:26:21.709 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:21.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:21.710 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:26:21.810 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:26:21.848 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:21.849 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:21.850 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:26:21.910 INFO:journalctl@ceph.node-exporter.b.smithi084.stdout:Sep 16 09:26:21 smithi084 bash[64320]: Trying to pull quay.io/prometheus/node-exporter:v1.7.0... 2024-09-16T09:26:21.950 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:26:21.987 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:21.987 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:21.988 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:26:22.005 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 9 16 23632 23616 163.943 115.562 0.00475947 0.00609396 2024-09-16T09:26:22.089 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:26:22.119 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:22.120 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:22.120 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:26:22.221 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:26:22.255 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:22.255 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:22.256 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:26:22.357 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:26:22.391 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:22.392 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:22.392 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:26:22.410 INFO:journalctl@ceph.node-exporter.b.smithi084.stdout:Sep 16 09:26:21 smithi084 bash[64320]: Getting image source signatures 2024-09-16T09:26:22.410 INFO:journalctl@ceph.node-exporter.b.smithi084.stdout:Sep 16 09:26:21 smithi084 bash[64320]: Copying blob sha256:324153f2810a9927fcce320af9e4e291e0b6e805cbdd1f338386c756b9defa24 2024-09-16T09:26:22.410 INFO:journalctl@ceph.node-exporter.b.smithi084.stdout:Sep 16 09:26:21 smithi084 bash[64320]: Copying blob sha256:455fd88e5221bc1e278ef2d059cd70e4df99a24e5af050ede621534276f6cf9a 2024-09-16T09:26:22.410 INFO:journalctl@ceph.node-exporter.b.smithi084.stdout:Sep 16 09:26:21 smithi084 bash[64320]: Copying blob sha256:2abcce694348cd2c949c0e98a7400ebdfd8341021bcf6b541bc72033ce982510 2024-09-16T09:26:22.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:22 smithi084 ceph-mon[32629]: from='client.14835 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:26:22.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:22 smithi084 ceph-mon[32629]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:26:22.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:22 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-16T09:26:22.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:22 smithi084 ceph-mon[32629]: osdmap e89: 8 total, 8 up, 8 in 2024-09-16T09:26:22.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:22 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:26:22.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:22 smithi084 ceph-mon[32629]: 5.18 scrub starts 2024-09-16T09:26:22.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:22 smithi084 ceph-mon[32629]: 5.18 scrub ok 2024-09-16T09:26:22.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:22 smithi070 ceph-mon[30388]: from='client.14835 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:26:22.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:22 smithi070 ceph-mon[30388]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:26:22.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:22 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-16T09:26:22.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:22 smithi070 ceph-mon[30388]: osdmap e89: 8 total, 8 up, 8 in 2024-09-16T09:26:22.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:22 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:26:22.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:22 smithi070 ceph-mon[30388]: 5.18 scrub starts 2024-09-16T09:26:22.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:22 smithi070 ceph-mon[30388]: 5.18 scrub ok 2024-09-16T09:26:22.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:22 smithi070 ceph-mon[40691]: from='client.14835 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:26:22.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:22 smithi070 ceph-mon[40691]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:26:22.425 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:22 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-16T09:26:22.425 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:22 smithi070 ceph-mon[40691]: osdmap e89: 8 total, 8 up, 8 in 2024-09-16T09:26:22.426 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:22 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:26:22.426 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:22 smithi070 ceph-mon[40691]: 5.18 scrub starts 2024-09-16T09:26:22.426 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:22 smithi070 ceph-mon[40691]: 5.18 scrub ok 2024-09-16T09:26:22.426 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:26:21 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:26:21] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:26:22.493 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:26:22.523 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:22.524 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:22.525 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:26:22.628 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:26:22.668 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:22.669 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:22.669 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:26:22.770 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:26:22.810 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:22.811 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:22.811 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:26:22.910 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:26:22 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:26:22.911 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:26:22.942 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:22.943 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:22.943 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:26:23.006 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 10 16 26030 26014 162.535 149.875 0.006565 0.0061465 2024-09-16T09:26:23.044 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:26:23.082 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:23.082 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:23.083 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:26:23.183 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:26:23.220 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:23.221 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:23.221 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:26:23.322 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:26:23.350 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:23.351 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:23.352 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:26:23.453 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:26:23.494 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:23.495 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:23.496 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:26:23.530 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:23 smithi070 ceph-mon[30388]: pgmap v44: 148 pgs: 148 active+clean; 865 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 144 MiB/s wr, 2.30k op/s 2024-09-16T09:26:23.531 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:23 smithi070 ceph-mon[30388]: 3.16 scrub starts 2024-09-16T09:26:23.531 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:23 smithi070 ceph-mon[30388]: 3.16 scrub ok 2024-09-16T09:26:23.531 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:23 smithi070 ceph-mon[30388]: 6.e scrub starts 2024-09-16T09:26:23.531 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:23 smithi070 ceph-mon[30388]: 6.e scrub ok 2024-09-16T09:26:23.531 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:23 smithi070 ceph-mon[40691]: pgmap v44: 148 pgs: 148 active+clean; 865 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 144 MiB/s wr, 2.30k op/s 2024-09-16T09:26:23.531 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:23 smithi070 ceph-mon[40691]: 3.16 scrub starts 2024-09-16T09:26:23.531 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:23 smithi070 ceph-mon[40691]: 3.16 scrub ok 2024-09-16T09:26:23.531 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:23 smithi070 ceph-mon[40691]: 6.e scrub starts 2024-09-16T09:26:23.531 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:23 smithi070 ceph-mon[40691]: 6.e scrub ok 2024-09-16T09:26:23.597 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:26:23.651 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:23.652 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:23.652 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:26:23.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:23 smithi084 ceph-mon[32629]: pgmap v44: 148 pgs: 148 active+clean; 865 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 144 MiB/s wr, 2.30k op/s 2024-09-16T09:26:23.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:23 smithi084 ceph-mon[32629]: 3.16 scrub starts 2024-09-16T09:26:23.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:23 smithi084 ceph-mon[32629]: 3.16 scrub ok 2024-09-16T09:26:23.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:23 smithi084 ceph-mon[32629]: 6.e scrub starts 2024-09-16T09:26:23.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:23 smithi084 ceph-mon[32629]: 6.e scrub ok 2024-09-16T09:26:23.752 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:26:23.774 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:23.775 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:23.775 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:26:23.876 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:26:23.911 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:23.912 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:23.912 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:26:24.006 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 11 16 27978 27962 158.827 121.75 0.010772 0.00629004 2024-09-16T09:26:24.013 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:26:24.036 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:24.037 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:24.037 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:26:24.138 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:26:24.180 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:24.181 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:24.181 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:26:24.282 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:26:24.322 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:24.323 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:24.323 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:26:24.388 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:24 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:26:24.388 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:24 smithi070 ceph-mon[30388]: 3.1d scrub starts 2024-09-16T09:26:24.388 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:24 smithi070 ceph-mon[30388]: 3.1d scrub ok 2024-09-16T09:26:24.388 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:24 smithi070 ceph-mon[30388]: 4.f scrub starts 2024-09-16T09:26:24.389 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:24 smithi070 ceph-mon[30388]: 4.f scrub ok 2024-09-16T09:26:24.389 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:24 smithi070 ceph-mon[30388]: 3.17 scrub starts 2024-09-16T09:26:24.389 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:24 smithi070 ceph-mon[30388]: 3.17 scrub ok 2024-09-16T09:26:24.389 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:24 smithi070 ceph-mon[30388]: 3.15 scrub starts 2024-09-16T09:26:24.389 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:24 smithi070 ceph-mon[30388]: 3.15 scrub ok 2024-09-16T09:26:24.389 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:24 smithi070 ceph-mon[30388]: 6.17 scrub starts 2024-09-16T09:26:24.389 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:24 smithi070 ceph-mon[30388]: 6.17 scrub ok 2024-09-16T09:26:24.389 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:24 smithi070 ceph-mon[30388]: osdmap e90: 8 total, 8 up, 8 in 2024-09-16T09:26:24.389 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:24 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:26:24.389 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:24 smithi070 ceph-mon[40691]: 3.1d scrub starts 2024-09-16T09:26:24.389 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:24 smithi070 ceph-mon[40691]: 3.1d scrub ok 2024-09-16T09:26:24.389 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:24 smithi070 ceph-mon[40691]: 4.f scrub starts 2024-09-16T09:26:24.390 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:24 smithi070 ceph-mon[40691]: 4.f scrub ok 2024-09-16T09:26:24.390 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:24 smithi070 ceph-mon[40691]: 3.17 scrub starts 2024-09-16T09:26:24.390 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:24 smithi070 ceph-mon[40691]: 3.17 scrub ok 2024-09-16T09:26:24.390 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:24 smithi070 ceph-mon[40691]: 3.15 scrub starts 2024-09-16T09:26:24.390 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:24 smithi070 ceph-mon[40691]: 3.15 scrub ok 2024-09-16T09:26:24.390 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:24 smithi070 ceph-mon[40691]: 6.17 scrub starts 2024-09-16T09:26:24.390 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:24 smithi070 ceph-mon[40691]: 6.17 scrub ok 2024-09-16T09:26:24.390 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:24 smithi070 ceph-mon[40691]: osdmap e90: 8 total, 8 up, 8 in 2024-09-16T09:26:24.425 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:26:24.449 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:26:24.473 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:24.473 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:24.477 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:26:24.578 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:26:24.611 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:24.611 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:24.612 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:26:24.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:24 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:26:24.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:24 smithi084 ceph-mon[32629]: 3.1d scrub starts 2024-09-16T09:26:24.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:24 smithi084 ceph-mon[32629]: 3.1d scrub ok 2024-09-16T09:26:24.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:24 smithi084 ceph-mon[32629]: 4.f scrub starts 2024-09-16T09:26:24.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:24 smithi084 ceph-mon[32629]: 4.f scrub ok 2024-09-16T09:26:24.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:24 smithi084 ceph-mon[32629]: 3.17 scrub starts 2024-09-16T09:26:24.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:24 smithi084 ceph-mon[32629]: 3.17 scrub ok 2024-09-16T09:26:24.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:24 smithi084 ceph-mon[32629]: 3.15 scrub starts 2024-09-16T09:26:24.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:24 smithi084 ceph-mon[32629]: 3.15 scrub ok 2024-09-16T09:26:24.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:24 smithi084 ceph-mon[32629]: 6.17 scrub starts 2024-09-16T09:26:24.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:24 smithi084 ceph-mon[32629]: 6.17 scrub ok 2024-09-16T09:26:24.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:24 smithi084 ceph-mon[32629]: osdmap e90: 8 total, 8 up, 8 in 2024-09-16T09:26:24.713 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:26:24.743 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:24.744 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:24.744 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:26:24.845 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:26:24.873 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:24.874 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:24.874 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:26:24.975 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:26:25.006 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 12 16 29735 29719 154.743 109.812 0.0114438 0.00645572 2024-09-16T09:26:25.010 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:25.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:25.011 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:26:25.112 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:26:25.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:25.145 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:25.146 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:26:25.247 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:26:25.272 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:25.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:25.273 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:26:25.374 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:26:25.406 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:25.406 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:25.407 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:26:25.507 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:26:25.531 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:25.532 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:25.532 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:26:25.574 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-16T09:26:25.609 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:25 smithi070 ceph-mon[30388]: 4.17 scrub starts 2024-09-16T09:26:25.609 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:25 smithi070 ceph-mon[30388]: 4.17 scrub ok 2024-09-16T09:26:25.609 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:25 smithi070 ceph-mon[30388]: pgmap v45: 148 pgs: 148 active+clean; 1.5 GiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 1.4 KiB/s rd, 211 MiB/s wr, 3.38k op/s 2024-09-16T09:26:25.609 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:25 smithi070 ceph-mon[30388]: 4.1f scrub starts 2024-09-16T09:26:25.609 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:25 smithi070 ceph-mon[30388]: 4.1f scrub ok 2024-09-16T09:26:25.609 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:25 smithi070 ceph-mon[30388]: 6.4 scrub starts 2024-09-16T09:26:25.610 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:25 smithi070 ceph-mon[30388]: 6.4 scrub ok 2024-09-16T09:26:25.610 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:25 smithi070 ceph-mon[30388]: 6.18 scrub starts 2024-09-16T09:26:25.610 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:25 smithi070 ceph-mon[30388]: 6.18 scrub ok 2024-09-16T09:26:25.610 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:25 smithi070 ceph-mon[30388]: 5.d scrub starts 2024-09-16T09:26:25.610 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:25 smithi070 ceph-mon[30388]: 5.d scrub ok 2024-09-16T09:26:25.610 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:25 smithi070 ceph-mon[40691]: 4.17 scrub starts 2024-09-16T09:26:25.610 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:25 smithi070 ceph-mon[40691]: 4.17 scrub ok 2024-09-16T09:26:25.610 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:25 smithi070 ceph-mon[40691]: pgmap v45: 148 pgs: 148 active+clean; 1.5 GiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 1.4 KiB/s rd, 211 MiB/s wr, 3.38k op/s 2024-09-16T09:26:25.610 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:25 smithi070 ceph-mon[40691]: 4.1f scrub starts 2024-09-16T09:26:25.610 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:25 smithi070 ceph-mon[40691]: 4.1f scrub ok 2024-09-16T09:26:25.610 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:25 smithi070 ceph-mon[40691]: 6.4 scrub starts 2024-09-16T09:26:25.610 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:25 smithi070 ceph-mon[40691]: 6.4 scrub ok 2024-09-16T09:26:25.611 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:25 smithi070 ceph-mon[40691]: 6.18 scrub starts 2024-09-16T09:26:25.611 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:25 smithi070 ceph-mon[40691]: 6.18 scrub ok 2024-09-16T09:26:25.611 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:25 smithi070 ceph-mon[40691]: 5.d scrub starts 2024-09-16T09:26:25.611 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:25 smithi070 ceph-mon[40691]: 5.d scrub ok 2024-09-16T09:26:25.632 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:26:25.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:25 smithi084 ceph-mon[32629]: 4.17 scrub starts 2024-09-16T09:26:25.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:25 smithi084 ceph-mon[32629]: 4.17 scrub ok 2024-09-16T09:26:25.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:25 smithi084 ceph-mon[32629]: pgmap v45: 148 pgs: 148 active+clean; 1.5 GiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 1.4 KiB/s rd, 211 MiB/s wr, 3.38k op/s 2024-09-16T09:26:25.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:25 smithi084 ceph-mon[32629]: 4.1f scrub starts 2024-09-16T09:26:25.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:25 smithi084 ceph-mon[32629]: 4.1f scrub ok 2024-09-16T09:26:25.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:25 smithi084 ceph-mon[32629]: 6.4 scrub starts 2024-09-16T09:26:25.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:25 smithi084 ceph-mon[32629]: 6.4 scrub ok 2024-09-16T09:26:25.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:25 smithi084 ceph-mon[32629]: 6.18 scrub starts 2024-09-16T09:26:25.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:25 smithi084 ceph-mon[32629]: 6.18 scrub ok 2024-09-16T09:26:25.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:25 smithi084 ceph-mon[32629]: 5.d scrub starts 2024-09-16T09:26:25.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:25 smithi084 ceph-mon[32629]: 5.d scrub ok 2024-09-16T09:26:25.671 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:25.672 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:25.672 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:26:25.773 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:26:25.816 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:25.817 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:25.817 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:26:25.918 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:26:25.948 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:25.948 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:25.949 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:26:26.006 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 13 16 31547 31531 151.551 113.25 0.00496081 0.00659295 2024-09-16T09:26:26.049 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:26:26.082 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:26.082 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:26.084 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:26:26.160 INFO:journalctl@ceph.node-exporter.b.smithi084.stdout:Sep 16 09:26:25 smithi084 bash[64320]: Copying config sha256:72c9c208898624938c9e4183d6686ea4a5fd3f912bc29bc3f00147924c521a3e 2024-09-16T09:26:26.160 INFO:journalctl@ceph.node-exporter.b.smithi084.stdout:Sep 16 09:26:25 smithi084 bash[64320]: Writing manifest to image destination 2024-09-16T09:26:26.184 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:26:26.214 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:26.215 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:26.215 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:26:26.316 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:26:26.352 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:26.353 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:26.353 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:26:26.427 INFO:journalctl@ceph.node-exporter.b.smithi084.stdout:Sep 16 09:26:26 smithi084 podman[64320]: 2024-09-16 09:26:26.154617649 +0000 UTC m=+4.622053033 image pull 72c9c208898624938c9e4183d6686ea4a5fd3f912bc29bc3f00147924c521a3e quay.io/prometheus/node-exporter:v1.7.0 2024-09-16T09:26:26.427 INFO:journalctl@ceph.node-exporter.b.smithi084.stdout:Sep 16 09:26:26 smithi084 podman[64320]: 2024-09-16 09:26:26.289544093 +0000 UTC m=+4.756979509 container create f12d713979d1922a6136120d97c2b707cbb7655c5517be5c89dfc636e2a8b641 (image=quay.io/prometheus/node-exporter:v1.7.0, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-16T09:26:26.427 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:26 smithi084 ceph-mon[32629]: 6.6 scrub starts 2024-09-16T09:26:26.428 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:26 smithi084 ceph-mon[32629]: 6.6 scrub ok 2024-09-16T09:26:26.428 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:26 smithi084 ceph-mon[32629]: 4.a scrub starts 2024-09-16T09:26:26.428 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:26 smithi084 ceph-mon[32629]: 4.a scrub ok 2024-09-16T09:26:26.428 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:26 smithi084 ceph-mon[32629]: 3.5 scrub starts 2024-09-16T09:26:26.428 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:26 smithi084 ceph-mon[32629]: 3.5 scrub ok 2024-09-16T09:26:26.444 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T09:26:26.454 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:26:26.485 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:26 smithi070 ceph-mon[30388]: 6.6 scrub starts 2024-09-16T09:26:26.485 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:26 smithi070 ceph-mon[30388]: 6.6 scrub ok 2024-09-16T09:26:26.485 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:26 smithi070 ceph-mon[30388]: 4.a scrub starts 2024-09-16T09:26:26.485 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:26 smithi070 ceph-mon[30388]: 4.a scrub ok 2024-09-16T09:26:26.485 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:26 smithi070 ceph-mon[30388]: 3.5 scrub starts 2024-09-16T09:26:26.485 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:26 smithi070 ceph-mon[30388]: 3.5 scrub ok 2024-09-16T09:26:26.486 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:26 smithi070 ceph-mon[40691]: 6.6 scrub starts 2024-09-16T09:26:26.486 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:26 smithi070 ceph-mon[40691]: 6.6 scrub ok 2024-09-16T09:26:26.486 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:26 smithi070 ceph-mon[40691]: 4.a scrub starts 2024-09-16T09:26:26.486 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:26 smithi070 ceph-mon[40691]: 4.a scrub ok 2024-09-16T09:26:26.486 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:26 smithi070 ceph-mon[40691]: 3.5 scrub starts 2024-09-16T09:26:26.486 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:26 smithi070 ceph-mon[40691]: 3.5 scrub ok 2024-09-16T09:26:26.539 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:26.540 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:26.540 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:26:26.641 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:26:26.654 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":42,"stamp":"2024-09-16T09:26:19.491273+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":906631707,"num_objects":14024,"num_object_clones":0,"num_object_copies":42072,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14024,"num_whiteouts":0,"num_read":918,"num_read_kb":775,"num_write":14374,"num_write_kb":886473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":14377,"ondisk_log_size":14377,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":444,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":2554236,"kb_used_data":2300572,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":747406980,"statfs":{"total":767960285184,"available":765344747520,"internally_reserved":0,"allocated":2355785728,"data_stored":2351496361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":259575346},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":906035200,"num_objects":13825,"num_object_clones":0,"num_object_copies":41475,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13825,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":13825,"num_write_kb":884800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.367199"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":68,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.149741+0000","last_change":"2024-09-16T09:25:01.709392+0000","last_active":"2024-09-16T09:26:16.149741+0000","last_peered":"2024-09-16T09:26:16.149741+0000","last_clean":"2024-09-16T09:26:16.149741+0000","last_became_active":"2024-09-16T09:25:01.709108+0000","last_became_peered":"2024-09-16T09:25:01.709108+0000","last_unstale":"2024-09-16T09:26:16.149741+0000","last_undegraded":"2024-09-16T09:26:16.149741+0000","last_fullsized":"2024-09-16T09:26:16.149741+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:06.619436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"66'9","reported_seq":80,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.142710+0000","last_change":"2024-09-16T09:25:03.715139+0000","last_active":"2024-09-16T09:26:16.142710+0000","last_peered":"2024-09-16T09:26:16.142710+0000","last_clean":"2024-09-16T09:26:16.142710+0000","last_became_active":"2024-09-16T09:25:03.714635+0000","last_became_peered":"2024-09-16T09:25:03.714635+0000","last_unstale":"2024-09-16T09:26:16.142710+0000","last_undegraded":"2024-09-16T09:26:16.142710+0000","last_fullsized":"2024-09-16T09:26:16.142710+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:19.723608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":60,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.154188+0000","last_change":"2024-09-16T09:25:05.716139+0000","last_active":"2024-09-16T09:26:16.154188+0000","last_peered":"2024-09-16T09:26:16.154188+0000","last_clean":"2024-09-16T09:26:16.154188+0000","last_became_active":"2024-09-16T09:25:05.715645+0000","last_became_peered":"2024-09-16T09:25:05.715645+0000","last_unstale":"2024-09-16T09:26:16.154188+0000","last_undegraded":"2024-09-16T09:26:16.154188+0000","last_fullsized":"2024-09-16T09:26:16.154188+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:31.956608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":56,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.142997+0000","last_change":"2024-09-16T09:25:07.715923+0000","last_active":"2024-09-16T09:26:16.142997+0000","last_peered":"2024-09-16T09:26:16.142997+0000","last_clean":"2024-09-16T09:26:16.142997+0000","last_became_active":"2024-09-16T09:25:07.715442+0000","last_became_peered":"2024-09-16T09:25:07.715442+0000","last_unstale":"2024-09-16T09:26:16.142997+0000","last_undegraded":"2024-09-16T09:26:16.142997+0000","last_fullsized":"2024-09-16T09:26:16.142997+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:35.494958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":56,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.141936+0000","last_change":"2024-09-16T09:25:07.718687+0000","last_active":"2024-09-16T09:26:16.141936+0000","last_peered":"2024-09-16T09:26:16.141936+0000","last_clean":"2024-09-16T09:26:16.141936+0000","last_became_active":"2024-09-16T09:25:07.718126+0000","last_became_peered":"2024-09-16T09:25:07.718126+0000","last_unstale":"2024-09-16T09:26:16.141936+0000","last_undegraded":"2024-09-16T09:26:16.141936+0000","last_fullsized":"2024-09-16T09:26:16.141936+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:23.444563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":68,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.146207+0000","last_change":"2024-09-16T09:25:01.708505+0000","last_active":"2024-09-16T09:26:16.146207+0000","last_peered":"2024-09-16T09:26:16.146207+0000","last_clean":"2024-09-16T09:26:16.146207+0000","last_became_active":"2024-09-16T09:25:01.708179+0000","last_became_peered":"2024-09-16T09:25:01.708179+0000","last_unstale":"2024-09-16T09:26:16.146207+0000","last_undegraded":"2024-09-16T09:26:16.146207+0000","last_fullsized":"2024-09-16T09:26:16.146207+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:22.060114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"66'15","reported_seq":89,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.142320+0000","last_change":"2024-09-16T09:25:03.709970+0000","last_active":"2024-09-16T09:26:16.142320+0000","last_peered":"2024-09-16T09:26:16.142320+0000","last_clean":"2024-09-16T09:26:16.142320+0000","last_became_active":"2024-09-16T09:25:03.709499+0000","last_became_peered":"2024-09-16T09:25:03.709499+0000","last_unstale":"2024-09-16T09:26:16.142320+0000","last_undegraded":"2024-09-16T09:26:16.142320+0000","last_fullsized":"2024-09-16T09:26:16.142320+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:09.600196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":60,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.142645+0000","last_change":"2024-09-16T09:25:05.717524+0000","last_active":"2024-09-16T09:26:16.142645+0000","last_peered":"2024-09-16T09:26:16.142645+0000","last_clean":"2024-09-16T09:26:16.142645+0000","last_became_active":"2024-09-16T09:25:05.717236+0000","last_became_peered":"2024-09-16T09:25:05.717236+0000","last_unstale":"2024-09-16T09:26:16.142645+0000","last_undegraded":"2024-09-16T09:26:16.142645+0000","last_fullsized":"2024-09-16T09:26:16.142645+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":67,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.147764+0000","last_change":"2024-09-16T09:25:01.705552+0000","last_active":"2024-09-16T09:26:16.147764+0000","last_peered":"2024-09-16T09:26:16.147764+0000","last_clean":"2024-09-16T09:26:16.147764+0000","last_became_active":"2024-09-16T09:25:01.705269+0000","last_became_peered":"2024-09-16T09:25:01.705269+0000","last_unstale":"2024-09-16T09:26:16.147764+0000","last_undegraded":"2024-09-16T09:26:16.147764+0000","last_fullsized":"2024-09-16T09:26:16.147764+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:18.445329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"66'9","reported_seq":80,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.143209+0000","last_change":"2024-09-16T09:25:03.709441+0000","last_active":"2024-09-16T09:26:16.143209+0000","last_peered":"2024-09-16T09:26:16.143209+0000","last_clean":"2024-09-16T09:26:16.143209+0000","last_became_active":"2024-09-16T09:25:03.709147+0000","last_became_peered":"2024-09-16T09:25:03.709147+0000","last_unstale":"2024-09-16T09:26:16.143209+0000","last_undegraded":"2024-09-16T09:26:16.143209+0000","last_fullsized":"2024-09-16T09:26:16.143209+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:21.309548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":59,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.147166+0000","last_change":"2024-09-16T09:25:05.718510+0000","last_active":"2024-09-16T09:26:16.147166+0000","last_peered":"2024-09-16T09:26:16.147166+0000","last_clean":"2024-09-16T09:26:16.147166+0000","last_became_active":"2024-09-16T09:25:05.718134+0000","last_became_peered":"2024-09-16T09:25:05.718134+0000","last_unstale":"2024-09-16T09:26:16.147166+0000","last_undegraded":"2024-09-16T09:26:16.147166+0000","last_fullsized":"2024-09-16T09:26:16.147166+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:30.593901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"66'1","reported_seq":57,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.147737+0000","last_change":"2024-09-16T09:25:07.716869+0000","last_active":"2024-09-16T09:26:16.147737+0000","last_peered":"2024-09-16T09:26:16.147737+0000","last_clean":"2024-09-16T09:26:16.147737+0000","last_became_active":"2024-09-16T09:25:07.716558+0000","last_became_peered":"2024-09-16T09:25:07.716558+0000","last_unstale":"2024-09-16T09:26:16.147737+0000","last_undegraded":"2024-09-16T09:26:16.147737+0000","last_fullsized":"2024-09-16T09:26:16.147737+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:17.035822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":67,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.147149+0000","last_change":"2024-09-16T09:25:01.697306+0000","last_active":"2024-09-16T09:26:16.147149+0000","last_peered":"2024-09-16T09:26:16.147149+0000","last_clean":"2024-09-16T09:26:16.147149+0000","last_became_active":"2024-09-16T09:25:01.696939+0000","last_became_peered":"2024-09-16T09:25:01.696939+0000","last_unstale":"2024-09-16T09:26:16.147149+0000","last_undegraded":"2024-09-16T09:26:16.147149+0000","last_fullsized":"2024-09-16T09:26:16.147149+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:26.024926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"66'5","reported_seq":74,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.143531+0000","last_change":"2024-09-16T09:25:03.716402+0000","last_active":"2024-09-16T09:26:16.143531+0000","last_peered":"2024-09-16T09:26:16.143531+0000","last_clean":"2024-09-16T09:26:16.143531+0000","last_became_active":"2024-09-16T09:25:03.716136+0000","last_became_peered":"2024-09-16T09:25:03.716136+0000","last_unstale":"2024-09-16T09:26:16.143531+0000","last_undegraded":"2024-09-16T09:26:16.143531+0000","last_fullsized":"2024-09-16T09:26:16.143531+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:32.360413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":59,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.147857+0000","last_change":"2024-09-16T09:25:05.718538+0000","last_active":"2024-09-16T09:26:16.147857+0000","last_peered":"2024-09-16T09:26:16.147857+0000","last_clean":"2024-09-16T09:26:16.147857+0000","last_became_active":"2024-09-16T09:25:05.718281+0000","last_became_peered":"2024-09-16T09:25:05.718281+0000","last_unstale":"2024-09-16T09:26:16.147857+0000","last_undegraded":"2024-09-16T09:26:16.147857+0000","last_fullsized":"2024-09-16T09:26:16.147857+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:29.527824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":55,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.146514+0000","last_change":"2024-09-16T09:25:07.717644+0000","last_active":"2024-09-16T09:26:16.146514+0000","last_peered":"2024-09-16T09:26:16.146514+0000","last_clean":"2024-09-16T09:26:16.146514+0000","last_became_active":"2024-09-16T09:25:07.716724+0000","last_became_peered":"2024-09-16T09:25:07.716724+0000","last_unstale":"2024-09-16T09:26:16.146514+0000","last_undegraded":"2024-09-16T09:26:16.146514+0000","last_fullsized":"2024-09-16T09:26:16.146514+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:32.145892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":56,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.144431+0000","last_change":"2024-09-16T09:25:07.721088+0000","last_active":"2024-09-16T09:26:16.144431+0000","last_peered":"2024-09-16T09:26:16.144431+0000","last_clean":"2024-09-16T09:26:16.144431+0000","last_became_active":"2024-09-16T09:25:07.720793+0000","last_became_peered":"2024-09-16T09:25:07.720793+0000","last_unstale":"2024-09-16T09:26:16.144431+0000","last_undegraded":"2024-09-16T09:26:16.144431+0000","last_fullsized":"2024-09-16T09:26:16.144431+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:16.293244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":68,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.149073+0000","last_change":"2024-09-16T09:25:01.706762+0000","last_active":"2024-09-16T09:26:16.149073+0000","last_peered":"2024-09-16T09:26:16.149073+0000","last_clean":"2024-09-16T09:26:16.149073+0000","last_became_active":"2024-09-16T09:25:01.706172+0000","last_became_peered":"2024-09-16T09:25:01.706172+0000","last_unstale":"2024-09-16T09:26:16.149073+0000","last_undegraded":"2024-09-16T09:26:16.149073+0000","last_fullsized":"2024-09-16T09:26:16.149073+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:08.677453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"66'15","reported_seq":89,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.145032+0000","last_change":"2024-09-16T09:25:03.714261+0000","last_active":"2024-09-16T09:26:16.145032+0000","last_peered":"2024-09-16T09:26:16.145032+0000","last_clean":"2024-09-16T09:26:16.145032+0000","last_became_active":"2024-09-16T09:25:03.713780+0000","last_became_peered":"2024-09-16T09:25:03.713780+0000","last_unstale":"2024-09-16T09:26:16.145032+0000","last_undegraded":"2024-09-16T09:26:16.145032+0000","last_fullsized":"2024-09-16T09:26:16.145032+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:29.826960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":60,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.155397+0000","last_change":"2024-09-16T09:25:05.714209+0000","last_active":"2024-09-16T09:26:16.155397+0000","last_peered":"2024-09-16T09:26:16.155397+0000","last_clean":"2024-09-16T09:26:16.155397+0000","last_became_active":"2024-09-16T09:25:05.713929+0000","last_became_peered":"2024-09-16T09:25:05.713929+0000","last_unstale":"2024-09-16T09:26:16.155397+0000","last_undegraded":"2024-09-16T09:26:16.155397+0000","last_fullsized":"2024-09-16T09:26:16.155397+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:09.142498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":56,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.145922+0000","last_change":"2024-09-16T09:25:07.719645+0000","last_active":"2024-09-16T09:26:16.145922+0000","last_peered":"2024-09-16T09:26:16.145922+0000","last_clean":"2024-09-16T09:26:16.145922+0000","last_became_active":"2024-09-16T09:25:07.719067+0000","last_became_peered":"2024-09-16T09:25:07.719067+0000","last_unstale":"2024-09-16T09:26:16.145922+0000","last_undegraded":"2024-09-16T09:26:16.145922+0000","last_fullsized":"2024-09-16T09:26:16.145922+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:19.447150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":68,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.143368+0000","last_change":"2024-09-16T09:25:01.706828+0000","last_active":"2024-09-16T09:26:16.143368+0000","last_peered":"2024-09-16T09:26:16.143368+0000","last_clean":"2024-09-16T09:26:16.143368+0000","last_became_active":"2024-09-16T09:25:01.706491+0000","last_became_peered":"2024-09-16T09:25:01.706491+0000","last_unstale":"2024-09-16T09:26:16.143368+0000","last_undegraded":"2024-09-16T09:26:16.143368+0000","last_fullsized":"2024-09-16T09:26:16.143368+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:16.321414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"66'12","reported_seq":87,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.145919+0000","last_change":"2024-09-16T09:25:03.709914+0000","last_active":"2024-09-16T09:26:16.145919+0000","last_peered":"2024-09-16T09:26:16.145919+0000","last_clean":"2024-09-16T09:26:16.145919+0000","last_became_active":"2024-09-16T09:25:03.709610+0000","last_became_peered":"2024-09-16T09:25:03.709610+0000","last_unstale":"2024-09-16T09:26:16.145919+0000","last_undegraded":"2024-09-16T09:26:16.145919+0000","last_fullsized":"2024-09-16T09:26:16.145919+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:26.267096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":60,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.143866+0000","last_change":"2024-09-16T09:25:05.710379+0000","last_active":"2024-09-16T09:26:16.143866+0000","last_peered":"2024-09-16T09:26:16.143866+0000","last_clean":"2024-09-16T09:26:16.143866+0000","last_became_active":"2024-09-16T09:25:05.710093+0000","last_became_peered":"2024-09-16T09:25:05.710093+0000","last_unstale":"2024-09-16T09:26:16.143866+0000","last_undegraded":"2024-09-16T09:26:16.143866+0000","last_fullsized":"2024-09-16T09:26:16.143866+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:28.802108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":56,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.147554+0000","last_change":"2024-09-16T09:25:07.719655+0000","last_active":"2024-09-16T09:26:16.147554+0000","last_peered":"2024-09-16T09:26:16.147554+0000","last_clean":"2024-09-16T09:26:16.147554+0000","last_became_active":"2024-09-16T09:25:07.719380+0000","last_became_peered":"2024-09-16T09:25:07.719380+0000","last_unstale":"2024-09-16T09:26:16.147554+0000","last_undegraded":"2024-09-16T09:26:16.147554+0000","last_fullsized":"2024-09-16T09:26:16.147554+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:36.059179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":69,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.154298+0000","last_change":"2024-09-16T09:25:01.700263+0000","last_active":"2024-09-16T09:26:16.154298+0000","last_peered":"2024-09-16T09:26:16.154298+0000","last_clean":"2024-09-16T09:26:16.154298+0000","last_became_active":"2024-09-16T09:25:01.699545+0000","last_became_peered":"2024-09-16T09:25:01.699545+0000","last_unstale":"2024-09-16T09:26:16.154298+0000","last_undegraded":"2024-09-16T09:26:16.154298+0000","last_fullsized":"2024-09-16T09:26:16.154298+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:06.109126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"66'10","reported_seq":79,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.150038+0000","last_change":"2024-09-16T09:25:03.720638+0000","last_active":"2024-09-16T09:26:16.150038+0000","last_peered":"2024-09-16T09:26:16.150038+0000","last_clean":"2024-09-16T09:26:16.150038+0000","last_became_active":"2024-09-16T09:25:03.720245+0000","last_became_peered":"2024-09-16T09:25:03.720245+0000","last_unstale":"2024-09-16T09:26:16.150038+0000","last_undegraded":"2024-09-16T09:26:16.150038+0000","last_fullsized":"2024-09-16T09:26:16.150038+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:21.451538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"66'8","reported_seq":87,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.151603+0000","last_change":"2024-09-16T09:25:05.713310+0000","last_active":"2024-09-16T09:26:16.151603+0000","last_peered":"2024-09-16T09:26:16.151603+0000","last_clean":"2024-09-16T09:26:16.151603+0000","last_became_active":"2024-09-16T09:25:05.712516+0000","last_became_peered":"2024-09-16T09:25:05.712516+0000","last_unstale":"2024-09-16T09:26:16.151603+0000","last_undegraded":"2024-09-16T09:26:16.151603+0000","last_fullsized":"2024-09-16T09:26:16.151603+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:24.505283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.c","version":"87'851","reported_seq":872,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:17.482721+0000","last_change":"2024-09-16T09:26:08.117066+0000","last_active":"2024-09-16T09:26:17.482721+0000","last_peered":"2024-09-16T09:26:17.482721+0000","last_clean":"2024-09-16T09:26:17.482721+0000","last_became_active":"2024-09-16T09:26:08.116783+0000","last_became_peered":"2024-09-16T09:26:08.116783+0000","last_unstale":"2024-09-16T09:26:17.482721+0000","last_undegraded":"2024-09-16T09:26:17.482721+0000","last_fullsized":"2024-09-16T09:26:17.482721+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_clean_scrub_stamp":"2024-09-16T09:26:07.096764+0000","objects_scrubbed":0,"log_size":851,"log_dups_size":0,"ondisk_log_size":851,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:16.079110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":55771136,"num_objects":851,"num_object_clones":0,"num_object_copies":2553,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":851,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":851,"num_write_kb":54464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.f","version":"66'15","reported_seq":89,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.155031+0000","last_change":"2024-09-16T09:25:03.719134+0000","last_active":"2024-09-16T09:26:16.155031+0000","last_peered":"2024-09-16T09:26:16.155031+0000","last_clean":"2024-09-16T09:26:16.155031+0000","last_became_active":"2024-09-16T09:25:03.718849+0000","last_became_peered":"2024-09-16T09:25:03.718849+0000","last_unstale":"2024-09-16T09:26:16.155031+0000","last_undegraded":"2024-09-16T09:26:16.155031+0000","last_fullsized":"2024-09-16T09:26:16.155031+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":68,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.141774+0000","last_change":"2024-09-16T09:25:01.702936+0000","last_active":"2024-09-16T09:26:16.141774+0000","last_peered":"2024-09-16T09:26:16.141774+0000","last_clean":"2024-09-16T09:26:16.141774+0000","last_became_active":"2024-09-16T09:25:01.702629+0000","last_became_peered":"2024-09-16T09:25:01.702629+0000","last_unstale":"2024-09-16T09:26:16.141774+0000","last_undegraded":"2024-09-16T09:26:16.141774+0000","last_fullsized":"2024-09-16T09:26:16.141774+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:20.061024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"66'8","reported_seq":84,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.144581+0000","last_change":"2024-09-16T09:25:05.714128+0000","last_active":"2024-09-16T09:26:16.144581+0000","last_peered":"2024-09-16T09:26:16.144581+0000","last_clean":"2024-09-16T09:26:16.144581+0000","last_became_active":"2024-09-16T09:25:05.713813+0000","last_became_peered":"2024-09-16T09:25:05.713813+0000","last_unstale":"2024-09-16T09:26:16.144581+0000","last_undegraded":"2024-09-16T09:26:16.144581+0000","last_fullsized":"2024-09-16T09:26:16.144581+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:26.013786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":55,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.146958+0000","last_change":"2024-09-16T09:25:07.717639+0000","last_active":"2024-09-16T09:26:16.146958+0000","last_peered":"2024-09-16T09:26:16.146958+0000","last_clean":"2024-09-16T09:26:16.146958+0000","last_became_active":"2024-09-16T09:25:07.717344+0000","last_became_peered":"2024-09-16T09:25:07.717344+0000","last_unstale":"2024-09-16T09:26:16.146958+0000","last_undegraded":"2024-09-16T09:26:16.146958+0000","last_fullsized":"2024-09-16T09:26:16.146958+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:24.054256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3","version":"87'880","reported_seq":901,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:17.465778+0000","last_change":"2024-09-16T09:26:08.119864+0000","last_active":"2024-09-16T09:26:17.465778+0000","last_peered":"2024-09-16T09:26:17.465778+0000","last_clean":"2024-09-16T09:26:17.465778+0000","last_became_active":"2024-09-16T09:26:08.119550+0000","last_became_peered":"2024-09-16T09:26:08.119550+0000","last_unstale":"2024-09-16T09:26:17.465778+0000","last_undegraded":"2024-09-16T09:26:17.465778+0000","last_fullsized":"2024-09-16T09:26:17.465778+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_clean_scrub_stamp":"2024-09-16T09:26:07.096764+0000","objects_scrubbed":0,"log_size":880,"log_dups_size":0,"ondisk_log_size":880,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:14.657800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":57671680,"num_objects":880,"num_object_clones":0,"num_object_copies":2640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":880,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":880,"num_write_kb":56320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4],"acting":[3,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"66'18","reported_seq":96,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.141676+0000","last_change":"2024-09-16T09:25:03.714508+0000","last_active":"2024-09-16T09:26:16.141676+0000","last_peered":"2024-09-16T09:26:16.141676+0000","last_clean":"2024-09-16T09:26:16.141676+0000","last_became_active":"2024-09-16T09:25:03.714208+0000","last_became_peered":"2024-09-16T09:25:03.714208+0000","last_unstale":"2024-09-16T09:26:16.141676+0000","last_undegraded":"2024-09-16T09:26:16.141676+0000","last_fullsized":"2024-09-16T09:26:16.141676+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:08.631080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":68,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.146168+0000","last_change":"2024-09-16T09:25:01.705362+0000","last_active":"2024-09-16T09:26:16.146168+0000","last_peered":"2024-09-16T09:26:16.146168+0000","last_clean":"2024-09-16T09:26:16.146168+0000","last_became_active":"2024-09-16T09:25:01.704994+0000","last_became_peered":"2024-09-16T09:25:01.704994+0000","last_unstale":"2024-09-16T09:26:16.146168+0000","last_undegraded":"2024-09-16T09:26:16.146168+0000","last_fullsized":"2024-09-16T09:26:16.146168+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:17.466780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":60,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.142876+0000","last_change":"2024-09-16T09:25:05.713687+0000","last_active":"2024-09-16T09:26:16.142876+0000","last_peered":"2024-09-16T09:26:16.142876+0000","last_clean":"2024-09-16T09:26:16.142876+0000","last_became_active":"2024-09-16T09:25:05.713360+0000","last_became_peered":"2024-09-16T09:25:05.713360+0000","last_unstale":"2024-09-16T09:26:16.142876+0000","last_undegraded":"2024-09-16T09:26:16.142876+0000","last_fullsized":"2024-09-16T09:26:16.142876+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:16.400241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":56,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.143111+0000","last_change":"2024-09-16T09:25:07.719412+0000","last_active":"2024-09-16T09:26:16.143111+0000","last_peered":"2024-09-16T09:26:16.143111+0000","last_clean":"2024-09-16T09:26:16.143111+0000","last_became_active":"2024-09-16T09:25:07.719159+0000","last_became_peered":"2024-09-16T09:25:07.719159+0000","last_unstale":"2024-09-16T09:26:16.143111+0000","last_undegraded":"2024-09-16T09:26:16.143111+0000","last_fullsized":"2024-09-16T09:26:16.143111+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:36.545361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2","version":"87'853","reported_seq":874,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:17.476425+0000","last_change":"2024-09-16T09:26:08.120627+0000","last_active":"2024-09-16T09:26:17.476425+0000","last_peered":"2024-09-16T09:26:17.476425+0000","last_clean":"2024-09-16T09:26:17.476425+0000","last_became_active":"2024-09-16T09:26:08.120338+0000","last_became_peered":"2024-09-16T09:26:08.120338+0000","last_unstale":"2024-09-16T09:26:17.476425+0000","last_undegraded":"2024-09-16T09:26:17.476425+0000","last_fullsized":"2024-09-16T09:26:17.476425+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_clean_scrub_stamp":"2024-09-16T09:26:07.096764+0000","objects_scrubbed":0,"log_size":853,"log_dups_size":0,"ondisk_log_size":853,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:29.425867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":55902208,"num_objects":853,"num_object_clones":0,"num_object_copies":2559,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":853,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":853,"num_write_kb":54592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"66'14","reported_seq":85,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.142912+0000","last_change":"2024-09-16T09:25:03.710438+0000","last_active":"2024-09-16T09:26:16.142912+0000","last_peered":"2024-09-16T09:26:16.142912+0000","last_clean":"2024-09-16T09:26:16.142912+0000","last_became_active":"2024-09-16T09:25:03.708830+0000","last_became_peered":"2024-09-16T09:25:03.708830+0000","last_unstale":"2024-09-16T09:26:16.142912+0000","last_undegraded":"2024-09-16T09:26:16.142912+0000","last_fullsized":"2024-09-16T09:26:16.142912+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:17.190767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":77,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.150167+0000","last_change":"2024-09-16T09:26:01.699622+0000","last_active":"2024-09-16T09:26:16.150167+0000","last_peered":"2024-09-16T09:26:16.150167+0000","last_clean":"2024-09-16T09:26:16.150167+0000","last_became_active":"2024-09-16T09:25:01.702992+0000","last_became_peered":"2024-09-16T09:25:01.702992+0000","last_unstale":"2024-09-16T09:26:16.150167+0000","last_undegraded":"2024-09-16T09:26:16.150167+0000","last_fullsized":"2024-09-16T09:26:16.150167+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:26:01.699571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:01.699571+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:11.258189+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"66'8","reported_seq":84,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.141153+0000","last_change":"2024-09-16T09:25:05.715927+0000","last_active":"2024-09-16T09:26:16.141153+0000","last_peered":"2024-09-16T09:26:16.141153+0000","last_clean":"2024-09-16T09:26:16.141153+0000","last_became_active":"2024-09-16T09:25:05.715606+0000","last_became_peered":"2024-09-16T09:25:05.715606+0000","last_unstale":"2024-09-16T09:26:16.141153+0000","last_undegraded":"2024-09-16T09:26:16.141153+0000","last_fullsized":"2024-09-16T09:26:16.141153+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:23.752404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":55,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.148103+0000","last_change":"2024-09-16T09:25:07.722287+0000","last_active":"2024-09-16T09:26:16.148103+0000","last_peered":"2024-09-16T09:26:16.148103+0000","last_clean":"2024-09-16T09:26:16.148103+0000","last_became_active":"2024-09-16T09:25:07.722017+0000","last_became_peered":"2024-09-16T09:25:07.722017+0000","last_unstale":"2024-09-16T09:26:16.148103+0000","last_undegraded":"2024-09-16T09:26:16.148103+0000","last_fullsized":"2024-09-16T09:26:16.148103+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:27.828108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1","version":"87'855","reported_seq":876,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:17.480203+0000","last_change":"2024-09-16T09:26:08.117045+0000","last_active":"2024-09-16T09:26:17.480203+0000","last_peered":"2024-09-16T09:26:17.480203+0000","last_clean":"2024-09-16T09:26:17.480203+0000","last_became_active":"2024-09-16T09:26:08.116726+0000","last_became_peered":"2024-09-16T09:26:08.116726+0000","last_unstale":"2024-09-16T09:26:17.480203+0000","last_undegraded":"2024-09-16T09:26:17.480203+0000","last_fullsized":"2024-09-16T09:26:17.480203+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_clean_scrub_stamp":"2024-09-16T09:26:07.096764+0000","objects_scrubbed":0,"log_size":855,"log_dups_size":0,"ondisk_log_size":855,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:31.324898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56033280,"num_objects":855,"num_object_clones":0,"num_object_copies":2565,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":855,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":855,"num_write_kb":54720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"66'10","reported_seq":79,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.154695+0000","last_change":"2024-09-16T09:25:03.713368+0000","last_active":"2024-09-16T09:26:16.154695+0000","last_peered":"2024-09-16T09:26:16.154695+0000","last_clean":"2024-09-16T09:26:16.154695+0000","last_became_active":"2024-09-16T09:25:03.712985+0000","last_became_peered":"2024-09-16T09:25:03.712985+0000","last_unstale":"2024-09-16T09:26:16.154695+0000","last_undegraded":"2024-09-16T09:26:16.154695+0000","last_fullsized":"2024-09-16T09:26:16.154695+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:21.169460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":67,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.147836+0000","last_change":"2024-09-16T09:25:01.706774+0000","last_active":"2024-09-16T09:26:16.147836+0000","last_peered":"2024-09-16T09:26:16.147836+0000","last_clean":"2024-09-16T09:26:16.147836+0000","last_became_active":"2024-09-16T09:25:01.706440+0000","last_became_peered":"2024-09-16T09:25:01.706440+0000","last_unstale":"2024-09-16T09:26:16.147836+0000","last_undegraded":"2024-09-16T09:26:16.147836+0000","last_fullsized":"2024-09-16T09:26:16.147836+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:18.805280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"66'8","reported_seq":84,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.148708+0000","last_change":"2024-09-16T09:25:05.714825+0000","last_active":"2024-09-16T09:26:16.148708+0000","last_peered":"2024-09-16T09:26:16.148708+0000","last_clean":"2024-09-16T09:26:16.148708+0000","last_became_active":"2024-09-16T09:25:05.714507+0000","last_became_peered":"2024-09-16T09:25:05.714507+0000","last_unstale":"2024-09-16T09:26:16.148708+0000","last_undegraded":"2024-09-16T09:26:16.148708+0000","last_fullsized":"2024-09-16T09:26:16.148708+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":56,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.147883+0000","last_change":"2024-09-16T09:25:07.719876+0000","last_active":"2024-09-16T09:26:16.147883+0000","last_peered":"2024-09-16T09:26:16.147883+0000","last_clean":"2024-09-16T09:26:16.147883+0000","last_became_active":"2024-09-16T09:25:07.719604+0000","last_became_peered":"2024-09-16T09:25:07.719604+0000","last_unstale":"2024-09-16T09:26:16.147883+0000","last_undegraded":"2024-09-16T09:26:16.147883+0000","last_fullsized":"2024-09-16T09:26:16.147883+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:20.929075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.0","version":"87'884","reported_seq":905,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:17.477959+0000","last_change":"2024-09-16T09:26:08.117892+0000","last_active":"2024-09-16T09:26:17.477959+0000","last_peered":"2024-09-16T09:26:17.477959+0000","last_clean":"2024-09-16T09:26:17.477959+0000","last_became_active":"2024-09-16T09:26:08.117581+0000","last_became_peered":"2024-09-16T09:26:08.117581+0000","last_unstale":"2024-09-16T09:26:17.477959+0000","last_undegraded":"2024-09-16T09:26:17.477959+0000","last_fullsized":"2024-09-16T09:26:17.477959+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_clean_scrub_stamp":"2024-09-16T09:26:07.096764+0000","objects_scrubbed":0,"log_size":884,"log_dups_size":0,"ondisk_log_size":884,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:14.557873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":57933824,"num_objects":884,"num_object_clones":0,"num_object_copies":2652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":884,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":884,"num_write_kb":56576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3","version":"66'19","reported_seq":100,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.148889+0000","last_change":"2024-09-16T09:25:03.715086+0000","last_active":"2024-09-16T09:26:16.148889+0000","last_peered":"2024-09-16T09:26:16.148889+0000","last_clean":"2024-09-16T09:26:16.148889+0000","last_became_active":"2024-09-16T09:25:03.714797+0000","last_became_peered":"2024-09-16T09:25:03.714797+0000","last_unstale":"2024-09-16T09:26:16.148889+0000","last_undegraded":"2024-09-16T09:26:16.148889+0000","last_fullsized":"2024-09-16T09:26:16.148889+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:23.683038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":76,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.154526+0000","last_change":"2024-09-16T09:26:03.194840+0000","last_active":"2024-09-16T09:26:16.154526+0000","last_peered":"2024-09-16T09:26:16.154526+0000","last_clean":"2024-09-16T09:26:16.154526+0000","last_became_active":"2024-09-16T09:25:01.706806+0000","last_became_peered":"2024-09-16T09:25:01.706806+0000","last_unstale":"2024-09-16T09:26:16.154526+0000","last_undegraded":"2024-09-16T09:26:16.154526+0000","last_fullsized":"2024-09-16T09:26:16.154526+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:03.194794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:03.194794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:23.821209+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":60,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.150583+0000","last_change":"2024-09-16T09:25:05.708949+0000","last_active":"2024-09-16T09:26:16.150583+0000","last_peered":"2024-09-16T09:26:16.150583+0000","last_clean":"2024-09-16T09:26:16.150583+0000","last_became_active":"2024-09-16T09:25:05.708663+0000","last_became_peered":"2024-09-16T09:25:05.708663+0000","last_unstale":"2024-09-16T09:26:16.150583+0000","last_undegraded":"2024-09-16T09:26:16.150583+0000","last_fullsized":"2024-09-16T09:26:16.150583+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:09.114054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":56,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.154067+0000","last_change":"2024-09-16T09:25:07.719568+0000","last_active":"2024-09-16T09:26:16.154067+0000","last_peered":"2024-09-16T09:26:16.154067+0000","last_clean":"2024-09-16T09:26:16.154067+0000","last_became_active":"2024-09-16T09:25:07.719267+0000","last_became_peered":"2024-09-16T09:25:07.719267+0000","last_unstale":"2024-09-16T09:26:16.154067+0000","last_undegraded":"2024-09-16T09:26:16.154067+0000","last_fullsized":"2024-09-16T09:26:16.154067+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:31.531856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"87'866","reported_seq":887,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:17.472677+0000","last_change":"2024-09-16T09:26:08.122122+0000","last_active":"2024-09-16T09:26:17.472677+0000","last_peered":"2024-09-16T09:26:17.472677+0000","last_clean":"2024-09-16T09:26:17.472677+0000","last_became_active":"2024-09-16T09:26:08.121828+0000","last_became_peered":"2024-09-16T09:26:08.121828+0000","last_unstale":"2024-09-16T09:26:17.472677+0000","last_undegraded":"2024-09-16T09:26:17.472677+0000","last_fullsized":"2024-09-16T09:26:17.472677+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_clean_scrub_stamp":"2024-09-16T09:26:07.096764+0000","objects_scrubbed":0,"log_size":866,"log_dups_size":0,"ondisk_log_size":866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:27.278648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56754176,"num_objects":866,"num_object_clones":0,"num_object_copies":2598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":866,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":866,"num_write_kb":55424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.4","version":"66'28","reported_seq":128,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.154427+0000","last_change":"2024-09-16T09:25:03.705300+0000","last_active":"2024-09-16T09:26:16.154427+0000","last_peered":"2024-09-16T09:26:16.154427+0000","last_clean":"2024-09-16T09:26:16.154427+0000","last_became_active":"2024-09-16T09:25:03.704878+0000","last_became_peered":"2024-09-16T09:25:03.704878+0000","last_unstale":"2024-09-16T09:26:16.154427+0000","last_undegraded":"2024-09-16T09:26:16.154427+0000","last_fullsized":"2024-09-16T09:26:16.154427+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:32.612703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":68,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.143804+0000","last_change":"2024-09-16T09:25:01.703394+0000","last_active":"2024-09-16T09:26:16.143804+0000","last_peered":"2024-09-16T09:26:16.143804+0000","last_clean":"2024-09-16T09:26:16.143804+0000","last_became_active":"2024-09-16T09:25:01.703076+0000","last_became_peered":"2024-09-16T09:25:01.703076+0000","last_unstale":"2024-09-16T09:26:16.143804+0000","last_undegraded":"2024-09-16T09:26:16.143804+0000","last_fullsized":"2024-09-16T09:26:16.143804+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:21.922041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":77,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.146975+0000","last_change":"2024-09-16T09:25:03.706980+0000","last_active":"2024-09-16T09:26:16.146975+0000","last_peered":"2024-09-16T09:26:16.146975+0000","last_clean":"2024-09-16T09:26:16.146975+0000","last_became_active":"2024-09-16T09:25:00.696237+0000","last_became_peered":"2024-09-16T09:25:00.696237+0000","last_unstale":"2024-09-16T09:26:16.146975+0000","last_undegraded":"2024-09-16T09:26:16.146975+0000","last_fullsized":"2024-09-16T09:26:16.146975+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:24:59.684950+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:24.893358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025137430000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":60,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.149401+0000","last_change":"2024-09-16T09:25:05.718319+0000","last_active":"2024-09-16T09:26:16.149401+0000","last_peered":"2024-09-16T09:26:16.149401+0000","last_clean":"2024-09-16T09:26:16.149401+0000","last_became_active":"2024-09-16T09:25:05.717591+0000","last_became_peered":"2024-09-16T09:25:05.717591+0000","last_unstale":"2024-09-16T09:26:16.149401+0000","last_undegraded":"2024-09-16T09:26:16.149401+0000","last_fullsized":"2024-09-16T09:26:16.149401+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:32.732085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":56,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.145656+0000","last_change":"2024-09-16T09:25:07.717720+0000","last_active":"2024-09-16T09:26:16.145656+0000","last_peered":"2024-09-16T09:26:16.145656+0000","last_clean":"2024-09-16T09:26:16.145656+0000","last_became_active":"2024-09-16T09:25:07.717436+0000","last_became_peered":"2024-09-16T09:25:07.717436+0000","last_unstale":"2024-09-16T09:26:16.145656+0000","last_undegraded":"2024-09-16T09:26:16.145656+0000","last_fullsized":"2024-09-16T09:26:16.145656+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:16.230936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"87'835","reported_seq":856,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:17.485701+0000","last_change":"2024-09-16T09:26:08.120333+0000","last_active":"2024-09-16T09:26:17.485701+0000","last_peered":"2024-09-16T09:26:17.485701+0000","last_clean":"2024-09-16T09:26:17.485701+0000","last_became_active":"2024-09-16T09:26:08.120051+0000","last_became_peered":"2024-09-16T09:26:08.120051+0000","last_unstale":"2024-09-16T09:26:17.485701+0000","last_undegraded":"2024-09-16T09:26:17.485701+0000","last_fullsized":"2024-09-16T09:26:17.485701+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_clean_scrub_stamp":"2024-09-16T09:26:07.096764+0000","objects_scrubbed":0,"log_size":835,"log_dups_size":0,"ondisk_log_size":835,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:36.075646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54722560,"num_objects":835,"num_object_clones":0,"num_object_copies":2505,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":835,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":835,"num_write_kb":53440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,1],"acting":[0,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"66'13","reported_seq":91,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.155163+0000","last_change":"2024-09-16T09:25:03.713631+0000","last_active":"2024-09-16T09:26:16.155163+0000","last_peered":"2024-09-16T09:26:16.155163+0000","last_clean":"2024-09-16T09:26:16.155163+0000","last_became_active":"2024-09-16T09:25:03.713241+0000","last_became_peered":"2024-09-16T09:25:03.713241+0000","last_unstale":"2024-09-16T09:26:16.155163+0000","last_undegraded":"2024-09-16T09:26:16.155163+0000","last_fullsized":"2024-09-16T09:26:16.155163+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:19.940601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":68,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.154024+0000","last_change":"2024-09-16T09:25:01.707526+0000","last_active":"2024-09-16T09:26:16.154024+0000","last_peered":"2024-09-16T09:26:16.154024+0000","last_clean":"2024-09-16T09:26:16.154024+0000","last_became_active":"2024-09-16T09:25:01.706828+0000","last_became_peered":"2024-09-16T09:25:01.706828+0000","last_unstale":"2024-09-16T09:26:16.154024+0000","last_undegraded":"2024-09-16T09:26:16.154024+0000","last_fullsized":"2024-09-16T09:26:16.154024+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:17.040873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":77,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.143576+0000","last_change":"2024-09-16T09:25:03.702772+0000","last_active":"2024-09-16T09:26:16.143576+0000","last_peered":"2024-09-16T09:26:16.143576+0000","last_clean":"2024-09-16T09:26:16.143576+0000","last_became_active":"2024-09-16T09:25:00.697822+0000","last_became_peered":"2024-09-16T09:25:00.697822+0000","last_unstale":"2024-09-16T09:26:16.143576+0000","last_undegraded":"2024-09-16T09:26:16.143576+0000","last_fullsized":"2024-09-16T09:26:16.143576+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:24:59.684950+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:06.026068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055022299999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":60,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.144608+0000","last_change":"2024-09-16T09:25:05.711840+0000","last_active":"2024-09-16T09:26:16.144608+0000","last_peered":"2024-09-16T09:26:16.144608+0000","last_clean":"2024-09-16T09:26:16.144608+0000","last_became_active":"2024-09-16T09:25:05.711539+0000","last_became_peered":"2024-09-16T09:25:05.711539+0000","last_unstale":"2024-09-16T09:26:16.144608+0000","last_undegraded":"2024-09-16T09:26:16.144608+0000","last_fullsized":"2024-09-16T09:26:16.144608+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:28.306667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":55,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.148924+0000","last_change":"2024-09-16T09:25:07.723554+0000","last_active":"2024-09-16T09:26:16.148924+0000","last_peered":"2024-09-16T09:26:16.148924+0000","last_clean":"2024-09-16T09:26:16.148924+0000","last_became_active":"2024-09-16T09:25:07.723282+0000","last_became_peered":"2024-09-16T09:25:07.723282+0000","last_unstale":"2024-09-16T09:26:16.148924+0000","last_undegraded":"2024-09-16T09:26:16.148924+0000","last_fullsized":"2024-09-16T09:26:16.148924+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:28.111790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"87'893","reported_seq":914,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:17.467444+0000","last_change":"2024-09-16T09:26:08.121383+0000","last_active":"2024-09-16T09:26:17.467444+0000","last_peered":"2024-09-16T09:26:17.467444+0000","last_clean":"2024-09-16T09:26:17.467444+0000","last_became_active":"2024-09-16T09:26:08.121117+0000","last_became_peered":"2024-09-16T09:26:08.121117+0000","last_unstale":"2024-09-16T09:26:17.467444+0000","last_undegraded":"2024-09-16T09:26:17.467444+0000","last_fullsized":"2024-09-16T09:26:17.467444+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_clean_scrub_stamp":"2024-09-16T09:26:07.096764+0000","objects_scrubbed":0,"log_size":893,"log_dups_size":0,"ondisk_log_size":893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:20.839977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58523648,"num_objects":893,"num_object_clones":0,"num_object_copies":2679,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":893,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":893,"num_write_kb":57152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,0],"acting":[7,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"66'12","reported_seq":82,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.150114+0000","last_change":"2024-09-16T09:25:03.715612+0000","last_active":"2024-09-16T09:26:16.150114+0000","last_peered":"2024-09-16T09:26:16.150114+0000","last_clean":"2024-09-16T09:26:16.150114+0000","last_became_active":"2024-09-16T09:25:03.715315+0000","last_became_peered":"2024-09-16T09:25:03.715315+0000","last_unstale":"2024-09-16T09:26:16.150114+0000","last_undegraded":"2024-09-16T09:26:16.150114+0000","last_fullsized":"2024-09-16T09:26:16.150114+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:31.110979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"59'1","reported_seq":77,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.147817+0000","last_change":"2024-09-16T09:26:07.112979+0000","last_active":"2024-09-16T09:26:16.147817+0000","last_peered":"2024-09-16T09:26:16.147817+0000","last_clean":"2024-09-16T09:26:16.147817+0000","last_became_active":"2024-09-16T09:25:01.701599+0000","last_became_peered":"2024-09-16T09:25:01.701599+0000","last_unstale":"2024-09-16T09:26:16.147817+0000","last_undegraded":"2024-09-16T09:26:16.147817+0000","last_fullsized":"2024-09-16T09:26:16.147817+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:59.657345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"67'5","reported_seq":148,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.147945+0000","last_change":"2024-09-16T09:25:03.711738+0000","last_active":"2024-09-16T09:26:16.147945+0000","last_peered":"2024-09-16T09:26:16.147945+0000","last_clean":"2024-09-16T09:26:16.147945+0000","last_became_active":"2024-09-16T09:25:00.696546+0000","last_became_peered":"2024-09-16T09:25:00.696546+0000","last_unstale":"2024-09-16T09:26:16.147945+0000","last_undegraded":"2024-09-16T09:26:16.147945+0000","last_fullsized":"2024-09-16T09:26:16.147945+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:24:59.684950+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:17.241966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0059770420000000001,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":70,"num_read_kb":65,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":59,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.148602+0000","last_change":"2024-09-16T09:25:05.714994+0000","last_active":"2024-09-16T09:26:16.148602+0000","last_peered":"2024-09-16T09:26:16.148602+0000","last_clean":"2024-09-16T09:26:16.148602+0000","last_became_active":"2024-09-16T09:25:05.714704+0000","last_became_peered":"2024-09-16T09:25:05.714704+0000","last_unstale":"2024-09-16T09:26:16.148602+0000","last_undegraded":"2024-09-16T09:26:16.148602+0000","last_fullsized":"2024-09-16T09:26:16.148602+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:32.353177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":56,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.154013+0000","last_change":"2024-09-16T09:25:07.714719+0000","last_active":"2024-09-16T09:26:16.154013+0000","last_peered":"2024-09-16T09:26:16.154013+0000","last_clean":"2024-09-16T09:26:16.154013+0000","last_became_active":"2024-09-16T09:25:07.714158+0000","last_became_peered":"2024-09-16T09:25:07.714158+0000","last_unstale":"2024-09-16T09:26:16.154013+0000","last_undegraded":"2024-09-16T09:26:16.154013+0000","last_fullsized":"2024-09-16T09:26:16.154013+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.6","version":"87'877","reported_seq":898,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:17.494638+0000","last_change":"2024-09-16T09:26:08.121544+0000","last_active":"2024-09-16T09:26:17.494638+0000","last_peered":"2024-09-16T09:26:17.494638+0000","last_clean":"2024-09-16T09:26:17.494638+0000","last_became_active":"2024-09-16T09:26:08.121253+0000","last_became_peered":"2024-09-16T09:26:08.121253+0000","last_unstale":"2024-09-16T09:26:17.494638+0000","last_undegraded":"2024-09-16T09:26:17.494638+0000","last_fullsized":"2024-09-16T09:26:17.494638+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_clean_scrub_stamp":"2024-09-16T09:26:07.096764+0000","objects_scrubbed":0,"log_size":877,"log_dups_size":0,"ondisk_log_size":877,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:14.407389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":57475072,"num_objects":877,"num_object_clones":0,"num_object_copies":2631,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":877,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":877,"num_write_kb":56128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"66'16","reported_seq":102,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.151288+0000","last_change":"2024-09-16T09:25:03.714405+0000","last_active":"2024-09-16T09:26:16.151288+0000","last_peered":"2024-09-16T09:26:16.151288+0000","last_clean":"2024-09-16T09:26:16.151288+0000","last_became_active":"2024-09-16T09:25:03.714118+0000","last_became_peered":"2024-09-16T09:25:03.714118+0000","last_unstale":"2024-09-16T09:26:16.151288+0000","last_undegraded":"2024-09-16T09:26:16.151288+0000","last_fullsized":"2024-09-16T09:26:16.151288+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:13.104229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":68,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.146458+0000","last_change":"2024-09-16T09:25:01.704246+0000","last_active":"2024-09-16T09:26:16.146458+0000","last_peered":"2024-09-16T09:26:16.146458+0000","last_clean":"2024-09-16T09:26:16.146458+0000","last_became_active":"2024-09-16T09:25:01.703730+0000","last_became_peered":"2024-09-16T09:25:01.703730+0000","last_unstale":"2024-09-16T09:26:16.146458+0000","last_undegraded":"2024-09-16T09:26:16.146458+0000","last_fullsized":"2024-09-16T09:26:16.146458+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:20.523303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"73'79","reported_seq":105,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.148095+0000","last_change":"2024-09-16T09:24:53.142015+0000","last_active":"2024-09-16T09:26:16.148095+0000","last_peered":"2024-09-16T09:26:16.148095+0000","last_clean":"2024-09-16T09:26:16.148095+0000","last_became_active":"2024-09-16T09:24:53.135890+0000","last_became_peered":"2024-09-16T09:24:53.135890+0000","last_unstale":"2024-09-16T09:26:16.148095+0000","last_undegraded":"2024-09-16T09:26:16.148095+0000","last_fullsized":"2024-09-16T09:26:16.148095+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"20'72","last_scrub_stamp":"2024-09-16T09:24:45.089989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:24:45.089989+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":59,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.147700+0000","last_change":"2024-09-16T09:25:05.714741+0000","last_active":"2024-09-16T09:26:16.147700+0000","last_peered":"2024-09-16T09:26:16.147700+0000","last_clean":"2024-09-16T09:26:16.147700+0000","last_became_active":"2024-09-16T09:25:05.714341+0000","last_became_peered":"2024-09-16T09:25:05.714341+0000","last_unstale":"2024-09-16T09:26:16.147700+0000","last_undegraded":"2024-09-16T09:26:16.147700+0000","last_fullsized":"2024-09-16T09:26:16.147700+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:21.685216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":55,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.148412+0000","last_change":"2024-09-16T09:25:07.714544+0000","last_active":"2024-09-16T09:26:16.148412+0000","last_peered":"2024-09-16T09:26:16.148412+0000","last_clean":"2024-09-16T09:26:16.148412+0000","last_became_active":"2024-09-16T09:25:07.714281+0000","last_became_peered":"2024-09-16T09:25:07.714281+0000","last_unstale":"2024-09-16T09:26:16.148412+0000","last_undegraded":"2024-09-16T09:26:16.148412+0000","last_fullsized":"2024-09-16T09:26:16.148412+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"87'852","reported_seq":873,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:17.466718+0000","last_change":"2024-09-16T09:26:08.122057+0000","last_active":"2024-09-16T09:26:17.466718+0000","last_peered":"2024-09-16T09:26:17.466718+0000","last_clean":"2024-09-16T09:26:17.466718+0000","last_became_active":"2024-09-16T09:26:08.121785+0000","last_became_peered":"2024-09-16T09:26:08.121785+0000","last_unstale":"2024-09-16T09:26:17.466718+0000","last_undegraded":"2024-09-16T09:26:17.466718+0000","last_fullsized":"2024-09-16T09:26:17.466718+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_clean_scrub_stamp":"2024-09-16T09:26:07.096764+0000","objects_scrubbed":0,"log_size":852,"log_dups_size":0,"ondisk_log_size":852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:16.969171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":55836672,"num_objects":852,"num_object_clones":0,"num_object_copies":2556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":852,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":852,"num_write_kb":54528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.e","version":"66'11","reported_seq":83,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.144691+0000","last_change":"2024-09-16T09:25:03.715654+0000","last_active":"2024-09-16T09:26:16.144691+0000","last_peered":"2024-09-16T09:26:16.144691+0000","last_clean":"2024-09-16T09:26:16.144691+0000","last_became_active":"2024-09-16T09:25:03.715373+0000","last_became_peered":"2024-09-16T09:25:03.715373+0000","last_unstale":"2024-09-16T09:26:16.144691+0000","last_undegraded":"2024-09-16T09:26:16.144691+0000","last_fullsized":"2024-09-16T09:26:16.144691+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:15.776474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":68,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.143149+0000","last_change":"2024-09-16T09:25:01.708568+0000","last_active":"2024-09-16T09:26:16.143149+0000","last_peered":"2024-09-16T09:26:16.143149+0000","last_clean":"2024-09-16T09:26:16.143149+0000","last_became_active":"2024-09-16T09:25:01.708270+0000","last_became_peered":"2024-09-16T09:25:01.708270+0000","last_unstale":"2024-09-16T09:26:16.143149+0000","last_undegraded":"2024-09-16T09:26:16.143149+0000","last_fullsized":"2024-09-16T09:26:16.143149+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:24.212331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":59,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.148328+0000","last_change":"2024-09-16T09:25:05.712303+0000","last_active":"2024-09-16T09:26:16.148328+0000","last_peered":"2024-09-16T09:26:16.148328+0000","last_clean":"2024-09-16T09:26:16.148328+0000","last_became_active":"2024-09-16T09:25:05.711964+0000","last_became_peered":"2024-09-16T09:25:05.711964+0000","last_unstale":"2024-09-16T09:26:16.148328+0000","last_undegraded":"2024-09-16T09:26:16.148328+0000","last_fullsized":"2024-09-16T09:26:16.148328+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:23.392105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":56,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.142495+0000","last_change":"2024-09-16T09:25:07.716780+0000","last_active":"2024-09-16T09:26:16.142495+0000","last_peered":"2024-09-16T09:26:16.142495+0000","last_clean":"2024-09-16T09:26:16.142495+0000","last_became_active":"2024-09-16T09:25:07.716514+0000","last_became_peered":"2024-09-16T09:25:07.716514+0000","last_unstale":"2024-09-16T09:26:16.142495+0000","last_undegraded":"2024-09-16T09:26:16.142495+0000","last_fullsized":"2024-09-16T09:26:16.142495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:12.593567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.e","version":"87'843","reported_seq":864,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:17.476873+0000","last_change":"2024-09-16T09:26:08.122247+0000","last_active":"2024-09-16T09:26:17.476873+0000","last_peered":"2024-09-16T09:26:17.476873+0000","last_clean":"2024-09-16T09:26:17.476873+0000","last_became_active":"2024-09-16T09:26:08.121969+0000","last_became_peered":"2024-09-16T09:26:08.121969+0000","last_unstale":"2024-09-16T09:26:17.476873+0000","last_undegraded":"2024-09-16T09:26:17.476873+0000","last_fullsized":"2024-09-16T09:26:17.476873+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_clean_scrub_stamp":"2024-09-16T09:26:07.096764+0000","objects_scrubbed":0,"log_size":843,"log_dups_size":0,"ondisk_log_size":843,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:13.996553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":55246848,"num_objects":843,"num_object_clones":0,"num_object_copies":2529,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":843,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":843,"num_write_kb":53952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"66'17","reported_seq":92,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.144111+0000","last_change":"2024-09-16T09:25:03.713691+0000","last_active":"2024-09-16T09:26:16.144111+0000","last_peered":"2024-09-16T09:26:16.144111+0000","last_clean":"2024-09-16T09:26:16.144111+0000","last_became_active":"2024-09-16T09:25:03.713358+0000","last_became_peered":"2024-09-16T09:25:03.713358+0000","last_unstale":"2024-09-16T09:26:16.144111+0000","last_undegraded":"2024-09-16T09:26:16.144111+0000","last_fullsized":"2024-09-16T09:26:16.144111+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":76,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.150773+0000","last_change":"2024-09-16T09:26:04.104188+0000","last_active":"2024-09-16T09:26:16.150773+0000","last_peered":"2024-09-16T09:26:16.150773+0000","last_clean":"2024-09-16T09:26:16.150773+0000","last_became_active":"2024-09-16T09:25:01.703355+0000","last_became_peered":"2024-09-16T09:25:01.703355+0000","last_unstale":"2024-09-16T09:26:16.150773+0000","last_undegraded":"2024-09-16T09:26:16.150773+0000","last_fullsized":"2024-09-16T09:26:16.150773+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:04.104130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:04.104130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:21.604480+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":60,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.154121+0000","last_change":"2024-09-16T09:25:05.717224+0000","last_active":"2024-09-16T09:26:16.154121+0000","last_peered":"2024-09-16T09:26:16.154121+0000","last_clean":"2024-09-16T09:26:16.154121+0000","last_became_active":"2024-09-16T09:25:05.716900+0000","last_became_peered":"2024-09-16T09:25:05.716900+0000","last_unstale":"2024-09-16T09:26:16.154121+0000","last_undegraded":"2024-09-16T09:26:16.154121+0000","last_fullsized":"2024-09-16T09:26:16.154121+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:30.026527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":56,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.143873+0000","last_change":"2024-09-16T09:25:07.721657+0000","last_active":"2024-09-16T09:26:16.143873+0000","last_peered":"2024-09-16T09:26:16.143873+0000","last_clean":"2024-09-16T09:26:16.143873+0000","last_became_active":"2024-09-16T09:25:07.721347+0000","last_became_peered":"2024-09-16T09:25:07.721347+0000","last_unstale":"2024-09-16T09:26:16.143873+0000","last_undegraded":"2024-09-16T09:26:16.143873+0000","last_fullsized":"2024-09-16T09:26:16.143873+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"87'857","reported_seq":877,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:17.495473+0000","last_change":"2024-09-16T09:26:08.120989+0000","last_active":"2024-09-16T09:26:17.495473+0000","last_peered":"2024-09-16T09:26:17.495473+0000","last_clean":"2024-09-16T09:26:17.495473+0000","last_became_active":"2024-09-16T09:26:08.120360+0000","last_became_peered":"2024-09-16T09:26:08.120360+0000","last_unstale":"2024-09-16T09:26:17.495473+0000","last_undegraded":"2024-09-16T09:26:17.495473+0000","last_fullsized":"2024-09-16T09:26:17.495473+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_clean_scrub_stamp":"2024-09-16T09:26:07.096764+0000","objects_scrubbed":0,"log_size":857,"log_dups_size":0,"ondisk_log_size":857,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:35.969883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56164352,"num_objects":857,"num_object_clones":0,"num_object_copies":2571,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":857,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":857,"num_write_kb":54848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"66'10","reported_seq":79,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.143865+0000","last_change":"2024-09-16T09:25:03.708144+0000","last_active":"2024-09-16T09:26:16.143865+0000","last_peered":"2024-09-16T09:26:16.143865+0000","last_clean":"2024-09-16T09:26:16.143865+0000","last_became_active":"2024-09-16T09:25:03.707723+0000","last_became_peered":"2024-09-16T09:25:03.707723+0000","last_unstale":"2024-09-16T09:26:16.143865+0000","last_undegraded":"2024-09-16T09:26:16.143865+0000","last_fullsized":"2024-09-16T09:26:16.143865+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:24.890527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":68,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.142293+0000","last_change":"2024-09-16T09:25:01.703388+0000","last_active":"2024-09-16T09:26:16.142293+0000","last_peered":"2024-09-16T09:26:16.142293+0000","last_clean":"2024-09-16T09:26:16.142293+0000","last_became_active":"2024-09-16T09:25:01.703094+0000","last_became_peered":"2024-09-16T09:25:01.703094+0000","last_unstale":"2024-09-16T09:26:16.142293+0000","last_undegraded":"2024-09-16T09:26:16.142293+0000","last_fullsized":"2024-09-16T09:26:16.142293+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:11.575598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":87,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.144150+0000","last_change":"2024-09-16T09:25:05.711891+0000","last_active":"2024-09-16T09:26:16.144150+0000","last_peered":"2024-09-16T09:26:16.144150+0000","last_clean":"2024-09-16T09:26:16.144150+0000","last_became_active":"2024-09-16T09:25:05.711590+0000","last_became_peered":"2024-09-16T09:25:05.711590+0000","last_unstale":"2024-09-16T09:26:16.144150+0000","last_undegraded":"2024-09-16T09:26:16.144150+0000","last_fullsized":"2024-09-16T09:26:16.144150+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:17.630914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":56,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.144335+0000","last_change":"2024-09-16T09:25:07.719496+0000","last_active":"2024-09-16T09:26:16.144335+0000","last_peered":"2024-09-16T09:26:16.144335+0000","last_clean":"2024-09-16T09:26:16.144335+0000","last_became_active":"2024-09-16T09:25:07.718958+0000","last_became_peered":"2024-09-16T09:25:07.718958+0000","last_unstale":"2024-09-16T09:26:16.144335+0000","last_undegraded":"2024-09-16T09:26:16.144335+0000","last_fullsized":"2024-09-16T09:26:16.144335+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.8","version":"87'867","reported_seq":888,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:17.469043+0000","last_change":"2024-09-16T09:26:08.118481+0000","last_active":"2024-09-16T09:26:17.469043+0000","last_peered":"2024-09-16T09:26:17.469043+0000","last_clean":"2024-09-16T09:26:17.469043+0000","last_became_active":"2024-09-16T09:26:08.118215+0000","last_became_peered":"2024-09-16T09:26:08.118215+0000","last_unstale":"2024-09-16T09:26:17.469043+0000","last_undegraded":"2024-09-16T09:26:17.469043+0000","last_fullsized":"2024-09-16T09:26:17.469043+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_clean_scrub_stamp":"2024-09-16T09:26:07.096764+0000","objects_scrubbed":0,"log_size":867,"log_dups_size":0,"ondisk_log_size":867,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:14.612304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56819712,"num_objects":867,"num_object_clones":0,"num_object_copies":2601,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":867,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":867,"num_write_kb":55488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"66'9","reported_seq":80,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.148804+0000","last_change":"2024-09-16T09:25:03.712553+0000","last_active":"2024-09-16T09:26:16.148804+0000","last_peered":"2024-09-16T09:26:16.148804+0000","last_clean":"2024-09-16T09:26:16.148804+0000","last_became_active":"2024-09-16T09:25:03.712230+0000","last_became_peered":"2024-09-16T09:25:03.712230+0000","last_unstale":"2024-09-16T09:26:16.148804+0000","last_undegraded":"2024-09-16T09:26:16.148804+0000","last_fullsized":"2024-09-16T09:26:16.148804+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:18.262732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":67,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.147271+0000","last_change":"2024-09-16T09:25:01.702943+0000","last_active":"2024-09-16T09:26:16.147271+0000","last_peered":"2024-09-16T09:26:16.147271+0000","last_clean":"2024-09-16T09:26:16.147271+0000","last_became_active":"2024-09-16T09:25:01.702629+0000","last_became_peered":"2024-09-16T09:25:01.702629+0000","last_unstale":"2024-09-16T09:26:16.147271+0000","last_undegraded":"2024-09-16T09:26:16.147271+0000","last_fullsized":"2024-09-16T09:26:16.147271+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:19.763325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":60,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.144052+0000","last_change":"2024-09-16T09:25:05.712932+0000","last_active":"2024-09-16T09:26:16.144052+0000","last_peered":"2024-09-16T09:26:16.144052+0000","last_clean":"2024-09-16T09:26:16.144052+0000","last_became_active":"2024-09-16T09:25:05.712573+0000","last_became_peered":"2024-09-16T09:25:05.712573+0000","last_unstale":"2024-09-16T09:26:16.144052+0000","last_undegraded":"2024-09-16T09:26:16.144052+0000","last_fullsized":"2024-09-16T09:26:16.144052+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:16.863815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":56,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.147729+0000","last_change":"2024-09-16T09:25:07.719073+0000","last_active":"2024-09-16T09:26:16.147729+0000","last_peered":"2024-09-16T09:26:16.147729+0000","last_clean":"2024-09-16T09:26:16.147729+0000","last_became_active":"2024-09-16T09:25:07.718778+0000","last_became_peered":"2024-09-16T09:25:07.718778+0000","last_unstale":"2024-09-16T09:26:16.147729+0000","last_undegraded":"2024-09-16T09:26:16.147729+0000","last_fullsized":"2024-09-16T09:26:16.147729+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:21.733588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"87'889","reported_seq":910,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:17.484755+0000","last_change":"2024-09-16T09:26:08.120869+0000","last_active":"2024-09-16T09:26:17.484755+0000","last_peered":"2024-09-16T09:26:17.484755+0000","last_clean":"2024-09-16T09:26:17.484755+0000","last_became_active":"2024-09-16T09:26:08.120579+0000","last_became_peered":"2024-09-16T09:26:08.120579+0000","last_unstale":"2024-09-16T09:26:17.484755+0000","last_undegraded":"2024-09-16T09:26:17.484755+0000","last_fullsized":"2024-09-16T09:26:17.484755+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_clean_scrub_stamp":"2024-09-16T09:26:07.096764+0000","objects_scrubbed":0,"log_size":889,"log_dups_size":0,"ondisk_log_size":889,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:28.849121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58261504,"num_objects":889,"num_object_clones":0,"num_object_copies":2667,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":889,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":889,"num_write_kb":56896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"66'19","reported_seq":95,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.150674+0000","last_change":"2024-09-16T09:25:03.713182+0000","last_active":"2024-09-16T09:26:16.150674+0000","last_peered":"2024-09-16T09:26:16.150674+0000","last_clean":"2024-09-16T09:26:16.150674+0000","last_became_active":"2024-09-16T09:25:03.712783+0000","last_became_peered":"2024-09-16T09:25:03.712783+0000","last_unstale":"2024-09-16T09:26:16.150674+0000","last_undegraded":"2024-09-16T09:26:16.150674+0000","last_fullsized":"2024-09-16T09:26:16.150674+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:24.025685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"59'1","reported_seq":73,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.149280+0000","last_change":"2024-09-16T09:25:01.708571+0000","last_active":"2024-09-16T09:26:16.149280+0000","last_peered":"2024-09-16T09:26:16.149280+0000","last_clean":"2024-09-16T09:26:16.149280+0000","last_became_active":"2024-09-16T09:25:01.708069+0000","last_became_peered":"2024-09-16T09:25:01.708069+0000","last_unstale":"2024-09-16T09:26:16.149280+0000","last_undegraded":"2024-09-16T09:26:16.149280+0000","last_fullsized":"2024-09-16T09:26:16.149280+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:06.677249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":60,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.145263+0000","last_change":"2024-09-16T09:25:05.709549+0000","last_active":"2024-09-16T09:26:16.145263+0000","last_peered":"2024-09-16T09:26:16.145263+0000","last_clean":"2024-09-16T09:26:16.145263+0000","last_became_active":"2024-09-16T09:25:05.709264+0000","last_became_peered":"2024-09-16T09:25:05.709264+0000","last_unstale":"2024-09-16T09:26:16.145263+0000","last_undegraded":"2024-09-16T09:26:16.145263+0000","last_fullsized":"2024-09-16T09:26:16.145263+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:10.982488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":55,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.147997+0000","last_change":"2024-09-16T09:25:07.719009+0000","last_active":"2024-09-16T09:26:16.147997+0000","last_peered":"2024-09-16T09:26:16.147997+0000","last_clean":"2024-09-16T09:26:16.147997+0000","last_became_active":"2024-09-16T09:25:07.718750+0000","last_became_peered":"2024-09-16T09:25:07.718750+0000","last_unstale":"2024-09-16T09:26:16.147997+0000","last_undegraded":"2024-09-16T09:26:16.147997+0000","last_fullsized":"2024-09-16T09:26:16.147997+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:12.691158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.a","version":"87'820","reported_seq":841,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:17.461248+0000","last_change":"2024-09-16T09:26:08.122847+0000","last_active":"2024-09-16T09:26:17.461248+0000","last_peered":"2024-09-16T09:26:17.461248+0000","last_clean":"2024-09-16T09:26:17.461248+0000","last_became_active":"2024-09-16T09:26:08.122557+0000","last_became_peered":"2024-09-16T09:26:08.122557+0000","last_unstale":"2024-09-16T09:26:17.461248+0000","last_undegraded":"2024-09-16T09:26:17.461248+0000","last_fullsized":"2024-09-16T09:26:17.461248+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_clean_scrub_stamp":"2024-09-16T09:26:07.096764+0000","objects_scrubbed":0,"log_size":820,"log_dups_size":0,"ondisk_log_size":820,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:21.791165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53739520,"num_objects":820,"num_object_clones":0,"num_object_copies":2460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":820,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":820,"num_write_kb":52480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"66'12","reported_seq":87,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.143086+0000","last_change":"2024-09-16T09:25:03.714309+0000","last_active":"2024-09-16T09:26:16.143086+0000","last_peered":"2024-09-16T09:26:16.143086+0000","last_clean":"2024-09-16T09:26:16.143086+0000","last_became_active":"2024-09-16T09:25:03.714027+0000","last_became_peered":"2024-09-16T09:25:03.714027+0000","last_unstale":"2024-09-16T09:26:16.143086+0000","last_undegraded":"2024-09-16T09:26:16.143086+0000","last_fullsized":"2024-09-16T09:26:16.143086+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:31.556984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":67,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.149239+0000","last_change":"2024-09-16T09:25:01.707905+0000","last_active":"2024-09-16T09:26:16.149239+0000","last_peered":"2024-09-16T09:26:16.149239+0000","last_clean":"2024-09-16T09:26:16.149239+0000","last_became_active":"2024-09-16T09:25:01.707528+0000","last_became_peered":"2024-09-16T09:25:01.707528+0000","last_unstale":"2024-09-16T09:26:16.149239+0000","last_undegraded":"2024-09-16T09:26:16.149239+0000","last_fullsized":"2024-09-16T09:26:16.149239+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":60,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.143757+0000","last_change":"2024-09-16T09:25:05.715584+0000","last_active":"2024-09-16T09:26:16.143757+0000","last_peered":"2024-09-16T09:26:16.143757+0000","last_clean":"2024-09-16T09:26:16.143757+0000","last_became_active":"2024-09-16T09:25:05.715253+0000","last_became_peered":"2024-09-16T09:25:05.715253+0000","last_unstale":"2024-09-16T09:26:16.143757+0000","last_undegraded":"2024-09-16T09:26:16.143757+0000","last_fullsized":"2024-09-16T09:26:16.143757+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:23.326757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":56,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.142016+0000","last_change":"2024-09-16T09:25:07.719338+0000","last_active":"2024-09-16T09:26:16.142016+0000","last_peered":"2024-09-16T09:26:16.142016+0000","last_clean":"2024-09-16T09:26:16.142016+0000","last_became_active":"2024-09-16T09:25:07.718899+0000","last_became_peered":"2024-09-16T09:25:07.718899+0000","last_unstale":"2024-09-16T09:26:16.142016+0000","last_undegraded":"2024-09-16T09:26:16.142016+0000","last_fullsized":"2024-09-16T09:26:16.142016+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:23.435495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"87'903","reported_seq":924,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:17.476581+0000","last_change":"2024-09-16T09:26:08.118161+0000","last_active":"2024-09-16T09:26:17.476581+0000","last_peered":"2024-09-16T09:26:17.476581+0000","last_clean":"2024-09-16T09:26:17.476581+0000","last_became_active":"2024-09-16T09:26:08.117869+0000","last_became_peered":"2024-09-16T09:26:08.117869+0000","last_unstale":"2024-09-16T09:26:17.476581+0000","last_undegraded":"2024-09-16T09:26:17.476581+0000","last_fullsized":"2024-09-16T09:26:17.476581+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_clean_scrub_stamp":"2024-09-16T09:26:07.096764+0000","objects_scrubbed":0,"log_size":903,"log_dups_size":0,"ondisk_log_size":903,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:19.443832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59179008,"num_objects":903,"num_object_clones":0,"num_object_copies":2709,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":903,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":903,"num_write_kb":57792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"66'15","reported_seq":88,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.146580+0000","last_change":"2024-09-16T09:25:03.716075+0000","last_active":"2024-09-16T09:26:16.146580+0000","last_peered":"2024-09-16T09:26:16.146580+0000","last_clean":"2024-09-16T09:26:16.146580+0000","last_became_active":"2024-09-16T09:25:03.715733+0000","last_became_peered":"2024-09-16T09:25:03.715733+0000","last_unstale":"2024-09-16T09:26:16.146580+0000","last_undegraded":"2024-09-16T09:26:16.146580+0000","last_fullsized":"2024-09-16T09:26:16.146580+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:20.577367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":79,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.149214+0000","last_change":"2024-09-16T09:25:01.707886+0000","last_active":"2024-09-16T09:26:16.149214+0000","last_peered":"2024-09-16T09:26:16.149214+0000","last_clean":"2024-09-16T09:26:16.149214+0000","last_became_active":"2024-09-16T09:25:01.707372+0000","last_became_peered":"2024-09-16T09:25:01.707372+0000","last_unstale":"2024-09-16T09:26:16.149214+0000","last_undegraded":"2024-09-16T09:26:16.149214+0000","last_fullsized":"2024-09-16T09:26:16.149214+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"66'8","reported_seq":83,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.147518+0000","last_change":"2024-09-16T09:25:05.716778+0000","last_active":"2024-09-16T09:26:16.147518+0000","last_peered":"2024-09-16T09:26:16.147518+0000","last_clean":"2024-09-16T09:26:16.147518+0000","last_became_active":"2024-09-16T09:25:05.716189+0000","last_became_peered":"2024-09-16T09:25:05.716189+0000","last_unstale":"2024-09-16T09:26:16.147518+0000","last_undegraded":"2024-09-16T09:26:16.147518+0000","last_fullsized":"2024-09-16T09:26:16.147518+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:31.128760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":55,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.146507+0000","last_change":"2024-09-16T09:25:07.717505+0000","last_active":"2024-09-16T09:26:16.146507+0000","last_peered":"2024-09-16T09:26:16.146507+0000","last_clean":"2024-09-16T09:26:16.146507+0000","last_became_active":"2024-09-16T09:25:07.717213+0000","last_became_peered":"2024-09-16T09:25:07.717213+0000","last_unstale":"2024-09-16T09:26:16.146507+0000","last_undegraded":"2024-09-16T09:26:16.146507+0000","last_fullsized":"2024-09-16T09:26:16.146507+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":68,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.150565+0000","last_change":"2024-09-16T09:25:01.703489+0000","last_active":"2024-09-16T09:26:16.150565+0000","last_peered":"2024-09-16T09:26:16.150565+0000","last_clean":"2024-09-16T09:26:16.150565+0000","last_became_active":"2024-09-16T09:25:01.703202+0000","last_became_peered":"2024-09-16T09:25:01.703202+0000","last_unstale":"2024-09-16T09:26:16.150565+0000","last_undegraded":"2024-09-16T09:26:16.150565+0000","last_fullsized":"2024-09-16T09:26:16.150565+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:30.581098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"66'6","reported_seq":73,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.146115+0000","last_change":"2024-09-16T09:25:03.713606+0000","last_active":"2024-09-16T09:26:16.146115+0000","last_peered":"2024-09-16T09:26:16.146115+0000","last_clean":"2024-09-16T09:26:16.146115+0000","last_became_active":"2024-09-16T09:25:03.713245+0000","last_became_peered":"2024-09-16T09:25:03.713245+0000","last_unstale":"2024-09-16T09:26:16.146115+0000","last_undegraded":"2024-09-16T09:26:16.146115+0000","last_fullsized":"2024-09-16T09:26:16.146115+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":59,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.148110+0000","last_change":"2024-09-16T09:25:05.715938+0000","last_active":"2024-09-16T09:26:16.148110+0000","last_peered":"2024-09-16T09:26:16.148110+0000","last_clean":"2024-09-16T09:26:16.148110+0000","last_became_active":"2024-09-16T09:25:05.715616+0000","last_became_peered":"2024-09-16T09:25:05.715616+0000","last_unstale":"2024-09-16T09:26:16.148110+0000","last_undegraded":"2024-09-16T09:26:16.148110+0000","last_fullsized":"2024-09-16T09:26:16.148110+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:25.224184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":55,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.147413+0000","last_change":"2024-09-16T09:25:07.721375+0000","last_active":"2024-09-16T09:26:16.147413+0000","last_peered":"2024-09-16T09:26:16.147413+0000","last_clean":"2024-09-16T09:26:16.147413+0000","last_became_active":"2024-09-16T09:25:07.721059+0000","last_became_peered":"2024-09-16T09:25:07.721059+0000","last_unstale":"2024-09-16T09:26:16.147413+0000","last_undegraded":"2024-09-16T09:26:16.147413+0000","last_fullsized":"2024-09-16T09:26:16.147413+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:16.275710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"66'9","reported_seq":80,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.150115+0000","last_change":"2024-09-16T09:25:03.719919+0000","last_active":"2024-09-16T09:26:16.150115+0000","last_peered":"2024-09-16T09:26:16.150115+0000","last_clean":"2024-09-16T09:26:16.150115+0000","last_became_active":"2024-09-16T09:25:03.719573+0000","last_became_peered":"2024-09-16T09:25:03.719573+0000","last_unstale":"2024-09-16T09:26:16.150115+0000","last_undegraded":"2024-09-16T09:26:16.150115+0000","last_fullsized":"2024-09-16T09:26:16.150115+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:23.239324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":67,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.147673+0000","last_change":"2024-09-16T09:25:01.707139+0000","last_active":"2024-09-16T09:26:16.147673+0000","last_peered":"2024-09-16T09:26:16.147673+0000","last_clean":"2024-09-16T09:26:16.147673+0000","last_became_active":"2024-09-16T09:25:01.706491+0000","last_became_peered":"2024-09-16T09:25:01.706491+0000","last_unstale":"2024-09-16T09:26:16.147673+0000","last_undegraded":"2024-09-16T09:26:16.147673+0000","last_fullsized":"2024-09-16T09:26:16.147673+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:09.678495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":60,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.146034+0000","last_change":"2024-09-16T09:25:05.715083+0000","last_active":"2024-09-16T09:26:16.146034+0000","last_peered":"2024-09-16T09:26:16.146034+0000","last_clean":"2024-09-16T09:26:16.146034+0000","last_became_active":"2024-09-16T09:25:05.714801+0000","last_became_peered":"2024-09-16T09:25:05.714801+0000","last_unstale":"2024-09-16T09:26:16.146034+0000","last_undegraded":"2024-09-16T09:26:16.146034+0000","last_fullsized":"2024-09-16T09:26:16.146034+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":56,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.144607+0000","last_change":"2024-09-16T09:25:07.722356+0000","last_active":"2024-09-16T09:26:16.144607+0000","last_peered":"2024-09-16T09:26:16.144607+0000","last_clean":"2024-09-16T09:26:16.144607+0000","last_became_active":"2024-09-16T09:25:07.722050+0000","last_became_peered":"2024-09-16T09:25:07.722050+0000","last_unstale":"2024-09-16T09:26:16.144607+0000","last_undegraded":"2024-09-16T09:26:16.144607+0000","last_fullsized":"2024-09-16T09:26:16.144607+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:35.793652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"66'9","reported_seq":79,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.147675+0000","last_change":"2024-09-16T09:25:03.717609+0000","last_active":"2024-09-16T09:26:16.147675+0000","last_peered":"2024-09-16T09:26:16.147675+0000","last_clean":"2024-09-16T09:26:16.147675+0000","last_became_active":"2024-09-16T09:25:03.717310+0000","last_became_peered":"2024-09-16T09:25:03.717310+0000","last_unstale":"2024-09-16T09:26:16.147675+0000","last_undegraded":"2024-09-16T09:26:16.147675+0000","last_fullsized":"2024-09-16T09:26:16.147675+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:27.281102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":68,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.148501+0000","last_change":"2024-09-16T09:25:01.702732+0000","last_active":"2024-09-16T09:26:16.148501+0000","last_peered":"2024-09-16T09:26:16.148501+0000","last_clean":"2024-09-16T09:26:16.148501+0000","last_became_active":"2024-09-16T09:25:01.702419+0000","last_became_peered":"2024-09-16T09:25:01.702419+0000","last_unstale":"2024-09-16T09:26:16.148501+0000","last_undegraded":"2024-09-16T09:26:16.148501+0000","last_fullsized":"2024-09-16T09:26:16.148501+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:12.623278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"66'8","reported_seq":84,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.143179+0000","last_change":"2024-09-16T09:25:05.711929+0000","last_active":"2024-09-16T09:26:16.143179+0000","last_peered":"2024-09-16T09:26:16.143179+0000","last_clean":"2024-09-16T09:26:16.143179+0000","last_became_active":"2024-09-16T09:25:05.711642+0000","last_became_peered":"2024-09-16T09:25:05.711642+0000","last_unstale":"2024-09-16T09:26:16.143179+0000","last_undegraded":"2024-09-16T09:26:16.143179+0000","last_fullsized":"2024-09-16T09:26:16.143179+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:29.307023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":56,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.144965+0000","last_change":"2024-09-16T09:25:07.719123+0000","last_active":"2024-09-16T09:26:16.144965+0000","last_peered":"2024-09-16T09:26:16.144965+0000","last_clean":"2024-09-16T09:26:16.144965+0000","last_became_active":"2024-09-16T09:25:07.718810+0000","last_became_peered":"2024-09-16T09:25:07.718810+0000","last_unstale":"2024-09-16T09:26:16.144965+0000","last_undegraded":"2024-09-16T09:26:16.144965+0000","last_fullsized":"2024-09-16T09:26:16.144965+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:16.185390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"66'10","reported_seq":79,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.145015+0000","last_change":"2024-09-16T09:25:03.712705+0000","last_active":"2024-09-16T09:26:16.145015+0000","last_peered":"2024-09-16T09:26:16.145015+0000","last_clean":"2024-09-16T09:26:16.145015+0000","last_became_active":"2024-09-16T09:25:03.712387+0000","last_became_peered":"2024-09-16T09:25:03.712387+0000","last_unstale":"2024-09-16T09:26:16.145015+0000","last_undegraded":"2024-09-16T09:26:16.145015+0000","last_fullsized":"2024-09-16T09:26:16.145015+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:26.552452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":67,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.148486+0000","last_change":"2024-09-16T09:25:01.708461+0000","last_active":"2024-09-16T09:26:16.148486+0000","last_peered":"2024-09-16T09:26:16.148486+0000","last_clean":"2024-09-16T09:26:16.148486+0000","last_became_active":"2024-09-16T09:25:01.707513+0000","last_became_peered":"2024-09-16T09:25:01.707513+0000","last_unstale":"2024-09-16T09:26:16.148486+0000","last_undegraded":"2024-09-16T09:26:16.148486+0000","last_fullsized":"2024-09-16T09:26:16.148486+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:15.732006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"66'8","reported_seq":83,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.147518+0000","last_change":"2024-09-16T09:25:05.714214+0000","last_active":"2024-09-16T09:26:16.147518+0000","last_peered":"2024-09-16T09:26:16.147518+0000","last_clean":"2024-09-16T09:26:16.147518+0000","last_became_active":"2024-09-16T09:25:05.713833+0000","last_became_peered":"2024-09-16T09:25:05.713833+0000","last_unstale":"2024-09-16T09:26:16.147518+0000","last_undegraded":"2024-09-16T09:26:16.147518+0000","last_fullsized":"2024-09-16T09:26:16.147518+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:11.934656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":56,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.149925+0000","last_change":"2024-09-16T09:25:07.717629+0000","last_active":"2024-09-16T09:26:16.149925+0000","last_peered":"2024-09-16T09:26:16.149925+0000","last_clean":"2024-09-16T09:26:16.149925+0000","last_became_active":"2024-09-16T09:25:07.717357+0000","last_became_peered":"2024-09-16T09:25:07.717357+0000","last_unstale":"2024-09-16T09:26:16.149925+0000","last_undegraded":"2024-09-16T09:26:16.149925+0000","last_fullsized":"2024-09-16T09:26:16.149925+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:28.965425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"66'11","reported_seq":83,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.143579+0000","last_change":"2024-09-16T09:25:03.715601+0000","last_active":"2024-09-16T09:26:16.143579+0000","last_peered":"2024-09-16T09:26:16.143579+0000","last_clean":"2024-09-16T09:26:16.143579+0000","last_became_active":"2024-09-16T09:25:03.715333+0000","last_became_peered":"2024-09-16T09:25:03.715333+0000","last_unstale":"2024-09-16T09:26:16.143579+0000","last_undegraded":"2024-09-16T09:26:16.143579+0000","last_fullsized":"2024-09-16T09:26:16.143579+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:05.795263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":68,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.144074+0000","last_change":"2024-09-16T09:25:01.705459+0000","last_active":"2024-09-16T09:26:16.144074+0000","last_peered":"2024-09-16T09:26:16.144074+0000","last_clean":"2024-09-16T09:26:16.144074+0000","last_became_active":"2024-09-16T09:25:01.705158+0000","last_became_peered":"2024-09-16T09:25:01.705158+0000","last_unstale":"2024-09-16T09:26:16.144074+0000","last_undegraded":"2024-09-16T09:26:16.144074+0000","last_fullsized":"2024-09-16T09:26:16.144074+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:17.236788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":60,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.154608+0000","last_change":"2024-09-16T09:25:05.713087+0000","last_active":"2024-09-16T09:26:16.154608+0000","last_peered":"2024-09-16T09:26:16.154608+0000","last_clean":"2024-09-16T09:26:16.154608+0000","last_became_active":"2024-09-16T09:25:05.712816+0000","last_became_peered":"2024-09-16T09:25:05.712816+0000","last_unstale":"2024-09-16T09:26:16.154608+0000","last_undegraded":"2024-09-16T09:26:16.154608+0000","last_fullsized":"2024-09-16T09:26:16.154608+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:29.267320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":56,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.141921+0000","last_change":"2024-09-16T09:25:07.713617+0000","last_active":"2024-09-16T09:26:16.141921+0000","last_peered":"2024-09-16T09:26:16.141921+0000","last_clean":"2024-09-16T09:26:16.141921+0000","last_became_active":"2024-09-16T09:25:07.713109+0000","last_became_peered":"2024-09-16T09:25:07.713109+0000","last_unstale":"2024-09-16T09:26:16.141921+0000","last_undegraded":"2024-09-16T09:26:16.141921+0000","last_fullsized":"2024-09-16T09:26:16.141921+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:17.988989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"66'9","reported_seq":80,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.154747+0000","last_change":"2024-09-16T09:25:03.706126+0000","last_active":"2024-09-16T09:26:16.154747+0000","last_peered":"2024-09-16T09:26:16.154747+0000","last_clean":"2024-09-16T09:26:16.154747+0000","last_became_active":"2024-09-16T09:25:03.705807+0000","last_became_peered":"2024-09-16T09:25:03.705807+0000","last_unstale":"2024-09-16T09:26:16.154747+0000","last_undegraded":"2024-09-16T09:26:16.154747+0000","last_fullsized":"2024-09-16T09:26:16.154747+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:25.655887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":67,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.149135+0000","last_change":"2024-09-16T09:25:01.707075+0000","last_active":"2024-09-16T09:26:16.149135+0000","last_peered":"2024-09-16T09:26:16.149135+0000","last_clean":"2024-09-16T09:26:16.149135+0000","last_became_active":"2024-09-16T09:25:01.706762+0000","last_became_peered":"2024-09-16T09:25:01.706762+0000","last_unstale":"2024-09-16T09:26:16.149135+0000","last_undegraded":"2024-09-16T09:26:16.149135+0000","last_fullsized":"2024-09-16T09:26:16.149135+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":60,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.142079+0000","last_change":"2024-09-16T09:25:05.714142+0000","last_active":"2024-09-16T09:26:16.142079+0000","last_peered":"2024-09-16T09:26:16.142079+0000","last_clean":"2024-09-16T09:26:16.142079+0000","last_became_active":"2024-09-16T09:25:05.713791+0000","last_became_peered":"2024-09-16T09:25:05.713791+0000","last_unstale":"2024-09-16T09:26:16.142079+0000","last_undegraded":"2024-09-16T09:26:16.142079+0000","last_fullsized":"2024-09-16T09:26:16.142079+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:20.280445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":56,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.147574+0000","last_change":"2024-09-16T09:25:07.715572+0000","last_active":"2024-09-16T09:26:16.147574+0000","last_peered":"2024-09-16T09:26:16.147574+0000","last_clean":"2024-09-16T09:26:16.147574+0000","last_became_active":"2024-09-16T09:25:07.715301+0000","last_became_peered":"2024-09-16T09:25:07.715301+0000","last_unstale":"2024-09-16T09:26:16.147574+0000","last_undegraded":"2024-09-16T09:26:16.147574+0000","last_fullsized":"2024-09-16T09:26:16.147574+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"66'11","reported_seq":83,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.142048+0000","last_change":"2024-09-16T09:25:03.712637+0000","last_active":"2024-09-16T09:26:16.142048+0000","last_peered":"2024-09-16T09:26:16.142048+0000","last_clean":"2024-09-16T09:26:16.142048+0000","last_became_active":"2024-09-16T09:25:03.712333+0000","last_became_peered":"2024-09-16T09:25:03.712333+0000","last_unstale":"2024-09-16T09:26:16.142048+0000","last_undegraded":"2024-09-16T09:26:16.142048+0000","last_fullsized":"2024-09-16T09:26:16.142048+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:10.561474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":67,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.148254+0000","last_change":"2024-09-16T09:25:01.701295+0000","last_active":"2024-09-16T09:26:16.148254+0000","last_peered":"2024-09-16T09:26:16.148254+0000","last_clean":"2024-09-16T09:26:16.148254+0000","last_became_active":"2024-09-16T09:25:01.700733+0000","last_became_peered":"2024-09-16T09:25:01.700733+0000","last_unstale":"2024-09-16T09:26:16.148254+0000","last_undegraded":"2024-09-16T09:26:16.148254+0000","last_fullsized":"2024-09-16T09:26:16.148254+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":59,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.147091+0000","last_change":"2024-09-16T09:25:05.713760+0000","last_active":"2024-09-16T09:26:16.147091+0000","last_peered":"2024-09-16T09:26:16.147091+0000","last_clean":"2024-09-16T09:26:16.147091+0000","last_became_active":"2024-09-16T09:25:05.713479+0000","last_became_peered":"2024-09-16T09:25:05.713479+0000","last_unstale":"2024-09-16T09:26:16.147091+0000","last_undegraded":"2024-09-16T09:26:16.147091+0000","last_fullsized":"2024-09-16T09:26:16.147091+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:31.316771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":56,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.142216+0000","last_change":"2024-09-16T09:25:07.715946+0000","last_active":"2024-09-16T09:26:16.142216+0000","last_peered":"2024-09-16T09:26:16.142216+0000","last_clean":"2024-09-16T09:26:16.142216+0000","last_became_active":"2024-09-16T09:25:07.715582+0000","last_became_peered":"2024-09-16T09:25:07.715582+0000","last_unstale":"2024-09-16T09:26:16.142216+0000","last_undegraded":"2024-09-16T09:26:16.142216+0000","last_fullsized":"2024-09-16T09:26:16.142216+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:18.541254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"66'4","reported_seq":70,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.141549+0000","last_change":"2024-09-16T09:25:03.705167+0000","last_active":"2024-09-16T09:26:16.141549+0000","last_peered":"2024-09-16T09:26:16.141549+0000","last_clean":"2024-09-16T09:26:16.141549+0000","last_became_active":"2024-09-16T09:25:03.704869+0000","last_became_peered":"2024-09-16T09:25:03.704869+0000","last_unstale":"2024-09-16T09:26:16.141549+0000","last_undegraded":"2024-09-16T09:26:16.141549+0000","last_fullsized":"2024-09-16T09:26:16.141549+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:17.043143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":68,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.149846+0000","last_change":"2024-09-16T09:25:01.705965+0000","last_active":"2024-09-16T09:26:16.149846+0000","last_peered":"2024-09-16T09:26:16.149846+0000","last_clean":"2024-09-16T09:26:16.149846+0000","last_became_active":"2024-09-16T09:25:01.705686+0000","last_became_peered":"2024-09-16T09:25:01.705686+0000","last_unstale":"2024-09-16T09:26:16.149846+0000","last_undegraded":"2024-09-16T09:26:16.149846+0000","last_fullsized":"2024-09-16T09:26:16.149846+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":60,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.150611+0000","last_change":"2024-09-16T09:25:05.712816+0000","last_active":"2024-09-16T09:26:16.150611+0000","last_peered":"2024-09-16T09:26:16.150611+0000","last_clean":"2024-09-16T09:26:16.150611+0000","last_became_active":"2024-09-16T09:25:05.712545+0000","last_became_peered":"2024-09-16T09:25:05.712545+0000","last_unstale":"2024-09-16T09:26:16.150611+0000","last_undegraded":"2024-09-16T09:26:16.150611+0000","last_fullsized":"2024-09-16T09:26:16.150611+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:10.110618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":55,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.147766+0000","last_change":"2024-09-16T09:25:07.720368+0000","last_active":"2024-09-16T09:26:16.147766+0000","last_peered":"2024-09-16T09:26:16.147766+0000","last_clean":"2024-09-16T09:26:16.147766+0000","last_became_active":"2024-09-16T09:25:07.720117+0000","last_became_peered":"2024-09-16T09:25:07.720117+0000","last_unstale":"2024-09-16T09:26:16.147766+0000","last_undegraded":"2024-09-16T09:26:16.147766+0000","last_fullsized":"2024-09-16T09:26:16.147766+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:31.504626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":56,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.155135+0000","last_change":"2024-09-16T09:25:07.712880+0000","last_active":"2024-09-16T09:26:16.155135+0000","last_peered":"2024-09-16T09:26:16.155135+0000","last_clean":"2024-09-16T09:26:16.155135+0000","last_became_active":"2024-09-16T09:25:07.712594+0000","last_became_peered":"2024-09-16T09:25:07.712594+0000","last_unstale":"2024-09-16T09:26:16.155135+0000","last_undegraded":"2024-09-16T09:26:16.155135+0000","last_fullsized":"2024-09-16T09:26:16.155135+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:33.865572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":68,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.141747+0000","last_change":"2024-09-16T09:25:01.704711+0000","last_active":"2024-09-16T09:26:16.141747+0000","last_peered":"2024-09-16T09:26:16.141747+0000","last_clean":"2024-09-16T09:26:16.141747+0000","last_became_active":"2024-09-16T09:25:01.703736+0000","last_became_peered":"2024-09-16T09:25:01.703736+0000","last_unstale":"2024-09-16T09:26:16.141747+0000","last_undegraded":"2024-09-16T09:26:16.141747+0000","last_fullsized":"2024-09-16T09:26:16.141747+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:14.597061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"66'11","reported_seq":83,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.151419+0000","last_change":"2024-09-16T09:25:03.708095+0000","last_active":"2024-09-16T09:26:16.151419+0000","last_peered":"2024-09-16T09:26:16.151419+0000","last_clean":"2024-09-16T09:26:16.151419+0000","last_became_active":"2024-09-16T09:25:03.707769+0000","last_became_peered":"2024-09-16T09:25:03.707769+0000","last_unstale":"2024-09-16T09:26:16.151419+0000","last_undegraded":"2024-09-16T09:26:16.151419+0000","last_fullsized":"2024-09-16T09:26:16.151419+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:21.606932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":60,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-09-16T09:26:16.150032+0000","last_change":"2024-09-16T09:25:05.712247+0000","last_active":"2024-09-16T09:26:16.150032+0000","last_peered":"2024-09-16T09:26:16.150032+0000","last_clean":"2024-09-16T09:26:16.150032+0000","last_became_active":"2024-09-16T09:25:05.711952+0000","last_became_peered":"2024-09-16T09:25:05.711952+0000","last_unstale":"2024-09-16T09:26:16.150032+0000","last_undegraded":"2024-09-16T09:26:16.150032+0000","last_fullsized":"2024-09-16T09:26:16.150032+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:26:30.229560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":16,"stat_sum":{"num_bytes":906035200,"num_objects":13825,"num_object_clones":0,"num_object_copies":41475,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13825,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13825,"num_write_kb":884800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2717974528,"data_stored":2717974528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13825,"ondisk_log_size":13825,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":393,"ondisk_log_size":393,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":70,"num_read_kb":65,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266708,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":260260,"kb_used_data":233232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93484892,"statfs":{"total":95995035648,"available":95728529408,"internally_reserved":0,"allocated":238829568,"data_stored":238289414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495641,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":283148,"kb_used_data":256120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93462004,"statfs":{"total":95995035648,"available":95705092096,"internally_reserved":0,"allocated":262266880,"data_stored":261750204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724576,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":236876,"kb_used_data":197368,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93508276,"statfs":{"total":95995035648,"available":95752474624,"internally_reserved":0,"allocated":202104832,"data_stored":201587971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953509,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":283492,"kb_used_data":239824,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":93461660,"statfs":{"total":95995035648,"available":95704739840,"internally_reserved":0,"allocated":245579776,"data_stored":245039142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":44693958},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149739,"num_pgs":65,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":427116,"kb_used_data":395928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93318036,"statfs":{"total":95995035648,"available":95557668864,"internally_reserved":0,"allocated":405430272,"data_stored":404881732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411376,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":319360,"kb_used_data":292332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93425792,"statfs":{"total":95995035648,"available":95668011008,"internally_reserved":0,"allocated":299347968,"data_stored":298842748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574903,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":351496,"kb_used_data":320308,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93393656,"statfs":{"total":95995035648,"available":95635103744,"internally_reserved":0,"allocated":327995392,"data_stored":327418094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738429,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":392488,"kb_used_data":365460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93352664,"statfs":{"total":95995035648,"available":95593127936,"internally_reserved":0,"allocated":374231040,"data_stored":373687056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":453574656,"data_stored":453574656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":341639168,"data_stored":341639168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":338034688,"data_stored":338034688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":506920960,"data_stored":506920960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":283574272,"data_stored":283574272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":226295808,"data_stored":226295808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":283705344,"data_stored":283705344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":284229632,"data_stored":284229632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T09:26:26.661 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2024-09-16T09:26:26.661 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd pool set .mgr pg_num 11 2024-09-16T09:26:26.670 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:26.671 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:26.671 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:26:26.772 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:26:26.838 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:26.838 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:26.839 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:26:26.910 INFO:journalctl@ceph.node-exporter.b.smithi084.stdout:Sep 16 09:26:26 smithi084 podman[64320]: 2024-09-16 09:26:26.592683449 +0000 UTC m=+5.060118842 container init f12d713979d1922a6136120d97c2b707cbb7655c5517be5c89dfc636e2a8b641 (image=quay.io/prometheus/node-exporter:v1.7.0, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-16T09:26:26.910 INFO:journalctl@ceph.node-exporter.b.smithi084.stdout:Sep 16 09:26:26 smithi084 podman[64320]: 2024-09-16 09:26:26.598938043 +0000 UTC m=+5.066373421 container start f12d713979d1922a6136120d97c2b707cbb7655c5517be5c89dfc636e2a8b641 (image=quay.io/prometheus/node-exporter:v1.7.0, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-16T09:26:26.910 INFO:journalctl@ceph.node-exporter.b.smithi084.stdout:Sep 16 09:26:26 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-b[64664]: ts=2024-09-16T09:26:26.602Z caller=node_exporter.go:192 level=info msg="Starting node_exporter" version="(version=1.7.0, branch=HEAD, revision=7333465abf9efba81876303bb57e6fadb946041b)" 2024-09-16T09:26:26.911 INFO:journalctl@ceph.node-exporter.b.smithi084.stdout:Sep 16 09:26:26 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-b[64664]: ts=2024-09-16T09:26:26.603Z caller=node_exporter.go:193 level=info msg="Build context" build_context="(go=go1.21.4, platform=linux/amd64, user=root@35918982f6d8, date=20231112-23:53:35, tags=netgo osusergo static_build)" 2024-09-16T09:26:26.911 INFO:journalctl@ceph.node-exporter.b.smithi084.stdout:Sep 16 09:26:26 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-b[64664]: ts=2024-09-16T09:26:26.603Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-09-16T09:26:26.911 INFO:journalctl@ceph.node-exporter.b.smithi084.stdout:Sep 16 09:26:26 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-b[64664]: ts=2024-09-16T09:26:26.603Z caller=diskstats_linux.go:265 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-09-16T09:26:26.911 INFO:journalctl@ceph.node-exporter.b.smithi084.stdout:Sep 16 09:26:26 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-b[64664]: ts=2024-09-16T09:26:26.603Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-09-16T09:26:26.911 INFO:journalctl@ceph.node-exporter.b.smithi084.stdout:Sep 16 09:26:26 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-b[64664]: ts=2024-09-16T09:26:26.603Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-09-16T09:26:26.911 INFO:journalctl@ceph.node-exporter.b.smithi084.stdout:Sep 16 09:26:26 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-b[64664]: ts=2024-09-16T09:26:26.605Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-09-16T09:26:26.911 INFO:journalctl@ceph.node-exporter.b.smithi084.stdout:Sep 16 09:26:26 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-b[64664]: ts=2024-09-16T09:26:26.605Z caller=node_exporter.go:117 level=info collector=arp 2024-09-16T09:26:26.911 INFO:journalctl@ceph.node-exporter.b.smithi084.stdout:Sep 16 09:26:26 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-b[64664]: ts=2024-09-16T09:26:26.605Z caller=node_exporter.go:117 level=info collector=bcache 2024-09-16T09:26:26.911 INFO:journalctl@ceph.node-exporter.b.smithi084.stdout:Sep 16 09:26:26 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-b[64664]: ts=2024-09-16T09:26:26.605Z caller=node_exporter.go:117 level=info collector=bonding 2024-09-16T09:26:26.911 INFO:journalctl@ceph.node-exporter.b.smithi084.stdout:Sep 16 09:26:26 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-b[64664]: ts=2024-09-16T09:26:26.605Z caller=node_exporter.go:117 level=info collector=btrfs 2024-09-16T09:26:26.911 INFO:journalctl@ceph.node-exporter.b.smithi084.stdout:Sep 16 09:26:26 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-b[64664]: ts=2024-09-16T09:26:26.605Z caller=node_exporter.go:117 level=info collector=conntrack 2024-09-16T09:26:26.911 INFO:journalctl@ceph.node-exporter.b.smithi084.stdout:Sep 16 09:26:26 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-b[64664]: ts=2024-09-16T09:26:26.605Z caller=node_exporter.go:117 level=info collector=cpu 2024-09-16T09:26:26.911 INFO:journalctl@ceph.node-exporter.b.smithi084.stdout:Sep 16 09:26:26 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-b[64664]: ts=2024-09-16T09:26:26.605Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-09-16T09:26:26.912 INFO:journalctl@ceph.node-exporter.b.smithi084.stdout:Sep 16 09:26:26 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-b[64664]: ts=2024-09-16T09:26:26.605Z caller=node_exporter.go:117 level=info collector=diskstats 2024-09-16T09:26:26.912 INFO:journalctl@ceph.node-exporter.b.smithi084.stdout:Sep 16 09:26:26 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-b[64664]: ts=2024-09-16T09:26:26.605Z caller=node_exporter.go:117 level=info collector=dmi 2024-09-16T09:26:26.912 INFO:journalctl@ceph.node-exporter.b.smithi084.stdout:Sep 16 09:26:26 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-b[64664]: ts=2024-09-16T09:26:26.605Z caller=node_exporter.go:117 level=info collector=edac 2024-09-16T09:26:26.912 INFO:journalctl@ceph.node-exporter.b.smithi084.stdout:Sep 16 09:26:26 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-b[64664]: ts=2024-09-16T09:26:26.605Z caller=node_exporter.go:117 level=info collector=entropy 2024-09-16T09:26:26.912 INFO:journalctl@ceph.node-exporter.b.smithi084.stdout:Sep 16 09:26:26 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-b[64664]: ts=2024-09-16T09:26:26.605Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-09-16T09:26:26.912 INFO:journalctl@ceph.node-exporter.b.smithi084.stdout:Sep 16 09:26:26 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-b[64664]: ts=2024-09-16T09:26:26.605Z caller=node_exporter.go:117 level=info collector=filefd 2024-09-16T09:26:26.912 INFO:journalctl@ceph.node-exporter.b.smithi084.stdout:Sep 16 09:26:26 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-b[64664]: ts=2024-09-16T09:26:26.605Z caller=node_exporter.go:117 level=info collector=filesystem 2024-09-16T09:26:26.913 INFO:journalctl@ceph.node-exporter.b.smithi084.stdout:Sep 16 09:26:26 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-b[64664]: ts=2024-09-16T09:26:26.605Z caller=node_exporter.go:117 level=info collector=hwmon 2024-09-16T09:26:26.913 INFO:journalctl@ceph.node-exporter.b.smithi084.stdout:Sep 16 09:26:26 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-b[64664]: ts=2024-09-16T09:26:26.605Z caller=node_exporter.go:117 level=info collector=infiniband 2024-09-16T09:26:26.913 INFO:journalctl@ceph.node-exporter.b.smithi084.stdout:Sep 16 09:26:26 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-b[64664]: ts=2024-09-16T09:26:26.605Z caller=node_exporter.go:117 level=info collector=ipvs 2024-09-16T09:26:26.913 INFO:journalctl@ceph.node-exporter.b.smithi084.stdout:Sep 16 09:26:26 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-b[64664]: ts=2024-09-16T09:26:26.605Z caller=node_exporter.go:117 level=info collector=loadavg 2024-09-16T09:26:26.913 INFO:journalctl@ceph.node-exporter.b.smithi084.stdout:Sep 16 09:26:26 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-b[64664]: ts=2024-09-16T09:26:26.605Z caller=node_exporter.go:117 level=info collector=mdadm 2024-09-16T09:26:26.913 INFO:journalctl@ceph.node-exporter.b.smithi084.stdout:Sep 16 09:26:26 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-b[64664]: ts=2024-09-16T09:26:26.605Z caller=node_exporter.go:117 level=info collector=meminfo 2024-09-16T09:26:26.913 INFO:journalctl@ceph.node-exporter.b.smithi084.stdout:Sep 16 09:26:26 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-b[64664]: ts=2024-09-16T09:26:26.605Z caller=node_exporter.go:117 level=info collector=netclass 2024-09-16T09:26:26.913 INFO:journalctl@ceph.node-exporter.b.smithi084.stdout:Sep 16 09:26:26 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-b[64664]: ts=2024-09-16T09:26:26.605Z caller=node_exporter.go:117 level=info collector=netdev 2024-09-16T09:26:26.913 INFO:journalctl@ceph.node-exporter.b.smithi084.stdout:Sep 16 09:26:26 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-b[64664]: ts=2024-09-16T09:26:26.605Z caller=node_exporter.go:117 level=info collector=netstat 2024-09-16T09:26:26.913 INFO:journalctl@ceph.node-exporter.b.smithi084.stdout:Sep 16 09:26:26 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-b[64664]: ts=2024-09-16T09:26:26.605Z caller=node_exporter.go:117 level=info collector=nfs 2024-09-16T09:26:26.913 INFO:journalctl@ceph.node-exporter.b.smithi084.stdout:Sep 16 09:26:26 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-b[64664]: ts=2024-09-16T09:26:26.606Z caller=node_exporter.go:117 level=info collector=nfsd 2024-09-16T09:26:26.913 INFO:journalctl@ceph.node-exporter.b.smithi084.stdout:Sep 16 09:26:26 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-b[64664]: ts=2024-09-16T09:26:26.606Z caller=node_exporter.go:117 level=info collector=nvme 2024-09-16T09:26:26.913 INFO:journalctl@ceph.node-exporter.b.smithi084.stdout:Sep 16 09:26:26 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-b[64664]: ts=2024-09-16T09:26:26.606Z caller=node_exporter.go:117 level=info collector=os 2024-09-16T09:26:26.913 INFO:journalctl@ceph.node-exporter.b.smithi084.stdout:Sep 16 09:26:26 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-b[64664]: ts=2024-09-16T09:26:26.606Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-09-16T09:26:26.914 INFO:journalctl@ceph.node-exporter.b.smithi084.stdout:Sep 16 09:26:26 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-b[64664]: ts=2024-09-16T09:26:26.606Z caller=node_exporter.go:117 level=info collector=pressure 2024-09-16T09:26:26.914 INFO:journalctl@ceph.node-exporter.b.smithi084.stdout:Sep 16 09:26:26 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-b[64664]: ts=2024-09-16T09:26:26.606Z caller=node_exporter.go:117 level=info collector=rapl 2024-09-16T09:26:26.914 INFO:journalctl@ceph.node-exporter.b.smithi084.stdout:Sep 16 09:26:26 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-b[64664]: ts=2024-09-16T09:26:26.606Z caller=node_exporter.go:117 level=info collector=schedstat 2024-09-16T09:26:26.914 INFO:journalctl@ceph.node-exporter.b.smithi084.stdout:Sep 16 09:26:26 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-b[64664]: ts=2024-09-16T09:26:26.606Z caller=node_exporter.go:117 level=info collector=selinux 2024-09-16T09:26:26.914 INFO:journalctl@ceph.node-exporter.b.smithi084.stdout:Sep 16 09:26:26 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-b[64664]: ts=2024-09-16T09:26:26.606Z caller=node_exporter.go:117 level=info collector=sockstat 2024-09-16T09:26:26.914 INFO:journalctl@ceph.node-exporter.b.smithi084.stdout:Sep 16 09:26:26 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-b[64664]: ts=2024-09-16T09:26:26.606Z caller=node_exporter.go:117 level=info collector=softnet 2024-09-16T09:26:26.914 INFO:journalctl@ceph.node-exporter.b.smithi084.stdout:Sep 16 09:26:26 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-b[64664]: ts=2024-09-16T09:26:26.606Z caller=node_exporter.go:117 level=info collector=stat 2024-09-16T09:26:26.914 INFO:journalctl@ceph.node-exporter.b.smithi084.stdout:Sep 16 09:26:26 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-b[64664]: ts=2024-09-16T09:26:26.606Z caller=node_exporter.go:117 level=info collector=tapestats 2024-09-16T09:26:26.914 INFO:journalctl@ceph.node-exporter.b.smithi084.stdout:Sep 16 09:26:26 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-b[64664]: ts=2024-09-16T09:26:26.606Z caller=node_exporter.go:117 level=info collector=textfile 2024-09-16T09:26:26.914 INFO:journalctl@ceph.node-exporter.b.smithi084.stdout:Sep 16 09:26:26 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-b[64664]: ts=2024-09-16T09:26:26.606Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-09-16T09:26:26.914 INFO:journalctl@ceph.node-exporter.b.smithi084.stdout:Sep 16 09:26:26 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-b[64664]: ts=2024-09-16T09:26:26.606Z caller=node_exporter.go:117 level=info collector=time 2024-09-16T09:26:26.914 INFO:journalctl@ceph.node-exporter.b.smithi084.stdout:Sep 16 09:26:26 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-b[64664]: ts=2024-09-16T09:26:26.606Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-09-16T09:26:26.914 INFO:journalctl@ceph.node-exporter.b.smithi084.stdout:Sep 16 09:26:26 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-b[64664]: ts=2024-09-16T09:26:26.606Z caller=node_exporter.go:117 level=info collector=uname 2024-09-16T09:26:26.914 INFO:journalctl@ceph.node-exporter.b.smithi084.stdout:Sep 16 09:26:26 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-b[64664]: ts=2024-09-16T09:26:26.606Z caller=node_exporter.go:117 level=info collector=vmstat 2024-09-16T09:26:26.914 INFO:journalctl@ceph.node-exporter.b.smithi084.stdout:Sep 16 09:26:26 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-b[64664]: ts=2024-09-16T09:26:26.606Z caller=node_exporter.go:117 level=info collector=xfs 2024-09-16T09:26:26.915 INFO:journalctl@ceph.node-exporter.b.smithi084.stdout:Sep 16 09:26:26 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-b[64664]: ts=2024-09-16T09:26:26.606Z caller=node_exporter.go:117 level=info collector=zfs 2024-09-16T09:26:26.915 INFO:journalctl@ceph.node-exporter.b.smithi084.stdout:Sep 16 09:26:26 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-b[64664]: ts=2024-09-16T09:26:26.607Z caller=tls_config.go:274 level=info msg="Listening on" address=[::]:9100 2024-09-16T09:26:26.915 INFO:journalctl@ceph.node-exporter.b.smithi084.stdout:Sep 16 09:26:26 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-b[64664]: ts=2024-09-16T09:26:26.607Z caller=tls_config.go:277 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-09-16T09:26:26.915 INFO:journalctl@ceph.node-exporter.b.smithi084.stdout:Sep 16 09:26:26 smithi084 bash[64320]: f12d713979d1922a6136120d97c2b707cbb7655c5517be5c89dfc636e2a8b641 2024-09-16T09:26:26.915 INFO:journalctl@ceph.node-exporter.b.smithi084.stdout:Sep 16 09:26:26 smithi084 systemd[1]: Started Ceph node-exporter.b for a1321834-740c-11ef-bceb-c7b262605968. 2024-09-16T09:26:26.939 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:26:26.984 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:26.985 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:26.985 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:26:27.007 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 14 16 33385 33369 148.922 114.875 0.0112316 0.0067093 2024-09-16T09:26:27.086 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:26:27.134 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:27.135 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:27.135 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:26:27.227 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T09:26:27.228 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [ 2024-09-16T09:26:27.228 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:26:27.228 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.24698.0:33647 7.4 e90/82 7:217f3c68:::benchmark_data_smithi070_78438_object33646:head v 90'2043, mlcod=90'2041)", 2024-09-16T09:26:27.228 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:26:27.218208+0000", 2024-09-16T09:26:27.228 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.006988228, 2024-09-16T09:26:27.228 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0070253039999999996, 2024-09-16T09:26:27.228 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:26:27.228 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:26:27.228 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T09:26:27.228 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:26:27.228 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:26:27.228 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:26:27.229 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:27.218208+0000", 2024-09-16T09:26:27.229 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:26:27.229 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:26:27.229 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:26:27.229 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:26:27.229 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:27.218208+0000", 2024-09-16T09:26:27.229 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:26:27.229 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:26:27.229 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:26:27.229 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:26:27.229 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:27.218209+0000", 2024-09-16T09:26:27.229 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1289999999999999e-06 2024-09-16T09:26:27.229 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:26:27.230 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:26:27.230 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:26:27.230 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:27.218241+0000", 2024-09-16T09:26:27.230 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2234999999999999e-05 2024-09-16T09:26:27.230 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:26:27.230 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:26:27.230 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:26:27.230 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:27.218255+0000", 2024-09-16T09:26:27.230 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.346e-05 2024-09-16T09:26:27.230 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:26:27.230 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:26:27.230 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:26:27.230 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:27.218263+0000", 2024-09-16T09:26:27.230 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.4109999999999996e-06 2024-09-16T09:26:27.231 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:26:27.231 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:26:27.231 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:26:27.231 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:27.220854+0000", 2024-09-16T09:26:27.231 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0025911469999999998 2024-09-16T09:26:27.231 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:26:27.231 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:26:27.231 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:26:27.231 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:27.220878+0000", 2024-09-16T09:26:27.231 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.4017999999999999e-05 2024-09-16T09:26:27.231 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:26:27.231 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:26:27.231 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:26:27.231 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:26:27.232 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:26:27.232 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.24698.0:33655 7.b 7:d9fb1baf:::benchmark_data_smithi070_78438_object33654:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-16T09:26:27.232 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:26:27.219408+0000", 2024-09-16T09:26:27.232 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.0057877149999999997, 2024-09-16T09:26:27.232 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0058936309999999999, 2024-09-16T09:26:27.232 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:26:27.232 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:26:27.232 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "waiting for sub ops", 2024-09-16T09:26:27.232 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:26:27.232 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.24698", 2024-09-16T09:26:27.232 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:26:27.232 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 33655 2024-09-16T09:26:27.233 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:26:27.233 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:26:27.233 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:26:27.233 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:26:27.233 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:27.219408+0000", 2024-09-16T09:26:27.233 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:26:27.233 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:26:27.233 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:26:27.233 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:26:27.233 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:27.219408+0000", 2024-09-16T09:26:27.233 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:26:27.233 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:26:27.233 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:26:27.233 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:26:27.234 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:27.219411+0000", 2024-09-16T09:26:27.234 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.119e-06 2024-09-16T09:26:27.234 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:26:27.234 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:26:27.234 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:26:27.234 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:27.219437+0000", 2024-09-16T09:26:27.234 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.6987000000000001e-05 2024-09-16T09:26:27.234 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:26:27.234 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:26:27.234 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:26:27.234 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:27.219454+0000", 2024-09-16T09:26:27.235 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6021000000000001e-05 2024-09-16T09:26:27.235 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:26:27.235 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:26:27.235 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:26:27.235 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:27.219462+0000", 2024-09-16T09:26:27.235 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.6149999999999997e-06 2024-09-16T09:26:27.235 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:26:27.235 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:26:27.235 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:26:27.235 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:27.219531+0000", 2024-09-16T09:26:27.235 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.9132000000000002e-05 2024-09-16T09:26:27.235 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:26:27.236 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:26:27.236 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:26:27.236 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:27.219660+0000", 2024-09-16T09:26:27.236 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00012917500000000001 2024-09-16T09:26:27.236 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:26:27.236 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:26:27.236 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 5,6", 2024-09-16T09:26:27.236 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:27.219786+0000", 2024-09-16T09:26:27.236 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000125559 2024-09-16T09:26:27.236 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:26:27.236 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:26:27.236 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:26:27.237 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:27.221361+0000", 2024-09-16T09:26:27.237 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0015751750000000001 2024-09-16T09:26:27.237 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:26:27.237 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:26:27.238 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:26:27.238 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:27.221405+0000", 2024-09-16T09:26:27.238 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.3961999999999997e-05 2024-09-16T09:26:27.238 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:26:27.238 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:26:27.238 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:26:27.238 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:26:27.238 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:26:27.239 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.24698.0:33648 7.9 e90/82 7:9b3cbfd3:::benchmark_data_smithi070_78438_object33647:head v 90'2104, mlcod=90'2101)", 2024-09-16T09:26:27.239 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:26:27.222447+0000", 2024-09-16T09:26:27.239 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.00274932, 2024-09-16T09:26:27.239 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0029325269999999999, 2024-09-16T09:26:27.239 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:26:27.239 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:26:27.239 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T09:26:27.239 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:26:27.239 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:26:27.240 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:26:27.240 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:27.222447+0000", 2024-09-16T09:26:27.240 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:26:27.240 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:26:27.240 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:26:27.240 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:26:27.240 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:27.222447+0000", 2024-09-16T09:26:27.240 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:26:27.240 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:26:27.241 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:26:27.241 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:26:27.241 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:27.222448+0000", 2024-09-16T09:26:27.241 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.1100000000000004e-07 2024-09-16T09:26:27.241 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:26:27.241 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:26:27.241 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:26:27.241 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:27.222476+0000", 2024-09-16T09:26:27.241 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.7943999999999999e-05 2024-09-16T09:26:27.241 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:26:27.242 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:26:27.242 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:26:27.242 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:27.222493+0000", 2024-09-16T09:26:27.242 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6974999999999999e-05 2024-09-16T09:26:27.242 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:26:27.242 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:26:27.242 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:26:27.242 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:27.222500+0000", 2024-09-16T09:26:27.242 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.9489999999999999e-06 2024-09-16T09:26:27.242 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:26:27.243 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:26:27.243 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:26:27.243 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:27.222576+0000", 2024-09-16T09:26:27.243 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6038000000000006e-05 2024-09-16T09:26:27.243 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:26:27.243 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:26:27.243 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:26:27.243 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:27.222599+0000", 2024-09-16T09:26:27.243 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3328000000000001e-05 2024-09-16T09:26:27.244 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:26:27.244 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:26:27.244 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:26:27.244 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:26:27.244 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-16T09:26:27.244 INFO:teuthology.orchestra.run.smithi070.stdout: "num_ops": 3 2024-09-16T09:26:27.244 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T09:26:27.245 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:26:27.294 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:27.295 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:27.295 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:26:27.397 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:26:27.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:27 smithi070 ceph-mon[30388]: pgmap v47: 148 pgs: 148 active+clean; 1.5 GiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 800 B/s rd, 108 MiB/s wr, 1.72k op/s 2024-09-16T09:26:27.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:27 smithi070 ceph-mon[30388]: 3.c scrub starts 2024-09-16T09:26:27.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:27 smithi070 ceph-mon[30388]: 3.c scrub ok 2024-09-16T09:26:27.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:27 smithi070 ceph-mon[30388]: 3.0 scrub starts 2024-09-16T09:26:27.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:27 smithi070 ceph-mon[30388]: 3.0 scrub ok 2024-09-16T09:26:27.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:27 smithi070 ceph-mon[30388]: 3.f scrub starts 2024-09-16T09:26:27.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:27 smithi070 ceph-mon[30388]: 3.f scrub ok 2024-09-16T09:26:27.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:27 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:26:27.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:27 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:26:27.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:27 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:26:27.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:27 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:26:27.422 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:27 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:26:27.422 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:27 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:26:27.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:27 smithi070 ceph-mon[40691]: pgmap v47: 148 pgs: 148 active+clean; 1.5 GiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 800 B/s rd, 108 MiB/s wr, 1.72k op/s 2024-09-16T09:26:27.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:27 smithi070 ceph-mon[40691]: 3.c scrub starts 2024-09-16T09:26:27.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:27 smithi070 ceph-mon[40691]: 3.c scrub ok 2024-09-16T09:26:27.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:27 smithi070 ceph-mon[40691]: 3.0 scrub starts 2024-09-16T09:26:27.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:27 smithi070 ceph-mon[40691]: 3.0 scrub ok 2024-09-16T09:26:27.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:27 smithi070 ceph-mon[40691]: 3.f scrub starts 2024-09-16T09:26:27.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:27 smithi070 ceph-mon[40691]: 3.f scrub ok 2024-09-16T09:26:27.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:27 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:26:27.423 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:27 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:26:27.423 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:27 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:26:27.423 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:27 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:26:27.423 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:27 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:26:27.423 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:27 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:26:27.426 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:27.426 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:27.427 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:26:27.524 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:27 smithi084 ceph-mon[32629]: pgmap v47: 148 pgs: 148 active+clean; 1.5 GiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 800 B/s rd, 108 MiB/s wr, 1.72k op/s 2024-09-16T09:26:27.525 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:27 smithi084 ceph-mon[32629]: 3.c scrub starts 2024-09-16T09:26:27.525 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:27 smithi084 ceph-mon[32629]: 3.c scrub ok 2024-09-16T09:26:27.525 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:27 smithi084 ceph-mon[32629]: 3.0 scrub starts 2024-09-16T09:26:27.525 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:27 smithi084 ceph-mon[32629]: 3.0 scrub ok 2024-09-16T09:26:27.525 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:27 smithi084 ceph-mon[32629]: 3.f scrub starts 2024-09-16T09:26:27.525 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:27 smithi084 ceph-mon[32629]: 3.f scrub ok 2024-09-16T09:26:27.525 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:27 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:26:27.525 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:27 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:26:27.525 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:27 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:26:27.525 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:27 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:26:27.525 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:27 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:26:27.525 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:27 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:26:27.528 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:26:27.575 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:27.576 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:27.576 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:26:27.677 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:26:27.714 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:27.715 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:27.715 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:26:27.816 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:26:27.856 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:27.857 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:27.857 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:26:27.958 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:26:27.998 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:27.999 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:27.999 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:26:28.007 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 15 15 35229 35214 146.682 115.312 0.00883878 0.00681216 2024-09-16T09:26:28.070 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:26:28.072 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:26:28.080 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:26:28.085 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.1 dump_blocked_ops 2024-09-16T09:26:28.100 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:26:28.180 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:28.181 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:28.181 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:26:28.282 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:26:28.311 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:28.311 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:28.312 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:26:28.366 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:28 smithi084 ceph-mon[32629]: 4.7 scrub starts 2024-09-16T09:26:28.366 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:28 smithi084 ceph-mon[32629]: 4.7 scrub ok 2024-09-16T09:26:28.366 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:28 smithi084 ceph-mon[32629]: 6.1e scrub starts 2024-09-16T09:26:28.366 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:28 smithi084 ceph-mon[32629]: 6.1e scrub ok 2024-09-16T09:26:28.413 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:26:28.415 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:28 smithi070 ceph-mon[30388]: 4.7 scrub starts 2024-09-16T09:26:28.415 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:28 smithi070 ceph-mon[30388]: 4.7 scrub ok 2024-09-16T09:26:28.415 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:28 smithi070 ceph-mon[30388]: 6.1e scrub starts 2024-09-16T09:26:28.415 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:28 smithi070 ceph-mon[30388]: 6.1e scrub ok 2024-09-16T09:26:28.416 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:28 smithi070 ceph-mon[40691]: 4.7 scrub starts 2024-09-16T09:26:28.416 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:28 smithi070 ceph-mon[40691]: 4.7 scrub ok 2024-09-16T09:26:28.416 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:28 smithi070 ceph-mon[40691]: 6.1e scrub starts 2024-09-16T09:26:28.416 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:28 smithi070 ceph-mon[40691]: 6.1e scrub ok 2024-09-16T09:26:28.439 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:28.440 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:28.440 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:26:28.541 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:26:28.571 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:28.572 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:28.572 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:26:28.673 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:26:28.724 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:28.725 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:28.725 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:26:28.826 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:26:28.861 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:28.861 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:28.862 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:26:28.963 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:26:29.006 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:29.007 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:29.007 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:26:29.007 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 16 16 36991 36975 144.393 110.062 0.0091808 0.0069186 2024-09-16T09:26:29.108 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:26:29.134 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:29.135 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:29.136 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:26:29.237 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:26:29.265 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:29.265 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:29.266 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:26:29.366 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:26:29.404 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:29.405 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:29.405 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:26:29.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:29 smithi070 ceph-mon[30388]: 4.10 scrub starts 2024-09-16T09:26:29.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:29 smithi070 ceph-mon[30388]: 4.10 scrub ok 2024-09-16T09:26:29.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:29 smithi070 ceph-mon[30388]: pgmap v48: 148 pgs: 148 active+clean; 1.9 GiB data, 6.1 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 141 MiB/s wr, 2.26k op/s 2024-09-16T09:26:29.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:29 smithi070 ceph-mon[30388]: 5.1f scrub starts 2024-09-16T09:26:29.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:29 smithi070 ceph-mon[30388]: 5.1f scrub ok 2024-09-16T09:26:29.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:29 smithi070 ceph-mon[30388]: 4.2 scrub starts 2024-09-16T09:26:29.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:29 smithi070 ceph-mon[30388]: 4.2 scrub ok 2024-09-16T09:26:29.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:29 smithi070 ceph-mon[30388]: 4.b scrub starts 2024-09-16T09:26:29.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:29 smithi070 ceph-mon[30388]: 4.b scrub ok 2024-09-16T09:26:29.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:29 smithi070 ceph-mon[30388]: 6.15 scrub starts 2024-09-16T09:26:29.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:29 smithi070 ceph-mon[30388]: 6.15 scrub ok 2024-09-16T09:26:29.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:29 smithi070 ceph-mon[40691]: 4.10 scrub starts 2024-09-16T09:26:29.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:29 smithi070 ceph-mon[40691]: 4.10 scrub ok 2024-09-16T09:26:29.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:29 smithi070 ceph-mon[40691]: pgmap v48: 148 pgs: 148 active+clean; 1.9 GiB data, 6.1 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 141 MiB/s wr, 2.26k op/s 2024-09-16T09:26:29.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:29 smithi070 ceph-mon[40691]: 5.1f scrub starts 2024-09-16T09:26:29.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:29 smithi070 ceph-mon[40691]: 5.1f scrub ok 2024-09-16T09:26:29.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:29 smithi070 ceph-mon[40691]: 4.2 scrub starts 2024-09-16T09:26:29.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:29 smithi070 ceph-mon[40691]: 4.2 scrub ok 2024-09-16T09:26:29.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:29 smithi070 ceph-mon[40691]: 4.b scrub starts 2024-09-16T09:26:29.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:29 smithi070 ceph-mon[40691]: 4.b scrub ok 2024-09-16T09:26:29.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:29 smithi070 ceph-mon[40691]: 6.15 scrub starts 2024-09-16T09:26:29.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:29 smithi070 ceph-mon[40691]: 6.15 scrub ok 2024-09-16T09:26:29.506 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:26:29.545 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:29.546 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:29.546 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:26:29.647 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:26:29.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:29 smithi084 ceph-mon[32629]: 4.10 scrub starts 2024-09-16T09:26:29.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:29 smithi084 ceph-mon[32629]: 4.10 scrub ok 2024-09-16T09:26:29.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:29 smithi084 ceph-mon[32629]: pgmap v48: 148 pgs: 148 active+clean; 1.9 GiB data, 6.1 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 141 MiB/s wr, 2.26k op/s 2024-09-16T09:26:29.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:29 smithi084 ceph-mon[32629]: 5.1f scrub starts 2024-09-16T09:26:29.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:29 smithi084 ceph-mon[32629]: 5.1f scrub ok 2024-09-16T09:26:29.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:29 smithi084 ceph-mon[32629]: 4.2 scrub starts 2024-09-16T09:26:29.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:29 smithi084 ceph-mon[32629]: 4.2 scrub ok 2024-09-16T09:26:29.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:29 smithi084 ceph-mon[32629]: 4.b scrub starts 2024-09-16T09:26:29.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:29 smithi084 ceph-mon[32629]: 4.b scrub ok 2024-09-16T09:26:29.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:29 smithi084 ceph-mon[32629]: 6.15 scrub starts 2024-09-16T09:26:29.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:29 smithi084 ceph-mon[32629]: 6.15 scrub ok 2024-09-16T09:26:29.684 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:29.685 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:29.685 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:26:29.786 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:26:29.836 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:29.837 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:29.837 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:26:29.938 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:26:29.987 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:29.988 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:29.988 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:26:30.007 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 17 16 38528 38512 141.55 96.0625 0.00778242 0.00705866 2024-09-16T09:26:30.024 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:26:30.090 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:26:30.097 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:26:30.098 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:26:30.119 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:30.120 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:30.120 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:26:30.130 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:26:30.130 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:26:30.170 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:26:30.170 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:26:30.205 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:26:30.206 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:26:30.220 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:26:30.238 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:26:30.238 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:26:30.259 INFO:teuthology.orchestra.run.smithi070.stderr:nodeep-scrub is unset 2024-09-16T09:26:30.263 INFO:teuthology.orchestra.run.smithi070.stderr:set pool 1 pg_num to 11 2024-09-16T09:26:30.277 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:26:30.278 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:26:30.278 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:30.279 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:30.279 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:26:30.315 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:26:30.315 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:26:30.379 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:26:30.379 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:26:30.380 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:26:30.409 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:30.409 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:30.410 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:26:30.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:30 smithi084 ceph-mon[32629]: 3.7 scrub starts 2024-09-16T09:26:30.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:30 smithi084 ceph-mon[32629]: 3.7 scrub ok 2024-09-16T09:26:30.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:30 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/4172696199' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:26:30.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:30 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1696370769' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "11"} : dispatch 2024-09-16T09:26:30.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:30 smithi084 ceph-mon[32629]: 6.0 scrub starts 2024-09-16T09:26:30.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:30 smithi084 ceph-mon[32629]: 6.0 scrub ok 2024-09-16T09:26:30.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:30 smithi084 ceph-mon[32629]: 5.8 scrub starts 2024-09-16T09:26:30.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:30 smithi084 ceph-mon[32629]: 5.8 scrub ok 2024-09-16T09:26:30.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:30 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3482762850' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:26:30.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:30 smithi070 ceph-mon[30388]: 3.7 scrub starts 2024-09-16T09:26:30.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:30 smithi070 ceph-mon[30388]: 3.7 scrub ok 2024-09-16T09:26:30.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:30 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/4172696199' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:26:30.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:30 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1696370769' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "11"} : dispatch 2024-09-16T09:26:30.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:30 smithi070 ceph-mon[30388]: 6.0 scrub starts 2024-09-16T09:26:30.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:30 smithi070 ceph-mon[30388]: 6.0 scrub ok 2024-09-16T09:26:30.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:30 smithi070 ceph-mon[30388]: 5.8 scrub starts 2024-09-16T09:26:30.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:30 smithi070 ceph-mon[30388]: 5.8 scrub ok 2024-09-16T09:26:30.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:30 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3482762850' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:26:30.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:30 smithi070 ceph-mon[40691]: 3.7 scrub starts 2024-09-16T09:26:30.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:30 smithi070 ceph-mon[40691]: 3.7 scrub ok 2024-09-16T09:26:30.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:30 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/4172696199' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:26:30.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:30 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1696370769' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "11"} : dispatch 2024-09-16T09:26:30.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:30 smithi070 ceph-mon[40691]: 6.0 scrub starts 2024-09-16T09:26:30.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:30 smithi070 ceph-mon[40691]: 6.0 scrub ok 2024-09-16T09:26:30.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:30 smithi070 ceph-mon[40691]: 5.8 scrub starts 2024-09-16T09:26:30.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:30 smithi070 ceph-mon[40691]: 5.8 scrub ok 2024-09-16T09:26:30.423 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:30 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3482762850' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:26:30.511 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:26:30.556 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:30.557 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:30.557 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:26:30.658 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:26:30.685 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:30.686 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:30.686 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:26:30.787 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:26:30.815 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:30.816 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:30.817 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:26:30.917 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:26:30.946 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:30.947 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:30.947 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:26:31.009 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 18 16 40219 40203 139.552 105.688 0.00361514 0.00716056 2024-09-16T09:26:31.048 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:26:31.082 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:31.083 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:31.083 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:26:31.115 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T09:26:31.115 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:26:31.184 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:26:31.212 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:31.213 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:31.214 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:26:31.315 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:26:31.355 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:31.356 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:31.356 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:26:31.457 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:26:31.487 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:31.488 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:31.488 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:26:31.550 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:31 smithi084 ceph-mon[32629]: pgmap v49: 148 pgs: 148 active+clean; 2.1 GiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 153 MiB/s wr, 2.45k op/s 2024-09-16T09:26:31.550 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:31 smithi084 ceph-mon[32629]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-16T09:26:31.550 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:31 smithi084 ceph-mon[32629]: Cluster is now healthy 2024-09-16T09:26:31.550 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:31 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/4172696199' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-16T09:26:31.551 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:31 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1696370769' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "11"}]': finished 2024-09-16T09:26:31.551 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:31 smithi084 ceph-mon[32629]: osdmap e91: 8 total, 8 up, 8 in 2024-09-16T09:26:31.551 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:31 smithi084 ceph-mon[32629]: 4.1e scrub starts 2024-09-16T09:26:31.551 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:31 smithi084 ceph-mon[32629]: 4.1e scrub ok 2024-09-16T09:26:31.551 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:31 smithi084 ceph-mon[32629]: 5.6 scrub starts 2024-09-16T09:26:31.551 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:31 smithi084 ceph-mon[32629]: 5.6 scrub ok 2024-09-16T09:26:31.589 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:26:31.631 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:31.632 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:31.632 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:26:31.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:31 smithi070 ceph-mon[30388]: pgmap v49: 148 pgs: 148 active+clean; 2.1 GiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 153 MiB/s wr, 2.45k op/s 2024-09-16T09:26:31.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:31 smithi070 ceph-mon[30388]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-16T09:26:31.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:31 smithi070 ceph-mon[30388]: Cluster is now healthy 2024-09-16T09:26:31.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:31 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/4172696199' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-16T09:26:31.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:31 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1696370769' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "11"}]': finished 2024-09-16T09:26:31.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:31 smithi070 ceph-mon[30388]: osdmap e91: 8 total, 8 up, 8 in 2024-09-16T09:26:31.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:31 smithi070 ceph-mon[30388]: 4.1e scrub starts 2024-09-16T09:26:31.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:31 smithi070 ceph-mon[30388]: 4.1e scrub ok 2024-09-16T09:26:31.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:31 smithi070 ceph-mon[30388]: 5.6 scrub starts 2024-09-16T09:26:31.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:31 smithi070 ceph-mon[30388]: 5.6 scrub ok 2024-09-16T09:26:31.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:31 smithi070 ceph-mon[40691]: pgmap v49: 148 pgs: 148 active+clean; 2.1 GiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 153 MiB/s wr, 2.45k op/s 2024-09-16T09:26:31.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:31 smithi070 ceph-mon[40691]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-16T09:26:31.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:31 smithi070 ceph-mon[40691]: Cluster is now healthy 2024-09-16T09:26:31.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:31 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/4172696199' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-16T09:26:31.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:31 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1696370769' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "11"}]': finished 2024-09-16T09:26:31.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:31 smithi070 ceph-mon[40691]: osdmap e91: 8 total, 8 up, 8 in 2024-09-16T09:26:31.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:31 smithi070 ceph-mon[40691]: 4.1e scrub starts 2024-09-16T09:26:31.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:31 smithi070 ceph-mon[40691]: 4.1e scrub ok 2024-09-16T09:26:31.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:31 smithi070 ceph-mon[40691]: 5.6 scrub starts 2024-09-16T09:26:31.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:31 smithi070 ceph-mon[40691]: 5.6 scrub ok 2024-09-16T09:26:31.733 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:26:31.776 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:31.776 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:31.777 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:26:31.878 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:26:31.916 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:31.916 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:31.917 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:26:32.008 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 19 16 42084 42068 138.341 116.562 0.00370795 0.00722241 2024-09-16T09:26:32.017 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:26:32.045 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:32.046 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:32.046 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:26:32.134 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:26:31 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:26:31] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:26:32.147 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:26:32.174 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:32.174 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:32.175 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:26:32.276 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:26:32.316 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:32.317 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:32.318 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:26:32.418 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:26:32.422 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:32 smithi070 ceph-mon[30388]: 6.11 scrub starts 2024-09-16T09:26:32.422 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:32 smithi070 ceph-mon[30388]: 6.11 scrub ok 2024-09-16T09:26:32.422 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:32 smithi070 ceph-mon[30388]: 4.12 scrub starts 2024-09-16T09:26:32.422 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:32 smithi070 ceph-mon[30388]: 4.12 scrub ok 2024-09-16T09:26:32.422 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:32 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "11"} : dispatch 2024-09-16T09:26:32.422 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:32 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "11"} : dispatch 2024-09-16T09:26:32.422 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:32 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]} : dispatch 2024-09-16T09:26:32.422 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:32 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]} : dispatch 2024-09-16T09:26:32.422 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:32 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 5]} : dispatch 2024-09-16T09:26:32.422 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:32 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]} : dispatch 2024-09-16T09:26:32.422 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:32 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]} : dispatch 2024-09-16T09:26:32.422 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:32 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 5]} : dispatch 2024-09-16T09:26:32.423 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:32 smithi070 ceph-mon[30388]: 1.0 scrub starts 2024-09-16T09:26:32.423 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:32 smithi070 ceph-mon[30388]: 1.0 scrub ok 2024-09-16T09:26:32.423 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:32 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:26:32.423 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:32 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:26:32.423 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:32 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:26:32.423 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:32 smithi070 ceph-mon[40691]: 6.11 scrub starts 2024-09-16T09:26:32.423 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:32 smithi070 ceph-mon[40691]: 6.11 scrub ok 2024-09-16T09:26:32.423 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:32 smithi070 ceph-mon[40691]: 4.12 scrub starts 2024-09-16T09:26:32.423 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:32 smithi070 ceph-mon[40691]: 4.12 scrub ok 2024-09-16T09:26:32.423 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:32 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "11"} : dispatch 2024-09-16T09:26:32.423 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:32 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "11"} : dispatch 2024-09-16T09:26:32.423 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:32 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]} : dispatch 2024-09-16T09:26:32.424 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:32 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]} : dispatch 2024-09-16T09:26:32.424 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:32 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 5]} : dispatch 2024-09-16T09:26:32.424 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:32 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]} : dispatch 2024-09-16T09:26:32.424 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:32 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]} : dispatch 2024-09-16T09:26:32.424 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:32 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 5]} : dispatch 2024-09-16T09:26:32.424 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:32 smithi070 ceph-mon[40691]: 1.0 scrub starts 2024-09-16T09:26:32.424 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:32 smithi070 ceph-mon[40691]: 1.0 scrub ok 2024-09-16T09:26:32.424 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:32 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:26:32.424 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:32 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:26:32.424 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:32 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:26:32.454 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:32.455 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:32.455 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:26:32.548 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:32 smithi084 ceph-mon[32629]: 6.11 scrub starts 2024-09-16T09:26:32.549 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:32 smithi084 ceph-mon[32629]: 6.11 scrub ok 2024-09-16T09:26:32.549 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:32 smithi084 ceph-mon[32629]: 4.12 scrub starts 2024-09-16T09:26:32.549 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:32 smithi084 ceph-mon[32629]: 4.12 scrub ok 2024-09-16T09:26:32.549 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:32 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "11"} : dispatch 2024-09-16T09:26:32.549 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:32 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "11"} : dispatch 2024-09-16T09:26:32.549 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:32 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]} : dispatch 2024-09-16T09:26:32.549 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:32 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]} : dispatch 2024-09-16T09:26:32.549 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:32 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 5]} : dispatch 2024-09-16T09:26:32.549 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:32 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]} : dispatch 2024-09-16T09:26:32.549 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:32 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]} : dispatch 2024-09-16T09:26:32.549 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:32 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 5]} : dispatch 2024-09-16T09:26:32.550 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:32 smithi084 ceph-mon[32629]: 1.0 scrub starts 2024-09-16T09:26:32.550 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:32 smithi084 ceph-mon[32629]: 1.0 scrub ok 2024-09-16T09:26:32.550 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:32 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:26:32.550 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:32 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:26:32.550 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:32 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:26:32.556 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:26:32.593 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:32.594 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:32.594 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:26:32.695 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:26:32.725 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:32.726 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:32.726 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:26:32.827 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:26:32.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:32.867 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:32.868 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:26:32.910 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:26:32 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:26:32.968 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:26:32.998 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:32.999 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:33.000 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:26:33.008 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:2024-09-16T09:26:33.007981+0000 min lat: 0.0012412 max lat: 0.110436 avg lat: 0.00728146 2024-09-16T09:26:33.008 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-16T09:26:33.008 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 20 16 43937 43921 137.215 115.812 0.00571194 0.00728146 2024-09-16T09:26:33.101 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:26:33.133 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:33.134 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:33.135 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:26:33.236 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:26:33.272 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:33.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:33.273 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:26:33.374 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:26:33.405 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:33 smithi084 ceph-mon[32629]: pgmap v51: 148 pgs: 148 active+clean; 2.1 GiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 639 B/s rd, 73 MiB/s wr, 1.16k op/s 2024-09-16T09:26:33.405 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:33 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "11"}]': finished 2024-09-16T09:26:33.405 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:33 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]': finished 2024-09-16T09:26:33.405 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:33 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]': finished 2024-09-16T09:26:33.405 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:33 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 5]}]': finished 2024-09-16T09:26:33.405 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:33 smithi084 ceph-mon[32629]: osdmap e92: 8 total, 8 up, 8 in 2024-09-16T09:26:33.405 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:33 smithi084 ceph-mon[32629]: 6.13 scrub starts 2024-09-16T09:26:33.406 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:33 smithi084 ceph-mon[32629]: 6.13 scrub ok 2024-09-16T09:26:33.406 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:33 smithi084 ceph-mon[32629]: 3.1a scrub starts 2024-09-16T09:26:33.406 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:33 smithi084 ceph-mon[32629]: 3.1a scrub ok 2024-09-16T09:26:33.413 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:33.413 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:33.414 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:26:33.515 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:26:33.545 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:33.545 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:33.546 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:26:33.647 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:26:33.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:33 smithi070 ceph-mon[30388]: pgmap v51: 148 pgs: 148 active+clean; 2.1 GiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 639 B/s rd, 73 MiB/s wr, 1.16k op/s 2024-09-16T09:26:33.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:33 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "11"}]': finished 2024-09-16T09:26:33.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:33 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]': finished 2024-09-16T09:26:33.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:33 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]': finished 2024-09-16T09:26:33.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:33 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 5]}]': finished 2024-09-16T09:26:33.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:33 smithi070 ceph-mon[30388]: osdmap e92: 8 total, 8 up, 8 in 2024-09-16T09:26:33.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:33 smithi070 ceph-mon[30388]: 6.13 scrub starts 2024-09-16T09:26:33.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:33 smithi070 ceph-mon[30388]: 6.13 scrub ok 2024-09-16T09:26:33.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:33 smithi070 ceph-mon[30388]: 3.1a scrub starts 2024-09-16T09:26:33.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:33 smithi070 ceph-mon[30388]: 3.1a scrub ok 2024-09-16T09:26:33.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:33 smithi070 ceph-mon[40691]: pgmap v51: 148 pgs: 148 active+clean; 2.1 GiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 639 B/s rd, 73 MiB/s wr, 1.16k op/s 2024-09-16T09:26:33.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:33 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "11"}]': finished 2024-09-16T09:26:33.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:33 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]': finished 2024-09-16T09:26:33.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:33 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]': finished 2024-09-16T09:26:33.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:33 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 5]}]': finished 2024-09-16T09:26:33.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:33 smithi070 ceph-mon[40691]: osdmap e92: 8 total, 8 up, 8 in 2024-09-16T09:26:33.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:33 smithi070 ceph-mon[40691]: 6.13 scrub starts 2024-09-16T09:26:33.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:33 smithi070 ceph-mon[40691]: 6.13 scrub ok 2024-09-16T09:26:33.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:33 smithi070 ceph-mon[40691]: 3.1a scrub starts 2024-09-16T09:26:33.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:33 smithi070 ceph-mon[40691]: 3.1a scrub ok 2024-09-16T09:26:33.674 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:33.675 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:33.676 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:26:33.776 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:26:33.804 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:33.804 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:33.805 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:26:33.817 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.1 dump_historic_ops 2024-09-16T09:26:33.906 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:26:33.944 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:33.945 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:33.945 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:26:34.008 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 21 15 45672 45657 135.846 108.5 0.00491018 0.00735582 2024-09-16T09:26:34.046 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:26:34.089 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:34.090 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:34.090 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:26:34.191 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:26:34.250 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:34.250 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:34.251 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:26:34.281 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:26:34.352 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:26:34.381 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:34.382 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:34.382 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:26:34.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:34 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:26:34.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:34 smithi070 ceph-mon[30388]: 4.8 scrub starts 2024-09-16T09:26:34.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:34 smithi070 ceph-mon[30388]: 3.10 scrub starts 2024-09-16T09:26:34.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:34 smithi070 ceph-mon[30388]: 4.8 scrub ok 2024-09-16T09:26:34.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:34 smithi070 ceph-mon[30388]: 3.10 scrub ok 2024-09-16T09:26:34.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:34 smithi070 ceph-mon[30388]: osdmap e93: 8 total, 8 up, 8 in 2024-09-16T09:26:34.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:34 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "11"} : dispatch 2024-09-16T09:26:34.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:34 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "11"} : dispatch 2024-09-16T09:26:34.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:34 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:26:34.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:34 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:26:34.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:34 smithi070 ceph-mon[30388]: 1.1 scrub starts 2024-09-16T09:26:34.422 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:34 smithi070 ceph-mon[30388]: 1.1 scrub ok 2024-09-16T09:26:34.422 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:34 smithi070 ceph-mon[30388]: 5.1 scrub starts 2024-09-16T09:26:34.422 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:34 smithi070 ceph-mon[30388]: 5.1 scrub ok 2024-09-16T09:26:34.422 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:34 smithi070 ceph-mon[30388]: 4.1c scrub starts 2024-09-16T09:26:34.422 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:34 smithi070 ceph-mon[30388]: 4.1c scrub ok 2024-09-16T09:26:34.422 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:34 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "11"}]': finished 2024-09-16T09:26:34.422 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:34 smithi070 ceph-mon[30388]: osdmap e94: 8 total, 8 up, 8 in 2024-09-16T09:26:34.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:34 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:26:34.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:34 smithi070 ceph-mon[40691]: 4.8 scrub starts 2024-09-16T09:26:34.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:34 smithi070 ceph-mon[40691]: 3.10 scrub starts 2024-09-16T09:26:34.423 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:34 smithi070 ceph-mon[40691]: 4.8 scrub ok 2024-09-16T09:26:34.423 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:34 smithi070 ceph-mon[40691]: 3.10 scrub ok 2024-09-16T09:26:34.423 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:34 smithi070 ceph-mon[40691]: osdmap e93: 8 total, 8 up, 8 in 2024-09-16T09:26:34.423 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:34 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "11"} : dispatch 2024-09-16T09:26:34.423 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:34 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "11"} : dispatch 2024-09-16T09:26:34.423 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:34 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:26:34.423 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:34 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:26:34.423 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:34 smithi070 ceph-mon[40691]: 1.1 scrub starts 2024-09-16T09:26:34.423 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:34 smithi070 ceph-mon[40691]: 1.1 scrub ok 2024-09-16T09:26:34.423 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:34 smithi070 ceph-mon[40691]: 5.1 scrub starts 2024-09-16T09:26:34.423 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:34 smithi070 ceph-mon[40691]: 5.1 scrub ok 2024-09-16T09:26:34.423 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:34 smithi070 ceph-mon[40691]: 4.1c scrub starts 2024-09-16T09:26:34.423 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:34 smithi070 ceph-mon[40691]: 4.1c scrub ok 2024-09-16T09:26:34.423 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:34 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "11"}]': finished 2024-09-16T09:26:34.424 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:34 smithi070 ceph-mon[40691]: osdmap e94: 8 total, 8 up, 8 in 2024-09-16T09:26:34.483 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:26:34.509 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:34.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:34.510 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:26:34.611 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:26:34.638 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:34.638 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:34.639 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:26:34.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:34 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:26:34.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:34 smithi084 ceph-mon[32629]: 4.8 scrub starts 2024-09-16T09:26:34.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:34 smithi084 ceph-mon[32629]: 3.10 scrub starts 2024-09-16T09:26:34.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:34 smithi084 ceph-mon[32629]: 4.8 scrub ok 2024-09-16T09:26:34.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:34 smithi084 ceph-mon[32629]: 3.10 scrub ok 2024-09-16T09:26:34.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:34 smithi084 ceph-mon[32629]: osdmap e93: 8 total, 8 up, 8 in 2024-09-16T09:26:34.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:34 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "11"} : dispatch 2024-09-16T09:26:34.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:34 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "11"} : dispatch 2024-09-16T09:26:34.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:34 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:26:34.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:34 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:26:34.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:34 smithi084 ceph-mon[32629]: 1.1 scrub starts 2024-09-16T09:26:34.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:34 smithi084 ceph-mon[32629]: 1.1 scrub ok 2024-09-16T09:26:34.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:34 smithi084 ceph-mon[32629]: 5.1 scrub starts 2024-09-16T09:26:34.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:34 smithi084 ceph-mon[32629]: 5.1 scrub ok 2024-09-16T09:26:34.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:34 smithi084 ceph-mon[32629]: 4.1c scrub starts 2024-09-16T09:26:34.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:34 smithi084 ceph-mon[32629]: 4.1c scrub ok 2024-09-16T09:26:34.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:34 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "11"}]': finished 2024-09-16T09:26:34.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:34 smithi084 ceph-mon[32629]: osdmap e94: 8 total, 8 up, 8 in 2024-09-16T09:26:34.740 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:26:34.783 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:34.783 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:34.784 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:26:34.884 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:26:34.914 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T09:26:34.923 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:34.923 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:34.924 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:26:35.008 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 22 16 47101 47085 133.728 89.25 0.00527762 0.00745258 2024-09-16T09:26:35.025 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:26:35.045 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:35.045 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:35.046 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:26:35.146 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:26:35.174 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:35.174 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:35.174 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:26:35.275 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:26:35.312 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:35.313 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:35.313 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:26:35.321 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T09:26:35.321 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:26:35.414 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:26:35.444 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:35.445 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:35.445 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:26:35.546 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:26:35.576 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:35.577 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:35.577 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:26:35.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:35 smithi084 ceph-mon[32629]: pgmap v54: 158 pgs: 4 peering, 10 unknown, 144 active+clean; 2.6 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 852 B/s rd, 115 MiB/s wr, 1.84k op/s 2024-09-16T09:26:35.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:35 smithi084 ceph-mon[32629]: 5.f scrub starts 2024-09-16T09:26:35.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:35 smithi084 ceph-mon[32629]: 5.f scrub ok 2024-09-16T09:26:35.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:35 smithi084 ceph-mon[32629]: 5.12 scrub starts 2024-09-16T09:26:35.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:35 smithi084 ceph-mon[32629]: 5.12 scrub ok 2024-09-16T09:26:35.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:35 smithi084 ceph-mon[32629]: 1.0 scrub starts 2024-09-16T09:26:35.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:35 smithi084 ceph-mon[32629]: 1.0 scrub ok 2024-09-16T09:26:35.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:35 smithi084 ceph-mon[32629]: osdmap e95: 8 total, 8 up, 8 in 2024-09-16T09:26:35.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:35 smithi070 ceph-mon[30388]: pgmap v54: 158 pgs: 4 peering, 10 unknown, 144 active+clean; 2.6 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 852 B/s rd, 115 MiB/s wr, 1.84k op/s 2024-09-16T09:26:35.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:35 smithi070 ceph-mon[30388]: 5.f scrub starts 2024-09-16T09:26:35.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:35 smithi070 ceph-mon[30388]: 5.f scrub ok 2024-09-16T09:26:35.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:35 smithi070 ceph-mon[30388]: 5.12 scrub starts 2024-09-16T09:26:35.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:35 smithi070 ceph-mon[30388]: 5.12 scrub ok 2024-09-16T09:26:35.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:35 smithi070 ceph-mon[30388]: 1.0 scrub starts 2024-09-16T09:26:35.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:35 smithi070 ceph-mon[30388]: 1.0 scrub ok 2024-09-16T09:26:35.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:35 smithi070 ceph-mon[30388]: osdmap e95: 8 total, 8 up, 8 in 2024-09-16T09:26:35.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:35 smithi070 ceph-mon[40691]: pgmap v54: 158 pgs: 4 peering, 10 unknown, 144 active+clean; 2.6 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 852 B/s rd, 115 MiB/s wr, 1.84k op/s 2024-09-16T09:26:35.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:35 smithi070 ceph-mon[40691]: 5.f scrub starts 2024-09-16T09:26:35.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:35 smithi070 ceph-mon[40691]: 5.f scrub ok 2024-09-16T09:26:35.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:35 smithi070 ceph-mon[40691]: 5.12 scrub starts 2024-09-16T09:26:35.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:35 smithi070 ceph-mon[40691]: 5.12 scrub ok 2024-09-16T09:26:35.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:35 smithi070 ceph-mon[40691]: 1.0 scrub starts 2024-09-16T09:26:35.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:35 smithi070 ceph-mon[40691]: 1.0 scrub ok 2024-09-16T09:26:35.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:35 smithi070 ceph-mon[40691]: osdmap e95: 8 total, 8 up, 8 in 2024-09-16T09:26:35.678 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:26:35.711 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:35.711 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:35.712 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:26:35.813 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:26:35.839 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:35.840 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:35.840 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:26:35.941 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:26:35.968 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:35.969 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:35.970 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:26:36.008 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 23 16 48466 48450 131.623 85.3125 0.00745256 0.00759193 2024-09-16T09:26:36.071 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:26:36.107 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:36.107 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:36.108 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:26:36.164 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd set noscrub 2024-09-16T09:26:36.209 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:26:36.235 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:36.236 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:36.236 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:26:36.337 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:26:36.364 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:36.364 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:36.365 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:26:36.466 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:26:36.508 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:36.508 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:36.509 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:26:36.610 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:26:36.635 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:36.635 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:36.636 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:26:36.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:36 smithi084 ceph-mon[32629]: 1.1 scrub starts 2024-09-16T09:26:36.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:36 smithi084 ceph-mon[32629]: 1.1 scrub ok 2024-09-16T09:26:36.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:36 smithi084 ceph-mon[32629]: 1.6 scrub starts 2024-09-16T09:26:36.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:36 smithi084 ceph-mon[32629]: 1.6 scrub ok 2024-09-16T09:26:36.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:36 smithi084 ceph-mon[32629]: 1.3 scrub starts 2024-09-16T09:26:36.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:36 smithi084 ceph-mon[32629]: 1.3 scrub ok 2024-09-16T09:26:36.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:36 smithi084 ceph-mon[32629]: 1.4 scrub starts 2024-09-16T09:26:36.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:36 smithi084 ceph-mon[32629]: 1.4 scrub ok 2024-09-16T09:26:36.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:36 smithi084 ceph-mon[32629]: pgmap v57: 158 pgs: 4 peering, 10 unknown, 144 active+clean; 2.6 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 1.2 KiB/s rd, 138 MiB/s wr, 2.20k op/s 2024-09-16T09:26:36.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:36 smithi084 ceph-mon[32629]: 6.9 scrub starts 2024-09-16T09:26:36.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:36 smithi084 ceph-mon[32629]: 6.9 scrub ok 2024-09-16T09:26:36.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:36 smithi084 ceph-mon[32629]: 1.5 scrub starts 2024-09-16T09:26:36.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:36 smithi084 ceph-mon[32629]: 1.5 scrub ok 2024-09-16T09:26:36.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:36 smithi084 ceph-mon[32629]: 1.2 scrub starts 2024-09-16T09:26:36.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:36 smithi084 ceph-mon[32629]: 1.2 scrub ok 2024-09-16T09:26:36.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:36 smithi084 ceph-mon[32629]: 6.14 scrub starts 2024-09-16T09:26:36.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:36 smithi084 ceph-mon[32629]: 6.14 scrub ok 2024-09-16T09:26:36.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:36 smithi070 ceph-mon[30388]: 1.1 scrub starts 2024-09-16T09:26:36.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:36 smithi070 ceph-mon[30388]: 1.1 scrub ok 2024-09-16T09:26:36.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:36 smithi070 ceph-mon[30388]: 1.6 scrub starts 2024-09-16T09:26:36.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:36 smithi070 ceph-mon[30388]: 1.6 scrub ok 2024-09-16T09:26:36.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:36 smithi070 ceph-mon[30388]: 1.3 scrub starts 2024-09-16T09:26:36.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:36 smithi070 ceph-mon[30388]: 1.3 scrub ok 2024-09-16T09:26:36.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:36 smithi070 ceph-mon[30388]: 1.4 scrub starts 2024-09-16T09:26:36.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:36 smithi070 ceph-mon[30388]: 1.4 scrub ok 2024-09-16T09:26:36.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:36 smithi070 ceph-mon[30388]: pgmap v57: 158 pgs: 4 peering, 10 unknown, 144 active+clean; 2.6 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 1.2 KiB/s rd, 138 MiB/s wr, 2.20k op/s 2024-09-16T09:26:36.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:36 smithi070 ceph-mon[30388]: 6.9 scrub starts 2024-09-16T09:26:36.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:36 smithi070 ceph-mon[30388]: 6.9 scrub ok 2024-09-16T09:26:36.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:36 smithi070 ceph-mon[30388]: 1.5 scrub starts 2024-09-16T09:26:36.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:36 smithi070 ceph-mon[30388]: 1.5 scrub ok 2024-09-16T09:26:36.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:36 smithi070 ceph-mon[30388]: 1.2 scrub starts 2024-09-16T09:26:36.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:36 smithi070 ceph-mon[30388]: 1.2 scrub ok 2024-09-16T09:26:36.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:36 smithi070 ceph-mon[30388]: 6.14 scrub starts 2024-09-16T09:26:36.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:36 smithi070 ceph-mon[30388]: 6.14 scrub ok 2024-09-16T09:26:36.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:36 smithi070 ceph-mon[40691]: 1.1 scrub starts 2024-09-16T09:26:36.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:36 smithi070 ceph-mon[40691]: 1.1 scrub ok 2024-09-16T09:26:36.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:36 smithi070 ceph-mon[40691]: 1.6 scrub starts 2024-09-16T09:26:36.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:36 smithi070 ceph-mon[40691]: 1.6 scrub ok 2024-09-16T09:26:36.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:36 smithi070 ceph-mon[40691]: 1.3 scrub starts 2024-09-16T09:26:36.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:36 smithi070 ceph-mon[40691]: 1.3 scrub ok 2024-09-16T09:26:36.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:36 smithi070 ceph-mon[40691]: 1.4 scrub starts 2024-09-16T09:26:36.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:36 smithi070 ceph-mon[40691]: 1.4 scrub ok 2024-09-16T09:26:36.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:36 smithi070 ceph-mon[40691]: pgmap v57: 158 pgs: 4 peering, 10 unknown, 144 active+clean; 2.6 GiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 1.2 KiB/s rd, 138 MiB/s wr, 2.20k op/s 2024-09-16T09:26:36.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:36 smithi070 ceph-mon[40691]: 6.9 scrub starts 2024-09-16T09:26:36.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:36 smithi070 ceph-mon[40691]: 6.9 scrub ok 2024-09-16T09:26:36.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:36 smithi070 ceph-mon[40691]: 1.5 scrub starts 2024-09-16T09:26:36.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:36 smithi070 ceph-mon[40691]: 1.5 scrub ok 2024-09-16T09:26:36.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:36 smithi070 ceph-mon[40691]: 1.2 scrub starts 2024-09-16T09:26:36.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:36 smithi070 ceph-mon[40691]: 1.2 scrub ok 2024-09-16T09:26:36.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:36 smithi070 ceph-mon[40691]: 6.14 scrub starts 2024-09-16T09:26:36.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:36 smithi070 ceph-mon[40691]: 6.14 scrub ok 2024-09-16T09:26:36.737 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:26:36.773 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:36.774 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:36.774 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:26:36.875 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:26:36.904 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:36.905 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:36.905 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:26:37.006 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:26:37.009 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 24 16 50029 50013 130.209 97.6875 0.00912164 0.00767458 2024-09-16T09:26:37.041 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:37.041 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:37.041 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:26:37.142 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:26:37.176 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:37.177 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:37.177 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:26:37.278 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:26:37.324 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:37.325 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:37.325 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:26:37.426 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:26:37.457 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:37.458 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:37.458 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:26:37.559 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:26:37.595 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:37.596 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:37.596 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:26:37.609 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:37 smithi070 ceph-mon[30388]: 1.a scrub starts 2024-09-16T09:26:37.609 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:37 smithi070 ceph-mon[30388]: 1.a scrub ok 2024-09-16T09:26:37.609 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:37 smithi070 ceph-mon[30388]: 1.7 scrub starts 2024-09-16T09:26:37.609 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:37 smithi070 ceph-mon[30388]: 1.7 scrub ok 2024-09-16T09:26:37.609 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:37 smithi070 ceph-mon[30388]: 5.c scrub starts 2024-09-16T09:26:37.609 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:37 smithi070 ceph-mon[30388]: 5.c scrub ok 2024-09-16T09:26:37.609 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:37 smithi070 ceph-mon[30388]: 1.9 scrub starts 2024-09-16T09:26:37.610 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:37 smithi070 ceph-mon[30388]: 1.9 scrub ok 2024-09-16T09:26:37.610 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:37 smithi070 ceph-mon[40691]: 1.a scrub starts 2024-09-16T09:26:37.610 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:37 smithi070 ceph-mon[40691]: 1.a scrub ok 2024-09-16T09:26:37.610 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:37 smithi070 ceph-mon[40691]: 1.7 scrub starts 2024-09-16T09:26:37.611 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:37 smithi070 ceph-mon[40691]: 1.7 scrub ok 2024-09-16T09:26:37.611 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:37 smithi070 ceph-mon[40691]: 5.c scrub starts 2024-09-16T09:26:37.611 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:37 smithi070 ceph-mon[40691]: 5.c scrub ok 2024-09-16T09:26:37.612 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:37 smithi070 ceph-mon[40691]: 1.9 scrub starts 2024-09-16T09:26:37.612 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:37 smithi070 ceph-mon[40691]: 1.9 scrub ok 2024-09-16T09:26:37.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:37 smithi084 ceph-mon[32629]: 1.a scrub starts 2024-09-16T09:26:37.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:37 smithi084 ceph-mon[32629]: 1.a scrub ok 2024-09-16T09:26:37.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:37 smithi084 ceph-mon[32629]: 1.7 scrub starts 2024-09-16T09:26:37.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:37 smithi084 ceph-mon[32629]: 1.7 scrub ok 2024-09-16T09:26:37.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:37 smithi084 ceph-mon[32629]: 5.c scrub starts 2024-09-16T09:26:37.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:37 smithi084 ceph-mon[32629]: 5.c scrub ok 2024-09-16T09:26:37.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:37 smithi084 ceph-mon[32629]: 1.9 scrub starts 2024-09-16T09:26:37.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:37 smithi084 ceph-mon[32629]: 1.9 scrub ok 2024-09-16T09:26:37.697 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:26:37.731 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:37.731 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:37.732 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:26:37.833 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:26:37.869 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:37.870 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:37.870 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:26:37.971 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:26:38.009 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 25 16 51482 51466 128.633 90.8125 0.0116447 0.00776871 2024-09-16T09:26:38.011 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:38.012 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:38.013 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:26:38.113 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:26:38.151 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:38.151 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:38.151 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:26:38.227 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T09:26:38.227 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T09:26:38.227 INFO:tasks.thrashosds.thrasher:inject_pause on osd.7 2024-09-16T09:26:38.227 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-16T09:26:38.228 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-16T09:26:38.228 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.7 config set filestore_inject_stall 3 2024-09-16T09:26:38.252 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:26:38.274 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:38.275 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:38.275 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:26:38.376 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:26:38.406 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:38.407 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:38.407 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:26:38.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:38 smithi070 ceph-mon[30388]: 1.8 scrub starts 2024-09-16T09:26:38.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:38 smithi070 ceph-mon[30388]: 1.8 scrub ok 2024-09-16T09:26:38.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:38 smithi070 ceph-mon[30388]: 6.1 scrub starts 2024-09-16T09:26:38.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:38 smithi070 ceph-mon[30388]: 6.1 scrub ok 2024-09-16T09:26:38.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:38 smithi070 ceph-mon[30388]: pgmap v58: 158 pgs: 1 unknown, 157 active+clean; 2.9 GiB data, 9.0 GiB used, 706 GiB / 715 GiB avail; 157 MiB/s wr, 2.49k op/s; 0 B/s, 2 objects/s recovering 2024-09-16T09:26:38.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:38 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:26:38.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:38 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:26:38.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:38 smithi070 ceph-mon[40691]: 1.8 scrub starts 2024-09-16T09:26:38.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:38 smithi070 ceph-mon[40691]: 1.8 scrub ok 2024-09-16T09:26:38.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:38 smithi070 ceph-mon[40691]: 6.1 scrub starts 2024-09-16T09:26:38.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:38 smithi070 ceph-mon[40691]: 6.1 scrub ok 2024-09-16T09:26:38.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:38 smithi070 ceph-mon[40691]: pgmap v58: 158 pgs: 1 unknown, 157 active+clean; 2.9 GiB data, 9.0 GiB used, 706 GiB / 715 GiB avail; 157 MiB/s wr, 2.49k op/s; 0 B/s, 2 objects/s recovering 2024-09-16T09:26:38.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:38 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:26:38.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:38 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:26:38.508 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:26:38.542 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:38.542 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:26:38.543 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:38.543 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:26:38.644 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:26:38.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:38 smithi084 ceph-mon[32629]: 1.8 scrub starts 2024-09-16T09:26:38.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:38 smithi084 ceph-mon[32629]: 1.8 scrub ok 2024-09-16T09:26:38.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:38 smithi084 ceph-mon[32629]: 6.1 scrub starts 2024-09-16T09:26:38.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:38 smithi084 ceph-mon[32629]: 6.1 scrub ok 2024-09-16T09:26:38.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:38 smithi084 ceph-mon[32629]: pgmap v58: 158 pgs: 1 unknown, 157 active+clean; 2.9 GiB data, 9.0 GiB used, 706 GiB / 715 GiB avail; 157 MiB/s wr, 2.49k op/s; 0 B/s, 2 objects/s recovering 2024-09-16T09:26:38.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:38 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:26:38.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:38 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:26:38.672 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:38.673 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:38.673 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:26:38.774 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:26:38.819 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:38.819 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:38.820 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:26:38.921 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:26:38.953 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:38.953 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:38.954 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:26:39.009 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 26 16 52694 52678 126.599 75.75 0.00965422 0.00789444 2024-09-16T09:26:39.056 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:26:39.086 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:39.086 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:39.087 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:26:39.127 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-09-16T09:26:39.127 INFO:teuthology.orchestra.run.smithi084.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-16T09:26:39.127 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-09-16T09:26:39.187 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:26:39.216 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.2 dump_ops_in_flight 2024-09-16T09:26:39.235 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:39.236 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:39.236 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:26:39.289 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:26:39.294 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:26:39.337 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:26:39.414 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:39.415 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:39.415 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:26:39.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:39 smithi070 ceph-mon[30388]: 6.d scrub starts 2024-09-16T09:26:39.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:39 smithi070 ceph-mon[30388]: 6.d scrub ok 2024-09-16T09:26:39.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:39 smithi070 ceph-mon[30388]: osdmap e96: 8 total, 8 up, 8 in 2024-09-16T09:26:39.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:39 smithi070 ceph-mon[30388]: 4.1 scrub starts 2024-09-16T09:26:39.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:39 smithi070 ceph-mon[30388]: 4.1 scrub ok 2024-09-16T09:26:39.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:39 smithi070 ceph-mon[30388]: 2.0 scrub starts 2024-09-16T09:26:39.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:39 smithi070 ceph-mon[30388]: 2.0 scrub ok 2024-09-16T09:26:39.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:39 smithi070 ceph-mon[40691]: 6.d scrub starts 2024-09-16T09:26:39.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:39 smithi070 ceph-mon[40691]: 6.d scrub ok 2024-09-16T09:26:39.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:39 smithi070 ceph-mon[40691]: osdmap e96: 8 total, 8 up, 8 in 2024-09-16T09:26:39.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:39 smithi070 ceph-mon[40691]: 4.1 scrub starts 2024-09-16T09:26:39.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:39 smithi070 ceph-mon[40691]: 4.1 scrub ok 2024-09-16T09:26:39.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:39 smithi070 ceph-mon[40691]: 2.0 scrub starts 2024-09-16T09:26:39.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:39 smithi070 ceph-mon[40691]: 2.0 scrub ok 2024-09-16T09:26:39.516 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:26:39.583 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:39.584 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:39.584 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:26:39.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:39 smithi084 ceph-mon[32629]: 6.d scrub starts 2024-09-16T09:26:39.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:39 smithi084 ceph-mon[32629]: 6.d scrub ok 2024-09-16T09:26:39.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:39 smithi084 ceph-mon[32629]: osdmap e96: 8 total, 8 up, 8 in 2024-09-16T09:26:39.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:39 smithi084 ceph-mon[32629]: 4.1 scrub starts 2024-09-16T09:26:39.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:39 smithi084 ceph-mon[32629]: 4.1 scrub ok 2024-09-16T09:26:39.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:39 smithi084 ceph-mon[32629]: 2.0 scrub starts 2024-09-16T09:26:39.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:39 smithi084 ceph-mon[32629]: 2.0 scrub ok 2024-09-16T09:26:39.685 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:26:39.737 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:39.738 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:39.739 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:26:39.839 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:26:39.878 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:39.879 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:39.879 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:26:39.980 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:26:40.009 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 27 16 53960 53944 124.84 79.125 0.00704155 0.00800474 2024-09-16T09:26:40.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:40.022 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:40.022 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:26:40.123 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:26:40.172 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:40.172 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:40.173 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:26:40.274 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:26:40.302 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:40.302 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:40.303 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:26:40.404 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:26:40.450 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:40.451 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:40.451 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:26:40.490 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:26:40.490 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:26:40.490 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:26:40.490 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:26:40.490 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:26:40.490 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:26:40.490 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:26:40.490 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:26:40.490 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:26:40.491 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:26:40.491 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:26:40.491 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:26:40.491 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:26:40.491 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:26:40.509 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:26:40.509 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:26:40.540 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:40 smithi070 ceph-mon[30388]: 6.1f scrub starts 2024-09-16T09:26:40.540 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:40 smithi070 ceph-mon[30388]: 6.1f scrub ok 2024-09-16T09:26:40.540 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:40 smithi070 ceph-mon[30388]: pgmap v60: 158 pgs: 158 active+clean; 3.1 GiB data, 9.5 GiB used, 706 GiB / 715 GiB avail; 81 MiB/s wr, 1.28k op/s; 0 B/s, 2 objects/s recovering 2024-09-16T09:26:40.540 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:40 smithi070 ceph-mon[30388]: 4.16 scrub starts 2024-09-16T09:26:40.540 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:40 smithi070 ceph-mon[30388]: 4.16 scrub ok 2024-09-16T09:26:40.540 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:40 smithi070 ceph-mon[30388]: 5.4 scrub starts 2024-09-16T09:26:40.540 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:40 smithi070 ceph-mon[30388]: 5.4 scrub ok 2024-09-16T09:26:40.540 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:40 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2940022856' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:26:40.541 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:40 smithi070 ceph-mon[40691]: 6.1f scrub starts 2024-09-16T09:26:40.541 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:40 smithi070 ceph-mon[40691]: 6.1f scrub ok 2024-09-16T09:26:40.541 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:40 smithi070 ceph-mon[40691]: pgmap v60: 158 pgs: 158 active+clean; 3.1 GiB data, 9.5 GiB used, 706 GiB / 715 GiB avail; 81 MiB/s wr, 1.28k op/s; 0 B/s, 2 objects/s recovering 2024-09-16T09:26:40.541 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:40 smithi070 ceph-mon[40691]: 4.16 scrub starts 2024-09-16T09:26:40.541 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:40 smithi070 ceph-mon[40691]: 4.16 scrub ok 2024-09-16T09:26:40.541 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:40 smithi070 ceph-mon[40691]: 5.4 scrub starts 2024-09-16T09:26:40.541 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:40 smithi070 ceph-mon[40691]: 5.4 scrub ok 2024-09-16T09:26:40.541 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:40 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/2940022856' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:26:40.552 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:26:40.578 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:40.578 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:40.578 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:26:40.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:40 smithi084 ceph-mon[32629]: 6.1f scrub starts 2024-09-16T09:26:40.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:40 smithi084 ceph-mon[32629]: 6.1f scrub ok 2024-09-16T09:26:40.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:40 smithi084 ceph-mon[32629]: pgmap v60: 158 pgs: 158 active+clean; 3.1 GiB data, 9.5 GiB used, 706 GiB / 715 GiB avail; 81 MiB/s wr, 1.28k op/s; 0 B/s, 2 objects/s recovering 2024-09-16T09:26:40.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:40 smithi084 ceph-mon[32629]: 4.16 scrub starts 2024-09-16T09:26:40.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:40 smithi084 ceph-mon[32629]: 4.16 scrub ok 2024-09-16T09:26:40.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:40 smithi084 ceph-mon[32629]: 5.4 scrub starts 2024-09-16T09:26:40.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:40 smithi084 ceph-mon[32629]: 5.4 scrub ok 2024-09-16T09:26:40.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:40 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/2940022856' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:26:40.679 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:26:40.708 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:40.708 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:40.709 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:26:40.810 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:26:40.855 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:40.855 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:40.856 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:26:40.957 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:26:40.985 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:40.986 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:40.986 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:26:41.009 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 28 16 55371 55355 123.531 88.1875 0.00529014 0.00808969 2024-09-16T09:26:41.086 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:26:41.127 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:41.128 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:41.129 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:26:41.229 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:26:41.257 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:41.258 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:41.258 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:26:41.359 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:26:41.386 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:41.387 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:41.387 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:26:41.488 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:26:41.525 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:41.526 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:41.526 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:26:41.627 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:26:41.656 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:41.657 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:41.657 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:26:41.659 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:41 smithi084 ceph-mon[32629]: 3.8 scrub starts 2024-09-16T09:26:41.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:41 smithi084 ceph-mon[32629]: 3.8 scrub ok 2024-09-16T09:26:41.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:41 smithi084 ceph-mon[32629]: 3.14 scrub starts 2024-09-16T09:26:41.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:41 smithi084 ceph-mon[32629]: 3.14 scrub ok 2024-09-16T09:26:41.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:41 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/2385900721' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:26:41.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:41 smithi084 ceph-mon[32629]: 6.3 scrub starts 2024-09-16T09:26:41.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:41 smithi084 ceph-mon[32629]: 6.3 scrub ok 2024-09-16T09:26:41.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:41 smithi070 ceph-mon[30388]: 3.8 scrub starts 2024-09-16T09:26:41.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:41 smithi070 ceph-mon[30388]: 3.8 scrub ok 2024-09-16T09:26:41.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:41 smithi070 ceph-mon[30388]: 3.14 scrub starts 2024-09-16T09:26:41.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:41 smithi070 ceph-mon[30388]: 3.14 scrub ok 2024-09-16T09:26:41.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:41 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2385900721' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:26:41.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:41 smithi070 ceph-mon[30388]: 6.3 scrub starts 2024-09-16T09:26:41.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:41 smithi070 ceph-mon[30388]: 6.3 scrub ok 2024-09-16T09:26:41.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:41 smithi070 ceph-mon[40691]: 3.8 scrub starts 2024-09-16T09:26:41.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:41 smithi070 ceph-mon[40691]: 3.8 scrub ok 2024-09-16T09:26:41.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:41 smithi070 ceph-mon[40691]: 3.14 scrub starts 2024-09-16T09:26:41.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:41 smithi070 ceph-mon[40691]: 3.14 scrub ok 2024-09-16T09:26:41.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:41 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/2385900721' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:26:41.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:41 smithi070 ceph-mon[40691]: 6.3 scrub starts 2024-09-16T09:26:41.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:41 smithi070 ceph-mon[40691]: 6.3 scrub ok 2024-09-16T09:26:41.759 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:26:41.793 INFO:teuthology.orchestra.run.smithi070.stderr:noscrub is set 2024-09-16T09:26:41.796 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:41.797 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:41.797 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:26:41.898 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:26:41.932 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:41.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:41.934 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:26:42.010 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 29 16 56906 56890 122.574 95.9375 0.00551037 0.00815241 2024-09-16T09:26:42.031 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:26:42.034 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:26:42.090 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:42.091 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:42.091 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:26:42.192 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:26:42.198 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:26:41 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:26:41] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:26:42.220 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:42.221 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:42.221 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:26:42.322 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:26:42.366 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:42.367 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:42.368 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:26:42.468 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:26:42.508 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:42.509 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:42.509 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:26:42.609 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:42 smithi070 ceph-mon[30388]: 5.19 scrub starts 2024-09-16T09:26:42.610 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:42 smithi070 ceph-mon[30388]: 5.19 scrub ok 2024-09-16T09:26:42.610 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:42 smithi070 ceph-mon[30388]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:26:42.610 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:42 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2385900721' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-16T09:26:42.610 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:42 smithi070 ceph-mon[30388]: osdmap e97: 8 total, 8 up, 8 in 2024-09-16T09:26:42.610 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:42 smithi070 ceph-mon[30388]: pgmap v62: 158 pgs: 158 active+clean; 3.1 GiB data, 9.5 GiB used, 706 GiB / 715 GiB avail; 76 MiB/s wr, 1.20k op/s; 0 B/s, 2 objects/s recovering 2024-09-16T09:26:42.610 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:42 smithi070 ceph-mon[40691]: 5.19 scrub starts 2024-09-16T09:26:42.610 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:42 smithi070 ceph-mon[40691]: 5.19 scrub ok 2024-09-16T09:26:42.610 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:42 smithi070 ceph-mon[40691]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:26:42.610 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:42 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/2385900721' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-16T09:26:42.610 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:42 smithi070 ceph-mon[40691]: osdmap e97: 8 total, 8 up, 8 in 2024-09-16T09:26:42.611 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:42 smithi070 ceph-mon[40691]: pgmap v62: 158 pgs: 158 active+clean; 3.1 GiB data, 9.5 GiB used, 706 GiB / 715 GiB avail; 76 MiB/s wr, 1.20k op/s; 0 B/s, 2 objects/s recovering 2024-09-16T09:26:42.611 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:26:42.660 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:26:42 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:26:42.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:42 smithi084 ceph-mon[32629]: 5.19 scrub starts 2024-09-16T09:26:42.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:42 smithi084 ceph-mon[32629]: 5.19 scrub ok 2024-09-16T09:26:42.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:42 smithi084 ceph-mon[32629]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:26:42.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:42 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/2385900721' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-16T09:26:42.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:42 smithi084 ceph-mon[32629]: osdmap e97: 8 total, 8 up, 8 in 2024-09-16T09:26:42.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:42 smithi084 ceph-mon[32629]: pgmap v62: 158 pgs: 158 active+clean; 3.1 GiB data, 9.5 GiB used, 706 GiB / 715 GiB avail; 76 MiB/s wr, 1.20k op/s; 0 B/s, 2 objects/s recovering 2024-09-16T09:26:42.665 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:42.666 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:42.666 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:26:42.767 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:26:42.810 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:42.810 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:42.811 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:26:42.911 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:26:42.942 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:42.943 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:42.943 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:26:43.011 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 30 16 58431 58415 121.665 95.3125 0.0158416 0.0082144 2024-09-16T09:26:43.043 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:26:43.078 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:43.078 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:43.079 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:26:43.179 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:26:43.209 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:43.210 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:43.210 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:26:43.311 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:26:43.357 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:43.357 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:43.357 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:26:43.427 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T09:26:43.427 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [ 2024-09-16T09:26:43.428 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:26:43.428 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.24698.0:58950 7.5 e97/82 7:af3b505f:::benchmark_data_smithi070_78438_object58949:head v 97'3793, mlcod=97'3791)", 2024-09-16T09:26:43.428 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:26:43.380671+0000", 2024-09-16T09:26:43.428 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.0055634880000000001, 2024-09-16T09:26:43.428 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0055943549999999996, 2024-09-16T09:26:43.428 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:26:43.428 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:26:43.428 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T09:26:43.428 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:26:43.428 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:26:43.428 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:26:43.428 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:43.380671+0000", 2024-09-16T09:26:43.429 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:26:43.429 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:26:43.429 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:26:43.429 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:26:43.429 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:43.380671+0000", 2024-09-16T09:26:43.429 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:26:43.429 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:26:43.429 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:26:43.429 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:26:43.429 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:43.380672+0000", 2024-09-16T09:26:43.429 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.113e-06 2024-09-16T09:26:43.429 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:26:43.429 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:26:43.430 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:26:43.430 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:43.380707+0000", 2024-09-16T09:26:43.430 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4712000000000002e-05 2024-09-16T09:26:43.430 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:26:43.430 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:26:43.430 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:26:43.430 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:43.380716+0000", 2024-09-16T09:26:43.430 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.6030000000000005e-06 2024-09-16T09:26:43.430 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:26:43.430 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:26:43.430 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:26:43.430 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:43.380721+0000", 2024-09-16T09:26:43.430 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.1250000000000001e-06 2024-09-16T09:26:43.431 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:26:43.431 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:26:43.431 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:26:43.431 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:43.383242+0000", 2024-09-16T09:26:43.431 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0025212020000000002 2024-09-16T09:26:43.431 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:26:43.431 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:26:43.431 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:26:43.431 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:43.383266+0000", 2024-09-16T09:26:43.431 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3600000000000001e-05 2024-09-16T09:26:43.431 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:26:43.431 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:26:43.431 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:26:43.432 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:26:43.432 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:26:43.432 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.24698.0:58949 7.a e97/82 7:5e646eb6:::benchmark_data_smithi070_78438_object58948:head v 97'3658, mlcod=97'3655)", 2024-09-16T09:26:43.432 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:26:43.380801+0000", 2024-09-16T09:26:43.432 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.0054339649999999998, 2024-09-16T09:26:43.432 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0055266730000000002, 2024-09-16T09:26:43.432 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:26:43.432 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:26:43.432 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T09:26:43.432 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:26:43.432 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:26:43.432 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:26:43.432 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:43.380801+0000", 2024-09-16T09:26:43.433 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:26:43.433 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:26:43.433 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:26:43.433 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:26:43.433 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:43.380801+0000", 2024-09-16T09:26:43.433 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:26:43.433 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:26:43.433 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:26:43.433 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:26:43.433 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:43.380801+0000", 2024-09-16T09:26:43.433 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1699999999999999e-07 2024-09-16T09:26:43.433 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:26:43.433 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:26:43.434 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:26:43.434 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:43.380824+0000", 2024-09-16T09:26:43.434 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.2408999999999998e-05 2024-09-16T09:26:43.434 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:26:43.434 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:26:43.434 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:26:43.434 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:43.380834+0000", 2024-09-16T09:26:43.434 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0589e-05 2024-09-16T09:26:43.434 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:26:43.434 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:26:43.434 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:26:43.434 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:43.380839+0000", 2024-09-16T09:26:43.434 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.7809999999999996e-06 2024-09-16T09:26:43.435 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:26:43.435 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:26:43.435 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:26:43.435 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:43.381830+0000", 2024-09-16T09:26:43.435 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00099064300000000004 2024-09-16T09:26:43.435 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:26:43.435 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:26:43.435 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:26:43.435 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:43.381846+0000", 2024-09-16T09:26:43.435 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.662e-05 2024-09-16T09:26:43.435 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:26:43.435 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:26:43.435 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:26:43.436 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:26:43.436 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:26:43.436 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.24698.0:58954 7.5 e97/82 7:a5c725be:::benchmark_data_smithi070_78438_object58953:head v 97'3794, mlcod=97'3791)", 2024-09-16T09:26:43.436 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:26:43.382315+0000", 2024-09-16T09:26:43.436 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.0039199200000000003, 2024-09-16T09:26:43.437 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0040739729999999998, 2024-09-16T09:26:43.437 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:26:43.437 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:26:43.437 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T09:26:43.437 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:26:43.437 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:26:43.437 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:26:43.437 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:43.382315+0000", 2024-09-16T09:26:43.437 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:26:43.437 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:26:43.437 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:26:43.437 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:26:43.437 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:43.382315+0000", 2024-09-16T09:26:43.438 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:26:43.438 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:26:43.438 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:26:43.438 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:26:43.438 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:43.382316+0000", 2024-09-16T09:26:43.438 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.9500000000000002e-07 2024-09-16T09:26:43.438 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:26:43.438 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:26:43.438 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:26:43.438 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:43.382415+0000", 2024-09-16T09:26:43.438 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.8883999999999997e-05 2024-09-16T09:26:43.438 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:26:43.438 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:26:43.438 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:26:43.439 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:43.382423+0000", 2024-09-16T09:26:43.439 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.7590000000000003e-06 2024-09-16T09:26:43.439 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:26:43.439 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:26:43.439 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:26:43.439 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:43.382429+0000", 2024-09-16T09:26:43.439 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.0599999999999996e-06 2024-09-16T09:26:43.439 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:26:43.439 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:26:43.439 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:26:43.439 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:43.385136+0000", 2024-09-16T09:26:43.439 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00270689 2024-09-16T09:26:43.440 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:26:43.440 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:26:43.440 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:26:43.440 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:43.385151+0000", 2024-09-16T09:26:43.440 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.4942e-05 2024-09-16T09:26:43.440 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:26:43.440 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:26:43.440 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:26:43.440 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:26:43.440 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:26:43.440 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.24698.0:58956 7.5 e97/82)", 2024-09-16T09:26:43.440 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:26:43.383868+0000", 2024-09-16T09:26:43.440 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.002367063, 2024-09-16T09:26:43.441 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0025683730000000001, 2024-09-16T09:26:43.441 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:26:43.441 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:26:43.441 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T09:26:43.441 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:26:43.441 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:26:43.441 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:26:43.441 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:43.383868+0000", 2024-09-16T09:26:43.441 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:26:43.441 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:26:43.441 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:26:43.441 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:26:43.441 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:43.383868+0000", 2024-09-16T09:26:43.441 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:26:43.442 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:26:43.442 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:26:43.442 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:26:43.442 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:43.383869+0000", 2024-09-16T09:26:43.442 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.9800000000000003e-07 2024-09-16T09:26:43.442 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:26:43.442 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:26:43.442 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:26:43.442 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:43.383941+0000", 2024-09-16T09:26:43.442 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.2738999999999996e-05 2024-09-16T09:26:43.442 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:26:43.442 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:26:43.442 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:26:43.442 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:43.383955+0000", 2024-09-16T09:26:43.443 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3818000000000001e-05 2024-09-16T09:26:43.443 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:26:43.443 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:26:43.443 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:26:43.443 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:43.383961+0000", 2024-09-16T09:26:43.443 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.2500000000000003e-06 2024-09-16T09:26:43.443 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:26:43.443 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:26:43.443 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:26:43.443 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:26:43.443 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-16T09:26:43.443 INFO:teuthology.orchestra.run.smithi070.stdout: "num_ops": 4 2024-09-16T09:26:43.443 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T09:26:43.458 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:26:43.487 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:43.488 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:43.489 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:26:43.590 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:26:43.624 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:43.624 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:43.625 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:26:43.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:43 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:26:43.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:43 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:26:43.670 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:43 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:26:43.726 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:26:43.730 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T09:26:43.747 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:43.748 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:43.748 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:26:43.849 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:26:43.879 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:43.880 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:43.880 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:26:43.981 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:26:44.011 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 31 16 59756 59740 120.409 82.8125 0.00915713 0.00829954 2024-09-16T09:26:44.029 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:44.030 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:44.031 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:26:44.131 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:26:44.159 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:44.160 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:44.160 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:26:44.261 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:26:44.298 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:44.298 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:44.299 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:26:44.400 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:26:44.421 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:44.422 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:44.422 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:26:44.523 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:26:44.553 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:44.554 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:44.554 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:26:44.655 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:26:44.673 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-16T09:26:44.685 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:44.686 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:44.686 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:26:44.787 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:26:44.809 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T09:26:44.809 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T09:26:44.809 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-16T09:26:44.810 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph tell osd.0 'trim stale osdmaps' 2024-09-16T09:26:44.811 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:44.812 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:44.812 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:26:44.913 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:26:44.939 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:44.940 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:44.940 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:26:45.012 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 32 16 60916 60900 118.911 72.5 0.00896919 0.00840386 2024-09-16T09:26:45.040 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:26:45.062 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:45.062 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:45.062 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:26:45.118 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:45 smithi070 ceph-mon[30388]: pgmap v63: 158 pgs: 158 active+clean; 3.5 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 115 MiB/s wr, 1.82k op/s; 0 B/s, 1 objects/s recovering 2024-09-16T09:26:45.118 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:45 smithi070 ceph-mon[30388]: osdmap e98: 8 total, 8 up, 8 in 2024-09-16T09:26:45.163 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:26:45.192 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:45.193 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:45.193 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:26:45.294 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:26:45.326 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:45.327 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:45.327 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:26:45.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:45 smithi084 ceph-mon[32629]: pgmap v63: 158 pgs: 158 active+clean; 3.5 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 115 MiB/s wr, 1.82k op/s; 0 B/s, 1 objects/s recovering 2024-09-16T09:26:45.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:45 smithi084 ceph-mon[32629]: osdmap e98: 8 total, 8 up, 8 in 2024-09-16T09:26:45.420 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:45 smithi070 ceph-mon[40691]: pgmap v63: 158 pgs: 158 active+clean; 3.5 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 115 MiB/s wr, 1.82k op/s; 0 B/s, 1 objects/s recovering 2024-09-16T09:26:45.420 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:45 smithi070 ceph-mon[40691]: osdmap e98: 8 total, 8 up, 8 in 2024-09-16T09:26:45.428 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:26:45.456 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:45.457 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:45.457 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:26:45.558 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:26:45.589 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:45.589 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:45.590 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:26:45.691 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:26:45.721 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:45.722 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:45.723 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:26:45.823 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:26:45.877 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:45.877 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:45.878 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:26:45.978 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:26:46.011 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:46.012 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:46.013 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:26:46.014 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 33 16 62257 62241 117.84 83.8125 0.015457 0.00848117 2024-09-16T09:26:46.113 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:26:46.142 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:46.142 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:46.143 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:26:46.244 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:26:46.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:46.274 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:46.274 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:26:46.375 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:26:46.403 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:46.403 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:46.404 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:26:46.504 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:26:46.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:46.536 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:46.536 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:26:46.637 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:26:46.675 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:46.676 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:46.676 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:26:46.777 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:26:46.814 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:46.814 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:46.815 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:26:46.916 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:26:46.954 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:46.954 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:46.955 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:26:47.015 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 34 16 63595 63579 116.832 83.625 0.00911113 0.00855324 2024-09-16T09:26:47.055 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:26:47.139 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:47.140 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:47.140 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:26:47.241 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:26:47.270 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:47.271 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:47.271 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:26:47.372 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:26:47.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:47 smithi084 ceph-mon[32629]: pgmap v65: 158 pgs: 158 active+clean; 3.5 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 712 B/s rd, 60 MiB/s wr, 958 op/s 2024-09-16T09:26:47.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:47 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:26:47.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:47 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:26:47.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:47 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:26:47.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:47 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:26:47.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:47 smithi070 ceph-mon[30388]: pgmap v65: 158 pgs: 158 active+clean; 3.5 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 712 B/s rd, 60 MiB/s wr, 958 op/s 2024-09-16T09:26:47.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:47 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:26:47.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:47 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:26:47.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:47 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:26:47.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:47 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:26:47.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:47 smithi070 ceph-mon[40691]: pgmap v65: 158 pgs: 158 active+clean; 3.5 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 712 B/s rd, 60 MiB/s wr, 958 op/s 2024-09-16T09:26:47.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:47 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:26:47.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:47 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:26:47.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:47 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:26:47.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:47 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:26:47.422 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:47.422 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:47.422 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:26:47.523 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:26:47.565 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:47.565 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:47.565 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:26:47.666 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:26:47.679 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.2 dump_blocked_ops 2024-09-16T09:26:47.693 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:47.693 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:47.694 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:26:47.783 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:26:47.795 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:26:47.804 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:26:47.817 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:26:47.855 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:47.855 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:47.856 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:26:47.957 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:26:47.981 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:47.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:47.982 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:26:48.015 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 35 16 64738 64722 115.535 71.4375 0.0140748 0.00865004 2024-09-16T09:26:48.082 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:26:48.108 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:48.109 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:48.109 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:26:48.210 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:26:48.235 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:48.236 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:48.236 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:26:48.337 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:26:48.381 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:48.381 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:48.382 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:26:48.409 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:48 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:26:48.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:48 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:26:48.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:48 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:26:48.420 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:48 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:26:48.420 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:48 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:26:48.420 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:48 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:26:48.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:48 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:26:48.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:48 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:26:48.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:48 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:26:48.482 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:26:48.509 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:48.509 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:48.510 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:26:48.611 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:26:48.642 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:48.642 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:48.643 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:26:48.744 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:26:48.770 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:48.771 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:48.771 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:26:48.872 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:26:48.916 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:48.917 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:48.917 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:26:48.922 INFO:teuthology.orchestra.run.smithi070.stderr:no valid command found; 10 closest matches: 2024-09-16T09:26:48.922 INFO:teuthology.orchestra.run.smithi070.stderr:0 2024-09-16T09:26:48.922 INFO:teuthology.orchestra.run.smithi070.stderr:1 2024-09-16T09:26:48.922 INFO:teuthology.orchestra.run.smithi070.stderr:2 2024-09-16T09:26:48.922 INFO:teuthology.orchestra.run.smithi070.stderr:abort 2024-09-16T09:26:48.922 INFO:teuthology.orchestra.run.smithi070.stderr:assert 2024-09-16T09:26:48.922 INFO:teuthology.orchestra.run.smithi070.stderr:bench [] [] [] [] 2024-09-16T09:26:48.923 INFO:teuthology.orchestra.run.smithi070.stderr:bluefs debug_inject_read_zeros 2024-09-16T09:26:48.923 INFO:teuthology.orchestra.run.smithi070.stderr:bluefs files list 2024-09-16T09:26:48.923 INFO:teuthology.orchestra.run.smithi070.stderr:bluefs stats 2024-09-16T09:26:48.923 INFO:teuthology.orchestra.run.smithi070.stderr:bluestore allocator dump block 2024-09-16T09:26:48.923 INFO:teuthology.orchestra.run.smithi070.stderr:Error EINVAL: invalid command 2024-09-16T09:26:49.014 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 36 15 65992 65977 114.505 78.4375 0.00827547 0.00872834 2024-09-16T09:26:49.018 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:26:49.043 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:49.044 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:49.044 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:26:49.145 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:26:49.186 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:49.187 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:49.187 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:26:49.288 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:26:49.315 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:49.316 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:49.316 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:26:49.358 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:49 smithi070 ceph-mon[30388]: pgmap v66: 158 pgs: 158 active+clean; 3.7 GiB data, 12 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 82 MiB/s wr, 1.32k op/s 2024-09-16T09:26:49.358 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:49 smithi070 ceph-mon[30388]: Reconfiguring alertmanager.a (dependencies changed)... 2024-09-16T09:26:49.358 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:49 smithi070 ceph-mon[30388]: Reconfiguring daemon alertmanager.a on smithi070 2024-09-16T09:26:49.358 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:49 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:26:49.358 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:49 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:26:49.358 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:49 smithi070 ceph-mon[40691]: pgmap v66: 158 pgs: 158 active+clean; 3.7 GiB data, 12 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 82 MiB/s wr, 1.32k op/s 2024-09-16T09:26:49.358 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:49 smithi070 ceph-mon[40691]: Reconfiguring alertmanager.a (dependencies changed)... 2024-09-16T09:26:49.358 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:49 smithi070 ceph-mon[40691]: Reconfiguring daemon alertmanager.a on smithi070 2024-09-16T09:26:49.359 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:49 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:26:49.359 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:49 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:26:49.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:49 smithi084 ceph-mon[32629]: pgmap v66: 158 pgs: 158 active+clean; 3.7 GiB data, 12 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 82 MiB/s wr, 1.32k op/s 2024-09-16T09:26:49.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:49 smithi084 ceph-mon[32629]: Reconfiguring alertmanager.a (dependencies changed)... 2024-09-16T09:26:49.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:49 smithi084 ceph-mon[32629]: Reconfiguring daemon alertmanager.a on smithi070 2024-09-16T09:26:49.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:49 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:26:49.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:49 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:26:49.417 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:26:49.447 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:49.447 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:49.448 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:26:49.549 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:26:49.588 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:49.588 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:49.588 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:26:49.689 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:26:49.739 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:49.740 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:49.740 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:26:49.841 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:26:49.877 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-16T09:26:49.878 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-16T09:26:49.878 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph tell osd.1 'trim stale osdmaps' 2024-09-16T09:26:49.895 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:49.896 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:49.896 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:26:49.997 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:26:50.017 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 37 16 67262 67246 113.554 79.3125 0.00928148 0.00880141 2024-09-16T09:26:50.048 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:50.049 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:50.049 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:26:50.150 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:26:50.175 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:50.176 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:50.177 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:26:50.177 INFO:teuthology.orchestra.run.smithi070.stderr:nodeep-scrub is set 2024-09-16T09:26:50.278 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:26:50.304 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:50.305 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:50.305 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:26:50.406 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:26:50.452 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:50.453 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:50.453 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:26:50.554 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:26:50.598 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:50.599 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:50.599 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:26:50.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:50 smithi084 ceph-mon[32629]: osdmap e99: 8 total, 8 up, 8 in 2024-09-16T09:26:50.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:50 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/2726405447' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:26:50.665 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:50 smithi070 ceph-mon[30388]: osdmap e99: 8 total, 8 up, 8 in 2024-09-16T09:26:50.665 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:50 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2726405447' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:26:50.665 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:50 smithi070 ceph-mon[40691]: osdmap e99: 8 total, 8 up, 8 in 2024-09-16T09:26:50.665 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:50 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/2726405447' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:26:50.701 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:26:50.743 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:50.743 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:50.744 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:26:50.845 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:26:50.872 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:50.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:50.873 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:26:50.974 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:26:51.003 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:51.003 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:51.004 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:26:51.015 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 38 16 68594 68578 112.756 83.25 0.0112806 0.00886387 2024-09-16T09:26:51.104 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:26:51.147 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:51.148 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:51.148 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:26:51.249 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:26:51.295 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:51.296 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:51.296 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:26:51.397 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:26:51.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:51 smithi070 ceph-mon[30388]: pgmap v68: 158 pgs: 158 active+clean; 3.9 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1.2 KiB/s rd, 105 MiB/s wr, 1.68k op/s 2024-09-16T09:26:51.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:51 smithi070 ceph-mon[30388]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:26:51.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:51 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2726405447' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-16T09:26:51.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:51 smithi070 ceph-mon[30388]: osdmap e100: 8 total, 8 up, 8 in 2024-09-16T09:26:51.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:51 smithi070 ceph-mon[40691]: pgmap v68: 158 pgs: 158 active+clean; 3.9 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1.2 KiB/s rd, 105 MiB/s wr, 1.68k op/s 2024-09-16T09:26:51.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:51 smithi070 ceph-mon[40691]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:26:51.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:51 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/2726405447' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-16T09:26:51.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:51 smithi070 ceph-mon[40691]: osdmap e100: 8 total, 8 up, 8 in 2024-09-16T09:26:51.426 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:51.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:51.428 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:26:51.528 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:26:51.566 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:51.566 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:51.567 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:26:51.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:51 smithi084 ceph-mon[32629]: pgmap v68: 158 pgs: 158 active+clean; 3.9 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1.2 KiB/s rd, 105 MiB/s wr, 1.68k op/s 2024-09-16T09:26:51.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:51 smithi084 ceph-mon[32629]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:26:51.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:51 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/2726405447' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-16T09:26:51.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:51 smithi084 ceph-mon[32629]: osdmap e100: 8 total, 8 up, 8 in 2024-09-16T09:26:51.667 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:26:51.695 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:51.696 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:51.696 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:26:51.784 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:26:51.797 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:26:51.819 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:51.819 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:51.820 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:26:51.921 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:26:51.950 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:51.950 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:51.952 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:26:52.015 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 39 16 69956 69940 112.048 85.125 0.00652023 0.00892006 2024-09-16T09:26:52.021 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:26:51 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:26:51] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:26:52.052 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:26:52.105 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:52.105 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:52.106 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:26:52.120 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:26:52.120 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:26:52.154 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:26:52.155 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:26:52.187 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:26:52.187 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:26:52.207 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:26:52.234 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:26:52.234 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:26:52.253 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:52.253 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:52.254 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:26:52.268 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:26:52.269 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:26:52.302 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:26:52.302 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:26:52.302 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:52 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:26:52.302 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:52 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:26:52.303 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:52 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/922866560' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:26:52.303 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:52 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:26:52.303 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:52 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:26:52.303 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:52 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/922866560' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:26:52.338 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:26:52.338 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:26:52.354 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:26:52.370 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:26:52.371 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:26:52.382 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:52.383 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:52.384 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:26:52.484 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:26:52.514 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:52.515 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:52.515 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:26:52.571 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:52 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:26:52.571 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:52 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:26:52.571 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:52 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/922866560' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:26:52.616 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:26:52.645 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:52.646 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:52.646 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:26:52.747 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:26:52.781 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:52.783 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:52.784 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:26:52.885 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:26:52.909 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:26:52 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:26:52.944 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:52.945 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:52.946 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:26:53.015 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:2024-09-16T09:26:53.014928+0000 min lat: 0.0012412 max lat: 0.156312 avg lat: 0.00898662 2024-09-16T09:26:53.015 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-16T09:26:53.015 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 40 16 71220 71204 111.221 79 0.00645534 0.00898662 2024-09-16T09:26:53.047 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:26:53.074 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:53.075 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:53.075 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:26:53.125 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:26:53.176 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:26:53.223 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:53.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:53.224 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:26:53.325 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:26:53.355 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:53.356 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:53.357 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:26:53.457 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:26:53.493 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:53.493 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:53.494 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:26:53.559 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:53 smithi070 ceph-mon[30388]: pgmap v70: 158 pgs: 158 active+clean; 3.9 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 692 B/s rd, 55 MiB/s wr, 886 op/s 2024-09-16T09:26:53.559 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:53 smithi070 ceph-mon[30388]: osdmap e101: 8 total, 8 up, 8 in 2024-09-16T09:26:53.560 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:53 smithi070 ceph-mon[40691]: pgmap v70: 158 pgs: 158 active+clean; 3.9 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 692 B/s rd, 55 MiB/s wr, 886 op/s 2024-09-16T09:26:53.560 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:53 smithi070 ceph-mon[40691]: osdmap e101: 8 total, 8 up, 8 in 2024-09-16T09:26:53.595 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:26:53.640 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:53.640 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:53.641 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:26:53.659 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:53 smithi084 ceph-mon[32629]: pgmap v70: 158 pgs: 158 active+clean; 3.9 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 692 B/s rd, 55 MiB/s wr, 886 op/s 2024-09-16T09:26:53.659 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:53 smithi084 ceph-mon[32629]: osdmap e101: 8 total, 8 up, 8 in 2024-09-16T09:26:53.741 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:26:53.773 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:53.774 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:53.775 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:26:53.813 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd unset noscrub 2024-09-16T09:26:53.875 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:26:53.902 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:53.903 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:53.903 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:26:54.004 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:26:54.015 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 41 16 72504 72488 110.464 80.25 0.0201269 0.00904778 2024-09-16T09:26:54.050 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:54.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:54.052 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:26:54.153 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:26:54.159 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:26:53 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=infra.usagestats t=2024-09-16T09:26:53.839910363Z level=info msg="Usage stats are ready to report" 2024-09-16T09:26:54.181 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:54.181 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:54.182 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:26:54.283 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:26:54.336 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:54.337 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:54.337 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:26:54.438 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:26:54.463 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:54.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:54.464 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:26:54.475 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:54 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:26:54.475 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:54 smithi070 ceph-mon[30388]: osdmap e102: 8 total, 8 up, 8 in 2024-09-16T09:26:54.476 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:54 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:26:54.476 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:54 smithi070 ceph-mon[40691]: osdmap e102: 8 total, 8 up, 8 in 2024-09-16T09:26:54.541 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T09:26:54.541 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:26:54.565 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:26:54.606 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:54.606 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:54.607 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:26:54.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:54 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:26:54.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:54 smithi084 ceph-mon[32629]: osdmap e102: 8 total, 8 up, 8 in 2024-09-16T09:26:54.707 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:26:54.735 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:54.735 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:54.736 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:26:54.837 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:26:54.877 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:54.878 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:54.879 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:26:54.980 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:26:55.015 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 42 16 73772 73756 109.721 79.25 0.00471605 0.00910914 2024-09-16T09:26:55.022 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:55.023 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:55.024 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:26:55.124 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:26:55.175 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:55.175 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:55.176 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:26:55.277 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:26:55.303 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:55.304 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:55.304 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:26:55.323 INFO:teuthology.orchestra.run.smithi070.stderr:no valid command found; 10 closest matches: 2024-09-16T09:26:55.323 INFO:teuthology.orchestra.run.smithi070.stderr:0 2024-09-16T09:26:55.323 INFO:teuthology.orchestra.run.smithi070.stderr:1 2024-09-16T09:26:55.323 INFO:teuthology.orchestra.run.smithi070.stderr:2 2024-09-16T09:26:55.324 INFO:teuthology.orchestra.run.smithi070.stderr:abort 2024-09-16T09:26:55.324 INFO:teuthology.orchestra.run.smithi070.stderr:assert 2024-09-16T09:26:55.324 INFO:teuthology.orchestra.run.smithi070.stderr:bench [] [] [] [] 2024-09-16T09:26:55.325 INFO:teuthology.orchestra.run.smithi070.stderr:bluefs debug_inject_read_zeros 2024-09-16T09:26:55.325 INFO:teuthology.orchestra.run.smithi070.stderr:bluefs files list 2024-09-16T09:26:55.325 INFO:teuthology.orchestra.run.smithi070.stderr:bluefs stats 2024-09-16T09:26:55.325 INFO:teuthology.orchestra.run.smithi070.stderr:bluestore allocator dump block 2024-09-16T09:26:55.325 INFO:teuthology.orchestra.run.smithi070.stderr:Error EINVAL: invalid command 2024-09-16T09:26:55.405 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:26:55.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:55 smithi070 ceph-mon[30388]: pgmap v72: 158 pgs: 158 active+clean; 4.3 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 853 B/s rd, 96 MiB/s wr, 1.53k op/s 2024-09-16T09:26:55.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:55 smithi070 ceph-mon[40691]: pgmap v72: 158 pgs: 158 active+clean; 4.3 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 853 B/s rd, 96 MiB/s wr, 1.53k op/s 2024-09-16T09:26:55.434 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:55.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:55.435 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:26:55.536 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:26:55.569 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:55.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:55.570 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:26:55.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:55 smithi084 ceph-mon[32629]: pgmap v72: 158 pgs: 158 active+clean; 4.3 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 853 B/s rd, 96 MiB/s wr, 1.53k op/s 2024-09-16T09:26:55.671 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:26:55.698 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:55.699 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:55.699 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:26:55.800 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:26:55.835 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:55.836 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:55.836 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:26:55.905 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.2 dump_historic_ops 2024-09-16T09:26:55.937 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:26:55.965 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:55.966 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:55.966 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:26:56.017 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 43 16 75061 75045 109.04 80.5625 0.0144068 0.00916617 2024-09-16T09:26:56.067 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:26:56.110 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:56.110 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:56.111 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:26:56.212 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:26:56.263 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:56.263 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:56.263 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:26:56.364 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:26:56.402 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:56.403 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:56.403 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:26:56.504 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:26:56.530 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:56.530 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:56.531 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:26:56.632 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:26:56.666 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:56.666 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:56.667 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:26:56.767 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:26:56.794 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:56.795 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:56.795 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:26:56.897 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:26:56.936 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:56.937 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:56.938 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:26:57.017 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 44 16 76346 76330 108.387 80.3125 0.00418654 0.00922112 2024-09-16T09:26:57.039 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:26:57.105 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:57.106 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:57.106 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:26:57.207 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:26:57.213 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Sep 16 09:26:57 smithi070 systemd[1]: Stopping Ceph alertmanager.a for a1321834-740c-11ef-bceb-c7b262605968... 2024-09-16T09:26:57.245 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:57.246 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:57.247 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:26:57.347 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:26:57.376 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:57.377 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:57.378 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:26:57.478 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:26:57.503 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:57.504 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:57.504 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:26:57.586 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:57 smithi070 ceph-mon[30388]: pgmap v74: 158 pgs: 158 active+clean; 4.3 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 853 B/s rd, 66 MiB/s wr, 1.05k op/s 2024-09-16T09:26:57.586 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:57 smithi070 ceph-mon[40691]: pgmap v74: 158 pgs: 158 active+clean; 4.3 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 853 B/s rd, 66 MiB/s wr, 1.05k op/s 2024-09-16T09:26:57.605 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:26:57.628 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:57.628 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:57.629 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:26:57.645 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-16T09:26:57.645 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-16T09:26:57.645 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph tell osd.2 'trim stale osdmaps' 2024-09-16T09:26:57.648 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:26:57.659 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:57 smithi084 ceph-mon[32629]: pgmap v74: 158 pgs: 158 active+clean; 4.3 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 853 B/s rd, 66 MiB/s wr, 1.05k op/s 2024-09-16T09:26:57.699 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:26:57.730 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:26:57.756 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:57.756 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:57.757 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:26:57.858 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:26:57.883 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:57.884 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:57.884 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:26:57.985 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:26:58.019 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 45 16 77499 77483 107.578 72.0625 0.00952609 0.00928991 2024-09-16T09:26:58.034 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:58.034 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:58.035 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:26:58.136 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:26:58.157 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:58.158 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:58.158 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:26:58.259 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:26:58.285 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:58.286 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:58.287 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:26:58.387 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:26:58.413 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:58.414 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:58.414 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:26:58.515 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:26:58.527 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T09:26:58.554 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:58.556 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:58.557 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:26:58.658 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:26:58.679 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:58.679 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:58.680 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:26:58.780 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:26:58.809 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:58.810 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:58.810 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:26:58.911 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:26:58.954 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:58.955 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:58.955 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:26:59.018 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 46 16 78644 78628 106.795 71.5625 0.0156153 0.00935904 2024-09-16T09:26:59.056 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:26:59.084 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:59.084 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:59.085 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:26:59.185 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:26:59.225 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:59.225 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:59.225 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:26:59.326 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:26:59.349 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:59.350 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:59.350 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:26:59.451 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:26:59.477 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:59.478 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:59.478 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:26:59.479 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:26:59 smithi070 ceph-mon[30388]: pgmap v75: 158 pgs: 158 active+clean; 4.5 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.4 KiB/s rd, 75 MiB/s wr, 1.19k op/s 2024-09-16T09:26:59.479 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:26:59 smithi070 ceph-mon[40691]: pgmap v75: 158 pgs: 158 active+clean; 4.5 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.4 KiB/s rd, 75 MiB/s wr, 1.19k op/s 2024-09-16T09:26:59.579 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:26:59.623 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:59.623 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:59.624 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:26:59.659 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:26:59 smithi084 ceph-mon[32629]: pgmap v75: 158 pgs: 158 active+clean; 4.5 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.4 KiB/s rd, 75 MiB/s wr, 1.19k op/s 2024-09-16T09:26:59.725 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:26:59.750 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:26:59.751 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:59.752 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:26:59.852 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:26:59.904 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:26:59.905 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:26:59.905 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:27:00.006 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:27:00.018 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 47 15 79747 79732 105.991 69 0.0118399 0.00942963 2024-09-16T09:27:00.049 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:00.049 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:00.050 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:27:00.151 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:27:00.179 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:00.180 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:00.180 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:27:00.227 INFO:teuthology.orchestra.run.smithi070.stderr:noscrub is unset 2024-09-16T09:27:00.281 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:27:00.307 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:00.308 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:00.308 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:27:00.372 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:00 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3064307626' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:27:00.372 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:00 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3064307626' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:27:00.372 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Sep 16 09:27:00 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-alertmanager-a[75718]: ts=2024-09-16T09:27:00.235Z caller=main.go:542 level=info msg="Received SIGTERM, exiting gracefully..." 2024-09-16T09:27:00.409 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:27:00.438 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:00.439 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:00.439 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:27:00.540 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:27:00.567 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:00.567 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:00.568 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:27:00.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:00 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3064307626' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:27:00.669 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:27:00.713 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:00.713 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:00.714 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:27:00.814 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:27:00.849 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:00.850 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:00.850 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:27:00.951 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:27:00.990 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:00.991 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:00.991 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:27:01.018 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 48 16 80899 80883 105.281 71.9375 0.0151792 0.00949394 2024-09-16T09:27:01.060 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T09:27:01.061 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:27:01.092 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:27:01.127 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:01.128 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:01.128 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:27:01.229 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:27:01.263 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:01.263 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:01.264 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:27:01.316 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:01 smithi070 ceph-mon[30388]: pgmap v76: 158 pgs: 158 active+clean; 4.7 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.2 KiB/s rd, 100 MiB/s wr, 1.60k op/s 2024-09-16T09:27:01.316 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:01 smithi070 ceph-mon[30388]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:27:01.316 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:01 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3064307626' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-16T09:27:01.316 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:01 smithi070 ceph-mon[30388]: osdmap e103: 8 total, 8 up, 8 in 2024-09-16T09:27:01.316 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:01 smithi070 ceph-mon[30388]: 3.1 scrub starts 2024-09-16T09:27:01.316 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:01 smithi070 ceph-mon[30388]: 3.1 scrub ok 2024-09-16T09:27:01.316 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:01 smithi070 ceph-mon[30388]: 5.1c scrub starts 2024-09-16T09:27:01.317 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:01 smithi070 ceph-mon[30388]: 5.1c scrub ok 2024-09-16T09:27:01.317 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:01 smithi070 ceph-mon[40691]: pgmap v76: 158 pgs: 158 active+clean; 4.7 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.2 KiB/s rd, 100 MiB/s wr, 1.60k op/s 2024-09-16T09:27:01.317 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:01 smithi070 ceph-mon[40691]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:27:01.318 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:01 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3064307626' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-16T09:27:01.318 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:01 smithi070 ceph-mon[40691]: osdmap e103: 8 total, 8 up, 8 in 2024-09-16T09:27:01.318 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:01 smithi070 ceph-mon[40691]: 3.1 scrub starts 2024-09-16T09:27:01.318 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:01 smithi070 ceph-mon[40691]: 3.1 scrub ok 2024-09-16T09:27:01.318 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:01 smithi070 ceph-mon[40691]: 5.1c scrub starts 2024-09-16T09:27:01.318 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:01 smithi070 ceph-mon[40691]: 5.1c scrub ok 2024-09-16T09:27:01.365 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:27:01.389 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:01.389 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:01.390 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:27:01.491 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:27:01.529 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:01.530 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:01.530 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:27:01.631 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:27:01.641 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Sep 16 09:27:01 smithi070 podman[84809]: 2024-09-16 09:27:01.314397544 +0000 UTC m=+1.226750995 container died f7b0530f3dbf8d8acb21cde05413a70b8de3412f7ad11a4fcff8fdf456b8593f (image=quay.io/prometheus/alertmanager:v0.27.0, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-16T09:27:01.658 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:01.659 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:01.659 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:27:01.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:01 smithi084 ceph-mon[32629]: pgmap v76: 158 pgs: 158 active+clean; 4.7 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.2 KiB/s rd, 100 MiB/s wr, 1.60k op/s 2024-09-16T09:27:01.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:01 smithi084 ceph-mon[32629]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:27:01.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:01 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3064307626' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-16T09:27:01.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:01 smithi084 ceph-mon[32629]: osdmap e103: 8 total, 8 up, 8 in 2024-09-16T09:27:01.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:01 smithi084 ceph-mon[32629]: 3.1 scrub starts 2024-09-16T09:27:01.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:01 smithi084 ceph-mon[32629]: 3.1 scrub ok 2024-09-16T09:27:01.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:01 smithi084 ceph-mon[32629]: 5.1c scrub starts 2024-09-16T09:27:01.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:01 smithi084 ceph-mon[32629]: 5.1c scrub ok 2024-09-16T09:27:01.760 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:27:01.792 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:01.792 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:01.793 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:27:01.847 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:27:01.893 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:27:01.920 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Sep 16 09:27:01 smithi070 podman[84809]: 2024-09-16 09:27:01.837731005 +0000 UTC m=+1.750084449 container cleanup f7b0530f3dbf8d8acb21cde05413a70b8de3412f7ad11a4fcff8fdf456b8593f (image=quay.io/prometheus/alertmanager:v0.27.0, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-16T09:27:01.921 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Sep 16 09:27:01 smithi070 bash[84809]: ceph-a1321834-740c-11ef-bceb-c7b262605968-alertmanager-a 2024-09-16T09:27:01.922 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:01.923 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:01.923 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:27:02.019 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 49 16 82150 82134 104.728 78.1875 0.00599777 0.00954407 2024-09-16T09:27:02.024 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:27:02.050 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:02.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:02.051 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:27:02.152 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:27:02.180 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:02.180 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:02.181 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:27:02.233 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Sep 16 09:27:02 smithi070 podman[84845]: 2024-09-16 09:27:02.123196739 +0000 UTC m=+1.830935707 container remove f7b0530f3dbf8d8acb21cde05413a70b8de3412f7ad11a4fcff8fdf456b8593f (image=quay.io/prometheus/alertmanager:v0.27.0, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-16T09:27:02.234 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:27:01 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:27:01] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:27:02.282 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:27:02.312 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:02.313 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:02.313 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:27:02.414 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:27:02.439 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:02.440 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:02.441 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:27:02.534 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:02 smithi070 ceph-mon[30388]: 6.19 scrub starts 2024-09-16T09:27:02.534 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:02 smithi070 ceph-mon[30388]: 6.19 scrub ok 2024-09-16T09:27:02.534 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:02 smithi070 ceph-mon[30388]: 4.9 scrub starts 2024-09-16T09:27:02.534 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:02 smithi070 ceph-mon[30388]: 4.9 scrub ok 2024-09-16T09:27:02.534 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:02 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:27:02.535 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:02 smithi070 ceph-mon[40691]: 6.19 scrub starts 2024-09-16T09:27:02.535 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:02 smithi070 ceph-mon[40691]: 6.19 scrub ok 2024-09-16T09:27:02.535 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:02 smithi070 ceph-mon[40691]: 4.9 scrub starts 2024-09-16T09:27:02.535 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:02 smithi070 ceph-mon[40691]: 4.9 scrub ok 2024-09-16T09:27:02.535 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:02 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:27:02.535 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Sep 16 09:27:02 smithi070 podman[84845]: 2024-09-16 09:27:02.418267879 +0000 UTC m=+2.126006859 volume remove 84d461f721208ab0f64ce78b1afdfc29060f206216c1c29a66dd70cb1a177c2c 2024-09-16T09:27:02.541 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:27:02.583 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:02 smithi084 ceph-mon[32629]: 6.19 scrub starts 2024-09-16T09:27:02.583 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:02 smithi084 ceph-mon[32629]: 6.19 scrub ok 2024-09-16T09:27:02.583 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:02 smithi084 ceph-mon[32629]: 4.9 scrub starts 2024-09-16T09:27:02.583 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:02 smithi084 ceph-mon[32629]: 4.9 scrub ok 2024-09-16T09:27:02.583 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:02 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:27:02.584 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:02.584 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:02.585 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:27:02.686 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:27:02.714 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:02.715 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:02.715 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:27:02.816 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:27:02.855 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:02.856 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:02.856 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:27:02.910 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:27:02 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:27:02.956 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:27:02.995 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:02.995 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:02.996 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:27:03.019 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 50 16 83326 83310 104.103 73.5 0.00921284 0.00960143 2024-09-16T09:27:03.097 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:27:03.124 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:03.124 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:03.125 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:27:03.226 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:27:03.250 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:03.250 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:03.251 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:27:03.308 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.3 dump_ops_in_flight 2024-09-16T09:27:03.351 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:27:03.445 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:03.445 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:03.445 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:27:03.546 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:27:03.575 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:03.575 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:03.576 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:27:03.608 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:03 smithi070 ceph-mon[30388]: pgmap v78: 158 pgs: 158 active+clean; 4.7 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 639 B/s rd, 50 MiB/s wr, 808 op/s 2024-09-16T09:27:03.609 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:03 smithi070 ceph-mon[30388]: 4.1d scrub starts 2024-09-16T09:27:03.609 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:03 smithi070 ceph-mon[30388]: 4.1d scrub ok 2024-09-16T09:27:03.609 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:03 smithi070 ceph-mon[40691]: pgmap v78: 158 pgs: 158 active+clean; 4.7 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 639 B/s rd, 50 MiB/s wr, 808 op/s 2024-09-16T09:27:03.609 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:03 smithi070 ceph-mon[40691]: 4.1d scrub starts 2024-09-16T09:27:03.609 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:03 smithi070 ceph-mon[40691]: 4.1d scrub ok 2024-09-16T09:27:03.659 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:03 smithi084 ceph-mon[32629]: pgmap v78: 158 pgs: 158 active+clean; 4.7 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 639 B/s rd, 50 MiB/s wr, 808 op/s 2024-09-16T09:27:03.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:03 smithi084 ceph-mon[32629]: 4.1d scrub starts 2024-09-16T09:27:03.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:03 smithi084 ceph-mon[32629]: 4.1d scrub ok 2024-09-16T09:27:03.677 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:27:03.699 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:27:03.730 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:03.731 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:03.731 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:27:03.832 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:27:03.859 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:03.859 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:03.860 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:27:03.961 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:27:03.984 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:03.984 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:03.985 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:27:04.020 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 51 16 84486 84470 103.481 72.5 0.00907866 0.00965851 2024-09-16T09:27:04.085 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:27:04.111 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:04.111 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:04.112 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:27:04.212 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:27:04.265 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:04.265 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:04.265 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:27:04.366 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:27:04.394 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:04.395 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:04.395 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:27:04.496 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:27:04.533 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:04 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:27:04.533 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:04 smithi070 ceph-mon[30388]: 4.14 scrub starts 2024-09-16T09:27:04.533 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:04 smithi070 ceph-mon[30388]: 4.14 scrub ok 2024-09-16T09:27:04.533 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:04 smithi070 ceph-mon[30388]: osdmap e104: 8 total, 8 up, 8 in 2024-09-16T09:27:04.533 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:04 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:27:04.533 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:04 smithi070 ceph-mon[40691]: 4.14 scrub starts 2024-09-16T09:27:04.533 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:04 smithi070 ceph-mon[40691]: 4.14 scrub ok 2024-09-16T09:27:04.533 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:04 smithi070 ceph-mon[40691]: osdmap e104: 8 total, 8 up, 8 in 2024-09-16T09:27:04.534 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:04.535 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:04.535 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:27:04.615 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:27:04.615 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:27:04.635 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:27:04.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:04 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:27:04.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:04 smithi084 ceph-mon[32629]: 4.14 scrub starts 2024-09-16T09:27:04.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:04 smithi084 ceph-mon[32629]: 4.14 scrub ok 2024-09-16T09:27:04.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:04 smithi084 ceph-mon[32629]: osdmap e104: 8 total, 8 up, 8 in 2024-09-16T09:27:04.660 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:27:04.661 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:27:04.663 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:04.663 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:04.664 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:27:04.693 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:27:04.693 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:27:04.734 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:27:04.734 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:27:04.765 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:27:04.768 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:27:04.769 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:27:04.810 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:27:04.811 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:27:04.811 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:04.812 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:04.812 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:27:04.845 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:27:04.845 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:27:04.887 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:27:04.888 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:27:04.912 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:27:04.937 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:04.938 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:04.938 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:27:04.951 INFO:teuthology.orchestra.run.smithi070.stderr:no valid command found; 10 closest matches: 2024-09-16T09:27:04.951 INFO:teuthology.orchestra.run.smithi070.stderr:0 2024-09-16T09:27:04.951 INFO:teuthology.orchestra.run.smithi070.stderr:1 2024-09-16T09:27:04.951 INFO:teuthology.orchestra.run.smithi070.stderr:2 2024-09-16T09:27:04.952 INFO:teuthology.orchestra.run.smithi070.stderr:abort 2024-09-16T09:27:04.952 INFO:teuthology.orchestra.run.smithi070.stderr:assert 2024-09-16T09:27:04.952 INFO:teuthology.orchestra.run.smithi070.stderr:bench [] [] [] [] 2024-09-16T09:27:04.952 INFO:teuthology.orchestra.run.smithi070.stderr:bluefs debug_inject_read_zeros 2024-09-16T09:27:04.952 INFO:teuthology.orchestra.run.smithi070.stderr:bluefs files list 2024-09-16T09:27:04.952 INFO:teuthology.orchestra.run.smithi070.stderr:bluefs stats 2024-09-16T09:27:04.952 INFO:teuthology.orchestra.run.smithi070.stderr:bluestore allocator dump block 2024-09-16T09:27:04.952 INFO:teuthology.orchestra.run.smithi070.stderr:Error EINVAL: invalid command 2024-09-16T09:27:05.020 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 52 16 85590 85574 102.817 69 0.0129868 0.00972103 2024-09-16T09:27:05.039 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:27:05.082 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:05.082 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:05.083 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:27:05.141 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Sep 16 09:27:04 smithi070 systemd[1]: ceph-a1321834-740c-11ef-bceb-c7b262605968@alertmanager.a.service: Deactivated successfully. 2024-09-16T09:27:05.141 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Sep 16 09:27:04 smithi070 systemd[1]: Stopped Ceph alertmanager.a for a1321834-740c-11ef-bceb-c7b262605968. 2024-09-16T09:27:05.141 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Sep 16 09:27:04 smithi070 systemd[1]: ceph-a1321834-740c-11ef-bceb-c7b262605968@alertmanager.a.service: Consumed 1.250s CPU time. 2024-09-16T09:27:05.141 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Sep 16 09:27:04 smithi070 systemd[1]: Starting Ceph alertmanager.a for a1321834-740c-11ef-bceb-c7b262605968... 2024-09-16T09:27:05.184 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:27:05.227 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:05.228 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:05.228 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:27:05.329 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:27:05.356 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:05.357 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:05.358 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:27:05.379 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:27:05.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:05 smithi070 ceph-mon[30388]: pgmap v79: 158 pgs: 158 active+clean; 5.0 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1.1 KiB/s rd, 81 MiB/s wr, 1.30k op/s 2024-09-16T09:27:05.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:05 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2812222882' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:27:05.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:05 smithi070 ceph-mon[40691]: pgmap v79: 158 pgs: 158 active+clean; 5.0 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1.1 KiB/s rd, 81 MiB/s wr, 1.30k op/s 2024-09-16T09:27:05.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:05 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/2812222882' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:27:05.458 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:27:05.503 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:05.503 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:05.504 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:27:05.605 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:27:05.632 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:05.633 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:05.633 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:27:05.659 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:05 smithi084 ceph-mon[32629]: pgmap v79: 158 pgs: 158 active+clean; 5.0 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1.1 KiB/s rd, 81 MiB/s wr, 1.30k op/s 2024-09-16T09:27:05.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:05 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/2812222882' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:27:05.666 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-16T09:27:05.735 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:27:05.760 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:05.760 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:05.761 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:27:05.862 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:27:05.941 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:05.941 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:05.942 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:27:06.023 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 53 16 86747 86731 102.237 72.3125 0.0107653 0.00977578 2024-09-16T09:27:06.043 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:27:06.093 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:06.094 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:06.094 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:27:06.195 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:27:06.216 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:06.217 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:06.217 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:27:06.318 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:27:06.361 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:06.362 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:06.362 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:27:06.463 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:27:06.497 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:06.498 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:06.498 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:27:06.599 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:27:06.627 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:06.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:06.628 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:27:06.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:06 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:27:06.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:06 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:27:06.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:06 smithi084 ceph-mon[32629]: osdmap e105: 8 total, 8 up, 8 in 2024-09-16T09:27:06.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:06 smithi084 ceph-mon[32629]: 4.1b scrub starts 2024-09-16T09:27:06.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:06 smithi084 ceph-mon[32629]: 4.1b scrub ok 2024-09-16T09:27:06.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:06 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:27:06.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:06 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:27:06.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:06 smithi070 ceph-mon[30388]: osdmap e105: 8 total, 8 up, 8 in 2024-09-16T09:27:06.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:06 smithi070 ceph-mon[30388]: 4.1b scrub starts 2024-09-16T09:27:06.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:06 smithi070 ceph-mon[30388]: 4.1b scrub ok 2024-09-16T09:27:06.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:06 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:27:06.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:06 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:27:06.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:06 smithi070 ceph-mon[40691]: osdmap e105: 8 total, 8 up, 8 in 2024-09-16T09:27:06.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:06 smithi070 ceph-mon[40691]: 4.1b scrub starts 2024-09-16T09:27:06.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:06 smithi070 ceph-mon[40691]: 4.1b scrub ok 2024-09-16T09:27:06.729 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:27:06.757 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:06.758 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:06.758 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:27:06.859 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:27:06.884 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:06.884 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:06.885 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:27:06.985 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:27:07.014 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:07.015 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:07.015 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:27:07.024 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 54 16 87856 87840 101.626 69.3125 0.012327 0.00983477 2024-09-16T09:27:07.116 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:27:07.149 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:07.149 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:07.149 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:27:07.250 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:27:07.272 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:07.272 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:07.272 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:27:07.373 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:27:07.414 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:07.415 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:07.415 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:27:07.516 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:27:07.540 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:07.541 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:07.541 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:27:07.566 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:07 smithi070 ceph-mon[30388]: pgmap v82: 158 pgs: 158 active+clean; 5.0 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 853 B/s rd, 60 MiB/s wr, 954 op/s 2024-09-16T09:27:07.566 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:07 smithi070 ceph-mon[30388]: 5.14 scrub starts 2024-09-16T09:27:07.566 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:07 smithi070 ceph-mon[30388]: 5.14 scrub ok 2024-09-16T09:27:07.566 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:07 smithi070 ceph-mon[30388]: 2.1 scrub starts 2024-09-16T09:27:07.566 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:07 smithi070 ceph-mon[30388]: 2.1 scrub ok 2024-09-16T09:27:07.566 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:07 smithi070 ceph-mon[40691]: pgmap v82: 158 pgs: 158 active+clean; 5.0 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 853 B/s rd, 60 MiB/s wr, 954 op/s 2024-09-16T09:27:07.567 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:07 smithi070 ceph-mon[40691]: 5.14 scrub starts 2024-09-16T09:27:07.567 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:07 smithi070 ceph-mon[40691]: 5.14 scrub ok 2024-09-16T09:27:07.567 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:07 smithi070 ceph-mon[40691]: 2.1 scrub starts 2024-09-16T09:27:07.567 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:07 smithi070 ceph-mon[40691]: 2.1 scrub ok 2024-09-16T09:27:07.643 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:27:07.659 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:07 smithi084 ceph-mon[32629]: pgmap v82: 158 pgs: 158 active+clean; 5.0 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 853 B/s rd, 60 MiB/s wr, 954 op/s 2024-09-16T09:27:07.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:07 smithi084 ceph-mon[32629]: 5.14 scrub starts 2024-09-16T09:27:07.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:07 smithi084 ceph-mon[32629]: 5.14 scrub ok 2024-09-16T09:27:07.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:07 smithi084 ceph-mon[32629]: 2.1 scrub starts 2024-09-16T09:27:07.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:07 smithi084 ceph-mon[32629]: 2.1 scrub ok 2024-09-16T09:27:07.682 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:07.683 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:07.683 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:27:07.784 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:27:07.809 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:07.809 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:07.810 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:27:07.911 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:27:07.918 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T09:27:07.935 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:07.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:07.936 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:27:08.024 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 55 16 88881 88865 100.943 64.0625 0.00317521 0.00989988 2024-09-16T09:27:08.037 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:27:08.059 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:08.060 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:08.060 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:27:08.161 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:27:08.206 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:08.207 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:08.207 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:27:08.308 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:27:08.334 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:08.334 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:08.335 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:27:08.435 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:27:08.462 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:08.462 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:08.463 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:27:08.564 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:27:08.588 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:08.589 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:08.589 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:27:08.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:08 smithi084 ceph-mon[32629]: 5.7 scrub starts 2024-09-16T09:27:08.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:08 smithi084 ceph-mon[32629]: 5.7 scrub ok 2024-09-16T09:27:08.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:08 smithi084 ceph-mon[32629]: 3.1f scrub starts 2024-09-16T09:27:08.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:08 smithi084 ceph-mon[32629]: 3.1f scrub ok 2024-09-16T09:27:08.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:08 smithi070 ceph-mon[30388]: 5.7 scrub starts 2024-09-16T09:27:08.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:08 smithi070 ceph-mon[30388]: 5.7 scrub ok 2024-09-16T09:27:08.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:08 smithi070 ceph-mon[30388]: 3.1f scrub starts 2024-09-16T09:27:08.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:08 smithi070 ceph-mon[30388]: 3.1f scrub ok 2024-09-16T09:27:08.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:08 smithi070 ceph-mon[40691]: 5.7 scrub starts 2024-09-16T09:27:08.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:08 smithi070 ceph-mon[40691]: 5.7 scrub ok 2024-09-16T09:27:08.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:08 smithi070 ceph-mon[40691]: 3.1f scrub starts 2024-09-16T09:27:08.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:08 smithi070 ceph-mon[40691]: 3.1f scrub ok 2024-09-16T09:27:08.690 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:27:08.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:08.729 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:08.729 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:27:08.830 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:27:08.882 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:08.883 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:08.884 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:27:08.984 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:27:09.024 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 56 16 89834 89818 100.204 59.5625 0.0278314 0.00997448 2024-09-16T09:27:09.027 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:09.028 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:09.028 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:27:09.129 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:27:09.165 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:09.166 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:09.166 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:27:09.267 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:27:09.296 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:09.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:09.297 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:27:09.301 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T09:27:09.302 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [ 2024-09-16T09:27:09.302 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:09.302 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.24698.0:89526 7.a 7:5160206a:::benchmark_data_smithi070_78438_object89525:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-16T09:27:09.302 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:27:08.700103+0000", 2024-09-16T09:27:09.302 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.01762822, 2024-09-16T09:27:09.302 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.017658879999999998, 2024-09-16T09:27:09.302 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:27:09.302 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:27:09.302 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "waiting for sub ops", 2024-09-16T09:27:09.302 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:27:09.302 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.24698", 2024-09-16T09:27:09.303 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:27:09.303 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89526 2024-09-16T09:27:09.303 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:09.303 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:27:09.303 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:09.303 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:27:09.303 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:08.700103+0000", 2024-09-16T09:27:09.303 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:27:09.303 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:09.303 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:09.303 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:27:09.303 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:08.700103+0000", 2024-09-16T09:27:09.303 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:27:09.303 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:09.304 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:09.304 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:27:09.304 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:08.700105+0000", 2024-09-16T09:27:09.304 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6279999999999999e-06 2024-09-16T09:27:09.304 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:09.304 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:09.304 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:27:09.304 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:08.700129+0000", 2024-09-16T09:27:09.304 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.4076000000000001e-05 2024-09-16T09:27:09.304 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:09.304 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:09.304 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:27:09.304 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:08.700145+0000", 2024-09-16T09:27:09.305 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6164e-05 2024-09-16T09:27:09.305 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:09.305 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:09.305 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:27:09.305 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:08.700153+0000", 2024-09-16T09:27:09.305 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.4449999999999999e-06 2024-09-16T09:27:09.305 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:09.305 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:09.305 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:27:09.305 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:08.701293+0000", 2024-09-16T09:27:09.305 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0011406980000000001 2024-09-16T09:27:09.305 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:09.305 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:09.306 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:27:09.306 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:08.701392+0000", 2024-09-16T09:27:09.306 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.8591999999999994e-05 2024-09-16T09:27:09.306 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:09.306 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:09.306 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 2,5", 2024-09-16T09:27:09.306 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:08.701534+0000", 2024-09-16T09:27:09.306 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00014257199999999999 2024-09-16T09:27:09.306 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:09.306 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:09.306 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:27:09.306 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:08.712561+0000", 2024-09-16T09:27:09.306 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.011026253999999999 2024-09-16T09:27:09.307 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:09.307 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:09.307 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:27:09.307 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:08.712750+0000", 2024-09-16T09:27:09.307 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00018939099999999999 2024-09-16T09:27:09.307 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:27:09.307 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:27:09.307 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:27:09.307 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:09.307 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:09.307 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.24698.0:89527 7.2 7:42c678ae:::benchmark_data_smithi070_78438_object89526:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-16T09:27:09.307 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:27:08.701212+0000", 2024-09-16T09:27:09.307 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.016519299000000001, 2024-09-16T09:27:09.308 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.016630265000000002, 2024-09-16T09:27:09.308 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:27:09.308 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:27:09.308 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "waiting for sub ops", 2024-09-16T09:27:09.308 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:27:09.308 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.24698", 2024-09-16T09:27:09.308 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:27:09.308 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89527 2024-09-16T09:27:09.308 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:09.308 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:27:09.308 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:09.308 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:27:09.308 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:08.701212+0000", 2024-09-16T09:27:09.308 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:27:09.309 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:09.309 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:09.309 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:27:09.309 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:08.701212+0000", 2024-09-16T09:27:09.309 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:27:09.309 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:09.309 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:09.309 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:27:09.309 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:08.701213+0000", 2024-09-16T09:27:09.309 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.3300000000000001e-07 2024-09-16T09:27:09.309 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:09.309 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:09.309 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:27:09.310 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:08.701242+0000", 2024-09-16T09:27:09.310 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.8479e-05 2024-09-16T09:27:09.310 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:09.310 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:09.310 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:27:09.310 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:08.701253+0000", 2024-09-16T09:27:09.310 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1228e-05 2024-09-16T09:27:09.310 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:09.310 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:09.310 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:27:09.310 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:08.701260+0000", 2024-09-16T09:27:09.310 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.0609999999999998e-06 2024-09-16T09:27:09.310 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:09.311 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:09.311 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:27:09.311 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:08.701488+0000", 2024-09-16T09:27:09.311 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000228266 2024-09-16T09:27:09.311 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:09.311 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:09.311 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:27:09.311 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:08.701648+0000", 2024-09-16T09:27:09.311 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00016019299999999999 2024-09-16T09:27:09.311 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:09.311 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:09.311 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,2", 2024-09-16T09:27:09.311 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:08.701773+0000", 2024-09-16T09:27:09.311 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00012505099999999999 2024-09-16T09:27:09.312 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:27:09.312 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:27:09.312 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:27:09.312 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:09.312 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:09.312 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.24698.0:89528 7.a 7:537db979:::benchmark_data_smithi070_78438_object89527:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-16T09:27:09.312 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:27:08.701273+0000", 2024-09-16T09:27:09.312 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.016458441000000001, 2024-09-16T09:27:09.312 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.016632596999999999, 2024-09-16T09:27:09.312 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:27:09.312 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:27:09.312 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "waiting for sub ops", 2024-09-16T09:27:09.312 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:27:09.312 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.24698", 2024-09-16T09:27:09.313 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:27:09.313 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89528 2024-09-16T09:27:09.313 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:09.313 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:27:09.313 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:09.313 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:27:09.313 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:08.701273+0000", 2024-09-16T09:27:09.313 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:27:09.313 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:09.313 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:09.313 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:27:09.313 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:08.701273+0000", 2024-09-16T09:27:09.313 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:27:09.314 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:09.314 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:09.314 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:27:09.314 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:08.701274+0000", 2024-09-16T09:27:09.314 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.3600000000000003e-07 2024-09-16T09:27:09.314 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:09.314 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:09.314 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:27:09.314 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:08.701293+0000", 2024-09-16T09:27:09.314 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9199999999999999e-05 2024-09-16T09:27:09.314 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:09.314 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:09.314 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:27:09.314 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:08.701300+0000", 2024-09-16T09:27:09.314 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3239999999999997e-06 2024-09-16T09:27:09.315 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:09.315 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:09.315 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:27:09.315 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:08.701306+0000", 2024-09-16T09:27:09.315 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4709999999999998e-06 2024-09-16T09:27:09.315 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:09.315 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:09.315 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:27:09.315 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:08.704010+0000", 2024-09-16T09:27:09.315 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0027046330000000001 2024-09-16T09:27:09.315 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:09.315 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:09.315 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:27:09.315 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:08.704128+0000", 2024-09-16T09:27:09.316 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00011752399999999999 2024-09-16T09:27:09.316 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:09.316 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:09.316 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 2,5", 2024-09-16T09:27:09.316 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:08.704262+0000", 2024-09-16T09:27:09.316 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000134172 2024-09-16T09:27:09.316 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:09.316 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:09.316 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:27:09.316 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:08.712740+0000", 2024-09-16T09:27:09.316 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0084775749999999993 2024-09-16T09:27:09.316 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:09.317 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:09.317 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:27:09.317 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:08.712754+0000", 2024-09-16T09:27:09.317 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.4790000000000001e-05 2024-09-16T09:27:09.317 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:27:09.317 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:27:09.317 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:27:09.317 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:09.317 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:09.317 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.24698.0:89530 7.2 7:43ab0c60:::benchmark_data_smithi070_78438_object89529:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-16T09:27:09.317 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:27:08.701456+0000", 2024-09-16T09:27:09.317 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.016275313999999999, 2024-09-16T09:27:09.317 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.016518036999999999, 2024-09-16T09:27:09.317 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:27:09.318 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:27:09.318 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "waiting for sub ops", 2024-09-16T09:27:09.318 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:27:09.318 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.24698", 2024-09-16T09:27:09.318 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:27:09.318 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89530 2024-09-16T09:27:09.318 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:09.318 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:27:09.318 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:09.318 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:27:09.318 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:08.701456+0000", 2024-09-16T09:27:09.318 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:27:09.318 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:09.318 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:09.319 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:27:09.319 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:08.701456+0000", 2024-09-16T09:27:09.319 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:27:09.319 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:09.319 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:09.319 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:27:09.319 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:08.701457+0000", 2024-09-16T09:27:09.319 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.8899999999999998e-07 2024-09-16T09:27:09.319 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:09.319 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:09.319 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:27:09.319 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:08.701479+0000", 2024-09-16T09:27:09.319 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.2232e-05 2024-09-16T09:27:09.319 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:09.320 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:09.320 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:27:09.320 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:08.701490+0000", 2024-09-16T09:27:09.320 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0866999999999999e-05 2024-09-16T09:27:09.320 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:09.320 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:09.320 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:27:09.320 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:08.701496+0000", 2024-09-16T09:27:09.320 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.3709999999999996e-06 2024-09-16T09:27:09.320 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:09.320 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:09.320 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:27:09.320 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:08.713505+0000", 2024-09-16T09:27:09.320 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.012008234 2024-09-16T09:27:09.321 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:09.321 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:09.321 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:27:09.321 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:08.716264+0000", 2024-09-16T09:27:09.321 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0027592799999999998 2024-09-16T09:27:09.321 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:09.321 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:09.321 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,2", 2024-09-16T09:27:09.321 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:08.716426+0000", 2024-09-16T09:27:09.321 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00016223100000000001 2024-09-16T09:27:09.321 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:27:09.321 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:27:09.321 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:27:09.321 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:09.322 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:09.322 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.24698.0:89532 7.a 7.8e934cda (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-16T09:27:09.322 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:27:08.701921+0000", 2024-09-16T09:27:09.322 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.015810472999999999, 2024-09-16T09:27:09.322 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.016111062999999998, 2024-09-16T09:27:09.322 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:27:09.322 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:27:09.322 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T09:27:09.322 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:27:09.322 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.24698", 2024-09-16T09:27:09.322 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:27:09.322 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89532 2024-09-16T09:27:09.322 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:09.322 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:27:09.323 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:09.323 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:27:09.323 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:08.701921+0000", 2024-09-16T09:27:09.323 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:27:09.323 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:09.323 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:09.323 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:27:09.323 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:08.701921+0000", 2024-09-16T09:27:09.323 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:27:09.323 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:09.323 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:09.323 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:27:09.323 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:08.701922+0000", 2024-09-16T09:27:09.323 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0759999999999999e-06 2024-09-16T09:27:09.323 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:09.324 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:09.324 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:27:09.324 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:08.701951+0000", 2024-09-16T09:27:09.324 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.9062000000000001e-05 2024-09-16T09:27:09.324 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:09.324 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:09.324 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:27:09.324 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:08.701967+0000", 2024-09-16T09:27:09.324 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6076e-05 2024-09-16T09:27:09.324 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:09.324 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:09.324 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:27:09.324 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:08.701975+0000", 2024-09-16T09:27:09.324 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.0849999999999997e-06 2024-09-16T09:27:09.324 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:27:09.325 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:27:09.325 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:27:09.325 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:09.325 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:09.325 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.24698.0:89537 7.0 e105/82 7:075aed29:::benchmark_data_smithi070_78438_object89536:head v 105'5667, mlcod=105'5662)", 2024-09-16T09:27:09.325 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:27:08.710326+0000", 2024-09-16T09:27:09.325 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.0074053549999999998, 2024-09-16T09:27:09.325 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.007753373, 2024-09-16T09:27:09.325 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:27:09.325 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:27:09.325 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T09:27:09.325 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:27:09.325 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:09.325 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:27:09.326 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:08.710326+0000", 2024-09-16T09:27:09.326 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:27:09.326 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:09.326 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:09.326 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:27:09.326 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:08.710326+0000", 2024-09-16T09:27:09.326 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:27:09.326 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:09.326 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:09.326 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:27:09.326 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:08.710327+0000", 2024-09-16T09:27:09.326 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.9999999999999996e-07 2024-09-16T09:27:09.326 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:09.327 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:09.327 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:27:09.327 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:08.710420+0000", 2024-09-16T09:27:09.327 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.3247999999999999e-05 2024-09-16T09:27:09.327 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:09.327 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:09.327 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:27:09.327 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:08.710430+0000", 2024-09-16T09:27:09.327 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.7429999999999997e-06 2024-09-16T09:27:09.327 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:09.327 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:09.327 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:27:09.327 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:08.710439+0000", 2024-09-16T09:27:09.328 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.8109999999999992e-06 2024-09-16T09:27:09.328 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:09.328 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:09.328 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:27:09.328 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:08.713562+0000", 2024-09-16T09:27:09.328 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0031230920000000001 2024-09-16T09:27:09.328 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:09.328 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:09.328 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:27:09.328 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:08.713722+0000", 2024-09-16T09:27:09.328 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00016010899999999999 2024-09-16T09:27:09.328 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:27:09.328 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:27:09.328 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:27:09.328 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:09.329 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:09.329 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop_reply(client.24698.0:89527 7.2 e105/82)", 2024-09-16T09:27:09.329 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:27:08.713349+0000", 2024-09-16T09:27:09.329 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.0043828340000000004, 2024-09-16T09:27:09.329 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0047743079999999997, 2024-09-16T09:27:09.329 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:27:09.329 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:27:09.329 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T09:27:09.329 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:27:09.329 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:09.329 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:27:09.329 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:08.713349+0000", 2024-09-16T09:27:09.329 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:27:09.330 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:09.330 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:09.330 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:27:09.330 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:08.713349+0000", 2024-09-16T09:27:09.330 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:27:09.330 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:09.330 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:09.330 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:27:09.330 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:08.713349+0000", 2024-09-16T09:27:09.330 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.7199999999999998e-07 2024-09-16T09:27:09.330 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:09.330 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:09.330 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:27:09.330 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:08.713354+0000", 2024-09-16T09:27:09.331 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.369e-06 2024-09-16T09:27:09.331 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:09.331 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:09.331 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:27:09.331 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:08.713364+0000", 2024-09-16T09:27:09.331 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0767999999999999e-05 2024-09-16T09:27:09.331 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:09.331 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:09.331 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:27:09.331 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:08.713375+0000", 2024-09-16T09:27:09.331 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0967e-05 2024-09-16T09:27:09.331 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:27:09.331 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:27:09.331 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:27:09.332 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:09.332 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:09.332 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.24698.0:89540 7.a 7.6f64e90a (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-16T09:27:09.332 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:27:08.716829+0000", 2024-09-16T09:27:09.332 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.00090257800000000002, 2024-09-16T09:27:09.332 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001333515, 2024-09-16T09:27:09.332 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:27:09.332 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:27:09.332 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T09:27:09.332 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:27:09.332 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.24698", 2024-09-16T09:27:09.332 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:27:09.332 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89540 2024-09-16T09:27:09.332 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:09.333 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:27:09.333 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:09.333 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:27:09.333 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:08.716829+0000", 2024-09-16T09:27:09.333 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:27:09.333 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:09.333 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:09.333 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:27:09.333 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:08.716829+0000", 2024-09-16T09:27:09.333 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:27:09.333 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:09.333 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:09.333 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:27:09.334 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:08.716830+0000", 2024-09-16T09:27:09.334 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.469e-06 2024-09-16T09:27:09.334 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:09.334 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:09.334 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:27:09.334 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:08.716856+0000", 2024-09-16T09:27:09.334 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.5111000000000001e-05 2024-09-16T09:27:09.334 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:09.334 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:09.334 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:27:09.334 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:08.716869+0000", 2024-09-16T09:27:09.334 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3793e-05 2024-09-16T09:27:09.334 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:09.334 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:09.334 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:27:09.335 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:08.716877+0000", 2024-09-16T09:27:09.335 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.8809999999999996e-06 2024-09-16T09:27:09.335 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:27:09.335 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:27:09.335 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:27:09.335 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:09.335 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:09.335 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.24698.0:89541 7.a 7.c445800a (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-16T09:27:09.335 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:27:08.716892+0000", 2024-09-16T09:27:09.335 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.00084000999999999998, 2024-09-16T09:27:09.335 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0013176559999999999, 2024-09-16T09:27:09.335 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:27:09.335 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:27:09.335 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T09:27:09.335 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:27:09.336 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.24698", 2024-09-16T09:27:09.336 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:27:09.336 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 89541 2024-09-16T09:27:09.336 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:09.336 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:27:09.336 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:09.336 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:27:09.336 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:08.716892+0000", 2024-09-16T09:27:09.336 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:27:09.336 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:09.336 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:09.336 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:27:09.336 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:08.716892+0000", 2024-09-16T09:27:09.337 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:27:09.337 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:09.337 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:09.337 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:27:09.337 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:08.716892+0000", 2024-09-16T09:27:09.337 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.75e-07 2024-09-16T09:27:09.337 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:09.337 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:09.337 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:27:09.337 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:08.716911+0000", 2024-09-16T09:27:09.337 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9354000000000001e-05 2024-09-16T09:27:09.337 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:09.337 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:09.337 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:27:09.338 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:08.716918+0000", 2024-09-16T09:27:09.338 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.4060000000000002e-06 2024-09-16T09:27:09.338 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:09.338 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:09.338 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:27:09.338 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:08.716923+0000", 2024-09-16T09:27:09.338 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.8679999999999998e-06 2024-09-16T09:27:09.338 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:27:09.338 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:27:09.338 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:27:09.338 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:27:09.338 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-16T09:27:09.338 INFO:teuthology.orchestra.run.smithi070.stdout: "num_ops": 9 2024-09-16T09:27:09.338 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T09:27:09.398 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:27:09.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:09 smithi070 ceph-mon[30388]: pgmap v83: 158 pgs: 158 active+clean; 5.2 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.4 KiB/s rd, 67 MiB/s wr, 1.08k op/s 2024-09-16T09:27:09.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:09 smithi070 ceph-mon[30388]: 3.19 scrub starts 2024-09-16T09:27:09.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:09 smithi070 ceph-mon[30388]: 3.19 scrub ok 2024-09-16T09:27:09.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:09 smithi070 ceph-mon[30388]: osdmap e106: 8 total, 8 up, 8 in 2024-09-16T09:27:09.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:09 smithi070 ceph-mon[40691]: pgmap v83: 158 pgs: 158 active+clean; 5.2 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.4 KiB/s rd, 67 MiB/s wr, 1.08k op/s 2024-09-16T09:27:09.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:09 smithi070 ceph-mon[40691]: 3.19 scrub starts 2024-09-16T09:27:09.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:09 smithi070 ceph-mon[40691]: 3.19 scrub ok 2024-09-16T09:27:09.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:09 smithi070 ceph-mon[40691]: osdmap e106: 8 total, 8 up, 8 in 2024-09-16T09:27:09.441 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:09.441 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:09.442 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:27:09.542 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:27:09.571 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:09.572 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:09.572 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:27:09.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:09 smithi084 ceph-mon[32629]: pgmap v83: 158 pgs: 158 active+clean; 5.2 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.4 KiB/s rd, 67 MiB/s wr, 1.08k op/s 2024-09-16T09:27:09.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:09 smithi084 ceph-mon[32629]: 3.19 scrub starts 2024-09-16T09:27:09.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:09 smithi084 ceph-mon[32629]: 3.19 scrub ok 2024-09-16T09:27:09.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:09 smithi084 ceph-mon[32629]: osdmap e106: 8 total, 8 up, 8 in 2024-09-16T09:27:09.673 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:27:09.714 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:09.715 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:09.715 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:27:09.816 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:27:09.859 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:09.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:09.861 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:27:09.961 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:27:09.998 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:09.999 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:09.999 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:27:10.024 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 57 16 91007 90991 99.7326 73.3125 0.0307043 0.0100221 2024-09-16T09:27:10.100 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:27:10.128 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:10.129 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:10.129 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:27:10.170 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-16T09:27:10.170 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-16T09:27:10.171 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph tell osd.3 'trim stale osdmaps' 2024-09-16T09:27:10.230 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:27:10.260 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:10.261 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:10.261 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:27:10.362 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:27:10.405 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:10.406 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:10.406 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:27:10.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:10 smithi070 ceph-mon[30388]: 4.0 scrub starts 2024-09-16T09:27:10.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:10 smithi070 ceph-mon[30388]: 4.0 scrub ok 2024-09-16T09:27:10.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:10 smithi070 ceph-mon[30388]: 5.2 scrub starts 2024-09-16T09:27:10.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:10 smithi070 ceph-mon[30388]: 5.2 scrub ok 2024-09-16T09:27:10.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:10 smithi070 ceph-mon[30388]: 3.1b scrub starts 2024-09-16T09:27:10.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:10 smithi070 ceph-mon[30388]: 3.1b scrub ok 2024-09-16T09:27:10.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:10 smithi070 ceph-mon[40691]: 4.0 scrub starts 2024-09-16T09:27:10.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:10 smithi070 ceph-mon[40691]: 4.0 scrub ok 2024-09-16T09:27:10.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:10 smithi070 ceph-mon[40691]: 5.2 scrub starts 2024-09-16T09:27:10.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:10 smithi070 ceph-mon[40691]: 5.2 scrub ok 2024-09-16T09:27:10.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:10 smithi070 ceph-mon[40691]: 3.1b scrub starts 2024-09-16T09:27:10.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:10 smithi070 ceph-mon[40691]: 3.1b scrub ok 2024-09-16T09:27:10.422 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Sep 16 09:27:10 smithi070 podman[85821]: 2024-09-16 09:27:10.15464409 +0000 UTC m=+0.466800824 image pull 11f11916f8cdfb17c1da74b20deb2075d778357b4d60474d82453303f8d461f5 quay.io/prometheus/alertmanager:v0.27.0 2024-09-16T09:27:10.422 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Sep 16 09:27:10 smithi070 podman[85821]: 2024-09-16 09:27:10.291362056 +0000 UTC m=+0.603518776 volume create 5968f77b31603c5a299e8158a688facd2b90309e305a1fff4f17f3ee954ca1fa 2024-09-16T09:27:10.431 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:27:10.507 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:27:10.538 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:10.539 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:10.539 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:27:10.640 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:27:10.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:10 smithi084 ceph-mon[32629]: 4.0 scrub starts 2024-09-16T09:27:10.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:10 smithi084 ceph-mon[32629]: 4.0 scrub ok 2024-09-16T09:27:10.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:10 smithi084 ceph-mon[32629]: 5.2 scrub starts 2024-09-16T09:27:10.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:10 smithi084 ceph-mon[32629]: 5.2 scrub ok 2024-09-16T09:27:10.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:10 smithi084 ceph-mon[32629]: 3.1b scrub starts 2024-09-16T09:27:10.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:10 smithi084 ceph-mon[32629]: 3.1b scrub ok 2024-09-16T09:27:10.699 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:10.699 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:10.700 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:27:10.712 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Sep 16 09:27:10 smithi070 podman[85821]: 2024-09-16 09:27:10.419670301 +0000 UTC m=+0.731827022 container create 3da3e64aa0d999725b464dc75c835413d99e831bc302e269d176a103332c2c79 (image=quay.io/prometheus/alertmanager:v0.27.0, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-16T09:27:10.800 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:27:10.828 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:10.829 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:10.829 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:27:10.930 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:27:10.955 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:10.955 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:10.956 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:27:11.025 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 58 16 92153 92137 99.2481 71.625 0.00481086 0.0100706 2024-09-16T09:27:11.056 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:27:11.106 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:11.107 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:11.107 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:27:11.208 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:27:11.249 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:11.250 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:11.250 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:27:11.325 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Sep 16 09:27:11 smithi070 podman[85821]: 2024-09-16 09:27:11.032824075 +0000 UTC m=+1.344980813 container init 3da3e64aa0d999725b464dc75c835413d99e831bc302e269d176a103332c2c79 (image=quay.io/prometheus/alertmanager:v0.27.0, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-16T09:27:11.325 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Sep 16 09:27:11 smithi070 podman[85821]: 2024-09-16 09:27:11.043880781 +0000 UTC m=+1.356037501 container start 3da3e64aa0d999725b464dc75c835413d99e831bc302e269d176a103332c2c79 (image=quay.io/prometheus/alertmanager:v0.27.0, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-16T09:27:11.325 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Sep 16 09:27:11 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-alertmanager-a[85965]: ts=2024-09-16T09:27:11.072Z caller=main.go:181 level=info msg="Starting Alertmanager" version="(version=0.27.0, branch=HEAD, revision=0aa3c2aad14cff039931923ab16b26b7481783b5)" 2024-09-16T09:27:11.326 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Sep 16 09:27:11 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-alertmanager-a[85965]: ts=2024-09-16T09:27:11.072Z caller=main.go:182 level=info build_context="(go=go1.21.7, platform=linux/amd64, user=root@22cd11f671e9, date=20240228-11:51:20, tags=netgo)" 2024-09-16T09:27:11.326 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Sep 16 09:27:11 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-alertmanager-a[85965]: ts=2024-09-16T09:27:11.077Z caller=cluster.go:186 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.70 port=9094 2024-09-16T09:27:11.326 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Sep 16 09:27:11 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-alertmanager-a[85965]: ts=2024-09-16T09:27:11.079Z caller=cluster.go:683 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-09-16T09:27:11.326 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Sep 16 09:27:11 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-alertmanager-a[85965]: ts=2024-09-16T09:27:11.126Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-16T09:27:11.326 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Sep 16 09:27:11 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-alertmanager-a[85965]: ts=2024-09-16T09:27:11.126Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-16T09:27:11.326 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Sep 16 09:27:11 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-alertmanager-a[85965]: ts=2024-09-16T09:27:11.129Z caller=tls_config.go:313 level=info msg="Listening on" address=[::]:9093 2024-09-16T09:27:11.327 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Sep 16 09:27:11 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-alertmanager-a[85965]: ts=2024-09-16T09:27:11.129Z caller=tls_config.go:316 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-09-16T09:27:11.327 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Sep 16 09:27:11 smithi070 bash[85821]: 3da3e64aa0d999725b464dc75c835413d99e831bc302e269d176a103332c2c79 2024-09-16T09:27:11.327 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Sep 16 09:27:11 smithi070 systemd[1]: Started Ceph alertmanager.a for a1321834-740c-11ef-bceb-c7b262605968. 2024-09-16T09:27:11.351 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:27:11.380 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:11.380 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:11.381 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:27:11.482 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:27:11.521 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:11.522 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:11.522 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:27:11.623 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:27:11.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:11 smithi084 ceph-mon[32629]: pgmap v85: 158 pgs: 158 active+clean; 5.4 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 853 B/s rd, 59 MiB/s wr, 947 op/s 2024-09-16T09:27:11.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:11 smithi084 ceph-mon[32629]: 4.19 scrub starts 2024-09-16T09:27:11.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:11 smithi084 ceph-mon[32629]: 4.19 scrub ok 2024-09-16T09:27:11.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:11 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:27:11.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:11 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:27:11.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:11 smithi084 ceph-mon[32629]: Reconfiguring prometheus.a (dependencies changed)... 2024-09-16T09:27:11.663 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:11.664 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:11.664 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:27:11.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:11 smithi070 ceph-mon[30388]: pgmap v85: 158 pgs: 158 active+clean; 5.4 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 853 B/s rd, 59 MiB/s wr, 947 op/s 2024-09-16T09:27:11.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:11 smithi070 ceph-mon[30388]: 4.19 scrub starts 2024-09-16T09:27:11.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:11 smithi070 ceph-mon[30388]: 4.19 scrub ok 2024-09-16T09:27:11.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:11 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:27:11.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:11 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:27:11.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:11 smithi070 ceph-mon[30388]: Reconfiguring prometheus.a (dependencies changed)... 2024-09-16T09:27:11.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:11 smithi070 ceph-mon[40691]: pgmap v85: 158 pgs: 158 active+clean; 5.4 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 853 B/s rd, 59 MiB/s wr, 947 op/s 2024-09-16T09:27:11.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:11 smithi070 ceph-mon[40691]: 4.19 scrub starts 2024-09-16T09:27:11.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:11 smithi070 ceph-mon[40691]: 4.19 scrub ok 2024-09-16T09:27:11.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:11 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:27:11.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:11 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:27:11.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:11 smithi070 ceph-mon[40691]: Reconfiguring prometheus.a (dependencies changed)... 2024-09-16T09:27:11.738 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.3 dump_blocked_ops 2024-09-16T09:27:11.765 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:27:11.822 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:11.823 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:11.823 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:27:11.924 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:27:11.991 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:11.992 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:11.993 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:27:12.019 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:27:12.028 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 59 16 93194 93178 98.6623 65.0625 0.0251625 0.0101293 2024-09-16T09:27:12.030 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:27:12.094 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:27:12.122 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:12.123 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:12.123 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:27:12.171 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:27:11 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:27:11] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:27:12.224 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:27:12.271 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:12.271 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:12.272 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:27:12.372 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:27:12.414 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:12.414 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:12.415 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:27:12.476 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:12 smithi070 ceph-mon[30388]: 5.b scrub starts 2024-09-16T09:27:12.476 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:12 smithi070 ceph-mon[30388]: 5.b scrub ok 2024-09-16T09:27:12.476 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:12 smithi070 ceph-mon[30388]: Reconfiguring daemon prometheus.a on smithi084 2024-09-16T09:27:12.477 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:12 smithi070 ceph-mon[30388]: pgmap v86: 158 pgs: 158 active+clean; 5.4 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 692 B/s rd, 48 MiB/s wr, 769 op/s 2024-09-16T09:27:12.477 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:12 smithi070 ceph-mon[30388]: 6.1a scrub starts 2024-09-16T09:27:12.477 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:12 smithi070 ceph-mon[30388]: 6.1a scrub ok 2024-09-16T09:27:12.477 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:12 smithi070 ceph-mon[40691]: 5.b scrub starts 2024-09-16T09:27:12.477 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:12 smithi070 ceph-mon[40691]: 5.b scrub ok 2024-09-16T09:27:12.477 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:12 smithi070 ceph-mon[40691]: Reconfiguring daemon prometheus.a on smithi084 2024-09-16T09:27:12.477 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:12 smithi070 ceph-mon[40691]: pgmap v86: 158 pgs: 158 active+clean; 5.4 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 692 B/s rd, 48 MiB/s wr, 769 op/s 2024-09-16T09:27:12.477 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:12 smithi070 ceph-mon[40691]: 6.1a scrub starts 2024-09-16T09:27:12.477 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:12 smithi070 ceph-mon[40691]: 6.1a scrub ok 2024-09-16T09:27:12.492 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:12 smithi084 ceph-mon[32629]: 5.b scrub starts 2024-09-16T09:27:12.492 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:12 smithi084 ceph-mon[32629]: 5.b scrub ok 2024-09-16T09:27:12.492 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:12 smithi084 ceph-mon[32629]: Reconfiguring daemon prometheus.a on smithi084 2024-09-16T09:27:12.493 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:12 smithi084 ceph-mon[32629]: pgmap v86: 158 pgs: 158 active+clean; 5.4 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 692 B/s rd, 48 MiB/s wr, 769 op/s 2024-09-16T09:27:12.493 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:12 smithi084 ceph-mon[32629]: 6.1a scrub starts 2024-09-16T09:27:12.493 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:12 smithi084 ceph-mon[32629]: 6.1a scrub ok 2024-09-16T09:27:12.515 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:27:12.555 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:12.555 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:12.556 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:27:12.657 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:27:12.699 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:12.700 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:12.701 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:27:12.801 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:27:12.830 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:12.831 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:12.832 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:27:12.909 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:27:12 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:27:12.933 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:27:12.959 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:12.959 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:12.960 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:27:13.027 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:27:13.027 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:27:13.028 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:2024-09-16T09:27:13.028303+0000 min lat: 0.0012412 max lat: 0.156312 avg lat: 0.0101881 2024-09-16T09:27:13.028 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-16T09:27:13.028 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 60 16 94243 94227 98.1107 65.5625 0.0147028 0.0101881 2024-09-16T09:27:13.041 INFO:teuthology.orchestra.run.smithi070.stderr:no valid command found; 10 closest matches: 2024-09-16T09:27:13.042 INFO:teuthology.orchestra.run.smithi070.stderr:0 2024-09-16T09:27:13.042 INFO:teuthology.orchestra.run.smithi070.stderr:1 2024-09-16T09:27:13.042 INFO:teuthology.orchestra.run.smithi070.stderr:2 2024-09-16T09:27:13.042 INFO:teuthology.orchestra.run.smithi070.stderr:abort 2024-09-16T09:27:13.042 INFO:teuthology.orchestra.run.smithi070.stderr:assert 2024-09-16T09:27:13.042 INFO:teuthology.orchestra.run.smithi070.stderr:bench [] [] [] [] 2024-09-16T09:27:13.042 INFO:teuthology.orchestra.run.smithi070.stderr:bluefs debug_inject_read_zeros 2024-09-16T09:27:13.042 INFO:teuthology.orchestra.run.smithi070.stderr:bluefs files list 2024-09-16T09:27:13.042 INFO:teuthology.orchestra.run.smithi070.stderr:bluefs stats 2024-09-16T09:27:13.042 INFO:teuthology.orchestra.run.smithi070.stderr:bluestore allocator dump block 2024-09-16T09:27:13.042 INFO:teuthology.orchestra.run.smithi070.stderr:Error EINVAL: invalid command 2024-09-16T09:27:13.061 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:27:13.064 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:27:13.064 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:27:13.089 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:13.090 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:13.090 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:27:13.100 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:27:13.100 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:27:13.137 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:27:13.137 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:27:13.170 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:27:13.171 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:27:13.190 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:27:13.205 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:27:13.206 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:27:13.231 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:13.232 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:13.232 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:27:13.237 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:27:13.238 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:27:13.273 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:27:13.274 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:27:13.333 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:27:13.346 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Sep 16 09:27:13 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-alertmanager-a[85965]: ts=2024-09-16T09:27:13.079Z caller=cluster.go:708 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000117648s 2024-09-16T09:27:13.346 INFO:teuthology.orchestra.run.smithi070.stderr:nodeep-scrub is unset 2024-09-16T09:27:13.360 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:13.361 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:13.362 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:27:13.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:13 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1391800332' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:27:13.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:13 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:27:13.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:13 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:27:13.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:13 smithi084 ceph-mon[32629]: 3.12 scrub starts 2024-09-16T09:27:13.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:13 smithi084 ceph-mon[32629]: 3.12 scrub ok 2024-09-16T09:27:13.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:13 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3212824024' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:27:13.463 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:27:13.499 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:13.500 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:13.500 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:27:13.601 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:27:13.643 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:13 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1391800332' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:27:13.643 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:13 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:27:13.644 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:13 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:27:13.644 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:13 smithi070 ceph-mon[30388]: 3.12 scrub starts 2024-09-16T09:27:13.644 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:13 smithi070 ceph-mon[30388]: 3.12 scrub ok 2024-09-16T09:27:13.644 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:13 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3212824024' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:27:13.644 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:13 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1391800332' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:27:13.644 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:13 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:27:13.644 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:13 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:27:13.644 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:13 smithi070 ceph-mon[40691]: 3.12 scrub starts 2024-09-16T09:27:13.644 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:13 smithi070 ceph-mon[40691]: 3.12 scrub ok 2024-09-16T09:27:13.644 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:13 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3212824024' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:27:13.645 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:13.646 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:13.646 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:27:13.747 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:27:13.776 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:13.776 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:13.777 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:27:13.878 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:27:13.905 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:13.905 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:13.906 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:27:14.007 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:27:14.028 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 61 16 95407 95391 97.695 72.75 0.00703916 0.010231 2024-09-16T09:27:14.047 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:14.048 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:14.048 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:27:14.149 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:27:14.176 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:14.177 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:14.177 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:27:14.278 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:27:14.318 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:14.318 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:14.319 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:27:14.420 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:27:14.445 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:14.446 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:14.446 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:27:14.547 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:27:14.574 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:14.575 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:14.575 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:27:14.599 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:14 smithi084 ceph-mon[32629]: 5.11 scrub starts 2024-09-16T09:27:14.599 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:14 smithi084 ceph-mon[32629]: 5.11 scrub ok 2024-09-16T09:27:14.599 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:14 smithi084 ceph-mon[32629]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-16T09:27:14.599 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:14 smithi084 ceph-mon[32629]: Cluster is now healthy 2024-09-16T09:27:14.599 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:14 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-16T09:27:14.600 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:14 smithi084 ceph-mon[32629]: osdmap e107: 8 total, 8 up, 8 in 2024-09-16T09:27:14.600 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:14 smithi084 ceph-mon[32629]: 5.15 scrub starts 2024-09-16T09:27:14.600 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:14 smithi084 ceph-mon[32629]: 5.15 scrub ok 2024-09-16T09:27:14.600 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:14 smithi084 ceph-mon[32629]: pgmap v88: 158 pgs: 158 active+clean; 5.7 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 86 MiB/s wr, 1.38k op/s 2024-09-16T09:27:14.600 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:14 smithi084 ceph-mon[32629]: 6.2 scrub starts 2024-09-16T09:27:14.600 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:14 smithi084 ceph-mon[32629]: 6.2 scrub ok 2024-09-16T09:27:14.600 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:14 smithi084 ceph-mon[32629]: osdmap e108: 8 total, 8 up, 8 in 2024-09-16T09:27:14.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:14 smithi070 ceph-mon[30388]: 5.11 scrub starts 2024-09-16T09:27:14.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:14 smithi070 ceph-mon[30388]: 5.11 scrub ok 2024-09-16T09:27:14.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:14 smithi070 ceph-mon[30388]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-16T09:27:14.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:14 smithi070 ceph-mon[30388]: Cluster is now healthy 2024-09-16T09:27:14.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:14 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-16T09:27:14.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:14 smithi070 ceph-mon[30388]: osdmap e107: 8 total, 8 up, 8 in 2024-09-16T09:27:14.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:14 smithi070 ceph-mon[30388]: 5.15 scrub starts 2024-09-16T09:27:14.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:14 smithi070 ceph-mon[30388]: 5.15 scrub ok 2024-09-16T09:27:14.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:14 smithi070 ceph-mon[30388]: pgmap v88: 158 pgs: 158 active+clean; 5.7 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 86 MiB/s wr, 1.38k op/s 2024-09-16T09:27:14.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:14 smithi070 ceph-mon[30388]: 6.2 scrub starts 2024-09-16T09:27:14.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:14 smithi070 ceph-mon[30388]: 6.2 scrub ok 2024-09-16T09:27:14.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:14 smithi070 ceph-mon[30388]: osdmap e108: 8 total, 8 up, 8 in 2024-09-16T09:27:14.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:14 smithi070 ceph-mon[40691]: 5.11 scrub starts 2024-09-16T09:27:14.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:14 smithi070 ceph-mon[40691]: 5.11 scrub ok 2024-09-16T09:27:14.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:14 smithi070 ceph-mon[40691]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-16T09:27:14.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:14 smithi070 ceph-mon[40691]: Cluster is now healthy 2024-09-16T09:27:14.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:14 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-16T09:27:14.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:14 smithi070 ceph-mon[40691]: osdmap e107: 8 total, 8 up, 8 in 2024-09-16T09:27:14.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:14 smithi070 ceph-mon[40691]: 5.15 scrub starts 2024-09-16T09:27:14.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:14 smithi070 ceph-mon[40691]: 5.15 scrub ok 2024-09-16T09:27:14.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:14 smithi070 ceph-mon[40691]: pgmap v88: 158 pgs: 158 active+clean; 5.7 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 86 MiB/s wr, 1.38k op/s 2024-09-16T09:27:14.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:14 smithi070 ceph-mon[40691]: 6.2 scrub starts 2024-09-16T09:27:14.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:14 smithi070 ceph-mon[40691]: 6.2 scrub ok 2024-09-16T09:27:14.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:14 smithi070 ceph-mon[40691]: osdmap e108: 8 total, 8 up, 8 in 2024-09-16T09:27:14.676 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:27:14.717 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:14.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:14.718 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:27:14.819 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:27:14.853 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:14.854 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:14.854 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:27:14.955 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:27:14.976 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:14.977 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:14.977 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:27:15.028 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 62 16 96456 96440 97.1768 65.5625 0.00699272 0.0102853 2024-09-16T09:27:15.078 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:27:15.106 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:15.107 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:15.107 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:27:15.208 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:27:15.235 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:15.236 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:15.236 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:27:15.337 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:27:15.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:15.366 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:15.366 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:27:15.467 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:27:15.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:15.513 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:15.513 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:27:15.614 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:27:15.629 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:15 smithi070 ceph-mon[30388]: 7.e scrub starts 2024-09-16T09:27:15.629 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:15 smithi070 ceph-mon[30388]: 4.5 scrub starts 2024-09-16T09:27:15.629 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:15 smithi070 ceph-mon[30388]: 4.5 scrub ok 2024-09-16T09:27:15.629 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:15 smithi070 ceph-mon[30388]: 3.6 scrub starts 2024-09-16T09:27:15.629 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:15 smithi070 ceph-mon[30388]: 3.6 scrub ok 2024-09-16T09:27:15.629 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:15 smithi070 ceph-mon[40691]: 7.e scrub starts 2024-09-16T09:27:15.629 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:15 smithi070 ceph-mon[40691]: 4.5 scrub starts 2024-09-16T09:27:15.629 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:15 smithi070 ceph-mon[40691]: 4.5 scrub ok 2024-09-16T09:27:15.630 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:15 smithi070 ceph-mon[40691]: 3.6 scrub starts 2024-09-16T09:27:15.630 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:15 smithi070 ceph-mon[40691]: 3.6 scrub ok 2024-09-16T09:27:15.645 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:15.645 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:15.646 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:27:15.660 INFO:journalctl@ceph.prometheus.a.smithi084.stdout:Sep 16 09:27:15 smithi084 systemd[1]: Stopping Ceph prometheus.a for a1321834-740c-11ef-bceb-c7b262605968... 2024-09-16T09:27:15.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:15 smithi084 ceph-mon[32629]: 7.e scrub starts 2024-09-16T09:27:15.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:15 smithi084 ceph-mon[32629]: 4.5 scrub starts 2024-09-16T09:27:15.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:15 smithi084 ceph-mon[32629]: 4.5 scrub ok 2024-09-16T09:27:15.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:15 smithi084 ceph-mon[32629]: 3.6 scrub starts 2024-09-16T09:27:15.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:15 smithi084 ceph-mon[32629]: 3.6 scrub ok 2024-09-16T09:27:15.700 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-16T09:27:15.700 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-16T09:27:15.700 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph tell osd.4 'trim stale osdmaps' 2024-09-16T09:27:15.707 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:27:15.746 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:27:15.772 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:15.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:15.774 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:27:15.875 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:27:15.932 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:15.933 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:15.933 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:27:16.029 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 63 16 97484 97468 96.6543 64.25 0.0129428 0.0103409 2024-09-16T09:27:16.035 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:27:16.098 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:16.105 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:16.106 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:27:16.160 INFO:journalctl@ceph.prometheus.a.smithi084.stdout:Sep 16 09:27:15 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-prometheus-a[61406]: ts=2024-09-16T09:27:15.862Z caller=main.go:964 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-09-16T09:27:16.160 INFO:journalctl@ceph.prometheus.a.smithi084.stdout:Sep 16 09:27:15 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-prometheus-a[61406]: ts=2024-09-16T09:27:15.862Z caller=main.go:988 level=info msg="Stopping scrape discovery manager..." 2024-09-16T09:27:16.161 INFO:journalctl@ceph.prometheus.a.smithi084.stdout:Sep 16 09:27:15 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-prometheus-a[61406]: ts=2024-09-16T09:27:15.862Z caller=main.go:1002 level=info msg="Stopping notify discovery manager..." 2024-09-16T09:27:16.161 INFO:journalctl@ceph.prometheus.a.smithi084.stdout:Sep 16 09:27:15 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-prometheus-a[61406]: ts=2024-09-16T09:27:15.862Z caller=manager.go:177 level=info component="rule manager" msg="Stopping rule manager..." 2024-09-16T09:27:16.161 INFO:journalctl@ceph.prometheus.a.smithi084.stdout:Sep 16 09:27:15 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-prometheus-a[61406]: ts=2024-09-16T09:27:15.862Z caller=main.go:984 level=info msg="Scrape discovery manager stopped" 2024-09-16T09:27:16.161 INFO:journalctl@ceph.prometheus.a.smithi084.stdout:Sep 16 09:27:15 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-prometheus-a[61406]: ts=2024-09-16T09:27:15.862Z caller=main.go:998 level=info msg="Notify discovery manager stopped" 2024-09-16T09:27:16.161 INFO:journalctl@ceph.prometheus.a.smithi084.stdout:Sep 16 09:27:15 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-prometheus-a[61406]: ts=2024-09-16T09:27:15.862Z caller=manager.go:187 level=info component="rule manager" msg="Rule manager stopped" 2024-09-16T09:27:16.161 INFO:journalctl@ceph.prometheus.a.smithi084.stdout:Sep 16 09:27:15 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-prometheus-a[61406]: ts=2024-09-16T09:27:15.862Z caller=main.go:1039 level=info msg="Stopping scrape manager..." 2024-09-16T09:27:16.161 INFO:journalctl@ceph.prometheus.a.smithi084.stdout:Sep 16 09:27:15 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-prometheus-a[61406]: ts=2024-09-16T09:27:15.862Z caller=main.go:1031 level=info msg="Scrape manager stopped" 2024-09-16T09:27:16.161 INFO:journalctl@ceph.prometheus.a.smithi084.stdout:Sep 16 09:27:15 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-prometheus-a[61406]: ts=2024-09-16T09:27:15.863Z caller=notifier.go:618 level=info component=notifier msg="Stopping notification manager..." 2024-09-16T09:27:16.161 INFO:journalctl@ceph.prometheus.a.smithi084.stdout:Sep 16 09:27:15 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-prometheus-a[61406]: ts=2024-09-16T09:27:15.863Z caller=main.go:1261 level=info msg="Notifier manager stopped" 2024-09-16T09:27:16.161 INFO:journalctl@ceph.prometheus.a.smithi084.stdout:Sep 16 09:27:15 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-prometheus-a[61406]: ts=2024-09-16T09:27:15.863Z caller=main.go:1273 level=info msg="See you next time!" 2024-09-16T09:27:16.161 INFO:journalctl@ceph.prometheus.a.smithi084.stdout:Sep 16 09:27:15 smithi084 podman[68588]: 2024-09-16 09:27:15.872987619 +0000 UTC m=+0.149022192 container died 4034d5ab5c22e3e89ae9028061761a10bdd8e8f976d568ff403087a1f3e2bbc6 (image=quay.io/prometheus/prometheus:v2.51.0, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-16T09:27:16.207 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:27:16.249 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:16.249 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:16.250 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:27:16.350 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:27:16.360 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T09:27:16.361 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:27:16.409 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:16.410 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:16.410 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:27:16.491 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T09:27:16.492 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:16 smithi084 ceph-mon[32629]: 7.0 scrub starts 2024-09-16T09:27:16.492 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:16 smithi084 ceph-mon[32629]: pgmap v90: 158 pgs: 158 active+clean; 5.7 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 800 B/s rd, 52 MiB/s wr, 834 op/s 2024-09-16T09:27:16.493 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:16 smithi084 ceph-mon[32629]: 7.8 scrub starts 2024-09-16T09:27:16.493 INFO:journalctl@ceph.prometheus.a.smithi084.stdout:Sep 16 09:27:16 smithi084 podman[68588]: 2024-09-16 09:27:16.266734231 +0000 UTC m=+0.542768810 container cleanup 4034d5ab5c22e3e89ae9028061761a10bdd8e8f976d568ff403087a1f3e2bbc6 (image=quay.io/prometheus/prometheus:v2.51.0, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-16T09:27:16.493 INFO:journalctl@ceph.prometheus.a.smithi084.stdout:Sep 16 09:27:16 smithi084 bash[68588]: ceph-a1321834-740c-11ef-bceb-c7b262605968-prometheus-a 2024-09-16T09:27:16.512 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:27:16.536 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:16 smithi070 ceph-mon[30388]: 7.0 scrub starts 2024-09-16T09:27:16.536 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:16 smithi070 ceph-mon[30388]: pgmap v90: 158 pgs: 158 active+clean; 5.7 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 800 B/s rd, 52 MiB/s wr, 834 op/s 2024-09-16T09:27:16.536 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:16 smithi070 ceph-mon[30388]: 7.8 scrub starts 2024-09-16T09:27:16.537 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:16 smithi070 ceph-mon[40691]: 7.0 scrub starts 2024-09-16T09:27:16.537 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:16 smithi070 ceph-mon[40691]: pgmap v90: 158 pgs: 158 active+clean; 5.7 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 800 B/s rd, 52 MiB/s wr, 834 op/s 2024-09-16T09:27:16.537 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:16 smithi070 ceph-mon[40691]: 7.8 scrub starts 2024-09-16T09:27:16.589 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:16.621 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:16.621 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:27:16.626 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:27:16.722 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:27:16.758 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:16.759 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:16.759 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:27:16.760 INFO:journalctl@ceph.prometheus.a.smithi084.stdout:Sep 16 09:27:16 smithi084 podman[68620]: 2024-09-16 09:27:16.560352734 +0000 UTC m=+0.688231892 container remove 4034d5ab5c22e3e89ae9028061761a10bdd8e8f976d568ff403087a1f3e2bbc6 (image=quay.io/prometheus/prometheus:v2.51.0, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-16T09:27:16.861 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:27:16.900 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:16.901 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:16.901 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:27:17.002 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:27:17.029 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 64 15 98434 98419 96.0729 59.4375 0.0178051 0.0104032 2024-09-16T09:27:17.045 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:17.046 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:17.047 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:27:17.075 INFO:journalctl@ceph.prometheus.a.smithi084.stdout:Sep 16 09:27:16 smithi084 systemd[1]: ceph-a1321834-740c-11ef-bceb-c7b262605968@prometheus.a.service: Deactivated successfully. 2024-09-16T09:27:17.076 INFO:journalctl@ceph.prometheus.a.smithi084.stdout:Sep 16 09:27:16 smithi084 systemd[1]: Stopped Ceph prometheus.a for a1321834-740c-11ef-bceb-c7b262605968. 2024-09-16T09:27:17.076 INFO:journalctl@ceph.prometheus.a.smithi084.stdout:Sep 16 09:27:16 smithi084 systemd[1]: ceph-a1321834-740c-11ef-bceb-c7b262605968@prometheus.a.service: Consumed 1.333s CPU time. 2024-09-16T09:27:17.076 INFO:journalctl@ceph.prometheus.a.smithi084.stdout:Sep 16 09:27:16 smithi084 systemd[1]: Starting Ceph prometheus.a for a1321834-740c-11ef-bceb-c7b262605968... 2024-09-16T09:27:17.149 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:27:17.158 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.3 dump_historic_ops 2024-09-16T09:27:17.194 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:17.194 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:17.195 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:27:17.295 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:27:17.341 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:17.341 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:17.341 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:27:17.374 INFO:journalctl@ceph.prometheus.a.smithi084.stdout:Sep 16 09:27:17 smithi084 podman[68771]: 2024-09-16 09:27:16.986481533 +0000 UTC m=+0.033500154 image pull 1d3b7f56885b6dd623f1785be963aa9c195f86bc256ea454e8d02a7980b79c53 quay.io/prometheus/prometheus:v2.51.0 2024-09-16T09:27:17.374 INFO:journalctl@ceph.prometheus.a.smithi084.stdout:Sep 16 09:27:17 smithi084 podman[68771]: 2024-09-16 09:27:17.245174883 +0000 UTC m=+0.292193487 container create 2d908358b02da43f2bced9837e86c8b789e5e15bc975249569f0cc85980f08dd (image=quay.io/prometheus/prometheus:v2.51.0, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-16T09:27:17.442 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:27:17.471 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:17.472 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:17.472 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:27:17.502 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd set noscrub 2024-09-16T09:27:17.573 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:27:17.646 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:17.647 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:17.647 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:27:17.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:17 smithi084 ceph-mon[32629]: 7.e scrub ok 2024-09-16T09:27:17.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:17 smithi084 ceph-mon[32629]: 5.1d scrub starts 2024-09-16T09:27:17.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:17 smithi084 ceph-mon[32629]: 5.1d scrub ok 2024-09-16T09:27:17.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:17 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:27:17.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:17 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:27:17.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:17 smithi084 ceph-mon[32629]: osdmap e109: 8 total, 8 up, 8 in 2024-09-16T09:27:17.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:17 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:27:17.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:17 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:27:17.661 INFO:journalctl@ceph.prometheus.a.smithi084.stdout:Sep 16 09:27:17 smithi084 podman[68771]: 2024-09-16 09:27:17.499482811 +0000 UTC m=+0.546501415 container init 2d908358b02da43f2bced9837e86c8b789e5e15bc975249569f0cc85980f08dd (image=quay.io/prometheus/prometheus:v2.51.0, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-16T09:27:17.661 INFO:journalctl@ceph.prometheus.a.smithi084.stdout:Sep 16 09:27:17 smithi084 podman[68771]: 2024-09-16 09:27:17.506874862 +0000 UTC m=+0.553893462 container start 2d908358b02da43f2bced9837e86c8b789e5e15bc975249569f0cc85980f08dd (image=quay.io/prometheus/prometheus:v2.51.0, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-16T09:27:17.661 INFO:journalctl@ceph.prometheus.a.smithi084.stdout:Sep 16 09:27:17 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-prometheus-a[68785]: ts=2024-09-16T09:27:17.538Z caller=main.go:617 level=info msg="Starting Prometheus Server" mode=server version="(version=2.51.0, branch=HEAD, revision=c05c15512acb675e3f6cd662a6727854e93fc024)" 2024-09-16T09:27:17.661 INFO:journalctl@ceph.prometheus.a.smithi084.stdout:Sep 16 09:27:17 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-prometheus-a[68785]: ts=2024-09-16T09:27:17.538Z caller=main.go:622 level=info build_context="(go=go1.22.1, platform=linux/amd64, user=root@b5723e458358, date=20240319-10:54:45, tags=netgo,builtinassets,stringlabels)" 2024-09-16T09:27:17.661 INFO:journalctl@ceph.prometheus.a.smithi084.stdout:Sep 16 09:27:17 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-prometheus-a[68785]: ts=2024-09-16T09:27:17.538Z caller=main.go:623 level=info host_details="(Linux 5.14.0-505.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Sep 5 07:54:07 UTC 2024 x86_64 smithi084 (none))" 2024-09-16T09:27:17.661 INFO:journalctl@ceph.prometheus.a.smithi084.stdout:Sep 16 09:27:17 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-prometheus-a[68785]: ts=2024-09-16T09:27:17.538Z caller=main.go:624 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-09-16T09:27:17.661 INFO:journalctl@ceph.prometheus.a.smithi084.stdout:Sep 16 09:27:17 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-prometheus-a[68785]: ts=2024-09-16T09:27:17.538Z caller=main.go:625 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-09-16T09:27:17.662 INFO:journalctl@ceph.prometheus.a.smithi084.stdout:Sep 16 09:27:17 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-prometheus-a[68785]: ts=2024-09-16T09:27:17.540Z caller=web.go:568 level=info component=web msg="Start listening for connections" address=:9095 2024-09-16T09:27:17.662 INFO:journalctl@ceph.prometheus.a.smithi084.stdout:Sep 16 09:27:17 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-prometheus-a[68785]: ts=2024-09-16T09:27:17.541Z caller=main.go:1129 level=info msg="Starting TSDB ..." 2024-09-16T09:27:17.662 INFO:journalctl@ceph.prometheus.a.smithi084.stdout:Sep 16 09:27:17 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-prometheus-a[68785]: ts=2024-09-16T09:27:17.541Z caller=tls_config.go:313 level=info component=web msg="Listening on" address=[::]:9095 2024-09-16T09:27:17.662 INFO:journalctl@ceph.prometheus.a.smithi084.stdout:Sep 16 09:27:17 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-prometheus-a[68785]: ts=2024-09-16T09:27:17.541Z caller=tls_config.go:316 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-09-16T09:27:17.662 INFO:journalctl@ceph.prometheus.a.smithi084.stdout:Sep 16 09:27:17 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-prometheus-a[68785]: ts=2024-09-16T09:27:17.544Z caller=head.go:616 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-09-16T09:27:17.662 INFO:journalctl@ceph.prometheus.a.smithi084.stdout:Sep 16 09:27:17 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-prometheus-a[68785]: ts=2024-09-16T09:27:17.544Z caller=head.go:698 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=4.633µs 2024-09-16T09:27:17.662 INFO:journalctl@ceph.prometheus.a.smithi084.stdout:Sep 16 09:27:17 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-prometheus-a[68785]: ts=2024-09-16T09:27:17.544Z caller=head.go:706 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-09-16T09:27:17.662 INFO:journalctl@ceph.prometheus.a.smithi084.stdout:Sep 16 09:27:17 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-prometheus-a[68785]: ts=2024-09-16T09:27:17.545Z caller=head.go:778 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=1 2024-09-16T09:27:17.662 INFO:journalctl@ceph.prometheus.a.smithi084.stdout:Sep 16 09:27:17 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-prometheus-a[68785]: ts=2024-09-16T09:27:17.547Z caller=head.go:778 level=info component=tsdb msg="WAL segment loaded" segment=1 maxSegment=1 2024-09-16T09:27:17.662 INFO:journalctl@ceph.prometheus.a.smithi084.stdout:Sep 16 09:27:17 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-prometheus-a[68785]: ts=2024-09-16T09:27:17.547Z caller=head.go:815 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=46.639µs wal_replay_duration=3.014882ms wbl_replay_duration=121ns total_replay_duration=3.138993ms 2024-09-16T09:27:17.662 INFO:journalctl@ceph.prometheus.a.smithi084.stdout:Sep 16 09:27:17 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-prometheus-a[68785]: ts=2024-09-16T09:27:17.550Z caller=main.go:1150 level=info fs_type=XFS_SUPER_MAGIC 2024-09-16T09:27:17.662 INFO:journalctl@ceph.prometheus.a.smithi084.stdout:Sep 16 09:27:17 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-prometheus-a[68785]: ts=2024-09-16T09:27:17.550Z caller=main.go:1153 level=info msg="TSDB started" 2024-09-16T09:27:17.662 INFO:journalctl@ceph.prometheus.a.smithi084.stdout:Sep 16 09:27:17 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-prometheus-a[68785]: ts=2024-09-16T09:27:17.550Z caller=main.go:1335 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-09-16T09:27:17.662 INFO:journalctl@ceph.prometheus.a.smithi084.stdout:Sep 16 09:27:17 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-prometheus-a[68785]: ts=2024-09-16T09:27:17.573Z caller=main.go:1372 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=23.077178ms db_storage=905ns remote_storage=1.901µs web_handler=426ns query_engine=710ns scrape=207.496µs scrape_sd=116.023µs notify=9.179µs notify_sd=8.186µs rules=22.372149ms tracing=8.221µs 2024-09-16T09:27:17.663 INFO:journalctl@ceph.prometheus.a.smithi084.stdout:Sep 16 09:27:17 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-prometheus-a[68785]: ts=2024-09-16T09:27:17.573Z caller=main.go:1114 level=info msg="Server is ready to receive web requests." 2024-09-16T09:27:17.663 INFO:journalctl@ceph.prometheus.a.smithi084.stdout:Sep 16 09:27:17 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-prometheus-a[68785]: ts=2024-09-16T09:27:17.573Z caller=manager.go:163 level=info component="rule manager" msg="Starting rule manager..." 2024-09-16T09:27:17.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:17 smithi070 ceph-mon[30388]: 7.e scrub ok 2024-09-16T09:27:17.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:17 smithi070 ceph-mon[30388]: 5.1d scrub starts 2024-09-16T09:27:17.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:17 smithi070 ceph-mon[30388]: 5.1d scrub ok 2024-09-16T09:27:17.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:17 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:27:17.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:17 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:27:17.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:17 smithi070 ceph-mon[30388]: osdmap e109: 8 total, 8 up, 8 in 2024-09-16T09:27:17.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:17 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:27:17.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:17 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:27:17.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:17 smithi070 ceph-mon[40691]: 7.e scrub ok 2024-09-16T09:27:17.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:17 smithi070 ceph-mon[40691]: 5.1d scrub starts 2024-09-16T09:27:17.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:17 smithi070 ceph-mon[40691]: 5.1d scrub ok 2024-09-16T09:27:17.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:17 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:27:17.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:17 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:27:17.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:17 smithi070 ceph-mon[40691]: osdmap e109: 8 total, 8 up, 8 in 2024-09-16T09:27:17.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:17 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:27:17.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:17 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:27:17.747 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:27:17.802 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:27:17.806 INFO:teuthology.orchestra.run.smithi070.stderr:no valid command found; 10 closest matches: 2024-09-16T09:27:17.807 INFO:teuthology.orchestra.run.smithi070.stderr:0 2024-09-16T09:27:17.807 INFO:teuthology.orchestra.run.smithi070.stderr:1 2024-09-16T09:27:17.807 INFO:teuthology.orchestra.run.smithi070.stderr:2 2024-09-16T09:27:17.807 INFO:teuthology.orchestra.run.smithi070.stderr:abort 2024-09-16T09:27:17.807 INFO:teuthology.orchestra.run.smithi070.stderr:assert 2024-09-16T09:27:17.807 INFO:teuthology.orchestra.run.smithi070.stderr:bench [] [] [] [] 2024-09-16T09:27:17.807 INFO:teuthology.orchestra.run.smithi070.stderr:bluefs debug_inject_read_zeros 2024-09-16T09:27:17.807 INFO:teuthology.orchestra.run.smithi070.stderr:bluefs files list 2024-09-16T09:27:17.807 INFO:teuthology.orchestra.run.smithi070.stderr:bluefs stats 2024-09-16T09:27:17.807 INFO:teuthology.orchestra.run.smithi070.stderr:bluestore allocator dump block 2024-09-16T09:27:17.807 INFO:teuthology.orchestra.run.smithi070.stderr:Error EINVAL: invalid command 2024-09-16T09:27:17.842 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:17.842 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:17.843 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:27:17.944 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:27:17.975 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:17.975 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:17.976 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:27:18.029 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 65 16 99340 99324 95.4651 56.5625 0.0099592 0.01047 2024-09-16T09:27:18.076 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:27:18.106 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:18.107 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:18.107 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:27:18.208 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:27:18.246 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:18.247 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:18.248 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:27:18.348 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:27:18.398 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:18.399 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:18.399 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:27:18.500 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:27:18.526 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:18.527 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:18.527 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:27:18.628 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:27:18.660 INFO:journalctl@ceph.prometheus.a.smithi084.stdout:Sep 16 09:27:18 smithi084 bash[68771]: 2d908358b02da43f2bced9837e86c8b789e5e15bc975249569f0cc85980f08dd 2024-09-16T09:27:18.660 INFO:journalctl@ceph.prometheus.a.smithi084.stdout:Sep 16 09:27:18 smithi084 systemd[1]: Started Ceph prometheus.a for a1321834-740c-11ef-bceb-c7b262605968. 2024-09-16T09:27:18.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:18 smithi084 ceph-mon[32629]: 7.0 scrub ok 2024-09-16T09:27:18.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:18 smithi084 ceph-mon[32629]: pgmap v92: 158 pgs: 158 active+clean; 5.8 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.7 KiB/s rd, 75 MiB/s wr, 1.21k op/s 2024-09-16T09:27:18.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:18 smithi084 ceph-mon[32629]: 3.d scrub starts 2024-09-16T09:27:18.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:18 smithi084 ceph-mon[32629]: 3.d scrub ok 2024-09-16T09:27:18.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:18 smithi084 ceph-mon[32629]: 5.3 scrub starts 2024-09-16T09:27:18.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:18 smithi084 ceph-mon[32629]: 5.3 scrub ok 2024-09-16T09:27:18.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:18 smithi084 ceph-mon[32629]: 7.8 scrub ok 2024-09-16T09:27:18.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:18 smithi084 ceph-mon[32629]: 6.f scrub starts 2024-09-16T09:27:18.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:18 smithi084 ceph-mon[32629]: 6.f scrub ok 2024-09-16T09:27:18.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:18 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:27:18.662 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:18 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:27:18.662 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:18 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "dashboard get-alertmanager-api-host"} : dispatch 2024-09-16T09:27:18.662 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:18 smithi084 ceph-mon[32629]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-16T09:27:18.662 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:18 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi070.front.sepia.ceph.com:9093"} : dispatch 2024-09-16T09:27:18.662 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:18 smithi084 ceph-mon[32629]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi070.front.sepia.ceph.com:9093"}]: dispatch 2024-09-16T09:27:18.662 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:18 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:27:18.662 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:18 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "dashboard get-prometheus-api-host"} : dispatch 2024-09-16T09:27:18.662 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:18 smithi084 ceph-mon[32629]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-16T09:27:18.662 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:18 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi084.front.sepia.ceph.com:9095"} : dispatch 2024-09-16T09:27:18.662 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:18 smithi084 ceph-mon[32629]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi084.front.sepia.ceph.com:9095"}]: dispatch 2024-09-16T09:27:18.662 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:18 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:27:18.662 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:18 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "dashboard get-grafana-api-url"} : dispatch 2024-09-16T09:27:18.662 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:18 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "dashboard set-grafana-api-url", "value": "https://smithi084.front.sepia.ceph.com:3000"} : dispatch 2024-09-16T09:27:18.662 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:18 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-16T09:27:18.663 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:18 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:27:18.674 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:18 smithi070 ceph-mon[30388]: 7.0 scrub ok 2024-09-16T09:27:18.674 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:18 smithi070 ceph-mon[30388]: pgmap v92: 158 pgs: 158 active+clean; 5.8 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.7 KiB/s rd, 75 MiB/s wr, 1.21k op/s 2024-09-16T09:27:18.674 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:18 smithi070 ceph-mon[30388]: 3.d scrub starts 2024-09-16T09:27:18.674 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:18 smithi070 ceph-mon[30388]: 3.d scrub ok 2024-09-16T09:27:18.674 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:18 smithi070 ceph-mon[30388]: 5.3 scrub starts 2024-09-16T09:27:18.674 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:18 smithi070 ceph-mon[30388]: 5.3 scrub ok 2024-09-16T09:27:18.675 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:18 smithi070 ceph-mon[30388]: 7.8 scrub ok 2024-09-16T09:27:18.675 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:18 smithi070 ceph-mon[30388]: 6.f scrub starts 2024-09-16T09:27:18.675 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:18 smithi070 ceph-mon[30388]: 6.f scrub ok 2024-09-16T09:27:18.675 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:18 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:27:18.675 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:18 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:27:18.675 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:18 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "dashboard get-alertmanager-api-host"} : dispatch 2024-09-16T09:27:18.675 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:18 smithi070 ceph-mon[30388]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-16T09:27:18.675 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:18 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi070.front.sepia.ceph.com:9093"} : dispatch 2024-09-16T09:27:18.675 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:18 smithi070 ceph-mon[30388]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi070.front.sepia.ceph.com:9093"}]: dispatch 2024-09-16T09:27:18.675 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:18 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:27:18.675 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:18 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "dashboard get-prometheus-api-host"} : dispatch 2024-09-16T09:27:18.675 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:18 smithi070 ceph-mon[30388]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-16T09:27:18.675 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:18 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi084.front.sepia.ceph.com:9095"} : dispatch 2024-09-16T09:27:18.675 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:18 smithi070 ceph-mon[30388]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi084.front.sepia.ceph.com:9095"}]: dispatch 2024-09-16T09:27:18.676 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:18 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:27:18.676 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:18 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "dashboard get-grafana-api-url"} : dispatch 2024-09-16T09:27:18.676 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:18 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "dashboard set-grafana-api-url", "value": "https://smithi084.front.sepia.ceph.com:3000"} : dispatch 2024-09-16T09:27:18.676 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:18 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-16T09:27:18.676 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:18 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:27:18.676 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:27:18 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: [16/Sep/2024:09:27:18] ENGINE Bus STOPPING 2024-09-16T09:27:18.676 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:27:18 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: [16/Sep/2024:09:27:18] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-16T09:27:18.676 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:27:18 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: [16/Sep/2024:09:27:18] ENGINE Bus STOPPED 2024-09-16T09:27:18.677 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:27:18 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: [16/Sep/2024:09:27:18] ENGINE Bus STARTING 2024-09-16T09:27:18.677 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:27:18 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: [16/Sep/2024:09:27:18] ENGINE Serving on http://:::9283 2024-09-16T09:27:18.677 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:27:18 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: [16/Sep/2024:09:27:18] ENGINE Bus STARTED 2024-09-16T09:27:18.677 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:27:18 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: [16/Sep/2024:09:27:18] ENGINE Bus STOPPING 2024-09-16T09:27:18.677 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:18 smithi070 ceph-mon[40691]: 7.0 scrub ok 2024-09-16T09:27:18.677 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:18 smithi070 ceph-mon[40691]: pgmap v92: 158 pgs: 158 active+clean; 5.8 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.7 KiB/s rd, 75 MiB/s wr, 1.21k op/s 2024-09-16T09:27:18.677 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:18 smithi070 ceph-mon[40691]: 3.d scrub starts 2024-09-16T09:27:18.677 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:18 smithi070 ceph-mon[40691]: 3.d scrub ok 2024-09-16T09:27:18.677 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:18 smithi070 ceph-mon[40691]: 5.3 scrub starts 2024-09-16T09:27:18.677 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:18 smithi070 ceph-mon[40691]: 5.3 scrub ok 2024-09-16T09:27:18.678 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:18 smithi070 ceph-mon[40691]: 7.8 scrub ok 2024-09-16T09:27:18.678 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:18 smithi070 ceph-mon[40691]: 6.f scrub starts 2024-09-16T09:27:18.678 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:18 smithi070 ceph-mon[40691]: 6.f scrub ok 2024-09-16T09:27:18.678 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:18 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:27:18.678 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:18 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:27:18.678 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:18 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "dashboard get-alertmanager-api-host"} : dispatch 2024-09-16T09:27:18.678 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:18 smithi070 ceph-mon[40691]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-16T09:27:18.678 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:18 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi070.front.sepia.ceph.com:9093"} : dispatch 2024-09-16T09:27:18.678 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:18 smithi070 ceph-mon[40691]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi070.front.sepia.ceph.com:9093"}]: dispatch 2024-09-16T09:27:18.678 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:18 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:27:18.678 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:18 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "dashboard get-prometheus-api-host"} : dispatch 2024-09-16T09:27:18.678 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:18 smithi070 ceph-mon[40691]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-16T09:27:18.678 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:18 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi084.front.sepia.ceph.com:9095"} : dispatch 2024-09-16T09:27:18.678 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:18 smithi070 ceph-mon[40691]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi084.front.sepia.ceph.com:9095"}]: dispatch 2024-09-16T09:27:18.679 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:18 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:27:18.679 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:18 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "dashboard get-grafana-api-url"} : dispatch 2024-09-16T09:27:18.679 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:18 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "dashboard set-grafana-api-url", "value": "https://smithi084.front.sepia.ceph.com:3000"} : dispatch 2024-09-16T09:27:18.679 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:18 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-16T09:27:18.679 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:18 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:27:18.680 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:18.680 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:18.680 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:27:18.765 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:27:18.782 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:27:18.876 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:18.877 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:18.877 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:27:18.978 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:27:19.008 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:19.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:19.009 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:27:19.029 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 66 16 100361 100345 94.9848 63.8125 0.0221996 0.0105231 2024-09-16T09:27:19.110 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:27:19.141 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:19.142 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:19.142 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:27:19.171 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:27:19 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: [16/Sep/2024:09:27:19] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-16T09:27:19.171 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:27:19 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: [16/Sep/2024:09:27:19] ENGINE Bus STOPPED 2024-09-16T09:27:19.171 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:27:19 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: [16/Sep/2024:09:27:19] ENGINE Bus STARTING 2024-09-16T09:27:19.171 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:27:19 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: [16/Sep/2024:09:27:19] ENGINE Serving on http://:::9283 2024-09-16T09:27:19.171 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:27:19 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: [16/Sep/2024:09:27:19] ENGINE Bus STARTED 2024-09-16T09:27:19.171 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:27:19 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: [16/Sep/2024:09:27:19] ENGINE Bus STOPPING 2024-09-16T09:27:19.242 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:27:19.272 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:19.272 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:19.273 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:27:19.373 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:27:19.405 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:19.406 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:19.406 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:27:19.455 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-16T09:27:19.456 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-16T09:27:19.456 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph tell osd.5 'trim stale osdmaps' 2024-09-16T09:27:19.506 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:27:19.532 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:19.533 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:19.534 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:27:19.579 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:27:19.622 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:19 smithi070 ceph-mon[40691]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-16T09:27:19.622 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:19 smithi070 ceph-mon[40691]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi084.front.sepia.ceph.com:3000"}]: dispatch 2024-09-16T09:27:19.622 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:19 smithi070 ceph-mon[40691]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-16T09:27:19.622 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:19 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:27:19.622 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:19 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-16T09:27:19.622 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:19 smithi070 ceph-mon[40691]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-16T09:27:19.622 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:19 smithi070 ceph-mon[40691]: osdmap e110: 8 total, 8 up, 8 in 2024-09-16T09:27:19.623 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:19 smithi070 ceph-mon[30388]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-16T09:27:19.623 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:19 smithi070 ceph-mon[30388]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi084.front.sepia.ceph.com:3000"}]: dispatch 2024-09-16T09:27:19.623 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:19 smithi070 ceph-mon[30388]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-16T09:27:19.623 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:19 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:27:19.623 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:19 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-16T09:27:19.623 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:19 smithi070 ceph-mon[30388]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-16T09:27:19.623 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:19 smithi070 ceph-mon[30388]: osdmap e110: 8 total, 8 up, 8 in 2024-09-16T09:27:19.623 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:27:19 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: [16/Sep/2024:09:27:19] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-16T09:27:19.623 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:27:19 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: [16/Sep/2024:09:27:19] ENGINE Bus STOPPED 2024-09-16T09:27:19.634 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:27:19.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:19 smithi084 ceph-mon[32629]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-16T09:27:19.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:19 smithi084 ceph-mon[32629]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi084.front.sepia.ceph.com:3000"}]: dispatch 2024-09-16T09:27:19.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:19 smithi084 ceph-mon[32629]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-16T09:27:19.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:19 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:27:19.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:19 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-16T09:27:19.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:19 smithi084 ceph-mon[32629]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-16T09:27:19.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:19 smithi084 ceph-mon[32629]: osdmap e110: 8 total, 8 up, 8 in 2024-09-16T09:27:19.687 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:19.688 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:19.688 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:27:19.789 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:27:19.817 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:19.817 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:19.818 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:27:19.920 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:27:19.921 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:27:19 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: [16/Sep/2024:09:27:19] ENGINE Bus STARTING 2024-09-16T09:27:19.921 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:27:19 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: [16/Sep/2024:09:27:19] ENGINE Serving on http://:::9283 2024-09-16T09:27:19.921 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:27:19 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: [16/Sep/2024:09:27:19] ENGINE Bus STARTED 2024-09-16T09:27:19.946 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:19.947 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:19.947 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:27:20.029 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 67 15 101443 101428 94.5774 67.6875 0.006598 0.010568 2024-09-16T09:27:20.048 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:27:20.077 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:20.077 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:20.078 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:27:20.179 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:27:20.213 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:20.214 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:20.214 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:27:20.315 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:27:20.342 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:20.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:20.343 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:27:20.444 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:27:20.479 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:20.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:20.481 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:27:20.581 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:27:20.616 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:20.616 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:20.617 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:27:20.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:20 smithi084 ceph-mon[32629]: pgmap v94: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 6.0 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 853 B/s rd, 54 MiB/s wr, 867 op/s 2024-09-16T09:27:20.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:20 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-16T09:27:20.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:20 smithi084 ceph-mon[32629]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-16T09:27:20.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:20 smithi084 ceph-mon[32629]: 5.a scrub starts 2024-09-16T09:27:20.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:20 smithi084 ceph-mon[32629]: 5.a scrub ok 2024-09-16T09:27:20.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:20 smithi070 ceph-mon[30388]: pgmap v94: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 6.0 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 853 B/s rd, 54 MiB/s wr, 867 op/s 2024-09-16T09:27:20.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:20 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-16T09:27:20.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:20 smithi070 ceph-mon[30388]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-16T09:27:20.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:20 smithi070 ceph-mon[30388]: 5.a scrub starts 2024-09-16T09:27:20.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:20 smithi070 ceph-mon[30388]: 5.a scrub ok 2024-09-16T09:27:20.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:20 smithi070 ceph-mon[40691]: pgmap v94: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 6.0 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 853 B/s rd, 54 MiB/s wr, 867 op/s 2024-09-16T09:27:20.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:20 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-16T09:27:20.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:20 smithi070 ceph-mon[40691]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-16T09:27:20.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:20 smithi070 ceph-mon[40691]: 5.a scrub starts 2024-09-16T09:27:20.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:20 smithi070 ceph-mon[40691]: 5.a scrub ok 2024-09-16T09:27:20.718 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:27:20.745 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:20.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:20.746 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:27:20.847 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:27:20.883 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:20.884 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:20.884 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:27:20.985 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:27:21.009 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:21.009 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:21.010 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:27:21.030 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 68 16 102475 102459 94.1341 64.4375 0.00935111 0.0106079 2024-09-16T09:27:21.110 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:27:21.144 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:21.145 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:21.145 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:27:21.246 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:27:21.274 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:21.274 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:21.275 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:27:21.376 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:27:21.409 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Sep 16 09:27:21 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-alertmanager-a[85965]: ts=2024-09-16T09:27:21.082Z caller=cluster.go:700 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.002654631s 2024-09-16T09:27:21.415 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:21.415 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:21.416 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:27:21.517 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:27:21.546 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:21.547 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:21.547 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:27:21.648 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:27:21.659 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:21 smithi084 ceph-mon[32629]: 6.a scrub starts 2024-09-16T09:27:21.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:21 smithi084 ceph-mon[32629]: 6.a scrub ok 2024-09-16T09:27:21.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:21 smithi084 ceph-mon[32629]: 7.b scrub starts 2024-09-16T09:27:21.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:21 smithi070 ceph-mon[30388]: 6.a scrub starts 2024-09-16T09:27:21.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:21 smithi070 ceph-mon[30388]: 6.a scrub ok 2024-09-16T09:27:21.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:21 smithi070 ceph-mon[30388]: 7.b scrub starts 2024-09-16T09:27:21.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:21 smithi070 ceph-mon[40691]: 6.a scrub starts 2024-09-16T09:27:21.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:21 smithi070 ceph-mon[40691]: 6.a scrub ok 2024-09-16T09:27:21.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:21 smithi070 ceph-mon[40691]: 7.b scrub starts 2024-09-16T09:27:21.679 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:21.679 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:21.679 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:27:21.780 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:27:21.812 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:21.813 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:21.813 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:27:21.914 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:27:21.951 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:21.951 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:21.951 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:27:22.030 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 69 16 103510 103494 93.7074 64.6875 0.0126228 0.0106627 2024-09-16T09:27:22.052 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:27:22.091 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:22.091 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:22.092 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:27:22.192 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:27:22.219 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:22.220 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:22.220 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:27:22.321 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:27:22.362 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:22.362 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:22.363 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:27:22.464 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:27:22.503 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:22.503 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:22.504 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:27:22.604 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:27:22.647 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:22.648 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:22.648 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:27:22.660 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:27:22 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:27:22.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:22 smithi084 ceph-mon[32629]: 4.11 scrub starts 2024-09-16T09:27:22.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:22 smithi084 ceph-mon[32629]: 4.11 scrub ok 2024-09-16T09:27:22.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:22 smithi084 ceph-mon[32629]: 6.7 scrub starts 2024-09-16T09:27:22.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:22 smithi084 ceph-mon[32629]: 6.7 scrub ok 2024-09-16T09:27:22.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:22 smithi084 ceph-mon[32629]: pgmap v95: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 6.0 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 692 B/s rd, 44 MiB/s wr, 703 op/s 2024-09-16T09:27:22.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:22 smithi084 ceph-mon[32629]: 6.10 scrub starts 2024-09-16T09:27:22.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:22 smithi084 ceph-mon[32629]: 6.10 scrub ok 2024-09-16T09:27:22.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:22 smithi070 ceph-mon[30388]: 4.11 scrub starts 2024-09-16T09:27:22.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:22 smithi070 ceph-mon[30388]: 4.11 scrub ok 2024-09-16T09:27:22.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:22 smithi070 ceph-mon[30388]: 6.7 scrub starts 2024-09-16T09:27:22.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:22 smithi070 ceph-mon[30388]: 6.7 scrub ok 2024-09-16T09:27:22.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:22 smithi070 ceph-mon[30388]: pgmap v95: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 6.0 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 692 B/s rd, 44 MiB/s wr, 703 op/s 2024-09-16T09:27:22.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:22 smithi070 ceph-mon[30388]: 6.10 scrub starts 2024-09-16T09:27:22.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:22 smithi070 ceph-mon[30388]: 6.10 scrub ok 2024-09-16T09:27:22.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:22 smithi070 ceph-mon[40691]: 4.11 scrub starts 2024-09-16T09:27:22.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:22 smithi070 ceph-mon[40691]: 4.11 scrub ok 2024-09-16T09:27:22.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:22 smithi070 ceph-mon[40691]: 6.7 scrub starts 2024-09-16T09:27:22.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:22 smithi070 ceph-mon[40691]: 6.7 scrub ok 2024-09-16T09:27:22.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:22 smithi070 ceph-mon[40691]: pgmap v95: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 6.0 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 692 B/s rd, 44 MiB/s wr, 703 op/s 2024-09-16T09:27:22.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:22 smithi070 ceph-mon[40691]: 6.10 scrub starts 2024-09-16T09:27:22.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:22 smithi070 ceph-mon[40691]: 6.10 scrub ok 2024-09-16T09:27:22.750 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:27:22.801 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:22.802 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:22.802 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:27:22.903 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:27:22.929 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:27:22.929 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:27:22.930 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:27:22.930 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:27:22.930 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:27:22.930 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:27:22.943 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:22.944 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:22.944 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:27:22.959 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:27:22.959 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:27:22.992 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:27:22.993 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:27:23.026 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:27:23.026 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:27:23.029 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 70 16 104584 104568 93.3277 67.125 0.00455156 0.0107095 2024-09-16T09:27:23.045 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:27:23.062 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:27:23.062 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:27:23.077 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:23.078 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:23.078 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:27:23.103 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:27:23.103 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:27:23.179 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:27:23.222 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T09:27:23.223 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 20, 2024-09-16T09:27:23.223 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 600, 2024-09-16T09:27:23.223 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [ 2024-09-16T09:27:23.223 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.223 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.24698.0:22904 7.3 7:c8205b43:::benchmark_data_smithi070_78438_object22903:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T09:27:23.223 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:26:21.573902+0000", 2024-09-16T09:27:23.223 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 61.642310670999997, 2024-09-16T09:27:23.223 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.083677513999999995, 2024-09-16T09:27:23.223 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:27:23.223 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:27:23.224 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:27:23.224 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:27:23.224 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.24698", 2024-09-16T09:27:23.224 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:27:23.224 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 22904 2024-09-16T09:27:23.224 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.224 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:27:23.224 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.224 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:27:23.224 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:21.573902+0000", 2024-09-16T09:27:23.224 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:27:23.224 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.224 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.224 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:27:23.225 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:21.573902+0000", 2024-09-16T09:27:23.225 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:27:23.225 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.225 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.225 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:27:23.225 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:21.573902+0000", 2024-09-16T09:27:23.225 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.2999999999999999e-07 2024-09-16T09:27:23.225 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.226 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.226 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:27:23.226 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:21.573929+0000", 2024-09-16T09:27:23.226 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.6424e-05 2024-09-16T09:27:23.226 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.226 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.226 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:27:23.226 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:21.573937+0000", 2024-09-16T09:27:23.226 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.9100000000000005e-06 2024-09-16T09:27:23.226 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.226 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.226 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:27:23.227 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:21.573943+0000", 2024-09-16T09:27:23.227 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.2190000000000001e-06 2024-09-16T09:27:23.227 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.227 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.227 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:27:23.227 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:21.575572+0000", 2024-09-16T09:27:23.227 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00162922 2024-09-16T09:27:23.227 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.228 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.228 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:27:23.228 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:21.575668+0000", 2024-09-16T09:27:23.228 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.5863e-05 2024-09-16T09:27:23.228 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.228 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.228 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 2,4", 2024-09-16T09:27:23.228 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:21.575783+0000", 2024-09-16T09:27:23.228 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000114817 2024-09-16T09:27:23.228 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.228 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.228 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:27:23.229 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:21.577585+0000", 2024-09-16T09:27:23.229 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0018025389999999999 2024-09-16T09:27:23.229 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.229 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.229 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:27:23.229 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:21.577816+0000", 2024-09-16T09:27:23.229 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00023053099999999999 2024-09-16T09:27:23.229 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.229 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.230 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:27:23.230 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:21.582756+0000", 2024-09-16T09:27:23.230 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0049396609999999997 2024-09-16T09:27:23.230 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.230 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.230 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:27:23.230 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:21.657534+0000", 2024-09-16T09:27:23.230 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.074778949999999997 2024-09-16T09:27:23.230 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.230 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.231 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:27:23.231 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:21.657579+0000", 2024-09-16T09:27:23.231 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.4549999999999999e-05 2024-09-16T09:27:23.231 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:27:23.231 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:27:23.231 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:27:23.231 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.231 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.231 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.24698.0:22926 7.3 7:c7975d6a:::benchmark_data_smithi070_78438_object22925:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T09:27:23.231 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:26:21.582159+0000", 2024-09-16T09:27:23.231 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 61.634053676999997, 2024-09-16T09:27:23.231 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.081201737999999996, 2024-09-16T09:27:23.232 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:27:23.232 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:27:23.232 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:27:23.232 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:27:23.232 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.24698", 2024-09-16T09:27:23.232 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:27:23.232 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 22926 2024-09-16T09:27:23.232 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.232 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:27:23.232 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.232 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:27:23.232 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:21.582159+0000", 2024-09-16T09:27:23.232 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:27:23.233 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.233 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.233 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:27:23.233 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:21.582159+0000", 2024-09-16T09:27:23.233 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:27:23.233 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.233 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.233 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:27:23.233 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:21.582160+0000", 2024-09-16T09:27:23.233 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.4410000000000001e-06 2024-09-16T09:27:23.233 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.233 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.233 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:27:23.234 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:21.582181+0000", 2024-09-16T09:27:23.234 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.1035000000000001e-05 2024-09-16T09:27:23.234 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.234 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.234 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:27:23.234 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:21.582193+0000", 2024-09-16T09:27:23.234 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1522000000000001e-05 2024-09-16T09:27:23.234 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.234 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.234 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:27:23.234 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:21.582201+0000", 2024-09-16T09:27:23.234 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.2919999999999996e-06 2024-09-16T09:27:23.234 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.235 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.235 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:27:23.235 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:21.658620+0000", 2024-09-16T09:27:23.235 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.076419070000000006 2024-09-16T09:27:23.235 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.235 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.235 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:27:23.235 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:21.658732+0000", 2024-09-16T09:27:23.235 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000111951 2024-09-16T09:27:23.235 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.235 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.235 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 2,4", 2024-09-16T09:27:23.235 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:21.658931+0000", 2024-09-16T09:27:23.235 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00019926399999999999 2024-09-16T09:27:23.236 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.236 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.236 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:27:23.236 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:21.660259+0000", 2024-09-16T09:27:23.236 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0013283940000000001 2024-09-16T09:27:23.236 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.236 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.236 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:27:23.236 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:21.660358+0000", 2024-09-16T09:27:23.236 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.8954000000000002e-05 2024-09-16T09:27:23.236 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.237 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.237 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:27:23.237 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:21.663259+0000", 2024-09-16T09:27:23.237 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0029010590000000001 2024-09-16T09:27:23.237 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.237 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.237 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:27:23.237 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:21.663315+0000", 2024-09-16T09:27:23.237 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5828000000000001e-05 2024-09-16T09:27:23.237 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.237 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.237 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:27:23.238 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:21.663360+0000", 2024-09-16T09:27:23.238 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.4928000000000001e-05 2024-09-16T09:27:23.238 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:27:23.238 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:27:23.238 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:27:23.238 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.238 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.238 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.24698.0:22931 7.a 7:54a65376:::benchmark_data_smithi070_78438_object22930:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T09:27:23.238 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:26:21.583023+0000", 2024-09-16T09:27:23.238 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 61.633188691999997, 2024-09-16T09:27:23.238 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.083393537000000004, 2024-09-16T09:27:23.238 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:27:23.239 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:27:23.239 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:27:23.239 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:27:23.239 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.24698", 2024-09-16T09:27:23.239 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:27:23.239 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 22931 2024-09-16T09:27:23.239 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.239 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:27:23.239 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.239 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:27:23.239 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:21.583023+0000", 2024-09-16T09:27:23.239 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:27:23.240 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.240 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.240 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:27:23.240 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:21.583023+0000", 2024-09-16T09:27:23.240 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:27:23.240 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.240 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.240 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:27:23.240 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:21.583024+0000", 2024-09-16T09:27:23.240 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.2600000000000001e-07 2024-09-16T09:27:23.240 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.240 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.240 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:27:23.241 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:21.583045+0000", 2024-09-16T09:27:23.241 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0704999999999999e-05 2024-09-16T09:27:23.241 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.241 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.241 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:27:23.241 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:21.656626+0000", 2024-09-16T09:27:23.241 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.07358083 2024-09-16T09:27:23.241 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.241 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.241 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:27:23.241 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:21.656639+0000", 2024-09-16T09:27:23.241 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2585000000000001e-05 2024-09-16T09:27:23.241 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.242 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.242 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:27:23.242 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:21.658079+0000", 2024-09-16T09:27:23.242 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001440099 2024-09-16T09:27:23.242 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.242 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.242 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:27:23.242 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:21.658213+0000", 2024-09-16T09:27:23.242 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00013391100000000001 2024-09-16T09:27:23.242 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.242 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.242 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 2,5", 2024-09-16T09:27:23.242 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:21.658351+0000", 2024-09-16T09:27:23.242 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000138682 2024-09-16T09:27:23.243 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.243 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.243 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:27:23.243 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:21.659819+0000", 2024-09-16T09:27:23.243 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001467643 2024-09-16T09:27:23.243 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.243 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.243 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:27:23.243 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:21.660246+0000", 2024-09-16T09:27:23.243 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000427553 2024-09-16T09:27:23.243 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.243 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.244 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:27:23.244 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:21.664278+0000", 2024-09-16T09:27:23.244 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0040312229999999996 2024-09-16T09:27:23.244 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.244 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.244 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:27:23.244 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:21.666368+0000", 2024-09-16T09:27:23.244 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0020906620000000001 2024-09-16T09:27:23.244 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.244 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.244 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:27:23.244 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:21.666417+0000", 2024-09-16T09:27:23.244 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.8717999999999997e-05 2024-09-16T09:27:23.244 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:27:23.245 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:27:23.245 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:27:23.245 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.245 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.245 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.24698.0:60230 7.3 7:cb5ea935:::benchmark_data_smithi070_78438_object60229:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-16T09:27:23.245 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:26:44.343517+0000", 2024-09-16T09:27:23.245 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 38.872695501000003, 2024-09-16T09:27:23.245 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.135916441, 2024-09-16T09:27:23.245 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:27:23.245 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:27:23.245 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:27:23.245 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:27:23.245 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.24698", 2024-09-16T09:27:23.246 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:27:23.246 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 60230 2024-09-16T09:27:23.246 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.246 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:27:23.246 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.246 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:27:23.246 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.343517+0000", 2024-09-16T09:27:23.246 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:27:23.246 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.246 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.246 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:27:23.246 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.343517+0000", 2024-09-16T09:27:23.246 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:27:23.246 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.246 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.246 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:27:23.247 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.343517+0000", 2024-09-16T09:27:23.247 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.8500000000000001e-07 2024-09-16T09:27:23.247 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.247 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.247 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:27:23.247 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.343537+0000", 2024-09-16T09:27:23.247 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9845999999999999e-05 2024-09-16T09:27:23.247 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.247 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.247 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:27:23.247 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.343545+0000", 2024-09-16T09:27:23.247 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.2209999999999997e-06 2024-09-16T09:27:23.247 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.247 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.247 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:27:23.247 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.343550+0000", 2024-09-16T09:27:23.248 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.899e-06 2024-09-16T09:27:23.248 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.248 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.248 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:27:23.248 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.347605+0000", 2024-09-16T09:27:23.248 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0040543519999999998 2024-09-16T09:27:23.248 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.248 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.248 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:27:23.248 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.347705+0000", 2024-09-16T09:27:23.248 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00010025 2024-09-16T09:27:23.248 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.248 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.249 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 2,4", 2024-09-16T09:27:23.249 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.347823+0000", 2024-09-16T09:27:23.249 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000117596 2024-09-16T09:27:23.249 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.249 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.249 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:27:23.249 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.351145+0000", 2024-09-16T09:27:23.249 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0033226839999999998 2024-09-16T09:27:23.249 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.249 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.249 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:27:23.249 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.351517+0000", 2024-09-16T09:27:23.249 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00037208299999999999 2024-09-16T09:27:23.249 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.250 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.250 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:27:23.250 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.479316+0000", 2024-09-16T09:27:23.250 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.12779896399999999 2024-09-16T09:27:23.250 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.250 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.250 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:27:23.250 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.479389+0000", 2024-09-16T09:27:23.250 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.2237000000000007e-05 2024-09-16T09:27:23.250 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.250 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.250 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:27:23.250 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.479433+0000", 2024-09-16T09:27:23.250 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.4623999999999998e-05 2024-09-16T09:27:23.250 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:27:23.250 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:27:23.251 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:27:23.251 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.251 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.251 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.24698.0:60241 7.6 e98/82 7:6fa435f1:::benchmark_data_smithi070_78438_object60240:head v 98'3736, mlcod=98'3734)", 2024-09-16T09:27:23.251 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:26:44.351029+0000", 2024-09-16T09:27:23.251 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 38.865183674000001, 2024-09-16T09:27:23.251 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.12807455000000001, 2024-09-16T09:27:23.251 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:27:23.251 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:27:23.251 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:27:23.251 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:27:23.251 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.251 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:27:23.251 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.351029+0000", 2024-09-16T09:27:23.252 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:27:23.252 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.252 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.252 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:27:23.252 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.351029+0000", 2024-09-16T09:27:23.252 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:27:23.252 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.252 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.252 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:27:23.252 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.351029+0000", 2024-09-16T09:27:23.252 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.89e-07 2024-09-16T09:27:23.252 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.252 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.252 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:27:23.253 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.351050+0000", 2024-09-16T09:27:23.253 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.1070999999999999e-05 2024-09-16T09:27:23.253 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.253 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.253 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:27:23.253 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.351059+0000", 2024-09-16T09:27:23.253 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.2820000000000002e-06 2024-09-16T09:27:23.254 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.254 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.254 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:27:23.254 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.351066+0000", 2024-09-16T09:27:23.254 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.2489999999999998e-06 2024-09-16T09:27:23.254 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.254 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.254 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:27:23.254 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.351140+0000", 2024-09-16T09:27:23.255 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3998000000000005e-05 2024-09-16T09:27:23.255 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.255 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.255 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:27:23.255 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.351157+0000", 2024-09-16T09:27:23.255 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7334999999999999e-05 2024-09-16T09:27:23.255 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.255 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.255 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:27:23.255 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.479051+0000", 2024-09-16T09:27:23.255 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.12789431600000001 2024-09-16T09:27:23.255 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.256 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.256 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:27:23.256 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.479103+0000", 2024-09-16T09:27:23.256 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.181e-05 2024-09-16T09:27:23.256 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:27:23.256 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:27:23.256 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:27:23.256 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.256 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.257 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.24698.0:60243 7.a 7:519efbc0:::benchmark_data_smithi070_78438_object60242:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-16T09:27:23.257 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:26:44.351884+0000", 2024-09-16T09:27:23.257 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 38.864327936999999, 2024-09-16T09:27:23.257 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.13656760500000001, 2024-09-16T09:27:23.257 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:27:23.257 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:27:23.257 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:27:23.257 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:27:23.257 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.24698", 2024-09-16T09:27:23.257 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:27:23.257 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 60243 2024-09-16T09:27:23.257 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.258 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:27:23.258 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.258 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:27:23.258 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.351884+0000", 2024-09-16T09:27:23.258 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:27:23.258 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.258 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.258 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:27:23.258 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.351884+0000", 2024-09-16T09:27:23.258 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:27:23.258 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.259 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.259 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:27:23.259 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.351885+0000", 2024-09-16T09:27:23.259 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.203e-06 2024-09-16T09:27:23.259 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.259 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.259 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:27:23.259 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.351909+0000", 2024-09-16T09:27:23.259 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3107000000000001e-05 2024-09-16T09:27:23.259 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.259 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.260 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:27:23.260 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.351919+0000", 2024-09-16T09:27:23.260 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0239e-05 2024-09-16T09:27:23.260 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.260 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.260 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:27:23.260 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.351940+0000", 2024-09-16T09:27:23.260 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0846999999999999e-05 2024-09-16T09:27:23.260 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.260 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.260 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:27:23.260 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.355041+0000", 2024-09-16T09:27:23.261 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0031018109999999999 2024-09-16T09:27:23.261 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.261 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.261 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:27:23.261 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.355147+0000", 2024-09-16T09:27:23.261 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000105263 2024-09-16T09:27:23.261 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.261 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.261 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 2,5", 2024-09-16T09:27:23.261 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.479270+0000", 2024-09-16T09:27:23.261 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.12412316 2024-09-16T09:27:23.262 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.262 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.262 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:27:23.262 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.488255+0000", 2024-09-16T09:27:23.262 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0089848919999999995 2024-09-16T09:27:23.262 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.262 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.262 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:27:23.262 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.488266+0000", 2024-09-16T09:27:23.262 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1381e-05 2024-09-16T09:27:23.263 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.263 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.263 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:27:23.263 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.488340+0000", 2024-09-16T09:27:23.263 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3609000000000006e-05 2024-09-16T09:27:23.263 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.263 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.263 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:27:23.263 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.488410+0000", 2024-09-16T09:27:23.263 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.9747999999999997e-05 2024-09-16T09:27:23.263 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.264 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.264 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:27:23.264 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.488452+0000", 2024-09-16T09:27:23.264 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2345000000000002e-05 2024-09-16T09:27:23.264 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:27:23.264 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:27:23.264 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:27:23.264 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.264 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.264 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.24698.0:60248 7.3 7:c2e4be92:::benchmark_data_smithi070_78438_object60247:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-16T09:27:23.264 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:26:44.352711+0000", 2024-09-16T09:27:23.265 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 38.863501102999997, 2024-09-16T09:27:23.265 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.12688961500000001, 2024-09-16T09:27:23.265 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:27:23.265 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:27:23.265 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:27:23.265 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:27:23.265 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.24698", 2024-09-16T09:27:23.265 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:27:23.265 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 60248 2024-09-16T09:27:23.265 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.265 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:27:23.265 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.266 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:27:23.266 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.352711+0000", 2024-09-16T09:27:23.266 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:27:23.266 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.266 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.266 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:27:23.266 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.352711+0000", 2024-09-16T09:27:23.266 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:27:23.267 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.267 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.267 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:27:23.267 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.352712+0000", 2024-09-16T09:27:23.267 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.16e-07 2024-09-16T09:27:23.267 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.267 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.267 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:27:23.267 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.352732+0000", 2024-09-16T09:27:23.267 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9944000000000001e-05 2024-09-16T09:27:23.267 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.268 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.268 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:27:23.268 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.352743+0000", 2024-09-16T09:27:23.268 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1053000000000001e-05 2024-09-16T09:27:23.268 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.268 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.268 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:27:23.269 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.352750+0000", 2024-09-16T09:27:23.269 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.1879999999999996e-06 2024-09-16T09:27:23.269 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.269 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.269 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:27:23.269 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.352871+0000", 2024-09-16T09:27:23.269 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000120828 2024-09-16T09:27:23.269 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.269 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.269 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:27:23.270 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.352997+0000", 2024-09-16T09:27:23.270 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00012576699999999999 2024-09-16T09:27:23.270 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.270 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.270 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 2,4", 2024-09-16T09:27:23.270 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.353118+0000", 2024-09-16T09:27:23.270 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000121045 2024-09-16T09:27:23.270 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.270 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.271 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:27:23.271 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.479259+0000", 2024-09-16T09:27:23.271 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.12614093800000001 2024-09-16T09:27:23.271 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.271 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.271 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:27:23.272 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:23.273 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:23.273 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:27:23.274 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.479477+0000", 2024-09-16T09:27:23.274 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00021839 2024-09-16T09:27:23.274 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.274 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.274 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:27:23.275 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.479517+0000", 2024-09-16T09:27:23.275 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9885999999999997e-05 2024-09-16T09:27:23.275 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.275 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.275 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:27:23.275 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.479566+0000", 2024-09-16T09:27:23.275 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.8819000000000001e-05 2024-09-16T09:27:23.275 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.275 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.275 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:27:23.276 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.479601+0000", 2024-09-16T09:27:23.276 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4841000000000001e-05 2024-09-16T09:27:23.276 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:27:23.276 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:27:23.276 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:27:23.276 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.276 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.276 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.24698.0:60245 7.7 e98/82 7:e5610e70:::benchmark_data_smithi070_78438_object60244:head v 98'3786, mlcod=98'3784)", 2024-09-16T09:27:23.276 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:26:44.352780+0000", 2024-09-16T09:27:23.276 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 38.863431816000002, 2024-09-16T09:27:23.276 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.12639468800000001, 2024-09-16T09:27:23.276 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:27:23.276 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:27:23.276 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:27:23.277 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:27:23.277 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.277 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:27:23.277 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.352780+0000", 2024-09-16T09:27:23.277 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:27:23.277 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.277 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.277 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:27:23.277 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.352780+0000", 2024-09-16T09:27:23.277 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:27:23.277 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.277 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.277 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:27:23.277 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.352781+0000", 2024-09-16T09:27:23.278 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0999999999999999e-07 2024-09-16T09:27:23.278 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.278 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.278 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:27:23.278 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.352808+0000", 2024-09-16T09:27:23.278 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.7022e-05 2024-09-16T09:27:23.278 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.278 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.278 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:27:23.278 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.352815+0000", 2024-09-16T09:27:23.278 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.4390000000000003e-06 2024-09-16T09:27:23.278 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.278 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.278 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:27:23.279 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.352820+0000", 2024-09-16T09:27:23.279 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.9819999999999999e-06 2024-09-16T09:27:23.279 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.279 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.279 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:27:23.279 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.353010+0000", 2024-09-16T09:27:23.279 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000189204 2024-09-16T09:27:23.279 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.279 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.279 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:27:23.279 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.353027+0000", 2024-09-16T09:27:23.279 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7907e-05 2024-09-16T09:27:23.279 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.279 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.280 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:27:23.280 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.355309+0000", 2024-09-16T09:27:23.280 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0022814799999999998 2024-09-16T09:27:23.280 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.280 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.280 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:27:23.280 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.479175+0000", 2024-09-16T09:27:23.280 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.123866144 2024-09-16T09:27:23.280 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:27:23.280 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:27:23.280 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:27:23.280 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.280 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.281 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.24698.0:60247 7.6 e98/82 7:6840b134:::benchmark_data_smithi070_78438_object60246:head v 98'3737, mlcod=98'3734)", 2024-09-16T09:27:23.281 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:26:44.352896+0000", 2024-09-16T09:27:23.281 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 38.863316056999999, 2024-09-16T09:27:23.281 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.12623531099999999, 2024-09-16T09:27:23.281 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:27:23.281 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:27:23.281 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:27:23.281 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:27:23.281 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.281 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:27:23.281 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.352896+0000", 2024-09-16T09:27:23.281 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:27:23.281 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.282 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.282 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:27:23.282 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.352896+0000", 2024-09-16T09:27:23.282 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:27:23.282 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.282 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.282 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:27:23.282 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.352897+0000", 2024-09-16T09:27:23.282 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.3700000000000001e-07 2024-09-16T09:27:23.282 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.282 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.282 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:27:23.282 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.352918+0000", 2024-09-16T09:27:23.283 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.1535e-05 2024-09-16T09:27:23.283 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.283 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.283 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:27:23.283 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.353538+0000", 2024-09-16T09:27:23.283 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00061983100000000005 2024-09-16T09:27:23.283 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.283 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.283 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:27:23.283 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.353544+0000", 2024-09-16T09:27:23.283 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.0279999999999997e-06 2024-09-16T09:27:23.283 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.283 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.284 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:27:23.284 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.353638+0000", 2024-09-16T09:27:23.284 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.4378999999999999e-05 2024-09-16T09:27:23.284 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.284 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.284 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:27:23.284 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.353655+0000", 2024-09-16T09:27:23.284 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6226e-05 2024-09-16T09:27:23.284 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.284 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.284 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:27:23.284 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.479108+0000", 2024-09-16T09:27:23.284 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.12545310500000001 2024-09-16T09:27:23.284 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.285 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.285 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:27:23.285 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.479131+0000", 2024-09-16T09:27:23.285 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3770000000000001e-05 2024-09-16T09:27:23.285 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:27:23.285 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:27:23.285 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:27:23.285 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.285 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.285 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.24698.0:60250 7.a 7:57008ad8:::benchmark_data_smithi070_78438_object60249:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-16T09:27:23.285 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:26:44.354902+0000", 2024-09-16T09:27:23.285 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 38.861310570999997, 2024-09-16T09:27:23.286 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.13525178299999999, 2024-09-16T09:27:23.286 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:27:23.286 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:27:23.286 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:27:23.286 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:27:23.286 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.24698", 2024-09-16T09:27:23.286 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:27:23.286 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 60250 2024-09-16T09:27:23.286 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.286 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:27:23.286 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.286 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:27:23.286 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.354902+0000", 2024-09-16T09:27:23.287 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:27:23.287 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.287 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.287 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:27:23.287 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.354902+0000", 2024-09-16T09:27:23.287 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:27:23.287 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.287 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.287 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:27:23.287 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.354903+0000", 2024-09-16T09:27:23.287 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.217e-06 2024-09-16T09:27:23.287 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.287 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.287 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:27:23.288 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.354926+0000", 2024-09-16T09:27:23.288 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3411e-05 2024-09-16T09:27:23.288 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.288 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.288 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:27:23.288 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.354937+0000", 2024-09-16T09:27:23.288 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0779e-05 2024-09-16T09:27:23.288 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.288 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.288 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:27:23.288 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.354944+0000", 2024-09-16T09:27:23.288 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.148e-06 2024-09-16T09:27:23.289 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.289 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.289 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:27:23.289 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.483018+0000", 2024-09-16T09:27:23.289 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.128073662 2024-09-16T09:27:23.289 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.289 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.289 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:27:23.289 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.483158+0000", 2024-09-16T09:27:23.289 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000140087 2024-09-16T09:27:23.290 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.290 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.290 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 2,5", 2024-09-16T09:27:23.290 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.483300+0000", 2024-09-16T09:27:23.290 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00014235499999999999 2024-09-16T09:27:23.290 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.290 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.290 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:27:23.290 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.488300+0000", 2024-09-16T09:27:23.290 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0049997389999999996 2024-09-16T09:27:23.291 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.291 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.291 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:27:23.291 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.488489+0000", 2024-09-16T09:27:23.291 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00018912200000000001 2024-09-16T09:27:23.291 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.291 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.291 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:27:23.291 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.490003+0000", 2024-09-16T09:27:23.291 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0015142440000000001 2024-09-16T09:27:23.291 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.291 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.292 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:27:23.292 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.490111+0000", 2024-09-16T09:27:23.292 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000107298 2024-09-16T09:27:23.292 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.292 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.292 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:27:23.292 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.490153+0000", 2024-09-16T09:27:23.292 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2721e-05 2024-09-16T09:27:23.292 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:27:23.292 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:27:23.293 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:27:23.293 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.293 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.293 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop_reply(client.24698.0:60248 7.3 e98/82 ondisk, result = 0)", 2024-09-16T09:27:23.293 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:26:44.355484+0000", 2024-09-16T09:27:23.293 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 38.860728186999999, 2024-09-16T09:27:23.293 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.124000344, 2024-09-16T09:27:23.293 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:27:23.293 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:27:23.293 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T09:27:23.293 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:27:23.293 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.293 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:27:23.294 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.355484+0000", 2024-09-16T09:27:23.294 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:27:23.294 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.294 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.294 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:27:23.294 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.355484+0000", 2024-09-16T09:27:23.294 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:27:23.294 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.294 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.294 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:27:23.295 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.355485+0000", 2024-09-16T09:27:23.295 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.1500000000000003e-07 2024-09-16T09:27:23.295 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.295 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.295 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:27:23.295 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.355488+0000", 2024-09-16T09:27:23.295 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6540000000000001e-06 2024-09-16T09:27:23.295 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.295 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.295 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:27:23.295 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.479245+0000", 2024-09-16T09:27:23.296 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.123756668 2024-09-16T09:27:23.296 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.296 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.296 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:27:23.296 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.479255+0000", 2024-09-16T09:27:23.296 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0030999999999999e-05 2024-09-16T09:27:23.296 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.296 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.296 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:27:23.296 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.479465+0000", 2024-09-16T09:27:23.296 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000209903 2024-09-16T09:27:23.296 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.297 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.297 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:27:23.297 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.479472+0000", 2024-09-16T09:27:23.297 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.922e-06 2024-09-16T09:27:23.297 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.297 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.297 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:27:23.297 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.479484+0000", 2024-09-16T09:27:23.297 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2351000000000001e-05 2024-09-16T09:27:23.297 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:27:23.297 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:27:23.297 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:27:23.298 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.298 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.298 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop_reply(client.24698.0:60230 7.3 e98/82 ondisk, result = 0)", 2024-09-16T09:27:23.298 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:26:44.356597+0000", 2024-09-16T09:27:23.298 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 38.859615441999999, 2024-09-16T09:27:23.298 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.122843122, 2024-09-16T09:27:23.298 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:27:23.298 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:27:23.298 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T09:27:23.298 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:27:23.299 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.299 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:27:23.299 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.356597+0000", 2024-09-16T09:27:23.299 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:27:23.299 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.299 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.299 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:27:23.299 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.356597+0000", 2024-09-16T09:27:23.299 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:27:23.300 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.300 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.300 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:27:23.300 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.356598+0000", 2024-09-16T09:27:23.300 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.9400000000000004e-07 2024-09-16T09:27:23.300 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.300 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.300 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:27:23.300 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.356601+0000", 2024-09-16T09:27:23.300 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9330000000000003e-06 2024-09-16T09:27:23.301 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.301 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.301 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:27:23.301 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.478958+0000", 2024-09-16T09:27:23.301 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.12235689499999999 2024-09-16T09:27:23.301 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.301 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.301 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:27:23.301 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.478971+0000", 2024-09-16T09:27:23.301 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2969999999999999e-05 2024-09-16T09:27:23.301 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.301 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.301 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:27:23.301 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.479302+0000", 2024-09-16T09:27:23.301 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00033111099999999998 2024-09-16T09:27:23.302 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.302 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.302 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:27:23.302 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.479311+0000", 2024-09-16T09:27:23.302 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.3620000000000006e-06 2024-09-16T09:27:23.302 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.302 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.302 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:27:23.302 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.479440+0000", 2024-09-16T09:27:23.302 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00012905699999999999 2024-09-16T09:27:23.302 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:27:23.303 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:27:23.303 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:27:23.303 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.303 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.303 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.24698.0:86884 7.a 7:5df900fe:::benchmark_data_smithi070_78438_object86883:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-16T09:27:23.303 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:27:06.131225+0000", 2024-09-16T09:27:23.303 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 17.084987565999999, 2024-09-16T09:27:23.303 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.091404366000000001, 2024-09-16T09:27:23.303 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:27:23.303 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:27:23.303 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:27:23.303 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:27:23.304 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.24698", 2024-09-16T09:27:23.304 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:27:23.304 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 86884 2024-09-16T09:27:23.304 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.304 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:27:23.304 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.304 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:27:23.304 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.131225+0000", 2024-09-16T09:27:23.304 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:27:23.304 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.304 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.304 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:27:23.305 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.131225+0000", 2024-09-16T09:27:23.305 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:27:23.305 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.305 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.305 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:27:23.305 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.131226+0000", 2024-09-16T09:27:23.305 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.28e-06 2024-09-16T09:27:23.305 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.305 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.305 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:27:23.305 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.131244+0000", 2024-09-16T09:27:23.306 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7677999999999999e-05 2024-09-16T09:27:23.306 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.306 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.306 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:27:23.306 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.131254+0000", 2024-09-16T09:27:23.306 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.9520000000000006e-06 2024-09-16T09:27:23.306 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.306 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.306 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:27:23.306 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.131259+0000", 2024-09-16T09:27:23.307 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7660000000000004e-06 2024-09-16T09:27:23.307 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.307 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.307 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:27:23.307 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.131983+0000", 2024-09-16T09:27:23.307 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00072386100000000001 2024-09-16T09:27:23.307 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.307 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.307 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:27:23.308 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.132087+0000", 2024-09-16T09:27:23.308 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00010430299999999999 2024-09-16T09:27:23.308 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.308 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.308 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 2,5", 2024-09-16T09:27:23.308 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.132211+0000", 2024-09-16T09:27:23.308 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000123089 2024-09-16T09:27:23.308 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.308 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.308 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:27:23.309 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.134857+0000", 2024-09-16T09:27:23.309 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.002646575 2024-09-16T09:27:23.309 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.309 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.309 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:27:23.309 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.136173+0000", 2024-09-16T09:27:23.309 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0013156019999999999 2024-09-16T09:27:23.309 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.309 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.310 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:27:23.310 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.222399+0000", 2024-09-16T09:27:23.310 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.086226593000000004 2024-09-16T09:27:23.310 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.310 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.310 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:27:23.310 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.222579+0000", 2024-09-16T09:27:23.310 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000179354 2024-09-16T09:27:23.310 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.310 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.310 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:27:23.310 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.222629+0000", 2024-09-16T09:27:23.311 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0312999999999998e-05 2024-09-16T09:27:23.311 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:27:23.311 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:27:23.311 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:27:23.311 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.311 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.311 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.24698.0:86887 7.c e105/82 7:347431d3:::benchmark_data_smithi070_78438_object86886:head v 105'5376, mlcod=105'5372)", 2024-09-16T09:27:23.311 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:27:06.135573+0000", 2024-09-16T09:27:23.311 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 17.080638807, 2024-09-16T09:27:23.311 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.087152681999999995, 2024-09-16T09:27:23.311 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:27:23.312 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:27:23.312 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:27:23.312 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:27:23.312 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.312 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:27:23.312 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.135573+0000", 2024-09-16T09:27:23.312 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:27:23.312 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.312 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.312 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:27:23.312 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.135573+0000", 2024-09-16T09:27:23.312 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:27:23.312 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.312 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.313 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:27:23.313 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.135574+0000", 2024-09-16T09:27:23.313 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.1999999999999998e-07 2024-09-16T09:27:23.313 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.313 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.313 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:27:23.313 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.135608+0000", 2024-09-16T09:27:23.313 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3321000000000002e-05 2024-09-16T09:27:23.313 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.313 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.313 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:27:23.313 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.135625+0000", 2024-09-16T09:27:23.313 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7041000000000002e-05 2024-09-16T09:27:23.314 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.314 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.314 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:27:23.314 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.135631+0000", 2024-09-16T09:27:23.314 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.3099999999999997e-06 2024-09-16T09:27:23.314 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.314 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.314 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:27:23.314 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.137126+0000", 2024-09-16T09:27:23.314 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001495084 2024-09-16T09:27:23.315 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.315 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.315 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:27:23.315 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.137142+0000", 2024-09-16T09:27:23.315 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6172999999999999e-05 2024-09-16T09:27:23.315 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.315 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.315 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:27:23.315 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.222688+0000", 2024-09-16T09:27:23.315 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.085545484000000005 2024-09-16T09:27:23.315 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.315 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.315 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:27:23.316 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.222726+0000", 2024-09-16T09:27:23.316 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8448999999999997e-05 2024-09-16T09:27:23.316 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:27:23.316 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:27:23.316 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:27:23.316 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.316 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.316 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.24698.0:86890 7.1 e105/82 7:8b0c621b:::benchmark_data_smithi070_78438_object86889:head v 105'5328, mlcod=105'5326)", 2024-09-16T09:27:23.316 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:27:06.137397+0000", 2024-09-16T09:27:23.316 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 17.078815059, 2024-09-16T09:27:23.316 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.082989971999999995, 2024-09-16T09:27:23.316 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:27:23.316 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:27:23.316 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:27:23.317 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:27:23.317 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.317 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:27:23.317 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.137397+0000", 2024-09-16T09:27:23.317 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:27:23.317 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.317 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.317 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:27:23.317 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.137397+0000", 2024-09-16T09:27:23.317 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:27:23.317 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.317 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.317 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:27:23.318 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.137398+0000", 2024-09-16T09:27:23.318 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.6899999999999996e-07 2024-09-16T09:27:23.318 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.318 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.318 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:27:23.318 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.137427+0000", 2024-09-16T09:27:23.318 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.9343e-05 2024-09-16T09:27:23.318 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.318 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.318 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:27:23.318 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.137438+0000", 2024-09-16T09:27:23.318 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0910000000000001e-05 2024-09-16T09:27:23.318 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.318 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.319 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:27:23.319 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.137444+0000", 2024-09-16T09:27:23.319 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.2210000000000002e-06 2024-09-16T09:27:23.319 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.319 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.319 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:27:23.319 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.137591+0000", 2024-09-16T09:27:23.319 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00014626 2024-09-16T09:27:23.319 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.319 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.319 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:27:23.319 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.137624+0000", 2024-09-16T09:27:23.319 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3704000000000001e-05 2024-09-16T09:27:23.319 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.319 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.320 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:27:23.320 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.220347+0000", 2024-09-16T09:27:23.320 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.082722093999999996 2024-09-16T09:27:23.320 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.320 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.320 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:27:23.320 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.220387+0000", 2024-09-16T09:27:23.320 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0571000000000002e-05 2024-09-16T09:27:23.320 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:27:23.320 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:27:23.320 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:27:23.320 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.320 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.321 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.24698.0:86892 7.f e105/82 7:f2806a39:::benchmark_data_smithi070_78438_object86891:head v 105'5504, mlcod=105'5502)", 2024-09-16T09:27:23.321 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:27:06.137486+0000", 2024-09-16T09:27:23.321 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 17.078725948999999, 2024-09-16T09:27:23.321 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.081603922999999995, 2024-09-16T09:27:23.321 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:27:23.321 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:27:23.321 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:27:23.321 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:27:23.321 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.321 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:27:23.321 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.137486+0000", 2024-09-16T09:27:23.321 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:27:23.321 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.322 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.322 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:27:23.322 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.137486+0000", 2024-09-16T09:27:23.322 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:27:23.322 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.322 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.322 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:27:23.322 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.137487+0000", 2024-09-16T09:27:23.322 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0600000000000001e-07 2024-09-16T09:27:23.322 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.322 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.322 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:27:23.322 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.137505+0000", 2024-09-16T09:27:23.322 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.8734000000000001e-05 2024-09-16T09:27:23.323 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.323 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.323 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:27:23.323 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.137512+0000", 2024-09-16T09:27:23.323 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.0500000000000003e-06 2024-09-16T09:27:23.323 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.323 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.323 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:27:23.323 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.137517+0000", 2024-09-16T09:27:23.323 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.4449999999999998e-06 2024-09-16T09:27:23.323 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.323 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.324 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:27:23.324 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.138110+0000", 2024-09-16T09:27:23.324 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00059345800000000005 2024-09-16T09:27:23.324 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.324 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.324 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:27:23.324 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.138128+0000", 2024-09-16T09:27:23.324 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7337e-05 2024-09-16T09:27:23.324 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.324 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.324 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:27:23.325 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.218923+0000", 2024-09-16T09:27:23.325 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.080795334999999996 2024-09-16T09:27:23.325 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.325 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.325 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:27:23.325 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.219090+0000", 2024-09-16T09:27:23.325 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00016715799999999999 2024-09-16T09:27:23.325 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:27:23.325 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:27:23.325 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:27:23.325 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.325 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.325 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.24698.0:86894 7.a 7:54e07782:::benchmark_data_smithi070_78438_object86893:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-16T09:27:23.326 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:27:06.138810+0000", 2024-09-16T09:27:23.326 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 17.077401967, 2024-09-16T09:27:23.326 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.093325733999999994, 2024-09-16T09:27:23.326 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:27:23.326 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:27:23.326 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:27:23.326 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:27:23.326 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.24698", 2024-09-16T09:27:23.327 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:27:23.327 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 86894 2024-09-16T09:27:23.327 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.327 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:27:23.327 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.327 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:27:23.327 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.138810+0000", 2024-09-16T09:27:23.327 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:27:23.327 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.327 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.327 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:27:23.327 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.138810+0000", 2024-09-16T09:27:23.327 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:27:23.327 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.328 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.328 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:27:23.328 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.138812+0000", 2024-09-16T09:27:23.328 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.378e-06 2024-09-16T09:27:23.328 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.328 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.328 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:27:23.328 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.138833+0000", 2024-09-16T09:27:23.328 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.1407999999999999e-05 2024-09-16T09:27:23.329 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.329 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.329 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:27:23.329 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.138843+0000", 2024-09-16T09:27:23.329 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.5740000000000002e-06 2024-09-16T09:27:23.329 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.329 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.329 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:27:23.329 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.138850+0000", 2024-09-16T09:27:23.329 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.0759999999999997e-06 2024-09-16T09:27:23.330 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.330 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.330 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:27:23.330 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.218942+0000", 2024-09-16T09:27:23.330 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.080092697000000004 2024-09-16T09:27:23.330 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.330 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.330 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:27:23.330 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.219090+0000", 2024-09-16T09:27:23.330 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000147455 2024-09-16T09:27:23.330 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.331 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.331 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 2,5", 2024-09-16T09:27:23.331 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.219244+0000", 2024-09-16T09:27:23.331 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000154344 2024-09-16T09:27:23.331 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.331 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.331 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:27:23.331 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.222668+0000", 2024-09-16T09:27:23.331 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0034239449999999999 2024-09-16T09:27:23.331 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.331 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.331 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:27:23.332 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.226556+0000", 2024-09-16T09:27:23.332 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.003888383 2024-09-16T09:27:23.332 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.332 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.332 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:27:23.332 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.232059+0000", 2024-09-16T09:27:23.332 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0055024050000000001 2024-09-16T09:27:23.332 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.332 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.333 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:27:23.333 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.232100+0000", 2024-09-16T09:27:23.333 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0809999999999997e-05 2024-09-16T09:27:23.333 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.333 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.333 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:27:23.333 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.232136+0000", 2024-09-16T09:27:23.333 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6259e-05 2024-09-16T09:27:23.334 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:27:23.334 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:27:23.334 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:27:23.334 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.334 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.334 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.24698.0:86900 7.3 7:c20e79e9:::benchmark_data_smithi070_78438_object86899:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-16T09:27:23.334 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:27:06.139507+0000", 2024-09-16T09:27:23.334 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 17.076704694, 2024-09-16T09:27:23.334 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.087967397000000003, 2024-09-16T09:27:23.334 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:27:23.334 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:27:23.334 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:27:23.335 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:27:23.335 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.24698", 2024-09-16T09:27:23.335 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:27:23.335 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 86900 2024-09-16T09:27:23.335 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.335 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:27:23.335 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.335 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:27:23.335 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.139507+0000", 2024-09-16T09:27:23.335 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:27:23.335 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.335 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.336 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:27:23.336 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.139507+0000", 2024-09-16T09:27:23.336 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:27:23.336 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.336 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.336 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:27:23.336 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.139508+0000", 2024-09-16T09:27:23.336 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.0400000000000005e-07 2024-09-16T09:27:23.336 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.336 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.336 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:27:23.336 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.139530+0000", 2024-09-16T09:27:23.336 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.1954e-05 2024-09-16T09:27:23.337 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.337 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.337 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:27:23.337 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.218888+0000", 2024-09-16T09:27:23.337 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.079357270999999993 2024-09-16T09:27:23.337 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.337 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.337 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:27:23.337 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.218902+0000", 2024-09-16T09:27:23.337 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.4269e-05 2024-09-16T09:27:23.337 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.338 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.338 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:27:23.338 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.218988+0000", 2024-09-16T09:27:23.338 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.6592e-05 2024-09-16T09:27:23.338 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.338 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.338 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:27:23.338 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.219110+0000", 2024-09-16T09:27:23.338 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000121554 2024-09-16T09:27:23.338 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.339 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.339 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 2,4", 2024-09-16T09:27:23.339 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.219324+0000", 2024-09-16T09:27:23.339 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00021373200000000001 2024-09-16T09:27:23.339 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.339 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.339 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:27:23.339 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.226401+0000", 2024-09-16T09:27:23.339 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0070768810000000001 2024-09-16T09:27:23.339 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.339 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.339 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:27:23.340 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.227313+0000", 2024-09-16T09:27:23.340 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00091269300000000001 2024-09-16T09:27:23.340 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.341 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.341 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:27:23.341 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.227369+0000", 2024-09-16T09:27:23.341 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5714999999999998e-05 2024-09-16T09:27:23.341 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.341 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.341 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:27:23.341 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.227428+0000", 2024-09-16T09:27:23.341 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.9367999999999998e-05 2024-09-16T09:27:23.341 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.342 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.342 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:27:23.342 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.227475+0000", 2024-09-16T09:27:23.342 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.6464000000000001e-05 2024-09-16T09:27:23.342 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:27:23.342 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:27:23.342 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:27:23.342 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.342 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.342 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop_reply(client.24698.0:86884 7.a e105/82 ondisk, result = 0)", 2024-09-16T09:27:23.342 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:27:06.139885+0000", 2024-09-16T09:27:23.342 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 17.076327227, 2024-09-16T09:27:23.342 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.082751624999999995, 2024-09-16T09:27:23.342 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:27:23.343 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:27:23.343 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T09:27:23.343 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:27:23.343 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.343 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:27:23.343 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.139885+0000", 2024-09-16T09:27:23.343 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:27:23.343 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.343 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.343 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:27:23.343 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.139885+0000", 2024-09-16T09:27:23.343 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:27:23.343 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.344 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.344 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:27:23.344 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.139886+0000", 2024-09-16T09:27:23.344 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8999999999999996e-07 2024-09-16T09:27:23.344 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.344 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.344 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:27:23.344 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.139890+0000", 2024-09-16T09:27:23.344 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0960000000000003e-06 2024-09-16T09:27:23.344 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.344 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.344 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:27:23.344 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.218876+0000", 2024-09-16T09:27:23.344 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.078986250999999993 2024-09-16T09:27:23.345 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.345 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.345 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:27:23.345 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.218891+0000", 2024-09-16T09:27:23.345 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5238999999999999e-05 2024-09-16T09:27:23.345 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.345 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.345 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:27:23.345 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.222387+0000", 2024-09-16T09:27:23.345 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0034954959999999998 2024-09-16T09:27:23.345 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.345 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.345 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:27:23.346 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.222394+0000", 2024-09-16T09:27:23.346 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3200000000000002e-06 2024-09-16T09:27:23.346 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.346 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.346 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:27:23.346 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.222637+0000", 2024-09-16T09:27:23.346 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00024263299999999999 2024-09-16T09:27:23.346 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:27:23.346 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:27:23.346 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:27:23.346 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.346 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.348 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.24698.0:86891 7.1 e105/82 7:8533eaa2:::benchmark_data_smithi070_78438_object86890:head v 105'5329, mlcod=105'5326)", 2024-09-16T09:27:23.348 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:27:06.140004+0000", 2024-09-16T09:27:23.348 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 17.076208489999999, 2024-09-16T09:27:23.348 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.087403198000000001, 2024-09-16T09:27:23.348 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:27:23.348 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:27:23.348 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:27:23.348 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:27:23.348 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.348 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:27:23.348 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.140004+0000", 2024-09-16T09:27:23.349 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:27:23.349 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.349 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.349 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:27:23.349 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.140004+0000", 2024-09-16T09:27:23.349 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:27:23.349 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.349 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.349 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:27:23.349 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.140004+0000", 2024-09-16T09:27:23.349 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.9999999999999997e-07 2024-09-16T09:27:23.349 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.349 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.349 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:27:23.350 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.140056+0000", 2024-09-16T09:27:23.350 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.1609e-05 2024-09-16T09:27:23.350 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.350 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.350 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:27:23.350 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.218896+0000", 2024-09-16T09:27:23.350 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.078839736999999993 2024-09-16T09:27:23.350 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.350 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.350 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:27:23.350 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.218906+0000", 2024-09-16T09:27:23.350 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0747999999999999e-05 2024-09-16T09:27:23.350 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.350 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.351 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:27:23.351 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.220428+0000", 2024-09-16T09:27:23.351 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0015211560000000001 2024-09-16T09:27:23.351 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.351 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.351 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:27:23.351 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.220445+0000", 2024-09-16T09:27:23.351 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7575e-05 2024-09-16T09:27:23.351 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.351 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.351 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:27:23.351 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.227359+0000", 2024-09-16T09:27:23.351 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0069135569999999999 2024-09-16T09:27:23.351 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:23.351 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:23.352 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:27:23.352 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.227407+0000", 2024-09-16T09:27:23.352 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.8216000000000001e-05 2024-09-16T09:27:23.352 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:27:23.352 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:27:23.352 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:27:23.352 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:27:23.352 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:27:23.352 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T09:27:23.375 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:27:23.414 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:23.415 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:23.415 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:27:23.516 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:27:23.547 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:23.547 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:23.548 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:27:23.648 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:27:23.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:23 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:27:23.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:23 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:27:23.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:23 smithi084 ceph-mon[32629]: 7.b scrub ok 2024-09-16T09:27:23.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:23 smithi084 ceph-mon[32629]: osdmap e111: 8 total, 8 up, 8 in 2024-09-16T09:27:23.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:23 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:27:23.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:23 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1977840317' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:27:23.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:23 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:27:23.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:23 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:27:23.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:23 smithi070 ceph-mon[30388]: 7.b scrub ok 2024-09-16T09:27:23.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:23 smithi070 ceph-mon[30388]: osdmap e111: 8 total, 8 up, 8 in 2024-09-16T09:27:23.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:23 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:27:23.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:23 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1977840317' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:27:23.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:23 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:27:23.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:23 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:27:23.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:23 smithi070 ceph-mon[40691]: 7.b scrub ok 2024-09-16T09:27:23.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:23 smithi070 ceph-mon[40691]: osdmap e111: 8 total, 8 up, 8 in 2024-09-16T09:27:23.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:23 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:27:23.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:23 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1977840317' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:27:23.680 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:23.681 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:23.681 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:27:23.782 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:27:23.811 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:23.812 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:23.812 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:27:23.913 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:27:23.955 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:23.955 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:23.955 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:27:24.030 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 71 16 105714 105698 93.0079 70.625 0.0110981 0.0107469 2024-09-16T09:27:24.056 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:27:24.083 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:24.084 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:24.084 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:27:24.185 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:27:24.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:24.218 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:24.218 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:27:24.319 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:27:24.349 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:24.350 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:24.350 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:27:24.451 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:27:24.477 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:24.478 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:24.478 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:27:24.579 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:27:24.606 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:24.607 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:24.607 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:27:24.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:24 smithi070 ceph-mon[30388]: 3.b scrub starts 2024-09-16T09:27:24.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:24 smithi070 ceph-mon[30388]: 3.b scrub ok 2024-09-16T09:27:24.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:24 smithi070 ceph-mon[30388]: pgmap v97: 158 pgs: 158 active+clean; 6.4 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 1.4 KiB/s rd, 91 MiB/s wr, 1.45k op/s 2024-09-16T09:27:24.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:24 smithi070 ceph-mon[30388]: osdmap e112: 8 total, 8 up, 8 in 2024-09-16T09:27:24.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:24 smithi070 ceph-mon[40691]: 3.b scrub starts 2024-09-16T09:27:24.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:24 smithi070 ceph-mon[40691]: 3.b scrub ok 2024-09-16T09:27:24.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:24 smithi070 ceph-mon[40691]: pgmap v97: 158 pgs: 158 active+clean; 6.4 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 1.4 KiB/s rd, 91 MiB/s wr, 1.45k op/s 2024-09-16T09:27:24.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:24 smithi070 ceph-mon[40691]: osdmap e112: 8 total, 8 up, 8 in 2024-09-16T09:27:24.708 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:27:24.739 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:24.739 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:24.740 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:27:24.841 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:27:24.887 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.4 dump_ops_in_flight 2024-09-16T09:27:24.895 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:24.896 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:24.896 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:27:24.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:24 smithi084 ceph-mon[32629]: 3.b scrub starts 2024-09-16T09:27:24.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:24 smithi084 ceph-mon[32629]: 3.b scrub ok 2024-09-16T09:27:24.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:24 smithi084 ceph-mon[32629]: pgmap v97: 158 pgs: 158 active+clean; 6.4 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 1.4 KiB/s rd, 91 MiB/s wr, 1.45k op/s 2024-09-16T09:27:24.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:24 smithi084 ceph-mon[32629]: osdmap e112: 8 total, 8 up, 8 in 2024-09-16T09:27:24.997 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:27:25.030 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 72 16 106871 106855 92.7205 72.3125 0.0131243 0.0107796 2024-09-16T09:27:25.032 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:25.033 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:25.034 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:27:25.134 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:27:25.173 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:25.174 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:25.174 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:27:25.255 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:27:25.276 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:27:25.321 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:25.322 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:25.322 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:27:25.423 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:27:25.470 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:25.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:25.471 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:27:25.572 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:27:25.629 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:25.630 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:25.630 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:27:25.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:25 smithi084 ceph-mon[32629]: 6.c scrub starts 2024-09-16T09:27:25.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:25 smithi084 ceph-mon[32629]: 6.c scrub ok 2024-09-16T09:27:25.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:25 smithi084 ceph-mon[32629]: 3.4 scrub starts 2024-09-16T09:27:25.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:25 smithi084 ceph-mon[32629]: 3.4 scrub ok 2024-09-16T09:27:25.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:25 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:27:25.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:25 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:27:25.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:25 smithi084 ceph-mon[32629]: 7.6 scrub starts 2024-09-16T09:27:25.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:25 smithi084 ceph-mon[32629]: 3.11 scrub starts 2024-09-16T09:27:25.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:25 smithi084 ceph-mon[32629]: 3.11 scrub ok 2024-09-16T09:27:25.670 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:25 smithi070 ceph-mon[40691]: 6.c scrub starts 2024-09-16T09:27:25.670 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:25 smithi070 ceph-mon[40691]: 6.c scrub ok 2024-09-16T09:27:25.670 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:25 smithi070 ceph-mon[40691]: 3.4 scrub starts 2024-09-16T09:27:25.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:25 smithi070 ceph-mon[40691]: 3.4 scrub ok 2024-09-16T09:27:25.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:25 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:27:25.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:25 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:27:25.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:25 smithi070 ceph-mon[40691]: 7.6 scrub starts 2024-09-16T09:27:25.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:25 smithi070 ceph-mon[40691]: 3.11 scrub starts 2024-09-16T09:27:25.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:25 smithi070 ceph-mon[40691]: 3.11 scrub ok 2024-09-16T09:27:25.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:25 smithi070 ceph-mon[30388]: 6.c scrub starts 2024-09-16T09:27:25.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:25 smithi070 ceph-mon[30388]: 6.c scrub ok 2024-09-16T09:27:25.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:25 smithi070 ceph-mon[30388]: 3.4 scrub starts 2024-09-16T09:27:25.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:25 smithi070 ceph-mon[30388]: 3.4 scrub ok 2024-09-16T09:27:25.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:25 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:27:25.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:25 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:27:25.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:25 smithi070 ceph-mon[30388]: 7.6 scrub starts 2024-09-16T09:27:25.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:25 smithi070 ceph-mon[30388]: 3.11 scrub starts 2024-09-16T09:27:25.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:25 smithi070 ceph-mon[30388]: 3.11 scrub ok 2024-09-16T09:27:25.732 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:27:25.778 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:25.779 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:25.779 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:27:25.838 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-09-16T09:27:25.838 INFO:teuthology.orchestra.run.smithi084.stdout: "ops": [ 2024-09-16T09:27:25.838 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:25.838 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:107640 7.1 7:8fa84824:::benchmark_data_smithi070_78438_object107639:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e112)", 2024-09-16T09:27:25.838 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:27:25.830887+0000", 2024-09-16T09:27:25.839 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 0.0055528820000000003, 2024-09-16T09:27:25.839 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0055765229999999999, 2024-09-16T09:27:25.839 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:27:25.839 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:27:25.839 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "waiting for sub ops", 2024-09-16T09:27:25.839 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:27:25.839 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:27:25.839 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:27:25.839 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 107640 2024-09-16T09:27:25.839 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:25.840 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:27:25.840 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:25.840 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:27:25.840 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:25.830887+0000", 2024-09-16T09:27:25.840 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:27:25.840 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:25.840 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:25.840 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:27:25.840 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:25.830887+0000", 2024-09-16T09:27:25.840 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:27:25.840 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:25.840 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:25.841 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:27:25.841 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:25.830889+0000", 2024-09-16T09:27:25.841 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.725e-06 2024-09-16T09:27:25.841 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:25.841 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:25.841 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:27:25.841 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:25.831053+0000", 2024-09-16T09:27:25.841 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00016371300000000001 2024-09-16T09:27:25.841 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:25.841 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:25.841 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:27:25.841 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:25.831066+0000", 2024-09-16T09:27:25.841 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.3783e-05 2024-09-16T09:27:25.842 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:25.842 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:25.842 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:27:25.842 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:25.831075+0000", 2024-09-16T09:27:25.842 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.1519999999999992e-06 2024-09-16T09:27:25.842 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:25.842 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:25.842 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:27:25.842 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:25.831136+0000", 2024-09-16T09:27:25.842 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.1299999999999999e-05 2024-09-16T09:27:25.842 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:25.842 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:25.842 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:27:25.842 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:25.831263+0000", 2024-09-16T09:27:25.842 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00012664100000000001 2024-09-16T09:27:25.842 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:25.843 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:25.843 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 3,6", 2024-09-16T09:27:25.843 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:25.831408+0000", 2024-09-16T09:27:25.843 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00014520200000000001 2024-09-16T09:27:25.843 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:25.843 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:25.843 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:27:25.843 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:25.834376+0000", 2024-09-16T09:27:25.844 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0029679929999999999 2024-09-16T09:27:25.844 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:25.844 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:25.844 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:27:25.844 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:25.834388+0000", 2024-09-16T09:27:25.844 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.2028000000000001e-05 2024-09-16T09:27:25.844 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:27:25.844 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:27:25.844 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:27:25.844 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:25.844 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:25.845 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_repop(client.24698.0:107642 7.4 e112/82 7:29a92f4c:::benchmark_data_smithi070_78438_object107641:head v 112'6647, mlcod=112'6644)", 2024-09-16T09:27:25.845 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:27:25.835031+0000", 2024-09-16T09:27:25.845 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 0.0014087030000000001, 2024-09-16T09:27:25.845 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0014776590000000001, 2024-09-16T09:27:25.845 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:27:25.845 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:27:25.845 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "started", 2024-09-16T09:27:25.845 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:27:25.845 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:25.845 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:27:25.845 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:25.835031+0000", 2024-09-16T09:27:25.846 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:27:25.846 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:25.846 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:25.846 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:27:25.846 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:25.835031+0000", 2024-09-16T09:27:25.846 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:27:25.846 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:25.846 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:25.846 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:27:25.846 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:25.835032+0000", 2024-09-16T09:27:25.847 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.8500000000000001e-07 2024-09-16T09:27:25.847 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:25.847 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:25.847 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:27:25.847 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:25.835085+0000", 2024-09-16T09:27:25.847 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.3307999999999999e-05 2024-09-16T09:27:25.847 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:25.848 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:25.848 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:27:25.848 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:25.835097+0000", 2024-09-16T09:27:25.848 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1151999999999999e-05 2024-09-16T09:27:25.848 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:25.848 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:25.849 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:27:25.849 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:25.835103+0000", 2024-09-16T09:27:25.849 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.3180000000000003e-06 2024-09-16T09:27:25.849 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:25.850 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:25.850 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:27:25.850 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:25.835147+0000", 2024-09-16T09:27:25.850 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.3730999999999999e-05 2024-09-16T09:27:25.850 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:25.850 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:25.851 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:27:25.851 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:25.835160+0000", 2024-09-16T09:27:25.851 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.3196999999999999e-05 2024-09-16T09:27:25.851 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:27:25.851 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:27:25.851 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:27:25.851 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:27:25.852 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-09-16T09:27:25.852 INFO:teuthology.orchestra.run.smithi084.stdout: "num_ops": 2 2024-09-16T09:27:25.852 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-09-16T09:27:25.880 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:27:25.913 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:25.914 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:25.915 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:27:26.016 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:27:26.030 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 73 16 107869 107853 92.3049 62.375 0.020762 0.0108293 2024-09-16T09:27:26.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:26.053 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:26.053 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:27:26.064 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:27:26.154 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:27:26.203 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:26.204 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:26.204 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:27:26.305 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:27:26.333 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:26.333 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:26.333 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:27:26.434 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:27:26.454 INFO:teuthology.orchestra.run.smithi070.stderr:noscrub is set 2024-09-16T09:27:26.460 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:26.460 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:26.461 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:27:26.562 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:27:26.579 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.4 dump_blocked_ops 2024-09-16T09:27:26.594 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:26.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:26.595 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:27:26.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:26 smithi070 ceph-mon[30388]: pgmap v99: 158 pgs: 158 active+clean; 6.4 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 800 B/s rd, 50 MiB/s wr, 807 op/s 2024-09-16T09:27:26.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:26 smithi070 ceph-mon[30388]: 4.d scrub starts 2024-09-16T09:27:26.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:26 smithi070 ceph-mon[30388]: 4.d scrub ok 2024-09-16T09:27:26.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:26 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/256183891' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:27:26.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:26 smithi070 ceph-mon[30388]: 6.8 scrub starts 2024-09-16T09:27:26.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:26 smithi070 ceph-mon[30388]: 6.8 scrub ok 2024-09-16T09:27:26.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:26 smithi070 ceph-mon[40691]: pgmap v99: 158 pgs: 158 active+clean; 6.4 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 800 B/s rd, 50 MiB/s wr, 807 op/s 2024-09-16T09:27:26.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:26 smithi070 ceph-mon[40691]: 4.d scrub starts 2024-09-16T09:27:26.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:26 smithi070 ceph-mon[40691]: 4.d scrub ok 2024-09-16T09:27:26.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:26 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/256183891' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:27:26.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:26 smithi070 ceph-mon[40691]: 6.8 scrub starts 2024-09-16T09:27:26.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:26 smithi070 ceph-mon[40691]: 6.8 scrub ok 2024-09-16T09:27:26.696 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:27:26.721 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:26.722 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:26.722 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:27:26.824 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:27:26.864 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:26.864 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:26.864 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:27:26.912 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:26 smithi084 ceph-mon[32629]: pgmap v99: 158 pgs: 158 active+clean; 6.4 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 800 B/s rd, 50 MiB/s wr, 807 op/s 2024-09-16T09:27:26.912 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:26 smithi084 ceph-mon[32629]: 4.d scrub starts 2024-09-16T09:27:26.912 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:26 smithi084 ceph-mon[32629]: 4.d scrub ok 2024-09-16T09:27:26.912 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:26 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/256183891' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:27:26.913 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:26 smithi084 ceph-mon[32629]: 6.8 scrub starts 2024-09-16T09:27:26.913 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:26 smithi084 ceph-mon[32629]: 6.8 scrub ok 2024-09-16T09:27:26.945 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:27:26.965 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:27:27.005 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:27.005 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:27.005 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:27:27.030 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 74 16 108966 108950 91.984 68.5625 0.00592854 0.0108666 2024-09-16T09:27:27.106 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:27:27.148 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:27.149 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:27.149 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:27:27.250 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:27:27.284 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:27.285 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:27.285 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:27:27.386 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:27:27.413 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:27.414 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:27.415 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:27:27.515 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:27:27.542 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:27.543 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:27.543 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:27:27.644 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:27:27.684 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:27.685 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:27.685 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:27:27.786 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:27:27.790 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:27 smithi070 ceph-mon[30388]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:27:27.790 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:27 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/256183891' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-16T09:27:27.790 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:27 smithi070 ceph-mon[30388]: osdmap e113: 8 total, 8 up, 8 in 2024-09-16T09:27:27.790 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:27 smithi070 ceph-mon[40691]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:27:27.790 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:27 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/256183891' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-16T09:27:27.790 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:27 smithi070 ceph-mon[40691]: osdmap e113: 8 total, 8 up, 8 in 2024-09-16T09:27:27.831 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:27.832 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:27.832 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:27:27.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:27 smithi084 ceph-mon[32629]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:27:27.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:27 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/256183891' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-16T09:27:27.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:27 smithi084 ceph-mon[32629]: osdmap e113: 8 total, 8 up, 8 in 2024-09-16T09:27:27.933 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:27:27.978 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:27.979 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:27.979 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:27:27.994 INFO:teuthology.orchestra.run.smithi070.stderr:no valid command found; 10 closest matches: 2024-09-16T09:27:27.995 INFO:teuthology.orchestra.run.smithi070.stderr:0 2024-09-16T09:27:27.995 INFO:teuthology.orchestra.run.smithi070.stderr:1 2024-09-16T09:27:27.995 INFO:teuthology.orchestra.run.smithi070.stderr:2 2024-09-16T09:27:27.995 INFO:teuthology.orchestra.run.smithi070.stderr:abort 2024-09-16T09:27:27.995 INFO:teuthology.orchestra.run.smithi070.stderr:assert 2024-09-16T09:27:27.995 INFO:teuthology.orchestra.run.smithi070.stderr:bench [] [] [] [] 2024-09-16T09:27:27.995 INFO:teuthology.orchestra.run.smithi070.stderr:bluefs debug_inject_read_zeros 2024-09-16T09:27:27.995 INFO:teuthology.orchestra.run.smithi070.stderr:bluefs files list 2024-09-16T09:27:27.995 INFO:teuthology.orchestra.run.smithi070.stderr:bluefs stats 2024-09-16T09:27:27.995 INFO:teuthology.orchestra.run.smithi070.stderr:bluestore allocator dump block 2024-09-16T09:27:27.995 INFO:teuthology.orchestra.run.smithi070.stderr:Error EINVAL: invalid command 2024-09-16T09:27:28.030 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 75 16 110119 110103 91.7184 72.0625 0.0149133 0.0108979 2024-09-16T09:27:28.080 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:27:28.101 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T09:27:28.105 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:28.106 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:28.106 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:27:28.207 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:27:28.232 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:28.233 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:28.233 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:27:28.334 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:27:28.360 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:28.361 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:28.361 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:27:28.462 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:27:28.518 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:28.519 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:28.519 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:27:28.620 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:27:28.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:28 smithi084 ceph-mon[32629]: pgmap v101: 158 pgs: 158 active+clean; 6.5 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.7 KiB/s rd, 75 MiB/s wr, 1.20k op/s 2024-09-16T09:27:28.665 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:28.667 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:28.667 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:27:28.768 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:27:28.790 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-09-16T09:27:28.791 INFO:teuthology.orchestra.run.smithi084.stdout: "ops": [], 2024-09-16T09:27:28.791 INFO:teuthology.orchestra.run.smithi084.stdout: "complaint_time": 30, 2024-09-16T09:27:28.791 INFO:teuthology.orchestra.run.smithi084.stdout: "num_blocked_ops": 0 2024-09-16T09:27:28.791 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-09-16T09:27:28.815 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:28.816 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:28.816 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:27:28.917 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:27:28.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:28 smithi070 ceph-mon[30388]: pgmap v101: 158 pgs: 158 active+clean; 6.5 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.7 KiB/s rd, 75 MiB/s wr, 1.20k op/s 2024-09-16T09:27:28.920 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:28 smithi070 ceph-mon[40691]: pgmap v101: 158 pgs: 158 active+clean; 6.5 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.7 KiB/s rd, 75 MiB/s wr, 1.20k op/s 2024-09-16T09:27:28.943 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:28.943 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:28.944 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:27:29.030 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 76 16 111224 111208 91.4203 69.0625 0.00391713 0.0109332 2024-09-16T09:27:29.045 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:27:29.094 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-16T09:27:29.095 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-16T09:27:29.095 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph tell osd.6 'trim stale osdmaps' 2024-09-16T09:27:29.096 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:29.097 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:29.097 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:27:29.198 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:27:29.220 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:29.221 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:29.221 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:27:29.322 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:27:29.349 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:29.349 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:29.350 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:27:29.450 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:27:29.478 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:29.479 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:29.480 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:27:29.539 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.4 dump_historic_ops 2024-09-16T09:27:29.581 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:27:29.617 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:29.618 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:29.619 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:27:29.658 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-16T09:27:29.719 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:27:29.761 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:29.762 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:29.762 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:27:29.863 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:27:29.884 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:27:29.894 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:29.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:29.895 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:27:29.996 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:27:30.034 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 77 15 112226 112211 91.0428 62.6875 0.0113375 0.010979 2024-09-16T09:27:30.049 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:30.050 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:30.051 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:27:30.151 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:27:30.194 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:30.195 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:30.195 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:27:30.269 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:30 smithi084 ceph-mon[32629]: osdmap e114: 8 total, 8 up, 8 in 2024-09-16T09:27:30.296 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:27:30.336 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:30.337 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:30.337 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:27:30.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:30 smithi070 ceph-mon[30388]: osdmap e114: 8 total, 8 up, 8 in 2024-09-16T09:27:30.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:30 smithi070 ceph-mon[40691]: osdmap e114: 8 total, 8 up, 8 in 2024-09-16T09:27:30.438 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:27:30.469 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:30.470 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:30.470 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:27:30.515 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-09-16T09:27:30.516 INFO:teuthology.orchestra.run.smithi084.stdout: "size": 20, 2024-09-16T09:27:30.516 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 600, 2024-09-16T09:27:30.516 INFO:teuthology.orchestra.run.smithi084.stdout: "ops": [ 2024-09-16T09:27:30.516 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.516 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:22925 7.1 7:89c34711:::benchmark_data_smithi070_78438_object22924:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T09:27:30.516 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:26:21.580979+0000", 2024-09-16T09:27:30.516 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 68.931342263000005, 2024-09-16T09:27:30.516 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.076893300999999997, 2024-09-16T09:27:30.516 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:27:30.516 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:27:30.517 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:27:30.517 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:27:30.517 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:27:30.517 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:27:30.517 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 22925 2024-09-16T09:27:30.517 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.517 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:27:30.517 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.517 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:27:30.517 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.580979+0000", 2024-09-16T09:27:30.517 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:27:30.517 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.517 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.518 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:27:30.518 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.580979+0000", 2024-09-16T09:27:30.518 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:27:30.518 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.518 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.518 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:27:30.518 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.580981+0000", 2024-09-16T09:27:30.518 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.3629999999999999e-06 2024-09-16T09:27:30.518 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.518 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.518 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:27:30.518 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.581034+0000", 2024-09-16T09:27:30.519 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.3300999999999997e-05 2024-09-16T09:27:30.519 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.519 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.519 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:27:30.519 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.581051+0000", 2024-09-16T09:27:30.519 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.6642000000000001e-05 2024-09-16T09:27:30.519 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.519 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.519 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:27:30.519 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.581059+0000", 2024-09-16T09:27:30.519 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.7970000000000001e-06 2024-09-16T09:27:30.519 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.519 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.520 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:27:30.520 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.581280+0000", 2024-09-16T09:27:30.520 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000221752 2024-09-16T09:27:30.520 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.520 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.520 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:27:30.520 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.581389+0000", 2024-09-16T09:27:30.520 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00010817799999999999 2024-09-16T09:27:30.520 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.520 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.520 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 3,6", 2024-09-16T09:27:30.520 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.581517+0000", 2024-09-16T09:27:30.521 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000128128 2024-09-16T09:27:30.521 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.521 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.521 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:27:30.521 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.584179+0000", 2024-09-16T09:27:30.521 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0026624830000000002 2024-09-16T09:27:30.521 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.521 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.521 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:27:30.521 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.585833+0000", 2024-09-16T09:27:30.521 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0016537520000000001 2024-09-16T09:27:30.521 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.521 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.522 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:27:30.522 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.657746+0000", 2024-09-16T09:27:30.522 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.071913191000000001 2024-09-16T09:27:30.522 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.522 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.522 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:27:30.522 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.657816+0000", 2024-09-16T09:27:30.522 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.9633000000000002e-05 2024-09-16T09:27:30.522 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.522 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.522 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:27:30.522 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.657873+0000", 2024-09-16T09:27:30.523 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.7080999999999998e-05 2024-09-16T09:27:30.523 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:27:30.523 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:27:30.523 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:27:30.523 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.523 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.523 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:22927 7.1 7:8fa982a9:::benchmark_data_smithi070_78438_object22926:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T09:27:30.523 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:26:21.582665+0000", 2024-09-16T09:27:30.523 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 68.929656279, 2024-09-16T09:27:30.523 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.077694033999999995, 2024-09-16T09:27:30.523 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:27:30.523 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:27:30.523 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:27:30.524 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:27:30.524 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:27:30.524 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:27:30.524 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 22927 2024-09-16T09:27:30.524 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.524 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:27:30.524 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.524 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:27:30.524 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.582665+0000", 2024-09-16T09:27:30.524 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:27:30.524 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.524 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.525 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:27:30.525 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.582665+0000", 2024-09-16T09:27:30.525 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:27:30.525 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.525 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.525 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:27:30.525 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.582667+0000", 2024-09-16T09:27:30.525 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.5179999999999999e-06 2024-09-16T09:27:30.525 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.525 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.525 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:27:30.526 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.582700+0000", 2024-09-16T09:27:30.526 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.2821000000000003e-05 2024-09-16T09:27:30.526 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.526 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.526 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:27:30.526 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.582719+0000", 2024-09-16T09:27:30.526 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.9255000000000001e-05 2024-09-16T09:27:30.526 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.526 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.526 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:27:30.526 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.582727+0000", 2024-09-16T09:27:30.527 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.661e-06 2024-09-16T09:27:30.527 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.527 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.527 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:27:30.527 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.582801+0000", 2024-09-16T09:27:30.527 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.4553000000000003e-05 2024-09-16T09:27:30.527 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.527 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.527 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:27:30.527 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.582991+0000", 2024-09-16T09:27:30.527 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00018939400000000001 2024-09-16T09:27:30.527 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.528 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.528 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 3,6", 2024-09-16T09:27:30.528 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.583125+0000", 2024-09-16T09:27:30.528 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00013424800000000001 2024-09-16T09:27:30.528 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.528 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.528 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:27:30.528 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.585851+0000", 2024-09-16T09:27:30.528 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0027256799999999999 2024-09-16T09:27:30.528 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.528 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.528 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:27:30.528 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.585910+0000", 2024-09-16T09:27:30.529 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.9552999999999997e-05 2024-09-16T09:27:30.529 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.529 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.529 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:27:30.529 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.660257+0000", 2024-09-16T09:27:30.529 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.074347094000000002 2024-09-16T09:27:30.529 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.529 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.529 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:27:30.529 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.660316+0000", 2024-09-16T09:27:30.529 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.8595999999999999e-05 2024-09-16T09:27:30.529 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.529 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.530 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:27:30.530 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.660360+0000", 2024-09-16T09:27:30.530 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.3661000000000001e-05 2024-09-16T09:27:30.530 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:27:30.530 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:27:30.530 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:27:30.530 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.530 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.530 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:22934 7.1 7:87f3da4d:::benchmark_data_smithi070_78438_object22933:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T09:27:30.530 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:26:21.583328+0000", 2024-09-16T09:27:30.530 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 68.928993461999994, 2024-09-16T09:27:30.530 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.077158353999999998, 2024-09-16T09:27:30.530 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:27:30.530 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:27:30.532 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:27:30.532 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:27:30.532 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:27:30.532 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:27:30.532 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 22934 2024-09-16T09:27:30.532 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.532 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:27:30.532 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.532 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:27:30.532 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.583328+0000", 2024-09-16T09:27:30.532 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:27:30.532 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.533 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.533 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:27:30.533 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.583328+0000", 2024-09-16T09:27:30.533 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:27:30.533 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.533 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.533 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:27:30.533 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.583329+0000", 2024-09-16T09:27:30.533 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0070000000000001e-06 2024-09-16T09:27:30.533 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.533 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.533 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:27:30.533 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.583429+0000", 2024-09-16T09:27:30.534 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.9981000000000004e-05 2024-09-16T09:27:30.534 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.534 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.534 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:27:30.534 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.583441+0000", 2024-09-16T09:27:30.534 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.2196e-05 2024-09-16T09:27:30.534 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.534 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.534 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:27:30.534 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.583448+0000", 2024-09-16T09:27:30.534 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.8460000000000002e-06 2024-09-16T09:27:30.534 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.534 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.534 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:27:30.535 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.584242+0000", 2024-09-16T09:27:30.535 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00079362599999999997 2024-09-16T09:27:30.535 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.535 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.535 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:27:30.535 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.584361+0000", 2024-09-16T09:27:30.535 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000119014 2024-09-16T09:27:30.535 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.535 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.535 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 3,6", 2024-09-16T09:27:30.535 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.584493+0000", 2024-09-16T09:27:30.535 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000132473 2024-09-16T09:27:30.535 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.536 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.536 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:27:30.536 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.585942+0000", 2024-09-16T09:27:30.536 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.001448509 2024-09-16T09:27:30.536 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.536 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.536 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:27:30.536 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.585970+0000", 2024-09-16T09:27:30.536 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.7932e-05 2024-09-16T09:27:30.536 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.536 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.536 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:27:30.536 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.660424+0000", 2024-09-16T09:27:30.537 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.074454300000000001 2024-09-16T09:27:30.537 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.537 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.537 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:27:30.537 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.660462+0000", 2024-09-16T09:27:30.537 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.8302e-05 2024-09-16T09:27:30.537 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.537 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.537 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:27:30.537 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.660487+0000", 2024-09-16T09:27:30.537 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.4168e-05 2024-09-16T09:27:30.537 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:27:30.538 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:27:30.538 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:27:30.538 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.538 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.538 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:26638 7.1 7:8fd77d04:::benchmark_data_smithi070_78438_object26637:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T09:27:30.538 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:26:23.276469+0000", 2024-09-16T09:27:30.538 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 67.235853062999993, 2024-09-16T09:27:30.538 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.051045984000000003, 2024-09-16T09:27:30.538 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:27:30.538 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:27:30.538 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:27:30.538 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:27:30.538 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:27:30.539 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:27:30.539 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 26638 2024-09-16T09:27:30.539 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.539 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:27:30.539 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.539 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:27:30.539 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.276469+0000", 2024-09-16T09:27:30.539 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:27:30.539 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.539 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.539 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:27:30.539 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.276469+0000", 2024-09-16T09:27:30.539 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:27:30.539 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.540 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.540 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:27:30.540 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.276470+0000", 2024-09-16T09:27:30.540 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.265e-06 2024-09-16T09:27:30.540 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.540 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.540 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:27:30.540 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.276723+0000", 2024-09-16T09:27:30.540 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00025323799999999998 2024-09-16T09:27:30.540 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.540 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.540 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:27:30.540 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.276735+0000", 2024-09-16T09:27:30.541 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.2201999999999999e-05 2024-09-16T09:27:30.541 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.541 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.541 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:27:30.541 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.276742+0000", 2024-09-16T09:27:30.541 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.097e-06 2024-09-16T09:27:30.541 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.541 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.541 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:27:30.541 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.276795+0000", 2024-09-16T09:27:30.541 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.2945000000000002e-05 2024-09-16T09:27:30.541 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.541 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.541 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:27:30.542 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.276912+0000", 2024-09-16T09:27:30.542 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00011658999999999999 2024-09-16T09:27:30.542 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.542 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.542 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 3,6", 2024-09-16T09:27:30.542 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.277036+0000", 2024-09-16T09:27:30.542 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00012364599999999999 2024-09-16T09:27:30.542 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.542 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.542 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:27:30.542 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.278637+0000", 2024-09-16T09:27:30.542 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0016011269999999999 2024-09-16T09:27:30.542 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.542 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.543 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:27:30.543 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.280947+0000", 2024-09-16T09:27:30.543 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0023097479999999999 2024-09-16T09:27:30.543 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.543 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.543 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:27:30.543 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.327359+0000", 2024-09-16T09:27:30.543 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.046412002000000001 2024-09-16T09:27:30.543 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.543 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.543 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:27:30.543 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.327462+0000", 2024-09-16T09:27:30.543 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000103546 2024-09-16T09:27:30.543 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.544 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.544 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:27:30.544 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.327515+0000", 2024-09-16T09:27:30.544 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.2577999999999997e-05 2024-09-16T09:27:30.544 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:27:30.544 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:27:30.544 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:27:30.544 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.544 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.544 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:26640 7.1 7:89612531:::benchmark_data_smithi070_78438_object26639:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T09:27:30.544 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:26:23.277543+0000", 2024-09-16T09:27:30.544 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 67.234778829000007, 2024-09-16T09:27:30.544 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.050231568999999997, 2024-09-16T09:27:30.544 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:27:30.545 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:27:30.545 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:27:30.545 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:27:30.545 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:27:30.545 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:27:30.545 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 26640 2024-09-16T09:27:30.545 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.545 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:27:30.545 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.545 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:27:30.545 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.277543+0000", 2024-09-16T09:27:30.545 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:27:30.545 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.545 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.546 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:27:30.546 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.277543+0000", 2024-09-16T09:27:30.546 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:27:30.546 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.546 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.546 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:27:30.546 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.277544+0000", 2024-09-16T09:27:30.546 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.156e-06 2024-09-16T09:27:30.546 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.546 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.546 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:27:30.546 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.277597+0000", 2024-09-16T09:27:30.546 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.2787000000000001e-05 2024-09-16T09:27:30.546 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.546 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.546 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:27:30.547 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.277608+0000", 2024-09-16T09:27:30.547 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1629e-05 2024-09-16T09:27:30.547 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.547 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.547 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:27:30.547 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.277614+0000", 2024-09-16T09:27:30.547 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.3709999999999999e-06 2024-09-16T09:27:30.547 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.547 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.547 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:27:30.547 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.277966+0000", 2024-09-16T09:27:30.547 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00035202500000000001 2024-09-16T09:27:30.547 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.547 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.547 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:27:30.548 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.278051+0000", 2024-09-16T09:27:30.548 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.4864999999999995e-05 2024-09-16T09:27:30.548 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.548 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.548 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 3,6", 2024-09-16T09:27:30.548 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.278126+0000", 2024-09-16T09:27:30.548 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.5603000000000001e-05 2024-09-16T09:27:30.548 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.548 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.548 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:27:30.548 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.278765+0000", 2024-09-16T09:27:30.548 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00063888499999999995 2024-09-16T09:27:30.549 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.549 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.549 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:27:30.549 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.281605+0000", 2024-09-16T09:27:30.549 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0028394940000000001 2024-09-16T09:27:30.549 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.549 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.549 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:27:30.549 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.327684+0000", 2024-09-16T09:27:30.549 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.046079684000000003 2024-09-16T09:27:30.549 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.549 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.549 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:27:30.550 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.327737+0000", 2024-09-16T09:27:30.550 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.2290000000000002e-05 2024-09-16T09:27:30.550 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.550 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.550 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:27:30.550 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.327774+0000", 2024-09-16T09:27:30.550 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.7780000000000001e-05 2024-09-16T09:27:30.550 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:27:30.550 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:27:30.550 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:27:30.550 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.550 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.550 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:27462 7.1 7:8b37553f:::benchmark_data_smithi070_78438_object27461:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T09:27:30.551 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:26:23.719582+0000", 2024-09-16T09:27:30.551 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 66.792739874000006, 2024-09-16T09:27:30.551 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.073272959999999998, 2024-09-16T09:27:30.551 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:27:30.551 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:27:30.551 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:27:30.551 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:27:30.551 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:27:30.551 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:27:30.551 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 27462 2024-09-16T09:27:30.551 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.551 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:27:30.551 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.552 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:27:30.552 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.719582+0000", 2024-09-16T09:27:30.552 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:27:30.552 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.552 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.552 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:27:30.552 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.719582+0000", 2024-09-16T09:27:30.552 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:27:30.552 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.552 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.552 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:27:30.552 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.719583+0000", 2024-09-16T09:27:30.553 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.192e-06 2024-09-16T09:27:30.553 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.553 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.553 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:27:30.553 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.719694+0000", 2024-09-16T09:27:30.553 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000110592 2024-09-16T09:27:30.553 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.553 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.553 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:27:30.553 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.719702+0000", 2024-09-16T09:27:30.553 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.3610000000000001e-06 2024-09-16T09:27:30.553 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.554 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.554 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:27:30.554 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.719707+0000", 2024-09-16T09:27:30.554 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.0640000000000001e-06 2024-09-16T09:27:30.554 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.554 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.554 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:27:30.554 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.719811+0000", 2024-09-16T09:27:30.554 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00010433399999999999 2024-09-16T09:27:30.554 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.554 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.554 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:27:30.555 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.719878+0000", 2024-09-16T09:27:30.555 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.6651999999999998e-05 2024-09-16T09:27:30.555 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.555 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.555 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 3,6", 2024-09-16T09:27:30.555 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.719964+0000", 2024-09-16T09:27:30.555 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.5840999999999993e-05 2024-09-16T09:27:30.555 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.555 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.555 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:27:30.555 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.789718+0000", 2024-09-16T09:27:30.555 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.069754134999999995 2024-09-16T09:27:30.555 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.555 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.556 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:27:30.556 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.792737+0000", 2024-09-16T09:27:30.556 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0030193170000000001 2024-09-16T09:27:30.556 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.556 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.556 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:27:30.556 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.792756+0000", 2024-09-16T09:27:30.556 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.8240000000000002e-05 2024-09-16T09:27:30.556 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.556 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.556 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:27:30.556 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.792812+0000", 2024-09-16T09:27:30.557 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.6081000000000001e-05 2024-09-16T09:27:30.557 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.557 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.557 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:27:30.557 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.792855+0000", 2024-09-16T09:27:30.557 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.3151000000000001e-05 2024-09-16T09:27:30.557 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:27:30.557 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:27:30.557 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:27:30.557 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.557 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.557 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_repop_reply(client.24698.0:27462 7.1 e89/82 ondisk, result = 0)", 2024-09-16T09:27:30.558 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:26:23.721262+0000", 2024-09-16T09:27:30.558 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 66.791059939999997, 2024-09-16T09:27:30.558 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.068463016000000002, 2024-09-16T09:27:30.558 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:27:30.558 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:27:30.558 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "started", 2024-09-16T09:27:30.558 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:27:30.558 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.558 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:27:30.558 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.721262+0000", 2024-09-16T09:27:30.558 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:27:30.558 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.559 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.559 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:27:30.559 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.721262+0000", 2024-09-16T09:27:30.559 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:27:30.559 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.559 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.559 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:27:30.559 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.721262+0000", 2024-09-16T09:27:30.559 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.3900000000000005e-07 2024-09-16T09:27:30.559 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.559 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.559 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:27:30.559 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.721266+0000", 2024-09-16T09:27:30.559 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.349e-06 2024-09-16T09:27:30.560 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.560 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.560 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:27:30.560 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.787935+0000", 2024-09-16T09:27:30.560 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.066669333999999997 2024-09-16T09:27:30.560 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.560 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.560 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:27:30.560 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.787946+0000", 2024-09-16T09:27:30.560 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1358999999999999e-05 2024-09-16T09:27:30.560 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.560 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.560 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:27:30.561 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.789705+0000", 2024-09-16T09:27:30.561 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.001758721 2024-09-16T09:27:30.561 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.561 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.561 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:27:30.561 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.789713+0000", 2024-09-16T09:27:30.561 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.3989999999999998e-06 2024-09-16T09:27:30.561 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.561 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.561 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:27:30.561 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.789725+0000", 2024-09-16T09:27:30.561 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.2315e-05 2024-09-16T09:27:30.562 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:27:30.562 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:27:30.562 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:27:30.562 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.562 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.562 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:27993 7.1 7:889d1acf:::benchmark_data_smithi070_78438_object27992:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T09:27:30.562 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:26:24.011608+0000", 2024-09-16T09:27:30.562 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 66.500713696000005, 2024-09-16T09:27:30.562 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.049084309999999999, 2024-09-16T09:27:30.562 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:27:30.562 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:27:30.562 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:27:30.562 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:27:30.563 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:27:30.563 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:27:30.563 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 27993 2024-09-16T09:27:30.563 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.563 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:27:30.563 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.563 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:27:30.563 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.011608+0000", 2024-09-16T09:27:30.563 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:27:30.563 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.563 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.563 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:27:30.563 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.011608+0000", 2024-09-16T09:27:30.564 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:27:30.564 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.564 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.564 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:27:30.564 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.011609+0000", 2024-09-16T09:27:30.564 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.33e-07 2024-09-16T09:27:30.564 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.564 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.564 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:27:30.564 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.011668+0000", 2024-09-16T09:27:30.564 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.8554999999999998e-05 2024-09-16T09:27:30.564 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.564 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.565 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:27:30.565 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.011675+0000", 2024-09-16T09:27:30.565 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.5190000000000003e-06 2024-09-16T09:27:30.565 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.565 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.565 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:27:30.565 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.011679+0000", 2024-09-16T09:27:30.565 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.3340000000000003e-06 2024-09-16T09:27:30.565 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.565 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.565 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:27:30.565 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.011709+0000", 2024-09-16T09:27:30.565 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.9938e-05 2024-09-16T09:27:30.566 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.566 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.566 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:27:30.566 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.011796+0000", 2024-09-16T09:27:30.566 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.6501000000000004e-05 2024-09-16T09:27:30.566 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.566 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.566 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 3,6", 2024-09-16T09:27:30.566 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.011871+0000", 2024-09-16T09:27:30.566 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.5537000000000005e-05 2024-09-16T09:27:30.566 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.566 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.566 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:27:30.567 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.012744+0000", 2024-09-16T09:27:30.567 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00087235800000000005 2024-09-16T09:27:30.567 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.567 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.567 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:27:30.567 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.013503+0000", 2024-09-16T09:27:30.567 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00075915200000000003 2024-09-16T09:27:30.567 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.567 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.567 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:27:30.567 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.060585+0000", 2024-09-16T09:27:30.567 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.047082042999999997 2024-09-16T09:27:30.567 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.568 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.568 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:27:30.568 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.060644+0000", 2024-09-16T09:27:30.568 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.9284999999999999e-05 2024-09-16T09:27:30.568 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.568 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.568 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:27:30.568 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.060692+0000", 2024-09-16T09:27:30.568 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.8155000000000002e-05 2024-09-16T09:27:30.568 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:27:30.568 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:27:30.568 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:27:30.568 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.569 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.569 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:28534 7.1 7:8ece20e3:::benchmark_data_smithi070_78438_object28533:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-16T09:27:30.569 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:26:24.310541+0000", 2024-09-16T09:27:30.569 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 66.201780571, 2024-09-16T09:27:30.569 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.049159799999999997, 2024-09-16T09:27:30.569 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:27:30.569 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:27:30.569 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:27:30.569 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:27:30.569 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:27:30.569 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:27:30.569 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 28534 2024-09-16T09:27:30.569 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.569 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:27:30.570 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.570 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:27:30.570 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.310541+0000", 2024-09-16T09:27:30.570 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:27:30.570 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.570 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.570 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:27:30.570 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.310541+0000", 2024-09-16T09:27:30.570 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:27:30.570 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.570 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.570 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:27:30.570 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.310542+0000", 2024-09-16T09:27:30.571 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.9400000000000001e-07 2024-09-16T09:27:30.571 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.571 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.571 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:27:30.571 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.310613+0000", 2024-09-16T09:27:30.571 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.1495000000000006e-05 2024-09-16T09:27:30.571 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.571 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.571 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:27:30.571 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.310618+0000", 2024-09-16T09:27:30.571 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.2639999999999999e-06 2024-09-16T09:27:30.571 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.571 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.571 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:27:30.572 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.310622+0000", 2024-09-16T09:27:30.572 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.292e-06 2024-09-16T09:27:30.572 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.572 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.572 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:27:30.572 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.311623+0000", 2024-09-16T09:27:30.572 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0010012230000000001 2024-09-16T09:27:30.572 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.572 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.572 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:27:30.572 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.311701+0000", 2024-09-16T09:27:30.572 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.8331999999999995e-05 2024-09-16T09:27:30.572 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.572 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.573 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 3,6", 2024-09-16T09:27:30.573 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.311818+0000", 2024-09-16T09:27:30.573 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000117119 2024-09-16T09:27:30.573 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.573 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.573 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:27:30.573 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.313396+0000", 2024-09-16T09:27:30.573 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0015781110000000001 2024-09-16T09:27:30.573 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.573 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.573 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:27:30.573 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.357044+0000", 2024-09-16T09:27:30.573 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.043647205000000001 2024-09-16T09:27:30.574 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.574 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.574 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:27:30.574 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.359597+0000", 2024-09-16T09:27:30.574 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0025538179999999998 2024-09-16T09:27:30.574 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.574 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.574 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:27:30.574 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.359662+0000", 2024-09-16T09:27:30.574 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.4134999999999995e-05 2024-09-16T09:27:30.574 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.574 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.574 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:27:30.575 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.359701+0000", 2024-09-16T09:27:30.575 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.9412000000000001e-05 2024-09-16T09:27:30.575 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:27:30.575 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:27:30.575 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:27:30.575 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.575 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.575 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:28536 7.1 7:8a1751b8:::benchmark_data_smithi070_78438_object28535:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-16T09:27:30.575 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:26:24.311161+0000", 2024-09-16T09:27:30.575 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 66.201161181000003, 2024-09-16T09:27:30.575 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.050263705999999998, 2024-09-16T09:27:30.575 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:27:30.575 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:27:30.576 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:27:30.576 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:27:30.576 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:27:30.576 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:27:30.576 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 28536 2024-09-16T09:27:30.576 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.576 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:27:30.576 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.576 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:27:30.576 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.311161+0000", 2024-09-16T09:27:30.576 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:27:30.576 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.576 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.576 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:27:30.577 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.311161+0000", 2024-09-16T09:27:30.577 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:27:30.577 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.577 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.577 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:27:30.577 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.311161+0000", 2024-09-16T09:27:30.577 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.2600000000000001e-07 2024-09-16T09:27:30.577 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.577 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.577 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:27:30.577 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.311199+0000", 2024-09-16T09:27:30.577 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.7957000000000003e-05 2024-09-16T09:27:30.577 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.578 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.578 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:27:30.578 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.311210+0000", 2024-09-16T09:27:30.578 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0424999999999999e-05 2024-09-16T09:27:30.578 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.578 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.578 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:27:30.578 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.311217+0000", 2024-09-16T09:27:30.578 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.6639999999999998e-06 2024-09-16T09:27:30.578 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.578 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.578 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:27:30.578 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.312694+0000", 2024-09-16T09:27:30.579 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0014767859999999999 2024-09-16T09:27:30.579 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.579 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.579 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:27:30.579 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.312764+0000", 2024-09-16T09:27:30.579 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.9597999999999993e-05 2024-09-16T09:27:30.579 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.579 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.579 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 3,6", 2024-09-16T09:27:30.579 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.312861+0000", 2024-09-16T09:27:30.579 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.7578999999999995e-05 2024-09-16T09:27:30.579 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.579 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.580 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:27:30.580 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.314081+0000", 2024-09-16T09:27:30.580 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.001219138 2024-09-16T09:27:30.580 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.580 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.580 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:27:30.580 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.357087+0000", 2024-09-16T09:27:30.580 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.043006123 2024-09-16T09:27:30.580 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.580 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.580 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:27:30.580 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.361332+0000", 2024-09-16T09:27:30.580 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00424552 2024-09-16T09:27:30.581 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.581 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.581 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:27:30.581 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.361384+0000", 2024-09-16T09:27:30.581 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.2009999999999998e-05 2024-09-16T09:27:30.581 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.581 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.581 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:27:30.581 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.361424+0000", 2024-09-16T09:27:30.581 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.0080000000000003e-05 2024-09-16T09:27:30.581 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:27:30.581 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:27:30.581 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:27:30.582 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.582 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.582 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:37503 7.1 7:898ab9a8:::benchmark_data_smithi070_78438_object37502:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-16T09:27:30.582 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:26:29.406093+0000", 2024-09-16T09:27:30.582 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 61.106228747999999, 2024-09-16T09:27:30.582 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.065952329000000004, 2024-09-16T09:27:30.582 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:27:30.582 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:27:30.582 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:27:30.582 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:27:30.582 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:27:30.582 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:27:30.582 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 37503 2024-09-16T09:27:30.582 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.583 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:27:30.583 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.583 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:27:30.583 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:29.406093+0000", 2024-09-16T09:27:30.583 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:27:30.583 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.583 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.583 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:27:30.583 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:29.406093+0000", 2024-09-16T09:27:30.583 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:27:30.583 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.583 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.583 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:27:30.584 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:29.406095+0000", 2024-09-16T09:27:30.584 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.5710000000000001e-06 2024-09-16T09:27:30.584 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.584 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.584 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:27:30.584 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:29.406143+0000", 2024-09-16T09:27:30.584 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.8238000000000001e-05 2024-09-16T09:27:30.584 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.584 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.584 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:27:30.584 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:29.406153+0000", 2024-09-16T09:27:30.584 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0292000000000001e-05 2024-09-16T09:27:30.584 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.585 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.585 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:27:30.585 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:29.406158+0000", 2024-09-16T09:27:30.585 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.9490000000000004e-06 2024-09-16T09:27:30.585 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.585 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.585 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:27:30.585 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:29.406212+0000", 2024-09-16T09:27:30.585 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.3687000000000003e-05 2024-09-16T09:27:30.585 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.585 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.585 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:27:30.585 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:29.406358+0000", 2024-09-16T09:27:30.586 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00014636599999999999 2024-09-16T09:27:30.586 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.586 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.586 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 3,6", 2024-09-16T09:27:30.586 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:29.406486+0000", 2024-09-16T09:27:30.586 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00012791899999999999 2024-09-16T09:27:30.586 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.586 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.586 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:27:30.586 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:29.407602+0000", 2024-09-16T09:27:30.586 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0011164129999999999 2024-09-16T09:27:30.586 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.586 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.587 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:27:30.587 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:29.413354+0000", 2024-09-16T09:27:30.587 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0057519659999999998 2024-09-16T09:27:30.587 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.587 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.587 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:27:30.587 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:29.471932+0000", 2024-09-16T09:27:30.587 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.058577683999999998 2024-09-16T09:27:30.587 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.587 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.587 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:27:30.587 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:29.471997+0000", 2024-09-16T09:27:30.587 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.4554000000000006e-05 2024-09-16T09:27:30.587 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.587 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.588 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:27:30.588 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:29.472045+0000", 2024-09-16T09:27:30.588 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.8690000000000003e-05 2024-09-16T09:27:30.588 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:27:30.588 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:27:30.588 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:27:30.588 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.588 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.588 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:37513 7.1 7:8f73be0a:::benchmark_data_smithi070_78438_object37512:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-16T09:27:30.588 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:26:29.411858+0000", 2024-09-16T09:27:30.588 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 61.100464017999997, 2024-09-16T09:27:30.588 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.060299758000000002, 2024-09-16T09:27:30.588 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:27:30.588 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:27:30.588 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:27:30.589 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:27:30.589 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:27:30.590 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:27:30.591 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:27:30.591 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:27:30.591 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 37513 2024-09-16T09:27:30.591 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.591 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:27:30.591 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.591 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:27:30.591 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:29.411858+0000", 2024-09-16T09:27:30.591 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:27:30.591 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.591 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.591 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:27:30.591 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:29.411858+0000", 2024-09-16T09:27:30.592 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:27:30.592 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.592 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.592 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:27:30.592 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:29.411861+0000", 2024-09-16T09:27:30.592 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.0860000000000002e-06 2024-09-16T09:27:30.592 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.592 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.592 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:27:30.592 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:29.411977+0000", 2024-09-16T09:27:30.592 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000115717 2024-09-16T09:27:30.592 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.592 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.593 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:27:30.593 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:29.411988+0000", 2024-09-16T09:27:30.593 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1153e-05 2024-09-16T09:27:30.593 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.593 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.593 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:27:30.593 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:29.411993+0000", 2024-09-16T09:27:30.593 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.1000000000000003e-06 2024-09-16T09:27:30.593 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.593 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.593 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:27:30.593 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:29.412040+0000", 2024-09-16T09:27:30.593 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.7586000000000001e-05 2024-09-16T09:27:30.594 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.594 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.594 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:27:30.594 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:29.412156+0000", 2024-09-16T09:27:30.594 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000115309 2024-09-16T09:27:30.594 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.594 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.594 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 3,6", 2024-09-16T09:27:30.594 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:29.412282+0000", 2024-09-16T09:27:30.594 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000126057 2024-09-16T09:27:30.594 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.594 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.594 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:27:30.594 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:29.413436+0000", 2024-09-16T09:27:30.595 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0011544859999999999 2024-09-16T09:27:30.595 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.595 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.595 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:27:30.595 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:29.414792+0000", 2024-09-16T09:27:30.595 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0013560709999999999 2024-09-16T09:27:30.595 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.595 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.595 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:27:30.595 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:29.472083+0000", 2024-09-16T09:27:30.595 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.057290747000000003 2024-09-16T09:27:30.595 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.595 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.596 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:27:30.596 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:29.472128+0000", 2024-09-16T09:27:30.596 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.4894000000000001e-05 2024-09-16T09:27:30.596 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.596 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.596 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:27:30.596 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:29.472157+0000", 2024-09-16T09:27:30.596 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.9552000000000001e-05 2024-09-16T09:27:30.596 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:27:30.596 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:27:30.596 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:27:30.596 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.596 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.596 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_repop(client.24698.0:37522 7.8 e90/82 7:14d9e782:::benchmark_data_smithi070_78438_object37521:head v 90'2363, mlcod=90'2360)", 2024-09-16T09:27:30.597 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:26:29.417519+0000", 2024-09-16T09:27:30.597 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 61.094802850000001, 2024-09-16T09:27:30.597 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.05336834, 2024-09-16T09:27:30.597 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:27:30.597 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:27:30.597 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:27:30.597 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:27:30.597 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.597 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:27:30.597 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:29.417519+0000", 2024-09-16T09:27:30.597 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:27:30.597 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.597 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.598 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:27:30.598 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:29.417519+0000", 2024-09-16T09:27:30.598 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:27:30.598 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.598 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.598 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:27:30.598 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:29.417519+0000", 2024-09-16T09:27:30.598 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.0399999999999996e-07 2024-09-16T09:27:30.598 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.598 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.598 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:27:30.598 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:29.417633+0000", 2024-09-16T09:27:30.598 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00011314700000000001 2024-09-16T09:27:30.598 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.599 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.599 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:27:30.599 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:29.417641+0000", 2024-09-16T09:27:30.599 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.2220000000000002e-06 2024-09-16T09:27:30.599 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.599 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.599 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:27:30.599 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:29.417645+0000", 2024-09-16T09:27:30.599 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.4739999999999999e-06 2024-09-16T09:27:30.599 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.599 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.599 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:27:30.599 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:29.417685+0000", 2024-09-16T09:27:30.600 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.0184e-05 2024-09-16T09:27:30.600 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.600 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.600 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:27:30.600 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:29.417697+0000", 2024-09-16T09:27:30.600 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1505000000000001e-05 2024-09-16T09:27:30.600 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.600 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.600 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:27:30.600 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:29.470756+0000", 2024-09-16T09:27:30.600 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.053058816000000002 2024-09-16T09:27:30.600 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.600 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.601 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:27:30.601 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:29.470887+0000", 2024-09-16T09:27:30.601 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000131488 2024-09-16T09:27:30.601 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:27:30.601 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:27:30.601 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:27:30.601 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.601 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.601 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_repop(client.24698.0:37524 7.3 e90/82 7:c3609e81:::benchmark_data_smithi070_78438_object37523:head v 90'2409, mlcod=90'2407)", 2024-09-16T09:27:30.601 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:26:29.418747+0000", 2024-09-16T09:27:30.601 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 61.093574836999998, 2024-09-16T09:27:30.601 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.057664957000000003, 2024-09-16T09:27:30.602 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:27:30.602 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:27:30.602 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:27:30.602 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:27:30.602 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.602 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:27:30.602 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:29.418747+0000", 2024-09-16T09:27:30.602 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:27:30.602 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.602 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.602 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:27:30.602 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:29.418747+0000", 2024-09-16T09:27:30.602 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:27:30.603 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.603 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.603 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:27:30.603 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:29.418748+0000", 2024-09-16T09:27:30.603 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.7000000000000004e-07 2024-09-16T09:27:30.603 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.603 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.603 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:27:30.603 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:29.418806+0000", 2024-09-16T09:27:30.603 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.8749000000000003e-05 2024-09-16T09:27:30.603 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.603 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.603 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:27:30.603 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:29.470751+0000", 2024-09-16T09:27:30.604 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0519446 2024-09-16T09:27:30.604 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.604 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.604 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:27:30.604 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:29.470777+0000", 2024-09-16T09:27:30.604 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.5473000000000002e-05 2024-09-16T09:27:30.604 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.604 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.604 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:27:30.604 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:29.471122+0000", 2024-09-16T09:27:30.604 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00034562200000000002 2024-09-16T09:27:30.604 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.604 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.605 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:27:30.605 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:29.471142+0000", 2024-09-16T09:27:30.605 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.0210000000000001e-05 2024-09-16T09:27:30.605 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.605 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.605 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:27:30.605 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:29.476382+0000", 2024-09-16T09:27:30.605 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.005239489 2024-09-16T09:27:30.605 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.605 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.605 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:27:30.605 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:29.476412+0000", 2024-09-16T09:27:30.605 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.0043999999999999e-05 2024-09-16T09:27:30.605 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:27:30.606 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:27:30.606 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:27:30.606 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.606 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.606 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:60611 7.1 7:87030956:::benchmark_data_smithi070_78438_object60610:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-16T09:27:30.606 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:26:44.705726+0000", 2024-09-16T09:27:30.606 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 45.806595528999999, 2024-09-16T09:27:30.606 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.069662907999999996, 2024-09-16T09:27:30.606 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:27:30.606 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:27:30.606 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:27:30.606 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:27:30.606 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:27:30.606 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:27:30.607 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 60611 2024-09-16T09:27:30.607 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.607 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:27:30.607 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.607 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:27:30.607 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.705726+0000", 2024-09-16T09:27:30.607 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:27:30.607 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.607 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.607 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:27:30.607 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.705726+0000", 2024-09-16T09:27:30.607 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:27:30.607 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.607 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.607 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:27:30.608 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.705729+0000", 2024-09-16T09:27:30.608 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.4880000000000001e-06 2024-09-16T09:27:30.608 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.608 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.608 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:27:30.608 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.705766+0000", 2024-09-16T09:27:30.608 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.7682000000000003e-05 2024-09-16T09:27:30.608 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.608 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.608 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:27:30.608 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.705781+0000", 2024-09-16T09:27:30.608 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.4499e-05 2024-09-16T09:27:30.608 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.608 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.609 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:27:30.609 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.705788+0000", 2024-09-16T09:27:30.609 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.5950000000000001e-06 2024-09-16T09:27:30.609 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.609 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.609 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:27:30.609 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.705827+0000", 2024-09-16T09:27:30.609 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.8266000000000003e-05 2024-09-16T09:27:30.609 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.609 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.609 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:27:30.609 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.705940+0000", 2024-09-16T09:27:30.610 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000112773 2024-09-16T09:27:30.610 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.610 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.610 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 3,6", 2024-09-16T09:27:30.610 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.706099+0000", 2024-09-16T09:27:30.610 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000159507 2024-09-16T09:27:30.610 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.610 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.610 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:27:30.610 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.707710+0000", 2024-09-16T09:27:30.610 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.001610867 2024-09-16T09:27:30.610 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.610 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.611 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:27:30.611 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.707986+0000", 2024-09-16T09:27:30.611 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00027570200000000001 2024-09-16T09:27:30.611 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.611 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.611 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:27:30.611 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.775304+0000", 2024-09-16T09:27:30.611 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.067318174999999994 2024-09-16T09:27:30.611 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.611 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.611 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:27:30.611 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.775351+0000", 2024-09-16T09:27:30.611 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.7630000000000003e-05 2024-09-16T09:27:30.611 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.612 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.612 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:27:30.612 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.775389+0000", 2024-09-16T09:27:30.612 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.7724e-05 2024-09-16T09:27:30.612 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:27:30.612 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:27:30.612 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:27:30.612 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.612 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.612 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:62039 7.1 7:8ca71bdb:::benchmark_data_smithi070_78438_object62038:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-16T09:27:30.612 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:26:45.815079+0000", 2024-09-16T09:27:30.612 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 44.697243030999999, 2024-09-16T09:27:30.613 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.062287673000000002, 2024-09-16T09:27:30.613 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:27:30.613 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:27:30.613 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:27:30.613 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:27:30.613 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:27:30.613 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:27:30.613 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 62039 2024-09-16T09:27:30.613 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.613 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:27:30.613 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.613 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:27:30.613 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.815079+0000", 2024-09-16T09:27:30.614 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:27:30.614 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.614 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.614 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:27:30.614 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.815079+0000", 2024-09-16T09:27:30.614 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:27:30.614 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.614 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.614 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:27:30.614 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.815081+0000", 2024-09-16T09:27:30.614 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.8759999999999999e-06 2024-09-16T09:27:30.614 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.614 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.614 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:27:30.615 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.815108+0000", 2024-09-16T09:27:30.615 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.7129999999999999e-05 2024-09-16T09:27:30.615 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.615 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.615 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:27:30.615 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.815124+0000", 2024-09-16T09:27:30.615 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.5942000000000001e-05 2024-09-16T09:27:30.615 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.615 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.615 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:27:30.615 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.815130+0000", 2024-09-16T09:27:30.615 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.2299999999999996e-06 2024-09-16T09:27:30.615 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.616 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.616 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:27:30.616 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.815189+0000", 2024-09-16T09:27:30.616 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.8690000000000002e-05 2024-09-16T09:27:30.616 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.616 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.616 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:27:30.616 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.815310+0000", 2024-09-16T09:27:30.616 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000121354 2024-09-16T09:27:30.616 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.616 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.616 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 3,6", 2024-09-16T09:27:30.616 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.815443+0000", 2024-09-16T09:27:30.617 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00013313100000000001 2024-09-16T09:27:30.617 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.617 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.617 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:27:30.617 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.817657+0000", 2024-09-16T09:27:30.617 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0022140110000000001 2024-09-16T09:27:30.617 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.617 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.617 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:27:30.617 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.818250+0000", 2024-09-16T09:27:30.617 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000592601 2024-09-16T09:27:30.617 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.617 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.617 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:27:30.618 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.877252+0000", 2024-09-16T09:27:30.618 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.059002286000000001 2024-09-16T09:27:30.618 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.618 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.618 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:27:30.618 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.877316+0000", 2024-09-16T09:27:30.618 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.4264999999999995e-05 2024-09-16T09:27:30.618 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.618 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.618 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:27:30.618 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.877366+0000", 2024-09-16T09:27:30.618 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.0157000000000001e-05 2024-09-16T09:27:30.618 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:27:30.618 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:27:30.619 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:27:30.619 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.619 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.619 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:86890 7.1 7:8b0c621b:::benchmark_data_smithi070_78438_object86889:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-16T09:27:30.619 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:27:06.136694+0000", 2024-09-16T09:27:30.619 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 24.375627973, 2024-09-16T09:27:30.619 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.086345176999999995, 2024-09-16T09:27:30.619 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:27:30.619 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:27:30.619 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:27:30.619 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:27:30.619 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:27:30.620 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:27:30.620 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 86890 2024-09-16T09:27:30.620 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.620 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:27:30.620 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.620 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:27:30.620 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.136694+0000", 2024-09-16T09:27:30.620 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:27:30.620 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.620 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.620 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:27:30.620 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.136694+0000", 2024-09-16T09:27:30.620 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:27:30.621 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.621 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.621 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:27:30.621 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.136695+0000", 2024-09-16T09:27:30.621 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.4160000000000001e-06 2024-09-16T09:27:30.621 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.621 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.621 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:27:30.621 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.136800+0000", 2024-09-16T09:27:30.621 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000104957 2024-09-16T09:27:30.621 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.621 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.621 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:27:30.622 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.136813+0000", 2024-09-16T09:27:30.622 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.3023000000000001e-05 2024-09-16T09:27:30.622 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.622 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.622 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:27:30.622 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.136820+0000", 2024-09-16T09:27:30.622 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.2180000000000002e-06 2024-09-16T09:27:30.622 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.622 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.622 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:27:30.622 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.136891+0000", 2024-09-16T09:27:30.622 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.0697000000000004e-05 2024-09-16T09:27:30.622 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.623 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.623 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:27:30.623 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.137034+0000", 2024-09-16T09:27:30.623 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000142421 2024-09-16T09:27:30.623 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.623 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.623 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 3,6", 2024-09-16T09:27:30.623 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.137177+0000", 2024-09-16T09:27:30.623 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00014350699999999999 2024-09-16T09:27:30.623 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.623 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.623 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:27:30.623 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.144870+0000", 2024-09-16T09:27:30.623 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0076926549999999996 2024-09-16T09:27:30.624 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.624 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.624 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:27:30.624 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.144970+0000", 2024-09-16T09:27:30.624 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000100511 2024-09-16T09:27:30.624 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.624 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.624 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:27:30.624 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.222926+0000", 2024-09-16T09:27:30.624 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.077955615000000006 2024-09-16T09:27:30.624 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.624 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.624 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:27:30.625 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.222988+0000", 2024-09-16T09:27:30.625 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.1916999999999997e-05 2024-09-16T09:27:30.625 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.625 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.625 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:27:30.625 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.223039+0000", 2024-09-16T09:27:30.625 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.1239999999999997e-05 2024-09-16T09:27:30.625 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:27:30.625 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:27:30.625 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:27:30.625 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.626 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.626 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:86891 7.1 7:8533eaa2:::benchmark_data_smithi070_78438_object86890:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-16T09:27:30.626 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:27:06.137963+0000", 2024-09-16T09:27:30.626 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 24.374358847, 2024-09-16T09:27:30.626 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.089654372999999996, 2024-09-16T09:27:30.626 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:27:30.626 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:27:30.626 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:27:30.626 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:27:30.626 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:27:30.626 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:27:30.626 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 86891 2024-09-16T09:27:30.626 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.627 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:27:30.627 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.627 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:27:30.627 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.137963+0000", 2024-09-16T09:27:30.627 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:27:30.627 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.627 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.627 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:27:30.627 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.137963+0000", 2024-09-16T09:27:30.627 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:27:30.627 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.627 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.627 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:27:30.628 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.137964+0000", 2024-09-16T09:27:30.628 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.5629999999999999e-06 2024-09-16T09:27:30.628 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.628 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.628 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:27:30.628 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.137999+0000", 2024-09-16T09:27:30.628 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.4564000000000003e-05 2024-09-16T09:27:30.628 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.628 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.628 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:27:30.628 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.138016+0000", 2024-09-16T09:27:30.628 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.6677999999999999e-05 2024-09-16T09:27:30.628 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.629 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.629 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:27:30.629 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.138023+0000", 2024-09-16T09:27:30.629 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.5059999999999998e-06 2024-09-16T09:27:30.629 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.629 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.629 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:27:30.629 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.139570+0000", 2024-09-16T09:27:30.629 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0015467809999999999 2024-09-16T09:27:30.629 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.629 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.629 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:27:30.629 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.139690+0000", 2024-09-16T09:27:30.629 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000120387 2024-09-16T09:27:30.630 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.630 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.630 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 3,6", 2024-09-16T09:27:30.630 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.139833+0000", 2024-09-16T09:27:30.630 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00014291799999999999 2024-09-16T09:27:30.630 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.630 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.630 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:27:30.630 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.144931+0000", 2024-09-16T09:27:30.630 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0050973110000000002 2024-09-16T09:27:30.630 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.630 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.630 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:27:30.630 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.144973+0000", 2024-09-16T09:27:30.630 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.2038e-05 2024-09-16T09:27:30.631 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.631 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.631 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:27:30.631 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.227507+0000", 2024-09-16T09:27:30.631 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.082533910000000002 2024-09-16T09:27:30.631 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.631 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.631 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:27:30.631 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.227568+0000", 2024-09-16T09:27:30.631 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.1229999999999995e-05 2024-09-16T09:27:30.631 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.631 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.631 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:27:30.631 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.227617+0000", 2024-09-16T09:27:30.631 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.9487000000000002e-05 2024-09-16T09:27:30.632 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:27:30.632 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:27:30.632 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:27:30.632 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.632 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.632 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:86895 7.1 7:852b1173:::benchmark_data_smithi070_78438_object86894:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-16T09:27:30.632 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:27:06.138894+0000", 2024-09-16T09:27:30.632 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 24.373427634999999, 2024-09-16T09:27:30.632 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.091395905, 2024-09-16T09:27:30.632 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:27:30.632 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:27:30.632 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:27:30.633 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:27:30.633 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:27:30.633 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:27:30.633 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 86895 2024-09-16T09:27:30.633 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.633 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:27:30.633 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.633 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:27:30.633 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.138894+0000", 2024-09-16T09:27:30.633 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:27:30.633 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.633 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.633 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:27:30.634 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.138894+0000", 2024-09-16T09:27:30.634 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:27:30.634 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.634 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.634 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:27:30.634 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.138895+0000", 2024-09-16T09:27:30.634 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.093e-06 2024-09-16T09:27:30.634 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.634 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.634 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:27:30.634 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.138932+0000", 2024-09-16T09:27:30.634 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.6934999999999998e-05 2024-09-16T09:27:30.634 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.634 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.635 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:27:30.635 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.138946+0000", 2024-09-16T09:27:30.635 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.3772e-05 2024-09-16T09:27:30.635 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.635 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.635 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:27:30.635 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.138953+0000", 2024-09-16T09:27:30.635 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.2810000000000003e-06 2024-09-16T09:27:30.635 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.635 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.635 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:27:30.635 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.142367+0000", 2024-09-16T09:27:30.635 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0034135559999999999 2024-09-16T09:27:30.635 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.636 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.636 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:27:30.636 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.142499+0000", 2024-09-16T09:27:30.636 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00013187 2024-09-16T09:27:30.636 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.636 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.636 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 3,6", 2024-09-16T09:27:30.636 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.142646+0000", 2024-09-16T09:27:30.636 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00014712500000000001 2024-09-16T09:27:30.636 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.636 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.636 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:27:30.637 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.144963+0000", 2024-09-16T09:27:30.637 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0023176260000000001 2024-09-16T09:27:30.637 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.637 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.637 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:27:30.637 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.144977+0000", 2024-09-16T09:27:30.637 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.3509000000000001e-05 2024-09-16T09:27:30.637 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.637 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.637 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:27:30.637 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.230194+0000", 2024-09-16T09:27:30.637 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.085216682000000002 2024-09-16T09:27:30.637 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.637 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.637 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:27:30.638 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.230252+0000", 2024-09-16T09:27:30.638 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.8023000000000003e-05 2024-09-16T09:27:30.638 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.638 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.638 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:27:30.638 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.230290+0000", 2024-09-16T09:27:30.638 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.8433000000000003e-05 2024-09-16T09:27:30.638 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:27:30.638 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:27:30.638 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:27:30.638 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.638 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.638 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:88353 7.1 7:8cc65f4e:::benchmark_data_smithi070_78438_object88352:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-16T09:27:30.638 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:27:07.475281+0000", 2024-09-16T09:27:30.638 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 23.037041126999998, 2024-09-16T09:27:30.639 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.057670512, 2024-09-16T09:27:30.639 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:27:30.639 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:27:30.639 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:27:30.639 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:27:30.639 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:27:30.639 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:27:30.639 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 88353 2024-09-16T09:27:30.639 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.639 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:27:30.639 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.639 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:27:30.639 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:07.475281+0000", 2024-09-16T09:27:30.639 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:27:30.640 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.640 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.640 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:27:30.640 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:07.475281+0000", 2024-09-16T09:27:30.640 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:27:30.640 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.640 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.640 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:27:30.640 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:07.475281+0000", 2024-09-16T09:27:30.640 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.3699999999999999e-07 2024-09-16T09:27:30.640 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.640 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.640 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:27:30.640 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:07.475337+0000", 2024-09-16T09:27:30.640 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.5787e-05 2024-09-16T09:27:30.641 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.641 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.641 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:27:30.641 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:07.475348+0000", 2024-09-16T09:27:30.641 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1233000000000001e-05 2024-09-16T09:27:30.641 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.641 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.641 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:27:30.641 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:07.475356+0000", 2024-09-16T09:27:30.641 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.2400000000000001e-06 2024-09-16T09:27:30.641 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.641 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.641 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:27:30.642 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:07.477364+0000", 2024-09-16T09:27:30.642 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0020087759999999999 2024-09-16T09:27:30.642 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.642 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.642 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:27:30.642 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:07.477490+0000", 2024-09-16T09:27:30.642 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000125938 2024-09-16T09:27:30.642 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.642 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.642 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 3,6", 2024-09-16T09:27:30.642 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:07.477673+0000", 2024-09-16T09:27:30.642 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00018255699999999999 2024-09-16T09:27:30.642 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.642 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.643 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:27:30.643 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:07.480013+0000", 2024-09-16T09:27:30.643 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0023395389999999999 2024-09-16T09:27:30.643 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.643 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.643 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:27:30.643 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:07.480362+0000", 2024-09-16T09:27:30.643 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000349694 2024-09-16T09:27:30.643 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.643 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.643 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:27:30.643 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:07.532825+0000", 2024-09-16T09:27:30.643 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.052463217999999999 2024-09-16T09:27:30.643 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.643 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.644 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:27:30.644 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:07.532874+0000", 2024-09-16T09:27:30.644 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.8992000000000001e-05 2024-09-16T09:27:30.644 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:30.644 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:30.644 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:27:30.644 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:07.532951+0000", 2024-09-16T09:27:30.644 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.6700999999999996e-05 2024-09-16T09:27:30.644 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:27:30.644 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:27:30.644 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:27:30.644 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:27:30.644 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:27:30.644 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-09-16T09:27:30.673 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:30.674 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:30.674 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:27:30.775 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:27:30.805 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:30.806 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:30.807 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:27:30.907 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:27:30.934 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:30.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:30.937 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:27:31.035 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 78 16 113365 113349 90.7865 71.125 0.0138662 0.0110104 2024-09-16T09:27:31.037 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:27:31.066 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:31.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:31.067 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:27:31.168 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:27:31.181 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:27:31.192 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:31.193 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:31.193 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:27:31.253 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:27:31.253 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:27:31.276 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:27:31.294 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:27:31.297 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:27:31.298 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:27:31.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:31.321 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:31.322 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:27:31.330 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:31 smithi070 ceph-mon[30388]: pgmap v103: 158 pgs: 158 active+clean; 6.7 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 852 B/s rd, 57 MiB/s wr, 920 op/s 2024-09-16T09:27:31.334 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:31 smithi070 ceph-mon[40691]: pgmap v103: 158 pgs: 158 active+clean; 6.7 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 852 B/s rd, 57 MiB/s wr, 920 op/s 2024-09-16T09:27:31.334 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:27:31.334 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:27:31.367 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:27:31.368 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:27:31.401 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:27:31.401 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:27:31.409 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:31 smithi084 ceph-mon[32629]: pgmap v103: 158 pgs: 158 active+clean; 6.7 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 852 B/s rd, 57 MiB/s wr, 920 op/s 2024-09-16T09:27:31.423 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:27:31.436 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:27:31.436 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:27:31.450 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:31.450 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:31.451 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:27:31.466 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:27:31.466 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:27:31.504 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:27:31.504 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:27:31.552 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:27:31.577 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:31.578 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:31.578 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:27:31.679 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:27:31.726 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:31.727 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:31.727 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:27:31.828 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:27:31.857 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.5 dump_ops_in_flight 2024-09-16T09:27:31.860 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:31.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:31.861 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:27:31.961 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:27:32.004 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:32.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:32.005 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:27:32.035 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 79 16 114369 114353 90.4314 62.75 0.030383 0.0110516 2024-09-16T09:27:32.106 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:27:32.137 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:32.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:32.139 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:27:32.158 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:27:31 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:27:31] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:27:32.194 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:27:32.240 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:27:32.267 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:32.267 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:32.268 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:27:32.369 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:27:32.397 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:32.397 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:32.398 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:27:32.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:32 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3617741837' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:27:32.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:32 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:27:32.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:32 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:27:32.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:32 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3617741837' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:27:32.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:32 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:27:32.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:32 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:27:32.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:32 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3617741837' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:27:32.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:32 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:27:32.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:32 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:27:32.498 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:27:32.533 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:32.534 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:32.534 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:27:32.635 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:27:32.664 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:32.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:32.665 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:27:32.755 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:27:32 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:27:32.768 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:27:32.800 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:32.800 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:32.801 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:27:32.806 INFO:teuthology.orchestra.run.smithi084.stdout:ERROR: (22) Invalid argument 2024-09-16T09:27:32.806 INFO:teuthology.orchestra.run.smithi084.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:27:32.901 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:27:32.937 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:32.937 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:32.938 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:27:33.036 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:2024-09-16T09:27:33.035389+0000 min lat: 0.0012412 max lat: 0.156312 avg lat: 0.0110849 2024-09-16T09:27:33.036 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-16T09:27:33.036 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 80 16 115486 115470 90.1737 69.8125 0.0237823 0.0110849 2024-09-16T09:27:33.038 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:27:33.086 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:33.087 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:33.087 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:27:33.128 INFO:teuthology.orchestra.run.smithi070.stderr:no valid command found; 10 closest matches: 2024-09-16T09:27:33.129 INFO:teuthology.orchestra.run.smithi070.stderr:0 2024-09-16T09:27:33.129 INFO:teuthology.orchestra.run.smithi070.stderr:1 2024-09-16T09:27:33.129 INFO:teuthology.orchestra.run.smithi070.stderr:2 2024-09-16T09:27:33.129 INFO:teuthology.orchestra.run.smithi070.stderr:abort 2024-09-16T09:27:33.129 INFO:teuthology.orchestra.run.smithi070.stderr:assert 2024-09-16T09:27:33.129 INFO:teuthology.orchestra.run.smithi070.stderr:bench [] [] [] [] 2024-09-16T09:27:33.129 INFO:teuthology.orchestra.run.smithi070.stderr:bluefs debug_inject_read_zeros 2024-09-16T09:27:33.129 INFO:teuthology.orchestra.run.smithi070.stderr:bluefs files list 2024-09-16T09:27:33.129 INFO:teuthology.orchestra.run.smithi070.stderr:bluefs stats 2024-09-16T09:27:33.129 INFO:teuthology.orchestra.run.smithi070.stderr:bluestore allocator dump block 2024-09-16T09:27:33.129 INFO:teuthology.orchestra.run.smithi070.stderr:Error EINVAL: invalid command 2024-09-16T09:27:33.169 INFO:teuthology.orchestra.run.smithi070.stderr:nodeep-scrub is set 2024-09-16T09:27:33.188 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:27:33.217 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:33.217 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:33.218 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:27:33.318 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:27:33.320 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:33 smithi070 ceph-mon[30388]: pgmap v104: 158 pgs: 158 active+clean; 6.7 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 692 B/s rd, 47 MiB/s wr, 747 op/s 2024-09-16T09:27:33.320 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:33 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3124002990' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:27:33.320 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:33 smithi070 ceph-mon[40691]: pgmap v104: 158 pgs: 158 active+clean; 6.7 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 692 B/s rd, 47 MiB/s wr, 747 op/s 2024-09-16T09:27:33.320 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:33 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3124002990' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:27:33.348 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:33.348 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:33.349 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:27:33.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:33 smithi084 ceph-mon[32629]: pgmap v104: 158 pgs: 158 active+clean; 6.7 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 692 B/s rd, 47 MiB/s wr, 747 op/s 2024-09-16T09:27:33.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:33 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3124002990' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:27:33.449 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:27:33.487 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:33.488 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:33.488 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:27:33.573 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.5 dump_blocked_ops 2024-09-16T09:27:33.589 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:27:33.646 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:33.646 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:33.647 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:27:33.747 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:27:33.780 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:33.781 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:33.781 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:27:33.882 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:27:33.923 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:33.923 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:33.924 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:27:33.926 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:27:34.025 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:27:34.035 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 81 16 116675 116659 89.9779 74.3125 0.00608094 0.0111076 2024-09-16T09:27:34.057 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-16T09:27:34.058 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-16T09:27:34.058 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph tell osd.7 'trim stale osdmaps' 2024-09-16T09:27:34.062 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:34.063 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:34.064 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:27:34.165 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:27:34.192 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:34.193 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:34.193 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:27:34.294 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:27:34.313 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:34 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:27:34.314 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:34 smithi084 ceph-mon[32629]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:27:34.314 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:34 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3124002990' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-16T09:27:34.314 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:34 smithi084 ceph-mon[32629]: osdmap e115: 8 total, 8 up, 8 in 2024-09-16T09:27:34.314 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:34 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:27:34.314 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:34 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:27:34.314 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:34 smithi084 ceph-mon[32629]: osdmap e116: 8 total, 8 up, 8 in 2024-09-16T09:27:34.324 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:34.325 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:34.325 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:27:34.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:34 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:27:34.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:34 smithi070 ceph-mon[30388]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:27:34.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:34 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3124002990' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-16T09:27:34.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:34 smithi070 ceph-mon[30388]: osdmap e115: 8 total, 8 up, 8 in 2024-09-16T09:27:34.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:34 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:27:34.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:34 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:27:34.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:34 smithi070 ceph-mon[30388]: osdmap e116: 8 total, 8 up, 8 in 2024-09-16T09:27:34.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:34 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:27:34.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:34 smithi070 ceph-mon[40691]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:27:34.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:34 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3124002990' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-16T09:27:34.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:34 smithi070 ceph-mon[40691]: osdmap e115: 8 total, 8 up, 8 in 2024-09-16T09:27:34.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:34 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:27:34.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:34 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:27:34.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:34 smithi070 ceph-mon[40691]: osdmap e116: 8 total, 8 up, 8 in 2024-09-16T09:27:34.426 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:27:34.466 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:34.469 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:34.469 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:27:34.570 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:27:34.599 INFO:teuthology.orchestra.run.smithi084.stdout:ERROR: (22) Invalid argument 2024-09-16T09:27:34.599 INFO:teuthology.orchestra.run.smithi084.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:27:34.605 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:34.606 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:34.606 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:27:34.707 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:27:34.746 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:34.747 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:34.747 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:27:34.848 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:27:34.900 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:34.900 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:34.900 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:27:35.001 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:27:35.014 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T09:27:35.037 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 82 16 117739 117723 89.6906 66.5 0.00845073 0.0111444 2024-09-16T09:27:35.055 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:35.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:35.056 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:27:35.157 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:27:35.184 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:35.185 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:35.185 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:27:35.286 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:27:35.310 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:35.311 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:35.311 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:27:35.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:35 smithi084 ceph-mon[32629]: pgmap v106: 158 pgs: 158 active+clean; 7.0 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 1.4 KiB/s rd, 96 MiB/s wr, 1.54k op/s 2024-09-16T09:27:35.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:35 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:27:35.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:35 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:27:35.412 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:27:35.436 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:35.437 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:35.437 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:27:35.464 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:35 smithi070 ceph-mon[40691]: pgmap v106: 158 pgs: 158 active+clean; 7.0 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 1.4 KiB/s rd, 96 MiB/s wr, 1.54k op/s 2024-09-16T09:27:35.464 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:35 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:27:35.464 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:35 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:27:35.464 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:35 smithi070 ceph-mon[30388]: pgmap v106: 158 pgs: 158 active+clean; 7.0 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 1.4 KiB/s rd, 96 MiB/s wr, 1.54k op/s 2024-09-16T09:27:35.464 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:35 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:27:35.465 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:35 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:27:35.538 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:27:35.564 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:35.564 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:35.565 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:27:35.570 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:27:35.665 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:27:35.707 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:35.708 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:35.709 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:27:35.809 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:27:35.845 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:35.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:35.846 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:27:35.947 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:27:35.970 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:35.971 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:35.971 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:27:36.037 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 83 16 118824 118808 89.427 67.8125 0.0143892 0.011178 2024-09-16T09:27:36.073 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:27:36.099 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:36.099 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:36.100 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:27:36.200 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:27:36.230 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:36.231 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:36.231 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:27:36.276 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.5 dump_historic_ops 2024-09-16T09:27:36.332 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:27:36.382 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:36.382 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:36.382 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:27:36.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:36 smithi070 ceph-mon[30388]: osdmap e117: 8 total, 8 up, 8 in 2024-09-16T09:27:36.420 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:36 smithi070 ceph-mon[40691]: osdmap e117: 8 total, 8 up, 8 in 2024-09-16T09:27:36.483 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:27:36.528 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:36.529 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:36.529 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:27:36.616 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:27:36.624 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:27:36.630 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:27:36.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:36 smithi084 ceph-mon[32629]: osdmap e117: 8 total, 8 up, 8 in 2024-09-16T09:27:36.661 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:36.662 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:36.662 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:27:36.718 INFO:teuthology.orchestra.run.smithi070.stderr:no valid command found; 10 closest matches: 2024-09-16T09:27:36.718 INFO:teuthology.orchestra.run.smithi070.stderr:0 2024-09-16T09:27:36.718 INFO:teuthology.orchestra.run.smithi070.stderr:1 2024-09-16T09:27:36.718 INFO:teuthology.orchestra.run.smithi070.stderr:2 2024-09-16T09:27:36.718 INFO:teuthology.orchestra.run.smithi070.stderr:abort 2024-09-16T09:27:36.718 INFO:teuthology.orchestra.run.smithi070.stderr:assert 2024-09-16T09:27:36.719 INFO:teuthology.orchestra.run.smithi070.stderr:bench [] [] [] [] 2024-09-16T09:27:36.719 INFO:teuthology.orchestra.run.smithi070.stderr:bluefs debug_inject_read_zeros 2024-09-16T09:27:36.719 INFO:teuthology.orchestra.run.smithi070.stderr:bluefs files list 2024-09-16T09:27:36.719 INFO:teuthology.orchestra.run.smithi070.stderr:bluefs stats 2024-09-16T09:27:36.719 INFO:teuthology.orchestra.run.smithi070.stderr:bluestore allocator dump block 2024-09-16T09:27:36.719 INFO:teuthology.orchestra.run.smithi070.stderr:Error EINVAL: invalid command 2024-09-16T09:27:36.763 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:27:36.810 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:36.811 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:36.812 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:27:36.912 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:27:36.913 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd unset noscrub 2024-09-16T09:27:36.957 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:36.958 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:36.959 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:27:37.038 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 84 16 119946 119930 89.1959 70.125 0.0105407 0.0112064 2024-09-16T09:27:37.060 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:27:37.092 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:37.093 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:37.093 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:27:37.194 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:27:37.197 INFO:teuthology.orchestra.run.smithi084.stdout:ERROR: (22) Invalid argument 2024-09-16T09:27:37.197 INFO:teuthology.orchestra.run.smithi084.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:27:37.232 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:37.233 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:37.233 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:27:37.334 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:27:37.360 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:37.360 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:37.361 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:27:37.409 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:37 smithi084 ceph-mon[32629]: pgmap v109: 158 pgs: 158 active+clean; 7.0 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 853 B/s rd, 56 MiB/s wr, 889 op/s 2024-09-16T09:27:37.422 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:37 smithi070 ceph-mon[30388]: pgmap v109: 158 pgs: 158 active+clean; 7.0 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 853 B/s rd, 56 MiB/s wr, 889 op/s 2024-09-16T09:27:37.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:37 smithi070 ceph-mon[40691]: pgmap v109: 158 pgs: 158 active+clean; 7.0 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 853 B/s rd, 56 MiB/s wr, 889 op/s 2024-09-16T09:27:37.461 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:27:37.505 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:37.505 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:37.505 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:27:37.606 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:27:37.635 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:37.635 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:37.636 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:27:37.737 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:27:37.773 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:37.774 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:37.774 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:27:37.875 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:27:37.882 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.6 dump_ops_in_flight 2024-09-16T09:27:37.909 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:37.910 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:37.911 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:27:38.011 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:27:38.032 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:38.033 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:38.033 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:27:38.038 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 85 16 121086 121070 88.9847 71.25 0.012215 0.0112303 2024-09-16T09:27:38.134 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:27:38.165 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:38.165 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:38.166 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:27:38.214 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:27:38.267 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:27:38.268 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-16T09:27:38.269 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-16T09:27:38.350 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:38.351 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:38.351 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:27:38.452 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:27:38.476 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:38.477 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:38.477 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:27:38.578 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:27:38.603 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:38.603 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:38.603 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:27:38.617 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:27:38.617 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:27:38.625 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:27:38.626 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:27:38.665 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:27:38.665 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:27:38.700 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:27:38.701 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:27:38.704 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:27:38.729 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:38.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:38.733 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:27:38.737 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:27:38.737 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:27:38.771 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:27:38.772 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:27:38.805 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:27:38.805 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:27:38.819 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-09-16T09:27:38.819 INFO:teuthology.orchestra.run.smithi084.stdout: "ops": [ 2024-09-16T09:27:38.819 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:38.819 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:121800 7.7 7:e81ace9d:::benchmark_data_smithi070_78438_object121799:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117)", 2024-09-16T09:27:38.820 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:27:38.817099+0000", 2024-09-16T09:27:38.820 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 0.001049812, 2024-09-16T09:27:38.820 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0010886540000000001, 2024-09-16T09:27:38.820 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:27:38.820 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:27:38.820 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "waiting for sub ops", 2024-09-16T09:27:38.820 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:27:38.820 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:27:38.820 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:27:38.820 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 121800 2024-09-16T09:27:38.820 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:38.820 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:27:38.820 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:38.821 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:27:38.821 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:38.817099+0000", 2024-09-16T09:27:38.821 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:27:38.821 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:38.821 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:38.821 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:27:38.821 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:38.817099+0000", 2024-09-16T09:27:38.821 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:27:38.821 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:38.821 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:38.821 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:27:38.821 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:38.817104+0000", 2024-09-16T09:27:38.821 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.074e-06 2024-09-16T09:27:38.822 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:38.822 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:38.822 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:27:38.822 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:38.817146+0000", 2024-09-16T09:27:38.822 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.1665000000000002e-05 2024-09-16T09:27:38.822 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:38.822 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:38.822 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:27:38.822 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:38.817162+0000", 2024-09-16T09:27:38.822 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.5957999999999999e-05 2024-09-16T09:27:38.822 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:38.822 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:38.822 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:27:38.822 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:38.817170+0000", 2024-09-16T09:27:38.823 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.8110000000000003e-06 2024-09-16T09:27:38.823 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:38.823 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:38.823 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:27:38.823 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:38.817227+0000", 2024-09-16T09:27:38.823 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.6823999999999997e-05 2024-09-16T09:27:38.823 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:38.823 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:38.823 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:27:38.823 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:38.817331+0000", 2024-09-16T09:27:38.823 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00010457 2024-09-16T09:27:38.823 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:38.823 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:38.823 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,3", 2024-09-16T09:27:38.824 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:38.817476+0000", 2024-09-16T09:27:38.824 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00014437900000000001 2024-09-16T09:27:38.824 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:27:38.824 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:27:38.824 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:27:38.824 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:27:38.824 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-09-16T09:27:38.824 INFO:teuthology.orchestra.run.smithi084.stdout: "num_ops": 1 2024-09-16T09:27:38.824 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-09-16T09:27:38.835 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:27:38.846 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:27:38.846 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:27:38.877 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:38.878 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:38.878 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:27:38.979 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:27:39.003 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:39.003 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:39.004 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:27:39.038 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 86 16 121998 121982 88.6129 57 0.0066357 0.0112798 2024-09-16T09:27:39.105 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:27:39.133 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:39.134 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:39.134 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:27:39.235 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:27:39.285 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:39.286 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:39.286 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:27:39.387 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:27:39.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:39 smithi084 ceph-mon[32629]: pgmap v110: 158 pgs: 158 active+clean; 7.1 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.7 KiB/s rd, 77 MiB/s wr, 1.24k op/s 2024-09-16T09:27:39.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:39 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/2595441959' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:27:39.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:39 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:27:39.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:39 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:27:39.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:39 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:27:39.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:39 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:27:39.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:39 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:27:39.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:39 smithi084 ceph-mon[32629]: osdmap e118: 8 total, 8 up, 8 in 2024-09-16T09:27:39.413 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:39.413 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:39.414 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:27:39.514 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:27:39.531 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.6 dump_blocked_ops 2024-09-16T09:27:39.541 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:39.541 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:39.541 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:27:39.642 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:27:39.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:39 smithi070 ceph-mon[30388]: pgmap v110: 158 pgs: 158 active+clean; 7.1 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.7 KiB/s rd, 77 MiB/s wr, 1.24k op/s 2024-09-16T09:27:39.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:39 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2595441959' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:27:39.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:39 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:27:39.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:39 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:27:39.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:39 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:27:39.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:39 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:27:39.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:39 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:27:39.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:39 smithi070 ceph-mon[30388]: osdmap e118: 8 total, 8 up, 8 in 2024-09-16T09:27:39.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:39 smithi070 ceph-mon[40691]: pgmap v110: 158 pgs: 158 active+clean; 7.1 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.7 KiB/s rd, 77 MiB/s wr, 1.24k op/s 2024-09-16T09:27:39.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:39 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/2595441959' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:27:39.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:39 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:27:39.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:39 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:27:39.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:39 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:27:39.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:39 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:27:39.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:39 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:27:39.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:39 smithi070 ceph-mon[40691]: osdmap e118: 8 total, 8 up, 8 in 2024-09-16T09:27:39.679 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:39.680 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:39.680 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:27:39.781 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:27:39.807 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:39.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:39.808 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:27:39.811 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:27:39.909 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:27:39.910 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:27:39.941 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:39.942 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:39.942 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:27:40.038 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 87 15 122879 122864 88.228 55.125 0.0178807 0.0113295 2024-09-16T09:27:40.043 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:27:40.068 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:40.068 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:40.069 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:27:40.170 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:27:40.199 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:40.200 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:40.200 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:27:40.301 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:27:40.333 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:40.334 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:40.335 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:27:40.436 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:27:40.470 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:40.470 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:40.471 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:27:40.509 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-09-16T09:27:40.509 INFO:teuthology.orchestra.run.smithi084.stdout: "ops": [], 2024-09-16T09:27:40.509 INFO:teuthology.orchestra.run.smithi084.stdout: "complaint_time": 30, 2024-09-16T09:27:40.509 INFO:teuthology.orchestra.run.smithi084.stdout: "num_blocked_ops": 0 2024-09-16T09:27:40.509 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-09-16T09:27:40.571 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:27:40.602 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:40.603 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:40.603 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:27:40.704 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:27:40.728 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:40.729 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:40.730 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:27:40.830 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:27:40.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:40.868 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:40.868 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:27:40.969 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:27:40.995 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:40.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:40.996 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:27:41.038 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 88 16 124087 124071 88.0827 75.4375 0.00964815 0.011348 2024-09-16T09:27:41.097 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:27:41.125 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:41.126 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:41.126 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:27:41.201 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.6 dump_historic_ops 2024-09-16T09:27:41.227 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:27:41.248 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:41.249 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:41.249 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:27:41.350 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:27:41.378 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:41.378 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:41.379 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:27:41.480 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:27:41.519 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:41.520 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:41.521 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:27:41.547 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:27:41.622 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:27:41.659 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:41.660 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:41.660 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:27:41.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:41 smithi084 ceph-mon[32629]: pgmap v112: 158 pgs: 158 active+clean; 7.4 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 853 B/s rd, 58 MiB/s wr, 935 op/s 2024-09-16T09:27:41.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:41 smithi070 ceph-mon[30388]: pgmap v112: 158 pgs: 158 active+clean; 7.4 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 853 B/s rd, 58 MiB/s wr, 935 op/s 2024-09-16T09:27:41.670 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:41 smithi070 ceph-mon[40691]: pgmap v112: 158 pgs: 158 active+clean; 7.4 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 853 B/s rd, 58 MiB/s wr, 935 op/s 2024-09-16T09:27:41.761 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:27:41.800 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:41.801 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:41.801 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:27:41.872 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T09:27:41.903 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:27:41.929 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:41.930 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:41.930 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:27:42.031 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:27:42.039 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 89 16 125105 125089 87.8079 63.625 0.0251079 0.0113832 2024-09-16T09:27:42.089 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:42.090 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:42.090 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:27:42.154 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-09-16T09:27:42.154 INFO:teuthology.orchestra.run.smithi084.stdout: "size": 20, 2024-09-16T09:27:42.154 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 600, 2024-09-16T09:27:42.154 INFO:teuthology.orchestra.run.smithi084.stdout: "ops": [ 2024-09-16T09:27:42.155 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.155 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:22928 7.c 7:3b5ff786:::benchmark_data_smithi070_78438_object22927:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T09:27:42.155 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:26:21.582673+0000", 2024-09-16T09:27:42.155 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 80.568013375000007, 2024-09-16T09:27:42.155 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.078021982000000004, 2024-09-16T09:27:42.155 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:27:42.155 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:27:42.155 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:27:42.155 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:27:42.155 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:27:42.155 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:27:42.155 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 22928 2024-09-16T09:27:42.155 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.156 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:27:42.156 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.156 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:27:42.156 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.582673+0000", 2024-09-16T09:27:42.156 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:27:42.156 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.156 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.156 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:27:42.156 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.582673+0000", 2024-09-16T09:27:42.157 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:27:42.157 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.157 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.157 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:27:42.157 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.582674+0000", 2024-09-16T09:27:42.157 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0899999999999999e-06 2024-09-16T09:27:42.157 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.157 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.157 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:27:42.157 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.582849+0000", 2024-09-16T09:27:42.158 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00017538100000000001 2024-09-16T09:27:42.158 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.158 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.159 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:27:42.159 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.582861+0000", 2024-09-16T09:27:42.159 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1402e-05 2024-09-16T09:27:42.159 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.159 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.160 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:27:42.160 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.582870+0000", 2024-09-16T09:27:42.160 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.5710000000000004e-06 2024-09-16T09:27:42.160 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.160 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.160 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:27:42.160 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.583221+0000", 2024-09-16T09:27:42.160 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000350978 2024-09-16T09:27:42.161 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.161 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.161 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:27:42.161 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.583337+0000", 2024-09-16T09:27:42.161 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000115828 2024-09-16T09:27:42.161 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.161 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.163 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 3,7", 2024-09-16T09:27:42.163 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.583477+0000", 2024-09-16T09:27:42.163 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000140178 2024-09-16T09:27:42.163 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.163 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.163 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:27:42.163 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.584911+0000", 2024-09-16T09:27:42.163 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0014333390000000001 2024-09-16T09:27:42.163 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.163 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.164 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:27:42.164 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.585824+0000", 2024-09-16T09:27:42.164 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00091349300000000003 2024-09-16T09:27:42.164 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.164 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.164 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:27:42.164 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.660608+0000", 2024-09-16T09:27:42.164 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.074783555000000002 2024-09-16T09:27:42.164 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.164 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.164 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:27:42.164 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.660657+0000", 2024-09-16T09:27:42.165 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.9633999999999999e-05 2024-09-16T09:27:42.165 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.165 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.165 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:27:42.165 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.660695+0000", 2024-09-16T09:27:42.165 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.7533000000000001e-05 2024-09-16T09:27:42.165 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:27:42.165 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:27:42.165 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:27:42.165 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.165 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.165 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:22937 7.c 7:30196b2c:::benchmark_data_smithi070_78438_object22936:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T09:27:42.165 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:26:21.585988+0000", 2024-09-16T09:27:42.166 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 80.564697787, 2024-09-16T09:27:42.166 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.074856724999999999, 2024-09-16T09:27:42.166 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:27:42.166 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:27:42.166 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:27:42.166 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:27:42.166 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:27:42.166 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:27:42.166 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 22937 2024-09-16T09:27:42.166 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.166 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:27:42.166 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.166 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:27:42.166 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.585988+0000", 2024-09-16T09:27:42.167 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:27:42.167 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.167 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.167 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:27:42.167 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.585988+0000", 2024-09-16T09:27:42.167 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:27:42.167 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.167 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.167 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:27:42.167 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.585990+0000", 2024-09-16T09:27:42.167 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.235e-06 2024-09-16T09:27:42.167 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.167 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.167 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:27:42.168 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.586041+0000", 2024-09-16T09:27:42.168 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.0979000000000001e-05 2024-09-16T09:27:42.168 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.168 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.168 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:27:42.168 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.586050+0000", 2024-09-16T09:27:42.168 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.2799999999999992e-06 2024-09-16T09:27:42.168 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.168 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.168 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:27:42.168 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.586054+0000", 2024-09-16T09:27:42.168 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.4009999999999999e-06 2024-09-16T09:27:42.168 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.168 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.169 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:27:42.169 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.586093+0000", 2024-09-16T09:27:42.169 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.9122000000000003e-05 2024-09-16T09:27:42.169 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.169 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.169 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:27:42.169 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.586166+0000", 2024-09-16T09:27:42.169 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.2849999999999995e-05 2024-09-16T09:27:42.169 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.170 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.170 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 3,7", 2024-09-16T09:27:42.170 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.586243+0000", 2024-09-16T09:27:42.170 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.6413000000000002e-05 2024-09-16T09:27:42.170 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.170 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.170 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:27:42.170 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.587005+0000", 2024-09-16T09:27:42.170 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00076235300000000001 2024-09-16T09:27:42.170 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.170 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.170 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:27:42.171 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.587161+0000", 2024-09-16T09:27:42.171 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000156117 2024-09-16T09:27:42.171 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.171 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.171 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:27:42.171 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.660740+0000", 2024-09-16T09:27:42.171 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.073579153999999994 2024-09-16T09:27:42.171 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.171 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.171 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:27:42.172 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.660798+0000", 2024-09-16T09:27:42.172 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.7648000000000001e-05 2024-09-16T09:27:42.172 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.172 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.172 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:27:42.172 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.660845+0000", 2024-09-16T09:27:42.172 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.7172999999999997e-05 2024-09-16T09:27:42.173 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:27:42.173 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:27:42.173 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:27:42.173 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.173 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.173 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:23489 7.7 7:ea3d3f31:::benchmark_data_smithi070_78438_object23488:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T09:27:42.173 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:26:21.886108+0000", 2024-09-16T09:27:42.173 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 80.264578470000004, 2024-09-16T09:27:42.174 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.074098949999999997, 2024-09-16T09:27:42.174 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:27:42.174 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:27:42.174 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:27:42.174 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:27:42.174 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:27:42.174 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:27:42.175 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 23489 2024-09-16T09:27:42.175 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.175 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:27:42.175 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.175 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:27:42.175 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.886108+0000", 2024-09-16T09:27:42.175 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:27:42.175 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.176 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.176 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:27:42.176 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.886108+0000", 2024-09-16T09:27:42.176 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:27:42.176 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.176 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.176 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:27:42.176 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.886111+0000", 2024-09-16T09:27:42.177 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.836e-06 2024-09-16T09:27:42.177 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.177 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.177 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:27:42.177 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.886173+0000", 2024-09-16T09:27:42.177 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.2782999999999998e-05 2024-09-16T09:27:42.177 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.178 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.178 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:27:42.178 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.886183+0000", 2024-09-16T09:27:42.178 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.764e-06 2024-09-16T09:27:42.178 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.178 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.178 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:27:42.179 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.886189+0000", 2024-09-16T09:27:42.179 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.3589999999999999e-06 2024-09-16T09:27:42.179 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.179 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.179 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:27:42.179 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.886247+0000", 2024-09-16T09:27:42.179 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.8740999999999999e-05 2024-09-16T09:27:42.179 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.179 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.179 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:27:42.179 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.886373+0000", 2024-09-16T09:27:42.180 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00012565800000000001 2024-09-16T09:27:42.180 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.180 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.180 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,3", 2024-09-16T09:27:42.180 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.886520+0000", 2024-09-16T09:27:42.180 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00014732999999999999 2024-09-16T09:27:42.180 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.181 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.181 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:27:42.181 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.888349+0000", 2024-09-16T09:27:42.181 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0018292549999999999 2024-09-16T09:27:42.181 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.181 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.181 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:27:42.182 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.888384+0000", 2024-09-16T09:27:42.182 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.4728999999999999e-05 2024-09-16T09:27:42.182 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.182 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.182 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:27:42.182 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.960031+0000", 2024-09-16T09:27:42.183 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.071646575000000004 2024-09-16T09:27:42.183 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.183 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.183 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:27:42.184 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.960118+0000", 2024-09-16T09:27:42.184 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.7422000000000004e-05 2024-09-16T09:27:42.184 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.184 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.184 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:27:42.184 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.960207+0000", 2024-09-16T09:27:42.184 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.8498000000000005e-05 2024-09-16T09:27:42.184 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:27:42.184 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:27:42.185 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:27:42.185 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.185 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.185 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:23490 7.7 7:e85ca95f:::benchmark_data_smithi070_78438_object23489:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T09:27:42.185 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:26:21.886202+0000", 2024-09-16T09:27:42.186 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 80.264484382000006, 2024-09-16T09:27:42.186 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.074161913999999995, 2024-09-16T09:27:42.186 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:27:42.186 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:27:42.186 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:27:42.186 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:27:42.186 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:27:42.186 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:27:42.186 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 23490 2024-09-16T09:27:42.186 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.187 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:27:42.189 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.189 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:27:42.190 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.886202+0000", 2024-09-16T09:27:42.190 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:27:42.190 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.190 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.190 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:27:42.190 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.886202+0000", 2024-09-16T09:27:42.190 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:27:42.190 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.190 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.190 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:27:42.190 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.886202+0000", 2024-09-16T09:27:42.190 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.44e-07 2024-09-16T09:27:42.190 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.190 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.191 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:27:42.191 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.886236+0000", 2024-09-16T09:27:42.191 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.3572000000000003e-05 2024-09-16T09:27:42.191 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.191 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.191 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:27:42.191 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.886243+0000", 2024-09-16T09:27:42.191 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.9940000000000003e-06 2024-09-16T09:27:42.191 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.191 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.191 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:27:42.191 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.886248+0000", 2024-09-16T09:27:42.191 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.9679999999999997e-06 2024-09-16T09:27:42.191 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.192 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.192 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:27:42.192 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.887340+0000", 2024-09-16T09:27:42.192 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0010923129999999999 2024-09-16T09:27:42.192 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.192 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.192 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:27:42.192 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.887448+0000", 2024-09-16T09:27:42.192 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00010743699999999999 2024-09-16T09:27:42.192 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.192 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.192 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,3", 2024-09-16T09:27:42.192 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.887600+0000", 2024-09-16T09:27:42.193 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00015224800000000001 2024-09-16T09:27:42.193 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.193 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.193 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:27:42.193 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.888460+0000", 2024-09-16T09:27:42.193 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00086027 2024-09-16T09:27:42.193 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.193 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.193 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:27:42.193 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.888959+0000", 2024-09-16T09:27:42.193 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00049850399999999998 2024-09-16T09:27:42.193 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.193 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.194 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:27:42.194 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.960280+0000", 2024-09-16T09:27:42.194 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.071321368999999996 2024-09-16T09:27:42.194 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.194 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.194 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:27:42.194 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.960331+0000", 2024-09-16T09:27:42.194 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.0810000000000003e-05 2024-09-16T09:27:42.194 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.194 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.194 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:27:42.194 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.960364+0000", 2024-09-16T09:27:42.194 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.2885000000000001e-05 2024-09-16T09:27:42.194 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:27:42.195 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:27:42.195 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:27:42.195 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.195 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.195 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:27455 7.7 7:efc48835:::benchmark_data_smithi070_78438_object27454:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T09:27:42.195 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:26:23.713213+0000", 2024-09-16T09:27:42.195 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 78.437473120000007, 2024-09-16T09:27:42.195 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.076424311999999994, 2024-09-16T09:27:42.195 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:27:42.195 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:27:42.195 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:27:42.195 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:27:42.195 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:27:42.195 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:27:42.196 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 27455 2024-09-16T09:27:42.196 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.196 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:27:42.196 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.196 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:27:42.196 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.713213+0000", 2024-09-16T09:27:42.196 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:27:42.196 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.196 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.196 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:27:42.196 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.713213+0000", 2024-09-16T09:27:42.196 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:27:42.197 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:27:42.198 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:27:41 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:27:41] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:27:42.198 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.199 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.199 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:27:42.199 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.713214+0000", 2024-09-16T09:27:42.199 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1310000000000001e-06 2024-09-16T09:27:42.199 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.199 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.199 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:27:42.199 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.713258+0000", 2024-09-16T09:27:42.199 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.4184999999999998e-05 2024-09-16T09:27:42.199 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.199 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.199 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:27:42.200 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.713270+0000", 2024-09-16T09:27:42.200 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1725999999999999e-05 2024-09-16T09:27:42.200 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.200 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.200 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:27:42.200 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.713277+0000", 2024-09-16T09:27:42.200 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.277e-06 2024-09-16T09:27:42.200 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.200 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.200 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:27:42.200 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.713324+0000", 2024-09-16T09:27:42.200 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.6136e-05 2024-09-16T09:27:42.200 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.200 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.201 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:27:42.201 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.713432+0000", 2024-09-16T09:27:42.201 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00010794300000000001 2024-09-16T09:27:42.201 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.201 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.201 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,3", 2024-09-16T09:27:42.201 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.713557+0000", 2024-09-16T09:27:42.201 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00012519699999999999 2024-09-16T09:27:42.201 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.201 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.201 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:27:42.201 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.714811+0000", 2024-09-16T09:27:42.201 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0012543750000000001 2024-09-16T09:27:42.201 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.202 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.202 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:27:42.202 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.719060+0000", 2024-09-16T09:27:42.202 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0042491819999999998 2024-09-16T09:27:42.202 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.202 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.202 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:27:42.202 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.789543+0000", 2024-09-16T09:27:42.202 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.070482793000000002 2024-09-16T09:27:42.202 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.202 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.202 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:27:42.202 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.789593+0000", 2024-09-16T09:27:42.202 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.0012000000000002e-05 2024-09-16T09:27:42.203 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.203 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.203 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:27:42.203 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.789637+0000", 2024-09-16T09:27:42.203 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.4354999999999998e-05 2024-09-16T09:27:42.203 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:27:42.203 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:27:42.203 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:27:42.203 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.203 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.203 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:27458 7.7 7:e5d53bde:::benchmark_data_smithi070_78438_object27457:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T09:27:42.203 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:26:23.716938+0000", 2024-09-16T09:27:42.203 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 78.433748614999999, 2024-09-16T09:27:42.203 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.072814842000000005, 2024-09-16T09:27:42.204 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:27:42.204 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:27:42.204 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:27:42.204 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:27:42.204 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:27:42.204 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:27:42.204 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 27458 2024-09-16T09:27:42.204 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.204 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:27:42.204 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.204 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:27:42.204 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.716938+0000", 2024-09-16T09:27:42.204 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:27:42.204 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.205 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.205 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:27:42.205 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.716938+0000", 2024-09-16T09:27:42.205 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:27:42.205 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.205 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.205 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:27:42.205 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.716939+0000", 2024-09-16T09:27:42.205 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.5039999999999999e-06 2024-09-16T09:27:42.205 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.205 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.205 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:27:42.205 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.717047+0000", 2024-09-16T09:27:42.205 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000108298 2024-09-16T09:27:42.206 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.206 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.206 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:27:42.206 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.717057+0000", 2024-09-16T09:27:42.206 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.9359999999999994e-06 2024-09-16T09:27:42.206 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.206 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.206 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:27:42.206 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.717064+0000", 2024-09-16T09:27:42.206 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.6390000000000003e-06 2024-09-16T09:27:42.206 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.206 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.206 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:27:42.206 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.717111+0000", 2024-09-16T09:27:42.207 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.6848999999999998e-05 2024-09-16T09:27:42.207 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.207 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.207 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:27:42.207 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.717181+0000", 2024-09-16T09:27:42.207 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.0151e-05 2024-09-16T09:27:42.207 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.207 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.207 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,3", 2024-09-16T09:27:42.207 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.717300+0000", 2024-09-16T09:27:42.207 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000119406 2024-09-16T09:27:42.207 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.207 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.207 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:27:42.207 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.718260+0000", 2024-09-16T09:27:42.208 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00095984499999999997 2024-09-16T09:27:42.208 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.208 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.208 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:27:42.208 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.788644+0000", 2024-09-16T09:27:42.208 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.070383904999999997 2024-09-16T09:27:42.208 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.208 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.208 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:27:42.208 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.789678+0000", 2024-09-16T09:27:42.208 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00103418 2024-09-16T09:27:42.208 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.208 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.208 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:27:42.209 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.789721+0000", 2024-09-16T09:27:42.209 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.2583999999999998e-05 2024-09-16T09:27:42.209 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.209 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.209 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:27:42.209 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.789752+0000", 2024-09-16T09:27:42.209 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.1544999999999997e-05 2024-09-16T09:27:42.209 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:27:42.209 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:27:42.212 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:27:42.212 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.212 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.212 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:27465 7.c 7:39cb8bcd:::benchmark_data_smithi070_78438_object27464:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T09:27:42.212 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:26:23.720047+0000", 2024-09-16T09:27:42.212 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 78.430639709000005, 2024-09-16T09:27:42.212 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.072980312000000006, 2024-09-16T09:27:42.212 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:27:42.213 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:27:42.213 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:27:42.213 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:27:42.213 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:27:42.213 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:27:42.213 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 27465 2024-09-16T09:27:42.213 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.213 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:27:42.213 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.213 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:27:42.213 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.720047+0000", 2024-09-16T09:27:42.213 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:27:42.213 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.213 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.214 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:27:42.214 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.720047+0000", 2024-09-16T09:27:42.214 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:27:42.214 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.214 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.214 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:27:42.214 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.720047+0000", 2024-09-16T09:27:42.214 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.2099999999999998e-07 2024-09-16T09:27:42.214 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.214 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.214 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:27:42.214 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.720069+0000", 2024-09-16T09:27:42.214 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.1993000000000001e-05 2024-09-16T09:27:42.214 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.215 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.215 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:27:42.215 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.720074+0000", 2024-09-16T09:27:42.215 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.724e-06 2024-09-16T09:27:42.215 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.215 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.215 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:27:42.215 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.720077+0000", 2024-09-16T09:27:42.215 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.1949999999999999e-06 2024-09-16T09:27:42.215 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.215 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.215 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:27:42.215 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.720124+0000", 2024-09-16T09:27:42.216 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.7309999999999999e-05 2024-09-16T09:27:42.216 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.216 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.216 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:27:42.216 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.788581+0000", 2024-09-16T09:27:42.216 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.068457341000000005 2024-09-16T09:27:42.216 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.216 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.216 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 3,7", 2024-09-16T09:27:42.216 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.788703+0000", 2024-09-16T09:27:42.216 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000121644 2024-09-16T09:27:42.216 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.216 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.216 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:27:42.217 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.789782+0000", 2024-09-16T09:27:42.217 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.001079317 2024-09-16T09:27:42.217 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.217 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.217 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:27:42.217 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.790071+0000", 2024-09-16T09:27:42.217 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00028908999999999999 2024-09-16T09:27:42.217 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.217 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.217 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:27:42.217 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.792927+0000", 2024-09-16T09:27:42.217 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0028552930000000001 2024-09-16T09:27:42.217 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.217 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.218 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:27:42.218 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.792991+0000", 2024-09-16T09:27:42.218 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.3923000000000004e-05 2024-09-16T09:27:42.218 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.218 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.218 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:27:42.218 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.793027+0000", 2024-09-16T09:27:42.218 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.6161000000000002e-05 2024-09-16T09:27:42.218 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:27:42.218 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:27:42.218 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:27:42.218 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.218 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.218 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:29083 7.7 7:ecb95ffb:::benchmark_data_smithi070_78438_object29082:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-16T09:27:42.219 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:26:24.649421+0000", 2024-09-16T09:27:42.219 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 77.501264895999995, 2024-09-16T09:27:42.219 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.075044315, 2024-09-16T09:27:42.219 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:27:42.219 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:27:42.219 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:27:42.219 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:27:42.219 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:27:42.219 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:27:42.219 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 29083 2024-09-16T09:27:42.219 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.219 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:27:42.219 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.219 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:27:42.220 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.649421+0000", 2024-09-16T09:27:42.220 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:27:42.220 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.220 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.220 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:27:42.220 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.649421+0000", 2024-09-16T09:27:42.220 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:27:42.220 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.220 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.220 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:27:42.220 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.649422+0000", 2024-09-16T09:27:42.220 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1379999999999999e-06 2024-09-16T09:27:42.220 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.220 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.221 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:27:42.221 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.649477+0000", 2024-09-16T09:27:42.221 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.4527999999999997e-05 2024-09-16T09:27:42.221 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.221 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.221 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:27:42.221 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.649488+0000", 2024-09-16T09:27:42.221 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0868e-05 2024-09-16T09:27:42.221 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.221 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.221 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:27:42.221 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.649493+0000", 2024-09-16T09:27:42.221 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.2109999999999997e-06 2024-09-16T09:27:42.221 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.222 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.222 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:27:42.222 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.649533+0000", 2024-09-16T09:27:42.222 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.0309999999999999e-05 2024-09-16T09:27:42.222 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.222 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.222 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:27:42.222 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.649618+0000", 2024-09-16T09:27:42.222 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.4889999999999995e-05 2024-09-16T09:27:42.222 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.222 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.222 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,3", 2024-09-16T09:27:42.222 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.649701+0000", 2024-09-16T09:27:42.222 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.2830000000000005e-05 2024-09-16T09:27:42.223 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.223 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.223 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:27:42.223 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.650834+0000", 2024-09-16T09:27:42.223 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0011331399999999999 2024-09-16T09:27:42.223 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.223 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.223 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:27:42.223 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.658254+0000", 2024-09-16T09:27:42.223 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.007420238 2024-09-16T09:27:42.223 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.223 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.223 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:27:42.223 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.724402+0000", 2024-09-16T09:27:42.223 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.066147819999999996 2024-09-16T09:27:42.224 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.224 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.224 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:27:42.224 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.724436+0000", 2024-09-16T09:27:42.224 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.3847999999999998e-05 2024-09-16T09:27:42.224 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.224 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.224 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:27:42.224 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.724466+0000", 2024-09-16T09:27:42.224 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.9493999999999999e-05 2024-09-16T09:27:42.224 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:27:42.224 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:27:42.224 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:27:42.225 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.225 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.225 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:29088 7.7 7:e96c9606:::benchmark_data_smithi070_78438_object29087:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-16T09:27:42.225 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:26:24.651003+0000", 2024-09-16T09:27:42.225 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 77.499682993999997, 2024-09-16T09:27:42.225 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.073554288999999995, 2024-09-16T09:27:42.225 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:27:42.225 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:27:42.225 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:27:42.225 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:27:42.225 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:27:42.225 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:27:42.225 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 29088 2024-09-16T09:27:42.225 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.226 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:27:42.226 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.226 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:27:42.226 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.651003+0000", 2024-09-16T09:27:42.226 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:27:42.226 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.226 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.226 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:27:42.226 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.651003+0000", 2024-09-16T09:27:42.226 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:27:42.226 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.226 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.226 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:27:42.226 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.651004+0000", 2024-09-16T09:27:42.227 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.9999999999999995e-07 2024-09-16T09:27:42.227 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.227 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.227 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:27:42.227 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.651063+0000", 2024-09-16T09:27:42.227 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.8925000000000002e-05 2024-09-16T09:27:42.227 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.227 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.227 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:27:42.227 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.651077+0000", 2024-09-16T09:27:42.227 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.3482999999999999e-05 2024-09-16T09:27:42.227 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.227 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.227 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:27:42.227 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.651084+0000", 2024-09-16T09:27:42.228 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.5870000000000004e-06 2024-09-16T09:27:42.228 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.228 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.228 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:27:42.228 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.651133+0000", 2024-09-16T09:27:42.228 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.8322999999999998e-05 2024-09-16T09:27:42.228 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.228 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.228 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:27:42.228 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.651254+0000", 2024-09-16T09:27:42.228 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000121161 2024-09-16T09:27:42.228 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.228 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.229 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,3", 2024-09-16T09:27:42.229 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.651384+0000", 2024-09-16T09:27:42.229 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00013054599999999999 2024-09-16T09:27:42.229 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.229 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.229 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:27:42.229 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.652452+0000", 2024-09-16T09:27:42.229 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.001068238 2024-09-16T09:27:42.229 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.229 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.229 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:27:42.229 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.658307+0000", 2024-09-16T09:27:42.229 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0058544369999999997 2024-09-16T09:27:42.230 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.230 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.230 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:27:42.230 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.724497+0000", 2024-09-16T09:27:42.230 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.066189577999999999 2024-09-16T09:27:42.230 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.230 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.230 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:27:42.230 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.724533+0000", 2024-09-16T09:27:42.230 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.6060999999999999e-05 2024-09-16T09:27:42.230 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.230 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.230 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:27:42.230 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.724558+0000", 2024-09-16T09:27:42.230 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.495e-05 2024-09-16T09:27:42.231 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:27:42.231 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:27:42.231 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:27:42.231 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.231 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.231 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:60245 7.7 7:e5610e70:::benchmark_data_smithi070_78438_object60244:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-16T09:27:42.231 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:26:44.352114+0000", 2024-09-16T09:27:42.231 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 57.798572708000002, 2024-09-16T09:27:42.231 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.127497575, 2024-09-16T09:27:42.231 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:27:42.231 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:27:42.231 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:27:42.231 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:27:42.231 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:27:42.232 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:27:42.232 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 60245 2024-09-16T09:27:42.232 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.232 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:27:42.232 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.232 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:27:42.232 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.352114+0000", 2024-09-16T09:27:42.232 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:27:42.232 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.232 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.232 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:27:42.232 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.352114+0000", 2024-09-16T09:27:42.232 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:27:42.232 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.233 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.233 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:27:42.233 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.352115+0000", 2024-09-16T09:27:42.233 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0449999999999999e-06 2024-09-16T09:27:42.233 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.233 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.233 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:27:42.233 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.352162+0000", 2024-09-16T09:27:42.233 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.7416000000000001e-05 2024-09-16T09:27:42.233 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.233 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.233 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:27:42.233 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.352174+0000", 2024-09-16T09:27:42.233 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.2347999999999999e-05 2024-09-16T09:27:42.234 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.234 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.234 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:27:42.234 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.352181+0000", 2024-09-16T09:27:42.234 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.8809999999999998e-06 2024-09-16T09:27:42.234 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.234 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.234 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:27:42.234 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.352231+0000", 2024-09-16T09:27:42.234 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.9292000000000002e-05 2024-09-16T09:27:42.234 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.234 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.234 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:27:42.234 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.352373+0000", 2024-09-16T09:27:42.235 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00014240699999999999 2024-09-16T09:27:42.235 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.235 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.235 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,3", 2024-09-16T09:27:42.235 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.352494+0000", 2024-09-16T09:27:42.235 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000121039 2024-09-16T09:27:42.235 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.235 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.235 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:27:42.235 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.354404+0000", 2024-09-16T09:27:42.235 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0019098979999999999 2024-09-16T09:27:42.235 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.235 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.235 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:27:42.236 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.355369+0000", 2024-09-16T09:27:42.236 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00096560899999999998 2024-09-16T09:27:42.236 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.236 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.236 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:27:42.236 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.479497+0000", 2024-09-16T09:27:42.236 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.124127398 2024-09-16T09:27:42.236 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.236 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.236 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:27:42.236 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.479559+0000", 2024-09-16T09:27:42.236 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.1692000000000005e-05 2024-09-16T09:27:42.236 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.236 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.237 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:27:42.237 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.479611+0000", 2024-09-16T09:27:42.237 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.2549999999999997e-05 2024-09-16T09:27:42.237 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:27:42.237 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:27:42.237 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:27:42.237 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.237 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.237 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:60246 7.d 7:b1444fdd:::benchmark_data_smithi070_78438_object60245:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-16T09:27:42.237 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:26:44.352405+0000", 2024-09-16T09:27:42.237 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 57.798281316999997, 2024-09-16T09:27:42.237 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.13626988500000001, 2024-09-16T09:27:42.237 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:27:42.237 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:27:42.238 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:27:42.238 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:27:42.238 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:27:42.238 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:27:42.238 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 60246 2024-09-16T09:27:42.238 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.238 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:27:42.238 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.238 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:27:42.238 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.352405+0000", 2024-09-16T09:27:42.238 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:27:42.238 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.238 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.238 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:27:42.238 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.352405+0000", 2024-09-16T09:27:42.239 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:27:42.239 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.239 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.239 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:27:42.239 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.352406+0000", 2024-09-16T09:27:42.239 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.85e-07 2024-09-16T09:27:42.239 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.239 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.239 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:27:42.239 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.352431+0000", 2024-09-16T09:27:42.239 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.5364e-05 2024-09-16T09:27:42.239 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.239 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.239 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:27:42.240 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.352440+0000", 2024-09-16T09:27:42.240 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.9509999999999995e-06 2024-09-16T09:27:42.240 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.240 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.240 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:27:42.240 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.352446+0000", 2024-09-16T09:27:42.240 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.2099999999999998e-06 2024-09-16T09:27:42.240 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.240 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.240 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:27:42.240 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.352486+0000", 2024-09-16T09:27:42.240 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.9475999999999999e-05 2024-09-16T09:27:42.240 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.240 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.241 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:27:42.241 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.352575+0000", 2024-09-16T09:27:42.241 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.9239000000000004e-05 2024-09-16T09:27:42.241 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.241 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.241 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,1", 2024-09-16T09:27:42.241 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.352693+0000", 2024-09-16T09:27:42.241 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000118248 2024-09-16T09:27:42.241 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.242 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.242 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:27:42.242 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.354498+0000", 2024-09-16T09:27:42.242 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0018052890000000001 2024-09-16T09:27:42.242 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.242 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.242 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:27:42.242 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.355812+0000", 2024-09-16T09:27:42.242 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0013137380000000001 2024-09-16T09:27:42.242 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.242 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.242 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:27:42.242 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.488574+0000", 2024-09-16T09:27:42.243 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.13276227400000001 2024-09-16T09:27:42.243 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.243 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.243 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:27:42.243 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.488632+0000", 2024-09-16T09:27:42.243 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.7970999999999998e-05 2024-09-16T09:27:42.243 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.243 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.243 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:27:42.243 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.488675+0000", 2024-09-16T09:27:42.243 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.2339999999999998e-05 2024-09-16T09:27:42.243 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:27:42.243 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:27:42.243 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:27:42.243 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.244 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.244 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:60253 7.d 7:bc8ca5cf:::benchmark_data_smithi070_78438_object60252:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-16T09:27:42.244 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:26:44.357181+0000", 2024-09-16T09:27:42.244 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 57.793505682000003, 2024-09-16T09:27:42.244 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.131612381, 2024-09-16T09:27:42.244 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:27:42.244 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:27:42.244 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:27:42.244 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:27:42.244 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:27:42.244 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:27:42.244 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 60253 2024-09-16T09:27:42.244 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.245 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:27:42.245 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.245 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:27:42.245 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.357181+0000", 2024-09-16T09:27:42.245 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:27:42.245 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.245 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.245 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:27:42.245 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.357181+0000", 2024-09-16T09:27:42.245 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:27:42.246 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.246 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.246 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:27:42.246 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.357182+0000", 2024-09-16T09:27:42.246 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.2270000000000001e-06 2024-09-16T09:27:42.246 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.246 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.246 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:27:42.246 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.357244+0000", 2024-09-16T09:27:42.246 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.2683999999999998e-05 2024-09-16T09:27:42.246 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.246 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.246 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:27:42.246 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.357255+0000", 2024-09-16T09:27:42.247 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0720000000000001e-05 2024-09-16T09:27:42.247 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.247 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.247 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:27:42.247 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.357262+0000", 2024-09-16T09:27:42.247 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.7800000000000003e-06 2024-09-16T09:27:42.247 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.247 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.247 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:27:42.247 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.357305+0000", 2024-09-16T09:27:42.247 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.2667999999999999e-05 2024-09-16T09:27:42.247 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.247 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.247 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:27:42.248 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.357399+0000", 2024-09-16T09:27:42.248 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.4217999999999998e-05 2024-09-16T09:27:42.248 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.248 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.248 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,1", 2024-09-16T09:27:42.248 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.357492+0000", 2024-09-16T09:27:42.248 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.3213000000000004e-05 2024-09-16T09:27:42.248 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.248 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.248 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:27:42.248 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.358931+0000", 2024-09-16T09:27:42.248 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.001439357 2024-09-16T09:27:42.248 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.248 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.249 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:27:42.249 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.359185+0000", 2024-09-16T09:27:42.249 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00025320799999999999 2024-09-16T09:27:42.249 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.249 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.249 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:27:42.249 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.488722+0000", 2024-09-16T09:27:42.249 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.12953768399999999 2024-09-16T09:27:42.249 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.249 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.249 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:27:42.249 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.488763+0000", 2024-09-16T09:27:42.249 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.1059000000000001e-05 2024-09-16T09:27:42.249 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.250 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.250 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:27:42.250 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.488793+0000", 2024-09-16T09:27:42.250 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.9563000000000001e-05 2024-09-16T09:27:42.250 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:27:42.250 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:27:42.250 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:27:42.250 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.250 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.250 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:62034 7.d 7:b4ff4322:::benchmark_data_smithi070_78438_object62033:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-16T09:27:42.250 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:26:45.811296+0000", 2024-09-16T09:27:42.250 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 56.339389847, 2024-09-16T09:27:42.250 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.074127372999999996, 2024-09-16T09:27:42.250 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:27:42.251 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:27:42.251 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:27:42.251 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:27:42.251 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:27:42.251 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:27:42.251 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 62034 2024-09-16T09:27:42.251 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.251 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:27:42.251 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.251 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:27:42.251 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.811296+0000", 2024-09-16T09:27:42.251 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:27:42.251 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.251 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.252 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:27:42.252 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.811296+0000", 2024-09-16T09:27:42.252 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:27:42.252 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.252 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.252 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:27:42.252 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.811298+0000", 2024-09-16T09:27:42.252 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.04e-06 2024-09-16T09:27:42.252 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.252 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.252 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:27:42.252 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.811351+0000", 2024-09-16T09:27:42.252 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.2772999999999998e-05 2024-09-16T09:27:42.252 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.253 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.253 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:27:42.253 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.811362+0000", 2024-09-16T09:27:42.253 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0871e-05 2024-09-16T09:27:42.253 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.253 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.253 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:27:42.253 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.811370+0000", 2024-09-16T09:27:42.253 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.4880000000000001e-06 2024-09-16T09:27:42.253 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.254 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.254 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:27:42.254 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.811424+0000", 2024-09-16T09:27:42.254 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.4246999999999997e-05 2024-09-16T09:27:42.254 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.254 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.254 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:27:42.254 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.811550+0000", 2024-09-16T09:27:42.254 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000125909 2024-09-16T09:27:42.254 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.254 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.254 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,1", 2024-09-16T09:27:42.254 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.811685+0000", 2024-09-16T09:27:42.254 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000135716 2024-09-16T09:27:42.255 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.255 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.255 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:27:42.255 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.816428+0000", 2024-09-16T09:27:42.255 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0047421690000000001 2024-09-16T09:27:42.255 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.255 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.255 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:27:42.255 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.816624+0000", 2024-09-16T09:27:42.255 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000195937 2024-09-16T09:27:42.255 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.255 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.255 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:27:42.255 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.885351+0000", 2024-09-16T09:27:42.256 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.068727413000000001 2024-09-16T09:27:42.256 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.256 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.256 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:27:42.256 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.885392+0000", 2024-09-16T09:27:42.256 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.0571999999999997e-05 2024-09-16T09:27:42.256 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.256 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.256 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:27:42.256 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.885424+0000", 2024-09-16T09:27:42.256 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.2237999999999999e-05 2024-09-16T09:27:42.256 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:27:42.256 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:27:42.257 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:27:42.257 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.257 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.257 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:62035 7.7 7:eb1581a3:::benchmark_data_smithi070_78438_object62034:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-16T09:27:42.257 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:26:45.812442+0000", 2024-09-16T09:27:42.257 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 56.338244232999998, 2024-09-16T09:27:42.257 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.073051295000000002, 2024-09-16T09:27:42.257 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:27:42.257 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:27:42.257 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:27:42.257 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:27:42.257 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:27:42.257 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:27:42.257 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 62035 2024-09-16T09:27:42.258 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.258 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:27:42.258 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.258 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:27:42.258 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.812442+0000", 2024-09-16T09:27:42.258 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:27:42.258 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.258 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.258 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:27:42.258 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.812442+0000", 2024-09-16T09:27:42.258 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:27:42.258 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.258 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.258 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:27:42.259 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.812443+0000", 2024-09-16T09:27:42.259 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.9800000000000003e-07 2024-09-16T09:27:42.259 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.259 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.259 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:27:42.259 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.812507+0000", 2024-09-16T09:27:42.259 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.4286000000000001e-05 2024-09-16T09:27:42.259 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.259 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.259 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:27:42.259 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.812517+0000", 2024-09-16T09:27:42.259 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0151e-05 2024-09-16T09:27:42.259 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.259 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.260 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:27:42.260 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.812522+0000", 2024-09-16T09:27:42.260 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.9200000000000003e-06 2024-09-16T09:27:42.260 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.260 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.260 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:27:42.260 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.812574+0000", 2024-09-16T09:27:42.260 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.1452999999999997e-05 2024-09-16T09:27:42.260 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.260 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.260 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:27:42.260 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.812694+0000", 2024-09-16T09:27:42.260 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000120697 2024-09-16T09:27:42.260 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.261 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.261 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,3", 2024-09-16T09:27:42.261 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.812889+0000", 2024-09-16T09:27:42.261 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00019489099999999999 2024-09-16T09:27:42.261 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.261 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.261 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:27:42.261 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.815273+0000", 2024-09-16T09:27:42.261 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.002384185 2024-09-16T09:27:42.261 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.261 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.261 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:27:42.261 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.877322+0000", 2024-09-16T09:27:42.261 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.062048535000000002 2024-09-16T09:27:42.262 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.262 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.262 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:27:42.262 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.885391+0000", 2024-09-16T09:27:42.262 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0080691510000000001 2024-09-16T09:27:42.262 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.262 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.262 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:27:42.262 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.885452+0000", 2024-09-16T09:27:42.262 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.0520999999999999e-05 2024-09-16T09:27:42.262 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.262 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.262 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:27:42.262 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.885493+0000", 2024-09-16T09:27:42.263 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.1706999999999999e-05 2024-09-16T09:27:42.263 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:27:42.263 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:27:42.263 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:27:42.263 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.263 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.263 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:62036 7.d 7:b755228e:::benchmark_data_smithi070_78438_object62035:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-16T09:27:42.263 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:26:45.812637+0000", 2024-09-16T09:27:42.263 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 56.338049581999996, 2024-09-16T09:27:42.263 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.078342939, 2024-09-16T09:27:42.263 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:27:42.263 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:27:42.263 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:27:42.263 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:27:42.264 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:27:42.264 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:27:42.264 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 62036 2024-09-16T09:27:42.264 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.264 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:27:42.264 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.264 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:27:42.264 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.812637+0000", 2024-09-16T09:27:42.264 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:27:42.264 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.264 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.264 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:27:42.264 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.812637+0000", 2024-09-16T09:27:42.264 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:27:42.265 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.265 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.265 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:27:42.265 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.812638+0000", 2024-09-16T09:27:42.265 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.7599999999999996e-07 2024-09-16T09:27:42.265 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.265 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.265 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:27:42.265 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.812759+0000", 2024-09-16T09:27:42.265 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000121045 2024-09-16T09:27:42.265 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.265 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.265 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:27:42.265 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.812769+0000", 2024-09-16T09:27:42.266 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0097e-05 2024-09-16T09:27:42.266 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.266 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.266 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:27:42.266 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.812774+0000", 2024-09-16T09:27:42.266 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.7089999999999999e-06 2024-09-16T09:27:42.266 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.266 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.266 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:27:42.266 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.814165+0000", 2024-09-16T09:27:42.266 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0013907189999999999 2024-09-16T09:27:42.266 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.266 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.266 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:27:42.267 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.814343+0000", 2024-09-16T09:27:42.267 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00017806500000000001 2024-09-16T09:27:42.267 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.267 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.267 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,1", 2024-09-16T09:27:42.267 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.814503+0000", 2024-09-16T09:27:42.267 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00015934599999999999 2024-09-16T09:27:42.267 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.267 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.267 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:27:42.267 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.816695+0000", 2024-09-16T09:27:42.267 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0021919970000000002 2024-09-16T09:27:42.267 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.267 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.268 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:27:42.268 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.817690+0000", 2024-09-16T09:27:42.268 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00099578899999999996 2024-09-16T09:27:42.268 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.268 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.268 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:27:42.268 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.890916+0000", 2024-09-16T09:27:42.268 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.073225861000000003 2024-09-16T09:27:42.268 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.268 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.268 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:27:42.268 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.890953+0000", 2024-09-16T09:27:42.268 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.6554999999999998e-05 2024-09-16T09:27:42.268 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.268 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.269 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:27:42.269 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.890980+0000", 2024-09-16T09:27:42.269 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.688e-05 2024-09-16T09:27:42.269 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:27:42.269 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:27:42.269 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:27:42.269 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.269 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.269 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:62038 7.7 7:e2c33fc5:::benchmark_data_smithi070_78438_object62037:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-16T09:27:42.269 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:26:45.814362+0000", 2024-09-16T09:27:42.269 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 56.336323735999997, 2024-09-16T09:27:42.269 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.076701338999999993, 2024-09-16T09:27:42.269 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:27:42.270 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:27:42.270 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:27:42.270 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:27:42.270 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:27:42.270 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:27:42.270 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 62038 2024-09-16T09:27:42.270 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.270 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:27:42.270 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.270 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:27:42.270 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.814362+0000", 2024-09-16T09:27:42.270 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:27:42.270 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.270 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.270 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:27:42.271 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.814362+0000", 2024-09-16T09:27:42.271 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:27:42.271 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.271 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.271 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:27:42.271 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.814364+0000", 2024-09-16T09:27:42.271 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.2750000000000001e-06 2024-09-16T09:27:42.271 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.271 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.271 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:27:42.271 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.814391+0000", 2024-09-16T09:27:42.271 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.6944000000000002e-05 2024-09-16T09:27:42.271 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.271 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.272 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:27:42.272 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.814400+0000", 2024-09-16T09:27:42.272 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.4979999999999995e-06 2024-09-16T09:27:42.272 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.272 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.272 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:27:42.272 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.814407+0000", 2024-09-16T09:27:42.272 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.4180000000000002e-06 2024-09-16T09:27:42.272 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.272 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.272 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:27:42.272 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.815301+0000", 2024-09-16T09:27:42.273 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000894837 2024-09-16T09:27:42.273 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.273 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.273 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:27:42.273 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.815436+0000", 2024-09-16T09:27:42.273 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00013441700000000001 2024-09-16T09:27:42.273 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.273 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.273 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,3", 2024-09-16T09:27:42.273 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.815570+0000", 2024-09-16T09:27:42.273 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00013415899999999999 2024-09-16T09:27:42.273 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.273 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.274 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:27:42.274 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.817959+0000", 2024-09-16T09:27:42.274 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.002389108 2024-09-16T09:27:42.274 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.274 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.274 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:27:42.274 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.877370+0000", 2024-09-16T09:27:42.274 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.059411322000000003 2024-09-16T09:27:42.274 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.274 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.274 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:27:42.274 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.890977+0000", 2024-09-16T09:27:42.274 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.013606114000000001 2024-09-16T09:27:42.274 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.275 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.275 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:27:42.275 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.891024+0000", 2024-09-16T09:27:42.275 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.7231000000000002e-05 2024-09-16T09:27:42.275 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.275 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.275 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:27:42.275 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.891064+0000", 2024-09-16T09:27:42.275 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.0015999999999998e-05 2024-09-16T09:27:42.275 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:27:42.275 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:27:42.275 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:27:42.275 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.275 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.276 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:86887 7.c 7:347431d3:::benchmark_data_smithi070_78438_object86886:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-16T09:27:42.276 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:27:06.134927+0000", 2024-09-16T09:27:42.276 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 36.015759258000003, 2024-09-16T09:27:42.276 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.088112020999999999, 2024-09-16T09:27:42.276 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:27:42.276 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:27:42.276 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:27:42.276 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:27:42.276 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:27:42.276 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:27:42.277 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 86887 2024-09-16T09:27:42.277 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.277 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:27:42.277 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.277 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:27:42.277 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.134927+0000", 2024-09-16T09:27:42.277 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:27:42.277 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.277 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.277 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:27:42.277 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.134927+0000", 2024-09-16T09:27:42.277 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:27:42.277 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.278 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.278 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:27:42.278 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.134928+0000", 2024-09-16T09:27:42.278 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.522e-06 2024-09-16T09:27:42.278 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.278 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.278 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:27:42.278 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.134975+0000", 2024-09-16T09:27:42.278 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.6757e-05 2024-09-16T09:27:42.278 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.278 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.278 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:27:42.278 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.134986+0000", 2024-09-16T09:27:42.278 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0477e-05 2024-09-16T09:27:42.279 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.279 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.279 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:27:42.279 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.134993+0000", 2024-09-16T09:27:42.279 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.0149999999999998e-06 2024-09-16T09:27:42.279 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.279 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.279 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:27:42.279 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.135043+0000", 2024-09-16T09:27:42.279 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.9991999999999998e-05 2024-09-16T09:27:42.279 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.279 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.279 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:27:42.279 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.135144+0000", 2024-09-16T09:27:42.280 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000100923 2024-09-16T09:27:42.280 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.280 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.280 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 3,7", 2024-09-16T09:27:42.280 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.135239+0000", 2024-09-16T09:27:42.280 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.5573000000000001e-05 2024-09-16T09:27:42.280 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.280 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.280 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:27:42.280 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.139237+0000", 2024-09-16T09:27:42.280 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0039979309999999997 2024-09-16T09:27:42.280 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.280 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.281 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:27:42.281 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.142141+0000", 2024-09-16T09:27:42.281 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0029041409999999998 2024-09-16T09:27:42.281 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.281 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.281 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:27:42.281 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.222928+0000", 2024-09-16T09:27:42.281 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.080786812999999999 2024-09-16T09:27:42.281 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.281 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.281 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:27:42.281 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.222989+0000", 2024-09-16T09:27:42.281 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.0958999999999997e-05 2024-09-16T09:27:42.281 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.282 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.282 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:27:42.282 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.223039+0000", 2024-09-16T09:27:42.282 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.9917999999999999e-05 2024-09-16T09:27:42.282 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:27:42.282 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:27:42.282 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:27:42.282 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.282 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.282 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:86896 7.c 7:341d906e:::benchmark_data_smithi070_78438_object86895:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-16T09:27:42.282 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:27:06.139141+0000", 2024-09-16T09:27:42.282 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 36.011544915999998, 2024-09-16T09:27:42.282 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.091441622, 2024-09-16T09:27:42.282 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:27:42.283 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:27:42.283 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:27:42.283 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:27:42.283 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:27:42.283 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:27:42.283 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 86896 2024-09-16T09:27:42.283 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.283 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:27:42.283 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.283 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:27:42.283 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.139141+0000", 2024-09-16T09:27:42.283 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:27:42.283 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.284 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.284 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:27:42.285 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.139141+0000", 2024-09-16T09:27:42.285 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:27:42.285 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.285 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.285 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:27:42.285 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.139143+0000", 2024-09-16T09:27:42.285 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.248e-06 2024-09-16T09:27:42.285 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.285 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.285 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:27:42.285 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.139165+0000", 2024-09-16T09:27:42.285 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.2052000000000002e-05 2024-09-16T09:27:42.285 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.286 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.286 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:27:42.286 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.139174+0000", 2024-09-16T09:27:42.286 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.6439999999999995e-06 2024-09-16T09:27:42.286 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.286 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.286 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:27:42.286 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.139181+0000", 2024-09-16T09:27:42.286 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.7340000000000002e-06 2024-09-16T09:27:42.286 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.286 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.286 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:27:42.286 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.139267+0000", 2024-09-16T09:27:42.286 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.5870000000000003e-05 2024-09-16T09:27:42.287 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.287 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.287 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:27:42.287 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.139433+0000", 2024-09-16T09:27:42.287 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00016583400000000001 2024-09-16T09:27:42.287 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.287 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.287 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 3,7", 2024-09-16T09:27:42.287 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.139576+0000", 2024-09-16T09:27:42.287 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00014307799999999999 2024-09-16T09:27:42.287 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.287 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.287 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:27:42.287 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.142331+0000", 2024-09-16T09:27:42.288 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0027555209999999999 2024-09-16T09:27:42.288 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.288 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.288 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:27:42.288 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.144623+0000", 2024-09-16T09:27:42.288 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0022914699999999999 2024-09-16T09:27:42.288 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.288 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.288 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:27:42.288 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.230508+0000", 2024-09-16T09:27:42.288 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.085885662000000002 2024-09-16T09:27:42.288 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.288 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.288 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:27:42.289 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.230545+0000", 2024-09-16T09:27:42.289 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.6637999999999997e-05 2024-09-16T09:27:42.289 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.289 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.289 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:27:42.289 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.230583+0000", 2024-09-16T09:27:42.289 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.7870999999999997e-05 2024-09-16T09:27:42.289 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:27:42.289 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:27:42.289 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:27:42.289 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.289 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.289 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:88362 7.7 7:e7b74c32:::benchmark_data_smithi070_78438_object88361:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-16T09:27:42.289 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:27:07.479055+0000", 2024-09-16T09:27:42.290 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 34.671631093999999, 2024-09-16T09:27:42.290 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.080788735, 2024-09-16T09:27:42.290 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:27:42.290 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:27:42.290 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:27:42.290 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:27:42.290 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:27:42.290 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:27:42.290 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 88362 2024-09-16T09:27:42.290 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.290 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:27:42.290 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.290 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:27:42.290 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:07.479055+0000", 2024-09-16T09:27:42.291 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:27:42.291 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.291 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.291 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:27:42.291 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:07.479055+0000", 2024-09-16T09:27:42.291 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:27:42.291 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.291 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.291 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:27:42.291 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:07.479057+0000", 2024-09-16T09:27:42.291 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.61e-06 2024-09-16T09:27:42.291 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.291 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.291 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:27:42.292 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:07.479112+0000", 2024-09-16T09:27:42.292 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.5143999999999999e-05 2024-09-16T09:27:42.292 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.292 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.292 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:27:42.292 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:07.479125+0000", 2024-09-16T09:27:42.292 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.3354e-05 2024-09-16T09:27:42.292 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.292 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.292 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:27:42.292 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:07.479132+0000", 2024-09-16T09:27:42.292 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.8709999999999999e-06 2024-09-16T09:27:42.292 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.292 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.293 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:27:42.293 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:07.481836+0000", 2024-09-16T09:27:42.293 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0027041330000000001 2024-09-16T09:27:42.293 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.293 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.293 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:27:42.293 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:07.481969+0000", 2024-09-16T09:27:42.293 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000132509 2024-09-16T09:27:42.293 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.293 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.293 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,3", 2024-09-16T09:27:42.293 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:07.482058+0000", 2024-09-16T09:27:42.293 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.9477999999999999e-05 2024-09-16T09:27:42.294 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.294 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.294 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:27:42.294 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:07.483713+0000", 2024-09-16T09:27:42.294 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0016550549999999999 2024-09-16T09:27:42.294 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.294 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.294 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:27:42.294 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:07.489193+0000", 2024-09-16T09:27:42.294 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0054799599999999999 2024-09-16T09:27:42.294 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.294 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.294 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:27:42.294 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:07.559739+0000", 2024-09-16T09:27:42.295 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.070545947999999997 2024-09-16T09:27:42.295 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.295 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.295 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:27:42.295 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:07.559798+0000", 2024-09-16T09:27:42.295 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.8743999999999999e-05 2024-09-16T09:27:42.295 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.295 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.295 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:27:42.295 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:07.559844+0000", 2024-09-16T09:27:42.295 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.5929e-05 2024-09-16T09:27:42.295 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:27:42.295 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:27:42.295 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:27:42.295 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.296 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.296 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:88371 7.7 7:e1f80517:::benchmark_data_smithi070_78438_object88370:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-16T09:27:42.296 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:27:07.490017+0000", 2024-09-16T09:27:42.296 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 34.660669087999999, 2024-09-16T09:27:42.296 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.078336972000000005, 2024-09-16T09:27:42.296 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:27:42.296 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:27:42.296 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:27:42.296 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:27:42.296 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:27:42.296 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:27:42.296 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 88371 2024-09-16T09:27:42.296 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.297 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:27:42.297 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.297 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:27:42.297 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:07.490017+0000", 2024-09-16T09:27:42.297 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:27:42.297 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.297 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.297 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:27:42.297 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:07.490017+0000", 2024-09-16T09:27:42.297 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:27:42.297 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.297 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.297 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:27:42.297 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:07.490018+0000", 2024-09-16T09:27:42.297 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.2440000000000001e-06 2024-09-16T09:27:42.298 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.298 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.298 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:27:42.298 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:07.490057+0000", 2024-09-16T09:27:42.298 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.8667000000000001e-05 2024-09-16T09:27:42.298 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.298 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.298 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:27:42.298 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:07.490067+0000", 2024-09-16T09:27:42.298 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.6579999999999997e-06 2024-09-16T09:27:42.298 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.298 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.298 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:27:42.298 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:07.490072+0000", 2024-09-16T09:27:42.299 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.994e-06 2024-09-16T09:27:42.299 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.299 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.299 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:27:42.299 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:07.490110+0000", 2024-09-16T09:27:42.299 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.8717000000000002e-05 2024-09-16T09:27:42.299 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.299 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.299 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:27:42.299 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:07.490217+0000", 2024-09-16T09:27:42.299 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000106519 2024-09-16T09:27:42.299 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.299 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.299 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,3", 2024-09-16T09:27:42.300 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:07.490315+0000", 2024-09-16T09:27:42.300 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.8445999999999999e-05 2024-09-16T09:27:42.300 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.300 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.300 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:27:42.300 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:07.492390+0000", 2024-09-16T09:27:42.300 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.002074486 2024-09-16T09:27:42.300 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.300 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.300 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:27:42.300 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:07.530535+0000", 2024-09-16T09:27:42.300 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.038145179000000001 2024-09-16T09:27:42.300 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.300 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.301 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:27:42.301 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:07.568239+0000", 2024-09-16T09:27:42.301 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.037704201 2024-09-16T09:27:42.301 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.301 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.301 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:27:42.301 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:07.568302+0000", 2024-09-16T09:27:42.301 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.3042999999999999e-05 2024-09-16T09:27:42.301 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:27:42.301 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:27:42.301 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:27:42.301 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:07.568354+0000", 2024-09-16T09:27:42.301 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.1817999999999998e-05 2024-09-16T09:27:42.301 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:27:42.302 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:27:42.302 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:27:42.302 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:27:42.302 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:27:42.302 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-09-16T09:27:42.333 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:42.333 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:42.334 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:27:42.435 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:27:42.461 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:27:42.487 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:42.488 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:42.489 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:27:42.589 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:27:42.632 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:42.632 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:42.633 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:27:42.734 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:27:42.773 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:42.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:42.774 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:27:42.874 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:27:42.904 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:42.905 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:42.905 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:27:42.909 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:27:42 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:27:43.006 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:27:43.028 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:43.028 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:43.029 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:27:43.039 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Total time run: 90.0086 2024-09-16T09:27:43.039 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Total writes made: 126128 2024-09-16T09:27:43.039 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Write size: 65536 2024-09-16T09:27:43.039 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Object size: 65536 2024-09-16T09:27:43.039 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Bandwidth (MB/sec): 87.5805 2024-09-16T09:27:43.039 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Stddev Bandwidth: 32.3024 2024-09-16T09:27:43.039 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Max bandwidth (MB/sec): 202.625 2024-09-16T09:27:43.040 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Min bandwidth (MB/sec): 55.125 2024-09-16T09:27:43.040 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Average IOPS: 1401 2024-09-16T09:27:43.040 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Stddev IOPS: 516.838 2024-09-16T09:27:43.040 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Max IOPS: 3242 2024-09-16T09:27:43.040 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Min IOPS: 882 2024-09-16T09:27:43.040 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Average Latency(s): 0.0114143 2024-09-16T09:27:43.040 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Stddev Latency(s): 0.0082261 2024-09-16T09:27:43.040 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Max latency(s): 0.156312 2024-09-16T09:27:43.040 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Min latency(s): 0.0012412 2024-09-16T09:27:43.043 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Cleaning up (deleting benchmark objects) 2024-09-16T09:27:43.122 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.7 dump_ops_in_flight 2024-09-16T09:27:43.130 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:27:43.194 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:43.194 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:43.195 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:27:43.225 INFO:teuthology.orchestra.run.smithi070.stderr:noscrub is unset 2024-09-16T09:27:43.270 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T09:27:43.270 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T09:27:43.270 INFO:tasks.thrashosds.thrasher:Shrinking pool unique_pool_0 2024-09-16T09:27:43.270 INFO:tasks.cephadm.ceph_manager.ceph:contract_pool unique_pool_0 by 10 min 8 2024-09-16T09:27:43.271 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-16T09:27:43.295 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:27:43.299 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:27:43.300 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:27:43.338 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:43.339 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:43.339 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:27:43.345 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:27:43.346 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:27:43.396 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:27:43.396 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:27:43.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:43 smithi084 ceph-mon[32629]: pgmap v113: 158 pgs: 158 active+clean; 7.4 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 692 B/s rd, 47 MiB/s wr, 759 op/s 2024-09-16T09:27:43.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:43 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1337032429' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:27:43.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:43 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:27:43.430 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:27:43.430 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:27:43.440 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:27:43.463 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:43 smithi070 ceph-mon[30388]: pgmap v113: 158 pgs: 158 active+clean; 7.4 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 692 B/s rd, 47 MiB/s wr, 759 op/s 2024-09-16T09:27:43.463 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:43 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1337032429' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:27:43.463 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:43 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:27:43.463 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:27:43.464 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:27:43.464 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:43 smithi070 ceph-mon[40691]: pgmap v113: 158 pgs: 158 active+clean; 7.4 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 692 B/s rd, 47 MiB/s wr, 759 op/s 2024-09-16T09:27:43.464 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:43 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1337032429' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:27:43.464 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:43 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:27:43.466 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:43.467 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:43.467 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:27:43.489 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:27:43.496 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:27:43.496 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:27:43.527 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:27:43.527 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:27:43.565 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:27:43.565 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:27:43.567 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:27:43.612 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:43.613 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:43.613 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:27:43.714 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:27:43.741 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:43.742 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:43.742 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:27:43.843 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:27:43.868 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:43.869 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:43.869 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:27:43.970 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:27:44.010 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:44.011 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:44.012 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:27:44.061 INFO:teuthology.orchestra.run.smithi084.stdout:ERROR: (22) Invalid argument 2024-09-16T09:27:44.061 INFO:teuthology.orchestra.run.smithi084.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:27:44.112 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:27:44.141 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:44.142 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:44.142 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:27:44.243 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:27:44.269 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:44.269 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:44.269 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:27:44.370 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:27:44.400 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:44.401 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:44.401 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:27:44.493 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:44 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:27:44.493 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:44 smithi070 ceph-mon[30388]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:27:44.493 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:44 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-16T09:27:44.493 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:44 smithi070 ceph-mon[30388]: osdmap e119: 8 total, 8 up, 8 in 2024-09-16T09:27:44.493 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:44 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/279278143' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:27:44.494 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:44 smithi070 ceph-mon[30388]: 4.1e scrub starts 2024-09-16T09:27:44.494 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:44 smithi070 ceph-mon[30388]: 4.1e scrub ok 2024-09-16T09:27:44.494 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:44 smithi070 ceph-mon[30388]: osdmap e120: 8 total, 8 up, 8 in 2024-09-16T09:27:44.494 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:44 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:27:44.494 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:44 smithi070 ceph-mon[40691]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:27:44.494 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:44 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-16T09:27:44.494 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:44 smithi070 ceph-mon[40691]: osdmap e119: 8 total, 8 up, 8 in 2024-09-16T09:27:44.494 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:44 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/279278143' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:27:44.494 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:44 smithi070 ceph-mon[40691]: 4.1e scrub starts 2024-09-16T09:27:44.494 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:44 smithi070 ceph-mon[40691]: 4.1e scrub ok 2024-09-16T09:27:44.494 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:44 smithi070 ceph-mon[40691]: osdmap e120: 8 total, 8 up, 8 in 2024-09-16T09:27:44.502 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:27:44.531 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:44.531 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:44.531 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:27:44.556 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:44 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:27:44.556 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:44 smithi084 ceph-mon[32629]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:27:44.556 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:44 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-16T09:27:44.556 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:44 smithi084 ceph-mon[32629]: osdmap e119: 8 total, 8 up, 8 in 2024-09-16T09:27:44.556 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:44 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/279278143' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:27:44.556 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:44 smithi084 ceph-mon[32629]: 4.1e scrub starts 2024-09-16T09:27:44.556 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:44 smithi084 ceph-mon[32629]: 4.1e scrub ok 2024-09-16T09:27:44.556 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:44 smithi084 ceph-mon[32629]: osdmap e120: 8 total, 8 up, 8 in 2024-09-16T09:27:44.632 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:27:44.657 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:44.658 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:44.658 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:27:44.759 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:27:44.781 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:44.781 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:44.782 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:27:44.883 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:27:44.918 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:44.918 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:44.918 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:27:44.923 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.7 dump_blocked_ops 2024-09-16T09:27:45.020 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:27:45.047 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:45.048 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:45.048 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:27:45.149 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:27:45.169 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:45.170 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:45.170 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:27:45.246 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:27:45.271 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:27:45.303 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:45.303 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:45.304 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:27:45.405 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:27:45.422 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:45 smithi084 ceph-mon[32629]: pgmap v115: 158 pgs: 158 active+clean; 7.7 GiB data, 24 GiB used, 692 GiB / 715 GiB avail; 1.2 KiB/s rd, 84 MiB/s wr, 1.34k op/s 2024-09-16T09:27:45.422 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:45 smithi084 ceph-mon[32629]: 4.7 scrub starts 2024-09-16T09:27:45.422 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:45 smithi084 ceph-mon[32629]: 4.7 scrub ok 2024-09-16T09:27:45.422 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:45 smithi084 ceph-mon[32629]: 2.2 scrub starts 2024-09-16T09:27:45.422 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:45 smithi084 ceph-mon[32629]: 2.2 scrub ok 2024-09-16T09:27:45.422 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:45 smithi084 ceph-mon[32629]: 5.13 scrub starts 2024-09-16T09:27:45.422 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:45 smithi084 ceph-mon[32629]: 5.13 scrub ok 2024-09-16T09:27:45.438 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:45.439 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:45.439 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:27:45.540 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:27:45.573 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:45.574 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:45.574 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:27:45.657 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:45 smithi070 ceph-mon[30388]: pgmap v115: 158 pgs: 158 active+clean; 7.7 GiB data, 24 GiB used, 692 GiB / 715 GiB avail; 1.2 KiB/s rd, 84 MiB/s wr, 1.34k op/s 2024-09-16T09:27:45.657 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:45 smithi070 ceph-mon[30388]: 4.7 scrub starts 2024-09-16T09:27:45.657 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:45 smithi070 ceph-mon[30388]: 4.7 scrub ok 2024-09-16T09:27:45.657 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:45 smithi070 ceph-mon[30388]: 2.2 scrub starts 2024-09-16T09:27:45.657 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:45 smithi070 ceph-mon[30388]: 2.2 scrub ok 2024-09-16T09:27:45.657 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:45 smithi070 ceph-mon[30388]: 5.13 scrub starts 2024-09-16T09:27:45.658 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:45 smithi070 ceph-mon[30388]: 5.13 scrub ok 2024-09-16T09:27:45.658 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:45 smithi070 ceph-mon[40691]: pgmap v115: 158 pgs: 158 active+clean; 7.7 GiB data, 24 GiB used, 692 GiB / 715 GiB avail; 1.2 KiB/s rd, 84 MiB/s wr, 1.34k op/s 2024-09-16T09:27:45.658 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:45 smithi070 ceph-mon[40691]: 4.7 scrub starts 2024-09-16T09:27:45.658 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:45 smithi070 ceph-mon[40691]: 4.7 scrub ok 2024-09-16T09:27:45.658 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:45 smithi070 ceph-mon[40691]: 2.2 scrub starts 2024-09-16T09:27:45.658 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:45 smithi070 ceph-mon[40691]: 2.2 scrub ok 2024-09-16T09:27:45.658 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:45 smithi070 ceph-mon[40691]: 5.13 scrub starts 2024-09-16T09:27:45.658 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:45 smithi070 ceph-mon[40691]: 5.13 scrub ok 2024-09-16T09:27:45.675 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:27:45.715 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:45.716 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:45.716 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:27:45.817 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:27:45.831 INFO:teuthology.orchestra.run.smithi084.stdout:ERROR: (22) Invalid argument 2024-09-16T09:27:45.831 INFO:teuthology.orchestra.run.smithi084.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:27:45.859 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:45.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:45.860 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:27:45.961 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:27:45.997 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:45.998 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:45.998 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:27:46.099 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:27:46.132 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:46.133 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:46.133 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:27:46.234 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:27:46.275 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:46.276 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:46.276 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:27:46.377 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:27:46.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:46 smithi084 ceph-mon[32629]: 6.6 scrub starts 2024-09-16T09:27:46.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:46 smithi084 ceph-mon[32629]: 6.6 scrub ok 2024-09-16T09:27:46.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:46 smithi084 ceph-mon[32629]: 4.18 scrub starts 2024-09-16T09:27:46.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:46 smithi084 ceph-mon[32629]: 4.18 scrub ok 2024-09-16T09:27:46.417 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:46.418 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:46 smithi070 ceph-mon[30388]: 6.6 scrub starts 2024-09-16T09:27:46.418 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:46 smithi070 ceph-mon[30388]: 6.6 scrub ok 2024-09-16T09:27:46.418 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:46 smithi070 ceph-mon[30388]: 4.18 scrub starts 2024-09-16T09:27:46.418 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:46 smithi070 ceph-mon[30388]: 4.18 scrub ok 2024-09-16T09:27:46.419 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:46 smithi070 ceph-mon[40691]: 6.6 scrub starts 2024-09-16T09:27:46.419 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:46 smithi070 ceph-mon[40691]: 6.6 scrub ok 2024-09-16T09:27:46.419 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:46 smithi070 ceph-mon[40691]: 4.18 scrub starts 2024-09-16T09:27:46.419 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:46 smithi070 ceph-mon[40691]: 4.18 scrub ok 2024-09-16T09:27:46.419 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:46.419 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:27:46.520 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:27:46.563 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:46.564 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:46.564 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:27:46.575 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:27:46.580 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.7 dump_historic_ops 2024-09-16T09:27:46.665 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:27:46.709 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:46.710 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:46.710 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:27:46.811 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:27:46.846 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:46.847 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:46.847 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:27:46.948 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:27:46.968 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:27:46.992 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:46.993 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:46.994 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:27:47.095 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:27:47.124 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:47.125 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:47.125 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:27:47.226 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:27:47.275 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:47.275 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:47.276 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:27:47.352 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:47 smithi070 ceph-mon[30388]: pgmap v117: 158 pgs: 158 active+clean; 7.7 GiB data, 24 GiB used, 692 GiB / 715 GiB avail; 801 B/s rd, 50 MiB/s wr, 799 op/s 2024-09-16T09:27:47.352 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:47 smithi070 ceph-mon[30388]: 5.12 scrub starts 2024-09-16T09:27:47.353 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:47 smithi070 ceph-mon[30388]: 5.12 scrub ok 2024-09-16T09:27:47.353 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:47 smithi070 ceph-mon[30388]: 3.16 scrub starts 2024-09-16T09:27:47.353 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:47 smithi070 ceph-mon[30388]: 3.16 scrub ok 2024-09-16T09:27:47.353 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:47 smithi070 ceph-mon[30388]: 3.7 scrub starts 2024-09-16T09:27:47.353 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:47 smithi070 ceph-mon[30388]: 3.7 scrub ok 2024-09-16T09:27:47.353 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:47 smithi070 ceph-mon[30388]: 6.e scrub starts 2024-09-16T09:27:47.353 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:47 smithi070 ceph-mon[30388]: 6.e scrub ok 2024-09-16T09:27:47.353 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:47 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:27:47.353 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:47 smithi070 ceph-mon[40691]: pgmap v117: 158 pgs: 158 active+clean; 7.7 GiB data, 24 GiB used, 692 GiB / 715 GiB avail; 801 B/s rd, 50 MiB/s wr, 799 op/s 2024-09-16T09:27:47.353 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:47 smithi070 ceph-mon[40691]: 5.12 scrub starts 2024-09-16T09:27:47.354 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:47 smithi070 ceph-mon[40691]: 5.12 scrub ok 2024-09-16T09:27:47.354 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:47 smithi070 ceph-mon[40691]: 3.16 scrub starts 2024-09-16T09:27:47.354 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:47 smithi070 ceph-mon[40691]: 3.16 scrub ok 2024-09-16T09:27:47.354 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:47 smithi070 ceph-mon[40691]: 3.7 scrub starts 2024-09-16T09:27:47.354 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:47 smithi070 ceph-mon[40691]: 3.7 scrub ok 2024-09-16T09:27:47.354 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:47 smithi070 ceph-mon[40691]: 6.e scrub starts 2024-09-16T09:27:47.354 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:47 smithi070 ceph-mon[40691]: 6.e scrub ok 2024-09-16T09:27:47.354 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:47 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:27:47.377 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:27:47.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:47 smithi084 ceph-mon[32629]: pgmap v117: 158 pgs: 158 active+clean; 7.7 GiB data, 24 GiB used, 692 GiB / 715 GiB avail; 801 B/s rd, 50 MiB/s wr, 799 op/s 2024-09-16T09:27:47.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:47 smithi084 ceph-mon[32629]: 5.12 scrub starts 2024-09-16T09:27:47.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:47 smithi084 ceph-mon[32629]: 5.12 scrub ok 2024-09-16T09:27:47.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:47 smithi084 ceph-mon[32629]: 3.16 scrub starts 2024-09-16T09:27:47.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:47 smithi084 ceph-mon[32629]: 3.16 scrub ok 2024-09-16T09:27:47.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:47 smithi084 ceph-mon[32629]: 3.7 scrub starts 2024-09-16T09:27:47.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:47 smithi084 ceph-mon[32629]: 3.7 scrub ok 2024-09-16T09:27:47.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:47 smithi084 ceph-mon[32629]: 6.e scrub starts 2024-09-16T09:27:47.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:47 smithi084 ceph-mon[32629]: 6.e scrub ok 2024-09-16T09:27:47.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:47 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:27:47.414 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:47.414 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:47.414 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:27:47.515 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:27:47.551 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:47.573 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:47.574 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:27:47.574 INFO:teuthology.orchestra.run.smithi084.stdout:ERROR: (22) Invalid argument 2024-09-16T09:27:47.574 INFO:teuthology.orchestra.run.smithi084.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:27:47.675 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:27:47.707 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-16T09:27:47.719 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:47.719 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:47.720 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:27:47.735 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:27:47.737 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2024-09-16T09:27:47.821 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:27:47.873 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:47.874 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:47.874 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:27:47.974 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:27:48.014 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:48.015 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:48.015 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:27:48.116 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:27:48.142 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:48.143 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:48.144 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:27:48.244 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:27:48.272 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:48.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:48.273 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:27:48.345 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.0 dump_ops_in_flight 2024-09-16T09:27:48.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:48 smithi070 ceph-mon[30388]: 6.5 scrub starts 2024-09-16T09:27:48.354 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:48 smithi070 ceph-mon[30388]: 6.5 scrub ok 2024-09-16T09:27:48.355 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:48 smithi070 ceph-mon[40691]: 6.5 scrub starts 2024-09-16T09:27:48.355 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:48 smithi070 ceph-mon[40691]: 6.5 scrub ok 2024-09-16T09:27:48.374 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:27:48.398 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:48.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:48.399 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:27:48.466 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T09:27:48.500 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:27:48.524 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:48.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:48.525 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:27:48.626 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:27:48.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:48 smithi084 ceph-mon[32629]: 6.5 scrub starts 2024-09-16T09:27:48.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:48 smithi084 ceph-mon[32629]: 6.5 scrub ok 2024-09-16T09:27:48.670 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:48.671 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:48.671 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:27:48.772 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:27:48.801 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:48.802 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:48.802 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:27:48.903 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:27:48.931 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:48.931 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:48.932 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:27:49.033 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:27:49.070 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:49.071 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:49.071 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:27:49.172 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:27:49.208 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:49.208 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:49.209 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:27:49.310 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:27:49.344 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:49.345 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:49.345 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:27:49.430 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":118,"stamp":"2024-09-16T09:27:47.525804+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":8118737459,"num_objects":124073,"num_object_clones":0,"num_object_copies":372219,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":124073,"num_whiteouts":0,"num_read":883,"num_read_kb":635,"num_write":128242,"num_write_kb":8053742,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":85248,"ondisk_log_size":85248,"up":474,"acting":474,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":474,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":24408608,"kb_used_data":23789856,"kb_used_omap":12,"kb_used_meta":618611,"kb_avail":725552608,"statfs":{"total":767960285184,"available":742965870592,"internally_reserved":0,"allocated":24360812544,"data_stored":24355451097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":633458226},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,4,5,3,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":218497048,"num_objects":3335,"num_object_clones":0,"num_object_copies":10005,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3335,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":7267,"num_write_kb":339201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002073"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":139,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T09:27:39.168580+0000","last_change":"2024-09-16T09:27:07.621760+0000","last_active":"2024-09-16T09:27:39.168580+0000","last_peered":"2024-09-16T09:27:39.168580+0000","last_clean":"2024-09-16T09:27:39.168580+0000","last_became_active":"2024-09-16T09:25:01.709108+0000","last_became_peered":"2024-09-16T09:25:01.709108+0000","last_unstale":"2024-09-16T09:27:39.168580+0000","last_undegraded":"2024-09-16T09:27:39.168580+0000","last_fullsized":"2024-09-16T09:27:39.168580+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:07.621692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:27:07.621692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:08.877709+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"66'9","reported_seq":155,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-16T09:27:45.630451+0000","last_change":"2024-09-16T09:27:45.630451+0000","last_active":"2024-09-16T09:27:45.630451+0000","last_peered":"2024-09-16T09:27:45.630451+0000","last_clean":"2024-09-16T09:27:45.630451+0000","last_became_active":"2024-09-16T09:25:03.714635+0000","last_became_peered":"2024-09-16T09:25:03.714635+0000","last_unstale":"2024-09-16T09:27:45.630451+0000","last_undegraded":"2024-09-16T09:27:45.630451+0000","last_fullsized":"2024-09-16T09:27:45.630451+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:27:45.630310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:45.630310+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:46.857234+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":131,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T09:27:39.133601+0000","last_change":"2024-09-16T09:26:41.241860+0000","last_active":"2024-09-16T09:27:39.133601+0000","last_peered":"2024-09-16T09:27:39.133601+0000","last_clean":"2024-09-16T09:27:39.133601+0000","last_became_active":"2024-09-16T09:25:05.715645+0000","last_became_peered":"2024-09-16T09:25:05.715645+0000","last_unstale":"2024-09-16T09:27:39.133601+0000","last_undegraded":"2024-09-16T09:27:39.133601+0000","last_fullsized":"2024-09-16T09:27:39.133601+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:41.241640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:26:41.241640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:47.534150+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":131,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-16T09:27:44.136461+0000","last_change":"2024-09-16T09:27:11.565306+0000","last_active":"2024-09-16T09:27:44.136461+0000","last_peered":"2024-09-16T09:27:44.136461+0000","last_clean":"2024-09-16T09:27:44.136461+0000","last_became_active":"2024-09-16T09:25:07.715442+0000","last_became_peered":"2024-09-16T09:25:07.715442+0000","last_unstale":"2024-09-16T09:27:44.136461+0000","last_undegraded":"2024-09-16T09:27:44.136461+0000","last_fullsized":"2024-09-16T09:27:44.136461+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:11.565220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:11.565220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:16.605980+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":119,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T09:27:39.151190+0000","last_change":"2024-09-16T09:25:07.718687+0000","last_active":"2024-09-16T09:27:39.151190+0000","last_peered":"2024-09-16T09:27:39.151190+0000","last_clean":"2024-09-16T09:27:39.151190+0000","last_became_active":"2024-09-16T09:25:07.718126+0000","last_became_peered":"2024-09-16T09:25:07.718126+0000","last_unstale":"2024-09-16T09:27:39.151190+0000","last_undegraded":"2024-09-16T09:27:39.151190+0000","last_fullsized":"2024-09-16T09:27:39.151190+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:54.549850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":131,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T09:27:39.151274+0000","last_change":"2024-09-16T09:25:01.708505+0000","last_active":"2024-09-16T09:27:39.151274+0000","last_peered":"2024-09-16T09:27:39.151274+0000","last_clean":"2024-09-16T09:27:39.151274+0000","last_became_active":"2024-09-16T09:25:01.708179+0000","last_became_peered":"2024-09-16T09:25:01.708179+0000","last_unstale":"2024-09-16T09:27:39.151274+0000","last_undegraded":"2024-09-16T09:27:39.151274+0000","last_fullsized":"2024-09-16T09:27:39.151274+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:50.452195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"66'15","reported_seq":160,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T09:27:39.152776+0000","last_change":"2024-09-16T09:27:09.887336+0000","last_active":"2024-09-16T09:27:39.152776+0000","last_peered":"2024-09-16T09:27:39.152776+0000","last_clean":"2024-09-16T09:27:39.152776+0000","last_became_active":"2024-09-16T09:25:03.709499+0000","last_became_peered":"2024-09-16T09:25:03.709499+0000","last_unstale":"2024-09-16T09:27:39.152776+0000","last_undegraded":"2024-09-16T09:27:39.152776+0000","last_fullsized":"2024-09-16T09:27:39.152776+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-16T09:27:09.887267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:09.887267+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:18.495993+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":135,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-16T09:27:44.134372+0000","last_change":"2024-09-16T09:26:21.827093+0000","last_active":"2024-09-16T09:27:44.134372+0000","last_peered":"2024-09-16T09:27:44.134372+0000","last_clean":"2024-09-16T09:27:44.134372+0000","last_became_active":"2024-09-16T09:25:05.717236+0000","last_became_peered":"2024-09-16T09:25:05.717236+0000","last_unstale":"2024-09-16T09:27:44.134372+0000","last_undegraded":"2024-09-16T09:27:44.134372+0000","last_fullsized":"2024-09-16T09:27:44.134372+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:21.827018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:26:21.827018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":142,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-16T09:27:44.138657+0000","last_change":"2024-09-16T09:26:23.041770+0000","last_active":"2024-09-16T09:27:44.138657+0000","last_peered":"2024-09-16T09:27:44.138657+0000","last_clean":"2024-09-16T09:27:44.138657+0000","last_became_active":"2024-09-16T09:25:01.705269+0000","last_became_peered":"2024-09-16T09:25:01.705269+0000","last_unstale":"2024-09-16T09:27:44.138657+0000","last_undegraded":"2024-09-16T09:27:44.138657+0000","last_fullsized":"2024-09-16T09:27:44.138657+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:23.041687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:23.041687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":28,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"66'9","reported_seq":147,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-16T09:27:44.136824+0000","last_change":"2024-09-16T09:25:03.709441+0000","last_active":"2024-09-16T09:27:44.136824+0000","last_peered":"2024-09-16T09:27:44.136824+0000","last_clean":"2024-09-16T09:27:44.136824+0000","last_became_active":"2024-09-16T09:25:03.709147+0000","last_became_peered":"2024-09-16T09:25:03.709147+0000","last_unstale":"2024-09-16T09:27:44.136824+0000","last_undegraded":"2024-09-16T09:27:44.136824+0000","last_fullsized":"2024-09-16T09:27:44.136824+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:49.730063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":126,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-16T09:27:44.138694+0000","last_change":"2024-09-16T09:25:05.718510+0000","last_active":"2024-09-16T09:27:44.138694+0000","last_peered":"2024-09-16T09:27:44.138694+0000","last_clean":"2024-09-16T09:27:44.138694+0000","last_became_active":"2024-09-16T09:25:05.718134+0000","last_became_peered":"2024-09-16T09:25:05.718134+0000","last_unstale":"2024-09-16T09:27:44.138694+0000","last_undegraded":"2024-09-16T09:27:44.138694+0000","last_fullsized":"2024-09-16T09:27:44.138694+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:55.212380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"66'1","reported_seq":128,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T09:27:39.154629+0000","last_change":"2024-09-16T09:26:24.547980+0000","last_active":"2024-09-16T09:27:39.154629+0000","last_peered":"2024-09-16T09:27:39.154629+0000","last_clean":"2024-09-16T09:27:39.154629+0000","last_became_active":"2024-09-16T09:25:07.716558+0000","last_became_peered":"2024-09-16T09:25:07.716558+0000","last_unstale":"2024-09-16T09:27:39.154629+0000","last_undegraded":"2024-09-16T09:27:39.154629+0000","last_fullsized":"2024-09-16T09:27:39.154629+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-16T09:26:24.547877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:24.547877+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:45.771067+0000","scrub_duration":52,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":134,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-16T09:27:44.139004+0000","last_change":"2024-09-16T09:25:01.697306+0000","last_active":"2024-09-16T09:27:44.139004+0000","last_peered":"2024-09-16T09:27:44.139004+0000","last_clean":"2024-09-16T09:27:44.139004+0000","last_became_active":"2024-09-16T09:25:01.696939+0000","last_became_peered":"2024-09-16T09:25:01.696939+0000","last_unstale":"2024-09-16T09:27:44.139004+0000","last_undegraded":"2024-09-16T09:27:44.139004+0000","last_fullsized":"2024-09-16T09:27:44.139004+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:49.230318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"66'5","reported_seq":149,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-16T09:27:44.136319+0000","last_change":"2024-09-16T09:27:05.694733+0000","last_active":"2024-09-16T09:27:44.136319+0000","last_peered":"2024-09-16T09:27:44.136319+0000","last_clean":"2024-09-16T09:27:44.136319+0000","last_became_active":"2024-09-16T09:25:03.716136+0000","last_became_peered":"2024-09-16T09:25:03.716136+0000","last_unstale":"2024-09-16T09:27:44.136319+0000","last_undegraded":"2024-09-16T09:27:44.136319+0000","last_fullsized":"2024-09-16T09:27:44.136319+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-16T09:27:05.694650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:05.694650+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:24.201532+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":126,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-16T09:27:44.148088+0000","last_change":"2024-09-16T09:25:05.718538+0000","last_active":"2024-09-16T09:27:44.148088+0000","last_peered":"2024-09-16T09:27:44.148088+0000","last_clean":"2024-09-16T09:27:44.148088+0000","last_became_active":"2024-09-16T09:25:05.718281+0000","last_became_peered":"2024-09-16T09:25:05.718281+0000","last_unstale":"2024-09-16T09:27:44.148088+0000","last_undegraded":"2024-09-16T09:27:44.148088+0000","last_fullsized":"2024-09-16T09:27:44.148088+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:46.610694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":130,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-16T09:27:44.138273+0000","last_change":"2024-09-16T09:27:01.285768+0000","last_active":"2024-09-16T09:27:44.138273+0000","last_peered":"2024-09-16T09:27:44.138273+0000","last_clean":"2024-09-16T09:27:44.138273+0000","last_became_active":"2024-09-16T09:25:07.716724+0000","last_became_peered":"2024-09-16T09:25:07.716724+0000","last_unstale":"2024-09-16T09:27:44.138273+0000","last_undegraded":"2024-09-16T09:27:44.138273+0000","last_fullsized":"2024-09-16T09:27:44.138273+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:01.285712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:01.285712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:23.640748+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":131,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-16T09:27:44.135541+0000","last_change":"2024-09-16T09:26:27.758217+0000","last_active":"2024-09-16T09:27:44.135541+0000","last_peered":"2024-09-16T09:27:44.135541+0000","last_clean":"2024-09-16T09:27:44.135541+0000","last_became_active":"2024-09-16T09:25:07.720793+0000","last_became_peered":"2024-09-16T09:25:07.720793+0000","last_unstale":"2024-09-16T09:27:44.135541+0000","last_undegraded":"2024-09-16T09:27:44.135541+0000","last_fullsized":"2024-09-16T09:27:44.135541+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:27.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:27.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":139,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T09:27:39.155688+0000","last_change":"2024-09-16T09:27:09.635443+0000","last_active":"2024-09-16T09:27:39.155688+0000","last_peered":"2024-09-16T09:27:39.155688+0000","last_clean":"2024-09-16T09:27:39.155688+0000","last_became_active":"2024-09-16T09:25:01.706172+0000","last_became_peered":"2024-09-16T09:25:01.706172+0000","last_unstale":"2024-09-16T09:27:39.155688+0000","last_undegraded":"2024-09-16T09:27:39.155688+0000","last_fullsized":"2024-09-16T09:27:39.155688+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:09.635363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:27:09.635363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:25.991704+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"66'15","reported_seq":160,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T09:27:39.153584+0000","last_change":"2024-09-16T09:26:33.873905+0000","last_active":"2024-09-16T09:27:39.153584+0000","last_peered":"2024-09-16T09:27:39.153584+0000","last_clean":"2024-09-16T09:27:39.153584+0000","last_became_active":"2024-09-16T09:25:03.713780+0000","last_became_peered":"2024-09-16T09:25:03.713780+0000","last_unstale":"2024-09-16T09:27:39.153584+0000","last_undegraded":"2024-09-16T09:27:39.153584+0000","last_fullsized":"2024-09-16T09:27:39.153584+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-16T09:26:33.873817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:26:33.873817+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:54.702210+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":131,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T09:27:39.134675+0000","last_change":"2024-09-16T09:27:16.266378+0000","last_active":"2024-09-16T09:27:39.134675+0000","last_peered":"2024-09-16T09:27:39.134675+0000","last_clean":"2024-09-16T09:27:39.134675+0000","last_became_active":"2024-09-16T09:25:05.713929+0000","last_became_peered":"2024-09-16T09:25:05.713929+0000","last_unstale":"2024-09-16T09:27:39.134675+0000","last_undegraded":"2024-09-16T09:27:39.134675+0000","last_fullsized":"2024-09-16T09:27:39.134675+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:16.266297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:16.266297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:38.874875+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":127,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T09:27:39.150500+0000","last_change":"2024-09-16T09:26:39.358604+0000","last_active":"2024-09-16T09:27:39.150500+0000","last_peered":"2024-09-16T09:27:39.150500+0000","last_clean":"2024-09-16T09:27:39.150500+0000","last_became_active":"2024-09-16T09:25:07.719067+0000","last_became_peered":"2024-09-16T09:25:07.719067+0000","last_unstale":"2024-09-16T09:27:39.150500+0000","last_undegraded":"2024-09-16T09:27:39.150500+0000","last_fullsized":"2024-09-16T09:27:39.150500+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:39.358091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:39.358091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:00.182589+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":143,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-16T09:27:44.136946+0000","last_change":"2024-09-16T09:26:32.756725+0000","last_active":"2024-09-16T09:27:44.136946+0000","last_peered":"2024-09-16T09:27:44.136946+0000","last_clean":"2024-09-16T09:27:44.136946+0000","last_became_active":"2024-09-16T09:25:01.706491+0000","last_became_peered":"2024-09-16T09:25:01.706491+0000","last_unstale":"2024-09-16T09:27:44.136946+0000","last_undegraded":"2024-09-16T09:27:44.136946+0000","last_fullsized":"2024-09-16T09:27:44.136946+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:32.756657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:32.756657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:55.818360+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"66'12","reported_seq":158,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T09:27:39.149937+0000","last_change":"2024-09-16T09:27:01.700219+0000","last_active":"2024-09-16T09:27:39.149937+0000","last_peered":"2024-09-16T09:27:39.149937+0000","last_clean":"2024-09-16T09:27:39.149937+0000","last_became_active":"2024-09-16T09:25:03.709610+0000","last_became_peered":"2024-09-16T09:25:03.709610+0000","last_unstale":"2024-09-16T09:27:39.149937+0000","last_undegraded":"2024-09-16T09:27:39.149937+0000","last_fullsized":"2024-09-16T09:27:39.149937+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-16T09:27:01.700139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:01.700139+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:28.256362+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":135,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-16T09:27:44.137687+0000","last_change":"2024-09-16T09:27:00.816500+0000","last_active":"2024-09-16T09:27:44.137687+0000","last_peered":"2024-09-16T09:27:44.137687+0000","last_clean":"2024-09-16T09:27:44.137687+0000","last_became_active":"2024-09-16T09:25:05.710093+0000","last_became_peered":"2024-09-16T09:25:05.710093+0000","last_unstale":"2024-09-16T09:27:44.137687+0000","last_undegraded":"2024-09-16T09:27:44.137687+0000","last_fullsized":"2024-09-16T09:27:44.137687+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:00.816424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:00.816424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:16.736124+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":123,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-16T09:27:44.148200+0000","last_change":"2024-09-16T09:25:07.719655+0000","last_active":"2024-09-16T09:27:44.148200+0000","last_peered":"2024-09-16T09:27:44.148200+0000","last_clean":"2024-09-16T09:27:44.148200+0000","last_became_active":"2024-09-16T09:25:07.719380+0000","last_became_peered":"2024-09-16T09:25:07.719380+0000","last_unstale":"2024-09-16T09:27:44.148200+0000","last_undegraded":"2024-09-16T09:27:44.148200+0000","last_fullsized":"2024-09-16T09:27:44.148200+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:51.590765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":140,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T09:27:39.133742+0000","last_change":"2024-09-16T09:27:08.225591+0000","last_active":"2024-09-16T09:27:39.133742+0000","last_peered":"2024-09-16T09:27:39.133742+0000","last_clean":"2024-09-16T09:27:39.133742+0000","last_became_active":"2024-09-16T09:25:01.699545+0000","last_became_peered":"2024-09-16T09:25:01.699545+0000","last_unstale":"2024-09-16T09:27:39.133742+0000","last_undegraded":"2024-09-16T09:27:39.133742+0000","last_fullsized":"2024-09-16T09:27:39.133742+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:27:08.225363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:27:08.225363+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:16.086554+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"66'10","reported_seq":150,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T09:27:39.158579+0000","last_change":"2024-09-16T09:26:30.598794+0000","last_active":"2024-09-16T09:27:39.158579+0000","last_peered":"2024-09-16T09:27:39.158579+0000","last_clean":"2024-09-16T09:27:39.158579+0000","last_became_active":"2024-09-16T09:25:03.720245+0000","last_became_peered":"2024-09-16T09:25:03.720245+0000","last_unstale":"2024-09-16T09:27:39.158579+0000","last_undegraded":"2024-09-16T09:27:39.158579+0000","last_fullsized":"2024-09-16T09:27:39.158579+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-16T09:26:30.598563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:26:30.598563+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:41.721204+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"66'8","reported_seq":180,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-16T09:27:44.143470+0000","last_change":"2024-09-16T09:26:28.003706+0000","last_active":"2024-09-16T09:27:44.143470+0000","last_peered":"2024-09-16T09:27:44.143470+0000","last_clean":"2024-09-16T09:27:44.143470+0000","last_became_active":"2024-09-16T09:25:05.712516+0000","last_became_peered":"2024-09-16T09:25:05.712516+0000","last_unstale":"2024-09-16T09:27:44.143470+0000","last_undegraded":"2024-09-16T09:27:44.143470+0000","last_fullsized":"2024-09-16T09:27:44.143470+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:26:28.003660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:26:28.003660+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.a","version":"73'79","reported_seq":70,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-16T09:27:44.138532+0000","last_change":"2024-09-16T09:26:36.169945+0000","last_active":"2024-09-16T09:27:44.138532+0000","last_peered":"2024-09-16T09:27:44.138532+0000","last_clean":"2024-09-16T09:27:44.138532+0000","last_became_active":"2024-09-16T09:26:35.127352+0000","last_became_peered":"2024-09-16T09:26:35.127352+0000","last_unstale":"2024-09-16T09:27:44.138532+0000","last_undegraded":"2024-09-16T09:27:44.138532+0000","last_fullsized":"2024-09-16T09:27:44.138532+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:26:36.169870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:26:36.169870+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:00.141321+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.c","version":"120'8155","reported_seq":8242,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-16T09:27:47.487585+0000","last_change":"2024-09-16T09:26:08.117066+0000","last_active":"2024-09-16T09:27:47.487585+0000","last_peered":"2024-09-16T09:27:47.487585+0000","last_clean":"2024-09-16T09:27:47.487585+0000","last_became_active":"2024-09-16T09:26:08.116783+0000","last_became_peered":"2024-09-16T09:26:08.116783+0000","last_unstale":"2024-09-16T09:27:47.487585+0000","last_undegraded":"2024-09-16T09:27:47.487585+0000","last_fullsized":"2024-09-16T09:27:47.487585+0000","mapping_epoch":82,"log_start":"91'2400","ondisk_log_start":"91'2400","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_clean_scrub_stamp":"2024-09-16T09:26:07.096764+0000","objects_scrubbed":0,"log_size":5755,"log_dups_size":0,"ondisk_log_size":5755,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:32.278987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":492109824,"num_objects":7509,"num_object_clones":0,"num_object_copies":22527,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7509,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8155,"num_write_kb":501248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.f","version":"66'15","reported_seq":78,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T09:27:39.153876+0000","last_change":"2024-09-16T09:26:33.348884+0000","last_active":"2024-09-16T09:27:39.153876+0000","last_peered":"2024-09-16T09:27:39.153876+0000","last_clean":"2024-09-16T09:27:39.153876+0000","last_became_active":"2024-09-16T09:26:33.295672+0000","last_became_peered":"2024-09-16T09:26:33.295672+0000","last_unstale":"2024-09-16T09:27:39.153876+0000","last_undegraded":"2024-09-16T09:27:39.153876+0000","last_fullsized":"2024-09-16T09:27:39.153876+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-16T09:26:23.191684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:26:23.191684+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:27.974437+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":139,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T09:27:39.157759+0000","last_change":"2024-09-16T09:26:40.364509+0000","last_active":"2024-09-16T09:27:39.157759+0000","last_peered":"2024-09-16T09:27:39.157759+0000","last_clean":"2024-09-16T09:27:39.157759+0000","last_became_active":"2024-09-16T09:25:01.702629+0000","last_became_peered":"2024-09-16T09:25:01.702629+0000","last_unstale":"2024-09-16T09:27:39.157759+0000","last_undegraded":"2024-09-16T09:27:39.157759+0000","last_fullsized":"2024-09-16T09:27:39.157759+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:40.364442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:40.364442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:02.622633+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"66'8","reported_seq":169,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-16T09:27:44.135650+0000","last_change":"2024-09-16T09:25:05.714128+0000","last_active":"2024-09-16T09:27:44.135650+0000","last_peered":"2024-09-16T09:27:44.135650+0000","last_clean":"2024-09-16T09:27:44.135650+0000","last_became_active":"2024-09-16T09:25:05.713813+0000","last_became_peered":"2024-09-16T09:25:05.713813+0000","last_unstale":"2024-09-16T09:27:44.135650+0000","last_undegraded":"2024-09-16T09:27:44.135650+0000","last_fullsized":"2024-09-16T09:27:44.135650+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:57.826189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":130,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-16T09:27:44.138461+0000","last_change":"2024-09-16T09:26:38.289448+0000","last_active":"2024-09-16T09:27:44.138461+0000","last_peered":"2024-09-16T09:27:44.138461+0000","last_clean":"2024-09-16T09:27:44.138461+0000","last_became_active":"2024-09-16T09:25:07.717344+0000","last_became_peered":"2024-09-16T09:25:07.717344+0000","last_unstale":"2024-09-16T09:27:44.138461+0000","last_undegraded":"2024-09-16T09:27:44.138461+0000","last_fullsized":"2024-09-16T09:27:44.138461+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:38.289387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:38.289387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:01.979399+0000","scrub_duration":114,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.5","version":"73'79","reported_seq":70,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-16T09:27:44.135199+0000","last_change":"2024-09-16T09:26:35.687824+0000","last_active":"2024-09-16T09:27:44.135199+0000","last_peered":"2024-09-16T09:27:44.135199+0000","last_clean":"2024-09-16T09:27:44.135199+0000","last_became_active":"2024-09-16T09:26:35.117714+0000","last_became_peered":"2024-09-16T09:26:35.117714+0000","last_unstale":"2024-09-16T09:27:44.135199+0000","last_undegraded":"2024-09-16T09:27:44.135199+0000","last_fullsized":"2024-09-16T09:27:44.135199+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:26:35.687778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:26:35.687778+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:04.588420+0000","scrub_duration":25,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3","version":"118'7944","reported_seq":8026,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T09:27:42.679848+0000","last_change":"2024-09-16T09:26:08.119864+0000","last_active":"2024-09-16T09:27:42.679848+0000","last_peered":"2024-09-16T09:27:42.679848+0000","last_clean":"2024-09-16T09:27:42.679848+0000","last_became_active":"2024-09-16T09:26:08.119550+0000","last_became_peered":"2024-09-16T09:26:08.119550+0000","last_unstale":"2024-09-16T09:27:42.679848+0000","last_undegraded":"2024-09-16T09:27:42.679848+0000","last_fullsized":"2024-09-16T09:27:42.679848+0000","mapping_epoch":82,"log_start":"96'3600","ondisk_log_start":"96'3600","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_clean_scrub_stamp":"2024-09-16T09:26:07.096764+0000","objects_scrubbed":0,"log_size":4344,"log_dups_size":0,"ondisk_log_size":4344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:33.053037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":520617984,"num_objects":7944,"num_object_clones":0,"num_object_copies":23832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7944,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7944,"num_write_kb":508416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4],"acting":[3,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"66'18","reported_seq":167,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T09:27:39.157598+0000","last_change":"2024-09-16T09:27:08.857567+0000","last_active":"2024-09-16T09:27:39.157598+0000","last_peered":"2024-09-16T09:27:39.157598+0000","last_clean":"2024-09-16T09:27:39.157598+0000","last_became_active":"2024-09-16T09:25:03.714208+0000","last_became_peered":"2024-09-16T09:25:03.714208+0000","last_unstale":"2024-09-16T09:27:39.157598+0000","last_undegraded":"2024-09-16T09:27:39.157598+0000","last_fullsized":"2024-09-16T09:27:39.157598+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'18","last_scrub_stamp":"2024-09-16T09:27:08.857484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:08.857484+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:31.036637+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":139,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T09:27:39.150913+0000","last_change":"2024-09-16T09:26:28.481262+0000","last_active":"2024-09-16T09:27:39.150913+0000","last_peered":"2024-09-16T09:27:39.150913+0000","last_clean":"2024-09-16T09:27:39.150913+0000","last_became_active":"2024-09-16T09:25:01.704994+0000","last_became_peered":"2024-09-16T09:25:01.704994+0000","last_unstale":"2024-09-16T09:27:39.150913+0000","last_undegraded":"2024-09-16T09:27:39.150913+0000","last_fullsized":"2024-09-16T09:27:39.150913+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:28.481194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:28.481194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:44.529826+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":135,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-16T09:27:44.134793+0000","last_change":"2024-09-16T09:26:33.729827+0000","last_active":"2024-09-16T09:27:44.134793+0000","last_peered":"2024-09-16T09:27:44.134793+0000","last_clean":"2024-09-16T09:27:44.134793+0000","last_became_active":"2024-09-16T09:25:05.713360+0000","last_became_peered":"2024-09-16T09:25:05.713360+0000","last_unstale":"2024-09-16T09:27:44.134793+0000","last_undegraded":"2024-09-16T09:27:44.134793+0000","last_fullsized":"2024-09-16T09:27:44.134793+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:33.729770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:26:33.729770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:54.598643+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":131,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-16T09:27:44.136746+0000","last_change":"2024-09-16T09:27:13.548667+0000","last_active":"2024-09-16T09:27:44.136746+0000","last_peered":"2024-09-16T09:27:44.136746+0000","last_clean":"2024-09-16T09:27:44.136746+0000","last_became_active":"2024-09-16T09:25:07.719159+0000","last_became_peered":"2024-09-16T09:25:07.719159+0000","last_unstale":"2024-09-16T09:27:44.136746+0000","last_undegraded":"2024-09-16T09:27:44.136746+0000","last_fullsized":"2024-09-16T09:27:44.136746+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:13.548608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:13.548608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:20.672287+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"73'79","reported_seq":71,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T09:27:39.149665+0000","last_change":"2024-09-16T09:26:35.426510+0000","last_active":"2024-09-16T09:27:39.149665+0000","last_peered":"2024-09-16T09:27:39.149665+0000","last_clean":"2024-09-16T09:27:39.149665+0000","last_became_active":"2024-09-16T09:26:35.131343+0000","last_became_peered":"2024-09-16T09:26:35.131343+0000","last_unstale":"2024-09-16T09:27:39.149665+0000","last_undegraded":"2024-09-16T09:27:39.149665+0000","last_fullsized":"2024-09-16T09:27:39.149665+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:26:35.426313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:26:35.426313+0000","objects_scrubbed":1,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:02.968954+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2","version":"118'7921","reported_seq":8004,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T09:27:42.630868+0000","last_change":"2024-09-16T09:26:08.120627+0000","last_active":"2024-09-16T09:27:42.630868+0000","last_peered":"2024-09-16T09:27:42.630868+0000","last_clean":"2024-09-16T09:27:42.630868+0000","last_became_active":"2024-09-16T09:26:08.120338+0000","last_became_peered":"2024-09-16T09:26:08.120338+0000","last_unstale":"2024-09-16T09:27:42.630868+0000","last_undegraded":"2024-09-16T09:27:42.630868+0000","last_fullsized":"2024-09-16T09:27:42.630868+0000","mapping_epoch":82,"log_start":"97'3600","ondisk_log_start":"97'3600","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_clean_scrub_stamp":"2024-09-16T09:26:07.096764+0000","objects_scrubbed":0,"log_size":4321,"log_dups_size":0,"ondisk_log_size":4321,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:38.116462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":519110656,"num_objects":7921,"num_object_clones":0,"num_object_copies":23763,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7921,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7921,"num_write_kb":506944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"66'14","reported_seq":160,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-16T09:27:44.134821+0000","last_change":"2024-09-16T09:26:38.595468+0000","last_active":"2024-09-16T09:27:44.134821+0000","last_peered":"2024-09-16T09:27:44.134821+0000","last_clean":"2024-09-16T09:27:44.134821+0000","last_became_active":"2024-09-16T09:25:03.708830+0000","last_became_peered":"2024-09-16T09:25:03.708830+0000","last_unstale":"2024-09-16T09:27:44.134821+0000","last_undegraded":"2024-09-16T09:27:44.134821+0000","last_fullsized":"2024-09-16T09:27:44.134821+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'14","last_scrub_stamp":"2024-09-16T09:26:38.595391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:26:38.595391+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:56.981868+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":148,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T09:27:39.158829+0000","last_change":"2024-09-16T09:27:14.637291+0000","last_active":"2024-09-16T09:27:39.158829+0000","last_peered":"2024-09-16T09:27:39.158829+0000","last_clean":"2024-09-16T09:27:39.158829+0000","last_became_active":"2024-09-16T09:25:01.702992+0000","last_became_peered":"2024-09-16T09:25:01.702992+0000","last_unstale":"2024-09-16T09:27:39.158829+0000","last_undegraded":"2024-09-16T09:27:39.158829+0000","last_fullsized":"2024-09-16T09:27:39.158829+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:27:14.637226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:27:14.637226+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:43.815503+0000","scrub_duration":37,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"66'8","reported_seq":164,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T09:27:39.156205+0000","last_change":"2024-09-16T09:25:05.715927+0000","last_active":"2024-09-16T09:27:39.156205+0000","last_peered":"2024-09-16T09:27:39.156205+0000","last_clean":"2024-09-16T09:27:39.156205+0000","last_became_active":"2024-09-16T09:25:05.715606+0000","last_became_peered":"2024-09-16T09:25:05.715606+0000","last_unstale":"2024-09-16T09:27:39.156205+0000","last_undegraded":"2024-09-16T09:27:39.156205+0000","last_fullsized":"2024-09-16T09:27:39.156205+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:57.590084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":130,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-16T09:27:44.146412+0000","last_change":"2024-09-16T09:26:40.699888+0000","last_active":"2024-09-16T09:27:44.146412+0000","last_peered":"2024-09-16T09:27:44.146412+0000","last_clean":"2024-09-16T09:27:44.146412+0000","last_became_active":"2024-09-16T09:25:07.722017+0000","last_became_peered":"2024-09-16T09:25:07.722017+0000","last_unstale":"2024-09-16T09:27:44.146412+0000","last_undegraded":"2024-09-16T09:27:44.146412+0000","last_fullsized":"2024-09-16T09:27:44.146412+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:40.699813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:40.699813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:54.431740+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"73'79","reported_seq":146,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-16T09:27:44.143229+0000","last_change":"2024-09-16T09:26:36.183925+0000","last_active":"2024-09-16T09:27:44.143229+0000","last_peered":"2024-09-16T09:27:44.143229+0000","last_clean":"2024-09-16T09:27:44.143229+0000","last_became_active":"2024-09-16T09:26:35.126759+0000","last_became_peered":"2024-09-16T09:26:35.126759+0000","last_unstale":"2024-09-16T09:27:44.143229+0000","last_undegraded":"2024-09-16T09:27:44.143229+0000","last_fullsized":"2024-09-16T09:27:44.143229+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:26:36.183868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:26:36.183868+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"120'8090","reported_seq":8177,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-16T09:27:47.451174+0000","last_change":"2024-09-16T09:26:08.117045+0000","last_active":"2024-09-16T09:27:47.451174+0000","last_peered":"2024-09-16T09:27:47.451174+0000","last_clean":"2024-09-16T09:27:47.451174+0000","last_became_active":"2024-09-16T09:26:08.116726+0000","last_became_peered":"2024-09-16T09:26:08.116726+0000","last_unstale":"2024-09-16T09:27:47.451174+0000","last_undegraded":"2024-09-16T09:27:47.451174+0000","last_fullsized":"2024-09-16T09:27:47.451174+0000","mapping_epoch":82,"log_start":"97'3500","ondisk_log_start":"97'3500","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_clean_scrub_stamp":"2024-09-16T09:26:07.096764+0000","objects_scrubbed":0,"log_size":4590,"log_dups_size":0,"ondisk_log_size":4590,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":488636416,"num_objects":7456,"num_object_clones":0,"num_object_copies":22368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7456,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8090,"num_write_kb":497472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"66'10","reported_seq":150,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T09:27:39.133707+0000","last_change":"2024-09-16T09:26:28.194465+0000","last_active":"2024-09-16T09:27:39.133707+0000","last_peered":"2024-09-16T09:27:39.133707+0000","last_clean":"2024-09-16T09:27:39.133707+0000","last_became_active":"2024-09-16T09:25:03.712985+0000","last_became_peered":"2024-09-16T09:25:03.712985+0000","last_unstale":"2024-09-16T09:27:39.133707+0000","last_undegraded":"2024-09-16T09:27:39.133707+0000","last_fullsized":"2024-09-16T09:27:39.133707+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-16T09:26:28.194023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:26:28.194023+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:30.198573+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":142,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-16T09:27:44.139122+0000","last_change":"2024-09-16T09:26:25.086388+0000","last_active":"2024-09-16T09:27:44.139122+0000","last_peered":"2024-09-16T09:27:44.139122+0000","last_clean":"2024-09-16T09:27:44.139122+0000","last_became_active":"2024-09-16T09:25:01.706440+0000","last_became_peered":"2024-09-16T09:25:01.706440+0000","last_unstale":"2024-09-16T09:27:44.139122+0000","last_undegraded":"2024-09-16T09:27:44.139122+0000","last_fullsized":"2024-09-16T09:27:44.139122+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:25.086322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:25.086322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:46.687420+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"66'8","reported_seq":172,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T09:27:39.155274+0000","last_change":"2024-09-16T09:27:17.664572+0000","last_active":"2024-09-16T09:27:39.155274+0000","last_peered":"2024-09-16T09:27:39.155274+0000","last_clean":"2024-09-16T09:27:39.155274+0000","last_became_active":"2024-09-16T09:25:05.714507+0000","last_became_peered":"2024-09-16T09:25:05.714507+0000","last_unstale":"2024-09-16T09:27:39.155274+0000","last_undegraded":"2024-09-16T09:27:39.155274+0000","last_fullsized":"2024-09-16T09:27:39.155274+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:27:17.664493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:17.664493+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:36.536988+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":127,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T09:27:39.154789+0000","last_change":"2024-09-16T09:26:29.632092+0000","last_active":"2024-09-16T09:27:39.154789+0000","last_peered":"2024-09-16T09:27:39.154789+0000","last_clean":"2024-09-16T09:27:39.154789+0000","last_became_active":"2024-09-16T09:25:07.719604+0000","last_became_peered":"2024-09-16T09:25:07.719604+0000","last_unstale":"2024-09-16T09:27:39.154789+0000","last_undegraded":"2024-09-16T09:27:39.154789+0000","last_fullsized":"2024-09-16T09:27:39.154789+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:29.632015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:29.632015+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:48.785098+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"73'79","reported_seq":146,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-16T09:27:44.143224+0000","last_change":"2024-09-16T09:26:35.170288+0000","last_active":"2024-09-16T09:27:44.143224+0000","last_peered":"2024-09-16T09:27:44.143224+0000","last_clean":"2024-09-16T09:27:44.143224+0000","last_became_active":"2024-09-16T09:26:35.125571+0000","last_became_peered":"2024-09-16T09:26:35.125571+0000","last_unstale":"2024-09-16T09:27:44.143224+0000","last_undegraded":"2024-09-16T09:27:44.143224+0000","last_fullsized":"2024-09-16T09:27:44.143224+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:26:35.170229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:26:35.170229+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:53.210344+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.0","version":"120'8239","reported_seq":8396,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-16T09:27:47.495586+0000","last_change":"2024-09-16T09:27:16.936395+0000","last_active":"2024-09-16T09:27:47.495586+0000","last_peered":"2024-09-16T09:27:47.495586+0000","last_clean":"2024-09-16T09:27:47.495586+0000","last_became_active":"2024-09-16T09:26:08.117581+0000","last_became_peered":"2024-09-16T09:26:08.117581+0000","last_unstale":"2024-09-16T09:27:47.495586+0000","last_undegraded":"2024-09-16T09:27:47.495586+0000","last_fullsized":"2024-09-16T09:27:47.495586+0000","mapping_epoch":82,"log_start":"92'2800","ondisk_log_start":"92'2800","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"109'6200","last_scrub_stamp":"2024-09-16T09:27:16.936218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_clean_scrub_stamp":"2024-09-16T09:27:16.936218+0000","objects_scrubbed":6157,"log_size":5439,"log_dups_size":0,"ondisk_log_size":5439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:45.561197+0000","scrub_duration":1612,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":497614848,"num_objects":7593,"num_object_clones":0,"num_object_copies":22779,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7593,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8239,"num_write_kb":506624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3","version":"66'19","reported_seq":163,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T09:27:39.155482+0000","last_change":"2024-09-16T09:25:03.715086+0000","last_active":"2024-09-16T09:27:39.155482+0000","last_peered":"2024-09-16T09:27:39.155482+0000","last_clean":"2024-09-16T09:27:39.155482+0000","last_became_active":"2024-09-16T09:25:03.714797+0000","last_became_peered":"2024-09-16T09:25:03.714797+0000","last_unstale":"2024-09-16T09:27:39.155482+0000","last_undegraded":"2024-09-16T09:27:39.155482+0000","last_fullsized":"2024-09-16T09:27:39.155482+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:42.494544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":147,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T09:27:39.136510+0000","last_change":"2024-09-16T09:27:24.219132+0000","last_active":"2024-09-16T09:27:39.136510+0000","last_peered":"2024-09-16T09:27:39.136510+0000","last_clean":"2024-09-16T09:27:39.136510+0000","last_became_active":"2024-09-16T09:25:01.706806+0000","last_became_peered":"2024-09-16T09:25:01.706806+0000","last_unstale":"2024-09-16T09:27:39.136510+0000","last_undegraded":"2024-09-16T09:27:39.136510+0000","last_fullsized":"2024-09-16T09:27:39.136510+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:24.218910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:27:24.218910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:52.294630+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":135,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-16T09:27:44.142703+0000","last_change":"2024-09-16T09:27:09.337607+0000","last_active":"2024-09-16T09:27:44.142703+0000","last_peered":"2024-09-16T09:27:44.142703+0000","last_clean":"2024-09-16T09:27:44.142703+0000","last_became_active":"2024-09-16T09:25:05.708663+0000","last_became_peered":"2024-09-16T09:25:05.708663+0000","last_unstale":"2024-09-16T09:27:44.142703+0000","last_undegraded":"2024-09-16T09:27:44.142703+0000","last_fullsized":"2024-09-16T09:27:44.142703+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:09.337528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:09.337528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:24.568824+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":127,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T09:27:39.133519+0000","last_change":"2024-09-16T09:26:37.282798+0000","last_active":"2024-09-16T09:27:39.133519+0000","last_peered":"2024-09-16T09:27:39.133519+0000","last_clean":"2024-09-16T09:27:39.133519+0000","last_became_active":"2024-09-16T09:25:07.719267+0000","last_became_peered":"2024-09-16T09:25:07.719267+0000","last_unstale":"2024-09-16T09:27:39.133519+0000","last_undegraded":"2024-09-16T09:27:39.133519+0000","last_fullsized":"2024-09-16T09:27:39.133519+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:37.282735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:37.282735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:54.832930+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"73'79","reported_seq":70,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-16T09:27:44.138173+0000","last_change":"2024-09-16T09:26:35.154953+0000","last_active":"2024-09-16T09:27:44.138173+0000","last_peered":"2024-09-16T09:27:44.138173+0000","last_clean":"2024-09-16T09:27:44.138173+0000","last_became_active":"2024-09-16T09:26:35.117627+0000","last_became_peered":"2024-09-16T09:26:35.117627+0000","last_unstale":"2024-09-16T09:27:44.138173+0000","last_undegraded":"2024-09-16T09:27:44.138173+0000","last_fullsized":"2024-09-16T09:27:44.138173+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:26:35.154875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:26:35.154875+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:57.010300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"120'8301","reported_seq":8388,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-16T09:27:47.495290+0000","last_change":"2024-09-16T09:26:08.122122+0000","last_active":"2024-09-16T09:27:47.495290+0000","last_peered":"2024-09-16T09:27:47.495290+0000","last_clean":"2024-09-16T09:27:47.495290+0000","last_became_active":"2024-09-16T09:26:08.121828+0000","last_became_peered":"2024-09-16T09:26:08.121828+0000","last_unstale":"2024-09-16T09:27:47.495290+0000","last_undegraded":"2024-09-16T09:27:47.495290+0000","last_fullsized":"2024-09-16T09:27:47.495290+0000","mapping_epoch":82,"log_start":"91'2600","ondisk_log_start":"91'2600","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_clean_scrub_stamp":"2024-09-16T09:26:07.096764+0000","objects_scrubbed":0,"log_size":5701,"log_dups_size":0,"ondisk_log_size":5701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:40.329742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":498401280,"num_objects":7605,"num_object_clones":0,"num_object_copies":22815,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7605,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8301,"num_write_kb":508992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.4","version":"66'28","reported_seq":208,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T09:27:39.136415+0000","last_change":"2024-09-16T09:25:03.705300+0000","last_active":"2024-09-16T09:27:39.136415+0000","last_peered":"2024-09-16T09:27:39.136415+0000","last_clean":"2024-09-16T09:27:39.136415+0000","last_became_active":"2024-09-16T09:25:03.704878+0000","last_became_peered":"2024-09-16T09:25:03.704878+0000","last_unstale":"2024-09-16T09:27:39.136415+0000","last_undegraded":"2024-09-16T09:27:39.136415+0000","last_fullsized":"2024-09-16T09:27:39.136415+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:46.163665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":135,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-16T09:27:44.136468+0000","last_change":"2024-09-16T09:25:01.703394+0000","last_active":"2024-09-16T09:27:44.136468+0000","last_peered":"2024-09-16T09:27:44.136468+0000","last_clean":"2024-09-16T09:27:44.136468+0000","last_became_active":"2024-09-16T09:25:01.703076+0000","last_became_peered":"2024-09-16T09:25:01.703076+0000","last_unstale":"2024-09-16T09:27:44.136468+0000","last_undegraded":"2024-09-16T09:27:44.136468+0000","last_fullsized":"2024-09-16T09:27:44.136468+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:51.678175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":152,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-16T09:27:44.183242+0000","last_change":"2024-09-16T09:27:44.183242+0000","last_active":"2024-09-16T09:27:44.183242+0000","last_peered":"2024-09-16T09:27:44.183242+0000","last_clean":"2024-09-16T09:27:44.183242+0000","last_became_active":"2024-09-16T09:25:00.696237+0000","last_became_peered":"2024-09-16T09:25:00.696237+0000","last_unstale":"2024-09-16T09:27:44.183242+0000","last_undegraded":"2024-09-16T09:27:44.183242+0000","last_fullsized":"2024-09-16T09:27:44.183242+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-16T09:27:44.183111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:27:44.183111+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:47.187301+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.0025137430000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":123,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T09:27:39.147693+0000","last_change":"2024-09-16T09:25:05.718319+0000","last_active":"2024-09-16T09:27:39.147693+0000","last_peered":"2024-09-16T09:27:39.147693+0000","last_clean":"2024-09-16T09:27:39.147693+0000","last_became_active":"2024-09-16T09:25:05.717591+0000","last_became_peered":"2024-09-16T09:25:05.717591+0000","last_unstale":"2024-09-16T09:27:39.147693+0000","last_undegraded":"2024-09-16T09:27:39.147693+0000","last_fullsized":"2024-09-16T09:27:39.147693+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:49.599882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":127,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T09:27:39.150773+0000","last_change":"2024-09-16T09:26:24.472634+0000","last_active":"2024-09-16T09:27:39.150773+0000","last_peered":"2024-09-16T09:27:39.150773+0000","last_clean":"2024-09-16T09:27:39.150773+0000","last_became_active":"2024-09-16T09:25:07.717436+0000","last_became_peered":"2024-09-16T09:25:07.717436+0000","last_unstale":"2024-09-16T09:27:39.150773+0000","last_undegraded":"2024-09-16T09:27:39.150773+0000","last_fullsized":"2024-09-16T09:27:39.150773+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:24.472574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:24.472574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"73'79","reported_seq":203,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-16T09:27:44.147926+0000","last_change":"2024-09-16T09:26:35.745908+0000","last_active":"2024-09-16T09:27:44.147926+0000","last_peered":"2024-09-16T09:27:44.147926+0000","last_clean":"2024-09-16T09:27:44.147926+0000","last_became_active":"2024-09-16T09:26:35.115364+0000","last_became_peered":"2024-09-16T09:26:35.115364+0000","last_unstale":"2024-09-16T09:27:44.147926+0000","last_undegraded":"2024-09-16T09:27:44.147926+0000","last_fullsized":"2024-09-16T09:27:44.147926+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:26:35.745837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:26:35.745837+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:48.253829+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.4","version":"118'7762","reported_seq":7845,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T09:27:42.621880+0000","last_change":"2024-09-16T09:26:08.120333+0000","last_active":"2024-09-16T09:27:42.621880+0000","last_peered":"2024-09-16T09:27:42.621880+0000","last_clean":"2024-09-16T09:27:42.621880+0000","last_became_active":"2024-09-16T09:26:08.120051+0000","last_became_peered":"2024-09-16T09:26:08.120051+0000","last_unstale":"2024-09-16T09:27:42.621880+0000","last_undegraded":"2024-09-16T09:27:42.621880+0000","last_fullsized":"2024-09-16T09:27:42.621880+0000","mapping_epoch":82,"log_start":"91'2400","ondisk_log_start":"91'2400","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_clean_scrub_stamp":"2024-09-16T09:26:07.096764+0000","objects_scrubbed":0,"log_size":5362,"log_dups_size":0,"ondisk_log_size":5362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:39.620556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":508690432,"num_objects":7762,"num_object_clones":0,"num_object_copies":23286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7762,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7762,"num_write_kb":496768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,1],"acting":[0,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"66'13","reported_seq":162,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T09:27:39.134710+0000","last_change":"2024-09-16T09:26:27.244130+0000","last_active":"2024-09-16T09:27:39.134710+0000","last_peered":"2024-09-16T09:27:39.134710+0000","last_clean":"2024-09-16T09:27:39.134710+0000","last_became_active":"2024-09-16T09:25:03.713241+0000","last_became_peered":"2024-09-16T09:25:03.713241+0000","last_unstale":"2024-09-16T09:27:39.134710+0000","last_undegraded":"2024-09-16T09:27:39.134710+0000","last_fullsized":"2024-09-16T09:27:39.134710+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'13","last_scrub_stamp":"2024-09-16T09:26:27.244001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:26:27.244001+0000","objects_scrubbed":5,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:42.934017+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":139,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T09:27:39.133814+0000","last_change":"2024-09-16T09:26:26.230421+0000","last_active":"2024-09-16T09:27:39.133814+0000","last_peered":"2024-09-16T09:27:39.133814+0000","last_clean":"2024-09-16T09:27:39.133814+0000","last_became_active":"2024-09-16T09:25:01.706828+0000","last_became_peered":"2024-09-16T09:25:01.706828+0000","last_unstale":"2024-09-16T09:27:39.133814+0000","last_undegraded":"2024-09-16T09:27:39.133814+0000","last_fullsized":"2024-09-16T09:27:39.133814+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:26.230338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:26.230338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:38.139199+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":148,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T09:27:39.145597+0000","last_change":"2024-09-16T09:27:06.062939+0000","last_active":"2024-09-16T09:27:39.145597+0000","last_peered":"2024-09-16T09:27:39.145597+0000","last_clean":"2024-09-16T09:27:39.145597+0000","last_became_active":"2024-09-16T09:25:00.697822+0000","last_became_peered":"2024-09-16T09:25:00.697822+0000","last_unstale":"2024-09-16T09:27:39.145597+0000","last_undegraded":"2024-09-16T09:27:39.145597+0000","last_fullsized":"2024-09-16T09:27:39.145597+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:27:06.062871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:27:06.062871+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:10.411432+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0.00055022299999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":131,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T09:27:39.156748+0000","last_change":"2024-09-16T09:26:30.833349+0000","last_active":"2024-09-16T09:27:39.156748+0000","last_peered":"2024-09-16T09:27:39.156748+0000","last_clean":"2024-09-16T09:27:39.156748+0000","last_became_active":"2024-09-16T09:25:05.711539+0000","last_became_peered":"2024-09-16T09:25:05.711539+0000","last_unstale":"2024-09-16T09:27:39.156748+0000","last_undegraded":"2024-09-16T09:27:39.156748+0000","last_fullsized":"2024-09-16T09:27:39.156748+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:30.833282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:26:30.833282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:46.240939+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":122,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-16T09:27:44.146944+0000","last_change":"2024-09-16T09:25:07.723554+0000","last_active":"2024-09-16T09:27:44.146944+0000","last_peered":"2024-09-16T09:27:44.146944+0000","last_clean":"2024-09-16T09:27:44.146944+0000","last_became_active":"2024-09-16T09:25:07.723282+0000","last_became_peered":"2024-09-16T09:25:07.723282+0000","last_unstale":"2024-09-16T09:27:44.146944+0000","last_undegraded":"2024-09-16T09:27:44.146944+0000","last_fullsized":"2024-09-16T09:27:44.146944+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:44.576561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"73'79","reported_seq":66,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T09:27:39.134202+0000","last_change":"2024-09-16T09:26:35.272980+0000","last_active":"2024-09-16T09:27:39.134202+0000","last_peered":"2024-09-16T09:27:39.134202+0000","last_clean":"2024-09-16T09:27:39.134202+0000","last_became_active":"2024-09-16T09:26:35.116315+0000","last_became_peered":"2024-09-16T09:26:35.116315+0000","last_unstale":"2024-09-16T09:27:39.134202+0000","last_undegraded":"2024-09-16T09:27:39.134202+0000","last_fullsized":"2024-09-16T09:27:39.134202+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:26:35.272938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:26:35.272938+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:47.275547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"120'8322","reported_seq":8409,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-16T09:27:47.507578+0000","last_change":"2024-09-16T09:26:08.121383+0000","last_active":"2024-09-16T09:27:47.507578+0000","last_peered":"2024-09-16T09:27:47.507578+0000","last_clean":"2024-09-16T09:27:47.507578+0000","last_became_active":"2024-09-16T09:26:08.121117+0000","last_became_peered":"2024-09-16T09:26:08.121117+0000","last_unstale":"2024-09-16T09:27:47.507578+0000","last_undegraded":"2024-09-16T09:27:47.507578+0000","last_fullsized":"2024-09-16T09:27:47.507578+0000","mapping_epoch":82,"log_start":"96'3400","ondisk_log_start":"96'3400","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_clean_scrub_stamp":"2024-09-16T09:26:07.096764+0000","objects_scrubbed":0,"log_size":4922,"log_dups_size":0,"ondisk_log_size":4922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:32.708766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":500563968,"num_objects":7638,"num_object_clones":0,"num_object_copies":22914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7638,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8322,"num_write_kb":510720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,0],"acting":[7,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"66'12","reported_seq":145,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T09:27:39.158656+0000","last_change":"2024-09-16T09:25:03.715612+0000","last_active":"2024-09-16T09:27:39.158656+0000","last_peered":"2024-09-16T09:27:39.158656+0000","last_clean":"2024-09-16T09:27:39.158656+0000","last_became_active":"2024-09-16T09:25:03.715315+0000","last_became_peered":"2024-09-16T09:25:03.715315+0000","last_unstale":"2024-09-16T09:27:39.158656+0000","last_undegraded":"2024-09-16T09:27:39.158656+0000","last_fullsized":"2024-09-16T09:27:39.158656+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:48.641271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"59'1","reported_seq":148,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T09:27:39.148749+0000","last_change":"2024-09-16T09:27:00.666748+0000","last_active":"2024-09-16T09:27:39.148749+0000","last_peered":"2024-09-16T09:27:39.148749+0000","last_clean":"2024-09-16T09:27:39.148749+0000","last_became_active":"2024-09-16T09:25:01.701599+0000","last_became_peered":"2024-09-16T09:25:01.701599+0000","last_unstale":"2024-09-16T09:27:39.148749+0000","last_undegraded":"2024-09-16T09:27:39.148749+0000","last_fullsized":"2024-09-16T09:27:39.148749+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:27:00.666673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:27:00.666673+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:13.128996+0000","scrub_duration":28,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"67'5","reported_seq":313,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-16T09:27:45.657157+0000","last_change":"2024-09-16T09:26:38.722150+0000","last_active":"2024-09-16T09:27:45.657157+0000","last_peered":"2024-09-16T09:27:45.657157+0000","last_clean":"2024-09-16T09:27:45.657157+0000","last_became_active":"2024-09-16T09:25:00.696546+0000","last_became_peered":"2024-09-16T09:25:00.696546+0000","last_unstale":"2024-09-16T09:27:45.657157+0000","last_undegraded":"2024-09-16T09:27:45.657157+0000","last_fullsized":"2024-09-16T09:27:45.657157+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'5","last_scrub_stamp":"2024-09-16T09:26:38.722074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:26:38.722074+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:03.198309+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0.0059770420000000001,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":160,"num_read_kb":155,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":134,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-16T09:27:44.139050+0000","last_change":"2024-09-16T09:27:07.337170+0000","last_active":"2024-09-16T09:27:44.139050+0000","last_peered":"2024-09-16T09:27:44.139050+0000","last_clean":"2024-09-16T09:27:44.139050+0000","last_became_active":"2024-09-16T09:25:05.714704+0000","last_became_peered":"2024-09-16T09:25:05.714704+0000","last_unstale":"2024-09-16T09:27:44.139050+0000","last_undegraded":"2024-09-16T09:27:44.139050+0000","last_fullsized":"2024-09-16T09:27:44.139050+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:07.337124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:07.337124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:33.974050+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":127,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T09:27:39.136002+0000","last_change":"2024-09-16T09:26:24.231324+0000","last_active":"2024-09-16T09:27:39.136002+0000","last_peered":"2024-09-16T09:27:39.136002+0000","last_clean":"2024-09-16T09:27:39.136002+0000","last_became_active":"2024-09-16T09:25:07.714158+0000","last_became_peered":"2024-09-16T09:25:07.714158+0000","last_unstale":"2024-09-16T09:27:39.136002+0000","last_undegraded":"2024-09-16T09:27:39.136002+0000","last_fullsized":"2024-09-16T09:27:39.136002+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:24.231167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:24.231167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:51.049493+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.6","version":"118'7852","reported_seq":7983,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T09:27:42.609758+0000","last_change":"2024-09-16T09:27:26.477113+0000","last_active":"2024-09-16T09:27:42.609758+0000","last_peered":"2024-09-16T09:27:42.609758+0000","last_clean":"2024-09-16T09:27:42.609758+0000","last_became_active":"2024-09-16T09:26:08.121253+0000","last_became_peered":"2024-09-16T09:26:08.121253+0000","last_unstale":"2024-09-16T09:27:42.609758+0000","last_undegraded":"2024-09-16T09:27:42.609758+0000","last_fullsized":"2024-09-16T09:27:42.609758+0000","mapping_epoch":82,"log_start":"91'2500","ondisk_log_start":"91'2500","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_clean_scrub_stamp":"2024-09-16T09:26:07.096764+0000","objects_scrubbed":4399,"log_size":5352,"log_dups_size":0,"ondisk_log_size":5352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:05.001537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":514588672,"num_objects":7852,"num_object_clones":0,"num_object_copies":23556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7852,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7852,"num_write_kb":502528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"66'16","reported_seq":195,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-16T09:27:44.143368+0000","last_change":"2024-09-16T09:27:14.265733+0000","last_active":"2024-09-16T09:27:44.143368+0000","last_peered":"2024-09-16T09:27:44.143368+0000","last_clean":"2024-09-16T09:27:44.143368+0000","last_became_active":"2024-09-16T09:25:03.714118+0000","last_became_peered":"2024-09-16T09:25:03.714118+0000","last_unstale":"2024-09-16T09:27:44.143368+0000","last_undegraded":"2024-09-16T09:27:44.143368+0000","last_fullsized":"2024-09-16T09:27:44.143368+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'16","last_scrub_stamp":"2024-09-16T09:27:14.265648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:14.265648+0000","objects_scrubbed":8,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:42.225176+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":131,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T09:27:39.150997+0000","last_change":"2024-09-16T09:25:01.704246+0000","last_active":"2024-09-16T09:27:39.150997+0000","last_peered":"2024-09-16T09:27:39.150997+0000","last_clean":"2024-09-16T09:27:39.150997+0000","last_became_active":"2024-09-16T09:25:01.703730+0000","last_became_peered":"2024-09-16T09:25:01.703730+0000","last_unstale":"2024-09-16T09:27:39.150997+0000","last_undegraded":"2024-09-16T09:27:39.150997+0000","last_fullsized":"2024-09-16T09:27:39.150997+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:48.457351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"73'79","reported_seq":198,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-16T09:27:44.146675+0000","last_change":"2024-09-16T09:26:34.751780+0000","last_active":"2024-09-16T09:27:44.146675+0000","last_peered":"2024-09-16T09:27:44.146675+0000","last_clean":"2024-09-16T09:27:44.146675+0000","last_became_active":"2024-09-16T09:26:33.297014+0000","last_became_peered":"2024-09-16T09:26:33.297014+0000","last_unstale":"2024-09-16T09:27:44.146675+0000","last_undegraded":"2024-09-16T09:27:44.146675+0000","last_fullsized":"2024-09-16T09:27:44.146675+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:26:34.751720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:26:34.751720+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:44.448893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":134,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-16T09:27:44.148347+0000","last_change":"2024-09-16T09:26:39.699133+0000","last_active":"2024-09-16T09:27:44.148347+0000","last_peered":"2024-09-16T09:27:44.148347+0000","last_clean":"2024-09-16T09:27:44.148347+0000","last_became_active":"2024-09-16T09:25:05.714341+0000","last_became_peered":"2024-09-16T09:25:05.714341+0000","last_unstale":"2024-09-16T09:27:44.148347+0000","last_undegraded":"2024-09-16T09:27:44.148347+0000","last_fullsized":"2024-09-16T09:27:44.148347+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:39.699059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:26:39.699059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:45.904027+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":130,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-16T09:27:44.138871+0000","last_change":"2024-09-16T09:27:21.192880+0000","last_active":"2024-09-16T09:27:44.138871+0000","last_peered":"2024-09-16T09:27:44.138871+0000","last_clean":"2024-09-16T09:27:44.138871+0000","last_became_active":"2024-09-16T09:25:07.714281+0000","last_became_peered":"2024-09-16T09:25:07.714281+0000","last_unstale":"2024-09-16T09:27:44.138871+0000","last_undegraded":"2024-09-16T09:27:44.138871+0000","last_fullsized":"2024-09-16T09:27:44.138871+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:21.192816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:21.192816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:44.084676+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"120'8129","reported_seq":8217,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-16T09:27:47.512547+0000","last_change":"2024-09-16T09:26:08.122057+0000","last_active":"2024-09-16T09:27:47.512547+0000","last_peered":"2024-09-16T09:27:47.512547+0000","last_clean":"2024-09-16T09:27:47.512547+0000","last_became_active":"2024-09-16T09:26:08.121785+0000","last_became_peered":"2024-09-16T09:26:08.121785+0000","last_unstale":"2024-09-16T09:27:47.512547+0000","last_undegraded":"2024-09-16T09:27:47.512547+0000","last_fullsized":"2024-09-16T09:27:47.512547+0000","mapping_epoch":82,"log_start":"90'2400","ondisk_log_start":"90'2400","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_clean_scrub_stamp":"2024-09-16T09:26:07.096764+0000","objects_scrubbed":0,"log_size":5729,"log_dups_size":0,"ondisk_log_size":5729,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:36.339662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":491651096,"num_objects":7503,"num_object_clones":0,"num_object_copies":22509,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7503,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8129,"num_write_kb":500161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.e","version":"66'11","reported_seq":169,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-16T09:27:44.135692+0000","last_change":"2024-09-16T09:26:33.316695+0000","last_active":"2024-09-16T09:27:44.135692+0000","last_peered":"2024-09-16T09:27:44.135692+0000","last_clean":"2024-09-16T09:27:44.135692+0000","last_became_active":"2024-09-16T09:26:33.287399+0000","last_became_peered":"2024-09-16T09:26:33.287399+0000","last_unstale":"2024-09-16T09:27:44.135692+0000","last_undegraded":"2024-09-16T09:27:44.135692+0000","last_fullsized":"2024-09-16T09:27:44.135692+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:53.689658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":135,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-16T09:27:44.135019+0000","last_change":"2024-09-16T09:25:01.708568+0000","last_active":"2024-09-16T09:27:44.135019+0000","last_peered":"2024-09-16T09:27:44.135019+0000","last_clean":"2024-09-16T09:27:44.135019+0000","last_became_active":"2024-09-16T09:25:01.708270+0000","last_became_peered":"2024-09-16T09:25:01.708270+0000","last_unstale":"2024-09-16T09:27:44.135019+0000","last_undegraded":"2024-09-16T09:27:44.135019+0000","last_fullsized":"2024-09-16T09:27:44.135019+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:54.717044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":134,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-16T09:27:44.138707+0000","last_change":"2024-09-16T09:26:34.171190+0000","last_active":"2024-09-16T09:27:44.138707+0000","last_peered":"2024-09-16T09:27:44.138707+0000","last_clean":"2024-09-16T09:27:44.138707+0000","last_became_active":"2024-09-16T09:25:05.711964+0000","last_became_peered":"2024-09-16T09:25:05.711964+0000","last_unstale":"2024-09-16T09:27:44.138707+0000","last_undegraded":"2024-09-16T09:27:44.138707+0000","last_fullsized":"2024-09-16T09:27:44.138707+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:34.171139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:26:34.171139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:39.195436+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":127,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T09:27:39.149780+0000","last_change":"2024-09-16T09:27:24.030747+0000","last_active":"2024-09-16T09:27:39.149780+0000","last_peered":"2024-09-16T09:27:39.149780+0000","last_clean":"2024-09-16T09:27:39.149780+0000","last_became_active":"2024-09-16T09:25:07.716514+0000","last_became_peered":"2024-09-16T09:25:07.716514+0000","last_unstale":"2024-09-16T09:27:39.149780+0000","last_undegraded":"2024-09-16T09:27:39.149780+0000","last_fullsized":"2024-09-16T09:27:39.149780+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:24.030677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:24.030677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:42.004108+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.8","version":"73'79","reported_seq":77,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-16T09:27:44.138123+0000","last_change":"2024-09-16T09:26:37.166480+0000","last_active":"2024-09-16T09:27:44.138123+0000","last_peered":"2024-09-16T09:27:44.138123+0000","last_clean":"2024-09-16T09:27:44.138123+0000","last_became_active":"2024-09-16T09:26:35.126217+0000","last_became_peered":"2024-09-16T09:26:35.126217+0000","last_unstale":"2024-09-16T09:27:44.138123+0000","last_undegraded":"2024-09-16T09:27:44.138123+0000","last_fullsized":"2024-09-16T09:27:44.138123+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:26:37.166434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:26:37.166434+0000","objects_scrubbed":1,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:58.101963+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262688,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.e","version":"118'7820","reported_seq":7972,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T09:27:42.556999+0000","last_change":"2024-09-16T09:27:16.142844+0000","last_active":"2024-09-16T09:27:42.556999+0000","last_peered":"2024-09-16T09:27:42.556999+0000","last_clean":"2024-09-16T09:27:42.556999+0000","last_became_active":"2024-09-16T09:26:08.121969+0000","last_became_peered":"2024-09-16T09:26:08.121969+0000","last_unstale":"2024-09-16T09:27:42.556999+0000","last_undegraded":"2024-09-16T09:27:42.556999+0000","last_fullsized":"2024-09-16T09:27:42.556999+0000","mapping_epoch":82,"log_start":"89'1600","ondisk_log_start":"89'1600","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"108'6031","last_scrub_stamp":"2024-09-16T09:27:16.142747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_clean_scrub_stamp":"2024-09-16T09:27:16.142747+0000","objects_scrubbed":5966,"log_size":6220,"log_dups_size":0,"ondisk_log_size":6220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:42.214405+0000","scrub_duration":2110,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":512491520,"num_objects":7820,"num_object_clones":0,"num_object_copies":23460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7820,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7820,"num_write_kb":500480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"66'17","reported_seq":167,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-16T09:27:44.135070+0000","last_change":"2024-09-16T09:27:25.615134+0000","last_active":"2024-09-16T09:27:44.135070+0000","last_peered":"2024-09-16T09:27:44.135070+0000","last_clean":"2024-09-16T09:27:44.135070+0000","last_became_active":"2024-09-16T09:25:03.713358+0000","last_became_peered":"2024-09-16T09:25:03.713358+0000","last_unstale":"2024-09-16T09:27:44.135070+0000","last_undegraded":"2024-09-16T09:27:44.135070+0000","last_fullsized":"2024-09-16T09:27:44.135070+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'17","last_scrub_stamp":"2024-09-16T09:27:25.615080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:25.615080+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:48.655184+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":143,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-16T09:27:44.142925+0000","last_change":"2024-09-16T09:26:04.104188+0000","last_active":"2024-09-16T09:27:44.142925+0000","last_peered":"2024-09-16T09:27:44.142925+0000","last_clean":"2024-09-16T09:27:44.142925+0000","last_became_active":"2024-09-16T09:25:01.703355+0000","last_became_peered":"2024-09-16T09:25:01.703355+0000","last_unstale":"2024-09-16T09:27:44.142925+0000","last_undegraded":"2024-09-16T09:27:44.142925+0000","last_fullsized":"2024-09-16T09:27:44.142925+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:04.104130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:04.104130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:39.294771+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":131,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T09:27:39.136100+0000","last_change":"2024-09-16T09:26:36.322641+0000","last_active":"2024-09-16T09:27:39.136100+0000","last_peered":"2024-09-16T09:27:39.136100+0000","last_clean":"2024-09-16T09:27:39.136100+0000","last_became_active":"2024-09-16T09:25:05.716900+0000","last_became_peered":"2024-09-16T09:25:05.716900+0000","last_unstale":"2024-09-16T09:27:39.136100+0000","last_undegraded":"2024-09-16T09:27:39.136100+0000","last_fullsized":"2024-09-16T09:27:39.136100+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:36.322568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:26:36.322568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:46.750549+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":127,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T09:27:39.154029+0000","last_change":"2024-09-16T09:27:17.949869+0000","last_active":"2024-09-16T09:27:39.154029+0000","last_peered":"2024-09-16T09:27:39.154029+0000","last_clean":"2024-09-16T09:27:39.154029+0000","last_became_active":"2024-09-16T09:25:07.721347+0000","last_became_peered":"2024-09-16T09:25:07.721347+0000","last_unstale":"2024-09-16T09:27:39.154029+0000","last_undegraded":"2024-09-16T09:27:39.154029+0000","last_fullsized":"2024-09-16T09:27:39.154029+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:17.949794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:17.949794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:30.022558+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.9","version":"73'79","reported_seq":203,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-16T09:27:44.146484+0000","last_change":"2024-09-16T09:26:36.771785+0000","last_active":"2024-09-16T09:27:44.146484+0000","last_peered":"2024-09-16T09:27:44.146484+0000","last_clean":"2024-09-16T09:27:44.146484+0000","last_became_active":"2024-09-16T09:26:35.126066+0000","last_became_peered":"2024-09-16T09:26:35.126066+0000","last_unstale":"2024-09-16T09:27:44.146484+0000","last_undegraded":"2024-09-16T09:27:44.146484+0000","last_fullsized":"2024-09-16T09:27:44.146484+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:26:36.771720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:26:36.771720+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:03.015639+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.f","version":"118'7937","reported_seq":8020,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T09:27:42.629096+0000","last_change":"2024-09-16T09:26:08.120989+0000","last_active":"2024-09-16T09:27:42.629096+0000","last_peered":"2024-09-16T09:27:42.629096+0000","last_clean":"2024-09-16T09:27:42.629096+0000","last_became_active":"2024-09-16T09:26:08.120360+0000","last_became_peered":"2024-09-16T09:26:08.120360+0000","last_unstale":"2024-09-16T09:27:42.629096+0000","last_undegraded":"2024-09-16T09:27:42.629096+0000","last_fullsized":"2024-09-16T09:27:42.629096+0000","mapping_epoch":82,"log_start":"91'2600","ondisk_log_start":"91'2600","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_clean_scrub_stamp":"2024-09-16T09:26:07.096764+0000","objects_scrubbed":0,"log_size":5337,"log_dups_size":0,"ondisk_log_size":5337,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:38.592305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":520159232,"num_objects":7937,"num_object_clones":0,"num_object_copies":23811,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7937,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7937,"num_write_kb":507968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"66'10","reported_seq":146,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-16T09:27:44.137731+0000","last_change":"2024-09-16T09:25:03.708144+0000","last_active":"2024-09-16T09:27:44.137731+0000","last_peered":"2024-09-16T09:27:44.137731+0000","last_clean":"2024-09-16T09:27:44.137731+0000","last_became_active":"2024-09-16T09:25:03.707723+0000","last_became_peered":"2024-09-16T09:25:03.707723+0000","last_unstale":"2024-09-16T09:27:44.137731+0000","last_undegraded":"2024-09-16T09:27:44.137731+0000","last_fullsized":"2024-09-16T09:27:44.137731+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:55.759028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":139,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T09:27:39.152931+0000","last_change":"2024-09-16T09:27:23.082314+0000","last_active":"2024-09-16T09:27:39.152931+0000","last_peered":"2024-09-16T09:27:39.152931+0000","last_clean":"2024-09-16T09:27:39.152931+0000","last_became_active":"2024-09-16T09:25:01.703094+0000","last_became_peered":"2024-09-16T09:25:01.703094+0000","last_unstale":"2024-09-16T09:27:39.152931+0000","last_undegraded":"2024-09-16T09:27:39.152931+0000","last_fullsized":"2024-09-16T09:27:39.152931+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:23.082107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:27:23.082107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:47.542492+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":175,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T09:27:39.146131+0000","last_change":"2024-09-16T09:26:24.837660+0000","last_active":"2024-09-16T09:27:39.146131+0000","last_peered":"2024-09-16T09:27:39.146131+0000","last_clean":"2024-09-16T09:27:39.146131+0000","last_became_active":"2024-09-16T09:25:05.711590+0000","last_became_peered":"2024-09-16T09:25:05.711590+0000","last_unstale":"2024-09-16T09:27:39.146131+0000","last_undegraded":"2024-09-16T09:27:39.146131+0000","last_fullsized":"2024-09-16T09:27:39.146131+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:26:24.837594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:26:24.837594+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:54.153527+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":139,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-16T09:27:46.596338+0000","last_change":"2024-09-16T09:27:46.596338+0000","last_active":"2024-09-16T09:27:46.596338+0000","last_peered":"2024-09-16T09:27:46.596338+0000","last_clean":"2024-09-16T09:27:46.596338+0000","last_became_active":"2024-09-16T09:25:07.718958+0000","last_became_peered":"2024-09-16T09:25:07.718958+0000","last_unstale":"2024-09-16T09:27:46.596338+0000","last_undegraded":"2024-09-16T09:27:46.596338+0000","last_fullsized":"2024-09-16T09:27:46.596338+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:46.596278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:46.596278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:00.041717+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.8","version":"118'7977","reported_seq":8131,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T09:27:42.500168+0000","last_change":"2024-09-16T09:27:17.928717+0000","last_active":"2024-09-16T09:27:42.500168+0000","last_peered":"2024-09-16T09:27:42.500168+0000","last_clean":"2024-09-16T09:27:42.500168+0000","last_became_active":"2024-09-16T09:26:08.118215+0000","last_became_peered":"2024-09-16T09:26:08.118215+0000","last_unstale":"2024-09-16T09:27:42.500168+0000","last_undegraded":"2024-09-16T09:27:42.500168+0000","last_fullsized":"2024-09-16T09:27:42.500168+0000","mapping_epoch":82,"log_start":"90'1800","ondisk_log_start":"90'1800","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"109'6266","last_scrub_stamp":"2024-09-16T09:27:17.927781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_clean_scrub_stamp":"2024-09-16T09:27:17.927781+0000","objects_scrubbed":6214,"log_size":6177,"log_dups_size":0,"ondisk_log_size":6177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:37.591987+0000","scrub_duration":1966,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":522780672,"num_objects":7977,"num_object_clones":0,"num_object_copies":23931,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7977,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7977,"num_write_kb":510528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"66'9","reported_seq":151,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T09:27:39.155388+0000","last_change":"2024-09-16T09:26:28.596878+0000","last_active":"2024-09-16T09:27:39.155388+0000","last_peered":"2024-09-16T09:27:39.155388+0000","last_clean":"2024-09-16T09:27:39.155388+0000","last_became_active":"2024-09-16T09:25:03.712230+0000","last_became_peered":"2024-09-16T09:25:03.712230+0000","last_unstale":"2024-09-16T09:27:39.155388+0000","last_undegraded":"2024-09-16T09:27:39.155388+0000","last_fullsized":"2024-09-16T09:27:39.155388+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:26:28.596805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:26:28.596805+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:48.802891+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":142,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-16T09:27:44.139155+0000","last_change":"2024-09-16T09:26:26.061173+0000","last_active":"2024-09-16T09:27:44.139155+0000","last_peered":"2024-09-16T09:27:44.139155+0000","last_clean":"2024-09-16T09:27:44.139155+0000","last_became_active":"2024-09-16T09:25:01.702629+0000","last_became_peered":"2024-09-16T09:25:01.702629+0000","last_unstale":"2024-09-16T09:27:44.139155+0000","last_undegraded":"2024-09-16T09:27:44.139155+0000","last_fullsized":"2024-09-16T09:27:44.139155+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:26.061106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:26.061106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:48.923532+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":131,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T09:27:39.142127+0000","last_change":"2024-09-16T09:27:19.997700+0000","last_active":"2024-09-16T09:27:39.142127+0000","last_peered":"2024-09-16T09:27:39.142127+0000","last_clean":"2024-09-16T09:27:39.142127+0000","last_became_active":"2024-09-16T09:25:05.712573+0000","last_became_peered":"2024-09-16T09:25:05.712573+0000","last_unstale":"2024-09-16T09:27:39.142127+0000","last_undegraded":"2024-09-16T09:27:39.142127+0000","last_fullsized":"2024-09-16T09:27:39.142127+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:19.997627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:19.997627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:47.478049+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":127,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T09:27:39.148675+0000","last_change":"2024-09-16T09:26:35.608021+0000","last_active":"2024-09-16T09:27:39.148675+0000","last_peered":"2024-09-16T09:27:39.148675+0000","last_clean":"2024-09-16T09:27:39.148675+0000","last_became_active":"2024-09-16T09:25:07.718778+0000","last_became_peered":"2024-09-16T09:25:07.718778+0000","last_unstale":"2024-09-16T09:27:39.148675+0000","last_undegraded":"2024-09-16T09:27:39.148675+0000","last_fullsized":"2024-09-16T09:27:39.148675+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:35.607956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:35.607956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:04.461371+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"118'7756","reported_seq":7839,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T09:27:42.573413+0000","last_change":"2024-09-16T09:26:08.120869+0000","last_active":"2024-09-16T09:27:42.573413+0000","last_peered":"2024-09-16T09:27:42.573413+0000","last_clean":"2024-09-16T09:27:42.573413+0000","last_became_active":"2024-09-16T09:26:08.120579+0000","last_became_peered":"2024-09-16T09:26:08.120579+0000","last_unstale":"2024-09-16T09:27:42.573413+0000","last_undegraded":"2024-09-16T09:27:42.573413+0000","last_fullsized":"2024-09-16T09:27:42.573413+0000","mapping_epoch":82,"log_start":"90'2400","ondisk_log_start":"90'2400","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_clean_scrub_stamp":"2024-09-16T09:26:07.096764+0000","objects_scrubbed":0,"log_size":5356,"log_dups_size":0,"ondisk_log_size":5356,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:30.609078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":508297216,"num_objects":7756,"num_object_clones":0,"num_object_copies":23268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7756,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7756,"num_write_kb":496384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"66'19","reported_seq":170,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-16T09:27:44.142792+0000","last_change":"2024-09-16T09:26:25.022657+0000","last_active":"2024-09-16T09:27:44.142792+0000","last_peered":"2024-09-16T09:27:44.142792+0000","last_clean":"2024-09-16T09:27:44.142792+0000","last_became_active":"2024-09-16T09:25:03.712783+0000","last_became_peered":"2024-09-16T09:25:03.712783+0000","last_unstale":"2024-09-16T09:27:44.142792+0000","last_undegraded":"2024-09-16T09:27:44.142792+0000","last_fullsized":"2024-09-16T09:27:44.142792+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-16T09:26:25.022598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:26:25.022598+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:49.230254+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"59'1","reported_seq":148,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-16T09:27:44.147263+0000","last_change":"2024-09-16T09:27:17.653505+0000","last_active":"2024-09-16T09:27:44.147263+0000","last_peered":"2024-09-16T09:27:44.147263+0000","last_clean":"2024-09-16T09:27:44.147263+0000","last_became_active":"2024-09-16T09:25:01.708069+0000","last_became_peered":"2024-09-16T09:25:01.708069+0000","last_unstale":"2024-09-16T09:27:44.147263+0000","last_undegraded":"2024-09-16T09:27:44.147263+0000","last_fullsized":"2024-09-16T09:27:44.147263+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:27:17.653425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:27:17.653425+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:38.391486+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":131,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T09:27:39.148874+0000","last_change":"2024-09-16T09:27:11.067059+0000","last_active":"2024-09-16T09:27:39.148874+0000","last_peered":"2024-09-16T09:27:39.148874+0000","last_clean":"2024-09-16T09:27:39.148874+0000","last_became_active":"2024-09-16T09:25:05.709264+0000","last_became_peered":"2024-09-16T09:25:05.709264+0000","last_unstale":"2024-09-16T09:27:39.148874+0000","last_undegraded":"2024-09-16T09:27:39.148874+0000","last_fullsized":"2024-09-16T09:27:39.148874+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:11.066986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:11.066986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:40.568379+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":130,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-16T09:27:44.146588+0000","last_change":"2024-09-16T09:27:25.865107+0000","last_active":"2024-09-16T09:27:44.146588+0000","last_peered":"2024-09-16T09:27:44.146588+0000","last_clean":"2024-09-16T09:27:44.146588+0000","last_became_active":"2024-09-16T09:25:07.718750+0000","last_became_peered":"2024-09-16T09:25:07.718750+0000","last_unstale":"2024-09-16T09:27:44.146588+0000","last_undegraded":"2024-09-16T09:27:44.146588+0000","last_fullsized":"2024-09-16T09:27:44.146588+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:25.865030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:25.865030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:35.295712+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.a","version":"118'7876","reported_seq":7959,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T09:27:42.671038+0000","last_change":"2024-09-16T09:26:08.122847+0000","last_active":"2024-09-16T09:27:42.671038+0000","last_peered":"2024-09-16T09:27:42.671038+0000","last_clean":"2024-09-16T09:27:42.671038+0000","last_became_active":"2024-09-16T09:26:08.122557+0000","last_became_peered":"2024-09-16T09:26:08.122557+0000","last_unstale":"2024-09-16T09:27:42.671038+0000","last_undegraded":"2024-09-16T09:27:42.671038+0000","last_fullsized":"2024-09-16T09:27:42.671038+0000","mapping_epoch":82,"log_start":"97'3500","ondisk_log_start":"97'3500","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_clean_scrub_stamp":"2024-09-16T09:26:07.096764+0000","objects_scrubbed":0,"log_size":4376,"log_dups_size":0,"ondisk_log_size":4376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:34.098109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":516161536,"num_objects":7876,"num_object_clones":0,"num_object_copies":23628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7876,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7876,"num_write_kb":504064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"66'12","reported_seq":162,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-16T09:27:44.135004+0000","last_change":"2024-09-16T09:27:01.771750+0000","last_active":"2024-09-16T09:27:44.135004+0000","last_peered":"2024-09-16T09:27:44.135004+0000","last_clean":"2024-09-16T09:27:44.135004+0000","last_became_active":"2024-09-16T09:25:03.714027+0000","last_became_peered":"2024-09-16T09:25:03.714027+0000","last_unstale":"2024-09-16T09:27:44.135004+0000","last_undegraded":"2024-09-16T09:27:44.135004+0000","last_fullsized":"2024-09-16T09:27:44.135004+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-16T09:27:01.771667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:01.771667+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:04.790349+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":134,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-16T09:27:44.147809+0000","last_change":"2024-09-16T09:25:01.707905+0000","last_active":"2024-09-16T09:27:44.147809+0000","last_peered":"2024-09-16T09:27:44.147809+0000","last_clean":"2024-09-16T09:27:44.147809+0000","last_became_active":"2024-09-16T09:25:01.707528+0000","last_became_peered":"2024-09-16T09:25:01.707528+0000","last_unstale":"2024-09-16T09:27:44.147809+0000","last_undegraded":"2024-09-16T09:27:44.147809+0000","last_fullsized":"2024-09-16T09:27:44.147809+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:31.737341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":131,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T09:27:39.141677+0000","last_change":"2024-09-16T09:26:29.793595+0000","last_active":"2024-09-16T09:27:39.141677+0000","last_peered":"2024-09-16T09:27:39.141677+0000","last_clean":"2024-09-16T09:27:39.141677+0000","last_became_active":"2024-09-16T09:25:05.715253+0000","last_became_peered":"2024-09-16T09:25:05.715253+0000","last_unstale":"2024-09-16T09:27:39.141677+0000","last_undegraded":"2024-09-16T09:27:39.141677+0000","last_fullsized":"2024-09-16T09:27:39.141677+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:29.793522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:26:29.793522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:49.039290+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":119,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T09:27:39.151273+0000","last_change":"2024-09-16T09:25:07.719338+0000","last_active":"2024-09-16T09:27:39.151273+0000","last_peered":"2024-09-16T09:27:39.151273+0000","last_clean":"2024-09-16T09:27:39.151273+0000","last_became_active":"2024-09-16T09:25:07.718899+0000","last_became_peered":"2024-09-16T09:25:07.718899+0000","last_unstale":"2024-09-16T09:27:39.151273+0000","last_undegraded":"2024-09-16T09:27:39.151273+0000","last_fullsized":"2024-09-16T09:27:39.151273+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:51.485861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"118'7725","reported_seq":7880,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T09:27:42.531025+0000","last_change":"2024-09-16T09:27:22.420461+0000","last_active":"2024-09-16T09:27:42.531025+0000","last_peered":"2024-09-16T09:27:42.531025+0000","last_clean":"2024-09-16T09:27:42.531025+0000","last_became_active":"2024-09-16T09:26:08.117869+0000","last_became_peered":"2024-09-16T09:26:08.117869+0000","last_unstale":"2024-09-16T09:27:42.531025+0000","last_undegraded":"2024-09-16T09:27:42.531025+0000","last_fullsized":"2024-09-16T09:27:42.531025+0000","mapping_epoch":82,"log_start":"94'2800","ondisk_log_start":"94'2800","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'6393","last_scrub_stamp":"2024-09-16T09:27:22.420266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_clean_scrub_stamp":"2024-09-16T09:27:22.420266+0000","objects_scrubbed":6319,"log_size":4925,"log_dups_size":0,"ondisk_log_size":4925,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:47.768482+0000","scrub_duration":2145,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":506265600,"num_objects":7725,"num_object_clones":0,"num_object_copies":23175,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7725,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7725,"num_write_kb":494400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"66'15","reported_seq":163,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-16T09:27:44.138287+0000","last_change":"2024-09-16T09:26:33.138828+0000","last_active":"2024-09-16T09:27:44.138287+0000","last_peered":"2024-09-16T09:27:44.138287+0000","last_clean":"2024-09-16T09:27:44.138287+0000","last_became_active":"2024-09-16T09:25:03.715733+0000","last_became_peered":"2024-09-16T09:25:03.715733+0000","last_unstale":"2024-09-16T09:27:44.138287+0000","last_undegraded":"2024-09-16T09:27:44.138287+0000","last_fullsized":"2024-09-16T09:27:44.138287+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-16T09:26:33.138603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:26:33.138603+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:42.149896+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":154,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-16T09:27:44.147330+0000","last_change":"2024-09-16T09:26:26.705847+0000","last_active":"2024-09-16T09:27:44.147330+0000","last_peered":"2024-09-16T09:27:44.147330+0000","last_clean":"2024-09-16T09:27:44.147330+0000","last_became_active":"2024-09-16T09:25:01.707372+0000","last_became_peered":"2024-09-16T09:25:01.707372+0000","last_unstale":"2024-09-16T09:27:44.147330+0000","last_undegraded":"2024-09-16T09:27:44.147330+0000","last_fullsized":"2024-09-16T09:27:44.147330+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-16T09:26:26.705773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:26.705773+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:40.764628+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"66'8","reported_seq":168,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-16T09:27:44.146437+0000","last_change":"2024-09-16T09:25:05.716778+0000","last_active":"2024-09-16T09:27:44.146437+0000","last_peered":"2024-09-16T09:27:44.146437+0000","last_clean":"2024-09-16T09:27:44.146437+0000","last_became_active":"2024-09-16T09:25:05.716189+0000","last_became_peered":"2024-09-16T09:25:05.716189+0000","last_unstale":"2024-09-16T09:27:44.146437+0000","last_undegraded":"2024-09-16T09:27:44.146437+0000","last_fullsized":"2024-09-16T09:27:44.146437+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:47.570349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":130,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-16T09:27:44.138652+0000","last_change":"2024-09-16T09:27:20.219376+0000","last_active":"2024-09-16T09:27:44.138652+0000","last_peered":"2024-09-16T09:27:44.138652+0000","last_clean":"2024-09-16T09:27:44.138652+0000","last_became_active":"2024-09-16T09:25:07.717213+0000","last_became_peered":"2024-09-16T09:25:07.717213+0000","last_unstale":"2024-09-16T09:27:44.138652+0000","last_undegraded":"2024-09-16T09:27:44.138652+0000","last_fullsized":"2024-09-16T09:27:44.138652+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:20.219333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:20.219333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:35.792434+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":143,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-16T09:27:44.142715+0000","last_change":"2024-09-16T09:26:33.139405+0000","last_active":"2024-09-16T09:27:44.142715+0000","last_peered":"2024-09-16T09:27:44.142715+0000","last_clean":"2024-09-16T09:27:44.142715+0000","last_became_active":"2024-09-16T09:25:01.703202+0000","last_became_peered":"2024-09-16T09:25:01.703202+0000","last_unstale":"2024-09-16T09:27:44.142715+0000","last_undegraded":"2024-09-16T09:27:44.142715+0000","last_fullsized":"2024-09-16T09:27:44.142715+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:33.139360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:33.139360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:58.524570+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"66'6","reported_seq":144,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T09:27:39.150786+0000","last_change":"2024-09-16T09:26:23.458888+0000","last_active":"2024-09-16T09:27:39.150786+0000","last_peered":"2024-09-16T09:27:39.150786+0000","last_clean":"2024-09-16T09:27:39.150786+0000","last_became_active":"2024-09-16T09:25:03.713245+0000","last_became_peered":"2024-09-16T09:25:03.713245+0000","last_unstale":"2024-09-16T09:27:39.150786+0000","last_undegraded":"2024-09-16T09:27:39.150786+0000","last_fullsized":"2024-09-16T09:27:39.150786+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'6","last_scrub_stamp":"2024-09-16T09:26:23.458794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:26:23.458794+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:36.044643+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":126,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-16T09:27:44.139010+0000","last_change":"2024-09-16T09:25:05.715938+0000","last_active":"2024-09-16T09:27:44.139010+0000","last_peered":"2024-09-16T09:27:44.139010+0000","last_clean":"2024-09-16T09:27:44.139010+0000","last_became_active":"2024-09-16T09:25:05.715616+0000","last_became_peered":"2024-09-16T09:25:05.715616+0000","last_unstale":"2024-09-16T09:27:44.139010+0000","last_undegraded":"2024-09-16T09:27:44.139010+0000","last_fullsized":"2024-09-16T09:27:44.139010+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:48.184793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":130,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-16T09:27:44.146855+0000","last_change":"2024-09-16T09:26:28.733381+0000","last_active":"2024-09-16T09:27:44.146855+0000","last_peered":"2024-09-16T09:27:44.146855+0000","last_clean":"2024-09-16T09:27:44.146855+0000","last_became_active":"2024-09-16T09:25:07.721059+0000","last_became_peered":"2024-09-16T09:25:07.721059+0000","last_unstale":"2024-09-16T09:27:44.146855+0000","last_undegraded":"2024-09-16T09:27:44.146855+0000","last_fullsized":"2024-09-16T09:27:44.146855+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:28.733324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:28.733324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:50.591599+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"66'9","reported_seq":151,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T09:27:39.158735+0000","last_change":"2024-09-16T09:26:39.554245+0000","last_active":"2024-09-16T09:27:39.158735+0000","last_peered":"2024-09-16T09:27:39.158735+0000","last_clean":"2024-09-16T09:27:39.158735+0000","last_became_active":"2024-09-16T09:25:03.719573+0000","last_became_peered":"2024-09-16T09:25:03.719573+0000","last_unstale":"2024-09-16T09:27:39.158735+0000","last_undegraded":"2024-09-16T09:27:39.158735+0000","last_fullsized":"2024-09-16T09:27:39.158735+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:26:39.554069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:26:39.554069+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:08.727745+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":142,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-16T09:27:44.146682+0000","last_change":"2024-09-16T09:27:24.854485+0000","last_active":"2024-09-16T09:27:44.146682+0000","last_peered":"2024-09-16T09:27:44.146682+0000","last_clean":"2024-09-16T09:27:44.146682+0000","last_became_active":"2024-09-16T09:25:01.706491+0000","last_became_peered":"2024-09-16T09:25:01.706491+0000","last_unstale":"2024-09-16T09:27:44.146682+0000","last_undegraded":"2024-09-16T09:27:44.146682+0000","last_fullsized":"2024-09-16T09:27:44.146682+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:24.854418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:27:24.854418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:49.764094+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":123,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T09:27:39.150582+0000","last_change":"2024-09-16T09:25:05.715083+0000","last_active":"2024-09-16T09:27:39.150582+0000","last_peered":"2024-09-16T09:27:39.150582+0000","last_clean":"2024-09-16T09:27:39.150582+0000","last_became_active":"2024-09-16T09:25:05.714801+0000","last_became_peered":"2024-09-16T09:25:05.714801+0000","last_unstale":"2024-09-16T09:27:39.150582+0000","last_undegraded":"2024-09-16T09:27:39.150582+0000","last_fullsized":"2024-09-16T09:27:39.150582+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:32.055938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":127,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T09:27:39.141704+0000","last_change":"2024-09-16T09:26:35.866582+0000","last_active":"2024-09-16T09:27:39.141704+0000","last_peered":"2024-09-16T09:27:39.141704+0000","last_clean":"2024-09-16T09:27:39.141704+0000","last_became_active":"2024-09-16T09:25:07.722050+0000","last_became_peered":"2024-09-16T09:25:07.722050+0000","last_unstale":"2024-09-16T09:27:39.141704+0000","last_undegraded":"2024-09-16T09:27:39.141704+0000","last_fullsized":"2024-09-16T09:27:39.141704+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:35.866512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:35.866512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:39.028701+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"66'9","reported_seq":146,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-16T09:27:44.138765+0000","last_change":"2024-09-16T09:25:03.717609+0000","last_active":"2024-09-16T09:27:44.138765+0000","last_peered":"2024-09-16T09:27:44.138765+0000","last_clean":"2024-09-16T09:27:44.138765+0000","last_became_active":"2024-09-16T09:25:03.717310+0000","last_became_peered":"2024-09-16T09:25:03.717310+0000","last_unstale":"2024-09-16T09:27:44.138765+0000","last_undegraded":"2024-09-16T09:27:44.138765+0000","last_fullsized":"2024-09-16T09:27:44.138765+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:50.212617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":139,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T09:27:39.150662+0000","last_change":"2024-09-16T09:27:12.652668+0000","last_active":"2024-09-16T09:27:39.150662+0000","last_peered":"2024-09-16T09:27:39.150662+0000","last_clean":"2024-09-16T09:27:39.150662+0000","last_became_active":"2024-09-16T09:25:01.702419+0000","last_became_peered":"2024-09-16T09:25:01.702419+0000","last_unstale":"2024-09-16T09:27:39.150662+0000","last_undegraded":"2024-09-16T09:27:39.150662+0000","last_fullsized":"2024-09-16T09:27:39.150662+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:12.652603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:27:12.652603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:16.848902+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"66'8","reported_seq":172,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T09:27:39.149978+0000","last_change":"2024-09-16T09:27:05.790385+0000","last_active":"2024-09-16T09:27:39.149978+0000","last_peered":"2024-09-16T09:27:39.149978+0000","last_clean":"2024-09-16T09:27:39.149978+0000","last_became_active":"2024-09-16T09:25:05.711642+0000","last_became_peered":"2024-09-16T09:25:05.711642+0000","last_unstale":"2024-09-16T09:27:39.149978+0000","last_undegraded":"2024-09-16T09:27:39.149978+0000","last_fullsized":"2024-09-16T09:27:39.149978+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:27:05.790301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:05.790301+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:25.945999+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":131,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-16T09:27:44.136144+0000","last_change":"2024-09-16T09:26:23.811194+0000","last_active":"2024-09-16T09:27:44.136144+0000","last_peered":"2024-09-16T09:27:44.136144+0000","last_clean":"2024-09-16T09:27:44.136144+0000","last_became_active":"2024-09-16T09:25:07.718810+0000","last_became_peered":"2024-09-16T09:25:07.718810+0000","last_unstale":"2024-09-16T09:27:44.136144+0000","last_undegraded":"2024-09-16T09:27:44.136144+0000","last_fullsized":"2024-09-16T09:27:44.136144+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:23.811127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:23.811127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"66'10","reported_seq":150,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T09:27:39.150287+0000","last_change":"2024-09-16T09:27:02.700022+0000","last_active":"2024-09-16T09:27:39.150287+0000","last_peered":"2024-09-16T09:27:39.150287+0000","last_clean":"2024-09-16T09:27:39.150287+0000","last_became_active":"2024-09-16T09:25:03.712387+0000","last_became_peered":"2024-09-16T09:25:03.712387+0000","last_unstale":"2024-09-16T09:27:39.150287+0000","last_undegraded":"2024-09-16T09:27:39.150287+0000","last_fullsized":"2024-09-16T09:27:39.150287+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-16T09:27:02.699947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:02.699947+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:29.333798+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":134,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-16T09:27:44.146910+0000","last_change":"2024-09-16T09:25:01.708461+0000","last_active":"2024-09-16T09:27:44.146910+0000","last_peered":"2024-09-16T09:27:44.146910+0000","last_clean":"2024-09-16T09:27:44.146910+0000","last_became_active":"2024-09-16T09:25:01.707513+0000","last_became_peered":"2024-09-16T09:25:01.707513+0000","last_unstale":"2024-09-16T09:27:44.146910+0000","last_undegraded":"2024-09-16T09:27:44.146910+0000","last_fullsized":"2024-09-16T09:27:44.146910+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:27.801162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"66'8","reported_seq":176,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-16T09:27:44.138571+0000","last_change":"2024-09-16T09:27:13.395353+0000","last_active":"2024-09-16T09:27:44.138571+0000","last_peered":"2024-09-16T09:27:44.138571+0000","last_clean":"2024-09-16T09:27:44.138571+0000","last_became_active":"2024-09-16T09:25:05.713833+0000","last_became_peered":"2024-09-16T09:25:05.713833+0000","last_unstale":"2024-09-16T09:27:44.138571+0000","last_undegraded":"2024-09-16T09:27:44.138571+0000","last_fullsized":"2024-09-16T09:27:44.138571+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:27:13.395272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:13.395272+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:19.438214+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":119,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T09:27:39.161396+0000","last_change":"2024-09-16T09:25:07.717629+0000","last_active":"2024-09-16T09:27:39.161396+0000","last_peered":"2024-09-16T09:27:39.161396+0000","last_clean":"2024-09-16T09:27:39.161396+0000","last_became_active":"2024-09-16T09:25:07.717357+0000","last_became_peered":"2024-09-16T09:25:07.717357+0000","last_unstale":"2024-09-16T09:27:39.161396+0000","last_undegraded":"2024-09-16T09:27:39.161396+0000","last_fullsized":"2024-09-16T09:27:39.161396+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:44.550344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"66'11","reported_seq":169,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-16T09:27:44.136307+0000","last_change":"2024-09-16T09:26:33.302713+0000","last_active":"2024-09-16T09:27:44.136307+0000","last_peered":"2024-09-16T09:27:44.136307+0000","last_clean":"2024-09-16T09:27:44.136307+0000","last_became_active":"2024-09-16T09:26:33.291265+0000","last_became_peered":"2024-09-16T09:26:33.291265+0000","last_unstale":"2024-09-16T09:27:44.136307+0000","last_undegraded":"2024-09-16T09:27:44.136307+0000","last_fullsized":"2024-09-16T09:27:44.136307+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:41.657340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":143,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-16T09:27:44.137833+0000","last_change":"2024-09-16T09:26:40.610512+0000","last_active":"2024-09-16T09:27:44.137833+0000","last_peered":"2024-09-16T09:27:44.137833+0000","last_clean":"2024-09-16T09:27:44.137833+0000","last_became_active":"2024-09-16T09:25:01.705158+0000","last_became_peered":"2024-09-16T09:25:01.705158+0000","last_unstale":"2024-09-16T09:27:44.137833+0000","last_undegraded":"2024-09-16T09:27:44.137833+0000","last_fullsized":"2024-09-16T09:27:44.137833+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:40.610383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:40.610383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:53.224993+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":131,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T09:27:39.133560+0000","last_change":"2024-09-16T09:26:34.264382+0000","last_active":"2024-09-16T09:27:39.133560+0000","last_peered":"2024-09-16T09:27:39.133560+0000","last_clean":"2024-09-16T09:27:39.133560+0000","last_became_active":"2024-09-16T09:25:05.712816+0000","last_became_peered":"2024-09-16T09:25:05.712816+0000","last_unstale":"2024-09-16T09:27:39.133560+0000","last_undegraded":"2024-09-16T09:27:39.133560+0000","last_fullsized":"2024-09-16T09:27:39.133560+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:34.264197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:26:34.264197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:45.753825+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":127,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T09:27:39.152380+0000","last_change":"2024-09-16T09:26:30.441010+0000","last_active":"2024-09-16T09:27:39.152380+0000","last_peered":"2024-09-16T09:27:39.152380+0000","last_clean":"2024-09-16T09:27:39.152380+0000","last_became_active":"2024-09-16T09:25:07.713109+0000","last_became_peered":"2024-09-16T09:25:07.713109+0000","last_unstale":"2024-09-16T09:27:39.152380+0000","last_undegraded":"2024-09-16T09:27:39.152380+0000","last_fullsized":"2024-09-16T09:27:39.152380+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:30.440945+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:30.440945+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:44.611954+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"66'9","reported_seq":151,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T09:27:39.133791+0000","last_change":"2024-09-16T09:26:31.216495+0000","last_active":"2024-09-16T09:27:39.133791+0000","last_peered":"2024-09-16T09:27:39.133791+0000","last_clean":"2024-09-16T09:27:39.133791+0000","last_became_active":"2024-09-16T09:25:03.705807+0000","last_became_peered":"2024-09-16T09:25:03.705807+0000","last_unstale":"2024-09-16T09:27:39.133791+0000","last_undegraded":"2024-09-16T09:27:39.133791+0000","last_fullsized":"2024-09-16T09:27:39.133791+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:26:31.216298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:26:31.216298+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:52.079512+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":142,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-16T09:27:44.147142+0000","last_change":"2024-09-16T09:26:23.796308+0000","last_active":"2024-09-16T09:27:44.147142+0000","last_peered":"2024-09-16T09:27:44.147142+0000","last_clean":"2024-09-16T09:27:44.147142+0000","last_became_active":"2024-09-16T09:25:01.706762+0000","last_became_peered":"2024-09-16T09:25:01.706762+0000","last_unstale":"2024-09-16T09:27:44.147142+0000","last_undegraded":"2024-09-16T09:27:44.147142+0000","last_fullsized":"2024-09-16T09:27:44.147142+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:23.796231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:23.796231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:38.747701+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":123,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T09:27:39.151474+0000","last_change":"2024-09-16T09:25:05.714142+0000","last_active":"2024-09-16T09:27:39.151474+0000","last_peered":"2024-09-16T09:27:39.151474+0000","last_clean":"2024-09-16T09:27:39.151474+0000","last_became_active":"2024-09-16T09:25:05.713791+0000","last_became_peered":"2024-09-16T09:25:05.713791+0000","last_unstale":"2024-09-16T09:27:39.151474+0000","last_undegraded":"2024-09-16T09:27:39.151474+0000","last_fullsized":"2024-09-16T09:27:39.151474+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:42.381071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":127,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T09:27:39.154468+0000","last_change":"2024-09-16T09:27:21.664905+0000","last_active":"2024-09-16T09:27:39.154468+0000","last_peered":"2024-09-16T09:27:39.154468+0000","last_clean":"2024-09-16T09:27:39.154468+0000","last_became_active":"2024-09-16T09:25:07.715301+0000","last_became_peered":"2024-09-16T09:25:07.715301+0000","last_unstale":"2024-09-16T09:27:39.154468+0000","last_undegraded":"2024-09-16T09:27:39.154468+0000","last_fullsized":"2024-09-16T09:27:39.154468+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:21.664836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:21.664836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:32.409182+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"66'11","reported_seq":154,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T09:27:39.152553+0000","last_change":"2024-09-16T09:27:21.062617+0000","last_active":"2024-09-16T09:27:39.152553+0000","last_peered":"2024-09-16T09:27:39.152553+0000","last_clean":"2024-09-16T09:27:39.152553+0000","last_became_active":"2024-09-16T09:25:03.712333+0000","last_became_peered":"2024-09-16T09:25:03.712333+0000","last_unstale":"2024-09-16T09:27:39.152553+0000","last_undegraded":"2024-09-16T09:27:39.152553+0000","last_fullsized":"2024-09-16T09:27:39.152553+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-16T09:27:21.062564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:21.062564+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:26.644417+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":150,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-16T09:27:46.175159+0000","last_change":"2024-09-16T09:27:46.175159+0000","last_active":"2024-09-16T09:27:46.175159+0000","last_peered":"2024-09-16T09:27:46.175159+0000","last_clean":"2024-09-16T09:27:46.175159+0000","last_became_active":"2024-09-16T09:25:01.700733+0000","last_became_peered":"2024-09-16T09:25:01.700733+0000","last_unstale":"2024-09-16T09:27:46.175159+0000","last_undegraded":"2024-09-16T09:27:46.175159+0000","last_fullsized":"2024-09-16T09:27:46.175159+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:46.175093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:27:46.175093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:15.134231+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":126,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-16T09:27:44.146211+0000","last_change":"2024-09-16T09:25:05.713760+0000","last_active":"2024-09-16T09:27:44.146211+0000","last_peered":"2024-09-16T09:27:44.146211+0000","last_clean":"2024-09-16T09:27:44.146211+0000","last_became_active":"2024-09-16T09:25:05.713479+0000","last_became_peered":"2024-09-16T09:25:05.713479+0000","last_unstale":"2024-09-16T09:27:44.146211+0000","last_undegraded":"2024-09-16T09:27:44.146211+0000","last_fullsized":"2024-09-16T09:27:44.146211+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:49.625090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":127,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T09:27:39.151597+0000","last_change":"2024-09-16T09:26:32.417222+0000","last_active":"2024-09-16T09:27:39.151597+0000","last_peered":"2024-09-16T09:27:39.151597+0000","last_clean":"2024-09-16T09:27:39.151597+0000","last_became_active":"2024-09-16T09:25:07.715582+0000","last_became_peered":"2024-09-16T09:25:07.715582+0000","last_unstale":"2024-09-16T09:27:39.151597+0000","last_undegraded":"2024-09-16T09:27:39.151597+0000","last_fullsized":"2024-09-16T09:27:39.151597+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:32.417150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:32.417150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:53.673620+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"66'4","reported_seq":141,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T09:27:39.157438+0000","last_change":"2024-09-16T09:26:27.474085+0000","last_active":"2024-09-16T09:27:39.157438+0000","last_peered":"2024-09-16T09:27:39.157438+0000","last_clean":"2024-09-16T09:27:39.157438+0000","last_became_active":"2024-09-16T09:25:03.704869+0000","last_became_peered":"2024-09-16T09:25:03.704869+0000","last_unstale":"2024-09-16T09:27:39.157438+0000","last_undegraded":"2024-09-16T09:27:39.157438+0000","last_fullsized":"2024-09-16T09:27:39.157438+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'4","last_scrub_stamp":"2024-09-16T09:26:27.473740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:26:27.473740+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:56.231643+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":139,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T09:27:39.164784+0000","last_change":"2024-09-16T09:26:23.494773+0000","last_active":"2024-09-16T09:27:39.164784+0000","last_peered":"2024-09-16T09:27:39.164784+0000","last_clean":"2024-09-16T09:27:39.164784+0000","last_became_active":"2024-09-16T09:25:01.705686+0000","last_became_peered":"2024-09-16T09:25:01.705686+0000","last_unstale":"2024-09-16T09:27:39.164784+0000","last_undegraded":"2024-09-16T09:27:39.164784+0000","last_fullsized":"2024-09-16T09:27:39.164784+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:23.494642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:23.494642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:29.637091+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":135,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-16T09:27:44.142556+0000","last_change":"2024-09-16T09:27:13.279354+0000","last_active":"2024-09-16T09:27:44.142556+0000","last_peered":"2024-09-16T09:27:44.142556+0000","last_clean":"2024-09-16T09:27:44.142556+0000","last_became_active":"2024-09-16T09:25:05.712545+0000","last_became_peered":"2024-09-16T09:25:05.712545+0000","last_unstale":"2024-09-16T09:27:44.142556+0000","last_undegraded":"2024-09-16T09:27:44.142556+0000","last_fullsized":"2024-09-16T09:27:44.142556+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:13.279303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:13.279303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:21.165370+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":122,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-16T09:27:44.148012+0000","last_change":"2024-09-16T09:25:07.720368+0000","last_active":"2024-09-16T09:27:44.148012+0000","last_peered":"2024-09-16T09:27:44.148012+0000","last_clean":"2024-09-16T09:27:44.148012+0000","last_became_active":"2024-09-16T09:25:07.720117+0000","last_became_peered":"2024-09-16T09:25:07.720117+0000","last_unstale":"2024-09-16T09:27:44.148012+0000","last_undegraded":"2024-09-16T09:27:44.148012+0000","last_fullsized":"2024-09-16T09:27:44.148012+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:50.589550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":119,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T09:27:39.134432+0000","last_change":"2024-09-16T09:25:07.712880+0000","last_active":"2024-09-16T09:27:39.134432+0000","last_peered":"2024-09-16T09:27:39.134432+0000","last_clean":"2024-09-16T09:27:39.134432+0000","last_became_active":"2024-09-16T09:25:07.712594+0000","last_became_peered":"2024-09-16T09:25:07.712594+0000","last_unstale":"2024-09-16T09:27:39.134432+0000","last_undegraded":"2024-09-16T09:27:39.134432+0000","last_fullsized":"2024-09-16T09:27:39.134432+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:50.161435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":131,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T09:27:39.157720+0000","last_change":"2024-09-16T09:25:01.704711+0000","last_active":"2024-09-16T09:27:39.157720+0000","last_peered":"2024-09-16T09:27:39.157720+0000","last_clean":"2024-09-16T09:27:39.157720+0000","last_became_active":"2024-09-16T09:25:01.703736+0000","last_became_peered":"2024-09-16T09:25:01.703736+0000","last_unstale":"2024-09-16T09:27:39.157720+0000","last_undegraded":"2024-09-16T09:27:39.157720+0000","last_fullsized":"2024-09-16T09:27:39.157720+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:31.100509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"66'11","reported_seq":158,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-16T09:27:44.143485+0000","last_change":"2024-09-16T09:26:24.090440+0000","last_active":"2024-09-16T09:27:44.143485+0000","last_peered":"2024-09-16T09:27:44.143485+0000","last_clean":"2024-09-16T09:27:44.143485+0000","last_became_active":"2024-09-16T09:25:03.707769+0000","last_became_peered":"2024-09-16T09:25:03.707769+0000","last_unstale":"2024-09-16T09:27:44.143485+0000","last_undegraded":"2024-09-16T09:27:44.143485+0000","last_fullsized":"2024-09-16T09:27:44.143485+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-16T09:26:24.090355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:26:24.090355+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":123,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-16T09:27:39.158506+0000","last_change":"2024-09-16T09:25:05.712247+0000","last_active":"2024-09-16T09:27:39.158506+0000","last_peered":"2024-09-16T09:27:39.158506+0000","last_clean":"2024-09-16T09:27:39.158506+0000","last_became_active":"2024-09-16T09:25:05.711952+0000","last_became_peered":"2024-09-16T09:25:05.711952+0000","last_unstale":"2024-09-16T09:27:39.158506+0000","last_undegraded":"2024-09-16T09:27:39.158506+0000","last_fullsized":"2024-09-16T09:27:39.158506+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:45.578865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":16,"stat_sum":{"num_bytes":8118140952,"num_objects":123874,"num_object_clones":0,"num_object_copies":371622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":123874,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":127806,"num_write_kb":8053697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24340795392,"data_stored":24340791320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83906,"ondisk_log_size":83906,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":393,"ondisk_log_size":393,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":160,"num_read_kb":155,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":11,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":869,"ondisk_log_size":869,"up":33,"acting":33,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266726,"num_pgs":62,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2519912,"kb_used_data":2456792,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":91225240,"statfs":{"total":95995035648,"available":93414645760,"internally_reserved":0,"allocated":2515755008,"data_stored":2515080690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":64616906},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495659,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2479764,"kb_used_data":2404164,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":91265388,"statfs":{"total":95995035648,"available":93455757312,"internally_reserved":0,"allocated":2461863936,"data_stored":2461209024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":77396422},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724594,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2001180,"kb_used_data":1938060,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":91743972,"statfs":{"total":95995035648,"available":93945827328,"internally_reserved":0,"allocated":1984573440,"data_stored":1983911325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64616902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953527,"num_pgs":66,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2509164,"kb_used_data":2433564,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":91235988,"statfs":{"total":95995035648,"available":93425651712,"internally_reserved":0,"allocated":2491969536,"data_stored":2491291186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":77396422},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149756,"num_pgs":70,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":4626720,"kb_used_data":4523344,"kb_used_omap":1,"kb_used_meta":103358,"kb_avail":89118432,"statfs":{"total":95995035648,"available":91257274368,"internally_reserved":0,"allocated":4631904256,"data_stored":4631222878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":105839047},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,4,2,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411393,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3094204,"kb_used_data":3026924,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":90650948,"statfs":{"total":95995035648,"available":92826570752,"internally_reserved":0,"allocated":3099570176,"data_stored":3098924146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,1,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574920,"num_pgs":61,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3055276,"kb_used_data":2987996,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":90689876,"statfs":{"total":95995035648,"available":92866433024,"internally_reserved":0,"allocated":3059707904,"data_stored":3059013278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738446,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":4122388,"kb_used_data":4019012,"kb_used_omap":1,"kb_used_meta":103358,"kb_avail":89622764,"statfs":{"total":95995035648,"available":91773710336,"internally_reserved":0,"allocated":4115468288,"data_stored":4114798570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":105839045},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4116512768,"data_stored":4116512768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3068854272,"data_stored":3068854272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3110076416,"data_stored":3110076416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4648337408,"data_stored":4648337408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":2469462016,"data_stored":2469462016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1976893440,"data_stored":1976893440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":2456817664,"data_stored":2456813592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":2493841408,"data_stored":2493841408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T09:27:49.437 INFO:tasks.cephadm.ceph_manager.ceph:would drop below min_pgs, proj 6, currently 16 2024-09-16T09:27:49.446 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:27:49.498 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:49.498 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:49.498 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:27:49.568 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:27:49.576 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:27:49.600 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:27:49.624 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:49.625 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:49.626 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:27:49.642 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:27:49.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:49 smithi084 ceph-mon[32629]: pgmap v118: 158 pgs: 158 active+clean; 7.6 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.4 KiB/s rd, 41 MiB/s wr, 909 op/s 2024-09-16T09:27:49.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:49 smithi084 ceph-mon[32629]: from='client.24769 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:27:49.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:49 smithi084 ceph-mon[32629]: 6.11 scrub starts 2024-09-16T09:27:49.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:49 smithi084 ceph-mon[32629]: 6.11 scrub ok 2024-09-16T09:27:49.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:49 smithi084 ceph-mon[32629]: 5.1a scrub starts 2024-09-16T09:27:49.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:49 smithi084 ceph-mon[32629]: 5.1a scrub ok 2024-09-16T09:27:49.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:49 smithi084 ceph-mon[32629]: 5.6 scrub starts 2024-09-16T09:27:49.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:49 smithi084 ceph-mon[32629]: 5.6 scrub ok 2024-09-16T09:27:49.673 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:49 smithi070 ceph-mon[30388]: pgmap v118: 158 pgs: 158 active+clean; 7.6 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.4 KiB/s rd, 41 MiB/s wr, 909 op/s 2024-09-16T09:27:49.673 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:49 smithi070 ceph-mon[30388]: from='client.24769 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:27:49.673 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:49 smithi070 ceph-mon[30388]: 6.11 scrub starts 2024-09-16T09:27:49.673 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:49 smithi070 ceph-mon[30388]: 6.11 scrub ok 2024-09-16T09:27:49.673 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:49 smithi070 ceph-mon[30388]: 5.1a scrub starts 2024-09-16T09:27:49.673 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:49 smithi070 ceph-mon[30388]: 5.1a scrub ok 2024-09-16T09:27:49.673 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:49 smithi070 ceph-mon[30388]: 5.6 scrub starts 2024-09-16T09:27:49.673 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:49 smithi070 ceph-mon[30388]: 5.6 scrub ok 2024-09-16T09:27:49.674 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:49 smithi070 ceph-mon[40691]: pgmap v118: 158 pgs: 158 active+clean; 7.6 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.4 KiB/s rd, 41 MiB/s wr, 909 op/s 2024-09-16T09:27:49.674 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:49 smithi070 ceph-mon[40691]: from='client.24769 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:27:49.674 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:49 smithi070 ceph-mon[40691]: 6.11 scrub starts 2024-09-16T09:27:49.674 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:49 smithi070 ceph-mon[40691]: 6.11 scrub ok 2024-09-16T09:27:49.674 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:49 smithi070 ceph-mon[40691]: 5.1a scrub starts 2024-09-16T09:27:49.674 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:49 smithi070 ceph-mon[40691]: 5.1a scrub ok 2024-09-16T09:27:49.674 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:49 smithi070 ceph-mon[40691]: 5.6 scrub starts 2024-09-16T09:27:49.674 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:49 smithi070 ceph-mon[40691]: 5.6 scrub ok 2024-09-16T09:27:49.727 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:27:49.755 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:49.756 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:49.757 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:27:49.857 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:27:49.900 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:49.901 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:49.901 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:27:50.002 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:27:50.030 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:50.030 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:50.031 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:27:50.131 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:27:50.173 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:50.174 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:50.174 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:27:50.275 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:27:50.303 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:50.303 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:50.304 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:27:50.405 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:27:50.444 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:50.445 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:50.446 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:27:50.546 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:27:50.588 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:50.589 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:50.589 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:27:50.599 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:50 smithi070 ceph-mon[30388]: 3.2 scrub starts 2024-09-16T09:27:50.600 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:50 smithi070 ceph-mon[30388]: 3.2 scrub ok 2024-09-16T09:27:50.600 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:50 smithi070 ceph-mon[30388]: 5.18 scrub starts 2024-09-16T09:27:50.600 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:50 smithi070 ceph-mon[30388]: 5.18 scrub ok 2024-09-16T09:27:50.600 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:50 smithi070 ceph-mon[30388]: 5.8 scrub starts 2024-09-16T09:27:50.600 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:50 smithi070 ceph-mon[30388]: 5.8 scrub ok 2024-09-16T09:27:50.600 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:50 smithi070 ceph-mon[40691]: 3.2 scrub starts 2024-09-16T09:27:50.600 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:50 smithi070 ceph-mon[40691]: 3.2 scrub ok 2024-09-16T09:27:50.600 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:50 smithi070 ceph-mon[40691]: 5.18 scrub starts 2024-09-16T09:27:50.600 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:50 smithi070 ceph-mon[40691]: 5.18 scrub ok 2024-09-16T09:27:50.600 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:50 smithi070 ceph-mon[40691]: 5.8 scrub starts 2024-09-16T09:27:50.600 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:50 smithi070 ceph-mon[40691]: 5.8 scrub ok 2024-09-16T09:27:50.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:50 smithi084 ceph-mon[32629]: 3.2 scrub starts 2024-09-16T09:27:50.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:50 smithi084 ceph-mon[32629]: 3.2 scrub ok 2024-09-16T09:27:50.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:50 smithi084 ceph-mon[32629]: 5.18 scrub starts 2024-09-16T09:27:50.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:50 smithi084 ceph-mon[32629]: 5.18 scrub ok 2024-09-16T09:27:50.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:50 smithi084 ceph-mon[32629]: 5.8 scrub starts 2024-09-16T09:27:50.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:50 smithi084 ceph-mon[32629]: 5.8 scrub ok 2024-09-16T09:27:50.690 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:27:50.709 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:27:50.710 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:27:50.727 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:50.727 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:50.728 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:27:50.728 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:27:50.729 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:27:50.787 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:27:50.787 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:27:50.829 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:27:50.834 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:27:50.834 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:27:50.855 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:50.855 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:50.856 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:27:50.872 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:27:50.872 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:27:50.914 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:27:50.914 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:27:50.956 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:27:50.962 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:27:50.962 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:27:50.995 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:50.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:50.996 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:27:50.997 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:27:50.997 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:27:51.097 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:27:51.122 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:51.123 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:51.123 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:27:51.224 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:27:51.259 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:51.259 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:51.260 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:27:51.303 INFO:teuthology.orchestra.run.smithi070.stderr:nodeep-scrub is unset 2024-09-16T09:27:51.361 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:27:51.400 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:51.401 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:51.402 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:27:51.502 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:27:51.528 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:51.529 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:51.529 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:27:51.630 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:27:51.658 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:51.658 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:51.658 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:27:51.659 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:51 smithi084 ceph-mon[32629]: pgmap v119: 158 pgs: 158 active+clean; 7.4 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.4 KiB/s rd, 44 MiB/s wr, 1.36k op/s 2024-09-16T09:27:51.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:51 smithi084 ceph-mon[32629]: 4.4 scrub starts 2024-09-16T09:27:51.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:51 smithi084 ceph-mon[32629]: 4.4 scrub ok 2024-09-16T09:27:51.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:51 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1710988926' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:27:51.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:51 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/2146581902' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:27:51.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:51 smithi070 ceph-mon[30388]: pgmap v119: 158 pgs: 158 active+clean; 7.4 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.4 KiB/s rd, 44 MiB/s wr, 1.36k op/s 2024-09-16T09:27:51.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:51 smithi070 ceph-mon[30388]: 4.4 scrub starts 2024-09-16T09:27:51.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:51 smithi070 ceph-mon[30388]: 4.4 scrub ok 2024-09-16T09:27:51.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:51 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1710988926' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:27:51.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:51 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2146581902' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:27:51.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:51 smithi070 ceph-mon[40691]: pgmap v119: 158 pgs: 158 active+clean; 7.4 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.4 KiB/s rd, 44 MiB/s wr, 1.36k op/s 2024-09-16T09:27:51.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:51 smithi070 ceph-mon[40691]: 4.4 scrub starts 2024-09-16T09:27:51.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:51 smithi070 ceph-mon[40691]: 4.4 scrub ok 2024-09-16T09:27:51.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:51 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1710988926' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:27:51.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:51 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/2146581902' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:27:51.760 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:27:51.787 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:51.787 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:51.788 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:27:51.799 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T09:27:51.800 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [ 2024-09-16T09:27:51.800 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:51.800 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.24698.0:135469 7.9 7:935e4602:::benchmark_data_smithi070_78438_object9338:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120)", 2024-09-16T09:27:51.800 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:27:51.222671+0000", 2024-09-16T09:27:51.800 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.001461429, 2024-09-16T09:27:51.800 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001496544, 2024-09-16T09:27:51.800 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:27:51.800 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:27:51.800 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "waiting for sub ops", 2024-09-16T09:27:51.800 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:27:51.800 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.24698", 2024-09-16T09:27:51.800 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:27:51.801 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 135469 2024-09-16T09:27:51.801 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:51.801 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:27:51.801 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:51.801 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:27:51.801 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:51.222671+0000", 2024-09-16T09:27:51.801 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:27:51.801 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:51.801 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:51.801 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:27:51.801 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:51.222671+0000", 2024-09-16T09:27:51.801 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:27:51.801 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:51.802 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:51.802 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:27:51.802 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:51.222672+0000", 2024-09-16T09:27:51.802 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6789999999999999e-06 2024-09-16T09:27:51.802 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:51.802 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:51.802 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:27:51.802 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:51.222678+0000", 2024-09-16T09:27:51.802 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7209999999999999e-06 2024-09-16T09:27:51.802 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:51.802 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:51.802 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:27:51.802 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:51.222696+0000", 2024-09-16T09:27:51.802 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.8559e-05 2024-09-16T09:27:51.803 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:51.803 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:51.803 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:27:51.803 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:51.222706+0000", 2024-09-16T09:27:51.803 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.2760000000000006e-06 2024-09-16T09:27:51.803 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:51.803 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:51.803 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:27:51.803 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:51.222793+0000", 2024-09-16T09:27:51.803 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.7263999999999997e-05 2024-09-16T09:27:51.803 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:51.803 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:51.803 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:27:51.803 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:51.223037+0000", 2024-09-16T09:27:51.804 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00024350700000000001 2024-09-16T09:27:51.804 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:51.804 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:51.804 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,4", 2024-09-16T09:27:51.804 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:51.223138+0000", 2024-09-16T09:27:51.804 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00010175000000000001 2024-09-16T09:27:51.804 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:27:51.804 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:27:51.804 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:27:51.804 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:51.804 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:51.804 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.24698.0:135470 7.f 7.3a1df54f (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e120)", 2024-09-16T09:27:51.805 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:27:51.222842+0000", 2024-09-16T09:27:51.805 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.001290097, 2024-09-16T09:27:51.805 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0013662399999999999, 2024-09-16T09:27:51.805 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:27:51.805 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:27:51.805 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T09:27:51.805 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:27:51.805 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.24698", 2024-09-16T09:27:51.805 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:27:51.805 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 135470 2024-09-16T09:27:51.805 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:51.805 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:27:51.805 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:51.805 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:27:51.806 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:51.222842+0000", 2024-09-16T09:27:51.806 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:27:51.806 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:51.806 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:51.806 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:27:51.806 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:51.222842+0000", 2024-09-16T09:27:51.806 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:27:51.806 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:51.806 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:51.806 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:27:51.806 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:51.222843+0000", 2024-09-16T09:27:51.806 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.5499999999999997e-07 2024-09-16T09:27:51.806 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:51.806 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:51.807 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:27:51.807 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:51.222847+0000", 2024-09-16T09:27:51.807 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8840000000000004e-06 2024-09-16T09:27:51.807 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:51.807 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:51.807 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:27:51.807 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:51.222855+0000", 2024-09-16T09:27:51.807 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.5250000000000005e-06 2024-09-16T09:27:51.807 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:27:51.807 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:27:51.807 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:27:51.807 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:51.222862+0000", 2024-09-16T09:27:51.807 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.4139999999999999e-06 2024-09-16T09:27:51.807 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:27:51.807 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:27:51.808 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:27:51.808 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:27:51.808 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-16T09:27:51.808 INFO:teuthology.orchestra.run.smithi070.stdout: "num_ops": 2 2024-09-16T09:27:51.808 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T09:27:51.888 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:27:51.931 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:51.931 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:51.932 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:27:52.033 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:27:52.057 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:52.058 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:52.058 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:27:52.150 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:27:51 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:27:51] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:27:52.158 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:27:52.186 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:52.187 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:52.187 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:27:52.288 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:27:52.314 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:52.315 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:52.315 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:27:52.416 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:27:52.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:52 smithi070 ceph-mon[30388]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-16T09:27:52.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:52 smithi070 ceph-mon[30388]: Cluster is now healthy 2024-09-16T09:27:52.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:52 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2146581902' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-16T09:27:52.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:52 smithi070 ceph-mon[30388]: osdmap e121: 8 total, 8 up, 8 in 2024-09-16T09:27:52.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:52 smithi070 ceph-mon[30388]: 5.1f scrub starts 2024-09-16T09:27:52.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:52 smithi070 ceph-mon[30388]: 5.1f scrub ok 2024-09-16T09:27:52.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:52 smithi070 ceph-mon[30388]: 6.16 scrub starts 2024-09-16T09:27:52.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:52 smithi070 ceph-mon[30388]: 6.16 scrub ok 2024-09-16T09:27:52.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:52 smithi070 ceph-mon[30388]: 5.9 scrub starts 2024-09-16T09:27:52.422 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:52 smithi070 ceph-mon[30388]: 5.9 scrub ok 2024-09-16T09:27:52.422 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:52 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:27:52.422 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:52 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:27:52.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:52 smithi070 ceph-mon[40691]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-16T09:27:52.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:52 smithi070 ceph-mon[40691]: Cluster is now healthy 2024-09-16T09:27:52.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:52 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/2146581902' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-16T09:27:52.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:52 smithi070 ceph-mon[40691]: osdmap e121: 8 total, 8 up, 8 in 2024-09-16T09:27:52.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:52 smithi070 ceph-mon[40691]: 5.1f scrub starts 2024-09-16T09:27:52.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:52 smithi070 ceph-mon[40691]: 5.1f scrub ok 2024-09-16T09:27:52.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:52 smithi070 ceph-mon[40691]: 6.16 scrub starts 2024-09-16T09:27:52.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:52 smithi070 ceph-mon[40691]: 6.16 scrub ok 2024-09-16T09:27:52.423 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:52 smithi070 ceph-mon[40691]: 5.9 scrub starts 2024-09-16T09:27:52.423 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:52 smithi070 ceph-mon[40691]: 5.9 scrub ok 2024-09-16T09:27:52.423 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:52 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:27:52.423 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:52 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:27:52.455 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:52.456 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:52.456 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:27:52.557 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:27:52.588 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:52.590 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:52.590 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:27:52.642 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:52 smithi084 ceph-mon[32629]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-16T09:27:52.642 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:52 smithi084 ceph-mon[32629]: Cluster is now healthy 2024-09-16T09:27:52.642 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:52 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/2146581902' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-16T09:27:52.642 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:52 smithi084 ceph-mon[32629]: osdmap e121: 8 total, 8 up, 8 in 2024-09-16T09:27:52.642 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:52 smithi084 ceph-mon[32629]: 5.1f scrub starts 2024-09-16T09:27:52.642 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:52 smithi084 ceph-mon[32629]: 5.1f scrub ok 2024-09-16T09:27:52.643 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:52 smithi084 ceph-mon[32629]: 6.16 scrub starts 2024-09-16T09:27:52.643 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:52 smithi084 ceph-mon[32629]: 6.16 scrub ok 2024-09-16T09:27:52.643 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:52 smithi084 ceph-mon[32629]: 5.9 scrub starts 2024-09-16T09:27:52.643 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:52 smithi084 ceph-mon[32629]: 5.9 scrub ok 2024-09-16T09:27:52.643 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:52 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:27:52.643 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:52 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:27:52.691 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:27:52.721 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:52.722 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:52.722 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:27:52.823 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:27:52.850 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:52.850 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:52.851 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:27:52.909 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:27:52 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:27:52.952 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:27:52.995 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:52.995 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:52.996 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:27:53.096 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:27:53.129 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:53.130 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:53.131 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:27:53.231 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:27:53.259 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:53.259 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:53.260 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:27:53.361 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:27:53.400 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:53.400 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:53.401 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:27:53.502 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:27:53.532 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:53.533 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:53.533 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:27:53.634 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:27:53.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:53 smithi084 ceph-mon[32629]: pgmap v121: 158 pgs: 158 active+clean; 7.4 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 767 B/s rd, 3.8 MiB/s wr, 720 op/s 2024-09-16T09:27:53.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:53 smithi084 ceph-mon[32629]: 6.1d scrub starts 2024-09-16T09:27:53.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:53 smithi084 ceph-mon[32629]: 6.1d scrub ok 2024-09-16T09:27:53.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:53 smithi084 ceph-mon[32629]: osdmap e122: 8 total, 8 up, 8 in 2024-09-16T09:27:53.667 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:53.667 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:53.668 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:27:53.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:53 smithi070 ceph-mon[30388]: pgmap v121: 158 pgs: 158 active+clean; 7.4 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 767 B/s rd, 3.8 MiB/s wr, 720 op/s 2024-09-16T09:27:53.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:53 smithi070 ceph-mon[30388]: 6.1d scrub starts 2024-09-16T09:27:53.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:53 smithi070 ceph-mon[30388]: 6.1d scrub ok 2024-09-16T09:27:53.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:53 smithi070 ceph-mon[30388]: osdmap e122: 8 total, 8 up, 8 in 2024-09-16T09:27:53.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:53 smithi070 ceph-mon[40691]: pgmap v121: 158 pgs: 158 active+clean; 7.4 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 767 B/s rd, 3.8 MiB/s wr, 720 op/s 2024-09-16T09:27:53.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:53 smithi070 ceph-mon[40691]: 6.1d scrub starts 2024-09-16T09:27:53.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:53 smithi070 ceph-mon[40691]: 6.1d scrub ok 2024-09-16T09:27:53.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:53 smithi070 ceph-mon[40691]: osdmap e122: 8 total, 8 up, 8 in 2024-09-16T09:27:53.769 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:27:53.815 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:53.816 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:53.816 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:27:53.886 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.0 dump_blocked_ops 2024-09-16T09:27:53.917 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:27:53.971 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:53.972 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:53.972 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:27:54.073 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:27:54.098 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:54.099 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:54.099 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:27:54.200 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:27:54.227 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:54.227 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:54.228 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:27:54.328 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:27:54.343 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:27:54.355 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:54.355 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:54.356 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:27:54.438 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T09:27:54.438 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T09:27:54.439 INFO:tasks.thrashosds.thrasher:Shrinking pool unique_pool_0 2024-09-16T09:27:54.439 INFO:tasks.cephadm.ceph_manager.ceph:contract_pool unique_pool_0 by 10 min 8 2024-09-16T09:27:54.439 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-16T09:27:54.456 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:27:54.485 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:54.485 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:54.486 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:27:54.586 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:27:54.613 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:54.614 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:54.614 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:27:54.640 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:54 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:27:54.640 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:54 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:27:54.659 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:54 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:27:54.716 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:27:54.743 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:54.744 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:54.744 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:27:54.845 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:27:54.892 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:54.893 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:54.893 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:27:54.901 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T09:27:54.994 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:27:55.017 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:55.018 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:55.018 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:27:55.042 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T09:27:55.042 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T09:27:55.042 INFO:teuthology.orchestra.run.smithi070.stdout: "complaint_time": 30, 2024-09-16T09:27:55.042 INFO:teuthology.orchestra.run.smithi070.stdout: "num_blocked_ops": 0 2024-09-16T09:27:55.043 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T09:27:55.119 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:27:55.165 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:55.166 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:55.166 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:27:55.267 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:27:55.292 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:55.293 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:55.293 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:27:55.327 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:27:55.394 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:27:55.434 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd set noscrub 2024-09-16T09:27:55.445 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:55.445 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:55.445 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:27:55.546 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:27:55.574 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:55.574 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:55.575 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:27:55.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:55 smithi084 ceph-mon[32629]: pgmap v123: 158 pgs: 158 active+clean; 7.0 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.4 KiB/s rd, 3.8 MiB/s wr, 1.44k op/s 2024-09-16T09:27:55.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:55 smithi084 ceph-mon[32629]: 5.c scrub starts 2024-09-16T09:27:55.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:55 smithi084 ceph-mon[32629]: 5.c scrub ok 2024-09-16T09:27:55.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:55 smithi084 ceph-mon[32629]: 3.1e scrub starts 2024-09-16T09:27:55.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:55 smithi084 ceph-mon[32629]: 3.1e scrub ok 2024-09-16T09:27:55.670 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:55 smithi070 ceph-mon[40691]: pgmap v123: 158 pgs: 158 active+clean; 7.0 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.4 KiB/s rd, 3.8 MiB/s wr, 1.44k op/s 2024-09-16T09:27:55.670 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:55 smithi070 ceph-mon[40691]: 5.c scrub starts 2024-09-16T09:27:55.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:55 smithi070 ceph-mon[40691]: 5.c scrub ok 2024-09-16T09:27:55.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:55 smithi070 ceph-mon[40691]: 3.1e scrub starts 2024-09-16T09:27:55.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:55 smithi070 ceph-mon[40691]: 3.1e scrub ok 2024-09-16T09:27:55.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:55 smithi070 ceph-mon[30388]: pgmap v123: 158 pgs: 158 active+clean; 7.0 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.4 KiB/s rd, 3.8 MiB/s wr, 1.44k op/s 2024-09-16T09:27:55.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:55 smithi070 ceph-mon[30388]: 5.c scrub starts 2024-09-16T09:27:55.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:55 smithi070 ceph-mon[30388]: 5.c scrub ok 2024-09-16T09:27:55.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:55 smithi070 ceph-mon[30388]: 3.1e scrub starts 2024-09-16T09:27:55.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:55 smithi070 ceph-mon[30388]: 3.1e scrub ok 2024-09-16T09:27:55.676 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:27:55.699 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:55.700 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:55.701 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:27:55.801 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:27:55.851 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:55.852 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:55.853 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:27:55.931 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.0 dump_historic_ops 2024-09-16T09:27:55.953 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:27:55.978 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:55.978 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:55.979 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:27:56.079 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:27:56.105 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:56.106 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:56.106 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:27:56.207 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:27:56.233 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:56.234 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:56.234 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:27:56.335 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:27:56.362 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:56.363 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:56.364 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:27:56.464 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:27:56.474 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:56 smithi070 ceph-mon[30388]: 5.16 scrub starts 2024-09-16T09:27:56.474 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:56 smithi070 ceph-mon[30388]: 5.16 scrub ok 2024-09-16T09:27:56.474 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:56 smithi070 ceph-mon[40691]: 5.16 scrub starts 2024-09-16T09:27:56.475 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:56 smithi070 ceph-mon[40691]: 5.16 scrub ok 2024-09-16T09:27:56.491 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:56.491 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:56.492 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:27:56.593 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:27:56.602 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:27:56.604 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:27:56.617 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:56.617 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:56.618 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:27:56.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:56 smithi084 ceph-mon[32629]: 5.16 scrub starts 2024-09-16T09:27:56.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:56 smithi084 ceph-mon[32629]: 5.16 scrub ok 2024-09-16T09:27:56.719 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:27:56.758 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:56.759 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:56.759 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:27:56.860 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:27:56.884 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:56.884 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:56.885 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:27:56.986 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:27:57.019 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:27:57.022 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2024-09-16T09:27:57.031 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:57.032 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:57.032 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:27:57.133 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:27:57.178 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:57.179 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:57.179 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:27:57.280 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:27:57.296 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:27:57.296 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:27:57.317 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:57.317 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:57.317 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:27:57.339 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:27:57.339 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:27:57.372 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:27:57.372 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:27:57.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:57 smithi084 ceph-mon[32629]: pgmap v124: 158 pgs: 158 active+clean; 7.0 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 639 B/s rd, 2.3 MiB/s wr, 1.17k op/s 2024-09-16T09:27:57.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:57 smithi084 ceph-mon[32629]: 6.b scrub starts 2024-09-16T09:27:57.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:57 smithi084 ceph-mon[32629]: 6.b scrub ok 2024-09-16T09:27:57.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:57 smithi084 ceph-mon[32629]: 3.1c scrub starts 2024-09-16T09:27:57.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:57 smithi084 ceph-mon[32629]: 3.1c scrub ok 2024-09-16T09:27:57.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:57 smithi084 ceph-mon[32629]: 5.10 scrub starts 2024-09-16T09:27:57.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:57 smithi084 ceph-mon[32629]: 5.10 scrub ok 2024-09-16T09:27:57.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:57 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3124664238' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:27:57.414 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:27:57.414 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:27:57.418 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:27:57.441 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:57.442 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:57.442 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:27:57.445 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:27:57.445 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:27:57.476 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:27:57.476 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:27:57.508 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:27:57.508 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:27:57.540 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:27:57.541 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:27:57.543 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:27:57.557 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:57 smithi070 ceph-mon[30388]: pgmap v124: 158 pgs: 158 active+clean; 7.0 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 639 B/s rd, 2.3 MiB/s wr, 1.17k op/s 2024-09-16T09:27:57.558 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:57 smithi070 ceph-mon[30388]: 6.b scrub starts 2024-09-16T09:27:57.558 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:57 smithi070 ceph-mon[30388]: 6.b scrub ok 2024-09-16T09:27:57.558 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:57 smithi070 ceph-mon[30388]: 3.1c scrub starts 2024-09-16T09:27:57.558 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:57 smithi070 ceph-mon[30388]: 3.1c scrub ok 2024-09-16T09:27:57.559 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:57 smithi070 ceph-mon[30388]: 5.10 scrub starts 2024-09-16T09:27:57.559 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:57 smithi070 ceph-mon[30388]: 5.10 scrub ok 2024-09-16T09:27:57.559 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:57 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3124664238' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:27:57.559 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:57 smithi070 ceph-mon[40691]: pgmap v124: 158 pgs: 158 active+clean; 7.0 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 639 B/s rd, 2.3 MiB/s wr, 1.17k op/s 2024-09-16T09:27:57.559 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:57 smithi070 ceph-mon[40691]: 6.b scrub starts 2024-09-16T09:27:57.560 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:57 smithi070 ceph-mon[40691]: 6.b scrub ok 2024-09-16T09:27:57.560 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:57 smithi070 ceph-mon[40691]: 3.1c scrub starts 2024-09-16T09:27:57.560 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:57 smithi070 ceph-mon[40691]: 3.1c scrub ok 2024-09-16T09:27:57.560 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:57 smithi070 ceph-mon[40691]: 5.10 scrub starts 2024-09-16T09:27:57.560 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:57 smithi070 ceph-mon[40691]: 5.10 scrub ok 2024-09-16T09:27:57.561 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:57 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3124664238' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:27:57.568 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:57.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:57.570 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:27:57.671 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:27:57.699 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:57.700 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:57.700 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:27:57.801 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:27:57.830 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:57.831 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:57.831 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:27:57.932 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:27:57.959 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:57.959 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:57.960 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:27:58.060 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:27:58.087 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:58.088 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:58.088 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:27:58.189 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:27:58.215 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:58.216 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:58.216 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:27:58.317 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:27:58.351 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:58.351 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:58.352 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:27:58.452 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:27:58.490 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:58.491 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:58.491 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:27:58.592 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:27:58.630 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:58.630 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:58.631 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:27:58.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:58 smithi084 ceph-mon[32629]: from='client.24860 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:27:58.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:58 smithi084 ceph-mon[32629]: 1.3 scrub starts 2024-09-16T09:27:58.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:58 smithi084 ceph-mon[32629]: 1.3 scrub ok 2024-09-16T09:27:58.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:58 smithi084 ceph-mon[32629]: 4.15 scrub starts 2024-09-16T09:27:58.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:58 smithi084 ceph-mon[32629]: 4.15 scrub ok 2024-09-16T09:27:58.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:58 smithi084 ceph-mon[32629]: 4.1f scrub starts 2024-09-16T09:27:58.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:58 smithi084 ceph-mon[32629]: 4.1f scrub ok 2024-09-16T09:27:58.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:58 smithi084 ceph-mon[32629]: 5.1e scrub starts 2024-09-16T09:27:58.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:58 smithi084 ceph-mon[32629]: 5.1e scrub ok 2024-09-16T09:27:58.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:58 smithi070 ceph-mon[30388]: from='client.24860 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:27:58.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:58 smithi070 ceph-mon[30388]: 1.3 scrub starts 2024-09-16T09:27:58.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:58 smithi070 ceph-mon[30388]: 1.3 scrub ok 2024-09-16T09:27:58.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:58 smithi070 ceph-mon[30388]: 4.15 scrub starts 2024-09-16T09:27:58.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:58 smithi070 ceph-mon[30388]: 4.15 scrub ok 2024-09-16T09:27:58.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:58 smithi070 ceph-mon[30388]: 4.1f scrub starts 2024-09-16T09:27:58.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:58 smithi070 ceph-mon[30388]: 4.1f scrub ok 2024-09-16T09:27:58.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:58 smithi070 ceph-mon[30388]: 5.1e scrub starts 2024-09-16T09:27:58.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:58 smithi070 ceph-mon[30388]: 5.1e scrub ok 2024-09-16T09:27:58.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:58 smithi070 ceph-mon[40691]: from='client.24860 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:27:58.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:58 smithi070 ceph-mon[40691]: 1.3 scrub starts 2024-09-16T09:27:58.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:58 smithi070 ceph-mon[40691]: 1.3 scrub ok 2024-09-16T09:27:58.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:58 smithi070 ceph-mon[40691]: 4.15 scrub starts 2024-09-16T09:27:58.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:58 smithi070 ceph-mon[40691]: 4.15 scrub ok 2024-09-16T09:27:58.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:58 smithi070 ceph-mon[40691]: 4.1f scrub starts 2024-09-16T09:27:58.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:58 smithi070 ceph-mon[40691]: 4.1f scrub ok 2024-09-16T09:27:58.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:58 smithi070 ceph-mon[40691]: 5.1e scrub starts 2024-09-16T09:27:58.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:58 smithi070 ceph-mon[40691]: 5.1e scrub ok 2024-09-16T09:27:58.731 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:27:58.766 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:58.767 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:58.767 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:27:58.868 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:27:58.909 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:58.910 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:58.911 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:27:59.012 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:27:59.047 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:59.048 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:59.048 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:27:59.149 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:27:59.182 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:59.182 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:59.183 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:27:59.283 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":124,"stamp":"2024-09-16T09:27:55.528236+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":7541824051,"num_objects":115270,"num_object_clones":0,"num_object_copies":345810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":115270,"num_whiteouts":0,"num_read":888,"num_read_kb":640,"num_write":137623,"num_write_kb":8072238,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":85029,"ondisk_log_size":85029,"up":474,"acting":474,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":474,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":22878644,"kb_used_data":22226768,"kb_used_omap":12,"kb_used_meta":650547,"kb_avail":727082572,"statfs":{"total":767960285184,"available":744532553728,"internally_reserved":0,"allocated":22760210432,"data_stored":22755630317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":666160690},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,5,9,4,4],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-576913408,"num_objects":-8803,"num_object_clones":0,"num_object_copies":-26409,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-8803,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":9381,"num_write_kb":18496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002432"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":147,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.321090+0000","last_change":"2024-09-16T09:27:07.621760+0000","last_active":"2024-09-16T09:27:52.321090+0000","last_peered":"2024-09-16T09:27:52.321090+0000","last_clean":"2024-09-16T09:27:52.321090+0000","last_became_active":"2024-09-16T09:25:01.709108+0000","last_became_peered":"2024-09-16T09:25:01.709108+0000","last_unstale":"2024-09-16T09:27:52.321090+0000","last_undegraded":"2024-09-16T09:27:52.321090+0000","last_fullsized":"2024-09-16T09:27:52.321090+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:07.621692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:27:07.621692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:08.877709+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"66'9","reported_seq":159,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.309786+0000","last_change":"2024-09-16T09:27:45.630451+0000","last_active":"2024-09-16T09:27:52.309786+0000","last_peered":"2024-09-16T09:27:52.309786+0000","last_clean":"2024-09-16T09:27:52.309786+0000","last_became_active":"2024-09-16T09:25:03.714635+0000","last_became_peered":"2024-09-16T09:25:03.714635+0000","last_unstale":"2024-09-16T09:27:52.309786+0000","last_undegraded":"2024-09-16T09:27:52.309786+0000","last_fullsized":"2024-09-16T09:27:52.309786+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:27:45.630310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:45.630310+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:46.857234+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":139,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.308529+0000","last_change":"2024-09-16T09:26:41.241860+0000","last_active":"2024-09-16T09:27:52.308529+0000","last_peered":"2024-09-16T09:27:52.308529+0000","last_clean":"2024-09-16T09:27:52.308529+0000","last_became_active":"2024-09-16T09:25:05.715645+0000","last_became_peered":"2024-09-16T09:25:05.715645+0000","last_unstale":"2024-09-16T09:27:52.308529+0000","last_undegraded":"2024-09-16T09:27:52.308529+0000","last_fullsized":"2024-09-16T09:27:52.308529+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:41.241640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:26:41.241640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":135,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.309885+0000","last_change":"2024-09-16T09:27:11.565306+0000","last_active":"2024-09-16T09:27:52.309885+0000","last_peered":"2024-09-16T09:27:52.309885+0000","last_clean":"2024-09-16T09:27:52.309885+0000","last_became_active":"2024-09-16T09:25:07.715442+0000","last_became_peered":"2024-09-16T09:25:07.715442+0000","last_unstale":"2024-09-16T09:27:52.309885+0000","last_undegraded":"2024-09-16T09:27:52.309885+0000","last_fullsized":"2024-09-16T09:27:52.309885+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:11.565220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:11.565220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:16.605980+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":127,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.317127+0000","last_change":"2024-09-16T09:25:07.718687+0000","last_active":"2024-09-16T09:27:52.317127+0000","last_peered":"2024-09-16T09:27:52.317127+0000","last_clean":"2024-09-16T09:27:52.317127+0000","last_became_active":"2024-09-16T09:25:07.718126+0000","last_became_peered":"2024-09-16T09:25:07.718126+0000","last_unstale":"2024-09-16T09:27:52.317127+0000","last_undegraded":"2024-09-16T09:27:52.317127+0000","last_fullsized":"2024-09-16T09:27:52.317127+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:54.549850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":139,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.318136+0000","last_change":"2024-09-16T09:25:01.708505+0000","last_active":"2024-09-16T09:27:52.318136+0000","last_peered":"2024-09-16T09:27:52.318136+0000","last_clean":"2024-09-16T09:27:52.318136+0000","last_became_active":"2024-09-16T09:25:01.708179+0000","last_became_peered":"2024-09-16T09:25:01.708179+0000","last_unstale":"2024-09-16T09:27:52.318136+0000","last_undegraded":"2024-09-16T09:27:52.318136+0000","last_fullsized":"2024-09-16T09:27:52.318136+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"66'15","reported_seq":168,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.322120+0000","last_change":"2024-09-16T09:27:09.887336+0000","last_active":"2024-09-16T09:27:52.322120+0000","last_peered":"2024-09-16T09:27:52.322120+0000","last_clean":"2024-09-16T09:27:52.322120+0000","last_became_active":"2024-09-16T09:25:03.709499+0000","last_became_peered":"2024-09-16T09:25:03.709499+0000","last_unstale":"2024-09-16T09:27:52.322120+0000","last_undegraded":"2024-09-16T09:27:52.322120+0000","last_fullsized":"2024-09-16T09:27:52.322120+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-16T09:27:09.887267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:09.887267+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:18.495993+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":147,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.309699+0000","last_change":"2024-09-16T09:27:49.653058+0000","last_active":"2024-09-16T09:27:52.309699+0000","last_peered":"2024-09-16T09:27:52.309699+0000","last_clean":"2024-09-16T09:27:52.309699+0000","last_became_active":"2024-09-16T09:25:05.717236+0000","last_became_peered":"2024-09-16T09:25:05.717236+0000","last_unstale":"2024-09-16T09:27:52.309699+0000","last_undegraded":"2024-09-16T09:27:52.309699+0000","last_fullsized":"2024-09-16T09:27:52.309699+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:49.653015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:49.653015+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:09.644746+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":146,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.314918+0000","last_change":"2024-09-16T09:26:23.041770+0000","last_active":"2024-09-16T09:27:52.314918+0000","last_peered":"2024-09-16T09:27:52.314918+0000","last_clean":"2024-09-16T09:27:52.314918+0000","last_became_active":"2024-09-16T09:25:01.705269+0000","last_became_peered":"2024-09-16T09:25:01.705269+0000","last_unstale":"2024-09-16T09:27:52.314918+0000","last_undegraded":"2024-09-16T09:27:52.314918+0000","last_fullsized":"2024-09-16T09:27:52.314918+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:23.041687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:23.041687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":28,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"66'9","reported_seq":151,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.310148+0000","last_change":"2024-09-16T09:25:03.709441+0000","last_active":"2024-09-16T09:27:52.310148+0000","last_peered":"2024-09-16T09:27:52.310148+0000","last_clean":"2024-09-16T09:27:52.310148+0000","last_became_active":"2024-09-16T09:25:03.709147+0000","last_became_peered":"2024-09-16T09:25:03.709147+0000","last_unstale":"2024-09-16T09:27:52.310148+0000","last_undegraded":"2024-09-16T09:27:52.310148+0000","last_fullsized":"2024-09-16T09:27:52.310148+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":130,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.315219+0000","last_change":"2024-09-16T09:25:05.718510+0000","last_active":"2024-09-16T09:27:52.315219+0000","last_peered":"2024-09-16T09:27:52.315219+0000","last_clean":"2024-09-16T09:27:52.315219+0000","last_became_active":"2024-09-16T09:25:05.718134+0000","last_became_peered":"2024-09-16T09:25:05.718134+0000","last_unstale":"2024-09-16T09:27:52.315219+0000","last_undegraded":"2024-09-16T09:27:52.315219+0000","last_fullsized":"2024-09-16T09:27:52.315219+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:55.212380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"66'1","reported_seq":136,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.320732+0000","last_change":"2024-09-16T09:26:24.547980+0000","last_active":"2024-09-16T09:27:52.320732+0000","last_peered":"2024-09-16T09:27:52.320732+0000","last_clean":"2024-09-16T09:27:52.320732+0000","last_became_active":"2024-09-16T09:25:07.716558+0000","last_became_peered":"2024-09-16T09:25:07.716558+0000","last_unstale":"2024-09-16T09:27:52.320732+0000","last_undegraded":"2024-09-16T09:27:52.320732+0000","last_fullsized":"2024-09-16T09:27:52.320732+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-16T09:26:24.547877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:24.547877+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":52,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":138,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.314743+0000","last_change":"2024-09-16T09:25:01.697306+0000","last_active":"2024-09-16T09:27:52.314743+0000","last_peered":"2024-09-16T09:27:52.314743+0000","last_clean":"2024-09-16T09:27:52.314743+0000","last_became_active":"2024-09-16T09:25:01.696939+0000","last_became_peered":"2024-09-16T09:25:01.696939+0000","last_unstale":"2024-09-16T09:27:52.314743+0000","last_undegraded":"2024-09-16T09:27:52.314743+0000","last_fullsized":"2024-09-16T09:27:52.314743+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"66'5","reported_seq":153,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.309998+0000","last_change":"2024-09-16T09:27:05.694733+0000","last_active":"2024-09-16T09:27:52.309998+0000","last_peered":"2024-09-16T09:27:52.309998+0000","last_clean":"2024-09-16T09:27:52.309998+0000","last_became_active":"2024-09-16T09:25:03.716136+0000","last_became_peered":"2024-09-16T09:25:03.716136+0000","last_unstale":"2024-09-16T09:27:52.309998+0000","last_undegraded":"2024-09-16T09:27:52.309998+0000","last_fullsized":"2024-09-16T09:27:52.309998+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-16T09:27:05.694650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:05.694650+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:24.201532+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":138,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.321276+0000","last_change":"2024-09-16T09:27:48.682578+0000","last_active":"2024-09-16T09:27:52.321276+0000","last_peered":"2024-09-16T09:27:52.321276+0000","last_clean":"2024-09-16T09:27:52.321276+0000","last_became_active":"2024-09-16T09:25:05.718281+0000","last_became_peered":"2024-09-16T09:25:05.718281+0000","last_unstale":"2024-09-16T09:27:52.321276+0000","last_undegraded":"2024-09-16T09:27:52.321276+0000","last_fullsized":"2024-09-16T09:27:52.321276+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:48.682528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:48.682528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:52.128525+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":134,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.314512+0000","last_change":"2024-09-16T09:27:01.285768+0000","last_active":"2024-09-16T09:27:52.314512+0000","last_peered":"2024-09-16T09:27:52.314512+0000","last_clean":"2024-09-16T09:27:52.314512+0000","last_became_active":"2024-09-16T09:25:07.716724+0000","last_became_peered":"2024-09-16T09:25:07.716724+0000","last_unstale":"2024-09-16T09:27:52.314512+0000","last_undegraded":"2024-09-16T09:27:52.314512+0000","last_fullsized":"2024-09-16T09:27:52.314512+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:01.285712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:01.285712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:23.640748+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":135,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.310733+0000","last_change":"2024-09-16T09:26:27.758217+0000","last_active":"2024-09-16T09:27:52.310733+0000","last_peered":"2024-09-16T09:27:52.310733+0000","last_clean":"2024-09-16T09:27:52.310733+0000","last_became_active":"2024-09-16T09:25:07.720793+0000","last_became_peered":"2024-09-16T09:25:07.720793+0000","last_unstale":"2024-09-16T09:27:52.310733+0000","last_undegraded":"2024-09-16T09:27:52.310733+0000","last_fullsized":"2024-09-16T09:27:52.310733+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:27.758151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:27.758151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":147,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.321169+0000","last_change":"2024-09-16T09:27:09.635443+0000","last_active":"2024-09-16T09:27:52.321169+0000","last_peered":"2024-09-16T09:27:52.321169+0000","last_clean":"2024-09-16T09:27:52.321169+0000","last_became_active":"2024-09-16T09:25:01.706172+0000","last_became_peered":"2024-09-16T09:25:01.706172+0000","last_unstale":"2024-09-16T09:27:52.321169+0000","last_undegraded":"2024-09-16T09:27:52.321169+0000","last_fullsized":"2024-09-16T09:27:52.321169+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:09.635363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:27:09.635363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:25.991704+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"66'15","reported_seq":168,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.319670+0000","last_change":"2024-09-16T09:26:33.873905+0000","last_active":"2024-09-16T09:27:52.319670+0000","last_peered":"2024-09-16T09:27:52.319670+0000","last_clean":"2024-09-16T09:27:52.319670+0000","last_became_active":"2024-09-16T09:25:03.713780+0000","last_became_peered":"2024-09-16T09:25:03.713780+0000","last_unstale":"2024-09-16T09:27:52.319670+0000","last_undegraded":"2024-09-16T09:27:52.319670+0000","last_fullsized":"2024-09-16T09:27:52.319670+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-16T09:26:33.873817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:26:33.873817+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:54.702210+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":139,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.311804+0000","last_change":"2024-09-16T09:27:16.266378+0000","last_active":"2024-09-16T09:27:52.311804+0000","last_peered":"2024-09-16T09:27:52.311804+0000","last_clean":"2024-09-16T09:27:52.311804+0000","last_became_active":"2024-09-16T09:25:05.713929+0000","last_became_peered":"2024-09-16T09:25:05.713929+0000","last_unstale":"2024-09-16T09:27:52.311804+0000","last_undegraded":"2024-09-16T09:27:52.311804+0000","last_fullsized":"2024-09-16T09:27:52.311804+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:16.266297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:16.266297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:38.874875+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":135,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.317508+0000","last_change":"2024-09-16T09:26:39.358604+0000","last_active":"2024-09-16T09:27:52.317508+0000","last_peered":"2024-09-16T09:27:52.317508+0000","last_clean":"2024-09-16T09:27:52.317508+0000","last_became_active":"2024-09-16T09:25:07.719067+0000","last_became_peered":"2024-09-16T09:25:07.719067+0000","last_unstale":"2024-09-16T09:27:52.317508+0000","last_undegraded":"2024-09-16T09:27:52.317508+0000","last_fullsized":"2024-09-16T09:27:52.317508+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:39.358091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:39.358091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:00.182589+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":147,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.310273+0000","last_change":"2024-09-16T09:26:32.756725+0000","last_active":"2024-09-16T09:27:52.310273+0000","last_peered":"2024-09-16T09:27:52.310273+0000","last_clean":"2024-09-16T09:27:52.310273+0000","last_became_active":"2024-09-16T09:25:01.706491+0000","last_became_peered":"2024-09-16T09:25:01.706491+0000","last_unstale":"2024-09-16T09:27:52.310273+0000","last_undegraded":"2024-09-16T09:27:52.310273+0000","last_fullsized":"2024-09-16T09:27:52.310273+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:32.756657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:32.756657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:55.818360+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"66'12","reported_seq":166,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.320326+0000","last_change":"2024-09-16T09:27:01.700219+0000","last_active":"2024-09-16T09:27:52.320326+0000","last_peered":"2024-09-16T09:27:52.320326+0000","last_clean":"2024-09-16T09:27:52.320326+0000","last_became_active":"2024-09-16T09:25:03.709610+0000","last_became_peered":"2024-09-16T09:25:03.709610+0000","last_unstale":"2024-09-16T09:27:52.320326+0000","last_undegraded":"2024-09-16T09:27:52.320326+0000","last_fullsized":"2024-09-16T09:27:52.320326+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-16T09:27:01.700139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:01.700139+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:28.256362+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":139,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.310625+0000","last_change":"2024-09-16T09:27:00.816500+0000","last_active":"2024-09-16T09:27:52.310625+0000","last_peered":"2024-09-16T09:27:52.310625+0000","last_clean":"2024-09-16T09:27:52.310625+0000","last_became_active":"2024-09-16T09:25:05.710093+0000","last_became_peered":"2024-09-16T09:25:05.710093+0000","last_unstale":"2024-09-16T09:27:52.310625+0000","last_undegraded":"2024-09-16T09:27:52.310625+0000","last_fullsized":"2024-09-16T09:27:52.310625+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:00.816424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:00.816424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:16.736124+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":127,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.321434+0000","last_change":"2024-09-16T09:25:07.719655+0000","last_active":"2024-09-16T09:27:52.321434+0000","last_peered":"2024-09-16T09:27:52.321434+0000","last_clean":"2024-09-16T09:27:52.321434+0000","last_became_active":"2024-09-16T09:25:07.719380+0000","last_became_peered":"2024-09-16T09:25:07.719380+0000","last_unstale":"2024-09-16T09:27:52.321434+0000","last_undegraded":"2024-09-16T09:27:52.321434+0000","last_fullsized":"2024-09-16T09:27:52.321434+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":148,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.308457+0000","last_change":"2024-09-16T09:27:08.225591+0000","last_active":"2024-09-16T09:27:52.308457+0000","last_peered":"2024-09-16T09:27:52.308457+0000","last_clean":"2024-09-16T09:27:52.308457+0000","last_became_active":"2024-09-16T09:25:01.699545+0000","last_became_peered":"2024-09-16T09:25:01.699545+0000","last_unstale":"2024-09-16T09:27:52.308457+0000","last_undegraded":"2024-09-16T09:27:52.308457+0000","last_fullsized":"2024-09-16T09:27:52.308457+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:27:08.225363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:27:08.225363+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:16.086554+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"66'10","reported_seq":166,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.332194+0000","last_change":"2024-09-16T09:27:43.556435+0000","last_active":"2024-09-16T09:27:52.332194+0000","last_peered":"2024-09-16T09:27:52.332194+0000","last_clean":"2024-09-16T09:27:52.332194+0000","last_became_active":"2024-09-16T09:25:03.720245+0000","last_became_peered":"2024-09-16T09:25:03.720245+0000","last_unstale":"2024-09-16T09:27:52.332194+0000","last_undegraded":"2024-09-16T09:27:52.332194+0000","last_fullsized":"2024-09-16T09:27:52.332194+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-16T09:27:43.556357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:43.556357+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:13.459515+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"66'8","reported_seq":193,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.313510+0000","last_change":"2024-09-16T09:27:51.374575+0000","last_active":"2024-09-16T09:27:52.313510+0000","last_peered":"2024-09-16T09:27:52.313510+0000","last_clean":"2024-09-16T09:27:52.313510+0000","last_became_active":"2024-09-16T09:25:05.712516+0000","last_became_peered":"2024-09-16T09:25:05.712516+0000","last_unstale":"2024-09-16T09:27:52.313510+0000","last_undegraded":"2024-09-16T09:27:52.313510+0000","last_fullsized":"2024-09-16T09:27:52.313510+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:27:51.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:51.374504+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:58.529957+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.a","version":"73'79","reported_seq":74,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.314595+0000","last_change":"2024-09-16T09:26:36.169945+0000","last_active":"2024-09-16T09:27:52.314595+0000","last_peered":"2024-09-16T09:27:52.314595+0000","last_clean":"2024-09-16T09:27:52.314595+0000","last_became_active":"2024-09-16T09:26:35.127352+0000","last_became_peered":"2024-09-16T09:26:35.127352+0000","last_unstale":"2024-09-16T09:27:52.314595+0000","last_undegraded":"2024-09-16T09:27:52.314595+0000","last_fullsized":"2024-09-16T09:27:52.314595+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:26:36.169870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:26:36.169870+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:00.141321+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.c","version":"122'8501","reported_seq":8590,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.501913+0000","last_change":"2024-09-16T09:26:08.117066+0000","last_active":"2024-09-16T09:27:52.501913+0000","last_peered":"2024-09-16T09:27:52.501913+0000","last_clean":"2024-09-16T09:27:52.501913+0000","last_became_active":"2024-09-16T09:26:08.116783+0000","last_became_peered":"2024-09-16T09:26:08.116783+0000","last_unstale":"2024-09-16T09:27:52.501913+0000","last_undegraded":"2024-09-16T09:27:52.501913+0000","last_fullsized":"2024-09-16T09:27:52.501913+0000","mapping_epoch":82,"log_start":"94'2800","ondisk_log_start":"94'2800","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_clean_scrub_stamp":"2024-09-16T09:26:07.096764+0000","objects_scrubbed":0,"log_size":5701,"log_dups_size":0,"ondisk_log_size":5701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:32.278987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":469434368,"num_objects":7163,"num_object_clones":0,"num_object_copies":21489,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7163,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8501,"num_write_kb":501248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.f","version":"66'15","reported_seq":86,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.320081+0000","last_change":"2024-09-16T09:26:33.348884+0000","last_active":"2024-09-16T09:27:52.320081+0000","last_peered":"2024-09-16T09:27:52.320081+0000","last_clean":"2024-09-16T09:27:52.320081+0000","last_became_active":"2024-09-16T09:26:33.295672+0000","last_became_peered":"2024-09-16T09:26:33.295672+0000","last_unstale":"2024-09-16T09:27:52.320081+0000","last_undegraded":"2024-09-16T09:27:52.320081+0000","last_fullsized":"2024-09-16T09:27:52.320081+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-16T09:26:23.191684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:26:23.191684+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:27.974437+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":147,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.322240+0000","last_change":"2024-09-16T09:26:40.364509+0000","last_active":"2024-09-16T09:27:52.322240+0000","last_peered":"2024-09-16T09:27:52.322240+0000","last_clean":"2024-09-16T09:27:52.322240+0000","last_became_active":"2024-09-16T09:25:01.702629+0000","last_became_peered":"2024-09-16T09:25:01.702629+0000","last_unstale":"2024-09-16T09:27:52.322240+0000","last_undegraded":"2024-09-16T09:27:52.322240+0000","last_fullsized":"2024-09-16T09:27:52.322240+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:40.364442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:40.364442+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:02.622633+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"66'8","reported_seq":174,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.310809+0000","last_change":"2024-09-16T09:25:05.714128+0000","last_active":"2024-09-16T09:27:52.310809+0000","last_peered":"2024-09-16T09:27:52.310809+0000","last_clean":"2024-09-16T09:27:52.310809+0000","last_became_active":"2024-09-16T09:25:05.713813+0000","last_became_peered":"2024-09-16T09:25:05.713813+0000","last_unstale":"2024-09-16T09:27:52.310809+0000","last_undegraded":"2024-09-16T09:27:52.310809+0000","last_fullsized":"2024-09-16T09:27:52.310809+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:57.826189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":134,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.314664+0000","last_change":"2024-09-16T09:26:38.289448+0000","last_active":"2024-09-16T09:27:52.314664+0000","last_peered":"2024-09-16T09:27:52.314664+0000","last_clean":"2024-09-16T09:27:52.314664+0000","last_became_active":"2024-09-16T09:25:07.717344+0000","last_became_peered":"2024-09-16T09:25:07.717344+0000","last_unstale":"2024-09-16T09:27:52.314664+0000","last_undegraded":"2024-09-16T09:27:52.314664+0000","last_fullsized":"2024-09-16T09:27:52.314664+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:38.289387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:38.289387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:01.979399+0000","scrub_duration":114,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.5","version":"73'79","reported_seq":74,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.311032+0000","last_change":"2024-09-16T09:26:35.687824+0000","last_active":"2024-09-16T09:27:52.311032+0000","last_peered":"2024-09-16T09:27:52.311032+0000","last_clean":"2024-09-16T09:27:52.311032+0000","last_became_active":"2024-09-16T09:26:35.117714+0000","last_became_peered":"2024-09-16T09:26:35.117714+0000","last_unstale":"2024-09-16T09:27:52.311032+0000","last_undegraded":"2024-09-16T09:27:52.311032+0000","last_fullsized":"2024-09-16T09:27:52.311032+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:26:35.687778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:26:35.687778+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:04.588420+0000","scrub_duration":25,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3","version":"122'8691","reported_seq":8782,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.694684+0000","last_change":"2024-09-16T09:26:08.119864+0000","last_active":"2024-09-16T09:27:52.694684+0000","last_peered":"2024-09-16T09:27:52.694684+0000","last_clean":"2024-09-16T09:27:52.694684+0000","last_became_active":"2024-09-16T09:26:08.119550+0000","last_became_peered":"2024-09-16T09:26:08.119550+0000","last_unstale":"2024-09-16T09:27:52.694684+0000","last_undegraded":"2024-09-16T09:27:52.694684+0000","last_fullsized":"2024-09-16T09:27:52.694684+0000","mapping_epoch":82,"log_start":"100'4400","ondisk_log_start":"100'4400","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_clean_scrub_stamp":"2024-09-16T09:26:07.096764+0000","objects_scrubbed":0,"log_size":4291,"log_dups_size":0,"ondisk_log_size":4291,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:33.053037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":474546176,"num_objects":7241,"num_object_clones":0,"num_object_copies":21723,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7241,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8691,"num_write_kb":509824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4],"acting":[3,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"66'18","reported_seq":175,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.322022+0000","last_change":"2024-09-16T09:27:08.857567+0000","last_active":"2024-09-16T09:27:52.322022+0000","last_peered":"2024-09-16T09:27:52.322022+0000","last_clean":"2024-09-16T09:27:52.322022+0000","last_became_active":"2024-09-16T09:25:03.714208+0000","last_became_peered":"2024-09-16T09:25:03.714208+0000","last_unstale":"2024-09-16T09:27:52.322022+0000","last_undegraded":"2024-09-16T09:27:52.322022+0000","last_fullsized":"2024-09-16T09:27:52.322022+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'18","last_scrub_stamp":"2024-09-16T09:27:08.857484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:08.857484+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:31.036637+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":155,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.318027+0000","last_change":"2024-09-16T09:27:46.208398+0000","last_active":"2024-09-16T09:27:52.318027+0000","last_peered":"2024-09-16T09:27:52.318027+0000","last_clean":"2024-09-16T09:27:52.318027+0000","last_became_active":"2024-09-16T09:25:01.704994+0000","last_became_peered":"2024-09-16T09:25:01.704994+0000","last_unstale":"2024-09-16T09:27:52.318027+0000","last_undegraded":"2024-09-16T09:27:52.318027+0000","last_fullsized":"2024-09-16T09:27:52.318027+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:46.208293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:27:46.208293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:00.017707+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":139,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.310246+0000","last_change":"2024-09-16T09:26:33.729827+0000","last_active":"2024-09-16T09:27:52.310246+0000","last_peered":"2024-09-16T09:27:52.310246+0000","last_clean":"2024-09-16T09:27:52.310246+0000","last_became_active":"2024-09-16T09:25:05.713360+0000","last_became_peered":"2024-09-16T09:25:05.713360+0000","last_unstale":"2024-09-16T09:27:52.310246+0000","last_undegraded":"2024-09-16T09:27:52.310246+0000","last_fullsized":"2024-09-16T09:27:52.310246+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:33.729770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:26:33.729770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:54.598643+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":135,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.310066+0000","last_change":"2024-09-16T09:27:13.548667+0000","last_active":"2024-09-16T09:27:52.310066+0000","last_peered":"2024-09-16T09:27:52.310066+0000","last_clean":"2024-09-16T09:27:52.310066+0000","last_became_active":"2024-09-16T09:25:07.719159+0000","last_became_peered":"2024-09-16T09:25:07.719159+0000","last_unstale":"2024-09-16T09:27:52.310066+0000","last_undegraded":"2024-09-16T09:27:52.310066+0000","last_fullsized":"2024-09-16T09:27:52.310066+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:13.548608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:13.548608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:20.672287+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"73'79","reported_seq":79,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.317638+0000","last_change":"2024-09-16T09:26:35.426510+0000","last_active":"2024-09-16T09:27:52.317638+0000","last_peered":"2024-09-16T09:27:52.317638+0000","last_clean":"2024-09-16T09:27:52.317638+0000","last_became_active":"2024-09-16T09:26:35.131343+0000","last_became_peered":"2024-09-16T09:26:35.131343+0000","last_unstale":"2024-09-16T09:27:52.317638+0000","last_undegraded":"2024-09-16T09:27:52.317638+0000","last_fullsized":"2024-09-16T09:27:52.317638+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:26:35.426313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:26:35.426313+0000","objects_scrubbed":1,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:02.968954+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2","version":"122'8631","reported_seq":8722,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.689135+0000","last_change":"2024-09-16T09:26:08.120627+0000","last_active":"2024-09-16T09:27:52.689135+0000","last_peered":"2024-09-16T09:27:52.689135+0000","last_clean":"2024-09-16T09:27:52.689135+0000","last_became_active":"2024-09-16T09:26:08.120338+0000","last_became_peered":"2024-09-16T09:26:08.120338+0000","last_unstale":"2024-09-16T09:27:52.689135+0000","last_undegraded":"2024-09-16T09:27:52.689135+0000","last_fullsized":"2024-09-16T09:27:52.689135+0000","mapping_epoch":82,"log_start":"100'4300","ondisk_log_start":"100'4300","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_clean_scrub_stamp":"2024-09-16T09:26:07.096764+0000","objects_scrubbed":0,"log_size":4331,"log_dups_size":0,"ondisk_log_size":4331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:38.116462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":477036544,"num_objects":7279,"num_object_clones":0,"num_object_copies":21837,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7279,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8631,"num_write_kb":509120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"66'14","reported_seq":164,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.310320+0000","last_change":"2024-09-16T09:26:38.595468+0000","last_active":"2024-09-16T09:27:52.310320+0000","last_peered":"2024-09-16T09:27:52.310320+0000","last_clean":"2024-09-16T09:27:52.310320+0000","last_became_active":"2024-09-16T09:25:03.708830+0000","last_became_peered":"2024-09-16T09:25:03.708830+0000","last_unstale":"2024-09-16T09:27:52.310320+0000","last_undegraded":"2024-09-16T09:27:52.310320+0000","last_fullsized":"2024-09-16T09:27:52.310320+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'14","last_scrub_stamp":"2024-09-16T09:26:38.595391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:26:38.595391+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:56.981868+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":156,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.332340+0000","last_change":"2024-09-16T09:27:14.637291+0000","last_active":"2024-09-16T09:27:52.332340+0000","last_peered":"2024-09-16T09:27:52.332340+0000","last_clean":"2024-09-16T09:27:52.332340+0000","last_became_active":"2024-09-16T09:25:01.702992+0000","last_became_peered":"2024-09-16T09:25:01.702992+0000","last_unstale":"2024-09-16T09:27:52.332340+0000","last_undegraded":"2024-09-16T09:27:52.332340+0000","last_fullsized":"2024-09-16T09:27:52.332340+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:27:14.637226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:27:14.637226+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:43.815503+0000","scrub_duration":37,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"66'8","reported_seq":174,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.321765+0000","last_change":"2024-09-16T09:25:05.715927+0000","last_active":"2024-09-16T09:27:52.321765+0000","last_peered":"2024-09-16T09:27:52.321765+0000","last_clean":"2024-09-16T09:27:52.321765+0000","last_became_active":"2024-09-16T09:25:05.715606+0000","last_became_peered":"2024-09-16T09:25:05.715606+0000","last_unstale":"2024-09-16T09:27:52.321765+0000","last_undegraded":"2024-09-16T09:27:52.321765+0000","last_fullsized":"2024-09-16T09:27:52.321765+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:57.590084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":134,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.321338+0000","last_change":"2024-09-16T09:26:40.699888+0000","last_active":"2024-09-16T09:27:52.321338+0000","last_peered":"2024-09-16T09:27:52.321338+0000","last_clean":"2024-09-16T09:27:52.321338+0000","last_became_active":"2024-09-16T09:25:07.722017+0000","last_became_peered":"2024-09-16T09:25:07.722017+0000","last_unstale":"2024-09-16T09:27:52.321338+0000","last_undegraded":"2024-09-16T09:27:52.321338+0000","last_fullsized":"2024-09-16T09:27:52.321338+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:40.699813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:40.699813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:54.431740+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"73'79","reported_seq":150,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.313336+0000","last_change":"2024-09-16T09:26:36.183925+0000","last_active":"2024-09-16T09:27:52.313336+0000","last_peered":"2024-09-16T09:27:52.313336+0000","last_clean":"2024-09-16T09:27:52.313336+0000","last_became_active":"2024-09-16T09:26:35.126759+0000","last_became_peered":"2024-09-16T09:26:35.126759+0000","last_unstale":"2024-09-16T09:27:52.313336+0000","last_undegraded":"2024-09-16T09:27:52.313336+0000","last_fullsized":"2024-09-16T09:27:52.313336+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:26:36.183868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:26:36.183868+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"122'8467","reported_seq":8558,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.493431+0000","last_change":"2024-09-16T09:26:08.117045+0000","last_active":"2024-09-16T09:27:52.493431+0000","last_peered":"2024-09-16T09:27:52.493431+0000","last_clean":"2024-09-16T09:27:52.493431+0000","last_became_active":"2024-09-16T09:26:08.116726+0000","last_became_peered":"2024-09-16T09:26:08.116726+0000","last_unstale":"2024-09-16T09:27:52.493431+0000","last_undegraded":"2024-09-16T09:27:52.493431+0000","last_fullsized":"2024-09-16T09:27:52.493431+0000","mapping_epoch":82,"log_start":"98'3900","ondisk_log_start":"98'3900","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_clean_scrub_stamp":"2024-09-16T09:26:07.096764+0000","objects_scrubbed":0,"log_size":4567,"log_dups_size":0,"ondisk_log_size":4567,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":463929344,"num_objects":7079,"num_object_clones":0,"num_object_copies":21237,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7079,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8467,"num_write_kb":497472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"66'10","reported_seq":158,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.310411+0000","last_change":"2024-09-16T09:26:28.194465+0000","last_active":"2024-09-16T09:27:52.310411+0000","last_peered":"2024-09-16T09:27:52.310411+0000","last_clean":"2024-09-16T09:27:52.310411+0000","last_became_active":"2024-09-16T09:25:03.712985+0000","last_became_peered":"2024-09-16T09:25:03.712985+0000","last_unstale":"2024-09-16T09:27:52.310411+0000","last_undegraded":"2024-09-16T09:27:52.310411+0000","last_fullsized":"2024-09-16T09:27:52.310411+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-16T09:26:28.194023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:26:28.194023+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:30.198573+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":146,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.315039+0000","last_change":"2024-09-16T09:26:25.086388+0000","last_active":"2024-09-16T09:27:52.315039+0000","last_peered":"2024-09-16T09:27:52.315039+0000","last_clean":"2024-09-16T09:27:52.315039+0000","last_became_active":"2024-09-16T09:25:01.706440+0000","last_became_peered":"2024-09-16T09:25:01.706440+0000","last_unstale":"2024-09-16T09:27:52.315039+0000","last_undegraded":"2024-09-16T09:27:52.315039+0000","last_fullsized":"2024-09-16T09:27:52.315039+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:25.086322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:25.086322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"66'8","reported_seq":182,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.320712+0000","last_change":"2024-09-16T09:27:17.664572+0000","last_active":"2024-09-16T09:27:52.320712+0000","last_peered":"2024-09-16T09:27:52.320712+0000","last_clean":"2024-09-16T09:27:52.320712+0000","last_became_active":"2024-09-16T09:25:05.714507+0000","last_became_peered":"2024-09-16T09:25:05.714507+0000","last_unstale":"2024-09-16T09:27:52.320712+0000","last_undegraded":"2024-09-16T09:27:52.320712+0000","last_fullsized":"2024-09-16T09:27:52.320712+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:27:17.664493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:17.664493+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:36.536988+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":135,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.320872+0000","last_change":"2024-09-16T09:26:29.632092+0000","last_active":"2024-09-16T09:27:52.320872+0000","last_peered":"2024-09-16T09:27:52.320872+0000","last_clean":"2024-09-16T09:27:52.320872+0000","last_became_active":"2024-09-16T09:25:07.719604+0000","last_became_peered":"2024-09-16T09:25:07.719604+0000","last_unstale":"2024-09-16T09:27:52.320872+0000","last_undegraded":"2024-09-16T09:27:52.320872+0000","last_fullsized":"2024-09-16T09:27:52.320872+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:29.632015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:29.632015+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"73'79","reported_seq":150,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.312907+0000","last_change":"2024-09-16T09:26:35.170288+0000","last_active":"2024-09-16T09:27:52.312907+0000","last_peered":"2024-09-16T09:27:52.312907+0000","last_clean":"2024-09-16T09:27:52.312907+0000","last_became_active":"2024-09-16T09:26:35.125571+0000","last_became_peered":"2024-09-16T09:26:35.125571+0000","last_unstale":"2024-09-16T09:27:52.312907+0000","last_undegraded":"2024-09-16T09:27:52.312907+0000","last_fullsized":"2024-09-16T09:27:52.312907+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:26:35.170229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:26:35.170229+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:53.210344+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.0","version":"122'8614","reported_seq":8774,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.471921+0000","last_change":"2024-09-16T09:27:16.936395+0000","last_active":"2024-09-16T09:27:52.471921+0000","last_peered":"2024-09-16T09:27:52.471921+0000","last_clean":"2024-09-16T09:27:52.471921+0000","last_became_active":"2024-09-16T09:26:08.117581+0000","last_became_peered":"2024-09-16T09:26:08.117581+0000","last_unstale":"2024-09-16T09:27:52.471921+0000","last_undegraded":"2024-09-16T09:27:52.471921+0000","last_fullsized":"2024-09-16T09:27:52.471921+0000","mapping_epoch":82,"log_start":"95'3200","ondisk_log_start":"95'3200","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"109'6200","last_scrub_stamp":"2024-09-16T09:27:16.936218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_clean_scrub_stamp":"2024-09-16T09:27:16.936218+0000","objects_scrubbed":6157,"log_size":5414,"log_dups_size":0,"ondisk_log_size":5414,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:45.561197+0000","scrub_duration":1612,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":473038848,"num_objects":7218,"num_object_clones":0,"num_object_copies":21654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7218,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8614,"num_write_kb":506624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3","version":"66'19","reported_seq":171,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.320977+0000","last_change":"2024-09-16T09:25:03.715086+0000","last_active":"2024-09-16T09:27:52.320977+0000","last_peered":"2024-09-16T09:27:52.320977+0000","last_clean":"2024-09-16T09:27:52.320977+0000","last_became_active":"2024-09-16T09:25:03.714797+0000","last_became_peered":"2024-09-16T09:25:03.714797+0000","last_unstale":"2024-09-16T09:27:52.320977+0000","last_undegraded":"2024-09-16T09:27:52.320977+0000","last_fullsized":"2024-09-16T09:27:52.320977+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:42.581621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":155,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.309729+0000","last_change":"2024-09-16T09:27:24.219132+0000","last_active":"2024-09-16T09:27:52.309729+0000","last_peered":"2024-09-16T09:27:52.309729+0000","last_clean":"2024-09-16T09:27:52.309729+0000","last_became_active":"2024-09-16T09:25:01.706806+0000","last_became_peered":"2024-09-16T09:25:01.706806+0000","last_unstale":"2024-09-16T09:27:52.309729+0000","last_undegraded":"2024-09-16T09:27:52.309729+0000","last_fullsized":"2024-09-16T09:27:52.309729+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:24.218910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:27:24.218910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:52.294630+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":139,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.312771+0000","last_change":"2024-09-16T09:27:09.337607+0000","last_active":"2024-09-16T09:27:52.312771+0000","last_peered":"2024-09-16T09:27:52.312771+0000","last_clean":"2024-09-16T09:27:52.312771+0000","last_became_active":"2024-09-16T09:25:05.708663+0000","last_became_peered":"2024-09-16T09:25:05.708663+0000","last_unstale":"2024-09-16T09:27:52.312771+0000","last_undegraded":"2024-09-16T09:27:52.312771+0000","last_fullsized":"2024-09-16T09:27:52.312771+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:09.337528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:09.337528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:24.568824+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":135,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.308271+0000","last_change":"2024-09-16T09:26:37.282798+0000","last_active":"2024-09-16T09:27:52.308271+0000","last_peered":"2024-09-16T09:27:52.308271+0000","last_clean":"2024-09-16T09:27:52.308271+0000","last_became_active":"2024-09-16T09:25:07.719267+0000","last_became_peered":"2024-09-16T09:25:07.719267+0000","last_unstale":"2024-09-16T09:27:52.308271+0000","last_undegraded":"2024-09-16T09:27:52.308271+0000","last_fullsized":"2024-09-16T09:27:52.308271+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:37.282735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:37.282735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:54.832930+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"73'79","reported_seq":74,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.314140+0000","last_change":"2024-09-16T09:26:35.154953+0000","last_active":"2024-09-16T09:27:52.314140+0000","last_peered":"2024-09-16T09:27:52.314140+0000","last_clean":"2024-09-16T09:27:52.314140+0000","last_became_active":"2024-09-16T09:26:35.117627+0000","last_became_peered":"2024-09-16T09:26:35.117627+0000","last_unstale":"2024-09-16T09:27:52.314140+0000","last_undegraded":"2024-09-16T09:27:52.314140+0000","last_fullsized":"2024-09-16T09:27:52.314140+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:26:35.154875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:26:35.154875+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:57.010300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"122'8650","reported_seq":8741,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.506826+0000","last_change":"2024-09-16T09:26:08.122122+0000","last_active":"2024-09-16T09:27:52.506826+0000","last_peered":"2024-09-16T09:27:52.506826+0000","last_clean":"2024-09-16T09:27:52.506826+0000","last_became_active":"2024-09-16T09:26:08.121828+0000","last_became_peered":"2024-09-16T09:26:08.121828+0000","last_unstale":"2024-09-16T09:27:52.506826+0000","last_undegraded":"2024-09-16T09:27:52.506826+0000","last_fullsized":"2024-09-16T09:27:52.506826+0000","mapping_epoch":82,"log_start":"94'2900","ondisk_log_start":"94'2900","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_clean_scrub_stamp":"2024-09-16T09:26:07.096764+0000","objects_scrubbed":0,"log_size":5750,"log_dups_size":0,"ondisk_log_size":5750,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:40.329742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":475529216,"num_objects":7256,"num_object_clones":0,"num_object_copies":21768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7256,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8650,"num_write_kb":508992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.4","version":"66'28","reported_seq":226,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.309608+0000","last_change":"2024-09-16T09:27:50.051292+0000","last_active":"2024-09-16T09:27:52.309608+0000","last_peered":"2024-09-16T09:27:52.309608+0000","last_clean":"2024-09-16T09:27:52.309608+0000","last_became_active":"2024-09-16T09:25:03.704878+0000","last_became_peered":"2024-09-16T09:25:03.704878+0000","last_unstale":"2024-09-16T09:27:52.309608+0000","last_undegraded":"2024-09-16T09:27:52.309608+0000","last_fullsized":"2024-09-16T09:27:52.309608+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'28","last_scrub_stamp":"2024-09-16T09:27:50.046365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:50.046365+0000","objects_scrubbed":10,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:08.615060+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":139,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.310624+0000","last_change":"2024-09-16T09:25:01.703394+0000","last_active":"2024-09-16T09:27:52.310624+0000","last_peered":"2024-09-16T09:27:52.310624+0000","last_clean":"2024-09-16T09:27:52.310624+0000","last_became_active":"2024-09-16T09:25:01.703076+0000","last_became_peered":"2024-09-16T09:25:01.703076+0000","last_unstale":"2024-09-16T09:27:52.310624+0000","last_undegraded":"2024-09-16T09:27:52.310624+0000","last_fullsized":"2024-09-16T09:27:52.310624+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":156,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.315158+0000","last_change":"2024-09-16T09:27:44.183242+0000","last_active":"2024-09-16T09:27:52.315158+0000","last_peered":"2024-09-16T09:27:52.315158+0000","last_clean":"2024-09-16T09:27:52.315158+0000","last_became_active":"2024-09-16T09:25:00.696237+0000","last_became_peered":"2024-09-16T09:25:00.696237+0000","last_unstale":"2024-09-16T09:27:52.315158+0000","last_undegraded":"2024-09-16T09:27:52.315158+0000","last_fullsized":"2024-09-16T09:27:52.315158+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-16T09:27:44.183111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:27:44.183111+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:47.187301+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.0025137430000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":131,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.321452+0000","last_change":"2024-09-16T09:25:05.718319+0000","last_active":"2024-09-16T09:27:52.321452+0000","last_peered":"2024-09-16T09:27:52.321452+0000","last_clean":"2024-09-16T09:27:52.321452+0000","last_became_active":"2024-09-16T09:25:05.717591+0000","last_became_peered":"2024-09-16T09:25:05.717591+0000","last_unstale":"2024-09-16T09:27:52.321452+0000","last_undegraded":"2024-09-16T09:27:52.321452+0000","last_fullsized":"2024-09-16T09:27:52.321452+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":143,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.317949+0000","last_change":"2024-09-16T09:27:45.201574+0000","last_active":"2024-09-16T09:27:52.317949+0000","last_peered":"2024-09-16T09:27:52.317949+0000","last_clean":"2024-09-16T09:27:52.317949+0000","last_became_active":"2024-09-16T09:25:07.717436+0000","last_became_peered":"2024-09-16T09:25:07.717436+0000","last_unstale":"2024-09-16T09:27:52.317949+0000","last_undegraded":"2024-09-16T09:27:52.317949+0000","last_fullsized":"2024-09-16T09:27:52.317949+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:45.201503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:45.201503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:09.839306+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"73'79","reported_seq":207,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.321146+0000","last_change":"2024-09-16T09:26:35.745908+0000","last_active":"2024-09-16T09:27:52.321146+0000","last_peered":"2024-09-16T09:27:52.321146+0000","last_clean":"2024-09-16T09:27:52.321146+0000","last_became_active":"2024-09-16T09:26:35.115364+0000","last_became_peered":"2024-09-16T09:26:35.115364+0000","last_unstale":"2024-09-16T09:27:52.321146+0000","last_undegraded":"2024-09-16T09:27:52.321146+0000","last_fullsized":"2024-09-16T09:27:52.321146+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:26:35.745837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:26:35.745837+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.4","version":"122'8457","reported_seq":8548,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.648701+0000","last_change":"2024-09-16T09:26:08.120333+0000","last_active":"2024-09-16T09:27:52.648701+0000","last_peered":"2024-09-16T09:27:52.648701+0000","last_clean":"2024-09-16T09:27:52.648701+0000","last_became_active":"2024-09-16T09:26:08.120051+0000","last_became_peered":"2024-09-16T09:26:08.120051+0000","last_unstale":"2024-09-16T09:27:52.648701+0000","last_undegraded":"2024-09-16T09:27:52.648701+0000","last_fullsized":"2024-09-16T09:27:52.648701+0000","mapping_epoch":82,"log_start":"95'3100","ondisk_log_start":"95'3100","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_clean_scrub_stamp":"2024-09-16T09:26:07.096764+0000","objects_scrubbed":0,"log_size":5357,"log_dups_size":0,"ondisk_log_size":5357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:39.620556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":466157568,"num_objects":7113,"num_object_clones":0,"num_object_copies":21339,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7113,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8457,"num_write_kb":498240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,1],"acting":[0,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"66'13","reported_seq":178,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.308931+0000","last_change":"2024-09-16T09:27:44.156057+0000","last_active":"2024-09-16T09:27:52.308931+0000","last_peered":"2024-09-16T09:27:52.308931+0000","last_clean":"2024-09-16T09:27:52.308931+0000","last_became_active":"2024-09-16T09:25:03.713241+0000","last_became_peered":"2024-09-16T09:25:03.713241+0000","last_unstale":"2024-09-16T09:27:52.308931+0000","last_undegraded":"2024-09-16T09:27:52.308931+0000","last_fullsized":"2024-09-16T09:27:52.308931+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'13","last_scrub_stamp":"2024-09-16T09:27:44.155969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:44.155969+0000","objects_scrubbed":5,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:49.037336+0000","scrub_duration":29,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":147,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.310315+0000","last_change":"2024-09-16T09:26:26.230421+0000","last_active":"2024-09-16T09:27:52.310315+0000","last_peered":"2024-09-16T09:27:52.310315+0000","last_clean":"2024-09-16T09:27:52.310315+0000","last_became_active":"2024-09-16T09:25:01.706828+0000","last_became_peered":"2024-09-16T09:25:01.706828+0000","last_unstale":"2024-09-16T09:27:52.310315+0000","last_undegraded":"2024-09-16T09:27:52.310315+0000","last_fullsized":"2024-09-16T09:27:52.310315+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:26.230338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:26.230338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:38.139199+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":156,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.322471+0000","last_change":"2024-09-16T09:27:06.062939+0000","last_active":"2024-09-16T09:27:52.322471+0000","last_peered":"2024-09-16T09:27:52.322471+0000","last_clean":"2024-09-16T09:27:52.322471+0000","last_became_active":"2024-09-16T09:25:00.697822+0000","last_became_peered":"2024-09-16T09:25:00.697822+0000","last_unstale":"2024-09-16T09:27:52.322471+0000","last_undegraded":"2024-09-16T09:27:52.322471+0000","last_fullsized":"2024-09-16T09:27:52.322471+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:27:06.062871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:27:06.062871+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:10.411432+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0.00055022299999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":147,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.320063+0000","last_change":"2024-09-16T09:27:48.986281+0000","last_active":"2024-09-16T09:27:52.320063+0000","last_peered":"2024-09-16T09:27:52.320063+0000","last_clean":"2024-09-16T09:27:52.320063+0000","last_became_active":"2024-09-16T09:25:05.711539+0000","last_became_peered":"2024-09-16T09:25:05.711539+0000","last_unstale":"2024-09-16T09:27:52.320063+0000","last_undegraded":"2024-09-16T09:27:52.320063+0000","last_fullsized":"2024-09-16T09:27:52.320063+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:48.986202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:48.986202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:54.667068+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":134,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.321535+0000","last_change":"2024-09-16T09:27:47.705213+0000","last_active":"2024-09-16T09:27:52.321535+0000","last_peered":"2024-09-16T09:27:52.321535+0000","last_clean":"2024-09-16T09:27:52.321535+0000","last_became_active":"2024-09-16T09:25:07.723282+0000","last_became_peered":"2024-09-16T09:25:07.723282+0000","last_unstale":"2024-09-16T09:27:52.321535+0000","last_undegraded":"2024-09-16T09:27:52.321535+0000","last_fullsized":"2024-09-16T09:27:52.321535+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:47.705151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:47.705151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:08.139782+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"73'79","reported_seq":74,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.308493+0000","last_change":"2024-09-16T09:26:35.272980+0000","last_active":"2024-09-16T09:27:52.308493+0000","last_peered":"2024-09-16T09:27:52.308493+0000","last_clean":"2024-09-16T09:27:52.308493+0000","last_became_active":"2024-09-16T09:26:35.116315+0000","last_became_peered":"2024-09-16T09:26:35.116315+0000","last_unstale":"2024-09-16T09:27:52.308493+0000","last_undegraded":"2024-09-16T09:27:52.308493+0000","last_fullsized":"2024-09-16T09:27:52.308493+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:26:35.272938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:26:35.272938+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"122'8675","reported_seq":8766,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.478340+0000","last_change":"2024-09-16T09:26:08.121383+0000","last_active":"2024-09-16T09:27:52.478340+0000","last_peered":"2024-09-16T09:27:52.478340+0000","last_clean":"2024-09-16T09:27:52.478340+0000","last_became_active":"2024-09-16T09:26:08.121117+0000","last_became_peered":"2024-09-16T09:26:08.121117+0000","last_unstale":"2024-09-16T09:27:52.478340+0000","last_undegraded":"2024-09-16T09:27:52.478340+0000","last_fullsized":"2024-09-16T09:27:52.478340+0000","mapping_epoch":82,"log_start":"97'3800","ondisk_log_start":"97'3800","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_clean_scrub_stamp":"2024-09-16T09:26:07.096764+0000","objects_scrubbed":0,"log_size":4875,"log_dups_size":0,"ondisk_log_size":4875,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:32.708766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":477429760,"num_objects":7285,"num_object_clones":0,"num_object_copies":21855,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7285,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8675,"num_write_kb":510720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,0],"acting":[7,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"66'12","reported_seq":153,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.332246+0000","last_change":"2024-09-16T09:25:03.715612+0000","last_active":"2024-09-16T09:27:52.332246+0000","last_peered":"2024-09-16T09:27:52.332246+0000","last_clean":"2024-09-16T09:27:52.332246+0000","last_became_active":"2024-09-16T09:25:03.715315+0000","last_became_peered":"2024-09-16T09:25:03.715315+0000","last_unstale":"2024-09-16T09:27:52.332246+0000","last_undegraded":"2024-09-16T09:27:52.332246+0000","last_fullsized":"2024-09-16T09:27:52.332246+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"59'1","reported_seq":156,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.320913+0000","last_change":"2024-09-16T09:27:00.666748+0000","last_active":"2024-09-16T09:27:52.320913+0000","last_peered":"2024-09-16T09:27:52.320913+0000","last_clean":"2024-09-16T09:27:52.320913+0000","last_became_active":"2024-09-16T09:25:01.701599+0000","last_became_peered":"2024-09-16T09:25:01.701599+0000","last_unstale":"2024-09-16T09:27:52.320913+0000","last_undegraded":"2024-09-16T09:27:52.320913+0000","last_fullsized":"2024-09-16T09:27:52.320913+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:27:00.666673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:27:00.666673+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:13.128996+0000","scrub_duration":28,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"67'5","reported_seq":322,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.321167+0000","last_change":"2024-09-16T09:26:38.722150+0000","last_active":"2024-09-16T09:27:52.321167+0000","last_peered":"2024-09-16T09:27:52.321167+0000","last_clean":"2024-09-16T09:27:52.321167+0000","last_became_active":"2024-09-16T09:25:00.696546+0000","last_became_peered":"2024-09-16T09:25:00.696546+0000","last_unstale":"2024-09-16T09:27:52.321167+0000","last_undegraded":"2024-09-16T09:27:52.321167+0000","last_fullsized":"2024-09-16T09:27:52.321167+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'5","last_scrub_stamp":"2024-09-16T09:26:38.722074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:26:38.722074+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:03.198309+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0.0059770420000000001,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":165,"num_read_kb":160,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":138,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.315280+0000","last_change":"2024-09-16T09:27:07.337170+0000","last_active":"2024-09-16T09:27:52.315280+0000","last_peered":"2024-09-16T09:27:52.315280+0000","last_clean":"2024-09-16T09:27:52.315280+0000","last_became_active":"2024-09-16T09:25:05.714704+0000","last_became_peered":"2024-09-16T09:25:05.714704+0000","last_unstale":"2024-09-16T09:27:52.315280+0000","last_undegraded":"2024-09-16T09:27:52.315280+0000","last_fullsized":"2024-09-16T09:27:52.315280+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:07.337124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:07.337124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:33.974050+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":135,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.309035+0000","last_change":"2024-09-16T09:26:24.231324+0000","last_active":"2024-09-16T09:27:52.309035+0000","last_peered":"2024-09-16T09:27:52.309035+0000","last_clean":"2024-09-16T09:27:52.309035+0000","last_became_active":"2024-09-16T09:25:07.714158+0000","last_became_peered":"2024-09-16T09:25:07.714158+0000","last_unstale":"2024-09-16T09:27:52.309035+0000","last_undegraded":"2024-09-16T09:27:52.309035+0000","last_fullsized":"2024-09-16T09:27:52.309035+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:24.231167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:24.231167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.6","version":"122'8582","reported_seq":8721,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.648469+0000","last_change":"2024-09-16T09:27:26.477113+0000","last_active":"2024-09-16T09:27:52.648469+0000","last_peered":"2024-09-16T09:27:52.648469+0000","last_clean":"2024-09-16T09:27:52.648469+0000","last_became_active":"2024-09-16T09:26:08.121253+0000","last_became_peered":"2024-09-16T09:26:08.121253+0000","last_unstale":"2024-09-16T09:27:52.648469+0000","last_undegraded":"2024-09-16T09:27:52.648469+0000","last_fullsized":"2024-09-16T09:27:52.648469+0000","mapping_epoch":82,"log_start":"96'3300","ondisk_log_start":"96'3300","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_clean_scrub_stamp":"2024-09-16T09:26:07.096764+0000","objects_scrubbed":4399,"log_size":5282,"log_dups_size":0,"ondisk_log_size":5282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:05.001537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":470548480,"num_objects":7180,"num_object_clones":0,"num_object_copies":21540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7180,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8582,"num_write_kb":504384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"66'16","reported_seq":200,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.313421+0000","last_change":"2024-09-16T09:27:14.265733+0000","last_active":"2024-09-16T09:27:52.313421+0000","last_peered":"2024-09-16T09:27:52.313421+0000","last_clean":"2024-09-16T09:27:52.313421+0000","last_became_active":"2024-09-16T09:25:03.714118+0000","last_became_peered":"2024-09-16T09:25:03.714118+0000","last_unstale":"2024-09-16T09:27:52.313421+0000","last_undegraded":"2024-09-16T09:27:52.313421+0000","last_fullsized":"2024-09-16T09:27:52.313421+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'16","last_scrub_stamp":"2024-09-16T09:27:14.265648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:14.265648+0000","objects_scrubbed":8,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:42.225176+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":147,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.324821+0000","last_change":"2024-09-16T09:27:49.195418+0000","last_active":"2024-09-16T09:27:52.324821+0000","last_peered":"2024-09-16T09:27:52.324821+0000","last_clean":"2024-09-16T09:27:52.324821+0000","last_became_active":"2024-09-16T09:25:01.703730+0000","last_became_peered":"2024-09-16T09:25:01.703730+0000","last_unstale":"2024-09-16T09:27:52.324821+0000","last_undegraded":"2024-09-16T09:27:52.324821+0000","last_fullsized":"2024-09-16T09:27:52.324821+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:49.195331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:27:49.195331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:10.193163+0000","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"73'79","reported_seq":202,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.321442+0000","last_change":"2024-09-16T09:26:34.751780+0000","last_active":"2024-09-16T09:27:52.321442+0000","last_peered":"2024-09-16T09:27:52.321442+0000","last_clean":"2024-09-16T09:27:52.321442+0000","last_became_active":"2024-09-16T09:26:33.297014+0000","last_became_peered":"2024-09-16T09:26:33.297014+0000","last_unstale":"2024-09-16T09:27:52.321442+0000","last_undegraded":"2024-09-16T09:27:52.321442+0000","last_fullsized":"2024-09-16T09:27:52.321442+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:26:34.751720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:26:34.751720+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":138,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.321530+0000","last_change":"2024-09-16T09:26:39.699133+0000","last_active":"2024-09-16T09:27:52.321530+0000","last_peered":"2024-09-16T09:27:52.321530+0000","last_clean":"2024-09-16T09:27:52.321530+0000","last_became_active":"2024-09-16T09:25:05.714341+0000","last_became_peered":"2024-09-16T09:25:05.714341+0000","last_unstale":"2024-09-16T09:27:52.321530+0000","last_undegraded":"2024-09-16T09:27:52.321530+0000","last_fullsized":"2024-09-16T09:27:52.321530+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:39.699059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:26:39.699059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":134,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.315112+0000","last_change":"2024-09-16T09:27:21.192880+0000","last_active":"2024-09-16T09:27:52.315112+0000","last_peered":"2024-09-16T09:27:52.315112+0000","last_clean":"2024-09-16T09:27:52.315112+0000","last_became_active":"2024-09-16T09:25:07.714281+0000","last_became_peered":"2024-09-16T09:25:07.714281+0000","last_unstale":"2024-09-16T09:27:52.315112+0000","last_undegraded":"2024-09-16T09:27:52.315112+0000","last_fullsized":"2024-09-16T09:27:52.315112+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:21.192816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:21.192816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:44.084676+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"122'8481","reported_seq":8573,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.490182+0000","last_change":"2024-09-16T09:26:08.122057+0000","last_active":"2024-09-16T09:27:52.490182+0000","last_peered":"2024-09-16T09:27:52.490182+0000","last_clean":"2024-09-16T09:27:52.490182+0000","last_became_active":"2024-09-16T09:26:08.121785+0000","last_became_peered":"2024-09-16T09:26:08.121785+0000","last_unstale":"2024-09-16T09:27:52.490182+0000","last_undegraded":"2024-09-16T09:27:52.490182+0000","last_fullsized":"2024-09-16T09:27:52.490182+0000","mapping_epoch":82,"log_start":"92'2800","ondisk_log_start":"92'2800","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_clean_scrub_stamp":"2024-09-16T09:26:07.096764+0000","objects_scrubbed":0,"log_size":5681,"log_dups_size":0,"ondisk_log_size":5681,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:36.339662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":468582424,"num_objects":7151,"num_object_clones":0,"num_object_copies":21453,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7151,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8481,"num_write_kb":500161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.e","version":"66'11","reported_seq":173,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.310819+0000","last_change":"2024-09-16T09:26:33.316695+0000","last_active":"2024-09-16T09:27:52.310819+0000","last_peered":"2024-09-16T09:27:52.310819+0000","last_clean":"2024-09-16T09:27:52.310819+0000","last_became_active":"2024-09-16T09:26:33.287399+0000","last_became_peered":"2024-09-16T09:26:33.287399+0000","last_unstale":"2024-09-16T09:27:52.310819+0000","last_undegraded":"2024-09-16T09:27:52.310819+0000","last_fullsized":"2024-09-16T09:27:52.310819+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:53.689658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":139,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.310583+0000","last_change":"2024-09-16T09:25:01.708568+0000","last_active":"2024-09-16T09:27:52.310583+0000","last_peered":"2024-09-16T09:27:52.310583+0000","last_clean":"2024-09-16T09:27:52.310583+0000","last_became_active":"2024-09-16T09:25:01.708270+0000","last_became_peered":"2024-09-16T09:25:01.708270+0000","last_unstale":"2024-09-16T09:27:52.310583+0000","last_undegraded":"2024-09-16T09:27:52.310583+0000","last_fullsized":"2024-09-16T09:27:52.310583+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:54.717044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":138,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.315025+0000","last_change":"2024-09-16T09:26:34.171190+0000","last_active":"2024-09-16T09:27:52.315025+0000","last_peered":"2024-09-16T09:27:52.315025+0000","last_clean":"2024-09-16T09:27:52.315025+0000","last_became_active":"2024-09-16T09:25:05.711964+0000","last_became_peered":"2024-09-16T09:25:05.711964+0000","last_unstale":"2024-09-16T09:27:52.315025+0000","last_undegraded":"2024-09-16T09:27:52.315025+0000","last_fullsized":"2024-09-16T09:27:52.315025+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:34.171139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:26:34.171139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:39.195436+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":135,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.317721+0000","last_change":"2024-09-16T09:27:24.030747+0000","last_active":"2024-09-16T09:27:52.317721+0000","last_peered":"2024-09-16T09:27:52.317721+0000","last_clean":"2024-09-16T09:27:52.317721+0000","last_became_active":"2024-09-16T09:25:07.716514+0000","last_became_peered":"2024-09-16T09:25:07.716514+0000","last_unstale":"2024-09-16T09:27:52.317721+0000","last_undegraded":"2024-09-16T09:27:52.317721+0000","last_fullsized":"2024-09-16T09:27:52.317721+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:24.030677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:24.030677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:42.004108+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.8","version":"73'79","reported_seq":81,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.314106+0000","last_change":"2024-09-16T09:26:37.166480+0000","last_active":"2024-09-16T09:27:52.314106+0000","last_peered":"2024-09-16T09:27:52.314106+0000","last_clean":"2024-09-16T09:27:52.314106+0000","last_became_active":"2024-09-16T09:26:35.126217+0000","last_became_peered":"2024-09-16T09:26:35.126217+0000","last_unstale":"2024-09-16T09:27:52.314106+0000","last_undegraded":"2024-09-16T09:27:52.314106+0000","last_fullsized":"2024-09-16T09:27:52.314106+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:26:37.166434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:26:37.166434+0000","objects_scrubbed":1,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:58.101963+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262688,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.e","version":"122'8520","reported_seq":8680,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.542911+0000","last_change":"2024-09-16T09:27:16.142844+0000","last_active":"2024-09-16T09:27:52.542911+0000","last_peered":"2024-09-16T09:27:52.542911+0000","last_clean":"2024-09-16T09:27:52.542911+0000","last_became_active":"2024-09-16T09:26:08.121969+0000","last_became_peered":"2024-09-16T09:26:08.121969+0000","last_unstale":"2024-09-16T09:27:52.542911+0000","last_undegraded":"2024-09-16T09:27:52.542911+0000","last_fullsized":"2024-09-16T09:27:52.542911+0000","mapping_epoch":82,"log_start":"90'2300","ondisk_log_start":"90'2300","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"108'6031","last_scrub_stamp":"2024-09-16T09:27:16.142747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_clean_scrub_stamp":"2024-09-16T09:27:16.142747+0000","objects_scrubbed":5966,"log_size":6220,"log_dups_size":0,"ondisk_log_size":6220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:42.214405+0000","scrub_duration":2110,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":470548480,"num_objects":7180,"num_object_clones":0,"num_object_copies":21540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7180,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8520,"num_write_kb":502400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"66'17","reported_seq":171,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.310967+0000","last_change":"2024-09-16T09:27:25.615134+0000","last_active":"2024-09-16T09:27:52.310967+0000","last_peered":"2024-09-16T09:27:52.310967+0000","last_clean":"2024-09-16T09:27:52.310967+0000","last_became_active":"2024-09-16T09:25:03.713358+0000","last_became_peered":"2024-09-16T09:25:03.713358+0000","last_unstale":"2024-09-16T09:27:52.310967+0000","last_undegraded":"2024-09-16T09:27:52.310967+0000","last_fullsized":"2024-09-16T09:27:52.310967+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'17","last_scrub_stamp":"2024-09-16T09:27:25.615080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:25.615080+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:48.655184+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":147,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.313071+0000","last_change":"2024-09-16T09:26:04.104188+0000","last_active":"2024-09-16T09:27:52.313071+0000","last_peered":"2024-09-16T09:27:52.313071+0000","last_clean":"2024-09-16T09:27:52.313071+0000","last_became_active":"2024-09-16T09:25:01.703355+0000","last_became_peered":"2024-09-16T09:25:01.703355+0000","last_unstale":"2024-09-16T09:27:52.313071+0000","last_undegraded":"2024-09-16T09:27:52.313071+0000","last_fullsized":"2024-09-16T09:27:52.313071+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:04.104130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:04.104130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:39.294771+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":139,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.309108+0000","last_change":"2024-09-16T09:26:36.322641+0000","last_active":"2024-09-16T09:27:52.309108+0000","last_peered":"2024-09-16T09:27:52.309108+0000","last_clean":"2024-09-16T09:27:52.309108+0000","last_became_active":"2024-09-16T09:25:05.716900+0000","last_became_peered":"2024-09-16T09:25:05.716900+0000","last_unstale":"2024-09-16T09:27:52.309108+0000","last_undegraded":"2024-09-16T09:27:52.309108+0000","last_fullsized":"2024-09-16T09:27:52.309108+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:36.322568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:26:36.322568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":135,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.320142+0000","last_change":"2024-09-16T09:27:17.949869+0000","last_active":"2024-09-16T09:27:52.320142+0000","last_peered":"2024-09-16T09:27:52.320142+0000","last_clean":"2024-09-16T09:27:52.320142+0000","last_became_active":"2024-09-16T09:25:07.721347+0000","last_became_peered":"2024-09-16T09:25:07.721347+0000","last_unstale":"2024-09-16T09:27:52.320142+0000","last_undegraded":"2024-09-16T09:27:52.320142+0000","last_fullsized":"2024-09-16T09:27:52.320142+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:17.949794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:17.949794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:30.022558+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.9","version":"73'79","reported_seq":207,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.321205+0000","last_change":"2024-09-16T09:26:36.771785+0000","last_active":"2024-09-16T09:27:52.321205+0000","last_peered":"2024-09-16T09:27:52.321205+0000","last_clean":"2024-09-16T09:27:52.321205+0000","last_became_active":"2024-09-16T09:26:35.126066+0000","last_became_peered":"2024-09-16T09:26:35.126066+0000","last_unstale":"2024-09-16T09:27:52.321205+0000","last_undegraded":"2024-09-16T09:27:52.321205+0000","last_fullsized":"2024-09-16T09:27:52.321205+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:26:36.771720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:26:36.771720+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:03.015639+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.f","version":"122'8655","reported_seq":8746,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.641223+0000","last_change":"2024-09-16T09:26:08.120989+0000","last_active":"2024-09-16T09:27:52.641223+0000","last_peered":"2024-09-16T09:27:52.641223+0000","last_clean":"2024-09-16T09:27:52.641223+0000","last_became_active":"2024-09-16T09:26:08.120360+0000","last_became_peered":"2024-09-16T09:26:08.120360+0000","last_unstale":"2024-09-16T09:27:52.641223+0000","last_undegraded":"2024-09-16T09:27:52.641223+0000","last_fullsized":"2024-09-16T09:27:52.641223+0000","mapping_epoch":82,"log_start":"96'3300","ondisk_log_start":"96'3300","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_clean_scrub_stamp":"2024-09-16T09:26:07.096764+0000","objects_scrubbed":0,"log_size":5355,"log_dups_size":0,"ondisk_log_size":5355,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:38.592305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":476643328,"num_objects":7273,"num_object_clones":0,"num_object_copies":21819,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7273,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8655,"num_write_kb":509696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"66'10","reported_seq":150,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.310766+0000","last_change":"2024-09-16T09:25:03.708144+0000","last_active":"2024-09-16T09:27:52.310766+0000","last_peered":"2024-09-16T09:27:52.310766+0000","last_clean":"2024-09-16T09:27:52.310766+0000","last_became_active":"2024-09-16T09:25:03.707723+0000","last_became_peered":"2024-09-16T09:25:03.707723+0000","last_unstale":"2024-09-16T09:27:52.310766+0000","last_undegraded":"2024-09-16T09:27:52.310766+0000","last_fullsized":"2024-09-16T09:27:52.310766+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:55.759028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":147,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.317549+0000","last_change":"2024-09-16T09:27:23.082314+0000","last_active":"2024-09-16T09:27:52.317549+0000","last_peered":"2024-09-16T09:27:52.317549+0000","last_clean":"2024-09-16T09:27:52.317549+0000","last_became_active":"2024-09-16T09:25:01.703094+0000","last_became_peered":"2024-09-16T09:25:01.703094+0000","last_unstale":"2024-09-16T09:27:52.317549+0000","last_undegraded":"2024-09-16T09:27:52.317549+0000","last_fullsized":"2024-09-16T09:27:52.317549+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:23.082107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:27:23.082107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:47.542492+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":185,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.319702+0000","last_change":"2024-09-16T09:26:24.837660+0000","last_active":"2024-09-16T09:27:52.319702+0000","last_peered":"2024-09-16T09:27:52.319702+0000","last_clean":"2024-09-16T09:27:52.319702+0000","last_became_active":"2024-09-16T09:25:05.711590+0000","last_became_peered":"2024-09-16T09:25:05.711590+0000","last_unstale":"2024-09-16T09:27:52.319702+0000","last_undegraded":"2024-09-16T09:27:52.319702+0000","last_fullsized":"2024-09-16T09:27:52.319702+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:26:24.837594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:26:24.837594+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:54.153527+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":143,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.310716+0000","last_change":"2024-09-16T09:27:46.596338+0000","last_active":"2024-09-16T09:27:52.310716+0000","last_peered":"2024-09-16T09:27:52.310716+0000","last_clean":"2024-09-16T09:27:52.310716+0000","last_became_active":"2024-09-16T09:25:07.718958+0000","last_became_peered":"2024-09-16T09:25:07.718958+0000","last_unstale":"2024-09-16T09:27:52.310716+0000","last_undegraded":"2024-09-16T09:27:52.310716+0000","last_fullsized":"2024-09-16T09:27:52.310716+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:46.596278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:46.596278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:00.041717+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.8","version":"122'8694","reported_seq":8855,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.553691+0000","last_change":"2024-09-16T09:27:17.928717+0000","last_active":"2024-09-16T09:27:52.553691+0000","last_peered":"2024-09-16T09:27:52.553691+0000","last_clean":"2024-09-16T09:27:52.553691+0000","last_became_active":"2024-09-16T09:26:08.118215+0000","last_became_peered":"2024-09-16T09:26:08.118215+0000","last_unstale":"2024-09-16T09:27:52.553691+0000","last_undegraded":"2024-09-16T09:27:52.553691+0000","last_fullsized":"2024-09-16T09:27:52.553691+0000","mapping_epoch":82,"log_start":"91'2500","ondisk_log_start":"91'2500","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"109'6266","last_scrub_stamp":"2024-09-16T09:27:17.927781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_clean_scrub_stamp":"2024-09-16T09:27:17.927781+0000","objects_scrubbed":6214,"log_size":6194,"log_dups_size":0,"ondisk_log_size":6194,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:37.591987+0000","scrub_duration":1966,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":479330304,"num_objects":7314,"num_object_clones":0,"num_object_copies":21942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7314,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8694,"num_write_kb":512256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"66'9","reported_seq":159,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.320883+0000","last_change":"2024-09-16T09:26:28.596878+0000","last_active":"2024-09-16T09:27:52.320883+0000","last_peered":"2024-09-16T09:27:52.320883+0000","last_clean":"2024-09-16T09:27:52.320883+0000","last_became_active":"2024-09-16T09:25:03.712230+0000","last_became_peered":"2024-09-16T09:25:03.712230+0000","last_unstale":"2024-09-16T09:27:52.320883+0000","last_undegraded":"2024-09-16T09:27:52.320883+0000","last_fullsized":"2024-09-16T09:27:52.320883+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:26:28.596805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:26:28.596805+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":146,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.314934+0000","last_change":"2024-09-16T09:26:26.061173+0000","last_active":"2024-09-16T09:27:52.314934+0000","last_peered":"2024-09-16T09:27:52.314934+0000","last_clean":"2024-09-16T09:27:52.314934+0000","last_became_active":"2024-09-16T09:25:01.702629+0000","last_became_peered":"2024-09-16T09:25:01.702629+0000","last_unstale":"2024-09-16T09:27:52.314934+0000","last_undegraded":"2024-09-16T09:27:52.314934+0000","last_fullsized":"2024-09-16T09:27:52.314934+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:26.061106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:26.061106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":139,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.322411+0000","last_change":"2024-09-16T09:27:19.997700+0000","last_active":"2024-09-16T09:27:52.322411+0000","last_peered":"2024-09-16T09:27:52.322411+0000","last_clean":"2024-09-16T09:27:52.322411+0000","last_became_active":"2024-09-16T09:25:05.712573+0000","last_became_peered":"2024-09-16T09:25:05.712573+0000","last_unstale":"2024-09-16T09:27:52.322411+0000","last_undegraded":"2024-09-16T09:27:52.322411+0000","last_fullsized":"2024-09-16T09:27:52.322411+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:19.997627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:19.997627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:47.478049+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":135,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.320823+0000","last_change":"2024-09-16T09:26:35.608021+0000","last_active":"2024-09-16T09:27:52.320823+0000","last_peered":"2024-09-16T09:27:52.320823+0000","last_clean":"2024-09-16T09:27:52.320823+0000","last_became_active":"2024-09-16T09:25:07.718778+0000","last_became_peered":"2024-09-16T09:25:07.718778+0000","last_unstale":"2024-09-16T09:27:52.320823+0000","last_undegraded":"2024-09-16T09:27:52.320823+0000","last_fullsized":"2024-09-16T09:27:52.320823+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:35.607956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:35.607956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:04.461371+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"122'8514","reported_seq":8605,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.658077+0000","last_change":"2024-09-16T09:26:08.120869+0000","last_active":"2024-09-16T09:27:52.658077+0000","last_peered":"2024-09-16T09:27:52.658077+0000","last_clean":"2024-09-16T09:27:52.658077+0000","last_became_active":"2024-09-16T09:26:08.120579+0000","last_became_peered":"2024-09-16T09:26:08.120579+0000","last_unstale":"2024-09-16T09:27:52.658077+0000","last_undegraded":"2024-09-16T09:27:52.658077+0000","last_fullsized":"2024-09-16T09:27:52.658077+0000","mapping_epoch":82,"log_start":"96'3200","ondisk_log_start":"96'3200","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_clean_scrub_stamp":"2024-09-16T09:26:07.096764+0000","objects_scrubbed":0,"log_size":5314,"log_dups_size":0,"ondisk_log_size":5314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:30.609078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":462553088,"num_objects":7058,"num_object_clones":0,"num_object_copies":21174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7058,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8514,"num_write_kb":498304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"66'19","reported_seq":174,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.312870+0000","last_change":"2024-09-16T09:26:25.022657+0000","last_active":"2024-09-16T09:27:52.312870+0000","last_peered":"2024-09-16T09:27:52.312870+0000","last_clean":"2024-09-16T09:27:52.312870+0000","last_became_active":"2024-09-16T09:25:03.712783+0000","last_became_peered":"2024-09-16T09:25:03.712783+0000","last_unstale":"2024-09-16T09:27:52.312870+0000","last_undegraded":"2024-09-16T09:27:52.312870+0000","last_fullsized":"2024-09-16T09:27:52.312870+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-16T09:26:25.022598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:26:25.022598+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"59'1","reported_seq":152,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.321736+0000","last_change":"2024-09-16T09:27:17.653505+0000","last_active":"2024-09-16T09:27:52.321736+0000","last_peered":"2024-09-16T09:27:52.321736+0000","last_clean":"2024-09-16T09:27:52.321736+0000","last_became_active":"2024-09-16T09:25:01.708069+0000","last_became_peered":"2024-09-16T09:25:01.708069+0000","last_unstale":"2024-09-16T09:27:52.321736+0000","last_undegraded":"2024-09-16T09:27:52.321736+0000","last_fullsized":"2024-09-16T09:27:52.321736+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:27:17.653425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:27:17.653425+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:38.391486+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":139,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.326793+0000","last_change":"2024-09-16T09:27:11.067059+0000","last_active":"2024-09-16T09:27:52.326793+0000","last_peered":"2024-09-16T09:27:52.326793+0000","last_clean":"2024-09-16T09:27:52.326793+0000","last_became_active":"2024-09-16T09:25:05.709264+0000","last_became_peered":"2024-09-16T09:25:05.709264+0000","last_unstale":"2024-09-16T09:27:52.326793+0000","last_undegraded":"2024-09-16T09:27:52.326793+0000","last_fullsized":"2024-09-16T09:27:52.326793+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:11.066986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:11.066986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:40.568379+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":134,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.321189+0000","last_change":"2024-09-16T09:27:25.865107+0000","last_active":"2024-09-16T09:27:52.321189+0000","last_peered":"2024-09-16T09:27:52.321189+0000","last_clean":"2024-09-16T09:27:52.321189+0000","last_became_active":"2024-09-16T09:25:07.718750+0000","last_became_peered":"2024-09-16T09:25:07.718750+0000","last_unstale":"2024-09-16T09:27:52.321189+0000","last_undegraded":"2024-09-16T09:27:52.321189+0000","last_fullsized":"2024-09-16T09:27:52.321189+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:25.865030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:25.865030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:35.295712+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.a","version":"122'8561","reported_seq":8652,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.679320+0000","last_change":"2024-09-16T09:26:08.122847+0000","last_active":"2024-09-16T09:27:52.679320+0000","last_peered":"2024-09-16T09:27:52.679320+0000","last_clean":"2024-09-16T09:27:52.679320+0000","last_became_active":"2024-09-16T09:26:08.122557+0000","last_became_peered":"2024-09-16T09:26:08.122557+0000","last_unstale":"2024-09-16T09:27:52.679320+0000","last_undegraded":"2024-09-16T09:27:52.679320+0000","last_fullsized":"2024-09-16T09:27:52.679320+0000","mapping_epoch":82,"log_start":"100'4200","ondisk_log_start":"100'4200","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_clean_scrub_stamp":"2024-09-16T09:26:07.096764+0000","objects_scrubbed":0,"log_size":4361,"log_dups_size":0,"ondisk_log_size":4361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:34.098109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":474808320,"num_objects":7245,"num_object_clones":0,"num_object_copies":21735,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7245,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8561,"num_write_kb":505792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"66'12","reported_seq":166,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.310492+0000","last_change":"2024-09-16T09:27:01.771750+0000","last_active":"2024-09-16T09:27:52.310492+0000","last_peered":"2024-09-16T09:27:52.310492+0000","last_clean":"2024-09-16T09:27:52.310492+0000","last_became_active":"2024-09-16T09:25:03.714027+0000","last_became_peered":"2024-09-16T09:25:03.714027+0000","last_unstale":"2024-09-16T09:27:52.310492+0000","last_undegraded":"2024-09-16T09:27:52.310492+0000","last_fullsized":"2024-09-16T09:27:52.310492+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-16T09:27:01.771667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:01.771667+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:04.790349+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":138,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.321881+0000","last_change":"2024-09-16T09:25:01.707905+0000","last_active":"2024-09-16T09:27:52.321881+0000","last_peered":"2024-09-16T09:27:52.321881+0000","last_clean":"2024-09-16T09:27:52.321881+0000","last_became_active":"2024-09-16T09:25:01.707528+0000","last_became_peered":"2024-09-16T09:25:01.707528+0000","last_unstale":"2024-09-16T09:27:52.321881+0000","last_undegraded":"2024-09-16T09:27:52.321881+0000","last_fullsized":"2024-09-16T09:27:52.321881+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:31.737341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":147,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.319492+0000","last_change":"2024-09-16T09:27:49.965628+0000","last_active":"2024-09-16T09:27:52.319492+0000","last_peered":"2024-09-16T09:27:52.319492+0000","last_clean":"2024-09-16T09:27:52.319492+0000","last_became_active":"2024-09-16T09:25:05.715253+0000","last_became_peered":"2024-09-16T09:25:05.715253+0000","last_unstale":"2024-09-16T09:27:52.319492+0000","last_undegraded":"2024-09-16T09:27:52.319492+0000","last_fullsized":"2024-09-16T09:27:52.319492+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:49.965424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:49.965424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:03.758011+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":127,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.317252+0000","last_change":"2024-09-16T09:25:07.719338+0000","last_active":"2024-09-16T09:27:52.317252+0000","last_peered":"2024-09-16T09:27:52.317252+0000","last_clean":"2024-09-16T09:27:52.317252+0000","last_became_active":"2024-09-16T09:25:07.718899+0000","last_became_peered":"2024-09-16T09:25:07.718899+0000","last_unstale":"2024-09-16T09:27:52.317252+0000","last_undegraded":"2024-09-16T09:27:52.317252+0000","last_fullsized":"2024-09-16T09:27:52.317252+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"122'8494","reported_seq":8657,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.530264+0000","last_change":"2024-09-16T09:27:22.420461+0000","last_active":"2024-09-16T09:27:52.530264+0000","last_peered":"2024-09-16T09:27:52.530264+0000","last_clean":"2024-09-16T09:27:52.530264+0000","last_became_active":"2024-09-16T09:26:08.117869+0000","last_became_peered":"2024-09-16T09:26:08.117869+0000","last_unstale":"2024-09-16T09:27:52.530264+0000","last_undegraded":"2024-09-16T09:27:52.530264+0000","last_fullsized":"2024-09-16T09:27:52.530264+0000","mapping_epoch":82,"log_start":"97'3500","ondisk_log_start":"97'3500","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'6393","last_scrub_stamp":"2024-09-16T09:27:22.420266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_clean_scrub_stamp":"2024-09-16T09:27:22.420266+0000","objects_scrubbed":6319,"log_size":4994,"log_dups_size":0,"ondisk_log_size":4994,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:47.768482+0000","scrub_duration":2145,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":461111296,"num_objects":7036,"num_object_clones":0,"num_object_copies":21108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7036,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8494,"num_write_kb":496960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"66'15","reported_seq":167,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.314223+0000","last_change":"2024-09-16T09:26:33.138828+0000","last_active":"2024-09-16T09:27:52.314223+0000","last_peered":"2024-09-16T09:27:52.314223+0000","last_clean":"2024-09-16T09:27:52.314223+0000","last_became_active":"2024-09-16T09:25:03.715733+0000","last_became_peered":"2024-09-16T09:25:03.715733+0000","last_unstale":"2024-09-16T09:27:52.314223+0000","last_undegraded":"2024-09-16T09:27:52.314223+0000","last_fullsized":"2024-09-16T09:27:52.314223+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-16T09:26:33.138603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:26:33.138603+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:42.149896+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":158,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.321875+0000","last_change":"2024-09-16T09:26:26.705847+0000","last_active":"2024-09-16T09:27:52.321875+0000","last_peered":"2024-09-16T09:27:52.321875+0000","last_clean":"2024-09-16T09:27:52.321875+0000","last_became_active":"2024-09-16T09:25:01.707372+0000","last_became_peered":"2024-09-16T09:25:01.707372+0000","last_unstale":"2024-09-16T09:27:52.321875+0000","last_undegraded":"2024-09-16T09:27:52.321875+0000","last_fullsized":"2024-09-16T09:27:52.321875+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-16T09:26:26.705773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:26.705773+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:40.764628+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"66'8","reported_seq":181,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.321201+0000","last_change":"2024-09-16T09:27:51.699524+0000","last_active":"2024-09-16T09:27:52.321201+0000","last_peered":"2024-09-16T09:27:52.321201+0000","last_clean":"2024-09-16T09:27:52.321201+0000","last_became_active":"2024-09-16T09:25:05.716189+0000","last_became_peered":"2024-09-16T09:25:05.716189+0000","last_unstale":"2024-09-16T09:27:52.321201+0000","last_undegraded":"2024-09-16T09:27:52.321201+0000","last_fullsized":"2024-09-16T09:27:52.321201+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:27:51.699463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:51.699463+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:58.162844+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":134,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.314741+0000","last_change":"2024-09-16T09:27:20.219376+0000","last_active":"2024-09-16T09:27:52.314741+0000","last_peered":"2024-09-16T09:27:52.314741+0000","last_clean":"2024-09-16T09:27:52.314741+0000","last_became_active":"2024-09-16T09:25:07.717213+0000","last_became_peered":"2024-09-16T09:25:07.717213+0000","last_unstale":"2024-09-16T09:27:52.314741+0000","last_undegraded":"2024-09-16T09:27:52.314741+0000","last_fullsized":"2024-09-16T09:27:52.314741+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:20.219333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:20.219333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:35.792434+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":147,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.312751+0000","last_change":"2024-09-16T09:26:33.139405+0000","last_active":"2024-09-16T09:27:52.312751+0000","last_peered":"2024-09-16T09:27:52.312751+0000","last_clean":"2024-09-16T09:27:52.312751+0000","last_became_active":"2024-09-16T09:25:01.703202+0000","last_became_peered":"2024-09-16T09:25:01.703202+0000","last_unstale":"2024-09-16T09:27:52.312751+0000","last_undegraded":"2024-09-16T09:27:52.312751+0000","last_fullsized":"2024-09-16T09:27:52.312751+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:33.139360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:33.139360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:58.524570+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"66'6","reported_seq":152,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.317766+0000","last_change":"2024-09-16T09:26:23.458888+0000","last_active":"2024-09-16T09:27:52.317766+0000","last_peered":"2024-09-16T09:27:52.317766+0000","last_clean":"2024-09-16T09:27:52.317766+0000","last_became_active":"2024-09-16T09:25:03.713245+0000","last_became_peered":"2024-09-16T09:25:03.713245+0000","last_unstale":"2024-09-16T09:27:52.317766+0000","last_undegraded":"2024-09-16T09:27:52.317766+0000","last_fullsized":"2024-09-16T09:27:52.317766+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'6","last_scrub_stamp":"2024-09-16T09:26:23.458794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:26:23.458794+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:36.044643+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":130,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.315254+0000","last_change":"2024-09-16T09:25:05.715938+0000","last_active":"2024-09-16T09:27:52.315254+0000","last_peered":"2024-09-16T09:27:52.315254+0000","last_clean":"2024-09-16T09:27:52.315254+0000","last_became_active":"2024-09-16T09:25:05.715616+0000","last_became_peered":"2024-09-16T09:25:05.715616+0000","last_unstale":"2024-09-16T09:27:52.315254+0000","last_undegraded":"2024-09-16T09:27:52.315254+0000","last_fullsized":"2024-09-16T09:27:52.315254+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":134,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.321472+0000","last_change":"2024-09-16T09:26:28.733381+0000","last_active":"2024-09-16T09:27:52.321472+0000","last_peered":"2024-09-16T09:27:52.321472+0000","last_clean":"2024-09-16T09:27:52.321472+0000","last_became_active":"2024-09-16T09:25:07.721059+0000","last_became_peered":"2024-09-16T09:25:07.721059+0000","last_unstale":"2024-09-16T09:27:52.321472+0000","last_undegraded":"2024-09-16T09:27:52.321472+0000","last_fullsized":"2024-09-16T09:27:52.321472+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:28.733324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:28.733324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"66'9","reported_seq":159,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.332273+0000","last_change":"2024-09-16T09:26:39.554245+0000","last_active":"2024-09-16T09:27:52.332273+0000","last_peered":"2024-09-16T09:27:52.332273+0000","last_clean":"2024-09-16T09:27:52.332273+0000","last_became_active":"2024-09-16T09:25:03.719573+0000","last_became_peered":"2024-09-16T09:25:03.719573+0000","last_unstale":"2024-09-16T09:27:52.332273+0000","last_undegraded":"2024-09-16T09:27:52.332273+0000","last_fullsized":"2024-09-16T09:27:52.332273+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:26:39.554069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:26:39.554069+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:08.727745+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":146,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.321370+0000","last_change":"2024-09-16T09:27:24.854485+0000","last_active":"2024-09-16T09:27:52.321370+0000","last_peered":"2024-09-16T09:27:52.321370+0000","last_clean":"2024-09-16T09:27:52.321370+0000","last_became_active":"2024-09-16T09:25:01.706491+0000","last_became_peered":"2024-09-16T09:25:01.706491+0000","last_unstale":"2024-09-16T09:27:52.321370+0000","last_undegraded":"2024-09-16T09:27:52.321370+0000","last_fullsized":"2024-09-16T09:27:52.321370+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:24.854418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:27:24.854418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:49.764094+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":131,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.317587+0000","last_change":"2024-09-16T09:25:05.715083+0000","last_active":"2024-09-16T09:27:52.317587+0000","last_peered":"2024-09-16T09:27:52.317587+0000","last_clean":"2024-09-16T09:27:52.317587+0000","last_became_active":"2024-09-16T09:25:05.714801+0000","last_became_peered":"2024-09-16T09:25:05.714801+0000","last_unstale":"2024-09-16T09:27:52.317587+0000","last_undegraded":"2024-09-16T09:27:52.317587+0000","last_fullsized":"2024-09-16T09:27:52.317587+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:32.055938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":135,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.319298+0000","last_change":"2024-09-16T09:26:35.866582+0000","last_active":"2024-09-16T09:27:52.319298+0000","last_peered":"2024-09-16T09:27:52.319298+0000","last_clean":"2024-09-16T09:27:52.319298+0000","last_became_active":"2024-09-16T09:25:07.722050+0000","last_became_peered":"2024-09-16T09:25:07.722050+0000","last_unstale":"2024-09-16T09:27:52.319298+0000","last_undegraded":"2024-09-16T09:27:52.319298+0000","last_fullsized":"2024-09-16T09:27:52.319298+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:35.866512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:35.866512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:39.028701+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"66'9","reported_seq":150,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.314918+0000","last_change":"2024-09-16T09:25:03.717609+0000","last_active":"2024-09-16T09:27:52.314918+0000","last_peered":"2024-09-16T09:27:52.314918+0000","last_clean":"2024-09-16T09:27:52.314918+0000","last_became_active":"2024-09-16T09:25:03.717310+0000","last_became_peered":"2024-09-16T09:25:03.717310+0000","last_unstale":"2024-09-16T09:27:52.314918+0000","last_undegraded":"2024-09-16T09:27:52.314918+0000","last_fullsized":"2024-09-16T09:27:52.314918+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":147,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.321425+0000","last_change":"2024-09-16T09:27:12.652668+0000","last_active":"2024-09-16T09:27:52.321425+0000","last_peered":"2024-09-16T09:27:52.321425+0000","last_clean":"2024-09-16T09:27:52.321425+0000","last_became_active":"2024-09-16T09:25:01.702419+0000","last_became_peered":"2024-09-16T09:25:01.702419+0000","last_unstale":"2024-09-16T09:27:52.321425+0000","last_undegraded":"2024-09-16T09:27:52.321425+0000","last_fullsized":"2024-09-16T09:27:52.321425+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:12.652603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:27:12.652603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:16.848902+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"66'8","reported_seq":182,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.317845+0000","last_change":"2024-09-16T09:27:05.790385+0000","last_active":"2024-09-16T09:27:52.317845+0000","last_peered":"2024-09-16T09:27:52.317845+0000","last_clean":"2024-09-16T09:27:52.317845+0000","last_became_active":"2024-09-16T09:25:05.711642+0000","last_became_peered":"2024-09-16T09:25:05.711642+0000","last_unstale":"2024-09-16T09:27:52.317845+0000","last_undegraded":"2024-09-16T09:27:52.317845+0000","last_fullsized":"2024-09-16T09:27:52.317845+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:27:05.790301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:05.790301+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:25.945999+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":135,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.310855+0000","last_change":"2024-09-16T09:26:23.811194+0000","last_active":"2024-09-16T09:27:52.310855+0000","last_peered":"2024-09-16T09:27:52.310855+0000","last_clean":"2024-09-16T09:27:52.310855+0000","last_became_active":"2024-09-16T09:25:07.718810+0000","last_became_peered":"2024-09-16T09:25:07.718810+0000","last_unstale":"2024-09-16T09:27:52.310855+0000","last_undegraded":"2024-09-16T09:27:52.310855+0000","last_fullsized":"2024-09-16T09:27:52.310855+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:23.811127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:23.811127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"66'10","reported_seq":158,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.318148+0000","last_change":"2024-09-16T09:27:02.700022+0000","last_active":"2024-09-16T09:27:52.318148+0000","last_peered":"2024-09-16T09:27:52.318148+0000","last_clean":"2024-09-16T09:27:52.318148+0000","last_became_active":"2024-09-16T09:25:03.712387+0000","last_became_peered":"2024-09-16T09:25:03.712387+0000","last_unstale":"2024-09-16T09:27:52.318148+0000","last_undegraded":"2024-09-16T09:27:52.318148+0000","last_fullsized":"2024-09-16T09:27:52.318148+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-16T09:27:02.699947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:02.699947+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:29.333798+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":138,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.321583+0000","last_change":"2024-09-16T09:25:01.708461+0000","last_active":"2024-09-16T09:27:52.321583+0000","last_peered":"2024-09-16T09:27:52.321583+0000","last_clean":"2024-09-16T09:27:52.321583+0000","last_became_active":"2024-09-16T09:25:01.707513+0000","last_became_peered":"2024-09-16T09:25:01.707513+0000","last_unstale":"2024-09-16T09:27:52.321583+0000","last_undegraded":"2024-09-16T09:27:52.321583+0000","last_fullsized":"2024-09-16T09:27:52.321583+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:27.801162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"66'8","reported_seq":181,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.314733+0000","last_change":"2024-09-16T09:27:13.395353+0000","last_active":"2024-09-16T09:27:52.314733+0000","last_peered":"2024-09-16T09:27:52.314733+0000","last_clean":"2024-09-16T09:27:52.314733+0000","last_became_active":"2024-09-16T09:25:05.713833+0000","last_became_peered":"2024-09-16T09:25:05.713833+0000","last_unstale":"2024-09-16T09:27:52.314733+0000","last_undegraded":"2024-09-16T09:27:52.314733+0000","last_fullsized":"2024-09-16T09:27:52.314733+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:27:13.395272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:13.395272+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:19.438214+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":135,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.323733+0000","last_change":"2024-09-16T09:27:51.639488+0000","last_active":"2024-09-16T09:27:52.323733+0000","last_peered":"2024-09-16T09:27:52.323733+0000","last_clean":"2024-09-16T09:27:52.323733+0000","last_became_active":"2024-09-16T09:25:07.717357+0000","last_became_peered":"2024-09-16T09:25:07.717357+0000","last_unstale":"2024-09-16T09:27:52.323733+0000","last_undegraded":"2024-09-16T09:27:52.323733+0000","last_fullsized":"2024-09-16T09:27:52.323733+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:51.639428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:51.639428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:19.906242+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"66'11","reported_seq":173,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.310377+0000","last_change":"2024-09-16T09:26:33.302713+0000","last_active":"2024-09-16T09:27:52.310377+0000","last_peered":"2024-09-16T09:27:52.310377+0000","last_clean":"2024-09-16T09:27:52.310377+0000","last_became_active":"2024-09-16T09:26:33.291265+0000","last_became_peered":"2024-09-16T09:26:33.291265+0000","last_unstale":"2024-09-16T09:27:52.310377+0000","last_undegraded":"2024-09-16T09:27:52.310377+0000","last_fullsized":"2024-09-16T09:27:52.310377+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:41.657340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":147,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.310993+0000","last_change":"2024-09-16T09:26:40.610512+0000","last_active":"2024-09-16T09:27:52.310993+0000","last_peered":"2024-09-16T09:27:52.310993+0000","last_clean":"2024-09-16T09:27:52.310993+0000","last_became_active":"2024-09-16T09:25:01.705158+0000","last_became_peered":"2024-09-16T09:25:01.705158+0000","last_unstale":"2024-09-16T09:27:52.310993+0000","last_undegraded":"2024-09-16T09:27:52.310993+0000","last_fullsized":"2024-09-16T09:27:52.310993+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:40.610383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:40.610383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:53.224993+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":147,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.310254+0000","last_change":"2024-09-16T09:27:46.034727+0000","last_active":"2024-09-16T09:27:52.310254+0000","last_peered":"2024-09-16T09:27:52.310254+0000","last_clean":"2024-09-16T09:27:52.310254+0000","last_became_active":"2024-09-16T09:25:05.712816+0000","last_became_peered":"2024-09-16T09:25:05.712816+0000","last_unstale":"2024-09-16T09:27:52.310254+0000","last_undegraded":"2024-09-16T09:27:52.310254+0000","last_fullsized":"2024-09-16T09:27:52.310254+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:46.034672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:46.034672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:49.704569+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":143,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.318339+0000","last_change":"2024-09-16T09:27:48.147819+0000","last_active":"2024-09-16T09:27:52.318339+0000","last_peered":"2024-09-16T09:27:52.318339+0000","last_clean":"2024-09-16T09:27:52.318339+0000","last_became_active":"2024-09-16T09:25:07.713109+0000","last_became_peered":"2024-09-16T09:25:07.713109+0000","last_unstale":"2024-09-16T09:27:52.318339+0000","last_undegraded":"2024-09-16T09:27:52.318339+0000","last_fullsized":"2024-09-16T09:27:52.318339+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:48.147759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:48.147759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:56.283015+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"66'9","reported_seq":159,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.310501+0000","last_change":"2024-09-16T09:26:31.216495+0000","last_active":"2024-09-16T09:27:52.310501+0000","last_peered":"2024-09-16T09:27:52.310501+0000","last_clean":"2024-09-16T09:27:52.310501+0000","last_became_active":"2024-09-16T09:25:03.705807+0000","last_became_peered":"2024-09-16T09:25:03.705807+0000","last_unstale":"2024-09-16T09:27:52.310501+0000","last_undegraded":"2024-09-16T09:27:52.310501+0000","last_fullsized":"2024-09-16T09:27:52.310501+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:26:31.216298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:26:31.216298+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":146,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.321644+0000","last_change":"2024-09-16T09:26:23.796308+0000","last_active":"2024-09-16T09:27:52.321644+0000","last_peered":"2024-09-16T09:27:52.321644+0000","last_clean":"2024-09-16T09:27:52.321644+0000","last_became_active":"2024-09-16T09:25:01.706762+0000","last_became_peered":"2024-09-16T09:25:01.706762+0000","last_unstale":"2024-09-16T09:27:52.321644+0000","last_undegraded":"2024-09-16T09:27:52.321644+0000","last_fullsized":"2024-09-16T09:27:52.321644+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:23.796231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:23.796231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:38.747701+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":139,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.317337+0000","last_change":"2024-09-16T09:27:44.242118+0000","last_active":"2024-09-16T09:27:52.317337+0000","last_peered":"2024-09-16T09:27:52.317337+0000","last_clean":"2024-09-16T09:27:52.317337+0000","last_became_active":"2024-09-16T09:25:05.713791+0000","last_became_peered":"2024-09-16T09:25:05.713791+0000","last_unstale":"2024-09-16T09:27:52.317337+0000","last_undegraded":"2024-09-16T09:27:52.317337+0000","last_fullsized":"2024-09-16T09:27:52.317337+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:44.241942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:44.241942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:12.701993+0000","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":135,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.320580+0000","last_change":"2024-09-16T09:27:21.664905+0000","last_active":"2024-09-16T09:27:52.320580+0000","last_peered":"2024-09-16T09:27:52.320580+0000","last_clean":"2024-09-16T09:27:52.320580+0000","last_became_active":"2024-09-16T09:25:07.715301+0000","last_became_peered":"2024-09-16T09:25:07.715301+0000","last_unstale":"2024-09-16T09:27:52.320580+0000","last_undegraded":"2024-09-16T09:27:52.320580+0000","last_fullsized":"2024-09-16T09:27:52.320580+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:21.664836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:21.664836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:32.409182+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"66'11","reported_seq":162,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.318528+0000","last_change":"2024-09-16T09:27:21.062617+0000","last_active":"2024-09-16T09:27:52.318528+0000","last_peered":"2024-09-16T09:27:52.318528+0000","last_clean":"2024-09-16T09:27:52.318528+0000","last_became_active":"2024-09-16T09:25:03.712333+0000","last_became_peered":"2024-09-16T09:25:03.712333+0000","last_unstale":"2024-09-16T09:27:52.318528+0000","last_undegraded":"2024-09-16T09:27:52.318528+0000","last_fullsized":"2024-09-16T09:27:52.318528+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-16T09:27:21.062564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:21.062564+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:26.644417+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":154,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.315322+0000","last_change":"2024-09-16T09:27:46.175159+0000","last_active":"2024-09-16T09:27:52.315322+0000","last_peered":"2024-09-16T09:27:52.315322+0000","last_clean":"2024-09-16T09:27:52.315322+0000","last_became_active":"2024-09-16T09:25:01.700733+0000","last_became_peered":"2024-09-16T09:25:01.700733+0000","last_unstale":"2024-09-16T09:27:52.315322+0000","last_undegraded":"2024-09-16T09:27:52.315322+0000","last_fullsized":"2024-09-16T09:27:52.315322+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:46.175093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:27:46.175093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:15.134231+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":130,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.320983+0000","last_change":"2024-09-16T09:25:05.713760+0000","last_active":"2024-09-16T09:27:52.320983+0000","last_peered":"2024-09-16T09:27:52.320983+0000","last_clean":"2024-09-16T09:27:52.320983+0000","last_became_active":"2024-09-16T09:25:05.713479+0000","last_became_peered":"2024-09-16T09:25:05.713479+0000","last_unstale":"2024-09-16T09:27:52.320983+0000","last_undegraded":"2024-09-16T09:27:52.320983+0000","last_fullsized":"2024-09-16T09:27:52.320983+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":135,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.317471+0000","last_change":"2024-09-16T09:26:32.417222+0000","last_active":"2024-09-16T09:27:52.317471+0000","last_peered":"2024-09-16T09:27:52.317471+0000","last_clean":"2024-09-16T09:27:52.317471+0000","last_became_active":"2024-09-16T09:25:07.715582+0000","last_became_peered":"2024-09-16T09:25:07.715582+0000","last_unstale":"2024-09-16T09:27:52.317471+0000","last_undegraded":"2024-09-16T09:27:52.317471+0000","last_fullsized":"2024-09-16T09:27:52.317471+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:32.417150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:32.417150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:53.673620+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"66'4","reported_seq":149,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.321918+0000","last_change":"2024-09-16T09:26:27.474085+0000","last_active":"2024-09-16T09:27:52.321918+0000","last_peered":"2024-09-16T09:27:52.321918+0000","last_clean":"2024-09-16T09:27:52.321918+0000","last_became_active":"2024-09-16T09:25:03.704869+0000","last_became_peered":"2024-09-16T09:25:03.704869+0000","last_unstale":"2024-09-16T09:27:52.321918+0000","last_undegraded":"2024-09-16T09:27:52.321918+0000","last_fullsized":"2024-09-16T09:27:52.321918+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'4","last_scrub_stamp":"2024-09-16T09:26:27.473740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:26:27.473740+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:27:56.231643+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":147,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.321170+0000","last_change":"2024-09-16T09:26:23.494773+0000","last_active":"2024-09-16T09:27:52.321170+0000","last_peered":"2024-09-16T09:27:52.321170+0000","last_clean":"2024-09-16T09:27:52.321170+0000","last_became_active":"2024-09-16T09:25:01.705686+0000","last_became_peered":"2024-09-16T09:25:01.705686+0000","last_unstale":"2024-09-16T09:27:52.321170+0000","last_undegraded":"2024-09-16T09:27:52.321170+0000","last_fullsized":"2024-09-16T09:27:52.321170+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:23.494642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:23.494642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:29.637091+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":139,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.313053+0000","last_change":"2024-09-16T09:27:13.279354+0000","last_active":"2024-09-16T09:27:52.313053+0000","last_peered":"2024-09-16T09:27:52.313053+0000","last_clean":"2024-09-16T09:27:52.313053+0000","last_became_active":"2024-09-16T09:25:05.712545+0000","last_became_peered":"2024-09-16T09:25:05.712545+0000","last_unstale":"2024-09-16T09:27:52.313053+0000","last_undegraded":"2024-09-16T09:27:52.313053+0000","last_fullsized":"2024-09-16T09:27:52.313053+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:13.279303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:13.279303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:21.165370+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":126,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.321165+0000","last_change":"2024-09-16T09:25:07.720368+0000","last_active":"2024-09-16T09:27:52.321165+0000","last_peered":"2024-09-16T09:27:52.321165+0000","last_clean":"2024-09-16T09:27:52.321165+0000","last_became_active":"2024-09-16T09:25:07.720117+0000","last_became_peered":"2024-09-16T09:25:07.720117+0000","last_unstale":"2024-09-16T09:27:52.321165+0000","last_undegraded":"2024-09-16T09:27:52.321165+0000","last_fullsized":"2024-09-16T09:27:52.321165+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:25:06.698714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":135,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.311593+0000","last_change":"2024-09-16T09:27:52.022115+0000","last_active":"2024-09-16T09:27:52.311593+0000","last_peered":"2024-09-16T09:27:52.311593+0000","last_clean":"2024-09-16T09:27:52.311593+0000","last_became_active":"2024-09-16T09:25:07.712594+0000","last_became_peered":"2024-09-16T09:25:07.712594+0000","last_unstale":"2024-09-16T09:27:52.311593+0000","last_undegraded":"2024-09-16T09:27:52.311593+0000","last_fullsized":"2024-09-16T09:27:52.311593+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:52.021842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:52.021842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:16.470972+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":139,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.324780+0000","last_change":"2024-09-16T09:25:01.704711+0000","last_active":"2024-09-16T09:27:52.324780+0000","last_peered":"2024-09-16T09:27:52.324780+0000","last_clean":"2024-09-16T09:27:52.324780+0000","last_became_active":"2024-09-16T09:25:01.703736+0000","last_became_peered":"2024-09-16T09:25:01.703736+0000","last_unstale":"2024-09-16T09:27:52.324780+0000","last_undegraded":"2024-09-16T09:27:52.324780+0000","last_fullsized":"2024-09-16T09:27:52.324780+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:31.100509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"66'11","reported_seq":162,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.313593+0000","last_change":"2024-09-16T09:26:24.090440+0000","last_active":"2024-09-16T09:27:52.313593+0000","last_peered":"2024-09-16T09:27:52.313593+0000","last_clean":"2024-09-16T09:27:52.313593+0000","last_became_active":"2024-09-16T09:25:03.707769+0000","last_became_peered":"2024-09-16T09:25:03.707769+0000","last_unstale":"2024-09-16T09:27:52.313593+0000","last_undegraded":"2024-09-16T09:27:52.313593+0000","last_fullsized":"2024-09-16T09:27:52.313593+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-16T09:26:24.090355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:26:24.090355+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":131,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-09-16T09:27:52.332089+0000","last_change":"2024-09-16T09:25:05.712247+0000","last_active":"2024-09-16T09:27:52.332089+0000","last_peered":"2024-09-16T09:27:52.332089+0000","last_clean":"2024-09-16T09:27:52.332089+0000","last_became_active":"2024-09-16T09:25:05.711952+0000","last_became_peered":"2024-09-16T09:25:05.711952+0000","last_unstale":"2024-09-16T09:27:52.332089+0000","last_undegraded":"2024-09-16T09:27:52.332089+0000","last_fullsized":"2024-09-16T09:27:52.332089+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":16,"stat_sum":{"num_bytes":7541227544,"num_objects":115071,"num_object_clones":0,"num_object_copies":345213,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":115071,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":137187,"num_write_kb":8072193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":22626185216,"data_stored":22626173000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83687,"ondisk_log_size":83687,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":393,"ondisk_log_size":393,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":165,"num_read_kb":160,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":11,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":869,"ondisk_log_size":869,"up":33,"acting":33,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266727,"num_pgs":62,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2413116,"kb_used_data":2341612,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":91332036,"statfs":{"total":95995035648,"available":93524004864,"internally_reserved":0,"allocated":2397810688,"data_stored":2397223325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":73136586},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495660,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2352052,"kb_used_data":2288620,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":91393100,"statfs":{"total":95995035648,"available":93586534400,"internally_reserved":0,"allocated":2343546880,"data_stored":2343042701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64616902},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724595,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1911596,"kb_used_data":1844192,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":91833556,"statfs":{"total":95995035648,"available":94037561344,"internally_reserved":0,"allocated":1888452608,"data_stored":1887877448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68876742},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953528,"num_pgs":66,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2385152,"kb_used_data":2317872,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":91360000,"statfs":{"total":95995035648,"available":93552640000,"internally_reserved":0,"allocated":2373500928,"data_stored":2372909533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68876742},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149758,"num_pgs":70,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":4290376,"kb_used_data":4182716,"kb_used_omap":1,"kb_used_meta":107518,"kb_avail":89454776,"statfs":{"total":95995035648,"available":91601690624,"internally_reserved":0,"allocated":4283101184,"data_stored":4282498331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":110098887},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,5,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411395,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2891620,"kb_used_data":2810200,"kb_used_omap":1,"kb_used_meta":81150,"kb_avail":90853532,"statfs":{"total":95995035648,"available":93034016768,"internally_reserved":0,"allocated":2877644800,"data_stored":2877077295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":83098053},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574922,"num_pgs":61,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2846872,"kb_used_data":2761356,"kb_used_omap":1,"kb_used_meta":85310,"kb_avail":90898280,"statfs":{"total":95995035648,"available":93079838720,"internally_reserved":0,"allocated":2827628544,"data_stored":2827008371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87357893},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738448,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3787860,"kb_used_data":3680200,"kb_used_omap":1,"kb_used_meta":107518,"kb_avail":89957292,"statfs":{"total":95995035648,"available":92116267008,"internally_reserved":0,"allocated":3768524800,"data_stored":3767993313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":110098885},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,5,0,1,4],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":3764981760,"data_stored":3764977688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2810056704,"data_stored":2810052632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2855665664,"data_stored":2855665664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4250009600,"data_stored":4250009600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":2349924352,"data_stored":2349924352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1881079808,"data_stored":1881079808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":2338721792,"data_stored":2338717720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":2375745536,"data_stored":2375745536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T09:27:59.293 INFO:tasks.cephadm.ceph_manager.ceph:would drop below min_pgs, proj 6, currently 16 2024-09-16T09:27:59.293 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:27:59.339 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:59.340 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:59.340 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:27:59.441 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:27:59.480 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:59.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:59.481 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:27:59.581 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:27:59.610 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:27:59.611 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:59.612 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:27:59.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:59 smithi084 ceph-mon[32629]: pgmap v125: 158 pgs: 158 active+clean; 6.9 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 986 op/s 2024-09-16T09:27:59.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:59 smithi084 ceph-mon[32629]: 5.19 scrub starts 2024-09-16T09:27:59.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:59 smithi084 ceph-mon[32629]: 5.19 scrub ok 2024-09-16T09:27:59.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:59 smithi084 ceph-mon[32629]: 6.12 scrub starts 2024-09-16T09:27:59.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:27:59 smithi084 ceph-mon[32629]: 6.12 scrub ok 2024-09-16T09:27:59.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:59 smithi070 ceph-mon[30388]: pgmap v125: 158 pgs: 158 active+clean; 6.9 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 986 op/s 2024-09-16T09:27:59.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:59 smithi070 ceph-mon[30388]: 5.19 scrub starts 2024-09-16T09:27:59.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:59 smithi070 ceph-mon[30388]: 5.19 scrub ok 2024-09-16T09:27:59.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:59 smithi070 ceph-mon[30388]: 6.12 scrub starts 2024-09-16T09:27:59.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:27:59 smithi070 ceph-mon[30388]: 6.12 scrub ok 2024-09-16T09:27:59.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:59 smithi070 ceph-mon[40691]: pgmap v125: 158 pgs: 158 active+clean; 6.9 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 986 op/s 2024-09-16T09:27:59.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:59 smithi070 ceph-mon[40691]: 5.19 scrub starts 2024-09-16T09:27:59.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:59 smithi070 ceph-mon[40691]: 5.19 scrub ok 2024-09-16T09:27:59.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:59 smithi070 ceph-mon[40691]: 6.12 scrub starts 2024-09-16T09:27:59.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:27:59 smithi070 ceph-mon[40691]: 6.12 scrub ok 2024-09-16T09:27:59.713 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:27:59.755 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:59.756 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:59.756 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:27:59.857 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:27:59.893 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:27:59.894 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:27:59.894 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:27:59.995 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:28:00.037 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:00.037 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:00.038 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:28:00.139 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:28:00.165 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:00.165 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:00.166 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:28:00.252 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:28:00.266 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:28:00.290 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:00.290 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:00.291 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:28:00.391 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:28:00.427 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:00.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:00.428 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:28:00.528 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:28:00.568 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:00.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:00.569 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:28:00.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:00 smithi084 ceph-mon[32629]: 5.1b scrub starts 2024-09-16T09:28:00.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:00 smithi084 ceph-mon[32629]: 5.1b scrub ok 2024-09-16T09:28:00.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:00 smithi084 ceph-mon[32629]: 1.7 scrub starts 2024-09-16T09:28:00.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:00 smithi084 ceph-mon[32629]: 1.7 scrub ok 2024-09-16T09:28:00.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:00 smithi084 ceph-mon[32629]: pgmap v126: 158 pgs: 158 active+clean; 6.7 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.39k op/s 2024-09-16T09:28:00.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:00 smithi084 ceph-mon[32629]: 4.1a scrub starts 2024-09-16T09:28:00.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:00 smithi084 ceph-mon[32629]: 4.1a scrub ok 2024-09-16T09:28:00.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:00 smithi084 ceph-mon[32629]: 6.1c scrub starts 2024-09-16T09:28:00.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:00 smithi084 ceph-mon[32629]: 6.1c scrub ok 2024-09-16T09:28:00.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:00 smithi084 ceph-mon[32629]: 5.d scrub starts 2024-09-16T09:28:00.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:00 smithi084 ceph-mon[32629]: 5.d scrub ok 2024-09-16T09:28:00.670 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:28:00.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:00 smithi070 ceph-mon[30388]: 5.1b scrub starts 2024-09-16T09:28:00.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:00 smithi070 ceph-mon[30388]: 5.1b scrub ok 2024-09-16T09:28:00.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:00 smithi070 ceph-mon[30388]: 1.7 scrub starts 2024-09-16T09:28:00.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:00 smithi070 ceph-mon[30388]: 1.7 scrub ok 2024-09-16T09:28:00.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:00 smithi070 ceph-mon[30388]: pgmap v126: 158 pgs: 158 active+clean; 6.7 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.39k op/s 2024-09-16T09:28:00.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:00 smithi070 ceph-mon[30388]: 4.1a scrub starts 2024-09-16T09:28:00.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:00 smithi070 ceph-mon[30388]: 4.1a scrub ok 2024-09-16T09:28:00.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:00 smithi070 ceph-mon[30388]: 6.1c scrub starts 2024-09-16T09:28:00.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:00 smithi070 ceph-mon[30388]: 6.1c scrub ok 2024-09-16T09:28:00.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:00 smithi070 ceph-mon[30388]: 5.d scrub starts 2024-09-16T09:28:00.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:00 smithi070 ceph-mon[30388]: 5.d scrub ok 2024-09-16T09:28:00.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:00 smithi070 ceph-mon[40691]: 5.1b scrub starts 2024-09-16T09:28:00.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:00 smithi070 ceph-mon[40691]: 5.1b scrub ok 2024-09-16T09:28:00.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:00 smithi070 ceph-mon[40691]: 1.7 scrub starts 2024-09-16T09:28:00.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:00 smithi070 ceph-mon[40691]: 1.7 scrub ok 2024-09-16T09:28:00.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:00 smithi070 ceph-mon[40691]: pgmap v126: 158 pgs: 158 active+clean; 6.7 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.39k op/s 2024-09-16T09:28:00.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:00 smithi070 ceph-mon[40691]: 4.1a scrub starts 2024-09-16T09:28:00.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:00 smithi070 ceph-mon[40691]: 4.1a scrub ok 2024-09-16T09:28:00.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:00 smithi070 ceph-mon[40691]: 6.1c scrub starts 2024-09-16T09:28:00.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:00 smithi070 ceph-mon[40691]: 6.1c scrub ok 2024-09-16T09:28:00.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:00 smithi070 ceph-mon[40691]: 5.d scrub starts 2024-09-16T09:28:00.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:00 smithi070 ceph-mon[40691]: 5.d scrub ok 2024-09-16T09:28:00.698 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:00.699 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:00.699 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:28:00.800 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:28:00.828 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:00.829 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:00.829 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:28:00.861 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T09:28:00.930 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:28:00.975 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:00.976 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:00.976 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:28:01.077 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:28:01.101 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:01.102 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:01.102 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:28:01.184 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T09:28:01.184 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:28:01.203 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:28:01.226 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:01.227 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:01.227 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:28:01.328 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:28:01.354 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:01.355 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:01.355 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:28:01.392 INFO:teuthology.orchestra.run.smithi070.stderr:noscrub is set 2024-09-16T09:28:01.456 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:28:01.481 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:01.482 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:01.482 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:28:01.583 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:28:01.625 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:01.625 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:01.625 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:28:01.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:01 smithi084 ceph-mon[32629]: 6.1b scrub starts 2024-09-16T09:28:01.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:01 smithi084 ceph-mon[32629]: 6.1b scrub ok 2024-09-16T09:28:01.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:01 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/4107298793' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:28:01.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:01 smithi084 ceph-mon[32629]: 3.3 scrub starts 2024-09-16T09:28:01.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:01 smithi084 ceph-mon[32629]: 3.3 scrub ok 2024-09-16T09:28:01.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:01 smithi070 ceph-mon[30388]: 6.1b scrub starts 2024-09-16T09:28:01.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:01 smithi070 ceph-mon[30388]: 6.1b scrub ok 2024-09-16T09:28:01.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:01 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/4107298793' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:28:01.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:01 smithi070 ceph-mon[30388]: 3.3 scrub starts 2024-09-16T09:28:01.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:01 smithi070 ceph-mon[30388]: 3.3 scrub ok 2024-09-16T09:28:01.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:01 smithi070 ceph-mon[40691]: 6.1b scrub starts 2024-09-16T09:28:01.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:01 smithi070 ceph-mon[40691]: 6.1b scrub ok 2024-09-16T09:28:01.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:01 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/4107298793' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:28:01.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:01 smithi070 ceph-mon[40691]: 3.3 scrub starts 2024-09-16T09:28:01.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:01 smithi070 ceph-mon[40691]: 3.3 scrub ok 2024-09-16T09:28:01.727 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:28:01.753 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:01.754 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:01.754 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:28:01.855 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:28:01.896 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:01.896 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:01.896 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:28:01.997 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:28:02.035 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:02.036 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:02.036 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:28:02.137 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:28:02.166 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:02.167 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:02.167 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:28:02.170 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:28:01 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:28:01] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:28:02.268 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:28:02.293 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:02.293 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:02.294 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:28:02.304 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.1 dump_ops_in_flight 2024-09-16T09:28:02.395 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:28:02.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:02.435 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:02.435 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:28:02.536 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:28:02.564 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:02.564 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:02.565 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:28:02.651 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:02 smithi084 ceph-mon[32629]: 3.1d scrub starts 2024-09-16T09:28:02.651 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:02 smithi084 ceph-mon[32629]: 3.1d scrub ok 2024-09-16T09:28:02.651 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:02 smithi084 ceph-mon[32629]: 5.0 scrub starts 2024-09-16T09:28:02.652 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:02 smithi084 ceph-mon[32629]: 5.0 scrub ok 2024-09-16T09:28:02.652 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:02 smithi084 ceph-mon[32629]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:28:02.652 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:02 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/4107298793' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-16T09:28:02.652 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:02 smithi084 ceph-mon[32629]: osdmap e123: 8 total, 8 up, 8 in 2024-09-16T09:28:02.652 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:02 smithi084 ceph-mon[32629]: pgmap v128: 158 pgs: 158 active+clean; 6.7 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.24k op/s 2024-09-16T09:28:02.652 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:02 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:28:02.666 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:28:02.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:02 smithi070 ceph-mon[30388]: 3.1d scrub starts 2024-09-16T09:28:02.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:02 smithi070 ceph-mon[30388]: 3.1d scrub ok 2024-09-16T09:28:02.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:02 smithi070 ceph-mon[30388]: 5.0 scrub starts 2024-09-16T09:28:02.673 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:02 smithi070 ceph-mon[30388]: 5.0 scrub ok 2024-09-16T09:28:02.673 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:02 smithi070 ceph-mon[30388]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:28:02.673 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:02 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/4107298793' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-16T09:28:02.673 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:02 smithi070 ceph-mon[30388]: osdmap e123: 8 total, 8 up, 8 in 2024-09-16T09:28:02.673 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:02 smithi070 ceph-mon[30388]: pgmap v128: 158 pgs: 158 active+clean; 6.7 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.24k op/s 2024-09-16T09:28:02.673 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:02 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:28:02.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:02 smithi070 ceph-mon[40691]: 3.1d scrub starts 2024-09-16T09:28:02.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:02 smithi070 ceph-mon[40691]: 3.1d scrub ok 2024-09-16T09:28:02.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:02 smithi070 ceph-mon[40691]: 5.0 scrub starts 2024-09-16T09:28:02.674 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:02 smithi070 ceph-mon[40691]: 5.0 scrub ok 2024-09-16T09:28:02.674 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:02 smithi070 ceph-mon[40691]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:28:02.674 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:02 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/4107298793' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-16T09:28:02.674 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:02 smithi070 ceph-mon[40691]: osdmap e123: 8 total, 8 up, 8 in 2024-09-16T09:28:02.674 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:02 smithi070 ceph-mon[40691]: pgmap v128: 158 pgs: 158 active+clean; 6.7 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.24k op/s 2024-09-16T09:28:02.674 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:02 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:28:02.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:02.709 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:02.709 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:28:02.759 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:28:02.811 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:28:02.870 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:02.871 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:02.871 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:28:02.910 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:28:02 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:28:02.971 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:28:03.000 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:03.000 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:03.001 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:28:03.102 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:28:03.140 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:03.141 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:03.141 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:28:03.242 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:28:03.276 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:03.277 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:03.277 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:28:03.378 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:28:03.405 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:03.406 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:03.406 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:28:03.507 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:28:03.534 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:03.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:03.535 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:28:03.636 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:28:03.659 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:03 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:28:03.663 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:03.664 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:03.665 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:28:03.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:03 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:28:03.670 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:03 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:28:03.765 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:28:03.794 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:03.795 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:03.795 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:28:03.896 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:28:03.921 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:03.921 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:03.922 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:28:04.023 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:28:04.064 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:04.064 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:04.065 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:28:04.166 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:28:04.200 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:04.201 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:04.201 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:28:04.294 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T09:28:04.294 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T09:28:04.295 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-16T09:28:04.295 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd dump -f json-pretty 2024-09-16T09:28:04.302 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:28:04.326 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:04.326 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:04.327 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:28:04.428 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:28:04.477 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:04.478 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:04.478 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:28:04.579 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:28:04.612 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:28:04.620 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:04 smithi070 ceph-mon[40691]: pgmap v129: 158 pgs: 158 active+clean; 6.3 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.17k op/s 2024-09-16T09:28:04.620 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:28:04.621 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:28:04.621 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:04 smithi070 ceph-mon[30388]: pgmap v129: 158 pgs: 158 active+clean; 6.3 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.17k op/s 2024-09-16T09:28:04.649 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:04.650 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:04.650 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:28:04.659 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:04 smithi084 ceph-mon[32629]: pgmap v129: 158 pgs: 158 active+clean; 6.3 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.17k op/s 2024-09-16T09:28:04.665 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:28:04.665 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:28:04.698 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:28:04.698 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:28:04.734 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:28:04.734 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:28:04.752 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:28:04.762 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:28:04.762 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:28:04.781 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:04.782 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:04.782 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:28:04.794 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:28:04.794 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:28:04.817 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-16T09:28:04.830 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:28:04.830 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:28:04.865 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:28:04.865 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:28:04.883 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:28:04.907 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:04.908 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:04.908 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:28:05.009 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:28:05.065 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:05.065 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:05.066 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:28:05.167 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:28:05.192 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:05.193 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:05.194 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:28:05.295 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:28:05.318 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:05.319 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:05.320 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:28:05.421 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:28:05.440 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T09:28:05.441 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [ 2024-09-16T09:28:05.441 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:05.441 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.24698.0:152048 7.4 e123/82 7:2017ab7e:::benchmark_data_smithi070_78438_object25917:head v 123'9375, mlcod=123'9372)", 2024-09-16T09:28:05.441 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:28:05.421370+0000", 2024-09-16T09:28:05.441 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.015660973000000002, 2024-09-16T09:28:05.441 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.015686240000000001, 2024-09-16T09:28:05.441 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:28:05.441 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:28:05.441 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:28:05.441 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:28:05.441 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:05.441 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:28:05.442 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:05.421370+0000", 2024-09-16T09:28:05.442 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:28:05.442 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:05.442 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:05.442 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:28:05.442 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:05.421370+0000", 2024-09-16T09:28:05.442 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:28:05.442 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:05.442 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:05.442 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:28:05.442 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:05.421370+0000", 2024-09-16T09:28:05.442 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4900000000000001e-07 2024-09-16T09:28:05.442 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:05.442 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:05.442 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:28:05.443 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:05.421379+0000", 2024-09-16T09:28:05.443 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.7539999999999995e-06 2024-09-16T09:28:05.443 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:05.443 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:05.443 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:28:05.443 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:05.421386+0000", 2024-09-16T09:28:05.443 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.7989999999999996e-06 2024-09-16T09:28:05.443 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:05.443 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:05.443 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:28:05.444 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:05.421390+0000", 2024-09-16T09:28:05.444 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.7489999999999998e-06 2024-09-16T09:28:05.444 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:05.444 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:05.444 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:28:05.444 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:05.427586+0000", 2024-09-16T09:28:05.444 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0061953829999999996 2024-09-16T09:28:05.444 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:05.444 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:05.444 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:28:05.444 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:05.427605+0000", 2024-09-16T09:28:05.444 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9100999999999999e-05 2024-09-16T09:28:05.444 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:05.445 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:05.445 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:28:05.445 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:05.436402+0000", 2024-09-16T09:28:05.445 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0087967099999999993 2024-09-16T09:28:05.445 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:05.445 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:28:05.446 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:05.446 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:05.446 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:05.446 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.24698.0:152058 7.b 7:d107177e:::benchmark_data_smithi070_78438_object25927:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123)", 2024-09-16T09:28:05.446 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:28:05.422693+0000", 2024-09-16T09:28:05.446 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.014338012000000001, 2024-09-16T09:28:05.446 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.014426682999999999, 2024-09-16T09:28:05.446 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:28:05.446 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:28:05.446 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "waiting for sub ops", 2024-09-16T09:28:05.446 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:28:05.447 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.24698", 2024-09-16T09:28:05.447 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:28:05.447 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 152058 2024-09-16T09:28:05.447 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:05.447 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:28:05.447 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:05.447 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:28:05.447 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:05.422693+0000", 2024-09-16T09:28:05.447 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:28:05.447 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:05.447 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:05.448 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:28:05.448 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:05.422693+0000", 2024-09-16T09:28:05.448 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:28:05.448 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:05.448 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:05.448 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:28:05.448 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:05.422694+0000", 2024-09-16T09:28:05.448 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5549999999999999e-06 2024-09-16T09:28:05.448 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:05.449 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:05.449 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:28:05.449 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:05.422699+0000", 2024-09-16T09:28:05.449 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.1499999999999998e-06 2024-09-16T09:28:05.449 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:05.449 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:05.449 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:28:05.449 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:05.422716+0000", 2024-09-16T09:28:05.449 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7054e-05 2024-09-16T09:28:05.450 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:05.450 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:05.450 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:28:05.450 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:05.422725+0000", 2024-09-16T09:28:05.450 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.0420000000000003e-06 2024-09-16T09:28:05.450 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:05.450 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:05.450 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:28:05.450 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:05.430148+0000", 2024-09-16T09:28:05.450 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0074233449999999996 2024-09-16T09:28:05.450 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:05.451 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:05.451 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:28:05.451 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:05.430245+0000", 2024-09-16T09:28:05.451 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.6798000000000004e-05 2024-09-16T09:28:05.451 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:05.451 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:05.451 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 5,6", 2024-09-16T09:28:05.451 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:05.430336+0000", 2024-09-16T09:28:05.451 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.1349999999999998e-05 2024-09-16T09:28:05.451 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:05.451 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:05.451 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:28:05.451 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:05.432941+0000", 2024-09-16T09:28:05.451 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.002605099 2024-09-16T09:28:05.452 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:05.452 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:05.452 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:28:05.452 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:05.435471+0000", 2024-09-16T09:28:05.452 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0025303880000000002 2024-09-16T09:28:05.452 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:05.452 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:28:05.452 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:05.452 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:05.452 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:05.452 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.24698.0:152054 7.2 e123/82 7:406f7def:::benchmark_data_smithi070_78438_object25923:head v 123'9582, mlcod=123'9580)", 2024-09-16T09:28:05.452 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:28:05.424516+0000", 2024-09-16T09:28:05.452 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.012514704999999999, 2024-09-16T09:28:05.453 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.012675067, 2024-09-16T09:28:05.453 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:28:05.453 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:28:05.453 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:28:05.453 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:28:05.453 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:05.453 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:28:05.453 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:05.424516+0000", 2024-09-16T09:28:05.453 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:28:05.453 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:05.453 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:05.453 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:28:05.453 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:05.424516+0000", 2024-09-16T09:28:05.454 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:28:05.454 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:05.454 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:05.454 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:28:05.454 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:05.424516+0000", 2024-09-16T09:28:05.454 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9099999999999999e-07 2024-09-16T09:28:05.454 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:05.454 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:05.454 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:28:05.454 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:05.424520+0000", 2024-09-16T09:28:05.454 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8e-06 2024-09-16T09:28:05.454 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:05.455 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:05.455 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:28:05.455 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:05.424527+0000", 2024-09-16T09:28:05.455 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.674e-06 2024-09-16T09:28:05.455 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:05.455 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:05.455 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:28:05.455 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:05.424531+0000", 2024-09-16T09:28:05.455 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2599999999999999e-06 2024-09-16T09:28:05.455 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:05.456 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:05.456 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:28:05.456 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:05.424601+0000", 2024-09-16T09:28:05.456 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.0098000000000005e-05 2024-09-16T09:28:05.456 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:05.456 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:05.456 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:28:05.456 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:05.424620+0000", 2024-09-16T09:28:05.456 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.8525999999999998e-05 2024-09-16T09:28:05.456 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:05.456 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:05.456 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:28:05.456 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:05.435591+0000", 2024-09-16T09:28:05.457 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.010971292000000001 2024-09-16T09:28:05.457 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:05.457 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:28:05.457 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:05.457 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:05.457 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:05.457 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop_reply(client.24698.0:152058 7.b e123/82)", 2024-09-16T09:28:05.457 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:28:05.435497+0000", 2024-09-16T09:28:05.457 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.0015338109999999999, 2024-09-16T09:28:05.457 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001742464, 2024-09-16T09:28:05.457 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:28:05.457 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:28:05.458 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T09:28:05.458 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:28:05.458 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:05.458 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:28:05.458 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:05.435497+0000", 2024-09-16T09:28:05.458 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:28:05.458 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:05.458 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:05.458 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:28:05.458 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:05.435497+0000", 2024-09-16T09:28:05.458 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:28:05.458 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:05.458 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:05.459 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:28:05.459 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:05.435498+0000", 2024-09-16T09:28:05.459 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2330000000000001e-06 2024-09-16T09:28:05.459 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:05.459 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:05.459 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:28:05.459 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:05.435504+0000", 2024-09-16T09:28:05.459 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4269999999999999e-06 2024-09-16T09:28:05.459 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:05.459 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:05.459 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:28:05.459 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:05.435522+0000", 2024-09-16T09:28:05.460 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.8102e-05 2024-09-16T09:28:05.460 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:05.460 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:05.460 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:28:05.460 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:05.435530+0000", 2024-09-16T09:28:05.460 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.0320000000000003e-06 2024-09-16T09:28:05.460 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:05.460 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:05.460 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:28:05.460 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:05.435714+0000", 2024-09-16T09:28:05.460 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00018404000000000001 2024-09-16T09:28:05.460 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:05.460 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:28:05.460 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:05.461 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:05.461 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-16T09:28:05.461 INFO:teuthology.orchestra.run.smithi070.stdout: "num_ops": 4 2024-09-16T09:28:05.461 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T09:28:05.491 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:05.492 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:05.492 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:28:05.593 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:28:05.622 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:05.622 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:05.623 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:28:05.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:05 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/737718714' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:28:05.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:05 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/737718714' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:28:05.695 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:28:05.724 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:28:05.748 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:05.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:05.749 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:28:05.850 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:28:05.892 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:05.893 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:05.893 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:28:05.909 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:05 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/737718714' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:28:05.994 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:28:06.024 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:06.024 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:06.025 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:28:06.125 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:28:06.154 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:06.154 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:06.155 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:28:06.255 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:28:06.282 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:06.283 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:06.283 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:28:06.384 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:28:06.423 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:06.424 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:06.424 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:28:06.525 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:28:06.556 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:06.557 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:06.557 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:28:06.658 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:28:06.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:06 smithi070 ceph-mon[30388]: pgmap v130: 158 pgs: 158 active+clean; 6.3 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.17k op/s 2024-09-16T09:28:06.670 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:06 smithi070 ceph-mon[40691]: pgmap v130: 158 pgs: 158 active+clean; 6.3 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.17k op/s 2024-09-16T09:28:06.687 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:06.687 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:06.688 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:28:06.788 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:28:06.822 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:06.822 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:06.823 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:28:06.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:06 smithi084 ceph-mon[32629]: pgmap v130: 158 pgs: 158 active+clean; 6.3 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.17k op/s 2024-09-16T09:28:06.923 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:28:06.960 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:06.961 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:06.961 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:28:07.062 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:28:07.089 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:07.089 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:07.090 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:28:07.190 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:28:07.211 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:07.212 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:07.212 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:28:07.313 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:28:07.350 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:07.351 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:07.351 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:28:07.452 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:28:07.493 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:07.494 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:07.494 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:28:07.595 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:28:07.642 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:07.642 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:07.643 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:28:07.743 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:28:07.768 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:07.768 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:07.769 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:28:07.870 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:28:07.905 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:07.906 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:07.906 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:28:08.007 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:28:08.042 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:08.043 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:08.043 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:28:08.144 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:28:08.167 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:08.168 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:08.168 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:28:08.268 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:28:08.293 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:08.294 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:08.294 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:28:08.395 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:28:08.435 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:08.436 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:08.436 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:28:08.537 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:28:08.579 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:08.579 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:08.579 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:28:08.680 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:28:08.708 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:08.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:08.709 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:28:08.810 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:28:08.833 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:08.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:08.834 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:28:08.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:08 smithi084 ceph-mon[32629]: pgmap v131: 158 pgs: 158 active+clean; 6.2 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.17k op/s 2024-09-16T09:28:08.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:08 smithi070 ceph-mon[30388]: pgmap v131: 158 pgs: 158 active+clean; 6.2 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.17k op/s 2024-09-16T09:28:08.920 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:08 smithi070 ceph-mon[40691]: pgmap v131: 158 pgs: 158 active+clean; 6.2 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.17k op/s 2024-09-16T09:28:08.935 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:28:08.958 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:08.959 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:08.959 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:28:09.003 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.1 dump_blocked_ops 2024-09-16T09:28:09.021 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:28:09.060 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:28:09.083 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:09.083 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:09.084 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:28:09.185 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:28:09.208 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:09.209 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:09.209 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:28:09.310 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:28:09.376 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:09.377 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:09.378 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:28:09.478 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:28:09.504 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:09.505 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:09.505 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:28:09.597 INFO:teuthology.orchestra.run.smithi070.stderr:nodeep-scrub is set 2024-09-16T09:28:09.605 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:28:09.646 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:09.647 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:09.647 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:28:09.748 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:28:09.773 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:09.774 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:09.774 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:28:09.875 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:28:09.909 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:09 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/920239287' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:28:09.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:09 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:28:09.917 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:09.917 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:09.918 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:28:09.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:09 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/920239287' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:28:09.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:09 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:28:09.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:09 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/920239287' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:28:09.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:09 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:28:10.019 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:28:10.046 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:10.047 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:10.047 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:28:10.120 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:28:10.122 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T09:28:10.123 INFO:teuthology.orchestra.run.smithi070.stdout: "epoch": 124, 2024-09-16T09:28:10.123 INFO:teuthology.orchestra.run.smithi070.stdout: "fsid": "a1321834-740c-11ef-bceb-c7b262605968", 2024-09-16T09:28:10.123 INFO:teuthology.orchestra.run.smithi070.stdout: "created": "2024-09-16T09:19:14.121498+0000", 2024-09-16T09:28:10.123 INFO:teuthology.orchestra.run.smithi070.stdout: "modified": "2024-09-16T09:28:09.587262+0000", 2024-09-16T09:28:10.123 INFO:teuthology.orchestra.run.smithi070.stdout: "last_up_change": "2024-09-16T09:24:51.124496+0000", 2024-09-16T09:28:10.123 INFO:teuthology.orchestra.run.smithi070.stdout: "last_in_change": "2024-09-16T09:24:28.432920+0000", 2024-09-16T09:28:10.123 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-16T09:28:10.123 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_num": 5806080, 2024-09-16T09:28:10.123 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_set": [ 2024-09-16T09:28:10.123 INFO:teuthology.orchestra.run.smithi070.stdout: "nodeep-scrub", 2024-09-16T09:28:10.123 INFO:teuthology.orchestra.run.smithi070.stdout: "noscrub", 2024-09-16T09:28:10.123 INFO:teuthology.orchestra.run.smithi070.stdout: "pglog_hardlimit", 2024-09-16T09:28:10.123 INFO:teuthology.orchestra.run.smithi070.stdout: "purged_snapdirs", 2024-09-16T09:28:10.124 INFO:teuthology.orchestra.run.smithi070.stdout: "recovery_deletes", 2024-09-16T09:28:10.124 INFO:teuthology.orchestra.run.smithi070.stdout: "sortbitwise" 2024-09-16T09:28:10.124 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-16T09:28:10.124 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_version": 18, 2024-09-16T09:28:10.124 INFO:teuthology.orchestra.run.smithi070.stdout: "full_ratio": 0.94999998807907104, 2024-09-16T09:28:10.124 INFO:teuthology.orchestra.run.smithi070.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-16T09:28:10.124 INFO:teuthology.orchestra.run.smithi070.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-16T09:28:10.124 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_snapshot": "", 2024-09-16T09:28:10.124 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_max": 7, 2024-09-16T09:28:10.124 INFO:teuthology.orchestra.run.smithi070.stdout: "max_osd": 8, 2024-09-16T09:28:10.124 INFO:teuthology.orchestra.run.smithi070.stdout: "require_min_compat_client": "luminous", 2024-09-16T09:28:10.124 INFO:teuthology.orchestra.run.smithi070.stdout: "min_compat_client": "luminous", 2024-09-16T09:28:10.125 INFO:teuthology.orchestra.run.smithi070.stdout: "require_osd_release": "squid", 2024-09-16T09:28:10.125 INFO:teuthology.orchestra.run.smithi070.stdout: "allow_crimson": false, 2024-09-16T09:28:10.125 INFO:teuthology.orchestra.run.smithi070.stdout: "pools": [ 2024-09-16T09:28:10.125 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:10.125 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 1, 2024-09-16T09:28:10.125 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": ".mgr", 2024-09-16T09:28:10.125 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2024-09-16T09:22:27.617694+0000", 2024-09-16T09:28:10.125 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2024-09-16T09:28:10.125 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2024-09-16T09:28:10.125 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2024-09-16T09:28:10.125 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 3, 2024-09-16T09:28:10.125 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 2, 2024-09-16T09:28:10.125 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2024-09-16T09:28:10.125 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2024-09-16T09:28:10.126 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2024-09-16T09:28:10.126 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T09:28:10.126 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T09:28:10.126 INFO:teuthology.orchestra.run.smithi070.stdout: "is_stretch_pool": false, 2024-09-16T09:28:10.126 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2024-09-16T09:28:10.126 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2024-09-16T09:28:10.126 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 11, 2024-09-16T09:28:10.126 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 11, 2024-09-16T09:28:10.126 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 11, 2024-09-16T09:28:10.126 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 11, 2024-09-16T09:28:10.126 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 11, 2024-09-16T09:28:10.126 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2024-09-16T09:28:10.126 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2024-09-16T09:28:10.127 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2024-09-16T09:28:10.127 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2024-09-16T09:28:10.127 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2024-09-16T09:28:10.127 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2024-09-16T09:28:10.127 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2024-09-16T09:28:10.127 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.127 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "94", 2024-09-16T09:28:10.127 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2024-09-16T09:28:10.127 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T09:28:10.127 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "92", 2024-09-16T09:28:10.127 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2024-09-16T09:28:10.127 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2024-09-16T09:28:10.127 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2024-09-16T09:28:10.128 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2024-09-16T09:28:10.128 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2024-09-16T09:28:10.128 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2024-09-16T09:28:10.128 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2024-09-16T09:28:10.128 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2024-09-16T09:28:10.128 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2024-09-16T09:28:10.128 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2024-09-16T09:28:10.128 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2024-09-16T09:28:10.128 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2024-09-16T09:28:10.128 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2024-09-16T09:28:10.128 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2024-09-16T09:28:10.128 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2024-09-16T09:28:10.128 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T09:28:10.129 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T09:28:10.129 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T09:28:10.129 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2024-09-16T09:28:10.129 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2024-09-16T09:28:10.129 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2024-09-16T09:28:10.129 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2024-09-16T09:28:10.129 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2024-09-16T09:28:10.129 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.129 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2024-09-16T09:28:10.129 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2024-09-16T09:28:10.129 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2024-09-16T09:28:10.129 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2024-09-16T09:28:10.129 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2024-09-16T09:28:10.129 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T09:28:10.130 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2024-09-16T09:28:10.130 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2024-09-16T09:28:10.130 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2024-09-16T09:28:10.130 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2024-09-16T09:28:10.130 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2024-09-16T09:28:10.130 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2024-09-16T09:28:10.130 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_max": 32, 2024-09-16T09:28:10.130 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_min": 1 2024-09-16T09:28:10.130 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.130 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2024-09-16T09:28:10.130 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": {} 2024-09-16T09:28:10.130 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.130 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2024-09-16T09:28:10.130 INFO:teuthology.orchestra.run.smithi070.stdout: "score_type": "Fair distribution", 2024-09-16T09:28:10.131 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 2.1800000667572021, 2024-09-16T09:28:10.131 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 2.1800000667572021, 2024-09-16T09:28:10.131 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1, 2024-09-16T09:28:10.131 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 2.1800000667572021, 2024-09-16T09:28:10.131 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 2.1800000667572021, 2024-09-16T09:28:10.131 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2024-09-16T09:28:10.131 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2024-09-16T09:28:10.131 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2024-09-16T09:28:10.131 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:10.132 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.132 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:10.132 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 2, 2024-09-16T09:28:10.132 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "datapool", 2024-09-16T09:28:10.132 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2024-09-16T09:24:59.169244+0000", 2024-09-16T09:28:10.132 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 8193, 2024-09-16T09:28:10.132 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-16T09:28:10.132 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2024-09-16T09:28:10.132 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 3, 2024-09-16T09:28:10.132 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 2, 2024-09-16T09:28:10.132 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2024-09-16T09:28:10.133 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2024-09-16T09:28:10.133 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2024-09-16T09:28:10.133 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T09:28:10.133 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T09:28:10.133 INFO:teuthology.orchestra.run.smithi070.stdout: "is_stretch_pool": false, 2024-09-16T09:28:10.133 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2024-09-16T09:28:10.133 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2024-09-16T09:28:10.133 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 3, 2024-09-16T09:28:10.133 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 3, 2024-09-16T09:28:10.133 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 3, 2024-09-16T09:28:10.133 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 3, 2024-09-16T09:28:10.133 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 3, 2024-09-16T09:28:10.133 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2024-09-16T09:28:10.133 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2024-09-16T09:28:10.134 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2024-09-16T09:28:10.134 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2024-09-16T09:28:10.134 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2024-09-16T09:28:10.134 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2024-09-16T09:28:10.134 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2024-09-16T09:28:10.134 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.134 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "61", 2024-09-16T09:28:10.134 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2024-09-16T09:28:10.134 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T09:28:10.134 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T09:28:10.134 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2024-09-16T09:28:10.134 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2024-09-16T09:28:10.134 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 2, 2024-09-16T09:28:10.134 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 61, 2024-09-16T09:28:10.134 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2024-09-16T09:28:10.134 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2024-09-16T09:28:10.135 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2024-09-16T09:28:10.135 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2024-09-16T09:28:10.135 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2024-09-16T09:28:10.135 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2024-09-16T09:28:10.135 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2024-09-16T09:28:10.135 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2024-09-16T09:28:10.135 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2024-09-16T09:28:10.135 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2024-09-16T09:28:10.135 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2024-09-16T09:28:10.135 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T09:28:10.135 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T09:28:10.135 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T09:28:10.135 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2024-09-16T09:28:10.136 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2024-09-16T09:28:10.136 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2024-09-16T09:28:10.136 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2024-09-16T09:28:10.136 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2024-09-16T09:28:10.136 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.136 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2024-09-16T09:28:10.136 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2024-09-16T09:28:10.136 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2024-09-16T09:28:10.136 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2024-09-16T09:28:10.136 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2024-09-16T09:28:10.136 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T09:28:10.136 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2024-09-16T09:28:10.136 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2024-09-16T09:28:10.136 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2024-09-16T09:28:10.136 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2024-09-16T09:28:10.137 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2024-09-16T09:28:10.137 INFO:teuthology.orchestra.run.smithi070.stdout: "options": {}, 2024-09-16T09:28:10.137 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2024-09-16T09:28:10.137 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd": {} 2024-09-16T09:28:10.137 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.137 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2024-09-16T09:28:10.137 INFO:teuthology.orchestra.run.smithi070.stdout: "score_type": "Fair distribution", 2024-09-16T09:28:10.137 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 2.6500000953674316, 2024-09-16T09:28:10.138 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 2.6500000953674316, 2024-09-16T09:28:10.138 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 0.87999999523162842, 2024-09-16T09:28:10.138 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 2.3299999237060547, 2024-09-16T09:28:10.138 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 2.3299999237060547, 2024-09-16T09:28:10.138 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2024-09-16T09:28:10.138 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2024-09-16T09:28:10.138 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2024-09-16T09:28:10.138 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:10.138 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.138 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:10.138 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 3, 2024-09-16T09:28:10.138 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": ".rgw.root", 2024-09-16T09:28:10.139 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2024-09-16T09:25:00.056119+0000", 2024-09-16T09:28:10.139 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2024-09-16T09:28:10.139 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2024-09-16T09:28:10.139 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2024-09-16T09:28:10.139 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 3, 2024-09-16T09:28:10.139 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 2, 2024-09-16T09:28:10.139 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2024-09-16T09:28:10.139 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2024-09-16T09:28:10.139 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2024-09-16T09:28:10.139 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T09:28:10.139 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T09:28:10.139 INFO:teuthology.orchestra.run.smithi070.stdout: "is_stretch_pool": false, 2024-09-16T09:28:10.139 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2024-09-16T09:28:10.139 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2024-09-16T09:28:10.139 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 32, 2024-09-16T09:28:10.140 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 32, 2024-09-16T09:28:10.140 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 32, 2024-09-16T09:28:10.140 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 32, 2024-09-16T09:28:10.140 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 32, 2024-09-16T09:28:10.140 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2024-09-16T09:28:10.140 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2024-09-16T09:28:10.140 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2024-09-16T09:28:10.140 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2024-09-16T09:28:10.140 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2024-09-16T09:28:10.140 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2024-09-16T09:28:10.140 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2024-09-16T09:28:10.140 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.140 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "60", 2024-09-16T09:28:10.141 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2024-09-16T09:28:10.141 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T09:28:10.141 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T09:28:10.141 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2024-09-16T09:28:10.141 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2024-09-16T09:28:10.141 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2024-09-16T09:28:10.141 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2024-09-16T09:28:10.141 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2024-09-16T09:28:10.141 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2024-09-16T09:28:10.141 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2024-09-16T09:28:10.141 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2024-09-16T09:28:10.141 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2024-09-16T09:28:10.141 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2024-09-16T09:28:10.142 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2024-09-16T09:28:10.142 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2024-09-16T09:28:10.142 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2024-09-16T09:28:10.142 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2024-09-16T09:28:10.142 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2024-09-16T09:28:10.142 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T09:28:10.142 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T09:28:10.142 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T09:28:10.142 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2024-09-16T09:28:10.142 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2024-09-16T09:28:10.142 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2024-09-16T09:28:10.142 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2024-09-16T09:28:10.142 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2024-09-16T09:28:10.142 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.143 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2024-09-16T09:28:10.143 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2024-09-16T09:28:10.143 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2024-09-16T09:28:10.143 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2024-09-16T09:28:10.143 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2024-09-16T09:28:10.143 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T09:28:10.143 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2024-09-16T09:28:10.143 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2024-09-16T09:28:10.143 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2024-09-16T09:28:10.143 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2024-09-16T09:28:10.143 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2024-09-16T09:28:10.143 INFO:teuthology.orchestra.run.smithi070.stdout: "options": {}, 2024-09-16T09:28:10.143 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2024-09-16T09:28:10.143 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw": {} 2024-09-16T09:28:10.144 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.144 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2024-09-16T09:28:10.144 INFO:teuthology.orchestra.run.smithi070.stdout: "score_type": "Fair distribution", 2024-09-16T09:28:10.144 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 1.5, 2024-09-16T09:28:10.144 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 1.5, 2024-09-16T09:28:10.144 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1, 2024-09-16T09:28:10.144 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 1.5, 2024-09-16T09:28:10.144 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 1.5, 2024-09-16T09:28:10.144 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2024-09-16T09:28:10.144 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2024-09-16T09:28:10.144 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2024-09-16T09:28:10.144 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:10.144 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.145 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:10.145 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 4, 2024-09-16T09:28:10.145 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "default.rgw.log", 2024-09-16T09:28:10.145 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2024-09-16T09:25:01.770016+0000", 2024-09-16T09:28:10.145 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2024-09-16T09:28:10.145 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2024-09-16T09:28:10.145 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2024-09-16T09:28:10.145 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 3, 2024-09-16T09:28:10.145 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 2, 2024-09-16T09:28:10.145 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2024-09-16T09:28:10.145 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2024-09-16T09:28:10.145 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2024-09-16T09:28:10.145 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T09:28:10.145 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T09:28:10.146 INFO:teuthology.orchestra.run.smithi070.stdout: "is_stretch_pool": false, 2024-09-16T09:28:10.146 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2024-09-16T09:28:10.146 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2024-09-16T09:28:10.146 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 32, 2024-09-16T09:28:10.146 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 32, 2024-09-16T09:28:10.146 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 32, 2024-09-16T09:28:10.146 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 32, 2024-09-16T09:28:10.146 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 32, 2024-09-16T09:28:10.146 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2024-09-16T09:28:10.146 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2024-09-16T09:28:10.146 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2024-09-16T09:28:10.146 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2024-09-16T09:28:10.146 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2024-09-16T09:28:10.146 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2024-09-16T09:28:10.147 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2024-09-16T09:28:10.147 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.147 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "62", 2024-09-16T09:28:10.147 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2024-09-16T09:28:10.147 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T09:28:10.147 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T09:28:10.147 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2024-09-16T09:28:10.147 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2024-09-16T09:28:10.147 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2024-09-16T09:28:10.147 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2024-09-16T09:28:10.147 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2024-09-16T09:28:10.147 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2024-09-16T09:28:10.147 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2024-09-16T09:28:10.147 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2024-09-16T09:28:10.148 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2024-09-16T09:28:10.148 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2024-09-16T09:28:10.148 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2024-09-16T09:28:10.148 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2024-09-16T09:28:10.148 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2024-09-16T09:28:10.148 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2024-09-16T09:28:10.148 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2024-09-16T09:28:10.148 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T09:28:10.148 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T09:28:10.148 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T09:28:10.148 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2024-09-16T09:28:10.148 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2024-09-16T09:28:10.148 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2024-09-16T09:28:10.149 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2024-09-16T09:28:10.149 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2024-09-16T09:28:10.149 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.149 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2024-09-16T09:28:10.149 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2024-09-16T09:28:10.149 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2024-09-16T09:28:10.149 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2024-09-16T09:28:10.149 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2024-09-16T09:28:10.149 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T09:28:10.149 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2024-09-16T09:28:10.149 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2024-09-16T09:28:10.149 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2024-09-16T09:28:10.149 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2024-09-16T09:28:10.150 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2024-09-16T09:28:10.150 INFO:teuthology.orchestra.run.smithi070.stdout: "options": {}, 2024-09-16T09:28:10.150 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2024-09-16T09:28:10.150 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw": {} 2024-09-16T09:28:10.150 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.150 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2024-09-16T09:28:10.150 INFO:teuthology.orchestra.run.smithi070.stdout: "score_type": "Fair distribution", 2024-09-16T09:28:10.150 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 2.25, 2024-09-16T09:28:10.150 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 2.25, 2024-09-16T09:28:10.150 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1, 2024-09-16T09:28:10.150 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 2.25, 2024-09-16T09:28:10.150 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 2.25, 2024-09-16T09:28:10.150 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2024-09-16T09:28:10.150 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2024-09-16T09:28:10.151 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2024-09-16T09:28:10.151 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:10.151 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.151 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:10.151 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 5, 2024-09-16T09:28:10.151 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "default.rgw.control", 2024-09-16T09:28:10.151 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2024-09-16T09:25:03.725555+0000", 2024-09-16T09:28:10.151 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2024-09-16T09:28:10.151 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2024-09-16T09:28:10.151 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2024-09-16T09:28:10.151 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 3, 2024-09-16T09:28:10.151 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 2, 2024-09-16T09:28:10.151 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2024-09-16T09:28:10.151 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2024-09-16T09:28:10.152 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2024-09-16T09:28:10.152 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T09:28:10.152 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T09:28:10.152 INFO:teuthology.orchestra.run.smithi070.stdout: "is_stretch_pool": false, 2024-09-16T09:28:10.152 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2024-09-16T09:28:10.152 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2024-09-16T09:28:10.152 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 32, 2024-09-16T09:28:10.152 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 32, 2024-09-16T09:28:10.152 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 32, 2024-09-16T09:28:10.152 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 32, 2024-09-16T09:28:10.152 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 32, 2024-09-16T09:28:10.152 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2024-09-16T09:28:10.152 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2024-09-16T09:28:10.152 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2024-09-16T09:28:10.153 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2024-09-16T09:28:10.153 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2024-09-16T09:28:10.153 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2024-09-16T09:28:10.153 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2024-09-16T09:28:10.153 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.153 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "64", 2024-09-16T09:28:10.153 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2024-09-16T09:28:10.153 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T09:28:10.153 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T09:28:10.153 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2024-09-16T09:28:10.153 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2024-09-16T09:28:10.153 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2024-09-16T09:28:10.153 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2024-09-16T09:28:10.154 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2024-09-16T09:28:10.154 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2024-09-16T09:28:10.154 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2024-09-16T09:28:10.154 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2024-09-16T09:28:10.154 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2024-09-16T09:28:10.154 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2024-09-16T09:28:10.154 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2024-09-16T09:28:10.154 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2024-09-16T09:28:10.154 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2024-09-16T09:28:10.154 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2024-09-16T09:28:10.154 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2024-09-16T09:28:10.154 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T09:28:10.154 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T09:28:10.154 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T09:28:10.154 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2024-09-16T09:28:10.154 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2024-09-16T09:28:10.155 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2024-09-16T09:28:10.155 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2024-09-16T09:28:10.155 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2024-09-16T09:28:10.155 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.155 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2024-09-16T09:28:10.155 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2024-09-16T09:28:10.155 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2024-09-16T09:28:10.155 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2024-09-16T09:28:10.155 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2024-09-16T09:28:10.155 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T09:28:10.155 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2024-09-16T09:28:10.155 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2024-09-16T09:28:10.155 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2024-09-16T09:28:10.155 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2024-09-16T09:28:10.156 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2024-09-16T09:28:10.156 INFO:teuthology.orchestra.run.smithi070.stdout: "options": {}, 2024-09-16T09:28:10.156 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2024-09-16T09:28:10.156 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw": {} 2024-09-16T09:28:10.156 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.156 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2024-09-16T09:28:10.156 INFO:teuthology.orchestra.run.smithi070.stdout: "score_type": "Fair distribution", 2024-09-16T09:28:10.156 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 1.25, 2024-09-16T09:28:10.156 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 1.25, 2024-09-16T09:28:10.156 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1, 2024-09-16T09:28:10.156 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 1.25, 2024-09-16T09:28:10.156 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 1.25, 2024-09-16T09:28:10.156 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2024-09-16T09:28:10.157 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2024-09-16T09:28:10.157 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:28:10.158 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2024-09-16T09:28:10.158 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:10.158 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.159 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:10.159 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 6, 2024-09-16T09:28:10.159 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "default.rgw.meta", 2024-09-16T09:28:10.159 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2024-09-16T09:25:05.773530+0000", 2024-09-16T09:28:10.159 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2024-09-16T09:28:10.159 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2024-09-16T09:28:10.159 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2024-09-16T09:28:10.159 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 3, 2024-09-16T09:28:10.159 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 2, 2024-09-16T09:28:10.159 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2024-09-16T09:28:10.159 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2024-09-16T09:28:10.159 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2024-09-16T09:28:10.159 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T09:28:10.159 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T09:28:10.160 INFO:teuthology.orchestra.run.smithi070.stdout: "is_stretch_pool": false, 2024-09-16T09:28:10.160 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2024-09-16T09:28:10.160 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2024-09-16T09:28:10.160 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 32, 2024-09-16T09:28:10.160 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 32, 2024-09-16T09:28:10.160 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 32, 2024-09-16T09:28:10.160 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 32, 2024-09-16T09:28:10.160 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 32, 2024-09-16T09:28:10.160 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2024-09-16T09:28:10.160 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2024-09-16T09:28:10.160 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2024-09-16T09:28:10.161 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2024-09-16T09:28:10.161 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2024-09-16T09:28:10.161 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2024-09-16T09:28:10.161 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2024-09-16T09:28:10.161 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.161 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "66", 2024-09-16T09:28:10.161 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2024-09-16T09:28:10.161 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T09:28:10.161 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T09:28:10.161 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2024-09-16T09:28:10.161 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2024-09-16T09:28:10.161 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2024-09-16T09:28:10.161 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2024-09-16T09:28:10.161 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2024-09-16T09:28:10.162 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2024-09-16T09:28:10.162 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2024-09-16T09:28:10.162 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2024-09-16T09:28:10.162 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2024-09-16T09:28:10.162 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2024-09-16T09:28:10.162 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2024-09-16T09:28:10.162 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2024-09-16T09:28:10.162 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2024-09-16T09:28:10.162 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2024-09-16T09:28:10.162 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2024-09-16T09:28:10.162 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T09:28:10.162 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T09:28:10.162 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T09:28:10.162 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2024-09-16T09:28:10.163 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2024-09-16T09:28:10.163 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2024-09-16T09:28:10.163 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2024-09-16T09:28:10.163 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2024-09-16T09:28:10.163 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.163 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2024-09-16T09:28:10.163 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2024-09-16T09:28:10.163 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2024-09-16T09:28:10.163 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2024-09-16T09:28:10.163 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2024-09-16T09:28:10.163 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T09:28:10.163 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2024-09-16T09:28:10.163 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2024-09-16T09:28:10.163 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2024-09-16T09:28:10.164 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2024-09-16T09:28:10.164 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2024-09-16T09:28:10.164 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2024-09-16T09:28:10.164 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_bias": 4 2024-09-16T09:28:10.164 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.164 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2024-09-16T09:28:10.164 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw": {} 2024-09-16T09:28:10.164 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.164 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2024-09-16T09:28:10.164 INFO:teuthology.orchestra.run.smithi070.stdout: "score_type": "Fair distribution", 2024-09-16T09:28:10.164 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 1.75, 2024-09-16T09:28:10.164 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 1.75, 2024-09-16T09:28:10.164 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1, 2024-09-16T09:28:10.165 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 1.75, 2024-09-16T09:28:10.165 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 1.75, 2024-09-16T09:28:10.165 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2024-09-16T09:28:10.165 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2024-09-16T09:28:10.165 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2024-09-16T09:28:10.165 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:10.165 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.165 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:10.165 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 7, 2024-09-16T09:28:10.165 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "unique_pool_0", 2024-09-16T09:28:10.165 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2024-09-16T09:26:06.241945+0000", 2024-09-16T09:28:10.165 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2024-09-16T09:28:10.165 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2024-09-16T09:28:10.166 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2024-09-16T09:28:10.166 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 3, 2024-09-16T09:28:10.166 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 2, 2024-09-16T09:28:10.166 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2024-09-16T09:28:10.166 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2024-09-16T09:28:10.166 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2024-09-16T09:28:10.166 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T09:28:10.166 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T09:28:10.166 INFO:teuthology.orchestra.run.smithi070.stdout: "is_stretch_pool": false, 2024-09-16T09:28:10.166 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2024-09-16T09:28:10.166 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2024-09-16T09:28:10.166 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 16, 2024-09-16T09:28:10.166 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 16, 2024-09-16T09:28:10.166 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 16, 2024-09-16T09:28:10.167 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 16, 2024-09-16T09:28:10.167 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 16, 2024-09-16T09:28:10.167 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2024-09-16T09:28:10.167 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2024-09-16T09:28:10.167 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2024-09-16T09:28:10.167 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2024-09-16T09:28:10.167 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2024-09-16T09:28:10.167 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2024-09-16T09:28:10.167 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2024-09-16T09:28:10.167 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.167 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "85", 2024-09-16T09:28:10.167 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2024-09-16T09:28:10.167 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T09:28:10.168 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T09:28:10.168 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2024-09-16T09:28:10.168 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2024-09-16T09:28:10.168 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2024-09-16T09:28:10.168 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2024-09-16T09:28:10.168 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2024-09-16T09:28:10.168 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2024-09-16T09:28:10.168 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2024-09-16T09:28:10.168 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2024-09-16T09:28:10.168 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2024-09-16T09:28:10.168 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2024-09-16T09:28:10.168 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2024-09-16T09:28:10.168 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2024-09-16T09:28:10.168 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2024-09-16T09:28:10.169 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2024-09-16T09:28:10.169 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2024-09-16T09:28:10.169 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T09:28:10.169 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T09:28:10.169 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T09:28:10.169 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2024-09-16T09:28:10.169 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2024-09-16T09:28:10.169 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2024-09-16T09:28:10.169 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2024-09-16T09:28:10.169 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2024-09-16T09:28:10.169 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.169 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2024-09-16T09:28:10.169 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2024-09-16T09:28:10.169 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2024-09-16T09:28:10.170 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2024-09-16T09:28:10.170 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2024-09-16T09:28:10.170 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T09:28:10.170 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2024-09-16T09:28:10.170 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2024-09-16T09:28:10.170 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2024-09-16T09:28:10.170 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2024-09-16T09:28:10.170 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2024-09-16T09:28:10.170 INFO:teuthology.orchestra.run.smithi070.stdout: "options": {}, 2024-09-16T09:28:10.170 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2024-09-16T09:28:10.170 INFO:teuthology.orchestra.run.smithi070.stdout: "rados": {} 2024-09-16T09:28:10.170 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.170 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2024-09-16T09:28:10.171 INFO:teuthology.orchestra.run.smithi070.stdout: "score_type": "Fair distribution", 2024-09-16T09:28:10.171 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 2, 2024-09-16T09:28:10.171 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 2, 2024-09-16T09:28:10.171 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1, 2024-09-16T09:28:10.171 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 2, 2024-09-16T09:28:10.171 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 2, 2024-09-16T09:28:10.171 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2024-09-16T09:28:10.171 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2024-09-16T09:28:10.171 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2024-09-16T09:28:10.171 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:10.171 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:10.171 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-16T09:28:10.171 INFO:teuthology.orchestra.run.smithi070.stdout: "osds": [ 2024-09-16T09:28:10.171 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:10.172 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 0, 2024-09-16T09:28:10.172 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "7ef288c8-3bf0-4814-8e86-f70a7de041ce", 2024-09-16T09:28:10.172 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2024-09-16T09:28:10.172 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2024-09-16T09:28:10.172 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2024-09-16T09:28:10.172 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2024-09-16T09:28:10.172 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2024-09-16T09:28:10.172 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2024-09-16T09:28:10.172 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 8, 2024-09-16T09:28:10.172 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 82, 2024-09-16T09:28:10.172 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2024-09-16T09:28:10.172 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2024-09-16T09:28:10.172 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2024-09-16T09:28:10.172 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:28:10.173 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:10.173 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:28:10.173 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6802", 2024-09-16T09:28:10.173 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2013075781 2024-09-16T09:28:10.173 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.173 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:10.173 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:28:10.173 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6803", 2024-09-16T09:28:10.173 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2013075781 2024-09-16T09:28:10.173 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:10.173 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:28:10.173 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.173 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2024-09-16T09:28:10.174 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:28:10.174 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:10.174 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:28:10.174 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6804", 2024-09-16T09:28:10.174 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2013075781 2024-09-16T09:28:10.174 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.174 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:10.174 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:28:10.174 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6805", 2024-09-16T09:28:10.174 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2013075781 2024-09-16T09:28:10.174 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:10.174 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:28:10.174 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.174 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2024-09-16T09:28:10.175 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:28:10.175 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:10.175 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:28:10.175 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6808", 2024-09-16T09:28:10.175 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2013075781 2024-09-16T09:28:10.175 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.175 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:10.175 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:28:10.175 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6809", 2024-09-16T09:28:10.175 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2013075781 2024-09-16T09:28:10.175 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:10.175 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:28:10.175 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.175 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2024-09-16T09:28:10.176 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:28:10.176 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:10.176 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:28:10.176 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6806", 2024-09-16T09:28:10.176 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2013075781 2024-09-16T09:28:10.176 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.176 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:10.176 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:28:10.176 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6807", 2024-09-16T09:28:10.176 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2013075781 2024-09-16T09:28:10.176 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:10.176 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:28:10.177 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.177 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6803/2013075781", 2024-09-16T09:28:10.177 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6805/2013075781", 2024-09-16T09:28:10.177 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6809/2013075781", 2024-09-16T09:28:10.177 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6807/2013075781", 2024-09-16T09:28:10.177 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2024-09-16T09:28:10.177 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2024-09-16T09:28:10.177 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2024-09-16T09:28:10.177 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:28:10.178 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.178 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:10.178 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 1, 2024-09-16T09:28:10.178 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "474d41d9-295a-450c-8f27-ce487a14678b", 2024-09-16T09:28:10.178 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2024-09-16T09:28:10.178 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2024-09-16T09:28:10.178 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2024-09-16T09:28:10.178 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2024-09-16T09:28:10.178 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2024-09-16T09:28:10.178 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2024-09-16T09:28:10.178 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2024-09-16T09:28:10.178 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 94, 2024-09-16T09:28:10.178 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2024-09-16T09:28:10.179 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2024-09-16T09:28:10.179 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2024-09-16T09:28:10.179 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:28:10.179 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:10.179 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:28:10.179 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6810", 2024-09-16T09:28:10.179 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1670012115 2024-09-16T09:28:10.179 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.179 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:10.179 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:28:10.179 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6811", 2024-09-16T09:28:10.179 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1670012115 2024-09-16T09:28:10.179 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:10.179 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:28:10.180 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.180 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2024-09-16T09:28:10.180 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:28:10.180 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:10.180 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:28:10.180 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6812", 2024-09-16T09:28:10.180 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1670012115 2024-09-16T09:28:10.180 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.180 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:10.180 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:28:10.180 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6813", 2024-09-16T09:28:10.180 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1670012115 2024-09-16T09:28:10.180 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:10.181 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:28:10.181 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.181 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2024-09-16T09:28:10.181 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:28:10.181 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:10.181 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:28:10.181 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6816", 2024-09-16T09:28:10.181 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1670012115 2024-09-16T09:28:10.181 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.181 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:10.181 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:28:10.181 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6817", 2024-09-16T09:28:10.181 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1670012115 2024-09-16T09:28:10.181 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:10.182 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:28:10.182 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.182 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2024-09-16T09:28:10.182 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:28:10.182 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:10.182 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:28:10.182 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6814", 2024-09-16T09:28:10.182 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1670012115 2024-09-16T09:28:10.182 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.182 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:10.183 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:28:10.183 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6815", 2024-09-16T09:28:10.183 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1670012115 2024-09-16T09:28:10.183 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:10.183 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:28:10.183 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.183 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6811/1670012115", 2024-09-16T09:28:10.183 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6813/1670012115", 2024-09-16T09:28:10.183 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6817/1670012115", 2024-09-16T09:28:10.183 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6815/1670012115", 2024-09-16T09:28:10.183 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2024-09-16T09:28:10.183 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2024-09-16T09:28:10.183 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2024-09-16T09:28:10.183 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:28:10.184 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.184 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:10.184 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 2, 2024-09-16T09:28:10.184 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "5b4ca4e8-cecc-4163-b291-c9209b02c4b7", 2024-09-16T09:28:10.184 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2024-09-16T09:28:10.184 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2024-09-16T09:28:10.184 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2024-09-16T09:28:10.184 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2024-09-16T09:28:10.184 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2024-09-16T09:28:10.184 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2024-09-16T09:28:10.184 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 18, 2024-09-16T09:28:10.184 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 92, 2024-09-16T09:28:10.184 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2024-09-16T09:28:10.185 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2024-09-16T09:28:10.185 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2024-09-16T09:28:10.185 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:28:10.185 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:10.185 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:28:10.185 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6818", 2024-09-16T09:28:10.185 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3601682336 2024-09-16T09:28:10.185 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.185 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:10.185 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:28:10.185 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6819", 2024-09-16T09:28:10.185 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3601682336 2024-09-16T09:28:10.185 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:10.185 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:28:10.186 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.186 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2024-09-16T09:28:10.186 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:28:10.186 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:10.186 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:28:10.186 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6820", 2024-09-16T09:28:10.186 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3601682336 2024-09-16T09:28:10.186 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.186 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:10.186 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:28:10.186 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6821", 2024-09-16T09:28:10.186 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3601682336 2024-09-16T09:28:10.186 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:10.186 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:28:10.187 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.187 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2024-09-16T09:28:10.187 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:28:10.187 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:10.187 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:28:10.187 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6824", 2024-09-16T09:28:10.187 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3601682336 2024-09-16T09:28:10.187 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.187 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:10.187 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:28:10.187 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6825", 2024-09-16T09:28:10.187 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3601682336 2024-09-16T09:28:10.187 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:10.187 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:28:10.188 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.188 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2024-09-16T09:28:10.188 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:28:10.188 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:10.188 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:28:10.188 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6822", 2024-09-16T09:28:10.188 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3601682336 2024-09-16T09:28:10.188 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.188 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:10.188 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:28:10.188 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6823", 2024-09-16T09:28:10.188 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3601682336 2024-09-16T09:28:10.188 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:10.188 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:28:10.189 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.189 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6819/3601682336", 2024-09-16T09:28:10.189 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6821/3601682336", 2024-09-16T09:28:10.189 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6825/3601682336", 2024-09-16T09:28:10.189 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6823/3601682336", 2024-09-16T09:28:10.189 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2024-09-16T09:28:10.189 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2024-09-16T09:28:10.189 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2024-09-16T09:28:10.189 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:28:10.189 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.189 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:10.189 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 3, 2024-09-16T09:28:10.189 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "ffb8d527-1b58-4ecd-808c-47d4ba1bf387", 2024-09-16T09:28:10.190 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2024-09-16T09:28:10.190 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2024-09-16T09:28:10.190 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2024-09-16T09:28:10.190 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2024-09-16T09:28:10.190 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2024-09-16T09:28:10.190 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2024-09-16T09:28:10.190 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 26, 2024-09-16T09:28:10.190 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 94, 2024-09-16T09:28:10.190 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2024-09-16T09:28:10.190 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2024-09-16T09:28:10.190 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2024-09-16T09:28:10.190 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:28:10.190 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:10.191 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:28:10.191 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6826", 2024-09-16T09:28:10.191 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2038083007 2024-09-16T09:28:10.191 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.191 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:10.191 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:28:10.191 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6827", 2024-09-16T09:28:10.191 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2038083007 2024-09-16T09:28:10.191 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:10.191 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:28:10.191 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.191 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2024-09-16T09:28:10.191 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:28:10.191 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:10.191 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:28:10.192 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6828", 2024-09-16T09:28:10.192 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2038083007 2024-09-16T09:28:10.192 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.192 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:10.192 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:28:10.192 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6829", 2024-09-16T09:28:10.192 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2038083007 2024-09-16T09:28:10.192 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:10.192 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:28:10.192 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.192 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2024-09-16T09:28:10.192 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:28:10.192 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:10.193 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:28:10.193 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6832", 2024-09-16T09:28:10.193 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2038083007 2024-09-16T09:28:10.193 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.193 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:10.193 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:28:10.193 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6833", 2024-09-16T09:28:10.193 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2038083007 2024-09-16T09:28:10.193 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:10.193 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:28:10.193 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.193 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2024-09-16T09:28:10.193 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:28:10.194 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:10.194 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:28:10.194 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6830", 2024-09-16T09:28:10.194 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2038083007 2024-09-16T09:28:10.194 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.194 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:10.194 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:28:10.194 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6831", 2024-09-16T09:28:10.194 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2038083007 2024-09-16T09:28:10.194 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:10.194 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:28:10.194 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.194 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6827/2038083007", 2024-09-16T09:28:10.195 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6829/2038083007", 2024-09-16T09:28:10.195 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6833/2038083007", 2024-09-16T09:28:10.195 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6831/2038083007", 2024-09-16T09:28:10.195 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2024-09-16T09:28:10.195 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2024-09-16T09:28:10.195 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2024-09-16T09:28:10.195 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:28:10.195 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.196 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:10.196 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 4, 2024-09-16T09:28:10.196 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "a946760b-4d91-4a4b-8e9b-a07a50589211", 2024-09-16T09:28:10.196 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2024-09-16T09:28:10.196 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2024-09-16T09:28:10.196 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2024-09-16T09:28:10.196 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2024-09-16T09:28:10.196 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2024-09-16T09:28:10.197 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2024-09-16T09:28:10.197 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 32, 2024-09-16T09:28:10.197 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 94, 2024-09-16T09:28:10.197 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2024-09-16T09:28:10.197 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2024-09-16T09:28:10.197 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2024-09-16T09:28:10.197 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:28:10.197 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:10.197 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:28:10.197 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6800", 2024-09-16T09:28:10.198 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2250892710 2024-09-16T09:28:10.198 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.198 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:10.198 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:28:10.198 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6801", 2024-09-16T09:28:10.198 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2250892710 2024-09-16T09:28:10.198 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:10.198 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:28:10.198 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.198 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2024-09-16T09:28:10.198 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:28:10.198 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:10.198 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:28:10.198 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6802", 2024-09-16T09:28:10.199 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2250892710 2024-09-16T09:28:10.199 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.199 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:10.199 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:28:10.199 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6803", 2024-09-16T09:28:10.199 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2250892710 2024-09-16T09:28:10.199 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:10.199 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:28:10.199 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.199 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2024-09-16T09:28:10.199 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:28:10.199 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:10.199 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:28:10.200 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6806", 2024-09-16T09:28:10.200 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2250892710 2024-09-16T09:28:10.200 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.200 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:10.200 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:28:10.200 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6807", 2024-09-16T09:28:10.200 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2250892710 2024-09-16T09:28:10.200 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:10.200 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:28:10.200 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.200 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2024-09-16T09:28:10.200 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:28:10.200 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:10.200 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:28:10.201 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6804", 2024-09-16T09:28:10.201 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2250892710 2024-09-16T09:28:10.201 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.201 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:10.201 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:28:10.201 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6805", 2024-09-16T09:28:10.201 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2250892710 2024-09-16T09:28:10.201 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:10.201 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:28:10.201 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.201 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.84:6801/2250892710", 2024-09-16T09:28:10.201 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.84:6803/2250892710", 2024-09-16T09:28:10.201 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.84:6807/2250892710", 2024-09-16T09:28:10.202 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.84:6805/2250892710", 2024-09-16T09:28:10.202 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2024-09-16T09:28:10.202 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2024-09-16T09:28:10.202 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2024-09-16T09:28:10.202 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:28:10.202 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.202 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:10.202 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 5, 2024-09-16T09:28:10.202 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "67681760-75a9-4846-946e-a935d42c5e05", 2024-09-16T09:28:10.202 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2024-09-16T09:28:10.202 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2024-09-16T09:28:10.202 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2024-09-16T09:28:10.202 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2024-09-16T09:28:10.202 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2024-09-16T09:28:10.202 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2024-09-16T09:28:10.203 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 39, 2024-09-16T09:28:10.203 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 94, 2024-09-16T09:28:10.203 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2024-09-16T09:28:10.203 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2024-09-16T09:28:10.203 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2024-09-16T09:28:10.203 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:28:10.203 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:10.203 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:28:10.203 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6808", 2024-09-16T09:28:10.203 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1803945348 2024-09-16T09:28:10.203 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.203 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:10.203 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:28:10.203 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6809", 2024-09-16T09:28:10.204 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1803945348 2024-09-16T09:28:10.204 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:10.204 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:28:10.204 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.204 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2024-09-16T09:28:10.204 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:28:10.204 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:10.204 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:28:10.204 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6810", 2024-09-16T09:28:10.204 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1803945348 2024-09-16T09:28:10.204 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.204 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:10.204 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:28:10.204 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6811", 2024-09-16T09:28:10.205 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1803945348 2024-09-16T09:28:10.205 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:10.205 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:28:10.205 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.205 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2024-09-16T09:28:10.205 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:28:10.205 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:10.205 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:28:10.205 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6814", 2024-09-16T09:28:10.205 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1803945348 2024-09-16T09:28:10.205 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.205 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:10.205 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:28:10.205 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6815", 2024-09-16T09:28:10.206 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1803945348 2024-09-16T09:28:10.206 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:10.206 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:28:10.206 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.206 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2024-09-16T09:28:10.206 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:28:10.206 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:10.206 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:28:10.206 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6812", 2024-09-16T09:28:10.206 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1803945348 2024-09-16T09:28:10.206 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.206 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:10.206 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:28:10.206 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6813", 2024-09-16T09:28:10.207 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1803945348 2024-09-16T09:28:10.207 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:10.207 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:28:10.207 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.207 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.84:6809/1803945348", 2024-09-16T09:28:10.207 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.84:6811/1803945348", 2024-09-16T09:28:10.207 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.84:6815/1803945348", 2024-09-16T09:28:10.207 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.84:6813/1803945348", 2024-09-16T09:28:10.207 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2024-09-16T09:28:10.207 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2024-09-16T09:28:10.207 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2024-09-16T09:28:10.207 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:28:10.207 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.208 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:10.208 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 6, 2024-09-16T09:28:10.208 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "7c8f0152-f539-4b7e-8113-91dd7529d3a1", 2024-09-16T09:28:10.208 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2024-09-16T09:28:10.208 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2024-09-16T09:28:10.208 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2024-09-16T09:28:10.208 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2024-09-16T09:28:10.208 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2024-09-16T09:28:10.208 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2024-09-16T09:28:10.208 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 46, 2024-09-16T09:28:10.208 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 94, 2024-09-16T09:28:10.208 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2024-09-16T09:28:10.208 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2024-09-16T09:28:10.208 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2024-09-16T09:28:10.209 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:28:10.209 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:10.209 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:28:10.209 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6816", 2024-09-16T09:28:10.209 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 610727189 2024-09-16T09:28:10.209 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.209 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:10.209 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:28:10.209 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6817", 2024-09-16T09:28:10.209 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 610727189 2024-09-16T09:28:10.209 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:10.209 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:28:10.209 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.210 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2024-09-16T09:28:10.210 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:28:10.210 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:10.210 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:28:10.210 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6818", 2024-09-16T09:28:10.210 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 610727189 2024-09-16T09:28:10.210 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.210 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:10.210 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:28:10.211 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6819", 2024-09-16T09:28:10.211 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 610727189 2024-09-16T09:28:10.211 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:10.211 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:28:10.211 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.211 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2024-09-16T09:28:10.212 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:28:10.212 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:10.212 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:28:10.212 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6822", 2024-09-16T09:28:10.212 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 610727189 2024-09-16T09:28:10.212 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.213 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:10.213 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:28:10.213 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6823", 2024-09-16T09:28:10.213 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 610727189 2024-09-16T09:28:10.213 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:10.213 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:28:10.214 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.214 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2024-09-16T09:28:10.214 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:28:10.214 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:10.214 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:28:10.214 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6820", 2024-09-16T09:28:10.214 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 610727189 2024-09-16T09:28:10.215 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.215 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:10.215 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:28:10.215 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6821", 2024-09-16T09:28:10.215 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 610727189 2024-09-16T09:28:10.215 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:10.216 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:28:10.216 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.216 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.84:6817/610727189", 2024-09-16T09:28:10.216 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.84:6819/610727189", 2024-09-16T09:28:10.216 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.84:6823/610727189", 2024-09-16T09:28:10.216 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.84:6821/610727189", 2024-09-16T09:28:10.217 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2024-09-16T09:28:10.217 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2024-09-16T09:28:10.217 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2024-09-16T09:28:10.217 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:28:10.217 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.217 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:10.218 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 7, 2024-09-16T09:28:10.218 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "945efaba-225e-49af-aefd-b4d58ca8093b", 2024-09-16T09:28:10.218 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2024-09-16T09:28:10.218 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2024-09-16T09:28:10.218 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2024-09-16T09:28:10.218 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2024-09-16T09:28:10.219 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2024-09-16T09:28:10.219 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2024-09-16T09:28:10.219 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 53, 2024-09-16T09:28:10.219 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 94, 2024-09-16T09:28:10.219 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2024-09-16T09:28:10.219 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2024-09-16T09:28:10.220 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2024-09-16T09:28:10.220 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:28:10.220 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:10.220 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:28:10.220 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6824", 2024-09-16T09:28:10.220 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1002252531 2024-09-16T09:28:10.221 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.221 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:10.221 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:28:10.221 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6825", 2024-09-16T09:28:10.221 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1002252531 2024-09-16T09:28:10.222 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:10.222 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:28:10.222 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.222 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2024-09-16T09:28:10.222 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:28:10.222 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:10.223 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:28:10.223 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6826", 2024-09-16T09:28:10.223 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1002252531 2024-09-16T09:28:10.223 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.223 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:10.223 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:28:10.224 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6827", 2024-09-16T09:28:10.224 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1002252531 2024-09-16T09:28:10.224 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:10.224 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:28:10.224 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.225 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2024-09-16T09:28:10.225 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:28:10.225 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:10.225 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:28:10.225 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6830", 2024-09-16T09:28:10.225 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1002252531 2024-09-16T09:28:10.226 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.226 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:10.226 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:28:10.226 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6831", 2024-09-16T09:28:10.226 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1002252531 2024-09-16T09:28:10.226 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:10.226 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:28:10.227 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.227 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2024-09-16T09:28:10.227 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:28:10.227 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:10.227 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:28:10.228 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6828", 2024-09-16T09:28:10.228 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1002252531 2024-09-16T09:28:10.228 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.228 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:10.228 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:28:10.228 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6829", 2024-09-16T09:28:10.229 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1002252531 2024-09-16T09:28:10.229 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:10.229 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:28:10.229 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.229 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.84:6825/1002252531", 2024-09-16T09:28:10.229 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.84:6827/1002252531", 2024-09-16T09:28:10.230 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.84:6831/1002252531", 2024-09-16T09:28:10.230 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.84:6829/1002252531", 2024-09-16T09:28:10.230 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2024-09-16T09:28:10.230 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2024-09-16T09:28:10.230 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2024-09-16T09:28:10.230 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:28:10.231 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:10.231 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-16T09:28:10.231 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_xinfo": [ 2024-09-16T09:28:10.231 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:10.231 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 0, 2024-09-16T09:28:10.231 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2024-09-16T09:28:10.232 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2024-09-16T09:28:10.232 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2024-09-16T09:28:10.232 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540719139924082687, 2024-09-16T09:28:10.232 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2024-09-16T09:28:10.232 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2024-09-16T09:27:51.817291+0000", 2024-09-16T09:28:10.233 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2024-09-16T09:28:10.233 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.233 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:10.233 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 1, 2024-09-16T09:28:10.233 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2024-09-16T09:28:10.233 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2024-09-16T09:28:10.234 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2024-09-16T09:28:10.234 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540719139924082687, 2024-09-16T09:28:10.234 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2024-09-16T09:28:10.234 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2024-09-16T09:27:05.258528+0000", 2024-09-16T09:28:10.234 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2024-09-16T09:28:10.234 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.235 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:10.235 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 2, 2024-09-16T09:28:10.235 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2024-09-16T09:28:10.235 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2024-09-16T09:28:10.235 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2024-09-16T09:28:10.235 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540719139924082687, 2024-09-16T09:28:10.236 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2024-09-16T09:28:10.236 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2024-09-16T09:27:22.363714+0000", 2024-09-16T09:28:10.236 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2024-09-16T09:28:10.236 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.236 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:10.237 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 3, 2024-09-16T09:28:10.237 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2024-09-16T09:28:10.237 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2024-09-16T09:28:10.237 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2024-09-16T09:28:10.237 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540719139924082687, 2024-09-16T09:28:10.237 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2024-09-16T09:28:10.238 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2024-09-16T09:26:51.368139+0000", 2024-09-16T09:28:10.238 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2024-09-16T09:28:10.238 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.238 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:10.238 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 4, 2024-09-16T09:28:10.238 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2024-09-16T09:28:10.238 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2024-09-16T09:28:10.238 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2024-09-16T09:28:10.239 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540719139924082687, 2024-09-16T09:28:10.239 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2024-09-16T09:28:10.239 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2024-09-16T09:27:16.342139+0000", 2024-09-16T09:28:10.239 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2024-09-16T09:28:10.239 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.239 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:10.240 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 5, 2024-09-16T09:28:10.240 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2024-09-16T09:28:10.240 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2024-09-16T09:28:10.240 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2024-09-16T09:28:10.240 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540719139924082687, 2024-09-16T09:28:10.240 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2024-09-16T09:28:10.240 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2024-09-16T09:27:33.462609+0000", 2024-09-16T09:28:10.240 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2024-09-16T09:28:10.240 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.240 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:10.240 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 6, 2024-09-16T09:28:10.241 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2024-09-16T09:28:10.241 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2024-09-16T09:28:10.241 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2024-09-16T09:28:10.241 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540719139924082687, 2024-09-16T09:28:10.241 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2024-09-16T09:28:10.241 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2024-09-16T09:26:48.803601+0000", 2024-09-16T09:28:10.241 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2024-09-16T09:28:10.241 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.241 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:10.241 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 7, 2024-09-16T09:28:10.241 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2024-09-16T09:28:10.241 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2024-09-16T09:28:10.241 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2024-09-16T09:28:10.242 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540719139924082687, 2024-09-16T09:28:10.242 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2024-09-16T09:28:10.242 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2024-09-16T09:27:34.588738+0000", 2024-09-16T09:28:10.242 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2024-09-16T09:28:10.242 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:10.242 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-16T09:28:10.242 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap": [], 2024-09-16T09:28:10.242 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap_items": [ 2024-09-16T09:28:10.242 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:10.242 INFO:teuthology.orchestra.run.smithi070.stdout: "pgid": "4.e", 2024-09-16T09:28:10.242 INFO:teuthology.orchestra.run.smithi070.stdout: "mappings": [ 2024-09-16T09:28:10.242 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:10.242 INFO:teuthology.orchestra.run.smithi070.stdout: "from": 1, 2024-09-16T09:28:10.242 INFO:teuthology.orchestra.run.smithi070.stdout: "to": 2 2024-09-16T09:28:10.243 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:10.243 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:28:10.243 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.243 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:10.243 INFO:teuthology.orchestra.run.smithi070.stdout: "pgid": "4.f", 2024-09-16T09:28:10.243 INFO:teuthology.orchestra.run.smithi070.stdout: "mappings": [ 2024-09-16T09:28:10.243 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:10.243 INFO:teuthology.orchestra.run.smithi070.stdout: "from": 1, 2024-09-16T09:28:10.243 INFO:teuthology.orchestra.run.smithi070.stdout: "to": 2 2024-09-16T09:28:10.243 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:10.243 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:28:10.243 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.243 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:10.243 INFO:teuthology.orchestra.run.smithi070.stdout: "pgid": "4.13", 2024-09-16T09:28:10.243 INFO:teuthology.orchestra.run.smithi070.stdout: "mappings": [ 2024-09-16T09:28:10.244 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:10.244 INFO:teuthology.orchestra.run.smithi070.stdout: "from": 1, 2024-09-16T09:28:10.244 INFO:teuthology.orchestra.run.smithi070.stdout: "to": 5 2024-09-16T09:28:10.244 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:10.244 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:28:10.244 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:10.244 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-16T09:28:10.244 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap_primaries": [], 2024-09-16T09:28:10.244 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_temp": [], 2024-09-16T09:28:10.244 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_temp": [], 2024-09-16T09:28:10.244 INFO:teuthology.orchestra.run.smithi070.stdout: "blocklist": { 2024-09-16T09:28:10.244 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/125491368": "2024-09-17T09:25:31.465167+0000", 2024-09-16T09:28:10.244 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/3354284852": "2024-09-17T09:19:43.205989+0000", 2024-09-16T09:28:10.245 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/3284625547": "2024-09-17T09:25:31.465167+0000", 2024-09-16T09:28:10.245 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/186565817": "2024-09-17T09:19:43.205989+0000", 2024-09-16T09:28:10.245 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/2213852865": "2024-09-17T09:19:43.205989+0000", 2024-09-16T09:28:10.245 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/1812373760": "2024-09-17T09:20:09.306330+0000", 2024-09-16T09:28:10.245 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:6800/837338094": "2024-09-17T09:19:43.205989+0000", 2024-09-16T09:28:10.245 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:6800/1876661229": "2024-09-17T09:25:31.465167+0000", 2024-09-16T09:28:10.245 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/3351688485": "2024-09-17T09:20:09.306330+0000", 2024-09-16T09:28:10.245 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:6800/3871553358": "2024-09-17T09:20:09.306330+0000", 2024-09-16T09:28:10.245 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/1230482871": "2024-09-17T09:25:31.465167+0000", 2024-09-16T09:28:10.245 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:6801/837338094": "2024-09-17T09:19:43.205989+0000", 2024-09-16T09:28:10.245 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:6801/3871553358": "2024-09-17T09:20:09.306330+0000", 2024-09-16T09:28:10.245 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/1403001486": "2024-09-17T09:20:09.306330+0000", 2024-09-16T09:28:10.245 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/1723643739": "2024-09-17T09:25:31.465167+0000", 2024-09-16T09:28:10.245 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:6801/1876661229": "2024-09-17T09:25:31.465167+0000", 2024-09-16T09:28:10.246 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/3456417396": "2024-09-17T09:25:31.465167+0000" 2024-09-16T09:28:10.246 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.246 INFO:teuthology.orchestra.run.smithi070.stdout: "range_blocklist": {}, 2024-09-16T09:28:10.246 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profiles": { 2024-09-16T09:28:10.246 INFO:teuthology.orchestra.run.smithi070.stdout: "default": { 2024-09-16T09:28:10.246 INFO:teuthology.orchestra.run.smithi070.stdout: "crush-failure-domain": "osd", 2024-09-16T09:28:10.246 INFO:teuthology.orchestra.run.smithi070.stdout: "k": "2", 2024-09-16T09:28:10.246 INFO:teuthology.orchestra.run.smithi070.stdout: "m": "1", 2024-09-16T09:28:10.246 INFO:teuthology.orchestra.run.smithi070.stdout: "plugin": "jerasure", 2024-09-16T09:28:10.246 INFO:teuthology.orchestra.run.smithi070.stdout: "technique": "reed_sol_van" 2024-09-16T09:28:10.246 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:10.246 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:10.246 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps_queue": [], 2024-09-16T09:28:10.246 INFO:teuthology.orchestra.run.smithi070.stdout: "new_removed_snaps": [], 2024-09-16T09:28:10.246 INFO:teuthology.orchestra.run.smithi070.stdout: "new_purged_snaps": [], 2024-09-16T09:28:10.247 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_node_flags": {}, 2024-09-16T09:28:10.247 INFO:teuthology.orchestra.run.smithi070.stdout: "device_class_flags": {}, 2024-09-16T09:28:10.247 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode": { 2024-09-16T09:28:10.247 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode_enabled": false, 2024-09-16T09:28:10.247 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_bucket_count": 0, 2024-09-16T09:28:10.247 INFO:teuthology.orchestra.run.smithi070.stdout: "degraded_stretch_mode": 0, 2024-09-16T09:28:10.247 INFO:teuthology.orchestra.run.smithi070.stdout: "recovering_stretch_mode": 0, 2024-09-16T09:28:10.247 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode_bucket": 0 2024-09-16T09:28:10.247 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:10.247 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T09:28:10.247 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T09:28:10.249 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:10.250 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:10.250 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:28:10.351 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:28:10.377 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:10.377 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:10.378 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:28:10.478 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:28:10.504 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:10.504 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:10.505 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:28:10.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:10 smithi070 ceph-mon[30388]: pgmap v132: 158 pgs: 158 active+clean; 6.0 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.18k op/s 2024-09-16T09:28:10.606 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:28:10.657 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:10.657 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:10.658 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:28:10.759 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:28:10.793 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:10.793 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:10.793 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:28:10.897 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:28:10.898 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:10 smithi070 ceph-mon[30388]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:28:10.898 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:10 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-16T09:28:10.898 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:10 smithi070 ceph-mon[30388]: osdmap e124: 8 total, 8 up, 8 in 2024-09-16T09:28:10.898 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:10 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2628807630' entity='client.admin' cmd={"prefix": "osd dump", "format": "json-pretty"} : dispatch 2024-09-16T09:28:10.899 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:10 smithi070 ceph-mon[40691]: pgmap v132: 158 pgs: 158 active+clean; 6.0 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.18k op/s 2024-09-16T09:28:10.899 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:10 smithi070 ceph-mon[40691]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:28:10.899 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:10 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-16T09:28:10.899 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:10 smithi070 ceph-mon[40691]: osdmap e124: 8 total, 8 up, 8 in 2024-09-16T09:28:10.899 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:10 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/2628807630' entity='client.admin' cmd={"prefix": "osd dump", "format": "json-pretty"} : dispatch 2024-09-16T09:28:10.909 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:10 smithi084 ceph-mon[32629]: pgmap v132: 158 pgs: 158 active+clean; 6.0 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.18k op/s 2024-09-16T09:28:10.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:10 smithi084 ceph-mon[32629]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:28:10.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:10 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-16T09:28:10.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:10 smithi084 ceph-mon[32629]: osdmap e124: 8 total, 8 up, 8 in 2024-09-16T09:28:10.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:10 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/2628807630' entity='client.admin' cmd={"prefix": "osd dump", "format": "json-pretty"} : dispatch 2024-09-16T09:28:10.924 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:10.925 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:10.925 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:28:11.026 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:28:11.062 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:11.063 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:11.063 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:28:11.164 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:28:11.190 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:11.190 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:11.191 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:28:11.292 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:28:11.318 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:11.319 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:11.319 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:28:11.354 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:28:11.383 INFO:tasks.thrashosds.thrasher:j is {'epoch': 124, 'fsid': 'a1321834-740c-11ef-bceb-c7b262605968', 'created': '2024-09-16T09:19:14.121498+0000', 'modified': '2024-09-16T09:28:09.587262+0000', 'last_up_change': '2024-09-16T09:24:51.124496+0000', 'last_in_change': '2024-09-16T09:24:28.432920+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 18, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 7, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T09:22:27.617694+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 11, 'pg_placement_num': 11, 'pg_placement_num_target': 11, 'pg_num_target': 11, 'pg_num_pending': 11, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '94', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '92', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.180000066757202, 'score_stable': 2.180000066757202, 'optimal_score': 1, 'raw_score_acting': 2.180000066757202, 'raw_score_stable': 2.180000066757202, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'datapool', 'create_time': '2024-09-16T09:24:59.169244+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 3, 'pg_placement_num': 3, 'pg_placement_num_target': 3, 'pg_num_target': 3, 'pg_num_pending': 3, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 61, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.6500000953674316, 'score_stable': 2.6500000953674316, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 2.3299999237060547, 'raw_score_stable': 2.3299999237060547, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': '.rgw.root', 'create_time': '2024-09-16T09:25:00.056119+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'default.rgw.log', 'create_time': '2024-09-16T09:25:01.770016+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '62', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'default.rgw.control', 'create_time': '2024-09-16T09:25:03.725555+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'default.rgw.meta', 'create_time': '2024-09-16T09:25:05.773530+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'unique_pool_0', 'create_time': '2024-09-16T09:26:06.241945+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 16, 'pg_placement_num': 16, 'pg_placement_num_target': 16, 'pg_num_target': 16, 'pg_num_pending': 16, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rados': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '7ef288c8-3bf0-4814-8e86-f70a7de041ce', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 8, 'up_thru': 82, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6802', 'nonce': 2013075781}, {'type': 'v1', 'addr': '172.21.15.70:6803', 'nonce': 2013075781}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6804', 'nonce': 2013075781}, {'type': 'v1', 'addr': '172.21.15.70:6805', 'nonce': 2013075781}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6808', 'nonce': 2013075781}, {'type': 'v1', 'addr': '172.21.15.70:6809', 'nonce': 2013075781}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6806', 'nonce': 2013075781}, {'type': 'v1', 'addr': '172.21.15.70:6807', 'nonce': 2013075781}]}, 'public_addr': '172.21.15.70:6803/2013075781', 'cluster_addr': '172.21.15.70:6805/2013075781', 'heartbeat_back_addr': '172.21.15.70:6809/2013075781', 'heartbeat_front_addr': '172.21.15.70:6807/2013075781', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '474d41d9-295a-450c-8f27-ce487a14678b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 94, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6810', 'nonce': 1670012115}, {'type': 'v1', 'addr': '172.21.15.70:6811', 'nonce': 1670012115}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6812', 'nonce': 1670012115}, {'type': 'v1', 'addr': '172.21.15.70:6813', 'nonce': 1670012115}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6816', 'nonce': 1670012115}, {'type': 'v1', 'addr': '172.21.15.70:6817', 'nonce': 1670012115}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6814', 'nonce': 1670012115}, {'type': 'v1', 'addr': '172.21.15.70:6815', 'nonce': 1670012115}]}, 'public_addr': '172.21.15.70:6811/1670012115', 'cluster_addr': '172.21.15.70:6813/1670012115', 'heartbeat_back_addr': '172.21.15.70:6817/1670012115', 'heartbeat_front_addr': '172.21.15.70:6815/1670012115', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '5b4ca4e8-cecc-4163-b291-c9209b02c4b7', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 18, 'up_thru': 92, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6818', 'nonce': 3601682336}, {'type': 'v1', 'addr': '172.21.15.70:6819', 'nonce': 3601682336}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6820', 'nonce': 3601682336}, {'type': 'v1', 'addr': '172.21.15.70:6821', 'nonce': 3601682336}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6824', 'nonce': 3601682336}, {'type': 'v1', 'addr': '172.21.15.70:6825', 'nonce': 3601682336}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6822', 'nonce': 3601682336}, {'type': 'v1', 'addr': '172.21.15.70:6823', 'nonce': 3601682336}]}, 'public_addr': '172.21.15.70:6819/3601682336', 'cluster_addr': '172.21.15.70:6821/3601682336', 'heartbeat_back_addr': '172.21.15.70:6825/3601682336', 'heartbeat_front_addr': '172.21.15.70:6823/3601682336', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'ffb8d527-1b58-4ecd-808c-47d4ba1bf387', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 26, 'up_thru': 94, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6826', 'nonce': 2038083007}, {'type': 'v1', 'addr': '172.21.15.70:6827', 'nonce': 2038083007}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6828', 'nonce': 2038083007}, {'type': 'v1', 'addr': '172.21.15.70:6829', 'nonce': 2038083007}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6832', 'nonce': 2038083007}, {'type': 'v1', 'addr': '172.21.15.70:6833', 'nonce': 2038083007}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6830', 'nonce': 2038083007}, {'type': 'v1', 'addr': '172.21.15.70:6831', 'nonce': 2038083007}]}, 'public_addr': '172.21.15.70:6827/2038083007', 'cluster_addr': '172.21.15.70:6829/2038083007', 'heartbeat_back_addr': '172.21.15.70:6833/2038083007', 'heartbeat_front_addr': '172.21.15.70:6831/2038083007', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'a946760b-4d91-4a4b-8e9b-a07a50589211', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 32, 'up_thru': 94, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6800', 'nonce': 2250892710}, {'type': 'v1', 'addr': '172.21.15.84:6801', 'nonce': 2250892710}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6802', 'nonce': 2250892710}, {'type': 'v1', 'addr': '172.21.15.84:6803', 'nonce': 2250892710}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6806', 'nonce': 2250892710}, {'type': 'v1', 'addr': '172.21.15.84:6807', 'nonce': 2250892710}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6804', 'nonce': 2250892710}, {'type': 'v1', 'addr': '172.21.15.84:6805', 'nonce': 2250892710}]}, 'public_addr': '172.21.15.84:6801/2250892710', 'cluster_addr': '172.21.15.84:6803/2250892710', 'heartbeat_back_addr': '172.21.15.84:6807/2250892710', 'heartbeat_front_addr': '172.21.15.84:6805/2250892710', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '67681760-75a9-4846-946e-a935d42c5e05', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 39, 'up_thru': 94, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6808', 'nonce': 1803945348}, {'type': 'v1', 'addr': '172.21.15.84:6809', 'nonce': 1803945348}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6810', 'nonce': 1803945348}, {'type': 'v1', 'addr': '172.21.15.84:6811', 'nonce': 1803945348}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6814', 'nonce': 1803945348}, {'type': 'v1', 'addr': '172.21.15.84:6815', 'nonce': 1803945348}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6812', 'nonce': 1803945348}, {'type': 'v1', 'addr': '172.21.15.84:6813', 'nonce': 1803945348}]}, 'public_addr': '172.21.15.84:6809/1803945348', 'cluster_addr': '172.21.15.84:6811/1803945348', 'heartbeat_back_addr': '172.21.15.84:6815/1803945348', 'heartbeat_front_addr': '172.21.15.84:6813/1803945348', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '7c8f0152-f539-4b7e-8113-91dd7529d3a1', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 46, 'up_thru': 94, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6816', 'nonce': 610727189}, {'type': 'v1', 'addr': '172.21.15.84:6817', 'nonce': 610727189}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6818', 'nonce': 610727189}, {'type': 'v1', 'addr': '172.21.15.84:6819', 'nonce': 610727189}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6822', 'nonce': 610727189}, {'type': 'v1', 'addr': '172.21.15.84:6823', 'nonce': 610727189}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6820', 'nonce': 610727189}, {'type': 'v1', 'addr': '172.21.15.84:6821', 'nonce': 610727189}]}, 'public_addr': '172.21.15.84:6817/610727189', 'cluster_addr': '172.21.15.84:6819/610727189', 'heartbeat_back_addr': '172.21.15.84:6823/610727189', 'heartbeat_front_addr': '172.21.15.84:6821/610727189', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '945efaba-225e-49af-aefd-b4d58ca8093b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 53, 'up_thru': 94, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6824', 'nonce': 1002252531}, {'type': 'v1', 'addr': '172.21.15.84:6825', 'nonce': 1002252531}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6826', 'nonce': 1002252531}, {'type': 'v1', 'addr': '172.21.15.84:6827', 'nonce': 1002252531}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6830', 'nonce': 1002252531}, {'type': 'v1', 'addr': '172.21.15.84:6831', 'nonce': 1002252531}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6828', 'nonce': 1002252531}, {'type': 'v1', 'addr': '172.21.15.84:6829', 'nonce': 1002252531}]}, 'public_addr': '172.21.15.84:6825/1002252531', 'cluster_addr': '172.21.15.84:6827/1002252531', 'heartbeat_back_addr': '172.21.15.84:6831/1002252531', 'heartbeat_front_addr': '172.21.15.84:6829/1002252531', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T09:27:51.817291+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T09:27:05.258528+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T09:27:22.363714+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T09:26:51.368139+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T09:27:16.342139+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T09:27:33.462609+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T09:26:48.803601+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T09:27:34.588738+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [{'pgid': '4.e', 'mappings': [{'from': 1, 'to': 2}]}, {'pgid': '4.f', 'mappings': [{'from': 1, 'to': 2}]}, {'pgid': '4.13', 'mappings': [{'from': 1, 'to': 5}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {'172.21.15.70:0/125491368': '2024-09-17T09:25:31.465167+0000', '172.21.15.70:0/3354284852': '2024-09-17T09:19:43.205989+0000', '172.21.15.70:0/3284625547': '2024-09-17T09:25:31.465167+0000', '172.21.15.70:0/186565817': '2024-09-17T09:19:43.205989+0000', '172.21.15.70:0/2213852865': '2024-09-17T09:19:43.205989+0000', '172.21.15.70:0/1812373760': '2024-09-17T09:20:09.306330+0000', '172.21.15.70:6800/837338094': '2024-09-17T09:19:43.205989+0000', '172.21.15.70:6800/1876661229': '2024-09-17T09:25:31.465167+0000', '172.21.15.70:0/3351688485': '2024-09-17T09:20:09.306330+0000', '172.21.15.70:6800/3871553358': '2024-09-17T09:20:09.306330+0000', '172.21.15.70:0/1230482871': '2024-09-17T09:25:31.465167+0000', '172.21.15.70:6801/837338094': '2024-09-17T09:19:43.205989+0000', '172.21.15.70:6801/3871553358': '2024-09-17T09:20:09.306330+0000', '172.21.15.70:0/1403001486': '2024-09-17T09:20:09.306330+0000', '172.21.15.70:0/1723643739': '2024-09-17T09:25:31.465167+0000', '172.21.15.70:6801/1876661229': '2024-09-17T09:25:31.465167+0000', '172.21.15.70:0/3456417396': '2024-09-17T09:25:31.465167+0000'}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-16T09:28:11.384 INFO:tasks.thrashosds.thrasher:No pg_upmap entries; doing nothing 2024-09-16T09:28:11.420 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:28:11.445 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:11.445 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:11.446 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:28:11.546 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:28:11.597 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:11.597 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:11.598 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:28:11.698 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:28:11.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:11.749 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:11.750 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:28:11.850 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:28:11.893 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:11.894 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:11.894 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:28:11.962 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:28:11.995 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:28:11.996 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T09:28:11.996 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T09:28:11.996 INFO:teuthology.orchestra.run.smithi070.stdout: "complaint_time": 30, 2024-09-16T09:28:11.996 INFO:teuthology.orchestra.run.smithi070.stdout: "num_blocked_ops": 0 2024-09-16T09:28:11.997 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T09:28:12.042 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:12.043 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:12.043 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:28:12.144 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:28:12.169 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:12.169 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:12.170 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:28:12.271 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:28:12.296 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:12.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:12.297 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:28:12.313 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:28:11 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:28:11] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:28:12.398 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:28:12.432 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd unset noscrub 2024-09-16T09:28:12.440 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:12.441 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:12.442 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:28:12.543 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:28:12.568 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:12.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:12.570 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:28:12.670 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:28:12.707 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:12.708 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:12.708 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:28:12.809 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:28:12.853 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:12.853 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:12.854 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:28:12.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:12 smithi084 ceph-mon[32629]: pgmap v134: 158 pgs: 158 active+clean; 6.0 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.18k op/s 2024-09-16T09:28:12.910 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:28:12 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:28:12.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:12 smithi070 ceph-mon[30388]: pgmap v134: 158 pgs: 158 active+clean; 6.0 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.18k op/s 2024-09-16T09:28:12.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:12 smithi070 ceph-mon[40691]: pgmap v134: 158 pgs: 158 active+clean; 6.0 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.18k op/s 2024-09-16T09:28:12.955 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:28:12.993 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:12.994 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:12.994 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:28:13.095 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:28:13.138 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.1 dump_historic_ops 2024-09-16T09:28:13.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:13.139 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:13.139 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:28:13.204 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:28:13.204 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:28:13.205 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:28:13.205 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:28:13.205 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:28:13.205 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:28:13.205 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:28:13.205 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:28:13.205 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:28:13.205 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:28:13.205 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:28:13.205 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:28:13.205 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:28:13.206 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:28:13.207 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:28:13.208 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:28:13.240 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:28:13.288 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:13.288 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:13.289 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:28:13.389 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:28:13.429 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:13.430 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:13.430 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:28:13.531 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:28:13.568 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:13.570 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:13.570 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:28:13.671 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:28:13.708 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:13.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:13.709 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:28:13.810 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:28:13.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:13.847 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:13.847 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:28:13.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:13 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:28:13.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:13 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/4172045871' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:28:13.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:13 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:28:13.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:13 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/4172045871' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:28:13.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:13 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:28:13.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:13 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/4172045871' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:28:13.948 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:28:13.971 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:13.972 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:13.972 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:28:13.997 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:28:14.073 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:28:14.104 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:28:14.126 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:14.127 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:14.128 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:28:14.228 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:28:14.254 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:14.255 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:14.255 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:28:14.356 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:28:14.391 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:14.392 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:14.392 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:28:14.493 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:28:14.518 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:14.519 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:14.520 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:28:14.621 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:28:14.648 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:14.649 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:14.649 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:28:14.750 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:28:14.786 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:14.787 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:14.787 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:28:14.883 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:14 smithi070 ceph-mon[30388]: pgmap v135: 158 pgs: 158 active+clean; 5.6 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.12k op/s 2024-09-16T09:28:14.883 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:14 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:28:14.883 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:14 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:28:14.883 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:14 smithi070 ceph-mon[40691]: pgmap v135: 158 pgs: 158 active+clean; 5.6 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.12k op/s 2024-09-16T09:28:14.883 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:14 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:28:14.884 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:14 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:28:14.888 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:28:14.909 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:14 smithi084 ceph-mon[32629]: pgmap v135: 158 pgs: 158 active+clean; 5.6 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.12k op/s 2024-09-16T09:28:14.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:14 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:28:14.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:14 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:28:14.938 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:14.939 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:14.939 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:28:15.040 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:28:15.067 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:15.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:15.068 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:28:15.169 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:28:15.192 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:15.193 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:15.193 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:28:15.294 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:28:15.331 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:15.332 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:15.332 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:28:15.433 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:28:15.462 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:15.463 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:15.463 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:28:15.564 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:28:15.603 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:15.604 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:15.604 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:28:15.658 INFO:teuthology.orchestra.run.smithi070.stderr:noscrub is unset 2024-09-16T09:28:15.705 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:28:15.746 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:15.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:15.747 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:28:15.774 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T09:28:15.774 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:28:15.847 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:28:15.889 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:15.890 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:15.891 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:28:15.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:15 smithi084 ceph-mon[32629]: osdmap e125: 8 total, 8 up, 8 in 2024-09-16T09:28:15.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:15 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1185511461' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:28:15.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:15 smithi070 ceph-mon[30388]: osdmap e125: 8 total, 8 up, 8 in 2024-09-16T09:28:15.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:15 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1185511461' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:28:15.920 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:15 smithi070 ceph-mon[40691]: osdmap e125: 8 total, 8 up, 8 in 2024-09-16T09:28:15.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:15 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1185511461' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:28:15.992 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:28:15.994 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T09:28:16.033 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:16.033 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:16.034 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:28:16.135 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:28:16.176 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:16.177 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:16.177 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:28:16.278 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:28:16.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:16.322 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:16.322 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:28:16.384 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T09:28:16.385 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T09:28:16.385 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-16T09:28:16.385 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd dump -f json-pretty 2024-09-16T09:28:16.423 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:28:16.481 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:16.481 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:16.482 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:28:16.582 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:28:16.632 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:16.633 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:16.634 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:28:16.735 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:28:16.780 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:16.780 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:16.781 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:28:16.881 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:28:16.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:16 smithi084 ceph-mon[32629]: pgmap v137: 158 pgs: 158 active+clean; 5.6 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.13k op/s 2024-09-16T09:28:16.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:16 smithi084 ceph-mon[32629]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:28:16.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:16 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1185511461' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-16T09:28:16.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:16 smithi084 ceph-mon[32629]: osdmap e126: 8 total, 8 up, 8 in 2024-09-16T09:28:16.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:16 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:28:16.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:16 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:28:16.916 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:16.917 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:16.918 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:28:16.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:16 smithi070 ceph-mon[30388]: pgmap v137: 158 pgs: 158 active+clean; 5.6 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.13k op/s 2024-09-16T09:28:16.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:16 smithi070 ceph-mon[30388]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:28:16.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:16 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1185511461' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-16T09:28:16.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:16 smithi070 ceph-mon[30388]: osdmap e126: 8 total, 8 up, 8 in 2024-09-16T09:28:16.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:16 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:28:16.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:16 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:28:16.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:16 smithi070 ceph-mon[40691]: pgmap v137: 158 pgs: 158 active+clean; 5.6 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.13k op/s 2024-09-16T09:28:16.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:16 smithi070 ceph-mon[40691]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:28:16.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:16 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1185511461' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-16T09:28:16.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:16 smithi070 ceph-mon[40691]: osdmap e126: 8 total, 8 up, 8 in 2024-09-16T09:28:16.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:16 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:28:16.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:16 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:28:17.018 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:28:17.046 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:17.047 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:17.047 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:28:17.126 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.2 dump_ops_in_flight 2024-09-16T09:28:17.148 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:28:17.176 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:17.177 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:17.177 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:28:17.196 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:28:17.225 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:28:17.278 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:28:17.317 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:17.318 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:17.318 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:28:17.419 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:28:17.449 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:17.450 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:17.450 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:28:17.551 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:28:17.610 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:17.611 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:17.611 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:28:17.712 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:28:17.755 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:17.755 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:17.756 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:28:17.857 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:28:17.858 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:17 smithi070 ceph-mon[30388]: osdmap e127: 8 total, 8 up, 8 in 2024-09-16T09:28:17.858 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:17 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:28:17.858 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:17 smithi070 ceph-mon[40691]: osdmap e127: 8 total, 8 up, 8 in 2024-09-16T09:28:17.858 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:17 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:28:17.901 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:17.901 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:17.902 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:28:17.909 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:17 smithi084 ceph-mon[32629]: osdmap e127: 8 total, 8 up, 8 in 2024-09-16T09:28:17.909 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:17 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:28:18.003 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:28:18.041 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:18.042 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:18.042 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:28:18.047 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:28:18.048 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:28:18.080 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:28:18.080 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:28:18.117 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:28:18.118 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:28:18.143 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:28:18.155 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:28:18.156 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:28:18.178 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:18.178 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:18.179 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:28:18.189 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:28:18.190 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:28:18.226 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:28:18.226 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:28:18.268 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:28:18.272 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:28:18.280 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:28:18.312 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:28:18.312 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:28:18.321 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:18.322 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:18.322 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:28:18.423 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:28:18.441 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-16T09:28:18.457 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:18.458 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:18.458 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:28:18.559 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:28:18.584 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:18.585 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:18.585 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:28:18.627 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:28:18.685 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:28:18.712 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:18.713 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:18.714 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:28:18.765 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:28:18.765 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T09:28:18.765 INFO:teuthology.orchestra.run.smithi070.stdout: "epoch": 127, 2024-09-16T09:28:18.765 INFO:teuthology.orchestra.run.smithi070.stdout: "fsid": "a1321834-740c-11ef-bceb-c7b262605968", 2024-09-16T09:28:18.765 INFO:teuthology.orchestra.run.smithi070.stdout: "created": "2024-09-16T09:19:14.121498+0000", 2024-09-16T09:28:18.765 INFO:teuthology.orchestra.run.smithi070.stdout: "modified": "2024-09-16T09:28:16.656306+0000", 2024-09-16T09:28:18.765 INFO:teuthology.orchestra.run.smithi070.stdout: "last_up_change": "2024-09-16T09:24:51.124496+0000", 2024-09-16T09:28:18.765 INFO:teuthology.orchestra.run.smithi070.stdout: "last_in_change": "2024-09-16T09:24:28.432920+0000", 2024-09-16T09:28:18.765 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-16T09:28:18.765 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_num": 5804032, 2024-09-16T09:28:18.766 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_set": [ 2024-09-16T09:28:18.766 INFO:teuthology.orchestra.run.smithi070.stdout: "nodeep-scrub", 2024-09-16T09:28:18.766 INFO:teuthology.orchestra.run.smithi070.stdout: "pglog_hardlimit", 2024-09-16T09:28:18.766 INFO:teuthology.orchestra.run.smithi070.stdout: "purged_snapdirs", 2024-09-16T09:28:18.766 INFO:teuthology.orchestra.run.smithi070.stdout: "recovery_deletes", 2024-09-16T09:28:18.766 INFO:teuthology.orchestra.run.smithi070.stdout: "sortbitwise" 2024-09-16T09:28:18.766 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-16T09:28:18.766 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_version": 18, 2024-09-16T09:28:18.766 INFO:teuthology.orchestra.run.smithi070.stdout: "full_ratio": 0.94999998807907104, 2024-09-16T09:28:18.766 INFO:teuthology.orchestra.run.smithi070.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-16T09:28:18.766 INFO:teuthology.orchestra.run.smithi070.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-16T09:28:18.766 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_snapshot": "", 2024-09-16T09:28:18.766 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_max": 7, 2024-09-16T09:28:18.767 INFO:teuthology.orchestra.run.smithi070.stdout: "max_osd": 8, 2024-09-16T09:28:18.767 INFO:teuthology.orchestra.run.smithi070.stdout: "require_min_compat_client": "luminous", 2024-09-16T09:28:18.767 INFO:teuthology.orchestra.run.smithi070.stdout: "min_compat_client": "luminous", 2024-09-16T09:28:18.767 INFO:teuthology.orchestra.run.smithi070.stdout: "require_osd_release": "squid", 2024-09-16T09:28:18.767 INFO:teuthology.orchestra.run.smithi070.stdout: "allow_crimson": false, 2024-09-16T09:28:18.767 INFO:teuthology.orchestra.run.smithi070.stdout: "pools": [ 2024-09-16T09:28:18.767 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:18.767 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 1, 2024-09-16T09:28:18.767 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": ".mgr", 2024-09-16T09:28:18.767 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2024-09-16T09:22:27.617694+0000", 2024-09-16T09:28:18.767 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2024-09-16T09:28:18.768 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2024-09-16T09:28:18.768 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2024-09-16T09:28:18.768 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 3, 2024-09-16T09:28:18.768 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 2, 2024-09-16T09:28:18.768 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2024-09-16T09:28:18.768 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2024-09-16T09:28:18.768 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2024-09-16T09:28:18.768 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T09:28:18.768 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T09:28:18.768 INFO:teuthology.orchestra.run.smithi070.stdout: "is_stretch_pool": false, 2024-09-16T09:28:18.768 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2024-09-16T09:28:18.768 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2024-09-16T09:28:18.768 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 11, 2024-09-16T09:28:18.768 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 11, 2024-09-16T09:28:18.769 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 11, 2024-09-16T09:28:18.769 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 11, 2024-09-16T09:28:18.769 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 11, 2024-09-16T09:28:18.769 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2024-09-16T09:28:18.769 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2024-09-16T09:28:18.769 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2024-09-16T09:28:18.769 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2024-09-16T09:28:18.769 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2024-09-16T09:28:18.769 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2024-09-16T09:28:18.769 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2024-09-16T09:28:18.769 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.769 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "94", 2024-09-16T09:28:18.769 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2024-09-16T09:28:18.769 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T09:28:18.770 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "92", 2024-09-16T09:28:18.770 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2024-09-16T09:28:18.770 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2024-09-16T09:28:18.770 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2024-09-16T09:28:18.770 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2024-09-16T09:28:18.770 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2024-09-16T09:28:18.770 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2024-09-16T09:28:18.770 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2024-09-16T09:28:18.770 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2024-09-16T09:28:18.770 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2024-09-16T09:28:18.770 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2024-09-16T09:28:18.770 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2024-09-16T09:28:18.770 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2024-09-16T09:28:18.771 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2024-09-16T09:28:18.771 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2024-09-16T09:28:18.771 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2024-09-16T09:28:18.771 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T09:28:18.771 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T09:28:18.771 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T09:28:18.771 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2024-09-16T09:28:18.771 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2024-09-16T09:28:18.771 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2024-09-16T09:28:18.771 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2024-09-16T09:28:18.771 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2024-09-16T09:28:18.771 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.771 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2024-09-16T09:28:18.771 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2024-09-16T09:28:18.772 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2024-09-16T09:28:18.772 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2024-09-16T09:28:18.772 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2024-09-16T09:28:18.772 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T09:28:18.772 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2024-09-16T09:28:18.772 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2024-09-16T09:28:18.772 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2024-09-16T09:28:18.772 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2024-09-16T09:28:18.772 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2024-09-16T09:28:18.772 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2024-09-16T09:28:18.772 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_max": 32, 2024-09-16T09:28:18.772 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_min": 1 2024-09-16T09:28:18.772 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.773 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2024-09-16T09:28:18.773 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": {} 2024-09-16T09:28:18.773 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.773 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2024-09-16T09:28:18.773 INFO:teuthology.orchestra.run.smithi070.stdout: "score_type": "Fair distribution", 2024-09-16T09:28:18.773 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 2.1800000667572021, 2024-09-16T09:28:18.773 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 2.1800000667572021, 2024-09-16T09:28:18.773 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1, 2024-09-16T09:28:18.773 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 2.1800000667572021, 2024-09-16T09:28:18.773 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 2.1800000667572021, 2024-09-16T09:28:18.773 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2024-09-16T09:28:18.773 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2024-09-16T09:28:18.774 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2024-09-16T09:28:18.774 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:18.774 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.774 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:18.774 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 2, 2024-09-16T09:28:18.774 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "datapool", 2024-09-16T09:28:18.774 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2024-09-16T09:24:59.169244+0000", 2024-09-16T09:28:18.774 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 8193, 2024-09-16T09:28:18.774 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-16T09:28:18.774 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2024-09-16T09:28:18.774 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 3, 2024-09-16T09:28:18.774 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 2, 2024-09-16T09:28:18.774 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2024-09-16T09:28:18.774 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2024-09-16T09:28:18.774 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2024-09-16T09:28:18.775 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T09:28:18.775 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T09:28:18.775 INFO:teuthology.orchestra.run.smithi070.stdout: "is_stretch_pool": false, 2024-09-16T09:28:18.775 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2024-09-16T09:28:18.775 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2024-09-16T09:28:18.775 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 3, 2024-09-16T09:28:18.775 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 3, 2024-09-16T09:28:18.775 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 3, 2024-09-16T09:28:18.775 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 3, 2024-09-16T09:28:18.775 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 3, 2024-09-16T09:28:18.775 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2024-09-16T09:28:18.775 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2024-09-16T09:28:18.775 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2024-09-16T09:28:18.776 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2024-09-16T09:28:18.776 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2024-09-16T09:28:18.776 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2024-09-16T09:28:18.776 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2024-09-16T09:28:18.776 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.776 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "61", 2024-09-16T09:28:18.776 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2024-09-16T09:28:18.776 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T09:28:18.776 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T09:28:18.777 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2024-09-16T09:28:18.777 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2024-09-16T09:28:18.777 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 2, 2024-09-16T09:28:18.777 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 61, 2024-09-16T09:28:18.777 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2024-09-16T09:28:18.777 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2024-09-16T09:28:18.777 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2024-09-16T09:28:18.777 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2024-09-16T09:28:18.777 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2024-09-16T09:28:18.778 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2024-09-16T09:28:18.778 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2024-09-16T09:28:18.778 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2024-09-16T09:28:18.778 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2024-09-16T09:28:18.778 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2024-09-16T09:28:18.778 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2024-09-16T09:28:18.778 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T09:28:18.778 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T09:28:18.778 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T09:28:18.778 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2024-09-16T09:28:18.778 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2024-09-16T09:28:18.778 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2024-09-16T09:28:18.778 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2024-09-16T09:28:18.778 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2024-09-16T09:28:18.779 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.779 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2024-09-16T09:28:18.779 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2024-09-16T09:28:18.779 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2024-09-16T09:28:18.779 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2024-09-16T09:28:18.779 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2024-09-16T09:28:18.779 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T09:28:18.779 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2024-09-16T09:28:18.779 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2024-09-16T09:28:18.779 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2024-09-16T09:28:18.779 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2024-09-16T09:28:18.779 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2024-09-16T09:28:18.779 INFO:teuthology.orchestra.run.smithi070.stdout: "options": {}, 2024-09-16T09:28:18.780 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2024-09-16T09:28:18.780 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd": {} 2024-09-16T09:28:18.780 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.780 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2024-09-16T09:28:18.780 INFO:teuthology.orchestra.run.smithi070.stdout: "score_type": "Fair distribution", 2024-09-16T09:28:18.780 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 2.6500000953674316, 2024-09-16T09:28:18.780 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 2.6500000953674316, 2024-09-16T09:28:18.780 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 0.87999999523162842, 2024-09-16T09:28:18.780 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 2.3299999237060547, 2024-09-16T09:28:18.780 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 2.3299999237060547, 2024-09-16T09:28:18.781 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2024-09-16T09:28:18.781 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2024-09-16T09:28:18.781 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2024-09-16T09:28:18.781 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:18.781 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.782 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:18.782 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 3, 2024-09-16T09:28:18.782 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": ".rgw.root", 2024-09-16T09:28:18.782 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2024-09-16T09:25:00.056119+0000", 2024-09-16T09:28:18.782 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2024-09-16T09:28:18.782 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2024-09-16T09:28:18.782 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2024-09-16T09:28:18.783 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 3, 2024-09-16T09:28:18.783 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 2, 2024-09-16T09:28:18.783 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2024-09-16T09:28:18.783 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2024-09-16T09:28:18.783 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2024-09-16T09:28:18.783 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T09:28:18.783 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T09:28:18.783 INFO:teuthology.orchestra.run.smithi070.stdout: "is_stretch_pool": false, 2024-09-16T09:28:18.783 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2024-09-16T09:28:18.783 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2024-09-16T09:28:18.783 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 32, 2024-09-16T09:28:18.783 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 32, 2024-09-16T09:28:18.784 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 32, 2024-09-16T09:28:18.784 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 32, 2024-09-16T09:28:18.784 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 32, 2024-09-16T09:28:18.784 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2024-09-16T09:28:18.784 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2024-09-16T09:28:18.784 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2024-09-16T09:28:18.785 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2024-09-16T09:28:18.785 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2024-09-16T09:28:18.785 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2024-09-16T09:28:18.785 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2024-09-16T09:28:18.785 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.785 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "60", 2024-09-16T09:28:18.785 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2024-09-16T09:28:18.785 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T09:28:18.785 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T09:28:18.785 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2024-09-16T09:28:18.785 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2024-09-16T09:28:18.785 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2024-09-16T09:28:18.786 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2024-09-16T09:28:18.786 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2024-09-16T09:28:18.786 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2024-09-16T09:28:18.786 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2024-09-16T09:28:18.786 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2024-09-16T09:28:18.786 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2024-09-16T09:28:18.786 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2024-09-16T09:28:18.786 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2024-09-16T09:28:18.786 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2024-09-16T09:28:18.786 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2024-09-16T09:28:18.786 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2024-09-16T09:28:18.786 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2024-09-16T09:28:18.786 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T09:28:18.787 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T09:28:18.787 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T09:28:18.787 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2024-09-16T09:28:18.787 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2024-09-16T09:28:18.787 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2024-09-16T09:28:18.787 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2024-09-16T09:28:18.787 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2024-09-16T09:28:18.787 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.787 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2024-09-16T09:28:18.787 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2024-09-16T09:28:18.787 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2024-09-16T09:28:18.787 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2024-09-16T09:28:18.787 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2024-09-16T09:28:18.787 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T09:28:18.788 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2024-09-16T09:28:18.788 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2024-09-16T09:28:18.788 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2024-09-16T09:28:18.788 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2024-09-16T09:28:18.788 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2024-09-16T09:28:18.788 INFO:teuthology.orchestra.run.smithi070.stdout: "options": {}, 2024-09-16T09:28:18.788 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2024-09-16T09:28:18.788 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw": {} 2024-09-16T09:28:18.788 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.788 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2024-09-16T09:28:18.788 INFO:teuthology.orchestra.run.smithi070.stdout: "score_type": "Fair distribution", 2024-09-16T09:28:18.788 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 1.5, 2024-09-16T09:28:18.788 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 1.5, 2024-09-16T09:28:18.788 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1, 2024-09-16T09:28:18.789 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 1.5, 2024-09-16T09:28:18.789 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 1.5, 2024-09-16T09:28:18.789 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2024-09-16T09:28:18.789 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2024-09-16T09:28:18.789 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2024-09-16T09:28:18.789 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:18.789 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.789 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:18.789 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 4, 2024-09-16T09:28:18.789 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "default.rgw.log", 2024-09-16T09:28:18.789 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2024-09-16T09:25:01.770016+0000", 2024-09-16T09:28:18.789 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2024-09-16T09:28:18.790 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2024-09-16T09:28:18.790 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2024-09-16T09:28:18.790 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 3, 2024-09-16T09:28:18.790 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 2, 2024-09-16T09:28:18.790 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2024-09-16T09:28:18.790 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2024-09-16T09:28:18.790 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2024-09-16T09:28:18.790 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T09:28:18.790 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T09:28:18.790 INFO:teuthology.orchestra.run.smithi070.stdout: "is_stretch_pool": false, 2024-09-16T09:28:18.790 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2024-09-16T09:28:18.790 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2024-09-16T09:28:18.791 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 32, 2024-09-16T09:28:18.791 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 32, 2024-09-16T09:28:18.791 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 32, 2024-09-16T09:28:18.791 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 32, 2024-09-16T09:28:18.791 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 32, 2024-09-16T09:28:18.791 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2024-09-16T09:28:18.791 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2024-09-16T09:28:18.791 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2024-09-16T09:28:18.791 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2024-09-16T09:28:18.791 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2024-09-16T09:28:18.791 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2024-09-16T09:28:18.791 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2024-09-16T09:28:18.791 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.792 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "62", 2024-09-16T09:28:18.792 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2024-09-16T09:28:18.792 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T09:28:18.792 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T09:28:18.792 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2024-09-16T09:28:18.792 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2024-09-16T09:28:18.792 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2024-09-16T09:28:18.792 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2024-09-16T09:28:18.792 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2024-09-16T09:28:18.792 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2024-09-16T09:28:18.792 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2024-09-16T09:28:18.792 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2024-09-16T09:28:18.792 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2024-09-16T09:28:18.793 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2024-09-16T09:28:18.793 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2024-09-16T09:28:18.793 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2024-09-16T09:28:18.793 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2024-09-16T09:28:18.793 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2024-09-16T09:28:18.793 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2024-09-16T09:28:18.793 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T09:28:18.793 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T09:28:18.793 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T09:28:18.793 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2024-09-16T09:28:18.793 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2024-09-16T09:28:18.793 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2024-09-16T09:28:18.793 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2024-09-16T09:28:18.794 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2024-09-16T09:28:18.794 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.794 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2024-09-16T09:28:18.794 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2024-09-16T09:28:18.794 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2024-09-16T09:28:18.794 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2024-09-16T09:28:18.794 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2024-09-16T09:28:18.794 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T09:28:18.794 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2024-09-16T09:28:18.794 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2024-09-16T09:28:18.794 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2024-09-16T09:28:18.794 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2024-09-16T09:28:18.795 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2024-09-16T09:28:18.795 INFO:teuthology.orchestra.run.smithi070.stdout: "options": {}, 2024-09-16T09:28:18.795 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2024-09-16T09:28:18.795 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw": {} 2024-09-16T09:28:18.795 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.795 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2024-09-16T09:28:18.795 INFO:teuthology.orchestra.run.smithi070.stdout: "score_type": "Fair distribution", 2024-09-16T09:28:18.795 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 2.25, 2024-09-16T09:28:18.795 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 2.25, 2024-09-16T09:28:18.795 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1, 2024-09-16T09:28:18.795 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 2.25, 2024-09-16T09:28:18.795 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 2.25, 2024-09-16T09:28:18.796 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2024-09-16T09:28:18.796 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2024-09-16T09:28:18.796 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2024-09-16T09:28:18.796 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:18.796 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.796 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:18.796 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 5, 2024-09-16T09:28:18.796 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "default.rgw.control", 2024-09-16T09:28:18.796 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2024-09-16T09:25:03.725555+0000", 2024-09-16T09:28:18.796 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2024-09-16T09:28:18.796 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2024-09-16T09:28:18.796 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2024-09-16T09:28:18.797 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 3, 2024-09-16T09:28:18.797 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 2, 2024-09-16T09:28:18.797 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2024-09-16T09:28:18.797 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2024-09-16T09:28:18.797 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2024-09-16T09:28:18.797 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T09:28:18.797 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T09:28:18.797 INFO:teuthology.orchestra.run.smithi070.stdout: "is_stretch_pool": false, 2024-09-16T09:28:18.797 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2024-09-16T09:28:18.797 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2024-09-16T09:28:18.798 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 32, 2024-09-16T09:28:18.798 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 32, 2024-09-16T09:28:18.798 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 32, 2024-09-16T09:28:18.798 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 32, 2024-09-16T09:28:18.798 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 32, 2024-09-16T09:28:18.798 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2024-09-16T09:28:18.798 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2024-09-16T09:28:18.798 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2024-09-16T09:28:18.798 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2024-09-16T09:28:18.798 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2024-09-16T09:28:18.798 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2024-09-16T09:28:18.798 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2024-09-16T09:28:18.798 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.798 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "64", 2024-09-16T09:28:18.799 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2024-09-16T09:28:18.799 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T09:28:18.799 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T09:28:18.799 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2024-09-16T09:28:18.799 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2024-09-16T09:28:18.799 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2024-09-16T09:28:18.799 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2024-09-16T09:28:18.799 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2024-09-16T09:28:18.799 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2024-09-16T09:28:18.799 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2024-09-16T09:28:18.799 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2024-09-16T09:28:18.799 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2024-09-16T09:28:18.799 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2024-09-16T09:28:18.799 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2024-09-16T09:28:18.799 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2024-09-16T09:28:18.800 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2024-09-16T09:28:18.800 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2024-09-16T09:28:18.800 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2024-09-16T09:28:18.800 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T09:28:18.800 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T09:28:18.800 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T09:28:18.800 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2024-09-16T09:28:18.800 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2024-09-16T09:28:18.800 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2024-09-16T09:28:18.800 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2024-09-16T09:28:18.800 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2024-09-16T09:28:18.800 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.800 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2024-09-16T09:28:18.801 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2024-09-16T09:28:18.801 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2024-09-16T09:28:18.801 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2024-09-16T09:28:18.801 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2024-09-16T09:28:18.801 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T09:28:18.801 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2024-09-16T09:28:18.801 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2024-09-16T09:28:18.801 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2024-09-16T09:28:18.801 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2024-09-16T09:28:18.801 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2024-09-16T09:28:18.801 INFO:teuthology.orchestra.run.smithi070.stdout: "options": {}, 2024-09-16T09:28:18.802 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2024-09-16T09:28:18.802 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw": {} 2024-09-16T09:28:18.802 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.802 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2024-09-16T09:28:18.802 INFO:teuthology.orchestra.run.smithi070.stdout: "score_type": "Fair distribution", 2024-09-16T09:28:18.802 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 1.25, 2024-09-16T09:28:18.802 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 1.25, 2024-09-16T09:28:18.802 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1, 2024-09-16T09:28:18.802 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 1.25, 2024-09-16T09:28:18.802 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 1.25, 2024-09-16T09:28:18.802 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2024-09-16T09:28:18.802 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2024-09-16T09:28:18.803 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2024-09-16T09:28:18.803 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:18.804 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.804 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:18.804 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 6, 2024-09-16T09:28:18.804 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "default.rgw.meta", 2024-09-16T09:28:18.804 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2024-09-16T09:25:05.773530+0000", 2024-09-16T09:28:18.804 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2024-09-16T09:28:18.804 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2024-09-16T09:28:18.804 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2024-09-16T09:28:18.804 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 3, 2024-09-16T09:28:18.804 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 2, 2024-09-16T09:28:18.804 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2024-09-16T09:28:18.804 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2024-09-16T09:28:18.804 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2024-09-16T09:28:18.805 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T09:28:18.805 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T09:28:18.805 INFO:teuthology.orchestra.run.smithi070.stdout: "is_stretch_pool": false, 2024-09-16T09:28:18.805 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2024-09-16T09:28:18.805 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2024-09-16T09:28:18.805 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 32, 2024-09-16T09:28:18.805 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 32, 2024-09-16T09:28:18.805 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 32, 2024-09-16T09:28:18.805 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 32, 2024-09-16T09:28:18.805 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 32, 2024-09-16T09:28:18.805 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2024-09-16T09:28:18.805 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2024-09-16T09:28:18.805 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2024-09-16T09:28:18.806 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2024-09-16T09:28:18.806 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2024-09-16T09:28:18.806 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2024-09-16T09:28:18.806 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2024-09-16T09:28:18.806 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.806 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "66", 2024-09-16T09:28:18.806 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2024-09-16T09:28:18.806 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T09:28:18.806 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T09:28:18.806 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2024-09-16T09:28:18.806 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2024-09-16T09:28:18.806 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2024-09-16T09:28:18.806 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2024-09-16T09:28:18.806 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2024-09-16T09:28:18.807 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2024-09-16T09:28:18.807 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2024-09-16T09:28:18.807 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2024-09-16T09:28:18.807 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2024-09-16T09:28:18.807 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2024-09-16T09:28:18.807 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2024-09-16T09:28:18.807 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2024-09-16T09:28:18.807 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2024-09-16T09:28:18.807 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2024-09-16T09:28:18.807 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2024-09-16T09:28:18.807 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T09:28:18.807 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T09:28:18.807 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T09:28:18.808 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2024-09-16T09:28:18.808 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2024-09-16T09:28:18.808 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2024-09-16T09:28:18.808 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2024-09-16T09:28:18.808 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2024-09-16T09:28:18.808 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.808 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2024-09-16T09:28:18.808 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2024-09-16T09:28:18.808 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2024-09-16T09:28:18.808 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2024-09-16T09:28:18.808 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2024-09-16T09:28:18.808 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T09:28:18.808 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2024-09-16T09:28:18.809 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2024-09-16T09:28:18.809 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2024-09-16T09:28:18.809 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2024-09-16T09:28:18.809 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2024-09-16T09:28:18.809 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2024-09-16T09:28:18.809 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_bias": 4 2024-09-16T09:28:18.809 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.809 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2024-09-16T09:28:18.809 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw": {} 2024-09-16T09:28:18.810 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.810 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2024-09-16T09:28:18.810 INFO:teuthology.orchestra.run.smithi070.stdout: "score_type": "Fair distribution", 2024-09-16T09:28:18.810 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 1.75, 2024-09-16T09:28:18.810 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 1.75, 2024-09-16T09:28:18.810 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1, 2024-09-16T09:28:18.810 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 1.75, 2024-09-16T09:28:18.810 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 1.75, 2024-09-16T09:28:18.810 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2024-09-16T09:28:18.811 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2024-09-16T09:28:18.811 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2024-09-16T09:28:18.811 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:18.811 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.811 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:18.811 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 7, 2024-09-16T09:28:18.812 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "unique_pool_0", 2024-09-16T09:28:18.812 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2024-09-16T09:26:06.241945+0000", 2024-09-16T09:28:18.812 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2024-09-16T09:28:18.812 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2024-09-16T09:28:18.812 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2024-09-16T09:28:18.813 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 3, 2024-09-16T09:28:18.813 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 2, 2024-09-16T09:28:18.813 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2024-09-16T09:28:18.813 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2024-09-16T09:28:18.813 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2024-09-16T09:28:18.813 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T09:28:18.813 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T09:28:18.813 INFO:teuthology.orchestra.run.smithi070.stdout: "is_stretch_pool": false, 2024-09-16T09:28:18.813 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2024-09-16T09:28:18.813 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2024-09-16T09:28:18.814 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 16, 2024-09-16T09:28:18.814 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 16, 2024-09-16T09:28:18.814 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 16, 2024-09-16T09:28:18.814 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 16, 2024-09-16T09:28:18.814 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 16, 2024-09-16T09:28:18.814 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2024-09-16T09:28:18.814 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2024-09-16T09:28:18.814 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2024-09-16T09:28:18.814 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2024-09-16T09:28:18.814 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2024-09-16T09:28:18.814 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2024-09-16T09:28:18.814 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2024-09-16T09:28:18.814 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.815 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "85", 2024-09-16T09:28:18.815 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2024-09-16T09:28:18.815 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T09:28:18.815 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T09:28:18.815 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2024-09-16T09:28:18.815 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2024-09-16T09:28:18.815 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2024-09-16T09:28:18.815 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2024-09-16T09:28:18.815 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2024-09-16T09:28:18.815 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2024-09-16T09:28:18.816 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2024-09-16T09:28:18.816 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2024-09-16T09:28:18.816 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2024-09-16T09:28:18.816 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2024-09-16T09:28:18.816 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2024-09-16T09:28:18.816 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2024-09-16T09:28:18.816 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2024-09-16T09:28:18.816 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2024-09-16T09:28:18.816 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2024-09-16T09:28:18.816 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T09:28:18.816 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T09:28:18.816 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T09:28:18.817 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2024-09-16T09:28:18.817 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2024-09-16T09:28:18.817 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2024-09-16T09:28:18.817 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2024-09-16T09:28:18.817 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2024-09-16T09:28:18.817 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.817 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2024-09-16T09:28:18.817 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2024-09-16T09:28:18.817 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2024-09-16T09:28:18.817 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2024-09-16T09:28:18.817 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2024-09-16T09:28:18.817 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T09:28:18.818 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2024-09-16T09:28:18.818 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2024-09-16T09:28:18.818 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2024-09-16T09:28:18.818 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2024-09-16T09:28:18.818 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2024-09-16T09:28:18.818 INFO:teuthology.orchestra.run.smithi070.stdout: "options": {}, 2024-09-16T09:28:18.818 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2024-09-16T09:28:18.818 INFO:teuthology.orchestra.run.smithi070.stdout: "rados": {} 2024-09-16T09:28:18.818 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.819 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2024-09-16T09:28:18.819 INFO:teuthology.orchestra.run.smithi070.stdout: "score_type": "Fair distribution", 2024-09-16T09:28:18.819 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 2, 2024-09-16T09:28:18.819 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 2, 2024-09-16T09:28:18.819 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1, 2024-09-16T09:28:18.819 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 2, 2024-09-16T09:28:18.819 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 2, 2024-09-16T09:28:18.819 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 1, 2024-09-16T09:28:18.819 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 1, 2024-09-16T09:28:18.820 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 1 2024-09-16T09:28:18.820 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:18.820 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:18.820 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-16T09:28:18.820 INFO:teuthology.orchestra.run.smithi070.stdout: "osds": [ 2024-09-16T09:28:18.820 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:18.820 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 0, 2024-09-16T09:28:18.820 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "7ef288c8-3bf0-4814-8e86-f70a7de041ce", 2024-09-16T09:28:18.820 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2024-09-16T09:28:18.820 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2024-09-16T09:28:18.820 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2024-09-16T09:28:18.820 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2024-09-16T09:28:18.820 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2024-09-16T09:28:18.821 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2024-09-16T09:28:18.821 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 8, 2024-09-16T09:28:18.821 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 82, 2024-09-16T09:28:18.821 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2024-09-16T09:28:18.821 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2024-09-16T09:28:18.821 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2024-09-16T09:28:18.821 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:28:18.821 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:18.821 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:28:18.821 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6802", 2024-09-16T09:28:18.821 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2013075781 2024-09-16T09:28:18.821 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.821 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:18.822 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:28:18.822 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6803", 2024-09-16T09:28:18.822 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2013075781 2024-09-16T09:28:18.822 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:18.822 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:28:18.822 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.822 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2024-09-16T09:28:18.822 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:28:18.822 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:18.822 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:28:18.822 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6804", 2024-09-16T09:28:18.822 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2013075781 2024-09-16T09:28:18.822 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.823 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:18.823 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:28:18.823 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6805", 2024-09-16T09:28:18.823 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2013075781 2024-09-16T09:28:18.823 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:18.823 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:28:18.823 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.823 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2024-09-16T09:28:18.823 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:28:18.823 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:18.823 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:28:18.823 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6808", 2024-09-16T09:28:18.823 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2013075781 2024-09-16T09:28:18.823 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.824 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:18.824 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:28:18.824 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6809", 2024-09-16T09:28:18.824 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2013075781 2024-09-16T09:28:18.824 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:18.824 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:28:18.824 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.824 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2024-09-16T09:28:18.824 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:28:18.824 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:18.824 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:28:18.824 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6806", 2024-09-16T09:28:18.824 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2013075781 2024-09-16T09:28:18.824 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.825 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:18.825 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:28:18.825 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6807", 2024-09-16T09:28:18.825 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2013075781 2024-09-16T09:28:18.825 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:18.825 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:28:18.825 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.825 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6803/2013075781", 2024-09-16T09:28:18.825 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6805/2013075781", 2024-09-16T09:28:18.825 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6809/2013075781", 2024-09-16T09:28:18.825 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6807/2013075781", 2024-09-16T09:28:18.825 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2024-09-16T09:28:18.825 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2024-09-16T09:28:18.826 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2024-09-16T09:28:18.826 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:28:18.826 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.826 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:18.826 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 1, 2024-09-16T09:28:18.826 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "474d41d9-295a-450c-8f27-ce487a14678b", 2024-09-16T09:28:18.826 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2024-09-16T09:28:18.826 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2024-09-16T09:28:18.826 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2024-09-16T09:28:18.827 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2024-09-16T09:28:18.827 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2024-09-16T09:28:18.827 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2024-09-16T09:28:18.827 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2024-09-16T09:28:18.827 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 94, 2024-09-16T09:28:18.827 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2024-09-16T09:28:18.827 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2024-09-16T09:28:18.827 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2024-09-16T09:28:18.827 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:28:18.827 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:18.827 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:28:18.827 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6810", 2024-09-16T09:28:18.828 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1670012115 2024-09-16T09:28:18.828 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.828 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:18.828 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:28:18.828 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6811", 2024-09-16T09:28:18.828 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1670012115 2024-09-16T09:28:18.828 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:18.828 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:28:18.828 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.828 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2024-09-16T09:28:18.828 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:28:18.828 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:18.828 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:28:18.829 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6812", 2024-09-16T09:28:18.829 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1670012115 2024-09-16T09:28:18.829 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.829 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:18.829 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:28:18.829 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6813", 2024-09-16T09:28:18.829 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1670012115 2024-09-16T09:28:18.829 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:18.829 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:28:18.829 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.829 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2024-09-16T09:28:18.829 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:28:18.830 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:18.830 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:28:18.830 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6816", 2024-09-16T09:28:18.830 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1670012115 2024-09-16T09:28:18.830 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.830 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:18.830 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:28:18.830 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6817", 2024-09-16T09:28:18.830 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1670012115 2024-09-16T09:28:18.830 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:18.831 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:28:18.831 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.831 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2024-09-16T09:28:18.831 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:28:18.831 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:18.831 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:28:18.831 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6814", 2024-09-16T09:28:18.831 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1670012115 2024-09-16T09:28:18.831 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.831 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:18.831 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:28:18.831 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6815", 2024-09-16T09:28:18.831 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1670012115 2024-09-16T09:28:18.831 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:18.832 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:28:18.832 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.832 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6811/1670012115", 2024-09-16T09:28:18.832 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6813/1670012115", 2024-09-16T09:28:18.832 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6817/1670012115", 2024-09-16T09:28:18.832 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6815/1670012115", 2024-09-16T09:28:18.832 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2024-09-16T09:28:18.832 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2024-09-16T09:28:18.832 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2024-09-16T09:28:18.832 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:28:18.833 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.833 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:18.833 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 2, 2024-09-16T09:28:18.833 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "5b4ca4e8-cecc-4163-b291-c9209b02c4b7", 2024-09-16T09:28:18.833 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2024-09-16T09:28:18.833 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2024-09-16T09:28:18.834 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2024-09-16T09:28:18.834 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2024-09-16T09:28:18.834 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2024-09-16T09:28:18.834 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2024-09-16T09:28:18.834 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 18, 2024-09-16T09:28:18.835 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 92, 2024-09-16T09:28:18.835 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2024-09-16T09:28:18.835 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2024-09-16T09:28:18.835 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2024-09-16T09:28:18.835 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:28:18.835 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:18.836 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:28:18.836 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6818", 2024-09-16T09:28:18.836 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3601682336 2024-09-16T09:28:18.836 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.836 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:18.837 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:28:18.837 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6819", 2024-09-16T09:28:18.837 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3601682336 2024-09-16T09:28:18.837 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:18.837 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:28:18.838 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.838 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2024-09-16T09:28:18.838 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:28:18.838 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:18.838 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:28:18.838 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6820", 2024-09-16T09:28:18.838 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3601682336 2024-09-16T09:28:18.838 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.838 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:18.839 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:28:18.839 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6821", 2024-09-16T09:28:18.839 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3601682336 2024-09-16T09:28:18.839 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:18.839 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:28:18.839 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.839 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2024-09-16T09:28:18.839 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:28:18.839 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:18.839 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:28:18.839 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6824", 2024-09-16T09:28:18.839 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3601682336 2024-09-16T09:28:18.839 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.840 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:18.840 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:28:18.840 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6825", 2024-09-16T09:28:18.840 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3601682336 2024-09-16T09:28:18.840 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:18.840 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:28:18.840 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.840 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2024-09-16T09:28:18.840 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:28:18.840 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:18.840 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:28:18.840 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6822", 2024-09-16T09:28:18.840 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3601682336 2024-09-16T09:28:18.840 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.841 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:18.841 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:28:18.841 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6823", 2024-09-16T09:28:18.841 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3601682336 2024-09-16T09:28:18.841 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:18.841 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:28:18.841 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.842 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6819/3601682336", 2024-09-16T09:28:18.842 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6821/3601682336", 2024-09-16T09:28:18.842 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6825/3601682336", 2024-09-16T09:28:18.842 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6823/3601682336", 2024-09-16T09:28:18.842 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2024-09-16T09:28:18.842 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2024-09-16T09:28:18.842 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2024-09-16T09:28:18.842 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:28:18.842 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.842 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:18.843 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 3, 2024-09-16T09:28:18.843 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "ffb8d527-1b58-4ecd-808c-47d4ba1bf387", 2024-09-16T09:28:18.843 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2024-09-16T09:28:18.843 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2024-09-16T09:28:18.843 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2024-09-16T09:28:18.843 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2024-09-16T09:28:18.843 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2024-09-16T09:28:18.843 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2024-09-16T09:28:18.843 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 26, 2024-09-16T09:28:18.843 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 94, 2024-09-16T09:28:18.843 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2024-09-16T09:28:18.843 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2024-09-16T09:28:18.843 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2024-09-16T09:28:18.844 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:28:18.844 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:18.844 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:28:18.844 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6826", 2024-09-16T09:28:18.844 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2038083007 2024-09-16T09:28:18.844 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.844 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:18.844 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:28:18.844 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6827", 2024-09-16T09:28:18.844 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2038083007 2024-09-16T09:28:18.844 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:18.844 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:28:18.844 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.844 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2024-09-16T09:28:18.845 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:28:18.845 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:18.845 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:28:18.845 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6828", 2024-09-16T09:28:18.845 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2038083007 2024-09-16T09:28:18.845 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.845 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:18.845 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:28:18.845 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6829", 2024-09-16T09:28:18.845 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2038083007 2024-09-16T09:28:18.845 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:18.845 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:28:18.845 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.846 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2024-09-16T09:28:18.846 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:28:18.846 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:18.846 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:28:18.846 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6832", 2024-09-16T09:28:18.846 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2038083007 2024-09-16T09:28:18.846 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.846 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:18.846 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:28:18.846 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6833", 2024-09-16T09:28:18.846 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2038083007 2024-09-16T09:28:18.846 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:18.846 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:28:18.846 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.847 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2024-09-16T09:28:18.847 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:28:18.847 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:18.847 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:28:18.847 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6830", 2024-09-16T09:28:18.847 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2038083007 2024-09-16T09:28:18.847 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.847 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:18.847 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:28:18.847 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6831", 2024-09-16T09:28:18.847 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2038083007 2024-09-16T09:28:18.847 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:18.847 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:28:18.847 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.848 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6827/2038083007", 2024-09-16T09:28:18.848 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6829/2038083007", 2024-09-16T09:28:18.848 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6833/2038083007", 2024-09-16T09:28:18.848 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6831/2038083007", 2024-09-16T09:28:18.848 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2024-09-16T09:28:18.848 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2024-09-16T09:28:18.848 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2024-09-16T09:28:18.848 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:28:18.848 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.848 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:18.848 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 4, 2024-09-16T09:28:18.849 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:28:18.850 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "a946760b-4d91-4a4b-8e9b-a07a50589211", 2024-09-16T09:28:18.850 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2024-09-16T09:28:18.850 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2024-09-16T09:28:18.850 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2024-09-16T09:28:18.851 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2024-09-16T09:28:18.851 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2024-09-16T09:28:18.851 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2024-09-16T09:28:18.851 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 32, 2024-09-16T09:28:18.851 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 94, 2024-09-16T09:28:18.851 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2024-09-16T09:28:18.851 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2024-09-16T09:28:18.851 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2024-09-16T09:28:18.851 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:28:18.851 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:18.851 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:28:18.851 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6800", 2024-09-16T09:28:18.852 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2250892710 2024-09-16T09:28:18.852 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.852 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:18.852 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:28:18.852 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6801", 2024-09-16T09:28:18.852 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2250892710 2024-09-16T09:28:18.852 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:18.852 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:28:18.852 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.852 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2024-09-16T09:28:18.852 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:28:18.852 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:18.852 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:28:18.853 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6802", 2024-09-16T09:28:18.853 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2250892710 2024-09-16T09:28:18.853 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.853 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:18.853 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:28:18.853 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6803", 2024-09-16T09:28:18.853 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2250892710 2024-09-16T09:28:18.853 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:18.853 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:28:18.854 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.854 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2024-09-16T09:28:18.854 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:28:18.854 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:18.854 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:28:18.854 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6806", 2024-09-16T09:28:18.854 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2250892710 2024-09-16T09:28:18.854 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.854 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:18.854 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:28:18.854 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6807", 2024-09-16T09:28:18.854 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2250892710 2024-09-16T09:28:18.854 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:18.855 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:28:18.855 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.855 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2024-09-16T09:28:18.855 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:28:18.855 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:18.855 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:28:18.855 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6804", 2024-09-16T09:28:18.855 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2250892710 2024-09-16T09:28:18.855 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.855 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:18.855 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:28:18.855 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6805", 2024-09-16T09:28:18.855 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2250892710 2024-09-16T09:28:18.856 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:18.856 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:28:18.856 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.856 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.84:6801/2250892710", 2024-09-16T09:28:18.856 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.84:6803/2250892710", 2024-09-16T09:28:18.856 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.84:6807/2250892710", 2024-09-16T09:28:18.856 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.84:6805/2250892710", 2024-09-16T09:28:18.856 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2024-09-16T09:28:18.856 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2024-09-16T09:28:18.856 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2024-09-16T09:28:18.856 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:28:18.856 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.856 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:18.857 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 5, 2024-09-16T09:28:18.857 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "67681760-75a9-4846-946e-a935d42c5e05", 2024-09-16T09:28:18.857 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2024-09-16T09:28:18.857 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2024-09-16T09:28:18.857 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2024-09-16T09:28:18.857 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2024-09-16T09:28:18.857 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2024-09-16T09:28:18.857 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2024-09-16T09:28:18.857 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 39, 2024-09-16T09:28:18.857 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 94, 2024-09-16T09:28:18.857 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2024-09-16T09:28:18.857 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2024-09-16T09:28:18.857 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2024-09-16T09:28:18.858 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:28:18.858 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:18.858 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:28:18.858 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6808", 2024-09-16T09:28:18.858 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1803945348 2024-09-16T09:28:18.858 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.858 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:18.858 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:28:18.858 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6809", 2024-09-16T09:28:18.858 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1803945348 2024-09-16T09:28:18.858 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:18.858 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:28:18.858 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.859 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2024-09-16T09:28:18.859 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:28:18.859 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:18.859 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:28:18.859 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6810", 2024-09-16T09:28:18.859 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1803945348 2024-09-16T09:28:18.859 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.859 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:18.859 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:28:18.859 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6811", 2024-09-16T09:28:18.859 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1803945348 2024-09-16T09:28:18.859 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:18.859 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:28:18.859 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.860 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2024-09-16T09:28:18.860 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:28:18.860 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:18.860 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:28:18.860 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6814", 2024-09-16T09:28:18.860 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1803945348 2024-09-16T09:28:18.860 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.860 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:18.860 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:28:18.860 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6815", 2024-09-16T09:28:18.860 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1803945348 2024-09-16T09:28:18.860 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:18.860 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:28:18.861 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.861 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2024-09-16T09:28:18.861 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:28:18.861 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:18.861 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:28:18.861 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6812", 2024-09-16T09:28:18.861 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1803945348 2024-09-16T09:28:18.861 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.861 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:18.861 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:28:18.861 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6813", 2024-09-16T09:28:18.861 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1803945348 2024-09-16T09:28:18.861 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:18.861 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:28:18.862 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.862 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.84:6809/1803945348", 2024-09-16T09:28:18.862 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.84:6811/1803945348", 2024-09-16T09:28:18.862 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.84:6815/1803945348", 2024-09-16T09:28:18.862 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.84:6813/1803945348", 2024-09-16T09:28:18.862 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2024-09-16T09:28:18.862 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2024-09-16T09:28:18.862 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2024-09-16T09:28:18.862 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:28:18.862 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.862 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:18.862 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 6, 2024-09-16T09:28:18.862 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "7c8f0152-f539-4b7e-8113-91dd7529d3a1", 2024-09-16T09:28:18.862 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2024-09-16T09:28:18.863 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2024-09-16T09:28:18.863 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2024-09-16T09:28:18.863 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2024-09-16T09:28:18.863 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2024-09-16T09:28:18.863 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2024-09-16T09:28:18.863 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 46, 2024-09-16T09:28:18.863 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 94, 2024-09-16T09:28:18.863 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2024-09-16T09:28:18.863 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2024-09-16T09:28:18.863 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2024-09-16T09:28:18.863 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:28:18.863 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:18.863 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:28:18.864 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6816", 2024-09-16T09:28:18.864 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 610727189 2024-09-16T09:28:18.864 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.864 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:18.864 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:28:18.864 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6817", 2024-09-16T09:28:18.864 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 610727189 2024-09-16T09:28:18.864 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:18.864 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:28:18.864 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.864 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2024-09-16T09:28:18.864 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:28:18.864 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:18.864 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:28:18.865 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6818", 2024-09-16T09:28:18.865 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 610727189 2024-09-16T09:28:18.865 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.865 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:18.865 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:28:18.865 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6819", 2024-09-16T09:28:18.865 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 610727189 2024-09-16T09:28:18.865 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:18.865 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:28:18.865 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.865 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2024-09-16T09:28:18.865 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:28:18.866 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:18.866 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:28:18.866 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6822", 2024-09-16T09:28:18.866 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 610727189 2024-09-16T09:28:18.866 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.866 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:18.866 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:28:18.866 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6823", 2024-09-16T09:28:18.866 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 610727189 2024-09-16T09:28:18.866 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:18.866 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:28:18.866 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.866 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2024-09-16T09:28:18.866 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:28:18.867 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:18.867 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:28:18.867 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6820", 2024-09-16T09:28:18.867 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 610727189 2024-09-16T09:28:18.867 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.867 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:18.867 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:28:18.867 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6821", 2024-09-16T09:28:18.867 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 610727189 2024-09-16T09:28:18.867 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:18.867 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:28:18.867 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.867 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.84:6817/610727189", 2024-09-16T09:28:18.867 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.84:6819/610727189", 2024-09-16T09:28:18.868 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.84:6823/610727189", 2024-09-16T09:28:18.868 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.84:6821/610727189", 2024-09-16T09:28:18.868 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2024-09-16T09:28:18.868 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2024-09-16T09:28:18.868 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2024-09-16T09:28:18.868 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:28:18.868 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.868 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:18.868 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 7, 2024-09-16T09:28:18.868 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "945efaba-225e-49af-aefd-b4d58ca8093b", 2024-09-16T09:28:18.868 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2024-09-16T09:28:18.868 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2024-09-16T09:28:18.868 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2024-09-16T09:28:18.868 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2024-09-16T09:28:18.869 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2024-09-16T09:28:18.869 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2024-09-16T09:28:18.869 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 53, 2024-09-16T09:28:18.869 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 94, 2024-09-16T09:28:18.869 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2024-09-16T09:28:18.869 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2024-09-16T09:28:18.869 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2024-09-16T09:28:18.869 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:28:18.869 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:18.869 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:28:18.869 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6824", 2024-09-16T09:28:18.869 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1002252531 2024-09-16T09:28:18.869 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.870 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:18.870 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:28:18.870 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6825", 2024-09-16T09:28:18.870 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1002252531 2024-09-16T09:28:18.870 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:18.870 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:28:18.870 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.870 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2024-09-16T09:28:18.870 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:28:18.870 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:18.870 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:28:18.870 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6826", 2024-09-16T09:28:18.870 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1002252531 2024-09-16T09:28:18.870 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.871 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:18.871 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:28:18.871 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6827", 2024-09-16T09:28:18.871 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1002252531 2024-09-16T09:28:18.871 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:18.871 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:28:18.871 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.871 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2024-09-16T09:28:18.871 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:28:18.871 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:18.871 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:28:18.871 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6830", 2024-09-16T09:28:18.871 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1002252531 2024-09-16T09:28:18.872 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.872 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:18.872 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:28:18.872 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6831", 2024-09-16T09:28:18.872 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1002252531 2024-09-16T09:28:18.872 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:18.872 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:28:18.872 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.872 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2024-09-16T09:28:18.873 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:28:18.873 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:18.873 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:28:18.873 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6828", 2024-09-16T09:28:18.873 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1002252531 2024-09-16T09:28:18.873 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.873 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:18.873 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:28:18.873 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6829", 2024-09-16T09:28:18.873 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1002252531 2024-09-16T09:28:18.873 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:18.873 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:28:18.874 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.874 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.84:6825/1002252531", 2024-09-16T09:28:18.874 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.84:6827/1002252531", 2024-09-16T09:28:18.874 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.84:6831/1002252531", 2024-09-16T09:28:18.874 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.84:6829/1002252531", 2024-09-16T09:28:18.874 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2024-09-16T09:28:18.874 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2024-09-16T09:28:18.874 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2024-09-16T09:28:18.874 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:28:18.874 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:18.874 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-16T09:28:18.874 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_xinfo": [ 2024-09-16T09:28:18.874 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:18.874 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 0, 2024-09-16T09:28:18.875 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2024-09-16T09:28:18.875 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2024-09-16T09:28:18.875 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2024-09-16T09:28:18.875 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540719139924082687, 2024-09-16T09:28:18.875 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2024-09-16T09:28:18.875 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2024-09-16T09:27:51.817291+0000", 2024-09-16T09:28:18.875 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2024-09-16T09:28:18.876 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.876 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:18.876 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 1, 2024-09-16T09:28:18.876 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2024-09-16T09:28:18.876 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2024-09-16T09:28:18.876 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2024-09-16T09:28:18.877 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540719139924082687, 2024-09-16T09:28:18.877 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2024-09-16T09:28:18.877 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2024-09-16T09:27:05.258528+0000", 2024-09-16T09:28:18.877 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2024-09-16T09:28:18.877 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.878 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:18.878 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 2, 2024-09-16T09:28:18.878 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2024-09-16T09:28:18.878 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2024-09-16T09:28:18.878 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2024-09-16T09:28:18.878 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540719139924082687, 2024-09-16T09:28:18.878 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2024-09-16T09:28:18.878 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2024-09-16T09:27:22.363714+0000", 2024-09-16T09:28:18.878 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2024-09-16T09:28:18.878 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.879 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:18.879 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 3, 2024-09-16T09:28:18.879 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2024-09-16T09:28:18.879 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2024-09-16T09:28:18.879 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2024-09-16T09:28:18.879 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540719139924082687, 2024-09-16T09:28:18.879 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2024-09-16T09:28:18.879 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2024-09-16T09:28:15.687422+0000", 2024-09-16T09:28:18.879 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2024-09-16T09:28:18.879 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.879 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:18.879 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 4, 2024-09-16T09:28:18.879 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2024-09-16T09:28:18.880 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2024-09-16T09:28:18.880 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2024-09-16T09:28:18.880 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540719139924082687, 2024-09-16T09:28:18.880 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2024-09-16T09:28:18.880 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2024-09-16T09:27:16.342139+0000", 2024-09-16T09:28:18.880 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2024-09-16T09:28:18.880 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.880 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:18.880 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 5, 2024-09-16T09:28:18.880 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2024-09-16T09:28:18.880 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2024-09-16T09:28:18.880 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2024-09-16T09:28:18.880 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540719139924082687, 2024-09-16T09:28:18.881 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2024-09-16T09:28:18.881 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2024-09-16T09:27:33.462609+0000", 2024-09-16T09:28:18.881 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2024-09-16T09:28:18.881 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.881 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:18.881 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 6, 2024-09-16T09:28:18.881 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2024-09-16T09:28:18.881 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2024-09-16T09:28:18.881 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2024-09-16T09:28:18.881 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540719139924082687, 2024-09-16T09:28:18.881 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2024-09-16T09:28:18.881 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2024-09-16T09:28:13.702381+0000", 2024-09-16T09:28:18.881 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2024-09-16T09:28:18.881 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.882 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:18.882 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 7, 2024-09-16T09:28:18.882 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2024-09-16T09:28:18.882 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2024-09-16T09:28:18.882 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2024-09-16T09:28:18.882 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540719139924082687, 2024-09-16T09:28:18.882 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2024-09-16T09:28:18.882 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2024-09-16T09:27:34.588738+0000", 2024-09-16T09:28:18.882 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2024-09-16T09:28:18.882 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:18.882 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-16T09:28:18.883 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap": [], 2024-09-16T09:28:18.883 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap_items": [ 2024-09-16T09:28:18.883 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:18.883 INFO:teuthology.orchestra.run.smithi070.stdout: "pgid": "4.e", 2024-09-16T09:28:18.883 INFO:teuthology.orchestra.run.smithi070.stdout: "mappings": [ 2024-09-16T09:28:18.883 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:18.883 INFO:teuthology.orchestra.run.smithi070.stdout: "from": 1, 2024-09-16T09:28:18.883 INFO:teuthology.orchestra.run.smithi070.stdout: "to": 2 2024-09-16T09:28:18.883 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:18.883 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:28:18.883 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.883 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:18.883 INFO:teuthology.orchestra.run.smithi070.stdout: "pgid": "4.f", 2024-09-16T09:28:18.884 INFO:teuthology.orchestra.run.smithi070.stdout: "mappings": [ 2024-09-16T09:28:18.884 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:18.884 INFO:teuthology.orchestra.run.smithi070.stdout: "from": 1, 2024-09-16T09:28:18.884 INFO:teuthology.orchestra.run.smithi070.stdout: "to": 2 2024-09-16T09:28:18.884 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:18.884 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:28:18.884 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.884 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:18.884 INFO:teuthology.orchestra.run.smithi070.stdout: "pgid": "4.13", 2024-09-16T09:28:18.884 INFO:teuthology.orchestra.run.smithi070.stdout: "mappings": [ 2024-09-16T09:28:18.884 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:18.884 INFO:teuthology.orchestra.run.smithi070.stdout: "from": 1, 2024-09-16T09:28:18.884 INFO:teuthology.orchestra.run.smithi070.stdout: "to": 5 2024-09-16T09:28:18.884 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:18.885 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:28:18.885 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:18.885 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-16T09:28:18.885 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap_primaries": [], 2024-09-16T09:28:18.885 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_temp": [], 2024-09-16T09:28:18.885 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_temp": [], 2024-09-16T09:28:18.885 INFO:teuthology.orchestra.run.smithi070.stdout: "blocklist": { 2024-09-16T09:28:18.885 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/125491368": "2024-09-17T09:25:31.465167+0000", 2024-09-16T09:28:18.885 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/3354284852": "2024-09-17T09:19:43.205989+0000", 2024-09-16T09:28:18.885 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/3284625547": "2024-09-17T09:25:31.465167+0000", 2024-09-16T09:28:18.885 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/186565817": "2024-09-17T09:19:43.205989+0000", 2024-09-16T09:28:18.886 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/2213852865": "2024-09-17T09:19:43.205989+0000", 2024-09-16T09:28:18.886 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/1812373760": "2024-09-17T09:20:09.306330+0000", 2024-09-16T09:28:18.886 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:6800/837338094": "2024-09-17T09:19:43.205989+0000", 2024-09-16T09:28:18.886 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:6800/1876661229": "2024-09-17T09:25:31.465167+0000", 2024-09-16T09:28:18.886 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/3351688485": "2024-09-17T09:20:09.306330+0000", 2024-09-16T09:28:18.886 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:6800/3871553358": "2024-09-17T09:20:09.306330+0000", 2024-09-16T09:28:18.886 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/1230482871": "2024-09-17T09:25:31.465167+0000", 2024-09-16T09:28:18.886 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:6801/837338094": "2024-09-17T09:19:43.205989+0000", 2024-09-16T09:28:18.886 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:6801/3871553358": "2024-09-17T09:20:09.306330+0000", 2024-09-16T09:28:18.886 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/1403001486": "2024-09-17T09:20:09.306330+0000", 2024-09-16T09:28:18.886 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/1723643739": "2024-09-17T09:25:31.465167+0000", 2024-09-16T09:28:18.886 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:6801/1876661229": "2024-09-17T09:25:31.465167+0000", 2024-09-16T09:28:18.887 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/3456417396": "2024-09-17T09:25:31.465167+0000" 2024-09-16T09:28:18.887 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.887 INFO:teuthology.orchestra.run.smithi070.stdout: "range_blocklist": {}, 2024-09-16T09:28:18.887 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profiles": { 2024-09-16T09:28:18.887 INFO:teuthology.orchestra.run.smithi070.stdout: "default": { 2024-09-16T09:28:18.887 INFO:teuthology.orchestra.run.smithi070.stdout: "crush-failure-domain": "osd", 2024-09-16T09:28:18.887 INFO:teuthology.orchestra.run.smithi070.stdout: "k": "2", 2024-09-16T09:28:18.887 INFO:teuthology.orchestra.run.smithi070.stdout: "m": "1", 2024-09-16T09:28:18.887 INFO:teuthology.orchestra.run.smithi070.stdout: "plugin": "jerasure", 2024-09-16T09:28:18.887 INFO:teuthology.orchestra.run.smithi070.stdout: "technique": "reed_sol_van" 2024-09-16T09:28:18.887 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:18.887 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:18.887 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps_queue": [], 2024-09-16T09:28:18.887 INFO:teuthology.orchestra.run.smithi070.stdout: "new_removed_snaps": [], 2024-09-16T09:28:18.888 INFO:teuthology.orchestra.run.smithi070.stdout: "new_purged_snaps": [], 2024-09-16T09:28:18.888 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_node_flags": {}, 2024-09-16T09:28:18.888 INFO:teuthology.orchestra.run.smithi070.stdout: "device_class_flags": {}, 2024-09-16T09:28:18.888 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode": { 2024-09-16T09:28:18.888 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode_enabled": false, 2024-09-16T09:28:18.888 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_bucket_count": 0, 2024-09-16T09:28:18.888 INFO:teuthology.orchestra.run.smithi070.stdout: "degraded_stretch_mode": 0, 2024-09-16T09:28:18.888 INFO:teuthology.orchestra.run.smithi070.stdout: "recovering_stretch_mode": 0, 2024-09-16T09:28:18.888 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode_bucket": 0 2024-09-16T09:28:18.888 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:18.888 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T09:28:18.889 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:18.890 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:18.890 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:28:18.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:18 smithi084 ceph-mon[32629]: pgmap v140: 158 pgs: 158 active+clean; 5.5 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.27k op/s 2024-09-16T09:28:18.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:18 smithi084 ceph-mon[32629]: 7.1 scrub starts 2024-09-16T09:28:18.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:18 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/2356124558' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:28:18.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:18 smithi070 ceph-mon[30388]: pgmap v140: 158 pgs: 158 active+clean; 5.5 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.27k op/s 2024-09-16T09:28:18.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:18 smithi070 ceph-mon[30388]: 7.1 scrub starts 2024-09-16T09:28:18.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:18 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2356124558' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:28:18.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:18 smithi070 ceph-mon[40691]: pgmap v140: 158 pgs: 158 active+clean; 5.5 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.27k op/s 2024-09-16T09:28:18.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:18 smithi070 ceph-mon[40691]: 7.1 scrub starts 2024-09-16T09:28:18.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:18 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/2356124558' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:28:18.991 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:28:19.018 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:19.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:19.019 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:28:19.120 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:28:19.149 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:19.150 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:19.150 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:28:19.251 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:28:19.306 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:19.306 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:19.306 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:28:19.407 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:28:19.433 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:19.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:19.434 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:28:19.535 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:28:19.568 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:19.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:19.569 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:28:19.670 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:28:19.696 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:19.697 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:19.697 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:28:19.798 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:28:19.840 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:19.841 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:19.841 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:28:19.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:19 smithi070 ceph-mon[30388]: 3.19 scrub starts 2024-09-16T09:28:19.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:19 smithi070 ceph-mon[30388]: 3.19 scrub ok 2024-09-16T09:28:19.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:19 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/783889785' entity='client.admin' cmd={"prefix": "osd dump", "format": "json-pretty"} : dispatch 2024-09-16T09:28:19.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:19 smithi070 ceph-mon[40691]: 3.19 scrub starts 2024-09-16T09:28:19.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:19 smithi070 ceph-mon[40691]: 3.19 scrub ok 2024-09-16T09:28:19.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:19 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/783889785' entity='client.admin' cmd={"prefix": "osd dump", "format": "json-pretty"} : dispatch 2024-09-16T09:28:19.942 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:28:19.978 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:19.979 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:19.979 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:28:20.080 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:28:20.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:20.115 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:20.115 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:28:20.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:19 smithi084 ceph-mon[32629]: 3.19 scrub starts 2024-09-16T09:28:20.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:19 smithi084 ceph-mon[32629]: 3.19 scrub ok 2024-09-16T09:28:20.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:19 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/783889785' entity='client.admin' cmd={"prefix": "osd dump", "format": "json-pretty"} : dispatch 2024-09-16T09:28:20.217 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:28:20.259 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:20.259 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:20.260 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:28:20.361 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:28:20.399 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:20.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:20.400 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:28:20.500 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:28:20.525 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:20.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:20.526 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:28:20.539 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T09:28:20.539 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [ 2024-09-16T09:28:20.539 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:20.540 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.24698.0:168869 7.a e127/82 7:5aea31bc:::benchmark_data_smithi070_78438_object42738:head v 127'10517, mlcod=127'10515)", 2024-09-16T09:28:20.540 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:28:20.535665+0000", 2024-09-16T09:28:20.540 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.0020103619999999999, 2024-09-16T09:28:20.540 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0020400050000000001, 2024-09-16T09:28:20.540 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:28:20.540 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:28:20.540 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T09:28:20.540 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:28:20.540 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:20.540 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:28:20.540 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:20.535665+0000", 2024-09-16T09:28:20.540 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:28:20.540 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:20.540 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:20.540 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:28:20.541 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:20.535665+0000", 2024-09-16T09:28:20.541 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:28:20.541 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:20.541 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:20.541 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:28:20.541 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:20.535667+0000", 2024-09-16T09:28:20.541 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.4190000000000001e-06 2024-09-16T09:28:20.541 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:20.541 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:20.541 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:28:20.541 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:20.535673+0000", 2024-09-16T09:28:20.541 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.7710000000000001e-06 2024-09-16T09:28:20.541 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:20.542 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:20.542 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:28:20.542 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:20.535686+0000", 2024-09-16T09:28:20.542 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2788e-05 2024-09-16T09:28:20.542 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:20.542 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:20.542 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:28:20.542 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:20.535694+0000", 2024-09-16T09:28:20.542 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.4560000000000003e-06 2024-09-16T09:28:20.542 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:20.542 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:20.542 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:28:20.542 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:20.535880+0000", 2024-09-16T09:28:20.542 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000186651 2024-09-16T09:28:20.543 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:20.543 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:20.543 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:28:20.543 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:20.535901+0000", 2024-09-16T09:28:20.543 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.1090999999999998e-05 2024-09-16T09:28:20.543 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:20.543 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:28:20.543 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:20.543 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:20.543 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:20.543 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.24698.0:168876 7.3 e127/82 7:c75971da:::benchmark_data_smithi070_78438_object42745:head v 127'10728, mlcod=127'10726)", 2024-09-16T09:28:20.543 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:28:20.536491+0000", 2024-09-16T09:28:20.543 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.001185097, 2024-09-16T09:28:20.543 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001271927, 2024-09-16T09:28:20.543 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:28:20.544 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:28:20.544 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T09:28:20.544 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:28:20.544 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:20.544 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:28:20.544 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:20.536491+0000", 2024-09-16T09:28:20.544 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:28:20.544 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:20.544 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:20.544 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:28:20.544 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:20.536491+0000", 2024-09-16T09:28:20.544 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:28:20.544 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:20.544 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:20.544 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:28:20.544 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:20.536491+0000", 2024-09-16T09:28:20.545 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6199999999999997e-07 2024-09-16T09:28:20.545 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:20.545 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:20.545 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:28:20.545 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:20.536497+0000", 2024-09-16T09:28:20.545 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4790000000000004e-06 2024-09-16T09:28:20.545 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:20.545 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:20.545 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:28:20.545 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:20.536511+0000", 2024-09-16T09:28:20.545 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.4562000000000001e-05 2024-09-16T09:28:20.545 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:20.545 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:20.545 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:28:20.545 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:20.536518+0000", 2024-09-16T09:28:20.545 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.7920000000000004e-06 2024-09-16T09:28:20.546 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:20.546 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:20.546 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:28:20.546 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:20.536582+0000", 2024-09-16T09:28:20.546 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.4183000000000005e-05 2024-09-16T09:28:20.546 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:20.546 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:20.546 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:28:20.546 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:20.536601+0000", 2024-09-16T09:28:20.546 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.8598e-05 2024-09-16T09:28:20.546 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:20.546 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:28:20.546 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:20.546 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:20.547 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-16T09:28:20.547 INFO:teuthology.orchestra.run.smithi070.stdout: "num_ops": 2 2024-09-16T09:28:20.547 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T09:28:20.626 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:28:20.669 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:20.670 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:20.670 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:28:20.771 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:28:20.800 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:20.801 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:20.801 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:28:20.902 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:28:20.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:20 smithi070 ceph-mon[30388]: 7.1 scrub ok 2024-09-16T09:28:20.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:20 smithi070 ceph-mon[30388]: pgmap v141: 158 pgs: 158 active+clean; 5.3 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 929 op/s 2024-09-16T09:28:20.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:20 smithi070 ceph-mon[40691]: 7.1 scrub ok 2024-09-16T09:28:20.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:20 smithi070 ceph-mon[40691]: pgmap v141: 158 pgs: 158 active+clean; 5.3 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 929 op/s 2024-09-16T09:28:20.926 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:20.927 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:20.927 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:28:21.028 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:28:21.055 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:21.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:21.056 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:28:21.157 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:28:21.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:20 smithi084 ceph-mon[32629]: 7.1 scrub ok 2024-09-16T09:28:21.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:20 smithi084 ceph-mon[32629]: pgmap v141: 158 pgs: 158 active+clean; 5.3 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 929 op/s 2024-09-16T09:28:21.181 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:21.181 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:21.182 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:28:21.278 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T09:28:21.282 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:28:21.362 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:21.363 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:21.363 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:28:21.464 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:28:21.488 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:21.489 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:21.489 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:28:21.590 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:28:21.616 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:21.617 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:21.617 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:28:21.718 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:28:21.745 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:21.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:21.746 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:28:21.847 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:28:21.884 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:21.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:21.885 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:28:21.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:21 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:28:21.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:21 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:28:21.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:21 smithi070 ceph-mon[30388]: 6.1e scrub starts 2024-09-16T09:28:21.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:21 smithi070 ceph-mon[30388]: 6.1e scrub ok 2024-09-16T09:28:21.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:21 smithi070 ceph-mon[30388]: osdmap e128: 8 total, 8 up, 8 in 2024-09-16T09:28:21.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:21 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:28:21.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:21 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:28:21.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:21 smithi070 ceph-mon[40691]: 6.1e scrub starts 2024-09-16T09:28:21.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:21 smithi070 ceph-mon[40691]: 6.1e scrub ok 2024-09-16T09:28:21.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:21 smithi070 ceph-mon[40691]: osdmap e128: 8 total, 8 up, 8 in 2024-09-16T09:28:21.986 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:28:22.025 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:22.026 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:22.027 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:28:22.115 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.2 dump_blocked_ops 2024-09-16T09:28:22.127 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:28:22.154 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:22.155 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:22.155 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:28:22.159 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:21 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:28:22.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:21 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:28:22.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:21 smithi084 ceph-mon[32629]: 6.1e scrub starts 2024-09-16T09:28:22.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:21 smithi084 ceph-mon[32629]: 6.1e scrub ok 2024-09-16T09:28:22.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:21 smithi084 ceph-mon[32629]: osdmap e128: 8 total, 8 up, 8 in 2024-09-16T09:28:22.255 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:28:22.282 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:22.283 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:22.284 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:28:22.384 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:28:22.425 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:28:21 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:28:21] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:28:22.431 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:22.432 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:22.432 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:28:22.504 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:28:22.533 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:28:22.554 INFO:tasks.thrashosds.thrasher:j is {'epoch': 127, 'fsid': 'a1321834-740c-11ef-bceb-c7b262605968', 'created': '2024-09-16T09:19:14.121498+0000', 'modified': '2024-09-16T09:28:16.656306+0000', 'last_up_change': '2024-09-16T09:24:51.124496+0000', 'last_in_change': '2024-09-16T09:24:28.432920+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 18, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 7, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T09:22:27.617694+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 11, 'pg_placement_num': 11, 'pg_placement_num_target': 11, 'pg_num_target': 11, 'pg_num_pending': 11, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '94', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '92', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.180000066757202, 'score_stable': 2.180000066757202, 'optimal_score': 1, 'raw_score_acting': 2.180000066757202, 'raw_score_stable': 2.180000066757202, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'datapool', 'create_time': '2024-09-16T09:24:59.169244+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 3, 'pg_placement_num': 3, 'pg_placement_num_target': 3, 'pg_num_target': 3, 'pg_num_pending': 3, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 61, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.6500000953674316, 'score_stable': 2.6500000953674316, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 2.3299999237060547, 'raw_score_stable': 2.3299999237060547, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': '.rgw.root', 'create_time': '2024-09-16T09:25:00.056119+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'default.rgw.log', 'create_time': '2024-09-16T09:25:01.770016+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '62', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'default.rgw.control', 'create_time': '2024-09-16T09:25:03.725555+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'default.rgw.meta', 'create_time': '2024-09-16T09:25:05.773530+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'unique_pool_0', 'create_time': '2024-09-16T09:26:06.241945+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 16, 'pg_placement_num': 16, 'pg_placement_num_target': 16, 'pg_num_target': 16, 'pg_num_pending': 16, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rados': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '7ef288c8-3bf0-4814-8e86-f70a7de041ce', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 8, 'up_thru': 82, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6802', 'nonce': 2013075781}, {'type': 'v1', 'addr': '172.21.15.70:6803', 'nonce': 2013075781}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6804', 'nonce': 2013075781}, {'type': 'v1', 'addr': '172.21.15.70:6805', 'nonce': 2013075781}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6808', 'nonce': 2013075781}, {'type': 'v1', 'addr': '172.21.15.70:6809', 'nonce': 2013075781}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6806', 'nonce': 2013075781}, {'type': 'v1', 'addr': '172.21.15.70:6807', 'nonce': 2013075781}]}, 'public_addr': '172.21.15.70:6803/2013075781', 'cluster_addr': '172.21.15.70:6805/2013075781', 'heartbeat_back_addr': '172.21.15.70:6809/2013075781', 'heartbeat_front_addr': '172.21.15.70:6807/2013075781', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '474d41d9-295a-450c-8f27-ce487a14678b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 94, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6810', 'nonce': 1670012115}, {'type': 'v1', 'addr': '172.21.15.70:6811', 'nonce': 1670012115}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6812', 'nonce': 1670012115}, {'type': 'v1', 'addr': '172.21.15.70:6813', 'nonce': 1670012115}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6816', 'nonce': 1670012115}, {'type': 'v1', 'addr': '172.21.15.70:6817', 'nonce': 1670012115}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6814', 'nonce': 1670012115}, {'type': 'v1', 'addr': '172.21.15.70:6815', 'nonce': 1670012115}]}, 'public_addr': '172.21.15.70:6811/1670012115', 'cluster_addr': '172.21.15.70:6813/1670012115', 'heartbeat_back_addr': '172.21.15.70:6817/1670012115', 'heartbeat_front_addr': '172.21.15.70:6815/1670012115', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '5b4ca4e8-cecc-4163-b291-c9209b02c4b7', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 18, 'up_thru': 92, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6818', 'nonce': 3601682336}, {'type': 'v1', 'addr': '172.21.15.70:6819', 'nonce': 3601682336}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6820', 'nonce': 3601682336}, {'type': 'v1', 'addr': '172.21.15.70:6821', 'nonce': 3601682336}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6824', 'nonce': 3601682336}, {'type': 'v1', 'addr': '172.21.15.70:6825', 'nonce': 3601682336}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6822', 'nonce': 3601682336}, {'type': 'v1', 'addr': '172.21.15.70:6823', 'nonce': 3601682336}]}, 'public_addr': '172.21.15.70:6819/3601682336', 'cluster_addr': '172.21.15.70:6821/3601682336', 'heartbeat_back_addr': '172.21.15.70:6825/3601682336', 'heartbeat_front_addr': '172.21.15.70:6823/3601682336', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'ffb8d527-1b58-4ecd-808c-47d4ba1bf387', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 26, 'up_thru': 94, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6826', 'nonce': 2038083007}, {'type': 'v1', 'addr': '172.21.15.70:6827', 'nonce': 2038083007}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6828', 'nonce': 2038083007}, {'type': 'v1', 'addr': '172.21.15.70:6829', 'nonce': 2038083007}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6832', 'nonce': 2038083007}, {'type': 'v1', 'addr': '172.21.15.70:6833', 'nonce': 2038083007}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6830', 'nonce': 2038083007}, {'type': 'v1', 'addr': '172.21.15.70:6831', 'nonce': 2038083007}]}, 'public_addr': '172.21.15.70:6827/2038083007', 'cluster_addr': '172.21.15.70:6829/2038083007', 'heartbeat_back_addr': '172.21.15.70:6833/2038083007', 'heartbeat_front_addr': '172.21.15.70:6831/2038083007', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'a946760b-4d91-4a4b-8e9b-a07a50589211', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 32, 'up_thru': 94, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6800', 'nonce': 2250892710}, {'type': 'v1', 'addr': '172.21.15.84:6801', 'nonce': 2250892710}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6802', 'nonce': 2250892710}, {'type': 'v1', 'addr': '172.21.15.84:6803', 'nonce': 2250892710}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6806', 'nonce': 2250892710}, {'type': 'v1', 'addr': '172.21.15.84:6807', 'nonce': 2250892710}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6804', 'nonce': 2250892710}, {'type': 'v1', 'addr': '172.21.15.84:6805', 'nonce': 2250892710}]}, 'public_addr': '172.21.15.84:6801/2250892710', 'cluster_addr': '172.21.15.84:6803/2250892710', 'heartbeat_back_addr': '172.21.15.84:6807/2250892710', 'heartbeat_front_addr': '172.21.15.84:6805/2250892710', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '67681760-75a9-4846-946e-a935d42c5e05', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 39, 'up_thru': 94, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6808', 'nonce': 1803945348}, {'type': 'v1', 'addr': '172.21.15.84:6809', 'nonce': 1803945348}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6810', 'nonce': 1803945348}, {'type': 'v1', 'addr': '172.21.15.84:6811', 'nonce': 1803945348}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6814', 'nonce': 1803945348}, {'type': 'v1', 'addr': '172.21.15.84:6815', 'nonce': 1803945348}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6812', 'nonce': 1803945348}, {'type': 'v1', 'addr': '172.21.15.84:6813', 'nonce': 1803945348}]}, 'public_addr': '172.21.15.84:6809/1803945348', 'cluster_addr': '172.21.15.84:6811/1803945348', 'heartbeat_back_addr': '172.21.15.84:6815/1803945348', 'heartbeat_front_addr': '172.21.15.84:6813/1803945348', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '7c8f0152-f539-4b7e-8113-91dd7529d3a1', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 46, 'up_thru': 94, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6816', 'nonce': 610727189}, {'type': 'v1', 'addr': '172.21.15.84:6817', 'nonce': 610727189}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6818', 'nonce': 610727189}, {'type': 'v1', 'addr': '172.21.15.84:6819', 'nonce': 610727189}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6822', 'nonce': 610727189}, {'type': 'v1', 'addr': '172.21.15.84:6823', 'nonce': 610727189}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6820', 'nonce': 610727189}, {'type': 'v1', 'addr': '172.21.15.84:6821', 'nonce': 610727189}]}, 'public_addr': '172.21.15.84:6817/610727189', 'cluster_addr': '172.21.15.84:6819/610727189', 'heartbeat_back_addr': '172.21.15.84:6823/610727189', 'heartbeat_front_addr': '172.21.15.84:6821/610727189', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '945efaba-225e-49af-aefd-b4d58ca8093b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 53, 'up_thru': 94, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6824', 'nonce': 1002252531}, {'type': 'v1', 'addr': '172.21.15.84:6825', 'nonce': 1002252531}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6826', 'nonce': 1002252531}, {'type': 'v1', 'addr': '172.21.15.84:6827', 'nonce': 1002252531}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6830', 'nonce': 1002252531}, {'type': 'v1', 'addr': '172.21.15.84:6831', 'nonce': 1002252531}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6828', 'nonce': 1002252531}, {'type': 'v1', 'addr': '172.21.15.84:6829', 'nonce': 1002252531}]}, 'public_addr': '172.21.15.84:6825/1002252531', 'cluster_addr': '172.21.15.84:6827/1002252531', 'heartbeat_back_addr': '172.21.15.84:6831/1002252531', 'heartbeat_front_addr': '172.21.15.84:6829/1002252531', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T09:27:51.817291+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T09:27:05.258528+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T09:27:22.363714+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T09:28:15.687422+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T09:27:16.342139+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T09:27:33.462609+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T09:28:13.702381+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T09:27:34.588738+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [{'pgid': '4.e', 'mappings': [{'from': 1, 'to': 2}]}, {'pgid': '4.f', 'mappings': [{'from': 1, 'to': 2}]}, {'pgid': '4.13', 'mappings': [{'from': 1, 'to': 5}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {'172.21.15.70:0/125491368': '2024-09-17T09:25:31.465167+0000', '172.21.15.70:0/3354284852': '2024-09-17T09:19:43.205989+0000', '172.21.15.70:0/3284625547': '2024-09-17T09:25:31.465167+0000', '172.21.15.70:0/186565817': '2024-09-17T09:19:43.205989+0000', '172.21.15.70:0/2213852865': '2024-09-17T09:19:43.205989+0000', '172.21.15.70:0/1812373760': '2024-09-17T09:20:09.306330+0000', '172.21.15.70:6800/837338094': '2024-09-17T09:19:43.205989+0000', '172.21.15.70:6800/1876661229': '2024-09-17T09:25:31.465167+0000', '172.21.15.70:0/3351688485': '2024-09-17T09:20:09.306330+0000', '172.21.15.70:6800/3871553358': '2024-09-17T09:20:09.306330+0000', '172.21.15.70:0/1230482871': '2024-09-17T09:25:31.465167+0000', '172.21.15.70:6801/837338094': '2024-09-17T09:19:43.205989+0000', '172.21.15.70:6801/3871553358': '2024-09-17T09:20:09.306330+0000', '172.21.15.70:0/1403001486': '2024-09-17T09:20:09.306330+0000', '172.21.15.70:0/1723643739': '2024-09-17T09:25:31.465167+0000', '172.21.15.70:6801/1876661229': '2024-09-17T09:25:31.465167+0000', '172.21.15.70:0/3456417396': '2024-09-17T09:25:31.465167+0000'}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-16T09:28:22.554 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-16T09:28:22.557 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:22.557 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:22.557 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:28:22.574 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:28:22.659 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:28:22.732 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:22.732 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:22.733 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:28:22.834 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:28:22.846 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:22 smithi070 ceph-mon[30388]: pgmap v143: 158 pgs: 158 active+clean; 5.3 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 929 op/s 2024-09-16T09:28:22.846 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:22 smithi070 ceph-mon[30388]: 5.11 scrub starts 2024-09-16T09:28:22.846 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:22 smithi070 ceph-mon[30388]: 5.11 scrub ok 2024-09-16T09:28:22.846 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:22 smithi070 ceph-mon[30388]: 3.8 scrub starts 2024-09-16T09:28:22.847 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:22 smithi070 ceph-mon[30388]: 3.8 scrub ok 2024-09-16T09:28:22.847 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:22 smithi070 ceph-mon[40691]: pgmap v143: 158 pgs: 158 active+clean; 5.3 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 929 op/s 2024-09-16T09:28:22.847 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:22 smithi070 ceph-mon[40691]: 5.11 scrub starts 2024-09-16T09:28:22.847 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:22 smithi070 ceph-mon[40691]: 5.11 scrub ok 2024-09-16T09:28:22.847 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:22 smithi070 ceph-mon[40691]: 3.8 scrub starts 2024-09-16T09:28:22.847 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:22 smithi070 ceph-mon[40691]: 3.8 scrub ok 2024-09-16T09:28:22.909 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:22.910 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:22.910 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:28:23.011 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:28:23.039 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:23.040 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:23.040 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:28:23.141 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:28:23.159 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:28:22 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:28:23.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:22 smithi084 ceph-mon[32629]: pgmap v143: 158 pgs: 158 active+clean; 5.3 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 929 op/s 2024-09-16T09:28:23.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:22 smithi084 ceph-mon[32629]: 5.11 scrub starts 2024-09-16T09:28:23.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:22 smithi084 ceph-mon[32629]: 5.11 scrub ok 2024-09-16T09:28:23.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:22 smithi084 ceph-mon[32629]: 3.8 scrub starts 2024-09-16T09:28:23.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:22 smithi084 ceph-mon[32629]: 3.8 scrub ok 2024-09-16T09:28:23.162 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:23.163 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:23.163 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:28:23.264 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:28:23.291 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:23.292 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:23.292 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:28:23.393 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:28:23.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:23.428 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:23.428 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:28:23.529 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:28:23.569 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:23.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:23.570 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:28:23.670 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:28:23.695 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:23.696 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:23.696 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:28:23.745 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:28:23.797 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:28:23.839 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:23.840 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:23.840 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:28:23.862 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:28:23.863 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:28:23.863 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:23 smithi070 ceph-mon[30388]: 6.0 scrub starts 2024-09-16T09:28:23.863 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:23 smithi070 ceph-mon[30388]: 6.0 scrub ok 2024-09-16T09:28:23.863 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:23 smithi070 ceph-mon[30388]: 6.17 scrub starts 2024-09-16T09:28:23.863 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:23 smithi070 ceph-mon[30388]: 6.17 scrub ok 2024-09-16T09:28:23.863 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:23 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:28:23.863 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:23 smithi070 ceph-mon[30388]: 1.4 scrub starts 2024-09-16T09:28:23.863 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:23 smithi070 ceph-mon[30388]: 1.4 scrub ok 2024-09-16T09:28:23.864 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:23 smithi070 ceph-mon[40691]: 6.0 scrub starts 2024-09-16T09:28:23.864 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:23 smithi070 ceph-mon[40691]: 6.0 scrub ok 2024-09-16T09:28:23.864 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:23 smithi070 ceph-mon[40691]: 6.17 scrub starts 2024-09-16T09:28:23.864 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:23 smithi070 ceph-mon[40691]: 6.17 scrub ok 2024-09-16T09:28:23.864 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:23 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:28:23.864 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:23 smithi070 ceph-mon[40691]: 1.4 scrub starts 2024-09-16T09:28:23.865 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:23 smithi070 ceph-mon[40691]: 1.4 scrub ok 2024-09-16T09:28:23.907 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:28:23.908 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:28:23.941 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:28:23.952 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:28:23.952 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:28:23.989 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:28:23.990 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:28:23.998 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:23.999 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:23.999 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:28:24.031 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:28:24.031 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:28:24.070 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:28:24.071 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:28:24.100 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:28:24.105 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:28:24.105 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:28:24.126 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:24.127 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:24.127 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:28:24.147 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:28:24.147 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:28:24.159 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:23 smithi084 ceph-mon[32629]: 6.0 scrub starts 2024-09-16T09:28:24.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:23 smithi084 ceph-mon[32629]: 6.0 scrub ok 2024-09-16T09:28:24.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:23 smithi084 ceph-mon[32629]: 6.17 scrub starts 2024-09-16T09:28:24.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:23 smithi084 ceph-mon[32629]: 6.17 scrub ok 2024-09-16T09:28:24.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:23 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:28:24.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:23 smithi084 ceph-mon[32629]: 1.4 scrub starts 2024-09-16T09:28:24.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:23 smithi084 ceph-mon[32629]: 1.4 scrub ok 2024-09-16T09:28:24.228 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:28:24.279 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:24.280 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:24.281 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:28:24.382 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:28:24.409 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:24.410 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:24.410 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:28:24.511 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:28:24.551 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:24.551 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:24.552 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:28:24.653 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:28:24.672 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:28:24.691 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:24.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:24.692 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:28:24.711 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T09:28:24.711 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:28:24.753 INFO:teuthology.orchestra.run.smithi070.stderr:nodeep-scrub is unset 2024-09-16T09:28:24.793 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:28:24.822 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:24 smithi070 ceph-mon[30388]: 5.15 scrub starts 2024-09-16T09:28:24.822 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:24 smithi070 ceph-mon[30388]: 5.15 scrub ok 2024-09-16T09:28:24.823 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:24 smithi070 ceph-mon[30388]: pgmap v144: 158 pgs: 158 active+clean; 4.9 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.43k op/s 2024-09-16T09:28:24.823 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:24 smithi070 ceph-mon[30388]: 4.b scrub starts 2024-09-16T09:28:24.823 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:24 smithi070 ceph-mon[30388]: 4.b scrub ok 2024-09-16T09:28:24.823 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:24 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2479388960' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:28:24.823 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:24 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3984059694' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:28:24.823 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:24 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:28:24.823 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:24 smithi070 ceph-mon[40691]: 5.15 scrub starts 2024-09-16T09:28:24.823 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:24 smithi070 ceph-mon[40691]: 5.15 scrub ok 2024-09-16T09:28:24.823 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:24 smithi070 ceph-mon[40691]: pgmap v144: 158 pgs: 158 active+clean; 4.9 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.43k op/s 2024-09-16T09:28:24.823 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:24 smithi070 ceph-mon[40691]: 4.b scrub starts 2024-09-16T09:28:24.823 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:24 smithi070 ceph-mon[40691]: 4.b scrub ok 2024-09-16T09:28:24.823 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:24 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/2479388960' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:28:24.824 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:24 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3984059694' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:28:24.824 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:24 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:28:24.841 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:24.842 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:24.842 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:28:24.943 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:28:24.970 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:24.971 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:24.971 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:28:25.072 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:28:25.100 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:25.101 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:25.101 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:28:25.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:24 smithi084 ceph-mon[32629]: 5.15 scrub starts 2024-09-16T09:28:25.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:24 smithi084 ceph-mon[32629]: 5.15 scrub ok 2024-09-16T09:28:25.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:24 smithi084 ceph-mon[32629]: pgmap v144: 158 pgs: 158 active+clean; 4.9 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.43k op/s 2024-09-16T09:28:25.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:24 smithi084 ceph-mon[32629]: 4.b scrub starts 2024-09-16T09:28:25.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:24 smithi084 ceph-mon[32629]: 4.b scrub ok 2024-09-16T09:28:25.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:24 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/2479388960' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:28:25.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:24 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3984059694' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:28:25.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:24 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:28:25.202 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:28:25.243 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:25.244 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:25.244 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:28:25.345 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:28:25.372 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:25.372 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:25.373 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:28:25.474 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:28:25.502 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:25.503 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:25.503 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:28:25.604 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:28:25.643 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:25.644 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:25.644 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:28:25.745 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:28:25.785 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:25.786 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:25.786 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:28:25.887 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:28:25.913 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:25.913 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:25.914 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:28:26.015 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:28:26.043 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:26.044 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:26.044 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:28:26.145 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:28:26.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:25 smithi084 ceph-mon[32629]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-16T09:28:26.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:25 smithi084 ceph-mon[32629]: Cluster is now healthy 2024-09-16T09:28:26.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:25 smithi084 ceph-mon[32629]: 5.2 scrub starts 2024-09-16T09:28:26.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:25 smithi084 ceph-mon[32629]: 5.2 scrub ok 2024-09-16T09:28:26.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:25 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-16T09:28:26.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:25 smithi084 ceph-mon[32629]: osdmap e129: 8 total, 8 up, 8 in 2024-09-16T09:28:26.170 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:25 smithi070 ceph-mon[30388]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-16T09:28:26.170 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:25 smithi070 ceph-mon[30388]: Cluster is now healthy 2024-09-16T09:28:26.170 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:25 smithi070 ceph-mon[30388]: 5.2 scrub starts 2024-09-16T09:28:26.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:25 smithi070 ceph-mon[30388]: 5.2 scrub ok 2024-09-16T09:28:26.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:25 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-16T09:28:26.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:25 smithi070 ceph-mon[30388]: osdmap e129: 8 total, 8 up, 8 in 2024-09-16T09:28:26.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:25 smithi070 ceph-mon[40691]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-16T09:28:26.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:25 smithi070 ceph-mon[40691]: Cluster is now healthy 2024-09-16T09:28:26.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:25 smithi070 ceph-mon[40691]: 5.2 scrub starts 2024-09-16T09:28:26.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:25 smithi070 ceph-mon[40691]: 5.2 scrub ok 2024-09-16T09:28:26.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:25 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-16T09:28:26.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:25 smithi070 ceph-mon[40691]: osdmap e129: 8 total, 8 up, 8 in 2024-09-16T09:28:26.172 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:26.172 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:26.172 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:28:26.273 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:28:26.304 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:26.305 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:26.305 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:28:26.406 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:28:26.444 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:26.445 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:26.446 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:28:26.546 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:28:26.572 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:26.573 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:26.573 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:28:26.674 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:28:26.707 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:26.708 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:26.708 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:28:26.809 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:28:26.835 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:26.836 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:26.837 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:28:26.937 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:28:26.967 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:26.968 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:26.968 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:28:27.069 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:28:27.096 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:27.096 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:27.097 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:28:27.099 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:26 smithi070 ceph-mon[30388]: 6.19 scrub starts 2024-09-16T09:28:27.099 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:26 smithi070 ceph-mon[30388]: 6.19 scrub ok 2024-09-16T09:28:27.099 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:26 smithi070 ceph-mon[30388]: pgmap v146: 158 pgs: 158 active+clean; 4.9 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.13k op/s 2024-09-16T09:28:27.099 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:26 smithi070 ceph-mon[30388]: 6.9 scrub starts 2024-09-16T09:28:27.099 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:26 smithi070 ceph-mon[30388]: 6.9 scrub ok 2024-09-16T09:28:27.099 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:26 smithi070 ceph-mon[40691]: 6.19 scrub starts 2024-09-16T09:28:27.100 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:26 smithi070 ceph-mon[40691]: 6.19 scrub ok 2024-09-16T09:28:27.100 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:26 smithi070 ceph-mon[40691]: pgmap v146: 158 pgs: 158 active+clean; 4.9 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.13k op/s 2024-09-16T09:28:27.100 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:26 smithi070 ceph-mon[40691]: 6.9 scrub starts 2024-09-16T09:28:27.100 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:26 smithi070 ceph-mon[40691]: 6.9 scrub ok 2024-09-16T09:28:27.149 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.2 dump_historic_ops 2024-09-16T09:28:27.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:26 smithi084 ceph-mon[32629]: 6.19 scrub starts 2024-09-16T09:28:27.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:26 smithi084 ceph-mon[32629]: 6.19 scrub ok 2024-09-16T09:28:27.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:26 smithi084 ceph-mon[32629]: pgmap v146: 158 pgs: 158 active+clean; 4.9 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.13k op/s 2024-09-16T09:28:27.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:26 smithi084 ceph-mon[32629]: 6.9 scrub starts 2024-09-16T09:28:27.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:26 smithi084 ceph-mon[32629]: 6.9 scrub ok 2024-09-16T09:28:27.200 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:28:27.226 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:27.227 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:27.228 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:28:27.329 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:28:27.353 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:27.354 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:27.355 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:28:27.455 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:28:27.481 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:27.482 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:27.482 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:28:27.583 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:28:27.625 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:27.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:27.626 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:28:27.727 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:28:27.753 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:27.754 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:27.755 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:28:27.800 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:28:27.808 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2024-09-16T09:28:27.855 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:28:27.890 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:27.891 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:27.891 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:28:27.992 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:28:28.033 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:28.034 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:28.034 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:28:28.135 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:28:28.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:27 smithi084 ceph-mon[32629]: 4.16 scrub starts 2024-09-16T09:28:28.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:27 smithi084 ceph-mon[32629]: 4.16 scrub ok 2024-09-16T09:28:28.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:27 smithi084 ceph-mon[32629]: 4.19 scrub starts 2024-09-16T09:28:28.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:27 smithi084 ceph-mon[32629]: 4.19 scrub ok 2024-09-16T09:28:28.170 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:27 smithi070 ceph-mon[30388]: 4.16 scrub starts 2024-09-16T09:28:28.170 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:27 smithi070 ceph-mon[30388]: 4.16 scrub ok 2024-09-16T09:28:28.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:27 smithi070 ceph-mon[30388]: 4.19 scrub starts 2024-09-16T09:28:28.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:27 smithi070 ceph-mon[30388]: 4.19 scrub ok 2024-09-16T09:28:28.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:27 smithi070 ceph-mon[40691]: 4.16 scrub starts 2024-09-16T09:28:28.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:27 smithi070 ceph-mon[40691]: 4.16 scrub ok 2024-09-16T09:28:28.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:27 smithi070 ceph-mon[40691]: 4.19 scrub starts 2024-09-16T09:28:28.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:27 smithi070 ceph-mon[40691]: 4.19 scrub ok 2024-09-16T09:28:28.172 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:28.173 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:28.174 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:28:28.274 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:28:28.307 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:28.308 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:28.309 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:28:28.409 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:28:28.450 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:28.451 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:28.451 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:28:28.551 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:28:28.584 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:28.585 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:28.585 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:28:28.686 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:28:28.723 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:28.724 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:28.724 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:28:28.825 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:28:28.850 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:28.851 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:28.851 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:28:28.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:28 smithi084 ceph-mon[32629]: pgmap v147: 158 pgs: 158 active+clean; 4.8 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 965 op/s 2024-09-16T09:28:28.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:28 smithi084 ceph-mon[32629]: from='client.24814 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:28:28.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:28 smithi084 ceph-mon[32629]: 4.f scrub starts 2024-09-16T09:28:28.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:28 smithi084 ceph-mon[32629]: 4.f scrub ok 2024-09-16T09:28:28.953 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:28:28.979 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:28.980 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:28.980 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:28:29.081 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:28:29.107 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:29.107 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:29.108 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:28:29.126 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:28 smithi070 ceph-mon[30388]: pgmap v147: 158 pgs: 158 active+clean; 4.8 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 965 op/s 2024-09-16T09:28:29.126 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:28 smithi070 ceph-mon[30388]: from='client.24814 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:28:29.126 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:28 smithi070 ceph-mon[30388]: 4.f scrub starts 2024-09-16T09:28:29.126 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:28 smithi070 ceph-mon[30388]: 4.f scrub ok 2024-09-16T09:28:29.127 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:28 smithi070 ceph-mon[40691]: pgmap v147: 158 pgs: 158 active+clean; 4.8 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 965 op/s 2024-09-16T09:28:29.127 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:28 smithi070 ceph-mon[40691]: from='client.24814 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:28:29.127 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:28 smithi070 ceph-mon[40691]: 4.f scrub starts 2024-09-16T09:28:29.127 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:28 smithi070 ceph-mon[40691]: 4.f scrub ok 2024-09-16T09:28:29.175 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":147,"stamp":"2024-09-16T09:28:27.542277+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":5172632115,"num_objects":79119,"num_object_clones":0,"num_object_copies":237357,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79119,"num_whiteouts":0,"num_read":923,"num_read_kb":675,"num_write":173774,"num_write_kb":8072238,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":84780,"ondisk_log_size":84780,"up":474,"acting":474,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":474,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":16057232,"kb_used_data":15257720,"kb_used_omap":12,"kb_used_meta":799091,"kb_avail":733903984,"statfs":{"total":767960285184,"available":751517679616,"internally_reserved":0,"allocated":15623905280,"data_stored":15621265389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":818269746},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3,2,6,3,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-505675776,"num_objects":-7716,"num_object_clones":0,"num_object_copies":-23148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7716,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":7716,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003627"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":159,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-16T09:28:20.707854+0000","last_change":"2024-09-16T09:27:07.621760+0000","last_active":"2024-09-16T09:28:20.707854+0000","last_peered":"2024-09-16T09:28:20.707854+0000","last_clean":"2024-09-16T09:28:20.707854+0000","last_became_active":"2024-09-16T09:25:01.709108+0000","last_became_peered":"2024-09-16T09:25:01.709108+0000","last_unstale":"2024-09-16T09:28:20.707854+0000","last_undegraded":"2024-09-16T09:28:20.707854+0000","last_fullsized":"2024-09-16T09:28:20.707854+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:07.621692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:27:07.621692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"66'9","reported_seq":173,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-16T09:28:24.739024+0000","last_change":"2024-09-16T09:27:45.630451+0000","last_active":"2024-09-16T09:28:24.739024+0000","last_peered":"2024-09-16T09:28:24.739024+0000","last_clean":"2024-09-16T09:28:24.739024+0000","last_became_active":"2024-09-16T09:25:03.714635+0000","last_became_peered":"2024-09-16T09:25:03.714635+0000","last_unstale":"2024-09-16T09:28:24.739024+0000","last_undegraded":"2024-09-16T09:28:24.739024+0000","last_fullsized":"2024-09-16T09:28:24.739024+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:27:45.630310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:45.630310+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:46.857234+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":159,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-16T09:28:20.698229+0000","last_change":"2024-09-16T09:27:58.014601+0000","last_active":"2024-09-16T09:28:20.698229+0000","last_peered":"2024-09-16T09:28:20.698229+0000","last_clean":"2024-09-16T09:28:20.698229+0000","last_became_active":"2024-09-16T09:25:05.715645+0000","last_became_peered":"2024-09-16T09:25:05.715645+0000","last_unstale":"2024-09-16T09:28:20.698229+0000","last_undegraded":"2024-09-16T09:28:20.698229+0000","last_fullsized":"2024-09-16T09:28:20.698229+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:58.014182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:58.014182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:06.935933+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":149,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-16T09:28:24.739139+0000","last_change":"2024-09-16T09:27:11.565306+0000","last_active":"2024-09-16T09:28:24.739139+0000","last_peered":"2024-09-16T09:28:24.739139+0000","last_clean":"2024-09-16T09:28:24.739139+0000","last_became_active":"2024-09-16T09:25:07.715442+0000","last_became_peered":"2024-09-16T09:25:07.715442+0000","last_unstale":"2024-09-16T09:28:24.739139+0000","last_undegraded":"2024-09-16T09:28:24.739139+0000","last_fullsized":"2024-09-16T09:28:24.739139+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:11.565220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:11.565220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":147,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-16T09:28:20.700592+0000","last_change":"2024-09-16T09:28:00.068631+0000","last_active":"2024-09-16T09:28:20.700592+0000","last_peered":"2024-09-16T09:28:20.700592+0000","last_clean":"2024-09-16T09:28:20.700592+0000","last_became_active":"2024-09-16T09:25:07.718126+0000","last_became_peered":"2024-09-16T09:25:07.718126+0000","last_unstale":"2024-09-16T09:28:20.700592+0000","last_undegraded":"2024-09-16T09:28:20.700592+0000","last_fullsized":"2024-09-16T09:28:20.700592+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:00.068570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:28:00.068570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:01.264621+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":159,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-16T09:28:20.701716+0000","last_change":"2024-09-16T09:27:54.078528+0000","last_active":"2024-09-16T09:28:20.701716+0000","last_peered":"2024-09-16T09:28:20.701716+0000","last_clean":"2024-09-16T09:28:20.701716+0000","last_became_active":"2024-09-16T09:25:01.708179+0000","last_became_peered":"2024-09-16T09:25:01.708179+0000","last_unstale":"2024-09-16T09:28:20.701716+0000","last_undegraded":"2024-09-16T09:28:20.701716+0000","last_fullsized":"2024-09-16T09:28:20.701716+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:54.078473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:27:54.078473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:55.232286+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"66'15","reported_seq":180,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-16T09:28:20.701021+0000","last_change":"2024-09-16T09:27:09.887336+0000","last_active":"2024-09-16T09:28:20.701021+0000","last_peered":"2024-09-16T09:28:20.701021+0000","last_clean":"2024-09-16T09:28:20.701021+0000","last_became_active":"2024-09-16T09:25:03.709499+0000","last_became_peered":"2024-09-16T09:25:03.709499+0000","last_unstale":"2024-09-16T09:28:20.701021+0000","last_undegraded":"2024-09-16T09:28:20.701021+0000","last_fullsized":"2024-09-16T09:28:20.701021+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-16T09:27:09.887267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:09.887267+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":161,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-16T09:28:24.738939+0000","last_change":"2024-09-16T09:27:49.653058+0000","last_active":"2024-09-16T09:28:24.738939+0000","last_peered":"2024-09-16T09:28:24.738939+0000","last_clean":"2024-09-16T09:28:24.738939+0000","last_became_active":"2024-09-16T09:25:05.717236+0000","last_became_peered":"2024-09-16T09:25:05.717236+0000","last_unstale":"2024-09-16T09:28:24.738939+0000","last_undegraded":"2024-09-16T09:28:24.738939+0000","last_fullsized":"2024-09-16T09:28:24.738939+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:49.653015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:49.653015+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:09.644746+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":168,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-16T09:28:24.749060+0000","last_change":"2024-09-16T09:28:01.038452+0000","last_active":"2024-09-16T09:28:24.749060+0000","last_peered":"2024-09-16T09:28:24.749060+0000","last_clean":"2024-09-16T09:28:24.749060+0000","last_became_active":"2024-09-16T09:25:01.705269+0000","last_became_peered":"2024-09-16T09:25:01.705269+0000","last_unstale":"2024-09-16T09:28:24.749060+0000","last_undegraded":"2024-09-16T09:28:24.749060+0000","last_fullsized":"2024-09-16T09:28:24.749060+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:01.038400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:28:01.038400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:15.348816+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"66'9","reported_seq":173,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-16T09:28:24.739411+0000","last_change":"2024-09-16T09:27:59.685896+0000","last_active":"2024-09-16T09:28:24.739411+0000","last_peered":"2024-09-16T09:28:24.739411+0000","last_clean":"2024-09-16T09:28:24.739411+0000","last_became_active":"2024-09-16T09:25:03.709147+0000","last_became_peered":"2024-09-16T09:25:03.709147+0000","last_unstale":"2024-09-16T09:28:24.739411+0000","last_undegraded":"2024-09-16T09:28:24.739411+0000","last_fullsized":"2024-09-16T09:28:24.739411+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:27:59.685837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:59.685837+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:25.373463+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":152,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-16T09:28:24.748839+0000","last_change":"2024-09-16T09:27:59.080369+0000","last_active":"2024-09-16T09:28:24.748839+0000","last_peered":"2024-09-16T09:28:24.748839+0000","last_clean":"2024-09-16T09:28:24.748839+0000","last_became_active":"2024-09-16T09:25:05.718134+0000","last_became_peered":"2024-09-16T09:25:05.718134+0000","last_unstale":"2024-09-16T09:28:24.748839+0000","last_undegraded":"2024-09-16T09:28:24.748839+0000","last_fullsized":"2024-09-16T09:28:24.748839+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:59.080307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:59.080307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:26.458257+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"66'1","reported_seq":148,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-16T09:28:20.706887+0000","last_change":"2024-09-16T09:26:24.547980+0000","last_active":"2024-09-16T09:28:20.706887+0000","last_peered":"2024-09-16T09:28:20.706887+0000","last_clean":"2024-09-16T09:28:20.706887+0000","last_became_active":"2024-09-16T09:25:07.716558+0000","last_became_peered":"2024-09-16T09:25:07.716558+0000","last_unstale":"2024-09-16T09:28:20.706887+0000","last_undegraded":"2024-09-16T09:28:20.706887+0000","last_fullsized":"2024-09-16T09:28:20.706887+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-16T09:26:24.547877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:24.547877+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:15.542611+0000","scrub_duration":52,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":160,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-16T09:28:24.748984+0000","last_change":"2024-09-16T09:27:56.191264+0000","last_active":"2024-09-16T09:28:24.748984+0000","last_peered":"2024-09-16T09:28:24.748984+0000","last_clean":"2024-09-16T09:28:24.748984+0000","last_became_active":"2024-09-16T09:25:01.696939+0000","last_became_peered":"2024-09-16T09:25:01.696939+0000","last_unstale":"2024-09-16T09:28:24.748984+0000","last_undegraded":"2024-09-16T09:28:24.748984+0000","last_fullsized":"2024-09-16T09:28:24.748984+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:56.191220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:27:56.191220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:15.886906+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"66'5","reported_seq":167,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-16T09:28:24.739367+0000","last_change":"2024-09-16T09:27:05.694733+0000","last_active":"2024-09-16T09:28:24.739367+0000","last_peered":"2024-09-16T09:28:24.739367+0000","last_clean":"2024-09-16T09:28:24.739367+0000","last_became_active":"2024-09-16T09:25:03.716136+0000","last_became_peered":"2024-09-16T09:25:03.716136+0000","last_unstale":"2024-09-16T09:28:24.739367+0000","last_undegraded":"2024-09-16T09:28:24.739367+0000","last_fullsized":"2024-09-16T09:28:24.739367+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-16T09:27:05.694650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:05.694650+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":152,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-16T09:28:24.757928+0000","last_change":"2024-09-16T09:27:48.682578+0000","last_active":"2024-09-16T09:28:24.757928+0000","last_peered":"2024-09-16T09:28:24.757928+0000","last_clean":"2024-09-16T09:28:24.757928+0000","last_became_active":"2024-09-16T09:25:05.718281+0000","last_became_peered":"2024-09-16T09:25:05.718281+0000","last_unstale":"2024-09-16T09:28:24.757928+0000","last_undegraded":"2024-09-16T09:28:24.757928+0000","last_fullsized":"2024-09-16T09:28:24.757928+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:48.682528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:48.682528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:52.128525+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":156,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-16T09:28:25.246140+0000","last_change":"2024-09-16T09:28:25.246140+0000","last_active":"2024-09-16T09:28:25.246140+0000","last_peered":"2024-09-16T09:28:25.246140+0000","last_clean":"2024-09-16T09:28:25.246140+0000","last_became_active":"2024-09-16T09:25:07.716724+0000","last_became_peered":"2024-09-16T09:25:07.716724+0000","last_unstale":"2024-09-16T09:28:25.246140+0000","last_undegraded":"2024-09-16T09:28:25.246140+0000","last_fullsized":"2024-09-16T09:28:25.246140+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:25.246067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:28:25.246067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:46.649209+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":157,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-16T09:28:24.739871+0000","last_change":"2024-09-16T09:28:20.662726+0000","last_active":"2024-09-16T09:28:24.739871+0000","last_peered":"2024-09-16T09:28:24.739871+0000","last_clean":"2024-09-16T09:28:24.739871+0000","last_became_active":"2024-09-16T09:25:07.720793+0000","last_became_peered":"2024-09-16T09:25:07.720793+0000","last_unstale":"2024-09-16T09:28:24.739871+0000","last_undegraded":"2024-09-16T09:28:24.739871+0000","last_fullsized":"2024-09-16T09:28:24.739871+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:20.662663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:28:20.662663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:46.306136+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":159,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-16T09:28:20.707882+0000","last_change":"2024-09-16T09:27:09.635443+0000","last_active":"2024-09-16T09:28:20.707882+0000","last_peered":"2024-09-16T09:28:20.707882+0000","last_clean":"2024-09-16T09:28:20.707882+0000","last_became_active":"2024-09-16T09:25:01.706172+0000","last_became_peered":"2024-09-16T09:25:01.706172+0000","last_unstale":"2024-09-16T09:28:20.707882+0000","last_undegraded":"2024-09-16T09:28:20.707882+0000","last_fullsized":"2024-09-16T09:28:20.707882+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:09.635363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:27:09.635363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:25.991704+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"66'15","reported_seq":180,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-16T09:28:20.706974+0000","last_change":"2024-09-16T09:26:33.873905+0000","last_active":"2024-09-16T09:28:20.706974+0000","last_peered":"2024-09-16T09:28:20.706974+0000","last_clean":"2024-09-16T09:28:20.706974+0000","last_became_active":"2024-09-16T09:25:03.713780+0000","last_became_peered":"2024-09-16T09:25:03.713780+0000","last_unstale":"2024-09-16T09:28:20.706974+0000","last_undegraded":"2024-09-16T09:28:20.706974+0000","last_fullsized":"2024-09-16T09:28:20.706974+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-16T09:26:33.873817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:26:33.873817+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:02.799432+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":151,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-16T09:28:20.699536+0000","last_change":"2024-09-16T09:27:16.266378+0000","last_active":"2024-09-16T09:28:20.699536+0000","last_peered":"2024-09-16T09:28:20.699536+0000","last_clean":"2024-09-16T09:28:20.699536+0000","last_became_active":"2024-09-16T09:25:05.713929+0000","last_became_peered":"2024-09-16T09:25:05.713929+0000","last_unstale":"2024-09-16T09:28:20.699536+0000","last_undegraded":"2024-09-16T09:28:20.699536+0000","last_fullsized":"2024-09-16T09:28:20.699536+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:16.266297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:16.266297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:38.874875+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":147,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-16T09:28:20.710903+0000","last_change":"2024-09-16T09:26:39.358604+0000","last_active":"2024-09-16T09:28:20.710903+0000","last_peered":"2024-09-16T09:28:20.710903+0000","last_clean":"2024-09-16T09:28:20.710903+0000","last_became_active":"2024-09-16T09:25:07.719067+0000","last_became_peered":"2024-09-16T09:25:07.719067+0000","last_unstale":"2024-09-16T09:28:20.710903+0000","last_undegraded":"2024-09-16T09:28:20.710903+0000","last_fullsized":"2024-09-16T09:28:20.710903+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:39.358091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:39.358091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:11.102980+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":161,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-16T09:28:24.739485+0000","last_change":"2024-09-16T09:26:32.756725+0000","last_active":"2024-09-16T09:28:24.739485+0000","last_peered":"2024-09-16T09:28:24.739485+0000","last_clean":"2024-09-16T09:28:24.739485+0000","last_became_active":"2024-09-16T09:25:01.706491+0000","last_became_peered":"2024-09-16T09:25:01.706491+0000","last_unstale":"2024-09-16T09:28:24.739485+0000","last_undegraded":"2024-09-16T09:28:24.739485+0000","last_fullsized":"2024-09-16T09:28:24.739485+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:32.756657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:32.756657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"66'12","reported_seq":178,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-16T09:28:20.699804+0000","last_change":"2024-09-16T09:27:01.700219+0000","last_active":"2024-09-16T09:28:20.699804+0000","last_peered":"2024-09-16T09:28:20.699804+0000","last_clean":"2024-09-16T09:28:20.699804+0000","last_became_active":"2024-09-16T09:25:03.709610+0000","last_became_peered":"2024-09-16T09:25:03.709610+0000","last_unstale":"2024-09-16T09:28:20.699804+0000","last_undegraded":"2024-09-16T09:28:20.699804+0000","last_fullsized":"2024-09-16T09:28:20.699804+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-16T09:27:01.700139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:01.700139+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:28.256362+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":153,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-16T09:28:24.739654+0000","last_change":"2024-09-16T09:27:00.816500+0000","last_active":"2024-09-16T09:28:24.739654+0000","last_peered":"2024-09-16T09:28:24.739654+0000","last_clean":"2024-09-16T09:28:24.739654+0000","last_became_active":"2024-09-16T09:25:05.710093+0000","last_became_peered":"2024-09-16T09:25:05.710093+0000","last_unstale":"2024-09-16T09:28:24.739654+0000","last_undegraded":"2024-09-16T09:28:24.739654+0000","last_fullsized":"2024-09-16T09:28:24.739654+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:00.816424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:00.816424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":149,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-16T09:28:24.758438+0000","last_change":"2024-09-16T09:27:59.801997+0000","last_active":"2024-09-16T09:28:24.758438+0000","last_peered":"2024-09-16T09:28:24.758438+0000","last_clean":"2024-09-16T09:28:24.758438+0000","last_became_active":"2024-09-16T09:25:07.719380+0000","last_became_peered":"2024-09-16T09:25:07.719380+0000","last_unstale":"2024-09-16T09:28:24.758438+0000","last_undegraded":"2024-09-16T09:28:24.758438+0000","last_fullsized":"2024-09-16T09:28:24.758438+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-16T09:27:59.801924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:59.801924+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:26.228249+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":168,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-16T09:28:20.698355+0000","last_change":"2024-09-16T09:28:17.993009+0000","last_active":"2024-09-16T09:28:20.698355+0000","last_peered":"2024-09-16T09:28:20.698355+0000","last_clean":"2024-09-16T09:28:20.698355+0000","last_became_active":"2024-09-16T09:25:01.699545+0000","last_became_peered":"2024-09-16T09:25:01.699545+0000","last_unstale":"2024-09-16T09:28:20.698355+0000","last_undegraded":"2024-09-16T09:28:20.698355+0000","last_fullsized":"2024-09-16T09:28:20.698355+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:28:17.992925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:28:17.992925+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:18.647927+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"66'10","reported_seq":178,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-16T09:28:20.708328+0000","last_change":"2024-09-16T09:27:43.556435+0000","last_active":"2024-09-16T09:28:20.708328+0000","last_peered":"2024-09-16T09:28:20.708328+0000","last_clean":"2024-09-16T09:28:20.708328+0000","last_became_active":"2024-09-16T09:25:03.720245+0000","last_became_peered":"2024-09-16T09:25:03.720245+0000","last_unstale":"2024-09-16T09:28:20.708328+0000","last_undegraded":"2024-09-16T09:28:20.708328+0000","last_fullsized":"2024-09-16T09:28:20.708328+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-16T09:27:43.556357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:43.556357+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:13.459515+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"66'8","reported_seq":214,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-16T09:28:24.752631+0000","last_change":"2024-09-16T09:27:51.374575+0000","last_active":"2024-09-16T09:28:24.752631+0000","last_peered":"2024-09-16T09:28:24.752631+0000","last_clean":"2024-09-16T09:28:24.752631+0000","last_became_active":"2024-09-16T09:25:05.712516+0000","last_became_peered":"2024-09-16T09:25:05.712516+0000","last_unstale":"2024-09-16T09:28:24.752631+0000","last_undegraded":"2024-09-16T09:28:24.752631+0000","last_fullsized":"2024-09-16T09:28:24.752631+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:27:51.374504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:51.374504+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:58.529957+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.a","version":"73'79","reported_seq":88,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-16T09:28:24.748721+0000","last_change":"2024-09-16T09:26:36.169945+0000","last_active":"2024-09-16T09:28:24.748721+0000","last_peered":"2024-09-16T09:28:24.748721+0000","last_clean":"2024-09-16T09:28:24.748721+0000","last_became_active":"2024-09-16T09:26:35.127352+0000","last_became_peered":"2024-09-16T09:26:35.127352+0000","last_unstale":"2024-09-16T09:28:24.748721+0000","last_undegraded":"2024-09-16T09:28:24.748721+0000","last_fullsized":"2024-09-16T09:28:24.748721+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:26:36.169870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:26:36.169870+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:13.960880+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.c","version":"129'10926","reported_seq":11032,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-16T09:28:27.508874+0000","last_change":"2024-09-16T09:26:08.117066+0000","last_active":"2024-09-16T09:28:27.508874+0000","last_peered":"2024-09-16T09:28:27.508874+0000","last_clean":"2024-09-16T09:28:27.508874+0000","last_became_active":"2024-09-16T09:26:08.116783+0000","last_became_peered":"2024-09-16T09:26:08.116783+0000","last_unstale":"2024-09-16T09:28:27.508874+0000","last_undegraded":"2024-09-16T09:28:27.508874+0000","last_fullsized":"2024-09-16T09:28:27.508874+0000","mapping_epoch":82,"log_start":"103'5200","ondisk_log_start":"103'5200","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_clean_scrub_stamp":"2024-09-16T09:26:07.096764+0000","objects_scrubbed":0,"log_size":5726,"log_dups_size":0,"ondisk_log_size":5726,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:32.278987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":310509568,"num_objects":4738,"num_object_clones":0,"num_object_copies":14214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4738,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10926,"num_write_kb":501248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.f","version":"66'15","reported_seq":98,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-16T09:28:20.707275+0000","last_change":"2024-09-16T09:26:33.348884+0000","last_active":"2024-09-16T09:28:20.707275+0000","last_peered":"2024-09-16T09:28:20.707275+0000","last_clean":"2024-09-16T09:28:20.707275+0000","last_became_active":"2024-09-16T09:26:33.295672+0000","last_became_peered":"2024-09-16T09:26:33.295672+0000","last_unstale":"2024-09-16T09:28:20.707275+0000","last_undegraded":"2024-09-16T09:28:20.707275+0000","last_fullsized":"2024-09-16T09:28:20.707275+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-16T09:26:23.191684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:26:23.191684+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:27.974437+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":167,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-16T09:28:22.091708+0000","last_change":"2024-09-16T09:28:22.091708+0000","last_active":"2024-09-16T09:28:22.091708+0000","last_peered":"2024-09-16T09:28:22.091708+0000","last_clean":"2024-09-16T09:28:22.091708+0000","last_became_active":"2024-09-16T09:25:01.702629+0000","last_became_peered":"2024-09-16T09:25:01.702629+0000","last_unstale":"2024-09-16T09:28:22.091708+0000","last_undegraded":"2024-09-16T09:28:22.091708+0000","last_fullsized":"2024-09-16T09:28:22.091708+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:22.091537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:28:22.091537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:51.478241+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"66'8","reported_seq":195,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-16T09:28:24.739744+0000","last_change":"2024-09-16T09:25:05.714128+0000","last_active":"2024-09-16T09:28:24.739744+0000","last_peered":"2024-09-16T09:28:24.739744+0000","last_clean":"2024-09-16T09:28:24.739744+0000","last_became_active":"2024-09-16T09:25:05.713813+0000","last_became_peered":"2024-09-16T09:25:05.713813+0000","last_unstale":"2024-09-16T09:28:24.739744+0000","last_undegraded":"2024-09-16T09:28:24.739744+0000","last_fullsized":"2024-09-16T09:28:24.739744+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:12.670604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":148,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-16T09:28:24.748726+0000","last_change":"2024-09-16T09:26:38.289448+0000","last_active":"2024-09-16T09:28:24.748726+0000","last_peered":"2024-09-16T09:28:24.748726+0000","last_clean":"2024-09-16T09:28:24.748726+0000","last_became_active":"2024-09-16T09:25:07.717344+0000","last_became_peered":"2024-09-16T09:25:07.717344+0000","last_unstale":"2024-09-16T09:28:24.748726+0000","last_undegraded":"2024-09-16T09:28:24.748726+0000","last_fullsized":"2024-09-16T09:28:24.748726+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:38.289387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:38.289387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:14.990613+0000","scrub_duration":114,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.5","version":"73'79","reported_seq":88,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-16T09:28:24.739786+0000","last_change":"2024-09-16T09:26:35.687824+0000","last_active":"2024-09-16T09:28:24.739786+0000","last_peered":"2024-09-16T09:28:24.739786+0000","last_clean":"2024-09-16T09:28:24.739786+0000","last_became_active":"2024-09-16T09:26:35.117714+0000","last_became_peered":"2024-09-16T09:26:35.117714+0000","last_unstale":"2024-09-16T09:28:24.739786+0000","last_undegraded":"2024-09-16T09:28:24.739786+0000","last_fullsized":"2024-09-16T09:28:24.739786+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:26:35.687778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:26:35.687778+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":25,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3","version":"128'10904","reported_seq":11008,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-16T09:28:22.779083+0000","last_change":"2024-09-16T09:26:08.119864+0000","last_active":"2024-09-16T09:28:22.779083+0000","last_peered":"2024-09-16T09:28:22.779083+0000","last_clean":"2024-09-16T09:28:22.779083+0000","last_became_active":"2024-09-16T09:26:08.119550+0000","last_became_peered":"2024-09-16T09:26:08.119550+0000","last_unstale":"2024-09-16T09:28:22.779083+0000","last_undegraded":"2024-09-16T09:28:22.779083+0000","last_fullsized":"2024-09-16T09:28:22.779083+0000","mapping_epoch":82,"log_start":"111'6600","ondisk_log_start":"111'6600","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_clean_scrub_stamp":"2024-09-16T09:26:07.096764+0000","objects_scrubbed":0,"log_size":4304,"log_dups_size":0,"ondisk_log_size":4304,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:33.053037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":329515008,"num_objects":5028,"num_object_clones":0,"num_object_copies":15084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5028,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10904,"num_write_kb":509824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4],"acting":[3,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"66'18","reported_seq":187,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-16T09:28:20.700012+0000","last_change":"2024-09-16T09:27:08.857567+0000","last_active":"2024-09-16T09:28:20.700012+0000","last_peered":"2024-09-16T09:28:20.700012+0000","last_clean":"2024-09-16T09:28:20.700012+0000","last_became_active":"2024-09-16T09:25:03.714208+0000","last_became_peered":"2024-09-16T09:25:03.714208+0000","last_unstale":"2024-09-16T09:28:20.700012+0000","last_undegraded":"2024-09-16T09:28:20.700012+0000","last_fullsized":"2024-09-16T09:28:20.700012+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'18","last_scrub_stamp":"2024-09-16T09:27:08.857484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:08.857484+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:31.036637+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":167,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-16T09:28:20.711568+0000","last_change":"2024-09-16T09:27:46.208398+0000","last_active":"2024-09-16T09:28:20.711568+0000","last_peered":"2024-09-16T09:28:20.711568+0000","last_clean":"2024-09-16T09:28:20.711568+0000","last_became_active":"2024-09-16T09:25:01.704994+0000","last_became_peered":"2024-09-16T09:25:01.704994+0000","last_unstale":"2024-09-16T09:28:20.711568+0000","last_undegraded":"2024-09-16T09:28:20.711568+0000","last_fullsized":"2024-09-16T09:28:20.711568+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:46.208293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:27:46.208293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:00.017707+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":153,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-16T09:28:24.739121+0000","last_change":"2024-09-16T09:26:33.729827+0000","last_active":"2024-09-16T09:28:24.739121+0000","last_peered":"2024-09-16T09:28:24.739121+0000","last_clean":"2024-09-16T09:28:24.739121+0000","last_became_active":"2024-09-16T09:25:05.713360+0000","last_became_peered":"2024-09-16T09:25:05.713360+0000","last_unstale":"2024-09-16T09:28:24.739121+0000","last_undegraded":"2024-09-16T09:28:24.739121+0000","last_fullsized":"2024-09-16T09:28:24.739121+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:33.729770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:26:33.729770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":149,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-16T09:28:24.739242+0000","last_change":"2024-09-16T09:27:13.548667+0000","last_active":"2024-09-16T09:28:24.739242+0000","last_peered":"2024-09-16T09:28:24.739242+0000","last_clean":"2024-09-16T09:28:24.739242+0000","last_became_active":"2024-09-16T09:25:07.719159+0000","last_became_peered":"2024-09-16T09:25:07.719159+0000","last_unstale":"2024-09-16T09:28:24.739242+0000","last_undegraded":"2024-09-16T09:28:24.739242+0000","last_fullsized":"2024-09-16T09:28:24.739242+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:13.548608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:13.548608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"73'79","reported_seq":91,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-16T09:28:20.703432+0000","last_change":"2024-09-16T09:26:35.426510+0000","last_active":"2024-09-16T09:28:20.703432+0000","last_peered":"2024-09-16T09:28:20.703432+0000","last_clean":"2024-09-16T09:28:20.703432+0000","last_became_active":"2024-09-16T09:26:35.131343+0000","last_became_peered":"2024-09-16T09:26:35.131343+0000","last_unstale":"2024-09-16T09:28:20.703432+0000","last_undegraded":"2024-09-16T09:28:20.703432+0000","last_fullsized":"2024-09-16T09:28:20.703432+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:26:35.426313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:26:35.426313+0000","objects_scrubbed":1,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2","version":"128'10863","reported_seq":10967,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-16T09:28:22.747974+0000","last_change":"2024-09-16T09:26:08.120627+0000","last_active":"2024-09-16T09:28:22.747974+0000","last_peered":"2024-09-16T09:28:22.747974+0000","last_clean":"2024-09-16T09:28:22.747974+0000","last_became_active":"2024-09-16T09:26:08.120338+0000","last_became_peered":"2024-09-16T09:26:08.120338+0000","last_unstale":"2024-09-16T09:28:22.747974+0000","last_undegraded":"2024-09-16T09:28:22.747974+0000","last_fullsized":"2024-09-16T09:28:22.747974+0000","mapping_epoch":82,"log_start":"110'6500","ondisk_log_start":"110'6500","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_clean_scrub_stamp":"2024-09-16T09:26:07.096764+0000","objects_scrubbed":0,"log_size":4363,"log_dups_size":0,"ondisk_log_size":4363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:38.116462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330760192,"num_objects":5047,"num_object_clones":0,"num_object_copies":15141,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5047,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10863,"num_write_kb":509120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"66'14","reported_seq":178,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-16T09:28:24.739168+0000","last_change":"2024-09-16T09:26:38.595468+0000","last_active":"2024-09-16T09:28:24.739168+0000","last_peered":"2024-09-16T09:28:24.739168+0000","last_clean":"2024-09-16T09:28:24.739168+0000","last_became_active":"2024-09-16T09:25:03.708830+0000","last_became_peered":"2024-09-16T09:25:03.708830+0000","last_unstale":"2024-09-16T09:28:24.739168+0000","last_undegraded":"2024-09-16T09:28:24.739168+0000","last_fullsized":"2024-09-16T09:28:24.739168+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'14","last_scrub_stamp":"2024-09-16T09:26:38.595391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:26:38.595391+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":168,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-16T09:28:20.708569+0000","last_change":"2024-09-16T09:27:14.637291+0000","last_active":"2024-09-16T09:28:20.708569+0000","last_peered":"2024-09-16T09:28:20.708569+0000","last_clean":"2024-09-16T09:28:20.708569+0000","last_became_active":"2024-09-16T09:25:01.702992+0000","last_became_peered":"2024-09-16T09:25:01.702992+0000","last_unstale":"2024-09-16T09:28:20.708569+0000","last_undegraded":"2024-09-16T09:28:20.708569+0000","last_fullsized":"2024-09-16T09:28:20.708569+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:27:14.637226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:27:14.637226+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:43.815503+0000","scrub_duration":37,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"66'8","reported_seq":200,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-16T09:28:20.699835+0000","last_change":"2024-09-16T09:28:01.096878+0000","last_active":"2024-09-16T09:28:20.699835+0000","last_peered":"2024-09-16T09:28:20.699835+0000","last_clean":"2024-09-16T09:28:20.699835+0000","last_became_active":"2024-09-16T09:25:05.715606+0000","last_became_peered":"2024-09-16T09:25:05.715606+0000","last_unstale":"2024-09-16T09:28:20.699835+0000","last_undegraded":"2024-09-16T09:28:20.699835+0000","last_fullsized":"2024-09-16T09:28:20.699835+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:28:01.096510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:28:01.096510+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:03.537462+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":148,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-16T09:28:24.757801+0000","last_change":"2024-09-16T09:26:40.699888+0000","last_active":"2024-09-16T09:28:24.757801+0000","last_peered":"2024-09-16T09:28:24.757801+0000","last_clean":"2024-09-16T09:28:24.757801+0000","last_became_active":"2024-09-16T09:25:07.722017+0000","last_became_peered":"2024-09-16T09:25:07.722017+0000","last_unstale":"2024-09-16T09:28:24.757801+0000","last_undegraded":"2024-09-16T09:28:24.757801+0000","last_fullsized":"2024-09-16T09:28:24.757801+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:40.699813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:40.699813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:10.826664+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"73'79","reported_seq":172,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-16T09:28:24.750701+0000","last_change":"2024-09-16T09:27:59.497431+0000","last_active":"2024-09-16T09:28:24.750701+0000","last_peered":"2024-09-16T09:28:24.750701+0000","last_clean":"2024-09-16T09:28:24.750701+0000","last_became_active":"2024-09-16T09:26:35.126759+0000","last_became_peered":"2024-09-16T09:26:35.126759+0000","last_unstale":"2024-09-16T09:28:24.750701+0000","last_undegraded":"2024-09-16T09:28:24.750701+0000","last_fullsized":"2024-09-16T09:28:24.750701+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:27:59.497364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:27:59.497364+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:12.907753+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"129'10867","reported_seq":11035,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-16T09:28:27.505472+0000","last_change":"2024-09-16T09:28:18.935082+0000","last_active":"2024-09-16T09:28:27.505472+0000","last_peered":"2024-09-16T09:28:27.505472+0000","last_clean":"2024-09-16T09:28:27.505472+0000","last_became_active":"2024-09-16T09:26:08.116726+0000","last_became_peered":"2024-09-16T09:26:08.116726+0000","last_unstale":"2024-09-16T09:28:27.505472+0000","last_undegraded":"2024-09-16T09:28:27.505472+0000","last_fullsized":"2024-09-16T09:28:27.505472+0000","mapping_epoch":82,"log_start":"110'6300","ondisk_log_start":"110'6300","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'10297","last_scrub_stamp":"2024-09-16T09:28:18.934970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_clean_scrub_stamp":"2024-09-16T09:28:18.934970+0000","objects_scrubbed":5294,"log_size":4567,"log_dups_size":0,"ondisk_log_size":4567,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:35.502825+0000","scrub_duration":1205,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":306642944,"num_objects":4679,"num_object_clones":0,"num_object_copies":14037,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4679,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10867,"num_write_kb":497472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"66'10","reported_seq":170,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-16T09:28:20.702386+0000","last_change":"2024-09-16T09:26:28.194465+0000","last_active":"2024-09-16T09:28:20.702386+0000","last_peered":"2024-09-16T09:28:20.702386+0000","last_clean":"2024-09-16T09:28:20.702386+0000","last_became_active":"2024-09-16T09:25:03.712985+0000","last_became_peered":"2024-09-16T09:25:03.712985+0000","last_unstale":"2024-09-16T09:28:20.702386+0000","last_undegraded":"2024-09-16T09:28:20.702386+0000","last_fullsized":"2024-09-16T09:28:20.702386+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-16T09:26:28.194023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:26:28.194023+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:30.198573+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":160,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-16T09:28:24.749109+0000","last_change":"2024-09-16T09:26:25.086388+0000","last_active":"2024-09-16T09:28:24.749109+0000","last_peered":"2024-09-16T09:28:24.749109+0000","last_clean":"2024-09-16T09:28:24.749109+0000","last_became_active":"2024-09-16T09:25:01.706440+0000","last_became_peered":"2024-09-16T09:25:01.706440+0000","last_unstale":"2024-09-16T09:28:24.749109+0000","last_undegraded":"2024-09-16T09:28:24.749109+0000","last_fullsized":"2024-09-16T09:28:24.749109+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:25.086322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:25.086322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:01.991189+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"66'8","reported_seq":200,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-16T09:28:20.707552+0000","last_change":"2024-09-16T09:27:17.664572+0000","last_active":"2024-09-16T09:28:20.707552+0000","last_peered":"2024-09-16T09:28:20.707552+0000","last_clean":"2024-09-16T09:28:20.707552+0000","last_became_active":"2024-09-16T09:25:05.714507+0000","last_became_peered":"2024-09-16T09:25:05.714507+0000","last_unstale":"2024-09-16T09:28:20.707552+0000","last_undegraded":"2024-09-16T09:28:20.707552+0000","last_fullsized":"2024-09-16T09:28:20.707552+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:27:17.664493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:17.664493+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:36.536988+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":155,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-16T09:28:22.563466+0000","last_change":"2024-09-16T09:28:22.563466+0000","last_active":"2024-09-16T09:28:22.563466+0000","last_peered":"2024-09-16T09:28:22.563466+0000","last_clean":"2024-09-16T09:28:22.563466+0000","last_became_active":"2024-09-16T09:25:07.719604+0000","last_became_peered":"2024-09-16T09:25:07.719604+0000","last_unstale":"2024-09-16T09:28:22.563466+0000","last_undegraded":"2024-09-16T09:28:22.563466+0000","last_fullsized":"2024-09-16T09:28:22.563466+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:22.563395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:28:22.563395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:45.474687+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"73'79","reported_seq":164,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-16T09:28:24.750396+0000","last_change":"2024-09-16T09:26:35.170288+0000","last_active":"2024-09-16T09:28:24.750396+0000","last_peered":"2024-09-16T09:28:24.750396+0000","last_clean":"2024-09-16T09:28:24.750396+0000","last_became_active":"2024-09-16T09:26:35.125571+0000","last_became_peered":"2024-09-16T09:26:35.125571+0000","last_unstale":"2024-09-16T09:28:24.750396+0000","last_undegraded":"2024-09-16T09:28:24.750396+0000","last_fullsized":"2024-09-16T09:28:24.750396+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:26:35.170229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:26:35.170229+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:04.472831+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.0","version":"129'11104","reported_seq":11280,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-16T09:28:27.496161+0000","last_change":"2024-09-16T09:27:16.936395+0000","last_active":"2024-09-16T09:28:27.496161+0000","last_peered":"2024-09-16T09:28:27.496161+0000","last_clean":"2024-09-16T09:28:27.496161+0000","last_became_active":"2024-09-16T09:26:08.117581+0000","last_became_peered":"2024-09-16T09:26:08.117581+0000","last_unstale":"2024-09-16T09:28:27.496161+0000","last_undegraded":"2024-09-16T09:28:27.496161+0000","last_fullsized":"2024-09-16T09:28:27.496161+0000","mapping_epoch":82,"log_start":"105'5700","ondisk_log_start":"105'5700","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"109'6200","last_scrub_stamp":"2024-09-16T09:27:16.936218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_clean_scrub_stamp":"2024-09-16T09:27:16.936218+0000","objects_scrubbed":6157,"log_size":5404,"log_dups_size":0,"ondisk_log_size":5404,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:45.561197+0000","scrub_duration":1612,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":309854208,"num_objects":4728,"num_object_clones":0,"num_object_copies":14184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4728,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11104,"num_write_kb":506624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3","version":"66'19","reported_seq":183,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-16T09:28:20.707687+0000","last_change":"2024-09-16T09:25:03.715086+0000","last_active":"2024-09-16T09:28:20.707687+0000","last_peered":"2024-09-16T09:28:20.707687+0000","last_clean":"2024-09-16T09:28:20.707687+0000","last_became_active":"2024-09-16T09:25:03.714797+0000","last_became_peered":"2024-09-16T09:25:03.714797+0000","last_unstale":"2024-09-16T09:28:20.707687+0000","last_undegraded":"2024-09-16T09:28:20.707687+0000","last_fullsized":"2024-09-16T09:28:20.707687+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:42.581621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":167,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-16T09:28:20.702753+0000","last_change":"2024-09-16T09:27:24.219132+0000","last_active":"2024-09-16T09:28:20.702753+0000","last_peered":"2024-09-16T09:28:20.702753+0000","last_clean":"2024-09-16T09:28:20.702753+0000","last_became_active":"2024-09-16T09:25:01.706806+0000","last_became_peered":"2024-09-16T09:25:01.706806+0000","last_unstale":"2024-09-16T09:28:20.702753+0000","last_undegraded":"2024-09-16T09:28:20.702753+0000","last_fullsized":"2024-09-16T09:28:20.702753+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:24.218910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:27:24.218910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:52.294630+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":161,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-16T09:28:24.749819+0000","last_change":"2024-09-16T09:28:24.737761+0000","last_active":"2024-09-16T09:28:24.749819+0000","last_peered":"2024-09-16T09:28:24.749819+0000","last_clean":"2024-09-16T09:28:24.749819+0000","last_became_active":"2024-09-16T09:25:05.708663+0000","last_became_peered":"2024-09-16T09:25:05.708663+0000","last_unstale":"2024-09-16T09:28:24.749819+0000","last_undegraded":"2024-09-16T09:28:24.749819+0000","last_fullsized":"2024-09-16T09:28:24.749819+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:24.737702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:28:24.737702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:40.005585+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":147,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-16T09:28:20.698144+0000","last_change":"2024-09-16T09:26:37.282798+0000","last_active":"2024-09-16T09:28:20.698144+0000","last_peered":"2024-09-16T09:28:20.698144+0000","last_clean":"2024-09-16T09:28:20.698144+0000","last_became_active":"2024-09-16T09:25:07.719267+0000","last_became_peered":"2024-09-16T09:25:07.719267+0000","last_unstale":"2024-09-16T09:28:20.698144+0000","last_undegraded":"2024-09-16T09:28:20.698144+0000","last_fullsized":"2024-09-16T09:28:20.698144+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:37.282735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:37.282735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:04.927965+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"73'79","reported_seq":88,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-16T09:28:24.748532+0000","last_change":"2024-09-16T09:26:35.154953+0000","last_active":"2024-09-16T09:28:24.748532+0000","last_peered":"2024-09-16T09:28:24.748532+0000","last_clean":"2024-09-16T09:28:24.748532+0000","last_became_active":"2024-09-16T09:26:35.117627+0000","last_became_peered":"2024-09-16T09:26:35.117627+0000","last_unstale":"2024-09-16T09:28:24.748532+0000","last_undegraded":"2024-09-16T09:28:24.748532+0000","last_fullsized":"2024-09-16T09:28:24.748532+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:26:35.154875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:26:35.154875+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:04.931144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"129'11120","reported_seq":11226,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-16T09:28:27.512866+0000","last_change":"2024-09-16T09:26:08.122122+0000","last_active":"2024-09-16T09:28:27.512866+0000","last_peered":"2024-09-16T09:28:27.512866+0000","last_clean":"2024-09-16T09:28:27.512866+0000","last_became_active":"2024-09-16T09:26:08.121828+0000","last_became_peered":"2024-09-16T09:26:08.121828+0000","last_unstale":"2024-09-16T09:28:27.512866+0000","last_undegraded":"2024-09-16T09:28:27.512866+0000","last_fullsized":"2024-09-16T09:28:27.512866+0000","mapping_epoch":82,"log_start":"104'5400","ondisk_log_start":"104'5400","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_clean_scrub_stamp":"2024-09-16T09:26:07.096764+0000","objects_scrubbed":0,"log_size":5720,"log_dups_size":0,"ondisk_log_size":5720,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:40.329742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":313655296,"num_objects":4786,"num_object_clones":0,"num_object_copies":14358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4786,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11120,"num_write_kb":508992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.4","version":"66'28","reported_seq":244,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-16T09:28:20.702641+0000","last_change":"2024-09-16T09:27:50.051292+0000","last_active":"2024-09-16T09:28:20.702641+0000","last_peered":"2024-09-16T09:28:20.702641+0000","last_clean":"2024-09-16T09:28:20.702641+0000","last_became_active":"2024-09-16T09:25:03.704878+0000","last_became_peered":"2024-09-16T09:25:03.704878+0000","last_unstale":"2024-09-16T09:28:20.702641+0000","last_undegraded":"2024-09-16T09:28:20.702641+0000","last_fullsized":"2024-09-16T09:28:20.702641+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'28","last_scrub_stamp":"2024-09-16T09:27:50.046365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:50.046365+0000","objects_scrubbed":10,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:08.615060+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":161,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-16T09:28:24.739605+0000","last_change":"2024-09-16T09:28:00.718726+0000","last_active":"2024-09-16T09:28:24.739605+0000","last_peered":"2024-09-16T09:28:24.739605+0000","last_clean":"2024-09-16T09:28:24.739605+0000","last_became_active":"2024-09-16T09:25:01.703076+0000","last_became_peered":"2024-09-16T09:25:01.703076+0000","last_unstale":"2024-09-16T09:28:24.739605+0000","last_undegraded":"2024-09-16T09:28:24.739605+0000","last_fullsized":"2024-09-16T09:28:24.739605+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:00.718684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:28:00.718684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:21.876192+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":170,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-16T09:28:24.749113+0000","last_change":"2024-09-16T09:27:44.183242+0000","last_active":"2024-09-16T09:28:24.749113+0000","last_peered":"2024-09-16T09:28:24.749113+0000","last_clean":"2024-09-16T09:28:24.749113+0000","last_became_active":"2024-09-16T09:25:00.696237+0000","last_became_peered":"2024-09-16T09:25:00.696237+0000","last_unstale":"2024-09-16T09:28:24.749113+0000","last_undegraded":"2024-09-16T09:28:24.749113+0000","last_fullsized":"2024-09-16T09:28:24.749113+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-16T09:27:44.183111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:27:44.183111+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:47.187301+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.0025137430000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":143,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-16T09:28:20.707649+0000","last_change":"2024-09-16T09:25:05.718319+0000","last_active":"2024-09-16T09:28:20.707649+0000","last_peered":"2024-09-16T09:28:20.707649+0000","last_clean":"2024-09-16T09:28:20.707649+0000","last_became_active":"2024-09-16T09:25:05.717591+0000","last_became_peered":"2024-09-16T09:25:05.717591+0000","last_unstale":"2024-09-16T09:28:20.707649+0000","last_undegraded":"2024-09-16T09:28:20.707649+0000","last_fullsized":"2024-09-16T09:28:20.707649+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:07.631783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":155,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-16T09:28:20.701518+0000","last_change":"2024-09-16T09:27:45.201574+0000","last_active":"2024-09-16T09:28:20.701518+0000","last_peered":"2024-09-16T09:28:20.701518+0000","last_clean":"2024-09-16T09:28:20.701518+0000","last_became_active":"2024-09-16T09:25:07.717436+0000","last_became_peered":"2024-09-16T09:25:07.717436+0000","last_unstale":"2024-09-16T09:28:20.701518+0000","last_undegraded":"2024-09-16T09:28:20.701518+0000","last_fullsized":"2024-09-16T09:28:20.701518+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:45.201503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:45.201503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:09.839306+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"73'79","reported_seq":221,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-16T09:28:24.757731+0000","last_change":"2024-09-16T09:26:35.745908+0000","last_active":"2024-09-16T09:28:24.757731+0000","last_peered":"2024-09-16T09:28:24.757731+0000","last_clean":"2024-09-16T09:28:24.757731+0000","last_became_active":"2024-09-16T09:26:35.115364+0000","last_became_peered":"2024-09-16T09:26:35.115364+0000","last_unstale":"2024-09-16T09:28:24.757731+0000","last_undegraded":"2024-09-16T09:28:24.757731+0000","last_fullsized":"2024-09-16T09:28:24.757731+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:26:35.745837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:26:35.745837+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:03.843804+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.4","version":"128'10512","reported_seq":10616,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-16T09:28:22.693564+0000","last_change":"2024-09-16T09:26:08.120333+0000","last_active":"2024-09-16T09:28:22.693564+0000","last_peered":"2024-09-16T09:28:22.693564+0000","last_clean":"2024-09-16T09:28:22.693564+0000","last_became_active":"2024-09-16T09:26:08.120051+0000","last_became_peered":"2024-09-16T09:26:08.120051+0000","last_unstale":"2024-09-16T09:28:22.693564+0000","last_undegraded":"2024-09-16T09:28:22.693564+0000","last_fullsized":"2024-09-16T09:28:22.693564+0000","mapping_epoch":82,"log_start":"103'5200","ondisk_log_start":"103'5200","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_clean_scrub_stamp":"2024-09-16T09:26:07.096764+0000","objects_scrubbed":0,"log_size":5312,"log_dups_size":0,"ondisk_log_size":5312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:39.620556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":331481088,"num_objects":5058,"num_object_clones":0,"num_object_copies":15174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5058,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10512,"num_write_kb":498240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,1],"acting":[0,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"66'13","reported_seq":190,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-16T09:28:20.698634+0000","last_change":"2024-09-16T09:27:44.156057+0000","last_active":"2024-09-16T09:28:20.698634+0000","last_peered":"2024-09-16T09:28:20.698634+0000","last_clean":"2024-09-16T09:28:20.698634+0000","last_became_active":"2024-09-16T09:25:03.713241+0000","last_became_peered":"2024-09-16T09:25:03.713241+0000","last_unstale":"2024-09-16T09:28:20.698634+0000","last_undegraded":"2024-09-16T09:28:20.698634+0000","last_fullsized":"2024-09-16T09:28:20.698634+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'13","last_scrub_stamp":"2024-09-16T09:27:44.155969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:44.155969+0000","objects_scrubbed":5,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:49.037336+0000","scrub_duration":29,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":159,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-16T09:28:20.699966+0000","last_change":"2024-09-16T09:26:26.230421+0000","last_active":"2024-09-16T09:28:20.699966+0000","last_peered":"2024-09-16T09:28:20.699966+0000","last_clean":"2024-09-16T09:28:20.699966+0000","last_became_active":"2024-09-16T09:25:01.706828+0000","last_became_peered":"2024-09-16T09:25:01.706828+0000","last_unstale":"2024-09-16T09:28:20.699966+0000","last_undegraded":"2024-09-16T09:28:20.699966+0000","last_fullsized":"2024-09-16T09:28:20.699966+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:26.230338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:26.230338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:38.139199+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":168,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-16T09:28:20.706530+0000","last_change":"2024-09-16T09:27:06.062939+0000","last_active":"2024-09-16T09:28:20.706530+0000","last_peered":"2024-09-16T09:28:20.706530+0000","last_clean":"2024-09-16T09:28:20.706530+0000","last_became_active":"2024-09-16T09:25:00.697822+0000","last_became_peered":"2024-09-16T09:25:00.697822+0000","last_unstale":"2024-09-16T09:28:20.706530+0000","last_undegraded":"2024-09-16T09:28:20.706530+0000","last_fullsized":"2024-09-16T09:28:20.706530+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:27:06.062871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:27:06.062871+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:11.909782+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0.00055022299999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":159,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-16T09:28:20.707577+0000","last_change":"2024-09-16T09:27:48.986281+0000","last_active":"2024-09-16T09:28:20.707577+0000","last_peered":"2024-09-16T09:28:20.707577+0000","last_clean":"2024-09-16T09:28:20.707577+0000","last_became_active":"2024-09-16T09:25:05.711539+0000","last_became_peered":"2024-09-16T09:25:05.711539+0000","last_unstale":"2024-09-16T09:28:20.707577+0000","last_undegraded":"2024-09-16T09:28:20.707577+0000","last_fullsized":"2024-09-16T09:28:20.707577+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:48.986202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:48.986202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:54.667068+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":148,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-16T09:28:24.758033+0000","last_change":"2024-09-16T09:27:47.705213+0000","last_active":"2024-09-16T09:28:24.758033+0000","last_peered":"2024-09-16T09:28:24.758033+0000","last_clean":"2024-09-16T09:28:24.758033+0000","last_became_active":"2024-09-16T09:25:07.723282+0000","last_became_peered":"2024-09-16T09:25:07.723282+0000","last_unstale":"2024-09-16T09:28:24.758033+0000","last_undegraded":"2024-09-16T09:28:24.758033+0000","last_fullsized":"2024-09-16T09:28:24.758033+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:47.705151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:47.705151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:08.139782+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"73'79","reported_seq":94,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-16T09:28:20.710547+0000","last_change":"2024-09-16T09:27:57.031941+0000","last_active":"2024-09-16T09:28:20.710547+0000","last_peered":"2024-09-16T09:28:20.710547+0000","last_clean":"2024-09-16T09:28:20.710547+0000","last_became_active":"2024-09-16T09:26:35.116315+0000","last_became_peered":"2024-09-16T09:26:35.116315+0000","last_unstale":"2024-09-16T09:28:20.710547+0000","last_undegraded":"2024-09-16T09:28:20.710547+0000","last_fullsized":"2024-09-16T09:28:20.710547+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:27:57.031564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:27:57.031564+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:25.604136+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"129'11241","reported_seq":11347,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-16T09:28:27.508779+0000","last_change":"2024-09-16T09:26:08.121383+0000","last_active":"2024-09-16T09:28:27.508779+0000","last_peered":"2024-09-16T09:28:27.508779+0000","last_clean":"2024-09-16T09:28:27.508779+0000","last_became_active":"2024-09-16T09:26:08.121117+0000","last_became_peered":"2024-09-16T09:26:08.121117+0000","last_unstale":"2024-09-16T09:28:27.508779+0000","last_undegraded":"2024-09-16T09:28:27.508779+0000","last_fullsized":"2024-09-16T09:28:27.508779+0000","mapping_epoch":82,"log_start":"109'6400","ondisk_log_start":"109'6400","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_clean_scrub_stamp":"2024-09-16T09:26:07.096764+0000","objects_scrubbed":0,"log_size":4841,"log_dups_size":0,"ondisk_log_size":4841,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:32.708766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":309264384,"num_objects":4719,"num_object_clones":0,"num_object_copies":14157,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4719,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11241,"num_write_kb":510720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,0],"acting":[7,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"66'12","reported_seq":165,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-16T09:28:20.708413+0000","last_change":"2024-09-16T09:25:03.715612+0000","last_active":"2024-09-16T09:28:20.708413+0000","last_peered":"2024-09-16T09:28:20.708413+0000","last_clean":"2024-09-16T09:28:20.708413+0000","last_became_active":"2024-09-16T09:25:03.715315+0000","last_became_peered":"2024-09-16T09:25:03.715315+0000","last_unstale":"2024-09-16T09:28:20.708413+0000","last_undegraded":"2024-09-16T09:28:20.708413+0000","last_fullsized":"2024-09-16T09:28:20.708413+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:02.537164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"59'1","reported_seq":168,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-16T09:28:20.707206+0000","last_change":"2024-09-16T09:27:00.666748+0000","last_active":"2024-09-16T09:28:20.707206+0000","last_peered":"2024-09-16T09:28:20.707206+0000","last_clean":"2024-09-16T09:28:20.707206+0000","last_became_active":"2024-09-16T09:25:01.701599+0000","last_became_peered":"2024-09-16T09:25:01.701599+0000","last_unstale":"2024-09-16T09:28:20.707206+0000","last_undegraded":"2024-09-16T09:28:20.707206+0000","last_fullsized":"2024-09-16T09:28:20.707206+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:27:00.666673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:27:00.666673+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":28,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"67'5","reported_seq":371,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-16T09:28:25.700280+0000","last_change":"2024-09-16T09:26:38.722150+0000","last_active":"2024-09-16T09:28:25.700280+0000","last_peered":"2024-09-16T09:28:25.700280+0000","last_clean":"2024-09-16T09:28:25.700280+0000","last_became_active":"2024-09-16T09:25:00.696546+0000","last_became_peered":"2024-09-16T09:25:00.696546+0000","last_unstale":"2024-09-16T09:28:25.700280+0000","last_undegraded":"2024-09-16T09:28:25.700280+0000","last_fullsized":"2024-09-16T09:28:25.700280+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'5","last_scrub_stamp":"2024-09-16T09:26:38.722074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:26:38.722074+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:13.894496+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0.0059770420000000001,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":200,"num_read_kb":195,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":152,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-16T09:28:24.749379+0000","last_change":"2024-09-16T09:27:07.337170+0000","last_active":"2024-09-16T09:28:24.749379+0000","last_peered":"2024-09-16T09:28:24.749379+0000","last_clean":"2024-09-16T09:28:24.749379+0000","last_became_active":"2024-09-16T09:25:05.714704+0000","last_became_peered":"2024-09-16T09:25:05.714704+0000","last_unstale":"2024-09-16T09:28:24.749379+0000","last_undegraded":"2024-09-16T09:28:24.749379+0000","last_fullsized":"2024-09-16T09:28:24.749379+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:07.337124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:07.337124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:33.974050+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":147,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-16T09:28:20.700621+0000","last_change":"2024-09-16T09:26:24.231324+0000","last_active":"2024-09-16T09:28:20.700621+0000","last_peered":"2024-09-16T09:28:20.700621+0000","last_clean":"2024-09-16T09:28:20.700621+0000","last_became_active":"2024-09-16T09:25:07.714158+0000","last_became_peered":"2024-09-16T09:25:07.714158+0000","last_unstale":"2024-09-16T09:28:20.700621+0000","last_undegraded":"2024-09-16T09:28:20.700621+0000","last_fullsized":"2024-09-16T09:28:20.700621+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:24.231167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:24.231167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:01.957543+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.6","version":"128'10683","reported_seq":10835,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-16T09:28:22.720964+0000","last_change":"2024-09-16T09:27:26.477113+0000","last_active":"2024-09-16T09:28:22.720964+0000","last_peered":"2024-09-16T09:28:22.720964+0000","last_clean":"2024-09-16T09:28:22.720964+0000","last_became_active":"2024-09-16T09:26:08.121253+0000","last_became_peered":"2024-09-16T09:26:08.121253+0000","last_unstale":"2024-09-16T09:28:22.720964+0000","last_undegraded":"2024-09-16T09:28:22.720964+0000","last_fullsized":"2024-09-16T09:28:22.720964+0000","mapping_epoch":82,"log_start":"105'5400","ondisk_log_start":"105'5400","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_clean_scrub_stamp":"2024-09-16T09:26:07.096764+0000","objects_scrubbed":4399,"log_size":5283,"log_dups_size":0,"ondisk_log_size":5283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:09.654519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":332857344,"num_objects":5079,"num_object_clones":0,"num_object_copies":15237,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5079,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10683,"num_write_kb":504384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"66'16","reported_seq":221,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-16T09:28:24.750752+0000","last_change":"2024-09-16T09:27:14.265733+0000","last_active":"2024-09-16T09:28:24.750752+0000","last_peered":"2024-09-16T09:28:24.750752+0000","last_clean":"2024-09-16T09:28:24.750752+0000","last_became_active":"2024-09-16T09:25:03.714118+0000","last_became_peered":"2024-09-16T09:25:03.714118+0000","last_unstale":"2024-09-16T09:28:24.750752+0000","last_undegraded":"2024-09-16T09:28:24.750752+0000","last_fullsized":"2024-09-16T09:28:24.750752+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'16","last_scrub_stamp":"2024-09-16T09:27:14.265648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:14.265648+0000","objects_scrubbed":8,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:42.225176+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":159,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-16T09:28:20.705768+0000","last_change":"2024-09-16T09:27:49.195418+0000","last_active":"2024-09-16T09:28:20.705768+0000","last_peered":"2024-09-16T09:28:20.705768+0000","last_clean":"2024-09-16T09:28:20.705768+0000","last_became_active":"2024-09-16T09:25:01.703730+0000","last_became_peered":"2024-09-16T09:25:01.703730+0000","last_unstale":"2024-09-16T09:28:20.705768+0000","last_undegraded":"2024-09-16T09:28:20.705768+0000","last_fullsized":"2024-09-16T09:28:20.705768+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:49.195331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:27:49.195331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:10.193163+0000","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"73'79","reported_seq":216,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-16T09:28:24.760317+0000","last_change":"2024-09-16T09:26:34.751780+0000","last_active":"2024-09-16T09:28:24.760317+0000","last_peered":"2024-09-16T09:28:24.760317+0000","last_clean":"2024-09-16T09:28:24.760317+0000","last_became_active":"2024-09-16T09:26:33.297014+0000","last_became_peered":"2024-09-16T09:26:33.297014+0000","last_unstale":"2024-09-16T09:28:24.760317+0000","last_undegraded":"2024-09-16T09:28:24.760317+0000","last_fullsized":"2024-09-16T09:28:24.760317+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:26:34.751720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:26:34.751720+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:01.840617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":152,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-16T09:28:24.758533+0000","last_change":"2024-09-16T09:26:39.699133+0000","last_active":"2024-09-16T09:28:24.758533+0000","last_peered":"2024-09-16T09:28:24.758533+0000","last_clean":"2024-09-16T09:28:24.758533+0000","last_became_active":"2024-09-16T09:25:05.714341+0000","last_became_peered":"2024-09-16T09:25:05.714341+0000","last_unstale":"2024-09-16T09:28:24.758533+0000","last_undegraded":"2024-09-16T09:28:24.758533+0000","last_fullsized":"2024-09-16T09:28:24.758533+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:39.699059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:26:39.699059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:02.876606+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":148,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-16T09:28:24.749221+0000","last_change":"2024-09-16T09:27:21.192880+0000","last_active":"2024-09-16T09:28:24.749221+0000","last_peered":"2024-09-16T09:28:24.749221+0000","last_clean":"2024-09-16T09:28:24.749221+0000","last_became_active":"2024-09-16T09:25:07.714281+0000","last_became_peered":"2024-09-16T09:25:07.714281+0000","last_unstale":"2024-09-16T09:28:24.749221+0000","last_undegraded":"2024-09-16T09:28:24.749221+0000","last_fullsized":"2024-09-16T09:28:24.749221+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:21.192816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:21.192816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:44.084676+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"129'11016","reported_seq":11123,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-16T09:28:27.501240+0000","last_change":"2024-09-16T09:26:08.122057+0000","last_active":"2024-09-16T09:28:27.501240+0000","last_peered":"2024-09-16T09:28:27.501240+0000","last_clean":"2024-09-16T09:28:27.501240+0000","last_became_active":"2024-09-16T09:26:08.121785+0000","last_became_peered":"2024-09-16T09:26:08.121785+0000","last_unstale":"2024-09-16T09:28:27.501240+0000","last_undegraded":"2024-09-16T09:28:27.501240+0000","last_fullsized":"2024-09-16T09:28:27.501240+0000","mapping_epoch":82,"log_start":"104'5300","ondisk_log_start":"104'5300","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_clean_scrub_stamp":"2024-09-16T09:26:07.096764+0000","objects_scrubbed":0,"log_size":5716,"log_dups_size":0,"ondisk_log_size":5716,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:36.339662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":302448664,"num_objects":4616,"num_object_clones":0,"num_object_copies":13848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4616,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11016,"num_write_kb":500161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.e","version":"66'11","reported_seq":187,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-16T09:28:24.739795+0000","last_change":"2024-09-16T09:26:33.316695+0000","last_active":"2024-09-16T09:28:24.739795+0000","last_peered":"2024-09-16T09:28:24.739795+0000","last_clean":"2024-09-16T09:28:24.739795+0000","last_became_active":"2024-09-16T09:26:33.287399+0000","last_became_peered":"2024-09-16T09:26:33.287399+0000","last_unstale":"2024-09-16T09:28:24.739795+0000","last_undegraded":"2024-09-16T09:28:24.739795+0000","last_fullsized":"2024-09-16T09:28:24.739795+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:03.666182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":153,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-16T09:28:24.739507+0000","last_change":"2024-09-16T09:25:01.708568+0000","last_active":"2024-09-16T09:28:24.739507+0000","last_peered":"2024-09-16T09:28:24.739507+0000","last_clean":"2024-09-16T09:28:24.739507+0000","last_became_active":"2024-09-16T09:25:01.708270+0000","last_became_peered":"2024-09-16T09:25:01.708270+0000","last_unstale":"2024-09-16T09:28:24.739507+0000","last_undegraded":"2024-09-16T09:28:24.739507+0000","last_fullsized":"2024-09-16T09:28:24.739507+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:09.662470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":152,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-16T09:28:24.749216+0000","last_change":"2024-09-16T09:26:34.171190+0000","last_active":"2024-09-16T09:28:24.749216+0000","last_peered":"2024-09-16T09:28:24.749216+0000","last_clean":"2024-09-16T09:28:24.749216+0000","last_became_active":"2024-09-16T09:25:05.711964+0000","last_became_peered":"2024-09-16T09:25:05.711964+0000","last_unstale":"2024-09-16T09:28:24.749216+0000","last_undegraded":"2024-09-16T09:28:24.749216+0000","last_fullsized":"2024-09-16T09:28:24.749216+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:34.171139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:26:34.171139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:39.195436+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":147,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-16T09:28:20.703523+0000","last_change":"2024-09-16T09:27:24.030747+0000","last_active":"2024-09-16T09:28:20.703523+0000","last_peered":"2024-09-16T09:28:20.703523+0000","last_clean":"2024-09-16T09:28:20.703523+0000","last_became_active":"2024-09-16T09:25:07.716514+0000","last_became_peered":"2024-09-16T09:25:07.716514+0000","last_unstale":"2024-09-16T09:28:20.703523+0000","last_undegraded":"2024-09-16T09:28:20.703523+0000","last_fullsized":"2024-09-16T09:28:20.703523+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:24.030677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:24.030677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:42.004108+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.8","version":"73'79","reported_seq":95,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-16T09:28:24.748499+0000","last_change":"2024-09-16T09:26:37.166480+0000","last_active":"2024-09-16T09:28:24.748499+0000","last_peered":"2024-09-16T09:28:24.748499+0000","last_clean":"2024-09-16T09:28:24.748499+0000","last_became_active":"2024-09-16T09:26:35.126217+0000","last_became_peered":"2024-09-16T09:26:35.126217+0000","last_unstale":"2024-09-16T09:28:24.748499+0000","last_undegraded":"2024-09-16T09:28:24.748499+0000","last_fullsized":"2024-09-16T09:28:24.748499+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:26:37.166434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:26:37.166434+0000","objects_scrubbed":1,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:06.932146+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262688,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.e","version":"128'10651","reported_seq":10824,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-16T09:28:22.590751+0000","last_change":"2024-09-16T09:27:16.142844+0000","last_active":"2024-09-16T09:28:22.590751+0000","last_peered":"2024-09-16T09:28:22.590751+0000","last_clean":"2024-09-16T09:28:22.590751+0000","last_became_active":"2024-09-16T09:26:08.121969+0000","last_became_peered":"2024-09-16T09:26:08.121969+0000","last_unstale":"2024-09-16T09:28:22.590751+0000","last_undegraded":"2024-09-16T09:28:22.590751+0000","last_fullsized":"2024-09-16T09:28:22.590751+0000","mapping_epoch":82,"log_start":"101'4500","ondisk_log_start":"101'4500","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"108'6031","last_scrub_stamp":"2024-09-16T09:27:16.142747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_clean_scrub_stamp":"2024-09-16T09:27:16.142747+0000","objects_scrubbed":5966,"log_size":6151,"log_dups_size":0,"ondisk_log_size":6151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:42.214405+0000","scrub_duration":2110,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330891264,"num_objects":5049,"num_object_clones":0,"num_object_copies":15147,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5049,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10651,"num_write_kb":502400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"66'17","reported_seq":185,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-16T09:28:24.739709+0000","last_change":"2024-09-16T09:27:25.615134+0000","last_active":"2024-09-16T09:28:24.739709+0000","last_peered":"2024-09-16T09:28:24.739709+0000","last_clean":"2024-09-16T09:28:24.739709+0000","last_became_active":"2024-09-16T09:25:03.713358+0000","last_became_peered":"2024-09-16T09:25:03.713358+0000","last_unstale":"2024-09-16T09:28:24.739709+0000","last_undegraded":"2024-09-16T09:28:24.739709+0000","last_fullsized":"2024-09-16T09:28:24.739709+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'17","last_scrub_stamp":"2024-09-16T09:27:25.615080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:25.615080+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:48.655184+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":161,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-16T09:28:24.750117+0000","last_change":"2024-09-16T09:26:04.104188+0000","last_active":"2024-09-16T09:28:24.750117+0000","last_peered":"2024-09-16T09:28:24.750117+0000","last_clean":"2024-09-16T09:28:24.750117+0000","last_became_active":"2024-09-16T09:25:01.703355+0000","last_became_peered":"2024-09-16T09:25:01.703355+0000","last_unstale":"2024-09-16T09:28:24.750117+0000","last_undegraded":"2024-09-16T09:28:24.750117+0000","last_fullsized":"2024-09-16T09:28:24.750117+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:04.104130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:04.104130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:39.294771+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":159,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-16T09:28:20.700996+0000","last_change":"2024-09-16T09:27:54.028056+0000","last_active":"2024-09-16T09:28:20.700996+0000","last_peered":"2024-09-16T09:28:20.700996+0000","last_clean":"2024-09-16T09:28:20.700996+0000","last_became_active":"2024-09-16T09:25:05.716900+0000","last_became_peered":"2024-09-16T09:25:05.716900+0000","last_unstale":"2024-09-16T09:28:20.700996+0000","last_undegraded":"2024-09-16T09:28:20.700996+0000","last_fullsized":"2024-09-16T09:28:20.700996+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:54.027864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:54.027864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:17.563463+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":147,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-16T09:28:20.707436+0000","last_change":"2024-09-16T09:27:17.949869+0000","last_active":"2024-09-16T09:28:20.707436+0000","last_peered":"2024-09-16T09:28:20.707436+0000","last_clean":"2024-09-16T09:28:20.707436+0000","last_became_active":"2024-09-16T09:25:07.721347+0000","last_became_peered":"2024-09-16T09:25:07.721347+0000","last_unstale":"2024-09-16T09:28:20.707436+0000","last_undegraded":"2024-09-16T09:28:20.707436+0000","last_fullsized":"2024-09-16T09:28:20.707436+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:17.949794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:17.949794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:30.022558+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.9","version":"73'79","reported_seq":221,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-16T09:28:24.757878+0000","last_change":"2024-09-16T09:26:36.771785+0000","last_active":"2024-09-16T09:28:24.757878+0000","last_peered":"2024-09-16T09:28:24.757878+0000","last_clean":"2024-09-16T09:28:24.757878+0000","last_became_active":"2024-09-16T09:26:35.126066+0000","last_became_peered":"2024-09-16T09:26:35.126066+0000","last_unstale":"2024-09-16T09:28:24.757878+0000","last_undegraded":"2024-09-16T09:28:24.757878+0000","last_fullsized":"2024-09-16T09:28:24.757878+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:26:36.771720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:26:36.771720+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:11.858317+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.f","version":"128'10836","reported_seq":10940,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-16T09:28:22.742726+0000","last_change":"2024-09-16T09:26:08.120989+0000","last_active":"2024-09-16T09:28:22.742726+0000","last_peered":"2024-09-16T09:28:22.742726+0000","last_clean":"2024-09-16T09:28:22.742726+0000","last_became_active":"2024-09-16T09:26:08.120360+0000","last_became_peered":"2024-09-16T09:26:08.120360+0000","last_unstale":"2024-09-16T09:28:22.742726+0000","last_undegraded":"2024-09-16T09:28:22.742726+0000","last_fullsized":"2024-09-16T09:28:22.742726+0000","mapping_epoch":82,"log_start":"105'5500","ondisk_log_start":"105'5500","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_clean_scrub_stamp":"2024-09-16T09:26:07.096764+0000","objects_scrubbed":0,"log_size":5336,"log_dups_size":0,"ondisk_log_size":5336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:38.592305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":333709312,"num_objects":5092,"num_object_clones":0,"num_object_copies":15276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5092,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10836,"num_write_kb":509696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"66'10","reported_seq":164,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-16T09:28:24.739760+0000","last_change":"2024-09-16T09:25:03.708144+0000","last_active":"2024-09-16T09:28:24.739760+0000","last_peered":"2024-09-16T09:28:24.739760+0000","last_clean":"2024-09-16T09:28:24.739760+0000","last_became_active":"2024-09-16T09:25:03.707723+0000","last_became_peered":"2024-09-16T09:25:03.707723+0000","last_unstale":"2024-09-16T09:28:24.739760+0000","last_undegraded":"2024-09-16T09:28:24.739760+0000","last_fullsized":"2024-09-16T09:28:24.739760+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:10.620842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":159,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-16T09:28:20.701028+0000","last_change":"2024-09-16T09:27:23.082314+0000","last_active":"2024-09-16T09:28:20.701028+0000","last_peered":"2024-09-16T09:28:20.701028+0000","last_clean":"2024-09-16T09:28:20.701028+0000","last_became_active":"2024-09-16T09:25:01.703094+0000","last_became_peered":"2024-09-16T09:25:01.703094+0000","last_unstale":"2024-09-16T09:28:20.701028+0000","last_undegraded":"2024-09-16T09:28:20.701028+0000","last_fullsized":"2024-09-16T09:28:20.701028+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:23.082107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:27:23.082107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:47.542492+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":211,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-16T09:28:20.707821+0000","last_change":"2024-09-16T09:27:59.829866+0000","last_active":"2024-09-16T09:28:20.707821+0000","last_peered":"2024-09-16T09:28:20.707821+0000","last_clean":"2024-09-16T09:28:20.707821+0000","last_became_active":"2024-09-16T09:25:05.711590+0000","last_became_peered":"2024-09-16T09:25:05.711590+0000","last_unstale":"2024-09-16T09:28:20.707821+0000","last_undegraded":"2024-09-16T09:28:20.707821+0000","last_fullsized":"2024-09-16T09:28:20.707821+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:27:59.829789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:59.829789+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:23.888787+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":157,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-16T09:28:24.739395+0000","last_change":"2024-09-16T09:27:46.596338+0000","last_active":"2024-09-16T09:28:24.739395+0000","last_peered":"2024-09-16T09:28:24.739395+0000","last_clean":"2024-09-16T09:28:24.739395+0000","last_became_active":"2024-09-16T09:25:07.718958+0000","last_became_peered":"2024-09-16T09:25:07.718958+0000","last_unstale":"2024-09-16T09:28:24.739395+0000","last_undegraded":"2024-09-16T09:28:24.739395+0000","last_fullsized":"2024-09-16T09:28:24.739395+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:46.596278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:46.596278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:00.041717+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.8","version":"128'10853","reported_seq":11027,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-16T09:28:22.588422+0000","last_change":"2024-09-16T09:27:17.928717+0000","last_active":"2024-09-16T09:28:22.588422+0000","last_peered":"2024-09-16T09:28:22.588422+0000","last_clean":"2024-09-16T09:28:22.588422+0000","last_became_active":"2024-09-16T09:26:08.118215+0000","last_became_peered":"2024-09-16T09:26:08.118215+0000","last_unstale":"2024-09-16T09:28:22.588422+0000","last_undegraded":"2024-09-16T09:28:22.588422+0000","last_fullsized":"2024-09-16T09:28:22.588422+0000","mapping_epoch":82,"log_start":"102'4700","ondisk_log_start":"102'4700","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"109'6266","last_scrub_stamp":"2024-09-16T09:27:17.927781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_clean_scrub_stamp":"2024-09-16T09:27:17.927781+0000","objects_scrubbed":6214,"log_size":6153,"log_dups_size":0,"ondisk_log_size":6153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:37.591987+0000","scrub_duration":1966,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":337838080,"num_objects":5155,"num_object_clones":0,"num_object_copies":15465,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5155,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10853,"num_write_kb":512256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"66'9","reported_seq":171,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-16T09:28:20.707648+0000","last_change":"2024-09-16T09:26:28.596878+0000","last_active":"2024-09-16T09:28:20.707648+0000","last_peered":"2024-09-16T09:28:20.707648+0000","last_clean":"2024-09-16T09:28:20.707648+0000","last_became_active":"2024-09-16T09:25:03.712230+0000","last_became_peered":"2024-09-16T09:25:03.712230+0000","last_unstale":"2024-09-16T09:28:20.707648+0000","last_undegraded":"2024-09-16T09:28:20.707648+0000","last_fullsized":"2024-09-16T09:28:20.707648+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:26:28.596805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:26:28.596805+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":160,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-16T09:28:24.749129+0000","last_change":"2024-09-16T09:26:26.061173+0000","last_active":"2024-09-16T09:28:24.749129+0000","last_peered":"2024-09-16T09:28:24.749129+0000","last_clean":"2024-09-16T09:28:24.749129+0000","last_became_active":"2024-09-16T09:25:01.702629+0000","last_became_peered":"2024-09-16T09:25:01.702629+0000","last_unstale":"2024-09-16T09:28:24.749129+0000","last_undegraded":"2024-09-16T09:28:24.749129+0000","last_fullsized":"2024-09-16T09:28:24.749129+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:26.061106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:26.061106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:02.946272+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":151,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-16T09:28:20.706465+0000","last_change":"2024-09-16T09:27:19.997700+0000","last_active":"2024-09-16T09:28:20.706465+0000","last_peered":"2024-09-16T09:28:20.706465+0000","last_clean":"2024-09-16T09:28:20.706465+0000","last_became_active":"2024-09-16T09:25:05.712573+0000","last_became_peered":"2024-09-16T09:25:05.712573+0000","last_unstale":"2024-09-16T09:28:20.706465+0000","last_undegraded":"2024-09-16T09:28:20.706465+0000","last_fullsized":"2024-09-16T09:28:20.706465+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:19.997627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:19.997627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:47.478049+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":147,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-16T09:28:20.707125+0000","last_change":"2024-09-16T09:26:35.608021+0000","last_active":"2024-09-16T09:28:20.707125+0000","last_peered":"2024-09-16T09:28:20.707125+0000","last_clean":"2024-09-16T09:28:20.707125+0000","last_became_active":"2024-09-16T09:25:07.718778+0000","last_became_peered":"2024-09-16T09:25:07.718778+0000","last_unstale":"2024-09-16T09:28:20.707125+0000","last_undegraded":"2024-09-16T09:28:20.707125+0000","last_fullsized":"2024-09-16T09:28:20.707125+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:35.607956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:35.607956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"128'10550","reported_seq":10654,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-16T09:28:22.691444+0000","last_change":"2024-09-16T09:26:08.120869+0000","last_active":"2024-09-16T09:28:22.691444+0000","last_peered":"2024-09-16T09:28:22.691444+0000","last_clean":"2024-09-16T09:28:22.691444+0000","last_became_active":"2024-09-16T09:26:08.120579+0000","last_became_peered":"2024-09-16T09:26:08.120579+0000","last_unstale":"2024-09-16T09:28:22.691444+0000","last_undegraded":"2024-09-16T09:28:22.691444+0000","last_fullsized":"2024-09-16T09:28:22.691444+0000","mapping_epoch":82,"log_start":"103'5200","ondisk_log_start":"103'5200","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_clean_scrub_stamp":"2024-09-16T09:26:07.096764+0000","objects_scrubbed":0,"log_size":5350,"log_dups_size":0,"ondisk_log_size":5350,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:30.609078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":329121792,"num_objects":5022,"num_object_clones":0,"num_object_copies":15066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5022,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10550,"num_write_kb":498304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"66'19","reported_seq":188,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-16T09:28:24.749924+0000","last_change":"2024-09-16T09:26:25.022657+0000","last_active":"2024-09-16T09:28:24.749924+0000","last_peered":"2024-09-16T09:28:24.749924+0000","last_clean":"2024-09-16T09:28:24.749924+0000","last_became_active":"2024-09-16T09:25:03.712783+0000","last_became_peered":"2024-09-16T09:25:03.712783+0000","last_unstale":"2024-09-16T09:28:24.749924+0000","last_undegraded":"2024-09-16T09:28:24.749924+0000","last_fullsized":"2024-09-16T09:28:24.749924+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-16T09:26:25.022598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:26:25.022598+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:01.454467+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"59'1","reported_seq":166,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-16T09:28:24.758256+0000","last_change":"2024-09-16T09:27:17.653505+0000","last_active":"2024-09-16T09:28:24.758256+0000","last_peered":"2024-09-16T09:28:24.758256+0000","last_clean":"2024-09-16T09:28:24.758256+0000","last_became_active":"2024-09-16T09:25:01.708069+0000","last_became_peered":"2024-09-16T09:25:01.708069+0000","last_unstale":"2024-09-16T09:28:24.758256+0000","last_undegraded":"2024-09-16T09:28:24.758256+0000","last_fullsized":"2024-09-16T09:28:24.758256+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:27:17.653425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:27:17.653425+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:38.391486+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":151,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-16T09:28:20.707758+0000","last_change":"2024-09-16T09:27:11.067059+0000","last_active":"2024-09-16T09:28:20.707758+0000","last_peered":"2024-09-16T09:28:20.707758+0000","last_clean":"2024-09-16T09:28:20.707758+0000","last_became_active":"2024-09-16T09:25:05.709264+0000","last_became_peered":"2024-09-16T09:25:05.709264+0000","last_unstale":"2024-09-16T09:28:20.707758+0000","last_undegraded":"2024-09-16T09:28:20.707758+0000","last_fullsized":"2024-09-16T09:28:20.707758+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:11.066986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:11.066986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:40.568379+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":148,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-16T09:28:24.760299+0000","last_change":"2024-09-16T09:27:25.865107+0000","last_active":"2024-09-16T09:28:24.760299+0000","last_peered":"2024-09-16T09:28:24.760299+0000","last_clean":"2024-09-16T09:28:24.760299+0000","last_became_active":"2024-09-16T09:25:07.718750+0000","last_became_peered":"2024-09-16T09:25:07.718750+0000","last_unstale":"2024-09-16T09:28:24.760299+0000","last_undegraded":"2024-09-16T09:28:24.760299+0000","last_fullsized":"2024-09-16T09:28:24.760299+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:25.865030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:25.865030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:35.295712+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.a","version":"128'10686","reported_seq":10789,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-16T09:28:22.760219+0000","last_change":"2024-09-16T09:26:08.122847+0000","last_active":"2024-09-16T09:28:22.760219+0000","last_peered":"2024-09-16T09:28:22.760219+0000","last_clean":"2024-09-16T09:28:22.760219+0000","last_became_active":"2024-09-16T09:26:08.122557+0000","last_became_peered":"2024-09-16T09:26:08.122557+0000","last_unstale":"2024-09-16T09:28:22.760219+0000","last_undegraded":"2024-09-16T09:28:22.760219+0000","last_fullsized":"2024-09-16T09:28:22.760219+0000","mapping_epoch":82,"log_start":"110'6400","ondisk_log_start":"110'6400","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_clean_scrub_stamp":"2024-09-16T09:26:07.096764+0000","objects_scrubbed":0,"log_size":4286,"log_dups_size":0,"ondisk_log_size":4286,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:34.098109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":335544320,"num_objects":5120,"num_object_clones":0,"num_object_copies":15360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5120,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10686,"num_write_kb":505792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"66'12","reported_seq":180,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-16T09:28:24.739319+0000","last_change":"2024-09-16T09:27:01.771750+0000","last_active":"2024-09-16T09:28:24.739319+0000","last_peered":"2024-09-16T09:28:24.739319+0000","last_clean":"2024-09-16T09:28:24.739319+0000","last_became_active":"2024-09-16T09:25:03.714027+0000","last_became_peered":"2024-09-16T09:25:03.714027+0000","last_unstale":"2024-09-16T09:28:24.739319+0000","last_undegraded":"2024-09-16T09:28:24.739319+0000","last_fullsized":"2024-09-16T09:28:24.739319+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-16T09:27:01.771667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:01.771667+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":152,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-16T09:28:24.759729+0000","last_change":"2024-09-16T09:25:01.707905+0000","last_active":"2024-09-16T09:28:24.759729+0000","last_peered":"2024-09-16T09:28:24.759729+0000","last_clean":"2024-09-16T09:28:24.759729+0000","last_became_active":"2024-09-16T09:25:01.707528+0000","last_became_peered":"2024-09-16T09:25:01.707528+0000","last_unstale":"2024-09-16T09:28:24.759729+0000","last_undegraded":"2024-09-16T09:28:24.759729+0000","last_fullsized":"2024-09-16T09:28:24.759729+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:31.737341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":159,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-16T09:28:20.706552+0000","last_change":"2024-09-16T09:27:49.965628+0000","last_active":"2024-09-16T09:28:20.706552+0000","last_peered":"2024-09-16T09:28:20.706552+0000","last_clean":"2024-09-16T09:28:20.706552+0000","last_became_active":"2024-09-16T09:25:05.715253+0000","last_became_peered":"2024-09-16T09:25:05.715253+0000","last_unstale":"2024-09-16T09:28:20.706552+0000","last_undegraded":"2024-09-16T09:28:20.706552+0000","last_fullsized":"2024-09-16T09:28:20.706552+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:49.965424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:49.965424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:03.758011+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":147,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-16T09:28:20.700674+0000","last_change":"2024-09-16T09:27:56.059755+0000","last_active":"2024-09-16T09:28:20.700674+0000","last_peered":"2024-09-16T09:28:20.700674+0000","last_clean":"2024-09-16T09:28:20.700674+0000","last_became_active":"2024-09-16T09:25:07.718899+0000","last_became_peered":"2024-09-16T09:25:07.718899+0000","last_unstale":"2024-09-16T09:28:20.700674+0000","last_undegraded":"2024-09-16T09:28:20.700674+0000","last_fullsized":"2024-09-16T09:28:20.700674+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:56.059687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:56.059687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:02.795352+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"128'10526","reported_seq":10701,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-16T09:28:22.585870+0000","last_change":"2024-09-16T09:27:22.420461+0000","last_active":"2024-09-16T09:28:22.585870+0000","last_peered":"2024-09-16T09:28:22.585870+0000","last_clean":"2024-09-16T09:28:22.585870+0000","last_became_active":"2024-09-16T09:26:08.117869+0000","last_became_peered":"2024-09-16T09:26:08.117869+0000","last_unstale":"2024-09-16T09:28:22.585870+0000","last_undegraded":"2024-09-16T09:28:22.585870+0000","last_fullsized":"2024-09-16T09:28:22.585870+0000","mapping_epoch":82,"log_start":"106'5600","ondisk_log_start":"106'5600","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"110'6393","last_scrub_stamp":"2024-09-16T09:27:22.420266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_clean_scrub_stamp":"2024-09-16T09:27:22.420266+0000","objects_scrubbed":6319,"log_size":4926,"log_dups_size":0,"ondisk_log_size":4926,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:47.768482+0000","scrub_duration":2145,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327942144,"num_objects":5004,"num_object_clones":0,"num_object_copies":15012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5004,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10526,"num_write_kb":496960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"66'15","reported_seq":181,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-16T09:28:24.748608+0000","last_change":"2024-09-16T09:26:33.138828+0000","last_active":"2024-09-16T09:28:24.748608+0000","last_peered":"2024-09-16T09:28:24.748608+0000","last_clean":"2024-09-16T09:28:24.748608+0000","last_became_active":"2024-09-16T09:25:03.715733+0000","last_became_peered":"2024-09-16T09:25:03.715733+0000","last_unstale":"2024-09-16T09:28:24.748608+0000","last_undegraded":"2024-09-16T09:28:24.748608+0000","last_fullsized":"2024-09-16T09:28:24.748608+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-16T09:26:33.138603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:26:33.138603+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:42.149896+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":172,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-16T09:28:24.758427+0000","last_change":"2024-09-16T09:26:26.705847+0000","last_active":"2024-09-16T09:28:24.758427+0000","last_peered":"2024-09-16T09:28:24.758427+0000","last_clean":"2024-09-16T09:28:24.758427+0000","last_became_active":"2024-09-16T09:25:01.707372+0000","last_became_peered":"2024-09-16T09:25:01.707372+0000","last_unstale":"2024-09-16T09:28:24.758427+0000","last_undegraded":"2024-09-16T09:28:24.758427+0000","last_fullsized":"2024-09-16T09:28:24.758427+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-16T09:26:26.705773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:26.705773+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:40.764628+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"66'8","reported_seq":202,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-16T09:28:24.757819+0000","last_change":"2024-09-16T09:27:51.699524+0000","last_active":"2024-09-16T09:28:24.757819+0000","last_peered":"2024-09-16T09:28:24.757819+0000","last_clean":"2024-09-16T09:28:24.757819+0000","last_became_active":"2024-09-16T09:25:05.716189+0000","last_became_peered":"2024-09-16T09:25:05.716189+0000","last_unstale":"2024-09-16T09:28:24.757819+0000","last_undegraded":"2024-09-16T09:28:24.757819+0000","last_fullsized":"2024-09-16T09:28:24.757819+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:27:51.699463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:51.699463+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:58.162844+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":148,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-16T09:28:24.748814+0000","last_change":"2024-09-16T09:27:20.219376+0000","last_active":"2024-09-16T09:28:24.748814+0000","last_peered":"2024-09-16T09:28:24.748814+0000","last_clean":"2024-09-16T09:28:24.748814+0000","last_became_active":"2024-09-16T09:25:07.717213+0000","last_became_peered":"2024-09-16T09:25:07.717213+0000","last_unstale":"2024-09-16T09:28:24.748814+0000","last_undegraded":"2024-09-16T09:28:24.748814+0000","last_fullsized":"2024-09-16T09:28:24.748814+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:20.219333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:20.219333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:35.792434+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":161,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-16T09:28:24.749769+0000","last_change":"2024-09-16T09:26:33.139405+0000","last_active":"2024-09-16T09:28:24.749769+0000","last_peered":"2024-09-16T09:28:24.749769+0000","last_clean":"2024-09-16T09:28:24.749769+0000","last_became_active":"2024-09-16T09:25:01.703202+0000","last_became_peered":"2024-09-16T09:25:01.703202+0000","last_unstale":"2024-09-16T09:28:24.749769+0000","last_undegraded":"2024-09-16T09:28:24.749769+0000","last_fullsized":"2024-09-16T09:28:24.749769+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:33.139360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:33.139360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:05.488909+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"66'6","reported_seq":164,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-16T09:28:20.711433+0000","last_change":"2024-09-16T09:26:23.458888+0000","last_active":"2024-09-16T09:28:20.711433+0000","last_peered":"2024-09-16T09:28:20.711433+0000","last_clean":"2024-09-16T09:28:20.711433+0000","last_became_active":"2024-09-16T09:25:03.713245+0000","last_became_peered":"2024-09-16T09:25:03.713245+0000","last_unstale":"2024-09-16T09:28:20.711433+0000","last_undegraded":"2024-09-16T09:28:20.711433+0000","last_fullsized":"2024-09-16T09:28:20.711433+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'6","last_scrub_stamp":"2024-09-16T09:26:23.458794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:26:23.458794+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:36.044643+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":152,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-16T09:28:24.749227+0000","last_change":"2024-09-16T09:27:55.154889+0000","last_active":"2024-09-16T09:28:24.749227+0000","last_peered":"2024-09-16T09:28:24.749227+0000","last_clean":"2024-09-16T09:28:24.749227+0000","last_became_active":"2024-09-16T09:25:05.715616+0000","last_became_peered":"2024-09-16T09:25:05.715616+0000","last_unstale":"2024-09-16T09:28:24.749227+0000","last_undegraded":"2024-09-16T09:28:24.749227+0000","last_fullsized":"2024-09-16T09:28:24.749227+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:55.154839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:55.154839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:58.175282+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":148,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-16T09:28:24.758009+0000","last_change":"2024-09-16T09:26:28.733381+0000","last_active":"2024-09-16T09:28:24.758009+0000","last_peered":"2024-09-16T09:28:24.758009+0000","last_clean":"2024-09-16T09:28:24.758009+0000","last_became_active":"2024-09-16T09:25:07.721059+0000","last_became_peered":"2024-09-16T09:25:07.721059+0000","last_unstale":"2024-09-16T09:28:24.758009+0000","last_undegraded":"2024-09-16T09:28:24.758009+0000","last_fullsized":"2024-09-16T09:28:24.758009+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:28.733324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:28.733324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:05.819123+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"66'9","reported_seq":171,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-16T09:28:20.708491+0000","last_change":"2024-09-16T09:26:39.554245+0000","last_active":"2024-09-16T09:28:20.708491+0000","last_peered":"2024-09-16T09:28:20.708491+0000","last_clean":"2024-09-16T09:28:20.708491+0000","last_became_active":"2024-09-16T09:25:03.719573+0000","last_became_peered":"2024-09-16T09:25:03.719573+0000","last_unstale":"2024-09-16T09:28:20.708491+0000","last_undegraded":"2024-09-16T09:28:20.708491+0000","last_fullsized":"2024-09-16T09:28:20.708491+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:26:39.554069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:26:39.554069+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":160,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-16T09:28:24.758092+0000","last_change":"2024-09-16T09:27:24.854485+0000","last_active":"2024-09-16T09:28:24.758092+0000","last_peered":"2024-09-16T09:28:24.758092+0000","last_clean":"2024-09-16T09:28:24.758092+0000","last_became_active":"2024-09-16T09:25:01.706491+0000","last_became_peered":"2024-09-16T09:25:01.706491+0000","last_unstale":"2024-09-16T09:28:24.758092+0000","last_undegraded":"2024-09-16T09:28:24.758092+0000","last_fullsized":"2024-09-16T09:28:24.758092+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:24.854418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:27:24.854418+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:49.764094+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":143,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-16T09:28:20.710980+0000","last_change":"2024-09-16T09:25:05.715083+0000","last_active":"2024-09-16T09:28:20.710980+0000","last_peered":"2024-09-16T09:28:20.710980+0000","last_clean":"2024-09-16T09:28:20.710980+0000","last_became_active":"2024-09-16T09:25:05.714801+0000","last_became_peered":"2024-09-16T09:25:05.714801+0000","last_unstale":"2024-09-16T09:28:20.710980+0000","last_undegraded":"2024-09-16T09:28:20.710980+0000","last_fullsized":"2024-09-16T09:28:20.710980+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:32.055938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":147,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-16T09:28:20.706637+0000","last_change":"2024-09-16T09:26:35.866582+0000","last_active":"2024-09-16T09:28:20.706637+0000","last_peered":"2024-09-16T09:28:20.706637+0000","last_clean":"2024-09-16T09:28:20.706637+0000","last_became_active":"2024-09-16T09:25:07.722050+0000","last_became_peered":"2024-09-16T09:25:07.722050+0000","last_unstale":"2024-09-16T09:28:20.706637+0000","last_undegraded":"2024-09-16T09:28:20.706637+0000","last_fullsized":"2024-09-16T09:28:20.706637+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:35.866512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:35.866512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:39.028701+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"66'9","reported_seq":172,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-16T09:28:24.749020+0000","last_change":"2024-09-16T09:27:57.144868+0000","last_active":"2024-09-16T09:28:24.749020+0000","last_peered":"2024-09-16T09:28:24.749020+0000","last_clean":"2024-09-16T09:28:24.749020+0000","last_became_active":"2024-09-16T09:25:03.717310+0000","last_became_peered":"2024-09-16T09:25:03.717310+0000","last_unstale":"2024-09-16T09:28:24.749020+0000","last_undegraded":"2024-09-16T09:28:24.749020+0000","last_fullsized":"2024-09-16T09:28:24.749020+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:27:57.144791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:57.144791+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:01.812129+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":159,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-16T09:28:20.707356+0000","last_change":"2024-09-16T09:27:12.652668+0000","last_active":"2024-09-16T09:28:20.707356+0000","last_peered":"2024-09-16T09:28:20.707356+0000","last_clean":"2024-09-16T09:28:20.707356+0000","last_became_active":"2024-09-16T09:25:01.702419+0000","last_became_peered":"2024-09-16T09:25:01.702419+0000","last_unstale":"2024-09-16T09:28:20.707356+0000","last_undegraded":"2024-09-16T09:28:20.707356+0000","last_fullsized":"2024-09-16T09:28:20.707356+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:12.652603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:27:12.652603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"66'8","reported_seq":200,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-16T09:28:20.703681+0000","last_change":"2024-09-16T09:27:05.790385+0000","last_active":"2024-09-16T09:28:20.703681+0000","last_peered":"2024-09-16T09:28:20.703681+0000","last_clean":"2024-09-16T09:28:20.703681+0000","last_became_active":"2024-09-16T09:25:05.711642+0000","last_became_peered":"2024-09-16T09:25:05.711642+0000","last_unstale":"2024-09-16T09:28:20.703681+0000","last_undegraded":"2024-09-16T09:28:20.703681+0000","last_fullsized":"2024-09-16T09:28:20.703681+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:27:05.790301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:05.790301+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:25.945999+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":157,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-16T09:28:24.740022+0000","last_change":"2024-09-16T09:28:22.600081+0000","last_active":"2024-09-16T09:28:24.740022+0000","last_peered":"2024-09-16T09:28:24.740022+0000","last_clean":"2024-09-16T09:28:24.740022+0000","last_became_active":"2024-09-16T09:25:07.718810+0000","last_became_peered":"2024-09-16T09:25:07.718810+0000","last_unstale":"2024-09-16T09:28:24.740022+0000","last_undegraded":"2024-09-16T09:28:24.740022+0000","last_fullsized":"2024-09-16T09:28:24.740022+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:22.600027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:28:22.600027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:36.033169+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"66'10","reported_seq":170,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-16T09:28:20.703968+0000","last_change":"2024-09-16T09:27:02.700022+0000","last_active":"2024-09-16T09:28:20.703968+0000","last_peered":"2024-09-16T09:28:20.703968+0000","last_clean":"2024-09-16T09:28:20.703968+0000","last_became_active":"2024-09-16T09:25:03.712387+0000","last_became_peered":"2024-09-16T09:25:03.712387+0000","last_unstale":"2024-09-16T09:28:20.703968+0000","last_undegraded":"2024-09-16T09:28:20.703968+0000","last_fullsized":"2024-09-16T09:28:20.703968+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-16T09:27:02.699947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:02.699947+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:29.333798+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":152,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-16T09:28:24.758110+0000","last_change":"2024-09-16T09:25:01.708461+0000","last_active":"2024-09-16T09:28:24.758110+0000","last_peered":"2024-09-16T09:28:24.758110+0000","last_clean":"2024-09-16T09:28:24.758110+0000","last_became_active":"2024-09-16T09:25:01.707513+0000","last_became_peered":"2024-09-16T09:25:01.707513+0000","last_unstale":"2024-09-16T09:28:24.758110+0000","last_undegraded":"2024-09-16T09:28:24.758110+0000","last_fullsized":"2024-09-16T09:28:24.758110+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:27.801162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"66'8","reported_seq":210,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-16T09:28:24.748957+0000","last_change":"2024-09-16T09:28:23.185312+0000","last_active":"2024-09-16T09:28:24.748957+0000","last_peered":"2024-09-16T09:28:24.748957+0000","last_clean":"2024-09-16T09:28:24.748957+0000","last_became_active":"2024-09-16T09:25:05.713833+0000","last_became_peered":"2024-09-16T09:25:05.713833+0000","last_unstale":"2024-09-16T09:28:24.748957+0000","last_undegraded":"2024-09-16T09:28:24.748957+0000","last_fullsized":"2024-09-16T09:28:24.748957+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:28:23.185266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:28:23.185266+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:33.284839+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":147,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-16T09:28:20.707986+0000","last_change":"2024-09-16T09:27:51.639488+0000","last_active":"2024-09-16T09:28:20.707986+0000","last_peered":"2024-09-16T09:28:20.707986+0000","last_clean":"2024-09-16T09:28:20.707986+0000","last_became_active":"2024-09-16T09:25:07.717357+0000","last_became_peered":"2024-09-16T09:25:07.717357+0000","last_unstale":"2024-09-16T09:28:20.707986+0000","last_undegraded":"2024-09-16T09:28:20.707986+0000","last_fullsized":"2024-09-16T09:28:20.707986+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:51.639428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:51.639428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:19.906242+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"66'11","reported_seq":187,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-16T09:28:24.739687+0000","last_change":"2024-09-16T09:26:33.302713+0000","last_active":"2024-09-16T09:28:24.739687+0000","last_peered":"2024-09-16T09:28:24.739687+0000","last_clean":"2024-09-16T09:28:24.739687+0000","last_became_active":"2024-09-16T09:26:33.291265+0000","last_became_peered":"2024-09-16T09:26:33.291265+0000","last_unstale":"2024-09-16T09:28:24.739687+0000","last_undegraded":"2024-09-16T09:28:24.739687+0000","last_fullsized":"2024-09-16T09:28:24.739687+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:41.657340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":161,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-16T09:28:24.739959+0000","last_change":"2024-09-16T09:26:40.610512+0000","last_active":"2024-09-16T09:28:24.739959+0000","last_peered":"2024-09-16T09:28:24.739959+0000","last_clean":"2024-09-16T09:28:24.739959+0000","last_became_active":"2024-09-16T09:25:01.705158+0000","last_became_peered":"2024-09-16T09:25:01.705158+0000","last_unstale":"2024-09-16T09:28:24.739959+0000","last_undegraded":"2024-09-16T09:28:24.739959+0000","last_fullsized":"2024-09-16T09:28:24.739959+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:40.610383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:40.610383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":159,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-16T09:28:20.702319+0000","last_change":"2024-09-16T09:27:46.034727+0000","last_active":"2024-09-16T09:28:20.702319+0000","last_peered":"2024-09-16T09:28:20.702319+0000","last_clean":"2024-09-16T09:28:20.702319+0000","last_became_active":"2024-09-16T09:25:05.712816+0000","last_became_peered":"2024-09-16T09:25:05.712816+0000","last_unstale":"2024-09-16T09:28:20.702319+0000","last_undegraded":"2024-09-16T09:28:20.702319+0000","last_fullsized":"2024-09-16T09:28:20.702319+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:46.034672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:46.034672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:49.704569+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":155,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-16T09:28:20.700102+0000","last_change":"2024-09-16T09:27:48.147819+0000","last_active":"2024-09-16T09:28:20.700102+0000","last_peered":"2024-09-16T09:28:20.700102+0000","last_clean":"2024-09-16T09:28:20.700102+0000","last_became_active":"2024-09-16T09:25:07.713109+0000","last_became_peered":"2024-09-16T09:25:07.713109+0000","last_unstale":"2024-09-16T09:28:20.700102+0000","last_undegraded":"2024-09-16T09:28:20.700102+0000","last_fullsized":"2024-09-16T09:28:20.700102+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:48.147759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:48.147759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:56.283015+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"66'9","reported_seq":171,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-16T09:28:20.702404+0000","last_change":"2024-09-16T09:26:31.216495+0000","last_active":"2024-09-16T09:28:20.702404+0000","last_peered":"2024-09-16T09:28:20.702404+0000","last_clean":"2024-09-16T09:28:20.702404+0000","last_became_active":"2024-09-16T09:25:03.705807+0000","last_became_peered":"2024-09-16T09:25:03.705807+0000","last_unstale":"2024-09-16T09:28:20.702404+0000","last_undegraded":"2024-09-16T09:28:20.702404+0000","last_fullsized":"2024-09-16T09:28:20.702404+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:26:31.216298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:26:31.216298+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:03.950024+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":160,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-16T09:28:24.758177+0000","last_change":"2024-09-16T09:26:23.796308+0000","last_active":"2024-09-16T09:28:24.758177+0000","last_peered":"2024-09-16T09:28:24.758177+0000","last_clean":"2024-09-16T09:28:24.758177+0000","last_became_active":"2024-09-16T09:25:01.706762+0000","last_became_peered":"2024-09-16T09:25:01.706762+0000","last_unstale":"2024-09-16T09:28:24.758177+0000","last_undegraded":"2024-09-16T09:28:24.758177+0000","last_fullsized":"2024-09-16T09:28:24.758177+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:23.796231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:23.796231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:38.747701+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":151,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-16T09:28:20.700770+0000","last_change":"2024-09-16T09:27:44.242118+0000","last_active":"2024-09-16T09:28:20.700770+0000","last_peered":"2024-09-16T09:28:20.700770+0000","last_clean":"2024-09-16T09:28:20.700770+0000","last_became_active":"2024-09-16T09:25:05.713791+0000","last_became_peered":"2024-09-16T09:25:05.713791+0000","last_unstale":"2024-09-16T09:28:20.700770+0000","last_undegraded":"2024-09-16T09:28:20.700770+0000","last_fullsized":"2024-09-16T09:28:20.700770+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:44.241942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:44.241942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:12.701993+0000","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":147,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-16T09:28:20.706783+0000","last_change":"2024-09-16T09:27:21.664905+0000","last_active":"2024-09-16T09:28:20.706783+0000","last_peered":"2024-09-16T09:28:20.706783+0000","last_clean":"2024-09-16T09:28:20.706783+0000","last_became_active":"2024-09-16T09:25:07.715301+0000","last_became_peered":"2024-09-16T09:25:07.715301+0000","last_unstale":"2024-09-16T09:28:20.706783+0000","last_undegraded":"2024-09-16T09:28:20.706783+0000","last_fullsized":"2024-09-16T09:28:20.706783+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:21.664836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:21.664836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:32.409182+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"66'11","reported_seq":174,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-16T09:28:20.700762+0000","last_change":"2024-09-16T09:27:21.062617+0000","last_active":"2024-09-16T09:28:20.700762+0000","last_peered":"2024-09-16T09:28:20.700762+0000","last_clean":"2024-09-16T09:28:20.700762+0000","last_became_active":"2024-09-16T09:25:03.712333+0000","last_became_peered":"2024-09-16T09:25:03.712333+0000","last_unstale":"2024-09-16T09:28:20.700762+0000","last_undegraded":"2024-09-16T09:28:20.700762+0000","last_fullsized":"2024-09-16T09:28:20.700762+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-16T09:27:21.062564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:21.062564+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:26.644417+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":168,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-16T09:28:24.749315+0000","last_change":"2024-09-16T09:27:46.175159+0000","last_active":"2024-09-16T09:28:24.749315+0000","last_peered":"2024-09-16T09:28:24.749315+0000","last_clean":"2024-09-16T09:28:24.749315+0000","last_became_active":"2024-09-16T09:25:01.700733+0000","last_became_peered":"2024-09-16T09:25:01.700733+0000","last_unstale":"2024-09-16T09:28:24.749315+0000","last_undegraded":"2024-09-16T09:28:24.749315+0000","last_fullsized":"2024-09-16T09:28:24.749315+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:46.175093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:27:46.175093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:15.134231+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":152,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-16T09:28:24.757573+0000","last_change":"2024-09-16T09:27:56.745764+0000","last_active":"2024-09-16T09:28:24.757573+0000","last_peered":"2024-09-16T09:28:24.757573+0000","last_clean":"2024-09-16T09:28:24.757573+0000","last_became_active":"2024-09-16T09:25:05.713479+0000","last_became_peered":"2024-09-16T09:25:05.713479+0000","last_unstale":"2024-09-16T09:28:24.757573+0000","last_undegraded":"2024-09-16T09:28:24.757573+0000","last_fullsized":"2024-09-16T09:28:24.757573+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:56.745720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:56.745720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:12.277156+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":147,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-16T09:28:20.700936+0000","last_change":"2024-09-16T09:26:32.417222+0000","last_active":"2024-09-16T09:28:20.700936+0000","last_peered":"2024-09-16T09:28:20.700936+0000","last_clean":"2024-09-16T09:28:20.700936+0000","last_became_active":"2024-09-16T09:25:07.715582+0000","last_became_peered":"2024-09-16T09:25:07.715582+0000","last_unstale":"2024-09-16T09:28:20.700936+0000","last_undegraded":"2024-09-16T09:28:20.700936+0000","last_fullsized":"2024-09-16T09:28:20.700936+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:32.417150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:32.417150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:06.124991+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"66'4","reported_seq":161,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-16T09:28:20.699932+0000","last_change":"2024-09-16T09:26:27.474085+0000","last_active":"2024-09-16T09:28:20.699932+0000","last_peered":"2024-09-16T09:28:20.699932+0000","last_clean":"2024-09-16T09:28:20.699932+0000","last_became_active":"2024-09-16T09:25:03.704869+0000","last_became_peered":"2024-09-16T09:25:03.704869+0000","last_unstale":"2024-09-16T09:28:20.699932+0000","last_undegraded":"2024-09-16T09:28:20.699932+0000","last_fullsized":"2024-09-16T09:28:20.699932+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'4","last_scrub_stamp":"2024-09-16T09:26:27.473740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:26:27.473740+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:08.111779+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":159,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-16T09:28:20.707933+0000","last_change":"2024-09-16T09:26:23.494773+0000","last_active":"2024-09-16T09:28:20.707933+0000","last_peered":"2024-09-16T09:28:20.707933+0000","last_clean":"2024-09-16T09:28:20.707933+0000","last_became_active":"2024-09-16T09:25:01.705686+0000","last_became_peered":"2024-09-16T09:25:01.705686+0000","last_unstale":"2024-09-16T09:28:20.707933+0000","last_undegraded":"2024-09-16T09:28:20.707933+0000","last_fullsized":"2024-09-16T09:28:20.707933+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:23.494642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:23.494642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:29.637091+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":161,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-16T09:28:24.749773+0000","last_change":"2024-09-16T09:28:21.658148+0000","last_active":"2024-09-16T09:28:24.749773+0000","last_peered":"2024-09-16T09:28:24.749773+0000","last_clean":"2024-09-16T09:28:24.749773+0000","last_became_active":"2024-09-16T09:25:05.712545+0000","last_became_peered":"2024-09-16T09:25:05.712545+0000","last_unstale":"2024-09-16T09:28:24.749773+0000","last_undegraded":"2024-09-16T09:28:24.749773+0000","last_fullsized":"2024-09-16T09:28:24.749773+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:21.658103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:28:21.658103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:48.560584+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":148,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-16T09:28:24.757831+0000","last_change":"2024-09-16T09:27:58.840101+0000","last_active":"2024-09-16T09:28:24.757831+0000","last_peered":"2024-09-16T09:28:24.757831+0000","last_clean":"2024-09-16T09:28:24.757831+0000","last_became_active":"2024-09-16T09:25:07.720117+0000","last_became_peered":"2024-09-16T09:25:07.720117+0000","last_unstale":"2024-09-16T09:28:24.757831+0000","last_undegraded":"2024-09-16T09:28:24.757831+0000","last_fullsized":"2024-09-16T09:28:24.757831+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:58.840039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:58.840039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:14.862907+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":147,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-16T09:28:20.698864+0000","last_change":"2024-09-16T09:27:52.022115+0000","last_active":"2024-09-16T09:28:20.698864+0000","last_peered":"2024-09-16T09:28:20.698864+0000","last_clean":"2024-09-16T09:28:20.698864+0000","last_became_active":"2024-09-16T09:25:07.712594+0000","last_became_peered":"2024-09-16T09:25:07.712594+0000","last_unstale":"2024-09-16T09:28:20.698864+0000","last_undegraded":"2024-09-16T09:28:20.698864+0000","last_fullsized":"2024-09-16T09:28:20.698864+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:52.021842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:52.021842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:16.470972+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":151,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-16T09:28:20.700140+0000","last_change":"2024-09-16T09:25:01.704711+0000","last_active":"2024-09-16T09:28:20.700140+0000","last_peered":"2024-09-16T09:28:20.700140+0000","last_clean":"2024-09-16T09:28:20.700140+0000","last_became_active":"2024-09-16T09:25:01.703736+0000","last_became_peered":"2024-09-16T09:25:01.703736+0000","last_unstale":"2024-09-16T09:28:20.700140+0000","last_undegraded":"2024-09-16T09:28:20.700140+0000","last_fullsized":"2024-09-16T09:28:20.700140+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:28:31.100509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"66'11","reported_seq":184,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-16T09:28:24.752671+0000","last_change":"2024-09-16T09:27:57.486459+0000","last_active":"2024-09-16T09:28:24.752671+0000","last_peered":"2024-09-16T09:28:24.752671+0000","last_clean":"2024-09-16T09:28:24.752671+0000","last_became_active":"2024-09-16T09:25:03.707769+0000","last_became_peered":"2024-09-16T09:25:03.707769+0000","last_unstale":"2024-09-16T09:28:24.752671+0000","last_undegraded":"2024-09-16T09:28:24.752671+0000","last_fullsized":"2024-09-16T09:28:24.752671+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-16T09:27:57.486381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:57.486381+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:03.415634+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":151,"reported_epoch":128,"state":"active+clean","last_fresh":"2024-09-16T09:28:20.708235+0000","last_change":"2024-09-16T09:27:57.640962+0000","last_active":"2024-09-16T09:28:20.708235+0000","last_peered":"2024-09-16T09:28:20.708235+0000","last_clean":"2024-09-16T09:28:20.708235+0000","last_became_active":"2024-09-16T09:25:05.711952+0000","last_became_peered":"2024-09-16T09:25:05.711952+0000","last_unstale":"2024-09-16T09:28:20.708235+0000","last_undegraded":"2024-09-16T09:28:20.708235+0000","last_fullsized":"2024-09-16T09:28:20.708235+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:57.640756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:57.640756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:09.172616+0000","scrub_duration":44,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":16,"stat_sum":{"num_bytes":5172035608,"num_objects":78920,"num_object_clones":0,"num_object_copies":236760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78920,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":173338,"num_write_kb":8072193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":15490691072,"data_stored":15490678856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83438,"ondisk_log_size":83438,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":393,"ondisk_log_size":393,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":200,"num_read_kb":195,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":11,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":869,"ondisk_log_size":869,"up":33,"acting":33,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266734,"num_pgs":62,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1628080,"kb_used_data":1538588,"kb_used_omap":1,"kb_used_meta":89470,"kb_avail":92117072,"statfs":{"total":95995035648,"available":94327881728,"internally_reserved":0,"allocated":1575514112,"data_stored":1575183114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":91617738},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495667,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1600600,"kb_used_data":1519432,"kb_used_omap":1,"kb_used_meta":81150,"kb_avail":92144552,"statfs":{"total":95995035648,"available":94356021248,"internally_reserved":0,"allocated":1555898368,"data_stored":1555586776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":83098054},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724602,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1291424,"kb_used_data":1215824,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":92453728,"statfs":{"total":95995035648,"available":94672617472,"internally_reserved":0,"allocated":1245003776,"data_stored":1244684981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":77396422},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953535,"num_pgs":66,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1613300,"kb_used_data":1527968,"kb_used_omap":1,"kb_used_meta":85310,"kb_avail":92131852,"statfs":{"total":95995035648,"available":94343016448,"internally_reserved":0,"allocated":1564639232,"data_stored":1564304202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":87357894},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149764,"num_pgs":70,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3083892,"kb_used_data":2944288,"kb_used_omap":1,"kb_used_meta":139454,"kb_avail":90661260,"statfs":{"total":95995035648,"available":92837130240,"internally_reserved":0,"allocated":3014950912,"data_stored":3014608639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":142801351},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411401,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2068880,"kb_used_data":1969660,"kb_used_omap":1,"kb_used_meta":99198,"kb_avail":91676272,"statfs":{"total":95995035648,"available":93876502528,"internally_reserved":0,"allocated":2016931840,"data_stored":2016624915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":101579205},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2],"upper_bound":8},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574928,"num_pgs":61,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2049092,"kb_used_data":1949872,"kb_used_omap":1,"kb_used_meta":99198,"kb_avail":91696060,"statfs":{"total":95995035648,"available":93896765440,"internally_reserved":0,"allocated":1996668928,"data_stored":1996309335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":101579205},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738454,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2721964,"kb_used_data":2592088,"kb_used_omap":1,"kb_used_meta":129726,"kb_avail":91023188,"statfs":{"total":95995035648,"available":93207744512,"internally_reserved":0,"allocated":2654298112,"data_stored":2653963427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":132839877},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,4,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2647724032,"data_stored":2647719960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1977749504,"data_stored":1977745432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2000224256,"data_stored":2000224256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2985754624,"data_stored":2985754624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1543307264,"data_stored":1543307264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1237778432,"data_stored":1237778432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1538658304,"data_stored":1538654232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1559494656,"data_stored":1559494656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T09:28:29.181 INFO:tasks.thrashosds.thrasher:Setting 3.1d to [1, 0, 7] 2024-09-16T09:28:29.182 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.1d', '1', '0', '7'] 2024-09-16T09:28:29.182 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd pg-upmap 3.1d 1 0 7 2024-09-16T09:28:29.208 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:28:29.232 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:29.233 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:29.233 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:28:29.334 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:28:29.359 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:29.360 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:29.360 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:28:29.461 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:28:29.489 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:29.489 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:29.490 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:28:29.591 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:28:29.630 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:29.631 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:29.631 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:28:29.732 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:28:29.767 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:29.768 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:29.768 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:28:29.869 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:28:29.907 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:29.908 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:29.908 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:28:29.938 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd set noscrub 2024-09-16T09:28:30.010 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:28:30.035 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:30.035 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:30.036 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:28:30.080 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:29 smithi070 ceph-mon[30388]: 3.14 scrub starts 2024-09-16T09:28:30.080 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:29 smithi070 ceph-mon[30388]: 3.14 scrub ok 2024-09-16T09:28:30.080 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:29 smithi070 ceph-mon[40691]: 3.14 scrub starts 2024-09-16T09:28:30.080 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:29 smithi070 ceph-mon[40691]: 3.14 scrub ok 2024-09-16T09:28:30.137 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:28:30.159 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:29 smithi084 ceph-mon[32629]: 3.14 scrub starts 2024-09-16T09:28:30.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:29 smithi084 ceph-mon[32629]: 3.14 scrub ok 2024-09-16T09:28:30.162 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:30.163 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:30.163 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:28:30.264 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:28:30.311 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:30.312 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:30.312 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:28:30.413 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:28:30.440 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:30.441 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:30.442 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:28:30.542 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:28:30.576 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:30.577 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:30.577 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:28:30.678 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:28:30.705 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:30.705 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:30.705 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:28:30.806 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:28:30.847 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:30.847 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:30.848 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:28:30.949 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:28:30.973 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:30.973 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:30.974 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:28:31.050 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:28:31.074 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:28:31.098 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:31.098 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:31.099 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:28:31.131 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:28:31.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:30 smithi084 ceph-mon[32629]: pgmap v148: 158 pgs: 158 active+clean; 4.6 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.27k op/s 2024-09-16T09:28:31.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:30 smithi084 ceph-mon[32629]: 3.1f scrub starts 2024-09-16T09:28:31.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:30 smithi084 ceph-mon[32629]: 3.1f scrub ok 2024-09-16T09:28:31.172 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:30 smithi070 ceph-mon[30388]: pgmap v148: 158 pgs: 158 active+clean; 4.6 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.27k op/s 2024-09-16T09:28:31.172 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:30 smithi070 ceph-mon[30388]: 3.1f scrub starts 2024-09-16T09:28:31.172 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:30 smithi070 ceph-mon[30388]: 3.1f scrub ok 2024-09-16T09:28:31.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:30 smithi070 ceph-mon[40691]: pgmap v148: 158 pgs: 158 active+clean; 4.6 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.27k op/s 2024-09-16T09:28:31.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:30 smithi070 ceph-mon[40691]: 3.1f scrub starts 2024-09-16T09:28:31.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:30 smithi070 ceph-mon[40691]: 3.1f scrub ok 2024-09-16T09:28:31.200 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:28:31.242 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:31.242 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:31.242 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:28:31.343 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:28:31.371 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:31.372 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:31.373 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:28:31.380 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:28:31.474 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:28:31.500 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:31.501 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:31.501 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:28:31.602 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:28:31.640 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:31.641 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:31.642 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:28:31.742 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:28:31.769 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:31.770 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:31.771 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:28:31.871 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:28:31.898 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:31.898 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:31.899 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:28:31.964 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T09:28:31.964 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:28:32.000 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:28:32.031 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:31 smithi070 ceph-mon[30388]: 5.1 scrub starts 2024-09-16T09:28:32.032 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:31 smithi070 ceph-mon[30388]: 5.1 scrub ok 2024-09-16T09:28:32.032 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:31 smithi070 ceph-mon[30388]: 4.2 scrub starts 2024-09-16T09:28:32.032 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:31 smithi070 ceph-mon[30388]: 4.2 scrub ok 2024-09-16T09:28:32.032 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:28:31 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:28:31] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:28:32.033 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:31 smithi070 ceph-mon[40691]: 5.1 scrub starts 2024-09-16T09:28:32.033 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:31 smithi070 ceph-mon[40691]: 5.1 scrub ok 2024-09-16T09:28:32.033 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:31 smithi070 ceph-mon[40691]: 4.2 scrub starts 2024-09-16T09:28:32.033 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:31 smithi070 ceph-mon[40691]: 4.2 scrub ok 2024-09-16T09:28:32.041 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:32.041 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:32.042 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:28:32.143 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:28:32.159 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:31 smithi084 ceph-mon[32629]: 5.1 scrub starts 2024-09-16T09:28:32.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:31 smithi084 ceph-mon[32629]: 5.1 scrub ok 2024-09-16T09:28:32.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:31 smithi084 ceph-mon[32629]: 4.2 scrub starts 2024-09-16T09:28:32.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:31 smithi084 ceph-mon[32629]: 4.2 scrub ok 2024-09-16T09:28:32.184 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:32.185 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:32.185 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:28:32.286 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:28:32.312 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:32.313 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:32.313 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:28:32.414 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:28:32.453 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:32.453 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:32.454 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:28:32.555 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:28:32.579 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:32.580 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:32.580 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:28:32.681 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:28:32.705 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:32.706 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:32.707 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:28:32.807 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:28:32.833 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:32.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:32.834 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:28:32.935 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:28:32.960 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:32.961 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:32.961 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:28:33.012 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T09:28:33.058 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:32 smithi070 ceph-mon[30388]: pgmap v149: 158 pgs: 158 active+clean; 4.6 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.12k op/s 2024-09-16T09:28:33.058 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:32 smithi070 ceph-mon[30388]: 3.1a scrub starts 2024-09-16T09:28:33.058 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:32 smithi070 ceph-mon[30388]: 3.1a scrub ok 2024-09-16T09:28:33.058 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:32 smithi070 ceph-mon[30388]: 3.18 scrub starts 2024-09-16T09:28:33.058 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:32 smithi070 ceph-mon[30388]: 3.18 scrub ok 2024-09-16T09:28:33.058 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:32 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:28:33.059 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:32 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:28:33.059 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:32 smithi070 ceph-mon[40691]: pgmap v149: 158 pgs: 158 active+clean; 4.6 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.12k op/s 2024-09-16T09:28:33.059 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:32 smithi070 ceph-mon[40691]: 3.1a scrub starts 2024-09-16T09:28:33.059 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:32 smithi070 ceph-mon[40691]: 3.1a scrub ok 2024-09-16T09:28:33.059 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:32 smithi070 ceph-mon[40691]: 3.18 scrub starts 2024-09-16T09:28:33.059 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:32 smithi070 ceph-mon[40691]: 3.18 scrub ok 2024-09-16T09:28:33.059 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:32 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:28:33.059 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:32 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:28:33.061 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:28:33.113 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:33.114 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:33.114 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:28:33.160 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:28:32 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:28:33.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:32 smithi084 ceph-mon[32629]: pgmap v149: 158 pgs: 158 active+clean; 4.6 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.12k op/s 2024-09-16T09:28:33.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:32 smithi084 ceph-mon[32629]: 3.1a scrub starts 2024-09-16T09:28:33.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:32 smithi084 ceph-mon[32629]: 3.1a scrub ok 2024-09-16T09:28:33.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:32 smithi084 ceph-mon[32629]: 3.18 scrub starts 2024-09-16T09:28:33.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:32 smithi084 ceph-mon[32629]: 3.18 scrub ok 2024-09-16T09:28:33.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:32 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:28:33.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:32 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:28:33.215 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:28:33.277 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:33.278 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:33.278 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:28:33.379 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:28:33.406 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:33.407 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:33.407 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:28:33.508 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:28:33.541 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:33.542 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:33.542 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:28:33.618 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.3 dump_ops_in_flight 2024-09-16T09:28:33.643 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:28:33.711 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:33.711 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:33.712 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:28:33.812 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:28:33.827 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:28:33.831 INFO:teuthology.orchestra.run.smithi070.stderr:set 3.1d pg_upmap mapping to [1,0,7] 2024-09-16T09:28:33.839 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:33.840 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:33.840 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:28:33.840 INFO:teuthology.orchestra.run.smithi070.stderr:noscrub is set 2024-09-16T09:28:33.941 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:28:33.995 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:33.996 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:33.996 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:28:34.097 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:28:34.098 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:33 smithi070 ceph-mon[40691]: 4.1 scrub starts 2024-09-16T09:28:34.098 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:33 smithi070 ceph-mon[40691]: 4.1 scrub ok 2024-09-16T09:28:34.098 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:33 smithi070 ceph-mon[40691]: 3.17 scrub starts 2024-09-16T09:28:34.098 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:33 smithi070 ceph-mon[40691]: 3.17 scrub ok 2024-09-16T09:28:34.098 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:33 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:28:34.098 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:33 smithi070 ceph-mon[40691]: 3.13 scrub starts 2024-09-16T09:28:34.098 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:33 smithi070 ceph-mon[40691]: 3.13 scrub ok 2024-09-16T09:28:34.098 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:33 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1599023776' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:28:34.098 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:33 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:28:34.098 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:33 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3748902411' entity='client.admin' cmd={"prefix": "osd pg-upmap", "pgid": "3.1d", "id": [1, 0, 7]} : dispatch 2024-09-16T09:28:34.099 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:33 smithi070 ceph-mon[30388]: 4.1 scrub starts 2024-09-16T09:28:34.099 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:33 smithi070 ceph-mon[30388]: 4.1 scrub ok 2024-09-16T09:28:34.099 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:33 smithi070 ceph-mon[30388]: 3.17 scrub starts 2024-09-16T09:28:34.099 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:33 smithi070 ceph-mon[30388]: 3.17 scrub ok 2024-09-16T09:28:34.099 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:33 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:28:34.099 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:33 smithi070 ceph-mon[30388]: 3.13 scrub starts 2024-09-16T09:28:34.099 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:33 smithi070 ceph-mon[30388]: 3.13 scrub ok 2024-09-16T09:28:34.099 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:33 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1599023776' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:28:34.099 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:33 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:28:34.099 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:33 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3748902411' entity='client.admin' cmd={"prefix": "osd pg-upmap", "pgid": "3.1d", "id": [1, 0, 7]} : dispatch 2024-09-16T09:28:34.125 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:34.126 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:34.126 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:28:34.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:33 smithi084 ceph-mon[32629]: 4.1 scrub starts 2024-09-16T09:28:34.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:33 smithi084 ceph-mon[32629]: 4.1 scrub ok 2024-09-16T09:28:34.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:33 smithi084 ceph-mon[32629]: 3.17 scrub starts 2024-09-16T09:28:34.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:33 smithi084 ceph-mon[32629]: 3.17 scrub ok 2024-09-16T09:28:34.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:33 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:28:34.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:33 smithi084 ceph-mon[32629]: 3.13 scrub starts 2024-09-16T09:28:34.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:33 smithi084 ceph-mon[32629]: 3.13 scrub ok 2024-09-16T09:28:34.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:33 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1599023776' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:28:34.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:33 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:28:34.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:33 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3748902411' entity='client.admin' cmd={"prefix": "osd pg-upmap", "pgid": "3.1d", "id": [1, 0, 7]} : dispatch 2024-09-16T09:28:34.227 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:28:34.256 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:34.257 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:34.257 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:28:34.358 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:28:34.388 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:34.388 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:34.388 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:28:34.489 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:28:34.515 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:34.515 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:34.516 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:28:34.616 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:28:34.644 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:34.644 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:34.645 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:28:34.746 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:28:34.800 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:34.800 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:34.801 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:28:34.902 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:28:34.930 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:34.930 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:34.931 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:28:35.031 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:28:35.070 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:35.071 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:35.071 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:28:35.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:34 smithi084 ceph-mon[32629]: 6.f scrub starts 2024-09-16T09:28:35.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:34 smithi084 ceph-mon[32629]: 6.f scrub ok 2024-09-16T09:28:35.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:34 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:28:35.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:34 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:28:35.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:34 smithi084 ceph-mon[32629]: pgmap v150: 158 pgs: 158 active+clean; 4.3 GiB data, 14 GiB used, 702 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.13k op/s 2024-09-16T09:28:35.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:34 smithi084 ceph-mon[32629]: 1.5 scrub starts 2024-09-16T09:28:35.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:34 smithi084 ceph-mon[32629]: 1.5 scrub ok 2024-09-16T09:28:35.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:34 smithi084 ceph-mon[32629]: 3.e scrub starts 2024-09-16T09:28:35.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:34 smithi084 ceph-mon[32629]: 3.e scrub ok 2024-09-16T09:28:35.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:34 smithi084 ceph-mon[32629]: 7.c scrub starts 2024-09-16T09:28:35.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:34 smithi084 ceph-mon[32629]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:28:35.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:34 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-16T09:28:35.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:34 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3748902411' entity='client.admin' cmd='[{"prefix": "osd pg-upmap", "pgid": "3.1d", "id": [1, 0, 7]}]': finished 2024-09-16T09:28:35.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:34 smithi084 ceph-mon[32629]: osdmap e130: 8 total, 8 up, 8 in 2024-09-16T09:28:35.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:34 smithi084 ceph-mon[32629]: osdmap e131: 8 total, 8 up, 8 in 2024-09-16T09:28:35.170 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:34 smithi070 ceph-mon[30388]: 6.f scrub starts 2024-09-16T09:28:35.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:34 smithi070 ceph-mon[30388]: 6.f scrub ok 2024-09-16T09:28:35.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:34 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:28:35.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:34 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:28:35.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:34 smithi070 ceph-mon[30388]: pgmap v150: 158 pgs: 158 active+clean; 4.3 GiB data, 14 GiB used, 702 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.13k op/s 2024-09-16T09:28:35.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:34 smithi070 ceph-mon[30388]: 1.5 scrub starts 2024-09-16T09:28:35.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:34 smithi070 ceph-mon[30388]: 1.5 scrub ok 2024-09-16T09:28:35.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:34 smithi070 ceph-mon[30388]: 3.e scrub starts 2024-09-16T09:28:35.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:34 smithi070 ceph-mon[30388]: 3.e scrub ok 2024-09-16T09:28:35.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:34 smithi070 ceph-mon[30388]: 7.c scrub starts 2024-09-16T09:28:35.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:34 smithi070 ceph-mon[30388]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:28:35.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:34 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-16T09:28:35.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:34 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3748902411' entity='client.admin' cmd='[{"prefix": "osd pg-upmap", "pgid": "3.1d", "id": [1, 0, 7]}]': finished 2024-09-16T09:28:35.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:34 smithi070 ceph-mon[30388]: osdmap e130: 8 total, 8 up, 8 in 2024-09-16T09:28:35.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:34 smithi070 ceph-mon[30388]: osdmap e131: 8 total, 8 up, 8 in 2024-09-16T09:28:35.172 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:28:35.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:34 smithi070 ceph-mon[40691]: 6.f scrub starts 2024-09-16T09:28:35.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:34 smithi070 ceph-mon[40691]: 6.f scrub ok 2024-09-16T09:28:35.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:34 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:28:35.173 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:34 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:28:35.173 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:34 smithi070 ceph-mon[40691]: pgmap v150: 158 pgs: 158 active+clean; 4.3 GiB data, 14 GiB used, 702 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.13k op/s 2024-09-16T09:28:35.173 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:34 smithi070 ceph-mon[40691]: 1.5 scrub starts 2024-09-16T09:28:35.173 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:34 smithi070 ceph-mon[40691]: 1.5 scrub ok 2024-09-16T09:28:35.173 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:34 smithi070 ceph-mon[40691]: 3.e scrub starts 2024-09-16T09:28:35.173 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:34 smithi070 ceph-mon[40691]: 3.e scrub ok 2024-09-16T09:28:35.173 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:34 smithi070 ceph-mon[40691]: 7.c scrub starts 2024-09-16T09:28:35.173 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:34 smithi070 ceph-mon[40691]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:28:35.173 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:34 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-16T09:28:35.173 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:34 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3748902411' entity='client.admin' cmd='[{"prefix": "osd pg-upmap", "pgid": "3.1d", "id": [1, 0, 7]}]': finished 2024-09-16T09:28:35.173 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:34 smithi070 ceph-mon[40691]: osdmap e130: 8 total, 8 up, 8 in 2024-09-16T09:28:35.173 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:34 smithi070 ceph-mon[40691]: osdmap e131: 8 total, 8 up, 8 in 2024-09-16T09:28:35.200 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:35.201 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:35.201 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:28:35.302 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:28:35.331 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:35.332 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:35.332 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:28:35.433 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:28:35.459 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:35.460 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:35.461 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:28:35.562 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:28:35.607 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:35.608 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:35.608 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:28:35.709 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:28:35.745 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:35.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:35.746 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:28:35.847 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:28:35.886 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:35.886 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:35.887 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:28:35.954 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:28:35.988 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:28:36.030 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:28:36.030 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:28:36.045 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:36.046 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:36.046 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:28:36.065 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:28:36.066 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:28:36.105 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:28:36.105 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:28:36.139 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:28:36.139 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:28:36.147 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:28:36.173 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:28:36.173 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:28:36.188 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:36.189 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:36.189 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:28:36.206 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:28:36.206 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:28:36.239 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:28:36.239 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:28:36.276 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:28:36.276 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:28:36.291 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:28:36.330 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:36.331 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:36.332 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:28:36.433 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:28:36.469 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:36.470 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:36.470 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:28:36.571 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:28:36.608 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:36.608 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:36.609 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:28:36.709 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:28:36.734 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:36.734 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:36.735 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:28:36.790 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T09:28:36.790 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [ 2024-09-16T09:28:36.790 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:36.790 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.24698.0:187267 7.2 7:41093740:::benchmark_data_smithi070_78438_object61136:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131)", 2024-09-16T09:28:36.791 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:28:36.771620+0000", 2024-09-16T09:28:36.791 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.016774463, 2024-09-16T09:28:36.791 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.016802362000000001, 2024-09-16T09:28:36.791 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:28:36.791 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:28:36.791 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "waiting for sub ops", 2024-09-16T09:28:36.791 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:28:36.791 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.24698", 2024-09-16T09:28:36.791 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:28:36.791 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 187267 2024-09-16T09:28:36.791 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:36.791 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:28:36.791 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:36.791 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:28:36.791 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:36.771620+0000", 2024-09-16T09:28:36.792 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:28:36.792 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:36.792 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:36.792 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:28:36.792 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:36.771620+0000", 2024-09-16T09:28:36.792 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:28:36.792 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:36.792 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:36.792 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:28:36.792 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:36.771621+0000", 2024-09-16T09:28:36.792 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.5999999999999991e-07 2024-09-16T09:28:36.792 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:36.792 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:36.792 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:28:36.793 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:36.771626+0000", 2024-09-16T09:28:36.793 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.9269999999999996e-06 2024-09-16T09:28:36.793 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:36.793 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:36.793 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:28:36.793 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:36.771638+0000", 2024-09-16T09:28:36.793 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1695999999999999e-05 2024-09-16T09:28:36.793 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:36.793 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:36.793 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:28:36.793 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:36.771645+0000", 2024-09-16T09:28:36.793 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.4529999999999996e-06 2024-09-16T09:28:36.793 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:36.794 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:36.794 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:28:36.794 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:36.774800+0000", 2024-09-16T09:28:36.794 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0031549210000000002 2024-09-16T09:28:36.794 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:36.794 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:36.794 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:28:36.794 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:36.774898+0000", 2024-09-16T09:28:36.794 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.7560999999999996e-05 2024-09-16T09:28:36.794 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:36.794 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:36.794 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,2", 2024-09-16T09:28:36.794 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:36.774996+0000", 2024-09-16T09:28:36.794 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.7992999999999995e-05 2024-09-16T09:28:36.795 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:36.795 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:36.795 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:28:36.795 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:36.782764+0000", 2024-09-16T09:28:36.795 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0077686270000000002 2024-09-16T09:28:36.795 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:36.795 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:36.795 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:28:36.795 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:36.787833+0000", 2024-09-16T09:28:36.795 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0050686050000000003 2024-09-16T09:28:36.795 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:36.795 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:28:36.795 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:36.796 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:36.796 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:36.796 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.24698.0:187270 7.2 7:441268b7:::benchmark_data_smithi070_78438_object61139:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131)", 2024-09-16T09:28:36.796 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:28:36.773920+0000", 2024-09-16T09:28:36.796 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.014474582999999999, 2024-09-16T09:28:36.796 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.014582078, 2024-09-16T09:28:36.796 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:28:36.796 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:28:36.797 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "waiting for sub ops", 2024-09-16T09:28:36.797 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:28:36.797 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.24698", 2024-09-16T09:28:36.797 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:28:36.797 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 187270 2024-09-16T09:28:36.797 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:36.797 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:28:36.797 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:36.797 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:28:36.797 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:36.773920+0000", 2024-09-16T09:28:36.797 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:28:36.797 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:36.797 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:36.798 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:28:36.798 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:36.773920+0000", 2024-09-16T09:28:36.798 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:28:36.798 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:36.798 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:36.798 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:28:36.798 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:36.773921+0000", 2024-09-16T09:28:36.798 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2109999999999999e-06 2024-09-16T09:28:36.798 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:36.798 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:36.798 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:28:36.798 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:36.773926+0000", 2024-09-16T09:28:36.798 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.7319999999999998e-06 2024-09-16T09:28:36.798 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:36.798 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:36.798 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:28:36.799 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:36.773941+0000", 2024-09-16T09:28:36.799 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5508000000000001e-05 2024-09-16T09:28:36.799 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:36.799 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:36.799 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:28:36.799 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:36.773949+0000", 2024-09-16T09:28:36.799 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.391e-06 2024-09-16T09:28:36.799 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:36.799 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:36.799 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:28:36.799 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:36.777288+0000", 2024-09-16T09:28:36.799 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0033393350000000001 2024-09-16T09:28:36.799 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:36.799 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:36.800 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:28:36.800 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:36.777408+0000", 2024-09-16T09:28:36.800 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00011967400000000001 2024-09-16T09:28:36.800 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:36.800 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:36.800 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,2", 2024-09-16T09:28:36.800 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:36.777514+0000", 2024-09-16T09:28:36.800 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000106282 2024-09-16T09:28:36.800 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:36.800 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:36.800 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:28:36.800 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:36.782769+0000", 2024-09-16T09:28:36.800 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0052551840000000004 2024-09-16T09:28:36.800 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:36.801 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:36.801 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:28:36.801 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:36.787895+0000", 2024-09-16T09:28:36.801 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0051260510000000004 2024-09-16T09:28:36.801 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:36.801 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:28:36.801 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:36.801 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:36.801 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:36.801 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.24698.0:187271 7.f e131/82 7:ff3efa8a:::benchmark_data_smithi070_78438_object61140:head v 131'11768, mlcod=131'11764)", 2024-09-16T09:28:36.801 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:28:36.782886+0000", 2024-09-16T09:28:36.801 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.0055084490000000003, 2024-09-16T09:28:36.801 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0056860180000000001, 2024-09-16T09:28:36.801 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:28:36.802 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:28:36.802 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T09:28:36.802 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:28:36.802 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:36.802 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:28:36.802 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:36.782886+0000", 2024-09-16T09:28:36.802 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:28:36.802 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:36.802 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:36.802 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:28:36.802 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:36.782886+0000", 2024-09-16T09:28:36.802 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:28:36.802 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:36.802 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:36.803 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:28:36.803 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:36.782887+0000", 2024-09-16T09:28:36.803 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.1900000000000002e-07 2024-09-16T09:28:36.803 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:36.803 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:36.803 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:28:36.803 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:36.782893+0000", 2024-09-16T09:28:36.803 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.9120000000000003e-06 2024-09-16T09:28:36.803 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:36.803 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:36.803 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:28:36.803 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:36.782903+0000", 2024-09-16T09:28:36.803 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.026e-05 2024-09-16T09:28:36.803 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:36.803 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:36.804 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:28:36.804 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:36.782910+0000", 2024-09-16T09:28:36.804 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.6409999999999996e-06 2024-09-16T09:28:36.804 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:36.804 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:36.804 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:28:36.804 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:36.783297+0000", 2024-09-16T09:28:36.804 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00038743000000000002 2024-09-16T09:28:36.804 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:36.804 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:36.804 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:28:36.805 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:36.783314+0000", 2024-09-16T09:28:36.805 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7178000000000001e-05 2024-09-16T09:28:36.805 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:36.805 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:28:36.805 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:36.805 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:36.805 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:36.805 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.24698.0:187272 7.c e131/82 7:35c46ffd:::benchmark_data_smithi070_78438_object61141:head v 131'11570, mlcod=131'11567)", 2024-09-16T09:28:36.805 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:28:36.784219+0000", 2024-09-16T09:28:36.805 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.0041752869999999997, 2024-09-16T09:28:36.806 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0043999920000000001, 2024-09-16T09:28:36.806 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:28:36.806 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:28:36.806 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T09:28:36.806 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:28:36.806 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:36.806 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:28:36.806 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:36.784219+0000", 2024-09-16T09:28:36.806 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:28:36.806 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:36.806 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:36.806 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:28:36.806 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:36.784219+0000", 2024-09-16T09:28:36.806 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:28:36.806 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:36.806 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:36.807 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:28:36.807 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:36.784220+0000", 2024-09-16T09:28:36.807 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.37e-07 2024-09-16T09:28:36.807 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:36.807 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:36.807 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:28:36.807 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:36.784225+0000", 2024-09-16T09:28:36.807 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.2229999999999998e-06 2024-09-16T09:28:36.807 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:36.807 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:36.807 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:28:36.807 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:36.784236+0000", 2024-09-16T09:28:36.807 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0747000000000001e-05 2024-09-16T09:28:36.807 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:36.807 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:36.808 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:28:36.808 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:36.784242+0000", 2024-09-16T09:28:36.808 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.3589999999999996e-06 2024-09-16T09:28:36.808 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:36.808 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:36.808 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:28:36.808 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:36.784555+0000", 2024-09-16T09:28:36.808 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00031265500000000001 2024-09-16T09:28:36.808 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:36.808 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:36.808 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:28:36.808 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:36.784643+0000", 2024-09-16T09:28:36.808 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.7913999999999999e-05 2024-09-16T09:28:36.808 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:36.808 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:28:36.809 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:36.809 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:36.809 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:36.809 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.24698.0:187273 7.6 e131/82 7:6d43b16b:::benchmark_data_smithi070_78438_object61142:head v 131'11668, mlcod=131'11666)", 2024-09-16T09:28:36.809 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:28:36.787961+0000", 2024-09-16T09:28:36.809 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.00043369300000000002, 2024-09-16T09:28:36.809 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00070309500000000004, 2024-09-16T09:28:36.809 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:28:36.809 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:28:36.809 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T09:28:36.809 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:28:36.809 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:36.809 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:28:36.809 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:36.787961+0000", 2024-09-16T09:28:36.810 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:28:36.810 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:36.810 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:36.810 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:28:36.810 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:36.787961+0000", 2024-09-16T09:28:36.810 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:28:36.810 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:36.810 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:36.810 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:28:36.810 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:36.787962+0000", 2024-09-16T09:28:36.810 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.3799999999999996e-07 2024-09-16T09:28:36.810 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:36.810 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:36.811 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:28:36.811 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:36.788008+0000", 2024-09-16T09:28:36.811 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.6258000000000003e-05 2024-09-16T09:28:36.811 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:36.811 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:36.811 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:28:36.811 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:36.788034+0000", 2024-09-16T09:28:36.811 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.6339999999999999e-05 2024-09-16T09:28:36.811 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:36.811 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:36.811 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:28:36.811 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:36.788046+0000", 2024-09-16T09:28:36.811 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2092e-05 2024-09-16T09:28:36.811 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:36.812 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:36.812 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:28:36.812 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:36.788156+0000", 2024-09-16T09:28:36.812 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00010966 2024-09-16T09:28:36.812 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:36.812 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:36.812 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:28:36.812 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:36.788174+0000", 2024-09-16T09:28:36.812 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7966999999999999e-05 2024-09-16T09:28:36.812 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:36.812 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:28:36.812 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:36.812 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:36.812 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:36.813 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.24698.0:187275 7.2 7.a3f78422 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e131)", 2024-09-16T09:28:36.813 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:28:36.788135+0000", 2024-09-16T09:28:36.813 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.00025960099999999998, 2024-09-16T09:28:36.813 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00057662200000000005, 2024-09-16T09:28:36.813 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:28:36.813 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:28:36.813 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T09:28:36.813 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:28:36.813 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.24698", 2024-09-16T09:28:36.813 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:28:36.813 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 187275 2024-09-16T09:28:36.813 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:36.813 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:28:36.813 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:36.814 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:28:36.814 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:36.788135+0000", 2024-09-16T09:28:36.814 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:28:36.814 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:36.814 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:36.814 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:28:36.814 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:36.788135+0000", 2024-09-16T09:28:36.814 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:28:36.814 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:36.814 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:36.814 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:28:36.814 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:36.788137+0000", 2024-09-16T09:28:36.814 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.669e-06 2024-09-16T09:28:36.814 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:36.815 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:36.815 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:28:36.815 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:36.788142+0000", 2024-09-16T09:28:36.815 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.6389999999999997e-06 2024-09-16T09:28:36.815 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:36.815 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:36.815 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:28:36.815 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:36.788155+0000", 2024-09-16T09:28:36.815 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2978e-05 2024-09-16T09:28:36.815 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:36.815 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:36.815 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:28:36.815 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:36.788162+0000", 2024-09-16T09:28:36.815 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.2640000000000002e-06 2024-09-16T09:28:36.815 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:36.816 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:28:36.816 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:36.816 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:36.816 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:36.816 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.24698.0:187265 7.2 7:4d256e68:::benchmark_data_smithi070_78438_object61134:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131)", 2024-09-16T09:28:36.816 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:28:36.771099+0000", 2024-09-16T09:28:36.816 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.017295268999999999, 2024-09-16T09:28:36.816 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.017660319000000001, 2024-09-16T09:28:36.816 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:28:36.816 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:28:36.816 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "waiting for sub ops", 2024-09-16T09:28:36.816 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:28:36.816 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.24698", 2024-09-16T09:28:36.816 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:28:36.816 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 187265 2024-09-16T09:28:36.817 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:36.817 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:28:36.817 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:36.817 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:28:36.817 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:36.771099+0000", 2024-09-16T09:28:36.817 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:28:36.817 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:36.817 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:36.817 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:28:36.817 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:36.771099+0000", 2024-09-16T09:28:36.817 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:28:36.817 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:36.817 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:36.817 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:28:36.817 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:36.771101+0000", 2024-09-16T09:28:36.817 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7269999999999999e-06 2024-09-16T09:28:36.818 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:36.818 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:36.818 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:28:36.818 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:36.771107+0000", 2024-09-16T09:28:36.818 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.3219999999999998e-06 2024-09-16T09:28:36.818 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:36.818 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:36.818 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:28:36.818 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:36.771121+0000", 2024-09-16T09:28:36.818 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3594000000000001e-05 2024-09-16T09:28:36.818 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:36.818 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:36.818 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:28:36.818 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:36.771129+0000", 2024-09-16T09:28:36.818 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.9279999999999993e-06 2024-09-16T09:28:36.818 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:36.819 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:36.819 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:28:36.819 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:36.771435+0000", 2024-09-16T09:28:36.819 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000305758 2024-09-16T09:28:36.819 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:36.819 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:36.819 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:28:36.819 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:36.771558+0000", 2024-09-16T09:28:36.819 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00012310899999999999 2024-09-16T09:28:36.819 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:36.819 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:36.819 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,2", 2024-09-16T09:28:36.819 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:36.771678+0000", 2024-09-16T09:28:36.819 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000120573 2024-09-16T09:28:36.819 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:36.819 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:36.820 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:28:36.820 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:36.782746+0000", 2024-09-16T09:28:36.820 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.011067832999999999 2024-09-16T09:28:36.820 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:28:36.820 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:28:36.820 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:28:36.820 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:36.782759+0000", 2024-09-16T09:28:36.820 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2761e-05 2024-09-16T09:28:36.820 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:36.820 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:28:36.820 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:36.820 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:28:36.820 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-16T09:28:36.820 INFO:teuthology.orchestra.run.smithi070.stdout: "num_ops": 7 2024-09-16T09:28:36.821 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T09:28:36.836 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:28:36.859 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:36.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:36.861 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:28:36.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:36 smithi070 ceph-mon[30388]: pgmap v153: 158 pgs: 158 active+clean; 4.3 GiB data, 14 GiB used, 702 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.16k op/s 2024-09-16T09:28:36.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:36 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1387170635' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:28:36.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:36 smithi070 ceph-mon[40691]: pgmap v153: 158 pgs: 158 active+clean; 4.3 GiB data, 14 GiB used, 702 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.16k op/s 2024-09-16T09:28:36.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:36 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1387170635' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:28:36.961 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:28:36.988 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:36.989 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:36.989 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:28:37.090 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:28:37.116 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:37.117 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:37.117 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:28:37.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:36 smithi084 ceph-mon[32629]: pgmap v153: 158 pgs: 158 active+clean; 4.3 GiB data, 14 GiB used, 702 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.16k op/s 2024-09-16T09:28:37.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:36 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1387170635' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:28:37.218 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:28:37.255 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:37.255 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:37.256 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:28:37.357 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:28:37.390 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-16T09:28:37.391 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:37.392 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:37.392 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:28:37.493 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:28:37.517 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:37.517 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:37.518 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:28:37.618 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:28:37.658 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:37.659 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:37.659 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:28:37.760 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:28:37.787 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:37.788 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:37.788 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:28:37.889 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:28:37.939 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.3 dump_blocked_ops 2024-09-16T09:28:37.943 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:37.943 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:37.944 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:28:38.044 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:28:38.069 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:38.070 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:38.070 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:28:38.171 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:28:38.231 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:38.232 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:38.232 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:28:38.333 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:28:38.383 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:38.383 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:38.384 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:28:38.484 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:28:38.507 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:38.507 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:38.508 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:28:38.514 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:28:38.518 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:28:38.608 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:28:38.634 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:38.635 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:38.635 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:28:38.737 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:28:38.839 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:38.840 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:38.840 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:28:38.941 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:28:38.982 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:38.983 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:38.984 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:28:39.064 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:38 smithi070 ceph-mon[30388]: pgmap v154: 158 pgs: 158 active+clean; 4.1 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 989 op/s 2024-09-16T09:28:39.064 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:38 smithi070 ceph-mon[40691]: pgmap v154: 158 pgs: 158 active+clean; 4.1 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 989 op/s 2024-09-16T09:28:39.085 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:28:39.128 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:39.129 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:39.129 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:28:39.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:38 smithi084 ceph-mon[32629]: pgmap v154: 158 pgs: 158 active+clean; 4.1 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 989 op/s 2024-09-16T09:28:39.230 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:28:39.254 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:39.255 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:39.255 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:28:39.356 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:28:39.381 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:39.382 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:39.382 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:28:39.483 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:28:39.522 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:39.522 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:39.523 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:28:39.576 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T09:28:39.576 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T09:28:39.576 INFO:teuthology.orchestra.run.smithi070.stdout: "complaint_time": 30, 2024-09-16T09:28:39.576 INFO:teuthology.orchestra.run.smithi070.stdout: "num_blocked_ops": 0 2024-09-16T09:28:39.576 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T09:28:39.624 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:28:39.670 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:39.671 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:39.671 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:28:39.772 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:28:39.809 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T09:28:39.809 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T09:28:39.809 INFO:tasks.thrashosds.thrasher:inject_pause on osd.6 2024-09-16T09:28:39.809 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-16T09:28:39.809 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-16T09:28:39.810 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.6 config set filestore_inject_stall 3 2024-09-16T09:28:39.832 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:39.833 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:39.833 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:28:39.869 INFO:teuthology.orchestra.run.smithi070.stderr:nodeep-scrub is set 2024-09-16T09:28:39.934 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:28:39.964 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:39.964 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:39.965 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:28:40.066 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:28:40.115 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:40.116 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:40.116 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:28:40.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:39 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:28:40.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:39 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/2484677867' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:28:40.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:39 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:28:40.170 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:39 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:28:40.170 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:39 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2484677867' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:28:40.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:39 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:28:40.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:39 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:28:40.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:39 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/2484677867' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:28:40.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:39 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:28:40.217 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:28:40.258 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:40.258 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:40.259 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:28:40.337 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.3 dump_historic_ops 2024-09-16T09:28:40.359 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:28:40.423 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:40.424 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:40.424 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:28:40.471 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T09:28:40.509 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:28:40.525 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:28:40.594 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:40.595 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:40.596 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:28:40.697 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:28:40.726 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:40.726 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:40.727 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:28:40.827 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:28:40.868 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:40.868 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:40.869 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:28:40.970 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:28:41.011 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:41.012 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:41.012 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:28:41.113 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:28:41.138 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:41.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:41.139 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:28:41.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:40 smithi084 ceph-mon[32629]: pgmap v155: 158 pgs: 158 active+clean; 3.9 GiB data, 13 GiB used, 703 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.43k op/s 2024-09-16T09:28:41.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:40 smithi084 ceph-mon[32629]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:28:41.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:40 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-16T09:28:41.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:40 smithi084 ceph-mon[32629]: osdmap e132: 8 total, 8 up, 8 in 2024-09-16T09:28:41.170 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:40 smithi070 ceph-mon[30388]: pgmap v155: 158 pgs: 158 active+clean; 3.9 GiB data, 13 GiB used, 703 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.43k op/s 2024-09-16T09:28:41.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:40 smithi070 ceph-mon[30388]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:28:41.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:40 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-16T09:28:41.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:40 smithi070 ceph-mon[30388]: osdmap e132: 8 total, 8 up, 8 in 2024-09-16T09:28:41.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:40 smithi070 ceph-mon[40691]: pgmap v155: 158 pgs: 158 active+clean; 3.9 GiB data, 13 GiB used, 703 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.43k op/s 2024-09-16T09:28:41.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:40 smithi070 ceph-mon[40691]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:28:41.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:40 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-16T09:28:41.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:40 smithi070 ceph-mon[40691]: osdmap e132: 8 total, 8 up, 8 in 2024-09-16T09:28:41.175 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:28:41.177 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:28:41.240 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:28:41.264 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:41.265 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:41.265 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:28:41.366 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:28:41.416 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:41.417 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:41.417 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:28:41.497 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-09-16T09:28:41.497 INFO:teuthology.orchestra.run.smithi084.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-16T09:28:41.497 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-09-16T09:28:41.518 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:28:41.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:41.569 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:41.569 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:28:41.670 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:28:41.713 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:41.714 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:41.715 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:28:41.815 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:28:41.853 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:41.854 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:41.855 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:28:41.955 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:28:41.965 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:41 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3966622739' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:28:41.965 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:28:41 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:28:41] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:28:41.967 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:41 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3966622739' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:28:42.007 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:42.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:42.008 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:28:42.070 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:28:42.070 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:28:42.070 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:28:42.070 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:28:42.070 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:28:42.070 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:28:42.071 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:28:42.071 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:28:42.071 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:28:42.071 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:28:42.082 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:28:42.083 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:28:42.109 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:28:42.114 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:28:42.114 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:28:42.136 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:42.136 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:42.137 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:28:42.150 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:28:42.150 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:28:42.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:41 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3966622739' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:28:42.238 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:28:42.264 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:42.265 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:42.266 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:28:42.362 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T09:28:42.362 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:28:42.367 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:28:42.415 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:42.416 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:42.416 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:28:42.516 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:28:42.558 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:42.559 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:42.559 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:28:42.660 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:28:42.694 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:42.694 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:42.694 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:28:42.795 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:28:42.821 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:42.821 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:42.822 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:28:42.884 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:28:42 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:28:42.924 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:28:42.959 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:42.960 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:42.960 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:28:43.061 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:28:43.086 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:43.086 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:43.087 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:28:43.109 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd unset noscrub 2024-09-16T09:28:43.159 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:42 smithi084 ceph-mon[32629]: pgmap v157: 158 pgs: 158 active+clean; 3.9 GiB data, 13 GiB used, 703 GiB / 715 GiB avail; 662 B/s rd, 0 B/s wr, 737 op/s 2024-09-16T09:28:43.170 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:42 smithi070 ceph-mon[30388]: pgmap v157: 158 pgs: 158 active+clean; 3.9 GiB data, 13 GiB used, 703 GiB / 715 GiB avail; 662 B/s rd, 0 B/s wr, 737 op/s 2024-09-16T09:28:43.170 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:42 smithi070 ceph-mon[40691]: pgmap v157: 158 pgs: 158 active+clean; 3.9 GiB data, 13 GiB used, 703 GiB / 715 GiB avail; 662 B/s rd, 0 B/s wr, 737 op/s 2024-09-16T09:28:43.188 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:28:43.215 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:43.215 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:43.216 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:28:43.317 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:28:43.356 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:43.357 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:43.357 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:28:43.458 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:28:43.503 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:43.504 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:43.504 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:28:43.605 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:28:43.641 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:43.642 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:43.642 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:28:43.743 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:28:43.783 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:43.784 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:43.784 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:28:43.885 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:28:43.892 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:43 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:28:43.918 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:43.919 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:43.919 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:28:44.020 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:28:44.048 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:44.049 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:44.049 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:28:44.150 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:28:44.170 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:43 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:28:44.170 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:43 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:28:44.182 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:44.183 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:44.183 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:28:44.284 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:28:44.330 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:44.331 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:44.331 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:28:44.432 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:28:44.470 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:44.470 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:44.471 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:28:44.571 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:28:44.611 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:44.612 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:44.612 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:28:44.713 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:28:44.763 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:44.764 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:44.765 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:28:44.865 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:28:44.909 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:44.910 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:44.910 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:28:45.011 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:28:45.046 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:45.046 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:45.047 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:28:45.148 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:28:45.151 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:44 smithi070 ceph-mon[30388]: pgmap v158: 158 pgs: 158 active+clean; 3.6 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.24k op/s 2024-09-16T09:28:45.151 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:44 smithi070 ceph-mon[40691]: pgmap v158: 158 pgs: 158 active+clean; 3.6 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.24k op/s 2024-09-16T09:28:45.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:44 smithi084 ceph-mon[32629]: pgmap v158: 158 pgs: 158 active+clean; 3.6 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.24k op/s 2024-09-16T09:28:45.211 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.4 dump_ops_in_flight 2024-09-16T09:28:45.218 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:45.219 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:45.220 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:28:45.223 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:28:45.321 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:28:45.355 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:45.356 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:45.356 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:28:45.457 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:28:45.489 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:45.489 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:45.490 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:28:45.591 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:28:45.625 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:45.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:45.626 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:28:45.727 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:28:45.753 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:45.753 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:45.754 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:28:45.855 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:28:45.895 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:45.896 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:45.896 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:28:45.962 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:28:45.997 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:28:46.033 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:46.034 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:46.034 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:28:46.135 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:28:46.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:45 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:28:46.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:45 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:28:46.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:45 smithi084 ceph-mon[32629]: osdmap e133: 8 total, 8 up, 8 in 2024-09-16T09:28:46.164 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:46.164 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:46.164 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:28:46.170 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:45 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:28:46.170 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:45 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:28:46.170 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:45 smithi070 ceph-mon[30388]: osdmap e133: 8 total, 8 up, 8 in 2024-09-16T09:28:46.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:45 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:28:46.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:45 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:28:46.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:45 smithi070 ceph-mon[40691]: osdmap e133: 8 total, 8 up, 8 in 2024-09-16T09:28:46.265 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:28:46.305 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:46.306 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T09:28:46.306 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:46.307 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:28:46.408 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:28:46.436 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:46.436 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:46.437 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:28:46.538 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:28:46.598 INFO:teuthology.orchestra.run.smithi084.stdout:ERROR: (22) Invalid argument 2024-09-16T09:28:46.598 INFO:teuthology.orchestra.run.smithi084.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:28:46.603 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:46.603 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:46.603 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:28:46.704 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:28:46.733 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:46.734 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:46.734 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:28:46.824 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:28:46.835 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:28:46.894 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:46.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:46.896 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:28:46.931 INFO:teuthology.orchestra.run.smithi070.stderr:noscrub is unset 2024-09-16T09:28:46.997 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:28:47.039 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:47.040 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:47.040 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:28:47.141 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:28:47.174 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:46 smithi070 ceph-mon[30388]: pgmap v160: 158 pgs: 158 active+clean; 3.6 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.08k op/s 2024-09-16T09:28:47.174 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:46 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1614861502' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:28:47.175 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:46 smithi070 ceph-mon[40691]: pgmap v160: 158 pgs: 158 active+clean; 3.6 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.08k op/s 2024-09-16T09:28:47.175 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:46 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1614861502' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:28:47.183 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:47.184 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:47.184 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:28:47.285 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:28:47.320 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:47.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:47.322 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:28:47.348 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:46 smithi084 ceph-mon[32629]: pgmap v160: 158 pgs: 158 active+clean; 3.6 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.08k op/s 2024-09-16T09:28:47.348 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:46 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1614861502' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:28:47.390 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.4 dump_blocked_ops 2024-09-16T09:28:47.422 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:28:47.466 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:47.467 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:47.467 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:28:47.568 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:28:47.603 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:47.604 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:47.604 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:28:47.705 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:28:47.729 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:47.729 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:47.730 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:28:47.831 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:28:47.866 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:47.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:47.867 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:28:47.914 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:28:47.915 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:28:47.915 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:28:47.915 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:28:47.915 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:28:47.915 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:28:47.915 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:28:47.915 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:28:47.915 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:28:47.915 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:28:47.915 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:28:47.915 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:28:47.916 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:28:47.916 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:28:47.916 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:28:47.916 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:28:47.937 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T09:28:47.938 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T09:28:47.938 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-16T09:28:47.938 INFO:tasks.thrashosds.thrasher:Setting osd 3 primary_affinity to 0.476536 2024-09-16T09:28:47.938 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd primary-affinity 3 0.4765362689388495 2024-09-16T09:28:47.968 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:28:48.018 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:48.020 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:48.021 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:28:48.122 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:28:48.148 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:48.148 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:48.149 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:28:48.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:47 smithi084 ceph-mon[32629]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:28:48.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:47 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1614861502' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-16T09:28:48.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:47 smithi084 ceph-mon[32629]: osdmap e134: 8 total, 8 up, 8 in 2024-09-16T09:28:48.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:47 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:28:48.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:47 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:28:48.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:47 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/2898328122' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:28:48.170 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:47 smithi070 ceph-mon[30388]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:28:48.170 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:47 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1614861502' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-16T09:28:48.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:47 smithi070 ceph-mon[30388]: osdmap e134: 8 total, 8 up, 8 in 2024-09-16T09:28:48.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:47 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:28:48.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:47 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:28:48.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:47 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2898328122' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:28:48.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:47 smithi070 ceph-mon[40691]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:28:48.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:47 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1614861502' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-16T09:28:48.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:47 smithi070 ceph-mon[40691]: osdmap e134: 8 total, 8 up, 8 in 2024-09-16T09:28:48.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:47 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:28:48.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:47 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:28:48.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:47 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/2898328122' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:28:48.250 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:28:48.288 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:48.289 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:48.289 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:28:48.390 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:28:48.418 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:48.419 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:48.419 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:28:48.520 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:28:48.547 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:48.548 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:48.548 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:28:48.602 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:28:48.649 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:28:48.676 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:48.676 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:48.677 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:28:48.777 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:28:48.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:48.808 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:48.809 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:28:48.910 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:28:48.949 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:48.950 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:48.950 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:28:49.049 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:48 smithi084 ceph-mon[32629]: 4.3 scrub starts 2024-09-16T09:28:49.049 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:48 smithi084 ceph-mon[32629]: 4.3 scrub ok 2024-09-16T09:28:49.049 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:48 smithi084 ceph-mon[32629]: pgmap v162: 158 pgs: 158 active+clean; 3.5 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 951 op/s 2024-09-16T09:28:49.050 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:48 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:28:49.050 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:48 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:28:49.050 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:48 smithi084 ceph-mon[32629]: 7.b scrub starts 2024-09-16T09:28:49.051 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:28:49.097 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:49.098 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:49.098 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:28:49.188 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-09-16T09:28:49.188 INFO:teuthology.orchestra.run.smithi084.stdout: "ops": [], 2024-09-16T09:28:49.188 INFO:teuthology.orchestra.run.smithi084.stdout: "complaint_time": 30, 2024-09-16T09:28:49.188 INFO:teuthology.orchestra.run.smithi084.stdout: "num_blocked_ops": 0 2024-09-16T09:28:49.188 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-09-16T09:28:49.199 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:28:49.233 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:49.234 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:49.235 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:28:49.252 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:48 smithi070 ceph-mon[30388]: 4.3 scrub starts 2024-09-16T09:28:49.252 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:48 smithi070 ceph-mon[30388]: 4.3 scrub ok 2024-09-16T09:28:49.253 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:48 smithi070 ceph-mon[30388]: pgmap v162: 158 pgs: 158 active+clean; 3.5 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 951 op/s 2024-09-16T09:28:49.253 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:48 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:28:49.253 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:48 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:28:49.253 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:48 smithi070 ceph-mon[30388]: 7.b scrub starts 2024-09-16T09:28:49.254 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:48 smithi070 ceph-mon[40691]: 4.3 scrub starts 2024-09-16T09:28:49.254 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:48 smithi070 ceph-mon[40691]: 4.3 scrub ok 2024-09-16T09:28:49.254 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:48 smithi070 ceph-mon[40691]: pgmap v162: 158 pgs: 158 active+clean; 3.5 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 951 op/s 2024-09-16T09:28:49.254 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:48 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:28:49.254 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:48 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:28:49.254 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:48 smithi070 ceph-mon[40691]: 7.b scrub starts 2024-09-16T09:28:49.336 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:28:49.363 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:49.364 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:49.364 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:28:49.465 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:28:49.493 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:49.494 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:49.494 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:28:49.595 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:28:49.645 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:49.646 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:49.646 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:28:49.747 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:28:49.763 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:28:49.774 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:49.774 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:49.775 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:28:49.875 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:28:49.916 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:49.917 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:49.918 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:28:49.987 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.4 dump_historic_ops 2024-09-16T09:28:50.019 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:28:50.057 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:50.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:50.058 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:28:50.159 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:28:50.211 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:50.211 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:50.211 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:28:50.237 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:49 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:28:50.238 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:49 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:28:50.238 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:49 smithi070 ceph-mon[30388]: 7.b scrub ok 2024-09-16T09:28:50.238 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:49 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:28:50.238 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:49 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:28:50.238 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:49 smithi070 ceph-mon[40691]: 7.b scrub ok 2024-09-16T09:28:50.312 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:28:50.314 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:28:50.349 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:50.350 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:50.350 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:28:50.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:49 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:28:50.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:49 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:28:50.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:49 smithi084 ceph-mon[32629]: 7.b scrub ok 2024-09-16T09:28:50.451 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:28:50.495 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:50.496 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:50.496 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:28:50.597 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:28:50.639 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:50.639 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:50.639 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:28:50.681 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T09:28:50.740 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:28:50.763 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:50.763 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:50.764 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:28:50.864 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:28:50.908 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:50.909 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:50.909 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:28:51.010 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:28:51.035 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:51.035 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:51.035 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:28:51.136 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:28:51.165 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:51.166 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:51.167 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:28:51.267 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:28:51.292 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:51.293 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:51.294 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:28:51.307 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-16T09:28:51.343 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:50 smithi070 ceph-mon[30388]: pgmap v163: 158 pgs: 158 active+clean; 3.2 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.34k op/s 2024-09-16T09:28:51.343 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:50 smithi070 ceph-mon[30388]: osdmap e135: 8 total, 8 up, 8 in 2024-09-16T09:28:51.344 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:50 smithi070 ceph-mon[40691]: pgmap v163: 158 pgs: 158 active+clean; 3.2 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.34k op/s 2024-09-16T09:28:51.344 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:50 smithi070 ceph-mon[40691]: osdmap e135: 8 total, 8 up, 8 in 2024-09-16T09:28:51.394 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:28:51.409 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:50 smithi084 ceph-mon[32629]: pgmap v163: 158 pgs: 158 active+clean; 3.2 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.34k op/s 2024-09-16T09:28:51.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:50 smithi084 ceph-mon[32629]: osdmap e135: 8 total, 8 up, 8 in 2024-09-16T09:28:51.418 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:51.419 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:51.419 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:28:51.520 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:28:51.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:51.570 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:51.570 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:28:51.671 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:28:51.694 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-09-16T09:28:51.694 INFO:teuthology.orchestra.run.smithi084.stdout: "size": 20, 2024-09-16T09:28:51.694 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 600, 2024-09-16T09:28:51.694 INFO:teuthology.orchestra.run.smithi084.stdout: "ops": [ 2024-09-16T09:28:51.694 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.694 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:22925 7.1 7:89c34711:::benchmark_data_smithi070_78438_object22924:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T09:28:51.694 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:26:21.580979+0000", 2024-09-16T09:28:51.694 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 149.99216136000001, 2024-09-16T09:28:51.695 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.076893300999999997, 2024-09-16T09:28:51.695 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:28:51.695 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:28:51.695 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:28:51.695 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:28:51.695 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:28:51.695 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:28:51.695 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 22925 2024-09-16T09:28:51.695 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.695 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:28:51.695 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.695 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:28:51.695 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.580979+0000", 2024-09-16T09:28:51.695 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:28:51.696 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.696 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.696 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:28:51.696 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.580979+0000", 2024-09-16T09:28:51.696 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:28:51.696 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.696 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.696 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:28:51.696 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.580981+0000", 2024-09-16T09:28:51.696 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.3629999999999999e-06 2024-09-16T09:28:51.696 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.696 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.696 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:28:51.697 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.581034+0000", 2024-09-16T09:28:51.697 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.3300999999999997e-05 2024-09-16T09:28:51.697 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.697 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.697 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:28:51.697 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.581051+0000", 2024-09-16T09:28:51.697 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.6642000000000001e-05 2024-09-16T09:28:51.697 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.697 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.697 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:28:51.697 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.581059+0000", 2024-09-16T09:28:51.697 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.7970000000000001e-06 2024-09-16T09:28:51.697 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.698 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.698 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:28:51.698 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.581280+0000", 2024-09-16T09:28:51.698 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000221752 2024-09-16T09:28:51.698 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.698 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.698 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:28:51.698 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.581389+0000", 2024-09-16T09:28:51.699 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00010817799999999999 2024-09-16T09:28:51.699 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.699 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.699 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 3,6", 2024-09-16T09:28:51.699 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.581517+0000", 2024-09-16T09:28:51.699 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000128128 2024-09-16T09:28:51.700 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.700 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.700 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:28:51.700 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.584179+0000", 2024-09-16T09:28:51.700 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0026624830000000002 2024-09-16T09:28:51.700 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.700 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.700 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:28:51.700 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.585833+0000", 2024-09-16T09:28:51.700 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0016537520000000001 2024-09-16T09:28:51.700 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.700 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.700 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:28:51.701 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.657746+0000", 2024-09-16T09:28:51.701 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.071913191000000001 2024-09-16T09:28:51.701 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.701 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.701 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:28:51.701 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.657816+0000", 2024-09-16T09:28:51.701 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.9633000000000002e-05 2024-09-16T09:28:51.701 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.701 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.701 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:28:51.701 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.657873+0000", 2024-09-16T09:28:51.701 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.7080999999999998e-05 2024-09-16T09:28:51.701 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:28:51.701 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:28:51.702 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:28:51.702 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.702 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.702 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:22927 7.1 7:8fa982a9:::benchmark_data_smithi070_78438_object22926:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T09:28:51.702 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:26:21.582665+0000", 2024-09-16T09:28:51.702 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 149.99047537600001, 2024-09-16T09:28:51.702 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.077694033999999995, 2024-09-16T09:28:51.702 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:28:51.702 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:28:51.702 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:28:51.702 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:28:51.702 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:28:51.702 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:28:51.703 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 22927 2024-09-16T09:28:51.703 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.703 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:28:51.703 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.703 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:28:51.703 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.582665+0000", 2024-09-16T09:28:51.703 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:28:51.703 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.703 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.703 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:28:51.703 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.582665+0000", 2024-09-16T09:28:51.703 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:28:51.703 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.704 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.704 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:28:51.704 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.582667+0000", 2024-09-16T09:28:51.704 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.5179999999999999e-06 2024-09-16T09:28:51.704 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.704 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.704 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:28:51.704 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.582700+0000", 2024-09-16T09:28:51.704 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.2821000000000003e-05 2024-09-16T09:28:51.704 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.704 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.704 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:28:51.705 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.582719+0000", 2024-09-16T09:28:51.705 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.9255000000000001e-05 2024-09-16T09:28:51.705 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.705 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.705 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:28:51.705 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.582727+0000", 2024-09-16T09:28:51.705 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.661e-06 2024-09-16T09:28:51.705 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.705 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.705 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:28:51.705 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.582801+0000", 2024-09-16T09:28:51.705 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.4553000000000003e-05 2024-09-16T09:28:51.705 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.705 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.705 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:28:51.706 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.582991+0000", 2024-09-16T09:28:51.706 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00018939400000000001 2024-09-16T09:28:51.706 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.706 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.706 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 3,6", 2024-09-16T09:28:51.706 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.583125+0000", 2024-09-16T09:28:51.706 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00013424800000000001 2024-09-16T09:28:51.706 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.706 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.706 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:28:51.706 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.585851+0000", 2024-09-16T09:28:51.706 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0027256799999999999 2024-09-16T09:28:51.706 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.706 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.707 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:28:51.707 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.585910+0000", 2024-09-16T09:28:51.707 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.9552999999999997e-05 2024-09-16T09:28:51.707 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.707 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.707 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:28:51.707 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.660257+0000", 2024-09-16T09:28:51.707 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.074347094000000002 2024-09-16T09:28:51.707 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.707 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.707 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:28:51.707 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.660316+0000", 2024-09-16T09:28:51.707 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.8595999999999999e-05 2024-09-16T09:28:51.707 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.707 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.708 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:28:51.708 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.660360+0000", 2024-09-16T09:28:51.708 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.3661000000000001e-05 2024-09-16T09:28:51.708 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:28:51.708 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:28:51.708 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:28:51.708 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.708 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.708 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:22934 7.1 7:87f3da4d:::benchmark_data_smithi070_78438_object22933:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T09:28:51.708 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:26:21.583328+0000", 2024-09-16T09:28:51.708 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 149.989812559, 2024-09-16T09:28:51.708 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.077158353999999998, 2024-09-16T09:28:51.708 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:28:51.709 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:28:51.709 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:28:51.709 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:28:51.709 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:28:51.709 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:28:51.709 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 22934 2024-09-16T09:28:51.709 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.709 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:28:51.709 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.709 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:28:51.709 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.583328+0000", 2024-09-16T09:28:51.709 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:28:51.710 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.710 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.710 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:28:51.710 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.583328+0000", 2024-09-16T09:28:51.710 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:28:51.710 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.710 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.710 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:28:51.710 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.583329+0000", 2024-09-16T09:28:51.710 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0070000000000001e-06 2024-09-16T09:28:51.710 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.710 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.711 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:28:51.711 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.583429+0000", 2024-09-16T09:28:51.711 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.9981000000000004e-05 2024-09-16T09:28:51.711 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.711 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.711 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:28:51.711 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.583441+0000", 2024-09-16T09:28:51.711 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.2196e-05 2024-09-16T09:28:51.711 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.711 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.711 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:28:51.711 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.583448+0000", 2024-09-16T09:28:51.712 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.8460000000000002e-06 2024-09-16T09:28:51.712 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.712 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.712 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:28:51.712 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.584242+0000", 2024-09-16T09:28:51.712 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00079362599999999997 2024-09-16T09:28:51.712 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.712 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.712 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:28:51.712 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.584361+0000", 2024-09-16T09:28:51.712 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000119014 2024-09-16T09:28:51.712 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.712 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.713 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 3,6", 2024-09-16T09:28:51.713 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.584493+0000", 2024-09-16T09:28:51.713 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000132473 2024-09-16T09:28:51.713 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.713 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.713 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:28:51.713 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.585942+0000", 2024-09-16T09:28:51.713 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.001448509 2024-09-16T09:28:51.713 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.713 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.713 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:28:51.713 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.585970+0000", 2024-09-16T09:28:51.713 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.7932e-05 2024-09-16T09:28:51.713 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.714 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.714 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:28:51.714 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.660424+0000", 2024-09-16T09:28:51.714 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.074454300000000001 2024-09-16T09:28:51.714 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.714 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.714 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:28:51.714 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.660462+0000", 2024-09-16T09:28:51.714 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.8302e-05 2024-09-16T09:28:51.714 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.715 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.715 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:28:51.715 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.660487+0000", 2024-09-16T09:28:51.715 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.4168e-05 2024-09-16T09:28:51.715 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:28:51.715 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:28:51.715 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:28:51.715 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.715 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.715 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:26638 7.1 7:8fd77d04:::benchmark_data_smithi070_78438_object26637:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T09:28:51.716 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:26:23.276469+0000", 2024-09-16T09:28:51.716 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 148.29667216000001, 2024-09-16T09:28:51.716 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.051045984000000003, 2024-09-16T09:28:51.716 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:28:51.716 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:28:51.716 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:28:51.716 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:28:51.716 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:28:51.716 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:28:51.716 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 26638 2024-09-16T09:28:51.716 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.717 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:28:51.717 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.717 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:28:51.717 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.276469+0000", 2024-09-16T09:28:51.717 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:28:51.717 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.717 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.717 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:28:51.717 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.276469+0000", 2024-09-16T09:28:51.717 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:28:51.718 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.718 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.718 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:28:51.718 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.276470+0000", 2024-09-16T09:28:51.718 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.265e-06 2024-09-16T09:28:51.718 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.718 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.718 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:28:51.718 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.276723+0000", 2024-09-16T09:28:51.718 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00025323799999999998 2024-09-16T09:28:51.718 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.718 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.718 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:28:51.719 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.276735+0000", 2024-09-16T09:28:51.719 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.2201999999999999e-05 2024-09-16T09:28:51.719 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.719 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.719 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:28:51.719 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.276742+0000", 2024-09-16T09:28:51.719 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.097e-06 2024-09-16T09:28:51.719 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.719 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.719 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:28:51.719 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.276795+0000", 2024-09-16T09:28:51.719 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.2945000000000002e-05 2024-09-16T09:28:51.719 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.719 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.719 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:28:51.720 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.276912+0000", 2024-09-16T09:28:51.720 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00011658999999999999 2024-09-16T09:28:51.720 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.720 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.720 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 3,6", 2024-09-16T09:28:51.720 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.277036+0000", 2024-09-16T09:28:51.720 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00012364599999999999 2024-09-16T09:28:51.720 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.720 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.720 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:28:51.720 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.278637+0000", 2024-09-16T09:28:51.720 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0016011269999999999 2024-09-16T09:28:51.720 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.721 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.721 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:28:51.721 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.280947+0000", 2024-09-16T09:28:51.721 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0023097479999999999 2024-09-16T09:28:51.721 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.721 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.721 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:28:51.721 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.327359+0000", 2024-09-16T09:28:51.721 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.046412002000000001 2024-09-16T09:28:51.721 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.721 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.721 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:28:51.722 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.327462+0000", 2024-09-16T09:28:51.722 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000103546 2024-09-16T09:28:51.722 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.722 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.722 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:28:51.722 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.327515+0000", 2024-09-16T09:28:51.722 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.2577999999999997e-05 2024-09-16T09:28:51.722 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:28:51.722 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:28:51.722 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:28:51.722 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.722 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.722 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:27462 7.1 7:8b37553f:::benchmark_data_smithi070_78438_object27461:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T09:28:51.722 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:26:23.719582+0000", 2024-09-16T09:28:51.722 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 147.85355897100001, 2024-09-16T09:28:51.723 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.073272959999999998, 2024-09-16T09:28:51.723 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:28:51.723 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:28:51.723 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:28:51.723 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:28:51.723 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:28:51.723 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:28:51.723 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 27462 2024-09-16T09:28:51.723 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.723 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:28:51.723 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.723 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:28:51.724 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.719582+0000", 2024-09-16T09:28:51.724 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:28:51.724 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.724 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.724 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:28:51.724 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.719582+0000", 2024-09-16T09:28:51.724 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:28:51.724 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.724 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.724 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:28:51.724 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.719583+0000", 2024-09-16T09:28:51.724 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.192e-06 2024-09-16T09:28:51.724 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.724 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.725 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:28:51.725 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.719694+0000", 2024-09-16T09:28:51.725 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000110592 2024-09-16T09:28:51.725 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.725 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.725 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:28:51.725 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.719702+0000", 2024-09-16T09:28:51.725 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.3610000000000001e-06 2024-09-16T09:28:51.725 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.726 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.726 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:28:51.726 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.719707+0000", 2024-09-16T09:28:51.726 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.0640000000000001e-06 2024-09-16T09:28:51.726 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.726 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.726 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:28:51.726 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.719811+0000", 2024-09-16T09:28:51.726 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00010433399999999999 2024-09-16T09:28:51.726 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.726 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.726 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:28:51.726 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.719878+0000", 2024-09-16T09:28:51.726 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.6651999999999998e-05 2024-09-16T09:28:51.727 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.727 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.727 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 3,6", 2024-09-16T09:28:51.727 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.719964+0000", 2024-09-16T09:28:51.727 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.5840999999999993e-05 2024-09-16T09:28:51.727 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.727 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.727 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:28:51.727 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.789718+0000", 2024-09-16T09:28:51.727 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.069754134999999995 2024-09-16T09:28:51.727 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.727 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.727 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:28:51.728 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.792737+0000", 2024-09-16T09:28:51.728 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0030193170000000001 2024-09-16T09:28:51.728 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.728 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.728 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:28:51.728 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.792756+0000", 2024-09-16T09:28:51.728 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.8240000000000002e-05 2024-09-16T09:28:51.728 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.728 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.728 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:28:51.728 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.792812+0000", 2024-09-16T09:28:51.729 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.6081000000000001e-05 2024-09-16T09:28:51.729 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.729 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.729 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:28:51.729 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.792855+0000", 2024-09-16T09:28:51.729 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.3151000000000001e-05 2024-09-16T09:28:51.729 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:28:51.729 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:28:51.729 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:28:51.729 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.729 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.729 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_repop_reply(client.24698.0:27462 7.1 e89/82 ondisk, result = 0)", 2024-09-16T09:28:51.729 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:26:23.721262+0000", 2024-09-16T09:28:51.729 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 147.851879037, 2024-09-16T09:28:51.730 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.068463016000000002, 2024-09-16T09:28:51.730 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:28:51.730 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:28:51.730 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "started", 2024-09-16T09:28:51.730 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:28:51.730 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.730 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:28:51.730 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.721262+0000", 2024-09-16T09:28:51.730 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:28:51.730 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.730 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.730 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:28:51.730 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.721262+0000", 2024-09-16T09:28:51.730 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:28:51.731 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.731 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.731 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:28:51.731 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.721262+0000", 2024-09-16T09:28:51.731 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.3900000000000005e-07 2024-09-16T09:28:51.731 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.731 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.731 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:28:51.731 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.721266+0000", 2024-09-16T09:28:51.731 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.349e-06 2024-09-16T09:28:51.731 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.731 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.731 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:28:51.731 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.787935+0000", 2024-09-16T09:28:51.732 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.066669333999999997 2024-09-16T09:28:51.732 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.732 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.732 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:28:51.732 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.787946+0000", 2024-09-16T09:28:51.732 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1358999999999999e-05 2024-09-16T09:28:51.732 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.732 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.732 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:28:51.732 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.789705+0000", 2024-09-16T09:28:51.732 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.001758721 2024-09-16T09:28:51.732 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.733 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.733 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:28:51.733 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.789713+0000", 2024-09-16T09:28:51.733 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.3989999999999998e-06 2024-09-16T09:28:51.733 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.733 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.733 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:28:51.733 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.789725+0000", 2024-09-16T09:28:51.733 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.2315e-05 2024-09-16T09:28:51.733 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:28:51.733 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:28:51.733 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:28:51.733 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.733 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.734 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:37503 7.1 7:898ab9a8:::benchmark_data_smithi070_78438_object37502:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-16T09:28:51.734 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:26:29.406093+0000", 2024-09-16T09:28:51.734 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 142.16704784500001, 2024-09-16T09:28:51.734 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.065952329000000004, 2024-09-16T09:28:51.734 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:28:51.734 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:28:51.734 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:28:51.734 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:28:51.734 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:28:51.734 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:28:51.734 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 37503 2024-09-16T09:28:51.734 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.734 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:28:51.734 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.735 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:28:51.735 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:29.406093+0000", 2024-09-16T09:28:51.735 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:28:51.735 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.735 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.735 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:28:51.735 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:29.406093+0000", 2024-09-16T09:28:51.735 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:28:51.735 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.735 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.735 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:28:51.735 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:29.406095+0000", 2024-09-16T09:28:51.736 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.5710000000000001e-06 2024-09-16T09:28:51.736 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.736 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.736 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:28:51.736 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:29.406143+0000", 2024-09-16T09:28:51.736 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.8238000000000001e-05 2024-09-16T09:28:51.736 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.736 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.736 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:28:51.736 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:29.406153+0000", 2024-09-16T09:28:51.736 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0292000000000001e-05 2024-09-16T09:28:51.737 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.737 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.737 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:28:51.737 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:29.406158+0000", 2024-09-16T09:28:51.737 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.9490000000000004e-06 2024-09-16T09:28:51.737 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.737 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.737 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:28:51.737 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:29.406212+0000", 2024-09-16T09:28:51.737 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.3687000000000003e-05 2024-09-16T09:28:51.737 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.737 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.737 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:28:51.737 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:29.406358+0000", 2024-09-16T09:28:51.738 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00014636599999999999 2024-09-16T09:28:51.738 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.738 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.738 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 3,6", 2024-09-16T09:28:51.738 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:29.406486+0000", 2024-09-16T09:28:51.738 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00012791899999999999 2024-09-16T09:28:51.738 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.739 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.739 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:28:51.739 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:29.407602+0000", 2024-09-16T09:28:51.739 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0011164129999999999 2024-09-16T09:28:51.739 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.739 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.739 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:28:51.739 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:29.413354+0000", 2024-09-16T09:28:51.739 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0057519659999999998 2024-09-16T09:28:51.739 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.739 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.739 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:28:51.740 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:29.471932+0000", 2024-09-16T09:28:51.740 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.058577683999999998 2024-09-16T09:28:51.740 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.740 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.740 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:28:51.740 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:29.471997+0000", 2024-09-16T09:28:51.740 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.4554000000000006e-05 2024-09-16T09:28:51.740 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.740 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.740 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:28:51.741 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:29.472045+0000", 2024-09-16T09:28:51.741 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.8690000000000003e-05 2024-09-16T09:28:51.741 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:28:51.741 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:28:51.741 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:28:51.741 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.741 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.741 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:37513 7.1 7:8f73be0a:::benchmark_data_smithi070_78438_object37512:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e90)", 2024-09-16T09:28:51.741 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:26:29.411858+0000", 2024-09-16T09:28:51.741 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 142.161283115, 2024-09-16T09:28:51.741 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.060299758000000002, 2024-09-16T09:28:51.741 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:28:51.741 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:28:51.741 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:28:51.742 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:28:51.742 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:28:51.742 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:28:51.742 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 37513 2024-09-16T09:28:51.742 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.742 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:28:51.742 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.742 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:28:51.742 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:29.411858+0000", 2024-09-16T09:28:51.742 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:28:51.742 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.742 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.742 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:28:51.742 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:29.411858+0000", 2024-09-16T09:28:51.742 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:28:51.743 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.743 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.743 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:28:51.743 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:29.411861+0000", 2024-09-16T09:28:51.743 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.0860000000000002e-06 2024-09-16T09:28:51.743 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.743 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.743 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:28:51.743 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:29.411977+0000", 2024-09-16T09:28:51.743 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000115717 2024-09-16T09:28:51.743 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.743 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.743 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:28:51.743 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:29.411988+0000", 2024-09-16T09:28:51.744 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1153e-05 2024-09-16T09:28:51.744 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.744 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.744 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:28:51.744 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:29.411993+0000", 2024-09-16T09:28:51.744 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.1000000000000003e-06 2024-09-16T09:28:51.744 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.744 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.744 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:28:51.744 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:29.412040+0000", 2024-09-16T09:28:51.744 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.7586000000000001e-05 2024-09-16T09:28:51.744 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.744 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.744 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:28:51.745 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:29.412156+0000", 2024-09-16T09:28:51.745 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000115309 2024-09-16T09:28:51.745 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.745 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.745 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 3,6", 2024-09-16T09:28:51.745 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:29.412282+0000", 2024-09-16T09:28:51.745 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000126057 2024-09-16T09:28:51.745 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.745 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.745 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:28:51.745 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:29.413436+0000", 2024-09-16T09:28:51.745 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0011544859999999999 2024-09-16T09:28:51.745 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.746 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.746 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:28:51.746 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:29.414792+0000", 2024-09-16T09:28:51.746 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0013560709999999999 2024-09-16T09:28:51.746 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.746 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.746 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:28:51.746 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:29.472083+0000", 2024-09-16T09:28:51.746 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.057290747000000003 2024-09-16T09:28:51.747 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.747 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.747 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:28:51.747 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:29.472128+0000", 2024-09-16T09:28:51.747 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.4894000000000001e-05 2024-09-16T09:28:51.747 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.747 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.747 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:28:51.747 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:29.472157+0000", 2024-09-16T09:28:51.747 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.9552000000000001e-05 2024-09-16T09:28:51.747 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:28:51.748 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:28:51.748 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:28:51.748 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.748 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.748 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_repop(client.24698.0:37522 7.8 e90/82 7:14d9e782:::benchmark_data_smithi070_78438_object37521:head v 90'2363, mlcod=90'2360)", 2024-09-16T09:28:51.748 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:26:29.417519+0000", 2024-09-16T09:28:51.748 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 142.15562194699999, 2024-09-16T09:28:51.748 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.05336834, 2024-09-16T09:28:51.748 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:28:51.748 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:28:51.749 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:28:51.749 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:28:51.749 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.749 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:28:51.749 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:29.417519+0000", 2024-09-16T09:28:51.749 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:28:51.749 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.749 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.749 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:28:51.749 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:29.417519+0000", 2024-09-16T09:28:51.750 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:28:51.750 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.750 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.750 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:28:51.750 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:29.417519+0000", 2024-09-16T09:28:51.750 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.0399999999999996e-07 2024-09-16T09:28:51.750 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.750 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.750 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:28:51.750 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:29.417633+0000", 2024-09-16T09:28:51.750 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00011314700000000001 2024-09-16T09:28:51.750 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.750 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.750 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:28:51.750 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:29.417641+0000", 2024-09-16T09:28:51.751 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.2220000000000002e-06 2024-09-16T09:28:51.751 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.751 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.751 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:28:51.751 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:29.417645+0000", 2024-09-16T09:28:51.751 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.4739999999999999e-06 2024-09-16T09:28:51.751 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.751 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.751 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:28:51.751 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:29.417685+0000", 2024-09-16T09:28:51.751 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.0184e-05 2024-09-16T09:28:51.751 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.751 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.751 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:28:51.751 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:29.417697+0000", 2024-09-16T09:28:51.752 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1505000000000001e-05 2024-09-16T09:28:51.752 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.752 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.752 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:28:51.752 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:29.470756+0000", 2024-09-16T09:28:51.752 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.053058816000000002 2024-09-16T09:28:51.752 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.752 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.752 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:28:51.752 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:29.470887+0000", 2024-09-16T09:28:51.752 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000131488 2024-09-16T09:28:51.752 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:28:51.752 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:28:51.753 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:28:51.753 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.753 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.753 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_repop(client.24698.0:37524 7.3 e90/82 7:c3609e81:::benchmark_data_smithi070_78438_object37523:head v 90'2409, mlcod=90'2407)", 2024-09-16T09:28:51.753 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:26:29.418747+0000", 2024-09-16T09:28:51.753 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 142.15439393400001, 2024-09-16T09:28:51.753 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.057664957000000003, 2024-09-16T09:28:51.753 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:28:51.753 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:28:51.753 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:28:51.753 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:28:51.754 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.754 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:28:51.754 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:29.418747+0000", 2024-09-16T09:28:51.754 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:28:51.754 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.754 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.754 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:28:51.754 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:29.418747+0000", 2024-09-16T09:28:51.754 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:28:51.754 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.754 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.754 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:28:51.754 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:29.418748+0000", 2024-09-16T09:28:51.754 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.7000000000000004e-07 2024-09-16T09:28:51.754 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.754 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.755 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:28:51.755 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:29.418806+0000", 2024-09-16T09:28:51.755 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.8749000000000003e-05 2024-09-16T09:28:51.755 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.755 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.755 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:28:51.755 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:29.470751+0000", 2024-09-16T09:28:51.755 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0519446 2024-09-16T09:28:51.755 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.755 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.755 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:28:51.755 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:29.470777+0000", 2024-09-16T09:28:51.755 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.5473000000000002e-05 2024-09-16T09:28:51.755 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.756 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.756 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:28:51.756 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:29.471122+0000", 2024-09-16T09:28:51.756 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00034562200000000002 2024-09-16T09:28:51.756 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.756 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.756 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:28:51.756 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:29.471142+0000", 2024-09-16T09:28:51.756 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.0210000000000001e-05 2024-09-16T09:28:51.756 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.756 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.756 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:28:51.757 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:29.476382+0000", 2024-09-16T09:28:51.757 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.005239489 2024-09-16T09:28:51.757 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.757 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.757 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:28:51.757 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:29.476412+0000", 2024-09-16T09:28:51.757 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.0043999999999999e-05 2024-09-16T09:28:51.757 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:28:51.757 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:28:51.757 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:28:51.757 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.757 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.757 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:60611 7.1 7:87030956:::benchmark_data_smithi070_78438_object60610:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-16T09:28:51.757 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:26:44.705726+0000", 2024-09-16T09:28:51.758 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 126.867414626, 2024-09-16T09:28:51.758 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.069662907999999996, 2024-09-16T09:28:51.758 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:28:51.758 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:28:51.758 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:28:51.758 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:28:51.758 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:28:51.758 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:28:51.758 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 60611 2024-09-16T09:28:51.758 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.758 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:28:51.758 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.758 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:28:51.758 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.705726+0000", 2024-09-16T09:28:51.758 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:28:51.759 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.759 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.759 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:28:51.759 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.705726+0000", 2024-09-16T09:28:51.759 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:28:51.759 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.759 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.759 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:28:51.760 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.705729+0000", 2024-09-16T09:28:51.760 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.4880000000000001e-06 2024-09-16T09:28:51.760 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.760 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.760 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:28:51.760 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.705766+0000", 2024-09-16T09:28:51.760 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.7682000000000003e-05 2024-09-16T09:28:51.760 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.760 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.760 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:28:51.760 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.705781+0000", 2024-09-16T09:28:51.760 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.4499e-05 2024-09-16T09:28:51.760 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.760 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.760 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:28:51.761 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.705788+0000", 2024-09-16T09:28:51.761 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.5950000000000001e-06 2024-09-16T09:28:51.761 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.761 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.761 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:28:51.761 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.705827+0000", 2024-09-16T09:28:51.761 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.8266000000000003e-05 2024-09-16T09:28:51.761 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.761 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.761 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:28:51.761 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.705940+0000", 2024-09-16T09:28:51.761 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000112773 2024-09-16T09:28:51.761 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.761 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.761 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 3,6", 2024-09-16T09:28:51.762 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.706099+0000", 2024-09-16T09:28:51.762 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000159507 2024-09-16T09:28:51.762 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.762 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.762 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:28:51.762 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.707710+0000", 2024-09-16T09:28:51.762 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.001610867 2024-09-16T09:28:51.762 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.762 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.762 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:28:51.762 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.707986+0000", 2024-09-16T09:28:51.762 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00027570200000000001 2024-09-16T09:28:51.762 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.762 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.763 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:28:51.763 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.775304+0000", 2024-09-16T09:28:51.763 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.067318174999999994 2024-09-16T09:28:51.763 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.763 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.763 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:28:51.763 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.775351+0000", 2024-09-16T09:28:51.763 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.7630000000000003e-05 2024-09-16T09:28:51.763 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.763 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.763 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:28:51.763 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.775389+0000", 2024-09-16T09:28:51.763 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.7724e-05 2024-09-16T09:28:51.763 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:28:51.763 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:28:51.763 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:28:51.764 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.764 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.764 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:62039 7.1 7:8ca71bdb:::benchmark_data_smithi070_78438_object62038:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-16T09:28:51.764 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:26:45.815079+0000", 2024-09-16T09:28:51.764 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 125.75806212800001, 2024-09-16T09:28:51.764 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.062287673000000002, 2024-09-16T09:28:51.764 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:28:51.764 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:28:51.764 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:28:51.764 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:28:51.764 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:28:51.764 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:28:51.764 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 62039 2024-09-16T09:28:51.764 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.764 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:28:51.765 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.765 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:28:51.765 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.815079+0000", 2024-09-16T09:28:51.765 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:28:51.765 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.765 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.765 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:28:51.765 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.815079+0000", 2024-09-16T09:28:51.765 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:28:51.765 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.765 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.765 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:28:51.765 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.815081+0000", 2024-09-16T09:28:51.765 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.8759999999999999e-06 2024-09-16T09:28:51.766 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.766 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.766 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:28:51.766 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.815108+0000", 2024-09-16T09:28:51.766 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.7129999999999999e-05 2024-09-16T09:28:51.766 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.766 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.766 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:28:51.766 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.815124+0000", 2024-09-16T09:28:51.766 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.5942000000000001e-05 2024-09-16T09:28:51.766 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.766 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.767 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:28:51.767 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.815130+0000", 2024-09-16T09:28:51.767 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.2299999999999996e-06 2024-09-16T09:28:51.767 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.767 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.767 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:28:51.767 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.815189+0000", 2024-09-16T09:28:51.767 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.8690000000000002e-05 2024-09-16T09:28:51.767 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.767 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.767 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:28:51.767 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.815310+0000", 2024-09-16T09:28:51.767 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000121354 2024-09-16T09:28:51.767 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.767 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.768 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 3,6", 2024-09-16T09:28:51.768 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.815443+0000", 2024-09-16T09:28:51.768 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00013313100000000001 2024-09-16T09:28:51.768 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.768 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.768 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:28:51.768 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.817657+0000", 2024-09-16T09:28:51.768 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0022140110000000001 2024-09-16T09:28:51.768 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.768 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.768 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:28:51.768 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.818250+0000", 2024-09-16T09:28:51.768 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000592601 2024-09-16T09:28:51.768 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.769 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.769 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:28:51.769 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.877252+0000", 2024-09-16T09:28:51.769 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.059002286000000001 2024-09-16T09:28:51.769 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.769 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.769 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:28:51.769 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.877316+0000", 2024-09-16T09:28:51.770 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.4264999999999995e-05 2024-09-16T09:28:51.770 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.770 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.770 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:28:51.770 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.877366+0000", 2024-09-16T09:28:51.770 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.0157000000000001e-05 2024-09-16T09:28:51.770 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:28:51.770 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:28:51.770 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:28:51.770 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.771 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.771 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:86890 7.1 7:8b0c621b:::benchmark_data_smithi070_78438_object86889:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-16T09:28:51.771 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:27:06.136694+0000", 2024-09-16T09:28:51.771 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 105.43644707, 2024-09-16T09:28:51.771 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.086345176999999995, 2024-09-16T09:28:51.771 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:28:51.771 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:28:51.771 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:28:51.771 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:28:51.771 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:28:51.771 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:28:51.771 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 86890 2024-09-16T09:28:51.771 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.772 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:28:51.772 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.772 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:28:51.772 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.136694+0000", 2024-09-16T09:28:51.772 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:28:51.772 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.772 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.772 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:28:51.772 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.136694+0000", 2024-09-16T09:28:51.772 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:28:51.772 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.772 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.772 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:28:51.773 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.136695+0000", 2024-09-16T09:28:51.773 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.4160000000000001e-06 2024-09-16T09:28:51.773 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.773 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.773 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:28:51.773 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.136800+0000", 2024-09-16T09:28:51.773 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000104957 2024-09-16T09:28:51.773 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.773 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.773 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:28:51.773 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.136813+0000", 2024-09-16T09:28:51.773 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.3023000000000001e-05 2024-09-16T09:28:51.773 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.773 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.774 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:28:51.774 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.136820+0000", 2024-09-16T09:28:51.774 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.2180000000000002e-06 2024-09-16T09:28:51.774 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.774 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.774 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:28:51.774 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.136891+0000", 2024-09-16T09:28:51.774 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.0697000000000004e-05 2024-09-16T09:28:51.774 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.774 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.774 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:28:51.774 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.137034+0000", 2024-09-16T09:28:51.774 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000142421 2024-09-16T09:28:51.774 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.774 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.775 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 3,6", 2024-09-16T09:28:51.775 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.137177+0000", 2024-09-16T09:28:51.775 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00014350699999999999 2024-09-16T09:28:51.775 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.775 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.775 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:28:51.775 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.144870+0000", 2024-09-16T09:28:51.775 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0076926549999999996 2024-09-16T09:28:51.775 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.775 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.775 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:28:51.775 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.144970+0000", 2024-09-16T09:28:51.775 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000100511 2024-09-16T09:28:51.775 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.776 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.776 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:28:51.776 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.222926+0000", 2024-09-16T09:28:51.776 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.077955615000000006 2024-09-16T09:28:51.776 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.776 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.776 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:28:51.776 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.222988+0000", 2024-09-16T09:28:51.776 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.1916999999999997e-05 2024-09-16T09:28:51.776 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.776 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.776 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:28:51.776 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.223039+0000", 2024-09-16T09:28:51.777 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.1239999999999997e-05 2024-09-16T09:28:51.777 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:28:51.777 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:28:51.777 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:28:51.777 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.777 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.777 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:86891 7.1 7:8533eaa2:::benchmark_data_smithi070_78438_object86890:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-16T09:28:51.777 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:27:06.137963+0000", 2024-09-16T09:28:51.777 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 105.435177944, 2024-09-16T09:28:51.777 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.089654372999999996, 2024-09-16T09:28:51.777 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:28:51.777 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:28:51.778 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:28:51.778 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:28:51.778 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:28:51.778 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:28:51.778 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 86891 2024-09-16T09:28:51.778 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.778 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:28:51.778 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.778 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:28:51.778 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.137963+0000", 2024-09-16T09:28:51.778 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:28:51.778 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.778 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.778 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:28:51.779 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.137963+0000", 2024-09-16T09:28:51.779 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:28:51.779 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.779 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.779 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:28:51.779 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.137964+0000", 2024-09-16T09:28:51.779 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.5629999999999999e-06 2024-09-16T09:28:51.779 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.779 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.779 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:28:51.779 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.137999+0000", 2024-09-16T09:28:51.779 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.4564000000000003e-05 2024-09-16T09:28:51.780 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.780 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.780 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:28:51.780 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.138016+0000", 2024-09-16T09:28:51.780 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.6677999999999999e-05 2024-09-16T09:28:51.780 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.780 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.780 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:28:51.781 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.138023+0000", 2024-09-16T09:28:51.781 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.5059999999999998e-06 2024-09-16T09:28:51.781 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.781 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.781 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:28:51.781 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.139570+0000", 2024-09-16T09:28:51.781 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0015467809999999999 2024-09-16T09:28:51.781 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.781 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.781 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:28:51.781 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.139690+0000", 2024-09-16T09:28:51.781 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000120387 2024-09-16T09:28:51.781 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.781 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.782 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 3,6", 2024-09-16T09:28:51.782 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.139833+0000", 2024-09-16T09:28:51.782 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00014291799999999999 2024-09-16T09:28:51.782 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.782 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.782 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:28:51.782 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.144931+0000", 2024-09-16T09:28:51.782 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0050973110000000002 2024-09-16T09:28:51.782 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.782 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.782 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:28:51.782 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.144973+0000", 2024-09-16T09:28:51.782 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.2038e-05 2024-09-16T09:28:51.782 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.783 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.783 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:28:51.783 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.227507+0000", 2024-09-16T09:28:51.783 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.082533910000000002 2024-09-16T09:28:51.783 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.783 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.783 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:28:51.783 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.227568+0000", 2024-09-16T09:28:51.783 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.1229999999999995e-05 2024-09-16T09:28:51.783 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.783 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.784 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:28:51.784 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.227617+0000", 2024-09-16T09:28:51.784 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.9487000000000002e-05 2024-09-16T09:28:51.784 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:28:51.784 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:28:51.784 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:28:51.784 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.784 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.784 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:86895 7.1 7:852b1173:::benchmark_data_smithi070_78438_object86894:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-16T09:28:51.785 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:27:06.138894+0000", 2024-09-16T09:28:51.785 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 105.43424673200001, 2024-09-16T09:28:51.785 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.091395905, 2024-09-16T09:28:51.785 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:28:51.785 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:28:51.785 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:28:51.785 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:28:51.786 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:28:51.786 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:28:51.786 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 86895 2024-09-16T09:28:51.786 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.786 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:28:51.786 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.787 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:28:51.787 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.138894+0000", 2024-09-16T09:28:51.787 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:28:51.787 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.787 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.787 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:28:51.788 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.138894+0000", 2024-09-16T09:28:51.788 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:28:51.788 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.788 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.788 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:28:51.788 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.138895+0000", 2024-09-16T09:28:51.789 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.093e-06 2024-09-16T09:28:51.789 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.789 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.789 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:28:51.789 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.138932+0000", 2024-09-16T09:28:51.789 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.6934999999999998e-05 2024-09-16T09:28:51.790 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.790 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.790 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:28:51.790 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.138946+0000", 2024-09-16T09:28:51.790 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.3772e-05 2024-09-16T09:28:51.790 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.791 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.791 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:28:51.791 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.138953+0000", 2024-09-16T09:28:51.791 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.2810000000000003e-06 2024-09-16T09:28:51.791 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.791 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.792 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:28:51.792 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.142367+0000", 2024-09-16T09:28:51.792 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0034135559999999999 2024-09-16T09:28:51.792 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.792 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.792 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:28:51.793 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.142499+0000", 2024-09-16T09:28:51.793 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00013187 2024-09-16T09:28:51.793 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.793 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.793 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 3,6", 2024-09-16T09:28:51.793 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.142646+0000", 2024-09-16T09:28:51.794 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00014712500000000001 2024-09-16T09:28:51.794 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.794 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.794 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:28:51.794 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.144963+0000", 2024-09-16T09:28:51.794 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0023176260000000001 2024-09-16T09:28:51.795 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.795 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.795 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:28:51.795 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.144977+0000", 2024-09-16T09:28:51.795 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.3509000000000001e-05 2024-09-16T09:28:51.796 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.796 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.796 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:28:51.796 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.230194+0000", 2024-09-16T09:28:51.796 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.085216682000000002 2024-09-16T09:28:51.796 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.797 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.797 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:28:51.797 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.230252+0000", 2024-09-16T09:28:51.797 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.8023000000000003e-05 2024-09-16T09:28:51.797 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.797 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.798 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:28:51.798 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.230290+0000", 2024-09-16T09:28:51.798 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.8433000000000003e-05 2024-09-16T09:28:51.798 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:28:51.798 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:28:51.798 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:28:51.799 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.799 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.799 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:88353 7.1 7:8cc65f4e:::benchmark_data_smithi070_78438_object88352:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-16T09:28:51.799 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:27:07.475281+0000", 2024-09-16T09:28:51.799 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 104.097860224, 2024-09-16T09:28:51.799 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.057670512, 2024-09-16T09:28:51.800 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:28:51.800 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:28:51.800 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:28:51.800 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:28:51.800 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:28:51.800 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:28:51.801 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 88353 2024-09-16T09:28:51.801 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.801 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:28:51.801 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.801 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:28:51.801 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:07.475281+0000", 2024-09-16T09:28:51.802 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:28:51.802 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.802 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.802 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:28:51.802 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:07.475281+0000", 2024-09-16T09:28:51.802 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:28:51.803 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.803 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.803 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:28:51.803 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:07.475281+0000", 2024-09-16T09:28:51.803 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.3699999999999999e-07 2024-09-16T09:28:51.803 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.804 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.804 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:28:51.804 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:07.475337+0000", 2024-09-16T09:28:51.804 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.5787e-05 2024-09-16T09:28:51.804 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.804 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.805 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:28:51.805 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:07.475348+0000", 2024-09-16T09:28:51.805 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1233000000000001e-05 2024-09-16T09:28:51.805 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.805 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.805 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:28:51.806 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:07.475356+0000", 2024-09-16T09:28:51.806 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.2400000000000001e-06 2024-09-16T09:28:51.806 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.806 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.806 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:28:51.806 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:07.477364+0000", 2024-09-16T09:28:51.807 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0020087759999999999 2024-09-16T09:28:51.807 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.807 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.807 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:28:51.807 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:07.477490+0000", 2024-09-16T09:28:51.807 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000125938 2024-09-16T09:28:51.808 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.808 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.808 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 3,6", 2024-09-16T09:28:51.808 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:07.477673+0000", 2024-09-16T09:28:51.808 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00018255699999999999 2024-09-16T09:28:51.808 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.809 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.809 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:28:51.809 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:07.480013+0000", 2024-09-16T09:28:51.809 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0023395389999999999 2024-09-16T09:28:51.809 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.809 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.810 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:28:51.810 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:07.480362+0000", 2024-09-16T09:28:51.810 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000349694 2024-09-16T09:28:51.810 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.810 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.810 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:28:51.810 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:07.532825+0000", 2024-09-16T09:28:51.810 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.052463217999999999 2024-09-16T09:28:51.811 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.811 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.811 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:28:51.811 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:07.532874+0000", 2024-09-16T09:28:51.811 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.8992000000000001e-05 2024-09-16T09:28:51.811 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.811 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.812 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:28:51.812 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:07.532951+0000", 2024-09-16T09:28:51.812 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.6700999999999996e-05 2024-09-16T09:28:51.812 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:28:51.812 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:28:51.812 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:28:51.812 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.812 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.812 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:202416 7.1 7:8fb2ceea:::benchmark_data_smithi070_78438_object76285:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135)", 2024-09-16T09:28:51.812 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:28:50.737455+0000", 2024-09-16T09:28:51.812 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 0.83568588899999996, 2024-09-16T09:28:51.812 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.073585405000000007, 2024-09-16T09:28:51.813 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:28:51.813 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:28:51.813 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:28:51.813 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:28:51.813 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:28:51.813 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:28:51.813 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 202416 2024-09-16T09:28:51.813 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.813 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:28:51.813 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.813 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:28:51.813 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.737455+0000", 2024-09-16T09:28:51.813 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:28:51.813 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.814 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.814 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:28:51.814 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.737455+0000", 2024-09-16T09:28:51.814 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:28:51.814 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.814 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.814 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:28:51.814 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.737456+0000", 2024-09-16T09:28:51.814 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.3400000000000001e-06 2024-09-16T09:28:51.814 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.814 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.814 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:28:51.814 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.737462+0000", 2024-09-16T09:28:51.814 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.429e-06 2024-09-16T09:28:51.815 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.815 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.815 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:28:51.815 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.737474+0000", 2024-09-16T09:28:51.815 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1875000000000001e-05 2024-09-16T09:28:51.815 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.815 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.815 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:28:51.815 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.737479+0000", 2024-09-16T09:28:51.815 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.3789999999999997e-06 2024-09-16T09:28:51.815 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.815 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.816 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:28:51.816 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.737540+0000", 2024-09-16T09:28:51.816 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.0568e-05 2024-09-16T09:28:51.816 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.816 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.816 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:28:51.816 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.737670+0000", 2024-09-16T09:28:51.816 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00013019799999999999 2024-09-16T09:28:51.816 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.816 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.816 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 3,6", 2024-09-16T09:28:51.816 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.737793+0000", 2024-09-16T09:28:51.816 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00012317599999999999 2024-09-16T09:28:51.816 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.817 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.817 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:28:51.817 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.740234+0000", 2024-09-16T09:28:51.817 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0024407489999999999 2024-09-16T09:28:51.817 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.817 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.817 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:28:51.817 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.741125+0000", 2024-09-16T09:28:51.817 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00089109699999999998 2024-09-16T09:28:51.817 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.817 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.818 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:28:51.818 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.810945+0000", 2024-09-16T09:28:51.818 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.069820241000000005 2024-09-16T09:28:51.818 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.818 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.818 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:28:51.818 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.811000+0000", 2024-09-16T09:28:51.818 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.4672000000000001e-05 2024-09-16T09:28:51.818 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.818 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.818 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:28:51.819 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.811040+0000", 2024-09-16T09:28:51.819 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.0680999999999999e-05 2024-09-16T09:28:51.819 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:28:51.819 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:28:51.819 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:28:51.819 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.819 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.819 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_repop(client.24698.0:202635 7.4 e135/82 7:2055e12a:::benchmark_data_smithi070_78438_object76504:head v 135'12468, mlcod=135'12465)", 2024-09-16T09:28:51.819 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:28:51.040989+0000", 2024-09-16T09:28:51.819 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 0.53215234099999997, 2024-09-16T09:28:51.819 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.105010197, 2024-09-16T09:28:51.819 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:28:51.819 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:28:51.819 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:28:51.820 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:28:51.820 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.820 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:28:51.820 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.040989+0000", 2024-09-16T09:28:51.820 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:28:51.820 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.820 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.820 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:28:51.820 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.040989+0000", 2024-09-16T09:28:51.820 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:28:51.820 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.820 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.820 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:28:51.820 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.040989+0000", 2024-09-16T09:28:51.821 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.1799999999999995e-07 2024-09-16T09:28:51.821 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.821 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.821 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:28:51.821 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.040993+0000", 2024-09-16T09:28:51.821 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.5269999999999999e-06 2024-09-16T09:28:51.821 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.821 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.821 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:28:51.821 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.041000+0000", 2024-09-16T09:28:51.822 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.3509999999999996e-06 2024-09-16T09:28:51.822 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.822 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.822 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:28:51.822 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.041004+0000", 2024-09-16T09:28:51.822 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.532e-06 2024-09-16T09:28:51.822 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.822 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.822 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:28:51.822 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.041044+0000", 2024-09-16T09:28:51.822 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.9258999999999998e-05 2024-09-16T09:28:51.823 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.823 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.823 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:28:51.823 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.041056+0000", 2024-09-16T09:28:51.823 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.2258e-05 2024-09-16T09:28:51.823 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.823 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.823 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:28:51.823 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.145950+0000", 2024-09-16T09:28:51.823 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.104893666 2024-09-16T09:28:51.823 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.823 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.823 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:28:51.823 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.145999+0000", 2024-09-16T09:28:51.823 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.9085999999999997e-05 2024-09-16T09:28:51.824 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:28:51.824 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:28:51.824 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:28:51.824 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.824 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.824 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:202638 7.1 7:8f557cfd:::benchmark_data_smithi070_78438_object76507:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135)", 2024-09-16T09:28:51.824 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:28:51.042963+0000", 2024-09-16T09:28:51.824 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 0.53017808200000005, 2024-09-16T09:28:51.824 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.111245755, 2024-09-16T09:28:51.824 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:28:51.824 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:28:51.825 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:28:51.825 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:28:51.825 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:28:51.825 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:28:51.825 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 202638 2024-09-16T09:28:51.825 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.825 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:28:51.825 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.825 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:28:51.825 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.042963+0000", 2024-09-16T09:28:51.825 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:28:51.825 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.825 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.826 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:28:51.826 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.042963+0000", 2024-09-16T09:28:51.826 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:28:51.826 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.826 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.826 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:28:51.826 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.042964+0000", 2024-09-16T09:28:51.826 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.5910000000000001e-06 2024-09-16T09:28:51.826 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.826 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.826 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:28:51.826 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.042970+0000", 2024-09-16T09:28:51.826 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.7960000000000001e-06 2024-09-16T09:28:51.826 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.826 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.827 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:28:51.827 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.137960+0000", 2024-09-16T09:28:51.827 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.094989799 2024-09-16T09:28:51.827 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.827 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.827 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:28:51.827 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.137972+0000", 2024-09-16T09:28:51.827 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1556000000000001e-05 2024-09-16T09:28:51.827 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.827 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.827 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:28:51.827 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.144247+0000", 2024-09-16T09:28:51.827 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0062759540000000003 2024-09-16T09:28:51.827 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.828 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.828 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:28:51.828 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.144377+0000", 2024-09-16T09:28:51.828 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00012968800000000001 2024-09-16T09:28:51.828 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.828 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.828 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 3,6", 2024-09-16T09:28:51.828 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.144499+0000", 2024-09-16T09:28:51.828 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00012222000000000001 2024-09-16T09:28:51.828 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.829 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.829 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:28:51.829 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.151527+0000", 2024-09-16T09:28:51.829 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0070274819999999998 2024-09-16T09:28:51.829 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.829 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.829 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:28:51.829 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.151636+0000", 2024-09-16T09:28:51.829 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000109224 2024-09-16T09:28:51.829 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.829 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.829 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:28:51.829 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.154123+0000", 2024-09-16T09:28:51.829 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0024866279999999998 2024-09-16T09:28:51.830 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.830 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.830 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:28:51.830 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.154169+0000", 2024-09-16T09:28:51.830 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.5902000000000002e-05 2024-09-16T09:28:51.830 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.830 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.830 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:28:51.830 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.154209+0000", 2024-09-16T09:28:51.830 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.9915e-05 2024-09-16T09:28:51.830 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:28:51.830 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:28:51.830 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:28:51.830 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.831 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.831 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_repop(client.24698.0:202642 7.4 e135/82 7:2c34e77c:::benchmark_data_smithi070_78438_object76511:head v 135'12469, mlcod=135'12466)", 2024-09-16T09:28:51.831 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:28:51.048625+0000", 2024-09-16T09:28:51.831 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 0.52451589799999998, 2024-09-16T09:28:51.831 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.104253999, 2024-09-16T09:28:51.831 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:28:51.831 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:28:51.831 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:28:51.831 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:28:51.831 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.831 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:28:51.831 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.048625+0000", 2024-09-16T09:28:51.831 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:28:51.831 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.832 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.832 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:28:51.832 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.048625+0000", 2024-09-16T09:28:51.832 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:28:51.832 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.832 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.832 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:28:51.832 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.048626+0000", 2024-09-16T09:28:51.832 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0219999999999999e-06 2024-09-16T09:28:51.832 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.832 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.832 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:28:51.832 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.048633+0000", 2024-09-16T09:28:51.832 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.6379999999999998e-06 2024-09-16T09:28:51.833 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.833 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.833 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:28:51.833 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.137040+0000", 2024-09-16T09:28:51.833 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.088407854999999994 2024-09-16T09:28:51.833 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.833 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.833 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:28:51.833 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.137056+0000", 2024-09-16T09:28:51.834 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.5583e-05 2024-09-16T09:28:51.834 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.834 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.834 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:28:51.834 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.139863+0000", 2024-09-16T09:28:51.834 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0028066559999999998 2024-09-16T09:28:51.834 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.834 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.834 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:28:51.834 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.139883+0000", 2024-09-16T09:28:51.834 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.0599e-05 2024-09-16T09:28:51.834 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.834 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.835 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:28:51.835 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.152841+0000", 2024-09-16T09:28:51.835 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.012958081999999999 2024-09-16T09:28:51.835 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:28:51.835 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:28:51.835 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:28:51.835 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.152879+0000", 2024-09-16T09:28:51.835 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.7564000000000002e-05 2024-09-16T09:28:51.835 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:28:51.835 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:28:51.835 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:28:51.835 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:28:51.835 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:28:51.835 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-09-16T09:28:51.836 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:51.837 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:51.837 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:28:51.938 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:28:51.966 INFO:teuthology.orchestra.run.smithi070.stderr:set osd.3 primary-affinity to 0.476536 (8312302) 2024-09-16T09:28:51.979 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:51.980 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:51.980 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:28:52.081 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:28:52.103 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:52.104 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:52.104 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:28:52.205 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:28:52.237 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:52.238 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:52.238 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:28:52.339 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:28:52.361 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:52.362 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:52.362 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:28:52.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:51 smithi084 ceph-mon[32629]: 3.11 scrub starts 2024-09-16T09:28:52.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:51 smithi084 ceph-mon[32629]: 3.11 scrub ok 2024-09-16T09:28:52.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:51 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1042325926' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 3, "weight": 0.4765362689388495} : dispatch 2024-09-16T09:28:52.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:51 smithi084 ceph-mon[32629]: 7.2 scrub starts 2024-09-16T09:28:52.420 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:28:51 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:28:51] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:28:52.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:51 smithi070 ceph-mon[30388]: 3.11 scrub starts 2024-09-16T09:28:52.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:51 smithi070 ceph-mon[30388]: 3.11 scrub ok 2024-09-16T09:28:52.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:51 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1042325926' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 3, "weight": 0.4765362689388495} : dispatch 2024-09-16T09:28:52.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:51 smithi070 ceph-mon[30388]: 7.2 scrub starts 2024-09-16T09:28:52.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:51 smithi070 ceph-mon[40691]: 3.11 scrub starts 2024-09-16T09:28:52.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:51 smithi070 ceph-mon[40691]: 3.11 scrub ok 2024-09-16T09:28:52.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:51 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1042325926' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 3, "weight": 0.4765362689388495} : dispatch 2024-09-16T09:28:52.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:51 smithi070 ceph-mon[40691]: 7.2 scrub starts 2024-09-16T09:28:52.463 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:28:52.486 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:52.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:52.487 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:28:52.587 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:28:52.608 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:52.608 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:52.609 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:28:52.710 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:28:52.732 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:52.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:52.733 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:28:52.750 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.5 dump_ops_in_flight 2024-09-16T09:28:52.833 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:28:52.855 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:52.856 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:52.856 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:28:52.957 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:28:52.972 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:28:52 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:28:52.973 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:52 smithi084 ceph-mon[32629]: pgmap v165: 158 pgs: 158 active+clean; 3.2 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 769 B/s rd, 0 B/s wr, 846 op/s 2024-09-16T09:28:52.973 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:52 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1042325926' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 3, "weight": 0.4765362689388495}]': finished 2024-09-16T09:28:52.973 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:52 smithi084 ceph-mon[32629]: osdmap e136: 8 total, 8 up, 8 in 2024-09-16T09:28:52.973 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:52 smithi084 ceph-mon[32629]: 7.2 scrub ok 2024-09-16T09:28:52.973 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:52 smithi084 ceph-mon[32629]: 4.18 scrub starts 2024-09-16T09:28:52.973 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:52 smithi084 ceph-mon[32629]: 4.18 scrub ok 2024-09-16T09:28:52.987 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:52.988 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:52.988 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:28:53.064 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:28:53.089 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:28:53.126 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:53.127 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:53.127 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:28:53.228 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:28:53.255 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:53.255 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:53.256 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:28:53.357 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:28:53.390 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:53.391 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:53.391 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:28:53.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:52 smithi070 ceph-mon[30388]: pgmap v165: 158 pgs: 158 active+clean; 3.2 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 769 B/s rd, 0 B/s wr, 846 op/s 2024-09-16T09:28:53.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:52 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1042325926' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 3, "weight": 0.4765362689388495}]': finished 2024-09-16T09:28:53.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:52 smithi070 ceph-mon[30388]: osdmap e136: 8 total, 8 up, 8 in 2024-09-16T09:28:53.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:52 smithi070 ceph-mon[30388]: 7.2 scrub ok 2024-09-16T09:28:53.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:52 smithi070 ceph-mon[30388]: 4.18 scrub starts 2024-09-16T09:28:53.422 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:52 smithi070 ceph-mon[30388]: 4.18 scrub ok 2024-09-16T09:28:53.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:52 smithi070 ceph-mon[40691]: pgmap v165: 158 pgs: 158 active+clean; 3.2 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 769 B/s rd, 0 B/s wr, 846 op/s 2024-09-16T09:28:53.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:52 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1042325926' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 3, "weight": 0.4765362689388495}]': finished 2024-09-16T09:28:53.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:52 smithi070 ceph-mon[40691]: osdmap e136: 8 total, 8 up, 8 in 2024-09-16T09:28:53.423 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:52 smithi070 ceph-mon[40691]: 7.2 scrub ok 2024-09-16T09:28:53.423 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:52 smithi070 ceph-mon[40691]: 4.18 scrub starts 2024-09-16T09:28:53.423 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:52 smithi070 ceph-mon[40691]: 4.18 scrub ok 2024-09-16T09:28:53.491 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:28:53.513 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:53.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:53.514 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:28:53.610 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-09-16T09:28:53.610 INFO:teuthology.orchestra.run.smithi084.stdout: "ops": [], 2024-09-16T09:28:53.610 INFO:teuthology.orchestra.run.smithi084.stdout: "num_ops": 0 2024-09-16T09:28:53.610 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-09-16T09:28:53.615 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:28:53.652 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:53.653 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:53.653 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:28:53.754 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:28:53.778 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:53.779 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:53.779 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:28:53.880 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:28:53.906 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:53.907 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:53.907 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:28:54.008 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:28:54.036 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:54.036 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:54.037 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:28:54.137 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:28:54.177 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:54.177 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:54.178 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:28:54.279 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:28:54.299 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:54.300 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:54.300 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:28:54.354 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:53 smithi084 ceph-mon[32629]: 6.10 scrub starts 2024-09-16T09:28:54.354 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:53 smithi084 ceph-mon[32629]: 6.10 scrub ok 2024-09-16T09:28:54.354 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:53 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:28:54.354 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:53 smithi084 ceph-mon[32629]: 5.1a scrub starts 2024-09-16T09:28:54.355 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:53 smithi084 ceph-mon[32629]: 5.1a scrub ok 2024-09-16T09:28:54.355 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:53 smithi084 ceph-mon[32629]: osdmap e137: 8 total, 8 up, 8 in 2024-09-16T09:28:54.355 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:53 smithi084 ceph-mon[32629]: 6.1f scrub starts 2024-09-16T09:28:54.355 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:53 smithi084 ceph-mon[32629]: 6.1f scrub ok 2024-09-16T09:28:54.355 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:53 smithi084 ceph-mon[32629]: 5.17 scrub starts 2024-09-16T09:28:54.355 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:53 smithi084 ceph-mon[32629]: 5.17 scrub ok 2024-09-16T09:28:54.396 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.5 dump_blocked_ops 2024-09-16T09:28:54.401 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:28:54.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:53 smithi070 ceph-mon[30388]: 6.10 scrub starts 2024-09-16T09:28:54.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:53 smithi070 ceph-mon[30388]: 6.10 scrub ok 2024-09-16T09:28:54.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:53 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:28:54.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:53 smithi070 ceph-mon[30388]: 5.1a scrub starts 2024-09-16T09:28:54.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:53 smithi070 ceph-mon[30388]: 5.1a scrub ok 2024-09-16T09:28:54.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:53 smithi070 ceph-mon[30388]: osdmap e137: 8 total, 8 up, 8 in 2024-09-16T09:28:54.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:53 smithi070 ceph-mon[30388]: 6.1f scrub starts 2024-09-16T09:28:54.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:53 smithi070 ceph-mon[30388]: 6.1f scrub ok 2024-09-16T09:28:54.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:53 smithi070 ceph-mon[30388]: 5.17 scrub starts 2024-09-16T09:28:54.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:53 smithi070 ceph-mon[30388]: 5.17 scrub ok 2024-09-16T09:28:54.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:53 smithi070 ceph-mon[40691]: 6.10 scrub starts 2024-09-16T09:28:54.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:53 smithi070 ceph-mon[40691]: 6.10 scrub ok 2024-09-16T09:28:54.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:53 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:28:54.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:53 smithi070 ceph-mon[40691]: 5.1a scrub starts 2024-09-16T09:28:54.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:53 smithi070 ceph-mon[40691]: 5.1a scrub ok 2024-09-16T09:28:54.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:53 smithi070 ceph-mon[40691]: osdmap e137: 8 total, 8 up, 8 in 2024-09-16T09:28:54.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:53 smithi070 ceph-mon[40691]: 6.1f scrub starts 2024-09-16T09:28:54.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:53 smithi070 ceph-mon[40691]: 6.1f scrub ok 2024-09-16T09:28:54.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:53 smithi070 ceph-mon[40691]: 5.17 scrub starts 2024-09-16T09:28:54.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:53 smithi070 ceph-mon[40691]: 5.17 scrub ok 2024-09-16T09:28:54.472 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:54.473 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:54.473 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:28:54.574 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:28:54.605 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:54.606 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:54.606 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:28:54.707 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:28:54.737 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:54.738 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:54.739 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:28:54.739 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:28:54.839 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:28:54.869 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:54.870 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:54.870 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:28:54.971 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:28:55.010 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:55.011 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:55.011 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:28:55.112 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:28:55.149 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:55.149 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:55.150 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:28:55.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:54 smithi084 ceph-mon[32629]: pgmap v168: 158 pgs: 19 peering, 139 active+clean; 3.0 GiB data, 9.8 GiB used, 705 GiB / 715 GiB avail; 0 B/s wr, 1.36k op/s 2024-09-16T09:28:55.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:54 smithi084 ceph-mon[32629]: 5.14 scrub starts 2024-09-16T09:28:55.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:54 smithi084 ceph-mon[32629]: 5.14 scrub ok 2024-09-16T09:28:55.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:54 smithi084 ceph-mon[32629]: Health check failed: Reduced data availability: 11 pgs inactive, 18 pgs peering (PG_AVAILABILITY) 2024-09-16T09:28:55.251 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:28:55.289 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:55.289 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:55.290 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:28:55.380 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-09-16T09:28:55.380 INFO:teuthology.orchestra.run.smithi084.stdout: "ops": [], 2024-09-16T09:28:55.380 INFO:teuthology.orchestra.run.smithi084.stdout: "complaint_time": 30, 2024-09-16T09:28:55.380 INFO:teuthology.orchestra.run.smithi084.stdout: "num_blocked_ops": 0 2024-09-16T09:28:55.380 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-09-16T09:28:55.391 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:28:55.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:54 smithi070 ceph-mon[30388]: pgmap v168: 158 pgs: 19 peering, 139 active+clean; 3.0 GiB data, 9.8 GiB used, 705 GiB / 715 GiB avail; 0 B/s wr, 1.36k op/s 2024-09-16T09:28:55.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:54 smithi070 ceph-mon[30388]: 5.14 scrub starts 2024-09-16T09:28:55.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:54 smithi070 ceph-mon[30388]: 5.14 scrub ok 2024-09-16T09:28:55.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:54 smithi070 ceph-mon[30388]: Health check failed: Reduced data availability: 11 pgs inactive, 18 pgs peering (PG_AVAILABILITY) 2024-09-16T09:28:55.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:54 smithi070 ceph-mon[40691]: pgmap v168: 158 pgs: 19 peering, 139 active+clean; 3.0 GiB data, 9.8 GiB used, 705 GiB / 715 GiB avail; 0 B/s wr, 1.36k op/s 2024-09-16T09:28:55.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:54 smithi070 ceph-mon[40691]: 5.14 scrub starts 2024-09-16T09:28:55.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:54 smithi070 ceph-mon[40691]: 5.14 scrub ok 2024-09-16T09:28:55.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:54 smithi070 ceph-mon[40691]: Health check failed: Reduced data availability: 11 pgs inactive, 18 pgs peering (PG_AVAILABILITY) 2024-09-16T09:28:55.432 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:55.433 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:55.433 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:28:55.536 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:28:55.576 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:55.577 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:55.577 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:28:55.678 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:28:55.709 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:55.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:55.710 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:28:55.810 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:28:55.836 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:55.837 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:55.837 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:28:55.938 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:28:55.989 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:55.990 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:55.990 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:28:56.091 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:28:56.119 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:56.120 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:56.120 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:28:56.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:55 smithi084 ceph-mon[32629]: 6.13 scrub starts 2024-09-16T09:28:56.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:55 smithi084 ceph-mon[32629]: 6.13 scrub ok 2024-09-16T09:28:56.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:55 smithi084 ceph-mon[32629]: 4.0 scrub starts 2024-09-16T09:28:56.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:55 smithi084 ceph-mon[32629]: 4.0 scrub ok 2024-09-16T09:28:56.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:55 smithi084 ceph-mon[32629]: 4.d scrub starts 2024-09-16T09:28:56.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:55 smithi084 ceph-mon[32629]: 4.d scrub ok 2024-09-16T09:28:56.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:55 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:28:56.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:55 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:28:56.221 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:28:56.256 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:56.257 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:56.257 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:28:56.274 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:55 smithi070 ceph-mon[30388]: 6.13 scrub starts 2024-09-16T09:28:56.274 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:55 smithi070 ceph-mon[30388]: 6.13 scrub ok 2024-09-16T09:28:56.274 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:55 smithi070 ceph-mon[30388]: 4.0 scrub starts 2024-09-16T09:28:56.274 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:55 smithi070 ceph-mon[30388]: 4.0 scrub ok 2024-09-16T09:28:56.275 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:55 smithi070 ceph-mon[30388]: 4.d scrub starts 2024-09-16T09:28:56.275 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:55 smithi070 ceph-mon[30388]: 4.d scrub ok 2024-09-16T09:28:56.275 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:55 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:28:56.275 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:55 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:28:56.275 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:55 smithi070 ceph-mon[40691]: 6.13 scrub starts 2024-09-16T09:28:56.275 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:55 smithi070 ceph-mon[40691]: 6.13 scrub ok 2024-09-16T09:28:56.275 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:55 smithi070 ceph-mon[40691]: 4.0 scrub starts 2024-09-16T09:28:56.275 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:55 smithi070 ceph-mon[40691]: 4.0 scrub ok 2024-09-16T09:28:56.275 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:55 smithi070 ceph-mon[40691]: 4.d scrub starts 2024-09-16T09:28:56.275 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:55 smithi070 ceph-mon[40691]: 4.d scrub ok 2024-09-16T09:28:56.275 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:55 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:28:56.275 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:55 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:28:56.303 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.5 dump_historic_ops 2024-09-16T09:28:56.359 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:28:56.368 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:28:56.397 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:28:56.410 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:56.411 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:56.411 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:28:56.512 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:28:56.589 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:56.589 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:56.589 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:28:56.655 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:28:56.690 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:28:56.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:56.749 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:56.749 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:28:56.850 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:28:56.892 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:56.892 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:56.893 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:28:56.994 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:28:57.035 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:56 smithi084 ceph-mon[32629]: pgmap v169: 158 pgs: 19 peering, 139 active+clean; 3.0 GiB data, 9.8 GiB used, 705 GiB / 715 GiB avail; 0 B/s wr, 783 op/s 2024-09-16T09:28:57.035 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:57 smithi084 ceph-mon[32629]: 4.11 scrub starts 2024-09-16T09:28:57.035 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:57 smithi084 ceph-mon[32629]: 4.11 scrub ok 2024-09-16T09:28:57.035 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:57 smithi084 ceph-mon[32629]: 6.c scrub starts 2024-09-16T09:28:57.035 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:57 smithi084 ceph-mon[32629]: 6.c scrub ok 2024-09-16T09:28:57.035 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:57 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:28:57.035 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:57 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:28:57.035 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:57 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:28:57.036 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:57 smithi084 ceph-mon[32629]: 4.14 scrub starts 2024-09-16T09:28:57.036 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:57 smithi084 ceph-mon[32629]: 4.14 scrub ok 2024-09-16T09:28:57.039 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:57.040 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:57.040 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:28:57.097 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:56 smithi070 ceph-mon[40691]: pgmap v169: 158 pgs: 19 peering, 139 active+clean; 3.0 GiB data, 9.8 GiB used, 705 GiB / 715 GiB avail; 0 B/s wr, 783 op/s 2024-09-16T09:28:57.097 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:56 smithi070 ceph-mon[40691]: 4.11 scrub starts 2024-09-16T09:28:57.097 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:56 smithi070 ceph-mon[40691]: 4.11 scrub ok 2024-09-16T09:28:57.097 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:56 smithi070 ceph-mon[40691]: 6.c scrub starts 2024-09-16T09:28:57.097 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:56 smithi070 ceph-mon[40691]: 6.c scrub ok 2024-09-16T09:28:57.097 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:56 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:28:57.098 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:56 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:28:57.098 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:56 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:28:57.098 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:56 smithi070 ceph-mon[40691]: 4.14 scrub starts 2024-09-16T09:28:57.098 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:56 smithi070 ceph-mon[40691]: 4.14 scrub ok 2024-09-16T09:28:57.100 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:56 smithi070 ceph-mon[30388]: pgmap v169: 158 pgs: 19 peering, 139 active+clean; 3.0 GiB data, 9.8 GiB used, 705 GiB / 715 GiB avail; 0 B/s wr, 783 op/s 2024-09-16T09:28:57.101 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:56 smithi070 ceph-mon[30388]: 4.11 scrub starts 2024-09-16T09:28:57.101 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:56 smithi070 ceph-mon[30388]: 4.11 scrub ok 2024-09-16T09:28:57.101 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:56 smithi070 ceph-mon[30388]: 6.c scrub starts 2024-09-16T09:28:57.101 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:56 smithi070 ceph-mon[30388]: 6.c scrub ok 2024-09-16T09:28:57.101 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:56 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:28:57.101 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:56 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:28:57.101 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:56 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:28:57.101 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:56 smithi070 ceph-mon[30388]: 4.14 scrub starts 2024-09-16T09:28:57.101 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:56 smithi070 ceph-mon[30388]: 4.14 scrub ok 2024-09-16T09:28:57.141 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:28:57.165 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:57.166 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:57.166 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:28:57.268 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:28:57.295 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:57.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:57.296 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:28:57.297 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:28:57.297 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:28:57.330 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:28:57.331 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:28:57.382 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:28:57.383 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:28:57.396 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:28:57.427 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:28:57.427 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:28:57.434 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:57.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:57.435 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:28:57.457 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:28:57.457 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:28:57.479 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:28:57.479 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:28:57.514 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:28:57.514 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:28:57.536 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:28:57.550 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:28:57.550 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:28:57.575 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:57.576 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:57.576 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:28:57.677 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:28:57.712 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:57.713 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:57.713 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:28:57.814 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:28:57.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:57.846 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:57.847 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:28:57.947 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:28:57.979 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:57.979 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:57.980 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:28:58.004 INFO:teuthology.orchestra.run.smithi070.stderr:nodeep-scrub is unset 2024-09-16T09:28:58.081 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:28:58.102 INFO:teuthology.orchestra.run.smithi084.stdout:ERROR: (22) Invalid argument 2024-09-16T09:28:58.102 INFO:teuthology.orchestra.run.smithi084.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:28:58.119 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:58.119 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:58.120 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:28:58.221 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:28:58.258 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:58.259 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:58.259 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:28:58.360 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:28:58.399 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:58.400 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:58.400 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:28:58.413 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:57 smithi084 ceph-mon[32629]: 7.a scrub starts 2024-09-16T09:28:58.413 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:57 smithi084 ceph-mon[32629]: 3.b scrub starts 2024-09-16T09:28:58.413 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:57 smithi084 ceph-mon[32629]: 3.b scrub ok 2024-09-16T09:28:58.413 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:57 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/2083010496' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:28:58.413 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:57 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/437240734' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:28:58.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:57 smithi070 ceph-mon[30388]: 7.a scrub starts 2024-09-16T09:28:58.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:57 smithi070 ceph-mon[30388]: 3.b scrub starts 2024-09-16T09:28:58.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:57 smithi070 ceph-mon[30388]: 3.b scrub ok 2024-09-16T09:28:58.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:57 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2083010496' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:28:58.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:57 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/437240734' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:28:58.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:57 smithi070 ceph-mon[40691]: 7.a scrub starts 2024-09-16T09:28:58.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:57 smithi070 ceph-mon[40691]: 3.b scrub starts 2024-09-16T09:28:58.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:57 smithi070 ceph-mon[40691]: 3.b scrub ok 2024-09-16T09:28:58.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:57 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/2083010496' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:28:58.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:57 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/437240734' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:28:58.502 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:28:58.544 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:58.544 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:58.545 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:28:58.646 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:28:58.675 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:58.675 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:58.676 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:28:58.777 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:28:58.804 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:58.805 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:58.805 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:28:58.906 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:28:58.935 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:58.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:58.936 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:28:59.012 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.6 dump_ops_in_flight 2024-09-16T09:28:59.037 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:28:59.083 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:59.083 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:59.084 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:28:59.170 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:58 smithi070 ceph-mon[30388]: 5.a scrub starts 2024-09-16T09:28:59.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:58 smithi070 ceph-mon[30388]: 5.a scrub ok 2024-09-16T09:28:59.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:58 smithi070 ceph-mon[30388]: 7.a scrub ok 2024-09-16T09:28:59.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:58 smithi070 ceph-mon[30388]: pgmap v170: 158 pgs: 11 peering, 147 active+clean; 2.9 GiB data, 9.5 GiB used, 706 GiB / 715 GiB avail; 672 B/s rd, 0 B/s wr, 855 op/s 2024-09-16T09:28:59.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:58 smithi070 ceph-mon[30388]: 5.3 scrub starts 2024-09-16T09:28:59.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:58 smithi070 ceph-mon[30388]: 5.3 scrub ok 2024-09-16T09:28:59.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:58 smithi070 ceph-mon[30388]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-16T09:28:59.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:58 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/437240734' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-16T09:28:59.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:58 smithi070 ceph-mon[30388]: osdmap e138: 8 total, 8 up, 8 in 2024-09-16T09:28:59.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:58 smithi070 ceph-mon[30388]: 5.1f scrub starts 2024-09-16T09:28:59.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:58 smithi070 ceph-mon[30388]: 5.1f scrub ok 2024-09-16T09:28:59.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:58 smithi070 ceph-mon[40691]: 5.a scrub starts 2024-09-16T09:28:59.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:58 smithi070 ceph-mon[40691]: 5.a scrub ok 2024-09-16T09:28:59.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:58 smithi070 ceph-mon[40691]: 7.a scrub ok 2024-09-16T09:28:59.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:58 smithi070 ceph-mon[40691]: pgmap v170: 158 pgs: 11 peering, 147 active+clean; 2.9 GiB data, 9.5 GiB used, 706 GiB / 715 GiB avail; 672 B/s rd, 0 B/s wr, 855 op/s 2024-09-16T09:28:59.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:58 smithi070 ceph-mon[40691]: 5.3 scrub starts 2024-09-16T09:28:59.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:58 smithi070 ceph-mon[40691]: 5.3 scrub ok 2024-09-16T09:28:59.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:58 smithi070 ceph-mon[40691]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-16T09:28:59.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:58 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/437240734' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-16T09:28:59.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:58 smithi070 ceph-mon[40691]: osdmap e138: 8 total, 8 up, 8 in 2024-09-16T09:28:59.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:58 smithi070 ceph-mon[40691]: 5.1f scrub starts 2024-09-16T09:28:59.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:58 smithi070 ceph-mon[40691]: 5.1f scrub ok 2024-09-16T09:28:59.185 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:28:59.222 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:59.234 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:59.235 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:28:59.336 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:28:59.368 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:28:59.368 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:59.368 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:59.369 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:28:59.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:58 smithi084 ceph-mon[32629]: 5.a scrub starts 2024-09-16T09:28:59.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:58 smithi084 ceph-mon[32629]: 5.a scrub ok 2024-09-16T09:28:59.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:58 smithi084 ceph-mon[32629]: 7.a scrub ok 2024-09-16T09:28:59.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:58 smithi084 ceph-mon[32629]: pgmap v170: 158 pgs: 11 peering, 147 active+clean; 2.9 GiB data, 9.5 GiB used, 706 GiB / 715 GiB avail; 672 B/s rd, 0 B/s wr, 855 op/s 2024-09-16T09:28:59.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:58 smithi084 ceph-mon[32629]: 5.3 scrub starts 2024-09-16T09:28:59.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:58 smithi084 ceph-mon[32629]: 5.3 scrub ok 2024-09-16T09:28:59.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:58 smithi084 ceph-mon[32629]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-16T09:28:59.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:58 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/437240734' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-16T09:28:59.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:58 smithi084 ceph-mon[32629]: osdmap e138: 8 total, 8 up, 8 in 2024-09-16T09:28:59.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:58 smithi084 ceph-mon[32629]: 5.1f scrub starts 2024-09-16T09:28:59.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:58 smithi084 ceph-mon[32629]: 5.1f scrub ok 2024-09-16T09:28:59.469 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:28:59.497 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:59.498 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:59.498 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:28:59.599 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:28:59.631 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:59.632 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:59.632 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:28:59.733 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:28:59.762 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:28:59.763 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:59.763 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:28:59.864 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:28:59.903 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:28:59.904 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:28:59.905 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:28:59.946 INFO:teuthology.orchestra.run.smithi084.stdout:ERROR: (22) Invalid argument 2024-09-16T09:28:59.946 INFO:teuthology.orchestra.run.smithi084.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:29:00.005 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:29:00.044 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:00.045 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:00.045 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:29:00.146 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:29:00.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:59 smithi084 ceph-mon[32629]: 3.6 scrub starts 2024-09-16T09:29:00.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:59 smithi084 ceph-mon[32629]: 3.6 scrub ok 2024-09-16T09:29:00.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:59 smithi084 ceph-mon[32629]: 3.7 scrub starts 2024-09-16T09:29:00.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:59 smithi084 ceph-mon[32629]: 3.7 scrub ok 2024-09-16T09:29:00.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:28:59 smithi084 ceph-mon[32629]: Health check update: Reduced data availability: 6 pgs inactive, 10 pgs peering (PG_AVAILABILITY) 2024-09-16T09:29:00.174 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:00.175 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:00.175 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:29:00.276 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:29:00.305 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:00.306 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:00.306 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:29:00.407 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:29:00.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:59 smithi070 ceph-mon[30388]: 3.6 scrub starts 2024-09-16T09:29:00.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:59 smithi070 ceph-mon[30388]: 3.6 scrub ok 2024-09-16T09:29:00.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:59 smithi070 ceph-mon[30388]: 3.7 scrub starts 2024-09-16T09:29:00.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:59 smithi070 ceph-mon[30388]: 3.7 scrub ok 2024-09-16T09:29:00.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:28:59 smithi070 ceph-mon[30388]: Health check update: Reduced data availability: 6 pgs inactive, 10 pgs peering (PG_AVAILABILITY) 2024-09-16T09:29:00.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:59 smithi070 ceph-mon[40691]: 3.6 scrub starts 2024-09-16T09:29:00.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:59 smithi070 ceph-mon[40691]: 3.6 scrub ok 2024-09-16T09:29:00.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:59 smithi070 ceph-mon[40691]: 3.7 scrub starts 2024-09-16T09:29:00.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:59 smithi070 ceph-mon[40691]: 3.7 scrub ok 2024-09-16T09:29:00.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:28:59 smithi070 ceph-mon[40691]: Health check update: Reduced data availability: 6 pgs inactive, 10 pgs peering (PG_AVAILABILITY) 2024-09-16T09:29:00.443 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:00.443 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:00.444 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:29:00.538 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T09:29:00.545 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:29:00.616 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:00.617 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:00.618 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:29:00.718 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:29:00.766 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:00.766 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:00.767 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:29:00.794 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.6 dump_blocked_ops 2024-09-16T09:29:00.868 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:29:00.921 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:00.922 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:00.923 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:29:00.972 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:29:01.024 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:29:01.056 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:01.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:01.057 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:29:01.115 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:29:01.158 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:29:01.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:00 smithi084 ceph-mon[32629]: pgmap v172: 158 pgs: 158 active+clean; 2.7 GiB data, 9.0 GiB used, 706 GiB / 715 GiB avail; 674 B/s rd, 0 B/s wr, 1.28k op/s 2024-09-16T09:29:01.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:00 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:29:01.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:00 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:29:01.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:00 smithi084 ceph-mon[32629]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 6 pgs inactive, 10 pgs peering) 2024-09-16T09:29:01.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:00 smithi084 ceph-mon[32629]: Cluster is now healthy 2024-09-16T09:29:01.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:00 smithi084 ceph-mon[32629]: osdmap e139: 8 total, 8 up, 8 in 2024-09-16T09:29:01.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:00 smithi084 ceph-mon[32629]: 6.7 scrub starts 2024-09-16T09:29:01.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:00 smithi084 ceph-mon[32629]: 6.7 scrub ok 2024-09-16T09:29:01.199 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:01.199 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:01.200 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:29:01.235 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd set noscrub 2024-09-16T09:29:01.280 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:00 smithi070 ceph-mon[30388]: pgmap v172: 158 pgs: 158 active+clean; 2.7 GiB data, 9.0 GiB used, 706 GiB / 715 GiB avail; 674 B/s rd, 0 B/s wr, 1.28k op/s 2024-09-16T09:29:01.281 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:00 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:29:01.281 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:00 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:29:01.281 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:00 smithi070 ceph-mon[30388]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 6 pgs inactive, 10 pgs peering) 2024-09-16T09:29:01.281 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:00 smithi070 ceph-mon[30388]: Cluster is now healthy 2024-09-16T09:29:01.281 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:00 smithi070 ceph-mon[30388]: osdmap e139: 8 total, 8 up, 8 in 2024-09-16T09:29:01.281 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:01 smithi070 ceph-mon[30388]: 6.7 scrub starts 2024-09-16T09:29:01.281 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:01 smithi070 ceph-mon[30388]: 6.7 scrub ok 2024-09-16T09:29:01.284 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:01 smithi070 ceph-mon[40691]: pgmap v172: 158 pgs: 158 active+clean; 2.7 GiB data, 9.0 GiB used, 706 GiB / 715 GiB avail; 674 B/s rd, 0 B/s wr, 1.28k op/s 2024-09-16T09:29:01.284 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:01 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:29:01.284 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:01 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:29:01.284 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:01 smithi070 ceph-mon[40691]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 6 pgs inactive, 10 pgs peering) 2024-09-16T09:29:01.284 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:01 smithi070 ceph-mon[40691]: Cluster is now healthy 2024-09-16T09:29:01.284 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:01 smithi070 ceph-mon[40691]: osdmap e139: 8 total, 8 up, 8 in 2024-09-16T09:29:01.284 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:01 smithi070 ceph-mon[40691]: 6.7 scrub starts 2024-09-16T09:29:01.284 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:01 smithi070 ceph-mon[40691]: 6.7 scrub ok 2024-09-16T09:29:01.301 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:29:01.328 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T09:29:01.328 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T09:29:01.328 INFO:tasks.thrashosds.thrasher:inject_pause on osd.4 2024-09-16T09:29:01.328 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-16T09:29:01.328 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-16T09:29:01.329 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.4 config set filestore_inject_stall 3 2024-09-16T09:29:01.351 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:01.352 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:01.352 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:29:01.453 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:29:01.492 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:01.493 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:01.493 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:29:01.594 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:29:01.643 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:01.643 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:01.643 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:29:01.702 INFO:teuthology.orchestra.run.smithi084.stdout:ERROR: (22) Invalid argument 2024-09-16T09:29:01.702 INFO:teuthology.orchestra.run.smithi084.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:29:01.744 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:29:01.755 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:29:01.756 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:29:01.790 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:29:01.790 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:01.791 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:01.791 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:29:01.792 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:29:01.792 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:29:01.830 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:29:01.830 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:29:01.864 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:29:01.864 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:29:01.892 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:29:01.894 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:29:01.894 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:29:01.918 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:01.919 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:01.919 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:29:01.926 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:29:01.926 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:29:01.960 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:29:01.961 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:29:01.994 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:29:01.995 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:29:02.020 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:29:02.047 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:02.048 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:02.049 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:29:02.108 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:29:01 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:29:01] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:29:02.109 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:02 smithi070 ceph-mon[30388]: 5.6 scrub starts 2024-09-16T09:29:02.109 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:02 smithi070 ceph-mon[30388]: 5.6 scrub ok 2024-09-16T09:29:02.109 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:02 smithi070 ceph-mon[30388]: 5.9 scrub starts 2024-09-16T09:29:02.109 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:02 smithi070 ceph-mon[30388]: 5.9 scrub ok 2024-09-16T09:29:02.109 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:02 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:29:02.110 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:02 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:29:02.110 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:02 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3780013235' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:29:02.110 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:02 smithi070 ceph-mon[40691]: 5.6 scrub starts 2024-09-16T09:29:02.111 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:02 smithi070 ceph-mon[40691]: 5.6 scrub ok 2024-09-16T09:29:02.111 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:02 smithi070 ceph-mon[40691]: 5.9 scrub starts 2024-09-16T09:29:02.111 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:02 smithi070 ceph-mon[40691]: 5.9 scrub ok 2024-09-16T09:29:02.111 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:02 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:29:02.111 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:02 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:29:02.111 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:02 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3780013235' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:29:02.150 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:29:02.185 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:02.185 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:02.186 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:29:02.286 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:29:02.320 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:02.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:02.321 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:29:02.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:02 smithi084 ceph-mon[32629]: 5.6 scrub starts 2024-09-16T09:29:02.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:02 smithi084 ceph-mon[32629]: 5.6 scrub ok 2024-09-16T09:29:02.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:02 smithi084 ceph-mon[32629]: 5.9 scrub starts 2024-09-16T09:29:02.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:02 smithi084 ceph-mon[32629]: 5.9 scrub ok 2024-09-16T09:29:02.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:02 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:29:02.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:02 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:29:02.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:02 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3780013235' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:29:02.422 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:29:02.450 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:02.451 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:02.451 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:29:02.483 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.6 dump_historic_ops 2024-09-16T09:29:02.532 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:29:02.552 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:29:02.590 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:02.591 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:02.591 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:29:02.692 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:29:02.719 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:02.720 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:02.720 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:29:02.821 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:29:02.874 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:02.875 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:02.876 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:29:02.977 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:29:03.013 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:29:02 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:29:03.018 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:03.018 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:03.019 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:29:03.120 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:29:03.156 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:03.157 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:03.157 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:29:03.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:03 smithi070 ceph-mon[30388]: pgmap v174: 158 pgs: 158 active+clean; 2.7 GiB data, 9.0 GiB used, 706 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 626 op/s 2024-09-16T09:29:03.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:03 smithi070 ceph-mon[30388]: osdmap e140: 8 total, 8 up, 8 in 2024-09-16T09:29:03.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:03 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:29:03.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:03 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:29:03.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:03 smithi070 ceph-mon[30388]: 4.a scrub starts 2024-09-16T09:29:03.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:03 smithi070 ceph-mon[30388]: 4.a scrub ok 2024-09-16T09:29:03.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:03 smithi070 ceph-mon[30388]: 6.e scrub starts 2024-09-16T09:29:03.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:03 smithi070 ceph-mon[30388]: 6.e scrub ok 2024-09-16T09:29:03.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:03 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2072686144' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:29:03.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:03 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:29:03.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:03 smithi070 ceph-mon[40691]: pgmap v174: 158 pgs: 158 active+clean; 2.7 GiB data, 9.0 GiB used, 706 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 626 op/s 2024-09-16T09:29:03.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:03 smithi070 ceph-mon[40691]: osdmap e140: 8 total, 8 up, 8 in 2024-09-16T09:29:03.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:03 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:29:03.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:03 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:29:03.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:03 smithi070 ceph-mon[40691]: 4.a scrub starts 2024-09-16T09:29:03.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:03 smithi070 ceph-mon[40691]: 4.a scrub ok 2024-09-16T09:29:03.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:03 smithi070 ceph-mon[40691]: 6.e scrub starts 2024-09-16T09:29:03.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:03 smithi070 ceph-mon[40691]: 6.e scrub ok 2024-09-16T09:29:03.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:03 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/2072686144' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:29:03.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:03 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:29:03.258 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:29:03.284 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:03.285 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:03.285 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:29:03.356 INFO:teuthology.orchestra.run.smithi070.stderr:noscrub is set 2024-09-16T09:29:03.386 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:29:03.407 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:03.407 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:03.408 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:29:03.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:03 smithi084 ceph-mon[32629]: pgmap v174: 158 pgs: 158 active+clean; 2.7 GiB data, 9.0 GiB used, 706 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 626 op/s 2024-09-16T09:29:03.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:03 smithi084 ceph-mon[32629]: osdmap e140: 8 total, 8 up, 8 in 2024-09-16T09:29:03.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:03 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:29:03.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:03 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:29:03.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:03 smithi084 ceph-mon[32629]: 4.a scrub starts 2024-09-16T09:29:03.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:03 smithi084 ceph-mon[32629]: 4.a scrub ok 2024-09-16T09:29:03.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:03 smithi084 ceph-mon[32629]: 6.e scrub starts 2024-09-16T09:29:03.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:03 smithi084 ceph-mon[32629]: 6.e scrub ok 2024-09-16T09:29:03.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:03 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/2072686144' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:29:03.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:03 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:29:03.508 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:29:03.536 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:03.536 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:03.537 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:29:03.638 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:29:03.671 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:03.671 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:03.672 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:29:03.773 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:29:03.794 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:03.794 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:03.795 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:29:03.896 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:29:03.917 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:03.917 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:03.918 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:29:04.019 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:29:04.053 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:04.053 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:04.054 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:29:04.154 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:29:04.170 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:04 smithi070 ceph-mon[30388]: 4.6 scrub starts 2024-09-16T09:29:04.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:04 smithi070 ceph-mon[30388]: 4.6 scrub ok 2024-09-16T09:29:04.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:04 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:29:04.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:04 smithi070 ceph-mon[30388]: 1.0 scrub starts 2024-09-16T09:29:04.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:04 smithi070 ceph-mon[30388]: 1.0 scrub ok 2024-09-16T09:29:04.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:04 smithi070 ceph-mon[30388]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:29:04.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:04 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-16T09:29:04.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:04 smithi070 ceph-mon[30388]: osdmap e141: 8 total, 8 up, 8 in 2024-09-16T09:29:04.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:04 smithi070 ceph-mon[40691]: 4.6 scrub starts 2024-09-16T09:29:04.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:04 smithi070 ceph-mon[40691]: 4.6 scrub ok 2024-09-16T09:29:04.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:04 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:29:04.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:04 smithi070 ceph-mon[40691]: 1.0 scrub starts 2024-09-16T09:29:04.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:04 smithi070 ceph-mon[40691]: 1.0 scrub ok 2024-09-16T09:29:04.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:04 smithi070 ceph-mon[40691]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:29:04.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:04 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-16T09:29:04.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:04 smithi070 ceph-mon[40691]: osdmap e141: 8 total, 8 up, 8 in 2024-09-16T09:29:04.194 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:04.195 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:04.195 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:29:04.200 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-09-16T09:29:04.200 INFO:teuthology.orchestra.run.smithi084.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-16T09:29:04.200 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-09-16T09:29:04.296 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:29:04.320 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:04.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:04.321 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:29:04.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:04 smithi084 ceph-mon[32629]: 4.6 scrub starts 2024-09-16T09:29:04.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:04 smithi084 ceph-mon[32629]: 4.6 scrub ok 2024-09-16T09:29:04.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:04 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:29:04.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:04 smithi084 ceph-mon[32629]: 1.0 scrub starts 2024-09-16T09:29:04.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:04 smithi084 ceph-mon[32629]: 1.0 scrub ok 2024-09-16T09:29:04.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:04 smithi084 ceph-mon[32629]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:29:04.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:04 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-16T09:29:04.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:04 smithi084 ceph-mon[32629]: osdmap e141: 8 total, 8 up, 8 in 2024-09-16T09:29:04.423 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:29:04.449 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:04.450 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:04.450 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:29:04.509 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:29:04.551 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:29:04.576 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:04.577 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:04.577 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:29:04.678 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:29:04.716 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:04.717 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:04.717 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:29:04.818 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:29:04.844 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:04.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:04.845 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:29:04.892 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T09:29:04.946 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:29:04.973 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:04.973 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:04.974 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:29:05.069 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-09-16T09:29:05.069 INFO:teuthology.orchestra.run.smithi084.stdout: "size": 20, 2024-09-16T09:29:05.069 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 600, 2024-09-16T09:29:05.069 INFO:teuthology.orchestra.run.smithi084.stdout: "ops": [ 2024-09-16T09:29:05.070 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.070 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:60245 7.7 7:e5610e70:::benchmark_data_smithi070_78438_object60244:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-16T09:29:05.070 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:26:44.352114+0000", 2024-09-16T09:29:05.070 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 140.71424364399999, 2024-09-16T09:29:05.070 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.127497575, 2024-09-16T09:29:05.070 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:29:05.070 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:29:05.070 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:29:05.070 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:29:05.070 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:29:05.070 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:29:05.070 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 60245 2024-09-16T09:29:05.071 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.071 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:29:05.071 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.071 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:29:05.071 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.352114+0000", 2024-09-16T09:29:05.071 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:29:05.071 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.071 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.071 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:29:05.071 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.352114+0000", 2024-09-16T09:29:05.071 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:29:05.071 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.071 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.071 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:29:05.072 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.352115+0000", 2024-09-16T09:29:05.072 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0449999999999999e-06 2024-09-16T09:29:05.072 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.072 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.072 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:29:05.072 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.352162+0000", 2024-09-16T09:29:05.072 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.7416000000000001e-05 2024-09-16T09:29:05.072 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.072 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.072 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:29:05.072 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.352174+0000", 2024-09-16T09:29:05.072 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.2347999999999999e-05 2024-09-16T09:29:05.072 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.073 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.073 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:29:05.073 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.352181+0000", 2024-09-16T09:29:05.073 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.8809999999999998e-06 2024-09-16T09:29:05.073 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.073 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.073 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:29:05.073 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.352231+0000", 2024-09-16T09:29:05.073 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.9292000000000002e-05 2024-09-16T09:29:05.073 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.073 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.073 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:29:05.074 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.352373+0000", 2024-09-16T09:29:05.074 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00014240699999999999 2024-09-16T09:29:05.074 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.074 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.074 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,3", 2024-09-16T09:29:05.074 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.352494+0000", 2024-09-16T09:29:05.074 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000121039 2024-09-16T09:29:05.074 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.074 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.074 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:29:05.074 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.354404+0000", 2024-09-16T09:29:05.074 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0019098979999999999 2024-09-16T09:29:05.074 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.074 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.075 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:05.075 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.355369+0000", 2024-09-16T09:29:05.075 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00096560899999999998 2024-09-16T09:29:05.075 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.075 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.075 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:05.075 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.479497+0000", 2024-09-16T09:29:05.075 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.124127398 2024-09-16T09:29:05.075 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.075 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.075 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:29:05.075 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.479559+0000", 2024-09-16T09:29:05.075 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.1692000000000005e-05 2024-09-16T09:29:05.075 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.075 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.076 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:29:05.076 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.479611+0000", 2024-09-16T09:29:05.076 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.2549999999999997e-05 2024-09-16T09:29:05.076 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:29:05.076 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:29:05.076 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:29:05.076 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.076 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.076 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:60246 7.d 7:b1444fdd:::benchmark_data_smithi070_78438_object60245:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-16T09:29:05.076 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:26:44.352405+0000", 2024-09-16T09:29:05.076 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 140.713952253, 2024-09-16T09:29:05.076 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.13626988500000001, 2024-09-16T09:29:05.076 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:29:05.076 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:29:05.077 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:29:05.077 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:29:05.077 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:29:05.077 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:29:05.077 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 60246 2024-09-16T09:29:05.077 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.077 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:29:05.077 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.077 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:29:05.077 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.352405+0000", 2024-09-16T09:29:05.077 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:29:05.077 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.077 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.078 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:29:05.078 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.352405+0000", 2024-09-16T09:29:05.078 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:29:05.078 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.078 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.078 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:29:05.078 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.352406+0000", 2024-09-16T09:29:05.078 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.85e-07 2024-09-16T09:29:05.078 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.078 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.078 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:29:05.078 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.352431+0000", 2024-09-16T09:29:05.078 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.5364e-05 2024-09-16T09:29:05.078 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.079 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.079 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:29:05.079 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.352440+0000", 2024-09-16T09:29:05.079 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.9509999999999995e-06 2024-09-16T09:29:05.079 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.079 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.079 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:29:05.079 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.352446+0000", 2024-09-16T09:29:05.079 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.2099999999999998e-06 2024-09-16T09:29:05.079 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.079 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.079 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:29:05.079 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.352486+0000", 2024-09-16T09:29:05.079 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.9475999999999999e-05 2024-09-16T09:29:05.080 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.080 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.080 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:29:05.080 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.352575+0000", 2024-09-16T09:29:05.080 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.9239000000000004e-05 2024-09-16T09:29:05.080 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.080 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.080 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,1", 2024-09-16T09:29:05.080 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.352693+0000", 2024-09-16T09:29:05.080 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000118248 2024-09-16T09:29:05.080 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.080 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.080 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:29:05.081 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.354498+0000", 2024-09-16T09:29:05.081 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0018052890000000001 2024-09-16T09:29:05.081 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.081 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.081 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:05.081 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.355812+0000", 2024-09-16T09:29:05.081 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0013137380000000001 2024-09-16T09:29:05.081 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.081 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.081 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:05.081 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.488574+0000", 2024-09-16T09:29:05.081 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.13276227400000001 2024-09-16T09:29:05.082 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.082 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.082 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:29:05.082 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.488632+0000", 2024-09-16T09:29:05.082 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.7970999999999998e-05 2024-09-16T09:29:05.082 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.082 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.082 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:29:05.082 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.488675+0000", 2024-09-16T09:29:05.082 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.2339999999999998e-05 2024-09-16T09:29:05.082 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:29:05.082 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:29:05.082 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:29:05.083 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.083 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.083 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:60253 7.d 7:bc8ca5cf:::benchmark_data_smithi070_78438_object60252:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-16T09:29:05.083 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:26:44.357181+0000", 2024-09-16T09:29:05.083 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 140.70917661799999, 2024-09-16T09:29:05.083 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.131612381, 2024-09-16T09:29:05.083 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:29:05.083 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:29:05.083 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:29:05.083 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:29:05.084 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:29:05.084 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:29:05.085 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:29:05.085 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 60253 2024-09-16T09:29:05.085 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.085 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:29:05.085 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.085 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:29:05.085 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.357181+0000", 2024-09-16T09:29:05.085 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:29:05.086 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.086 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.086 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:29:05.086 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.357181+0000", 2024-09-16T09:29:05.086 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:29:05.086 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.086 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.086 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:29:05.086 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.357182+0000", 2024-09-16T09:29:05.086 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.2270000000000001e-06 2024-09-16T09:29:05.086 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.086 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.087 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:29:05.087 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.357244+0000", 2024-09-16T09:29:05.087 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.2683999999999998e-05 2024-09-16T09:29:05.087 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.087 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.087 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:29:05.087 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.357255+0000", 2024-09-16T09:29:05.087 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0720000000000001e-05 2024-09-16T09:29:05.087 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.087 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.088 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:29:05.088 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.357262+0000", 2024-09-16T09:29:05.088 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.7800000000000003e-06 2024-09-16T09:29:05.088 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.088 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.088 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:29:05.088 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.357305+0000", 2024-09-16T09:29:05.088 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.2667999999999999e-05 2024-09-16T09:29:05.088 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.088 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.088 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:29:05.089 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.357399+0000", 2024-09-16T09:29:05.089 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.4217999999999998e-05 2024-09-16T09:29:05.089 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.089 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.089 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,1", 2024-09-16T09:29:05.089 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.357492+0000", 2024-09-16T09:29:05.089 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.3213000000000004e-05 2024-09-16T09:29:05.089 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.089 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.089 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:29:05.089 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.358931+0000", 2024-09-16T09:29:05.089 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.001439357 2024-09-16T09:29:05.089 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.090 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.090 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:05.090 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.359185+0000", 2024-09-16T09:29:05.090 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00025320799999999999 2024-09-16T09:29:05.090 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.090 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.090 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:05.090 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.488722+0000", 2024-09-16T09:29:05.090 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.12953768399999999 2024-09-16T09:29:05.090 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.091 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.091 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:29:05.091 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.488763+0000", 2024-09-16T09:29:05.091 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.1059000000000001e-05 2024-09-16T09:29:05.091 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.091 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.091 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:29:05.091 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.488793+0000", 2024-09-16T09:29:05.091 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.9563000000000001e-05 2024-09-16T09:29:05.091 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:29:05.091 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:29:05.091 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:29:05.092 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.092 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.092 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:86887 7.c 7:347431d3:::benchmark_data_smithi070_78438_object86886:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-16T09:29:05.092 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:27:06.134927+0000", 2024-09-16T09:29:05.092 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 118.931430194, 2024-09-16T09:29:05.092 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.088112020999999999, 2024-09-16T09:29:05.093 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:29:05.093 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:29:05.093 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:29:05.093 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:29:05.093 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:29:05.093 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:29:05.093 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 86887 2024-09-16T09:29:05.093 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.094 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:29:05.094 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.094 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:29:05.094 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.134927+0000", 2024-09-16T09:29:05.094 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:29:05.094 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.094 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.095 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:29:05.095 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.134927+0000", 2024-09-16T09:29:05.095 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:29:05.095 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.095 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.095 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:29:05.095 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.134928+0000", 2024-09-16T09:29:05.095 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.522e-06 2024-09-16T09:29:05.095 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.095 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.096 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:29:05.096 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.134975+0000", 2024-09-16T09:29:05.096 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.6757e-05 2024-09-16T09:29:05.096 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.096 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.096 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:29:05.096 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.134986+0000", 2024-09-16T09:29:05.096 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0477e-05 2024-09-16T09:29:05.096 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.096 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.096 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:29:05.097 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.134993+0000", 2024-09-16T09:29:05.097 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.0149999999999998e-06 2024-09-16T09:29:05.097 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.097 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.097 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:29:05.097 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.135043+0000", 2024-09-16T09:29:05.097 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.9991999999999998e-05 2024-09-16T09:29:05.097 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.097 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.097 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:29:05.097 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.135144+0000", 2024-09-16T09:29:05.097 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000100923 2024-09-16T09:29:05.097 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.097 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.098 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 3,7", 2024-09-16T09:29:05.098 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.135239+0000", 2024-09-16T09:29:05.098 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.5573000000000001e-05 2024-09-16T09:29:05.098 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.098 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.098 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:05.098 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.139237+0000", 2024-09-16T09:29:05.098 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0039979309999999997 2024-09-16T09:29:05.098 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.098 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.098 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:29:05.098 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.142141+0000", 2024-09-16T09:29:05.098 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0029041409999999998 2024-09-16T09:29:05.098 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.099 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.099 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:05.099 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.222928+0000", 2024-09-16T09:29:05.099 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.080786812999999999 2024-09-16T09:29:05.099 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.099 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.099 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:29:05.099 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.222989+0000", 2024-09-16T09:29:05.099 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.0958999999999997e-05 2024-09-16T09:29:05.099 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.100 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.100 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:29:05.100 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.223039+0000", 2024-09-16T09:29:05.100 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.9917999999999999e-05 2024-09-16T09:29:05.100 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:29:05.100 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:29:05.100 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:29:05.100 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.100 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.100 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:86896 7.c 7:341d906e:::benchmark_data_smithi070_78438_object86895:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-16T09:29:05.100 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:27:06.139141+0000", 2024-09-16T09:29:05.101 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 118.927215852, 2024-09-16T09:29:05.101 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.091441622, 2024-09-16T09:29:05.101 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:29:05.101 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:29:05.101 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:29:05.101 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:29:05.101 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:29:05.101 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:29:05.101 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 86896 2024-09-16T09:29:05.101 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.101 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:29:05.101 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.102 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:29:05.102 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.139141+0000", 2024-09-16T09:29:05.102 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:29:05.102 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.102 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.102 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:29:05.102 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.139141+0000", 2024-09-16T09:29:05.102 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:29:05.102 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.102 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.102 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:29:05.103 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.139143+0000", 2024-09-16T09:29:05.103 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.248e-06 2024-09-16T09:29:05.103 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.103 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.103 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:29:05.103 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.139165+0000", 2024-09-16T09:29:05.103 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.2052000000000002e-05 2024-09-16T09:29:05.103 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.103 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.103 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:29:05.103 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.139174+0000", 2024-09-16T09:29:05.103 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.6439999999999995e-06 2024-09-16T09:29:05.103 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.103 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.104 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:29:05.104 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.139181+0000", 2024-09-16T09:29:05.104 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.7340000000000002e-06 2024-09-16T09:29:05.104 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.104 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.104 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:29:05.104 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.139267+0000", 2024-09-16T09:29:05.104 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.5870000000000003e-05 2024-09-16T09:29:05.104 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.104 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.104 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:29:05.105 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.139433+0000", 2024-09-16T09:29:05.105 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00016583400000000001 2024-09-16T09:29:05.105 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.105 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.105 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 3,7", 2024-09-16T09:29:05.105 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.139576+0000", 2024-09-16T09:29:05.105 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00014307799999999999 2024-09-16T09:29:05.105 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.105 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.105 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:29:05.105 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.142331+0000", 2024-09-16T09:29:05.105 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0027555209999999999 2024-09-16T09:29:05.105 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.105 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.106 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:05.106 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.144623+0000", 2024-09-16T09:29:05.106 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0022914699999999999 2024-09-16T09:29:05.106 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.106 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.106 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:05.106 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.230508+0000", 2024-09-16T09:29:05.106 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.085885662000000002 2024-09-16T09:29:05.106 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.106 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.106 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:29:05.106 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.230545+0000", 2024-09-16T09:29:05.106 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.6637999999999997e-05 2024-09-16T09:29:05.107 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.107 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.107 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:29:05.107 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.230583+0000", 2024-09-16T09:29:05.107 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.7870999999999997e-05 2024-09-16T09:29:05.107 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:29:05.107 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:29:05.107 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:29:05.107 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.107 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.107 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:202388 7.7 7:ed754f08:::benchmark_data_smithi070_78438_object76257:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135)", 2024-09-16T09:29:05.107 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:28:50.710298+0000", 2024-09-16T09:29:05.107 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 14.356059278, 2024-09-16T09:29:05.108 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.097457902999999999, 2024-09-16T09:29:05.108 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:29:05.108 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:29:05.108 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:29:05.108 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:29:05.108 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:29:05.108 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:29:05.108 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 202388 2024-09-16T09:29:05.108 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.108 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:29:05.108 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.109 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:29:05.109 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.710298+0000", 2024-09-16T09:29:05.109 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:29:05.109 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.109 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.109 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:29:05.109 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.710298+0000", 2024-09-16T09:29:05.109 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:29:05.109 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.109 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.109 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:29:05.109 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.710299+0000", 2024-09-16T09:29:05.109 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.9000000000000005e-07 2024-09-16T09:29:05.110 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.110 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.110 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:29:05.110 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.710303+0000", 2024-09-16T09:29:05.110 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.924e-06 2024-09-16T09:29:05.110 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.110 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.110 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:29:05.110 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.710311+0000", 2024-09-16T09:29:05.110 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.4290000000000001e-06 2024-09-16T09:29:05.110 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.110 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.111 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:29:05.111 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.710316+0000", 2024-09-16T09:29:05.111 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.4750000000000004e-06 2024-09-16T09:29:05.111 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.111 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.111 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:29:05.111 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.710363+0000", 2024-09-16T09:29:05.111 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.7332e-05 2024-09-16T09:29:05.111 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.111 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.111 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:29:05.111 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.710467+0000", 2024-09-16T09:29:05.111 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000104288 2024-09-16T09:29:05.111 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.112 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.112 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,3", 2024-09-16T09:29:05.112 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.710569+0000", 2024-09-16T09:29:05.112 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000101499 2024-09-16T09:29:05.112 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.112 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.112 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:29:05.112 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.713003+0000", 2024-09-16T09:29:05.112 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0024343059999999998 2024-09-16T09:29:05.112 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.112 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.112 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:05.112 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.807079+0000", 2024-09-16T09:29:05.113 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.094075385999999997 2024-09-16T09:29:05.113 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.113 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.113 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:05.113 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.807669+0000", 2024-09-16T09:29:05.113 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00059051199999999996 2024-09-16T09:29:05.113 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.113 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.113 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:29:05.113 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.807717+0000", 2024-09-16T09:29:05.113 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.7716999999999997e-05 2024-09-16T09:29:05.113 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.113 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.113 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:29:05.114 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.807756+0000", 2024-09-16T09:29:05.114 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.9045000000000003e-05 2024-09-16T09:29:05.114 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:29:05.114 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:29:05.114 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:29:05.114 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.114 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.114 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:202394 7.7 7:e0c03bf8:::benchmark_data_smithi070_78438_object76263:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135)", 2024-09-16T09:29:05.114 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:28:50.713789+0000", 2024-09-16T09:29:05.114 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 14.352568146999999, 2024-09-16T09:29:05.114 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.094102293000000004, 2024-09-16T09:29:05.114 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:29:05.114 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:29:05.115 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:29:05.115 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:29:05.115 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:29:05.115 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:29:05.115 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 202394 2024-09-16T09:29:05.115 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.115 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:29:05.115 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.115 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:29:05.115 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.713789+0000", 2024-09-16T09:29:05.115 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:29:05.115 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.116 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.116 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:29:05.116 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.713789+0000", 2024-09-16T09:29:05.116 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:29:05.116 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.116 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.116 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:29:05.116 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.713790+0000", 2024-09-16T09:29:05.116 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.9000000000000005e-07 2024-09-16T09:29:05.116 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.116 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.116 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:29:05.116 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.713794+0000", 2024-09-16T09:29:05.116 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.8770000000000003e-06 2024-09-16T09:29:05.117 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.117 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.117 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:29:05.117 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.713803+0000", 2024-09-16T09:29:05.117 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.6619999999999994e-06 2024-09-16T09:29:05.117 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.117 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.117 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:29:05.117 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.713807+0000", 2024-09-16T09:29:05.117 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.9049999999999996e-06 2024-09-16T09:29:05.117 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.117 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.117 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:29:05.117 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.713841+0000", 2024-09-16T09:29:05.117 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.3575999999999998e-05 2024-09-16T09:29:05.118 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.118 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.118 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:29:05.118 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.713919+0000", 2024-09-16T09:29:05.118 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.8446000000000001e-05 2024-09-16T09:29:05.118 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.118 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.118 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,3", 2024-09-16T09:29:05.118 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.713983+0000", 2024-09-16T09:29:05.118 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.3911000000000005e-05 2024-09-16T09:29:05.118 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.118 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.118 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:29:05.119 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.720846+0000", 2024-09-16T09:29:05.119 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0068624310000000004 2024-09-16T09:29:05.119 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.119 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.119 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:05.119 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.807591+0000", 2024-09-16T09:29:05.119 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.086744842000000003 2024-09-16T09:29:05.119 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.119 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.119 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:05.119 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.807817+0000", 2024-09-16T09:29:05.119 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00022680500000000001 2024-09-16T09:29:05.119 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.119 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.119 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:29:05.119 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.807856+0000", 2024-09-16T09:29:05.120 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.8909999999999998e-05 2024-09-16T09:29:05.120 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.120 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.120 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:29:05.120 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.807891+0000", 2024-09-16T09:29:05.120 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.4938000000000003e-05 2024-09-16T09:29:05.120 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:29:05.120 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:29:05.120 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:29:05.120 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.120 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.120 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:202395 7.7 7:e38f3473:::benchmark_data_smithi070_78438_object76264:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135)", 2024-09-16T09:29:05.120 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:28:50.713818+0000", 2024-09-16T09:29:05.120 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 14.35253874, 2024-09-16T09:29:05.120 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.094181156000000002, 2024-09-16T09:29:05.121 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:29:05.121 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:29:05.121 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:29:05.121 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:29:05.121 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:29:05.121 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:29:05.121 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 202395 2024-09-16T09:29:05.121 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.121 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:29:05.121 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.121 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:29:05.121 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.713818+0000", 2024-09-16T09:29:05.121 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:29:05.121 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.122 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.122 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:29:05.122 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.713818+0000", 2024-09-16T09:29:05.122 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:29:05.122 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.122 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.122 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:29:05.122 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.713819+0000", 2024-09-16T09:29:05.122 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.0499999999999999e-07 2024-09-16T09:29:05.122 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.122 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.123 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:29:05.123 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.713820+0000", 2024-09-16T09:29:05.123 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.4980000000000001e-06 2024-09-16T09:29:05.123 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.123 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.123 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:29:05.123 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.713824+0000", 2024-09-16T09:29:05.123 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.5130000000000001e-06 2024-09-16T09:29:05.123 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.123 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.124 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:29:05.124 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.713827+0000", 2024-09-16T09:29:05.124 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.1590000000000002e-06 2024-09-16T09:29:05.124 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.124 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.124 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:29:05.124 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.716144+0000", 2024-09-16T09:29:05.124 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0023175159999999999 2024-09-16T09:29:05.124 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.124 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.124 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:29:05.124 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.716246+0000", 2024-09-16T09:29:05.124 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000101902 2024-09-16T09:29:05.124 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.125 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.125 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,3", 2024-09-16T09:29:05.125 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.716348+0000", 2024-09-16T09:29:05.125 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000102099 2024-09-16T09:29:05.125 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.126 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.126 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:29:05.126 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.720849+0000", 2024-09-16T09:29:05.126 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0045008069999999999 2024-09-16T09:29:05.126 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.126 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.127 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:05.127 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.807634+0000", 2024-09-16T09:29:05.127 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.086784617999999994 2024-09-16T09:29:05.127 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.127 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.127 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:05.128 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.807928+0000", 2024-09-16T09:29:05.128 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00029464299999999998 2024-09-16T09:29:05.128 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.128 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.128 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:29:05.128 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.807969+0000", 2024-09-16T09:29:05.128 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.0917999999999997e-05 2024-09-16T09:29:05.128 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.128 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.128 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:29:05.128 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.808000+0000", 2024-09-16T09:29:05.128 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.0178000000000001e-05 2024-09-16T09:29:05.129 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:29:05.129 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:29:05.129 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:29:05.129 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.129 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.129 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:202398 7.7 7:ed5121df:::benchmark_data_smithi070_78438_object76267:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135)", 2024-09-16T09:29:05.129 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:28:50.713837+0000", 2024-09-16T09:29:05.129 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 14.352520116999999, 2024-09-16T09:29:05.129 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.094273179999999998, 2024-09-16T09:29:05.129 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:29:05.129 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:29:05.129 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:29:05.129 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:29:05.129 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:29:05.130 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:29:05.130 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 202398 2024-09-16T09:29:05.130 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.130 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:29:05.130 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.130 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:29:05.130 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.713837+0000", 2024-09-16T09:29:05.130 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:29:05.130 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.130 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.130 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:29:05.130 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.713837+0000", 2024-09-16T09:29:05.130 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:29:05.130 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.130 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.130 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:29:05.131 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.713837+0000", 2024-09-16T09:29:05.131 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.2100000000000001e-07 2024-09-16T09:29:05.131 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.131 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.131 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:29:05.131 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.713839+0000", 2024-09-16T09:29:05.131 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.297e-06 2024-09-16T09:29:05.131 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.131 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.131 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:29:05.131 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.713842+0000", 2024-09-16T09:29:05.131 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.4350000000000001e-06 2024-09-16T09:29:05.131 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.131 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.131 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:29:05.131 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.713845+0000", 2024-09-16T09:29:05.131 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.8430000000000001e-06 2024-09-16T09:29:05.132 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.132 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.132 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:29:05.132 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.718636+0000", 2024-09-16T09:29:05.132 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0047914630000000001 2024-09-16T09:29:05.132 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.132 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.132 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:29:05.132 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.718741+0000", 2024-09-16T09:29:05.132 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000104498 2024-09-16T09:29:05.132 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.132 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.132 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,3", 2024-09-16T09:29:05.134 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.718844+0000", 2024-09-16T09:29:05.134 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00010359800000000001 2024-09-16T09:29:05.134 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.134 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.134 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:29:05.134 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.720976+0000", 2024-09-16T09:29:05.134 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0021316080000000001 2024-09-16T09:29:05.134 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.134 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.134 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:05.134 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.807788+0000", 2024-09-16T09:29:05.134 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.086811865000000002 2024-09-16T09:29:05.134 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.134 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.135 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:05.135 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.808037+0000", 2024-09-16T09:29:05.135 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000248905 2024-09-16T09:29:05.135 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.135 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.135 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:29:05.135 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.808077+0000", 2024-09-16T09:29:05.135 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.0547000000000003e-05 2024-09-16T09:29:05.135 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.135 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.135 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:29:05.135 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.808110+0000", 2024-09-16T09:29:05.135 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.29e-05 2024-09-16T09:29:05.135 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:29:05.135 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:29:05.136 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:29:05.136 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.136 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.136 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:202402 7.c 7:386004fd:::benchmark_data_smithi070_78438_object76271:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135)", 2024-09-16T09:29:05.136 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:28:50.720561+0000", 2024-09-16T09:29:05.136 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 14.345795990999999, 2024-09-16T09:29:05.136 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.087074283000000002, 2024-09-16T09:29:05.136 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:29:05.136 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:29:05.136 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:29:05.136 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:29:05.136 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:29:05.136 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:29:05.136 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 202402 2024-09-16T09:29:05.137 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.137 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:29:05.137 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.137 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:29:05.137 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.720561+0000", 2024-09-16T09:29:05.137 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:29:05.137 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.137 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.137 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:29:05.137 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.720561+0000", 2024-09-16T09:29:05.137 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:29:05.137 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.137 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.137 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:29:05.137 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.720562+0000", 2024-09-16T09:29:05.138 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0109999999999999e-06 2024-09-16T09:29:05.138 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.138 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.138 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:29:05.138 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.720567+0000", 2024-09-16T09:29:05.138 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.3460000000000004e-06 2024-09-16T09:29:05.138 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.138 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.138 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:29:05.138 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.720575+0000", 2024-09-16T09:29:05.138 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.5569999999999996e-06 2024-09-16T09:29:05.138 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.138 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.139 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:29:05.139 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.720580+0000", 2024-09-16T09:29:05.139 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.091e-06 2024-09-16T09:29:05.139 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.139 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.139 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:29:05.139 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.720614+0000", 2024-09-16T09:29:05.139 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.3331999999999999e-05 2024-09-16T09:29:05.139 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.139 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.139 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:29:05.139 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.720684+0000", 2024-09-16T09:29:05.139 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.0964999999999996e-05 2024-09-16T09:29:05.139 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.139 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.139 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 3,7", 2024-09-16T09:29:05.139 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.720745+0000", 2024-09-16T09:29:05.140 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.0309999999999997e-05 2024-09-16T09:29:05.140 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.140 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.140 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:29:05.140 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.722990+0000", 2024-09-16T09:29:05.140 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0022451810000000002 2024-09-16T09:29:05.140 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.140 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.140 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:05.140 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.723555+0000", 2024-09-16T09:29:05.140 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00056466999999999997 2024-09-16T09:29:05.140 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.140 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.141 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:05.141 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.807545+0000", 2024-09-16T09:29:05.141 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.083990597 2024-09-16T09:29:05.141 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.141 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.141 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:29:05.141 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.807594+0000", 2024-09-16T09:29:05.141 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.8931999999999998e-05 2024-09-16T09:29:05.141 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.141 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.141 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:29:05.141 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.807635+0000", 2024-09-16T09:29:05.141 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.1291000000000002e-05 2024-09-16T09:29:05.142 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:29:05.142 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:29:05.142 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:29:05.142 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.142 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.142 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:203433 7.c 7:3eef1e44:::benchmark_data_smithi070_78438_object77302:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135)", 2024-09-16T09:29:05.142 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:28:51.980722+0000", 2024-09-16T09:29:05.142 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 13.0856355, 2024-09-16T09:29:05.142 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.14639702299999999, 2024-09-16T09:29:05.142 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:29:05.143 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:29:05.143 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:29:05.143 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:29:05.143 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:29:05.143 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:29:05.143 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 203433 2024-09-16T09:29:05.143 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.143 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:29:05.143 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.143 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:29:05.143 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.980722+0000", 2024-09-16T09:29:05.143 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:29:05.143 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.143 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.144 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:29:05.144 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.980722+0000", 2024-09-16T09:29:05.144 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:29:05.144 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.144 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.144 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:29:05.144 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.980723+0000", 2024-09-16T09:29:05.144 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.2559999999999999e-06 2024-09-16T09:29:05.144 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.144 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.144 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:29:05.144 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.980727+0000", 2024-09-16T09:29:05.144 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.9990000000000002e-06 2024-09-16T09:29:05.144 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.145 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.145 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:29:05.145 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.980742+0000", 2024-09-16T09:29:05.145 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.4761e-05 2024-09-16T09:29:05.145 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.145 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.145 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:29:05.145 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.980750+0000", 2024-09-16T09:29:05.145 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.9389999999999997e-06 2024-09-16T09:29:05.145 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.145 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.145 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:29:05.145 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.980831+0000", 2024-09-16T09:29:05.145 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.1773999999999993e-05 2024-09-16T09:29:05.146 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.146 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.146 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:29:05.146 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.980990+0000", 2024-09-16T09:29:05.146 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00015910100000000001 2024-09-16T09:29:05.146 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.146 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.146 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 3,7", 2024-09-16T09:29:05.146 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.981083+0000", 2024-09-16T09:29:05.146 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.2274000000000005e-05 2024-09-16T09:29:05.146 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.146 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.146 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:29:05.146 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.984389+0000", 2024-09-16T09:29:05.146 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.003306504 2024-09-16T09:29:05.147 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.147 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.147 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:05.147 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.985173+0000", 2024-09-16T09:29:05.147 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00078339399999999997 2024-09-16T09:29:05.147 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.147 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.147 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:05.147 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:52.127014+0000", 2024-09-16T09:29:05.147 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.14184139400000001 2024-09-16T09:29:05.147 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.147 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.147 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:29:05.147 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:52.127071+0000", 2024-09-16T09:29:05.148 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.6552999999999999e-05 2024-09-16T09:29:05.148 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.148 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.148 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:29:05.148 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:52.127119+0000", 2024-09-16T09:29:05.148 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.8074000000000001e-05 2024-09-16T09:29:05.148 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:29:05.148 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:29:05.148 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:29:05.148 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.148 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.148 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:203435 7.c 7:3a18a4c7:::benchmark_data_smithi070_78438_object77304:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136)", 2024-09-16T09:29:05.148 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:28:51.987097+0000", 2024-09-16T09:29:05.148 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 13.079259913, 2024-09-16T09:29:05.149 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.14016357900000001, 2024-09-16T09:29:05.149 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:29:05.149 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:29:05.149 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:29:05.149 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:29:05.149 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:29:05.149 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:29:05.149 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 203435 2024-09-16T09:29:05.149 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.149 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:29:05.149 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.149 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:29:05.149 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.987097+0000", 2024-09-16T09:29:05.149 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:29:05.149 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.150 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.150 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:29:05.150 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.987097+0000", 2024-09-16T09:29:05.150 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:29:05.150 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.150 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.150 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:29:05.150 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.987106+0000", 2024-09-16T09:29:05.150 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.4840000000000004e-06 2024-09-16T09:29:05.150 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.150 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.150 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:29:05.150 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.987110+0000", 2024-09-16T09:29:05.150 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.053e-06 2024-09-16T09:29:05.150 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.151 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.151 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:29:05.151 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.987118+0000", 2024-09-16T09:29:05.151 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.5730000000000008e-06 2024-09-16T09:29:05.151 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.151 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.151 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:29:05.151 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.987125+0000", 2024-09-16T09:29:05.151 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.6220000000000003e-06 2024-09-16T09:29:05.151 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.151 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.151 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:29:05.151 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.987168+0000", 2024-09-16T09:29:05.151 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.2617999999999998e-05 2024-09-16T09:29:05.151 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.151 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.152 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:29:05.152 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.987308+0000", 2024-09-16T09:29:05.152 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00014014499999999999 2024-09-16T09:29:05.152 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.152 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.152 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 3,7", 2024-09-16T09:29:05.152 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.987386+0000", 2024-09-16T09:29:05.152 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.8529999999999995e-05 2024-09-16T09:29:05.152 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.152 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.152 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:29:05.152 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.989686+0000", 2024-09-16T09:29:05.152 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0023000989999999999 2024-09-16T09:29:05.153 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.153 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.153 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:05.153 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.990324+0000", 2024-09-16T09:29:05.153 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00063796899999999995 2024-09-16T09:29:05.153 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.153 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.153 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:05.153 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:52.127169+0000", 2024-09-16T09:29:05.153 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.13684418100000001 2024-09-16T09:29:05.153 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.153 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.153 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:29:05.153 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:52.127222+0000", 2024-09-16T09:29:05.154 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.3458000000000002e-05 2024-09-16T09:29:05.154 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.154 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.154 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:29:05.154 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:52.127261+0000", 2024-09-16T09:29:05.154 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.8847000000000003e-05 2024-09-16T09:29:05.154 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:29:05.154 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:29:05.154 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:29:05.154 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.154 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.154 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:215668 7.c 7:31105ff3:::benchmark_data_smithi070_78438_object89537:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e141)", 2024-09-16T09:29:05.154 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:29:03.671611+0000", 2024-09-16T09:29:05.154 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 1.3947463680000001, 2024-09-16T09:29:05.154 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.254303314, 2024-09-16T09:29:05.154 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:29:05.154 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:29:05.155 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:29:05.155 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:29:05.155 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:29:05.155 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:29:05.155 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 215668 2024-09-16T09:29:05.155 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.155 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:29:05.155 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.155 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:29:05.155 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.671611+0000", 2024-09-16T09:29:05.155 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:29:05.155 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.155 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.155 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:29:05.155 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.671611+0000", 2024-09-16T09:29:05.155 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:29:05.156 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.156 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.156 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:29:05.156 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.671612+0000", 2024-09-16T09:29:05.156 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1379999999999999e-06 2024-09-16T09:29:05.156 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.156 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.156 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:29:05.156 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.671617+0000", 2024-09-16T09:29:05.156 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.5959999999999997e-06 2024-09-16T09:29:05.156 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.156 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.156 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:29:05.157 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.721873+0000", 2024-09-16T09:29:05.157 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.050256728000000001 2024-09-16T09:29:05.157 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.157 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.157 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:29:05.157 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.721883+0000", 2024-09-16T09:29:05.157 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.8730000000000002e-06 2024-09-16T09:29:05.157 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.157 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.157 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:29:05.157 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.722610+0000", 2024-09-16T09:29:05.157 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000726449 2024-09-16T09:29:05.157 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.157 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.157 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:29:05.158 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.722746+0000", 2024-09-16T09:29:05.158 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000136231 2024-09-16T09:29:05.158 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.158 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.158 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 3,7", 2024-09-16T09:29:05.158 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.722862+0000", 2024-09-16T09:29:05.158 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000116125 2024-09-16T09:29:05.158 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.158 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.158 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:05.158 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.925722+0000", 2024-09-16T09:29:05.158 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.20286009799999999 2024-09-16T09:29:05.158 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.158 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.158 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:29:05.158 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.925733+0000", 2024-09-16T09:29:05.159 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0635000000000001e-05 2024-09-16T09:29:05.159 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.159 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.159 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:05.159 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.925820+0000", 2024-09-16T09:29:05.159 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.7356999999999997e-05 2024-09-16T09:29:05.159 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.159 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.159 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:29:05.159 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.925868+0000", 2024-09-16T09:29:05.159 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.7519999999999999e-05 2024-09-16T09:29:05.159 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.159 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.159 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:29:05.159 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.925914+0000", 2024-09-16T09:29:05.159 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.6563999999999997e-05 2024-09-16T09:29:05.159 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:29:05.160 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:29:05.160 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:29:05.160 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.160 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.160 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:215673 7.c 7:37240b0f:::benchmark_data_smithi070_78438_object89542:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e141)", 2024-09-16T09:29:05.160 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:29:03.721919+0000", 2024-09-16T09:29:05.160 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 1.3444385969999999, 2024-09-16T09:29:05.160 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.20416785100000001, 2024-09-16T09:29:05.160 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:29:05.160 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:29:05.160 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:29:05.160 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:29:05.160 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:29:05.160 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:29:05.160 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 215673 2024-09-16T09:29:05.161 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.161 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:29:05.161 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.161 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:29:05.161 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.721919+0000", 2024-09-16T09:29:05.161 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:29:05.161 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.161 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.161 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:29:05.161 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.721919+0000", 2024-09-16T09:29:05.161 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:29:05.161 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.161 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.161 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:29:05.161 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.721920+0000", 2024-09-16T09:29:05.161 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.373e-06 2024-09-16T09:29:05.162 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.162 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.162 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:29:05.162 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.721925+0000", 2024-09-16T09:29:05.162 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.6789999999999996e-06 2024-09-16T09:29:05.162 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.162 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.162 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:29:05.162 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.721934+0000", 2024-09-16T09:29:05.162 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.0489999999999993e-06 2024-09-16T09:29:05.162 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.162 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.162 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:29:05.162 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.721939+0000", 2024-09-16T09:29:05.162 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.0080000000000002e-06 2024-09-16T09:29:05.163 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.163 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.163 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:29:05.163 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.725737+0000", 2024-09-16T09:29:05.163 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0037982279999999998 2024-09-16T09:29:05.163 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.163 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.163 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:29:05.163 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.725857+0000", 2024-09-16T09:29:05.163 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00012021 2024-09-16T09:29:05.163 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.163 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.163 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 3,7", 2024-09-16T09:29:05.163 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.725981+0000", 2024-09-16T09:29:05.163 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000124282 2024-09-16T09:29:05.163 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.164 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.164 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:29:05.164 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.925739+0000", 2024-09-16T09:29:05.164 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.19975721499999999 2024-09-16T09:29:05.164 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.164 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.164 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:05.164 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.925767+0000", 2024-09-16T09:29:05.164 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.8090999999999999e-05 2024-09-16T09:29:05.164 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.164 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.164 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:05.164 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.926009+0000", 2024-09-16T09:29:05.164 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00024180299999999999 2024-09-16T09:29:05.164 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.164 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.165 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:29:05.165 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.926051+0000", 2024-09-16T09:29:05.165 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.2327999999999999e-05 2024-09-16T09:29:05.165 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.165 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.165 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:29:05.165 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.926086+0000", 2024-09-16T09:29:05.165 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.5584999999999999e-05 2024-09-16T09:29:05.165 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:29:05.165 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:29:05.165 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:29:05.165 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.165 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.165 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:215702 7.c 7:352e61ee:::benchmark_data_smithi070_78438_object89571:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e141)", 2024-09-16T09:29:05.166 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:29:03.721946+0000", 2024-09-16T09:29:05.166 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 1.344411566, 2024-09-16T09:29:05.166 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.20431201800000001, 2024-09-16T09:29:05.166 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:29:05.166 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:29:05.166 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:29:05.166 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:29:05.166 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:29:05.166 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:29:05.166 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 215702 2024-09-16T09:29:05.166 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.166 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:29:05.166 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.167 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:29:05.167 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.721946+0000", 2024-09-16T09:29:05.167 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:29:05.167 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.167 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.167 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:29:05.167 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.721946+0000", 2024-09-16T09:29:05.167 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:29:05.167 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.167 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.167 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:29:05.167 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.721946+0000", 2024-09-16T09:29:05.167 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.5199999999999998e-07 2024-09-16T09:29:05.167 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.167 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.167 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:29:05.168 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.721948+0000", 2024-09-16T09:29:05.168 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.0980000000000001e-06 2024-09-16T09:29:05.168 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.168 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.168 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:29:05.168 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.721953+0000", 2024-09-16T09:29:05.168 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.8940000000000001e-06 2024-09-16T09:29:05.168 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.168 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.168 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:29:05.168 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.721957+0000", 2024-09-16T09:29:05.168 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.0300000000000004e-06 2024-09-16T09:29:05.168 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.168 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.168 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:29:05.169 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.728650+0000", 2024-09-16T09:29:05.169 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0066934969999999996 2024-09-16T09:29:05.169 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.169 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.169 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:29:05.169 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.728753+0000", 2024-09-16T09:29:05.169 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000102785 2024-09-16T09:29:05.169 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.169 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.169 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 3,7", 2024-09-16T09:29:05.169 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.728856+0000", 2024-09-16T09:29:05.169 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000102303 2024-09-16T09:29:05.169 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.169 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.169 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:29:05.169 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.925926+0000", 2024-09-16T09:29:05.170 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.19707012099999999 2024-09-16T09:29:05.170 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.170 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.170 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:05.170 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.925968+0000", 2024-09-16T09:29:05.170 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.2169999999999998e-05 2024-09-16T09:29:05.170 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.170 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.170 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:05.170 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.926178+0000", 2024-09-16T09:29:05.170 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00021038400000000001 2024-09-16T09:29:05.170 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.170 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.170 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:29:05.170 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.926221+0000", 2024-09-16T09:29:05.170 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.3211999999999999e-05 2024-09-16T09:29:05.171 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.171 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.171 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:29:05.171 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.926258+0000", 2024-09-16T09:29:05.171 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.6171999999999999e-05 2024-09-16T09:29:05.171 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:29:05.171 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:29:05.171 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:29:05.171 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.171 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.171 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:215709 7.7 7:e925da47:::benchmark_data_smithi070_78438_object89578:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e141)", 2024-09-16T09:29:05.171 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:29:03.722069+0000", 2024-09-16T09:29:05.171 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 1.3442878710000001, 2024-09-16T09:29:05.171 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.203720754, 2024-09-16T09:29:05.171 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:29:05.171 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:29:05.172 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:29:05.172 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:29:05.172 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:29:05.172 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:29:05.172 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 215709 2024-09-16T09:29:05.172 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.172 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:29:05.172 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.172 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:29:05.172 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.722069+0000", 2024-09-16T09:29:05.172 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:29:05.172 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.172 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.172 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:29:05.172 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.722069+0000", 2024-09-16T09:29:05.173 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:29:05.173 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.173 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.173 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:29:05.173 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.722070+0000", 2024-09-16T09:29:05.173 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.9299999999999998e-07 2024-09-16T09:29:05.173 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.173 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.173 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:29:05.173 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.722073+0000", 2024-09-16T09:29:05.173 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.83e-06 2024-09-16T09:29:05.173 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.173 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.173 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:29:05.174 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.722079+0000", 2024-09-16T09:29:05.174 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.4010000000000002e-06 2024-09-16T09:29:05.174 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.174 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.174 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:29:05.174 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.722084+0000", 2024-09-16T09:29:05.174 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.2689999999999999e-06 2024-09-16T09:29:05.174 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.174 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.176 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:29:05.176 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.726106+0000", 2024-09-16T09:29:05.176 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.004021663 2024-09-16T09:29:05.176 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.176 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.176 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:29:05.176 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.726207+0000", 2024-09-16T09:29:05.176 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000101337 2024-09-16T09:29:05.176 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.176 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.176 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,3", 2024-09-16T09:29:05.177 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.726308+0000", 2024-09-16T09:29:05.177 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000100815 2024-09-16T09:29:05.177 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.177 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.177 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:29:05.177 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.729071+0000", 2024-09-16T09:29:05.177 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.002763111 2024-09-16T09:29:05.177 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.177 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.177 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:05.177 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.731946+0000", 2024-09-16T09:29:05.177 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.002874457 2024-09-16T09:29:05.177 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.177 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.177 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:05.177 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.925721+0000", 2024-09-16T09:29:05.178 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.193775314 2024-09-16T09:29:05.178 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.178 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.178 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:29:05.178 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.925757+0000", 2024-09-16T09:29:05.178 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.6494e-05 2024-09-16T09:29:05.178 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.178 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.178 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:29:05.178 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.925790+0000", 2024-09-16T09:29:05.178 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.2570000000000002e-05 2024-09-16T09:29:05.178 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:29:05.178 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:29:05.178 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:29:05.179 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.179 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.179 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:215721 7.c 7:3859c12b:::benchmark_data_smithi070_78438_object89590:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e141)", 2024-09-16T09:29:05.179 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:29:03.729262+0000", 2024-09-16T09:29:05.179 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 1.337094805, 2024-09-16T09:29:05.179 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.197115755, 2024-09-16T09:29:05.179 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:29:05.179 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:29:05.179 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:29:05.179 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:29:05.179 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:29:05.179 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:29:05.179 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 215721 2024-09-16T09:29:05.179 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.180 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:29:05.180 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.180 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:29:05.180 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.729262+0000", 2024-09-16T09:29:05.180 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:29:05.180 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.180 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.180 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:29:05.180 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.729262+0000", 2024-09-16T09:29:05.180 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:29:05.180 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.180 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.180 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:29:05.180 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.729263+0000", 2024-09-16T09:29:05.180 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.2999999999999999e-07 2024-09-16T09:29:05.180 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.181 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.181 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:29:05.181 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.729267+0000", 2024-09-16T09:29:05.181 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.822e-06 2024-09-16T09:29:05.181 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.181 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.181 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:29:05.181 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.729276+0000", 2024-09-16T09:29:05.181 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.3979999999999999e-06 2024-09-16T09:29:05.181 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.181 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.181 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:29:05.181 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.729280+0000", 2024-09-16T09:29:05.182 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.826e-06 2024-09-16T09:29:05.182 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.182 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.182 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:29:05.182 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.731471+0000", 2024-09-16T09:29:05.182 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.002190619 2024-09-16T09:29:05.182 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.182 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.182 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:29:05.182 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.731551+0000", 2024-09-16T09:29:05.182 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.9630999999999997e-05 2024-09-16T09:29:05.182 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.182 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.182 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 3,7", 2024-09-16T09:29:05.182 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.731616+0000", 2024-09-16T09:29:05.182 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.5062999999999997e-05 2024-09-16T09:29:05.183 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.183 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.183 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:29:05.183 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.925931+0000", 2024-09-16T09:29:05.183 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.19431493799999999 2024-09-16T09:29:05.183 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.183 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.183 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:05.183 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.926133+0000", 2024-09-16T09:29:05.183 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00020227600000000001 2024-09-16T09:29:05.183 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.183 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.184 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:05.184 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.926304+0000", 2024-09-16T09:29:05.184 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00017123099999999999 2024-09-16T09:29:05.184 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.184 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.184 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:29:05.184 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.926347+0000", 2024-09-16T09:29:05.184 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.2651000000000002e-05 2024-09-16T09:29:05.184 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.184 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.184 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:29:05.184 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.926378+0000", 2024-09-16T09:29:05.184 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.137e-05 2024-09-16T09:29:05.184 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:29:05.184 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:29:05.184 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:29:05.185 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.185 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.185 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_repop_reply(client.24698.0:215668 7.c e141/82 ondisk, result = 0)", 2024-09-16T09:29:05.185 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:29:03.731265+0000", 2024-09-16T09:29:05.185 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 1.3350924449999999, 2024-09-16T09:29:05.185 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.194475603, 2024-09-16T09:29:05.185 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:29:05.185 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:29:05.185 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "started", 2024-09-16T09:29:05.185 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:29:05.185 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.186 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:29:05.186 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.731265+0000", 2024-09-16T09:29:05.186 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:29:05.186 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.186 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.186 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:29:05.186 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.731265+0000", 2024-09-16T09:29:05.186 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:29:05.186 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.186 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.186 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:29:05.186 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.731266+0000", 2024-09-16T09:29:05.187 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.13e-06 2024-09-16T09:29:05.187 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.187 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.187 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:29:05.187 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.731279+0000", 2024-09-16T09:29:05.187 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.2967e-05 2024-09-16T09:29:05.187 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.187 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.187 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:29:05.187 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.731291+0000", 2024-09-16T09:29:05.187 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.2045000000000001e-05 2024-09-16T09:29:05.187 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.187 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.187 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:29:05.187 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.731299+0000", 2024-09-16T09:29:05.188 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.7559999999999999e-06 2024-09-16T09:29:05.188 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.188 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.188 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:29:05.188 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.925711+0000", 2024-09-16T09:29:05.188 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.194412262 2024-09-16T09:29:05.188 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.188 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.188 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:29:05.188 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.925716+0000", 2024-09-16T09:29:05.188 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.3229999999999997e-06 2024-09-16T09:29:05.188 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.188 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.189 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:29:05.189 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.925740+0000", 2024-09-16T09:29:05.189 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.4119999999999999e-05 2024-09-16T09:29:05.189 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:29:05.189 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:29:05.189 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:29:05.189 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.189 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.189 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_repop_reply(client.24698.0:215673 7.c e141/82 ondisk, result = 0)", 2024-09-16T09:29:05.189 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:29:03.733382+0000", 2024-09-16T09:29:05.189 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 1.332974842, 2024-09-16T09:29:05.189 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.19239313599999999, 2024-09-16T09:29:05.189 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:29:05.189 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:29:05.190 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "started", 2024-09-16T09:29:05.190 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:29:05.190 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.190 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:29:05.190 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.733382+0000", 2024-09-16T09:29:05.190 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:29:05.190 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.190 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.190 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:29:05.190 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.733382+0000", 2024-09-16T09:29:05.190 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:29:05.190 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.190 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.190 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:29:05.191 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.733383+0000", 2024-09-16T09:29:05.191 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.4099999999999997e-07 2024-09-16T09:29:05.191 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.191 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.191 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:29:05.191 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.733387+0000", 2024-09-16T09:29:05.191 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.3189999999999996e-06 2024-09-16T09:29:05.191 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.191 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.191 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:29:05.191 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.733398+0000", 2024-09-16T09:29:05.191 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0699e-05 2024-09-16T09:29:05.191 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.191 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.191 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:29:05.192 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.925646+0000", 2024-09-16T09:29:05.192 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.19224790999999999 2024-09-16T09:29:05.192 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.192 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.192 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:29:05.192 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.925759+0000", 2024-09-16T09:29:05.192 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000113022 2024-09-16T09:29:05.192 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.192 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.192 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:29:05.192 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.925764+0000", 2024-09-16T09:29:05.192 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.8559999999999998e-06 2024-09-16T09:29:05.192 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.192 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.193 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:29:05.193 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.925775+0000", 2024-09-16T09:29:05.193 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1489e-05 2024-09-16T09:29:05.193 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:29:05.193 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:29:05.193 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:29:05.193 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.193 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.193 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_repop_reply(client.24698.0:215709 7.7 e141/82 ondisk, result = 0)", 2024-09-16T09:29:05.193 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:29:03.733756+0000", 2024-09-16T09:29:05.194 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 1.3326013589999999, 2024-09-16T09:29:05.194 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.19203947800000001, 2024-09-16T09:29:05.194 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:29:05.194 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:29:05.194 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "started", 2024-09-16T09:29:05.194 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:29:05.194 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.194 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:29:05.194 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.733756+0000", 2024-09-16T09:29:05.195 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:29:05.195 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.195 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.195 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:29:05.195 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.733756+0000", 2024-09-16T09:29:05.195 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:29:05.195 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.195 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.196 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:29:05.196 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.733756+0000", 2024-09-16T09:29:05.196 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.6599999999999996e-07 2024-09-16T09:29:05.196 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.196 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.196 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:29:05.196 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.733761+0000", 2024-09-16T09:29:05.196 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.2950000000000004e-06 2024-09-16T09:29:05.196 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.197 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.197 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:29:05.197 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.925646+0000", 2024-09-16T09:29:05.197 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.19188530400000001 2024-09-16T09:29:05.197 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.197 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.197 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:29:05.197 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.925661+0000", 2024-09-16T09:29:05.197 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.5029e-05 2024-09-16T09:29:05.197 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.197 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.197 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:29:05.197 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.925709+0000", 2024-09-16T09:29:05.198 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.7506000000000002e-05 2024-09-16T09:29:05.198 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.198 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.198 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:29:05.198 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.925716+0000", 2024-09-16T09:29:05.198 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.435e-06 2024-09-16T09:29:05.198 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:05.198 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:05.198 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:29:05.198 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.925795+0000", 2024-09-16T09:29:05.198 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.9243e-05 2024-09-16T09:29:05.198 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:29:05.198 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:29:05.198 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:29:05.198 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:29:05.198 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:29:05.199 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-09-16T09:29:05.200 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:05 smithi084 ceph-mon[32629]: 4.1c scrub starts 2024-09-16T09:29:05.200 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:05 smithi084 ceph-mon[32629]: 4.1c scrub ok 2024-09-16T09:29:05.200 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:05 smithi084 ceph-mon[32629]: pgmap v177: 158 pgs: 158 active+clean; 2.3 GiB data, 7.9 GiB used, 707 GiB / 715 GiB avail; 922 B/s rd, 0 B/s wr, 1.05k op/s 2024-09-16T09:29:05.200 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:05.201 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:05.201 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:29:05.302 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:29:05.347 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:05.348 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:05.348 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:29:05.348 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:29:05.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:05 smithi070 ceph-mon[40691]: 4.1c scrub starts 2024-09-16T09:29:05.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:05 smithi070 ceph-mon[40691]: 4.1c scrub ok 2024-09-16T09:29:05.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:05 smithi070 ceph-mon[40691]: pgmap v177: 158 pgs: 158 active+clean; 2.3 GiB data, 7.9 GiB used, 707 GiB / 715 GiB avail; 922 B/s rd, 0 B/s wr, 1.05k op/s 2024-09-16T09:29:05.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:05 smithi070 ceph-mon[30388]: 4.1c scrub starts 2024-09-16T09:29:05.422 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:05 smithi070 ceph-mon[30388]: 4.1c scrub ok 2024-09-16T09:29:05.422 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:05 smithi070 ceph-mon[30388]: pgmap v177: 158 pgs: 158 active+clean; 2.3 GiB data, 7.9 GiB used, 707 GiB / 715 GiB avail; 922 B/s rd, 0 B/s wr, 1.05k op/s 2024-09-16T09:29:05.449 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:29:05.503 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:05.503 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:05.503 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:29:05.604 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:29:05.632 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:05.632 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:05.633 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:29:05.734 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:29:05.766 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:05.767 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:05.768 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:29:05.868 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:29:05.906 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:05.906 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:05.907 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:29:06.008 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:29:06.043 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:06.044 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:06.044 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:29:06.145 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:29:06.182 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:06.183 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:06.184 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:29:06.284 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:29:06.331 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:06.332 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:06.333 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:29:06.433 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:29:06.476 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:06.476 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:06.477 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:29:06.488 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:29:06.488 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:29:06.524 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:29:06.524 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:29:06.557 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:29:06.557 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:29:06.577 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:29:06.590 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:29:06.590 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:29:06.607 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:06.607 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:06.608 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:29:06.608 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-16T09:29:06.620 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:29:06.621 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:29:06.652 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:29:06.652 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:29:06.685 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:29:06.686 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:29:06.709 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:29:06.718 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:29:06.719 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:29:06.734 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:06.734 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:06.735 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:29:06.836 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:29:06.867 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.7 dump_ops_in_flight 2024-09-16T09:29:06.890 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:06.891 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:06.891 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:29:06.992 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:29:07.025 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:07.025 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:07.026 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:29:07.127 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:29:07.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:07 smithi084 ceph-mon[32629]: pgmap v178: 158 pgs: 158 active+clean; 2.3 GiB data, 7.9 GiB used, 707 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 974 op/s 2024-09-16T09:29:07.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:07 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/2852999816' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:29:07.167 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:07.167 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:07.167 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:29:07.225 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:29:07.268 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:29:07.299 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:07.300 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:07.300 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:29:07.401 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:29:07.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:07 smithi070 ceph-mon[30388]: pgmap v178: 158 pgs: 158 active+clean; 2.3 GiB data, 7.9 GiB used, 707 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 974 op/s 2024-09-16T09:29:07.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:07 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2852999816' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:29:07.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:07 smithi070 ceph-mon[40691]: pgmap v178: 158 pgs: 158 active+clean; 2.3 GiB data, 7.9 GiB used, 707 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 974 op/s 2024-09-16T09:29:07.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:07 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/2852999816' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:29:07.428 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:07.429 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:07.429 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:29:07.530 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:29:07.546 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:29:07.576 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:07.577 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:07.577 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:29:07.678 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:29:07.708 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:07.708 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:07.709 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:29:07.795 INFO:teuthology.orchestra.run.smithi084.stdout:ERROR: (22) Invalid argument 2024-09-16T09:29:07.795 INFO:teuthology.orchestra.run.smithi084.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:29:07.809 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:29:07.840 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:07.841 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:07.841 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:29:07.942 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:29:07.976 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:07.976 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:07.977 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:29:08.078 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:29:08.104 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:08.105 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:08.105 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:29:08.206 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:29:08.230 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:08.231 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:08.231 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:29:08.332 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:29:08.375 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:08.376 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:08.376 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:29:08.477 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:29:08.501 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:08.502 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:08.503 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:29:08.603 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:29:08.627 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:08.628 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:08.628 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:29:08.696 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.7 dump_blocked_ops 2024-09-16T09:29:08.729 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:29:08.770 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:08.771 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:08.771 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:29:08.871 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:29:08.911 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:08.911 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:08.911 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:29:09.012 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:29:09.034 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:29:09.047 INFO:teuthology.orchestra.run.smithi070.stderr:nodeep-scrub is set 2024-09-16T09:29:09.062 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:09.063 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:09.064 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:29:09.164 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:29:09.190 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:09.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:09.191 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:29:09.292 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:29:09.318 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:09.319 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:09.319 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:29:09.344 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:09 smithi084 ceph-mon[32629]: pgmap v179: 158 pgs: 158 active+clean; 2.2 GiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.05k op/s 2024-09-16T09:29:09.344 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:09 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3331782499' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:29:09.345 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:09 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:29:09.409 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:09 smithi070 ceph-mon[30388]: pgmap v179: 158 pgs: 158 active+clean; 2.2 GiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.05k op/s 2024-09-16T09:29:09.410 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:09 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3331782499' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:29:09.410 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:09 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:29:09.410 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:09 smithi070 ceph-mon[40691]: pgmap v179: 158 pgs: 158 active+clean; 2.2 GiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.05k op/s 2024-09-16T09:29:09.410 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:09 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3331782499' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:29:09.410 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:09 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:29:09.419 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:29:09.445 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:09.446 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:09.446 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:29:09.493 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T09:29:09.547 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:29:09.568 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:09.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:09.569 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:29:09.669 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:29:09.712 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:09.713 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:09.713 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:29:09.722 INFO:teuthology.orchestra.run.smithi084.stdout:ERROR: (22) Invalid argument 2024-09-16T09:29:09.722 INFO:teuthology.orchestra.run.smithi084.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:29:09.814 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:29:09.840 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:09.840 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:09.841 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:29:09.942 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:29:09.966 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:09.967 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:09.967 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:29:10.068 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:29:10.098 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:10.098 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:10.099 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:29:10.199 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:29:10.225 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:10.225 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:10.226 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:29:10.304 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:10 smithi070 ceph-mon[30388]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:29:10.304 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:10 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-16T09:29:10.304 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:10 smithi070 ceph-mon[30388]: osdmap e142: 8 total, 8 up, 8 in 2024-09-16T09:29:10.305 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:10 smithi070 ceph-mon[40691]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:29:10.305 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:10 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-16T09:29:10.305 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:10 smithi070 ceph-mon[40691]: osdmap e142: 8 total, 8 up, 8 in 2024-09-16T09:29:10.326 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:29:10.352 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:10.353 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:10.354 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:29:10.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:10 smithi084 ceph-mon[32629]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:29:10.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:10 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-16T09:29:10.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:10 smithi084 ceph-mon[32629]: osdmap e142: 8 total, 8 up, 8 in 2024-09-16T09:29:10.454 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:29:10.469 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:29:10.480 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:10.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:10.481 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:29:10.582 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:29:10.623 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:10.623 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:10.624 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:29:10.724 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:29:10.760 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:10.761 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:10.761 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:29:10.851 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.7 dump_historic_ops 2024-09-16T09:29:10.862 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:29:10.927 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:10.928 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:10.928 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:29:11.029 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:29:11.067 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:11.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:11.068 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:29:11.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:11 smithi084 ceph-mon[32629]: pgmap v181: 158 pgs: 158 active+clean; 2.0 GiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.51k op/s 2024-09-16T09:29:11.168 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:29:11.182 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:29:11.188 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:29:11.189 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:29:11.200 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:11.201 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:11.201 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:29:11.230 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:29:11.231 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:29:11.265 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:11 smithi070 ceph-mon[30388]: pgmap v181: 158 pgs: 158 active+clean; 2.0 GiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.51k op/s 2024-09-16T09:29:11.265 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:29:11.265 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:29:11.266 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:11 smithi070 ceph-mon[40691]: pgmap v181: 158 pgs: 158 active+clean; 2.0 GiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.51k op/s 2024-09-16T09:29:11.297 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:29:11.298 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:29:11.302 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:29:11.330 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:11.330 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:11.331 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:29:11.331 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:29:11.331 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:29:11.342 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T09:29:11.342 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T09:29:11.342 INFO:tasks.thrashosds.thrasher:fixing pg num pool unique_pool_0 2024-09-16T09:29:11.343 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-16T09:29:11.368 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:29:11.368 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:29:11.402 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:29:11.403 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:29:11.431 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:29:11.435 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:29:11.435 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:29:11.477 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:11.477 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:11.478 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:29:11.579 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:29:11.608 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:11.608 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:11.609 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:29:11.710 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:29:11.752 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:11.753 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:11.753 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:29:11.761 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-09-16T09:29:11.762 INFO:teuthology.orchestra.run.smithi084.stdout: "size": 20, 2024-09-16T09:29:11.762 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 600, 2024-09-16T09:29:11.762 INFO:teuthology.orchestra.run.smithi084.stdout: "ops": [ 2024-09-16T09:29:11.762 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.762 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:22924 7.5 7:a7d40976:::benchmark_data_smithi070_78438_object22923:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T09:29:11.762 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:26:21.580769+0000", 2024-09-16T09:29:11.763 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 170.178018522, 2024-09-16T09:29:11.763 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.079602870000000006, 2024-09-16T09:29:11.763 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:29:11.763 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:29:11.763 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:29:11.763 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:29:11.764 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:29:11.764 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:29:11.764 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 22924 2024-09-16T09:29:11.764 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.764 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:29:11.764 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.765 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:29:11.765 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.580769+0000", 2024-09-16T09:29:11.765 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:29:11.765 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.765 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.765 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:29:11.766 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.580769+0000", 2024-09-16T09:29:11.766 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:29:11.766 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.766 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.766 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:29:11.766 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.580770+0000", 2024-09-16T09:29:11.767 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.3200000000000001e-06 2024-09-16T09:29:11.767 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.767 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.767 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:29:11.767 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.580848+0000", 2024-09-16T09:29:11.767 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.7514000000000004e-05 2024-09-16T09:29:11.768 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.768 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.768 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:29:11.768 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.580867+0000", 2024-09-16T09:29:11.768 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.8782000000000001e-05 2024-09-16T09:29:11.768 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.768 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.769 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:29:11.769 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.580874+0000", 2024-09-16T09:29:11.769 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.1269999999999997e-06 2024-09-16T09:29:11.769 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.770 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.770 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:29:11.770 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.580941+0000", 2024-09-16T09:29:11.770 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.7010999999999999e-05 2024-09-16T09:29:11.771 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.771 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.771 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:29:11.771 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.581054+0000", 2024-09-16T09:29:11.771 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000113509 2024-09-16T09:29:11.772 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.772 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.772 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,2", 2024-09-16T09:29:11.772 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.581176+0000", 2024-09-16T09:29:11.772 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000122187 2024-09-16T09:29:11.772 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.773 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.773 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:29:11.773 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.582818+0000", 2024-09-16T09:29:11.773 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0016419200000000001 2024-09-16T09:29:11.773 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.774 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.774 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:11.774 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.583304+0000", 2024-09-16T09:29:11.774 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00048596099999999998 2024-09-16T09:29:11.774 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.774 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.775 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:11.775 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.660281+0000", 2024-09-16T09:29:11.775 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.076976634000000002 2024-09-16T09:29:11.775 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.775 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.775 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:29:11.775 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.660322+0000", 2024-09-16T09:29:11.775 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.1125999999999999e-05 2024-09-16T09:29:11.776 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.776 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.776 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:29:11.776 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.660372+0000", 2024-09-16T09:29:11.776 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.9778999999999999e-05 2024-09-16T09:29:11.776 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:29:11.777 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:29:11.777 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:29:11.777 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.777 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.777 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:23476 7.5 7:ab69eea0:::benchmark_data_smithi070_78438_object23475:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T09:29:11.777 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:26:21.880371+0000", 2024-09-16T09:29:11.777 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 169.87841618900001, 2024-09-16T09:29:11.777 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.069588092000000004, 2024-09-16T09:29:11.777 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:29:11.778 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:29:11.778 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:29:11.778 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:29:11.778 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:29:11.778 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:29:11.778 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 23476 2024-09-16T09:29:11.778 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.778 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:29:11.779 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.779 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:29:11.779 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.880371+0000", 2024-09-16T09:29:11.779 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:29:11.779 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.779 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.779 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:29:11.780 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.880371+0000", 2024-09-16T09:29:11.780 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:29:11.780 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.780 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.780 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:29:11.780 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.880372+0000", 2024-09-16T09:29:11.781 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.4799999999999997e-07 2024-09-16T09:29:11.781 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.781 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.781 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:29:11.781 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.880461+0000", 2024-09-16T09:29:11.781 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.9232000000000002e-05 2024-09-16T09:29:11.781 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.782 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.782 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:29:11.782 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.880473+0000", 2024-09-16T09:29:11.782 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1687e-05 2024-09-16T09:29:11.782 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.782 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.783 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:29:11.783 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.880480+0000", 2024-09-16T09:29:11.783 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.3000000000000004e-06 2024-09-16T09:29:11.783 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.783 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.783 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:29:11.784 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.881388+0000", 2024-09-16T09:29:11.784 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00090752200000000002 2024-09-16T09:29:11.784 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.784 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.784 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:29:11.784 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.881520+0000", 2024-09-16T09:29:11.785 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00013243400000000001 2024-09-16T09:29:11.785 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.785 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.785 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,2", 2024-09-16T09:29:11.785 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.881649+0000", 2024-09-16T09:29:11.785 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00012842299999999999 2024-09-16T09:29:11.786 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.786 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.786 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:29:11.786 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.882506+0000", 2024-09-16T09:29:11.786 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000857047 2024-09-16T09:29:11.786 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.787 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.787 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:11.787 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.884941+0000", 2024-09-16T09:29:11.787 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0024354149999999998 2024-09-16T09:29:11.787 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.788 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.788 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:11.788 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.949871+0000", 2024-09-16T09:29:11.788 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.064929929999999997 2024-09-16T09:29:11.788 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.788 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.789 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:29:11.789 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.949921+0000", 2024-09-16T09:29:11.789 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.9660999999999998e-05 2024-09-16T09:29:11.789 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.789 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.789 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:29:11.790 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.949959+0000", 2024-09-16T09:29:11.790 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.8692999999999997e-05 2024-09-16T09:29:11.790 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:29:11.790 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:29:11.790 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:29:11.790 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.791 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.791 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:23481 7.5 7:a58daa7e:::benchmark_data_smithi070_78438_object23480:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T09:29:11.791 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:26:21.883877+0000", 2024-09-16T09:29:11.791 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 169.87491000099999, 2024-09-16T09:29:11.791 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.067196770000000003, 2024-09-16T09:29:11.791 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:29:11.792 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:29:11.793 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:29:11.794 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:29:11.794 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:29:11.794 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:29:11.794 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 23481 2024-09-16T09:29:11.794 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.794 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:29:11.795 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.795 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:29:11.795 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.883877+0000", 2024-09-16T09:29:11.795 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:29:11.795 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.795 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.796 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:29:11.796 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.883877+0000", 2024-09-16T09:29:11.796 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:29:11.796 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.796 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.796 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:29:11.796 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.883879+0000", 2024-09-16T09:29:11.797 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.144e-06 2024-09-16T09:29:11.797 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.797 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.797 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:29:11.797 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.883931+0000", 2024-09-16T09:29:11.797 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.2123000000000002e-05 2024-09-16T09:29:11.797 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.797 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.798 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:29:11.798 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.883943+0000", 2024-09-16T09:29:11.798 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.2323999999999999e-05 2024-09-16T09:29:11.798 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.798 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.798 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:29:11.799 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.883950+0000", 2024-09-16T09:29:11.799 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.3900000000000004e-06 2024-09-16T09:29:11.799 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.799 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.799 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:29:11.799 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.884006+0000", 2024-09-16T09:29:11.800 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.5387999999999999e-05 2024-09-16T09:29:11.800 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.800 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.800 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:29:11.800 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.884117+0000", 2024-09-16T09:29:11.800 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000111461 2024-09-16T09:29:11.801 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.801 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.801 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,2", 2024-09-16T09:29:11.801 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.884259+0000", 2024-09-16T09:29:11.801 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000141686 2024-09-16T09:29:11.801 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.802 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.802 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:29:11.802 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.885063+0000", 2024-09-16T09:29:11.802 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00080437399999999995 2024-09-16T09:29:11.802 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.803 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.803 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:11.803 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.886156+0000", 2024-09-16T09:29:11.803 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0010922130000000001 2024-09-16T09:29:11.803 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.803 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.804 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:11.804 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.950978+0000", 2024-09-16T09:29:11.804 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.064822639000000001 2024-09-16T09:29:11.804 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.804 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.805 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:29:11.805 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.951036+0000", 2024-09-16T09:29:11.805 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.7666e-05 2024-09-16T09:29:11.805 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.805 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.805 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:29:11.805 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.951074+0000", 2024-09-16T09:29:11.806 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.8362000000000003e-05 2024-09-16T09:29:11.806 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:29:11.806 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:29:11.806 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:29:11.806 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.806 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.806 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_repop(client.24698.0:27466 7.6 e89/82 7:60c44c8b:::benchmark_data_smithi070_78438_object27465:head v 89'1726, mlcod=89'1723)", 2024-09-16T09:29:11.806 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:26:23.722364+0000", 2024-09-16T09:29:11.807 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 168.03642363399999, 2024-09-16T09:29:11.807 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.071550814000000004, 2024-09-16T09:29:11.807 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:29:11.807 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:29:11.807 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:29:11.807 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:29:11.807 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.807 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:29:11.808 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.722364+0000", 2024-09-16T09:29:11.808 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:29:11.808 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.808 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.808 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:29:11.808 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.722364+0000", 2024-09-16T09:29:11.808 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:29:11.808 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.809 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.809 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:29:11.809 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.722364+0000", 2024-09-16T09:29:11.809 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.0699999999999998e-07 2024-09-16T09:29:11.809 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.809 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.809 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:29:11.809 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.722426+0000", 2024-09-16T09:29:11.810 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.1278999999999994e-05 2024-09-16T09:29:11.810 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.810 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.810 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:29:11.810 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.722432+0000", 2024-09-16T09:29:11.810 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.003e-06 2024-09-16T09:29:11.811 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.811 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.811 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:29:11.811 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.722435+0000", 2024-09-16T09:29:11.811 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.4010000000000001e-06 2024-09-16T09:29:11.811 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.812 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.812 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:29:11.812 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.722461+0000", 2024-09-16T09:29:11.812 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.5894999999999999e-05 2024-09-16T09:29:11.812 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.812 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.812 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:29:11.813 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.722471+0000", 2024-09-16T09:29:11.813 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0251000000000001e-05 2024-09-16T09:29:11.813 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.813 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.813 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:29:11.813 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.793863+0000", 2024-09-16T09:29:11.813 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.071392294999999995 2024-09-16T09:29:11.816 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.816 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.816 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:29:11.816 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.793915+0000", 2024-09-16T09:29:11.816 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.1282999999999997e-05 2024-09-16T09:29:11.816 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:29:11.816 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:29:11.816 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:29:11.816 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.817 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.817 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_repop(client.24698.0:27468 7.8 e89/82 7:1de713de:::benchmark_data_smithi070_78438_object27467:head v 89'1721, mlcod=89'1718)", 2024-09-16T09:29:11.817 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:26:23.722887+0000", 2024-09-16T09:29:11.817 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 168.03590061400001, 2024-09-16T09:29:11.817 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.069612414999999997, 2024-09-16T09:29:11.817 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:29:11.818 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:29:11.818 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:29:11.818 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:29:11.818 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.818 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:29:11.818 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.722887+0000", 2024-09-16T09:29:11.818 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:29:11.818 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.818 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.819 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:29:11.819 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.722887+0000", 2024-09-16T09:29:11.819 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:29:11.819 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.819 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.819 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:29:11.819 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.722887+0000", 2024-09-16T09:29:11.819 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.0100000000000005e-07 2024-09-16T09:29:11.819 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.820 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.820 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:29:11.820 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.722949+0000", 2024-09-16T09:29:11.820 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.1298999999999997e-05 2024-09-16T09:29:11.820 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.820 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.820 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:29:11.820 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.790454+0000", 2024-09-16T09:29:11.821 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.067505342999999995 2024-09-16T09:29:11.821 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.821 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.821 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:29:11.821 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.790465+0000", 2024-09-16T09:29:11.821 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1242e-05 2024-09-16T09:29:11.821 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.821 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.821 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:29:11.822 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.790558+0000", 2024-09-16T09:29:11.822 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.3227999999999996e-05 2024-09-16T09:29:11.822 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.822 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.822 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:29:11.822 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.790578+0000", 2024-09-16T09:29:11.822 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.9114999999999999e-05 2024-09-16T09:29:11.822 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.823 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.823 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:29:11.823 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.792380+0000", 2024-09-16T09:29:11.823 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0018023329999999999 2024-09-16T09:29:11.823 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.823 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.823 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:29:11.823 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.792499+0000", 2024-09-16T09:29:11.824 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000119354 2024-09-16T09:29:11.824 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:29:11.824 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:29:11.824 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:29:11.824 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.824 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.824 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:27473 7.5 7:a272c321:::benchmark_data_smithi070_78438_object27472:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T09:29:11.824 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:26:23.723821+0000", 2024-09-16T09:29:11.825 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 168.03496602300001, 2024-09-16T09:29:11.825 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.073668624000000002, 2024-09-16T09:29:11.825 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:29:11.825 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:29:11.825 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:29:11.825 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:29:11.825 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:29:11.826 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:29:11.826 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 27473 2024-09-16T09:29:11.826 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.826 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:29:11.826 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.826 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:29:11.826 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.723821+0000", 2024-09-16T09:29:11.826 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:29:11.827 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.827 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.827 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:29:11.827 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.723821+0000", 2024-09-16T09:29:11.827 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:29:11.827 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.827 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.827 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:29:11.827 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.723823+0000", 2024-09-16T09:29:11.827 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1999999999999999e-06 2024-09-16T09:29:11.827 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.827 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.827 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:29:11.828 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.723884+0000", 2024-09-16T09:29:11.828 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.0971000000000003e-05 2024-09-16T09:29:11.828 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.828 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.828 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:29:11.828 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.790196+0000", 2024-09-16T09:29:11.828 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.066312372999999994 2024-09-16T09:29:11.828 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.828 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.828 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:29:11.828 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.790213+0000", 2024-09-16T09:29:11.828 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.6974e-05 2024-09-16T09:29:11.828 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.829 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.829 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:29:11.829 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.790559+0000", 2024-09-16T09:29:11.829 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00034600599999999999 2024-09-16T09:29:11.829 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.829 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.829 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:29:11.829 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.790669+0000", 2024-09-16T09:29:11.829 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000110276 2024-09-16T09:29:11.829 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.829 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.829 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,2", 2024-09-16T09:29:11.829 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.790788+0000", 2024-09-16T09:29:11.829 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000119096 2024-09-16T09:29:11.830 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.830 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.830 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:11.830 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.793102+0000", 2024-09-16T09:29:11.830 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0023135030000000002 2024-09-16T09:29:11.830 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.830 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.830 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:29:11.830 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.793337+0000", 2024-09-16T09:29:11.830 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000235481 2024-09-16T09:29:11.830 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.830 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.830 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:11.830 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.797381+0000", 2024-09-16T09:29:11.831 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0040435699999999998 2024-09-16T09:29:11.831 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.831 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.831 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:29:11.831 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.797446+0000", 2024-09-16T09:29:11.831 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.4759999999999997e-05 2024-09-16T09:29:11.831 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.831 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.831 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:29:11.831 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.797490+0000", 2024-09-16T09:29:11.831 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.4413999999999999e-05 2024-09-16T09:29:11.831 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:29:11.831 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:29:11.832 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:29:11.832 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.832 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.832 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_repop(client.24698.0:28810 7.6 e90/82 7:63b6347d:::benchmark_data_smithi070_78438_object28809:head v 90'1809, mlcod=90'1805)", 2024-09-16T09:29:11.832 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:26:24.482328+0000", 2024-09-16T09:29:11.832 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 167.27645918499999, 2024-09-16T09:29:11.832 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.063714949000000007, 2024-09-16T09:29:11.832 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:29:11.832 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:29:11.832 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:29:11.832 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:29:11.832 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.832 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:29:11.832 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.482328+0000", 2024-09-16T09:29:11.833 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:29:11.833 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.833 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.833 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:29:11.833 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.482328+0000", 2024-09-16T09:29:11.833 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:29:11.833 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.833 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.833 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:29:11.833 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.482329+0000", 2024-09-16T09:29:11.833 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.0200000000000002e-07 2024-09-16T09:29:11.833 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.833 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.833 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:29:11.834 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.482394+0000", 2024-09-16T09:29:11.834 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.4706e-05 2024-09-16T09:29:11.834 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.834 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.834 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:29:11.834 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.543399+0000", 2024-09-16T09:29:11.834 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.061005750999999997 2024-09-16T09:29:11.834 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.834 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.834 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:29:11.834 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.543411+0000", 2024-09-16T09:29:11.834 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1687e-05 2024-09-16T09:29:11.834 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.835 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.835 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:29:11.835 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.543539+0000", 2024-09-16T09:29:11.835 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000128298 2024-09-16T09:29:11.835 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.835 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.835 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:29:11.835 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.543558+0000", 2024-09-16T09:29:11.835 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.8323999999999999e-05 2024-09-16T09:29:11.835 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.835 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.835 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:29:11.835 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.546006+0000", 2024-09-16T09:29:11.835 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0024483180000000001 2024-09-16T09:29:11.835 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.836 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.836 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:29:11.836 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.546043+0000", 2024-09-16T09:29:11.836 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.7262999999999999e-05 2024-09-16T09:29:11.836 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:29:11.836 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:29:11.836 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:29:11.836 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.836 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.836 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:60617 7.5 7:a21a3d15:::benchmark_data_smithi070_78438_object60616:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-16T09:29:11.836 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:26:44.711293+0000", 2024-09-16T09:29:11.836 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 147.047494532, 2024-09-16T09:29:11.836 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.063119306999999999, 2024-09-16T09:29:11.836 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:29:11.837 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:29:11.837 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:29:11.837 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:29:11.837 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:29:11.837 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:29:11.837 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 60617 2024-09-16T09:29:11.837 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.837 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:29:11.837 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.837 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:29:11.837 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.711293+0000", 2024-09-16T09:29:11.837 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:29:11.837 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.837 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.838 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:29:11.838 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.711293+0000", 2024-09-16T09:29:11.838 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:29:11.838 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.838 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.838 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:29:11.838 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.711294+0000", 2024-09-16T09:29:11.838 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.406e-06 2024-09-16T09:29:11.838 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.838 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.838 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:29:11.838 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.711347+0000", 2024-09-16T09:29:11.838 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.2803999999999998e-05 2024-09-16T09:29:11.838 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.839 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.839 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:29:11.839 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.711360+0000", 2024-09-16T09:29:11.839 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.2937e-05 2024-09-16T09:29:11.839 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.839 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.839 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:29:11.839 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.711365+0000", 2024-09-16T09:29:11.839 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.3009999999999997e-06 2024-09-16T09:29:11.839 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.839 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.839 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:29:11.839 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.711414+0000", 2024-09-16T09:29:11.839 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.8568e-05 2024-09-16T09:29:11.840 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.840 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.840 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:29:11.840 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.711505+0000", 2024-09-16T09:29:11.840 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.1026999999999994e-05 2024-09-16T09:29:11.840 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.840 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.840 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,2", 2024-09-16T09:29:11.840 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.711588+0000", 2024-09-16T09:29:11.840 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.3224000000000001e-05 2024-09-16T09:29:11.840 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.840 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.840 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:29:11.840 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.713160+0000", 2024-09-16T09:29:11.841 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0015717120000000001 2024-09-16T09:29:11.841 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.841 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.841 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:11.841 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.714523+0000", 2024-09-16T09:29:11.841 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0013628030000000001 2024-09-16T09:29:11.841 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.841 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.841 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:11.841 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.774315+0000", 2024-09-16T09:29:11.841 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.059791959999999998 2024-09-16T09:29:11.841 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.841 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.841 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:29:11.842 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.774367+0000", 2024-09-16T09:29:11.842 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.2552000000000001e-05 2024-09-16T09:29:11.842 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.842 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.842 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:29:11.842 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.774412+0000", 2024-09-16T09:29:11.842 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.5012999999999997e-05 2024-09-16T09:29:11.842 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:29:11.842 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:29:11.842 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:29:11.842 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.842 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.842 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:62033 7.5 7:a5dfc1ea:::benchmark_data_smithi070_78438_object62032:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-16T09:29:11.842 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:26:45.811217+0000", 2024-09-16T09:29:11.843 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 145.94757065900001, 2024-09-16T09:29:11.843 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.071578969000000006, 2024-09-16T09:29:11.843 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:29:11.843 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:29:11.843 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:29:11.843 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:29:11.843 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:29:11.843 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:29:11.843 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 62033 2024-09-16T09:29:11.843 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.843 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:29:11.843 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.843 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:29:11.843 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.811217+0000", 2024-09-16T09:29:11.844 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:29:11.844 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.844 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.844 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:29:11.844 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.811217+0000", 2024-09-16T09:29:11.844 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:29:11.844 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.844 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.844 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:29:11.844 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.811218+0000", 2024-09-16T09:29:11.844 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.5290000000000001e-06 2024-09-16T09:29:11.844 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.844 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.844 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:29:11.845 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.811262+0000", 2024-09-16T09:29:11.845 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.3850999999999997e-05 2024-09-16T09:29:11.845 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.845 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.845 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:29:11.845 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.811278+0000", 2024-09-16T09:29:11.845 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.5753999999999999e-05 2024-09-16T09:29:11.845 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.845 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.845 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:29:11.846 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.811286+0000", 2024-09-16T09:29:11.846 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.7519999999999996e-06 2024-09-16T09:29:11.846 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.846 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.846 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:29:11.846 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.811333+0000", 2024-09-16T09:29:11.846 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.7203000000000002e-05 2024-09-16T09:29:11.846 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.846 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.846 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:29:11.846 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.811456+0000", 2024-09-16T09:29:11.846 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00012353900000000001 2024-09-16T09:29:11.846 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.846 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.847 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,2", 2024-09-16T09:29:11.847 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.811644+0000", 2024-09-16T09:29:11.847 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00018799100000000001 2024-09-16T09:29:11.847 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.847 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.847 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:29:11.847 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.814035+0000", 2024-09-16T09:29:11.847 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0023902009999999998 2024-09-16T09:29:11.847 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.847 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.847 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:11.847 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.815646+0000", 2024-09-16T09:29:11.847 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0016112399999999999 2024-09-16T09:29:11.847 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.848 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.848 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:11.848 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.882683+0000", 2024-09-16T09:29:11.848 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.067037332000000005 2024-09-16T09:29:11.848 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.848 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.848 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:29:11.848 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.882753+0000", 2024-09-16T09:29:11.848 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.9534000000000002e-05 2024-09-16T09:29:11.848 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.848 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.848 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:29:11.848 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.882796+0000", 2024-09-16T09:29:11.848 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.3043000000000001e-05 2024-09-16T09:29:11.849 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:29:11.849 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:29:11.849 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:29:11.849 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.849 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.849 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:62044 7.5 7:aee12d22:::benchmark_data_smithi070_78438_object62043:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-16T09:29:11.849 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:26:45.819031+0000", 2024-09-16T09:29:11.849 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 145.93975679900001, 2024-09-16T09:29:11.849 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.063892420000000005, 2024-09-16T09:29:11.849 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:29:11.849 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:29:11.849 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:29:11.849 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:29:11.849 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:29:11.849 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:29:11.850 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 62044 2024-09-16T09:29:11.850 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.850 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:29:11.850 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.850 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:29:11.850 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.819031+0000", 2024-09-16T09:29:11.850 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:29:11.850 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.850 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.850 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:29:11.850 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.819031+0000", 2024-09-16T09:29:11.850 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:29:11.850 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.850 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.851 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:29:11.851 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.819032+0000", 2024-09-16T09:29:11.851 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.3120000000000001e-06 2024-09-16T09:29:11.851 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.851 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.851 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:29:11.851 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.819071+0000", 2024-09-16T09:29:11.851 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.8599000000000001e-05 2024-09-16T09:29:11.851 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.851 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.851 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:29:11.851 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.819081+0000", 2024-09-16T09:29:11.851 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0812000000000001e-05 2024-09-16T09:29:11.851 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.852 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.852 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:29:11.852 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.819089+0000", 2024-09-16T09:29:11.852 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.8089999999999993e-06 2024-09-16T09:29:11.852 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.852 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.852 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:29:11.852 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.819138+0000", 2024-09-16T09:29:11.852 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.8361e-05 2024-09-16T09:29:11.852 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.852 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.852 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:29:11.852 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.819248+0000", 2024-09-16T09:29:11.853 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000110819 2024-09-16T09:29:11.853 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.853 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.853 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,2", 2024-09-16T09:29:11.853 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.819399+0000", 2024-09-16T09:29:11.853 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000150674 2024-09-16T09:29:11.853 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.853 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.853 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:29:11.853 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.820979+0000", 2024-09-16T09:29:11.853 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0015803589999999999 2024-09-16T09:29:11.853 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.853 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.853 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:11.853 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.821238+0000", 2024-09-16T09:29:11.854 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00025873699999999999 2024-09-16T09:29:11.854 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.854 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.854 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:11.854 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.882832+0000", 2024-09-16T09:29:11.854 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.061593649 2024-09-16T09:29:11.854 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.854 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.854 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:29:11.854 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.882879+0000", 2024-09-16T09:29:11.854 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.6879000000000003e-05 2024-09-16T09:29:11.854 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.854 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.854 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:29:11.855 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.882923+0000", 2024-09-16T09:29:11.855 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.4409999999999997e-05 2024-09-16T09:29:11.855 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:29:11.855 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:29:11.855 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:29:11.855 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.855 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.855 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:63392 7.5 7:af2804a3:::benchmark_data_smithi070_78438_object63391:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-16T09:29:11.855 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:26:46.805497+0000", 2024-09-16T09:29:11.855 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 144.95329066900001, 2024-09-16T09:29:11.855 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.064734542000000006, 2024-09-16T09:29:11.855 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:29:11.855 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:29:11.855 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:29:11.856 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:29:11.856 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:29:11.856 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:29:11.856 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 63392 2024-09-16T09:29:11.856 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.856 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:29:11.856 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.856 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:29:11.856 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:46.805497+0000", 2024-09-16T09:29:11.856 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:29:11.856 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.856 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.856 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:29:11.857 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:46.805497+0000", 2024-09-16T09:29:11.857 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:29:11.857 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.857 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.857 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:29:11.857 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:46.805498+0000", 2024-09-16T09:29:11.857 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.6789999999999999e-06 2024-09-16T09:29:11.857 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.857 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.857 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:29:11.857 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:46.805542+0000", 2024-09-16T09:29:11.857 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.3123e-05 2024-09-16T09:29:11.857 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.857 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.858 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:29:11.858 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:46.805555+0000", 2024-09-16T09:29:11.858 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.3777e-05 2024-09-16T09:29:11.858 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.858 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.858 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:29:11.858 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:46.805563+0000", 2024-09-16T09:29:11.858 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.4290000000000004e-06 2024-09-16T09:29:11.858 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.858 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.858 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:29:11.858 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:46.805618+0000", 2024-09-16T09:29:11.858 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.5559000000000001e-05 2024-09-16T09:29:11.859 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.859 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.859 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:29:11.859 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:46.805705+0000", 2024-09-16T09:29:11.859 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.7059999999999997e-05 2024-09-16T09:29:11.859 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.859 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.859 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,2", 2024-09-16T09:29:11.859 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:46.805786+0000", 2024-09-16T09:29:11.859 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.0076999999999999e-05 2024-09-16T09:29:11.859 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.859 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.859 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:29:11.860 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:46.807389+0000", 2024-09-16T09:29:11.860 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0016034160000000001 2024-09-16T09:29:11.860 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.860 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.860 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:11.860 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:46.807761+0000", 2024-09-16T09:29:11.860 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00037185799999999998 2024-09-16T09:29:11.860 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.860 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.860 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:11.860 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:46.870158+0000", 2024-09-16T09:29:11.860 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.062397059999999997 2024-09-16T09:29:11.860 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.861 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.861 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:29:11.861 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:46.870193+0000", 2024-09-16T09:29:11.861 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.4777000000000003e-05 2024-09-16T09:29:11.861 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.861 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.861 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:29:11.861 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:46.870231+0000", 2024-09-16T09:29:11.861 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.8726999999999997e-05 2024-09-16T09:29:11.861 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:29:11.861 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:29:11.861 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:29:11.861 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.861 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.861 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:202401 7.5 7:a2fb40f6:::benchmark_data_smithi070_78438_object76270:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135)", 2024-09-16T09:29:11.862 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:28:50.720099+0000", 2024-09-16T09:29:11.862 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 21.038688163, 2024-09-16T09:29:11.862 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.089978474000000003, 2024-09-16T09:29:11.862 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:29:11.862 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:29:11.862 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:29:11.862 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:29:11.862 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:29:11.862 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:29:11.862 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 202401 2024-09-16T09:29:11.862 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.862 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:29:11.862 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.862 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:29:11.863 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.720099+0000", 2024-09-16T09:29:11.863 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:29:11.863 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.863 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.863 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:29:11.863 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.720099+0000", 2024-09-16T09:29:11.863 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:29:11.863 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.863 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.863 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:29:11.863 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.720101+0000", 2024-09-16T09:29:11.863 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.5889999999999999e-06 2024-09-16T09:29:11.863 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.863 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.864 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:29:11.864 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.720107+0000", 2024-09-16T09:29:11.864 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.6520000000000003e-06 2024-09-16T09:29:11.864 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.864 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.864 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:29:11.864 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.720117+0000", 2024-09-16T09:29:11.864 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0254999999999999e-05 2024-09-16T09:29:11.864 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.864 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.864 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:29:11.864 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.720122+0000", 2024-09-16T09:29:11.865 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.2580000000000003e-06 2024-09-16T09:29:11.865 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.865 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.865 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:29:11.865 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.720178+0000", 2024-09-16T09:29:11.865 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.5501000000000003e-05 2024-09-16T09:29:11.865 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.865 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.865 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:29:11.865 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.720290+0000", 2024-09-16T09:29:11.866 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000112837 2024-09-16T09:29:11.866 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.866 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.866 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,2", 2024-09-16T09:29:11.866 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.720394+0000", 2024-09-16T09:29:11.866 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000103613 2024-09-16T09:29:11.866 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.866 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.866 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:29:11.866 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.722666+0000", 2024-09-16T09:29:11.866 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0022723800000000001 2024-09-16T09:29:11.866 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.866 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.866 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:11.867 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.725231+0000", 2024-09-16T09:29:11.867 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0025650759999999999 2024-09-16T09:29:11.867 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.867 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.867 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:11.867 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.809799+0000", 2024-09-16T09:29:11.867 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.084568014999999996 2024-09-16T09:29:11.867 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.867 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.867 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:29:11.867 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.810033+0000", 2024-09-16T09:29:11.867 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00023360000000000001 2024-09-16T09:29:11.867 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.867 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.868 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:29:11.868 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.810078+0000", 2024-09-16T09:29:11.868 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.4697999999999998e-05 2024-09-16T09:29:11.868 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:29:11.868 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:29:11.868 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:29:11.868 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.868 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.868 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:202408 7.5 7:a7ac5683:::benchmark_data_smithi070_78438_object76277:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135)", 2024-09-16T09:29:11.868 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:28:50.725540+0000", 2024-09-16T09:29:11.868 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 21.033247815999999, 2024-09-16T09:29:11.868 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.090820330000000005, 2024-09-16T09:29:11.868 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:29:11.868 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:29:11.869 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:29:11.869 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:29:11.869 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:29:11.869 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:29:11.869 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 202408 2024-09-16T09:29:11.869 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.869 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:29:11.869 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.869 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:29:11.869 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.725540+0000", 2024-09-16T09:29:11.869 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:29:11.869 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.869 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.870 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:29:11.870 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.725540+0000", 2024-09-16T09:29:11.870 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:29:11.870 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.870 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.870 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:29:11.870 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.725541+0000", 2024-09-16T09:29:11.870 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.5069999999999999e-06 2024-09-16T09:29:11.870 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.870 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.870 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:29:11.870 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.725546+0000", 2024-09-16T09:29:11.870 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.113e-06 2024-09-16T09:29:11.871 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.871 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.871 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:29:11.871 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.725558+0000", 2024-09-16T09:29:11.871 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.2164000000000001e-05 2024-09-16T09:29:11.871 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.871 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.871 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:29:11.871 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.725565+0000", 2024-09-16T09:29:11.871 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.8850000000000001e-06 2024-09-16T09:29:11.871 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.871 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.871 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:29:11.871 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.725617+0000", 2024-09-16T09:29:11.872 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.1848000000000002e-05 2024-09-16T09:29:11.872 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.872 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.872 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:29:11.872 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.725728+0000", 2024-09-16T09:29:11.872 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00011048300000000001 2024-09-16T09:29:11.872 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.872 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.872 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,2", 2024-09-16T09:29:11.872 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.725839+0000", 2024-09-16T09:29:11.872 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00011105200000000001 2024-09-16T09:29:11.872 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.872 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.872 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:11.873 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.810121+0000", 2024-09-16T09:29:11.873 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.084282652 2024-09-16T09:29:11.873 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.873 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.873 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:29:11.873 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.816197+0000", 2024-09-16T09:29:11.873 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0060759309999999997 2024-09-16T09:29:11.873 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.873 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.873 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:11.873 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.816253+0000", 2024-09-16T09:29:11.873 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.5912999999999998e-05 2024-09-16T09:29:11.873 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.873 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.874 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:29:11.874 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.816308+0000", 2024-09-16T09:29:11.874 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.4511e-05 2024-09-16T09:29:11.874 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.874 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.874 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:29:11.874 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.816360+0000", 2024-09-16T09:29:11.874 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.2271000000000001e-05 2024-09-16T09:29:11.874 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:29:11.874 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:29:11.874 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:29:11.874 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.874 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.874 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24424.0:218 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch (1) in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e134)", 2024-09-16T09:29:11.875 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:28:50.729011+0000", 2024-09-16T09:29:11.875 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 21.029776687999998, 2024-09-16T09:29:11.875 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.078508791999999994, 2024-09-16T09:29:11.875 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:29:11.875 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:29:11.875 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "started", 2024-09-16T09:29:11.875 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:29:11.875 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24424", 2024-09-16T09:29:11.875 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.84:0/1181633158", 2024-09-16T09:29:11.875 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 218 2024-09-16T09:29:11.875 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.875 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:29:11.875 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.875 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:29:11.876 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.729011+0000", 2024-09-16T09:29:11.876 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:29:11.876 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.876 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.876 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:29:11.876 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.729011+0000", 2024-09-16T09:29:11.876 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:29:11.876 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.876 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.876 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:29:11.876 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.729012+0000", 2024-09-16T09:29:11.876 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.6270000000000001e-06 2024-09-16T09:29:11.876 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.876 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.877 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:29:11.877 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.729019+0000", 2024-09-16T09:29:11.877 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.8889999999999996e-06 2024-09-16T09:29:11.877 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.877 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.877 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:29:11.877 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.806924+0000", 2024-09-16T09:29:11.877 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.077904235000000002 2024-09-16T09:29:11.877 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.877 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.877 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:29:11.877 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.806944+0000", 2024-09-16T09:29:11.877 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.039e-05 2024-09-16T09:29:11.877 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.878 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.878 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:29:11.878 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.807345+0000", 2024-09-16T09:29:11.878 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00040116499999999999 2024-09-16T09:29:11.878 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.878 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.878 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:29:11.878 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.807432+0000", 2024-09-16T09:29:11.878 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.6515999999999996e-05 2024-09-16T09:29:11.878 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.878 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.878 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:29:11.878 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.807520+0000", 2024-09-16T09:29:11.878 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.797e-05 2024-09-16T09:29:11.878 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:29:11.879 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:29:11.879 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:29:11.879 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.879 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.879 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_repop_reply(client.24698.0:202401 7.5 e135/82 ondisk, result = 0)", 2024-09-16T09:29:11.879 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:28:50.730604+0000", 2024-09-16T09:29:11.879 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 21.028183557999998, 2024-09-16T09:29:11.879 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.079479822000000006, 2024-09-16T09:29:11.879 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:29:11.879 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:29:11.879 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "started", 2024-09-16T09:29:11.879 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:29:11.879 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.879 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:29:11.879 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.730604+0000", 2024-09-16T09:29:11.880 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:29:11.880 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.880 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.880 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:29:11.880 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.730604+0000", 2024-09-16T09:29:11.880 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:29:11.880 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.880 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:29:11.882 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.882 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:29:11.882 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.730605+0000", 2024-09-16T09:29:11.882 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.0999999999999997e-07 2024-09-16T09:29:11.882 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.882 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.882 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:29:11.882 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.730609+0000", 2024-09-16T09:29:11.882 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.9929999999999997e-06 2024-09-16T09:29:11.882 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.882 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.883 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:29:11.883 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.806968+0000", 2024-09-16T09:29:11.883 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.076359716999999994 2024-09-16T09:29:11.883 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.883 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.883 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:29:11.883 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.806980+0000", 2024-09-16T09:29:11.883 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1551e-05 2024-09-16T09:29:11.883 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.883 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.883 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:29:11.883 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.809786+0000", 2024-09-16T09:29:11.883 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0028062339999999999 2024-09-16T09:29:11.884 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.884 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.884 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:29:11.884 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.809794+0000", 2024-09-16T09:29:11.884 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.4630000000000004e-06 2024-09-16T09:29:11.884 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.884 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.884 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:29:11.884 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.810084+0000", 2024-09-16T09:29:11.884 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00029005399999999998 2024-09-16T09:29:11.884 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:29:11.884 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:29:11.884 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:29:11.884 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.885 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.885 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:202411 7.5 7:a7b52676:::benchmark_data_smithi070_78438_object76280:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135)", 2024-09-16T09:29:11.885 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:28:50.732821+0000", 2024-09-16T09:29:11.885 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 21.025966523000001, 2024-09-16T09:29:11.885 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.10429561900000001, 2024-09-16T09:29:11.885 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:29:11.885 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:29:11.885 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:29:11.885 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:29:11.885 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:29:11.885 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:29:11.885 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 202411 2024-09-16T09:29:11.885 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.886 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:29:11.886 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.886 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:29:11.886 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.732821+0000", 2024-09-16T09:29:11.886 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:29:11.886 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.886 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.886 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:29:11.886 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.732821+0000", 2024-09-16T09:29:11.886 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:29:11.886 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.886 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.886 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:29:11.886 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.732822+0000", 2024-09-16T09:29:11.887 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.46e-06 2024-09-16T09:29:11.887 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.887 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.887 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:29:11.887 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.732828+0000", 2024-09-16T09:29:11.887 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.4240000000000001e-06 2024-09-16T09:29:11.887 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.887 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.887 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:29:11.887 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.806990+0000", 2024-09-16T09:29:11.887 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.074162464999999997 2024-09-16T09:29:11.887 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.887 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.887 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:29:11.888 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.806998+0000", 2024-09-16T09:29:11.888 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.0369999999999995e-06 2024-09-16T09:29:11.888 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.888 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.888 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:29:11.888 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.810149+0000", 2024-09-16T09:29:11.888 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0031503339999999999 2024-09-16T09:29:11.888 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.888 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.888 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:29:11.888 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.810253+0000", 2024-09-16T09:29:11.888 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00010388200000000001 2024-09-16T09:29:11.888 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.888 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.889 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,2", 2024-09-16T09:29:11.889 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.810356+0000", 2024-09-16T09:29:11.889 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000103319 2024-09-16T09:29:11.889 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.889 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.889 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:29:11.889 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.816207+0000", 2024-09-16T09:29:11.889 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.005850984 2024-09-16T09:29:11.889 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.889 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.889 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:11.889 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.828515+0000", 2024-09-16T09:29:11.889 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.012307904 2024-09-16T09:29:11.889 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.890 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.890 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:11.890 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.837029+0000", 2024-09-16T09:29:11.890 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0085145419999999999 2024-09-16T09:29:11.890 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.890 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.890 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:29:11.890 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.837071+0000", 2024-09-16T09:29:11.890 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.1718999999999998e-05 2024-09-16T09:29:11.890 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.890 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.890 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:29:11.890 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.837117+0000", 2024-09-16T09:29:11.890 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.5549e-05 2024-09-16T09:29:11.891 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:29:11.891 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:29:11.891 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:29:11.891 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.891 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.891 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:204901 7.5 7:af219aea:::benchmark_data_smithi070_78438_object78770:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137)", 2024-09-16T09:29:11.891 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:28:54.030373+0000", 2024-09-16T09:29:11.891 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 17.728414853, 2024-09-16T09:29:11.891 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.077126564999999994, 2024-09-16T09:29:11.891 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:29:11.891 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:29:11.891 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:29:11.891 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:29:11.891 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:29:11.892 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:29:11.892 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 204901 2024-09-16T09:29:11.892 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.892 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:29:11.892 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.892 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:29:11.892 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.030373+0000", 2024-09-16T09:29:11.892 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:29:11.892 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.892 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.892 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:29:11.892 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.030373+0000", 2024-09-16T09:29:11.892 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:29:11.892 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.893 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.893 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:29:11.893 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.030374+0000", 2024-09-16T09:29:11.893 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.389e-06 2024-09-16T09:29:11.893 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.893 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.893 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:29:11.893 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.030378+0000", 2024-09-16T09:29:11.893 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.1679999999999997e-06 2024-09-16T09:29:11.893 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.893 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.893 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:29:11.893 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.030389+0000", 2024-09-16T09:29:11.893 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.097e-05 2024-09-16T09:29:11.894 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.894 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.894 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:29:11.894 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.030395+0000", 2024-09-16T09:29:11.894 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.229e-06 2024-09-16T09:29:11.894 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.894 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.894 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:29:11.894 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.030446+0000", 2024-09-16T09:29:11.894 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.0757999999999998e-05 2024-09-16T09:29:11.894 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.894 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.894 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:29:11.894 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.030553+0000", 2024-09-16T09:29:11.895 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00010718900000000001 2024-09-16T09:29:11.895 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.895 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.895 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,2", 2024-09-16T09:29:11.895 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.030664+0000", 2024-09-16T09:29:11.895 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00011086600000000001 2024-09-16T09:29:11.895 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.895 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.895 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:29:11.895 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.033351+0000", 2024-09-16T09:29:11.895 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.002686875 2024-09-16T09:29:11.895 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.895 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.895 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:11.896 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.033716+0000", 2024-09-16T09:29:11.896 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00036531200000000002 2024-09-16T09:29:11.896 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.896 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.896 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:11.896 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.107394+0000", 2024-09-16T09:29:11.896 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.073677379000000001 2024-09-16T09:29:11.896 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.896 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.896 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:29:11.896 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.107450+0000", 2024-09-16T09:29:11.896 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.6351999999999999e-05 2024-09-16T09:29:11.896 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.897 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.897 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:29:11.897 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.107499+0000", 2024-09-16T09:29:11.897 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.9078e-05 2024-09-16T09:29:11.897 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:29:11.897 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:29:11.897 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:29:11.897 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.897 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.897 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:204910 7.5 7:a16392ce:::benchmark_data_smithi070_78438_object78779:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137)", 2024-09-16T09:29:11.897 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:28:54.036796+0000", 2024-09-16T09:29:11.897 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 17.721991375999998, 2024-09-16T09:29:11.897 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.070834338999999996, 2024-09-16T09:29:11.897 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:29:11.898 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:29:11.898 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:29:11.898 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:29:11.898 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:29:11.898 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:29:11.898 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 204910 2024-09-16T09:29:11.898 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.898 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:29:11.898 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.898 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:29:11.898 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.036796+0000", 2024-09-16T09:29:11.898 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:29:11.898 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.898 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.899 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:29:11.899 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.036796+0000", 2024-09-16T09:29:11.899 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:29:11.899 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.899 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.899 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:29:11.899 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.036798+0000", 2024-09-16T09:29:11.899 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.649e-06 2024-09-16T09:29:11.899 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.899 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.899 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:29:11.899 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.036805+0000", 2024-09-16T09:29:11.899 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.0840000000000003e-06 2024-09-16T09:29:11.899 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.900 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.900 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:29:11.900 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.036824+0000", 2024-09-16T09:29:11.900 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.9318e-05 2024-09-16T09:29:11.900 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.900 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.900 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:29:11.900 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.036832+0000", 2024-09-16T09:29:11.900 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.4000000000000003e-06 2024-09-16T09:29:11.900 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.900 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.900 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:29:11.900 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.036897+0000", 2024-09-16T09:29:11.901 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.5238000000000001e-05 2024-09-16T09:29:11.901 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.901 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.901 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:29:11.901 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.037029+0000", 2024-09-16T09:29:11.901 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00013190399999999999 2024-09-16T09:29:11.901 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.901 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.901 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,2", 2024-09-16T09:29:11.901 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.037149+0000", 2024-09-16T09:29:11.901 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000119833 2024-09-16T09:29:11.901 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.901 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.902 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:29:11.902 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.039175+0000", 2024-09-16T09:29:11.902 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0020263389999999998 2024-09-16T09:29:11.902 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.902 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.902 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:11.902 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.106199+0000", 2024-09-16T09:29:11.902 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.067023694999999994 2024-09-16T09:29:11.902 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.902 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.902 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:11.902 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.107546+0000", 2024-09-16T09:29:11.902 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.001347902 2024-09-16T09:29:11.902 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.903 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.903 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:29:11.903 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.107593+0000", 2024-09-16T09:29:11.903 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.6736000000000001e-05 2024-09-16T09:29:11.903 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.903 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.903 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:29:11.903 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.107630+0000", 2024-09-16T09:29:11.903 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.7240999999999998e-05 2024-09-16T09:29:11.903 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:29:11.903 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:29:11.903 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:29:11.903 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.903 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.904 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:215340 7.5 7:a4f699d2:::benchmark_data_smithi070_78438_object89209:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e141)", 2024-09-16T09:29:11.904 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:29:03.365151+0000", 2024-09-16T09:29:11.904 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 8.3936365550000005, 2024-09-16T09:29:11.904 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.068201946999999999, 2024-09-16T09:29:11.904 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:29:11.904 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:29:11.904 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:29:11.904 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:29:11.904 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:29:11.904 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:29:11.904 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 215340 2024-09-16T09:29:11.904 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.904 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:29:11.905 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.905 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:29:11.905 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.365151+0000", 2024-09-16T09:29:11.905 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:29:11.905 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.905 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.905 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:29:11.905 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.365151+0000", 2024-09-16T09:29:11.905 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:29:11.905 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.905 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.905 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:29:11.905 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.365153+0000", 2024-09-16T09:29:11.905 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.0540000000000002e-06 2024-09-16T09:29:11.905 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.906 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.906 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:29:11.906 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.365162+0000", 2024-09-16T09:29:11.906 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.38e-06 2024-09-16T09:29:11.906 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.906 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.906 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:29:11.906 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.365186+0000", 2024-09-16T09:29:11.906 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.3986e-05 2024-09-16T09:29:11.906 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.906 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.906 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:29:11.906 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.365200+0000", 2024-09-16T09:29:11.906 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.3586e-05 2024-09-16T09:29:11.907 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.907 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.907 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:29:11.907 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.365312+0000", 2024-09-16T09:29:11.907 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000112422 2024-09-16T09:29:11.907 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.907 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.907 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op must wait for map", 2024-09-16T09:29:11.907 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.365322+0000", 2024-09-16T09:29:11.907 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.5289999999999997e-06 2024-09-16T09:29:11.907 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.907 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.907 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:29:11.907 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.422102+0000", 2024-09-16T09:29:11.908 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.056780023999999998 2024-09-16T09:29:11.908 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.908 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.908 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:29:11.908 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.422202+0000", 2024-09-16T09:29:11.908 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.9617999999999994e-05 2024-09-16T09:29:11.908 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.908 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.908 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,2", 2024-09-16T09:29:11.908 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.422578+0000", 2024-09-16T09:29:11.908 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00037663600000000002 2024-09-16T09:29:11.908 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.908 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.908 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:11.909 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.431303+0000", 2024-09-16T09:29:11.909 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0087248670000000007 2024-09-16T09:29:11.909 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.909 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.909 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:11.909 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.431404+0000", 2024-09-16T09:29:11.909 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00010138500000000001 2024-09-16T09:29:11.909 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.909 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.909 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:29:11.909 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.433215+0000", 2024-09-16T09:29:11.909 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0018102979999999999 2024-09-16T09:29:11.909 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.909 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.910 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:29:11.910 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.433306+0000", 2024-09-16T09:29:11.910 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.1774999999999995e-05 2024-09-16T09:29:11.910 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.910 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.910 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:29:11.910 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.433353+0000", 2024-09-16T09:29:11.910 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.6387000000000002e-05 2024-09-16T09:29:11.910 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:29:11.910 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:29:11.910 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:29:11.910 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.910 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.910 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:215342 7.5 7:a84e8ca2:::benchmark_data_smithi070_78438_object89211:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e141)", 2024-09-16T09:29:11.911 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:29:03.366631+0000", 2024-09-16T09:29:11.911 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 8.3921559769999998, 2024-09-16T09:29:11.911 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.074197923999999998, 2024-09-16T09:29:11.911 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:29:11.911 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:29:11.911 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:29:11.911 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:29:11.911 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:29:11.911 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:29:11.911 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 215342 2024-09-16T09:29:11.911 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.911 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:29:11.911 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.911 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:29:11.912 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.366631+0000", 2024-09-16T09:29:11.912 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:29:11.912 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.912 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.912 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:29:11.912 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.366631+0000", 2024-09-16T09:29:11.912 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:29:11.912 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.912 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.912 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:29:11.912 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.366633+0000", 2024-09-16T09:29:11.912 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.2839999999999999e-06 2024-09-16T09:29:11.912 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.912 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.913 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:29:11.913 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.366640+0000", 2024-09-16T09:29:11.913 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.6329999999999996e-06 2024-09-16T09:29:11.913 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.913 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.913 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:29:11.913 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.420402+0000", 2024-09-16T09:29:11.913 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.053761807000000002 2024-09-16T09:29:11.913 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.913 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.913 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:29:11.913 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.420414+0000", 2024-09-16T09:29:11.913 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1885000000000001e-05 2024-09-16T09:29:11.913 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.914 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.914 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:29:11.914 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.426991+0000", 2024-09-16T09:29:11.914 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0065772399999999998 2024-09-16T09:29:11.914 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.914 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.914 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:29:11.914 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.427091+0000", 2024-09-16T09:29:11.914 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.9407999999999994e-05 2024-09-16T09:29:11.914 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.914 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.914 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,2", 2024-09-16T09:29:11.914 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.427202+0000", 2024-09-16T09:29:11.914 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000111298 2024-09-16T09:29:11.914 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.915 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.915 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:29:11.915 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.433361+0000", 2024-09-16T09:29:11.915 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0061589390000000004 2024-09-16T09:29:11.915 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.915 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.915 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:11.915 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.437028+0000", 2024-09-16T09:29:11.915 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0036673130000000002 2024-09-16T09:29:11.915 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.915 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.915 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:11.915 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.440723+0000", 2024-09-16T09:29:11.916 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0036951480000000001 2024-09-16T09:29:11.916 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.916 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.916 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:29:11.916 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.440791+0000", 2024-09-16T09:29:11.916 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.7496000000000006e-05 2024-09-16T09:29:11.916 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:29:11.916 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:29:11.916 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:29:11.916 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.440829+0000", 2024-09-16T09:29:11.916 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.8473000000000002e-05 2024-09-16T09:29:11.916 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:29:11.916 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:29:11.916 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:29:11.917 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:29:11.917 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:29:11.917 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-09-16T09:29:11.920 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:11.920 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:11.921 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:29:12.022 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:29:12.061 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:12.062 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:12.063 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:29:12.072 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:29:11 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:29:11] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:29:12.164 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:29:12.216 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:12.217 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:12.217 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:29:12.244 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:29:12.319 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:29:12.355 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd unset noscrub 2024-09-16T09:29:12.373 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:12.374 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:12.374 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:29:12.384 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:12 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/720020972' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:29:12.385 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:12 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/720020972' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:29:12.409 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:12 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/720020972' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:29:12.475 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:29:12.525 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:12.526 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:12.527 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:29:12.628 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:29:12.672 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:12.673 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:12.673 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:29:12.774 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:29:12.805 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:12.806 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:12.806 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:29:12.907 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:29:12.932 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:12.932 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:12.933 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:29:12.991 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:29:12 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:29:12.997 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:29:13.030 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.0 dump_ops_in_flight 2024-09-16T09:29:13.033 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:29:13.059 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:13.060 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:13.060 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:29:13.161 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:29:13.186 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:13.187 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:13.187 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:29:13.240 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:29:13.242 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2024-09-16T09:29:13.288 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:29:13.325 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:13 smithi070 ceph-mon[30388]: pgmap v182: 158 pgs: 158 active+clean; 2.0 GiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 624 B/s rd, 0 B/s wr, 674 op/s 2024-09-16T09:29:13.326 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:13 smithi070 ceph-mon[40691]: pgmap v182: 158 pgs: 158 active+clean; 2.0 GiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 624 B/s rd, 0 B/s wr, 674 op/s 2024-09-16T09:29:13.347 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:13.348 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:13.348 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:29:13.409 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:13 smithi084 ceph-mon[32629]: pgmap v182: 158 pgs: 158 active+clean; 2.0 GiB data, 6.9 GiB used, 708 GiB / 715 GiB avail; 624 B/s rd, 0 B/s wr, 674 op/s 2024-09-16T09:29:13.449 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:29:13.491 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:13.491 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:13.491 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:29:13.592 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:29:13.619 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:13.619 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:13.620 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:29:13.720 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:29:13.764 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:13.765 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:13.765 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:29:13.866 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:29:13.904 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:13.904 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:13.905 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:29:14.008 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:29:14.032 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":182,"stamp":"2024-09-16T09:29:11.558336+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2107316787,"num_objects":32346,"num_object_clones":0,"num_object_copies":97038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32346,"num_whiteouts":0,"num_read":958,"num_read_kb":710,"num_write":220547,"num_write_kb":8072238,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":85822,"ondisk_log_size":85822,"up":474,"acting":474,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":474,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":7193832,"kb_used_data":6291496,"kb_used_omap":12,"kb_used_meta":901811,"kb_avail":742767384,"statfs":{"total":767960285184,"available":760593801216,"internally_reserved":0,"allocated":6442491904,"data_stored":6439814569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":923455026},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,4,6,2,6,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-362217472,"num_objects":-5527,"num_object_clones":0,"num_object_copies":-16581,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5527,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5527,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.204584"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":193,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.364224+0000","last_change":"2024-09-16T09:28:29.639297+0000","last_active":"2024-09-16T09:29:03.364224+0000","last_peered":"2024-09-16T09:29:03.364224+0000","last_clean":"2024-09-16T09:29:03.364224+0000","last_became_active":"2024-09-16T09:25:01.709108+0000","last_became_peered":"2024-09-16T09:25:01.709108+0000","last_unstale":"2024-09-16T09:29:03.364224+0000","last_undegraded":"2024-09-16T09:29:03.364224+0000","last_fullsized":"2024-09-16T09:29:03.364224+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:29.639071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:28:29.639071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:38.316022+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"66'9","reported_seq":205,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.373255+0000","last_change":"2024-09-16T09:28:52.689373+0000","last_active":"2024-09-16T09:29:03.373255+0000","last_peered":"2024-09-16T09:29:03.373255+0000","last_clean":"2024-09-16T09:29:03.373255+0000","last_became_active":"2024-09-16T09:25:03.714635+0000","last_became_peered":"2024-09-16T09:25:03.714635+0000","last_unstale":"2024-09-16T09:29:03.373255+0000","last_undegraded":"2024-09-16T09:29:03.373255+0000","last_fullsized":"2024-09-16T09:29:03.373255+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:28:52.689304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:52.689304+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:21.292656+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":185,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.356809+0000","last_change":"2024-09-16T09:27:58.014601+0000","last_active":"2024-09-16T09:29:03.356809+0000","last_peered":"2024-09-16T09:29:03.356809+0000","last_clean":"2024-09-16T09:29:03.356809+0000","last_became_active":"2024-09-16T09:25:05.715645+0000","last_became_peered":"2024-09-16T09:25:05.715645+0000","last_unstale":"2024-09-16T09:29:03.356809+0000","last_undegraded":"2024-09-16T09:29:03.356809+0000","last_fullsized":"2024-09-16T09:29:03.356809+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:58.014182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:58.014182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:06.935933+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":173,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.373270+0000","last_change":"2024-09-16T09:27:11.565306+0000","last_active":"2024-09-16T09:29:03.373270+0000","last_peered":"2024-09-16T09:29:03.373270+0000","last_clean":"2024-09-16T09:29:03.373270+0000","last_became_active":"2024-09-16T09:25:07.715442+0000","last_became_peered":"2024-09-16T09:25:07.715442+0000","last_unstale":"2024-09-16T09:29:03.373270+0000","last_undegraded":"2024-09-16T09:29:03.373270+0000","last_fullsized":"2024-09-16T09:29:03.373270+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:11.565220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:11.565220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:36.627819+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":173,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.359302+0000","last_change":"2024-09-16T09:28:00.068631+0000","last_active":"2024-09-16T09:29:03.359302+0000","last_peered":"2024-09-16T09:29:03.359302+0000","last_clean":"2024-09-16T09:29:03.359302+0000","last_became_active":"2024-09-16T09:25:07.718126+0000","last_became_peered":"2024-09-16T09:25:07.718126+0000","last_unstale":"2024-09-16T09:29:03.359302+0000","last_undegraded":"2024-09-16T09:29:03.359302+0000","last_fullsized":"2024-09-16T09:29:03.359302+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:00.068570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:28:00.068570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.409320+0000","last_change":"2024-09-16T09:28:52.975560+0000","last_active":"2024-09-16T09:29:03.409320+0000","last_peered":"2024-09-16T09:29:03.409320+0000","last_clean":"2024-09-16T09:29:03.409320+0000","last_became_active":"2024-09-16T09:28:52.975294+0000","last_became_peered":"2024-09-16T09:28:52.975294+0000","last_unstale":"2024-09-16T09:29:03.409320+0000","last_undegraded":"2024-09-16T09:29:03.409320+0000","last_fullsized":"2024-09-16T09:29:03.409320+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:54.078473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:27:54.078473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:12.438302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.19","version":"66'15","reported_seq":66,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.365944+0000","last_change":"2024-09-16T09:28:52.983190+0000","last_active":"2024-09-16T09:29:03.365944+0000","last_peered":"2024-09-16T09:29:03.365944+0000","last_clean":"2024-09-16T09:29:03.365944+0000","last_became_active":"2024-09-16T09:28:52.982553+0000","last_became_peered":"2024-09-16T09:28:52.982553+0000","last_unstale":"2024-09-16T09:29:03.365944+0000","last_undegraded":"2024-09-16T09:29:03.365944+0000","last_fullsized":"2024-09-16T09:29:03.365944+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-16T09:28:27.126326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:27.126326+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:53.195556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":185,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.373189+0000","last_change":"2024-09-16T09:27:49.653058+0000","last_active":"2024-09-16T09:29:03.373189+0000","last_peered":"2024-09-16T09:29:03.373189+0000","last_clean":"2024-09-16T09:29:03.373189+0000","last_became_active":"2024-09-16T09:25:05.717236+0000","last_became_peered":"2024-09-16T09:25:05.717236+0000","last_unstale":"2024-09-16T09:29:03.373189+0000","last_undegraded":"2024-09-16T09:29:03.373189+0000","last_fullsized":"2024-09-16T09:29:03.373189+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:49.653015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:49.653015+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:09.644746+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":32,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.356191+0000","last_change":"2024-09-16T09:28:34.839909+0000","last_active":"2024-09-16T09:29:03.356191+0000","last_peered":"2024-09-16T09:29:03.356191+0000","last_clean":"2024-09-16T09:29:03.356191+0000","last_became_active":"2024-09-16T09:28:34.839435+0000","last_became_peered":"2024-09-16T09:28:34.839435+0000","last_unstale":"2024-09-16T09:29:03.356191+0000","last_undegraded":"2024-09-16T09:29:03.356191+0000","last_fullsized":"2024-09-16T09:29:03.356191+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:01.038400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:28:01.038400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:17.493355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"66'9","reported_seq":197,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.373420+0000","last_change":"2024-09-16T09:27:59.685896+0000","last_active":"2024-09-16T09:29:03.373420+0000","last_peered":"2024-09-16T09:29:03.373420+0000","last_clean":"2024-09-16T09:29:03.373420+0000","last_became_active":"2024-09-16T09:25:03.709147+0000","last_became_peered":"2024-09-16T09:25:03.709147+0000","last_unstale":"2024-09-16T09:29:03.373420+0000","last_undegraded":"2024-09-16T09:29:03.373420+0000","last_fullsized":"2024-09-16T09:29:03.373420+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:27:59.685837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:59.685837+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:25.373463+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":176,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.418024+0000","last_change":"2024-09-16T09:27:59.080369+0000","last_active":"2024-09-16T09:29:03.418024+0000","last_peered":"2024-09-16T09:29:03.418024+0000","last_clean":"2024-09-16T09:29:03.418024+0000","last_became_active":"2024-09-16T09:25:05.718134+0000","last_became_peered":"2024-09-16T09:25:05.718134+0000","last_unstale":"2024-09-16T09:29:03.418024+0000","last_undegraded":"2024-09-16T09:29:03.418024+0000","last_fullsized":"2024-09-16T09:29:03.418024+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:59.080307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:59.080307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:26.458257+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"66'1","reported_seq":174,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.362225+0000","last_change":"2024-09-16T09:26:24.547980+0000","last_active":"2024-09-16T09:29:03.362225+0000","last_peered":"2024-09-16T09:29:03.362225+0000","last_clean":"2024-09-16T09:29:03.362225+0000","last_became_active":"2024-09-16T09:25:07.716558+0000","last_became_peered":"2024-09-16T09:25:07.716558+0000","last_unstale":"2024-09-16T09:29:03.362225+0000","last_undegraded":"2024-09-16T09:29:03.362225+0000","last_fullsized":"2024-09-16T09:29:03.362225+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-16T09:26:24.547877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:24.547877+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:15.542611+0000","scrub_duration":52,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":184,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.409451+0000","last_change":"2024-09-16T09:27:56.191264+0000","last_active":"2024-09-16T09:29:03.409451+0000","last_peered":"2024-09-16T09:29:03.409451+0000","last_clean":"2024-09-16T09:29:03.409451+0000","last_became_active":"2024-09-16T09:25:01.696939+0000","last_became_peered":"2024-09-16T09:25:01.696939+0000","last_unstale":"2024-09-16T09:29:03.409451+0000","last_undegraded":"2024-09-16T09:29:03.409451+0000","last_fullsized":"2024-09-16T09:29:03.409451+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:56.191220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:27:56.191220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:15.886906+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"66'5","reported_seq":191,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.373352+0000","last_change":"2024-09-16T09:27:05.694733+0000","last_active":"2024-09-16T09:29:03.373352+0000","last_peered":"2024-09-16T09:29:03.373352+0000","last_clean":"2024-09-16T09:29:03.373352+0000","last_became_active":"2024-09-16T09:25:03.716136+0000","last_became_peered":"2024-09-16T09:25:03.716136+0000","last_unstale":"2024-09-16T09:29:03.373352+0000","last_undegraded":"2024-09-16T09:29:03.373352+0000","last_fullsized":"2024-09-16T09:29:03.373352+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-16T09:27:05.694650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:05.694650+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:39.677272+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":184,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.417630+0000","last_change":"2024-09-16T09:28:52.746046+0000","last_active":"2024-09-16T09:29:03.417630+0000","last_peered":"2024-09-16T09:29:03.417630+0000","last_clean":"2024-09-16T09:29:03.417630+0000","last_became_active":"2024-09-16T09:25:05.718281+0000","last_became_peered":"2024-09-16T09:25:05.718281+0000","last_unstale":"2024-09-16T09:29:03.417630+0000","last_undegraded":"2024-09-16T09:29:03.417630+0000","last_fullsized":"2024-09-16T09:29:03.417630+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:52.745999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:28:52.745999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:22.253990+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":180,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.408918+0000","last_change":"2024-09-16T09:28:25.246140+0000","last_active":"2024-09-16T09:29:03.408918+0000","last_peered":"2024-09-16T09:29:03.408918+0000","last_clean":"2024-09-16T09:29:03.408918+0000","last_became_active":"2024-09-16T09:25:07.716724+0000","last_became_peered":"2024-09-16T09:25:07.716724+0000","last_unstale":"2024-09-16T09:29:03.408918+0000","last_undegraded":"2024-09-16T09:29:03.408918+0000","last_fullsized":"2024-09-16T09:29:03.408918+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:25.246067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:28:25.246067+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:46.649209+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":181,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.417137+0000","last_change":"2024-09-16T09:28:20.662726+0000","last_active":"2024-09-16T09:29:03.417137+0000","last_peered":"2024-09-16T09:29:03.417137+0000","last_clean":"2024-09-16T09:29:03.417137+0000","last_became_active":"2024-09-16T09:25:07.720793+0000","last_became_peered":"2024-09-16T09:25:07.720793+0000","last_unstale":"2024-09-16T09:29:03.417137+0000","last_undegraded":"2024-09-16T09:29:03.417137+0000","last_fullsized":"2024-09-16T09:29:03.417137+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:20.662663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:28:20.662663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:46.306136+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":185,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.362482+0000","last_change":"2024-09-16T09:27:09.635443+0000","last_active":"2024-09-16T09:29:03.362482+0000","last_peered":"2024-09-16T09:29:03.362482+0000","last_clean":"2024-09-16T09:29:03.362482+0000","last_became_active":"2024-09-16T09:25:01.706172+0000","last_became_peered":"2024-09-16T09:25:01.706172+0000","last_unstale":"2024-09-16T09:29:03.362482+0000","last_undegraded":"2024-09-16T09:29:03.362482+0000","last_fullsized":"2024-09-16T09:29:03.362482+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:09.635363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:27:09.635363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:38.538435+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"66'15","reported_seq":214,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.366227+0000","last_change":"2024-09-16T09:29:03.155623+0000","last_active":"2024-09-16T09:29:03.366227+0000","last_peered":"2024-09-16T09:29:03.366227+0000","last_clean":"2024-09-16T09:29:03.366227+0000","last_became_active":"2024-09-16T09:25:03.713780+0000","last_became_peered":"2024-09-16T09:25:03.713780+0000","last_unstale":"2024-09-16T09:29:03.366227+0000","last_undegraded":"2024-09-16T09:29:03.366227+0000","last_fullsized":"2024-09-16T09:29:03.366227+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-16T09:29:03.155568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:29:03.155568+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:30.605184+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":177,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.358282+0000","last_change":"2024-09-16T09:27:16.266378+0000","last_active":"2024-09-16T09:29:03.358282+0000","last_peered":"2024-09-16T09:29:03.358282+0000","last_clean":"2024-09-16T09:29:03.358282+0000","last_became_active":"2024-09-16T09:25:05.713929+0000","last_became_peered":"2024-09-16T09:25:05.713929+0000","last_unstale":"2024-09-16T09:29:03.358282+0000","last_undegraded":"2024-09-16T09:29:03.358282+0000","last_fullsized":"2024-09-16T09:29:03.358282+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:16.266297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:16.266297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:44.686289+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":30,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.419251+0000","last_change":"2024-09-16T09:28:53.566347+0000","last_active":"2024-09-16T09:29:03.419251+0000","last_peered":"2024-09-16T09:29:03.419251+0000","last_clean":"2024-09-16T09:29:03.419251+0000","last_became_active":"2024-09-16T09:28:52.974255+0000","last_became_peered":"2024-09-16T09:28:52.974255+0000","last_unstale":"2024-09-16T09:29:03.419251+0000","last_undegraded":"2024-09-16T09:29:03.419251+0000","last_fullsized":"2024-09-16T09:29:03.419251+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:53.566299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:28:53.566299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:23.088282+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":193,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.416318+0000","last_change":"2024-09-16T09:28:31.565260+0000","last_active":"2024-09-16T09:29:03.416318+0000","last_peered":"2024-09-16T09:29:03.416318+0000","last_clean":"2024-09-16T09:29:03.416318+0000","last_became_active":"2024-09-16T09:25:01.706491+0000","last_became_peered":"2024-09-16T09:25:01.706491+0000","last_unstale":"2024-09-16T09:29:03.416318+0000","last_undegraded":"2024-09-16T09:29:03.416318+0000","last_fullsized":"2024-09-16T09:29:03.416318+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:31.565184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:28:31.565184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:51.771792+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"66'12","reported_seq":64,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.358424+0000","last_change":"2024-09-16T09:28:52.981384+0000","last_active":"2024-09-16T09:29:03.358424+0000","last_peered":"2024-09-16T09:29:03.358424+0000","last_clean":"2024-09-16T09:29:03.358424+0000","last_became_active":"2024-09-16T09:28:52.981011+0000","last_became_peered":"2024-09-16T09:28:52.981011+0000","last_unstale":"2024-09-16T09:29:03.358424+0000","last_undegraded":"2024-09-16T09:29:03.358424+0000","last_fullsized":"2024-09-16T09:29:03.358424+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-16T09:27:01.700139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:01.700139+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":24,"num_read_kb":15,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":177,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.373412+0000","last_change":"2024-09-16T09:27:00.816500+0000","last_active":"2024-09-16T09:29:03.373412+0000","last_peered":"2024-09-16T09:29:03.373412+0000","last_clean":"2024-09-16T09:29:03.373412+0000","last_became_active":"2024-09-16T09:25:05.710093+0000","last_became_peered":"2024-09-16T09:25:05.710093+0000","last_unstale":"2024-09-16T09:29:03.373412+0000","last_undegraded":"2024-09-16T09:29:03.373412+0000","last_fullsized":"2024-09-16T09:29:03.373412+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:00.816424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:00.816424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:37.654650+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":173,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.416841+0000","last_change":"2024-09-16T09:27:59.801997+0000","last_active":"2024-09-16T09:29:03.416841+0000","last_peered":"2024-09-16T09:29:03.416841+0000","last_clean":"2024-09-16T09:29:03.416841+0000","last_became_active":"2024-09-16T09:25:07.719380+0000","last_became_peered":"2024-09-16T09:25:07.719380+0000","last_unstale":"2024-09-16T09:29:03.416841+0000","last_undegraded":"2024-09-16T09:29:03.416841+0000","last_fullsized":"2024-09-16T09:29:03.416841+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-16T09:27:59.801924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:59.801924+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:26.228249+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":194,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.356905+0000","last_change":"2024-09-16T09:28:17.993009+0000","last_active":"2024-09-16T09:29:03.356905+0000","last_peered":"2024-09-16T09:29:03.356905+0000","last_clean":"2024-09-16T09:29:03.356905+0000","last_became_active":"2024-09-16T09:25:01.699545+0000","last_became_peered":"2024-09-16T09:25:01.699545+0000","last_unstale":"2024-09-16T09:29:03.356905+0000","last_undegraded":"2024-09-16T09:29:03.356905+0000","last_fullsized":"2024-09-16T09:29:03.356905+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:28:17.992925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:28:17.992925+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:18.647927+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"66'10","reported_seq":205,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.364397+0000","last_change":"2024-09-16T09:27:43.556435+0000","last_active":"2024-09-16T09:29:03.364397+0000","last_peered":"2024-09-16T09:29:03.364397+0000","last_clean":"2024-09-16T09:29:03.364397+0000","last_became_active":"2024-09-16T09:25:03.720245+0000","last_became_peered":"2024-09-16T09:25:03.720245+0000","last_unstale":"2024-09-16T09:29:03.364397+0000","last_undegraded":"2024-09-16T09:29:03.364397+0000","last_fullsized":"2024-09-16T09:29:03.364397+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-16T09:27:43.556357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:43.556357+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:13.459515+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"66'8","reported_seq":254,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.419497+0000","last_change":"2024-09-16T09:28:58.597645+0000","last_active":"2024-09-16T09:29:03.419497+0000","last_peered":"2024-09-16T09:29:03.419497+0000","last_clean":"2024-09-16T09:29:03.419497+0000","last_became_active":"2024-09-16T09:25:05.712516+0000","last_became_peered":"2024-09-16T09:25:05.712516+0000","last_unstale":"2024-09-16T09:29:03.419497+0000","last_undegraded":"2024-09-16T09:29:03.419497+0000","last_fullsized":"2024-09-16T09:29:03.419497+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:28:58.597586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:28:58.597586+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:23.196728+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.a","version":"73'79","reported_seq":113,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.408716+0000","last_change":"2024-09-16T09:26:36.169945+0000","last_active":"2024-09-16T09:29:03.408716+0000","last_peered":"2024-09-16T09:29:03.408716+0000","last_clean":"2024-09-16T09:29:03.408716+0000","last_became_active":"2024-09-16T09:26:35.127352+0000","last_became_peered":"2024-09-16T09:26:35.127352+0000","last_unstale":"2024-09-16T09:29:03.408716+0000","last_undegraded":"2024-09-16T09:29:03.408716+0000","last_fullsized":"2024-09-16T09:29:03.408716+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:26:36.169870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:26:36.169870+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:13.960880+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.c","version":"141'13638","reported_seq":13772,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:07.514803+0000","last_change":"2024-09-16T09:28:33.904045+0000","last_active":"2024-09-16T09:29:07.514803+0000","last_peered":"2024-09-16T09:29:07.514803+0000","last_clean":"2024-09-16T09:29:07.514803+0000","last_became_active":"2024-09-16T09:26:08.116783+0000","last_became_peered":"2024-09-16T09:26:08.116783+0000","last_unstale":"2024-09-16T09:29:07.514803+0000","last_undegraded":"2024-09-16T09:29:07.514803+0000","last_fullsized":"2024-09-16T09:29:07.514803+0000","mapping_epoch":82,"log_start":"118'7800","ondisk_log_start":"118'7800","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_clean_scrub_stamp":"2024-09-16T09:26:07.096764+0000","objects_scrubbed":99,"log_size":5838,"log_dups_size":0,"ondisk_log_size":5838,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:05.486600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":132775936,"num_objects":2026,"num_object_clones":0,"num_object_copies":6078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2026,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13638,"num_write_kb":501248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.f","version":"66'15","reported_seq":133,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.366034+0000","last_change":"2024-09-16T09:28:28.065781+0000","last_active":"2024-09-16T09:29:03.366034+0000","last_peered":"2024-09-16T09:29:03.366034+0000","last_clean":"2024-09-16T09:29:03.366034+0000","last_became_active":"2024-09-16T09:26:33.295672+0000","last_became_peered":"2024-09-16T09:26:33.295672+0000","last_unstale":"2024-09-16T09:29:03.366034+0000","last_undegraded":"2024-09-16T09:29:03.366034+0000","last_fullsized":"2024-09-16T09:29:03.366034+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-16T09:28:28.065699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:28.065699+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:32.934741+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":193,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.363357+0000","last_change":"2024-09-16T09:28:22.091708+0000","last_active":"2024-09-16T09:29:03.363357+0000","last_peered":"2024-09-16T09:29:03.363357+0000","last_clean":"2024-09-16T09:29:03.363357+0000","last_became_active":"2024-09-16T09:25:01.702629+0000","last_became_peered":"2024-09-16T09:25:01.702629+0000","last_unstale":"2024-09-16T09:29:03.363357+0000","last_undegraded":"2024-09-16T09:29:03.363357+0000","last_fullsized":"2024-09-16T09:29:03.363357+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:22.091537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:28:22.091537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:51.478241+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"66'8","reported_seq":227,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.417278+0000","last_change":"2024-09-16T09:25:05.714128+0000","last_active":"2024-09-16T09:29:03.417278+0000","last_peered":"2024-09-16T09:29:03.417278+0000","last_clean":"2024-09-16T09:29:03.417278+0000","last_became_active":"2024-09-16T09:25:05.713813+0000","last_became_peered":"2024-09-16T09:25:05.713813+0000","last_unstale":"2024-09-16T09:29:03.417278+0000","last_undegraded":"2024-09-16T09:29:03.417278+0000","last_fullsized":"2024-09-16T09:29:03.417278+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:12.670604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":172,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.409663+0000","last_change":"2024-09-16T09:26:38.289448+0000","last_active":"2024-09-16T09:29:03.409663+0000","last_peered":"2024-09-16T09:29:03.409663+0000","last_clean":"2024-09-16T09:29:03.409663+0000","last_became_active":"2024-09-16T09:25:07.717344+0000","last_became_peered":"2024-09-16T09:25:07.717344+0000","last_unstale":"2024-09-16T09:29:03.409663+0000","last_undegraded":"2024-09-16T09:29:03.409663+0000","last_fullsized":"2024-09-16T09:29:03.409663+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:38.289387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:38.289387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:14.990613+0000","scrub_duration":114,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.5","version":"73'79","reported_seq":121,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.417541+0000","last_change":"2024-09-16T09:28:33.628146+0000","last_active":"2024-09-16T09:29:03.417541+0000","last_peered":"2024-09-16T09:29:03.417541+0000","last_clean":"2024-09-16T09:29:03.417541+0000","last_became_active":"2024-09-16T09:26:35.117714+0000","last_became_peered":"2024-09-16T09:26:35.117714+0000","last_unstale":"2024-09-16T09:29:03.417541+0000","last_undegraded":"2024-09-16T09:29:03.417541+0000","last_fullsized":"2024-09-16T09:29:03.417541+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:28:33.628085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:28:33.628085+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:49.100297+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3","version":"141'13934","reported_seq":14064,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:07.835195+0000","last_change":"2024-09-16T09:26:08.119864+0000","last_active":"2024-09-16T09:29:07.835195+0000","last_peered":"2024-09-16T09:29:07.835195+0000","last_clean":"2024-09-16T09:29:07.835195+0000","last_became_active":"2024-09-16T09:26:08.119550+0000","last_became_peered":"2024-09-16T09:26:08.119550+0000","last_unstale":"2024-09-16T09:29:07.835195+0000","last_undegraded":"2024-09-16T09:29:07.835195+0000","last_fullsized":"2024-09-16T09:29:07.835195+0000","mapping_epoch":82,"log_start":"123'9600","ondisk_log_start":"123'9600","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_clean_scrub_stamp":"2024-09-16T09:26:07.096764+0000","objects_scrubbed":0,"log_size":4334,"log_dups_size":0,"ondisk_log_size":4334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:43.125248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":130940928,"num_objects":1998,"num_object_clones":0,"num_object_copies":5994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1998,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13934,"num_write_kb":509824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4],"acting":[3,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"66'18","reported_seq":81,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.417163+0000","last_change":"2024-09-16T09:28:54.700083+0000","last_active":"2024-09-16T09:29:03.417163+0000","last_peered":"2024-09-16T09:29:03.417163+0000","last_clean":"2024-09-16T09:29:03.417163+0000","last_became_active":"2024-09-16T09:28:52.979255+0000","last_became_peered":"2024-09-16T09:28:52.979255+0000","last_unstale":"2024-09-16T09:29:03.417163+0000","last_undegraded":"2024-09-16T09:29:03.417163+0000","last_fullsized":"2024-09-16T09:29:03.417163+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'18","last_scrub_stamp":"2024-09-16T09:28:54.699778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:54.699778+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:16.500734+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":33,"num_read_kb":21,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":29,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.423040+0000","last_change":"2024-09-16T09:28:58.705169+0000","last_active":"2024-09-16T09:29:03.423040+0000","last_peered":"2024-09-16T09:29:03.423040+0000","last_clean":"2024-09-16T09:29:03.423040+0000","last_became_active":"2024-09-16T09:28:52.983543+0000","last_became_peered":"2024-09-16T09:28:52.983543+0000","last_unstale":"2024-09-16T09:29:03.423040+0000","last_undegraded":"2024-09-16T09:29:03.423040+0000","last_fullsized":"2024-09-16T09:29:03.423040+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:58.705097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:28:58.705097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:13.544726+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":185,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.417130+0000","last_change":"2024-09-16T09:28:30.522177+0000","last_active":"2024-09-16T09:29:03.417130+0000","last_peered":"2024-09-16T09:29:03.417130+0000","last_clean":"2024-09-16T09:29:03.417130+0000","last_became_active":"2024-09-16T09:25:05.713360+0000","last_became_peered":"2024-09-16T09:25:05.713360+0000","last_unstale":"2024-09-16T09:29:03.417130+0000","last_undegraded":"2024-09-16T09:29:03.417130+0000","last_fullsized":"2024-09-16T09:29:03.417130+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:30.522117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:28:30.522117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:38.366286+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":173,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.373383+0000","last_change":"2024-09-16T09:27:13.548667+0000","last_active":"2024-09-16T09:29:03.373383+0000","last_peered":"2024-09-16T09:29:03.373383+0000","last_clean":"2024-09-16T09:29:03.373383+0000","last_became_active":"2024-09-16T09:25:07.719159+0000","last_became_peered":"2024-09-16T09:25:07.719159+0000","last_unstale":"2024-09-16T09:29:03.373383+0000","last_undegraded":"2024-09-16T09:29:03.373383+0000","last_fullsized":"2024-09-16T09:29:03.373383+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:13.548608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:13.548608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:38.699396+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"73'79","reported_seq":126,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.362456+0000","last_change":"2024-09-16T09:28:23.121906+0000","last_active":"2024-09-16T09:29:03.362456+0000","last_peered":"2024-09-16T09:29:03.362456+0000","last_clean":"2024-09-16T09:29:03.362456+0000","last_became_active":"2024-09-16T09:26:35.131343+0000","last_became_peered":"2024-09-16T09:26:35.131343+0000","last_unstale":"2024-09-16T09:29:03.362456+0000","last_undegraded":"2024-09-16T09:29:03.362456+0000","last_fullsized":"2024-09-16T09:29:03.362456+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:28:23.121808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:28:23.121808+0000","objects_scrubbed":1,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:50.968563+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2","version":"141'13866","reported_seq":14036,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:07.815572+0000","last_change":"2024-09-16T09:28:52.474208+0000","last_active":"2024-09-16T09:29:07.815572+0000","last_peered":"2024-09-16T09:29:07.815572+0000","last_clean":"2024-09-16T09:29:07.815572+0000","last_became_active":"2024-09-16T09:26:08.120338+0000","last_became_peered":"2024-09-16T09:26:08.120338+0000","last_unstale":"2024-09-16T09:29:07.815572+0000","last_undegraded":"2024-09-16T09:29:07.815572+0000","last_fullsized":"2024-09-16T09:29:07.815572+0000","mapping_epoch":82,"log_start":"123'9500","ondisk_log_start":"123'9500","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"136'12797","last_scrub_stamp":"2024-09-16T09:28:52.474060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_clean_scrub_stamp":"2024-09-16T09:28:52.474060+0000","objects_scrubbed":3130,"log_size":4366,"log_dups_size":0,"ondisk_log_size":4366,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:19.773815+0000","scrub_duration":1231,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":133955584,"num_objects":2044,"num_object_clones":0,"num_object_copies":6132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2044,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13866,"num_write_kb":509120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"66'14","reported_seq":210,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.417217+0000","last_change":"2024-09-16T09:28:32.595064+0000","last_active":"2024-09-16T09:29:03.417217+0000","last_peered":"2024-09-16T09:29:03.417217+0000","last_clean":"2024-09-16T09:29:03.417217+0000","last_became_active":"2024-09-16T09:25:03.708830+0000","last_became_peered":"2024-09-16T09:25:03.708830+0000","last_unstale":"2024-09-16T09:29:03.417217+0000","last_undegraded":"2024-09-16T09:29:03.417217+0000","last_fullsized":"2024-09-16T09:29:03.417217+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'14","last_scrub_stamp":"2024-09-16T09:28:32.594988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:32.594988+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:00.546313+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":202,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.364945+0000","last_change":"2024-09-16T09:28:58.602786+0000","last_active":"2024-09-16T09:29:03.364945+0000","last_peered":"2024-09-16T09:29:03.364945+0000","last_clean":"2024-09-16T09:29:03.364945+0000","last_became_active":"2024-09-16T09:25:01.702992+0000","last_became_peered":"2024-09-16T09:25:01.702992+0000","last_unstale":"2024-09-16T09:29:03.364945+0000","last_undegraded":"2024-09-16T09:29:03.364945+0000","last_fullsized":"2024-09-16T09:29:03.364945+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:28:58.602717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:28:58.602717+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:25.183069+0000","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"137'9","reported_seq":50,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.354692+0000","last_change":"2024-09-16T09:28:52.978543+0000","last_active":"2024-09-16T09:29:03.354692+0000","last_peered":"2024-09-16T09:29:03.354692+0000","last_clean":"2024-09-16T09:29:03.354692+0000","last_became_active":"2024-09-16T09:28:52.978049+0000","last_became_peered":"2024-09-16T09:28:52.978049+0000","last_unstale":"2024-09-16T09:29:03.354692+0000","last_undegraded":"2024-09-16T09:29:03.354692+0000","last_fullsized":"2024-09-16T09:29:03.354692+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:28:01.096510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:28:01.096510+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:16.869632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":172,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.419267+0000","last_change":"2024-09-16T09:26:40.699888+0000","last_active":"2024-09-16T09:29:03.419267+0000","last_peered":"2024-09-16T09:29:03.419267+0000","last_clean":"2024-09-16T09:29:03.419267+0000","last_became_active":"2024-09-16T09:25:07.722017+0000","last_became_peered":"2024-09-16T09:25:07.722017+0000","last_unstale":"2024-09-16T09:29:03.419267+0000","last_undegraded":"2024-09-16T09:29:03.419267+0000","last_fullsized":"2024-09-16T09:29:03.419267+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:40.699813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:40.699813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:10.826664+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"73'79","reported_seq":197,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.419158+0000","last_change":"2024-09-16T09:27:59.497431+0000","last_active":"2024-09-16T09:29:03.419158+0000","last_peered":"2024-09-16T09:29:03.419158+0000","last_clean":"2024-09-16T09:29:03.419158+0000","last_became_active":"2024-09-16T09:26:35.126759+0000","last_became_peered":"2024-09-16T09:26:35.126759+0000","last_unstale":"2024-09-16T09:29:03.419158+0000","last_undegraded":"2024-09-16T09:29:03.419158+0000","last_fullsized":"2024-09-16T09:29:03.419158+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:27:59.497364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:27:59.497364+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:12.907753+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"141'13533","reported_seq":13725,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:07.496845+0000","last_change":"2024-09-16T09:28:18.935082+0000","last_active":"2024-09-16T09:29:07.496845+0000","last_peered":"2024-09-16T09:29:07.496845+0000","last_clean":"2024-09-16T09:29:07.496845+0000","last_became_active":"2024-09-16T09:26:08.116726+0000","last_became_peered":"2024-09-16T09:26:08.116726+0000","last_unstale":"2024-09-16T09:29:07.496845+0000","last_undegraded":"2024-09-16T09:29:07.496845+0000","last_fullsized":"2024-09-16T09:29:07.496845+0000","mapping_epoch":82,"log_start":"122'8900","ondisk_log_start":"122'8900","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'10297","last_scrub_stamp":"2024-09-16T09:28:18.934970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_clean_scrub_stamp":"2024-09-16T09:28:18.934970+0000","objects_scrubbed":5294,"log_size":4633,"log_dups_size":0,"ondisk_log_size":4633,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:35.502825+0000","scrub_duration":1205,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131923968,"num_objects":2013,"num_object_clones":0,"num_object_copies":6039,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2013,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13533,"num_write_kb":497472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"66'10","reported_seq":204,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.358308+0000","last_change":"2024-09-16T09:28:30.792507+0000","last_active":"2024-09-16T09:29:03.358308+0000","last_peered":"2024-09-16T09:29:03.358308+0000","last_clean":"2024-09-16T09:29:03.358308+0000","last_became_active":"2024-09-16T09:25:03.712985+0000","last_became_peered":"2024-09-16T09:25:03.712985+0000","last_unstale":"2024-09-16T09:29:03.358308+0000","last_undegraded":"2024-09-16T09:29:03.358308+0000","last_fullsized":"2024-09-16T09:29:03.358308+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-16T09:28:30.792412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:30.792412+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:57.669767+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":184,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.410095+0000","last_change":"2024-09-16T09:26:25.086388+0000","last_active":"2024-09-16T09:29:03.410095+0000","last_peered":"2024-09-16T09:29:03.410095+0000","last_clean":"2024-09-16T09:29:03.410095+0000","last_became_active":"2024-09-16T09:25:01.706440+0000","last_became_peered":"2024-09-16T09:25:01.706440+0000","last_unstale":"2024-09-16T09:29:03.410095+0000","last_undegraded":"2024-09-16T09:29:03.410095+0000","last_fullsized":"2024-09-16T09:29:03.410095+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:25.086322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:25.086322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"66'8","reported_seq":243,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.362069+0000","last_change":"2024-09-16T09:28:57.628615+0000","last_active":"2024-09-16T09:29:03.362069+0000","last_peered":"2024-09-16T09:29:03.362069+0000","last_clean":"2024-09-16T09:29:03.362069+0000","last_became_active":"2024-09-16T09:25:05.714507+0000","last_became_peered":"2024-09-16T09:25:05.714507+0000","last_unstale":"2024-09-16T09:29:03.362069+0000","last_undegraded":"2024-09-16T09:29:03.362069+0000","last_fullsized":"2024-09-16T09:29:03.362069+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:28:57.628543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:28:57.628543+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:19.830302+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":181,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.361914+0000","last_change":"2024-09-16T09:28:22.563466+0000","last_active":"2024-09-16T09:29:03.361914+0000","last_peered":"2024-09-16T09:29:03.361914+0000","last_clean":"2024-09-16T09:29:03.361914+0000","last_became_active":"2024-09-16T09:25:07.719604+0000","last_became_peered":"2024-09-16T09:25:07.719604+0000","last_unstale":"2024-09-16T09:29:03.361914+0000","last_undegraded":"2024-09-16T09:29:03.361914+0000","last_fullsized":"2024-09-16T09:29:03.361914+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:22.563395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:28:22.563395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:45.474687+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"73'79","reported_seq":189,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.409106+0000","last_change":"2024-09-16T09:26:35.170288+0000","last_active":"2024-09-16T09:29:03.409106+0000","last_peered":"2024-09-16T09:29:03.409106+0000","last_clean":"2024-09-16T09:29:03.409106+0000","last_became_active":"2024-09-16T09:26:35.125571+0000","last_became_peered":"2024-09-16T09:26:35.125571+0000","last_unstale":"2024-09-16T09:29:03.409106+0000","last_undegraded":"2024-09-16T09:29:03.409106+0000","last_fullsized":"2024-09-16T09:29:03.409106+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:26:35.170229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:26:35.170229+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:04.472831+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.0","version":"141'13848","reported_seq":14048,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:07.438399+0000","last_change":"2024-09-16T09:27:16.936395+0000","last_active":"2024-09-16T09:29:07.438399+0000","last_peered":"2024-09-16T09:29:07.438399+0000","last_clean":"2024-09-16T09:29:07.438399+0000","last_became_active":"2024-09-16T09:26:08.117581+0000","last_became_peered":"2024-09-16T09:26:08.117581+0000","last_unstale":"2024-09-16T09:29:07.438399+0000","last_undegraded":"2024-09-16T09:29:07.438399+0000","last_fullsized":"2024-09-16T09:29:07.438399+0000","mapping_epoch":82,"log_start":"120'8300","ondisk_log_start":"120'8300","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"109'6200","last_scrub_stamp":"2024-09-16T09:27:16.936218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_clean_scrub_stamp":"2024-09-16T09:27:16.936218+0000","objects_scrubbed":6157,"log_size":5548,"log_dups_size":0,"ondisk_log_size":5548,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"queued for scrub","scrub_duration":1612,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":130023424,"num_objects":1984,"num_object_clones":0,"num_object_copies":5952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1984,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13848,"num_write_kb":506624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3","version":"66'19","reported_seq":217,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.362245+0000","last_change":"2024-09-16T09:28:47.551606+0000","last_active":"2024-09-16T09:29:03.362245+0000","last_peered":"2024-09-16T09:29:03.362245+0000","last_clean":"2024-09-16T09:29:03.362245+0000","last_became_active":"2024-09-16T09:25:03.714797+0000","last_became_peered":"2024-09-16T09:25:03.714797+0000","last_unstale":"2024-09-16T09:29:03.362245+0000","last_undegraded":"2024-09-16T09:29:03.362245+0000","last_fullsized":"2024-09-16T09:29:03.362245+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-16T09:28:47.551527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:47.551527+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:59.123170+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":193,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.355975+0000","last_change":"2024-09-16T09:27:24.219132+0000","last_active":"2024-09-16T09:29:03.355975+0000","last_peered":"2024-09-16T09:29:03.355975+0000","last_clean":"2024-09-16T09:29:03.355975+0000","last_became_active":"2024-09-16T09:25:01.706806+0000","last_became_peered":"2024-09-16T09:25:01.706806+0000","last_unstale":"2024-09-16T09:29:03.355975+0000","last_undegraded":"2024-09-16T09:29:03.355975+0000","last_fullsized":"2024-09-16T09:29:03.355975+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:24.218910+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:27:24.218910+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":185,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.418649+0000","last_change":"2024-09-16T09:28:24.737761+0000","last_active":"2024-09-16T09:29:03.418649+0000","last_peered":"2024-09-16T09:29:03.418649+0000","last_clean":"2024-09-16T09:29:03.418649+0000","last_became_active":"2024-09-16T09:25:05.708663+0000","last_became_peered":"2024-09-16T09:25:05.708663+0000","last_unstale":"2024-09-16T09:29:03.418649+0000","last_undegraded":"2024-09-16T09:29:03.418649+0000","last_fullsized":"2024-09-16T09:29:03.418649+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:24.737702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:28:24.737702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:40.005585+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":173,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.356744+0000","last_change":"2024-09-16T09:26:37.282798+0000","last_active":"2024-09-16T09:29:03.356744+0000","last_peered":"2024-09-16T09:29:03.356744+0000","last_clean":"2024-09-16T09:29:03.356744+0000","last_became_active":"2024-09-16T09:25:07.719267+0000","last_became_peered":"2024-09-16T09:25:07.719267+0000","last_unstale":"2024-09-16T09:29:03.356744+0000","last_undegraded":"2024-09-16T09:29:03.356744+0000","last_fullsized":"2024-09-16T09:29:03.356744+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:37.282735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:37.282735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:04.927965+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"73'79","reported_seq":113,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.408715+0000","last_change":"2024-09-16T09:26:35.154953+0000","last_active":"2024-09-16T09:29:03.408715+0000","last_peered":"2024-09-16T09:29:03.408715+0000","last_clean":"2024-09-16T09:29:03.408715+0000","last_became_active":"2024-09-16T09:26:35.117627+0000","last_became_peered":"2024-09-16T09:26:35.117627+0000","last_unstale":"2024-09-16T09:29:03.408715+0000","last_undegraded":"2024-09-16T09:29:03.408715+0000","last_fullsized":"2024-09-16T09:29:03.408715+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:26:35.154875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:26:35.154875+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:04.931144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"141'13880","reported_seq":14010,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:07.516585+0000","last_change":"2024-09-16T09:26:08.122122+0000","last_active":"2024-09-16T09:29:07.516585+0000","last_peered":"2024-09-16T09:29:07.516585+0000","last_clean":"2024-09-16T09:29:07.516585+0000","last_became_active":"2024-09-16T09:26:08.121828+0000","last_became_peered":"2024-09-16T09:26:08.121828+0000","last_unstale":"2024-09-16T09:29:07.516585+0000","last_undegraded":"2024-09-16T09:29:07.516585+0000","last_fullsized":"2024-09-16T09:29:07.516585+0000","mapping_epoch":82,"log_start":"120'8100","ondisk_log_start":"120'8100","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_clean_scrub_stamp":"2024-09-16T09:26:07.096764+0000","objects_scrubbed":0,"log_size":5780,"log_dups_size":0,"ondisk_log_size":5780,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:44.607436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":132775936,"num_objects":2026,"num_object_clones":0,"num_object_copies":6078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2026,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13880,"num_write_kb":508992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.4","version":"66'28","reported_seq":279,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.355872+0000","last_change":"2024-09-16T09:27:50.051292+0000","last_active":"2024-09-16T09:29:03.355872+0000","last_peered":"2024-09-16T09:29:03.355872+0000","last_clean":"2024-09-16T09:29:03.355872+0000","last_became_active":"2024-09-16T09:25:03.704878+0000","last_became_peered":"2024-09-16T09:25:03.704878+0000","last_unstale":"2024-09-16T09:29:03.355872+0000","last_undegraded":"2024-09-16T09:29:03.355872+0000","last_fullsized":"2024-09-16T09:29:03.355872+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'28","last_scrub_stamp":"2024-09-16T09:27:50.046365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:50.046365+0000","objects_scrubbed":10,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:08.615060+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":185,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.416455+0000","last_change":"2024-09-16T09:28:00.718726+0000","last_active":"2024-09-16T09:29:03.416455+0000","last_peered":"2024-09-16T09:29:03.416455+0000","last_clean":"2024-09-16T09:29:03.416455+0000","last_became_active":"2024-09-16T09:25:01.703076+0000","last_became_peered":"2024-09-16T09:25:01.703076+0000","last_unstale":"2024-09-16T09:29:03.416455+0000","last_undegraded":"2024-09-16T09:29:03.416455+0000","last_fullsized":"2024-09-16T09:29:03.416455+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:00.718684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:28:00.718684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:21.876192+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":194,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.417962+0000","last_change":"2024-09-16T09:27:44.183242+0000","last_active":"2024-09-16T09:29:03.417962+0000","last_peered":"2024-09-16T09:29:03.417962+0000","last_clean":"2024-09-16T09:29:03.417962+0000","last_became_active":"2024-09-16T09:25:00.696237+0000","last_became_peered":"2024-09-16T09:25:00.696237+0000","last_unstale":"2024-09-16T09:29:03.417962+0000","last_undegraded":"2024-09-16T09:29:03.417962+0000","last_fullsized":"2024-09-16T09:29:03.417962+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-16T09:27:44.183111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:27:44.183111+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.0025137430000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":169,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.362633+0000","last_change":"2024-09-16T09:25:05.718319+0000","last_active":"2024-09-16T09:29:03.362633+0000","last_peered":"2024-09-16T09:29:03.362633+0000","last_clean":"2024-09-16T09:29:03.362633+0000","last_became_active":"2024-09-16T09:25:05.717591+0000","last_became_peered":"2024-09-16T09:25:05.717591+0000","last_unstale":"2024-09-16T09:29:03.362633+0000","last_undegraded":"2024-09-16T09:29:03.362633+0000","last_fullsized":"2024-09-16T09:29:03.362633+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:07.631783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":181,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.353233+0000","last_change":"2024-09-16T09:27:45.201574+0000","last_active":"2024-09-16T09:29:03.353233+0000","last_peered":"2024-09-16T09:29:03.353233+0000","last_clean":"2024-09-16T09:29:03.353233+0000","last_became_active":"2024-09-16T09:25:07.717436+0000","last_became_peered":"2024-09-16T09:25:07.717436+0000","last_unstale":"2024-09-16T09:29:03.353233+0000","last_undegraded":"2024-09-16T09:29:03.353233+0000","last_fullsized":"2024-09-16T09:29:03.353233+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:45.201503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:45.201503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:09.839306+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"73'79","reported_seq":246,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.417462+0000","last_change":"2024-09-16T09:26:35.745908+0000","last_active":"2024-09-16T09:29:03.417462+0000","last_peered":"2024-09-16T09:29:03.417462+0000","last_clean":"2024-09-16T09:29:03.417462+0000","last_became_active":"2024-09-16T09:26:35.115364+0000","last_became_peered":"2024-09-16T09:26:35.115364+0000","last_unstale":"2024-09-16T09:29:03.417462+0000","last_undegraded":"2024-09-16T09:29:03.417462+0000","last_fullsized":"2024-09-16T09:29:03.417462+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:26:35.745837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:26:35.745837+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:03.843804+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.4","version":"141'13568","reported_seq":13698,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:07.811556+0000","last_change":"2024-09-16T09:26:08.120333+0000","last_active":"2024-09-16T09:29:07.811556+0000","last_peered":"2024-09-16T09:29:07.811556+0000","last_clean":"2024-09-16T09:29:07.811556+0000","last_became_active":"2024-09-16T09:26:08.120051+0000","last_became_peered":"2024-09-16T09:26:08.120051+0000","last_unstale":"2024-09-16T09:29:07.811556+0000","last_undegraded":"2024-09-16T09:29:07.811556+0000","last_fullsized":"2024-09-16T09:29:07.811556+0000","mapping_epoch":82,"log_start":"121'8385","ondisk_log_start":"121'8385","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_clean_scrub_stamp":"2024-09-16T09:26:07.096764+0000","objects_scrubbed":0,"log_size":5183,"log_dups_size":0,"ondisk_log_size":5183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:41.623770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131203072,"num_objects":2002,"num_object_clones":0,"num_object_copies":6006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2002,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13568,"num_write_kb":498240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,1],"acting":[0,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"66'13","reported_seq":216,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.354505+0000","last_change":"2024-09-16T09:27:44.156057+0000","last_active":"2024-09-16T09:29:03.354505+0000","last_peered":"2024-09-16T09:29:03.354505+0000","last_clean":"2024-09-16T09:29:03.354505+0000","last_became_active":"2024-09-16T09:25:03.713241+0000","last_became_peered":"2024-09-16T09:25:03.713241+0000","last_unstale":"2024-09-16T09:29:03.354505+0000","last_undegraded":"2024-09-16T09:29:03.354505+0000","last_fullsized":"2024-09-16T09:29:03.354505+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'13","last_scrub_stamp":"2024-09-16T09:27:44.155969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:44.155969+0000","objects_scrubbed":5,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":29,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":186,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.356518+0000","last_change":"2024-09-16T09:26:26.230421+0000","last_active":"2024-09-16T09:29:03.356518+0000","last_peered":"2024-09-16T09:29:03.356518+0000","last_clean":"2024-09-16T09:29:03.356518+0000","last_became_active":"2024-09-16T09:25:01.706828+0000","last_became_peered":"2024-09-16T09:25:01.706828+0000","last_unstale":"2024-09-16T09:29:03.356518+0000","last_undegraded":"2024-09-16T09:29:03.356518+0000","last_fullsized":"2024-09-16T09:29:03.356518+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:26.230338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:26.230338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:39.777245+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":194,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.366041+0000","last_change":"2024-09-16T09:27:06.062939+0000","last_active":"2024-09-16T09:29:03.366041+0000","last_peered":"2024-09-16T09:29:03.366041+0000","last_clean":"2024-09-16T09:29:03.366041+0000","last_became_active":"2024-09-16T09:25:00.697822+0000","last_became_peered":"2024-09-16T09:25:00.697822+0000","last_unstale":"2024-09-16T09:29:03.366041+0000","last_undegraded":"2024-09-16T09:29:03.366041+0000","last_fullsized":"2024-09-16T09:29:03.366041+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:27:06.062871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:27:06.062871+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:11.909782+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0.00055022299999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":193,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.366782+0000","last_change":"2024-09-16T09:29:00.169141+0000","last_active":"2024-09-16T09:29:03.366782+0000","last_peered":"2024-09-16T09:29:03.366782+0000","last_clean":"2024-09-16T09:29:03.366782+0000","last_became_active":"2024-09-16T09:25:05.711539+0000","last_became_peered":"2024-09-16T09:25:05.711539+0000","last_unstale":"2024-09-16T09:29:03.366782+0000","last_undegraded":"2024-09-16T09:29:03.366782+0000","last_fullsized":"2024-09-16T09:29:03.366782+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:00.169077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:00.169077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:17.394265+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":172,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.421689+0000","last_change":"2024-09-16T09:27:47.705213+0000","last_active":"2024-09-16T09:29:03.421689+0000","last_peered":"2024-09-16T09:29:03.421689+0000","last_clean":"2024-09-16T09:29:03.421689+0000","last_became_active":"2024-09-16T09:25:07.723282+0000","last_became_peered":"2024-09-16T09:25:07.723282+0000","last_unstale":"2024-09-16T09:29:03.421689+0000","last_undegraded":"2024-09-16T09:29:03.421689+0000","last_fullsized":"2024-09-16T09:29:03.421689+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:47.705151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:47.705151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:08.139782+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"73'79","reported_seq":121,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.354745+0000","last_change":"2024-09-16T09:27:57.031941+0000","last_active":"2024-09-16T09:29:03.354745+0000","last_peered":"2024-09-16T09:29:03.354745+0000","last_clean":"2024-09-16T09:29:03.354745+0000","last_became_active":"2024-09-16T09:26:35.116315+0000","last_became_peered":"2024-09-16T09:26:35.116315+0000","last_unstale":"2024-09-16T09:29:03.354745+0000","last_undegraded":"2024-09-16T09:29:03.354745+0000","last_fullsized":"2024-09-16T09:29:03.354745+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:27:57.031564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:27:57.031564+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:25.604136+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"141'13965","reported_seq":14095,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:07.518738+0000","last_change":"2024-09-16T09:26:08.121383+0000","last_active":"2024-09-16T09:29:07.518738+0000","last_peered":"2024-09-16T09:29:07.518738+0000","last_clean":"2024-09-16T09:29:07.518738+0000","last_became_active":"2024-09-16T09:26:08.121117+0000","last_became_peered":"2024-09-16T09:26:08.121117+0000","last_unstale":"2024-09-16T09:29:07.518738+0000","last_undegraded":"2024-09-16T09:29:07.518738+0000","last_fullsized":"2024-09-16T09:29:07.518738+0000","mapping_epoch":82,"log_start":"122'9105","ondisk_log_start":"122'9105","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_clean_scrub_stamp":"2024-09-16T09:26:07.096764+0000","objects_scrubbed":0,"log_size":4860,"log_dups_size":0,"ondisk_log_size":4860,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:34.776754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":130744320,"num_objects":1995,"num_object_clones":0,"num_object_copies":5985,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1995,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13965,"num_write_kb":510720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,0],"acting":[7,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"66'12","reported_seq":199,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.362839+0000","last_change":"2024-09-16T09:29:02.713335+0000","last_active":"2024-09-16T09:29:03.362839+0000","last_peered":"2024-09-16T09:29:03.362839+0000","last_clean":"2024-09-16T09:29:03.362839+0000","last_became_active":"2024-09-16T09:25:03.715315+0000","last_became_peered":"2024-09-16T09:25:03.715315+0000","last_unstale":"2024-09-16T09:29:03.362839+0000","last_undegraded":"2024-09-16T09:29:03.362839+0000","last_fullsized":"2024-09-16T09:29:03.362839+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-16T09:29:02.713259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:29:02.713259+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:06.150014+0000","scrub_duration":26,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"59'1","reported_seq":194,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.365280+0000","last_change":"2024-09-16T09:27:00.666748+0000","last_active":"2024-09-16T09:29:03.365280+0000","last_peered":"2024-09-16T09:29:03.365280+0000","last_clean":"2024-09-16T09:29:03.365280+0000","last_became_active":"2024-09-16T09:25:01.701599+0000","last_became_peered":"2024-09-16T09:25:01.701599+0000","last_unstale":"2024-09-16T09:29:03.365280+0000","last_undegraded":"2024-09-16T09:29:03.365280+0000","last_fullsized":"2024-09-16T09:29:03.365280+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:27:00.666673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:27:00.666673+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:36.573227+0000","scrub_duration":28,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"67'5","reported_seq":436,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:05.827261+0000","last_change":"2024-09-16T09:26:38.722150+0000","last_active":"2024-09-16T09:29:05.827261+0000","last_peered":"2024-09-16T09:29:05.827261+0000","last_clean":"2024-09-16T09:29:05.827261+0000","last_became_active":"2024-09-16T09:25:00.696546+0000","last_became_peered":"2024-09-16T09:25:00.696546+0000","last_unstale":"2024-09-16T09:29:05.827261+0000","last_undegraded":"2024-09-16T09:29:05.827261+0000","last_fullsized":"2024-09-16T09:29:05.827261+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'5","last_scrub_stamp":"2024-09-16T09:26:38.722074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:26:38.722074+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:13.894496+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0.0059770420000000001,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":240,"num_read_kb":235,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":176,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.409716+0000","last_change":"2024-09-16T09:27:07.337170+0000","last_active":"2024-09-16T09:29:03.409716+0000","last_peered":"2024-09-16T09:29:03.409716+0000","last_clean":"2024-09-16T09:29:03.409716+0000","last_became_active":"2024-09-16T09:25:05.714704+0000","last_became_peered":"2024-09-16T09:25:05.714704+0000","last_unstale":"2024-09-16T09:29:03.409716+0000","last_undegraded":"2024-09-16T09:29:03.409716+0000","last_fullsized":"2024-09-16T09:29:03.409716+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:07.337124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:07.337124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:34.096386+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":173,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.354213+0000","last_change":"2024-09-16T09:26:24.231324+0000","last_active":"2024-09-16T09:29:03.354213+0000","last_peered":"2024-09-16T09:29:03.354213+0000","last_clean":"2024-09-16T09:29:03.354213+0000","last_became_active":"2024-09-16T09:25:07.714158+0000","last_became_peered":"2024-09-16T09:25:07.714158+0000","last_unstale":"2024-09-16T09:29:03.354213+0000","last_undegraded":"2024-09-16T09:29:03.354213+0000","last_fullsized":"2024-09-16T09:29:03.354213+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:24.231167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:24.231167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.6","version":"141'13762","reported_seq":13940,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:07.852713+0000","last_change":"2024-09-16T09:27:26.477113+0000","last_active":"2024-09-16T09:29:07.852713+0000","last_peered":"2024-09-16T09:29:07.852713+0000","last_clean":"2024-09-16T09:29:07.852713+0000","last_became_active":"2024-09-16T09:26:08.121253+0000","last_became_peered":"2024-09-16T09:26:08.121253+0000","last_unstale":"2024-09-16T09:29:07.852713+0000","last_undegraded":"2024-09-16T09:29:07.852713+0000","last_fullsized":"2024-09-16T09:29:07.852713+0000","mapping_epoch":82,"log_start":"122'8565","ondisk_log_start":"122'8565","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_clean_scrub_stamp":"2024-09-16T09:26:07.096764+0000","objects_scrubbed":4399,"log_size":5197,"log_dups_size":0,"ondisk_log_size":5197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:09.654519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131072000,"num_objects":2000,"num_object_clones":0,"num_object_copies":6000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2000,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13762,"num_write_kb":504384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"66'16","reported_seq":253,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.419707+0000","last_change":"2024-09-16T09:27:14.265733+0000","last_active":"2024-09-16T09:29:03.419707+0000","last_peered":"2024-09-16T09:29:03.419707+0000","last_clean":"2024-09-16T09:29:03.419707+0000","last_became_active":"2024-09-16T09:25:03.714118+0000","last_became_peered":"2024-09-16T09:25:03.714118+0000","last_unstale":"2024-09-16T09:29:03.419707+0000","last_undegraded":"2024-09-16T09:29:03.419707+0000","last_fullsized":"2024-09-16T09:29:03.419707+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'16","last_scrub_stamp":"2024-09-16T09:27:14.265648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:14.265648+0000","objects_scrubbed":8,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:46.704565+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.417811+0000","last_change":"2024-09-16T09:28:52.972799+0000","last_active":"2024-09-16T09:29:03.417811+0000","last_peered":"2024-09-16T09:29:03.417811+0000","last_clean":"2024-09-16T09:29:03.417811+0000","last_became_active":"2024-09-16T09:28:52.972303+0000","last_became_peered":"2024-09-16T09:28:52.972303+0000","last_unstale":"2024-09-16T09:29:03.417811+0000","last_undegraded":"2024-09-16T09:29:03.417811+0000","last_fullsized":"2024-09-16T09:29:03.417811+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:49.195331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:27:49.195331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:05.796804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"73'79","reported_seq":249,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.417703+0000","last_change":"2024-09-16T09:29:02.723875+0000","last_active":"2024-09-16T09:29:03.417703+0000","last_peered":"2024-09-16T09:29:03.417703+0000","last_clean":"2024-09-16T09:29:03.417703+0000","last_became_active":"2024-09-16T09:26:33.297014+0000","last_became_peered":"2024-09-16T09:26:33.297014+0000","last_unstale":"2024-09-16T09:29:03.417703+0000","last_undegraded":"2024-09-16T09:29:03.417703+0000","last_fullsized":"2024-09-16T09:29:03.417703+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:02.723794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:02.723794+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:11.669882+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":176,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.416946+0000","last_change":"2024-09-16T09:26:39.699133+0000","last_active":"2024-09-16T09:29:03.416946+0000","last_peered":"2024-09-16T09:29:03.416946+0000","last_clean":"2024-09-16T09:29:03.416946+0000","last_became_active":"2024-09-16T09:25:05.714341+0000","last_became_peered":"2024-09-16T09:25:05.714341+0000","last_unstale":"2024-09-16T09:29:03.416946+0000","last_undegraded":"2024-09-16T09:29:03.416946+0000","last_fullsized":"2024-09-16T09:29:03.416946+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:39.699059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:26:39.699059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":180,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.409517+0000","last_change":"2024-09-16T09:29:00.243528+0000","last_active":"2024-09-16T09:29:03.409517+0000","last_peered":"2024-09-16T09:29:03.409517+0000","last_clean":"2024-09-16T09:29:03.409517+0000","last_became_active":"2024-09-16T09:25:07.714281+0000","last_became_peered":"2024-09-16T09:25:07.714281+0000","last_unstale":"2024-09-16T09:29:03.409517+0000","last_undegraded":"2024-09-16T09:29:03.409517+0000","last_fullsized":"2024-09-16T09:29:03.409517+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:00.243482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:00.243482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:20.041664+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"141'13675","reported_seq":13804,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:07.517644+0000","last_change":"2024-09-16T09:26:08.122057+0000","last_active":"2024-09-16T09:29:07.517644+0000","last_peered":"2024-09-16T09:29:07.517644+0000","last_clean":"2024-09-16T09:29:07.517644+0000","last_became_active":"2024-09-16T09:26:08.121785+0000","last_became_peered":"2024-09-16T09:26:08.121785+0000","last_unstale":"2024-09-16T09:29:07.517644+0000","last_undegraded":"2024-09-16T09:29:07.517644+0000","last_fullsized":"2024-09-16T09:29:07.517644+0000","mapping_epoch":82,"log_start":"120'7900","ondisk_log_start":"120'7900","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_clean_scrub_stamp":"2024-09-16T09:26:07.096764+0000","objects_scrubbed":0,"log_size":5775,"log_dups_size":0,"ondisk_log_size":5775,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:38.757046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":128188440,"num_objects":1957,"num_object_clones":0,"num_object_copies":5871,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1957,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13675,"num_write_kb":500161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.e","version":"66'11","reported_seq":211,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.417329+0000","last_change":"2024-09-16T09:26:33.316695+0000","last_active":"2024-09-16T09:29:03.417329+0000","last_peered":"2024-09-16T09:29:03.417329+0000","last_clean":"2024-09-16T09:29:03.417329+0000","last_became_active":"2024-09-16T09:26:33.287399+0000","last_became_peered":"2024-09-16T09:26:33.287399+0000","last_unstale":"2024-09-16T09:29:03.417329+0000","last_undegraded":"2024-09-16T09:29:03.417329+0000","last_fullsized":"2024-09-16T09:29:03.417329+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:03.666182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":177,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.417558+0000","last_change":"2024-09-16T09:25:01.708568+0000","last_active":"2024-09-16T09:29:03.417558+0000","last_peered":"2024-09-16T09:29:03.417558+0000","last_clean":"2024-09-16T09:29:03.417558+0000","last_became_active":"2024-09-16T09:25:01.708270+0000","last_became_peered":"2024-09-16T09:25:01.708270+0000","last_unstale":"2024-09-16T09:29:03.417558+0000","last_undegraded":"2024-09-16T09:29:03.417558+0000","last_fullsized":"2024-09-16T09:29:03.417558+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:09.662470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":176,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.409431+0000","last_change":"2024-09-16T09:26:34.171190+0000","last_active":"2024-09-16T09:29:03.409431+0000","last_peered":"2024-09-16T09:29:03.409431+0000","last_clean":"2024-09-16T09:29:03.409431+0000","last_became_active":"2024-09-16T09:25:05.711964+0000","last_became_peered":"2024-09-16T09:25:05.711964+0000","last_unstale":"2024-09-16T09:29:03.409431+0000","last_undegraded":"2024-09-16T09:29:03.409431+0000","last_fullsized":"2024-09-16T09:29:03.409431+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:34.171139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:26:34.171139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:42.124422+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":30,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.409302+0000","last_change":"2024-09-16T09:28:56.556198+0000","last_active":"2024-09-16T09:29:03.409302+0000","last_peered":"2024-09-16T09:29:03.409302+0000","last_clean":"2024-09-16T09:29:03.409302+0000","last_became_active":"2024-09-16T09:28:52.977909+0000","last_became_peered":"2024-09-16T09:28:52.977909+0000","last_unstale":"2024-09-16T09:29:03.409302+0000","last_undegraded":"2024-09-16T09:29:03.409302+0000","last_fullsized":"2024-09-16T09:29:03.409302+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:56.556130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:28:56.556130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:06.061012+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.8","version":"73'79","reported_seq":120,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.408891+0000","last_change":"2024-09-16T09:26:37.166480+0000","last_active":"2024-09-16T09:29:03.408891+0000","last_peered":"2024-09-16T09:29:03.408891+0000","last_clean":"2024-09-16T09:29:03.408891+0000","last_became_active":"2024-09-16T09:26:35.126217+0000","last_became_peered":"2024-09-16T09:26:35.126217+0000","last_unstale":"2024-09-16T09:29:03.408891+0000","last_undegraded":"2024-09-16T09:29:03.408891+0000","last_fullsized":"2024-09-16T09:29:03.408891+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:26:37.166434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:26:37.166434+0000","objects_scrubbed":1,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:06.932146+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262688,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.e","version":"141'13661","reported_seq":13860,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:07.605567+0000","last_change":"2024-09-16T09:27:16.142844+0000","last_active":"2024-09-16T09:29:07.605567+0000","last_peered":"2024-09-16T09:29:07.605567+0000","last_clean":"2024-09-16T09:29:07.605567+0000","last_became_active":"2024-09-16T09:26:08.121969+0000","last_became_peered":"2024-09-16T09:26:08.121969+0000","last_unstale":"2024-09-16T09:29:07.605567+0000","last_undegraded":"2024-09-16T09:29:07.605567+0000","last_fullsized":"2024-09-16T09:29:07.605567+0000","mapping_epoch":82,"log_start":"117'7500","ondisk_log_start":"117'7500","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"108'6031","last_scrub_stamp":"2024-09-16T09:27:16.142747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_clean_scrub_stamp":"2024-09-16T09:27:16.142747+0000","objects_scrubbed":5966,"log_size":6161,"log_dups_size":0,"ondisk_log_size":6161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:46.250402+0000","scrub_duration":2110,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":133627904,"num_objects":2039,"num_object_clones":0,"num_object_copies":6117,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2039,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13661,"num_write_kb":502400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"66'17","reported_seq":217,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.417385+0000","last_change":"2024-09-16T09:28:55.633865+0000","last_active":"2024-09-16T09:29:03.417385+0000","last_peered":"2024-09-16T09:29:03.417385+0000","last_clean":"2024-09-16T09:29:03.417385+0000","last_became_active":"2024-09-16T09:25:03.713358+0000","last_became_peered":"2024-09-16T09:25:03.713358+0000","last_unstale":"2024-09-16T09:29:03.417385+0000","last_undegraded":"2024-09-16T09:29:03.417385+0000","last_fullsized":"2024-09-16T09:29:03.417385+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'17","last_scrub_stamp":"2024-09-16T09:28:55.633770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:55.633770+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:58.863492+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":185,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.418967+0000","last_change":"2024-09-16T09:26:04.104188+0000","last_active":"2024-09-16T09:29:03.418967+0000","last_peered":"2024-09-16T09:29:03.418967+0000","last_clean":"2024-09-16T09:29:03.418967+0000","last_became_active":"2024-09-16T09:25:01.703355+0000","last_became_peered":"2024-09-16T09:25:01.703355+0000","last_unstale":"2024-09-16T09:29:03.418967+0000","last_undegraded":"2024-09-16T09:29:03.418967+0000","last_fullsized":"2024-09-16T09:29:03.418967+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:04.104130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:04.104130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:39.750641+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":185,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.354302+0000","last_change":"2024-09-16T09:27:54.028056+0000","last_active":"2024-09-16T09:29:03.354302+0000","last_peered":"2024-09-16T09:29:03.354302+0000","last_clean":"2024-09-16T09:29:03.354302+0000","last_became_active":"2024-09-16T09:25:05.716900+0000","last_became_peered":"2024-09-16T09:25:05.716900+0000","last_unstale":"2024-09-16T09:29:03.354302+0000","last_undegraded":"2024-09-16T09:29:03.354302+0000","last_fullsized":"2024-09-16T09:29:03.354302+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:54.027864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:54.027864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:17.563463+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":181,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.366232+0000","last_change":"2024-09-16T09:28:33.225543+0000","last_active":"2024-09-16T09:29:03.366232+0000","last_peered":"2024-09-16T09:29:03.366232+0000","last_clean":"2024-09-16T09:29:03.366232+0000","last_became_active":"2024-09-16T09:25:07.721347+0000","last_became_peered":"2024-09-16T09:25:07.721347+0000","last_unstale":"2024-09-16T09:29:03.366232+0000","last_undegraded":"2024-09-16T09:29:03.366232+0000","last_fullsized":"2024-09-16T09:29:03.366232+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:33.225473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:28:33.225473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:55.185217+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.9","version":"73'79","reported_seq":246,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.417070+0000","last_change":"2024-09-16T09:26:36.771785+0000","last_active":"2024-09-16T09:29:03.417070+0000","last_peered":"2024-09-16T09:29:03.417070+0000","last_clean":"2024-09-16T09:29:03.417070+0000","last_became_active":"2024-09-16T09:26:35.126066+0000","last_became_peered":"2024-09-16T09:26:35.126066+0000","last_unstale":"2024-09-16T09:29:03.417070+0000","last_undegraded":"2024-09-16T09:29:03.417070+0000","last_fullsized":"2024-09-16T09:29:03.417070+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:26:36.771720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:26:36.771720+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:11.858317+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.f","version":"141'13918","reported_seq":14048,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:07.810062+0000","last_change":"2024-09-16T09:26:08.120989+0000","last_active":"2024-09-16T09:29:07.810062+0000","last_peered":"2024-09-16T09:29:07.810062+0000","last_clean":"2024-09-16T09:29:07.810062+0000","last_became_active":"2024-09-16T09:26:08.120360+0000","last_became_peered":"2024-09-16T09:26:08.120360+0000","last_unstale":"2024-09-16T09:29:07.810062+0000","last_undegraded":"2024-09-16T09:29:07.810062+0000","last_fullsized":"2024-09-16T09:29:07.810062+0000","mapping_epoch":82,"log_start":"122'8658","ondisk_log_start":"122'8658","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_clean_scrub_stamp":"2024-09-16T09:26:07.096764+0000","objects_scrubbed":0,"log_size":5260,"log_dups_size":0,"ondisk_log_size":5260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:39.571615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131727360,"num_objects":2010,"num_object_clones":0,"num_object_copies":6030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2010,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13918,"num_write_kb":509696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"66'10","reported_seq":188,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.373431+0000","last_change":"2024-09-16T09:25:03.708144+0000","last_active":"2024-09-16T09:29:03.373431+0000","last_peered":"2024-09-16T09:29:03.373431+0000","last_clean":"2024-09-16T09:29:03.373431+0000","last_became_active":"2024-09-16T09:25:03.707723+0000","last_became_peered":"2024-09-16T09:25:03.707723+0000","last_unstale":"2024-09-16T09:29:03.373431+0000","last_undegraded":"2024-09-16T09:29:03.373431+0000","last_fullsized":"2024-09-16T09:29:03.373431+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:10.620842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":29,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.362411+0000","last_change":"2024-09-16T09:28:56.590999+0000","last_active":"2024-09-16T09:29:03.362411+0000","last_peered":"2024-09-16T09:29:03.362411+0000","last_clean":"2024-09-16T09:29:03.362411+0000","last_became_active":"2024-09-16T09:28:52.983530+0000","last_became_peered":"2024-09-16T09:28:52.983530+0000","last_unstale":"2024-09-16T09:29:03.362411+0000","last_undegraded":"2024-09-16T09:29:03.362411+0000","last_fullsized":"2024-09-16T09:29:03.362411+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:56.590937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:28:56.590937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:13.348023+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":246,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.366220+0000","last_change":"2024-09-16T09:27:59.829866+0000","last_active":"2024-09-16T09:29:03.366220+0000","last_peered":"2024-09-16T09:29:03.366220+0000","last_clean":"2024-09-16T09:29:03.366220+0000","last_became_active":"2024-09-16T09:25:05.711590+0000","last_became_peered":"2024-09-16T09:25:05.711590+0000","last_unstale":"2024-09-16T09:29:03.366220+0000","last_undegraded":"2024-09-16T09:29:03.366220+0000","last_fullsized":"2024-09-16T09:29:03.366220+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:27:59.829789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:59.829789+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:23.888787+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":189,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.417038+0000","last_change":"2024-09-16T09:29:02.667332+0000","last_active":"2024-09-16T09:29:03.417038+0000","last_peered":"2024-09-16T09:29:03.417038+0000","last_clean":"2024-09-16T09:29:03.417038+0000","last_became_active":"2024-09-16T09:25:07.718958+0000","last_became_peered":"2024-09-16T09:25:07.718958+0000","last_unstale":"2024-09-16T09:29:03.417038+0000","last_undegraded":"2024-09-16T09:29:03.417038+0000","last_fullsized":"2024-09-16T09:29:03.417038+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:02.667259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:02.667259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:28.829982+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.8","version":"141'13926","reported_seq":14126,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:07.600389+0000","last_change":"2024-09-16T09:27:17.928717+0000","last_active":"2024-09-16T09:29:07.600389+0000","last_peered":"2024-09-16T09:29:07.600389+0000","last_clean":"2024-09-16T09:29:07.600389+0000","last_became_active":"2024-09-16T09:26:08.118215+0000","last_became_peered":"2024-09-16T09:26:08.118215+0000","last_unstale":"2024-09-16T09:29:07.600389+0000","last_undegraded":"2024-09-16T09:29:07.600389+0000","last_fullsized":"2024-09-16T09:29:07.600389+0000","mapping_epoch":82,"log_start":"118'7800","ondisk_log_start":"118'7800","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"109'6266","last_scrub_stamp":"2024-09-16T09:27:17.927781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_clean_scrub_stamp":"2024-09-16T09:27:17.927781+0000","objects_scrubbed":6214,"log_size":6126,"log_dups_size":0,"ondisk_log_size":6126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:41.349335+0000","scrub_duration":1966,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":136445952,"num_objects":2082,"num_object_clones":0,"num_object_copies":6246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2082,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13926,"num_write_kb":512256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"66'9","reported_seq":205,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.362164+0000","last_change":"2024-09-16T09:28:23.570956+0000","last_active":"2024-09-16T09:29:03.362164+0000","last_peered":"2024-09-16T09:29:03.362164+0000","last_clean":"2024-09-16T09:29:03.362164+0000","last_became_active":"2024-09-16T09:25:03.712230+0000","last_became_peered":"2024-09-16T09:25:03.712230+0000","last_unstale":"2024-09-16T09:29:03.362164+0000","last_undegraded":"2024-09-16T09:29:03.362164+0000","last_fullsized":"2024-09-16T09:29:03.362164+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:28:23.570883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:23.570883+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:50.668824+0000","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":184,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.409569+0000","last_change":"2024-09-16T09:26:26.061173+0000","last_active":"2024-09-16T09:29:03.409569+0000","last_peered":"2024-09-16T09:29:03.409569+0000","last_clean":"2024-09-16T09:29:03.409569+0000","last_became_active":"2024-09-16T09:25:01.702629+0000","last_became_peered":"2024-09-16T09:25:01.702629+0000","last_unstale":"2024-09-16T09:29:03.409569+0000","last_undegraded":"2024-09-16T09:29:03.409569+0000","last_fullsized":"2024-09-16T09:29:03.409569+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:26.061106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:26.061106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":185,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.366057+0000","last_change":"2024-09-16T09:28:57.220125+0000","last_active":"2024-09-16T09:29:03.366057+0000","last_peered":"2024-09-16T09:29:03.366057+0000","last_clean":"2024-09-16T09:29:03.366057+0000","last_became_active":"2024-09-16T09:25:05.712573+0000","last_became_peered":"2024-09-16T09:25:05.712573+0000","last_unstale":"2024-09-16T09:29:03.366057+0000","last_undegraded":"2024-09-16T09:29:03.366057+0000","last_fullsized":"2024-09-16T09:29:03.366057+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:57.220068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:28:57.220068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:23.707294+0000","scrub_duration":33,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":181,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.365192+0000","last_change":"2024-09-16T09:28:25.593045+0000","last_active":"2024-09-16T09:29:03.365192+0000","last_peered":"2024-09-16T09:29:03.365192+0000","last_clean":"2024-09-16T09:29:03.365192+0000","last_became_active":"2024-09-16T09:25:07.718778+0000","last_became_peered":"2024-09-16T09:25:07.718778+0000","last_unstale":"2024-09-16T09:29:03.365192+0000","last_undegraded":"2024-09-16T09:29:03.365192+0000","last_fullsized":"2024-09-16T09:29:03.365192+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:25.592728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:28:25.592728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:42.172423+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"141'13664","reported_seq":13794,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:07.850966+0000","last_change":"2024-09-16T09:26:08.120869+0000","last_active":"2024-09-16T09:29:07.850966+0000","last_peered":"2024-09-16T09:29:07.850966+0000","last_clean":"2024-09-16T09:29:07.850966+0000","last_became_active":"2024-09-16T09:26:08.120579+0000","last_became_peered":"2024-09-16T09:26:08.120579+0000","last_unstale":"2024-09-16T09:29:07.850966+0000","last_undegraded":"2024-09-16T09:29:07.850966+0000","last_fullsized":"2024-09-16T09:29:07.850966+0000","mapping_epoch":82,"log_start":"121'8464","ondisk_log_start":"121'8464","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_clean_scrub_stamp":"2024-09-16T09:26:07.096764+0000","objects_scrubbed":0,"log_size":5200,"log_dups_size":0,"ondisk_log_size":5200,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:34.644187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":125042688,"num_objects":1908,"num_object_clones":0,"num_object_copies":5724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1908,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13664,"num_write_kb":498304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"66'19","reported_seq":220,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.418754+0000","last_change":"2024-09-16T09:29:02.554357+0000","last_active":"2024-09-16T09:29:03.418754+0000","last_peered":"2024-09-16T09:29:03.418754+0000","last_clean":"2024-09-16T09:29:03.418754+0000","last_became_active":"2024-09-16T09:25:03.712783+0000","last_became_peered":"2024-09-16T09:25:03.712783+0000","last_unstale":"2024-09-16T09:29:03.418754+0000","last_undegraded":"2024-09-16T09:29:03.418754+0000","last_fullsized":"2024-09-16T09:29:03.418754+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-16T09:29:02.554274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:29:02.554274+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:14.754043+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"59'1","reported_seq":190,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.422023+0000","last_change":"2024-09-16T09:27:17.653505+0000","last_active":"2024-09-16T09:29:03.422023+0000","last_peered":"2024-09-16T09:29:03.422023+0000","last_clean":"2024-09-16T09:29:03.422023+0000","last_became_active":"2024-09-16T09:25:01.708069+0000","last_became_peered":"2024-09-16T09:25:01.708069+0000","last_unstale":"2024-09-16T09:29:03.422023+0000","last_undegraded":"2024-09-16T09:29:03.422023+0000","last_fullsized":"2024-09-16T09:29:03.422023+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:27:17.653425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:27:17.653425+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:41.906822+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":177,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.366073+0000","last_change":"2024-09-16T09:27:11.067059+0000","last_active":"2024-09-16T09:29:03.366073+0000","last_peered":"2024-09-16T09:29:03.366073+0000","last_clean":"2024-09-16T09:29:03.366073+0000","last_became_active":"2024-09-16T09:25:05.709264+0000","last_became_peered":"2024-09-16T09:25:05.709264+0000","last_unstale":"2024-09-16T09:29:03.366073+0000","last_undegraded":"2024-09-16T09:29:03.366073+0000","last_fullsized":"2024-09-16T09:29:03.366073+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:11.066986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:11.066986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:42.381021+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":172,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.417613+0000","last_change":"2024-09-16T09:27:25.865107+0000","last_active":"2024-09-16T09:29:03.417613+0000","last_peered":"2024-09-16T09:29:03.417613+0000","last_clean":"2024-09-16T09:29:03.417613+0000","last_became_active":"2024-09-16T09:25:07.718750+0000","last_became_peered":"2024-09-16T09:25:07.718750+0000","last_unstale":"2024-09-16T09:29:03.417613+0000","last_undegraded":"2024-09-16T09:29:03.417613+0000","last_fullsized":"2024-09-16T09:29:03.417613+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:25.865030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:25.865030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:35.816657+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.a","version":"141'13754","reported_seq":13926,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:07.644792+0000","last_change":"2024-09-16T09:28:57.251732+0000","last_active":"2024-09-16T09:29:07.644792+0000","last_peered":"2024-09-16T09:29:07.644792+0000","last_clean":"2024-09-16T09:29:07.644792+0000","last_became_active":"2024-09-16T09:28:52.977773+0000","last_became_peered":"2024-09-16T09:28:52.977773+0000","last_unstale":"2024-09-16T09:29:07.644792+0000","last_undegraded":"2024-09-16T09:29:07.644792+0000","last_fullsized":"2024-09-16T09:29:07.644792+0000","mapping_epoch":136,"log_start":"120'8400","ondisk_log_start":"120'8400","created":82,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'13003","last_scrub_stamp":"2024-09-16T09:28:57.251446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_clean_scrub_stamp":"2024-09-16T09:28:57.251446+0000","objects_scrubbed":2837,"log_size":5354,"log_dups_size":0,"ondisk_log_size":5354,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:17.858604+0000","scrub_duration":1085,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":134479872,"num_objects":2052,"num_object_clones":0,"num_object_copies":6156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2052,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13754,"num_write_kb":505792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,5],"acting":[2,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"66'12","reported_seq":204,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.417409+0000","last_change":"2024-09-16T09:27:01.771750+0000","last_active":"2024-09-16T09:29:03.417409+0000","last_peered":"2024-09-16T09:29:03.417409+0000","last_clean":"2024-09-16T09:29:03.417409+0000","last_became_active":"2024-09-16T09:25:03.714027+0000","last_became_peered":"2024-09-16T09:25:03.714027+0000","last_unstale":"2024-09-16T09:29:03.417409+0000","last_undegraded":"2024-09-16T09:29:03.417409+0000","last_fullsized":"2024-09-16T09:29:03.417409+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-16T09:27:01.771667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:01.771667+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:34.629871+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":184,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.422666+0000","last_change":"2024-09-16T09:28:33.761891+0000","last_active":"2024-09-16T09:29:03.422666+0000","last_peered":"2024-09-16T09:29:03.422666+0000","last_clean":"2024-09-16T09:29:03.422666+0000","last_became_active":"2024-09-16T09:25:01.707528+0000","last_became_peered":"2024-09-16T09:25:01.707528+0000","last_unstale":"2024-09-16T09:29:03.422666+0000","last_undegraded":"2024-09-16T09:29:03.422666+0000","last_fullsized":"2024-09-16T09:29:03.422666+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:33.761829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:28:33.761829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:35.167780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":185,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.366310+0000","last_change":"2024-09-16T09:27:49.965628+0000","last_active":"2024-09-16T09:29:03.366310+0000","last_peered":"2024-09-16T09:29:03.366310+0000","last_clean":"2024-09-16T09:29:03.366310+0000","last_became_active":"2024-09-16T09:25:05.715253+0000","last_became_peered":"2024-09-16T09:25:05.715253+0000","last_unstale":"2024-09-16T09:29:03.366310+0000","last_undegraded":"2024-09-16T09:29:03.366310+0000","last_fullsized":"2024-09-16T09:29:03.366310+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:49.965424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:49.965424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:03.758011+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":173,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.359459+0000","last_change":"2024-09-16T09:27:56.059755+0000","last_active":"2024-09-16T09:29:03.359459+0000","last_peered":"2024-09-16T09:29:03.359459+0000","last_clean":"2024-09-16T09:29:03.359459+0000","last_became_active":"2024-09-16T09:25:07.718899+0000","last_became_peered":"2024-09-16T09:25:07.718899+0000","last_unstale":"2024-09-16T09:29:03.359459+0000","last_undegraded":"2024-09-16T09:29:03.359459+0000","last_fullsized":"2024-09-16T09:29:03.359459+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:56.059687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:56.059687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"141'13519","reported_seq":13761,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:07.614767+0000","last_change":"2024-09-16T09:28:49.711492+0000","last_active":"2024-09-16T09:29:07.614767+0000","last_peered":"2024-09-16T09:29:07.614767+0000","last_clean":"2024-09-16T09:29:07.614767+0000","last_became_active":"2024-09-16T09:26:08.117869+0000","last_became_peered":"2024-09-16T09:26:08.117869+0000","last_unstale":"2024-09-16T09:29:07.614767+0000","last_undegraded":"2024-09-16T09:29:07.614767+0000","last_fullsized":"2024-09-16T09:29:07.614767+0000","mapping_epoch":82,"log_start":"122'8656","ondisk_log_start":"122'8656","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'12346","last_scrub_stamp":"2024-09-16T09:28:49.711386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:26:07.096764+0000","last_clean_scrub_stamp":"2024-09-16T09:28:49.711386+0000","objects_scrubbed":3213,"log_size":4863,"log_dups_size":0,"ondisk_log_size":4863,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:59.088493+0000","scrub_duration":918,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131792896,"num_objects":2011,"num_object_clones":0,"num_object_copies":6033,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2011,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13519,"num_write_kb":496960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"66'15","reported_seq":205,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.409000+0000","last_change":"2024-09-16T09:26:33.138828+0000","last_active":"2024-09-16T09:29:03.409000+0000","last_peered":"2024-09-16T09:29:03.409000+0000","last_clean":"2024-09-16T09:29:03.409000+0000","last_became_active":"2024-09-16T09:25:03.715733+0000","last_became_peered":"2024-09-16T09:25:03.715733+0000","last_unstale":"2024-09-16T09:29:03.409000+0000","last_undegraded":"2024-09-16T09:29:03.409000+0000","last_fullsized":"2024-09-16T09:29:03.409000+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-16T09:26:33.138603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:26:33.138603+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:46.098021+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":196,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.423123+0000","last_change":"2024-09-16T09:26:26.705847+0000","last_active":"2024-09-16T09:29:03.423123+0000","last_peered":"2024-09-16T09:29:03.423123+0000","last_clean":"2024-09-16T09:29:03.423123+0000","last_became_active":"2024-09-16T09:25:01.707372+0000","last_became_peered":"2024-09-16T09:25:01.707372+0000","last_unstale":"2024-09-16T09:29:03.423123+0000","last_undegraded":"2024-09-16T09:29:03.423123+0000","last_fullsized":"2024-09-16T09:29:03.423123+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-16T09:26:26.705773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:26.705773+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:40.875025+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"66'8","reported_seq":242,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.417015+0000","last_change":"2024-09-16T09:29:00.749254+0000","last_active":"2024-09-16T09:29:03.417015+0000","last_peered":"2024-09-16T09:29:03.417015+0000","last_clean":"2024-09-16T09:29:03.417015+0000","last_became_active":"2024-09-16T09:25:05.716189+0000","last_became_peered":"2024-09-16T09:25:05.716189+0000","last_unstale":"2024-09-16T09:29:03.417015+0000","last_undegraded":"2024-09-16T09:29:03.417015+0000","last_fullsized":"2024-09-16T09:29:03.417015+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:29:00.749177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:00.749177+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:03.301570+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":172,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.417581+0000","last_change":"2024-09-16T09:27:20.219376+0000","last_active":"2024-09-16T09:29:03.417581+0000","last_peered":"2024-09-16T09:29:03.417581+0000","last_clean":"2024-09-16T09:29:03.417581+0000","last_became_active":"2024-09-16T09:25:07.717213+0000","last_became_peered":"2024-09-16T09:25:07.717213+0000","last_unstale":"2024-09-16T09:29:03.417581+0000","last_undegraded":"2024-09-16T09:29:03.417581+0000","last_fullsized":"2024-09-16T09:29:03.417581+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:20.219333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:20.219333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:37.118971+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":185,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.409672+0000","last_change":"2024-09-16T09:26:33.139405+0000","last_active":"2024-09-16T09:29:03.409672+0000","last_peered":"2024-09-16T09:29:03.409672+0000","last_clean":"2024-09-16T09:29:03.409672+0000","last_became_active":"2024-09-16T09:25:01.703202+0000","last_became_peered":"2024-09-16T09:25:01.703202+0000","last_unstale":"2024-09-16T09:29:03.409672+0000","last_undegraded":"2024-09-16T09:29:03.409672+0000","last_fullsized":"2024-09-16T09:29:03.409672+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:33.139360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:33.139360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:05.488909+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"66'6","reported_seq":190,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.353384+0000","last_change":"2024-09-16T09:26:23.458888+0000","last_active":"2024-09-16T09:29:03.353384+0000","last_peered":"2024-09-16T09:29:03.353384+0000","last_clean":"2024-09-16T09:29:03.353384+0000","last_became_active":"2024-09-16T09:25:03.713245+0000","last_became_peered":"2024-09-16T09:25:03.713245+0000","last_unstale":"2024-09-16T09:29:03.353384+0000","last_undegraded":"2024-09-16T09:29:03.353384+0000","last_fullsized":"2024-09-16T09:29:03.353384+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'6","last_scrub_stamp":"2024-09-16T09:26:23.458794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:26:23.458794+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:46.087500+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":176,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.418320+0000","last_change":"2024-09-16T09:27:55.154889+0000","last_active":"2024-09-16T09:29:03.418320+0000","last_peered":"2024-09-16T09:29:03.418320+0000","last_clean":"2024-09-16T09:29:03.418320+0000","last_became_active":"2024-09-16T09:25:05.715616+0000","last_became_peered":"2024-09-16T09:25:05.715616+0000","last_unstale":"2024-09-16T09:29:03.418320+0000","last_undegraded":"2024-09-16T09:29:03.418320+0000","last_fullsized":"2024-09-16T09:29:03.418320+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:55.154839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:55.154839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":172,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.421674+0000","last_change":"2024-09-16T09:26:28.733381+0000","last_active":"2024-09-16T09:29:03.421674+0000","last_peered":"2024-09-16T09:29:03.421674+0000","last_clean":"2024-09-16T09:29:03.421674+0000","last_became_active":"2024-09-16T09:25:07.721059+0000","last_became_peered":"2024-09-16T09:25:07.721059+0000","last_unstale":"2024-09-16T09:29:03.421674+0000","last_undegraded":"2024-09-16T09:29:03.421674+0000","last_fullsized":"2024-09-16T09:29:03.421674+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:28.733324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:28.733324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:05.819123+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"66'9","reported_seq":205,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.362978+0000","last_change":"2024-09-16T09:28:26.591121+0000","last_active":"2024-09-16T09:29:03.362978+0000","last_peered":"2024-09-16T09:29:03.362978+0000","last_clean":"2024-09-16T09:29:03.362978+0000","last_became_active":"2024-09-16T09:25:03.719573+0000","last_became_peered":"2024-09-16T09:25:03.719573+0000","last_unstale":"2024-09-16T09:29:03.362978+0000","last_undegraded":"2024-09-16T09:29:03.362978+0000","last_fullsized":"2024-09-16T09:29:03.362978+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:28:26.590949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:26.590949+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:36.843827+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":192,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.417364+0000","last_change":"2024-09-16T09:28:50.698331+0000","last_active":"2024-09-16T09:29:03.417364+0000","last_peered":"2024-09-16T09:29:03.417364+0000","last_clean":"2024-09-16T09:29:03.417364+0000","last_became_active":"2024-09-16T09:25:01.706491+0000","last_became_peered":"2024-09-16T09:25:01.706491+0000","last_unstale":"2024-09-16T09:29:03.417364+0000","last_undegraded":"2024-09-16T09:29:03.417364+0000","last_fullsized":"2024-09-16T09:29:03.417364+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:50.698288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:28:50.698288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:05.036276+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":29,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.354091+0000","last_change":"2024-09-16T09:28:53.810644+0000","last_active":"2024-09-16T09:29:03.354091+0000","last_peered":"2024-09-16T09:29:03.354091+0000","last_clean":"2024-09-16T09:29:03.354091+0000","last_became_active":"2024-09-16T09:28:52.977688+0000","last_became_peered":"2024-09-16T09:28:52.977688+0000","last_unstale":"2024-09-16T09:29:03.354091+0000","last_undegraded":"2024-09-16T09:29:03.354091+0000","last_fullsized":"2024-09-16T09:29:03.354091+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:53.810580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:28:53.810580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:00.227485+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":173,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.365861+0000","last_change":"2024-09-16T09:26:35.866582+0000","last_active":"2024-09-16T09:29:03.365861+0000","last_peered":"2024-09-16T09:29:03.365861+0000","last_clean":"2024-09-16T09:29:03.365861+0000","last_became_active":"2024-09-16T09:25:07.722050+0000","last_became_peered":"2024-09-16T09:25:07.722050+0000","last_unstale":"2024-09-16T09:29:03.365861+0000","last_undegraded":"2024-09-16T09:29:03.365861+0000","last_fullsized":"2024-09-16T09:29:03.365861+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:35.866512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:35.866512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:40.319226+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"66'9","reported_seq":196,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.410003+0000","last_change":"2024-09-16T09:27:57.144868+0000","last_active":"2024-09-16T09:29:03.410003+0000","last_peered":"2024-09-16T09:29:03.410003+0000","last_clean":"2024-09-16T09:29:03.410003+0000","last_became_active":"2024-09-16T09:25:03.717310+0000","last_became_peered":"2024-09-16T09:25:03.717310+0000","last_unstale":"2024-09-16T09:29:03.410003+0000","last_undegraded":"2024-09-16T09:29:03.410003+0000","last_fullsized":"2024-09-16T09:29:03.410003+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:27:57.144791+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:57.144791+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":185,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.362714+0000","last_change":"2024-09-16T09:27:12.652668+0000","last_active":"2024-09-16T09:29:03.362714+0000","last_peered":"2024-09-16T09:29:03.362714+0000","last_clean":"2024-09-16T09:29:03.362714+0000","last_became_active":"2024-09-16T09:25:01.702419+0000","last_became_peered":"2024-09-16T09:25:01.702419+0000","last_unstale":"2024-09-16T09:29:03.362714+0000","last_undegraded":"2024-09-16T09:29:03.362714+0000","last_fullsized":"2024-09-16T09:29:03.362714+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:12.652603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:27:12.652603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:37.554413+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"137'9","reported_seq":58,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.416926+0000","last_change":"2024-09-16T09:28:53.714478+0000","last_active":"2024-09-16T09:29:03.416926+0000","last_peered":"2024-09-16T09:29:03.416926+0000","last_clean":"2024-09-16T09:29:03.416926+0000","last_became_active":"2024-09-16T09:28:52.977923+0000","last_became_peered":"2024-09-16T09:28:52.977923+0000","last_unstale":"2024-09-16T09:29:03.416926+0000","last_undegraded":"2024-09-16T09:29:03.416926+0000","last_fullsized":"2024-09-16T09:29:03.416926+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:28:53.714413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:28:53.714413+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:08.966865+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":181,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.417684+0000","last_change":"2024-09-16T09:28:22.600081+0000","last_active":"2024-09-16T09:29:03.417684+0000","last_peered":"2024-09-16T09:29:03.417684+0000","last_clean":"2024-09-16T09:29:03.417684+0000","last_became_active":"2024-09-16T09:25:07.718810+0000","last_became_peered":"2024-09-16T09:25:07.718810+0000","last_unstale":"2024-09-16T09:29:03.417684+0000","last_undegraded":"2024-09-16T09:29:03.417684+0000","last_fullsized":"2024-09-16T09:29:03.417684+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:22.600027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:28:22.600027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:36.033169+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"66'10","reported_seq":64,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.354487+0000","last_change":"2024-09-16T09:28:56.825675+0000","last_active":"2024-09-16T09:29:03.354487+0000","last_peered":"2024-09-16T09:29:03.354487+0000","last_clean":"2024-09-16T09:29:03.354487+0000","last_became_active":"2024-09-16T09:28:52.979059+0000","last_became_peered":"2024-09-16T09:28:52.979059+0000","last_unstale":"2024-09-16T09:29:03.354487+0000","last_undegraded":"2024-09-16T09:29:03.354487+0000","last_fullsized":"2024-09-16T09:29:03.354487+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-16T09:28:56.825476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:56.825476+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:02.744116+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":14,"num_read_kb":9,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":184,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.419529+0000","last_change":"2024-09-16T09:28:32.753479+0000","last_active":"2024-09-16T09:29:03.419529+0000","last_peered":"2024-09-16T09:29:03.419529+0000","last_clean":"2024-09-16T09:29:03.419529+0000","last_became_active":"2024-09-16T09:25:01.707513+0000","last_became_peered":"2024-09-16T09:25:01.707513+0000","last_unstale":"2024-09-16T09:29:03.419529+0000","last_undegraded":"2024-09-16T09:29:03.419529+0000","last_fullsized":"2024-09-16T09:29:03.419529+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:32.753408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:28:32.753408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:49.069771+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"66'8","reported_seq":242,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.409840+0000","last_change":"2024-09-16T09:28:23.185312+0000","last_active":"2024-09-16T09:29:03.409840+0000","last_peered":"2024-09-16T09:29:03.409840+0000","last_clean":"2024-09-16T09:29:03.409840+0000","last_became_active":"2024-09-16T09:25:05.713833+0000","last_became_peered":"2024-09-16T09:25:05.713833+0000","last_unstale":"2024-09-16T09:29:03.409840+0000","last_undegraded":"2024-09-16T09:29:03.409840+0000","last_fullsized":"2024-09-16T09:29:03.409840+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:28:23.185266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:28:23.185266+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:33.284839+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":173,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.362544+0000","last_change":"2024-09-16T09:27:51.639488+0000","last_active":"2024-09-16T09:29:03.362544+0000","last_peered":"2024-09-16T09:29:03.362544+0000","last_clean":"2024-09-16T09:29:03.362544+0000","last_became_active":"2024-09-16T09:25:07.717357+0000","last_became_peered":"2024-09-16T09:25:07.717357+0000","last_unstale":"2024-09-16T09:29:03.362544+0000","last_undegraded":"2024-09-16T09:29:03.362544+0000","last_fullsized":"2024-09-16T09:29:03.362544+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:51.639428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:51.639428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:19.906242+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"66'11","reported_seq":211,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.373409+0000","last_change":"2024-09-16T09:26:33.302713+0000","last_active":"2024-09-16T09:29:03.373409+0000","last_peered":"2024-09-16T09:29:03.373409+0000","last_clean":"2024-09-16T09:29:03.373409+0000","last_became_active":"2024-09-16T09:26:33.291265+0000","last_became_peered":"2024-09-16T09:26:33.291265+0000","last_unstale":"2024-09-16T09:29:03.373409+0000","last_undegraded":"2024-09-16T09:29:03.373409+0000","last_fullsized":"2024-09-16T09:29:03.373409+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:42.620583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":193,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.416529+0000","last_change":"2024-09-16T09:28:28.573613+0000","last_active":"2024-09-16T09:29:03.416529+0000","last_peered":"2024-09-16T09:29:03.416529+0000","last_clean":"2024-09-16T09:29:03.416529+0000","last_became_active":"2024-09-16T09:25:01.705158+0000","last_became_peered":"2024-09-16T09:25:01.705158+0000","last_unstale":"2024-09-16T09:29:03.416529+0000","last_undegraded":"2024-09-16T09:29:03.416529+0000","last_fullsized":"2024-09-16T09:29:03.416529+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:28.573434+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:28:28.573434+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:43.428009+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":185,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.358047+0000","last_change":"2024-09-16T09:27:46.034727+0000","last_active":"2024-09-16T09:29:03.358047+0000","last_peered":"2024-09-16T09:29:03.358047+0000","last_clean":"2024-09-16T09:29:03.358047+0000","last_became_active":"2024-09-16T09:25:05.712816+0000","last_became_peered":"2024-09-16T09:25:05.712816+0000","last_unstale":"2024-09-16T09:29:03.358047+0000","last_undegraded":"2024-09-16T09:29:03.358047+0000","last_fullsized":"2024-09-16T09:29:03.358047+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:46.034672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:46.034672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":22,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.365095+0000","last_change":"2024-09-16T09:28:52.982570+0000","last_active":"2024-09-16T09:29:03.365095+0000","last_peered":"2024-09-16T09:29:03.365095+0000","last_clean":"2024-09-16T09:29:03.365095+0000","last_became_active":"2024-09-16T09:28:52.982243+0000","last_became_peered":"2024-09-16T09:28:52.982243+0000","last_unstale":"2024-09-16T09:29:03.365095+0000","last_undegraded":"2024-09-16T09:29:03.365095+0000","last_fullsized":"2024-09-16T09:29:03.365095+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:48.147759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:48.147759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:05.941445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"66'9","reported_seq":197,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.358402+0000","last_change":"2024-09-16T09:26:31.216495+0000","last_active":"2024-09-16T09:29:03.358402+0000","last_peered":"2024-09-16T09:29:03.358402+0000","last_clean":"2024-09-16T09:29:03.358402+0000","last_became_active":"2024-09-16T09:25:03.705807+0000","last_became_peered":"2024-09-16T09:25:03.705807+0000","last_unstale":"2024-09-16T09:29:03.358402+0000","last_undegraded":"2024-09-16T09:29:03.358402+0000","last_fullsized":"2024-09-16T09:29:03.358402+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:26:31.216298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:26:31.216298+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:03.950024+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":184,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.421883+0000","last_change":"2024-09-16T09:26:23.796308+0000","last_active":"2024-09-16T09:29:03.421883+0000","last_peered":"2024-09-16T09:29:03.421883+0000","last_clean":"2024-09-16T09:29:03.421883+0000","last_became_active":"2024-09-16T09:25:01.706762+0000","last_became_peered":"2024-09-16T09:25:01.706762+0000","last_unstale":"2024-09-16T09:29:03.421883+0000","last_undegraded":"2024-09-16T09:29:03.421883+0000","last_fullsized":"2024-09-16T09:29:03.421883+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:23.796231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:23.796231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:39.839678+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":21,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.361930+0000","last_change":"2024-09-16T09:28:52.983144+0000","last_active":"2024-09-16T09:29:03.361930+0000","last_peered":"2024-09-16T09:29:03.361930+0000","last_clean":"2024-09-16T09:29:03.361930+0000","last_became_active":"2024-09-16T09:28:52.982865+0000","last_became_peered":"2024-09-16T09:28:52.982865+0000","last_unstale":"2024-09-16T09:29:03.361930+0000","last_undegraded":"2024-09-16T09:29:03.361930+0000","last_fullsized":"2024-09-16T09:29:03.361930+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:44.241942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:44.241942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":181,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.361565+0000","last_change":"2024-09-16T09:28:52.552567+0000","last_active":"2024-09-16T09:29:03.361565+0000","last_peered":"2024-09-16T09:29:03.361565+0000","last_clean":"2024-09-16T09:29:03.361565+0000","last_became_active":"2024-09-16T09:25:07.715301+0000","last_became_peered":"2024-09-16T09:25:07.715301+0000","last_unstale":"2024-09-16T09:29:03.361565+0000","last_undegraded":"2024-09-16T09:29:03.361565+0000","last_fullsized":"2024-09-16T09:29:03.361565+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:52.552495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:28:52.552495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:11.151721+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"66'11","reported_seq":68,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.417165+0000","last_change":"2024-09-16T09:28:55.643505+0000","last_active":"2024-09-16T09:29:03.417165+0000","last_peered":"2024-09-16T09:29:03.417165+0000","last_clean":"2024-09-16T09:29:03.417165+0000","last_became_active":"2024-09-16T09:28:52.971132+0000","last_became_peered":"2024-09-16T09:28:52.971132+0000","last_unstale":"2024-09-16T09:29:03.417165+0000","last_undegraded":"2024-09-16T09:29:03.417165+0000","last_fullsized":"2024-09-16T09:29:03.417165+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-16T09:28:55.643448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:55.643448+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:24.166714+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":19,"num_read_kb":12,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":192,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.418448+0000","last_change":"2024-09-16T09:27:46.175159+0000","last_active":"2024-09-16T09:29:03.418448+0000","last_peered":"2024-09-16T09:29:03.418448+0000","last_clean":"2024-09-16T09:29:03.418448+0000","last_became_active":"2024-09-16T09:25:01.700733+0000","last_became_peered":"2024-09-16T09:25:01.700733+0000","last_unstale":"2024-09-16T09:29:03.418448+0000","last_undegraded":"2024-09-16T09:29:03.418448+0000","last_fullsized":"2024-09-16T09:29:03.418448+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:46.175093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:27:46.175093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:15.134231+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":176,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.416635+0000","last_change":"2024-09-16T09:27:56.745764+0000","last_active":"2024-09-16T09:29:03.416635+0000","last_peered":"2024-09-16T09:29:03.416635+0000","last_clean":"2024-09-16T09:29:03.416635+0000","last_became_active":"2024-09-16T09:25:05.713479+0000","last_became_peered":"2024-09-16T09:25:05.713479+0000","last_unstale":"2024-09-16T09:29:03.416635+0000","last_undegraded":"2024-09-16T09:29:03.416635+0000","last_fullsized":"2024-09-16T09:29:03.416635+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:56.745720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:56.745720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:12.277156+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":30,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.362326+0000","last_change":"2024-09-16T09:28:54.589320+0000","last_active":"2024-09-16T09:29:03.362326+0000","last_peered":"2024-09-16T09:29:03.362326+0000","last_clean":"2024-09-16T09:29:03.362326+0000","last_became_active":"2024-09-16T09:28:52.982251+0000","last_became_peered":"2024-09-16T09:28:52.982251+0000","last_unstale":"2024-09-16T09:29:03.362326+0000","last_undegraded":"2024-09-16T09:29:03.362326+0000","last_fullsized":"2024-09-16T09:29:03.362326+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:54.589258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:28:54.589258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:22.262118+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"66'4","reported_seq":187,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.363151+0000","last_change":"2024-09-16T09:26:27.474085+0000","last_active":"2024-09-16T09:29:03.363151+0000","last_peered":"2024-09-16T09:29:03.363151+0000","last_clean":"2024-09-16T09:29:03.363151+0000","last_became_active":"2024-09-16T09:25:03.704869+0000","last_became_peered":"2024-09-16T09:25:03.704869+0000","last_unstale":"2024-09-16T09:29:03.363151+0000","last_undegraded":"2024-09-16T09:29:03.363151+0000","last_fullsized":"2024-09-16T09:29:03.363151+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'4","last_scrub_stamp":"2024-09-16T09:26:27.473740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:26:27.473740+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:08.111779+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":193,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.364301+0000","last_change":"2024-09-16T09:28:32.638113+0000","last_active":"2024-09-16T09:29:03.364301+0000","last_peered":"2024-09-16T09:29:03.364301+0000","last_clean":"2024-09-16T09:29:03.364301+0000","last_became_active":"2024-09-16T09:25:01.705686+0000","last_became_peered":"2024-09-16T09:25:01.705686+0000","last_unstale":"2024-09-16T09:29:03.364301+0000","last_undegraded":"2024-09-16T09:29:03.364301+0000","last_fullsized":"2024-09-16T09:29:03.364301+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:32.638048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:28:32.638048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:55.366007+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":185,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.409931+0000","last_change":"2024-09-16T09:28:21.658148+0000","last_active":"2024-09-16T09:29:03.409931+0000","last_peered":"2024-09-16T09:29:03.409931+0000","last_clean":"2024-09-16T09:29:03.409931+0000","last_became_active":"2024-09-16T09:25:05.712545+0000","last_became_peered":"2024-09-16T09:25:05.712545+0000","last_unstale":"2024-09-16T09:29:03.409931+0000","last_undegraded":"2024-09-16T09:29:03.409931+0000","last_fullsized":"2024-09-16T09:29:03.409931+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:21.658103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:28:21.658103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:48.560584+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":172,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.417549+0000","last_change":"2024-09-16T09:27:58.840101+0000","last_active":"2024-09-16T09:29:03.417549+0000","last_peered":"2024-09-16T09:29:03.417549+0000","last_clean":"2024-09-16T09:29:03.417549+0000","last_became_active":"2024-09-16T09:25:07.720117+0000","last_became_peered":"2024-09-16T09:25:07.720117+0000","last_unstale":"2024-09-16T09:29:03.417549+0000","last_undegraded":"2024-09-16T09:29:03.417549+0000","last_fullsized":"2024-09-16T09:29:03.417549+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:58.840039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:58.840039+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:14.862907+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":173,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.358057+0000","last_change":"2024-09-16T09:27:52.022115+0000","last_active":"2024-09-16T09:29:03.358057+0000","last_peered":"2024-09-16T09:29:03.358057+0000","last_clean":"2024-09-16T09:29:03.358057+0000","last_became_active":"2024-09-16T09:25:07.712594+0000","last_became_peered":"2024-09-16T09:25:07.712594+0000","last_unstale":"2024-09-16T09:29:03.358057+0000","last_undegraded":"2024-09-16T09:29:03.358057+0000","last_fullsized":"2024-09-16T09:29:03.358057+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:52.021842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:52.021842+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:16.470972+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.362091+0000","last_change":"2024-09-16T09:28:52.982847+0000","last_active":"2024-09-16T09:29:03.362091+0000","last_peered":"2024-09-16T09:29:03.362091+0000","last_clean":"2024-09-16T09:29:03.362091+0000","last_became_active":"2024-09-16T09:28:52.982171+0000","last_became_peered":"2024-09-16T09:28:52.982171+0000","last_unstale":"2024-09-16T09:29:03.362091+0000","last_undegraded":"2024-09-16T09:29:03.362091+0000","last_fullsized":"2024-09-16T09:29:03.362091+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:32.168204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:28:32.168204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:38.622006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"66'11","reported_seq":208,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.419541+0000","last_change":"2024-09-16T09:27:57.486459+0000","last_active":"2024-09-16T09:29:03.419541+0000","last_peered":"2024-09-16T09:29:03.419541+0000","last_clean":"2024-09-16T09:29:03.419541+0000","last_became_active":"2024-09-16T09:25:03.707769+0000","last_became_peered":"2024-09-16T09:25:03.707769+0000","last_unstale":"2024-09-16T09:29:03.419541+0000","last_undegraded":"2024-09-16T09:29:03.419541+0000","last_fullsized":"2024-09-16T09:29:03.419541+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-16T09:27:57.486381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:57.486381+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":177,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-09-16T09:29:03.362762+0000","last_change":"2024-09-16T09:27:57.640962+0000","last_active":"2024-09-16T09:29:03.362762+0000","last_peered":"2024-09-16T09:29:03.362762+0000","last_clean":"2024-09-16T09:29:03.362762+0000","last_became_active":"2024-09-16T09:25:05.711952+0000","last_became_peered":"2024-09-16T09:25:05.711952+0000","last_unstale":"2024-09-16T09:29:03.362762+0000","last_undegraded":"2024-09-16T09:29:03.362762+0000","last_fullsized":"2024-09-16T09:29:03.362762+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:57.640756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:57.640756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:09.172616+0000","scrub_duration":44,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":16,"stat_sum":{"num_bytes":2106720280,"num_objects":32147,"num_object_clones":0,"num_object_copies":96441,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32147,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":220111,"num_write_kb":8072193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":6321811456,"data_stored":6321799240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":84478,"ondisk_log_size":84478,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":66,"ondisk_log_size":66,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":693,"num_read_kb":450,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":393,"ondisk_log_size":393,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":240,"num_read_kb":235,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":11,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":869,"ondisk_log_size":869,"up":33,"acting":33,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266742,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":783428,"kb_used_data":671732,"kb_used_omap":1,"kb_used_meta":111678,"kb_avail":92961724,"statfs":{"total":95995035648,"available":95192805376,"internally_reserved":0,"allocated":687853568,"data_stored":687519997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":114358730},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495675,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":747312,"kb_used_data":643936,"kb_used_omap":1,"kb_used_meta":103358,"kb_avail":92997840,"statfs":{"total":95995035648,"available":95229788160,"internally_reserved":0,"allocated":659390464,"data_stored":659076299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":105839046},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724610,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":590200,"kb_used_data":522920,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":93154952,"statfs":{"total":95995035648,"available":95390670848,"internally_reserved":0,"allocated":535470080,"data_stored":535148712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68876742},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953543,"num_pgs":65,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":772040,"kb_used_data":664504,"kb_used_omap":1,"kb_used_meta":107518,"kb_avail":92973112,"statfs":{"total":95995035648,"available":95204466688,"internally_reserved":0,"allocated":680452096,"data_stored":680114493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":110098886},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149773,"num_pgs":70,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1346260,"kb_used_data":1180356,"kb_used_omap":1,"kb_used_meta":165822,"kb_avail":92398892,"statfs":{"total":95995035648,"available":94616465408,"internally_reserved":0,"allocated":1208684544,"data_stored":1208335483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":169802183},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411410,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":898032,"kb_used_data":798624,"kb_used_omap":1,"kb_used_meta":99198,"kb_avail":92847120,"statfs":{"total":95995035648,"available":95075450880,"internally_reserved":0,"allocated":817790976,"data_stored":817477263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":101579205},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3,0,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574937,"num_pgs":62,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":885156,"kb_used_data":785812,"kb_used_omap":1,"kb_used_meta":99198,"kb_avail":92859996,"statfs":{"total":95995035648,"available":95088635904,"internally_reserved":0,"allocated":804671488,"data_stored":804305107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":101579205},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738463,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1171404,"kb_used_data":1023612,"kb_used_omap":1,"kb_used_meta":147774,"kb_avail":92573748,"statfs":{"total":95995035648,"available":94795517952,"internally_reserved":0,"allocated":1048178688,"data_stored":1047837215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":151321029},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,4,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1038487552,"data_stored":1038483480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":782962688,"data_stored":782958616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":801439744,"data_stored":801439744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1182662656,"data_stored":1182662656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":660406272,"data_stored":660406272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":531365888,"data_stored":531365888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":657723392,"data_stored":657719320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":666763264,"data_stored":666763264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T09:29:14.039 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd pool set unique_pool_0 pgp_num 16 2024-09-16T09:29:14.040 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:14.040 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:14.040 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:29:14.141 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:29:14.169 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:14.170 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:14.170 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:29:14.180 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T09:29:14.244 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:14 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:29:14.244 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:14 smithi070 ceph-mon[30388]: from='client.15072 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:29:14.245 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:14 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:29:14.245 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:14 smithi070 ceph-mon[40691]: from='client.15072 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:29:14.270 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:29:14.295 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:14.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:14.296 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:29:14.390 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:29:14.397 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:29:14.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:14 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:29:14.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:14 smithi084 ceph-mon[32629]: from='client.15072 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:29:14.421 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:14.422 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:14.422 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:29:14.523 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:29:14.603 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:14.632 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:14.633 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:29:14.734 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:29:14.761 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:14.761 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:14.762 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:29:14.862 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:29:14.888 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:14.889 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:14.889 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:29:14.990 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:29:15.015 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:15.016 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:15.016 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:29:15.057 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T09:29:15.057 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [ 2024-09-16T09:29:15.057 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:15.058 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.24698.0:227824 7.5 e142/82 7:ac97f22c:::benchmark_data_smithi070_78438_object101693:head v 142'14453, mlcod=142'14451)", 2024-09-16T09:29:15.058 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:29:15.033525+0000", 2024-09-16T09:29:15.058 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.020957104000000001, 2024-09-16T09:29:15.058 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.020984605, 2024-09-16T09:29:15.058 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:29:15.058 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:29:15.058 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T09:29:15.058 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:29:15.059 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:15.059 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:29:15.059 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:15.033525+0000", 2024-09-16T09:29:15.059 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:29:15.059 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:15.059 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:15.059 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:29:15.060 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:15.033525+0000", 2024-09-16T09:29:15.060 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:29:15.060 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:15.060 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:15.060 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:29:15.060 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:15.033526+0000", 2024-09-16T09:29:15.060 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.5300000000000004e-07 2024-09-16T09:29:15.061 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:15.061 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:15.061 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:29:15.061 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:15.033531+0000", 2024-09-16T09:29:15.061 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0980000000000001e-06 2024-09-16T09:29:15.061 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:15.061 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:15.061 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:29:15.061 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:15.033538+0000", 2024-09-16T09:29:15.061 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.469e-06 2024-09-16T09:29:15.061 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:15.061 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:15.062 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:29:15.062 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:15.033544+0000", 2024-09-16T09:29:15.062 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5990000000000001e-06 2024-09-16T09:29:15.062 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:15.062 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:15.062 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:29:15.062 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:15.039232+0000", 2024-09-16T09:29:15.062 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0056885060000000003 2024-09-16T09:29:15.062 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:15.062 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:15.062 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:29:15.062 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:15.039252+0000", 2024-09-16T09:29:15.063 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0058e-05 2024-09-16T09:29:15.063 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:29:15.063 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:29:15.063 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:29:15.063 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:15.063 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:15.063 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.24698.0:227825 7.7 e142/82 7:ed9f2b2c:::benchmark_data_smithi070_78438_object101694:head v 142'14372, mlcod=142'14369)", 2024-09-16T09:29:15.063 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:29:15.034170+0000", 2024-09-16T09:29:15.063 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.020312050000000002, 2024-09-16T09:29:15.063 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.020395483999999998, 2024-09-16T09:29:15.063 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:29:15.063 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:29:15.064 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T09:29:15.064 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:29:15.064 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:15.064 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:29:15.064 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:15.034170+0000", 2024-09-16T09:29:15.064 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:29:15.064 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:15.064 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:15.064 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:29:15.064 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:15.034170+0000", 2024-09-16T09:29:15.064 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:29:15.064 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:15.065 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:15.065 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:29:15.065 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:15.034171+0000", 2024-09-16T09:29:15.065 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.4600000000000004e-07 2024-09-16T09:29:15.065 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:15.065 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:15.065 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:29:15.065 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:15.034176+0000", 2024-09-16T09:29:15.065 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.1399999999999999e-06 2024-09-16T09:29:15.065 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:15.065 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:15.065 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:29:15.065 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:15.034186+0000", 2024-09-16T09:29:15.065 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.8069999999999994e-06 2024-09-16T09:29:15.066 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:15.066 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:15.066 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:29:15.066 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:15.034192+0000", 2024-09-16T09:29:15.066 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.9449999999999998e-06 2024-09-16T09:29:15.066 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:15.066 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:15.066 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:29:15.066 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:15.036294+0000", 2024-09-16T09:29:15.066 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.002102114 2024-09-16T09:29:15.066 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:15.066 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:15.066 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:29:15.066 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:15.036312+0000", 2024-09-16T09:29:15.067 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7872000000000001e-05 2024-09-16T09:29:15.067 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:29:15.067 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:29:15.067 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:29:15.067 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:15.067 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:15.067 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.24698.0:227827 7.d e142/82 7:b6a3a28f:::benchmark_data_smithi070_78438_object101696:head v 142'14143, mlcod=142'14140)", 2024-09-16T09:29:15.067 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:29:15.034970+0000", 2024-09-16T09:29:15.067 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.019512096999999999, 2024-09-16T09:29:15.067 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.019643362000000001, 2024-09-16T09:29:15.067 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:29:15.067 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:29:15.067 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T09:29:15.067 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:29:15.068 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:15.068 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:29:15.068 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:15.034970+0000", 2024-09-16T09:29:15.068 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:29:15.068 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:15.068 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:15.068 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:29:15.068 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:15.034970+0000", 2024-09-16T09:29:15.068 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:29:15.068 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:15.068 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:15.068 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:29:15.068 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:15.034971+0000", 2024-09-16T09:29:15.068 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.75e-07 2024-09-16T09:29:15.069 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:15.069 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:15.069 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:29:15.069 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:15.034976+0000", 2024-09-16T09:29:15.069 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.9189999999999998e-06 2024-09-16T09:29:15.069 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:15.069 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:15.069 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:29:15.069 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:15.035002+0000", 2024-09-16T09:29:15.069 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.6665999999999998e-05 2024-09-16T09:29:15.069 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:15.069 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:15.069 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:29:15.069 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:15.035008+0000", 2024-09-16T09:29:15.070 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1079999999999998e-06 2024-09-16T09:29:15.070 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:15.070 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:15.070 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:29:15.070 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:15.044122+0000", 2024-09-16T09:29:15.070 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0091140679999999995 2024-09-16T09:29:15.070 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:15.070 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:15.070 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:29:15.070 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:15.044140+0000", 2024-09-16T09:29:15.070 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7346e-05 2024-09-16T09:29:15.071 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:29:15.071 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:29:15.071 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:29:15.071 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:15.071 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:15.071 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.24698.0:227830 7.6 7:6596533c:::benchmark_data_smithi070_78438_object101699:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e142)", 2024-09-16T09:29:15.071 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:29:15.036300+0000", 2024-09-16T09:29:15.071 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.018181691999999999, 2024-09-16T09:29:15.071 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.018365056000000001, 2024-09-16T09:29:15.071 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:29:15.071 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:29:15.071 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "waiting for sub ops", 2024-09-16T09:29:15.072 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:29:15.072 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.24698", 2024-09-16T09:29:15.072 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:29:15.072 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 227830 2024-09-16T09:29:15.072 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:15.072 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:29:15.072 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:15.072 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:29:15.072 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:15.036300+0000", 2024-09-16T09:29:15.072 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:29:15.072 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:15.072 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:15.073 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:29:15.073 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:15.036300+0000", 2024-09-16T09:29:15.073 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:29:15.073 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:15.073 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:15.073 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:29:15.073 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:15.036301+0000", 2024-09-16T09:29:15.073 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.105e-06 2024-09-16T09:29:15.073 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:15.073 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:15.073 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:29:15.073 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:15.036306+0000", 2024-09-16T09:29:15.074 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.8319999999999996e-06 2024-09-16T09:29:15.074 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:15.074 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:15.074 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:29:15.074 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:15.039447+0000", 2024-09-16T09:29:15.074 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0031409519999999998 2024-09-16T09:29:15.074 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:15.074 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:15.074 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:29:15.074 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:15.039462+0000", 2024-09-16T09:29:15.074 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5199e-05 2024-09-16T09:29:15.074 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:15.074 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:15.074 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:29:15.075 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:15.040563+0000", 2024-09-16T09:29:15.075 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0011007390000000001 2024-09-16T09:29:15.075 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:15.075 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:15.075 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:29:15.075 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:15.040743+0000", 2024-09-16T09:29:15.076 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00018021399999999999 2024-09-16T09:29:15.076 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:15.076 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:15.076 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 3,7", 2024-09-16T09:29:15.076 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:15.043048+0000", 2024-09-16T09:29:15.076 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0023041289999999998 2024-09-16T09:29:15.077 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:15.077 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:15.077 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:15.077 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:15.048737+0000", 2024-09-16T09:29:15.077 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0056892189999999997 2024-09-16T09:29:15.077 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:15.078 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:15.078 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:15.078 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:15.052359+0000", 2024-09-16T09:29:15.078 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0036218919999999998 2024-09-16T09:29:15.078 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:29:15.078 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:29:15.079 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:29:15.079 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:15.079 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:15.079 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.24698.0:227833 7.e e142/82 7:733545d2:::benchmark_data_smithi070_78438_object101702:head v 142'14167, mlcod=142'14165)", 2024-09-16T09:29:15.079 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:29:15.052498+0000", 2024-09-16T09:29:15.079 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.0019844160000000001, 2024-09-16T09:29:15.079 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0022382589999999998, 2024-09-16T09:29:15.079 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:29:15.080 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:29:15.080 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T09:29:15.080 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:29:15.080 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:15.080 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:29:15.081 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:15.052498+0000", 2024-09-16T09:29:15.081 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:29:15.081 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:15.081 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:15.081 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:29:15.081 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:15.052498+0000", 2024-09-16T09:29:15.081 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:29:15.081 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:15.082 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:15.082 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:29:15.082 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:15.052499+0000", 2024-09-16T09:29:15.082 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.6099999999999999e-07 2024-09-16T09:29:15.082 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:15.082 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:15.082 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:29:15.082 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:15.052506+0000", 2024-09-16T09:29:15.083 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.644e-06 2024-09-16T09:29:15.083 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:15.084 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:15.084 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:29:15.084 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:15.052521+0000", 2024-09-16T09:29:15.084 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5192e-05 2024-09-16T09:29:15.084 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:15.084 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:15.084 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:29:15.084 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:15.052528+0000", 2024-09-16T09:29:15.084 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.652e-06 2024-09-16T09:29:15.084 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:15.085 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:15.085 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:29:15.085 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:15.052764+0000", 2024-09-16T09:29:15.085 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000236152 2024-09-16T09:29:15.085 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:15.085 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:15.085 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:29:15.085 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:15.052783+0000", 2024-09-16T09:29:15.085 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9190000000000001e-05 2024-09-16T09:29:15.085 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:29:15.085 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:29:15.085 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:29:15.085 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:15.085 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:15.086 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.24698.0:227834 7.5 e142/82)", 2024-09-16T09:29:15.086 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:29:15.054699+0000", 2024-09-16T09:29:15.086 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 4294967295.999783, 2024-09-16T09:29:15.086 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.0853999999999995e-05, 2024-09-16T09:29:15.086 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:29:15.086 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:29:15.086 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T09:29:15.086 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:29:15.086 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:15.086 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:29:15.086 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:15.054699+0000", 2024-09-16T09:29:15.086 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:29:15.086 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:15.086 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:15.087 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:29:15.087 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:15.054699+0000", 2024-09-16T09:29:15.087 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:29:15.087 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:15.087 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:15.087 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:29:15.087 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:15.054700+0000", 2024-09-16T09:29:15.087 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0279999999999999e-06 2024-09-16T09:29:15.087 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:15.087 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:15.087 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:29:15.087 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:15.054707+0000", 2024-09-16T09:29:15.087 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.7989999999999996e-06 2024-09-16T09:29:15.087 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:15.087 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:15.088 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:29:15.088 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:15.054722+0000", 2024-09-16T09:29:15.088 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5037e-05 2024-09-16T09:29:15.088 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:15.088 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:15.088 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:29:15.088 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:15.054729+0000", 2024-09-16T09:29:15.088 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.6919999999999996e-06 2024-09-16T09:29:15.088 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:29:15.088 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:29:15.088 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:29:15.088 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:29:15.088 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-16T09:29:15.088 INFO:teuthology.orchestra.run.smithi070.stdout: "num_ops": 6 2024-09-16T09:29:15.089 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T09:29:15.117 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:29:15.145 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:15.146 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:15.147 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:29:15.175 INFO:teuthology.orchestra.run.smithi070.stderr:noscrub is unset 2024-09-16T09:29:15.247 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:29:15.275 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:15.276 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:15.276 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:29:15.377 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:29:15.408 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:15.409 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:15.409 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:29:15.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:15 smithi070 ceph-mon[30388]: pgmap v183: 158 pgs: 158 active+clean; 1.6 GiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.08k op/s 2024-09-16T09:29:15.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:15 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/4172428644' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:29:15.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:15 smithi070 ceph-mon[40691]: pgmap v183: 158 pgs: 158 active+clean; 1.6 GiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.08k op/s 2024-09-16T09:29:15.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:15 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/4172428644' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:29:15.510 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:29:15.541 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:15.542 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:15.542 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:29:15.643 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:29:15.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:15 smithi084 ceph-mon[32629]: pgmap v183: 158 pgs: 158 active+clean; 1.6 GiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.08k op/s 2024-09-16T09:29:15.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:15 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/4172428644' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:29:15.676 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:15.677 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:15.677 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:29:15.778 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:29:15.818 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:15.818 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:15.819 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:29:15.920 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:29:15.953 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:15.953 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:15.954 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:29:16.055 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:29:16.089 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:16.090 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:16.090 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:29:16.191 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:29:16.223 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:16.223 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:16.223 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:29:16.324 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:29:16.356 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:16.356 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:16.357 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:29:16.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:16 smithi084 ceph-mon[32629]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:29:16.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:16 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/4172428644' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-16T09:29:16.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:16 smithi084 ceph-mon[32629]: osdmap e143: 8 total, 8 up, 8 in 2024-09-16T09:29:16.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:16 smithi084 ceph-mon[32629]: 1.1 scrub starts 2024-09-16T09:29:16.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:16 smithi084 ceph-mon[32629]: 1.1 scrub ok 2024-09-16T09:29:16.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:16 smithi084 ceph-mon[32629]: 5.18 scrub starts 2024-09-16T09:29:16.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:16 smithi084 ceph-mon[32629]: 5.18 scrub ok 2024-09-16T09:29:16.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:16 smithi084 ceph-mon[32629]: 5.12 scrub starts 2024-09-16T09:29:16.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:16 smithi084 ceph-mon[32629]: 5.12 scrub ok 2024-09-16T09:29:16.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:16 smithi070 ceph-mon[30388]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:29:16.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:16 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/4172428644' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-16T09:29:16.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:16 smithi070 ceph-mon[30388]: osdmap e143: 8 total, 8 up, 8 in 2024-09-16T09:29:16.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:16 smithi070 ceph-mon[30388]: 1.1 scrub starts 2024-09-16T09:29:16.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:16 smithi070 ceph-mon[30388]: 1.1 scrub ok 2024-09-16T09:29:16.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:16 smithi070 ceph-mon[30388]: 5.18 scrub starts 2024-09-16T09:29:16.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:16 smithi070 ceph-mon[30388]: 5.18 scrub ok 2024-09-16T09:29:16.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:16 smithi070 ceph-mon[30388]: 5.12 scrub starts 2024-09-16T09:29:16.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:16 smithi070 ceph-mon[30388]: 5.12 scrub ok 2024-09-16T09:29:16.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:16 smithi070 ceph-mon[40691]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:29:16.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:16 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/4172428644' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-16T09:29:16.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:16 smithi070 ceph-mon[40691]: osdmap e143: 8 total, 8 up, 8 in 2024-09-16T09:29:16.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:16 smithi070 ceph-mon[40691]: 1.1 scrub starts 2024-09-16T09:29:16.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:16 smithi070 ceph-mon[40691]: 1.1 scrub ok 2024-09-16T09:29:16.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:16 smithi070 ceph-mon[40691]: 5.18 scrub starts 2024-09-16T09:29:16.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:16 smithi070 ceph-mon[40691]: 5.18 scrub ok 2024-09-16T09:29:16.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:16 smithi070 ceph-mon[40691]: 5.12 scrub starts 2024-09-16T09:29:16.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:16 smithi070 ceph-mon[40691]: 5.12 scrub ok 2024-09-16T09:29:16.457 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:29:16.488 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:16.489 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:16.489 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:29:16.590 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:29:16.625 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:16.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:16.626 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:29:16.727 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:29:16.767 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:16.767 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:16.768 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:29:16.868 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:29:16.896 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:16.897 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:16.897 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:29:16.998 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:29:17.034 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:17.035 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:17.035 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:29:17.136 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:29:17.172 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:17.172 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:17.172 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:29:17.273 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:29:17.309 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:17.310 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:17.310 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:29:17.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:17 smithi084 ceph-mon[32629]: pgmap v185: 158 pgs: 158 active+clean; 1.6 GiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.09k op/s 2024-09-16T09:29:17.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:17 smithi084 ceph-mon[32629]: 6.18 scrub starts 2024-09-16T09:29:17.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:17 smithi084 ceph-mon[32629]: 6.18 scrub ok 2024-09-16T09:29:17.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:17 smithi084 ceph-mon[32629]: 2.1 scrub starts 2024-09-16T09:29:17.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:17 smithi084 ceph-mon[32629]: 2.1 scrub ok 2024-09-16T09:29:17.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:17 smithi084 ceph-mon[32629]: 3.1e scrub starts 2024-09-16T09:29:17.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:17 smithi084 ceph-mon[32629]: 3.1e scrub ok 2024-09-16T09:29:17.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:17 smithi084 ceph-mon[32629]: 3.4 scrub starts 2024-09-16T09:29:17.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:17 smithi084 ceph-mon[32629]: 3.4 scrub ok 2024-09-16T09:29:17.411 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:29:17.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:17 smithi070 ceph-mon[30388]: pgmap v185: 158 pgs: 158 active+clean; 1.6 GiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.09k op/s 2024-09-16T09:29:17.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:17 smithi070 ceph-mon[30388]: 6.18 scrub starts 2024-09-16T09:29:17.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:17 smithi070 ceph-mon[30388]: 6.18 scrub ok 2024-09-16T09:29:17.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:17 smithi070 ceph-mon[30388]: 2.1 scrub starts 2024-09-16T09:29:17.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:17 smithi070 ceph-mon[30388]: 2.1 scrub ok 2024-09-16T09:29:17.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:17 smithi070 ceph-mon[30388]: 3.1e scrub starts 2024-09-16T09:29:17.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:17 smithi070 ceph-mon[30388]: 3.1e scrub ok 2024-09-16T09:29:17.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:17 smithi070 ceph-mon[30388]: 3.4 scrub starts 2024-09-16T09:29:17.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:17 smithi070 ceph-mon[30388]: 3.4 scrub ok 2024-09-16T09:29:17.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:17 smithi070 ceph-mon[40691]: pgmap v185: 158 pgs: 158 active+clean; 1.6 GiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.09k op/s 2024-09-16T09:29:17.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:17 smithi070 ceph-mon[40691]: 6.18 scrub starts 2024-09-16T09:29:17.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:17 smithi070 ceph-mon[40691]: 6.18 scrub ok 2024-09-16T09:29:17.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:17 smithi070 ceph-mon[40691]: 2.1 scrub starts 2024-09-16T09:29:17.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:17 smithi070 ceph-mon[40691]: 2.1 scrub ok 2024-09-16T09:29:17.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:17 smithi070 ceph-mon[40691]: 3.1e scrub starts 2024-09-16T09:29:17.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:17 smithi070 ceph-mon[40691]: 3.1e scrub ok 2024-09-16T09:29:17.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:17 smithi070 ceph-mon[40691]: 3.4 scrub starts 2024-09-16T09:29:17.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:17 smithi070 ceph-mon[40691]: 3.4 scrub ok 2024-09-16T09:29:17.459 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:17.459 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:17.460 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:29:17.561 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:29:17.605 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:17.606 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:17.606 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:29:17.707 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:29:17.736 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:17.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:17.737 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:29:17.838 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:29:17.865 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:17.866 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:17.866 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:29:17.967 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:29:17.993 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:17.994 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:17.994 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:29:18.011 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:29:18.095 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:29:18.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:18.146 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:18.146 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:29:18.247 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:29:18.272 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:18.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:18.273 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:29:18.374 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:29:18.408 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:18.409 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:18.409 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:29:18.512 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:29:18.514 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.0 dump_blocked_ops 2024-09-16T09:29:18.542 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:18.543 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:18.543 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:29:18.644 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:29:18.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:18 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:29:18.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:18 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:29:18.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:18 smithi084 ceph-mon[32629]: 1.8 scrub starts 2024-09-16T09:29:18.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:18 smithi084 ceph-mon[32629]: 1.8 scrub ok 2024-09-16T09:29:18.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:18 smithi084 ceph-mon[32629]: 1.7 scrub starts 2024-09-16T09:29:18.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:18 smithi084 ceph-mon[32629]: 1.7 scrub ok 2024-09-16T09:29:18.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:18 smithi084 ceph-mon[32629]: 5.19 scrub starts 2024-09-16T09:29:18.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:18 smithi084 ceph-mon[32629]: 5.19 scrub ok 2024-09-16T09:29:18.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:18 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:29:18.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:18 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:29:18.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:18 smithi070 ceph-mon[30388]: 1.8 scrub starts 2024-09-16T09:29:18.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:18 smithi070 ceph-mon[30388]: 1.8 scrub ok 2024-09-16T09:29:18.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:18 smithi070 ceph-mon[30388]: 1.7 scrub starts 2024-09-16T09:29:18.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:18 smithi070 ceph-mon[30388]: 1.7 scrub ok 2024-09-16T09:29:18.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:18 smithi070 ceph-mon[30388]: 5.19 scrub starts 2024-09-16T09:29:18.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:18 smithi070 ceph-mon[30388]: 5.19 scrub ok 2024-09-16T09:29:18.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:18 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:29:18.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:18 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:29:18.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:18 smithi070 ceph-mon[40691]: 1.8 scrub starts 2024-09-16T09:29:18.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:18 smithi070 ceph-mon[40691]: 1.8 scrub ok 2024-09-16T09:29:18.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:18 smithi070 ceph-mon[40691]: 1.7 scrub starts 2024-09-16T09:29:18.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:18 smithi070 ceph-mon[40691]: 1.7 scrub ok 2024-09-16T09:29:18.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:18 smithi070 ceph-mon[40691]: 5.19 scrub starts 2024-09-16T09:29:18.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:18 smithi070 ceph-mon[40691]: 5.19 scrub ok 2024-09-16T09:29:18.694 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:18.694 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:18.694 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:29:18.795 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:29:18.835 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:18.836 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:18.836 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:29:18.937 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:29:18.968 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:18.969 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:18.969 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:29:19.070 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:29:19.109 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:19.110 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:19.110 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:29:19.159 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:29:19.211 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:29:19.237 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:19.239 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:19.239 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:29:19.340 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:29:19.365 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:19.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:19.366 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:29:19.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:19 smithi070 ceph-mon[30388]: pgmap v186: 158 pgs: 158 active+clean; 1.5 GiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 855 op/s 2024-09-16T09:29:19.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:19 smithi070 ceph-mon[30388]: 1.a scrub starts 2024-09-16T09:29:19.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:19 smithi070 ceph-mon[30388]: 1.a scrub ok 2024-09-16T09:29:19.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:19 smithi070 ceph-mon[30388]: 1.9 scrub starts 2024-09-16T09:29:19.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:19 smithi070 ceph-mon[30388]: 1.9 scrub ok 2024-09-16T09:29:19.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:19 smithi070 ceph-mon[40691]: pgmap v186: 158 pgs: 158 active+clean; 1.5 GiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 855 op/s 2024-09-16T09:29:19.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:19 smithi070 ceph-mon[40691]: 1.a scrub starts 2024-09-16T09:29:19.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:19 smithi070 ceph-mon[40691]: 1.a scrub ok 2024-09-16T09:29:19.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:19 smithi070 ceph-mon[40691]: 1.9 scrub starts 2024-09-16T09:29:19.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:19 smithi070 ceph-mon[40691]: 1.9 scrub ok 2024-09-16T09:29:19.467 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:29:19.492 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:19.493 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:19.494 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:29:19.559 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:29:19.595 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:29:19.636 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:19.637 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:19.637 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:29:19.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:19 smithi084 ceph-mon[32629]: pgmap v186: 158 pgs: 158 active+clean; 1.5 GiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 855 op/s 2024-09-16T09:29:19.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:19 smithi084 ceph-mon[32629]: 1.a scrub starts 2024-09-16T09:29:19.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:19 smithi084 ceph-mon[32629]: 1.a scrub ok 2024-09-16T09:29:19.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:19 smithi084 ceph-mon[32629]: 1.9 scrub starts 2024-09-16T09:29:19.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:19 smithi084 ceph-mon[32629]: 1.9 scrub ok 2024-09-16T09:29:19.738 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:29:19.786 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:19.787 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:19.787 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:29:19.848 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:29:19.849 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:29:19.884 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:29:19.884 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:29:19.888 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:29:19.913 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:19.914 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:19.914 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:29:19.920 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:29:19.920 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:29:19.952 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:29:19.952 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:29:19.979 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-16T09:29:19.983 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:29:19.983 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:29:20.015 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:29:20.019 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:29:20.020 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:29:20.040 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:20.040 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:20.041 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:29:20.052 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:29:20.052 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:29:20.085 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:29:20.085 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:29:20.142 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:29:20.165 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:20.166 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:20.166 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:29:20.267 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:29:20.309 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:20.310 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:20.310 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:29:20.409 INFO:teuthology.orchestra.run.smithi070.stderr:set pool 7 pgp_num to 16 2024-09-16T09:29:20.411 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:29:20.436 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:20.436 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:20.437 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:29:20.537 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:29:20.564 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:20.565 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:20.565 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:29:20.615 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:20 smithi070 ceph-mon[30388]: 4.4 scrub starts 2024-09-16T09:29:20.615 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:20 smithi070 ceph-mon[30388]: 4.4 scrub ok 2024-09-16T09:29:20.615 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:20 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1005069721' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_0", "var": "pgp_num", "val": "16"} : dispatch 2024-09-16T09:29:20.615 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:20 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_0", "var": "pgp_num", "val": "16"} : dispatch 2024-09-16T09:29:20.616 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:20 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2449500604' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:29:20.616 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:20 smithi070 ceph-mon[30388]: 6.1d scrub starts 2024-09-16T09:29:20.616 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:20 smithi070 ceph-mon[30388]: 6.1d scrub ok 2024-09-16T09:29:20.616 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:20 smithi070 ceph-mon[30388]: 2.0 scrub starts 2024-09-16T09:29:20.616 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:20 smithi070 ceph-mon[30388]: 2.0 scrub ok 2024-09-16T09:29:20.616 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:20 smithi070 ceph-mon[40691]: 4.4 scrub starts 2024-09-16T09:29:20.616 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:20 smithi070 ceph-mon[40691]: 4.4 scrub ok 2024-09-16T09:29:20.616 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:20 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1005069721' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_0", "var": "pgp_num", "val": "16"} : dispatch 2024-09-16T09:29:20.616 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:20 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_0", "var": "pgp_num", "val": "16"} : dispatch 2024-09-16T09:29:20.616 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:20 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/2449500604' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:29:20.617 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:20 smithi070 ceph-mon[40691]: 6.1d scrub starts 2024-09-16T09:29:20.617 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:20 smithi070 ceph-mon[40691]: 6.1d scrub ok 2024-09-16T09:29:20.617 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:20 smithi070 ceph-mon[40691]: 2.0 scrub starts 2024-09-16T09:29:20.617 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:20 smithi070 ceph-mon[40691]: 2.0 scrub ok 2024-09-16T09:29:20.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:20 smithi084 ceph-mon[32629]: 4.4 scrub starts 2024-09-16T09:29:20.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:20 smithi084 ceph-mon[32629]: 4.4 scrub ok 2024-09-16T09:29:20.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:20 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1005069721' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_0", "var": "pgp_num", "val": "16"} : dispatch 2024-09-16T09:29:20.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:20 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_0", "var": "pgp_num", "val": "16"} : dispatch 2024-09-16T09:29:20.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:20 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/2449500604' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:29:20.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:20 smithi084 ceph-mon[32629]: 6.1d scrub starts 2024-09-16T09:29:20.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:20 smithi084 ceph-mon[32629]: 6.1d scrub ok 2024-09-16T09:29:20.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:20 smithi084 ceph-mon[32629]: 2.0 scrub starts 2024-09-16T09:29:20.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:20 smithi084 ceph-mon[32629]: 2.0 scrub ok 2024-09-16T09:29:20.666 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:29:20.706 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:20.706 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:20.707 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:29:20.789 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T09:29:20.790 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:29:20.808 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:29:20.837 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:20.838 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:20.838 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:29:20.939 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:29:20.981 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:20.982 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:20.983 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:29:21.083 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:29:21.110 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:21.111 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:21.112 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:29:21.212 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:29:21.247 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:21.248 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:21.248 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:29:21.349 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:29:21.374 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:21.375 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:21.375 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:29:21.476 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:29:21.511 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:21.512 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:21.512 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:29:21.613 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:29:21.639 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:21.640 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:21.640 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:29:21.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:21 smithi084 ceph-mon[32629]: pgmap v187: 158 pgs: 158 active+clean; 1.3 GiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.07k op/s 2024-09-16T09:29:21.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:21 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_0", "var": "pgp_num", "val": "16"}]': finished 2024-09-16T09:29:21.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:21 smithi084 ceph-mon[32629]: osdmap e144: 8 total, 8 up, 8 in 2024-09-16T09:29:21.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:21 smithi070 ceph-mon[30388]: pgmap v187: 158 pgs: 158 active+clean; 1.3 GiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.07k op/s 2024-09-16T09:29:21.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:21 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_0", "var": "pgp_num", "val": "16"}]': finished 2024-09-16T09:29:21.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:21 smithi070 ceph-mon[30388]: osdmap e144: 8 total, 8 up, 8 in 2024-09-16T09:29:21.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:21 smithi070 ceph-mon[40691]: pgmap v187: 158 pgs: 158 active+clean; 1.3 GiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.07k op/s 2024-09-16T09:29:21.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:21 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_0", "var": "pgp_num", "val": "16"}]': finished 2024-09-16T09:29:21.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:21 smithi070 ceph-mon[40691]: osdmap e144: 8 total, 8 up, 8 in 2024-09-16T09:29:21.741 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:29:21.746 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.0 dump_historic_ops 2024-09-16T09:29:21.763 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:29:21.764 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:21.765 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:21.765 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:29:21.866 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:29:21.934 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:21.934 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:21.935 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:29:22.035 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:29:22.074 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:22.075 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:22.075 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:29:22.171 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:29:21 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:29:21] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:29:22.176 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:29:22.214 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:22.215 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:22.215 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:29:22.316 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:29:22.356 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:22.356 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:22.357 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:29:22.458 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:29:22.484 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:22.485 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:22.485 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:29:22.586 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:29:22.615 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:22.615 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:22.615 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:29:22.641 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:22 smithi070 ceph-mon[30388]: pgmap v189: 158 pgs: 158 active+clean; 1.3 GiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 674 op/s 2024-09-16T09:29:22.641 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:22 smithi070 ceph-mon[30388]: 5.0 scrub starts 2024-09-16T09:29:22.641 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:22 smithi070 ceph-mon[30388]: 5.0 scrub ok 2024-09-16T09:29:22.641 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:22 smithi070 ceph-mon[30388]: 7.5 scrub starts 2024-09-16T09:29:22.642 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:22 smithi070 ceph-mon[30388]: 7.e scrub starts 2024-09-16T09:29:22.642 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:22 smithi070 ceph-mon[40691]: pgmap v189: 158 pgs: 158 active+clean; 1.3 GiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 674 op/s 2024-09-16T09:29:22.642 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:22 smithi070 ceph-mon[40691]: 5.0 scrub starts 2024-09-16T09:29:22.642 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:22 smithi070 ceph-mon[40691]: 5.0 scrub ok 2024-09-16T09:29:22.642 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:22 smithi070 ceph-mon[40691]: 7.5 scrub starts 2024-09-16T09:29:22.642 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:22 smithi070 ceph-mon[40691]: 7.e scrub starts 2024-09-16T09:29:22.716 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:29:22.741 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:22 smithi084 ceph-mon[32629]: pgmap v189: 158 pgs: 158 active+clean; 1.3 GiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 674 op/s 2024-09-16T09:29:22.741 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:22 smithi084 ceph-mon[32629]: 5.0 scrub starts 2024-09-16T09:29:22.741 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:22 smithi084 ceph-mon[32629]: 5.0 scrub ok 2024-09-16T09:29:22.741 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:22 smithi084 ceph-mon[32629]: 7.5 scrub starts 2024-09-16T09:29:22.741 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:22 smithi084 ceph-mon[32629]: 7.e scrub starts 2024-09-16T09:29:22.748 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:22.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:22.749 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:29:22.850 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:29:22.891 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:22.892 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:22.892 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:29:22.993 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:29:23.028 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:23.029 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:23.029 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:29:23.130 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:29:23.158 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:23.159 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:23.159 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:29:23.160 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:29:22 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:29:23.260 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:29:23.290 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:23.290 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:23.290 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:29:23.391 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:29:23.424 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:23.425 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:23.426 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:29:23.526 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:29:23.562 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:23.563 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:23.563 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:29:23.664 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:29:23.707 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:23.708 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:23.708 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:29:23.719 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:23 smithi070 ceph-mon[30388]: 7.3 scrub starts 2024-09-16T09:29:23.719 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:23 smithi070 ceph-mon[30388]: 7.5 scrub ok 2024-09-16T09:29:23.719 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:23 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:29:23.719 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:23 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:29:23.720 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:23 smithi070 ceph-mon[30388]: 7.e scrub ok 2024-09-16T09:29:23.720 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:23 smithi070 ceph-mon[30388]: 7.3 scrub ok 2024-09-16T09:29:23.720 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:23 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:29:23.720 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:23 smithi070 ceph-mon[30388]: 6.5 scrub starts 2024-09-16T09:29:23.720 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:23 smithi070 ceph-mon[30388]: 6.5 scrub ok 2024-09-16T09:29:23.720 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:23 smithi070 ceph-mon[30388]: 7.8 scrub starts 2024-09-16T09:29:23.720 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:23 smithi070 ceph-mon[40691]: 7.3 scrub starts 2024-09-16T09:29:23.720 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:23 smithi070 ceph-mon[40691]: 7.5 scrub ok 2024-09-16T09:29:23.720 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:23 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:29:23.720 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:23 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:29:23.720 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:23 smithi070 ceph-mon[40691]: 7.e scrub ok 2024-09-16T09:29:23.721 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:23 smithi070 ceph-mon[40691]: 7.3 scrub ok 2024-09-16T09:29:23.721 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:23 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:29:23.721 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:23 smithi070 ceph-mon[40691]: 6.5 scrub starts 2024-09-16T09:29:23.721 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:23 smithi070 ceph-mon[40691]: 6.5 scrub ok 2024-09-16T09:29:23.721 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:23 smithi070 ceph-mon[40691]: 7.8 scrub starts 2024-09-16T09:29:23.809 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:29:23.847 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:23.847 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:23.847 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:29:23.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:23 smithi084 ceph-mon[32629]: 7.3 scrub starts 2024-09-16T09:29:23.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:23 smithi084 ceph-mon[32629]: 7.5 scrub ok 2024-09-16T09:29:23.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:23 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:29:23.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:23 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:29:23.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:23 smithi084 ceph-mon[32629]: 7.e scrub ok 2024-09-16T09:29:23.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:23 smithi084 ceph-mon[32629]: 7.3 scrub ok 2024-09-16T09:29:23.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:23 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:29:23.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:23 smithi084 ceph-mon[32629]: 6.5 scrub starts 2024-09-16T09:29:23.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:23 smithi084 ceph-mon[32629]: 6.5 scrub ok 2024-09-16T09:29:23.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:23 smithi084 ceph-mon[32629]: 7.8 scrub starts 2024-09-16T09:29:23.948 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:29:23.974 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:23.974 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:23.975 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:29:24.076 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:29:24.123 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:24.124 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:24.124 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:29:24.225 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:29:24.262 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:24.263 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:24.263 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:29:24.319 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:29:24.364 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:29:24.390 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:24.390 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:24.391 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:29:24.491 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:29:24.515 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:24.517 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:24.517 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:29:24.618 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:29:24.636 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:24 smithi070 ceph-mon[30388]: 7.8 scrub ok 2024-09-16T09:29:24.636 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:24 smithi070 ceph-mon[30388]: osdmap e145: 8 total, 8 up, 8 in 2024-09-16T09:29:24.636 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:24 smithi070 ceph-mon[30388]: pgmap v191: 158 pgs: 158 active+clean; 998 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.36k op/s 2024-09-16T09:29:24.636 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:24 smithi070 ceph-mon[30388]: 7.6 scrub starts 2024-09-16T09:29:24.636 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:24 smithi070 ceph-mon[30388]: 7.6 scrub ok 2024-09-16T09:29:24.637 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:24 smithi070 ceph-mon[40691]: 7.8 scrub ok 2024-09-16T09:29:24.637 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:24 smithi070 ceph-mon[40691]: osdmap e145: 8 total, 8 up, 8 in 2024-09-16T09:29:24.637 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:24 smithi070 ceph-mon[40691]: pgmap v191: 158 pgs: 158 active+clean; 998 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.36k op/s 2024-09-16T09:29:24.637 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:24 smithi070 ceph-mon[40691]: 7.6 scrub starts 2024-09-16T09:29:24.637 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:24 smithi070 ceph-mon[40691]: 7.6 scrub ok 2024-09-16T09:29:24.643 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:24.644 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:24.645 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:29:24.746 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:29:24.774 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:24.775 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:24.775 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:29:24.876 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:29:24.909 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:24 smithi084 ceph-mon[32629]: 7.8 scrub ok 2024-09-16T09:29:24.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:24 smithi084 ceph-mon[32629]: osdmap e145: 8 total, 8 up, 8 in 2024-09-16T09:29:24.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:24 smithi084 ceph-mon[32629]: pgmap v191: 158 pgs: 158 active+clean; 998 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.36k op/s 2024-09-16T09:29:24.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:24 smithi084 ceph-mon[32629]: 7.6 scrub starts 2024-09-16T09:29:24.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:24 smithi084 ceph-mon[32629]: 7.6 scrub ok 2024-09-16T09:29:24.921 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:24.921 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:24.922 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:29:24.970 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T09:29:24.970 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:29:25.023 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:29:25.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:25.058 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:25.058 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:29:25.159 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:29:25.188 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:25.189 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:25.189 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:29:25.290 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:29:25.327 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:25.328 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:25.328 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:29:25.429 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:29:25.456 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:25.457 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:25.457 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:29:25.467 INFO:teuthology.orchestra.run.smithi070.stderr:nodeep-scrub is unset 2024-09-16T09:29:25.558 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:29:25.593 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:25.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:25.594 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:29:25.695 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:29:25.722 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T09:29:25.732 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:25.732 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:25.732 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:29:25.772 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:25 smithi070 ceph-mon[30388]: 7.d scrub starts 2024-09-16T09:29:25.773 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:25 smithi070 ceph-mon[30388]: 6.d scrub starts 2024-09-16T09:29:25.773 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:25 smithi070 ceph-mon[30388]: 6.d scrub ok 2024-09-16T09:29:25.773 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:25 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/368904597' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:29:25.773 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:25 smithi070 ceph-mon[30388]: 7.d scrub ok 2024-09-16T09:29:25.773 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:25 smithi070 ceph-mon[30388]: 5.10 scrub starts 2024-09-16T09:29:25.773 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:25 smithi070 ceph-mon[30388]: 5.10 scrub ok 2024-09-16T09:29:25.773 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:25 smithi070 ceph-mon[40691]: 7.d scrub starts 2024-09-16T09:29:25.773 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:25 smithi070 ceph-mon[40691]: 6.d scrub starts 2024-09-16T09:29:25.773 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:25 smithi070 ceph-mon[40691]: 6.d scrub ok 2024-09-16T09:29:25.773 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:25 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/368904597' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:29:25.773 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:25 smithi070 ceph-mon[40691]: 7.d scrub ok 2024-09-16T09:29:25.774 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:25 smithi070 ceph-mon[40691]: 5.10 scrub starts 2024-09-16T09:29:25.774 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:25 smithi070 ceph-mon[40691]: 5.10 scrub ok 2024-09-16T09:29:25.833 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:29:25.860 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:25.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:25.861 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:29:25.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:25 smithi084 ceph-mon[32629]: 7.d scrub starts 2024-09-16T09:29:25.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:25 smithi084 ceph-mon[32629]: 6.d scrub starts 2024-09-16T09:29:25.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:25 smithi084 ceph-mon[32629]: 6.d scrub ok 2024-09-16T09:29:25.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:25 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/368904597' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:29:25.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:25 smithi084 ceph-mon[32629]: 7.d scrub ok 2024-09-16T09:29:25.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:25 smithi084 ceph-mon[32629]: 5.10 scrub starts 2024-09-16T09:29:25.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:25 smithi084 ceph-mon[32629]: 5.10 scrub ok 2024-09-16T09:29:25.962 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:29:25.988 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:25.989 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:25.989 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:29:26.090 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:29:26.116 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:26.116 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:26.117 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:29:26.217 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:29:26.270 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:26.271 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:26.272 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:29:26.286 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.1 dump_ops_in_flight 2024-09-16T09:29:26.373 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:29:26.416 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:26.417 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:26.417 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:29:26.518 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:29:26.564 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:26.565 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:26.565 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:29:26.666 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:29:26.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:26 smithi070 ceph-mon[30388]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-16T09:29:26.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:26 smithi070 ceph-mon[30388]: Cluster is now healthy 2024-09-16T09:29:26.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:26 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/368904597' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-16T09:29:26.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:26 smithi070 ceph-mon[30388]: osdmap e146: 8 total, 8 up, 8 in 2024-09-16T09:29:26.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:26 smithi070 ceph-mon[30388]: 7.0 scrub starts 2024-09-16T09:29:26.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:26 smithi070 ceph-mon[30388]: pgmap v193: 158 pgs: 158 active+clean; 998 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 917 op/s 2024-09-16T09:29:26.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:26 smithi070 ceph-mon[40691]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-16T09:29:26.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:26 smithi070 ceph-mon[40691]: Cluster is now healthy 2024-09-16T09:29:26.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:26 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/368904597' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-16T09:29:26.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:26 smithi070 ceph-mon[40691]: osdmap e146: 8 total, 8 up, 8 in 2024-09-16T09:29:26.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:26 smithi070 ceph-mon[40691]: 7.0 scrub starts 2024-09-16T09:29:26.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:26 smithi070 ceph-mon[40691]: pgmap v193: 158 pgs: 158 active+clean; 998 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 917 op/s 2024-09-16T09:29:26.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:26.710 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:26.710 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:29:26.791 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:29:26.811 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:29:26.847 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:29:26.856 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:26.857 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:26.857 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:29:26.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:26 smithi084 ceph-mon[32629]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-16T09:29:26.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:26 smithi084 ceph-mon[32629]: Cluster is now healthy 2024-09-16T09:29:26.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:26 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/368904597' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-16T09:29:26.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:26 smithi084 ceph-mon[32629]: osdmap e146: 8 total, 8 up, 8 in 2024-09-16T09:29:26.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:26 smithi084 ceph-mon[32629]: 7.0 scrub starts 2024-09-16T09:29:26.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:26 smithi084 ceph-mon[32629]: pgmap v193: 158 pgs: 158 active+clean; 998 MiB data, 4.0 GiB used, 711 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 917 op/s 2024-09-16T09:29:26.958 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:29:27.001 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:27.001 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:27.002 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:29:27.103 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:29:27.136 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:27.137 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:27.137 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:29:27.238 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:29:27.275 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:27.275 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:27.276 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:29:27.377 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:29:27.406 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:27.407 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:27.407 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:29:27.508 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:29:27.544 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:27.544 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:27.544 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:29:27.645 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:29:27.674 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:27.674 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:27.675 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:29:27.685 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T09:29:27.685 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T09:29:27.685 INFO:tasks.thrashosds.thrasher:inject_pause on osd.1 2024-09-16T09:29:27.686 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-16T09:29:27.686 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-16T09:29:27.686 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.1 config set filestore_inject_stall 3 2024-09-16T09:29:27.775 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:29:27.800 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:27.801 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:27.801 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:29:27.902 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:29:27.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:27 smithi084 ceph-mon[32629]: 7.0 scrub ok 2024-09-16T09:29:27.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:27 smithi084 ceph-mon[32629]: 3.3 scrub starts 2024-09-16T09:29:27.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:27 smithi084 ceph-mon[32629]: 3.3 scrub ok 2024-09-16T09:29:27.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:27 smithi084 ceph-mon[32629]: 6.12 scrub starts 2024-09-16T09:29:27.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:27 smithi084 ceph-mon[32629]: 6.12 scrub ok 2024-09-16T09:29:27.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:27 smithi070 ceph-mon[30388]: 7.0 scrub ok 2024-09-16T09:29:27.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:27 smithi070 ceph-mon[30388]: 3.3 scrub starts 2024-09-16T09:29:27.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:27 smithi070 ceph-mon[30388]: 3.3 scrub ok 2024-09-16T09:29:27.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:27 smithi070 ceph-mon[30388]: 6.12 scrub starts 2024-09-16T09:29:27.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:27 smithi070 ceph-mon[30388]: 6.12 scrub ok 2024-09-16T09:29:27.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:27 smithi070 ceph-mon[40691]: 7.0 scrub ok 2024-09-16T09:29:27.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:27 smithi070 ceph-mon[40691]: 3.3 scrub starts 2024-09-16T09:29:27.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:27 smithi070 ceph-mon[40691]: 3.3 scrub ok 2024-09-16T09:29:27.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:27 smithi070 ceph-mon[40691]: 6.12 scrub starts 2024-09-16T09:29:27.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:27 smithi070 ceph-mon[40691]: 6.12 scrub ok 2024-09-16T09:29:27.923 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:27.924 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:27.924 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:29:28.025 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:29:28.053 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:28.054 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:28.055 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:29:28.155 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:29:28.213 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:28.214 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:28.214 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:29:28.315 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:29:28.356 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:28.357 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:28.357 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:29:28.458 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:29:28.501 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:28.502 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:28.502 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:29:28.603 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:29:28.628 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:28.629 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:28.629 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:29:28.643 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:28 smithi070 ceph-mon[30388]: 7.7 scrub starts 2024-09-16T09:29:28.643 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:28 smithi070 ceph-mon[30388]: pgmap v194: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 871 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1.05k op/s 2024-09-16T09:29:28.643 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:28 smithi070 ceph-mon[30388]: 7.7 scrub ok 2024-09-16T09:29:28.643 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:28 smithi070 ceph-mon[40691]: 7.7 scrub starts 2024-09-16T09:29:28.644 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:28 smithi070 ceph-mon[40691]: pgmap v194: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 871 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1.05k op/s 2024-09-16T09:29:28.644 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:28 smithi070 ceph-mon[40691]: 7.7 scrub ok 2024-09-16T09:29:28.730 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:29:28.742 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:29:28.742 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:29:28.747 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:29:28.747 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:29:28.767 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd set noscrub 2024-09-16T09:29:28.768 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:28.770 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:28.771 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:29:28.801 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:29:28.802 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:29:28.851 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T09:29:28.852 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [ 2024-09-16T09:29:28.852 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:28.852 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.24698.0:242888 7.b 7:d7e6709d:::benchmark_data_smithi070_78438_object116757:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e146)", 2024-09-16T09:29:28.852 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:29:28.799619+0000", 2024-09-16T09:29:28.852 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.016403299, 2024-09-16T09:29:28.852 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.016433891999999999, 2024-09-16T09:29:28.852 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:29:28.852 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:29:28.852 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "waiting for sub ops", 2024-09-16T09:29:28.852 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:29:28.852 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.24698", 2024-09-16T09:29:28.852 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:29:28.852 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 242888 2024-09-16T09:29:28.853 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:28.853 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:29:28.853 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:28.853 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:29:28.853 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:28.799619+0000", 2024-09-16T09:29:28.853 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:29:28.853 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:28.853 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:28.853 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:29:28.853 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:28.799619+0000", 2024-09-16T09:29:28.853 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:29:28.853 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:28.853 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:28.853 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:29:28.854 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:28.799620+0000", 2024-09-16T09:29:28.854 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6640000000000001e-06 2024-09-16T09:29:28.854 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:28.854 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:28.854 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:29:28.854 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:28.799626+0000", 2024-09-16T09:29:28.854 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.3090000000000003e-06 2024-09-16T09:29:28.854 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:28.854 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:28.854 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:29:28.854 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:28.799637+0000", 2024-09-16T09:29:28.854 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0829000000000001e-05 2024-09-16T09:29:28.854 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:28.854 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:28.854 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:29:28.855 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:28.799644+0000", 2024-09-16T09:29:28.855 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.4089999999999997e-06 2024-09-16T09:29:28.855 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:28.855 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:28.855 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:29:28.855 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:28.808006+0000", 2024-09-16T09:29:28.855 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0083623510000000005 2024-09-16T09:29:28.855 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:28.855 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:28.855 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:29:28.855 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:28.808256+0000", 2024-09-16T09:29:28.855 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00024932300000000003 2024-09-16T09:29:28.855 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:28.856 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:28.856 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 5,6", 2024-09-16T09:29:28.856 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:28.808622+0000", 2024-09-16T09:29:28.856 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00036619700000000002 2024-09-16T09:29:28.856 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:28.856 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:28.856 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:29:28.856 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:28.812774+0000", 2024-09-16T09:29:28.856 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.004151811 2024-09-16T09:29:28.856 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:28.856 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:28.856 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:28.856 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:28.815829+0000", 2024-09-16T09:29:28.857 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0030554919999999999 2024-09-16T09:29:28.857 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:28.857 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:28.857 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:28.857 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:28.816025+0000", 2024-09-16T09:29:28.857 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000195582 2024-09-16T09:29:28.857 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:29:28.857 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:29:28.857 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:29:28.857 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:28.857 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:28.857 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.24698.0:242899 7.9 e146/82 7:934dbf6b:::benchmark_data_smithi070_78438_object116768:head v 146'15025, mlcod=146'15023)", 2024-09-16T09:29:28.857 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:29:28.813257+0000", 2024-09-16T09:29:28.857 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.0027651199999999998, 2024-09-16T09:29:28.858 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0028792430000000001, 2024-09-16T09:29:28.858 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:29:28.858 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:29:28.858 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T09:29:28.858 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:29:28.858 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:28.858 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:29:28.858 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:28.813257+0000", 2024-09-16T09:29:28.858 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:29:28.858 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:28.858 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:28.858 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:29:28.858 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:28.813257+0000", 2024-09-16T09:29:28.859 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:29:28.859 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:28.859 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:28.859 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:29:28.859 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:28.813258+0000", 2024-09-16T09:29:28.859 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0279999999999999e-06 2024-09-16T09:29:28.859 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:28.859 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:28.859 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:29:28.859 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:28.813265+0000", 2024-09-16T09:29:28.859 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.9630000000000001e-06 2024-09-16T09:29:28.859 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:28.859 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:28.859 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:29:28.859 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:28.813279+0000", 2024-09-16T09:29:28.860 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.4066000000000001e-05 2024-09-16T09:29:28.860 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:28.860 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:28.860 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:29:28.860 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:28.813286+0000", 2024-09-16T09:29:28.860 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.5400000000000001e-06 2024-09-16T09:29:28.860 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:28.860 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:28.860 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:29:28.860 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:28.813383+0000", 2024-09-16T09:29:28.860 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.7335000000000002e-05 2024-09-16T09:29:28.860 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:28.860 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:28.861 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:29:28.861 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:28.813404+0000", 2024-09-16T09:29:28.861 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0754000000000002e-05 2024-09-16T09:29:28.861 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:29:28.861 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:29:28.861 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:29:28.861 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:28.861 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:28.861 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.24698.0:242896 7.4 e146/82 7:23e56b13:::benchmark_data_smithi070_78438_object116765:head v 146'15012, mlcod=146'15010)", 2024-09-16T09:29:28.861 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:29:28.813306+0000", 2024-09-16T09:29:28.861 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.0027163859999999999, 2024-09-16T09:29:28.861 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0028785519999999999, 2024-09-16T09:29:28.861 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:29:28.861 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:29:28.862 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T09:29:28.862 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:29:28.862 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:28.862 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:29:28.862 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:28.813306+0000", 2024-09-16T09:29:28.862 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:29:28.862 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:28.862 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:28.862 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:29:28.862 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:28.813306+0000", 2024-09-16T09:29:28.862 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:29:28.862 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:28.862 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:28.862 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:29:28.863 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:28.813306+0000", 2024-09-16T09:29:28.863 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.72e-07 2024-09-16T09:29:28.863 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:28.863 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:28.863 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:29:28.863 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:28.813310+0000", 2024-09-16T09:29:28.863 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.608e-06 2024-09-16T09:29:28.863 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:28.863 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:28.863 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:29:28.863 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:28.813316+0000", 2024-09-16T09:29:28.863 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.2920000000000001e-06 2024-09-16T09:29:28.863 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:28.863 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:28.863 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:29:28.863 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:28.813320+0000", 2024-09-16T09:29:28.864 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0099999999999997e-06 2024-09-16T09:29:28.864 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:28.864 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:28.864 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:29:28.864 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:28.813449+0000", 2024-09-16T09:29:28.864 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00012860899999999999 2024-09-16T09:29:28.864 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:28.864 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:28.864 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:29:28.864 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:28.813466+0000", 2024-09-16T09:29:28.864 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7526000000000001e-05 2024-09-16T09:29:28.864 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:29:28.864 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:29:28.864 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:29:28.864 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:28.865 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:28.865 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop_reply(client.24698.0:242888 7.b e146/82 ondisk, result = 0)", 2024-09-16T09:29:28.865 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:29:28.815907+0000", 2024-09-16T09:29:28.865 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.000115129, 2024-09-16T09:29:28.865 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00032398399999999998, 2024-09-16T09:29:28.865 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:29:28.865 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:29:28.865 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T09:29:28.865 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:29:28.865 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:28.865 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:29:28.865 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:28.815907+0000", 2024-09-16T09:29:28.865 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:29:28.865 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:28.866 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:28.866 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:29:28.866 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:28.815907+0000", 2024-09-16T09:29:28.866 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:29:28.866 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:28.866 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:28.866 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:29:28.866 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:28.815908+0000", 2024-09-16T09:29:28.866 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.034e-06 2024-09-16T09:29:28.866 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:28.866 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:28.866 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:29:28.866 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:28.815913+0000", 2024-09-16T09:29:28.866 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.2360000000000004e-06 2024-09-16T09:29:28.866 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:28.867 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:28.867 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:29:28.867 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:28.815926+0000", 2024-09-16T09:29:28.867 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2813000000000001e-05 2024-09-16T09:29:28.867 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:28.867 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:28.867 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:29:28.867 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:28.815934+0000", 2024-09-16T09:29:28.867 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.7149999999999998e-06 2024-09-16T09:29:28.867 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:28.867 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:28.867 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:29:28.867 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:28.816013+0000", 2024-09-16T09:29:28.867 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.9741999999999996e-05 2024-09-16T09:29:28.867 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:28.868 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:28.868 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:29:28.868 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:28.816020+0000", 2024-09-16T09:29:28.868 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.5899999999999996e-06 2024-09-16T09:29:28.868 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:29:28.868 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:29:28.868 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:29:28.868 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:29:28.868 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-16T09:29:28.868 INFO:teuthology.orchestra.run.smithi070.stdout: "num_ops": 4 2024-09-16T09:29:28.868 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T09:29:28.869 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:29:28.869 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:29:28.872 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:29:28.896 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:29:28.896 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:29:28.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:28 smithi084 ceph-mon[32629]: 7.7 scrub starts 2024-09-16T09:29:28.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:28 smithi084 ceph-mon[32629]: pgmap v194: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 871 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1.05k op/s 2024-09-16T09:29:28.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:28 smithi084 ceph-mon[32629]: 7.7 scrub ok 2024-09-16T09:29:28.935 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:29:28.935 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:29:28.943 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:28.944 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:28.944 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:29:28.965 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:29:28.965 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:29:29.001 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:29:29.001 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:29:29.045 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:29:29.093 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:29.094 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:29.094 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:29:29.195 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:29:29.246 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:29.247 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:29.247 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:29:29.348 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:29:29.376 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:29.377 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:29.377 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:29:29.478 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:29:29.506 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:29.507 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:29.508 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:29:29.609 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:29:29.638 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:29.639 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:29.639 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:29:29.740 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:29:29.782 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:29.783 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:29.783 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:29:29.871 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:29 smithi070 ceph-mon[30388]: 4.1a scrub starts 2024-09-16T09:29:29.871 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:29 smithi070 ceph-mon[30388]: 4.1a scrub ok 2024-09-16T09:29:29.872 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:29 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/619340519' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:29:29.872 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:29 smithi070 ceph-mon[40691]: 4.1a scrub starts 2024-09-16T09:29:29.872 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:29 smithi070 ceph-mon[40691]: 4.1a scrub ok 2024-09-16T09:29:29.872 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:29 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/619340519' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:29:29.884 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:29:29.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:29 smithi084 ceph-mon[32629]: 4.1a scrub starts 2024-09-16T09:29:29.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:29 smithi084 ceph-mon[32629]: 4.1a scrub ok 2024-09-16T09:29:29.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:29 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/619340519' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:29:29.929 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:29.930 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:29.930 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:29:29.960 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:29:29.963 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:29:30.031 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:29:30.057 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:30.058 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:30.058 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:29:30.159 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:29:30.184 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:30.185 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:30.185 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:29:30.286 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:29:30.312 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:30.313 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:30.313 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:29:30.414 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:29:30.462 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.1 dump_blocked_ops 2024-09-16T09:29:30.477 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:30.477 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:30.478 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:29:30.578 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:29:30.620 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:30.621 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:30.621 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:29:30.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:30 smithi070 ceph-mon[30388]: 7.c scrub starts 2024-09-16T09:29:30.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:30 smithi070 ceph-mon[30388]: 2.2 scrub starts 2024-09-16T09:29:30.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:30 smithi070 ceph-mon[30388]: 2.2 scrub ok 2024-09-16T09:29:30.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:30 smithi070 ceph-mon[30388]: pgmap v195: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 655 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.38k op/s 2024-09-16T09:29:30.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:30 smithi070 ceph-mon[30388]: 7.c scrub ok 2024-09-16T09:29:30.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:30 smithi070 ceph-mon[30388]: 5.d scrub starts 2024-09-16T09:29:30.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:30 smithi070 ceph-mon[30388]: 5.d scrub ok 2024-09-16T09:29:30.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:30 smithi070 ceph-mon[30388]: 6.1c scrub starts 2024-09-16T09:29:30.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:30 smithi070 ceph-mon[30388]: 6.1c scrub ok 2024-09-16T09:29:30.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:30 smithi070 ceph-mon[40691]: 7.c scrub starts 2024-09-16T09:29:30.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:30 smithi070 ceph-mon[40691]: 2.2 scrub starts 2024-09-16T09:29:30.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:30 smithi070 ceph-mon[40691]: 2.2 scrub ok 2024-09-16T09:29:30.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:30 smithi070 ceph-mon[40691]: pgmap v195: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 655 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.38k op/s 2024-09-16T09:29:30.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:30 smithi070 ceph-mon[40691]: 7.c scrub ok 2024-09-16T09:29:30.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:30 smithi070 ceph-mon[40691]: 5.d scrub starts 2024-09-16T09:29:30.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:30 smithi070 ceph-mon[40691]: 5.d scrub ok 2024-09-16T09:29:30.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:30 smithi070 ceph-mon[40691]: 6.1c scrub starts 2024-09-16T09:29:30.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:30 smithi070 ceph-mon[40691]: 6.1c scrub ok 2024-09-16T09:29:30.723 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:29:30.784 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:30.785 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:30.785 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:29:30.886 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:29:30.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:30 smithi084 ceph-mon[32629]: 7.c scrub starts 2024-09-16T09:29:30.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:30 smithi084 ceph-mon[32629]: 2.2 scrub starts 2024-09-16T09:29:30.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:30 smithi084 ceph-mon[32629]: 2.2 scrub ok 2024-09-16T09:29:30.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:30 smithi084 ceph-mon[32629]: pgmap v195: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 655 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.38k op/s 2024-09-16T09:29:30.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:30 smithi084 ceph-mon[32629]: 7.c scrub ok 2024-09-16T09:29:30.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:30 smithi084 ceph-mon[32629]: 5.d scrub starts 2024-09-16T09:29:30.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:30 smithi084 ceph-mon[32629]: 5.d scrub ok 2024-09-16T09:29:30.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:30 smithi084 ceph-mon[32629]: 6.1c scrub starts 2024-09-16T09:29:30.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:30 smithi084 ceph-mon[32629]: 6.1c scrub ok 2024-09-16T09:29:30.911 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:30.912 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:30.912 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:29:31.013 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:29:31.055 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:31.056 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:31.056 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:29:31.157 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:29:31.192 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:31.192 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:31.193 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:29:31.293 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:29:31.314 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:31.315 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:31.315 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:29:31.416 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:29:31.442 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:31.443 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:31.443 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:29:31.544 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:29:31.572 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:31.573 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:31.573 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:29:31.674 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:29:31.698 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:31.698 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:31.699 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:29:31.800 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:29:31.825 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:31.826 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:31.826 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:29:31.919 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T09:29:31.927 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:29:31.952 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:31.953 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:31.953 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:29:32.054 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:29:32.069 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T09:29:32.069 INFO:teuthology.orchestra.run.smithi070.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-16T09:29:32.069 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T09:29:32.119 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:32.120 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:32.120 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:29:32.172 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:29:31 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:29:31] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:29:32.222 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:29:32.245 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:32.246 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:32.246 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:29:32.347 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:29:32.370 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:32.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:32.371 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:29:32.472 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:29:32.500 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:32.501 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:32.501 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:29:32.602 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:29:32.631 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:32.631 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:32.632 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:29:32.645 INFO:teuthology.orchestra.run.smithi070.stderr:noscrub is set 2024-09-16T09:29:32.732 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:29:32.773 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:32.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:32.774 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:29:32.874 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:29:32.910 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:32.911 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:32.911 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:29:32.911 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:29:32 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:29:32.912 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:32 smithi084 ceph-mon[32629]: 5.16 scrub starts 2024-09-16T09:29:32.912 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:32 smithi084 ceph-mon[32629]: 5.16 scrub ok 2024-09-16T09:29:32.912 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:32 smithi084 ceph-mon[32629]: pgmap v196: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 655 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 628 B/s rd, 0 B/s wr, 675 op/s 2024-09-16T09:29:32.912 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:32 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/728304271' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:29:32.912 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:32 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:29:32.912 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:32 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:29:32.912 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:32 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:29:32.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:32 smithi070 ceph-mon[30388]: 5.16 scrub starts 2024-09-16T09:29:32.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:32 smithi070 ceph-mon[30388]: 5.16 scrub ok 2024-09-16T09:29:32.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:32 smithi070 ceph-mon[30388]: pgmap v196: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 655 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 628 B/s rd, 0 B/s wr, 675 op/s 2024-09-16T09:29:32.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:32 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/728304271' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:29:32.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:32 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:29:32.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:32 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:29:32.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:32 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:29:32.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:32 smithi070 ceph-mon[40691]: 5.16 scrub starts 2024-09-16T09:29:32.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:32 smithi070 ceph-mon[40691]: 5.16 scrub ok 2024-09-16T09:29:32.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:32 smithi070 ceph-mon[40691]: pgmap v196: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 655 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 628 B/s rd, 0 B/s wr, 675 op/s 2024-09-16T09:29:32.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:32 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/728304271' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:29:32.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:32 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:29:32.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:32 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:29:32.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:32 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:29:33.012 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:29:33.038 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:33.038 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:33.039 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:29:33.140 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:29:33.165 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:33.166 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:33.166 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:29:33.267 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:29:33.294 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:33.295 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:33.295 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:29:33.396 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:29:33.435 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:33.436 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:33.436 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:29:33.537 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:29:33.573 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:33.573 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:33.574 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:29:33.674 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:29:33.710 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:33.710 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:33.711 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:29:33.805 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:33 smithi070 ceph-mon[30388]: 4.15 scrub starts 2024-09-16T09:29:33.805 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:33 smithi070 ceph-mon[30388]: 4.15 scrub ok 2024-09-16T09:29:33.805 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:33 smithi070 ceph-mon[30388]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:29:33.806 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:33 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-16T09:29:33.806 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:33 smithi070 ceph-mon[30388]: osdmap e147: 8 total, 8 up, 8 in 2024-09-16T09:29:33.806 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:33 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:29:33.806 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:33 smithi070 ceph-mon[40691]: 4.15 scrub starts 2024-09-16T09:29:33.806 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:33 smithi070 ceph-mon[40691]: 4.15 scrub ok 2024-09-16T09:29:33.806 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:33 smithi070 ceph-mon[40691]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:29:33.806 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:33 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-16T09:29:33.806 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:33 smithi070 ceph-mon[40691]: osdmap e147: 8 total, 8 up, 8 in 2024-09-16T09:29:33.806 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:33 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:29:33.812 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:29:33.881 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:29:33.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:33.886 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:33.886 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:29:33.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:33 smithi084 ceph-mon[32629]: 4.15 scrub starts 2024-09-16T09:29:33.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:33 smithi084 ceph-mon[32629]: 4.15 scrub ok 2024-09-16T09:29:33.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:33 smithi084 ceph-mon[32629]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:29:33.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:33 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-16T09:29:33.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:33 smithi084 ceph-mon[32629]: osdmap e147: 8 total, 8 up, 8 in 2024-09-16T09:29:33.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:33 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:29:33.987 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:29:34.012 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:34.013 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:34.013 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:29:34.114 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:29:34.144 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:34.144 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:34.145 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:29:34.246 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:29:34.270 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:34.271 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:34.271 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:29:34.372 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:29:34.399 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:34.400 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:34.400 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:29:34.491 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:29:34.501 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:29:34.547 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:34.547 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:34.548 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:29:34.549 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T09:29:34.549 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T09:29:34.549 INFO:teuthology.orchestra.run.smithi070.stdout: "complaint_time": 30, 2024-09-16T09:29:34.549 INFO:teuthology.orchestra.run.smithi070.stdout: "num_blocked_ops": 0 2024-09-16T09:29:34.549 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T09:29:34.649 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:29:34.689 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:34.690 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:34.691 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:29:34.791 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:29:34.830 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:34.830 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:34.831 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:29:34.931 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:29:34.964 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:34.965 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:34.965 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:29:35.066 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:29:35.097 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:35.098 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:35.098 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:29:35.199 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:29:35.234 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:35.235 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:35.235 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:29:35.326 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:29:35.326 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:29:35.336 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:29:35.366 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:29:35.366 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:29:35.379 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:35.380 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:35.380 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:29:35.405 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:29:35.405 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:29:35.405 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:35 smithi070 ceph-mon[30388]: pgmap v198: 158 pgs: 158 active+clean; 323 MiB data, 2.0 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.33k op/s 2024-09-16T09:29:35.405 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:35 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:29:35.406 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:35 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:29:35.406 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:35 smithi070 ceph-mon[40691]: pgmap v198: 158 pgs: 158 active+clean; 323 MiB data, 2.0 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.33k op/s 2024-09-16T09:29:35.406 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:35 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:29:35.406 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:35 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:29:35.440 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:29:35.440 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:29:35.476 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:29:35.476 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:29:35.481 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:29:35.506 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:29:35.506 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:29:35.508 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:35.509 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:35.509 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:29:35.538 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:29:35.538 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:29:35.569 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:29:35.569 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:29:35.610 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:29:35.635 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:35.635 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:35.636 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:29:35.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:35 smithi084 ceph-mon[32629]: pgmap v198: 158 pgs: 158 active+clean; 323 MiB data, 2.0 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.33k op/s 2024-09-16T09:29:35.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:35 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:29:35.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:35 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:29:35.736 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:29:35.776 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:35.776 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:35.777 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:29:35.859 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-16T09:29:35.877 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:29:35.945 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:35.945 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:35.946 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:29:36.001 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.1 dump_historic_ops 2024-09-16T09:29:36.047 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:29:36.072 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:36.072 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:36.073 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:29:36.173 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:29:36.198 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:36.199 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:36.199 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:29:36.300 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:29:36.327 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:36.328 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:36.328 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:29:36.429 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:29:36.476 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:36 smithi070 ceph-mon[30388]: osdmap e148: 8 total, 8 up, 8 in 2024-09-16T09:29:36.477 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:36 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3855318516' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:29:36.477 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:36 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:29:36.477 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:36 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:29:36.477 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:36 smithi070 ceph-mon[40691]: osdmap e148: 8 total, 8 up, 8 in 2024-09-16T09:29:36.477 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:36 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3855318516' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:29:36.477 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:36 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:29:36.477 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:36 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:29:36.497 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:36.498 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:36.498 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:29:36.549 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:29:36.599 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:29:36.645 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:36.645 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:36.646 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:29:36.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:36 smithi084 ceph-mon[32629]: osdmap e148: 8 total, 8 up, 8 in 2024-09-16T09:29:36.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:36 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3855318516' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:29:36.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:36 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:29:36.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:36 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:29:36.746 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:29:36.786 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:36.787 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:36.787 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:29:36.888 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:29:36.912 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:36.913 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:36.913 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:29:37.014 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:29:37.045 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:37.046 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:37.046 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:29:37.147 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:29:37.170 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:37.170 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:37.171 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:29:37.212 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:29:37.219 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T09:29:37.219 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:29:37.271 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:29:37.301 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:37.301 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:37.302 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:29:37.403 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:29:37.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:37 smithi070 ceph-mon[30388]: pgmap v200: 158 pgs: 158 active+clean; 323 MiB data, 2.0 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.10k op/s 2024-09-16T09:29:37.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:37 smithi070 ceph-mon[30388]: osdmap e149: 8 total, 8 up, 8 in 2024-09-16T09:29:37.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:37 smithi070 ceph-mon[40691]: pgmap v200: 158 pgs: 158 active+clean; 323 MiB data, 2.0 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.10k op/s 2024-09-16T09:29:37.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:37 smithi070 ceph-mon[40691]: osdmap e149: 8 total, 8 up, 8 in 2024-09-16T09:29:37.427 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:37.428 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:37.428 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:29:37.438 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Removed 126128 objects 2024-09-16T09:29:37.438 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Clean up completed and total clean up time :114.395 2024-09-16T09:29:37.529 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:29:37.551 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:37.551 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:37.552 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:29:37.653 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:29:37.659 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:37 smithi084 ceph-mon[32629]: pgmap v200: 158 pgs: 158 active+clean; 323 MiB data, 2.0 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.10k op/s 2024-09-16T09:29:37.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:37 smithi084 ceph-mon[32629]: osdmap e149: 8 total, 8 up, 8 in 2024-09-16T09:29:37.673 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:37.674 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:37.674 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:29:37.775 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:29:37.797 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:37.798 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:37.798 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:29:37.899 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:29:37.921 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:37.921 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:37.921 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:29:38.022 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:29:38.043 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:38.043 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:38.044 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:29:38.137 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.2 dump_ops_in_flight 2024-09-16T09:29:38.144 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:29:38.156 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T09:29:38.157 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T09:29:38.157 INFO:tasks.thrashosds.thrasher:inject_pause on osd.7 2024-09-16T09:29:38.157 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-16T09:29:38.157 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-16T09:29:38.157 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.7 config set filestore_inject_stall 3 2024-09-16T09:29:38.167 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:38.167 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:38.167 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:29:38.269 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:29:38.291 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:38.291 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:38.292 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:29:38.393 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:29:38.411 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:29:38.413 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:38.414 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:38.414 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:29:38.482 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T09:29:38.515 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:29:38.547 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:38.548 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:38.548 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:29:38.649 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:29:38.672 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:38.673 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:38.673 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:29:38.774 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:29:38.796 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:38.797 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:38.797 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:29:38.898 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:29:38.918 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:38.919 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:38.919 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:29:38.919 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-09-16T09:29:38.920 INFO:teuthology.orchestra.run.smithi084.stdout: "success": "" 2024-09-16T09:29:38.920 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-09-16T09:29:39.020 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:29:39.041 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:39.041 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:39.042 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:29:39.142 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:29:39.166 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:39.167 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:39.167 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:29:39.267 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:29:39.289 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:39.289 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:39.289 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:29:39.390 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:29:39.411 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:39.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:39.412 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:29:39.513 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:29:39.534 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:39.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:39.535 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:29:39.636 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:29:39.658 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:39.658 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:39.659 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:29:39.659 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:39 smithi084 ceph-mon[32629]: pgmap v202: 158 pgs: 158 active+clean; 198 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.22k op/s 2024-09-16T09:29:39.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:39 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/2773603905' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:29:39.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:39 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:29:39.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:39 smithi070 ceph-mon[30388]: pgmap v202: 158 pgs: 158 active+clean; 198 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.22k op/s 2024-09-16T09:29:39.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:39 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2773603905' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:29:39.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:39 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:29:39.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:39 smithi070 ceph-mon[40691]: pgmap v202: 158 pgs: 158 active+clean; 198 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.22k op/s 2024-09-16T09:29:39.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:39 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/2773603905' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:29:39.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:39 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:29:39.760 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:29:39.782 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:39.782 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:39.782 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:29:39.883 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:29:39.905 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:39.906 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:39.906 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:29:40.003 INFO:teuthology.orchestra.run.smithi070.stderr:nodeep-scrub is set 2024-09-16T09:29:40.007 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:29:40.029 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:40.030 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:40.030 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:29:40.131 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:29:40.153 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:40.153 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:40.153 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:29:40.254 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:29:40.276 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:40.276 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:40.277 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:29:40.377 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:29:40.385 INFO:tasks.cephadm.ceph_manager.ceph:removing pool_name unique_pool_0 2024-09-16T09:29:40.386 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd pool rm unique_pool_0 unique_pool_0 --yes-i-really-really-mean-it 2024-09-16T09:29:40.399 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:40.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:40.400 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:29:40.412 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:40 smithi070 ceph-mon[30388]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:29:40.413 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:40 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-16T09:29:40.413 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:40 smithi070 ceph-mon[30388]: osdmap e150: 8 total, 8 up, 8 in 2024-09-16T09:29:40.413 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:40 smithi070 ceph-mon[40691]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:29:40.413 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:40 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-16T09:29:40.413 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:40 smithi070 ceph-mon[40691]: osdmap e150: 8 total, 8 up, 8 in 2024-09-16T09:29:40.501 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:29:40.523 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:40.524 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:40.524 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:29:40.625 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:29:40.646 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:40.647 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:40.647 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:29:40.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:40 smithi084 ceph-mon[32629]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:29:40.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:40 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-16T09:29:40.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:40 smithi084 ceph-mon[32629]: osdmap e150: 8 total, 8 up, 8 in 2024-09-16T09:29:40.748 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:29:40.770 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:40.770 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:40.770 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:29:40.871 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:29:40.892 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:40.893 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:40.893 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:29:40.982 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:29:40.994 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:29:41.020 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:41.020 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:41.020 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:29:41.121 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:29:41.144 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:41.144 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:41.145 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:29:41.248 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:29:41.270 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:41.270 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:41.270 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:29:41.371 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:29:41.375 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:41 smithi070 ceph-mon[30388]: pgmap v204: 158 pgs: 158 active+clean; 583 KiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 859 op/s 2024-09-16T09:29:41.375 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:41 smithi070 ceph-mon[40691]: pgmap v204: 158 pgs: 158 active+clean; 583 KiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 859 op/s 2024-09-16T09:29:41.397 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:41.397 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:41.398 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:29:41.413 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:29:41.499 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:29:41.524 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:41.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:41.525 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:29:41.626 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:29:41.652 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:41.652 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:41.653 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:29:41.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:41 smithi084 ceph-mon[32629]: pgmap v204: 158 pgs: 158 active+clean; 583 KiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 859 op/s 2024-09-16T09:29:41.753 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:29:41.778 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:41.778 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:41.779 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:29:41.880 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:29:41.901 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:41.902 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:41.902 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:29:42.003 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:29:42.028 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:42.029 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:42.029 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:29:42.072 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:29:41 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:29:41] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:29:42.130 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:29:42.156 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:42.157 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:42.157 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:29:42.258 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:29:42.284 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:42.284 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:42.285 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:29:42.385 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:29:42.408 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:42.408 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:42.409 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:29:42.432 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T09:29:42.432 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:29:42.510 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:29:42.535 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:42.536 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:42.536 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:29:42.637 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:29:42.659 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:42.660 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:42.660 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:29:42.761 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:29:42.781 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:42.782 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:42.783 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:29:42.883 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:29:42.909 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:42.910 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:42.910 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:29:43.011 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:29:43.032 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:43.033 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:43.033 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:29:43.134 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:29:43.155 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:43.156 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:43.156 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:29:43.160 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:29:42 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:29:43.223 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'unique_pool_0' removed 2024-09-16T09:29:43.257 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:29:43.284 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:43.284 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:43.285 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:29:43.295 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.2 dump_blocked_ops 2024-09-16T09:29:43.385 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:29:43.408 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:43.409 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:43.409 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:29:43.410 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd unset noscrub 2024-09-16T09:29:43.509 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:29:43.530 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:43.531 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:43.531 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:29:43.592 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:43 smithi070 ceph-mon[30388]: pgmap v205: 158 pgs: 158 active+clean; 583 KiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 802 B/s rd, 0 B/s wr, 809 op/s 2024-09-16T09:29:43.592 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:43 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/871428859' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true} : dispatch 2024-09-16T09:29:43.593 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:43 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true} : dispatch 2024-09-16T09:29:43.593 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:43 smithi070 ceph-mon[40691]: pgmap v205: 158 pgs: 158 active+clean; 583 KiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 802 B/s rd, 0 B/s wr, 809 op/s 2024-09-16T09:29:43.593 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:43 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/871428859' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true} : dispatch 2024-09-16T09:29:43.593 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:43 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true} : dispatch 2024-09-16T09:29:43.632 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:29:43.653 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:43.653 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:43.654 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:29:43.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:43 smithi084 ceph-mon[32629]: pgmap v205: 158 pgs: 158 active+clean; 583 KiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 802 B/s rd, 0 B/s wr, 809 op/s 2024-09-16T09:29:43.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:43 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/871428859' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true} : dispatch 2024-09-16T09:29:43.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:43 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true} : dispatch 2024-09-16T09:29:43.754 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:29:43.777 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:43.778 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:43.778 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:29:43.879 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:29:43.900 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:43.901 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:43.901 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:29:43.940 INFO:teuthology.task.full_sequential:In full_sequential, running task radosbench... 2024-09-16T09:29:43.941 INFO:tasks.radosbench:Beginning radosbench... 2024-09-16T09:29:43.941 INFO:tasks.cephadm.ceph_manager.ceph:creating pool_name unique_pool_1 2024-09-16T09:29:43.942 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd pool create unique_pool_1 16 2024-09-16T09:29:44.002 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:29:44.043 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:29:44.060 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:29:44.089 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:44.090 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:44.090 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:29:44.090 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:29:44.191 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:29:44.213 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:44.213 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:44.214 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:29:44.315 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:29:44.336 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:44.337 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:44.337 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:29:44.438 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:29:44.455 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:44 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:29:44.455 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:44 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true}]': finished 2024-09-16T09:29:44.455 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:44 smithi070 ceph-mon[30388]: osdmap e151: 8 total, 8 up, 8 in 2024-09-16T09:29:44.455 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:44 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:29:44.456 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:44 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true}]': finished 2024-09-16T09:29:44.456 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:44 smithi070 ceph-mon[40691]: osdmap e151: 8 total, 8 up, 8 in 2024-09-16T09:29:44.459 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:44.459 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:44.460 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:29:44.561 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:29:44.582 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:44.583 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:44.583 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:29:44.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:44 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:29:44.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:44 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true}]': finished 2024-09-16T09:29:44.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:44 smithi084 ceph-mon[32629]: osdmap e151: 8 total, 8 up, 8 in 2024-09-16T09:29:44.683 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:29:44.690 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T09:29:44.690 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T09:29:44.691 INFO:tasks.thrashosds.thrasher:inject_pause on osd.0 2024-09-16T09:29:44.691 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-16T09:29:44.691 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-16T09:29:44.691 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.0 config set filestore_inject_stall 3 2024-09-16T09:29:44.705 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:44.706 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:44.706 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:29:44.807 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:29:44.828 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:44.829 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:44.829 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:29:44.930 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:29:44.952 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:44.953 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:44.953 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:29:45.054 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:29:45.076 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:45.077 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:45.077 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:29:45.178 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:29:45.199 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:45.200 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:45.200 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:29:45.301 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:29:45.346 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:45.347 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:45.347 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:29:45.416 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:29:45.416 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:29:45.416 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:29:45.416 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:29:45.416 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:29:45.416 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:29:45.416 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:29:45.416 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:29:45.417 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:29:45.417 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:29:45.417 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:29:45.417 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:29:45.417 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:29:45.417 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:29:45.417 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:29:45.417 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:29:45.448 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:29:45.470 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:45.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:45.471 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:29:45.561 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:45 smithi084 ceph-mon[32629]: pgmap v207: 142 pgs: 142 active+clean; 583 KiB data, 994 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:29:45.561 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:45 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3420741445' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:29:45.573 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:29:45.594 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:45.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:45.595 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:29:45.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:45 smithi070 ceph-mon[30388]: pgmap v207: 142 pgs: 142 active+clean; 583 KiB data, 994 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:29:45.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:45 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3420741445' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:29:45.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:45 smithi070 ceph-mon[40691]: pgmap v207: 142 pgs: 142 active+clean; 583 KiB data, 994 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:29:45.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:45 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3420741445' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:29:45.696 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:29:45.722 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:45.723 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:45.723 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:29:45.824 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:29:45.836 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T09:29:45.836 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T09:29:45.836 INFO:teuthology.orchestra.run.smithi070.stdout: "complaint_time": 30, 2024-09-16T09:29:45.836 INFO:teuthology.orchestra.run.smithi070.stdout: "num_blocked_ops": 0 2024-09-16T09:29:45.836 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T09:29:45.846 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:45.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:45.847 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:29:45.947 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:29:45.969 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:45.969 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:45.970 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:29:46.070 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:29:46.092 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:46.092 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:46.093 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:29:46.193 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:29:46.219 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:46.219 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:46.220 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:29:46.320 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:29:46.341 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:46.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:46.342 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:29:46.443 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:29:46.464 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:46.465 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:46.465 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:29:46.566 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:29:46.587 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:46.588 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:46.588 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:29:46.689 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:29:46.718 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:46.719 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:46.719 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:29:46.821 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:29:46.842 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:46.843 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:46.843 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:29:46.944 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:29:46.966 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:46.966 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:46.967 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:29:47.067 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:29:47.089 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:47.089 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:47.089 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:29:47.166 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.2 dump_historic_ops 2024-09-16T09:29:47.190 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:29:47.232 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:47.233 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:47.233 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:29:47.334 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:29:47.355 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:47.356 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:47.356 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:29:47.380 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:47 smithi070 ceph-mon[30388]: pgmap v208: 142 pgs: 142 active+clean; 583 KiB data, 994 MiB used, 714 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-09-16T09:29:47.381 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:47 smithi070 ceph-mon[40691]: pgmap v208: 142 pgs: 142 active+clean; 583 KiB data, 994 MiB used, 714 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-09-16T09:29:47.386 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:29:47.457 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:29:47.478 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:47.479 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:47.479 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:29:47.580 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:29:47.601 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:47.602 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:47.602 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:29:47.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:47 smithi084 ceph-mon[32629]: pgmap v208: 142 pgs: 142 active+clean; 583 KiB data, 994 MiB used, 714 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-09-16T09:29:47.704 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:29:47.725 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:47.725 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:47.726 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:29:47.827 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:29:47.848 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:47.849 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:47.849 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:29:47.916 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:29:47.951 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:29:47.956 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:29:47.974 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:47.975 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:47.975 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:29:48.076 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:29:48.097 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:48.098 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:48.098 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:29:48.199 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:29:48.221 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:48.222 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:48.222 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:29:48.323 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:29:48.344 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:48.345 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:48.345 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:29:48.439 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'unique_pool_1' created 2024-09-16T09:29:48.439 INFO:teuthology.orchestra.run.smithi070.stderr:noscrub is unset 2024-09-16T09:29:48.447 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:29:48.469 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:48.470 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:48.470 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:29:48.478 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:48 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:29:48.478 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:48 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:29:48.478 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:48 smithi070 ceph-mon[30388]: pgmap v209: 142 pgs: 142 active+clean; 583 KiB data, 995 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-16T09:29:48.478 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:48 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1298063026' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:29:48.478 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:48 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3240934423' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "unique_pool_1", "pg_num": 16} : dispatch 2024-09-16T09:29:48.478 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:48 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:29:48.478 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:48 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:29:48.478 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:48 smithi070 ceph-mon[40691]: pgmap v209: 142 pgs: 142 active+clean; 583 KiB data, 995 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-16T09:29:48.478 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:48 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1298063026' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:29:48.479 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:48 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3240934423' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "unique_pool_1", "pg_num": 16} : dispatch 2024-09-16T09:29:48.571 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:29:48.592 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:48.593 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:48.593 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:29:48.694 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:29:48.716 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:48.716 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:48.717 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:29:48.732 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T09:29:48.818 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:29:48.843 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:48.844 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:48.844 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:29:48.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:48 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:29:48.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:48 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:29:48.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:48 smithi084 ceph-mon[32629]: pgmap v209: 142 pgs: 142 active+clean; 583 KiB data, 995 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-16T09:29:48.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:48 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1298063026' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:29:48.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:48 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3240934423' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "unique_pool_1", "pg_num": 16} : dispatch 2024-09-16T09:29:48.945 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:29:48.966 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:48.966 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:48.967 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:29:49.067 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:29:49.092 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:49.093 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:49.094 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:29:49.195 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:29:49.218 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:49.219 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:49.219 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:29:49.320 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:29:49.332 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd pool application enable unique_pool_1 rados --yes-i-really-mean-it || true 2024-09-16T09:29:49.344 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:49.345 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:49.345 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:29:49.446 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:29:49.470 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:49.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:49.471 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:29:49.564 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:49 smithi070 ceph-mon[30388]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:29:49.565 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:49 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1298063026' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-16T09:29:49.565 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:49 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3240934423' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_1", "pg_num": 16}]': finished 2024-09-16T09:29:49.565 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:49 smithi070 ceph-mon[30388]: osdmap e152: 8 total, 8 up, 8 in 2024-09-16T09:29:49.565 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:49 smithi070 ceph-mon[30388]: 5.15 scrub starts 2024-09-16T09:29:49.565 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:49 smithi070 ceph-mon[30388]: 5.15 scrub ok 2024-09-16T09:29:49.565 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:49 smithi070 ceph-mon[30388]: 3.12 scrub starts 2024-09-16T09:29:49.565 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:49 smithi070 ceph-mon[30388]: 3.12 scrub ok 2024-09-16T09:29:49.565 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:49 smithi070 ceph-mon[30388]: osdmap e153: 8 total, 8 up, 8 in 2024-09-16T09:29:49.565 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:49 smithi070 ceph-mon[40691]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:29:49.565 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:49 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1298063026' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-16T09:29:49.566 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:49 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3240934423' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_1", "pg_num": 16}]': finished 2024-09-16T09:29:49.566 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:49 smithi070 ceph-mon[40691]: osdmap e152: 8 total, 8 up, 8 in 2024-09-16T09:29:49.566 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:49 smithi070 ceph-mon[40691]: 5.15 scrub starts 2024-09-16T09:29:49.566 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:49 smithi070 ceph-mon[40691]: 5.15 scrub ok 2024-09-16T09:29:49.566 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:49 smithi070 ceph-mon[40691]: 3.12 scrub starts 2024-09-16T09:29:49.566 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:49 smithi070 ceph-mon[40691]: 3.12 scrub ok 2024-09-16T09:29:49.566 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:49 smithi070 ceph-mon[40691]: osdmap e153: 8 total, 8 up, 8 in 2024-09-16T09:29:49.572 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:29:49.578 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T09:29:49.578 INFO:teuthology.orchestra.run.smithi070.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-16T09:29:49.578 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T09:29:49.596 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:49.597 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:49.597 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:29:49.698 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:29:49.724 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:49.725 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:49.725 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:29:49.826 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:29:49.852 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:49.853 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:49.854 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:29:49.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:49 smithi084 ceph-mon[32629]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:29:49.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:49 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1298063026' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-16T09:29:49.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:49 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3240934423' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_1", "pg_num": 16}]': finished 2024-09-16T09:29:49.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:49 smithi084 ceph-mon[32629]: osdmap e152: 8 total, 8 up, 8 in 2024-09-16T09:29:49.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:49 smithi084 ceph-mon[32629]: 5.15 scrub starts 2024-09-16T09:29:49.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:49 smithi084 ceph-mon[32629]: 5.15 scrub ok 2024-09-16T09:29:49.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:49 smithi084 ceph-mon[32629]: 3.12 scrub starts 2024-09-16T09:29:49.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:49 smithi084 ceph-mon[32629]: 3.12 scrub ok 2024-09-16T09:29:49.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:49 smithi084 ceph-mon[32629]: osdmap e153: 8 total, 8 up, 8 in 2024-09-16T09:29:49.955 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:29:49.976 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:49.976 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:49.977 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:29:50.077 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:29:50.099 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:50.100 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:50.100 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:29:50.201 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:29:50.227 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:50.228 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:50.228 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:29:50.329 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:29:50.355 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:50.356 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:50.356 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:29:50.457 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:29:50.484 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:50.484 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:50.485 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:29:50.585 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:29:50.612 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:50.612 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:50.613 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:29:50.714 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:29:50.736 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:50.736 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:50.736 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:29:50.737 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:50 smithi070 ceph-mon[30388]: 6.17 scrub starts 2024-09-16T09:29:50.737 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:50 smithi070 ceph-mon[30388]: 6.17 scrub ok 2024-09-16T09:29:50.737 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:50 smithi070 ceph-mon[30388]: pgmap v212: 158 pgs: 16 unknown, 142 active+clean; 583 KiB data, 999 MiB used, 714 GiB / 715 GiB avail; 805 B/s rd, 0 op/s 2024-09-16T09:29:50.737 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:50 smithi070 ceph-mon[30388]: osdmap e154: 8 total, 8 up, 8 in 2024-09-16T09:29:50.738 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:50 smithi070 ceph-mon[40691]: 6.17 scrub starts 2024-09-16T09:29:50.738 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:50 smithi070 ceph-mon[40691]: 6.17 scrub ok 2024-09-16T09:29:50.738 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:50 smithi070 ceph-mon[40691]: pgmap v212: 158 pgs: 16 unknown, 142 active+clean; 583 KiB data, 999 MiB used, 714 GiB / 715 GiB avail; 805 B/s rd, 0 op/s 2024-09-16T09:29:50.738 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:50 smithi070 ceph-mon[40691]: osdmap e154: 8 total, 8 up, 8 in 2024-09-16T09:29:50.838 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:29:50.840 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T09:29:50.840 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 20, 2024-09-16T09:29:50.840 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 600, 2024-09-16T09:29:50.841 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [ 2024-09-16T09:29:50.841 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.841 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.24698.0:60244 7.e 7:7b2ed1e2:::benchmark_data_smithi070_78438_object60243:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-16T09:29:50.841 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:26:44.351525+0000", 2024-09-16T09:29:50.841 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 186.48616150699999, 2024-09-16T09:29:50.841 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.142576176, 2024-09-16T09:29:50.841 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:29:50.844 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:29:50.844 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:29:50.844 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:29:50.845 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.24698", 2024-09-16T09:29:50.845 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:29:50.845 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 60244 2024-09-16T09:29:50.845 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.845 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:29:50.845 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.845 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:29:50.845 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.351525+0000", 2024-09-16T09:29:50.845 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:29:50.845 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.845 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.845 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:29:50.845 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.351525+0000", 2024-09-16T09:29:50.846 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:29:50.846 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.846 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.846 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:29:50.846 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.351526+0000", 2024-09-16T09:29:50.846 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.342e-06 2024-09-16T09:29:50.846 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.846 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.846 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:29:50.846 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.351550+0000", 2024-09-16T09:29:50.846 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3652999999999999e-05 2024-09-16T09:29:50.846 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.846 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.846 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:29:50.847 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.351561+0000", 2024-09-16T09:29:50.847 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0848e-05 2024-09-16T09:29:50.847 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.847 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.847 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:29:50.847 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.351567+0000", 2024-09-16T09:29:50.847 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.742e-06 2024-09-16T09:29:50.847 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.847 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.847 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:29:50.847 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.351668+0000", 2024-09-16T09:29:50.847 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000100317 2024-09-16T09:29:50.847 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.847 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.848 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:29:50.848 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.355242+0000", 2024-09-16T09:29:50.848 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0035748020000000002 2024-09-16T09:29:50.848 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.848 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.848 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 0,5", 2024-09-16T09:29:50.848 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.355413+0000", 2024-09-16T09:29:50.848 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000170965 2024-09-16T09:29:50.848 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.848 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.848 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:29:50.848 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.357521+0000", 2024-09-16T09:29:50.849 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0021074829999999998 2024-09-16T09:29:50.849 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.849 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.849 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:50.849 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.357578+0000", 2024-09-16T09:29:50.849 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7408000000000003e-05 2024-09-16T09:29:50.849 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.849 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.849 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:50.849 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.493988+0000", 2024-09-16T09:29:50.849 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.13640938399999999 2024-09-16T09:29:50.849 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.849 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.850 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:29:50.850 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.494061+0000", 2024-09-16T09:29:50.850 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.2917000000000007e-05 2024-09-16T09:29:50.850 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.850 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.850 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:29:50.850 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.494101+0000", 2024-09-16T09:29:50.850 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0315000000000003e-05 2024-09-16T09:29:50.850 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:29:50.850 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:29:50.850 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:29:50.850 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.850 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.851 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.24698.0:124205 7.8 7:1b181d5b:::benchmark_data_smithi070_78438_object124204:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-16T09:29:50.851 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:27:41.126806+0000", 2024-09-16T09:29:50.851 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 129.71088015999999, 2024-09-16T09:29:50.851 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.14566695599999999, 2024-09-16T09:29:50.851 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:29:50.851 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:29:50.851 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:29:50.851 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:29:50.851 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.24698", 2024-09-16T09:29:50.851 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:29:50.851 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 124205 2024-09-16T09:29:50.851 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.851 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:29:50.852 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.852 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:29:50.852 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.126806+0000", 2024-09-16T09:29:50.852 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:29:50.852 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.852 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.852 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:29:50.852 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.126806+0000", 2024-09-16T09:29:50.852 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:29:50.852 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.852 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.852 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:29:50.852 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.126807+0000", 2024-09-16T09:29:50.853 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.02e-06 2024-09-16T09:29:50.853 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.853 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.853 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:29:50.853 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.126827+0000", 2024-09-16T09:29:50.853 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0109e-05 2024-09-16T09:29:50.853 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.853 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.853 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:29:50.853 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.126839+0000", 2024-09-16T09:29:50.853 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1562999999999999e-05 2024-09-16T09:29:50.853 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.853 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.854 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:29:50.854 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.126846+0000", 2024-09-16T09:29:50.854 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.1049999999999997e-06 2024-09-16T09:29:50.854 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.854 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.854 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:29:50.854 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.134161+0000", 2024-09-16T09:29:50.854 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0073147730000000001 2024-09-16T09:29:50.854 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.854 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.854 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:29:50.854 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.256083+0000", 2024-09-16T09:29:50.854 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.12192246700000001 2024-09-16T09:29:50.854 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.855 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.855 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 4,7", 2024-09-16T09:29:50.855 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.257668+0000", 2024-09-16T09:29:50.855 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0015851280000000001 2024-09-16T09:29:50.855 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.855 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.855 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:50.855 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.261746+0000", 2024-09-16T09:29:50.855 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0040775999999999998 2024-09-16T09:29:50.856 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.856 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.856 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:50.856 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.268846+0000", 2024-09-16T09:29:50.856 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0070999050000000001 2024-09-16T09:29:50.856 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.856 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.856 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:29:50.856 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.272337+0000", 2024-09-16T09:29:50.856 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.003490922 2024-09-16T09:29:50.856 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.856 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.856 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:29:50.856 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.272430+0000", 2024-09-16T09:29:50.856 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.3037999999999999e-05 2024-09-16T09:29:50.857 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.857 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.857 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:29:50.857 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.272473+0000", 2024-09-16T09:29:50.857 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.3325999999999998e-05 2024-09-16T09:29:50.857 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:29:50.857 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:29:50.857 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:29:50.857 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.857 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.857 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.24698.0:124212 7.8 7:178dba36:::benchmark_data_smithi070_78438_object124211:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-16T09:29:50.857 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:27:41.131813+0000", 2024-09-16T09:29:50.857 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 129.70587282400001, 2024-09-16T09:29:50.858 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.14434644899999999, 2024-09-16T09:29:50.858 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:29:50.858 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:29:50.858 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:29:50.858 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:29:50.860 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.24698", 2024-09-16T09:29:50.860 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:29:50.860 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 124212 2024-09-16T09:29:50.860 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.860 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:29:50.860 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.860 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:29:50.860 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.131813+0000", 2024-09-16T09:29:50.860 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:29:50.860 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.860 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.861 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:29:50.861 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.131813+0000", 2024-09-16T09:29:50.861 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:29:50.861 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.861 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.861 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:29:50.861 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.131815+0000", 2024-09-16T09:29:50.861 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2720000000000001e-06 2024-09-16T09:29:50.861 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.861 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.862 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:29:50.862 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.131834+0000", 2024-09-16T09:29:50.862 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9828999999999999e-05 2024-09-16T09:29:50.862 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.862 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.862 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:29:50.862 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.131846+0000", 2024-09-16T09:29:50.862 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1256e-05 2024-09-16T09:29:50.862 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.862 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.862 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:29:50.862 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.131853+0000", 2024-09-16T09:29:50.862 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.8719999999999996e-06 2024-09-16T09:29:50.862 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.863 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.863 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:29:50.863 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.261784+0000", 2024-09-16T09:29:50.863 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.12993149800000001 2024-09-16T09:29:50.863 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.863 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.863 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:29:50.863 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.261920+0000", 2024-09-16T09:29:50.863 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000136367 2024-09-16T09:29:50.863 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.863 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.863 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 4,7", 2024-09-16T09:29:50.863 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.262081+0000", 2024-09-16T09:29:50.864 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000160493 2024-09-16T09:29:50.864 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.864 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.864 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:29:50.864 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.272481+0000", 2024-09-16T09:29:50.864 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.010399864999999999 2024-09-16T09:29:50.864 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.864 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.864 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:50.864 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.272526+0000", 2024-09-16T09:29:50.864 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5537000000000001e-05 2024-09-16T09:29:50.864 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.864 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.865 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:50.865 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.276017+0000", 2024-09-16T09:29:50.865 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.003490314 2024-09-16T09:29:50.865 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.865 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.865 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:29:50.865 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.276081+0000", 2024-09-16T09:29:50.865 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.4176000000000004e-05 2024-09-16T09:29:50.865 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.865 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.865 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:29:50.865 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.276160+0000", 2024-09-16T09:29:50.865 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.8969999999999998e-05 2024-09-16T09:29:50.865 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:29:50.865 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:29:50.865 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:29:50.866 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.866 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.866 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.24698.0:203427 7.e 7:71b09c45:::benchmark_data_smithi070_78438_object77296:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135)", 2024-09-16T09:29:50.866 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:28:51.971563+0000", 2024-09-16T09:29:50.866 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 58.866123612000003, 2024-09-16T09:29:50.866 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.15078747000000001, 2024-09-16T09:29:50.866 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:29:50.866 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:29:50.866 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:29:50.866 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:29:50.866 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.24698", 2024-09-16T09:29:50.866 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:29:50.866 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 203427 2024-09-16T09:29:50.866 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.867 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:29:50.867 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.867 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:29:50.867 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.971563+0000", 2024-09-16T09:29:50.867 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:29:50.867 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.867 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.867 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:29:50.867 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.971563+0000", 2024-09-16T09:29:50.867 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:29:50.867 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.867 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.867 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:29:50.867 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.971564+0000", 2024-09-16T09:29:50.867 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.339e-06 2024-09-16T09:29:50.868 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.868 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.868 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:29:50.868 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.971567+0000", 2024-09-16T09:29:50.868 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.512e-06 2024-09-16T09:29:50.868 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.868 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.868 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:29:50.868 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.971580+0000", 2024-09-16T09:29:50.868 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2958e-05 2024-09-16T09:29:50.868 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.868 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.868 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:29:50.868 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.971589+0000", 2024-09-16T09:29:50.869 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.7809999999999994e-06 2024-09-16T09:29:50.869 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.869 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.869 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:29:50.869 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.971702+0000", 2024-09-16T09:29:50.869 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000112611 2024-09-16T09:29:50.869 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.869 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.869 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:29:50.869 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.971839+0000", 2024-09-16T09:29:50.869 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000137298 2024-09-16T09:29:50.869 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.869 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.869 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 0,5", 2024-09-16T09:29:50.870 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.971954+0000", 2024-09-16T09:29:50.870 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00011476699999999999 2024-09-16T09:29:50.870 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.870 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.870 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:50.870 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.980144+0000", 2024-09-16T09:29:50.870 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0081902120000000005 2024-09-16T09:29:50.870 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.870 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.870 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:29:50.870 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.980166+0000", 2024-09-16T09:29:50.870 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.1750999999999999e-05 2024-09-16T09:29:50.870 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.870 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.870 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:50.870 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.122241+0000", 2024-09-16T09:29:50.871 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.142074912 2024-09-16T09:29:50.871 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.871 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.871 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:29:50.871 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.122303+0000", 2024-09-16T09:29:50.871 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.2509999999999996e-05 2024-09-16T09:29:50.871 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.871 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.871 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:29:50.871 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.122350+0000", 2024-09-16T09:29:50.871 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.6819e-05 2024-09-16T09:29:50.871 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:29:50.871 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:29:50.871 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:29:50.872 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.872 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.872 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.24698.0:203420 7.a 7:52851f87:::benchmark_data_smithi070_78438_object77289:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e136)", 2024-09-16T09:29:50.872 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:28:51.981780+0000", 2024-09-16T09:29:50.872 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 58.855906068000003, 2024-09-16T09:29:50.872 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.020255924, 2024-09-16T09:29:50.872 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:29:50.872 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:29:50.872 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T09:29:50.872 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:29:50.872 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.24698", 2024-09-16T09:29:50.872 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:29:50.872 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 203420 2024-09-16T09:29:50.873 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.873 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:29:50.873 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.873 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:29:50.873 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.981780+0000", 2024-09-16T09:29:50.873 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:29:50.873 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.873 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.873 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:29:50.873 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.981780+0000", 2024-09-16T09:29:50.873 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:29:50.873 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.874 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.874 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:29:50.874 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.981782+0000", 2024-09-16T09:29:50.874 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5400000000000001e-06 2024-09-16T09:29:50.874 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.874 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.874 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:29:50.874 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.981788+0000", 2024-09-16T09:29:50.874 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.0129999999999999e-06 2024-09-16T09:29:50.874 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.874 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.874 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:29:50.874 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.981802+0000", 2024-09-16T09:29:50.874 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3876000000000001e-05 2024-09-16T09:29:50.874 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.875 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.875 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:29:50.875 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.981810+0000", 2024-09-16T09:29:50.875 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.7709999999999995e-06 2024-09-16T09:29:50.875 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.875 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.875 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:29:50.875 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.982159+0000", 2024-09-16T09:29:50.875 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000348389 2024-09-16T09:29:50.875 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.875 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.875 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op must wait for map", 2024-09-16T09:29:50.875 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.982179+0000", 2024-09-16T09:29:50.876 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0489999999999999e-05 2024-09-16T09:29:50.876 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.876 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.876 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:29:50.876 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.985921+0000", 2024-09-16T09:29:50.876 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0037419480000000001 2024-09-16T09:29:50.876 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.876 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.876 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:29:50.876 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.985932+0000", 2024-09-16T09:29:50.876 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0929e-05 2024-09-16T09:29:50.876 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.876 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.876 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:29:50.877 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.001181+0000", 2024-09-16T09:29:50.877 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0152491459999999 2024-09-16T09:29:50.877 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.877 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.877 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:29:50.877 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.002036+0000", 2024-09-16T09:29:50.877 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00085482199999999998 2024-09-16T09:29:50.877 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:29:50.877 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:29:50.877 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:29:50.877 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.877 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.877 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.24698.0:203422 7.a 7:5a4cd988:::benchmark_data_smithi070_78438_object77291:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e136)", 2024-09-16T09:29:50.877 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:28:51.981838+0000", 2024-09-16T09:29:50.878 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 58.855848407000003, 2024-09-16T09:29:50.878 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.020646688, 2024-09-16T09:29:50.878 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:29:50.878 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:29:50.878 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T09:29:50.878 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:29:50.878 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.24698", 2024-09-16T09:29:50.878 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:29:50.878 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 203422 2024-09-16T09:29:50.878 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.878 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:29:50.878 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.878 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:29:50.878 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.981838+0000", 2024-09-16T09:29:50.878 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:29:50.878 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.879 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.879 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:29:50.879 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.981838+0000", 2024-09-16T09:29:50.879 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:29:50.879 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.879 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.879 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:29:50.879 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.981838+0000", 2024-09-16T09:29:50.879 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0500000000000004e-07 2024-09-16T09:29:50.879 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.879 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.879 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:29:50.879 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.981841+0000", 2024-09-16T09:29:50.879 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.061e-06 2024-09-16T09:29:50.879 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.879 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.880 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:29:50.880 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.981848+0000", 2024-09-16T09:29:50.880 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.8730000000000001e-06 2024-09-16T09:29:50.880 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.880 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.880 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:29:50.880 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.981854+0000", 2024-09-16T09:29:50.880 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.6879999999999996e-06 2024-09-16T09:29:50.880 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.880 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.880 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:29:50.880 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.982493+0000", 2024-09-16T09:29:50.880 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00063907000000000005 2024-09-16T09:29:50.880 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.880 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.881 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting_for_map not empty", 2024-09-16T09:29:50.881 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.982513+0000", 2024-09-16T09:29:50.881 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.965e-05 2024-09-16T09:29:50.881 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.881 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.881 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:29:50.881 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.986082+0000", 2024-09-16T09:29:50.881 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0035691189999999999 2024-09-16T09:29:50.881 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.881 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.881 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:29:50.881 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.986091+0000", 2024-09-16T09:29:50.881 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.9099999999999994e-06 2024-09-16T09:29:50.881 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.881 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.881 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:29:50.882 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.002421+0000", 2024-09-16T09:29:50.882 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0163305250000001 2024-09-16T09:29:50.882 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.882 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.882 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:29:50.882 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.002484+0000", 2024-09-16T09:29:50.882 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.3287000000000006e-05 2024-09-16T09:29:50.882 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:29:50.882 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:29:50.882 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:29:50.882 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.882 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.882 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.24698.0:203455 7.a 7:5b32df42:::benchmark_data_smithi070_78438_object77324:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136)", 2024-09-16T09:29:50.882 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:28:52.145758+0000", 2024-09-16T09:29:50.882 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 58.691927890999999, 2024-09-16T09:29:50.882 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.86280117499999998, 2024-09-16T09:29:50.883 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:29:50.883 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:29:50.883 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:29:50.883 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:29:50.883 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.24698", 2024-09-16T09:29:50.883 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:29:50.883 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 203455 2024-09-16T09:29:50.883 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.883 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:29:50.883 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.883 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:29:50.883 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.145758+0000", 2024-09-16T09:29:50.883 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:29:50.883 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.883 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.884 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:29:50.884 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.145758+0000", 2024-09-16T09:29:50.884 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:29:50.884 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.884 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.884 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:29:50.884 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.145760+0000", 2024-09-16T09:29:50.884 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.703e-06 2024-09-16T09:29:50.884 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.884 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.884 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:29:50.884 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.145767+0000", 2024-09-16T09:29:50.884 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.5590000000000002e-06 2024-09-16T09:29:50.884 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.884 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.885 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:29:50.885 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.145785+0000", 2024-09-16T09:29:50.885 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.8501000000000001e-05 2024-09-16T09:29:50.885 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.885 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.885 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:29:50.885 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.145793+0000", 2024-09-16T09:29:50.885 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.2830000000000001e-06 2024-09-16T09:29:50.885 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.885 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.885 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:29:50.885 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.145867+0000", 2024-09-16T09:29:50.885 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3374000000000006e-05 2024-09-16T09:29:50.885 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.886 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.886 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:29:50.886 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.145878+0000", 2024-09-16T09:29:50.886 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0942e-05 2024-09-16T09:29:50.886 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.886 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.886 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:29:50.886 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.002527+0000", 2024-09-16T09:29:50.886 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.85664891300000001 2024-09-16T09:29:50.886 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.886 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.886 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:29:50.886 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.002770+0000", 2024-09-16T09:29:50.886 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000243754 2024-09-16T09:29:50.886 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.887 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.887 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 3,5", 2024-09-16T09:29:50.887 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.002875+0000", 2024-09-16T09:29:50.887 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000105033 2024-09-16T09:29:50.887 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.887 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.887 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:50.887 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.008335+0000", 2024-09-16T09:29:50.887 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0054600509999999996 2024-09-16T09:29:50.887 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.887 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.887 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:29:50.887 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.008442+0000", 2024-09-16T09:29:50.887 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00010692 2024-09-16T09:29:50.888 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.888 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.888 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:50.888 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.008478+0000", 2024-09-16T09:29:50.888 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5590000000000003e-05 2024-09-16T09:29:50.888 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.888 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.888 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:29:50.888 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.008516+0000", 2024-09-16T09:29:50.888 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8565000000000001e-05 2024-09-16T09:29:50.888 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.888 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.888 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:29:50.888 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.008559+0000", 2024-09-16T09:29:50.888 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2987e-05 2024-09-16T09:29:50.889 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:29:50.889 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:29:50.889 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:29:50.889 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.889 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.889 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.24698.0:203461 7.a 7:5b0a38e6:::benchmark_data_smithi070_78438_object77330:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136)", 2024-09-16T09:29:50.889 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:28:52.150646+0000", 2024-09-16T09:29:50.889 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 58.687040285999998, 2024-09-16T09:29:50.889 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.85954066799999995, 2024-09-16T09:29:50.889 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:29:50.889 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:29:50.889 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:29:50.889 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:29:50.889 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.24698", 2024-09-16T09:29:50.890 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:29:50.890 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 203461 2024-09-16T09:29:50.890 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.890 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:29:50.890 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.890 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:29:50.890 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.150646+0000", 2024-09-16T09:29:50.890 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:29:50.890 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.890 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.890 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:29:50.890 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.150646+0000", 2024-09-16T09:29:50.890 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:29:50.890 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.891 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.891 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:29:50.891 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.150647+0000", 2024-09-16T09:29:50.891 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.1800000000000005e-07 2024-09-16T09:29:50.891 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.891 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.891 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:29:50.891 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.150666+0000", 2024-09-16T09:29:50.891 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.8927e-05 2024-09-16T09:29:50.891 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.891 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.891 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:29:50.891 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.150676+0000", 2024-09-16T09:29:50.891 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0113e-05 2024-09-16T09:29:50.892 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.892 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.892 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:29:50.892 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.150682+0000", 2024-09-16T09:29:50.892 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.3980000000000004e-06 2024-09-16T09:29:50.892 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.892 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.892 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:29:50.892 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.150779+0000", 2024-09-16T09:29:50.892 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.6989000000000003e-05 2024-09-16T09:29:50.892 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.892 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.892 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:29:50.893 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.150788+0000", 2024-09-16T09:29:50.893 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.4409999999999999e-06 2024-09-16T09:29:50.893 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.893 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.893 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:29:50.893 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.004849+0000", 2024-09-16T09:29:50.893 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.85406023600000003 2024-09-16T09:29:50.893 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.893 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.893 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:29:50.893 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.004962+0000", 2024-09-16T09:29:50.893 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00011376599999999999 2024-09-16T09:29:50.893 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.894 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.894 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 3,5", 2024-09-16T09:29:50.894 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.005070+0000", 2024-09-16T09:29:50.894 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000107599 2024-09-16T09:29:50.894 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.894 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.894 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:50.894 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.008390+0000", 2024-09-16T09:29:50.894 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0033195080000000001 2024-09-16T09:29:50.894 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.894 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.894 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:29:50.894 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.010026+0000", 2024-09-16T09:29:50.894 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001636167 2024-09-16T09:29:50.895 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.895 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.895 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:50.895 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.010073+0000", 2024-09-16T09:29:50.895 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.7425000000000001e-05 2024-09-16T09:29:50.895 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.895 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.895 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:29:50.895 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.010138+0000", 2024-09-16T09:29:50.895 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.4800000000000003e-05 2024-09-16T09:29:50.895 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.895 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.895 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:29:50.895 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.010187+0000", 2024-09-16T09:29:50.896 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.8581000000000001e-05 2024-09-16T09:29:50.896 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:29:50.896 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:29:50.896 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:29:50.896 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.896 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.896 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.24698.0:203478 7.a 7:5aee0767:::benchmark_data_smithi070_78438_object77347:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136)", 2024-09-16T09:29:50.896 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:28:52.164088+0000", 2024-09-16T09:29:50.896 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 58.673598544999997, 2024-09-16T09:29:50.896 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.85042330099999996, 2024-09-16T09:29:50.896 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:29:50.896 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:29:50.896 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:29:50.896 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:29:50.897 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.24698", 2024-09-16T09:29:50.897 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:29:50.897 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 203478 2024-09-16T09:29:50.897 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.897 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:29:50.897 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.897 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:29:50.897 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.164088+0000", 2024-09-16T09:29:50.897 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:29:50.897 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.897 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.897 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:29:50.897 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.164088+0000", 2024-09-16T09:29:50.897 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:29:50.898 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.898 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.898 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:29:50.898 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.164088+0000", 2024-09-16T09:29:50.898 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.9900000000000001e-07 2024-09-16T09:29:50.898 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.898 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.898 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:29:50.898 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.164091+0000", 2024-09-16T09:29:50.898 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.5469999999999999e-06 2024-09-16T09:29:50.898 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.898 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.898 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:29:50.898 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.164097+0000", 2024-09-16T09:29:50.899 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.4949999999999996e-06 2024-09-16T09:29:50.899 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.899 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.899 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:29:50.899 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.164103+0000", 2024-09-16T09:29:50.899 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4249999999999997e-06 2024-09-16T09:29:50.899 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.899 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.899 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:29:50.899 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.164297+0000", 2024-09-16T09:29:50.899 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00019394400000000001 2024-09-16T09:29:50.899 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.899 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.899 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:29:50.899 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.164307+0000", 2024-09-16T09:29:50.900 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0781000000000001e-05 2024-09-16T09:29:50.900 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.900 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.900 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:29:50.900 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.006832+0000", 2024-09-16T09:29:50.900 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.84252505899999997 2024-09-16T09:29:50.900 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.900 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.900 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:29:50.900 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.006941+0000", 2024-09-16T09:29:50.900 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00010823300000000001 2024-09-16T09:29:50.900 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.900 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.900 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 3,5", 2024-09-16T09:29:50.900 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.007051+0000", 2024-09-16T09:29:50.901 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000110491 2024-09-16T09:29:50.901 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.901 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.901 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:29:50.901 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.010031+0000", 2024-09-16T09:29:50.901 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0029797629999999999 2024-09-16T09:29:50.901 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.901 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.901 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:50.901 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.010244+0000", 2024-09-16T09:29:50.901 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00021353600000000001 2024-09-16T09:29:50.901 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.901 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.901 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:50.902 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.014448+0000", 2024-09-16T09:29:50.902 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0042034200000000002 2024-09-16T09:29:50.902 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.902 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.902 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:29:50.902 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.014481+0000", 2024-09-16T09:29:50.902 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3259000000000001e-05 2024-09-16T09:29:50.902 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.902 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.902 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:29:50.902 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.014511+0000", 2024-09-16T09:29:50.902 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.9848999999999998e-05 2024-09-16T09:29:50.902 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:29:50.902 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:29:50.903 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:29:50.903 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.903 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.903 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.24698.0:203490 7.a 7:5b988e00:::benchmark_data_smithi070_78438_object77359:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136)", 2024-09-16T09:29:50.903 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:28:52.173441+0000", 2024-09-16T09:29:50.903 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 58.664244674000003, 2024-09-16T09:29:50.903 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.84119520299999995, 2024-09-16T09:29:50.903 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:29:50.903 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:29:50.903 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:29:50.903 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:29:50.903 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.24698", 2024-09-16T09:29:50.903 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:29:50.903 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 203490 2024-09-16T09:29:50.904 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.904 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:29:50.904 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.904 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:29:50.904 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.173441+0000", 2024-09-16T09:29:50.904 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:29:50.904 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.904 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.904 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:29:50.904 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.173441+0000", 2024-09-16T09:29:50.904 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:29:50.904 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.904 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.904 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:29:50.904 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.173443+0000", 2024-09-16T09:29:50.905 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.426e-06 2024-09-16T09:29:50.905 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.905 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.905 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:29:50.905 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.173449+0000", 2024-09-16T09:29:50.905 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.6210000000000001e-06 2024-09-16T09:29:50.905 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.905 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.905 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:29:50.905 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.173464+0000", 2024-09-16T09:29:50.905 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5287000000000001e-05 2024-09-16T09:29:50.905 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.905 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.905 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:29:50.906 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.173471+0000", 2024-09-16T09:29:50.906 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.2910000000000002e-06 2024-09-16T09:29:50.906 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.906 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.906 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:29:50.906 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.173534+0000", 2024-09-16T09:29:50.906 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.3219000000000006e-05 2024-09-16T09:29:50.906 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.906 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.906 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:29:50.906 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.173547+0000", 2024-09-16T09:29:50.906 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2451e-05 2024-09-16T09:29:50.906 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.906 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.907 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:29:50.907 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.008597+0000", 2024-09-16T09:29:50.907 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.83504982800000005 2024-09-16T09:29:50.907 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.907 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.907 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:29:50.907 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.008699+0000", 2024-09-16T09:29:50.907 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000102577 2024-09-16T09:29:50.907 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.907 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.907 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 3,5", 2024-09-16T09:29:50.907 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.008804+0000", 2024-09-16T09:29:50.907 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000105265 2024-09-16T09:29:50.907 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.908 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.908 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:50.908 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.014398+0000", 2024-09-16T09:29:50.908 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0055940340000000003 2024-09-16T09:29:50.908 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.908 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.908 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:29:50.908 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.014414+0000", 2024-09-16T09:29:50.908 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5449999999999999e-05 2024-09-16T09:29:50.908 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.908 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.908 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:50.908 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.014556+0000", 2024-09-16T09:29:50.908 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000141786 2024-09-16T09:29:50.908 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.909 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.909 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:29:50.909 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.014601+0000", 2024-09-16T09:29:50.909 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5532999999999999e-05 2024-09-16T09:29:50.909 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.909 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.909 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:29:50.909 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.014637+0000", 2024-09-16T09:29:50.909 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5435000000000002e-05 2024-09-16T09:29:50.909 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:29:50.909 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:29:50.909 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:29:50.910 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.910 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.910 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.24698.0:203508 7.a 7:50f2d387:::benchmark_data_smithi070_78438_object77377:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136)", 2024-09-16T09:29:50.910 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:28:52.190869+0000", 2024-09-16T09:29:50.910 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 58.646817298000002, 2024-09-16T09:29:50.910 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.82400409200000002, 2024-09-16T09:29:50.910 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:29:50.910 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:29:50.910 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:29:50.910 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:29:50.910 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.24698", 2024-09-16T09:29:50.910 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:29:50.910 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 203508 2024-09-16T09:29:50.910 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.910 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:29:50.911 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.911 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:29:50.911 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.190869+0000", 2024-09-16T09:29:50.911 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:29:50.911 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.911 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.911 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:29:50.911 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.190869+0000", 2024-09-16T09:29:50.911 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:29:50.911 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.911 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.911 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:29:50.911 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.190870+0000", 2024-09-16T09:29:50.911 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5290000000000001e-06 2024-09-16T09:29:50.912 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.912 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.912 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:29:50.912 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.190876+0000", 2024-09-16T09:29:50.912 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4389999999999999e-06 2024-09-16T09:29:50.912 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.912 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.912 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:29:50.912 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.190887+0000", 2024-09-16T09:29:50.912 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0847e-05 2024-09-16T09:29:50.912 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.912 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.912 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:29:50.913 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.190893+0000", 2024-09-16T09:29:50.913 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.5100000000000004e-06 2024-09-16T09:29:50.913 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.913 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.913 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:29:50.913 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.191097+0000", 2024-09-16T09:29:50.913 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00020403700000000001 2024-09-16T09:29:50.913 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.913 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.913 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:29:50.913 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.191106+0000", 2024-09-16T09:29:50.913 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.7600000000000008e-06 2024-09-16T09:29:50.913 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.914 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.914 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:29:50.914 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.010277+0000", 2024-09-16T09:29:50.914 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.81917143299999995 2024-09-16T09:29:50.914 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.914 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.914 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:29:50.914 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.010378+0000", 2024-09-16T09:29:50.914 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000100559 2024-09-16T09:29:50.914 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.914 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.914 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 3,5", 2024-09-16T09:29:50.914 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.010474+0000", 2024-09-16T09:29:50.914 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.6433000000000003e-05 2024-09-16T09:29:50.914 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.915 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.915 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:29:50.915 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.014647+0000", 2024-09-16T09:29:50.915 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.004172111 2024-09-16T09:29:50.915 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.915 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.915 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:50.915 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.014703+0000", 2024-09-16T09:29:50.915 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.6196999999999997e-05 2024-09-16T09:29:50.915 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.915 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.915 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:50.915 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.014771+0000", 2024-09-16T09:29:50.915 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.8649e-05 2024-09-16T09:29:50.916 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.916 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.916 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:29:50.916 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.014833+0000", 2024-09-16T09:29:50.916 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1499000000000002e-05 2024-09-16T09:29:50.916 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.916 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.916 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:29:50.916 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.014873+0000", 2024-09-16T09:29:50.916 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0089000000000002e-05 2024-09-16T09:29:50.916 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:29:50.916 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:29:50.916 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:29:50.916 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.916 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.917 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.24698.0:203532 7.a 7:50322cd6:::benchmark_data_smithi070_78438_object77401:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136)", 2024-09-16T09:29:50.917 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:28:52.208817+0000", 2024-09-16T09:29:50.917 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 58.628868990999997, 2024-09-16T09:29:50.917 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.81510171200000003, 2024-09-16T09:29:50.917 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:29:50.917 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:29:50.917 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:29:50.917 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:29:50.917 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.24698", 2024-09-16T09:29:50.917 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:29:50.917 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 203532 2024-09-16T09:29:50.917 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.917 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:29:50.918 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.918 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:29:50.918 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.208817+0000", 2024-09-16T09:29:50.918 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:29:50.918 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.918 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.918 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:29:50.918 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.208817+0000", 2024-09-16T09:29:50.918 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:29:50.918 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.918 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.918 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:29:50.918 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.208819+0000", 2024-09-16T09:29:50.918 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6869999999999999e-06 2024-09-16T09:29:50.919 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.919 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.919 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:29:50.919 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.208825+0000", 2024-09-16T09:29:50.919 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.0510000000000002e-06 2024-09-16T09:29:50.919 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.919 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.919 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:29:50.919 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.208842+0000", 2024-09-16T09:29:50.919 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7286e-05 2024-09-16T09:29:50.919 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.919 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.919 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:29:50.919 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.208850+0000", 2024-09-16T09:29:50.919 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6669999999999996e-06 2024-09-16T09:29:50.920 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.920 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.920 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:29:50.920 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.208919+0000", 2024-09-16T09:29:50.920 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.9596000000000002e-05 2024-09-16T09:29:50.920 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.920 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.920 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:29:50.920 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.208932+0000", 2024-09-16T09:29:50.920 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2170999999999999e-05 2024-09-16T09:29:50.920 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.920 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.920 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:29:50.920 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.012264+0000", 2024-09-16T09:29:50.921 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.803332245 2024-09-16T09:29:50.921 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.921 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.921 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:29:50.921 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.012380+0000", 2024-09-16T09:29:50.921 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000116592 2024-09-16T09:29:50.921 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.921 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.921 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 3,5", 2024-09-16T09:29:50.921 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.012477+0000", 2024-09-16T09:29:50.921 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.6370000000000001e-05 2024-09-16T09:29:50.921 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.921 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.922 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:29:50.922 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.014652+0000", 2024-09-16T09:29:50.922 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0021752859999999998 2024-09-16T09:29:50.922 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.922 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.922 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:50.922 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.014915+0000", 2024-09-16T09:29:50.922 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00026334699999999998 2024-09-16T09:29:50.922 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.922 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.922 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:50.922 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.023829+0000", 2024-09-16T09:29:50.922 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0089137379999999992 2024-09-16T09:29:50.922 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.923 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.923 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:29:50.923 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.023879+0000", 2024-09-16T09:29:50.923 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.9595999999999997e-05 2024-09-16T09:29:50.923 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.923 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.923 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:29:50.923 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.023919+0000", 2024-09-16T09:29:50.923 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0080000000000003e-05 2024-09-16T09:29:50.923 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:29:50.923 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:29:50.923 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:29:50.923 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.923 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.923 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.24698.0:203542 7.a 7:54d67e0e:::benchmark_data_smithi070_78438_object77411:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136)", 2024-09-16T09:29:50.924 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:28:52.223327+0000", 2024-09-16T09:29:50.924 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 58.614359149999999, 2024-09-16T09:29:50.924 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.80535596300000001, 2024-09-16T09:29:50.924 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:29:50.924 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:29:50.924 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:29:50.924 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:29:50.924 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.24698", 2024-09-16T09:29:50.924 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:29:50.924 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 203542 2024-09-16T09:29:50.924 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.924 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:29:50.924 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.924 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:29:50.925 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.223327+0000", 2024-09-16T09:29:50.925 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:29:50.925 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.925 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.925 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:29:50.925 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.223327+0000", 2024-09-16T09:29:50.925 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:29:50.925 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.925 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.925 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:29:50.925 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.223328+0000", 2024-09-16T09:29:50.925 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.291e-06 2024-09-16T09:29:50.925 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.925 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.926 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:29:50.926 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.223333+0000", 2024-09-16T09:29:50.926 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.1789999999999999e-06 2024-09-16T09:29:50.926 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.926 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.926 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:29:50.926 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.223347+0000", 2024-09-16T09:29:50.926 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3696999999999999e-05 2024-09-16T09:29:50.926 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.926 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.926 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:29:50.926 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.223355+0000", 2024-09-16T09:29:50.926 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.486e-06 2024-09-16T09:29:50.926 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.926 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.927 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:29:50.927 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.223423+0000", 2024-09-16T09:29:50.927 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.8592999999999999e-05 2024-09-16T09:29:50.927 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.927 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.927 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:29:50.927 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.223433+0000", 2024-09-16T09:29:50.927 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.3219999999999999e-06 2024-09-16T09:29:50.927 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.927 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.928 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:50.929 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:29:50.929 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.014946+0000", 2024-09-16T09:29:50.929 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.79151325800000005 2024-09-16T09:29:50.929 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.929 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.929 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:29:50.929 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.015054+0000", 2024-09-16T09:29:50.929 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00010777600000000001 2024-09-16T09:29:50.929 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.929 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.929 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 3,5", 2024-09-16T09:29:50.930 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.015166+0000", 2024-09-16T09:29:50.930 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000111987 2024-09-16T09:29:50.930 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.930 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.930 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:50.930 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.023955+0000", 2024-09-16T09:29:50.930 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0087896369999999995 2024-09-16T09:29:50.930 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.930 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.930 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:50.930 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.028597+0000", 2024-09-16T09:29:50.930 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0046413829999999998 2024-09-16T09:29:50.931 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.931 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.931 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:29:50.931 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.028608+0000", 2024-09-16T09:29:50.931 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1442e-05 2024-09-16T09:29:50.931 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.931 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.931 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:29:50.931 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.028642+0000", 2024-09-16T09:29:50.931 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3893000000000002e-05 2024-09-16T09:29:50.931 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.931 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.931 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:29:50.931 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.028683+0000", 2024-09-16T09:29:50.932 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.1019000000000001e-05 2024-09-16T09:29:50.932 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:29:50.932 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:29:50.932 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:29:50.932 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.932 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.932 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.24698.0:203546 7.a 7:5c0c2c2e:::benchmark_data_smithi070_78438_object77415:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136)", 2024-09-16T09:29:50.932 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:28:52.223460+0000", 2024-09-16T09:29:50.932 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 58.614225812000001, 2024-09-16T09:29:50.932 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.80539693000000001, 2024-09-16T09:29:50.932 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:29:50.932 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:29:50.932 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:29:50.932 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:29:50.933 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.24698", 2024-09-16T09:29:50.933 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:29:50.933 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 203546 2024-09-16T09:29:50.933 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.933 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:29:50.933 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.933 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:29:50.933 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.223460+0000", 2024-09-16T09:29:50.933 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:29:50.933 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.933 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.933 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:29:50.933 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.223460+0000", 2024-09-16T09:29:50.933 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:29:50.934 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.934 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.934 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:29:50.934 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.223461+0000", 2024-09-16T09:29:50.934 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0380000000000001e-06 2024-09-16T09:29:50.934 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.934 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.934 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:29:50.934 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.223466+0000", 2024-09-16T09:29:50.934 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.9910000000000002e-06 2024-09-16T09:29:50.934 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.934 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.934 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:29:50.934 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.223480+0000", 2024-09-16T09:29:50.934 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3917000000000001e-05 2024-09-16T09:29:50.935 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.935 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.935 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:29:50.935 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.223487+0000", 2024-09-16T09:29:50.935 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.9840000000000004e-06 2024-09-16T09:29:50.935 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.935 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.935 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:29:50.935 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.224169+0000", 2024-09-16T09:29:50.935 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00068157799999999996 2024-09-16T09:29:50.935 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.935 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.935 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:29:50.935 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.224177+0000", 2024-09-16T09:29:50.935 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.1219999999999995e-06 2024-09-16T09:29:50.935 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.935 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.936 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:29:50.936 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.017181+0000", 2024-09-16T09:29:50.936 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.79300421899999995 2024-09-16T09:29:50.936 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.936 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.936 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:29:50.936 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.017275+0000", 2024-09-16T09:29:50.936 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.3654999999999997e-05 2024-09-16T09:29:50.936 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.936 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.936 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 3,5", 2024-09-16T09:29:50.936 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.017384+0000", 2024-09-16T09:29:50.936 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000108768 2024-09-16T09:29:50.936 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.936 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.937 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:29:50.937 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.028693+0000", 2024-09-16T09:29:50.937 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.011309817999999999 2024-09-16T09:29:50.937 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.937 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.937 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:50.937 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.028743+0000", 2024-09-16T09:29:50.937 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.9947000000000001e-05 2024-09-16T09:29:50.937 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.937 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.937 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:50.937 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.028784+0000", 2024-09-16T09:29:50.937 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.015e-05 2024-09-16T09:29:50.937 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.937 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.937 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:29:50.938 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.028823+0000", 2024-09-16T09:29:50.938 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9246999999999999e-05 2024-09-16T09:29:50.938 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.938 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.938 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:29:50.938 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.028857+0000", 2024-09-16T09:29:50.938 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4496000000000003e-05 2024-09-16T09:29:50.938 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:29:50.938 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:29:50.938 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:29:50.938 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.938 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.938 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.24698.0:203550 7.a 7:5010ec5d:::benchmark_data_smithi070_78438_object77419:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136)", 2024-09-16T09:29:50.938 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:28:52.230377+0000", 2024-09-16T09:29:50.938 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 58.607309503000003, 2024-09-16T09:29:50.939 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.80332031299999995, 2024-09-16T09:29:50.939 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:29:50.939 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:29:50.939 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:29:50.939 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:29:50.939 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.24698", 2024-09-16T09:29:50.939 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:29:50.939 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 203550 2024-09-16T09:29:50.939 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.939 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:29:50.939 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.939 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:29:50.939 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.230377+0000", 2024-09-16T09:29:50.939 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:29:50.939 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.940 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.940 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:29:50.940 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.230377+0000", 2024-09-16T09:29:50.940 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:29:50.940 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.940 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.940 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:29:50.940 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.230379+0000", 2024-09-16T09:29:50.940 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.148e-06 2024-09-16T09:29:50.940 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.940 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.940 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:29:50.940 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.230387+0000", 2024-09-16T09:29:50.940 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.3319999999999992e-06 2024-09-16T09:29:50.940 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.940 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.941 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:29:50.941 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.230410+0000", 2024-09-16T09:29:50.941 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.2585000000000001e-05 2024-09-16T09:29:50.941 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.941 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.941 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:29:50.941 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.230419+0000", 2024-09-16T09:29:50.941 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.516e-06 2024-09-16T09:29:50.941 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.941 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.941 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:29:50.941 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.230500+0000", 2024-09-16T09:29:50.941 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.0457999999999994e-05 2024-09-16T09:29:50.941 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.941 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.941 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:29:50.942 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.230511+0000", 2024-09-16T09:29:50.942 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1549000000000001e-05 2024-09-16T09:29:50.942 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.942 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.942 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:29:50.942 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.019416+0000", 2024-09-16T09:29:50.942 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.78890451800000005 2024-09-16T09:29:50.942 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.942 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.942 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:29:50.942 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.019508+0000", 2024-09-16T09:29:50.942 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.1932e-05 2024-09-16T09:29:50.942 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.942 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.943 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 3,5", 2024-09-16T09:29:50.943 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.019600+0000", 2024-09-16T09:29:50.943 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.2604999999999999e-05 2024-09-16T09:29:50.943 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.943 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.943 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:29:50.943 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.028698+0000", 2024-09-16T09:29:50.943 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.009097628 2024-09-16T09:29:50.943 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.943 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.943 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:50.943 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.033536+0000", 2024-09-16T09:29:50.943 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0048381350000000004 2024-09-16T09:29:50.943 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.943 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.944 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:50.944 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.033612+0000", 2024-09-16T09:29:50.944 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6386999999999999e-05 2024-09-16T09:29:50.944 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.944 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.944 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:29:50.944 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.033659+0000", 2024-09-16T09:29:50.944 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.6350000000000002e-05 2024-09-16T09:29:50.944 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.944 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.944 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:29:50.944 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.033697+0000", 2024-09-16T09:29:50.944 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8170000000000002e-05 2024-09-16T09:29:50.944 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:29:50.945 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:29:50.945 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:29:50.945 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.945 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.945 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.24698.0:203558 7.a 7:5a81c49c:::benchmark_data_smithi070_78438_object77427:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136)", 2024-09-16T09:29:50.945 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:28:52.238828+0000", 2024-09-16T09:29:50.945 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 58.598858305, 2024-09-16T09:29:50.945 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.794966693, 2024-09-16T09:29:50.945 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:29:50.945 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:29:50.945 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:29:50.945 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:29:50.945 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.24698", 2024-09-16T09:29:50.945 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:29:50.945 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 203558 2024-09-16T09:29:50.945 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.946 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:29:50.946 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.946 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:29:50.946 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.238828+0000", 2024-09-16T09:29:50.946 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:29:50.946 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.946 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.946 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:29:50.946 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.238828+0000", 2024-09-16T09:29:50.946 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:29:50.946 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.946 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.946 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:29:50.946 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.238829+0000", 2024-09-16T09:29:50.946 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1880000000000001e-06 2024-09-16T09:29:50.947 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.947 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.947 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:29:50.947 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.238833+0000", 2024-09-16T09:29:50.947 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.1760000000000003e-06 2024-09-16T09:29:50.947 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.947 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.947 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:29:50.947 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.238844+0000", 2024-09-16T09:29:50.947 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1172000000000001e-05 2024-09-16T09:29:50.947 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.947 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.947 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:29:50.947 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.238852+0000", 2024-09-16T09:29:50.948 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3170000000000004e-06 2024-09-16T09:29:50.948 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.948 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.948 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:29:50.948 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.238911+0000", 2024-09-16T09:29:50.948 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.9190000000000001e-05 2024-09-16T09:29:50.948 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.948 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.948 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:29:50.948 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.238921+0000", 2024-09-16T09:29:50.948 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0129000000000001e-05 2024-09-16T09:29:50.948 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.948 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.948 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:29:50.949 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.021618+0000", 2024-09-16T09:29:50.949 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.78269724100000004 2024-09-16T09:29:50.949 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.949 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.949 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:29:50.949 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.021719+0000", 2024-09-16T09:29:50.949 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00010029100000000001 2024-09-16T09:29:50.949 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.949 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.949 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 3,5", 2024-09-16T09:29:50.949 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.021818+0000", 2024-09-16T09:29:50.950 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.9066000000000003e-05 2024-09-16T09:29:50.950 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.950 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.950 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:29:50.950 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.028703+0000", 2024-09-16T09:29:50.950 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0068851210000000001 2024-09-16T09:29:50.950 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.950 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.950 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:50.950 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.033575+0000", 2024-09-16T09:29:50.950 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0048726500000000001 2024-09-16T09:29:50.950 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.950 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.951 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:50.951 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.033729+0000", 2024-09-16T09:29:50.951 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00015408800000000001 2024-09-16T09:29:50.951 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.951 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.951 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:29:50.951 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.033766+0000", 2024-09-16T09:29:50.951 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6412999999999998e-05 2024-09-16T09:29:50.951 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.951 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.951 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:29:50.951 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.033795+0000", 2024-09-16T09:29:50.951 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.8651000000000001e-05 2024-09-16T09:29:50.951 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:29:50.951 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:29:50.952 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:29:50.952 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.952 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.952 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.24698.0:203641 7.a 7:5c484e8b:::benchmark_data_smithi070_78438_object77510:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136)", 2024-09-16T09:29:50.952 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:28:52.342560+0000", 2024-09-16T09:29:50.952 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 58.495126583999998, 2024-09-16T09:29:50.952 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.69139150400000005, 2024-09-16T09:29:50.952 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:29:50.952 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:29:50.952 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:29:50.952 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:29:50.952 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.24698", 2024-09-16T09:29:50.952 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:29:50.952 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 203641 2024-09-16T09:29:50.953 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.953 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:29:50.953 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.953 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:29:50.953 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.342560+0000", 2024-09-16T09:29:50.953 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:29:50.953 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.953 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.953 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:29:50.953 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.342560+0000", 2024-09-16T09:29:50.953 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:29:50.953 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.953 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.953 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:29:50.954 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.342561+0000", 2024-09-16T09:29:50.954 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.285e-06 2024-09-16T09:29:50.954 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.954 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.954 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:29:50.954 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.342566+0000", 2024-09-16T09:29:50.954 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.972e-06 2024-09-16T09:29:50.954 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.954 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.954 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:29:50.954 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.342577+0000", 2024-09-16T09:29:50.954 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0961999999999999e-05 2024-09-16T09:29:50.954 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.954 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.954 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:29:50.955 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.342585+0000", 2024-09-16T09:29:50.955 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.7919999999999993e-06 2024-09-16T09:29:50.955 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.955 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.955 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:29:50.955 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.342652+0000", 2024-09-16T09:29:50.955 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.7025000000000003e-05 2024-09-16T09:29:50.955 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.955 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.955 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:29:50.955 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.342663+0000", 2024-09-16T09:29:50.955 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1562999999999999e-05 2024-09-16T09:29:50.955 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.955 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.955 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:29:50.956 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.023982+0000", 2024-09-16T09:29:50.956 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.68131918899999999 2024-09-16T09:29:50.956 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.956 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.956 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:29:50.956 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.024072+0000", 2024-09-16T09:29:50.956 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.9392000000000001e-05 2024-09-16T09:29:50.956 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.956 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.956 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 3,5", 2024-09-16T09:29:50.956 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.024161+0000", 2024-09-16T09:29:50.956 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.9412000000000004e-05 2024-09-16T09:29:50.956 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.956 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.956 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:29:50.957 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.028707+0000", 2024-09-16T09:29:50.957 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.004546161 2024-09-16T09:29:50.957 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.957 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.957 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:50.957 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.033826+0000", 2024-09-16T09:29:50.957 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0051183080000000002 2024-09-16T09:29:50.957 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.957 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.957 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:50.957 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.033887+0000", 2024-09-16T09:29:50.957 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1198999999999995e-05 2024-09-16T09:29:50.957 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.957 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.958 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:29:50.958 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.033923+0000", 2024-09-16T09:29:50.958 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6049e-05 2024-09-16T09:29:50.958 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.958 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.958 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:29:50.958 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.033951+0000", 2024-09-16T09:29:50.958 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.8195e-05 2024-09-16T09:29:50.958 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:29:50.958 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:29:50.958 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:29:50.958 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.958 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.958 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.24698.0:203652 7.a 7:5453db86:::benchmark_data_smithi070_78438_object77521:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136)", 2024-09-16T09:29:50.959 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:28:52.357854+0000", 2024-09-16T09:29:50.959 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 58.479831773999997, 2024-09-16T09:29:50.959 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.67619149199999995, 2024-09-16T09:29:50.959 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:29:50.959 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:29:50.959 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:29:50.959 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:29:50.959 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.24698", 2024-09-16T09:29:50.959 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:29:50.959 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 203652 2024-09-16T09:29:50.959 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.959 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:29:50.959 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.959 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:29:50.960 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.357854+0000", 2024-09-16T09:29:50.960 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:29:50.960 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.960 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.960 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:29:50.960 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.357854+0000", 2024-09-16T09:29:50.960 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:29:50.960 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.960 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.960 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:29:50.960 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.357855+0000", 2024-09-16T09:29:50.960 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.023e-06 2024-09-16T09:29:50.960 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.960 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.960 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:29:50.961 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.357860+0000", 2024-09-16T09:29:50.961 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2939999999999999e-06 2024-09-16T09:29:50.961 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.961 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.961 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:29:50.961 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.357869+0000", 2024-09-16T09:29:50.961 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.1030000000000008e-06 2024-09-16T09:29:50.961 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.961 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.961 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:29:50.961 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.357874+0000", 2024-09-16T09:29:50.962 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0370000000000002e-06 2024-09-16T09:29:50.962 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.962 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.962 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:29:50.962 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.357930+0000", 2024-09-16T09:29:50.962 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.6468999999999998e-05 2024-09-16T09:29:50.962 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.962 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.962 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:29:50.962 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.357940+0000", 2024-09-16T09:29:50.962 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.5850000000000006e-06 2024-09-16T09:29:50.962 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.963 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.963 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:29:50.963 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.026230+0000", 2024-09-16T09:29:50.963 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.66828984300000005 2024-09-16T09:29:50.963 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.963 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.963 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:29:50.963 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.026333+0000", 2024-09-16T09:29:50.963 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00010361099999999999 2024-09-16T09:29:50.963 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.963 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.963 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 3,5", 2024-09-16T09:29:50.963 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.026438+0000", 2024-09-16T09:29:50.963 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00010440700000000001 2024-09-16T09:29:50.964 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.964 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.964 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:29:50.964 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.033500+0000", 2024-09-16T09:29:50.964 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0070627210000000001 2024-09-16T09:29:50.964 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.964 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.964 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:50.964 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.033857+0000", 2024-09-16T09:29:50.964 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00035613700000000002 2024-09-16T09:29:50.964 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.964 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.964 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:50.965 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.033982+0000", 2024-09-16T09:29:50.965 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00012542900000000001 2024-09-16T09:29:50.965 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.965 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.965 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:29:50.965 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.034018+0000", 2024-09-16T09:29:50.965 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5757999999999999e-05 2024-09-16T09:29:50.965 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.965 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.965 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:29:50.965 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.034046+0000", 2024-09-16T09:29:50.965 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.8075000000000001e-05 2024-09-16T09:29:50.965 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:29:50.965 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:29:50.966 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:29:50.966 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.966 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.966 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.24698.0:203750 7.a 7:52e49071:::benchmark_data_smithi070_78438_object77619:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136)", 2024-09-16T09:29:50.966 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:28:52.581478+0000", 2024-09-16T09:29:50.966 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 58.256207959000001, 2024-09-16T09:29:50.966 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.45793680799999997, 2024-09-16T09:29:50.966 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:29:50.966 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:29:50.966 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:29:50.966 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:29:50.966 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.24698", 2024-09-16T09:29:50.966 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:29:50.966 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 203750 2024-09-16T09:29:50.966 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.967 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:29:50.967 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.967 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:29:50.967 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.581478+0000", 2024-09-16T09:29:50.967 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:29:50.967 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.967 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.967 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:29:50.967 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.581478+0000", 2024-09-16T09:29:50.967 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:29:50.967 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.967 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.967 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:29:50.967 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.581480+0000", 2024-09-16T09:29:50.968 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.08e-06 2024-09-16T09:29:50.968 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.968 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.968 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:29:50.968 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.581486+0000", 2024-09-16T09:29:50.968 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8279999999999999e-06 2024-09-16T09:29:50.968 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.968 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.968 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:29:50.968 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.581502+0000", 2024-09-16T09:29:50.968 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5852999999999999e-05 2024-09-16T09:29:50.968 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.968 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.968 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:29:50.968 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.581510+0000", 2024-09-16T09:29:50.969 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.0700000000000007e-06 2024-09-16T09:29:50.969 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.969 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.969 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:29:50.969 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.581576+0000", 2024-09-16T09:29:50.969 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.6338000000000001e-05 2024-09-16T09:29:50.969 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.969 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.969 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:29:50.969 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.581587+0000", 2024-09-16T09:29:50.969 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0725e-05 2024-09-16T09:29:50.969 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.969 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.969 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:29:50.970 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.028904+0000", 2024-09-16T09:29:50.970 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.44731681699999998 2024-09-16T09:29:50.970 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.970 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.970 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:29:50.970 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.029003+0000", 2024-09-16T09:29:50.970 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.8913000000000007e-05 2024-09-16T09:29:50.970 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.970 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.970 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 3,5", 2024-09-16T09:29:50.970 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.029098+0000", 2024-09-16T09:29:50.970 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.5378999999999996e-05 2024-09-16T09:29:50.970 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.970 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.971 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:29:50.971 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.033546+0000", 2024-09-16T09:29:50.971 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0044476680000000001 2024-09-16T09:29:50.971 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.971 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.971 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:50.971 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.034091+0000", 2024-09-16T09:29:50.971 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00054530099999999999 2024-09-16T09:29:50.971 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.971 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.971 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:50.971 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.039290+0000", 2024-09-16T09:29:50.971 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0051983799999999998 2024-09-16T09:29:50.971 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.971 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.972 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:29:50.972 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.039347+0000", 2024-09-16T09:29:50.972 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7201999999999999e-05 2024-09-16T09:29:50.972 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.972 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.972 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:29:50.972 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.039415+0000", 2024-09-16T09:29:50.972 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.8254000000000001e-05 2024-09-16T09:29:50.972 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:29:50.972 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:29:50.972 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:29:50.972 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.972 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.972 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.24698.0:203753 7.a 7:5a1d367b:::benchmark_data_smithi070_78438_object77622:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136)", 2024-09-16T09:29:50.973 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:28:52.587237+0000", 2024-09-16T09:29:50.973 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 58.250449248000002, 2024-09-16T09:29:50.973 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.45243622999999999, 2024-09-16T09:29:50.973 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:29:50.973 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:29:50.973 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:29:50.973 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:29:50.973 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.24698", 2024-09-16T09:29:50.973 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:29:50.973 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 203753 2024-09-16T09:29:50.973 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.973 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:29:50.973 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.973 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:29:50.974 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.587237+0000", 2024-09-16T09:29:50.974 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:29:50.974 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.974 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.974 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:29:50.974 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.587237+0000", 2024-09-16T09:29:50.974 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:29:50.974 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.974 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.974 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:29:50.974 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.587238+0000", 2024-09-16T09:29:50.974 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2139999999999999e-06 2024-09-16T09:29:50.974 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.974 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.974 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:29:50.975 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.587242+0000", 2024-09-16T09:29:50.975 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2259999999999999e-06 2024-09-16T09:29:50.975 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.975 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.975 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:29:50.975 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.587254+0000", 2024-09-16T09:29:50.975 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2069000000000001e-05 2024-09-16T09:29:50.975 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.975 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.975 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:29:50.975 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.587260+0000", 2024-09-16T09:29:50.975 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.835e-06 2024-09-16T09:29:50.975 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.975 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.975 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:29:50.976 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.587318+0000", 2024-09-16T09:29:50.976 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7497000000000002e-05 2024-09-16T09:29:50.976 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.976 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.976 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:29:50.976 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.587329+0000", 2024-09-16T09:29:50.976 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1158000000000001e-05 2024-09-16T09:29:50.976 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.976 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.976 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:29:50.976 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.031439+0000", 2024-09-16T09:29:50.976 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.444110216 2024-09-16T09:29:50.976 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.976 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.976 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:29:50.977 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.031537+0000", 2024-09-16T09:29:50.977 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.7628999999999996e-05 2024-09-16T09:29:50.977 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.977 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.977 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 3,5", 2024-09-16T09:29:50.977 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.031628+0000", 2024-09-16T09:29:50.977 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.0997999999999998e-05 2024-09-16T09:29:50.977 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.977 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.977 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:29:50.977 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.039223+0000", 2024-09-16T09:29:50.977 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0075953080000000003 2024-09-16T09:29:50.977 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.978 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.978 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:50.978 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.039460+0000", 2024-09-16T09:29:50.978 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00023686599999999999 2024-09-16T09:29:50.978 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.978 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.978 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:29:50.978 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.039544+0000", 2024-09-16T09:29:50.978 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.3762000000000002e-05 2024-09-16T09:29:50.978 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.978 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.978 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:29:50.978 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.039636+0000", 2024-09-16T09:29:50.978 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.1879000000000006e-05 2024-09-16T09:29:50.978 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:29:50.979 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:29:50.979 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:29:50.979 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.039673+0000", 2024-09-16T09:29:50.979 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7573000000000001e-05 2024-09-16T09:29:50.979 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:29:50.979 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:29:50.979 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:29:50.979 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:29:50.979 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:29:50.979 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T09:29:50.979 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:50.980 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:29:50.980 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:50 smithi084 ceph-mon[32629]: 6.17 scrub starts 2024-09-16T09:29:50.980 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:50 smithi084 ceph-mon[32629]: 6.17 scrub ok 2024-09-16T09:29:50.980 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:50 smithi084 ceph-mon[32629]: pgmap v212: 158 pgs: 16 unknown, 142 active+clean; 583 KiB data, 999 MiB used, 714 GiB / 715 GiB avail; 805 B/s rd, 0 op/s 2024-09-16T09:29:50.980 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:50 smithi084 ceph-mon[32629]: osdmap e154: 8 total, 8 up, 8 in 2024-09-16T09:29:51.081 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:29:51.102 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:51.103 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:51.104 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:29:51.204 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:29:51.226 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:51.226 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:51.227 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:29:51.327 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:29:51.348 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:51.349 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:51.349 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:29:51.450 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:29:51.477 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:51.478 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:51.478 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:29:51.579 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:29:51.601 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:51.602 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:51.602 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:29:51.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:51 smithi070 ceph-mon[30388]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T09:29:51.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:51 smithi070 ceph-mon[30388]: 6.19 scrub starts 2024-09-16T09:29:51.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:51 smithi070 ceph-mon[30388]: 6.19 scrub ok 2024-09-16T09:29:51.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:51 smithi070 ceph-mon[30388]: 5.1 scrub starts 2024-09-16T09:29:51.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:51 smithi070 ceph-mon[30388]: 5.1 scrub ok 2024-09-16T09:29:51.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:51 smithi070 ceph-mon[30388]: 3.0 scrub starts 2024-09-16T09:29:51.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:51 smithi070 ceph-mon[30388]: 3.0 scrub ok 2024-09-16T09:29:51.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:51 smithi070 ceph-mon[30388]: 3.1b scrub starts 2024-09-16T09:29:51.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:51 smithi070 ceph-mon[30388]: 3.1b scrub ok 2024-09-16T09:29:51.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:51 smithi070 ceph-mon[30388]: 3.d scrub starts 2024-09-16T09:29:51.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:51 smithi070 ceph-mon[30388]: 3.d scrub ok 2024-09-16T09:29:51.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:51 smithi070 ceph-mon[40691]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T09:29:51.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:51 smithi070 ceph-mon[40691]: 6.19 scrub starts 2024-09-16T09:29:51.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:51 smithi070 ceph-mon[40691]: 6.19 scrub ok 2024-09-16T09:29:51.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:51 smithi070 ceph-mon[40691]: 5.1 scrub starts 2024-09-16T09:29:51.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:51 smithi070 ceph-mon[40691]: 5.1 scrub ok 2024-09-16T09:29:51.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:51 smithi070 ceph-mon[40691]: 3.0 scrub starts 2024-09-16T09:29:51.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:51 smithi070 ceph-mon[40691]: 3.0 scrub ok 2024-09-16T09:29:51.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:51 smithi070 ceph-mon[40691]: 3.1b scrub starts 2024-09-16T09:29:51.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:51 smithi070 ceph-mon[40691]: 3.1b scrub ok 2024-09-16T09:29:51.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:51 smithi070 ceph-mon[40691]: 3.d scrub starts 2024-09-16T09:29:51.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:51 smithi070 ceph-mon[40691]: 3.d scrub ok 2024-09-16T09:29:51.703 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:29:51.730 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:51.731 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:51.731 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:29:51.832 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:29:51.853 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:51.854 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:51.854 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:29:51.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:51 smithi084 ceph-mon[32629]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T09:29:51.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:51 smithi084 ceph-mon[32629]: 6.19 scrub starts 2024-09-16T09:29:51.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:51 smithi084 ceph-mon[32629]: 6.19 scrub ok 2024-09-16T09:29:51.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:51 smithi084 ceph-mon[32629]: 5.1 scrub starts 2024-09-16T09:29:51.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:51 smithi084 ceph-mon[32629]: 5.1 scrub ok 2024-09-16T09:29:51.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:51 smithi084 ceph-mon[32629]: 3.0 scrub starts 2024-09-16T09:29:51.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:51 smithi084 ceph-mon[32629]: 3.0 scrub ok 2024-09-16T09:29:51.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:51 smithi084 ceph-mon[32629]: 3.1b scrub starts 2024-09-16T09:29:51.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:51 smithi084 ceph-mon[32629]: 3.1b scrub ok 2024-09-16T09:29:51.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:51 smithi084 ceph-mon[32629]: 3.d scrub starts 2024-09-16T09:29:51.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:51 smithi084 ceph-mon[32629]: 3.d scrub ok 2024-09-16T09:29:51.954 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:29:51.976 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:51.977 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:51.977 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:29:52.022 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:29:51 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:29:51] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:29:52.049 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.3 dump_ops_in_flight 2024-09-16T09:29:52.078 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:29:52.104 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:52.104 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:52.105 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:29:52.198 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:29:52.205 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:29:52.205 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:29:52.231 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:52.231 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:52.232 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:29:52.332 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:29:52.358 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:52.359 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:52.359 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:29:52.460 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:29:52.486 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:52.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:52.487 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:29:52.587 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:29:52.609 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:52.610 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:52.610 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:29:52.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:52 smithi070 ceph-mon[30388]: 3.14 scrub starts 2024-09-16T09:29:52.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:52 smithi070 ceph-mon[30388]: 3.14 scrub ok 2024-09-16T09:29:52.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:52 smithi070 ceph-mon[30388]: pgmap v214: 158 pgs: 16 unknown, 142 active+clean; 583 KiB data, 999 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:29:52.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:52 smithi070 ceph-mon[40691]: 3.14 scrub starts 2024-09-16T09:29:52.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:52 smithi070 ceph-mon[40691]: 3.14 scrub ok 2024-09-16T09:29:52.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:52 smithi070 ceph-mon[40691]: pgmap v214: 158 pgs: 16 unknown, 142 active+clean; 583 KiB data, 999 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:29:52.711 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:29:52.732 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:52.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:52.733 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:29:52.773 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:52 smithi084 ceph-mon[32629]: 3.14 scrub starts 2024-09-16T09:29:52.773 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:52 smithi084 ceph-mon[32629]: 3.14 scrub ok 2024-09-16T09:29:52.773 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:52 smithi084 ceph-mon[32629]: pgmap v214: 158 pgs: 16 unknown, 142 active+clean; 583 KiB data, 999 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:29:52.834 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:29:52.856 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:52.856 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:52.857 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:29:52.957 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:29:52.978 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:52.979 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:52.979 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:29:53.080 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:29:53.106 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:53.107 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:53.107 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:29:53.137 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:29:53.138 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:29:53.160 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:29:52 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:29:53.161 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:29:53.161 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:29:53.163 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:29:53.182 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:29:53.183 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:29:53.200 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:29:53.200 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:29:53.209 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:29:53.217 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:29:53.218 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:29:53.230 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:53.231 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:53.231 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:29:53.236 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:29:53.236 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:29:53.253 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:29:53.253 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:29:53.269 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:29:53.270 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:29:53.332 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:29:53.356 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:53.357 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:53.357 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:29:53.458 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:29:53.474 INFO:teuthology.orchestra.run.smithi070.stderr:enabled application 'rados' on pool 'unique_pool_1' 2024-09-16T09:29:53.484 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:53.484 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:53.485 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:29:53.586 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:29:53.610 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:53.611 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:53.611 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:29:53.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:53 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:29:53.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:53 smithi070 ceph-mon[30388]: 6.16 scrub starts 2024-09-16T09:29:53.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:53 smithi070 ceph-mon[30388]: 6.16 scrub ok 2024-09-16T09:29:53.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:53 smithi070 ceph-mon[30388]: 3.e scrub starts 2024-09-16T09:29:53.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:53 smithi070 ceph-mon[30388]: 3.e scrub ok 2024-09-16T09:29:53.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:53 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2547806926' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "unique_pool_1", "app": "rados", "yes_i_really_mean_it": true} : dispatch 2024-09-16T09:29:53.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:53 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1796864498' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:29:53.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:53 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:29:53.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:53 smithi070 ceph-mon[40691]: 6.16 scrub starts 2024-09-16T09:29:53.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:53 smithi070 ceph-mon[40691]: 6.16 scrub ok 2024-09-16T09:29:53.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:53 smithi070 ceph-mon[40691]: 3.e scrub starts 2024-09-16T09:29:53.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:53 smithi070 ceph-mon[40691]: 3.e scrub ok 2024-09-16T09:29:53.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:53 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/2547806926' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "unique_pool_1", "app": "rados", "yes_i_really_mean_it": true} : dispatch 2024-09-16T09:29:53.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:53 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1796864498' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:29:53.713 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:29:53.739 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:53.740 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:53.740 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:29:53.841 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:29:53.863 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:53.863 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:53.864 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:29:53.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:53 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:29:53.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:53 smithi084 ceph-mon[32629]: 6.16 scrub starts 2024-09-16T09:29:53.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:53 smithi084 ceph-mon[32629]: 6.16 scrub ok 2024-09-16T09:29:53.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:53 smithi084 ceph-mon[32629]: 3.e scrub starts 2024-09-16T09:29:53.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:53 smithi084 ceph-mon[32629]: 3.e scrub ok 2024-09-16T09:29:53.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:53 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/2547806926' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "unique_pool_1", "app": "rados", "yes_i_really_mean_it": true} : dispatch 2024-09-16T09:29:53.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:53 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1796864498' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:29:53.964 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:29:53.986 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:53.987 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:53.987 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:29:54.088 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:29:54.110 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:54.111 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:54.111 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:29:54.204 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-16T09:29:54.212 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:29:54.238 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:54.239 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:54.239 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:29:54.340 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:29:54.361 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:54.362 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:54.362 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:29:54.463 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:29:54.491 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:54.491 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:54.492 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:29:54.592 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:29:54.614 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:54.615 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:54.615 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:29:54.716 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:29:54.742 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:54.743 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:54.743 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:29:54.844 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:29:54.861 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:54 smithi070 ceph-mon[30388]: 5.11 scrub starts 2024-09-16T09:29:54.861 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:54 smithi070 ceph-mon[30388]: 5.11 scrub ok 2024-09-16T09:29:54.861 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:54 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2547806926' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_1", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-16T09:29:54.861 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:54 smithi070 ceph-mon[30388]: osdmap e155: 8 total, 8 up, 8 in 2024-09-16T09:29:54.861 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:54 smithi070 ceph-mon[30388]: pgmap v216: 158 pgs: 158 active+clean; 583 KiB data, 1002 MiB used, 714 GiB / 715 GiB avail; 997 B/s rd, 0 op/s 2024-09-16T09:29:54.862 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:54 smithi070 ceph-mon[40691]: 5.11 scrub starts 2024-09-16T09:29:54.862 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:54 smithi070 ceph-mon[40691]: 5.11 scrub ok 2024-09-16T09:29:54.862 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:54 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/2547806926' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_1", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-16T09:29:54.862 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:54 smithi070 ceph-mon[40691]: osdmap e155: 8 total, 8 up, 8 in 2024-09-16T09:29:54.862 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:54 smithi070 ceph-mon[40691]: pgmap v216: 158 pgs: 158 active+clean; 583 KiB data, 1002 MiB used, 714 GiB / 715 GiB avail; 997 B/s rd, 0 op/s 2024-09-16T09:29:54.870 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:54.870 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:54.871 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:29:54.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:54 smithi084 ceph-mon[32629]: 5.11 scrub starts 2024-09-16T09:29:54.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:54 smithi084 ceph-mon[32629]: 5.11 scrub ok 2024-09-16T09:29:54.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:54 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/2547806926' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_1", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-16T09:29:54.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:54 smithi084 ceph-mon[32629]: osdmap e155: 8 total, 8 up, 8 in 2024-09-16T09:29:54.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:54 smithi084 ceph-mon[32629]: pgmap v216: 158 pgs: 158 active+clean; 583 KiB data, 1002 MiB used, 714 GiB / 715 GiB avail; 997 B/s rd, 0 op/s 2024-09-16T09:29:54.971 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:29:54.994 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:54.995 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:54.995 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:29:55.097 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:29:55.104 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T09:29:55.104 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:29:55.119 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:55.119 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:55.120 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:29:55.220 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:29:55.242 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:55.242 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:55.242 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:29:55.343 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:29:55.365 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:55.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:55.366 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:29:55.467 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:29:55.489 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:55.489 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:55.490 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:29:55.590 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:29:55.598 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T09:29:55.599 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T09:29:55.617 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:55.617 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:55.618 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:29:55.719 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:29:55.742 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:55.742 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:55.742 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:29:55.801 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:55 smithi070 ceph-mon[40691]: 1.4 scrub starts 2024-09-16T09:29:55.801 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:55 smithi070 ceph-mon[40691]: 1.4 scrub ok 2024-09-16T09:29:55.801 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:55 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:29:55.801 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:55 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:29:55.801 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:55 smithi070 ceph-mon[40691]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-16T09:29:55.801 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:55 smithi070 ceph-mon[40691]: osdmap e156: 8 total, 8 up, 8 in 2024-09-16T09:29:55.801 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:55 smithi070 ceph-mon[30388]: 1.4 scrub starts 2024-09-16T09:29:55.802 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:55 smithi070 ceph-mon[30388]: 1.4 scrub ok 2024-09-16T09:29:55.802 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:55 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:29:55.802 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:55 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:29:55.802 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:55 smithi070 ceph-mon[30388]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-16T09:29:55.802 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:55 smithi070 ceph-mon[30388]: osdmap e156: 8 total, 8 up, 8 in 2024-09-16T09:29:55.821 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:29:55.830 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.3 dump_blocked_ops 2024-09-16T09:29:55.843 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:29:55.889 DEBUG:teuthology.orchestra.run.smithi070:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.0 -b 65536 --object-size 65536 -p unique_pool_1 bench 90 write' 2024-09-16T09:29:55.889 INFO:tasks.thrashosds.thrasher:Growing pool .mgr 2024-09-16T09:29:55.890 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-16T09:29:55.894 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:55.894 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:55.895 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:29:55.895 INFO:tasks.radosbench:joining radosbench (timing out after 3000s) 2024-09-16T09:29:55.895 INFO:teuthology.orchestra.run:waiting for 3000 2024-09-16T09:29:55.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:55 smithi084 ceph-mon[32629]: 1.4 scrub starts 2024-09-16T09:29:55.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:55 smithi084 ceph-mon[32629]: 1.4 scrub ok 2024-09-16T09:29:55.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:55 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:29:55.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:55 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:29:55.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:55 smithi084 ceph-mon[32629]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-16T09:29:55.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:55 smithi084 ceph-mon[32629]: osdmap e156: 8 total, 8 up, 8 in 2024-09-16T09:29:55.948 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:hints = 1 2024-09-16T09:29:55.948 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Maintaining 16 concurrent writes of 65536 bytes to objects of size 65536 for up to 90 seconds or 0 objects 2024-09-16T09:29:55.948 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Object prefix: benchmark_data_smithi070_107919 2024-09-16T09:29:55.948 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-16T09:29:55.948 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 0 0 0 0 0 0 - 0 2024-09-16T09:29:55.996 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:29:56.030 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:56.030 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:56.031 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:29:56.111 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T09:29:56.131 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:29:56.157 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:56.157 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:56.158 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:29:56.259 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:29:56.301 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:56.301 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:56.301 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:29:56.402 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:29:56.428 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:56.429 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:56.430 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:29:56.467 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:29:56.531 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:29:56.560 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:56.561 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:56.561 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:29:56.656 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:56 smithi070 ceph-mon[30388]: pgmap v218: 158 pgs: 158 active+clean; 583 KiB data, 1002 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:29:56.656 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:56 smithi070 ceph-mon[30388]: 5.1d scrub starts 2024-09-16T09:29:56.656 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:56 smithi070 ceph-mon[30388]: 5.1d scrub ok 2024-09-16T09:29:56.656 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:56 smithi070 ceph-mon[30388]: 3.13 scrub starts 2024-09-16T09:29:56.656 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:56 smithi070 ceph-mon[30388]: 3.13 scrub ok 2024-09-16T09:29:56.656 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:56 smithi070 ceph-mon[40691]: pgmap v218: 158 pgs: 158 active+clean; 583 KiB data, 1002 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:29:56.656 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:56 smithi070 ceph-mon[40691]: 5.1d scrub starts 2024-09-16T09:29:56.656 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:56 smithi070 ceph-mon[40691]: 5.1d scrub ok 2024-09-16T09:29:56.657 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:56 smithi070 ceph-mon[40691]: 3.13 scrub starts 2024-09-16T09:29:56.657 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:56 smithi070 ceph-mon[40691]: 3.13 scrub ok 2024-09-16T09:29:56.659 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:56 smithi084 ceph-mon[32629]: pgmap v218: 158 pgs: 158 active+clean; 583 KiB data, 1002 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:29:56.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:56 smithi084 ceph-mon[32629]: 5.1d scrub starts 2024-09-16T09:29:56.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:56 smithi084 ceph-mon[32629]: 5.1d scrub ok 2024-09-16T09:29:56.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:56 smithi084 ceph-mon[32629]: 3.13 scrub starts 2024-09-16T09:29:56.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:56 smithi084 ceph-mon[32629]: 3.13 scrub ok 2024-09-16T09:29:56.662 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:29:56.696 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:56.697 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:56.697 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:29:56.791 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:29:56.799 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:29:56.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:56.910 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:56.910 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:29:56.948 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 1 15 3099 3084 192.612 192.75 0.00485028 0.00517486 2024-09-16T09:29:57.011 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:29:57.047 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:57.047 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:57.048 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:29:57.148 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:29:57.179 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:57.180 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:57.180 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:29:57.281 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:29:57.298 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T09:29:57.298 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:29:57.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:57.320 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:57.321 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:29:57.422 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:29:57.456 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:57.457 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:57.457 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:29:57.497 INFO:teuthology.orchestra.run.smithi070.stderr:nodeep-scrub is unset 2024-09-16T09:29:57.558 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:29:57.602 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:57.602 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:57.603 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:29:57.703 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:29:57.746 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:57.747 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:57.747 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:29:57.848 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:29:57.885 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:57.886 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:57.886 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:29:57.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:57 smithi084 ceph-mon[32629]: 6.1e scrub starts 2024-09-16T09:29:57.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:57 smithi084 ceph-mon[32629]: 6.1e scrub ok 2024-09-16T09:29:57.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:57 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1614638141' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:29:57.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:57 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:29:57.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:57 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:29:57.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:57 smithi070 ceph-mon[30388]: 6.1e scrub starts 2024-09-16T09:29:57.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:57 smithi070 ceph-mon[30388]: 6.1e scrub ok 2024-09-16T09:29:57.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:57 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1614638141' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:29:57.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:57 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:29:57.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:57 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:29:57.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:57 smithi070 ceph-mon[40691]: 6.1e scrub starts 2024-09-16T09:29:57.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:57 smithi070 ceph-mon[40691]: 6.1e scrub ok 2024-09-16T09:29:57.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:57 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1614638141' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:29:57.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:57 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:29:57.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:57 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:29:57.948 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 2 15 6298 6283 196.263 199.938 0.00491841 0.00508395 2024-09-16T09:29:57.988 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:29:58.021 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:58.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:58.022 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:29:58.122 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:29:58.156 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:58.157 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:58.157 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:29:58.258 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:29:58.291 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:58.292 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:58.292 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:29:58.389 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:29:58.392 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2024-09-16T09:29:58.393 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:29:58.435 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:58.436 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:58.437 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:29:58.537 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:29:58.568 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:58.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:58.569 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:29:58.670 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:29:58.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:58 smithi070 ceph-mon[30388]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-16T09:29:58.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:58 smithi070 ceph-mon[30388]: Cluster is now healthy 2024-09-16T09:29:58.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:58 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-16T09:29:58.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:58 smithi070 ceph-mon[30388]: osdmap e157: 8 total, 8 up, 8 in 2024-09-16T09:29:58.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:58 smithi070 ceph-mon[30388]: pgmap v220: 158 pgs: 158 active+clean; 155 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 26 MiB/s wr, 412 op/s 2024-09-16T09:29:58.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:58 smithi070 ceph-mon[30388]: 4.19 scrub starts 2024-09-16T09:29:58.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:58 smithi070 ceph-mon[30388]: 4.19 scrub ok 2024-09-16T09:29:58.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:58 smithi070 ceph-mon[30388]: from='client.25076 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:29:58.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:58 smithi070 ceph-mon[40691]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-16T09:29:58.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:58 smithi070 ceph-mon[40691]: Cluster is now healthy 2024-09-16T09:29:58.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:58 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-16T09:29:58.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:58 smithi070 ceph-mon[40691]: osdmap e157: 8 total, 8 up, 8 in 2024-09-16T09:29:58.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:58 smithi070 ceph-mon[40691]: pgmap v220: 158 pgs: 158 active+clean; 155 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 26 MiB/s wr, 412 op/s 2024-09-16T09:29:58.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:58 smithi070 ceph-mon[40691]: 4.19 scrub starts 2024-09-16T09:29:58.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:58 smithi070 ceph-mon[40691]: 4.19 scrub ok 2024-09-16T09:29:58.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:58 smithi070 ceph-mon[40691]: from='client.25076 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:29:58.701 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:58.702 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:58.702 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:29:58.803 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:29:58.807 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.3 dump_historic_ops 2024-09-16T09:29:58.840 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:58.841 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:58.841 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:29:58.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:58 smithi084 ceph-mon[32629]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-16T09:29:58.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:58 smithi084 ceph-mon[32629]: Cluster is now healthy 2024-09-16T09:29:58.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:58 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-16T09:29:58.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:58 smithi084 ceph-mon[32629]: osdmap e157: 8 total, 8 up, 8 in 2024-09-16T09:29:58.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:58 smithi084 ceph-mon[32629]: pgmap v220: 158 pgs: 158 active+clean; 155 MiB data, 1.3 GiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 26 MiB/s wr, 412 op/s 2024-09-16T09:29:58.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:58 smithi084 ceph-mon[32629]: 4.19 scrub starts 2024-09-16T09:29:58.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:58 smithi084 ceph-mon[32629]: 4.19 scrub ok 2024-09-16T09:29:58.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:58 smithi084 ceph-mon[32629]: from='client.25076 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:29:58.942 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:29:58.948 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 3 16 9413 9397 195.712 194.625 0.00644474 0.00510011 2024-09-16T09:29:58.978 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:58.978 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:58.979 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:29:59.079 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:29:59.126 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:59.127 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:59.127 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:29:59.194 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:29:59.228 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:29:59.256 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:59.257 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:59.258 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:29:59.358 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:29:59.391 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:59.391 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:59.392 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:29:59.493 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:29:59.524 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:59.524 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:59.524 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:29:59.625 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:29:59.661 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:59.661 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:59.662 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:29:59.762 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:29:59.795 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:29:59.796 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:59.797 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:29:59.801 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:59 smithi070 ceph-mon[30388]: 1.5 scrub starts 2024-09-16T09:29:59.801 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:59 smithi070 ceph-mon[30388]: 1.5 scrub ok 2024-09-16T09:29:59.801 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:59 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:29:59.802 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:59 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:29:59.802 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:59 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:29:59.802 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:59 smithi070 ceph-mon[30388]: 5.c scrub starts 2024-09-16T09:29:59.802 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:59 smithi070 ceph-mon[30388]: 5.c scrub ok 2024-09-16T09:29:59.802 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:59 smithi070 ceph-mon[30388]: 4.16 scrub starts 2024-09-16T09:29:59.802 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:29:59 smithi070 ceph-mon[30388]: 4.16 scrub ok 2024-09-16T09:29:59.802 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:59 smithi070 ceph-mon[40691]: 1.5 scrub starts 2024-09-16T09:29:59.802 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:59 smithi070 ceph-mon[40691]: 1.5 scrub ok 2024-09-16T09:29:59.802 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:59 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:29:59.803 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:59 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:29:59.803 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:59 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:29:59.803 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:59 smithi070 ceph-mon[40691]: 5.c scrub starts 2024-09-16T09:29:59.803 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:59 smithi070 ceph-mon[40691]: 5.c scrub ok 2024-09-16T09:29:59.803 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:59 smithi070 ceph-mon[40691]: 4.16 scrub starts 2024-09-16T09:29:59.803 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:29:59 smithi070 ceph-mon[40691]: 4.16 scrub ok 2024-09-16T09:29:59.898 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:29:59.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:59 smithi084 ceph-mon[32629]: 1.5 scrub starts 2024-09-16T09:29:59.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:59 smithi084 ceph-mon[32629]: 1.5 scrub ok 2024-09-16T09:29:59.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:59 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:29:59.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:59 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:29:59.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:59 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:29:59.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:59 smithi084 ceph-mon[32629]: 5.c scrub starts 2024-09-16T09:29:59.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:59 smithi084 ceph-mon[32629]: 5.c scrub ok 2024-09-16T09:29:59.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:59 smithi084 ceph-mon[32629]: 4.16 scrub starts 2024-09-16T09:29:59.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:29:59 smithi084 ceph-mon[32629]: 4.16 scrub ok 2024-09-16T09:29:59.933 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:29:59.934 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:29:59.934 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:29:59.949 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 4 16 12353 12337 192.718 183.75 0.00255333 0.00518022 2024-09-16T09:30:00.035 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:30:00.072 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:00.073 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:00.073 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:30:00.152 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:30:00.152 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:30:00.152 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:30:00.152 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:30:00.153 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:30:00.153 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:30:00.153 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:30:00.153 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:30:00.153 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:30:00.153 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:30:00.153 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:30:00.153 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:30:00.153 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:30:00.153 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:30:00.153 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:30:00.154 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:30:00.174 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:30:00.214 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:00.214 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:00.215 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:30:00.315 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:30:00.351 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:00.352 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:00.352 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:30:00.407 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:30:00.453 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:30:00.494 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:00.495 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:00.495 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:30:00.596 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:30:00.634 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:00.635 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:00.635 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:30:00.736 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:30:00.772 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:00.773 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:00.773 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:30:00.841 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:00 smithi070 ceph-mon[30388]: pgmap v221: 158 pgs: 158 active+clean; 344 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 839 B/s rd, 56 MiB/s wr, 901 op/s 2024-09-16T09:30:00.842 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:00 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/4240059469' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:30:00.842 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:00 smithi070 ceph-mon[30388]: overall HEALTH_OK 2024-09-16T09:30:00.842 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:00 smithi070 ceph-mon[40691]: pgmap v221: 158 pgs: 158 active+clean; 344 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 839 B/s rd, 56 MiB/s wr, 901 op/s 2024-09-16T09:30:00.842 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:00 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/4240059469' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:30:00.842 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:00 smithi070 ceph-mon[40691]: overall HEALTH_OK 2024-09-16T09:30:00.875 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:30:00.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:00 smithi084 ceph-mon[32629]: pgmap v221: 158 pgs: 158 active+clean; 344 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 839 B/s rd, 56 MiB/s wr, 901 op/s 2024-09-16T09:30:00.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:00 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/4240059469' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:30:00.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:00 smithi084 ceph-mon[32629]: overall HEALTH_OK 2024-09-16T09:30:00.918 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:00.918 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:00.919 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:30:00.949 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 5 16 15299 15283 190.996 184.125 0.00379694 0.00522911 2024-09-16T09:30:01.019 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:30:01.032 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":220,"stamp":"2024-09-16T09:29:57.574221+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":162273819,"num_objects":2666,"num_object_clones":0,"num_object_copies":7998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2666,"num_whiteouts":0,"num_read":1007,"num_read_kb":759,"num_write":2903,"num_write_kb":157933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3811,"ondisk_log_size":3811,"up":474,"acting":474,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":474,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":1365688,"kb_used_data":346972,"kb_used_omap":12,"kb_used_meta":1018419,"kb_avail":748595528,"statfs":{"total":767960285184,"available":766561820672,"internally_reserved":0,"allocated":355299328,"data_stored":352849593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":1042861618},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":161677312,"num_objects":2467,"num_object_clones":0,"num_object_copies":7401,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2467,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2467,"num_write_kb":157888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001348"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":220,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-16T09:29:50.442647+0000","last_change":"2024-09-16T09:28:29.639297+0000","last_active":"2024-09-16T09:29:50.442647+0000","last_peered":"2024-09-16T09:29:50.442647+0000","last_clean":"2024-09-16T09:29:50.442647+0000","last_became_active":"2024-09-16T09:25:01.709108+0000","last_became_peered":"2024-09-16T09:25:01.709108+0000","last_unstale":"2024-09-16T09:29:50.442647+0000","last_undegraded":"2024-09-16T09:29:50.442647+0000","last_fullsized":"2024-09-16T09:29:50.442647+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:29.639071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:28:29.639071+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"66'9","reported_seq":238,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-16T09:29:57.499064+0000","last_change":"2024-09-16T09:28:52.689373+0000","last_active":"2024-09-16T09:29:57.499064+0000","last_peered":"2024-09-16T09:29:57.499064+0000","last_clean":"2024-09-16T09:29:57.499064+0000","last_became_active":"2024-09-16T09:25:03.714635+0000","last_became_peered":"2024-09-16T09:25:03.714635+0000","last_unstale":"2024-09-16T09:29:57.499064+0000","last_undegraded":"2024-09-16T09:29:57.499064+0000","last_fullsized":"2024-09-16T09:29:57.499064+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:28:52.689304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:52.689304+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:21.292656+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":219,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-16T09:29:50.444331+0000","last_change":"2024-09-16T09:29:17.910367+0000","last_active":"2024-09-16T09:29:50.444331+0000","last_peered":"2024-09-16T09:29:50.444331+0000","last_clean":"2024-09-16T09:29:50.444331+0000","last_became_active":"2024-09-16T09:25:05.715645+0000","last_became_peered":"2024-09-16T09:25:05.715645+0000","last_unstale":"2024-09-16T09:29:50.444331+0000","last_undegraded":"2024-09-16T09:29:50.444331+0000","last_fullsized":"2024-09-16T09:29:50.444331+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:17.910305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:17.910305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:31.298101+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":205,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-16T09:29:57.499753+0000","last_change":"2024-09-16T09:27:11.565306+0000","last_active":"2024-09-16T09:29:57.499753+0000","last_peered":"2024-09-16T09:29:57.499753+0000","last_clean":"2024-09-16T09:29:57.499753+0000","last_became_active":"2024-09-16T09:25:07.715442+0000","last_became_peered":"2024-09-16T09:25:07.715442+0000","last_unstale":"2024-09-16T09:29:57.499753+0000","last_undegraded":"2024-09-16T09:29:57.499753+0000","last_fullsized":"2024-09-16T09:29:57.499753+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:11.565220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:11.565220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:41.628205+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":199,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-16T09:29:50.468330+0000","last_change":"2024-09-16T09:28:00.068631+0000","last_active":"2024-09-16T09:29:50.468330+0000","last_peered":"2024-09-16T09:29:50.468330+0000","last_clean":"2024-09-16T09:29:50.468330+0000","last_became_active":"2024-09-16T09:25:07.718126+0000","last_became_peered":"2024-09-16T09:25:07.718126+0000","last_unstale":"2024-09-16T09:29:50.468330+0000","last_undegraded":"2024-09-16T09:29:50.468330+0000","last_fullsized":"2024-09-16T09:29:50.468330+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:00.068570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:28:00.068570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:06.242441+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":61,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-16T09:29:57.495127+0000","last_change":"2024-09-16T09:29:16.575296+0000","last_active":"2024-09-16T09:29:57.495127+0000","last_peered":"2024-09-16T09:29:57.495127+0000","last_clean":"2024-09-16T09:29:57.495127+0000","last_became_active":"2024-09-16T09:28:52.975294+0000","last_became_peered":"2024-09-16T09:28:52.975294+0000","last_unstale":"2024-09-16T09:29:57.495127+0000","last_undegraded":"2024-09-16T09:29:57.495127+0000","last_fullsized":"2024-09-16T09:29:57.495127+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:16.575227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:16.575227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:44.190525+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.19","version":"66'15","reported_seq":92,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-16T09:29:50.822999+0000","last_change":"2024-09-16T09:28:52.983190+0000","last_active":"2024-09-16T09:29:50.822999+0000","last_peered":"2024-09-16T09:29:50.822999+0000","last_clean":"2024-09-16T09:29:50.822999+0000","last_became_active":"2024-09-16T09:28:52.982553+0000","last_became_peered":"2024-09-16T09:28:52.982553+0000","last_unstale":"2024-09-16T09:29:50.822999+0000","last_undegraded":"2024-09-16T09:29:50.822999+0000","last_fullsized":"2024-09-16T09:29:50.822999+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-16T09:28:27.126326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:27.126326+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:53.195556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":226,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-16T09:29:57.499188+0000","last_change":"2024-09-16T09:29:15.628676+0000","last_active":"2024-09-16T09:29:57.499188+0000","last_peered":"2024-09-16T09:29:57.499188+0000","last_clean":"2024-09-16T09:29:57.499188+0000","last_became_active":"2024-09-16T09:25:05.717236+0000","last_became_peered":"2024-09-16T09:25:05.717236+0000","last_unstale":"2024-09-16T09:29:57.499188+0000","last_undegraded":"2024-09-16T09:29:57.499188+0000","last_fullsized":"2024-09-16T09:29:57.499188+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:15.628467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:15.628467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:44.502659+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":58,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-16T09:29:50.444537+0000","last_change":"2024-09-16T09:28:34.839909+0000","last_active":"2024-09-16T09:29:50.444537+0000","last_peered":"2024-09-16T09:29:50.444537+0000","last_clean":"2024-09-16T09:29:50.444537+0000","last_became_active":"2024-09-16T09:28:34.839435+0000","last_became_peered":"2024-09-16T09:28:34.839435+0000","last_unstale":"2024-09-16T09:29:50.444537+0000","last_undegraded":"2024-09-16T09:29:50.444537+0000","last_fullsized":"2024-09-16T09:29:50.444537+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:01.038400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:28:01.038400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:36.847392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"66'9","reported_seq":237,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-16T09:29:57.499966+0000","last_change":"2024-09-16T09:29:28.622647+0000","last_active":"2024-09-16T09:29:57.499966+0000","last_peered":"2024-09-16T09:29:57.499966+0000","last_clean":"2024-09-16T09:29:57.499966+0000","last_became_active":"2024-09-16T09:25:03.709147+0000","last_became_peered":"2024-09-16T09:25:03.709147+0000","last_unstale":"2024-09-16T09:29:57.499966+0000","last_undegraded":"2024-09-16T09:29:57.499966+0000","last_fullsized":"2024-09-16T09:29:57.499966+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:29:28.622558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:29:28.622558+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:48.836980+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":208,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-16T09:29:57.501098+0000","last_change":"2024-09-16T09:27:59.080369+0000","last_active":"2024-09-16T09:29:57.501098+0000","last_peered":"2024-09-16T09:29:57.501098+0000","last_clean":"2024-09-16T09:29:57.501098+0000","last_became_active":"2024-09-16T09:25:05.718134+0000","last_became_peered":"2024-09-16T09:25:05.718134+0000","last_unstale":"2024-09-16T09:29:57.501098+0000","last_undegraded":"2024-09-16T09:29:57.501098+0000","last_fullsized":"2024-09-16T09:29:57.501098+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:59.080307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:59.080307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:47.486214+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"66'1","reported_seq":208,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-16T09:29:50.442077+0000","last_change":"2024-09-16T09:29:15.787990+0000","last_active":"2024-09-16T09:29:50.442077+0000","last_peered":"2024-09-16T09:29:50.442077+0000","last_clean":"2024-09-16T09:29:50.442077+0000","last_became_active":"2024-09-16T09:25:07.716558+0000","last_became_peered":"2024-09-16T09:25:07.716558+0000","last_unstale":"2024-09-16T09:29:50.442077+0000","last_undegraded":"2024-09-16T09:29:50.442077+0000","last_fullsized":"2024-09-16T09:29:50.442077+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-16T09:29:15.787922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:15.787922+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:27.924198+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":216,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-16T09:29:57.500687+0000","last_change":"2024-09-16T09:27:56.191264+0000","last_active":"2024-09-16T09:29:57.500687+0000","last_peered":"2024-09-16T09:29:57.500687+0000","last_clean":"2024-09-16T09:29:57.500687+0000","last_became_active":"2024-09-16T09:25:01.696939+0000","last_became_peered":"2024-09-16T09:25:01.696939+0000","last_unstale":"2024-09-16T09:29:57.500687+0000","last_undegraded":"2024-09-16T09:29:57.500687+0000","last_fullsized":"2024-09-16T09:29:57.500687+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:56.191220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:27:56.191220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:45.493367+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"66'5","reported_seq":223,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-16T09:29:57.502765+0000","last_change":"2024-09-16T09:27:05.694733+0000","last_active":"2024-09-16T09:29:57.502765+0000","last_peered":"2024-09-16T09:29:57.502765+0000","last_clean":"2024-09-16T09:29:57.502765+0000","last_became_active":"2024-09-16T09:25:03.716136+0000","last_became_peered":"2024-09-16T09:25:03.716136+0000","last_unstale":"2024-09-16T09:29:57.502765+0000","last_undegraded":"2024-09-16T09:29:57.502765+0000","last_fullsized":"2024-09-16T09:29:57.502765+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-16T09:27:05.694650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:05.694650+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:47.504795+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":216,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-16T09:29:57.498572+0000","last_change":"2024-09-16T09:28:52.746046+0000","last_active":"2024-09-16T09:29:57.498572+0000","last_peered":"2024-09-16T09:29:57.498572+0000","last_clean":"2024-09-16T09:29:57.498572+0000","last_became_active":"2024-09-16T09:25:05.718281+0000","last_became_peered":"2024-09-16T09:25:05.718281+0000","last_unstale":"2024-09-16T09:29:57.498572+0000","last_undegraded":"2024-09-16T09:29:57.498572+0000","last_fullsized":"2024-09-16T09:29:57.498572+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:52.745999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:28:52.745999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:22.253990+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":220,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-16T09:29:57.499591+0000","last_change":"2024-09-16T09:29:50.468199+0000","last_active":"2024-09-16T09:29:57.499591+0000","last_peered":"2024-09-16T09:29:57.499591+0000","last_clean":"2024-09-16T09:29:57.499591+0000","last_became_active":"2024-09-16T09:25:07.716724+0000","last_became_peered":"2024-09-16T09:25:07.716724+0000","last_unstale":"2024-09-16T09:29:57.499591+0000","last_undegraded":"2024-09-16T09:29:57.499591+0000","last_fullsized":"2024-09-16T09:29:57.499591+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:50.468134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:50.468134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:31:15.288787+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":222,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-16T09:29:57.500518+0000","last_change":"2024-09-16T09:29:56.605555+0000","last_active":"2024-09-16T09:29:57.500518+0000","last_peered":"2024-09-16T09:29:57.500518+0000","last_clean":"2024-09-16T09:29:57.500518+0000","last_became_active":"2024-09-16T09:25:07.720793+0000","last_became_peered":"2024-09-16T09:25:07.720793+0000","last_unstale":"2024-09-16T09:29:57.500518+0000","last_undegraded":"2024-09-16T09:29:57.500518+0000","last_fullsized":"2024-09-16T09:29:57.500518+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:56.605488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:56.605488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:31:10.696779+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":219,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-16T09:29:50.913029+0000","last_change":"2024-09-16T09:29:50.913029+0000","last_active":"2024-09-16T09:29:50.913029+0000","last_peered":"2024-09-16T09:29:50.913029+0000","last_clean":"2024-09-16T09:29:50.913029+0000","last_became_active":"2024-09-16T09:25:01.706172+0000","last_became_peered":"2024-09-16T09:25:01.706172+0000","last_unstale":"2024-09-16T09:29:50.913029+0000","last_undegraded":"2024-09-16T09:29:50.913029+0000","last_fullsized":"2024-09-16T09:29:50.913029+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:50.912979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:50.912979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:55.204409+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"66'15","reported_seq":240,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-16T09:29:50.823423+0000","last_change":"2024-09-16T09:29:03.155623+0000","last_active":"2024-09-16T09:29:50.823423+0000","last_peered":"2024-09-16T09:29:50.823423+0000","last_clean":"2024-09-16T09:29:50.823423+0000","last_became_active":"2024-09-16T09:25:03.713780+0000","last_became_peered":"2024-09-16T09:25:03.713780+0000","last_unstale":"2024-09-16T09:29:50.823423+0000","last_undegraded":"2024-09-16T09:29:50.823423+0000","last_fullsized":"2024-09-16T09:29:50.823423+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-16T09:29:03.155568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:29:03.155568+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:30.605184+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":203,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-16T09:29:50.444921+0000","last_change":"2024-09-16T09:27:16.266378+0000","last_active":"2024-09-16T09:29:50.444921+0000","last_peered":"2024-09-16T09:29:50.444921+0000","last_clean":"2024-09-16T09:29:50.444921+0000","last_became_active":"2024-09-16T09:25:05.713929+0000","last_became_peered":"2024-09-16T09:25:05.713929+0000","last_unstale":"2024-09-16T09:29:50.444921+0000","last_undegraded":"2024-09-16T09:29:50.444921+0000","last_fullsized":"2024-09-16T09:29:50.444921+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:16.266297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:16.266297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":62,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-16T09:29:57.495045+0000","last_change":"2024-09-16T09:28:53.566347+0000","last_active":"2024-09-16T09:29:57.495045+0000","last_peered":"2024-09-16T09:29:57.495045+0000","last_clean":"2024-09-16T09:29:57.495045+0000","last_became_active":"2024-09-16T09:28:52.974255+0000","last_became_peered":"2024-09-16T09:28:52.974255+0000","last_unstale":"2024-09-16T09:29:57.495045+0000","last_undegraded":"2024-09-16T09:29:57.495045+0000","last_fullsized":"2024-09-16T09:29:57.495045+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:53.566299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:28:53.566299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:23.088282+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":225,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-16T09:29:57.500084+0000","last_change":"2024-09-16T09:28:31.565260+0000","last_active":"2024-09-16T09:29:57.500084+0000","last_peered":"2024-09-16T09:29:57.500084+0000","last_clean":"2024-09-16T09:29:57.500084+0000","last_became_active":"2024-09-16T09:25:01.706491+0000","last_became_peered":"2024-09-16T09:25:01.706491+0000","last_unstale":"2024-09-16T09:29:57.500084+0000","last_undegraded":"2024-09-16T09:29:57.500084+0000","last_fullsized":"2024-09-16T09:29:57.500084+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:31.565184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:28:31.565184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"66'12","reported_seq":90,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-16T09:29:50.445072+0000","last_change":"2024-09-16T09:28:52.981384+0000","last_active":"2024-09-16T09:29:50.445072+0000","last_peered":"2024-09-16T09:29:50.445072+0000","last_clean":"2024-09-16T09:29:50.445072+0000","last_became_active":"2024-09-16T09:28:52.981011+0000","last_became_peered":"2024-09-16T09:28:52.981011+0000","last_unstale":"2024-09-16T09:29:50.445072+0000","last_undegraded":"2024-09-16T09:29:50.445072+0000","last_fullsized":"2024-09-16T09:29:50.445072+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-16T09:27:01.700139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:01.700139+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:11.951654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":24,"num_read_kb":15,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":209,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-16T09:29:57.500980+0000","last_change":"2024-09-16T09:27:00.816500+0000","last_active":"2024-09-16T09:29:57.500980+0000","last_peered":"2024-09-16T09:29:57.500980+0000","last_clean":"2024-09-16T09:29:57.500980+0000","last_became_active":"2024-09-16T09:25:05.710093+0000","last_became_peered":"2024-09-16T09:25:05.710093+0000","last_unstale":"2024-09-16T09:29:57.500980+0000","last_undegraded":"2024-09-16T09:29:57.500980+0000","last_fullsized":"2024-09-16T09:29:57.500980+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:00.816424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:00.816424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:42.602227+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":213,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-16T09:29:57.497225+0000","last_change":"2024-09-16T09:29:29.918433+0000","last_active":"2024-09-16T09:29:57.497225+0000","last_peered":"2024-09-16T09:29:57.497225+0000","last_clean":"2024-09-16T09:29:57.497225+0000","last_became_active":"2024-09-16T09:25:07.719380+0000","last_became_peered":"2024-09-16T09:25:07.719380+0000","last_unstale":"2024-09-16T09:29:57.497225+0000","last_undegraded":"2024-09-16T09:29:57.497225+0000","last_fullsized":"2024-09-16T09:29:57.497225+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-16T09:29:29.918368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:29.918368+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:59.870894+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":220,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-16T09:29:50.444431+0000","last_change":"2024-09-16T09:28:17.993009+0000","last_active":"2024-09-16T09:29:50.444431+0000","last_peered":"2024-09-16T09:29:50.444431+0000","last_clean":"2024-09-16T09:29:50.444431+0000","last_became_active":"2024-09-16T09:25:01.699545+0000","last_became_peered":"2024-09-16T09:25:01.699545+0000","last_unstale":"2024-09-16T09:29:50.444431+0000","last_undegraded":"2024-09-16T09:29:50.444431+0000","last_fullsized":"2024-09-16T09:29:50.444431+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:28:17.992925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:28:17.992925+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"66'10","reported_seq":231,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-16T09:29:50.443345+0000","last_change":"2024-09-16T09:27:43.556435+0000","last_active":"2024-09-16T09:29:50.443345+0000","last_peered":"2024-09-16T09:29:50.443345+0000","last_clean":"2024-09-16T09:29:50.443345+0000","last_became_active":"2024-09-16T09:25:03.720245+0000","last_became_peered":"2024-09-16T09:25:03.720245+0000","last_unstale":"2024-09-16T09:29:50.443345+0000","last_undegraded":"2024-09-16T09:29:50.443345+0000","last_fullsized":"2024-09-16T09:29:50.443345+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-16T09:27:43.556357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:43.556357+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:13.698617+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"66'8","reported_seq":297,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-16T09:29:57.495212+0000","last_change":"2024-09-16T09:28:58.597645+0000","last_active":"2024-09-16T09:29:57.495212+0000","last_peered":"2024-09-16T09:29:57.495212+0000","last_clean":"2024-09-16T09:29:57.495212+0000","last_became_active":"2024-09-16T09:25:05.712516+0000","last_became_peered":"2024-09-16T09:25:05.712516+0000","last_unstale":"2024-09-16T09:29:57.495212+0000","last_undegraded":"2024-09-16T09:29:57.495212+0000","last_fullsized":"2024-09-16T09:29:57.495212+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:28:58.597586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:28:58.597586+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:23.196728+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-16T09:29:50.445121+0000","last_change":"2024-09-16T09:29:49.449011+0000","last_active":"2024-09-16T09:29:50.445121+0000","last_peered":"2024-09-16T09:29:50.445121+0000","last_clean":"2024-09-16T09:29:50.445121+0000","last_became_active":"2024-09-16T09:29:49.448473+0000","last_became_peered":"2024-09-16T09:29:49.448473+0000","last_unstale":"2024-09-16T09:29:50.445121+0000","last_undegraded":"2024-09-16T09:29:50.445121+0000","last_fullsized":"2024-09-16T09:29:50.445121+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:31:03.371211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.a","version":"73'79","reported_seq":153,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-16T09:29:57.499377+0000","last_change":"2024-09-16T09:29:18.498887+0000","last_active":"2024-09-16T09:29:57.499377+0000","last_peered":"2024-09-16T09:29:57.499377+0000","last_clean":"2024-09-16T09:29:57.499377+0000","last_became_active":"2024-09-16T09:26:35.127352+0000","last_became_peered":"2024-09-16T09:26:35.127352+0000","last_unstale":"2024-09-16T09:29:57.499377+0000","last_undegraded":"2024-09-16T09:29:57.499377+0000","last_fullsized":"2024-09-16T09:29:57.499377+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:18.498766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:18.498766+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:37.869780+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"66'15","reported_seq":159,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-16T09:29:50.823482+0000","last_change":"2024-09-16T09:28:28.065781+0000","last_active":"2024-09-16T09:29:50.823482+0000","last_peered":"2024-09-16T09:29:50.823482+0000","last_clean":"2024-09-16T09:29:50.823482+0000","last_became_active":"2024-09-16T09:26:33.295672+0000","last_became_peered":"2024-09-16T09:26:33.295672+0000","last_unstale":"2024-09-16T09:29:50.823482+0000","last_undegraded":"2024-09-16T09:29:50.823482+0000","last_fullsized":"2024-09-16T09:29:50.823482+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-16T09:28:28.065699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:28.065699+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:33.850277+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":220,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-16T09:29:50.468286+0000","last_change":"2024-09-16T09:28:22.091708+0000","last_active":"2024-09-16T09:29:50.468286+0000","last_peered":"2024-09-16T09:29:50.468286+0000","last_clean":"2024-09-16T09:29:50.468286+0000","last_became_active":"2024-09-16T09:25:01.702629+0000","last_became_peered":"2024-09-16T09:25:01.702629+0000","last_unstale":"2024-09-16T09:29:50.468286+0000","last_undegraded":"2024-09-16T09:29:50.468286+0000","last_fullsized":"2024-09-16T09:29:50.468286+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:22.091537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:28:22.091537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:51.478241+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"66'8","reported_seq":270,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-16T09:29:57.500601+0000","last_change":"2024-09-16T09:25:05.714128+0000","last_active":"2024-09-16T09:29:57.500601+0000","last_peered":"2024-09-16T09:29:57.500601+0000","last_clean":"2024-09-16T09:29:57.500601+0000","last_became_active":"2024-09-16T09:25:05.713813+0000","last_became_peered":"2024-09-16T09:25:05.713813+0000","last_unstale":"2024-09-16T09:29:57.500601+0000","last_undegraded":"2024-09-16T09:29:57.500601+0000","last_fullsized":"2024-09-16T09:29:57.500601+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:14.664591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":213,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-16T09:29:57.500465+0000","last_change":"2024-09-16T09:29:24.537802+0000","last_active":"2024-09-16T09:29:57.500465+0000","last_peered":"2024-09-16T09:29:57.500465+0000","last_clean":"2024-09-16T09:29:57.500465+0000","last_became_active":"2024-09-16T09:25:07.717344+0000","last_became_peered":"2024-09-16T09:25:07.717344+0000","last_unstale":"2024-09-16T09:29:57.500465+0000","last_undegraded":"2024-09-16T09:29:57.500465+0000","last_fullsized":"2024-09-16T09:29:57.500465+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:24.537731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:24.537731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:33.718527+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-16T09:29:50.442475+0000","last_change":"2024-09-16T09:29:50.008199+0000","last_active":"2024-09-16T09:29:50.442475+0000","last_peered":"2024-09-16T09:29:50.442475+0000","last_clean":"2024-09-16T09:29:50.442475+0000","last_became_active":"2024-09-16T09:29:50.007935+0000","last_became_peered":"2024-09-16T09:29:50.007935+0000","last_unstale":"2024-09-16T09:29:50.442475+0000","last_undegraded":"2024-09-16T09:29:50.442475+0000","last_fullsized":"2024-09-16T09:29:50.442475+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:57.381276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"73'79","reported_seq":153,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-16T09:29:57.500869+0000","last_change":"2024-09-16T09:28:33.628146+0000","last_active":"2024-09-16T09:29:57.500869+0000","last_peered":"2024-09-16T09:29:57.500869+0000","last_clean":"2024-09-16T09:29:57.500869+0000","last_became_active":"2024-09-16T09:26:35.117714+0000","last_became_peered":"2024-09-16T09:26:35.117714+0000","last_unstale":"2024-09-16T09:29:57.500869+0000","last_undegraded":"2024-09-16T09:29:57.500869+0000","last_fullsized":"2024-09-16T09:29:57.500869+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:28:33.628085+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:28:33.628085+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"66'18","reported_seq":113,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-16T09:29:57.497467+0000","last_change":"2024-09-16T09:28:54.700083+0000","last_active":"2024-09-16T09:29:57.497467+0000","last_peered":"2024-09-16T09:29:57.497467+0000","last_clean":"2024-09-16T09:29:57.497467+0000","last_became_active":"2024-09-16T09:28:52.979255+0000","last_became_peered":"2024-09-16T09:28:52.979255+0000","last_unstale":"2024-09-16T09:29:57.497467+0000","last_undegraded":"2024-09-16T09:29:57.497467+0000","last_fullsized":"2024-09-16T09:29:57.497467+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'18","last_scrub_stamp":"2024-09-16T09:28:54.699778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:54.699778+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:16.500734+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":33,"num_read_kb":21,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":61,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-16T09:29:57.498413+0000","last_change":"2024-09-16T09:28:58.705169+0000","last_active":"2024-09-16T09:29:57.498413+0000","last_peered":"2024-09-16T09:29:57.498413+0000","last_clean":"2024-09-16T09:29:57.498413+0000","last_became_active":"2024-09-16T09:28:52.983543+0000","last_became_peered":"2024-09-16T09:28:52.983543+0000","last_unstale":"2024-09-16T09:29:57.498413+0000","last_undegraded":"2024-09-16T09:29:57.498413+0000","last_fullsized":"2024-09-16T09:29:57.498413+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:58.705097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:28:58.705097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:13.544726+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":226,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-16T09:29:57.499101+0000","last_change":"2024-09-16T09:29:50.482296+0000","last_active":"2024-09-16T09:29:57.499101+0000","last_peered":"2024-09-16T09:29:57.499101+0000","last_clean":"2024-09-16T09:29:57.499101+0000","last_became_active":"2024-09-16T09:25:05.713360+0000","last_became_peered":"2024-09-16T09:25:05.713360+0000","last_unstale":"2024-09-16T09:29:57.499101+0000","last_undegraded":"2024-09-16T09:29:57.499101+0000","last_fullsized":"2024-09-16T09:29:57.499101+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:50.482240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:50.482240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:53.077515+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":206,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-16T09:29:57.499926+0000","last_change":"2024-09-16T09:27:13.548667+0000","last_active":"2024-09-16T09:29:57.499926+0000","last_peered":"2024-09-16T09:29:57.499926+0000","last_clean":"2024-09-16T09:29:57.499926+0000","last_became_active":"2024-09-16T09:25:07.719159+0000","last_became_peered":"2024-09-16T09:25:07.719159+0000","last_unstale":"2024-09-16T09:29:57.499926+0000","last_undegraded":"2024-09-16T09:29:57.499926+0000","last_fullsized":"2024-09-16T09:29:57.499926+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:13.548608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:13.548608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:46.516250+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.d","version":"157'294","reported_seq":315,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-16T09:29:57.534705+0000","last_change":"2024-09-16T09:29:50.010860+0000","last_active":"2024-09-16T09:29:57.534705+0000","last_peered":"2024-09-16T09:29:57.534705+0000","last_clean":"2024-09-16T09:29:57.534705+0000","last_became_active":"2024-09-16T09:29:50.010421+0000","last_became_peered":"2024-09-16T09:29:50.010421+0000","last_unstale":"2024-09-16T09:29:57.534705+0000","last_undegraded":"2024-09-16T09:29:57.534705+0000","last_fullsized":"2024-09-16T09:29:57.534705+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":294,"log_dups_size":0,"ondisk_log_size":294,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:31:13.299355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19267584,"num_objects":294,"num_object_clones":0,"num_object_copies":882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":294,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":294,"num_write_kb":18816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,2],"acting":[5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.4","version":"73'79","reported_seq":152,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-16T09:29:50.468369+0000","last_change":"2024-09-16T09:28:23.121906+0000","last_active":"2024-09-16T09:29:50.468369+0000","last_peered":"2024-09-16T09:29:50.468369+0000","last_clean":"2024-09-16T09:29:50.468369+0000","last_became_active":"2024-09-16T09:26:35.131343+0000","last_became_peered":"2024-09-16T09:26:35.131343+0000","last_unstale":"2024-09-16T09:29:50.468369+0000","last_undegraded":"2024-09-16T09:29:50.468369+0000","last_fullsized":"2024-09-16T09:29:50.468369+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:28:23.121808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:28:23.121808+0000","objects_scrubbed":1,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:50.968563+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"66'14","reported_seq":243,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-16T09:29:57.499188+0000","last_change":"2024-09-16T09:28:32.595064+0000","last_active":"2024-09-16T09:29:57.499188+0000","last_peered":"2024-09-16T09:29:57.499188+0000","last_clean":"2024-09-16T09:29:57.499188+0000","last_became_active":"2024-09-16T09:25:03.708830+0000","last_became_peered":"2024-09-16T09:25:03.708830+0000","last_unstale":"2024-09-16T09:29:57.499188+0000","last_undegraded":"2024-09-16T09:29:57.499188+0000","last_fullsized":"2024-09-16T09:29:57.499188+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'14","last_scrub_stamp":"2024-09-16T09:28:32.594988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:32.594988+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:00.546313+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":229,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-16T09:29:50.443466+0000","last_change":"2024-09-16T09:28:58.602786+0000","last_active":"2024-09-16T09:29:50.443466+0000","last_peered":"2024-09-16T09:29:50.443466+0000","last_clean":"2024-09-16T09:29:50.443466+0000","last_became_active":"2024-09-16T09:25:01.702992+0000","last_became_peered":"2024-09-16T09:25:01.702992+0000","last_unstale":"2024-09-16T09:29:50.443466+0000","last_undegraded":"2024-09-16T09:29:50.443466+0000","last_fullsized":"2024-09-16T09:29:50.443466+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:28:58.602717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:28:58.602717+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:25.183069+0000","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"137'9","reported_seq":93,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-16T09:29:50.443700+0000","last_change":"2024-09-16T09:29:21.898605+0000","last_active":"2024-09-16T09:29:50.443700+0000","last_peered":"2024-09-16T09:29:50.443700+0000","last_clean":"2024-09-16T09:29:50.443700+0000","last_became_active":"2024-09-16T09:28:52.978049+0000","last_became_peered":"2024-09-16T09:28:52.978049+0000","last_unstale":"2024-09-16T09:29:50.443700+0000","last_undegraded":"2024-09-16T09:29:50.443700+0000","last_fullsized":"2024-09-16T09:29:50.443700+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'9","last_scrub_stamp":"2024-09-16T09:29:21.898533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:21.898533+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:50.363286+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":205,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-16T09:29:57.498058+0000","last_change":"2024-09-16T09:26:40.699888+0000","last_active":"2024-09-16T09:29:57.498058+0000","last_peered":"2024-09-16T09:29:57.498058+0000","last_clean":"2024-09-16T09:29:57.498058+0000","last_became_active":"2024-09-16T09:25:07.722017+0000","last_became_peered":"2024-09-16T09:25:07.722017+0000","last_unstale":"2024-09-16T09:29:57.498058+0000","last_undegraded":"2024-09-16T09:29:57.498058+0000","last_fullsized":"2024-09-16T09:29:57.498058+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:40.699813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:40.699813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:13.912121+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-16T09:29:50.443148+0000","last_change":"2024-09-16T09:29:49.448081+0000","last_active":"2024-09-16T09:29:50.443148+0000","last_peered":"2024-09-16T09:29:50.443148+0000","last_clean":"2024-09-16T09:29:50.443148+0000","last_became_active":"2024-09-16T09:29:49.447767+0000","last_became_peered":"2024-09-16T09:29:49.447767+0000","last_unstale":"2024-09-16T09:29:50.443148+0000","last_undegraded":"2024-09-16T09:29:50.443148+0000","last_fullsized":"2024-09-16T09:29:50.443148+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:54.362837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"73'79","reported_seq":237,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-16T09:29:57.494950+0000","last_change":"2024-09-16T09:29:17.534500+0000","last_active":"2024-09-16T09:29:57.494950+0000","last_peered":"2024-09-16T09:29:57.494950+0000","last_clean":"2024-09-16T09:29:57.494950+0000","last_became_active":"2024-09-16T09:26:35.126759+0000","last_became_peered":"2024-09-16T09:26:35.126759+0000","last_unstale":"2024-09-16T09:29:57.494950+0000","last_undegraded":"2024-09-16T09:29:57.494950+0000","last_fullsized":"2024-09-16T09:29:57.494950+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:17.534429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:17.534429+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:21.094441+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"66'10","reported_seq":231,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-16T09:29:50.444412+0000","last_change":"2024-09-16T09:28:30.792507+0000","last_active":"2024-09-16T09:29:50.444412+0000","last_peered":"2024-09-16T09:29:50.444412+0000","last_clean":"2024-09-16T09:29:50.444412+0000","last_became_active":"2024-09-16T09:25:03.712985+0000","last_became_peered":"2024-09-16T09:25:03.712985+0000","last_unstale":"2024-09-16T09:29:50.444412+0000","last_undegraded":"2024-09-16T09:29:50.444412+0000","last_fullsized":"2024-09-16T09:29:50.444412+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-16T09:28:30.792412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:30.792412+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:57.669767+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":217,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-16T09:29:57.500915+0000","last_change":"2024-09-16T09:26:25.086388+0000","last_active":"2024-09-16T09:29:57.500915+0000","last_peered":"2024-09-16T09:29:57.500915+0000","last_clean":"2024-09-16T09:29:57.500915+0000","last_became_active":"2024-09-16T09:25:01.706440+0000","last_became_peered":"2024-09-16T09:25:01.706440+0000","last_unstale":"2024-09-16T09:29:57.500915+0000","last_undegraded":"2024-09-16T09:29:57.500915+0000","last_fullsized":"2024-09-16T09:29:57.500915+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:25.086322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:25.086322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:06.339933+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"66'8","reported_seq":279,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-16T09:29:50.442451+0000","last_change":"2024-09-16T09:28:57.628615+0000","last_active":"2024-09-16T09:29:50.442451+0000","last_peered":"2024-09-16T09:29:50.442451+0000","last_clean":"2024-09-16T09:29:50.442451+0000","last_became_active":"2024-09-16T09:25:05.714507+0000","last_became_peered":"2024-09-16T09:25:05.714507+0000","last_unstale":"2024-09-16T09:29:50.442451+0000","last_undegraded":"2024-09-16T09:29:50.442451+0000","last_fullsized":"2024-09-16T09:29:50.442451+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:28:57.628543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:28:57.628543+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:19.830302+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":208,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-16T09:29:50.442174+0000","last_change":"2024-09-16T09:28:22.563466+0000","last_active":"2024-09-16T09:29:50.442174+0000","last_peered":"2024-09-16T09:29:50.442174+0000","last_clean":"2024-09-16T09:29:50.442174+0000","last_became_active":"2024-09-16T09:25:07.719604+0000","last_became_peered":"2024-09-16T09:25:07.719604+0000","last_unstale":"2024-09-16T09:29:50.442174+0000","last_undegraded":"2024-09-16T09:29:50.442174+0000","last_fullsized":"2024-09-16T09:29:50.442174+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:22.563395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:28:22.563395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-16T09:29:50.823622+0000","last_change":"2024-09-16T09:29:49.838498+0000","last_active":"2024-09-16T09:29:50.823622+0000","last_peered":"2024-09-16T09:29:50.823622+0000","last_clean":"2024-09-16T09:29:50.823622+0000","last_became_active":"2024-09-16T09:29:49.838146+0000","last_became_peered":"2024-09-16T09:29:49.838146+0000","last_unstale":"2024-09-16T09:29:50.823622+0000","last_undegraded":"2024-09-16T09:29:50.823622+0000","last_fullsized":"2024-09-16T09:29:50.823622+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:31:06.092338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"73'79","reported_seq":221,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-16T09:29:57.494912+0000","last_change":"2024-09-16T09:26:35.170288+0000","last_active":"2024-09-16T09:29:57.494912+0000","last_peered":"2024-09-16T09:29:57.494912+0000","last_clean":"2024-09-16T09:29:57.494912+0000","last_became_active":"2024-09-16T09:26:35.125571+0000","last_became_peered":"2024-09-16T09:26:35.125571+0000","last_unstale":"2024-09-16T09:29:57.494912+0000","last_undegraded":"2024-09-16T09:29:57.494912+0000","last_fullsized":"2024-09-16T09:29:57.494912+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:26:35.170229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:26:35.170229+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:06.520557+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"66'19","reported_seq":244,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-16T09:29:50.442532+0000","last_change":"2024-09-16T09:28:47.551606+0000","last_active":"2024-09-16T09:29:50.442532+0000","last_peered":"2024-09-16T09:29:50.442532+0000","last_clean":"2024-09-16T09:29:50.442532+0000","last_became_active":"2024-09-16T09:25:03.714797+0000","last_became_peered":"2024-09-16T09:25:03.714797+0000","last_unstale":"2024-09-16T09:29:50.442532+0000","last_undegraded":"2024-09-16T09:29:50.442532+0000","last_fullsized":"2024-09-16T09:29:50.442532+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-16T09:28:47.551527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:47.551527+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:59.123170+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":228,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-16T09:29:50.444919+0000","last_change":"2024-09-16T09:29:16.888357+0000","last_active":"2024-09-16T09:29:50.444919+0000","last_peered":"2024-09-16T09:29:50.444919+0000","last_clean":"2024-09-16T09:29:50.444919+0000","last_became_active":"2024-09-16T09:25:01.706806+0000","last_became_peered":"2024-09-16T09:25:01.706806+0000","last_unstale":"2024-09-16T09:29:50.444919+0000","last_undegraded":"2024-09-16T09:29:50.444919+0000","last_fullsized":"2024-09-16T09:29:50.444919+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:16.888236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:16.888236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:30.989136+0000","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":218,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-16T09:29:57.494208+0000","last_change":"2024-09-16T09:28:24.737761+0000","last_active":"2024-09-16T09:29:57.494208+0000","last_peered":"2024-09-16T09:29:57.494208+0000","last_clean":"2024-09-16T09:29:57.494208+0000","last_became_active":"2024-09-16T09:25:05.708663+0000","last_became_peered":"2024-09-16T09:25:05.708663+0000","last_unstale":"2024-09-16T09:29:57.494208+0000","last_undegraded":"2024-09-16T09:29:57.494208+0000","last_fullsized":"2024-09-16T09:29:57.494208+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:24.737702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:28:24.737702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:48.283381+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":200,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-16T09:29:50.444289+0000","last_change":"2024-09-16T09:26:37.282798+0000","last_active":"2024-09-16T09:29:50.444289+0000","last_peered":"2024-09-16T09:29:50.444289+0000","last_clean":"2024-09-16T09:29:50.444289+0000","last_became_active":"2024-09-16T09:25:07.719267+0000","last_became_peered":"2024-09-16T09:25:07.719267+0000","last_unstale":"2024-09-16T09:29:50.444289+0000","last_undegraded":"2024-09-16T09:29:50.444289+0000","last_fullsized":"2024-09-16T09:29:50.444289+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:37.282735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:37.282735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:10.938413+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.8","version":"157'322","reported_seq":342,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-16T09:29:57.534004+0000","last_change":"2024-09-16T09:29:50.009019+0000","last_active":"2024-09-16T09:29:57.534004+0000","last_peered":"2024-09-16T09:29:57.534004+0000","last_clean":"2024-09-16T09:29:57.534004+0000","last_became_active":"2024-09-16T09:29:50.008664+0000","last_became_peered":"2024-09-16T09:29:50.008664+0000","last_unstale":"2024-09-16T09:29:57.534004+0000","last_undegraded":"2024-09-16T09:29:57.534004+0000","last_fullsized":"2024-09-16T09:29:57.534004+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":322,"log_dups_size":0,"ondisk_log_size":322,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:55.925345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21102592,"num_objects":322,"num_object_clones":0,"num_object_copies":966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":322,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":322,"num_write_kb":20608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1","version":"73'79","reported_seq":153,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-16T09:29:57.499436+0000","last_change":"2024-09-16T09:29:15.460244+0000","last_active":"2024-09-16T09:29:57.499436+0000","last_peered":"2024-09-16T09:29:57.499436+0000","last_clean":"2024-09-16T09:29:57.499436+0000","last_became_active":"2024-09-16T09:26:35.117627+0000","last_became_peered":"2024-09-16T09:26:35.117627+0000","last_unstale":"2024-09-16T09:29:57.499436+0000","last_undegraded":"2024-09-16T09:29:57.499436+0000","last_fullsized":"2024-09-16T09:29:57.499436+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:15.460179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:15.460179+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:27.800189+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"66'28","reported_seq":323,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-16T09:29:50.444816+0000","last_change":"2024-09-16T09:29:18.937715+0000","last_active":"2024-09-16T09:29:50.444816+0000","last_peered":"2024-09-16T09:29:50.444816+0000","last_clean":"2024-09-16T09:29:50.444816+0000","last_became_active":"2024-09-16T09:25:03.704878+0000","last_became_peered":"2024-09-16T09:25:03.704878+0000","last_unstale":"2024-09-16T09:29:50.444816+0000","last_undegraded":"2024-09-16T09:29:50.444816+0000","last_fullsized":"2024-09-16T09:29:50.444816+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'28","last_scrub_stamp":"2024-09-16T09:29:18.937623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:29:18.937623+0000","objects_scrubbed":10,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:19.964476+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":226,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-16T09:29:57.503040+0000","last_change":"2024-09-16T09:29:26.637070+0000","last_active":"2024-09-16T09:29:57.503040+0000","last_peered":"2024-09-16T09:29:57.503040+0000","last_clean":"2024-09-16T09:29:57.503040+0000","last_became_active":"2024-09-16T09:25:01.703076+0000","last_became_peered":"2024-09-16T09:25:01.703076+0000","last_unstale":"2024-09-16T09:29:57.503040+0000","last_undegraded":"2024-09-16T09:29:57.503040+0000","last_fullsized":"2024-09-16T09:29:57.503040+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:26.637027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:26.637027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:51.048084+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":235,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-16T09:29:57.500788+0000","last_change":"2024-09-16T09:29:29.486284+0000","last_active":"2024-09-16T09:29:57.500788+0000","last_peered":"2024-09-16T09:29:57.500788+0000","last_clean":"2024-09-16T09:29:57.500788+0000","last_became_active":"2024-09-16T09:25:00.696237+0000","last_became_peered":"2024-09-16T09:25:00.696237+0000","last_unstale":"2024-09-16T09:29:57.500788+0000","last_undegraded":"2024-09-16T09:29:57.500788+0000","last_fullsized":"2024-09-16T09:29:57.500788+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-16T09:29:29.486188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:29:29.486188+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:47.593613+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.0025137430000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":196,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-16T09:29:50.442868+0000","last_change":"2024-09-16T09:25:05.718319+0000","last_active":"2024-09-16T09:29:50.442868+0000","last_peered":"2024-09-16T09:29:50.442868+0000","last_clean":"2024-09-16T09:29:50.442868+0000","last_became_active":"2024-09-16T09:25:05.717591+0000","last_became_peered":"2024-09-16T09:25:05.717591+0000","last_unstale":"2024-09-16T09:29:50.442868+0000","last_undegraded":"2024-09-16T09:29:50.442868+0000","last_fullsized":"2024-09-16T09:29:50.442868+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:08.672802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":208,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-16T09:29:50.468861+0000","last_change":"2024-09-16T09:27:45.201574+0000","last_active":"2024-09-16T09:29:50.468861+0000","last_peered":"2024-09-16T09:29:50.468861+0000","last_clean":"2024-09-16T09:29:50.468861+0000","last_became_active":"2024-09-16T09:25:07.717436+0000","last_became_peered":"2024-09-16T09:25:07.717436+0000","last_unstale":"2024-09-16T09:29:50.468861+0000","last_undegraded":"2024-09-16T09:29:50.468861+0000","last_fullsized":"2024-09-16T09:29:50.468861+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:45.201503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:45.201503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:12.240024+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"157'294","reported_seq":315,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-16T09:29:57.518069+0000","last_change":"2024-09-16T09:29:49.839026+0000","last_active":"2024-09-16T09:29:57.518069+0000","last_peered":"2024-09-16T09:29:57.518069+0000","last_clean":"2024-09-16T09:29:57.518069+0000","last_became_active":"2024-09-16T09:29:49.838725+0000","last_became_peered":"2024-09-16T09:29:49.838725+0000","last_unstale":"2024-09-16T09:29:57.518069+0000","last_undegraded":"2024-09-16T09:29:57.518069+0000","last_fullsized":"2024-09-16T09:29:57.518069+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":294,"log_dups_size":0,"ondisk_log_size":294,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:59.738135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19267584,"num_objects":294,"num_object_clones":0,"num_object_copies":882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":294,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":294,"num_write_kb":18816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.2","version":"73'79","reported_seq":278,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-16T09:29:57.497817+0000","last_change":"2024-09-16T09:26:35.745908+0000","last_active":"2024-09-16T09:29:57.497817+0000","last_peered":"2024-09-16T09:29:57.497817+0000","last_clean":"2024-09-16T09:29:57.497817+0000","last_became_active":"2024-09-16T09:26:35.115364+0000","last_became_peered":"2024-09-16T09:26:35.115364+0000","last_unstale":"2024-09-16T09:29:57.497817+0000","last_undegraded":"2024-09-16T09:29:57.497817+0000","last_fullsized":"2024-09-16T09:29:57.497817+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:26:35.745837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:26:35.745837+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:09.826176+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"66'13","reported_seq":243,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-16T09:29:50.445190+0000","last_change":"2024-09-16T09:27:44.156057+0000","last_active":"2024-09-16T09:29:50.445190+0000","last_peered":"2024-09-16T09:29:50.445190+0000","last_clean":"2024-09-16T09:29:50.445190+0000","last_became_active":"2024-09-16T09:25:03.713241+0000","last_became_peered":"2024-09-16T09:25:03.713241+0000","last_unstale":"2024-09-16T09:29:50.445190+0000","last_undegraded":"2024-09-16T09:29:50.445190+0000","last_fullsized":"2024-09-16T09:29:50.445190+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'13","last_scrub_stamp":"2024-09-16T09:27:44.155969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:44.155969+0000","objects_scrubbed":5,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:14.867164+0000","scrub_duration":29,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":220,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-16T09:29:50.827103+0000","last_change":"2024-09-16T09:29:50.827103+0000","last_active":"2024-09-16T09:29:50.827103+0000","last_peered":"2024-09-16T09:29:50.827103+0000","last_clean":"2024-09-16T09:29:50.827103+0000","last_became_active":"2024-09-16T09:25:01.706828+0000","last_became_peered":"2024-09-16T09:25:01.706828+0000","last_unstale":"2024-09-16T09:29:50.827103+0000","last_undegraded":"2024-09-16T09:29:50.827103+0000","last_fullsized":"2024-09-16T09:29:50.827103+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:50.827057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:50.827057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:31:04.210633+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":229,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-16T09:29:50.823003+0000","last_change":"2024-09-16T09:29:16.104298+0000","last_active":"2024-09-16T09:29:50.823003+0000","last_peered":"2024-09-16T09:29:50.823003+0000","last_clean":"2024-09-16T09:29:50.823003+0000","last_became_active":"2024-09-16T09:25:00.697822+0000","last_became_peered":"2024-09-16T09:25:00.697822+0000","last_unstale":"2024-09-16T09:29:50.823003+0000","last_undegraded":"2024-09-16T09:29:50.823003+0000","last_fullsized":"2024-09-16T09:29:50.823003+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:29:16.104209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:29:16.104209+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:22.205357+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0.00055022299999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":220,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-16T09:29:50.823503+0000","last_change":"2024-09-16T09:29:00.169141+0000","last_active":"2024-09-16T09:29:50.823503+0000","last_peered":"2024-09-16T09:29:50.823503+0000","last_clean":"2024-09-16T09:29:50.823503+0000","last_became_active":"2024-09-16T09:25:05.711539+0000","last_became_peered":"2024-09-16T09:25:05.711539+0000","last_unstale":"2024-09-16T09:29:50.823503+0000","last_undegraded":"2024-09-16T09:29:50.823503+0000","last_fullsized":"2024-09-16T09:29:50.823503+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:00.169077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:00.169077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:17.394265+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":213,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-16T09:29:57.499506+0000","last_change":"2024-09-16T09:29:22.968971+0000","last_active":"2024-09-16T09:29:57.499506+0000","last_peered":"2024-09-16T09:29:57.499506+0000","last_clean":"2024-09-16T09:29:57.499506+0000","last_became_active":"2024-09-16T09:25:07.723282+0000","last_became_peered":"2024-09-16T09:25:07.723282+0000","last_unstale":"2024-09-16T09:29:57.499506+0000","last_undegraded":"2024-09-16T09:29:57.499506+0000","last_fullsized":"2024-09-16T09:29:57.499506+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:22.968908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:22.968908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:41.020638+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-16T09:29:50.444262+0000","last_change":"2024-09-16T09:29:49.448898+0000","last_active":"2024-09-16T09:29:50.444262+0000","last_peered":"2024-09-16T09:29:50.444262+0000","last_clean":"2024-09-16T09:29:50.444262+0000","last_became_active":"2024-09-16T09:29:49.448628+0000","last_became_peered":"2024-09-16T09:29:49.448628+0000","last_unstale":"2024-09-16T09:29:50.444262+0000","last_undegraded":"2024-09-16T09:29:50.444262+0000","last_fullsized":"2024-09-16T09:29:50.444262+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:31:17.951300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"73'79","reported_seq":147,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-16T09:29:50.444946+0000","last_change":"2024-09-16T09:27:57.031941+0000","last_active":"2024-09-16T09:29:50.444946+0000","last_peered":"2024-09-16T09:29:50.444946+0000","last_clean":"2024-09-16T09:29:50.444946+0000","last_became_active":"2024-09-16T09:26:35.116315+0000","last_became_peered":"2024-09-16T09:26:35.116315+0000","last_unstale":"2024-09-16T09:29:50.444946+0000","last_undegraded":"2024-09-16T09:29:50.444946+0000","last_fullsized":"2024-09-16T09:29:50.444946+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:27:57.031564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:27:57.031564+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"66'12","reported_seq":226,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-16T09:29:50.443387+0000","last_change":"2024-09-16T09:29:02.713335+0000","last_active":"2024-09-16T09:29:50.443387+0000","last_peered":"2024-09-16T09:29:50.443387+0000","last_clean":"2024-09-16T09:29:50.443387+0000","last_became_active":"2024-09-16T09:25:03.715315+0000","last_became_peered":"2024-09-16T09:25:03.715315+0000","last_unstale":"2024-09-16T09:29:50.443387+0000","last_undegraded":"2024-09-16T09:29:50.443387+0000","last_fullsized":"2024-09-16T09:29:50.443387+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-16T09:29:02.713259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:29:02.713259+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:06.150014+0000","scrub_duration":26,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"59'1","reported_seq":221,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-16T09:29:50.442525+0000","last_change":"2024-09-16T09:27:00.666748+0000","last_active":"2024-09-16T09:29:50.442525+0000","last_peered":"2024-09-16T09:29:50.442525+0000","last_clean":"2024-09-16T09:29:50.442525+0000","last_became_active":"2024-09-16T09:25:01.701599+0000","last_became_peered":"2024-09-16T09:25:01.701599+0000","last_unstale":"2024-09-16T09:29:50.442525+0000","last_undegraded":"2024-09-16T09:29:50.442525+0000","last_fullsized":"2024-09-16T09:29:50.442525+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:27:00.666673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:27:00.666673+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:41.935727+0000","scrub_duration":28,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"67'5","reported_seq":526,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-16T09:29:57.497601+0000","last_change":"2024-09-16T09:29:19.994791+0000","last_active":"2024-09-16T09:29:57.497601+0000","last_peered":"2024-09-16T09:29:57.497601+0000","last_clean":"2024-09-16T09:29:57.497601+0000","last_became_active":"2024-09-16T09:25:00.696546+0000","last_became_peered":"2024-09-16T09:25:00.696546+0000","last_unstale":"2024-09-16T09:29:57.497601+0000","last_undegraded":"2024-09-16T09:29:57.497601+0000","last_fullsized":"2024-09-16T09:29:57.497601+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'5","last_scrub_stamp":"2024-09-16T09:29:19.994716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:29:19.994716+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:30.571740+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0.0059770420000000001,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":290,"num_read_kb":285,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":209,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-16T09:29:57.500773+0000","last_change":"2024-09-16T09:27:07.337170+0000","last_active":"2024-09-16T09:29:57.500773+0000","last_peered":"2024-09-16T09:29:57.500773+0000","last_clean":"2024-09-16T09:29:57.500773+0000","last_became_active":"2024-09-16T09:25:05.714704+0000","last_became_peered":"2024-09-16T09:25:05.714704+0000","last_unstale":"2024-09-16T09:29:57.500773+0000","last_undegraded":"2024-09-16T09:29:57.500773+0000","last_fullsized":"2024-09-16T09:29:57.500773+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:07.337124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:07.337124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:34.478935+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":200,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-16T09:29:50.444502+0000","last_change":"2024-09-16T09:26:24.231324+0000","last_active":"2024-09-16T09:29:50.444502+0000","last_peered":"2024-09-16T09:29:50.444502+0000","last_clean":"2024-09-16T09:29:50.444502+0000","last_became_active":"2024-09-16T09:25:07.714158+0000","last_became_peered":"2024-09-16T09:25:07.714158+0000","last_unstale":"2024-09-16T09:29:50.444502+0000","last_undegraded":"2024-09-16T09:29:50.444502+0000","last_fullsized":"2024-09-16T09:29:50.444502+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:24.231167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:24.231167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:07.867061+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"157'320","reported_seq":341,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-16T09:29:57.563398+0000","last_change":"2024-09-16T09:29:49.450943+0000","last_active":"2024-09-16T09:29:57.563398+0000","last_peered":"2024-09-16T09:29:57.563398+0000","last_clean":"2024-09-16T09:29:57.563398+0000","last_became_active":"2024-09-16T09:29:49.450643+0000","last_became_peered":"2024-09-16T09:29:49.450643+0000","last_unstale":"2024-09-16T09:29:57.563398+0000","last_undegraded":"2024-09-16T09:29:57.563398+0000","last_fullsized":"2024-09-16T09:29:57.563398+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":320,"log_dups_size":0,"ondisk_log_size":320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:48.501094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":320,"num_object_clones":0,"num_object_copies":960,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":320,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":320,"num_write_kb":20480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"66'16","reported_seq":296,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-16T09:29:57.496135+0000","last_change":"2024-09-16T09:27:14.265733+0000","last_active":"2024-09-16T09:29:57.496135+0000","last_peered":"2024-09-16T09:29:57.496135+0000","last_clean":"2024-09-16T09:29:57.496135+0000","last_became_active":"2024-09-16T09:25:03.714118+0000","last_became_peered":"2024-09-16T09:25:03.714118+0000","last_unstale":"2024-09-16T09:29:57.496135+0000","last_undegraded":"2024-09-16T09:29:57.496135+0000","last_fullsized":"2024-09-16T09:29:57.496135+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'16","last_scrub_stamp":"2024-09-16T09:27:14.265648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:14.265648+0000","objects_scrubbed":8,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:47.270775+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":53,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-16T09:29:57.500704+0000","last_change":"2024-09-16T09:28:52.972799+0000","last_active":"2024-09-16T09:29:57.500704+0000","last_peered":"2024-09-16T09:29:57.500704+0000","last_clean":"2024-09-16T09:29:57.500704+0000","last_became_active":"2024-09-16T09:28:52.972303+0000","last_became_peered":"2024-09-16T09:28:52.972303+0000","last_unstale":"2024-09-16T09:29:57.500704+0000","last_undegraded":"2024-09-16T09:29:57.500704+0000","last_fullsized":"2024-09-16T09:29:57.500704+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:49.195331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:27:49.195331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:39.522852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"73'79","reported_seq":281,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-16T09:29:57.497760+0000","last_change":"2024-09-16T09:29:02.723875+0000","last_active":"2024-09-16T09:29:57.497760+0000","last_peered":"2024-09-16T09:29:57.497760+0000","last_clean":"2024-09-16T09:29:57.497760+0000","last_became_active":"2024-09-16T09:26:33.297014+0000","last_became_peered":"2024-09-16T09:26:33.297014+0000","last_unstale":"2024-09-16T09:29:57.497760+0000","last_undegraded":"2024-09-16T09:29:57.497760+0000","last_fullsized":"2024-09-16T09:29:57.497760+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:02.723794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:02.723794+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:11.669882+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":209,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-16T09:29:57.497475+0000","last_change":"2024-09-16T09:26:39.699133+0000","last_active":"2024-09-16T09:29:57.497475+0000","last_peered":"2024-09-16T09:29:57.497475+0000","last_clean":"2024-09-16T09:29:57.497475+0000","last_became_active":"2024-09-16T09:25:05.714341+0000","last_became_peered":"2024-09-16T09:25:05.714341+0000","last_unstale":"2024-09-16T09:29:57.497475+0000","last_undegraded":"2024-09-16T09:29:57.497475+0000","last_fullsized":"2024-09-16T09:29:57.497475+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:39.699059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:26:39.699059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:06.744032+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":213,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-16T09:29:57.500577+0000","last_change":"2024-09-16T09:29:00.243528+0000","last_active":"2024-09-16T09:29:57.500577+0000","last_peered":"2024-09-16T09:29:57.500577+0000","last_clean":"2024-09-16T09:29:57.500577+0000","last_became_active":"2024-09-16T09:25:07.714281+0000","last_became_peered":"2024-09-16T09:25:07.714281+0000","last_unstale":"2024-09-16T09:29:57.500577+0000","last_undegraded":"2024-09-16T09:29:57.500577+0000","last_fullsized":"2024-09-16T09:29:57.500577+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:00.243482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:00.243482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:20.041664+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.2","version":"157'327","reported_seq":348,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-16T09:29:57.524910+0000","last_change":"2024-09-16T09:29:49.445324+0000","last_active":"2024-09-16T09:29:57.524910+0000","last_peered":"2024-09-16T09:29:57.524910+0000","last_clean":"2024-09-16T09:29:57.524910+0000","last_became_active":"2024-09-16T09:29:49.445022+0000","last_became_peered":"2024-09-16T09:29:49.445022+0000","last_unstale":"2024-09-16T09:29:57.524910+0000","last_undegraded":"2024-09-16T09:29:57.524910+0000","last_fullsized":"2024-09-16T09:29:57.524910+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":327,"log_dups_size":0,"ondisk_log_size":327,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:31:16.543438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21430272,"num_objects":327,"num_object_clones":0,"num_object_copies":981,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":327,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":327,"num_write_kb":20928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"66'11","reported_seq":244,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-16T09:29:57.500694+0000","last_change":"2024-09-16T09:26:33.316695+0000","last_active":"2024-09-16T09:29:57.500694+0000","last_peered":"2024-09-16T09:29:57.500694+0000","last_clean":"2024-09-16T09:29:57.500694+0000","last_became_active":"2024-09-16T09:26:33.287399+0000","last_became_peered":"2024-09-16T09:26:33.287399+0000","last_unstale":"2024-09-16T09:29:57.500694+0000","last_undegraded":"2024-09-16T09:29:57.500694+0000","last_fullsized":"2024-09-16T09:29:57.500694+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:04.697081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":210,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-16T09:29:57.499521+0000","last_change":"2024-09-16T09:25:01.708568+0000","last_active":"2024-09-16T09:29:57.499521+0000","last_peered":"2024-09-16T09:29:57.499521+0000","last_clean":"2024-09-16T09:29:57.499521+0000","last_became_active":"2024-09-16T09:25:01.708270+0000","last_became_peered":"2024-09-16T09:25:01.708270+0000","last_unstale":"2024-09-16T09:29:57.499521+0000","last_undegraded":"2024-09-16T09:29:57.499521+0000","last_fullsized":"2024-09-16T09:29:57.499521+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:09.670719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":209,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-16T09:29:57.500462+0000","last_change":"2024-09-16T09:26:34.171190+0000","last_active":"2024-09-16T09:29:57.500462+0000","last_peered":"2024-09-16T09:29:57.500462+0000","last_clean":"2024-09-16T09:29:57.500462+0000","last_became_active":"2024-09-16T09:25:05.711964+0000","last_became_peered":"2024-09-16T09:25:05.711964+0000","last_unstale":"2024-09-16T09:29:57.500462+0000","last_undegraded":"2024-09-16T09:29:57.500462+0000","last_fullsized":"2024-09-16T09:29:57.500462+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:34.171139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:26:34.171139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:44.467427+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":62,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-16T09:29:57.495015+0000","last_change":"2024-09-16T09:28:56.556198+0000","last_active":"2024-09-16T09:29:57.495015+0000","last_peered":"2024-09-16T09:29:57.495015+0000","last_clean":"2024-09-16T09:29:57.495015+0000","last_became_active":"2024-09-16T09:28:52.977909+0000","last_became_peered":"2024-09-16T09:28:52.977909+0000","last_unstale":"2024-09-16T09:29:57.495015+0000","last_undegraded":"2024-09-16T09:29:57.495015+0000","last_fullsized":"2024-09-16T09:29:57.495015+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:56.556130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:28:56.556130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:06.061012+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-16T09:29:50.444268+0000","last_change":"2024-09-16T09:29:49.449796+0000","last_active":"2024-09-16T09:29:50.444268+0000","last_peered":"2024-09-16T09:29:50.444268+0000","last_clean":"2024-09-16T09:29:50.444268+0000","last_became_active":"2024-09-16T09:29:49.449475+0000","last_became_peered":"2024-09-16T09:29:49.449475+0000","last_unstale":"2024-09-16T09:29:50.444268+0000","last_undegraded":"2024-09-16T09:29:50.444268+0000","last_fullsized":"2024-09-16T09:29:50.444268+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:49.983415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.8","version":"73'79","reported_seq":160,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-16T09:29:57.499142+0000","last_change":"2024-09-16T09:29:17.482847+0000","last_active":"2024-09-16T09:29:57.499142+0000","last_peered":"2024-09-16T09:29:57.499142+0000","last_clean":"2024-09-16T09:29:57.499142+0000","last_became_active":"2024-09-16T09:26:35.126217+0000","last_became_peered":"2024-09-16T09:26:35.126217+0000","last_unstale":"2024-09-16T09:29:57.499142+0000","last_undegraded":"2024-09-16T09:29:57.499142+0000","last_fullsized":"2024-09-16T09:29:57.499142+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:17.482769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:17.482769+0000","objects_scrubbed":1,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:21.070300+0000","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262688,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"66'17","reported_seq":250,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-16T09:29:57.500704+0000","last_change":"2024-09-16T09:28:55.633865+0000","last_active":"2024-09-16T09:29:57.500704+0000","last_peered":"2024-09-16T09:29:57.500704+0000","last_clean":"2024-09-16T09:29:57.500704+0000","last_became_active":"2024-09-16T09:25:03.713358+0000","last_became_peered":"2024-09-16T09:25:03.713358+0000","last_unstale":"2024-09-16T09:29:57.500704+0000","last_undegraded":"2024-09-16T09:29:57.500704+0000","last_fullsized":"2024-09-16T09:29:57.500704+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'17","last_scrub_stamp":"2024-09-16T09:28:55.633770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:55.633770+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:58.863492+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":218,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-16T09:29:57.494468+0000","last_change":"2024-09-16T09:26:04.104188+0000","last_active":"2024-09-16T09:29:57.494468+0000","last_peered":"2024-09-16T09:29:57.494468+0000","last_clean":"2024-09-16T09:29:57.494468+0000","last_became_active":"2024-09-16T09:25:01.703355+0000","last_became_peered":"2024-09-16T09:25:01.703355+0000","last_unstale":"2024-09-16T09:29:57.494468+0000","last_undegraded":"2024-09-16T09:29:57.494468+0000","last_fullsized":"2024-09-16T09:29:57.494468+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:04.104130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:04.104130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:44.256993+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":212,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-16T09:29:50.444608+0000","last_change":"2024-09-16T09:27:54.028056+0000","last_active":"2024-09-16T09:29:50.444608+0000","last_peered":"2024-09-16T09:29:50.444608+0000","last_clean":"2024-09-16T09:29:50.444608+0000","last_became_active":"2024-09-16T09:25:05.716900+0000","last_became_peered":"2024-09-16T09:25:05.716900+0000","last_unstale":"2024-09-16T09:29:50.444608+0000","last_undegraded":"2024-09-16T09:29:50.444608+0000","last_fullsized":"2024-09-16T09:29:50.444608+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:54.027864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:54.027864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":208,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-16T09:29:50.823623+0000","last_change":"2024-09-16T09:28:33.225543+0000","last_active":"2024-09-16T09:29:50.823623+0000","last_peered":"2024-09-16T09:29:50.823623+0000","last_clean":"2024-09-16T09:29:50.823623+0000","last_became_active":"2024-09-16T09:25:07.721347+0000","last_became_peered":"2024-09-16T09:25:07.721347+0000","last_unstale":"2024-09-16T09:29:50.823623+0000","last_undegraded":"2024-09-16T09:29:50.823623+0000","last_fullsized":"2024-09-16T09:29:50.823623+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:33.225473+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:28:33.225473+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:55.185217+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.0","version":"157'297","reported_seq":318,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-16T09:29:57.514145+0000","last_change":"2024-09-16T09:29:49.837604+0000","last_active":"2024-09-16T09:29:57.514145+0000","last_peered":"2024-09-16T09:29:57.514145+0000","last_clean":"2024-09-16T09:29:57.514145+0000","last_became_active":"2024-09-16T09:29:49.837311+0000","last_became_peered":"2024-09-16T09:29:49.837311+0000","last_unstale":"2024-09-16T09:29:57.514145+0000","last_undegraded":"2024-09-16T09:29:57.514145+0000","last_fullsized":"2024-09-16T09:29:57.514145+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":297,"log_dups_size":0,"ondisk_log_size":297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:31:02.332617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19464192,"num_objects":297,"num_object_clones":0,"num_object_copies":891,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":297,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":297,"num_write_kb":19008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.9","version":"73'79","reported_seq":286,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-16T09:29:57.497506+0000","last_change":"2024-09-16T09:29:19.013948+0000","last_active":"2024-09-16T09:29:57.497506+0000","last_peered":"2024-09-16T09:29:57.497506+0000","last_clean":"2024-09-16T09:29:57.497506+0000","last_became_active":"2024-09-16T09:26:35.126066+0000","last_became_peered":"2024-09-16T09:26:35.126066+0000","last_unstale":"2024-09-16T09:29:57.497506+0000","last_undegraded":"2024-09-16T09:29:57.497506+0000","last_fullsized":"2024-09-16T09:29:57.497506+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:19.013865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:19.013865+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:42.903583+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"66'10","reported_seq":221,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-16T09:29:57.501015+0000","last_change":"2024-09-16T09:25:03.708144+0000","last_active":"2024-09-16T09:29:57.501015+0000","last_peered":"2024-09-16T09:29:57.501015+0000","last_clean":"2024-09-16T09:29:57.501015+0000","last_became_active":"2024-09-16T09:25:03.707723+0000","last_became_peered":"2024-09-16T09:25:03.707723+0000","last_unstale":"2024-09-16T09:29:57.501015+0000","last_undegraded":"2024-09-16T09:29:57.501015+0000","last_fullsized":"2024-09-16T09:29:57.501015+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:12.716564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":55,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-16T09:29:50.442652+0000","last_change":"2024-09-16T09:28:56.590999+0000","last_active":"2024-09-16T09:29:50.442652+0000","last_peered":"2024-09-16T09:29:50.442652+0000","last_clean":"2024-09-16T09:29:50.442652+0000","last_became_active":"2024-09-16T09:28:52.983530+0000","last_became_peered":"2024-09-16T09:28:52.983530+0000","last_unstale":"2024-09-16T09:29:50.442652+0000","last_undegraded":"2024-09-16T09:29:50.442652+0000","last_fullsized":"2024-09-16T09:29:50.442652+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:56.590937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:28:56.590937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:13.348023+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":290,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-16T09:29:50.823545+0000","last_change":"2024-09-16T09:29:29.866520+0000","last_active":"2024-09-16T09:29:50.823545+0000","last_peered":"2024-09-16T09:29:50.823545+0000","last_clean":"2024-09-16T09:29:50.823545+0000","last_became_active":"2024-09-16T09:25:05.711590+0000","last_became_peered":"2024-09-16T09:25:05.711590+0000","last_unstale":"2024-09-16T09:29:50.823545+0000","last_undegraded":"2024-09-16T09:29:50.823545+0000","last_fullsized":"2024-09-16T09:29:50.823545+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:29:29.866460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:29.866460+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:51.037099+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":222,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-16T09:29:57.500429+0000","last_change":"2024-09-16T09:29:02.667332+0000","last_active":"2024-09-16T09:29:57.500429+0000","last_peered":"2024-09-16T09:29:57.500429+0000","last_clean":"2024-09-16T09:29:57.500429+0000","last_became_active":"2024-09-16T09:25:07.718958+0000","last_became_peered":"2024-09-16T09:25:07.718958+0000","last_unstale":"2024-09-16T09:29:57.500429+0000","last_undegraded":"2024-09-16T09:29:57.500429+0000","last_fullsized":"2024-09-16T09:29:57.500429+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:02.667259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:02.667259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:28.829982+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-16T09:29:50.445043+0000","last_change":"2024-09-16T09:29:49.839740+0000","last_active":"2024-09-16T09:29:50.445043+0000","last_peered":"2024-09-16T09:29:50.445043+0000","last_clean":"2024-09-16T09:29:50.445043+0000","last_became_active":"2024-09-16T09:29:49.839437+0000","last_became_peered":"2024-09-16T09:29:49.839437+0000","last_unstale":"2024-09-16T09:29:50.445043+0000","last_undegraded":"2024-09-16T09:29:50.445043+0000","last_fullsized":"2024-09-16T09:29:50.445043+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:31:10.650761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"66'9","reported_seq":232,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-16T09:29:50.442514+0000","last_change":"2024-09-16T09:28:23.570956+0000","last_active":"2024-09-16T09:29:50.442514+0000","last_peered":"2024-09-16T09:29:50.442514+0000","last_clean":"2024-09-16T09:29:50.442514+0000","last_became_active":"2024-09-16T09:25:03.712230+0000","last_became_peered":"2024-09-16T09:25:03.712230+0000","last_unstale":"2024-09-16T09:29:50.442514+0000","last_undegraded":"2024-09-16T09:29:50.442514+0000","last_fullsized":"2024-09-16T09:29:50.442514+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:28:23.570883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:23.570883+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:50.668824+0000","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":217,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-16T09:29:57.500827+0000","last_change":"2024-09-16T09:26:26.061173+0000","last_active":"2024-09-16T09:29:57.500827+0000","last_peered":"2024-09-16T09:29:57.500827+0000","last_clean":"2024-09-16T09:29:57.500827+0000","last_became_active":"2024-09-16T09:25:01.702629+0000","last_became_peered":"2024-09-16T09:25:01.702629+0000","last_unstale":"2024-09-16T09:29:57.500827+0000","last_undegraded":"2024-09-16T09:29:57.500827+0000","last_fullsized":"2024-09-16T09:29:57.500827+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:26.061106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:26.061106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:08.347220+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":212,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-16T09:29:50.823003+0000","last_change":"2024-09-16T09:28:57.220125+0000","last_active":"2024-09-16T09:29:50.823003+0000","last_peered":"2024-09-16T09:29:50.823003+0000","last_clean":"2024-09-16T09:29:50.823003+0000","last_became_active":"2024-09-16T09:25:05.712573+0000","last_became_peered":"2024-09-16T09:25:05.712573+0000","last_unstale":"2024-09-16T09:29:50.823003+0000","last_undegraded":"2024-09-16T09:29:50.823003+0000","last_fullsized":"2024-09-16T09:29:50.823003+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:57.220068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:28:57.220068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:23.707294+0000","scrub_duration":33,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":208,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-16T09:29:50.442295+0000","last_change":"2024-09-16T09:28:25.593045+0000","last_active":"2024-09-16T09:29:50.442295+0000","last_peered":"2024-09-16T09:29:50.442295+0000","last_clean":"2024-09-16T09:29:50.442295+0000","last_became_active":"2024-09-16T09:25:07.718778+0000","last_became_peered":"2024-09-16T09:25:07.718778+0000","last_unstale":"2024-09-16T09:29:50.442295+0000","last_undegraded":"2024-09-16T09:29:50.442295+0000","last_fullsized":"2024-09-16T09:29:50.442295+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:25.592728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:28:25.592728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-16T09:29:50.443327+0000","last_change":"2024-09-16T09:29:50.008888+0000","last_active":"2024-09-16T09:29:50.443327+0000","last_peered":"2024-09-16T09:29:50.443327+0000","last_clean":"2024-09-16T09:29:50.443327+0000","last_became_active":"2024-09-16T09:29:50.008656+0000","last_became_peered":"2024-09-16T09:29:50.008656+0000","last_unstale":"2024-09-16T09:29:50.443327+0000","last_undegraded":"2024-09-16T09:29:50.443327+0000","last_fullsized":"2024-09-16T09:29:50.443327+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:31:15.258613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"66'19","reported_seq":253,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-16T09:29:57.494291+0000","last_change":"2024-09-16T09:29:02.554357+0000","last_active":"2024-09-16T09:29:57.494291+0000","last_peered":"2024-09-16T09:29:57.494291+0000","last_clean":"2024-09-16T09:29:57.494291+0000","last_became_active":"2024-09-16T09:25:03.712783+0000","last_became_peered":"2024-09-16T09:25:03.712783+0000","last_unstale":"2024-09-16T09:29:57.494291+0000","last_undegraded":"2024-09-16T09:29:57.494291+0000","last_fullsized":"2024-09-16T09:29:57.494291+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-16T09:29:02.554274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:29:02.554274+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:14.754043+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"59'1","reported_seq":231,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-16T09:29:57.499425+0000","last_change":"2024-09-16T09:29:51.025505+0000","last_active":"2024-09-16T09:29:57.499425+0000","last_peered":"2024-09-16T09:29:57.499425+0000","last_clean":"2024-09-16T09:29:57.499425+0000","last_became_active":"2024-09-16T09:25:01.708069+0000","last_became_peered":"2024-09-16T09:25:01.708069+0000","last_unstale":"2024-09-16T09:29:57.499425+0000","last_undegraded":"2024-09-16T09:29:57.499425+0000","last_fullsized":"2024-09-16T09:29:57.499425+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:29:51.025444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:51.025444+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:51.262232+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":204,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-16T09:29:50.823169+0000","last_change":"2024-09-16T09:27:11.067059+0000","last_active":"2024-09-16T09:29:50.823169+0000","last_peered":"2024-09-16T09:29:50.823169+0000","last_clean":"2024-09-16T09:29:50.823169+0000","last_became_active":"2024-09-16T09:25:05.709264+0000","last_became_peered":"2024-09-16T09:25:05.709264+0000","last_unstale":"2024-09-16T09:29:50.823169+0000","last_undegraded":"2024-09-16T09:29:50.823169+0000","last_fullsized":"2024-09-16T09:29:50.823169+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:11.066986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:11.066986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:47.015327+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":205,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-16T09:29:57.497680+0000","last_change":"2024-09-16T09:27:25.865107+0000","last_active":"2024-09-16T09:29:57.497680+0000","last_peered":"2024-09-16T09:29:57.497680+0000","last_clean":"2024-09-16T09:29:57.497680+0000","last_became_active":"2024-09-16T09:25:07.718750+0000","last_became_peered":"2024-09-16T09:25:07.718750+0000","last_unstale":"2024-09-16T09:29:57.497680+0000","last_undegraded":"2024-09-16T09:29:57.497680+0000","last_fullsized":"2024-09-16T09:29:57.497680+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:25.865030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:25.865030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:35.955976+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"157'309","reported_seq":330,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-16T09:29:57.553872+0000","last_change":"2024-09-16T09:29:50.010183+0000","last_active":"2024-09-16T09:29:57.553872+0000","last_peered":"2024-09-16T09:29:57.553872+0000","last_clean":"2024-09-16T09:29:57.553872+0000","last_became_active":"2024-09-16T09:29:50.009013+0000","last_became_peered":"2024-09-16T09:29:50.009013+0000","last_unstale":"2024-09-16T09:29:57.553872+0000","last_undegraded":"2024-09-16T09:29:57.553872+0000","last_fullsized":"2024-09-16T09:29:57.553872+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":309,"log_dups_size":0,"ondisk_log_size":309,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:31:02.782637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20250624,"num_objects":309,"num_object_clones":0,"num_object_copies":927,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":309,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":309,"num_write_kb":19776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"66'12","reported_seq":237,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-16T09:29:57.499380+0000","last_change":"2024-09-16T09:27:01.771750+0000","last_active":"2024-09-16T09:29:57.499380+0000","last_peered":"2024-09-16T09:29:57.499380+0000","last_clean":"2024-09-16T09:29:57.499380+0000","last_became_active":"2024-09-16T09:25:03.714027+0000","last_became_peered":"2024-09-16T09:25:03.714027+0000","last_unstale":"2024-09-16T09:29:57.499380+0000","last_undegraded":"2024-09-16T09:29:57.499380+0000","last_fullsized":"2024-09-16T09:29:57.499380+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-16T09:27:01.771667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:01.771667+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:37.679800+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":225,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-16T09:29:57.498086+0000","last_change":"2024-09-16T09:29:53.075288+0000","last_active":"2024-09-16T09:29:57.498086+0000","last_peered":"2024-09-16T09:29:57.498086+0000","last_clean":"2024-09-16T09:29:57.498086+0000","last_became_active":"2024-09-16T09:25:01.707528+0000","last_became_peered":"2024-09-16T09:25:01.707528+0000","last_unstale":"2024-09-16T09:29:57.498086+0000","last_undegraded":"2024-09-16T09:29:57.498086+0000","last_fullsized":"2024-09-16T09:29:57.498086+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:53.075232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:53.075232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:31:02.701620+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":212,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-16T09:29:50.822765+0000","last_change":"2024-09-16T09:27:49.965628+0000","last_active":"2024-09-16T09:29:50.822765+0000","last_peered":"2024-09-16T09:29:50.822765+0000","last_clean":"2024-09-16T09:29:50.822765+0000","last_became_active":"2024-09-16T09:25:05.715253+0000","last_became_peered":"2024-09-16T09:25:05.715253+0000","last_unstale":"2024-09-16T09:29:50.822765+0000","last_undegraded":"2024-09-16T09:29:50.822765+0000","last_fullsized":"2024-09-16T09:29:50.822765+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:49.965424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:49.965424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:05.121441+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":200,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-16T09:29:50.468272+0000","last_change":"2024-09-16T09:27:56.059755+0000","last_active":"2024-09-16T09:29:50.468272+0000","last_peered":"2024-09-16T09:29:50.468272+0000","last_clean":"2024-09-16T09:29:50.468272+0000","last_became_active":"2024-09-16T09:25:07.718899+0000","last_became_peered":"2024-09-16T09:25:07.718899+0000","last_unstale":"2024-09-16T09:29:50.468272+0000","last_undegraded":"2024-09-16T09:29:50.468272+0000","last_fullsized":"2024-09-16T09:29:50.468272+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:56.059687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:56.059687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:11.275881+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.4","version":"157'304","reported_seq":325,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-16T09:29:57.527225+0000","last_change":"2024-09-16T09:29:49.444857+0000","last_active":"2024-09-16T09:29:57.527225+0000","last_peered":"2024-09-16T09:29:57.527225+0000","last_clean":"2024-09-16T09:29:57.527225+0000","last_became_active":"2024-09-16T09:29:49.444515+0000","last_became_peered":"2024-09-16T09:29:49.444515+0000","last_unstale":"2024-09-16T09:29:57.527225+0000","last_undegraded":"2024-09-16T09:29:57.527225+0000","last_fullsized":"2024-09-16T09:29:57.527225+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":304,"log_dups_size":0,"ondisk_log_size":304,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:31:14.656695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19922944,"num_objects":304,"num_object_clones":0,"num_object_copies":912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":304,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":304,"num_write_kb":19456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"66'15","reported_seq":238,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-16T09:29:57.499259+0000","last_change":"2024-09-16T09:26:33.138828+0000","last_active":"2024-09-16T09:29:57.499259+0000","last_peered":"2024-09-16T09:29:57.499259+0000","last_clean":"2024-09-16T09:29:57.499259+0000","last_became_active":"2024-09-16T09:25:03.715733+0000","last_became_peered":"2024-09-16T09:25:03.715733+0000","last_unstale":"2024-09-16T09:29:57.499259+0000","last_undegraded":"2024-09-16T09:29:57.499259+0000","last_fullsized":"2024-09-16T09:29:57.499259+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-16T09:26:33.138603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:26:33.138603+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:46.502077+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":229,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-16T09:29:57.498493+0000","last_change":"2024-09-16T09:26:26.705847+0000","last_active":"2024-09-16T09:29:57.498493+0000","last_peered":"2024-09-16T09:29:57.498493+0000","last_clean":"2024-09-16T09:29:57.498493+0000","last_became_active":"2024-09-16T09:25:01.707372+0000","last_became_peered":"2024-09-16T09:25:01.707372+0000","last_unstale":"2024-09-16T09:29:57.498493+0000","last_undegraded":"2024-09-16T09:29:57.498493+0000","last_fullsized":"2024-09-16T09:29:57.498493+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-16T09:26:26.705773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:26.705773+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:44.096466+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"66'8","reported_seq":285,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-16T09:29:57.497417+0000","last_change":"2024-09-16T09:29:00.749254+0000","last_active":"2024-09-16T09:29:57.497417+0000","last_peered":"2024-09-16T09:29:57.497417+0000","last_clean":"2024-09-16T09:29:57.497417+0000","last_became_active":"2024-09-16T09:25:05.716189+0000","last_became_peered":"2024-09-16T09:25:05.716189+0000","last_unstale":"2024-09-16T09:29:57.497417+0000","last_undegraded":"2024-09-16T09:29:57.497417+0000","last_fullsized":"2024-09-16T09:29:57.497417+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:29:00.749177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:00.749177+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:03.301570+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":205,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-16T09:29:57.500257+0000","last_change":"2024-09-16T09:27:20.219376+0000","last_active":"2024-09-16T09:29:57.500257+0000","last_peered":"2024-09-16T09:29:57.500257+0000","last_clean":"2024-09-16T09:29:57.500257+0000","last_became_active":"2024-09-16T09:25:07.717213+0000","last_became_peered":"2024-09-16T09:25:07.717213+0000","last_unstale":"2024-09-16T09:29:57.500257+0000","last_undegraded":"2024-09-16T09:29:57.500257+0000","last_fullsized":"2024-09-16T09:29:57.500257+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:20.219333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:20.219333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:37.526567+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":218,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-16T09:29:57.493823+0000","last_change":"2024-09-16T09:26:33.139405+0000","last_active":"2024-09-16T09:29:57.493823+0000","last_peered":"2024-09-16T09:29:57.493823+0000","last_clean":"2024-09-16T09:29:57.493823+0000","last_became_active":"2024-09-16T09:25:01.703202+0000","last_became_peered":"2024-09-16T09:25:01.703202+0000","last_unstale":"2024-09-16T09:29:57.493823+0000","last_undegraded":"2024-09-16T09:29:57.493823+0000","last_fullsized":"2024-09-16T09:29:57.493823+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:33.139360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:33.139360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:07.496071+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"66'6","reported_seq":216,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-16T09:29:50.469097+0000","last_change":"2024-09-16T09:26:23.458888+0000","last_active":"2024-09-16T09:29:50.469097+0000","last_peered":"2024-09-16T09:29:50.469097+0000","last_clean":"2024-09-16T09:29:50.469097+0000","last_became_active":"2024-09-16T09:25:03.713245+0000","last_became_peered":"2024-09-16T09:25:03.713245+0000","last_unstale":"2024-09-16T09:29:50.469097+0000","last_undegraded":"2024-09-16T09:29:50.469097+0000","last_fullsized":"2024-09-16T09:29:50.469097+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'6","last_scrub_stamp":"2024-09-16T09:26:23.458794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:26:23.458794+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:47.831778+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":217,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-16T09:29:57.500448+0000","last_change":"2024-09-16T09:29:31.496374+0000","last_active":"2024-09-16T09:29:57.500448+0000","last_peered":"2024-09-16T09:29:57.500448+0000","last_clean":"2024-09-16T09:29:57.500448+0000","last_became_active":"2024-09-16T09:25:05.715616+0000","last_became_peered":"2024-09-16T09:25:05.715616+0000","last_unstale":"2024-09-16T09:29:57.500448+0000","last_undegraded":"2024-09-16T09:29:57.500448+0000","last_fullsized":"2024-09-16T09:29:57.500448+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:31.496304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:31.496304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:44.164855+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":205,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-16T09:29:57.499052+0000","last_change":"2024-09-16T09:26:28.733381+0000","last_active":"2024-09-16T09:29:57.499052+0000","last_peered":"2024-09-16T09:29:57.499052+0000","last_clean":"2024-09-16T09:29:57.499052+0000","last_became_active":"2024-09-16T09:25:07.721059+0000","last_became_peered":"2024-09-16T09:25:07.721059+0000","last_unstale":"2024-09-16T09:29:57.499052+0000","last_undegraded":"2024-09-16T09:29:57.499052+0000","last_fullsized":"2024-09-16T09:29:57.499052+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:28.733324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:28.733324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:12.942687+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"66'9","reported_seq":232,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-16T09:29:50.443487+0000","last_change":"2024-09-16T09:28:26.591121+0000","last_active":"2024-09-16T09:29:50.443487+0000","last_peered":"2024-09-16T09:29:50.443487+0000","last_clean":"2024-09-16T09:29:50.443487+0000","last_became_active":"2024-09-16T09:25:03.719573+0000","last_became_peered":"2024-09-16T09:25:03.719573+0000","last_unstale":"2024-09-16T09:29:50.443487+0000","last_undegraded":"2024-09-16T09:29:50.443487+0000","last_fullsized":"2024-09-16T09:29:50.443487+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:28:26.590949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:26.590949+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":225,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-16T09:29:57.497723+0000","last_change":"2024-09-16T09:28:50.698331+0000","last_active":"2024-09-16T09:29:57.497723+0000","last_peered":"2024-09-16T09:29:57.497723+0000","last_clean":"2024-09-16T09:29:57.497723+0000","last_became_active":"2024-09-16T09:25:01.706491+0000","last_became_peered":"2024-09-16T09:25:01.706491+0000","last_unstale":"2024-09-16T09:29:57.497723+0000","last_undegraded":"2024-09-16T09:29:57.497723+0000","last_fullsized":"2024-09-16T09:29:57.497723+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:50.698288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:28:50.698288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:05.036276+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":55,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-16T09:29:50.445001+0000","last_change":"2024-09-16T09:28:53.810644+0000","last_active":"2024-09-16T09:29:50.445001+0000","last_peered":"2024-09-16T09:29:50.445001+0000","last_clean":"2024-09-16T09:29:50.445001+0000","last_became_active":"2024-09-16T09:28:52.977688+0000","last_became_peered":"2024-09-16T09:28:52.977688+0000","last_unstale":"2024-09-16T09:29:50.445001+0000","last_undegraded":"2024-09-16T09:29:50.445001+0000","last_fullsized":"2024-09-16T09:29:50.445001+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:53.810580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:28:53.810580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:00.227485+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":200,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-16T09:29:50.823126+0000","last_change":"2024-09-16T09:26:35.866582+0000","last_active":"2024-09-16T09:29:50.823126+0000","last_peered":"2024-09-16T09:29:50.823126+0000","last_clean":"2024-09-16T09:29:50.823126+0000","last_became_active":"2024-09-16T09:25:07.722050+0000","last_became_peered":"2024-09-16T09:25:07.722050+0000","last_unstale":"2024-09-16T09:29:50.823126+0000","last_undegraded":"2024-09-16T09:29:50.823126+0000","last_fullsized":"2024-09-16T09:29:50.823126+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:35.866512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:35.866512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:40.909599+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"66'9","reported_seq":237,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-16T09:29:57.500815+0000","last_change":"2024-09-16T09:29:32.485165+0000","last_active":"2024-09-16T09:29:57.500815+0000","last_peered":"2024-09-16T09:29:57.500815+0000","last_clean":"2024-09-16T09:29:57.500815+0000","last_became_active":"2024-09-16T09:25:03.717310+0000","last_became_peered":"2024-09-16T09:25:03.717310+0000","last_unstale":"2024-09-16T09:29:57.500815+0000","last_undegraded":"2024-09-16T09:29:57.500815+0000","last_fullsized":"2024-09-16T09:29:57.500815+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:29:32.485084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:29:32.485084+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:32.698063+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":220,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-16T09:29:50.442862+0000","last_change":"2024-09-16T09:29:48.950188+0000","last_active":"2024-09-16T09:29:50.442862+0000","last_peered":"2024-09-16T09:29:50.442862+0000","last_clean":"2024-09-16T09:29:50.442862+0000","last_became_active":"2024-09-16T09:25:01.702419+0000","last_became_peered":"2024-09-16T09:25:01.702419+0000","last_unstale":"2024-09-16T09:29:50.442862+0000","last_undegraded":"2024-09-16T09:29:50.442862+0000","last_fullsized":"2024-09-16T09:29:50.442862+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.950144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.950144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:31:14.085276+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"137'9","reported_seq":100,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-16T09:29:57.497387+0000","last_change":"2024-09-16T09:28:53.714478+0000","last_active":"2024-09-16T09:29:57.497387+0000","last_peered":"2024-09-16T09:29:57.497387+0000","last_clean":"2024-09-16T09:29:57.497387+0000","last_became_active":"2024-09-16T09:28:52.977923+0000","last_became_peered":"2024-09-16T09:28:52.977923+0000","last_unstale":"2024-09-16T09:29:57.497387+0000","last_undegraded":"2024-09-16T09:29:57.497387+0000","last_fullsized":"2024-09-16T09:29:57.497387+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:28:53.714413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:28:53.714413+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:08.966865+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":222,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-16T09:29:57.500066+0000","last_change":"2024-09-16T09:29:49.474440+0000","last_active":"2024-09-16T09:29:57.500066+0000","last_peered":"2024-09-16T09:29:57.500066+0000","last_clean":"2024-09-16T09:29:57.500066+0000","last_became_active":"2024-09-16T09:25:07.718810+0000","last_became_peered":"2024-09-16T09:25:07.718810+0000","last_unstale":"2024-09-16T09:29:57.500066+0000","last_undegraded":"2024-09-16T09:29:57.500066+0000","last_fullsized":"2024-09-16T09:29:57.500066+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:49.474391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:49.474391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:59.740100+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"66'10","reported_seq":90,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-16T09:29:50.444762+0000","last_change":"2024-09-16T09:28:56.825675+0000","last_active":"2024-09-16T09:29:50.444762+0000","last_peered":"2024-09-16T09:29:50.444762+0000","last_clean":"2024-09-16T09:29:50.444762+0000","last_became_active":"2024-09-16T09:28:52.979059+0000","last_became_peered":"2024-09-16T09:28:52.979059+0000","last_unstale":"2024-09-16T09:29:50.444762+0000","last_undegraded":"2024-09-16T09:29:50.444762+0000","last_fullsized":"2024-09-16T09:29:50.444762+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-16T09:28:56.825476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:56.825476+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:02.744116+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":14,"num_read_kb":9,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":225,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-16T09:29:57.498204+0000","last_change":"2024-09-16T09:29:56.021413+0000","last_active":"2024-09-16T09:29:57.498204+0000","last_peered":"2024-09-16T09:29:57.498204+0000","last_clean":"2024-09-16T09:29:57.498204+0000","last_became_active":"2024-09-16T09:25:01.707513+0000","last_became_peered":"2024-09-16T09:25:01.707513+0000","last_unstale":"2024-09-16T09:29:57.498204+0000","last_undegraded":"2024-09-16T09:29:57.498204+0000","last_fullsized":"2024-09-16T09:29:57.498204+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:56.021340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:56.021340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:31:04.402691+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"66'8","reported_seq":293,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-16T09:29:57.500651+0000","last_change":"2024-09-16T09:29:48.485781+0000","last_active":"2024-09-16T09:29:57.500651+0000","last_peered":"2024-09-16T09:29:57.500651+0000","last_clean":"2024-09-16T09:29:57.500651+0000","last_became_active":"2024-09-16T09:25:05.713833+0000","last_became_peered":"2024-09-16T09:25:05.713833+0000","last_unstale":"2024-09-16T09:29:57.500651+0000","last_undegraded":"2024-09-16T09:29:57.500651+0000","last_fullsized":"2024-09-16T09:29:57.500651+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:29:48.485727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.485727+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:31:03.042280+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":207,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-16T09:29:52.958394+0000","last_change":"2024-09-16T09:29:52.958394+0000","last_active":"2024-09-16T09:29:52.958394+0000","last_peered":"2024-09-16T09:29:52.958394+0000","last_clean":"2024-09-16T09:29:52.958394+0000","last_became_active":"2024-09-16T09:25:07.717357+0000","last_became_peered":"2024-09-16T09:25:07.717357+0000","last_unstale":"2024-09-16T09:29:52.958394+0000","last_undegraded":"2024-09-16T09:29:52.958394+0000","last_fullsized":"2024-09-16T09:29:52.958394+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:52.958303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:52.958303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:58.908393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"66'11","reported_seq":244,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-16T09:29:57.502841+0000","last_change":"2024-09-16T09:26:33.302713+0000","last_active":"2024-09-16T09:29:57.502841+0000","last_peered":"2024-09-16T09:29:57.502841+0000","last_clean":"2024-09-16T09:29:57.502841+0000","last_became_active":"2024-09-16T09:26:33.291265+0000","last_became_peered":"2024-09-16T09:26:33.291265+0000","last_unstale":"2024-09-16T09:29:57.502841+0000","last_undegraded":"2024-09-16T09:29:57.502841+0000","last_fullsized":"2024-09-16T09:29:57.502841+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:44.612142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":234,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-16T09:29:57.501121+0000","last_change":"2024-09-16T09:29:51.517112+0000","last_active":"2024-09-16T09:29:57.501121+0000","last_peered":"2024-09-16T09:29:57.501121+0000","last_clean":"2024-09-16T09:29:57.501121+0000","last_became_active":"2024-09-16T09:25:01.705158+0000","last_became_peered":"2024-09-16T09:25:01.705158+0000","last_unstale":"2024-09-16T09:29:57.501121+0000","last_undegraded":"2024-09-16T09:29:57.501121+0000","last_fullsized":"2024-09-16T09:29:57.501121+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:51.517054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:51.517054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:31:13.265239+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":220,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-16T09:29:50.444253+0000","last_change":"2024-09-16T09:29:15.910903+0000","last_active":"2024-09-16T09:29:50.444253+0000","last_peered":"2024-09-16T09:29:50.444253+0000","last_clean":"2024-09-16T09:29:50.444253+0000","last_became_active":"2024-09-16T09:25:05.712816+0000","last_became_peered":"2024-09-16T09:25:05.712816+0000","last_unstale":"2024-09-16T09:29:50.444253+0000","last_undegraded":"2024-09-16T09:29:50.444253+0000","last_fullsized":"2024-09-16T09:29:50.444253+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:15.910824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:15.910824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:37.511760+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":48,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-16T09:29:50.442203+0000","last_change":"2024-09-16T09:28:52.982570+0000","last_active":"2024-09-16T09:29:50.442203+0000","last_peered":"2024-09-16T09:29:50.442203+0000","last_clean":"2024-09-16T09:29:50.442203+0000","last_became_active":"2024-09-16T09:28:52.982243+0000","last_became_peered":"2024-09-16T09:28:52.982243+0000","last_unstale":"2024-09-16T09:29:50.442203+0000","last_undegraded":"2024-09-16T09:29:50.442203+0000","last_fullsized":"2024-09-16T09:29:50.442203+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:48.147759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:48.147759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:10.722960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"66'9","reported_seq":224,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-16T09:29:50.444511+0000","last_change":"2024-09-16T09:26:31.216495+0000","last_active":"2024-09-16T09:29:50.444511+0000","last_peered":"2024-09-16T09:29:50.444511+0000","last_clean":"2024-09-16T09:29:50.444511+0000","last_became_active":"2024-09-16T09:25:03.705807+0000","last_became_peered":"2024-09-16T09:25:03.705807+0000","last_unstale":"2024-09-16T09:29:50.444511+0000","last_undegraded":"2024-09-16T09:29:50.444511+0000","last_fullsized":"2024-09-16T09:29:50.444511+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:26:31.216298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:26:31.216298+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:09.907044+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":217,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-16T09:29:57.499278+0000","last_change":"2024-09-16T09:26:23.796308+0000","last_active":"2024-09-16T09:29:57.499278+0000","last_peered":"2024-09-16T09:29:57.499278+0000","last_clean":"2024-09-16T09:29:57.499278+0000","last_became_active":"2024-09-16T09:25:01.706762+0000","last_became_peered":"2024-09-16T09:25:01.706762+0000","last_unstale":"2024-09-16T09:29:57.499278+0000","last_undegraded":"2024-09-16T09:29:57.499278+0000","last_fullsized":"2024-09-16T09:29:57.499278+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:23.796231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:23.796231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:41.024875+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":47,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-16T09:29:50.442303+0000","last_change":"2024-09-16T09:28:52.983144+0000","last_active":"2024-09-16T09:29:50.442303+0000","last_peered":"2024-09-16T09:29:50.442303+0000","last_clean":"2024-09-16T09:29:50.442303+0000","last_became_active":"2024-09-16T09:28:52.982865+0000","last_became_peered":"2024-09-16T09:28:52.982865+0000","last_unstale":"2024-09-16T09:29:50.442303+0000","last_undegraded":"2024-09-16T09:29:50.442303+0000","last_fullsized":"2024-09-16T09:29:50.442303+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:44.241942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:44.241942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:04.706066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":208,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-16T09:29:50.442003+0000","last_change":"2024-09-16T09:28:52.552567+0000","last_active":"2024-09-16T09:29:50.442003+0000","last_peered":"2024-09-16T09:29:50.442003+0000","last_clean":"2024-09-16T09:29:50.442003+0000","last_became_active":"2024-09-16T09:25:07.715301+0000","last_became_peered":"2024-09-16T09:25:07.715301+0000","last_unstale":"2024-09-16T09:29:50.442003+0000","last_undegraded":"2024-09-16T09:29:50.442003+0000","last_fullsized":"2024-09-16T09:29:50.442003+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:52.552495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:28:52.552495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:11.151721+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"66'11","reported_seq":100,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-16T09:29:57.497594+0000","last_change":"2024-09-16T09:28:55.643505+0000","last_active":"2024-09-16T09:29:57.497594+0000","last_peered":"2024-09-16T09:29:57.497594+0000","last_clean":"2024-09-16T09:29:57.497594+0000","last_became_active":"2024-09-16T09:28:52.971132+0000","last_became_peered":"2024-09-16T09:28:52.971132+0000","last_unstale":"2024-09-16T09:29:57.497594+0000","last_undegraded":"2024-09-16T09:29:57.497594+0000","last_fullsized":"2024-09-16T09:29:57.497594+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-16T09:28:55.643448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:55.643448+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:24.166714+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":19,"num_read_kb":12,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":225,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-16T09:29:57.500553+0000","last_change":"2024-09-16T09:27:46.175159+0000","last_active":"2024-09-16T09:29:57.500553+0000","last_peered":"2024-09-16T09:29:57.500553+0000","last_clean":"2024-09-16T09:29:57.500553+0000","last_became_active":"2024-09-16T09:25:01.700733+0000","last_became_peered":"2024-09-16T09:25:01.700733+0000","last_unstale":"2024-09-16T09:29:57.500553+0000","last_undegraded":"2024-09-16T09:29:57.500553+0000","last_fullsized":"2024-09-16T09:29:57.500553+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:46.175093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:27:46.175093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:40.495938+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":217,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-16T09:29:57.496785+0000","last_change":"2024-09-16T09:29:25.017736+0000","last_active":"2024-09-16T09:29:57.496785+0000","last_peered":"2024-09-16T09:29:57.496785+0000","last_clean":"2024-09-16T09:29:57.496785+0000","last_became_active":"2024-09-16T09:25:05.713479+0000","last_became_peered":"2024-09-16T09:25:05.713479+0000","last_unstale":"2024-09-16T09:29:57.496785+0000","last_undegraded":"2024-09-16T09:29:57.496785+0000","last_fullsized":"2024-09-16T09:29:57.496785+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:25.017610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:25.017610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:36.761657+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":56,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-16T09:29:50.442645+0000","last_change":"2024-09-16T09:28:54.589320+0000","last_active":"2024-09-16T09:29:50.442645+0000","last_peered":"2024-09-16T09:29:50.442645+0000","last_clean":"2024-09-16T09:29:50.442645+0000","last_became_active":"2024-09-16T09:28:52.982251+0000","last_became_peered":"2024-09-16T09:28:52.982251+0000","last_unstale":"2024-09-16T09:29:50.442645+0000","last_undegraded":"2024-09-16T09:29:50.442645+0000","last_fullsized":"2024-09-16T09:29:50.442645+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:54.589258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:28:54.589258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:22.262118+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"66'4","reported_seq":214,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-16T09:29:50.468066+0000","last_change":"2024-09-16T09:26:27.474085+0000","last_active":"2024-09-16T09:29:50.468066+0000","last_peered":"2024-09-16T09:29:50.468066+0000","last_clean":"2024-09-16T09:29:50.468066+0000","last_became_active":"2024-09-16T09:25:03.704869+0000","last_became_peered":"2024-09-16T09:25:03.704869+0000","last_unstale":"2024-09-16T09:29:50.468066+0000","last_undegraded":"2024-09-16T09:29:50.468066+0000","last_fullsized":"2024-09-16T09:29:50.468066+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'4","last_scrub_stamp":"2024-09-16T09:26:27.473740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:26:27.473740+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:10.261117+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":220,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-16T09:29:50.442793+0000","last_change":"2024-09-16T09:28:32.638113+0000","last_active":"2024-09-16T09:29:50.442793+0000","last_peered":"2024-09-16T09:29:50.442793+0000","last_clean":"2024-09-16T09:29:50.442793+0000","last_became_active":"2024-09-16T09:25:01.705686+0000","last_became_peered":"2024-09-16T09:25:01.705686+0000","last_unstale":"2024-09-16T09:29:50.442793+0000","last_undegraded":"2024-09-16T09:29:50.442793+0000","last_fullsized":"2024-09-16T09:29:50.442793+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:32.638048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:28:32.638048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:29:55.366007+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":226,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-16T09:29:57.494431+0000","last_change":"2024-09-16T09:29:53.244633+0000","last_active":"2024-09-16T09:29:57.494431+0000","last_peered":"2024-09-16T09:29:57.494431+0000","last_clean":"2024-09-16T09:29:57.494431+0000","last_became_active":"2024-09-16T09:25:05.712545+0000","last_became_peered":"2024-09-16T09:25:05.712545+0000","last_unstale":"2024-09-16T09:29:57.494431+0000","last_undegraded":"2024-09-16T09:29:57.494431+0000","last_fullsized":"2024-09-16T09:29:57.494431+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:53.244578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:53.244578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:31:05.703567+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":213,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-16T09:29:57.497874+0000","last_change":"2024-09-16T09:29:27.012810+0000","last_active":"2024-09-16T09:29:57.497874+0000","last_peered":"2024-09-16T09:29:57.497874+0000","last_clean":"2024-09-16T09:29:57.497874+0000","last_became_active":"2024-09-16T09:25:07.720117+0000","last_became_peered":"2024-09-16T09:25:07.720117+0000","last_unstale":"2024-09-16T09:29:57.497874+0000","last_undegraded":"2024-09-16T09:29:57.497874+0000","last_fullsized":"2024-09-16T09:29:57.497874+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:27.012756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:27.012756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:42.125653+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":207,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-16T09:29:50.444634+0000","last_change":"2024-09-16T09:29:19.888825+0000","last_active":"2024-09-16T09:29:50.444634+0000","last_peered":"2024-09-16T09:29:50.444634+0000","last_clean":"2024-09-16T09:29:50.444634+0000","last_became_active":"2024-09-16T09:25:07.712594+0000","last_became_peered":"2024-09-16T09:25:07.712594+0000","last_unstale":"2024-09-16T09:29:50.444634+0000","last_undegraded":"2024-09-16T09:29:50.444634+0000","last_fullsized":"2024-09-16T09:29:50.444634+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:19.888759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:19.888759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:31.404497+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":47,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-16T09:29:50.442346+0000","last_change":"2024-09-16T09:28:52.982847+0000","last_active":"2024-09-16T09:29:50.442346+0000","last_peered":"2024-09-16T09:29:50.442346+0000","last_clean":"2024-09-16T09:29:50.442346+0000","last_became_active":"2024-09-16T09:28:52.982171+0000","last_became_peered":"2024-09-16T09:28:52.982171+0000","last_unstale":"2024-09-16T09:29:50.442346+0000","last_undegraded":"2024-09-16T09:29:50.442346+0000","last_fullsized":"2024-09-16T09:29:50.442346+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:32.168204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:28:32.168204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"66'11","reported_seq":241,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-09-16T09:29:57.495301+0000","last_change":"2024-09-16T09:27:57.486459+0000","last_active":"2024-09-16T09:29:57.495301+0000","last_peered":"2024-09-16T09:29:57.495301+0000","last_clean":"2024-09-16T09:29:57.495301+0000","last_became_active":"2024-09-16T09:25:03.707769+0000","last_became_peered":"2024-09-16T09:25:03.707769+0000","last_unstale":"2024-09-16T09:29:57.495301+0000","last_undegraded":"2024-09-16T09:29:57.495301+0000","last_fullsized":"2024-09-16T09:29:57.495301+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-16T09:27:57.486381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:57.486381+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:09.501888+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":204,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-16T09:29:50.443258+0000","last_change":"2024-09-16T09:27:57.640962+0000","last_active":"2024-09-16T09:29:50.443258+0000","last_peered":"2024-09-16T09:29:50.443258+0000","last_clean":"2024-09-16T09:29:50.443258+0000","last_became_active":"2024-09-16T09:25:05.711952+0000","last_became_peered":"2024-09-16T09:25:05.711952+0000","last_unstale":"2024-09-16T09:29:50.443258+0000","last_undegraded":"2024-09-16T09:29:50.443258+0000","last_fullsized":"2024-09-16T09:29:50.443258+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:57.640756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:57.640756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:12.723072+0000","scrub_duration":44,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":16,"stat_sum":{"num_bytes":161677312,"num_objects":2467,"num_object_clones":0,"num_object_copies":7401,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2467,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2467,"num_write_kb":157888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":475332608,"data_stored":475332608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2467,"ondisk_log_size":2467,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":66,"ondisk_log_size":66,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":693,"num_read_kb":450,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":393,"ondisk_log_size":393,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":290,"num_read_kb":285,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":11,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":869,"ondisk_log_size":869,"up":33,"acting":33,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266752,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":177920,"kb_used_data":70384,"kb_used_omap":1,"kb_used_meta":107518,"kb_avail":93567232,"statfs":{"total":95995035648,"available":95812845568,"internally_reserved":0,"allocated":72073216,"data_stored":71762613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":110098890},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495685,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":180072,"kb_used_data":80856,"kb_used_omap":1,"kb_used_meta":99198,"kb_avail":93565080,"statfs":{"total":95995035648,"available":95810641920,"internally_reserved":0,"allocated":82796544,"data_stored":82509419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":101579206},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724620,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":264308,"kb_used_data":152612,"kb_used_omap":1,"kb_used_meta":111678,"kb_avail":93480844,"statfs":{"total":95995035648,"available":95724384256,"internally_reserved":0,"allocated":156274688,"data_stored":155976288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":114358726},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953553,"num_pgs":65,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":142724,"kb_used_data":39348,"kb_used_omap":1,"kb_used_meta":103358,"kb_avail":93602428,"statfs":{"total":95995035648,"available":95848886272,"internally_reserved":0,"allocated":40292352,"data_stored":39977717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":105839046},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149782,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":181056,"kb_used_data":1320,"kb_used_omap":1,"kb_used_meta":179710,"kb_avail":93564096,"statfs":{"total":95995035648,"available":95809634304,"internally_reserved":0,"allocated":1351680,"data_stored":1033497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":184023495},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411419,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":130716,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":129726,"kb_avail":93614436,"statfs":{"total":95995035648,"available":95861182464,"internally_reserved":0,"allocated":987136,"data_stored":704301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":132839877},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574946,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":130508,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":129726,"kb_avail":93614644,"statfs":{"total":95995035648,"available":95861395456,"internally_reserved":0,"allocated":774144,"data_stored":442713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":132839877},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738472,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":158384,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":157502,"kb_avail":93586768,"statfs":{"total":95995035648,"available":95832850432,"internally_reserved":0,"allocated":749568,"data_stored":443045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":161282501},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":101711872,"data_stored":101711872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":166068224,"data_stored":166068224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":106168320,"data_stored":106168320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":101384192,"data_stored":101384192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T09:30:01.112 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2024-09-16T09:30:01.113 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd pool set .mgr pg_num 21 2024-09-16T09:30:01.114 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:01.114 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:01.115 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:30:01.215 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:30:01.246 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd set noscrub 2024-09-16T09:30:01.247 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:01.247 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:01.248 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:30:01.348 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:30:01.397 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:01.398 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:01.398 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:30:01.499 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:30:01.538 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:01.539 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:01.539 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:30:01.640 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:30:01.679 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:01.680 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:01.680 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:30:01.781 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:30:01.821 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:01.822 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:01.823 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:30:01.924 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:30:01.949 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 6 16 17894 17878 186.193 162.188 0.00477134 0.0053648 2024-09-16T09:30:01.958 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:01.958 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:01.959 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:30:02.060 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:30:02.104 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:02.104 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:02.105 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:30:02.206 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:30:02.215 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T09:30:02.215 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 20, 2024-09-16T09:30:02.215 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 600, 2024-09-16T09:30:02.215 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [ 2024-09-16T09:30:02.215 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.215 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.24698.0:60230 7.3 7:cb5ea935:::benchmark_data_smithi070_78438_object60229:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-16T09:30:02.215 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:26:44.343517+0000", 2024-09-16T09:30:02.216 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 197.866424841, 2024-09-16T09:30:02.216 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.135916441, 2024-09-16T09:30:02.216 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:30:02.216 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:30:02.216 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:30:02.216 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:30:02.217 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.24698", 2024-09-16T09:30:02.217 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:30:02.217 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 60230 2024-09-16T09:30:02.217 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.217 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:30:02.217 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.217 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:30:02.217 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.343517+0000", 2024-09-16T09:30:02.218 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:30:02.218 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.218 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.218 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:30:02.218 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.343517+0000", 2024-09-16T09:30:02.218 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:30:02.218 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.218 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.218 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:30:02.218 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.343517+0000", 2024-09-16T09:30:02.218 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.8500000000000001e-07 2024-09-16T09:30:02.218 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.218 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.219 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:30:02.219 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.343537+0000", 2024-09-16T09:30:02.219 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9845999999999999e-05 2024-09-16T09:30:02.219 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.219 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.219 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:30:02.219 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.343545+0000", 2024-09-16T09:30:02.219 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.2209999999999997e-06 2024-09-16T09:30:02.219 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.219 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.219 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:30:02.219 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.343550+0000", 2024-09-16T09:30:02.219 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.899e-06 2024-09-16T09:30:02.219 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.219 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.220 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:30:02.220 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.347605+0000", 2024-09-16T09:30:02.220 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0040543519999999998 2024-09-16T09:30:02.220 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.220 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.220 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:30:02.220 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.347705+0000", 2024-09-16T09:30:02.220 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00010025 2024-09-16T09:30:02.220 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.220 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.220 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 2,4", 2024-09-16T09:30:02.220 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.347823+0000", 2024-09-16T09:30:02.220 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000117596 2024-09-16T09:30:02.221 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.221 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.221 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:30:02.221 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.351145+0000", 2024-09-16T09:30:02.221 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0033226839999999998 2024-09-16T09:30:02.221 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.221 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.221 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:30:02.221 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.351517+0000", 2024-09-16T09:30:02.221 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00037208299999999999 2024-09-16T09:30:02.221 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.221 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.221 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:30:02.222 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.479316+0000", 2024-09-16T09:30:02.222 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.12779896399999999 2024-09-16T09:30:02.222 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.222 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.222 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:30:02.222 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.479389+0000", 2024-09-16T09:30:02.222 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.2237000000000007e-05 2024-09-16T09:30:02.222 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.222 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.222 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:30:02.222 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.479433+0000", 2024-09-16T09:30:02.223 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.4623999999999998e-05 2024-09-16T09:30:02.223 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:30:02.223 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:30:02.223 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:30:02.223 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.223 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.223 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.24698.0:60241 7.6 e98/82 7:6fa435f1:::benchmark_data_smithi070_78438_object60240:head v 98'3736, mlcod=98'3734)", 2024-09-16T09:30:02.223 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:26:44.351029+0000", 2024-09-16T09:30:02.223 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 197.858913014, 2024-09-16T09:30:02.224 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.12807455000000001, 2024-09-16T09:30:02.224 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:30:02.224 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:30:02.224 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:30:02.224 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:30:02.224 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.224 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:30:02.225 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.351029+0000", 2024-09-16T09:30:02.225 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:30:02.225 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.225 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.225 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:30:02.225 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.351029+0000", 2024-09-16T09:30:02.225 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:30:02.225 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.225 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.225 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:30:02.225 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.351029+0000", 2024-09-16T09:30:02.225 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.89e-07 2024-09-16T09:30:02.225 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.226 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.226 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:30:02.226 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.351050+0000", 2024-09-16T09:30:02.226 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.1070999999999999e-05 2024-09-16T09:30:02.226 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.226 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.226 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:30:02.226 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.351059+0000", 2024-09-16T09:30:02.226 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.2820000000000002e-06 2024-09-16T09:30:02.226 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.226 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.226 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:30:02.226 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.351066+0000", 2024-09-16T09:30:02.226 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.2489999999999998e-06 2024-09-16T09:30:02.227 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.227 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.227 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:30:02.227 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.351140+0000", 2024-09-16T09:30:02.227 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3998000000000005e-05 2024-09-16T09:30:02.227 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.227 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.227 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:30:02.227 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.351157+0000", 2024-09-16T09:30:02.227 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7334999999999999e-05 2024-09-16T09:30:02.227 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.227 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.227 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:30:02.228 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.479051+0000", 2024-09-16T09:30:02.228 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.12789431600000001 2024-09-16T09:30:02.228 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.228 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.228 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:30:02.228 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.479103+0000", 2024-09-16T09:30:02.228 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.181e-05 2024-09-16T09:30:02.228 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:30:02.228 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:30:02.228 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:30:02.228 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.228 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.228 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.24698.0:60243 7.a 7:519efbc0:::benchmark_data_smithi070_78438_object60242:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-16T09:30:02.229 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:26:44.351884+0000", 2024-09-16T09:30:02.229 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 197.858057277, 2024-09-16T09:30:02.229 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.13656760500000001, 2024-09-16T09:30:02.229 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:30:02.229 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:30:02.229 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:30:02.229 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:30:02.229 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.24698", 2024-09-16T09:30:02.229 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:30:02.229 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 60243 2024-09-16T09:30:02.230 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.230 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:30:02.230 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.230 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:30:02.230 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.351884+0000", 2024-09-16T09:30:02.230 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:30:02.230 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.230 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.230 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:30:02.230 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.351884+0000", 2024-09-16T09:30:02.231 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:30:02.231 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.231 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.231 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:30:02.231 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.351885+0000", 2024-09-16T09:30:02.231 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.203e-06 2024-09-16T09:30:02.231 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.231 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.231 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:30:02.231 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.351909+0000", 2024-09-16T09:30:02.231 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3107000000000001e-05 2024-09-16T09:30:02.232 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.232 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.232 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:30:02.232 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.351919+0000", 2024-09-16T09:30:02.232 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0239e-05 2024-09-16T09:30:02.232 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.232 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.232 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:30:02.232 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.351940+0000", 2024-09-16T09:30:02.232 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0846999999999999e-05 2024-09-16T09:30:02.232 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.233 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.233 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:30:02.233 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.355041+0000", 2024-09-16T09:30:02.233 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0031018109999999999 2024-09-16T09:30:02.233 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.233 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.233 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:30:02.233 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.355147+0000", 2024-09-16T09:30:02.233 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000105263 2024-09-16T09:30:02.233 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.233 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.234 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 2,5", 2024-09-16T09:30:02.234 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.479270+0000", 2024-09-16T09:30:02.234 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.12412316 2024-09-16T09:30:02.234 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.234 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.234 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:30:02.234 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.488255+0000", 2024-09-16T09:30:02.234 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0089848919999999995 2024-09-16T09:30:02.234 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.234 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.234 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:30:02.234 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.488266+0000", 2024-09-16T09:30:02.234 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1381e-05 2024-09-16T09:30:02.234 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.235 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.235 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:30:02.235 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.488340+0000", 2024-09-16T09:30:02.235 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3609000000000006e-05 2024-09-16T09:30:02.235 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.235 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.235 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:30:02.235 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.488410+0000", 2024-09-16T09:30:02.235 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.9747999999999997e-05 2024-09-16T09:30:02.235 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.235 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.235 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:30:02.236 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.488452+0000", 2024-09-16T09:30:02.236 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2345000000000002e-05 2024-09-16T09:30:02.236 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:30:02.236 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:30:02.236 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:30:02.236 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.236 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.236 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.24698.0:60248 7.3 7:c2e4be92:::benchmark_data_smithi070_78438_object60247:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-16T09:30:02.236 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:26:44.352711+0000", 2024-09-16T09:30:02.236 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 197.85723044299999, 2024-09-16T09:30:02.236 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.12688961500000001, 2024-09-16T09:30:02.236 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:30:02.236 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:30:02.236 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:30:02.237 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:30:02.237 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.24698", 2024-09-16T09:30:02.237 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:30:02.237 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 60248 2024-09-16T09:30:02.237 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.237 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:30:02.237 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.237 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:30:02.237 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.352711+0000", 2024-09-16T09:30:02.237 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:30:02.237 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.237 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.237 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:30:02.237 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.352711+0000", 2024-09-16T09:30:02.238 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:30:02.238 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.238 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.238 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:30:02.238 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.352712+0000", 2024-09-16T09:30:02.238 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.16e-07 2024-09-16T09:30:02.238 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.238 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.238 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:30:02.238 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.352732+0000", 2024-09-16T09:30:02.238 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9944000000000001e-05 2024-09-16T09:30:02.238 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.238 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.238 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:30:02.239 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.352743+0000", 2024-09-16T09:30:02.239 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1053000000000001e-05 2024-09-16T09:30:02.239 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.239 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.239 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:30:02.239 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.352750+0000", 2024-09-16T09:30:02.239 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.1879999999999996e-06 2024-09-16T09:30:02.239 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.239 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.239 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:30:02.239 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.352871+0000", 2024-09-16T09:30:02.239 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000120828 2024-09-16T09:30:02.239 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.240 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.240 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:30:02.240 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.352997+0000", 2024-09-16T09:30:02.240 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00012576699999999999 2024-09-16T09:30:02.240 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.240 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.240 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 2,4", 2024-09-16T09:30:02.240 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.353118+0000", 2024-09-16T09:30:02.240 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000121045 2024-09-16T09:30:02.240 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.240 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.240 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:30:02.240 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.479259+0000", 2024-09-16T09:30:02.240 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.12614093800000001 2024-09-16T09:30:02.241 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.241 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.241 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:30:02.241 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.479477+0000", 2024-09-16T09:30:02.241 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00021839 2024-09-16T09:30:02.241 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.241 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.241 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:30:02.241 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.479517+0000", 2024-09-16T09:30:02.241 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9885999999999997e-05 2024-09-16T09:30:02.241 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.242 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.242 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:30:02.242 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.479566+0000", 2024-09-16T09:30:02.242 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.8819000000000001e-05 2024-09-16T09:30:02.242 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.242 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.242 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:30:02.242 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.479601+0000", 2024-09-16T09:30:02.242 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4841000000000001e-05 2024-09-16T09:30:02.242 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:30:02.242 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:30:02.242 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:30:02.242 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.242 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.242 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.24698.0:60245 7.7 e98/82 7:e5610e70:::benchmark_data_smithi070_78438_object60244:head v 98'3786, mlcod=98'3784)", 2024-09-16T09:30:02.242 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:26:44.352780+0000", 2024-09-16T09:30:02.243 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 197.85716115599999, 2024-09-16T09:30:02.243 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.12639468800000001, 2024-09-16T09:30:02.243 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:30:02.243 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:30:02.243 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:30:02.243 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:30:02.243 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.243 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:30:02.243 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.352780+0000", 2024-09-16T09:30:02.243 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:30:02.243 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.243 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.243 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:30:02.243 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.352780+0000", 2024-09-16T09:30:02.243 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:30:02.244 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.244 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.244 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:30:02.244 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.352781+0000", 2024-09-16T09:30:02.244 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0999999999999999e-07 2024-09-16T09:30:02.244 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.244 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.244 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:30:02.244 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.352808+0000", 2024-09-16T09:30:02.244 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.7022e-05 2024-09-16T09:30:02.244 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.244 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.244 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:30:02.244 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.352815+0000", 2024-09-16T09:30:02.244 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.4390000000000003e-06 2024-09-16T09:30:02.244 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.245 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.245 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:30:02.245 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.352820+0000", 2024-09-16T09:30:02.245 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.9819999999999999e-06 2024-09-16T09:30:02.245 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.245 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.245 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:30:02.245 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.353010+0000", 2024-09-16T09:30:02.245 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000189204 2024-09-16T09:30:02.246 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.246 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.246 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:30:02.246 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.353027+0000", 2024-09-16T09:30:02.246 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7907e-05 2024-09-16T09:30:02.246 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.246 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.246 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:30:02.246 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.355309+0000", 2024-09-16T09:30:02.246 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0022814799999999998 2024-09-16T09:30:02.246 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.246 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.247 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:30:01 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:30:01] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:30:02.247 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:30:02.247 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.479175+0000", 2024-09-16T09:30:02.247 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.123866144 2024-09-16T09:30:02.248 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:30:02.248 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:30:02.248 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:30:02.248 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.248 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.248 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.24698.0:60247 7.6 e98/82 7:6840b134:::benchmark_data_smithi070_78438_object60246:head v 98'3737, mlcod=98'3734)", 2024-09-16T09:30:02.248 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:26:44.352896+0000", 2024-09-16T09:30:02.248 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 197.85704539700001, 2024-09-16T09:30:02.249 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.12623531099999999, 2024-09-16T09:30:02.249 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:30:02.249 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:30:02.249 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:30:02.249 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:30:02.249 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.249 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:30:02.249 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.352896+0000", 2024-09-16T09:30:02.249 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:30:02.249 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.249 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.250 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:30:02.250 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.352896+0000", 2024-09-16T09:30:02.250 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:30:02.250 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.250 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.250 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:30:02.250 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.352897+0000", 2024-09-16T09:30:02.250 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.3700000000000001e-07 2024-09-16T09:30:02.250 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.251 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.251 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:30:02.251 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.352918+0000", 2024-09-16T09:30:02.251 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.1535e-05 2024-09-16T09:30:02.251 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.251 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.251 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:30:02.251 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.353538+0000", 2024-09-16T09:30:02.251 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00061983100000000005 2024-09-16T09:30:02.252 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.252 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.252 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:30:02.252 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.353544+0000", 2024-09-16T09:30:02.252 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.0279999999999997e-06 2024-09-16T09:30:02.252 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.252 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.252 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:30:02.252 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.353638+0000", 2024-09-16T09:30:02.252 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.4378999999999999e-05 2024-09-16T09:30:02.252 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.253 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.253 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:30:02.253 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.353655+0000", 2024-09-16T09:30:02.253 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6226e-05 2024-09-16T09:30:02.253 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.253 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.253 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:30:02.253 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.479108+0000", 2024-09-16T09:30:02.253 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.12545310500000001 2024-09-16T09:30:02.253 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.253 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.253 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:30:02.254 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.479131+0000", 2024-09-16T09:30:02.254 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3770000000000001e-05 2024-09-16T09:30:02.254 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:30:02.254 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:30:02.254 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:30:02.254 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.254 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.254 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.24698.0:60250 7.a 7:57008ad8:::benchmark_data_smithi070_78438_object60249:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-16T09:30:02.255 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:26:44.354902+0000", 2024-09-16T09:30:02.255 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 197.85503991100001, 2024-09-16T09:30:02.255 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.13525178299999999, 2024-09-16T09:30:02.255 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:30:02.255 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:30:02.255 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:30:02.255 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:30:02.255 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.24698", 2024-09-16T09:30:02.255 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:30:02.255 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 60250 2024-09-16T09:30:02.255 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.255 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:30:02.256 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.256 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:30:02.256 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.354902+0000", 2024-09-16T09:30:02.256 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:30:02.256 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.256 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.256 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:30:02.256 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.354902+0000", 2024-09-16T09:30:02.257 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:30:02.257 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.257 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.257 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:30:02.257 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.354903+0000", 2024-09-16T09:30:02.257 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.217e-06 2024-09-16T09:30:02.257 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.257 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.257 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:30:02.257 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.354926+0000", 2024-09-16T09:30:02.257 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3411e-05 2024-09-16T09:30:02.257 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.257 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.257 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:30:02.258 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.354937+0000", 2024-09-16T09:30:02.258 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0779e-05 2024-09-16T09:30:02.258 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.258 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.258 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:30:02.258 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.354944+0000", 2024-09-16T09:30:02.258 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.148e-06 2024-09-16T09:30:02.258 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.258 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.258 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:30:02.258 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.483018+0000", 2024-09-16T09:30:02.258 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.128073662 2024-09-16T09:30:02.258 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.258 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.258 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:30:02.258 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.483158+0000", 2024-09-16T09:30:02.259 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000140087 2024-09-16T09:30:02.259 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.259 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.259 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 2,5", 2024-09-16T09:30:02.259 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.483300+0000", 2024-09-16T09:30:02.259 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00014235499999999999 2024-09-16T09:30:02.259 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.259 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.259 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:30:02.259 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.488300+0000", 2024-09-16T09:30:02.259 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0049997389999999996 2024-09-16T09:30:02.259 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.259 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.259 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:30:02.260 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.488489+0000", 2024-09-16T09:30:02.260 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00018912200000000001 2024-09-16T09:30:02.260 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.260 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.260 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:30:02.260 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.490003+0000", 2024-09-16T09:30:02.260 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0015142440000000001 2024-09-16T09:30:02.260 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.260 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.260 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:30:02.260 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.490111+0000", 2024-09-16T09:30:02.260 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000107298 2024-09-16T09:30:02.260 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.260 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.260 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:30:02.261 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.490153+0000", 2024-09-16T09:30:02.261 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2721e-05 2024-09-16T09:30:02.261 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:30:02.261 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:30:02.261 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:30:02.261 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.261 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.261 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop_reply(client.24698.0:60248 7.3 e98/82 ondisk, result = 0)", 2024-09-16T09:30:02.261 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:26:44.355484+0000", 2024-09-16T09:30:02.261 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 197.85445752699999, 2024-09-16T09:30:02.261 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.124000344, 2024-09-16T09:30:02.261 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:30:02.262 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:30:02.262 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T09:30:02.262 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:30:02.262 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.262 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:30:02.262 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.355484+0000", 2024-09-16T09:30:02.262 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:30:02.262 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.262 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.262 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:30:02.262 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.355484+0000", 2024-09-16T09:30:02.262 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:30:02.262 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.263 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.263 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:30:02.263 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.355485+0000", 2024-09-16T09:30:02.263 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.1500000000000003e-07 2024-09-16T09:30:02.263 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.263 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.263 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:30:02.263 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.355488+0000", 2024-09-16T09:30:02.263 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6540000000000001e-06 2024-09-16T09:30:02.263 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.263 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.263 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:30:02.263 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.479245+0000", 2024-09-16T09:30:02.263 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.123756668 2024-09-16T09:30:02.263 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.263 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.264 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:30:02.264 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.479255+0000", 2024-09-16T09:30:02.264 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0030999999999999e-05 2024-09-16T09:30:02.264 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.264 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.264 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:30:02.264 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.479465+0000", 2024-09-16T09:30:02.264 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000209903 2024-09-16T09:30:02.264 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.264 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.264 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:30:02.264 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.479472+0000", 2024-09-16T09:30:02.264 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.922e-06 2024-09-16T09:30:02.264 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.265 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.265 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:30:02.265 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.479484+0000", 2024-09-16T09:30:02.265 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2351000000000001e-05 2024-09-16T09:30:02.265 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:30:02.265 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:30:02.265 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:30:02.265 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.265 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.265 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop_reply(client.24698.0:60230 7.3 e98/82 ondisk, result = 0)", 2024-09-16T09:30:02.265 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:26:44.356597+0000", 2024-09-16T09:30:02.265 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 197.85334478199999, 2024-09-16T09:30:02.265 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.122843122, 2024-09-16T09:30:02.265 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:30:02.265 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:30:02.265 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T09:30:02.266 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:30:02.266 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.266 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:30:02.266 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.356597+0000", 2024-09-16T09:30:02.266 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:30:02.266 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.266 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.266 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:30:02.266 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.356597+0000", 2024-09-16T09:30:02.266 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:30:02.266 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.266 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.267 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:30:02.267 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.356598+0000", 2024-09-16T09:30:02.267 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.9400000000000004e-07 2024-09-16T09:30:02.267 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.267 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.267 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:30:02.267 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.356601+0000", 2024-09-16T09:30:02.267 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9330000000000003e-06 2024-09-16T09:30:02.267 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.267 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.268 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:30:02.268 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.478958+0000", 2024-09-16T09:30:02.268 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.12235689499999999 2024-09-16T09:30:02.268 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.268 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.268 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:30:02.268 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.478971+0000", 2024-09-16T09:30:02.268 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2969999999999999e-05 2024-09-16T09:30:02.268 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.268 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.268 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:30:02.268 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.479302+0000", 2024-09-16T09:30:02.268 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00033111099999999998 2024-09-16T09:30:02.269 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.269 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.269 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:30:02.269 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.479311+0000", 2024-09-16T09:30:02.269 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.3620000000000006e-06 2024-09-16T09:30:02.269 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.269 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.269 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:30:02.269 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.479440+0000", 2024-09-16T09:30:02.269 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00012905699999999999 2024-09-16T09:30:02.269 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:30:02.269 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:30:02.269 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:30:02.270 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.270 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.270 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.24698.0:86894 7.a 7:54e07782:::benchmark_data_smithi070_78438_object86893:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-16T09:30:02.270 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:27:06.138810+0000", 2024-09-16T09:30:02.270 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 176.071131307, 2024-09-16T09:30:02.270 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.093325733999999994, 2024-09-16T09:30:02.270 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:30:02.270 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:30:02.270 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:30:02.270 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:30:02.270 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.24698", 2024-09-16T09:30:02.270 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:30:02.270 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 86894 2024-09-16T09:30:02.270 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.271 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:30:02.271 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.271 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:30:02.271 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.138810+0000", 2024-09-16T09:30:02.271 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:30:02.271 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.271 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.271 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:30:02.271 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.138810+0000", 2024-09-16T09:30:02.271 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:30:02.271 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.271 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.271 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:30:02.272 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.138812+0000", 2024-09-16T09:30:02.272 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.378e-06 2024-09-16T09:30:02.272 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.272 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.272 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:30:02.272 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.138833+0000", 2024-09-16T09:30:02.272 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.1407999999999999e-05 2024-09-16T09:30:02.272 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.272 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.272 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:30:02.272 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.138843+0000", 2024-09-16T09:30:02.272 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.5740000000000002e-06 2024-09-16T09:30:02.272 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.273 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.273 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:30:02.273 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.138850+0000", 2024-09-16T09:30:02.273 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.0759999999999997e-06 2024-09-16T09:30:02.273 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.273 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.273 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:30:02.273 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.218942+0000", 2024-09-16T09:30:02.273 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.080092697000000004 2024-09-16T09:30:02.273 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.273 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.273 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:30:02.274 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.219090+0000", 2024-09-16T09:30:02.274 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000147455 2024-09-16T09:30:02.274 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.274 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.274 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 2,5", 2024-09-16T09:30:02.274 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.219244+0000", 2024-09-16T09:30:02.274 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000154344 2024-09-16T09:30:02.274 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.274 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.274 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:30:02.274 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.222668+0000", 2024-09-16T09:30:02.275 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0034239449999999999 2024-09-16T09:30:02.275 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.275 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.275 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:30:02.275 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.226556+0000", 2024-09-16T09:30:02.275 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.003888383 2024-09-16T09:30:02.275 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.275 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.275 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:30:02.275 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.232059+0000", 2024-09-16T09:30:02.275 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0055024050000000001 2024-09-16T09:30:02.275 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.275 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.276 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:30:02.276 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.232100+0000", 2024-09-16T09:30:02.276 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0809999999999997e-05 2024-09-16T09:30:02.276 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.276 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.276 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:30:02.276 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:06.232136+0000", 2024-09-16T09:30:02.276 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6259e-05 2024-09-16T09:30:02.276 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:30:02.276 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:30:02.276 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:30:02.276 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.276 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.276 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.24698.0:124211 7.2 7:4df4e160:::benchmark_data_smithi070_78438_object124210:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-16T09:30:02.276 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:27:41.131290+0000", 2024-09-16T09:30:02.277 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 141.07865152299999, 2024-09-16T09:30:02.277 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.12849359899999999, 2024-09-16T09:30:02.277 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:30:02.277 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:30:02.277 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:30:02.277 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:30:02.277 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.24698", 2024-09-16T09:30:02.277 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:30:02.278 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 124211 2024-09-16T09:30:02.278 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.278 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:30:02.278 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.278 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:30:02.278 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.131290+0000", 2024-09-16T09:30:02.278 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:30:02.278 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.278 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.278 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:30:02.278 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.131290+0000", 2024-09-16T09:30:02.278 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:30:02.278 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.278 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.278 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:30:02.279 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.131291+0000", 2024-09-16T09:30:02.279 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.5000000000000001e-07 2024-09-16T09:30:02.279 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.279 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.279 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:30:02.279 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.131310+0000", 2024-09-16T09:30:02.279 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.8989999999999999e-05 2024-09-16T09:30:02.279 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.279 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.279 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:30:02.279 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.131318+0000", 2024-09-16T09:30:02.279 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.4300000000000006e-06 2024-09-16T09:30:02.279 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.279 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.279 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:30:02.280 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.131323+0000", 2024-09-16T09:30:02.280 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.9150000000000004e-06 2024-09-16T09:30:02.280 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.280 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.280 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:30:02.280 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.133498+0000", 2024-09-16T09:30:02.280 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.002174273 2024-09-16T09:30:02.280 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.280 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.280 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:30:02.280 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.133574+0000", 2024-09-16T09:30:02.280 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6100999999999995e-05 2024-09-16T09:30:02.280 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.280 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.280 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,2", 2024-09-16T09:30:02.281 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.133656+0000", 2024-09-16T09:30:02.281 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.2003999999999996e-05 2024-09-16T09:30:02.281 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.281 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.281 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:30:02.281 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.136823+0000", 2024-09-16T09:30:02.281 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0031675620000000001 2024-09-16T09:30:02.281 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.281 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.281 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:30:02.281 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.137133+0000", 2024-09-16T09:30:02.281 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00030980800000000001 2024-09-16T09:30:02.281 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.281 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.282 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:30:02.282 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.259639+0000", 2024-09-16T09:30:02.282 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.12250636299999999 2024-09-16T09:30:02.282 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.282 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.282 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:30:02.282 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.259740+0000", 2024-09-16T09:30:02.282 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00010049599999999999 2024-09-16T09:30:02.282 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.283 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.283 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:30:02.283 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.259784+0000", 2024-09-16T09:30:02.283 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.3707e-05 2024-09-16T09:30:02.283 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:30:02.283 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:30:02.283 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:30:02.283 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.283 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.283 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.24698.0:124215 7.2 7:4f5f8fec:::benchmark_data_smithi070_78438_object124214:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-16T09:30:02.283 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:27:41.132773+0000", 2024-09-16T09:30:02.284 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 141.07716816199999, 2024-09-16T09:30:02.284 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.13255043599999999, 2024-09-16T09:30:02.284 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:30:02.284 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:30:02.284 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:30:02.284 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:30:02.284 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.24698", 2024-09-16T09:30:02.284 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:30:02.284 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 124215 2024-09-16T09:30:02.284 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.284 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:30:02.284 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.284 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:30:02.285 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.132773+0000", 2024-09-16T09:30:02.285 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:30:02.285 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.285 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.285 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:30:02.285 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.132773+0000", 2024-09-16T09:30:02.285 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:30:02.285 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.285 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.285 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:30:02.285 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.132774+0000", 2024-09-16T09:30:02.285 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.9800000000000002e-07 2024-09-16T09:30:02.285 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.285 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.286 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:30:02.286 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.132794+0000", 2024-09-16T09:30:02.286 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9610999999999999e-05 2024-09-16T09:30:02.286 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.286 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.286 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:30:02.286 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.132805+0000", 2024-09-16T09:30:02.286 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1420000000000001e-05 2024-09-16T09:30:02.286 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.286 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.286 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:30:02.286 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.132812+0000", 2024-09-16T09:30:02.286 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.6780000000000003e-06 2024-09-16T09:30:02.286 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.286 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.286 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:30:02.287 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.135223+0000", 2024-09-16T09:30:02.287 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00241055 2024-09-16T09:30:02.287 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.287 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.287 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:30:02.287 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.135331+0000", 2024-09-16T09:30:02.287 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000108818 2024-09-16T09:30:02.287 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.287 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.288 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,2", 2024-09-16T09:30:02.288 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.135427+0000", 2024-09-16T09:30:02.288 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.5394000000000002e-05 2024-09-16T09:30:02.288 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.288 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.289 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:30:02.289 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.136968+0000", 2024-09-16T09:30:02.289 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0015413860000000001 2024-09-16T09:30:02.289 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.289 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.289 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:30:02.289 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.137281+0000", 2024-09-16T09:30:02.289 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00031255999999999998 2024-09-16T09:30:02.289 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.289 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.289 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:30:02.289 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.265227+0000", 2024-09-16T09:30:02.289 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.127945907 2024-09-16T09:30:02.290 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.290 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.290 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:30:02.290 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.265281+0000", 2024-09-16T09:30:02.290 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.3825000000000001e-05 2024-09-16T09:30:02.290 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.290 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.290 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:30:02.290 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.265324+0000", 2024-09-16T09:30:02.290 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.3288999999999998e-05 2024-09-16T09:30:02.290 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:30:02.290 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:30:02.290 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:30:02.290 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.290 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.291 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.24698.0:203420 7.a 7:52851f87:::benchmark_data_smithi070_78438_object77289:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135)", 2024-09-16T09:30:02.291 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:28:51.960186+0000", 2024-09-16T09:30:02.291 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 70.249755604000001, 2024-09-16T09:30:02.291 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.16906342799999999, 2024-09-16T09:30:02.291 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:30:02.291 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:30:02.291 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "waiting for sub ops", 2024-09-16T09:30:02.291 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:30:02.291 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.24698", 2024-09-16T09:30:02.291 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:30:02.291 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 203420 2024-09-16T09:30:02.291 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.291 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:30:02.291 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.291 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:30:02.292 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.960186+0000", 2024-09-16T09:30:02.292 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:30:02.292 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.292 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.292 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:30:02.292 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.960186+0000", 2024-09-16T09:30:02.292 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:30:02.292 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.292 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.292 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:30:02.292 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.960187+0000", 2024-09-16T09:30:02.292 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.4589999999999999e-06 2024-09-16T09:30:02.293 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.293 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.293 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:30:02.293 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.960192+0000", 2024-09-16T09:30:02.293 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.4449999999999998e-06 2024-09-16T09:30:02.293 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.293 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.293 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:30:02.293 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.960201+0000", 2024-09-16T09:30:02.293 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.5289999999999997e-06 2024-09-16T09:30:02.293 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.293 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.293 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:30:02.293 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.960208+0000", 2024-09-16T09:30:02.294 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.6499999999999999e-06 2024-09-16T09:30:02.294 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.294 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.294 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:30:02.294 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.962224+0000", 2024-09-16T09:30:02.294 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0020158580000000001 2024-09-16T09:30:02.294 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.294 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.295 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:30:02.295 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.962328+0000", 2024-09-16T09:30:02.295 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000103984 2024-09-16T09:30:02.295 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.295 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.295 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 2,5", 2024-09-16T09:30:02.295 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.962437+0000", 2024-09-16T09:30:02.295 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000108879 2024-09-16T09:30:02.295 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.295 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.295 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:30:02.295 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.967574+0000", 2024-09-16T09:30:02.296 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0051370909999999999 2024-09-16T09:30:02.296 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.296 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.296 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:30:02.296 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.129249+0000", 2024-09-16T09:30:02.296 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.16167553300000001 2024-09-16T09:30:02.296 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:30:02.296 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:30:02.296 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:30:02.296 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.296 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.296 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.24698.0:203422 7.a 7:5a4cd988:::benchmark_data_smithi070_78438_object77291:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135)", 2024-09-16T09:30:02.296 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:28:51.966795+0000", 2024-09-16T09:30:02.296 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 70.243146379999999, 2024-09-16T09:30:02.297 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.16245894399999999, 2024-09-16T09:30:02.297 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:30:02.297 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:30:02.297 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "waiting for sub ops", 2024-09-16T09:30:02.297 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:30:02.297 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.24698", 2024-09-16T09:30:02.297 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:30:02.297 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 203422 2024-09-16T09:30:02.297 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.297 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:30:02.297 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.297 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:30:02.297 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.966795+0000", 2024-09-16T09:30:02.298 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:30:02.298 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.298 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.298 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:30:02.298 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.966795+0000", 2024-09-16T09:30:02.298 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:30:02.298 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.298 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.298 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:30:02.298 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.966797+0000", 2024-09-16T09:30:02.298 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.243e-06 2024-09-16T09:30:02.298 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.299 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.299 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:30:02.299 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.966805+0000", 2024-09-16T09:30:02.299 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.7649999999999993e-06 2024-09-16T09:30:02.299 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.299 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.299 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:30:02.299 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.966819+0000", 2024-09-16T09:30:02.299 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3859000000000001e-05 2024-09-16T09:30:02.299 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.300 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.300 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:30:02.300 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.966829+0000", 2024-09-16T09:30:02.300 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.5070000000000006e-06 2024-09-16T09:30:02.300 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.300 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.300 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:30:02.300 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.967612+0000", 2024-09-16T09:30:02.300 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00078302500000000002 2024-09-16T09:30:02.300 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.300 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.300 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:30:02.300 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.967723+0000", 2024-09-16T09:30:02.300 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00011114400000000001 2024-09-16T09:30:02.301 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.301 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.301 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 2,5", 2024-09-16T09:30:02.301 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.968124+0000", 2024-09-16T09:30:02.301 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00040120299999999999 2024-09-16T09:30:02.301 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.301 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.301 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:30:02.301 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.129254+0000", 2024-09-16T09:30:02.301 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.161130198 2024-09-16T09:30:02.301 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:30:02.301 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:30:02.302 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:30:02.302 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.302 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.302 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop_reply(client.24698.0:203422 7.a e135/82)", 2024-09-16T09:30:02.302 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:28:51.979481+0000", 2024-09-16T09:30:02.302 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 70.230460679000004, 2024-09-16T09:30:02.302 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.14986957100000001, 2024-09-16T09:30:02.302 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:30:02.302 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:30:02.302 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T09:30:02.302 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:30:02.302 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.302 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:30:02.303 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.979481+0000", 2024-09-16T09:30:02.303 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:30:02.303 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.303 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.303 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:30:02.303 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.979481+0000", 2024-09-16T09:30:02.303 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:30:02.303 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.303 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.303 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:30:02.303 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.979482+0000", 2024-09-16T09:30:02.303 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.4499999999999996e-07 2024-09-16T09:30:02.303 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.303 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.303 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:30:02.304 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.979485+0000", 2024-09-16T09:30:02.304 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3969999999999998e-06 2024-09-16T09:30:02.304 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.304 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.304 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:30:02.304 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.983819+0000", 2024-09-16T09:30:02.304 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0043340779999999999 2024-09-16T09:30:02.304 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.304 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.304 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:30:02.304 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.983991+0000", 2024-09-16T09:30:02.304 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00017179599999999999 2024-09-16T09:30:02.304 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.304 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.304 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:30:02.305 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.129339+0000", 2024-09-16T09:30:02.305 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.14534812899999999 2024-09-16T09:30:02.305 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.305 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.305 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:30:02.305 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.129350+0000", 2024-09-16T09:30:02.305 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1326e-05 2024-09-16T09:30:02.305 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:30:02.305 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:30:02.305 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:30:02.305 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.305 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.305 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.24698.0:203428 7.f e135/82 7:fc13f757:::benchmark_data_smithi070_78438_object77297:head v 135'12855, mlcod=135'12853)", 2024-09-16T09:30:02.305 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:28:51.979535+0000", 2024-09-16T09:30:02.306 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 70.230406518999999, 2024-09-16T09:30:02.306 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.14576935799999999, 2024-09-16T09:30:02.306 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:30:02.306 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:30:02.306 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:30:02.306 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:30:02.306 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.306 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:30:02.306 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.979535+0000", 2024-09-16T09:30:02.306 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:30:02.306 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.306 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.306 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:30:02.306 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.979535+0000", 2024-09-16T09:30:02.306 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:30:02.306 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.307 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.307 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:30:02.307 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.979535+0000", 2024-09-16T09:30:02.307 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.5600000000000002e-07 2024-09-16T09:30:02.307 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.307 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.307 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:30:02.307 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.979540+0000", 2024-09-16T09:30:02.307 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.8489999999999997e-06 2024-09-16T09:30:02.307 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.307 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.307 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:30:02.308 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.984131+0000", 2024-09-16T09:30:02.308 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.004591365 2024-09-16T09:30:02.308 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.308 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.308 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:30:02.308 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.984138+0000", 2024-09-16T09:30:02.308 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.9770000000000003e-06 2024-09-16T09:30:02.308 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.308 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.308 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:30:02.308 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.119540+0000", 2024-09-16T09:30:02.308 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.135401732 2024-09-16T09:30:02.308 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.309 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.309 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:30:02.309 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.119558+0000", 2024-09-16T09:30:02.309 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.8306000000000001e-05 2024-09-16T09:30:02.309 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.309 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.309 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:02.310 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:02.310 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:30:02.311 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:30:02.311 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.125260+0000", 2024-09-16T09:30:02.311 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.005701472 2024-09-16T09:30:02.311 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.311 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.311 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:30:02.311 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.125304+0000", 2024-09-16T09:30:02.311 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.4400999999999997e-05 2024-09-16T09:30:02.311 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:30:02.311 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:30:02.311 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:30:02.311 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.312 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.312 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop_reply(client.24698.0:203420 7.a e135/82)", 2024-09-16T09:30:02.312 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:28:51.979576+0000", 2024-09-16T09:30:02.312 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 70.230365899999995, 2024-09-16T09:30:02.312 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.14971880700000001, 2024-09-16T09:30:02.312 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:30:02.312 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:30:02.312 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T09:30:02.312 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:30:02.312 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.313 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:30:02.313 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.979576+0000", 2024-09-16T09:30:02.313 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:30:02.313 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.313 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.313 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:30:02.313 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.979576+0000", 2024-09-16T09:30:02.313 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:30:02.313 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.313 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.313 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:30:02.313 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.979576+0000", 2024-09-16T09:30:02.313 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.5899999999999996e-07 2024-09-16T09:30:02.314 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.314 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.314 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:30:02.314 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.979582+0000", 2024-09-16T09:30:02.314 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8050000000000003e-06 2024-09-16T09:30:02.314 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.314 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.314 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:30:02.314 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.980649+0000", 2024-09-16T09:30:02.314 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001066865 2024-09-16T09:30:02.314 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.314 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.314 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:30:02.314 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.980693+0000", 2024-09-16T09:30:02.315 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.4469999999999999e-05 2024-09-16T09:30:02.315 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.315 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.315 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:30:02.315 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.129280+0000", 2024-09-16T09:30:02.315 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.14858617700000001 2024-09-16T09:30:02.315 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.315 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.315 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:30:02.315 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.129294+0000", 2024-09-16T09:30:02.315 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.4831000000000001e-05 2024-09-16T09:30:02.315 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:30:02.315 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:30:02.316 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:30:02.316 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.316 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.316 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.24698.0:203433 7.c e136/82 7:3eef1e44:::benchmark_data_smithi070_78438_object77302:head v 136'12615, mlcod=135'12613)", 2024-09-16T09:30:02.316 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:28:51.983475+0000", 2024-09-16T09:30:02.316 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 70.226466161999994, 2024-09-16T09:30:02.316 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.14347006700000001, 2024-09-16T09:30:02.316 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:30:02.316 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:30:02.316 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:30:02.316 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:30:02.316 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.316 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:30:02.317 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.983475+0000", 2024-09-16T09:30:02.317 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:30:02.317 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.317 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.317 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:30:02.317 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.983475+0000", 2024-09-16T09:30:02.317 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:30:02.317 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.317 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.317 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:30:02.317 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.983476+0000", 2024-09-16T09:30:02.317 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.08e-06 2024-09-16T09:30:02.317 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.317 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.317 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:30:02.318 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.983483+0000", 2024-09-16T09:30:02.318 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.7510000000000002e-06 2024-09-16T09:30:02.318 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.318 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.318 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:30:02.318 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.983495+0000", 2024-09-16T09:30:02.318 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1326e-05 2024-09-16T09:30:02.318 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.318 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.318 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:30:02.318 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.983502+0000", 2024-09-16T09:30:02.318 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3540000000000002e-06 2024-09-16T09:30:02.318 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.318 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.318 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:30:02.319 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.118393+0000", 2024-09-16T09:30:02.319 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.13489093399999999 2024-09-16T09:30:02.319 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.319 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.319 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:30:02.319 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.118414+0000", 2024-09-16T09:30:02.319 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0696999999999999e-05 2024-09-16T09:30:02.319 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.319 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.319 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:30:02.319 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.126864+0000", 2024-09-16T09:30:02.319 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0084504490000000005 2024-09-16T09:30:02.319 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.319 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.319 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:30:02.319 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.126945+0000", 2024-09-16T09:30:02.320 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.1476000000000004e-05 2024-09-16T09:30:02.320 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:30:02.320 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:30:02.320 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:30:02.320 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.320 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.320 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop_reply(client.24698.0:203422 7.a e135/82)", 2024-09-16T09:30:02.320 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:28:51.983541+0000", 2024-09-16T09:30:02.320 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 70.226400462000001, 2024-09-16T09:30:02.321 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.14578239300000001, 2024-09-16T09:30:02.321 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:30:02.321 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:30:02.321 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T09:30:02.321 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:30:02.321 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.321 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:30:02.321 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.983541+0000", 2024-09-16T09:30:02.321 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:30:02.321 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.321 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.321 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:30:02.321 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.983541+0000", 2024-09-16T09:30:02.321 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:30:02.321 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.322 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.322 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:30:02.322 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.983542+0000", 2024-09-16T09:30:02.322 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5600000000000001e-07 2024-09-16T09:30:02.322 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.322 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.322 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:30:02.322 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.983544+0000", 2024-09-16T09:30:02.322 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.932e-06 2024-09-16T09:30:02.322 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.322 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.322 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:30:02.322 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.983552+0000", 2024-09-16T09:30:02.322 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.4699999999999996e-06 2024-09-16T09:30:02.322 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.323 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.323 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:30:02.323 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.983557+0000", 2024-09-16T09:30:02.323 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.3199999999999999e-06 2024-09-16T09:30:02.323 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.323 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.323 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:30:02.323 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.129312+0000", 2024-09-16T09:30:02.323 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.14575458999999999 2024-09-16T09:30:02.323 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.323 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.324 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:30:02.324 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.129323+0000", 2024-09-16T09:30:02.324 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1625e-05 2024-09-16T09:30:02.324 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:30:02.324 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:30:02.324 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:30:02.324 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.324 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.324 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.24698.0:203434 7.2 7:4b27635e:::benchmark_data_smithi070_78438_object77303:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135)", 2024-09-16T09:30:02.324 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:28:51.985566+0000", 2024-09-16T09:30:02.324 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 70.224375797999997, 2024-09-16T09:30:02.324 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.16551761700000001, 2024-09-16T09:30:02.324 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:30:02.324 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:30:02.325 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:30:02.325 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:30:02.325 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.24698", 2024-09-16T09:30:02.325 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:30:02.325 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 203434 2024-09-16T09:30:02.325 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.325 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:30:02.325 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.325 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:30:02.325 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.985566+0000", 2024-09-16T09:30:02.325 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:30:02.325 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.326 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.326 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:30:02.326 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.985566+0000", 2024-09-16T09:30:02.326 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:30:02.326 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.326 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.326 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:30:02.326 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.985567+0000", 2024-09-16T09:30:02.326 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5689999999999999e-06 2024-09-16T09:30:02.326 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.326 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.326 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:30:02.326 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.985573+0000", 2024-09-16T09:30:02.326 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.339e-06 2024-09-16T09:30:02.327 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.327 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.327 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:30:02.327 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.117695+0000", 2024-09-16T09:30:02.327 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.13212251 2024-09-16T09:30:02.327 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.327 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.327 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:30:02.327 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.117706+0000", 2024-09-16T09:30:02.327 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0431000000000001e-05 2024-09-16T09:30:02.327 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.327 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.327 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:30:02.327 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.130808+0000", 2024-09-16T09:30:02.328 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.013101936999999999 2024-09-16T09:30:02.328 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.328 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.328 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:30:02.328 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.130930+0000", 2024-09-16T09:30:02.328 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00012259099999999999 2024-09-16T09:30:02.328 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.328 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.328 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,2", 2024-09-16T09:30:02.328 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.131054+0000", 2024-09-16T09:30:02.328 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000123695 2024-09-16T09:30:02.328 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.328 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.328 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:30:02.328 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.137688+0000", 2024-09-16T09:30:02.329 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0066342689999999999 2024-09-16T09:30:02.329 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.329 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.329 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:30:02.329 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.137822+0000", 2024-09-16T09:30:02.329 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00013432900000000001 2024-09-16T09:30:02.329 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.329 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.329 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:30:02.329 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.150917+0000", 2024-09-16T09:30:02.329 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.013094252000000001 2024-09-16T09:30:02.329 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.329 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.330 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:30:02.330 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.151020+0000", 2024-09-16T09:30:02.330 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000103112 2024-09-16T09:30:02.330 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:02.330 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:02.330 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:30:02.330 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.151083+0000", 2024-09-16T09:30:02.330 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.3583000000000004e-05 2024-09-16T09:30:02.330 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:30:02.330 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:30:02.330 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:30:02.330 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:30:02.330 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:30:02.331 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T09:30:02.411 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:30:02.444 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:02.444 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:02.445 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:30:02.545 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:30:02.579 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:02.579 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:02.579 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:30:02.680 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:30:02.710 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:02.710 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:02.711 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:30:02.811 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:30:02.849 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:02.850 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:02.851 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:30:02.900 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T09:30:02.910 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:30:02 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:30:02.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:02 smithi084 ceph-mon[32629]: pgmap v222: 158 pgs: 158 active+clean; 344 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 43 MiB/s wr, 687 op/s 2024-09-16T09:30:02.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:02 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:30:02.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:02 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:30:02.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:02 smithi070 ceph-mon[30388]: pgmap v222: 158 pgs: 158 active+clean; 344 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 43 MiB/s wr, 687 op/s 2024-09-16T09:30:02.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:02 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:30:02.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:02 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:30:02.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:02 smithi070 ceph-mon[40691]: pgmap v222: 158 pgs: 158 active+clean; 344 MiB data, 1.7 GiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 43 MiB/s wr, 687 op/s 2024-09-16T09:30:02.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:02 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:30:02.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:02 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:30:02.949 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 7 16 20444 20428 182.354 159.375 0.00506405 0.00547808 2024-09-16T09:30:02.951 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:30:02.997 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:02.998 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:02.998 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:30:03.099 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:30:03.132 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:03.133 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:03.134 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:30:03.234 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:30:03.270 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:03.271 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:03.271 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:30:03.372 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:30:03.410 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:03.411 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:03.411 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:30:03.460 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.4 dump_ops_in_flight 2024-09-16T09:30:03.512 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:30:03.516 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:30:03.527 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:30:03.545 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:03.546 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:03.546 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:30:03.576 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:30:03.646 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:30:03.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:03 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:30:03.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:03 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:30:03.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:03 smithi070 ceph-mon[30388]: osdmap e158: 8 total, 8 up, 8 in 2024-09-16T09:30:03.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:03 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:30:03.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:03 smithi070 ceph-mon[30388]: 3.19 scrub starts 2024-09-16T09:30:03.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:03 smithi070 ceph-mon[30388]: 3.19 scrub ok 2024-09-16T09:30:03.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:03 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:30:03.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:03 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:30:03.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:03 smithi070 ceph-mon[40691]: osdmap e158: 8 total, 8 up, 8 in 2024-09-16T09:30:03.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:03 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:30:03.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:03 smithi070 ceph-mon[40691]: 3.19 scrub starts 2024-09-16T09:30:03.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:03 smithi070 ceph-mon[40691]: 3.19 scrub ok 2024-09-16T09:30:03.691 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:03.693 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:03.693 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:30:03.794 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:30:03.835 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:03.835 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:03.836 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:30:03.864 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:30:03.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:03 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:30:03.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:03 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:30:03.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:03 smithi084 ceph-mon[32629]: osdmap e158: 8 total, 8 up, 8 in 2024-09-16T09:30:03.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:03 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:30:03.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:03 smithi084 ceph-mon[32629]: 3.19 scrub starts 2024-09-16T09:30:03.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:03 smithi084 ceph-mon[32629]: 3.19 scrub ok 2024-09-16T09:30:03.937 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:30:03.951 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 8 16 22916 22900 178.821 154.5 0.0111406 0.00558593 2024-09-16T09:30:03.978 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:03.978 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:03.978 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:30:04.079 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:30:04.118 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:04.119 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:04.119 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:30:04.220 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:30:04.259 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:04.260 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:04.260 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:30:04.361 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:30:04.398 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:04.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:04.399 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:30:04.481 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-09-16T09:30:04.481 INFO:teuthology.orchestra.run.smithi084.stdout: "ops": [ 2024-09-16T09:30:04.482 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:04.482 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_repop(client.25064.0:24156 8.5 e158/152 8:abfef68f:::benchmark_data_smithi070_107919_object24155:head v 158'1458, mlcod=158'1456)", 2024-09-16T09:30:04.482 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:30:04.475908+0000", 2024-09-16T09:30:04.482 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 0.0019547969999999999, 2024-09-16T09:30:04.482 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0019812829999999999, 2024-09-16T09:30:04.482 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:30:04.482 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:30:04.482 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:30:04.482 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:30:04.482 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:04.482 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:30:04.483 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:04.475908+0000", 2024-09-16T09:30:04.483 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:30:04.483 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:04.483 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:04.483 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:30:04.483 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:04.475908+0000", 2024-09-16T09:30:04.483 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:30:04.483 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:04.483 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:04.483 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:30:04.483 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:04.475909+0000", 2024-09-16T09:30:04.483 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.7800000000000001e-07 2024-09-16T09:30:04.483 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:04.484 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:04.484 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:30:04.484 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:04.475930+0000", 2024-09-16T09:30:04.484 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.1132999999999999e-05 2024-09-16T09:30:04.484 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:04.484 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:04.484 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:30:04.484 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:04.475939+0000", 2024-09-16T09:30:04.484 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.7539999999999995e-06 2024-09-16T09:30:04.485 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:04.485 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:04.485 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:30:04.485 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:04.475945+0000", 2024-09-16T09:30:04.485 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.1850000000000001e-06 2024-09-16T09:30:04.485 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:04.485 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:04.485 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:30:04.485 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:04.476412+0000", 2024-09-16T09:30:04.486 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000467164 2024-09-16T09:30:04.486 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:04.486 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:04.486 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:30:04.486 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:04.476429+0000", 2024-09-16T09:30:04.486 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.6952999999999998e-05 2024-09-16T09:30:04.486 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:04.486 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:04.486 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:30:04.486 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:04.477882+0000", 2024-09-16T09:30:04.486 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.001452591 2024-09-16T09:30:04.486 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:04.486 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:30:04.486 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:04.487 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:04.487 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-09-16T09:30:04.487 INFO:teuthology.orchestra.run.smithi084.stdout: "num_ops": 1 2024-09-16T09:30:04.487 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-09-16T09:30:04.500 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:30:04.541 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:04.542 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:04.542 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:30:04.643 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:30:04.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:04 smithi084 ceph-mon[32629]: pgmap v224: 158 pgs: 158 active+clean; 1.2 GiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 1.2 KiB/s rd, 155 MiB/s wr, 2.48k op/s 2024-09-16T09:30:04.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:04 smithi084 ceph-mon[32629]: 3.1a scrub starts 2024-09-16T09:30:04.679 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:04.680 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:04.680 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:30:04.781 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:30:04.801 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:04 smithi070 ceph-mon[30388]: pgmap v224: 158 pgs: 158 active+clean; 1.2 GiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 1.2 KiB/s rd, 155 MiB/s wr, 2.48k op/s 2024-09-16T09:30:04.801 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:04 smithi070 ceph-mon[30388]: 3.1a scrub starts 2024-09-16T09:30:04.801 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:04 smithi070 ceph-mon[30388]: 3.1a scrub ok 2024-09-16T09:30:04.801 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:04 smithi070 ceph-mon[30388]: 3.1f scrub starts 2024-09-16T09:30:04.801 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:04 smithi070 ceph-mon[30388]: 3.1f scrub ok 2024-09-16T09:30:04.801 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:04 smithi070 ceph-mon[30388]: 1.3 scrub starts 2024-09-16T09:30:04.802 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:04 smithi070 ceph-mon[30388]: 1.3 scrub ok 2024-09-16T09:30:04.802 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:04 smithi070 ceph-mon[30388]: 6.f scrub starts 2024-09-16T09:30:04.802 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:04 smithi070 ceph-mon[30388]: 6.f scrub ok 2024-09-16T09:30:04.802 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:04 smithi070 ceph-mon[40691]: pgmap v224: 158 pgs: 158 active+clean; 1.2 GiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 1.2 KiB/s rd, 155 MiB/s wr, 2.48k op/s 2024-09-16T09:30:04.802 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:04 smithi070 ceph-mon[40691]: 3.1a scrub starts 2024-09-16T09:30:04.802 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:04 smithi070 ceph-mon[40691]: 3.1a scrub ok 2024-09-16T09:30:04.802 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:04 smithi070 ceph-mon[40691]: 3.1f scrub starts 2024-09-16T09:30:04.802 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:04 smithi070 ceph-mon[40691]: 3.1f scrub ok 2024-09-16T09:30:04.802 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:04 smithi070 ceph-mon[40691]: 1.3 scrub starts 2024-09-16T09:30:04.802 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:04 smithi070 ceph-mon[40691]: 1.3 scrub ok 2024-09-16T09:30:04.802 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:04 smithi070 ceph-mon[40691]: 6.f scrub starts 2024-09-16T09:30:04.802 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:04 smithi070 ceph-mon[40691]: 6.f scrub ok 2024-09-16T09:30:04.814 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:04.815 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:04.816 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:30:04.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:04 smithi084 ceph-mon[32629]: 3.1a scrub ok 2024-09-16T09:30:04.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:04 smithi084 ceph-mon[32629]: 3.1f scrub starts 2024-09-16T09:30:04.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:04 smithi084 ceph-mon[32629]: 3.1f scrub ok 2024-09-16T09:30:04.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:04 smithi084 ceph-mon[32629]: 1.3 scrub starts 2024-09-16T09:30:04.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:04 smithi084 ceph-mon[32629]: 1.3 scrub ok 2024-09-16T09:30:04.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:04 smithi084 ceph-mon[32629]: 6.f scrub starts 2024-09-16T09:30:04.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:04 smithi084 ceph-mon[32629]: 6.f scrub ok 2024-09-16T09:30:04.916 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:30:04.951 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:04.951 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:04.952 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:30:04.952 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 9 15 25221 25206 174.966 144.125 0.00400404 0.00571027 2024-09-16T09:30:05.035 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:30:05.036 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:30:05.036 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:30:05.036 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:30:05.052 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:30:05.067 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:30:05.067 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:30:05.082 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:05.082 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:05.083 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:30:05.110 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:30:05.110 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:30:05.146 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:30:05.147 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:30:05.184 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:30:05.185 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:30:05.185 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:30:05.223 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:05.223 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:05.224 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:30:05.224 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:30:05.224 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:30:05.258 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:30:05.259 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:30:05.325 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:30:05.358 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:05.358 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:05.359 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:30:05.379 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.4 dump_blocked_ops 2024-09-16T09:30:05.459 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:30:05.492 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:05.494 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:05.494 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:30:05.595 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:30:05.635 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:05.635 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:05.636 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:30:05.668 INFO:teuthology.orchestra.run.smithi070.stderr:noscrub is set 2024-09-16T09:30:05.671 INFO:teuthology.orchestra.run.smithi070.stderr:set pool 1 pg_num to 21 2024-09-16T09:30:05.737 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:30:05.772 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:05.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:05.774 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:30:05.789 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:30:05.875 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:30:05.914 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:05.915 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:05.915 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:30:05.917 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:05 smithi070 ceph-mon[40691]: 4.d scrub starts 2024-09-16T09:30:05.917 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:05 smithi070 ceph-mon[40691]: 4.d scrub ok 2024-09-16T09:30:05.917 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:05 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/300234240' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:30:05.917 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:05 smithi070 ceph-mon[40691]: 4.2 scrub starts 2024-09-16T09:30:05.917 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:05 smithi070 ceph-mon[40691]: 4.2 scrub ok 2024-09-16T09:30:05.918 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:05 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3714776741' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:30:05.918 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:05 smithi070 ceph-mon[40691]: 5.9 scrub starts 2024-09-16T09:30:05.918 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:05 smithi070 ceph-mon[40691]: 5.9 scrub ok 2024-09-16T09:30:05.918 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:05 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3340864835' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "21"} : dispatch 2024-09-16T09:30:05.918 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:05 smithi070 ceph-mon[30388]: 4.d scrub starts 2024-09-16T09:30:05.918 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:05 smithi070 ceph-mon[30388]: 4.d scrub ok 2024-09-16T09:30:05.918 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:05 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/300234240' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:30:05.918 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:05 smithi070 ceph-mon[30388]: 4.2 scrub starts 2024-09-16T09:30:05.919 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:05 smithi070 ceph-mon[30388]: 4.2 scrub ok 2024-09-16T09:30:05.919 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:05 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3714776741' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:30:05.919 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:05 smithi070 ceph-mon[30388]: 5.9 scrub starts 2024-09-16T09:30:05.919 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:05 smithi070 ceph-mon[30388]: 5.9 scrub ok 2024-09-16T09:30:05.919 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:05 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3340864835' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "21"} : dispatch 2024-09-16T09:30:05.919 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:05 smithi084 ceph-mon[32629]: 4.d scrub starts 2024-09-16T09:30:05.919 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:05 smithi084 ceph-mon[32629]: 4.d scrub ok 2024-09-16T09:30:05.920 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:05 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/300234240' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:30:05.920 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:05 smithi084 ceph-mon[32629]: 4.2 scrub starts 2024-09-16T09:30:05.920 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:05 smithi084 ceph-mon[32629]: 4.2 scrub ok 2024-09-16T09:30:05.920 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:05 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3714776741' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:30:05.920 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:05 smithi084 ceph-mon[32629]: 5.9 scrub starts 2024-09-16T09:30:05.920 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:05 smithi084 ceph-mon[32629]: 5.9 scrub ok 2024-09-16T09:30:05.920 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:05 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3340864835' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "21"} : dispatch 2024-09-16T09:30:05.952 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 10 16 27495 27479 171.676 142.062 0.00349693 0.00581935 2024-09-16T09:30:06.016 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:30:06.054 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:06.055 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:06.055 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:30:06.156 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:30:06.177 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:06.178 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:06.178 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:30:06.279 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:30:06.309 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:06.310 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:06.310 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:30:06.411 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:30:06.440 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:06.441 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:06.441 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:30:06.542 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:30:06.562 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:06.563 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:06.563 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:30:06.664 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:30:06.697 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:06.698 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:06.698 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:30:06.799 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:30:06.832 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:06.833 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:06.833 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:30:06.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:06 smithi070 ceph-mon[30388]: pgmap v225: 158 pgs: 158 active+clean; 1.2 GiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 1.2 KiB/s rd, 153 MiB/s wr, 2.45k op/s 2024-09-16T09:30:06.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:06 smithi070 ceph-mon[30388]: 4.e scrub starts 2024-09-16T09:30:06.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:06 smithi070 ceph-mon[30388]: 4.e scrub ok 2024-09-16T09:30:06.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:06 smithi070 ceph-mon[30388]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:30:06.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:06 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/300234240' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-16T09:30:06.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:06 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3340864835' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "21"}]': finished 2024-09-16T09:30:06.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:06 smithi070 ceph-mon[30388]: osdmap e159: 8 total, 8 up, 8 in 2024-09-16T09:30:06.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:06 smithi070 ceph-mon[40691]: pgmap v225: 158 pgs: 158 active+clean; 1.2 GiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 1.2 KiB/s rd, 153 MiB/s wr, 2.45k op/s 2024-09-16T09:30:06.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:06 smithi070 ceph-mon[40691]: 4.e scrub starts 2024-09-16T09:30:06.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:06 smithi070 ceph-mon[40691]: 4.e scrub ok 2024-09-16T09:30:06.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:06 smithi070 ceph-mon[40691]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:30:06.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:06 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/300234240' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-16T09:30:06.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:06 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3340864835' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "21"}]': finished 2024-09-16T09:30:06.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:06 smithi070 ceph-mon[40691]: osdmap e159: 8 total, 8 up, 8 in 2024-09-16T09:30:06.922 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:06 smithi084 ceph-mon[32629]: pgmap v225: 158 pgs: 158 active+clean; 1.2 GiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 1.2 KiB/s rd, 153 MiB/s wr, 2.45k op/s 2024-09-16T09:30:06.922 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:06 smithi084 ceph-mon[32629]: 4.e scrub starts 2024-09-16T09:30:06.922 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:06 smithi084 ceph-mon[32629]: 4.e scrub ok 2024-09-16T09:30:06.923 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:06 smithi084 ceph-mon[32629]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:30:06.923 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:06 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/300234240' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-16T09:30:06.923 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:06 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3340864835' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "21"}]': finished 2024-09-16T09:30:06.923 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:06 smithi084 ceph-mon[32629]: osdmap e159: 8 total, 8 up, 8 in 2024-09-16T09:30:06.935 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:30:06.951 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 11 16 29645 29629 168.285 134.375 0.00312277 0.00593643 2024-09-16T09:30:06.969 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:06.970 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:06.970 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:30:07.071 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:30:07.104 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:07.104 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:07.105 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:30:07.205 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:30:07.227 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:07.228 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:07.228 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:30:07.329 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:30:07.366 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:07.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:07.367 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:30:07.467 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:30:07.499 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:07.500 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:07.500 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:30:07.601 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:30:07.638 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:07.639 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:07.640 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:30:07.740 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:30:07.766 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:07.767 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:07.767 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:30:07.868 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:30:07.903 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:07.903 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:07.904 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:30:07.952 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 12 15 31412 31397 163.47 110.5 0.00939339 0.00611234 2024-09-16T09:30:07.974 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T09:30:07.995 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:07 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "21"} : dispatch 2024-09-16T09:30:07.995 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:07 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "21"} : dispatch 2024-09-16T09:30:07.995 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:07 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "21"} : dispatch 2024-09-16T09:30:07.996 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:07 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "21"} : dispatch 2024-09-16T09:30:07.996 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:07 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "21"} : dispatch 2024-09-16T09:30:07.996 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:07 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "21"} : dispatch 2024-09-16T09:30:08.005 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:30:08.029 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:08.030 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:08.030 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:30:08.131 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:30:08.152 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:08.153 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:08.153 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:30:08.254 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:30:08.293 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:08.294 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:08.294 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:30:08.395 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:30:08.425 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:08.425 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:08.426 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:30:08.526 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:30:08.560 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:08.561 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:08.561 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:30:08.662 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:30:08.683 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:08.684 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:08.684 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:30:08.785 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:30:08.806 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:08.806 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:08.807 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:30:08.907 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:30:08.943 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:08.944 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:08.944 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:30:08.952 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 13 16 32757 32741 157.358 84 0.00635945 0.00635009 2024-09-16T09:30:09.038 INFO:teuthology.orchestra.run.smithi084.stdout:ERROR: (22) Invalid argument 2024-09-16T09:30:09.039 INFO:teuthology.orchestra.run.smithi084.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:30:09.039 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:08 smithi084 ceph-mon[32629]: pgmap v227: 158 pgs: 158 active+clean; 1.6 GiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 156 MiB/s wr, 2.49k op/s 2024-09-16T09:30:09.039 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:08 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "21"}]': finished 2024-09-16T09:30:09.039 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:08 smithi084 ceph-mon[32629]: osdmap e160: 8 total, 8 up, 8 in 2024-09-16T09:30:09.039 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:08 smithi084 ceph-mon[32629]: osdmap e161: 8 total, 8 up, 8 in 2024-09-16T09:30:09.045 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:30:09.066 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:09.066 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:09.066 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:30:09.167 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:30:09.170 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:08 smithi070 ceph-mon[30388]: pgmap v227: 158 pgs: 158 active+clean; 1.6 GiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 156 MiB/s wr, 2.49k op/s 2024-09-16T09:30:09.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:08 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "21"}]': finished 2024-09-16T09:30:09.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:08 smithi070 ceph-mon[30388]: osdmap e160: 8 total, 8 up, 8 in 2024-09-16T09:30:09.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:08 smithi070 ceph-mon[30388]: osdmap e161: 8 total, 8 up, 8 in 2024-09-16T09:30:09.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:08 smithi070 ceph-mon[40691]: pgmap v227: 158 pgs: 158 active+clean; 1.6 GiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 156 MiB/s wr, 2.49k op/s 2024-09-16T09:30:09.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:08 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "21"}]': finished 2024-09-16T09:30:09.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:08 smithi070 ceph-mon[40691]: osdmap e160: 8 total, 8 up, 8 in 2024-09-16T09:30:09.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:08 smithi070 ceph-mon[40691]: osdmap e161: 8 total, 8 up, 8 in 2024-09-16T09:30:09.192 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:09.192 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:09.192 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:30:09.293 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:30:09.330 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:09.331 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:09.331 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:30:09.421 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-16T09:30:09.432 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:30:09.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:09.486 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:09.486 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:30:09.587 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:30:09.625 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:09.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:09.627 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:30:09.728 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:30:09.749 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:09.750 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:09.751 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:30:09.851 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:30:09.880 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:09.881 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:09.881 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:30:09.952 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 14 16 34034 34018 151.819 79.8125 0.00185839 0.0065761 2024-09-16T09:30:09.982 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:30:10.009 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:10.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:10.010 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:30:10.111 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:30:10.143 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:10.144 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:10.144 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:30:10.245 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:30:10.270 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:10.270 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:10.270 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:30:10.371 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:30:10.403 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:10.404 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:10.404 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:30:10.505 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:30:10.543 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:10.544 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:10.544 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:30:10.645 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:30:10.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:10 smithi084 ceph-mon[32629]: 1.4 scrub starts 2024-09-16T09:30:10.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:10 smithi084 ceph-mon[32629]: 1.2 scrub starts 2024-09-16T09:30:10.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:10 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:30:10.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:10 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:30:10.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:10 smithi084 ceph-mon[32629]: 1.7 scrub starts 2024-09-16T09:30:10.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:10 smithi084 ceph-mon[32629]: 1.1 scrub starts 2024-09-16T09:30:10.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:10 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "21"} : dispatch 2024-09-16T09:30:10.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:10 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "21"} : dispatch 2024-09-16T09:30:10.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:10 smithi084 ceph-mon[32629]: 1.5 scrub starts 2024-09-16T09:30:10.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:10 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "21"}]': finished 2024-09-16T09:30:10.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:10 smithi084 ceph-mon[32629]: osdmap e162: 8 total, 8 up, 8 in 2024-09-16T09:30:10.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:10 smithi084 ceph-mon[32629]: 1.3 scrub starts 2024-09-16T09:30:10.670 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:10 smithi070 ceph-mon[40691]: 1.4 scrub starts 2024-09-16T09:30:10.670 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:10 smithi070 ceph-mon[40691]: 1.2 scrub starts 2024-09-16T09:30:10.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:10 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:30:10.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:10 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:30:10.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:10 smithi070 ceph-mon[40691]: 1.7 scrub starts 2024-09-16T09:30:10.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:10 smithi070 ceph-mon[40691]: 1.1 scrub starts 2024-09-16T09:30:10.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:10 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "21"} : dispatch 2024-09-16T09:30:10.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:10 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "21"} : dispatch 2024-09-16T09:30:10.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:10 smithi070 ceph-mon[40691]: 1.5 scrub starts 2024-09-16T09:30:10.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:10 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "21"}]': finished 2024-09-16T09:30:10.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:10 smithi070 ceph-mon[40691]: osdmap e162: 8 total, 8 up, 8 in 2024-09-16T09:30:10.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:10 smithi070 ceph-mon[40691]: 1.3 scrub starts 2024-09-16T09:30:10.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:10 smithi070 ceph-mon[30388]: 1.4 scrub starts 2024-09-16T09:30:10.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:10 smithi070 ceph-mon[30388]: 1.2 scrub starts 2024-09-16T09:30:10.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:10 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:30:10.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:10 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:30:10.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:10 smithi070 ceph-mon[30388]: 1.7 scrub starts 2024-09-16T09:30:10.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:10 smithi070 ceph-mon[30388]: 1.1 scrub starts 2024-09-16T09:30:10.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:10 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "21"} : dispatch 2024-09-16T09:30:10.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:10 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "21"} : dispatch 2024-09-16T09:30:10.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:10 smithi070 ceph-mon[30388]: 1.5 scrub starts 2024-09-16T09:30:10.673 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:10 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "21"}]': finished 2024-09-16T09:30:10.673 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:10 smithi070 ceph-mon[30388]: osdmap e162: 8 total, 8 up, 8 in 2024-09-16T09:30:10.673 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:10 smithi070 ceph-mon[30388]: 1.3 scrub starts 2024-09-16T09:30:10.676 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:10.677 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:10.678 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:30:10.778 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:30:10.814 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:10.814 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:10.815 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:30:10.915 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:30:10.936 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:10.937 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:10.937 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:30:10.952 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 15 16 35488 35472 147.757 90.875 0.00486548 0.00673427 2024-09-16T09:30:11.039 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:30:11.074 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:11.075 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:11.076 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:30:11.176 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:30:11.212 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:11.213 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:11.213 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:30:11.314 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:30:11.348 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:11.348 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:11.349 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:30:11.449 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:30:11.470 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:11.470 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:11.471 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:30:11.571 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:30:11.614 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:11.614 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:11.615 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:30:11.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:11 smithi084 ceph-mon[32629]: pgmap v230: 168 pgs: 2 peering, 10 unknown, 156 active+clean; 1.9 GiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 115 MiB/s wr, 1.84k op/s 2024-09-16T09:30:11.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:11 smithi084 ceph-mon[32629]: 1.4 scrub starts 2024-09-16T09:30:11.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:11 smithi084 ceph-mon[32629]: 1.2 scrub starts 2024-09-16T09:30:11.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:11 smithi084 ceph-mon[32629]: 1.7 scrub starts 2024-09-16T09:30:11.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:11 smithi084 ceph-mon[32629]: 1.1 scrub starts 2024-09-16T09:30:11.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:11 smithi084 ceph-mon[32629]: 1.5 scrub starts 2024-09-16T09:30:11.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:11 smithi084 ceph-mon[32629]: osdmap e163: 8 total, 8 up, 8 in 2024-09-16T09:30:11.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:11 smithi084 ceph-mon[32629]: 1.13 scrub starts 2024-09-16T09:30:11.670 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:11 smithi070 ceph-mon[40691]: pgmap v230: 168 pgs: 2 peering, 10 unknown, 156 active+clean; 1.9 GiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 115 MiB/s wr, 1.84k op/s 2024-09-16T09:30:11.670 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:11 smithi070 ceph-mon[40691]: 1.4 scrub starts 2024-09-16T09:30:11.670 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:11 smithi070 ceph-mon[40691]: 1.2 scrub starts 2024-09-16T09:30:11.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:11 smithi070 ceph-mon[40691]: 1.7 scrub starts 2024-09-16T09:30:11.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:11 smithi070 ceph-mon[40691]: 1.1 scrub starts 2024-09-16T09:30:11.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:11 smithi070 ceph-mon[40691]: 1.5 scrub starts 2024-09-16T09:30:11.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:11 smithi070 ceph-mon[40691]: osdmap e163: 8 total, 8 up, 8 in 2024-09-16T09:30:11.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:11 smithi070 ceph-mon[40691]: 1.13 scrub starts 2024-09-16T09:30:11.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:11 smithi070 ceph-mon[30388]: pgmap v230: 168 pgs: 2 peering, 10 unknown, 156 active+clean; 1.9 GiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 115 MiB/s wr, 1.84k op/s 2024-09-16T09:30:11.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:11 smithi070 ceph-mon[30388]: 1.4 scrub starts 2024-09-16T09:30:11.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:11 smithi070 ceph-mon[30388]: 1.2 scrub starts 2024-09-16T09:30:11.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:11 smithi070 ceph-mon[30388]: 1.7 scrub starts 2024-09-16T09:30:11.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:11 smithi070 ceph-mon[30388]: 1.1 scrub starts 2024-09-16T09:30:11.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:11 smithi070 ceph-mon[30388]: 1.5 scrub starts 2024-09-16T09:30:11.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:11 smithi070 ceph-mon[30388]: osdmap e163: 8 total, 8 up, 8 in 2024-09-16T09:30:11.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:11 smithi070 ceph-mon[30388]: 1.13 scrub starts 2024-09-16T09:30:11.716 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:30:11.752 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:11.752 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:11.753 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:30:11.853 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:30:11.874 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:11.875 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:11.875 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:30:11.952 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 16 15 37243 37228 145.381 109.75 0.00714212 0.00687272 2024-09-16T09:30:11.977 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:30:12.010 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:12.011 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:12.011 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:30:12.112 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:30:12.145 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:12.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:12.146 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:30:12.190 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:30:11 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:30:11] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:30:12.247 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:30:12.271 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.4 dump_historic_ops 2024-09-16T09:30:12.283 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:12.284 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:12.285 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:30:12.385 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:30:12.424 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:12.425 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:12.425 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:30:12.526 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:30:12.562 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:12.563 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:12.563 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:30:12.660 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:30:12.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:12 smithi084 ceph-mon[32629]: 1.11 scrub starts 2024-09-16T09:30:12.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:12 smithi084 ceph-mon[32629]: 1.c scrub starts 2024-09-16T09:30:12.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:12 smithi084 ceph-mon[32629]: 1.14 scrub starts 2024-09-16T09:30:12.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:12 smithi084 ceph-mon[32629]: 1.0 scrub starts 2024-09-16T09:30:12.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:12 smithi084 ceph-mon[32629]: 1.6 scrub starts 2024-09-16T09:30:12.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:12 smithi084 ceph-mon[32629]: 1.10 scrub starts 2024-09-16T09:30:12.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:12 smithi084 ceph-mon[32629]: 1.e scrub starts 2024-09-16T09:30:12.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:12 smithi084 ceph-mon[32629]: 1.b scrub starts 2024-09-16T09:30:12.664 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:30:12.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:12 smithi070 ceph-mon[30388]: 1.11 scrub starts 2024-09-16T09:30:12.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:12 smithi070 ceph-mon[30388]: 1.c scrub starts 2024-09-16T09:30:12.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:12 smithi070 ceph-mon[30388]: 1.14 scrub starts 2024-09-16T09:30:12.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:12 smithi070 ceph-mon[30388]: 1.0 scrub starts 2024-09-16T09:30:12.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:12 smithi070 ceph-mon[30388]: 1.6 scrub starts 2024-09-16T09:30:12.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:12 smithi070 ceph-mon[30388]: 1.10 scrub starts 2024-09-16T09:30:12.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:12 smithi070 ceph-mon[30388]: 1.e scrub starts 2024-09-16T09:30:12.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:12 smithi070 ceph-mon[30388]: 1.b scrub starts 2024-09-16T09:30:12.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:12 smithi070 ceph-mon[40691]: 1.11 scrub starts 2024-09-16T09:30:12.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:12 smithi070 ceph-mon[40691]: 1.c scrub starts 2024-09-16T09:30:12.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:12 smithi070 ceph-mon[40691]: 1.14 scrub starts 2024-09-16T09:30:12.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:12 smithi070 ceph-mon[40691]: 1.0 scrub starts 2024-09-16T09:30:12.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:12 smithi070 ceph-mon[40691]: 1.6 scrub starts 2024-09-16T09:30:12.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:12 smithi070 ceph-mon[40691]: 1.10 scrub starts 2024-09-16T09:30:12.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:12 smithi070 ceph-mon[40691]: 1.e scrub starts 2024-09-16T09:30:12.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:12 smithi070 ceph-mon[40691]: 1.b scrub starts 2024-09-16T09:30:12.711 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:12.712 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:12.712 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:30:12.813 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:30:12.850 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:12.851 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:12.851 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:30:12.952 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:30:12.953 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 17 16 39215 39199 144.075 123.188 0.00402222 0.00693524 2024-09-16T09:30:13.000 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:13.001 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:13.001 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:30:13.081 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:30:13.102 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:30:13.115 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:30:13.117 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:30:12 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:30:13.137 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:13.137 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:13.138 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:30:13.238 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:30:13.280 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:13.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:13.281 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:30:13.282 INFO:teuthology.orchestra.run.smithi084.stdout:ERROR: (22) Invalid argument 2024-09-16T09:30:13.282 INFO:teuthology.orchestra.run.smithi084.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:30:13.382 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:30:13.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:13 smithi084 ceph-mon[32629]: pgmap v233: 168 pgs: 2 peering, 10 unknown, 156 active+clean; 1.9 GiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 86 MiB/s wr, 1.37k op/s 2024-09-16T09:30:13.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:13 smithi084 ceph-mon[32629]: 1.d scrub starts 2024-09-16T09:30:13.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:13 smithi084 ceph-mon[32629]: 1.f scrub starts 2024-09-16T09:30:13.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:13 smithi084 ceph-mon[32629]: 1.12 scrub starts 2024-09-16T09:30:13.430 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:13.430 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:13.431 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:30:13.450 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:13 smithi070 ceph-mon[30388]: pgmap v233: 168 pgs: 2 peering, 10 unknown, 156 active+clean; 1.9 GiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 86 MiB/s wr, 1.37k op/s 2024-09-16T09:30:13.450 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:13 smithi070 ceph-mon[30388]: 1.d scrub starts 2024-09-16T09:30:13.450 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:13 smithi070 ceph-mon[30388]: 1.f scrub starts 2024-09-16T09:30:13.450 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:13 smithi070 ceph-mon[30388]: 1.12 scrub starts 2024-09-16T09:30:13.450 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:13 smithi070 ceph-mon[40691]: pgmap v233: 168 pgs: 2 peering, 10 unknown, 156 active+clean; 1.9 GiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 86 MiB/s wr, 1.37k op/s 2024-09-16T09:30:13.450 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:13 smithi070 ceph-mon[40691]: 1.d scrub starts 2024-09-16T09:30:13.450 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:13 smithi070 ceph-mon[40691]: 1.f scrub starts 2024-09-16T09:30:13.450 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:13 smithi070 ceph-mon[40691]: 1.12 scrub starts 2024-09-16T09:30:13.532 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:30:13.570 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:13.570 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:13.570 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:30:13.671 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:30:13.714 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:13.715 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:13.715 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:30:13.816 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:30:13.852 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:13.853 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:13.854 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:30:13.952 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 18 16 41104 41088 142.63 118.062 0.00259865 0.00700528 2024-09-16T09:30:13.954 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:30:13.985 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:13.985 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:13.986 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:30:14.086 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:30:14.116 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:14.117 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:14.117 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:30:14.169 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.5 dump_ops_in_flight 2024-09-16T09:30:14.202 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:30:14.203 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:30:14.218 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:30:14.248 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:14.249 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:14.249 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:30:14.250 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:30:14.251 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:30:14.296 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:30:14.296 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:30:14.334 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:30:14.334 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:30:14.350 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:30:14.367 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:30:14.367 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:30:14.386 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:14.386 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:14.387 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:30:14.403 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:30:14.403 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:30:14.404 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:14 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:30:14.404 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:14 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/82279552' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:30:14.404 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:14 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:30:14.404 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:14 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/82279552' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:30:14.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:14 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:30:14.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:14 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/82279552' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:30:14.437 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:30:14.438 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:30:14.474 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:30:14.474 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:30:14.488 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:30:14.535 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:14.536 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:14.536 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:30:14.588 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:30:14.637 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:30:14.671 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:14.672 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:14.672 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:30:14.773 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:30:14.809 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:14.810 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:14.811 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:30:14.911 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:30:14.947 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:14.948 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:14.949 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:30:14.952 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 19 16 42933 42917 141.139 114.312 0.00498951 0.00708014 2024-09-16T09:30:15.050 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:30:15.088 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:15.089 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:15.089 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:30:15.190 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:30:15.224 INFO:teuthology.orchestra.run.smithi070.stderr:nodeep-scrub is set 2024-09-16T09:30:15.230 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:15.231 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:15.232 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:30:15.249 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-09-16T09:30:15.249 INFO:teuthology.orchestra.run.smithi084.stdout: "ops": [ 2024-09-16T09:30:15.250 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:15.250 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.25064.0:43475 8.d 8:b08d1cf3:::benchmark_data_smithi070_107919_object43474:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e165)", 2024-09-16T09:30:15.250 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:30:15.242265+0000", 2024-09-16T09:30:15.250 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 0.0033318140000000002, 2024-09-16T09:30:15.250 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0033626530000000002, 2024-09-16T09:30:15.250 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:30:15.250 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:30:15.250 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "waiting for sub ops", 2024-09-16T09:30:15.250 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:30:15.251 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.25064", 2024-09-16T09:30:15.251 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/2578256439", 2024-09-16T09:30:15.251 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 43475 2024-09-16T09:30:15.251 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:15.251 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:30:15.251 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:15.251 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:30:15.251 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:15.242265+0000", 2024-09-16T09:30:15.251 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:30:15.251 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:15.251 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:15.251 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:30:15.251 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:15.242265+0000", 2024-09-16T09:30:15.252 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:30:15.252 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:15.252 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:15.252 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:30:15.252 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:15.242266+0000", 2024-09-16T09:30:15.252 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.736e-06 2024-09-16T09:30:15.252 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:15.252 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:15.252 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:30:15.253 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:15.242307+0000", 2024-09-16T09:30:15.253 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.0423999999999998e-05 2024-09-16T09:30:15.253 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:15.253 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:15.253 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:30:15.253 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:15.242328+0000", 2024-09-16T09:30:15.253 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.1075999999999999e-05 2024-09-16T09:30:15.253 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:15.253 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:15.253 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:30:15.253 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:15.242337+0000", 2024-09-16T09:30:15.253 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.5349999999999993e-06 2024-09-16T09:30:15.254 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:15.254 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:15.254 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:30:15.254 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:15.242552+0000", 2024-09-16T09:30:15.254 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000214203 2024-09-16T09:30:15.254 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:15.254 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:15.254 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:30:15.254 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:15.242696+0000", 2024-09-16T09:30:15.254 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00014406 2024-09-16T09:30:15.254 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:15.255 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:15.255 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 2,7", 2024-09-16T09:30:15.255 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:15.242844+0000", 2024-09-16T09:30:15.255 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000148846 2024-09-16T09:30:15.255 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:15.255 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:15.255 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:30:15.255 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:15.244640+0000", 2024-09-16T09:30:15.256 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00179591 2024-09-16T09:30:15.256 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:15.256 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:30:15.256 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:15.256 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:15.256 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:15.256 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.25064.0:43479 8.0 8:0e6a4026:::benchmark_data_smithi070_107919_object43478:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e165)", 2024-09-16T09:30:15.256 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:30:15.243550+0000", 2024-09-16T09:30:15.256 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 0.002046726, 2024-09-16T09:30:15.256 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0021539850000000002, 2024-09-16T09:30:15.256 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:30:15.256 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:30:15.256 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "waiting for sub ops", 2024-09-16T09:30:15.257 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:30:15.257 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.25064", 2024-09-16T09:30:15.257 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/2578256439", 2024-09-16T09:30:15.257 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 43479 2024-09-16T09:30:15.257 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:15.257 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:30:15.257 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:15.257 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:30:15.257 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:15.243550+0000", 2024-09-16T09:30:15.257 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:30:15.257 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:15.257 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:15.258 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:30:15.258 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:15.243550+0000", 2024-09-16T09:30:15.258 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:30:15.258 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:15.258 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:15.258 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:30:15.258 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:15.243551+0000", 2024-09-16T09:30:15.258 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.082e-06 2024-09-16T09:30:15.258 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:15.258 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:15.258 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:30:15.259 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:15.243582+0000", 2024-09-16T09:30:15.259 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.1086e-05 2024-09-16T09:30:15.259 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:15.259 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:15.259 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:30:15.259 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:15.243596+0000", 2024-09-16T09:30:15.259 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.4469e-05 2024-09-16T09:30:15.259 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:15.259 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:15.259 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:30:15.259 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:15.243603+0000", 2024-09-16T09:30:15.259 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.8329999999999996e-06 2024-09-16T09:30:15.259 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:15.259 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:15.260 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:30:15.260 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:15.243782+0000", 2024-09-16T09:30:15.260 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000178517 2024-09-16T09:30:15.260 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:15.260 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:15.260 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:30:15.260 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:15.243898+0000", 2024-09-16T09:30:15.260 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00011603399999999999 2024-09-16T09:30:15.260 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:15.260 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:15.260 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 1,3", 2024-09-16T09:30:15.260 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:15.244025+0000", 2024-09-16T09:30:15.261 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000127379 2024-09-16T09:30:15.261 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:15.261 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:30:15.261 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:15.261 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:15.261 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:15.261 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.25064.0:43480 8.0 8.1f07ee30 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e165)", 2024-09-16T09:30:15.261 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:30:15.243618+0000", 2024-09-16T09:30:15.261 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 0.001978596, 2024-09-16T09:30:15.261 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0021493910000000001, 2024-09-16T09:30:15.261 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:30:15.261 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:30:15.262 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "queued for pg", 2024-09-16T09:30:15.262 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:30:15.262 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.25064", 2024-09-16T09:30:15.262 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/2578256439", 2024-09-16T09:30:15.262 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 43480 2024-09-16T09:30:15.262 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:15.262 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:30:15.262 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:15.262 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:30:15.262 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:15.243618+0000", 2024-09-16T09:30:15.262 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:30:15.263 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:15.263 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:15.263 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:30:15.263 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:15.243618+0000", 2024-09-16T09:30:15.263 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:30:15.263 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:15.263 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:15.263 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:30:15.263 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:15.243618+0000", 2024-09-16T09:30:15.263 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.5600000000000001e-07 2024-09-16T09:30:15.263 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:15.263 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:15.263 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:30:15.264 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:15.243647+0000", 2024-09-16T09:30:15.264 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.8951000000000001e-05 2024-09-16T09:30:15.264 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:15.264 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:15.264 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:30:15.264 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:15.243657+0000", 2024-09-16T09:30:15.264 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.6900000000000004e-06 2024-09-16T09:30:15.264 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:15.264 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:15.264 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:30:15.264 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:15.243662+0000", 2024-09-16T09:30:15.264 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.1529999999999996e-06 2024-09-16T09:30:15.265 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:15.265 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:30:15.265 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:15.265 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:15.265 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:15.265 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.25064.0:43481 8.4 8:2bd2c881:::benchmark_data_smithi070_107919_object43480:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e165)", 2024-09-16T09:30:15.265 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:30:15.243727+0000", 2024-09-16T09:30:15.265 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 0.0018693819999999999, 2024-09-16T09:30:15.265 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.002092549, 2024-09-16T09:30:15.265 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:30:15.265 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:30:15.266 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "waiting for sub ops", 2024-09-16T09:30:15.266 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:30:15.266 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.25064", 2024-09-16T09:30:15.267 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/2578256439", 2024-09-16T09:30:15.267 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 43481 2024-09-16T09:30:15.267 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:15.267 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:30:15.267 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:15.267 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:30:15.267 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:15.243727+0000", 2024-09-16T09:30:15.267 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:30:15.267 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:15.267 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:15.268 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:30:15.268 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:15.243727+0000", 2024-09-16T09:30:15.268 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:30:15.268 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:15.268 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:15.268 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:30:15.268 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:15.243728+0000", 2024-09-16T09:30:15.268 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.3499999999999996e-07 2024-09-16T09:30:15.268 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:15.268 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:15.268 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:30:15.268 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:15.244694+0000", 2024-09-16T09:30:15.269 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00096668299999999997 2024-09-16T09:30:15.269 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:15.269 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:15.269 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:30:15.269 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:15.244706+0000", 2024-09-16T09:30:15.269 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1364e-05 2024-09-16T09:30:15.269 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:15.269 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:15.269 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:30:15.269 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:15.244713+0000", 2024-09-16T09:30:15.270 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.7740000000000004e-06 2024-09-16T09:30:15.270 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:15.270 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:15.270 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:30:15.270 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:15.244851+0000", 2024-09-16T09:30:15.270 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00013810999999999999 2024-09-16T09:30:15.270 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:15.270 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:15.270 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:30:15.270 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:15.245056+0000", 2024-09-16T09:30:15.270 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000204255 2024-09-16T09:30:15.270 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:15.270 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:15.270 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 2,4", 2024-09-16T09:30:15.271 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:15.245190+0000", 2024-09-16T09:30:15.271 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00013404700000000001 2024-09-16T09:30:15.271 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:15.271 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:30:15.271 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:15.271 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:15.271 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:15.271 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.25064.0:43483 8.4 8.dffc96e4 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e165)", 2024-09-16T09:30:15.271 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:30:15.244739+0000", 2024-09-16T09:30:15.271 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 0.00085737900000000002, 2024-09-16T09:30:15.271 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.001140973, 2024-09-16T09:30:15.271 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:30:15.271 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:30:15.271 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "queued for pg", 2024-09-16T09:30:15.272 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:30:15.272 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.25064", 2024-09-16T09:30:15.272 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/2578256439", 2024-09-16T09:30:15.272 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 43483 2024-09-16T09:30:15.272 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:15.272 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:30:15.272 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:15.272 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:30:15.272 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:15.244739+0000", 2024-09-16T09:30:15.272 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:30:15.272 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:15.272 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:15.272 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:30:15.272 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:15.244739+0000", 2024-09-16T09:30:15.273 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:30:15.273 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:15.273 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:15.273 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:30:15.273 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:15.244740+0000", 2024-09-16T09:30:15.273 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1230000000000001e-06 2024-09-16T09:30:15.273 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:15.273 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:15.273 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:30:15.273 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:15.244786+0000", 2024-09-16T09:30:15.273 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.5537000000000001e-05 2024-09-16T09:30:15.273 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:15.273 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:15.273 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:30:15.274 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:15.244795+0000", 2024-09-16T09:30:15.274 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.3910000000000004e-06 2024-09-16T09:30:15.274 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:15.274 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:15.274 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:30:15.274 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:15.244801+0000", 2024-09-16T09:30:15.274 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.6189999999999999e-06 2024-09-16T09:30:15.274 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:15.274 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:30:15.274 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:15.274 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:15.274 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:15.274 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.25064.0:43484 8.8 8:1e50685d:::benchmark_data_smithi070_107919_object43483:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e165)", 2024-09-16T09:30:15.274 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:30:15.244809+0000", 2024-09-16T09:30:15.275 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 0.00078707699999999996, 2024-09-16T09:30:15.275 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0011315380000000001, 2024-09-16T09:30:15.275 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:30:15.275 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:30:15.275 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "waiting for sub ops", 2024-09-16T09:30:15.275 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:30:15.275 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.25064", 2024-09-16T09:30:15.275 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/2578256439", 2024-09-16T09:30:15.275 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 43484 2024-09-16T09:30:15.275 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:15.275 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:30:15.275 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:15.275 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:30:15.276 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:15.244809+0000", 2024-09-16T09:30:15.276 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:30:15.276 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:15.276 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:15.276 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:30:15.276 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:15.244809+0000", 2024-09-16T09:30:15.276 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:30:15.276 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:15.276 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:15.276 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:30:15.276 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:15.244810+0000", 2024-09-16T09:30:15.277 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.2600000000000002e-07 2024-09-16T09:30:15.277 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:15.277 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:15.277 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:30:15.277 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:15.244846+0000", 2024-09-16T09:30:15.277 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.6106000000000003e-05 2024-09-16T09:30:15.277 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:15.277 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:15.277 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:30:15.277 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:15.244854+0000", 2024-09-16T09:30:15.278 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.8869999999999992e-06 2024-09-16T09:30:15.278 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:15.278 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:15.278 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:30:15.278 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:15.244859+0000", 2024-09-16T09:30:15.278 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.2399999999999998e-06 2024-09-16T09:30:15.278 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:15.278 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:15.278 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:30:15.278 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:15.245623+0000", 2024-09-16T09:30:15.278 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000764225 2024-09-16T09:30:15.278 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:15.278 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:15.278 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:30:15.278 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:15.245737+0000", 2024-09-16T09:30:15.279 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000113368 2024-09-16T09:30:15.279 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:15.279 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:15.279 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 3,7", 2024-09-16T09:30:15.279 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:15.245862+0000", 2024-09-16T09:30:15.279 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000125678 2024-09-16T09:30:15.279 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:15.279 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:30:15.279 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:15.279 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:15.279 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:15.279 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.25064.0:43485 8.4 8.b02d3714 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e165)", 2024-09-16T09:30:15.279 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:30:15.244867+0000", 2024-09-16T09:30:15.279 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 0.00072945699999999998, 2024-09-16T09:30:15.279 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0011343460000000001, 2024-09-16T09:30:15.279 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:30:15.280 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:30:15.280 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "queued for pg", 2024-09-16T09:30:15.280 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:30:15.280 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.25064", 2024-09-16T09:30:15.280 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/2578256439", 2024-09-16T09:30:15.280 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 43485 2024-09-16T09:30:15.280 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:15.280 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:30:15.280 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:15.280 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:30:15.280 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:15.244867+0000", 2024-09-16T09:30:15.280 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:30:15.280 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:15.280 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:15.280 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:30:15.281 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:15.244867+0000", 2024-09-16T09:30:15.281 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:30:15.281 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:15.281 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:15.281 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:30:15.281 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:15.244868+0000", 2024-09-16T09:30:15.281 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.6899999999999997e-07 2024-09-16T09:30:15.281 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:15.281 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:15.281 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:30:15.281 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:15.244898+0000", 2024-09-16T09:30:15.281 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.0395e-05 2024-09-16T09:30:15.281 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:15.282 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:15.282 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:30:15.282 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:15.244916+0000", 2024-09-16T09:30:15.282 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.7944999999999998e-05 2024-09-16T09:30:15.282 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:15.282 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:15.282 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:30:15.282 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:15.244922+0000", 2024-09-16T09:30:15.282 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.8610000000000003e-06 2024-09-16T09:30:15.282 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:15.282 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:30:15.282 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:15.282 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:15.283 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:15.283 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.25064.0:43486 8.d 8:b30def38:::benchmark_data_smithi070_107919_object43485:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e165)", 2024-09-16T09:30:15.283 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:30:15.244932+0000", 2024-09-16T09:30:15.283 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 0.00066471900000000003, 2024-09-16T09:30:15.283 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0011263600000000001, 2024-09-16T09:30:15.283 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:30:15.283 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:30:15.283 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "waiting for sub ops", 2024-09-16T09:30:15.283 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:30:15.283 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.25064", 2024-09-16T09:30:15.283 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/2578256439", 2024-09-16T09:30:15.283 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 43486 2024-09-16T09:30:15.283 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:15.283 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:30:15.284 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:15.284 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:30:15.284 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:15.244932+0000", 2024-09-16T09:30:15.284 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:30:15.284 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:15.284 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:15.284 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:30:15.284 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:15.244932+0000", 2024-09-16T09:30:15.284 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:30:15.284 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:15.284 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:15.284 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:30:15.285 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:15.244932+0000", 2024-09-16T09:30:15.285 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.1099999999999996e-07 2024-09-16T09:30:15.285 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:15.285 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:15.285 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:30:15.285 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:15.244961+0000", 2024-09-16T09:30:15.285 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.9300999999999999e-05 2024-09-16T09:30:15.285 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:15.285 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:15.285 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:30:15.285 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:15.244974+0000", 2024-09-16T09:30:15.285 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.2077e-05 2024-09-16T09:30:15.285 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:15.286 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:15.286 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:30:15.286 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:15.244979+0000", 2024-09-16T09:30:15.286 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.699e-06 2024-09-16T09:30:15.286 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:15.286 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:15.286 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:30:15.286 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:15.245142+0000", 2024-09-16T09:30:15.286 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00016301600000000001 2024-09-16T09:30:15.286 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:15.286 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:15.286 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:30:15.286 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:15.245247+0000", 2024-09-16T09:30:15.286 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000105033 2024-09-16T09:30:15.287 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:15.287 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:15.287 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 2,7", 2024-09-16T09:30:15.287 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:15.245358+0000", 2024-09-16T09:30:15.287 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00011028500000000001 2024-09-16T09:30:15.287 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:15.287 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:30:15.287 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:15.287 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:15.288 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:15.288 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_repop(client.25064.0:43482 8.a e165/152 8:5ffde28d:::benchmark_data_smithi070_107919_object43481:head v 165'2656, mlcod=164'2654)", 2024-09-16T09:30:15.288 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:30:15.245001+0000", 2024-09-16T09:30:15.288 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 0.00059539100000000002, 2024-09-16T09:30:15.288 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.001119753, 2024-09-16T09:30:15.288 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:30:15.288 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:30:15.288 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "started", 2024-09-16T09:30:15.288 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:30:15.288 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:15.288 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:30:15.289 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:15.245001+0000", 2024-09-16T09:30:15.289 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:30:15.289 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:15.289 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:15.289 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:30:15.289 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:15.245001+0000", 2024-09-16T09:30:15.289 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:30:15.289 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:15.289 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:15.289 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:30:15.289 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:15.245002+0000", 2024-09-16T09:30:15.289 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.0200000000000002e-07 2024-09-16T09:30:15.289 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:15.290 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:15.290 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:30:15.290 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:15.245029+0000", 2024-09-16T09:30:15.290 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.7917000000000001e-05 2024-09-16T09:30:15.290 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:15.290 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:15.290 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:30:15.290 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:15.245037+0000", 2024-09-16T09:30:15.290 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.8010000000000004e-06 2024-09-16T09:30:15.290 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:15.290 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:15.290 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:30:15.290 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:15.245042+0000", 2024-09-16T09:30:15.290 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.1290000000000004e-06 2024-09-16T09:30:15.290 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:15.291 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:15.291 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:30:15.291 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:15.245516+0000", 2024-09-16T09:30:15.291 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00047392400000000002 2024-09-16T09:30:15.291 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:15.291 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:15.291 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:30:15.291 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:15.245534+0000", 2024-09-16T09:30:15.291 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.7388999999999999e-05 2024-09-16T09:30:15.291 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:15.291 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:30:15.291 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:15.291 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:15.291 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-09-16T09:30:15.291 INFO:teuthology.orchestra.run.smithi084.stdout: "num_ops": 9 2024-09-16T09:30:15.292 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-09-16T09:30:15.332 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:30:15.374 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:15.374 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:15.375 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:30:15.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:15 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:30:15.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:15 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:30:15.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:15 smithi084 ceph-mon[32629]: pgmap v234: 168 pgs: 5 active+clean+scrubbing, 163 active+clean; 2.4 GiB data, 8.1 GiB used, 707 GiB / 715 GiB avail; 9.3 KiB/s rd, 141 MiB/s wr, 2.26k op/s; 76 KiB/s, 0 objects/s recovering 2024-09-16T09:30:15.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:15 smithi084 ceph-mon[32629]: osdmap e164: 8 total, 8 up, 8 in 2024-09-16T09:30:15.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:15 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/2744321922' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:30:15.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:15 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:30:15.476 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:30:15.510 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:15.511 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:15.511 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:30:15.598 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:15 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:30:15.599 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:15 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:30:15.599 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:15 smithi070 ceph-mon[30388]: pgmap v234: 168 pgs: 5 active+clean+scrubbing, 163 active+clean; 2.4 GiB data, 8.1 GiB used, 707 GiB / 715 GiB avail; 9.3 KiB/s rd, 141 MiB/s wr, 2.26k op/s; 76 KiB/s, 0 objects/s recovering 2024-09-16T09:30:15.599 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:15 smithi070 ceph-mon[30388]: osdmap e164: 8 total, 8 up, 8 in 2024-09-16T09:30:15.599 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:15 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2744321922' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:30:15.599 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:15 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:30:15.599 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:15 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:30:15.599 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:15 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:30:15.599 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:15 smithi070 ceph-mon[40691]: pgmap v234: 168 pgs: 5 active+clean+scrubbing, 163 active+clean; 2.4 GiB data, 8.1 GiB used, 707 GiB / 715 GiB avail; 9.3 KiB/s rd, 141 MiB/s wr, 2.26k op/s; 76 KiB/s, 0 objects/s recovering 2024-09-16T09:30:15.599 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:15 smithi070 ceph-mon[40691]: osdmap e164: 8 total, 8 up, 8 in 2024-09-16T09:30:15.600 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:15 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/2744321922' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:30:15.600 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:15 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:30:15.612 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:30:15.650 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:15.650 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:15.650 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:30:15.751 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:30:15.787 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:15.788 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:15.788 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:30:15.889 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:30:15.924 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:15.925 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:15.925 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:30:15.953 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:2024-09-16T09:30:15.952332+0000 min lat: 0.00116281 max lat: 0.228591 avg lat: 0.00714162 2024-09-16T09:30:15.953 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-16T09:30:15.953 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 20 16 44803 44787 139.925 116.875 0.0036269 0.00714162 2024-09-16T09:30:16.026 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:30:16.057 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:16.058 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:16.058 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:30:16.066 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.5 dump_blocked_ops 2024-09-16T09:30:16.159 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:30:16.187 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:16.188 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:16.188 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:30:16.289 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:30:16.295 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:16 smithi070 ceph-mon[30388]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:30:16.295 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:16 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-16T09:30:16.295 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:16 smithi070 ceph-mon[30388]: osdmap e165: 8 total, 8 up, 8 in 2024-09-16T09:30:16.296 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:16 smithi070 ceph-mon[40691]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:30:16.296 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:16 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-16T09:30:16.296 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:16 smithi070 ceph-mon[40691]: osdmap e165: 8 total, 8 up, 8 in 2024-09-16T09:30:16.325 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:16.326 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:16.326 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:30:16.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:16 smithi084 ceph-mon[32629]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:30:16.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:16 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-16T09:30:16.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:16 smithi084 ceph-mon[32629]: osdmap e165: 8 total, 8 up, 8 in 2024-09-16T09:30:16.428 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:30:16.478 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:16.479 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:16.479 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:30:16.513 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:30:16.580 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:30:16.623 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:16.623 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:16.624 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:30:16.724 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:30:16.760 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:16.761 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:16.761 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:30:16.863 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:30:16.896 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:16.897 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:16.897 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:30:16.952 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 21 15 46569 46554 138.521 110.438 0.0120034 0.00721447 2024-09-16T09:30:16.998 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:30:17.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:17.020 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:17.020 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:30:17.121 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:30:17.156 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:17.157 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:17.157 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:30:17.258 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:30:17.280 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:17.280 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:17.280 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:30:17.382 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:30:17.415 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:17.416 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:17.416 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:30:17.517 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:30:17.551 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:17.552 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:17.552 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:30:17.553 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:17 smithi084 ceph-mon[32629]: pgmap v237: 168 pgs: 5 active+clean+scrubbing, 163 active+clean; 2.4 GiB data, 8.1 GiB used, 707 GiB / 715 GiB avail; 9.3 KiB/s rd, 83 MiB/s wr, 1.33k op/s; 77 KiB/s, 0 objects/s recovering 2024-09-16T09:30:17.652 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T09:30:17.653 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:30:17.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:17 smithi070 ceph-mon[30388]: pgmap v237: 168 pgs: 5 active+clean+scrubbing, 163 active+clean; 2.4 GiB data, 8.1 GiB used, 707 GiB / 715 GiB avail; 9.3 KiB/s rd, 83 MiB/s wr, 1.33k op/s; 77 KiB/s, 0 objects/s recovering 2024-09-16T09:30:17.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:17 smithi070 ceph-mon[40691]: pgmap v237: 168 pgs: 5 active+clean+scrubbing, 163 active+clean; 2.4 GiB data, 8.1 GiB used, 707 GiB / 715 GiB avail; 9.3 KiB/s rd, 83 MiB/s wr, 1.33k op/s; 77 KiB/s, 0 objects/s recovering 2024-09-16T09:30:17.691 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:17.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:17.692 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:30:17.793 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:30:17.845 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:17.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:17.846 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:30:17.858 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-09-16T09:30:17.858 INFO:teuthology.orchestra.run.smithi084.stdout: "ops": [], 2024-09-16T09:30:17.858 INFO:teuthology.orchestra.run.smithi084.stdout: "complaint_time": 30, 2024-09-16T09:30:17.858 INFO:teuthology.orchestra.run.smithi084.stdout: "num_blocked_ops": 0 2024-09-16T09:30:17.858 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-09-16T09:30:17.947 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:30:17.953 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 22 16 48148 48132 136.702 98.625 0.00937261 0.0073097 2024-09-16T09:30:17.970 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T09:30:17.970 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T09:30:17.970 INFO:tasks.thrashosds.thrasher:Reweighting osd 3 to 0.3394905982962045 2024-09-16T09:30:17.970 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd reweight 3 0.3394905982962045 2024-09-16T09:30:17.989 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:17.990 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:17.990 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:30:18.091 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:30:18.111 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:30:18.134 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:18.137 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:18.138 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:30:18.238 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:30:18.270 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:18.271 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:18.271 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:30:18.348 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd unset noscrub 2024-09-16T09:30:18.372 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:30:18.459 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:18.460 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:18.461 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:30:18.561 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:30:18.589 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:18.590 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:18.590 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:30:18.609 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:18 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:30:18.609 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:18 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:30:18.609 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:18 smithi070 ceph-mon[30388]: pgmap v238: 168 pgs: 3 active+clean+scrubbing, 165 active+clean; 2.6 GiB data, 8.9 GiB used, 706 GiB / 715 GiB avail; 8.7 KiB/s rd, 112 MiB/s wr, 1.80k op/s; 65 KiB/s, 0 objects/s recovering 2024-09-16T09:30:18.610 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:18 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:30:18.610 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:18 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:30:18.610 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:18 smithi070 ceph-mon[40691]: pgmap v238: 168 pgs: 3 active+clean+scrubbing, 165 active+clean; 2.6 GiB data, 8.9 GiB used, 706 GiB / 715 GiB avail; 8.7 KiB/s rd, 112 MiB/s wr, 1.80k op/s; 65 KiB/s, 0 objects/s recovering 2024-09-16T09:30:18.677 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.5 dump_historic_ops 2024-09-16T09:30:18.691 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:30:18.753 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:18.754 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:18.754 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:30:18.797 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:30:18.811 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:30:18.811 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:30:18.850 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:30:18.850 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:30:18.855 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:30:18.887 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:30:18.887 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:30:18.888 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:18.888 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:18.888 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:30:18.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:18 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:30:18.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:18 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:30:18.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:18 smithi084 ceph-mon[32629]: pgmap v238: 168 pgs: 3 active+clean+scrubbing, 165 active+clean; 2.6 GiB data, 8.9 GiB used, 706 GiB / 715 GiB avail; 8.7 KiB/s rd, 112 MiB/s wr, 1.80k op/s; 65 KiB/s, 0 objects/s recovering 2024-09-16T09:30:18.921 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:30:18.921 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:30:18.955 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 23 16 49702 49686 134.978 97.125 0.00789653 0.00740387 2024-09-16T09:30:18.958 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:30:18.958 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:30:18.989 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:30:18.993 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:30:18.993 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:30:19.026 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:19.026 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:19.027 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:30:19.028 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:30:19.028 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:30:19.058 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:30:19.058 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:30:19.067 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:30:19.128 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:30:19.164 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:19.164 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:19.165 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:30:19.265 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:30:19.297 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:19.298 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:19.298 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:30:19.399 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:30:19.441 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:19.441 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:19.441 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:30:19.542 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:30:19.580 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:19.580 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:19.581 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:30:19.629 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:30:19.670 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:19 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/2732522225' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:30:19.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:19 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2732522225' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:30:19.681 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:30:19.693 INFO:teuthology.orchestra.run.smithi084.stdout:ERROR: (22) Invalid argument 2024-09-16T09:30:19.694 INFO:teuthology.orchestra.run.smithi084.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:30:19.720 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:19.720 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:19.720 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:30:19.821 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:30:19.852 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:19.852 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:19.853 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:30:19.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:19 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/2732522225' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:30:19.953 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:30:19.954 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 24 16 51323 51307 133.575 101.312 0.00639945 0.00748096 2024-09-16T09:30:19.988 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:19.988 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:19.988 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:30:20.089 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:30:20.122 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:20.123 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:20.123 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:30:20.224 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:30:20.259 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:20.260 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:20.260 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:30:20.361 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:30:20.390 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:20.390 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:20.391 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:30:20.492 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:30:20.502 INFO:teuthology.orchestra.run.smithi070.stderr:reweighted osd.3 to 0.339491 (56e8) 2024-09-16T09:30:20.547 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:20.549 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:20.549 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:30:20.552 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.6 dump_ops_in_flight 2024-09-16T09:30:20.650 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:30:20.672 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:20.672 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:20.673 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:30:20.748 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:20 smithi070 ceph-mon[30388]: pgmap v239: 168 pgs: 168 active+clean; 2.9 GiB data, 9.8 GiB used, 705 GiB / 715 GiB avail; 7.5 KiB/s rd, 131 MiB/s wr, 2.11k op/s; 56 KiB/s, 0 objects/s recovering 2024-09-16T09:30:20.749 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:20 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1084377916' entity='client.admin' cmd={"prefix": "osd reweight", "id": 3, "weight": 0.3394905982962045} : dispatch 2024-09-16T09:30:20.749 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:20 smithi070 ceph-mon[40691]: pgmap v239: 168 pgs: 168 active+clean; 2.9 GiB data, 9.8 GiB used, 705 GiB / 715 GiB avail; 7.5 KiB/s rd, 131 MiB/s wr, 2.11k op/s; 56 KiB/s, 0 objects/s recovering 2024-09-16T09:30:20.749 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:20 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1084377916' entity='client.admin' cmd={"prefix": "osd reweight", "id": 3, "weight": 0.3394905982962045} : dispatch 2024-09-16T09:30:20.774 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:30:20.797 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:20.797 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:20.798 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:30:20.809 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:30:20.899 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:30:20.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:20 smithi084 ceph-mon[32629]: pgmap v239: 168 pgs: 168 active+clean; 2.9 GiB data, 9.8 GiB used, 705 GiB / 715 GiB avail; 7.5 KiB/s rd, 131 MiB/s wr, 2.11k op/s; 56 KiB/s, 0 objects/s recovering 2024-09-16T09:30:20.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:20 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1084377916' entity='client.admin' cmd={"prefix": "osd reweight", "id": 3, "weight": 0.3394905982962045} : dispatch 2024-09-16T09:30:20.920 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:20.921 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:20.921 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:30:20.954 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 25 16 52412 52396 130.955 68.0625 0.00221855 0.0075127 2024-09-16T09:30:21.022 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:30:21.044 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:21.044 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:21.045 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:30:21.145 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:30:21.167 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:21.167 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:21.168 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:30:21.268 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:30:21.285 INFO:teuthology.orchestra.run.smithi084.stdout:ERROR: (22) Invalid argument 2024-09-16T09:30:21.285 INFO:teuthology.orchestra.run.smithi084.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:30:21.290 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:21.290 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:21.291 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:30:21.392 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:30:21.413 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:21.414 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:21.414 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:30:21.503 INFO:teuthology.orchestra.run.smithi070.stderr:noscrub is unset 2024-09-16T09:30:21.515 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:30:21.555 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:21.556 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:21.556 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:30:21.658 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:30:21.679 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:21.680 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:21.680 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:30:21.781 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:30:21.803 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:21.804 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:21.804 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:30:21.905 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:30:21.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:21 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1084377916' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 0.3394905982962045}]': finished 2024-09-16T09:30:21.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:21 smithi084 ceph-mon[32629]: osdmap e166: 8 total, 8 up, 8 in 2024-09-16T09:30:21.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:21 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3077254788' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:30:21.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:21 smithi084 ceph-mon[32629]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:30:21.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:21 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3077254788' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-16T09:30:21.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:21 smithi084 ceph-mon[32629]: osdmap e167: 8 total, 8 up, 8 in 2024-09-16T09:30:21.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:21 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1084377916' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 0.3394905982962045}]': finished 2024-09-16T09:30:21.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:21 smithi070 ceph-mon[30388]: osdmap e166: 8 total, 8 up, 8 in 2024-09-16T09:30:21.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:21 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3077254788' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:30:21.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:21 smithi070 ceph-mon[30388]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:30:21.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:21 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3077254788' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-16T09:30:21.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:21 smithi070 ceph-mon[30388]: osdmap e167: 8 total, 8 up, 8 in 2024-09-16T09:30:21.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:21 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1084377916' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 0.3394905982962045}]': finished 2024-09-16T09:30:21.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:21 smithi070 ceph-mon[40691]: osdmap e166: 8 total, 8 up, 8 in 2024-09-16T09:30:21.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:21 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3077254788' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:30:21.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:21 smithi070 ceph-mon[40691]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:30:21.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:21 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3077254788' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-16T09:30:21.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:21 smithi070 ceph-mon[40691]: osdmap e167: 8 total, 8 up, 8 in 2024-09-16T09:30:21.926 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:21.927 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:21.927 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:30:21.954 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 26 16 52412 52396 125.919 0 - 0.0075127 2024-09-16T09:30:21.982 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.6 dump_blocked_ops 2024-09-16T09:30:22.028 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:30:22.049 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:22.050 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:22.050 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:30:22.151 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:30:22.173 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:22.174 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:22.174 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:30:22.236 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:30:22.241 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T09:30:22.263 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:30:21 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:30:21] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:30:22.275 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:30:22.296 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:22.296 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:22.297 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:30:22.397 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:30:22.418 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:22.419 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:22.419 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:30:22.495 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:30:22.521 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:30:22.558 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:22.559 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:22.559 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:30:22.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:22 smithi084 ceph-mon[32629]: pgmap v242: 168 pgs: 168 active+clean; 2.9 GiB data, 9.8 GiB used, 705 GiB / 715 GiB avail; 804 B/s rd, 89 MiB/s wr, 1.42k op/s 2024-09-16T09:30:22.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:22 smithi084 ceph-mon[32629]: 1.11 scrub starts 2024-09-16T09:30:22.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:22 smithi084 ceph-mon[32629]: 1.11 scrub ok 2024-09-16T09:30:22.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:22 smithi084 ceph-mon[32629]: 1.13 scrub starts 2024-09-16T09:30:22.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:22 smithi084 ceph-mon[32629]: 1.13 scrub ok 2024-09-16T09:30:22.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:22 smithi084 ceph-mon[32629]: 4.9 scrub starts 2024-09-16T09:30:22.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:22 smithi084 ceph-mon[32629]: 4.9 scrub ok 2024-09-16T09:30:22.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:22 smithi084 ceph-mon[32629]: 4.1c scrub starts 2024-09-16T09:30:22.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:22 smithi084 ceph-mon[32629]: 4.1c scrub ok 2024-09-16T09:30:22.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:22 smithi084 ceph-mon[32629]: osdmap e168: 8 total, 8 up, 8 in 2024-09-16T09:30:22.661 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:30:22.684 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:22.685 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:22.686 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:30:22.786 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:30:22.816 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:22.816 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:22.817 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:30:22.848 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:22 smithi070 ceph-mon[30388]: pgmap v242: 168 pgs: 168 active+clean; 2.9 GiB data, 9.8 GiB used, 705 GiB / 715 GiB avail; 804 B/s rd, 89 MiB/s wr, 1.42k op/s 2024-09-16T09:30:22.848 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:22 smithi070 ceph-mon[30388]: 1.11 scrub starts 2024-09-16T09:30:22.849 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:22 smithi070 ceph-mon[30388]: 1.11 scrub ok 2024-09-16T09:30:22.849 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:22 smithi070 ceph-mon[30388]: 1.13 scrub starts 2024-09-16T09:30:22.849 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:22 smithi070 ceph-mon[30388]: 1.13 scrub ok 2024-09-16T09:30:22.849 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:22 smithi070 ceph-mon[30388]: 4.9 scrub starts 2024-09-16T09:30:22.849 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:22 smithi070 ceph-mon[30388]: 4.9 scrub ok 2024-09-16T09:30:22.849 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:22 smithi070 ceph-mon[30388]: 4.1c scrub starts 2024-09-16T09:30:22.849 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:22 smithi070 ceph-mon[30388]: 4.1c scrub ok 2024-09-16T09:30:22.849 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:22 smithi070 ceph-mon[30388]: osdmap e168: 8 total, 8 up, 8 in 2024-09-16T09:30:22.849 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:22 smithi070 ceph-mon[40691]: pgmap v242: 168 pgs: 168 active+clean; 2.9 GiB data, 9.8 GiB used, 705 GiB / 715 GiB avail; 804 B/s rd, 89 MiB/s wr, 1.42k op/s 2024-09-16T09:30:22.849 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:22 smithi070 ceph-mon[40691]: 1.11 scrub starts 2024-09-16T09:30:22.850 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:22 smithi070 ceph-mon[40691]: 1.11 scrub ok 2024-09-16T09:30:22.850 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:22 smithi070 ceph-mon[40691]: 1.13 scrub starts 2024-09-16T09:30:22.850 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:22 smithi070 ceph-mon[40691]: 1.13 scrub ok 2024-09-16T09:30:22.850 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:22 smithi070 ceph-mon[40691]: 4.9 scrub starts 2024-09-16T09:30:22.850 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:22 smithi070 ceph-mon[40691]: 4.9 scrub ok 2024-09-16T09:30:22.850 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:22 smithi070 ceph-mon[40691]: 4.1c scrub starts 2024-09-16T09:30:22.850 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:22 smithi070 ceph-mon[40691]: 4.1c scrub ok 2024-09-16T09:30:22.850 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:22 smithi070 ceph-mon[40691]: osdmap e168: 8 total, 8 up, 8 in 2024-09-16T09:30:22.918 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:30:22.951 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:22.952 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:22.953 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:30:22.954 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 27 16 52782 52766 122.112 11.5625 0.00985145 0.00818329 2024-09-16T09:30:23.054 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:30:23.060 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:30:22 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:30:23.085 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:23.086 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:23.086 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:30:23.119 INFO:teuthology.orchestra.run.smithi084.stdout:ERROR: (22) Invalid argument 2024-09-16T09:30:23.119 INFO:teuthology.orchestra.run.smithi084.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:30:23.187 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:30:23.220 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:23.221 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:23.222 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:30:23.322 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:30:23.358 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:23.358 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:23.359 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:30:23.417 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:30:23.417 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:30:23.444 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:30:23.444 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:30:23.460 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:30:23.476 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:30:23.476 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:30:23.493 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:23.494 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:23.494 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:30:23.504 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:30:23.504 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:30:23.540 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:30:23.541 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:30:23.575 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:30:23.575 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:30:23.595 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:30:23.605 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:30:23.606 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:30:23.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:23.626 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:23.627 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:30:23.634 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:30:23.635 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:30:23.727 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:30:23.763 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:23.763 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:23.764 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:30:23.865 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:30:23.892 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:23 smithi084 ceph-mon[32629]: 1.6 scrub starts 2024-09-16T09:30:23.892 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:23 smithi084 ceph-mon[32629]: 1.6 scrub ok 2024-09-16T09:30:23.892 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:23 smithi084 ceph-mon[32629]: 1.4 scrub starts 2024-09-16T09:30:23.892 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:23 smithi084 ceph-mon[32629]: 1.4 scrub ok 2024-09-16T09:30:23.892 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:23 smithi084 ceph-mon[32629]: 1.c scrub starts 2024-09-16T09:30:23.892 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:23 smithi084 ceph-mon[32629]: 1.c scrub ok 2024-09-16T09:30:23.892 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:23 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:30:23.892 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:23 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1767364120' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:30:23.895 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:23.896 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:23.896 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:30:23.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:23 smithi070 ceph-mon[30388]: 1.6 scrub starts 2024-09-16T09:30:23.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:23 smithi070 ceph-mon[30388]: 1.6 scrub ok 2024-09-16T09:30:23.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:23 smithi070 ceph-mon[30388]: 1.4 scrub starts 2024-09-16T09:30:23.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:23 smithi070 ceph-mon[30388]: 1.4 scrub ok 2024-09-16T09:30:23.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:23 smithi070 ceph-mon[30388]: 1.c scrub starts 2024-09-16T09:30:23.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:23 smithi070 ceph-mon[30388]: 1.c scrub ok 2024-09-16T09:30:23.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:23 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:30:23.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:23 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1767364120' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:30:23.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:23 smithi070 ceph-mon[40691]: 1.6 scrub starts 2024-09-16T09:30:23.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:23 smithi070 ceph-mon[40691]: 1.6 scrub ok 2024-09-16T09:30:23.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:23 smithi070 ceph-mon[40691]: 1.4 scrub starts 2024-09-16T09:30:23.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:23 smithi070 ceph-mon[40691]: 1.4 scrub ok 2024-09-16T09:30:23.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:23 smithi070 ceph-mon[40691]: 1.c scrub starts 2024-09-16T09:30:23.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:23 smithi070 ceph-mon[40691]: 1.c scrub ok 2024-09-16T09:30:23.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:23 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:30:23.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:23 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1767364120' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:30:23.950 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.6 dump_historic_ops 2024-09-16T09:30:23.954 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 28 16 54115 54099 120.727 83.3125 0.00366242 0.00827579 2024-09-16T09:30:23.998 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:30:24.029 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:24.029 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:24.030 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:30:24.130 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:30:24.176 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:24.177 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:24.177 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:30:24.278 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:30:24.337 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:24.338 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:24.338 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:30:24.396 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-16T09:30:24.399 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:30:24.439 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:30:24.480 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:24.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:24.481 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:30:24.582 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:30:24.613 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:24.613 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:24.614 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:30:24.714 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:30:24.755 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:24.756 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:24.756 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:30:24.816 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:30:24.858 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:30:24.862 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:24 smithi084 ceph-mon[32629]: 1.7 scrub starts 2024-09-16T09:30:24.862 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:24 smithi084 ceph-mon[32629]: 1.7 scrub ok 2024-09-16T09:30:24.863 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:24 smithi084 ceph-mon[32629]: pgmap v244: 168 pgs: 1 activating+undersized+remapped, 2 activating+undersized+degraded+remapped, 165 active+clean; 3.2 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 96 MiB/s wr, 1.54k op/s; 9922/158139 objects degraded (6.274%); 0 B/s, 9 objects/s recovering 2024-09-16T09:30:24.863 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:24 smithi084 ceph-mon[32629]: osdmap e169: 8 total, 8 up, 8 in 2024-09-16T09:30:24.897 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:24.897 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:24.897 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:30:24.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:24 smithi070 ceph-mon[30388]: 1.7 scrub starts 2024-09-16T09:30:24.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:24 smithi070 ceph-mon[30388]: 1.7 scrub ok 2024-09-16T09:30:24.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:24 smithi070 ceph-mon[30388]: pgmap v244: 168 pgs: 1 activating+undersized+remapped, 2 activating+undersized+degraded+remapped, 165 active+clean; 3.2 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 96 MiB/s wr, 1.54k op/s; 9922/158139 objects degraded (6.274%); 0 B/s, 9 objects/s recovering 2024-09-16T09:30:24.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:24 smithi070 ceph-mon[30388]: osdmap e169: 8 total, 8 up, 8 in 2024-09-16T09:30:24.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:24 smithi070 ceph-mon[40691]: 1.7 scrub starts 2024-09-16T09:30:24.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:24 smithi070 ceph-mon[40691]: 1.7 scrub ok 2024-09-16T09:30:24.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:24 smithi070 ceph-mon[40691]: pgmap v244: 168 pgs: 1 activating+undersized+remapped, 2 activating+undersized+degraded+remapped, 165 active+clean; 3.2 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 96 MiB/s wr, 1.54k op/s; 9922/158139 objects degraded (6.274%); 0 B/s, 9 objects/s recovering 2024-09-16T09:30:24.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:24 smithi070 ceph-mon[40691]: osdmap e169: 8 total, 8 up, 8 in 2024-09-16T09:30:24.955 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 29 15 55394 55379 119.322 80 0.00994136 0.00837479 2024-09-16T09:30:24.999 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:30:25.035 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:25.035 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:25.035 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:30:25.105 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-09-16T09:30:25.105 INFO:teuthology.orchestra.run.smithi084.stdout: "size": 20, 2024-09-16T09:30:25.105 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 600, 2024-09-16T09:30:25.105 INFO:teuthology.orchestra.run.smithi084.stdout: "ops": [ 2024-09-16T09:30:25.105 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.105 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:60245 7.7 7:e5610e70:::benchmark_data_smithi070_78438_object60244:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-16T09:30:25.105 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:26:44.352114+0000", 2024-09-16T09:30:25.105 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 220.715831251, 2024-09-16T09:30:25.105 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.127497575, 2024-09-16T09:30:25.106 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:30:25.106 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:30:25.106 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:30:25.106 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:30:25.106 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:30:25.106 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:30:25.106 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 60245 2024-09-16T09:30:25.106 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.106 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:30:25.106 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.106 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:30:25.106 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.352114+0000", 2024-09-16T09:30:25.106 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:30:25.106 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.107 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.107 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:30:25.107 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.352114+0000", 2024-09-16T09:30:25.107 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:30:25.107 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.107 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.107 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:30:25.107 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.352115+0000", 2024-09-16T09:30:25.107 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0449999999999999e-06 2024-09-16T09:30:25.107 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.108 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.108 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:30:25.108 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.352162+0000", 2024-09-16T09:30:25.108 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.7416000000000001e-05 2024-09-16T09:30:25.108 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.108 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.108 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:30:25.108 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.352174+0000", 2024-09-16T09:30:25.108 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.2347999999999999e-05 2024-09-16T09:30:25.109 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.109 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.109 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:30:25.109 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.352181+0000", 2024-09-16T09:30:25.109 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.8809999999999998e-06 2024-09-16T09:30:25.109 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.109 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.109 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:30:25.109 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.352231+0000", 2024-09-16T09:30:25.109 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.9292000000000002e-05 2024-09-16T09:30:25.109 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.109 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.109 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:30:25.109 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.352373+0000", 2024-09-16T09:30:25.110 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00014240699999999999 2024-09-16T09:30:25.110 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.110 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.110 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,3", 2024-09-16T09:30:25.110 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.352494+0000", 2024-09-16T09:30:25.110 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000121039 2024-09-16T09:30:25.110 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.110 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.110 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:30:25.110 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.354404+0000", 2024-09-16T09:30:25.110 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0019098979999999999 2024-09-16T09:30:25.110 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.110 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.110 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:30:25.111 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.355369+0000", 2024-09-16T09:30:25.111 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00096560899999999998 2024-09-16T09:30:25.111 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.111 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.111 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:30:25.111 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.479497+0000", 2024-09-16T09:30:25.111 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.124127398 2024-09-16T09:30:25.111 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.111 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.111 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:30:25.111 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.479559+0000", 2024-09-16T09:30:25.111 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.1692000000000005e-05 2024-09-16T09:30:25.111 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.112 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.112 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:30:25.112 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.479611+0000", 2024-09-16T09:30:25.112 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.2549999999999997e-05 2024-09-16T09:30:25.112 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:25.112 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:30:25.112 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:25.112 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.112 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.112 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:60246 7.d 7:b1444fdd:::benchmark_data_smithi070_78438_object60245:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-16T09:30:25.112 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:26:44.352405+0000", 2024-09-16T09:30:25.112 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 220.71553986000001, 2024-09-16T09:30:25.113 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.13626988500000001, 2024-09-16T09:30:25.113 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:30:25.113 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:30:25.113 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:30:25.113 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:30:25.113 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:30:25.113 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:30:25.113 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 60246 2024-09-16T09:30:25.113 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.113 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:30:25.113 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.114 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:30:25.114 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.352405+0000", 2024-09-16T09:30:25.114 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:30:25.114 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.114 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.114 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:30:25.115 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.352405+0000", 2024-09-16T09:30:25.115 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:30:25.115 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.115 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.115 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:30:25.115 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.352406+0000", 2024-09-16T09:30:25.115 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.85e-07 2024-09-16T09:30:25.115 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.115 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.115 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:30:25.115 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.352431+0000", 2024-09-16T09:30:25.115 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.5364e-05 2024-09-16T09:30:25.115 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.115 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.116 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:30:25.116 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.352440+0000", 2024-09-16T09:30:25.116 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.9509999999999995e-06 2024-09-16T09:30:25.116 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.116 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.116 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:30:25.116 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.352446+0000", 2024-09-16T09:30:25.116 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.2099999999999998e-06 2024-09-16T09:30:25.116 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.116 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.116 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:30:25.116 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.352486+0000", 2024-09-16T09:30:25.116 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.9475999999999999e-05 2024-09-16T09:30:25.116 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.117 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.117 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:30:25.117 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.352575+0000", 2024-09-16T09:30:25.117 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.9239000000000004e-05 2024-09-16T09:30:25.117 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.117 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.117 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,1", 2024-09-16T09:30:25.117 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.352693+0000", 2024-09-16T09:30:25.117 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000118248 2024-09-16T09:30:25.117 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.117 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.117 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:30:25.117 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.354498+0000", 2024-09-16T09:30:25.117 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0018052890000000001 2024-09-16T09:30:25.118 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.118 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.118 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:30:25.118 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.355812+0000", 2024-09-16T09:30:25.118 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0013137380000000001 2024-09-16T09:30:25.118 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.118 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.118 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:30:25.118 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.488574+0000", 2024-09-16T09:30:25.118 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.13276227400000001 2024-09-16T09:30:25.119 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.119 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.119 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:30:25.119 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.488632+0000", 2024-09-16T09:30:25.119 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.7970999999999998e-05 2024-09-16T09:30:25.119 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.119 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.119 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:30:25.119 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.488675+0000", 2024-09-16T09:30:25.119 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.2339999999999998e-05 2024-09-16T09:30:25.119 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:25.119 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:30:25.119 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:25.119 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.119 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.120 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:60253 7.d 7:bc8ca5cf:::benchmark_data_smithi070_78438_object60252:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-16T09:30:25.120 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:26:44.357181+0000", 2024-09-16T09:30:25.120 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 220.71076422499999, 2024-09-16T09:30:25.120 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.131612381, 2024-09-16T09:30:25.120 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:30:25.120 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:30:25.120 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:30:25.120 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:30:25.120 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:30:25.120 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:30:25.120 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 60253 2024-09-16T09:30:25.120 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.120 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:30:25.121 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.121 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:30:25.121 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.357181+0000", 2024-09-16T09:30:25.121 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:30:25.121 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.121 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.121 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:30:25.121 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.357181+0000", 2024-09-16T09:30:25.121 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:30:25.121 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.121 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.121 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:30:25.121 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.357182+0000", 2024-09-16T09:30:25.121 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.2270000000000001e-06 2024-09-16T09:30:25.122 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.122 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.122 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:30:25.122 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.357244+0000", 2024-09-16T09:30:25.122 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.2683999999999998e-05 2024-09-16T09:30:25.122 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.122 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.122 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:30:25.122 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.357255+0000", 2024-09-16T09:30:25.122 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0720000000000001e-05 2024-09-16T09:30:25.122 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.122 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.122 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:30:25.122 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.357262+0000", 2024-09-16T09:30:25.123 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.7800000000000003e-06 2024-09-16T09:30:25.123 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.123 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.123 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:30:25.123 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.357305+0000", 2024-09-16T09:30:25.123 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.2667999999999999e-05 2024-09-16T09:30:25.124 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.124 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.124 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:30:25.124 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.357399+0000", 2024-09-16T09:30:25.124 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.4217999999999998e-05 2024-09-16T09:30:25.124 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.124 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.124 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,1", 2024-09-16T09:30:25.124 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.357492+0000", 2024-09-16T09:30:25.124 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.3213000000000004e-05 2024-09-16T09:30:25.124 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.124 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.125 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:30:25.125 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.358931+0000", 2024-09-16T09:30:25.125 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.001439357 2024-09-16T09:30:25.125 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.125 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.125 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:30:25.125 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.359185+0000", 2024-09-16T09:30:25.125 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00025320799999999999 2024-09-16T09:30:25.125 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.125 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.125 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:30:25.125 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.488722+0000", 2024-09-16T09:30:25.125 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.12953768399999999 2024-09-16T09:30:25.126 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.126 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.126 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:30:25.126 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.488763+0000", 2024-09-16T09:30:25.126 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.1059000000000001e-05 2024-09-16T09:30:25.126 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.126 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.126 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:30:25.126 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:44.488793+0000", 2024-09-16T09:30:25.126 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.9563000000000001e-05 2024-09-16T09:30:25.126 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:25.126 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:30:25.126 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:25.127 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.127 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.127 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:202388 7.7 7:ed754f08:::benchmark_data_smithi070_78438_object76257:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135)", 2024-09-16T09:30:25.127 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:28:50.710298+0000", 2024-09-16T09:30:25.127 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 94.357646884999994, 2024-09-16T09:30:25.127 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.097457902999999999, 2024-09-16T09:30:25.127 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:30:25.127 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:30:25.127 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:30:25.127 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:30:25.127 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:30:25.127 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:30:25.128 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 202388 2024-09-16T09:30:25.128 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.128 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:30:25.128 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.128 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:30:25.128 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.710298+0000", 2024-09-16T09:30:25.128 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:30:25.128 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.128 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.128 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:30:25.128 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.710298+0000", 2024-09-16T09:30:25.128 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:30:25.128 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.129 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.129 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:30:25.129 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.710299+0000", 2024-09-16T09:30:25.129 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.9000000000000005e-07 2024-09-16T09:30:25.129 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.129 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.129 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:30:25.129 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.710303+0000", 2024-09-16T09:30:25.129 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.924e-06 2024-09-16T09:30:25.129 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.129 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.129 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:30:25.129 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.710311+0000", 2024-09-16T09:30:25.130 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.4290000000000001e-06 2024-09-16T09:30:25.130 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.130 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.130 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:30:25.130 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.710316+0000", 2024-09-16T09:30:25.130 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.4750000000000004e-06 2024-09-16T09:30:25.130 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.130 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.130 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:30:25.130 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.710363+0000", 2024-09-16T09:30:25.130 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.7332e-05 2024-09-16T09:30:25.130 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.130 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.130 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:30:25.130 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.710467+0000", 2024-09-16T09:30:25.130 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000104288 2024-09-16T09:30:25.131 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.131 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.131 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,3", 2024-09-16T09:30:25.131 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.710569+0000", 2024-09-16T09:30:25.131 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000101499 2024-09-16T09:30:25.131 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.131 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.131 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:30:25.131 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.713003+0000", 2024-09-16T09:30:25.131 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0024343059999999998 2024-09-16T09:30:25.131 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.131 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.131 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:30:25.131 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.807079+0000", 2024-09-16T09:30:25.132 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.094075385999999997 2024-09-16T09:30:25.132 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.132 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.132 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:30:25.132 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.807669+0000", 2024-09-16T09:30:25.132 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00059051199999999996 2024-09-16T09:30:25.132 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.132 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.132 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:30:25.132 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.807717+0000", 2024-09-16T09:30:25.132 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.7716999999999997e-05 2024-09-16T09:30:25.132 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.132 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.132 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:30:25.133 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.807756+0000", 2024-09-16T09:30:25.133 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.9045000000000003e-05 2024-09-16T09:30:25.133 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:25.133 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:30:25.133 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:25.133 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.133 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.133 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:202394 7.7 7:e0c03bf8:::benchmark_data_smithi070_78438_object76263:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135)", 2024-09-16T09:30:25.133 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:28:50.713789+0000", 2024-09-16T09:30:25.133 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 94.354155754000004, 2024-09-16T09:30:25.133 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.094102293000000004, 2024-09-16T09:30:25.133 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:30:25.133 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:30:25.133 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:30:25.133 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:30:25.134 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:30:25.134 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:30:25.134 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 202394 2024-09-16T09:30:25.134 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.134 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:30:25.134 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.134 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:30:25.134 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.713789+0000", 2024-09-16T09:30:25.134 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:30:25.134 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.134 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.134 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:30:25.134 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.713789+0000", 2024-09-16T09:30:25.134 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:30:25.134 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.134 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.135 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:30:25.135 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.713790+0000", 2024-09-16T09:30:25.135 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.9000000000000005e-07 2024-09-16T09:30:25.135 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.135 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.135 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:30:25.135 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.713794+0000", 2024-09-16T09:30:25.135 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.8770000000000003e-06 2024-09-16T09:30:25.135 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.135 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.135 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:30:25.135 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.713803+0000", 2024-09-16T09:30:25.135 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.6619999999999994e-06 2024-09-16T09:30:25.135 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.135 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.136 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:30:25.136 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.713807+0000", 2024-09-16T09:30:25.136 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.9049999999999996e-06 2024-09-16T09:30:25.136 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.136 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.136 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:30:25.136 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.713841+0000", 2024-09-16T09:30:25.136 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.3575999999999998e-05 2024-09-16T09:30:25.136 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.136 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.136 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:30:25.136 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.713919+0000", 2024-09-16T09:30:25.136 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.8446000000000001e-05 2024-09-16T09:30:25.136 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.136 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.137 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,3", 2024-09-16T09:30:25.137 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.713983+0000", 2024-09-16T09:30:25.137 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.3911000000000005e-05 2024-09-16T09:30:25.137 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.137 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.137 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:30:25.137 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.720846+0000", 2024-09-16T09:30:25.137 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0068624310000000004 2024-09-16T09:30:25.137 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.137 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.137 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:30:25.137 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.807591+0000", 2024-09-16T09:30:25.137 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.086744842000000003 2024-09-16T09:30:25.137 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.137 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.137 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:30:25.138 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.807817+0000", 2024-09-16T09:30:25.138 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00022680500000000001 2024-09-16T09:30:25.138 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.138 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.138 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:30:25.138 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.807856+0000", 2024-09-16T09:30:25.138 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.8909999999999998e-05 2024-09-16T09:30:25.138 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.138 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.138 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:30:25.138 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.807891+0000", 2024-09-16T09:30:25.138 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.4938000000000003e-05 2024-09-16T09:30:25.138 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:25.138 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:30:25.139 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:25.139 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.139 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.139 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:202395 7.7 7:e38f3473:::benchmark_data_smithi070_78438_object76264:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135)", 2024-09-16T09:30:25.139 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:28:50.713818+0000", 2024-09-16T09:30:25.139 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 94.354126347000005, 2024-09-16T09:30:25.139 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.094181156000000002, 2024-09-16T09:30:25.139 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:30:25.139 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:30:25.139 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:30:25.139 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:30:25.139 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:30:25.139 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:30:25.139 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 202395 2024-09-16T09:30:25.140 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.140 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:30:25.140 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.140 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:30:25.140 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.713818+0000", 2024-09-16T09:30:25.140 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:30:25.140 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.140 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.140 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:30:25.140 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.713818+0000", 2024-09-16T09:30:25.140 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:30:25.140 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.140 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.140 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:30:25.140 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.713819+0000", 2024-09-16T09:30:25.141 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.0499999999999999e-07 2024-09-16T09:30:25.141 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.141 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.141 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:30:25.141 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.713820+0000", 2024-09-16T09:30:25.141 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.4980000000000001e-06 2024-09-16T09:30:25.141 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.141 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.141 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:30:25.141 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.713824+0000", 2024-09-16T09:30:25.141 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.5130000000000001e-06 2024-09-16T09:30:25.141 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.141 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.142 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:30:25.142 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.713827+0000", 2024-09-16T09:30:25.142 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.1590000000000002e-06 2024-09-16T09:30:25.142 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.142 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.142 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:30:25.142 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.716144+0000", 2024-09-16T09:30:25.142 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0023175159999999999 2024-09-16T09:30:25.142 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.142 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.142 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:30:25.142 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.716246+0000", 2024-09-16T09:30:25.142 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000101902 2024-09-16T09:30:25.143 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.143 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.143 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,3", 2024-09-16T09:30:25.143 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.716348+0000", 2024-09-16T09:30:25.143 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000102099 2024-09-16T09:30:25.143 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.143 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.143 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:30:25.143 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.720849+0000", 2024-09-16T09:30:25.143 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0045008069999999999 2024-09-16T09:30:25.143 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.143 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.143 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:30:25.143 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.807634+0000", 2024-09-16T09:30:25.144 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.086784617999999994 2024-09-16T09:30:25.144 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.144 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.144 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:30:25.144 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.807928+0000", 2024-09-16T09:30:25.144 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00029464299999999998 2024-09-16T09:30:25.144 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.144 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.144 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:30:25.144 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.807969+0000", 2024-09-16T09:30:25.144 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.0917999999999997e-05 2024-09-16T09:30:25.144 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.144 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.144 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:30:25.145 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.808000+0000", 2024-09-16T09:30:25.145 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.0178000000000001e-05 2024-09-16T09:30:25.145 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:25.145 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:30:25.145 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:25.145 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.145 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.145 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:202398 7.7 7:ed5121df:::benchmark_data_smithi070_78438_object76267:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135)", 2024-09-16T09:30:25.145 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:28:50.713837+0000", 2024-09-16T09:30:25.145 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 94.354107724000002, 2024-09-16T09:30:25.145 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.094273179999999998, 2024-09-16T09:30:25.145 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:30:25.145 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:30:25.145 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:30:25.145 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:30:25.146 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:30:25.146 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:30:25.146 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 202398 2024-09-16T09:30:25.146 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.146 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:30:25.146 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.146 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:30:25.146 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.713837+0000", 2024-09-16T09:30:25.146 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:30:25.146 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.146 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.147 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:30:25.147 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.713837+0000", 2024-09-16T09:30:25.147 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:30:25.147 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.147 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.147 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:30:25.147 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.713837+0000", 2024-09-16T09:30:25.147 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.2100000000000001e-07 2024-09-16T09:30:25.147 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.147 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.147 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:30:25.148 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.713839+0000", 2024-09-16T09:30:25.148 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.297e-06 2024-09-16T09:30:25.148 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.148 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.148 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:30:25.148 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.713842+0000", 2024-09-16T09:30:25.148 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.4350000000000001e-06 2024-09-16T09:30:25.148 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.149 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.149 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:30:25.149 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.713845+0000", 2024-09-16T09:30:25.149 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.8430000000000001e-06 2024-09-16T09:30:25.149 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.149 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.149 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:30:25.149 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.718636+0000", 2024-09-16T09:30:25.149 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0047914630000000001 2024-09-16T09:30:25.150 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.150 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.150 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:30:25.150 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.718741+0000", 2024-09-16T09:30:25.150 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000104498 2024-09-16T09:30:25.150 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.150 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.150 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,3", 2024-09-16T09:30:25.150 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.718844+0000", 2024-09-16T09:30:25.150 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00010359800000000001 2024-09-16T09:30:25.150 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.150 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.151 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:30:25.151 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.720976+0000", 2024-09-16T09:30:25.151 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0021316080000000001 2024-09-16T09:30:25.151 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.151 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.151 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:30:25.151 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.807788+0000", 2024-09-16T09:30:25.151 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.086811865000000002 2024-09-16T09:30:25.151 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.151 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.151 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:30:25.151 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.808037+0000", 2024-09-16T09:30:25.151 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000248905 2024-09-16T09:30:25.151 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.152 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.152 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:30:25.152 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.808077+0000", 2024-09-16T09:30:25.152 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.0547000000000003e-05 2024-09-16T09:30:25.152 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.152 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.152 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:30:25.152 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.808110+0000", 2024-09-16T09:30:25.152 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.29e-05 2024-09-16T09:30:25.152 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:25.152 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:30:25.152 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:25.153 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.153 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.153 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:203433 7.c 7:3eef1e44:::benchmark_data_smithi070_78438_object77302:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135)", 2024-09-16T09:30:25.153 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:28:51.980722+0000", 2024-09-16T09:30:25.153 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 93.087223107, 2024-09-16T09:30:25.153 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.14639702299999999, 2024-09-16T09:30:25.153 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:30:25.153 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:30:25.153 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:30:25.153 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:30:25.153 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:30:25.153 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:30:25.154 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 203433 2024-09-16T09:30:25.154 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.154 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:30:25.154 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.154 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:30:25.154 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.980722+0000", 2024-09-16T09:30:25.154 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:30:25.154 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.154 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.154 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:30:25.154 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.980722+0000", 2024-09-16T09:30:25.154 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:30:25.155 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.155 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.155 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:30:25.155 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.980723+0000", 2024-09-16T09:30:25.155 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.2559999999999999e-06 2024-09-16T09:30:25.155 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.155 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.155 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:30:25.155 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.980727+0000", 2024-09-16T09:30:25.155 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.9990000000000002e-06 2024-09-16T09:30:25.155 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.155 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.155 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:30:25.156 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.980742+0000", 2024-09-16T09:30:25.156 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.4761e-05 2024-09-16T09:30:25.156 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.156 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.156 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:30:25.156 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.980750+0000", 2024-09-16T09:30:25.156 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.9389999999999997e-06 2024-09-16T09:30:25.156 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.156 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.156 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:30:25.156 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.980831+0000", 2024-09-16T09:30:25.156 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.1773999999999993e-05 2024-09-16T09:30:25.156 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.157 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.157 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:30:25.157 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.980990+0000", 2024-09-16T09:30:25.157 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00015910100000000001 2024-09-16T09:30:25.157 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.157 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.157 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 3,7", 2024-09-16T09:30:25.157 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.981083+0000", 2024-09-16T09:30:25.157 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.2274000000000005e-05 2024-09-16T09:30:25.157 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.157 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.157 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:30:25.158 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.984389+0000", 2024-09-16T09:30:25.158 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.003306504 2024-09-16T09:30:25.158 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.158 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.158 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:30:25.158 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.985173+0000", 2024-09-16T09:30:25.158 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00078339399999999997 2024-09-16T09:30:25.158 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.158 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.158 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:30:25.158 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:52.127014+0000", 2024-09-16T09:30:25.158 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.14184139400000001 2024-09-16T09:30:25.158 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.158 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.159 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:30:25.159 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:52.127071+0000", 2024-09-16T09:30:25.159 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.6552999999999999e-05 2024-09-16T09:30:25.159 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.159 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.159 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:30:25.159 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:52.127119+0000", 2024-09-16T09:30:25.159 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.8074000000000001e-05 2024-09-16T09:30:25.159 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:25.159 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:30:25.159 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:25.159 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.159 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.159 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:203435 7.c 7:3a18a4c7:::benchmark_data_smithi070_78438_object77304:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136)", 2024-09-16T09:30:25.160 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:28:51.987097+0000", 2024-09-16T09:30:25.160 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 93.080847520000006, 2024-09-16T09:30:25.160 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.14016357900000001, 2024-09-16T09:30:25.160 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:30:25.160 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:30:25.160 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:30:25.160 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:30:25.160 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:30:25.160 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:30:25.160 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 203435 2024-09-16T09:30:25.160 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.160 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:30:25.161 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.161 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:30:25.161 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.987097+0000", 2024-09-16T09:30:25.161 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:30:25.161 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.161 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.161 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:30:25.161 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.987097+0000", 2024-09-16T09:30:25.161 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:30:25.161 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.161 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.161 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:30:25.161 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.987106+0000", 2024-09-16T09:30:25.161 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.4840000000000004e-06 2024-09-16T09:30:25.162 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.162 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.162 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:30:25.162 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.987110+0000", 2024-09-16T09:30:25.162 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.053e-06 2024-09-16T09:30:25.162 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.162 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.162 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:30:25.162 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.987118+0000", 2024-09-16T09:30:25.162 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.5730000000000008e-06 2024-09-16T09:30:25.162 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.162 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.162 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:30:25.162 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.987125+0000", 2024-09-16T09:30:25.163 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.6220000000000003e-06 2024-09-16T09:30:25.163 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.163 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.163 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:30:25.163 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.987168+0000", 2024-09-16T09:30:25.163 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.2617999999999998e-05 2024-09-16T09:30:25.163 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.163 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.163 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:30:25.163 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.987308+0000", 2024-09-16T09:30:25.163 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00014014499999999999 2024-09-16T09:30:25.163 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.163 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.163 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 3,7", 2024-09-16T09:30:25.164 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.987386+0000", 2024-09-16T09:30:25.164 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.8529999999999995e-05 2024-09-16T09:30:25.164 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.164 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.164 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:30:25.164 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.989686+0000", 2024-09-16T09:30:25.164 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0023000989999999999 2024-09-16T09:30:25.164 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.164 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.164 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:30:25.164 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.990324+0000", 2024-09-16T09:30:25.164 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00063796899999999995 2024-09-16T09:30:25.165 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.165 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.165 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:30:25.165 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:52.127169+0000", 2024-09-16T09:30:25.165 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.13684418100000001 2024-09-16T09:30:25.165 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.165 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.165 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:30:25.165 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:52.127222+0000", 2024-09-16T09:30:25.165 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.3458000000000002e-05 2024-09-16T09:30:25.165 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.165 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.165 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:30:25.165 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:52.127261+0000", 2024-09-16T09:30:25.166 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.8847000000000003e-05 2024-09-16T09:30:25.166 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:25.166 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:30:25.166 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:25.166 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.166 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.166 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:215668 7.c 7:31105ff3:::benchmark_data_smithi070_78438_object89537:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e141)", 2024-09-16T09:30:25.166 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:29:03.671611+0000", 2024-09-16T09:30:25.166 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 81.396333975000005, 2024-09-16T09:30:25.166 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.254303314, 2024-09-16T09:30:25.166 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:30:25.166 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:30:25.166 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:30:25.166 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:30:25.167 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:30:25.167 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:30:25.167 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 215668 2024-09-16T09:30:25.167 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.167 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:30:25.167 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.167 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:30:25.167 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.671611+0000", 2024-09-16T09:30:25.167 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:30:25.167 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.167 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.167 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:30:25.167 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.671611+0000", 2024-09-16T09:30:25.167 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:30:25.168 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.168 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.168 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:30:25.168 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.671612+0000", 2024-09-16T09:30:25.168 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1379999999999999e-06 2024-09-16T09:30:25.168 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.168 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.168 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:30:25.168 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.671617+0000", 2024-09-16T09:30:25.168 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.5959999999999997e-06 2024-09-16T09:30:25.168 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.168 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.168 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:30:25.168 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.721873+0000", 2024-09-16T09:30:25.169 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.050256728000000001 2024-09-16T09:30:25.169 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.169 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.169 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:30:25.169 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.721883+0000", 2024-09-16T09:30:25.169 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.8730000000000002e-06 2024-09-16T09:30:25.169 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.169 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.169 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:30:25.169 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.722610+0000", 2024-09-16T09:30:25.169 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000726449 2024-09-16T09:30:25.169 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.169 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.170 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:30:25.170 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.722746+0000", 2024-09-16T09:30:25.170 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000136231 2024-09-16T09:30:25.170 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.170 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.170 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 3,7", 2024-09-16T09:30:25.170 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.722862+0000", 2024-09-16T09:30:25.170 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000116125 2024-09-16T09:30:25.170 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.170 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.170 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:30:25.170 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.925722+0000", 2024-09-16T09:30:25.170 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.20286009799999999 2024-09-16T09:30:25.170 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.171 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.171 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:30:25.171 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.925733+0000", 2024-09-16T09:30:25.171 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0635000000000001e-05 2024-09-16T09:30:25.171 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.171 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.171 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:30:25.171 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.925820+0000", 2024-09-16T09:30:25.171 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.7356999999999997e-05 2024-09-16T09:30:25.171 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.171 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.171 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:30:25.171 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.925868+0000", 2024-09-16T09:30:25.172 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.7519999999999999e-05 2024-09-16T09:30:25.172 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.172 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.172 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:30:25.172 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.925914+0000", 2024-09-16T09:30:25.172 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.6563999999999997e-05 2024-09-16T09:30:25.172 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:25.172 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:30:25.172 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:25.173 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.173 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.173 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:215673 7.c 7:37240b0f:::benchmark_data_smithi070_78438_object89542:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e141)", 2024-09-16T09:30:25.173 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:29:03.721919+0000", 2024-09-16T09:30:25.173 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 81.346026203999998, 2024-09-16T09:30:25.173 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.20416785100000001, 2024-09-16T09:30:25.173 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:30:25.173 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:30:25.173 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:30:25.173 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:30:25.173 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:30:25.173 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:30:25.173 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 215673 2024-09-16T09:30:25.173 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.173 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:30:25.174 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.174 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:30:25.174 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.721919+0000", 2024-09-16T09:30:25.174 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:30:25.174 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.174 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.174 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:30:25.174 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.721919+0000", 2024-09-16T09:30:25.174 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:30:25.174 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.174 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.174 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:30:25.175 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.721920+0000", 2024-09-16T09:30:25.175 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.373e-06 2024-09-16T09:30:25.175 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.175 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.175 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:30:25.175 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.721925+0000", 2024-09-16T09:30:25.175 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.6789999999999996e-06 2024-09-16T09:30:25.175 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.175 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.175 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:30:25.175 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.721934+0000", 2024-09-16T09:30:25.175 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.0489999999999993e-06 2024-09-16T09:30:25.176 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.176 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.176 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:30:25.176 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.721939+0000", 2024-09-16T09:30:25.176 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.0080000000000002e-06 2024-09-16T09:30:25.176 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.176 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.176 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:30:25.176 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.725737+0000", 2024-09-16T09:30:25.176 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0037982279999999998 2024-09-16T09:30:25.176 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.176 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.176 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:30:25.176 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.725857+0000", 2024-09-16T09:30:25.177 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00012021 2024-09-16T09:30:25.177 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.177 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.177 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 3,7", 2024-09-16T09:30:25.177 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.725981+0000", 2024-09-16T09:30:25.177 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000124282 2024-09-16T09:30:25.177 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.177 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.177 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:30:25.177 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.925739+0000", 2024-09-16T09:30:25.177 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.19975721499999999 2024-09-16T09:30:25.177 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.177 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.177 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:30:25.177 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.925767+0000", 2024-09-16T09:30:25.178 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.8090999999999999e-05 2024-09-16T09:30:25.178 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.178 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.178 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:30:25.178 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.926009+0000", 2024-09-16T09:30:25.178 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00024180299999999999 2024-09-16T09:30:25.178 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.178 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:30:25.180 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.180 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:30:25.180 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.926051+0000", 2024-09-16T09:30:25.180 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.2327999999999999e-05 2024-09-16T09:30:25.180 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.180 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.180 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:30:25.180 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.926086+0000", 2024-09-16T09:30:25.180 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.5584999999999999e-05 2024-09-16T09:30:25.180 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:25.180 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:30:25.180 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:25.181 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.181 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.181 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:215702 7.c 7:352e61ee:::benchmark_data_smithi070_78438_object89571:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e141)", 2024-09-16T09:30:25.181 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:29:03.721946+0000", 2024-09-16T09:30:25.181 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 81.345999172999996, 2024-09-16T09:30:25.181 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.20431201800000001, 2024-09-16T09:30:25.181 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:30:25.181 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:30:25.181 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:30:25.181 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:30:25.181 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:30:25.181 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:30:25.181 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 215702 2024-09-16T09:30:25.182 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.182 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:30:25.182 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.182 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:30:25.182 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.721946+0000", 2024-09-16T09:30:25.182 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:30:25.182 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.182 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.182 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:30:25.182 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.721946+0000", 2024-09-16T09:30:25.182 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:30:25.182 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.182 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.182 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:30:25.183 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.721946+0000", 2024-09-16T09:30:25.183 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.5199999999999998e-07 2024-09-16T09:30:25.183 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.183 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.183 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:30:25.183 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.721948+0000", 2024-09-16T09:30:25.183 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.0980000000000001e-06 2024-09-16T09:30:25.183 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.183 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.183 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:30:25.183 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.721953+0000", 2024-09-16T09:30:25.183 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.8940000000000001e-06 2024-09-16T09:30:25.183 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.183 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.184 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:30:25.184 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.721957+0000", 2024-09-16T09:30:25.184 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.0300000000000004e-06 2024-09-16T09:30:25.184 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.184 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.184 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:30:25.184 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.728650+0000", 2024-09-16T09:30:25.184 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0066934969999999996 2024-09-16T09:30:25.184 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.184 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.184 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:30:25.184 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.728753+0000", 2024-09-16T09:30:25.184 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000102785 2024-09-16T09:30:25.185 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.185 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.185 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 3,7", 2024-09-16T09:30:25.185 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.728856+0000", 2024-09-16T09:30:25.185 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000102303 2024-09-16T09:30:25.185 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.185 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.185 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:30:25.185 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.925926+0000", 2024-09-16T09:30:25.185 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.19707012099999999 2024-09-16T09:30:25.185 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.185 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.185 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:30:25.186 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.925968+0000", 2024-09-16T09:30:25.186 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.2169999999999998e-05 2024-09-16T09:30:25.186 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.186 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.186 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:30:25.186 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.926178+0000", 2024-09-16T09:30:25.186 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00021038400000000001 2024-09-16T09:30:25.186 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.186 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.186 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:30:25.186 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.926221+0000", 2024-09-16T09:30:25.186 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.3211999999999999e-05 2024-09-16T09:30:25.186 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.186 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.187 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:30:25.187 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.926258+0000", 2024-09-16T09:30:25.187 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.6171999999999999e-05 2024-09-16T09:30:25.187 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:25.187 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:30:25.187 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:25.187 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.187 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.187 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:215709 7.7 7:e925da47:::benchmark_data_smithi070_78438_object89578:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e141)", 2024-09-16T09:30:25.187 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:29:03.722069+0000", 2024-09-16T09:30:25.187 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 81.345875477999996, 2024-09-16T09:30:25.187 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.203720754, 2024-09-16T09:30:25.187 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:30:25.187 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:30:25.188 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:30:25.188 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:30:25.188 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:30:25.188 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:30:25.188 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 215709 2024-09-16T09:30:25.188 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.188 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:30:25.188 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.188 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:30:25.188 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.722069+0000", 2024-09-16T09:30:25.188 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:30:25.188 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.188 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.188 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:30:25.189 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.722069+0000", 2024-09-16T09:30:25.189 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:30:25.189 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.189 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.189 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:30:25.189 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.722070+0000", 2024-09-16T09:30:25.189 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.9299999999999998e-07 2024-09-16T09:30:25.189 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.189 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.189 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:30:25.189 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.722073+0000", 2024-09-16T09:30:25.189 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.83e-06 2024-09-16T09:30:25.189 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.189 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.190 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:30:25.190 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.722079+0000", 2024-09-16T09:30:25.190 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.4010000000000002e-06 2024-09-16T09:30:25.190 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.190 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.190 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:30:25.190 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.722084+0000", 2024-09-16T09:30:25.190 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.2689999999999999e-06 2024-09-16T09:30:25.190 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.190 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.190 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:30:25.190 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.726106+0000", 2024-09-16T09:30:25.190 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.004021663 2024-09-16T09:30:25.190 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.190 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.191 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:30:25.191 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.726207+0000", 2024-09-16T09:30:25.191 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000101337 2024-09-16T09:30:25.191 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.191 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.191 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,3", 2024-09-16T09:30:25.191 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.726308+0000", 2024-09-16T09:30:25.191 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000100815 2024-09-16T09:30:25.191 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.191 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.191 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:30:25.191 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.729071+0000", 2024-09-16T09:30:25.191 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.002763111 2024-09-16T09:30:25.191 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.191 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.192 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:30:25.192 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.731946+0000", 2024-09-16T09:30:25.192 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.002874457 2024-09-16T09:30:25.192 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.192 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.192 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:30:25.192 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.925721+0000", 2024-09-16T09:30:25.192 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.193775314 2024-09-16T09:30:25.192 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.192 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.192 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:30:25.192 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.925757+0000", 2024-09-16T09:30:25.192 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.6494e-05 2024-09-16T09:30:25.192 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.193 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.193 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:30:25.193 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.925790+0000", 2024-09-16T09:30:25.193 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.2570000000000002e-05 2024-09-16T09:30:25.193 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:25.193 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:30:25.193 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:25.193 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.193 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.193 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:215721 7.c 7:3859c12b:::benchmark_data_smithi070_78438_object89590:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e141)", 2024-09-16T09:30:25.193 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:29:03.729262+0000", 2024-09-16T09:30:25.193 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 81.338682411999997, 2024-09-16T09:30:25.193 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.197115755, 2024-09-16T09:30:25.193 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:30:25.194 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:30:25.194 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:30:25.194 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:30:25.194 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:30:25.194 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:30:25.194 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 215721 2024-09-16T09:30:25.194 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.194 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:30:25.194 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.194 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:30:25.194 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.729262+0000", 2024-09-16T09:30:25.194 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:30:25.194 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.194 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.195 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:30:25.195 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.729262+0000", 2024-09-16T09:30:25.195 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:30:25.195 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.195 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.195 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:30:25.195 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.729263+0000", 2024-09-16T09:30:25.195 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.2999999999999999e-07 2024-09-16T09:30:25.195 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.195 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.195 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:30:25.195 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.729267+0000", 2024-09-16T09:30:25.195 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.822e-06 2024-09-16T09:30:25.195 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.195 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.196 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:30:25.196 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.729276+0000", 2024-09-16T09:30:25.196 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.3979999999999999e-06 2024-09-16T09:30:25.196 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.196 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.196 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:30:25.196 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.729280+0000", 2024-09-16T09:30:25.196 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.826e-06 2024-09-16T09:30:25.196 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.196 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.196 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:30:25.196 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.731471+0000", 2024-09-16T09:30:25.196 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.002190619 2024-09-16T09:30:25.197 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.197 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.197 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:30:25.197 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.731551+0000", 2024-09-16T09:30:25.197 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.9630999999999997e-05 2024-09-16T09:30:25.197 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.197 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.197 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 3,7", 2024-09-16T09:30:25.197 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.731616+0000", 2024-09-16T09:30:25.197 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.5062999999999997e-05 2024-09-16T09:30:25.197 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.197 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.198 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:30:25.198 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.925931+0000", 2024-09-16T09:30:25.198 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.19431493799999999 2024-09-16T09:30:25.198 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.198 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.198 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:30:25.198 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.926133+0000", 2024-09-16T09:30:25.198 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00020227600000000001 2024-09-16T09:30:25.198 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.198 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.198 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:30:25.198 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.926304+0000", 2024-09-16T09:30:25.198 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00017123099999999999 2024-09-16T09:30:25.198 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.199 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.199 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:30:25.199 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.926347+0000", 2024-09-16T09:30:25.199 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.2651000000000002e-05 2024-09-16T09:30:25.199 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.199 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.199 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:30:25.199 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.926378+0000", 2024-09-16T09:30:25.199 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.137e-05 2024-09-16T09:30:25.199 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:25.199 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:30:25.199 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:25.200 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.200 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.200 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_repop_reply(client.24698.0:215668 7.c e141/82 ondisk, result = 0)", 2024-09-16T09:30:25.200 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:29:03.731265+0000", 2024-09-16T09:30:25.200 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 81.336680052000006, 2024-09-16T09:30:25.200 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.194475603, 2024-09-16T09:30:25.200 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:30:25.200 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:30:25.200 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "started", 2024-09-16T09:30:25.200 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:30:25.200 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.201 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:30:25.201 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.731265+0000", 2024-09-16T09:30:25.201 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:30:25.201 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.201 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.201 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:30:25.201 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.731265+0000", 2024-09-16T09:30:25.201 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:30:25.201 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.201 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.201 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:30:25.202 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.731266+0000", 2024-09-16T09:30:25.202 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.13e-06 2024-09-16T09:30:25.202 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.202 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.202 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:30:25.202 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.731279+0000", 2024-09-16T09:30:25.202 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.2967e-05 2024-09-16T09:30:25.202 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.202 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.202 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:30:25.202 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.731291+0000", 2024-09-16T09:30:25.202 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.2045000000000001e-05 2024-09-16T09:30:25.202 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.203 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.203 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:30:25.203 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.731299+0000", 2024-09-16T09:30:25.203 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.7559999999999999e-06 2024-09-16T09:30:25.203 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.203 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.203 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:30:25.203 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.925711+0000", 2024-09-16T09:30:25.204 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.194412262 2024-09-16T09:30:25.204 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.204 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.204 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:30:25.204 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.925716+0000", 2024-09-16T09:30:25.204 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.3229999999999997e-06 2024-09-16T09:30:25.204 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.204 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.204 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:30:25.204 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.925740+0000", 2024-09-16T09:30:25.204 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.4119999999999999e-05 2024-09-16T09:30:25.204 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:25.204 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:30:25.205 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:25.205 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.205 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.205 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_repop_reply(client.24698.0:215673 7.c e141/82 ondisk, result = 0)", 2024-09-16T09:30:25.205 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:29:03.733382+0000", 2024-09-16T09:30:25.205 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 81.334562449000003, 2024-09-16T09:30:25.205 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.19239313599999999, 2024-09-16T09:30:25.205 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:30:25.205 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:30:25.205 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "started", 2024-09-16T09:30:25.205 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:30:25.205 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.205 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:30:25.206 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.733382+0000", 2024-09-16T09:30:25.206 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:30:25.206 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.206 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.206 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:30:25.206 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.733382+0000", 2024-09-16T09:30:25.206 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:30:25.206 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.206 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.206 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:30:25.206 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.733383+0000", 2024-09-16T09:30:25.206 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.4099999999999997e-07 2024-09-16T09:30:25.206 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.206 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.207 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:30:25.207 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.733387+0000", 2024-09-16T09:30:25.207 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.3189999999999996e-06 2024-09-16T09:30:25.207 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.207 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.207 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:30:25.207 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.733398+0000", 2024-09-16T09:30:25.207 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0699e-05 2024-09-16T09:30:25.207 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.207 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.207 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:30:25.207 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.925646+0000", 2024-09-16T09:30:25.207 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.19224790999999999 2024-09-16T09:30:25.208 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.208 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.208 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:30:25.208 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.925759+0000", 2024-09-16T09:30:25.208 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000113022 2024-09-16T09:30:25.208 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.208 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.208 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:30:25.208 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.925764+0000", 2024-09-16T09:30:25.208 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.8559999999999998e-06 2024-09-16T09:30:25.208 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.208 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.208 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:30:25.209 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.925775+0000", 2024-09-16T09:30:25.209 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1489e-05 2024-09-16T09:30:25.209 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:25.209 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:30:25.209 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:25.209 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.209 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.209 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_repop_reply(client.24698.0:215709 7.7 e141/82 ondisk, result = 0)", 2024-09-16T09:30:25.209 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:29:03.733756+0000", 2024-09-16T09:30:25.209 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 81.334188965999999, 2024-09-16T09:30:25.209 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.19203947800000001, 2024-09-16T09:30:25.209 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:30:25.210 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:30:25.210 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "started", 2024-09-16T09:30:25.210 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:30:25.210 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.210 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:30:25.210 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.733756+0000", 2024-09-16T09:30:25.210 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:30:25.210 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.210 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.210 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:30:25.210 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.733756+0000", 2024-09-16T09:30:25.210 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:30:25.210 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.210 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.211 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:30:25.211 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.733756+0000", 2024-09-16T09:30:25.211 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.6599999999999996e-07 2024-09-16T09:30:25.211 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.211 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.211 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:30:25.211 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.733761+0000", 2024-09-16T09:30:25.211 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.2950000000000004e-06 2024-09-16T09:30:25.211 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.211 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.211 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:30:25.211 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.925646+0000", 2024-09-16T09:30:25.211 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.19188530400000001 2024-09-16T09:30:25.211 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.212 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.212 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:30:25.212 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.925661+0000", 2024-09-16T09:30:25.212 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.5029e-05 2024-09-16T09:30:25.212 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.212 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.212 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:30:25.212 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.925709+0000", 2024-09-16T09:30:25.212 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.7506000000000002e-05 2024-09-16T09:30:25.212 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.212 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.212 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:30:25.213 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.925716+0000", 2024-09-16T09:30:25.213 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.435e-06 2024-09-16T09:30:25.213 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.213 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.213 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:30:25.213 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.925795+0000", 2024-09-16T09:30:25.213 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.9243e-05 2024-09-16T09:30:25.213 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:25.213 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:30:25.213 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:25.213 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.213 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.213 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:230151 7.d 7:b538372a:::benchmark_data_smithi070_78438_object104020:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e143)", 2024-09-16T09:30:25.213 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:29:17.048472+0000", 2024-09-16T09:30:25.214 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 68.019472984999993, 2024-09-16T09:30:25.214 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.097967526999999999, 2024-09-16T09:30:25.214 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:30:25.214 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:30:25.214 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:30:25.214 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:30:25.214 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:30:25.214 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:30:25.214 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 230151 2024-09-16T09:30:25.214 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.214 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:30:25.214 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.214 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:30:25.214 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:17.048472+0000", 2024-09-16T09:30:25.215 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:30:25.215 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.215 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.215 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:30:25.215 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:17.048472+0000", 2024-09-16T09:30:25.215 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:30:25.215 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.215 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.215 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:30:25.215 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:17.048472+0000", 2024-09-16T09:30:25.215 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.0999999999999999e-07 2024-09-16T09:30:25.215 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.215 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.215 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:30:25.215 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:17.048474+0000", 2024-09-16T09:30:25.216 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.2539999999999999e-06 2024-09-16T09:30:25.216 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.216 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.216 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:30:25.216 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:17.048480+0000", 2024-09-16T09:30:25.216 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.9750000000000004e-06 2024-09-16T09:30:25.216 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.216 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.216 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:30:25.216 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:17.048485+0000", 2024-09-16T09:30:25.216 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.5920000000000002e-06 2024-09-16T09:30:25.216 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.216 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.216 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:30:25.217 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:17.048529+0000", 2024-09-16T09:30:25.217 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.4200999999999999e-05 2024-09-16T09:30:25.217 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.217 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.217 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:30:25.217 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:17.048627+0000", 2024-09-16T09:30:25.217 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.8185999999999998e-05 2024-09-16T09:30:25.217 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.217 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.217 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,1", 2024-09-16T09:30:25.217 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:17.048724+0000", 2024-09-16T09:30:25.217 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.6636000000000001e-05 2024-09-16T09:30:25.217 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.217 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.218 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:30:25.218 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:17.053219+0000", 2024-09-16T09:30:25.218 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0044945879999999999 2024-09-16T09:30:25.218 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.218 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.218 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:30:25.218 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:17.056036+0000", 2024-09-16T09:30:25.218 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0028171289999999998 2024-09-16T09:30:25.218 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.218 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.218 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:30:25.218 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:17.146336+0000", 2024-09-16T09:30:25.218 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.090300633000000005 2024-09-16T09:30:25.218 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.219 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.219 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:30:25.219 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:17.146389+0000", 2024-09-16T09:30:25.219 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.2988999999999997e-05 2024-09-16T09:30:25.219 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.219 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.219 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:30:25.219 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:17.146439+0000", 2024-09-16T09:30:25.219 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.9934e-05 2024-09-16T09:30:25.219 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:25.219 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:30:25.219 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:25.219 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.219 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.219 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:230164 7.7 7:eb6c19a4:::benchmark_data_smithi070_78438_object104033:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e143)", 2024-09-16T09:30:25.220 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:29:17.058827+0000", 2024-09-16T09:30:25.220 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 68.009117278000005, 2024-09-16T09:30:25.220 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.101866417, 2024-09-16T09:30:25.220 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:30:25.220 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:30:25.220 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:30:25.220 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:30:25.220 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:30:25.220 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:30:25.220 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 230164 2024-09-16T09:30:25.220 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.220 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:30:25.220 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.220 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:30:25.220 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:17.058827+0000", 2024-09-16T09:30:25.221 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:30:25.221 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.221 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.221 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:30:25.221 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:17.058827+0000", 2024-09-16T09:30:25.221 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:30:25.221 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.221 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.221 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:30:25.221 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:17.058829+0000", 2024-09-16T09:30:25.221 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.206e-06 2024-09-16T09:30:25.221 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.221 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.221 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:30:25.221 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:17.058832+0000", 2024-09-16T09:30:25.222 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.5480000000000002e-06 2024-09-16T09:30:25.222 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.222 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.222 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:30:25.222 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:17.058843+0000", 2024-09-16T09:30:25.222 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0715e-05 2024-09-16T09:30:25.222 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.222 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.222 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:30:25.222 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:17.058851+0000", 2024-09-16T09:30:25.222 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.8469999999999996e-06 2024-09-16T09:30:25.222 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.222 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.222 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:30:25.223 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:17.058906+0000", 2024-09-16T09:30:25.223 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.4962e-05 2024-09-16T09:30:25.223 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.223 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.223 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:30:25.223 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:17.058995+0000", 2024-09-16T09:30:25.223 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.8753999999999998e-05 2024-09-16T09:30:25.223 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.223 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.223 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,3", 2024-09-16T09:30:25.223 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:17.059070+0000", 2024-09-16T09:30:25.223 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.5609000000000001e-05 2024-09-16T09:30:25.223 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.224 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.224 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:30:25.224 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:17.063126+0000", 2024-09-16T09:30:25.224 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0040554809999999997 2024-09-16T09:30:25.224 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.224 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.224 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:30:25.224 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:17.063429+0000", 2024-09-16T09:30:25.224 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00030373899999999997 2024-09-16T09:30:25.224 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.224 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.224 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:30:25.224 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:17.160632+0000", 2024-09-16T09:30:25.224 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.097202175000000002 2024-09-16T09:30:25.225 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.225 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.225 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:30:25.225 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:17.160664+0000", 2024-09-16T09:30:25.225 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.2657999999999998e-05 2024-09-16T09:30:25.225 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.225 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.225 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:30:25.225 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:17.160694+0000", 2024-09-16T09:30:25.225 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.9723e-05 2024-09-16T09:30:25.225 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:25.225 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:30:25.225 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:25.225 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.226 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.226 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:230165 7.7 7:e1df84c2:::benchmark_data_smithi070_78438_object104034:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e143)", 2024-09-16T09:30:25.226 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:29:17.061006+0000", 2024-09-16T09:30:25.226 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 68.006939179, 2024-09-16T09:30:25.226 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.099808531000000006, 2024-09-16T09:30:25.226 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:30:25.226 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:30:25.226 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:30:25.226 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:30:25.226 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:30:25.226 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:30:25.226 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 230165 2024-09-16T09:30:25.226 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.227 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:30:25.227 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.227 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:30:25.227 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:17.061006+0000", 2024-09-16T09:30:25.227 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:30:25.227 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.227 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.227 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:30:25.227 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:17.061006+0000", 2024-09-16T09:30:25.227 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:30:25.227 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.227 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.227 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:30:25.228 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:17.061006+0000", 2024-09-16T09:30:25.228 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.7800000000000001e-07 2024-09-16T09:30:25.228 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.228 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.228 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:30:25.228 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:17.061009+0000", 2024-09-16T09:30:25.228 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.0419999999999998e-06 2024-09-16T09:30:25.228 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.228 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.228 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:30:25.228 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:17.061018+0000", 2024-09-16T09:30:25.228 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.4819999999999994e-06 2024-09-16T09:30:25.228 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.228 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.228 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:30:25.229 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:17.061024+0000", 2024-09-16T09:30:25.229 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.3300000000000004e-06 2024-09-16T09:30:25.229 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.229 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.229 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:30:25.229 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:17.061146+0000", 2024-09-16T09:30:25.229 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000121785 2024-09-16T09:30:25.229 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.229 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.229 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:30:25.229 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:17.061261+0000", 2024-09-16T09:30:25.229 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00011545200000000001 2024-09-16T09:30:25.229 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.229 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.230 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,3", 2024-09-16T09:30:25.230 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:17.061378+0000", 2024-09-16T09:30:25.230 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00011623799999999999 2024-09-16T09:30:25.230 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.230 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.230 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:30:25.230 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:17.063246+0000", 2024-09-16T09:30:25.230 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0018686830000000001 2024-09-16T09:30:25.230 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.230 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.230 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:30:25.230 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:17.063452+0000", 2024-09-16T09:30:25.230 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00020538199999999999 2024-09-16T09:30:25.230 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.230 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.231 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:30:25.231 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:17.160738+0000", 2024-09-16T09:30:25.231 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.097286891 2024-09-16T09:30:25.231 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.231 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.231 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:30:25.231 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:17.160781+0000", 2024-09-16T09:30:25.231 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.2549000000000002e-05 2024-09-16T09:30:25.231 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:25.231 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:25.231 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:30:25.231 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:17.160814+0000", 2024-09-16T09:30:25.231 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.3118999999999999e-05 2024-09-16T09:30:25.231 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:25.231 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:30:25.232 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:25.232 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:25.232 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:30:25.232 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-09-16T09:30:25.262 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:25.263 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:25.263 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:30:25.364 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:30:25.397 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:25.397 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:25.398 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:30:25.499 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:30:25.527 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:25.528 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:25.528 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:30:25.629 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:30:25.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:25.665 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:25.666 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:30:25.766 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:30:25.803 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:25.804 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:25.804 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:30:25.905 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:30:25.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:25 smithi084 ceph-mon[32629]: 6.1f scrub starts 2024-09-16T09:30:25.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:25 smithi084 ceph-mon[32629]: 6.1f scrub ok 2024-09-16T09:30:25.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:25 smithi084 ceph-mon[32629]: Health check failed: Degraded data redundancy: 9922/158139 objects degraded (6.274%), 2 pgs degraded (PG_DEGRADED) 2024-09-16T09:30:25.920 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:25 smithi070 ceph-mon[40691]: 6.1f scrub starts 2024-09-16T09:30:25.920 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:25 smithi070 ceph-mon[40691]: 6.1f scrub ok 2024-09-16T09:30:25.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:25 smithi070 ceph-mon[40691]: Health check failed: Degraded data redundancy: 9922/158139 objects degraded (6.274%), 2 pgs degraded (PG_DEGRADED) 2024-09-16T09:30:25.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:25 smithi070 ceph-mon[30388]: 6.1f scrub starts 2024-09-16T09:30:25.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:25 smithi070 ceph-mon[30388]: 6.1f scrub ok 2024-09-16T09:30:25.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:25 smithi070 ceph-mon[30388]: Health check failed: Degraded data redundancy: 9922/158139 objects degraded (6.274%), 2 pgs degraded (PG_DEGRADED) 2024-09-16T09:30:25.939 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:25.940 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:25.940 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:30:25.955 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 30 16 56707 56691 118.078 82 0.00721323 0.00846407 2024-09-16T09:30:26.042 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:30:26.067 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:26.068 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:26.068 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:30:26.169 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:30:26.202 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:26.203 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:26.203 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:30:26.304 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:30:26.334 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:26.334 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:26.335 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:30:26.348 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.7 dump_ops_in_flight 2024-09-16T09:30:26.436 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:30:26.473 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:26.474 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:26.474 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:30:26.547 INFO:teuthology.orchestra.run.smithi070.stderr:nodeep-scrub is unset 2024-09-16T09:30:26.556 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T09:30:26.575 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:30:26.614 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:26.615 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:26.615 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:30:26.631 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T09:30:26.631 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T09:30:26.631 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-16T09:30:26.631 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd dump -f json-pretty 2024-09-16T09:30:26.717 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:30:26.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:26.774 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:26.774 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:30:26.793 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:30:26.875 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:30:26.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:26 smithi084 ceph-mon[32629]: pgmap v246: 168 pgs: 1 activating+undersized+remapped, 2 activating+undersized+degraded+remapped, 165 active+clean; 3.2 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 59 MiB/s wr, 940 op/s; 9922/158139 objects degraded (6.274%); 0 B/s, 11 objects/s recovering 2024-09-16T09:30:26.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:26 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/2669310488' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:30:26.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:26 smithi084 ceph-mon[32629]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-16T09:30:26.915 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:26.915 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:26.916 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:30:26.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:26 smithi070 ceph-mon[30388]: pgmap v246: 168 pgs: 1 activating+undersized+remapped, 2 activating+undersized+degraded+remapped, 165 active+clean; 3.2 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 59 MiB/s wr, 940 op/s; 9922/158139 objects degraded (6.274%); 0 B/s, 11 objects/s recovering 2024-09-16T09:30:26.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:26 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2669310488' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:30:26.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:26 smithi070 ceph-mon[30388]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-16T09:30:26.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:26 smithi070 ceph-mon[40691]: pgmap v246: 168 pgs: 1 activating+undersized+remapped, 2 activating+undersized+degraded+remapped, 165 active+clean; 3.2 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 59 MiB/s wr, 940 op/s; 9922/158139 objects degraded (6.274%); 0 B/s, 11 objects/s recovering 2024-09-16T09:30:26.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:26 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/2669310488' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:30:26.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:26 smithi070 ceph-mon[40691]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-16T09:30:26.955 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 31 16 57936 57920 116.746 76.8125 0.00791494 0.0085583 2024-09-16T09:30:27.017 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:30:27.053 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:27.053 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:27.054 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:30:27.154 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:30:27.187 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:27.188 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:27.188 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:30:27.289 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:30:27.327 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:27.327 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:27.328 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:30:27.429 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:30:27.471 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:27.472 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:27.472 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:30:27.519 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:30:27.573 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:30:27.574 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:27 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/2669310488' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-16T09:30:27.574 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:27 smithi084 ceph-mon[32629]: osdmap e170: 8 total, 8 up, 8 in 2024-09-16T09:30:27.619 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:27.620 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:27.620 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:30:27.658 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:27 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2669310488' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-16T09:30:27.658 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:27 smithi070 ceph-mon[30388]: osdmap e170: 8 total, 8 up, 8 in 2024-09-16T09:30:27.658 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:27 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/2669310488' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-16T09:30:27.658 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:27 smithi070 ceph-mon[40691]: osdmap e170: 8 total, 8 up, 8 in 2024-09-16T09:30:27.702 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:30:27.721 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:30:27.735 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-09-16T09:30:27.735 INFO:teuthology.orchestra.run.smithi084.stdout: "ops": [ 2024-09-16T09:30:27.735 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:27.735 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_repop(client.25064.0:58900 8.8 e171/167 8:12747873:::benchmark_data_smithi070_107919_object58899:head v 171'3776, mlcod=156'87)", 2024-09-16T09:30:27.736 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:30:27.733237+0000", 2024-09-16T09:30:27.736 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 0.00058616600000000005, 2024-09-16T09:30:27.736 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000606889, 2024-09-16T09:30:27.736 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:30:27.736 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:30:27.736 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "started", 2024-09-16T09:30:27.736 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:30:27.736 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:27.736 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:30:27.736 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:27.733237+0000", 2024-09-16T09:30:27.736 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:30:27.736 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:27.736 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:27.737 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:30:27.737 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:27.733237+0000", 2024-09-16T09:30:27.737 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:30:27.737 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:27.737 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:27.737 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:30:27.737 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:27.733238+0000", 2024-09-16T09:30:27.737 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.094e-06 2024-09-16T09:30:27.737 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:27.737 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:27.737 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:30:27.737 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:27.733265+0000", 2024-09-16T09:30:27.737 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.6758e-05 2024-09-16T09:30:27.737 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:27.738 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:27.738 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:30:27.738 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:27.733284+0000", 2024-09-16T09:30:27.738 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.9381e-05 2024-09-16T09:30:27.738 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:27.738 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:27.738 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:30:27.738 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:27.733292+0000", 2024-09-16T09:30:27.738 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.8879999999999997e-06 2024-09-16T09:30:27.738 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:27.738 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:27.738 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:30:27.738 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:27.733349+0000", 2024-09-16T09:30:27.738 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.7370999999999997e-05 2024-09-16T09:30:27.739 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:27.739 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:27.739 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:30:27.739 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:27.733383+0000", 2024-09-16T09:30:27.739 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.3977999999999999e-05 2024-09-16T09:30:27.739 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:27.739 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:30:27.739 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:27.739 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:27.739 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-09-16T09:30:27.739 INFO:teuthology.orchestra.run.smithi084.stdout: "num_ops": 1 2024-09-16T09:30:27.739 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-09-16T09:30:27.757 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:27.758 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:27.758 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:30:27.859 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:30:27.894 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:27.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:27.895 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:30:27.955 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 32 16 59204 59188 115.574 79.25 0.00555415 0.00864583 2024-09-16T09:30:27.996 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:30:28.036 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:28.037 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:28.037 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:30:28.138 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:30:28.172 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:28.173 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:28.173 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:30:28.274 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:30:28.299 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:28.300 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:28.300 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:30:28.401 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:30:28.425 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:28.426 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:28.426 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:30:28.527 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:30:28.565 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:28.565 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:28.566 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:30:28.667 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:30:28.702 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:28.703 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:28.703 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:30:28.745 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:28 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:30:28.746 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:28 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:30:28.746 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:28 smithi070 ceph-mon[30388]: osdmap e171: 8 total, 8 up, 8 in 2024-09-16T09:30:28.746 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:28 smithi070 ceph-mon[30388]: pgmap v249: 168 pgs: 1 activating+undersized+remapped, 2 activating+undersized+degraded+remapped, 165 active+clean; 3.3 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 0 B/s rd, 69 MiB/s wr, 1.10k op/s; 9922/160518 objects degraded (6.181%); 0 B/s, 11 objects/s recovering 2024-09-16T09:30:28.746 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:28 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2394102935' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:30:28.746 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:28 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:30:28.746 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:28 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:30:28.746 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:28 smithi070 ceph-mon[40691]: osdmap e171: 8 total, 8 up, 8 in 2024-09-16T09:30:28.746 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:28 smithi070 ceph-mon[40691]: pgmap v249: 168 pgs: 1 activating+undersized+remapped, 2 activating+undersized+degraded+remapped, 165 active+clean; 3.3 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 0 B/s rd, 69 MiB/s wr, 1.10k op/s; 9922/160518 objects degraded (6.181%); 0 B/s, 11 objects/s recovering 2024-09-16T09:30:28.746 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:28 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/2394102935' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:30:28.747 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:30:28.748 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T09:30:28.748 INFO:teuthology.orchestra.run.smithi070.stdout: "epoch": 171, 2024-09-16T09:30:28.748 INFO:teuthology.orchestra.run.smithi070.stdout: "fsid": "a1321834-740c-11ef-bceb-c7b262605968", 2024-09-16T09:30:28.748 INFO:teuthology.orchestra.run.smithi070.stdout: "created": "2024-09-16T09:19:14.121498+0000", 2024-09-16T09:30:28.748 INFO:teuthology.orchestra.run.smithi070.stdout: "modified": "2024-09-16T09:30:27.554985+0000", 2024-09-16T09:30:28.748 INFO:teuthology.orchestra.run.smithi070.stdout: "last_up_change": "2024-09-16T09:24:51.124496+0000", 2024-09-16T09:30:28.748 INFO:teuthology.orchestra.run.smithi070.stdout: "last_in_change": "2024-09-16T09:24:28.432920+0000", 2024-09-16T09:30:28.748 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-16T09:30:28.748 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_num": 5799936, 2024-09-16T09:30:28.748 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_set": [ 2024-09-16T09:30:28.748 INFO:teuthology.orchestra.run.smithi070.stdout: "pglog_hardlimit", 2024-09-16T09:30:28.749 INFO:teuthology.orchestra.run.smithi070.stdout: "purged_snapdirs", 2024-09-16T09:30:28.749 INFO:teuthology.orchestra.run.smithi070.stdout: "recovery_deletes", 2024-09-16T09:30:28.749 INFO:teuthology.orchestra.run.smithi070.stdout: "sortbitwise" 2024-09-16T09:30:28.749 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-16T09:30:28.749 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_version": 18, 2024-09-16T09:30:28.749 INFO:teuthology.orchestra.run.smithi070.stdout: "full_ratio": 0.94999998807907104, 2024-09-16T09:30:28.749 INFO:teuthology.orchestra.run.smithi070.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-16T09:30:28.749 INFO:teuthology.orchestra.run.smithi070.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-16T09:30:28.749 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_snapshot": "", 2024-09-16T09:30:28.749 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_max": 8, 2024-09-16T09:30:28.749 INFO:teuthology.orchestra.run.smithi070.stdout: "max_osd": 8, 2024-09-16T09:30:28.749 INFO:teuthology.orchestra.run.smithi070.stdout: "require_min_compat_client": "luminous", 2024-09-16T09:30:28.749 INFO:teuthology.orchestra.run.smithi070.stdout: "min_compat_client": "luminous", 2024-09-16T09:30:28.750 INFO:teuthology.orchestra.run.smithi070.stdout: "require_osd_release": "squid", 2024-09-16T09:30:28.750 INFO:teuthology.orchestra.run.smithi070.stdout: "allow_crimson": false, 2024-09-16T09:30:28.750 INFO:teuthology.orchestra.run.smithi070.stdout: "pools": [ 2024-09-16T09:30:28.750 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:28.750 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 1, 2024-09-16T09:30:28.750 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": ".mgr", 2024-09-16T09:30:28.750 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2024-09-16T09:22:27.617694+0000", 2024-09-16T09:30:28.750 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2024-09-16T09:30:28.750 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2024-09-16T09:30:28.750 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2024-09-16T09:30:28.750 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 3, 2024-09-16T09:30:28.750 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 2, 2024-09-16T09:30:28.750 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2024-09-16T09:30:28.750 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2024-09-16T09:30:28.750 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2024-09-16T09:30:28.751 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T09:30:28.751 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T09:30:28.751 INFO:teuthology.orchestra.run.smithi070.stdout: "is_stretch_pool": false, 2024-09-16T09:30:28.751 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2024-09-16T09:30:28.751 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2024-09-16T09:30:28.751 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 21, 2024-09-16T09:30:28.751 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 21, 2024-09-16T09:30:28.751 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 21, 2024-09-16T09:30:28.751 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 21, 2024-09-16T09:30:28.751 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 21, 2024-09-16T09:30:28.751 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2024-09-16T09:30:28.751 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2024-09-16T09:30:28.751 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2024-09-16T09:30:28.752 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2024-09-16T09:30:28.752 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2024-09-16T09:30:28.752 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2024-09-16T09:30:28.752 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2024-09-16T09:30:28.752 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.752 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "162", 2024-09-16T09:30:28.752 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2024-09-16T09:30:28.752 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T09:30:28.752 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "160", 2024-09-16T09:30:28.752 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2024-09-16T09:30:28.752 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2024-09-16T09:30:28.752 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2024-09-16T09:30:28.752 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2024-09-16T09:30:28.753 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2024-09-16T09:30:28.753 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2024-09-16T09:30:28.753 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2024-09-16T09:30:28.753 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2024-09-16T09:30:28.753 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2024-09-16T09:30:28.753 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2024-09-16T09:30:28.753 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2024-09-16T09:30:28.753 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2024-09-16T09:30:28.753 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2024-09-16T09:30:28.753 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2024-09-16T09:30:28.753 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2024-09-16T09:30:28.753 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T09:30:28.753 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T09:30:28.754 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T09:30:28.754 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2024-09-16T09:30:28.754 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2024-09-16T09:30:28.754 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2024-09-16T09:30:28.754 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2024-09-16T09:30:28.754 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2024-09-16T09:30:28.754 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.754 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2024-09-16T09:30:28.754 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2024-09-16T09:30:28.754 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2024-09-16T09:30:28.754 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2024-09-16T09:30:28.754 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2024-09-16T09:30:28.754 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T09:30:28.755 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2024-09-16T09:30:28.755 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2024-09-16T09:30:28.755 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2024-09-16T09:30:28.755 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2024-09-16T09:30:28.755 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2024-09-16T09:30:28.755 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2024-09-16T09:30:28.755 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_max": 32, 2024-09-16T09:30:28.755 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_min": 1 2024-09-16T09:30:28.755 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.755 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2024-09-16T09:30:28.755 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": {} 2024-09-16T09:30:28.755 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.756 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2024-09-16T09:30:28.756 INFO:teuthology.orchestra.run.smithi070.stdout: "score_type": "Fair distribution", 2024-09-16T09:30:28.756 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 2.2899999618530273, 2024-09-16T09:30:28.756 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 2.2899999618530273, 2024-09-16T09:30:28.756 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1, 2024-09-16T09:30:28.756 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 2.2899999618530273, 2024-09-16T09:30:28.756 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 2.2899999618530273, 2024-09-16T09:30:28.756 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 0.93456649780273438, 2024-09-16T09:30:28.756 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 0.93000000715255737, 2024-09-16T09:30:28.756 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 0.93000000715255737 2024-09-16T09:30:28.756 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:30:28.756 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.757 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:28.757 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 2, 2024-09-16T09:30:28.757 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "datapool", 2024-09-16T09:30:28.757 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2024-09-16T09:24:59.169244+0000", 2024-09-16T09:30:28.757 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 8193, 2024-09-16T09:30:28.757 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-16T09:30:28.757 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2024-09-16T09:30:28.757 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 3, 2024-09-16T09:30:28.757 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 2, 2024-09-16T09:30:28.757 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2024-09-16T09:30:28.757 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2024-09-16T09:30:28.757 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2024-09-16T09:30:28.757 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T09:30:28.758 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T09:30:28.758 INFO:teuthology.orchestra.run.smithi070.stdout: "is_stretch_pool": false, 2024-09-16T09:30:28.758 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2024-09-16T09:30:28.758 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2024-09-16T09:30:28.758 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 3, 2024-09-16T09:30:28.758 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 3, 2024-09-16T09:30:28.758 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 3, 2024-09-16T09:30:28.758 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 3, 2024-09-16T09:30:28.758 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 3, 2024-09-16T09:30:28.758 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2024-09-16T09:30:28.758 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2024-09-16T09:30:28.758 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2024-09-16T09:30:28.758 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2024-09-16T09:30:28.758 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2024-09-16T09:30:28.759 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2024-09-16T09:30:28.759 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2024-09-16T09:30:28.759 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.759 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "61", 2024-09-16T09:30:28.759 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2024-09-16T09:30:28.759 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T09:30:28.759 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T09:30:28.759 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2024-09-16T09:30:28.759 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2024-09-16T09:30:28.759 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 2, 2024-09-16T09:30:28.759 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 61, 2024-09-16T09:30:28.759 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2024-09-16T09:30:28.759 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2024-09-16T09:30:28.760 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2024-09-16T09:30:28.760 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2024-09-16T09:30:28.760 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2024-09-16T09:30:28.760 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2024-09-16T09:30:28.760 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2024-09-16T09:30:28.760 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2024-09-16T09:30:28.760 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2024-09-16T09:30:28.760 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2024-09-16T09:30:28.760 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2024-09-16T09:30:28.760 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T09:30:28.760 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T09:30:28.760 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T09:30:28.760 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2024-09-16T09:30:28.760 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2024-09-16T09:30:28.761 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2024-09-16T09:30:28.761 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2024-09-16T09:30:28.761 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2024-09-16T09:30:28.761 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.761 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2024-09-16T09:30:28.761 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2024-09-16T09:30:28.761 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2024-09-16T09:30:28.761 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2024-09-16T09:30:28.761 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2024-09-16T09:30:28.761 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T09:30:28.761 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2024-09-16T09:30:28.761 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2024-09-16T09:30:28.761 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2024-09-16T09:30:28.762 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2024-09-16T09:30:28.762 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2024-09-16T09:30:28.762 INFO:teuthology.orchestra.run.smithi070.stdout: "options": {}, 2024-09-16T09:30:28.762 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2024-09-16T09:30:28.762 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd": {} 2024-09-16T09:30:28.762 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.762 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2024-09-16T09:30:28.762 INFO:teuthology.orchestra.run.smithi070.stdout: "score_type": "Fair distribution", 2024-09-16T09:30:28.762 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 2.6700000762939453, 2024-09-16T09:30:28.762 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 2.6700000762939453, 2024-09-16T09:30:28.762 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 0.75, 2024-09-16T09:30:28.762 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 2, 2024-09-16T09:30:28.762 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 2, 2024-09-16T09:30:28.762 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 0.93456649780273438, 2024-09-16T09:30:28.763 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 0.93000000715255737, 2024-09-16T09:30:28.763 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 0.93000000715255737 2024-09-16T09:30:28.763 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:30:28.763 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.763 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:28.763 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 3, 2024-09-16T09:30:28.763 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": ".rgw.root", 2024-09-16T09:30:28.763 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2024-09-16T09:25:00.056119+0000", 2024-09-16T09:30:28.763 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2024-09-16T09:30:28.763 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2024-09-16T09:30:28.763 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2024-09-16T09:30:28.763 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 3, 2024-09-16T09:30:28.763 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 2, 2024-09-16T09:30:28.763 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2024-09-16T09:30:28.764 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2024-09-16T09:30:28.764 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2024-09-16T09:30:28.764 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T09:30:28.764 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T09:30:28.764 INFO:teuthology.orchestra.run.smithi070.stdout: "is_stretch_pool": false, 2024-09-16T09:30:28.764 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2024-09-16T09:30:28.764 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2024-09-16T09:30:28.764 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 32, 2024-09-16T09:30:28.764 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 32, 2024-09-16T09:30:28.764 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 32, 2024-09-16T09:30:28.764 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 32, 2024-09-16T09:30:28.764 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 32, 2024-09-16T09:30:28.764 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2024-09-16T09:30:28.765 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2024-09-16T09:30:28.765 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2024-09-16T09:30:28.765 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2024-09-16T09:30:28.765 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2024-09-16T09:30:28.765 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2024-09-16T09:30:28.765 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2024-09-16T09:30:28.765 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.765 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "60", 2024-09-16T09:30:28.765 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2024-09-16T09:30:28.765 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T09:30:28.765 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T09:30:28.765 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2024-09-16T09:30:28.765 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2024-09-16T09:30:28.766 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2024-09-16T09:30:28.766 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2024-09-16T09:30:28.766 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2024-09-16T09:30:28.766 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2024-09-16T09:30:28.766 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2024-09-16T09:30:28.766 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2024-09-16T09:30:28.766 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2024-09-16T09:30:28.766 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2024-09-16T09:30:28.766 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2024-09-16T09:30:28.766 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2024-09-16T09:30:28.766 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2024-09-16T09:30:28.767 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2024-09-16T09:30:28.767 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2024-09-16T09:30:28.767 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T09:30:28.767 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T09:30:28.767 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T09:30:28.767 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2024-09-16T09:30:28.767 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2024-09-16T09:30:28.767 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2024-09-16T09:30:28.767 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2024-09-16T09:30:28.767 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2024-09-16T09:30:28.767 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.767 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2024-09-16T09:30:28.767 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2024-09-16T09:30:28.768 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2024-09-16T09:30:28.768 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2024-09-16T09:30:28.768 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2024-09-16T09:30:28.768 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T09:30:28.768 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2024-09-16T09:30:28.768 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2024-09-16T09:30:28.768 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2024-09-16T09:30:28.768 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2024-09-16T09:30:28.768 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2024-09-16T09:30:28.768 INFO:teuthology.orchestra.run.smithi070.stdout: "options": {}, 2024-09-16T09:30:28.768 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2024-09-16T09:30:28.768 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw": {} 2024-09-16T09:30:28.769 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.769 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2024-09-16T09:30:28.769 INFO:teuthology.orchestra.run.smithi070.stdout: "score_type": "Fair distribution", 2024-09-16T09:30:28.769 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 2, 2024-09-16T09:30:28.769 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 2, 2024-09-16T09:30:28.769 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1, 2024-09-16T09:30:28.769 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 2, 2024-09-16T09:30:28.769 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 2, 2024-09-16T09:30:28.769 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 0.93456649780273438, 2024-09-16T09:30:28.769 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 0.93000000715255737, 2024-09-16T09:30:28.769 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 0.93000000715255737 2024-09-16T09:30:28.769 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:30:28.770 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.770 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:28.770 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 4, 2024-09-16T09:30:28.770 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "default.rgw.log", 2024-09-16T09:30:28.770 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2024-09-16T09:25:01.770016+0000", 2024-09-16T09:30:28.770 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2024-09-16T09:30:28.770 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2024-09-16T09:30:28.770 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2024-09-16T09:30:28.770 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 3, 2024-09-16T09:30:28.770 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 2, 2024-09-16T09:30:28.770 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2024-09-16T09:30:28.770 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2024-09-16T09:30:28.770 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2024-09-16T09:30:28.771 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T09:30:28.771 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T09:30:28.771 INFO:teuthology.orchestra.run.smithi070.stdout: "is_stretch_pool": false, 2024-09-16T09:30:28.771 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2024-09-16T09:30:28.771 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2024-09-16T09:30:28.771 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 32, 2024-09-16T09:30:28.771 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 32, 2024-09-16T09:30:28.771 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 32, 2024-09-16T09:30:28.771 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 32, 2024-09-16T09:30:28.771 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 32, 2024-09-16T09:30:28.771 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2024-09-16T09:30:28.771 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2024-09-16T09:30:28.771 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2024-09-16T09:30:28.771 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2024-09-16T09:30:28.772 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2024-09-16T09:30:28.772 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2024-09-16T09:30:28.772 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2024-09-16T09:30:28.772 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.772 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "62", 2024-09-16T09:30:28.772 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2024-09-16T09:30:28.772 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T09:30:28.772 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T09:30:28.772 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2024-09-16T09:30:28.772 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2024-09-16T09:30:28.772 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2024-09-16T09:30:28.772 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2024-09-16T09:30:28.772 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2024-09-16T09:30:28.773 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2024-09-16T09:30:28.773 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2024-09-16T09:30:28.773 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2024-09-16T09:30:28.773 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2024-09-16T09:30:28.773 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2024-09-16T09:30:28.773 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2024-09-16T09:30:28.773 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2024-09-16T09:30:28.773 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2024-09-16T09:30:28.773 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2024-09-16T09:30:28.773 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2024-09-16T09:30:28.773 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T09:30:28.773 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T09:30:28.773 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T09:30:28.773 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2024-09-16T09:30:28.774 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2024-09-16T09:30:28.774 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2024-09-16T09:30:28.774 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2024-09-16T09:30:28.774 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2024-09-16T09:30:28.774 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.774 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2024-09-16T09:30:28.774 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2024-09-16T09:30:28.774 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2024-09-16T09:30:28.774 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2024-09-16T09:30:28.774 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2024-09-16T09:30:28.774 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T09:30:28.774 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2024-09-16T09:30:28.774 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2024-09-16T09:30:28.775 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2024-09-16T09:30:28.775 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2024-09-16T09:30:28.775 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2024-09-16T09:30:28.775 INFO:teuthology.orchestra.run.smithi070.stdout: "options": {}, 2024-09-16T09:30:28.775 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2024-09-16T09:30:28.775 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw": {} 2024-09-16T09:30:28.775 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.775 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2024-09-16T09:30:28.775 INFO:teuthology.orchestra.run.smithi070.stdout: "score_type": "Fair distribution", 2024-09-16T09:30:28.775 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 2.25, 2024-09-16T09:30:28.775 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 2.25, 2024-09-16T09:30:28.775 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1, 2024-09-16T09:30:28.775 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 2.25, 2024-09-16T09:30:28.776 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 2.25, 2024-09-16T09:30:28.776 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 0.93456649780273438, 2024-09-16T09:30:28.776 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 0.93000000715255737, 2024-09-16T09:30:28.776 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 0.93000000715255737 2024-09-16T09:30:28.776 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:30:28.776 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.776 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:28.776 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 5, 2024-09-16T09:30:28.776 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "default.rgw.control", 2024-09-16T09:30:28.776 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2024-09-16T09:25:03.725555+0000", 2024-09-16T09:30:28.776 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2024-09-16T09:30:28.776 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2024-09-16T09:30:28.776 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2024-09-16T09:30:28.776 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 3, 2024-09-16T09:30:28.777 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 2, 2024-09-16T09:30:28.777 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2024-09-16T09:30:28.777 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2024-09-16T09:30:28.777 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2024-09-16T09:30:28.777 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T09:30:28.777 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T09:30:28.777 INFO:teuthology.orchestra.run.smithi070.stdout: "is_stretch_pool": false, 2024-09-16T09:30:28.777 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2024-09-16T09:30:28.777 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2024-09-16T09:30:28.777 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 32, 2024-09-16T09:30:28.777 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 32, 2024-09-16T09:30:28.777 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 32, 2024-09-16T09:30:28.777 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 32, 2024-09-16T09:30:28.777 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 32, 2024-09-16T09:30:28.778 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2024-09-16T09:30:28.778 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2024-09-16T09:30:28.778 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2024-09-16T09:30:28.778 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2024-09-16T09:30:28.778 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2024-09-16T09:30:28.778 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2024-09-16T09:30:28.778 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2024-09-16T09:30:28.778 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.778 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "64", 2024-09-16T09:30:28.778 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2024-09-16T09:30:28.778 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T09:30:28.778 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T09:30:28.778 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2024-09-16T09:30:28.778 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2024-09-16T09:30:28.778 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2024-09-16T09:30:28.779 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2024-09-16T09:30:28.779 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2024-09-16T09:30:28.779 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2024-09-16T09:30:28.779 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2024-09-16T09:30:28.779 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2024-09-16T09:30:28.779 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2024-09-16T09:30:28.779 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2024-09-16T09:30:28.779 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2024-09-16T09:30:28.779 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2024-09-16T09:30:28.779 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2024-09-16T09:30:28.779 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2024-09-16T09:30:28.779 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2024-09-16T09:30:28.779 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T09:30:28.779 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T09:30:28.779 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T09:30:28.780 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2024-09-16T09:30:28.780 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2024-09-16T09:30:28.780 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2024-09-16T09:30:28.780 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2024-09-16T09:30:28.780 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2024-09-16T09:30:28.780 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.780 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2024-09-16T09:30:28.780 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2024-09-16T09:30:28.780 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2024-09-16T09:30:28.780 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2024-09-16T09:30:28.780 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2024-09-16T09:30:28.780 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T09:30:28.780 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2024-09-16T09:30:28.780 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2024-09-16T09:30:28.781 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2024-09-16T09:30:28.781 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2024-09-16T09:30:28.781 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2024-09-16T09:30:28.781 INFO:teuthology.orchestra.run.smithi070.stdout: "options": {}, 2024-09-16T09:30:28.781 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2024-09-16T09:30:28.781 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw": {} 2024-09-16T09:30:28.781 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.781 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2024-09-16T09:30:28.781 INFO:teuthology.orchestra.run.smithi070.stdout: "score_type": "Fair distribution", 2024-09-16T09:30:28.781 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 1.5, 2024-09-16T09:30:28.781 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 1.5, 2024-09-16T09:30:28.781 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1, 2024-09-16T09:30:28.781 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 1.5, 2024-09-16T09:30:28.781 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 1.5, 2024-09-16T09:30:28.782 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 0.93456649780273438, 2024-09-16T09:30:28.782 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 0.93000000715255737, 2024-09-16T09:30:28.782 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 0.93000000715255737 2024-09-16T09:30:28.782 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:30:28.783 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.783 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:28.783 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 6, 2024-09-16T09:30:28.783 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "default.rgw.meta", 2024-09-16T09:30:28.783 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2024-09-16T09:25:05.773530+0000", 2024-09-16T09:30:28.783 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2024-09-16T09:30:28.783 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2024-09-16T09:30:28.783 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2024-09-16T09:30:28.783 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 3, 2024-09-16T09:30:28.783 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 2, 2024-09-16T09:30:28.783 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2024-09-16T09:30:28.783 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2024-09-16T09:30:28.783 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2024-09-16T09:30:28.783 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T09:30:28.784 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T09:30:28.784 INFO:teuthology.orchestra.run.smithi070.stdout: "is_stretch_pool": false, 2024-09-16T09:30:28.784 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2024-09-16T09:30:28.784 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2024-09-16T09:30:28.784 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 32, 2024-09-16T09:30:28.784 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 32, 2024-09-16T09:30:28.784 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 32, 2024-09-16T09:30:28.784 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 32, 2024-09-16T09:30:28.784 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 32, 2024-09-16T09:30:28.784 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2024-09-16T09:30:28.784 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2024-09-16T09:30:28.784 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2024-09-16T09:30:28.785 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2024-09-16T09:30:28.785 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2024-09-16T09:30:28.785 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2024-09-16T09:30:28.785 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2024-09-16T09:30:28.785 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.785 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "66", 2024-09-16T09:30:28.785 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2024-09-16T09:30:28.785 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T09:30:28.785 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T09:30:28.785 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2024-09-16T09:30:28.785 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2024-09-16T09:30:28.785 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2024-09-16T09:30:28.785 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2024-09-16T09:30:28.786 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2024-09-16T09:30:28.786 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2024-09-16T09:30:28.786 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2024-09-16T09:30:28.786 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2024-09-16T09:30:28.786 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2024-09-16T09:30:28.786 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2024-09-16T09:30:28.786 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2024-09-16T09:30:28.786 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2024-09-16T09:30:28.786 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2024-09-16T09:30:28.786 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2024-09-16T09:30:28.786 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2024-09-16T09:30:28.786 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T09:30:28.786 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T09:30:28.786 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T09:30:28.787 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2024-09-16T09:30:28.787 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2024-09-16T09:30:28.787 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2024-09-16T09:30:28.787 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2024-09-16T09:30:28.787 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2024-09-16T09:30:28.787 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.787 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2024-09-16T09:30:28.787 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2024-09-16T09:30:28.787 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2024-09-16T09:30:28.787 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2024-09-16T09:30:28.787 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2024-09-16T09:30:28.787 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T09:30:28.787 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2024-09-16T09:30:28.787 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2024-09-16T09:30:28.788 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2024-09-16T09:30:28.788 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2024-09-16T09:30:28.788 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2024-09-16T09:30:28.788 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2024-09-16T09:30:28.788 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_bias": 4 2024-09-16T09:30:28.788 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.788 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2024-09-16T09:30:28.788 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw": {} 2024-09-16T09:30:28.788 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.788 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2024-09-16T09:30:28.788 INFO:teuthology.orchestra.run.smithi070.stdout: "score_type": "Fair distribution", 2024-09-16T09:30:28.788 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 1.75, 2024-09-16T09:30:28.788 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 1.75, 2024-09-16T09:30:28.789 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1, 2024-09-16T09:30:28.789 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 1.75, 2024-09-16T09:30:28.789 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 1.75, 2024-09-16T09:30:28.789 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 0.93456649780273438, 2024-09-16T09:30:28.789 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 0.93000000715255737, 2024-09-16T09:30:28.789 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 0.93000000715255737 2024-09-16T09:30:28.789 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:30:28.789 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.789 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:28.789 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 8, 2024-09-16T09:30:28.789 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "unique_pool_1", 2024-09-16T09:30:28.789 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2024-09-16T09:29:48.079163+0000", 2024-09-16T09:30:28.789 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2024-09-16T09:30:28.789 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2024-09-16T09:30:28.790 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2024-09-16T09:30:28.790 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 3, 2024-09-16T09:30:28.790 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 2, 2024-09-16T09:30:28.790 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2024-09-16T09:30:28.790 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2024-09-16T09:30:28.790 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2024-09-16T09:30:28.790 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T09:30:28.790 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T09:30:28.790 INFO:teuthology.orchestra.run.smithi070.stdout: "is_stretch_pool": false, 2024-09-16T09:30:28.790 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2024-09-16T09:30:28.790 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2024-09-16T09:30:28.790 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 16, 2024-09-16T09:30:28.790 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 16, 2024-09-16T09:30:28.791 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 16, 2024-09-16T09:30:28.791 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 16, 2024-09-16T09:30:28.791 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 16, 2024-09-16T09:30:28.791 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2024-09-16T09:30:28.791 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2024-09-16T09:30:28.791 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2024-09-16T09:30:28.791 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2024-09-16T09:30:28.791 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2024-09-16T09:30:28.791 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2024-09-16T09:30:28.791 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2024-09-16T09:30:28.791 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.791 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "155", 2024-09-16T09:30:28.791 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2024-09-16T09:30:28.791 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T09:30:28.792 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T09:30:28.792 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2024-09-16T09:30:28.792 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2024-09-16T09:30:28.792 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2024-09-16T09:30:28.792 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2024-09-16T09:30:28.792 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2024-09-16T09:30:28.792 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2024-09-16T09:30:28.792 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2024-09-16T09:30:28.792 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2024-09-16T09:30:28.792 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2024-09-16T09:30:28.792 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2024-09-16T09:30:28.792 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2024-09-16T09:30:28.792 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2024-09-16T09:30:28.793 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2024-09-16T09:30:28.793 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2024-09-16T09:30:28.793 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2024-09-16T09:30:28.793 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T09:30:28.793 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T09:30:28.793 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T09:30:28.793 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2024-09-16T09:30:28.793 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2024-09-16T09:30:28.793 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2024-09-16T09:30:28.793 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2024-09-16T09:30:28.793 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2024-09-16T09:30:28.793 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.793 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2024-09-16T09:30:28.794 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2024-09-16T09:30:28.794 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2024-09-16T09:30:28.794 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2024-09-16T09:30:28.794 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2024-09-16T09:30:28.794 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T09:30:28.794 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2024-09-16T09:30:28.794 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2024-09-16T09:30:28.794 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2024-09-16T09:30:28.794 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2024-09-16T09:30:28.794 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2024-09-16T09:30:28.794 INFO:teuthology.orchestra.run.smithi070.stdout: "options": {}, 2024-09-16T09:30:28.794 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2024-09-16T09:30:28.795 INFO:teuthology.orchestra.run.smithi070.stdout: "rados": {} 2024-09-16T09:30:28.795 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.795 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2024-09-16T09:30:28.795 INFO:teuthology.orchestra.run.smithi070.stdout: "score_type": "Fair distribution", 2024-09-16T09:30:28.795 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 2, 2024-09-16T09:30:28.795 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 2, 2024-09-16T09:30:28.795 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1, 2024-09-16T09:30:28.795 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 2, 2024-09-16T09:30:28.795 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 2, 2024-09-16T09:30:28.795 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 0.93456649780273438, 2024-09-16T09:30:28.795 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 0.93000000715255737, 2024-09-16T09:30:28.795 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 0.93000000715255737 2024-09-16T09:30:28.795 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:30:28.795 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:30:28.796 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-16T09:30:28.796 INFO:teuthology.orchestra.run.smithi070.stdout: "osds": [ 2024-09-16T09:30:28.796 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:28.796 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 0, 2024-09-16T09:30:28.796 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "7ef288c8-3bf0-4814-8e86-f70a7de041ce", 2024-09-16T09:30:28.796 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2024-09-16T09:30:28.796 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2024-09-16T09:30:28.796 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2024-09-16T09:30:28.796 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2024-09-16T09:30:28.796 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2024-09-16T09:30:28.796 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2024-09-16T09:30:28.796 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 8, 2024-09-16T09:30:28.796 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 166, 2024-09-16T09:30:28.797 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2024-09-16T09:30:28.797 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2024-09-16T09:30:28.797 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2024-09-16T09:30:28.797 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:30:28.797 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:28.797 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:30:28.797 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6802", 2024-09-16T09:30:28.797 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2013075781 2024-09-16T09:30:28.797 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.797 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:28.797 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:30:28.797 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6803", 2024-09-16T09:30:28.797 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2013075781 2024-09-16T09:30:28.798 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:30:28.798 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:30:28.798 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.798 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2024-09-16T09:30:28.798 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:30:28.798 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:28.798 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:30:28.798 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6804", 2024-09-16T09:30:28.798 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2013075781 2024-09-16T09:30:28.798 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.798 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:28.798 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:30:28.798 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6805", 2024-09-16T09:30:28.799 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2013075781 2024-09-16T09:30:28.799 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:30:28.799 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:30:28.799 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.799 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2024-09-16T09:30:28.799 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:30:28.799 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:28.799 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:30:28.799 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6808", 2024-09-16T09:30:28.799 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2013075781 2024-09-16T09:30:28.799 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.799 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:28.799 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:30:28.799 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6809", 2024-09-16T09:30:28.800 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2013075781 2024-09-16T09:30:28.800 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:30:28.800 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:30:28.800 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.800 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2024-09-16T09:30:28.800 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:30:28.800 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:28.800 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:30:28.800 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6806", 2024-09-16T09:30:28.800 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2013075781 2024-09-16T09:30:28.800 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.800 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:28.801 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:30:28.801 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6807", 2024-09-16T09:30:28.801 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2013075781 2024-09-16T09:30:28.801 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:30:28.801 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:30:28.801 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.801 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6803/2013075781", 2024-09-16T09:30:28.801 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6805/2013075781", 2024-09-16T09:30:28.801 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6809/2013075781", 2024-09-16T09:30:28.801 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6807/2013075781", 2024-09-16T09:30:28.801 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2024-09-16T09:30:28.801 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2024-09-16T09:30:28.802 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2024-09-16T09:30:28.802 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:30:28.802 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.802 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:28.802 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 1, 2024-09-16T09:30:28.802 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "474d41d9-295a-450c-8f27-ce487a14678b", 2024-09-16T09:30:28.802 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2024-09-16T09:30:28.802 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2024-09-16T09:30:28.802 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2024-09-16T09:30:28.802 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2024-09-16T09:30:28.802 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2024-09-16T09:30:28.802 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2024-09-16T09:30:28.802 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2024-09-16T09:30:28.803 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 166, 2024-09-16T09:30:28.803 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2024-09-16T09:30:28.803 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2024-09-16T09:30:28.803 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2024-09-16T09:30:28.803 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:30:28.803 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:28.803 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:30:28.803 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6810", 2024-09-16T09:30:28.803 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1670012115 2024-09-16T09:30:28.803 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.803 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:28.803 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:30:28.803 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6811", 2024-09-16T09:30:28.804 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1670012115 2024-09-16T09:30:28.804 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:30:28.804 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:30:28.804 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.804 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2024-09-16T09:30:28.804 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:30:28.804 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:28.804 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:30:28.804 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6812", 2024-09-16T09:30:28.804 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1670012115 2024-09-16T09:30:28.804 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.804 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:28.804 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:30:28.805 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6813", 2024-09-16T09:30:28.805 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1670012115 2024-09-16T09:30:28.805 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:30:28.805 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:30:28.805 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.805 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2024-09-16T09:30:28.805 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:30:28.805 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:28.805 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:30:28.805 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6816", 2024-09-16T09:30:28.805 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1670012115 2024-09-16T09:30:28.805 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.805 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:28.805 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:30:28.806 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6817", 2024-09-16T09:30:28.806 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1670012115 2024-09-16T09:30:28.806 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:30:28.806 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:30:28.806 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.806 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2024-09-16T09:30:28.806 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:30:28.806 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:28.806 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:30:28.806 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6814", 2024-09-16T09:30:28.806 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1670012115 2024-09-16T09:30:28.806 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.806 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:28.806 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:30:28.807 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6815", 2024-09-16T09:30:28.807 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1670012115 2024-09-16T09:30:28.807 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:30:28.807 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:30:28.807 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.807 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6811/1670012115", 2024-09-16T09:30:28.807 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6813/1670012115", 2024-09-16T09:30:28.807 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6817/1670012115", 2024-09-16T09:30:28.807 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6815/1670012115", 2024-09-16T09:30:28.807 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2024-09-16T09:30:28.807 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2024-09-16T09:30:28.807 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2024-09-16T09:30:28.807 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:30:28.807 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.807 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:28.808 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 2, 2024-09-16T09:30:28.808 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "5b4ca4e8-cecc-4163-b291-c9209b02c4b7", 2024-09-16T09:30:28.808 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2024-09-16T09:30:28.808 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2024-09-16T09:30:28.808 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2024-09-16T09:30:28.808 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2024-09-16T09:30:28.808 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2024-09-16T09:30:28.808 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2024-09-16T09:30:28.808 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 18, 2024-09-16T09:30:28.808 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 166, 2024-09-16T09:30:28.808 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2024-09-16T09:30:28.808 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2024-09-16T09:30:28.808 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2024-09-16T09:30:28.808 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:30:28.809 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:28.809 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:30:28.809 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6818", 2024-09-16T09:30:28.809 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3601682336 2024-09-16T09:30:28.809 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.809 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:28.809 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:30:28.809 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6819", 2024-09-16T09:30:28.809 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3601682336 2024-09-16T09:30:28.809 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:30:28.809 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:30:28.809 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.809 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2024-09-16T09:30:28.810 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:30:28.810 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:28.810 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:30:28.810 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6820", 2024-09-16T09:30:28.810 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3601682336 2024-09-16T09:30:28.810 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.810 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:28.810 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:30:28.810 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6821", 2024-09-16T09:30:28.810 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3601682336 2024-09-16T09:30:28.810 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:30:28.810 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:30:28.810 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.811 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2024-09-16T09:30:28.811 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:30:28.811 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:28.811 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:30:28.811 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6824", 2024-09-16T09:30:28.811 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3601682336 2024-09-16T09:30:28.811 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.811 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:28.811 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:30:28.811 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6825", 2024-09-16T09:30:28.811 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3601682336 2024-09-16T09:30:28.811 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:30:28.811 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:30:28.812 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.812 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2024-09-16T09:30:28.812 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:30:28.812 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:28.812 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:30:28.812 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6822", 2024-09-16T09:30:28.812 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3601682336 2024-09-16T09:30:28.812 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.812 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:28.812 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:30:28.812 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6823", 2024-09-16T09:30:28.812 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3601682336 2024-09-16T09:30:28.812 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:30:28.813 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:30:28.813 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.813 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6819/3601682336", 2024-09-16T09:30:28.813 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6821/3601682336", 2024-09-16T09:30:28.813 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6825/3601682336", 2024-09-16T09:30:28.813 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6823/3601682336", 2024-09-16T09:30:28.813 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2024-09-16T09:30:28.813 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2024-09-16T09:30:28.813 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2024-09-16T09:30:28.813 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:30:28.813 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.813 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:28.814 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 3, 2024-09-16T09:30:28.814 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "ffb8d527-1b58-4ecd-808c-47d4ba1bf387", 2024-09-16T09:30:28.814 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2024-09-16T09:30:28.814 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2024-09-16T09:30:28.814 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 0.3394775390625, 2024-09-16T09:30:28.814 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 0.476531982421875, 2024-09-16T09:30:28.814 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2024-09-16T09:30:28.814 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2024-09-16T09:30:28.814 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 26, 2024-09-16T09:30:28.814 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 162, 2024-09-16T09:30:28.814 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2024-09-16T09:30:28.814 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2024-09-16T09:30:28.814 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2024-09-16T09:30:28.814 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:30:28.815 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:28.815 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:30:28.815 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6826", 2024-09-16T09:30:28.815 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2038083007 2024-09-16T09:30:28.815 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.815 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:28.815 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:30:28.815 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6827", 2024-09-16T09:30:28.815 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2038083007 2024-09-16T09:30:28.815 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:30:28.815 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:30:28.815 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.815 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2024-09-16T09:30:28.815 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:30:28.816 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:28.816 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:30:28.816 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6828", 2024-09-16T09:30:28.816 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2038083007 2024-09-16T09:30:28.816 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.816 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:28.816 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:30:28.816 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6829", 2024-09-16T09:30:28.816 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2038083007 2024-09-16T09:30:28.816 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:30:28.816 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:30:28.816 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.816 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2024-09-16T09:30:28.817 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:30:28.817 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:28.817 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:30:28.817 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6832", 2024-09-16T09:30:28.817 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2038083007 2024-09-16T09:30:28.817 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.818 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:28.818 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:30:28.818 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6833", 2024-09-16T09:30:28.818 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2038083007 2024-09-16T09:30:28.818 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:30:28.818 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:30:28.818 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.818 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2024-09-16T09:30:28.818 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:30:28.818 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:28.818 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:30:28.818 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6830", 2024-09-16T09:30:28.818 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2038083007 2024-09-16T09:30:28.819 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.819 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:28.819 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:30:28.819 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6831", 2024-09-16T09:30:28.819 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2038083007 2024-09-16T09:30:28.819 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:30:28.819 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:30:28.819 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.819 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6827/2038083007", 2024-09-16T09:30:28.819 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:30:28.820 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6829/2038083007", 2024-09-16T09:30:28.820 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6833/2038083007", 2024-09-16T09:30:28.820 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6831/2038083007", 2024-09-16T09:30:28.820 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2024-09-16T09:30:28.821 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2024-09-16T09:30:28.821 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2024-09-16T09:30:28.821 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:30:28.821 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.821 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:28.821 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 4, 2024-09-16T09:30:28.821 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "a946760b-4d91-4a4b-8e9b-a07a50589211", 2024-09-16T09:30:28.821 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2024-09-16T09:30:28.821 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2024-09-16T09:30:28.821 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2024-09-16T09:30:28.821 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2024-09-16T09:30:28.821 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2024-09-16T09:30:28.821 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2024-09-16T09:30:28.822 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 32, 2024-09-16T09:30:28.822 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 167, 2024-09-16T09:30:28.822 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2024-09-16T09:30:28.822 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2024-09-16T09:30:28.822 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2024-09-16T09:30:28.822 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:30:28.822 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:28.822 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:30:28.822 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6800", 2024-09-16T09:30:28.822 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2250892710 2024-09-16T09:30:28.822 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.822 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:28.822 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:30:28.823 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6801", 2024-09-16T09:30:28.823 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2250892710 2024-09-16T09:30:28.823 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:30:28.823 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:30:28.823 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.823 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2024-09-16T09:30:28.823 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:30:28.823 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:28.823 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:30:28.823 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6802", 2024-09-16T09:30:28.823 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2250892710 2024-09-16T09:30:28.823 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.823 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:28.824 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:30:28.824 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6803", 2024-09-16T09:30:28.824 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2250892710 2024-09-16T09:30:28.824 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:30:28.824 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:30:28.824 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.824 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2024-09-16T09:30:28.824 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:30:28.824 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:28.824 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:30:28.824 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6806", 2024-09-16T09:30:28.824 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2250892710 2024-09-16T09:30:28.824 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.825 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:28.825 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:30:28.825 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6807", 2024-09-16T09:30:28.825 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2250892710 2024-09-16T09:30:28.825 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:30:28.825 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:30:28.825 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.825 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2024-09-16T09:30:28.825 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:30:28.825 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:28.825 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:30:28.825 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6804", 2024-09-16T09:30:28.826 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2250892710 2024-09-16T09:30:28.826 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.826 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:28.826 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:30:28.826 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6805", 2024-09-16T09:30:28.826 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2250892710 2024-09-16T09:30:28.826 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:30:28.826 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:30:28.826 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.826 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.84:6801/2250892710", 2024-09-16T09:30:28.826 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.84:6803/2250892710", 2024-09-16T09:30:28.826 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.84:6807/2250892710", 2024-09-16T09:30:28.826 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.84:6805/2250892710", 2024-09-16T09:30:28.827 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2024-09-16T09:30:28.827 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2024-09-16T09:30:28.827 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2024-09-16T09:30:28.827 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:30:28.827 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.827 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:28.827 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 5, 2024-09-16T09:30:28.827 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "67681760-75a9-4846-946e-a935d42c5e05", 2024-09-16T09:30:28.827 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2024-09-16T09:30:28.827 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2024-09-16T09:30:28.827 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2024-09-16T09:30:28.827 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2024-09-16T09:30:28.827 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2024-09-16T09:30:28.827 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2024-09-16T09:30:28.827 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 39, 2024-09-16T09:30:28.828 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 167, 2024-09-16T09:30:28.828 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2024-09-16T09:30:28.828 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2024-09-16T09:30:28.828 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2024-09-16T09:30:28.828 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:30:28.828 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:28.828 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:30:28.828 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6808", 2024-09-16T09:30:28.828 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1803945348 2024-09-16T09:30:28.828 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.828 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:28.828 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:30:28.828 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6809", 2024-09-16T09:30:28.828 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1803945348 2024-09-16T09:30:28.829 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:30:28.829 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:30:28.829 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.829 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2024-09-16T09:30:28.829 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:30:28.829 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:28.829 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:30:28.829 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6810", 2024-09-16T09:30:28.829 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1803945348 2024-09-16T09:30:28.829 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.829 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:28.829 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:30:28.829 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6811", 2024-09-16T09:30:28.830 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1803945348 2024-09-16T09:30:28.830 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:30:28.830 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:30:28.830 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.830 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2024-09-16T09:30:28.830 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:30:28.830 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:28.830 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:30:28.830 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6814", 2024-09-16T09:30:28.830 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1803945348 2024-09-16T09:30:28.830 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.830 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:28.830 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:30:28.831 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6815", 2024-09-16T09:30:28.831 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1803945348 2024-09-16T09:30:28.831 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:30:28.831 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:30:28.831 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.831 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2024-09-16T09:30:28.831 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:30:28.831 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:28.831 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:30:28.831 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6812", 2024-09-16T09:30:28.831 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1803945348 2024-09-16T09:30:28.831 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.831 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:28.832 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:30:28.832 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6813", 2024-09-16T09:30:28.832 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1803945348 2024-09-16T09:30:28.832 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:30:28.832 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:30:28.832 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.832 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.84:6809/1803945348", 2024-09-16T09:30:28.832 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.84:6811/1803945348", 2024-09-16T09:30:28.832 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.84:6815/1803945348", 2024-09-16T09:30:28.832 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.84:6813/1803945348", 2024-09-16T09:30:28.832 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2024-09-16T09:30:28.832 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2024-09-16T09:30:28.833 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2024-09-16T09:30:28.833 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:30:28.833 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.833 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:28.833 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 6, 2024-09-16T09:30:28.833 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "7c8f0152-f539-4b7e-8113-91dd7529d3a1", 2024-09-16T09:30:28.833 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2024-09-16T09:30:28.833 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2024-09-16T09:30:28.833 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2024-09-16T09:30:28.833 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2024-09-16T09:30:28.833 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2024-09-16T09:30:28.833 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2024-09-16T09:30:28.833 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 46, 2024-09-16T09:30:28.834 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 166, 2024-09-16T09:30:28.834 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2024-09-16T09:30:28.834 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2024-09-16T09:30:28.834 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2024-09-16T09:30:28.834 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:30:28.834 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:28.834 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:30:28.834 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6816", 2024-09-16T09:30:28.834 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 610727189 2024-09-16T09:30:28.834 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.834 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:28.834 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:30:28.834 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6817", 2024-09-16T09:30:28.834 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 610727189 2024-09-16T09:30:28.835 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:30:28.835 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:30:28.835 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.835 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2024-09-16T09:30:28.835 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:30:28.835 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:28.835 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:30:28.835 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6818", 2024-09-16T09:30:28.835 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 610727189 2024-09-16T09:30:28.835 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.835 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:28.835 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:30:28.835 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6819", 2024-09-16T09:30:28.836 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 610727189 2024-09-16T09:30:28.836 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:30:28.836 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:30:28.836 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.836 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2024-09-16T09:30:28.836 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:30:28.836 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:28.836 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:30:28.836 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6822", 2024-09-16T09:30:28.836 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 610727189 2024-09-16T09:30:28.836 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.836 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:28.836 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:30:28.836 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6823", 2024-09-16T09:30:28.837 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 610727189 2024-09-16T09:30:28.837 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:30:28.837 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:30:28.837 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.837 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2024-09-16T09:30:28.837 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:30:28.837 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:28.837 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:30:28.837 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6820", 2024-09-16T09:30:28.837 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 610727189 2024-09-16T09:30:28.837 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.837 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:28.837 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:30:28.838 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6821", 2024-09-16T09:30:28.838 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 610727189 2024-09-16T09:30:28.838 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:30:28.838 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:30:28.838 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.838 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.84:6817/610727189", 2024-09-16T09:30:28.838 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.84:6819/610727189", 2024-09-16T09:30:28.838 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.84:6823/610727189", 2024-09-16T09:30:28.838 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.84:6821/610727189", 2024-09-16T09:30:28.838 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2024-09-16T09:30:28.838 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2024-09-16T09:30:28.838 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2024-09-16T09:30:28.838 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:30:28.838 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.838 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:28.839 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 7, 2024-09-16T09:30:28.839 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "945efaba-225e-49af-aefd-b4d58ca8093b", 2024-09-16T09:30:28.839 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2024-09-16T09:30:28.839 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2024-09-16T09:30:28.839 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2024-09-16T09:30:28.839 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2024-09-16T09:30:28.839 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2024-09-16T09:30:28.839 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2024-09-16T09:30:28.839 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 53, 2024-09-16T09:30:28.839 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 166, 2024-09-16T09:30:28.839 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2024-09-16T09:30:28.839 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2024-09-16T09:30:28.839 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2024-09-16T09:30:28.840 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:30:28.840 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:28.840 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:30:28.840 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6824", 2024-09-16T09:30:28.840 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1002252531 2024-09-16T09:30:28.840 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.840 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:28.840 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:30:28.840 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6825", 2024-09-16T09:30:28.840 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1002252531 2024-09-16T09:30:28.840 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:30:28.840 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:30:28.840 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.841 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2024-09-16T09:30:28.841 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:30:28.841 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:28.841 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:30:28.841 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6826", 2024-09-16T09:30:28.841 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1002252531 2024-09-16T09:30:28.841 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.841 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:28.841 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:30:28.841 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6827", 2024-09-16T09:30:28.841 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1002252531 2024-09-16T09:30:28.841 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:30:28.841 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:30:28.841 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.842 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2024-09-16T09:30:28.842 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:30:28.842 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:28.842 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:30:28.842 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6830", 2024-09-16T09:30:28.842 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1002252531 2024-09-16T09:30:28.842 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.842 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:28.842 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:30:28.842 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6831", 2024-09-16T09:30:28.842 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1002252531 2024-09-16T09:30:28.842 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:30:28.842 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:30:28.843 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.843 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2024-09-16T09:30:28.843 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:30:28.843 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:28.843 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:30:28.843 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6828", 2024-09-16T09:30:28.843 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1002252531 2024-09-16T09:30:28.843 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.843 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:28.843 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:30:28.843 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6829", 2024-09-16T09:30:28.843 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1002252531 2024-09-16T09:30:28.843 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:30:28.844 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:30:28.844 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.844 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.84:6825/1002252531", 2024-09-16T09:30:28.844 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.84:6827/1002252531", 2024-09-16T09:30:28.844 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.84:6831/1002252531", 2024-09-16T09:30:28.844 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.84:6829/1002252531", 2024-09-16T09:30:28.844 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2024-09-16T09:30:28.844 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2024-09-16T09:30:28.844 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2024-09-16T09:30:28.844 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:30:28.844 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:30:28.845 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-16T09:30:28.845 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_xinfo": [ 2024-09-16T09:30:28.845 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:28.845 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 0, 2024-09-16T09:30:28.845 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2024-09-16T09:30:28.845 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2024-09-16T09:30:28.845 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2024-09-16T09:30:28.845 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540719139924082687, 2024-09-16T09:30:28.845 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2024-09-16T09:30:28.845 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2024-09-16T09:30:27.369260+0000", 2024-09-16T09:30:28.845 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2024-09-16T09:30:28.845 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.845 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:28.846 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 1, 2024-09-16T09:30:28.846 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2024-09-16T09:30:28.846 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2024-09-16T09:30:28.846 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2024-09-16T09:30:28.846 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540719139924082687, 2024-09-16T09:30:28.846 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2024-09-16T09:30:28.846 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2024-09-16T09:29:54.420066+0000", 2024-09-16T09:30:28.846 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2024-09-16T09:30:28.846 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.846 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:28.846 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 2, 2024-09-16T09:30:28.846 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2024-09-16T09:30:28.846 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2024-09-16T09:30:28.847 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2024-09-16T09:30:28.847 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540719139924082687, 2024-09-16T09:30:28.847 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2024-09-16T09:30:28.847 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2024-09-16T09:30:13.478032+0000", 2024-09-16T09:30:28.847 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2024-09-16T09:30:28.847 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.847 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:28.847 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 3, 2024-09-16T09:30:28.847 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2024-09-16T09:30:28.847 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2024-09-16T09:30:28.847 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2024-09-16T09:30:28.847 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540719139924082687, 2024-09-16T09:30:28.848 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2024-09-16T09:30:28.848 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2024-09-16T09:29:35.654697+0000", 2024-09-16T09:30:28.848 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2024-09-16T09:30:28.848 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.848 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:28.848 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 4, 2024-09-16T09:30:28.848 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2024-09-16T09:30:28.848 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2024-09-16T09:30:28.848 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2024-09-16T09:30:28.848 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540719139924082687, 2024-09-16T09:30:28.848 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2024-09-16T09:30:28.848 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2024-09-16T09:29:34.352275+0000", 2024-09-16T09:30:28.849 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2024-09-16T09:30:28.849 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.849 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:28.849 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 5, 2024-09-16T09:30:28.849 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2024-09-16T09:30:28.849 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2024-09-16T09:30:28.849 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2024-09-16T09:30:28.849 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540719139924082687, 2024-09-16T09:30:28.849 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2024-09-16T09:30:28.849 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2024-09-16T09:30:02.228975+0000", 2024-09-16T09:30:28.849 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2024-09-16T09:30:28.849 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.850 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:28.850 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 6, 2024-09-16T09:30:28.850 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2024-09-16T09:30:28.850 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2024-09-16T09:30:28.850 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2024-09-16T09:30:28.850 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540719139924082687, 2024-09-16T09:30:28.850 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2024-09-16T09:30:28.850 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2024-09-16T09:29:22.502600+0000", 2024-09-16T09:30:28.850 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2024-09-16T09:30:28.850 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.850 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:28.850 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 7, 2024-09-16T09:30:28.850 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2024-09-16T09:30:28.851 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2024-09-16T09:30:28.851 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2024-09-16T09:30:28.851 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540719139924082687, 2024-09-16T09:30:28.851 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2024-09-16T09:30:28.851 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2024-09-16T09:30:09.147856+0000", 2024-09-16T09:30:28.851 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2024-09-16T09:30:28.851 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:30:28.851 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-16T09:30:28.851 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap": [ 2024-09-16T09:30:28.851 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:28.851 INFO:teuthology.orchestra.run.smithi070.stdout: "pgid": "3.1d", 2024-09-16T09:30:28.851 INFO:teuthology.orchestra.run.smithi070.stdout: "osds": [ 2024-09-16T09:30:28.852 INFO:teuthology.orchestra.run.smithi070.stdout: 1, 2024-09-16T09:30:28.852 INFO:teuthology.orchestra.run.smithi070.stdout: 0, 2024-09-16T09:30:28.852 INFO:teuthology.orchestra.run.smithi070.stdout: 7 2024-09-16T09:30:28.852 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:30:28.852 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:30:28.852 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-16T09:30:28.852 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap_items": [ 2024-09-16T09:30:28.852 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:28.852 INFO:teuthology.orchestra.run.smithi070.stdout: "pgid": "4.e", 2024-09-16T09:30:28.852 INFO:teuthology.orchestra.run.smithi070.stdout: "mappings": [ 2024-09-16T09:30:28.852 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:28.852 INFO:teuthology.orchestra.run.smithi070.stdout: "from": 1, 2024-09-16T09:30:28.852 INFO:teuthology.orchestra.run.smithi070.stdout: "to": 2 2024-09-16T09:30:28.853 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:30:28.853 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:30:28.853 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.853 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:28.853 INFO:teuthology.orchestra.run.smithi070.stdout: "pgid": "4.f", 2024-09-16T09:30:28.853 INFO:teuthology.orchestra.run.smithi070.stdout: "mappings": [ 2024-09-16T09:30:28.853 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:28.853 INFO:teuthology.orchestra.run.smithi070.stdout: "from": 1, 2024-09-16T09:30:28.853 INFO:teuthology.orchestra.run.smithi070.stdout: "to": 2 2024-09-16T09:30:28.853 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:30:28.853 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:30:28.853 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.853 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:28.853 INFO:teuthology.orchestra.run.smithi070.stdout: "pgid": "4.13", 2024-09-16T09:30:28.854 INFO:teuthology.orchestra.run.smithi070.stdout: "mappings": [ 2024-09-16T09:30:28.854 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:28.854 INFO:teuthology.orchestra.run.smithi070.stdout: "from": 1, 2024-09-16T09:30:28.854 INFO:teuthology.orchestra.run.smithi070.stdout: "to": 5 2024-09-16T09:30:28.854 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:30:28.854 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:30:28.854 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:30:28.854 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-16T09:30:28.854 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap_primaries": [], 2024-09-16T09:30:28.854 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_temp": [ 2024-09-16T09:30:28.854 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:28.854 INFO:teuthology.orchestra.run.smithi070.stdout: "pgid": "8.0", 2024-09-16T09:30:28.854 INFO:teuthology.orchestra.run.smithi070.stdout: "osds": [ 2024-09-16T09:30:28.854 INFO:teuthology.orchestra.run.smithi070.stdout: 5, 2024-09-16T09:30:28.854 INFO:teuthology.orchestra.run.smithi070.stdout: 1 2024-09-16T09:30:28.855 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:30:28.855 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.855 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:28.855 INFO:teuthology.orchestra.run.smithi070.stdout: "pgid": "8.8", 2024-09-16T09:30:28.855 INFO:teuthology.orchestra.run.smithi070.stdout: "osds": [ 2024-09-16T09:30:28.855 INFO:teuthology.orchestra.run.smithi070.stdout: 5, 2024-09-16T09:30:28.855 INFO:teuthology.orchestra.run.smithi070.stdout: 7 2024-09-16T09:30:28.855 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:30:28.855 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.855 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:28.855 INFO:teuthology.orchestra.run.smithi070.stdout: "pgid": "8.b", 2024-09-16T09:30:28.855 INFO:teuthology.orchestra.run.smithi070.stdout: "osds": [ 2024-09-16T09:30:28.855 INFO:teuthology.orchestra.run.smithi070.stdout: 4, 2024-09-16T09:30:28.855 INFO:teuthology.orchestra.run.smithi070.stdout: 1 2024-09-16T09:30:28.855 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:30:28.856 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:30:28.856 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-16T09:30:28.856 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_temp": [], 2024-09-16T09:30:28.856 INFO:teuthology.orchestra.run.smithi070.stdout: "blocklist": { 2024-09-16T09:30:28.856 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/125491368": "2024-09-17T09:25:31.465167+0000", 2024-09-16T09:30:28.856 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/3354284852": "2024-09-17T09:19:43.205989+0000", 2024-09-16T09:30:28.856 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/3284625547": "2024-09-17T09:25:31.465167+0000", 2024-09-16T09:30:28.856 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/186565817": "2024-09-17T09:19:43.205989+0000", 2024-09-16T09:30:28.856 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/2213852865": "2024-09-17T09:19:43.205989+0000", 2024-09-16T09:30:28.856 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/1812373760": "2024-09-17T09:20:09.306330+0000", 2024-09-16T09:30:28.856 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:6800/837338094": "2024-09-17T09:19:43.205989+0000", 2024-09-16T09:30:28.856 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:6800/1876661229": "2024-09-17T09:25:31.465167+0000", 2024-09-16T09:30:28.856 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/3351688485": "2024-09-17T09:20:09.306330+0000", 2024-09-16T09:30:28.856 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:6800/3871553358": "2024-09-17T09:20:09.306330+0000", 2024-09-16T09:30:28.857 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/1230482871": "2024-09-17T09:25:31.465167+0000", 2024-09-16T09:30:28.857 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:6801/837338094": "2024-09-17T09:19:43.205989+0000", 2024-09-16T09:30:28.857 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:6801/3871553358": "2024-09-17T09:20:09.306330+0000", 2024-09-16T09:30:28.857 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/1403001486": "2024-09-17T09:20:09.306330+0000", 2024-09-16T09:30:28.857 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/1723643739": "2024-09-17T09:25:31.465167+0000", 2024-09-16T09:30:28.857 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:6801/1876661229": "2024-09-17T09:25:31.465167+0000", 2024-09-16T09:30:28.857 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/3456417396": "2024-09-17T09:25:31.465167+0000" 2024-09-16T09:30:28.857 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.857 INFO:teuthology.orchestra.run.smithi070.stdout: "range_blocklist": {}, 2024-09-16T09:30:28.857 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profiles": { 2024-09-16T09:30:28.857 INFO:teuthology.orchestra.run.smithi070.stdout: "default": { 2024-09-16T09:30:28.857 INFO:teuthology.orchestra.run.smithi070.stdout: "crush-failure-domain": "osd", 2024-09-16T09:30:28.857 INFO:teuthology.orchestra.run.smithi070.stdout: "k": "2", 2024-09-16T09:30:28.857 INFO:teuthology.orchestra.run.smithi070.stdout: "m": "1", 2024-09-16T09:30:28.858 INFO:teuthology.orchestra.run.smithi070.stdout: "plugin": "jerasure", 2024-09-16T09:30:28.858 INFO:teuthology.orchestra.run.smithi070.stdout: "technique": "reed_sol_van" 2024-09-16T09:30:28.858 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:30:28.858 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:28.858 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps_queue": [], 2024-09-16T09:30:28.858 INFO:teuthology.orchestra.run.smithi070.stdout: "new_removed_snaps": [], 2024-09-16T09:30:28.858 INFO:teuthology.orchestra.run.smithi070.stdout: "new_purged_snaps": [], 2024-09-16T09:30:28.858 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_node_flags": {}, 2024-09-16T09:30:28.858 INFO:teuthology.orchestra.run.smithi070.stdout: "device_class_flags": {}, 2024-09-16T09:30:28.858 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode": { 2024-09-16T09:30:28.858 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode_enabled": false, 2024-09-16T09:30:28.858 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_bucket_count": 0, 2024-09-16T09:30:28.858 INFO:teuthology.orchestra.run.smithi070.stdout: "degraded_stretch_mode": 0, 2024-09-16T09:30:28.859 INFO:teuthology.orchestra.run.smithi070.stdout: "recovering_stretch_mode": 0, 2024-09-16T09:30:28.859 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode_bucket": 0 2024-09-16T09:30:28.859 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:30:28.859 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T09:30:28.894 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:28.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:28.895 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:30:28.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:28 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:30:28.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:28 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:30:28.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:28 smithi084 ceph-mon[32629]: osdmap e171: 8 total, 8 up, 8 in 2024-09-16T09:30:28.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:28 smithi084 ceph-mon[32629]: pgmap v249: 168 pgs: 1 activating+undersized+remapped, 2 activating+undersized+degraded+remapped, 165 active+clean; 3.3 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 0 B/s rd, 69 MiB/s wr, 1.10k op/s; 9922/160518 objects degraded (6.181%); 0 B/s, 11 objects/s recovering 2024-09-16T09:30:28.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:28 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/2394102935' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:30:28.955 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 33 16 60338 60322 114.22 70.875 0.00487445 0.00874931 2024-09-16T09:30:28.997 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:30:29.027 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:29.028 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:29.028 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:30:29.078 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:30:29.078 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:30:29.079 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:30:29.079 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:30:29.079 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:30:29.079 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:30:29.079 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:30:29.079 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:30:29.079 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:30:29.079 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:30:29.080 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:30:29.080 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:30:29.080 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:30:29.080 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:30:29.080 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:30:29.080 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:30:29.129 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:30:29.163 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:29.163 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:29.164 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:30:29.264 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:30:29.286 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:29.286 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:29.287 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:30:29.387 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:30:29.421 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:29.422 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:29.422 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:30:29.449 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd set noscrub 2024-09-16T09:30:29.523 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:30:29.567 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:29.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:29.568 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:30:29.669 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:30:29.706 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:29.707 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:29.707 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:30:29.808 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:30:29.848 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:29.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:29.849 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:30:29.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:29 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1346443271' entity='client.admin' cmd={"prefix": "osd dump", "format": "json-pretty"} : dispatch 2024-09-16T09:30:29.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:29 smithi084 ceph-mon[32629]: osdmap e172: 8 total, 8 up, 8 in 2024-09-16T09:30:29.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:29 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1346443271' entity='client.admin' cmd={"prefix": "osd dump", "format": "json-pretty"} : dispatch 2024-09-16T09:30:29.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:29 smithi070 ceph-mon[30388]: osdmap e172: 8 total, 8 up, 8 in 2024-09-16T09:30:29.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:29 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1346443271' entity='client.admin' cmd={"prefix": "osd dump", "format": "json-pretty"} : dispatch 2024-09-16T09:30:29.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:29 smithi070 ceph-mon[40691]: osdmap e172: 8 total, 8 up, 8 in 2024-09-16T09:30:29.950 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:30:29.955 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 34 16 61632 61616 113.239 80.875 0.00986356 0.00882599 2024-09-16T09:30:29.986 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:29.986 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:29.987 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:30:30.088 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:30:30.117 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:30.118 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:30.118 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:30:30.219 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:30:30.258 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:30.258 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:30.258 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:30:30.359 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:30:30.393 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:30.393 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:30.394 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:30:30.495 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:30:30.525 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:30.526 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:30.526 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:30:30.627 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:30:30.659 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:30.660 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:30.660 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:30:30.761 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:30:30.785 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.7 dump_blocked_ops 2024-09-16T09:30:30.789 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:30.789 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:30.790 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:30:30.862 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:30 smithi070 ceph-mon[30388]: pgmap v251: 168 pgs: 3 active+recovering+undersized+remapped, 165 active+clean; 3.6 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 946 B/s rd, 74 MiB/s wr, 1.19k op/s; 9558/177369 objects misplaced (5.389%); 4.2 MiB/s, 66 objects/s recovering 2024-09-16T09:30:30.863 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:30 smithi070 ceph-mon[40691]: pgmap v251: 168 pgs: 3 active+recovering+undersized+remapped, 165 active+clean; 3.6 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 946 B/s rd, 74 MiB/s wr, 1.19k op/s; 9558/177369 objects misplaced (5.389%); 4.2 MiB/s, 66 objects/s recovering 2024-09-16T09:30:30.891 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:30:30.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:30 smithi084 ceph-mon[32629]: pgmap v251: 168 pgs: 3 active+recovering+undersized+remapped, 165 active+clean; 3.6 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 946 B/s rd, 74 MiB/s wr, 1.19k op/s; 9558/177369 objects misplaced (5.389%); 4.2 MiB/s, 66 objects/s recovering 2024-09-16T09:30:30.929 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:30.929 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:30.930 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:30:30.955 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 35 16 62853 62837 112.184 76.3125 0.0294831 0.00890808 2024-09-16T09:30:31.030 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:30:31.072 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:31.073 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:31.073 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:30:31.174 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:30:31.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:31.206 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:31.206 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:30:31.220 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:30:31.307 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:30:31.341 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:31.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:31.342 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:30:31.378 INFO:tasks.thrashosds.thrasher:j is {'epoch': 171, 'fsid': 'a1321834-740c-11ef-bceb-c7b262605968', 'created': '2024-09-16T09:19:14.121498+0000', 'modified': '2024-09-16T09:30:27.554985+0000', 'last_up_change': '2024-09-16T09:24:51.124496+0000', 'last_in_change': '2024-09-16T09:24:28.432920+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 18, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 8, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T09:22:27.617694+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 21, 'pg_placement_num': 21, 'pg_placement_num_target': 21, 'pg_num_target': 21, 'pg_num_pending': 21, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '162', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '160', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.2899999618530273, 'score_stable': 2.2899999618530273, 'optimal_score': 1, 'raw_score_acting': 2.2899999618530273, 'raw_score_stable': 2.2899999618530273, 'primary_affinity_weighted': 0.9345664978027344, 'average_primary_affinity': 0.9300000071525574, 'average_primary_affinity_weighted': 0.9300000071525574}}, {'pool': 2, 'pool_name': 'datapool', 'create_time': '2024-09-16T09:24:59.169244+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 3, 'pg_placement_num': 3, 'pg_placement_num_target': 3, 'pg_num_target': 3, 'pg_num_pending': 3, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 61, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.6700000762939453, 'score_stable': 2.6700000762939453, 'optimal_score': 0.75, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.9345664978027344, 'average_primary_affinity': 0.9300000071525574, 'average_primary_affinity_weighted': 0.9300000071525574}}, {'pool': 3, 'pool_name': '.rgw.root', 'create_time': '2024-09-16T09:25:00.056119+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.9345664978027344, 'average_primary_affinity': 0.9300000071525574, 'average_primary_affinity_weighted': 0.9300000071525574}}, {'pool': 4, 'pool_name': 'default.rgw.log', 'create_time': '2024-09-16T09:25:01.770016+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '62', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9345664978027344, 'average_primary_affinity': 0.9300000071525574, 'average_primary_affinity_weighted': 0.9300000071525574}}, {'pool': 5, 'pool_name': 'default.rgw.control', 'create_time': '2024-09-16T09:25:03.725555+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9345664978027344, 'average_primary_affinity': 0.9300000071525574, 'average_primary_affinity_weighted': 0.9300000071525574}}, {'pool': 6, 'pool_name': 'default.rgw.meta', 'create_time': '2024-09-16T09:25:05.773530+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 0.9345664978027344, 'average_primary_affinity': 0.9300000071525574, 'average_primary_affinity_weighted': 0.9300000071525574}}, {'pool': 8, 'pool_name': 'unique_pool_1', 'create_time': '2024-09-16T09:29:48.079163+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 16, 'pg_placement_num': 16, 'pg_placement_num_target': 16, 'pg_num_target': 16, 'pg_num_pending': 16, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '155', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rados': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.9345664978027344, 'average_primary_affinity': 0.9300000071525574, 'average_primary_affinity_weighted': 0.9300000071525574}}], 'osds': [{'osd': 0, 'uuid': '7ef288c8-3bf0-4814-8e86-f70a7de041ce', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 8, 'up_thru': 166, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6802', 'nonce': 2013075781}, {'type': 'v1', 'addr': '172.21.15.70:6803', 'nonce': 2013075781}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6804', 'nonce': 2013075781}, {'type': 'v1', 'addr': '172.21.15.70:6805', 'nonce': 2013075781}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6808', 'nonce': 2013075781}, {'type': 'v1', 'addr': '172.21.15.70:6809', 'nonce': 2013075781}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6806', 'nonce': 2013075781}, {'type': 'v1', 'addr': '172.21.15.70:6807', 'nonce': 2013075781}]}, 'public_addr': '172.21.15.70:6803/2013075781', 'cluster_addr': '172.21.15.70:6805/2013075781', 'heartbeat_back_addr': '172.21.15.70:6809/2013075781', 'heartbeat_front_addr': '172.21.15.70:6807/2013075781', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '474d41d9-295a-450c-8f27-ce487a14678b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 166, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6810', 'nonce': 1670012115}, {'type': 'v1', 'addr': '172.21.15.70:6811', 'nonce': 1670012115}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6812', 'nonce': 1670012115}, {'type': 'v1', 'addr': '172.21.15.70:6813', 'nonce': 1670012115}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6816', 'nonce': 1670012115}, {'type': 'v1', 'addr': '172.21.15.70:6817', 'nonce': 1670012115}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6814', 'nonce': 1670012115}, {'type': 'v1', 'addr': '172.21.15.70:6815', 'nonce': 1670012115}]}, 'public_addr': '172.21.15.70:6811/1670012115', 'cluster_addr': '172.21.15.70:6813/1670012115', 'heartbeat_back_addr': '172.21.15.70:6817/1670012115', 'heartbeat_front_addr': '172.21.15.70:6815/1670012115', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '5b4ca4e8-cecc-4163-b291-c9209b02c4b7', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 18, 'up_thru': 166, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6818', 'nonce': 3601682336}, {'type': 'v1', 'addr': '172.21.15.70:6819', 'nonce': 3601682336}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6820', 'nonce': 3601682336}, {'type': 'v1', 'addr': '172.21.15.70:6821', 'nonce': 3601682336}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6824', 'nonce': 3601682336}, {'type': 'v1', 'addr': '172.21.15.70:6825', 'nonce': 3601682336}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6822', 'nonce': 3601682336}, {'type': 'v1', 'addr': '172.21.15.70:6823', 'nonce': 3601682336}]}, 'public_addr': '172.21.15.70:6819/3601682336', 'cluster_addr': '172.21.15.70:6821/3601682336', 'heartbeat_back_addr': '172.21.15.70:6825/3601682336', 'heartbeat_front_addr': '172.21.15.70:6823/3601682336', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'ffb8d527-1b58-4ecd-808c-47d4ba1bf387', 'up': 1, 'in': 1, 'weight': 0.3394775390625, 'primary_affinity': 0.476531982421875, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 26, 'up_thru': 162, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6826', 'nonce': 2038083007}, {'type': 'v1', 'addr': '172.21.15.70:6827', 'nonce': 2038083007}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6828', 'nonce': 2038083007}, {'type': 'v1', 'addr': '172.21.15.70:6829', 'nonce': 2038083007}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6832', 'nonce': 2038083007}, {'type': 'v1', 'addr': '172.21.15.70:6833', 'nonce': 2038083007}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6830', 'nonce': 2038083007}, {'type': 'v1', 'addr': '172.21.15.70:6831', 'nonce': 2038083007}]}, 'public_addr': '172.21.15.70:6827/2038083007', 'cluster_addr': '172.21.15.70:6829/2038083007', 'heartbeat_back_addr': '172.21.15.70:6833/2038083007', 'heartbeat_front_addr': '172.21.15.70:6831/2038083007', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'a946760b-4d91-4a4b-8e9b-a07a50589211', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 32, 'up_thru': 167, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6800', 'nonce': 2250892710}, {'type': 'v1', 'addr': '172.21.15.84:6801', 'nonce': 2250892710}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6802', 'nonce': 2250892710}, {'type': 'v1', 'addr': '172.21.15.84:6803', 'nonce': 2250892710}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6806', 'nonce': 2250892710}, {'type': 'v1', 'addr': '172.21.15.84:6807', 'nonce': 2250892710}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6804', 'nonce': 2250892710}, {'type': 'v1', 'addr': '172.21.15.84:6805', 'nonce': 2250892710}]}, 'public_addr': '172.21.15.84:6801/2250892710', 'cluster_addr': '172.21.15.84:6803/2250892710', 'heartbeat_back_addr': '172.21.15.84:6807/2250892710', 'heartbeat_front_addr': '172.21.15.84:6805/2250892710', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '67681760-75a9-4846-946e-a935d42c5e05', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 39, 'up_thru': 167, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6808', 'nonce': 1803945348}, {'type': 'v1', 'addr': '172.21.15.84:6809', 'nonce': 1803945348}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6810', 'nonce': 1803945348}, {'type': 'v1', 'addr': '172.21.15.84:6811', 'nonce': 1803945348}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6814', 'nonce': 1803945348}, {'type': 'v1', 'addr': '172.21.15.84:6815', 'nonce': 1803945348}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6812', 'nonce': 1803945348}, {'type': 'v1', 'addr': '172.21.15.84:6813', 'nonce': 1803945348}]}, 'public_addr': '172.21.15.84:6809/1803945348', 'cluster_addr': '172.21.15.84:6811/1803945348', 'heartbeat_back_addr': '172.21.15.84:6815/1803945348', 'heartbeat_front_addr': '172.21.15.84:6813/1803945348', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '7c8f0152-f539-4b7e-8113-91dd7529d3a1', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 46, 'up_thru': 166, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6816', 'nonce': 610727189}, {'type': 'v1', 'addr': '172.21.15.84:6817', 'nonce': 610727189}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6818', 'nonce': 610727189}, {'type': 'v1', 'addr': '172.21.15.84:6819', 'nonce': 610727189}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6822', 'nonce': 610727189}, {'type': 'v1', 'addr': '172.21.15.84:6823', 'nonce': 610727189}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6820', 'nonce': 610727189}, {'type': 'v1', 'addr': '172.21.15.84:6821', 'nonce': 610727189}]}, 'public_addr': '172.21.15.84:6817/610727189', 'cluster_addr': '172.21.15.84:6819/610727189', 'heartbeat_back_addr': '172.21.15.84:6823/610727189', 'heartbeat_front_addr': '172.21.15.84:6821/610727189', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '945efaba-225e-49af-aefd-b4d58ca8093b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 53, 'up_thru': 166, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6824', 'nonce': 1002252531}, {'type': 'v1', 'addr': '172.21.15.84:6825', 'nonce': 1002252531}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6826', 'nonce': 1002252531}, {'type': 'v1', 'addr': '172.21.15.84:6827', 'nonce': 1002252531}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6830', 'nonce': 1002252531}, {'type': 'v1', 'addr': '172.21.15.84:6831', 'nonce': 1002252531}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6828', 'nonce': 1002252531}, {'type': 'v1', 'addr': '172.21.15.84:6829', 'nonce': 1002252531}]}, 'public_addr': '172.21.15.84:6825/1002252531', 'cluster_addr': '172.21.15.84:6827/1002252531', 'heartbeat_back_addr': '172.21.15.84:6831/1002252531', 'heartbeat_front_addr': '172.21.15.84:6829/1002252531', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T09:30:27.369260+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T09:29:54.420066+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T09:30:13.478032+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T09:29:35.654697+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T09:29:34.352275+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T09:30:02.228975+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T09:29:22.502600+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T09:30:09.147856+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.1d', 'osds': [1, 0, 7]}], 'pg_upmap_items': [{'pgid': '4.e', 'mappings': [{'from': 1, 'to': 2}]}, {'pgid': '4.f', 'mappings': [{'from': 1, 'to': 2}]}, {'pgid': '4.13', 'mappings': [{'from': 1, 'to': 5}]}], 'pg_upmap_primaries': [], 'pg_temp': [{'pgid': '8.0', 'osds': [5, 1]}, {'pgid': '8.8', 'osds': [5, 7]}, {'pgid': '8.b', 'osds': [4, 1]}], 'primary_temp': [], 'blocklist': {'172.21.15.70:0/125491368': '2024-09-17T09:25:31.465167+0000', '172.21.15.70:0/3354284852': '2024-09-17T09:19:43.205989+0000', '172.21.15.70:0/3284625547': '2024-09-17T09:25:31.465167+0000', '172.21.15.70:0/186565817': '2024-09-17T09:19:43.205989+0000', '172.21.15.70:0/2213852865': '2024-09-17T09:19:43.205989+0000', '172.21.15.70:0/1812373760': '2024-09-17T09:20:09.306330+0000', '172.21.15.70:6800/837338094': '2024-09-17T09:19:43.205989+0000', '172.21.15.70:6800/1876661229': '2024-09-17T09:25:31.465167+0000', '172.21.15.70:0/3351688485': '2024-09-17T09:20:09.306330+0000', '172.21.15.70:6800/3871553358': '2024-09-17T09:20:09.306330+0000', '172.21.15.70:0/1230482871': '2024-09-17T09:25:31.465167+0000', '172.21.15.70:6801/837338094': '2024-09-17T09:19:43.205989+0000', '172.21.15.70:6801/3871553358': '2024-09-17T09:20:09.306330+0000', '172.21.15.70:0/1403001486': '2024-09-17T09:20:09.306330+0000', '172.21.15.70:0/1723643739': '2024-09-17T09:25:31.465167+0000', '172.21.15.70:6801/1876661229': '2024-09-17T09:25:31.465167+0000', '172.21.15.70:0/3456417396': '2024-09-17T09:25:31.465167+0000'}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-16T09:30:31.378 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-16T09:30:31.443 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:30:31.481 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:31.481 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:31.481 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:30:31.528 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:30:31.582 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:30:31.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:31 smithi070 ceph-mon[30388]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 9922/160518 objects degraded (6.181%), 2 pgs degraded) 2024-09-16T09:30:31.609 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:31 smithi084 ceph-mon[32629]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 9922/160518 objects degraded (6.181%), 2 pgs degraded) 2024-09-16T09:30:31.609 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:31 smithi084 ceph-mon[32629]: Cluster is now healthy 2024-09-16T09:30:31.618 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:31.619 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:31.619 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:30:31.720 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:30:31.753 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:31.753 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:31.754 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:30:31.855 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:30:31.886 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:31 smithi070 ceph-mon[40691]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 9922/160518 objects degraded (6.181%), 2 pgs degraded) 2024-09-16T09:30:31.886 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:31 smithi070 ceph-mon[40691]: Cluster is now healthy 2024-09-16T09:30:31.886 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:31.887 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:31 smithi070 ceph-mon[30388]: Cluster is now healthy 2024-09-16T09:30:31.887 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:31.887 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:30:31.955 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 36 16 64025 64009 111.102 73.25 0.0070991 0.00899389 2024-09-16T09:30:31.988 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:30:32.021 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:32.022 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:32.022 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:30:32.123 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:30:32.153 INFO:teuthology.orchestra.run.smithi084.stdout:ERROR: (22) Invalid argument 2024-09-16T09:30:32.153 INFO:teuthology.orchestra.run.smithi084.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:30:32.159 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:32.160 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:32.160 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:30:32.170 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:30:31 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:30:31] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:30:32.261 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:30:32.294 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:32.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:32.295 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:30:32.368 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:30:32.396 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:30:32.424 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:32.425 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:32.425 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:30:32.526 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:30:32.569 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:32.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:32.570 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:30:32.671 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:30:32.707 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:32.708 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:32.708 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:30:32.809 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:30:32.818 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:32 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:30:32.819 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:32 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:30:32.819 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:32 smithi084 ceph-mon[32629]: 5.1f scrub starts 2024-09-16T09:30:32.819 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:32 smithi084 ceph-mon[32629]: 5.1f scrub ok 2024-09-16T09:30:32.819 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:32 smithi084 ceph-mon[32629]: pgmap v252: 168 pgs: 3 active+recovering+undersized+remapped, 165 active+clean; 3.6 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 853 B/s rd, 67 MiB/s wr, 1.07k op/s; 9558/177369 objects misplaced (5.389%); 3.8 MiB/s, 60 objects/s recovering 2024-09-16T09:30:32.819 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:32 smithi084 ceph-mon[32629]: osdmap e173: 8 total, 8 up, 8 in 2024-09-16T09:30:32.819 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:32 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:30:32.819 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:32 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:30:32.844 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:32.844 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:32.844 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:30:32.852 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:32 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:30:32.852 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:32 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:30:32.852 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:32 smithi070 ceph-mon[30388]: 5.1f scrub starts 2024-09-16T09:30:32.852 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:32 smithi070 ceph-mon[30388]: 5.1f scrub ok 2024-09-16T09:30:32.852 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:32 smithi070 ceph-mon[30388]: pgmap v252: 168 pgs: 3 active+recovering+undersized+remapped, 165 active+clean; 3.6 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 853 B/s rd, 67 MiB/s wr, 1.07k op/s; 9558/177369 objects misplaced (5.389%); 3.8 MiB/s, 60 objects/s recovering 2024-09-16T09:30:32.852 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:32 smithi070 ceph-mon[30388]: osdmap e173: 8 total, 8 up, 8 in 2024-09-16T09:30:32.852 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:32 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:30:32.852 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:32 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:30:32.853 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:32 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:30:32.853 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:32 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:30:32.853 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:32 smithi070 ceph-mon[40691]: 5.1f scrub starts 2024-09-16T09:30:32.853 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:32 smithi070 ceph-mon[40691]: 5.1f scrub ok 2024-09-16T09:30:32.853 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:32 smithi070 ceph-mon[40691]: pgmap v252: 168 pgs: 3 active+recovering+undersized+remapped, 165 active+clean; 3.6 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 853 B/s rd, 67 MiB/s wr, 1.07k op/s; 9558/177369 objects misplaced (5.389%); 3.8 MiB/s, 60 objects/s recovering 2024-09-16T09:30:32.853 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:32 smithi070 ceph-mon[40691]: osdmap e173: 8 total, 8 up, 8 in 2024-09-16T09:30:32.853 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:32 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:30:32.853 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:32 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:30:32.865 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T09:30:32.945 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:30:32.955 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 37 16 65225 65209 110.127 75 0.00602894 0.00907379 2024-09-16T09:30:32.982 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:32.983 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:32.984 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:30:33.084 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:30:33.123 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:33.123 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:33.123 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:30:33.160 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:30:32 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:30:33.224 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:30:33.254 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:33.255 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:33.255 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:30:33.297 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:30:33.340 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:30:33.343 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2024-09-16T09:30:33.356 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:30:33.402 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:33.403 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:33.404 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:30:33.505 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:30:33.539 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:33.539 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:33.539 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:30:33.616 INFO:teuthology.orchestra.run.smithi070.stderr:noscrub is set 2024-09-16T09:30:33.618 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:33 smithi070 ceph-mon[30388]: 3.1e scrub starts 2024-09-16T09:30:33.618 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:33 smithi070 ceph-mon[30388]: 3.1e scrub ok 2024-09-16T09:30:33.618 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:33 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/4266489376' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:30:33.618 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:33 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:30:33.618 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:33 smithi070 ceph-mon[30388]: from='client.25154 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:30:33.640 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:30:33.676 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:33.676 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:33.676 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:30:33.759 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.7 dump_historic_ops 2024-09-16T09:30:33.777 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:30:33.813 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:33.814 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:33.814 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:30:33.868 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:33 smithi070 ceph-mon[40691]: 3.1e scrub starts 2024-09-16T09:30:33.868 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:33 smithi070 ceph-mon[40691]: 3.1e scrub ok 2024-09-16T09:30:33.868 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:33 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/4266489376' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:30:33.869 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:33 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:30:33.869 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:33 smithi070 ceph-mon[40691]: from='client.25154 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:30:33.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:33 smithi084 ceph-mon[32629]: 3.1e scrub starts 2024-09-16T09:30:33.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:33 smithi084 ceph-mon[32629]: 3.1e scrub ok 2024-09-16T09:30:33.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:33 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/4266489376' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:30:33.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:33 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:30:33.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:33 smithi084 ceph-mon[32629]: from='client.25154 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:30:33.915 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:30:33.953 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:33.954 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:33.954 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:30:33.955 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 38 16 66282 66266 108.967 66.0625 0.0519641 0.00916966 2024-09-16T09:30:34.055 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:30:34.107 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:34.108 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:34.108 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:30:34.194 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:30:34.209 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:30:34.239 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:34.240 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:34.240 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:30:34.341 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:30:34.385 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:34.386 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:34.387 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:30:34.487 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:30:34.521 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:34.522 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:34.522 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:30:34.623 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:30:34.661 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:34.661 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:34.662 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:30:34.762 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:30:34.794 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:34.795 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:34.795 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:30:34.896 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:30:34.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:34 smithi084 ceph-mon[32629]: 6.c scrub starts 2024-09-16T09:30:34.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:34 smithi084 ceph-mon[32629]: 6.c scrub ok 2024-09-16T09:30:34.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:34 smithi084 ceph-mon[32629]: pgmap v254: 168 pgs: 3 active+recovering+undersized+remapped, 165 active+clean; 4.0 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 851 B/s rd, 129 MiB/s wr, 2.07k op/s; 9166/195441 objects misplaced (4.690%); 7.8 MiB/s, 125 objects/s recovering 2024-09-16T09:30:34.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:34 smithi084 ceph-mon[32629]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:30:34.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:34 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/4266489376' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-16T09:30:34.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:34 smithi084 ceph-mon[32629]: osdmap e174: 8 total, 8 up, 8 in 2024-09-16T09:30:34.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:34 smithi070 ceph-mon[30388]: 6.c scrub starts 2024-09-16T09:30:34.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:34 smithi070 ceph-mon[30388]: 6.c scrub ok 2024-09-16T09:30:34.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:34 smithi070 ceph-mon[30388]: pgmap v254: 168 pgs: 3 active+recovering+undersized+remapped, 165 active+clean; 4.0 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 851 B/s rd, 129 MiB/s wr, 2.07k op/s; 9166/195441 objects misplaced (4.690%); 7.8 MiB/s, 125 objects/s recovering 2024-09-16T09:30:34.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:34 smithi070 ceph-mon[30388]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:30:34.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:34 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/4266489376' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-16T09:30:34.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:34 smithi070 ceph-mon[30388]: osdmap e174: 8 total, 8 up, 8 in 2024-09-16T09:30:34.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:34 smithi070 ceph-mon[40691]: 6.c scrub starts 2024-09-16T09:30:34.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:34 smithi070 ceph-mon[40691]: 6.c scrub ok 2024-09-16T09:30:34.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:34 smithi070 ceph-mon[40691]: pgmap v254: 168 pgs: 3 active+recovering+undersized+remapped, 165 active+clean; 4.0 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 851 B/s rd, 129 MiB/s wr, 2.07k op/s; 9166/195441 objects misplaced (4.690%); 7.8 MiB/s, 125 objects/s recovering 2024-09-16T09:30:34.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:34 smithi070 ceph-mon[40691]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:30:34.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:34 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/4266489376' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-16T09:30:34.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:34 smithi070 ceph-mon[40691]: osdmap e174: 8 total, 8 up, 8 in 2024-09-16T09:30:34.930 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:34.930 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:34.931 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:30:34.956 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 39 16 67507 67491 108.136 76.5625 0.00630906 0.00923576 2024-09-16T09:30:35.032 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:30:35.072 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:35.072 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:35.073 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:30:35.173 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:30:35.206 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:35.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:35.207 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:30:35.308 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:30:35.354 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:35.354 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:35.354 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:30:35.455 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:30:35.485 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:35.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:35.486 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:30:35.587 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:30:35.622 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:35.622 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:35.623 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:30:35.724 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:30:35.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:35.750 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:35.750 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:30:35.851 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:30:35.888 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":253,"stamp":"2024-09-16T09:30:31.605924+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3862390802,"num_objects":59123,"num_object_clones":0,"num_object_copies":177369,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":9558,"num_objects_unfound":0,"num_objects_dirty":59123,"num_whiteouts":0,"num_read":1056,"num_read_kb":827,"num_write":59380,"num_write_kb":3771413,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":441,"num_bytes_recovered":24096731,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":61060,"ondisk_log_size":61060,"up":504,"acting":501,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":507,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":12539368,"kb_used_data":11195912,"kb_used_omap":12,"kb_used_meta":1343283,"kb_avail":737421848,"statfs":{"total":767960285184,"available":755119972352,"internally_reserved":0,"allocated":11464613888,"data_stored":11461686924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":1375522354},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[7,2,7,4,1,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":420085760,"num_objects":6410,"num_object_clones":0,"num_object_copies":19230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-9922,"num_objects_misplaced":9558,"num_objects_unfound":0,"num_objects_dirty":6410,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":6410,"num_write_kb":410240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":362,"num_bytes_recovered":23724032,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.058924"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":262,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.581919+0000","last_change":"2024-09-16T09:30:03.848439+0000","last_active":"2024-09-16T09:30:27.581919+0000","last_peered":"2024-09-16T09:30:27.581919+0000","last_clean":"2024-09-16T09:30:27.581919+0000","last_became_active":"2024-09-16T09:25:01.709108+0000","last_became_peered":"2024-09-16T09:25:01.709108+0000","last_unstale":"2024-09-16T09:30:27.581919+0000","last_undegraded":"2024-09-16T09:30:27.581919+0000","last_fullsized":"2024-09-16T09:30:27.581919+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:30:03.848354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:30:03.848354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:31:26.175389+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"66'9","reported_seq":266,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.576935+0000","last_change":"2024-09-16T09:28:52.689373+0000","last_active":"2024-09-16T09:30:27.576935+0000","last_peered":"2024-09-16T09:30:27.576935+0000","last_clean":"2024-09-16T09:30:27.576935+0000","last_became_active":"2024-09-16T09:25:03.714635+0000","last_became_peered":"2024-09-16T09:25:03.714635+0000","last_unstale":"2024-09-16T09:30:27.576935+0000","last_undegraded":"2024-09-16T09:30:27.576935+0000","last_fullsized":"2024-09-16T09:30:27.576935+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:28:52.689304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:52.689304+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":254,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.580301+0000","last_change":"2024-09-16T09:29:17.910367+0000","last_active":"2024-09-16T09:30:27.580301+0000","last_peered":"2024-09-16T09:30:27.580301+0000","last_clean":"2024-09-16T09:30:27.580301+0000","last_became_active":"2024-09-16T09:25:05.715645+0000","last_became_peered":"2024-09-16T09:25:05.715645+0000","last_unstale":"2024-09-16T09:30:27.580301+0000","last_undegraded":"2024-09-16T09:30:27.580301+0000","last_fullsized":"2024-09-16T09:30:27.580301+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:17.910305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:17.910305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:31.298101+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":234,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.577335+0000","last_change":"2024-09-16T09:27:11.565306+0000","last_active":"2024-09-16T09:30:27.577335+0000","last_peered":"2024-09-16T09:30:27.577335+0000","last_clean":"2024-09-16T09:30:27.577335+0000","last_became_active":"2024-09-16T09:25:07.715442+0000","last_became_peered":"2024-09-16T09:25:07.715442+0000","last_unstale":"2024-09-16T09:30:27.577335+0000","last_undegraded":"2024-09-16T09:30:27.577335+0000","last_fullsized":"2024-09-16T09:30:27.577335+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:11.565220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:11.565220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:41.628205+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":20,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-16T09:30:26.549917+0000","last_change":"2024-09-16T09:30:21.541340+0000","last_active":"2024-09-16T09:30:26.549917+0000","last_peered":"2024-09-16T09:30:26.549917+0000","last_clean":"2024-09-16T09:30:26.549917+0000","last_became_active":"2024-09-16T09:30:21.541064+0000","last_became_peered":"2024-09-16T09:30:21.541064+0000","last_unstale":"2024-09-16T09:30:26.549917+0000","last_undegraded":"2024-09-16T09:30:26.549917+0000","last_fullsized":"2024-09-16T09:30:26.549917+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:00.068570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:28:00.068570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":94,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-16T09:30:26.565262+0000","last_change":"2024-09-16T09:30:21.528197+0000","last_active":"2024-09-16T09:30:26.565262+0000","last_peered":"2024-09-16T09:30:26.565262+0000","last_clean":"2024-09-16T09:30:26.565262+0000","last_became_active":"2024-09-16T09:30:21.527743+0000","last_became_peered":"2024-09-16T09:30:21.527743+0000","last_unstale":"2024-09-16T09:30:26.565262+0000","last_undegraded":"2024-09-16T09:30:26.565262+0000","last_fullsized":"2024-09-16T09:30:26.565262+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:16.575227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:16.575227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,1],"acting":[6,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.19","version":"66'15","reported_seq":157,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.587868+0000","last_change":"2024-09-16T09:30:21.553791+0000","last_active":"2024-09-16T09:30:27.587868+0000","last_peered":"2024-09-16T09:30:27.587868+0000","last_clean":"2024-09-16T09:30:27.587868+0000","last_became_active":"2024-09-16T09:30:21.535180+0000","last_became_peered":"2024-09-16T09:30:21.535180+0000","last_unstale":"2024-09-16T09:30:27.587868+0000","last_undegraded":"2024-09-16T09:30:27.587868+0000","last_fullsized":"2024-09-16T09:30:27.587868+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-16T09:29:57.968162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:29:57.968162+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:31:13.183599+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":254,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.576848+0000","last_change":"2024-09-16T09:29:15.628676+0000","last_active":"2024-09-16T09:30:27.576848+0000","last_peered":"2024-09-16T09:30:27.576848+0000","last_clean":"2024-09-16T09:30:27.576848+0000","last_became_active":"2024-09-16T09:25:05.717236+0000","last_became_peered":"2024-09-16T09:25:05.717236+0000","last_unstale":"2024-09-16T09:30:27.576848+0000","last_undegraded":"2024-09-16T09:30:27.576848+0000","last_fullsized":"2024-09-16T09:30:27.576848+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:15.628467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:15.628467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:44.502659+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":92,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.577542+0000","last_change":"2024-09-16T09:28:34.839909+0000","last_active":"2024-09-16T09:30:27.577542+0000","last_peered":"2024-09-16T09:30:27.577542+0000","last_clean":"2024-09-16T09:30:27.577542+0000","last_became_active":"2024-09-16T09:28:34.839435+0000","last_became_peered":"2024-09-16T09:28:34.839435+0000","last_unstale":"2024-09-16T09:30:27.577542+0000","last_undegraded":"2024-09-16T09:30:27.577542+0000","last_fullsized":"2024-09-16T09:30:27.577542+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:01.038400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:28:01.038400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:36.847392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"66'9","reported_seq":266,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.577485+0000","last_change":"2024-09-16T09:29:28.622647+0000","last_active":"2024-09-16T09:30:27.577485+0000","last_peered":"2024-09-16T09:30:27.577485+0000","last_clean":"2024-09-16T09:30:27.577485+0000","last_became_active":"2024-09-16T09:25:03.709147+0000","last_became_peered":"2024-09-16T09:25:03.709147+0000","last_unstale":"2024-09-16T09:30:27.577485+0000","last_undegraded":"2024-09-16T09:30:27.577485+0000","last_fullsized":"2024-09-16T09:30:27.577485+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:29:28.622558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:29:28.622558+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:48.836980+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":237,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.581228+0000","last_change":"2024-09-16T09:27:59.080369+0000","last_active":"2024-09-16T09:30:27.581228+0000","last_peered":"2024-09-16T09:30:27.581228+0000","last_clean":"2024-09-16T09:30:27.581228+0000","last_became_active":"2024-09-16T09:25:05.718134+0000","last_became_peered":"2024-09-16T09:25:05.718134+0000","last_unstale":"2024-09-16T09:30:27.581228+0000","last_undegraded":"2024-09-16T09:30:27.581228+0000","last_fullsized":"2024-09-16T09:30:27.581228+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:59.080307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:59.080307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:47.486214+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"66'1","reported_seq":243,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.581578+0000","last_change":"2024-09-16T09:29:15.787990+0000","last_active":"2024-09-16T09:30:27.581578+0000","last_peered":"2024-09-16T09:30:27.581578+0000","last_clean":"2024-09-16T09:30:27.581578+0000","last_became_active":"2024-09-16T09:25:07.716558+0000","last_became_peered":"2024-09-16T09:25:07.716558+0000","last_unstale":"2024-09-16T09:30:27.581578+0000","last_undegraded":"2024-09-16T09:30:27.581578+0000","last_fullsized":"2024-09-16T09:30:27.581578+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-16T09:29:15.787922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:15.787922+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:27.924198+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":245,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.583037+0000","last_change":"2024-09-16T09:27:56.191264+0000","last_active":"2024-09-16T09:30:27.583037+0000","last_peered":"2024-09-16T09:30:27.583037+0000","last_clean":"2024-09-16T09:30:27.583037+0000","last_became_active":"2024-09-16T09:25:01.696939+0000","last_became_peered":"2024-09-16T09:25:01.696939+0000","last_unstale":"2024-09-16T09:30:27.583037+0000","last_undegraded":"2024-09-16T09:30:27.583037+0000","last_fullsized":"2024-09-16T09:30:27.583037+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:56.191220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:27:56.191220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:45.493367+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"66'5","reported_seq":252,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.579741+0000","last_change":"2024-09-16T09:27:05.694733+0000","last_active":"2024-09-16T09:30:27.579741+0000","last_peered":"2024-09-16T09:30:27.579741+0000","last_clean":"2024-09-16T09:30:27.579741+0000","last_became_active":"2024-09-16T09:25:03.716136+0000","last_became_peered":"2024-09-16T09:25:03.716136+0000","last_unstale":"2024-09-16T09:30:27.579741+0000","last_undegraded":"2024-09-16T09:30:27.579741+0000","last_fullsized":"2024-09-16T09:30:27.579741+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-16T09:27:05.694650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:05.694650+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:47.504795+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":243,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-16T09:30:26.550679+0000","last_change":"2024-09-16T09:28:52.746046+0000","last_active":"2024-09-16T09:30:26.550679+0000","last_peered":"2024-09-16T09:30:26.550679+0000","last_clean":"2024-09-16T09:30:26.550679+0000","last_became_active":"2024-09-16T09:25:05.718281+0000","last_became_peered":"2024-09-16T09:25:05.718281+0000","last_unstale":"2024-09-16T09:30:26.550679+0000","last_undegraded":"2024-09-16T09:30:26.550679+0000","last_fullsized":"2024-09-16T09:30:26.550679+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:52.745999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:28:52.745999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":256,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.581284+0000","last_change":"2024-09-16T09:30:21.527156+0000","last_active":"2024-09-16T09:30:27.581284+0000","last_peered":"2024-09-16T09:30:27.581284+0000","last_clean":"2024-09-16T09:30:27.581284+0000","last_became_active":"2024-09-16T09:30:21.526737+0000","last_became_peered":"2024-09-16T09:30:21.526737+0000","last_unstale":"2024-09-16T09:30:27.581284+0000","last_undegraded":"2024-09-16T09:30:27.581284+0000","last_fullsized":"2024-09-16T09:30:27.581284+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:50.468134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:50.468134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:31:16.020015+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,2],"acting":[5,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":250,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.579117+0000","last_change":"2024-09-16T09:29:56.605555+0000","last_active":"2024-09-16T09:30:27.579117+0000","last_peered":"2024-09-16T09:30:27.579117+0000","last_clean":"2024-09-16T09:30:27.579117+0000","last_became_active":"2024-09-16T09:25:07.720793+0000","last_became_peered":"2024-09-16T09:25:07.720793+0000","last_unstale":"2024-09-16T09:30:27.579117+0000","last_undegraded":"2024-09-16T09:30:27.579117+0000","last_fullsized":"2024-09-16T09:30:27.579117+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:56.605488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:56.605488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:31:10.696779+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":254,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.582108+0000","last_change":"2024-09-16T09:29:50.913029+0000","last_active":"2024-09-16T09:30:27.582108+0000","last_peered":"2024-09-16T09:30:27.582108+0000","last_clean":"2024-09-16T09:30:27.582108+0000","last_became_active":"2024-09-16T09:25:01.706172+0000","last_became_peered":"2024-09-16T09:25:01.706172+0000","last_unstale":"2024-09-16T09:30:27.582108+0000","last_undegraded":"2024-09-16T09:30:27.582108+0000","last_fullsized":"2024-09-16T09:30:27.582108+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:50.912979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:50.912979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:55.204409+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"66'15","reported_seq":306,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.585969+0000","last_change":"2024-09-16T09:30:21.932668+0000","last_active":"2024-09-16T09:30:27.585969+0000","last_peered":"2024-09-16T09:30:27.585969+0000","last_clean":"2024-09-16T09:30:27.585969+0000","last_became_active":"2024-09-16T09:30:21.524507+0000","last_became_peered":"2024-09-16T09:30:21.524507+0000","last_unstale":"2024-09-16T09:30:27.585969+0000","last_undegraded":"2024-09-16T09:30:27.585969+0000","last_fullsized":"2024-09-16T09:30:27.585969+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-16T09:30:21.932604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:30:21.932604+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:31:50.563357+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":246,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.580600+0000","last_change":"2024-09-16T09:29:55.875536+0000","last_active":"2024-09-16T09:30:27.580600+0000","last_peered":"2024-09-16T09:30:27.580600+0000","last_clean":"2024-09-16T09:30:27.580600+0000","last_became_active":"2024-09-16T09:25:05.713929+0000","last_became_peered":"2024-09-16T09:25:05.713929+0000","last_unstale":"2024-09-16T09:30:27.580600+0000","last_undegraded":"2024-09-16T09:30:27.580600+0000","last_fullsized":"2024-09-16T09:30:27.580600+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:55.875480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:55.875480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:31:15.525615+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":96,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-16T09:30:26.565742+0000","last_change":"2024-09-16T09:30:24.412733+0000","last_active":"2024-09-16T09:30:26.565742+0000","last_peered":"2024-09-16T09:30:26.565742+0000","last_clean":"2024-09-16T09:30:26.565742+0000","last_became_active":"2024-09-16T09:28:52.974255+0000","last_became_peered":"2024-09-16T09:28:52.974255+0000","last_unstale":"2024-09-16T09:30:26.565742+0000","last_undegraded":"2024-09-16T09:30:26.565742+0000","last_fullsized":"2024-09-16T09:30:26.565742+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:30:24.412571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:30:24.412571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:31:36.120832+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":262,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.577744+0000","last_change":"2024-09-16T09:30:03.628882+0000","last_active":"2024-09-16T09:30:27.577744+0000","last_peered":"2024-09-16T09:30:27.577744+0000","last_clean":"2024-09-16T09:30:27.577744+0000","last_became_active":"2024-09-16T09:25:01.706491+0000","last_became_peered":"2024-09-16T09:25:01.706491+0000","last_unstale":"2024-09-16T09:30:27.577744+0000","last_undegraded":"2024-09-16T09:30:27.577744+0000","last_fullsized":"2024-09-16T09:30:27.577744+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:30:03.628812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:30:03.628812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:31:12.621371+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"66'12","reported_seq":143,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.580810+0000","last_change":"2024-09-16T09:30:21.553703+0000","last_active":"2024-09-16T09:30:27.580810+0000","last_peered":"2024-09-16T09:30:27.580810+0000","last_clean":"2024-09-16T09:30:27.580810+0000","last_became_active":"2024-09-16T09:30:21.521062+0000","last_became_peered":"2024-09-16T09:30:21.521062+0000","last_unstale":"2024-09-16T09:30:27.580810+0000","last_undegraded":"2024-09-16T09:30:27.580810+0000","last_fullsized":"2024-09-16T09:30:27.580810+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-16T09:27:01.700139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:01.700139+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":24,"num_read_kb":15,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,6],"acting":[1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":238,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.577980+0000","last_change":"2024-09-16T09:27:00.816500+0000","last_active":"2024-09-16T09:30:27.577980+0000","last_peered":"2024-09-16T09:30:27.577980+0000","last_clean":"2024-09-16T09:30:27.577980+0000","last_became_active":"2024-09-16T09:25:05.710093+0000","last_became_peered":"2024-09-16T09:25:05.710093+0000","last_unstale":"2024-09-16T09:30:27.577980+0000","last_undegraded":"2024-09-16T09:30:27.577980+0000","last_fullsized":"2024-09-16T09:30:27.577980+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:00.816424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:00.816424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:42.602227+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":240,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-16T09:30:26.550621+0000","last_change":"2024-09-16T09:29:29.918433+0000","last_active":"2024-09-16T09:30:26.550621+0000","last_peered":"2024-09-16T09:30:26.550621+0000","last_clean":"2024-09-16T09:30:26.550621+0000","last_became_active":"2024-09-16T09:25:07.719380+0000","last_became_peered":"2024-09-16T09:25:07.719380+0000","last_unstale":"2024-09-16T09:30:26.550621+0000","last_undegraded":"2024-09-16T09:30:26.550621+0000","last_fullsized":"2024-09-16T09:30:26.550621+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-16T09:29:29.918368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:29.918368+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:59.870894+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":263,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.580432+0000","last_change":"2024-09-16T09:30:02.843932+0000","last_active":"2024-09-16T09:30:27.580432+0000","last_peered":"2024-09-16T09:30:27.580432+0000","last_clean":"2024-09-16T09:30:27.580432+0000","last_became_active":"2024-09-16T09:25:01.699545+0000","last_became_peered":"2024-09-16T09:25:01.699545+0000","last_unstale":"2024-09-16T09:30:27.580432+0000","last_undegraded":"2024-09-16T09:30:27.580432+0000","last_fullsized":"2024-09-16T09:30:27.580432+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:30:02.843856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:30:02.843856+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:31:09.364650+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"66'10","reported_seq":284,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.582159+0000","last_change":"2024-09-16T09:30:21.559975+0000","last_active":"2024-09-16T09:30:27.582159+0000","last_peered":"2024-09-16T09:30:27.582159+0000","last_clean":"2024-09-16T09:30:27.582159+0000","last_became_active":"2024-09-16T09:30:21.526982+0000","last_became_peered":"2024-09-16T09:30:21.526982+0000","last_unstale":"2024-09-16T09:30:27.582159+0000","last_undegraded":"2024-09-16T09:30:27.582159+0000","last_fullsized":"2024-09-16T09:30:27.582159+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-16T09:27:43.556357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:43.556357+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"66'8","reported_seq":329,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-16T09:30:26.565867+0000","last_change":"2024-09-16T09:28:58.597645+0000","last_active":"2024-09-16T09:30:26.565867+0000","last_peered":"2024-09-16T09:30:26.565867+0000","last_clean":"2024-09-16T09:30:26.565867+0000","last_became_active":"2024-09-16T09:25:05.712516+0000","last_became_peered":"2024-09-16T09:25:05.712516+0000","last_unstale":"2024-09-16T09:30:26.565867+0000","last_undegraded":"2024-09-16T09:30:26.565867+0000","last_fullsized":"2024-09-16T09:30:26.565867+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:28:58.597586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:28:58.597586+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"171'3769","reported_seq":3818,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.764385+0000","last_change":"2024-09-16T09:29:49.449011+0000","last_active":"2024-09-16T09:30:27.764385+0000","last_peered":"2024-09-16T09:30:27.764385+0000","last_clean":"2024-09-16T09:30:27.764385+0000","last_became_active":"2024-09-16T09:29:49.448473+0000","last_became_peered":"2024-09-16T09:29:49.448473+0000","last_unstale":"2024-09-16T09:30:27.764385+0000","last_undegraded":"2024-09-16T09:30:27.764385+0000","last_fullsized":"2024-09-16T09:30:27.764385+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":3769,"log_dups_size":0,"ondisk_log_size":3769,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:31:17.161691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":247005184,"num_objects":3769,"num_object_clones":0,"num_object_copies":11307,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3769,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3769,"num_write_kb":241216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.a","version":"73'79","reported_seq":184,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.580978+0000","last_change":"2024-09-16T09:29:18.498887+0000","last_active":"2024-09-16T09:30:27.580978+0000","last_peered":"2024-09-16T09:30:27.580978+0000","last_clean":"2024-09-16T09:30:27.580978+0000","last_became_active":"2024-09-16T09:26:35.127352+0000","last_became_peered":"2024-09-16T09:26:35.127352+0000","last_unstale":"2024-09-16T09:30:27.580978+0000","last_undegraded":"2024-09-16T09:30:27.580978+0000","last_fullsized":"2024-09-16T09:30:27.580978+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:18.498766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:18.498766+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"66'15","reported_seq":193,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.585604+0000","last_change":"2024-09-16T09:28:28.065781+0000","last_active":"2024-09-16T09:30:27.585604+0000","last_peered":"2024-09-16T09:30:27.585604+0000","last_clean":"2024-09-16T09:30:27.585604+0000","last_became_active":"2024-09-16T09:26:33.295672+0000","last_became_peered":"2024-09-16T09:26:33.295672+0000","last_unstale":"2024-09-16T09:30:27.585604+0000","last_undegraded":"2024-09-16T09:30:27.585604+0000","last_fullsized":"2024-09-16T09:30:27.585604+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-16T09:28:28.065699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:28.065699+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:33.850277+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.577349+0000","last_change":"2024-09-16T09:30:21.540666+0000","last_active":"2024-09-16T09:30:27.577349+0000","last_peered":"2024-09-16T09:30:27.577349+0000","last_clean":"2024-09-16T09:30:27.577349+0000","last_became_active":"2024-09-16T09:30:21.540065+0000","last_became_peered":"2024-09-16T09:30:21.540065+0000","last_unstale":"2024-09-16T09:30:27.577349+0000","last_undegraded":"2024-09-16T09:30:27.577349+0000","last_fullsized":"2024-09-16T09:30:27.577349+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:22.091537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:28:22.091537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,4],"acting":[1,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"66'8","reported_seq":304,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.579188+0000","last_change":"2024-09-16T09:25:05.714128+0000","last_active":"2024-09-16T09:30:27.579188+0000","last_peered":"2024-09-16T09:30:27.579188+0000","last_clean":"2024-09-16T09:30:27.579188+0000","last_became_active":"2024-09-16T09:25:05.713813+0000","last_became_peered":"2024-09-16T09:25:05.713813+0000","last_unstale":"2024-09-16T09:30:27.579188+0000","last_undegraded":"2024-09-16T09:30:27.579188+0000","last_fullsized":"2024-09-16T09:30:27.579188+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:31:17.740413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":241,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.581424+0000","last_change":"2024-09-16T09:29:24.537802+0000","last_active":"2024-09-16T09:30:27.581424+0000","last_peered":"2024-09-16T09:30:27.581424+0000","last_clean":"2024-09-16T09:30:27.581424+0000","last_became_active":"2024-09-16T09:25:07.717344+0000","last_became_peered":"2024-09-16T09:25:07.717344+0000","last_unstale":"2024-09-16T09:30:27.581424+0000","last_undegraded":"2024-09-16T09:30:27.581424+0000","last_fullsized":"2024-09-16T09:30:27.581424+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:24.537731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:24.537731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:33.718527+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.c","version":"171'3711","reported_seq":3760,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.991297+0000","last_change":"2024-09-16T09:29:50.008199+0000","last_active":"2024-09-16T09:30:27.991297+0000","last_peered":"2024-09-16T09:30:27.991297+0000","last_clean":"2024-09-16T09:30:27.991297+0000","last_became_active":"2024-09-16T09:29:50.007935+0000","last_became_peered":"2024-09-16T09:29:50.007935+0000","last_unstale":"2024-09-16T09:30:27.991297+0000","last_undegraded":"2024-09-16T09:30:27.991297+0000","last_fullsized":"2024-09-16T09:30:27.991297+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":3711,"log_dups_size":0,"ondisk_log_size":3711,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:31:07.013285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":243204096,"num_objects":3711,"num_object_clones":0,"num_object_copies":11133,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3711,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3711,"num_write_kb":237504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"73'79","reported_seq":204,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.578675+0000","last_change":"2024-09-16T09:30:10.687467+0000","last_active":"2024-09-16T09:30:27.578675+0000","last_peered":"2024-09-16T09:30:27.578675+0000","last_clean":"2024-09-16T09:30:27.578675+0000","last_became_active":"2024-09-16T09:30:08.763683+0000","last_became_peered":"2024-09-16T09:30:08.763683+0000","last_unstale":"2024-09-16T09:30:27.578675+0000","last_undegraded":"2024-09-16T09:30:27.578675+0000","last_fullsized":"2024-09-16T09:30:27.578675+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:58.568841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:58.568841+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:30:40.661693+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"66'18","reported_seq":164,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-16T09:30:26.550041+0000","last_change":"2024-09-16T09:30:21.563937+0000","last_active":"2024-09-16T09:30:26.550041+0000","last_peered":"2024-09-16T09:30:26.550041+0000","last_clean":"2024-09-16T09:30:26.550041+0000","last_became_active":"2024-09-16T09:30:21.545778+0000","last_became_peered":"2024-09-16T09:30:21.545778+0000","last_unstale":"2024-09-16T09:30:26.550041+0000","last_undegraded":"2024-09-16T09:30:26.550041+0000","last_fullsized":"2024-09-16T09:30:26.550041+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'18","last_scrub_stamp":"2024-09-16T09:28:54.699778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:54.699778+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":33,"num_read_kb":21,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":87,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-16T09:30:26.550353+0000","last_change":"2024-09-16T09:28:58.705169+0000","last_active":"2024-09-16T09:30:26.550353+0000","last_peered":"2024-09-16T09:30:26.550353+0000","last_clean":"2024-09-16T09:30:26.550353+0000","last_became_active":"2024-09-16T09:28:52.983543+0000","last_became_peered":"2024-09-16T09:28:52.983543+0000","last_unstale":"2024-09-16T09:30:26.550353+0000","last_undegraded":"2024-09-16T09:30:26.550353+0000","last_fullsized":"2024-09-16T09:30:26.550353+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:58.705097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:28:58.705097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":261,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.576477+0000","last_change":"2024-09-16T09:30:21.525252+0000","last_active":"2024-09-16T09:30:27.576477+0000","last_peered":"2024-09-16T09:30:27.576477+0000","last_clean":"2024-09-16T09:30:27.576477+0000","last_became_active":"2024-09-16T09:30:21.524834+0000","last_became_peered":"2024-09-16T09:30:21.524834+0000","last_unstale":"2024-09-16T09:30:27.576477+0000","last_undegraded":"2024-09-16T09:30:27.576477+0000","last_fullsized":"2024-09-16T09:30:27.576477+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:50.482240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:50.482240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:31:06.041061+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":241,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.577866+0000","last_change":"2024-09-16T09:30:21.517801+0000","last_active":"2024-09-16T09:30:27.577866+0000","last_peered":"2024-09-16T09:30:27.577866+0000","last_clean":"2024-09-16T09:30:27.577866+0000","last_became_active":"2024-09-16T09:30:21.517321+0000","last_became_peered":"2024-09-16T09:30:21.517321+0000","last_unstale":"2024-09-16T09:30:27.577866+0000","last_undegraded":"2024-09-16T09:30:27.577866+0000","last_fullsized":"2024-09-16T09:30:27.577866+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:13.548608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:13.548608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.d","version":"171'3681","reported_seq":3730,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.607033+0000","last_change":"2024-09-16T09:29:50.010860+0000","last_active":"2024-09-16T09:30:27.607033+0000","last_peered":"2024-09-16T09:30:27.607033+0000","last_clean":"2024-09-16T09:30:27.607033+0000","last_became_active":"2024-09-16T09:29:50.010421+0000","last_became_peered":"2024-09-16T09:29:50.010421+0000","last_unstale":"2024-09-16T09:30:27.607033+0000","last_undegraded":"2024-09-16T09:30:27.607033+0000","last_fullsized":"2024-09-16T09:30:27.607033+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":3681,"log_dups_size":0,"ondisk_log_size":3681,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:31:13.299355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":241238016,"num_objects":3681,"num_object_clones":0,"num_object_copies":11043,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3681,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3681,"num_write_kb":235584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,2],"acting":[5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.4","version":"73'79","reported_seq":30,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.582948+0000","last_change":"2024-09-16T09:30:22.438448+0000","last_active":"2024-09-16T09:30:27.582948+0000","last_peered":"2024-09-16T09:30:27.582948+0000","last_clean":"2024-09-16T09:30:27.582948+0000","last_became_active":"2024-09-16T09:30:21.521291+0000","last_became_peered":"2024-09-16T09:30:21.521291+0000","last_unstale":"2024-09-16T09:30:27.582948+0000","last_undegraded":"2024-09-16T09:30:27.582948+0000","last_fullsized":"2024-09-16T09:30:27.582948+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:30:22.438378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:30:22.438378+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:31:49.074818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1","version":"66'14","reported_seq":271,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.576564+0000","last_change":"2024-09-16T09:28:32.595064+0000","last_active":"2024-09-16T09:30:27.576564+0000","last_peered":"2024-09-16T09:30:27.576564+0000","last_clean":"2024-09-16T09:30:27.576564+0000","last_became_active":"2024-09-16T09:25:03.708830+0000","last_became_peered":"2024-09-16T09:25:03.708830+0000","last_unstale":"2024-09-16T09:30:27.576564+0000","last_undegraded":"2024-09-16T09:30:27.576564+0000","last_fullsized":"2024-09-16T09:30:27.576564+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'14","last_scrub_stamp":"2024-09-16T09:28:32.594988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:32.594988+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:31:07.612225+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":263,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.582433+0000","last_change":"2024-09-16T09:28:58.602786+0000","last_active":"2024-09-16T09:30:27.582433+0000","last_peered":"2024-09-16T09:30:27.582433+0000","last_clean":"2024-09-16T09:30:27.582433+0000","last_became_active":"2024-09-16T09:25:01.702992+0000","last_became_peered":"2024-09-16T09:25:01.702992+0000","last_unstale":"2024-09-16T09:30:27.582433+0000","last_undegraded":"2024-09-16T09:30:27.582433+0000","last_fullsized":"2024-09-16T09:30:27.582433+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:28:58.602717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:28:58.602717+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"167'10","reported_seq":145,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.576971+0000","last_change":"2024-09-16T09:30:21.556484+0000","last_active":"2024-09-16T09:30:27.576971+0000","last_peered":"2024-09-16T09:30:27.576971+0000","last_clean":"2024-09-16T09:30:27.576971+0000","last_became_active":"2024-09-16T09:30:21.537847+0000","last_became_peered":"2024-09-16T09:30:21.537847+0000","last_unstale":"2024-09-16T09:30:27.576971+0000","last_undegraded":"2024-09-16T09:30:27.576971+0000","last_fullsized":"2024-09-16T09:30:27.576971+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'9","last_scrub_stamp":"2024-09-16T09:29:21.898533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:21.898533+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:43.257921+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":231,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-16T09:30:26.549677+0000","last_change":"2024-09-16T09:26:40.699888+0000","last_active":"2024-09-16T09:30:26.549677+0000","last_peered":"2024-09-16T09:30:26.549677+0000","last_clean":"2024-09-16T09:30:26.549677+0000","last_became_active":"2024-09-16T09:25:07.722017+0000","last_became_peered":"2024-09-16T09:25:07.722017+0000","last_unstale":"2024-09-16T09:30:26.549677+0000","last_undegraded":"2024-09-16T09:30:26.549677+0000","last_fullsized":"2024-09-16T09:30:26.549677+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:40.699813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:40.699813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:31:19.140134+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.e","version":"171'3748","reported_seq":3797,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:28.016252+0000","last_change":"2024-09-16T09:29:49.448081+0000","last_active":"2024-09-16T09:30:28.016252+0000","last_peered":"2024-09-16T09:30:28.016252+0000","last_clean":"2024-09-16T09:30:28.016252+0000","last_became_active":"2024-09-16T09:29:49.447767+0000","last_became_peered":"2024-09-16T09:29:49.447767+0000","last_unstale":"2024-09-16T09:30:28.016252+0000","last_undegraded":"2024-09-16T09:30:28.016252+0000","last_fullsized":"2024-09-16T09:30:28.016252+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":3748,"log_dups_size":0,"ondisk_log_size":3748,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:31:11.883242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":245628928,"num_objects":3748,"num_object_clones":0,"num_object_copies":11244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3748,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3748,"num_write_kb":239872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"73'79","reported_seq":293,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-16T09:30:26.565646+0000","last_change":"2024-09-16T09:30:23.455463+0000","last_active":"2024-09-16T09:30:26.565646+0000","last_peered":"2024-09-16T09:30:26.565646+0000","last_clean":"2024-09-16T09:30:26.565646+0000","last_became_active":"2024-09-16T09:30:21.526086+0000","last_became_peered":"2024-09-16T09:30:21.526086+0000","last_unstale":"2024-09-16T09:30:26.565646+0000","last_undegraded":"2024-09-16T09:30:26.565646+0000","last_fullsized":"2024-09-16T09:30:26.565646+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:30:23.455388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:30:23.455388+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:31:41.702024+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,5],"acting":[6,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"66'10","reported_seq":273,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.582445+0000","last_change":"2024-09-16T09:30:04.848306+0000","last_active":"2024-09-16T09:30:27.582445+0000","last_peered":"2024-09-16T09:30:27.582445+0000","last_clean":"2024-09-16T09:30:27.582445+0000","last_became_active":"2024-09-16T09:25:03.712985+0000","last_became_peered":"2024-09-16T09:25:03.712985+0000","last_unstale":"2024-09-16T09:30:27.582445+0000","last_undegraded":"2024-09-16T09:30:27.582445+0000","last_fullsized":"2024-09-16T09:30:27.582445+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-16T09:30:04.848234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:30:04.848234+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:31:22.324284+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":252,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.582543+0000","last_change":"2024-09-16T09:30:21.523461+0000","last_active":"2024-09-16T09:30:27.582543+0000","last_peered":"2024-09-16T09:30:27.582543+0000","last_clean":"2024-09-16T09:30:27.582543+0000","last_became_active":"2024-09-16T09:30:21.523024+0000","last_became_peered":"2024-09-16T09:30:21.523024+0000","last_unstale":"2024-09-16T09:30:27.582543+0000","last_undegraded":"2024-09-16T09:30:27.582543+0000","last_fullsized":"2024-09-16T09:30:27.582543+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:25.086322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:25.086322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"66'8","reported_seq":320,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.581644+0000","last_change":"2024-09-16T09:28:57.628615+0000","last_active":"2024-09-16T09:30:27.581644+0000","last_peered":"2024-09-16T09:30:27.581644+0000","last_clean":"2024-09-16T09:30:27.581644+0000","last_became_active":"2024-09-16T09:25:05.714507+0000","last_became_peered":"2024-09-16T09:25:05.714507+0000","last_unstale":"2024-09-16T09:30:27.581644+0000","last_undegraded":"2024-09-16T09:30:27.581644+0000","last_fullsized":"2024-09-16T09:30:27.581644+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:28:57.628543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:28:57.628543+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":249,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.581728+0000","last_change":"2024-09-16T09:30:21.517513+0000","last_active":"2024-09-16T09:30:27.581728+0000","last_peered":"2024-09-16T09:30:27.581728+0000","last_clean":"2024-09-16T09:30:27.581728+0000","last_became_active":"2024-09-16T09:30:21.517167+0000","last_became_peered":"2024-09-16T09:30:21.517167+0000","last_unstale":"2024-09-16T09:30:27.581728+0000","last_undegraded":"2024-09-16T09:30:27.581728+0000","last_fullsized":"2024-09-16T09:30:27.581728+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:22.563395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:28:22.563395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,5],"acting":[0,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.f","version":"171'3644","reported_seq":3693,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.743678+0000","last_change":"2024-09-16T09:29:49.838498+0000","last_active":"2024-09-16T09:30:27.743678+0000","last_peered":"2024-09-16T09:30:27.743678+0000","last_clean":"2024-09-16T09:30:27.743678+0000","last_became_active":"2024-09-16T09:29:49.838146+0000","last_became_peered":"2024-09-16T09:29:49.838146+0000","last_unstale":"2024-09-16T09:30:27.743678+0000","last_undegraded":"2024-09-16T09:30:27.743678+0000","last_fullsized":"2024-09-16T09:30:27.743678+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":3644,"log_dups_size":0,"ondisk_log_size":3644,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:31:09.838041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":238813184,"num_objects":3644,"num_object_clones":0,"num_object_copies":10932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3644,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3644,"num_write_kb":233216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"73'79","reported_seq":274,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-16T09:30:26.565337+0000","last_change":"2024-09-16T09:30:22.415099+0000","last_active":"2024-09-16T09:30:26.565337+0000","last_peered":"2024-09-16T09:30:26.565337+0000","last_clean":"2024-09-16T09:30:26.565337+0000","last_became_active":"2024-09-16T09:30:21.521457+0000","last_became_peered":"2024-09-16T09:30:21.521457+0000","last_unstale":"2024-09-16T09:30:26.565337+0000","last_undegraded":"2024-09-16T09:30:26.565337+0000","last_fullsized":"2024-09-16T09:30:26.565337+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:30:22.415051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:30:22.415051+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:31:24.612301+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,4],"acting":[6,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"66'19","reported_seq":278,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.581867+0000","last_change":"2024-09-16T09:28:47.551606+0000","last_active":"2024-09-16T09:30:27.581867+0000","last_peered":"2024-09-16T09:30:27.581867+0000","last_clean":"2024-09-16T09:30:27.581867+0000","last_became_active":"2024-09-16T09:25:03.714797+0000","last_became_peered":"2024-09-16T09:25:03.714797+0000","last_unstale":"2024-09-16T09:30:27.581867+0000","last_undegraded":"2024-09-16T09:30:27.581867+0000","last_fullsized":"2024-09-16T09:30:27.581867+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-16T09:28:47.551527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:47.551527+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":262,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.580960+0000","last_change":"2024-09-16T09:29:16.888357+0000","last_active":"2024-09-16T09:30:27.580960+0000","last_peered":"2024-09-16T09:30:27.580960+0000","last_clean":"2024-09-16T09:30:27.580960+0000","last_became_active":"2024-09-16T09:25:01.706806+0000","last_became_peered":"2024-09-16T09:25:01.706806+0000","last_unstale":"2024-09-16T09:30:27.580960+0000","last_undegraded":"2024-09-16T09:30:27.580960+0000","last_fullsized":"2024-09-16T09:30:27.580960+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:16.888236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:16.888236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:30.989136+0000","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":244,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-16T09:30:26.563787+0000","last_change":"2024-09-16T09:28:24.737761+0000","last_active":"2024-09-16T09:30:26.563787+0000","last_peered":"2024-09-16T09:30:26.563787+0000","last_clean":"2024-09-16T09:30:26.563787+0000","last_became_active":"2024-09-16T09:25:05.708663+0000","last_became_peered":"2024-09-16T09:25:05.708663+0000","last_unstale":"2024-09-16T09:30:26.563787+0000","last_undegraded":"2024-09-16T09:30:26.563787+0000","last_fullsized":"2024-09-16T09:30:26.563787+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:24.737702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:28:24.737702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:48.283381+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":234,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.580227+0000","last_change":"2024-09-16T09:26:37.282798+0000","last_active":"2024-09-16T09:30:27.580227+0000","last_peered":"2024-09-16T09:30:27.580227+0000","last_clean":"2024-09-16T09:30:27.580227+0000","last_became_active":"2024-09-16T09:25:07.719267+0000","last_became_peered":"2024-09-16T09:25:07.719267+0000","last_unstale":"2024-09-16T09:30:27.580227+0000","last_undegraded":"2024-09-16T09:30:27.580227+0000","last_fullsized":"2024-09-16T09:30:27.580227+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:37.282735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:37.282735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:31:17.659346+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.8","version":"171'3769","reported_seq":3994,"reported_epoch":171,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-16T09:30:27.636199+0000","last_change":"2024-09-16T09:30:22.707970+0000","last_active":"2024-09-16T09:30:27.636199+0000","last_peered":"2024-09-16T09:30:27.636199+0000","last_clean":"2024-09-16T09:30:20.474651+0000","last_became_active":"2024-09-16T09:30:22.688829+0000","last_became_peered":"2024-09-16T09:30:22.688829+0000","last_unstale":"2024-09-16T09:30:27.636199+0000","last_undegraded":"2024-09-16T09:30:27.636199+0000","last_fullsized":"2024-09-16T09:30:22.504174+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":3769,"log_dups_size":0,"ondisk_log_size":3769,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":247005184,"num_objects":3769,"num_object_clones":0,"num_object_copies":11307,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3256,"num_objects_unfound":0,"num_objects_dirty":3769,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3769,"num_write_kb":241216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":5636096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,0],"acting":[5,7],"avail_no_missing":["5","3","7"],"object_location_counts":[{"shards":"3,5,7","objects":3256},{"shards":"5,7","objects":513}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1","version":"73'79","reported_seq":196,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.581050+0000","last_change":"2024-09-16T09:30:10.688265+0000","last_active":"2024-09-16T09:30:27.581050+0000","last_peered":"2024-09-16T09:30:27.581050+0000","last_clean":"2024-09-16T09:30:27.581050+0000","last_became_active":"2024-09-16T09:30:08.766350+0000","last_became_peered":"2024-09-16T09:30:08.766350+0000","last_unstale":"2024-09-16T09:30:27.581050+0000","last_undegraded":"2024-09-16T09:30:27.581050+0000","last_fullsized":"2024-09-16T09:30:27.581050+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:15.460179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:15.460179+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:30:40.423853+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"167'29","reported_seq":393,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.577379+0000","last_change":"2024-09-16T09:30:21.559310+0000","last_active":"2024-09-16T09:30:27.577379+0000","last_peered":"2024-09-16T09:30:27.577379+0000","last_clean":"2024-09-16T09:30:27.577379+0000","last_became_active":"2024-09-16T09:30:21.537755+0000","last_became_peered":"2024-09-16T09:30:21.537755+0000","last_unstale":"2024-09-16T09:30:27.577379+0000","last_undegraded":"2024-09-16T09:30:27.577379+0000","last_fullsized":"2024-09-16T09:30:27.577379+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'28","last_scrub_stamp":"2024-09-16T09:29:18.937623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:29:18.937623+0000","objects_scrubbed":10,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:31.146984+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":254,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.589506+0000","last_change":"2024-09-16T09:29:26.637070+0000","last_active":"2024-09-16T09:30:27.589506+0000","last_peered":"2024-09-16T09:30:27.589506+0000","last_clean":"2024-09-16T09:30:27.589506+0000","last_became_active":"2024-09-16T09:25:01.703076+0000","last_became_peered":"2024-09-16T09:25:01.703076+0000","last_unstale":"2024-09-16T09:30:27.589506+0000","last_undegraded":"2024-09-16T09:30:27.589506+0000","last_fullsized":"2024-09-16T09:30:27.589506+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:26.637027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:26.637027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:51.048084+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":263,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.581580+0000","last_change":"2024-09-16T09:29:29.486284+0000","last_active":"2024-09-16T09:30:27.581580+0000","last_peered":"2024-09-16T09:30:27.581580+0000","last_clean":"2024-09-16T09:30:27.581580+0000","last_became_active":"2024-09-16T09:25:00.696237+0000","last_became_peered":"2024-09-16T09:25:00.696237+0000","last_unstale":"2024-09-16T09:30:27.581580+0000","last_undegraded":"2024-09-16T09:30:27.581580+0000","last_fullsized":"2024-09-16T09:30:27.581580+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-16T09:29:29.486188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:29:29.486188+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:47.593613+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.0025137430000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":230,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.582207+0000","last_change":"2024-09-16T09:25:05.718319+0000","last_active":"2024-09-16T09:30:27.582207+0000","last_peered":"2024-09-16T09:30:27.582207+0000","last_clean":"2024-09-16T09:30:27.582207+0000","last_became_active":"2024-09-16T09:25:05.717591+0000","last_became_peered":"2024-09-16T09:25:05.717591+0000","last_unstale":"2024-09-16T09:30:27.582207+0000","last_undegraded":"2024-09-16T09:30:27.582207+0000","last_fullsized":"2024-09-16T09:30:27.582207+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:31:09.804564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":22,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.579357+0000","last_change":"2024-09-16T09:30:21.520726+0000","last_active":"2024-09-16T09:30:27.579357+0000","last_peered":"2024-09-16T09:30:27.579357+0000","last_clean":"2024-09-16T09:30:27.579357+0000","last_became_active":"2024-09-16T09:30:21.520285+0000","last_became_peered":"2024-09-16T09:30:21.520285+0000","last_unstale":"2024-09-16T09:30:27.579357+0000","last_undegraded":"2024-09-16T09:30:27.579357+0000","last_fullsized":"2024-09-16T09:30:27.579357+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:45.201503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:45.201503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.b","version":"170'3684","reported_seq":4093,"reported_epoch":171,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-16T09:30:27.586771+0000","last_change":"2024-09-16T09:30:22.697123+0000","last_active":"2024-09-16T09:30:27.586771+0000","last_peered":"2024-09-16T09:30:27.586771+0000","last_clean":"2024-09-16T09:30:20.492754+0000","last_became_active":"2024-09-16T09:30:22.680179+0000","last_became_peered":"2024-09-16T09:30:22.680179+0000","last_unstale":"2024-09-16T09:30:27.586771+0000","last_undegraded":"2024-09-16T09:30:27.586771+0000","last_fullsized":"2024-09-16T09:30:22.505234+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":3684,"log_dups_size":0,"ondisk_log_size":3684,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":241434624,"num_objects":3684,"num_object_clones":0,"num_object_copies":11052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3105,"num_objects_unfound":0,"num_objects_dirty":3684,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3684,"num_write_kb":235776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":178,"num_bytes_recovered":11665408,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1],"avail_no_missing":["4","1","3"],"object_location_counts":[{"shards":"1,3,4","objects":3105},{"shards":"1,4","objects":579}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.2","version":"73'79","reported_seq":319,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-16T09:30:26.550513+0000","last_change":"2024-09-16T09:30:10.689124+0000","last_active":"2024-09-16T09:30:26.550513+0000","last_peered":"2024-09-16T09:30:26.550513+0000","last_clean":"2024-09-16T09:30:26.550513+0000","last_became_active":"2024-09-16T09:30:08.765943+0000","last_became_peered":"2024-09-16T09:30:08.765943+0000","last_unstale":"2024-09-16T09:30:26.550513+0000","last_undegraded":"2024-09-16T09:30:26.550513+0000","last_fullsized":"2024-09-16T09:30:26.550513+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:26:35.745837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:26:35.745837+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:30:40.156395+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"66'13","reported_seq":277,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.583149+0000","last_change":"2024-09-16T09:27:44.156057+0000","last_active":"2024-09-16T09:30:27.583149+0000","last_peered":"2024-09-16T09:30:27.583149+0000","last_clean":"2024-09-16T09:30:27.583149+0000","last_became_active":"2024-09-16T09:25:03.713241+0000","last_became_peered":"2024-09-16T09:25:03.713241+0000","last_unstale":"2024-09-16T09:30:27.583149+0000","last_undegraded":"2024-09-16T09:30:27.583149+0000","last_fullsized":"2024-09-16T09:30:27.583149+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'13","last_scrub_stamp":"2024-09-16T09:27:44.155969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:44.155969+0000","objects_scrubbed":5,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:31:20.696787+0000","scrub_duration":29,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":254,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.577584+0000","last_change":"2024-09-16T09:29:50.827103+0000","last_active":"2024-09-16T09:30:27.577584+0000","last_peered":"2024-09-16T09:30:27.577584+0000","last_clean":"2024-09-16T09:30:27.577584+0000","last_became_active":"2024-09-16T09:25:01.706828+0000","last_became_peered":"2024-09-16T09:25:01.706828+0000","last_unstale":"2024-09-16T09:30:27.577584+0000","last_undegraded":"2024-09-16T09:30:27.577584+0000","last_fullsized":"2024-09-16T09:30:27.577584+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:50.827057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:50.827057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:31:04.210633+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":274,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.587954+0000","last_change":"2024-09-16T09:30:21.550904+0000","last_active":"2024-09-16T09:30:27.587954+0000","last_peered":"2024-09-16T09:30:27.587954+0000","last_clean":"2024-09-16T09:30:27.587954+0000","last_became_active":"2024-09-16T09:30:21.528216+0000","last_became_peered":"2024-09-16T09:30:21.528216+0000","last_unstale":"2024-09-16T09:30:27.587954+0000","last_undegraded":"2024-09-16T09:30:27.587954+0000","last_fullsized":"2024-09-16T09:30:27.587954+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:29:16.104209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:29:16.104209+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:34.444682+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0.00055022299999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":254,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.586366+0000","last_change":"2024-09-16T09:29:00.169141+0000","last_active":"2024-09-16T09:30:27.586366+0000","last_peered":"2024-09-16T09:30:27.586366+0000","last_clean":"2024-09-16T09:30:27.586366+0000","last_became_active":"2024-09-16T09:25:05.711539+0000","last_became_peered":"2024-09-16T09:25:05.711539+0000","last_unstale":"2024-09-16T09:30:27.586366+0000","last_undegraded":"2024-09-16T09:30:27.586366+0000","last_fullsized":"2024-09-16T09:30:27.586366+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:00.169077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:00.169077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:31:17.980553+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":239,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-16T09:30:26.550704+0000","last_change":"2024-09-16T09:29:22.968971+0000","last_active":"2024-09-16T09:30:26.550704+0000","last_peered":"2024-09-16T09:30:26.550704+0000","last_clean":"2024-09-16T09:30:26.550704+0000","last_became_active":"2024-09-16T09:25:07.723282+0000","last_became_peered":"2024-09-16T09:25:07.723282+0000","last_unstale":"2024-09-16T09:30:26.550704+0000","last_undegraded":"2024-09-16T09:30:26.550704+0000","last_fullsized":"2024-09-16T09:30:26.550704+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:22.968908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:22.968908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:41.020638+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"171'3651","reported_seq":3699,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.766434+0000","last_change":"2024-09-16T09:29:49.448898+0000","last_active":"2024-09-16T09:30:27.766434+0000","last_peered":"2024-09-16T09:30:27.766434+0000","last_clean":"2024-09-16T09:30:27.766434+0000","last_became_active":"2024-09-16T09:29:49.448628+0000","last_became_peered":"2024-09-16T09:29:49.448628+0000","last_unstale":"2024-09-16T09:30:27.766434+0000","last_undegraded":"2024-09-16T09:30:27.766434+0000","last_fullsized":"2024-09-16T09:30:27.766434+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":3651,"log_dups_size":0,"ondisk_log_size":3651,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:31:15.665299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":239271936,"num_objects":3651,"num_object_clones":0,"num_object_copies":10953,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3651,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3651,"num_write_kb":233664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"73'79","reported_seq":201,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.582943+0000","last_change":"2024-09-16T09:30:10.707160+0000","last_active":"2024-09-16T09:30:27.582943+0000","last_peered":"2024-09-16T09:30:27.582943+0000","last_clean":"2024-09-16T09:30:27.582943+0000","last_became_active":"2024-09-16T09:30:08.877110+0000","last_became_peered":"2024-09-16T09:30:08.877110+0000","last_unstale":"2024-09-16T09:30:27.582943+0000","last_undegraded":"2024-09-16T09:30:27.582943+0000","last_fullsized":"2024-09-16T09:30:27.582943+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:30:03.887571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:30:03.887571+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:30:39.807998+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"66'12","reported_seq":260,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.582254+0000","last_change":"2024-09-16T09:29:02.713335+0000","last_active":"2024-09-16T09:30:27.582254+0000","last_peered":"2024-09-16T09:30:27.582254+0000","last_clean":"2024-09-16T09:30:27.582254+0000","last_became_active":"2024-09-16T09:25:03.715315+0000","last_became_peered":"2024-09-16T09:25:03.715315+0000","last_unstale":"2024-09-16T09:30:27.582254+0000","last_undegraded":"2024-09-16T09:30:27.582254+0000","last_fullsized":"2024-09-16T09:30:27.582254+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-16T09:29:02.713259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:29:02.713259+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":26,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"59'1","reported_seq":266,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.584999+0000","last_change":"2024-09-16T09:30:21.540586+0000","last_active":"2024-09-16T09:30:27.584999+0000","last_peered":"2024-09-16T09:30:27.584999+0000","last_clean":"2024-09-16T09:30:27.584999+0000","last_became_active":"2024-09-16T09:30:21.530134+0000","last_became_peered":"2024-09-16T09:30:21.530134+0000","last_unstale":"2024-09-16T09:30:27.584999+0000","last_undegraded":"2024-09-16T09:30:27.584999+0000","last_fullsized":"2024-09-16T09:30:27.584999+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:27:00.666673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:27:00.666673+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":28,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":993,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"67'5","reported_seq":582,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-16T09:30:26.550208+0000","last_change":"2024-09-16T09:29:19.994791+0000","last_active":"2024-09-16T09:30:26.550208+0000","last_peered":"2024-09-16T09:30:26.550208+0000","last_clean":"2024-09-16T09:30:26.550208+0000","last_became_active":"2024-09-16T09:25:00.696546+0000","last_became_peered":"2024-09-16T09:25:00.696546+0000","last_unstale":"2024-09-16T09:30:26.550208+0000","last_undegraded":"2024-09-16T09:30:26.550208+0000","last_fullsized":"2024-09-16T09:30:26.550208+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'5","last_scrub_stamp":"2024-09-16T09:29:19.994716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:29:19.994716+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:30.571740+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0.0059770420000000001,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":320,"num_read_kb":315,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":237,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.583007+0000","last_change":"2024-09-16T09:27:07.337170+0000","last_active":"2024-09-16T09:30:27.583007+0000","last_peered":"2024-09-16T09:30:27.583007+0000","last_clean":"2024-09-16T09:30:27.583007+0000","last_became_active":"2024-09-16T09:25:05.714704+0000","last_became_peered":"2024-09-16T09:25:05.714704+0000","last_unstale":"2024-09-16T09:30:27.583007+0000","last_undegraded":"2024-09-16T09:30:27.583007+0000","last_fullsized":"2024-09-16T09:30:27.583007+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:07.337124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:07.337124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:34.478935+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":241,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.577074+0000","last_change":"2024-09-16T09:30:21.526407+0000","last_active":"2024-09-16T09:30:27.577074+0000","last_peered":"2024-09-16T09:30:27.577074+0000","last_clean":"2024-09-16T09:30:27.577074+0000","last_became_active":"2024-09-16T09:30:21.525946+0000","last_became_peered":"2024-09-16T09:30:21.525946+0000","last_unstale":"2024-09-16T09:30:27.577074+0000","last_undegraded":"2024-09-16T09:30:27.577074+0000","last_fullsized":"2024-09-16T09:30:27.577074+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:24.231167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:24.231167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"170'3620","reported_seq":3667,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.537916+0000","last_change":"2024-09-16T09:29:49.450943+0000","last_active":"2024-09-16T09:30:27.537916+0000","last_peered":"2024-09-16T09:30:27.537916+0000","last_clean":"2024-09-16T09:30:27.537916+0000","last_became_active":"2024-09-16T09:29:49.450643+0000","last_became_peered":"2024-09-16T09:29:49.450643+0000","last_unstale":"2024-09-16T09:30:27.537916+0000","last_undegraded":"2024-09-16T09:30:27.537916+0000","last_fullsized":"2024-09-16T09:30:27.537916+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":3620,"log_dups_size":0,"ondisk_log_size":3620,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:48.501094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":237240320,"num_objects":3620,"num_object_clones":0,"num_object_copies":10860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3620,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3620,"num_write_kb":231680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"66'16","reported_seq":328,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-16T09:30:26.565514+0000","last_change":"2024-09-16T09:27:14.265733+0000","last_active":"2024-09-16T09:30:26.565514+0000","last_peered":"2024-09-16T09:30:26.565514+0000","last_clean":"2024-09-16T09:30:26.565514+0000","last_became_active":"2024-09-16T09:25:03.714118+0000","last_became_peered":"2024-09-16T09:25:03.714118+0000","last_unstale":"2024-09-16T09:30:26.565514+0000","last_undegraded":"2024-09-16T09:30:26.565514+0000","last_fullsized":"2024-09-16T09:30:26.565514+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'16","last_scrub_stamp":"2024-09-16T09:27:14.265648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:14.265648+0000","objects_scrubbed":8,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:47.270775+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":88,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.581492+0000","last_change":"2024-09-16T09:30:21.518836+0000","last_active":"2024-09-16T09:30:27.581492+0000","last_peered":"2024-09-16T09:30:27.581492+0000","last_clean":"2024-09-16T09:30:27.581492+0000","last_became_active":"2024-09-16T09:30:21.518447+0000","last_became_peered":"2024-09-16T09:30:21.518447+0000","last_unstale":"2024-09-16T09:30:27.581492+0000","last_undegraded":"2024-09-16T09:30:27.581492+0000","last_fullsized":"2024-09-16T09:30:27.581492+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:49.195331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:27:49.195331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,4],"acting":[5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"73'79","reported_seq":319,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-16T09:30:26.550332+0000","last_change":"2024-09-16T09:30:14.218545+0000","last_active":"2024-09-16T09:30:26.550332+0000","last_peered":"2024-09-16T09:30:26.550332+0000","last_clean":"2024-09-16T09:30:26.550332+0000","last_became_active":"2024-09-16T09:30:08.870394+0000","last_became_peered":"2024-09-16T09:30:08.870394+0000","last_unstale":"2024-09-16T09:30:26.550332+0000","last_undegraded":"2024-09-16T09:30:26.550332+0000","last_fullsized":"2024-09-16T09:30:26.550332+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:02.723794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:02.723794+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:30:41.107669+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":235,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-16T09:30:26.550849+0000","last_change":"2024-09-16T09:26:39.699133+0000","last_active":"2024-09-16T09:30:26.550849+0000","last_peered":"2024-09-16T09:30:26.550849+0000","last_clean":"2024-09-16T09:30:26.550849+0000","last_became_active":"2024-09-16T09:25:05.714341+0000","last_became_peered":"2024-09-16T09:25:05.714341+0000","last_unstale":"2024-09-16T09:30:26.550849+0000","last_undegraded":"2024-09-16T09:30:26.550849+0000","last_fullsized":"2024-09-16T09:30:26.550849+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:39.699059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:26:39.699059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:31:15.129214+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":241,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.581960+0000","last_change":"2024-09-16T09:29:00.243528+0000","last_active":"2024-09-16T09:30:27.581960+0000","last_peered":"2024-09-16T09:30:27.581960+0000","last_clean":"2024-09-16T09:30:27.581960+0000","last_became_active":"2024-09-16T09:25:07.714281+0000","last_became_peered":"2024-09-16T09:25:07.714281+0000","last_unstale":"2024-09-16T09:30:27.581960+0000","last_undegraded":"2024-09-16T09:30:27.581960+0000","last_fullsized":"2024-09-16T09:30:27.581960+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:00.243482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:00.243482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.b","version":"73'79","reported_seq":207,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.582400+0000","last_change":"2024-09-16T09:30:14.212834+0000","last_active":"2024-09-16T09:30:27.582400+0000","last_peered":"2024-09-16T09:30:27.582400+0000","last_clean":"2024-09-16T09:30:27.582400+0000","last_became_active":"2024-09-16T09:30:10.784688+0000","last_became_peered":"2024-09-16T09:30:10.784688+0000","last_unstale":"2024-09-16T09:30:27.582400+0000","last_undegraded":"2024-09-16T09:30:27.582400+0000","last_fullsized":"2024-09-16T09:30:27.582400+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:30:03.887571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:30:03.887571+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:30:41.776745+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,5],"acting":[1,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2","version":"170'3612","reported_seq":3661,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.577141+0000","last_change":"2024-09-16T09:29:49.445324+0000","last_active":"2024-09-16T09:30:27.577141+0000","last_peered":"2024-09-16T09:30:27.577141+0000","last_clean":"2024-09-16T09:30:27.577141+0000","last_became_active":"2024-09-16T09:29:49.445022+0000","last_became_peered":"2024-09-16T09:29:49.445022+0000","last_unstale":"2024-09-16T09:30:27.577141+0000","last_undegraded":"2024-09-16T09:30:27.577141+0000","last_fullsized":"2024-09-16T09:30:27.577141+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":3612,"log_dups_size":0,"ondisk_log_size":3612,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:31:16.543438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":236716032,"num_objects":3612,"num_object_clones":0,"num_object_copies":10836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3612,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3612,"num_write_kb":231168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"66'11","reported_seq":280,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.579265+0000","last_change":"2024-09-16T09:30:05.646701+0000","last_active":"2024-09-16T09:30:27.579265+0000","last_peered":"2024-09-16T09:30:27.579265+0000","last_clean":"2024-09-16T09:30:27.579265+0000","last_became_active":"2024-09-16T09:26:33.287399+0000","last_became_peered":"2024-09-16T09:26:33.287399+0000","last_unstale":"2024-09-16T09:30:27.579265+0000","last_undegraded":"2024-09-16T09:30:27.579265+0000","last_fullsized":"2024-09-16T09:30:27.579265+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-16T09:30:05.646500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:30:05.646500+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:31:29.727455+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":238,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.576847+0000","last_change":"2024-09-16T09:25:01.708568+0000","last_active":"2024-09-16T09:30:27.576847+0000","last_peered":"2024-09-16T09:30:27.576847+0000","last_clean":"2024-09-16T09:30:27.576847+0000","last_became_active":"2024-09-16T09:25:01.708270+0000","last_became_peered":"2024-09-16T09:25:01.708270+0000","last_unstale":"2024-09-16T09:30:27.576847+0000","last_undegraded":"2024-09-16T09:30:27.576847+0000","last_fullsized":"2024-09-16T09:30:27.576847+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:31:14.651548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":237,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.581831+0000","last_change":"2024-09-16T09:26:34.171190+0000","last_active":"2024-09-16T09:30:27.581831+0000","last_peered":"2024-09-16T09:30:27.581831+0000","last_clean":"2024-09-16T09:30:27.581831+0000","last_became_active":"2024-09-16T09:25:05.711964+0000","last_became_peered":"2024-09-16T09:25:05.711964+0000","last_unstale":"2024-09-16T09:30:27.581831+0000","last_undegraded":"2024-09-16T09:30:27.581831+0000","last_fullsized":"2024-09-16T09:30:27.581831+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:34.171139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:26:34.171139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:44.467427+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":95,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-16T09:30:26.564576+0000","last_change":"2024-09-16T09:30:21.520160+0000","last_active":"2024-09-16T09:30:26.564576+0000","last_peered":"2024-09-16T09:30:26.564576+0000","last_clean":"2024-09-16T09:30:26.564576+0000","last_became_active":"2024-09-16T09:30:21.519743+0000","last_became_peered":"2024-09-16T09:30:21.519743+0000","last_unstale":"2024-09-16T09:30:26.564576+0000","last_undegraded":"2024-09-16T09:30:26.564576+0000","last_fullsized":"2024-09-16T09:30:26.564576+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:56.556130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:28:56.556130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,7],"acting":[6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1","version":"171'3520","reported_seq":3569,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.766389+0000","last_change":"2024-09-16T09:29:49.449796+0000","last_active":"2024-09-16T09:30:27.766389+0000","last_peered":"2024-09-16T09:30:27.766389+0000","last_clean":"2024-09-16T09:30:27.766389+0000","last_became_active":"2024-09-16T09:29:49.449475+0000","last_became_peered":"2024-09-16T09:29:49.449475+0000","last_unstale":"2024-09-16T09:30:27.766389+0000","last_undegraded":"2024-09-16T09:30:27.766389+0000","last_fullsized":"2024-09-16T09:30:27.766389+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":3520,"log_dups_size":0,"ondisk_log_size":3520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:31:15.964272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":230686720,"num_objects":3520,"num_object_clones":0,"num_object_copies":10560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3520,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3520,"num_write_kb":225280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.8","version":"73'79","reported_seq":202,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.590329+0000","last_change":"2024-09-16T09:30:21.529897+0000","last_active":"2024-09-16T09:30:27.590329+0000","last_peered":"2024-09-16T09:30:27.590329+0000","last_clean":"2024-09-16T09:30:27.590329+0000","last_became_active":"2024-09-16T09:30:21.517682+0000","last_became_peered":"2024-09-16T09:30:21.517682+0000","last_unstale":"2024-09-16T09:30:27.590329+0000","last_undegraded":"2024-09-16T09:30:27.590329+0000","last_fullsized":"2024-09-16T09:30:27.590329+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:17.482769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:17.482769+0000","objects_scrubbed":1,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:32.341275+0000","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262688,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":262688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"66'17","reported_seq":286,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.578494+0000","last_change":"2024-09-16T09:30:04.636115+0000","last_active":"2024-09-16T09:30:27.578494+0000","last_peered":"2024-09-16T09:30:27.578494+0000","last_clean":"2024-09-16T09:30:27.578494+0000","last_became_active":"2024-09-16T09:25:03.713358+0000","last_became_peered":"2024-09-16T09:25:03.713358+0000","last_unstale":"2024-09-16T09:30:27.578494+0000","last_undegraded":"2024-09-16T09:30:27.578494+0000","last_fullsized":"2024-09-16T09:30:27.578494+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'17","last_scrub_stamp":"2024-09-16T09:30:04.636039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:30:04.636039+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:31:18.451284+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":244,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-16T09:30:26.564086+0000","last_change":"2024-09-16T09:26:04.104188+0000","last_active":"2024-09-16T09:30:26.564086+0000","last_peered":"2024-09-16T09:30:26.564086+0000","last_clean":"2024-09-16T09:30:26.564086+0000","last_became_active":"2024-09-16T09:25:01.703355+0000","last_became_peered":"2024-09-16T09:25:01.703355+0000","last_unstale":"2024-09-16T09:30:26.564086+0000","last_undegraded":"2024-09-16T09:30:26.564086+0000","last_fullsized":"2024-09-16T09:30:26.564086+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:04.104130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:04.104130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:44.256993+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":254,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.577157+0000","last_change":"2024-09-16T09:29:58.839132+0000","last_active":"2024-09-16T09:30:27.577157+0000","last_peered":"2024-09-16T09:30:27.577157+0000","last_clean":"2024-09-16T09:30:27.577157+0000","last_became_active":"2024-09-16T09:25:05.716900+0000","last_became_peered":"2024-09-16T09:25:05.716900+0000","last_unstale":"2024-09-16T09:30:27.577157+0000","last_undegraded":"2024-09-16T09:30:27.577157+0000","last_fullsized":"2024-09-16T09:30:27.577157+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:58.839051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:58.839051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:31:05.210188+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":257,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.585731+0000","last_change":"2024-09-16T09:30:21.518960+0000","last_active":"2024-09-16T09:30:27.585731+0000","last_peered":"2024-09-16T09:30:27.585731+0000","last_clean":"2024-09-16T09:30:27.585731+0000","last_became_active":"2024-09-16T09:30:21.518434+0000","last_became_peered":"2024-09-16T09:30:21.518434+0000","last_unstale":"2024-09-16T09:30:27.585731+0000","last_undegraded":"2024-09-16T09:30:27.585731+0000","last_fullsized":"2024-09-16T09:30:27.585731+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:30:03.946025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:30:03.946025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:31:16.213027+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,6],"acting":[2,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.0","version":"171'3682","reported_seq":3929,"reported_epoch":171,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-16T09:30:27.620913+0000","last_change":"2024-09-16T09:30:22.640774+0000","last_active":"2024-09-16T09:30:27.620913+0000","last_peered":"2024-09-16T09:30:27.620913+0000","last_clean":"2024-09-16T09:30:20.486290+0000","last_became_active":"2024-09-16T09:30:22.600935+0000","last_became_peered":"2024-09-16T09:30:22.600935+0000","last_unstale":"2024-09-16T09:30:27.620913+0000","last_undegraded":"2024-09-16T09:30:27.620913+0000","last_fullsized":"2024-09-16T09:30:22.504263+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":3682,"log_dups_size":0,"ondisk_log_size":3682,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":241303552,"num_objects":3682,"num_object_clones":0,"num_object_copies":11046,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3197,"num_objects_unfound":0,"num_objects_dirty":3682,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3682,"num_write_kb":235648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":98,"num_bytes_recovered":6422528,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1],"avail_no_missing":["5","1","3"],"object_location_counts":[{"shards":"1,3,5","objects":3197},{"shards":"1,5","objects":485}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.9","version":"73'79","reported_seq":315,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-16T09:30:26.551069+0000","last_change":"2024-09-16T09:29:19.013948+0000","last_active":"2024-09-16T09:30:26.551069+0000","last_peered":"2024-09-16T09:30:26.551069+0000","last_clean":"2024-09-16T09:30:26.551069+0000","last_became_active":"2024-09-16T09:26:35.126066+0000","last_became_peered":"2024-09-16T09:26:35.126066+0000","last_unstale":"2024-09-16T09:30:26.551069+0000","last_undegraded":"2024-09-16T09:30:26.551069+0000","last_fullsized":"2024-09-16T09:30:26.551069+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:19.013865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:19.013865+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"66'10","reported_seq":249,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.578094+0000","last_change":"2024-09-16T09:25:03.708144+0000","last_active":"2024-09-16T09:30:27.578094+0000","last_peered":"2024-09-16T09:30:27.578094+0000","last_clean":"2024-09-16T09:30:27.578094+0000","last_became_active":"2024-09-16T09:25:03.707723+0000","last_became_peered":"2024-09-16T09:25:03.707723+0000","last_unstale":"2024-09-16T09:30:27.578094+0000","last_undegraded":"2024-09-16T09:30:27.578094+0000","last_fullsized":"2024-09-16T09:30:27.578094+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:31:16.694153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":96,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.582033+0000","last_change":"2024-09-16T09:30:21.526533+0000","last_active":"2024-09-16T09:30:27.582033+0000","last_peered":"2024-09-16T09:30:27.582033+0000","last_clean":"2024-09-16T09:30:27.582033+0000","last_became_active":"2024-09-16T09:30:21.525581+0000","last_became_peered":"2024-09-16T09:30:21.525581+0000","last_unstale":"2024-09-16T09:30:27.582033+0000","last_undegraded":"2024-09-16T09:30:27.582033+0000","last_fullsized":"2024-09-16T09:30:27.582033+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:56.590937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:28:56.590937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,1],"acting":[0,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":331,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.585985+0000","last_change":"2024-09-16T09:29:29.866520+0000","last_active":"2024-09-16T09:30:27.585985+0000","last_peered":"2024-09-16T09:30:27.585985+0000","last_clean":"2024-09-16T09:30:27.585985+0000","last_became_active":"2024-09-16T09:25:05.711590+0000","last_became_peered":"2024-09-16T09:25:05.711590+0000","last_unstale":"2024-09-16T09:30:27.585985+0000","last_undegraded":"2024-09-16T09:30:27.585985+0000","last_fullsized":"2024-09-16T09:30:27.585985+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:29:29.866460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:29.866460+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:51.037099+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":250,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.579045+0000","last_change":"2024-09-16T09:29:02.667332+0000","last_active":"2024-09-16T09:30:27.579045+0000","last_peered":"2024-09-16T09:30:27.579045+0000","last_clean":"2024-09-16T09:30:27.579045+0000","last_became_active":"2024-09-16T09:25:07.718958+0000","last_became_peered":"2024-09-16T09:25:07.718958+0000","last_unstale":"2024-09-16T09:30:27.579045+0000","last_undegraded":"2024-09-16T09:30:27.579045+0000","last_fullsized":"2024-09-16T09:30:27.579045+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:02.667259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:02.667259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:28.829982+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.e","version":"73'79","reported_seq":31,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.578891+0000","last_change":"2024-09-16T09:30:12.543108+0000","last_active":"2024-09-16T09:30:27.578891+0000","last_peered":"2024-09-16T09:30:27.578891+0000","last_clean":"2024-09-16T09:30:27.578891+0000","last_became_active":"2024-09-16T09:30:10.705072+0000","last_became_peered":"2024-09-16T09:30:10.705072+0000","last_unstale":"2024-09-16T09:30:27.578891+0000","last_undegraded":"2024-09-16T09:30:27.578891+0000","last_fullsized":"2024-09-16T09:30:27.578891+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:26:35.170229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:26:35.170229+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:30:41.648470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26835,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":10,"num_write":5,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":26835,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,3],"acting":[4,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.7","version":"171'3666","reported_seq":3715,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.722937+0000","last_change":"2024-09-16T09:29:49.839740+0000","last_active":"2024-09-16T09:30:27.722937+0000","last_peered":"2024-09-16T09:30:27.722937+0000","last_clean":"2024-09-16T09:30:27.722937+0000","last_became_active":"2024-09-16T09:29:49.839437+0000","last_became_peered":"2024-09-16T09:29:49.839437+0000","last_unstale":"2024-09-16T09:30:27.722937+0000","last_undegraded":"2024-09-16T09:30:27.722937+0000","last_fullsized":"2024-09-16T09:30:27.722937+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":3666,"log_dups_size":0,"ondisk_log_size":3666,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:54.973078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":240254976,"num_objects":3666,"num_object_clones":0,"num_object_copies":10998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3666,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3666,"num_write_kb":234624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"66'9","reported_seq":266,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.581782+0000","last_change":"2024-09-16T09:28:23.570956+0000","last_active":"2024-09-16T09:30:27.581782+0000","last_peered":"2024-09-16T09:30:27.581782+0000","last_clean":"2024-09-16T09:30:27.581782+0000","last_became_active":"2024-09-16T09:25:03.712230+0000","last_became_peered":"2024-09-16T09:25:03.712230+0000","last_unstale":"2024-09-16T09:30:27.581782+0000","last_undegraded":"2024-09-16T09:30:27.581782+0000","last_fullsized":"2024-09-16T09:30:27.581782+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:28:23.570883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:23.570883+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":252,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.583171+0000","last_change":"2024-09-16T09:30:21.523697+0000","last_active":"2024-09-16T09:30:27.583171+0000","last_peered":"2024-09-16T09:30:27.583171+0000","last_clean":"2024-09-16T09:30:27.583171+0000","last_became_active":"2024-09-16T09:30:21.523265+0000","last_became_peered":"2024-09-16T09:30:21.523265+0000","last_unstale":"2024-09-16T09:30:27.583171+0000","last_undegraded":"2024-09-16T09:30:27.583171+0000","last_fullsized":"2024-09-16T09:30:27.583171+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:26.061106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:26.061106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":253,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.585436+0000","last_change":"2024-09-16T09:30:21.522402+0000","last_active":"2024-09-16T09:30:27.585436+0000","last_peered":"2024-09-16T09:30:27.585436+0000","last_clean":"2024-09-16T09:30:27.585436+0000","last_became_active":"2024-09-16T09:30:21.522052+0000","last_became_peered":"2024-09-16T09:30:21.522052+0000","last_unstale":"2024-09-16T09:30:27.585436+0000","last_undegraded":"2024-09-16T09:30:27.585436+0000","last_fullsized":"2024-09-16T09:30:27.585436+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:57.220068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:28:57.220068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":33,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,0],"acting":[2,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":242,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.584919+0000","last_change":"2024-09-16T09:28:25.593045+0000","last_active":"2024-09-16T09:30:27.584919+0000","last_peered":"2024-09-16T09:30:27.584919+0000","last_clean":"2024-09-16T09:30:27.584919+0000","last_became_active":"2024-09-16T09:25:07.718778+0000","last_became_peered":"2024-09-16T09:25:07.718778+0000","last_unstale":"2024-09-16T09:30:27.584919+0000","last_undegraded":"2024-09-16T09:30:27.584919+0000","last_fullsized":"2024-09-16T09:30:27.584919+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:25.592728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:28:25.592728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.f","version":"73'79","reported_seq":281,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-16T09:30:26.566181+0000","last_change":"2024-09-16T09:30:12.570468+0000","last_active":"2024-09-16T09:30:26.566181+0000","last_peered":"2024-09-16T09:30:26.566181+0000","last_clean":"2024-09-16T09:30:26.566181+0000","last_became_active":"2024-09-16T09:30:10.693477+0000","last_became_peered":"2024-09-16T09:30:10.693477+0000","last_unstale":"2024-09-16T09:30:26.566181+0000","last_undegraded":"2024-09-16T09:30:26.566181+0000","last_fullsized":"2024-09-16T09:30:26.566181+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:17.534429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:17.534429+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:30:42.362212+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.6","version":"171'3729","reported_seq":3778,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:28.010774+0000","last_change":"2024-09-16T09:29:50.008888+0000","last_active":"2024-09-16T09:30:28.010774+0000","last_peered":"2024-09-16T09:30:28.010774+0000","last_clean":"2024-09-16T09:30:28.010774+0000","last_became_active":"2024-09-16T09:29:50.008656+0000","last_became_peered":"2024-09-16T09:29:50.008656+0000","last_unstale":"2024-09-16T09:30:28.010774+0000","last_undegraded":"2024-09-16T09:30:28.010774+0000","last_fullsized":"2024-09-16T09:30:28.010774+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":3729,"log_dups_size":0,"ondisk_log_size":3729,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:31:18.079408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":244383744,"num_objects":3729,"num_object_clones":0,"num_object_copies":11187,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3729,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3729,"num_write_kb":238656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"66'19","reported_seq":279,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-16T09:30:26.563876+0000","last_change":"2024-09-16T09:29:02.554357+0000","last_active":"2024-09-16T09:30:26.563876+0000","last_peered":"2024-09-16T09:30:26.563876+0000","last_clean":"2024-09-16T09:30:26.563876+0000","last_became_active":"2024-09-16T09:25:03.712783+0000","last_became_peered":"2024-09-16T09:25:03.712783+0000","last_unstale":"2024-09-16T09:30:26.563876+0000","last_undegraded":"2024-09-16T09:30:26.563876+0000","last_fullsized":"2024-09-16T09:30:26.563876+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-16T09:29:02.554274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:29:02.554274+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:31:15.390724+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"59'1","reported_seq":257,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-16T09:30:26.551122+0000","last_change":"2024-09-16T09:29:51.025505+0000","last_active":"2024-09-16T09:30:26.551122+0000","last_peered":"2024-09-16T09:30:26.551122+0000","last_clean":"2024-09-16T09:30:26.551122+0000","last_became_active":"2024-09-16T09:25:01.708069+0000","last_became_peered":"2024-09-16T09:25:01.708069+0000","last_unstale":"2024-09-16T09:30:26.551122+0000","last_undegraded":"2024-09-16T09:30:26.551122+0000","last_fullsized":"2024-09-16T09:30:26.551122+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:29:51.025444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:51.025444+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:51.262232+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":238,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.587067+0000","last_change":"2024-09-16T09:27:11.067059+0000","last_active":"2024-09-16T09:30:27.587067+0000","last_peered":"2024-09-16T09:30:27.587067+0000","last_clean":"2024-09-16T09:30:27.587067+0000","last_became_active":"2024-09-16T09:25:05.709264+0000","last_became_peered":"2024-09-16T09:25:05.709264+0000","last_unstale":"2024-09-16T09:30:27.587067+0000","last_undegraded":"2024-09-16T09:30:27.587067+0000","last_fullsized":"2024-09-16T09:30:27.587067+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:11.066986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:11.066986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:47.015327+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":231,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-16T09:30:26.550252+0000","last_change":"2024-09-16T09:27:25.865107+0000","last_active":"2024-09-16T09:30:26.550252+0000","last_peered":"2024-09-16T09:30:26.550252+0000","last_clean":"2024-09-16T09:30:26.550252+0000","last_became_active":"2024-09-16T09:25:07.718750+0000","last_became_peered":"2024-09-16T09:25:07.718750+0000","last_unstale":"2024-09-16T09:30:26.550252+0000","last_undegraded":"2024-09-16T09:30:26.550252+0000","last_fullsized":"2024-09-16T09:30:26.550252+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:25.865030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:25.865030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:35.955976+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.c","version":"73'79","reported_seq":217,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.582385+0000","last_change":"2024-09-16T09:30:22.614537+0000","last_active":"2024-09-16T09:30:27.582385+0000","last_peered":"2024-09-16T09:30:27.582385+0000","last_clean":"2024-09-16T09:30:27.582385+0000","last_became_active":"2024-09-16T09:30:21.526976+0000","last_became_peered":"2024-09-16T09:30:21.526976+0000","last_unstale":"2024-09-16T09:30:27.582385+0000","last_undegraded":"2024-09-16T09:30:27.582385+0000","last_fullsized":"2024-09-16T09:30:27.582385+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:30:22.614462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:30:22.614462+0000","objects_scrubbed":1,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:31:35.978749+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.5","version":"170'3693","reported_seq":3740,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.535104+0000","last_change":"2024-09-16T09:29:50.010183+0000","last_active":"2024-09-16T09:30:27.535104+0000","last_peered":"2024-09-16T09:30:27.535104+0000","last_clean":"2024-09-16T09:30:27.535104+0000","last_became_active":"2024-09-16T09:29:50.009013+0000","last_became_peered":"2024-09-16T09:29:50.009013+0000","last_unstale":"2024-09-16T09:30:27.535104+0000","last_undegraded":"2024-09-16T09:30:27.535104+0000","last_fullsized":"2024-09-16T09:30:27.535104+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":3693,"log_dups_size":0,"ondisk_log_size":3693,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:31:02.782637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":242024448,"num_objects":3693,"num_object_clones":0,"num_object_copies":11079,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3693,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3693,"num_write_kb":236352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"66'12","reported_seq":292,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.576713+0000","last_change":"2024-09-16T09:30:21.728723+0000","last_active":"2024-09-16T09:30:27.576713+0000","last_peered":"2024-09-16T09:30:27.576713+0000","last_clean":"2024-09-16T09:30:27.576713+0000","last_became_active":"2024-09-16T09:30:21.521343+0000","last_became_peered":"2024-09-16T09:30:21.521343+0000","last_unstale":"2024-09-16T09:30:27.576713+0000","last_undegraded":"2024-09-16T09:30:27.576713+0000","last_fullsized":"2024-09-16T09:30:27.576713+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-16T09:30:21.728668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:30:21.728668+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:31:22.144268+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6],"acting":[4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":251,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-16T09:30:26.552421+0000","last_change":"2024-09-16T09:29:53.075288+0000","last_active":"2024-09-16T09:30:26.552421+0000","last_peered":"2024-09-16T09:30:26.552421+0000","last_clean":"2024-09-16T09:30:26.552421+0000","last_became_active":"2024-09-16T09:25:01.707528+0000","last_became_peered":"2024-09-16T09:25:01.707528+0000","last_unstale":"2024-09-16T09:30:26.552421+0000","last_undegraded":"2024-09-16T09:30:26.552421+0000","last_fullsized":"2024-09-16T09:30:26.552421+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:53.075232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:53.075232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:31:02.701620+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":246,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.585169+0000","last_change":"2024-09-16T09:27:49.965628+0000","last_active":"2024-09-16T09:30:27.585169+0000","last_peered":"2024-09-16T09:30:27.585169+0000","last_clean":"2024-09-16T09:30:27.585169+0000","last_became_active":"2024-09-16T09:25:05.715253+0000","last_became_peered":"2024-09-16T09:25:05.715253+0000","last_unstale":"2024-09-16T09:30:27.585169+0000","last_undegraded":"2024-09-16T09:30:27.585169+0000","last_fullsized":"2024-09-16T09:30:27.585169+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:49.965424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:49.965424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:31:06.007398+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":234,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.578940+0000","last_change":"2024-09-16T09:27:56.059755+0000","last_active":"2024-09-16T09:30:27.578940+0000","last_peered":"2024-09-16T09:30:27.578940+0000","last_clean":"2024-09-16T09:30:27.578940+0000","last_became_active":"2024-09-16T09:25:07.718899+0000","last_became_peered":"2024-09-16T09:25:07.718899+0000","last_unstale":"2024-09-16T09:30:27.578940+0000","last_undegraded":"2024-09-16T09:30:27.578940+0000","last_fullsized":"2024-09-16T09:30:27.578940+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:56.059687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:56.059687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:31:20.007543+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.d","version":"73'79","reported_seq":29,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-16T09:30:26.550428+0000","last_change":"2024-09-16T09:30:12.542549+0000","last_active":"2024-09-16T09:30:26.550428+0000","last_peered":"2024-09-16T09:30:26.550428+0000","last_clean":"2024-09-16T09:30:26.550428+0000","last_became_active":"2024-09-16T09:30:10.709958+0000","last_became_peered":"2024-09-16T09:30:10.709958+0000","last_unstale":"2024-09-16T09:30:26.550428+0000","last_undegraded":"2024-09-16T09:30:26.550428+0000","last_fullsized":"2024-09-16T09:30:26.550428+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:58.568841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:58.568841+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:30:42.131031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26835,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":10,"num_write":5,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":26835,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.4","version":"171'3744","reported_seq":3793,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.603943+0000","last_change":"2024-09-16T09:29:49.444857+0000","last_active":"2024-09-16T09:30:27.603943+0000","last_peered":"2024-09-16T09:30:27.603943+0000","last_clean":"2024-09-16T09:30:27.603943+0000","last_became_active":"2024-09-16T09:29:49.444515+0000","last_became_peered":"2024-09-16T09:29:49.444515+0000","last_unstale":"2024-09-16T09:30:27.603943+0000","last_undegraded":"2024-09-16T09:30:27.603943+0000","last_fullsized":"2024-09-16T09:30:27.603943+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":3744,"log_dups_size":0,"ondisk_log_size":3744,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:31:14.656695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":245366784,"num_objects":3744,"num_object_clones":0,"num_object_copies":11232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3744,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3744,"num_write_kb":239616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"66'15","reported_seq":266,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.590507+0000","last_change":"2024-09-16T09:26:33.138828+0000","last_active":"2024-09-16T09:30:27.590507+0000","last_peered":"2024-09-16T09:30:27.590507+0000","last_clean":"2024-09-16T09:30:27.590507+0000","last_became_active":"2024-09-16T09:25:03.715733+0000","last_became_peered":"2024-09-16T09:25:03.715733+0000","last_unstale":"2024-09-16T09:30:27.590507+0000","last_undegraded":"2024-09-16T09:30:27.590507+0000","last_fullsized":"2024-09-16T09:30:27.590507+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-16T09:26:33.138603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:26:33.138603+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:46.502077+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":255,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-16T09:30:26.551968+0000","last_change":"2024-09-16T09:26:26.705847+0000","last_active":"2024-09-16T09:30:26.551968+0000","last_peered":"2024-09-16T09:30:26.551968+0000","last_clean":"2024-09-16T09:30:26.551968+0000","last_became_active":"2024-09-16T09:25:01.707372+0000","last_became_peered":"2024-09-16T09:25:01.707372+0000","last_unstale":"2024-09-16T09:30:26.551968+0000","last_undegraded":"2024-09-16T09:30:26.551968+0000","last_fullsized":"2024-09-16T09:30:26.551968+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-16T09:26:26.705773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:26.705773+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:44.096466+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"66'8","reported_seq":325,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-16T09:30:26.551021+0000","last_change":"2024-09-16T09:30:05.138362+0000","last_active":"2024-09-16T09:30:26.551021+0000","last_peered":"2024-09-16T09:30:26.551021+0000","last_clean":"2024-09-16T09:30:26.551021+0000","last_became_active":"2024-09-16T09:25:05.716189+0000","last_became_peered":"2024-09-16T09:25:05.716189+0000","last_unstale":"2024-09-16T09:30:26.551021+0000","last_undegraded":"2024-09-16T09:30:26.551021+0000","last_fullsized":"2024-09-16T09:30:26.551021+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:30:05.138288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:30:05.138288+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:31:30.927152+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":233,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.581105+0000","last_change":"2024-09-16T09:27:20.219376+0000","last_active":"2024-09-16T09:30:27.581105+0000","last_peered":"2024-09-16T09:30:27.581105+0000","last_clean":"2024-09-16T09:30:27.581105+0000","last_became_active":"2024-09-16T09:25:07.717213+0000","last_became_peered":"2024-09-16T09:25:07.717213+0000","last_unstale":"2024-09-16T09:30:27.581105+0000","last_undegraded":"2024-09-16T09:30:27.581105+0000","last_fullsized":"2024-09-16T09:30:27.581105+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:20.219333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:20.219333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:37.526567+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.12","version":"73'79","reported_seq":31,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.581340+0000","last_change":"2024-09-16T09:30:12.543909+0000","last_active":"2024-09-16T09:30:27.581340+0000","last_peered":"2024-09-16T09:30:27.581340+0000","last_clean":"2024-09-16T09:30:27.581340+0000","last_became_active":"2024-09-16T09:30:10.731413+0000","last_became_peered":"2024-09-16T09:30:10.731413+0000","last_unstale":"2024-09-16T09:30:27.581340+0000","last_undegraded":"2024-09-16T09:30:27.581340+0000","last_fullsized":"2024-09-16T09:30:27.581340+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:26:35.745837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:26:35.745837+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:30:42.366646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26835,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":10,"num_write":6,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":26835,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":244,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-16T09:30:26.563116+0000","last_change":"2024-09-16T09:26:33.139405+0000","last_active":"2024-09-16T09:30:26.563116+0000","last_peered":"2024-09-16T09:30:26.563116+0000","last_clean":"2024-09-16T09:30:26.563116+0000","last_became_active":"2024-09-16T09:25:01.703202+0000","last_became_peered":"2024-09-16T09:25:01.703202+0000","last_unstale":"2024-09-16T09:30:26.563116+0000","last_undegraded":"2024-09-16T09:30:26.563116+0000","last_fullsized":"2024-09-16T09:30:26.563116+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:33.139360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:33.139360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:31:08.384301+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"66'6","reported_seq":56,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-16T09:30:26.550266+0000","last_change":"2024-09-16T09:30:21.547476+0000","last_active":"2024-09-16T09:30:26.550266+0000","last_peered":"2024-09-16T09:30:26.550266+0000","last_clean":"2024-09-16T09:30:26.550266+0000","last_became_active":"2024-09-16T09:30:21.526308+0000","last_became_peered":"2024-09-16T09:30:21.526308+0000","last_unstale":"2024-09-16T09:30:26.550266+0000","last_undegraded":"2024-09-16T09:30:26.550266+0000","last_fullsized":"2024-09-16T09:30:26.550266+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'6","last_scrub_stamp":"2024-09-16T09:26:23.458794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:26:23.458794+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":8,"num_read_kb":5,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":245,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.581748+0000","last_change":"2024-09-16T09:29:31.496374+0000","last_active":"2024-09-16T09:30:27.581748+0000","last_peered":"2024-09-16T09:30:27.581748+0000","last_clean":"2024-09-16T09:30:27.581748+0000","last_became_active":"2024-09-16T09:25:05.715616+0000","last_became_peered":"2024-09-16T09:25:05.715616+0000","last_unstale":"2024-09-16T09:30:27.581748+0000","last_undegraded":"2024-09-16T09:30:27.581748+0000","last_fullsized":"2024-09-16T09:30:27.581748+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:31.496304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:31.496304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:44.164855+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":231,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-16T09:30:26.550628+0000","last_change":"2024-09-16T09:26:28.733381+0000","last_active":"2024-09-16T09:30:26.550628+0000","last_peered":"2024-09-16T09:30:26.550628+0000","last_clean":"2024-09-16T09:30:26.550628+0000","last_became_active":"2024-09-16T09:25:07.721059+0000","last_became_peered":"2024-09-16T09:25:07.721059+0000","last_unstale":"2024-09-16T09:30:26.550628+0000","last_undegraded":"2024-09-16T09:30:26.550628+0000","last_fullsized":"2024-09-16T09:30:26.550628+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:28.733324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:28.733324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:31:18.119032+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.13","version":"73'79","reported_seq":222,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.582773+0000","last_change":"2024-09-16T09:30:21.714434+0000","last_active":"2024-09-16T09:30:27.582773+0000","last_peered":"2024-09-16T09:30:27.582773+0000","last_clean":"2024-09-16T09:30:27.582773+0000","last_became_active":"2024-09-16T09:30:21.522796+0000","last_became_peered":"2024-09-16T09:30:21.522796+0000","last_unstale":"2024-09-16T09:30:27.582773+0000","last_undegraded":"2024-09-16T09:30:27.582773+0000","last_fullsized":"2024-09-16T09:30:27.582773+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:30:21.714369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:30:21.714369+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:31:29.735821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.16","version":"66'9","reported_seq":291,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.582344+0000","last_change":"2024-09-16T09:30:21.550522+0000","last_active":"2024-09-16T09:30:27.582344+0000","last_peered":"2024-09-16T09:30:27.582344+0000","last_clean":"2024-09-16T09:30:27.582344+0000","last_became_active":"2024-09-16T09:30:21.528908+0000","last_became_peered":"2024-09-16T09:30:21.528908+0000","last_unstale":"2024-09-16T09:30:27.582344+0000","last_undegraded":"2024-09-16T09:30:27.582344+0000","last_fullsized":"2024-09-16T09:30:27.582344+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:29:58.867513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:29:58.867513+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:31:27.174087+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":251,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-16T09:30:26.551290+0000","last_change":"2024-09-16T09:28:50.698331+0000","last_active":"2024-09-16T09:30:26.551290+0000","last_peered":"2024-09-16T09:30:26.551290+0000","last_clean":"2024-09-16T09:30:26.551290+0000","last_became_active":"2024-09-16T09:25:01.706491+0000","last_became_peered":"2024-09-16T09:25:01.706491+0000","last_unstale":"2024-09-16T09:30:26.551290+0000","last_undegraded":"2024-09-16T09:30:26.551290+0000","last_fullsized":"2024-09-16T09:30:26.551290+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:50.698288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:28:50.698288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:31:21.149848+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":96,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.582949+0000","last_change":"2024-09-16T09:30:21.523953+0000","last_active":"2024-09-16T09:30:27.582949+0000","last_peered":"2024-09-16T09:30:27.582949+0000","last_clean":"2024-09-16T09:30:27.582949+0000","last_became_active":"2024-09-16T09:30:21.523670+0000","last_became_peered":"2024-09-16T09:30:21.523670+0000","last_unstale":"2024-09-16T09:30:27.582949+0000","last_undegraded":"2024-09-16T09:30:27.582949+0000","last_fullsized":"2024-09-16T09:30:27.582949+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:53.810580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:28:53.810580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":234,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.585084+0000","last_change":"2024-09-16T09:26:35.866582+0000","last_active":"2024-09-16T09:30:27.585084+0000","last_peered":"2024-09-16T09:30:27.585084+0000","last_clean":"2024-09-16T09:30:27.585084+0000","last_became_active":"2024-09-16T09:25:07.722050+0000","last_became_peered":"2024-09-16T09:25:07.722050+0000","last_unstale":"2024-09-16T09:30:27.585084+0000","last_undegraded":"2024-09-16T09:30:27.585084+0000","last_fullsized":"2024-09-16T09:30:27.585084+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:35.866512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:35.866512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:40.909599+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.10","version":"73'79","reported_seq":31,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.583038+0000","last_change":"2024-09-16T09:30:12.543869+0000","last_active":"2024-09-16T09:30:27.583038+0000","last_peered":"2024-09-16T09:30:27.583038+0000","last_clean":"2024-09-16T09:30:27.583038+0000","last_became_active":"2024-09-16T09:30:10.692770+0000","last_became_peered":"2024-09-16T09:30:10.692770+0000","last_unstale":"2024-09-16T09:30:27.583038+0000","last_undegraded":"2024-09-16T09:30:27.583038+0000","last_fullsized":"2024-09-16T09:30:27.583038+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:02.723794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:02.723794+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:30:41.398719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26835,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":10,"num_write":5,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":26835,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"66'9","reported_seq":265,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.582020+0000","last_change":"2024-09-16T09:29:32.485165+0000","last_active":"2024-09-16T09:30:27.582020+0000","last_peered":"2024-09-16T09:30:27.582020+0000","last_clean":"2024-09-16T09:30:27.582020+0000","last_became_active":"2024-09-16T09:25:03.717310+0000","last_became_peered":"2024-09-16T09:25:03.717310+0000","last_unstale":"2024-09-16T09:30:27.582020+0000","last_undegraded":"2024-09-16T09:30:27.582020+0000","last_fullsized":"2024-09-16T09:30:27.582020+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:29:32.485084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:29:32.485084+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:32.698063+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":261,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.584301+0000","last_change":"2024-09-16T09:30:21.527647+0000","last_active":"2024-09-16T09:30:27.584301+0000","last_peered":"2024-09-16T09:30:27.584301+0000","last_clean":"2024-09-16T09:30:27.584301+0000","last_became_active":"2024-09-16T09:30:21.527121+0000","last_became_peered":"2024-09-16T09:30:21.527121+0000","last_unstale":"2024-09-16T09:30:27.584301+0000","last_undegraded":"2024-09-16T09:30:27.584301+0000","last_fullsized":"2024-09-16T09:30:27.584301+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.950144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.950144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:52.621786+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"167'10","reported_seq":143,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-16T09:30:26.550806+0000","last_change":"2024-09-16T09:30:21.542265+0000","last_active":"2024-09-16T09:30:26.550806+0000","last_peered":"2024-09-16T09:30:26.550806+0000","last_clean":"2024-09-16T09:30:26.550806+0000","last_became_active":"2024-09-16T09:30:21.522242+0000","last_became_peered":"2024-09-16T09:30:21.522242+0000","last_unstale":"2024-09-16T09:30:26.550806+0000","last_undegraded":"2024-09-16T09:30:26.550806+0000","last_fullsized":"2024-09-16T09:30:26.550806+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:28:53.714413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:28:53.714413+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,1],"acting":[7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":250,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.578692+0000","last_change":"2024-09-16T09:29:49.474440+0000","last_active":"2024-09-16T09:30:27.578692+0000","last_peered":"2024-09-16T09:30:27.578692+0000","last_clean":"2024-09-16T09:30:27.578692+0000","last_became_active":"2024-09-16T09:25:07.718810+0000","last_became_peered":"2024-09-16T09:25:07.718810+0000","last_unstale":"2024-09-16T09:30:27.578692+0000","last_undegraded":"2024-09-16T09:30:27.578692+0000","last_fullsized":"2024-09-16T09:30:27.578692+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:49.474391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:49.474391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:59.740100+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.11","version":"73'79","reported_seq":122,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.585090+0000","last_change":"2024-09-16T09:30:21.651624+0000","last_active":"2024-09-16T09:30:27.585090+0000","last_peered":"2024-09-16T09:30:27.585090+0000","last_clean":"2024-09-16T09:30:27.585090+0000","last_became_active":"2024-09-16T09:30:21.534283+0000","last_became_peered":"2024-09-16T09:30:21.534283+0000","last_unstale":"2024-09-16T09:30:27.585090+0000","last_undegraded":"2024-09-16T09:30:27.585090+0000","last_fullsized":"2024-09-16T09:30:27.585090+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:30:21.651575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:30:21.651575+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:31:41.891004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.14","version":"66'10","reported_seq":124,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.577288+0000","last_change":"2024-09-16T09:28:56.825675+0000","last_active":"2024-09-16T09:30:27.577288+0000","last_peered":"2024-09-16T09:30:27.577288+0000","last_clean":"2024-09-16T09:30:27.577288+0000","last_became_active":"2024-09-16T09:28:52.979059+0000","last_became_peered":"2024-09-16T09:28:52.979059+0000","last_unstale":"2024-09-16T09:30:27.577288+0000","last_undegraded":"2024-09-16T09:30:27.577288+0000","last_fullsized":"2024-09-16T09:30:27.577288+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-16T09:28:56.825476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:56.825476+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":14,"num_read_kb":9,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":251,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-16T09:30:26.550003+0000","last_change":"2024-09-16T09:29:56.021413+0000","last_active":"2024-09-16T09:30:26.550003+0000","last_peered":"2024-09-16T09:30:26.550003+0000","last_clean":"2024-09-16T09:30:26.550003+0000","last_became_active":"2024-09-16T09:25:01.707513+0000","last_became_peered":"2024-09-16T09:25:01.707513+0000","last_unstale":"2024-09-16T09:30:26.550003+0000","last_undegraded":"2024-09-16T09:30:26.550003+0000","last_fullsized":"2024-09-16T09:30:26.550003+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:56.021340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:56.021340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:31:04.402691+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"66'8","reported_seq":327,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.581619+0000","last_change":"2024-09-16T09:29:48.485781+0000","last_active":"2024-09-16T09:30:27.581619+0000","last_peered":"2024-09-16T09:30:27.581619+0000","last_clean":"2024-09-16T09:30:27.581619+0000","last_became_active":"2024-09-16T09:25:05.713833+0000","last_became_peered":"2024-09-16T09:25:05.713833+0000","last_unstale":"2024-09-16T09:30:27.581619+0000","last_undegraded":"2024-09-16T09:30:27.581619+0000","last_fullsized":"2024-09-16T09:30:27.581619+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:29:48.485727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.485727+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:31:03.042280+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":249,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.581791+0000","last_change":"2024-09-16T09:30:21.533128+0000","last_active":"2024-09-16T09:30:27.581791+0000","last_peered":"2024-09-16T09:30:27.581791+0000","last_clean":"2024-09-16T09:30:27.581791+0000","last_became_active":"2024-09-16T09:30:21.525049+0000","last_became_peered":"2024-09-16T09:30:21.525049+0000","last_unstale":"2024-09-16T09:30:27.581791+0000","last_undegraded":"2024-09-16T09:30:27.581791+0000","last_fullsized":"2024-09-16T09:30:27.581791+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:52.958303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:52.958303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:59.950324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"66'11","reported_seq":272,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.579834+0000","last_change":"2024-09-16T09:26:33.302713+0000","last_active":"2024-09-16T09:30:27.579834+0000","last_peered":"2024-09-16T09:30:27.579834+0000","last_clean":"2024-09-16T09:30:27.579834+0000","last_became_active":"2024-09-16T09:26:33.291265+0000","last_became_peered":"2024-09-16T09:26:33.291265+0000","last_unstale":"2024-09-16T09:30:27.579834+0000","last_undegraded":"2024-09-16T09:30:27.579834+0000","last_fullsized":"2024-09-16T09:30:27.579834+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:44.612142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":262,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.578178+0000","last_change":"2024-09-16T09:29:51.517112+0000","last_active":"2024-09-16T09:30:27.578178+0000","last_peered":"2024-09-16T09:30:27.578178+0000","last_clean":"2024-09-16T09:30:27.578178+0000","last_became_active":"2024-09-16T09:25:01.705158+0000","last_became_peered":"2024-09-16T09:25:01.705158+0000","last_unstale":"2024-09-16T09:30:27.578178+0000","last_undegraded":"2024-09-16T09:30:27.578178+0000","last_fullsized":"2024-09-16T09:30:27.578178+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:51.517054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:51.517054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:31:13.265239+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":261,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.582228+0000","last_change":"2024-09-16T09:30:21.526541+0000","last_active":"2024-09-16T09:30:27.582228+0000","last_peered":"2024-09-16T09:30:27.582228+0000","last_clean":"2024-09-16T09:30:27.582228+0000","last_became_active":"2024-09-16T09:30:21.526238+0000","last_became_peered":"2024-09-16T09:30:21.526238+0000","last_unstale":"2024-09-16T09:30:27.582228+0000","last_undegraded":"2024-09-16T09:30:27.582228+0000","last_fullsized":"2024-09-16T09:30:27.582228+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:15.910824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:15.910824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":89,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.584831+0000","last_change":"2024-09-16T09:30:21.531966+0000","last_active":"2024-09-16T09:30:27.584831+0000","last_peered":"2024-09-16T09:30:27.584831+0000","last_clean":"2024-09-16T09:30:27.584831+0000","last_became_active":"2024-09-16T09:30:21.528860+0000","last_became_peered":"2024-09-16T09:30:21.528860+0000","last_unstale":"2024-09-16T09:30:27.584831+0000","last_undegraded":"2024-09-16T09:30:27.584831+0000","last_fullsized":"2024-09-16T09:30:27.584831+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:48.147759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:48.147759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"66'9","reported_seq":258,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.582766+0000","last_change":"2024-09-16T09:26:31.216495+0000","last_active":"2024-09-16T09:30:27.582766+0000","last_peered":"2024-09-16T09:30:27.582766+0000","last_clean":"2024-09-16T09:30:27.582766+0000","last_became_active":"2024-09-16T09:25:03.705807+0000","last_became_peered":"2024-09-16T09:25:03.705807+0000","last_unstale":"2024-09-16T09:30:27.582766+0000","last_undegraded":"2024-09-16T09:30:27.582766+0000","last_fullsized":"2024-09-16T09:30:27.582766+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:26:31.216298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:26:31.216298+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:31:16.684288+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":243,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-16T09:30:26.550992+0000","last_change":"2024-09-16T09:26:23.796308+0000","last_active":"2024-09-16T09:30:26.550992+0000","last_peered":"2024-09-16T09:30:26.550992+0000","last_clean":"2024-09-16T09:30:26.550992+0000","last_became_active":"2024-09-16T09:25:01.706762+0000","last_became_peered":"2024-09-16T09:25:01.706762+0000","last_unstale":"2024-09-16T09:30:26.550992+0000","last_undegraded":"2024-09-16T09:30:26.550992+0000","last_fullsized":"2024-09-16T09:30:26.550992+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:23.796231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:23.796231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:41.024875+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":81,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.581562+0000","last_change":"2024-09-16T09:28:52.983144+0000","last_active":"2024-09-16T09:30:27.581562+0000","last_peered":"2024-09-16T09:30:27.581562+0000","last_clean":"2024-09-16T09:30:27.581562+0000","last_became_active":"2024-09-16T09:28:52.982865+0000","last_became_peered":"2024-09-16T09:28:52.982865+0000","last_unstale":"2024-09-16T09:30:27.581562+0000","last_undegraded":"2024-09-16T09:30:27.581562+0000","last_fullsized":"2024-09-16T09:30:27.581562+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:44.241942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:44.241942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:31:06.834707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":242,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.581316+0000","last_change":"2024-09-16T09:28:52.552567+0000","last_active":"2024-09-16T09:30:27.581316+0000","last_peered":"2024-09-16T09:30:27.581316+0000","last_clean":"2024-09-16T09:30:27.581316+0000","last_became_active":"2024-09-16T09:25:07.715301+0000","last_became_peered":"2024-09-16T09:25:07.715301+0000","last_unstale":"2024-09-16T09:30:27.581316+0000","last_undegraded":"2024-09-16T09:30:27.581316+0000","last_fullsized":"2024-09-16T09:30:27.581316+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:52.552495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:28:52.552495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.14","version":"73'79","reported_seq":31,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.584974+0000","last_change":"2024-09-16T09:30:12.753961+0000","last_active":"2024-09-16T09:30:27.584974+0000","last_peered":"2024-09-16T09:30:27.584974+0000","last_clean":"2024-09-16T09:30:27.584974+0000","last_became_active":"2024-09-16T09:30:10.700866+0000","last_became_peered":"2024-09-16T09:30:10.700866+0000","last_unstale":"2024-09-16T09:30:27.584974+0000","last_undegraded":"2024-09-16T09:30:27.584974+0000","last_fullsized":"2024-09-16T09:30:27.584974+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:53.916505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:53.916505+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:30:41.060729+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109227,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"66'11","reported_seq":126,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-16T09:30:26.551154+0000","last_change":"2024-09-16T09:28:55.643505+0000","last_active":"2024-09-16T09:30:26.551154+0000","last_peered":"2024-09-16T09:30:26.551154+0000","last_clean":"2024-09-16T09:30:26.551154+0000","last_became_active":"2024-09-16T09:28:52.971132+0000","last_became_peered":"2024-09-16T09:28:52.971132+0000","last_unstale":"2024-09-16T09:30:26.551154+0000","last_undegraded":"2024-09-16T09:30:26.551154+0000","last_fullsized":"2024-09-16T09:30:26.551154+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-16T09:28:55.643448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:55.643448+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":19,"num_read_kb":12,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":253,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.581865+0000","last_change":"2024-09-16T09:27:46.175159+0000","last_active":"2024-09-16T09:30:27.581865+0000","last_peered":"2024-09-16T09:30:27.581865+0000","last_clean":"2024-09-16T09:30:27.581865+0000","last_became_active":"2024-09-16T09:25:01.700733+0000","last_became_peered":"2024-09-16T09:25:01.700733+0000","last_unstale":"2024-09-16T09:30:27.581865+0000","last_undegraded":"2024-09-16T09:30:27.581865+0000","last_fullsized":"2024-09-16T09:30:27.581865+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:46.175093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:27:46.175093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:40.495938+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":243,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-16T09:30:26.549020+0000","last_change":"2024-09-16T09:29:25.017736+0000","last_active":"2024-09-16T09:30:26.549020+0000","last_peered":"2024-09-16T09:30:26.549020+0000","last_clean":"2024-09-16T09:30:26.549020+0000","last_became_active":"2024-09-16T09:25:05.713479+0000","last_became_peered":"2024-09-16T09:25:05.713479+0000","last_unstale":"2024-09-16T09:30:26.549020+0000","last_undegraded":"2024-09-16T09:30:26.549020+0000","last_fullsized":"2024-09-16T09:30:26.549020+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:25.017610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:25.017610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:36.761657+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":97,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.581950+0000","last_change":"2024-09-16T09:30:21.524646+0000","last_active":"2024-09-16T09:30:27.581950+0000","last_peered":"2024-09-16T09:30:27.581950+0000","last_clean":"2024-09-16T09:30:27.581950+0000","last_became_active":"2024-09-16T09:30:21.524308+0000","last_became_peered":"2024-09-16T09:30:21.524308+0000","last_unstale":"2024-09-16T09:30:27.581950+0000","last_undegraded":"2024-09-16T09:30:27.581950+0000","last_fullsized":"2024-09-16T09:30:27.581950+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:54.589258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:28:54.589258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,2],"acting":[0,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"66'4","reported_seq":53,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.577204+0000","last_change":"2024-09-16T09:30:21.559461+0000","last_active":"2024-09-16T09:30:27.577204+0000","last_peered":"2024-09-16T09:30:27.577204+0000","last_clean":"2024-09-16T09:30:27.577204+0000","last_became_active":"2024-09-16T09:30:21.538240+0000","last_became_peered":"2024-09-16T09:30:21.538240+0000","last_unstale":"2024-09-16T09:30:27.577204+0000","last_undegraded":"2024-09-16T09:30:27.577204+0000","last_fullsized":"2024-09-16T09:30:27.577204+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'4","last_scrub_stamp":"2024-09-16T09:26:27.473740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:26:27.473740+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":261,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.582009+0000","last_change":"2024-09-16T09:30:21.526452+0000","last_active":"2024-09-16T09:30:27.582009+0000","last_peered":"2024-09-16T09:30:27.582009+0000","last_clean":"2024-09-16T09:30:27.582009+0000","last_became_active":"2024-09-16T09:30:21.525960+0000","last_became_peered":"2024-09-16T09:30:21.525960+0000","last_unstale":"2024-09-16T09:30:27.582009+0000","last_undegraded":"2024-09-16T09:30:27.582009+0000","last_fullsized":"2024-09-16T09:30:27.582009+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:32.638048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:28:32.638048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,6],"acting":[0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":252,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-16T09:30:26.562947+0000","last_change":"2024-09-16T09:29:53.244633+0000","last_active":"2024-09-16T09:30:26.562947+0000","last_peered":"2024-09-16T09:30:26.562947+0000","last_clean":"2024-09-16T09:30:26.562947+0000","last_became_active":"2024-09-16T09:25:05.712545+0000","last_became_peered":"2024-09-16T09:25:05.712545+0000","last_unstale":"2024-09-16T09:30:26.562947+0000","last_undegraded":"2024-09-16T09:30:26.562947+0000","last_fullsized":"2024-09-16T09:30:26.562947+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:53.244578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:53.244578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:31:05.703567+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":239,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-16T09:30:26.550600+0000","last_change":"2024-09-16T09:29:27.012810+0000","last_active":"2024-09-16T09:30:26.550600+0000","last_peered":"2024-09-16T09:30:26.550600+0000","last_clean":"2024-09-16T09:30:26.550600+0000","last_became_active":"2024-09-16T09:25:07.720117+0000","last_became_peered":"2024-09-16T09:25:07.720117+0000","last_unstale":"2024-09-16T09:30:26.550600+0000","last_undegraded":"2024-09-16T09:30:26.550600+0000","last_fullsized":"2024-09-16T09:30:26.550600+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:27.012756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:27.012756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:42.125653+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":242,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.580318+0000","last_change":"2024-09-16T09:29:19.888825+0000","last_active":"2024-09-16T09:30:27.580318+0000","last_peered":"2024-09-16T09:30:27.580318+0000","last_clean":"2024-09-16T09:30:27.580318+0000","last_became_active":"2024-09-16T09:25:07.712594+0000","last_became_peered":"2024-09-16T09:25:07.712594+0000","last_unstale":"2024-09-16T09:30:27.580318+0000","last_undegraded":"2024-09-16T09:30:27.580318+0000","last_fullsized":"2024-09-16T09:30:27.580318+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:19.888759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:19.888759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:30:31.404497+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":88,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.581932+0000","last_change":"2024-09-16T09:30:21.543696+0000","last_active":"2024-09-16T09:30:27.581932+0000","last_peered":"2024-09-16T09:30:27.581932+0000","last_clean":"2024-09-16T09:30:27.581932+0000","last_became_active":"2024-09-16T09:30:21.543315+0000","last_became_peered":"2024-09-16T09:30:21.543315+0000","last_unstale":"2024-09-16T09:30:27.581932+0000","last_undegraded":"2024-09-16T09:30:27.581932+0000","last_fullsized":"2024-09-16T09:30:27.581932+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:32.168204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:28:32.168204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"66'11","reported_seq":267,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-09-16T09:30:26.565948+0000","last_change":"2024-09-16T09:27:57.486459+0000","last_active":"2024-09-16T09:30:26.565948+0000","last_peered":"2024-09-16T09:30:26.565948+0000","last_clean":"2024-09-16T09:30:26.565948+0000","last_became_active":"2024-09-16T09:25:03.707769+0000","last_became_peered":"2024-09-16T09:25:03.707769+0000","last_unstale":"2024-09-16T09:30:26.565948+0000","last_undegraded":"2024-09-16T09:30:26.565948+0000","last_fullsized":"2024-09-16T09:30:26.565948+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-16T09:27:57.486381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:57.486381+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:31:13.387386+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":238,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-09-16T09:30:27.582072+0000","last_change":"2024-09-16T09:27:57.640962+0000","last_active":"2024-09-16T09:30:27.582072+0000","last_peered":"2024-09-16T09:30:27.582072+0000","last_clean":"2024-09-16T09:30:27.582072+0000","last_became_active":"2024-09-16T09:25:05.711952+0000","last_became_peered":"2024-09-16T09:25:05.711952+0000","last_unstale":"2024-09-16T09:30:27.582072+0000","last_undegraded":"2024-09-16T09:30:27.582072+0000","last_fullsized":"2024-09-16T09:30:27.582072+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:57.640756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:57.640756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":44,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":16,"stat_sum":{"num_bytes":3861577728,"num_objects":58923,"num_object_clones":0,"num_object_copies":176769,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":9558,"num_objects_unfound":0,"num_objects_dirty":58923,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":58923,"num_write_kb":3771072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":362,"num_bytes_recovered":23724032,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":11609767936,"data_stored":11609767936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":58923,"ondisk_log_size":58923,"up":48,"acting":45,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":68,"ondisk_log_size":68,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":691,"num_read_kb":448,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":73,"num_bytes_recovered":1678,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":394,"ondisk_log_size":394,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":993,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":320,"num_read_kb":315,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":21,"stat_sum":{"num_bytes":806935,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":21,"num_read_kb":40,"num_write":21,"num_write_kb":296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":370028,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1659,"ondisk_log_size":1659,"up":63,"acting":63,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266758,"num_pgs":69,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1299436,"kb_used_data":1173848,"kb_used_omap":1,"kb_used_meta":125566,"kb_avail":92445716,"statfs":{"total":95995035648,"available":94664413184,"internally_reserved":0,"allocated":1202020352,"data_stored":1201654552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":128580042},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495691,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1290680,"kb_used_data":1173412,"kb_used_omap":1,"kb_used_meta":117246,"kb_avail":92454472,"statfs":{"total":95995035648,"available":94673379328,"internally_reserved":0,"allocated":1201573888,"data_stored":1201196044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":120060358},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724626,"num_pgs":68,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2022372,"kb_used_data":1878740,"kb_used_omap":1,"kb_used_meta":143614,"kb_avail":91722780,"statfs":{"total":95995035648,"available":93924126720,"internally_reserved":0,"allocated":1923829760,"data_stored":1923469074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":147061190},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,3,1,1,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953559,"num_pgs":77,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1330608,"kb_used_data":1191132,"kb_used_omap":1,"kb_used_meta":139454,"kb_avail":92414544,"statfs":{"total":95995035648,"available":94632493056,"internally_reserved":0,"allocated":1219719168,"data_stored":1219349336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":142801350},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149789,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1558592,"kb_used_data":1328876,"kb_used_omap":1,"kb_used_meta":229694,"kb_avail":92186560,"statfs":{"total":95995035648,"available":94399037440,"internally_reserved":0,"allocated":1360769024,"data_stored":1360446289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":235207111},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411426,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1572188,"kb_used_data":1410504,"kb_used_omap":1,"kb_used_meta":161662,"kb_avail":92172964,"statfs":{"total":95995035648,"available":94385115136,"internally_reserved":0,"allocated":1444356096,"data_stored":1444006168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":165542341},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574953,"num_pgs":77,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1777680,"kb_used_data":1607676,"kb_used_omap":1,"kb_used_meta":169982,"kb_avail":91967472,"statfs":{"total":95995035648,"available":94174691328,"internally_reserved":0,"allocated":1646260224,"data_stored":1645857714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":174062021},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738479,"num_pgs":66,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1687812,"kb_used_data":1431724,"kb_used_omap":1,"kb_used_meta":256062,"kb_avail":92057340,"statfs":{"total":95995035648,"available":94266716160,"internally_reserved":0,"allocated":1466085376,"data_stored":1465707747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":262207941},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":1832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1481310208,"data_stored":1481310208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1680867328,"data_stored":1680867328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1479999488,"data_stored":1479999488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1374617600,"data_stored":1374617600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1241120768,"data_stored":1241120768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1933836288,"data_stored":1933836288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1205927936,"data_stored":1205927936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1212088320,"data_stored":1212088320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T09:30:35.895 INFO:tasks.thrashosds.thrasher:Setting 3.1b to [0, 6, 4, 7, 1, 3] 2024-09-16T09:30:35.895 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.1b', '0', '6', '4', '7', '1', '3'] 2024-09-16T09:30:35.895 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd pg-upmap-items 3.1b 0 6 4 7 1 3 2024-09-16T09:30:35.896 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:35.896 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:35.897 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:30:35.956 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:2024-09-16T09:30:35.955995+0000 min lat: 0.00116281 max lat: 2.20004 avg lat: 0.00930569 2024-09-16T09:30:35.956 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-16T09:30:35.956 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 40 16 68769 68753 107.404 78.875 0.00880059 0.00930569 2024-09-16T09:30:35.998 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:30:36.027 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:36.028 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:36.028 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:30:36.129 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:30:36.170 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:36.170 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:36.171 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:30:36.213 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:30:36.213 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:30:36.213 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:30:36.214 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:30:36.214 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:30:36.214 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:30:36.238 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:30:36.238 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:30:36.271 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:30:36.273 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:30:36.273 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:30:36.310 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:30:36.310 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:30:36.325 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:36.326 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:36.326 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:30:36.344 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:30:36.344 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:30:36.375 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:30:36.378 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:30:36.379 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:30:36.427 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:30:36.466 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:36.466 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:36.467 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:30:36.568 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:30:36.602 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:36.602 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:36.603 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:30:36.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:36 smithi084 ceph-mon[32629]: pgmap v256: 168 pgs: 3 active+recovering+undersized+remapped, 165 active+clean; 4.0 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 799 B/s rd, 59 MiB/s wr, 941 op/s; 9166/195441 objects misplaced (4.690%); 3.8 MiB/s, 61 objects/s recovering 2024-09-16T09:30:36.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:36 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/883553781' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:30:36.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:36 smithi070 ceph-mon[30388]: pgmap v256: 168 pgs: 3 active+recovering+undersized+remapped, 165 active+clean; 4.0 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 799 B/s rd, 59 MiB/s wr, 941 op/s; 9166/195441 objects misplaced (4.690%); 3.8 MiB/s, 61 objects/s recovering 2024-09-16T09:30:36.670 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:36 smithi070 ceph-mon[40691]: pgmap v256: 168 pgs: 3 active+recovering+undersized+remapped, 165 active+clean; 4.0 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 799 B/s rd, 59 MiB/s wr, 941 op/s; 9166/195441 objects misplaced (4.690%); 3.8 MiB/s, 61 objects/s recovering 2024-09-16T09:30:36.704 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:30:36.725 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:36.726 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:36.726 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:30:36.827 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:30:36.860 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:36.861 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:36.861 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:30:36.956 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 41 16 69837 69821 106.412 66.75 0.00656031 0.00939058 2024-09-16T09:30:36.963 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:30:37.000 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:37.001 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:37.001 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:30:37.085 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:36 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/883553781' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:30:37.085 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:36 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/883553781' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:30:37.102 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:30:37.123 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-16T09:30:37.132 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:37.133 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:37.133 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:30:37.227 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-09-16T09:30:37.228 INFO:teuthology.orchestra.run.smithi084.stdout: "size": 20, 2024-09-16T09:30:37.228 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 600, 2024-09-16T09:30:37.228 INFO:teuthology.orchestra.run.smithi084.stdout: "ops": [ 2024-09-16T09:30:37.228 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.228 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:22924 7.5 7:a7d40976:::benchmark_data_smithi070_78438_object22923:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T09:30:37.228 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:26:21.580769+0000", 2024-09-16T09:30:37.228 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 255.64340148599999, 2024-09-16T09:30:37.228 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.079602870000000006, 2024-09-16T09:30:37.228 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:30:37.228 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:30:37.229 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:30:37.229 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:30:37.229 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:30:37.229 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:30:37.229 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 22924 2024-09-16T09:30:37.229 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.229 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:30:37.229 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.229 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:30:37.229 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.580769+0000", 2024-09-16T09:30:37.229 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:30:37.229 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.229 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.229 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:30:37.230 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.580769+0000", 2024-09-16T09:30:37.230 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:30:37.230 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.230 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.230 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:30:37.230 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.580770+0000", 2024-09-16T09:30:37.230 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.3200000000000001e-06 2024-09-16T09:30:37.230 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.230 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.230 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:30:37.230 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.580848+0000", 2024-09-16T09:30:37.230 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.7514000000000004e-05 2024-09-16T09:30:37.230 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.230 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.230 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:30:37.231 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.580867+0000", 2024-09-16T09:30:37.231 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.8782000000000001e-05 2024-09-16T09:30:37.231 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.231 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.231 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:30:37.231 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.580874+0000", 2024-09-16T09:30:37.231 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.1269999999999997e-06 2024-09-16T09:30:37.231 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.231 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.231 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:30:37.231 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.580941+0000", 2024-09-16T09:30:37.231 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.7010999999999999e-05 2024-09-16T09:30:37.231 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.231 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.232 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:30:37.232 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.581054+0000", 2024-09-16T09:30:37.232 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000113509 2024-09-16T09:30:37.232 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.232 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.232 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,2", 2024-09-16T09:30:37.232 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.581176+0000", 2024-09-16T09:30:37.232 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000122187 2024-09-16T09:30:37.232 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.232 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.232 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:30:37.232 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.582818+0000", 2024-09-16T09:30:37.232 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0016419200000000001 2024-09-16T09:30:37.232 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.233 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.233 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:30:37.233 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.583304+0000", 2024-09-16T09:30:37.233 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00048596099999999998 2024-09-16T09:30:37.233 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.233 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.233 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:30:37.233 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.660281+0000", 2024-09-16T09:30:37.233 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.076976634000000002 2024-09-16T09:30:37.233 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.233 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.233 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:30:37.233 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.660322+0000", 2024-09-16T09:30:37.234 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.1125999999999999e-05 2024-09-16T09:30:37.234 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.234 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.234 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:30:37.234 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.660372+0000", 2024-09-16T09:30:37.234 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.9778999999999999e-05 2024-09-16T09:30:37.234 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:37.234 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:30:37.234 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:37.234 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.234 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.234 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:23476 7.5 7:ab69eea0:::benchmark_data_smithi070_78438_object23475:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T09:30:37.234 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:26:21.880371+0000", 2024-09-16T09:30:37.234 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 255.34379915299999, 2024-09-16T09:30:37.234 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.069588092000000004, 2024-09-16T09:30:37.235 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:30:37.235 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:30:37.235 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:30:37.235 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:30:37.235 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:30:37.235 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:30:37.235 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 23476 2024-09-16T09:30:37.235 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.235 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:30:37.235 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.235 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:30:37.235 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.880371+0000", 2024-09-16T09:30:37.235 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:30:37.235 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.235 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.236 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:30:37.236 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.880371+0000", 2024-09-16T09:30:37.236 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:30:37.236 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.236 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.236 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:30:37.236 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.880372+0000", 2024-09-16T09:30:37.236 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.4799999999999997e-07 2024-09-16T09:30:37.236 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.236 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.236 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:30:37.236 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.880461+0000", 2024-09-16T09:30:37.236 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.9232000000000002e-05 2024-09-16T09:30:37.236 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.236 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.237 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:30:37.237 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.880473+0000", 2024-09-16T09:30:37.237 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1687e-05 2024-09-16T09:30:37.237 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.237 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.237 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:30:37.237 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.880480+0000", 2024-09-16T09:30:37.237 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.3000000000000004e-06 2024-09-16T09:30:37.237 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.237 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.237 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:30:37.237 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.881388+0000", 2024-09-16T09:30:37.237 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00090752200000000002 2024-09-16T09:30:37.237 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.238 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.238 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:30:37.238 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.881520+0000", 2024-09-16T09:30:37.238 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00013243400000000001 2024-09-16T09:30:37.238 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.238 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.238 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,2", 2024-09-16T09:30:37.238 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.881649+0000", 2024-09-16T09:30:37.238 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00012842299999999999 2024-09-16T09:30:37.238 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.238 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.238 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:30:37.238 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.882506+0000", 2024-09-16T09:30:37.239 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000857047 2024-09-16T09:30:37.239 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.239 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.239 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:30:37.239 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.884941+0000", 2024-09-16T09:30:37.239 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0024354149999999998 2024-09-16T09:30:37.239 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.239 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.239 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:30:37.239 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.949871+0000", 2024-09-16T09:30:37.239 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.064929929999999997 2024-09-16T09:30:37.239 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.239 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.239 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:30:37.239 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.949921+0000", 2024-09-16T09:30:37.240 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.9660999999999998e-05 2024-09-16T09:30:37.240 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.240 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.240 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:30:37.240 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.949959+0000", 2024-09-16T09:30:37.240 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.8692999999999997e-05 2024-09-16T09:30:37.240 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:37.240 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:30:37.240 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:37.240 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.240 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.240 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:23481 7.5 7:a58daa7e:::benchmark_data_smithi070_78438_object23480:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T09:30:37.240 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:26:21.883877+0000", 2024-09-16T09:30:37.241 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 255.340292965, 2024-09-16T09:30:37.241 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.067196770000000003, 2024-09-16T09:30:37.241 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:30:37.241 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:30:37.241 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:30:37.242 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:30:37.242 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:30:37.242 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:30:37.242 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:30:37.242 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 23481 2024-09-16T09:30:37.242 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.242 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:30:37.243 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.243 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:30:37.243 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.883877+0000", 2024-09-16T09:30:37.243 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:30:37.243 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.243 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.243 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:30:37.243 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.883877+0000", 2024-09-16T09:30:37.243 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:30:37.243 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.243 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.243 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:30:37.243 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.883879+0000", 2024-09-16T09:30:37.243 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.144e-06 2024-09-16T09:30:37.244 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.244 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.244 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:30:37.244 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.883931+0000", 2024-09-16T09:30:37.244 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.2123000000000002e-05 2024-09-16T09:30:37.244 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.244 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.244 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:30:37.244 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.883943+0000", 2024-09-16T09:30:37.244 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.2323999999999999e-05 2024-09-16T09:30:37.244 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.244 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.244 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:30:37.245 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.883950+0000", 2024-09-16T09:30:37.245 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.3900000000000004e-06 2024-09-16T09:30:37.245 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.245 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.245 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:30:37.245 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.884006+0000", 2024-09-16T09:30:37.245 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.5387999999999999e-05 2024-09-16T09:30:37.245 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.245 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.245 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:30:37.245 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.884117+0000", 2024-09-16T09:30:37.245 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000111461 2024-09-16T09:30:37.245 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.245 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.246 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,2", 2024-09-16T09:30:37.246 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.884259+0000", 2024-09-16T09:30:37.246 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000141686 2024-09-16T09:30:37.246 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.246 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.246 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:30:37.246 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.885063+0000", 2024-09-16T09:30:37.246 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00080437399999999995 2024-09-16T09:30:37.246 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.246 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.246 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:30:37.246 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.886156+0000", 2024-09-16T09:30:37.246 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0010922130000000001 2024-09-16T09:30:37.246 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.246 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.247 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:30:37.247 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.950978+0000", 2024-09-16T09:30:37.247 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.064822639000000001 2024-09-16T09:30:37.247 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.247 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.247 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:30:37.247 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.951036+0000", 2024-09-16T09:30:37.247 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.7666e-05 2024-09-16T09:30:37.247 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.247 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.247 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:30:37.247 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.951074+0000", 2024-09-16T09:30:37.247 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.8362000000000003e-05 2024-09-16T09:30:37.248 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:37.248 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:30:37.248 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:37.248 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.248 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.248 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_repop(client.24698.0:27466 7.6 e89/82 7:60c44c8b:::benchmark_data_smithi070_78438_object27465:head v 89'1726, mlcod=89'1723)", 2024-09-16T09:30:37.248 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:26:23.722364+0000", 2024-09-16T09:30:37.248 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 253.501806598, 2024-09-16T09:30:37.248 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.071550814000000004, 2024-09-16T09:30:37.248 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:30:37.248 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:30:37.248 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:30:37.248 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:30:37.248 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.249 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:30:37.249 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.722364+0000", 2024-09-16T09:30:37.249 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:30:37.249 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.249 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.249 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:30:37.249 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.722364+0000", 2024-09-16T09:30:37.249 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:30:37.249 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.249 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.249 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:30:37.249 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.722364+0000", 2024-09-16T09:30:37.249 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.0699999999999998e-07 2024-09-16T09:30:37.249 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.250 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.250 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:30:37.250 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.722426+0000", 2024-09-16T09:30:37.250 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.1278999999999994e-05 2024-09-16T09:30:37.250 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.250 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.250 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:30:37.250 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.722432+0000", 2024-09-16T09:30:37.250 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.003e-06 2024-09-16T09:30:37.250 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.250 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.250 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:30:37.250 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.722435+0000", 2024-09-16T09:30:37.250 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.4010000000000001e-06 2024-09-16T09:30:37.251 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.251 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.251 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:30:37.251 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.722461+0000", 2024-09-16T09:30:37.251 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.5894999999999999e-05 2024-09-16T09:30:37.251 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.251 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.251 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:30:37.251 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.722471+0000", 2024-09-16T09:30:37.251 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0251000000000001e-05 2024-09-16T09:30:37.251 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.251 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.251 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:30:37.251 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.793863+0000", 2024-09-16T09:30:37.251 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.071392294999999995 2024-09-16T09:30:37.252 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.252 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.252 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:30:37.252 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.793915+0000", 2024-09-16T09:30:37.252 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.1282999999999997e-05 2024-09-16T09:30:37.252 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:37.252 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:30:37.252 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:37.252 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.252 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.252 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_repop(client.24698.0:27468 7.8 e89/82 7:1de713de:::benchmark_data_smithi070_78438_object27467:head v 89'1721, mlcod=89'1718)", 2024-09-16T09:30:37.252 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:26:23.722887+0000", 2024-09-16T09:30:37.252 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 253.501283578, 2024-09-16T09:30:37.253 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.069612414999999997, 2024-09-16T09:30:37.253 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:30:37.253 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:30:37.253 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:30:37.253 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:30:37.253 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.253 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:30:37.253 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.722887+0000", 2024-09-16T09:30:37.253 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:30:37.253 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.253 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.253 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:30:37.253 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.722887+0000", 2024-09-16T09:30:37.253 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:30:37.254 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.254 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.254 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:30:37.254 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.722887+0000", 2024-09-16T09:30:37.254 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.0100000000000005e-07 2024-09-16T09:30:37.254 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.254 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.254 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:30:37.254 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.722949+0000", 2024-09-16T09:30:37.254 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.1298999999999997e-05 2024-09-16T09:30:37.254 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.254 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.254 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:30:37.254 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.790454+0000", 2024-09-16T09:30:37.255 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.067505342999999995 2024-09-16T09:30:37.255 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.255 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.255 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:30:37.255 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.790465+0000", 2024-09-16T09:30:37.255 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1242e-05 2024-09-16T09:30:37.255 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.255 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.255 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:30:37.255 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.790558+0000", 2024-09-16T09:30:37.255 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.3227999999999996e-05 2024-09-16T09:30:37.255 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.255 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.255 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:30:37.256 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.790578+0000", 2024-09-16T09:30:37.256 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.9114999999999999e-05 2024-09-16T09:30:37.256 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.256 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.256 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:30:37.256 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.792380+0000", 2024-09-16T09:30:37.256 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0018023329999999999 2024-09-16T09:30:37.256 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.256 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.256 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:30:37.256 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.792499+0000", 2024-09-16T09:30:37.256 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000119354 2024-09-16T09:30:37.256 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:37.256 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:30:37.257 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:37.257 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.257 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.257 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:27473 7.5 7:a272c321:::benchmark_data_smithi070_78438_object27472:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T09:30:37.257 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:26:23.723821+0000", 2024-09-16T09:30:37.257 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 253.500348987, 2024-09-16T09:30:37.257 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.073668624000000002, 2024-09-16T09:30:37.257 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:30:37.257 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:30:37.257 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:30:37.257 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:30:37.257 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:30:37.257 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:30:37.257 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 27473 2024-09-16T09:30:37.258 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.258 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:30:37.258 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.258 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:30:37.258 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.723821+0000", 2024-09-16T09:30:37.258 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:30:37.258 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.258 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.258 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:30:37.258 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.723821+0000", 2024-09-16T09:30:37.258 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:30:37.258 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.258 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.258 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:30:37.259 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.723823+0000", 2024-09-16T09:30:37.259 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1999999999999999e-06 2024-09-16T09:30:37.259 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.259 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.259 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:30:37.259 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.723884+0000", 2024-09-16T09:30:37.259 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.0971000000000003e-05 2024-09-16T09:30:37.259 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.259 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.259 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:30:37.259 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.790196+0000", 2024-09-16T09:30:37.259 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.066312372999999994 2024-09-16T09:30:37.259 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.259 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.259 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:30:37.260 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.790213+0000", 2024-09-16T09:30:37.260 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.6974e-05 2024-09-16T09:30:37.260 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.260 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.260 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:30:37.260 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.790559+0000", 2024-09-16T09:30:37.260 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00034600599999999999 2024-09-16T09:30:37.260 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.260 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.260 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:30:37.260 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.790669+0000", 2024-09-16T09:30:37.260 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000110276 2024-09-16T09:30:37.260 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.260 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.261 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,2", 2024-09-16T09:30:37.261 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.790788+0000", 2024-09-16T09:30:37.261 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000119096 2024-09-16T09:30:37.261 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.261 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.261 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:30:37.261 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.793102+0000", 2024-09-16T09:30:37.261 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0023135030000000002 2024-09-16T09:30:37.261 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.261 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.261 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:30:37.261 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.793337+0000", 2024-09-16T09:30:37.261 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000235481 2024-09-16T09:30:37.262 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.262 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.262 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:30:37.262 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.797381+0000", 2024-09-16T09:30:37.262 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0040435699999999998 2024-09-16T09:30:37.262 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.262 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.262 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:30:37.262 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.797446+0000", 2024-09-16T09:30:37.262 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.4759999999999997e-05 2024-09-16T09:30:37.262 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.262 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.262 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:30:37.262 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.797490+0000", 2024-09-16T09:30:37.263 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.4413999999999999e-05 2024-09-16T09:30:37.263 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:37.263 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:30:37.263 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:37.263 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.263 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.263 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_repop(client.24698.0:28810 7.6 e90/82 7:63b6347d:::benchmark_data_smithi070_78438_object28809:head v 90'1809, mlcod=90'1805)", 2024-09-16T09:30:37.263 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:26:24.482328+0000", 2024-09-16T09:30:37.263 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 252.74184214900001, 2024-09-16T09:30:37.263 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.063714949000000007, 2024-09-16T09:30:37.263 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:30:37.263 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:30:37.263 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:30:37.263 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:30:37.264 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.264 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:30:37.264 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.482328+0000", 2024-09-16T09:30:37.264 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:30:37.264 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.264 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.264 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:30:37.264 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.482328+0000", 2024-09-16T09:30:37.264 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:30:37.264 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.264 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.264 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:30:37.264 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.482329+0000", 2024-09-16T09:30:37.265 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.0200000000000002e-07 2024-09-16T09:30:37.265 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.265 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.265 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:30:37.265 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.482394+0000", 2024-09-16T09:30:37.265 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.4706e-05 2024-09-16T09:30:37.265 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.265 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.265 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:30:37.265 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.543399+0000", 2024-09-16T09:30:37.265 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.061005750999999997 2024-09-16T09:30:37.265 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.265 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.265 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:30:37.265 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.543411+0000", 2024-09-16T09:30:37.266 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1687e-05 2024-09-16T09:30:37.266 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.266 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.266 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:30:37.266 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.543539+0000", 2024-09-16T09:30:37.266 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000128298 2024-09-16T09:30:37.266 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.266 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.266 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:30:37.266 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.543558+0000", 2024-09-16T09:30:37.266 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.8323999999999999e-05 2024-09-16T09:30:37.266 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.266 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.266 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:30:37.267 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.546006+0000", 2024-09-16T09:30:37.267 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0024483180000000001 2024-09-16T09:30:37.267 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.267 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.267 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:30:37.267 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.546043+0000", 2024-09-16T09:30:37.267 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.7262999999999999e-05 2024-09-16T09:30:37.267 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:37.267 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:30:37.267 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:37.267 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.267 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.267 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:62033 7.5 7:a5dfc1ea:::benchmark_data_smithi070_78438_object62032:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-16T09:30:37.267 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:26:45.811217+0000", 2024-09-16T09:30:37.268 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 231.41295362299999, 2024-09-16T09:30:37.268 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.071578969000000006, 2024-09-16T09:30:37.268 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:30:37.268 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:30:37.268 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:30:37.268 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:30:37.268 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:30:37.268 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:30:37.268 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 62033 2024-09-16T09:30:37.268 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.268 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:30:37.268 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.268 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:30:37.268 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.811217+0000", 2024-09-16T09:30:37.269 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:30:37.269 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.269 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.269 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:30:37.269 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.811217+0000", 2024-09-16T09:30:37.269 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:30:37.269 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.269 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.269 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:30:37.269 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.811218+0000", 2024-09-16T09:30:37.269 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.5290000000000001e-06 2024-09-16T09:30:37.269 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.269 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.269 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:30:37.269 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.811262+0000", 2024-09-16T09:30:37.270 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.3850999999999997e-05 2024-09-16T09:30:37.271 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.271 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.271 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:30:37.271 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.811278+0000", 2024-09-16T09:30:37.271 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.5753999999999999e-05 2024-09-16T09:30:37.271 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.271 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.271 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:30:37.271 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.811286+0000", 2024-09-16T09:30:37.271 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.7519999999999996e-06 2024-09-16T09:30:37.271 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.271 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.271 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:30:37.272 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.811333+0000", 2024-09-16T09:30:37.272 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.7203000000000002e-05 2024-09-16T09:30:37.272 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.272 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.272 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:30:37.272 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.811456+0000", 2024-09-16T09:30:37.272 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00012353900000000001 2024-09-16T09:30:37.272 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.272 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.272 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,2", 2024-09-16T09:30:37.272 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.811644+0000", 2024-09-16T09:30:37.272 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00018799100000000001 2024-09-16T09:30:37.272 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.272 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.273 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:30:37.273 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.814035+0000", 2024-09-16T09:30:37.273 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0023902009999999998 2024-09-16T09:30:37.273 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.273 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.273 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:30:37.273 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.815646+0000", 2024-09-16T09:30:37.273 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0016112399999999999 2024-09-16T09:30:37.273 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.273 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.273 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:30:37.273 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.882683+0000", 2024-09-16T09:30:37.273 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.067037332000000005 2024-09-16T09:30:37.273 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.274 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.274 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:30:37.274 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.882753+0000", 2024-09-16T09:30:37.274 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.9534000000000002e-05 2024-09-16T09:30:37.274 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.274 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.274 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:30:37.274 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.882796+0000", 2024-09-16T09:30:37.274 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.3043000000000001e-05 2024-09-16T09:30:37.274 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:37.274 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:30:37.274 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:37.274 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.274 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.275 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:62044 7.5 7:aee12d22:::benchmark_data_smithi070_78438_object62043:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-16T09:30:37.275 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:26:45.819031+0000", 2024-09-16T09:30:37.275 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 231.40513976299999, 2024-09-16T09:30:37.275 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.063892420000000005, 2024-09-16T09:30:37.275 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:30:37.275 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:30:37.275 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:30:37.275 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:30:37.275 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:30:37.275 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:30:37.275 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 62044 2024-09-16T09:30:37.275 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.275 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:30:37.276 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.276 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:30:37.276 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.819031+0000", 2024-09-16T09:30:37.276 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:30:37.276 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.276 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.276 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:30:37.276 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.819031+0000", 2024-09-16T09:30:37.276 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:30:37.276 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.276 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.276 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:30:37.276 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.819032+0000", 2024-09-16T09:30:37.276 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.3120000000000001e-06 2024-09-16T09:30:37.276 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.277 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.277 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:30:37.277 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.819071+0000", 2024-09-16T09:30:37.277 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.8599000000000001e-05 2024-09-16T09:30:37.277 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.277 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.277 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:30:37.277 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.819081+0000", 2024-09-16T09:30:37.277 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0812000000000001e-05 2024-09-16T09:30:37.277 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.277 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.277 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:30:37.277 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.819089+0000", 2024-09-16T09:30:37.277 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.8089999999999993e-06 2024-09-16T09:30:37.278 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.278 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.278 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:30:37.278 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.819138+0000", 2024-09-16T09:30:37.278 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.8361e-05 2024-09-16T09:30:37.278 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.278 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.278 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:30:37.278 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.819248+0000", 2024-09-16T09:30:37.278 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000110819 2024-09-16T09:30:37.278 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.278 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.278 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,2", 2024-09-16T09:30:37.278 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.819399+0000", 2024-09-16T09:30:37.279 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000150674 2024-09-16T09:30:37.279 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.279 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.279 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:30:37.279 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.820979+0000", 2024-09-16T09:30:37.279 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0015803589999999999 2024-09-16T09:30:37.279 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.279 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.279 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:30:37.279 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.821238+0000", 2024-09-16T09:30:37.279 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00025873699999999999 2024-09-16T09:30:37.279 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.279 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.279 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:30:37.280 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.882832+0000", 2024-09-16T09:30:37.280 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.061593649 2024-09-16T09:30:37.280 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.280 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.280 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:30:37.280 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.882879+0000", 2024-09-16T09:30:37.280 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.6879000000000003e-05 2024-09-16T09:30:37.280 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.280 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.280 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:30:37.280 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.882923+0000", 2024-09-16T09:30:37.280 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.4409999999999997e-05 2024-09-16T09:30:37.280 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:37.280 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:30:37.281 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:37.281 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.281 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.281 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:63392 7.5 7:af2804a3:::benchmark_data_smithi070_78438_object63391:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-16T09:30:37.281 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:26:46.805497+0000", 2024-09-16T09:30:37.281 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 230.418673633, 2024-09-16T09:30:37.281 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.064734542000000006, 2024-09-16T09:30:37.281 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:30:37.281 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:30:37.281 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:30:37.281 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:30:37.281 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:30:37.281 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:30:37.281 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 63392 2024-09-16T09:30:37.282 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.282 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:30:37.282 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.282 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:30:37.282 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:46.805497+0000", 2024-09-16T09:30:37.282 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:30:37.282 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.282 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.282 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:30:37.282 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:46.805497+0000", 2024-09-16T09:30:37.282 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:30:37.282 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.282 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.282 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:30:37.282 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:46.805498+0000", 2024-09-16T09:30:37.283 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.6789999999999999e-06 2024-09-16T09:30:37.283 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.283 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.283 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:30:37.283 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:46.805542+0000", 2024-09-16T09:30:37.283 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.3123e-05 2024-09-16T09:30:37.283 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.283 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.283 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:30:37.283 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:46.805555+0000", 2024-09-16T09:30:37.283 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.3777e-05 2024-09-16T09:30:37.283 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.283 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.283 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:30:37.283 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:46.805563+0000", 2024-09-16T09:30:37.284 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.4290000000000004e-06 2024-09-16T09:30:37.284 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.284 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.284 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:30:37.284 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:46.805618+0000", 2024-09-16T09:30:37.284 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.5559000000000001e-05 2024-09-16T09:30:37.284 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.284 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.284 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:30:37.284 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:46.805705+0000", 2024-09-16T09:30:37.284 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.7059999999999997e-05 2024-09-16T09:30:37.284 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.284 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.285 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,2", 2024-09-16T09:30:37.285 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:46.805786+0000", 2024-09-16T09:30:37.285 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.0076999999999999e-05 2024-09-16T09:30:37.285 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.285 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.285 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:30:37.285 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:46.807389+0000", 2024-09-16T09:30:37.285 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0016034160000000001 2024-09-16T09:30:37.285 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.285 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.285 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:30:37.285 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:46.807761+0000", 2024-09-16T09:30:37.285 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00037185799999999998 2024-09-16T09:30:37.285 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.286 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.286 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:30:37.286 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:46.870158+0000", 2024-09-16T09:30:37.286 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.062397059999999997 2024-09-16T09:30:37.286 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.286 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.286 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:30:37.286 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:46.870193+0000", 2024-09-16T09:30:37.286 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.4777000000000003e-05 2024-09-16T09:30:37.286 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.286 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.286 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:30:37.286 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:46.870231+0000", 2024-09-16T09:30:37.286 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.8726999999999997e-05 2024-09-16T09:30:37.286 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:37.287 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:30:37.287 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:37.287 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.287 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.287 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:202401 7.5 7:a2fb40f6:::benchmark_data_smithi070_78438_object76270:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135)", 2024-09-16T09:30:37.287 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:28:50.720099+0000", 2024-09-16T09:30:37.287 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 106.504071127, 2024-09-16T09:30:37.287 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.089978474000000003, 2024-09-16T09:30:37.287 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:30:37.287 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:30:37.287 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:30:37.287 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:30:37.287 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:30:37.287 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:30:37.288 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 202401 2024-09-16T09:30:37.288 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.288 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:30:37.288 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.288 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:30:37.288 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.720099+0000", 2024-09-16T09:30:37.288 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:30:37.288 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.288 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.288 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:30:37.288 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.720099+0000", 2024-09-16T09:30:37.288 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:30:37.288 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.288 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.289 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:30:37.289 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.720101+0000", 2024-09-16T09:30:37.289 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.5889999999999999e-06 2024-09-16T09:30:37.289 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.289 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.289 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:30:37.289 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.720107+0000", 2024-09-16T09:30:37.289 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.6520000000000003e-06 2024-09-16T09:30:37.289 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.289 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.289 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:30:37.289 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.720117+0000", 2024-09-16T09:30:37.289 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0254999999999999e-05 2024-09-16T09:30:37.289 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.290 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.290 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:30:37.290 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.720122+0000", 2024-09-16T09:30:37.290 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.2580000000000003e-06 2024-09-16T09:30:37.290 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.290 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.290 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:30:37.290 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.720178+0000", 2024-09-16T09:30:37.290 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.5501000000000003e-05 2024-09-16T09:30:37.290 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.290 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.290 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:30:37.290 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.720290+0000", 2024-09-16T09:30:37.290 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000112837 2024-09-16T09:30:37.291 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.291 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.291 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,2", 2024-09-16T09:30:37.291 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.720394+0000", 2024-09-16T09:30:37.291 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000103613 2024-09-16T09:30:37.291 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.291 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.291 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:30:37.291 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.722666+0000", 2024-09-16T09:30:37.291 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0022723800000000001 2024-09-16T09:30:37.291 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.291 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.291 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:30:37.292 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.725231+0000", 2024-09-16T09:30:37.292 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0025650759999999999 2024-09-16T09:30:37.292 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.292 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.292 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:30:37.292 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.809799+0000", 2024-09-16T09:30:37.292 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.084568014999999996 2024-09-16T09:30:37.292 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.292 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.292 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:30:37.292 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.810033+0000", 2024-09-16T09:30:37.292 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00023360000000000001 2024-09-16T09:30:37.292 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.292 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.292 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:30:37.292 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.810078+0000", 2024-09-16T09:30:37.293 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.4697999999999998e-05 2024-09-16T09:30:37.293 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:37.293 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:30:37.293 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:37.293 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.293 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.293 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:202408 7.5 7:a7ac5683:::benchmark_data_smithi070_78438_object76277:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135)", 2024-09-16T09:30:37.293 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:28:50.725540+0000", 2024-09-16T09:30:37.293 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 106.49863078, 2024-09-16T09:30:37.293 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.090820330000000005, 2024-09-16T09:30:37.293 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:30:37.293 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:30:37.293 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:30:37.293 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:30:37.294 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:30:37.294 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:30:37.294 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 202408 2024-09-16T09:30:37.294 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.294 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:30:37.294 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.294 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:30:37.294 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.725540+0000", 2024-09-16T09:30:37.294 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:30:37.294 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.294 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.294 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:30:37.294 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.725540+0000", 2024-09-16T09:30:37.294 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:30:37.295 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.295 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.295 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:30:37.295 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.725541+0000", 2024-09-16T09:30:37.295 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.5069999999999999e-06 2024-09-16T09:30:37.295 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.295 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.295 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:30:37.295 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.725546+0000", 2024-09-16T09:30:37.295 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.113e-06 2024-09-16T09:30:37.295 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.295 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.295 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:30:37.295 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.725558+0000", 2024-09-16T09:30:37.295 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.2164000000000001e-05 2024-09-16T09:30:37.296 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.296 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.296 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:30:37.296 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.725565+0000", 2024-09-16T09:30:37.296 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.8850000000000001e-06 2024-09-16T09:30:37.296 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.296 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.296 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:30:37.296 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.725617+0000", 2024-09-16T09:30:37.296 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.1848000000000002e-05 2024-09-16T09:30:37.296 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.296 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.296 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:30:37.296 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.725728+0000", 2024-09-16T09:30:37.297 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00011048300000000001 2024-09-16T09:30:37.297 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.297 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.297 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,2", 2024-09-16T09:30:37.297 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.725839+0000", 2024-09-16T09:30:37.297 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00011105200000000001 2024-09-16T09:30:37.297 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.297 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.297 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:30:37.297 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.810121+0000", 2024-09-16T09:30:37.297 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.084282652 2024-09-16T09:30:37.297 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.297 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.298 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:30:37.298 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.816197+0000", 2024-09-16T09:30:37.298 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0060759309999999997 2024-09-16T09:30:37.298 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.298 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.298 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:30:37.298 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.816253+0000", 2024-09-16T09:30:37.298 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.5912999999999998e-05 2024-09-16T09:30:37.298 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.298 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.298 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:30:37.298 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.816308+0000", 2024-09-16T09:30:37.298 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.4511e-05 2024-09-16T09:30:37.298 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.298 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.299 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:30:37.299 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.816360+0000", 2024-09-16T09:30:37.299 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.2271000000000001e-05 2024-09-16T09:30:37.299 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:37.299 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:30:37.299 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:37.299 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.299 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.299 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24424.0:218 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch (1) in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e134)", 2024-09-16T09:30:37.299 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:28:50.729011+0000", 2024-09-16T09:30:37.299 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 106.495159652, 2024-09-16T09:30:37.299 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.078508791999999994, 2024-09-16T09:30:37.299 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:30:37.299 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:30:37.300 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "started", 2024-09-16T09:30:37.300 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:30:37.300 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24424", 2024-09-16T09:30:37.300 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.84:0/1181633158", 2024-09-16T09:30:37.300 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 218 2024-09-16T09:30:37.300 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.300 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:30:37.300 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.300 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:30:37.300 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.729011+0000", 2024-09-16T09:30:37.300 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:30:37.300 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.300 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.300 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:30:37.301 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.729011+0000", 2024-09-16T09:30:37.301 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:30:37.301 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.301 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.301 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:30:37.301 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.729012+0000", 2024-09-16T09:30:37.301 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.6270000000000001e-06 2024-09-16T09:30:37.301 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.301 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.301 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:30:37.301 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.729019+0000", 2024-09-16T09:30:37.301 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.8889999999999996e-06 2024-09-16T09:30:37.301 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.301 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.302 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:30:37.302 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.806924+0000", 2024-09-16T09:30:37.302 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.077904235000000002 2024-09-16T09:30:37.302 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.302 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.302 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:30:37.302 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.806944+0000", 2024-09-16T09:30:37.302 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.039e-05 2024-09-16T09:30:37.302 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.302 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.302 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:30:37.302 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.807345+0000", 2024-09-16T09:30:37.302 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00040116499999999999 2024-09-16T09:30:37.302 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.303 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.303 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:30:37.303 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.807432+0000", 2024-09-16T09:30:37.303 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.6515999999999996e-05 2024-09-16T09:30:37.303 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.303 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.303 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:30:37.303 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.807520+0000", 2024-09-16T09:30:37.303 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.797e-05 2024-09-16T09:30:37.303 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:37.303 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:30:37.303 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:37.303 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.303 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.303 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_repop_reply(client.24698.0:202401 7.5 e135/82 ondisk, result = 0)", 2024-09-16T09:30:37.304 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:28:50.730604+0000", 2024-09-16T09:30:37.304 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 106.49356652199999, 2024-09-16T09:30:37.304 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.079479822000000006, 2024-09-16T09:30:37.304 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:30:37.304 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:30:37.304 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "started", 2024-09-16T09:30:37.304 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:30:37.304 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.304 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:30:37.304 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.730604+0000", 2024-09-16T09:30:37.304 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:30:37.304 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.304 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.305 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:30:37.305 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.730604+0000", 2024-09-16T09:30:37.305 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:30:37.305 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.305 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.305 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:30:37.305 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.730605+0000", 2024-09-16T09:30:37.305 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.0999999999999997e-07 2024-09-16T09:30:37.305 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.305 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.305 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:30:37.305 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.730609+0000", 2024-09-16T09:30:37.305 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.9929999999999997e-06 2024-09-16T09:30:37.305 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.305 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.306 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:30:37.306 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.806968+0000", 2024-09-16T09:30:37.306 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.076359716999999994 2024-09-16T09:30:37.306 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.306 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.306 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:30:37.306 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.806980+0000", 2024-09-16T09:30:37.306 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1551e-05 2024-09-16T09:30:37.306 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.306 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.306 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:30:37.306 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.809786+0000", 2024-09-16T09:30:37.306 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0028062339999999999 2024-09-16T09:30:37.306 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.307 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.307 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:30:37.307 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.809794+0000", 2024-09-16T09:30:37.307 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.4630000000000004e-06 2024-09-16T09:30:37.307 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.307 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.307 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:30:37.307 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.810084+0000", 2024-09-16T09:30:37.307 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00029005399999999998 2024-09-16T09:30:37.307 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:37.307 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:30:37.307 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:37.307 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.307 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.308 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:202411 7.5 7:a7b52676:::benchmark_data_smithi070_78438_object76280:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135)", 2024-09-16T09:30:37.308 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:28:50.732821+0000", 2024-09-16T09:30:37.308 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 106.49134948699999, 2024-09-16T09:30:37.308 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.10429561900000001, 2024-09-16T09:30:37.308 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:30:37.308 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:30:37.308 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:30:37.308 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:30:37.308 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:30:37.308 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:30:37.308 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 202411 2024-09-16T09:30:37.308 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.308 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:30:37.308 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.309 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:30:37.309 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.732821+0000", 2024-09-16T09:30:37.309 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:30:37.309 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.309 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.309 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:30:37.309 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.732821+0000", 2024-09-16T09:30:37.309 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:30:37.309 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.309 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.309 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:30:37.309 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.732822+0000", 2024-09-16T09:30:37.309 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.46e-06 2024-09-16T09:30:37.309 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.309 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.310 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:30:37.310 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.732828+0000", 2024-09-16T09:30:37.310 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.4240000000000001e-06 2024-09-16T09:30:37.310 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.310 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.310 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:30:37.310 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.806990+0000", 2024-09-16T09:30:37.310 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.074162464999999997 2024-09-16T09:30:37.310 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.310 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.310 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:30:37.310 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.806998+0000", 2024-09-16T09:30:37.310 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.0369999999999995e-06 2024-09-16T09:30:37.310 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.310 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.311 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:30:37.311 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.810149+0000", 2024-09-16T09:30:37.311 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0031503339999999999 2024-09-16T09:30:37.311 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.311 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.311 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:30:37.312 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.810253+0000", 2024-09-16T09:30:37.312 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00010388200000000001 2024-09-16T09:30:37.312 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.312 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.313 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,2", 2024-09-16T09:30:37.313 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.810356+0000", 2024-09-16T09:30:37.313 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000103319 2024-09-16T09:30:37.313 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.313 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.313 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:30:37.313 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.816207+0000", 2024-09-16T09:30:37.313 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.005850984 2024-09-16T09:30:37.313 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.313 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.313 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:30:37.313 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.828515+0000", 2024-09-16T09:30:37.313 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.012307904 2024-09-16T09:30:37.313 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.314 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.314 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:30:37.314 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.837029+0000", 2024-09-16T09:30:37.314 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0085145419999999999 2024-09-16T09:30:37.314 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.314 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.314 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:30:37.314 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.837071+0000", 2024-09-16T09:30:37.314 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.1718999999999998e-05 2024-09-16T09:30:37.314 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.314 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.314 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:30:37.314 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.837117+0000", 2024-09-16T09:30:37.314 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.5549e-05 2024-09-16T09:30:37.315 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:37.315 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:30:37.315 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:37.315 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.315 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.315 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:204901 7.5 7:af219aea:::benchmark_data_smithi070_78438_object78770:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137)", 2024-09-16T09:30:37.315 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:28:54.030373+0000", 2024-09-16T09:30:37.315 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 103.193797817, 2024-09-16T09:30:37.315 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.077126564999999994, 2024-09-16T09:30:37.315 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:30:37.315 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:30:37.315 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:30:37.315 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:30:37.315 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:30:37.315 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:30:37.316 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 204901 2024-09-16T09:30:37.316 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.316 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:30:37.316 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.316 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:30:37.316 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.030373+0000", 2024-09-16T09:30:37.316 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:30:37.316 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.316 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.316 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:30:37.316 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.030373+0000", 2024-09-16T09:30:37.316 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:30:37.316 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.316 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.317 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:30:37.317 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.030374+0000", 2024-09-16T09:30:37.317 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.389e-06 2024-09-16T09:30:37.317 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.317 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.317 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:30:37.317 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.030378+0000", 2024-09-16T09:30:37.317 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.1679999999999997e-06 2024-09-16T09:30:37.317 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.317 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.317 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:30:37.317 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.030389+0000", 2024-09-16T09:30:37.317 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.097e-05 2024-09-16T09:30:37.317 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.317 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.318 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:30:37.318 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.030395+0000", 2024-09-16T09:30:37.318 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.229e-06 2024-09-16T09:30:37.318 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.318 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.318 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:30:37.318 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.030446+0000", 2024-09-16T09:30:37.318 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.0757999999999998e-05 2024-09-16T09:30:37.318 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.318 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.318 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:30:37.318 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.030553+0000", 2024-09-16T09:30:37.318 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00010718900000000001 2024-09-16T09:30:37.318 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.319 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.319 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,2", 2024-09-16T09:30:37.319 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.030664+0000", 2024-09-16T09:30:37.319 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00011086600000000001 2024-09-16T09:30:37.319 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.319 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.319 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:30:37.319 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.033351+0000", 2024-09-16T09:30:37.319 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.002686875 2024-09-16T09:30:37.319 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.319 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.319 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:30:37.319 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.033716+0000", 2024-09-16T09:30:37.319 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00036531200000000002 2024-09-16T09:30:37.319 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.320 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.320 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:30:37.320 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.107394+0000", 2024-09-16T09:30:37.320 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.073677379000000001 2024-09-16T09:30:37.320 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.320 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.320 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:30:37.320 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.107450+0000", 2024-09-16T09:30:37.320 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.6351999999999999e-05 2024-09-16T09:30:37.320 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.320 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.320 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:30:37.320 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.107499+0000", 2024-09-16T09:30:37.321 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.9078e-05 2024-09-16T09:30:37.321 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:37.321 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:30:37.321 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:37.321 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.321 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.321 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:204910 7.5 7:a16392ce:::benchmark_data_smithi070_78438_object78779:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137)", 2024-09-16T09:30:37.321 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:28:54.036796+0000", 2024-09-16T09:30:37.321 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 103.18737434000001, 2024-09-16T09:30:37.321 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.070834338999999996, 2024-09-16T09:30:37.321 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:30:37.321 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:30:37.321 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:30:37.321 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:30:37.322 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:30:37.322 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:30:37.322 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 204910 2024-09-16T09:30:37.322 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.322 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:30:37.322 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.322 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:30:37.322 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.036796+0000", 2024-09-16T09:30:37.322 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:30:37.322 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.322 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.322 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:30:37.322 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.036796+0000", 2024-09-16T09:30:37.322 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:30:37.322 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.323 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.323 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:30:37.323 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.036798+0000", 2024-09-16T09:30:37.323 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.649e-06 2024-09-16T09:30:37.323 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.323 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.323 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:30:37.323 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.036805+0000", 2024-09-16T09:30:37.323 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.0840000000000003e-06 2024-09-16T09:30:37.323 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.323 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.323 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:30:37.323 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.036824+0000", 2024-09-16T09:30:37.323 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.9318e-05 2024-09-16T09:30:37.323 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.324 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.324 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:30:37.324 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.036832+0000", 2024-09-16T09:30:37.324 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.4000000000000003e-06 2024-09-16T09:30:37.324 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.324 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.324 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:30:37.324 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.036897+0000", 2024-09-16T09:30:37.324 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.5238000000000001e-05 2024-09-16T09:30:37.324 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.324 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.324 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:30:37.324 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.037029+0000", 2024-09-16T09:30:37.324 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00013190399999999999 2024-09-16T09:30:37.325 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.325 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.325 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,2", 2024-09-16T09:30:37.325 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.037149+0000", 2024-09-16T09:30:37.325 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000119833 2024-09-16T09:30:37.325 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.325 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.325 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:30:37.325 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.039175+0000", 2024-09-16T09:30:37.325 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0020263389999999998 2024-09-16T09:30:37.325 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.325 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.325 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:30:37.326 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.106199+0000", 2024-09-16T09:30:37.326 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.067023694999999994 2024-09-16T09:30:37.326 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.326 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.326 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:30:37.326 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.107546+0000", 2024-09-16T09:30:37.326 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.001347902 2024-09-16T09:30:37.326 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.326 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.326 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:30:37.326 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.107593+0000", 2024-09-16T09:30:37.326 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.6736000000000001e-05 2024-09-16T09:30:37.326 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.326 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.327 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:30:37.327 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.107630+0000", 2024-09-16T09:30:37.327 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.7240999999999998e-05 2024-09-16T09:30:37.327 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:37.327 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:30:37.327 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:37.327 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.327 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.327 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:215340 7.5 7:a4f699d2:::benchmark_data_smithi070_78438_object89209:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e141)", 2024-09-16T09:30:37.327 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:29:03.365151+0000", 2024-09-16T09:30:37.327 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 93.859019519, 2024-09-16T09:30:37.327 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.068201946999999999, 2024-09-16T09:30:37.327 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:30:37.327 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:30:37.328 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:30:37.328 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:30:37.328 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:30:37.328 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:30:37.328 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 215340 2024-09-16T09:30:37.328 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.328 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:30:37.328 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.328 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:30:37.328 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.365151+0000", 2024-09-16T09:30:37.328 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:30:37.328 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.328 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.329 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:30:37.329 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.365151+0000", 2024-09-16T09:30:37.329 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:30:37.329 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.329 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.329 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:30:37.329 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.365153+0000", 2024-09-16T09:30:37.329 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.0540000000000002e-06 2024-09-16T09:30:37.329 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.329 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.329 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:30:37.329 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.365162+0000", 2024-09-16T09:30:37.329 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.38e-06 2024-09-16T09:30:37.329 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.330 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.330 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:30:37.330 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.365186+0000", 2024-09-16T09:30:37.330 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.3986e-05 2024-09-16T09:30:37.330 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.330 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.330 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:30:37.330 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.365200+0000", 2024-09-16T09:30:37.330 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.3586e-05 2024-09-16T09:30:37.330 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.330 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.330 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:30:37.330 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.365312+0000", 2024-09-16T09:30:37.330 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000112422 2024-09-16T09:30:37.331 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.331 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.331 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op must wait for map", 2024-09-16T09:30:37.331 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.365322+0000", 2024-09-16T09:30:37.331 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.5289999999999997e-06 2024-09-16T09:30:37.331 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.331 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.331 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:30:37.331 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.422102+0000", 2024-09-16T09:30:37.331 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.056780023999999998 2024-09-16T09:30:37.331 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.331 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.331 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:30:37.331 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.422202+0000", 2024-09-16T09:30:37.332 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.9617999999999994e-05 2024-09-16T09:30:37.332 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.332 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.332 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,2", 2024-09-16T09:30:37.332 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.422578+0000", 2024-09-16T09:30:37.332 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00037663600000000002 2024-09-16T09:30:37.332 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.332 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.332 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:30:37.332 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.431303+0000", 2024-09-16T09:30:37.332 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0087248670000000007 2024-09-16T09:30:37.332 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.332 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.332 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:30:37.333 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.431404+0000", 2024-09-16T09:30:37.333 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00010138500000000001 2024-09-16T09:30:37.333 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.333 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.333 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:30:37.333 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.433215+0000", 2024-09-16T09:30:37.333 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0018102979999999999 2024-09-16T09:30:37.333 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.333 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.333 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:30:37.333 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.433306+0000", 2024-09-16T09:30:37.333 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.1774999999999995e-05 2024-09-16T09:30:37.333 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.333 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.334 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:30:37.334 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.433353+0000", 2024-09-16T09:30:37.334 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.6387000000000002e-05 2024-09-16T09:30:37.334 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:37.334 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:30:37.334 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:37.334 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.334 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.334 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:215342 7.5 7:a84e8ca2:::benchmark_data_smithi070_78438_object89211:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e141)", 2024-09-16T09:30:37.334 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:29:03.366631+0000", 2024-09-16T09:30:37.334 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 93.857538941000001, 2024-09-16T09:30:37.334 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.074197923999999998, 2024-09-16T09:30:37.334 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:30:37.334 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:30:37.334 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:30:37.335 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:30:37.335 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:30:37.335 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:30:37.335 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 215342 2024-09-16T09:30:37.335 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.335 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:30:37.335 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.335 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:30:37.335 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.366631+0000", 2024-09-16T09:30:37.335 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:30:37.335 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.335 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.335 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:30:37.335 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.366631+0000", 2024-09-16T09:30:37.335 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:30:37.336 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.336 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.336 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:30:37.336 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.366633+0000", 2024-09-16T09:30:37.336 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.2839999999999999e-06 2024-09-16T09:30:37.336 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.336 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.336 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:30:37.336 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.366640+0000", 2024-09-16T09:30:37.336 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.6329999999999996e-06 2024-09-16T09:30:37.336 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.336 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.336 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:30:37.336 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.420402+0000", 2024-09-16T09:30:37.337 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.053761807000000002 2024-09-16T09:30:37.337 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.337 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.337 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:30:37.337 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.420414+0000", 2024-09-16T09:30:37.337 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1885000000000001e-05 2024-09-16T09:30:37.337 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.337 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.337 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:30:37.337 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.426991+0000", 2024-09-16T09:30:37.337 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0065772399999999998 2024-09-16T09:30:37.337 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.337 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.337 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:30:37.338 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.427091+0000", 2024-09-16T09:30:37.338 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.9407999999999994e-05 2024-09-16T09:30:37.338 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.338 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.338 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,2", 2024-09-16T09:30:37.338 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.427202+0000", 2024-09-16T09:30:37.338 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000111298 2024-09-16T09:30:37.338 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.338 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.338 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:30:37.338 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.433361+0000", 2024-09-16T09:30:37.338 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0061589390000000004 2024-09-16T09:30:37.338 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.338 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.339 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:30:37.339 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.437028+0000", 2024-09-16T09:30:37.339 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0036673130000000002 2024-09-16T09:30:37.339 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.339 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.339 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:30:37.339 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.440723+0000", 2024-09-16T09:30:37.339 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0036951480000000001 2024-09-16T09:30:37.339 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.339 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.339 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:30:37.339 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.440791+0000", 2024-09-16T09:30:37.339 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.7496000000000006e-05 2024-09-16T09:30:37.339 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.340 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.340 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:30:37.340 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.440829+0000", 2024-09-16T09:30:37.340 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.8473000000000002e-05 2024-09-16T09:30:37.340 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:37.340 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:30:37.340 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:37.340 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.340 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.340 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:247035 7.5 7:aba136a1:::benchmark_data_smithi070_78438_object120904:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e147)", 2024-09-16T09:30:37.340 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:29:32.642597+0000", 2024-09-16T09:30:37.340 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 64.581573828000003, 2024-09-16T09:30:37.340 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.089290376000000005, 2024-09-16T09:30:37.340 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:30:37.340 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:30:37.341 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:30:37.341 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:30:37.341 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:30:37.341 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:30:37.341 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 247035 2024-09-16T09:30:37.341 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.341 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:30:37.341 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.341 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:30:37.341 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:32.642597+0000", 2024-09-16T09:30:37.341 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:30:37.341 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.341 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.342 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:30:37.342 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:32.642597+0000", 2024-09-16T09:30:37.342 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:30:37.342 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.342 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.342 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:30:37.342 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:32.642598+0000", 2024-09-16T09:30:37.342 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.511e-06 2024-09-16T09:30:37.342 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.342 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.342 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:30:37.342 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:32.642604+0000", 2024-09-16T09:30:37.342 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.6080000000000003e-06 2024-09-16T09:30:37.342 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.343 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.343 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:30:37.343 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:32.642616+0000", 2024-09-16T09:30:37.343 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.2532e-05 2024-09-16T09:30:37.343 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.343 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.343 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:30:37.343 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:32.642622+0000", 2024-09-16T09:30:37.343 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.8869999999999997e-06 2024-09-16T09:30:37.343 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.343 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.343 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:30:37.343 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:32.642686+0000", 2024-09-16T09:30:37.343 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.3368999999999996e-05 2024-09-16T09:30:37.343 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.344 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.344 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op must wait for map", 2024-09-16T09:30:37.344 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:32.642693+0000", 2024-09-16T09:30:37.344 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.2219999999999996e-06 2024-09-16T09:30:37.344 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.344 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.344 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:30:37.344 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:32.644822+0000", 2024-09-16T09:30:37.344 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0021290290000000002 2024-09-16T09:30:37.344 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.344 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.344 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:30:37.344 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:32.644912+0000", 2024-09-16T09:30:37.344 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.0234000000000004e-05 2024-09-16T09:30:37.345 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.345 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.345 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,2", 2024-09-16T09:30:37.345 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:32.644981+0000", 2024-09-16T09:30:37.345 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.8956999999999997e-05 2024-09-16T09:30:37.345 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.345 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.345 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:30:37.345 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:32.647401+0000", 2024-09-16T09:30:37.345 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0024197620000000002 2024-09-16T09:30:37.345 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.345 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.345 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:30:37.345 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:32.723230+0000", 2024-09-16T09:30:37.346 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.075829222000000002 2024-09-16T09:30:37.346 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.346 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.346 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:30:37.346 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:32.731807+0000", 2024-09-16T09:30:37.346 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.008576719 2024-09-16T09:30:37.346 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.346 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.346 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:30:37.346 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:32.731856+0000", 2024-09-16T09:30:37.346 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.9051999999999997e-05 2024-09-16T09:30:37.346 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:30:37.346 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:30:37.346 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:30:37.347 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:32.731887+0000", 2024-09-16T09:30:37.347 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.1272000000000001e-05 2024-09-16T09:30:37.347 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:37.347 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:30:37.347 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:37.347 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:30:37.347 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:30:37.347 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-09-16T09:30:37.348 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:37.348 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:37.348 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:30:37.449 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:30:37.488 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:37.489 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:37.489 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:30:37.590 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:30:37.629 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:37.629 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:37.629 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:30:37.730 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:30:37.765 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:37.766 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:37.766 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:30:37.867 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:30:37.903 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:37.904 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:37.905 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:30:37.956 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 42 16 71022 71006 105.642 74.0625 0.0103686 0.0094601 2024-09-16T09:30:38.005 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:30:38.042 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:38.043 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:38.043 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:30:38.144 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:30:38.179 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:38.180 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:38.180 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:30:38.230 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:30:38.281 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:30:38.318 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:38.319 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:38.319 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:30:38.420 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:30:38.434 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.0 dump_ops_in_flight 2024-09-16T09:30:38.459 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:38.460 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:38.460 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:30:38.561 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:30:38.595 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:38.596 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:38.596 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:30:38.649 INFO:teuthology.orchestra.run.smithi070.stderr:set 3.1b pg_upmap_items mapping to [0->6,4->7,1->3] 2024-09-16T09:30:38.697 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:30:38.747 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:38.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:38.749 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:30:38.850 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:30:38.880 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:38.880 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:38.881 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:30:38.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:38 smithi084 ceph-mon[32629]: pgmap v257: 168 pgs: 3 active+recovering+undersized+remapped, 165 active+clean; 4.0 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.2 KiB/s rd, 53 MiB/s wr, 845 op/s; 9166/197631 objects misplaced (4.638%); 3.1 MiB/s, 48 objects/s recovering 2024-09-16T09:30:38.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:38 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3527555197' entity='client.admin' cmd={"prefix": "osd pg-upmap-items", "pgid": "3.1b", "id": [0, 6, 4, 7, 1, 3]} : dispatch 2024-09-16T09:30:38.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:38 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pg-upmap-items", "pgid": "3.1b", "id": [0, 6, 4, 7, 1, 3]} : dispatch 2024-09-16T09:30:38.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:38 smithi070 ceph-mon[30388]: pgmap v257: 168 pgs: 3 active+recovering+undersized+remapped, 165 active+clean; 4.0 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.2 KiB/s rd, 53 MiB/s wr, 845 op/s; 9166/197631 objects misplaced (4.638%); 3.1 MiB/s, 48 objects/s recovering 2024-09-16T09:30:38.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:38 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3527555197' entity='client.admin' cmd={"prefix": "osd pg-upmap-items", "pgid": "3.1b", "id": [0, 6, 4, 7, 1, 3]} : dispatch 2024-09-16T09:30:38.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:38 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pg-upmap-items", "pgid": "3.1b", "id": [0, 6, 4, 7, 1, 3]} : dispatch 2024-09-16T09:30:38.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:38 smithi070 ceph-mon[40691]: pgmap v257: 168 pgs: 3 active+recovering+undersized+remapped, 165 active+clean; 4.0 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.2 KiB/s rd, 53 MiB/s wr, 845 op/s; 9166/197631 objects misplaced (4.638%); 3.1 MiB/s, 48 objects/s recovering 2024-09-16T09:30:38.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:38 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3527555197' entity='client.admin' cmd={"prefix": "osd pg-upmap-items", "pgid": "3.1b", "id": [0, 6, 4, 7, 1, 3]} : dispatch 2024-09-16T09:30:38.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:38 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pg-upmap-items", "pgid": "3.1b", "id": [0, 6, 4, 7, 1, 3]} : dispatch 2024-09-16T09:30:38.957 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 43 16 72161 72145 104.839 71.1875 0.0098121 0.00953336 2024-09-16T09:30:38.981 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:30:39.027 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:39.027 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:39.028 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:30:39.129 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:30:39.165 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:39.166 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:39.166 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:30:39.267 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:30:39.299 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:39.299 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:39.300 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:30:39.401 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:30:39.434 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:39.435 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:39.435 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:30:39.536 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:30:39.559 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T09:30:39.570 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:39.571 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:39.571 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:30:39.673 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:30:39.692 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:30:39.703 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:39.703 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:39.704 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:30:39.805 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:30:39.834 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:39.835 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:39.835 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:30:39.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:39 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:30:39.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:39 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:30:39.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:39 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pg-upmap-items", "pgid": "3.1b", "id": [0, 6, 4, 7, 1, 3]}]': finished 2024-09-16T09:30:39.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:39 smithi084 ceph-mon[32629]: osdmap e175: 8 total, 8 up, 8 in 2024-09-16T09:30:39.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:39 smithi084 ceph-mon[32629]: osdmap e176: 8 total, 8 up, 8 in 2024-09-16T09:30:39.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:39 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1731174827' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:30:39.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:39 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:30:39.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:39 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:30:39.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:39 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pg-upmap-items", "pgid": "3.1b", "id": [0, 6, 4, 7, 1, 3]}]': finished 2024-09-16T09:30:39.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:39 smithi070 ceph-mon[30388]: osdmap e175: 8 total, 8 up, 8 in 2024-09-16T09:30:39.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:39 smithi070 ceph-mon[30388]: osdmap e176: 8 total, 8 up, 8 in 2024-09-16T09:30:39.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:39 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1731174827' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:30:39.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:39 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:30:39.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:39 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:30:39.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:39 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pg-upmap-items", "pgid": "3.1b", "id": [0, 6, 4, 7, 1, 3]}]': finished 2024-09-16T09:30:39.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:39 smithi070 ceph-mon[40691]: osdmap e175: 8 total, 8 up, 8 in 2024-09-16T09:30:39.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:39 smithi070 ceph-mon[40691]: osdmap e176: 8 total, 8 up, 8 in 2024-09-16T09:30:39.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:39 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1731174827' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:30:39.936 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:30:39.957 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 44 16 73359 73343 104.158 74.875 0.00907534 0.00959488 2024-09-16T09:30:39.962 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:39.962 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:39.963 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:30:40.063 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:30:40.101 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:40.102 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:40.102 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:30:40.197 INFO:teuthology.orchestra.run.smithi070.stderr:nodeep-scrub is set 2024-09-16T09:30:40.203 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:30:40.239 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:40.240 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:40.240 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:30:40.341 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:30:40.378 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:40.379 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:40.379 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:30:40.480 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:30:40.513 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:40.514 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:40.514 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:30:40.615 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:30:40.652 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:40.653 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:40.653 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:30:40.754 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:30:40.789 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:40.789 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:40.790 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:30:40.841 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T09:30:40.841 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [ 2024-09-16T09:30:40.841 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:40.842 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.25064.0:74409 8.8 e177/167)", 2024-09-16T09:30:40.842 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:30:40.837479+0000", 2024-09-16T09:30:40.842 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.0012019159999999999, 2024-09-16T09:30:40.842 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0012239309999999999, 2024-09-16T09:30:40.842 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:30:40.842 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:30:40.842 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T09:30:40.842 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:30:40.842 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:40.842 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:30:40.842 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:30:40.837479+0000", 2024-09-16T09:30:40.842 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:30:40.842 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:40.842 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:40.843 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:30:40.843 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:30:40.837479+0000", 2024-09-16T09:30:40.843 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:30:40.843 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:40.843 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:40.843 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:30:40.843 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:30:40.837480+0000", 2024-09-16T09:30:40.843 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0359999999999999e-06 2024-09-16T09:30:40.843 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:40.843 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:40.843 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:30:40.843 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:30:40.837512+0000", 2024-09-16T09:30:40.843 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1621000000000001e-05 2024-09-16T09:30:40.843 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:40.844 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:40.844 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:30:40.844 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:30:40.837528+0000", 2024-09-16T09:30:40.844 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6121999999999999e-05 2024-09-16T09:30:40.844 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:40.844 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:40.844 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:30:40.844 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:30:40.837535+0000", 2024-09-16T09:30:40.844 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.379e-06 2024-09-16T09:30:40.844 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:30:40.844 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:30:40.845 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:30:40.845 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:40.845 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:40.845 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.25064.0:74390 8.8 e177/167 8:141df075:::benchmark_data_smithi070_107919_object74389:head v 177'4724, mlcod=157'355)", 2024-09-16T09:30:40.845 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:30:40.829511+0000", 2024-09-16T09:30:40.845 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.0091702190000000003, 2024-09-16T09:30:40.845 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0092389459999999996, 2024-09-16T09:30:40.845 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:30:40.845 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:30:40.845 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T09:30:40.845 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:30:40.845 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:40.846 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:30:40.846 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:30:40.829511+0000", 2024-09-16T09:30:40.846 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:30:40.846 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:40.846 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:40.846 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:30:40.846 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:30:40.829511+0000", 2024-09-16T09:30:40.846 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:30:40.846 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:40.846 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:40.847 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:30:40.847 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:30:40.829512+0000", 2024-09-16T09:30:40.847 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.1699999999999997e-07 2024-09-16T09:30:40.847 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:40.847 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:40.847 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:30:40.847 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:30:40.829557+0000", 2024-09-16T09:30:40.847 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5513999999999998e-05 2024-09-16T09:30:40.847 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:40.847 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:40.847 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:30:40.847 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:30:40.829568+0000", 2024-09-16T09:30:40.847 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1005999999999999e-05 2024-09-16T09:30:40.847 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:40.848 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:40.848 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:30:40.848 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:30:40.829576+0000", 2024-09-16T09:30:40.848 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.9799999999999998e-06 2024-09-16T09:30:40.848 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:40.848 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:40.848 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:30:40.848 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:30:40.829653+0000", 2024-09-16T09:30:40.848 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6680000000000004e-05 2024-09-16T09:30:40.848 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:40.848 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:40.848 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:30:40.848 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:30:40.829677+0000", 2024-09-16T09:30:40.849 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.4117e-05 2024-09-16T09:30:40.849 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:30:40.849 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:30:40.849 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:30:40.849 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:40.849 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:40.849 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.25064.0:74398 8.8 e177/167 8:130f04e5:::benchmark_data_smithi070_107919_object74397:head v 177'4725, mlcod=157'355)", 2024-09-16T09:30:40.849 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:30:40.832635+0000", 2024-09-16T09:30:40.849 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.0060461050000000004, 2024-09-16T09:30:40.849 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0061631239999999999, 2024-09-16T09:30:40.849 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:30:40.849 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:30:40.849 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T09:30:40.849 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:30:40.850 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:40.850 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:30:40.850 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:30:40.832635+0000", 2024-09-16T09:30:40.850 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:30:40.850 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:40.850 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:40.850 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:30:40.850 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:30:40.832635+0000", 2024-09-16T09:30:40.850 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:30:40.850 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:40.850 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:40.850 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:30:40.850 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:30:40.832636+0000", 2024-09-16T09:30:40.850 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.6600000000000005e-07 2024-09-16T09:30:40.851 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:40.851 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:40.851 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:30:40.851 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:30:40.832680+0000", 2024-09-16T09:30:40.851 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.4619000000000001e-05 2024-09-16T09:30:40.851 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:40.851 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:40.851 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:30:40.851 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:30:40.832691+0000", 2024-09-16T09:30:40.851 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0197000000000001e-05 2024-09-16T09:30:40.851 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:40.851 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:40.851 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:30:40.851 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:30:40.832697+0000", 2024-09-16T09:30:40.852 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.3899999999999998e-06 2024-09-16T09:30:40.852 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:40.852 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:40.852 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:30:40.852 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:30:40.838100+0000", 2024-09-16T09:30:40.852 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0054034249999999999 2024-09-16T09:30:40.852 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:40.852 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:40.852 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:30:40.852 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:30:40.838125+0000", 2024-09-16T09:30:40.852 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.4501000000000001e-05 2024-09-16T09:30:40.852 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:30:40.852 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:30:40.852 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:30:40.853 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:40.853 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:40.853 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.25064.0:74410 8.c 8:340cc17b:::benchmark_data_smithi070_107919_object74409:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e177)", 2024-09-16T09:30:40.853 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:30:40.833795+0000", 2024-09-16T09:30:40.853 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.004886334, 2024-09-16T09:30:40.853 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0050581310000000004, 2024-09-16T09:30:40.853 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:30:40.853 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:30:40.853 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "waiting for sub ops", 2024-09-16T09:30:40.853 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:30:40.853 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.25064", 2024-09-16T09:30:40.853 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/2578256439", 2024-09-16T09:30:40.853 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 74410 2024-09-16T09:30:40.853 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:40.854 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:30:40.854 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:40.854 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:30:40.854 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:30:40.833795+0000", 2024-09-16T09:30:40.854 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:30:40.854 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:40.854 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:40.854 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:30:40.854 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:30:40.833795+0000", 2024-09-16T09:30:40.854 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:30:40.854 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:40.854 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:40.854 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:30:40.854 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:30:40.833796+0000", 2024-09-16T09:30:40.854 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.9900000000000009e-07 2024-09-16T09:30:40.855 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:40.855 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:40.855 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:30:40.855 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:30:40.833813+0000", 2024-09-16T09:30:40.855 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6914000000000001e-05 2024-09-16T09:30:40.855 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:40.855 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:40.855 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:30:40.855 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:30:40.833824+0000", 2024-09-16T09:30:40.855 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1898e-05 2024-09-16T09:30:40.855 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:40.855 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:40.855 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:30:40.855 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:30:40.833830+0000", 2024-09-16T09:30:40.855 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.9669999999999998e-06 2024-09-16T09:30:40.856 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:40.856 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:40.856 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:30:40.856 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:30:40.833873+0000", 2024-09-16T09:30:40.856 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2120999999999999e-05 2024-09-16T09:30:40.856 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:40.856 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:40.856 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:30:40.856 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:30:40.834099+0000", 2024-09-16T09:30:40.856 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00022668800000000001 2024-09-16T09:30:40.856 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:40.856 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:40.856 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 3,7", 2024-09-16T09:30:40.857 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:30:40.834364+0000", 2024-09-16T09:30:40.857 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00026489299999999999 2024-09-16T09:30:40.857 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:40.857 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:40.857 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:30:40.857 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:30:40.837648+0000", 2024-09-16T09:30:40.857 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0032841810000000002 2024-09-16T09:30:40.857 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:40.857 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:40.857 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:30:40.857 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:30:40.837845+0000", 2024-09-16T09:30:40.857 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00019693600000000001 2024-09-16T09:30:40.857 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:30:40.857 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:30:40.857 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:30:40.858 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:40.858 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:40.858 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.25064.0:74411 8.e 8:7a919a87:::benchmark_data_smithi070_107919_object74410:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e177)", 2024-09-16T09:30:40.858 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:30:40.834140+0000", 2024-09-16T09:30:40.858 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.0045410210000000001, 2024-09-16T09:30:40.858 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0047865939999999999, 2024-09-16T09:30:40.858 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:30:40.858 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:30:40.858 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "waiting for sub ops", 2024-09-16T09:30:40.858 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:30:40.858 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.25064", 2024-09-16T09:30:40.858 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/2578256439", 2024-09-16T09:30:40.858 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 74411 2024-09-16T09:30:40.858 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:40.859 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:30:40.859 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:40.859 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:30:40.859 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:30:40.834140+0000", 2024-09-16T09:30:40.859 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:30:40.859 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:40.859 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:40.859 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:30:40.859 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:30:40.834140+0000", 2024-09-16T09:30:40.859 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:30:40.859 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:40.859 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:40.859 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:30:40.859 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:30:40.834141+0000", 2024-09-16T09:30:40.859 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.4900000000000005e-07 2024-09-16T09:30:40.860 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:40.860 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:40.860 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:30:40.860 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:30:40.834162+0000", 2024-09-16T09:30:40.860 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.1137000000000001e-05 2024-09-16T09:30:40.860 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:40.860 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:40.860 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:30:40.860 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:30:40.834172+0000", 2024-09-16T09:30:40.860 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0362e-05 2024-09-16T09:30:40.860 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:40.860 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:40.860 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:30:40.860 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:30:40.834180+0000", 2024-09-16T09:30:40.861 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.5619999999999998e-06 2024-09-16T09:30:40.861 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:40.861 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:40.861 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:30:40.861 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:30:40.834254+0000", 2024-09-16T09:30:40.861 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.4578000000000003e-05 2024-09-16T09:30:40.861 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:40.861 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:40.861 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:30:40.861 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:30:40.834400+0000", 2024-09-16T09:30:40.861 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00014587399999999999 2024-09-16T09:30:40.861 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:40.861 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:40.862 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 2,4", 2024-09-16T09:30:40.862 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:30:40.834543+0000", 2024-09-16T09:30:40.862 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000142523 2024-09-16T09:30:40.862 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:30:40.862 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:30:40.862 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:30:40.862 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:30:40.837641+0000", 2024-09-16T09:30:40.862 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0030977119999999999 2024-09-16T09:30:40.862 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:30:40.862 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:30:40.862 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:30:40.862 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:30:40.862 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-16T09:30:40.862 INFO:teuthology.orchestra.run.smithi070.stdout: "num_ops": 5 2024-09-16T09:30:40.862 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T09:30:40.891 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:30:40.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:40 smithi084 ceph-mon[32629]: pgmap v260: 168 pgs: 3 active+recovering+undersized+remapped, 165 active+clean; 4.3 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 853 B/s rd, 61 MiB/s wr, 973 op/s; 8735/212949 objects misplaced (4.102%); 4.5 MiB/s, 71 objects/s recovering 2024-09-16T09:30:40.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:40 smithi084 ceph-mon[32629]: 1.3 scrub starts 2024-09-16T09:30:40.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:40 smithi084 ceph-mon[32629]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:30:40.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:40 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1731174827' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-16T09:30:40.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:40 smithi084 ceph-mon[32629]: osdmap e177: 8 total, 8 up, 8 in 2024-09-16T09:30:40.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:40 smithi070 ceph-mon[30388]: pgmap v260: 168 pgs: 3 active+recovering+undersized+remapped, 165 active+clean; 4.3 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 853 B/s rd, 61 MiB/s wr, 973 op/s; 8735/212949 objects misplaced (4.102%); 4.5 MiB/s, 71 objects/s recovering 2024-09-16T09:30:40.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:40 smithi070 ceph-mon[30388]: 1.3 scrub starts 2024-09-16T09:30:40.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:40 smithi070 ceph-mon[30388]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:30:40.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:40 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1731174827' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-16T09:30:40.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:40 smithi070 ceph-mon[30388]: osdmap e177: 8 total, 8 up, 8 in 2024-09-16T09:30:40.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:40 smithi070 ceph-mon[40691]: pgmap v260: 168 pgs: 3 active+recovering+undersized+remapped, 165 active+clean; 4.3 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 853 B/s rd, 61 MiB/s wr, 973 op/s; 8735/212949 objects misplaced (4.102%); 4.5 MiB/s, 71 objects/s recovering 2024-09-16T09:30:40.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:40 smithi070 ceph-mon[40691]: 1.3 scrub starts 2024-09-16T09:30:40.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:40 smithi070 ceph-mon[40691]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:30:40.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:40 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1731174827' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-16T09:30:40.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:40 smithi070 ceph-mon[40691]: osdmap e177: 8 total, 8 up, 8 in 2024-09-16T09:30:40.926 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:40.927 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:40.927 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:30:40.957 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 45 16 74559 74543 103.51 75 0.0196519 0.00965574 2024-09-16T09:30:41.027 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:30:41.063 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:41.064 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:41.064 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:30:41.165 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:30:41.202 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:41.203 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:41.203 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:30:41.304 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:30:41.338 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:41.339 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:41.339 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:30:41.440 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:30:41.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:41.472 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:41.472 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:30:41.573 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:30:41.608 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:41.608 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:41.609 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:30:41.710 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:30:41.745 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:41.745 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:41.746 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:30:41.842 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.0 dump_blocked_ops 2024-09-16T09:30:41.847 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:30:41.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:41 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:30:41.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:41 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:30:41.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:41 smithi084 ceph-mon[32629]: 1.5 scrub starts 2024-09-16T09:30:41.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:41 smithi084 ceph-mon[32629]: 1.2 scrub starts 2024-09-16T09:30:41.919 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:41.920 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:41.921 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:30:41.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:41 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:30:41.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:41 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:30:41.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:41 smithi070 ceph-mon[30388]: 1.5 scrub starts 2024-09-16T09:30:41.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:41 smithi070 ceph-mon[30388]: 1.2 scrub starts 2024-09-16T09:30:41.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:41 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:30:41.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:41 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:30:41.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:41 smithi070 ceph-mon[40691]: 1.5 scrub starts 2024-09-16T09:30:41.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:41 smithi070 ceph-mon[40691]: 1.2 scrub starts 2024-09-16T09:30:41.957 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 46 16 75872 75856 103.044 82.0625 0.00795022 0.00969901 2024-09-16T09:30:42.022 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:30:42.063 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:42.064 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:42.065 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:30:42.165 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:30:42.201 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:42.201 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:42.202 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:30:42.228 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:30:41 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:30:41] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:30:42.243 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:30:42.302 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:30:42.342 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:42.343 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:42.343 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:30:42.444 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:30:42.481 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:42.482 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:42.482 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:30:42.559 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:30:42.583 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:30:42.626 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:42.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:42.627 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:30:42.728 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:30:42.763 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:42.763 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:42.764 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:30:42.856 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:42 smithi070 ceph-mon[30388]: 1.1 scrub starts 2024-09-16T09:30:42.856 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:42 smithi070 ceph-mon[30388]: pgmap v262: 168 pgs: 3 active+recovering+undersized+remapped, 165 active+clean; 4.3 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 853 B/s rd, 61 MiB/s wr, 973 op/s; 8735/212949 objects misplaced (4.102%); 4.5 MiB/s, 71 objects/s recovering 2024-09-16T09:30:42.856 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:42 smithi070 ceph-mon[30388]: 1.e scrub starts 2024-09-16T09:30:42.856 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:42 smithi070 ceph-mon[30388]: 1.14 scrub starts 2024-09-16T09:30:42.857 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:42 smithi070 ceph-mon[30388]: 1.b scrub starts 2024-09-16T09:30:42.857 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:42 smithi070 ceph-mon[30388]: 1.0 scrub starts 2024-09-16T09:30:42.857 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:42 smithi070 ceph-mon[40691]: 1.1 scrub starts 2024-09-16T09:30:42.857 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:42 smithi070 ceph-mon[40691]: pgmap v262: 168 pgs: 3 active+recovering+undersized+remapped, 165 active+clean; 4.3 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 853 B/s rd, 61 MiB/s wr, 973 op/s; 8735/212949 objects misplaced (4.102%); 4.5 MiB/s, 71 objects/s recovering 2024-09-16T09:30:42.857 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:42 smithi070 ceph-mon[40691]: 1.e scrub starts 2024-09-16T09:30:42.857 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:42 smithi070 ceph-mon[40691]: 1.14 scrub starts 2024-09-16T09:30:42.857 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:42 smithi070 ceph-mon[40691]: 1.b scrub starts 2024-09-16T09:30:42.857 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:42 smithi070 ceph-mon[40691]: 1.0 scrub starts 2024-09-16T09:30:42.865 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:30:42.899 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:42.899 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:42.900 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:30:42.957 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 47 16 77097 77081 102.48 76.5625 0.0210732 0.00975258 2024-09-16T09:30:43.001 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:30:43.045 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:43.046 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:43.046 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:30:43.147 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:30:43.160 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:30:42 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:30:43.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:42 smithi084 ceph-mon[32629]: 1.1 scrub starts 2024-09-16T09:30:43.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:42 smithi084 ceph-mon[32629]: pgmap v262: 168 pgs: 3 active+recovering+undersized+remapped, 165 active+clean; 4.3 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 853 B/s rd, 61 MiB/s wr, 973 op/s; 8735/212949 objects misplaced (4.102%); 4.5 MiB/s, 71 objects/s recovering 2024-09-16T09:30:43.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:42 smithi084 ceph-mon[32629]: 1.e scrub starts 2024-09-16T09:30:43.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:42 smithi084 ceph-mon[32629]: 1.14 scrub starts 2024-09-16T09:30:43.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:42 smithi084 ceph-mon[32629]: 1.b scrub starts 2024-09-16T09:30:43.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:42 smithi084 ceph-mon[32629]: 1.0 scrub starts 2024-09-16T09:30:43.181 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:43.182 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:43.182 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:30:43.221 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T09:30:43.221 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:30:43.282 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:30:43.298 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:30:43.298 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:30:43.299 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:30:43.299 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:30:43.299 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:30:43.299 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:30:43.299 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:30:43.300 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:30:43.301 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:30:43.301 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:30:43.315 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:43.316 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:43.316 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:30:43.344 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:30:43.345 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:30:43.384 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:30:43.384 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:30:43.416 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:30:43.416 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:30:43.417 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:30:43.453 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:43.454 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:43.454 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:30:43.555 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:30:43.596 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:43.597 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:43.597 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:30:43.698 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:30:43.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:43.734 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:43.734 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:30:43.835 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:30:43.873 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:43.874 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:43.874 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:30:43.957 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 48 16 78337 78321 101.96 77.5 0.0176404 0.00980259 2024-09-16T09:30:43.976 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:30:44.007 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:44.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:44.008 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:30:44.112 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:30:44.146 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:44.147 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:44.147 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:30:44.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:43 smithi084 ceph-mon[32629]: 1.10 scrub starts 2024-09-16T09:30:44.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:43 smithi084 ceph-mon[32629]: 1.f scrub starts 2024-09-16T09:30:44.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:43 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:30:44.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:43 smithi084 ceph-mon[32629]: 1.d scrub starts 2024-09-16T09:30:44.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:43 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1294592562' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:30:44.170 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:43 smithi070 ceph-mon[30388]: 1.10 scrub starts 2024-09-16T09:30:44.170 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:43 smithi070 ceph-mon[30388]: 1.f scrub starts 2024-09-16T09:30:44.170 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:43 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:30:44.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:43 smithi070 ceph-mon[30388]: 1.d scrub starts 2024-09-16T09:30:44.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:43 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1294592562' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:30:44.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:43 smithi070 ceph-mon[40691]: 1.10 scrub starts 2024-09-16T09:30:44.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:43 smithi070 ceph-mon[40691]: 1.f scrub starts 2024-09-16T09:30:44.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:43 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:30:44.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:43 smithi070 ceph-mon[40691]: 1.d scrub starts 2024-09-16T09:30:44.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:43 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1294592562' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:30:44.234 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd unset noscrub 2024-09-16T09:30:44.248 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:30:44.281 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:44.282 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:44.282 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:30:44.383 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:30:44.422 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:44.422 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:44.423 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:30:44.476 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.0 dump_historic_ops 2024-09-16T09:30:44.524 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:30:44.579 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:44.580 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:44.580 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:30:44.673 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T09:30:44.674 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd reweight 0 1 2024-09-16T09:30:44.680 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:30:44.774 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:44.775 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:44.775 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:30:44.865 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:44 smithi070 ceph-mon[30388]: 1.12 scrub starts 2024-09-16T09:30:44.865 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:44 smithi070 ceph-mon[30388]: pgmap v263: 168 pgs: 8 active+clean+scrubbing, 3 active+recovering+undersized+remapped, 157 active+clean; 4.7 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 853 B/s rd, 116 MiB/s wr, 1.86k op/s; 8265/231099 objects misplaced (3.576%); 9.4 MiB/s, 150 objects/s recovering 2024-09-16T09:30:44.865 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:44 smithi070 ceph-mon[30388]: osdmap e178: 8 total, 8 up, 8 in 2024-09-16T09:30:44.865 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:44 smithi070 ceph-mon[40691]: 1.12 scrub starts 2024-09-16T09:30:44.865 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:44 smithi070 ceph-mon[40691]: pgmap v263: 168 pgs: 8 active+clean+scrubbing, 3 active+recovering+undersized+remapped, 157 active+clean; 4.7 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 853 B/s rd, 116 MiB/s wr, 1.86k op/s; 8265/231099 objects misplaced (3.576%); 9.4 MiB/s, 150 objects/s recovering 2024-09-16T09:30:44.865 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:44 smithi070 ceph-mon[40691]: osdmap e178: 8 total, 8 up, 8 in 2024-09-16T09:30:44.876 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:30:44.903 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:44.903 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:44.904 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:30:44.957 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 49 16 79490 79474 101.35 72.0625 0.0132503 0.00986141 2024-09-16T09:30:45.004 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:30:45.019 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:30:45.037 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:30:45.038 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:45.039 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:45.039 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:30:45.060 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:44 smithi084 ceph-mon[32629]: 1.12 scrub starts 2024-09-16T09:30:45.061 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:44 smithi084 ceph-mon[32629]: pgmap v263: 168 pgs: 8 active+clean+scrubbing, 3 active+recovering+undersized+remapped, 157 active+clean; 4.7 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 853 B/s rd, 116 MiB/s wr, 1.86k op/s; 8265/231099 objects misplaced (3.576%); 9.4 MiB/s, 150 objects/s recovering 2024-09-16T09:30:45.061 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:44 smithi084 ceph-mon[32629]: osdmap e178: 8 total, 8 up, 8 in 2024-09-16T09:30:45.140 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:30:45.179 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:45.179 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:45.180 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:30:45.280 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:30:45.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:45.321 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:45.321 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:30:45.422 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:30:45.450 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:45.451 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:45.451 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:30:45.552 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:30:45.588 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:45.589 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:45.589 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:30:45.690 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:30:45.714 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T09:30:45.714 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:30:45.726 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:45.727 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:45.727 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:30:45.828 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:30:45.861 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:45.861 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:45.862 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:30:45.957 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 50 16 80645 80629 100.766 72.1875 0.0127097 0.00991886 2024-09-16T09:30:45.963 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:30:46.004 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:46.005 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:46.005 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:30:46.106 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:30:46.141 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:46.142 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:46.142 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:30:46.243 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:30:46.272 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:46.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:46.273 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:30:46.374 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:30:46.414 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:46.414 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:46.415 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:30:46.515 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:30:46.558 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:46.559 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:46.559 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:30:46.660 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:30:46.693 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.1 dump_ops_in_flight 2024-09-16T09:30:46.717 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:46.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:46.718 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:30:46.819 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:30:46.859 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:46.859 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:46.860 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:30:46.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:46 smithi070 ceph-mon[30388]: pgmap v265: 168 pgs: 8 active+clean+scrubbing, 3 active+recovering+undersized+remapped, 157 active+clean; 4.7 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 799 B/s rd, 59 MiB/s wr, 945 op/s; 8265/231099 objects misplaced (3.576%); 4.6 MiB/s, 73 objects/s recovering 2024-09-16T09:30:46.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:46 smithi070 ceph-mon[40691]: pgmap v265: 168 pgs: 8 active+clean+scrubbing, 3 active+recovering+undersized+remapped, 157 active+clean; 4.7 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 799 B/s rd, 59 MiB/s wr, 945 op/s; 8265/231099 objects misplaced (3.576%); 4.6 MiB/s, 73 objects/s recovering 2024-09-16T09:30:46.933 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T09:30:46.959 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 51 16 81763 81747 100.159 69.875 0.0141935 0.00997782 2024-09-16T09:30:46.960 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:30:46.970 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:30:47.024 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:47.025 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:47.025 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:30:47.126 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:30:47.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:46 smithi084 ceph-mon[32629]: pgmap v265: 168 pgs: 8 active+clean+scrubbing, 3 active+recovering+undersized+remapped, 157 active+clean; 4.7 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 799 B/s rd, 59 MiB/s wr, 945 op/s; 8265/231099 objects misplaced (3.576%); 4.6 MiB/s, 73 objects/s recovering 2024-09-16T09:30:47.164 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:47.165 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:47.166 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:30:47.267 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:30:47.330 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:47.331 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:47.331 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:30:47.432 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:30:47.469 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:47.469 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:47.469 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:30:47.570 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:30:47.593 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:47.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:47.595 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:30:47.694 INFO:teuthology.orchestra.run.smithi070.stderr:noscrub is unset 2024-09-16T09:30:47.696 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:30:47.743 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:47.744 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:47.744 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:30:47.845 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:30:47.886 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:47.887 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:47.887 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:30:47.959 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 52 16 82839 82823 99.5258 67.25 0.0217826 0.0100425 2024-09-16T09:30:47.988 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:30:48.023 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:48.023 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:48.023 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:30:48.104 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:30:48.125 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:30:48.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:47 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1328717453' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:30:48.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:47 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:30:48.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:47 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:30:48.170 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:47 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1328717453' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:30:48.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:47 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:30:48.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:47 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:30:48.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:47 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1328717453' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:30:48.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:47 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:30:48.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:47 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:30:48.173 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:48.174 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:48.175 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:30:48.276 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:30:48.308 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:48.309 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:48.309 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:30:48.410 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:30:48.447 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:48.448 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:48.448 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:30:48.549 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:30:48.582 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:48.583 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:48.583 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:30:48.660 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:30:48.685 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:30:48.710 INFO:teuthology.orchestra.run.smithi070.stderr:reweighted osd.0 to 1 (10000) 2024-09-16T09:30:48.732 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:48.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:48.733 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:30:48.834 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:30:48.870 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:48.871 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:48.871 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:30:48.907 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:48 smithi070 ceph-mon[30388]: pgmap v266: 168 pgs: 6 active+clean+scrubbing, 3 active+recovering+undersized+remapped, 159 active+clean; 4.7 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 1.2 KiB/s rd, 53 MiB/s wr, 851 op/s; 8265/233367 objects misplaced (3.542%); 3.7 MiB/s, 58 objects/s recovering 2024-09-16T09:30:48.907 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:48 smithi070 ceph-mon[30388]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:30:48.907 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:48 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1328717453' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-16T09:30:48.907 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:48 smithi070 ceph-mon[30388]: osdmap e179: 8 total, 8 up, 8 in 2024-09-16T09:30:48.907 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:48 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3006191733' entity='client.admin' cmd={"prefix": "osd reweight", "id": 0, "weight": 1.0} : dispatch 2024-09-16T09:30:48.907 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:48 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 0, "weight": 1.0} : dispatch 2024-09-16T09:30:48.908 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:48 smithi070 ceph-mon[40691]: pgmap v266: 168 pgs: 6 active+clean+scrubbing, 3 active+recovering+undersized+remapped, 159 active+clean; 4.7 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 1.2 KiB/s rd, 53 MiB/s wr, 851 op/s; 8265/233367 objects misplaced (3.542%); 3.7 MiB/s, 58 objects/s recovering 2024-09-16T09:30:48.908 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:48 smithi070 ceph-mon[40691]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:30:48.908 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:48 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1328717453' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-16T09:30:48.908 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:48 smithi070 ceph-mon[40691]: osdmap e179: 8 total, 8 up, 8 in 2024-09-16T09:30:48.908 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:48 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3006191733' entity='client.admin' cmd={"prefix": "osd reweight", "id": 0, "weight": 1.0} : dispatch 2024-09-16T09:30:48.908 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:48 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 0, "weight": 1.0} : dispatch 2024-09-16T09:30:48.959 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 53 16 84058 84042 99.0854 76.1875 0.012798 0.0100876 2024-09-16T09:30:48.972 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:30:49.005 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:49.005 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:49.006 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:30:49.106 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:30:49.137 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:49.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:49.138 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:30:49.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:48 smithi084 ceph-mon[32629]: pgmap v266: 168 pgs: 6 active+clean+scrubbing, 3 active+recovering+undersized+remapped, 159 active+clean; 4.7 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 1.2 KiB/s rd, 53 MiB/s wr, 851 op/s; 8265/233367 objects misplaced (3.542%); 3.7 MiB/s, 58 objects/s recovering 2024-09-16T09:30:49.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:48 smithi084 ceph-mon[32629]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:30:49.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:48 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1328717453' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-16T09:30:49.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:48 smithi084 ceph-mon[32629]: osdmap e179: 8 total, 8 up, 8 in 2024-09-16T09:30:49.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:48 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3006191733' entity='client.admin' cmd={"prefix": "osd reweight", "id": 0, "weight": 1.0} : dispatch 2024-09-16T09:30:49.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:48 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 0, "weight": 1.0} : dispatch 2024-09-16T09:30:49.239 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:30:49.274 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:49.275 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:49.275 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:30:49.376 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:30:49.411 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:49.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:49.412 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:30:49.513 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:30:49.553 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:49.553 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:49.554 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:30:49.655 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:30:49.697 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:49.698 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:49.698 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:30:49.779 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T09:30:49.779 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:30:49.799 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:30:49.835 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:49.836 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:49.836 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:30:49.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:49 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-16T09:30:49.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:49 smithi070 ceph-mon[30388]: osdmap e180: 8 total, 8 up, 8 in 2024-09-16T09:30:49.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:49 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-16T09:30:49.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:49 smithi070 ceph-mon[40691]: osdmap e180: 8 total, 8 up, 8 in 2024-09-16T09:30:49.938 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:30:49.959 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 54 16 85316 85300 98.7064 78.625 0.00753764 0.0101262 2024-09-16T09:30:49.973 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:49.974 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:49.975 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:30:50.075 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:30:50.106 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:50.107 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:50.108 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:30:50.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:49 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-16T09:30:50.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:49 smithi084 ceph-mon[32629]: osdmap e180: 8 total, 8 up, 8 in 2024-09-16T09:30:50.208 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:30:50.242 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd reweight 1 1 2024-09-16T09:30:50.260 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:50.260 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:50.261 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:30:50.362 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:30:50.394 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:50.395 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:50.395 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:30:50.496 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:30:50.534 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:50.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:50.535 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:30:50.636 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:30:50.672 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:50.673 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:50.673 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:30:50.710 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-16T09:30:50.774 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:30:50.809 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:50.810 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:50.810 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:30:50.911 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:30:50.940 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:50.941 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:50.941 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:30:50.959 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 55 15 86535 86520 98.298 76.25 0.015204 0.0101681 2024-09-16T09:30:51.041 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:30:51.104 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:51.105 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:51.105 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:30:51.138 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:30:51.139 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:30:51.173 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:30:51.173 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:30:51.202 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:30:51.202 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:30:51.206 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:30:51.237 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:30:51.237 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:30:51.246 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:51.247 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:51.247 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:30:51.270 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:30:51.270 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:30:51.308 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:30:51.308 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:30:51.340 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:30:51.340 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:30:51.348 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:30:51.375 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:30:51.375 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:30:51.385 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:51.385 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:51.385 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:30:51.486 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:30:51.524 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:51.524 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:51.525 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:30:51.626 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:30:51.654 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:51.655 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:51.655 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:30:51.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:51 smithi084 ceph-mon[32629]: pgmap v269: 168 pgs: 3 active+recovering+undersized+remapped, 165 active+clean; 5.0 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 853 B/s rd, 60 MiB/s wr, 965 op/s; 7781/248466 objects misplaced (3.132%); 5.0 MiB/s, 80 objects/s recovering 2024-09-16T09:30:51.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:51 smithi084 ceph-mon[32629]: 4.5 scrub starts 2024-09-16T09:30:51.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:51 smithi084 ceph-mon[32629]: 4.5 scrub ok 2024-09-16T09:30:51.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:51 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/2074268154' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:30:51.664 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:51 smithi070 ceph-mon[30388]: pgmap v269: 168 pgs: 3 active+recovering+undersized+remapped, 165 active+clean; 5.0 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 853 B/s rd, 60 MiB/s wr, 965 op/s; 7781/248466 objects misplaced (3.132%); 5.0 MiB/s, 80 objects/s recovering 2024-09-16T09:30:51.664 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:51 smithi070 ceph-mon[30388]: 4.5 scrub starts 2024-09-16T09:30:51.665 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:51 smithi070 ceph-mon[30388]: 4.5 scrub ok 2024-09-16T09:30:51.665 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:51 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2074268154' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:30:51.665 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:51 smithi070 ceph-mon[40691]: pgmap v269: 168 pgs: 3 active+recovering+undersized+remapped, 165 active+clean; 5.0 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 853 B/s rd, 60 MiB/s wr, 965 op/s; 7781/248466 objects misplaced (3.132%); 5.0 MiB/s, 80 objects/s recovering 2024-09-16T09:30:51.665 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:51 smithi070 ceph-mon[40691]: 4.5 scrub starts 2024-09-16T09:30:51.665 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:51 smithi070 ceph-mon[40691]: 4.5 scrub ok 2024-09-16T09:30:51.665 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:51 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/2074268154' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:30:51.717 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.1 dump_blocked_ops 2024-09-16T09:30:51.756 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:30:51.794 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:51.794 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:51.795 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:30:51.895 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:30:51.927 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:51.928 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:51.928 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:30:51.938 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:30:51.959 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 56 16 87736 87720 97.8813 75 0.0146284 0.0102094 2024-09-16T09:30:52.030 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:30:52.075 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:52.075 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:52.076 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:30:52.176 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:30:52.197 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:30:51 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:30:51] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:30:52.215 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:52.215 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:52.215 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:30:52.316 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:30:52.353 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:52.353 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:52.353 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:30:52.454 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:30:52.476 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:52.476 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:52.477 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:30:52.577 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:30:52.609 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:52.609 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:52.610 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:30:52.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:52 smithi084 ceph-mon[32629]: 5.2 scrub starts 2024-09-16T09:30:52.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:52 smithi084 ceph-mon[32629]: 5.2 scrub ok 2024-09-16T09:30:52.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:52 smithi070 ceph-mon[30388]: 5.2 scrub starts 2024-09-16T09:30:52.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:52 smithi070 ceph-mon[30388]: 5.2 scrub ok 2024-09-16T09:30:52.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:52 smithi070 ceph-mon[40691]: 5.2 scrub starts 2024-09-16T09:30:52.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:52 smithi070 ceph-mon[40691]: 5.2 scrub ok 2024-09-16T09:30:52.711 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:30:52.712 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:30:52.779 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:52.779 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:52.780 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:30:52.880 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:30:52.919 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:52.919 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:52.919 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:30:52.959 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 57 16 88927 88911 97.47 74.4375 0.0135832 0.0102548 2024-09-16T09:30:53.020 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:30:53.056 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:53.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:53.057 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:30:53.158 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:30:53.160 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:30:52 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:30:53.195 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:53.196 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:53.197 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:30:53.297 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:30:53.330 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:53.330 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:53.331 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:30:53.432 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:30:53.464 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:53 smithi070 ceph-mon[30388]: pgmap v270: 168 pgs: 3 active+recovering+undersized+remapped, 165 active+clean; 5.0 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 691 B/s rd, 49 MiB/s wr, 782 op/s; 7781/248466 objects misplaced (3.132%); 4.1 MiB/s, 65 objects/s recovering 2024-09-16T09:30:53.464 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:53 smithi070 ceph-mon[40691]: pgmap v270: 168 pgs: 3 active+recovering+undersized+remapped, 165 active+clean; 5.0 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 691 B/s rd, 49 MiB/s wr, 782 op/s; 7781/248466 objects misplaced (3.132%); 4.1 MiB/s, 65 objects/s recovering 2024-09-16T09:30:53.465 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:53.466 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:53.466 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:30:53.567 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:30:53.596 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:53.597 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:53.597 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:30:53.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:53 smithi084 ceph-mon[32629]: pgmap v270: 168 pgs: 3 active+recovering+undersized+remapped, 165 active+clean; 5.0 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 691 B/s rd, 49 MiB/s wr, 782 op/s; 7781/248466 objects misplaced (3.132%); 4.1 MiB/s, 65 objects/s recovering 2024-09-16T09:30:53.698 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:30:53.731 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:53.732 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:53.732 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:30:53.833 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:30:53.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:53.873 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:53.873 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:30:53.960 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 58 16 90115 90099 97.0696 74.25 0.00984765 0.0102957 2024-09-16T09:30:53.974 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:30:54.005 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:54.006 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:54.006 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:30:54.107 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:30:54.144 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:54.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:54.145 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:30:54.246 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:30:54.288 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:54.289 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:54.289 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:30:54.389 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:30:54.425 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:54.426 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:54.426 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:30:54.527 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:30:54.561 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:54.562 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:54.562 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:30:54.611 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:54 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:30:54.611 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:54 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:30:54.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:54 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:30:54.663 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:30:54.696 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:54.696 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:54.696 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:30:54.750 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T09:30:54.798 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:30:54.855 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:54.855 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:54.856 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:30:54.956 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:30:54.959 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 59 16 91258 91242 96.635 71.4375 0.0106105 0.0103433 2024-09-16T09:30:55.025 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:55.025 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:55.026 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:30:55.127 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:30:55.163 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:55.163 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:55.164 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:30:55.228 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:30:55.231 INFO:teuthology.orchestra.run.smithi070.stderr:reweighted osd.1 to 1 (10000) 2024-09-16T09:30:55.265 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:30:55.315 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:55.317 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:55.318 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:30:55.419 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:30:55.453 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:55.454 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:55.454 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:30:55.555 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:30:55.596 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:55.596 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:55.597 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:30:55.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:55 smithi084 ceph-mon[32629]: pgmap v271: 168 pgs: 3 active+recovering+undersized+remapped, 165 active+clean; 5.4 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 93 MiB/s wr, 1.49k op/s; 7258/266745 objects misplaced (2.721%); 7.9 MiB/s, 125 objects/s recovering 2024-09-16T09:30:55.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:55 smithi084 ceph-mon[32629]: 8.9 scrub starts 2024-09-16T09:30:55.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:55 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/806102198' entity='client.admin' cmd={"prefix": "osd reweight", "id": 1, "weight": 1.0} : dispatch 2024-09-16T09:30:55.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:55 smithi070 ceph-mon[30388]: pgmap v271: 168 pgs: 3 active+recovering+undersized+remapped, 165 active+clean; 5.4 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 93 MiB/s wr, 1.49k op/s; 7258/266745 objects misplaced (2.721%); 7.9 MiB/s, 125 objects/s recovering 2024-09-16T09:30:55.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:55 smithi070 ceph-mon[30388]: 8.9 scrub starts 2024-09-16T09:30:55.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:55 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/806102198' entity='client.admin' cmd={"prefix": "osd reweight", "id": 1, "weight": 1.0} : dispatch 2024-09-16T09:30:55.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:55 smithi070 ceph-mon[40691]: pgmap v271: 168 pgs: 3 active+recovering+undersized+remapped, 165 active+clean; 5.4 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 93 MiB/s wr, 1.49k op/s; 7258/266745 objects misplaced (2.721%); 7.9 MiB/s, 125 objects/s recovering 2024-09-16T09:30:55.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:55 smithi070 ceph-mon[40691]: 8.9 scrub starts 2024-09-16T09:30:55.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:55 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/806102198' entity='client.admin' cmd={"prefix": "osd reweight", "id": 1, "weight": 1.0} : dispatch 2024-09-16T09:30:55.697 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:30:55.736 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:55.736 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:55.737 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:30:55.837 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:30:55.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:55.872 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:55.872 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:30:55.960 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:2024-09-16T09:30:55.959554+0000 min lat: 0.00116281 max lat: 2.20004 avg lat: 0.0103898 2024-09-16T09:30:55.960 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-16T09:30:55.960 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 60 16 92391 92375 96.2046 70.8125 0.0186186 0.0103898 2024-09-16T09:30:55.973 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:30:56.013 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:56.013 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:56.014 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:30:56.115 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:30:56.144 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:56.144 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:56.145 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:30:56.236 INFO:teuthology.orchestra.run.smithi070.stderr:nodeep-scrub is unset 2024-09-16T09:30:56.246 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:30:56.283 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:56.284 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:56.284 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:30:56.385 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:30:56.396 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd reweight 2 1 2024-09-16T09:30:56.425 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:56.425 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:56.426 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:30:56.502 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:56 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/806102198' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-16T09:30:56.503 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:56 smithi084 ceph-mon[32629]: osdmap e181: 8 total, 8 up, 8 in 2024-09-16T09:30:56.503 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:56 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/620868435' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:30:56.526 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:30:56.560 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:56.560 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:56.560 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:30:56.661 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:30:56.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:56 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/806102198' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-16T09:30:56.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:56 smithi070 ceph-mon[30388]: osdmap e181: 8 total, 8 up, 8 in 2024-09-16T09:30:56.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:56 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/620868435' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:30:56.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:56 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/806102198' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-16T09:30:56.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:56 smithi070 ceph-mon[40691]: osdmap e181: 8 total, 8 up, 8 in 2024-09-16T09:30:56.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:56 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/620868435' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:30:56.691 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:56.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:56.693 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:30:56.793 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:30:56.831 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:56.832 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:56.832 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:30:56.933 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:30:56.960 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 61 16 93594 93578 95.8599 75.1875 0.0157164 0.0104269 2024-09-16T09:30:56.966 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:56.967 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:56.968 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:30:57.069 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:30:57.111 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:57.112 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:57.112 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:30:57.210 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T09:30:57.211 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T09:30:57.211 INFO:teuthology.orchestra.run.smithi070.stdout: "complaint_time": 30, 2024-09-16T09:30:57.211 INFO:teuthology.orchestra.run.smithi070.stdout: "num_blocked_ops": 0 2024-09-16T09:30:57.211 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T09:30:57.213 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:30:57.253 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:57.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:57.254 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:30:57.355 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:30:57.389 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:57.390 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:57.390 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:30:57.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:57 smithi070 ceph-mon[30388]: pgmap v273: 168 pgs: 3 active+recovering+undersized+remapped, 165 active+clean; 5.4 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 648 B/s rd, 83 MiB/s wr, 1.32k op/s; 7258/266745 objects misplaced (2.721%); 6.1 MiB/s, 97 objects/s recovering 2024-09-16T09:30:57.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:57 smithi070 ceph-mon[30388]: 8.9 scrub ok 2024-09-16T09:30:57.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:57 smithi070 ceph-mon[30388]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-16T09:30:57.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:57 smithi070 ceph-mon[30388]: Cluster is now healthy 2024-09-16T09:30:57.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:57 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/620868435' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-16T09:30:57.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:57 smithi070 ceph-mon[30388]: osdmap e182: 8 total, 8 up, 8 in 2024-09-16T09:30:57.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:57 smithi070 ceph-mon[40691]: pgmap v273: 168 pgs: 3 active+recovering+undersized+remapped, 165 active+clean; 5.4 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 648 B/s rd, 83 MiB/s wr, 1.32k op/s; 7258/266745 objects misplaced (2.721%); 6.1 MiB/s, 97 objects/s recovering 2024-09-16T09:30:57.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:57 smithi070 ceph-mon[40691]: 8.9 scrub ok 2024-09-16T09:30:57.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:57 smithi070 ceph-mon[40691]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-16T09:30:57.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:57 smithi070 ceph-mon[40691]: Cluster is now healthy 2024-09-16T09:30:57.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:57 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/620868435' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-16T09:30:57.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:57 smithi070 ceph-mon[40691]: osdmap e182: 8 total, 8 up, 8 in 2024-09-16T09:30:57.491 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:30:57.524 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:57.524 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:57.525 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:30:57.626 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:30:57.661 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:57.661 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:57.662 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:30:57.662 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:57 smithi084 ceph-mon[32629]: pgmap v273: 168 pgs: 3 active+recovering+undersized+remapped, 165 active+clean; 5.4 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 648 B/s rd, 83 MiB/s wr, 1.32k op/s; 7258/266745 objects misplaced (2.721%); 6.1 MiB/s, 97 objects/s recovering 2024-09-16T09:30:57.662 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:57 smithi084 ceph-mon[32629]: 8.9 scrub ok 2024-09-16T09:30:57.662 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:57 smithi084 ceph-mon[32629]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-16T09:30:57.662 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:57 smithi084 ceph-mon[32629]: Cluster is now healthy 2024-09-16T09:30:57.662 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:57 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/620868435' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-16T09:30:57.662 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:57 smithi084 ceph-mon[32629]: osdmap e182: 8 total, 8 up, 8 in 2024-09-16T09:30:57.729 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:30:57.762 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:30:57.792 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:57.793 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:57.793 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:30:57.894 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:30:57.926 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:57.927 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:57.928 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:30:57.960 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 62 16 94854 94838 95.5839 78.75 0.0119086 0.0104574 2024-09-16T09:30:58.029 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:30:58.065 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:58.066 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:58.067 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:30:58.167 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:30:58.202 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:58.202 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:58.203 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:30:58.304 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:30:58.343 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:58.343 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:58.344 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:30:58.445 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:30:58.478 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:58.479 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:58.479 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:30:58.580 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:30:58.615 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:58.616 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:58.616 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:30:58.717 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:30:58.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:58.750 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:58.751 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:30:58.852 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:30:58.885 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:58.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:58.886 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:30:58.960 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 63 16 96103 96087 95.3057 78.0625 0.0101702 0.0104867 2024-09-16T09:30:58.987 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:30:59.033 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:59.033 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:59.034 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:30:59.134 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:30:59.170 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:59.171 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:59.171 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:30:59.272 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:30:59.306 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:59.307 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:59.307 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:30:59.408 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:30:59.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:59 smithi070 ceph-mon[30388]: pgmap v275: 168 pgs: 3 active+recovering+undersized+remapped, 165 active+clean; 5.5 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 53 MiB/s wr, 850 op/s; 7258/268851 objects misplaced (2.700%); 4.1 MiB/s, 65 objects/s recovering 2024-09-16T09:30:59.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:30:59 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:30:59.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:59 smithi070 ceph-mon[40691]: pgmap v275: 168 pgs: 3 active+recovering+undersized+remapped, 165 active+clean; 5.5 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 53 MiB/s wr, 850 op/s; 7258/268851 objects misplaced (2.700%); 4.1 MiB/s, 65 objects/s recovering 2024-09-16T09:30:59.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:30:59 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:30:59.464 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:59.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:59.465 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:30:59.549 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.1 dump_historic_ops 2024-09-16T09:30:59.566 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:30:59.616 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:30:59.617 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:59.617 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:30:59.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:59 smithi084 ceph-mon[32629]: pgmap v275: 168 pgs: 3 active+recovering+undersized+remapped, 165 active+clean; 5.5 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 53 MiB/s wr, 850 op/s; 7258/268851 objects misplaced (2.700%); 4.1 MiB/s, 65 objects/s recovering 2024-09-16T09:30:59.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:30:59 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:30:59.719 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:30:59.760 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:59.761 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:59.761 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:30:59.777 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd set noscrub 2024-09-16T09:30:59.862 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:30:59.908 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:30:59.908 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:30:59.908 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:30:59.961 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 64 16 97218 97202 94.9046 69.6875 0.0212839 0.010531 2024-09-16T09:30:59.973 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:30:59.995 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:30:59.995 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:31:00.010 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:31:00.041 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:31:00.041 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:31:00.092 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:00.092 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:00.093 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:31:00.096 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:31:00.096 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:31:00.130 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:31:00.130 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:31:00.167 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:31:00.167 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:31:00.193 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:31:00.201 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:31:00.202 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:31:00.237 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:00.238 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:00.238 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:31:00.238 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:31:00.239 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:31:00.272 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:31:00.272 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:31:00.339 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:31:00.383 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:00.384 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:00.384 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:31:00.485 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:31:00.522 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:00.523 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:00.523 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:31:00.624 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:31:00.672 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:00.673 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:00.673 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:31:00.774 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:31:00.808 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:00.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:00.809 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:31:00.900 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:31:00.910 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:31:00.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:00 smithi084 ceph-mon[32629]: pgmap v276: 168 pgs: 3 active+recovering+undersized+remapped, 165 active+clean; 5.8 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1.2 KiB/s rd, 94 MiB/s wr, 1.50k op/s; 6721/284505 objects misplaced (2.362%); 8.3 MiB/s, 132 objects/s recovering 2024-09-16T09:31:00.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:00 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:31:00.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:00 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:31:00.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:00 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/4072636209' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:31:00.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:00 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:31:00.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:00 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:31:00.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:00 smithi070 ceph-mon[30388]: pgmap v276: 168 pgs: 3 active+recovering+undersized+remapped, 165 active+clean; 5.8 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1.2 KiB/s rd, 94 MiB/s wr, 1.50k op/s; 6721/284505 objects misplaced (2.362%); 8.3 MiB/s, 132 objects/s recovering 2024-09-16T09:31:00.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:00 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:31:00.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:00 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:31:00.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:00 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/4072636209' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:31:00.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:00 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:31:00.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:00 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:31:00.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:00 smithi070 ceph-mon[40691]: pgmap v276: 168 pgs: 3 active+recovering+undersized+remapped, 165 active+clean; 5.8 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1.2 KiB/s rd, 94 MiB/s wr, 1.50k op/s; 6721/284505 objects misplaced (2.362%); 8.3 MiB/s, 132 objects/s recovering 2024-09-16T09:31:00.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:00 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:31:00.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:00 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:31:00.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:00 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/4072636209' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:31:00.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:00 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:31:00.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:00 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:31:00.961 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 65 16 98230 98214 94.417 63.25 0.0088343 0.010586 2024-09-16T09:31:00.984 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:00.984 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:00.985 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:31:01.085 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:31:01.123 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:01.123 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:01.124 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:31:01.224 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:31:01.255 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:01.255 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:01.255 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:31:01.356 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:31:01.393 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:01.394 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:01.394 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:31:01.495 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:31:01.532 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:01.532 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:01.533 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:31:01.633 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:31:01.667 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:01.668 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:01.668 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:31:01.769 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:31:01.805 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:01.806 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:01.807 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:31:01.818 INFO:teuthology.orchestra.run.smithi070.stderr:reweighted osd.2 to 1 (10000) 2024-09-16T09:31:01.907 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:31:01.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:01 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:31:01.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:01 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:31:01.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:01 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:31:01.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:01 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1359606034' entity='client.admin' cmd={"prefix": "osd reweight", "id": 2, "weight": 1.0} : dispatch 2024-09-16T09:31:01.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:01 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:31:01.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:01 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:31:01.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:01 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:31:01.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:01 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1359606034' entity='client.admin' cmd={"prefix": "osd reweight", "id": 2, "weight": 1.0} : dispatch 2024-09-16T09:31:01.940 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:01.941 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:01.941 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:31:01.961 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 66 16 99361 99345 94.0574 70.6875 0.00990967 0.0106271 2024-09-16T09:31:02.042 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:31:02.075 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:02.075 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:02.076 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:31:02.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:01 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:31:02.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:01 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:31:02.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:01 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:31:02.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:01 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1359606034' entity='client.admin' cmd={"prefix": "osd reweight", "id": 2, "weight": 1.0} : dispatch 2024-09-16T09:31:02.177 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:31:02.212 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:02.213 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:02.213 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:31:02.314 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:31:02.346 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:02.346 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:02.346 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:31:02.420 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:31:01 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:31:01] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:31:02.447 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:31:02.467 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:02.468 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:02.468 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:31:02.569 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:31:02.604 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:02.605 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:02.605 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:31:02.706 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:31:02.742 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:02.743 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:02.743 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:31:02.844 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:31:02.886 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:02.887 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:02.888 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:31:02.961 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 67 15 100566 100551 93.7785 75.375 0.00862183 0.0106591 2024-09-16T09:31:02.989 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:31:03.022 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:03.023 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:03.023 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:31:03.124 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:31:03.156 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:03.157 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:03.157 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:31:03.160 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:31:02 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:31:03.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:02 smithi084 ceph-mon[32629]: pgmap v277: 168 pgs: 3 active+recovering+undersized+remapped, 165 active+clean; 5.8 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 639 B/s rd, 46 MiB/s wr, 740 op/s; 6721/284505 objects misplaced (2.362%); 4.2 MiB/s, 67 objects/s recovering 2024-09-16T09:31:03.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:02 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1359606034' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-16T09:31:03.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:02 smithi084 ceph-mon[32629]: osdmap e183: 8 total, 8 up, 8 in 2024-09-16T09:31:03.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:02 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:31:03.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:02 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:31:03.170 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:02 smithi070 ceph-mon[30388]: pgmap v277: 168 pgs: 3 active+recovering+undersized+remapped, 165 active+clean; 5.8 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 639 B/s rd, 46 MiB/s wr, 740 op/s; 6721/284505 objects misplaced (2.362%); 4.2 MiB/s, 67 objects/s recovering 2024-09-16T09:31:03.170 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:02 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1359606034' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-16T09:31:03.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:02 smithi070 ceph-mon[30388]: osdmap e183: 8 total, 8 up, 8 in 2024-09-16T09:31:03.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:02 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:31:03.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:02 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:31:03.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:02 smithi070 ceph-mon[40691]: pgmap v277: 168 pgs: 3 active+recovering+undersized+remapped, 165 active+clean; 5.8 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 639 B/s rd, 46 MiB/s wr, 740 op/s; 6721/284505 objects misplaced (2.362%); 4.2 MiB/s, 67 objects/s recovering 2024-09-16T09:31:03.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:02 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1359606034' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-16T09:31:03.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:02 smithi070 ceph-mon[40691]: osdmap e183: 8 total, 8 up, 8 in 2024-09-16T09:31:03.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:02 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:31:03.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:02 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:31:03.259 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:31:03.295 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:03.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:03.296 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:31:03.396 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:31:03.435 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:03.435 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:03.436 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:31:03.536 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:31:03.571 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:03.572 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:03.572 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:31:03.673 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:31:03.711 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:03.712 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:03.712 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:31:03.761 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd reweight 3 1 2024-09-16T09:31:03.801 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:31:03.813 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:31:03.843 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:03.844 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:03.844 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:31:03.945 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:31:03.961 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 68 16 101846 101830 93.5748 79.9375 0.0149811 0.0106818 2024-09-16T09:31:03.995 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:03.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:03.997 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:31:04.031 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:03 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:31:04.032 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:03 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:31:04.032 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:03 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:31:04.032 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:03 smithi070 ceph-mon[30388]: osdmap e184: 8 total, 8 up, 8 in 2024-09-16T09:31:04.033 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:03 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:31:04.033 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:03 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:31:04.033 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:03 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:31:04.033 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:03 smithi070 ceph-mon[40691]: osdmap e184: 8 total, 8 up, 8 in 2024-09-16T09:31:04.098 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:31:04.137 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:04.137 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:04.138 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:31:04.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:03 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:31:04.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:03 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:31:04.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:03 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:31:04.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:03 smithi084 ceph-mon[32629]: osdmap e184: 8 total, 8 up, 8 in 2024-09-16T09:31:04.239 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:31:04.279 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:04.279 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:04.279 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:31:04.380 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:31:04.420 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:04.421 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:04.421 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:31:04.426 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T09:31:04.426 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:31:04.522 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:31:04.563 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:04.564 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:04.565 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:31:04.665 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:31:04.703 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:04.704 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:04.704 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:31:04.805 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:31:04.834 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:04.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:04.835 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:31:04.936 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:31:04.957 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T09:31:04.961 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 69 15 103032 103017 93.2938 74.1875 0.011399 0.0107139 2024-09-16T09:31:04.967 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:04.968 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:04.968 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:31:05.069 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:31:05.094 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:05.094 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:05.095 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:31:05.195 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:31:05.229 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:05.229 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:05.230 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:31:05.330 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:31:05.360 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:05.361 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:05.361 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:31:05.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:05 smithi070 ceph-mon[30388]: pgmap v280: 168 pgs: 3 active+recovering+undersized+remapped, 165 active+clean; 6.1 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.4 KiB/s rd, 99 MiB/s wr, 1.58k op/s; 6099/301560 objects misplaced (2.022%); 9.8 MiB/s, 157 objects/s recovering 2024-09-16T09:31:05.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:05 smithi070 ceph-mon[30388]: 8.5 scrub starts 2024-09-16T09:31:05.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:05 smithi070 ceph-mon[40691]: pgmap v280: 168 pgs: 3 active+recovering+undersized+remapped, 165 active+clean; 6.1 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.4 KiB/s rd, 99 MiB/s wr, 1.58k op/s; 6099/301560 objects misplaced (2.022%); 9.8 MiB/s, 157 objects/s recovering 2024-09-16T09:31:05.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:05 smithi070 ceph-mon[40691]: 8.5 scrub starts 2024-09-16T09:31:05.463 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:31:05.505 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:05.506 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:05.506 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:31:05.607 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:31:05.636 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:05.636 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:05.637 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:31:05.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:05 smithi084 ceph-mon[32629]: pgmap v280: 168 pgs: 3 active+recovering+undersized+remapped, 165 active+clean; 6.1 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.4 KiB/s rd, 99 MiB/s wr, 1.58k op/s; 6099/301560 objects misplaced (2.022%); 9.8 MiB/s, 157 objects/s recovering 2024-09-16T09:31:05.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:05 smithi084 ceph-mon[32629]: 8.5 scrub starts 2024-09-16T09:31:05.737 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:31:05.771 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:05.772 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:05.773 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:31:05.873 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:31:05.912 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:05.913 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:05.913 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:31:05.961 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 70 16 104216 104200 93.0172 73.9375 0.00775389 0.0107458 2024-09-16T09:31:06.014 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:31:06.049 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:06.050 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:06.050 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:31:06.151 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:31:06.186 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:06.187 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:06.187 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:31:06.288 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:31:06.327 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:06.328 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:06.328 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:31:06.375 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.2 dump_ops_in_flight 2024-09-16T09:31:06.429 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:31:06.466 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:06.466 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:06.466 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:31:06.567 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:31:06.615 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:06.616 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:06.616 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:31:06.717 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:31:06.746 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:06.747 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:06.747 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:31:06.821 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:31:06.832 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:31:06.848 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:31:06.877 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:06.878 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:06.878 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:31:06.963 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 71 16 105374 105358 92.7252 72.375 0.0133188 0.0107795 2024-09-16T09:31:06.979 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:31:07.015 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:07.016 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:07.017 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:31:07.117 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:31:07.142 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:31:07.166 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:07.167 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:07.167 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:31:07.244 INFO:teuthology.orchestra.run.smithi070.stderr:noscrub is set 2024-09-16T09:31:07.268 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:31:07.298 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:07.299 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:07.299 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:31:07.400 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:31:07.439 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:07.439 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:07.440 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:31:07.484 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:07 smithi070 ceph-mon[30388]: pgmap v281: 168 pgs: 3 active+recovering+undersized+remapped, 165 active+clean; 6.1 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 639 B/s rd, 85 MiB/s wr, 1.36k op/s; 6099/301560 objects misplaced (2.022%); 9.1 MiB/s, 144 objects/s recovering 2024-09-16T09:31:07.484 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:07 smithi070 ceph-mon[30388]: 8.5 scrub ok 2024-09-16T09:31:07.484 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:07 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:31:07.484 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:07 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:31:07.484 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:07 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/4148682582' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:31:07.484 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:07 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:31:07.485 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:07 smithi070 ceph-mon[40691]: pgmap v281: 168 pgs: 3 active+recovering+undersized+remapped, 165 active+clean; 6.1 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 639 B/s rd, 85 MiB/s wr, 1.36k op/s; 6099/301560 objects misplaced (2.022%); 9.1 MiB/s, 144 objects/s recovering 2024-09-16T09:31:07.485 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:07 smithi070 ceph-mon[40691]: 8.5 scrub ok 2024-09-16T09:31:07.485 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:07 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:31:07.485 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:07 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:31:07.485 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:07 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/4148682582' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:31:07.486 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:07 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:31:07.541 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:31:07.582 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:07.582 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:07.582 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:31:07.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:07 smithi084 ceph-mon[32629]: pgmap v281: 168 pgs: 3 active+recovering+undersized+remapped, 165 active+clean; 6.1 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 639 B/s rd, 85 MiB/s wr, 1.36k op/s; 6099/301560 objects misplaced (2.022%); 9.1 MiB/s, 144 objects/s recovering 2024-09-16T09:31:07.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:07 smithi084 ceph-mon[32629]: 8.5 scrub ok 2024-09-16T09:31:07.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:07 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:31:07.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:07 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:31:07.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:07 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/4148682582' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:31:07.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:07 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:31:07.683 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:31:07.721 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:07.721 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:07.722 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:31:07.822 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:31:07.859 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:07.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:07.860 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:31:07.961 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:31:07.963 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 72 16 106585 106569 92.4885 75.6875 0.0129453 0.0108077 2024-09-16T09:31:07.996 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:07.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:07.997 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:31:08.097 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:31:08.140 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:08.140 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:08.141 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:31:08.241 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:31:08.253 INFO:teuthology.orchestra.run.smithi070.stderr:reweighted osd.3 to 1 (10000) 2024-09-16T09:31:08.257 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:08 smithi070 ceph-mon[30388]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:31:08.257 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:08 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-16T09:31:08.257 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:08 smithi070 ceph-mon[30388]: osdmap e185: 8 total, 8 up, 8 in 2024-09-16T09:31:08.257 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:08 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2668508894' entity='client.admin' cmd={"prefix": "osd reweight", "id": 3, "weight": 1.0} : dispatch 2024-09-16T09:31:08.257 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:08 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 3, "weight": 1.0} : dispatch 2024-09-16T09:31:08.257 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:08 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1302221813' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:31:08.257 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:08 smithi070 ceph-mon[40691]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:31:08.257 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:08 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-16T09:31:08.257 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:08 smithi070 ceph-mon[40691]: osdmap e185: 8 total, 8 up, 8 in 2024-09-16T09:31:08.257 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:08 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/2668508894' entity='client.admin' cmd={"prefix": "osd reweight", "id": 3, "weight": 1.0} : dispatch 2024-09-16T09:31:08.258 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:08 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 3, "weight": 1.0} : dispatch 2024-09-16T09:31:08.258 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:08 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1302221813' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:31:08.258 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T09:31:08.258 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:31:08.280 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:08.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:08.282 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:31:08.382 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:31:08.403 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:08.404 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:08.404 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:31:08.505 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:31:08.509 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:31:08.509 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:31:08.509 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:31:08.509 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:31:08.510 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:31:08.510 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:31:08.510 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:31:08.510 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:31:08.510 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:31:08.510 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:31:08.510 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:31:08.510 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:31:08.511 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:31:08.511 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:31:08.511 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:31:08.511 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:31:08.526 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:08.527 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:08.527 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:31:08.628 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:31:08.651 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:08.651 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:08.652 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:31:08.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:08 smithi084 ceph-mon[32629]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:31:08.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:08 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-16T09:31:08.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:08 smithi084 ceph-mon[32629]: osdmap e185: 8 total, 8 up, 8 in 2024-09-16T09:31:08.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:08 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/2668508894' entity='client.admin' cmd={"prefix": "osd reweight", "id": 3, "weight": 1.0} : dispatch 2024-09-16T09:31:08.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:08 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 3, "weight": 1.0} : dispatch 2024-09-16T09:31:08.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:08 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1302221813' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:31:08.753 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:31:08.811 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:08.812 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:08.812 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:31:08.913 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:31:08.935 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:08.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:08.936 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:31:08.962 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 73 16 106991 106975 91.5693 25.375 0.0028164 0.0108189 2024-09-16T09:31:09.037 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:31:09.060 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:09.061 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:09.061 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:31:09.162 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:31:09.187 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:09.187 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:09.188 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:31:09.289 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:31:09.321 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:09.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:09.322 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:31:09.423 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:31:09.453 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:09.454 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:09.454 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:31:09.555 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:31:09.590 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:09.590 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:09.591 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:31:09.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:09 smithi084 ceph-mon[32629]: pgmap v283: 168 pgs: 3 active+recovering+undersized+remapped, 165 active+clean; 6.2 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.7 KiB/s rd, 67 MiB/s wr, 1.08k op/s; 6099/303867 objects misplaced (2.007%); 6.5 MiB/s, 103 objects/s recovering 2024-09-16T09:31:09.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:09 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-16T09:31:09.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:09 smithi084 ceph-mon[32629]: osdmap e186: 8 total, 8 up, 8 in 2024-09-16T09:31:09.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:09 smithi070 ceph-mon[30388]: pgmap v283: 168 pgs: 3 active+recovering+undersized+remapped, 165 active+clean; 6.2 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.7 KiB/s rd, 67 MiB/s wr, 1.08k op/s; 6099/303867 objects misplaced (2.007%); 6.5 MiB/s, 103 objects/s recovering 2024-09-16T09:31:09.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:09 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-16T09:31:09.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:09 smithi070 ceph-mon[30388]: osdmap e186: 8 total, 8 up, 8 in 2024-09-16T09:31:09.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:09 smithi070 ceph-mon[40691]: pgmap v283: 168 pgs: 3 active+recovering+undersized+remapped, 165 active+clean; 6.2 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.7 KiB/s rd, 67 MiB/s wr, 1.08k op/s; 6099/303867 objects misplaced (2.007%); 6.5 MiB/s, 103 objects/s recovering 2024-09-16T09:31:09.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:09 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-16T09:31:09.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:09 smithi070 ceph-mon[40691]: osdmap e186: 8 total, 8 up, 8 in 2024-09-16T09:31:09.691 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:31:09.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:09.728 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:09.729 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:31:09.830 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:31:09.864 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:09.864 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:09.864 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:31:09.963 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 74 16 107570 107554 90.8209 36.1875 0.0105044 0.011005 2024-09-16T09:31:09.965 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:31:10.001 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:10.002 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:10.002 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:31:10.103 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:31:10.135 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:10.136 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:10.136 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:31:10.237 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:31:10.277 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:10.278 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:10.278 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:31:10.379 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:31:10.412 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:10.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:10.413 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:31:10.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:10 smithi070 ceph-mon[30388]: osdmap e187: 8 total, 8 up, 8 in 2024-09-16T09:31:10.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:10 smithi070 ceph-mon[30388]: 1.3 scrub starts 2024-09-16T09:31:10.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:10 smithi070 ceph-mon[40691]: osdmap e187: 8 total, 8 up, 8 in 2024-09-16T09:31:10.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:10 smithi070 ceph-mon[40691]: 1.3 scrub starts 2024-09-16T09:31:10.513 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:31:10.546 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:10.547 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:10.547 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:31:10.648 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:31:10.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:10 smithi084 ceph-mon[32629]: osdmap e187: 8 total, 8 up, 8 in 2024-09-16T09:31:10.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:10 smithi084 ceph-mon[32629]: 1.3 scrub starts 2024-09-16T09:31:10.682 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:10.683 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:10.684 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:31:10.764 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.2 dump_blocked_ops 2024-09-16T09:31:10.785 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:31:10.822 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:10.822 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:10.823 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:31:10.923 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:31:10.963 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:10.964 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:10.965 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:31:10.965 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 75 16 108685 108669 90.5387 69.6875 0.0164287 0.0110394 2024-09-16T09:31:11.066 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:31:11.099 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:11.099 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:11.100 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:31:11.201 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:31:11.229 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:11.229 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:11.230 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:31:11.331 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:31:11.367 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:11.368 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:11.369 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:31:11.469 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:31:11.499 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:11.500 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:11.500 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:31:11.601 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:31:11.621 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:11 smithi070 ceph-mon[30388]: pgmap v286: 168 pgs: 3 active+recovering+undersized+remapped, 165 active+clean; 6.5 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 852 B/s rd, 63 MiB/s wr, 1.02k op/s; 5456/319848 objects misplaced (1.706%); 6.7 MiB/s, 107 objects/s recovering 2024-09-16T09:31:11.621 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:11 smithi070 ceph-mon[30388]: 1.5 scrub starts 2024-09-16T09:31:11.621 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:11 smithi070 ceph-mon[40691]: pgmap v286: 168 pgs: 3 active+recovering+undersized+remapped, 165 active+clean; 6.5 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 852 B/s rd, 63 MiB/s wr, 1.02k op/s; 5456/319848 objects misplaced (1.706%); 6.7 MiB/s, 107 objects/s recovering 2024-09-16T09:31:11.621 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:11 smithi070 ceph-mon[40691]: 1.5 scrub starts 2024-09-16T09:31:11.641 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:11.642 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:11.642 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:31:11.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:11 smithi084 ceph-mon[32629]: pgmap v286: 168 pgs: 3 active+recovering+undersized+remapped, 165 active+clean; 6.5 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 852 B/s rd, 63 MiB/s wr, 1.02k op/s; 5456/319848 objects misplaced (1.706%); 6.7 MiB/s, 107 objects/s recovering 2024-09-16T09:31:11.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:11 smithi084 ceph-mon[32629]: 1.5 scrub starts 2024-09-16T09:31:11.676 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd reweight 4 1 2024-09-16T09:31:11.744 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:31:11.778 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:11.779 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:11.779 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:31:11.880 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:31:11.929 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:11.930 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:11.930 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:31:11.963 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 76 16 109769 109753 90.2388 67.75 0.00772745 0.0110774 2024-09-16T09:31:12.031 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:31:12.062 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:12.062 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:12.063 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:31:12.163 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:31:12.193 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:31:11 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:31:11] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:31:12.205 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:12.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:12.206 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:31:12.225 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-16T09:31:12.307 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:31:12.361 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:31:12.361 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:12.362 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:12.362 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:31:12.377 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:31:12.463 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:31:12.497 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:12 smithi070 ceph-mon[30388]: 1.2 scrub starts 2024-09-16T09:31:12.498 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:12 smithi070 ceph-mon[30388]: 1.1 scrub starts 2024-09-16T09:31:12.498 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:12 smithi070 ceph-mon[30388]: 1.e scrub starts 2024-09-16T09:31:12.498 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:12 smithi070 ceph-mon[40691]: 1.2 scrub starts 2024-09-16T09:31:12.498 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:12 smithi070 ceph-mon[40691]: 1.1 scrub starts 2024-09-16T09:31:12.498 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:12 smithi070 ceph-mon[40691]: 1.e scrub starts 2024-09-16T09:31:12.506 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:12.506 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:12.507 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:31:12.608 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:31:12.644 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:12.644 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:12.645 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:31:12.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:12 smithi084 ceph-mon[32629]: 1.2 scrub starts 2024-09-16T09:31:12.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:12 smithi084 ceph-mon[32629]: 1.1 scrub starts 2024-09-16T09:31:12.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:12 smithi084 ceph-mon[32629]: 1.e scrub starts 2024-09-16T09:31:12.745 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:31:12.783 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:12.784 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:12.784 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:31:12.885 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:31:12.921 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:12.922 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:12.923 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:31:12.964 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 77 15 110789 110774 89.895 63.8125 0.0239284 0.0111194 2024-09-16T09:31:13.024 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:31:13.054 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:13.055 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:13.056 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:31:13.156 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:31:13.160 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:31:12 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:31:13.188 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:13.189 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:13.189 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:31:13.290 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:31:13.323 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:13.324 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:13.324 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:31:13.425 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:31:13.459 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:13.460 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:13.460 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:31:13.561 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:31:13.602 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:13.603 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:13.603 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:31:13.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:13 smithi084 ceph-mon[32629]: pgmap v287: 168 pgs: 3 active+recovering+undersized+remapped, 165 active+clean; 6.5 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 852 B/s rd, 63 MiB/s wr, 1.02k op/s; 5456/319848 objects misplaced (1.706%); 6.7 MiB/s, 107 objects/s recovering 2024-09-16T09:31:13.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:13 smithi084 ceph-mon[32629]: 1.10 scrub starts 2024-09-16T09:31:13.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:13 smithi084 ceph-mon[32629]: 1.0 scrub starts 2024-09-16T09:31:13.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:13 smithi084 ceph-mon[32629]: 1.b scrub starts 2024-09-16T09:31:13.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:13 smithi070 ceph-mon[30388]: pgmap v287: 168 pgs: 3 active+recovering+undersized+remapped, 165 active+clean; 6.5 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 852 B/s rd, 63 MiB/s wr, 1.02k op/s; 5456/319848 objects misplaced (1.706%); 6.7 MiB/s, 107 objects/s recovering 2024-09-16T09:31:13.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:13 smithi070 ceph-mon[30388]: 1.10 scrub starts 2024-09-16T09:31:13.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:13 smithi070 ceph-mon[30388]: 1.0 scrub starts 2024-09-16T09:31:13.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:13 smithi070 ceph-mon[30388]: 1.b scrub starts 2024-09-16T09:31:13.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:13 smithi070 ceph-mon[40691]: pgmap v287: 168 pgs: 3 active+recovering+undersized+remapped, 165 active+clean; 6.5 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 852 B/s rd, 63 MiB/s wr, 1.02k op/s; 5456/319848 objects misplaced (1.706%); 6.7 MiB/s, 107 objects/s recovering 2024-09-16T09:31:13.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:13 smithi070 ceph-mon[40691]: 1.10 scrub starts 2024-09-16T09:31:13.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:13 smithi070 ceph-mon[40691]: 1.0 scrub starts 2024-09-16T09:31:13.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:13 smithi070 ceph-mon[40691]: 1.b scrub starts 2024-09-16T09:31:13.705 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:31:13.740 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:13.741 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:13.741 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:31:13.842 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:31:13.894 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:13.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:13.895 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:31:13.964 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 78 16 111869 111853 89.6071 67.4375 0.00950702 0.0111542 2024-09-16T09:31:13.990 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T09:31:13.991 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T09:31:13.991 INFO:teuthology.orchestra.run.smithi070.stdout: "complaint_time": 30, 2024-09-16T09:31:13.991 INFO:teuthology.orchestra.run.smithi070.stdout: "num_blocked_ops": 0 2024-09-16T09:31:13.991 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T09:31:13.996 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:31:14.022 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:14.022 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:14.022 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:31:14.123 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:31:14.159 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:14.160 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:14.160 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:31:14.257 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T09:31:14.261 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:31:14.290 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:14.290 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:14.291 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:31:14.391 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:31:14.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:14 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:31:14.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:14 smithi070 ceph-mon[30388]: 1.14 scrub starts 2024-09-16T09:31:14.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:14 smithi070 ceph-mon[30388]: 1.12 scrub starts 2024-09-16T09:31:14.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:14 smithi070 ceph-mon[30388]: 1.d scrub starts 2024-09-16T09:31:14.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:14 smithi070 ceph-mon[30388]: 1.f scrub starts 2024-09-16T09:31:14.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:14 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:31:14.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:14 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:31:14.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:14 smithi070 ceph-mon[30388]: osdmap e188: 8 total, 8 up, 8 in 2024-09-16T09:31:14.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:14 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:31:14.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:14 smithi070 ceph-mon[40691]: 1.14 scrub starts 2024-09-16T09:31:14.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:14 smithi070 ceph-mon[40691]: 1.12 scrub starts 2024-09-16T09:31:14.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:14 smithi070 ceph-mon[40691]: 1.d scrub starts 2024-09-16T09:31:14.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:14 smithi070 ceph-mon[40691]: 1.f scrub starts 2024-09-16T09:31:14.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:14 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:31:14.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:14 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:31:14.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:14 smithi070 ceph-mon[40691]: osdmap e188: 8 total, 8 up, 8 in 2024-09-16T09:31:14.450 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:14.451 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:14.452 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:31:14.553 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:31:14.579 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:14.579 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:14.580 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:31:14.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:14 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:31:14.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:14 smithi084 ceph-mon[32629]: 1.14 scrub starts 2024-09-16T09:31:14.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:14 smithi084 ceph-mon[32629]: 1.12 scrub starts 2024-09-16T09:31:14.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:14 smithi084 ceph-mon[32629]: 1.d scrub starts 2024-09-16T09:31:14.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:14 smithi084 ceph-mon[32629]: 1.f scrub starts 2024-09-16T09:31:14.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:14 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:31:14.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:14 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:31:14.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:14 smithi084 ceph-mon[32629]: osdmap e188: 8 total, 8 up, 8 in 2024-09-16T09:31:14.680 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:31:14.723 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:14.724 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:14.724 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:31:14.825 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:31:14.856 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:14.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:14.857 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:31:14.958 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:31:14.964 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 79 15 112840 112825 89.2418 60.75 0.0134113 0.0112006 2024-09-16T09:31:14.991 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:14.991 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:14.991 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:31:15.064 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.2 dump_historic_ops 2024-09-16T09:31:15.071 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:31:15.092 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:31:15.119 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:15.120 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:15.120 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:31:15.221 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:31:15.278 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:15.279 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:15.279 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:31:15.324 INFO:teuthology.orchestra.run.smithi070.stderr:reweighted osd.4 to 1 (10000) 2024-09-16T09:31:15.380 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:31:15.415 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:15.416 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:15.416 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:31:15.517 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:31:15.552 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:15.552 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:15.553 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:31:15.656 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:31:15.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:15 smithi084 ceph-mon[32629]: pgmap v288: 168 pgs: 1 active+recovery_wait+degraded, 7 active+clean+scrubbing, 3 active+recovering+undersized+remapped, 157 active+clean; 6.8 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.6 KiB/s rd, 101 MiB/s wr, 1.62k op/s; 1/332436 objects degraded (0.000%); 9915/332436 objects misplaced (2.983%); 9.6 MiB/s, 162 objects/s recovering 2024-09-16T09:31:15.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:15 smithi084 ceph-mon[32629]: Health check failed: Degraded data redundancy: 1/332436 objects degraded (0.000%), 1 pg degraded (PG_DEGRADED) 2024-09-16T09:31:15.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:15 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/2792611745' entity='client.admin' cmd={"prefix": "osd reweight", "id": 4, "weight": 1.0} : dispatch 2024-09-16T09:31:15.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:15 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 4, "weight": 1.0} : dispatch 2024-09-16T09:31:15.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:15 smithi070 ceph-mon[30388]: pgmap v288: 168 pgs: 1 active+recovery_wait+degraded, 7 active+clean+scrubbing, 3 active+recovering+undersized+remapped, 157 active+clean; 6.8 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.6 KiB/s rd, 101 MiB/s wr, 1.62k op/s; 1/332436 objects degraded (0.000%); 9915/332436 objects misplaced (2.983%); 9.6 MiB/s, 162 objects/s recovering 2024-09-16T09:31:15.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:15 smithi070 ceph-mon[30388]: Health check failed: Degraded data redundancy: 1/332436 objects degraded (0.000%), 1 pg degraded (PG_DEGRADED) 2024-09-16T09:31:15.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:15 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2792611745' entity='client.admin' cmd={"prefix": "osd reweight", "id": 4, "weight": 1.0} : dispatch 2024-09-16T09:31:15.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:15 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 4, "weight": 1.0} : dispatch 2024-09-16T09:31:15.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:15 smithi070 ceph-mon[40691]: pgmap v288: 168 pgs: 1 active+recovery_wait+degraded, 7 active+clean+scrubbing, 3 active+recovering+undersized+remapped, 157 active+clean; 6.8 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.6 KiB/s rd, 101 MiB/s wr, 1.62k op/s; 1/332436 objects degraded (0.000%); 9915/332436 objects misplaced (2.983%); 9.6 MiB/s, 162 objects/s recovering 2024-09-16T09:31:15.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:15 smithi070 ceph-mon[40691]: Health check failed: Degraded data redundancy: 1/332436 objects degraded (0.000%), 1 pg degraded (PG_DEGRADED) 2024-09-16T09:31:15.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:15 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/2792611745' entity='client.admin' cmd={"prefix": "osd reweight", "id": 4, "weight": 1.0} : dispatch 2024-09-16T09:31:15.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:15 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 4, "weight": 1.0} : dispatch 2024-09-16T09:31:15.688 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:15.689 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:15.689 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:31:15.790 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:31:15.823 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:15.824 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:15.824 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:31:15.925 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:31:15.956 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:15.957 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:15.957 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:31:15.964 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:2024-09-16T09:31:15.963955+0000 min lat: 0.00116281 max lat: 2.20004 avg lat: 0.0112349 2024-09-16T09:31:15.964 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-16T09:31:15.964 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 80 16 113922 113906 88.9707 67.5625 0.0111305 0.0112349 2024-09-16T09:31:16.058 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:31:16.096 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:16.097 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:16.098 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:31:16.199 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:31:16.211 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:31:16.228 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:16.229 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:16.229 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:31:16.330 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:31:16.375 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:16.376 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:16.376 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:31:16.477 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:31:16.513 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:16.514 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:16.515 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:31:16.616 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:31:16.650 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:16.651 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:16.651 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:31:16.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:16 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-16T09:31:16.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:16 smithi084 ceph-mon[32629]: osdmap e189: 8 total, 8 up, 8 in 2024-09-16T09:31:16.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:16 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-16T09:31:16.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:16 smithi070 ceph-mon[30388]: osdmap e189: 8 total, 8 up, 8 in 2024-09-16T09:31:16.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:16 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-16T09:31:16.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:16 smithi070 ceph-mon[40691]: osdmap e189: 8 total, 8 up, 8 in 2024-09-16T09:31:16.752 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:31:16.758 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd reweight 5 1 2024-09-16T09:31:16.788 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:16.789 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:16.789 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:31:16.890 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:31:16.936 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:16.937 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:16.937 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:31:16.964 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 81 16 114942 114926 88.6591 63.75 0.00740071 0.0112742 2024-09-16T09:31:17.038 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:31:17.081 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:17.081 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:17.081 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:31:17.169 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:31:17.182 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:31:17.218 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:17.219 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:17.219 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:31:17.320 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:31:17.338 INFO:teuthology.orchestra.run.smithi070.stderr:nodeep-scrub is set 2024-09-16T09:31:17.363 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:17.364 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:17.364 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:31:17.408 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:17 smithi070 ceph-mon[30388]: pgmap v291: 168 pgs: 1 active+recovery_wait+degraded, 7 active+clean+scrubbing, 3 active+recovering+undersized+remapped, 157 active+clean; 6.8 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 805 B/s rd, 41 MiB/s wr, 660 op/s; 1/332436 objects degraded (0.000%); 9915/332436 objects misplaced (2.983%); 3.3 MiB/s, 61 objects/s recovering 2024-09-16T09:31:17.408 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:17 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2264370250' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:31:17.408 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:17 smithi070 ceph-mon[40691]: pgmap v291: 168 pgs: 1 active+recovery_wait+degraded, 7 active+clean+scrubbing, 3 active+recovering+undersized+remapped, 157 active+clean; 6.8 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 805 B/s rd, 41 MiB/s wr, 660 op/s; 1/332436 objects degraded (0.000%); 9915/332436 objects misplaced (2.983%); 3.3 MiB/s, 61 objects/s recovering 2024-09-16T09:31:17.408 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:17 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/2264370250' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:31:17.465 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:31:17.474 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:31:17.474 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:31:17.504 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:17.505 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:17.505 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:31:17.507 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:31:17.507 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:31:17.538 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:31:17.538 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:31:17.570 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:31:17.571 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:31:17.603 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:31:17.603 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:31:17.606 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:31:17.635 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:17.635 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:17.636 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:31:17.640 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:31:17.640 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:31:17.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:17 smithi084 ceph-mon[32629]: pgmap v291: 168 pgs: 1 active+recovery_wait+degraded, 7 active+clean+scrubbing, 3 active+recovering+undersized+remapped, 157 active+clean; 6.8 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 805 B/s rd, 41 MiB/s wr, 660 op/s; 1/332436 objects degraded (0.000%); 9915/332436 objects misplaced (2.983%); 3.3 MiB/s, 61 objects/s recovering 2024-09-16T09:31:17.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:17 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/2264370250' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:31:17.678 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:31:17.678 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:31:17.710 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:31:17.710 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:31:17.737 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:31:17.772 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:17.772 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:17.773 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:31:17.874 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:31:17.912 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:17.913 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:17.913 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:31:17.965 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 82 16 115950 115934 88.3462 63 0.00583217 0.0113144 2024-09-16T09:31:18.014 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:31:18.060 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:18.061 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:18.061 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:31:18.162 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:31:18.190 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:18.190 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:18.191 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:31:18.291 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:31:18.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:18.321 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:18.321 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:31:18.422 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:31:18.452 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:18.452 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:18.453 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:31:18.554 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:31:18.589 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:18.589 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:18.589 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:31:18.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:18 smithi084 ceph-mon[32629]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:31:18.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:18 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/2264370250' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-16T09:31:18.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:18 smithi084 ceph-mon[32629]: osdmap e190: 8 total, 8 up, 8 in 2024-09-16T09:31:18.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:18 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/4219614853' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:31:18.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:18 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:31:18.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:18 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:31:18.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:18 smithi070 ceph-mon[30388]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:31:18.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:18 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2264370250' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-16T09:31:18.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:18 smithi070 ceph-mon[30388]: osdmap e190: 8 total, 8 up, 8 in 2024-09-16T09:31:18.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:18 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/4219614853' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:31:18.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:18 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:31:18.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:18 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:31:18.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:18 smithi070 ceph-mon[40691]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:31:18.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:18 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/2264370250' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-16T09:31:18.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:18 smithi070 ceph-mon[40691]: osdmap e190: 8 total, 8 up, 8 in 2024-09-16T09:31:18.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:18 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/4219614853' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:31:18.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:18 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:31:18.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:18 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:31:18.690 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:31:18.722 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:18.724 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:18.724 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:31:18.825 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:31:18.866 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:18.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:18.868 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:31:18.965 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 83 16 117091 117075 88.1409 71.3125 0.00520678 0.01134 2024-09-16T09:31:18.968 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:31:19.005 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:19.005 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:19.006 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:31:19.106 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:31:19.149 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:19.149 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:19.149 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:31:19.250 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:31:19.275 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:31:19.280 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:19.280 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:19.281 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:31:19.327 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T09:31:19.327 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:31:19.382 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:31:19.421 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:19.422 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:19.422 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:31:19.523 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:31:19.557 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:19.558 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:19.558 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:31:19.572 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:19 smithi070 ceph-mon[30388]: pgmap v293: 168 pgs: 1 active+recovery_wait+degraded, 5 active+clean+scrubbing, 3 active+recovering+undersized+remapped, 159 active+clean; 6.8 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.7 KiB/s rd, 51 MiB/s wr, 810 op/s; 1/334413 objects degraded (0.000%); 9915/334413 objects misplaced (2.965%); 3.5 MiB/s, 64 objects/s recovering 2024-09-16T09:31:19.572 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:19 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:31:19.572 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:19 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:31:19.572 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:19 smithi070 ceph-mon[40691]: pgmap v293: 168 pgs: 1 active+recovery_wait+degraded, 5 active+clean+scrubbing, 3 active+recovering+undersized+remapped, 159 active+clean; 6.8 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.7 KiB/s rd, 51 MiB/s wr, 810 op/s; 1/334413 objects degraded (0.000%); 9915/334413 objects misplaced (2.965%); 3.5 MiB/s, 64 objects/s recovering 2024-09-16T09:31:19.573 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:19 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:31:19.573 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:19 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:31:19.659 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:31:19.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:19 smithi084 ceph-mon[32629]: pgmap v293: 168 pgs: 1 active+recovery_wait+degraded, 5 active+clean+scrubbing, 3 active+recovering+undersized+remapped, 159 active+clean; 6.8 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.7 KiB/s rd, 51 MiB/s wr, 810 op/s; 1/334413 objects degraded (0.000%); 9915/334413 objects misplaced (2.965%); 3.5 MiB/s, 64 objects/s recovering 2024-09-16T09:31:19.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:19 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:31:19.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:19 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:31:19.697 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:19.698 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:19.699 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:31:19.799 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:31:19.836 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:19.836 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:19.837 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:31:19.937 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:31:19.965 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 84 16 118195 118179 87.913 69 0.00878258 0.0113698 2024-09-16T09:31:19.973 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:19.974 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:19.974 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:31:20.075 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:31:20.118 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:20.119 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:20.119 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:31:20.155 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd unset noscrub 2024-09-16T09:31:20.220 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:31:20.250 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:20.250 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:20.250 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:31:20.351 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:31:20.382 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:20.382 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:20.383 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:31:20.484 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:31:20.527 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:20.528 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:20.528 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:31:20.608 INFO:teuthology.orchestra.run.smithi070.stderr:reweighted osd.5 to 1 (10000) 2024-09-16T09:31:20.620 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.3 dump_ops_in_flight 2024-09-16T09:31:20.629 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:31:20.714 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:20.714 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:20.715 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:31:20.815 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:31:20.857 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:20.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:20.858 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:31:20.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:20 smithi084 ceph-mon[32629]: osdmap e191: 8 total, 8 up, 8 in 2024-09-16T09:31:20.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:20 smithi084 ceph-mon[32629]: pgmap v295: 168 pgs: 1 active+recovery_wait+degraded, 3 active+recovering+undersized+remapped, 164 active+clean; 7.1 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 945 B/s rd, 60 MiB/s wr, 957 op/s; 1/347979 objects degraded (0.000%); 9523/347979 objects misplaced (2.737%); 4.5 MiB/s, 72 objects/s recovering 2024-09-16T09:31:20.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:20 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1501926151' entity='client.admin' cmd={"prefix": "osd reweight", "id": 5, "weight": 1.0} : dispatch 2024-09-16T09:31:20.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:20 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 5, "weight": 1.0} : dispatch 2024-09-16T09:31:20.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:20 smithi070 ceph-mon[30388]: osdmap e191: 8 total, 8 up, 8 in 2024-09-16T09:31:20.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:20 smithi070 ceph-mon[30388]: pgmap v295: 168 pgs: 1 active+recovery_wait+degraded, 3 active+recovering+undersized+remapped, 164 active+clean; 7.1 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 945 B/s rd, 60 MiB/s wr, 957 op/s; 1/347979 objects degraded (0.000%); 9523/347979 objects misplaced (2.737%); 4.5 MiB/s, 72 objects/s recovering 2024-09-16T09:31:20.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:20 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1501926151' entity='client.admin' cmd={"prefix": "osd reweight", "id": 5, "weight": 1.0} : dispatch 2024-09-16T09:31:20.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:20 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 5, "weight": 1.0} : dispatch 2024-09-16T09:31:20.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:20 smithi070 ceph-mon[40691]: osdmap e191: 8 total, 8 up, 8 in 2024-09-16T09:31:20.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:20 smithi070 ceph-mon[40691]: pgmap v295: 168 pgs: 1 active+recovery_wait+degraded, 3 active+recovering+undersized+remapped, 164 active+clean; 7.1 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 945 B/s rd, 60 MiB/s wr, 957 op/s; 1/347979 objects degraded (0.000%); 9523/347979 objects misplaced (2.737%); 4.5 MiB/s, 72 objects/s recovering 2024-09-16T09:31:20.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:20 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1501926151' entity='client.admin' cmd={"prefix": "osd reweight", "id": 5, "weight": 1.0} : dispatch 2024-09-16T09:31:20.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:20 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 5, "weight": 1.0} : dispatch 2024-09-16T09:31:20.959 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:31:20.965 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 85 15 119200 119185 87.6184 62.875 0.00585385 0.0114079 2024-09-16T09:31:20.988 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:20.989 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:20.990 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:31:21.056 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T09:31:21.090 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:31:21.123 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:21.124 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:21.124 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:31:21.224 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:31:21.259 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:21.260 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:21.260 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:31:21.361 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:31:21.400 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:21.401 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:21.401 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:31:21.502 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:31:21.540 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:21.541 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:21.542 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:31:21.642 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:31:21.676 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:21.677 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:21.677 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:31:21.778 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:31:21.781 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd reweight 6 1 2024-09-16T09:31:21.800 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:31:21.854 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:21.855 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:21.855 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:31:21.857 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:31:21.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:21 smithi084 ceph-mon[32629]: Health check update: Degraded data redundancy: 1/347979 objects degraded (0.000%), 1 pg degraded (PG_DEGRADED) 2024-09-16T09:31:21.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:21 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-16T09:31:21.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:21 smithi084 ceph-mon[32629]: osdmap e192: 8 total, 8 up, 8 in 2024-09-16T09:31:21.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:21 smithi070 ceph-mon[30388]: Health check update: Degraded data redundancy: 1/347979 objects degraded (0.000%), 1 pg degraded (PG_DEGRADED) 2024-09-16T09:31:21.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:21 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-16T09:31:21.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:21 smithi070 ceph-mon[30388]: osdmap e192: 8 total, 8 up, 8 in 2024-09-16T09:31:21.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:21 smithi070 ceph-mon[40691]: Health check update: Degraded data redundancy: 1/347979 objects degraded (0.000%), 1 pg degraded (PG_DEGRADED) 2024-09-16T09:31:21.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:21 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-16T09:31:21.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:21 smithi070 ceph-mon[40691]: osdmap e192: 8 total, 8 up, 8 in 2024-09-16T09:31:21.956 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:31:21.965 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 86 16 120253 120237 87.3639 65.75 0.0104755 0.0114412 2024-09-16T09:31:21.998 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:21.999 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:21.999 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:31:22.100 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:31:22.129 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:22.131 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:22.131 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:31:22.208 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:31:21 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:31:21] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:31:22.232 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:31:22.277 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:22.277 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:22.278 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:31:22.379 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:31:22.415 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:22.416 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:22.416 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:31:22.517 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:31:22.520 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:31:22.565 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:22.566 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:22.566 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:31:22.666 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:31:22.696 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:22.696 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:22.697 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:31:22.767 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:22 smithi070 ceph-mon[30388]: pgmap v297: 168 pgs: 1 active+recovery_wait+degraded, 3 active+recovering+undersized+remapped, 164 active+clean; 7.1 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 852 B/s rd, 54 MiB/s wr, 863 op/s; 1/347979 objects degraded (0.000%); 9523/347979 objects misplaced (2.737%); 4.1 MiB/s, 65 objects/s recovering 2024-09-16T09:31:22.767 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:22 smithi070 ceph-mon[40691]: pgmap v297: 168 pgs: 1 active+recovery_wait+degraded, 3 active+recovering+undersized+remapped, 164 active+clean; 7.1 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 852 B/s rd, 54 MiB/s wr, 863 op/s; 1/347979 objects degraded (0.000%); 9523/347979 objects misplaced (2.737%); 4.1 MiB/s, 65 objects/s recovering 2024-09-16T09:31:22.798 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:31:22.836 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:22.837 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:22.837 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:31:22.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:22 smithi084 ceph-mon[32629]: pgmap v297: 168 pgs: 1 active+recovery_wait+degraded, 3 active+recovering+undersized+remapped, 164 active+clean; 7.1 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 852 B/s rd, 54 MiB/s wr, 863 op/s; 1/347979 objects degraded (0.000%); 9523/347979 objects misplaced (2.737%); 4.1 MiB/s, 65 objects/s recovering 2024-09-16T09:31:22.910 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:31:22 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:31:22.938 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:31:22.966 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 87 16 121285 121269 87.1009 64.5 0.0132645 0.0114761 2024-09-16T09:31:22.976 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:22.977 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:22.977 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:31:23.078 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:31:23.117 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:23.117 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:23.118 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:31:23.218 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:31:23.225 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:31:23.225 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:31:23.228 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:31:23.228 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:31:23.246 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:23.247 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:23.247 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:31:23.261 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:31:23.261 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:31:23.295 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:31:23.295 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:31:23.333 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:31:23.333 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:31:23.348 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:31:23.373 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:31:23.373 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:31:23.390 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:23.390 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:23.391 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:31:23.420 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:31:23.420 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:31:23.454 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:31:23.455 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:31:23.491 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:31:23.500 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T09:31:23.500 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T09:31:23.500 INFO:teuthology.orchestra.run.smithi070.stdout: "num_ops": 0 2024-09-16T09:31:23.500 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T09:31:23.541 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:23.542 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:23.542 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:31:23.643 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:31:23.644 INFO:teuthology.orchestra.run.smithi070.stderr:noscrub is unset 2024-09-16T09:31:23.674 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:23.675 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:23.675 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:31:23.718 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:23 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:31:23.718 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:23 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/4216582930' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:31:23.719 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:23 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:31:23.719 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:23 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/80019623' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:31:23.719 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:23 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:31:23.719 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:23 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/4216582930' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:31:23.719 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:23 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:31:23.719 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:23 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/80019623' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:31:23.777 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:31:23.812 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:23.813 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:23.813 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:31:23.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:23 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:31:23.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:23 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/4216582930' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:31:23.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:23 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:31:23.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:23 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/80019623' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:31:23.915 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:31:23.953 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:23.954 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:23.954 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:31:23.966 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 88 15 122341 122326 86.8618 66.0625 0.0045687 0.0115078 2024-09-16T09:31:24.055 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:31:24.104 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:24.130 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:24.130 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:31:24.231 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:31:24.272 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:24.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:24.273 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:31:24.374 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:31:24.412 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:24.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:24.413 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:31:24.513 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:31:24.544 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:24.544 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:24.545 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:31:24.645 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:31:24.677 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:24.677 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:24.678 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:31:24.778 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:31:24.808 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:24.809 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:24.809 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:31:24.876 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:24 smithi070 ceph-mon[30388]: pgmap v298: 168 pgs: 1 active+recovery_wait+degraded, 3 active+recovering+undersized+remapped, 164 active+clean; 7.4 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.6 KiB/s rd, 105 MiB/s wr, 1.68k op/s; 1/364035 objects degraded (0.000%); 9076/364035 objects misplaced (2.493%); 8.4 MiB/s, 133 objects/s recovering 2024-09-16T09:31:24.876 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:24 smithi070 ceph-mon[30388]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:31:24.877 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:24 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-16T09:31:24.877 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:24 smithi070 ceph-mon[30388]: osdmap e193: 8 total, 8 up, 8 in 2024-09-16T09:31:24.877 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:24 smithi070 ceph-mon[40691]: pgmap v298: 168 pgs: 1 active+recovery_wait+degraded, 3 active+recovering+undersized+remapped, 164 active+clean; 7.4 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.6 KiB/s rd, 105 MiB/s wr, 1.68k op/s; 1/364035 objects degraded (0.000%); 9076/364035 objects misplaced (2.493%); 8.4 MiB/s, 133 objects/s recovering 2024-09-16T09:31:24.877 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:24 smithi070 ceph-mon[40691]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:31:24.877 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:24 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-16T09:31:24.877 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:24 smithi070 ceph-mon[40691]: osdmap e193: 8 total, 8 up, 8 in 2024-09-16T09:31:24.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:24 smithi084 ceph-mon[32629]: pgmap v298: 168 pgs: 1 active+recovery_wait+degraded, 3 active+recovering+undersized+remapped, 164 active+clean; 7.4 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.6 KiB/s rd, 105 MiB/s wr, 1.68k op/s; 1/364035 objects degraded (0.000%); 9076/364035 objects misplaced (2.493%); 8.4 MiB/s, 133 objects/s recovering 2024-09-16T09:31:24.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:24 smithi084 ceph-mon[32629]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:31:24.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:24 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-16T09:31:24.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:24 smithi084 ceph-mon[32629]: osdmap e193: 8 total, 8 up, 8 in 2024-09-16T09:31:24.910 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:31:24.950 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:24.950 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:24.950 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:31:24.965 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 89 15 123511 123496 86.7074 73.125 0.0109258 0.0115277 2024-09-16T09:31:25.051 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:31:25.080 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:25.081 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:25.082 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:31:25.183 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:31:25.219 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:25.220 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:25.220 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:31:25.238 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:31:25.277 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.3 dump_blocked_ops 2024-09-16T09:31:25.321 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:31:25.355 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:25.356 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:25.357 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:31:25.457 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:31:25.496 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:25.496 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:25.497 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:31:25.597 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:31:25.640 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:25.640 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:25.641 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:31:25.742 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:31:25.789 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:25.790 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:25.790 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:31:25.891 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:31:25.929 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:25.930 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:25.930 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:31:25.965 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Total time run: 90.0123 2024-09-16T09:31:25.966 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Total writes made: 124592 2024-09-16T09:31:25.966 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Write size: 65536 2024-09-16T09:31:25.966 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Object size: 65536 2024-09-16T09:31:25.966 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Bandwidth (MB/sec): 86.5104 2024-09-16T09:31:25.966 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Stddev Bandwidth: 36.6926 2024-09-16T09:31:25.966 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Max bandwidth (MB/sec): 199.938 2024-09-16T09:31:25.966 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Min bandwidth (MB/sec): 0 2024-09-16T09:31:25.966 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Average IOPS: 1384 2024-09-16T09:31:25.966 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Stddev IOPS: 587.081 2024-09-16T09:31:25.966 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Max IOPS: 3199 2024-09-16T09:31:25.966 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Min IOPS: 0 2024-09-16T09:31:25.966 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Average Latency(s): 0.0115549 2024-09-16T09:31:25.966 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Stddev Latency(s): 0.0289272 2024-09-16T09:31:25.967 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Max latency(s): 2.20004 2024-09-16T09:31:25.967 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Min latency(s): 0.00116281 2024-09-16T09:31:25.969 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Cleaning up (deleting benchmark objects) 2024-09-16T09:31:26.030 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:31:26.077 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:26.077 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:26.078 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:31:26.178 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:31:26.217 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:26.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:26.218 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:31:26.319 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:31:26.374 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:31:26.375 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:26.375 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:26.375 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:31:26.476 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:31:26.506 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:26.506 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:26.506 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:31:26.600 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T09:31:26.608 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:31:26.641 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:26.641 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:26.641 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:31:26.742 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:31:26.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:26.774 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:26.774 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:31:26.875 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:31:26.906 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:26.907 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:26.907 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:31:26.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:26 smithi084 ceph-mon[32629]: pgmap v300: 168 pgs: 1 active+recovery_wait+degraded, 3 active+recovering+undersized+remapped, 164 active+clean; 7.4 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 850 B/s rd, 56 MiB/s wr, 890 op/s; 1/364035 objects degraded (0.000%); 9076/364035 objects misplaced (2.493%); 4.6 MiB/s, 74 objects/s recovering 2024-09-16T09:31:26.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:26 smithi084 ceph-mon[32629]: 3.18 scrub starts 2024-09-16T09:31:26.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:26 smithi084 ceph-mon[32629]: 3.18 scrub ok 2024-09-16T09:31:26.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:26 smithi070 ceph-mon[30388]: pgmap v300: 168 pgs: 1 active+recovery_wait+degraded, 3 active+recovering+undersized+remapped, 164 active+clean; 7.4 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 850 B/s rd, 56 MiB/s wr, 890 op/s; 1/364035 objects degraded (0.000%); 9076/364035 objects misplaced (2.493%); 4.6 MiB/s, 74 objects/s recovering 2024-09-16T09:31:26.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:26 smithi070 ceph-mon[30388]: 3.18 scrub starts 2024-09-16T09:31:26.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:26 smithi070 ceph-mon[30388]: 3.18 scrub ok 2024-09-16T09:31:26.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:26 smithi070 ceph-mon[40691]: pgmap v300: 168 pgs: 1 active+recovery_wait+degraded, 3 active+recovering+undersized+remapped, 164 active+clean; 7.4 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 850 B/s rd, 56 MiB/s wr, 890 op/s; 1/364035 objects degraded (0.000%); 9076/364035 objects misplaced (2.493%); 4.6 MiB/s, 74 objects/s recovering 2024-09-16T09:31:26.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:26 smithi070 ceph-mon[40691]: 3.18 scrub starts 2024-09-16T09:31:26.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:26 smithi070 ceph-mon[40691]: 3.18 scrub ok 2024-09-16T09:31:27.009 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:31:27.043 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:27.044 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:27.044 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:31:27.145 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:31:27.176 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:27.177 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:27.177 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:31:27.278 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:31:27.311 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:27.312 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:27.312 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:31:27.329 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T09:31:27.329 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T09:31:27.329 INFO:teuthology.orchestra.run.smithi070.stdout: "complaint_time": 30, 2024-09-16T09:31:27.329 INFO:teuthology.orchestra.run.smithi070.stdout: "num_blocked_ops": 0 2024-09-16T09:31:27.329 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T09:31:27.413 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:31:27.445 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:27.445 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:27.446 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:31:27.546 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:31:27.582 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:27.583 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:27.583 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:31:27.658 INFO:teuthology.orchestra.run.smithi070.stderr:reweighted osd.6 to 1 (10000) 2024-09-16T09:31:27.684 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:31:27.714 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:27.714 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:27.714 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:31:27.751 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-16T09:31:27.815 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:31:27.836 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:27.837 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:27.837 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:31:27.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:27 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1643214919' entity='client.admin' cmd={"prefix": "osd reweight", "id": 6, "weight": 1.0} : dispatch 2024-09-16T09:31:27.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:27 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 6, "weight": 1.0} : dispatch 2024-09-16T09:31:27.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:27 smithi070 ceph-mon[30388]: 6.9 scrub starts 2024-09-16T09:31:27.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:27 smithi070 ceph-mon[30388]: 6.9 scrub ok 2024-09-16T09:31:27.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:27 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1643214919' entity='client.admin' cmd={"prefix": "osd reweight", "id": 6, "weight": 1.0} : dispatch 2024-09-16T09:31:27.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:27 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 6, "weight": 1.0} : dispatch 2024-09-16T09:31:27.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:27 smithi070 ceph-mon[40691]: 6.9 scrub starts 2024-09-16T09:31:27.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:27 smithi070 ceph-mon[40691]: 6.9 scrub ok 2024-09-16T09:31:27.939 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:31:27.971 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:27.972 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:27.972 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:31:28.073 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:31:28.115 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:28.116 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:28.117 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:31:28.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:27 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1643214919' entity='client.admin' cmd={"prefix": "osd reweight", "id": 6, "weight": 1.0} : dispatch 2024-09-16T09:31:28.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:27 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 6, "weight": 1.0} : dispatch 2024-09-16T09:31:28.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:27 smithi084 ceph-mon[32629]: 6.9 scrub starts 2024-09-16T09:31:28.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:27 smithi084 ceph-mon[32629]: 6.9 scrub ok 2024-09-16T09:31:28.217 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:31:28.232 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.3 dump_historic_ops 2024-09-16T09:31:28.260 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:28.260 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:28.260 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:31:28.361 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:31:28.400 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:28.401 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:28.401 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:31:28.502 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:31:28.552 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:28.553 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:28.553 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:31:28.569 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:31:28.654 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:31:28.705 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:28.706 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:28.706 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:31:28.807 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:31:28.843 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:28.844 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:28.844 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:31:28.945 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:31:28.983 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:28.983 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:28.984 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:31:28.994 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:28 smithi070 ceph-mon[30388]: pgmap v301: 168 pgs: 1 active+recovery_wait+degraded, 3 active+recovering+undersized+remapped, 164 active+clean; 7.4 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.2 KiB/s rd, 45 MiB/s wr, 753 op/s; 1/364677 objects degraded (0.000%); 9076/364677 objects misplaced (2.489%); 3.5 MiB/s, 55 objects/s recovering 2024-09-16T09:31:28.994 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:28 smithi070 ceph-mon[30388]: Health check update: Degraded data redundancy: 1/364677 objects degraded (0.000%), 1 pg degraded (PG_DEGRADED) 2024-09-16T09:31:28.994 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:28 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-16T09:31:28.994 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:28 smithi070 ceph-mon[30388]: osdmap e194: 8 total, 8 up, 8 in 2024-09-16T09:31:28.994 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:28 smithi070 ceph-mon[40691]: pgmap v301: 168 pgs: 1 active+recovery_wait+degraded, 3 active+recovering+undersized+remapped, 164 active+clean; 7.4 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.2 KiB/s rd, 45 MiB/s wr, 753 op/s; 1/364677 objects degraded (0.000%); 9076/364677 objects misplaced (2.489%); 3.5 MiB/s, 55 objects/s recovering 2024-09-16T09:31:28.994 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:28 smithi070 ceph-mon[40691]: Health check update: Degraded data redundancy: 1/364677 objects degraded (0.000%), 1 pg degraded (PG_DEGRADED) 2024-09-16T09:31:28.995 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:28 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-16T09:31:28.995 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:28 smithi070 ceph-mon[40691]: osdmap e194: 8 total, 8 up, 8 in 2024-09-16T09:31:29.003 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:31:29.047 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd reweight 7 1 2024-09-16T09:31:29.084 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:31:29.116 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:29.117 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:29.117 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:31:29.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:28 smithi084 ceph-mon[32629]: pgmap v301: 168 pgs: 1 active+recovery_wait+degraded, 3 active+recovering+undersized+remapped, 164 active+clean; 7.4 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.2 KiB/s rd, 45 MiB/s wr, 753 op/s; 1/364677 objects degraded (0.000%); 9076/364677 objects misplaced (2.489%); 3.5 MiB/s, 55 objects/s recovering 2024-09-16T09:31:29.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:28 smithi084 ceph-mon[32629]: Health check update: Degraded data redundancy: 1/364677 objects degraded (0.000%), 1 pg degraded (PG_DEGRADED) 2024-09-16T09:31:29.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:28 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-16T09:31:29.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:28 smithi084 ceph-mon[32629]: osdmap e194: 8 total, 8 up, 8 in 2024-09-16T09:31:29.219 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:31:29.252 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:29.253 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:29.253 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:31:29.354 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:31:29.385 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:29.386 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:29.386 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:31:29.487 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:31:29.524 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:29.524 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:29.525 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:31:29.626 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:31:29.661 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:29.661 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:29.662 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:31:29.763 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:31:29.799 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:29.800 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:29.800 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:31:29.901 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:31:29.929 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:29.929 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:29.930 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:31:30.030 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:31:30.064 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:30.065 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:30.065 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:31:30.166 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:31:30.194 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:30.195 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:30.195 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:31:30.296 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:31:30.320 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:30.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:30.321 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:31:30.347 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:31:30.348 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:31:30.373 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:31:30.395 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:31:30.396 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:31:30.403 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:31:30.421 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:31:30.440 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:31:30.440 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:31:30.444 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:30.445 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:30.445 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:31:30.480 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:31:30.480 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:31:30.521 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:31:30.521 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:31:30.546 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:31:30.555 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:31:30.555 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:31:30.585 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:30.585 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:31:30.586 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:31:30.586 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:30.586 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:31:30.617 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:31:30.617 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:31:30.677 INFO:teuthology.orchestra.run.smithi070.stderr:nodeep-scrub is unset 2024-09-16T09:31:30.687 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:31:30.708 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:30.708 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:30.709 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:31:30.809 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:31:30.843 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:30.843 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:30.844 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:31:30.876 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:30 smithi070 ceph-mon[30388]: pgmap v303: 168 pgs: 1 active+recovery_wait+degraded, 3 active+recovering+undersized+remapped, 164 active+clean; 7.5 GiB data, 25 GiB used, 691 GiB / 715 GiB avail; 1.4 KiB/s rd, 69 MiB/s wr, 1.36k op/s; 1/368235 objects degraded (0.000%); 8591/368235 objects misplaced (2.333%); 7.3 MiB/s, 116 objects/s recovering 2024-09-16T09:31:30.876 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:30 smithi070 ceph-mon[30388]: 3.17 scrub starts 2024-09-16T09:31:30.876 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:30 smithi070 ceph-mon[30388]: 3.17 scrub ok 2024-09-16T09:31:30.877 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:30 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2666530067' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:31:30.877 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:30 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/981791922' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:31:30.877 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:30 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:31:30.877 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:30 smithi070 ceph-mon[40691]: pgmap v303: 168 pgs: 1 active+recovery_wait+degraded, 3 active+recovering+undersized+remapped, 164 active+clean; 7.5 GiB data, 25 GiB used, 691 GiB / 715 GiB avail; 1.4 KiB/s rd, 69 MiB/s wr, 1.36k op/s; 1/368235 objects degraded (0.000%); 8591/368235 objects misplaced (2.333%); 7.3 MiB/s, 116 objects/s recovering 2024-09-16T09:31:30.877 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:30 smithi070 ceph-mon[40691]: 3.17 scrub starts 2024-09-16T09:31:30.877 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:30 smithi070 ceph-mon[40691]: 3.17 scrub ok 2024-09-16T09:31:30.877 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:30 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/2666530067' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:31:30.877 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:30 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/981791922' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:31:30.877 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:30 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:31:30.945 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:31:30.977 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:30.978 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:30.978 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:31:31.079 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:31:31.115 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:31.116 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:31.116 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:31:31.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:30 smithi084 ceph-mon[32629]: pgmap v303: 168 pgs: 1 active+recovery_wait+degraded, 3 active+recovering+undersized+remapped, 164 active+clean; 7.5 GiB data, 25 GiB used, 691 GiB / 715 GiB avail; 1.4 KiB/s rd, 69 MiB/s wr, 1.36k op/s; 1/368235 objects degraded (0.000%); 8591/368235 objects misplaced (2.333%); 7.3 MiB/s, 116 objects/s recovering 2024-09-16T09:31:31.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:30 smithi084 ceph-mon[32629]: 3.17 scrub starts 2024-09-16T09:31:31.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:30 smithi084 ceph-mon[32629]: 3.17 scrub ok 2024-09-16T09:31:31.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:30 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/2666530067' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:31:31.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:30 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/981791922' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:31:31.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:30 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:31:31.217 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:31:31.248 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:31.249 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:31.249 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:31:31.350 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:31:31.357 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T09:31:31.357 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:31:31.383 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:31.384 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:31.385 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:31:31.485 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:31:31.522 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:31.522 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:31.522 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:31:31.623 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:31:31.663 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:31.663 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:31.664 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:31:31.764 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:31:31.799 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:31.800 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:31.800 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:31:31.901 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:31:31.936 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:31.937 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:31.937 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:31:32.009 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:31 smithi070 ceph-mon[30388]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-16T09:31:32.009 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:31 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-16T09:31:32.009 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:31 smithi070 ceph-mon[30388]: osdmap e195: 8 total, 8 up, 8 in 2024-09-16T09:31:32.009 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:31 smithi070 ceph-mon[30388]: 4.b scrub starts 2024-09-16T09:31:32.009 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:31 smithi070 ceph-mon[30388]: 4.b scrub ok 2024-09-16T09:31:32.009 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:31 smithi070 ceph-mon[40691]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-16T09:31:32.009 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:31 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-16T09:31:32.010 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:31 smithi070 ceph-mon[40691]: osdmap e195: 8 total, 8 up, 8 in 2024-09-16T09:31:32.010 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:31 smithi070 ceph-mon[40691]: 4.b scrub starts 2024-09-16T09:31:32.010 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:31 smithi070 ceph-mon[40691]: 4.b scrub ok 2024-09-16T09:31:32.010 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:31:31 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:31:31] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:31:32.038 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:31:32.071 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:32.071 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:32.072 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:31:32.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:31 smithi084 ceph-mon[32629]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-16T09:31:32.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:31 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-16T09:31:32.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:31 smithi084 ceph-mon[32629]: osdmap e195: 8 total, 8 up, 8 in 2024-09-16T09:31:32.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:31 smithi084 ceph-mon[32629]: 4.b scrub starts 2024-09-16T09:31:32.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:31 smithi084 ceph-mon[32629]: 4.b scrub ok 2024-09-16T09:31:32.173 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:31:32.205 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:32.205 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:32.205 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:31:32.306 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:31:32.339 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:32.340 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:32.340 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:31:32.441 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:31:32.473 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:32.474 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:32.474 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:31:32.575 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:31:32.608 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:32.608 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:32.609 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:31:32.710 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:31:32.745 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:32.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:32.746 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:31:32.847 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:31:32.884 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:32.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:32.885 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:31:32.986 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:31:33.014 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:33.014 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:33.015 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:31:33.115 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:31:33.155 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:33.156 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:33.156 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:31:33.160 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:31:32 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:31:33.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:32 smithi084 ceph-mon[32629]: pgmap v305: 168 pgs: 1 active+recovery_wait+degraded, 3 active+recovering+undersized+remapped, 164 active+clean; 7.5 GiB data, 25 GiB used, 691 GiB / 715 GiB avail; 770 B/s rd, 27 MiB/s wr, 689 op/s; 1/368235 objects degraded (0.000%); 8591/368235 objects misplaced (2.333%); 3.8 MiB/s, 60 objects/s recovering 2024-09-16T09:31:33.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:32 smithi084 ceph-mon[32629]: 3.b scrub starts 2024-09-16T09:31:33.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:32 smithi084 ceph-mon[32629]: 3.b scrub ok 2024-09-16T09:31:33.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:32 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:31:33.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:32 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:31:33.170 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:32 smithi070 ceph-mon[30388]: pgmap v305: 168 pgs: 1 active+recovery_wait+degraded, 3 active+recovering+undersized+remapped, 164 active+clean; 7.5 GiB data, 25 GiB used, 691 GiB / 715 GiB avail; 770 B/s rd, 27 MiB/s wr, 689 op/s; 1/368235 objects degraded (0.000%); 8591/368235 objects misplaced (2.333%); 3.8 MiB/s, 60 objects/s recovering 2024-09-16T09:31:33.170 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:32 smithi070 ceph-mon[30388]: 3.b scrub starts 2024-09-16T09:31:33.170 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:32 smithi070 ceph-mon[30388]: 3.b scrub ok 2024-09-16T09:31:33.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:32 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:31:33.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:32 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:31:33.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:32 smithi070 ceph-mon[40691]: pgmap v305: 168 pgs: 1 active+recovery_wait+degraded, 3 active+recovering+undersized+remapped, 164 active+clean; 7.5 GiB data, 25 GiB used, 691 GiB / 715 GiB avail; 770 B/s rd, 27 MiB/s wr, 689 op/s; 1/368235 objects degraded (0.000%); 8591/368235 objects misplaced (2.333%); 3.8 MiB/s, 60 objects/s recovering 2024-09-16T09:31:33.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:32 smithi070 ceph-mon[40691]: 3.b scrub starts 2024-09-16T09:31:33.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:32 smithi070 ceph-mon[40691]: 3.b scrub ok 2024-09-16T09:31:33.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:32 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:31:33.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:32 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:31:33.258 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:31:33.296 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:33.296 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:33.297 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:31:33.398 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:31:33.436 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:33.437 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:33.438 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:31:33.539 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:31:33.572 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:33.573 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:33.573 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:31:33.674 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:31:33.714 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:33.715 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:33.715 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:31:33.816 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:31:33.849 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:33.850 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:33.850 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:31:33.909 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T09:31:33.950 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:33 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:31:33.950 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:33 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:31:33.951 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:31:33.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:33.982 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:33.982 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:31:34.083 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:31:34.117 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:34.118 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:34.118 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:31:34.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:33 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:31:34.219 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:31:34.259 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:34.259 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:34.260 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:31:34.360 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:31:34.402 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:34.402 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:34.403 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:31:34.503 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:31:34.556 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:34.557 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:34.557 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:31:34.658 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:31:34.689 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:34.689 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:34.690 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:31:34.790 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:31:34.835 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.4 dump_ops_in_flight 2024-09-16T09:31:34.845 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:34.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:34.847 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:31:34.936 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:31:34.947 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:31:34.989 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:34.990 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:34.990 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:31:35.068 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:34 smithi070 ceph-mon[30388]: pgmap v306: 168 pgs: 1 active+recovery_wait+degraded, 3 active+recovering+undersized+remapped, 164 active+clean; 7.1 GiB data, 24 GiB used, 692 GiB / 715 GiB avail; 1.4 KiB/s rd, 27 MiB/s wr, 1.39k op/s; 1/351348 objects degraded (0.000%); 8108/351348 objects misplaced (2.308%); 7.6 MiB/s, 120 objects/s recovering 2024-09-16T09:31:35.068 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:34 smithi070 ceph-mon[30388]: Health check update: Degraded data redundancy: 1/351348 objects degraded (0.000%), 1 pg degraded (PG_DEGRADED) 2024-09-16T09:31:35.069 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:34 smithi070 ceph-mon[40691]: pgmap v306: 168 pgs: 1 active+recovery_wait+degraded, 3 active+recovering+undersized+remapped, 164 active+clean; 7.1 GiB data, 24 GiB used, 692 GiB / 715 GiB avail; 1.4 KiB/s rd, 27 MiB/s wr, 1.39k op/s; 1/351348 objects degraded (0.000%); 8108/351348 objects misplaced (2.308%); 7.6 MiB/s, 120 objects/s recovering 2024-09-16T09:31:35.069 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:34 smithi070 ceph-mon[40691]: Health check update: Degraded data redundancy: 1/351348 objects degraded (0.000%), 1 pg degraded (PG_DEGRADED) 2024-09-16T09:31:35.091 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:31:35.123 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:35.124 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:35.124 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:31:35.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:34 smithi084 ceph-mon[32629]: pgmap v306: 168 pgs: 1 active+recovery_wait+degraded, 3 active+recovering+undersized+remapped, 164 active+clean; 7.1 GiB data, 24 GiB used, 692 GiB / 715 GiB avail; 1.4 KiB/s rd, 27 MiB/s wr, 1.39k op/s; 1/351348 objects degraded (0.000%); 8108/351348 objects misplaced (2.308%); 7.6 MiB/s, 120 objects/s recovering 2024-09-16T09:31:35.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:34 smithi084 ceph-mon[32629]: Health check update: Degraded data redundancy: 1/351348 objects degraded (0.000%), 1 pg degraded (PG_DEGRADED) 2024-09-16T09:31:35.225 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:31:35.225 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:31:35.261 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:35.261 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:35.262 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:31:35.362 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:31:35.406 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:35.407 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:35.407 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:31:35.508 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:31:35.540 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:35.541 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:35.541 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:31:35.642 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:31:35.677 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:35.677 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:35.678 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:31:35.718 INFO:teuthology.orchestra.run.smithi070.stderr:reweighted osd.7 to 1 (10000) 2024-09-16T09:31:35.778 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:31:35.811 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:35.812 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:35.812 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:31:35.913 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:31:35.937 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:35.938 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:35.938 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:31:35.948 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd set noscrub 2024-09-16T09:31:35.979 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:35 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/4232749439' entity='client.admin' cmd={"prefix": "osd reweight", "id": 7, "weight": 1.0} : dispatch 2024-09-16T09:31:35.979 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:35 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 7, "weight": 1.0} : dispatch 2024-09-16T09:31:35.979 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:35 smithi070 ceph-mon[30388]: 4.3 scrub starts 2024-09-16T09:31:35.980 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:35 smithi070 ceph-mon[30388]: 4.3 scrub ok 2024-09-16T09:31:35.980 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:35 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/4232749439' entity='client.admin' cmd={"prefix": "osd reweight", "id": 7, "weight": 1.0} : dispatch 2024-09-16T09:31:35.980 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:35 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 7, "weight": 1.0} : dispatch 2024-09-16T09:31:35.980 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:35 smithi070 ceph-mon[40691]: 4.3 scrub starts 2024-09-16T09:31:35.980 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:35 smithi070 ceph-mon[40691]: 4.3 scrub ok 2024-09-16T09:31:36.039 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:31:36.072 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:36.073 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:36.073 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:31:36.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:35 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/4232749439' entity='client.admin' cmd={"prefix": "osd reweight", "id": 7, "weight": 1.0} : dispatch 2024-09-16T09:31:36.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:35 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 7, "weight": 1.0} : dispatch 2024-09-16T09:31:36.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:35 smithi084 ceph-mon[32629]: 4.3 scrub starts 2024-09-16T09:31:36.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:35 smithi084 ceph-mon[32629]: 4.3 scrub ok 2024-09-16T09:31:36.174 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:31:36.207 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:36.208 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:36.208 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:31:36.309 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:31:36.342 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:36.343 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:36.343 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:31:36.444 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:31:36.472 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:36.473 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:36.473 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:31:36.574 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:31:36.607 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:36.608 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:36.608 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:31:36.709 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:31:36.736 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:36.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:36.737 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:31:36.838 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:31:36.877 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:36.878 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:36.879 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:31:36.980 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:31:37.012 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:37.012 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:37.012 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:31:37.113 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:31:37.144 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:37.145 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:37.145 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:31:37.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:36 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:31:37.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:36 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:31:37.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:36 smithi084 ceph-mon[32629]: pgmap v307: 168 pgs: 1 active+recovery_wait+degraded, 3 active+recovering+undersized+remapped, 164 active+clean; 7.1 GiB data, 24 GiB used, 692 GiB / 715 GiB avail; 767 B/s rd, 23 MiB/s wr, 1.31k op/s; 1/351348 objects degraded (0.000%); 8108/351348 objects misplaced (2.308%); 7.6 MiB/s, 120 objects/s recovering 2024-09-16T09:31:37.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:36 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-16T09:31:37.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:36 smithi084 ceph-mon[32629]: osdmap e196: 8 total, 8 up, 8 in 2024-09-16T09:31:37.170 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:36 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:31:37.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:36 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:31:37.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:36 smithi070 ceph-mon[30388]: pgmap v307: 168 pgs: 1 active+recovery_wait+degraded, 3 active+recovering+undersized+remapped, 164 active+clean; 7.1 GiB data, 24 GiB used, 692 GiB / 715 GiB avail; 767 B/s rd, 23 MiB/s wr, 1.31k op/s; 1/351348 objects degraded (0.000%); 8108/351348 objects misplaced (2.308%); 7.6 MiB/s, 120 objects/s recovering 2024-09-16T09:31:37.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:36 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-16T09:31:37.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:36 smithi070 ceph-mon[30388]: osdmap e196: 8 total, 8 up, 8 in 2024-09-16T09:31:37.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:36 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:31:37.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:36 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:31:37.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:36 smithi070 ceph-mon[40691]: pgmap v307: 168 pgs: 1 active+recovery_wait+degraded, 3 active+recovering+undersized+remapped, 164 active+clean; 7.1 GiB data, 24 GiB used, 692 GiB / 715 GiB avail; 767 B/s rd, 23 MiB/s wr, 1.31k op/s; 1/351348 objects degraded (0.000%); 8108/351348 objects misplaced (2.308%); 7.6 MiB/s, 120 objects/s recovering 2024-09-16T09:31:37.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:36 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-16T09:31:37.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:36 smithi070 ceph-mon[40691]: osdmap e196: 8 total, 8 up, 8 in 2024-09-16T09:31:37.246 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:31:37.272 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:37.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:37.273 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:31:37.374 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:31:37.401 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:37.402 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:37.403 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:31:37.503 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:31:37.539 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:37.540 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:37.540 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:31:37.641 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:31:37.662 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:37.663 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:37.663 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:31:37.764 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:31:37.798 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:37.799 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:37.800 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:31:37.900 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:31:37.932 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:37.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:37.933 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:31:38.034 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:31:38.060 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:38.061 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:38.061 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:31:38.162 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:31:38.186 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:38.186 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:38.186 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:31:38.189 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:31:38.189 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:31:38.206 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:31:38.206 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:31:38.221 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:31:38.222 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:31:38.237 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:31:38.238 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:31:38.272 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:31:38.272 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:31:38.287 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:31:38.307 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:31:38.308 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:31:38.323 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:38.323 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:38.324 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:31:38.342 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:31:38.342 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:31:38.374 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:31:38.374 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:31:38.425 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:31:38.458 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:38.459 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:38.459 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:31:38.560 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:31:38.589 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:38.589 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:38.590 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:31:38.623 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-09-16T09:31:38.623 INFO:teuthology.orchestra.run.smithi084.stdout: "ops": [ 2024-09-16T09:31:38.623 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:38.624 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "MOSDPGPushReply(8.b 196/186 [PushReplyOp(8:d6aae8ed:::benchmark_data_smithi070_107919_object73097:head)])", 2024-09-16T09:31:38.624 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:31:38.618030+0000", 2024-09-16T09:31:38.624 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 0.002604019, 2024-09-16T09:31:38.624 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0026275560000000001, 2024-09-16T09:31:38.624 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:31:38.624 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:31:38.624 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "reached pg", 2024-09-16T09:31:38.624 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:31:38.624 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:38.624 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:31:38.624 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:38.618030+0000", 2024-09-16T09:31:38.624 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:31:38.625 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:38.625 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:38.625 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:31:38.625 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:38.618030+0000", 2024-09-16T09:31:38.625 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:31:38.625 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:38.625 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:38.625 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:31:38.625 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:38.618031+0000", 2024-09-16T09:31:38.625 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.1200000000000001e-07 2024-09-16T09:31:38.625 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:38.625 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:38.626 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:31:38.626 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:38.618037+0000", 2024-09-16T09:31:38.626 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.7180000000000001e-06 2024-09-16T09:31:38.626 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:38.626 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:38.626 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:31:38.626 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:38.618050+0000", 2024-09-16T09:31:38.626 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.2700999999999999e-05 2024-09-16T09:31:38.626 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:38.626 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:38.627 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:31:38.627 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:38.618057+0000", 2024-09-16T09:31:38.627 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.6569999999999997e-06 2024-09-16T09:31:38.627 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:38.627 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:38.627 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:31:38.627 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:38.620021+0000", 2024-09-16T09:31:38.627 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0019633889999999998 2024-09-16T09:31:38.627 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:31:38.627 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:31:38.627 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:31:38.627 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:38.627 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:38.627 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.25064.0:138157 8.2 8:483e7e4b:::benchmark_data_smithi070_107919_object13562:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e196)", 2024-09-16T09:31:38.628 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:31:38.619838+0000", 2024-09-16T09:31:38.628 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 0.00079607199999999999, 2024-09-16T09:31:38.628 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00086980599999999999, 2024-09-16T09:31:38.628 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:31:38.628 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:31:38.628 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "waiting for sub ops", 2024-09-16T09:31:38.628 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:31:38.628 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.25064", 2024-09-16T09:31:38.628 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/2578256439", 2024-09-16T09:31:38.628 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 138157 2024-09-16T09:31:38.628 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:38.628 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:31:38.628 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:38.629 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:31:38.629 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:38.619838+0000", 2024-09-16T09:31:38.629 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:31:38.629 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:38.629 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:38.629 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:31:38.629 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:38.619838+0000", 2024-09-16T09:31:38.629 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:31:38.629 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:38.629 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:38.629 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:31:38.629 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:38.619840+0000", 2024-09-16T09:31:38.630 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.3340000000000001e-06 2024-09-16T09:31:38.630 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:38.630 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:38.630 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:31:38.630 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:38.619845+0000", 2024-09-16T09:31:38.630 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.772e-06 2024-09-16T09:31:38.630 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:38.630 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:38.630 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:31:38.630 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:38.619860+0000", 2024-09-16T09:31:38.630 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.466e-05 2024-09-16T09:31:38.630 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:38.630 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:38.630 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:31:38.631 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:38.619868+0000", 2024-09-16T09:31:38.631 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.3350000000000007e-06 2024-09-16T09:31:38.631 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:38.631 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:38.631 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:31:38.631 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:38.619936+0000", 2024-09-16T09:31:38.631 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.7259000000000001e-05 2024-09-16T09:31:38.631 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:38.631 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:38.631 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:31:38.631 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:38.620081+0000", 2024-09-16T09:31:38.631 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000145575 2024-09-16T09:31:38.631 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:38.631 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:38.631 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,5", 2024-09-16T09:31:38.632 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:38.620198+0000", 2024-09-16T09:31:38.632 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00011647899999999999 2024-09-16T09:31:38.632 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:31:38.632 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:31:38.632 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:31:38.632 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:31:38.632 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-09-16T09:31:38.632 INFO:teuthology.orchestra.run.smithi084.stdout: "num_ops": 2 2024-09-16T09:31:38.632 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-09-16T09:31:38.691 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:31:38.737 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:38.738 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:38.738 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:31:38.839 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:31:38.869 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:38.870 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:38.870 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:31:38.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:38 smithi084 ceph-mon[32629]: pgmap v309: 168 pgs: 1 active+recovery_wait+degraded, 3 active+recovering+undersized+remapped, 164 active+clean; 7.1 GiB data, 24 GiB used, 692 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 786 op/s; 1/349374 objects degraded (0.000%); 8108/349374 objects misplaced (2.321%); 3.8 MiB/s, 60 objects/s recovering 2024-09-16T09:31:38.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:38 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1899918527' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:31:38.971 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:31:39.007 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:39.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:39.008 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:31:39.109 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:31:39.141 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:39.142 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:39.142 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:31:39.157 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:38 smithi070 ceph-mon[30388]: pgmap v309: 168 pgs: 1 active+recovery_wait+degraded, 3 active+recovering+undersized+remapped, 164 active+clean; 7.1 GiB data, 24 GiB used, 692 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 786 op/s; 1/349374 objects degraded (0.000%); 8108/349374 objects misplaced (2.321%); 3.8 MiB/s, 60 objects/s recovering 2024-09-16T09:31:39.157 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:38 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1899918527' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:31:39.158 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:38 smithi070 ceph-mon[40691]: pgmap v309: 168 pgs: 1 active+recovery_wait+degraded, 3 active+recovering+undersized+remapped, 164 active+clean; 7.1 GiB data, 24 GiB used, 692 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 786 op/s; 1/349374 objects degraded (0.000%); 8108/349374 objects misplaced (2.321%); 3.8 MiB/s, 60 objects/s recovering 2024-09-16T09:31:39.158 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:38 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1899918527' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:31:39.207 INFO:tasks.cephadm.ceph_manager.ceph:waiting for recovery to complete 2024-09-16T09:31:39.208 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-16T09:31:39.243 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:31:39.287 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:39.288 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:39.288 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:31:39.389 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:31:39.421 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:39.424 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:39.425 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:31:39.524 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.4 dump_blocked_ops 2024-09-16T09:31:39.525 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:31:39.566 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:39.567 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:39.567 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:31:39.668 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:31:39.701 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:31:39.733 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:39.734 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:39.734 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:31:39.738 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:31:39.836 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:31:39.876 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:39.876 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:39.877 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:31:39.879 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:39 smithi070 ceph-mon[30388]: Health check update: Degraded data redundancy: 1/349374 objects degraded (0.000%), 1 pg degraded (PG_DEGRADED) 2024-09-16T09:31:39.880 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:39 smithi070 ceph-mon[40691]: Health check update: Degraded data redundancy: 1/349374 objects degraded (0.000%), 1 pg degraded (PG_DEGRADED) 2024-09-16T09:31:39.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:39 smithi084 ceph-mon[32629]: Health check update: Degraded data redundancy: 1/349374 objects degraded (0.000%), 1 pg degraded (PG_DEGRADED) 2024-09-16T09:31:39.923 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:31:39.977 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:31:40.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:40.020 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:40.020 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:31:40.121 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:31:40.154 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:40.154 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:40.155 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:31:40.255 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:31:40.286 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:40.286 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:40.287 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:31:40.388 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:31:40.412 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:40.413 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:40.413 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:31:40.514 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:31:40.541 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:40.542 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:40.542 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:31:40.643 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:31:40.686 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:40.687 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:40.687 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:31:40.788 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:31:40.823 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:40.824 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:40.824 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:31:40.885 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-09-16T09:31:40.885 INFO:teuthology.orchestra.run.smithi084.stdout: "ops": [], 2024-09-16T09:31:40.885 INFO:teuthology.orchestra.run.smithi084.stdout: "complaint_time": 30, 2024-09-16T09:31:40.885 INFO:teuthology.orchestra.run.smithi084.stdout: "num_blocked_ops": 0 2024-09-16T09:31:40.885 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-09-16T09:31:40.925 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:31:40.959 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:40.960 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:40.960 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:31:41.061 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:31:41.098 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:41.098 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:41.099 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:31:41.138 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:31:41.141 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2024-09-16T09:31:41.199 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:31:41.209 INFO:teuthology.orchestra.run.smithi070.stderr:noscrub is set 2024-09-16T09:31:41.243 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:41.244 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:41.244 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:31:41.345 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:31:41.384 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:41.384 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:41.385 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:31:41.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:41 smithi070 ceph-mon[30388]: pgmap v310: 168 pgs: 1 active+recovery_wait+degraded, 3 active+recovering+undersized+remapped, 164 active+clean; 6.8 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.20k op/s; 1/335994 objects degraded (0.000%); 7625/335994 objects misplaced (2.269%); 6.8 MiB/s, 108 objects/s recovering 2024-09-16T09:31:41.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:41 smithi070 ceph-mon[30388]: 1.3 scrub starts 2024-09-16T09:31:41.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:41 smithi070 ceph-mon[30388]: 1.3 scrub ok 2024-09-16T09:31:41.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:41 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/905557029' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:31:41.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:41 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:31:41.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:41 smithi070 ceph-mon[40691]: pgmap v310: 168 pgs: 1 active+recovery_wait+degraded, 3 active+recovering+undersized+remapped, 164 active+clean; 6.8 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.20k op/s; 1/335994 objects degraded (0.000%); 7625/335994 objects misplaced (2.269%); 6.8 MiB/s, 108 objects/s recovering 2024-09-16T09:31:41.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:41 smithi070 ceph-mon[40691]: 1.3 scrub starts 2024-09-16T09:31:41.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:41 smithi070 ceph-mon[40691]: 1.3 scrub ok 2024-09-16T09:31:41.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:41 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/905557029' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:31:41.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:41 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:31:41.486 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:31:41.518 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:41.519 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:41.520 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:31:41.553 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:41 smithi084 ceph-mon[32629]: pgmap v310: 168 pgs: 1 active+recovery_wait+degraded, 3 active+recovering+undersized+remapped, 164 active+clean; 6.8 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.20k op/s; 1/335994 objects degraded (0.000%); 7625/335994 objects misplaced (2.269%); 6.8 MiB/s, 108 objects/s recovering 2024-09-16T09:31:41.553 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:41 smithi084 ceph-mon[32629]: 1.3 scrub starts 2024-09-16T09:31:41.553 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:41 smithi084 ceph-mon[32629]: 1.3 scrub ok 2024-09-16T09:31:41.554 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:41 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/905557029' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:31:41.554 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:41 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:31:41.603 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.4 dump_historic_ops 2024-09-16T09:31:41.620 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:31:41.650 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T09:31:41.653 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:41.654 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:41.654 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:31:41.755 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:31:41.814 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:41.815 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:41.815 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:31:41.916 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:31:41.953 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:41.954 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:41.954 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:31:41.989 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:31:42.022 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":310,"stamp":"2024-09-16T09:31:39.617688+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":7327541290,"num_objects":111998,"num_object_clones":0,"num_object_copies":335994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":7625,"num_objects_unfound":0,"num_objects_dirty":111998,"num_whiteouts":0,"num_read":1127,"num_read_kb":898,"num_write":137845,"num_write_kb":7974230,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7221,"num_bytes_recovered":464959744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79415,"ondisk_log_size":79415,"up":504,"acting":501,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":508,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":23899968,"kb_used_data":22015080,"kb_used_omap":12,"kb_used_meta":1884723,"kb_avail":726061248,"statfs":{"total":767960285184,"available":743486717952,"internally_reserved":0,"allocated":22543441920,"data_stored":22540985745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":1929956914},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-704315392,"num_objects":-10747,"num_object_clones":0,"num_object_copies":-32241,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-966,"num_objects_unfound":0,"num_objects_dirty":-10747,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":10747,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":966,"num_bytes_recovered":63307776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.943939"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":313,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.728260+0000","last_change":"2024-09-16T09:30:03.848439+0000","last_active":"2024-09-16T09:31:35.728260+0000","last_peered":"2024-09-16T09:31:35.728260+0000","last_clean":"2024-09-16T09:31:35.728260+0000","last_became_active":"2024-09-16T09:25:01.709108+0000","last_became_peered":"2024-09-16T09:25:01.709108+0000","last_unstale":"2024-09-16T09:31:35.728260+0000","last_undegraded":"2024-09-16T09:31:35.728260+0000","last_fullsized":"2024-09-16T09:31:35.728260+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:30:03.848354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:30:03.848354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"66'9","reported_seq":316,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.716445+0000","last_change":"2024-09-16T09:28:52.689373+0000","last_active":"2024-09-16T09:31:35.716445+0000","last_peered":"2024-09-16T09:31:35.716445+0000","last_clean":"2024-09-16T09:31:35.716445+0000","last_became_active":"2024-09-16T09:25:03.714635+0000","last_became_peered":"2024-09-16T09:25:03.714635+0000","last_unstale":"2024-09-16T09:31:35.716445+0000","last_undegraded":"2024-09-16T09:31:35.716445+0000","last_fullsized":"2024-09-16T09:31:35.716445+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:28:52.689304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:52.689304+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":304,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.720467+0000","last_change":"2024-09-16T09:29:17.910367+0000","last_active":"2024-09-16T09:31:35.720467+0000","last_peered":"2024-09-16T09:31:35.720467+0000","last_clean":"2024-09-16T09:31:35.720467+0000","last_became_active":"2024-09-16T09:25:05.715645+0000","last_became_peered":"2024-09-16T09:25:05.715645+0000","last_unstale":"2024-09-16T09:31:35.720467+0000","last_undegraded":"2024-09-16T09:31:35.720467+0000","last_fullsized":"2024-09-16T09:31:35.720467+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:17.910305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:17.910305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":284,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.717156+0000","last_change":"2024-09-16T09:27:11.565306+0000","last_active":"2024-09-16T09:31:35.717156+0000","last_peered":"2024-09-16T09:31:35.717156+0000","last_clean":"2024-09-16T09:31:35.717156+0000","last_became_active":"2024-09-16T09:25:07.715442+0000","last_became_peered":"2024-09-16T09:25:07.715442+0000","last_unstale":"2024-09-16T09:31:35.717156+0000","last_undegraded":"2024-09-16T09:31:35.717156+0000","last_fullsized":"2024-09-16T09:31:35.717156+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:11.565220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:11.565220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":30,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.717463+0000","last_change":"2024-09-16T09:31:09.269700+0000","last_active":"2024-09-16T09:31:35.717463+0000","last_peered":"2024-09-16T09:31:35.717463+0000","last_clean":"2024-09-16T09:31:35.717463+0000","last_became_active":"2024-09-16T09:31:09.269342+0000","last_became_peered":"2024-09-16T09:31:09.269342+0000","last_unstale":"2024-09-16T09:31:35.717463+0000","last_undegraded":"2024-09-16T09:31:35.717463+0000","last_fullsized":"2024-09-16T09:31:35.717463+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:00.068570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:28:00.068570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":162,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.725816+0000","last_change":"2024-09-16T09:31:09.276349+0000","last_active":"2024-09-16T09:31:35.725816+0000","last_peered":"2024-09-16T09:31:35.725816+0000","last_clean":"2024-09-16T09:31:35.725816+0000","last_became_active":"2024-09-16T09:31:09.275136+0000","last_became_peered":"2024-09-16T09:31:09.275136+0000","last_unstale":"2024-09-16T09:31:35.725816+0000","last_undegraded":"2024-09-16T09:31:35.725816+0000","last_fullsized":"2024-09-16T09:31:35.725816+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:30:32.389008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:30:32.389008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:31:56.454141+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.19","version":"66'15","reported_seq":231,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.727553+0000","last_change":"2024-09-16T09:31:09.302903+0000","last_active":"2024-09-16T09:31:35.727553+0000","last_peered":"2024-09-16T09:31:35.727553+0000","last_clean":"2024-09-16T09:31:35.727553+0000","last_became_active":"2024-09-16T09:31:09.268750+0000","last_became_peered":"2024-09-16T09:31:09.268750+0000","last_unstale":"2024-09-16T09:31:35.727553+0000","last_undegraded":"2024-09-16T09:31:35.727553+0000","last_fullsized":"2024-09-16T09:31:35.727553+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-16T09:29:57.968162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:29:57.968162+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":304,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.716371+0000","last_change":"2024-09-16T09:29:15.628676+0000","last_active":"2024-09-16T09:31:35.716371+0000","last_peered":"2024-09-16T09:31:35.716371+0000","last_clean":"2024-09-16T09:31:35.716371+0000","last_became_active":"2024-09-16T09:25:05.717236+0000","last_became_peered":"2024-09-16T09:25:05.717236+0000","last_unstale":"2024-09-16T09:31:35.716371+0000","last_undegraded":"2024-09-16T09:31:35.716371+0000","last_fullsized":"2024-09-16T09:31:35.716371+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:15.628467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:15.628467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":143,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.723584+0000","last_change":"2024-09-16T09:28:34.839909+0000","last_active":"2024-09-16T09:31:35.723584+0000","last_peered":"2024-09-16T09:31:35.723584+0000","last_clean":"2024-09-16T09:31:35.723584+0000","last_became_active":"2024-09-16T09:28:34.839435+0000","last_became_peered":"2024-09-16T09:28:34.839435+0000","last_unstale":"2024-09-16T09:31:35.723584+0000","last_undegraded":"2024-09-16T09:31:35.723584+0000","last_fullsized":"2024-09-16T09:31:35.723584+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:01.038400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:28:01.038400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:09.066525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"66'9","reported_seq":316,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.717432+0000","last_change":"2024-09-16T09:29:28.622647+0000","last_active":"2024-09-16T09:31:35.717432+0000","last_peered":"2024-09-16T09:31:35.717432+0000","last_clean":"2024-09-16T09:31:35.717432+0000","last_became_active":"2024-09-16T09:25:03.709147+0000","last_became_peered":"2024-09-16T09:25:03.709147+0000","last_unstale":"2024-09-16T09:31:35.717432+0000","last_undegraded":"2024-09-16T09:31:35.717432+0000","last_fullsized":"2024-09-16T09:31:35.717432+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:29:28.622558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:29:28.622558+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":287,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.722428+0000","last_change":"2024-09-16T09:27:59.080369+0000","last_active":"2024-09-16T09:31:35.722428+0000","last_peered":"2024-09-16T09:31:35.722428+0000","last_clean":"2024-09-16T09:31:35.722428+0000","last_became_active":"2024-09-16T09:25:05.718134+0000","last_became_peered":"2024-09-16T09:25:05.718134+0000","last_unstale":"2024-09-16T09:31:35.722428+0000","last_undegraded":"2024-09-16T09:31:35.722428+0000","last_fullsized":"2024-09-16T09:31:35.722428+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:59.080307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:59.080307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"66'1","reported_seq":293,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.726955+0000","last_change":"2024-09-16T09:29:15.787990+0000","last_active":"2024-09-16T09:31:35.726955+0000","last_peered":"2024-09-16T09:31:35.726955+0000","last_clean":"2024-09-16T09:31:35.726955+0000","last_became_active":"2024-09-16T09:25:07.716558+0000","last_became_peered":"2024-09-16T09:25:07.716558+0000","last_unstale":"2024-09-16T09:31:35.726955+0000","last_undegraded":"2024-09-16T09:31:35.726955+0000","last_fullsized":"2024-09-16T09:31:35.726955+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-16T09:29:15.787922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:15.787922+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":295,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.723583+0000","last_change":"2024-09-16T09:27:56.191264+0000","last_active":"2024-09-16T09:31:35.723583+0000","last_peered":"2024-09-16T09:31:35.723583+0000","last_clean":"2024-09-16T09:31:35.723583+0000","last_became_active":"2024-09-16T09:25:01.696939+0000","last_became_peered":"2024-09-16T09:25:01.696939+0000","last_unstale":"2024-09-16T09:31:35.723583+0000","last_undegraded":"2024-09-16T09:31:35.723583+0000","last_fullsized":"2024-09-16T09:31:35.723583+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:56.191220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:27:56.191220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"66'5","reported_seq":302,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.717877+0000","last_change":"2024-09-16T09:27:05.694733+0000","last_active":"2024-09-16T09:31:35.717877+0000","last_peered":"2024-09-16T09:31:35.717877+0000","last_clean":"2024-09-16T09:31:35.717877+0000","last_became_active":"2024-09-16T09:25:03.716136+0000","last_became_peered":"2024-09-16T09:25:03.716136+0000","last_unstale":"2024-09-16T09:31:35.717877+0000","last_undegraded":"2024-09-16T09:31:35.717877+0000","last_fullsized":"2024-09-16T09:31:35.717877+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-16T09:27:05.694650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:05.694650+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":295,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.728152+0000","last_change":"2024-09-16T09:28:52.746046+0000","last_active":"2024-09-16T09:31:35.728152+0000","last_peered":"2024-09-16T09:31:35.728152+0000","last_clean":"2024-09-16T09:31:35.728152+0000","last_became_active":"2024-09-16T09:25:05.718281+0000","last_became_peered":"2024-09-16T09:25:05.718281+0000","last_unstale":"2024-09-16T09:31:35.728152+0000","last_undegraded":"2024-09-16T09:31:35.728152+0000","last_fullsized":"2024-09-16T09:31:35.728152+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:52.745999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:28:52.745999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":313,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.723124+0000","last_change":"2024-09-16T09:31:09.276016+0000","last_active":"2024-09-16T09:31:35.723124+0000","last_peered":"2024-09-16T09:31:35.723124+0000","last_clean":"2024-09-16T09:31:35.723124+0000","last_became_active":"2024-09-16T09:31:09.275656+0000","last_became_peered":"2024-09-16T09:31:09.275656+0000","last_unstale":"2024-09-16T09:31:35.723124+0000","last_undegraded":"2024-09-16T09:31:35.723124+0000","last_fullsized":"2024-09-16T09:31:35.723124+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:50.468134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:50.468134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":301,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.718399+0000","last_change":"2024-09-16T09:29:56.605555+0000","last_active":"2024-09-16T09:31:35.718399+0000","last_peered":"2024-09-16T09:31:35.718399+0000","last_clean":"2024-09-16T09:31:35.718399+0000","last_became_active":"2024-09-16T09:25:07.720793+0000","last_became_peered":"2024-09-16T09:25:07.720793+0000","last_unstale":"2024-09-16T09:31:35.718399+0000","last_undegraded":"2024-09-16T09:31:35.718399+0000","last_fullsized":"2024-09-16T09:31:35.718399+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:56.605488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:56.605488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":52,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.723760+0000","last_change":"2024-09-16T09:30:39.202178+0000","last_active":"2024-09-16T09:31:35.723760+0000","last_peered":"2024-09-16T09:31:35.723760+0000","last_clean":"2024-09-16T09:31:35.723760+0000","last_became_active":"2024-09-16T09:30:39.201824+0000","last_became_peered":"2024-09-16T09:30:39.201824+0000","last_unstale":"2024-09-16T09:31:35.723760+0000","last_undegraded":"2024-09-16T09:31:35.723760+0000","last_fullsized":"2024-09-16T09:31:35.723760+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:50.912979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:50.912979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:20.431823+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"66'15","reported_seq":379,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.727325+0000","last_change":"2024-09-16T09:31:09.302018+0000","last_active":"2024-09-16T09:31:35.727325+0000","last_peered":"2024-09-16T09:31:35.727325+0000","last_clean":"2024-09-16T09:31:35.727325+0000","last_became_active":"2024-09-16T09:31:09.263534+0000","last_became_peered":"2024-09-16T09:31:09.263534+0000","last_unstale":"2024-09-16T09:31:35.727325+0000","last_undegraded":"2024-09-16T09:31:35.727325+0000","last_fullsized":"2024-09-16T09:31:35.727325+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-16T09:30:21.932604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:30:21.932604+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:31:39.071159+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":296,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.723895+0000","last_change":"2024-09-16T09:29:55.875536+0000","last_active":"2024-09-16T09:31:35.723895+0000","last_peered":"2024-09-16T09:31:35.723895+0000","last_clean":"2024-09-16T09:31:35.723895+0000","last_became_active":"2024-09-16T09:25:05.713929+0000","last_became_peered":"2024-09-16T09:25:05.713929+0000","last_unstale":"2024-09-16T09:31:35.723895+0000","last_undegraded":"2024-09-16T09:31:35.723895+0000","last_fullsized":"2024-09-16T09:31:35.723895+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:55.875480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:55.875480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":149,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.724436+0000","last_change":"2024-09-16T09:30:24.412733+0000","last_active":"2024-09-16T09:31:35.724436+0000","last_peered":"2024-09-16T09:31:35.724436+0000","last_clean":"2024-09-16T09:31:35.724436+0000","last_became_active":"2024-09-16T09:28:52.974255+0000","last_became_peered":"2024-09-16T09:28:52.974255+0000","last_unstale":"2024-09-16T09:31:35.724436+0000","last_undegraded":"2024-09-16T09:31:35.724436+0000","last_fullsized":"2024-09-16T09:31:35.724436+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:30:24.412571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:30:24.412571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:31:36.120832+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":312,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.717568+0000","last_change":"2024-09-16T09:30:03.628882+0000","last_active":"2024-09-16T09:31:35.717568+0000","last_peered":"2024-09-16T09:31:35.717568+0000","last_clean":"2024-09-16T09:31:35.717568+0000","last_became_active":"2024-09-16T09:25:01.706491+0000","last_became_peered":"2024-09-16T09:25:01.706491+0000","last_unstale":"2024-09-16T09:31:35.717568+0000","last_undegraded":"2024-09-16T09:31:35.717568+0000","last_fullsized":"2024-09-16T09:31:35.717568+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:30:03.628812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:30:03.628812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"66'12","reported_seq":213,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.724082+0000","last_change":"2024-09-16T09:31:09.286673+0000","last_active":"2024-09-16T09:31:35.724082+0000","last_peered":"2024-09-16T09:31:35.724082+0000","last_clean":"2024-09-16T09:31:35.724082+0000","last_became_active":"2024-09-16T09:31:09.268588+0000","last_became_peered":"2024-09-16T09:31:09.268588+0000","last_unstale":"2024-09-16T09:31:35.724082+0000","last_undegraded":"2024-09-16T09:31:35.724082+0000","last_fullsized":"2024-09-16T09:31:35.724082+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-16T09:27:01.700139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:01.700139+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":24,"num_read_kb":15,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":288,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.717750+0000","last_change":"2024-09-16T09:27:00.816500+0000","last_active":"2024-09-16T09:31:35.717750+0000","last_peered":"2024-09-16T09:31:35.717750+0000","last_clean":"2024-09-16T09:31:35.717750+0000","last_became_active":"2024-09-16T09:25:05.710093+0000","last_became_peered":"2024-09-16T09:25:05.710093+0000","last_unstale":"2024-09-16T09:31:35.717750+0000","last_undegraded":"2024-09-16T09:31:35.717750+0000","last_fullsized":"2024-09-16T09:31:35.717750+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:00.816424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:00.816424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":292,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.728695+0000","last_change":"2024-09-16T09:29:29.918433+0000","last_active":"2024-09-16T09:31:35.728695+0000","last_peered":"2024-09-16T09:31:35.728695+0000","last_clean":"2024-09-16T09:31:35.728695+0000","last_became_active":"2024-09-16T09:25:07.719380+0000","last_became_peered":"2024-09-16T09:25:07.719380+0000","last_unstale":"2024-09-16T09:31:35.728695+0000","last_undegraded":"2024-09-16T09:31:35.728695+0000","last_fullsized":"2024-09-16T09:31:35.728695+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-16T09:29:29.918368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:29.918368+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":313,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.720586+0000","last_change":"2024-09-16T09:30:02.843932+0000","last_active":"2024-09-16T09:31:35.720586+0000","last_peered":"2024-09-16T09:31:35.720586+0000","last_clean":"2024-09-16T09:31:35.720586+0000","last_became_active":"2024-09-16T09:25:01.699545+0000","last_became_peered":"2024-09-16T09:25:01.699545+0000","last_unstale":"2024-09-16T09:31:35.720586+0000","last_undegraded":"2024-09-16T09:31:35.720586+0000","last_fullsized":"2024-09-16T09:31:35.720586+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:30:02.843856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:30:02.843856+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"66'10","reported_seq":354,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.729047+0000","last_change":"2024-09-16T09:31:09.328513+0000","last_active":"2024-09-16T09:31:35.729047+0000","last_peered":"2024-09-16T09:31:35.729047+0000","last_clean":"2024-09-16T09:31:35.729047+0000","last_became_active":"2024-09-16T09:31:09.275050+0000","last_became_peered":"2024-09-16T09:31:09.275050+0000","last_unstale":"2024-09-16T09:31:35.729047+0000","last_undegraded":"2024-09-16T09:31:35.729047+0000","last_fullsized":"2024-09-16T09:31:35.729047+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-16T09:27:43.556357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:43.556357+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:15.194576+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"66'8","reported_seq":404,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.724538+0000","last_change":"2024-09-16T09:30:31.392771+0000","last_active":"2024-09-16T09:31:35.724538+0000","last_peered":"2024-09-16T09:31:35.724538+0000","last_clean":"2024-09-16T09:31:35.724538+0000","last_became_active":"2024-09-16T09:25:05.712516+0000","last_became_peered":"2024-09-16T09:25:05.712516+0000","last_unstale":"2024-09-16T09:31:35.724538+0000","last_undegraded":"2024-09-16T09:31:35.724538+0000","last_fullsized":"2024-09-16T09:31:35.724538+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:30:31.392399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:30:31.392399+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"196'8624","reported_seq":8723,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:37.828793+0000","last_change":"2024-09-16T09:29:49.449011+0000","last_active":"2024-09-16T09:31:37.828793+0000","last_peered":"2024-09-16T09:31:37.828793+0000","last_clean":"2024-09-16T09:31:37.828793+0000","last_became_active":"2024-09-16T09:29:49.448473+0000","last_became_peered":"2024-09-16T09:29:49.448473+0000","last_unstale":"2024-09-16T09:31:37.828793+0000","last_undegraded":"2024-09-16T09:31:37.828793+0000","last_fullsized":"2024-09-16T09:31:37.828793+0000","mapping_epoch":152,"log_start":"173'4100","ondisk_log_start":"173'4100","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":4524,"log_dups_size":0,"ondisk_log_size":4524,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":458358784,"num_objects":6994,"num_object_clones":0,"num_object_copies":20982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6994,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8624,"num_write_kb":499776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.a","version":"73'79","reported_seq":235,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.721756+0000","last_change":"2024-09-16T09:29:18.498887+0000","last_active":"2024-09-16T09:31:35.721756+0000","last_peered":"2024-09-16T09:31:35.721756+0000","last_clean":"2024-09-16T09:31:35.721756+0000","last_became_active":"2024-09-16T09:26:35.127352+0000","last_became_peered":"2024-09-16T09:26:35.127352+0000","last_unstale":"2024-09-16T09:31:35.721756+0000","last_undegraded":"2024-09-16T09:31:35.721756+0000","last_fullsized":"2024-09-16T09:31:35.721756+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:18.498766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:18.498766+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"66'15","reported_seq":244,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.727777+0000","last_change":"2024-09-16T09:28:28.065781+0000","last_active":"2024-09-16T09:31:35.727777+0000","last_peered":"2024-09-16T09:31:35.727777+0000","last_clean":"2024-09-16T09:31:35.727777+0000","last_became_active":"2024-09-16T09:26:33.295672+0000","last_became_peered":"2024-09-16T09:26:33.295672+0000","last_unstale":"2024-09-16T09:31:35.727777+0000","last_undegraded":"2024-09-16T09:31:35.727777+0000","last_fullsized":"2024-09-16T09:31:35.727777+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-16T09:28:28.065699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:28.065699+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":30,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.726582+0000","last_change":"2024-09-16T09:31:09.276858+0000","last_active":"2024-09-16T09:31:35.726582+0000","last_peered":"2024-09-16T09:31:35.726582+0000","last_clean":"2024-09-16T09:31:35.726582+0000","last_became_active":"2024-09-16T09:31:09.274534+0000","last_became_peered":"2024-09-16T09:31:09.274534+0000","last_unstale":"2024-09-16T09:31:35.726582+0000","last_undegraded":"2024-09-16T09:31:35.726582+0000","last_fullsized":"2024-09-16T09:31:35.726582+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:22.091537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:28:22.091537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"66'8","reported_seq":368,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.718525+0000","last_change":"2024-09-16T09:25:05.714128+0000","last_active":"2024-09-16T09:31:35.718525+0000","last_peered":"2024-09-16T09:31:35.718525+0000","last_clean":"2024-09-16T09:31:35.718525+0000","last_became_active":"2024-09-16T09:25:05.713813+0000","last_became_peered":"2024-09-16T09:25:05.713813+0000","last_unstale":"2024-09-16T09:31:35.718525+0000","last_undegraded":"2024-09-16T09:31:35.718525+0000","last_fullsized":"2024-09-16T09:31:35.718525+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":291,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.727912+0000","last_change":"2024-09-16T09:29:24.537802+0000","last_active":"2024-09-16T09:31:35.727912+0000","last_peered":"2024-09-16T09:31:35.727912+0000","last_clean":"2024-09-16T09:31:35.727912+0000","last_became_active":"2024-09-16T09:25:07.717344+0000","last_became_peered":"2024-09-16T09:25:07.717344+0000","last_unstale":"2024-09-16T09:31:35.727912+0000","last_undegraded":"2024-09-16T09:31:35.727912+0000","last_fullsized":"2024-09-16T09:31:35.727912+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:24.537731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:24.537731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.c","version":"196'8596","reported_seq":8695,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:38.123821+0000","last_change":"2024-09-16T09:29:50.008199+0000","last_active":"2024-09-16T09:31:38.123821+0000","last_peered":"2024-09-16T09:31:38.123821+0000","last_clean":"2024-09-16T09:31:38.123821+0000","last_became_active":"2024-09-16T09:29:50.007935+0000","last_became_peered":"2024-09-16T09:29:50.007935+0000","last_unstale":"2024-09-16T09:31:38.123821+0000","last_undegraded":"2024-09-16T09:31:38.123821+0000","last_fullsized":"2024-09-16T09:31:38.123821+0000","mapping_epoch":152,"log_start":"169'3500","ondisk_log_start":"169'3500","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":5096,"log_dups_size":0,"ondisk_log_size":5096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":455606272,"num_objects":6952,"num_object_clones":0,"num_object_copies":20856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6952,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8596,"num_write_kb":497536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"73'79","reported_seq":261,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.717719+0000","last_change":"2024-09-16T09:31:14.198357+0000","last_active":"2024-09-16T09:31:35.717719+0000","last_peered":"2024-09-16T09:31:35.717719+0000","last_clean":"2024-09-16T09:31:35.717719+0000","last_became_active":"2024-09-16T09:30:08.763683+0000","last_became_peered":"2024-09-16T09:30:08.763683+0000","last_unstale":"2024-09-16T09:31:35.717719+0000","last_undegraded":"2024-09-16T09:31:35.717719+0000","last_fullsized":"2024-09-16T09:31:35.717719+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:58.568841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:58.568841+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:31:40.802987+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"66'18","reported_seq":242,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.727388+0000","last_change":"2024-09-16T09:31:09.407700+0000","last_active":"2024-09-16T09:31:35.727388+0000","last_peered":"2024-09-16T09:31:35.727388+0000","last_clean":"2024-09-16T09:31:35.727388+0000","last_became_active":"2024-09-16T09:31:09.270434+0000","last_became_peered":"2024-09-16T09:31:09.270434+0000","last_unstale":"2024-09-16T09:31:35.727388+0000","last_undegraded":"2024-09-16T09:31:35.727388+0000","last_fullsized":"2024-09-16T09:31:35.727388+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'18","last_scrub_stamp":"2024-09-16T09:28:54.699778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:54.699778+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":33,"num_read_kb":21,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":140,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.729275+0000","last_change":"2024-09-16T09:28:58.705169+0000","last_active":"2024-09-16T09:31:35.729275+0000","last_peered":"2024-09-16T09:31:35.729275+0000","last_clean":"2024-09-16T09:31:35.729275+0000","last_became_active":"2024-09-16T09:28:52.983543+0000","last_became_peered":"2024-09-16T09:28:52.983543+0000","last_unstale":"2024-09-16T09:31:35.729275+0000","last_undegraded":"2024-09-16T09:31:35.729275+0000","last_fullsized":"2024-09-16T09:31:35.729275+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:58.705097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:28:58.705097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":319,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.716414+0000","last_change":"2024-09-16T09:31:09.269234+0000","last_active":"2024-09-16T09:31:35.716414+0000","last_peered":"2024-09-16T09:31:35.716414+0000","last_clean":"2024-09-16T09:31:35.716414+0000","last_became_active":"2024-09-16T09:31:09.268864+0000","last_became_peered":"2024-09-16T09:31:09.268864+0000","last_unstale":"2024-09-16T09:31:35.716414+0000","last_undegraded":"2024-09-16T09:31:35.716414+0000","last_fullsized":"2024-09-16T09:31:35.716414+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:50.482240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:50.482240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":299,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.717343+0000","last_change":"2024-09-16T09:31:09.269842+0000","last_active":"2024-09-16T09:31:35.717343+0000","last_peered":"2024-09-16T09:31:35.717343+0000","last_clean":"2024-09-16T09:31:35.717343+0000","last_became_active":"2024-09-16T09:31:09.269433+0000","last_became_peered":"2024-09-16T09:31:09.269433+0000","last_unstale":"2024-09-16T09:31:35.717343+0000","last_undegraded":"2024-09-16T09:31:35.717343+0000","last_fullsized":"2024-09-16T09:31:35.717343+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:13.548608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:13.548608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.d","version":"196'8602","reported_seq":8702,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:37.860555+0000","last_change":"2024-09-16T09:29:50.010860+0000","last_active":"2024-09-16T09:31:37.860555+0000","last_peered":"2024-09-16T09:31:37.860555+0000","last_clean":"2024-09-16T09:31:37.860555+0000","last_became_active":"2024-09-16T09:29:50.010421+0000","last_became_peered":"2024-09-16T09:29:50.010421+0000","last_unstale":"2024-09-16T09:31:37.860555+0000","last_undegraded":"2024-09-16T09:31:37.860555+0000","last_fullsized":"2024-09-16T09:31:37.860555+0000","mapping_epoch":152,"log_start":"172'3900","ondisk_log_start":"172'3900","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":4702,"log_dups_size":0,"ondisk_log_size":4702,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":456327192,"num_objects":6964,"num_object_clones":0,"num_object_copies":20892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6964,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8602,"num_write_kb":498049,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,2],"acting":[5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.4","version":"73'79","reported_seq":30,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.719621+0000","last_change":"2024-09-16T09:31:09.275499+0000","last_active":"2024-09-16T09:31:35.719621+0000","last_peered":"2024-09-16T09:31:35.719621+0000","last_clean":"2024-09-16T09:31:35.719621+0000","last_became_active":"2024-09-16T09:31:09.275139+0000","last_became_peered":"2024-09-16T09:31:09.275139+0000","last_unstale":"2024-09-16T09:31:35.719621+0000","last_undegraded":"2024-09-16T09:31:35.719621+0000","last_fullsized":"2024-09-16T09:31:35.719621+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:30:22.438378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:30:22.438378+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"66'14","reported_seq":322,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.716512+0000","last_change":"2024-09-16T09:28:32.595064+0000","last_active":"2024-09-16T09:31:35.716512+0000","last_peered":"2024-09-16T09:31:35.716512+0000","last_clean":"2024-09-16T09:31:35.716512+0000","last_became_active":"2024-09-16T09:25:03.708830+0000","last_became_peered":"2024-09-16T09:25:03.708830+0000","last_unstale":"2024-09-16T09:31:35.716512+0000","last_undegraded":"2024-09-16T09:31:35.716512+0000","last_fullsized":"2024-09-16T09:31:35.716512+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'14","last_scrub_stamp":"2024-09-16T09:28:32.594988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:32.594988+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":314,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.729291+0000","last_change":"2024-09-16T09:28:58.602786+0000","last_active":"2024-09-16T09:31:35.729291+0000","last_peered":"2024-09-16T09:31:35.729291+0000","last_clean":"2024-09-16T09:31:35.729291+0000","last_became_active":"2024-09-16T09:25:01.702992+0000","last_became_peered":"2024-09-16T09:25:01.702992+0000","last_unstale":"2024-09-16T09:31:35.729291+0000","last_undegraded":"2024-09-16T09:31:35.729291+0000","last_fullsized":"2024-09-16T09:31:35.729291+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:28:58.602717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:28:58.602717+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"187'11","reported_seq":221,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.718936+0000","last_change":"2024-09-16T09:31:09.397758+0000","last_active":"2024-09-16T09:31:35.718936+0000","last_peered":"2024-09-16T09:31:35.718936+0000","last_clean":"2024-09-16T09:31:35.718936+0000","last_became_active":"2024-09-16T09:31:09.269173+0000","last_became_peered":"2024-09-16T09:31:09.269173+0000","last_unstale":"2024-09-16T09:31:35.718936+0000","last_undegraded":"2024-09-16T09:31:35.718936+0000","last_fullsized":"2024-09-16T09:31:35.718936+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'9","last_scrub_stamp":"2024-09-16T09:29:21.898533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:21.898533+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":284,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.728332+0000","last_change":"2024-09-16T09:26:40.699888+0000","last_active":"2024-09-16T09:31:35.728332+0000","last_peered":"2024-09-16T09:31:35.728332+0000","last_clean":"2024-09-16T09:31:35.728332+0000","last_became_active":"2024-09-16T09:25:07.722017+0000","last_became_peered":"2024-09-16T09:25:07.722017+0000","last_unstale":"2024-09-16T09:31:35.728332+0000","last_undegraded":"2024-09-16T09:31:35.728332+0000","last_fullsized":"2024-09-16T09:31:35.728332+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:40.699813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:40.699813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.e","version":"196'8637","reported_seq":8736,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:38.078108+0000","last_change":"2024-09-16T09:29:49.448081+0000","last_active":"2024-09-16T09:31:38.078108+0000","last_peered":"2024-09-16T09:31:38.078108+0000","last_clean":"2024-09-16T09:31:38.078108+0000","last_became_active":"2024-09-16T09:29:49.447767+0000","last_became_peered":"2024-09-16T09:29:49.447767+0000","last_unstale":"2024-09-16T09:31:38.078108+0000","last_undegraded":"2024-09-16T09:31:38.078108+0000","last_fullsized":"2024-09-16T09:31:38.078108+0000","mapping_epoch":152,"log_start":"169'3500","ondisk_log_start":"169'3500","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":5137,"log_dups_size":0,"ondisk_log_size":5137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":456720384,"num_objects":6969,"num_object_clones":0,"num_object_copies":20907,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6969,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8637,"num_write_kb":499392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"73'79","reported_seq":353,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.724351+0000","last_change":"2024-09-16T09:31:09.275413+0000","last_active":"2024-09-16T09:31:35.724351+0000","last_peered":"2024-09-16T09:31:35.724351+0000","last_clean":"2024-09-16T09:31:35.724351+0000","last_became_active":"2024-09-16T09:31:09.274707+0000","last_became_peered":"2024-09-16T09:31:09.274707+0000","last_unstale":"2024-09-16T09:31:35.724351+0000","last_undegraded":"2024-09-16T09:31:35.724351+0000","last_fullsized":"2024-09-16T09:31:35.724351+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:30:23.455388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:30:23.455388+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"66'10","reported_seq":324,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.724077+0000","last_change":"2024-09-16T09:30:04.848306+0000","last_active":"2024-09-16T09:31:35.724077+0000","last_peered":"2024-09-16T09:31:35.724077+0000","last_clean":"2024-09-16T09:31:35.724077+0000","last_became_active":"2024-09-16T09:25:03.712985+0000","last_became_peered":"2024-09-16T09:25:03.712985+0000","last_unstale":"2024-09-16T09:31:35.724077+0000","last_undegraded":"2024-09-16T09:31:35.724077+0000","last_fullsized":"2024-09-16T09:31:35.724077+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-16T09:30:04.848234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:30:04.848234+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":310,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.728302+0000","last_change":"2024-09-16T09:31:09.272049+0000","last_active":"2024-09-16T09:31:35.728302+0000","last_peered":"2024-09-16T09:31:35.728302+0000","last_clean":"2024-09-16T09:31:35.728302+0000","last_became_active":"2024-09-16T09:31:09.270619+0000","last_became_peered":"2024-09-16T09:31:09.270619+0000","last_unstale":"2024-09-16T09:31:35.728302+0000","last_undegraded":"2024-09-16T09:31:35.728302+0000","last_fullsized":"2024-09-16T09:31:35.728302+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:25.086322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:25.086322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"66'8","reported_seq":385,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.728218+0000","last_change":"2024-09-16T09:28:57.628615+0000","last_active":"2024-09-16T09:31:35.728218+0000","last_peered":"2024-09-16T09:31:35.728218+0000","last_clean":"2024-09-16T09:31:35.728218+0000","last_became_active":"2024-09-16T09:25:05.714507+0000","last_became_peered":"2024-09-16T09:25:05.714507+0000","last_unstale":"2024-09-16T09:31:35.728218+0000","last_undegraded":"2024-09-16T09:31:35.728218+0000","last_fullsized":"2024-09-16T09:31:35.728218+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:28:57.628543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:28:57.628543+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":307,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.727133+0000","last_change":"2024-09-16T09:31:09.283816+0000","last_active":"2024-09-16T09:31:35.727133+0000","last_peered":"2024-09-16T09:31:35.727133+0000","last_clean":"2024-09-16T09:31:35.727133+0000","last_became_active":"2024-09-16T09:31:09.283192+0000","last_became_peered":"2024-09-16T09:31:09.283192+0000","last_unstale":"2024-09-16T09:31:35.727133+0000","last_undegraded":"2024-09-16T09:31:35.727133+0000","last_fullsized":"2024-09-16T09:31:35.727133+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:22.563395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:28:22.563395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:22.152175+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.f","version":"196'8655","reported_seq":8754,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:37.585808+0000","last_change":"2024-09-16T09:29:49.838498+0000","last_active":"2024-09-16T09:31:37.585808+0000","last_peered":"2024-09-16T09:31:37.585808+0000","last_clean":"2024-09-16T09:31:37.585808+0000","last_became_active":"2024-09-16T09:29:49.838146+0000","last_became_peered":"2024-09-16T09:29:49.838146+0000","last_unstale":"2024-09-16T09:31:37.585808+0000","last_undegraded":"2024-09-16T09:31:37.585808+0000","last_fullsized":"2024-09-16T09:31:37.585808+0000","mapping_epoch":152,"log_start":"165'2700","ondisk_log_start":"165'2700","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":5955,"log_dups_size":0,"ondisk_log_size":5955,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":464715776,"num_objects":7091,"num_object_clones":0,"num_object_copies":21273,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7091,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8655,"num_write_kb":503872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"73'79","reported_seq":334,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.725638+0000","last_change":"2024-09-16T09:31:09.274975+0000","last_active":"2024-09-16T09:31:35.725638+0000","last_peered":"2024-09-16T09:31:35.725638+0000","last_clean":"2024-09-16T09:31:35.725638+0000","last_became_active":"2024-09-16T09:31:09.274565+0000","last_became_peered":"2024-09-16T09:31:09.274565+0000","last_unstale":"2024-09-16T09:31:35.725638+0000","last_undegraded":"2024-09-16T09:31:35.725638+0000","last_fullsized":"2024-09-16T09:31:35.725638+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:30:22.415051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:30:22.415051+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:31:41.553155+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"66'19","reported_seq":337,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.728425+0000","last_change":"2024-09-16T09:31:35.249952+0000","last_active":"2024-09-16T09:31:35.728425+0000","last_peered":"2024-09-16T09:31:35.728425+0000","last_clean":"2024-09-16T09:31:35.728425+0000","last_became_active":"2024-09-16T09:25:03.714797+0000","last_became_peered":"2024-09-16T09:25:03.714797+0000","last_unstale":"2024-09-16T09:31:35.728425+0000","last_undegraded":"2024-09-16T09:31:35.728425+0000","last_fullsized":"2024-09-16T09:31:35.728425+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-16T09:31:35.249864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:31:35.249864+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:40.632025+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":313,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.723672+0000","last_change":"2024-09-16T09:29:16.888357+0000","last_active":"2024-09-16T09:31:35.723672+0000","last_peered":"2024-09-16T09:31:35.723672+0000","last_clean":"2024-09-16T09:31:35.723672+0000","last_became_active":"2024-09-16T09:25:01.706806+0000","last_became_peered":"2024-09-16T09:25:01.706806+0000","last_unstale":"2024-09-16T09:31:35.723672+0000","last_undegraded":"2024-09-16T09:31:35.723672+0000","last_fullsized":"2024-09-16T09:31:35.723672+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:16.888236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:16.888236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":305,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.723495+0000","last_change":"2024-09-16T09:30:51.409917+0000","last_active":"2024-09-16T09:31:35.723495+0000","last_peered":"2024-09-16T09:31:35.723495+0000","last_clean":"2024-09-16T09:31:35.723495+0000","last_became_active":"2024-09-16T09:25:05.708663+0000","last_became_peered":"2024-09-16T09:25:05.708663+0000","last_unstale":"2024-09-16T09:31:35.723495+0000","last_undegraded":"2024-09-16T09:31:35.723495+0000","last_fullsized":"2024-09-16T09:31:35.723495+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:30:51.409834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:30:51.409834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:31:55.112422+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":285,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.720391+0000","last_change":"2024-09-16T09:26:37.282798+0000","last_active":"2024-09-16T09:31:35.720391+0000","last_peered":"2024-09-16T09:31:35.720391+0000","last_clean":"2024-09-16T09:31:35.720391+0000","last_became_active":"2024-09-16T09:25:07.719267+0000","last_became_peered":"2024-09-16T09:25:07.719267+0000","last_unstale":"2024-09-16T09:31:35.720391+0000","last_undegraded":"2024-09-16T09:31:35.720391+0000","last_fullsized":"2024-09-16T09:31:35.720391+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:37.282735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:37.282735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.8","version":"196'8669","reported_seq":12238,"reported_epoch":196,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-16T09:31:37.863735+0000","last_change":"2024-09-16T09:31:09.438251+0000","last_active":"2024-09-16T09:31:37.863735+0000","last_peered":"2024-09-16T09:31:37.863735+0000","last_clean":"2024-09-16T09:30:20.474651+0000","last_became_active":"2024-09-16T09:31:09.409873+0000","last_became_peered":"2024-09-16T09:31:09.409873+0000","last_unstale":"2024-09-16T09:31:37.863735+0000","last_undegraded":"2024-09-16T09:31:37.863735+0000","last_fullsized":"2024-09-16T09:31:09.256524+0000","mapping_epoch":186,"log_start":"172'3900","ondisk_log_start":"172'3900","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":4769,"log_dups_size":0,"ondisk_log_size":4769,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":462225408,"num_objects":7053,"num_object_clones":0,"num_object_copies":21159,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2726,"num_objects_unfound":0,"num_objects_dirty":7053,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8669,"num_write_kb":503104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1730,"num_bytes_recovered":113377280,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,7],"avail_no_missing":["5","7"],"object_location_counts":[{"shards":"0,5,7","objects":2726},{"shards":"5,7","objects":4327}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1","version":"73'79","reported_seq":253,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.722716+0000","last_change":"2024-09-16T09:31:14.195247+0000","last_active":"2024-09-16T09:31:35.722716+0000","last_peered":"2024-09-16T09:31:35.722716+0000","last_clean":"2024-09-16T09:31:35.722716+0000","last_became_active":"2024-09-16T09:30:08.766350+0000","last_became_peered":"2024-09-16T09:30:08.766350+0000","last_unstale":"2024-09-16T09:31:35.722716+0000","last_undegraded":"2024-09-16T09:31:35.722716+0000","last_fullsized":"2024-09-16T09:31:35.722716+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:15.460179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:15.460179+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:31:41.441452+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"187'30","reported_seq":487,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.719294+0000","last_change":"2024-09-16T09:31:09.323281+0000","last_active":"2024-09-16T09:31:35.719294+0000","last_peered":"2024-09-16T09:31:35.719294+0000","last_clean":"2024-09-16T09:31:35.719294+0000","last_became_active":"2024-09-16T09:31:09.278839+0000","last_became_peered":"2024-09-16T09:31:09.278839+0000","last_unstale":"2024-09-16T09:31:35.719294+0000","last_undegraded":"2024-09-16T09:31:35.719294+0000","last_fullsized":"2024-09-16T09:31:35.719294+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'28","last_scrub_stamp":"2024-09-16T09:29:18.937623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:29:18.937623+0000","objects_scrubbed":10,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":305,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.718719+0000","last_change":"2024-09-16T09:29:26.637070+0000","last_active":"2024-09-16T09:31:35.718719+0000","last_peered":"2024-09-16T09:31:35.718719+0000","last_clean":"2024-09-16T09:31:35.718719+0000","last_became_active":"2024-09-16T09:25:01.703076+0000","last_became_peered":"2024-09-16T09:25:01.703076+0000","last_unstale":"2024-09-16T09:31:35.718719+0000","last_undegraded":"2024-09-16T09:31:35.718719+0000","last_fullsized":"2024-09-16T09:31:35.718719+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:26.637027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:26.637027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":314,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.725543+0000","last_change":"2024-09-16T09:29:29.486284+0000","last_active":"2024-09-16T09:31:35.725543+0000","last_peered":"2024-09-16T09:31:35.725543+0000","last_clean":"2024-09-16T09:31:35.725543+0000","last_became_active":"2024-09-16T09:25:00.696237+0000","last_became_peered":"2024-09-16T09:25:00.696237+0000","last_unstale":"2024-09-16T09:31:35.725543+0000","last_undegraded":"2024-09-16T09:31:35.725543+0000","last_fullsized":"2024-09-16T09:31:35.725543+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-16T09:29:29.486188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:29:29.486188+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.0025137430000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":281,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.727950+0000","last_change":"2024-09-16T09:25:05.718319+0000","last_active":"2024-09-16T09:31:35.727950+0000","last_peered":"2024-09-16T09:31:35.727950+0000","last_clean":"2024-09-16T09:31:35.727950+0000","last_became_active":"2024-09-16T09:25:05.717591+0000","last_became_peered":"2024-09-16T09:25:05.717591+0000","last_unstale":"2024-09-16T09:31:35.727950+0000","last_undegraded":"2024-09-16T09:31:35.727950+0000","last_fullsized":"2024-09-16T09:31:35.727950+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:10.167420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":30,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.720159+0000","last_change":"2024-09-16T09:31:09.272967+0000","last_active":"2024-09-16T09:31:35.720159+0000","last_peered":"2024-09-16T09:31:35.720159+0000","last_clean":"2024-09-16T09:31:35.720159+0000","last_became_active":"2024-09-16T09:31:09.272609+0000","last_became_peered":"2024-09-16T09:31:09.272609+0000","last_unstale":"2024-09-16T09:31:35.720159+0000","last_undegraded":"2024-09-16T09:31:35.720159+0000","last_fullsized":"2024-09-16T09:31:35.720159+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:45.201503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:45.201503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"196'8556","reported_seq":15525,"reported_epoch":196,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-16T09:31:37.750106+0000","last_change":"2024-09-16T09:31:09.423422+0000","last_active":"2024-09-16T09:31:37.750106+0000","last_peered":"2024-09-16T09:31:37.750106+0000","last_clean":"2024-09-16T09:30:20.492754+0000","last_became_active":"2024-09-16T09:31:09.398447+0000","last_became_peered":"2024-09-16T09:31:09.398447+0000","last_unstale":"2024-09-16T09:31:37.750106+0000","last_undegraded":"2024-09-16T09:31:37.750106+0000","last_fullsized":"2024-09-16T09:31:09.253245+0000","mapping_epoch":186,"log_start":"174'4300","ondisk_log_start":"174'4300","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":4256,"log_dups_size":0,"ondisk_log_size":4256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":457310208,"num_objects":6978,"num_object_clones":0,"num_object_copies":20934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2121,"num_objects_unfound":0,"num_objects_dirty":6978,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8556,"num_write_kb":497088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3430,"num_bytes_recovered":224788480,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,1],"avail_no_missing":["4","1"],"object_location_counts":[{"shards":"1,2,4","objects":2121},{"shards":"1,4","objects":4857}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.2","version":"73'79","reported_seq":378,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.727978+0000","last_change":"2024-09-16T09:31:14.214356+0000","last_active":"2024-09-16T09:31:35.727978+0000","last_peered":"2024-09-16T09:31:35.727978+0000","last_clean":"2024-09-16T09:31:35.727978+0000","last_became_active":"2024-09-16T09:30:08.765943+0000","last_became_peered":"2024-09-16T09:30:08.765943+0000","last_unstale":"2024-09-16T09:31:35.727978+0000","last_undegraded":"2024-09-16T09:31:35.727978+0000","last_fullsized":"2024-09-16T09:31:35.727978+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:26:35.745837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:26:35.745837+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:31:41.389452+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"66'13","reported_seq":328,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.722116+0000","last_change":"2024-09-16T09:27:44.156057+0000","last_active":"2024-09-16T09:31:35.722116+0000","last_peered":"2024-09-16T09:31:35.722116+0000","last_clean":"2024-09-16T09:31:35.722116+0000","last_became_active":"2024-09-16T09:25:03.713241+0000","last_became_peered":"2024-09-16T09:25:03.713241+0000","last_unstale":"2024-09-16T09:31:35.722116+0000","last_undegraded":"2024-09-16T09:31:35.722116+0000","last_fullsized":"2024-09-16T09:31:35.722116+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'13","last_scrub_stamp":"2024-09-16T09:27:44.155969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:44.155969+0000","objects_scrubbed":5,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":29,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":305,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.723538+0000","last_change":"2024-09-16T09:29:50.827103+0000","last_active":"2024-09-16T09:31:35.723538+0000","last_peered":"2024-09-16T09:31:35.723538+0000","last_clean":"2024-09-16T09:31:35.723538+0000","last_became_active":"2024-09-16T09:25:01.706828+0000","last_became_peered":"2024-09-16T09:25:01.706828+0000","last_unstale":"2024-09-16T09:31:35.723538+0000","last_undegraded":"2024-09-16T09:31:35.723538+0000","last_fullsized":"2024-09-16T09:31:35.723538+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:50.827057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:50.827057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":336,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.727699+0000","last_change":"2024-09-16T09:31:09.281163+0000","last_active":"2024-09-16T09:31:35.727699+0000","last_peered":"2024-09-16T09:31:35.727699+0000","last_clean":"2024-09-16T09:31:35.727699+0000","last_became_active":"2024-09-16T09:31:09.266338+0000","last_became_peered":"2024-09-16T09:31:09.266338+0000","last_unstale":"2024-09-16T09:31:35.727699+0000","last_undegraded":"2024-09-16T09:31:35.727699+0000","last_fullsized":"2024-09-16T09:31:35.727699+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:29:16.104209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:29:16.104209+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0.00055022299999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":305,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.726189+0000","last_change":"2024-09-16T09:29:00.169141+0000","last_active":"2024-09-16T09:31:35.726189+0000","last_peered":"2024-09-16T09:31:35.726189+0000","last_clean":"2024-09-16T09:31:35.726189+0000","last_became_active":"2024-09-16T09:25:05.711539+0000","last_became_peered":"2024-09-16T09:25:05.711539+0000","last_unstale":"2024-09-16T09:31:35.726189+0000","last_undegraded":"2024-09-16T09:31:35.726189+0000","last_fullsized":"2024-09-16T09:31:35.726189+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:00.169077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:00.169077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":292,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.730555+0000","last_change":"2024-09-16T09:29:22.968971+0000","last_active":"2024-09-16T09:31:35.730555+0000","last_peered":"2024-09-16T09:31:35.730555+0000","last_clean":"2024-09-16T09:31:35.730555+0000","last_became_active":"2024-09-16T09:25:07.723282+0000","last_became_peered":"2024-09-16T09:25:07.723282+0000","last_unstale":"2024-09-16T09:31:35.730555+0000","last_undegraded":"2024-09-16T09:31:35.730555+0000","last_fullsized":"2024-09-16T09:31:35.730555+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:22.968908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:22.968908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"196'8610","reported_seq":8709,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:37.831036+0000","last_change":"2024-09-16T09:29:49.448898+0000","last_active":"2024-09-16T09:31:37.831036+0000","last_peered":"2024-09-16T09:31:37.831036+0000","last_clean":"2024-09-16T09:31:37.831036+0000","last_became_active":"2024-09-16T09:29:49.448628+0000","last_became_peered":"2024-09-16T09:29:49.448628+0000","last_unstale":"2024-09-16T09:31:37.831036+0000","last_undegraded":"2024-09-16T09:31:37.831036+0000","last_fullsized":"2024-09-16T09:31:37.831036+0000","mapping_epoch":152,"log_start":"173'4100","ondisk_log_start":"173'4100","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":4510,"log_dups_size":0,"ondisk_log_size":4510,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":460324864,"num_objects":7024,"num_object_clones":0,"num_object_copies":21072,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7024,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8610,"num_write_kb":500288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"73'79","reported_seq":258,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.724619+0000","last_change":"2024-09-16T09:31:14.199817+0000","last_active":"2024-09-16T09:31:35.724619+0000","last_peered":"2024-09-16T09:31:35.724619+0000","last_clean":"2024-09-16T09:31:35.724619+0000","last_became_active":"2024-09-16T09:30:08.877110+0000","last_became_peered":"2024-09-16T09:30:08.877110+0000","last_unstale":"2024-09-16T09:31:35.724619+0000","last_undegraded":"2024-09-16T09:31:35.724619+0000","last_fullsized":"2024-09-16T09:31:35.724619+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:30:03.887571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:30:03.887571+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:31:40.053493+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"66'12","reported_seq":311,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.729128+0000","last_change":"2024-09-16T09:29:02.713335+0000","last_active":"2024-09-16T09:31:35.729128+0000","last_peered":"2024-09-16T09:31:35.729128+0000","last_clean":"2024-09-16T09:31:35.729128+0000","last_became_active":"2024-09-16T09:25:03.715315+0000","last_became_peered":"2024-09-16T09:25:03.715315+0000","last_unstale":"2024-09-16T09:31:35.729128+0000","last_undegraded":"2024-09-16T09:31:35.729128+0000","last_fullsized":"2024-09-16T09:31:35.729128+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-16T09:29:02.713259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:29:02.713259+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":26,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"59'1","reported_seq":328,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.728554+0000","last_change":"2024-09-16T09:31:09.286309+0000","last_active":"2024-09-16T09:31:35.728554+0000","last_peered":"2024-09-16T09:31:35.728554+0000","last_clean":"2024-09-16T09:31:35.728554+0000","last_became_active":"2024-09-16T09:31:09.269627+0000","last_became_peered":"2024-09-16T09:31:09.269627+0000","last_unstale":"2024-09-16T09:31:35.728554+0000","last_undegraded":"2024-09-16T09:31:35.728554+0000","last_fullsized":"2024-09-16T09:31:35.728554+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:27:00.666673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:27:00.666673+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:13.191274+0000","scrub_duration":28,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1986,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"67'5","reported_seq":705,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:36.073244+0000","last_change":"2024-09-16T09:29:19.994791+0000","last_active":"2024-09-16T09:31:36.073244+0000","last_peered":"2024-09-16T09:31:36.073244+0000","last_clean":"2024-09-16T09:31:36.073244+0000","last_became_active":"2024-09-16T09:25:00.696546+0000","last_became_peered":"2024-09-16T09:25:00.696546+0000","last_unstale":"2024-09-16T09:31:36.073244+0000","last_undegraded":"2024-09-16T09:31:36.073244+0000","last_fullsized":"2024-09-16T09:31:36.073244+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'5","last_scrub_stamp":"2024-09-16T09:29:19.994716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:29:19.994716+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0.0059770420000000001,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":390,"num_read_kb":385,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":288,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.725310+0000","last_change":"2024-09-16T09:27:07.337170+0000","last_active":"2024-09-16T09:31:35.725310+0000","last_peered":"2024-09-16T09:31:35.725310+0000","last_clean":"2024-09-16T09:31:35.725310+0000","last_became_active":"2024-09-16T09:25:05.714704+0000","last_became_peered":"2024-09-16T09:25:05.714704+0000","last_unstale":"2024-09-16T09:31:35.725310+0000","last_undegraded":"2024-09-16T09:31:35.725310+0000","last_fullsized":"2024-09-16T09:31:35.725310+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:07.337124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:07.337124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":299,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.718924+0000","last_change":"2024-09-16T09:31:09.271453+0000","last_active":"2024-09-16T09:31:35.718924+0000","last_peered":"2024-09-16T09:31:35.718924+0000","last_clean":"2024-09-16T09:31:35.718924+0000","last_became_active":"2024-09-16T09:31:09.271111+0000","last_became_peered":"2024-09-16T09:31:09.271111+0000","last_unstale":"2024-09-16T09:31:35.718924+0000","last_undegraded":"2024-09-16T09:31:35.718924+0000","last_fullsized":"2024-09-16T09:31:35.718924+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:24.231167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:24.231167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"196'8584","reported_seq":8748,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:37.590844+0000","last_change":"2024-09-16T09:30:56.023710+0000","last_active":"2024-09-16T09:31:37.590844+0000","last_peered":"2024-09-16T09:31:37.590844+0000","last_clean":"2024-09-16T09:31:37.590844+0000","last_became_active":"2024-09-16T09:29:49.450643+0000","last_became_peered":"2024-09-16T09:29:49.450643+0000","last_unstale":"2024-09-16T09:31:37.590844+0000","last_undegraded":"2024-09-16T09:31:37.590844+0000","last_fullsized":"2024-09-16T09:31:37.590844+0000","mapping_epoch":152,"log_start":"169'3400","ondisk_log_start":"169'3400","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"181'5780","last_scrub_stamp":"2024-09-16T09:30:56.023622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:30:56.023622+0000","objects_scrubbed":5716,"log_size":5184,"log_dups_size":0,"ondisk_log_size":5184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:07.379468+0000","scrub_duration":1526,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":457441280,"num_objects":6980,"num_object_clones":0,"num_object_copies":20940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6980,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8584,"num_write_kb":498048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"66'16","reported_seq":403,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.724356+0000","last_change":"2024-09-16T09:30:50.441875+0000","last_active":"2024-09-16T09:31:35.724356+0000","last_peered":"2024-09-16T09:31:35.724356+0000","last_clean":"2024-09-16T09:31:35.724356+0000","last_became_active":"2024-09-16T09:25:03.714118+0000","last_became_peered":"2024-09-16T09:25:03.714118+0000","last_unstale":"2024-09-16T09:31:35.724356+0000","last_undegraded":"2024-09-16T09:31:35.724356+0000","last_fullsized":"2024-09-16T09:31:35.724356+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'16","last_scrub_stamp":"2024-09-16T09:30:50.441795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:30:50.441795+0000","objects_scrubbed":8,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:00.328016+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":146,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.725455+0000","last_change":"2024-09-16T09:31:09.270603+0000","last_active":"2024-09-16T09:31:35.725455+0000","last_peered":"2024-09-16T09:31:35.725455+0000","last_clean":"2024-09-16T09:31:35.725455+0000","last_became_active":"2024-09-16T09:31:09.270196+0000","last_became_peered":"2024-09-16T09:31:09.270196+0000","last_unstale":"2024-09-16T09:31:35.725455+0000","last_undegraded":"2024-09-16T09:31:35.725455+0000","last_fullsized":"2024-09-16T09:31:35.725455+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:49.195331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:27:49.195331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"73'79","reported_seq":378,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.727674+0000","last_change":"2024-09-16T09:31:14.216083+0000","last_active":"2024-09-16T09:31:35.727674+0000","last_peered":"2024-09-16T09:31:35.727674+0000","last_clean":"2024-09-16T09:31:35.727674+0000","last_became_active":"2024-09-16T09:30:08.870394+0000","last_became_peered":"2024-09-16T09:30:08.870394+0000","last_unstale":"2024-09-16T09:31:35.727674+0000","last_undegraded":"2024-09-16T09:31:35.727674+0000","last_fullsized":"2024-09-16T09:31:35.727674+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:02.723794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:02.723794+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:31:42.411236+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":288,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.728888+0000","last_change":"2024-09-16T09:26:39.699133+0000","last_active":"2024-09-16T09:31:35.728888+0000","last_peered":"2024-09-16T09:31:35.728888+0000","last_clean":"2024-09-16T09:31:35.728888+0000","last_became_active":"2024-09-16T09:25:05.714341+0000","last_became_peered":"2024-09-16T09:25:05.714341+0000","last_unstale":"2024-09-16T09:31:35.728888+0000","last_undegraded":"2024-09-16T09:31:35.728888+0000","last_fullsized":"2024-09-16T09:31:35.728888+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:39.699059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:26:39.699059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":292,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.725495+0000","last_change":"2024-09-16T09:29:00.243528+0000","last_active":"2024-09-16T09:31:35.725495+0000","last_peered":"2024-09-16T09:31:35.725495+0000","last_clean":"2024-09-16T09:31:35.725495+0000","last_became_active":"2024-09-16T09:25:07.714281+0000","last_became_peered":"2024-09-16T09:25:07.714281+0000","last_unstale":"2024-09-16T09:31:35.725495+0000","last_undegraded":"2024-09-16T09:31:35.725495+0000","last_fullsized":"2024-09-16T09:31:35.725495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:00.243482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:00.243482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.b","version":"73'79","reported_seq":263,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.721574+0000","last_change":"2024-09-16T09:31:14.199282+0000","last_active":"2024-09-16T09:31:35.721574+0000","last_peered":"2024-09-16T09:31:35.721574+0000","last_clean":"2024-09-16T09:31:35.721574+0000","last_became_active":"2024-09-16T09:30:10.784688+0000","last_became_peered":"2024-09-16T09:30:10.784688+0000","last_unstale":"2024-09-16T09:31:35.721574+0000","last_undegraded":"2024-09-16T09:31:35.721574+0000","last_fullsized":"2024-09-16T09:31:35.721574+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:30:03.887571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:30:03.887571+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:31:43.035265+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,5],"acting":[1,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2","version":"196'8398","reported_seq":8497,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:37.764916+0000","last_change":"2024-09-16T09:29:49.445324+0000","last_active":"2024-09-16T09:31:37.764916+0000","last_peered":"2024-09-16T09:31:37.764916+0000","last_clean":"2024-09-16T09:31:37.764916+0000","last_became_active":"2024-09-16T09:29:49.445022+0000","last_became_peered":"2024-09-16T09:29:49.445022+0000","last_unstale":"2024-09-16T09:31:37.764916+0000","last_undegraded":"2024-09-16T09:31:37.764916+0000","last_fullsized":"2024-09-16T09:31:37.764916+0000","mapping_epoch":152,"log_start":"174'4100","ondisk_log_start":"174'4100","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":4298,"log_dups_size":0,"ondisk_log_size":4298,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":448135168,"num_objects":6838,"num_object_clones":0,"num_object_copies":20514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6838,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8398,"num_write_kb":487552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"66'11","reported_seq":330,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.718577+0000","last_change":"2024-09-16T09:30:05.646701+0000","last_active":"2024-09-16T09:31:35.718577+0000","last_peered":"2024-09-16T09:31:35.718577+0000","last_clean":"2024-09-16T09:31:35.718577+0000","last_became_active":"2024-09-16T09:26:33.287399+0000","last_became_peered":"2024-09-16T09:26:33.287399+0000","last_unstale":"2024-09-16T09:31:35.718577+0000","last_undegraded":"2024-09-16T09:31:35.718577+0000","last_fullsized":"2024-09-16T09:31:35.718577+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-16T09:30:05.646500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:30:05.646500+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":289,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.716806+0000","last_change":"2024-09-16T09:25:01.708568+0000","last_active":"2024-09-16T09:31:35.716806+0000","last_peered":"2024-09-16T09:31:35.716806+0000","last_clean":"2024-09-16T09:31:35.716806+0000","last_became_active":"2024-09-16T09:25:01.708270+0000","last_became_peered":"2024-09-16T09:25:01.708270+0000","last_unstale":"2024-09-16T09:31:35.716806+0000","last_undegraded":"2024-09-16T09:31:35.716806+0000","last_fullsized":"2024-09-16T09:31:35.716806+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":287,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.722093+0000","last_change":"2024-09-16T09:26:34.171190+0000","last_active":"2024-09-16T09:31:35.722093+0000","last_peered":"2024-09-16T09:31:35.722093+0000","last_clean":"2024-09-16T09:31:35.722093+0000","last_became_active":"2024-09-16T09:25:05.711964+0000","last_became_peered":"2024-09-16T09:25:05.711964+0000","last_unstale":"2024-09-16T09:31:35.722093+0000","last_undegraded":"2024-09-16T09:31:35.722093+0000","last_fullsized":"2024-09-16T09:31:35.722093+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:34.171139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:26:34.171139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":163,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.724022+0000","last_change":"2024-09-16T09:31:09.278959+0000","last_active":"2024-09-16T09:31:35.724022+0000","last_peered":"2024-09-16T09:31:35.724022+0000","last_clean":"2024-09-16T09:31:35.724022+0000","last_became_active":"2024-09-16T09:31:09.278618+0000","last_became_peered":"2024-09-16T09:31:09.278618+0000","last_unstale":"2024-09-16T09:31:35.724022+0000","last_undegraded":"2024-09-16T09:31:35.724022+0000","last_fullsized":"2024-09-16T09:31:35.724022+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:30:33.410232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:30:33.410232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:00.093285+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1","version":"196'8380","reported_seq":8477,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:37.819862+0000","last_change":"2024-09-16T09:29:49.449796+0000","last_active":"2024-09-16T09:31:37.819862+0000","last_peered":"2024-09-16T09:31:37.819862+0000","last_clean":"2024-09-16T09:31:37.819862+0000","last_became_active":"2024-09-16T09:29:49.449475+0000","last_became_peered":"2024-09-16T09:29:49.449475+0000","last_unstale":"2024-09-16T09:31:37.819862+0000","last_undegraded":"2024-09-16T09:31:37.819862+0000","last_fullsized":"2024-09-16T09:31:37.819862+0000","mapping_epoch":152,"log_start":"173'3900","ondisk_log_start":"173'3900","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":4480,"log_dups_size":0,"ondisk_log_size":4480,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":450756608,"num_objects":6878,"num_object_clones":0,"num_object_copies":20634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6878,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8380,"num_write_kb":488256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.8","version":"73'79","reported_seq":260,"reported_epoch":196,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-16T09:31:35.731323+0000","last_change":"2024-09-16T09:31:09.430087+0000","last_active":"2024-09-16T09:31:35.731323+0000","last_peered":"2024-09-16T09:31:35.731323+0000","last_clean":"2024-09-16T09:31:07.261934+0000","last_became_active":"2024-09-16T09:31:09.406638+0000","last_became_peered":"2024-09-16T09:31:09.406638+0000","last_unstale":"2024-09-16T09:31:35.731323+0000","last_undegraded":"2024-09-16T09:31:09.427685+0000","last_fullsized":"2024-09-16T09:31:35.731323+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:17.482769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:17.482769+0000","objects_scrubbed":1,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262688,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":262688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":["5","2","6"],"object_location_counts":[{"shards":"2,5,6","objects":1}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"66'17","reported_seq":336,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.717515+0000","last_change":"2024-09-16T09:30:04.636115+0000","last_active":"2024-09-16T09:31:35.717515+0000","last_peered":"2024-09-16T09:31:35.717515+0000","last_clean":"2024-09-16T09:31:35.717515+0000","last_became_active":"2024-09-16T09:25:03.713358+0000","last_became_peered":"2024-09-16T09:25:03.713358+0000","last_unstale":"2024-09-16T09:31:35.717515+0000","last_undegraded":"2024-09-16T09:31:35.717515+0000","last_fullsized":"2024-09-16T09:31:35.717515+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'17","last_scrub_stamp":"2024-09-16T09:30:04.636039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:30:04.636039+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":297,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.723652+0000","last_change":"2024-09-16T09:26:04.104188+0000","last_active":"2024-09-16T09:31:35.723652+0000","last_peered":"2024-09-16T09:31:35.723652+0000","last_clean":"2024-09-16T09:31:35.723652+0000","last_became_active":"2024-09-16T09:25:01.703355+0000","last_became_peered":"2024-09-16T09:25:01.703355+0000","last_unstale":"2024-09-16T09:31:35.723652+0000","last_undegraded":"2024-09-16T09:31:35.723652+0000","last_fullsized":"2024-09-16T09:31:35.723652+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:04.104130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:04.104130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:31:46.426148+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":304,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.719026+0000","last_change":"2024-09-16T09:29:58.839132+0000","last_active":"2024-09-16T09:31:35.719026+0000","last_peered":"2024-09-16T09:31:35.719026+0000","last_clean":"2024-09-16T09:31:35.719026+0000","last_became_active":"2024-09-16T09:25:05.716900+0000","last_became_peered":"2024-09-16T09:25:05.716900+0000","last_unstale":"2024-09-16T09:31:35.719026+0000","last_undegraded":"2024-09-16T09:31:35.719026+0000","last_fullsized":"2024-09-16T09:31:35.719026+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:58.839051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:58.839051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":314,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.727856+0000","last_change":"2024-09-16T09:31:09.276869+0000","last_active":"2024-09-16T09:31:35.727856+0000","last_peered":"2024-09-16T09:31:35.727856+0000","last_clean":"2024-09-16T09:31:35.727856+0000","last_became_active":"2024-09-16T09:31:09.276275+0000","last_became_peered":"2024-09-16T09:31:09.276275+0000","last_unstale":"2024-09-16T09:31:35.727856+0000","last_undegraded":"2024-09-16T09:31:35.727856+0000","last_fullsized":"2024-09-16T09:31:35.727856+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:30:03.946025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:30:03.946025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.0","version":"196'8538","reported_seq":12494,"reported_epoch":196,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-16T09:31:37.853646+0000","last_change":"2024-09-16T09:31:09.476408+0000","last_active":"2024-09-16T09:31:37.853646+0000","last_peered":"2024-09-16T09:31:37.853646+0000","last_clean":"2024-09-16T09:30:20.486290+0000","last_became_active":"2024-09-16T09:31:09.399514+0000","last_became_peered":"2024-09-16T09:31:09.399514+0000","last_unstale":"2024-09-16T09:31:37.853646+0000","last_undegraded":"2024-09-16T09:31:37.853646+0000","last_fullsized":"2024-09-16T09:31:09.254572+0000","mapping_epoch":186,"log_start":"172'3800","ondisk_log_start":"172'3800","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":4738,"log_dups_size":0,"ondisk_log_size":4738,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":458620928,"num_objects":6998,"num_object_clones":0,"num_object_copies":20994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2778,"num_objects_unfound":0,"num_objects_dirty":6998,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8538,"num_write_kb":497152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1924,"num_bytes_recovered":126091264,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,1],"avail_no_missing":["5","1"],"object_location_counts":[{"shards":"1,4,5","objects":2778},{"shards":"1,5","objects":4220}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.9","version":"73'79","reported_seq":368,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.728004+0000","last_change":"2024-09-16T09:29:19.013948+0000","last_active":"2024-09-16T09:31:35.728004+0000","last_peered":"2024-09-16T09:31:35.728004+0000","last_clean":"2024-09-16T09:31:35.728004+0000","last_became_active":"2024-09-16T09:26:35.126066+0000","last_became_peered":"2024-09-16T09:26:35.126066+0000","last_unstale":"2024-09-16T09:31:35.728004+0000","last_undegraded":"2024-09-16T09:31:35.728004+0000","last_fullsized":"2024-09-16T09:31:35.728004+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:19.013865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:19.013865+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"66'10","reported_seq":299,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.717884+0000","last_change":"2024-09-16T09:25:03.708144+0000","last_active":"2024-09-16T09:31:35.717884+0000","last_peered":"2024-09-16T09:31:35.717884+0000","last_clean":"2024-09-16T09:31:35.717884+0000","last_became_active":"2024-09-16T09:25:03.707723+0000","last_became_peered":"2024-09-16T09:25:03.707723+0000","last_unstale":"2024-09-16T09:31:35.717884+0000","last_undegraded":"2024-09-16T09:31:35.717884+0000","last_fullsized":"2024-09-16T09:31:35.717884+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":162,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.728619+0000","last_change":"2024-09-16T09:31:32.253397+0000","last_active":"2024-09-16T09:31:35.728619+0000","last_peered":"2024-09-16T09:31:35.728619+0000","last_clean":"2024-09-16T09:31:35.728619+0000","last_became_active":"2024-09-16T09:31:09.288825+0000","last_became_peered":"2024-09-16T09:31:09.288825+0000","last_unstale":"2024-09-16T09:31:35.728619+0000","last_undegraded":"2024-09-16T09:31:35.728619+0000","last_fullsized":"2024-09-16T09:31:35.728619+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:31:32.253306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:31:32.253306+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:45.682494+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":395,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.728366+0000","last_change":"2024-09-16T09:29:29.866520+0000","last_active":"2024-09-16T09:31:35.728366+0000","last_peered":"2024-09-16T09:31:35.728366+0000","last_clean":"2024-09-16T09:31:35.728366+0000","last_became_active":"2024-09-16T09:25:05.711590+0000","last_became_peered":"2024-09-16T09:25:05.711590+0000","last_unstale":"2024-09-16T09:31:35.728366+0000","last_undegraded":"2024-09-16T09:31:35.728366+0000","last_fullsized":"2024-09-16T09:31:35.728366+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:29:29.866460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:29.866460+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":300,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.718315+0000","last_change":"2024-09-16T09:29:02.667332+0000","last_active":"2024-09-16T09:31:35.718315+0000","last_peered":"2024-09-16T09:31:35.718315+0000","last_clean":"2024-09-16T09:31:35.718315+0000","last_became_active":"2024-09-16T09:25:07.718958+0000","last_became_peered":"2024-09-16T09:25:07.718958+0000","last_unstale":"2024-09-16T09:31:35.718315+0000","last_undegraded":"2024-09-16T09:31:35.718315+0000","last_fullsized":"2024-09-16T09:31:35.718315+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:02.667259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:02.667259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.e","version":"73'79","reported_seq":87,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.718170+0000","last_change":"2024-09-16T09:31:14.198121+0000","last_active":"2024-09-16T09:31:35.718170+0000","last_peered":"2024-09-16T09:31:35.718170+0000","last_clean":"2024-09-16T09:31:35.718170+0000","last_became_active":"2024-09-16T09:30:10.705072+0000","last_became_peered":"2024-09-16T09:30:10.705072+0000","last_unstale":"2024-09-16T09:31:35.718170+0000","last_undegraded":"2024-09-16T09:31:35.718170+0000","last_fullsized":"2024-09-16T09:31:35.718170+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:26:35.170229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:26:35.170229+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:31:41.814604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26835,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":10,"num_write":5,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":26835,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,3],"acting":[4,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.7","version":"196'8519","reported_seq":8618,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:37.802649+0000","last_change":"2024-09-16T09:29:49.839740+0000","last_active":"2024-09-16T09:31:37.802649+0000","last_peered":"2024-09-16T09:31:37.802649+0000","last_clean":"2024-09-16T09:31:37.802649+0000","last_became_active":"2024-09-16T09:29:49.839437+0000","last_became_peered":"2024-09-16T09:29:49.839437+0000","last_unstale":"2024-09-16T09:31:37.802649+0000","last_undegraded":"2024-09-16T09:31:37.802649+0000","last_fullsized":"2024-09-16T09:31:37.802649+0000","mapping_epoch":152,"log_start":"173'4000","ondisk_log_start":"173'4000","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":4519,"log_dups_size":0,"ondisk_log_size":4519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":453836800,"num_objects":6925,"num_object_clones":0,"num_object_copies":20775,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6925,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8519,"num_write_kb":494208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"66'9","reported_seq":324,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.728323+0000","last_change":"2024-09-16T09:31:31.211495+0000","last_active":"2024-09-16T09:31:35.728323+0000","last_peered":"2024-09-16T09:31:35.728323+0000","last_clean":"2024-09-16T09:31:35.728323+0000","last_became_active":"2024-09-16T09:25:03.712230+0000","last_became_peered":"2024-09-16T09:25:03.712230+0000","last_unstale":"2024-09-16T09:31:35.728323+0000","last_undegraded":"2024-09-16T09:31:35.728323+0000","last_fullsized":"2024-09-16T09:31:35.728323+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:31:31.211420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:31:31.211420+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:36.499658+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":309,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.725523+0000","last_change":"2024-09-16T09:31:09.277413+0000","last_active":"2024-09-16T09:31:35.725523+0000","last_peered":"2024-09-16T09:31:35.725523+0000","last_clean":"2024-09-16T09:31:35.725523+0000","last_became_active":"2024-09-16T09:31:09.276516+0000","last_became_peered":"2024-09-16T09:31:09.276516+0000","last_unstale":"2024-09-16T09:31:35.725523+0000","last_undegraded":"2024-09-16T09:31:35.725523+0000","last_fullsized":"2024-09-16T09:31:35.725523+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:26.061106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:26.061106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":311,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.726021+0000","last_change":"2024-09-16T09:31:09.275219+0000","last_active":"2024-09-16T09:31:35.726021+0000","last_peered":"2024-09-16T09:31:35.726021+0000","last_clean":"2024-09-16T09:31:35.726021+0000","last_became_active":"2024-09-16T09:31:09.274702+0000","last_became_peered":"2024-09-16T09:31:09.274702+0000","last_unstale":"2024-09-16T09:31:35.726021+0000","last_undegraded":"2024-09-16T09:31:35.726021+0000","last_fullsized":"2024-09-16T09:31:35.726021+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:57.220068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:28:57.220068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":33,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":301,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.728503+0000","last_change":"2024-09-16T09:31:27.185957+0000","last_active":"2024-09-16T09:31:35.728503+0000","last_peered":"2024-09-16T09:31:35.728503+0000","last_clean":"2024-09-16T09:31:35.728503+0000","last_became_active":"2024-09-16T09:25:07.718778+0000","last_became_peered":"2024-09-16T09:25:07.718778+0000","last_unstale":"2024-09-16T09:31:35.728503+0000","last_undegraded":"2024-09-16T09:31:35.728503+0000","last_fullsized":"2024-09-16T09:31:35.728503+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:31:27.185801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:31:27.185801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:39.017009+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.f","version":"73'79","reported_seq":339,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.724564+0000","last_change":"2024-09-16T09:31:14.210208+0000","last_active":"2024-09-16T09:31:35.724564+0000","last_peered":"2024-09-16T09:31:35.724564+0000","last_clean":"2024-09-16T09:31:35.724564+0000","last_became_active":"2024-09-16T09:30:10.693477+0000","last_became_peered":"2024-09-16T09:30:10.693477+0000","last_unstale":"2024-09-16T09:31:35.724564+0000","last_undegraded":"2024-09-16T09:31:35.724564+0000","last_fullsized":"2024-09-16T09:31:35.724564+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:17.534429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:17.534429+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:31:43.408019+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.6","version":"196'8773","reported_seq":8872,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:38.120451+0000","last_change":"2024-09-16T09:29:50.008888+0000","last_active":"2024-09-16T09:31:38.120451+0000","last_peered":"2024-09-16T09:31:38.120451+0000","last_clean":"2024-09-16T09:31:38.120451+0000","last_became_active":"2024-09-16T09:29:50.008656+0000","last_became_peered":"2024-09-16T09:29:50.008656+0000","last_unstale":"2024-09-16T09:31:38.120451+0000","last_undegraded":"2024-09-16T09:31:38.120451+0000","last_fullsized":"2024-09-16T09:31:38.120451+0000","mapping_epoch":152,"log_start":"169'3600","ondisk_log_start":"169'3600","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":5173,"log_dups_size":0,"ondisk_log_size":5173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":461963264,"num_objects":7049,"num_object_clones":0,"num_object_copies":21147,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7049,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8773,"num_write_kb":506304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"66'19","reported_seq":332,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.723598+0000","last_change":"2024-09-16T09:29:02.554357+0000","last_active":"2024-09-16T09:31:35.723598+0000","last_peered":"2024-09-16T09:31:35.723598+0000","last_clean":"2024-09-16T09:31:35.723598+0000","last_became_active":"2024-09-16T09:25:03.712783+0000","last_became_peered":"2024-09-16T09:25:03.712783+0000","last_unstale":"2024-09-16T09:31:35.723598+0000","last_undegraded":"2024-09-16T09:31:35.723598+0000","last_fullsized":"2024-09-16T09:31:35.723598+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-16T09:29:02.554274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:29:02.554274+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:17.371584+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"59'1","reported_seq":309,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.730921+0000","last_change":"2024-09-16T09:29:51.025505+0000","last_active":"2024-09-16T09:31:35.730921+0000","last_peered":"2024-09-16T09:31:35.730921+0000","last_clean":"2024-09-16T09:31:35.730921+0000","last_became_active":"2024-09-16T09:25:01.708069+0000","last_became_peered":"2024-09-16T09:25:01.708069+0000","last_unstale":"2024-09-16T09:31:35.730921+0000","last_undegraded":"2024-09-16T09:31:35.730921+0000","last_fullsized":"2024-09-16T09:31:35.730921+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:29:51.025444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:51.025444+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":288,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.725877+0000","last_change":"2024-09-16T09:27:11.067059+0000","last_active":"2024-09-16T09:31:35.725877+0000","last_peered":"2024-09-16T09:31:35.725877+0000","last_clean":"2024-09-16T09:31:35.725877+0000","last_became_active":"2024-09-16T09:25:05.709264+0000","last_became_peered":"2024-09-16T09:25:05.709264+0000","last_unstale":"2024-09-16T09:31:35.725877+0000","last_undegraded":"2024-09-16T09:31:35.725877+0000","last_fullsized":"2024-09-16T09:31:35.725877+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:11.066986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:11.066986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":283,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.727589+0000","last_change":"2024-09-16T09:27:25.865107+0000","last_active":"2024-09-16T09:31:35.727589+0000","last_peered":"2024-09-16T09:31:35.727589+0000","last_clean":"2024-09-16T09:31:35.727589+0000","last_became_active":"2024-09-16T09:25:07.718750+0000","last_became_peered":"2024-09-16T09:25:07.718750+0000","last_unstale":"2024-09-16T09:31:35.727589+0000","last_undegraded":"2024-09-16T09:31:35.727589+0000","last_fullsized":"2024-09-16T09:31:35.727589+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:25.865030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:25.865030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.c","version":"73'79","reported_seq":34,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.719783+0000","last_change":"2024-09-16T09:31:09.290919+0000","last_active":"2024-09-16T09:31:35.719783+0000","last_peered":"2024-09-16T09:31:35.719783+0000","last_clean":"2024-09-16T09:31:35.719783+0000","last_became_active":"2024-09-16T09:31:09.273770+0000","last_became_peered":"2024-09-16T09:31:09.273770+0000","last_unstale":"2024-09-16T09:31:35.719783+0000","last_undegraded":"2024-09-16T09:31:35.719783+0000","last_fullsized":"2024-09-16T09:31:35.719783+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:30:22.614462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:30:22.614462+0000","objects_scrubbed":1,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:31:40.667658+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":327680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.5","version":"196'8628","reported_seq":8800,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:37.548096+0000","last_change":"2024-09-16T09:31:06.121217+0000","last_active":"2024-09-16T09:31:37.548096+0000","last_peered":"2024-09-16T09:31:37.548096+0000","last_clean":"2024-09-16T09:31:37.548096+0000","last_became_active":"2024-09-16T09:29:50.009013+0000","last_became_peered":"2024-09-16T09:29:50.009013+0000","last_unstale":"2024-09-16T09:31:37.548096+0000","last_undegraded":"2024-09-16T09:31:37.548096+0000","last_fullsized":"2024-09-16T09:31:37.548096+0000","mapping_epoch":152,"log_start":"168'3413","ondisk_log_start":"168'3413","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"184'6587","last_scrub_stamp":"2024-09-16T09:31:06.120990+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:31:06.120990+0000","objects_scrubbed":6523,"log_size":5215,"log_dups_size":0,"ondisk_log_size":5215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:20.260788+0000","scrub_duration":1690,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":466354176,"num_objects":7116,"num_object_clones":0,"num_object_copies":21348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7116,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8628,"num_write_kb":503808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"66'12","reported_seq":362,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.716598+0000","last_change":"2024-09-16T09:31:09.296915+0000","last_active":"2024-09-16T09:31:35.716598+0000","last_peered":"2024-09-16T09:31:35.716598+0000","last_clean":"2024-09-16T09:31:35.716598+0000","last_became_active":"2024-09-16T09:31:09.269047+0000","last_became_peered":"2024-09-16T09:31:09.269047+0000","last_unstale":"2024-09-16T09:31:35.716598+0000","last_undegraded":"2024-09-16T09:31:35.716598+0000","last_fullsized":"2024-09-16T09:31:35.716598+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-16T09:30:21.728668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:30:21.728668+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":303,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.729166+0000","last_change":"2024-09-16T09:29:53.075288+0000","last_active":"2024-09-16T09:31:35.729166+0000","last_peered":"2024-09-16T09:31:35.729166+0000","last_clean":"2024-09-16T09:31:35.729166+0000","last_became_active":"2024-09-16T09:25:01.707528+0000","last_became_peered":"2024-09-16T09:25:01.707528+0000","last_unstale":"2024-09-16T09:31:35.729166+0000","last_undegraded":"2024-09-16T09:31:35.729166+0000","last_fullsized":"2024-09-16T09:31:35.729166+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:53.075232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:53.075232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":297,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.725682+0000","last_change":"2024-09-16T09:27:49.965628+0000","last_active":"2024-09-16T09:31:35.725682+0000","last_peered":"2024-09-16T09:31:35.725682+0000","last_clean":"2024-09-16T09:31:35.725682+0000","last_became_active":"2024-09-16T09:25:05.715253+0000","last_became_peered":"2024-09-16T09:25:05.715253+0000","last_unstale":"2024-09-16T09:31:35.725682+0000","last_undegraded":"2024-09-16T09:31:35.725682+0000","last_fullsized":"2024-09-16T09:31:35.725682+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:49.965424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:49.965424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":284,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.717609+0000","last_change":"2024-09-16T09:27:56.059755+0000","last_active":"2024-09-16T09:31:35.717609+0000","last_peered":"2024-09-16T09:31:35.717609+0000","last_clean":"2024-09-16T09:31:35.717609+0000","last_became_active":"2024-09-16T09:25:07.718899+0000","last_became_peered":"2024-09-16T09:25:07.718899+0000","last_unstale":"2024-09-16T09:31:35.717609+0000","last_undegraded":"2024-09-16T09:31:35.717609+0000","last_fullsized":"2024-09-16T09:31:35.717609+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:56.059687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:56.059687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.d","version":"73'79","reported_seq":87,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.730357+0000","last_change":"2024-09-16T09:31:14.215723+0000","last_active":"2024-09-16T09:31:35.730357+0000","last_peered":"2024-09-16T09:31:35.730357+0000","last_clean":"2024-09-16T09:31:35.730357+0000","last_became_active":"2024-09-16T09:30:10.709958+0000","last_became_peered":"2024-09-16T09:30:10.709958+0000","last_unstale":"2024-09-16T09:31:35.730357+0000","last_undegraded":"2024-09-16T09:31:35.730357+0000","last_fullsized":"2024-09-16T09:31:35.730357+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:58.568841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:58.568841+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:31:43.390312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26835,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":10,"num_write":5,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":26835,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.4","version":"196'8619","reported_seq":8718,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:37.860697+0000","last_change":"2024-09-16T09:29:49.444857+0000","last_active":"2024-09-16T09:31:37.860697+0000","last_peered":"2024-09-16T09:31:37.860697+0000","last_clean":"2024-09-16T09:31:37.860697+0000","last_became_active":"2024-09-16T09:29:49.444515+0000","last_became_peered":"2024-09-16T09:29:49.444515+0000","last_unstale":"2024-09-16T09:31:37.860697+0000","last_undegraded":"2024-09-16T09:31:37.860697+0000","last_fullsized":"2024-09-16T09:31:37.860697+0000","mapping_epoch":152,"log_start":"172'3900","ondisk_log_start":"172'3900","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":4719,"log_dups_size":0,"ondisk_log_size":4719,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":458031104,"num_objects":6989,"num_object_clones":0,"num_object_copies":20967,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6989,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8619,"num_write_kb":499456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"66'15","reported_seq":317,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.731498+0000","last_change":"2024-09-16T09:26:33.138828+0000","last_active":"2024-09-16T09:31:35.731498+0000","last_peered":"2024-09-16T09:31:35.731498+0000","last_clean":"2024-09-16T09:31:35.731498+0000","last_became_active":"2024-09-16T09:25:03.715733+0000","last_became_peered":"2024-09-16T09:25:03.715733+0000","last_unstale":"2024-09-16T09:31:35.731498+0000","last_undegraded":"2024-09-16T09:31:35.731498+0000","last_fullsized":"2024-09-16T09:31:35.731498+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-16T09:26:33.138603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:26:33.138603+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":307,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.729316+0000","last_change":"2024-09-16T09:26:26.705847+0000","last_active":"2024-09-16T09:31:35.729316+0000","last_peered":"2024-09-16T09:31:35.729316+0000","last_clean":"2024-09-16T09:31:35.729316+0000","last_became_active":"2024-09-16T09:25:01.707372+0000","last_became_peered":"2024-09-16T09:25:01.707372+0000","last_unstale":"2024-09-16T09:31:35.729316+0000","last_undegraded":"2024-09-16T09:31:35.729316+0000","last_fullsized":"2024-09-16T09:31:35.729316+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-16T09:26:26.705773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:26.705773+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"66'8","reported_seq":392,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:36.839984+0000","last_change":"2024-09-16T09:30:05.138362+0000","last_active":"2024-09-16T09:31:36.839984+0000","last_peered":"2024-09-16T09:31:36.839984+0000","last_clean":"2024-09-16T09:31:36.839984+0000","last_became_active":"2024-09-16T09:25:05.716189+0000","last_became_peered":"2024-09-16T09:25:05.716189+0000","last_unstale":"2024-09-16T09:31:36.839984+0000","last_undegraded":"2024-09-16T09:31:36.839984+0000","last_fullsized":"2024-09-16T09:31:36.839984+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:30:05.138288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:30:05.138288+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":283,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.723404+0000","last_change":"2024-09-16T09:27:20.219376+0000","last_active":"2024-09-16T09:31:35.723404+0000","last_peered":"2024-09-16T09:31:35.723404+0000","last_clean":"2024-09-16T09:31:35.723404+0000","last_became_active":"2024-09-16T09:25:07.717213+0000","last_became_peered":"2024-09-16T09:25:07.717213+0000","last_unstale":"2024-09-16T09:31:35.723404+0000","last_undegraded":"2024-09-16T09:31:35.723404+0000","last_fullsized":"2024-09-16T09:31:35.723404+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:20.219333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:20.219333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.12","version":"73'79","reported_seq":87,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.725303+0000","last_change":"2024-09-16T09:31:14.196160+0000","last_active":"2024-09-16T09:31:35.725303+0000","last_peered":"2024-09-16T09:31:35.725303+0000","last_clean":"2024-09-16T09:31:35.725303+0000","last_became_active":"2024-09-16T09:30:10.731413+0000","last_became_peered":"2024-09-16T09:30:10.731413+0000","last_unstale":"2024-09-16T09:31:35.725303+0000","last_undegraded":"2024-09-16T09:31:35.725303+0000","last_fullsized":"2024-09-16T09:31:35.725303+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:26:35.745837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:26:35.745837+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:31:43.386292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26835,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":10,"num_write":6,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":26835,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":296,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.723282+0000","last_change":"2024-09-16T09:26:33.139405+0000","last_active":"2024-09-16T09:31:35.723282+0000","last_peered":"2024-09-16T09:31:35.723282+0000","last_clean":"2024-09-16T09:31:35.723282+0000","last_became_active":"2024-09-16T09:25:01.703202+0000","last_became_peered":"2024-09-16T09:25:01.703202+0000","last_unstale":"2024-09-16T09:31:35.723282+0000","last_undegraded":"2024-09-16T09:31:35.723282+0000","last_fullsized":"2024-09-16T09:31:35.723282+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:33.139360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:33.139360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:10.508509+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"66'6","reported_seq":38,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.717770+0000","last_change":"2024-09-16T09:31:09.297514+0000","last_active":"2024-09-16T09:31:35.717770+0000","last_peered":"2024-09-16T09:31:35.717770+0000","last_clean":"2024-09-16T09:31:35.717770+0000","last_became_active":"2024-09-16T09:31:09.270685+0000","last_became_peered":"2024-09-16T09:31:09.270685+0000","last_unstale":"2024-09-16T09:31:35.717770+0000","last_undegraded":"2024-09-16T09:31:35.717770+0000","last_fullsized":"2024-09-16T09:31:35.717770+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'6","last_scrub_stamp":"2024-09-16T09:26:23.458794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:26:23.458794+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":8,"num_read_kb":5,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":295,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.722244+0000","last_change":"2024-09-16T09:29:31.496374+0000","last_active":"2024-09-16T09:31:35.722244+0000","last_peered":"2024-09-16T09:31:35.722244+0000","last_clean":"2024-09-16T09:31:35.722244+0000","last_became_active":"2024-09-16T09:25:05.715616+0000","last_became_peered":"2024-09-16T09:25:05.715616+0000","last_unstale":"2024-09-16T09:31:35.722244+0000","last_undegraded":"2024-09-16T09:31:35.722244+0000","last_fullsized":"2024-09-16T09:31:35.722244+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:31.496304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:31.496304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":283,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.730526+0000","last_change":"2024-09-16T09:26:28.733381+0000","last_active":"2024-09-16T09:31:35.730526+0000","last_peered":"2024-09-16T09:31:35.730526+0000","last_clean":"2024-09-16T09:31:35.730526+0000","last_became_active":"2024-09-16T09:25:07.721059+0000","last_became_peered":"2024-09-16T09:25:07.721059+0000","last_unstale":"2024-09-16T09:31:35.730526+0000","last_undegraded":"2024-09-16T09:31:35.730526+0000","last_fullsized":"2024-09-16T09:31:35.730526+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:28.733324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:28.733324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.13","version":"73'79","reported_seq":279,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.724468+0000","last_change":"2024-09-16T09:31:09.288677+0000","last_active":"2024-09-16T09:31:35.724468+0000","last_peered":"2024-09-16T09:31:35.724468+0000","last_clean":"2024-09-16T09:31:35.724468+0000","last_became_active":"2024-09-16T09:31:09.288233+0000","last_became_peered":"2024-09-16T09:31:09.288233+0000","last_unstale":"2024-09-16T09:31:35.724468+0000","last_undegraded":"2024-09-16T09:31:35.724468+0000","last_fullsized":"2024-09-16T09:31:35.724468+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:30:21.714369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:30:21.714369+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:31:50.034245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.16","version":"66'9","reported_seq":358,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.729210+0000","last_change":"2024-09-16T09:31:09.309570+0000","last_active":"2024-09-16T09:31:35.729210+0000","last_peered":"2024-09-16T09:31:35.729210+0000","last_clean":"2024-09-16T09:31:35.729210+0000","last_became_active":"2024-09-16T09:31:09.273498+0000","last_became_peered":"2024-09-16T09:31:09.273498+0000","last_unstale":"2024-09-16T09:31:35.729210+0000","last_undegraded":"2024-09-16T09:31:35.729210+0000","last_fullsized":"2024-09-16T09:31:35.729210+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:29:58.867513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:29:58.867513+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":303,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.728222+0000","last_change":"2024-09-16T09:28:50.698331+0000","last_active":"2024-09-16T09:31:35.728222+0000","last_peered":"2024-09-16T09:31:35.728222+0000","last_clean":"2024-09-16T09:31:35.728222+0000","last_became_active":"2024-09-16T09:25:01.706491+0000","last_became_peered":"2024-09-16T09:25:01.706491+0000","last_unstale":"2024-09-16T09:31:35.728222+0000","last_undegraded":"2024-09-16T09:31:35.728222+0000","last_fullsized":"2024-09-16T09:31:35.728222+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:50.698288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:28:50.698288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":154,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.721941+0000","last_change":"2024-09-16T09:31:09.271558+0000","last_active":"2024-09-16T09:31:35.721941+0000","last_peered":"2024-09-16T09:31:35.721941+0000","last_clean":"2024-09-16T09:31:35.721941+0000","last_became_active":"2024-09-16T09:31:09.271223+0000","last_became_peered":"2024-09-16T09:31:09.271223+0000","last_unstale":"2024-09-16T09:31:35.721941+0000","last_undegraded":"2024-09-16T09:31:35.721941+0000","last_fullsized":"2024-09-16T09:31:35.721941+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:53.810580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:28:53.810580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":284,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.726932+0000","last_change":"2024-09-16T09:26:35.866582+0000","last_active":"2024-09-16T09:31:35.726932+0000","last_peered":"2024-09-16T09:31:35.726932+0000","last_clean":"2024-09-16T09:31:35.726932+0000","last_became_active":"2024-09-16T09:25:07.722050+0000","last_became_peered":"2024-09-16T09:25:07.722050+0000","last_unstale":"2024-09-16T09:31:35.726932+0000","last_undegraded":"2024-09-16T09:31:35.726932+0000","last_fullsized":"2024-09-16T09:31:35.726932+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:35.866512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:35.866512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:08.904503+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.10","version":"73'79","reported_seq":87,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.725094+0000","last_change":"2024-09-16T09:31:14.195080+0000","last_active":"2024-09-16T09:31:35.725094+0000","last_peered":"2024-09-16T09:31:35.725094+0000","last_clean":"2024-09-16T09:31:35.725094+0000","last_became_active":"2024-09-16T09:30:10.692770+0000","last_became_peered":"2024-09-16T09:30:10.692770+0000","last_unstale":"2024-09-16T09:31:35.725094+0000","last_undegraded":"2024-09-16T09:31:35.725094+0000","last_fullsized":"2024-09-16T09:31:35.725094+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:02.723794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:02.723794+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:31:42.406568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26835,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":10,"num_write":5,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":26835,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"66'9","reported_seq":315,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.728217+0000","last_change":"2024-09-16T09:29:32.485165+0000","last_active":"2024-09-16T09:31:35.728217+0000","last_peered":"2024-09-16T09:31:35.728217+0000","last_clean":"2024-09-16T09:31:35.728217+0000","last_became_active":"2024-09-16T09:25:03.717310+0000","last_became_peered":"2024-09-16T09:25:03.717310+0000","last_unstale":"2024-09-16T09:31:35.728217+0000","last_undegraded":"2024-09-16T09:31:35.728217+0000","last_fullsized":"2024-09-16T09:31:35.728217+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:29:32.485084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:29:32.485084+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":318,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.728619+0000","last_change":"2024-09-16T09:31:09.273453+0000","last_active":"2024-09-16T09:31:35.728619+0000","last_peered":"2024-09-16T09:31:35.728619+0000","last_clean":"2024-09-16T09:31:35.728619+0000","last_became_active":"2024-09-16T09:31:09.272848+0000","last_became_peered":"2024-09-16T09:31:09.272848+0000","last_unstale":"2024-09-16T09:31:35.728619+0000","last_undegraded":"2024-09-16T09:31:35.728619+0000","last_fullsized":"2024-09-16T09:31:35.728619+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.950144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.950144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"187'11","reported_seq":221,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.728876+0000","last_change":"2024-09-16T09:31:09.311243+0000","last_active":"2024-09-16T09:31:35.728876+0000","last_peered":"2024-09-16T09:31:35.728876+0000","last_clean":"2024-09-16T09:31:35.728876+0000","last_became_active":"2024-09-16T09:31:09.277688+0000","last_became_peered":"2024-09-16T09:31:09.277688+0000","last_unstale":"2024-09-16T09:31:35.728876+0000","last_undegraded":"2024-09-16T09:31:35.728876+0000","last_fullsized":"2024-09-16T09:31:35.728876+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:28:53.714413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:28:53.714413+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":300,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.717365+0000","last_change":"2024-09-16T09:29:49.474440+0000","last_active":"2024-09-16T09:31:35.717365+0000","last_peered":"2024-09-16T09:31:35.717365+0000","last_clean":"2024-09-16T09:31:35.717365+0000","last_became_active":"2024-09-16T09:25:07.718810+0000","last_became_peered":"2024-09-16T09:25:07.718810+0000","last_unstale":"2024-09-16T09:31:35.717365+0000","last_undegraded":"2024-09-16T09:31:35.717365+0000","last_fullsized":"2024-09-16T09:31:35.717365+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:49.474391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:49.474391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.11","version":"73'79","reported_seq":179,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.728656+0000","last_change":"2024-09-16T09:31:09.272938+0000","last_active":"2024-09-16T09:31:35.728656+0000","last_peered":"2024-09-16T09:31:35.728656+0000","last_clean":"2024-09-16T09:31:35.728656+0000","last_became_active":"2024-09-16T09:31:09.272208+0000","last_became_peered":"2024-09-16T09:31:09.272208+0000","last_unstale":"2024-09-16T09:31:35.728656+0000","last_undegraded":"2024-09-16T09:31:35.728656+0000","last_fullsized":"2024-09-16T09:31:35.728656+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:30:21.651575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:30:21.651575+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.14","version":"66'10","reported_seq":175,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.719171+0000","last_change":"2024-09-16T09:28:56.825675+0000","last_active":"2024-09-16T09:31:35.719171+0000","last_peered":"2024-09-16T09:31:35.719171+0000","last_clean":"2024-09-16T09:31:35.719171+0000","last_became_active":"2024-09-16T09:28:52.979059+0000","last_became_peered":"2024-09-16T09:28:52.979059+0000","last_unstale":"2024-09-16T09:31:35.719171+0000","last_undegraded":"2024-09-16T09:31:35.719171+0000","last_fullsized":"2024-09-16T09:31:35.719171+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-16T09:28:56.825476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:56.825476+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":14,"num_read_kb":9,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":303,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.728520+0000","last_change":"2024-09-16T09:29:56.021413+0000","last_active":"2024-09-16T09:31:35.728520+0000","last_peered":"2024-09-16T09:31:35.728520+0000","last_clean":"2024-09-16T09:31:35.728520+0000","last_became_active":"2024-09-16T09:25:01.707513+0000","last_became_peered":"2024-09-16T09:25:01.707513+0000","last_unstale":"2024-09-16T09:31:35.728520+0000","last_undegraded":"2024-09-16T09:31:35.728520+0000","last_fullsized":"2024-09-16T09:31:35.728520+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:56.021340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:56.021340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"66'8","reported_seq":391,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.728071+0000","last_change":"2024-09-16T09:29:48.485781+0000","last_active":"2024-09-16T09:31:35.728071+0000","last_peered":"2024-09-16T09:31:35.728071+0000","last_clean":"2024-09-16T09:31:35.728071+0000","last_became_active":"2024-09-16T09:25:05.713833+0000","last_became_peered":"2024-09-16T09:25:05.713833+0000","last_unstale":"2024-09-16T09:31:35.728071+0000","last_undegraded":"2024-09-16T09:31:35.728071+0000","last_fullsized":"2024-09-16T09:31:35.728071+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:29:48.485727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.485727+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":306,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.728867+0000","last_change":"2024-09-16T09:31:09.276853+0000","last_active":"2024-09-16T09:31:35.728867+0000","last_peered":"2024-09-16T09:31:35.728867+0000","last_clean":"2024-09-16T09:31:35.728867+0000","last_became_active":"2024-09-16T09:31:09.275168+0000","last_became_peered":"2024-09-16T09:31:09.275168+0000","last_unstale":"2024-09-16T09:31:35.728867+0000","last_undegraded":"2024-09-16T09:31:35.728867+0000","last_fullsized":"2024-09-16T09:31:35.728867+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:52.958303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:52.958303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"66'11","reported_seq":322,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.717990+0000","last_change":"2024-09-16T09:26:33.302713+0000","last_active":"2024-09-16T09:31:35.717990+0000","last_peered":"2024-09-16T09:31:35.717990+0000","last_clean":"2024-09-16T09:31:35.717990+0000","last_became_active":"2024-09-16T09:26:33.291265+0000","last_became_peered":"2024-09-16T09:26:33.291265+0000","last_unstale":"2024-09-16T09:31:35.717990+0000","last_undegraded":"2024-09-16T09:31:35.717990+0000","last_fullsized":"2024-09-16T09:31:35.717990+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":312,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.718106+0000","last_change":"2024-09-16T09:29:51.517112+0000","last_active":"2024-09-16T09:31:35.718106+0000","last_peered":"2024-09-16T09:31:35.718106+0000","last_clean":"2024-09-16T09:31:35.718106+0000","last_became_active":"2024-09-16T09:25:01.705158+0000","last_became_peered":"2024-09-16T09:25:01.705158+0000","last_unstale":"2024-09-16T09:31:35.718106+0000","last_undegraded":"2024-09-16T09:31:35.718106+0000","last_fullsized":"2024-09-16T09:31:35.718106+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:51.517054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:51.517054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":318,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.724015+0000","last_change":"2024-09-16T09:31:09.266820+0000","last_active":"2024-09-16T09:31:35.724015+0000","last_peered":"2024-09-16T09:31:35.724015+0000","last_clean":"2024-09-16T09:31:35.724015+0000","last_became_active":"2024-09-16T09:31:09.266326+0000","last_became_peered":"2024-09-16T09:31:09.266326+0000","last_unstale":"2024-09-16T09:31:35.724015+0000","last_undegraded":"2024-09-16T09:31:35.724015+0000","last_fullsized":"2024-09-16T09:31:35.724015+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:15.910824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:15.910824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":147,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.728356+0000","last_change":"2024-09-16T09:31:09.277239+0000","last_active":"2024-09-16T09:31:35.728356+0000","last_peered":"2024-09-16T09:31:35.728356+0000","last_clean":"2024-09-16T09:31:35.728356+0000","last_became_active":"2024-09-16T09:31:09.276042+0000","last_became_peered":"2024-09-16T09:31:09.276042+0000","last_unstale":"2024-09-16T09:31:35.728356+0000","last_undegraded":"2024-09-16T09:31:35.728356+0000","last_fullsized":"2024-09-16T09:31:35.728356+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:48.147759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:48.147759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:14.230973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"66'9","reported_seq":308,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.724115+0000","last_change":"2024-09-16T09:26:31.216495+0000","last_active":"2024-09-16T09:31:35.724115+0000","last_peered":"2024-09-16T09:31:35.724115+0000","last_clean":"2024-09-16T09:31:35.724115+0000","last_became_active":"2024-09-16T09:25:03.705807+0000","last_became_peered":"2024-09-16T09:25:03.705807+0000","last_unstale":"2024-09-16T09:31:35.724115+0000","last_undegraded":"2024-09-16T09:31:35.724115+0000","last_fullsized":"2024-09-16T09:31:35.724115+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:26:31.216298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:26:31.216298+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":295,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.730782+0000","last_change":"2024-09-16T09:26:23.796308+0000","last_active":"2024-09-16T09:31:35.730782+0000","last_peered":"2024-09-16T09:31:35.730782+0000","last_clean":"2024-09-16T09:31:35.730782+0000","last_became_active":"2024-09-16T09:25:01.706762+0000","last_became_peered":"2024-09-16T09:25:01.706762+0000","last_unstale":"2024-09-16T09:31:35.730782+0000","last_undegraded":"2024-09-16T09:31:35.730782+0000","last_fullsized":"2024-09-16T09:31:35.730782+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:23.796231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:23.796231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":132,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.728078+0000","last_change":"2024-09-16T09:28:52.983144+0000","last_active":"2024-09-16T09:31:35.728078+0000","last_peered":"2024-09-16T09:31:35.728078+0000","last_clean":"2024-09-16T09:31:35.728078+0000","last_became_active":"2024-09-16T09:28:52.982865+0000","last_became_peered":"2024-09-16T09:28:52.982865+0000","last_unstale":"2024-09-16T09:31:35.728078+0000","last_undegraded":"2024-09-16T09:31:35.728078+0000","last_fullsized":"2024-09-16T09:31:35.728078+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:44.241942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:44.241942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:09.215097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":292,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.726722+0000","last_change":"2024-09-16T09:28:52.552567+0000","last_active":"2024-09-16T09:31:35.726722+0000","last_peered":"2024-09-16T09:31:35.726722+0000","last_clean":"2024-09-16T09:31:35.726722+0000","last_became_active":"2024-09-16T09:25:07.715301+0000","last_became_peered":"2024-09-16T09:25:07.715301+0000","last_unstale":"2024-09-16T09:31:35.726722+0000","last_undegraded":"2024-09-16T09:31:35.726722+0000","last_fullsized":"2024-09-16T09:31:35.726722+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:52.552495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:28:52.552495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.14","version":"73'79","reported_seq":87,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.726837+0000","last_change":"2024-09-16T09:31:14.201538+0000","last_active":"2024-09-16T09:31:35.726837+0000","last_peered":"2024-09-16T09:31:35.726837+0000","last_clean":"2024-09-16T09:31:35.726837+0000","last_became_active":"2024-09-16T09:30:10.700866+0000","last_became_peered":"2024-09-16T09:30:10.700866+0000","last_unstale":"2024-09-16T09:31:35.726837+0000","last_undegraded":"2024-09-16T09:31:35.726837+0000","last_fullsized":"2024-09-16T09:31:35.726837+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:53.916505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:53.916505+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:31:42.929883+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109227,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"66'11","reported_seq":179,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.728089+0000","last_change":"2024-09-16T09:28:55.643505+0000","last_active":"2024-09-16T09:31:35.728089+0000","last_peered":"2024-09-16T09:31:35.728089+0000","last_clean":"2024-09-16T09:31:35.728089+0000","last_became_active":"2024-09-16T09:28:52.971132+0000","last_became_peered":"2024-09-16T09:28:52.971132+0000","last_unstale":"2024-09-16T09:31:35.728089+0000","last_undegraded":"2024-09-16T09:31:35.728089+0000","last_fullsized":"2024-09-16T09:31:35.728089+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-16T09:28:55.643448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:55.643448+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":19,"num_read_kb":12,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":303,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.722326+0000","last_change":"2024-09-16T09:27:46.175159+0000","last_active":"2024-09-16T09:31:35.722326+0000","last_peered":"2024-09-16T09:31:35.722326+0000","last_clean":"2024-09-16T09:31:35.722326+0000","last_became_active":"2024-09-16T09:25:01.700733+0000","last_became_peered":"2024-09-16T09:25:01.700733+0000","last_unstale":"2024-09-16T09:31:35.722326+0000","last_undegraded":"2024-09-16T09:31:35.722326+0000","last_fullsized":"2024-09-16T09:31:35.722326+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:46.175093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:27:46.175093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":295,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.727185+0000","last_change":"2024-09-16T09:29:25.017736+0000","last_active":"2024-09-16T09:31:35.727185+0000","last_peered":"2024-09-16T09:31:35.727185+0000","last_clean":"2024-09-16T09:31:35.727185+0000","last_became_active":"2024-09-16T09:25:05.713479+0000","last_became_peered":"2024-09-16T09:25:05.713479+0000","last_unstale":"2024-09-16T09:31:35.727185+0000","last_undegraded":"2024-09-16T09:31:35.727185+0000","last_fullsized":"2024-09-16T09:31:35.727185+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:25.017610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:25.017610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":155,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.728521+0000","last_change":"2024-09-16T09:31:09.275121+0000","last_active":"2024-09-16T09:31:35.728521+0000","last_peered":"2024-09-16T09:31:35.728521+0000","last_clean":"2024-09-16T09:31:35.728521+0000","last_became_active":"2024-09-16T09:31:09.274797+0000","last_became_peered":"2024-09-16T09:31:09.274797+0000","last_unstale":"2024-09-16T09:31:35.728521+0000","last_undegraded":"2024-09-16T09:31:35.728521+0000","last_fullsized":"2024-09-16T09:31:35.728521+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:54.589258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:28:54.589258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"66'4","reported_seq":36,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.726488+0000","last_change":"2024-09-16T09:31:09.401626+0000","last_active":"2024-09-16T09:31:35.726488+0000","last_peered":"2024-09-16T09:31:35.726488+0000","last_clean":"2024-09-16T09:31:35.726488+0000","last_became_active":"2024-09-16T09:31:09.273682+0000","last_became_peered":"2024-09-16T09:31:09.273682+0000","last_unstale":"2024-09-16T09:31:35.726488+0000","last_undegraded":"2024-09-16T09:31:35.726488+0000","last_fullsized":"2024-09-16T09:31:35.726488+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'4","last_scrub_stamp":"2024-09-16T09:26:27.473740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:26:27.473740+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":326,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.728341+0000","last_change":"2024-09-16T09:31:30.235761+0000","last_active":"2024-09-16T09:31:35.728341+0000","last_peered":"2024-09-16T09:31:35.728341+0000","last_clean":"2024-09-16T09:31:35.728341+0000","last_became_active":"2024-09-16T09:31:09.274719+0000","last_became_peered":"2024-09-16T09:31:09.274719+0000","last_unstale":"2024-09-16T09:31:35.728341+0000","last_undegraded":"2024-09-16T09:31:35.728341+0000","last_fullsized":"2024-09-16T09:31:35.728341+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:31:30.235692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:31:30.235692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:49.736157+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":304,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.723054+0000","last_change":"2024-09-16T09:29:53.244633+0000","last_active":"2024-09-16T09:31:35.723054+0000","last_peered":"2024-09-16T09:31:35.723054+0000","last_clean":"2024-09-16T09:31:35.723054+0000","last_became_active":"2024-09-16T09:25:05.712545+0000","last_became_peered":"2024-09-16T09:25:05.712545+0000","last_unstale":"2024-09-16T09:31:35.723054+0000","last_undegraded":"2024-09-16T09:31:35.723054+0000","last_fullsized":"2024-09-16T09:31:35.723054+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:53.244578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:53.244578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:07.420423+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":291,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.728068+0000","last_change":"2024-09-16T09:29:27.012810+0000","last_active":"2024-09-16T09:31:35.728068+0000","last_peered":"2024-09-16T09:31:35.728068+0000","last_clean":"2024-09-16T09:31:35.728068+0000","last_became_active":"2024-09-16T09:25:07.720117+0000","last_became_peered":"2024-09-16T09:25:07.720117+0000","last_unstale":"2024-09-16T09:31:35.728068+0000","last_undegraded":"2024-09-16T09:31:35.728068+0000","last_fullsized":"2024-09-16T09:31:35.728068+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:27.012756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:27.012756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":292,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.723668+0000","last_change":"2024-09-16T09:29:19.888825+0000","last_active":"2024-09-16T09:31:35.723668+0000","last_peered":"2024-09-16T09:31:35.723668+0000","last_clean":"2024-09-16T09:31:35.723668+0000","last_became_active":"2024-09-16T09:25:07.712594+0000","last_became_peered":"2024-09-16T09:25:07.712594+0000","last_unstale":"2024-09-16T09:31:35.723668+0000","last_undegraded":"2024-09-16T09:31:35.723668+0000","last_fullsized":"2024-09-16T09:31:35.723668+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:19.888759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:19.888759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":154,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.727327+0000","last_change":"2024-09-16T09:31:26.211625+0000","last_active":"2024-09-16T09:31:35.727327+0000","last_peered":"2024-09-16T09:31:35.727327+0000","last_clean":"2024-09-16T09:31:35.727327+0000","last_became_active":"2024-09-16T09:31:09.271662+0000","last_became_peered":"2024-09-16T09:31:09.271662+0000","last_unstale":"2024-09-16T09:31:35.727327+0000","last_undegraded":"2024-09-16T09:31:35.727327+0000","last_fullsized":"2024-09-16T09:31:35.727327+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:31:26.211363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:31:26.211363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:32.873753+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"66'11","reported_seq":320,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.724634+0000","last_change":"2024-09-16T09:27:57.486459+0000","last_active":"2024-09-16T09:31:35.724634+0000","last_peered":"2024-09-16T09:31:35.724634+0000","last_clean":"2024-09-16T09:31:35.724634+0000","last_became_active":"2024-09-16T09:25:03.707769+0000","last_became_peered":"2024-09-16T09:25:03.707769+0000","last_unstale":"2024-09-16T09:31:35.724634+0000","last_undegraded":"2024-09-16T09:31:35.724634+0000","last_fullsized":"2024-09-16T09:31:35.724634+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-16T09:27:57.486381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:57.486381+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:16.375833+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":289,"reported_epoch":196,"state":"active+clean","last_fresh":"2024-09-16T09:31:35.728964+0000","last_change":"2024-09-16T09:27:57.640962+0000","last_active":"2024-09-16T09:31:35.728964+0000","last_peered":"2024-09-16T09:31:35.728964+0000","last_clean":"2024-09-16T09:31:35.728964+0000","last_became_active":"2024-09-16T09:25:05.711952+0000","last_became_peered":"2024-09-16T09:25:05.711952+0000","last_unstale":"2024-09-16T09:31:35.728964+0000","last_undegraded":"2024-09-16T09:31:35.728964+0000","last_fullsized":"2024-09-16T09:31:35.728964+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:57.640756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:57.640756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:21.189003+0000","scrub_duration":44,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":16,"stat_sum":{"num_bytes":7326728216,"num_objects":111798,"num_object_clones":0,"num_object_copies":335394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":7625,"num_objects_unfound":0,"num_objects_dirty":111798,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":137388,"num_write_kb":7973889,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7084,"num_bytes_recovered":464257024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":22448123904,"data_stored":22448111688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":77275,"ondisk_log_size":77275,"up":48,"acting":45,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":70,"ondisk_log_size":70,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":691,"num_read_kb":448,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":126,"num_bytes_recovered":3026,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":395,"ondisk_log_size":395,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1986,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":390,"num_read_kb":385,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":21,"stat_sum":{"num_bytes":806935,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":21,"num_read_kb":40,"num_write":21,"num_write_kb":296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":697708,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1659,"ondisk_log_size":1659,"up":63,"acting":63,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266772,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2417748,"kb_used_data":2260224,"kb_used_omap":1,"kb_used_meta":157502,"kb_avail":91327404,"statfs":{"total":95995035648,"available":93519261696,"internally_reserved":0,"allocated":2314469376,"data_stored":2314159378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":161282506},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495705,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2413820,"kb_used_data":2252136,"kb_used_omap":1,"kb_used_meta":161662,"kb_avail":91331332,"statfs":{"total":95995035648,"available":93523283968,"internally_reserved":0,"allocated":2306187264,"data_stored":2305901264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":165542342},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724640,"num_pgs":64,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3848388,"kb_used_data":3588148,"kb_used_omap":1,"kb_used_meta":260222,"kb_avail":89896764,"statfs":{"total":95995035648,"available":92054286336,"internally_reserved":0,"allocated":3674263552,"data_stored":3673965757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":266467782},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953573,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2739152,"kb_used_data":2537216,"kb_used_omap":1,"kb_used_meta":201918,"kb_avail":91006000,"statfs":{"total":95995035648,"available":93190144000,"internally_reserved":0,"allocated":2598109184,"data_stored":2597799226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":206764486},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149803,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2493916,"kb_used_data":2205896,"kb_used_omap":1,"kb_used_meta":287998,"kb_avail":91251236,"statfs":{"total":95995035648,"available":93441265664,"internally_reserved":0,"allocated":2258837504,"data_stored":2258519640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":294910407},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411440,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3326848,"kb_used_data":3061036,"kb_used_omap":1,"kb_used_meta":265790,"kb_avail":90418304,"statfs":{"total":95995035648,"available":92588343296,"internally_reserved":0,"allocated":3134500864,"data_stored":3134210724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":272169413},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574967,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3407532,"kb_used_data":3141720,"kb_used_omap":1,"kb_used_meta":265790,"kb_avail":90337620,"statfs":{"total":95995035648,"available":92505722880,"internally_reserved":0,"allocated":3217121280,"data_stored":3216786616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":272169413},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738493,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3252564,"kb_used_data":2968704,"kb_used_omap":1,"kb_used_meta":283838,"kb_avail":90492588,"statfs":{"total":95995035648,"available":92664410112,"internally_reserved":0,"allocated":3039952896,"data_stored":3039643140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":290650565},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":3017736192,"data_stored":3017736192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3203399680,"data_stored":3203399680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3113095168,"data_stored":3113091096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2249195520,"data_stored":2249195520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":2597060608,"data_stored":2597060608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":3658027008,"data_stored":3658022936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":2300968960,"data_stored":2300968960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":2308640768,"data_stored":2308636696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T09:31:42.031 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-16T09:31:42.055 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:31:42.119 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:42.120 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:42.120 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:31:42.221 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:31:42.224 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:31:41 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:31:41] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:31:42.250 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:42.251 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:42.251 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:31:42.352 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:31:42.385 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:42 smithi084 ceph-mon[32629]: from='client.25289 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:31:42.386 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:42 smithi084 ceph-mon[32629]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:31:42.386 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:42 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-16T09:31:42.386 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:42 smithi084 ceph-mon[32629]: osdmap e197: 8 total, 8 up, 8 in 2024-09-16T09:31:42.386 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:42 smithi084 ceph-mon[32629]: 1.5 scrub starts 2024-09-16T09:31:42.395 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:42.395 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:42.396 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:31:42.497 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:31:42.534 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:42.535 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:42.535 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:31:42.636 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:31:42.669 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-09-16T09:31:42.669 INFO:teuthology.orchestra.run.smithi084.stdout: "size": 20, 2024-09-16T09:31:42.669 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 600, 2024-09-16T09:31:42.670 INFO:teuthology.orchestra.run.smithi084.stdout: "ops": [ 2024-09-16T09:31:42.670 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.670 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:22927 7.1 7:8fa982a9:::benchmark_data_smithi070_78438_object22926:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T09:31:42.670 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:26:21.582665+0000", 2024-09-16T09:31:42.670 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 321.08198896499999, 2024-09-16T09:31:42.670 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.077694033999999995, 2024-09-16T09:31:42.670 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:31:42.670 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:31:42.670 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:31:42.670 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:31:42.670 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:31:42.670 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:31:42.670 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 22927 2024-09-16T09:31:42.671 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.671 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:31:42.671 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.671 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:31:42.671 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.582665+0000", 2024-09-16T09:31:42.671 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:31:42.671 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.671 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.671 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:31:42.671 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.582665+0000", 2024-09-16T09:31:42.671 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:31:42.671 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.671 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.672 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:31:42.672 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.582667+0000", 2024-09-16T09:31:42.672 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.5179999999999999e-06 2024-09-16T09:31:42.672 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.672 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.672 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:31:42.672 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.582700+0000", 2024-09-16T09:31:42.672 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.2821000000000003e-05 2024-09-16T09:31:42.672 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.672 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.673 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:31:42.673 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.582719+0000", 2024-09-16T09:31:42.673 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.9255000000000001e-05 2024-09-16T09:31:42.673 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.673 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.673 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:31:42.673 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.582727+0000", 2024-09-16T09:31:42.673 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.661e-06 2024-09-16T09:31:42.673 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.673 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.673 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:31:42.673 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.582801+0000", 2024-09-16T09:31:42.673 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.4553000000000003e-05 2024-09-16T09:31:42.674 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.674 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.674 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:31:42.674 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.582991+0000", 2024-09-16T09:31:42.674 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00018939400000000001 2024-09-16T09:31:42.674 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.674 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.674 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 3,6", 2024-09-16T09:31:42.674 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.583125+0000", 2024-09-16T09:31:42.674 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00013424800000000001 2024-09-16T09:31:42.674 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.674 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.674 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:31:42.674 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.585851+0000", 2024-09-16T09:31:42.675 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0027256799999999999 2024-09-16T09:31:42.675 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.675 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.675 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:31:42.675 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.585910+0000", 2024-09-16T09:31:42.675 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.9552999999999997e-05 2024-09-16T09:31:42.675 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.675 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.675 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:31:42.675 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.660257+0000", 2024-09-16T09:31:42.675 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.074347094000000002 2024-09-16T09:31:42.675 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.676 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.676 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:31:42.676 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.660316+0000", 2024-09-16T09:31:42.676 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.8595999999999999e-05 2024-09-16T09:31:42.676 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.676 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.676 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:31:42.676 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.660360+0000", 2024-09-16T09:31:42.676 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.3661000000000001e-05 2024-09-16T09:31:42.676 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:31:42.676 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:31:42.677 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:31:42.677 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.677 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.677 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:22934 7.1 7:87f3da4d:::benchmark_data_smithi070_78438_object22933:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T09:31:42.677 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:26:21.583328+0000", 2024-09-16T09:31:42.677 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 321.08132614800002, 2024-09-16T09:31:42.677 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.077158353999999998, 2024-09-16T09:31:42.677 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:31:42.677 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:31:42.678 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:31:42.678 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:31:42.678 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:31:42.678 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:31:42.678 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 22934 2024-09-16T09:31:42.678 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.678 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:31:42.678 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.678 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:31:42.678 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.583328+0000", 2024-09-16T09:31:42.678 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:31:42.678 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.678 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.678 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:31:42.679 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.583328+0000", 2024-09-16T09:31:42.679 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:31:42.679 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.679 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.679 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:31:42.679 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.583329+0000", 2024-09-16T09:31:42.679 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0070000000000001e-06 2024-09-16T09:31:42.679 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.679 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.679 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:31:42.679 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.583429+0000", 2024-09-16T09:31:42.679 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.9981000000000004e-05 2024-09-16T09:31:42.679 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.680 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.680 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:31:42.680 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.583441+0000", 2024-09-16T09:31:42.680 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.2196e-05 2024-09-16T09:31:42.680 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.680 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.680 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:31:42.680 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.583448+0000", 2024-09-16T09:31:42.680 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.8460000000000002e-06 2024-09-16T09:31:42.680 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.680 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.680 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:31:42.680 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.584242+0000", 2024-09-16T09:31:42.681 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00079362599999999997 2024-09-16T09:31:42.681 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.681 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.681 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:31:42.681 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.584361+0000", 2024-09-16T09:31:42.681 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000119014 2024-09-16T09:31:42.681 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.681 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.681 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 3,6", 2024-09-16T09:31:42.681 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.584493+0000", 2024-09-16T09:31:42.681 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000132473 2024-09-16T09:31:42.681 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.681 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.682 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:31:42.682 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.585942+0000", 2024-09-16T09:31:42.682 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.001448509 2024-09-16T09:31:42.682 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.682 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.682 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:31:42.682 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.585970+0000", 2024-09-16T09:31:42.682 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.7932e-05 2024-09-16T09:31:42.682 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.682 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.682 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:31:42.682 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.660424+0000", 2024-09-16T09:31:42.683 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.074454300000000001 2024-09-16T09:31:42.683 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.683 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.683 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:31:42.683 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.660462+0000", 2024-09-16T09:31:42.683 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.8302e-05 2024-09-16T09:31:42.683 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.683 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.683 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:31:42.683 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.660487+0000", 2024-09-16T09:31:42.683 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.4168e-05 2024-09-16T09:31:42.683 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:31:42.683 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:31:42.684 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:31:42.684 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.684 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.684 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:86890 7.1 7:8b0c621b:::benchmark_data_smithi070_78438_object86889:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-16T09:31:42.684 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:27:06.136694+0000", 2024-09-16T09:31:42.684 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 276.52796065899997, 2024-09-16T09:31:42.684 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.086345176999999995, 2024-09-16T09:31:42.684 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:31:42.684 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:31:42.684 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:31:42.684 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:31:42.684 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:31:42.685 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:31:42.685 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 86890 2024-09-16T09:31:42.685 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.685 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:31:42.685 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.685 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:31:42.685 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.136694+0000", 2024-09-16T09:31:42.685 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:31:42.685 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.685 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.685 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:31:42.685 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.136694+0000", 2024-09-16T09:31:42.686 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:31:42.686 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.686 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.686 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:31:42.686 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.136695+0000", 2024-09-16T09:31:42.686 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.4160000000000001e-06 2024-09-16T09:31:42.686 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.686 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.686 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:31:42.686 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.136800+0000", 2024-09-16T09:31:42.686 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000104957 2024-09-16T09:31:42.686 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.686 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.687 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:31:42.687 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.136813+0000", 2024-09-16T09:31:42.687 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.3023000000000001e-05 2024-09-16T09:31:42.687 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.687 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.687 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:31:42.687 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.136820+0000", 2024-09-16T09:31:42.687 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.2180000000000002e-06 2024-09-16T09:31:42.687 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.688 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.688 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:31:42.688 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.136891+0000", 2024-09-16T09:31:42.688 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.0697000000000004e-05 2024-09-16T09:31:42.688 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.688 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.688 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:31:42.688 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.137034+0000", 2024-09-16T09:31:42.688 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000142421 2024-09-16T09:31:42.688 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.688 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.688 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 3,6", 2024-09-16T09:31:42.689 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.137177+0000", 2024-09-16T09:31:42.689 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00014350699999999999 2024-09-16T09:31:42.689 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.689 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.689 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:31:42.689 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.144870+0000", 2024-09-16T09:31:42.689 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0076926549999999996 2024-09-16T09:31:42.689 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.689 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.689 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:31:42.689 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.144970+0000", 2024-09-16T09:31:42.689 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000100511 2024-09-16T09:31:42.689 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.690 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.690 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:31:42.690 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.222926+0000", 2024-09-16T09:31:42.690 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.077955615000000006 2024-09-16T09:31:42.690 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.690 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.690 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:31:42.690 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.222988+0000", 2024-09-16T09:31:42.690 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.1916999999999997e-05 2024-09-16T09:31:42.690 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.690 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.690 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:31:42.690 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.223039+0000", 2024-09-16T09:31:42.690 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.1239999999999997e-05 2024-09-16T09:31:42.690 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:31:42.691 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:31:42.691 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:31:42.691 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.691 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.691 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:86891 7.1 7:8533eaa2:::benchmark_data_smithi070_78438_object86890:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-16T09:31:42.691 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:27:06.137963+0000", 2024-09-16T09:31:42.691 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 276.52669153300002, 2024-09-16T09:31:42.691 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.089654372999999996, 2024-09-16T09:31:42.691 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:31:42.691 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:31:42.691 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:31:42.691 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:31:42.691 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:31:42.691 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:31:42.692 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 86891 2024-09-16T09:31:42.692 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.692 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:31:42.692 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.692 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:31:42.692 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.137963+0000", 2024-09-16T09:31:42.692 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:31:42.692 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.692 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.692 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:31:42.692 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.137963+0000", 2024-09-16T09:31:42.692 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:31:42.692 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.692 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.693 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:31:42.693 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.137964+0000", 2024-09-16T09:31:42.693 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.5629999999999999e-06 2024-09-16T09:31:42.693 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.693 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.693 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:31:42.693 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.137999+0000", 2024-09-16T09:31:42.693 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.4564000000000003e-05 2024-09-16T09:31:42.693 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.693 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.693 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:31:42.693 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.138016+0000", 2024-09-16T09:31:42.693 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.6677999999999999e-05 2024-09-16T09:31:42.693 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.693 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.694 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:31:42.694 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.138023+0000", 2024-09-16T09:31:42.694 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.5059999999999998e-06 2024-09-16T09:31:42.694 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.694 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.694 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:31:42.694 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.139570+0000", 2024-09-16T09:31:42.694 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0015467809999999999 2024-09-16T09:31:42.694 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.694 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.694 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:31:42.694 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.139690+0000", 2024-09-16T09:31:42.694 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000120387 2024-09-16T09:31:42.694 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.694 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.695 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 3,6", 2024-09-16T09:31:42.695 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.139833+0000", 2024-09-16T09:31:42.695 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00014291799999999999 2024-09-16T09:31:42.695 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.695 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.695 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:31:42.695 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.144931+0000", 2024-09-16T09:31:42.695 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0050973110000000002 2024-09-16T09:31:42.695 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.695 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.695 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:31:42.695 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.144973+0000", 2024-09-16T09:31:42.695 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.2038e-05 2024-09-16T09:31:42.695 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.695 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.696 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:31:42.696 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.227507+0000", 2024-09-16T09:31:42.696 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.082533910000000002 2024-09-16T09:31:42.696 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.696 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.696 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:31:42.696 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.227568+0000", 2024-09-16T09:31:42.696 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.1229999999999995e-05 2024-09-16T09:31:42.696 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.696 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.697 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:31:42.697 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.227617+0000", 2024-09-16T09:31:42.697 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.9487000000000002e-05 2024-09-16T09:31:42.697 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:31:42.697 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:31:42.697 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:31:42.697 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.697 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.697 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:86895 7.1 7:852b1173:::benchmark_data_smithi070_78438_object86894:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-16T09:31:42.697 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:27:06.138894+0000", 2024-09-16T09:31:42.697 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 276.52576032100001, 2024-09-16T09:31:42.697 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.091395905, 2024-09-16T09:31:42.698 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:31:42.698 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:31:42.698 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:31:42.698 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:31:42.698 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:31:42.698 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:31:42.698 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 86895 2024-09-16T09:31:42.698 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.698 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:31:42.698 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.698 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:31:42.698 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.138894+0000", 2024-09-16T09:31:42.698 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:31:42.698 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.699 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.699 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:31:42.699 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.138894+0000", 2024-09-16T09:31:42.699 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:31:42.699 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.700 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:42 smithi070 ceph-mon[30388]: from='client.25289 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:31:42.700 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:42 smithi070 ceph-mon[30388]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:31:42.700 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:42 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-16T09:31:42.700 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:42 smithi070 ceph-mon[30388]: osdmap e197: 8 total, 8 up, 8 in 2024-09-16T09:31:42.700 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:42 smithi070 ceph-mon[30388]: 1.5 scrub starts 2024-09-16T09:31:42.700 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.700 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:31:42.700 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.138895+0000", 2024-09-16T09:31:42.700 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.093e-06 2024-09-16T09:31:42.700 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.701 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.701 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:31:42.701 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.138932+0000", 2024-09-16T09:31:42.701 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.6934999999999998e-05 2024-09-16T09:31:42.701 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.701 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.701 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:31:42.701 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.138946+0000", 2024-09-16T09:31:42.701 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.3772e-05 2024-09-16T09:31:42.701 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.701 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.701 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:31:42.702 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.138953+0000", 2024-09-16T09:31:42.702 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.2810000000000003e-06 2024-09-16T09:31:42.702 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.702 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.702 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:31:42.702 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.142367+0000", 2024-09-16T09:31:42.702 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0034135559999999999 2024-09-16T09:31:42.702 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.702 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.702 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:31:42.702 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.142499+0000", 2024-09-16T09:31:42.703 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00013187 2024-09-16T09:31:42.703 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.703 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.703 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 3,6", 2024-09-16T09:31:42.703 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.142646+0000", 2024-09-16T09:31:42.703 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00014712500000000001 2024-09-16T09:31:42.703 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.703 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.704 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:31:42.704 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.144963+0000", 2024-09-16T09:31:42.704 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0023176260000000001 2024-09-16T09:31:42.704 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.704 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.704 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:31:42.704 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.144977+0000", 2024-09-16T09:31:42.704 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.3509000000000001e-05 2024-09-16T09:31:42.704 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.704 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.704 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:31:42.704 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.230194+0000", 2024-09-16T09:31:42.704 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.085216682000000002 2024-09-16T09:31:42.705 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.705 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.705 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:31:42.705 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.230252+0000", 2024-09-16T09:31:42.705 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.8023000000000003e-05 2024-09-16T09:31:42.705 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.705 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.705 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:31:42.705 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:27:06.230290+0000", 2024-09-16T09:31:42.705 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.8433000000000003e-05 2024-09-16T09:31:42.705 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:31:42.705 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:31:42.706 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:31:42.706 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.706 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.706 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_repop(client.24698.0:202635 7.4 e135/82 7:2055e12a:::benchmark_data_smithi070_78438_object76504:head v 135'12468, mlcod=135'12465)", 2024-09-16T09:31:42.706 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:28:51.040989+0000", 2024-09-16T09:31:42.706 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 171.62366592999999, 2024-09-16T09:31:42.706 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.105010197, 2024-09-16T09:31:42.706 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:31:42.706 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:31:42.706 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:31:42.706 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:31:42.706 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.706 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:31:42.707 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.040989+0000", 2024-09-16T09:31:42.707 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:31:42.707 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.707 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.707 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:31:42.707 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.040989+0000", 2024-09-16T09:31:42.707 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:31:42.707 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.707 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.707 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:31:42.707 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.040989+0000", 2024-09-16T09:31:42.707 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.1799999999999995e-07 2024-09-16T09:31:42.708 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.708 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.708 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:31:42.708 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.040993+0000", 2024-09-16T09:31:42.708 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.5269999999999999e-06 2024-09-16T09:31:42.708 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.708 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.708 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:31:42.708 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.041000+0000", 2024-09-16T09:31:42.708 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.3509999999999996e-06 2024-09-16T09:31:42.708 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.708 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.709 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:31:42.709 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.041004+0000", 2024-09-16T09:31:42.709 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.532e-06 2024-09-16T09:31:42.709 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.709 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.709 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:31:42.709 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.041044+0000", 2024-09-16T09:31:42.709 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.9258999999999998e-05 2024-09-16T09:31:42.709 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.709 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.709 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:31:42.709 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.041056+0000", 2024-09-16T09:31:42.709 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.2258e-05 2024-09-16T09:31:42.710 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.710 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.710 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:31:42.710 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.145950+0000", 2024-09-16T09:31:42.710 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.104893666 2024-09-16T09:31:42.710 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.710 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.710 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:31:42.710 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.145999+0000", 2024-09-16T09:31:42.710 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.9085999999999997e-05 2024-09-16T09:31:42.710 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:31:42.710 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:31:42.710 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:31:42.711 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.711 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.711 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:202638 7.1 7:8f557cfd:::benchmark_data_smithi070_78438_object76507:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135)", 2024-09-16T09:31:42.711 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:28:51.042963+0000", 2024-09-16T09:31:42.711 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 171.62169167100001, 2024-09-16T09:31:42.711 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.111245755, 2024-09-16T09:31:42.711 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:31:42.711 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:31:42.711 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:31:42.711 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:31:42.711 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:31:42.711 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:31:42.711 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 202638 2024-09-16T09:31:42.711 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.711 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:31:42.711 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.712 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:31:42.712 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.042963+0000", 2024-09-16T09:31:42.712 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:31:42.712 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.712 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.712 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:31:42.712 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.042963+0000", 2024-09-16T09:31:42.712 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:31:42.712 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.712 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.712 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:31:42.712 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.042964+0000", 2024-09-16T09:31:42.712 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.5910000000000001e-06 2024-09-16T09:31:42.712 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.713 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.713 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:31:42.713 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.042970+0000", 2024-09-16T09:31:42.713 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.7960000000000001e-06 2024-09-16T09:31:42.713 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.713 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.713 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:31:42.713 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.137960+0000", 2024-09-16T09:31:42.713 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.094989799 2024-09-16T09:31:42.713 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.713 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.713 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:31:42.713 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.137972+0000", 2024-09-16T09:31:42.713 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1556000000000001e-05 2024-09-16T09:31:42.714 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.714 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.714 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:31:42.714 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.144247+0000", 2024-09-16T09:31:42.714 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0062759540000000003 2024-09-16T09:31:42.714 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.714 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.714 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:31:42.714 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.144377+0000", 2024-09-16T09:31:42.714 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00012968800000000001 2024-09-16T09:31:42.714 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.714 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.714 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 3,6", 2024-09-16T09:31:42.715 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.144499+0000", 2024-09-16T09:31:42.715 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00012222000000000001 2024-09-16T09:31:42.715 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.715 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.715 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:31:42.715 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.151527+0000", 2024-09-16T09:31:42.715 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0070274819999999998 2024-09-16T09:31:42.715 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.715 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.715 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:31:42.715 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:42 smithi070 ceph-mon[40691]: from='client.25289 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:31:42.715 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:42 smithi070 ceph-mon[40691]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:31:42.716 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:42 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-16T09:31:42.716 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:42 smithi070 ceph-mon[40691]: osdmap e197: 8 total, 8 up, 8 in 2024-09-16T09:31:42.716 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:42 smithi070 ceph-mon[40691]: 1.5 scrub starts 2024-09-16T09:31:42.716 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:42.717 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:42.717 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:31:42.718 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.151636+0000", 2024-09-16T09:31:42.718 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000109224 2024-09-16T09:31:42.718 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.718 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.718 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:31:42.718 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.154123+0000", 2024-09-16T09:31:42.718 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0024866279999999998 2024-09-16T09:31:42.718 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.718 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.718 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:31:42.719 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.154169+0000", 2024-09-16T09:31:42.719 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.5902000000000002e-05 2024-09-16T09:31:42.719 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.719 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.719 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:31:42.719 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.154209+0000", 2024-09-16T09:31:42.719 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.9915e-05 2024-09-16T09:31:42.719 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:31:42.719 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:31:42.719 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:31:42.719 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.720 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.720 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_repop(client.24698.0:202642 7.4 e135/82 7:2c34e77c:::benchmark_data_smithi070_78438_object76511:head v 135'12469, mlcod=135'12466)", 2024-09-16T09:31:42.720 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:28:51.048625+0000", 2024-09-16T09:31:42.720 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 171.61602948699999, 2024-09-16T09:31:42.720 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.104253999, 2024-09-16T09:31:42.720 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:31:42.720 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:31:42.720 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:31:42.720 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:31:42.720 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.720 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:31:42.720 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.048625+0000", 2024-09-16T09:31:42.720 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:31:42.721 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.721 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.721 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:31:42.721 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.048625+0000", 2024-09-16T09:31:42.721 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:31:42.721 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.721 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.721 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:31:42.721 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.048626+0000", 2024-09-16T09:31:42.721 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0219999999999999e-06 2024-09-16T09:31:42.721 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.721 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.721 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:31:42.722 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.048633+0000", 2024-09-16T09:31:42.722 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.6379999999999998e-06 2024-09-16T09:31:42.722 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.722 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.722 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:31:42.722 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.137040+0000", 2024-09-16T09:31:42.722 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.088407854999999994 2024-09-16T09:31:42.722 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.722 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.722 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:31:42.722 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.137056+0000", 2024-09-16T09:31:42.722 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.5583e-05 2024-09-16T09:31:42.722 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.722 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.723 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:31:42.723 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.139863+0000", 2024-09-16T09:31:42.723 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0028066559999999998 2024-09-16T09:31:42.723 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.723 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.723 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:31:42.723 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.139883+0000", 2024-09-16T09:31:42.723 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.0599e-05 2024-09-16T09:31:42.723 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.723 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.723 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:31:42.723 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.152841+0000", 2024-09-16T09:31:42.723 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.012958081999999999 2024-09-16T09:31:42.723 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.723 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.724 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:31:42.724 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:51.152879+0000", 2024-09-16T09:31:42.724 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.7564000000000002e-05 2024-09-16T09:31:42.724 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:31:42.724 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:31:42.724 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:31:42.724 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.724 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.724 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:215736 7.1 7:8d3aa3e8:::benchmark_data_smithi070_78438_object89605:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e141)", 2024-09-16T09:31:42.724 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:29:03.742609+0000", 2024-09-16T09:31:42.724 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 158.922045626, 2024-09-16T09:31:42.724 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.19267857899999999, 2024-09-16T09:31:42.724 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:31:42.725 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:31:42.725 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:31:42.725 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:31:42.725 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:31:42.725 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:31:42.725 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 215736 2024-09-16T09:31:42.725 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.725 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:31:42.725 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.725 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:31:42.725 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.742609+0000", 2024-09-16T09:31:42.725 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:31:42.725 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.725 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.726 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:31:42.726 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.742609+0000", 2024-09-16T09:31:42.726 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:31:42.726 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.726 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.726 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:31:42.726 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.742610+0000", 2024-09-16T09:31:42.726 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.387e-06 2024-09-16T09:31:42.726 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.727 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.727 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:31:42.727 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.742615+0000", 2024-09-16T09:31:42.727 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.5469999999999998e-06 2024-09-16T09:31:42.727 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.727 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.727 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:31:42.727 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.742627+0000", 2024-09-16T09:31:42.727 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.2577e-05 2024-09-16T09:31:42.727 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.727 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.727 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:31:42.727 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.742635+0000", 2024-09-16T09:31:42.728 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.7349999999999996e-06 2024-09-16T09:31:42.728 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.728 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.728 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:31:42.728 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.742684+0000", 2024-09-16T09:31:42.728 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.9285e-05 2024-09-16T09:31:42.728 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.728 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.728 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:31:42.728 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.742816+0000", 2024-09-16T09:31:42.728 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00013143599999999999 2024-09-16T09:31:42.728 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.728 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.729 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 3,6", 2024-09-16T09:31:42.729 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.742948+0000", 2024-09-16T09:31:42.729 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00013233400000000001 2024-09-16T09:31:42.729 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.729 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.729 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:31:42.729 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.744957+0000", 2024-09-16T09:31:42.729 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.002008953 2024-09-16T09:31:42.729 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.729 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.729 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:31:42.729 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.746331+0000", 2024-09-16T09:31:42.729 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.001373967 2024-09-16T09:31:42.729 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.729 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.730 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:31:42.730 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.935196+0000", 2024-09-16T09:31:42.730 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.18886488200000001 2024-09-16T09:31:42.730 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.730 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.730 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:31:42.730 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.935241+0000", 2024-09-16T09:31:42.730 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.5334999999999999e-05 2024-09-16T09:31:42.730 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.730 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.730 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:31:42.730 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.935287+0000", 2024-09-16T09:31:42.730 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.6140999999999998e-05 2024-09-16T09:31:42.730 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:31:42.731 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:31:42.731 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:31:42.731 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.731 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.731 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.25064.0:46974 8.2 8:41eba22e:::benchmark_data_smithi070_107919_object46973:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e165)", 2024-09-16T09:31:42.731 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:30:17.214234+0000", 2024-09-16T09:31:42.731 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 85.450420167999994, 2024-09-16T09:31:42.731 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.089142967000000004, 2024-09-16T09:31:42.731 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:31:42.731 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:31:42.731 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:31:42.731 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:31:42.731 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.25064", 2024-09-16T09:31:42.731 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/2578256439", 2024-09-16T09:31:42.732 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 46974 2024-09-16T09:31:42.732 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.732 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:31:42.732 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.732 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:31:42.732 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:17.214234+0000", 2024-09-16T09:31:42.732 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:31:42.732 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.732 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.732 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:31:42.732 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:17.214234+0000", 2024-09-16T09:31:42.732 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:31:42.732 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.733 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.733 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:31:42.733 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:17.214235+0000", 2024-09-16T09:31:42.733 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.3799999999999996e-07 2024-09-16T09:31:42.733 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.733 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.733 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:31:42.733 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:17.214337+0000", 2024-09-16T09:31:42.733 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00010166900000000001 2024-09-16T09:31:42.733 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.733 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.733 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:31:42.733 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:17.214348+0000", 2024-09-16T09:31:42.734 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1351e-05 2024-09-16T09:31:42.734 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.734 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.734 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:31:42.734 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:17.214356+0000", 2024-09-16T09:31:42.734 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.4789999999999999e-06 2024-09-16T09:31:42.734 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.734 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.734 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:31:42.734 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:17.214400+0000", 2024-09-16T09:31:42.734 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.4437999999999997e-05 2024-09-16T09:31:42.734 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.734 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.734 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:31:42.734 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:17.214504+0000", 2024-09-16T09:31:42.735 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00010417 2024-09-16T09:31:42.735 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.735 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.735 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,5", 2024-09-16T09:31:42.735 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:17.214618+0000", 2024-09-16T09:31:42.735 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00011418599999999999 2024-09-16T09:31:42.735 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.735 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.735 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:31:42.735 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:17.216201+0000", 2024-09-16T09:31:42.735 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00158299 2024-09-16T09:31:42.735 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.735 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.735 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:31:42.736 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:17.217406+0000", 2024-09-16T09:31:42.736 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0012043589999999999 2024-09-16T09:31:42.736 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.736 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.736 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:31:42.736 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:17.303276+0000", 2024-09-16T09:31:42.736 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.085870418000000004 2024-09-16T09:31:42.736 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.736 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.736 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:31:42.736 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:17.303332+0000", 2024-09-16T09:31:42.736 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.6261000000000002e-05 2024-09-16T09:31:42.737 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.737 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.737 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:31:42.737 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:17.303377+0000", 2024-09-16T09:31:42.737 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.4808000000000002e-05 2024-09-16T09:31:42.737 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:31:42.737 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:31:42.737 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:31:42.737 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.737 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.737 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.25064.0:46979 8.b 8:d2da9167:::benchmark_data_smithi070_107919_object46978:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e165)", 2024-09-16T09:31:42.737 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:30:17.218130+0000", 2024-09-16T09:31:42.737 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 85.446524874000005, 2024-09-16T09:31:42.738 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.079397475999999995, 2024-09-16T09:31:42.738 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:31:42.738 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:31:42.738 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:31:42.738 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:31:42.738 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.25064", 2024-09-16T09:31:42.738 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/2578256439", 2024-09-16T09:31:42.738 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 46979 2024-09-16T09:31:42.738 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.738 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:31:42.738 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.738 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:31:42.738 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:17.218130+0000", 2024-09-16T09:31:42.738 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:31:42.739 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.739 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.739 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:31:42.739 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:17.218130+0000", 2024-09-16T09:31:42.739 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:31:42.739 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.739 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.739 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:31:42.739 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:17.218130+0000", 2024-09-16T09:31:42.739 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.9899999999999999e-07 2024-09-16T09:31:42.740 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.740 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.740 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:31:42.740 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:17.218196+0000", 2024-09-16T09:31:42.740 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.5576999999999999e-05 2024-09-16T09:31:42.740 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.740 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.740 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:31:42.740 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:17.218203+0000", 2024-09-16T09:31:42.740 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.0129999999999996e-06 2024-09-16T09:31:42.740 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.740 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.740 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:31:42.741 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:17.218207+0000", 2024-09-16T09:31:42.741 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.1679999999999997e-06 2024-09-16T09:31:42.741 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.741 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.741 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:31:42.741 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:17.218235+0000", 2024-09-16T09:31:42.741 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.7597e-05 2024-09-16T09:31:42.741 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.741 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.741 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:31:42.741 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:17.218304+0000", 2024-09-16T09:31:42.741 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.9210999999999999e-05 2024-09-16T09:31:42.741 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.741 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.742 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 1,3", 2024-09-16T09:31:42.742 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:17.218505+0000", 2024-09-16T09:31:42.742 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00020078500000000001 2024-09-16T09:31:42.742 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.742 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.742 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:31:42.742 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:17.219881+0000", 2024-09-16T09:31:42.742 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.001376565 2024-09-16T09:31:42.742 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.742 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.742 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:31:42.742 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:17.297398+0000", 2024-09-16T09:31:42.742 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.077516285000000004 2024-09-16T09:31:42.742 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.743 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.743 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:31:42.743 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:17.297433+0000", 2024-09-16T09:31:42.743 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.5429000000000002e-05 2024-09-16T09:31:42.743 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.743 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.743 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:31:42.743 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:17.297487+0000", 2024-09-16T09:31:42.743 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.4344e-05 2024-09-16T09:31:42.743 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.743 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.743 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:31:42.743 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:17.297527+0000", 2024-09-16T09:31:42.744 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.9703000000000003e-05 2024-09-16T09:31:42.744 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:31:42.744 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:31:42.744 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:31:42.744 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.744 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.744 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.25064.0:46984 8.2 8:45706fdb:::benchmark_data_smithi070_107919_object46983:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e165)", 2024-09-16T09:31:42.744 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:30:17.219923+0000", 2024-09-16T09:31:42.744 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 85.444731156000003, 2024-09-16T09:31:42.744 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.083882824999999994, 2024-09-16T09:31:42.744 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:31:42.744 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:31:42.744 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:31:42.744 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:31:42.745 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.25064", 2024-09-16T09:31:42.745 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/2578256439", 2024-09-16T09:31:42.745 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 46984 2024-09-16T09:31:42.745 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.745 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:31:42.745 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.745 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:31:42.745 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:17.219923+0000", 2024-09-16T09:31:42.745 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:31:42.745 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.745 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.745 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:31:42.745 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:17.219923+0000", 2024-09-16T09:31:42.745 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:31:42.746 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.746 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.746 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:31:42.746 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:17.219924+0000", 2024-09-16T09:31:42.746 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1119999999999999e-06 2024-09-16T09:31:42.746 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.746 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.746 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:31:42.746 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:17.219975+0000", 2024-09-16T09:31:42.746 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.0377000000000002e-05 2024-09-16T09:31:42.746 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.746 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.746 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:31:42.746 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:17.219986+0000", 2024-09-16T09:31:42.747 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1396000000000001e-05 2024-09-16T09:31:42.747 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.747 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.747 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:31:42.747 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:17.219993+0000", 2024-09-16T09:31:42.747 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.9600000000000003e-06 2024-09-16T09:31:42.747 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.747 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.747 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:31:42.747 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:17.295982+0000", 2024-09-16T09:31:42.747 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.075989142999999995 2024-09-16T09:31:42.747 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.748 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.748 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:31:42.748 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:17.296143+0000", 2024-09-16T09:31:42.748 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00016059700000000001 2024-09-16T09:31:42.748 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.748 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.748 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,5", 2024-09-16T09:31:42.748 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:17.296294+0000", 2024-09-16T09:31:42.748 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00015154599999999999 2024-09-16T09:31:42.748 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.748 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.748 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:31:42.748 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:17.299274+0000", 2024-09-16T09:31:42.749 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0029792730000000002 2024-09-16T09:31:42.749 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.749 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.749 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:31:42.749 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:17.303455+0000", 2024-09-16T09:31:42.749 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0041808669999999996 2024-09-16T09:31:42.749 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.749 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.749 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:31:42.749 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:17.303474+0000", 2024-09-16T09:31:42.749 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.9483e-05 2024-09-16T09:31:42.749 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.749 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.749 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:31:42.750 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:17.303553+0000", 2024-09-16T09:31:42.750 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.9122999999999994e-05 2024-09-16T09:31:42.750 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.751 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.751 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:31:42.751 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:17.303806+0000", 2024-09-16T09:31:42.752 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00025294800000000002 2024-09-16T09:31:42.752 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:31:42.752 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:31:42.752 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:31:42.752 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.752 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.752 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_repop(client.25064.0:46982 8.5 e165/152 8:a153e887:::benchmark_data_smithi070_107919_object46981:head v 165'2939, mlcod=165'2937)", 2024-09-16T09:31:42.752 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:30:17.219966+0000", 2024-09-16T09:31:42.752 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 85.444688796999998, 2024-09-16T09:31:42.752 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.081178690999999997, 2024-09-16T09:31:42.752 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:31:42.753 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:31:42.753 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:31:42.753 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:31:42.753 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.753 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:31:42.753 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:17.219966+0000", 2024-09-16T09:31:42.753 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:31:42.753 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.753 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.754 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:31:42.754 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:17.219966+0000", 2024-09-16T09:31:42.754 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:31:42.754 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.754 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.754 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:31:42.755 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:17.219966+0000", 2024-09-16T09:31:42.755 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.1699999999999997e-07 2024-09-16T09:31:42.755 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.755 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.755 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:31:42.755 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:17.219990+0000", 2024-09-16T09:31:42.755 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.3791e-05 2024-09-16T09:31:42.756 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.756 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.756 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:31:42.756 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:17.220001+0000", 2024-09-16T09:31:42.756 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0533000000000001e-05 2024-09-16T09:31:42.756 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.757 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.757 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:31:42.757 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:17.220007+0000", 2024-09-16T09:31:42.757 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.4289999999999998e-06 2024-09-16T09:31:42.757 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.758 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.758 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:31:42.758 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:17.299230+0000", 2024-09-16T09:31:42.758 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.079222687 2024-09-16T09:31:42.758 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.758 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.758 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:31:42.759 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:17.299249+0000", 2024-09-16T09:31:42.759 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.8828e-05 2024-09-16T09:31:42.759 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.759 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.759 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:31:42.760 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:17.301104+0000", 2024-09-16T09:31:42.760 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.001855808 2024-09-16T09:31:42.760 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.760 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.760 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:31:42.760 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:17.301144+0000", 2024-09-16T09:31:42.761 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.9898000000000003e-05 2024-09-16T09:31:42.761 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:31:42.761 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:31:42.761 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:31:42.761 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.761 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.762 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.25064.0:58729 8.b 8:d496684a:::benchmark_data_smithi070_107919_object58728:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e170)", 2024-09-16T09:31:42.762 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:30:27.568367+0000", 2024-09-16T09:31:42.762 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 75.096287654999998, 2024-09-16T09:31:42.762 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.087170015000000003, 2024-09-16T09:31:42.762 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:31:42.762 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:31:42.763 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:31:42.763 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:31:42.763 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.25064", 2024-09-16T09:31:42.763 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/2578256439", 2024-09-16T09:31:42.763 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 58729 2024-09-16T09:31:42.763 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.764 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:31:42.764 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.764 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:31:42.764 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:27.568367+0000", 2024-09-16T09:31:42.764 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:31:42.764 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.765 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.765 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:31:42.765 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:27.568367+0000", 2024-09-16T09:31:42.765 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:31:42.765 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.766 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.766 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:31:42.766 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:27.568367+0000", 2024-09-16T09:31:42.766 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.5899999999999996e-07 2024-09-16T09:31:42.766 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.766 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.767 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:31:42.767 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:27.568399+0000", 2024-09-16T09:31:42.767 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.2098999999999999e-05 2024-09-16T09:31:42.767 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.767 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.767 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:31:42.768 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:27.568407+0000", 2024-09-16T09:31:42.768 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.4649999999999997e-06 2024-09-16T09:31:42.768 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.768 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.768 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:31:42.769 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:27.568413+0000", 2024-09-16T09:31:42.769 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.2650000000000002e-06 2024-09-16T09:31:42.769 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.769 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.769 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:31:42.770 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:27.594594+0000", 2024-09-16T09:31:42.770 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.026180878000000001 2024-09-16T09:31:42.770 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.770 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.770 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:31:42.770 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:27.594733+0000", 2024-09-16T09:31:42.770 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00013914099999999999 2024-09-16T09:31:42.770 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.771 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.771 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 1,2", 2024-09-16T09:31:42.771 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:27.594879+0000", 2024-09-16T09:31:42.771 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000145981 2024-09-16T09:31:42.771 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.772 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.772 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:31:42.772 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:27.611800+0000", 2024-09-16T09:31:42.772 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.016921268999999999 2024-09-16T09:31:42.772 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.772 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.772 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:31:42.773 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:27.612066+0000", 2024-09-16T09:31:42.773 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00026519599999999999 2024-09-16T09:31:42.773 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.773 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.773 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:31:42.773 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:27.653146+0000", 2024-09-16T09:31:42.773 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.041080083000000003 2024-09-16T09:31:42.773 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.773 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.773 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:31:42.774 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:27.653220+0000", 2024-09-16T09:31:42.774 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.4717999999999999e-05 2024-09-16T09:31:42.774 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.774 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.774 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:31:42.774 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:27.655537+0000", 2024-09-16T09:31:42.774 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0023163609999999999 2024-09-16T09:31:42.774 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:31:42.774 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:31:42.774 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:31:42.774 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.774 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.775 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.25064.0:69511 8.2 8:4648a4fb:::benchmark_data_smithi070_107919_object69510:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e174)", 2024-09-16T09:31:42.775 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:30:36.630107+0000", 2024-09-16T09:31:42.775 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 66.034547461000002, 2024-09-16T09:31:42.775 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.10345686, 2024-09-16T09:31:42.775 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:31:42.775 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:31:42.775 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:31:42.775 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:31:42.775 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.25064", 2024-09-16T09:31:42.775 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/2578256439", 2024-09-16T09:31:42.775 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 69511 2024-09-16T09:31:42.775 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.775 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:31:42.776 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.776 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:31:42.776 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.630107+0000", 2024-09-16T09:31:42.776 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:31:42.776 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.776 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.776 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:31:42.776 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.630107+0000", 2024-09-16T09:31:42.776 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:31:42.776 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.776 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.776 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:31:42.776 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.630109+0000", 2024-09-16T09:31:42.777 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.5719999999999999e-06 2024-09-16T09:31:42.777 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.777 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.777 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:31:42.777 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.630148+0000", 2024-09-16T09:31:42.777 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.9671e-05 2024-09-16T09:31:42.777 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.777 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.777 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:31:42.777 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.630162+0000", 2024-09-16T09:31:42.777 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.4270000000000001e-05 2024-09-16T09:31:42.777 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.778 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.778 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:31:42.778 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.630171+0000", 2024-09-16T09:31:42.778 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.4519999999999997e-06 2024-09-16T09:31:42.778 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.778 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.778 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:31:42.778 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.634766+0000", 2024-09-16T09:31:42.778 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0045951869999999997 2024-09-16T09:31:42.778 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.778 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.778 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:31:42.779 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.634884+0000", 2024-09-16T09:31:42.779 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00011819699999999999 2024-09-16T09:31:42.779 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.779 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.779 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,5", 2024-09-16T09:31:42.779 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.635031+0000", 2024-09-16T09:31:42.779 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00014644899999999999 2024-09-16T09:31:42.779 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.779 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.779 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:31:42.779 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.637283+0000", 2024-09-16T09:31:42.779 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0022525940000000001 2024-09-16T09:31:42.780 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.780 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.780 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:31:42.780 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.651674+0000", 2024-09-16T09:31:42.780 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.014391111999999999 2024-09-16T09:31:42.780 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.781 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.781 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:31:42.781 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.733447+0000", 2024-09-16T09:31:42.781 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.081772519000000002 2024-09-16T09:31:42.781 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.782 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.782 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:31:42.782 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.733505+0000", 2024-09-16T09:31:42.782 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.7907e-05 2024-09-16T09:31:42.782 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.782 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.782 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:31:42.782 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.733564+0000", 2024-09-16T09:31:42.782 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.893e-05 2024-09-16T09:31:42.782 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:31:42.783 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:31:42.783 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:31:42.783 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.783 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.783 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.25064.0:69513 8.2 8:42a1c798:::benchmark_data_smithi070_107919_object69512:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e174)", 2024-09-16T09:31:42.783 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:30:36.631827+0000", 2024-09-16T09:31:42.783 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 66.032827087000001, 2024-09-16T09:31:42.783 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.101899426, 2024-09-16T09:31:42.783 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:31:42.783 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:31:42.784 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:31:42.784 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:31:42.784 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.25064", 2024-09-16T09:31:42.784 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/2578256439", 2024-09-16T09:31:42.784 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 69513 2024-09-16T09:31:42.784 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.784 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:31:42.784 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.784 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:31:42.784 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.631827+0000", 2024-09-16T09:31:42.784 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:31:42.784 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.784 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.785 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:31:42.785 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.631827+0000", 2024-09-16T09:31:42.785 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:31:42.785 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.785 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.785 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:31:42.785 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.631829+0000", 2024-09-16T09:31:42.785 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.4890000000000001e-06 2024-09-16T09:31:42.785 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.785 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.785 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:31:42.785 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.631866+0000", 2024-09-16T09:31:42.785 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.7302999999999998e-05 2024-09-16T09:31:42.785 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.786 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.786 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:31:42.786 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.631879+0000", 2024-09-16T09:31:42.786 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.2418e-05 2024-09-16T09:31:42.786 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.786 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.786 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:31:42.786 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.631886+0000", 2024-09-16T09:31:42.786 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.8099999999999998e-06 2024-09-16T09:31:42.786 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.786 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.786 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:31:42.786 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.637377+0000", 2024-09-16T09:31:42.786 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0054910790000000003 2024-09-16T09:31:42.786 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.787 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.787 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:31:42.787 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.637474+0000", 2024-09-16T09:31:42.787 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.6710000000000001e-05 2024-09-16T09:31:42.787 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.787 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.787 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,5", 2024-09-16T09:31:42.787 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.637603+0000", 2024-09-16T09:31:42.787 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000128422 2024-09-16T09:31:42.787 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.787 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.787 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:31:42.788 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.645753+0000", 2024-09-16T09:31:42.788 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0081507920000000005 2024-09-16T09:31:42.788 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.788 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.788 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:31:42.788 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.651711+0000", 2024-09-16T09:31:42.788 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0059576710000000003 2024-09-16T09:31:42.788 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.788 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.788 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:31:42.789 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.733629+0000", 2024-09-16T09:31:42.789 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.081918284999999993 2024-09-16T09:31:42.789 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.789 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.789 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:31:42.789 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.733689+0000", 2024-09-16T09:31:42.789 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.9978000000000001e-05 2024-09-16T09:31:42.789 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.789 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.789 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:31:42.789 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.733727+0000", 2024-09-16T09:31:42.789 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.7469000000000003e-05 2024-09-16T09:31:42.789 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:31:42.790 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:31:42.790 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:31:42.790 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.790 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.790 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.25064.0:69516 8.2 8:4c59d842:::benchmark_data_smithi070_107919_object69515:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e174)", 2024-09-16T09:31:42.791 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:30:36.631908+0000", 2024-09-16T09:31:42.791 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 66.032746625000001, 2024-09-16T09:31:42.791 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.101966154, 2024-09-16T09:31:42.791 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:31:42.791 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:31:42.791 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:31:42.791 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:31:42.791 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.25064", 2024-09-16T09:31:42.791 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/2578256439", 2024-09-16T09:31:42.791 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 69516 2024-09-16T09:31:42.791 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.791 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:31:42.791 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.791 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:31:42.792 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.631908+0000", 2024-09-16T09:31:42.792 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:31:42.792 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.792 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.792 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:31:42.792 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.631908+0000", 2024-09-16T09:31:42.792 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:31:42.792 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.792 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.792 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:31:42.792 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.631908+0000", 2024-09-16T09:31:42.792 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.4700000000000001e-07 2024-09-16T09:31:42.793 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.793 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.793 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:31:42.793 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.631940+0000", 2024-09-16T09:31:42.793 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.1381999999999999e-05 2024-09-16T09:31:42.793 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.793 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.793 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:31:42.793 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.631948+0000", 2024-09-16T09:31:42.793 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.9589999999999995e-06 2024-09-16T09:31:42.793 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.793 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.794 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:31:42.794 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.631953+0000", 2024-09-16T09:31:42.794 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.728e-06 2024-09-16T09:31:42.794 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.794 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.794 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:31:42.794 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.642019+0000", 2024-09-16T09:31:42.794 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.010065988 2024-09-16T09:31:42.794 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.794 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.794 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:31:42.794 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.642124+0000", 2024-09-16T09:31:42.794 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000104385 2024-09-16T09:31:42.794 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.795 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.795 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,5", 2024-09-16T09:31:42.795 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.642253+0000", 2024-09-16T09:31:42.795 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000129111 2024-09-16T09:31:42.795 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.795 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.795 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:31:42.795 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.645760+0000", 2024-09-16T09:31:42.795 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.003507025 2024-09-16T09:31:42.795 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.795 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.795 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:31:42.795 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.651738+0000", 2024-09-16T09:31:42.795 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0059782209999999997 2024-09-16T09:31:42.796 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.796 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.796 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:31:42.796 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.733782+0000", 2024-09-16T09:31:42.796 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.082044254999999996 2024-09-16T09:31:42.796 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.796 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.796 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:31:42.796 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.733834+0000", 2024-09-16T09:31:42.796 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.1610999999999998e-05 2024-09-16T09:31:42.796 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.796 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.797 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:31:42.797 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.733874+0000", 2024-09-16T09:31:42.797 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.9941999999999998e-05 2024-09-16T09:31:42.797 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:31:42.797 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:31:42.797 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:31:42.797 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.797 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.797 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.25064.0:69529 8.2 8:4d722993:::benchmark_data_smithi070_107919_object69528:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e174)", 2024-09-16T09:31:42.797 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:30:36.645961+0000", 2024-09-16T09:31:42.798 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 66.018693691999999, 2024-09-16T09:31:42.798 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.092111787000000001, 2024-09-16T09:31:42.798 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:31:42.798 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:31:42.798 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:31:42.798 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:31:42.798 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.25064", 2024-09-16T09:31:42.798 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/2578256439", 2024-09-16T09:31:42.798 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 69529 2024-09-16T09:31:42.798 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.798 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:31:42.798 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.798 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:31:42.799 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.645961+0000", 2024-09-16T09:31:42.799 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:31:42.799 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.799 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.799 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:31:42.799 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.645961+0000", 2024-09-16T09:31:42.799 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:31:42.799 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.799 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.799 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:31:42.799 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.645962+0000", 2024-09-16T09:31:42.799 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.7260000000000001e-06 2024-09-16T09:31:42.799 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.799 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.800 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:31:42.800 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.646007+0000", 2024-09-16T09:31:42.800 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.4472999999999999e-05 2024-09-16T09:31:42.800 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.800 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.800 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:31:42.800 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.646018+0000", 2024-09-16T09:31:42.800 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1476e-05 2024-09-16T09:31:42.800 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.800 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.800 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:31:42.801 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.646026+0000", 2024-09-16T09:31:42.801 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.8320000000000006e-06 2024-09-16T09:31:42.801 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.801 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.801 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:31:42.801 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.646077+0000", 2024-09-16T09:31:42.801 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.1099e-05 2024-09-16T09:31:42.801 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.801 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.801 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:31:42.801 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.646224+0000", 2024-09-16T09:31:42.801 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00014714 2024-09-16T09:31:42.801 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.801 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.802 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,5", 2024-09-16T09:31:42.802 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.646384+0000", 2024-09-16T09:31:42.802 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00015904299999999999 2024-09-16T09:31:42.802 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.802 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.802 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:31:42.802 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.648511+0000", 2024-09-16T09:31:42.802 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0021279300000000001 2024-09-16T09:31:42.802 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.802 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.802 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:31:42.802 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.651758+0000", 2024-09-16T09:31:42.802 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0032464350000000002 2024-09-16T09:31:42.803 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.803 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.803 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:31:42.803 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.733986+0000", 2024-09-16T09:31:42.803 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.082227763999999995 2024-09-16T09:31:42.803 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.803 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.803 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:31:42.803 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.734031+0000", 2024-09-16T09:31:42.803 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.5707999999999997e-05 2024-09-16T09:31:42.804 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.804 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.804 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:31:42.804 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.738073+0000", 2024-09-16T09:31:42.804 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0040411609999999997 2024-09-16T09:31:42.804 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:31:42.804 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:31:42.804 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:31:42.804 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.804 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.804 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.25064.0:108832 8.b 8:d3bdb339:::benchmark_data_smithi070_107919_object108831:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e187)", 2024-09-16T09:31:42.805 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:31:11.090521+0000", 2024-09-16T09:31:42.805 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 31.574133577000001, 2024-09-16T09:31:42.805 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.077086900999999999, 2024-09-16T09:31:42.805 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:31:42.805 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:31:42.805 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:31:42.805 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:31:42.805 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.25064", 2024-09-16T09:31:42.805 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/2578256439", 2024-09-16T09:31:42.805 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 108832 2024-09-16T09:31:42.805 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.805 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:31:42.806 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.806 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:31:42.806 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:11.090521+0000", 2024-09-16T09:31:42.806 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:31:42.806 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.806 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.806 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:31:42.806 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:11.090521+0000", 2024-09-16T09:31:42.806 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:31:42.806 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.806 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.806 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:31:42.806 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:11.090523+0000", 2024-09-16T09:31:42.806 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.8649999999999999e-06 2024-09-16T09:31:42.807 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.807 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.807 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:31:42.807 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:11.090558+0000", 2024-09-16T09:31:42.807 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.5114999999999998e-05 2024-09-16T09:31:42.807 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.807 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.807 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:31:42.807 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:11.090570+0000", 2024-09-16T09:31:42.807 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.2048999999999999e-05 2024-09-16T09:31:42.807 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.807 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.807 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:31:42.807 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:11.090578+0000", 2024-09-16T09:31:42.808 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.5280000000000003e-06 2024-09-16T09:31:42.808 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.808 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.808 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:31:42.808 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:11.095720+0000", 2024-09-16T09:31:42.808 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0051411770000000002 2024-09-16T09:31:42.808 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.808 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.808 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:31:42.808 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:11.095894+0000", 2024-09-16T09:31:42.808 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000174291 2024-09-16T09:31:42.808 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.808 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.809 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 1,3", 2024-09-16T09:31:42.809 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:11.096066+0000", 2024-09-16T09:31:42.809 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000171879 2024-09-16T09:31:42.809 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.809 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.809 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:31:42.809 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:11.103363+0000", 2024-09-16T09:31:42.809 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.007297123 2024-09-16T09:31:42.809 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.809 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.809 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:31:42.809 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:11.103385+0000", 2024-09-16T09:31:42.809 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.2266e-05 2024-09-16T09:31:42.810 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.810 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.810 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:31:42.810 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:11.163645+0000", 2024-09-16T09:31:42.810 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.060259423999999999 2024-09-16T09:31:42.810 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.810 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.810 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:31:42.810 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:11.163720+0000", 2024-09-16T09:31:42.810 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.5444999999999993e-05 2024-09-16T09:31:42.810 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.810 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.810 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:31:42.810 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:11.167608+0000", 2024-09-16T09:31:42.811 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0038877389999999999 2024-09-16T09:31:42.811 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:31:42.811 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:31:42.811 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:31:42.811 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.811 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.811 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.25064.0:112692 8.b 8:db6cf8c4:::benchmark_data_smithi070_107919_object112691:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e188)", 2024-09-16T09:31:42.811 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:31:14.829160+0000", 2024-09-16T09:31:42.811 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 27.835494851, 2024-09-16T09:31:42.811 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.080842331000000003, 2024-09-16T09:31:42.811 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:31:42.811 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:31:42.811 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:31:42.812 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:31:42.812 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.25064", 2024-09-16T09:31:42.812 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/2578256439", 2024-09-16T09:31:42.812 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 112692 2024-09-16T09:31:42.812 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.812 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:31:42.812 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.812 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:31:42.812 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:14.829160+0000", 2024-09-16T09:31:42.812 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:31:42.812 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.812 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.812 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:31:42.812 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:14.829160+0000", 2024-09-16T09:31:42.813 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:31:42.813 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.813 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.813 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:31:42.813 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:14.829161+0000", 2024-09-16T09:31:42.813 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.7039999999999999e-06 2024-09-16T09:31:42.813 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.813 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.813 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:31:42.813 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:14.829219+0000", 2024-09-16T09:31:42.813 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.8167000000000001e-05 2024-09-16T09:31:42.813 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.813 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.814 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:31:42.814 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:14.829234+0000", 2024-09-16T09:31:42.814 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.4952e-05 2024-09-16T09:31:42.814 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.814 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.814 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:31:42.814 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:14.829242+0000", 2024-09-16T09:31:42.814 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.7200000000000006e-06 2024-09-16T09:31:42.814 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.814 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.814 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:31:42.814 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:14.832414+0000", 2024-09-16T09:31:42.815 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.003171994 2024-09-16T09:31:42.815 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.815 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.815 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:31:42.815 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:14.832571+0000", 2024-09-16T09:31:42.815 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00015684 2024-09-16T09:31:42.815 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.815 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.815 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 1,3", 2024-09-16T09:31:42.815 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:14.832759+0000", 2024-09-16T09:31:42.815 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00018798999999999999 2024-09-16T09:31:42.815 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.815 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.815 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:31:42.816 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:14.835540+0000", 2024-09-16T09:31:42.816 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0027815040000000002 2024-09-16T09:31:42.816 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.816 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.816 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:31:42.816 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:14.836665+0000", 2024-09-16T09:31:42.816 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.001124948 2024-09-16T09:31:42.816 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.816 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.816 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:31:42.816 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:14.907871+0000", 2024-09-16T09:31:42.816 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.071205953000000002 2024-09-16T09:31:42.816 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.817 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.817 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:31:42.817 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:14.907957+0000", 2024-09-16T09:31:42.817 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.5576999999999997e-05 2024-09-16T09:31:42.817 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:42.817 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:42.817 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:31:42.817 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:14.910002+0000", 2024-09-16T09:31:42.817 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0020449819999999999 2024-09-16T09:31:42.817 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:31:42.817 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:31:42.817 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:31:42.817 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:31:42.817 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:31:42.818 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-09-16T09:31:42.818 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:31:42.854 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:42.855 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:42.855 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:31:42.956 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:31:42.986 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:42.987 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:42.987 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:31:43.054 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:31:42 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:31:43.089 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:31:43.140 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:31:43.149 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:43.149 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:43.150 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:31:43.250 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:31:43.285 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:43.286 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:43.286 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:31:43.319 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:43 smithi070 ceph-mon[30388]: 1.2 scrub starts 2024-09-16T09:31:43.319 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:43 smithi070 ceph-mon[30388]: pgmap v312: 168 pgs: 1 active+recovery_wait+degraded, 3 active+recovering+undersized+remapped, 164 active+clean; 6.8 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 640 op/s; 1/335994 objects degraded (0.000%); 7625/335994 objects misplaced (2.269%); 3.8 MiB/s, 60 objects/s recovering 2024-09-16T09:31:43.320 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:43 smithi070 ceph-mon[30388]: 1.1 scrub starts 2024-09-16T09:31:43.320 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:43 smithi070 ceph-mon[30388]: 1.e scrub starts 2024-09-16T09:31:43.320 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:43 smithi070 ceph-mon[30388]: 1.b scrub starts 2024-09-16T09:31:43.320 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:43 smithi070 ceph-mon[40691]: 1.2 scrub starts 2024-09-16T09:31:43.320 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:43 smithi070 ceph-mon[40691]: pgmap v312: 168 pgs: 1 active+recovery_wait+degraded, 3 active+recovering+undersized+remapped, 164 active+clean; 6.8 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 640 op/s; 1/335994 objects degraded (0.000%); 7625/335994 objects misplaced (2.269%); 3.8 MiB/s, 60 objects/s recovering 2024-09-16T09:31:43.320 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:43 smithi070 ceph-mon[40691]: 1.1 scrub starts 2024-09-16T09:31:43.320 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:43 smithi070 ceph-mon[40691]: 1.e scrub starts 2024-09-16T09:31:43.320 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:43 smithi070 ceph-mon[40691]: 1.b scrub starts 2024-09-16T09:31:43.387 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:31:43.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:43 smithi084 ceph-mon[32629]: 1.2 scrub starts 2024-09-16T09:31:43.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:43 smithi084 ceph-mon[32629]: pgmap v312: 168 pgs: 1 active+recovery_wait+degraded, 3 active+recovering+undersized+remapped, 164 active+clean; 6.8 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 640 op/s; 1/335994 objects degraded (0.000%); 7625/335994 objects misplaced (2.269%); 3.8 MiB/s, 60 objects/s recovering 2024-09-16T09:31:43.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:43 smithi084 ceph-mon[32629]: 1.1 scrub starts 2024-09-16T09:31:43.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:43 smithi084 ceph-mon[32629]: 1.e scrub starts 2024-09-16T09:31:43.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:43 smithi084 ceph-mon[32629]: 1.b scrub starts 2024-09-16T09:31:43.425 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:43.426 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:43.426 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:31:43.527 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:31:43.560 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:43.560 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:43.561 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:31:43.662 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:31:43.694 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:43.694 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:43.695 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:31:43.796 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:31:43.824 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:43.825 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:43.825 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:31:43.918 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.5 dump_ops_in_flight 2024-09-16T09:31:43.926 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:31:43.957 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:43.958 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:43.959 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:31:44.059 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:31:44.109 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:44.110 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:44.110 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:31:44.211 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:31:44.243 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:44.244 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:44.244 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:31:44.304 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:31:44.323 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:31:44.323 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:31:44.324 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:31:44.324 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:31:44.324 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:31:44.324 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:31:44.324 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:31:44.324 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:31:44.324 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:31:44.324 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:31:44.325 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:31:44.325 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:31:44.325 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:31:44.325 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:31:44.325 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:31:44.325 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:31:44.344 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:31:44.375 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:44.376 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:44.376 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:31:44.477 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:31:44.519 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:44.520 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:44.520 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:31:44.552 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:31:44.621 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:31:44.655 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:44.656 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:44.656 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:31:44.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:44 smithi084 ceph-mon[32629]: 1.0 scrub starts 2024-09-16T09:31:44.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:44 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:31:44.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:44 smithi084 ceph-mon[32629]: 1.10 scrub starts 2024-09-16T09:31:44.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:44 smithi084 ceph-mon[32629]: 1.f scrub starts 2024-09-16T09:31:44.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:44 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3329661144' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:31:44.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:44 smithi084 ceph-mon[32629]: Health check update: Degraded data redundancy: 1/335994 objects degraded (0.000%), 1 pg degraded (PG_DEGRADED) 2024-09-16T09:31:44.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:44 smithi070 ceph-mon[30388]: 1.0 scrub starts 2024-09-16T09:31:44.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:44 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:31:44.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:44 smithi070 ceph-mon[30388]: 1.10 scrub starts 2024-09-16T09:31:44.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:44 smithi070 ceph-mon[30388]: 1.f scrub starts 2024-09-16T09:31:44.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:44 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3329661144' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:31:44.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:44 smithi070 ceph-mon[30388]: Health check update: Degraded data redundancy: 1/335994 objects degraded (0.000%), 1 pg degraded (PG_DEGRADED) 2024-09-16T09:31:44.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:44 smithi070 ceph-mon[40691]: 1.0 scrub starts 2024-09-16T09:31:44.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:44 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:31:44.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:44 smithi070 ceph-mon[40691]: 1.10 scrub starts 2024-09-16T09:31:44.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:44 smithi070 ceph-mon[40691]: 1.f scrub starts 2024-09-16T09:31:44.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:44 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3329661144' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:31:44.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:44 smithi070 ceph-mon[40691]: Health check update: Degraded data redundancy: 1/335994 objects degraded (0.000%), 1 pg degraded (PG_DEGRADED) 2024-09-16T09:31:44.757 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:31:44.794 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:44.794 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:44.794 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:31:44.895 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:31:44.932 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:44.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:44.933 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:31:45.034 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:31:45.077 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:45.078 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:45.078 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:31:45.116 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-16T09:31:45.179 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:31:45.201 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:45.202 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:45.203 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:31:45.303 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:31:45.334 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:45.334 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:45.335 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:31:45.422 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:45 smithi070 ceph-mon[30388]: 1.d scrub starts 2024-09-16T09:31:45.423 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:45 smithi070 ceph-mon[30388]: pgmap v313: 168 pgs: 1 active+recovery_wait+degraded, 4 active+clean+scrubbing, 3 active+recovering+undersized+remapped, 160 active+clean; 6.5 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.33k op/s; 1/319491 objects degraded (0.000%); 7145/319491 objects misplaced (2.236%); 7.5 MiB/s, 120 objects/s recovering 2024-09-16T09:31:45.423 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:45 smithi070 ceph-mon[30388]: 1.14 scrub starts 2024-09-16T09:31:45.423 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:45 smithi070 ceph-mon[30388]: 1.12 scrub starts 2024-09-16T09:31:45.423 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:45 smithi070 ceph-mon[40691]: 1.d scrub starts 2024-09-16T09:31:45.423 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:45 smithi070 ceph-mon[40691]: pgmap v313: 168 pgs: 1 active+recovery_wait+degraded, 4 active+clean+scrubbing, 3 active+recovering+undersized+remapped, 160 active+clean; 6.5 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.33k op/s; 1/319491 objects degraded (0.000%); 7145/319491 objects misplaced (2.236%); 7.5 MiB/s, 120 objects/s recovering 2024-09-16T09:31:45.423 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:45 smithi070 ceph-mon[40691]: 1.14 scrub starts 2024-09-16T09:31:45.423 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:45 smithi070 ceph-mon[40691]: 1.12 scrub starts 2024-09-16T09:31:45.436 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:31:45.463 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:45.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:45.464 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:31:45.565 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:31:45.606 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:45.607 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:45.607 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:31:45.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:45 smithi084 ceph-mon[32629]: 1.d scrub starts 2024-09-16T09:31:45.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:45 smithi084 ceph-mon[32629]: pgmap v313: 168 pgs: 1 active+recovery_wait+degraded, 4 active+clean+scrubbing, 3 active+recovering+undersized+remapped, 160 active+clean; 6.5 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.33k op/s; 1/319491 objects degraded (0.000%); 7145/319491 objects misplaced (2.236%); 7.5 MiB/s, 120 objects/s recovering 2024-09-16T09:31:45.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:45 smithi084 ceph-mon[32629]: 1.14 scrub starts 2024-09-16T09:31:45.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:45 smithi084 ceph-mon[32629]: 1.12 scrub starts 2024-09-16T09:31:45.708 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:31:45.734 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:45.735 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:45.735 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:31:45.836 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:31:45.860 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:31:45.868 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:45.869 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:45.869 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:31:45.970 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:31:45.993 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:45.993 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:45.994 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:31:46.094 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:31:46.129 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:46.130 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:46.131 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:31:46.174 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:31:46.177 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2024-09-16T09:31:46.232 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:31:46.265 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:46.266 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:46.266 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:31:46.367 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:31:46.401 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:46.401 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:46.402 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:31:46.502 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:31:46.536 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:46.537 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:46.537 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:31:46.638 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:31:46.671 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:46.671 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:46.672 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:31:46.773 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:31:46.802 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:46.803 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:46.803 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:31:46.904 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:31:46.941 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:46.942 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:46.942 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:31:47.043 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:31:47.073 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:47.073 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:47.074 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:31:47.148 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":314,"stamp":"2024-09-16T09:31:45.622318+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":6967027754,"num_objects":106497,"num_object_clones":0,"num_object_copies":319491,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":7145,"num_objects_unfound":0,"num_objects_dirty":106497,"num_whiteouts":0,"num_read":1132,"num_read_kb":903,"num_write":143346,"num_write_kb":7974230,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7701,"num_bytes_recovered":496417024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79416,"ondisk_log_size":79416,"up":504,"acting":501,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":508,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":22916004,"kb_used_data":21039432,"kb_used_omap":12,"kb_used_meta":1876403,"kb_avail":727045212,"statfs":{"total":767960285184,"available":744494297088,"internally_reserved":0,"allocated":21544378368,"data_stored":21541889985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":1921437234},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6,4,8,4,10,6],"upper_bound":64},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-695926784,"num_objects":-10619,"num_object_clones":0,"num_object_copies":-31857,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-963,"num_objects_unfound":0,"num_objects_dirty":-10619,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":10619,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":963,"num_bytes_recovered":63111168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.906567"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":315,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.215923+0000","last_change":"2024-09-16T09:30:03.848439+0000","last_active":"2024-09-16T09:31:41.215923+0000","last_peered":"2024-09-16T09:31:41.215923+0000","last_clean":"2024-09-16T09:31:41.215923+0000","last_became_active":"2024-09-16T09:25:01.709108+0000","last_became_peered":"2024-09-16T09:25:01.709108+0000","last_unstale":"2024-09-16T09:31:41.215923+0000","last_undegraded":"2024-09-16T09:31:41.215923+0000","last_fullsized":"2024-09-16T09:31:41.215923+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:30:03.848354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:30:03.848354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"66'9","reported_seq":318,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.203441+0000","last_change":"2024-09-16T09:28:52.689373+0000","last_active":"2024-09-16T09:31:41.203441+0000","last_peered":"2024-09-16T09:31:41.203441+0000","last_clean":"2024-09-16T09:31:41.203441+0000","last_became_active":"2024-09-16T09:25:03.714635+0000","last_became_peered":"2024-09-16T09:25:03.714635+0000","last_unstale":"2024-09-16T09:31:41.203441+0000","last_undegraded":"2024-09-16T09:31:41.203441+0000","last_fullsized":"2024-09-16T09:31:41.203441+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:28:52.689304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:52.689304+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":306,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.205868+0000","last_change":"2024-09-16T09:29:17.910367+0000","last_active":"2024-09-16T09:31:41.205868+0000","last_peered":"2024-09-16T09:31:41.205868+0000","last_clean":"2024-09-16T09:31:41.205868+0000","last_became_active":"2024-09-16T09:25:05.715645+0000","last_became_peered":"2024-09-16T09:25:05.715645+0000","last_unstale":"2024-09-16T09:31:41.205868+0000","last_undegraded":"2024-09-16T09:31:41.205868+0000","last_fullsized":"2024-09-16T09:31:41.205868+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:17.910305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:17.910305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":286,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.203806+0000","last_change":"2024-09-16T09:27:11.565306+0000","last_active":"2024-09-16T09:31:41.203806+0000","last_peered":"2024-09-16T09:31:41.203806+0000","last_clean":"2024-09-16T09:31:41.203806+0000","last_became_active":"2024-09-16T09:25:07.715442+0000","last_became_peered":"2024-09-16T09:25:07.715442+0000","last_unstale":"2024-09-16T09:31:41.203806+0000","last_undegraded":"2024-09-16T09:31:41.203806+0000","last_fullsized":"2024-09-16T09:31:41.203806+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:11.565220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:11.565220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":32,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.211204+0000","last_change":"2024-09-16T09:31:09.269700+0000","last_active":"2024-09-16T09:31:41.211204+0000","last_peered":"2024-09-16T09:31:41.211204+0000","last_clean":"2024-09-16T09:31:41.211204+0000","last_became_active":"2024-09-16T09:31:09.269342+0000","last_became_peered":"2024-09-16T09:31:09.269342+0000","last_unstale":"2024-09-16T09:31:41.211204+0000","last_undegraded":"2024-09-16T09:31:41.211204+0000","last_fullsized":"2024-09-16T09:31:41.211204+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:00.068570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:28:00.068570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":164,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.213628+0000","last_change":"2024-09-16T09:31:09.276349+0000","last_active":"2024-09-16T09:31:41.213628+0000","last_peered":"2024-09-16T09:31:41.213628+0000","last_clean":"2024-09-16T09:31:41.213628+0000","last_became_active":"2024-09-16T09:31:09.275136+0000","last_became_peered":"2024-09-16T09:31:09.275136+0000","last_unstale":"2024-09-16T09:31:41.213628+0000","last_undegraded":"2024-09-16T09:31:41.213628+0000","last_fullsized":"2024-09-16T09:31:41.213628+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:30:32.389008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:30:32.389008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:31:56.454141+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.19","version":"66'15","reported_seq":233,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.222859+0000","last_change":"2024-09-16T09:31:09.302903+0000","last_active":"2024-09-16T09:31:41.222859+0000","last_peered":"2024-09-16T09:31:41.222859+0000","last_clean":"2024-09-16T09:31:41.222859+0000","last_became_active":"2024-09-16T09:31:09.268750+0000","last_became_peered":"2024-09-16T09:31:09.268750+0000","last_unstale":"2024-09-16T09:31:41.222859+0000","last_undegraded":"2024-09-16T09:31:41.222859+0000","last_fullsized":"2024-09-16T09:31:41.222859+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-16T09:29:57.968162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:29:57.968162+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":306,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.203354+0000","last_change":"2024-09-16T09:29:15.628676+0000","last_active":"2024-09-16T09:31:41.203354+0000","last_peered":"2024-09-16T09:31:41.203354+0000","last_clean":"2024-09-16T09:31:41.203354+0000","last_became_active":"2024-09-16T09:25:05.717236+0000","last_became_peered":"2024-09-16T09:25:05.717236+0000","last_unstale":"2024-09-16T09:31:41.203354+0000","last_undegraded":"2024-09-16T09:31:41.203354+0000","last_fullsized":"2024-09-16T09:31:41.203354+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:15.628467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:15.628467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":145,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.205064+0000","last_change":"2024-09-16T09:28:34.839909+0000","last_active":"2024-09-16T09:31:41.205064+0000","last_peered":"2024-09-16T09:31:41.205064+0000","last_clean":"2024-09-16T09:31:41.205064+0000","last_became_active":"2024-09-16T09:28:34.839435+0000","last_became_peered":"2024-09-16T09:28:34.839435+0000","last_unstale":"2024-09-16T09:31:41.205064+0000","last_undegraded":"2024-09-16T09:31:41.205064+0000","last_fullsized":"2024-09-16T09:31:41.205064+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:01.038400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:28:01.038400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:09.066525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"66'9","reported_seq":318,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.204160+0000","last_change":"2024-09-16T09:29:28.622647+0000","last_active":"2024-09-16T09:31:41.204160+0000","last_peered":"2024-09-16T09:31:41.204160+0000","last_clean":"2024-09-16T09:31:41.204160+0000","last_became_active":"2024-09-16T09:25:03.709147+0000","last_became_peered":"2024-09-16T09:25:03.709147+0000","last_unstale":"2024-09-16T09:31:41.204160+0000","last_undegraded":"2024-09-16T09:31:41.204160+0000","last_fullsized":"2024-09-16T09:31:41.204160+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:29:28.622558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:29:28.622558+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":289,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.208459+0000","last_change":"2024-09-16T09:27:59.080369+0000","last_active":"2024-09-16T09:31:41.208459+0000","last_peered":"2024-09-16T09:31:41.208459+0000","last_clean":"2024-09-16T09:31:41.208459+0000","last_became_active":"2024-09-16T09:25:05.718134+0000","last_became_peered":"2024-09-16T09:25:05.718134+0000","last_unstale":"2024-09-16T09:31:41.208459+0000","last_undegraded":"2024-09-16T09:31:41.208459+0000","last_fullsized":"2024-09-16T09:31:41.208459+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:59.080307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:59.080307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"66'1","reported_seq":295,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.214120+0000","last_change":"2024-09-16T09:29:15.787990+0000","last_active":"2024-09-16T09:31:41.214120+0000","last_peered":"2024-09-16T09:31:41.214120+0000","last_clean":"2024-09-16T09:31:41.214120+0000","last_became_active":"2024-09-16T09:25:07.716558+0000","last_became_peered":"2024-09-16T09:25:07.716558+0000","last_unstale":"2024-09-16T09:31:41.214120+0000","last_undegraded":"2024-09-16T09:31:41.214120+0000","last_fullsized":"2024-09-16T09:31:41.214120+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-16T09:29:15.787922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:15.787922+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":297,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.208645+0000","last_change":"2024-09-16T09:27:56.191264+0000","last_active":"2024-09-16T09:31:41.208645+0000","last_peered":"2024-09-16T09:31:41.208645+0000","last_clean":"2024-09-16T09:31:41.208645+0000","last_became_active":"2024-09-16T09:25:01.696939+0000","last_became_peered":"2024-09-16T09:25:01.696939+0000","last_unstale":"2024-09-16T09:31:41.208645+0000","last_undegraded":"2024-09-16T09:31:41.208645+0000","last_fullsized":"2024-09-16T09:31:41.208645+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:56.191220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:27:56.191220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"66'5","reported_seq":304,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.206392+0000","last_change":"2024-09-16T09:27:05.694733+0000","last_active":"2024-09-16T09:31:41.206392+0000","last_peered":"2024-09-16T09:31:41.206392+0000","last_clean":"2024-09-16T09:31:41.206392+0000","last_became_active":"2024-09-16T09:25:03.716136+0000","last_became_peered":"2024-09-16T09:25:03.716136+0000","last_unstale":"2024-09-16T09:31:41.206392+0000","last_undegraded":"2024-09-16T09:31:41.206392+0000","last_fullsized":"2024-09-16T09:31:41.206392+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-16T09:27:05.694650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:05.694650+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":297,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.217885+0000","last_change":"2024-09-16T09:28:52.746046+0000","last_active":"2024-09-16T09:31:41.217885+0000","last_peered":"2024-09-16T09:31:41.217885+0000","last_clean":"2024-09-16T09:31:41.217885+0000","last_became_active":"2024-09-16T09:25:05.718281+0000","last_became_peered":"2024-09-16T09:25:05.718281+0000","last_unstale":"2024-09-16T09:31:41.217885+0000","last_undegraded":"2024-09-16T09:31:41.217885+0000","last_fullsized":"2024-09-16T09:31:41.217885+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:52.745999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:28:52.745999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":315,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.210003+0000","last_change":"2024-09-16T09:31:09.276016+0000","last_active":"2024-09-16T09:31:41.210003+0000","last_peered":"2024-09-16T09:31:41.210003+0000","last_clean":"2024-09-16T09:31:41.210003+0000","last_became_active":"2024-09-16T09:31:09.275656+0000","last_became_peered":"2024-09-16T09:31:09.275656+0000","last_unstale":"2024-09-16T09:31:41.210003+0000","last_undegraded":"2024-09-16T09:31:41.210003+0000","last_fullsized":"2024-09-16T09:31:41.210003+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:50.468134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:50.468134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":303,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.204079+0000","last_change":"2024-09-16T09:29:56.605555+0000","last_active":"2024-09-16T09:31:41.204079+0000","last_peered":"2024-09-16T09:31:41.204079+0000","last_clean":"2024-09-16T09:31:41.204079+0000","last_became_active":"2024-09-16T09:25:07.720793+0000","last_became_peered":"2024-09-16T09:25:07.720793+0000","last_unstale":"2024-09-16T09:31:41.204079+0000","last_undegraded":"2024-09-16T09:31:41.204079+0000","last_fullsized":"2024-09-16T09:31:41.204079+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:56.605488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:56.605488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":54,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.212379+0000","last_change":"2024-09-16T09:30:39.202178+0000","last_active":"2024-09-16T09:31:41.212379+0000","last_peered":"2024-09-16T09:31:41.212379+0000","last_clean":"2024-09-16T09:31:41.212379+0000","last_became_active":"2024-09-16T09:30:39.201824+0000","last_became_peered":"2024-09-16T09:30:39.201824+0000","last_unstale":"2024-09-16T09:31:41.212379+0000","last_undegraded":"2024-09-16T09:31:41.212379+0000","last_fullsized":"2024-09-16T09:31:41.212379+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:50.912979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:50.912979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:20.431823+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"66'15","reported_seq":381,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.220769+0000","last_change":"2024-09-16T09:31:09.302018+0000","last_active":"2024-09-16T09:31:41.220769+0000","last_peered":"2024-09-16T09:31:41.220769+0000","last_clean":"2024-09-16T09:31:41.220769+0000","last_became_active":"2024-09-16T09:31:09.263534+0000","last_became_peered":"2024-09-16T09:31:09.263534+0000","last_unstale":"2024-09-16T09:31:41.220769+0000","last_undegraded":"2024-09-16T09:31:41.220769+0000","last_fullsized":"2024-09-16T09:31:41.220769+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-16T09:30:21.932604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:30:21.932604+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":298,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.205397+0000","last_change":"2024-09-16T09:29:55.875536+0000","last_active":"2024-09-16T09:31:41.205397+0000","last_peered":"2024-09-16T09:31:41.205397+0000","last_clean":"2024-09-16T09:31:41.205397+0000","last_became_active":"2024-09-16T09:25:05.713929+0000","last_became_peered":"2024-09-16T09:25:05.713929+0000","last_unstale":"2024-09-16T09:31:41.205397+0000","last_undegraded":"2024-09-16T09:31:41.205397+0000","last_fullsized":"2024-09-16T09:31:41.205397+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:55.875480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:55.875480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":151,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.213444+0000","last_change":"2024-09-16T09:30:24.412733+0000","last_active":"2024-09-16T09:31:41.213444+0000","last_peered":"2024-09-16T09:31:41.213444+0000","last_clean":"2024-09-16T09:31:41.213444+0000","last_became_active":"2024-09-16T09:28:52.974255+0000","last_became_peered":"2024-09-16T09:28:52.974255+0000","last_unstale":"2024-09-16T09:31:41.213444+0000","last_undegraded":"2024-09-16T09:31:41.213444+0000","last_fullsized":"2024-09-16T09:31:41.213444+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:30:24.412571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:30:24.412571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":314,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.204285+0000","last_change":"2024-09-16T09:30:03.628882+0000","last_active":"2024-09-16T09:31:41.204285+0000","last_peered":"2024-09-16T09:31:41.204285+0000","last_clean":"2024-09-16T09:31:41.204285+0000","last_became_active":"2024-09-16T09:25:01.706491+0000","last_became_peered":"2024-09-16T09:25:01.706491+0000","last_unstale":"2024-09-16T09:31:41.204285+0000","last_undegraded":"2024-09-16T09:31:41.204285+0000","last_fullsized":"2024-09-16T09:31:41.204285+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:30:03.628812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:30:03.628812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"66'12","reported_seq":215,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.205624+0000","last_change":"2024-09-16T09:31:09.286673+0000","last_active":"2024-09-16T09:31:41.205624+0000","last_peered":"2024-09-16T09:31:41.205624+0000","last_clean":"2024-09-16T09:31:41.205624+0000","last_became_active":"2024-09-16T09:31:09.268588+0000","last_became_peered":"2024-09-16T09:31:09.268588+0000","last_unstale":"2024-09-16T09:31:41.205624+0000","last_undegraded":"2024-09-16T09:31:41.205624+0000","last_fullsized":"2024-09-16T09:31:41.205624+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-16T09:27:01.700139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:01.700139+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":24,"num_read_kb":15,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":290,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.206139+0000","last_change":"2024-09-16T09:27:00.816500+0000","last_active":"2024-09-16T09:31:41.206139+0000","last_peered":"2024-09-16T09:31:41.206139+0000","last_clean":"2024-09-16T09:31:41.206139+0000","last_became_active":"2024-09-16T09:25:05.710093+0000","last_became_peered":"2024-09-16T09:25:05.710093+0000","last_unstale":"2024-09-16T09:31:41.206139+0000","last_undegraded":"2024-09-16T09:31:41.206139+0000","last_fullsized":"2024-09-16T09:31:41.206139+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:00.816424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:00.816424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":294,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.218243+0000","last_change":"2024-09-16T09:29:29.918433+0000","last_active":"2024-09-16T09:31:41.218243+0000","last_peered":"2024-09-16T09:31:41.218243+0000","last_clean":"2024-09-16T09:31:41.218243+0000","last_became_active":"2024-09-16T09:25:07.719380+0000","last_became_peered":"2024-09-16T09:25:07.719380+0000","last_unstale":"2024-09-16T09:31:41.218243+0000","last_undegraded":"2024-09-16T09:31:41.218243+0000","last_fullsized":"2024-09-16T09:31:41.218243+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-16T09:29:29.918368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:29.918368+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":315,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.205998+0000","last_change":"2024-09-16T09:30:02.843932+0000","last_active":"2024-09-16T09:31:41.205998+0000","last_peered":"2024-09-16T09:31:41.205998+0000","last_clean":"2024-09-16T09:31:41.205998+0000","last_became_active":"2024-09-16T09:25:01.699545+0000","last_became_peered":"2024-09-16T09:25:01.699545+0000","last_unstale":"2024-09-16T09:31:41.205998+0000","last_undegraded":"2024-09-16T09:31:41.205998+0000","last_fullsized":"2024-09-16T09:31:41.205998+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:30:02.843856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:30:02.843856+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"66'10","reported_seq":356,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.219749+0000","last_change":"2024-09-16T09:31:09.328513+0000","last_active":"2024-09-16T09:31:41.219749+0000","last_peered":"2024-09-16T09:31:41.219749+0000","last_clean":"2024-09-16T09:31:41.219749+0000","last_became_active":"2024-09-16T09:31:09.275050+0000","last_became_peered":"2024-09-16T09:31:09.275050+0000","last_unstale":"2024-09-16T09:31:41.219749+0000","last_undegraded":"2024-09-16T09:31:41.219749+0000","last_fullsized":"2024-09-16T09:31:41.219749+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-16T09:27:43.556357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:43.556357+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:15.194576+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"66'8","reported_seq":407,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.213631+0000","last_change":"2024-09-16T09:30:31.392771+0000","last_active":"2024-09-16T09:31:41.213631+0000","last_peered":"2024-09-16T09:31:41.213631+0000","last_clean":"2024-09-16T09:31:41.213631+0000","last_became_active":"2024-09-16T09:25:05.712516+0000","last_became_peered":"2024-09-16T09:25:05.712516+0000","last_unstale":"2024-09-16T09:31:41.213631+0000","last_undegraded":"2024-09-16T09:31:41.213631+0000","last_fullsized":"2024-09-16T09:31:41.213631+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:30:31.392399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:30:31.392399+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"197'8979","reported_seq":9078,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:42.848602+0000","last_change":"2024-09-16T09:29:49.449011+0000","last_active":"2024-09-16T09:31:42.848602+0000","last_peered":"2024-09-16T09:31:42.848602+0000","last_clean":"2024-09-16T09:31:42.848602+0000","last_became_active":"2024-09-16T09:29:49.448473+0000","last_became_peered":"2024-09-16T09:29:49.448473+0000","last_unstale":"2024-09-16T09:31:42.848602+0000","last_undegraded":"2024-09-16T09:31:42.848602+0000","last_fullsized":"2024-09-16T09:31:42.848602+0000","mapping_epoch":152,"log_start":"174'4500","ondisk_log_start":"174'4500","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":4479,"log_dups_size":0,"ondisk_log_size":4479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":435093504,"num_objects":6639,"num_object_clones":0,"num_object_copies":19917,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6639,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8979,"num_write_kb":499776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.a","version":"73'79","reported_seq":237,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.208212+0000","last_change":"2024-09-16T09:29:18.498887+0000","last_active":"2024-09-16T09:31:41.208212+0000","last_peered":"2024-09-16T09:31:41.208212+0000","last_clean":"2024-09-16T09:31:41.208212+0000","last_became_active":"2024-09-16T09:26:35.127352+0000","last_became_peered":"2024-09-16T09:26:35.127352+0000","last_unstale":"2024-09-16T09:31:41.208212+0000","last_undegraded":"2024-09-16T09:31:41.208212+0000","last_fullsized":"2024-09-16T09:31:41.208212+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:18.498766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:18.498766+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"66'15","reported_seq":246,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.226520+0000","last_change":"2024-09-16T09:28:28.065781+0000","last_active":"2024-09-16T09:31:41.226520+0000","last_peered":"2024-09-16T09:31:41.226520+0000","last_clean":"2024-09-16T09:31:41.226520+0000","last_became_active":"2024-09-16T09:26:33.295672+0000","last_became_peered":"2024-09-16T09:26:33.295672+0000","last_unstale":"2024-09-16T09:31:41.226520+0000","last_undegraded":"2024-09-16T09:31:41.226520+0000","last_fullsized":"2024-09-16T09:31:41.226520+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-16T09:28:28.065699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:28.065699+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":32,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.210765+0000","last_change":"2024-09-16T09:31:09.276858+0000","last_active":"2024-09-16T09:31:41.210765+0000","last_peered":"2024-09-16T09:31:41.210765+0000","last_clean":"2024-09-16T09:31:41.210765+0000","last_became_active":"2024-09-16T09:31:09.274534+0000","last_became_peered":"2024-09-16T09:31:09.274534+0000","last_unstale":"2024-09-16T09:31:41.210765+0000","last_undegraded":"2024-09-16T09:31:41.210765+0000","last_fullsized":"2024-09-16T09:31:41.210765+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:22.091537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:28:22.091537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"66'8","reported_seq":371,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.204253+0000","last_change":"2024-09-16T09:25:05.714128+0000","last_active":"2024-09-16T09:31:41.204253+0000","last_peered":"2024-09-16T09:31:41.204253+0000","last_clean":"2024-09-16T09:31:41.204253+0000","last_became_active":"2024-09-16T09:25:05.713813+0000","last_became_peered":"2024-09-16T09:25:05.713813+0000","last_unstale":"2024-09-16T09:31:41.204253+0000","last_undegraded":"2024-09-16T09:31:41.204253+0000","last_fullsized":"2024-09-16T09:31:41.204253+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":293,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.208576+0000","last_change":"2024-09-16T09:29:24.537802+0000","last_active":"2024-09-16T09:31:41.208576+0000","last_peered":"2024-09-16T09:31:41.208576+0000","last_clean":"2024-09-16T09:31:41.208576+0000","last_became_active":"2024-09-16T09:25:07.717344+0000","last_became_peered":"2024-09-16T09:25:07.717344+0000","last_unstale":"2024-09-16T09:31:41.208576+0000","last_undegraded":"2024-09-16T09:31:41.208576+0000","last_fullsized":"2024-09-16T09:31:41.208576+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:24.537731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:24.537731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.c","version":"197'8930","reported_seq":9031,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:43.105672+0000","last_change":"2024-09-16T09:29:50.008199+0000","last_active":"2024-09-16T09:31:43.105672+0000","last_peered":"2024-09-16T09:31:43.105672+0000","last_clean":"2024-09-16T09:31:43.105672+0000","last_became_active":"2024-09-16T09:29:50.007935+0000","last_became_peered":"2024-09-16T09:29:50.007935+0000","last_unstale":"2024-09-16T09:31:43.105672+0000","last_undegraded":"2024-09-16T09:31:43.105672+0000","last_fullsized":"2024-09-16T09:31:43.105672+0000","mapping_epoch":152,"log_start":"172'3800","ondisk_log_start":"172'3800","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":5130,"log_dups_size":0,"ondisk_log_size":5130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":433717248,"num_objects":6618,"num_object_clones":0,"num_object_copies":19854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6618,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8930,"num_write_kb":497536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"73'79","reported_seq":266,"reported_epoch":197,"state":"active+clean+scrubbing","last_fresh":"2024-09-16T09:31:41.825134+0000","last_change":"2024-09-16T09:31:41.825014+0000","last_active":"2024-09-16T09:31:41.825134+0000","last_peered":"2024-09-16T09:31:41.825134+0000","last_clean":"2024-09-16T09:31:41.825134+0000","last_became_active":"2024-09-16T09:30:08.763683+0000","last_became_peered":"2024-09-16T09:30:08.763683+0000","last_unstale":"2024-09-16T09:31:41.825134+0000","last_undegraded":"2024-09-16T09:31:41.825134+0000","last_fullsized":"2024-09-16T09:31:41.825134+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:58.568841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:58.568841+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"scrubbing for 0s","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"66'18","reported_seq":244,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.217724+0000","last_change":"2024-09-16T09:31:09.407700+0000","last_active":"2024-09-16T09:31:41.217724+0000","last_peered":"2024-09-16T09:31:41.217724+0000","last_clean":"2024-09-16T09:31:41.217724+0000","last_became_active":"2024-09-16T09:31:09.270434+0000","last_became_peered":"2024-09-16T09:31:09.270434+0000","last_unstale":"2024-09-16T09:31:41.217724+0000","last_undegraded":"2024-09-16T09:31:41.217724+0000","last_fullsized":"2024-09-16T09:31:41.217724+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'18","last_scrub_stamp":"2024-09-16T09:28:54.699778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:54.699778+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":33,"num_read_kb":21,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":142,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.220344+0000","last_change":"2024-09-16T09:28:58.705169+0000","last_active":"2024-09-16T09:31:41.220344+0000","last_peered":"2024-09-16T09:31:41.220344+0000","last_clean":"2024-09-16T09:31:41.220344+0000","last_became_active":"2024-09-16T09:28:52.983543+0000","last_became_peered":"2024-09-16T09:28:52.983543+0000","last_unstale":"2024-09-16T09:31:41.220344+0000","last_undegraded":"2024-09-16T09:31:41.220344+0000","last_fullsized":"2024-09-16T09:31:41.220344+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:58.705097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:28:58.705097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":321,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.203577+0000","last_change":"2024-09-16T09:31:09.269234+0000","last_active":"2024-09-16T09:31:41.203577+0000","last_peered":"2024-09-16T09:31:41.203577+0000","last_clean":"2024-09-16T09:31:41.203577+0000","last_became_active":"2024-09-16T09:31:09.268864+0000","last_became_peered":"2024-09-16T09:31:09.268864+0000","last_unstale":"2024-09-16T09:31:41.203577+0000","last_undegraded":"2024-09-16T09:31:41.203577+0000","last_fullsized":"2024-09-16T09:31:41.203577+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:50.482240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:50.482240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":301,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.204077+0000","last_change":"2024-09-16T09:31:09.269842+0000","last_active":"2024-09-16T09:31:41.204077+0000","last_peered":"2024-09-16T09:31:41.204077+0000","last_clean":"2024-09-16T09:31:41.204077+0000","last_became_active":"2024-09-16T09:31:09.269433+0000","last_became_peered":"2024-09-16T09:31:09.269433+0000","last_unstale":"2024-09-16T09:31:41.204077+0000","last_undegraded":"2024-09-16T09:31:41.204077+0000","last_fullsized":"2024-09-16T09:31:41.204077+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:13.548608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:13.548608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.d","version":"197'8921","reported_seq":9023,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:42.863970+0000","last_change":"2024-09-16T09:29:50.010860+0000","last_active":"2024-09-16T09:31:42.863970+0000","last_peered":"2024-09-16T09:31:42.863970+0000","last_clean":"2024-09-16T09:31:42.863970+0000","last_became_active":"2024-09-16T09:29:50.010421+0000","last_became_peered":"2024-09-16T09:29:50.010421+0000","last_unstale":"2024-09-16T09:31:42.863970+0000","last_undegraded":"2024-09-16T09:31:42.863970+0000","last_fullsized":"2024-09-16T09:31:42.863970+0000","mapping_epoch":152,"log_start":"174'4200","ondisk_log_start":"174'4200","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":4721,"log_dups_size":0,"ondisk_log_size":4721,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":435421208,"num_objects":6645,"num_object_clones":0,"num_object_copies":19935,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6645,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8921,"num_write_kb":498049,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,2],"acting":[5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.4","version":"73'79","reported_seq":32,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.211115+0000","last_change":"2024-09-16T09:31:09.275499+0000","last_active":"2024-09-16T09:31:41.211115+0000","last_peered":"2024-09-16T09:31:41.211115+0000","last_clean":"2024-09-16T09:31:41.211115+0000","last_became_active":"2024-09-16T09:31:09.275139+0000","last_became_peered":"2024-09-16T09:31:09.275139+0000","last_unstale":"2024-09-16T09:31:41.211115+0000","last_undegraded":"2024-09-16T09:31:41.211115+0000","last_fullsized":"2024-09-16T09:31:41.211115+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:30:22.438378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:30:22.438378+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"66'14","reported_seq":324,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.203668+0000","last_change":"2024-09-16T09:28:32.595064+0000","last_active":"2024-09-16T09:31:41.203668+0000","last_peered":"2024-09-16T09:31:41.203668+0000","last_clean":"2024-09-16T09:31:41.203668+0000","last_became_active":"2024-09-16T09:25:03.708830+0000","last_became_peered":"2024-09-16T09:25:03.708830+0000","last_unstale":"2024-09-16T09:31:41.203668+0000","last_undegraded":"2024-09-16T09:31:41.203668+0000","last_fullsized":"2024-09-16T09:31:41.203668+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'14","last_scrub_stamp":"2024-09-16T09:28:32.594988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:32.594988+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":316,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.219991+0000","last_change":"2024-09-16T09:28:58.602786+0000","last_active":"2024-09-16T09:31:41.219991+0000","last_peered":"2024-09-16T09:31:41.219991+0000","last_clean":"2024-09-16T09:31:41.219991+0000","last_became_active":"2024-09-16T09:25:01.702992+0000","last_became_peered":"2024-09-16T09:25:01.702992+0000","last_unstale":"2024-09-16T09:31:41.219991+0000","last_undegraded":"2024-09-16T09:31:41.219991+0000","last_fullsized":"2024-09-16T09:31:41.219991+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:28:58.602717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:28:58.602717+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"187'11","reported_seq":224,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.204950+0000","last_change":"2024-09-16T09:31:09.397758+0000","last_active":"2024-09-16T09:31:41.204950+0000","last_peered":"2024-09-16T09:31:41.204950+0000","last_clean":"2024-09-16T09:31:41.204950+0000","last_became_active":"2024-09-16T09:31:09.269173+0000","last_became_peered":"2024-09-16T09:31:09.269173+0000","last_unstale":"2024-09-16T09:31:41.204950+0000","last_undegraded":"2024-09-16T09:31:41.204950+0000","last_fullsized":"2024-09-16T09:31:41.204950+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'9","last_scrub_stamp":"2024-09-16T09:29:21.898533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:21.898533+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":286,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.217827+0000","last_change":"2024-09-16T09:26:40.699888+0000","last_active":"2024-09-16T09:31:41.217827+0000","last_peered":"2024-09-16T09:31:41.217827+0000","last_clean":"2024-09-16T09:31:41.217827+0000","last_became_active":"2024-09-16T09:25:07.722017+0000","last_became_peered":"2024-09-16T09:25:07.722017+0000","last_unstale":"2024-09-16T09:31:41.217827+0000","last_undegraded":"2024-09-16T09:31:41.217827+0000","last_fullsized":"2024-09-16T09:31:41.217827+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:40.699813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:40.699813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.e","version":"197'8988","reported_seq":9089,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:43.129630+0000","last_change":"2024-09-16T09:29:49.448081+0000","last_active":"2024-09-16T09:31:43.129630+0000","last_peered":"2024-09-16T09:31:43.129630+0000","last_clean":"2024-09-16T09:31:43.129630+0000","last_became_active":"2024-09-16T09:29:49.447767+0000","last_became_peered":"2024-09-16T09:29:49.447767+0000","last_unstale":"2024-09-16T09:31:43.129630+0000","last_undegraded":"2024-09-16T09:31:43.129630+0000","last_fullsized":"2024-09-16T09:31:43.129630+0000","mapping_epoch":152,"log_start":"172'3900","ondisk_log_start":"172'3900","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":5088,"log_dups_size":0,"ondisk_log_size":5088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":433717248,"num_objects":6618,"num_object_clones":0,"num_object_copies":19854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6618,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8988,"num_write_kb":499392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"73'79","reported_seq":355,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.213441+0000","last_change":"2024-09-16T09:31:09.275413+0000","last_active":"2024-09-16T09:31:41.213441+0000","last_peered":"2024-09-16T09:31:41.213441+0000","last_clean":"2024-09-16T09:31:41.213441+0000","last_became_active":"2024-09-16T09:31:09.274707+0000","last_became_peered":"2024-09-16T09:31:09.274707+0000","last_unstale":"2024-09-16T09:31:41.213441+0000","last_undegraded":"2024-09-16T09:31:41.213441+0000","last_fullsized":"2024-09-16T09:31:41.213441+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:30:23.455388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:30:23.455388+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"66'10","reported_seq":326,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.204838+0000","last_change":"2024-09-16T09:30:04.848306+0000","last_active":"2024-09-16T09:31:41.204838+0000","last_peered":"2024-09-16T09:31:41.204838+0000","last_clean":"2024-09-16T09:31:41.204838+0000","last_became_active":"2024-09-16T09:25:03.712985+0000","last_became_peered":"2024-09-16T09:25:03.712985+0000","last_unstale":"2024-09-16T09:31:41.204838+0000","last_undegraded":"2024-09-16T09:31:41.204838+0000","last_fullsized":"2024-09-16T09:31:41.204838+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-16T09:30:04.848234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:30:04.848234+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":312,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.208894+0000","last_change":"2024-09-16T09:31:09.272049+0000","last_active":"2024-09-16T09:31:41.208894+0000","last_peered":"2024-09-16T09:31:41.208894+0000","last_clean":"2024-09-16T09:31:41.208894+0000","last_became_active":"2024-09-16T09:31:09.270619+0000","last_became_peered":"2024-09-16T09:31:09.270619+0000","last_unstale":"2024-09-16T09:31:41.208894+0000","last_undegraded":"2024-09-16T09:31:41.208894+0000","last_fullsized":"2024-09-16T09:31:41.208894+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:25.086322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:25.086322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"66'8","reported_seq":388,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.214863+0000","last_change":"2024-09-16T09:28:57.628615+0000","last_active":"2024-09-16T09:31:41.214863+0000","last_peered":"2024-09-16T09:31:41.214863+0000","last_clean":"2024-09-16T09:31:41.214863+0000","last_became_active":"2024-09-16T09:25:05.714507+0000","last_became_peered":"2024-09-16T09:25:05.714507+0000","last_unstale":"2024-09-16T09:31:41.214863+0000","last_undegraded":"2024-09-16T09:31:41.214863+0000","last_fullsized":"2024-09-16T09:31:41.214863+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:28:57.628543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:28:57.628543+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":309,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.214268+0000","last_change":"2024-09-16T09:31:09.283816+0000","last_active":"2024-09-16T09:31:41.214268+0000","last_peered":"2024-09-16T09:31:41.214268+0000","last_clean":"2024-09-16T09:31:41.214268+0000","last_became_active":"2024-09-16T09:31:09.283192+0000","last_became_peered":"2024-09-16T09:31:09.283192+0000","last_unstale":"2024-09-16T09:31:41.214268+0000","last_undegraded":"2024-09-16T09:31:41.214268+0000","last_fullsized":"2024-09-16T09:31:41.214268+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:22.563395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:28:22.563395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:22.152175+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.f","version":"197'8998","reported_seq":9099,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:42.797444+0000","last_change":"2024-09-16T09:29:49.838498+0000","last_active":"2024-09-16T09:31:42.797444+0000","last_peered":"2024-09-16T09:31:42.797444+0000","last_clean":"2024-09-16T09:31:42.797444+0000","last_became_active":"2024-09-16T09:29:49.838146+0000","last_became_peered":"2024-09-16T09:29:49.838146+0000","last_unstale":"2024-09-16T09:31:42.797444+0000","last_undegraded":"2024-09-16T09:31:42.797444+0000","last_fullsized":"2024-09-16T09:31:42.797444+0000","mapping_epoch":152,"log_start":"165'3100","ondisk_log_start":"165'3100","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":5898,"log_dups_size":0,"ondisk_log_size":5898,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":442236928,"num_objects":6748,"num_object_clones":0,"num_object_copies":20244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6748,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8998,"num_write_kb":503872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"73'79","reported_seq":336,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.213475+0000","last_change":"2024-09-16T09:31:09.274975+0000","last_active":"2024-09-16T09:31:41.213475+0000","last_peered":"2024-09-16T09:31:41.213475+0000","last_clean":"2024-09-16T09:31:41.213475+0000","last_became_active":"2024-09-16T09:31:09.274565+0000","last_became_peered":"2024-09-16T09:31:09.274565+0000","last_unstale":"2024-09-16T09:31:41.213475+0000","last_undegraded":"2024-09-16T09:31:41.213475+0000","last_fullsized":"2024-09-16T09:31:41.213475+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:30:22.415051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:30:22.415051+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:31:41.553155+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"66'19","reported_seq":339,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.215054+0000","last_change":"2024-09-16T09:31:35.249952+0000","last_active":"2024-09-16T09:31:41.215054+0000","last_peered":"2024-09-16T09:31:41.215054+0000","last_clean":"2024-09-16T09:31:41.215054+0000","last_became_active":"2024-09-16T09:25:03.714797+0000","last_became_peered":"2024-09-16T09:25:03.714797+0000","last_unstale":"2024-09-16T09:31:41.215054+0000","last_undegraded":"2024-09-16T09:31:41.215054+0000","last_fullsized":"2024-09-16T09:31:41.215054+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-16T09:31:35.249864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:31:35.249864+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:40.632025+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":315,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.204968+0000","last_change":"2024-09-16T09:29:16.888357+0000","last_active":"2024-09-16T09:31:41.204968+0000","last_peered":"2024-09-16T09:31:41.204968+0000","last_clean":"2024-09-16T09:31:41.204968+0000","last_became_active":"2024-09-16T09:25:01.706806+0000","last_became_peered":"2024-09-16T09:25:01.706806+0000","last_unstale":"2024-09-16T09:31:41.204968+0000","last_undegraded":"2024-09-16T09:31:41.204968+0000","last_fullsized":"2024-09-16T09:31:41.204968+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:16.888236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:16.888236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":307,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.212534+0000","last_change":"2024-09-16T09:30:51.409917+0000","last_active":"2024-09-16T09:31:41.212534+0000","last_peered":"2024-09-16T09:31:41.212534+0000","last_clean":"2024-09-16T09:31:41.212534+0000","last_became_active":"2024-09-16T09:25:05.708663+0000","last_became_peered":"2024-09-16T09:25:05.708663+0000","last_unstale":"2024-09-16T09:31:41.212534+0000","last_undegraded":"2024-09-16T09:31:41.212534+0000","last_fullsized":"2024-09-16T09:31:41.212534+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:30:51.409834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:30:51.409834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:31:55.112422+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":287,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.205788+0000","last_change":"2024-09-16T09:26:37.282798+0000","last_active":"2024-09-16T09:31:41.205788+0000","last_peered":"2024-09-16T09:31:41.205788+0000","last_clean":"2024-09-16T09:31:41.205788+0000","last_became_active":"2024-09-16T09:25:07.719267+0000","last_became_peered":"2024-09-16T09:25:07.719267+0000","last_unstale":"2024-09-16T09:31:41.205788+0000","last_undegraded":"2024-09-16T09:31:41.205788+0000","last_fullsized":"2024-09-16T09:31:41.205788+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:37.282735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:37.282735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.8","version":"197'9062","reported_seq":12850,"reported_epoch":197,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-16T09:31:42.865637+0000","last_change":"2024-09-16T09:31:09.438251+0000","last_active":"2024-09-16T09:31:42.865637+0000","last_peered":"2024-09-16T09:31:42.865637+0000","last_clean":"2024-09-16T09:30:20.474651+0000","last_became_active":"2024-09-16T09:31:09.409873+0000","last_became_peered":"2024-09-16T09:31:09.409873+0000","last_unstale":"2024-09-16T09:31:42.865637+0000","last_undegraded":"2024-09-16T09:31:42.865637+0000","last_fullsized":"2024-09-16T09:31:09.256524+0000","mapping_epoch":186,"log_start":"174'4300","ondisk_log_start":"174'4300","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":4762,"log_dups_size":0,"ondisk_log_size":4762,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436469760,"num_objects":6660,"num_object_clones":0,"num_object_copies":19980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2618,"num_objects_unfound":0,"num_objects_dirty":6660,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9062,"num_write_kb":503104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1838,"num_bytes_recovered":120455168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,7],"avail_no_missing":["5","7"],"object_location_counts":[{"shards":"0,5,7","objects":2618},{"shards":"5,7","objects":4042}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1","version":"73'79","reported_seq":258,"reported_epoch":197,"state":"active+clean+scrubbing","last_fresh":"2024-09-16T09:31:42.386918+0000","last_change":"2024-09-16T09:31:42.386773+0000","last_active":"2024-09-16T09:31:42.386918+0000","last_peered":"2024-09-16T09:31:42.386918+0000","last_clean":"2024-09-16T09:31:42.386918+0000","last_became_active":"2024-09-16T09:30:08.766350+0000","last_became_peered":"2024-09-16T09:30:08.766350+0000","last_unstale":"2024-09-16T09:31:42.386918+0000","last_undegraded":"2024-09-16T09:31:42.386918+0000","last_fullsized":"2024-09-16T09:31:42.386918+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:15.460179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:15.460179+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"scrubbing for 0s","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"187'30","reported_seq":490,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.206664+0000","last_change":"2024-09-16T09:31:09.323281+0000","last_active":"2024-09-16T09:31:41.206664+0000","last_peered":"2024-09-16T09:31:41.206664+0000","last_clean":"2024-09-16T09:31:41.206664+0000","last_became_active":"2024-09-16T09:31:09.278839+0000","last_became_peered":"2024-09-16T09:31:09.278839+0000","last_unstale":"2024-09-16T09:31:41.206664+0000","last_undegraded":"2024-09-16T09:31:41.206664+0000","last_fullsized":"2024-09-16T09:31:41.206664+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'28","last_scrub_stamp":"2024-09-16T09:29:18.937623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:29:18.937623+0000","objects_scrubbed":10,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":307,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.207224+0000","last_change":"2024-09-16T09:29:26.637070+0000","last_active":"2024-09-16T09:31:41.207224+0000","last_peered":"2024-09-16T09:31:41.207224+0000","last_clean":"2024-09-16T09:31:41.207224+0000","last_became_active":"2024-09-16T09:25:01.703076+0000","last_became_peered":"2024-09-16T09:25:01.703076+0000","last_unstale":"2024-09-16T09:31:41.207224+0000","last_undegraded":"2024-09-16T09:31:41.207224+0000","last_fullsized":"2024-09-16T09:31:41.207224+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:26.637027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:26.637027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":316,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.208834+0000","last_change":"2024-09-16T09:29:29.486284+0000","last_active":"2024-09-16T09:31:41.208834+0000","last_peered":"2024-09-16T09:31:41.208834+0000","last_clean":"2024-09-16T09:31:41.208834+0000","last_became_active":"2024-09-16T09:25:00.696237+0000","last_became_peered":"2024-09-16T09:25:00.696237+0000","last_unstale":"2024-09-16T09:31:41.208834+0000","last_undegraded":"2024-09-16T09:31:41.208834+0000","last_fullsized":"2024-09-16T09:31:41.208834+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-16T09:29:29.486188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:29:29.486188+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.0025137430000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":283,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.215601+0000","last_change":"2024-09-16T09:25:05.718319+0000","last_active":"2024-09-16T09:31:41.215601+0000","last_peered":"2024-09-16T09:31:41.215601+0000","last_clean":"2024-09-16T09:31:41.215601+0000","last_became_active":"2024-09-16T09:25:05.717591+0000","last_became_peered":"2024-09-16T09:25:05.717591+0000","last_unstale":"2024-09-16T09:31:41.215601+0000","last_undegraded":"2024-09-16T09:31:41.215601+0000","last_fullsized":"2024-09-16T09:31:41.215601+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:10.167420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":32,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.212025+0000","last_change":"2024-09-16T09:31:09.272967+0000","last_active":"2024-09-16T09:31:41.212025+0000","last_peered":"2024-09-16T09:31:41.212025+0000","last_clean":"2024-09-16T09:31:41.212025+0000","last_became_active":"2024-09-16T09:31:09.272609+0000","last_became_peered":"2024-09-16T09:31:09.272609+0000","last_unstale":"2024-09-16T09:31:41.212025+0000","last_undegraded":"2024-09-16T09:31:41.212025+0000","last_fullsized":"2024-09-16T09:31:41.212025+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:45.201503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:45.201503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"197'8915","reported_seq":16422,"reported_epoch":197,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-16T09:31:42.773488+0000","last_change":"2024-09-16T09:31:09.423422+0000","last_active":"2024-09-16T09:31:42.773488+0000","last_peered":"2024-09-16T09:31:42.773488+0000","last_clean":"2024-09-16T09:30:20.492754+0000","last_became_active":"2024-09-16T09:31:09.398447+0000","last_became_peered":"2024-09-16T09:31:09.398447+0000","last_unstale":"2024-09-16T09:31:42.773488+0000","last_undegraded":"2024-09-16T09:31:42.773488+0000","last_fullsized":"2024-09-16T09:31:09.253245+0000","mapping_epoch":186,"log_start":"177'4600","ondisk_log_start":"177'4600","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":4315,"log_dups_size":0,"ondisk_log_size":4315,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":433782784,"num_objects":6619,"num_object_clones":0,"num_object_copies":19857,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1853,"num_objects_unfound":0,"num_objects_dirty":6619,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8915,"num_write_kb":497088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3698,"num_bytes_recovered":242352128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,1],"avail_no_missing":["4","1"],"object_location_counts":[{"shards":"1,2,4","objects":1853},{"shards":"1,4","objects":4766}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.2","version":"73'79","reported_seq":383,"reported_epoch":197,"state":"active+clean+scrubbing","last_fresh":"2024-09-16T09:31:41.475451+0000","last_change":"2024-09-16T09:31:41.475268+0000","last_active":"2024-09-16T09:31:41.475451+0000","last_peered":"2024-09-16T09:31:41.475451+0000","last_clean":"2024-09-16T09:31:41.475451+0000","last_became_active":"2024-09-16T09:30:08.765943+0000","last_became_peered":"2024-09-16T09:30:08.765943+0000","last_unstale":"2024-09-16T09:31:41.475451+0000","last_undegraded":"2024-09-16T09:31:41.475451+0000","last_fullsized":"2024-09-16T09:31:41.475451+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:26:35.745837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:26:35.745837+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"scrubbing for 0s","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"66'13","reported_seq":330,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.205390+0000","last_change":"2024-09-16T09:27:44.156057+0000","last_active":"2024-09-16T09:31:41.205390+0000","last_peered":"2024-09-16T09:31:41.205390+0000","last_clean":"2024-09-16T09:31:41.205390+0000","last_became_active":"2024-09-16T09:25:03.713241+0000","last_became_peered":"2024-09-16T09:25:03.713241+0000","last_unstale":"2024-09-16T09:31:41.205390+0000","last_undegraded":"2024-09-16T09:31:41.205390+0000","last_fullsized":"2024-09-16T09:31:41.205390+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'13","last_scrub_stamp":"2024-09-16T09:27:44.155969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:44.155969+0000","objects_scrubbed":5,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":29,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":307,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.205331+0000","last_change":"2024-09-16T09:29:50.827103+0000","last_active":"2024-09-16T09:31:41.205331+0000","last_peered":"2024-09-16T09:31:41.205331+0000","last_clean":"2024-09-16T09:31:41.205331+0000","last_became_active":"2024-09-16T09:25:01.706828+0000","last_became_peered":"2024-09-16T09:25:01.706828+0000","last_unstale":"2024-09-16T09:31:41.205331+0000","last_undegraded":"2024-09-16T09:31:41.205331+0000","last_fullsized":"2024-09-16T09:31:41.205331+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:50.827057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:50.827057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":338,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.222949+0000","last_change":"2024-09-16T09:31:09.281163+0000","last_active":"2024-09-16T09:31:41.222949+0000","last_peered":"2024-09-16T09:31:41.222949+0000","last_clean":"2024-09-16T09:31:41.222949+0000","last_became_active":"2024-09-16T09:31:09.266338+0000","last_became_peered":"2024-09-16T09:31:09.266338+0000","last_unstale":"2024-09-16T09:31:41.222949+0000","last_undegraded":"2024-09-16T09:31:41.222949+0000","last_fullsized":"2024-09-16T09:31:41.222949+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:29:16.104209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:29:16.104209+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0.00055022299999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":307,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.220052+0000","last_change":"2024-09-16T09:29:00.169141+0000","last_active":"2024-09-16T09:31:41.220052+0000","last_peered":"2024-09-16T09:31:41.220052+0000","last_clean":"2024-09-16T09:31:41.220052+0000","last_became_active":"2024-09-16T09:25:05.711539+0000","last_became_peered":"2024-09-16T09:25:05.711539+0000","last_unstale":"2024-09-16T09:31:41.220052+0000","last_undegraded":"2024-09-16T09:31:41.220052+0000","last_fullsized":"2024-09-16T09:31:41.220052+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:00.169077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:00.169077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":294,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.220775+0000","last_change":"2024-09-16T09:29:22.968971+0000","last_active":"2024-09-16T09:31:41.220775+0000","last_peered":"2024-09-16T09:31:41.220775+0000","last_clean":"2024-09-16T09:31:41.220775+0000","last_became_active":"2024-09-16T09:25:07.723282+0000","last_became_peered":"2024-09-16T09:25:07.723282+0000","last_unstale":"2024-09-16T09:31:41.220775+0000","last_undegraded":"2024-09-16T09:31:41.220775+0000","last_fullsized":"2024-09-16T09:31:41.220775+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:22.968908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:22.968908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"197'8963","reported_seq":9064,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:42.809114+0000","last_change":"2024-09-16T09:29:49.448898+0000","last_active":"2024-09-16T09:31:42.809114+0000","last_peered":"2024-09-16T09:31:42.809114+0000","last_clean":"2024-09-16T09:31:42.809114+0000","last_became_active":"2024-09-16T09:29:49.448628+0000","last_became_peered":"2024-09-16T09:29:49.448628+0000","last_unstale":"2024-09-16T09:31:42.809114+0000","last_undegraded":"2024-09-16T09:31:42.809114+0000","last_fullsized":"2024-09-16T09:31:42.809114+0000","mapping_epoch":152,"log_start":"174'4400","ondisk_log_start":"174'4400","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":4563,"log_dups_size":0,"ondisk_log_size":4563,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":437190656,"num_objects":6671,"num_object_clones":0,"num_object_copies":20013,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6671,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8963,"num_write_kb":500288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"73'79","reported_seq":268,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.206516+0000","last_change":"2024-09-16T09:31:40.212968+0000","last_active":"2024-09-16T09:31:41.206516+0000","last_peered":"2024-09-16T09:31:41.206516+0000","last_clean":"2024-09-16T09:31:41.206516+0000","last_became_active":"2024-09-16T09:30:08.877110+0000","last_became_peered":"2024-09-16T09:30:08.877110+0000","last_unstale":"2024-09-16T09:31:41.206516+0000","last_undegraded":"2024-09-16T09:31:41.206516+0000","last_fullsized":"2024-09-16T09:31:41.206516+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:31:40.212889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:31:40.212889+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:45.708324+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"66'12","reported_seq":313,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.219833+0000","last_change":"2024-09-16T09:29:02.713335+0000","last_active":"2024-09-16T09:31:41.219833+0000","last_peered":"2024-09-16T09:31:41.219833+0000","last_clean":"2024-09-16T09:31:41.219833+0000","last_became_active":"2024-09-16T09:25:03.715315+0000","last_became_peered":"2024-09-16T09:25:03.715315+0000","last_unstale":"2024-09-16T09:31:41.219833+0000","last_undegraded":"2024-09-16T09:31:41.219833+0000","last_fullsized":"2024-09-16T09:31:41.219833+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-16T09:29:02.713259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:29:02.713259+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":26,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"59'1","reported_seq":330,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.215672+0000","last_change":"2024-09-16T09:31:09.286309+0000","last_active":"2024-09-16T09:31:41.215672+0000","last_peered":"2024-09-16T09:31:41.215672+0000","last_clean":"2024-09-16T09:31:41.215672+0000","last_became_active":"2024-09-16T09:31:09.269627+0000","last_became_peered":"2024-09-16T09:31:09.269627+0000","last_unstale":"2024-09-16T09:31:41.215672+0000","last_undegraded":"2024-09-16T09:31:41.215672+0000","last_fullsized":"2024-09-16T09:31:41.215672+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:27:00.666673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:27:00.666673+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:13.191274+0000","scrub_duration":28,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1986,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"67'5","reported_seq":712,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.217907+0000","last_change":"2024-09-16T09:29:19.994791+0000","last_active":"2024-09-16T09:31:41.217907+0000","last_peered":"2024-09-16T09:31:41.217907+0000","last_clean":"2024-09-16T09:31:41.217907+0000","last_became_active":"2024-09-16T09:25:00.696546+0000","last_became_peered":"2024-09-16T09:25:00.696546+0000","last_unstale":"2024-09-16T09:31:41.217907+0000","last_undegraded":"2024-09-16T09:31:41.217907+0000","last_fullsized":"2024-09-16T09:31:41.217907+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'5","last_scrub_stamp":"2024-09-16T09:29:19.994716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:29:19.994716+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0.0059770420000000001,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":395,"num_read_kb":390,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":290,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.209232+0000","last_change":"2024-09-16T09:27:07.337170+0000","last_active":"2024-09-16T09:31:41.209232+0000","last_peered":"2024-09-16T09:31:41.209232+0000","last_clean":"2024-09-16T09:31:41.209232+0000","last_became_active":"2024-09-16T09:25:05.714704+0000","last_became_peered":"2024-09-16T09:25:05.714704+0000","last_unstale":"2024-09-16T09:31:41.209232+0000","last_undegraded":"2024-09-16T09:31:41.209232+0000","last_fullsized":"2024-09-16T09:31:41.209232+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:07.337124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:07.337124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":301,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.204581+0000","last_change":"2024-09-16T09:31:09.271453+0000","last_active":"2024-09-16T09:31:41.204581+0000","last_peered":"2024-09-16T09:31:41.204581+0000","last_clean":"2024-09-16T09:31:41.204581+0000","last_became_active":"2024-09-16T09:31:09.271111+0000","last_became_peered":"2024-09-16T09:31:09.271111+0000","last_unstale":"2024-09-16T09:31:41.204581+0000","last_undegraded":"2024-09-16T09:31:41.204581+0000","last_fullsized":"2024-09-16T09:31:41.204581+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:24.231167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:24.231167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"197'8891","reported_seq":9057,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:42.597332+0000","last_change":"2024-09-16T09:30:56.023710+0000","last_active":"2024-09-16T09:31:42.597332+0000","last_peered":"2024-09-16T09:31:42.597332+0000","last_clean":"2024-09-16T09:31:42.597332+0000","last_became_active":"2024-09-16T09:29:49.450643+0000","last_became_peered":"2024-09-16T09:29:49.450643+0000","last_unstale":"2024-09-16T09:31:42.597332+0000","last_undegraded":"2024-09-16T09:31:42.597332+0000","last_fullsized":"2024-09-16T09:31:42.597332+0000","mapping_epoch":152,"log_start":"171'3700","ondisk_log_start":"171'3700","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"181'5780","last_scrub_stamp":"2024-09-16T09:30:56.023622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:30:56.023622+0000","objects_scrubbed":5716,"log_size":5191,"log_dups_size":0,"ondisk_log_size":5191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:07.379468+0000","scrub_duration":1526,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":437321728,"num_objects":6673,"num_object_clones":0,"num_object_copies":20019,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6673,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8891,"num_write_kb":498048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"66'16","reported_seq":406,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.213364+0000","last_change":"2024-09-16T09:30:50.441875+0000","last_active":"2024-09-16T09:31:41.213364+0000","last_peered":"2024-09-16T09:31:41.213364+0000","last_clean":"2024-09-16T09:31:41.213364+0000","last_became_active":"2024-09-16T09:25:03.714118+0000","last_became_peered":"2024-09-16T09:25:03.714118+0000","last_unstale":"2024-09-16T09:31:41.213364+0000","last_undegraded":"2024-09-16T09:31:41.213364+0000","last_fullsized":"2024-09-16T09:31:41.213364+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'16","last_scrub_stamp":"2024-09-16T09:30:50.441795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:30:50.441795+0000","objects_scrubbed":8,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:00.328016+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":148,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.208735+0000","last_change":"2024-09-16T09:31:09.270603+0000","last_active":"2024-09-16T09:31:41.208735+0000","last_peered":"2024-09-16T09:31:41.208735+0000","last_clean":"2024-09-16T09:31:41.208735+0000","last_became_active":"2024-09-16T09:31:09.270196+0000","last_became_peered":"2024-09-16T09:31:09.270196+0000","last_unstale":"2024-09-16T09:31:41.208735+0000","last_undegraded":"2024-09-16T09:31:41.208735+0000","last_fullsized":"2024-09-16T09:31:41.208735+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:49.195331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:27:49.195331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"73'79","reported_seq":383,"reported_epoch":197,"state":"active+clean+scrubbing","last_fresh":"2024-09-16T09:31:42.473066+0000","last_change":"2024-09-16T09:31:42.472639+0000","last_active":"2024-09-16T09:31:42.473066+0000","last_peered":"2024-09-16T09:31:42.473066+0000","last_clean":"2024-09-16T09:31:42.473066+0000","last_became_active":"2024-09-16T09:30:08.870394+0000","last_became_peered":"2024-09-16T09:30:08.870394+0000","last_unstale":"2024-09-16T09:31:42.473066+0000","last_undegraded":"2024-09-16T09:31:42.473066+0000","last_fullsized":"2024-09-16T09:31:42.473066+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:02.723794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:02.723794+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"scrubbing for 0s","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":290,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.218597+0000","last_change":"2024-09-16T09:26:39.699133+0000","last_active":"2024-09-16T09:31:41.218597+0000","last_peered":"2024-09-16T09:31:41.218597+0000","last_clean":"2024-09-16T09:31:41.218597+0000","last_became_active":"2024-09-16T09:25:05.714341+0000","last_became_peered":"2024-09-16T09:25:05.714341+0000","last_unstale":"2024-09-16T09:31:41.218597+0000","last_undegraded":"2024-09-16T09:31:41.218597+0000","last_fullsized":"2024-09-16T09:31:41.218597+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:39.699059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:26:39.699059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":294,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.209127+0000","last_change":"2024-09-16T09:29:00.243528+0000","last_active":"2024-09-16T09:31:41.209127+0000","last_peered":"2024-09-16T09:31:41.209127+0000","last_clean":"2024-09-16T09:31:41.209127+0000","last_became_active":"2024-09-16T09:25:07.714281+0000","last_became_peered":"2024-09-16T09:25:07.714281+0000","last_unstale":"2024-09-16T09:31:41.209127+0000","last_undegraded":"2024-09-16T09:31:41.209127+0000","last_fullsized":"2024-09-16T09:31:41.209127+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:00.243482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:00.243482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.b","version":"73'79","reported_seq":265,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.205583+0000","last_change":"2024-09-16T09:31:14.199282+0000","last_active":"2024-09-16T09:31:41.205583+0000","last_peered":"2024-09-16T09:31:41.205583+0000","last_clean":"2024-09-16T09:31:41.205583+0000","last_became_active":"2024-09-16T09:30:10.784688+0000","last_became_peered":"2024-09-16T09:30:10.784688+0000","last_unstale":"2024-09-16T09:31:41.205583+0000","last_undegraded":"2024-09-16T09:31:41.205583+0000","last_fullsized":"2024-09-16T09:31:41.205583+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:30:03.887571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:30:03.887571+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:31:43.035265+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,5],"acting":[1,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2","version":"197'8729","reported_seq":8830,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:42.764259+0000","last_change":"2024-09-16T09:29:49.445324+0000","last_active":"2024-09-16T09:31:42.764259+0000","last_peered":"2024-09-16T09:31:42.764259+0000","last_clean":"2024-09-16T09:31:42.764259+0000","last_became_active":"2024-09-16T09:29:49.445022+0000","last_became_peered":"2024-09-16T09:29:49.445022+0000","last_unstale":"2024-09-16T09:31:42.764259+0000","last_undegraded":"2024-09-16T09:31:42.764259+0000","last_fullsized":"2024-09-16T09:31:42.764259+0000","mapping_epoch":152,"log_start":"176'4500","ondisk_log_start":"176'4500","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":4229,"log_dups_size":0,"ondisk_log_size":4229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":426442752,"num_objects":6507,"num_object_clones":0,"num_object_copies":19521,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6507,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8729,"num_write_kb":487552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"66'11","reported_seq":332,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.204301+0000","last_change":"2024-09-16T09:30:05.646701+0000","last_active":"2024-09-16T09:31:41.204301+0000","last_peered":"2024-09-16T09:31:41.204301+0000","last_clean":"2024-09-16T09:31:41.204301+0000","last_became_active":"2024-09-16T09:26:33.287399+0000","last_became_peered":"2024-09-16T09:26:33.287399+0000","last_unstale":"2024-09-16T09:31:41.204301+0000","last_undegraded":"2024-09-16T09:31:41.204301+0000","last_fullsized":"2024-09-16T09:31:41.204301+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-16T09:30:05.646500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:30:05.646500+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":291,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.203986+0000","last_change":"2024-09-16T09:25:01.708568+0000","last_active":"2024-09-16T09:31:41.203986+0000","last_peered":"2024-09-16T09:31:41.203986+0000","last_clean":"2024-09-16T09:31:41.203986+0000","last_became_active":"2024-09-16T09:25:01.708270+0000","last_became_peered":"2024-09-16T09:25:01.708270+0000","last_unstale":"2024-09-16T09:31:41.203986+0000","last_undegraded":"2024-09-16T09:31:41.203986+0000","last_fullsized":"2024-09-16T09:31:41.203986+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":289,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.209102+0000","last_change":"2024-09-16T09:26:34.171190+0000","last_active":"2024-09-16T09:31:41.209102+0000","last_peered":"2024-09-16T09:31:41.209102+0000","last_clean":"2024-09-16T09:31:41.209102+0000","last_became_active":"2024-09-16T09:25:05.711964+0000","last_became_peered":"2024-09-16T09:25:05.711964+0000","last_unstale":"2024-09-16T09:31:41.209102+0000","last_undegraded":"2024-09-16T09:31:41.209102+0000","last_fullsized":"2024-09-16T09:31:41.209102+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:34.171139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:26:34.171139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":165,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.213027+0000","last_change":"2024-09-16T09:31:09.278959+0000","last_active":"2024-09-16T09:31:41.213027+0000","last_peered":"2024-09-16T09:31:41.213027+0000","last_clean":"2024-09-16T09:31:41.213027+0000","last_became_active":"2024-09-16T09:31:09.278618+0000","last_became_peered":"2024-09-16T09:31:09.278618+0000","last_unstale":"2024-09-16T09:31:41.213027+0000","last_undegraded":"2024-09-16T09:31:41.213027+0000","last_fullsized":"2024-09-16T09:31:41.213027+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:30:33.410232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:30:33.410232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:00.093285+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1","version":"197'8714","reported_seq":8815,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:42.852907+0000","last_change":"2024-09-16T09:29:49.449796+0000","last_active":"2024-09-16T09:31:42.852907+0000","last_peered":"2024-09-16T09:31:42.852907+0000","last_clean":"2024-09-16T09:31:42.852907+0000","last_became_active":"2024-09-16T09:29:49.449475+0000","last_became_peered":"2024-09-16T09:29:49.449475+0000","last_unstale":"2024-09-16T09:31:42.852907+0000","last_undegraded":"2024-09-16T09:31:42.852907+0000","last_fullsized":"2024-09-16T09:31:42.852907+0000","mapping_epoch":152,"log_start":"174'4200","ondisk_log_start":"174'4200","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":4514,"log_dups_size":0,"ondisk_log_size":4514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":428867584,"num_objects":6544,"num_object_clones":0,"num_object_copies":19632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6544,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8714,"num_write_kb":488256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.8","version":"73'79","reported_seq":262,"reported_epoch":197,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-16T09:31:41.214157+0000","last_change":"2024-09-16T09:31:09.430087+0000","last_active":"2024-09-16T09:31:41.214157+0000","last_peered":"2024-09-16T09:31:41.214157+0000","last_clean":"2024-09-16T09:31:07.261934+0000","last_became_active":"2024-09-16T09:31:09.406638+0000","last_became_peered":"2024-09-16T09:31:09.406638+0000","last_unstale":"2024-09-16T09:31:41.214157+0000","last_undegraded":"2024-09-16T09:31:09.427685+0000","last_fullsized":"2024-09-16T09:31:41.214157+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:17.482769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:17.482769+0000","objects_scrubbed":1,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262688,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":262688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":["5","2","6"],"object_location_counts":[{"shards":"2,5,6","objects":1}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"66'17","reported_seq":338,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.204840+0000","last_change":"2024-09-16T09:30:04.636115+0000","last_active":"2024-09-16T09:31:41.204840+0000","last_peered":"2024-09-16T09:31:41.204840+0000","last_clean":"2024-09-16T09:31:41.204840+0000","last_became_active":"2024-09-16T09:25:03.713358+0000","last_became_peered":"2024-09-16T09:25:03.713358+0000","last_unstale":"2024-09-16T09:31:41.204840+0000","last_undegraded":"2024-09-16T09:31:41.204840+0000","last_fullsized":"2024-09-16T09:31:41.204840+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'17","last_scrub_stamp":"2024-09-16T09:30:04.636039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:30:04.636039+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":299,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.212733+0000","last_change":"2024-09-16T09:26:04.104188+0000","last_active":"2024-09-16T09:31:41.212733+0000","last_peered":"2024-09-16T09:31:41.212733+0000","last_clean":"2024-09-16T09:31:41.212733+0000","last_became_active":"2024-09-16T09:25:01.703355+0000","last_became_peered":"2024-09-16T09:25:01.703355+0000","last_unstale":"2024-09-16T09:31:41.212733+0000","last_undegraded":"2024-09-16T09:31:41.212733+0000","last_fullsized":"2024-09-16T09:31:41.212733+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:04.104130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:04.104130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:31:46.426148+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":306,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.204665+0000","last_change":"2024-09-16T09:29:58.839132+0000","last_active":"2024-09-16T09:31:41.204665+0000","last_peered":"2024-09-16T09:31:41.204665+0000","last_clean":"2024-09-16T09:31:41.204665+0000","last_became_active":"2024-09-16T09:25:05.716900+0000","last_became_peered":"2024-09-16T09:25:05.716900+0000","last_unstale":"2024-09-16T09:31:41.204665+0000","last_undegraded":"2024-09-16T09:31:41.204665+0000","last_fullsized":"2024-09-16T09:31:41.204665+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:58.839051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:58.839051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":316,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.226651+0000","last_change":"2024-09-16T09:31:09.276869+0000","last_active":"2024-09-16T09:31:41.226651+0000","last_peered":"2024-09-16T09:31:41.226651+0000","last_clean":"2024-09-16T09:31:41.226651+0000","last_became_active":"2024-09-16T09:31:09.276275+0000","last_became_peered":"2024-09-16T09:31:09.276275+0000","last_unstale":"2024-09-16T09:31:41.226651+0000","last_undegraded":"2024-09-16T09:31:41.226651+0000","last_fullsized":"2024-09-16T09:31:41.226651+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:30:03.946025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:30:03.946025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.0","version":"197'8920","reported_seq":13087,"reported_epoch":197,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-16T09:31:42.869370+0000","last_change":"2024-09-16T09:31:09.476408+0000","last_active":"2024-09-16T09:31:42.869370+0000","last_peered":"2024-09-16T09:31:42.869370+0000","last_clean":"2024-09-16T09:30:20.486290+0000","last_became_active":"2024-09-16T09:31:09.399514+0000","last_became_peered":"2024-09-16T09:31:09.399514+0000","last_unstale":"2024-09-16T09:31:42.869370+0000","last_undegraded":"2024-09-16T09:31:42.869370+0000","last_fullsized":"2024-09-16T09:31:09.254572+0000","mapping_epoch":186,"log_start":"174'4200","ondisk_log_start":"174'4200","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":4720,"log_dups_size":0,"ondisk_log_size":4720,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":433586176,"num_objects":6616,"num_object_clones":0,"num_object_copies":19848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2674,"num_objects_unfound":0,"num_objects_dirty":6616,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8920,"num_write_kb":497152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2028,"num_bytes_recovered":132907008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,1],"avail_no_missing":["5","1"],"object_location_counts":[{"shards":"1,4,5","objects":2674},{"shards":"1,5","objects":3942}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.9","version":"73'79","reported_seq":370,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.218066+0000","last_change":"2024-09-16T09:29:19.013948+0000","last_active":"2024-09-16T09:31:41.218066+0000","last_peered":"2024-09-16T09:31:41.218066+0000","last_clean":"2024-09-16T09:31:41.218066+0000","last_became_active":"2024-09-16T09:26:35.126066+0000","last_became_peered":"2024-09-16T09:26:35.126066+0000","last_unstale":"2024-09-16T09:31:41.218066+0000","last_undegraded":"2024-09-16T09:31:41.218066+0000","last_fullsized":"2024-09-16T09:31:41.218066+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:19.013865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:19.013865+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"66'10","reported_seq":301,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.206275+0000","last_change":"2024-09-16T09:25:03.708144+0000","last_active":"2024-09-16T09:31:41.206275+0000","last_peered":"2024-09-16T09:31:41.206275+0000","last_clean":"2024-09-16T09:31:41.206275+0000","last_became_active":"2024-09-16T09:25:03.707723+0000","last_became_peered":"2024-09-16T09:25:03.707723+0000","last_unstale":"2024-09-16T09:31:41.206275+0000","last_undegraded":"2024-09-16T09:31:41.206275+0000","last_fullsized":"2024-09-16T09:31:41.206275+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":164,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.215225+0000","last_change":"2024-09-16T09:31:32.253397+0000","last_active":"2024-09-16T09:31:41.215225+0000","last_peered":"2024-09-16T09:31:41.215225+0000","last_clean":"2024-09-16T09:31:41.215225+0000","last_became_active":"2024-09-16T09:31:09.288825+0000","last_became_peered":"2024-09-16T09:31:09.288825+0000","last_unstale":"2024-09-16T09:31:41.215225+0000","last_undegraded":"2024-09-16T09:31:41.215225+0000","last_fullsized":"2024-09-16T09:31:41.215225+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:31:32.253306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:31:32.253306+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:45.682494+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":398,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.225429+0000","last_change":"2024-09-16T09:29:29.866520+0000","last_active":"2024-09-16T09:31:41.225429+0000","last_peered":"2024-09-16T09:31:41.225429+0000","last_clean":"2024-09-16T09:31:41.225429+0000","last_became_active":"2024-09-16T09:25:05.711590+0000","last_became_peered":"2024-09-16T09:25:05.711590+0000","last_unstale":"2024-09-16T09:31:41.225429+0000","last_undegraded":"2024-09-16T09:31:41.225429+0000","last_fullsized":"2024-09-16T09:31:41.225429+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:29:29.866460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:29.866460+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":302,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.203994+0000","last_change":"2024-09-16T09:29:02.667332+0000","last_active":"2024-09-16T09:31:41.203994+0000","last_peered":"2024-09-16T09:31:41.203994+0000","last_clean":"2024-09-16T09:31:41.203994+0000","last_became_active":"2024-09-16T09:25:07.718958+0000","last_became_peered":"2024-09-16T09:25:07.718958+0000","last_unstale":"2024-09-16T09:31:41.203994+0000","last_undegraded":"2024-09-16T09:31:41.203994+0000","last_fullsized":"2024-09-16T09:31:41.203994+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:02.667259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:02.667259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.e","version":"73'79","reported_seq":89,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.203450+0000","last_change":"2024-09-16T09:31:14.198121+0000","last_active":"2024-09-16T09:31:41.203450+0000","last_peered":"2024-09-16T09:31:41.203450+0000","last_clean":"2024-09-16T09:31:41.203450+0000","last_became_active":"2024-09-16T09:30:10.705072+0000","last_became_peered":"2024-09-16T09:30:10.705072+0000","last_unstale":"2024-09-16T09:31:41.203450+0000","last_undegraded":"2024-09-16T09:31:41.203450+0000","last_fullsized":"2024-09-16T09:31:41.203450+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:26:35.170229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:26:35.170229+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:31:41.814604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26835,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":10,"num_write":5,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":26835,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,3],"acting":[4,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.7","version":"197'8863","reported_seq":8964,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:42.839498+0000","last_change":"2024-09-16T09:29:49.839740+0000","last_active":"2024-09-16T09:31:42.839498+0000","last_peered":"2024-09-16T09:31:42.839498+0000","last_clean":"2024-09-16T09:31:42.839498+0000","last_became_active":"2024-09-16T09:29:49.839437+0000","last_became_peered":"2024-09-16T09:29:49.839437+0000","last_unstale":"2024-09-16T09:31:42.839498+0000","last_undegraded":"2024-09-16T09:31:42.839498+0000","last_fullsized":"2024-09-16T09:31:42.839498+0000","mapping_epoch":152,"log_start":"174'4300","ondisk_log_start":"174'4300","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":4563,"log_dups_size":0,"ondisk_log_size":4563,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":431292416,"num_objects":6581,"num_object_clones":0,"num_object_copies":19743,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6581,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8863,"num_write_kb":494208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"66'9","reported_seq":326,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.214962+0000","last_change":"2024-09-16T09:31:31.211495+0000","last_active":"2024-09-16T09:31:41.214962+0000","last_peered":"2024-09-16T09:31:41.214962+0000","last_clean":"2024-09-16T09:31:41.214962+0000","last_became_active":"2024-09-16T09:25:03.712230+0000","last_became_peered":"2024-09-16T09:25:03.712230+0000","last_unstale":"2024-09-16T09:31:41.214962+0000","last_undegraded":"2024-09-16T09:31:41.214962+0000","last_fullsized":"2024-09-16T09:31:41.214962+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:31:31.211420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:31:31.211420+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:36.499658+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":311,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.208736+0000","last_change":"2024-09-16T09:31:09.277413+0000","last_active":"2024-09-16T09:31:41.208736+0000","last_peered":"2024-09-16T09:31:41.208736+0000","last_clean":"2024-09-16T09:31:41.208736+0000","last_became_active":"2024-09-16T09:31:09.276516+0000","last_became_peered":"2024-09-16T09:31:09.276516+0000","last_unstale":"2024-09-16T09:31:41.208736+0000","last_undegraded":"2024-09-16T09:31:41.208736+0000","last_fullsized":"2024-09-16T09:31:41.208736+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:26.061106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:26.061106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":313,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.225141+0000","last_change":"2024-09-16T09:31:09.275219+0000","last_active":"2024-09-16T09:31:41.225141+0000","last_peered":"2024-09-16T09:31:41.225141+0000","last_clean":"2024-09-16T09:31:41.225141+0000","last_became_active":"2024-09-16T09:31:09.274702+0000","last_became_peered":"2024-09-16T09:31:09.274702+0000","last_unstale":"2024-09-16T09:31:41.225141+0000","last_undegraded":"2024-09-16T09:31:41.225141+0000","last_fullsized":"2024-09-16T09:31:41.225141+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:57.220068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:28:57.220068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":33,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":303,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.215576+0000","last_change":"2024-09-16T09:31:27.185957+0000","last_active":"2024-09-16T09:31:41.215576+0000","last_peered":"2024-09-16T09:31:41.215576+0000","last_clean":"2024-09-16T09:31:41.215576+0000","last_became_active":"2024-09-16T09:25:07.718778+0000","last_became_peered":"2024-09-16T09:25:07.718778+0000","last_unstale":"2024-09-16T09:31:41.215576+0000","last_undegraded":"2024-09-16T09:31:41.215576+0000","last_fullsized":"2024-09-16T09:31:41.215576+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:31:27.185801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:31:27.185801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:39.017009+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.f","version":"73'79","reported_seq":341,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.213653+0000","last_change":"2024-09-16T09:31:14.210208+0000","last_active":"2024-09-16T09:31:41.213653+0000","last_peered":"2024-09-16T09:31:41.213653+0000","last_clean":"2024-09-16T09:31:41.213653+0000","last_became_active":"2024-09-16T09:30:10.693477+0000","last_became_peered":"2024-09-16T09:30:10.693477+0000","last_unstale":"2024-09-16T09:31:41.213653+0000","last_undegraded":"2024-09-16T09:31:41.213653+0000","last_fullsized":"2024-09-16T09:31:41.213653+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:17.534429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:17.534429+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:31:43.408019+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.6","version":"197'9138","reported_seq":9239,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:43.137321+0000","last_change":"2024-09-16T09:29:50.008888+0000","last_active":"2024-09-16T09:31:43.137321+0000","last_peered":"2024-09-16T09:31:43.137321+0000","last_clean":"2024-09-16T09:31:43.137321+0000","last_became_active":"2024-09-16T09:29:50.008656+0000","last_became_peered":"2024-09-16T09:29:50.008656+0000","last_unstale":"2024-09-16T09:31:43.137321+0000","last_undegraded":"2024-09-16T09:31:43.137321+0000","last_fullsized":"2024-09-16T09:31:43.137321+0000","mapping_epoch":152,"log_start":"172'4000","ondisk_log_start":"172'4000","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":5138,"log_dups_size":0,"ondisk_log_size":5138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":438042624,"num_objects":6684,"num_object_clones":0,"num_object_copies":20052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6684,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9138,"num_write_kb":506304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"66'19","reported_seq":334,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.212623+0000","last_change":"2024-09-16T09:29:02.554357+0000","last_active":"2024-09-16T09:31:41.212623+0000","last_peered":"2024-09-16T09:31:41.212623+0000","last_clean":"2024-09-16T09:31:41.212623+0000","last_became_active":"2024-09-16T09:25:03.712783+0000","last_became_peered":"2024-09-16T09:25:03.712783+0000","last_unstale":"2024-09-16T09:31:41.212623+0000","last_undegraded":"2024-09-16T09:31:41.212623+0000","last_fullsized":"2024-09-16T09:31:41.212623+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-16T09:29:02.554274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:29:02.554274+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:17.371584+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"59'1","reported_seq":311,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.221089+0000","last_change":"2024-09-16T09:29:51.025505+0000","last_active":"2024-09-16T09:31:41.221089+0000","last_peered":"2024-09-16T09:31:41.221089+0000","last_clean":"2024-09-16T09:31:41.221089+0000","last_became_active":"2024-09-16T09:25:01.708069+0000","last_became_peered":"2024-09-16T09:25:01.708069+0000","last_unstale":"2024-09-16T09:31:41.221089+0000","last_undegraded":"2024-09-16T09:31:41.221089+0000","last_fullsized":"2024-09-16T09:31:41.221089+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:29:51.025444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:51.025444+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":290,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.219730+0000","last_change":"2024-09-16T09:27:11.067059+0000","last_active":"2024-09-16T09:31:41.219730+0000","last_peered":"2024-09-16T09:31:41.219730+0000","last_clean":"2024-09-16T09:31:41.219730+0000","last_became_active":"2024-09-16T09:25:05.709264+0000","last_became_peered":"2024-09-16T09:25:05.709264+0000","last_unstale":"2024-09-16T09:31:41.219730+0000","last_undegraded":"2024-09-16T09:31:41.219730+0000","last_fullsized":"2024-09-16T09:31:41.219730+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:11.066986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:11.066986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":286,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.217995+0000","last_change":"2024-09-16T09:27:25.865107+0000","last_active":"2024-09-16T09:31:41.217995+0000","last_peered":"2024-09-16T09:31:41.217995+0000","last_clean":"2024-09-16T09:31:41.217995+0000","last_became_active":"2024-09-16T09:25:07.718750+0000","last_became_peered":"2024-09-16T09:25:07.718750+0000","last_unstale":"2024-09-16T09:31:41.217995+0000","last_undegraded":"2024-09-16T09:31:41.217995+0000","last_fullsized":"2024-09-16T09:31:41.217995+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:25.865030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:25.865030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.c","version":"73'79","reported_seq":36,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.211359+0000","last_change":"2024-09-16T09:31:09.290919+0000","last_active":"2024-09-16T09:31:41.211359+0000","last_peered":"2024-09-16T09:31:41.211359+0000","last_clean":"2024-09-16T09:31:41.211359+0000","last_became_active":"2024-09-16T09:31:09.273770+0000","last_became_peered":"2024-09-16T09:31:09.273770+0000","last_unstale":"2024-09-16T09:31:41.211359+0000","last_undegraded":"2024-09-16T09:31:41.211359+0000","last_fullsized":"2024-09-16T09:31:41.211359+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:30:22.614462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:30:22.614462+0000","objects_scrubbed":1,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":327680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.5","version":"197'8949","reported_seq":9123,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:42.598678+0000","last_change":"2024-09-16T09:31:06.121217+0000","last_active":"2024-09-16T09:31:42.598678+0000","last_peered":"2024-09-16T09:31:42.598678+0000","last_clean":"2024-09-16T09:31:42.598678+0000","last_became_active":"2024-09-16T09:29:50.009013+0000","last_became_peered":"2024-09-16T09:29:50.009013+0000","last_unstale":"2024-09-16T09:31:42.598678+0000","last_undegraded":"2024-09-16T09:31:42.598678+0000","last_fullsized":"2024-09-16T09:31:42.598678+0000","mapping_epoch":152,"log_start":"171'3713","ondisk_log_start":"171'3713","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"184'6587","last_scrub_stamp":"2024-09-16T09:31:06.120990+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:31:06.120990+0000","objects_scrubbed":6523,"log_size":5236,"log_dups_size":0,"ondisk_log_size":5236,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:20.260788+0000","scrub_duration":1690,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":445317120,"num_objects":6795,"num_object_clones":0,"num_object_copies":20385,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6795,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8949,"num_write_kb":503808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"66'12","reported_seq":364,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.203872+0000","last_change":"2024-09-16T09:31:09.296915+0000","last_active":"2024-09-16T09:31:41.203872+0000","last_peered":"2024-09-16T09:31:41.203872+0000","last_clean":"2024-09-16T09:31:41.203872+0000","last_became_active":"2024-09-16T09:31:09.269047+0000","last_became_peered":"2024-09-16T09:31:09.269047+0000","last_unstale":"2024-09-16T09:31:41.203872+0000","last_undegraded":"2024-09-16T09:31:41.203872+0000","last_fullsized":"2024-09-16T09:31:41.203872+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-16T09:30:21.728668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:30:21.728668+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":305,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.220887+0000","last_change":"2024-09-16T09:29:53.075288+0000","last_active":"2024-09-16T09:31:41.220887+0000","last_peered":"2024-09-16T09:31:41.220887+0000","last_clean":"2024-09-16T09:31:41.220887+0000","last_became_active":"2024-09-16T09:25:01.707528+0000","last_became_peered":"2024-09-16T09:25:01.707528+0000","last_unstale":"2024-09-16T09:31:41.220887+0000","last_undegraded":"2024-09-16T09:31:41.220887+0000","last_fullsized":"2024-09-16T09:31:41.220887+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:53.075232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:53.075232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":299,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.219984+0000","last_change":"2024-09-16T09:27:49.965628+0000","last_active":"2024-09-16T09:31:41.219984+0000","last_peered":"2024-09-16T09:31:41.219984+0000","last_clean":"2024-09-16T09:31:41.219984+0000","last_became_active":"2024-09-16T09:25:05.715253+0000","last_became_peered":"2024-09-16T09:25:05.715253+0000","last_unstale":"2024-09-16T09:31:41.219984+0000","last_undegraded":"2024-09-16T09:31:41.219984+0000","last_fullsized":"2024-09-16T09:31:41.219984+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:49.965424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:49.965424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":286,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.214563+0000","last_change":"2024-09-16T09:27:56.059755+0000","last_active":"2024-09-16T09:31:41.214563+0000","last_peered":"2024-09-16T09:31:41.214563+0000","last_clean":"2024-09-16T09:31:41.214563+0000","last_became_active":"2024-09-16T09:25:07.718899+0000","last_became_peered":"2024-09-16T09:25:07.718899+0000","last_unstale":"2024-09-16T09:31:41.214563+0000","last_undegraded":"2024-09-16T09:31:41.214563+0000","last_fullsized":"2024-09-16T09:31:41.214563+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:56.059687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:56.059687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.d","version":"73'79","reported_seq":89,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.218066+0000","last_change":"2024-09-16T09:31:14.215723+0000","last_active":"2024-09-16T09:31:41.218066+0000","last_peered":"2024-09-16T09:31:41.218066+0000","last_clean":"2024-09-16T09:31:41.218066+0000","last_became_active":"2024-09-16T09:30:10.709958+0000","last_became_peered":"2024-09-16T09:30:10.709958+0000","last_unstale":"2024-09-16T09:31:41.218066+0000","last_undegraded":"2024-09-16T09:31:41.218066+0000","last_fullsized":"2024-09-16T09:31:41.218066+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:58.568841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:58.568841+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:31:43.390312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26835,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":10,"num_write":5,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":26835,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.4","version":"197'8929","reported_seq":9030,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:42.864783+0000","last_change":"2024-09-16T09:29:49.444857+0000","last_active":"2024-09-16T09:31:42.864783+0000","last_peered":"2024-09-16T09:31:42.864783+0000","last_clean":"2024-09-16T09:31:42.864783+0000","last_became_active":"2024-09-16T09:29:49.444515+0000","last_became_peered":"2024-09-16T09:29:49.444515+0000","last_unstale":"2024-09-16T09:31:42.864783+0000","last_undegraded":"2024-09-16T09:31:42.864783+0000","last_fullsized":"2024-09-16T09:31:42.864783+0000","mapping_epoch":152,"log_start":"173'4200","ondisk_log_start":"173'4200","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":4729,"log_dups_size":0,"ondisk_log_size":4729,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":437714944,"num_objects":6679,"num_object_clones":0,"num_object_copies":20037,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6679,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8929,"num_write_kb":499456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"66'15","reported_seq":319,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.214476+0000","last_change":"2024-09-16T09:26:33.138828+0000","last_active":"2024-09-16T09:31:41.214476+0000","last_peered":"2024-09-16T09:31:41.214476+0000","last_clean":"2024-09-16T09:31:41.214476+0000","last_became_active":"2024-09-16T09:25:03.715733+0000","last_became_peered":"2024-09-16T09:25:03.715733+0000","last_unstale":"2024-09-16T09:31:41.214476+0000","last_undegraded":"2024-09-16T09:31:41.214476+0000","last_fullsized":"2024-09-16T09:31:41.214476+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-16T09:26:33.138603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:26:33.138603+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":309,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.220417+0000","last_change":"2024-09-16T09:26:26.705847+0000","last_active":"2024-09-16T09:31:41.220417+0000","last_peered":"2024-09-16T09:31:41.220417+0000","last_clean":"2024-09-16T09:31:41.220417+0000","last_became_active":"2024-09-16T09:25:01.707372+0000","last_became_peered":"2024-09-16T09:25:01.707372+0000","last_unstale":"2024-09-16T09:31:41.220417+0000","last_undegraded":"2024-09-16T09:31:41.220417+0000","last_fullsized":"2024-09-16T09:31:41.220417+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-16T09:26:26.705773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:26.705773+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"66'8","reported_seq":395,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.218016+0000","last_change":"2024-09-16T09:30:05.138362+0000","last_active":"2024-09-16T09:31:41.218016+0000","last_peered":"2024-09-16T09:31:41.218016+0000","last_clean":"2024-09-16T09:31:41.218016+0000","last_became_active":"2024-09-16T09:25:05.716189+0000","last_became_peered":"2024-09-16T09:25:05.716189+0000","last_unstale":"2024-09-16T09:31:41.218016+0000","last_undegraded":"2024-09-16T09:31:41.218016+0000","last_fullsized":"2024-09-16T09:31:41.218016+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:30:05.138288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:30:05.138288+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":286,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.209490+0000","last_change":"2024-09-16T09:27:20.219376+0000","last_active":"2024-09-16T09:31:41.209490+0000","last_peered":"2024-09-16T09:31:41.209490+0000","last_clean":"2024-09-16T09:31:41.209490+0000","last_became_active":"2024-09-16T09:25:07.717213+0000","last_became_peered":"2024-09-16T09:25:07.717213+0000","last_unstale":"2024-09-16T09:31:41.209490+0000","last_undegraded":"2024-09-16T09:31:41.209490+0000","last_fullsized":"2024-09-16T09:31:41.209490+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:20.219333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:20.219333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.12","version":"73'79","reported_seq":89,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.208559+0000","last_change":"2024-09-16T09:31:14.196160+0000","last_active":"2024-09-16T09:31:41.208559+0000","last_peered":"2024-09-16T09:31:41.208559+0000","last_clean":"2024-09-16T09:31:41.208559+0000","last_became_active":"2024-09-16T09:30:10.731413+0000","last_became_peered":"2024-09-16T09:30:10.731413+0000","last_unstale":"2024-09-16T09:31:41.208559+0000","last_undegraded":"2024-09-16T09:31:41.208559+0000","last_fullsized":"2024-09-16T09:31:41.208559+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:26:35.745837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:26:35.745837+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:31:43.386292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26835,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":10,"num_write":6,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":26835,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":298,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.212503+0000","last_change":"2024-09-16T09:26:33.139405+0000","last_active":"2024-09-16T09:31:41.212503+0000","last_peered":"2024-09-16T09:31:41.212503+0000","last_clean":"2024-09-16T09:31:41.212503+0000","last_became_active":"2024-09-16T09:25:01.703202+0000","last_became_peered":"2024-09-16T09:25:01.703202+0000","last_unstale":"2024-09-16T09:31:41.212503+0000","last_undegraded":"2024-09-16T09:31:41.212503+0000","last_fullsized":"2024-09-16T09:31:41.212503+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:33.139360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:33.139360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:10.508509+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"66'6","reported_seq":40,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.212303+0000","last_change":"2024-09-16T09:31:09.297514+0000","last_active":"2024-09-16T09:31:41.212303+0000","last_peered":"2024-09-16T09:31:41.212303+0000","last_clean":"2024-09-16T09:31:41.212303+0000","last_became_active":"2024-09-16T09:31:09.270685+0000","last_became_peered":"2024-09-16T09:31:09.270685+0000","last_unstale":"2024-09-16T09:31:41.212303+0000","last_undegraded":"2024-09-16T09:31:41.212303+0000","last_fullsized":"2024-09-16T09:31:41.212303+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'6","last_scrub_stamp":"2024-09-16T09:26:23.458794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:26:23.458794+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":8,"num_read_kb":5,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":297,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.208783+0000","last_change":"2024-09-16T09:29:31.496374+0000","last_active":"2024-09-16T09:31:41.208783+0000","last_peered":"2024-09-16T09:31:41.208783+0000","last_clean":"2024-09-16T09:31:41.208783+0000","last_became_active":"2024-09-16T09:25:05.715616+0000","last_became_peered":"2024-09-16T09:25:05.715616+0000","last_unstale":"2024-09-16T09:31:41.208783+0000","last_undegraded":"2024-09-16T09:31:41.208783+0000","last_fullsized":"2024-09-16T09:31:41.208783+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:31.496304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:31.496304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":285,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.220806+0000","last_change":"2024-09-16T09:26:28.733381+0000","last_active":"2024-09-16T09:31:41.220806+0000","last_peered":"2024-09-16T09:31:41.220806+0000","last_clean":"2024-09-16T09:31:41.220806+0000","last_became_active":"2024-09-16T09:25:07.721059+0000","last_became_peered":"2024-09-16T09:25:07.721059+0000","last_unstale":"2024-09-16T09:31:41.220806+0000","last_undegraded":"2024-09-16T09:31:41.220806+0000","last_fullsized":"2024-09-16T09:31:41.220806+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:28.733324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:28.733324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.13","version":"73'79","reported_seq":281,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.206358+0000","last_change":"2024-09-16T09:31:09.288677+0000","last_active":"2024-09-16T09:31:41.206358+0000","last_peered":"2024-09-16T09:31:41.206358+0000","last_clean":"2024-09-16T09:31:41.206358+0000","last_became_active":"2024-09-16T09:31:09.288233+0000","last_became_peered":"2024-09-16T09:31:09.288233+0000","last_unstale":"2024-09-16T09:31:41.206358+0000","last_undegraded":"2024-09-16T09:31:41.206358+0000","last_fullsized":"2024-09-16T09:31:41.206358+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:30:21.714369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:30:21.714369+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:31:50.034245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.16","version":"66'9","reported_seq":360,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.219909+0000","last_change":"2024-09-16T09:31:09.309570+0000","last_active":"2024-09-16T09:31:41.219909+0000","last_peered":"2024-09-16T09:31:41.219909+0000","last_clean":"2024-09-16T09:31:41.219909+0000","last_became_active":"2024-09-16T09:31:09.273498+0000","last_became_peered":"2024-09-16T09:31:09.273498+0000","last_unstale":"2024-09-16T09:31:41.219909+0000","last_undegraded":"2024-09-16T09:31:41.219909+0000","last_fullsized":"2024-09-16T09:31:41.219909+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:29:58.867513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:29:58.867513+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":305,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.218563+0000","last_change":"2024-09-16T09:28:50.698331+0000","last_active":"2024-09-16T09:31:41.218563+0000","last_peered":"2024-09-16T09:31:41.218563+0000","last_clean":"2024-09-16T09:31:41.218563+0000","last_became_active":"2024-09-16T09:25:01.706491+0000","last_became_peered":"2024-09-16T09:25:01.706491+0000","last_unstale":"2024-09-16T09:31:41.218563+0000","last_undegraded":"2024-09-16T09:31:41.218563+0000","last_fullsized":"2024-09-16T09:31:41.218563+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:50.698288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:28:50.698288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":156,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.205202+0000","last_change":"2024-09-16T09:31:09.271558+0000","last_active":"2024-09-16T09:31:41.205202+0000","last_peered":"2024-09-16T09:31:41.205202+0000","last_clean":"2024-09-16T09:31:41.205202+0000","last_became_active":"2024-09-16T09:31:09.271223+0000","last_became_peered":"2024-09-16T09:31:09.271223+0000","last_unstale":"2024-09-16T09:31:41.205202+0000","last_undegraded":"2024-09-16T09:31:41.205202+0000","last_fullsized":"2024-09-16T09:31:41.205202+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:53.810580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:28:53.810580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":286,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.220408+0000","last_change":"2024-09-16T09:26:35.866582+0000","last_active":"2024-09-16T09:31:41.220408+0000","last_peered":"2024-09-16T09:31:41.220408+0000","last_clean":"2024-09-16T09:31:41.220408+0000","last_became_active":"2024-09-16T09:25:07.722050+0000","last_became_peered":"2024-09-16T09:25:07.722050+0000","last_unstale":"2024-09-16T09:31:41.220408+0000","last_undegraded":"2024-09-16T09:31:41.220408+0000","last_fullsized":"2024-09-16T09:31:41.220408+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:35.866512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:35.866512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:08.904503+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.10","version":"73'79","reported_seq":89,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.209968+0000","last_change":"2024-09-16T09:31:14.195080+0000","last_active":"2024-09-16T09:31:41.209968+0000","last_peered":"2024-09-16T09:31:41.209968+0000","last_clean":"2024-09-16T09:31:41.209968+0000","last_became_active":"2024-09-16T09:30:10.692770+0000","last_became_peered":"2024-09-16T09:30:10.692770+0000","last_unstale":"2024-09-16T09:31:41.209968+0000","last_undegraded":"2024-09-16T09:31:41.209968+0000","last_fullsized":"2024-09-16T09:31:41.209968+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:02.723794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:02.723794+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:31:42.406568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26835,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":10,"num_write":5,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":26835,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"66'9","reported_seq":317,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.208864+0000","last_change":"2024-09-16T09:29:32.485165+0000","last_active":"2024-09-16T09:31:41.208864+0000","last_peered":"2024-09-16T09:31:41.208864+0000","last_clean":"2024-09-16T09:31:41.208864+0000","last_became_active":"2024-09-16T09:25:03.717310+0000","last_became_peered":"2024-09-16T09:25:03.717310+0000","last_unstale":"2024-09-16T09:31:41.208864+0000","last_undegraded":"2024-09-16T09:31:41.208864+0000","last_fullsized":"2024-09-16T09:31:41.208864+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:29:32.485084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:29:32.485084+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":320,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.214612+0000","last_change":"2024-09-16T09:31:09.273453+0000","last_active":"2024-09-16T09:31:41.214612+0000","last_peered":"2024-09-16T09:31:41.214612+0000","last_clean":"2024-09-16T09:31:41.214612+0000","last_became_active":"2024-09-16T09:31:09.272848+0000","last_became_peered":"2024-09-16T09:31:09.272848+0000","last_unstale":"2024-09-16T09:31:41.214612+0000","last_undegraded":"2024-09-16T09:31:41.214612+0000","last_fullsized":"2024-09-16T09:31:41.214612+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.950144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.950144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"187'11","reported_seq":224,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.218547+0000","last_change":"2024-09-16T09:31:09.311243+0000","last_active":"2024-09-16T09:31:41.218547+0000","last_peered":"2024-09-16T09:31:41.218547+0000","last_clean":"2024-09-16T09:31:41.218547+0000","last_became_active":"2024-09-16T09:31:09.277688+0000","last_became_peered":"2024-09-16T09:31:09.277688+0000","last_unstale":"2024-09-16T09:31:41.218547+0000","last_undegraded":"2024-09-16T09:31:41.218547+0000","last_fullsized":"2024-09-16T09:31:41.218547+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:28:53.714413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:28:53.714413+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":302,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.203880+0000","last_change":"2024-09-16T09:29:49.474440+0000","last_active":"2024-09-16T09:31:41.203880+0000","last_peered":"2024-09-16T09:31:41.203880+0000","last_clean":"2024-09-16T09:31:41.203880+0000","last_became_active":"2024-09-16T09:25:07.718810+0000","last_became_peered":"2024-09-16T09:25:07.718810+0000","last_unstale":"2024-09-16T09:31:41.203880+0000","last_undegraded":"2024-09-16T09:31:41.203880+0000","last_fullsized":"2024-09-16T09:31:41.203880+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:49.474391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:49.474391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.11","version":"73'79","reported_seq":181,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.215771+0000","last_change":"2024-09-16T09:31:09.272938+0000","last_active":"2024-09-16T09:31:41.215771+0000","last_peered":"2024-09-16T09:31:41.215771+0000","last_clean":"2024-09-16T09:31:41.215771+0000","last_became_active":"2024-09-16T09:31:09.272208+0000","last_became_peered":"2024-09-16T09:31:09.272208+0000","last_unstale":"2024-09-16T09:31:41.215771+0000","last_undegraded":"2024-09-16T09:31:41.215771+0000","last_fullsized":"2024-09-16T09:31:41.215771+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:30:21.651575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:30:21.651575+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.14","version":"66'10","reported_seq":177,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.204790+0000","last_change":"2024-09-16T09:28:56.825675+0000","last_active":"2024-09-16T09:31:41.204790+0000","last_peered":"2024-09-16T09:31:41.204790+0000","last_clean":"2024-09-16T09:31:41.204790+0000","last_became_active":"2024-09-16T09:28:52.979059+0000","last_became_peered":"2024-09-16T09:28:52.979059+0000","last_unstale":"2024-09-16T09:31:41.204790+0000","last_undegraded":"2024-09-16T09:31:41.204790+0000","last_fullsized":"2024-09-16T09:31:41.204790+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-16T09:28:56.825476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:56.825476+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":14,"num_read_kb":9,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":305,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.218285+0000","last_change":"2024-09-16T09:29:56.021413+0000","last_active":"2024-09-16T09:31:41.218285+0000","last_peered":"2024-09-16T09:31:41.218285+0000","last_clean":"2024-09-16T09:31:41.218285+0000","last_became_active":"2024-09-16T09:25:01.707513+0000","last_became_peered":"2024-09-16T09:25:01.707513+0000","last_unstale":"2024-09-16T09:31:41.218285+0000","last_undegraded":"2024-09-16T09:31:41.218285+0000","last_fullsized":"2024-09-16T09:31:41.218285+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:56.021340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:56.021340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"66'8","reported_seq":394,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.208799+0000","last_change":"2024-09-16T09:29:48.485781+0000","last_active":"2024-09-16T09:31:41.208799+0000","last_peered":"2024-09-16T09:31:41.208799+0000","last_clean":"2024-09-16T09:31:41.208799+0000","last_became_active":"2024-09-16T09:25:05.713833+0000","last_became_peered":"2024-09-16T09:25:05.713833+0000","last_unstale":"2024-09-16T09:31:41.208799+0000","last_undegraded":"2024-09-16T09:31:41.208799+0000","last_fullsized":"2024-09-16T09:31:41.208799+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:29:48.485727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.485727+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":308,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.219447+0000","last_change":"2024-09-16T09:31:09.276853+0000","last_active":"2024-09-16T09:31:41.219447+0000","last_peered":"2024-09-16T09:31:41.219447+0000","last_clean":"2024-09-16T09:31:41.219447+0000","last_became_active":"2024-09-16T09:31:09.275168+0000","last_became_peered":"2024-09-16T09:31:09.275168+0000","last_unstale":"2024-09-16T09:31:41.219447+0000","last_undegraded":"2024-09-16T09:31:41.219447+0000","last_fullsized":"2024-09-16T09:31:41.219447+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:52.958303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:52.958303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"66'11","reported_seq":324,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.206481+0000","last_change":"2024-09-16T09:26:33.302713+0000","last_active":"2024-09-16T09:31:41.206481+0000","last_peered":"2024-09-16T09:31:41.206481+0000","last_clean":"2024-09-16T09:31:41.206481+0000","last_became_active":"2024-09-16T09:26:33.291265+0000","last_became_peered":"2024-09-16T09:26:33.291265+0000","last_unstale":"2024-09-16T09:31:41.206481+0000","last_undegraded":"2024-09-16T09:31:41.206481+0000","last_fullsized":"2024-09-16T09:31:41.206481+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":314,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.206459+0000","last_change":"2024-09-16T09:29:51.517112+0000","last_active":"2024-09-16T09:31:41.206459+0000","last_peered":"2024-09-16T09:31:41.206459+0000","last_clean":"2024-09-16T09:31:41.206459+0000","last_became_active":"2024-09-16T09:25:01.705158+0000","last_became_peered":"2024-09-16T09:25:01.705158+0000","last_unstale":"2024-09-16T09:31:41.206459+0000","last_undegraded":"2024-09-16T09:31:41.206459+0000","last_fullsized":"2024-09-16T09:31:41.206459+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:51.517054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:51.517054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":320,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.204648+0000","last_change":"2024-09-16T09:31:09.266820+0000","last_active":"2024-09-16T09:31:41.204648+0000","last_peered":"2024-09-16T09:31:41.204648+0000","last_clean":"2024-09-16T09:31:41.204648+0000","last_became_active":"2024-09-16T09:31:09.266326+0000","last_became_peered":"2024-09-16T09:31:09.266326+0000","last_unstale":"2024-09-16T09:31:41.204648+0000","last_undegraded":"2024-09-16T09:31:41.204648+0000","last_fullsized":"2024-09-16T09:31:41.204648+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:15.910824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:15.910824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":149,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.215606+0000","last_change":"2024-09-16T09:31:09.277239+0000","last_active":"2024-09-16T09:31:41.215606+0000","last_peered":"2024-09-16T09:31:41.215606+0000","last_clean":"2024-09-16T09:31:41.215606+0000","last_became_active":"2024-09-16T09:31:09.276042+0000","last_became_peered":"2024-09-16T09:31:09.276042+0000","last_unstale":"2024-09-16T09:31:41.215606+0000","last_undegraded":"2024-09-16T09:31:41.215606+0000","last_fullsized":"2024-09-16T09:31:41.215606+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:48.147759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:48.147759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:14.230973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"66'9","reported_seq":310,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.204969+0000","last_change":"2024-09-16T09:26:31.216495+0000","last_active":"2024-09-16T09:31:41.204969+0000","last_peered":"2024-09-16T09:31:41.204969+0000","last_clean":"2024-09-16T09:31:41.204969+0000","last_became_active":"2024-09-16T09:25:03.705807+0000","last_became_peered":"2024-09-16T09:25:03.705807+0000","last_unstale":"2024-09-16T09:31:41.204969+0000","last_undegraded":"2024-09-16T09:31:41.204969+0000","last_fullsized":"2024-09-16T09:31:41.204969+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:26:31.216298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:26:31.216298+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":297,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.220999+0000","last_change":"2024-09-16T09:26:23.796308+0000","last_active":"2024-09-16T09:31:41.220999+0000","last_peered":"2024-09-16T09:31:41.220999+0000","last_clean":"2024-09-16T09:31:41.220999+0000","last_became_active":"2024-09-16T09:25:01.706762+0000","last_became_peered":"2024-09-16T09:25:01.706762+0000","last_unstale":"2024-09-16T09:31:41.220999+0000","last_undegraded":"2024-09-16T09:31:41.220999+0000","last_fullsized":"2024-09-16T09:31:41.220999+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:23.796231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:23.796231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":134,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.214736+0000","last_change":"2024-09-16T09:28:52.983144+0000","last_active":"2024-09-16T09:31:41.214736+0000","last_peered":"2024-09-16T09:31:41.214736+0000","last_clean":"2024-09-16T09:31:41.214736+0000","last_became_active":"2024-09-16T09:28:52.982865+0000","last_became_peered":"2024-09-16T09:28:52.982865+0000","last_unstale":"2024-09-16T09:31:41.214736+0000","last_undegraded":"2024-09-16T09:31:41.214736+0000","last_fullsized":"2024-09-16T09:31:41.214736+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:44.241942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:44.241942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:09.215097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":294,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.213896+0000","last_change":"2024-09-16T09:28:52.552567+0000","last_active":"2024-09-16T09:31:41.213896+0000","last_peered":"2024-09-16T09:31:41.213896+0000","last_clean":"2024-09-16T09:31:41.213896+0000","last_became_active":"2024-09-16T09:25:07.715301+0000","last_became_peered":"2024-09-16T09:25:07.715301+0000","last_unstale":"2024-09-16T09:31:41.213896+0000","last_undegraded":"2024-09-16T09:31:41.213896+0000","last_fullsized":"2024-09-16T09:31:41.213896+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:52.552495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:28:52.552495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.14","version":"73'79","reported_seq":89,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.220315+0000","last_change":"2024-09-16T09:31:14.201538+0000","last_active":"2024-09-16T09:31:41.220315+0000","last_peered":"2024-09-16T09:31:41.220315+0000","last_clean":"2024-09-16T09:31:41.220315+0000","last_became_active":"2024-09-16T09:30:10.700866+0000","last_became_peered":"2024-09-16T09:30:10.700866+0000","last_unstale":"2024-09-16T09:31:41.220315+0000","last_undegraded":"2024-09-16T09:31:41.220315+0000","last_fullsized":"2024-09-16T09:31:41.220315+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:53.916505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:53.916505+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:31:42.929883+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109227,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"66'11","reported_seq":181,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.218429+0000","last_change":"2024-09-16T09:28:55.643505+0000","last_active":"2024-09-16T09:31:41.218429+0000","last_peered":"2024-09-16T09:31:41.218429+0000","last_clean":"2024-09-16T09:31:41.218429+0000","last_became_active":"2024-09-16T09:28:52.971132+0000","last_became_peered":"2024-09-16T09:28:52.971132+0000","last_unstale":"2024-09-16T09:31:41.218429+0000","last_undegraded":"2024-09-16T09:31:41.218429+0000","last_fullsized":"2024-09-16T09:31:41.218429+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-16T09:28:55.643448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:55.643448+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":19,"num_read_kb":12,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":305,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.209045+0000","last_change":"2024-09-16T09:27:46.175159+0000","last_active":"2024-09-16T09:31:41.209045+0000","last_peered":"2024-09-16T09:31:41.209045+0000","last_clean":"2024-09-16T09:31:41.209045+0000","last_became_active":"2024-09-16T09:25:01.700733+0000","last_became_peered":"2024-09-16T09:25:01.700733+0000","last_unstale":"2024-09-16T09:31:41.209045+0000","last_undegraded":"2024-09-16T09:31:41.209045+0000","last_fullsized":"2024-09-16T09:31:41.209045+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:46.175093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:27:46.175093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":297,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.217500+0000","last_change":"2024-09-16T09:29:25.017736+0000","last_active":"2024-09-16T09:31:41.217500+0000","last_peered":"2024-09-16T09:31:41.217500+0000","last_clean":"2024-09-16T09:31:41.217500+0000","last_became_active":"2024-09-16T09:25:05.713479+0000","last_became_peered":"2024-09-16T09:25:05.713479+0000","last_unstale":"2024-09-16T09:31:41.217500+0000","last_undegraded":"2024-09-16T09:31:41.217500+0000","last_fullsized":"2024-09-16T09:31:41.217500+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:25.017610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:25.017610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":157,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.215142+0000","last_change":"2024-09-16T09:31:09.275121+0000","last_active":"2024-09-16T09:31:41.215142+0000","last_peered":"2024-09-16T09:31:41.215142+0000","last_clean":"2024-09-16T09:31:41.215142+0000","last_became_active":"2024-09-16T09:31:09.274797+0000","last_became_peered":"2024-09-16T09:31:09.274797+0000","last_unstale":"2024-09-16T09:31:41.215142+0000","last_undegraded":"2024-09-16T09:31:41.215142+0000","last_fullsized":"2024-09-16T09:31:41.215142+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:54.589258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:28:54.589258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"66'4","reported_seq":38,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.210666+0000","last_change":"2024-09-16T09:31:09.401626+0000","last_active":"2024-09-16T09:31:41.210666+0000","last_peered":"2024-09-16T09:31:41.210666+0000","last_clean":"2024-09-16T09:31:41.210666+0000","last_became_active":"2024-09-16T09:31:09.273682+0000","last_became_peered":"2024-09-16T09:31:09.273682+0000","last_unstale":"2024-09-16T09:31:41.210666+0000","last_undegraded":"2024-09-16T09:31:41.210666+0000","last_fullsized":"2024-09-16T09:31:41.210666+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'4","last_scrub_stamp":"2024-09-16T09:26:27.473740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:26:27.473740+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":328,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.216003+0000","last_change":"2024-09-16T09:31:30.235761+0000","last_active":"2024-09-16T09:31:41.216003+0000","last_peered":"2024-09-16T09:31:41.216003+0000","last_clean":"2024-09-16T09:31:41.216003+0000","last_became_active":"2024-09-16T09:31:09.274719+0000","last_became_peered":"2024-09-16T09:31:09.274719+0000","last_unstale":"2024-09-16T09:31:41.216003+0000","last_undegraded":"2024-09-16T09:31:41.216003+0000","last_fullsized":"2024-09-16T09:31:41.216003+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:31:30.235692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:31:30.235692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:49.736157+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":306,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.212832+0000","last_change":"2024-09-16T09:29:53.244633+0000","last_active":"2024-09-16T09:31:41.212832+0000","last_peered":"2024-09-16T09:31:41.212832+0000","last_clean":"2024-09-16T09:31:41.212832+0000","last_became_active":"2024-09-16T09:25:05.712545+0000","last_became_peered":"2024-09-16T09:25:05.712545+0000","last_unstale":"2024-09-16T09:31:41.212832+0000","last_undegraded":"2024-09-16T09:31:41.212832+0000","last_fullsized":"2024-09-16T09:31:41.212832+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:53.244578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:53.244578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:07.420423+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":293,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.217831+0000","last_change":"2024-09-16T09:29:27.012810+0000","last_active":"2024-09-16T09:31:41.217831+0000","last_peered":"2024-09-16T09:31:41.217831+0000","last_clean":"2024-09-16T09:31:41.217831+0000","last_became_active":"2024-09-16T09:25:07.720117+0000","last_became_peered":"2024-09-16T09:25:07.720117+0000","last_unstale":"2024-09-16T09:31:41.217831+0000","last_undegraded":"2024-09-16T09:31:41.217831+0000","last_fullsized":"2024-09-16T09:31:41.217831+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:27.012756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:27.012756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":294,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.205144+0000","last_change":"2024-09-16T09:29:19.888825+0000","last_active":"2024-09-16T09:31:41.205144+0000","last_peered":"2024-09-16T09:31:41.205144+0000","last_clean":"2024-09-16T09:31:41.205144+0000","last_became_active":"2024-09-16T09:25:07.712594+0000","last_became_peered":"2024-09-16T09:25:07.712594+0000","last_unstale":"2024-09-16T09:31:41.205144+0000","last_undegraded":"2024-09-16T09:31:41.205144+0000","last_fullsized":"2024-09-16T09:31:41.205144+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:19.888759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:19.888759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":156,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.214474+0000","last_change":"2024-09-16T09:31:26.211625+0000","last_active":"2024-09-16T09:31:41.214474+0000","last_peered":"2024-09-16T09:31:41.214474+0000","last_clean":"2024-09-16T09:31:41.214474+0000","last_became_active":"2024-09-16T09:31:09.271662+0000","last_became_peered":"2024-09-16T09:31:09.271662+0000","last_unstale":"2024-09-16T09:31:41.214474+0000","last_undegraded":"2024-09-16T09:31:41.214474+0000","last_fullsized":"2024-09-16T09:31:41.214474+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:31:26.211363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:31:26.211363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:32.873753+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"66'11","reported_seq":322,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.213677+0000","last_change":"2024-09-16T09:27:57.486459+0000","last_active":"2024-09-16T09:31:41.213677+0000","last_peered":"2024-09-16T09:31:41.213677+0000","last_clean":"2024-09-16T09:31:41.213677+0000","last_became_active":"2024-09-16T09:25:03.707769+0000","last_became_peered":"2024-09-16T09:25:03.707769+0000","last_unstale":"2024-09-16T09:31:41.213677+0000","last_undegraded":"2024-09-16T09:31:41.213677+0000","last_fullsized":"2024-09-16T09:31:41.213677+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-16T09:27:57.486381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:57.486381+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:16.375833+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":291,"reported_epoch":197,"state":"active+clean","last_fresh":"2024-09-16T09:31:41.219679+0000","last_change":"2024-09-16T09:27:57.640962+0000","last_active":"2024-09-16T09:31:41.219679+0000","last_peered":"2024-09-16T09:31:41.219679+0000","last_clean":"2024-09-16T09:31:41.219679+0000","last_became_active":"2024-09-16T09:25:05.711952+0000","last_became_peered":"2024-09-16T09:25:05.711952+0000","last_unstale":"2024-09-16T09:31:41.219679+0000","last_undegraded":"2024-09-16T09:31:41.219679+0000","last_fullsized":"2024-09-16T09:31:41.219679+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:57.640756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:57.640756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:21.189003+0000","scrub_duration":44,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":16,"stat_sum":{"num_bytes":6966214680,"num_objects":106297,"num_object_clones":0,"num_object_copies":318891,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":7145,"num_objects_unfound":0,"num_objects_dirty":106297,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":142889,"num_write_kb":7973889,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7564,"num_bytes_recovered":495714304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":21400072192,"data_stored":21400059976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":77276,"ondisk_log_size":77276,"up":48,"acting":45,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":70,"ondisk_log_size":70,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":691,"num_read_kb":448,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":126,"num_bytes_recovered":3026,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":395,"ondisk_log_size":395,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1986,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":395,"num_read_kb":390,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":21,"stat_sum":{"num_bytes":806935,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":21,"num_read_kb":40,"num_write":21,"num_write_kb":296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":697708,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1659,"ondisk_log_size":1659,"up":63,"acting":63,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266773,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2314976,"kb_used_data":2149132,"kb_used_omap":1,"kb_used_meta":165822,"kb_avail":91430176,"statfs":{"total":95995035648,"available":93624500224,"internally_reserved":0,"allocated":2200711168,"data_stored":2200397144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":169802186},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495706,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2298884,"kb_used_data":2145524,"kb_used_omap":1,"kb_used_meta":153342,"kb_avail":91446268,"statfs":{"total":95995035648,"available":93640978432,"internally_reserved":0,"allocated":2197016576,"data_stored":2196726550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":157022662},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724641,"num_pgs":64,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3667860,"kb_used_data":3415936,"kb_used_omap":1,"kb_used_meta":251902,"kb_avail":90077292,"statfs":{"total":95995035648,"available":92239147008,"internally_reserved":0,"allocated":3497918464,"data_stored":3497616643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":257948102},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,1,2,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953574,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2644896,"kb_used_data":2451276,"kb_used_omap":1,"kb_used_meta":193598,"kb_avail":91100256,"statfs":{"total":95995035648,"available":93286662144,"internally_reserved":0,"allocated":2510106624,"data_stored":2509792640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":198244806},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,2,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149804,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2382568,"kb_used_data":2102868,"kb_used_omap":1,"kb_used_meta":279678,"kb_avail":91362584,"statfs":{"total":95995035648,"available":93555286016,"internally_reserved":0,"allocated":2153336832,"data_stored":2153014942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":286390727},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411441,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3207244,"kb_used_data":2937272,"kb_used_omap":1,"kb_used_meta":269950,"kb_avail":90537908,"statfs":{"total":95995035648,"available":92710817792,"internally_reserved":0,"allocated":3007766528,"data_stored":3007472362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":276429253},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,2,1,0,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574968,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3266616,"kb_used_data":2992484,"kb_used_omap":1,"kb_used_meta":274110,"kb_avail":90478536,"statfs":{"total":95995035648,"available":92650020864,"internally_reserved":0,"allocated":3064303616,"data_stored":3063964926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":280689093},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738494,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3132960,"kb_used_data":2844940,"kb_used_omap":1,"kb_used_meta":287998,"kb_avail":90612192,"statfs":{"total":95995035648,"available":92786884608,"internally_reserved":0,"allocated":2913218560,"data_stored":2912904778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":294910405},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2884370432,"data_stored":2884370432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3041787904,"data_stored":3041787904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2981367808,"data_stored":2981363736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2139553792,"data_stored":2139553792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":2486632448,"data_stored":2486632448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":3477082112,"data_stored":3477078040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":2191982592,"data_stored":2191982592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":2197295104,"data_stored":2197291032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T09:31:47.155 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph status --format=json 2024-09-16T09:31:47.174 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:31:47.207 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T09:31:47.242 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:47.243 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:47.244 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:31:47.253 INFO:teuthology.orchestra.run.smithi070.stderr:nodeep-scrub is set 2024-09-16T09:31:47.344 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:31:47.353 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:47 smithi084 ceph-mon[32629]: pgmap v314: 168 pgs: 1 active+recovery_wait+degraded, 4 active+clean+scrubbing, 3 active+recovering+undersized+remapped, 160 active+clean; 6.5 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.07k op/s; 1/319491 objects degraded (0.000%); 7145/319491 objects misplaced (2.236%); 6.1 MiB/s, 97 objects/s recovering 2024-09-16T09:31:47.353 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:47 smithi084 ceph-mon[32629]: from='client.15369 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:31:47.353 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:47 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3288491746' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:31:47.353 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:47 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:31:47.379 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:47.379 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:47.380 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:31:47.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:47 smithi070 ceph-mon[30388]: pgmap v314: 168 pgs: 1 active+recovery_wait+degraded, 4 active+clean+scrubbing, 3 active+recovering+undersized+remapped, 160 active+clean; 6.5 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.07k op/s; 1/319491 objects degraded (0.000%); 7145/319491 objects misplaced (2.236%); 6.1 MiB/s, 97 objects/s recovering 2024-09-16T09:31:47.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:47 smithi070 ceph-mon[30388]: from='client.15369 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:31:47.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:47 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3288491746' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:31:47.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:47 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:31:47.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:47 smithi070 ceph-mon[40691]: pgmap v314: 168 pgs: 1 active+recovery_wait+degraded, 4 active+clean+scrubbing, 3 active+recovering+undersized+remapped, 160 active+clean; 6.5 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.07k op/s; 1/319491 objects degraded (0.000%); 7145/319491 objects misplaced (2.236%); 6.1 MiB/s, 97 objects/s recovering 2024-09-16T09:31:47.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:47 smithi070 ceph-mon[40691]: from='client.15369 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:31:47.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:47 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3288491746' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:31:47.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:47 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:31:47.481 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:31:47.502 INFO:teuthology.orchestra.run.smithi084.stdout:ERROR: (22) Invalid argument 2024-09-16T09:31:47.502 INFO:teuthology.orchestra.run.smithi084.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:31:47.518 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:47.519 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:47.519 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:31:47.620 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:31:47.652 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:47.653 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:47.653 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:31:47.754 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:31:47.787 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:47.788 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:47.788 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:31:47.889 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:31:47.917 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:47.918 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:47.918 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:31:48.019 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:31:48.053 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:48.054 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:48.054 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:31:48.155 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:31:48.192 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:48.192 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:48.193 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:31:48.223 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.5 dump_blocked_ops 2024-09-16T09:31:48.293 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:31:48.353 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:48.354 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:48.354 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:31:48.455 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:31:48.492 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:48.493 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:48.493 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:31:48.594 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:31:48.641 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:48.642 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:48.642 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:31:48.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:48 smithi084 ceph-mon[32629]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:31:48.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:48 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-16T09:31:48.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:48 smithi084 ceph-mon[32629]: osdmap e198: 8 total, 8 up, 8 in 2024-09-16T09:31:48.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:48 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:31:48.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:48 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:31:48.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:48 smithi070 ceph-mon[30388]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:31:48.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:48 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-16T09:31:48.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:48 smithi070 ceph-mon[30388]: osdmap e198: 8 total, 8 up, 8 in 2024-09-16T09:31:48.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:48 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:31:48.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:48 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:31:48.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:48 smithi070 ceph-mon[40691]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:31:48.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:48 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-16T09:31:48.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:48 smithi070 ceph-mon[40691]: osdmap e198: 8 total, 8 up, 8 in 2024-09-16T09:31:48.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:48 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:31:48.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:48 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:31:48.686 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:31:48.743 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:31:48.773 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:48.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:48.774 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:31:48.831 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:31:48.844 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:31:48.874 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:31:48.906 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:48.907 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:48.908 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:31:49.008 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:31:49.040 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:49.041 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:49.042 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:31:49.142 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:31:49.181 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:49.182 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:49.183 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:31:49.283 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:31:49.322 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:49.324 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:49.324 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:31:49.333 INFO:teuthology.orchestra.run.smithi084.stdout:ERROR: (22) Invalid argument 2024-09-16T09:31:49.333 INFO:teuthology.orchestra.run.smithi084.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:31:49.425 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:31:49.460 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:49.460 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:49.461 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:31:49.529 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:31:49.530 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:31:49.560 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:31:49.560 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:49 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:31:49.560 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:49 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:31:49.560 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:49 smithi070 ceph-mon[30388]: pgmap v316: 168 pgs: 1 active+recovery_wait+degraded, 2 active+clean+scrubbing, 3 active+recovering+undersized+remapped, 162 active+clean; 6.4 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 778 op/s; 1/317331 objects degraded (0.000%); 7145/317331 objects misplaced (2.252%); 3.7 MiB/s, 59 objects/s recovering 2024-09-16T09:31:49.560 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:49 smithi070 ceph-mon[30388]: osdmap e199: 8 total, 8 up, 8 in 2024-09-16T09:31:49.560 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:49 smithi070 ceph-mon[30388]: Health check update: Degraded data redundancy: 1/317331 objects degraded (0.000%), 1 pg degraded (PG_DEGRADED) 2024-09-16T09:31:49.561 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:49 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:31:49.561 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:49 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:31:49.561 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:49 smithi070 ceph-mon[40691]: pgmap v316: 168 pgs: 1 active+recovery_wait+degraded, 2 active+clean+scrubbing, 3 active+recovering+undersized+remapped, 162 active+clean; 6.4 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 778 op/s; 1/317331 objects degraded (0.000%); 7145/317331 objects misplaced (2.252%); 3.7 MiB/s, 59 objects/s recovering 2024-09-16T09:31:49.561 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:49 smithi070 ceph-mon[40691]: osdmap e199: 8 total, 8 up, 8 in 2024-09-16T09:31:49.561 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:49 smithi070 ceph-mon[40691]: Health check update: Degraded data redundancy: 1/317331 objects degraded (0.000%), 1 pg degraded (PG_DEGRADED) 2024-09-16T09:31:49.562 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:31:49.562 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:31:49.595 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:49.596 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:49.596 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:31:49.596 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:31:49.596 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:31:49.631 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:31:49.631 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:31:49.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:49 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:31:49.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:49 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:31:49.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:49 smithi084 ceph-mon[32629]: pgmap v316: 168 pgs: 1 active+recovery_wait+degraded, 2 active+clean+scrubbing, 3 active+recovering+undersized+remapped, 162 active+clean; 6.4 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 778 op/s; 1/317331 objects degraded (0.000%); 7145/317331 objects misplaced (2.252%); 3.7 MiB/s, 59 objects/s recovering 2024-09-16T09:31:49.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:49 smithi084 ceph-mon[32629]: osdmap e199: 8 total, 8 up, 8 in 2024-09-16T09:31:49.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:49 smithi084 ceph-mon[32629]: Health check update: Degraded data redundancy: 1/317331 objects degraded (0.000%), 1 pg degraded (PG_DEGRADED) 2024-09-16T09:31:49.665 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:31:49.665 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:31:49.697 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:31:49.699 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:31:49.699 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:31:49.730 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:31:49.730 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:31:49.741 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:49.742 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:49.742 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:31:49.767 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:31:49.768 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:31:49.843 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:31:49.876 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:49.877 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:49.877 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:31:49.978 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:31:50.011 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:50.011 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:50.012 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:31:50.107 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.5 dump_historic_ops 2024-09-16T09:31:50.112 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:31:50.196 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:50.197 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:50.197 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:31:50.300 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:31:50.338 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:50.339 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:50.339 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:31:50.395 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:50 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/964370642' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:31:50.395 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:50 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/964370642' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:31:50.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:50 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/964370642' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:31:50.440 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:31:50.479 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:50.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:50.481 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:31:50.520 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:31:50.581 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:31:50.617 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:50.618 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:50.618 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:31:50.719 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:31:50.745 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:31:50.745 INFO:teuthology.orchestra.run.smithi070.stdout:{"fsid":"a1321834-740c-11ef-bceb-c7b262605968","health":{"status":"HEALTH_WARN","checks":{"OSDMAP_FLAGS":{"severity":"HEALTH_WARN","summary":{"message":"noscrub,nodeep-scrub flag(s) set","count":20},"muted":false},"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 1/302955 objects degraded (0.000%), 1 pg degraded","count":1},"muted":false}},"mutes":[]},"election_epoch":14,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":658,"monmap":{"epoch":3,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":199,"num_osds":8,"num_up_osds":8,"osd_up_since":1726478691,"num_in_osds":8,"osd_in_since":1726478668,"num_remapped_pgs":3},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":164},{"state_name":"active+recovering+undersized+remapped","count":3},{"state_name":"active+recovery_wait+degraded","count":1}],"num_pgs":168,"num_pools":7,"num_objects":100985,"data_bytes":6605793322,"bytes_used":22429982720,"bytes_avail":745530302464,"bytes_total":767960285184,"degraded_objects":1,"degraded_total":302955,"degraded_ratio":3.3008202538330776e-06,"misplaced_objects":6599,"misplaced_total":302955,"misplaced_ratio":0.021782112855044479,"recovering_objects_per_sec":128,"recovering_bytes_per_sec":8399693,"recovering_keys_per_sec":0,"num_objects_recovered":1026,"num_bytes_recovered":67239936,"num_keys_recovered":0,"read_bytes_sec":1279,"write_bytes_sec":0,"read_op_per_sec":1,"write_op_per_sec":1375},"fsmap":{"epoch":1,"btime":"2024-09-16T09:19:14:121324+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":1,"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"services":{"dashboard":"https://172.21.15.70:8443/","prometheus":"http://172.21.15.70:9283/"}},"servicemap":{"epoch":14,"modified":"2024-09-16T09:26:33.498407+0000","services":{"rgw":{"daemons":{"summary":"","14475":{"start_epoch":12,"start_stamp":"2024-09-16T09:25:08.794234+0000","gid":14475,"addr":"172.21.15.70:0/1049757559","metadata":{"arch":"x86_64","ceph_release":"squid","ceph_version":"ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev)","ceph_version_short":"19.3.0-4917-g8293d73f","container_hostname":"smithi070","container_image":"quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce","cpu":"Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz","distro":"centos","distro_description":"CentOS Stream 9","distro_version":"9","frontend_config#0":"beast port=80","frontend_type#0":"beast","hostname":"smithi070","id":"foo.a","kernel_description":"#1 SMP PREEMPT_DYNAMIC Thu Sep 5 07:54:07 UTC 2024","kernel_version":"5.14.0-505.el9.x86_64","mem_swap_kb":"0","mem_total_kb":"32487212","num_handles":"1","os":"Linux","pid":"7","realm_id":"","realm_name":"","zone_id":"09633d1c-04b1-424a-bf26-5fa0f61f30f8","zone_name":"default","zonegroup_id":"c6682a31-37e4-465e-b890-750b49769dfa","zonegroup_name":"default"},"task_status":{}}}}}},"progress_events":{}} 2024-09-16T09:31:50.755 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:50.756 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:50.756 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:31:50.807 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd unset noscrub 2024-09-16T09:31:50.858 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:31:50.914 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:50.914 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:50.914 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:31:51.015 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:31:51.045 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:51.046 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:51.046 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:31:51.146 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:31:51.186 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:51.187 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:51.187 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:31:51.263 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-09-16T09:31:51.263 INFO:teuthology.orchestra.run.smithi084.stdout: "size": 20, 2024-09-16T09:31:51.264 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 600, 2024-09-16T09:31:51.264 INFO:teuthology.orchestra.run.smithi084.stdout: "ops": [ 2024-09-16T09:31:51.264 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.264 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "MOSDPGPushReply(8.0 168/167 [PushReplyOp(8:08f3f0b5:::benchmark_data_smithi070_107919_object491:head)])", 2024-09-16T09:31:51.264 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:30:23.824987+0000", 2024-09-16T09:31:51.264 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 87.435185462000007, 2024-09-16T09:31:51.264 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.13437887400000001, 2024-09-16T09:31:51.264 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:31:51.264 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:31:51.264 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "reached pg", 2024-09-16T09:31:51.264 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:31:51.264 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.265 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:31:51.265 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:23.824987+0000", 2024-09-16T09:31:51.265 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:31:51.265 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.265 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.265 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:31:51.265 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:23.824987+0000", 2024-09-16T09:31:51.265 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:31:51.265 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.265 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.265 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:31:51.265 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:23.824988+0000", 2024-09-16T09:31:51.265 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.5900000000000005e-07 2024-09-16T09:31:51.266 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.266 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.266 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:31:51.266 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:23.824994+0000", 2024-09-16T09:31:51.266 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.088e-06 2024-09-16T09:31:51.266 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.266 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.266 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:31:51.266 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:23.825004+0000", 2024-09-16T09:31:51.266 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0492999999999999e-05 2024-09-16T09:31:51.266 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.266 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.266 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:31:51.267 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:23.825012+0000", 2024-09-16T09:31:51.267 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.277e-06 2024-09-16T09:31:51.267 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.267 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.267 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:31:51.267 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:23.953117+0000", 2024-09-16T09:31:51.267 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.12810587600000001 2024-09-16T09:31:51.267 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.267 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.268 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:31:51.268 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:23.959366+0000", 2024-09-16T09:31:51.268 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0062481810000000002 2024-09-16T09:31:51.268 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:31:51.268 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:31:51.268 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:31:51.268 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.268 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.268 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "MOSDPGPushReply(8.8 168/167 [PushReplyOp(8:11a4176e:::benchmark_data_smithi070_107919_object218:head)])", 2024-09-16T09:31:51.268 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:30:23.878467+0000", 2024-09-16T09:31:51.268 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 87.381704855999999, 2024-09-16T09:31:51.268 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.12165545899999999, 2024-09-16T09:31:51.268 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:31:51.269 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:31:51.269 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "reached pg", 2024-09-16T09:31:51.269 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:31:51.269 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.269 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:31:51.269 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:23.878467+0000", 2024-09-16T09:31:51.269 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:31:51.269 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.269 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.269 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:31:51.269 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:23.878467+0000", 2024-09-16T09:31:51.269 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:31:51.269 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.270 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.270 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:31:51.270 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:23.878468+0000", 2024-09-16T09:31:51.270 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.0100000000000005e-07 2024-09-16T09:31:51.270 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.270 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.270 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:31:51.270 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:23.878473+0000", 2024-09-16T09:31:51.270 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.6460000000000001e-06 2024-09-16T09:31:51.270 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.270 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.270 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:31:51.270 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:23.878480+0000", 2024-09-16T09:31:51.270 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.6480000000000003e-06 2024-09-16T09:31:51.271 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.271 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.271 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:31:51.271 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:23.878486+0000", 2024-09-16T09:31:51.271 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.7810000000000002e-06 2024-09-16T09:31:51.271 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.271 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.271 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:31:51.271 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:23.995434+0000", 2024-09-16T09:31:51.271 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.11694816800000001 2024-09-16T09:31:51.271 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.271 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.271 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:31:51.272 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:24.000123+0000", 2024-09-16T09:31:51.272 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0046886150000000001 2024-09-16T09:31:51.272 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:31:51.272 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:31:51.272 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:31:51.272 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.272 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.272 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "MOSDPGPushReply(8.8 169/167 [PushReplyOp(8:156d5b64:::benchmark_data_smithi070_107919_object361:head)])", 2024-09-16T09:31:51.272 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:30:24.607597+0000", 2024-09-16T09:31:51.272 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 86.652575294000002, 2024-09-16T09:31:51.272 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.116072945, 2024-09-16T09:31:51.272 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:31:51.272 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:31:51.272 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "reached pg", 2024-09-16T09:31:51.273 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:31:51.273 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.273 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:31:51.273 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:24.607597+0000", 2024-09-16T09:31:51.273 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:31:51.273 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.273 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.273 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:31:51.273 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:24.607597+0000", 2024-09-16T09:31:51.273 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:31:51.273 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.273 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.273 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:31:51.274 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:24.607598+0000", 2024-09-16T09:31:51.274 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.5099999999999999e-07 2024-09-16T09:31:51.274 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.274 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.274 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:31:51.274 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:24.607603+0000", 2024-09-16T09:31:51.274 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.4859999999999997e-06 2024-09-16T09:31:51.274 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.274 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.274 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:31:51.274 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:24.607613+0000", 2024-09-16T09:31:51.274 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0022e-05 2024-09-16T09:31:51.274 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.274 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.275 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:31:51.275 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:24.607621+0000", 2024-09-16T09:31:51.275 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.7230000000000004e-06 2024-09-16T09:31:51.275 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.275 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.275 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:31:51.275 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:24.719485+0000", 2024-09-16T09:31:51.275 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.111864667 2024-09-16T09:31:51.275 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.275 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.275 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:31:51.275 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:24.723670+0000", 2024-09-16T09:31:51.275 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0041843959999999999 2024-09-16T09:31:51.276 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:31:51.276 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:31:51.276 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:31:51.276 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.276 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.276 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "MOSDPGPushReply(8.8 169/167 [PushReplyOp(8:10968269:::benchmark_data_smithi070_107919_object493:head)])", 2024-09-16T09:31:51.276 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:30:25.294720+0000", 2024-09-16T09:31:51.276 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 85.965452490000004, 2024-09-16T09:31:51.276 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.110359677, 2024-09-16T09:31:51.276 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:31:51.276 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:31:51.277 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "reached pg", 2024-09-16T09:31:51.277 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:31:51.277 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.277 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:31:51.277 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:25.294720+0000", 2024-09-16T09:31:51.277 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:31:51.277 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.277 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.277 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:31:51.277 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:25.294720+0000", 2024-09-16T09:31:51.277 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:31:51.277 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.277 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.277 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:31:51.278 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:25.294720+0000", 2024-09-16T09:31:51.278 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.0199999999999998e-07 2024-09-16T09:31:51.278 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.278 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.278 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:31:51.278 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:25.294724+0000", 2024-09-16T09:31:51.278 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.8890000000000004e-06 2024-09-16T09:31:51.278 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.278 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.278 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:31:51.278 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:25.294731+0000", 2024-09-16T09:31:51.278 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.6649999999999997e-06 2024-09-16T09:31:51.278 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.278 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.279 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:31:51.279 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:25.294736+0000", 2024-09-16T09:31:51.279 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.4979999999999997e-06 2024-09-16T09:31:51.279 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.279 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.280 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:31:51.280 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:25.399626+0000", 2024-09-16T09:31:51.280 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.104890238 2024-09-16T09:31:51.280 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.280 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.280 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:31:51.280 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:25.405079+0000", 2024-09-16T09:31:51.280 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0054530849999999999 2024-09-16T09:31:51.280 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:31:51.280 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:31:51.280 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:31:51.281 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.281 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.281 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "MOSDPGPushReply(8.0 171/167 [PushReplyOp(8:032c0fa5:::benchmark_data_smithi070_107919_object1896:head)])", 2024-09-16T09:31:51.281 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:30:27.623384+0000", 2024-09-16T09:31:51.281 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 83.636788085999996, 2024-09-16T09:31:51.281 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.104204386, 2024-09-16T09:31:51.281 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:31:51.281 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:31:51.281 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "reached pg", 2024-09-16T09:31:51.281 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:31:51.281 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.281 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:31:51.281 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:27.623384+0000", 2024-09-16T09:31:51.281 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:31:51.282 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.282 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.282 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:31:51.282 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:27.623384+0000", 2024-09-16T09:31:51.282 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:31:51.282 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.282 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.282 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:31:51.282 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:27.623384+0000", 2024-09-16T09:31:51.282 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.5999999999999999e-07 2024-09-16T09:31:51.282 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.282 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.282 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:31:51.282 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:27.623388+0000", 2024-09-16T09:31:51.283 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.5980000000000001e-06 2024-09-16T09:31:51.283 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.283 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.283 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:31:51.283 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:27.623395+0000", 2024-09-16T09:31:51.283 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.9909999999999996e-06 2024-09-16T09:31:51.283 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.283 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.283 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:31:51.283 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:27.623401+0000", 2024-09-16T09:31:51.283 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.959e-06 2024-09-16T09:31:51.283 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.283 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.283 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:31:51.284 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:27.714155+0000", 2024-09-16T09:31:51.284 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.090754192999999997 2024-09-16T09:31:51.284 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.284 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.284 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:31:51.284 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:27.727588+0000", 2024-09-16T09:31:51.284 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.013433285 2024-09-16T09:31:51.284 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:31:51.284 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:31:51.284 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:31:51.284 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.284 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.284 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "MOSDPGPushReply(8.0 171/167 [PushReplyOp(8:03e7bdc2:::benchmark_data_smithi070_107919_object2053:head)])", 2024-09-16T09:31:51.284 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:30:28.027268+0000", 2024-09-16T09:31:51.285 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 83.232903851000003, 2024-09-16T09:31:51.285 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.14247446499999999, 2024-09-16T09:31:51.285 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:31:51.285 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:31:51.285 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "reached pg", 2024-09-16T09:31:51.285 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:31:51.285 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.285 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:31:51.285 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.027268+0000", 2024-09-16T09:31:51.285 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:31:51.285 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.285 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.285 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:31:51.286 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.027268+0000", 2024-09-16T09:31:51.286 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:31:51.286 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.286 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.286 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:31:51.286 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.027269+0000", 2024-09-16T09:31:51.286 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.5800000000000001e-07 2024-09-16T09:31:51.286 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.286 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.286 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:31:51.286 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.027273+0000", 2024-09-16T09:31:51.286 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.3900000000000003e-06 2024-09-16T09:31:51.286 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.287 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.287 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:31:51.287 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.027281+0000", 2024-09-16T09:31:51.287 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.9740000000000002e-06 2024-09-16T09:31:51.287 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.287 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.287 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:31:51.287 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.027288+0000", 2024-09-16T09:31:51.287 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.8689999999999998e-06 2024-09-16T09:31:51.287 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.287 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.287 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:31:51.287 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.165751+0000", 2024-09-16T09:31:51.287 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.13846350599999999 2024-09-16T09:31:51.288 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.288 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.288 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:31:51.288 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.169743+0000", 2024-09-16T09:31:51.288 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0039914679999999998 2024-09-16T09:31:51.288 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:31:51.288 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:31:51.288 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:31:51.288 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.288 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.288 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.25064.0:59358 8.8 8:119768b4:::benchmark_data_smithi070_107919_object59357:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e171)", 2024-09-16T09:31:51.288 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:30:28.069811+0000", 2024-09-16T09:31:51.288 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 83.190361103000001, 2024-09-16T09:31:51.288 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.097315094000000005, 2024-09-16T09:31:51.289 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:31:51.289 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:31:51.289 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:31:51.289 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:31:51.289 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.25064", 2024-09-16T09:31:51.289 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/2578256439", 2024-09-16T09:31:51.289 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 59358 2024-09-16T09:31:51.289 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.289 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:31:51.289 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.289 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:31:51.289 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.069811+0000", 2024-09-16T09:31:51.290 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:31:51.290 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.290 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.290 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:31:51.290 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.069811+0000", 2024-09-16T09:31:51.290 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:31:51.290 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.290 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.290 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:31:51.290 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.069813+0000", 2024-09-16T09:31:51.290 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.641e-06 2024-09-16T09:31:51.290 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.291 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.291 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:31:51.291 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.069845+0000", 2024-09-16T09:31:51.291 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.1977999999999998e-05 2024-09-16T09:31:51.291 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.291 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.291 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:31:51.291 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.069859+0000", 2024-09-16T09:31:51.291 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.4438e-05 2024-09-16T09:31:51.291 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.291 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.291 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:31:51.291 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.069865+0000", 2024-09-16T09:31:51.291 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.7409999999999998e-06 2024-09-16T09:31:51.292 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.292 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.292 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:31:51.292 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.126331+0000", 2024-09-16T09:31:51.292 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.056465818000000001 2024-09-16T09:31:51.292 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.292 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.292 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:31:51.292 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.126477+0000", 2024-09-16T09:31:51.292 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000146179 2024-09-16T09:31:51.292 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.292 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.292 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,7", 2024-09-16T09:31:51.292 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.126622+0000", 2024-09-16T09:31:51.293 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00014503899999999999 2024-09-16T09:31:51.293 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.293 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.293 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:31:51.293 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.130879+0000", 2024-09-16T09:31:51.293 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0042567880000000001 2024-09-16T09:31:51.293 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.293 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.293 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:31:51.293 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.137098+0000", 2024-09-16T09:31:51.293 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.006219423 2024-09-16T09:31:51.293 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.293 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.293 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:31:51.294 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.158728+0000", 2024-09-16T09:31:51.294 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.02162967 2024-09-16T09:31:51.294 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.294 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.294 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:31:51.294 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.158785+0000", 2024-09-16T09:31:51.294 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.7704000000000002e-05 2024-09-16T09:31:51.294 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.294 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.294 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:31:51.294 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.167126+0000", 2024-09-16T09:31:51.295 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0083406750000000005 2024-09-16T09:31:51.295 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:31:51.295 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:31:51.295 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:31:51.295 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.295 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.295 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.25064.0:59367 8.8 8:18af80a6:::benchmark_data_smithi070_107919_object59366:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e171)", 2024-09-16T09:31:51.295 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:30:28.075551+0000", 2024-09-16T09:31:51.295 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 83.184620967000001, 2024-09-16T09:31:51.295 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.096823518999999997, 2024-09-16T09:31:51.295 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:31:51.295 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:31:51.295 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:31:51.296 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:31:51.296 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.25064", 2024-09-16T09:31:51.296 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/2578256439", 2024-09-16T09:31:51.296 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 59367 2024-09-16T09:31:51.296 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.296 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:31:51.296 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.296 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:31:51.296 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.075551+0000", 2024-09-16T09:31:51.296 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:31:51.296 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.296 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.296 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:31:51.296 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.075551+0000", 2024-09-16T09:31:51.297 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:31:51.297 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.297 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.297 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:31:51.297 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.075552+0000", 2024-09-16T09:31:51.297 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.6499999999999998e-07 2024-09-16T09:31:51.297 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.297 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.297 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:31:51.297 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.075606+0000", 2024-09-16T09:31:51.297 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.3754000000000001e-05 2024-09-16T09:31:51.297 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.297 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.298 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:31:51.298 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.075615+0000", 2024-09-16T09:31:51.298 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.6719999999999999e-06 2024-09-16T09:31:51.298 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.298 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.298 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:31:51.298 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.075622+0000", 2024-09-16T09:31:51.298 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.8329999999999996e-06 2024-09-16T09:31:51.298 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.298 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.298 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:31:51.298 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.130931+0000", 2024-09-16T09:31:51.298 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.055308615999999998 2024-09-16T09:31:51.299 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.299 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.299 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:31:51.299 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.131073+0000", 2024-09-16T09:31:51.299 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00014213299999999999 2024-09-16T09:31:51.299 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.299 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.299 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,7", 2024-09-16T09:31:51.299 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.131224+0000", 2024-09-16T09:31:51.299 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00015121500000000001 2024-09-16T09:31:51.299 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.299 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.299 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:31:51.300 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.137226+0000", 2024-09-16T09:31:51.300 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0060017100000000004 2024-09-16T09:31:51.300 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.300 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.300 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:31:51.300 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.139735+0000", 2024-09-16T09:31:51.300 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0025091480000000001 2024-09-16T09:31:51.300 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.300 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.300 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:31:51.300 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.169289+0000", 2024-09-16T09:31:51.300 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.029553890999999999 2024-09-16T09:31:51.300 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.301 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.301 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:31:51.301 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.169354+0000", 2024-09-16T09:31:51.301 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.4559000000000003e-05 2024-09-16T09:31:51.301 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.301 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.301 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:31:51.301 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.172375+0000", 2024-09-16T09:31:51.301 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0030211230000000001 2024-09-16T09:31:51.301 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:31:51.301 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:31:51.301 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:31:51.301 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.301 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.302 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.25064.0:59370 8.8 8:17355324:::benchmark_data_smithi070_107919_object59369:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e171)", 2024-09-16T09:31:51.302 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:30:28.078337+0000", 2024-09-16T09:31:51.302 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 83.181835011999993, 2024-09-16T09:31:51.302 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.104820753, 2024-09-16T09:31:51.302 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:31:51.302 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:31:51.302 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:31:51.302 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:31:51.302 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.25064", 2024-09-16T09:31:51.302 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/2578256439", 2024-09-16T09:31:51.302 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 59370 2024-09-16T09:31:51.302 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.302 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:31:51.302 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.303 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:31:51.303 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.078337+0000", 2024-09-16T09:31:51.303 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:31:51.303 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.303 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.303 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:31:51.303 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.078337+0000", 2024-09-16T09:31:51.303 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:31:51.303 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.303 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.303 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:31:51.303 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.078338+0000", 2024-09-16T09:31:51.303 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.054e-06 2024-09-16T09:31:51.303 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.304 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.304 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:31:51.304 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.078394+0000", 2024-09-16T09:31:51.304 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.6239999999999997e-05 2024-09-16T09:31:51.304 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.304 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.304 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:31:51.304 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.078406+0000", 2024-09-16T09:31:51.304 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1309e-05 2024-09-16T09:31:51.304 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.304 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.304 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:31:51.304 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.078414+0000", 2024-09-16T09:31:51.304 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.5339999999999999e-06 2024-09-16T09:31:51.305 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.305 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.305 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:31:51.305 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.137262+0000", 2024-09-16T09:31:51.305 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.058847305000000003 2024-09-16T09:31:51.305 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.305 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.305 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:31:51.305 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.137457+0000", 2024-09-16T09:31:51.305 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000195613 2024-09-16T09:31:51.305 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.305 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.305 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,7", 2024-09-16T09:31:51.305 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.137628+0000", 2024-09-16T09:31:51.306 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00017081999999999999 2024-09-16T09:31:51.306 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.306 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.306 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:31:51.306 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.146675+0000", 2024-09-16T09:31:51.306 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0090471179999999998 2024-09-16T09:31:51.306 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.306 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.306 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:31:51.306 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.146722+0000", 2024-09-16T09:31:51.306 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.7058000000000002e-05 2024-09-16T09:31:51.306 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.306 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.306 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:31:51.307 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.179468+0000", 2024-09-16T09:31:51.307 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.032746022 2024-09-16T09:31:51.307 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.307 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.307 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:31:51.307 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.179514+0000", 2024-09-16T09:31:51.307 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.6162999999999998e-05 2024-09-16T09:31:51.307 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.307 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.307 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:31:51.307 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.183158+0000", 2024-09-16T09:31:51.307 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0036435170000000002 2024-09-16T09:31:51.307 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:31:51.307 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:31:51.308 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:31:51.308 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.308 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.308 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.25064.0:59373 8.8 8:14d238bf:::benchmark_data_smithi070_107919_object59372:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e171)", 2024-09-16T09:31:51.308 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:30:28.080847+0000", 2024-09-16T09:31:51.308 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 83.179324704999999, 2024-09-16T09:31:51.308 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.105629741, 2024-09-16T09:31:51.308 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:31:51.308 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:31:51.308 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:31:51.308 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:31:51.308 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.25064", 2024-09-16T09:31:51.308 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/2578256439", 2024-09-16T09:31:51.308 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 59373 2024-09-16T09:31:51.309 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.309 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:31:51.309 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.309 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:31:51.309 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.080847+0000", 2024-09-16T09:31:51.309 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:31:51.309 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.309 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.309 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:31:51.309 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.080847+0000", 2024-09-16T09:31:51.309 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:31:51.309 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.309 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.310 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:31:51.310 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.080849+0000", 2024-09-16T09:31:51.310 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0979999999999999e-06 2024-09-16T09:31:51.310 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.310 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.310 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:31:51.310 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.080878+0000", 2024-09-16T09:31:51.310 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.9929000000000001e-05 2024-09-16T09:31:51.310 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.310 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.310 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:31:51.310 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.080889+0000", 2024-09-16T09:31:51.310 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0549e-05 2024-09-16T09:31:51.310 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.311 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.311 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:31:51.311 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.080896+0000", 2024-09-16T09:31:51.311 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.7649999999999996e-06 2024-09-16T09:31:51.311 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.311 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.311 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:31:51.311 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.139771+0000", 2024-09-16T09:31:51.311 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.058875234999999998 2024-09-16T09:31:51.311 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.311 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.311 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:31:51.311 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.140006+0000", 2024-09-16T09:31:51.312 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000235233 2024-09-16T09:31:51.312 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.312 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.312 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,7", 2024-09-16T09:31:51.312 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.140160+0000", 2024-09-16T09:31:51.312 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000153494 2024-09-16T09:31:51.312 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.312 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.312 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:31:51.312 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.155408+0000", 2024-09-16T09:31:51.312 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.015248177999999999 2024-09-16T09:31:51.312 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.312 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.313 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:31:51.313 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.155520+0000", 2024-09-16T09:31:51.313 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00011243699999999999 2024-09-16T09:31:51.313 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.313 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.313 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:31:51.313 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.183169+0000", 2024-09-16T09:31:51.313 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.027648546 2024-09-16T09:31:51.313 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.313 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.313 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:31:51.313 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.183216+0000", 2024-09-16T09:31:51.313 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.7154000000000003e-05 2024-09-16T09:31:51.314 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.314 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.314 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:31:51.314 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.186477+0000", 2024-09-16T09:31:51.314 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0032611229999999999 2024-09-16T09:31:51.314 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:31:51.314 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:31:51.314 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:31:51.314 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.314 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.314 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.25064.0:59381 8.8 8:13feaa7e:::benchmark_data_smithi070_107919_object59380:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e171)", 2024-09-16T09:31:51.314 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:30:28.091934+0000", 2024-09-16T09:31:51.314 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 83.168238212999995, 2024-09-16T09:31:51.315 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.096736820000000001, 2024-09-16T09:31:51.315 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:31:51.315 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:31:51.315 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:31:51.315 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:31:51.315 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.25064", 2024-09-16T09:31:51.315 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/2578256439", 2024-09-16T09:31:51.315 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 59381 2024-09-16T09:31:51.315 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.315 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:31:51.315 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.315 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:31:51.315 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.091934+0000", 2024-09-16T09:31:51.315 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:31:51.316 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.316 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.316 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:31:51.316 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.091934+0000", 2024-09-16T09:31:51.316 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:31:51.316 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.316 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.316 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:31:51.316 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.091936+0000", 2024-09-16T09:31:51.316 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.9240000000000001e-06 2024-09-16T09:31:51.316 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.316 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.316 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:31:51.316 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.091991+0000", 2024-09-16T09:31:51.317 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.5274e-05 2024-09-16T09:31:51.317 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.317 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.317 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:31:51.317 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.126533+0000", 2024-09-16T09:31:51.317 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.034541545999999999 2024-09-16T09:31:51.317 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.317 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.317 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:31:51.317 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.126541+0000", 2024-09-16T09:31:51.317 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.4989999999999994e-06 2024-09-16T09:31:51.317 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.317 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.318 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:31:51.318 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.142082+0000", 2024-09-16T09:31:51.318 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.015541038 2024-09-16T09:31:51.318 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.318 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.318 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:31:51.318 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.142210+0000", 2024-09-16T09:31:51.318 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00012744899999999999 2024-09-16T09:31:51.318 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.318 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.318 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,7", 2024-09-16T09:31:51.318 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.142356+0000", 2024-09-16T09:31:51.318 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00014586699999999999 2024-09-16T09:31:51.318 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.318 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.319 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:31:51.319 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.155569+0000", 2024-09-16T09:31:51.319 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.013213341999999999 2024-09-16T09:31:51.319 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.319 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.319 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:31:51.319 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.155625+0000", 2024-09-16T09:31:51.319 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.6264999999999997e-05 2024-09-16T09:31:51.319 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.319 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.319 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:31:51.319 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.186490+0000", 2024-09-16T09:31:51.319 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.030864756 2024-09-16T09:31:51.319 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.320 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.320 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:31:51.320 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.186542+0000", 2024-09-16T09:31:51.320 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.2240000000000001e-05 2024-09-16T09:31:51.320 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.320 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.320 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:31:51.320 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.188671+0000", 2024-09-16T09:31:51.320 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0021286199999999999 2024-09-16T09:31:51.320 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:31:51.320 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:31:51.320 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:31:51.320 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.320 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.321 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.25064.0:69521 8.8 8:118957f9:::benchmark_data_smithi070_107919_object69520:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e174)", 2024-09-16T09:31:51.321 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:30:36.635667+0000", 2024-09-16T09:31:51.321 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 74.624505075000002, 2024-09-16T09:31:51.321 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.10586423, 2024-09-16T09:31:51.321 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:31:51.321 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:31:51.321 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:31:51.321 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:31:51.321 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.25064", 2024-09-16T09:31:51.321 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/2578256439", 2024-09-16T09:31:51.321 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 69521 2024-09-16T09:31:51.321 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.321 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:31:51.321 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.321 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:31:51.322 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.635667+0000", 2024-09-16T09:31:51.322 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:31:51.323 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:31:51.323 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.323 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.323 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:31:51.323 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.635667+0000", 2024-09-16T09:31:51.323 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:31:51.323 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.324 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.324 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:31:51.324 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.635669+0000", 2024-09-16T09:31:51.324 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.823e-06 2024-09-16T09:31:51.324 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.324 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.324 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:31:51.324 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.635697+0000", 2024-09-16T09:31:51.324 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.7994999999999999e-05 2024-09-16T09:31:51.324 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.324 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.324 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:31:51.324 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.635938+0000", 2024-09-16T09:31:51.324 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00024088899999999999 2024-09-16T09:31:51.325 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.325 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.325 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:31:51.325 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.635946+0000", 2024-09-16T09:31:51.325 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.7619999999999995e-06 2024-09-16T09:31:51.325 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.328 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.328 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:31:51.328 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.636579+0000", 2024-09-16T09:31:51.328 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00063343600000000003 2024-09-16T09:31:51.328 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.328 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.328 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:31:51.328 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.636752+0000", 2024-09-16T09:31:51.328 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00017257700000000001 2024-09-16T09:31:51.329 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.329 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.329 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,7", 2024-09-16T09:31:51.329 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.636946+0000", 2024-09-16T09:31:51.329 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00019409599999999999 2024-09-16T09:31:51.329 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.329 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.329 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:31:51.329 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.645412+0000", 2024-09-16T09:31:51.329 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0084667370000000002 2024-09-16T09:31:51.329 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.329 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.329 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:31:51.329 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.645426+0000", 2024-09-16T09:31:51.330 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.3464e-05 2024-09-16T09:31:51.330 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.330 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.330 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:31:51.330 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.737605+0000", 2024-09-16T09:31:51.330 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.092179443999999999 2024-09-16T09:31:51.330 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.330 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.330 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:31:51.330 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.737701+0000", 2024-09-16T09:31:51.330 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.5650999999999996e-05 2024-09-16T09:31:51.330 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.330 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.330 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:31:51.331 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.741531+0000", 2024-09-16T09:31:51.331 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.003830356 2024-09-16T09:31:51.331 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:31:51.331 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:31:51.331 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:31:51.331 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.331 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.331 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.25064.0:69527 8.8 8:1bdf553a:::benchmark_data_smithi070_107919_object69526:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e174)", 2024-09-16T09:31:51.331 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:30:36.637575+0000", 2024-09-16T09:31:51.332 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 74.622596966000003, 2024-09-16T09:31:51.332 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.106121862, 2024-09-16T09:31:51.332 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:31:51.332 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:31:51.332 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:31:51.332 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:31:51.332 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.25064", 2024-09-16T09:31:51.332 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/2578256439", 2024-09-16T09:31:51.332 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 69527 2024-09-16T09:31:51.333 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.333 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:31:51.333 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.333 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:31:51.333 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.637575+0000", 2024-09-16T09:31:51.333 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:31:51.333 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.333 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.333 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:31:51.333 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.637575+0000", 2024-09-16T09:31:51.333 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:31:51.333 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.333 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.334 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:31:51.334 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.637576+0000", 2024-09-16T09:31:51.334 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.7700000000000004e-07 2024-09-16T09:31:51.334 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.334 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.334 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:31:51.334 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.637608+0000", 2024-09-16T09:31:51.334 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.2075e-05 2024-09-16T09:31:51.334 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.335 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.335 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:31:51.335 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.637614+0000", 2024-09-16T09:31:51.335 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.3690000000000003e-06 2024-09-16T09:31:51.335 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.335 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.335 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:31:51.335 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.637620+0000", 2024-09-16T09:31:51.335 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.6010000000000002e-06 2024-09-16T09:31:51.335 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.335 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.335 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:31:51.335 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.641975+0000", 2024-09-16T09:31:51.335 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0043553430000000002 2024-09-16T09:31:51.336 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.336 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.336 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:31:51.336 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.642107+0000", 2024-09-16T09:31:51.336 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00013171100000000001 2024-09-16T09:31:51.336 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.336 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.336 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,7", 2024-09-16T09:31:51.336 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.642260+0000", 2024-09-16T09:31:51.336 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00015355499999999999 2024-09-16T09:31:51.336 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.336 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.336 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:31:51.336 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.649121+0000", 2024-09-16T09:31:51.337 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0068601160000000003 2024-09-16T09:31:51.337 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.337 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.337 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:31:51.337 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.649142+0000", 2024-09-16T09:31:51.337 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.1352000000000002e-05 2024-09-16T09:31:51.337 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.337 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.337 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:31:51.337 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.741591+0000", 2024-09-16T09:31:51.337 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.092448942000000006 2024-09-16T09:31:51.337 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.337 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.338 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:31:51.338 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.741649+0000", 2024-09-16T09:31:51.338 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.7769000000000002e-05 2024-09-16T09:31:51.338 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.338 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.338 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:31:51.338 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.743697+0000", 2024-09-16T09:31:51.338 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0020484520000000001 2024-09-16T09:31:51.338 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:31:51.338 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:31:51.338 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:31:51.338 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.338 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.338 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "MOSDPGPushReply(8.8 174/167 [PushReplyOp(8:1c8c4107:::benchmark_data_smithi070_107919_object4388:head)])", 2024-09-16T09:31:51.338 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:30:37.857431+0000", 2024-09-16T09:31:51.339 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 73.402741328000005, 2024-09-16T09:31:51.339 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.115708227, 2024-09-16T09:31:51.339 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:31:51.339 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:31:51.339 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "reached pg", 2024-09-16T09:31:51.339 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:31:51.339 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.339 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:31:51.339 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:37.857431+0000", 2024-09-16T09:31:51.339 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:31:51.339 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.339 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.339 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:31:51.339 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:37.857431+0000", 2024-09-16T09:31:51.340 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:31:51.340 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.340 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.340 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:31:51.340 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:37.857431+0000", 2024-09-16T09:31:51.340 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.3000000000000002e-07 2024-09-16T09:31:51.340 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.340 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.340 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:31:51.340 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:37.857436+0000", 2024-09-16T09:31:51.340 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.4680000000000003e-06 2024-09-16T09:31:51.340 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.340 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.341 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:31:51.341 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:37.857444+0000", 2024-09-16T09:31:51.341 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.9799999999999998e-06 2024-09-16T09:31:51.341 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.341 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.341 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:31:51.341 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:37.857450+0000", 2024-09-16T09:31:51.341 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.0129999999999999e-06 2024-09-16T09:31:51.341 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.341 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.341 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:31:51.341 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:37.967635+0000", 2024-09-16T09:31:51.341 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.110185139 2024-09-16T09:31:51.342 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.342 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.342 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:31:51.342 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:37.973139+0000", 2024-09-16T09:31:51.342 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.005504297 2024-09-16T09:31:51.342 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:31:51.342 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:31:51.342 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:31:51.342 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.342 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.342 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "MOSDPGPushReply(8.0 176/167 [PushReplyOp(8:0296ea1e:::benchmark_data_smithi070_107919_object6825:head)])", 2024-09-16T09:31:51.342 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:30:39.823571+0000", 2024-09-16T09:31:51.342 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 71.436601002000003, 2024-09-16T09:31:51.342 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.13149123800000001, 2024-09-16T09:31:51.343 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:31:51.343 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:31:51.343 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "reached pg", 2024-09-16T09:31:51.343 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:31:51.343 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.343 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:31:51.343 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:39.823571+0000", 2024-09-16T09:31:51.343 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:31:51.343 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.343 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.343 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:31:51.343 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:39.823571+0000", 2024-09-16T09:31:51.343 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:31:51.343 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.343 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.344 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:31:51.344 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:39.823572+0000", 2024-09-16T09:31:51.344 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.066e-06 2024-09-16T09:31:51.344 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.344 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.344 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:31:51.344 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:39.823578+0000", 2024-09-16T09:31:51.344 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.2650000000000002e-06 2024-09-16T09:31:51.344 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.344 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.344 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:31:51.344 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:39.823592+0000", 2024-09-16T09:31:51.344 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.3444999999999999e-05 2024-09-16T09:31:51.344 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.345 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.345 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:31:51.345 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:39.823600+0000", 2024-09-16T09:31:51.345 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.7829999999999999e-06 2024-09-16T09:31:51.345 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.345 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.345 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:31:51.345 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:39.951069+0000", 2024-09-16T09:31:51.345 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.127468785 2024-09-16T09:31:51.345 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.345 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.345 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:31:51.345 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:39.955062+0000", 2024-09-16T09:31:51.345 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0039938939999999996 2024-09-16T09:31:51.346 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:31:51.346 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:31:51.346 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:31:51.346 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.346 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.346 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "MOSDPGPushReply(8.0 180/167 [PushReplyOp(8:0cd6d5e7:::benchmark_data_smithi070_107919_object12497:head)])", 2024-09-16T09:31:51.346 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:30:53.252304+0000", 2024-09-16T09:31:51.346 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 58.007867924000003, 2024-09-16T09:31:51.346 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.13780235099999999, 2024-09-16T09:31:51.346 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:31:51.346 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:31:51.346 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "reached pg", 2024-09-16T09:31:51.346 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:31:51.347 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.347 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:31:51.348 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:53.252304+0000", 2024-09-16T09:31:51.348 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:31:51.348 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.348 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.348 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:31:51.348 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:53.252304+0000", 2024-09-16T09:31:51.348 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:31:51.348 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.348 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.348 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:31:51.348 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:53.252305+0000", 2024-09-16T09:31:51.348 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.6000000000000004e-07 2024-09-16T09:31:51.348 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.349 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.349 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:31:51.349 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:53.252310+0000", 2024-09-16T09:31:51.349 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.8210000000000001e-06 2024-09-16T09:31:51.349 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.349 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.349 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:31:51.349 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:53.252317+0000", 2024-09-16T09:31:51.349 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.5410000000000003e-06 2024-09-16T09:31:51.349 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.349 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.349 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:31:51.349 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:53.252324+0000", 2024-09-16T09:31:51.350 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.3790000000000003e-06 2024-09-16T09:31:51.350 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.350 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.350 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:31:51.350 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:53.387129+0000", 2024-09-16T09:31:51.350 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.13480577499999999 2024-09-16T09:31:51.350 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.350 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.350 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:31:51.350 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:53.390107+0000", 2024-09-16T09:31:51.350 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0029772750000000001 2024-09-16T09:31:51.350 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:31:51.350 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:31:51.350 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:31:51.350 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.351 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.351 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "MOSDPGPushReply(8.8 180/167 [PushReplyOp(8:13aee2bf:::benchmark_data_smithi070_107919_object10150:head)])", 2024-09-16T09:31:51.351 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:30:53.264356+0000", 2024-09-16T09:31:51.351 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 57.995815710999999, 2024-09-16T09:31:51.351 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.15374286400000001, 2024-09-16T09:31:51.351 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:31:51.351 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:31:51.351 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "reached pg", 2024-09-16T09:31:51.351 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:31:51.351 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.351 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:31:51.351 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:53.264356+0000", 2024-09-16T09:31:51.351 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:31:51.352 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.352 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.352 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:31:51.352 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:53.264356+0000", 2024-09-16T09:31:51.352 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:31:51.352 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.352 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.352 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:31:51.352 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:53.264357+0000", 2024-09-16T09:31:51.352 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.7800000000000001e-07 2024-09-16T09:31:51.352 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.352 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.352 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:31:51.353 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:53.264363+0000", 2024-09-16T09:31:51.353 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.5629999999999999e-06 2024-09-16T09:31:51.353 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.353 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.353 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:31:51.353 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:53.264372+0000", 2024-09-16T09:31:51.353 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.3670000000000003e-06 2024-09-16T09:31:51.353 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.353 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.353 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:31:51.353 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:53.264380+0000", 2024-09-16T09:31:51.353 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.371e-06 2024-09-16T09:31:51.353 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.353 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.353 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:31:51.354 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:53.411592+0000", 2024-09-16T09:31:51.354 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.14721152800000001 2024-09-16T09:31:51.354 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.354 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.354 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:31:51.354 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:53.418099+0000", 2024-09-16T09:31:51.354 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0065073570000000001 2024-09-16T09:31:51.354 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:31:51.354 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:31:51.354 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:31:51.354 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.354 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.354 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "MOSDPGPushReply(8.0 180/167 [PushReplyOp(8:014384d9:::benchmark_data_smithi070_107919_object12876:head)])", 2024-09-16T09:31:51.354 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:30:54.077469+0000", 2024-09-16T09:31:51.355 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 57.182703394000001, 2024-09-16T09:31:51.355 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.10240925200000001, 2024-09-16T09:31:51.355 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:31:51.355 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:31:51.355 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "reached pg", 2024-09-16T09:31:51.355 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:31:51.355 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.355 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:31:51.355 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:54.077469+0000", 2024-09-16T09:31:51.355 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:31:51.355 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.355 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.355 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:31:51.355 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:54.077469+0000", 2024-09-16T09:31:51.355 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:31:51.356 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.356 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.356 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:31:51.356 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:54.077469+0000", 2024-09-16T09:31:51.356 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.0999999999999998e-07 2024-09-16T09:31:51.356 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.356 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.356 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:31:51.356 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:54.077476+0000", 2024-09-16T09:31:51.356 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.0340000000000002e-06 2024-09-16T09:31:51.356 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.356 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.356 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:31:51.356 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:54.077486+0000", 2024-09-16T09:31:51.357 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0925e-05 2024-09-16T09:31:51.357 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.357 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.357 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:31:51.357 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:54.077494+0000", 2024-09-16T09:31:51.357 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.0990000000000001e-06 2024-09-16T09:31:51.357 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.357 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.357 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:31:51.357 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:54.177117+0000", 2024-09-16T09:31:51.357 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.099623664000000001 2024-09-16T09:31:51.357 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.357 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.357 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:31:51.358 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:54.179878+0000", 2024-09-16T09:31:51.358 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0027608200000000002 2024-09-16T09:31:51.358 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:31:51.358 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:31:51.358 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:31:51.358 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.358 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.358 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.25064.0:106991 8.8 8:1dd7c7a4:::benchmark_data_smithi070_107919_object106990:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e186)", 2024-09-16T09:31:51.358 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:31:08.373667+0000", 2024-09-16T09:31:51.358 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 42.886505569000001, 2024-09-16T09:31:51.358 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1058546549999999, 2024-09-16T09:31:51.358 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:31:51.358 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:31:51.358 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:31:51.359 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:31:51.359 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.25064", 2024-09-16T09:31:51.359 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/2578256439", 2024-09-16T09:31:51.359 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 106991 2024-09-16T09:31:51.359 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.359 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:31:51.359 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.359 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:31:51.359 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:08.373667+0000", 2024-09-16T09:31:51.359 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:31:51.359 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.359 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.359 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:31:51.360 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:08.373667+0000", 2024-09-16T09:31:51.360 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:31:51.360 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.360 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.360 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:31:51.360 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:08.373667+0000", 2024-09-16T09:31:51.360 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.47e-07 2024-09-16T09:31:51.360 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.360 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.360 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:31:51.360 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:08.373710+0000", 2024-09-16T09:31:51.360 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.2564000000000001e-05 2024-09-16T09:31:51.360 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.361 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.361 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:31:51.361 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:08.373721+0000", 2024-09-16T09:31:51.361 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1045e-05 2024-09-16T09:31:51.361 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.361 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.361 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:31:51.361 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:08.373727+0000", 2024-09-16T09:31:51.361 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.5160000000000002e-06 2024-09-16T09:31:51.361 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.361 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.361 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:31:51.361 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:08.373779+0000", 2024-09-16T09:31:51.361 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.2237000000000001e-05 2024-09-16T09:31:51.362 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.362 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.362 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:31:51.362 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:08.373786+0000", 2024-09-16T09:31:51.362 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.1389999999999998e-06 2024-09-16T09:31:51.362 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.362 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.362 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:31:51.362 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:09.430374+0000", 2024-09-16T09:31:51.362 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0565881580000001 2024-09-16T09:31:51.362 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.362 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.362 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:31:51.362 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:09.430493+0000", 2024-09-16T09:31:51.362 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000118845 2024-09-16T09:31:51.363 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.363 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.363 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 3,7", 2024-09-16T09:31:51.363 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:09.430648+0000", 2024-09-16T09:31:51.363 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000155477 2024-09-16T09:31:51.363 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.363 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.363 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:31:51.363 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:09.436479+0000", 2024-09-16T09:31:51.363 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0058305120000000004 2024-09-16T09:31:51.363 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.363 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.363 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:31:51.363 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:09.436530+0000", 2024-09-16T09:31:51.364 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.1523999999999997e-05 2024-09-16T09:31:51.364 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.364 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.364 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:31:51.364 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:09.476247+0000", 2024-09-16T09:31:51.364 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.039716695000000003 2024-09-16T09:31:51.364 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.364 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.364 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:31:51.364 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:09.476349+0000", 2024-09-16T09:31:51.364 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000102117 2024-09-16T09:31:51.364 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.364 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.365 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:31:51.365 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:09.479521+0000", 2024-09-16T09:31:51.365 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0031719790000000001 2024-09-16T09:31:51.365 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:31:51.365 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:31:51.365 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:31:51.365 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.365 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.365 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "MOSDPGPushReply(8.0 187/186 [PushReplyOp(8:04425f91:::benchmark_data_smithi070_107919_object52626:head)])", 2024-09-16T09:31:51.365 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:31:10.232177+0000", 2024-09-16T09:31:51.365 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 41.027995363000002, 2024-09-16T09:31:51.365 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.13291494300000001, 2024-09-16T09:31:51.365 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:31:51.365 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:31:51.366 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "reached pg", 2024-09-16T09:31:51.366 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:31:51.366 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.366 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:31:51.366 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:10.232177+0000", 2024-09-16T09:31:51.366 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:31:51.366 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.366 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.366 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:31:51.366 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:10.232177+0000", 2024-09-16T09:31:51.366 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:31:51.366 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.366 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.366 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:31:51.367 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:10.232177+0000", 2024-09-16T09:31:51.367 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.2700000000000001e-07 2024-09-16T09:31:51.367 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.367 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.367 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:31:51.367 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:10.232179+0000", 2024-09-16T09:31:51.367 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.2340000000000001e-06 2024-09-16T09:31:51.367 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.367 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.367 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:31:51.367 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:10.232186+0000", 2024-09-16T09:31:51.367 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.3980000000000004e-06 2024-09-16T09:31:51.367 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.367 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.368 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:31:51.368 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:10.232190+0000", 2024-09-16T09:31:51.368 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.5990000000000004e-06 2024-09-16T09:31:51.368 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.368 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.368 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:31:51.368 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:10.358728+0000", 2024-09-16T09:31:51.368 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.12653795800000001 2024-09-16T09:31:51.368 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:51.368 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:51.368 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:31:51.368 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:10.365092+0000", 2024-09-16T09:31:51.368 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0063635269999999999 2024-09-16T09:31:51.368 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:31:51.369 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:31:51.369 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:31:51.369 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:31:51.369 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:31:51.369 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-09-16T09:31:51.405 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:51.405 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:51.406 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:31:51.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:51 smithi084 ceph-mon[32629]: pgmap v318: 168 pgs: 1 active+recovery_wait+degraded, 3 active+recovering+undersized+remapped, 164 active+clean; 6.2 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.38k op/s; 1/302955 objects degraded (0.000%); 6599/302955 objects misplaced (2.178%); 8.0 MiB/s, 128 objects/s recovering 2024-09-16T09:31:51.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:51 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/4232833954' entity='client.admin' cmd={"prefix": "status", "format": "json"} : dispatch 2024-09-16T09:31:51.506 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:31:51.543 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:51.543 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:51.544 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:31:51.591 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:51 smithi070 ceph-mon[30388]: pgmap v318: 168 pgs: 1 active+recovery_wait+degraded, 3 active+recovering+undersized+remapped, 164 active+clean; 6.2 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.38k op/s; 1/302955 objects degraded (0.000%); 6599/302955 objects misplaced (2.178%); 8.0 MiB/s, 128 objects/s recovering 2024-09-16T09:31:51.591 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:51 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/4232833954' entity='client.admin' cmd={"prefix": "status", "format": "json"} : dispatch 2024-09-16T09:31:51.591 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:51 smithi070 ceph-mon[40691]: pgmap v318: 168 pgs: 1 active+recovery_wait+degraded, 3 active+recovering+undersized+remapped, 164 active+clean; 6.2 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.38k op/s; 1/302955 objects degraded (0.000%); 6599/302955 objects misplaced (2.178%); 8.0 MiB/s, 128 objects/s recovering 2024-09-16T09:31:51.592 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:51 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/4232833954' entity='client.admin' cmd={"prefix": "status", "format": "json"} : dispatch 2024-09-16T09:31:51.644 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:31:51.649 INFO:tasks.cephadm.ceph_manager.ceph:making progress, resetting timeout 2024-09-16T09:31:51.649 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-16T09:31:51.675 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:51.675 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:51.676 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:31:51.681 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:31:51.777 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:31:51.805 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:51.806 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:51.807 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:31:51.907 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:31:51.937 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:51.938 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:51.938 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:31:52.039 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:31:52.067 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:52.068 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:52.068 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:31:52.169 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:31:52.170 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:31:51 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:31:51] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:31:52.200 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:52.201 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:52.201 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:31:52.295 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.6 dump_ops_in_flight 2024-09-16T09:31:52.302 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:31:52.372 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:52.374 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:52.374 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:31:52.384 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:31:52.475 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:31:52.521 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:52.522 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:52.522 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:31:52.623 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:31:52.664 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:52.664 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:52.665 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:31:52.694 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:31:52.766 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:31:52.801 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:52.801 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:52.802 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:31:52.902 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:31:52.911 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T09:31:52.945 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:52.946 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:52.946 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:31:53.047 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:31:53.084 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:53.084 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:53.085 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:31:53.160 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:31:52 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:31:53.185 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:31:53.246 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:53.246 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:53.247 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:31:53.313 INFO:teuthology.orchestra.run.smithi070.stderr:noscrub is unset 2024-09-16T09:31:53.331 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-09-16T09:31:53.331 INFO:teuthology.orchestra.run.smithi084.stdout: "ops": [ 2024-09-16T09:31:53.332 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:53.332 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.25064.0:154462 8.9 8:988c29df:::benchmark_data_smithi070_107919_object29867:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e199)", 2024-09-16T09:31:53.332 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:31:53.309511+0000", 2024-09-16T09:31:53.332 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 0.019797869999999999, 2024-09-16T09:31:53.332 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.019823206999999999, 2024-09-16T09:31:53.332 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:31:53.332 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:31:53.332 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "waiting for sub ops", 2024-09-16T09:31:53.332 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:31:53.332 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.25064", 2024-09-16T09:31:53.332 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/2578256439", 2024-09-16T09:31:53.332 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 154462 2024-09-16T09:31:53.332 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:53.333 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:31:53.333 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:53.333 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:31:53.333 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:53.309511+0000", 2024-09-16T09:31:53.333 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:31:53.333 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:53.333 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:53.333 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:31:53.333 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:53.309511+0000", 2024-09-16T09:31:53.333 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:31:53.333 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:53.333 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:53.334 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:31:53.334 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:53.309513+0000", 2024-09-16T09:31:53.334 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.734e-06 2024-09-16T09:31:53.334 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:53.334 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:53.334 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:31:53.334 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:53.309518+0000", 2024-09-16T09:31:53.334 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.5539999999999997e-06 2024-09-16T09:31:53.334 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:53.334 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:53.334 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:31:53.334 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:53.309533+0000", 2024-09-16T09:31:53.334 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.4296e-05 2024-09-16T09:31:53.335 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:53.335 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:53.335 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:31:53.335 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:53.309542+0000", 2024-09-16T09:31:53.335 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.9849999999999995e-06 2024-09-16T09:31:53.335 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:53.335 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:53.335 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:31:53.335 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:53.311136+0000", 2024-09-16T09:31:53.335 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0015947629999999999 2024-09-16T09:31:53.335 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:53.335 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:53.335 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:31:53.335 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:53.311270+0000", 2024-09-16T09:31:53.335 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00013371300000000001 2024-09-16T09:31:53.336 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:53.336 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:53.336 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,5", 2024-09-16T09:31:53.336 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:53.311380+0000", 2024-09-16T09:31:53.336 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000109341 2024-09-16T09:31:53.336 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:53.336 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:53.336 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:31:53.336 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:53.319336+0000", 2024-09-16T09:31:53.336 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0079565469999999996 2024-09-16T09:31:53.336 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:53.336 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:53.336 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:31:53.336 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:53.324858+0000", 2024-09-16T09:31:53.336 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0055218289999999998 2024-09-16T09:31:53.337 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:31:53.337 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:31:53.337 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:31:53.337 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:53.337 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:53.337 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.25064.0:154470 8.9 8:92703fb3:::benchmark_data_smithi070_107919_object29875:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e200)", 2024-09-16T09:31:53.337 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:31:53.317772+0000", 2024-09-16T09:31:53.337 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 0.011537102, 2024-09-16T09:31:53.337 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.011636189999999999, 2024-09-16T09:31:53.337 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:31:53.337 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:31:53.337 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "waiting for sub ops", 2024-09-16T09:31:53.337 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:31:53.338 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.25064", 2024-09-16T09:31:53.338 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/2578256439", 2024-09-16T09:31:53.338 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 154470 2024-09-16T09:31:53.338 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:53.338 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:31:53.338 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:53.338 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:31:53.338 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:53.317772+0000", 2024-09-16T09:31:53.338 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:31:53.338 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:53.338 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:53.338 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:31:53.338 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:53.317772+0000", 2024-09-16T09:31:53.338 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:31:53.338 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:53.339 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:53.339 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:31:53.339 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:53.317773+0000", 2024-09-16T09:31:53.339 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.3e-06 2024-09-16T09:31:53.339 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:53.339 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:53.339 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:31:53.339 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:53.317778+0000", 2024-09-16T09:31:53.339 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.4920000000000004e-06 2024-09-16T09:31:53.340 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:53.340 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:53.340 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:31:53.340 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:53.317787+0000", 2024-09-16T09:31:53.340 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.5589999999999994e-06 2024-09-16T09:31:53.340 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:53.340 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:53.340 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:31:53.340 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:53.317794+0000", 2024-09-16T09:31:53.340 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.2230000000000001e-06 2024-09-16T09:31:53.340 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:53.340 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:53.340 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:31:53.340 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:53.319366+0000", 2024-09-16T09:31:53.341 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.001571802 2024-09-16T09:31:53.341 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:53.341 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:53.341 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:31:53.341 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:53.319475+0000", 2024-09-16T09:31:53.341 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00010874600000000001 2024-09-16T09:31:53.341 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:53.341 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:53.341 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,5", 2024-09-16T09:31:53.341 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:53.319586+0000", 2024-09-16T09:31:53.341 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000111063 2024-09-16T09:31:53.341 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:53.341 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:53.341 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:31:53.342 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:53.327680+0000", 2024-09-16T09:31:53.342 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.008093767 2024-09-16T09:31:53.342 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:53.342 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:53.342 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:31:53.342 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:53.327769+0000", 2024-09-16T09:31:53.342 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.9384999999999999e-05 2024-09-16T09:31:53.342 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:31:53.342 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:31:53.342 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:31:53.342 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:53.342 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:53.342 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_repop(client.25064.0:154468 8.3 e200/152 8:c50989e4:::benchmark_data_smithi070_107919_object29873:head v 200'9720, mlcod=199'9717)", 2024-09-16T09:31:53.342 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:31:53.328502+0000", 2024-09-16T09:31:53.342 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 0.00080691400000000005, 2024-09-16T09:31:53.342 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00097263099999999995, 2024-09-16T09:31:53.343 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:31:53.343 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:31:53.343 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "started", 2024-09-16T09:31:53.343 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:31:53.343 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:53.343 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:31:53.343 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:53.328502+0000", 2024-09-16T09:31:53.343 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:31:53.343 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:53.343 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:53.343 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:31:53.343 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:53.328502+0000", 2024-09-16T09:31:53.343 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:31:53.343 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:53.343 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:53.344 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:31:53.344 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:53.328502+0000", 2024-09-16T09:31:53.344 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.7599999999999998e-07 2024-09-16T09:31:53.344 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:53.344 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:53.344 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:31:53.344 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:53.328506+0000", 2024-09-16T09:31:53.344 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.8130000000000002e-06 2024-09-16T09:31:53.344 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:53.344 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:53.344 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:31:53.344 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:53.328514+0000", 2024-09-16T09:31:53.344 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.6520000000000006e-06 2024-09-16T09:31:53.345 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:53.345 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:53.345 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:31:53.345 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:53.328520+0000", 2024-09-16T09:31:53.345 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.6729999999999997e-06 2024-09-16T09:31:53.345 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:53.345 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:53.345 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:31:53.345 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:53.328580+0000", 2024-09-16T09:31:53.345 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.0834000000000001e-05 2024-09-16T09:31:53.345 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:53.345 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:53.345 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:31:53.345 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:53.328599+0000", 2024-09-16T09:31:53.346 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.8929000000000001e-05 2024-09-16T09:31:53.346 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:31:53.346 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:31:53.346 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:31:53.346 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:53.346 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:53.346 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_repop(client.25064.0:154469 8.3 e200/152)", 2024-09-16T09:31:53.346 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:31:53.328537+0000", 2024-09-16T09:31:53.346 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 0.00077211599999999999, 2024-09-16T09:31:53.346 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00097980500000000004, 2024-09-16T09:31:53.346 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:31:53.346 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:31:53.346 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "queued for pg", 2024-09-16T09:31:53.347 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:31:53.347 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:53.347 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:31:53.347 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:53.328537+0000", 2024-09-16T09:31:53.347 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:31:53.347 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:53.347 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:53.347 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:31:53.347 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:53.328537+0000", 2024-09-16T09:31:53.347 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:31:53.347 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:53.347 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:53.347 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:31:53.348 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:53.328537+0000", 2024-09-16T09:31:53.348 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.8599999999999999e-07 2024-09-16T09:31:53.348 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:53.348 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:53.348 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:31:53.348 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:53.328540+0000", 2024-09-16T09:31:53.348 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.0199999999999999e-06 2024-09-16T09:31:53.348 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:53.348 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:53.348 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:31:53.348 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:53.328547+0000", 2024-09-16T09:31:53.348 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.9890000000000003e-06 2024-09-16T09:31:53.348 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:53.348 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:53.349 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:31:53.349 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:53.328552+0000", 2024-09-16T09:31:53.349 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.3329999999999998e-06 2024-09-16T09:31:53.349 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:31:53.349 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:31:53.349 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:31:53.349 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:53.349 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:53.349 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_repop(client.25064.0:154473 8.3 e200/152)", 2024-09-16T09:31:53.349 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:31:53.329220+0000", 2024-09-16T09:31:53.349 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 8.9378999999999999e-05, 2024-09-16T09:31:53.349 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00032967899999999999, 2024-09-16T09:31:53.349 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:31:53.350 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:31:53.350 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "queued for pg", 2024-09-16T09:31:53.350 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:31:53.350 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:53.350 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:31:53.350 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:53.329220+0000", 2024-09-16T09:31:53.350 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:31:53.350 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:53.350 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:53.350 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:31:53.350 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:53.329220+0000", 2024-09-16T09:31:53.350 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:31:53.350 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:53.350 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:53.351 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:31:53.351 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:53.329220+0000", 2024-09-16T09:31:53.351 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.6199999999999997e-07 2024-09-16T09:31:53.351 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:53.351 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:53.351 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:31:53.351 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:53.329226+0000", 2024-09-16T09:31:53.351 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.4e-06 2024-09-16T09:31:53.351 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:53.351 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:53.351 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:31:53.351 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:53.329237+0000", 2024-09-16T09:31:53.351 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1080999999999999e-05 2024-09-16T09:31:53.351 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:31:53.351 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:31:53.351 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:31:53.352 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:53.329243+0000", 2024-09-16T09:31:53.352 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.8749999999999997e-06 2024-09-16T09:31:53.352 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:31:53.352 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:31:53.352 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:31:53.352 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:31:53.352 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-09-16T09:31:53.352 INFO:teuthology.orchestra.run.smithi084.stdout: "num_ops": 5 2024-09-16T09:31:53.352 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-09-16T09:31:53.352 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:31:53.368 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:31:53.371 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2024-09-16T09:31:53.381 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:53.382 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:53.383 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:31:53.483 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:31:53.506 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:53.507 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:53.507 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:31:53.608 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:31:53.648 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:53.648 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:53.648 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:31:53.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:53 smithi084 ceph-mon[32629]: pgmap v319: 168 pgs: 1 active+recovery_wait+degraded, 3 active+recovering+undersized+remapped, 164 active+clean; 6.2 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 689 op/s; 1/302955 objects degraded (0.000%); 6599/302955 objects misplaced (2.178%); 4.3 MiB/s, 68 objects/s recovering 2024-09-16T09:31:53.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:53 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1379259081' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:31:53.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:53 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:31:53.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:53 smithi070 ceph-mon[30388]: pgmap v319: 168 pgs: 1 active+recovery_wait+degraded, 3 active+recovering+undersized+remapped, 164 active+clean; 6.2 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 689 op/s; 1/302955 objects degraded (0.000%); 6599/302955 objects misplaced (2.178%); 4.3 MiB/s, 68 objects/s recovering 2024-09-16T09:31:53.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:53 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1379259081' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:31:53.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:53 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:31:53.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:53 smithi070 ceph-mon[40691]: pgmap v319: 168 pgs: 1 active+recovery_wait+degraded, 3 active+recovering+undersized+remapped, 164 active+clean; 6.2 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 689 op/s; 1/302955 objects degraded (0.000%); 6599/302955 objects misplaced (2.178%); 4.3 MiB/s, 68 objects/s recovering 2024-09-16T09:31:53.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:53 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1379259081' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:31:53.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:53 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:31:53.749 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:31:53.783 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:53.784 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:53.784 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:31:53.885 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:31:53.923 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:53.924 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:53.924 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:31:54.025 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:31:54.061 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:54.062 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:54.062 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:31:54.156 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.6 dump_blocked_ops 2024-09-16T09:31:54.163 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:31:54.202 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:54.202 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:54.203 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:31:54.303 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:31:54.341 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:54.341 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:54.342 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:31:54.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:54 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:31:54.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:54 smithi084 ceph-mon[32629]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:31:54.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:54 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-16T09:31:54.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:54 smithi084 ceph-mon[32629]: osdmap e200: 8 total, 8 up, 8 in 2024-09-16T09:31:54.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:54 smithi084 ceph-mon[32629]: from='client.25093 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:31:54.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:54 smithi084 ceph-mon[32629]: Health check update: Degraded data redundancy: 1/302955 objects degraded (0.000%), 1 pg degraded (PG_DEGRADED) 2024-09-16T09:31:54.442 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:31:54.481 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:54.482 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:54.482 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:31:54.550 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:31:54.583 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:31:54.615 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:54.616 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:54.616 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:31:54.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:54 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:31:54.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:54 smithi070 ceph-mon[30388]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:31:54.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:54 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-16T09:31:54.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:54 smithi070 ceph-mon[30388]: osdmap e200: 8 total, 8 up, 8 in 2024-09-16T09:31:54.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:54 smithi070 ceph-mon[30388]: from='client.25093 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:31:54.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:54 smithi070 ceph-mon[30388]: Health check update: Degraded data redundancy: 1/302955 objects degraded (0.000%), 1 pg degraded (PG_DEGRADED) 2024-09-16T09:31:54.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:54 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:31:54.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:54 smithi070 ceph-mon[40691]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:31:54.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:54 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-16T09:31:54.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:54 smithi070 ceph-mon[40691]: osdmap e200: 8 total, 8 up, 8 in 2024-09-16T09:31:54.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:54 smithi070 ceph-mon[40691]: from='client.25093 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:31:54.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:54 smithi070 ceph-mon[40691]: Health check update: Degraded data redundancy: 1/302955 objects degraded (0.000%), 1 pg degraded (PG_DEGRADED) 2024-09-16T09:31:54.717 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:31:54.749 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:54.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:54.750 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:31:54.850 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:31:54.887 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:54.887 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:54.888 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:31:54.989 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:31:55.020 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:55.020 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:55.021 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:31:55.122 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:31:55.153 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:55.154 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:55.154 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:31:55.199 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-09-16T09:31:55.199 INFO:teuthology.orchestra.run.smithi084.stdout: "ops": [], 2024-09-16T09:31:55.199 INFO:teuthology.orchestra.run.smithi084.stdout: "complaint_time": 30, 2024-09-16T09:31:55.200 INFO:teuthology.orchestra.run.smithi084.stdout: "num_blocked_ops": 0 2024-09-16T09:31:55.200 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-09-16T09:31:55.231 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":320,"stamp":"2024-09-16T09:31:53.306560+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":6237874218,"num_objects":95371,"num_object_clones":0,"num_object_copies":286113,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":6003,"num_objects_unfound":0,"num_objects_dirty":95371,"num_whiteouts":0,"num_read":1142,"num_read_kb":913,"num_write":154472,"num_write_kb":7974230,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8843,"num_bytes_recovered":571259136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79342,"ondisk_log_size":79342,"up":504,"acting":501,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":508,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":20895688,"kb_used_data":18930056,"kb_used_omap":12,"kb_used_meta":1965235,"kb_avail":729065528,"statfs":{"total":767960285184,"available":746563100672,"internally_reserved":0,"allocated":19384377344,"data_stored":19381824945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":2012401202},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,2,4,8,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-729153536,"num_objects":-11126,"num_object_clones":0,"num_object_copies":-33378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-1142,"num_objects_unfound":0,"num_objects_dirty":-11126,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":11126,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1142,"num_bytes_recovered":74842112,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.684242"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":319,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.289210+0000","last_change":"2024-09-16T09:30:03.848439+0000","last_active":"2024-09-16T09:31:48.289210+0000","last_peered":"2024-09-16T09:31:48.289210+0000","last_clean":"2024-09-16T09:31:48.289210+0000","last_became_active":"2024-09-16T09:25:01.709108+0000","last_became_peered":"2024-09-16T09:25:01.709108+0000","last_unstale":"2024-09-16T09:31:48.289210+0000","last_undegraded":"2024-09-16T09:31:48.289210+0000","last_fullsized":"2024-09-16T09:31:48.289210+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:30:03.848354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:30:03.848354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"66'9","reported_seq":322,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.266598+0000","last_change":"2024-09-16T09:28:52.689373+0000","last_active":"2024-09-16T09:31:48.266598+0000","last_peered":"2024-09-16T09:31:48.266598+0000","last_clean":"2024-09-16T09:31:48.266598+0000","last_became_active":"2024-09-16T09:25:03.714635+0000","last_became_peered":"2024-09-16T09:25:03.714635+0000","last_unstale":"2024-09-16T09:31:48.266598+0000","last_undegraded":"2024-09-16T09:31:48.266598+0000","last_fullsized":"2024-09-16T09:31:48.266598+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:28:52.689304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:52.689304+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":310,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.267593+0000","last_change":"2024-09-16T09:29:17.910367+0000","last_active":"2024-09-16T09:31:48.267593+0000","last_peered":"2024-09-16T09:31:48.267593+0000","last_clean":"2024-09-16T09:31:48.267593+0000","last_became_active":"2024-09-16T09:25:05.715645+0000","last_became_peered":"2024-09-16T09:25:05.715645+0000","last_unstale":"2024-09-16T09:31:48.267593+0000","last_undegraded":"2024-09-16T09:31:48.267593+0000","last_fullsized":"2024-09-16T09:31:48.267593+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:17.910305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:17.910305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":290,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.268478+0000","last_change":"2024-09-16T09:27:11.565306+0000","last_active":"2024-09-16T09:31:48.268478+0000","last_peered":"2024-09-16T09:31:48.268478+0000","last_clean":"2024-09-16T09:31:48.268478+0000","last_became_active":"2024-09-16T09:25:07.715442+0000","last_became_peered":"2024-09-16T09:25:07.715442+0000","last_unstale":"2024-09-16T09:31:48.268478+0000","last_undegraded":"2024-09-16T09:31:48.268478+0000","last_fullsized":"2024-09-16T09:31:48.268478+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:11.565220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:11.565220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":36,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.283939+0000","last_change":"2024-09-16T09:31:09.269700+0000","last_active":"2024-09-16T09:31:48.283939+0000","last_peered":"2024-09-16T09:31:48.283939+0000","last_clean":"2024-09-16T09:31:48.283939+0000","last_became_active":"2024-09-16T09:31:09.269342+0000","last_became_peered":"2024-09-16T09:31:09.269342+0000","last_unstale":"2024-09-16T09:31:48.283939+0000","last_undegraded":"2024-09-16T09:31:48.283939+0000","last_fullsized":"2024-09-16T09:31:48.283939+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:00.068570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:28:00.068570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":168,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.273302+0000","last_change":"2024-09-16T09:31:09.276349+0000","last_active":"2024-09-16T09:31:48.273302+0000","last_peered":"2024-09-16T09:31:48.273302+0000","last_clean":"2024-09-16T09:31:48.273302+0000","last_became_active":"2024-09-16T09:31:09.275136+0000","last_became_peered":"2024-09-16T09:31:09.275136+0000","last_unstale":"2024-09-16T09:31:48.273302+0000","last_undegraded":"2024-09-16T09:31:48.273302+0000","last_fullsized":"2024-09-16T09:31:48.273302+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:30:32.389008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:30:32.389008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:31:56.454141+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.19","version":"66'15","reported_seq":237,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.283152+0000","last_change":"2024-09-16T09:31:09.302903+0000","last_active":"2024-09-16T09:31:48.283152+0000","last_peered":"2024-09-16T09:31:48.283152+0000","last_clean":"2024-09-16T09:31:48.283152+0000","last_became_active":"2024-09-16T09:31:09.268750+0000","last_became_peered":"2024-09-16T09:31:09.268750+0000","last_unstale":"2024-09-16T09:31:48.283152+0000","last_undegraded":"2024-09-16T09:31:48.283152+0000","last_fullsized":"2024-09-16T09:31:48.283152+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-16T09:29:57.968162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:29:57.968162+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":310,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.266506+0000","last_change":"2024-09-16T09:29:15.628676+0000","last_active":"2024-09-16T09:31:48.266506+0000","last_peered":"2024-09-16T09:31:48.266506+0000","last_clean":"2024-09-16T09:31:48.266506+0000","last_became_active":"2024-09-16T09:25:05.717236+0000","last_became_peered":"2024-09-16T09:25:05.717236+0000","last_unstale":"2024-09-16T09:31:48.266506+0000","last_undegraded":"2024-09-16T09:31:48.266506+0000","last_fullsized":"2024-09-16T09:31:48.266506+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:15.628467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:15.628467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":149,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.270436+0000","last_change":"2024-09-16T09:28:34.839909+0000","last_active":"2024-09-16T09:31:48.270436+0000","last_peered":"2024-09-16T09:31:48.270436+0000","last_clean":"2024-09-16T09:31:48.270436+0000","last_became_active":"2024-09-16T09:28:34.839435+0000","last_became_peered":"2024-09-16T09:28:34.839435+0000","last_unstale":"2024-09-16T09:31:48.270436+0000","last_undegraded":"2024-09-16T09:31:48.270436+0000","last_fullsized":"2024-09-16T09:31:48.270436+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:01.038400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:28:01.038400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:09.066525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"66'9","reported_seq":322,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.268678+0000","last_change":"2024-09-16T09:29:28.622647+0000","last_active":"2024-09-16T09:31:48.268678+0000","last_peered":"2024-09-16T09:31:48.268678+0000","last_clean":"2024-09-16T09:31:48.268678+0000","last_became_active":"2024-09-16T09:25:03.709147+0000","last_became_peered":"2024-09-16T09:25:03.709147+0000","last_unstale":"2024-09-16T09:31:48.268678+0000","last_undegraded":"2024-09-16T09:31:48.268678+0000","last_fullsized":"2024-09-16T09:31:48.268678+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:29:28.622558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:29:28.622558+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":293,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.273125+0000","last_change":"2024-09-16T09:27:59.080369+0000","last_active":"2024-09-16T09:31:48.273125+0000","last_peered":"2024-09-16T09:31:48.273125+0000","last_clean":"2024-09-16T09:31:48.273125+0000","last_became_active":"2024-09-16T09:25:05.718134+0000","last_became_peered":"2024-09-16T09:25:05.718134+0000","last_unstale":"2024-09-16T09:31:48.273125+0000","last_undegraded":"2024-09-16T09:31:48.273125+0000","last_fullsized":"2024-09-16T09:31:48.273125+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:59.080307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:59.080307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"66'1","reported_seq":299,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.283957+0000","last_change":"2024-09-16T09:29:15.787990+0000","last_active":"2024-09-16T09:31:48.283957+0000","last_peered":"2024-09-16T09:31:48.283957+0000","last_clean":"2024-09-16T09:31:48.283957+0000","last_became_active":"2024-09-16T09:25:07.716558+0000","last_became_peered":"2024-09-16T09:25:07.716558+0000","last_unstale":"2024-09-16T09:31:48.283957+0000","last_undegraded":"2024-09-16T09:31:48.283957+0000","last_fullsized":"2024-09-16T09:31:48.283957+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-16T09:29:15.787922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:15.787922+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":301,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.278039+0000","last_change":"2024-09-16T09:27:56.191264+0000","last_active":"2024-09-16T09:31:48.278039+0000","last_peered":"2024-09-16T09:31:48.278039+0000","last_clean":"2024-09-16T09:31:48.278039+0000","last_became_active":"2024-09-16T09:25:01.696939+0000","last_became_peered":"2024-09-16T09:25:01.696939+0000","last_unstale":"2024-09-16T09:31:48.278039+0000","last_undegraded":"2024-09-16T09:31:48.278039+0000","last_fullsized":"2024-09-16T09:31:48.278039+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:56.191220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:27:56.191220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"66'5","reported_seq":308,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.267293+0000","last_change":"2024-09-16T09:27:05.694733+0000","last_active":"2024-09-16T09:31:48.267293+0000","last_peered":"2024-09-16T09:31:48.267293+0000","last_clean":"2024-09-16T09:31:48.267293+0000","last_became_active":"2024-09-16T09:25:03.716136+0000","last_became_peered":"2024-09-16T09:25:03.716136+0000","last_unstale":"2024-09-16T09:31:48.267293+0000","last_undegraded":"2024-09-16T09:31:48.267293+0000","last_fullsized":"2024-09-16T09:31:48.267293+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-16T09:27:05.694650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:05.694650+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":301,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.276211+0000","last_change":"2024-09-16T09:28:52.746046+0000","last_active":"2024-09-16T09:31:48.276211+0000","last_peered":"2024-09-16T09:31:48.276211+0000","last_clean":"2024-09-16T09:31:48.276211+0000","last_became_active":"2024-09-16T09:25:05.718281+0000","last_became_peered":"2024-09-16T09:25:05.718281+0000","last_unstale":"2024-09-16T09:31:48.276211+0000","last_undegraded":"2024-09-16T09:31:48.276211+0000","last_fullsized":"2024-09-16T09:31:48.276211+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:52.745999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:28:52.745999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":319,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.273226+0000","last_change":"2024-09-16T09:31:09.276016+0000","last_active":"2024-09-16T09:31:48.273226+0000","last_peered":"2024-09-16T09:31:48.273226+0000","last_clean":"2024-09-16T09:31:48.273226+0000","last_became_active":"2024-09-16T09:31:09.275656+0000","last_became_peered":"2024-09-16T09:31:09.275656+0000","last_unstale":"2024-09-16T09:31:48.273226+0000","last_undegraded":"2024-09-16T09:31:48.273226+0000","last_fullsized":"2024-09-16T09:31:48.273226+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:50.468134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:50.468134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":307,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.268799+0000","last_change":"2024-09-16T09:29:56.605555+0000","last_active":"2024-09-16T09:31:48.268799+0000","last_peered":"2024-09-16T09:31:48.268799+0000","last_clean":"2024-09-16T09:31:48.268799+0000","last_became_active":"2024-09-16T09:25:07.720793+0000","last_became_peered":"2024-09-16T09:25:07.720793+0000","last_unstale":"2024-09-16T09:31:48.268799+0000","last_undegraded":"2024-09-16T09:31:48.268799+0000","last_fullsized":"2024-09-16T09:31:48.268799+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:56.605488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:56.605488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":58,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.272173+0000","last_change":"2024-09-16T09:30:39.202178+0000","last_active":"2024-09-16T09:31:48.272173+0000","last_peered":"2024-09-16T09:31:48.272173+0000","last_clean":"2024-09-16T09:31:48.272173+0000","last_became_active":"2024-09-16T09:30:39.201824+0000","last_became_peered":"2024-09-16T09:30:39.201824+0000","last_unstale":"2024-09-16T09:31:48.272173+0000","last_undegraded":"2024-09-16T09:31:48.272173+0000","last_fullsized":"2024-09-16T09:31:48.272173+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:50.912979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:50.912979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:20.431823+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"66'15","reported_seq":385,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.275581+0000","last_change":"2024-09-16T09:31:09.302018+0000","last_active":"2024-09-16T09:31:48.275581+0000","last_peered":"2024-09-16T09:31:48.275581+0000","last_clean":"2024-09-16T09:31:48.275581+0000","last_became_active":"2024-09-16T09:31:09.263534+0000","last_became_peered":"2024-09-16T09:31:09.263534+0000","last_unstale":"2024-09-16T09:31:48.275581+0000","last_undegraded":"2024-09-16T09:31:48.275581+0000","last_fullsized":"2024-09-16T09:31:48.275581+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-16T09:30:21.932604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:30:21.932604+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":302,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.271487+0000","last_change":"2024-09-16T09:29:55.875536+0000","last_active":"2024-09-16T09:31:48.271487+0000","last_peered":"2024-09-16T09:31:48.271487+0000","last_clean":"2024-09-16T09:31:48.271487+0000","last_became_active":"2024-09-16T09:25:05.713929+0000","last_became_peered":"2024-09-16T09:25:05.713929+0000","last_unstale":"2024-09-16T09:31:48.271487+0000","last_undegraded":"2024-09-16T09:31:48.271487+0000","last_fullsized":"2024-09-16T09:31:48.271487+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:55.875480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:55.875480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":155,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.273232+0000","last_change":"2024-09-16T09:30:24.412733+0000","last_active":"2024-09-16T09:31:48.273232+0000","last_peered":"2024-09-16T09:31:48.273232+0000","last_clean":"2024-09-16T09:31:48.273232+0000","last_became_active":"2024-09-16T09:28:52.974255+0000","last_became_peered":"2024-09-16T09:28:52.974255+0000","last_unstale":"2024-09-16T09:31:48.273232+0000","last_undegraded":"2024-09-16T09:31:48.273232+0000","last_fullsized":"2024-09-16T09:31:48.273232+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:30:24.412571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:30:24.412571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":318,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.268798+0000","last_change":"2024-09-16T09:30:03.628882+0000","last_active":"2024-09-16T09:31:48.268798+0000","last_peered":"2024-09-16T09:31:48.268798+0000","last_clean":"2024-09-16T09:31:48.268798+0000","last_became_active":"2024-09-16T09:25:01.706491+0000","last_became_peered":"2024-09-16T09:25:01.706491+0000","last_unstale":"2024-09-16T09:31:48.268798+0000","last_undegraded":"2024-09-16T09:31:48.268798+0000","last_fullsized":"2024-09-16T09:31:48.268798+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:30:03.628812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:30:03.628812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"66'12","reported_seq":219,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.271717+0000","last_change":"2024-09-16T09:31:09.286673+0000","last_active":"2024-09-16T09:31:48.271717+0000","last_peered":"2024-09-16T09:31:48.271717+0000","last_clean":"2024-09-16T09:31:48.271717+0000","last_became_active":"2024-09-16T09:31:09.268588+0000","last_became_peered":"2024-09-16T09:31:09.268588+0000","last_unstale":"2024-09-16T09:31:48.271717+0000","last_undegraded":"2024-09-16T09:31:48.271717+0000","last_fullsized":"2024-09-16T09:31:48.271717+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-16T09:27:01.700139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:01.700139+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":24,"num_read_kb":15,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":294,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.267455+0000","last_change":"2024-09-16T09:27:00.816500+0000","last_active":"2024-09-16T09:31:48.267455+0000","last_peered":"2024-09-16T09:31:48.267455+0000","last_clean":"2024-09-16T09:31:48.267455+0000","last_became_active":"2024-09-16T09:25:05.710093+0000","last_became_peered":"2024-09-16T09:25:05.710093+0000","last_unstale":"2024-09-16T09:31:48.267455+0000","last_undegraded":"2024-09-16T09:31:48.267455+0000","last_fullsized":"2024-09-16T09:31:48.267455+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:00.816424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:00.816424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":298,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.276696+0000","last_change":"2024-09-16T09:29:29.918433+0000","last_active":"2024-09-16T09:31:48.276696+0000","last_peered":"2024-09-16T09:31:48.276696+0000","last_clean":"2024-09-16T09:31:48.276696+0000","last_became_active":"2024-09-16T09:25:07.719380+0000","last_became_peered":"2024-09-16T09:25:07.719380+0000","last_unstale":"2024-09-16T09:31:48.276696+0000","last_undegraded":"2024-09-16T09:31:48.276696+0000","last_fullsized":"2024-09-16T09:31:48.276696+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-16T09:29:29.918368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:29.918368+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":319,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.267735+0000","last_change":"2024-09-16T09:30:02.843932+0000","last_active":"2024-09-16T09:31:48.267735+0000","last_peered":"2024-09-16T09:31:48.267735+0000","last_clean":"2024-09-16T09:31:48.267735+0000","last_became_active":"2024-09-16T09:25:01.699545+0000","last_became_peered":"2024-09-16T09:25:01.699545+0000","last_unstale":"2024-09-16T09:31:48.267735+0000","last_undegraded":"2024-09-16T09:31:48.267735+0000","last_fullsized":"2024-09-16T09:31:48.267735+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:30:02.843856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:30:02.843856+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"66'10","reported_seq":360,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.290194+0000","last_change":"2024-09-16T09:31:09.328513+0000","last_active":"2024-09-16T09:31:48.290194+0000","last_peered":"2024-09-16T09:31:48.290194+0000","last_clean":"2024-09-16T09:31:48.290194+0000","last_became_active":"2024-09-16T09:31:09.275050+0000","last_became_peered":"2024-09-16T09:31:09.275050+0000","last_unstale":"2024-09-16T09:31:48.290194+0000","last_undegraded":"2024-09-16T09:31:48.290194+0000","last_fullsized":"2024-09-16T09:31:48.290194+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-16T09:27:43.556357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:43.556357+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:15.194576+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"66'8","reported_seq":413,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.273385+0000","last_change":"2024-09-16T09:30:31.392771+0000","last_active":"2024-09-16T09:31:48.273385+0000","last_peered":"2024-09-16T09:31:48.273385+0000","last_clean":"2024-09-16T09:31:48.273385+0000","last_became_active":"2024-09-16T09:25:05.712516+0000","last_became_peered":"2024-09-16T09:25:05.712516+0000","last_unstale":"2024-09-16T09:31:48.273385+0000","last_undegraded":"2024-09-16T09:31:48.273385+0000","last_fullsized":"2024-09-16T09:31:48.273385+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:30:31.392399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:30:31.392399+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:44.469073+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"199'9686","reported_seq":9791,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:52.816844+0000","last_change":"2024-09-16T09:29:49.449011+0000","last_active":"2024-09-16T09:31:52.816844+0000","last_peered":"2024-09-16T09:31:52.816844+0000","last_clean":"2024-09-16T09:31:52.816844+0000","last_became_active":"2024-09-16T09:29:49.448473+0000","last_became_peered":"2024-09-16T09:29:49.448473+0000","last_unstale":"2024-09-16T09:31:52.816844+0000","last_undegraded":"2024-09-16T09:31:52.816844+0000","last_fullsized":"2024-09-16T09:31:52.816844+0000","mapping_epoch":152,"log_start":"178'5200","ondisk_log_start":"178'5200","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":4486,"log_dups_size":0,"ondisk_log_size":4486,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":388759552,"num_objects":5932,"num_object_clones":0,"num_object_copies":17796,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5932,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9686,"num_write_kb":499776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.a","version":"73'79","reported_seq":241,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.273884+0000","last_change":"2024-09-16T09:29:18.498887+0000","last_active":"2024-09-16T09:31:48.273884+0000","last_peered":"2024-09-16T09:31:48.273884+0000","last_clean":"2024-09-16T09:31:48.273884+0000","last_became_active":"2024-09-16T09:26:35.127352+0000","last_became_peered":"2024-09-16T09:26:35.127352+0000","last_unstale":"2024-09-16T09:31:48.273884+0000","last_undegraded":"2024-09-16T09:31:48.273884+0000","last_fullsized":"2024-09-16T09:31:48.273884+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:18.498766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:18.498766+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"66'15","reported_seq":250,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.276002+0000","last_change":"2024-09-16T09:28:28.065781+0000","last_active":"2024-09-16T09:31:48.276002+0000","last_peered":"2024-09-16T09:31:48.276002+0000","last_clean":"2024-09-16T09:31:48.276002+0000","last_became_active":"2024-09-16T09:26:33.295672+0000","last_became_peered":"2024-09-16T09:26:33.295672+0000","last_unstale":"2024-09-16T09:31:48.276002+0000","last_undegraded":"2024-09-16T09:31:48.276002+0000","last_fullsized":"2024-09-16T09:31:48.276002+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-16T09:28:28.065699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:28.065699+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":36,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.278422+0000","last_change":"2024-09-16T09:31:09.276858+0000","last_active":"2024-09-16T09:31:48.278422+0000","last_peered":"2024-09-16T09:31:48.278422+0000","last_clean":"2024-09-16T09:31:48.278422+0000","last_became_active":"2024-09-16T09:31:09.274534+0000","last_became_peered":"2024-09-16T09:31:09.274534+0000","last_unstale":"2024-09-16T09:31:48.278422+0000","last_undegraded":"2024-09-16T09:31:48.278422+0000","last_fullsized":"2024-09-16T09:31:48.278422+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:22.091537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:28:22.091537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"66'8","reported_seq":378,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:51.841850+0000","last_change":"2024-09-16T09:25:05.714128+0000","last_active":"2024-09-16T09:31:51.841850+0000","last_peered":"2024-09-16T09:31:51.841850+0000","last_clean":"2024-09-16T09:31:51.841850+0000","last_became_active":"2024-09-16T09:25:05.713813+0000","last_became_peered":"2024-09-16T09:25:05.713813+0000","last_unstale":"2024-09-16T09:31:51.841850+0000","last_undegraded":"2024-09-16T09:31:51.841850+0000","last_fullsized":"2024-09-16T09:31:51.841850+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":298,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.273480+0000","last_change":"2024-09-16T09:29:24.537802+0000","last_active":"2024-09-16T09:31:48.273480+0000","last_peered":"2024-09-16T09:31:48.273480+0000","last_clean":"2024-09-16T09:31:48.273480+0000","last_became_active":"2024-09-16T09:25:07.717344+0000","last_became_peered":"2024-09-16T09:25:07.717344+0000","last_unstale":"2024-09-16T09:31:48.273480+0000","last_undegraded":"2024-09-16T09:31:48.273480+0000","last_fullsized":"2024-09-16T09:31:48.273480+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:24.537731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:24.537731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.c","version":"199'9651","reported_seq":9756,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.150588+0000","last_change":"2024-09-16T09:29:50.008199+0000","last_active":"2024-09-16T09:31:53.150588+0000","last_peered":"2024-09-16T09:31:53.150588+0000","last_clean":"2024-09-16T09:31:53.150588+0000","last_became_active":"2024-09-16T09:29:50.007935+0000","last_became_peered":"2024-09-16T09:29:50.007935+0000","last_unstale":"2024-09-16T09:31:53.150588+0000","last_undegraded":"2024-09-16T09:31:53.150588+0000","last_fullsized":"2024-09-16T09:31:53.150588+0000","mapping_epoch":152,"log_start":"176'4500","ondisk_log_start":"176'4500","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":5151,"log_dups_size":0,"ondisk_log_size":5151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":386465792,"num_objects":5897,"num_object_clones":0,"num_object_copies":17691,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5897,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9651,"num_write_kb":497536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"73'79","reported_seq":270,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.268362+0000","last_change":"2024-09-16T09:31:47.249715+0000","last_active":"2024-09-16T09:31:48.268362+0000","last_peered":"2024-09-16T09:31:48.268362+0000","last_clean":"2024-09-16T09:31:48.268362+0000","last_became_active":"2024-09-16T09:30:08.763683+0000","last_became_peered":"2024-09-16T09:30:08.763683+0000","last_unstale":"2024-09-16T09:31:48.268362+0000","last_undegraded":"2024-09-16T09:31:48.268362+0000","last_fullsized":"2024-09-16T09:31:48.268362+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:58.568841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:58.568841+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:32:11.825261+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"66'18","reported_seq":248,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.276186+0000","last_change":"2024-09-16T09:31:09.407700+0000","last_active":"2024-09-16T09:31:48.276186+0000","last_peered":"2024-09-16T09:31:48.276186+0000","last_clean":"2024-09-16T09:31:48.276186+0000","last_became_active":"2024-09-16T09:31:09.270434+0000","last_became_peered":"2024-09-16T09:31:09.270434+0000","last_unstale":"2024-09-16T09:31:48.276186+0000","last_undegraded":"2024-09-16T09:31:48.276186+0000","last_fullsized":"2024-09-16T09:31:48.276186+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'18","last_scrub_stamp":"2024-09-16T09:28:54.699778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:54.699778+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":33,"num_read_kb":21,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":146,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.278036+0000","last_change":"2024-09-16T09:28:58.705169+0000","last_active":"2024-09-16T09:31:48.278036+0000","last_peered":"2024-09-16T09:31:48.278036+0000","last_clean":"2024-09-16T09:31:48.278036+0000","last_became_active":"2024-09-16T09:28:52.983543+0000","last_became_peered":"2024-09-16T09:28:52.983543+0000","last_unstale":"2024-09-16T09:31:48.278036+0000","last_undegraded":"2024-09-16T09:31:48.278036+0000","last_fullsized":"2024-09-16T09:31:48.278036+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:58.705097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:28:58.705097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":325,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.266531+0000","last_change":"2024-09-16T09:31:09.269234+0000","last_active":"2024-09-16T09:31:48.266531+0000","last_peered":"2024-09-16T09:31:48.266531+0000","last_clean":"2024-09-16T09:31:48.266531+0000","last_became_active":"2024-09-16T09:31:09.268864+0000","last_became_peered":"2024-09-16T09:31:09.268864+0000","last_unstale":"2024-09-16T09:31:48.266531+0000","last_undegraded":"2024-09-16T09:31:48.266531+0000","last_fullsized":"2024-09-16T09:31:48.266531+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:50.482240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:50.482240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":305,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.268629+0000","last_change":"2024-09-16T09:31:09.269842+0000","last_active":"2024-09-16T09:31:48.268629+0000","last_peered":"2024-09-16T09:31:48.268629+0000","last_clean":"2024-09-16T09:31:48.268629+0000","last_became_active":"2024-09-16T09:31:09.269433+0000","last_became_peered":"2024-09-16T09:31:09.269433+0000","last_unstale":"2024-09-16T09:31:48.268629+0000","last_undegraded":"2024-09-16T09:31:48.268629+0000","last_fullsized":"2024-09-16T09:31:48.268629+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:13.548608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:13.548608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.d","version":"199'9597","reported_seq":9703,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:52.883195+0000","last_change":"2024-09-16T09:29:50.010860+0000","last_active":"2024-09-16T09:31:52.883195+0000","last_peered":"2024-09-16T09:31:52.883195+0000","last_clean":"2024-09-16T09:31:52.883195+0000","last_became_active":"2024-09-16T09:29:50.010421+0000","last_became_peered":"2024-09-16T09:29:50.010421+0000","last_unstale":"2024-09-16T09:31:52.883195+0000","last_undegraded":"2024-09-16T09:31:52.883195+0000","last_fullsized":"2024-09-16T09:31:52.883195+0000","mapping_epoch":152,"log_start":"177'4900","ondisk_log_start":"177'4900","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":4697,"log_dups_size":0,"ondisk_log_size":4697,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":391118872,"num_objects":5969,"num_object_clones":0,"num_object_copies":17907,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5969,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9597,"num_write_kb":498049,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,2],"acting":[5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.4","version":"73'79","reported_seq":36,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.278558+0000","last_change":"2024-09-16T09:31:09.275499+0000","last_active":"2024-09-16T09:31:48.278558+0000","last_peered":"2024-09-16T09:31:48.278558+0000","last_clean":"2024-09-16T09:31:48.278558+0000","last_became_active":"2024-09-16T09:31:09.275139+0000","last_became_peered":"2024-09-16T09:31:09.275139+0000","last_unstale":"2024-09-16T09:31:48.278558+0000","last_undegraded":"2024-09-16T09:31:48.278558+0000","last_fullsized":"2024-09-16T09:31:48.278558+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:30:22.438378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:30:22.438378+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"66'14","reported_seq":328,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.266610+0000","last_change":"2024-09-16T09:28:32.595064+0000","last_active":"2024-09-16T09:31:48.266610+0000","last_peered":"2024-09-16T09:31:48.266610+0000","last_clean":"2024-09-16T09:31:48.266610+0000","last_became_active":"2024-09-16T09:25:03.708830+0000","last_became_peered":"2024-09-16T09:25:03.708830+0000","last_unstale":"2024-09-16T09:31:48.266610+0000","last_undegraded":"2024-09-16T09:31:48.266610+0000","last_fullsized":"2024-09-16T09:31:48.266610+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'14","last_scrub_stamp":"2024-09-16T09:28:32.594988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:32.594988+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":320,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.291991+0000","last_change":"2024-09-16T09:28:58.602786+0000","last_active":"2024-09-16T09:31:48.291991+0000","last_peered":"2024-09-16T09:31:48.291991+0000","last_clean":"2024-09-16T09:31:48.291991+0000","last_became_active":"2024-09-16T09:25:01.702992+0000","last_became_peered":"2024-09-16T09:25:01.702992+0000","last_unstale":"2024-09-16T09:31:48.291991+0000","last_undegraded":"2024-09-16T09:31:48.291991+0000","last_fullsized":"2024-09-16T09:31:48.291991+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:28:58.602717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:28:58.602717+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"187'11","reported_seq":230,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.268663+0000","last_change":"2024-09-16T09:31:09.397758+0000","last_active":"2024-09-16T09:31:48.268663+0000","last_peered":"2024-09-16T09:31:48.268663+0000","last_clean":"2024-09-16T09:31:48.268663+0000","last_became_active":"2024-09-16T09:31:09.269173+0000","last_became_peered":"2024-09-16T09:31:09.269173+0000","last_unstale":"2024-09-16T09:31:48.268663+0000","last_undegraded":"2024-09-16T09:31:48.268663+0000","last_fullsized":"2024-09-16T09:31:48.268663+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'9","last_scrub_stamp":"2024-09-16T09:29:21.898533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:21.898533+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":290,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.277411+0000","last_change":"2024-09-16T09:26:40.699888+0000","last_active":"2024-09-16T09:31:48.277411+0000","last_peered":"2024-09-16T09:31:48.277411+0000","last_clean":"2024-09-16T09:31:48.277411+0000","last_became_active":"2024-09-16T09:25:07.722017+0000","last_became_peered":"2024-09-16T09:25:07.722017+0000","last_unstale":"2024-09-16T09:31:48.277411+0000","last_undegraded":"2024-09-16T09:31:48.277411+0000","last_fullsized":"2024-09-16T09:31:48.277411+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:40.699813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:40.699813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.e","version":"199'9704","reported_seq":9808,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.132565+0000","last_change":"2024-09-16T09:29:49.448081+0000","last_active":"2024-09-16T09:31:53.132565+0000","last_peered":"2024-09-16T09:31:53.132565+0000","last_clean":"2024-09-16T09:31:53.132565+0000","last_became_active":"2024-09-16T09:29:49.447767+0000","last_became_peered":"2024-09-16T09:29:49.447767+0000","last_unstale":"2024-09-16T09:31:53.132565+0000","last_undegraded":"2024-09-16T09:31:53.132565+0000","last_fullsized":"2024-09-16T09:31:53.132565+0000","mapping_epoch":152,"log_start":"176'4600","ondisk_log_start":"176'4600","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":5104,"log_dups_size":0,"ondisk_log_size":5104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":386793472,"num_objects":5902,"num_object_clones":0,"num_object_copies":17706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5902,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9704,"num_write_kb":499392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"73'79","reported_seq":359,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.272512+0000","last_change":"2024-09-16T09:31:09.275413+0000","last_active":"2024-09-16T09:31:48.272512+0000","last_peered":"2024-09-16T09:31:48.272512+0000","last_clean":"2024-09-16T09:31:48.272512+0000","last_became_active":"2024-09-16T09:31:09.274707+0000","last_became_peered":"2024-09-16T09:31:09.274707+0000","last_unstale":"2024-09-16T09:31:48.272512+0000","last_undegraded":"2024-09-16T09:31:48.272512+0000","last_fullsized":"2024-09-16T09:31:48.272512+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:30:23.455388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:30:23.455388+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:41.421930+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"66'10","reported_seq":330,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.267757+0000","last_change":"2024-09-16T09:30:04.848306+0000","last_active":"2024-09-16T09:31:48.267757+0000","last_peered":"2024-09-16T09:31:48.267757+0000","last_clean":"2024-09-16T09:31:48.267757+0000","last_became_active":"2024-09-16T09:25:03.712985+0000","last_became_peered":"2024-09-16T09:25:03.712985+0000","last_unstale":"2024-09-16T09:31:48.267757+0000","last_undegraded":"2024-09-16T09:31:48.267757+0000","last_fullsized":"2024-09-16T09:31:48.267757+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-16T09:30:04.848234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:30:04.848234+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":316,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.273808+0000","last_change":"2024-09-16T09:31:09.272049+0000","last_active":"2024-09-16T09:31:48.273808+0000","last_peered":"2024-09-16T09:31:48.273808+0000","last_clean":"2024-09-16T09:31:48.273808+0000","last_became_active":"2024-09-16T09:31:09.270619+0000","last_became_peered":"2024-09-16T09:31:09.270619+0000","last_unstale":"2024-09-16T09:31:48.273808+0000","last_undegraded":"2024-09-16T09:31:48.273808+0000","last_fullsized":"2024-09-16T09:31:48.273808+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:25.086322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:25.086322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"66'8","reported_seq":394,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.287913+0000","last_change":"2024-09-16T09:28:57.628615+0000","last_active":"2024-09-16T09:31:48.287913+0000","last_peered":"2024-09-16T09:31:48.287913+0000","last_clean":"2024-09-16T09:31:48.287913+0000","last_became_active":"2024-09-16T09:25:05.714507+0000","last_became_peered":"2024-09-16T09:25:05.714507+0000","last_unstale":"2024-09-16T09:31:48.287913+0000","last_undegraded":"2024-09-16T09:31:48.287913+0000","last_fullsized":"2024-09-16T09:31:48.287913+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:28:57.628543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:28:57.628543+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:51.253847+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":313,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.284087+0000","last_change":"2024-09-16T09:31:09.283816+0000","last_active":"2024-09-16T09:31:48.284087+0000","last_peered":"2024-09-16T09:31:48.284087+0000","last_clean":"2024-09-16T09:31:48.284087+0000","last_became_active":"2024-09-16T09:31:09.283192+0000","last_became_peered":"2024-09-16T09:31:09.283192+0000","last_unstale":"2024-09-16T09:31:48.284087+0000","last_undegraded":"2024-09-16T09:31:48.284087+0000","last_fullsized":"2024-09-16T09:31:48.284087+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:22.563395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:28:22.563395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:22.152175+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.f","version":"199'9696","reported_seq":9799,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:52.818305+0000","last_change":"2024-09-16T09:29:49.838498+0000","last_active":"2024-09-16T09:31:52.818305+0000","last_peered":"2024-09-16T09:31:52.818305+0000","last_clean":"2024-09-16T09:31:52.818305+0000","last_became_active":"2024-09-16T09:29:49.838146+0000","last_became_peered":"2024-09-16T09:29:49.838146+0000","last_unstale":"2024-09-16T09:31:52.818305+0000","last_undegraded":"2024-09-16T09:31:52.818305+0000","last_fullsized":"2024-09-16T09:31:52.818305+0000","mapping_epoch":152,"log_start":"172'3800","ondisk_log_start":"172'3800","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":5896,"log_dups_size":0,"ondisk_log_size":5896,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":396492800,"num_objects":6050,"num_object_clones":0,"num_object_copies":18150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6050,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9696,"num_write_kb":503872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"73'79","reported_seq":340,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.272457+0000","last_change":"2024-09-16T09:31:09.274975+0000","last_active":"2024-09-16T09:31:48.272457+0000","last_peered":"2024-09-16T09:31:48.272457+0000","last_clean":"2024-09-16T09:31:48.272457+0000","last_became_active":"2024-09-16T09:31:09.274565+0000","last_became_peered":"2024-09-16T09:31:09.274565+0000","last_unstale":"2024-09-16T09:31:48.272457+0000","last_undegraded":"2024-09-16T09:31:48.272457+0000","last_fullsized":"2024-09-16T09:31:48.272457+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:30:22.415051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:30:22.415051+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"66'19","reported_seq":343,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.288073+0000","last_change":"2024-09-16T09:31:35.249952+0000","last_active":"2024-09-16T09:31:48.288073+0000","last_peered":"2024-09-16T09:31:48.288073+0000","last_clean":"2024-09-16T09:31:48.288073+0000","last_became_active":"2024-09-16T09:25:03.714797+0000","last_became_peered":"2024-09-16T09:25:03.714797+0000","last_unstale":"2024-09-16T09:31:48.288073+0000","last_undegraded":"2024-09-16T09:31:48.288073+0000","last_fullsized":"2024-09-16T09:31:48.288073+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-16T09:31:35.249864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:31:35.249864+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:40.632025+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":319,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.270751+0000","last_change":"2024-09-16T09:29:16.888357+0000","last_active":"2024-09-16T09:31:48.270751+0000","last_peered":"2024-09-16T09:31:48.270751+0000","last_clean":"2024-09-16T09:31:48.270751+0000","last_became_active":"2024-09-16T09:25:01.706806+0000","last_became_peered":"2024-09-16T09:25:01.706806+0000","last_unstale":"2024-09-16T09:31:48.270751+0000","last_undegraded":"2024-09-16T09:31:48.270751+0000","last_fullsized":"2024-09-16T09:31:48.270751+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:16.888236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:16.888236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":311,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.272323+0000","last_change":"2024-09-16T09:30:51.409917+0000","last_active":"2024-09-16T09:31:48.272323+0000","last_peered":"2024-09-16T09:31:48.272323+0000","last_clean":"2024-09-16T09:31:48.272323+0000","last_became_active":"2024-09-16T09:25:05.708663+0000","last_became_peered":"2024-09-16T09:25:05.708663+0000","last_unstale":"2024-09-16T09:31:48.272323+0000","last_undegraded":"2024-09-16T09:31:48.272323+0000","last_fullsized":"2024-09-16T09:31:48.272323+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:30:51.409834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:30:51.409834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:31:55.112422+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":291,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.267503+0000","last_change":"2024-09-16T09:26:37.282798+0000","last_active":"2024-09-16T09:31:48.267503+0000","last_peered":"2024-09-16T09:31:48.267503+0000","last_clean":"2024-09-16T09:31:48.267503+0000","last_became_active":"2024-09-16T09:25:07.719267+0000","last_became_peered":"2024-09-16T09:25:07.719267+0000","last_unstale":"2024-09-16T09:31:48.267503+0000","last_undegraded":"2024-09-16T09:31:48.267503+0000","last_fullsized":"2024-09-16T09:31:48.267503+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:37.282735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:37.282735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.8","version":"199'9735","reported_seq":14048,"reported_epoch":199,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-16T09:31:52.899923+0000","last_change":"2024-09-16T09:31:09.438251+0000","last_active":"2024-09-16T09:31:52.899923+0000","last_peered":"2024-09-16T09:31:52.899923+0000","last_clean":"2024-09-16T09:30:20.474651+0000","last_became_active":"2024-09-16T09:31:09.409873+0000","last_became_peered":"2024-09-16T09:31:09.409873+0000","last_unstale":"2024-09-16T09:31:52.899923+0000","last_undegraded":"2024-09-16T09:31:52.899923+0000","last_fullsized":"2024-09-16T09:31:09.256524+0000","mapping_epoch":186,"log_start":"178'5000","ondisk_log_start":"178'5000","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":4735,"log_dups_size":0,"ondisk_log_size":4735,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":392364032,"num_objects":5987,"num_object_clones":0,"num_object_copies":17961,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2357,"num_objects_unfound":0,"num_objects_dirty":5987,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9735,"num_write_kb":503104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2099,"num_bytes_recovered":137560064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,7],"avail_no_missing":["5","7"],"object_location_counts":[{"shards":"0,5,7","objects":2357},{"shards":"5,7","objects":3630}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1","version":"73'79","reported_seq":262,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.273027+0000","last_change":"2024-09-16T09:31:47.248649+0000","last_active":"2024-09-16T09:31:48.273027+0000","last_peered":"2024-09-16T09:31:48.273027+0000","last_clean":"2024-09-16T09:31:48.273027+0000","last_became_active":"2024-09-16T09:30:08.766350+0000","last_became_peered":"2024-09-16T09:30:08.766350+0000","last_unstale":"2024-09-16T09:31:48.273027+0000","last_undegraded":"2024-09-16T09:31:48.273027+0000","last_fullsized":"2024-09-16T09:31:48.273027+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:15.460179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:15.460179+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:32:12.387091+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"187'30","reported_seq":496,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.270657+0000","last_change":"2024-09-16T09:31:09.323281+0000","last_active":"2024-09-16T09:31:48.270657+0000","last_peered":"2024-09-16T09:31:48.270657+0000","last_clean":"2024-09-16T09:31:48.270657+0000","last_became_active":"2024-09-16T09:31:09.278839+0000","last_became_peered":"2024-09-16T09:31:09.278839+0000","last_unstale":"2024-09-16T09:31:48.270657+0000","last_undegraded":"2024-09-16T09:31:48.270657+0000","last_fullsized":"2024-09-16T09:31:48.270657+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'28","last_scrub_stamp":"2024-09-16T09:29:18.937623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:29:18.937623+0000","objects_scrubbed":10,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":311,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.267624+0000","last_change":"2024-09-16T09:29:26.637070+0000","last_active":"2024-09-16T09:31:48.267624+0000","last_peered":"2024-09-16T09:31:48.267624+0000","last_clean":"2024-09-16T09:31:48.267624+0000","last_became_active":"2024-09-16T09:25:01.703076+0000","last_became_peered":"2024-09-16T09:25:01.703076+0000","last_unstale":"2024-09-16T09:31:48.267624+0000","last_undegraded":"2024-09-16T09:31:48.267624+0000","last_fullsized":"2024-09-16T09:31:48.267624+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:26.637027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:26.637027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":320,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.275096+0000","last_change":"2024-09-16T09:29:29.486284+0000","last_active":"2024-09-16T09:31:48.275096+0000","last_peered":"2024-09-16T09:31:48.275096+0000","last_clean":"2024-09-16T09:31:48.275096+0000","last_became_active":"2024-09-16T09:25:00.696237+0000","last_became_peered":"2024-09-16T09:25:00.696237+0000","last_unstale":"2024-09-16T09:31:48.275096+0000","last_undegraded":"2024-09-16T09:31:48.275096+0000","last_fullsized":"2024-09-16T09:31:48.275096+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-16T09:29:29.486188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:29:29.486188+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.0025137430000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":287,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.288919+0000","last_change":"2024-09-16T09:25:05.718319+0000","last_active":"2024-09-16T09:31:48.288919+0000","last_peered":"2024-09-16T09:31:48.288919+0000","last_clean":"2024-09-16T09:31:48.288919+0000","last_became_active":"2024-09-16T09:25:05.717591+0000","last_became_peered":"2024-09-16T09:25:05.717591+0000","last_unstale":"2024-09-16T09:31:48.288919+0000","last_undegraded":"2024-09-16T09:31:48.288919+0000","last_fullsized":"2024-09-16T09:31:48.288919+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:10.167420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":36,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.279913+0000","last_change":"2024-09-16T09:31:09.272967+0000","last_active":"2024-09-16T09:31:48.279913+0000","last_peered":"2024-09-16T09:31:48.279913+0000","last_clean":"2024-09-16T09:31:48.279913+0000","last_became_active":"2024-09-16T09:31:09.272609+0000","last_became_peered":"2024-09-16T09:31:09.272609+0000","last_unstale":"2024-09-16T09:31:48.279913+0000","last_undegraded":"2024-09-16T09:31:48.279913+0000","last_fullsized":"2024-09-16T09:31:48.279913+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:45.201503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:45.201503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"199'9574","reported_seq":18346,"reported_epoch":199,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-16T09:31:52.789514+0000","last_change":"2024-09-16T09:31:09.423422+0000","last_active":"2024-09-16T09:31:52.789514+0000","last_peered":"2024-09-16T09:31:52.789514+0000","last_clean":"2024-09-16T09:30:20.492754+0000","last_became_active":"2024-09-16T09:31:09.398447+0000","last_became_peered":"2024-09-16T09:31:09.398447+0000","last_unstale":"2024-09-16T09:31:52.789514+0000","last_undegraded":"2024-09-16T09:31:52.789514+0000","last_fullsized":"2024-09-16T09:31:09.253245+0000","mapping_epoch":186,"log_start":"180'5300","ondisk_log_start":"180'5300","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":4274,"log_dups_size":0,"ondisk_log_size":4274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":390594560,"num_objects":5960,"num_object_clones":0,"num_object_copies":17880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1222,"num_objects_unfound":0,"num_objects_dirty":5960,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9574,"num_write_kb":497088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4329,"num_bytes_recovered":283705344,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,1],"avail_no_missing":["4","1"],"object_location_counts":[{"shards":"1,2,4","objects":1222},{"shards":"1,4","objects":4738}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.2","version":"73'79","reported_seq":387,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.276321+0000","last_change":"2024-09-16T09:31:47.263716+0000","last_active":"2024-09-16T09:31:48.276321+0000","last_peered":"2024-09-16T09:31:48.276321+0000","last_clean":"2024-09-16T09:31:48.276321+0000","last_became_active":"2024-09-16T09:30:08.765943+0000","last_became_peered":"2024-09-16T09:30:08.765943+0000","last_unstale":"2024-09-16T09:31:48.276321+0000","last_undegraded":"2024-09-16T09:31:48.276321+0000","last_fullsized":"2024-09-16T09:31:48.276321+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:26:35.745837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:26:35.745837+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:32:11.475590+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"66'13","reported_seq":334,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.280649+0000","last_change":"2024-09-16T09:27:44.156057+0000","last_active":"2024-09-16T09:31:48.280649+0000","last_peered":"2024-09-16T09:31:48.280649+0000","last_clean":"2024-09-16T09:31:48.280649+0000","last_became_active":"2024-09-16T09:25:03.713241+0000","last_became_peered":"2024-09-16T09:25:03.713241+0000","last_unstale":"2024-09-16T09:31:48.280649+0000","last_undegraded":"2024-09-16T09:31:48.280649+0000","last_fullsized":"2024-09-16T09:31:48.280649+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'13","last_scrub_stamp":"2024-09-16T09:27:44.155969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:44.155969+0000","objects_scrubbed":5,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":29,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":311,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.269084+0000","last_change":"2024-09-16T09:29:50.827103+0000","last_active":"2024-09-16T09:31:48.269084+0000","last_peered":"2024-09-16T09:31:48.269084+0000","last_clean":"2024-09-16T09:31:48.269084+0000","last_became_active":"2024-09-16T09:25:01.706828+0000","last_became_peered":"2024-09-16T09:25:01.706828+0000","last_unstale":"2024-09-16T09:31:48.269084+0000","last_undegraded":"2024-09-16T09:31:48.269084+0000","last_fullsized":"2024-09-16T09:31:48.269084+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:50.827057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:50.827057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":342,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.285633+0000","last_change":"2024-09-16T09:31:09.281163+0000","last_active":"2024-09-16T09:31:48.285633+0000","last_peered":"2024-09-16T09:31:48.285633+0000","last_clean":"2024-09-16T09:31:48.285633+0000","last_became_active":"2024-09-16T09:31:09.266338+0000","last_became_peered":"2024-09-16T09:31:09.266338+0000","last_unstale":"2024-09-16T09:31:48.285633+0000","last_undegraded":"2024-09-16T09:31:48.285633+0000","last_fullsized":"2024-09-16T09:31:48.285633+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:29:16.104209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:29:16.104209+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0.00055022299999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":311,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.276013+0000","last_change":"2024-09-16T09:29:00.169141+0000","last_active":"2024-09-16T09:31:48.276013+0000","last_peered":"2024-09-16T09:31:48.276013+0000","last_clean":"2024-09-16T09:31:48.276013+0000","last_became_active":"2024-09-16T09:25:05.711539+0000","last_became_peered":"2024-09-16T09:25:05.711539+0000","last_unstale":"2024-09-16T09:31:48.276013+0000","last_undegraded":"2024-09-16T09:31:48.276013+0000","last_fullsized":"2024-09-16T09:31:48.276013+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:00.169077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:00.169077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":298,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.276759+0000","last_change":"2024-09-16T09:29:22.968971+0000","last_active":"2024-09-16T09:31:48.276759+0000","last_peered":"2024-09-16T09:31:48.276759+0000","last_clean":"2024-09-16T09:31:48.276759+0000","last_became_active":"2024-09-16T09:25:07.723282+0000","last_became_peered":"2024-09-16T09:25:07.723282+0000","last_unstale":"2024-09-16T09:31:48.276759+0000","last_undegraded":"2024-09-16T09:31:48.276759+0000","last_fullsized":"2024-09-16T09:31:48.276759+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:22.968908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:22.968908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"199'9597","reported_seq":9702,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:52.797908+0000","last_change":"2024-09-16T09:29:49.448898+0000","last_active":"2024-09-16T09:31:52.797908+0000","last_peered":"2024-09-16T09:31:52.797908+0000","last_clean":"2024-09-16T09:31:52.797908+0000","last_became_active":"2024-09-16T09:29:49.448628+0000","last_became_peered":"2024-09-16T09:29:49.448628+0000","last_unstale":"2024-09-16T09:31:52.797908+0000","last_undegraded":"2024-09-16T09:31:52.797908+0000","last_fullsized":"2024-09-16T09:31:52.797908+0000","mapping_epoch":152,"log_start":"178'5100","ondisk_log_start":"178'5100","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":4497,"log_dups_size":0,"ondisk_log_size":4497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":395640832,"num_objects":6037,"num_object_clones":0,"num_object_copies":18111,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6037,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9597,"num_write_kb":500288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"73'79","reported_seq":272,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.271112+0000","last_change":"2024-09-16T09:31:40.212968+0000","last_active":"2024-09-16T09:31:48.271112+0000","last_peered":"2024-09-16T09:31:48.271112+0000","last_clean":"2024-09-16T09:31:48.271112+0000","last_became_active":"2024-09-16T09:30:08.877110+0000","last_became_peered":"2024-09-16T09:30:08.877110+0000","last_unstale":"2024-09-16T09:31:48.271112+0000","last_undegraded":"2024-09-16T09:31:48.271112+0000","last_fullsized":"2024-09-16T09:31:48.271112+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:31:40.212889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:31:40.212889+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:45.708324+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"66'12","reported_seq":317,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.290274+0000","last_change":"2024-09-16T09:29:02.713335+0000","last_active":"2024-09-16T09:31:48.290274+0000","last_peered":"2024-09-16T09:31:48.290274+0000","last_clean":"2024-09-16T09:31:48.290274+0000","last_became_active":"2024-09-16T09:25:03.715315+0000","last_became_peered":"2024-09-16T09:25:03.715315+0000","last_unstale":"2024-09-16T09:31:48.290274+0000","last_undegraded":"2024-09-16T09:31:48.290274+0000","last_fullsized":"2024-09-16T09:31:48.290274+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-16T09:29:02.713259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:29:02.713259+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:43.220661+0000","scrub_duration":26,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"59'1","reported_seq":334,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.290132+0000","last_change":"2024-09-16T09:31:09.286309+0000","last_active":"2024-09-16T09:31:48.290132+0000","last_peered":"2024-09-16T09:31:48.290132+0000","last_clean":"2024-09-16T09:31:48.290132+0000","last_became_active":"2024-09-16T09:31:09.269627+0000","last_became_peered":"2024-09-16T09:31:09.269627+0000","last_unstale":"2024-09-16T09:31:48.290132+0000","last_undegraded":"2024-09-16T09:31:48.290132+0000","last_fullsized":"2024-09-16T09:31:48.290132+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:27:00.666673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:27:00.666673+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:13.191274+0000","scrub_duration":28,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1986,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"67'5","reported_seq":726,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:51.099076+0000","last_change":"2024-09-16T09:29:19.994791+0000","last_active":"2024-09-16T09:31:51.099076+0000","last_peered":"2024-09-16T09:31:51.099076+0000","last_clean":"2024-09-16T09:31:51.099076+0000","last_became_active":"2024-09-16T09:25:00.696546+0000","last_became_peered":"2024-09-16T09:25:00.696546+0000","last_unstale":"2024-09-16T09:31:51.099076+0000","last_undegraded":"2024-09-16T09:31:51.099076+0000","last_fullsized":"2024-09-16T09:31:51.099076+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'5","last_scrub_stamp":"2024-09-16T09:29:19.994716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:29:19.994716+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0.0059770420000000001,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":405,"num_read_kb":400,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":294,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.273865+0000","last_change":"2024-09-16T09:27:07.337170+0000","last_active":"2024-09-16T09:31:48.273865+0000","last_peered":"2024-09-16T09:31:48.273865+0000","last_clean":"2024-09-16T09:31:48.273865+0000","last_became_active":"2024-09-16T09:25:05.714704+0000","last_became_peered":"2024-09-16T09:25:05.714704+0000","last_unstale":"2024-09-16T09:31:48.273865+0000","last_undegraded":"2024-09-16T09:31:48.273865+0000","last_fullsized":"2024-09-16T09:31:48.273865+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:07.337124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:07.337124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":305,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.270435+0000","last_change":"2024-09-16T09:31:09.271453+0000","last_active":"2024-09-16T09:31:48.270435+0000","last_peered":"2024-09-16T09:31:48.270435+0000","last_clean":"2024-09-16T09:31:48.270435+0000","last_became_active":"2024-09-16T09:31:09.271111+0000","last_became_peered":"2024-09-16T09:31:09.271111+0000","last_unstale":"2024-09-16T09:31:48.270435+0000","last_undegraded":"2024-09-16T09:31:48.270435+0000","last_fullsized":"2024-09-16T09:31:48.270435+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:24.231167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:24.231167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"199'9607","reported_seq":9776,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:52.580272+0000","last_change":"2024-09-16T09:30:56.023710+0000","last_active":"2024-09-16T09:31:52.580272+0000","last_peered":"2024-09-16T09:31:52.580272+0000","last_clean":"2024-09-16T09:31:52.580272+0000","last_became_active":"2024-09-16T09:29:49.450643+0000","last_became_peered":"2024-09-16T09:29:49.450643+0000","last_unstale":"2024-09-16T09:31:52.580272+0000","last_undegraded":"2024-09-16T09:31:52.580272+0000","last_fullsized":"2024-09-16T09:31:52.580272+0000","mapping_epoch":152,"log_start":"174'4400","ondisk_log_start":"174'4400","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"181'5780","last_scrub_stamp":"2024-09-16T09:30:56.023622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:30:56.023622+0000","objects_scrubbed":5716,"log_size":5207,"log_dups_size":0,"ondisk_log_size":5207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:07.379468+0000","scrub_duration":1526,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":390397952,"num_objects":5957,"num_object_clones":0,"num_object_copies":17871,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5957,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9607,"num_write_kb":498048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"66'16","reported_seq":412,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.272406+0000","last_change":"2024-09-16T09:30:50.441875+0000","last_active":"2024-09-16T09:31:48.272406+0000","last_peered":"2024-09-16T09:31:48.272406+0000","last_clean":"2024-09-16T09:31:48.272406+0000","last_became_active":"2024-09-16T09:25:03.714118+0000","last_became_peered":"2024-09-16T09:25:03.714118+0000","last_unstale":"2024-09-16T09:31:48.272406+0000","last_undegraded":"2024-09-16T09:31:48.272406+0000","last_fullsized":"2024-09-16T09:31:48.272406+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'16","last_scrub_stamp":"2024-09-16T09:30:50.441795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:30:50.441795+0000","objects_scrubbed":8,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:00.328016+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":152,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.275011+0000","last_change":"2024-09-16T09:31:09.270603+0000","last_active":"2024-09-16T09:31:48.275011+0000","last_peered":"2024-09-16T09:31:48.275011+0000","last_clean":"2024-09-16T09:31:48.275011+0000","last_became_active":"2024-09-16T09:31:09.270196+0000","last_became_peered":"2024-09-16T09:31:09.270196+0000","last_unstale":"2024-09-16T09:31:48.275011+0000","last_undegraded":"2024-09-16T09:31:48.275011+0000","last_fullsized":"2024-09-16T09:31:48.275011+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:49.195331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:27:49.195331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"73'79","reported_seq":387,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.276513+0000","last_change":"2024-09-16T09:31:47.263760+0000","last_active":"2024-09-16T09:31:48.276513+0000","last_peered":"2024-09-16T09:31:48.276513+0000","last_clean":"2024-09-16T09:31:48.276513+0000","last_became_active":"2024-09-16T09:30:08.870394+0000","last_became_peered":"2024-09-16T09:30:08.870394+0000","last_unstale":"2024-09-16T09:31:48.276513+0000","last_undegraded":"2024-09-16T09:31:48.276513+0000","last_fullsized":"2024-09-16T09:31:48.276513+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:02.723794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:02.723794+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:32:12.473499+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":294,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.276923+0000","last_change":"2024-09-16T09:26:39.699133+0000","last_active":"2024-09-16T09:31:48.276923+0000","last_peered":"2024-09-16T09:31:48.276923+0000","last_clean":"2024-09-16T09:31:48.276923+0000","last_became_active":"2024-09-16T09:25:05.714341+0000","last_became_peered":"2024-09-16T09:25:05.714341+0000","last_unstale":"2024-09-16T09:31:48.276923+0000","last_undegraded":"2024-09-16T09:31:48.276923+0000","last_fullsized":"2024-09-16T09:31:48.276923+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:39.699059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:26:39.699059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":298,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.273701+0000","last_change":"2024-09-16T09:29:00.243528+0000","last_active":"2024-09-16T09:31:48.273701+0000","last_peered":"2024-09-16T09:31:48.273701+0000","last_clean":"2024-09-16T09:31:48.273701+0000","last_became_active":"2024-09-16T09:25:07.714281+0000","last_became_peered":"2024-09-16T09:25:07.714281+0000","last_unstale":"2024-09-16T09:31:48.273701+0000","last_undegraded":"2024-09-16T09:31:48.273701+0000","last_fullsized":"2024-09-16T09:31:48.273701+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:00.243482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:00.243482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.b","version":"73'79","reported_seq":272,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.271812+0000","last_change":"2024-09-16T09:31:47.267322+0000","last_active":"2024-09-16T09:31:48.271812+0000","last_peered":"2024-09-16T09:31:48.271812+0000","last_clean":"2024-09-16T09:31:48.271812+0000","last_became_active":"2024-09-16T09:30:10.784688+0000","last_became_peered":"2024-09-16T09:30:10.784688+0000","last_unstale":"2024-09-16T09:31:48.271812+0000","last_undegraded":"2024-09-16T09:31:48.271812+0000","last_fullsized":"2024-09-16T09:31:48.271812+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:30:03.887571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:30:03.887571+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:32:13.164874+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,5],"acting":[1,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2","version":"199'9452","reported_seq":9557,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:52.777851+0000","last_change":"2024-09-16T09:29:49.445324+0000","last_active":"2024-09-16T09:31:52.777851+0000","last_peered":"2024-09-16T09:31:52.777851+0000","last_clean":"2024-09-16T09:31:52.777851+0000","last_became_active":"2024-09-16T09:29:49.445022+0000","last_became_peered":"2024-09-16T09:29:49.445022+0000","last_unstale":"2024-09-16T09:31:52.777851+0000","last_undegraded":"2024-09-16T09:31:52.777851+0000","last_fullsized":"2024-09-16T09:31:52.777851+0000","mapping_epoch":152,"log_start":"180'5200","ondisk_log_start":"180'5200","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":4252,"log_dups_size":0,"ondisk_log_size":4252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":379060224,"num_objects":5784,"num_object_clones":0,"num_object_copies":17352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5784,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9452,"num_write_kb":487552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"66'11","reported_seq":337,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.268884+0000","last_change":"2024-09-16T09:30:05.646701+0000","last_active":"2024-09-16T09:31:48.268884+0000","last_peered":"2024-09-16T09:31:48.268884+0000","last_clean":"2024-09-16T09:31:48.268884+0000","last_became_active":"2024-09-16T09:26:33.287399+0000","last_became_peered":"2024-09-16T09:26:33.287399+0000","last_unstale":"2024-09-16T09:31:48.268884+0000","last_undegraded":"2024-09-16T09:31:48.268884+0000","last_fullsized":"2024-09-16T09:31:48.268884+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-16T09:30:05.646500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:30:05.646500+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":295,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.267051+0000","last_change":"2024-09-16T09:25:01.708568+0000","last_active":"2024-09-16T09:31:48.267051+0000","last_peered":"2024-09-16T09:31:48.267051+0000","last_clean":"2024-09-16T09:31:48.267051+0000","last_became_active":"2024-09-16T09:25:01.708270+0000","last_became_peered":"2024-09-16T09:25:01.708270+0000","last_unstale":"2024-09-16T09:31:48.267051+0000","last_undegraded":"2024-09-16T09:31:48.267051+0000","last_fullsized":"2024-09-16T09:31:48.267051+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":293,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.273531+0000","last_change":"2024-09-16T09:26:34.171190+0000","last_active":"2024-09-16T09:31:48.273531+0000","last_peered":"2024-09-16T09:31:48.273531+0000","last_clean":"2024-09-16T09:31:48.273531+0000","last_became_active":"2024-09-16T09:25:05.711964+0000","last_became_peered":"2024-09-16T09:25:05.711964+0000","last_unstale":"2024-09-16T09:31:48.273531+0000","last_undegraded":"2024-09-16T09:31:48.273531+0000","last_fullsized":"2024-09-16T09:31:48.273531+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:34.171139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:26:34.171139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":169,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.271987+0000","last_change":"2024-09-16T09:31:09.278959+0000","last_active":"2024-09-16T09:31:48.271987+0000","last_peered":"2024-09-16T09:31:48.271987+0000","last_clean":"2024-09-16T09:31:48.271987+0000","last_became_active":"2024-09-16T09:31:09.278618+0000","last_became_peered":"2024-09-16T09:31:09.278618+0000","last_unstale":"2024-09-16T09:31:48.271987+0000","last_undegraded":"2024-09-16T09:31:48.271987+0000","last_fullsized":"2024-09-16T09:31:48.271987+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:30:33.410232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:30:33.410232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:00.093285+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1","version":"199'9389","reported_seq":9494,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:52.843760+0000","last_change":"2024-09-16T09:29:49.449796+0000","last_active":"2024-09-16T09:31:52.843760+0000","last_peered":"2024-09-16T09:31:52.843760+0000","last_clean":"2024-09-16T09:31:52.843760+0000","last_became_active":"2024-09-16T09:29:49.449475+0000","last_became_peered":"2024-09-16T09:29:49.449475+0000","last_unstale":"2024-09-16T09:31:52.843760+0000","last_undegraded":"2024-09-16T09:31:52.843760+0000","last_fullsized":"2024-09-16T09:31:52.843760+0000","mapping_epoch":152,"log_start":"178'4900","ondisk_log_start":"178'4900","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":4489,"log_dups_size":0,"ondisk_log_size":4489,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":384630784,"num_objects":5869,"num_object_clones":0,"num_object_copies":17607,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5869,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9389,"num_write_kb":488256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.8","version":"73'79","reported_seq":266,"reported_epoch":199,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-16T09:31:48.279537+0000","last_change":"2024-09-16T09:31:09.430087+0000","last_active":"2024-09-16T09:31:48.279537+0000","last_peered":"2024-09-16T09:31:48.279537+0000","last_clean":"2024-09-16T09:31:07.261934+0000","last_became_active":"2024-09-16T09:31:09.406638+0000","last_became_peered":"2024-09-16T09:31:09.406638+0000","last_unstale":"2024-09-16T09:31:48.279537+0000","last_undegraded":"2024-09-16T09:31:09.427685+0000","last_fullsized":"2024-09-16T09:31:48.279537+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:17.482769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:17.482769+0000","objects_scrubbed":1,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262688,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":262688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":["5","2","6"],"object_location_counts":[{"shards":"2,5,6","objects":1}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"66'17","reported_seq":343,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.268164+0000","last_change":"2024-09-16T09:30:04.636115+0000","last_active":"2024-09-16T09:31:48.268164+0000","last_peered":"2024-09-16T09:31:48.268164+0000","last_clean":"2024-09-16T09:31:48.268164+0000","last_became_active":"2024-09-16T09:25:03.713358+0000","last_became_peered":"2024-09-16T09:25:03.713358+0000","last_unstale":"2024-09-16T09:31:48.268164+0000","last_undegraded":"2024-09-16T09:31:48.268164+0000","last_fullsized":"2024-09-16T09:31:48.268164+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'17","last_scrub_stamp":"2024-09-16T09:30:04.636039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:30:04.636039+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":303,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.272809+0000","last_change":"2024-09-16T09:26:04.104188+0000","last_active":"2024-09-16T09:31:48.272809+0000","last_peered":"2024-09-16T09:31:48.272809+0000","last_clean":"2024-09-16T09:31:48.272809+0000","last_became_active":"2024-09-16T09:25:01.703355+0000","last_became_peered":"2024-09-16T09:25:01.703355+0000","last_unstale":"2024-09-16T09:31:48.272809+0000","last_undegraded":"2024-09-16T09:31:48.272809+0000","last_fullsized":"2024-09-16T09:31:48.272809+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:04.104130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:04.104130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":311,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.270511+0000","last_change":"2024-09-16T09:29:58.839132+0000","last_active":"2024-09-16T09:31:48.270511+0000","last_peered":"2024-09-16T09:31:48.270511+0000","last_clean":"2024-09-16T09:31:48.270511+0000","last_became_active":"2024-09-16T09:25:05.716900+0000","last_became_peered":"2024-09-16T09:25:05.716900+0000","last_unstale":"2024-09-16T09:31:48.270511+0000","last_undegraded":"2024-09-16T09:31:48.270511+0000","last_fullsized":"2024-09-16T09:31:48.270511+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:58.839051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:58.839051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":321,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.276176+0000","last_change":"2024-09-16T09:31:09.276869+0000","last_active":"2024-09-16T09:31:48.276176+0000","last_peered":"2024-09-16T09:31:48.276176+0000","last_clean":"2024-09-16T09:31:48.276176+0000","last_became_active":"2024-09-16T09:31:09.276275+0000","last_became_peered":"2024-09-16T09:31:09.276275+0000","last_unstale":"2024-09-16T09:31:48.276176+0000","last_undegraded":"2024-09-16T09:31:48.276176+0000","last_fullsized":"2024-09-16T09:31:48.276176+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:30:03.946025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:30:03.946025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.0","version":"199'9646","reported_seq":14318,"reported_epoch":199,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-16T09:31:52.898873+0000","last_change":"2024-09-16T09:31:09.476408+0000","last_active":"2024-09-16T09:31:52.898873+0000","last_peered":"2024-09-16T09:31:52.898873+0000","last_clean":"2024-09-16T09:30:20.486290+0000","last_became_active":"2024-09-16T09:31:09.399514+0000","last_became_peered":"2024-09-16T09:31:09.399514+0000","last_unstale":"2024-09-16T09:31:52.898873+0000","last_undegraded":"2024-09-16T09:31:52.898873+0000","last_fullsized":"2024-09-16T09:31:09.254572+0000","mapping_epoch":186,"log_start":"177'4900","ondisk_log_start":"177'4900","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":4746,"log_dups_size":0,"ondisk_log_size":4746,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":386007040,"num_objects":5890,"num_object_clones":0,"num_object_copies":17670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2424,"num_objects_unfound":0,"num_objects_dirty":5890,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9646,"num_write_kb":497152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2278,"num_bytes_recovered":149291008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,1],"avail_no_missing":["5","1"],"object_location_counts":[{"shards":"1,4,5","objects":2424},{"shards":"1,5","objects":3466}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.9","version":"73'79","reported_seq":374,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.276307+0000","last_change":"2024-09-16T09:29:19.013948+0000","last_active":"2024-09-16T09:31:48.276307+0000","last_peered":"2024-09-16T09:31:48.276307+0000","last_clean":"2024-09-16T09:31:48.276307+0000","last_became_active":"2024-09-16T09:26:35.126066+0000","last_became_peered":"2024-09-16T09:26:35.126066+0000","last_unstale":"2024-09-16T09:31:48.276307+0000","last_undegraded":"2024-09-16T09:31:48.276307+0000","last_fullsized":"2024-09-16T09:31:48.276307+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:19.013865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:19.013865+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"66'10","reported_seq":306,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.267590+0000","last_change":"2024-09-16T09:25:03.708144+0000","last_active":"2024-09-16T09:31:48.267590+0000","last_peered":"2024-09-16T09:31:48.267590+0000","last_clean":"2024-09-16T09:31:48.267590+0000","last_became_active":"2024-09-16T09:25:03.707723+0000","last_became_peered":"2024-09-16T09:25:03.707723+0000","last_unstale":"2024-09-16T09:31:48.267590+0000","last_undegraded":"2024-09-16T09:31:48.267590+0000","last_fullsized":"2024-09-16T09:31:48.267590+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":168,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.288224+0000","last_change":"2024-09-16T09:31:32.253397+0000","last_active":"2024-09-16T09:31:48.288224+0000","last_peered":"2024-09-16T09:31:48.288224+0000","last_clean":"2024-09-16T09:31:48.288224+0000","last_became_active":"2024-09-16T09:31:09.288825+0000","last_became_peered":"2024-09-16T09:31:09.288825+0000","last_unstale":"2024-09-16T09:31:48.288224+0000","last_undegraded":"2024-09-16T09:31:48.288224+0000","last_fullsized":"2024-09-16T09:31:48.288224+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:31:32.253306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:31:32.253306+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:45.682494+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":405,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:51.843803+0000","last_change":"2024-09-16T09:29:29.866520+0000","last_active":"2024-09-16T09:31:51.843803+0000","last_peered":"2024-09-16T09:31:51.843803+0000","last_clean":"2024-09-16T09:31:51.843803+0000","last_became_active":"2024-09-16T09:25:05.711590+0000","last_became_peered":"2024-09-16T09:25:05.711590+0000","last_unstale":"2024-09-16T09:31:51.843803+0000","last_undegraded":"2024-09-16T09:31:51.843803+0000","last_fullsized":"2024-09-16T09:31:51.843803+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:29:29.866460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:29.866460+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":307,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.268777+0000","last_change":"2024-09-16T09:29:02.667332+0000","last_active":"2024-09-16T09:31:48.268777+0000","last_peered":"2024-09-16T09:31:48.268777+0000","last_clean":"2024-09-16T09:31:48.268777+0000","last_became_active":"2024-09-16T09:25:07.718958+0000","last_became_peered":"2024-09-16T09:25:07.718958+0000","last_unstale":"2024-09-16T09:31:48.268777+0000","last_undegraded":"2024-09-16T09:31:48.268777+0000","last_fullsized":"2024-09-16T09:31:48.268777+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:02.667259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:02.667259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.e","version":"73'79","reported_seq":96,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.267867+0000","last_change":"2024-09-16T09:31:47.249302+0000","last_active":"2024-09-16T09:31:48.267867+0000","last_peered":"2024-09-16T09:31:48.267867+0000","last_clean":"2024-09-16T09:31:48.267867+0000","last_became_active":"2024-09-16T09:30:10.705072+0000","last_became_peered":"2024-09-16T09:30:10.705072+0000","last_unstale":"2024-09-16T09:31:48.267867+0000","last_undegraded":"2024-09-16T09:31:48.267867+0000","last_fullsized":"2024-09-16T09:31:48.267867+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:26:35.170229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:26:35.170229+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:32:12.841915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26835,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":10,"num_write":5,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":26835,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,3],"acting":[4,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.7","version":"199'9555","reported_seq":9660,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:52.811350+0000","last_change":"2024-09-16T09:29:49.839740+0000","last_active":"2024-09-16T09:31:52.811350+0000","last_peered":"2024-09-16T09:31:52.811350+0000","last_clean":"2024-09-16T09:31:52.811350+0000","last_became_active":"2024-09-16T09:29:49.839437+0000","last_became_peered":"2024-09-16T09:29:49.839437+0000","last_unstale":"2024-09-16T09:31:52.811350+0000","last_undegraded":"2024-09-16T09:31:52.811350+0000","last_fullsized":"2024-09-16T09:31:52.811350+0000","mapping_epoch":152,"log_start":"178'5000","ondisk_log_start":"178'5000","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":4555,"log_dups_size":0,"ondisk_log_size":4555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":385941504,"num_objects":5889,"num_object_clones":0,"num_object_copies":17667,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5889,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9555,"num_write_kb":494208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"66'9","reported_seq":331,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.287997+0000","last_change":"2024-09-16T09:31:31.211495+0000","last_active":"2024-09-16T09:31:48.287997+0000","last_peered":"2024-09-16T09:31:48.287997+0000","last_clean":"2024-09-16T09:31:48.287997+0000","last_became_active":"2024-09-16T09:25:03.712230+0000","last_became_peered":"2024-09-16T09:25:03.712230+0000","last_unstale":"2024-09-16T09:31:48.287997+0000","last_undegraded":"2024-09-16T09:31:48.287997+0000","last_fullsized":"2024-09-16T09:31:48.287997+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:31:31.211420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:31:31.211420+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:36.499658+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":316,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.278187+0000","last_change":"2024-09-16T09:31:09.277413+0000","last_active":"2024-09-16T09:31:48.278187+0000","last_peered":"2024-09-16T09:31:48.278187+0000","last_clean":"2024-09-16T09:31:48.278187+0000","last_became_active":"2024-09-16T09:31:09.276516+0000","last_became_peered":"2024-09-16T09:31:09.276516+0000","last_unstale":"2024-09-16T09:31:48.278187+0000","last_undegraded":"2024-09-16T09:31:48.278187+0000","last_fullsized":"2024-09-16T09:31:48.278187+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:26.061106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:26.061106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":317,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.275437+0000","last_change":"2024-09-16T09:31:09.275219+0000","last_active":"2024-09-16T09:31:48.275437+0000","last_peered":"2024-09-16T09:31:48.275437+0000","last_clean":"2024-09-16T09:31:48.275437+0000","last_became_active":"2024-09-16T09:31:09.274702+0000","last_became_peered":"2024-09-16T09:31:09.274702+0000","last_unstale":"2024-09-16T09:31:48.275437+0000","last_undegraded":"2024-09-16T09:31:48.275437+0000","last_fullsized":"2024-09-16T09:31:48.275437+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:57.220068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:28:57.220068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":33,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":307,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.290049+0000","last_change":"2024-09-16T09:31:27.185957+0000","last_active":"2024-09-16T09:31:48.290049+0000","last_peered":"2024-09-16T09:31:48.290049+0000","last_clean":"2024-09-16T09:31:48.290049+0000","last_became_active":"2024-09-16T09:25:07.718778+0000","last_became_peered":"2024-09-16T09:25:07.718778+0000","last_unstale":"2024-09-16T09:31:48.290049+0000","last_undegraded":"2024-09-16T09:31:48.290049+0000","last_fullsized":"2024-09-16T09:31:48.290049+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:31:27.185801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:31:27.185801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:39.017009+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.f","version":"73'79","reported_seq":348,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.273388+0000","last_change":"2024-09-16T09:31:47.259689+0000","last_active":"2024-09-16T09:31:48.273388+0000","last_peered":"2024-09-16T09:31:48.273388+0000","last_clean":"2024-09-16T09:31:48.273388+0000","last_became_active":"2024-09-16T09:30:10.693477+0000","last_became_peered":"2024-09-16T09:30:10.693477+0000","last_unstale":"2024-09-16T09:31:48.273388+0000","last_undegraded":"2024-09-16T09:31:48.273388+0000","last_fullsized":"2024-09-16T09:31:48.273388+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:17.534429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:17.534429+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:32:13.472806+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.6","version":"199'9851","reported_seq":9956,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.042411+0000","last_change":"2024-09-16T09:29:50.008888+0000","last_active":"2024-09-16T09:31:53.042411+0000","last_peered":"2024-09-16T09:31:53.042411+0000","last_clean":"2024-09-16T09:31:53.042411+0000","last_became_active":"2024-09-16T09:29:50.008656+0000","last_became_peered":"2024-09-16T09:29:50.008656+0000","last_unstale":"2024-09-16T09:31:53.042411+0000","last_undegraded":"2024-09-16T09:31:53.042411+0000","last_fullsized":"2024-09-16T09:31:53.042411+0000","mapping_epoch":152,"log_start":"177'4700","ondisk_log_start":"177'4700","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":5151,"log_dups_size":0,"ondisk_log_size":5151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":391315456,"num_objects":5971,"num_object_clones":0,"num_object_copies":17913,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5971,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9851,"num_write_kb":506304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"66'19","reported_seq":338,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.272529+0000","last_change":"2024-09-16T09:29:02.554357+0000","last_active":"2024-09-16T09:31:48.272529+0000","last_peered":"2024-09-16T09:31:48.272529+0000","last_clean":"2024-09-16T09:31:48.272529+0000","last_became_active":"2024-09-16T09:25:03.712783+0000","last_became_peered":"2024-09-16T09:25:03.712783+0000","last_unstale":"2024-09-16T09:31:48.272529+0000","last_undegraded":"2024-09-16T09:31:48.272529+0000","last_fullsized":"2024-09-16T09:31:48.272529+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-16T09:29:02.554274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:29:02.554274+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:17.371584+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"59'1","reported_seq":316,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.277054+0000","last_change":"2024-09-16T09:29:51.025505+0000","last_active":"2024-09-16T09:31:48.277054+0000","last_peered":"2024-09-16T09:31:48.277054+0000","last_clean":"2024-09-16T09:31:48.277054+0000","last_became_active":"2024-09-16T09:25:01.708069+0000","last_became_peered":"2024-09-16T09:25:01.708069+0000","last_unstale":"2024-09-16T09:31:48.277054+0000","last_undegraded":"2024-09-16T09:31:48.277054+0000","last_fullsized":"2024-09-16T09:31:48.277054+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:29:51.025444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:51.025444+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":295,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.274852+0000","last_change":"2024-09-16T09:27:11.067059+0000","last_active":"2024-09-16T09:31:48.274852+0000","last_peered":"2024-09-16T09:31:48.274852+0000","last_clean":"2024-09-16T09:31:48.274852+0000","last_became_active":"2024-09-16T09:25:05.709264+0000","last_became_peered":"2024-09-16T09:25:05.709264+0000","last_unstale":"2024-09-16T09:31:48.274852+0000","last_undegraded":"2024-09-16T09:31:48.274852+0000","last_fullsized":"2024-09-16T09:31:48.274852+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:11.066986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:11.066986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":290,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.276419+0000","last_change":"2024-09-16T09:27:25.865107+0000","last_active":"2024-09-16T09:31:48.276419+0000","last_peered":"2024-09-16T09:31:48.276419+0000","last_clean":"2024-09-16T09:31:48.276419+0000","last_became_active":"2024-09-16T09:25:07.718750+0000","last_became_peered":"2024-09-16T09:25:07.718750+0000","last_unstale":"2024-09-16T09:31:48.276419+0000","last_undegraded":"2024-09-16T09:31:48.276419+0000","last_fullsized":"2024-09-16T09:31:48.276419+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:25.865030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:25.865030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.c","version":"73'79","reported_seq":40,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.278730+0000","last_change":"2024-09-16T09:31:09.290919+0000","last_active":"2024-09-16T09:31:48.278730+0000","last_peered":"2024-09-16T09:31:48.278730+0000","last_clean":"2024-09-16T09:31:48.278730+0000","last_became_active":"2024-09-16T09:31:09.273770+0000","last_became_peered":"2024-09-16T09:31:09.273770+0000","last_unstale":"2024-09-16T09:31:48.278730+0000","last_undegraded":"2024-09-16T09:31:48.278730+0000","last_fullsized":"2024-09-16T09:31:48.278730+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:30:22.614462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:30:22.614462+0000","objects_scrubbed":1,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":327680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.5","version":"199'9663","reported_seq":9841,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:52.595693+0000","last_change":"2024-09-16T09:31:06.121217+0000","last_active":"2024-09-16T09:31:52.595693+0000","last_peered":"2024-09-16T09:31:52.595693+0000","last_clean":"2024-09-16T09:31:52.595693+0000","last_became_active":"2024-09-16T09:29:50.009013+0000","last_became_peered":"2024-09-16T09:29:50.009013+0000","last_unstale":"2024-09-16T09:31:52.595693+0000","last_undegraded":"2024-09-16T09:31:52.595693+0000","last_fullsized":"2024-09-16T09:31:52.595693+0000","mapping_epoch":152,"log_start":"174'4413","ondisk_log_start":"174'4413","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"184'6587","last_scrub_stamp":"2024-09-16T09:31:06.120990+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:31:06.120990+0000","objects_scrubbed":6523,"log_size":5250,"log_dups_size":0,"ondisk_log_size":5250,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:20.260788+0000","scrub_duration":1690,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":398524416,"num_objects":6081,"num_object_clones":0,"num_object_copies":18243,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6081,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9663,"num_write_kb":503808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"66'12","reported_seq":368,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.266861+0000","last_change":"2024-09-16T09:31:09.296915+0000","last_active":"2024-09-16T09:31:48.266861+0000","last_peered":"2024-09-16T09:31:48.266861+0000","last_clean":"2024-09-16T09:31:48.266861+0000","last_became_active":"2024-09-16T09:31:09.269047+0000","last_became_peered":"2024-09-16T09:31:09.269047+0000","last_unstale":"2024-09-16T09:31:48.266861+0000","last_undegraded":"2024-09-16T09:31:48.266861+0000","last_fullsized":"2024-09-16T09:31:48.266861+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-16T09:30:21.728668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:30:21.728668+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":310,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.276810+0000","last_change":"2024-09-16T09:29:53.075288+0000","last_active":"2024-09-16T09:31:48.276810+0000","last_peered":"2024-09-16T09:31:48.276810+0000","last_clean":"2024-09-16T09:31:48.276810+0000","last_became_active":"2024-09-16T09:25:01.707528+0000","last_became_peered":"2024-09-16T09:25:01.707528+0000","last_unstale":"2024-09-16T09:31:48.276810+0000","last_undegraded":"2024-09-16T09:31:48.276810+0000","last_fullsized":"2024-09-16T09:31:48.276810+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:53.075232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:53.075232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":303,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.274793+0000","last_change":"2024-09-16T09:27:49.965628+0000","last_active":"2024-09-16T09:31:48.274793+0000","last_peered":"2024-09-16T09:31:48.274793+0000","last_clean":"2024-09-16T09:31:48.274793+0000","last_became_active":"2024-09-16T09:25:05.715253+0000","last_became_peered":"2024-09-16T09:25:05.715253+0000","last_unstale":"2024-09-16T09:31:48.274793+0000","last_undegraded":"2024-09-16T09:31:48.274793+0000","last_fullsized":"2024-09-16T09:31:48.274793+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:49.965424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:49.965424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":291,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.284117+0000","last_change":"2024-09-16T09:27:56.059755+0000","last_active":"2024-09-16T09:31:48.284117+0000","last_peered":"2024-09-16T09:31:48.284117+0000","last_clean":"2024-09-16T09:31:48.284117+0000","last_became_active":"2024-09-16T09:25:07.718899+0000","last_became_peered":"2024-09-16T09:25:07.718899+0000","last_unstale":"2024-09-16T09:31:48.284117+0000","last_undegraded":"2024-09-16T09:31:48.284117+0000","last_fullsized":"2024-09-16T09:31:48.284117+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:56.059687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:56.059687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.d","version":"73'79","reported_seq":96,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.276508+0000","last_change":"2024-09-16T09:31:47.263954+0000","last_active":"2024-09-16T09:31:48.276508+0000","last_peered":"2024-09-16T09:31:48.276508+0000","last_clean":"2024-09-16T09:31:48.276508+0000","last_became_active":"2024-09-16T09:30:10.709958+0000","last_became_peered":"2024-09-16T09:30:10.709958+0000","last_unstale":"2024-09-16T09:31:48.276508+0000","last_undegraded":"2024-09-16T09:31:48.276508+0000","last_fullsized":"2024-09-16T09:31:48.276508+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:58.568841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:58.568841+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:32:13.456459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26835,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":10,"num_write":5,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":26835,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.4","version":"199'9612","reported_seq":9717,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:52.908063+0000","last_change":"2024-09-16T09:29:49.444857+0000","last_active":"2024-09-16T09:31:52.908063+0000","last_peered":"2024-09-16T09:31:52.908063+0000","last_clean":"2024-09-16T09:31:52.908063+0000","last_became_active":"2024-09-16T09:29:49.444515+0000","last_became_peered":"2024-09-16T09:29:49.444515+0000","last_unstale":"2024-09-16T09:31:52.908063+0000","last_undegraded":"2024-09-16T09:31:52.908063+0000","last_fullsized":"2024-09-16T09:31:52.908063+0000","mapping_epoch":152,"log_start":"177'4900","ondisk_log_start":"177'4900","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":4712,"log_dups_size":0,"ondisk_log_size":4712,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":392953856,"num_objects":5996,"num_object_clones":0,"num_object_copies":17988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5996,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9612,"num_write_kb":499456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"66'15","reported_seq":323,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.279789+0000","last_change":"2024-09-16T09:26:33.138828+0000","last_active":"2024-09-16T09:31:48.279789+0000","last_peered":"2024-09-16T09:31:48.279789+0000","last_clean":"2024-09-16T09:31:48.279789+0000","last_became_active":"2024-09-16T09:25:03.715733+0000","last_became_peered":"2024-09-16T09:25:03.715733+0000","last_unstale":"2024-09-16T09:31:48.279789+0000","last_undegraded":"2024-09-16T09:31:48.279789+0000","last_fullsized":"2024-09-16T09:31:48.279789+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-16T09:26:33.138603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:26:33.138603+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":314,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.278107+0000","last_change":"2024-09-16T09:26:26.705847+0000","last_active":"2024-09-16T09:31:48.278107+0000","last_peered":"2024-09-16T09:31:48.278107+0000","last_clean":"2024-09-16T09:31:48.278107+0000","last_became_active":"2024-09-16T09:25:01.707372+0000","last_became_peered":"2024-09-16T09:25:01.707372+0000","last_unstale":"2024-09-16T09:31:48.278107+0000","last_undegraded":"2024-09-16T09:31:48.278107+0000","last_fullsized":"2024-09-16T09:31:48.278107+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-16T09:26:26.705773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:26.705773+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"66'8","reported_seq":401,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.276212+0000","last_change":"2024-09-16T09:30:05.138362+0000","last_active":"2024-09-16T09:31:48.276212+0000","last_peered":"2024-09-16T09:31:48.276212+0000","last_clean":"2024-09-16T09:31:48.276212+0000","last_became_active":"2024-09-16T09:25:05.716189+0000","last_became_peered":"2024-09-16T09:25:05.716189+0000","last_unstale":"2024-09-16T09:31:48.276212+0000","last_undegraded":"2024-09-16T09:31:48.276212+0000","last_fullsized":"2024-09-16T09:31:48.276212+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:30:05.138288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:30:05.138288+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":290,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.274170+0000","last_change":"2024-09-16T09:27:20.219376+0000","last_active":"2024-09-16T09:31:48.274170+0000","last_peered":"2024-09-16T09:31:48.274170+0000","last_clean":"2024-09-16T09:31:48.274170+0000","last_became_active":"2024-09-16T09:25:07.717213+0000","last_became_peered":"2024-09-16T09:25:07.717213+0000","last_unstale":"2024-09-16T09:31:48.274170+0000","last_undegraded":"2024-09-16T09:31:48.274170+0000","last_fullsized":"2024-09-16T09:31:48.274170+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:20.219333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:20.219333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.12","version":"73'79","reported_seq":96,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.274917+0000","last_change":"2024-09-16T09:31:47.251225+0000","last_active":"2024-09-16T09:31:48.274917+0000","last_peered":"2024-09-16T09:31:48.274917+0000","last_clean":"2024-09-16T09:31:48.274917+0000","last_became_active":"2024-09-16T09:30:10.731413+0000","last_became_peered":"2024-09-16T09:30:10.731413+0000","last_unstale":"2024-09-16T09:31:48.274917+0000","last_undegraded":"2024-09-16T09:31:48.274917+0000","last_fullsized":"2024-09-16T09:31:48.274917+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:26:35.745837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:26:35.745837+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:32:14.398546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26835,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":10,"num_write":6,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":26835,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":302,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.272006+0000","last_change":"2024-09-16T09:26:33.139405+0000","last_active":"2024-09-16T09:31:48.272006+0000","last_peered":"2024-09-16T09:31:48.272006+0000","last_clean":"2024-09-16T09:31:48.272006+0000","last_became_active":"2024-09-16T09:25:01.703202+0000","last_became_peered":"2024-09-16T09:25:01.703202+0000","last_unstale":"2024-09-16T09:31:48.272006+0000","last_undegraded":"2024-09-16T09:31:48.272006+0000","last_fullsized":"2024-09-16T09:31:48.272006+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:33.139360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:33.139360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:10.508509+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"66'6","reported_seq":44,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.278840+0000","last_change":"2024-09-16T09:31:09.297514+0000","last_active":"2024-09-16T09:31:48.278840+0000","last_peered":"2024-09-16T09:31:48.278840+0000","last_clean":"2024-09-16T09:31:48.278840+0000","last_became_active":"2024-09-16T09:31:09.270685+0000","last_became_peered":"2024-09-16T09:31:09.270685+0000","last_unstale":"2024-09-16T09:31:48.278840+0000","last_undegraded":"2024-09-16T09:31:48.278840+0000","last_fullsized":"2024-09-16T09:31:48.278840+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'6","last_scrub_stamp":"2024-09-16T09:26:23.458794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:26:23.458794+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":8,"num_read_kb":5,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":301,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.273418+0000","last_change":"2024-09-16T09:29:31.496374+0000","last_active":"2024-09-16T09:31:48.273418+0000","last_peered":"2024-09-16T09:31:48.273418+0000","last_clean":"2024-09-16T09:31:48.273418+0000","last_became_active":"2024-09-16T09:25:05.715616+0000","last_became_peered":"2024-09-16T09:25:05.715616+0000","last_unstale":"2024-09-16T09:31:48.273418+0000","last_undegraded":"2024-09-16T09:31:48.273418+0000","last_fullsized":"2024-09-16T09:31:48.273418+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:31.496304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:31.496304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":289,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.276668+0000","last_change":"2024-09-16T09:26:28.733381+0000","last_active":"2024-09-16T09:31:48.276668+0000","last_peered":"2024-09-16T09:31:48.276668+0000","last_clean":"2024-09-16T09:31:48.276668+0000","last_became_active":"2024-09-16T09:25:07.721059+0000","last_became_peered":"2024-09-16T09:25:07.721059+0000","last_unstale":"2024-09-16T09:31:48.276668+0000","last_undegraded":"2024-09-16T09:31:48.276668+0000","last_fullsized":"2024-09-16T09:31:48.276668+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:28.733324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:28.733324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.13","version":"73'79","reported_seq":285,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.270971+0000","last_change":"2024-09-16T09:31:09.288677+0000","last_active":"2024-09-16T09:31:48.270971+0000","last_peered":"2024-09-16T09:31:48.270971+0000","last_clean":"2024-09-16T09:31:48.270971+0000","last_became_active":"2024-09-16T09:31:09.288233+0000","last_became_peered":"2024-09-16T09:31:09.288233+0000","last_unstale":"2024-09-16T09:31:48.270971+0000","last_undegraded":"2024-09-16T09:31:48.270971+0000","last_fullsized":"2024-09-16T09:31:48.270971+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:30:21.714369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:30:21.714369+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:31:50.034245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.16","version":"66'9","reported_seq":364,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.291907+0000","last_change":"2024-09-16T09:31:09.309570+0000","last_active":"2024-09-16T09:31:48.291907+0000","last_peered":"2024-09-16T09:31:48.291907+0000","last_clean":"2024-09-16T09:31:48.291907+0000","last_became_active":"2024-09-16T09:31:09.273498+0000","last_became_peered":"2024-09-16T09:31:09.273498+0000","last_unstale":"2024-09-16T09:31:48.291907+0000","last_undegraded":"2024-09-16T09:31:48.291907+0000","last_fullsized":"2024-09-16T09:31:48.291907+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:29:58.867513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:29:58.867513+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":309,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.276541+0000","last_change":"2024-09-16T09:28:50.698331+0000","last_active":"2024-09-16T09:31:48.276541+0000","last_peered":"2024-09-16T09:31:48.276541+0000","last_clean":"2024-09-16T09:31:48.276541+0000","last_became_active":"2024-09-16T09:25:01.706491+0000","last_became_peered":"2024-09-16T09:25:01.706491+0000","last_unstale":"2024-09-16T09:31:48.276541+0000","last_undegraded":"2024-09-16T09:31:48.276541+0000","last_fullsized":"2024-09-16T09:31:48.276541+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:50.698288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:28:50.698288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":160,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.280455+0000","last_change":"2024-09-16T09:31:09.271558+0000","last_active":"2024-09-16T09:31:48.280455+0000","last_peered":"2024-09-16T09:31:48.280455+0000","last_clean":"2024-09-16T09:31:48.280455+0000","last_became_active":"2024-09-16T09:31:09.271223+0000","last_became_peered":"2024-09-16T09:31:09.271223+0000","last_unstale":"2024-09-16T09:31:48.280455+0000","last_undegraded":"2024-09-16T09:31:48.280455+0000","last_fullsized":"2024-09-16T09:31:48.280455+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:53.810580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:28:53.810580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":290,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.275181+0000","last_change":"2024-09-16T09:26:35.866582+0000","last_active":"2024-09-16T09:31:48.275181+0000","last_peered":"2024-09-16T09:31:48.275181+0000","last_clean":"2024-09-16T09:31:48.275181+0000","last_became_active":"2024-09-16T09:25:07.722050+0000","last_became_peered":"2024-09-16T09:25:07.722050+0000","last_unstale":"2024-09-16T09:31:48.275181+0000","last_undegraded":"2024-09-16T09:31:48.275181+0000","last_fullsized":"2024-09-16T09:31:48.275181+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:35.866512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:35.866512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:08.904503+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.10","version":"73'79","reported_seq":96,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.273061+0000","last_change":"2024-09-16T09:31:47.249070+0000","last_active":"2024-09-16T09:31:48.273061+0000","last_peered":"2024-09-16T09:31:48.273061+0000","last_clean":"2024-09-16T09:31:48.273061+0000","last_became_active":"2024-09-16T09:30:10.692770+0000","last_became_peered":"2024-09-16T09:30:10.692770+0000","last_unstale":"2024-09-16T09:31:48.273061+0000","last_undegraded":"2024-09-16T09:31:48.273061+0000","last_fullsized":"2024-09-16T09:31:48.273061+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:02.723794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:02.723794+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:32:13.396012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26835,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":10,"num_write":5,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":26835,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"66'9","reported_seq":321,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.273731+0000","last_change":"2024-09-16T09:29:32.485165+0000","last_active":"2024-09-16T09:31:48.273731+0000","last_peered":"2024-09-16T09:31:48.273731+0000","last_clean":"2024-09-16T09:31:48.273731+0000","last_became_active":"2024-09-16T09:25:03.717310+0000","last_became_peered":"2024-09-16T09:25:03.717310+0000","last_unstale":"2024-09-16T09:31:48.273731+0000","last_undegraded":"2024-09-16T09:31:48.273731+0000","last_fullsized":"2024-09-16T09:31:48.273731+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:29:32.485084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:29:32.485084+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":324,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.288529+0000","last_change":"2024-09-16T09:31:09.273453+0000","last_active":"2024-09-16T09:31:48.288529+0000","last_peered":"2024-09-16T09:31:48.288529+0000","last_clean":"2024-09-16T09:31:48.288529+0000","last_became_active":"2024-09-16T09:31:09.272848+0000","last_became_peered":"2024-09-16T09:31:09.272848+0000","last_unstale":"2024-09-16T09:31:48.288529+0000","last_undegraded":"2024-09-16T09:31:48.288529+0000","last_fullsized":"2024-09-16T09:31:48.288529+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.950144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.950144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"187'11","reported_seq":230,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.276877+0000","last_change":"2024-09-16T09:31:09.311243+0000","last_active":"2024-09-16T09:31:48.276877+0000","last_peered":"2024-09-16T09:31:48.276877+0000","last_clean":"2024-09-16T09:31:48.276877+0000","last_became_active":"2024-09-16T09:31:09.277688+0000","last_became_peered":"2024-09-16T09:31:09.277688+0000","last_unstale":"2024-09-16T09:31:48.276877+0000","last_undegraded":"2024-09-16T09:31:48.276877+0000","last_fullsized":"2024-09-16T09:31:48.276877+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:28:53.714413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:28:53.714413+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":306,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.267495+0000","last_change":"2024-09-16T09:29:49.474440+0000","last_active":"2024-09-16T09:31:48.267495+0000","last_peered":"2024-09-16T09:31:48.267495+0000","last_clean":"2024-09-16T09:31:48.267495+0000","last_became_active":"2024-09-16T09:25:07.718810+0000","last_became_peered":"2024-09-16T09:25:07.718810+0000","last_unstale":"2024-09-16T09:31:48.267495+0000","last_undegraded":"2024-09-16T09:31:48.267495+0000","last_fullsized":"2024-09-16T09:31:48.267495+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:49.474391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:49.474391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.11","version":"73'79","reported_seq":185,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.290216+0000","last_change":"2024-09-16T09:31:09.272938+0000","last_active":"2024-09-16T09:31:48.290216+0000","last_peered":"2024-09-16T09:31:48.290216+0000","last_clean":"2024-09-16T09:31:48.290216+0000","last_became_active":"2024-09-16T09:31:09.272208+0000","last_became_peered":"2024-09-16T09:31:09.272208+0000","last_unstale":"2024-09-16T09:31:48.290216+0000","last_undegraded":"2024-09-16T09:31:48.290216+0000","last_fullsized":"2024-09-16T09:31:48.290216+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:30:21.651575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:30:21.651575+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.14","version":"66'10","reported_seq":181,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.270611+0000","last_change":"2024-09-16T09:28:56.825675+0000","last_active":"2024-09-16T09:31:48.270611+0000","last_peered":"2024-09-16T09:31:48.270611+0000","last_clean":"2024-09-16T09:31:48.270611+0000","last_became_active":"2024-09-16T09:28:52.979059+0000","last_became_peered":"2024-09-16T09:28:52.979059+0000","last_unstale":"2024-09-16T09:31:48.270611+0000","last_undegraded":"2024-09-16T09:31:48.270611+0000","last_fullsized":"2024-09-16T09:31:48.270611+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-16T09:28:56.825476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:56.825476+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":14,"num_read_kb":9,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":309,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.277673+0000","last_change":"2024-09-16T09:29:56.021413+0000","last_active":"2024-09-16T09:31:48.277673+0000","last_peered":"2024-09-16T09:31:48.277673+0000","last_clean":"2024-09-16T09:31:48.277673+0000","last_became_active":"2024-09-16T09:25:01.707513+0000","last_became_peered":"2024-09-16T09:25:01.707513+0000","last_unstale":"2024-09-16T09:31:48.277673+0000","last_undegraded":"2024-09-16T09:31:48.277673+0000","last_fullsized":"2024-09-16T09:31:48.277673+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:56.021340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:56.021340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"66'8","reported_seq":400,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.273602+0000","last_change":"2024-09-16T09:29:48.485781+0000","last_active":"2024-09-16T09:31:48.273602+0000","last_peered":"2024-09-16T09:31:48.273602+0000","last_clean":"2024-09-16T09:31:48.273602+0000","last_became_active":"2024-09-16T09:25:05.713833+0000","last_became_peered":"2024-09-16T09:25:05.713833+0000","last_unstale":"2024-09-16T09:31:48.273602+0000","last_undegraded":"2024-09-16T09:31:48.273602+0000","last_fullsized":"2024-09-16T09:31:48.273602+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:29:48.485727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.485727+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":312,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.289832+0000","last_change":"2024-09-16T09:31:09.276853+0000","last_active":"2024-09-16T09:31:48.289832+0000","last_peered":"2024-09-16T09:31:48.289832+0000","last_clean":"2024-09-16T09:31:48.289832+0000","last_became_active":"2024-09-16T09:31:09.275168+0000","last_became_peered":"2024-09-16T09:31:09.275168+0000","last_unstale":"2024-09-16T09:31:48.289832+0000","last_undegraded":"2024-09-16T09:31:48.289832+0000","last_fullsized":"2024-09-16T09:31:48.289832+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:52.958303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:52.958303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"66'11","reported_seq":328,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.267362+0000","last_change":"2024-09-16T09:26:33.302713+0000","last_active":"2024-09-16T09:31:48.267362+0000","last_peered":"2024-09-16T09:31:48.267362+0000","last_clean":"2024-09-16T09:31:48.267362+0000","last_became_active":"2024-09-16T09:26:33.291265+0000","last_became_peered":"2024-09-16T09:26:33.291265+0000","last_unstale":"2024-09-16T09:31:48.267362+0000","last_undegraded":"2024-09-16T09:31:48.267362+0000","last_fullsized":"2024-09-16T09:31:48.267362+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":318,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.268998+0000","last_change":"2024-09-16T09:29:51.517112+0000","last_active":"2024-09-16T09:31:48.268998+0000","last_peered":"2024-09-16T09:31:48.268998+0000","last_clean":"2024-09-16T09:31:48.268998+0000","last_became_active":"2024-09-16T09:25:01.705158+0000","last_became_peered":"2024-09-16T09:25:01.705158+0000","last_unstale":"2024-09-16T09:31:48.268998+0000","last_undegraded":"2024-09-16T09:31:48.268998+0000","last_fullsized":"2024-09-16T09:31:48.268998+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:51.517054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:51.517054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":324,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.267591+0000","last_change":"2024-09-16T09:31:09.266820+0000","last_active":"2024-09-16T09:31:48.267591+0000","last_peered":"2024-09-16T09:31:48.267591+0000","last_clean":"2024-09-16T09:31:48.267591+0000","last_became_active":"2024-09-16T09:31:09.266326+0000","last_became_peered":"2024-09-16T09:31:09.266326+0000","last_unstale":"2024-09-16T09:31:48.267591+0000","last_undegraded":"2024-09-16T09:31:48.267591+0000","last_fullsized":"2024-09-16T09:31:48.267591+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:15.910824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:15.910824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":153,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.289919+0000","last_change":"2024-09-16T09:31:09.277239+0000","last_active":"2024-09-16T09:31:48.289919+0000","last_peered":"2024-09-16T09:31:48.289919+0000","last_clean":"2024-09-16T09:31:48.289919+0000","last_became_active":"2024-09-16T09:31:09.276042+0000","last_became_peered":"2024-09-16T09:31:09.276042+0000","last_unstale":"2024-09-16T09:31:48.289919+0000","last_undegraded":"2024-09-16T09:31:48.289919+0000","last_fullsized":"2024-09-16T09:31:48.289919+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:48.147759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:48.147759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:14.230973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"66'9","reported_seq":314,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.267879+0000","last_change":"2024-09-16T09:26:31.216495+0000","last_active":"2024-09-16T09:31:48.267879+0000","last_peered":"2024-09-16T09:31:48.267879+0000","last_clean":"2024-09-16T09:31:48.267879+0000","last_became_active":"2024-09-16T09:25:03.705807+0000","last_became_peered":"2024-09-16T09:25:03.705807+0000","last_unstale":"2024-09-16T09:31:48.267879+0000","last_undegraded":"2024-09-16T09:31:48.267879+0000","last_fullsized":"2024-09-16T09:31:48.267879+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:26:31.216298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:26:31.216298+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":301,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.276970+0000","last_change":"2024-09-16T09:26:23.796308+0000","last_active":"2024-09-16T09:31:48.276970+0000","last_peered":"2024-09-16T09:31:48.276970+0000","last_clean":"2024-09-16T09:31:48.276970+0000","last_became_active":"2024-09-16T09:25:01.706762+0000","last_became_peered":"2024-09-16T09:25:01.706762+0000","last_unstale":"2024-09-16T09:31:48.276970+0000","last_undegraded":"2024-09-16T09:31:48.276970+0000","last_fullsized":"2024-09-16T09:31:48.276970+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:23.796231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:23.796231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":138,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.287783+0000","last_change":"2024-09-16T09:28:52.983144+0000","last_active":"2024-09-16T09:31:48.287783+0000","last_peered":"2024-09-16T09:31:48.287783+0000","last_clean":"2024-09-16T09:31:48.287783+0000","last_became_active":"2024-09-16T09:28:52.982865+0000","last_became_peered":"2024-09-16T09:28:52.982865+0000","last_unstale":"2024-09-16T09:31:48.287783+0000","last_undegraded":"2024-09-16T09:31:48.287783+0000","last_fullsized":"2024-09-16T09:31:48.287783+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:44.241942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:44.241942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:09.215097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":298,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.283766+0000","last_change":"2024-09-16T09:28:52.552567+0000","last_active":"2024-09-16T09:31:48.283766+0000","last_peered":"2024-09-16T09:31:48.283766+0000","last_clean":"2024-09-16T09:31:48.283766+0000","last_became_active":"2024-09-16T09:25:07.715301+0000","last_became_peered":"2024-09-16T09:25:07.715301+0000","last_unstale":"2024-09-16T09:31:48.283766+0000","last_undegraded":"2024-09-16T09:31:48.283766+0000","last_fullsized":"2024-09-16T09:31:48.283766+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:52.552495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:28:52.552495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:45.269179+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.14","version":"73'79","reported_seq":96,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.275091+0000","last_change":"2024-09-16T09:31:47.253568+0000","last_active":"2024-09-16T09:31:48.275091+0000","last_peered":"2024-09-16T09:31:48.275091+0000","last_clean":"2024-09-16T09:31:48.275091+0000","last_became_active":"2024-09-16T09:30:10.700866+0000","last_became_peered":"2024-09-16T09:30:10.700866+0000","last_unstale":"2024-09-16T09:31:48.275091+0000","last_undegraded":"2024-09-16T09:31:48.275091+0000","last_fullsized":"2024-09-16T09:31:48.275091+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:53.916505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:53.916505+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:32:13.898748+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109227,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"66'11","reported_seq":185,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.276399+0000","last_change":"2024-09-16T09:28:55.643505+0000","last_active":"2024-09-16T09:31:48.276399+0000","last_peered":"2024-09-16T09:31:48.276399+0000","last_clean":"2024-09-16T09:31:48.276399+0000","last_became_active":"2024-09-16T09:28:52.971132+0000","last_became_peered":"2024-09-16T09:28:52.971132+0000","last_unstale":"2024-09-16T09:31:48.276399+0000","last_undegraded":"2024-09-16T09:31:48.276399+0000","last_fullsized":"2024-09-16T09:31:48.276399+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-16T09:28:55.643448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:55.643448+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":19,"num_read_kb":12,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":309,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.273570+0000","last_change":"2024-09-16T09:27:46.175159+0000","last_active":"2024-09-16T09:31:48.273570+0000","last_peered":"2024-09-16T09:31:48.273570+0000","last_clean":"2024-09-16T09:31:48.273570+0000","last_became_active":"2024-09-16T09:25:01.700733+0000","last_became_peered":"2024-09-16T09:25:01.700733+0000","last_unstale":"2024-09-16T09:31:48.273570+0000","last_undegraded":"2024-09-16T09:31:48.273570+0000","last_fullsized":"2024-09-16T09:31:48.273570+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:46.175093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:27:46.175093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":301,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.275799+0000","last_change":"2024-09-16T09:29:25.017736+0000","last_active":"2024-09-16T09:31:48.275799+0000","last_peered":"2024-09-16T09:31:48.275799+0000","last_clean":"2024-09-16T09:31:48.275799+0000","last_became_active":"2024-09-16T09:25:05.713479+0000","last_became_peered":"2024-09-16T09:25:05.713479+0000","last_unstale":"2024-09-16T09:31:48.275799+0000","last_undegraded":"2024-09-16T09:31:48.275799+0000","last_fullsized":"2024-09-16T09:31:48.275799+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:25.017610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:25.017610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":161,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.288148+0000","last_change":"2024-09-16T09:31:09.275121+0000","last_active":"2024-09-16T09:31:48.288148+0000","last_peered":"2024-09-16T09:31:48.288148+0000","last_clean":"2024-09-16T09:31:48.288148+0000","last_became_active":"2024-09-16T09:31:09.274797+0000","last_became_peered":"2024-09-16T09:31:09.274797+0000","last_unstale":"2024-09-16T09:31:48.288148+0000","last_undegraded":"2024-09-16T09:31:48.288148+0000","last_fullsized":"2024-09-16T09:31:48.288148+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:54.589258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:28:54.589258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:48.251310+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"66'4","reported_seq":42,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.278312+0000","last_change":"2024-09-16T09:31:09.401626+0000","last_active":"2024-09-16T09:31:48.278312+0000","last_peered":"2024-09-16T09:31:48.278312+0000","last_clean":"2024-09-16T09:31:48.278312+0000","last_became_active":"2024-09-16T09:31:09.273682+0000","last_became_peered":"2024-09-16T09:31:09.273682+0000","last_unstale":"2024-09-16T09:31:48.278312+0000","last_undegraded":"2024-09-16T09:31:48.278312+0000","last_fullsized":"2024-09-16T09:31:48.278312+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'4","last_scrub_stamp":"2024-09-16T09:26:27.473740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:26:27.473740+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":332,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.289334+0000","last_change":"2024-09-16T09:31:30.235761+0000","last_active":"2024-09-16T09:31:48.289334+0000","last_peered":"2024-09-16T09:31:48.289334+0000","last_clean":"2024-09-16T09:31:48.289334+0000","last_became_active":"2024-09-16T09:31:09.274719+0000","last_became_peered":"2024-09-16T09:31:09.274719+0000","last_unstale":"2024-09-16T09:31:48.289334+0000","last_undegraded":"2024-09-16T09:31:48.289334+0000","last_fullsized":"2024-09-16T09:31:48.289334+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:31:30.235692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:31:30.235692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:49.736157+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":310,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.273694+0000","last_change":"2024-09-16T09:29:53.244633+0000","last_active":"2024-09-16T09:31:48.273694+0000","last_peered":"2024-09-16T09:31:48.273694+0000","last_clean":"2024-09-16T09:31:48.273694+0000","last_became_active":"2024-09-16T09:25:05.712545+0000","last_became_peered":"2024-09-16T09:25:05.712545+0000","last_unstale":"2024-09-16T09:31:48.273694+0000","last_undegraded":"2024-09-16T09:31:48.273694+0000","last_fullsized":"2024-09-16T09:31:48.273694+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:53.244578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:53.244578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:07.420423+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":297,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.276113+0000","last_change":"2024-09-16T09:29:27.012810+0000","last_active":"2024-09-16T09:31:48.276113+0000","last_peered":"2024-09-16T09:31:48.276113+0000","last_clean":"2024-09-16T09:31:48.276113+0000","last_became_active":"2024-09-16T09:25:07.720117+0000","last_became_peered":"2024-09-16T09:25:07.720117+0000","last_unstale":"2024-09-16T09:31:48.276113+0000","last_undegraded":"2024-09-16T09:31:48.276113+0000","last_fullsized":"2024-09-16T09:31:48.276113+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:27.012756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:27.012756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":298,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.271272+0000","last_change":"2024-09-16T09:29:19.888825+0000","last_active":"2024-09-16T09:31:48.271272+0000","last_peered":"2024-09-16T09:31:48.271272+0000","last_clean":"2024-09-16T09:31:48.271272+0000","last_became_active":"2024-09-16T09:25:07.712594+0000","last_became_peered":"2024-09-16T09:25:07.712594+0000","last_unstale":"2024-09-16T09:31:48.271272+0000","last_undegraded":"2024-09-16T09:31:48.271272+0000","last_fullsized":"2024-09-16T09:31:48.271272+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:19.888759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:19.888759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":160,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.284255+0000","last_change":"2024-09-16T09:31:26.211625+0000","last_active":"2024-09-16T09:31:48.284255+0000","last_peered":"2024-09-16T09:31:48.284255+0000","last_clean":"2024-09-16T09:31:48.284255+0000","last_became_active":"2024-09-16T09:31:09.271662+0000","last_became_peered":"2024-09-16T09:31:09.271662+0000","last_unstale":"2024-09-16T09:31:48.284255+0000","last_undegraded":"2024-09-16T09:31:48.284255+0000","last_fullsized":"2024-09-16T09:31:48.284255+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:31:26.211363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:31:26.211363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:32.873753+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"66'11","reported_seq":326,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.273424+0000","last_change":"2024-09-16T09:27:57.486459+0000","last_active":"2024-09-16T09:31:48.273424+0000","last_peered":"2024-09-16T09:31:48.273424+0000","last_clean":"2024-09-16T09:31:48.273424+0000","last_became_active":"2024-09-16T09:25:03.707769+0000","last_became_peered":"2024-09-16T09:25:03.707769+0000","last_unstale":"2024-09-16T09:31:48.273424+0000","last_undegraded":"2024-09-16T09:31:48.273424+0000","last_fullsized":"2024-09-16T09:31:48.273424+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-16T09:27:57.486381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:57.486381+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:16.375833+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":295,"reported_epoch":199,"state":"active+clean","last_fresh":"2024-09-16T09:31:48.290121+0000","last_change":"2024-09-16T09:27:57.640962+0000","last_active":"2024-09-16T09:31:48.290121+0000","last_peered":"2024-09-16T09:31:48.290121+0000","last_clean":"2024-09-16T09:31:48.290121+0000","last_became_active":"2024-09-16T09:25:05.711952+0000","last_became_peered":"2024-09-16T09:25:05.711952+0000","last_unstale":"2024-09-16T09:31:48.290121+0000","last_undegraded":"2024-09-16T09:31:48.290121+0000","last_fullsized":"2024-09-16T09:31:48.290121+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:57.640756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:57.640756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:21.189003+0000","scrub_duration":44,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":16,"stat_sum":{"num_bytes":6237061144,"num_objects":95171,"num_object_clones":0,"num_object_copies":285513,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":6003,"num_objects_unfound":0,"num_objects_dirty":95171,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":154015,"num_write_kb":7973889,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8706,"num_bytes_recovered":570556416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":19287257088,"data_stored":19287244872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":77202,"ondisk_log_size":77202,"up":48,"acting":45,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":70,"ondisk_log_size":70,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":691,"num_read_kb":448,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":126,"num_bytes_recovered":3026,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":395,"ondisk_log_size":395,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1986,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":405,"num_read_kb":400,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":21,"stat_sum":{"num_bytes":806935,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":21,"num_read_kb":40,"num_write":21,"num_write_kb":296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":697708,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1659,"ondisk_log_size":1659,"up":63,"acting":63,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266775,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2105588,"kb_used_data":1925732,"kb_used_omap":1,"kb_used_meta":179710,"kb_avail":91639564,"statfs":{"total":95995035648,"available":93838913536,"internally_reserved":0,"allocated":1971949568,"data_stored":1971627542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":184023498},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495708,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2086556,"kb_used_data":1920716,"kb_used_omap":1,"kb_used_meta":165822,"kb_avail":91658596,"statfs":{"total":95995035648,"available":93858402304,"internally_reserved":0,"allocated":1966813184,"data_stored":1966515156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":169802182},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724643,"num_pgs":64,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3316648,"kb_used_data":3060504,"kb_used_omap":1,"kb_used_meta":256062,"kb_avail":90428504,"statfs":{"total":95995035648,"available":92598788096,"internally_reserved":0,"allocated":3133956096,"data_stored":3133646273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":262207942},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953576,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2395380,"kb_used_data":2205924,"kb_used_omap":1,"kb_used_meta":189438,"kb_avail":91349772,"statfs":{"total":95995035648,"available":93542166528,"internally_reserved":0,"allocated":2258866176,"data_stored":2258544190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":193984966},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149806,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2191548,"kb_used_data":1907628,"kb_used_omap":1,"kb_used_meta":283838,"kb_avail":91553604,"statfs":{"total":95995035648,"available":93750890496,"internally_reserved":0,"allocated":1953411072,"data_stored":1953081180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":290650567},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411443,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2940448,"kb_used_data":2652432,"kb_used_omap":1,"kb_used_meta":287998,"kb_avail":90804704,"statfs":{"total":95995035648,"available":92984016896,"internally_reserved":0,"allocated":2716090368,"data_stored":2715788200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":294910405},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574970,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2985932,"kb_used_data":2689596,"kb_used_omap":1,"kb_used_meta":296318,"kb_avail":90759220,"statfs":{"total":95995035648,"available":92937441280,"internally_reserved":0,"allocated":2754146304,"data_stored":2753799612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":303430085},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,4],"upper_bound":16},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738496,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2873588,"kb_used_data":2567524,"kb_used_omap":1,"kb_used_meta":306046,"kb_avail":90871564,"statfs":{"total":95995035648,"available":93052481536,"internally_reserved":0,"allocated":2629144576,"data_stored":2628822792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":313391557},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,4,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2604924928,"data_stored":2604924928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2727411712,"data_stored":2727411712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2713456640,"data_stored":2713452568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1942814720,"data_stored":1942814720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":2257846272,"data_stored":2257846272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":3113553920,"data_stored":3113549848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1959591936,"data_stored":1959591936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1967656960,"data_stored":1967652888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T09:31:55.255 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:31:55.292 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:55.293 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:55.293 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:31:55.394 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:31:55.432 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:55.433 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:55.433 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:31:55.534 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:31:55.566 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:55.567 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:55.567 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:31:55.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:55 smithi084 ceph-mon[32629]: pgmap v321: 168 pgs: 1 active+recovery_wait+degraded, 3 active+recovering+undersized+remapped, 164 active+clean; 5.8 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1.75k op/s; 1/286113 objects degraded (0.000%); 6003/286113 objects misplaced (2.098%); 11 MiB/s, 179 objects/s recovering 2024-09-16T09:31:55.667 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:31:55.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:55 smithi070 ceph-mon[30388]: pgmap v321: 168 pgs: 1 active+recovery_wait+degraded, 3 active+recovering+undersized+remapped, 164 active+clean; 5.8 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1.75k op/s; 1/286113 objects degraded (0.000%); 6003/286113 objects misplaced (2.098%); 11 MiB/s, 179 objects/s recovering 2024-09-16T09:31:55.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:55 smithi070 ceph-mon[40691]: pgmap v321: 168 pgs: 1 active+recovery_wait+degraded, 3 active+recovering+undersized+remapped, 164 active+clean; 5.8 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1.75k op/s; 1/286113 objects degraded (0.000%); 6003/286113 objects misplaced (2.098%); 11 MiB/s, 179 objects/s recovering 2024-09-16T09:31:55.699 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:55.700 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:55.700 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:31:55.778 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:31:55.801 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:31:55.832 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:55.833 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:55.833 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:31:55.906 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.6 dump_historic_ops 2024-09-16T09:31:55.933 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:31:55.969 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:55.969 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:55.970 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:31:56.070 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:31:56.103 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:56.103 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:56.104 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:31:56.205 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:31:56.243 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:56.244 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:56.244 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:31:56.299 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:31:56.345 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:31:56.375 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:56.375 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:56.376 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:31:56.460 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:31:56.460 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:31:56.476 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:31:56.493 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:31:56.493 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:31:56.509 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:56.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:56.510 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:31:56.525 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:31:56.526 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:31:56.557 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:31:56.557 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:31:56.586 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:31:56.586 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:31:56.611 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:31:56.620 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:56 smithi070 ceph-mon[30388]: 5.2 scrub starts 2024-09-16T09:31:56.620 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:56 smithi070 ceph-mon[30388]: 5.2 scrub ok 2024-09-16T09:31:56.620 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:56 smithi070 ceph-mon[40691]: 5.2 scrub starts 2024-09-16T09:31:56.620 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:56 smithi070 ceph-mon[40691]: 5.2 scrub ok 2024-09-16T09:31:56.621 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:31:56.621 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:31:56.648 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:56.649 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:56.649 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:31:56.652 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:31:56.652 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:31:56.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:56 smithi084 ceph-mon[32629]: 5.2 scrub starts 2024-09-16T09:31:56.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:56 smithi084 ceph-mon[32629]: 5.2 scrub ok 2024-09-16T09:31:56.684 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:31:56.684 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:31:56.750 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:31:56.783 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:56.783 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:56.784 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:31:56.885 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:31:56.919 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:56.920 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:56.920 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:31:57.021 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:31:57.055 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:57.055 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:57.055 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:31:57.157 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:31:57.191 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:57.192 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:57.192 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:31:57.293 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:31:57.335 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:57.336 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:57.336 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:31:57.371 INFO:teuthology.orchestra.run.smithi084.stdout:ERROR: (22) Invalid argument 2024-09-16T09:31:57.371 INFO:teuthology.orchestra.run.smithi084.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:31:57.437 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:31:57.481 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:57.482 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:57.482 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:31:57.583 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:31:57.619 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:57.620 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:57.621 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:31:57.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:57 smithi084 ceph-mon[32629]: pgmap v322: 168 pgs: 1 active+recovery_wait+degraded, 3 active+recovering+undersized+remapped, 164 active+clean; 5.8 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.30k op/s; 1/286113 objects degraded (0.000%); 6003/286113 objects misplaced (2.098%); 8.9 MiB/s, 142 objects/s recovering 2024-09-16T09:31:57.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:57 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/562591358' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:31:57.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:57 smithi084 ceph-mon[32629]: 3.1e scrub starts 2024-09-16T09:31:57.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:57 smithi084 ceph-mon[32629]: 3.1e scrub ok 2024-09-16T09:31:57.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:57 smithi070 ceph-mon[30388]: pgmap v322: 168 pgs: 1 active+recovery_wait+degraded, 3 active+recovering+undersized+remapped, 164 active+clean; 5.8 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.30k op/s; 1/286113 objects degraded (0.000%); 6003/286113 objects misplaced (2.098%); 8.9 MiB/s, 142 objects/s recovering 2024-09-16T09:31:57.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:57 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/562591358' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:31:57.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:57 smithi070 ceph-mon[30388]: 3.1e scrub starts 2024-09-16T09:31:57.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:57 smithi070 ceph-mon[30388]: 3.1e scrub ok 2024-09-16T09:31:57.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:57 smithi070 ceph-mon[40691]: pgmap v322: 168 pgs: 1 active+recovery_wait+degraded, 3 active+recovering+undersized+remapped, 164 active+clean; 5.8 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.30k op/s; 1/286113 objects degraded (0.000%); 6003/286113 objects misplaced (2.098%); 8.9 MiB/s, 142 objects/s recovering 2024-09-16T09:31:57.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:57 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/562591358' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:31:57.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:57 smithi070 ceph-mon[40691]: 3.1e scrub starts 2024-09-16T09:31:57.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:57 smithi070 ceph-mon[40691]: 3.1e scrub ok 2024-09-16T09:31:57.721 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:31:57.757 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:57.757 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:57.758 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:31:57.775 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-16T09:31:57.858 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:31:57.896 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:57.897 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:57.898 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:31:57.998 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:31:58.032 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:58.033 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:58.033 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:31:58.134 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:31:58.145 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.7 dump_ops_in_flight 2024-09-16T09:31:58.184 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:58.185 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:58.185 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:31:58.214 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:31:58.238 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-16T09:31:58.286 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:31:58.320 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:58.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:58.322 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:31:58.423 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:31:58.467 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:58.468 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:58.468 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:31:58.545 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:31:58.569 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:31:58.610 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:58.611 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:58.611 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:31:58.712 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:31:58.744 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:58.745 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:58.745 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:31:58.846 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:31:58.884 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:58.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:58.886 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:31:58.986 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:31:59.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:59.020 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:59.020 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:31:59.121 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:31:59.161 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:59.162 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:59.162 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:31:59.263 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:31:59.301 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:59.301 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:59.302 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:31:59.377 INFO:teuthology.orchestra.run.smithi084.stdout:ERROR: (22) Invalid argument 2024-09-16T09:31:59.377 INFO:teuthology.orchestra.run.smithi084.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:31:59.403 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:31:59.409 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T09:31:59.437 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:59.438 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:59.439 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:31:59.449 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:31:59.540 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:31:59.582 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:59.583 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:59.583 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:31:59.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:59 smithi084 ceph-mon[32629]: 3.6 scrub starts 2024-09-16T09:31:59.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:59 smithi084 ceph-mon[32629]: 3.6 scrub ok 2024-09-16T09:31:59.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:59 smithi084 ceph-mon[32629]: pgmap v323: 168 pgs: 1 active+recovery_wait+degraded, 3 active+recovering+undersized+remapped, 164 active+clean; 5.8 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 675 op/s; 1/284022 objects degraded (0.000%); 6003/284022 objects misplaced (2.114%); 4.0 MiB/s, 63 objects/s recovering 2024-09-16T09:31:59.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:31:59 smithi084 ceph-mon[32629]: Health check update: Degraded data redundancy: 1/284022 objects degraded (0.000%), 1 pg degraded (PG_DEGRADED) 2024-09-16T09:31:59.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:59 smithi070 ceph-mon[30388]: 3.6 scrub starts 2024-09-16T09:31:59.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:59 smithi070 ceph-mon[30388]: 3.6 scrub ok 2024-09-16T09:31:59.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:59 smithi070 ceph-mon[30388]: pgmap v323: 168 pgs: 1 active+recovery_wait+degraded, 3 active+recovering+undersized+remapped, 164 active+clean; 5.8 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 675 op/s; 1/284022 objects degraded (0.000%); 6003/284022 objects misplaced (2.114%); 4.0 MiB/s, 63 objects/s recovering 2024-09-16T09:31:59.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:31:59 smithi070 ceph-mon[30388]: Health check update: Degraded data redundancy: 1/284022 objects degraded (0.000%), 1 pg degraded (PG_DEGRADED) 2024-09-16T09:31:59.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:59 smithi070 ceph-mon[40691]: 3.6 scrub starts 2024-09-16T09:31:59.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:59 smithi070 ceph-mon[40691]: 3.6 scrub ok 2024-09-16T09:31:59.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:59 smithi070 ceph-mon[40691]: pgmap v323: 168 pgs: 1 active+recovery_wait+degraded, 3 active+recovering+undersized+remapped, 164 active+clean; 5.8 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 675 op/s; 1/284022 objects degraded (0.000%); 6003/284022 objects misplaced (2.114%); 4.0 MiB/s, 63 objects/s recovering 2024-09-16T09:31:59.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:31:59 smithi070 ceph-mon[40691]: Health check update: Degraded data redundancy: 1/284022 objects degraded (0.000%), 1 pg degraded (PG_DEGRADED) 2024-09-16T09:31:59.684 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:31:59.728 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:31:59.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:59.729 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:31:59.830 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:31:59.865 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:31:59.865 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:31:59.866 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:31:59.967 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:32:00.002 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:00.002 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:00.003 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:32:00.103 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:32:00.111 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.7 dump_blocked_ops 2024-09-16T09:32:00.134 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:00.134 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:00.134 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:32:00.236 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:32:00.279 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:00.280 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:00.280 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:32:00.373 INFO:teuthology.orchestra.run.smithi070.stderr:nodeep-scrub is unset 2024-09-16T09:32:00.381 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:32:00.423 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:00.424 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:00.424 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:32:00.521 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:32:00.525 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:32:00.559 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:00.560 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:00.560 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:32:00.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:00 smithi084 ceph-mon[32629]: 6.18 scrub starts 2024-09-16T09:32:00.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:00 smithi084 ceph-mon[32629]: 6.18 scrub ok 2024-09-16T09:32:00.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:00 smithi084 ceph-mon[32629]: 3.12 scrub starts 2024-09-16T09:32:00.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:00 smithi084 ceph-mon[32629]: 3.12 scrub ok 2024-09-16T09:32:00.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:00 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/336677614' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:32:00.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:00 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:32:00.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:00 smithi084 ceph-mon[32629]: 6.16 scrub starts 2024-09-16T09:32:00.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:00 smithi084 ceph-mon[32629]: 6.16 scrub ok 2024-09-16T09:32:00.661 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:32:00.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:00 smithi070 ceph-mon[30388]: 6.18 scrub starts 2024-09-16T09:32:00.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:00 smithi070 ceph-mon[30388]: 6.18 scrub ok 2024-09-16T09:32:00.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:00 smithi070 ceph-mon[30388]: 3.12 scrub starts 2024-09-16T09:32:00.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:00 smithi070 ceph-mon[30388]: 3.12 scrub ok 2024-09-16T09:32:00.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:00 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/336677614' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:32:00.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:00 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:32:00.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:00 smithi070 ceph-mon[30388]: 6.16 scrub starts 2024-09-16T09:32:00.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:00 smithi070 ceph-mon[30388]: 6.16 scrub ok 2024-09-16T09:32:00.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:00 smithi070 ceph-mon[40691]: 6.18 scrub starts 2024-09-16T09:32:00.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:00 smithi070 ceph-mon[40691]: 6.18 scrub ok 2024-09-16T09:32:00.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:00 smithi070 ceph-mon[40691]: 3.12 scrub starts 2024-09-16T09:32:00.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:00 smithi070 ceph-mon[40691]: 3.12 scrub ok 2024-09-16T09:32:00.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:00 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/336677614' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:32:00.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:00 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:32:00.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:00 smithi070 ceph-mon[40691]: 6.16 scrub starts 2024-09-16T09:32:00.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:00 smithi070 ceph-mon[40691]: 6.16 scrub ok 2024-09-16T09:32:00.690 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:00.691 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:00.691 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:32:00.792 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:32:00.809 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:32:00.812 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2024-09-16T09:32:00.838 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:00.839 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:00.839 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:32:00.940 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:32:00.976 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:00.977 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:00.977 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:32:01.078 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:32:01.110 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:01.110 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:01.111 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:32:01.192 INFO:teuthology.orchestra.run.smithi084.stdout:ERROR: (22) Invalid argument 2024-09-16T09:32:01.192 INFO:teuthology.orchestra.run.smithi084.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:32:01.211 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:32:01.248 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:01.249 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:01.249 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:32:01.350 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:32:01.392 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:01.393 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:01.393 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:32:01.494 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:32:01.549 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:01.550 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:01.550 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:32:01.624 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:01 smithi070 ceph-mon[30388]: pgmap v324: 168 pgs: 1 active+recovery_wait+degraded, 3 active+recovering+undersized+remapped, 164 active+clean; 5.5 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.14k op/s; 1/268683 objects degraded (0.000%); 5350/268683 objects misplaced (1.991%); 7.8 MiB/s, 124 objects/s recovering 2024-09-16T09:32:01.624 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:01 smithi070 ceph-mon[30388]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-16T09:32:01.624 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:01 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-16T09:32:01.624 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:01 smithi070 ceph-mon[30388]: osdmap e201: 8 total, 8 up, 8 in 2024-09-16T09:32:01.625 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:01 smithi070 ceph-mon[30388]: 6.c scrub starts 2024-09-16T09:32:01.625 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:01 smithi070 ceph-mon[30388]: 6.c scrub ok 2024-09-16T09:32:01.625 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:01 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:32:01.625 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:01 smithi070 ceph-mon[30388]: 4.16 scrub starts 2024-09-16T09:32:01.625 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:01 smithi070 ceph-mon[30388]: 4.16 scrub ok 2024-09-16T09:32:01.626 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:01 smithi070 ceph-mon[40691]: pgmap v324: 168 pgs: 1 active+recovery_wait+degraded, 3 active+recovering+undersized+remapped, 164 active+clean; 5.5 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.14k op/s; 1/268683 objects degraded (0.000%); 5350/268683 objects misplaced (1.991%); 7.8 MiB/s, 124 objects/s recovering 2024-09-16T09:32:01.626 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:01 smithi070 ceph-mon[40691]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-16T09:32:01.626 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:01 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-16T09:32:01.626 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:01 smithi070 ceph-mon[40691]: osdmap e201: 8 total, 8 up, 8 in 2024-09-16T09:32:01.626 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:01 smithi070 ceph-mon[40691]: 6.c scrub starts 2024-09-16T09:32:01.626 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:01 smithi070 ceph-mon[40691]: 6.c scrub ok 2024-09-16T09:32:01.626 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:01 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:32:01.627 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:01 smithi070 ceph-mon[40691]: 4.16 scrub starts 2024-09-16T09:32:01.627 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:01 smithi070 ceph-mon[40691]: 4.16 scrub ok 2024-09-16T09:32:01.651 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:32:01.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:01 smithi084 ceph-mon[32629]: pgmap v324: 168 pgs: 1 active+recovery_wait+degraded, 3 active+recovering+undersized+remapped, 164 active+clean; 5.5 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.14k op/s; 1/268683 objects degraded (0.000%); 5350/268683 objects misplaced (1.991%); 7.8 MiB/s, 124 objects/s recovering 2024-09-16T09:32:01.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:01 smithi084 ceph-mon[32629]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-16T09:32:01.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:01 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-16T09:32:01.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:01 smithi084 ceph-mon[32629]: osdmap e201: 8 total, 8 up, 8 in 2024-09-16T09:32:01.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:01 smithi084 ceph-mon[32629]: 6.c scrub starts 2024-09-16T09:32:01.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:01 smithi084 ceph-mon[32629]: 6.c scrub ok 2024-09-16T09:32:01.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:01 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:32:01.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:01 smithi084 ceph-mon[32629]: 4.16 scrub starts 2024-09-16T09:32:01.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:01 smithi084 ceph-mon[32629]: 4.16 scrub ok 2024-09-16T09:32:01.689 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:01.689 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:01.690 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:32:01.790 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:32:01.823 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:01.824 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:01.824 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:32:01.925 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:32:01.963 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:01.963 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:01.964 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:32:02.065 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:32:02.094 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:32:02.121 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":325,"stamp":"2024-09-16T09:32:00.364286+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":5857110058,"num_objects":89561,"num_object_clones":0,"num_object_copies":268683,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":5350,"num_objects_unfound":0,"num_objects_dirty":89561,"num_whiteouts":0,"num_read":1147,"num_read_kb":918,"num_write":160282,"num_write_kb":7974230,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9496,"num_bytes_recovered":614054144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79452,"ondisk_log_size":79452,"up":504,"acting":501,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":508,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":19873896,"kb_used_data":17905704,"kb_used_omap":12,"kb_used_meta":1968051,"kb_avail":730087320,"statfs":{"total":767960285184,"available":747609415680,"internally_reserved":0,"allocated":18335440896,"data_stored":18332856289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":2015284786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,4,2,5,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-748683264,"num_objects":-11424,"num_object_clones":0,"num_object_copies":-34272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-1249,"num_objects_unfound":0,"num_objects_dirty":-11424,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":11424,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1249,"num_bytes_recovered":81854464,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.740165"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":321,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.328189+0000","last_change":"2024-09-16T09:30:03.848439+0000","last_active":"2024-09-16T09:31:53.328189+0000","last_peered":"2024-09-16T09:31:53.328189+0000","last_clean":"2024-09-16T09:31:53.328189+0000","last_became_active":"2024-09-16T09:25:01.709108+0000","last_became_peered":"2024-09-16T09:25:01.709108+0000","last_unstale":"2024-09-16T09:31:53.328189+0000","last_undegraded":"2024-09-16T09:31:53.328189+0000","last_fullsized":"2024-09-16T09:31:53.328189+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:30:03.848354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:30:03.848354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"66'9","reported_seq":324,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.308964+0000","last_change":"2024-09-16T09:28:52.689373+0000","last_active":"2024-09-16T09:31:53.308964+0000","last_peered":"2024-09-16T09:31:53.308964+0000","last_clean":"2024-09-16T09:31:53.308964+0000","last_became_active":"2024-09-16T09:25:03.714635+0000","last_became_peered":"2024-09-16T09:25:03.714635+0000","last_unstale":"2024-09-16T09:31:53.308964+0000","last_undegraded":"2024-09-16T09:31:53.308964+0000","last_fullsized":"2024-09-16T09:31:53.308964+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:28:52.689304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:52.689304+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":312,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.320336+0000","last_change":"2024-09-16T09:29:17.910367+0000","last_active":"2024-09-16T09:31:53.320336+0000","last_peered":"2024-09-16T09:31:53.320336+0000","last_clean":"2024-09-16T09:31:53.320336+0000","last_became_active":"2024-09-16T09:25:05.715645+0000","last_became_peered":"2024-09-16T09:25:05.715645+0000","last_unstale":"2024-09-16T09:31:53.320336+0000","last_undegraded":"2024-09-16T09:31:53.320336+0000","last_fullsized":"2024-09-16T09:31:53.320336+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:17.910305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:17.910305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":292,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.307704+0000","last_change":"2024-09-16T09:27:11.565306+0000","last_active":"2024-09-16T09:31:53.307704+0000","last_peered":"2024-09-16T09:31:53.307704+0000","last_clean":"2024-09-16T09:31:53.307704+0000","last_became_active":"2024-09-16T09:25:07.715442+0000","last_became_peered":"2024-09-16T09:25:07.715442+0000","last_unstale":"2024-09-16T09:31:53.307704+0000","last_undegraded":"2024-09-16T09:31:53.307704+0000","last_fullsized":"2024-09-16T09:31:53.307704+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:11.565220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:11.565220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":38,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.318323+0000","last_change":"2024-09-16T09:31:09.269700+0000","last_active":"2024-09-16T09:31:53.318323+0000","last_peered":"2024-09-16T09:31:53.318323+0000","last_clean":"2024-09-16T09:31:53.318323+0000","last_became_active":"2024-09-16T09:31:09.269342+0000","last_became_peered":"2024-09-16T09:31:09.269342+0000","last_unstale":"2024-09-16T09:31:53.318323+0000","last_undegraded":"2024-09-16T09:31:53.318323+0000","last_fullsized":"2024-09-16T09:31:53.318323+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:00.068570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:28:00.068570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":178,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:56.507595+0000","last_change":"2024-09-16T09:31:56.507595+0000","last_active":"2024-09-16T09:31:56.507595+0000","last_peered":"2024-09-16T09:31:56.507595+0000","last_clean":"2024-09-16T09:31:56.507595+0000","last_became_active":"2024-09-16T09:31:09.275136+0000","last_became_peered":"2024-09-16T09:31:09.275136+0000","last_unstale":"2024-09-16T09:31:56.507595+0000","last_undegraded":"2024-09-16T09:31:56.507595+0000","last_fullsized":"2024-09-16T09:31:56.507595+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:31:56.507528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:31:56.507528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:08.491873+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.19","version":"66'15","reported_seq":239,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.309918+0000","last_change":"2024-09-16T09:31:09.302903+0000","last_active":"2024-09-16T09:31:53.309918+0000","last_peered":"2024-09-16T09:31:53.309918+0000","last_clean":"2024-09-16T09:31:53.309918+0000","last_became_active":"2024-09-16T09:31:09.268750+0000","last_became_peered":"2024-09-16T09:31:09.268750+0000","last_unstale":"2024-09-16T09:31:53.309918+0000","last_undegraded":"2024-09-16T09:31:53.309918+0000","last_fullsized":"2024-09-16T09:31:53.309918+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-16T09:29:57.968162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:29:57.968162+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":312,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.308842+0000","last_change":"2024-09-16T09:29:15.628676+0000","last_active":"2024-09-16T09:31:53.308842+0000","last_peered":"2024-09-16T09:31:53.308842+0000","last_clean":"2024-09-16T09:31:53.308842+0000","last_became_active":"2024-09-16T09:25:05.717236+0000","last_became_peered":"2024-09-16T09:25:05.717236+0000","last_unstale":"2024-09-16T09:31:53.308842+0000","last_undegraded":"2024-09-16T09:31:53.308842+0000","last_fullsized":"2024-09-16T09:31:53.308842+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:15.628467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:15.628467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":151,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.317574+0000","last_change":"2024-09-16T09:28:34.839909+0000","last_active":"2024-09-16T09:31:53.317574+0000","last_peered":"2024-09-16T09:31:53.317574+0000","last_clean":"2024-09-16T09:31:53.317574+0000","last_became_active":"2024-09-16T09:28:34.839435+0000","last_became_peered":"2024-09-16T09:28:34.839435+0000","last_unstale":"2024-09-16T09:31:53.317574+0000","last_undegraded":"2024-09-16T09:31:53.317574+0000","last_fullsized":"2024-09-16T09:31:53.317574+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:01.038400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:28:01.038400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:09.066525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"66'9","reported_seq":324,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.307983+0000","last_change":"2024-09-16T09:29:28.622647+0000","last_active":"2024-09-16T09:31:53.307983+0000","last_peered":"2024-09-16T09:31:53.307983+0000","last_clean":"2024-09-16T09:31:53.307983+0000","last_became_active":"2024-09-16T09:25:03.709147+0000","last_became_peered":"2024-09-16T09:25:03.709147+0000","last_unstale":"2024-09-16T09:31:53.307983+0000","last_undegraded":"2024-09-16T09:31:53.307983+0000","last_fullsized":"2024-09-16T09:31:53.307983+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:29:28.622558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:29:28.622558+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":295,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.320484+0000","last_change":"2024-09-16T09:27:59.080369+0000","last_active":"2024-09-16T09:31:53.320484+0000","last_peered":"2024-09-16T09:31:53.320484+0000","last_clean":"2024-09-16T09:31:53.320484+0000","last_became_active":"2024-09-16T09:25:05.718134+0000","last_became_peered":"2024-09-16T09:25:05.718134+0000","last_unstale":"2024-09-16T09:31:53.320484+0000","last_undegraded":"2024-09-16T09:31:53.320484+0000","last_fullsized":"2024-09-16T09:31:53.320484+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:59.080307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:59.080307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"66'1","reported_seq":301,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.319155+0000","last_change":"2024-09-16T09:29:15.787990+0000","last_active":"2024-09-16T09:31:53.319155+0000","last_peered":"2024-09-16T09:31:53.319155+0000","last_clean":"2024-09-16T09:31:53.319155+0000","last_became_active":"2024-09-16T09:25:07.716558+0000","last_became_peered":"2024-09-16T09:25:07.716558+0000","last_unstale":"2024-09-16T09:31:53.319155+0000","last_undegraded":"2024-09-16T09:31:53.319155+0000","last_fullsized":"2024-09-16T09:31:53.319155+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-16T09:29:15.787922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:15.787922+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":303,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.320085+0000","last_change":"2024-09-16T09:27:56.191264+0000","last_active":"2024-09-16T09:31:53.320085+0000","last_peered":"2024-09-16T09:31:53.320085+0000","last_clean":"2024-09-16T09:31:53.320085+0000","last_became_active":"2024-09-16T09:25:01.696939+0000","last_became_peered":"2024-09-16T09:25:01.696939+0000","last_unstale":"2024-09-16T09:31:53.320085+0000","last_undegraded":"2024-09-16T09:31:53.320085+0000","last_fullsized":"2024-09-16T09:31:53.320085+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:56.191220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:27:56.191220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"66'5","reported_seq":310,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.310892+0000","last_change":"2024-09-16T09:27:05.694733+0000","last_active":"2024-09-16T09:31:53.310892+0000","last_peered":"2024-09-16T09:31:53.310892+0000","last_clean":"2024-09-16T09:31:53.310892+0000","last_became_active":"2024-09-16T09:25:03.716136+0000","last_became_peered":"2024-09-16T09:25:03.716136+0000","last_unstale":"2024-09-16T09:31:53.310892+0000","last_undegraded":"2024-09-16T09:31:53.310892+0000","last_fullsized":"2024-09-16T09:31:53.310892+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-16T09:27:05.694650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:05.694650+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":303,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.325117+0000","last_change":"2024-09-16T09:28:52.746046+0000","last_active":"2024-09-16T09:31:53.325117+0000","last_peered":"2024-09-16T09:31:53.325117+0000","last_clean":"2024-09-16T09:31:53.325117+0000","last_became_active":"2024-09-16T09:25:05.718281+0000","last_became_peered":"2024-09-16T09:25:05.718281+0000","last_unstale":"2024-09-16T09:31:53.325117+0000","last_undegraded":"2024-09-16T09:31:53.325117+0000","last_fullsized":"2024-09-16T09:31:53.325117+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:52.745999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:28:52.745999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":321,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.318921+0000","last_change":"2024-09-16T09:31:09.276016+0000","last_active":"2024-09-16T09:31:53.318921+0000","last_peered":"2024-09-16T09:31:53.318921+0000","last_clean":"2024-09-16T09:31:53.318921+0000","last_became_active":"2024-09-16T09:31:09.275656+0000","last_became_peered":"2024-09-16T09:31:09.275656+0000","last_unstale":"2024-09-16T09:31:53.318921+0000","last_undegraded":"2024-09-16T09:31:53.318921+0000","last_fullsized":"2024-09-16T09:31:53.318921+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:50.468134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:50.468134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":309,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.306731+0000","last_change":"2024-09-16T09:29:56.605555+0000","last_active":"2024-09-16T09:31:53.306731+0000","last_peered":"2024-09-16T09:31:53.306731+0000","last_clean":"2024-09-16T09:31:53.306731+0000","last_became_active":"2024-09-16T09:25:07.720793+0000","last_became_peered":"2024-09-16T09:25:07.720793+0000","last_unstale":"2024-09-16T09:31:53.306731+0000","last_undegraded":"2024-09-16T09:31:53.306731+0000","last_fullsized":"2024-09-16T09:31:53.306731+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:56.605488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:56.605488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":60,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.317790+0000","last_change":"2024-09-16T09:30:39.202178+0000","last_active":"2024-09-16T09:31:53.317790+0000","last_peered":"2024-09-16T09:31:53.317790+0000","last_clean":"2024-09-16T09:31:53.317790+0000","last_became_active":"2024-09-16T09:30:39.201824+0000","last_became_peered":"2024-09-16T09:30:39.201824+0000","last_unstale":"2024-09-16T09:31:53.317790+0000","last_undegraded":"2024-09-16T09:31:53.317790+0000","last_fullsized":"2024-09-16T09:31:53.317790+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:50.912979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:50.912979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:20.431823+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"66'15","reported_seq":387,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.310604+0000","last_change":"2024-09-16T09:31:09.302018+0000","last_active":"2024-09-16T09:31:53.310604+0000","last_peered":"2024-09-16T09:31:53.310604+0000","last_clean":"2024-09-16T09:31:53.310604+0000","last_became_active":"2024-09-16T09:31:09.263534+0000","last_became_peered":"2024-09-16T09:31:09.263534+0000","last_unstale":"2024-09-16T09:31:53.310604+0000","last_undegraded":"2024-09-16T09:31:53.310604+0000","last_fullsized":"2024-09-16T09:31:53.310604+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-16T09:30:21.932604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:30:21.932604+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":304,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.317845+0000","last_change":"2024-09-16T09:29:55.875536+0000","last_active":"2024-09-16T09:31:53.317845+0000","last_peered":"2024-09-16T09:31:53.317845+0000","last_clean":"2024-09-16T09:31:53.317845+0000","last_became_active":"2024-09-16T09:25:05.713929+0000","last_became_peered":"2024-09-16T09:25:05.713929+0000","last_unstale":"2024-09-16T09:31:53.317845+0000","last_undegraded":"2024-09-16T09:31:53.317845+0000","last_fullsized":"2024-09-16T09:31:53.317845+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:55.875480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:55.875480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":157,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.319771+0000","last_change":"2024-09-16T09:30:24.412733+0000","last_active":"2024-09-16T09:31:53.319771+0000","last_peered":"2024-09-16T09:31:53.319771+0000","last_clean":"2024-09-16T09:31:53.319771+0000","last_became_active":"2024-09-16T09:28:52.974255+0000","last_became_peered":"2024-09-16T09:28:52.974255+0000","last_unstale":"2024-09-16T09:31:53.319771+0000","last_undegraded":"2024-09-16T09:31:53.319771+0000","last_fullsized":"2024-09-16T09:31:53.319771+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:30:24.412571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:30:24.412571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:49.424390+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":320,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.308122+0000","last_change":"2024-09-16T09:30:03.628882+0000","last_active":"2024-09-16T09:31:53.308122+0000","last_peered":"2024-09-16T09:31:53.308122+0000","last_clean":"2024-09-16T09:31:53.308122+0000","last_became_active":"2024-09-16T09:25:01.706491+0000","last_became_peered":"2024-09-16T09:25:01.706491+0000","last_unstale":"2024-09-16T09:31:53.308122+0000","last_undegraded":"2024-09-16T09:31:53.308122+0000","last_fullsized":"2024-09-16T09:31:53.308122+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:30:03.628812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:30:03.628812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"66'12","reported_seq":221,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.318011+0000","last_change":"2024-09-16T09:31:09.286673+0000","last_active":"2024-09-16T09:31:53.318011+0000","last_peered":"2024-09-16T09:31:53.318011+0000","last_clean":"2024-09-16T09:31:53.318011+0000","last_became_active":"2024-09-16T09:31:09.268588+0000","last_became_peered":"2024-09-16T09:31:09.268588+0000","last_unstale":"2024-09-16T09:31:53.318011+0000","last_undegraded":"2024-09-16T09:31:53.318011+0000","last_fullsized":"2024-09-16T09:31:53.318011+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-16T09:27:01.700139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:01.700139+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":24,"num_read_kb":15,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":296,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.308238+0000","last_change":"2024-09-16T09:27:00.816500+0000","last_active":"2024-09-16T09:31:53.308238+0000","last_peered":"2024-09-16T09:31:53.308238+0000","last_clean":"2024-09-16T09:31:53.308238+0000","last_became_active":"2024-09-16T09:25:05.710093+0000","last_became_peered":"2024-09-16T09:25:05.710093+0000","last_unstale":"2024-09-16T09:31:53.308238+0000","last_undegraded":"2024-09-16T09:31:53.308238+0000","last_fullsized":"2024-09-16T09:31:53.308238+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:00.816424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:00.816424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":300,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.326767+0000","last_change":"2024-09-16T09:29:29.918433+0000","last_active":"2024-09-16T09:31:53.326767+0000","last_peered":"2024-09-16T09:31:53.326767+0000","last_clean":"2024-09-16T09:31:53.326767+0000","last_became_active":"2024-09-16T09:25:07.719380+0000","last_became_peered":"2024-09-16T09:25:07.719380+0000","last_unstale":"2024-09-16T09:31:53.326767+0000","last_undegraded":"2024-09-16T09:31:53.326767+0000","last_fullsized":"2024-09-16T09:31:53.326767+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-16T09:29:29.918368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:29.918368+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":321,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.320469+0000","last_change":"2024-09-16T09:30:02.843932+0000","last_active":"2024-09-16T09:31:53.320469+0000","last_peered":"2024-09-16T09:31:53.320469+0000","last_clean":"2024-09-16T09:31:53.320469+0000","last_became_active":"2024-09-16T09:25:01.699545+0000","last_became_peered":"2024-09-16T09:25:01.699545+0000","last_unstale":"2024-09-16T09:31:53.320469+0000","last_undegraded":"2024-09-16T09:31:53.320469+0000","last_fullsized":"2024-09-16T09:31:53.320469+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:30:02.843856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:30:02.843856+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"66'10","reported_seq":362,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.328011+0000","last_change":"2024-09-16T09:31:09.328513+0000","last_active":"2024-09-16T09:31:53.328011+0000","last_peered":"2024-09-16T09:31:53.328011+0000","last_clean":"2024-09-16T09:31:53.328011+0000","last_became_active":"2024-09-16T09:31:09.275050+0000","last_became_peered":"2024-09-16T09:31:09.275050+0000","last_unstale":"2024-09-16T09:31:53.328011+0000","last_undegraded":"2024-09-16T09:31:53.328011+0000","last_fullsized":"2024-09-16T09:31:53.328011+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-16T09:27:43.556357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:43.556357+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:15.194576+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"66'8","reported_seq":416,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.319947+0000","last_change":"2024-09-16T09:30:31.392771+0000","last_active":"2024-09-16T09:31:53.319947+0000","last_peered":"2024-09-16T09:31:53.319947+0000","last_clean":"2024-09-16T09:31:53.319947+0000","last_became_active":"2024-09-16T09:25:05.712516+0000","last_became_peered":"2024-09-16T09:25:05.712516+0000","last_unstale":"2024-09-16T09:31:53.319947+0000","last_undegraded":"2024-09-16T09:31:53.319947+0000","last_fullsized":"2024-09-16T09:31:53.319947+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:30:31.392399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:30:31.392399+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:44.469073+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"200'10051","reported_seq":10157,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:57.779314+0000","last_change":"2024-09-16T09:29:49.449011+0000","last_active":"2024-09-16T09:31:57.779314+0000","last_peered":"2024-09-16T09:31:57.779314+0000","last_clean":"2024-09-16T09:31:57.779314+0000","last_became_active":"2024-09-16T09:29:49.448473+0000","last_became_peered":"2024-09-16T09:29:49.448473+0000","last_unstale":"2024-09-16T09:31:57.779314+0000","last_undegraded":"2024-09-16T09:31:57.779314+0000","last_fullsized":"2024-09-16T09:31:57.779314+0000","mapping_epoch":152,"log_start":"180'5500","ondisk_log_start":"180'5500","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":4551,"log_dups_size":0,"ondisk_log_size":4551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":364838912,"num_objects":5567,"num_object_clones":0,"num_object_copies":16701,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5567,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10051,"num_write_kb":499776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.a","version":"73'79","reported_seq":243,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.320701+0000","last_change":"2024-09-16T09:29:18.498887+0000","last_active":"2024-09-16T09:31:53.320701+0000","last_peered":"2024-09-16T09:31:53.320701+0000","last_clean":"2024-09-16T09:31:53.320701+0000","last_became_active":"2024-09-16T09:26:35.127352+0000","last_became_peered":"2024-09-16T09:26:35.127352+0000","last_unstale":"2024-09-16T09:31:53.320701+0000","last_undegraded":"2024-09-16T09:31:53.320701+0000","last_fullsized":"2024-09-16T09:31:53.320701+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:18.498766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:18.498766+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"66'15","reported_seq":252,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.313407+0000","last_change":"2024-09-16T09:28:28.065781+0000","last_active":"2024-09-16T09:31:53.313407+0000","last_peered":"2024-09-16T09:31:53.313407+0000","last_clean":"2024-09-16T09:31:53.313407+0000","last_became_active":"2024-09-16T09:26:33.295672+0000","last_became_peered":"2024-09-16T09:26:33.295672+0000","last_unstale":"2024-09-16T09:31:53.313407+0000","last_undegraded":"2024-09-16T09:31:53.313407+0000","last_fullsized":"2024-09-16T09:31:53.313407+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-16T09:28:28.065699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:28.065699+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":38,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.327394+0000","last_change":"2024-09-16T09:31:09.276858+0000","last_active":"2024-09-16T09:31:53.327394+0000","last_peered":"2024-09-16T09:31:53.327394+0000","last_clean":"2024-09-16T09:31:53.327394+0000","last_became_active":"2024-09-16T09:31:09.274534+0000","last_became_peered":"2024-09-16T09:31:09.274534+0000","last_unstale":"2024-09-16T09:31:53.327394+0000","last_undegraded":"2024-09-16T09:31:53.327394+0000","last_fullsized":"2024-09-16T09:31:53.327394+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:22.091537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:28:22.091537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"66'8","reported_seq":381,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.306844+0000","last_change":"2024-09-16T09:25:05.714128+0000","last_active":"2024-09-16T09:31:53.306844+0000","last_peered":"2024-09-16T09:31:53.306844+0000","last_clean":"2024-09-16T09:31:53.306844+0000","last_became_active":"2024-09-16T09:25:05.713813+0000","last_became_peered":"2024-09-16T09:25:05.713813+0000","last_unstale":"2024-09-16T09:31:53.306844+0000","last_undegraded":"2024-09-16T09:31:53.306844+0000","last_fullsized":"2024-09-16T09:31:53.306844+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":300,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.318916+0000","last_change":"2024-09-16T09:29:24.537802+0000","last_active":"2024-09-16T09:31:53.318916+0000","last_peered":"2024-09-16T09:31:53.318916+0000","last_clean":"2024-09-16T09:31:53.318916+0000","last_became_active":"2024-09-16T09:25:07.717344+0000","last_became_peered":"2024-09-16T09:25:07.717344+0000","last_unstale":"2024-09-16T09:31:53.318916+0000","last_undegraded":"2024-09-16T09:31:53.318916+0000","last_fullsized":"2024-09-16T09:31:53.318916+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:24.537731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:24.537731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.c","version":"200'9999","reported_seq":10106,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:58.161296+0000","last_change":"2024-09-16T09:29:50.008199+0000","last_active":"2024-09-16T09:31:58.161296+0000","last_peered":"2024-09-16T09:31:58.161296+0000","last_clean":"2024-09-16T09:31:58.161296+0000","last_became_active":"2024-09-16T09:29:50.007935+0000","last_became_peered":"2024-09-16T09:29:50.007935+0000","last_unstale":"2024-09-16T09:31:58.161296+0000","last_undegraded":"2024-09-16T09:31:58.161296+0000","last_fullsized":"2024-09-16T09:31:58.161296+0000","mapping_epoch":152,"log_start":"178'4900","ondisk_log_start":"178'4900","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":5099,"log_dups_size":0,"ondisk_log_size":5099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":363659264,"num_objects":5549,"num_object_clones":0,"num_object_copies":16647,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5549,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9999,"num_write_kb":497536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"73'79","reported_seq":272,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.310204+0000","last_change":"2024-09-16T09:31:47.249715+0000","last_active":"2024-09-16T09:31:53.310204+0000","last_peered":"2024-09-16T09:31:53.310204+0000","last_clean":"2024-09-16T09:31:53.310204+0000","last_became_active":"2024-09-16T09:30:08.763683+0000","last_became_peered":"2024-09-16T09:30:08.763683+0000","last_unstale":"2024-09-16T09:31:53.310204+0000","last_undegraded":"2024-09-16T09:31:53.310204+0000","last_fullsized":"2024-09-16T09:31:53.310204+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:58.568841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:58.568841+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:32:11.825261+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"66'18","reported_seq":250,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.325086+0000","last_change":"2024-09-16T09:31:09.407700+0000","last_active":"2024-09-16T09:31:53.325086+0000","last_peered":"2024-09-16T09:31:53.325086+0000","last_clean":"2024-09-16T09:31:53.325086+0000","last_became_active":"2024-09-16T09:31:09.270434+0000","last_became_peered":"2024-09-16T09:31:09.270434+0000","last_unstale":"2024-09-16T09:31:53.325086+0000","last_undegraded":"2024-09-16T09:31:53.325086+0000","last_fullsized":"2024-09-16T09:31:53.325086+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'18","last_scrub_stamp":"2024-09-16T09:28:54.699778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:54.699778+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":33,"num_read_kb":21,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":148,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.326551+0000","last_change":"2024-09-16T09:28:58.705169+0000","last_active":"2024-09-16T09:31:53.326551+0000","last_peered":"2024-09-16T09:31:53.326551+0000","last_clean":"2024-09-16T09:31:53.326551+0000","last_became_active":"2024-09-16T09:28:52.983543+0000","last_became_peered":"2024-09-16T09:28:52.983543+0000","last_unstale":"2024-09-16T09:31:53.326551+0000","last_undegraded":"2024-09-16T09:31:53.326551+0000","last_fullsized":"2024-09-16T09:31:53.326551+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:58.705097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:28:58.705097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":327,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.306357+0000","last_change":"2024-09-16T09:31:09.269234+0000","last_active":"2024-09-16T09:31:53.306357+0000","last_peered":"2024-09-16T09:31:53.306357+0000","last_clean":"2024-09-16T09:31:53.306357+0000","last_became_active":"2024-09-16T09:31:09.268864+0000","last_became_peered":"2024-09-16T09:31:09.268864+0000","last_unstale":"2024-09-16T09:31:53.306357+0000","last_undegraded":"2024-09-16T09:31:53.306357+0000","last_fullsized":"2024-09-16T09:31:53.306357+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:50.482240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:50.482240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":307,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.307888+0000","last_change":"2024-09-16T09:31:09.269842+0000","last_active":"2024-09-16T09:31:53.307888+0000","last_peered":"2024-09-16T09:31:53.307888+0000","last_clean":"2024-09-16T09:31:53.307888+0000","last_became_active":"2024-09-16T09:31:09.269433+0000","last_became_peered":"2024-09-16T09:31:09.269433+0000","last_unstale":"2024-09-16T09:31:53.307888+0000","last_undegraded":"2024-09-16T09:31:53.307888+0000","last_fullsized":"2024-09-16T09:31:53.307888+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:13.548608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:13.548608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.d","version":"200'9983","reported_seq":10091,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:57.868083+0000","last_change":"2024-09-16T09:29:50.010860+0000","last_active":"2024-09-16T09:31:57.868083+0000","last_peered":"2024-09-16T09:31:57.868083+0000","last_clean":"2024-09-16T09:31:57.868083+0000","last_became_active":"2024-09-16T09:29:50.010421+0000","last_became_peered":"2024-09-16T09:29:50.010421+0000","last_unstale":"2024-09-16T09:31:57.868083+0000","last_undegraded":"2024-09-16T09:31:57.868083+0000","last_fullsized":"2024-09-16T09:31:57.868083+0000","mapping_epoch":152,"log_start":"179'5200","ondisk_log_start":"179'5200","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":4783,"log_dups_size":0,"ondisk_log_size":4783,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":365821976,"num_objects":5583,"num_object_clones":0,"num_object_copies":16749,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5583,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9983,"num_write_kb":498049,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,2],"acting":[5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.4","version":"73'79","reported_seq":38,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.319299+0000","last_change":"2024-09-16T09:31:09.275499+0000","last_active":"2024-09-16T09:31:53.319299+0000","last_peered":"2024-09-16T09:31:53.319299+0000","last_clean":"2024-09-16T09:31:53.319299+0000","last_became_active":"2024-09-16T09:31:09.275139+0000","last_became_peered":"2024-09-16T09:31:09.275139+0000","last_unstale":"2024-09-16T09:31:53.319299+0000","last_undegraded":"2024-09-16T09:31:53.319299+0000","last_fullsized":"2024-09-16T09:31:53.319299+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:30:22.438378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:30:22.438378+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"66'14","reported_seq":330,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.306437+0000","last_change":"2024-09-16T09:28:32.595064+0000","last_active":"2024-09-16T09:31:53.306437+0000","last_peered":"2024-09-16T09:31:53.306437+0000","last_clean":"2024-09-16T09:31:53.306437+0000","last_became_active":"2024-09-16T09:25:03.708830+0000","last_became_peered":"2024-09-16T09:25:03.708830+0000","last_unstale":"2024-09-16T09:31:53.306437+0000","last_undegraded":"2024-09-16T09:31:53.306437+0000","last_fullsized":"2024-09-16T09:31:53.306437+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'14","last_scrub_stamp":"2024-09-16T09:28:32.594988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:32.594988+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":330,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:57.373606+0000","last_change":"2024-09-16T09:31:57.373606+0000","last_active":"2024-09-16T09:31:57.373606+0000","last_peered":"2024-09-16T09:31:57.373606+0000","last_clean":"2024-09-16T09:31:57.373606+0000","last_became_active":"2024-09-16T09:25:01.702992+0000","last_became_peered":"2024-09-16T09:25:01.702992+0000","last_unstale":"2024-09-16T09:31:57.373606+0000","last_undegraded":"2024-09-16T09:31:57.373606+0000","last_fullsized":"2024-09-16T09:31:57.373606+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:31:57.373544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:31:57.373544+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:24.193683+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"187'11","reported_seq":233,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.319254+0000","last_change":"2024-09-16T09:31:09.397758+0000","last_active":"2024-09-16T09:31:53.319254+0000","last_peered":"2024-09-16T09:31:53.319254+0000","last_clean":"2024-09-16T09:31:53.319254+0000","last_became_active":"2024-09-16T09:31:09.269173+0000","last_became_peered":"2024-09-16T09:31:09.269173+0000","last_unstale":"2024-09-16T09:31:53.319254+0000","last_undegraded":"2024-09-16T09:31:53.319254+0000","last_fullsized":"2024-09-16T09:31:53.319254+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'9","last_scrub_stamp":"2024-09-16T09:29:21.898533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:21.898533+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":292,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.325346+0000","last_change":"2024-09-16T09:26:40.699888+0000","last_active":"2024-09-16T09:31:53.325346+0000","last_peered":"2024-09-16T09:31:53.325346+0000","last_clean":"2024-09-16T09:31:53.325346+0000","last_became_active":"2024-09-16T09:25:07.722017+0000","last_became_peered":"2024-09-16T09:25:07.722017+0000","last_unstale":"2024-09-16T09:31:53.325346+0000","last_undegraded":"2024-09-16T09:31:53.325346+0000","last_fullsized":"2024-09-16T09:31:53.325346+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:40.699813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:40.699813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.e","version":"200'10066","reported_seq":10173,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:58.146773+0000","last_change":"2024-09-16T09:29:49.448081+0000","last_active":"2024-09-16T09:31:58.146773+0000","last_peered":"2024-09-16T09:31:58.146773+0000","last_clean":"2024-09-16T09:31:58.146773+0000","last_became_active":"2024-09-16T09:29:49.447767+0000","last_became_peered":"2024-09-16T09:29:49.447767+0000","last_unstale":"2024-09-16T09:31:58.146773+0000","last_undegraded":"2024-09-16T09:31:58.146773+0000","last_fullsized":"2024-09-16T09:31:58.146773+0000","mapping_epoch":152,"log_start":"177'4900","ondisk_log_start":"177'4900","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":5166,"log_dups_size":0,"ondisk_log_size":5166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":363069440,"num_objects":5540,"num_object_clones":0,"num_object_copies":16620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5540,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10066,"num_write_kb":499392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"73'79","reported_seq":361,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.319599+0000","last_change":"2024-09-16T09:31:09.275413+0000","last_active":"2024-09-16T09:31:53.319599+0000","last_peered":"2024-09-16T09:31:53.319599+0000","last_clean":"2024-09-16T09:31:53.319599+0000","last_became_active":"2024-09-16T09:31:09.274707+0000","last_became_peered":"2024-09-16T09:31:09.274707+0000","last_unstale":"2024-09-16T09:31:53.319599+0000","last_undegraded":"2024-09-16T09:31:53.319599+0000","last_fullsized":"2024-09-16T09:31:53.319599+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:30:23.455388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:30:23.455388+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:41.421930+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"66'10","reported_seq":332,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.326416+0000","last_change":"2024-09-16T09:30:04.848306+0000","last_active":"2024-09-16T09:31:53.326416+0000","last_peered":"2024-09-16T09:31:53.326416+0000","last_clean":"2024-09-16T09:31:53.326416+0000","last_became_active":"2024-09-16T09:25:03.712985+0000","last_became_peered":"2024-09-16T09:25:03.712985+0000","last_unstale":"2024-09-16T09:31:53.326416+0000","last_undegraded":"2024-09-16T09:31:53.326416+0000","last_fullsized":"2024-09-16T09:31:53.326416+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-16T09:30:04.848234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:30:04.848234+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":318,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.319372+0000","last_change":"2024-09-16T09:31:09.272049+0000","last_active":"2024-09-16T09:31:53.319372+0000","last_peered":"2024-09-16T09:31:53.319372+0000","last_clean":"2024-09-16T09:31:53.319372+0000","last_became_active":"2024-09-16T09:31:09.270619+0000","last_became_peered":"2024-09-16T09:31:09.270619+0000","last_unstale":"2024-09-16T09:31:53.319372+0000","last_undegraded":"2024-09-16T09:31:53.319372+0000","last_fullsized":"2024-09-16T09:31:53.319372+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:25.086322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:25.086322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"66'8","reported_seq":397,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.327379+0000","last_change":"2024-09-16T09:28:57.628615+0000","last_active":"2024-09-16T09:31:53.327379+0000","last_peered":"2024-09-16T09:31:53.327379+0000","last_clean":"2024-09-16T09:31:53.327379+0000","last_became_active":"2024-09-16T09:25:05.714507+0000","last_became_peered":"2024-09-16T09:25:05.714507+0000","last_unstale":"2024-09-16T09:31:53.327379+0000","last_undegraded":"2024-09-16T09:31:53.327379+0000","last_fullsized":"2024-09-16T09:31:53.327379+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:28:57.628543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:28:57.628543+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:51.253847+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":315,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.319309+0000","last_change":"2024-09-16T09:31:09.283816+0000","last_active":"2024-09-16T09:31:53.319309+0000","last_peered":"2024-09-16T09:31:53.319309+0000","last_clean":"2024-09-16T09:31:53.319309+0000","last_became_active":"2024-09-16T09:31:09.283192+0000","last_became_peered":"2024-09-16T09:31:09.283192+0000","last_unstale":"2024-09-16T09:31:53.319309+0000","last_undegraded":"2024-09-16T09:31:53.319309+0000","last_fullsized":"2024-09-16T09:31:53.319309+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:22.563395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:28:22.563395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:22.152175+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.f","version":"200'10047","reported_seq":10154,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:57.770730+0000","last_change":"2024-09-16T09:29:49.838498+0000","last_active":"2024-09-16T09:31:57.770730+0000","last_peered":"2024-09-16T09:31:57.770730+0000","last_clean":"2024-09-16T09:31:57.770730+0000","last_became_active":"2024-09-16T09:29:49.838146+0000","last_became_peered":"2024-09-16T09:29:49.838146+0000","last_unstale":"2024-09-16T09:31:57.770730+0000","last_undegraded":"2024-09-16T09:31:57.770730+0000","last_fullsized":"2024-09-16T09:31:57.770730+0000","mapping_epoch":152,"log_start":"174'4100","ondisk_log_start":"174'4100","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":5947,"log_dups_size":0,"ondisk_log_size":5947,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":373489664,"num_objects":5699,"num_object_clones":0,"num_object_copies":17097,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5699,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10047,"num_write_kb":503872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"73'79","reported_seq":342,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.319920+0000","last_change":"2024-09-16T09:31:09.274975+0000","last_active":"2024-09-16T09:31:53.319920+0000","last_peered":"2024-09-16T09:31:53.319920+0000","last_clean":"2024-09-16T09:31:53.319920+0000","last_became_active":"2024-09-16T09:31:09.274565+0000","last_became_peered":"2024-09-16T09:31:09.274565+0000","last_unstale":"2024-09-16T09:31:53.319920+0000","last_undegraded":"2024-09-16T09:31:53.319920+0000","last_fullsized":"2024-09-16T09:31:53.319920+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:30:22.415051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:30:22.415051+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:52.385618+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"66'19","reported_seq":345,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.327139+0000","last_change":"2024-09-16T09:31:35.249952+0000","last_active":"2024-09-16T09:31:53.327139+0000","last_peered":"2024-09-16T09:31:53.327139+0000","last_clean":"2024-09-16T09:31:53.327139+0000","last_became_active":"2024-09-16T09:25:03.714797+0000","last_became_peered":"2024-09-16T09:25:03.714797+0000","last_unstale":"2024-09-16T09:31:53.327139+0000","last_undegraded":"2024-09-16T09:31:53.327139+0000","last_fullsized":"2024-09-16T09:31:53.327139+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-16T09:31:35.249864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:31:35.249864+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:40.632025+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":321,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.317488+0000","last_change":"2024-09-16T09:29:16.888357+0000","last_active":"2024-09-16T09:31:53.317488+0000","last_peered":"2024-09-16T09:31:53.317488+0000","last_clean":"2024-09-16T09:31:53.317488+0000","last_became_active":"2024-09-16T09:25:01.706806+0000","last_became_peered":"2024-09-16T09:25:01.706806+0000","last_unstale":"2024-09-16T09:31:53.317488+0000","last_undegraded":"2024-09-16T09:31:53.317488+0000","last_fullsized":"2024-09-16T09:31:53.317488+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:16.888236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:16.888236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":321,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:55.456669+0000","last_change":"2024-09-16T09:31:55.456669+0000","last_active":"2024-09-16T09:31:55.456669+0000","last_peered":"2024-09-16T09:31:55.456669+0000","last_clean":"2024-09-16T09:31:55.456669+0000","last_became_active":"2024-09-16T09:25:05.708663+0000","last_became_peered":"2024-09-16T09:25:05.708663+0000","last_unstale":"2024-09-16T09:31:55.456669+0000","last_undegraded":"2024-09-16T09:31:55.456669+0000","last_fullsized":"2024-09-16T09:31:55.456669+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:31:55.456592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:31:55.456592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:56.006473+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":293,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.320257+0000","last_change":"2024-09-16T09:26:37.282798+0000","last_active":"2024-09-16T09:31:53.320257+0000","last_peered":"2024-09-16T09:31:53.320257+0000","last_clean":"2024-09-16T09:31:53.320257+0000","last_became_active":"2024-09-16T09:25:07.719267+0000","last_became_peered":"2024-09-16T09:25:07.719267+0000","last_unstale":"2024-09-16T09:31:53.320257+0000","last_undegraded":"2024-09-16T09:31:53.320257+0000","last_fullsized":"2024-09-16T09:31:53.320257+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:37.282735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:37.282735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.8","version":"200'10106","reported_seq":14698,"reported_epoch":200,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-16T09:31:57.870833+0000","last_change":"2024-09-16T09:31:09.438251+0000","last_active":"2024-09-16T09:31:57.870833+0000","last_peered":"2024-09-16T09:31:57.870833+0000","last_clean":"2024-09-16T09:30:20.474651+0000","last_became_active":"2024-09-16T09:31:09.409873+0000","last_became_peered":"2024-09-16T09:31:09.409873+0000","last_unstale":"2024-09-16T09:31:57.870833+0000","last_undegraded":"2024-09-16T09:31:57.870833+0000","last_fullsized":"2024-09-16T09:31:09.256524+0000","mapping_epoch":186,"log_start":"180'5400","ondisk_log_start":"180'5400","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":4706,"log_dups_size":0,"ondisk_log_size":4706,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":368050176,"num_objects":5616,"num_object_clones":0,"num_object_copies":16848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2219,"num_objects_unfound":0,"num_objects_dirty":5616,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10106,"num_write_kb":503104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2237,"num_bytes_recovered":146604032,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,7],"avail_no_missing":["5","7"],"object_location_counts":[{"shards":"0,5,7","objects":2219},{"shards":"5,7","objects":3397}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1","version":"73'79","reported_seq":264,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.318701+0000","last_change":"2024-09-16T09:31:47.248649+0000","last_active":"2024-09-16T09:31:53.318701+0000","last_peered":"2024-09-16T09:31:53.318701+0000","last_clean":"2024-09-16T09:31:53.318701+0000","last_became_active":"2024-09-16T09:30:08.766350+0000","last_became_peered":"2024-09-16T09:30:08.766350+0000","last_unstale":"2024-09-16T09:31:53.318701+0000","last_undegraded":"2024-09-16T09:31:53.318701+0000","last_fullsized":"2024-09-16T09:31:53.318701+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:15.460179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:15.460179+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:32:12.387091+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"187'30","reported_seq":499,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.317405+0000","last_change":"2024-09-16T09:31:09.323281+0000","last_active":"2024-09-16T09:31:53.317405+0000","last_peered":"2024-09-16T09:31:53.317405+0000","last_clean":"2024-09-16T09:31:53.317405+0000","last_became_active":"2024-09-16T09:31:09.278839+0000","last_became_peered":"2024-09-16T09:31:09.278839+0000","last_unstale":"2024-09-16T09:31:53.317405+0000","last_undegraded":"2024-09-16T09:31:53.317405+0000","last_fullsized":"2024-09-16T09:31:53.317405+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'28","last_scrub_stamp":"2024-09-16T09:29:18.937623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:29:18.937623+0000","objects_scrubbed":10,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":313,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.312113+0000","last_change":"2024-09-16T09:29:26.637070+0000","last_active":"2024-09-16T09:31:53.312113+0000","last_peered":"2024-09-16T09:31:53.312113+0000","last_clean":"2024-09-16T09:31:53.312113+0000","last_became_active":"2024-09-16T09:25:01.703076+0000","last_became_peered":"2024-09-16T09:25:01.703076+0000","last_unstale":"2024-09-16T09:31:53.312113+0000","last_undegraded":"2024-09-16T09:31:53.312113+0000","last_fullsized":"2024-09-16T09:31:53.312113+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:26.637027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:26.637027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":322,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.321388+0000","last_change":"2024-09-16T09:29:29.486284+0000","last_active":"2024-09-16T09:31:53.321388+0000","last_peered":"2024-09-16T09:31:53.321388+0000","last_clean":"2024-09-16T09:31:53.321388+0000","last_became_active":"2024-09-16T09:25:00.696237+0000","last_became_peered":"2024-09-16T09:25:00.696237+0000","last_unstale":"2024-09-16T09:31:53.321388+0000","last_undegraded":"2024-09-16T09:31:53.321388+0000","last_fullsized":"2024-09-16T09:31:53.321388+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-16T09:29:29.486188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:29:29.486188+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.0025137430000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":289,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.327875+0000","last_change":"2024-09-16T09:25:05.718319+0000","last_active":"2024-09-16T09:31:53.327875+0000","last_peered":"2024-09-16T09:31:53.327875+0000","last_clean":"2024-09-16T09:31:53.327875+0000","last_became_active":"2024-09-16T09:25:05.717591+0000","last_became_peered":"2024-09-16T09:25:05.717591+0000","last_unstale":"2024-09-16T09:31:53.327875+0000","last_undegraded":"2024-09-16T09:31:53.327875+0000","last_fullsized":"2024-09-16T09:31:53.327875+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:10.167420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":38,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.329599+0000","last_change":"2024-09-16T09:31:09.272967+0000","last_active":"2024-09-16T09:31:53.329599+0000","last_peered":"2024-09-16T09:31:53.329599+0000","last_clean":"2024-09-16T09:31:53.329599+0000","last_became_active":"2024-09-16T09:31:09.272609+0000","last_became_peered":"2024-09-16T09:31:09.272609+0000","last_unstale":"2024-09-16T09:31:53.329599+0000","last_undegraded":"2024-09-16T09:31:53.329599+0000","last_fullsized":"2024-09-16T09:31:53.329599+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:45.201503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:45.201503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"200'9948","reported_seq":19482,"reported_epoch":200,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-16T09:31:57.796796+0000","last_change":"2024-09-16T09:31:09.423422+0000","last_active":"2024-09-16T09:31:57.796796+0000","last_peered":"2024-09-16T09:31:57.796796+0000","last_clean":"2024-09-16T09:30:20.492754+0000","last_became_active":"2024-09-16T09:31:09.398447+0000","last_became_peered":"2024-09-16T09:31:09.398447+0000","last_unstale":"2024-09-16T09:31:57.796796+0000","last_undegraded":"2024-09-16T09:31:57.796796+0000","last_fullsized":"2024-09-16T09:31:09.253245+0000","mapping_epoch":186,"log_start":"181'5700","ondisk_log_start":"181'5700","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":4248,"log_dups_size":0,"ondisk_log_size":4248,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":366084096,"num_objects":5586,"num_object_clones":0,"num_object_copies":16758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":842,"num_objects_unfound":0,"num_objects_dirty":5586,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9948,"num_write_kb":497088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4709,"num_bytes_recovered":308609024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,1],"avail_no_missing":["4","1"],"object_location_counts":[{"shards":"1,2,4","objects":842},{"shards":"1,4","objects":4744}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.2","version":"73'79","reported_seq":389,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.324954+0000","last_change":"2024-09-16T09:31:47.263716+0000","last_active":"2024-09-16T09:31:53.324954+0000","last_peered":"2024-09-16T09:31:53.324954+0000","last_clean":"2024-09-16T09:31:53.324954+0000","last_became_active":"2024-09-16T09:30:08.765943+0000","last_became_peered":"2024-09-16T09:30:08.765943+0000","last_unstale":"2024-09-16T09:31:53.324954+0000","last_undegraded":"2024-09-16T09:31:53.324954+0000","last_fullsized":"2024-09-16T09:31:53.324954+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:26:35.745837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:26:35.745837+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:32:11.475590+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"66'13","reported_seq":336,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.327295+0000","last_change":"2024-09-16T09:27:44.156057+0000","last_active":"2024-09-16T09:31:53.327295+0000","last_peered":"2024-09-16T09:31:53.327295+0000","last_clean":"2024-09-16T09:31:53.327295+0000","last_became_active":"2024-09-16T09:25:03.713241+0000","last_became_peered":"2024-09-16T09:25:03.713241+0000","last_unstale":"2024-09-16T09:31:53.327295+0000","last_undegraded":"2024-09-16T09:31:53.327295+0000","last_fullsized":"2024-09-16T09:31:53.327295+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'13","last_scrub_stamp":"2024-09-16T09:27:44.155969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:44.155969+0000","objects_scrubbed":5,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":29,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":313,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.319643+0000","last_change":"2024-09-16T09:29:50.827103+0000","last_active":"2024-09-16T09:31:53.319643+0000","last_peered":"2024-09-16T09:31:53.319643+0000","last_clean":"2024-09-16T09:31:53.319643+0000","last_became_active":"2024-09-16T09:25:01.706828+0000","last_became_peered":"2024-09-16T09:25:01.706828+0000","last_unstale":"2024-09-16T09:31:53.319643+0000","last_undegraded":"2024-09-16T09:31:53.319643+0000","last_fullsized":"2024-09-16T09:31:53.319643+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:50.827057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:50.827057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":344,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.310011+0000","last_change":"2024-09-16T09:31:09.281163+0000","last_active":"2024-09-16T09:31:53.310011+0000","last_peered":"2024-09-16T09:31:53.310011+0000","last_clean":"2024-09-16T09:31:53.310011+0000","last_became_active":"2024-09-16T09:31:09.266338+0000","last_became_peered":"2024-09-16T09:31:09.266338+0000","last_unstale":"2024-09-16T09:31:53.310011+0000","last_undegraded":"2024-09-16T09:31:53.310011+0000","last_fullsized":"2024-09-16T09:31:53.310011+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:29:16.104209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:29:16.104209+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0.00055022299999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":313,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.310547+0000","last_change":"2024-09-16T09:29:00.169141+0000","last_active":"2024-09-16T09:31:53.310547+0000","last_peered":"2024-09-16T09:31:53.310547+0000","last_clean":"2024-09-16T09:31:53.310547+0000","last_became_active":"2024-09-16T09:25:05.711539+0000","last_became_peered":"2024-09-16T09:25:05.711539+0000","last_unstale":"2024-09-16T09:31:53.310547+0000","last_undegraded":"2024-09-16T09:31:53.310547+0000","last_fullsized":"2024-09-16T09:31:53.310547+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:00.169077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:00.169077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":300,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.328132+0000","last_change":"2024-09-16T09:29:22.968971+0000","last_active":"2024-09-16T09:31:53.328132+0000","last_peered":"2024-09-16T09:31:53.328132+0000","last_clean":"2024-09-16T09:31:53.328132+0000","last_became_active":"2024-09-16T09:25:07.723282+0000","last_became_peered":"2024-09-16T09:25:07.723282+0000","last_unstale":"2024-09-16T09:31:53.328132+0000","last_undegraded":"2024-09-16T09:31:53.328132+0000","last_fullsized":"2024-09-16T09:31:53.328132+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:22.968908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:22.968908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"200'9979","reported_seq":10086,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:57.861636+0000","last_change":"2024-09-16T09:29:49.448898+0000","last_active":"2024-09-16T09:31:57.861636+0000","last_peered":"2024-09-16T09:31:57.861636+0000","last_clean":"2024-09-16T09:31:57.861636+0000","last_became_active":"2024-09-16T09:29:49.448628+0000","last_became_peered":"2024-09-16T09:29:49.448628+0000","last_unstale":"2024-09-16T09:31:57.861636+0000","last_undegraded":"2024-09-16T09:31:57.861636+0000","last_fullsized":"2024-09-16T09:31:57.861636+0000","mapping_epoch":152,"log_start":"180'5500","ondisk_log_start":"180'5500","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":4479,"log_dups_size":0,"ondisk_log_size":4479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":370606080,"num_objects":5655,"num_object_clones":0,"num_object_copies":16965,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5655,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9979,"num_write_kb":500288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"73'79","reported_seq":274,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.320236+0000","last_change":"2024-09-16T09:31:40.212968+0000","last_active":"2024-09-16T09:31:53.320236+0000","last_peered":"2024-09-16T09:31:53.320236+0000","last_clean":"2024-09-16T09:31:53.320236+0000","last_became_active":"2024-09-16T09:30:08.877110+0000","last_became_peered":"2024-09-16T09:30:08.877110+0000","last_unstale":"2024-09-16T09:31:53.320236+0000","last_undegraded":"2024-09-16T09:31:53.320236+0000","last_fullsized":"2024-09-16T09:31:53.320236+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:31:40.212889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:31:40.212889+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:45.708324+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"66'12","reported_seq":319,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.328094+0000","last_change":"2024-09-16T09:29:02.713335+0000","last_active":"2024-09-16T09:31:53.328094+0000","last_peered":"2024-09-16T09:31:53.328094+0000","last_clean":"2024-09-16T09:31:53.328094+0000","last_became_active":"2024-09-16T09:25:03.715315+0000","last_became_peered":"2024-09-16T09:25:03.715315+0000","last_unstale":"2024-09-16T09:31:53.328094+0000","last_undegraded":"2024-09-16T09:31:53.328094+0000","last_fullsized":"2024-09-16T09:31:53.328094+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-16T09:29:02.713259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:29:02.713259+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:43.220661+0000","scrub_duration":26,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"59'1","reported_seq":336,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.332582+0000","last_change":"2024-09-16T09:31:09.286309+0000","last_active":"2024-09-16T09:31:53.332582+0000","last_peered":"2024-09-16T09:31:53.332582+0000","last_clean":"2024-09-16T09:31:53.332582+0000","last_became_active":"2024-09-16T09:31:09.269627+0000","last_became_peered":"2024-09-16T09:31:09.269627+0000","last_unstale":"2024-09-16T09:31:53.332582+0000","last_undegraded":"2024-09-16T09:31:53.332582+0000","last_fullsized":"2024-09-16T09:31:53.332582+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:27:00.666673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:27:00.666673+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:13.191274+0000","scrub_duration":28,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1986,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"67'5","reported_seq":733,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:56.108294+0000","last_change":"2024-09-16T09:29:19.994791+0000","last_active":"2024-09-16T09:31:56.108294+0000","last_peered":"2024-09-16T09:31:56.108294+0000","last_clean":"2024-09-16T09:31:56.108294+0000","last_became_active":"2024-09-16T09:25:00.696546+0000","last_became_peered":"2024-09-16T09:25:00.696546+0000","last_unstale":"2024-09-16T09:31:56.108294+0000","last_undegraded":"2024-09-16T09:31:56.108294+0000","last_fullsized":"2024-09-16T09:31:56.108294+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'5","last_scrub_stamp":"2024-09-16T09:29:19.994716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:29:19.994716+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0.0059770420000000001,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":410,"num_read_kb":405,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":296,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.319219+0000","last_change":"2024-09-16T09:27:07.337170+0000","last_active":"2024-09-16T09:31:53.319219+0000","last_peered":"2024-09-16T09:31:53.319219+0000","last_clean":"2024-09-16T09:31:53.319219+0000","last_became_active":"2024-09-16T09:25:05.714704+0000","last_became_peered":"2024-09-16T09:25:05.714704+0000","last_unstale":"2024-09-16T09:31:53.319219+0000","last_undegraded":"2024-09-16T09:31:53.319219+0000","last_fullsized":"2024-09-16T09:31:53.319219+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:07.337124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:07.337124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":307,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.317067+0000","last_change":"2024-09-16T09:31:09.271453+0000","last_active":"2024-09-16T09:31:53.317067+0000","last_peered":"2024-09-16T09:31:53.317067+0000","last_clean":"2024-09-16T09:31:53.317067+0000","last_became_active":"2024-09-16T09:31:09.271111+0000","last_became_peered":"2024-09-16T09:31:09.271111+0000","last_unstale":"2024-09-16T09:31:53.317067+0000","last_undegraded":"2024-09-16T09:31:53.317067+0000","last_fullsized":"2024-09-16T09:31:53.317067+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:24.231167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:24.231167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"200'9944","reported_seq":10116,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:57.576536+0000","last_change":"2024-09-16T09:30:56.023710+0000","last_active":"2024-09-16T09:31:57.576536+0000","last_peered":"2024-09-16T09:31:57.576536+0000","last_clean":"2024-09-16T09:31:57.576536+0000","last_became_active":"2024-09-16T09:29:49.450643+0000","last_became_peered":"2024-09-16T09:29:49.450643+0000","last_unstale":"2024-09-16T09:31:57.576536+0000","last_undegraded":"2024-09-16T09:31:57.576536+0000","last_fullsized":"2024-09-16T09:31:57.576536+0000","mapping_epoch":152,"log_start":"177'4700","ondisk_log_start":"177'4700","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"181'5780","last_scrub_stamp":"2024-09-16T09:30:56.023622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:30:56.023622+0000","objects_scrubbed":5716,"log_size":5244,"log_dups_size":0,"ondisk_log_size":5244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:07.379468+0000","scrub_duration":1526,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":368312320,"num_objects":5620,"num_object_clones":0,"num_object_copies":16860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5620,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9944,"num_write_kb":498048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"66'16","reported_seq":415,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.320591+0000","last_change":"2024-09-16T09:30:50.441875+0000","last_active":"2024-09-16T09:31:53.320591+0000","last_peered":"2024-09-16T09:31:53.320591+0000","last_clean":"2024-09-16T09:31:53.320591+0000","last_became_active":"2024-09-16T09:25:03.714118+0000","last_became_peered":"2024-09-16T09:25:03.714118+0000","last_unstale":"2024-09-16T09:31:53.320591+0000","last_undegraded":"2024-09-16T09:31:53.320591+0000","last_fullsized":"2024-09-16T09:31:53.320591+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'16","last_scrub_stamp":"2024-09-16T09:30:50.441795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:30:50.441795+0000","objects_scrubbed":8,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:00.328016+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":154,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.321299+0000","last_change":"2024-09-16T09:31:09.270603+0000","last_active":"2024-09-16T09:31:53.321299+0000","last_peered":"2024-09-16T09:31:53.321299+0000","last_clean":"2024-09-16T09:31:53.321299+0000","last_became_active":"2024-09-16T09:31:09.270196+0000","last_became_peered":"2024-09-16T09:31:09.270196+0000","last_unstale":"2024-09-16T09:31:53.321299+0000","last_undegraded":"2024-09-16T09:31:53.321299+0000","last_fullsized":"2024-09-16T09:31:53.321299+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:49.195331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:27:49.195331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"73'79","reported_seq":389,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.325367+0000","last_change":"2024-09-16T09:31:47.263760+0000","last_active":"2024-09-16T09:31:53.325367+0000","last_peered":"2024-09-16T09:31:53.325367+0000","last_clean":"2024-09-16T09:31:53.325367+0000","last_became_active":"2024-09-16T09:30:08.870394+0000","last_became_peered":"2024-09-16T09:30:08.870394+0000","last_unstale":"2024-09-16T09:31:53.325367+0000","last_undegraded":"2024-09-16T09:31:53.325367+0000","last_fullsized":"2024-09-16T09:31:53.325367+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:02.723794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:02.723794+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:32:12.473499+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":296,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.327002+0000","last_change":"2024-09-16T09:26:39.699133+0000","last_active":"2024-09-16T09:31:53.327002+0000","last_peered":"2024-09-16T09:31:53.327002+0000","last_clean":"2024-09-16T09:31:53.327002+0000","last_became_active":"2024-09-16T09:25:05.714341+0000","last_became_peered":"2024-09-16T09:25:05.714341+0000","last_unstale":"2024-09-16T09:31:53.327002+0000","last_undegraded":"2024-09-16T09:31:53.327002+0000","last_fullsized":"2024-09-16T09:31:53.327002+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:39.699059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:26:39.699059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":300,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.319089+0000","last_change":"2024-09-16T09:29:00.243528+0000","last_active":"2024-09-16T09:31:53.319089+0000","last_peered":"2024-09-16T09:31:53.319089+0000","last_clean":"2024-09-16T09:31:53.319089+0000","last_became_active":"2024-09-16T09:25:07.714281+0000","last_became_peered":"2024-09-16T09:25:07.714281+0000","last_unstale":"2024-09-16T09:31:53.319089+0000","last_undegraded":"2024-09-16T09:31:53.319089+0000","last_fullsized":"2024-09-16T09:31:53.319089+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:00.243482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:00.243482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.b","version":"73'79","reported_seq":274,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.319732+0000","last_change":"2024-09-16T09:31:47.267322+0000","last_active":"2024-09-16T09:31:53.319732+0000","last_peered":"2024-09-16T09:31:53.319732+0000","last_clean":"2024-09-16T09:31:53.319732+0000","last_became_active":"2024-09-16T09:30:10.784688+0000","last_became_peered":"2024-09-16T09:30:10.784688+0000","last_unstale":"2024-09-16T09:31:53.319732+0000","last_undegraded":"2024-09-16T09:31:53.319732+0000","last_fullsized":"2024-09-16T09:31:53.319732+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:30:03.887571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:30:03.887571+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:32:13.164874+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,5],"acting":[1,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2","version":"200'9789","reported_seq":9897,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:57.795514+0000","last_change":"2024-09-16T09:29:49.445324+0000","last_active":"2024-09-16T09:31:57.795514+0000","last_peered":"2024-09-16T09:31:57.795514+0000","last_clean":"2024-09-16T09:31:57.795514+0000","last_became_active":"2024-09-16T09:29:49.445022+0000","last_became_peered":"2024-09-16T09:29:49.445022+0000","last_unstale":"2024-09-16T09:31:57.795514+0000","last_undegraded":"2024-09-16T09:31:57.795514+0000","last_fullsized":"2024-09-16T09:31:57.795514+0000","mapping_epoch":152,"log_start":"180'5500","ondisk_log_start":"180'5500","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":4289,"log_dups_size":0,"ondisk_log_size":4289,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":356974592,"num_objects":5447,"num_object_clones":0,"num_object_copies":16341,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5447,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9789,"num_write_kb":487552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"66'11","reported_seq":339,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.306888+0000","last_change":"2024-09-16T09:30:05.646701+0000","last_active":"2024-09-16T09:31:53.306888+0000","last_peered":"2024-09-16T09:31:53.306888+0000","last_clean":"2024-09-16T09:31:53.306888+0000","last_became_active":"2024-09-16T09:26:33.287399+0000","last_became_peered":"2024-09-16T09:26:33.287399+0000","last_unstale":"2024-09-16T09:31:53.306888+0000","last_undegraded":"2024-09-16T09:31:53.306888+0000","last_fullsized":"2024-09-16T09:31:53.306888+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-16T09:30:05.646500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:30:05.646500+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":297,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.306748+0000","last_change":"2024-09-16T09:25:01.708568+0000","last_active":"2024-09-16T09:31:53.306748+0000","last_peered":"2024-09-16T09:31:53.306748+0000","last_clean":"2024-09-16T09:31:53.306748+0000","last_became_active":"2024-09-16T09:25:01.708270+0000","last_became_peered":"2024-09-16T09:25:01.708270+0000","last_unstale":"2024-09-16T09:31:53.306748+0000","last_undegraded":"2024-09-16T09:31:53.306748+0000","last_fullsized":"2024-09-16T09:31:53.306748+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":296,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.319005+0000","last_change":"2024-09-16T09:26:34.171190+0000","last_active":"2024-09-16T09:31:53.319005+0000","last_peered":"2024-09-16T09:31:53.319005+0000","last_clean":"2024-09-16T09:31:53.319005+0000","last_became_active":"2024-09-16T09:25:05.711964+0000","last_became_peered":"2024-09-16T09:25:05.711964+0000","last_unstale":"2024-09-16T09:31:53.319005+0000","last_undegraded":"2024-09-16T09:31:53.319005+0000","last_fullsized":"2024-09-16T09:31:53.319005+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:34.171139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:26:34.171139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":171,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.318870+0000","last_change":"2024-09-16T09:31:09.278959+0000","last_active":"2024-09-16T09:31:53.318870+0000","last_peered":"2024-09-16T09:31:53.318870+0000","last_clean":"2024-09-16T09:31:53.318870+0000","last_became_active":"2024-09-16T09:31:09.278618+0000","last_became_peered":"2024-09-16T09:31:09.278618+0000","last_unstale":"2024-09-16T09:31:53.318870+0000","last_undegraded":"2024-09-16T09:31:53.318870+0000","last_fullsized":"2024-09-16T09:31:53.318870+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:30:33.410232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:30:33.410232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:00.093285+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1","version":"200'9725","reported_seq":9833,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:57.835627+0000","last_change":"2024-09-16T09:29:49.449796+0000","last_active":"2024-09-16T09:31:57.835627+0000","last_peered":"2024-09-16T09:31:57.835627+0000","last_clean":"2024-09-16T09:31:57.835627+0000","last_became_active":"2024-09-16T09:29:49.449475+0000","last_became_peered":"2024-09-16T09:29:49.449475+0000","last_unstale":"2024-09-16T09:31:57.835627+0000","last_undegraded":"2024-09-16T09:31:57.835627+0000","last_fullsized":"2024-09-16T09:31:57.835627+0000","mapping_epoch":152,"log_start":"180'5200","ondisk_log_start":"180'5200","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":4525,"log_dups_size":0,"ondisk_log_size":4525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362610688,"num_objects":5533,"num_object_clones":0,"num_object_copies":16599,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5533,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9725,"num_write_kb":488256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.8","version":"73'79","reported_seq":268,"reported_epoch":200,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-16T09:31:53.324742+0000","last_change":"2024-09-16T09:31:09.430087+0000","last_active":"2024-09-16T09:31:53.324742+0000","last_peered":"2024-09-16T09:31:53.324742+0000","last_clean":"2024-09-16T09:31:07.261934+0000","last_became_active":"2024-09-16T09:31:09.406638+0000","last_became_peered":"2024-09-16T09:31:09.406638+0000","last_unstale":"2024-09-16T09:31:53.324742+0000","last_undegraded":"2024-09-16T09:31:09.427685+0000","last_fullsized":"2024-09-16T09:31:53.324742+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:17.482769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:17.482769+0000","objects_scrubbed":1,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262688,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":262688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":["5","2","6"],"object_location_counts":[{"shards":"2,5,6","objects":1}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"66'17","reported_seq":345,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.310062+0000","last_change":"2024-09-16T09:30:04.636115+0000","last_active":"2024-09-16T09:31:53.310062+0000","last_peered":"2024-09-16T09:31:53.310062+0000","last_clean":"2024-09-16T09:31:53.310062+0000","last_became_active":"2024-09-16T09:25:03.713358+0000","last_became_peered":"2024-09-16T09:25:03.713358+0000","last_unstale":"2024-09-16T09:31:53.310062+0000","last_undegraded":"2024-09-16T09:31:53.310062+0000","last_fullsized":"2024-09-16T09:31:53.310062+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'17","last_scrub_stamp":"2024-09-16T09:30:04.636039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:30:04.636039+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":305,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.318469+0000","last_change":"2024-09-16T09:26:04.104188+0000","last_active":"2024-09-16T09:31:53.318469+0000","last_peered":"2024-09-16T09:31:53.318469+0000","last_clean":"2024-09-16T09:31:53.318469+0000","last_became_active":"2024-09-16T09:25:01.703355+0000","last_became_peered":"2024-09-16T09:25:01.703355+0000","last_unstale":"2024-09-16T09:31:53.318469+0000","last_undegraded":"2024-09-16T09:31:53.318469+0000","last_fullsized":"2024-09-16T09:31:53.318469+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:04.104130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:04.104130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:48.432563+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":313,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.317159+0000","last_change":"2024-09-16T09:29:58.839132+0000","last_active":"2024-09-16T09:31:53.317159+0000","last_peered":"2024-09-16T09:31:53.317159+0000","last_clean":"2024-09-16T09:31:53.317159+0000","last_became_active":"2024-09-16T09:25:05.716900+0000","last_became_peered":"2024-09-16T09:25:05.716900+0000","last_unstale":"2024-09-16T09:31:53.317159+0000","last_undegraded":"2024-09-16T09:31:53.317159+0000","last_fullsized":"2024-09-16T09:31:53.317159+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:58.839051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:58.839051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":323,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.314383+0000","last_change":"2024-09-16T09:31:09.276869+0000","last_active":"2024-09-16T09:31:53.314383+0000","last_peered":"2024-09-16T09:31:53.314383+0000","last_clean":"2024-09-16T09:31:53.314383+0000","last_became_active":"2024-09-16T09:31:09.276275+0000","last_became_peered":"2024-09-16T09:31:09.276275+0000","last_unstale":"2024-09-16T09:31:53.314383+0000","last_undegraded":"2024-09-16T09:31:53.314383+0000","last_fullsized":"2024-09-16T09:31:53.314383+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:30:03.946025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:30:03.946025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.0","version":"200'10027","reported_seq":14972,"reported_epoch":200,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-16T09:31:57.913830+0000","last_change":"2024-09-16T09:31:09.476408+0000","last_active":"2024-09-16T09:31:57.913830+0000","last_peered":"2024-09-16T09:31:57.913830+0000","last_clean":"2024-09-16T09:30:20.486290+0000","last_became_active":"2024-09-16T09:31:09.399514+0000","last_became_peered":"2024-09-16T09:31:09.399514+0000","last_unstale":"2024-09-16T09:31:57.913830+0000","last_undegraded":"2024-09-16T09:31:57.913830+0000","last_fullsized":"2024-09-16T09:31:09.254572+0000","mapping_epoch":186,"log_start":"180'5300","ondisk_log_start":"180'5300","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":4727,"log_dups_size":0,"ondisk_log_size":4727,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":361037824,"num_objects":5509,"num_object_clones":0,"num_object_copies":16527,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2289,"num_objects_unfound":0,"num_objects_dirty":5509,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10027,"num_write_kb":497152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2413,"num_bytes_recovered":158138368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,1],"avail_no_missing":["5","1"],"object_location_counts":[{"shards":"1,4,5","objects":2289},{"shards":"1,5","objects":3220}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.9","version":"73'79","reported_seq":376,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.325787+0000","last_change":"2024-09-16T09:29:19.013948+0000","last_active":"2024-09-16T09:31:53.325787+0000","last_peered":"2024-09-16T09:31:53.325787+0000","last_clean":"2024-09-16T09:31:53.325787+0000","last_became_active":"2024-09-16T09:26:35.126066+0000","last_became_peered":"2024-09-16T09:26:35.126066+0000","last_unstale":"2024-09-16T09:31:53.325787+0000","last_undegraded":"2024-09-16T09:31:53.325787+0000","last_fullsized":"2024-09-16T09:31:53.325787+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:19.013865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:19.013865+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"66'10","reported_seq":308,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.308372+0000","last_change":"2024-09-16T09:25:03.708144+0000","last_active":"2024-09-16T09:31:53.308372+0000","last_peered":"2024-09-16T09:31:53.308372+0000","last_clean":"2024-09-16T09:31:53.308372+0000","last_became_active":"2024-09-16T09:25:03.707723+0000","last_became_peered":"2024-09-16T09:25:03.707723+0000","last_unstale":"2024-09-16T09:31:53.308372+0000","last_undegraded":"2024-09-16T09:31:53.308372+0000","last_fullsized":"2024-09-16T09:31:53.308372+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":170,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.328394+0000","last_change":"2024-09-16T09:31:32.253397+0000","last_active":"2024-09-16T09:31:53.328394+0000","last_peered":"2024-09-16T09:31:53.328394+0000","last_clean":"2024-09-16T09:31:53.328394+0000","last_became_active":"2024-09-16T09:31:09.288825+0000","last_became_peered":"2024-09-16T09:31:09.288825+0000","last_unstale":"2024-09-16T09:31:53.328394+0000","last_undegraded":"2024-09-16T09:31:53.328394+0000","last_fullsized":"2024-09-16T09:31:53.328394+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:31:32.253306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:31:32.253306+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:45.682494+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":408,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.310264+0000","last_change":"2024-09-16T09:29:29.866520+0000","last_active":"2024-09-16T09:31:53.310264+0000","last_peered":"2024-09-16T09:31:53.310264+0000","last_clean":"2024-09-16T09:31:53.310264+0000","last_became_active":"2024-09-16T09:25:05.711590+0000","last_became_peered":"2024-09-16T09:25:05.711590+0000","last_unstale":"2024-09-16T09:31:53.310264+0000","last_undegraded":"2024-09-16T09:31:53.310264+0000","last_fullsized":"2024-09-16T09:31:53.310264+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:29:29.866460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:29.866460+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":309,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.306649+0000","last_change":"2024-09-16T09:29:02.667332+0000","last_active":"2024-09-16T09:31:53.306649+0000","last_peered":"2024-09-16T09:31:53.306649+0000","last_clean":"2024-09-16T09:31:53.306649+0000","last_became_active":"2024-09-16T09:25:07.718958+0000","last_became_peered":"2024-09-16T09:25:07.718958+0000","last_unstale":"2024-09-16T09:31:53.306649+0000","last_undegraded":"2024-09-16T09:31:53.306649+0000","last_fullsized":"2024-09-16T09:31:53.306649+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:02.667259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:02.667259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.e","version":"73'79","reported_seq":98,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.306496+0000","last_change":"2024-09-16T09:31:47.249302+0000","last_active":"2024-09-16T09:31:53.306496+0000","last_peered":"2024-09-16T09:31:53.306496+0000","last_clean":"2024-09-16T09:31:53.306496+0000","last_became_active":"2024-09-16T09:30:10.705072+0000","last_became_peered":"2024-09-16T09:30:10.705072+0000","last_unstale":"2024-09-16T09:31:53.306496+0000","last_undegraded":"2024-09-16T09:31:53.306496+0000","last_fullsized":"2024-09-16T09:31:53.306496+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:26:35.170229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:26:35.170229+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:32:12.841915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26835,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":10,"num_write":5,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":26835,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,3],"acting":[4,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.7","version":"200'9913","reported_seq":10020,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:57.833951+0000","last_change":"2024-09-16T09:29:49.839740+0000","last_active":"2024-09-16T09:31:57.833951+0000","last_peered":"2024-09-16T09:31:57.833951+0000","last_clean":"2024-09-16T09:31:57.833951+0000","last_became_active":"2024-09-16T09:29:49.839437+0000","last_became_peered":"2024-09-16T09:29:49.839437+0000","last_unstale":"2024-09-16T09:31:57.833951+0000","last_undegraded":"2024-09-16T09:31:57.833951+0000","last_fullsized":"2024-09-16T09:31:57.833951+0000","mapping_epoch":152,"log_start":"180'5400","ondisk_log_start":"180'5400","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":4513,"log_dups_size":0,"ondisk_log_size":4513,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362479616,"num_objects":5531,"num_object_clones":0,"num_object_copies":16593,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5531,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9913,"num_write_kb":494208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"66'9","reported_seq":333,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.327047+0000","last_change":"2024-09-16T09:31:31.211495+0000","last_active":"2024-09-16T09:31:53.327047+0000","last_peered":"2024-09-16T09:31:53.327047+0000","last_clean":"2024-09-16T09:31:53.327047+0000","last_became_active":"2024-09-16T09:25:03.712230+0000","last_became_peered":"2024-09-16T09:25:03.712230+0000","last_unstale":"2024-09-16T09:31:53.327047+0000","last_undegraded":"2024-09-16T09:31:53.327047+0000","last_fullsized":"2024-09-16T09:31:53.327047+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:31:31.211420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:31:31.211420+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:36.499658+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":318,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.320216+0000","last_change":"2024-09-16T09:31:09.277413+0000","last_active":"2024-09-16T09:31:53.320216+0000","last_peered":"2024-09-16T09:31:53.320216+0000","last_clean":"2024-09-16T09:31:53.320216+0000","last_became_active":"2024-09-16T09:31:09.276516+0000","last_became_peered":"2024-09-16T09:31:09.276516+0000","last_unstale":"2024-09-16T09:31:53.320216+0000","last_undegraded":"2024-09-16T09:31:53.320216+0000","last_fullsized":"2024-09-16T09:31:53.320216+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:26.061106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:26.061106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":319,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.311860+0000","last_change":"2024-09-16T09:31:09.275219+0000","last_active":"2024-09-16T09:31:53.311860+0000","last_peered":"2024-09-16T09:31:53.311860+0000","last_clean":"2024-09-16T09:31:53.311860+0000","last_became_active":"2024-09-16T09:31:09.274702+0000","last_became_peered":"2024-09-16T09:31:09.274702+0000","last_unstale":"2024-09-16T09:31:53.311860+0000","last_undegraded":"2024-09-16T09:31:53.311860+0000","last_fullsized":"2024-09-16T09:31:53.311860+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:57.220068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:28:57.220068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":33,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":309,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.332049+0000","last_change":"2024-09-16T09:31:27.185957+0000","last_active":"2024-09-16T09:31:53.332049+0000","last_peered":"2024-09-16T09:31:53.332049+0000","last_clean":"2024-09-16T09:31:53.332049+0000","last_became_active":"2024-09-16T09:25:07.718778+0000","last_became_peered":"2024-09-16T09:25:07.718778+0000","last_unstale":"2024-09-16T09:31:53.332049+0000","last_undegraded":"2024-09-16T09:31:53.332049+0000","last_fullsized":"2024-09-16T09:31:53.332049+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:31:27.185801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:31:27.185801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:39.017009+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.f","version":"73'79","reported_seq":350,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.319998+0000","last_change":"2024-09-16T09:31:47.259689+0000","last_active":"2024-09-16T09:31:53.319998+0000","last_peered":"2024-09-16T09:31:53.319998+0000","last_clean":"2024-09-16T09:31:53.319998+0000","last_became_active":"2024-09-16T09:30:10.693477+0000","last_became_peered":"2024-09-16T09:30:10.693477+0000","last_unstale":"2024-09-16T09:31:53.319998+0000","last_undegraded":"2024-09-16T09:31:53.319998+0000","last_fullsized":"2024-09-16T09:31:53.319998+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:17.534429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:17.534429+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:32:13.472806+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.6","version":"200'10218","reported_seq":10325,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:58.131529+0000","last_change":"2024-09-16T09:29:50.008888+0000","last_active":"2024-09-16T09:31:58.131529+0000","last_peered":"2024-09-16T09:31:58.131529+0000","last_clean":"2024-09-16T09:31:58.131529+0000","last_became_active":"2024-09-16T09:29:50.008656+0000","last_became_peered":"2024-09-16T09:29:50.008656+0000","last_unstale":"2024-09-16T09:31:58.131529+0000","last_undegraded":"2024-09-16T09:31:58.131529+0000","last_fullsized":"2024-09-16T09:31:58.131529+0000","mapping_epoch":152,"log_start":"178'5100","ondisk_log_start":"178'5100","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":5118,"log_dups_size":0,"ondisk_log_size":5118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":367263744,"num_objects":5604,"num_object_clones":0,"num_object_copies":16812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5604,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10218,"num_write_kb":506304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"66'19","reported_seq":340,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.318277+0000","last_change":"2024-09-16T09:29:02.554357+0000","last_active":"2024-09-16T09:31:53.318277+0000","last_peered":"2024-09-16T09:31:53.318277+0000","last_clean":"2024-09-16T09:31:53.318277+0000","last_became_active":"2024-09-16T09:25:03.712783+0000","last_became_peered":"2024-09-16T09:25:03.712783+0000","last_unstale":"2024-09-16T09:31:53.318277+0000","last_undegraded":"2024-09-16T09:31:53.318277+0000","last_fullsized":"2024-09-16T09:31:53.318277+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-16T09:29:02.554274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:29:02.554274+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:17.371584+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"59'1","reported_seq":318,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.328457+0000","last_change":"2024-09-16T09:29:51.025505+0000","last_active":"2024-09-16T09:31:53.328457+0000","last_peered":"2024-09-16T09:31:53.328457+0000","last_clean":"2024-09-16T09:31:53.328457+0000","last_became_active":"2024-09-16T09:25:01.708069+0000","last_became_peered":"2024-09-16T09:25:01.708069+0000","last_unstale":"2024-09-16T09:31:53.328457+0000","last_undegraded":"2024-09-16T09:31:53.328457+0000","last_fullsized":"2024-09-16T09:31:53.328457+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:29:51.025444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:51.025444+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":297,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.309990+0000","last_change":"2024-09-16T09:27:11.067059+0000","last_active":"2024-09-16T09:31:53.309990+0000","last_peered":"2024-09-16T09:31:53.309990+0000","last_clean":"2024-09-16T09:31:53.309990+0000","last_became_active":"2024-09-16T09:25:05.709264+0000","last_became_peered":"2024-09-16T09:25:05.709264+0000","last_unstale":"2024-09-16T09:31:53.309990+0000","last_undegraded":"2024-09-16T09:31:53.309990+0000","last_fullsized":"2024-09-16T09:31:53.309990+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:11.066986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:11.066986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":292,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.325292+0000","last_change":"2024-09-16T09:27:25.865107+0000","last_active":"2024-09-16T09:31:53.325292+0000","last_peered":"2024-09-16T09:31:53.325292+0000","last_clean":"2024-09-16T09:31:53.325292+0000","last_became_active":"2024-09-16T09:25:07.718750+0000","last_became_peered":"2024-09-16T09:25:07.718750+0000","last_unstale":"2024-09-16T09:31:53.325292+0000","last_undegraded":"2024-09-16T09:31:53.325292+0000","last_fullsized":"2024-09-16T09:31:53.325292+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:25.865030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:25.865030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.c","version":"73'79","reported_seq":42,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.319801+0000","last_change":"2024-09-16T09:31:09.290919+0000","last_active":"2024-09-16T09:31:53.319801+0000","last_peered":"2024-09-16T09:31:53.319801+0000","last_clean":"2024-09-16T09:31:53.319801+0000","last_became_active":"2024-09-16T09:31:09.273770+0000","last_became_peered":"2024-09-16T09:31:09.273770+0000","last_unstale":"2024-09-16T09:31:53.319801+0000","last_undegraded":"2024-09-16T09:31:53.319801+0000","last_fullsized":"2024-09-16T09:31:53.319801+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:30:22.614462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:30:22.614462+0000","objects_scrubbed":1,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":327680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.5","version":"200'10023","reported_seq":10204,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:57.586829+0000","last_change":"2024-09-16T09:31:06.121217+0000","last_active":"2024-09-16T09:31:57.586829+0000","last_peered":"2024-09-16T09:31:57.586829+0000","last_clean":"2024-09-16T09:31:57.586829+0000","last_became_active":"2024-09-16T09:29:50.009013+0000","last_became_peered":"2024-09-16T09:29:50.009013+0000","last_unstale":"2024-09-16T09:31:57.586829+0000","last_undegraded":"2024-09-16T09:31:57.586829+0000","last_fullsized":"2024-09-16T09:31:57.586829+0000","mapping_epoch":152,"log_start":"177'4813","ondisk_log_start":"177'4813","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"184'6587","last_scrub_stamp":"2024-09-16T09:31:06.120990+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:31:06.120990+0000","objects_scrubbed":6523,"log_size":5210,"log_dups_size":0,"ondisk_log_size":5210,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:20.260788+0000","scrub_duration":1690,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":374931456,"num_objects":5721,"num_object_clones":0,"num_object_copies":17163,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5721,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10023,"num_write_kb":503808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"66'12","reported_seq":370,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.306611+0000","last_change":"2024-09-16T09:31:09.296915+0000","last_active":"2024-09-16T09:31:53.306611+0000","last_peered":"2024-09-16T09:31:53.306611+0000","last_clean":"2024-09-16T09:31:53.306611+0000","last_became_active":"2024-09-16T09:31:09.269047+0000","last_became_peered":"2024-09-16T09:31:09.269047+0000","last_unstale":"2024-09-16T09:31:53.306611+0000","last_undegraded":"2024-09-16T09:31:53.306611+0000","last_fullsized":"2024-09-16T09:31:53.306611+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-16T09:30:21.728668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:30:21.728668+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":312,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.327615+0000","last_change":"2024-09-16T09:29:53.075288+0000","last_active":"2024-09-16T09:31:53.327615+0000","last_peered":"2024-09-16T09:31:53.327615+0000","last_clean":"2024-09-16T09:31:53.327615+0000","last_became_active":"2024-09-16T09:25:01.707528+0000","last_became_peered":"2024-09-16T09:25:01.707528+0000","last_unstale":"2024-09-16T09:31:53.327615+0000","last_undegraded":"2024-09-16T09:31:53.327615+0000","last_fullsized":"2024-09-16T09:31:53.327615+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:53.075232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:53.075232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":305,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.309614+0000","last_change":"2024-09-16T09:27:49.965628+0000","last_active":"2024-09-16T09:31:53.309614+0000","last_peered":"2024-09-16T09:31:53.309614+0000","last_clean":"2024-09-16T09:31:53.309614+0000","last_became_active":"2024-09-16T09:25:05.715253+0000","last_became_peered":"2024-09-16T09:25:05.715253+0000","last_unstale":"2024-09-16T09:31:53.309614+0000","last_undegraded":"2024-09-16T09:31:53.309614+0000","last_fullsized":"2024-09-16T09:31:53.309614+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:49.965424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:49.965424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":293,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.318472+0000","last_change":"2024-09-16T09:27:56.059755+0000","last_active":"2024-09-16T09:31:53.318472+0000","last_peered":"2024-09-16T09:31:53.318472+0000","last_clean":"2024-09-16T09:31:53.318472+0000","last_became_active":"2024-09-16T09:25:07.718899+0000","last_became_peered":"2024-09-16T09:25:07.718899+0000","last_unstale":"2024-09-16T09:31:53.318472+0000","last_undegraded":"2024-09-16T09:31:53.318472+0000","last_fullsized":"2024-09-16T09:31:53.318472+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:56.059687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:56.059687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.d","version":"73'79","reported_seq":98,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.327196+0000","last_change":"2024-09-16T09:31:47.263954+0000","last_active":"2024-09-16T09:31:53.327196+0000","last_peered":"2024-09-16T09:31:53.327196+0000","last_clean":"2024-09-16T09:31:53.327196+0000","last_became_active":"2024-09-16T09:30:10.709958+0000","last_became_peered":"2024-09-16T09:30:10.709958+0000","last_unstale":"2024-09-16T09:31:53.327196+0000","last_undegraded":"2024-09-16T09:31:53.327196+0000","last_fullsized":"2024-09-16T09:31:53.327196+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:58.568841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:58.568841+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:32:13.456459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26835,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":10,"num_write":5,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":26835,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.4","version":"200'10007","reported_seq":10114,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:57.918591+0000","last_change":"2024-09-16T09:29:49.444857+0000","last_active":"2024-09-16T09:31:57.918591+0000","last_peered":"2024-09-16T09:31:57.918591+0000","last_clean":"2024-09-16T09:31:57.918591+0000","last_became_active":"2024-09-16T09:29:49.444515+0000","last_became_peered":"2024-09-16T09:29:49.444515+0000","last_unstale":"2024-09-16T09:31:57.918591+0000","last_undegraded":"2024-09-16T09:31:57.918591+0000","last_fullsized":"2024-09-16T09:31:57.918591+0000","mapping_epoch":152,"log_start":"180'5300","ondisk_log_start":"180'5300","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":4707,"log_dups_size":0,"ondisk_log_size":4707,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":367067136,"num_objects":5601,"num_object_clones":0,"num_object_copies":16803,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5601,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10007,"num_write_kb":499456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"66'15","reported_seq":325,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.325020+0000","last_change":"2024-09-16T09:26:33.138828+0000","last_active":"2024-09-16T09:31:53.325020+0000","last_peered":"2024-09-16T09:31:53.325020+0000","last_clean":"2024-09-16T09:31:53.325020+0000","last_became_active":"2024-09-16T09:25:03.715733+0000","last_became_peered":"2024-09-16T09:25:03.715733+0000","last_unstale":"2024-09-16T09:31:53.325020+0000","last_undegraded":"2024-09-16T09:31:53.325020+0000","last_fullsized":"2024-09-16T09:31:53.325020+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-16T09:26:33.138603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:26:33.138603+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":316,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.326642+0000","last_change":"2024-09-16T09:26:26.705847+0000","last_active":"2024-09-16T09:31:53.326642+0000","last_peered":"2024-09-16T09:31:53.326642+0000","last_clean":"2024-09-16T09:31:53.326642+0000","last_became_active":"2024-09-16T09:25:01.707372+0000","last_became_peered":"2024-09-16T09:25:01.707372+0000","last_unstale":"2024-09-16T09:31:53.326642+0000","last_undegraded":"2024-09-16T09:31:53.326642+0000","last_fullsized":"2024-09-16T09:31:53.326642+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-16T09:26:26.705773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:26.705773+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"66'8","reported_seq":404,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.325741+0000","last_change":"2024-09-16T09:30:05.138362+0000","last_active":"2024-09-16T09:31:53.325741+0000","last_peered":"2024-09-16T09:31:53.325741+0000","last_clean":"2024-09-16T09:31:53.325741+0000","last_became_active":"2024-09-16T09:25:05.716189+0000","last_became_peered":"2024-09-16T09:25:05.716189+0000","last_unstale":"2024-09-16T09:31:53.325741+0000","last_undegraded":"2024-09-16T09:31:53.325741+0000","last_fullsized":"2024-09-16T09:31:53.325741+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:30:05.138288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:30:05.138288+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":292,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.320915+0000","last_change":"2024-09-16T09:27:20.219376+0000","last_active":"2024-09-16T09:31:53.320915+0000","last_peered":"2024-09-16T09:31:53.320915+0000","last_clean":"2024-09-16T09:31:53.320915+0000","last_became_active":"2024-09-16T09:25:07.717213+0000","last_became_peered":"2024-09-16T09:25:07.717213+0000","last_unstale":"2024-09-16T09:31:53.320915+0000","last_undegraded":"2024-09-16T09:31:53.320915+0000","last_fullsized":"2024-09-16T09:31:53.320915+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:20.219333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:20.219333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.12","version":"73'79","reported_seq":98,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.321162+0000","last_change":"2024-09-16T09:31:47.251225+0000","last_active":"2024-09-16T09:31:53.321162+0000","last_peered":"2024-09-16T09:31:53.321162+0000","last_clean":"2024-09-16T09:31:53.321162+0000","last_became_active":"2024-09-16T09:30:10.731413+0000","last_became_peered":"2024-09-16T09:30:10.731413+0000","last_unstale":"2024-09-16T09:31:53.321162+0000","last_undegraded":"2024-09-16T09:31:53.321162+0000","last_fullsized":"2024-09-16T09:31:53.321162+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:26:35.745837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:26:35.745837+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:32:14.398546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26835,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":10,"num_write":6,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":26835,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":305,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.318064+0000","last_change":"2024-09-16T09:26:33.139405+0000","last_active":"2024-09-16T09:31:53.318064+0000","last_peered":"2024-09-16T09:31:53.318064+0000","last_clean":"2024-09-16T09:31:53.318064+0000","last_became_active":"2024-09-16T09:25:01.703202+0000","last_became_peered":"2024-09-16T09:25:01.703202+0000","last_unstale":"2024-09-16T09:31:53.318064+0000","last_undegraded":"2024-09-16T09:31:53.318064+0000","last_fullsized":"2024-09-16T09:31:53.318064+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:33.139360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:33.139360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:10.508509+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"66'6","reported_seq":46,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.318708+0000","last_change":"2024-09-16T09:31:09.297514+0000","last_active":"2024-09-16T09:31:53.318708+0000","last_peered":"2024-09-16T09:31:53.318708+0000","last_clean":"2024-09-16T09:31:53.318708+0000","last_became_active":"2024-09-16T09:31:09.270685+0000","last_became_peered":"2024-09-16T09:31:09.270685+0000","last_unstale":"2024-09-16T09:31:53.318708+0000","last_undegraded":"2024-09-16T09:31:53.318708+0000","last_fullsized":"2024-09-16T09:31:53.318708+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'6","last_scrub_stamp":"2024-09-16T09:26:23.458794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:26:23.458794+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":8,"num_read_kb":5,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":303,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.319965+0000","last_change":"2024-09-16T09:29:31.496374+0000","last_active":"2024-09-16T09:31:53.319965+0000","last_peered":"2024-09-16T09:31:53.319965+0000","last_clean":"2024-09-16T09:31:53.319965+0000","last_became_active":"2024-09-16T09:25:05.715616+0000","last_became_peered":"2024-09-16T09:25:05.715616+0000","last_unstale":"2024-09-16T09:31:53.319965+0000","last_undegraded":"2024-09-16T09:31:53.319965+0000","last_fullsized":"2024-09-16T09:31:53.319965+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:31.496304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:31.496304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":291,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.328112+0000","last_change":"2024-09-16T09:26:28.733381+0000","last_active":"2024-09-16T09:31:53.328112+0000","last_peered":"2024-09-16T09:31:53.328112+0000","last_clean":"2024-09-16T09:31:53.328112+0000","last_became_active":"2024-09-16T09:25:07.721059+0000","last_became_peered":"2024-09-16T09:25:07.721059+0000","last_unstale":"2024-09-16T09:31:53.328112+0000","last_undegraded":"2024-09-16T09:31:53.328112+0000","last_fullsized":"2024-09-16T09:31:53.328112+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:28.733324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:28.733324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.13","version":"73'79","reported_seq":287,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.320090+0000","last_change":"2024-09-16T09:31:09.288677+0000","last_active":"2024-09-16T09:31:53.320090+0000","last_peered":"2024-09-16T09:31:53.320090+0000","last_clean":"2024-09-16T09:31:53.320090+0000","last_became_active":"2024-09-16T09:31:09.288233+0000","last_became_peered":"2024-09-16T09:31:09.288233+0000","last_unstale":"2024-09-16T09:31:53.320090+0000","last_undegraded":"2024-09-16T09:31:53.320090+0000","last_fullsized":"2024-09-16T09:31:53.320090+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:30:21.714369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:30:21.714369+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.16","version":"66'9","reported_seq":366,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.328178+0000","last_change":"2024-09-16T09:31:09.309570+0000","last_active":"2024-09-16T09:31:53.328178+0000","last_peered":"2024-09-16T09:31:53.328178+0000","last_clean":"2024-09-16T09:31:53.328178+0000","last_became_active":"2024-09-16T09:31:09.273498+0000","last_became_peered":"2024-09-16T09:31:09.273498+0000","last_unstale":"2024-09-16T09:31:53.328178+0000","last_undegraded":"2024-09-16T09:31:53.328178+0000","last_fullsized":"2024-09-16T09:31:53.328178+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:29:58.867513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:29:58.867513+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":311,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.326010+0000","last_change":"2024-09-16T09:28:50.698331+0000","last_active":"2024-09-16T09:31:53.326010+0000","last_peered":"2024-09-16T09:31:53.326010+0000","last_clean":"2024-09-16T09:31:53.326010+0000","last_became_active":"2024-09-16T09:25:01.706491+0000","last_became_peered":"2024-09-16T09:25:01.706491+0000","last_unstale":"2024-09-16T09:31:53.326010+0000","last_undegraded":"2024-09-16T09:31:53.326010+0000","last_fullsized":"2024-09-16T09:31:53.326010+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:50.698288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:28:50.698288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":162,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.327116+0000","last_change":"2024-09-16T09:31:09.271558+0000","last_active":"2024-09-16T09:31:53.327116+0000","last_peered":"2024-09-16T09:31:53.327116+0000","last_clean":"2024-09-16T09:31:53.327116+0000","last_became_active":"2024-09-16T09:31:09.271223+0000","last_became_peered":"2024-09-16T09:31:09.271223+0000","last_unstale":"2024-09-16T09:31:53.327116+0000","last_undegraded":"2024-09-16T09:31:53.327116+0000","last_fullsized":"2024-09-16T09:31:53.327116+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:53.810580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:28:53.810580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":292,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.310195+0000","last_change":"2024-09-16T09:26:35.866582+0000","last_active":"2024-09-16T09:31:53.310195+0000","last_peered":"2024-09-16T09:31:53.310195+0000","last_clean":"2024-09-16T09:31:53.310195+0000","last_became_active":"2024-09-16T09:25:07.722050+0000","last_became_peered":"2024-09-16T09:25:07.722050+0000","last_unstale":"2024-09-16T09:31:53.310195+0000","last_undegraded":"2024-09-16T09:31:53.310195+0000","last_fullsized":"2024-09-16T09:31:53.310195+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:35.866512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:35.866512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:08.904503+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.10","version":"73'79","reported_seq":98,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.320407+0000","last_change":"2024-09-16T09:31:47.249070+0000","last_active":"2024-09-16T09:31:53.320407+0000","last_peered":"2024-09-16T09:31:53.320407+0000","last_clean":"2024-09-16T09:31:53.320407+0000","last_became_active":"2024-09-16T09:30:10.692770+0000","last_became_peered":"2024-09-16T09:30:10.692770+0000","last_unstale":"2024-09-16T09:31:53.320407+0000","last_undegraded":"2024-09-16T09:31:53.320407+0000","last_fullsized":"2024-09-16T09:31:53.320407+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:02.723794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:02.723794+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:32:13.396012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26835,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":10,"num_write":5,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":26835,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"66'9","reported_seq":323,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.319276+0000","last_change":"2024-09-16T09:29:32.485165+0000","last_active":"2024-09-16T09:31:53.319276+0000","last_peered":"2024-09-16T09:31:53.319276+0000","last_clean":"2024-09-16T09:31:53.319276+0000","last_became_active":"2024-09-16T09:25:03.717310+0000","last_became_peered":"2024-09-16T09:25:03.717310+0000","last_unstale":"2024-09-16T09:31:53.319276+0000","last_undegraded":"2024-09-16T09:31:53.319276+0000","last_fullsized":"2024-09-16T09:31:53.319276+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:29:32.485084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:29:32.485084+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":326,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.327559+0000","last_change":"2024-09-16T09:31:09.273453+0000","last_active":"2024-09-16T09:31:53.327559+0000","last_peered":"2024-09-16T09:31:53.327559+0000","last_clean":"2024-09-16T09:31:53.327559+0000","last_became_active":"2024-09-16T09:31:09.272848+0000","last_became_peered":"2024-09-16T09:31:09.272848+0000","last_unstale":"2024-09-16T09:31:53.327559+0000","last_undegraded":"2024-09-16T09:31:53.327559+0000","last_fullsized":"2024-09-16T09:31:53.327559+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.950144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.950144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"187'11","reported_seq":233,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.326955+0000","last_change":"2024-09-16T09:31:09.311243+0000","last_active":"2024-09-16T09:31:53.326955+0000","last_peered":"2024-09-16T09:31:53.326955+0000","last_clean":"2024-09-16T09:31:53.326955+0000","last_became_active":"2024-09-16T09:31:09.277688+0000","last_became_peered":"2024-09-16T09:31:09.277688+0000","last_unstale":"2024-09-16T09:31:53.326955+0000","last_undegraded":"2024-09-16T09:31:53.326955+0000","last_fullsized":"2024-09-16T09:31:53.326955+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:28:53.714413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:28:53.714413+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":308,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.306990+0000","last_change":"2024-09-16T09:29:49.474440+0000","last_active":"2024-09-16T09:31:53.306990+0000","last_peered":"2024-09-16T09:31:53.306990+0000","last_clean":"2024-09-16T09:31:53.306990+0000","last_became_active":"2024-09-16T09:25:07.718810+0000","last_became_peered":"2024-09-16T09:25:07.718810+0000","last_unstale":"2024-09-16T09:31:53.306990+0000","last_undegraded":"2024-09-16T09:31:53.306990+0000","last_fullsized":"2024-09-16T09:31:53.306990+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:49.474391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:49.474391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.11","version":"73'79","reported_seq":187,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.332983+0000","last_change":"2024-09-16T09:31:09.272938+0000","last_active":"2024-09-16T09:31:53.332983+0000","last_peered":"2024-09-16T09:31:53.332983+0000","last_clean":"2024-09-16T09:31:53.332983+0000","last_became_active":"2024-09-16T09:31:09.272208+0000","last_became_peered":"2024-09-16T09:31:09.272208+0000","last_unstale":"2024-09-16T09:31:53.332983+0000","last_undegraded":"2024-09-16T09:31:53.332983+0000","last_fullsized":"2024-09-16T09:31:53.332983+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:30:21.651575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:30:21.651575+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.14","version":"66'10","reported_seq":183,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.317279+0000","last_change":"2024-09-16T09:28:56.825675+0000","last_active":"2024-09-16T09:31:53.317279+0000","last_peered":"2024-09-16T09:31:53.317279+0000","last_clean":"2024-09-16T09:31:53.317279+0000","last_became_active":"2024-09-16T09:28:52.979059+0000","last_became_peered":"2024-09-16T09:28:52.979059+0000","last_unstale":"2024-09-16T09:31:53.317279+0000","last_undegraded":"2024-09-16T09:31:53.317279+0000","last_fullsized":"2024-09-16T09:31:53.317279+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-16T09:28:56.825476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:56.825476+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":14,"num_read_kb":9,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":311,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.326310+0000","last_change":"2024-09-16T09:29:56.021413+0000","last_active":"2024-09-16T09:31:53.326310+0000","last_peered":"2024-09-16T09:31:53.326310+0000","last_clean":"2024-09-16T09:31:53.326310+0000","last_became_active":"2024-09-16T09:25:01.707513+0000","last_became_peered":"2024-09-16T09:25:01.707513+0000","last_unstale":"2024-09-16T09:31:53.326310+0000","last_undegraded":"2024-09-16T09:31:53.326310+0000","last_fullsized":"2024-09-16T09:31:53.326310+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:56.021340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:56.021340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"66'8","reported_seq":403,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.319105+0000","last_change":"2024-09-16T09:29:48.485781+0000","last_active":"2024-09-16T09:31:53.319105+0000","last_peered":"2024-09-16T09:31:53.319105+0000","last_clean":"2024-09-16T09:31:53.319105+0000","last_became_active":"2024-09-16T09:25:05.713833+0000","last_became_peered":"2024-09-16T09:25:05.713833+0000","last_unstale":"2024-09-16T09:31:53.319105+0000","last_undegraded":"2024-09-16T09:31:53.319105+0000","last_fullsized":"2024-09-16T09:31:53.319105+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:29:48.485727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.485727+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":314,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.327764+0000","last_change":"2024-09-16T09:31:09.276853+0000","last_active":"2024-09-16T09:31:53.327764+0000","last_peered":"2024-09-16T09:31:53.327764+0000","last_clean":"2024-09-16T09:31:53.327764+0000","last_became_active":"2024-09-16T09:31:09.275168+0000","last_became_peered":"2024-09-16T09:31:09.275168+0000","last_unstale":"2024-09-16T09:31:53.327764+0000","last_undegraded":"2024-09-16T09:31:53.327764+0000","last_fullsized":"2024-09-16T09:31:53.327764+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:52.958303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:52.958303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"66'11","reported_seq":330,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.311005+0000","last_change":"2024-09-16T09:26:33.302713+0000","last_active":"2024-09-16T09:31:53.311005+0000","last_peered":"2024-09-16T09:31:53.311005+0000","last_clean":"2024-09-16T09:31:53.311005+0000","last_became_active":"2024-09-16T09:26:33.291265+0000","last_became_peered":"2024-09-16T09:26:33.291265+0000","last_unstale":"2024-09-16T09:31:53.311005+0000","last_undegraded":"2024-09-16T09:31:53.311005+0000","last_fullsized":"2024-09-16T09:31:53.311005+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":320,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.308566+0000","last_change":"2024-09-16T09:29:51.517112+0000","last_active":"2024-09-16T09:31:53.308566+0000","last_peered":"2024-09-16T09:31:53.308566+0000","last_clean":"2024-09-16T09:31:53.308566+0000","last_became_active":"2024-09-16T09:25:01.705158+0000","last_became_peered":"2024-09-16T09:25:01.705158+0000","last_unstale":"2024-09-16T09:31:53.308566+0000","last_undegraded":"2024-09-16T09:31:53.308566+0000","last_fullsized":"2024-09-16T09:31:53.308566+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:51.517054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:51.517054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":326,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.326173+0000","last_change":"2024-09-16T09:31:09.266820+0000","last_active":"2024-09-16T09:31:53.326173+0000","last_peered":"2024-09-16T09:31:53.326173+0000","last_clean":"2024-09-16T09:31:53.326173+0000","last_became_active":"2024-09-16T09:31:09.266326+0000","last_became_peered":"2024-09-16T09:31:09.266326+0000","last_unstale":"2024-09-16T09:31:53.326173+0000","last_undegraded":"2024-09-16T09:31:53.326173+0000","last_fullsized":"2024-09-16T09:31:53.326173+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:15.910824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:15.910824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":155,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.333406+0000","last_change":"2024-09-16T09:31:09.277239+0000","last_active":"2024-09-16T09:31:53.333406+0000","last_peered":"2024-09-16T09:31:53.333406+0000","last_clean":"2024-09-16T09:31:53.333406+0000","last_became_active":"2024-09-16T09:31:09.276042+0000","last_became_peered":"2024-09-16T09:31:09.276042+0000","last_unstale":"2024-09-16T09:31:53.333406+0000","last_undegraded":"2024-09-16T09:31:53.333406+0000","last_fullsized":"2024-09-16T09:31:53.333406+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:48.147759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:48.147759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:14.230973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"66'9","reported_seq":316,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.326507+0000","last_change":"2024-09-16T09:26:31.216495+0000","last_active":"2024-09-16T09:31:53.326507+0000","last_peered":"2024-09-16T09:31:53.326507+0000","last_clean":"2024-09-16T09:31:53.326507+0000","last_became_active":"2024-09-16T09:25:03.705807+0000","last_became_peered":"2024-09-16T09:25:03.705807+0000","last_unstale":"2024-09-16T09:31:53.326507+0000","last_undegraded":"2024-09-16T09:31:53.326507+0000","last_fullsized":"2024-09-16T09:31:53.326507+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:26:31.216298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:26:31.216298+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":303,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.328319+0000","last_change":"2024-09-16T09:26:23.796308+0000","last_active":"2024-09-16T09:31:53.328319+0000","last_peered":"2024-09-16T09:31:53.328319+0000","last_clean":"2024-09-16T09:31:53.328319+0000","last_became_active":"2024-09-16T09:25:01.706762+0000","last_became_peered":"2024-09-16T09:25:01.706762+0000","last_unstale":"2024-09-16T09:31:53.328319+0000","last_undegraded":"2024-09-16T09:31:53.328319+0000","last_fullsized":"2024-09-16T09:31:53.328319+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:23.796231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:23.796231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":140,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.326896+0000","last_change":"2024-09-16T09:28:52.983144+0000","last_active":"2024-09-16T09:31:53.326896+0000","last_peered":"2024-09-16T09:31:53.326896+0000","last_clean":"2024-09-16T09:31:53.326896+0000","last_became_active":"2024-09-16T09:28:52.982865+0000","last_became_peered":"2024-09-16T09:28:52.982865+0000","last_unstale":"2024-09-16T09:31:53.326896+0000","last_undegraded":"2024-09-16T09:31:53.326896+0000","last_fullsized":"2024-09-16T09:31:53.326896+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:44.241942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:44.241942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:09.215097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":300,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.318919+0000","last_change":"2024-09-16T09:28:52.552567+0000","last_active":"2024-09-16T09:31:53.318919+0000","last_peered":"2024-09-16T09:31:53.318919+0000","last_clean":"2024-09-16T09:31:53.318919+0000","last_became_active":"2024-09-16T09:25:07.715301+0000","last_became_peered":"2024-09-16T09:25:07.715301+0000","last_unstale":"2024-09-16T09:31:53.318919+0000","last_undegraded":"2024-09-16T09:31:53.318919+0000","last_fullsized":"2024-09-16T09:31:53.318919+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:52.552495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:28:52.552495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:45.269179+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.14","version":"73'79","reported_seq":98,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.309992+0000","last_change":"2024-09-16T09:31:47.253568+0000","last_active":"2024-09-16T09:31:53.309992+0000","last_peered":"2024-09-16T09:31:53.309992+0000","last_clean":"2024-09-16T09:31:53.309992+0000","last_became_active":"2024-09-16T09:30:10.700866+0000","last_became_peered":"2024-09-16T09:30:10.700866+0000","last_unstale":"2024-09-16T09:31:53.309992+0000","last_undegraded":"2024-09-16T09:31:53.309992+0000","last_fullsized":"2024-09-16T09:31:53.309992+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:53.916505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:53.916505+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:32:13.898748+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109227,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"66'11","reported_seq":187,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.325865+0000","last_change":"2024-09-16T09:28:55.643505+0000","last_active":"2024-09-16T09:31:53.325865+0000","last_peered":"2024-09-16T09:31:53.325865+0000","last_clean":"2024-09-16T09:31:53.325865+0000","last_became_active":"2024-09-16T09:28:52.971132+0000","last_became_peered":"2024-09-16T09:28:52.971132+0000","last_unstale":"2024-09-16T09:31:53.325865+0000","last_undegraded":"2024-09-16T09:31:53.325865+0000","last_fullsized":"2024-09-16T09:31:53.325865+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-16T09:28:55.643448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:55.643448+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":19,"num_read_kb":12,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":311,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.320202+0000","last_change":"2024-09-16T09:27:46.175159+0000","last_active":"2024-09-16T09:31:53.320202+0000","last_peered":"2024-09-16T09:31:53.320202+0000","last_clean":"2024-09-16T09:31:53.320202+0000","last_became_active":"2024-09-16T09:25:01.700733+0000","last_became_peered":"2024-09-16T09:25:01.700733+0000","last_unstale":"2024-09-16T09:31:53.320202+0000","last_undegraded":"2024-09-16T09:31:53.320202+0000","last_fullsized":"2024-09-16T09:31:53.320202+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:46.175093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:27:46.175093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":303,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.324830+0000","last_change":"2024-09-16T09:29:25.017736+0000","last_active":"2024-09-16T09:31:53.324830+0000","last_peered":"2024-09-16T09:31:53.324830+0000","last_clean":"2024-09-16T09:31:53.324830+0000","last_became_active":"2024-09-16T09:25:05.713479+0000","last_became_peered":"2024-09-16T09:25:05.713479+0000","last_unstale":"2024-09-16T09:31:53.324830+0000","last_undegraded":"2024-09-16T09:31:53.324830+0000","last_fullsized":"2024-09-16T09:31:53.324830+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:25.017610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:25.017610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":163,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.327229+0000","last_change":"2024-09-16T09:31:09.275121+0000","last_active":"2024-09-16T09:31:53.327229+0000","last_peered":"2024-09-16T09:31:53.327229+0000","last_clean":"2024-09-16T09:31:53.327229+0000","last_became_active":"2024-09-16T09:31:09.274797+0000","last_became_peered":"2024-09-16T09:31:09.274797+0000","last_unstale":"2024-09-16T09:31:53.327229+0000","last_undegraded":"2024-09-16T09:31:53.327229+0000","last_fullsized":"2024-09-16T09:31:53.327229+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:54.589258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:28:54.589258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:48.251310+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"66'4","reported_seq":44,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.327298+0000","last_change":"2024-09-16T09:31:09.401626+0000","last_active":"2024-09-16T09:31:53.327298+0000","last_peered":"2024-09-16T09:31:53.327298+0000","last_clean":"2024-09-16T09:31:53.327298+0000","last_became_active":"2024-09-16T09:31:09.273682+0000","last_became_peered":"2024-09-16T09:31:09.273682+0000","last_unstale":"2024-09-16T09:31:53.327298+0000","last_undegraded":"2024-09-16T09:31:53.327298+0000","last_fullsized":"2024-09-16T09:31:53.327298+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'4","last_scrub_stamp":"2024-09-16T09:26:27.473740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:26:27.473740+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":334,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.328267+0000","last_change":"2024-09-16T09:31:30.235761+0000","last_active":"2024-09-16T09:31:53.328267+0000","last_peered":"2024-09-16T09:31:53.328267+0000","last_clean":"2024-09-16T09:31:53.328267+0000","last_became_active":"2024-09-16T09:31:09.274719+0000","last_became_peered":"2024-09-16T09:31:09.274719+0000","last_unstale":"2024-09-16T09:31:53.328267+0000","last_undegraded":"2024-09-16T09:31:53.328267+0000","last_fullsized":"2024-09-16T09:31:53.328267+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:31:30.235692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:31:30.235692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:49.736157+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":312,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.318920+0000","last_change":"2024-09-16T09:29:53.244633+0000","last_active":"2024-09-16T09:31:53.318920+0000","last_peered":"2024-09-16T09:31:53.318920+0000","last_clean":"2024-09-16T09:31:53.318920+0000","last_became_active":"2024-09-16T09:25:05.712545+0000","last_became_peered":"2024-09-16T09:25:05.712545+0000","last_unstale":"2024-09-16T09:31:53.318920+0000","last_undegraded":"2024-09-16T09:31:53.318920+0000","last_fullsized":"2024-09-16T09:31:53.318920+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:53.244578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:53.244578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:07.420423+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":299,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.325043+0000","last_change":"2024-09-16T09:29:27.012810+0000","last_active":"2024-09-16T09:31:53.325043+0000","last_peered":"2024-09-16T09:31:53.325043+0000","last_clean":"2024-09-16T09:31:53.325043+0000","last_became_active":"2024-09-16T09:25:07.720117+0000","last_became_peered":"2024-09-16T09:25:07.720117+0000","last_unstale":"2024-09-16T09:31:53.325043+0000","last_undegraded":"2024-09-16T09:31:53.325043+0000","last_fullsized":"2024-09-16T09:31:53.325043+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:27.012756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:27.012756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":300,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.317644+0000","last_change":"2024-09-16T09:29:19.888825+0000","last_active":"2024-09-16T09:31:53.317644+0000","last_peered":"2024-09-16T09:31:53.317644+0000","last_clean":"2024-09-16T09:31:53.317644+0000","last_became_active":"2024-09-16T09:25:07.712594+0000","last_became_peered":"2024-09-16T09:25:07.712594+0000","last_unstale":"2024-09-16T09:31:53.317644+0000","last_undegraded":"2024-09-16T09:31:53.317644+0000","last_fullsized":"2024-09-16T09:31:53.317644+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:19.888759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:19.888759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":162,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.319521+0000","last_change":"2024-09-16T09:31:26.211625+0000","last_active":"2024-09-16T09:31:53.319521+0000","last_peered":"2024-09-16T09:31:53.319521+0000","last_clean":"2024-09-16T09:31:53.319521+0000","last_became_active":"2024-09-16T09:31:09.271662+0000","last_became_peered":"2024-09-16T09:31:09.271662+0000","last_unstale":"2024-09-16T09:31:53.319521+0000","last_undegraded":"2024-09-16T09:31:53.319521+0000","last_fullsized":"2024-09-16T09:31:53.319521+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:31:26.211363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:31:26.211363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:32.873753+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"66'11","reported_seq":328,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.320107+0000","last_change":"2024-09-16T09:27:57.486459+0000","last_active":"2024-09-16T09:31:53.320107+0000","last_peered":"2024-09-16T09:31:53.320107+0000","last_clean":"2024-09-16T09:31:53.320107+0000","last_became_active":"2024-09-16T09:25:03.707769+0000","last_became_peered":"2024-09-16T09:25:03.707769+0000","last_unstale":"2024-09-16T09:31:53.320107+0000","last_undegraded":"2024-09-16T09:31:53.320107+0000","last_fullsized":"2024-09-16T09:31:53.320107+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-16T09:27:57.486381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:57.486381+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:16.375833+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":297,"reported_epoch":200,"state":"active+clean","last_fresh":"2024-09-16T09:31:53.327937+0000","last_change":"2024-09-16T09:27:57.640962+0000","last_active":"2024-09-16T09:31:53.327937+0000","last_peered":"2024-09-16T09:31:53.327937+0000","last_clean":"2024-09-16T09:31:53.327937+0000","last_became_active":"2024-09-16T09:25:05.711952+0000","last_became_peered":"2024-09-16T09:25:05.711952+0000","last_unstale":"2024-09-16T09:31:53.327937+0000","last_undegraded":"2024-09-16T09:31:53.327937+0000","last_fullsized":"2024-09-16T09:31:53.327937+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:57.640756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:57.640756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:21.189003+0000","scrub_duration":44,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":16,"stat_sum":{"num_bytes":5856296984,"num_objects":89361,"num_object_clones":0,"num_object_copies":268083,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":5350,"num_objects_unfound":0,"num_objects_dirty":89361,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":159825,"num_write_kb":7973889,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9359,"num_bytes_recovered":613351424,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":18188611584,"data_stored":18188599368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":77312,"ondisk_log_size":77312,"up":48,"acting":45,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":70,"ondisk_log_size":70,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":691,"num_read_kb":448,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":126,"num_bytes_recovered":3026,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":395,"ondisk_log_size":395,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1986,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":410,"num_read_kb":405,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":21,"stat_sum":{"num_bytes":806935,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":21,"num_read_kb":40,"num_write":21,"num_write_kb":296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":697708,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1659,"ondisk_log_size":1659,"up":63,"acting":63,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266776,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1980800,"kb_used_data":1809392,"kb_used_omap":1,"kb_used_meta":171390,"kb_avail":91764352,"statfs":{"total":95995035648,"available":93966696448,"internally_reserved":0,"allocated":1852817408,"data_stored":1852491356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":175503818},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495709,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1961320,"kb_used_data":1803800,"kb_used_omap":1,"kb_used_meta":157502,"kb_avail":91783832,"statfs":{"total":95995035648,"available":93986643968,"internally_reserved":0,"allocated":1847091200,"data_stored":1846789146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":161282502},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724644,"num_pgs":64,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3120436,"kb_used_data":2872676,"kb_used_omap":1,"kb_used_meta":247742,"kb_avail":90624716,"statfs":{"total":95995035648,"available":92799709184,"internally_reserved":0,"allocated":2941620224,"data_stored":2941306375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":253688262},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953577,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2310784,"kb_used_data":2113008,"kb_used_omap":1,"kb_used_meta":197758,"kb_avail":91434368,"statfs":{"total":95995035648,"available":93628792832,"internally_reserved":0,"allocated":2163720192,"data_stored":2163394180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":202504646},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149807,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2136328,"kb_used_data":1808056,"kb_used_omap":1,"kb_used_meta":328254,"kb_avail":91608824,"statfs":{"total":95995035648,"available":93807435776,"internally_reserved":0,"allocated":1851449344,"data_stored":1851115426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":336132551},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,0,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411444,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2816876,"kb_used_data":2537180,"kb_used_omap":1,"kb_used_meta":279678,"kb_avail":90928276,"statfs":{"total":95995035648,"available":93110554624,"internally_reserved":0,"allocated":2598072320,"data_stored":2597766126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":286390725},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574971,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2816088,"kb_used_data":2528072,"kb_used_omap":1,"kb_used_meta":287998,"kb_avail":90929064,"statfs":{"total":95995035648,"available":93111361536,"internally_reserved":0,"allocated":2588745728,"data_stored":2588395010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":294910405},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738497,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2731264,"kb_used_data":2433520,"kb_used_omap":1,"kb_used_meta":297726,"kb_avail":91013888,"statfs":{"total":95995035648,"available":93198221312,"internally_reserved":0,"allocated":2491924480,"data_stored":2491598670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":304871877},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2468216832,"data_stored":2468216832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2561081344,"data_stored":2561081344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2568491008,"data_stored":2568486936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1841758208,"data_stored":1841758208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":2137391104,"data_stored":2137391104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":2919895040,"data_stored":2919890968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1842872320,"data_stored":1842872320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1848905728,"data_stored":1848901656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T09:32:02.127 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph status --format=json 2024-09-16T09:32:02.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:02.159 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:02.159 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:32:02.199 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.7 dump_historic_ops 2024-09-16T09:32:02.227 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:32:01 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:32:01] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:32:02.260 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:32:02.320 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:02.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:02.321 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:32:02.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:02 smithi084 ceph-mon[32629]: from='client.25331 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:32:02.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:02 smithi084 ceph-mon[32629]: 4.5 scrub starts 2024-09-16T09:32:02.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:02 smithi084 ceph-mon[32629]: 4.5 scrub ok 2024-09-16T09:32:02.422 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:32:02.454 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:02.454 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:02.455 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:32:02.556 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:32:02.574 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:02 smithi070 ceph-mon[30388]: from='client.25331 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:32:02.574 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:02 smithi070 ceph-mon[30388]: 4.5 scrub starts 2024-09-16T09:32:02.574 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:02 smithi070 ceph-mon[30388]: 4.5 scrub ok 2024-09-16T09:32:02.574 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:02 smithi070 ceph-mon[40691]: from='client.25331 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:32:02.574 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:02 smithi070 ceph-mon[40691]: 4.5 scrub starts 2024-09-16T09:32:02.574 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:02 smithi070 ceph-mon[40691]: 4.5 scrub ok 2024-09-16T09:32:02.601 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:02.601 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:32:02.602 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:02.602 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:32:02.703 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:32:02.734 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:02.735 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:02.735 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:32:02.746 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:32:02.746 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:32:02.781 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:32:02.782 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:32:02.817 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:32:02.817 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:32:02.836 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:32:02.842 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:32:02.852 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:32:02.853 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:32:02.888 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:32:02.888 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:32:02.911 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:02.912 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:02.912 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:32:02.922 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:32:02.922 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:32:02.953 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:32:02.953 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:32:02.988 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:32:02.988 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:32:03.014 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:32:03.039 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:32:02 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:32:03.049 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:03.050 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:03.050 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:32:03.151 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:32:03.181 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:03.182 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:03.182 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:32:03.283 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:32:03.294 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd set noscrub 2024-09-16T09:32:03.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:03.321 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:03.322 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:32:03.423 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:32:03.457 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:03.458 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:03.458 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:32:03.493 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-09-16T09:32:03.493 INFO:teuthology.orchestra.run.smithi084.stdout: "size": 20, 2024-09-16T09:32:03.493 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 600, 2024-09-16T09:32:03.493 INFO:teuthology.orchestra.run.smithi084.stdout: "ops": [ 2024-09-16T09:32:03.493 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.493 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:22924 7.5 7:a7d40976:::benchmark_data_smithi070_78438_object22923:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T09:32:03.493 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:26:21.580769+0000", 2024-09-16T09:32:03.493 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 341.90688414700003, 2024-09-16T09:32:03.494 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.079602870000000006, 2024-09-16T09:32:03.494 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:32:03.494 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:32:03.494 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:32:03.494 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:32:03.494 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:32:03.494 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:32:03.494 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 22924 2024-09-16T09:32:03.494 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.494 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:32:03.494 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.495 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:32:03.495 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.580769+0000", 2024-09-16T09:32:03.495 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:32:03.495 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.495 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.495 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:32:03.495 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.580769+0000", 2024-09-16T09:32:03.495 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:32:03.495 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.495 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.495 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:32:03.495 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.580770+0000", 2024-09-16T09:32:03.495 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.3200000000000001e-06 2024-09-16T09:32:03.496 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.496 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.496 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:32:03.496 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.580848+0000", 2024-09-16T09:32:03.496 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.7514000000000004e-05 2024-09-16T09:32:03.496 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.496 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.496 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:32:03.496 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.580867+0000", 2024-09-16T09:32:03.496 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.8782000000000001e-05 2024-09-16T09:32:03.496 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.496 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.496 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:32:03.497 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.580874+0000", 2024-09-16T09:32:03.497 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.1269999999999997e-06 2024-09-16T09:32:03.497 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.497 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.497 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:32:03.497 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.580941+0000", 2024-09-16T09:32:03.497 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.7010999999999999e-05 2024-09-16T09:32:03.497 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.497 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.497 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:32:03.497 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.581054+0000", 2024-09-16T09:32:03.497 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000113509 2024-09-16T09:32:03.497 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.497 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.498 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,2", 2024-09-16T09:32:03.498 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.581176+0000", 2024-09-16T09:32:03.498 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000122187 2024-09-16T09:32:03.498 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.498 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.498 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:32:03.498 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.582818+0000", 2024-09-16T09:32:03.498 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0016419200000000001 2024-09-16T09:32:03.499 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.499 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.499 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:32:03.499 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.583304+0000", 2024-09-16T09:32:03.499 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00048596099999999998 2024-09-16T09:32:03.499 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.499 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.499 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:32:03.499 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.660281+0000", 2024-09-16T09:32:03.499 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.076976634000000002 2024-09-16T09:32:03.499 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.499 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.499 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:32:03.500 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.660322+0000", 2024-09-16T09:32:03.500 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.1125999999999999e-05 2024-09-16T09:32:03.500 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.500 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.500 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:32:03.500 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.660372+0000", 2024-09-16T09:32:03.500 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.9778999999999999e-05 2024-09-16T09:32:03.500 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:32:03.500 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:32:03.500 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:32:03.500 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.500 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.500 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:23476 7.5 7:ab69eea0:::benchmark_data_smithi070_78438_object23475:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T09:32:03.501 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:26:21.880371+0000", 2024-09-16T09:32:03.501 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 341.60728181399998, 2024-09-16T09:32:03.501 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.069588092000000004, 2024-09-16T09:32:03.501 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:32:03.501 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:32:03.501 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:32:03.501 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:32:03.501 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:32:03.501 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:32:03.501 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 23476 2024-09-16T09:32:03.501 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.501 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:32:03.501 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.502 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:32:03.502 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.880371+0000", 2024-09-16T09:32:03.502 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:32:03.502 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.502 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.502 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:32:03.502 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.880371+0000", 2024-09-16T09:32:03.502 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:32:03.502 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.502 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.502 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:32:03.502 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.880372+0000", 2024-09-16T09:32:03.502 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.4799999999999997e-07 2024-09-16T09:32:03.502 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.503 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.503 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:32:03.503 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.880461+0000", 2024-09-16T09:32:03.503 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.9232000000000002e-05 2024-09-16T09:32:03.503 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.503 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.503 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:32:03.503 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.880473+0000", 2024-09-16T09:32:03.503 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1687e-05 2024-09-16T09:32:03.503 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.504 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.504 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:32:03.504 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.880480+0000", 2024-09-16T09:32:03.504 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.3000000000000004e-06 2024-09-16T09:32:03.504 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.504 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.504 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:32:03.504 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.881388+0000", 2024-09-16T09:32:03.504 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00090752200000000002 2024-09-16T09:32:03.504 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.504 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.504 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:32:03.504 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.881520+0000", 2024-09-16T09:32:03.504 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00013243400000000001 2024-09-16T09:32:03.505 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.505 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.505 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,2", 2024-09-16T09:32:03.505 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.881649+0000", 2024-09-16T09:32:03.505 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00012842299999999999 2024-09-16T09:32:03.505 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.505 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.505 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:32:03.505 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.882506+0000", 2024-09-16T09:32:03.505 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000857047 2024-09-16T09:32:03.505 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.505 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.505 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:32:03.506 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.884941+0000", 2024-09-16T09:32:03.506 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0024354149999999998 2024-09-16T09:32:03.506 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.506 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.506 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:32:03.506 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.949871+0000", 2024-09-16T09:32:03.506 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.064929929999999997 2024-09-16T09:32:03.506 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.506 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.506 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:32:03.506 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.949921+0000", 2024-09-16T09:32:03.506 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.9660999999999998e-05 2024-09-16T09:32:03.506 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.507 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.507 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:32:03.507 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.949959+0000", 2024-09-16T09:32:03.507 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.8692999999999997e-05 2024-09-16T09:32:03.507 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:32:03.507 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:32:03.508 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:32:03.508 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.508 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.508 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:23481 7.5 7:a58daa7e:::benchmark_data_smithi070_78438_object23480:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T09:32:03.508 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:26:21.883877+0000", 2024-09-16T09:32:03.508 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 341.60377562600002, 2024-09-16T09:32:03.508 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.067196770000000003, 2024-09-16T09:32:03.508 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:32:03.508 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:32:03.508 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:32:03.508 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:32:03.508 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:32:03.508 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:32:03.508 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 23481 2024-09-16T09:32:03.509 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.509 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:32:03.509 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.509 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:32:03.509 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.883877+0000", 2024-09-16T09:32:03.509 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:32:03.509 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.509 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.509 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:32:03.509 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.883877+0000", 2024-09-16T09:32:03.509 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:32:03.509 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.509 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.510 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:32:03.510 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.883879+0000", 2024-09-16T09:32:03.510 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.144e-06 2024-09-16T09:32:03.510 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.510 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.510 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:32:03.510 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.883931+0000", 2024-09-16T09:32:03.510 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.2123000000000002e-05 2024-09-16T09:32:03.510 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.510 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.510 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:32:03.510 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.883943+0000", 2024-09-16T09:32:03.510 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.2323999999999999e-05 2024-09-16T09:32:03.511 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.511 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.511 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:32:03.511 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.883950+0000", 2024-09-16T09:32:03.511 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.3900000000000004e-06 2024-09-16T09:32:03.511 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.511 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.511 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:32:03.511 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.884006+0000", 2024-09-16T09:32:03.511 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.5387999999999999e-05 2024-09-16T09:32:03.511 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.511 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.511 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:32:03.512 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.884117+0000", 2024-09-16T09:32:03.512 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000111461 2024-09-16T09:32:03.512 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.512 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.512 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,2", 2024-09-16T09:32:03.512 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.884259+0000", 2024-09-16T09:32:03.512 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000141686 2024-09-16T09:32:03.512 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.513 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.513 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:32:03.513 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.885063+0000", 2024-09-16T09:32:03.514 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00080437399999999995 2024-09-16T09:32:03.514 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.514 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.514 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:32:03.514 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.886156+0000", 2024-09-16T09:32:03.514 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0010922130000000001 2024-09-16T09:32:03.514 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.514 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.514 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:32:03.514 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.950978+0000", 2024-09-16T09:32:03.514 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.064822639000000001 2024-09-16T09:32:03.514 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.514 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.515 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:32:03.515 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.951036+0000", 2024-09-16T09:32:03.515 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.7666e-05 2024-09-16T09:32:03.515 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.515 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.515 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:32:03.515 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.951074+0000", 2024-09-16T09:32:03.515 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.8362000000000003e-05 2024-09-16T09:32:03.515 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:32:03.515 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:32:03.515 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:32:03.515 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.515 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.515 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_repop(client.24698.0:27466 7.6 e89/82 7:60c44c8b:::benchmark_data_smithi070_78438_object27465:head v 89'1726, mlcod=89'1723)", 2024-09-16T09:32:03.516 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:26:23.722364+0000", 2024-09-16T09:32:03.516 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 339.76528925899999, 2024-09-16T09:32:03.516 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.071550814000000004, 2024-09-16T09:32:03.516 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:32:03.516 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:32:03.516 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:32:03.516 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:32:03.516 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.516 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:32:03.516 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.722364+0000", 2024-09-16T09:32:03.516 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:32:03.516 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.516 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.517 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:32:03.517 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.722364+0000", 2024-09-16T09:32:03.517 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:32:03.517 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.517 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.517 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:32:03.517 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.722364+0000", 2024-09-16T09:32:03.517 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.0699999999999998e-07 2024-09-16T09:32:03.517 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.517 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.517 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:32:03.517 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.722426+0000", 2024-09-16T09:32:03.517 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.1278999999999994e-05 2024-09-16T09:32:03.518 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.518 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.518 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:32:03.518 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.722432+0000", 2024-09-16T09:32:03.518 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.003e-06 2024-09-16T09:32:03.518 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.518 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.518 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:32:03.518 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.722435+0000", 2024-09-16T09:32:03.518 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.4010000000000001e-06 2024-09-16T09:32:03.518 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.518 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.519 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:32:03.519 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.722461+0000", 2024-09-16T09:32:03.519 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.5894999999999999e-05 2024-09-16T09:32:03.519 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.519 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.519 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:32:03.519 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.722471+0000", 2024-09-16T09:32:03.519 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0251000000000001e-05 2024-09-16T09:32:03.519 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.519 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.519 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:32:03.519 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.793863+0000", 2024-09-16T09:32:03.520 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.071392294999999995 2024-09-16T09:32:03.520 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.520 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.520 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:32:03.520 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.793915+0000", 2024-09-16T09:32:03.520 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.1282999999999997e-05 2024-09-16T09:32:03.520 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:32:03.520 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:32:03.520 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:32:03.520 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.520 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.520 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_repop(client.24698.0:27468 7.8 e89/82 7:1de713de:::benchmark_data_smithi070_78438_object27467:head v 89'1721, mlcod=89'1718)", 2024-09-16T09:32:03.521 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:26:23.722887+0000", 2024-09-16T09:32:03.521 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 339.76476623899998, 2024-09-16T09:32:03.521 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.069612414999999997, 2024-09-16T09:32:03.521 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:32:03.521 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:32:03.521 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:32:03.521 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:32:03.521 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.521 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:32:03.521 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.722887+0000", 2024-09-16T09:32:03.521 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:32:03.521 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.521 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.522 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:32:03.522 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.722887+0000", 2024-09-16T09:32:03.522 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:32:03.522 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.522 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.522 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:32:03.522 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.722887+0000", 2024-09-16T09:32:03.522 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.0100000000000005e-07 2024-09-16T09:32:03.522 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.522 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.522 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:32:03.522 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.722949+0000", 2024-09-16T09:32:03.522 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.1298999999999997e-05 2024-09-16T09:32:03.523 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.523 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.523 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:32:03.523 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.790454+0000", 2024-09-16T09:32:03.523 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.067505342999999995 2024-09-16T09:32:03.523 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.524 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.524 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:32:03.524 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.790465+0000", 2024-09-16T09:32:03.524 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1242e-05 2024-09-16T09:32:03.524 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.524 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.524 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:32:03.524 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.790558+0000", 2024-09-16T09:32:03.524 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.3227999999999996e-05 2024-09-16T09:32:03.525 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.525 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.525 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:32:03.525 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.790578+0000", 2024-09-16T09:32:03.525 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.9114999999999999e-05 2024-09-16T09:32:03.525 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.525 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.525 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:32:03.525 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.792380+0000", 2024-09-16T09:32:03.526 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:03 smithi070 ceph-mon[30388]: pgmap v326: 168 pgs: 1 active+recovery_wait+degraded, 3 active+recovering+undersized+remapped, 164 active+clean; 5.5 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 615 B/s rd, 0 B/s wr, 698 op/s; 1/268683 objects degraded (0.000%); 5350/268683 objects misplaced (1.991%); 4.9 MiB/s, 78 objects/s recovering 2024-09-16T09:32:03.526 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:03 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:32:03.526 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:03 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:32:03.526 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:03 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:32:03.526 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:03 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:32:03.526 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:03 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:32:03.527 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:03 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:32:03.527 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:03 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:32:03.527 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:03 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/813645725' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:32:03.527 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:03 smithi070 ceph-mon[40691]: pgmap v326: 168 pgs: 1 active+recovery_wait+degraded, 3 active+recovering+undersized+remapped, 164 active+clean; 5.5 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 615 B/s rd, 0 B/s wr, 698 op/s; 1/268683 objects degraded (0.000%); 5350/268683 objects misplaced (1.991%); 4.9 MiB/s, 78 objects/s recovering 2024-09-16T09:32:03.527 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:03 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:32:03.527 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:03 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:32:03.527 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:03 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:32:03.527 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:03 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:32:03.527 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:03 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:32:03.528 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:03 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:32:03.528 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:03 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:32:03.528 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:03 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/813645725' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:32:03.528 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0018023329999999999 2024-09-16T09:32:03.528 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.528 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.528 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:32:03.528 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.792499+0000", 2024-09-16T09:32:03.529 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000119354 2024-09-16T09:32:03.529 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:32:03.529 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:32:03.529 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:32:03.529 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.529 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.529 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:27473 7.5 7:a272c321:::benchmark_data_smithi070_78438_object27472:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T09:32:03.529 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:26:23.723821+0000", 2024-09-16T09:32:03.529 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 339.76383164800001, 2024-09-16T09:32:03.529 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.073668624000000002, 2024-09-16T09:32:03.529 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:32:03.529 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:32:03.529 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:32:03.529 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:32:03.530 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:32:03.530 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:32:03.530 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 27473 2024-09-16T09:32:03.530 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.530 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:32:03.530 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.530 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:32:03.530 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.723821+0000", 2024-09-16T09:32:03.530 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:32:03.530 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.530 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.530 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:32:03.530 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.723821+0000", 2024-09-16T09:32:03.530 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:32:03.531 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.531 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.531 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:32:03.531 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.723823+0000", 2024-09-16T09:32:03.531 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1999999999999999e-06 2024-09-16T09:32:03.531 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.531 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.531 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:32:03.531 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.723884+0000", 2024-09-16T09:32:03.531 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.0971000000000003e-05 2024-09-16T09:32:03.531 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.531 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.531 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:32:03.532 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.790196+0000", 2024-09-16T09:32:03.532 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.066312372999999994 2024-09-16T09:32:03.532 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.532 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.532 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:32:03.532 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.790213+0000", 2024-09-16T09:32:03.532 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.6974e-05 2024-09-16T09:32:03.532 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.532 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.532 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:32:03.532 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.790559+0000", 2024-09-16T09:32:03.532 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00034600599999999999 2024-09-16T09:32:03.532 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.532 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.533 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:32:03.533 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.790669+0000", 2024-09-16T09:32:03.533 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000110276 2024-09-16T09:32:03.533 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.533 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.533 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,2", 2024-09-16T09:32:03.533 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.790788+0000", 2024-09-16T09:32:03.533 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000119096 2024-09-16T09:32:03.533 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.533 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.533 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:32:03.533 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.793102+0000", 2024-09-16T09:32:03.533 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0023135030000000002 2024-09-16T09:32:03.533 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.534 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.534 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:32:03.534 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.793337+0000", 2024-09-16T09:32:03.534 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000235481 2024-09-16T09:32:03.534 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.534 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.534 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:32:03.534 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.797381+0000", 2024-09-16T09:32:03.534 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0040435699999999998 2024-09-16T09:32:03.534 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.534 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.534 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:32:03.534 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.797446+0000", 2024-09-16T09:32:03.534 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.4759999999999997e-05 2024-09-16T09:32:03.535 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.535 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.535 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:32:03.535 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.797490+0000", 2024-09-16T09:32:03.535 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.4413999999999999e-05 2024-09-16T09:32:03.535 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:32:03.535 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:32:03.535 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:32:03.535 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.535 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.535 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_repop(client.24698.0:28810 7.6 e90/82 7:63b6347d:::benchmark_data_smithi070_78438_object28809:head v 90'1809, mlcod=90'1805)", 2024-09-16T09:32:03.535 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:26:24.482328+0000", 2024-09-16T09:32:03.535 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 339.00532480999999, 2024-09-16T09:32:03.536 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.063714949000000007, 2024-09-16T09:32:03.536 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:32:03.536 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:32:03.536 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:32:03.536 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:32:03.536 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.536 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:32:03.536 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.482328+0000", 2024-09-16T09:32:03.536 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:32:03.536 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.536 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.537 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:32:03.537 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.482328+0000", 2024-09-16T09:32:03.537 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:32:03.537 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.537 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.537 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:32:03.537 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.482329+0000", 2024-09-16T09:32:03.537 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.0200000000000002e-07 2024-09-16T09:32:03.537 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.537 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.537 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:32:03.537 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.482394+0000", 2024-09-16T09:32:03.537 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.4706e-05 2024-09-16T09:32:03.537 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.537 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.538 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:32:03.538 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.543399+0000", 2024-09-16T09:32:03.538 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.061005750999999997 2024-09-16T09:32:03.538 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.538 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.538 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:32:03.538 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.543411+0000", 2024-09-16T09:32:03.538 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1687e-05 2024-09-16T09:32:03.538 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.538 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.538 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:32:03.538 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.543539+0000", 2024-09-16T09:32:03.538 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000128298 2024-09-16T09:32:03.538 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.539 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.539 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:32:03.539 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.543558+0000", 2024-09-16T09:32:03.539 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.8323999999999999e-05 2024-09-16T09:32:03.539 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.539 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.539 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:32:03.539 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.546006+0000", 2024-09-16T09:32:03.539 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0024483180000000001 2024-09-16T09:32:03.539 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.539 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.539 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:32:03.539 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.546043+0000", 2024-09-16T09:32:03.539 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.7262999999999999e-05 2024-09-16T09:32:03.539 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:32:03.540 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:32:03.540 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:32:03.540 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.540 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.540 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:62033 7.5 7:a5dfc1ea:::benchmark_data_smithi070_78438_object62032:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-16T09:32:03.540 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:26:45.811217+0000", 2024-09-16T09:32:03.540 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 317.67643628399998, 2024-09-16T09:32:03.540 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.071578969000000006, 2024-09-16T09:32:03.540 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:32:03.540 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:32:03.540 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:32:03.540 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:32:03.540 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:32:03.541 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:32:03.541 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 62033 2024-09-16T09:32:03.541 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.541 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:32:03.541 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.541 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:32:03.541 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.811217+0000", 2024-09-16T09:32:03.541 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:32:03.541 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.541 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.541 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:32:03.541 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.811217+0000", 2024-09-16T09:32:03.541 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:32:03.542 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.542 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.542 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:32:03.542 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.811218+0000", 2024-09-16T09:32:03.542 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.5290000000000001e-06 2024-09-16T09:32:03.542 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.542 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.542 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:32:03.542 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.811262+0000", 2024-09-16T09:32:03.542 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.3850999999999997e-05 2024-09-16T09:32:03.542 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.542 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.542 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:32:03.543 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.811278+0000", 2024-09-16T09:32:03.543 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.5753999999999999e-05 2024-09-16T09:32:03.543 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.543 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.543 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:32:03.543 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.811286+0000", 2024-09-16T09:32:03.543 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.7519999999999996e-06 2024-09-16T09:32:03.543 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.543 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.543 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:32:03.543 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.811333+0000", 2024-09-16T09:32:03.544 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.7203000000000002e-05 2024-09-16T09:32:03.544 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.544 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.544 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:32:03.544 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.811456+0000", 2024-09-16T09:32:03.544 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00012353900000000001 2024-09-16T09:32:03.544 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.544 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.544 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,2", 2024-09-16T09:32:03.544 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.811644+0000", 2024-09-16T09:32:03.544 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00018799100000000001 2024-09-16T09:32:03.544 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.544 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.545 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:32:03.545 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.814035+0000", 2024-09-16T09:32:03.545 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0023902009999999998 2024-09-16T09:32:03.545 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.545 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.545 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:32:03.545 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.815646+0000", 2024-09-16T09:32:03.545 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0016112399999999999 2024-09-16T09:32:03.545 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.545 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.545 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:32:03.545 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.882683+0000", 2024-09-16T09:32:03.545 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.067037332000000005 2024-09-16T09:32:03.545 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.546 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.546 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:32:03.546 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.882753+0000", 2024-09-16T09:32:03.546 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.9534000000000002e-05 2024-09-16T09:32:03.546 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.546 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.546 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:32:03.546 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.882796+0000", 2024-09-16T09:32:03.546 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.3043000000000001e-05 2024-09-16T09:32:03.546 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:32:03.546 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:32:03.547 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:32:03.547 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.547 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.547 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:62044 7.5 7:aee12d22:::benchmark_data_smithi070_78438_object62043:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-16T09:32:03.547 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:26:45.819031+0000", 2024-09-16T09:32:03.547 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 317.66862242399998, 2024-09-16T09:32:03.547 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.063892420000000005, 2024-09-16T09:32:03.547 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:32:03.547 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:32:03.547 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:32:03.547 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:32:03.547 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:32:03.547 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:32:03.548 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 62044 2024-09-16T09:32:03.548 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.548 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:32:03.548 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.548 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:32:03.548 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.819031+0000", 2024-09-16T09:32:03.548 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:32:03.548 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.548 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.548 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:32:03.548 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.819031+0000", 2024-09-16T09:32:03.548 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:32:03.548 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.548 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.549 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:32:03.549 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.819032+0000", 2024-09-16T09:32:03.549 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.3120000000000001e-06 2024-09-16T09:32:03.549 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.549 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.549 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:32:03.549 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.819071+0000", 2024-09-16T09:32:03.549 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.8599000000000001e-05 2024-09-16T09:32:03.549 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.549 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.549 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:32:03.549 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.819081+0000", 2024-09-16T09:32:03.550 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0812000000000001e-05 2024-09-16T09:32:03.550 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.550 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.550 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:32:03.550 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.819089+0000", 2024-09-16T09:32:03.550 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.8089999999999993e-06 2024-09-16T09:32:03.550 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.550 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.550 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:32:03.550 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.819138+0000", 2024-09-16T09:32:03.550 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.8361e-05 2024-09-16T09:32:03.550 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.550 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.551 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:32:03.551 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.819248+0000", 2024-09-16T09:32:03.551 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000110819 2024-09-16T09:32:03.551 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.551 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.551 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,2", 2024-09-16T09:32:03.551 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.819399+0000", 2024-09-16T09:32:03.551 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000150674 2024-09-16T09:32:03.551 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.551 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.551 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:32:03.551 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.820979+0000", 2024-09-16T09:32:03.551 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0015803589999999999 2024-09-16T09:32:03.551 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.551 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.552 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:32:03.552 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.821238+0000", 2024-09-16T09:32:03.552 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00025873699999999999 2024-09-16T09:32:03.552 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.552 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.552 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:32:03.552 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.882832+0000", 2024-09-16T09:32:03.552 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.061593649 2024-09-16T09:32:03.552 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.552 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.552 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:32:03.552 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.882879+0000", 2024-09-16T09:32:03.552 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.6879000000000003e-05 2024-09-16T09:32:03.552 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.553 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.553 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:32:03.553 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.882923+0000", 2024-09-16T09:32:03.553 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.4409999999999997e-05 2024-09-16T09:32:03.553 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:32:03.553 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:32:03.553 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:32:03.553 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.553 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.553 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:63392 7.5 7:af2804a3:::benchmark_data_smithi070_78438_object63391:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-16T09:32:03.553 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:26:46.805497+0000", 2024-09-16T09:32:03.553 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 316.68215629399998, 2024-09-16T09:32:03.553 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.064734542000000006, 2024-09-16T09:32:03.553 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:32:03.554 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:32:03.554 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:32:03.554 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:32:03.554 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:32:03.554 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:32:03.554 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 63392 2024-09-16T09:32:03.554 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.554 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:32:03.554 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.554 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:32:03.554 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:46.805497+0000", 2024-09-16T09:32:03.554 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:32:03.554 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.554 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.555 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:32:03.555 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:46.805497+0000", 2024-09-16T09:32:03.555 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:32:03.555 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.555 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.555 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:32:03.555 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:46.805498+0000", 2024-09-16T09:32:03.555 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.6789999999999999e-06 2024-09-16T09:32:03.555 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.555 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.555 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:32:03.555 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:46.805542+0000", 2024-09-16T09:32:03.555 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.3123e-05 2024-09-16T09:32:03.555 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.556 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.556 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:32:03.556 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:46.805555+0000", 2024-09-16T09:32:03.556 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.3777e-05 2024-09-16T09:32:03.556 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.556 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.556 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:32:03.556 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:46.805563+0000", 2024-09-16T09:32:03.556 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.4290000000000004e-06 2024-09-16T09:32:03.556 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.556 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.556 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:32:03.556 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:46.805618+0000", 2024-09-16T09:32:03.557 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.5559000000000001e-05 2024-09-16T09:32:03.557 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.557 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.557 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:32:03.557 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:46.805705+0000", 2024-09-16T09:32:03.557 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.7059999999999997e-05 2024-09-16T09:32:03.557 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.557 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.557 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,2", 2024-09-16T09:32:03.557 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:46.805786+0000", 2024-09-16T09:32:03.557 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.0076999999999999e-05 2024-09-16T09:32:03.557 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.558 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.558 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:32:03.558 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:46.807389+0000", 2024-09-16T09:32:03.558 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0016034160000000001 2024-09-16T09:32:03.558 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.558 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.558 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:32:03.558 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:46.807761+0000", 2024-09-16T09:32:03.558 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00037185799999999998 2024-09-16T09:32:03.558 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.558 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.558 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:32:03.558 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:46.870158+0000", 2024-09-16T09:32:03.558 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.062397059999999997 2024-09-16T09:32:03.559 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.559 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.559 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:32:03.559 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:46.870193+0000", 2024-09-16T09:32:03.559 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.4777000000000003e-05 2024-09-16T09:32:03.559 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.559 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.559 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:32:03.559 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:46.870231+0000", 2024-09-16T09:32:03.559 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.8726999999999997e-05 2024-09-16T09:32:03.559 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:32:03.559 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:32:03.559 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:32:03.559 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.560 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.560 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:202401 7.5 7:a2fb40f6:::benchmark_data_smithi070_78438_object76270:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135)", 2024-09-16T09:32:03.560 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:28:50.720099+0000", 2024-09-16T09:32:03.560 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 192.76755378799999, 2024-09-16T09:32:03.560 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.089978474000000003, 2024-09-16T09:32:03.560 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:32:03.560 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:32:03.560 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:32:03.560 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:32:03.560 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:32:03.560 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:32:03.560 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 202401 2024-09-16T09:32:03.561 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.561 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:32:03.561 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.561 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:32:03.561 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.720099+0000", 2024-09-16T09:32:03.561 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:32:03.561 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.561 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.561 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:32:03.561 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.720099+0000", 2024-09-16T09:32:03.561 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:32:03.561 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.561 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.561 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:32:03.562 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.720101+0000", 2024-09-16T09:32:03.562 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.5889999999999999e-06 2024-09-16T09:32:03.562 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.562 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.562 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:32:03.562 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.720107+0000", 2024-09-16T09:32:03.562 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.6520000000000003e-06 2024-09-16T09:32:03.562 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.562 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.562 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:32:03.562 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.720117+0000", 2024-09-16T09:32:03.562 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0254999999999999e-05 2024-09-16T09:32:03.562 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.562 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.563 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:32:03.563 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.720122+0000", 2024-09-16T09:32:03.563 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.2580000000000003e-06 2024-09-16T09:32:03.563 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.563 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.563 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:32:03.563 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.720178+0000", 2024-09-16T09:32:03.563 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.5501000000000003e-05 2024-09-16T09:32:03.563 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.563 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.563 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:32:03.563 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.720290+0000", 2024-09-16T09:32:03.563 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000112837 2024-09-16T09:32:03.564 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.564 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.564 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,2", 2024-09-16T09:32:03.564 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.720394+0000", 2024-09-16T09:32:03.564 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000103613 2024-09-16T09:32:03.564 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.564 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.564 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:32:03.564 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.722666+0000", 2024-09-16T09:32:03.564 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0022723800000000001 2024-09-16T09:32:03.564 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.564 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.564 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:32:03.565 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.725231+0000", 2024-09-16T09:32:03.565 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0025650759999999999 2024-09-16T09:32:03.565 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.565 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.565 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:32:03.565 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.809799+0000", 2024-09-16T09:32:03.565 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.084568014999999996 2024-09-16T09:32:03.565 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.565 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.565 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:32:03.565 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.810033+0000", 2024-09-16T09:32:03.565 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00023360000000000001 2024-09-16T09:32:03.565 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.566 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.566 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:32:03.566 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.810078+0000", 2024-09-16T09:32:03.566 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.4697999999999998e-05 2024-09-16T09:32:03.566 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:32:03.566 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:32:03.566 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:32:03.566 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.566 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.566 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:202408 7.5 7:a7ac5683:::benchmark_data_smithi070_78438_object76277:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135)", 2024-09-16T09:32:03.566 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:28:50.725540+0000", 2024-09-16T09:32:03.566 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 192.762113441, 2024-09-16T09:32:03.566 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.090820330000000005, 2024-09-16T09:32:03.566 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:32:03.567 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:32:03.567 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:32:03.567 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:32:03.567 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:32:03.567 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:32:03.567 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 202408 2024-09-16T09:32:03.567 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.567 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:32:03.567 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.567 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:32:03.567 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.725540+0000", 2024-09-16T09:32:03.567 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:32:03.567 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.567 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.568 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:32:03.568 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.725540+0000", 2024-09-16T09:32:03.568 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:32:03.568 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.568 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.568 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:32:03.568 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.725541+0000", 2024-09-16T09:32:03.568 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.5069999999999999e-06 2024-09-16T09:32:03.568 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.568 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.568 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:32:03.568 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.725546+0000", 2024-09-16T09:32:03.568 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.113e-06 2024-09-16T09:32:03.568 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.569 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.569 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:32:03.569 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.725558+0000", 2024-09-16T09:32:03.569 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.2164000000000001e-05 2024-09-16T09:32:03.569 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.569 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.569 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:32:03.569 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.725565+0000", 2024-09-16T09:32:03.569 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.8850000000000001e-06 2024-09-16T09:32:03.569 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.569 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.569 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:32:03.569 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.725617+0000", 2024-09-16T09:32:03.570 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.1848000000000002e-05 2024-09-16T09:32:03.570 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.570 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.570 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:32:03.570 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.725728+0000", 2024-09-16T09:32:03.570 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00011048300000000001 2024-09-16T09:32:03.570 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.570 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.570 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,2", 2024-09-16T09:32:03.570 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.725839+0000", 2024-09-16T09:32:03.570 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00011105200000000001 2024-09-16T09:32:03.570 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.570 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.570 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:32:03.571 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.810121+0000", 2024-09-16T09:32:03.571 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.084282652 2024-09-16T09:32:03.571 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.571 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.571 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:32:03.571 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.816197+0000", 2024-09-16T09:32:03.571 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0060759309999999997 2024-09-16T09:32:03.571 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.571 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.571 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:32:03.571 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.816253+0000", 2024-09-16T09:32:03.572 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.5912999999999998e-05 2024-09-16T09:32:03.572 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.572 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.572 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:32:03.572 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.816308+0000", 2024-09-16T09:32:03.572 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.4511e-05 2024-09-16T09:32:03.572 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.572 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.572 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:32:03.572 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.816360+0000", 2024-09-16T09:32:03.572 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.2271000000000001e-05 2024-09-16T09:32:03.572 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:32:03.573 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:32:03.573 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:32:03.573 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.573 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.573 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24424.0:218 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch (1) in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e134)", 2024-09-16T09:32:03.573 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:28:50.729011+0000", 2024-09-16T09:32:03.573 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 192.758642313, 2024-09-16T09:32:03.573 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.078508791999999994, 2024-09-16T09:32:03.573 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:32:03.573 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:32:03.573 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "started", 2024-09-16T09:32:03.573 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:32:03.573 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24424", 2024-09-16T09:32:03.574 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.84:0/1181633158", 2024-09-16T09:32:03.574 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 218 2024-09-16T09:32:03.574 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.574 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:32:03.574 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.574 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:32:03.574 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.729011+0000", 2024-09-16T09:32:03.574 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:32:03.574 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.574 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.574 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:32:03.575 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.729011+0000", 2024-09-16T09:32:03.575 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:32:03.575 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.575 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.575 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:32:03.575 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.729012+0000", 2024-09-16T09:32:03.575 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.6270000000000001e-06 2024-09-16T09:32:03.575 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.575 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.575 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:32:03.575 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.729019+0000", 2024-09-16T09:32:03.575 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.8889999999999996e-06 2024-09-16T09:32:03.575 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.575 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.576 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:32:03.576 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.806924+0000", 2024-09-16T09:32:03.576 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.077904235000000002 2024-09-16T09:32:03.576 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.576 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.576 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:32:03.576 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.806944+0000", 2024-09-16T09:32:03.576 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.039e-05 2024-09-16T09:32:03.576 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.576 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.576 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:32:03.576 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.807345+0000", 2024-09-16T09:32:03.576 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00040116499999999999 2024-09-16T09:32:03.576 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.577 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.577 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:32:03.577 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.807432+0000", 2024-09-16T09:32:03.577 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.6515999999999996e-05 2024-09-16T09:32:03.577 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.577 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.577 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:32:03.577 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.807520+0000", 2024-09-16T09:32:03.577 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.797e-05 2024-09-16T09:32:03.577 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:32:03.578 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:32:03.578 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:32:03.578 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.578 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.578 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_repop_reply(client.24698.0:202401 7.5 e135/82 ondisk, result = 0)", 2024-09-16T09:32:03.578 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:28:50.730604+0000", 2024-09-16T09:32:03.578 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 192.75704918299999, 2024-09-16T09:32:03.578 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.079479822000000006, 2024-09-16T09:32:03.578 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:32:03.578 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:32:03.579 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "started", 2024-09-16T09:32:03.579 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:32:03.579 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.579 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:32:03.579 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.730604+0000", 2024-09-16T09:32:03.579 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:32:03.579 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.579 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.579 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:32:03.579 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.730604+0000", 2024-09-16T09:32:03.579 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:32:03.579 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.579 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.580 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:32:03.580 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.730605+0000", 2024-09-16T09:32:03.580 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.0999999999999997e-07 2024-09-16T09:32:03.580 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.580 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.580 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:32:03.580 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.730609+0000", 2024-09-16T09:32:03.580 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.9929999999999997e-06 2024-09-16T09:32:03.580 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.580 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.580 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:32:03.580 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.806968+0000", 2024-09-16T09:32:03.580 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.076359716999999994 2024-09-16T09:32:03.580 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.581 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.581 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:32:03.581 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.806980+0000", 2024-09-16T09:32:03.581 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1551e-05 2024-09-16T09:32:03.581 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.581 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.581 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:32:03.581 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.809786+0000", 2024-09-16T09:32:03.581 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0028062339999999999 2024-09-16T09:32:03.581 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.581 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.581 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:32:03.582 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.809794+0000", 2024-09-16T09:32:03.582 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.4630000000000004e-06 2024-09-16T09:32:03.582 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.582 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.582 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:32:03.582 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.810084+0000", 2024-09-16T09:32:03.582 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00029005399999999998 2024-09-16T09:32:03.582 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:32:03.582 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:32:03.582 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:32:03.582 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.582 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.582 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:202411 7.5 7:a7b52676:::benchmark_data_smithi070_78438_object76280:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135)", 2024-09-16T09:32:03.582 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:28:50.732821+0000", 2024-09-16T09:32:03.582 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 192.75483214799999, 2024-09-16T09:32:03.583 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.10429561900000001, 2024-09-16T09:32:03.583 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:32:03.583 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:32:03.583 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:32:03.583 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:32:03.583 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:32:03.583 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:32:03.583 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 202411 2024-09-16T09:32:03.583 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.583 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:32:03.583 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.583 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:32:03.583 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.732821+0000", 2024-09-16T09:32:03.584 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:32:03.584 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.584 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.584 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:32:03.584 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.732821+0000", 2024-09-16T09:32:03.584 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:32:03.584 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.584 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.584 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:32:03.584 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.732822+0000", 2024-09-16T09:32:03.585 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.46e-06 2024-09-16T09:32:03.585 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.585 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.585 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:32:03.585 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.732828+0000", 2024-09-16T09:32:03.585 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.4240000000000001e-06 2024-09-16T09:32:03.585 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.586 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.586 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:32:03.586 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.806990+0000", 2024-09-16T09:32:03.586 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.074162464999999997 2024-09-16T09:32:03.586 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.586 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.586 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:32:03.586 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.806998+0000", 2024-09-16T09:32:03.586 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.0369999999999995e-06 2024-09-16T09:32:03.586 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.586 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.586 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:32:03.586 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.810149+0000", 2024-09-16T09:32:03.586 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0031503339999999999 2024-09-16T09:32:03.586 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.587 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.587 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:32:03.587 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:32:03.588 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.810253+0000", 2024-09-16T09:32:03.588 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00010388200000000001 2024-09-16T09:32:03.588 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.588 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.588 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,2", 2024-09-16T09:32:03.588 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.810356+0000", 2024-09-16T09:32:03.588 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000103319 2024-09-16T09:32:03.589 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.589 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.589 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:32:03.589 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.816207+0000", 2024-09-16T09:32:03.589 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.005850984 2024-09-16T09:32:03.589 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.589 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.589 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:32:03.589 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.828515+0000", 2024-09-16T09:32:03.589 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.012307904 2024-09-16T09:32:03.589 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.589 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.590 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:32:03.590 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.837029+0000", 2024-09-16T09:32:03.590 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0085145419999999999 2024-09-16T09:32:03.590 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.590 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.590 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:32:03.590 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.837071+0000", 2024-09-16T09:32:03.590 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.1718999999999998e-05 2024-09-16T09:32:03.590 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.590 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.590 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:32:03.590 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.837117+0000", 2024-09-16T09:32:03.590 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.5549e-05 2024-09-16T09:32:03.590 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:32:03.591 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:32:03.591 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:32:03.591 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.591 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.591 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:204901 7.5 7:af219aea:::benchmark_data_smithi070_78438_object78770:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137)", 2024-09-16T09:32:03.591 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:28:54.030373+0000", 2024-09-16T09:32:03.591 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 189.457280478, 2024-09-16T09:32:03.591 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.077126564999999994, 2024-09-16T09:32:03.591 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:32:03.591 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:32:03.591 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:32:03.591 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:32:03.591 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:32:03.592 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:32:03.592 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 204901 2024-09-16T09:32:03.592 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.592 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:32:03.592 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.592 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:32:03.592 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.030373+0000", 2024-09-16T09:32:03.592 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:32:03.592 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.592 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.592 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:32:03.592 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.030373+0000", 2024-09-16T09:32:03.593 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:32:03.593 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.593 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.593 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:32:03.593 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.030374+0000", 2024-09-16T09:32:03.593 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.389e-06 2024-09-16T09:32:03.593 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.593 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.593 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:32:03.593 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.030378+0000", 2024-09-16T09:32:03.593 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.1679999999999997e-06 2024-09-16T09:32:03.593 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.594 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.594 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:32:03.594 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.030389+0000", 2024-09-16T09:32:03.594 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.097e-05 2024-09-16T09:32:03.594 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.594 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.594 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:32:03.594 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.030395+0000", 2024-09-16T09:32:03.594 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.229e-06 2024-09-16T09:32:03.594 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.594 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.594 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:32:03.594 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.030446+0000", 2024-09-16T09:32:03.594 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.0757999999999998e-05 2024-09-16T09:32:03.595 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.595 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.595 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:32:03.595 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.030553+0000", 2024-09-16T09:32:03.595 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00010718900000000001 2024-09-16T09:32:03.595 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.595 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.595 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,2", 2024-09-16T09:32:03.595 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.030664+0000", 2024-09-16T09:32:03.595 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00011086600000000001 2024-09-16T09:32:03.595 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.595 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.595 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:32:03.596 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.033351+0000", 2024-09-16T09:32:03.596 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.002686875 2024-09-16T09:32:03.596 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.596 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.596 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:32:03.596 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.033716+0000", 2024-09-16T09:32:03.596 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00036531200000000002 2024-09-16T09:32:03.596 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.596 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.596 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:32:03.596 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.107394+0000", 2024-09-16T09:32:03.596 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.073677379000000001 2024-09-16T09:32:03.596 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.597 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.597 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:32:03.597 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.107450+0000", 2024-09-16T09:32:03.597 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.6351999999999999e-05 2024-09-16T09:32:03.597 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.597 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.597 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:32:03.597 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.107499+0000", 2024-09-16T09:32:03.597 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.9078e-05 2024-09-16T09:32:03.597 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:32:03.597 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:32:03.597 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:32:03.597 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.597 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.598 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:204910 7.5 7:a16392ce:::benchmark_data_smithi070_78438_object78779:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137)", 2024-09-16T09:32:03.598 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:28:54.036796+0000", 2024-09-16T09:32:03.598 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 189.450857001, 2024-09-16T09:32:03.598 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.070834338999999996, 2024-09-16T09:32:03.598 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:32:03.598 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:32:03.598 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:32:03.598 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:32:03.598 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:32:03.598 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:32:03.598 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 204910 2024-09-16T09:32:03.598 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.598 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:32:03.599 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.599 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:32:03.599 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.036796+0000", 2024-09-16T09:32:03.599 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:32:03.599 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.599 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.599 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:32:03.599 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.036796+0000", 2024-09-16T09:32:03.599 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:32:03.599 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.599 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.599 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:32:03.599 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.036798+0000", 2024-09-16T09:32:03.599 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.649e-06 2024-09-16T09:32:03.600 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.600 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.600 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:32:03.600 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.036805+0000", 2024-09-16T09:32:03.600 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.0840000000000003e-06 2024-09-16T09:32:03.600 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.600 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.600 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:32:03.600 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.036824+0000", 2024-09-16T09:32:03.600 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.9318e-05 2024-09-16T09:32:03.600 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.600 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.600 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:32:03.600 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.036832+0000", 2024-09-16T09:32:03.601 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.4000000000000003e-06 2024-09-16T09:32:03.601 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.601 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.601 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:32:03.601 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.036897+0000", 2024-09-16T09:32:03.601 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.5238000000000001e-05 2024-09-16T09:32:03.601 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.601 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.601 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:32:03.601 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.037029+0000", 2024-09-16T09:32:03.601 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00013190399999999999 2024-09-16T09:32:03.601 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.601 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.602 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,2", 2024-09-16T09:32:03.602 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.037149+0000", 2024-09-16T09:32:03.602 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000119833 2024-09-16T09:32:03.602 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.602 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.602 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:32:03.602 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.039175+0000", 2024-09-16T09:32:03.602 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0020263389999999998 2024-09-16T09:32:03.602 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.602 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.602 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:32:03.602 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.106199+0000", 2024-09-16T09:32:03.602 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.067023694999999994 2024-09-16T09:32:03.602 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.603 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.603 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:32:03.603 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.107546+0000", 2024-09-16T09:32:03.603 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.001347902 2024-09-16T09:32:03.603 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.603 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.603 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:32:03.603 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.107593+0000", 2024-09-16T09:32:03.603 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.6736000000000001e-05 2024-09-16T09:32:03.603 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.603 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.603 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:32:03.603 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.107630+0000", 2024-09-16T09:32:03.603 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.7240999999999998e-05 2024-09-16T09:32:03.604 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:32:03.604 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:32:03.604 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:32:03.604 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.604 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.604 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:215340 7.5 7:a4f699d2:::benchmark_data_smithi070_78438_object89209:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e141)", 2024-09-16T09:32:03.604 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:29:03.365151+0000", 2024-09-16T09:32:03.604 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 180.12250218, 2024-09-16T09:32:03.604 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.068201946999999999, 2024-09-16T09:32:03.604 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:32:03.604 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:32:03.604 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:32:03.604 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:32:03.604 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:32:03.605 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:32:03.605 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 215340 2024-09-16T09:32:03.605 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.605 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:32:03.605 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.605 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:32:03.605 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.365151+0000", 2024-09-16T09:32:03.605 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:32:03.605 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.605 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.605 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:32:03.605 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.365151+0000", 2024-09-16T09:32:03.605 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:32:03.605 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.606 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.606 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:32:03.606 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.365153+0000", 2024-09-16T09:32:03.606 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.0540000000000002e-06 2024-09-16T09:32:03.606 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.606 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.606 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:32:03.606 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.365162+0000", 2024-09-16T09:32:03.606 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.38e-06 2024-09-16T09:32:03.606 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.606 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.606 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:32:03.606 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.365186+0000", 2024-09-16T09:32:03.606 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.3986e-05 2024-09-16T09:32:03.607 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.607 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.607 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:32:03.607 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.365200+0000", 2024-09-16T09:32:03.607 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.3586e-05 2024-09-16T09:32:03.607 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.607 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.607 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:32:03.607 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.365312+0000", 2024-09-16T09:32:03.607 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000112422 2024-09-16T09:32:03.607 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.607 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.607 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op must wait for map", 2024-09-16T09:32:03.607 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.365322+0000", 2024-09-16T09:32:03.608 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.5289999999999997e-06 2024-09-16T09:32:03.608 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.608 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.608 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:32:03.608 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.422102+0000", 2024-09-16T09:32:03.608 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.056780023999999998 2024-09-16T09:32:03.608 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.608 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.608 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:32:03.608 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.422202+0000", 2024-09-16T09:32:03.608 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.9617999999999994e-05 2024-09-16T09:32:03.608 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.608 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.608 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,2", 2024-09-16T09:32:03.609 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.422578+0000", 2024-09-16T09:32:03.609 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00037663600000000002 2024-09-16T09:32:03.609 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.609 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.609 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:32:03.609 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.431303+0000", 2024-09-16T09:32:03.609 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0087248670000000007 2024-09-16T09:32:03.609 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.609 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.609 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:32:03.609 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.431404+0000", 2024-09-16T09:32:03.609 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00010138500000000001 2024-09-16T09:32:03.609 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.609 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.610 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:32:03.610 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.433215+0000", 2024-09-16T09:32:03.610 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0018102979999999999 2024-09-16T09:32:03.610 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.610 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.610 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:32:03.610 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.433306+0000", 2024-09-16T09:32:03.610 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.1774999999999995e-05 2024-09-16T09:32:03.610 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.610 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.610 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:32:03.610 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.433353+0000", 2024-09-16T09:32:03.610 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.6387000000000002e-05 2024-09-16T09:32:03.610 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:32:03.610 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:32:03.611 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:32:03.611 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.611 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.611 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:215342 7.5 7:a84e8ca2:::benchmark_data_smithi070_78438_object89211:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e141)", 2024-09-16T09:32:03.611 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:29:03.366631+0000", 2024-09-16T09:32:03.611 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 180.12102160200001, 2024-09-16T09:32:03.611 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.074197923999999998, 2024-09-16T09:32:03.611 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:32:03.611 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:32:03.611 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:32:03.611 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:32:03.611 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:32:03.611 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:32:03.611 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 215342 2024-09-16T09:32:03.612 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.612 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:32:03.612 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.612 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:32:03.612 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.366631+0000", 2024-09-16T09:32:03.612 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:32:03.612 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.612 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.612 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:32:03.612 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.366631+0000", 2024-09-16T09:32:03.612 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:32:03.612 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.612 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.612 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:32:03.613 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.366633+0000", 2024-09-16T09:32:03.613 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.2839999999999999e-06 2024-09-16T09:32:03.613 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.613 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.613 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:32:03.613 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.366640+0000", 2024-09-16T09:32:03.613 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.6329999999999996e-06 2024-09-16T09:32:03.613 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.613 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.613 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:32:03.613 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.420402+0000", 2024-09-16T09:32:03.613 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.053761807000000002 2024-09-16T09:32:03.613 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.614 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.614 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:32:03.614 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.420414+0000", 2024-09-16T09:32:03.614 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1885000000000001e-05 2024-09-16T09:32:03.614 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.614 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.614 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:32:03.614 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.426991+0000", 2024-09-16T09:32:03.614 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0065772399999999998 2024-09-16T09:32:03.614 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.614 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.614 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:32:03.614 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.427091+0000", 2024-09-16T09:32:03.614 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.9407999999999994e-05 2024-09-16T09:32:03.615 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.615 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.615 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,2", 2024-09-16T09:32:03.615 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.427202+0000", 2024-09-16T09:32:03.615 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000111298 2024-09-16T09:32:03.615 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.615 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.615 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:32:03.615 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.433361+0000", 2024-09-16T09:32:03.615 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0061589390000000004 2024-09-16T09:32:03.615 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.615 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.615 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:32:03.615 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.437028+0000", 2024-09-16T09:32:03.616 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0036673130000000002 2024-09-16T09:32:03.616 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.616 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.616 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:32:03.616 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.440723+0000", 2024-09-16T09:32:03.616 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0036951480000000001 2024-09-16T09:32:03.616 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.616 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.616 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:32:03.616 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.440791+0000", 2024-09-16T09:32:03.616 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.7496000000000006e-05 2024-09-16T09:32:03.616 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.616 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.616 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:32:03.617 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.440829+0000", 2024-09-16T09:32:03.617 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.8473000000000002e-05 2024-09-16T09:32:03.620 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:32:03.620 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:32:03.620 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:32:03.620 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.620 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.620 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:247035 7.5 7:aba136a1:::benchmark_data_smithi070_78438_object120904:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e147)", 2024-09-16T09:32:03.620 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:29:32.642597+0000", 2024-09-16T09:32:03.621 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 150.845056489, 2024-09-16T09:32:03.621 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.089290376000000005, 2024-09-16T09:32:03.621 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:32:03.621 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:32:03.621 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:32:03.621 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:32:03.621 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:32:03.621 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:32:03.621 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 247035 2024-09-16T09:32:03.621 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.621 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:32:03.621 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.621 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:32:03.622 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:32.642597+0000", 2024-09-16T09:32:03.622 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:32:03.622 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.622 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.622 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:32:03.622 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:32.642597+0000", 2024-09-16T09:32:03.622 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:32:03.622 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.622 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.622 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:32:03.622 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:32.642598+0000", 2024-09-16T09:32:03.622 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.511e-06 2024-09-16T09:32:03.622 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.622 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.623 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:32:03.623 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:32.642604+0000", 2024-09-16T09:32:03.623 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.6080000000000003e-06 2024-09-16T09:32:03.623 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.623 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.623 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:32:03.623 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:32.642616+0000", 2024-09-16T09:32:03.623 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.2532e-05 2024-09-16T09:32:03.623 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.623 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.623 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:32:03.623 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:32.642622+0000", 2024-09-16T09:32:03.623 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.8869999999999997e-06 2024-09-16T09:32:03.623 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.624 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.624 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:32:03.624 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:32.642686+0000", 2024-09-16T09:32:03.624 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.3368999999999996e-05 2024-09-16T09:32:03.624 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.624 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.624 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op must wait for map", 2024-09-16T09:32:03.624 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:32.642693+0000", 2024-09-16T09:32:03.624 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.2219999999999996e-06 2024-09-16T09:32:03.624 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.624 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.624 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:32:03.625 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:32.644822+0000", 2024-09-16T09:32:03.625 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0021290290000000002 2024-09-16T09:32:03.625 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.625 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.625 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:32:03.625 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:32.644912+0000", 2024-09-16T09:32:03.625 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.0234000000000004e-05 2024-09-16T09:32:03.625 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.625 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.625 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,2", 2024-09-16T09:32:03.626 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:32.644981+0000", 2024-09-16T09:32:03.626 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.8956999999999997e-05 2024-09-16T09:32:03.626 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.626 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.626 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:32:03.626 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:32.647401+0000", 2024-09-16T09:32:03.626 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0024197620000000002 2024-09-16T09:32:03.626 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.626 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.626 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:32:03.626 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:32.723230+0000", 2024-09-16T09:32:03.626 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.075829222000000002 2024-09-16T09:32:03.626 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.627 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.627 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:32:03.627 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:32.731807+0000", 2024-09-16T09:32:03.627 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.008576719 2024-09-16T09:32:03.627 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.627 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.627 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:32:03.627 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:32.731856+0000", 2024-09-16T09:32:03.627 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.9051999999999997e-05 2024-09-16T09:32:03.627 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:32:03.627 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:32:03.627 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:32:03.627 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:32.731887+0000", 2024-09-16T09:32:03.627 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.1272000000000001e-05 2024-09-16T09:32:03.628 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:32:03.628 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:32:03.628 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:32:03.628 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:32:03.628 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:32:03.628 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-09-16T09:32:03.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:03 smithi084 ceph-mon[32629]: pgmap v326: 168 pgs: 1 active+recovery_wait+degraded, 3 active+recovering+undersized+remapped, 164 active+clean; 5.5 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 615 B/s rd, 0 B/s wr, 698 op/s; 1/268683 objects degraded (0.000%); 5350/268683 objects misplaced (1.991%); 4.9 MiB/s, 78 objects/s recovering 2024-09-16T09:32:03.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:03 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:32:03.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:03 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:32:03.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:03 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:32:03.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:03 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:32:03.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:03 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:32:03.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:03 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:32:03.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:03 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:32:03.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:03 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/813645725' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:32:03.665 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:03.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:03.666 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:32:03.766 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:32:03.804 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:03.804 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:03.805 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:32:03.906 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:32:03.955 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:03.955 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:03.956 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:32:04.057 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:32:04.092 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:04.092 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:04.092 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:32:04.193 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:32:04.227 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:04.227 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:04.228 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:32:04.328 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:32:04.355 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:04.356 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:04.356 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:32:04.457 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:32:04.487 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:04.488 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:04.489 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:32:04.589 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:32:04.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:04.628 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:04.628 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:32:04.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:04 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:32:04.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:04 smithi084 ceph-mon[32629]: osdmap e202: 8 total, 8 up, 8 in 2024-09-16T09:32:04.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:04 smithi084 ceph-mon[32629]: Health check update: Degraded data redundancy: 1/268683 objects degraded (0.000%), 1 pg degraded (PG_DEGRADED) 2024-09-16T09:32:04.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:04 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:32:04.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:04 smithi070 ceph-mon[30388]: osdmap e202: 8 total, 8 up, 8 in 2024-09-16T09:32:04.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:04 smithi070 ceph-mon[30388]: Health check update: Degraded data redundancy: 1/268683 objects degraded (0.000%), 1 pg degraded (PG_DEGRADED) 2024-09-16T09:32:04.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:04 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:32:04.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:04 smithi070 ceph-mon[40691]: osdmap e202: 8 total, 8 up, 8 in 2024-09-16T09:32:04.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:04 smithi070 ceph-mon[40691]: Health check update: Degraded data redundancy: 1/268683 objects degraded (0.000%), 1 pg degraded (PG_DEGRADED) 2024-09-16T09:32:04.728 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:32:04.744 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.0 dump_ops_in_flight 2024-09-16T09:32:04.767 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:04.768 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:04.768 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:32:04.869 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:32:04.898 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:04.898 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:04.899 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:32:04.938 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:32:04.939 INFO:teuthology.orchestra.run.smithi070.stdout:{"fsid":"a1321834-740c-11ef-bceb-c7b262605968","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 1/252006 objects degraded (0.000%), 1 pg degraded","count":1},"muted":false}},"mutes":[]},"election_epoch":14,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":671,"monmap":{"epoch":3,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":203,"num_osds":8,"num_up_osds":8,"osd_up_since":1726478691,"num_in_osds":8,"osd_in_since":1726478668,"num_remapped_pgs":3},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":164},{"state_name":"active+recovering+undersized+remapped","count":3},{"state_name":"active+recovery_wait+degraded","count":1}],"num_pgs":168,"num_pools":7,"num_objects":84002,"data_bytes":5492795434,"bytes_used":19293097984,"bytes_avail":748667187200,"bytes_total":767960285184,"degraded_objects":1,"degraded_total":252006,"degraded_ratio":3.9681594882661521e-06,"misplaced_objects":4670,"misplaced_total":252006,"misplaced_ratio":0.018531304810202931,"recovering_objects_per_sec":166,"recovering_bytes_per_sec":10916665,"recovering_keys_per_sec":0,"num_objects_recovered":1333,"num_bytes_recovered":87359488,"num_keys_recovered":0,"read_bytes_sec":1279,"write_bytes_sec":0,"read_op_per_sec":1,"write_op_per_sec":1420},"fsmap":{"epoch":1,"btime":"2024-09-16T09:19:14:121324+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":1,"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"services":{"dashboard":"https://172.21.15.70:8443/","prometheus":"http://172.21.15.70:9283/"}},"servicemap":{"epoch":14,"modified":"2024-09-16T09:26:33.498407+0000","services":{"rgw":{"daemons":{"summary":"","14475":{"start_epoch":12,"start_stamp":"2024-09-16T09:25:08.794234+0000","gid":14475,"addr":"172.21.15.70:0/1049757559","metadata":{"arch":"x86_64","ceph_release":"squid","ceph_version":"ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev)","ceph_version_short":"19.3.0-4917-g8293d73f","container_hostname":"smithi070","container_image":"quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce","cpu":"Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz","distro":"centos","distro_description":"CentOS Stream 9","distro_version":"9","frontend_config#0":"beast port=80","frontend_type#0":"beast","hostname":"smithi070","id":"foo.a","kernel_description":"#1 SMP PREEMPT_DYNAMIC Thu Sep 5 07:54:07 UTC 2024","kernel_version":"5.14.0-505.el9.x86_64","mem_swap_kb":"0","mem_total_kb":"32487212","num_handles":"1","os":"Linux","pid":"7","realm_id":"","realm_name":"","zone_id":"09633d1c-04b1-424a-bf26-5fa0f61f30f8","zone_name":"default","zonegroup_id":"c6682a31-37e4-465e-b890-750b49769dfa","zonegroup_name":"default"},"task_status":{}}}}}},"progress_events":{}} 2024-09-16T09:32:04.999 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:32:05.040 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:05.041 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:05.041 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:32:05.142 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:32:05.181 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:05.182 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:05.182 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:32:05.283 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:32:05.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:05.322 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:05.322 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:32:05.423 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:32:05.459 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:05.460 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:05.460 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:32:05.561 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:32:05.593 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:05.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:05.594 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:32:05.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:05 smithi084 ceph-mon[32629]: pgmap v328: 168 pgs: 1 active+recovery_wait+degraded, 3 active+recovering+undersized+remapped, 164 active+clean; 5.1 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.42k op/s; 1/252006 objects degraded (0.000%); 4670/252006 objects misplaced (1.853%); 10 MiB/s, 166 objects/s recovering 2024-09-16T09:32:05.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:05 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:32:05.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:05 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:32:05.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:05 smithi084 ceph-mon[32629]: osdmap e203: 8 total, 8 up, 8 in 2024-09-16T09:32:05.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:05 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3987608792' entity='client.admin' cmd={"prefix": "status", "format": "json"} : dispatch 2024-09-16T09:32:05.673 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:05 smithi070 ceph-mon[30388]: pgmap v328: 168 pgs: 1 active+recovery_wait+degraded, 3 active+recovering+undersized+remapped, 164 active+clean; 5.1 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.42k op/s; 1/252006 objects degraded (0.000%); 4670/252006 objects misplaced (1.853%); 10 MiB/s, 166 objects/s recovering 2024-09-16T09:32:05.673 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:05 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:32:05.673 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:05 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:32:05.673 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:05 smithi070 ceph-mon[30388]: osdmap e203: 8 total, 8 up, 8 in 2024-09-16T09:32:05.673 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:05 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3987608792' entity='client.admin' cmd={"prefix": "status", "format": "json"} : dispatch 2024-09-16T09:32:05.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:05 smithi070 ceph-mon[40691]: pgmap v328: 168 pgs: 1 active+recovery_wait+degraded, 3 active+recovering+undersized+remapped, 164 active+clean; 5.1 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.42k op/s; 1/252006 objects degraded (0.000%); 4670/252006 objects misplaced (1.853%); 10 MiB/s, 166 objects/s recovering 2024-09-16T09:32:05.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:05 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:32:05.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:05 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:32:05.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:05 smithi070 ceph-mon[40691]: osdmap e203: 8 total, 8 up, 8 in 2024-09-16T09:32:05.674 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:05 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3987608792' entity='client.admin' cmd={"prefix": "status", "format": "json"} : dispatch 2024-09-16T09:32:05.695 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:32:05.730 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:05.731 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:05.731 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:32:05.832 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:32:05.868 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:05.868 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:05.868 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:32:05.929 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T09:32:05.970 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:32:06.003 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:06.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:06.004 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:32:06.105 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:32:06.139 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:06.140 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:06.140 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:32:06.241 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:32:06.267 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:06.268 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:06.269 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:32:06.369 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:32:06.408 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:06.409 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:06.409 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:32:06.510 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:32:06.548 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:06.549 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:06.549 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:32:06.650 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:32:06.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:06 smithi084 ceph-mon[32629]: 8.c scrub starts 2024-09-16T09:32:06.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:06 smithi070 ceph-mon[30388]: 8.c scrub starts 2024-09-16T09:32:06.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:06 smithi070 ceph-mon[40691]: 8.c scrub starts 2024-09-16T09:32:06.680 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:06.681 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:06.682 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:32:06.783 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:32:06.823 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:06.823 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:06.824 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:32:06.874 INFO:tasks.cephadm.ceph_manager.ceph:making progress, resetting timeout 2024-09-16T09:32:06.874 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-16T09:32:06.885 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:32:06.925 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:32:06.929 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:32:06.949 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:06.949 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:06.950 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:32:07.050 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:32:07.086 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:07.086 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:07.086 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:32:07.187 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:32:07.214 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:07.215 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:07.215 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:32:07.316 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:32:07.351 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:07.352 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:07.352 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:32:07.453 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:32:07.479 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:07.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:07.481 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:32:07.581 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:32:07.607 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:07.608 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:07.608 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:32:07.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:07 smithi084 ceph-mon[32629]: pgmap v330: 168 pgs: 1 active+recovery_wait+degraded, 3 active+recovering+undersized+remapped, 164 active+clean; 5.1 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 927 op/s; 1/252006 objects degraded (0.000%); 4670/252006 objects misplaced (1.853%); 7.1 MiB/s, 113 objects/s recovering 2024-09-16T09:32:07.669 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:07 smithi070 ceph-mon[30388]: pgmap v330: 168 pgs: 1 active+recovery_wait+degraded, 3 active+recovering+undersized+remapped, 164 active+clean; 5.1 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 927 op/s; 1/252006 objects degraded (0.000%); 4670/252006 objects misplaced (1.853%); 7.1 MiB/s, 113 objects/s recovering 2024-09-16T09:32:07.669 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:07 smithi070 ceph-mon[40691]: pgmap v330: 168 pgs: 1 active+recovery_wait+degraded, 3 active+recovering+undersized+remapped, 164 active+clean; 5.1 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 927 op/s; 1/252006 objects degraded (0.000%); 4670/252006 objects misplaced (1.853%); 7.1 MiB/s, 113 objects/s recovering 2024-09-16T09:32:07.702 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T09:32:07.702 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T09:32:07.703 INFO:teuthology.orchestra.run.smithi070.stdout: "num_ops": 0 2024-09-16T09:32:07.703 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T09:32:07.709 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:32:07.731 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:07.732 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:07.732 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:32:07.833 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:32:07.855 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:07.856 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:07.856 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:32:07.957 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:32:07.979 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:07.980 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:07.980 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:32:08.081 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:32:08.105 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:08.105 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:08.106 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:32:08.151 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:32:08.207 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:32:08.229 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:08.230 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:08.230 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:32:08.331 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:32:08.353 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:08.354 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:08.354 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:32:08.420 INFO:teuthology.orchestra.run.smithi070.stderr:noscrub is set 2024-09-16T09:32:08.455 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:32:08.491 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:08.493 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:08.493 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:32:08.594 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:32:08.609 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:08 smithi070 ceph-mon[30388]: osdmap e204: 8 total, 8 up, 8 in 2024-09-16T09:32:08.610 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:08 smithi070 ceph-mon[30388]: 5.11 scrub starts 2024-09-16T09:32:08.610 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:08 smithi070 ceph-mon[30388]: 5.11 scrub ok 2024-09-16T09:32:08.610 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:08 smithi070 ceph-mon[30388]: 8.c scrub ok 2024-09-16T09:32:08.610 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:08 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3955503331' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:32:08.610 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:08 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:32:08.610 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:08 smithi070 ceph-mon[40691]: osdmap e204: 8 total, 8 up, 8 in 2024-09-16T09:32:08.610 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:08 smithi070 ceph-mon[40691]: 5.11 scrub starts 2024-09-16T09:32:08.610 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:08 smithi070 ceph-mon[40691]: 5.11 scrub ok 2024-09-16T09:32:08.610 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:08 smithi070 ceph-mon[40691]: 8.c scrub ok 2024-09-16T09:32:08.611 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:08 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3955503331' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:32:08.611 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:08 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:32:08.629 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:08.629 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:08.630 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:32:08.656 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.0 dump_blocked_ops 2024-09-16T09:32:08.731 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:32:08.762 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:08.763 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:08.763 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:32:08.853 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:32:08.864 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:32:08.893 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:08.894 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:08.894 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:32:08.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:08 smithi084 ceph-mon[32629]: osdmap e204: 8 total, 8 up, 8 in 2024-09-16T09:32:08.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:08 smithi084 ceph-mon[32629]: 5.11 scrub starts 2024-09-16T09:32:08.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:08 smithi084 ceph-mon[32629]: 5.11 scrub ok 2024-09-16T09:32:08.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:08 smithi084 ceph-mon[32629]: 8.c scrub ok 2024-09-16T09:32:08.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:08 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3955503331' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:32:08.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:08 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:32:08.995 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:32:09.031 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:09.032 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:09.032 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:32:09.133 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:32:09.163 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:09.164 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:09.165 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:32:09.265 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:32:09.308 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:09.309 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:09.309 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:32:09.410 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:32:09.445 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:09.445 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:09.446 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:32:09.546 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:32:09.550 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:09 smithi070 ceph-mon[30388]: pgmap v332: 168 pgs: 1 active+recovery_wait+degraded, 3 active+recovering+undersized+remapped, 164 active+clean; 5.1 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.05k op/s; 1/249855 objects degraded (0.000%); 4670/249855 objects misplaced (1.869%); 7.1 MiB/s, 113 objects/s recovering 2024-09-16T09:32:09.551 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:09 smithi070 ceph-mon[30388]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:32:09.551 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:09 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-16T09:32:09.551 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:09 smithi070 ceph-mon[30388]: osdmap e205: 8 total, 8 up, 8 in 2024-09-16T09:32:09.551 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:09 smithi070 ceph-mon[30388]: Health check update: Degraded data redundancy: 1/249855 objects degraded (0.000%), 1 pg degraded (PG_DEGRADED) 2024-09-16T09:32:09.551 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:09 smithi070 ceph-mon[40691]: pgmap v332: 168 pgs: 1 active+recovery_wait+degraded, 3 active+recovering+undersized+remapped, 164 active+clean; 5.1 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.05k op/s; 1/249855 objects degraded (0.000%); 4670/249855 objects misplaced (1.869%); 7.1 MiB/s, 113 objects/s recovering 2024-09-16T09:32:09.551 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:09 smithi070 ceph-mon[40691]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:32:09.551 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:09 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-16T09:32:09.551 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:09 smithi070 ceph-mon[40691]: osdmap e205: 8 total, 8 up, 8 in 2024-09-16T09:32:09.551 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:09 smithi070 ceph-mon[40691]: Health check update: Degraded data redundancy: 1/249855 objects degraded (0.000%), 1 pg degraded (PG_DEGRADED) 2024-09-16T09:32:09.589 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:09.590 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:09.590 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:32:09.691 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:32:09.726 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:09.727 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:09.727 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:32:09.828 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:32:09.863 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:09.863 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:09.864 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:32:09.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:09 smithi084 ceph-mon[32629]: pgmap v332: 168 pgs: 1 active+recovery_wait+degraded, 3 active+recovering+undersized+remapped, 164 active+clean; 5.1 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.05k op/s; 1/249855 objects degraded (0.000%); 4670/249855 objects misplaced (1.869%); 7.1 MiB/s, 113 objects/s recovering 2024-09-16T09:32:09.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:09 smithi084 ceph-mon[32629]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:32:09.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:09 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-16T09:32:09.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:09 smithi084 ceph-mon[32629]: osdmap e205: 8 total, 8 up, 8 in 2024-09-16T09:32:09.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:09 smithi084 ceph-mon[32629]: Health check update: Degraded data redundancy: 1/249855 objects degraded (0.000%), 1 pg degraded (PG_DEGRADED) 2024-09-16T09:32:09.965 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:32:10.008 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:10.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:10.009 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:32:10.109 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:32:10.151 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:10.151 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:10.152 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:32:10.253 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:32:10.284 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:10.285 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:10.285 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:32:10.388 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:32:10.409 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:10.410 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:10.410 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:32:10.511 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:32:10.542 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:10.543 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:10.544 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:32:10.644 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:32:10.683 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:10.683 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:10.684 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:32:10.784 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:32:10.819 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:10.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:10.821 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:32:10.921 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:32:10.951 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:10.952 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:10.952 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:32:11.053 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:32:11.091 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:11.091 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:11.092 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:32:11.193 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:32:11.222 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:11.222 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:11.223 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:32:11.323 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:32:11.361 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:11.361 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:11.361 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:32:11.368 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:32:11.371 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2024-09-16T09:32:11.462 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:32:11.501 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:11.502 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:11.502 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:32:11.602 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:32:11.646 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:11.646 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:11.647 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:32:11.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:11 smithi070 ceph-mon[30388]: pgmap v334: 168 pgs: 1 peering, 2 active+recovering+undersized+remapped, 165 active+clean; 4.8 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 890 op/s; 3829/235989 objects misplaced (1.623%); 8.8 MiB/s, 140 objects/s recovering 2024-09-16T09:32:11.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:11 smithi070 ceph-mon[30388]: from='client.25352 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:32:11.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:11 smithi070 ceph-mon[40691]: pgmap v334: 168 pgs: 1 peering, 2 active+recovering+undersized+remapped, 165 active+clean; 4.8 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 890 op/s; 3829/235989 objects misplaced (1.623%); 8.8 MiB/s, 140 objects/s recovering 2024-09-16T09:32:11.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:11 smithi070 ceph-mon[40691]: from='client.25352 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:32:11.748 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:32:11.777 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:32:11.778 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:32:11.784 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:11.785 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:11.785 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:32:11.813 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:32:11.813 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:32:11.847 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:32:11.848 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:32:11.880 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:32:11.880 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:32:11.885 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:32:11.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:11 smithi084 ceph-mon[32629]: pgmap v334: 168 pgs: 1 peering, 2 active+recovering+undersized+remapped, 165 active+clean; 4.8 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 890 op/s; 3829/235989 objects misplaced (1.623%); 8.8 MiB/s, 140 objects/s recovering 2024-09-16T09:32:11.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:11 smithi084 ceph-mon[32629]: from='client.25352 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:32:11.913 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:32:11.913 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:32:11.920 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:11.921 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:11.921 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:32:11.949 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:32:11.949 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:32:11.983 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:32:11.984 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:32:11.984 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:32:11 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:32:11] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:32:12.018 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:32:12.018 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:32:12.024 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:32:12.053 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:12.054 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:12.054 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:32:12.155 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:32:12.191 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:12.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:12.192 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:32:12.293 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:32:12.332 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:12.332 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:12.333 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:32:12.434 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:32:12.469 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:12.469 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:12.470 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:32:12.571 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:32:12.603 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:12.604 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:12.604 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:32:12.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:12 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2681353372' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:32:12.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:12 smithi070 ceph-mon[30388]: 1.5 scrub starts 2024-09-16T09:32:12.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:12 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/2681353372' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:32:12.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:12 smithi070 ceph-mon[40691]: 1.5 scrub starts 2024-09-16T09:32:12.704 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:32:12.729 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":334,"stamp":"2024-09-16T09:32:09.628902+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":5142898730,"num_objects":78663,"num_object_clones":0,"num_object_copies":235989,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3829,"num_objects_unfound":0,"num_objects_dirty":78663,"num_whiteouts":0,"num_read":1157,"num_read_kb":928,"num_write":171180,"num_write_kb":7974230,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11018,"num_bytes_recovered":713997088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79450,"ondisk_log_size":79450,"up":504,"acting":502,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":508,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":17816264,"kb_used_data":15811768,"kb_used_omap":12,"kb_used_meta":2004147,"kb_avail":732144952,"statfs":{"total":767960285184,"available":749716430848,"internally_reserved":0,"allocated":16191250432,"data_stored":16188950577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":2052247090},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,6,5,7,14,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-349896704,"num_objects":-5339,"num_object_clones":0,"num_object_copies":-16017,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1,"num_objects_misplaced":-841,"num_objects_unfound":0,"num_objects_dirty":-5339,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5339,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":842,"num_bytes_recovered":55378464,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001592"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":329,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.427600+0000","last_change":"2024-09-16T09:30:03.848439+0000","last_active":"2024-09-16T09:32:07.427600+0000","last_peered":"2024-09-16T09:32:07.427600+0000","last_clean":"2024-09-16T09:32:07.427600+0000","last_became_active":"2024-09-16T09:25:01.709108+0000","last_became_peered":"2024-09-16T09:25:01.709108+0000","last_unstale":"2024-09-16T09:32:07.427600+0000","last_undegraded":"2024-09-16T09:32:07.427600+0000","last_fullsized":"2024-09-16T09:32:07.427600+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:30:03.848354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:30:03.848354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"66'9","reported_seq":332,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.412408+0000","last_change":"2024-09-16T09:28:52.689373+0000","last_active":"2024-09-16T09:32:07.412408+0000","last_peered":"2024-09-16T09:32:07.412408+0000","last_clean":"2024-09-16T09:32:07.412408+0000","last_became_active":"2024-09-16T09:25:03.714635+0000","last_became_peered":"2024-09-16T09:25:03.714635+0000","last_unstale":"2024-09-16T09:32:07.412408+0000","last_undegraded":"2024-09-16T09:32:07.412408+0000","last_fullsized":"2024-09-16T09:32:07.412408+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:28:52.689304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:52.689304+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":320,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.425518+0000","last_change":"2024-09-16T09:29:17.910367+0000","last_active":"2024-09-16T09:32:07.425518+0000","last_peered":"2024-09-16T09:32:07.425518+0000","last_clean":"2024-09-16T09:32:07.425518+0000","last_became_active":"2024-09-16T09:25:05.715645+0000","last_became_peered":"2024-09-16T09:25:05.715645+0000","last_unstale":"2024-09-16T09:32:07.425518+0000","last_undegraded":"2024-09-16T09:32:07.425518+0000","last_fullsized":"2024-09-16T09:32:07.425518+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:17.910305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:17.910305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":300,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.413321+0000","last_change":"2024-09-16T09:27:11.565306+0000","last_active":"2024-09-16T09:32:07.413321+0000","last_peered":"2024-09-16T09:32:07.413321+0000","last_clean":"2024-09-16T09:32:07.413321+0000","last_became_active":"2024-09-16T09:25:07.715442+0000","last_became_peered":"2024-09-16T09:25:07.715442+0000","last_unstale":"2024-09-16T09:32:07.413321+0000","last_undegraded":"2024-09-16T09:32:07.413321+0000","last_fullsized":"2024-09-16T09:32:07.413321+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:11.565220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:11.565220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":46,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.419776+0000","last_change":"2024-09-16T09:31:09.269700+0000","last_active":"2024-09-16T09:32:07.419776+0000","last_peered":"2024-09-16T09:32:07.419776+0000","last_clean":"2024-09-16T09:32:07.419776+0000","last_became_active":"2024-09-16T09:31:09.269342+0000","last_became_peered":"2024-09-16T09:31:09.269342+0000","last_unstale":"2024-09-16T09:32:07.419776+0000","last_undegraded":"2024-09-16T09:32:07.419776+0000","last_fullsized":"2024-09-16T09:32:07.419776+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:00.068570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:28:00.068570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":186,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.469717+0000","last_change":"2024-09-16T09:31:56.507595+0000","last_active":"2024-09-16T09:32:07.469717+0000","last_peered":"2024-09-16T09:32:07.469717+0000","last_clean":"2024-09-16T09:32:07.469717+0000","last_became_active":"2024-09-16T09:31:09.275136+0000","last_became_peered":"2024-09-16T09:31:09.275136+0000","last_unstale":"2024-09-16T09:32:07.469717+0000","last_undegraded":"2024-09-16T09:32:07.469717+0000","last_fullsized":"2024-09-16T09:32:07.469717+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:31:56.507528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:31:56.507528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:08.491873+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.19","version":"66'15","reported_seq":247,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.417360+0000","last_change":"2024-09-16T09:31:09.302903+0000","last_active":"2024-09-16T09:32:07.417360+0000","last_peered":"2024-09-16T09:32:07.417360+0000","last_clean":"2024-09-16T09:32:07.417360+0000","last_became_active":"2024-09-16T09:31:09.268750+0000","last_became_peered":"2024-09-16T09:31:09.268750+0000","last_unstale":"2024-09-16T09:32:07.417360+0000","last_undegraded":"2024-09-16T09:32:07.417360+0000","last_fullsized":"2024-09-16T09:32:07.417360+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-16T09:29:57.968162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:29:57.968162+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":320,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.412297+0000","last_change":"2024-09-16T09:29:15.628676+0000","last_active":"2024-09-16T09:32:07.412297+0000","last_peered":"2024-09-16T09:32:07.412297+0000","last_clean":"2024-09-16T09:32:07.412297+0000","last_became_active":"2024-09-16T09:25:05.717236+0000","last_became_peered":"2024-09-16T09:25:05.717236+0000","last_unstale":"2024-09-16T09:32:07.412297+0000","last_undegraded":"2024-09-16T09:32:07.412297+0000","last_fullsized":"2024-09-16T09:32:07.412297+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:15.628467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:15.628467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":159,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.425283+0000","last_change":"2024-09-16T09:28:34.839909+0000","last_active":"2024-09-16T09:32:07.425283+0000","last_peered":"2024-09-16T09:32:07.425283+0000","last_clean":"2024-09-16T09:32:07.425283+0000","last_became_active":"2024-09-16T09:28:34.839435+0000","last_became_peered":"2024-09-16T09:28:34.839435+0000","last_unstale":"2024-09-16T09:32:07.425283+0000","last_undegraded":"2024-09-16T09:32:07.425283+0000","last_fullsized":"2024-09-16T09:32:07.425283+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:01.038400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:28:01.038400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:09.066525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"66'9","reported_seq":332,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.413565+0000","last_change":"2024-09-16T09:29:28.622647+0000","last_active":"2024-09-16T09:32:07.413565+0000","last_peered":"2024-09-16T09:32:07.413565+0000","last_clean":"2024-09-16T09:32:07.413565+0000","last_became_active":"2024-09-16T09:25:03.709147+0000","last_became_peered":"2024-09-16T09:25:03.709147+0000","last_unstale":"2024-09-16T09:32:07.413565+0000","last_undegraded":"2024-09-16T09:32:07.413565+0000","last_fullsized":"2024-09-16T09:32:07.413565+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:29:28.622558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:29:28.622558+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":303,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.417337+0000","last_change":"2024-09-16T09:27:59.080369+0000","last_active":"2024-09-16T09:32:07.417337+0000","last_peered":"2024-09-16T09:32:07.417337+0000","last_clean":"2024-09-16T09:32:07.417337+0000","last_became_active":"2024-09-16T09:25:05.718134+0000","last_became_peered":"2024-09-16T09:25:05.718134+0000","last_unstale":"2024-09-16T09:32:07.417337+0000","last_undegraded":"2024-09-16T09:32:07.417337+0000","last_fullsized":"2024-09-16T09:32:07.417337+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:59.080307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:59.080307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"66'1","reported_seq":317,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.426952+0000","last_change":"2024-09-16T09:31:58.367903+0000","last_active":"2024-09-16T09:32:07.426952+0000","last_peered":"2024-09-16T09:32:07.426952+0000","last_clean":"2024-09-16T09:32:07.426952+0000","last_became_active":"2024-09-16T09:25:07.716558+0000","last_became_peered":"2024-09-16T09:25:07.716558+0000","last_unstale":"2024-09-16T09:32:07.426952+0000","last_undegraded":"2024-09-16T09:32:07.426952+0000","last_fullsized":"2024-09-16T09:32:07.426952+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-16T09:31:58.367837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:31:58.367837+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:20.121824+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":311,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.418344+0000","last_change":"2024-09-16T09:27:56.191264+0000","last_active":"2024-09-16T09:32:07.418344+0000","last_peered":"2024-09-16T09:32:07.418344+0000","last_clean":"2024-09-16T09:32:07.418344+0000","last_became_active":"2024-09-16T09:25:01.696939+0000","last_became_peered":"2024-09-16T09:25:01.696939+0000","last_unstale":"2024-09-16T09:32:07.418344+0000","last_undegraded":"2024-09-16T09:32:07.418344+0000","last_fullsized":"2024-09-16T09:32:07.418344+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:56.191220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:27:56.191220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"66'5","reported_seq":318,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.413594+0000","last_change":"2024-09-16T09:27:05.694733+0000","last_active":"2024-09-16T09:32:07.413594+0000","last_peered":"2024-09-16T09:32:07.413594+0000","last_clean":"2024-09-16T09:32:07.413594+0000","last_became_active":"2024-09-16T09:25:03.716136+0000","last_became_peered":"2024-09-16T09:25:03.716136+0000","last_unstale":"2024-09-16T09:32:07.413594+0000","last_undegraded":"2024-09-16T09:32:07.413594+0000","last_fullsized":"2024-09-16T09:32:07.413594+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-16T09:27:05.694650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:05.694650+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":311,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.417558+0000","last_change":"2024-09-16T09:28:52.746046+0000","last_active":"2024-09-16T09:32:07.417558+0000","last_peered":"2024-09-16T09:32:07.417558+0000","last_clean":"2024-09-16T09:32:07.417558+0000","last_became_active":"2024-09-16T09:25:05.718281+0000","last_became_peered":"2024-09-16T09:25:05.718281+0000","last_unstale":"2024-09-16T09:32:07.417558+0000","last_undegraded":"2024-09-16T09:32:07.417558+0000","last_fullsized":"2024-09-16T09:32:07.417558+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:52.745999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:28:52.745999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":329,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.417152+0000","last_change":"2024-09-16T09:31:09.276016+0000","last_active":"2024-09-16T09:32:07.417152+0000","last_peered":"2024-09-16T09:32:07.417152+0000","last_clean":"2024-09-16T09:32:07.417152+0000","last_became_active":"2024-09-16T09:31:09.275656+0000","last_became_peered":"2024-09-16T09:31:09.275656+0000","last_unstale":"2024-09-16T09:32:07.417152+0000","last_undegraded":"2024-09-16T09:32:07.417152+0000","last_fullsized":"2024-09-16T09:32:07.417152+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:50.468134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:50.468134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":317,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.413268+0000","last_change":"2024-09-16T09:29:56.605555+0000","last_active":"2024-09-16T09:32:07.413268+0000","last_peered":"2024-09-16T09:32:07.413268+0000","last_clean":"2024-09-16T09:32:07.413268+0000","last_became_active":"2024-09-16T09:25:07.720793+0000","last_became_peered":"2024-09-16T09:25:07.720793+0000","last_unstale":"2024-09-16T09:32:07.413268+0000","last_undegraded":"2024-09-16T09:32:07.413268+0000","last_fullsized":"2024-09-16T09:32:07.413268+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:56.605488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:56.605488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":68,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.425452+0000","last_change":"2024-09-16T09:30:39.202178+0000","last_active":"2024-09-16T09:32:07.425452+0000","last_peered":"2024-09-16T09:32:07.425452+0000","last_clean":"2024-09-16T09:32:07.425452+0000","last_became_active":"2024-09-16T09:30:39.201824+0000","last_became_peered":"2024-09-16T09:30:39.201824+0000","last_unstale":"2024-09-16T09:32:07.425452+0000","last_undegraded":"2024-09-16T09:32:07.425452+0000","last_fullsized":"2024-09-16T09:32:07.425452+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:50.912979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:50.912979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:20.431823+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"66'15","reported_seq":395,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.422885+0000","last_change":"2024-09-16T09:31:09.302018+0000","last_active":"2024-09-16T09:32:07.422885+0000","last_peered":"2024-09-16T09:32:07.422885+0000","last_clean":"2024-09-16T09:32:07.422885+0000","last_became_active":"2024-09-16T09:31:09.263534+0000","last_became_peered":"2024-09-16T09:31:09.263534+0000","last_unstale":"2024-09-16T09:32:07.422885+0000","last_undegraded":"2024-09-16T09:32:07.422885+0000","last_fullsized":"2024-09-16T09:32:07.422885+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-16T09:30:21.932604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:30:21.932604+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":312,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.425579+0000","last_change":"2024-09-16T09:29:55.875536+0000","last_active":"2024-09-16T09:32:07.425579+0000","last_peered":"2024-09-16T09:32:07.425579+0000","last_clean":"2024-09-16T09:32:07.425579+0000","last_became_active":"2024-09-16T09:25:05.713929+0000","last_became_peered":"2024-09-16T09:25:05.713929+0000","last_unstale":"2024-09-16T09:32:07.425579+0000","last_undegraded":"2024-09-16T09:32:07.425579+0000","last_fullsized":"2024-09-16T09:32:07.425579+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:55.875480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:55.875480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":165,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.471633+0000","last_change":"2024-09-16T09:30:24.412733+0000","last_active":"2024-09-16T09:32:07.471633+0000","last_peered":"2024-09-16T09:32:07.471633+0000","last_clean":"2024-09-16T09:32:07.471633+0000","last_became_active":"2024-09-16T09:28:52.974255+0000","last_became_peered":"2024-09-16T09:28:52.974255+0000","last_unstale":"2024-09-16T09:32:07.471633+0000","last_undegraded":"2024-09-16T09:32:07.471633+0000","last_fullsized":"2024-09-16T09:32:07.471633+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:30:24.412571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:30:24.412571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:49.424390+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":328,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.413690+0000","last_change":"2024-09-16T09:30:03.628882+0000","last_active":"2024-09-16T09:32:07.413690+0000","last_peered":"2024-09-16T09:32:07.413690+0000","last_clean":"2024-09-16T09:32:07.413690+0000","last_became_active":"2024-09-16T09:25:01.706491+0000","last_became_peered":"2024-09-16T09:25:01.706491+0000","last_unstale":"2024-09-16T09:32:07.413690+0000","last_undegraded":"2024-09-16T09:32:07.413690+0000","last_fullsized":"2024-09-16T09:32:07.413690+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:30:03.628812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:30:03.628812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"66'12","reported_seq":229,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.426109+0000","last_change":"2024-09-16T09:31:09.286673+0000","last_active":"2024-09-16T09:32:07.426109+0000","last_peered":"2024-09-16T09:32:07.426109+0000","last_clean":"2024-09-16T09:32:07.426109+0000","last_became_active":"2024-09-16T09:31:09.268588+0000","last_became_peered":"2024-09-16T09:31:09.268588+0000","last_unstale":"2024-09-16T09:32:07.426109+0000","last_undegraded":"2024-09-16T09:32:07.426109+0000","last_fullsized":"2024-09-16T09:32:07.426109+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-16T09:27:01.700139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:01.700139+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":24,"num_read_kb":15,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":304,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.413984+0000","last_change":"2024-09-16T09:27:00.816500+0000","last_active":"2024-09-16T09:32:07.413984+0000","last_peered":"2024-09-16T09:32:07.413984+0000","last_clean":"2024-09-16T09:32:07.413984+0000","last_became_active":"2024-09-16T09:25:05.710093+0000","last_became_peered":"2024-09-16T09:25:05.710093+0000","last_unstale":"2024-09-16T09:32:07.413984+0000","last_undegraded":"2024-09-16T09:32:07.413984+0000","last_fullsized":"2024-09-16T09:32:07.413984+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:00.816424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:00.816424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":308,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.418978+0000","last_change":"2024-09-16T09:29:29.918433+0000","last_active":"2024-09-16T09:32:07.418978+0000","last_peered":"2024-09-16T09:32:07.418978+0000","last_clean":"2024-09-16T09:32:07.418978+0000","last_became_active":"2024-09-16T09:25:07.719380+0000","last_became_peered":"2024-09-16T09:25:07.719380+0000","last_unstale":"2024-09-16T09:32:07.418978+0000","last_undegraded":"2024-09-16T09:32:07.418978+0000","last_fullsized":"2024-09-16T09:32:07.418978+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-16T09:29:29.918368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:29.918368+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":329,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.425695+0000","last_change":"2024-09-16T09:30:02.843932+0000","last_active":"2024-09-16T09:32:07.425695+0000","last_peered":"2024-09-16T09:32:07.425695+0000","last_clean":"2024-09-16T09:32:07.425695+0000","last_became_active":"2024-09-16T09:25:01.699545+0000","last_became_peered":"2024-09-16T09:25:01.699545+0000","last_unstale":"2024-09-16T09:32:07.425695+0000","last_undegraded":"2024-09-16T09:32:07.425695+0000","last_fullsized":"2024-09-16T09:32:07.425695+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:30:02.843856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:30:02.843856+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"66'10","reported_seq":370,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.428327+0000","last_change":"2024-09-16T09:31:09.328513+0000","last_active":"2024-09-16T09:32:07.428327+0000","last_peered":"2024-09-16T09:32:07.428327+0000","last_clean":"2024-09-16T09:32:07.428327+0000","last_became_active":"2024-09-16T09:31:09.275050+0000","last_became_peered":"2024-09-16T09:31:09.275050+0000","last_unstale":"2024-09-16T09:32:07.428327+0000","last_undegraded":"2024-09-16T09:32:07.428327+0000","last_fullsized":"2024-09-16T09:32:07.428327+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-16T09:27:43.556357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:43.556357+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:15.194576+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"66'8","reported_seq":426,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.471793+0000","last_change":"2024-09-16T09:30:31.392771+0000","last_active":"2024-09-16T09:32:07.471793+0000","last_peered":"2024-09-16T09:32:07.471793+0000","last_clean":"2024-09-16T09:32:07.471793+0000","last_became_active":"2024-09-16T09:25:05.712516+0000","last_became_peered":"2024-09-16T09:25:05.712516+0000","last_unstale":"2024-09-16T09:32:07.471793+0000","last_undegraded":"2024-09-16T09:32:07.471793+0000","last_fullsized":"2024-09-16T09:32:07.471793+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:30:31.392399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:30:31.392399+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:44.469073+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"204'10740","reported_seq":10856,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.618863+0000","last_change":"2024-09-16T09:29:49.449011+0000","last_active":"2024-09-16T09:32:07.618863+0000","last_peered":"2024-09-16T09:32:07.618863+0000","last_clean":"2024-09-16T09:32:07.618863+0000","last_became_active":"2024-09-16T09:29:49.448473+0000","last_became_peered":"2024-09-16T09:29:49.448473+0000","last_unstale":"2024-09-16T09:32:07.618863+0000","last_undegraded":"2024-09-16T09:32:07.618863+0000","last_fullsized":"2024-09-16T09:32:07.618863+0000","mapping_epoch":152,"log_start":"182'6200","ondisk_log_start":"182'6200","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":4540,"log_dups_size":0,"ondisk_log_size":4540,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":319684608,"num_objects":4878,"num_object_clones":0,"num_object_copies":14634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4878,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10740,"num_write_kb":499776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.a","version":"73'79","reported_seq":251,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.416443+0000","last_change":"2024-09-16T09:29:18.498887+0000","last_active":"2024-09-16T09:32:07.416443+0000","last_peered":"2024-09-16T09:32:07.416443+0000","last_clean":"2024-09-16T09:32:07.416443+0000","last_became_active":"2024-09-16T09:26:35.127352+0000","last_became_peered":"2024-09-16T09:26:35.127352+0000","last_unstale":"2024-09-16T09:32:07.416443+0000","last_undegraded":"2024-09-16T09:32:07.416443+0000","last_fullsized":"2024-09-16T09:32:07.416443+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:18.498766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:18.498766+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"66'15","reported_seq":260,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.422562+0000","last_change":"2024-09-16T09:28:28.065781+0000","last_active":"2024-09-16T09:32:07.422562+0000","last_peered":"2024-09-16T09:32:07.422562+0000","last_clean":"2024-09-16T09:32:07.422562+0000","last_became_active":"2024-09-16T09:26:33.295672+0000","last_became_peered":"2024-09-16T09:26:33.295672+0000","last_unstale":"2024-09-16T09:32:07.422562+0000","last_undegraded":"2024-09-16T09:32:07.422562+0000","last_fullsized":"2024-09-16T09:32:07.422562+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-16T09:28:28.065699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:28.065699+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":46,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.423283+0000","last_change":"2024-09-16T09:31:09.276858+0000","last_active":"2024-09-16T09:32:07.423283+0000","last_peered":"2024-09-16T09:32:07.423283+0000","last_clean":"2024-09-16T09:32:07.423283+0000","last_became_active":"2024-09-16T09:31:09.274534+0000","last_became_peered":"2024-09-16T09:31:09.274534+0000","last_unstale":"2024-09-16T09:32:07.423283+0000","last_undegraded":"2024-09-16T09:32:07.423283+0000","last_fullsized":"2024-09-16T09:32:07.423283+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:22.091537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:28:22.091537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"66'8","reported_seq":391,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.413362+0000","last_change":"2024-09-16T09:25:05.714128+0000","last_active":"2024-09-16T09:32:07.413362+0000","last_peered":"2024-09-16T09:32:07.413362+0000","last_clean":"2024-09-16T09:32:07.413362+0000","last_became_active":"2024-09-16T09:25:05.713813+0000","last_became_peered":"2024-09-16T09:25:05.713813+0000","last_unstale":"2024-09-16T09:32:07.413362+0000","last_undegraded":"2024-09-16T09:32:07.413362+0000","last_fullsized":"2024-09-16T09:32:07.413362+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":308,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.417935+0000","last_change":"2024-09-16T09:29:24.537802+0000","last_active":"2024-09-16T09:32:07.417935+0000","last_peered":"2024-09-16T09:32:07.417935+0000","last_clean":"2024-09-16T09:32:07.417935+0000","last_became_active":"2024-09-16T09:25:07.717344+0000","last_became_peered":"2024-09-16T09:25:07.717344+0000","last_unstale":"2024-09-16T09:32:07.417935+0000","last_undegraded":"2024-09-16T09:32:07.417935+0000","last_fullsized":"2024-09-16T09:32:07.417935+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:24.537731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:24.537731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.c","version":"204'10667","reported_seq":10841,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.676399+0000","last_change":"2024-09-16T09:32:07.676294+0000","last_active":"2024-09-16T09:32:07.676399+0000","last_peered":"2024-09-16T09:32:07.676399+0000","last_clean":"2024-09-16T09:32:07.676399+0000","last_became_active":"2024-09-16T09:29:50.007935+0000","last_became_peered":"2024-09-16T09:29:50.007935+0000","last_unstale":"2024-09-16T09:32:07.676399+0000","last_undegraded":"2024-09-16T09:32:07.676399+0000","last_fullsized":"2024-09-16T09:32:07.676399+0000","mapping_epoch":152,"log_start":"180'5500","ondisk_log_start":"180'5500","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"204'10667","last_scrub_stamp":"2024-09-16T09:32:07.676020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:32:07.676020+0000","objects_scrubbed":4920,"log_size":5167,"log_dups_size":0,"ondisk_log_size":5167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:18.894513+0000","scrub_duration":1433,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":319881216,"num_objects":4881,"num_object_clones":0,"num_object_copies":14643,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4881,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10667,"num_write_kb":497536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"73'79","reported_seq":280,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.415062+0000","last_change":"2024-09-16T09:31:47.249715+0000","last_active":"2024-09-16T09:32:07.415062+0000","last_peered":"2024-09-16T09:32:07.415062+0000","last_clean":"2024-09-16T09:32:07.415062+0000","last_became_active":"2024-09-16T09:30:08.763683+0000","last_became_peered":"2024-09-16T09:30:08.763683+0000","last_unstale":"2024-09-16T09:32:07.415062+0000","last_undegraded":"2024-09-16T09:32:07.415062+0000","last_fullsized":"2024-09-16T09:32:07.415062+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:58.568841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:58.568841+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:32:11.825261+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"66'18","reported_seq":258,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.419025+0000","last_change":"2024-09-16T09:31:09.407700+0000","last_active":"2024-09-16T09:32:07.419025+0000","last_peered":"2024-09-16T09:32:07.419025+0000","last_clean":"2024-09-16T09:32:07.419025+0000","last_became_active":"2024-09-16T09:31:09.270434+0000","last_became_peered":"2024-09-16T09:31:09.270434+0000","last_unstale":"2024-09-16T09:32:07.419025+0000","last_undegraded":"2024-09-16T09:32:07.419025+0000","last_fullsized":"2024-09-16T09:32:07.419025+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'18","last_scrub_stamp":"2024-09-16T09:28:54.699778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:54.699778+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":33,"num_read_kb":21,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":156,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.419779+0000","last_change":"2024-09-16T09:28:58.705169+0000","last_active":"2024-09-16T09:32:07.419779+0000","last_peered":"2024-09-16T09:32:07.419779+0000","last_clean":"2024-09-16T09:32:07.419779+0000","last_became_active":"2024-09-16T09:28:52.983543+0000","last_became_peered":"2024-09-16T09:28:52.983543+0000","last_unstale":"2024-09-16T09:32:07.419779+0000","last_undegraded":"2024-09-16T09:32:07.419779+0000","last_fullsized":"2024-09-16T09:32:07.419779+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:58.705097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:28:58.705097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":335,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.412374+0000","last_change":"2024-09-16T09:31:09.269234+0000","last_active":"2024-09-16T09:32:07.412374+0000","last_peered":"2024-09-16T09:32:07.412374+0000","last_clean":"2024-09-16T09:32:07.412374+0000","last_became_active":"2024-09-16T09:31:09.268864+0000","last_became_peered":"2024-09-16T09:31:09.268864+0000","last_unstale":"2024-09-16T09:32:07.412374+0000","last_undegraded":"2024-09-16T09:32:07.412374+0000","last_fullsized":"2024-09-16T09:32:07.412374+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:50.482240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:50.482240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":315,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.413482+0000","last_change":"2024-09-16T09:31:09.269842+0000","last_active":"2024-09-16T09:32:07.413482+0000","last_peered":"2024-09-16T09:32:07.413482+0000","last_clean":"2024-09-16T09:32:07.413482+0000","last_became_active":"2024-09-16T09:31:09.269433+0000","last_became_peered":"2024-09-16T09:31:09.269433+0000","last_unstale":"2024-09-16T09:32:07.413482+0000","last_undegraded":"2024-09-16T09:32:07.413482+0000","last_fullsized":"2024-09-16T09:32:07.413482+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:13.548608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:13.548608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.d","version":"204'10695","reported_seq":10812,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.603386+0000","last_change":"2024-09-16T09:29:50.010860+0000","last_active":"2024-09-16T09:32:07.603386+0000","last_peered":"2024-09-16T09:32:07.603386+0000","last_clean":"2024-09-16T09:32:07.603386+0000","last_became_active":"2024-09-16T09:29:50.010421+0000","last_became_peered":"2024-09-16T09:29:50.010421+0000","last_unstale":"2024-09-16T09:32:07.603386+0000","last_undegraded":"2024-09-16T09:32:07.603386+0000","last_fullsized":"2024-09-16T09:32:07.603386+0000","mapping_epoch":152,"log_start":"182'6000","ondisk_log_start":"182'6000","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":4695,"log_dups_size":0,"ondisk_log_size":4695,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":319160344,"num_objects":4871,"num_object_clones":0,"num_object_copies":14613,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4871,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10695,"num_write_kb":498049,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,2],"acting":[5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.4","version":"73'79","reported_seq":46,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.420225+0000","last_change":"2024-09-16T09:31:09.275499+0000","last_active":"2024-09-16T09:32:07.420225+0000","last_peered":"2024-09-16T09:32:07.420225+0000","last_clean":"2024-09-16T09:32:07.420225+0000","last_became_active":"2024-09-16T09:31:09.275139+0000","last_became_peered":"2024-09-16T09:31:09.275139+0000","last_unstale":"2024-09-16T09:32:07.420225+0000","last_undegraded":"2024-09-16T09:32:07.420225+0000","last_fullsized":"2024-09-16T09:32:07.420225+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:30:22.438378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:30:22.438378+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"66'14","reported_seq":338,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.412454+0000","last_change":"2024-09-16T09:28:32.595064+0000","last_active":"2024-09-16T09:32:07.412454+0000","last_peered":"2024-09-16T09:32:07.412454+0000","last_clean":"2024-09-16T09:32:07.412454+0000","last_became_active":"2024-09-16T09:25:03.708830+0000","last_became_peered":"2024-09-16T09:25:03.708830+0000","last_unstale":"2024-09-16T09:32:07.412454+0000","last_undegraded":"2024-09-16T09:32:07.412454+0000","last_fullsized":"2024-09-16T09:32:07.412454+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'14","last_scrub_stamp":"2024-09-16T09:28:32.594988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:32.594988+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":338,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.428609+0000","last_change":"2024-09-16T09:31:57.373606+0000","last_active":"2024-09-16T09:32:07.428609+0000","last_peered":"2024-09-16T09:32:07.428609+0000","last_clean":"2024-09-16T09:32:07.428609+0000","last_became_active":"2024-09-16T09:25:01.702992+0000","last_became_peered":"2024-09-16T09:25:01.702992+0000","last_unstale":"2024-09-16T09:32:07.428609+0000","last_undegraded":"2024-09-16T09:32:07.428609+0000","last_fullsized":"2024-09-16T09:32:07.428609+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:31:57.373544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:31:57.373544+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:24.193683+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"187'11","reported_seq":243,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.424599+0000","last_change":"2024-09-16T09:31:09.397758+0000","last_active":"2024-09-16T09:32:07.424599+0000","last_peered":"2024-09-16T09:32:07.424599+0000","last_clean":"2024-09-16T09:32:07.424599+0000","last_became_active":"2024-09-16T09:31:09.269173+0000","last_became_peered":"2024-09-16T09:31:09.269173+0000","last_unstale":"2024-09-16T09:32:07.424599+0000","last_undegraded":"2024-09-16T09:32:07.424599+0000","last_fullsized":"2024-09-16T09:32:07.424599+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'9","last_scrub_stamp":"2024-09-16T09:29:21.898533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:21.898533+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":300,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.417765+0000","last_change":"2024-09-16T09:26:40.699888+0000","last_active":"2024-09-16T09:32:07.417765+0000","last_peered":"2024-09-16T09:32:07.417765+0000","last_clean":"2024-09-16T09:32:07.417765+0000","last_became_active":"2024-09-16T09:25:07.722017+0000","last_became_peered":"2024-09-16T09:25:07.722017+0000","last_unstale":"2024-09-16T09:32:07.417765+0000","last_undegraded":"2024-09-16T09:32:07.417765+0000","last_fullsized":"2024-09-16T09:32:07.417765+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:40.699813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:40.699813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.e","version":"204'10713","reported_seq":10829,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.625259+0000","last_change":"2024-09-16T09:29:49.448081+0000","last_active":"2024-09-16T09:32:07.625259+0000","last_peered":"2024-09-16T09:32:07.625259+0000","last_clean":"2024-09-16T09:32:07.625259+0000","last_became_active":"2024-09-16T09:29:49.447767+0000","last_became_peered":"2024-09-16T09:29:49.447767+0000","last_unstale":"2024-09-16T09:32:07.625259+0000","last_undegraded":"2024-09-16T09:32:07.625259+0000","last_fullsized":"2024-09-16T09:32:07.625259+0000","mapping_epoch":152,"log_start":"180'5600","ondisk_log_start":"180'5600","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":5113,"log_dups_size":0,"ondisk_log_size":5113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":320667648,"num_objects":4893,"num_object_clones":0,"num_object_copies":14679,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4893,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10713,"num_write_kb":499392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"73'79","reported_seq":369,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.471521+0000","last_change":"2024-09-16T09:31:09.275413+0000","last_active":"2024-09-16T09:32:07.471521+0000","last_peered":"2024-09-16T09:32:07.471521+0000","last_clean":"2024-09-16T09:32:07.471521+0000","last_became_active":"2024-09-16T09:31:09.274707+0000","last_became_peered":"2024-09-16T09:31:09.274707+0000","last_unstale":"2024-09-16T09:32:07.471521+0000","last_undegraded":"2024-09-16T09:32:07.471521+0000","last_fullsized":"2024-09-16T09:32:07.471521+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:30:23.455388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:30:23.455388+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:41.421930+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"66'10","reported_seq":340,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.424988+0000","last_change":"2024-09-16T09:30:04.848306+0000","last_active":"2024-09-16T09:32:07.424988+0000","last_peered":"2024-09-16T09:32:07.424988+0000","last_clean":"2024-09-16T09:32:07.424988+0000","last_became_active":"2024-09-16T09:25:03.712985+0000","last_became_peered":"2024-09-16T09:25:03.712985+0000","last_unstale":"2024-09-16T09:32:07.424988+0000","last_undegraded":"2024-09-16T09:32:07.424988+0000","last_fullsized":"2024-09-16T09:32:07.424988+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-16T09:30:04.848234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:30:04.848234+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":326,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.417854+0000","last_change":"2024-09-16T09:31:09.272049+0000","last_active":"2024-09-16T09:32:07.417854+0000","last_peered":"2024-09-16T09:32:07.417854+0000","last_clean":"2024-09-16T09:32:07.417854+0000","last_became_active":"2024-09-16T09:31:09.270619+0000","last_became_peered":"2024-09-16T09:31:09.270619+0000","last_unstale":"2024-09-16T09:32:07.417854+0000","last_undegraded":"2024-09-16T09:32:07.417854+0000","last_fullsized":"2024-09-16T09:32:07.417854+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:25.086322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:25.086322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"66'8","reported_seq":407,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.427237+0000","last_change":"2024-09-16T09:28:57.628615+0000","last_active":"2024-09-16T09:32:07.427237+0000","last_peered":"2024-09-16T09:32:07.427237+0000","last_clean":"2024-09-16T09:32:07.427237+0000","last_became_active":"2024-09-16T09:25:05.714507+0000","last_became_peered":"2024-09-16T09:25:05.714507+0000","last_unstale":"2024-09-16T09:32:07.427237+0000","last_undegraded":"2024-09-16T09:32:07.427237+0000","last_fullsized":"2024-09-16T09:32:07.427237+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:28:57.628543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:28:57.628543+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:51.253847+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":323,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.427110+0000","last_change":"2024-09-16T09:31:09.283816+0000","last_active":"2024-09-16T09:32:07.427110+0000","last_peered":"2024-09-16T09:32:07.427110+0000","last_clean":"2024-09-16T09:32:07.427110+0000","last_became_active":"2024-09-16T09:31:09.283192+0000","last_became_peered":"2024-09-16T09:31:09.283192+0000","last_unstale":"2024-09-16T09:32:07.427110+0000","last_undegraded":"2024-09-16T09:32:07.427110+0000","last_fullsized":"2024-09-16T09:32:07.427110+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:22.563395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:28:22.563395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:22.152175+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.f","version":"204'10725","reported_seq":10841,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.641830+0000","last_change":"2024-09-16T09:29:49.838498+0000","last_active":"2024-09-16T09:32:07.641830+0000","last_peered":"2024-09-16T09:32:07.641830+0000","last_clean":"2024-09-16T09:32:07.641830+0000","last_became_active":"2024-09-16T09:29:49.838146+0000","last_became_peered":"2024-09-16T09:29:49.838146+0000","last_unstale":"2024-09-16T09:32:07.641830+0000","last_undegraded":"2024-09-16T09:32:07.641830+0000","last_fullsized":"2024-09-16T09:32:07.641830+0000","mapping_epoch":152,"log_start":"177'4800","ondisk_log_start":"177'4800","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":5925,"log_dups_size":0,"ondisk_log_size":5925,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":329056256,"num_objects":5021,"num_object_clones":0,"num_object_copies":15063,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5021,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10725,"num_write_kb":503872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"73'79","reported_seq":350,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.469506+0000","last_change":"2024-09-16T09:31:09.274975+0000","last_active":"2024-09-16T09:32:07.469506+0000","last_peered":"2024-09-16T09:32:07.469506+0000","last_clean":"2024-09-16T09:32:07.469506+0000","last_became_active":"2024-09-16T09:31:09.274565+0000","last_became_peered":"2024-09-16T09:31:09.274565+0000","last_unstale":"2024-09-16T09:32:07.469506+0000","last_undegraded":"2024-09-16T09:32:07.469506+0000","last_fullsized":"2024-09-16T09:32:07.469506+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:30:22.415051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:30:22.415051+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:52.385618+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"66'19","reported_seq":353,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.427454+0000","last_change":"2024-09-16T09:31:35.249952+0000","last_active":"2024-09-16T09:32:07.427454+0000","last_peered":"2024-09-16T09:32:07.427454+0000","last_clean":"2024-09-16T09:32:07.427454+0000","last_became_active":"2024-09-16T09:25:03.714797+0000","last_became_peered":"2024-09-16T09:25:03.714797+0000","last_unstale":"2024-09-16T09:32:07.427454+0000","last_undegraded":"2024-09-16T09:32:07.427454+0000","last_fullsized":"2024-09-16T09:32:07.427454+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-16T09:31:35.249864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:31:35.249864+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:40.632025+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":329,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.425474+0000","last_change":"2024-09-16T09:29:16.888357+0000","last_active":"2024-09-16T09:32:07.425474+0000","last_peered":"2024-09-16T09:32:07.425474+0000","last_clean":"2024-09-16T09:32:07.425474+0000","last_became_active":"2024-09-16T09:25:01.706806+0000","last_became_peered":"2024-09-16T09:25:01.706806+0000","last_unstale":"2024-09-16T09:32:07.425474+0000","last_undegraded":"2024-09-16T09:32:07.425474+0000","last_fullsized":"2024-09-16T09:32:07.425474+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:16.888236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:16.888236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":329,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.466916+0000","last_change":"2024-09-16T09:31:55.456669+0000","last_active":"2024-09-16T09:32:07.466916+0000","last_peered":"2024-09-16T09:32:07.466916+0000","last_clean":"2024-09-16T09:32:07.466916+0000","last_became_active":"2024-09-16T09:25:05.708663+0000","last_became_peered":"2024-09-16T09:25:05.708663+0000","last_unstale":"2024-09-16T09:32:07.466916+0000","last_undegraded":"2024-09-16T09:32:07.466916+0000","last_fullsized":"2024-09-16T09:32:07.466916+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:31:55.456592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:31:55.456592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:56.006473+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":301,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.425432+0000","last_change":"2024-09-16T09:26:37.282798+0000","last_active":"2024-09-16T09:32:07.425432+0000","last_peered":"2024-09-16T09:32:07.425432+0000","last_clean":"2024-09-16T09:32:07.425432+0000","last_became_active":"2024-09-16T09:25:07.719267+0000","last_became_peered":"2024-09-16T09:25:07.719267+0000","last_unstale":"2024-09-16T09:32:07.425432+0000","last_undegraded":"2024-09-16T09:32:07.425432+0000","last_fullsized":"2024-09-16T09:32:07.425432+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:37.282735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:37.282735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.8","version":"204'10818","reported_seq":16103,"reported_epoch":204,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-16T09:32:07.930027+0000","last_change":"2024-09-16T09:31:09.438251+0000","last_active":"2024-09-16T09:32:07.930027+0000","last_peered":"2024-09-16T09:32:07.930027+0000","last_clean":"2024-09-16T09:30:20.474651+0000","last_became_active":"2024-09-16T09:31:09.409873+0000","last_became_peered":"2024-09-16T09:31:09.409873+0000","last_unstale":"2024-09-16T09:32:07.930027+0000","last_undegraded":"2024-09-16T09:32:07.930027+0000","last_fullsized":"2024-09-16T09:31:09.256524+0000","mapping_epoch":186,"log_start":"182'6100","ondisk_log_start":"182'6100","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":4718,"log_dups_size":0,"ondisk_log_size":4718,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":321388544,"num_objects":4904,"num_object_clones":0,"num_object_copies":14712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1877,"num_objects_unfound":0,"num_objects_dirty":4904,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10818,"num_write_kb":503104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2579,"num_bytes_recovered":169017344,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,7],"avail_no_missing":["5","7"],"object_location_counts":[{"shards":"0,5,7","objects":1877},{"shards":"5,7","objects":3027}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1","version":"73'79","reported_seq":272,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.416276+0000","last_change":"2024-09-16T09:31:47.248649+0000","last_active":"2024-09-16T09:32:07.416276+0000","last_peered":"2024-09-16T09:32:07.416276+0000","last_clean":"2024-09-16T09:32:07.416276+0000","last_became_active":"2024-09-16T09:30:08.766350+0000","last_became_peered":"2024-09-16T09:30:08.766350+0000","last_unstale":"2024-09-16T09:32:07.416276+0000","last_undegraded":"2024-09-16T09:32:07.416276+0000","last_fullsized":"2024-09-16T09:32:07.416276+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:15.460179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:15.460179+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:32:12.387091+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"187'30","reported_seq":509,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.425285+0000","last_change":"2024-09-16T09:31:09.323281+0000","last_active":"2024-09-16T09:32:07.425285+0000","last_peered":"2024-09-16T09:32:07.425285+0000","last_clean":"2024-09-16T09:32:07.425285+0000","last_became_active":"2024-09-16T09:31:09.278839+0000","last_became_peered":"2024-09-16T09:31:09.278839+0000","last_unstale":"2024-09-16T09:32:07.425285+0000","last_undegraded":"2024-09-16T09:32:07.425285+0000","last_fullsized":"2024-09-16T09:32:07.425285+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'28","last_scrub_stamp":"2024-09-16T09:29:18.937623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:29:18.937623+0000","objects_scrubbed":10,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":321,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.413879+0000","last_change":"2024-09-16T09:29:26.637070+0000","last_active":"2024-09-16T09:32:07.413879+0000","last_peered":"2024-09-16T09:32:07.413879+0000","last_clean":"2024-09-16T09:32:07.413879+0000","last_became_active":"2024-09-16T09:25:01.703076+0000","last_became_peered":"2024-09-16T09:25:01.703076+0000","last_unstale":"2024-09-16T09:32:07.413879+0000","last_undegraded":"2024-09-16T09:32:07.413879+0000","last_fullsized":"2024-09-16T09:32:07.413879+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:26.637027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:26.637027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":330,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.418687+0000","last_change":"2024-09-16T09:29:29.486284+0000","last_active":"2024-09-16T09:32:07.418687+0000","last_peered":"2024-09-16T09:32:07.418687+0000","last_clean":"2024-09-16T09:32:07.418687+0000","last_became_active":"2024-09-16T09:25:00.696237+0000","last_became_peered":"2024-09-16T09:25:00.696237+0000","last_unstale":"2024-09-16T09:32:07.418687+0000","last_undegraded":"2024-09-16T09:32:07.418687+0000","last_fullsized":"2024-09-16T09:32:07.418687+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-16T09:29:29.486188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:29:29.486188+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.0025137430000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":297,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.427932+0000","last_change":"2024-09-16T09:25:05.718319+0000","last_active":"2024-09-16T09:32:07.427932+0000","last_peered":"2024-09-16T09:32:07.427932+0000","last_clean":"2024-09-16T09:32:07.427932+0000","last_became_active":"2024-09-16T09:25:05.717591+0000","last_became_peered":"2024-09-16T09:25:05.717591+0000","last_unstale":"2024-09-16T09:32:07.427932+0000","last_undegraded":"2024-09-16T09:32:07.427932+0000","last_fullsized":"2024-09-16T09:32:07.427932+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:10.167420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":46,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.419475+0000","last_change":"2024-09-16T09:31:09.272967+0000","last_active":"2024-09-16T09:32:07.419475+0000","last_peered":"2024-09-16T09:32:07.419475+0000","last_clean":"2024-09-16T09:32:07.419475+0000","last_became_active":"2024-09-16T09:31:09.272609+0000","last_became_peered":"2024-09-16T09:31:09.272609+0000","last_unstale":"2024-09-16T09:32:07.419475+0000","last_undegraded":"2024-09-16T09:32:07.419475+0000","last_fullsized":"2024-09-16T09:32:07.419475+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:45.201503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:45.201503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"203'10640","reported_seq":21868,"reported_epoch":204,"state":"peering","last_fresh":"2024-09-16T09:32:07.416644+0000","last_change":"2024-09-16T09:32:07.416644+0000","last_active":"2024-09-16T09:32:07.416644+0000","last_peered":"2024-09-16T09:32:07.399382+0000","last_clean":"2024-09-16T09:30:20.492754+0000","last_became_active":"2024-09-16T09:31:09.398447+0000","last_became_peered":"2024-09-16T09:31:09.398447+0000","last_unstale":"2024-09-16T09:32:07.416644+0000","last_undegraded":"2024-09-16T09:32:07.416644+0000","last_fullsized":"2024-09-16T09:32:07.416644+0000","mapping_epoch":204,"log_start":"184'6400","ondisk_log_start":"184'6400","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":4240,"log_dups_size":0,"ondisk_log_size":4240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":320733184,"num_objects":4894,"num_object_clones":0,"num_object_copies":14682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4894,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10640,"num_write_kb":497088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5551,"num_bytes_recovered":363790336,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.2","version":"73'79","reported_seq":397,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.417397+0000","last_change":"2024-09-16T09:31:47.263716+0000","last_active":"2024-09-16T09:32:07.417397+0000","last_peered":"2024-09-16T09:32:07.417397+0000","last_clean":"2024-09-16T09:32:07.417397+0000","last_became_active":"2024-09-16T09:30:08.765943+0000","last_became_peered":"2024-09-16T09:30:08.765943+0000","last_unstale":"2024-09-16T09:32:07.417397+0000","last_undegraded":"2024-09-16T09:32:07.417397+0000","last_fullsized":"2024-09-16T09:32:07.417397+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:26:35.745837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:26:35.745837+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:32:11.475590+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"66'13","reported_seq":344,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.426196+0000","last_change":"2024-09-16T09:27:44.156057+0000","last_active":"2024-09-16T09:32:07.426196+0000","last_peered":"2024-09-16T09:32:07.426196+0000","last_clean":"2024-09-16T09:32:07.426196+0000","last_became_active":"2024-09-16T09:25:03.713241+0000","last_became_peered":"2024-09-16T09:25:03.713241+0000","last_unstale":"2024-09-16T09:32:07.426196+0000","last_undegraded":"2024-09-16T09:32:07.426196+0000","last_fullsized":"2024-09-16T09:32:07.426196+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'13","last_scrub_stamp":"2024-09-16T09:27:44.155969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:44.155969+0000","objects_scrubbed":5,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":29,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":321,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.425069+0000","last_change":"2024-09-16T09:29:50.827103+0000","last_active":"2024-09-16T09:32:07.425069+0000","last_peered":"2024-09-16T09:32:07.425069+0000","last_clean":"2024-09-16T09:32:07.425069+0000","last_became_active":"2024-09-16T09:25:01.706828+0000","last_became_peered":"2024-09-16T09:25:01.706828+0000","last_unstale":"2024-09-16T09:32:07.425069+0000","last_undegraded":"2024-09-16T09:32:07.425069+0000","last_fullsized":"2024-09-16T09:32:07.425069+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:50.827057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:50.827057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":352,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.417460+0000","last_change":"2024-09-16T09:31:09.281163+0000","last_active":"2024-09-16T09:32:07.417460+0000","last_peered":"2024-09-16T09:32:07.417460+0000","last_clean":"2024-09-16T09:32:07.417460+0000","last_became_active":"2024-09-16T09:31:09.266338+0000","last_became_peered":"2024-09-16T09:31:09.266338+0000","last_unstale":"2024-09-16T09:32:07.417460+0000","last_undegraded":"2024-09-16T09:32:07.417460+0000","last_fullsized":"2024-09-16T09:32:07.417460+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:29:16.104209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:29:16.104209+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0.00055022299999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":321,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.417351+0000","last_change":"2024-09-16T09:29:00.169141+0000","last_active":"2024-09-16T09:32:07.417351+0000","last_peered":"2024-09-16T09:32:07.417351+0000","last_clean":"2024-09-16T09:32:07.417351+0000","last_became_active":"2024-09-16T09:25:05.711539+0000","last_became_peered":"2024-09-16T09:25:05.711539+0000","last_unstale":"2024-09-16T09:32:07.417351+0000","last_undegraded":"2024-09-16T09:32:07.417351+0000","last_fullsized":"2024-09-16T09:32:07.417351+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:00.169077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:00.169077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":308,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.420115+0000","last_change":"2024-09-16T09:29:22.968971+0000","last_active":"2024-09-16T09:32:07.420115+0000","last_peered":"2024-09-16T09:32:07.420115+0000","last_clean":"2024-09-16T09:32:07.420115+0000","last_became_active":"2024-09-16T09:25:07.723282+0000","last_became_peered":"2024-09-16T09:25:07.723282+0000","last_unstale":"2024-09-16T09:32:07.420115+0000","last_undegraded":"2024-09-16T09:32:07.420115+0000","last_fullsized":"2024-09-16T09:32:07.420115+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:22.968908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:22.968908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"204'10635","reported_seq":10751,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.623034+0000","last_change":"2024-09-16T09:29:49.448898+0000","last_active":"2024-09-16T09:32:07.623034+0000","last_peered":"2024-09-16T09:32:07.623034+0000","last_clean":"2024-09-16T09:32:07.623034+0000","last_became_active":"2024-09-16T09:29:49.448628+0000","last_became_peered":"2024-09-16T09:29:49.448628+0000","last_unstale":"2024-09-16T09:32:07.623034+0000","last_undegraded":"2024-09-16T09:32:07.623034+0000","last_fullsized":"2024-09-16T09:32:07.623034+0000","mapping_epoch":152,"log_start":"182'6100","ondisk_log_start":"182'6100","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":4535,"log_dups_size":0,"ondisk_log_size":4535,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327614464,"num_objects":4999,"num_object_clones":0,"num_object_copies":14997,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4999,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10635,"num_write_kb":500288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"73'79","reported_seq":282,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.427398+0000","last_change":"2024-09-16T09:31:40.212968+0000","last_active":"2024-09-16T09:32:07.427398+0000","last_peered":"2024-09-16T09:32:07.427398+0000","last_clean":"2024-09-16T09:32:07.427398+0000","last_became_active":"2024-09-16T09:30:08.877110+0000","last_became_peered":"2024-09-16T09:30:08.877110+0000","last_unstale":"2024-09-16T09:32:07.427398+0000","last_undegraded":"2024-09-16T09:32:07.427398+0000","last_fullsized":"2024-09-16T09:32:07.427398+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:31:40.212889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:31:40.212889+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:45.708324+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"66'12","reported_seq":327,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.428425+0000","last_change":"2024-09-16T09:29:02.713335+0000","last_active":"2024-09-16T09:32:07.428425+0000","last_peered":"2024-09-16T09:32:07.428425+0000","last_clean":"2024-09-16T09:32:07.428425+0000","last_became_active":"2024-09-16T09:25:03.715315+0000","last_became_peered":"2024-09-16T09:25:03.715315+0000","last_unstale":"2024-09-16T09:32:07.428425+0000","last_undegraded":"2024-09-16T09:32:07.428425+0000","last_fullsized":"2024-09-16T09:32:07.428425+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-16T09:29:02.713259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:29:02.713259+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:43.220661+0000","scrub_duration":26,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"59'1","reported_seq":344,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.428451+0000","last_change":"2024-09-16T09:31:09.286309+0000","last_active":"2024-09-16T09:32:07.428451+0000","last_peered":"2024-09-16T09:32:07.428451+0000","last_clean":"2024-09-16T09:32:07.428451+0000","last_became_active":"2024-09-16T09:31:09.269627+0000","last_became_peered":"2024-09-16T09:31:09.269627+0000","last_unstale":"2024-09-16T09:32:07.428451+0000","last_undegraded":"2024-09-16T09:32:07.428451+0000","last_fullsized":"2024-09-16T09:32:07.428451+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:27:00.666673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:27:00.666673+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:13.191274+0000","scrub_duration":28,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1986,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"67'5","reported_seq":751,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.419177+0000","last_change":"2024-09-16T09:29:19.994791+0000","last_active":"2024-09-16T09:32:07.419177+0000","last_peered":"2024-09-16T09:32:07.419177+0000","last_clean":"2024-09-16T09:32:07.419177+0000","last_became_active":"2024-09-16T09:25:00.696546+0000","last_became_peered":"2024-09-16T09:25:00.696546+0000","last_unstale":"2024-09-16T09:32:07.419177+0000","last_undegraded":"2024-09-16T09:32:07.419177+0000","last_fullsized":"2024-09-16T09:32:07.419177+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'5","last_scrub_stamp":"2024-09-16T09:29:19.994716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:29:19.994716+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0.0059770420000000001,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":420,"num_read_kb":415,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":304,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.417879+0000","last_change":"2024-09-16T09:27:07.337170+0000","last_active":"2024-09-16T09:32:07.417879+0000","last_peered":"2024-09-16T09:32:07.417879+0000","last_clean":"2024-09-16T09:32:07.417879+0000","last_became_active":"2024-09-16T09:25:05.714704+0000","last_became_peered":"2024-09-16T09:25:05.714704+0000","last_unstale":"2024-09-16T09:32:07.417879+0000","last_undegraded":"2024-09-16T09:32:07.417879+0000","last_fullsized":"2024-09-16T09:32:07.417879+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:07.337124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:07.337124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":315,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.425002+0000","last_change":"2024-09-16T09:31:09.271453+0000","last_active":"2024-09-16T09:32:07.425002+0000","last_peered":"2024-09-16T09:32:07.425002+0000","last_clean":"2024-09-16T09:32:07.425002+0000","last_became_active":"2024-09-16T09:31:09.271111+0000","last_became_peered":"2024-09-16T09:31:09.271111+0000","last_unstale":"2024-09-16T09:32:07.425002+0000","last_undegraded":"2024-09-16T09:32:07.425002+0000","last_fullsized":"2024-09-16T09:32:07.425002+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:24.231167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:24.231167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"204'10629","reported_seq":10810,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.554633+0000","last_change":"2024-09-16T09:30:56.023710+0000","last_active":"2024-09-16T09:32:07.554633+0000","last_peered":"2024-09-16T09:32:07.554633+0000","last_clean":"2024-09-16T09:32:07.554633+0000","last_became_active":"2024-09-16T09:29:49.450643+0000","last_became_peered":"2024-09-16T09:29:49.450643+0000","last_unstale":"2024-09-16T09:32:07.554633+0000","last_undegraded":"2024-09-16T09:32:07.554633+0000","last_fullsized":"2024-09-16T09:32:07.554633+0000","mapping_epoch":152,"log_start":"180'5400","ondisk_log_start":"180'5400","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"181'5780","last_scrub_stamp":"2024-09-16T09:30:56.023622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:30:56.023622+0000","objects_scrubbed":5716,"log_size":5229,"log_dups_size":0,"ondisk_log_size":5229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"queued for scrub","scrub_duration":1526,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":323420160,"num_objects":4935,"num_object_clones":0,"num_object_copies":14805,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4935,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10629,"num_write_kb":498048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"66'16","reported_seq":433,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.467895+0000","last_change":"2024-09-16T09:32:01.564577+0000","last_active":"2024-09-16T09:32:07.467895+0000","last_peered":"2024-09-16T09:32:07.467895+0000","last_clean":"2024-09-16T09:32:07.467895+0000","last_became_active":"2024-09-16T09:25:03.714118+0000","last_became_peered":"2024-09-16T09:25:03.714118+0000","last_unstale":"2024-09-16T09:32:07.467895+0000","last_undegraded":"2024-09-16T09:32:07.467895+0000","last_fullsized":"2024-09-16T09:32:07.467895+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'16","last_scrub_stamp":"2024-09-16T09:32:01.564489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:32:01.564489+0000","objects_scrubbed":8,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:10.625142+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":162,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.418606+0000","last_change":"2024-09-16T09:31:09.270603+0000","last_active":"2024-09-16T09:32:07.418606+0000","last_peered":"2024-09-16T09:32:07.418606+0000","last_clean":"2024-09-16T09:32:07.418606+0000","last_became_active":"2024-09-16T09:31:09.270196+0000","last_became_peered":"2024-09-16T09:31:09.270196+0000","last_unstale":"2024-09-16T09:32:07.418606+0000","last_undegraded":"2024-09-16T09:32:07.418606+0000","last_fullsized":"2024-09-16T09:32:07.418606+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:49.195331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:27:49.195331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"73'79","reported_seq":397,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.419375+0000","last_change":"2024-09-16T09:31:47.263760+0000","last_active":"2024-09-16T09:32:07.419375+0000","last_peered":"2024-09-16T09:32:07.419375+0000","last_clean":"2024-09-16T09:32:07.419375+0000","last_became_active":"2024-09-16T09:30:08.870394+0000","last_became_peered":"2024-09-16T09:30:08.870394+0000","last_unstale":"2024-09-16T09:32:07.419375+0000","last_undegraded":"2024-09-16T09:32:07.419375+0000","last_fullsized":"2024-09-16T09:32:07.419375+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:02.723794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:02.723794+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:32:12.473499+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":304,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.419470+0000","last_change":"2024-09-16T09:26:39.699133+0000","last_active":"2024-09-16T09:32:07.419470+0000","last_peered":"2024-09-16T09:32:07.419470+0000","last_clean":"2024-09-16T09:32:07.419470+0000","last_became_active":"2024-09-16T09:25:05.714341+0000","last_became_peered":"2024-09-16T09:25:05.714341+0000","last_unstale":"2024-09-16T09:32:07.419470+0000","last_undegraded":"2024-09-16T09:32:07.419470+0000","last_fullsized":"2024-09-16T09:32:07.419470+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:39.699059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:26:39.699059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":308,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.417145+0000","last_change":"2024-09-16T09:29:00.243528+0000","last_active":"2024-09-16T09:32:07.417145+0000","last_peered":"2024-09-16T09:32:07.417145+0000","last_clean":"2024-09-16T09:32:07.417145+0000","last_became_active":"2024-09-16T09:25:07.714281+0000","last_became_peered":"2024-09-16T09:25:07.714281+0000","last_unstale":"2024-09-16T09:32:07.417145+0000","last_undegraded":"2024-09-16T09:32:07.417145+0000","last_fullsized":"2024-09-16T09:32:07.417145+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:00.243482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:00.243482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.b","version":"73'79","reported_seq":282,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.426885+0000","last_change":"2024-09-16T09:31:47.267322+0000","last_active":"2024-09-16T09:32:07.426885+0000","last_peered":"2024-09-16T09:32:07.426885+0000","last_clean":"2024-09-16T09:32:07.426885+0000","last_became_active":"2024-09-16T09:30:10.784688+0000","last_became_peered":"2024-09-16T09:30:10.784688+0000","last_unstale":"2024-09-16T09:32:07.426885+0000","last_undegraded":"2024-09-16T09:32:07.426885+0000","last_fullsized":"2024-09-16T09:32:07.426885+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:30:03.887571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:30:03.887571+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:32:13.164874+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,5],"acting":[1,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2","version":"204'10464","reported_seq":10580,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.602184+0000","last_change":"2024-09-16T09:29:49.445324+0000","last_active":"2024-09-16T09:32:07.602184+0000","last_peered":"2024-09-16T09:32:07.602184+0000","last_clean":"2024-09-16T09:32:07.602184+0000","last_became_active":"2024-09-16T09:29:49.445022+0000","last_became_peered":"2024-09-16T09:29:49.445022+0000","last_unstale":"2024-09-16T09:32:07.602184+0000","last_undegraded":"2024-09-16T09:32:07.602184+0000","last_fullsized":"2024-09-16T09:32:07.602184+0000","mapping_epoch":152,"log_start":"183'6200","ondisk_log_start":"183'6200","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":4264,"log_dups_size":0,"ondisk_log_size":4264,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":312737792,"num_objects":4772,"num_object_clones":0,"num_object_copies":14316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4772,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10464,"num_write_kb":487552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"66'11","reported_seq":347,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.413446+0000","last_change":"2024-09-16T09:30:05.646701+0000","last_active":"2024-09-16T09:32:07.413446+0000","last_peered":"2024-09-16T09:32:07.413446+0000","last_clean":"2024-09-16T09:32:07.413446+0000","last_became_active":"2024-09-16T09:26:33.287399+0000","last_became_peered":"2024-09-16T09:26:33.287399+0000","last_unstale":"2024-09-16T09:32:07.413446+0000","last_undegraded":"2024-09-16T09:32:07.413446+0000","last_fullsized":"2024-09-16T09:32:07.413446+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-16T09:30:05.646500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:30:05.646500+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":305,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.412740+0000","last_change":"2024-09-16T09:25:01.708568+0000","last_active":"2024-09-16T09:32:07.412740+0000","last_peered":"2024-09-16T09:32:07.412740+0000","last_clean":"2024-09-16T09:32:07.412740+0000","last_became_active":"2024-09-16T09:25:01.708270+0000","last_became_peered":"2024-09-16T09:25:01.708270+0000","last_unstale":"2024-09-16T09:32:07.412740+0000","last_undegraded":"2024-09-16T09:32:07.412740+0000","last_fullsized":"2024-09-16T09:32:07.412740+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":304,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.416920+0000","last_change":"2024-09-16T09:26:34.171190+0000","last_active":"2024-09-16T09:32:07.416920+0000","last_peered":"2024-09-16T09:32:07.416920+0000","last_clean":"2024-09-16T09:32:07.416920+0000","last_became_active":"2024-09-16T09:25:05.711964+0000","last_became_peered":"2024-09-16T09:25:05.711964+0000","last_unstale":"2024-09-16T09:32:07.416920+0000","last_undegraded":"2024-09-16T09:32:07.416920+0000","last_fullsized":"2024-09-16T09:32:07.416920+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:34.171139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:26:34.171139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":187,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.467146+0000","last_change":"2024-09-16T09:32:00.526741+0000","last_active":"2024-09-16T09:32:07.467146+0000","last_peered":"2024-09-16T09:32:07.467146+0000","last_clean":"2024-09-16T09:32:07.467146+0000","last_became_active":"2024-09-16T09:31:09.278618+0000","last_became_peered":"2024-09-16T09:31:09.278618+0000","last_unstale":"2024-09-16T09:32:07.467146+0000","last_undegraded":"2024-09-16T09:32:07.467146+0000","last_fullsized":"2024-09-16T09:32:07.467146+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:00.526674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:32:00.526674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:25.768941+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1","version":"204'10416","reported_seq":10532,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.651597+0000","last_change":"2024-09-16T09:29:49.449796+0000","last_active":"2024-09-16T09:32:07.651597+0000","last_peered":"2024-09-16T09:32:07.651597+0000","last_clean":"2024-09-16T09:32:07.651597+0000","last_became_active":"2024-09-16T09:29:49.449475+0000","last_became_peered":"2024-09-16T09:29:49.449475+0000","last_unstale":"2024-09-16T09:32:07.651597+0000","last_undegraded":"2024-09-16T09:32:07.651597+0000","last_fullsized":"2024-09-16T09:32:07.651597+0000","mapping_epoch":152,"log_start":"182'5900","ondisk_log_start":"182'5900","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":4516,"log_dups_size":0,"ondisk_log_size":4516,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":317325312,"num_objects":4842,"num_object_clones":0,"num_object_copies":14526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4842,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10416,"num_write_kb":488256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.8","version":"73'79","reported_seq":283,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.421873+0000","last_change":"2024-09-16T09:32:07.068820+0000","last_active":"2024-09-16T09:32:07.421873+0000","last_peered":"2024-09-16T09:32:07.421873+0000","last_clean":"2024-09-16T09:32:07.421873+0000","last_became_active":"2024-09-16T09:31:09.406638+0000","last_became_peered":"2024-09-16T09:31:09.406638+0000","last_unstale":"2024-09-16T09:32:07.421873+0000","last_undegraded":"2024-09-16T09:32:07.421873+0000","last_fullsized":"2024-09-16T09:32:07.421873+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:17.482769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:17.482769+0000","objects_scrubbed":1,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262688,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":525376,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"66'17","reported_seq":353,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.414940+0000","last_change":"2024-09-16T09:30:04.636115+0000","last_active":"2024-09-16T09:32:07.414940+0000","last_peered":"2024-09-16T09:32:07.414940+0000","last_clean":"2024-09-16T09:32:07.414940+0000","last_became_active":"2024-09-16T09:25:03.713358+0000","last_became_peered":"2024-09-16T09:25:03.713358+0000","last_unstale":"2024-09-16T09:32:07.414940+0000","last_undegraded":"2024-09-16T09:32:07.414940+0000","last_fullsized":"2024-09-16T09:32:07.414940+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'17","last_scrub_stamp":"2024-09-16T09:30:04.636039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:30:04.636039+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":313,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.467167+0000","last_change":"2024-09-16T09:26:04.104188+0000","last_active":"2024-09-16T09:32:07.467167+0000","last_peered":"2024-09-16T09:32:07.467167+0000","last_clean":"2024-09-16T09:32:07.467167+0000","last_became_active":"2024-09-16T09:25:01.703355+0000","last_became_peered":"2024-09-16T09:25:01.703355+0000","last_unstale":"2024-09-16T09:32:07.467167+0000","last_undegraded":"2024-09-16T09:32:07.467167+0000","last_fullsized":"2024-09-16T09:32:07.467167+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:04.104130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:04.104130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:48.432563+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":321,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.425105+0000","last_change":"2024-09-16T09:29:58.839132+0000","last_active":"2024-09-16T09:32:07.425105+0000","last_peered":"2024-09-16T09:32:07.425105+0000","last_clean":"2024-09-16T09:32:07.425105+0000","last_became_active":"2024-09-16T09:25:05.716900+0000","last_became_peered":"2024-09-16T09:25:05.716900+0000","last_unstale":"2024-09-16T09:32:07.425105+0000","last_undegraded":"2024-09-16T09:32:07.425105+0000","last_fullsized":"2024-09-16T09:32:07.425105+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:58.839051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:58.839051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":331,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.422735+0000","last_change":"2024-09-16T09:31:09.276869+0000","last_active":"2024-09-16T09:32:07.422735+0000","last_peered":"2024-09-16T09:32:07.422735+0000","last_clean":"2024-09-16T09:32:07.422735+0000","last_became_active":"2024-09-16T09:31:09.276275+0000","last_became_peered":"2024-09-16T09:31:09.276275+0000","last_unstale":"2024-09-16T09:32:07.422735+0000","last_undegraded":"2024-09-16T09:32:07.422735+0000","last_fullsized":"2024-09-16T09:32:07.422735+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:30:03.946025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:30:03.946025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.0","version":"204'10694","reported_seq":16321,"reported_epoch":204,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-16T09:32:07.930052+0000","last_change":"2024-09-16T09:31:09.476408+0000","last_active":"2024-09-16T09:32:07.930052+0000","last_peered":"2024-09-16T09:32:07.930052+0000","last_clean":"2024-09-16T09:30:20.486290+0000","last_became_active":"2024-09-16T09:31:09.399514+0000","last_became_peered":"2024-09-16T09:31:09.399514+0000","last_unstale":"2024-09-16T09:32:07.930052+0000","last_undegraded":"2024-09-16T09:32:07.930052+0000","last_fullsized":"2024-09-16T09:31:09.254572+0000","mapping_epoch":186,"log_start":"182'6000","ondisk_log_start":"182'6000","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":4694,"log_dups_size":0,"ondisk_log_size":4694,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":317325312,"num_objects":4842,"num_object_clones":0,"num_object_copies":14526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1952,"num_objects_unfound":0,"num_objects_dirty":4842,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10694,"num_write_kb":497152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2750,"num_bytes_recovered":180224000,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,1],"avail_no_missing":["5","1"],"object_location_counts":[{"shards":"1,4,5","objects":1952},{"shards":"1,5","objects":2890}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.9","version":"73'79","reported_seq":384,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.418337+0000","last_change":"2024-09-16T09:29:19.013948+0000","last_active":"2024-09-16T09:32:07.418337+0000","last_peered":"2024-09-16T09:32:07.418337+0000","last_clean":"2024-09-16T09:32:07.418337+0000","last_became_active":"2024-09-16T09:26:35.126066+0000","last_became_peered":"2024-09-16T09:26:35.126066+0000","last_unstale":"2024-09-16T09:32:07.418337+0000","last_undegraded":"2024-09-16T09:32:07.418337+0000","last_fullsized":"2024-09-16T09:32:07.418337+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:19.013865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:19.013865+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"66'10","reported_seq":316,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.414157+0000","last_change":"2024-09-16T09:25:03.708144+0000","last_active":"2024-09-16T09:32:07.414157+0000","last_peered":"2024-09-16T09:32:07.414157+0000","last_clean":"2024-09-16T09:32:07.414157+0000","last_became_active":"2024-09-16T09:25:03.707723+0000","last_became_peered":"2024-09-16T09:25:03.707723+0000","last_unstale":"2024-09-16T09:32:07.414157+0000","last_undegraded":"2024-09-16T09:32:07.414157+0000","last_fullsized":"2024-09-16T09:32:07.414157+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":178,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.427616+0000","last_change":"2024-09-16T09:31:32.253397+0000","last_active":"2024-09-16T09:32:07.427616+0000","last_peered":"2024-09-16T09:32:07.427616+0000","last_clean":"2024-09-16T09:32:07.427616+0000","last_became_active":"2024-09-16T09:31:09.288825+0000","last_became_peered":"2024-09-16T09:31:09.288825+0000","last_unstale":"2024-09-16T09:32:07.427616+0000","last_undegraded":"2024-09-16T09:32:07.427616+0000","last_fullsized":"2024-09-16T09:32:07.427616+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:31:32.253306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:31:32.253306+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:45.682494+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":418,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.417058+0000","last_change":"2024-09-16T09:29:29.866520+0000","last_active":"2024-09-16T09:32:07.417058+0000","last_peered":"2024-09-16T09:32:07.417058+0000","last_clean":"2024-09-16T09:32:07.417058+0000","last_became_active":"2024-09-16T09:25:05.711590+0000","last_became_peered":"2024-09-16T09:25:05.711590+0000","last_unstale":"2024-09-16T09:32:07.417058+0000","last_undegraded":"2024-09-16T09:32:07.417058+0000","last_fullsized":"2024-09-16T09:32:07.417058+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:29:29.866460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:29.866460+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":317,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.413186+0000","last_change":"2024-09-16T09:29:02.667332+0000","last_active":"2024-09-16T09:32:07.413186+0000","last_peered":"2024-09-16T09:32:07.413186+0000","last_clean":"2024-09-16T09:32:07.413186+0000","last_became_active":"2024-09-16T09:25:07.718958+0000","last_became_peered":"2024-09-16T09:25:07.718958+0000","last_unstale":"2024-09-16T09:32:07.413186+0000","last_undegraded":"2024-09-16T09:32:07.413186+0000","last_fullsized":"2024-09-16T09:32:07.413186+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:02.667259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:02.667259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.e","version":"73'79","reported_seq":106,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.413012+0000","last_change":"2024-09-16T09:31:47.249302+0000","last_active":"2024-09-16T09:32:07.413012+0000","last_peered":"2024-09-16T09:32:07.413012+0000","last_clean":"2024-09-16T09:32:07.413012+0000","last_became_active":"2024-09-16T09:30:10.705072+0000","last_became_peered":"2024-09-16T09:30:10.705072+0000","last_unstale":"2024-09-16T09:32:07.413012+0000","last_undegraded":"2024-09-16T09:32:07.413012+0000","last_fullsized":"2024-09-16T09:32:07.413012+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:26:35.170229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:26:35.170229+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:32:12.841915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26835,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":10,"num_write":5,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":26835,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,3],"acting":[4,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.7","version":"204'10552","reported_seq":10668,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.661611+0000","last_change":"2024-09-16T09:29:49.839740+0000","last_active":"2024-09-16T09:32:07.661611+0000","last_peered":"2024-09-16T09:32:07.661611+0000","last_clean":"2024-09-16T09:32:07.661611+0000","last_became_active":"2024-09-16T09:29:49.839437+0000","last_became_peered":"2024-09-16T09:29:49.839437+0000","last_unstale":"2024-09-16T09:32:07.661611+0000","last_undegraded":"2024-09-16T09:32:07.661611+0000","last_fullsized":"2024-09-16T09:32:07.661611+0000","mapping_epoch":152,"log_start":"182'6000","ondisk_log_start":"182'6000","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":4552,"log_dups_size":0,"ondisk_log_size":4552,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":320602112,"num_objects":4892,"num_object_clones":0,"num_object_copies":14676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4892,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10552,"num_write_kb":494208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"66'9","reported_seq":341,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.427367+0000","last_change":"2024-09-16T09:31:31.211495+0000","last_active":"2024-09-16T09:32:07.427367+0000","last_peered":"2024-09-16T09:32:07.427367+0000","last_clean":"2024-09-16T09:32:07.427367+0000","last_became_active":"2024-09-16T09:25:03.712230+0000","last_became_peered":"2024-09-16T09:25:03.712230+0000","last_unstale":"2024-09-16T09:32:07.427367+0000","last_undegraded":"2024-09-16T09:32:07.427367+0000","last_fullsized":"2024-09-16T09:32:07.427367+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:31:31.211420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:31:31.211420+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:36.499658+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":326,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.418490+0000","last_change":"2024-09-16T09:31:09.277413+0000","last_active":"2024-09-16T09:32:07.418490+0000","last_peered":"2024-09-16T09:32:07.418490+0000","last_clean":"2024-09-16T09:32:07.418490+0000","last_became_active":"2024-09-16T09:31:09.276516+0000","last_became_peered":"2024-09-16T09:31:09.276516+0000","last_unstale":"2024-09-16T09:32:07.418490+0000","last_undegraded":"2024-09-16T09:32:07.418490+0000","last_fullsized":"2024-09-16T09:32:07.418490+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:26.061106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:26.061106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":327,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.419490+0000","last_change":"2024-09-16T09:31:09.275219+0000","last_active":"2024-09-16T09:32:07.419490+0000","last_peered":"2024-09-16T09:32:07.419490+0000","last_clean":"2024-09-16T09:32:07.419490+0000","last_became_active":"2024-09-16T09:31:09.274702+0000","last_became_peered":"2024-09-16T09:31:09.274702+0000","last_unstale":"2024-09-16T09:32:07.419490+0000","last_undegraded":"2024-09-16T09:32:07.419490+0000","last_fullsized":"2024-09-16T09:32:07.419490+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:57.220068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:28:57.220068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":33,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":317,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.428257+0000","last_change":"2024-09-16T09:31:27.185957+0000","last_active":"2024-09-16T09:32:07.428257+0000","last_peered":"2024-09-16T09:32:07.428257+0000","last_clean":"2024-09-16T09:32:07.428257+0000","last_became_active":"2024-09-16T09:25:07.718778+0000","last_became_peered":"2024-09-16T09:25:07.718778+0000","last_unstale":"2024-09-16T09:32:07.428257+0000","last_undegraded":"2024-09-16T09:32:07.428257+0000","last_fullsized":"2024-09-16T09:32:07.428257+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:31:27.185801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:31:27.185801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:39.017009+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.f","version":"73'79","reported_seq":358,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.471887+0000","last_change":"2024-09-16T09:31:47.259689+0000","last_active":"2024-09-16T09:32:07.471887+0000","last_peered":"2024-09-16T09:32:07.471887+0000","last_clean":"2024-09-16T09:32:07.471887+0000","last_became_active":"2024-09-16T09:30:10.693477+0000","last_became_peered":"2024-09-16T09:30:10.693477+0000","last_unstale":"2024-09-16T09:32:07.471887+0000","last_undegraded":"2024-09-16T09:32:07.471887+0000","last_fullsized":"2024-09-16T09:32:07.471887+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:17.534429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:17.534429+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:32:13.472806+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.6","version":"204'10860","reported_seq":10976,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.593460+0000","last_change":"2024-09-16T09:29:50.008888+0000","last_active":"2024-09-16T09:32:07.593460+0000","last_peered":"2024-09-16T09:32:07.593460+0000","last_clean":"2024-09-16T09:32:07.593460+0000","last_became_active":"2024-09-16T09:29:50.008656+0000","last_became_peered":"2024-09-16T09:29:50.008656+0000","last_unstale":"2024-09-16T09:32:07.593460+0000","last_undegraded":"2024-09-16T09:32:07.593460+0000","last_fullsized":"2024-09-16T09:32:07.593460+0000","mapping_epoch":152,"log_start":"180'5700","ondisk_log_start":"180'5700","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":5160,"log_dups_size":0,"ondisk_log_size":5160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":325189632,"num_objects":4962,"num_object_clones":0,"num_object_copies":14886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4962,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10860,"num_write_kb":506304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"66'19","reported_seq":348,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.467029+0000","last_change":"2024-09-16T09:29:02.554357+0000","last_active":"2024-09-16T09:32:07.467029+0000","last_peered":"2024-09-16T09:32:07.467029+0000","last_clean":"2024-09-16T09:32:07.467029+0000","last_became_active":"2024-09-16T09:25:03.712783+0000","last_became_peered":"2024-09-16T09:25:03.712783+0000","last_unstale":"2024-09-16T09:32:07.467029+0000","last_undegraded":"2024-09-16T09:32:07.467029+0000","last_fullsized":"2024-09-16T09:32:07.467029+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-16T09:29:02.554274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:29:02.554274+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:17.371584+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"59'1","reported_seq":326,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.420355+0000","last_change":"2024-09-16T09:29:51.025505+0000","last_active":"2024-09-16T09:32:07.420355+0000","last_peered":"2024-09-16T09:32:07.420355+0000","last_clean":"2024-09-16T09:32:07.420355+0000","last_became_active":"2024-09-16T09:25:01.708069+0000","last_became_peered":"2024-09-16T09:25:01.708069+0000","last_unstale":"2024-09-16T09:32:07.420355+0000","last_undegraded":"2024-09-16T09:32:07.420355+0000","last_fullsized":"2024-09-16T09:32:07.420355+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:29:51.025444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:51.025444+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":305,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.416520+0000","last_change":"2024-09-16T09:27:11.067059+0000","last_active":"2024-09-16T09:32:07.416520+0000","last_peered":"2024-09-16T09:32:07.416520+0000","last_clean":"2024-09-16T09:32:07.416520+0000","last_became_active":"2024-09-16T09:25:05.709264+0000","last_became_peered":"2024-09-16T09:25:05.709264+0000","last_unstale":"2024-09-16T09:32:07.416520+0000","last_undegraded":"2024-09-16T09:32:07.416520+0000","last_fullsized":"2024-09-16T09:32:07.416520+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:11.066986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:11.066986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":300,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.419279+0000","last_change":"2024-09-16T09:27:25.865107+0000","last_active":"2024-09-16T09:32:07.419279+0000","last_peered":"2024-09-16T09:32:07.419279+0000","last_clean":"2024-09-16T09:32:07.419279+0000","last_became_active":"2024-09-16T09:25:07.718750+0000","last_became_peered":"2024-09-16T09:25:07.718750+0000","last_unstale":"2024-09-16T09:32:07.419279+0000","last_undegraded":"2024-09-16T09:32:07.419279+0000","last_fullsized":"2024-09-16T09:32:07.419279+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:25.865030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:25.865030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.c","version":"73'79","reported_seq":50,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.420395+0000","last_change":"2024-09-16T09:31:09.290919+0000","last_active":"2024-09-16T09:32:07.420395+0000","last_peered":"2024-09-16T09:32:07.420395+0000","last_clean":"2024-09-16T09:32:07.420395+0000","last_became_active":"2024-09-16T09:31:09.273770+0000","last_became_peered":"2024-09-16T09:31:09.273770+0000","last_unstale":"2024-09-16T09:32:07.420395+0000","last_undegraded":"2024-09-16T09:32:07.420395+0000","last_fullsized":"2024-09-16T09:32:07.420395+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:30:22.614462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:30:22.614462+0000","objects_scrubbed":1,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":327680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.5","version":"204'10748","reported_seq":10937,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.569927+0000","last_change":"2024-09-16T09:31:06.121217+0000","last_active":"2024-09-16T09:32:07.569927+0000","last_peered":"2024-09-16T09:32:07.569927+0000","last_clean":"2024-09-16T09:32:07.569927+0000","last_became_active":"2024-09-16T09:29:50.009013+0000","last_became_peered":"2024-09-16T09:29:50.009013+0000","last_unstale":"2024-09-16T09:32:07.569927+0000","last_undegraded":"2024-09-16T09:32:07.569927+0000","last_fullsized":"2024-09-16T09:32:07.569927+0000","mapping_epoch":152,"log_start":"180'5513","ondisk_log_start":"180'5513","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"184'6587","last_scrub_stamp":"2024-09-16T09:31:06.120990+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:31:06.120990+0000","objects_scrubbed":6523,"log_size":5235,"log_dups_size":0,"ondisk_log_size":5235,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:20.260788+0000","scrub_duration":1690,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327417856,"num_objects":4996,"num_object_clones":0,"num_object_copies":14988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4996,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10748,"num_write_kb":503808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"66'12","reported_seq":378,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.412618+0000","last_change":"2024-09-16T09:31:09.296915+0000","last_active":"2024-09-16T09:32:07.412618+0000","last_peered":"2024-09-16T09:32:07.412618+0000","last_clean":"2024-09-16T09:32:07.412618+0000","last_became_active":"2024-09-16T09:31:09.269047+0000","last_became_peered":"2024-09-16T09:31:09.269047+0000","last_unstale":"2024-09-16T09:32:07.412618+0000","last_undegraded":"2024-09-16T09:32:07.412618+0000","last_fullsized":"2024-09-16T09:32:07.412618+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-16T09:30:21.728668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:30:21.728668+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":320,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.419885+0000","last_change":"2024-09-16T09:29:53.075288+0000","last_active":"2024-09-16T09:32:07.419885+0000","last_peered":"2024-09-16T09:32:07.419885+0000","last_clean":"2024-09-16T09:32:07.419885+0000","last_became_active":"2024-09-16T09:25:01.707528+0000","last_became_peered":"2024-09-16T09:25:01.707528+0000","last_unstale":"2024-09-16T09:32:07.419885+0000","last_undegraded":"2024-09-16T09:32:07.419885+0000","last_fullsized":"2024-09-16T09:32:07.419885+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:53.075232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:53.075232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":313,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.416664+0000","last_change":"2024-09-16T09:27:49.965628+0000","last_active":"2024-09-16T09:32:07.416664+0000","last_peered":"2024-09-16T09:32:07.416664+0000","last_clean":"2024-09-16T09:32:07.416664+0000","last_became_active":"2024-09-16T09:25:05.715253+0000","last_became_peered":"2024-09-16T09:25:05.715253+0000","last_unstale":"2024-09-16T09:32:07.416664+0000","last_undegraded":"2024-09-16T09:32:07.416664+0000","last_fullsized":"2024-09-16T09:32:07.416664+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:49.965424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:49.965424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":301,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.419885+0000","last_change":"2024-09-16T09:27:56.059755+0000","last_active":"2024-09-16T09:32:07.419885+0000","last_peered":"2024-09-16T09:32:07.419885+0000","last_clean":"2024-09-16T09:32:07.419885+0000","last_became_active":"2024-09-16T09:25:07.718899+0000","last_became_peered":"2024-09-16T09:25:07.718899+0000","last_unstale":"2024-09-16T09:32:07.419885+0000","last_undegraded":"2024-09-16T09:32:07.419885+0000","last_fullsized":"2024-09-16T09:32:07.419885+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:56.059687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:56.059687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.d","version":"73'79","reported_seq":106,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.418883+0000","last_change":"2024-09-16T09:31:47.263954+0000","last_active":"2024-09-16T09:32:07.418883+0000","last_peered":"2024-09-16T09:32:07.418883+0000","last_clean":"2024-09-16T09:32:07.418883+0000","last_became_active":"2024-09-16T09:30:10.709958+0000","last_became_peered":"2024-09-16T09:30:10.709958+0000","last_unstale":"2024-09-16T09:32:07.418883+0000","last_undegraded":"2024-09-16T09:32:07.418883+0000","last_fullsized":"2024-09-16T09:32:07.418883+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:58.568841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:58.568841+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:32:13.456459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26835,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":10,"num_write":5,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":26835,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.4","version":"204'10727","reported_seq":10843,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.623180+0000","last_change":"2024-09-16T09:29:49.444857+0000","last_active":"2024-09-16T09:32:07.623180+0000","last_peered":"2024-09-16T09:32:07.623180+0000","last_clean":"2024-09-16T09:32:07.623180+0000","last_became_active":"2024-09-16T09:29:49.444515+0000","last_became_peered":"2024-09-16T09:29:49.444515+0000","last_unstale":"2024-09-16T09:32:07.623180+0000","last_undegraded":"2024-09-16T09:32:07.623180+0000","last_fullsized":"2024-09-16T09:32:07.623180+0000","mapping_epoch":152,"log_start":"182'6000","ondisk_log_start":"182'6000","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":4727,"log_dups_size":0,"ondisk_log_size":4727,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":319881216,"num_objects":4881,"num_object_clones":0,"num_object_copies":14643,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4881,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10727,"num_write_kb":499456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"66'15","reported_seq":333,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.422416+0000","last_change":"2024-09-16T09:26:33.138828+0000","last_active":"2024-09-16T09:32:07.422416+0000","last_peered":"2024-09-16T09:32:07.422416+0000","last_clean":"2024-09-16T09:32:07.422416+0000","last_became_active":"2024-09-16T09:25:03.715733+0000","last_became_peered":"2024-09-16T09:25:03.715733+0000","last_unstale":"2024-09-16T09:32:07.422416+0000","last_undegraded":"2024-09-16T09:32:07.422416+0000","last_fullsized":"2024-09-16T09:32:07.422416+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-16T09:26:33.138603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:26:33.138603+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":324,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.419834+0000","last_change":"2024-09-16T09:26:26.705847+0000","last_active":"2024-09-16T09:32:07.419834+0000","last_peered":"2024-09-16T09:32:07.419834+0000","last_clean":"2024-09-16T09:32:07.419834+0000","last_became_active":"2024-09-16T09:25:01.707372+0000","last_became_peered":"2024-09-16T09:25:01.707372+0000","last_unstale":"2024-09-16T09:32:07.419834+0000","last_undegraded":"2024-09-16T09:32:07.419834+0000","last_fullsized":"2024-09-16T09:32:07.419834+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-16T09:26:26.705773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:26.705773+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"66'8","reported_seq":414,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.418243+0000","last_change":"2024-09-16T09:30:05.138362+0000","last_active":"2024-09-16T09:32:07.418243+0000","last_peered":"2024-09-16T09:32:07.418243+0000","last_clean":"2024-09-16T09:32:07.418243+0000","last_became_active":"2024-09-16T09:25:05.716189+0000","last_became_peered":"2024-09-16T09:25:05.716189+0000","last_unstale":"2024-09-16T09:32:07.418243+0000","last_undegraded":"2024-09-16T09:32:07.418243+0000","last_fullsized":"2024-09-16T09:32:07.418243+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:30:05.138288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:30:05.138288+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":300,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.418205+0000","last_change":"2024-09-16T09:27:20.219376+0000","last_active":"2024-09-16T09:32:07.418205+0000","last_peered":"2024-09-16T09:32:07.418205+0000","last_clean":"2024-09-16T09:32:07.418205+0000","last_became_active":"2024-09-16T09:25:07.717213+0000","last_became_peered":"2024-09-16T09:25:07.717213+0000","last_unstale":"2024-09-16T09:32:07.418205+0000","last_undegraded":"2024-09-16T09:32:07.418205+0000","last_fullsized":"2024-09-16T09:32:07.418205+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:20.219333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:20.219333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.12","version":"73'79","reported_seq":106,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.418431+0000","last_change":"2024-09-16T09:31:47.251225+0000","last_active":"2024-09-16T09:32:07.418431+0000","last_peered":"2024-09-16T09:32:07.418431+0000","last_clean":"2024-09-16T09:32:07.418431+0000","last_became_active":"2024-09-16T09:30:10.731413+0000","last_became_peered":"2024-09-16T09:30:10.731413+0000","last_unstale":"2024-09-16T09:32:07.418431+0000","last_undegraded":"2024-09-16T09:32:07.418431+0000","last_fullsized":"2024-09-16T09:32:07.418431+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:26:35.745837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:26:35.745837+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:32:14.398546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26835,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":10,"num_write":6,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":26835,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":313,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.425507+0000","last_change":"2024-09-16T09:26:33.139405+0000","last_active":"2024-09-16T09:32:07.425507+0000","last_peered":"2024-09-16T09:32:07.425507+0000","last_clean":"2024-09-16T09:32:07.425507+0000","last_became_active":"2024-09-16T09:25:01.703202+0000","last_became_peered":"2024-09-16T09:25:01.703202+0000","last_unstale":"2024-09-16T09:32:07.425507+0000","last_undegraded":"2024-09-16T09:32:07.425507+0000","last_fullsized":"2024-09-16T09:32:07.425507+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:33.139360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:33.139360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:10.508509+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"66'6","reported_seq":54,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.424134+0000","last_change":"2024-09-16T09:31:09.297514+0000","last_active":"2024-09-16T09:32:07.424134+0000","last_peered":"2024-09-16T09:32:07.424134+0000","last_clean":"2024-09-16T09:32:07.424134+0000","last_became_active":"2024-09-16T09:31:09.270685+0000","last_became_peered":"2024-09-16T09:31:09.270685+0000","last_unstale":"2024-09-16T09:32:07.424134+0000","last_undegraded":"2024-09-16T09:32:07.424134+0000","last_fullsized":"2024-09-16T09:32:07.424134+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'6","last_scrub_stamp":"2024-09-16T09:26:23.458794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:26:23.458794+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":8,"num_read_kb":5,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":311,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.416516+0000","last_change":"2024-09-16T09:29:31.496374+0000","last_active":"2024-09-16T09:32:07.416516+0000","last_peered":"2024-09-16T09:32:07.416516+0000","last_clean":"2024-09-16T09:32:07.416516+0000","last_became_active":"2024-09-16T09:25:05.715616+0000","last_became_peered":"2024-09-16T09:25:05.715616+0000","last_unstale":"2024-09-16T09:32:07.416516+0000","last_undegraded":"2024-09-16T09:32:07.416516+0000","last_fullsized":"2024-09-16T09:32:07.416516+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:31.496304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:31.496304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":299,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.420099+0000","last_change":"2024-09-16T09:26:28.733381+0000","last_active":"2024-09-16T09:32:07.420099+0000","last_peered":"2024-09-16T09:32:07.420099+0000","last_clean":"2024-09-16T09:32:07.420099+0000","last_became_active":"2024-09-16T09:25:07.721059+0000","last_became_peered":"2024-09-16T09:25:07.721059+0000","last_unstale":"2024-09-16T09:32:07.420099+0000","last_undegraded":"2024-09-16T09:32:07.420099+0000","last_fullsized":"2024-09-16T09:32:07.420099+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:28.733324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:28.733324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.13","version":"73'79","reported_seq":295,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.427246+0000","last_change":"2024-09-16T09:31:09.288677+0000","last_active":"2024-09-16T09:32:07.427246+0000","last_peered":"2024-09-16T09:32:07.427246+0000","last_clean":"2024-09-16T09:32:07.427246+0000","last_became_active":"2024-09-16T09:31:09.288233+0000","last_became_peered":"2024-09-16T09:31:09.288233+0000","last_unstale":"2024-09-16T09:32:07.427246+0000","last_undegraded":"2024-09-16T09:32:07.427246+0000","last_fullsized":"2024-09-16T09:32:07.427246+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:30:21.714369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:30:21.714369+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.16","version":"66'9","reported_seq":382,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.428515+0000","last_change":"2024-09-16T09:32:01.319481+0000","last_active":"2024-09-16T09:32:07.428515+0000","last_peered":"2024-09-16T09:32:07.428515+0000","last_clean":"2024-09-16T09:32:07.428515+0000","last_became_active":"2024-09-16T09:31:09.273498+0000","last_became_peered":"2024-09-16T09:31:09.273498+0000","last_unstale":"2024-09-16T09:32:07.428515+0000","last_undegraded":"2024-09-16T09:32:07.428515+0000","last_fullsized":"2024-09-16T09:32:07.428515+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:32:01.319390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:32:01.319390+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:29.275572+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":320,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.418540+0000","last_change":"2024-09-16T09:28:50.698331+0000","last_active":"2024-09-16T09:32:07.418540+0000","last_peered":"2024-09-16T09:32:07.418540+0000","last_clean":"2024-09-16T09:32:07.418540+0000","last_became_active":"2024-09-16T09:25:01.706491+0000","last_became_peered":"2024-09-16T09:25:01.706491+0000","last_unstale":"2024-09-16T09:32:07.418540+0000","last_undegraded":"2024-09-16T09:32:07.418540+0000","last_fullsized":"2024-09-16T09:32:07.418540+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:50.698288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:28:50.698288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":170,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.425961+0000","last_change":"2024-09-16T09:31:09.271558+0000","last_active":"2024-09-16T09:32:07.425961+0000","last_peered":"2024-09-16T09:32:07.425961+0000","last_clean":"2024-09-16T09:32:07.425961+0000","last_became_active":"2024-09-16T09:31:09.271223+0000","last_became_peered":"2024-09-16T09:31:09.271223+0000","last_unstale":"2024-09-16T09:32:07.425961+0000","last_undegraded":"2024-09-16T09:32:07.425961+0000","last_fullsized":"2024-09-16T09:32:07.425961+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:53.810580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:28:53.810580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":301,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.422337+0000","last_change":"2024-09-16T09:26:35.866582+0000","last_active":"2024-09-16T09:32:07.422337+0000","last_peered":"2024-09-16T09:32:07.422337+0000","last_clean":"2024-09-16T09:32:07.422337+0000","last_became_active":"2024-09-16T09:25:07.722050+0000","last_became_peered":"2024-09-16T09:25:07.722050+0000","last_unstale":"2024-09-16T09:32:07.422337+0000","last_undegraded":"2024-09-16T09:32:07.422337+0000","last_fullsized":"2024-09-16T09:32:07.422337+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:35.866512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:35.866512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:08.904503+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.10","version":"73'79","reported_seq":106,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.416534+0000","last_change":"2024-09-16T09:31:47.249070+0000","last_active":"2024-09-16T09:32:07.416534+0000","last_peered":"2024-09-16T09:32:07.416534+0000","last_clean":"2024-09-16T09:32:07.416534+0000","last_became_active":"2024-09-16T09:30:10.692770+0000","last_became_peered":"2024-09-16T09:30:10.692770+0000","last_unstale":"2024-09-16T09:32:07.416534+0000","last_undegraded":"2024-09-16T09:32:07.416534+0000","last_fullsized":"2024-09-16T09:32:07.416534+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:02.723794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:02.723794+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:32:13.396012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26835,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":10,"num_write":5,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":26835,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"66'9","reported_seq":331,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.417779+0000","last_change":"2024-09-16T09:29:32.485165+0000","last_active":"2024-09-16T09:32:07.417779+0000","last_peered":"2024-09-16T09:32:07.417779+0000","last_clean":"2024-09-16T09:32:07.417779+0000","last_became_active":"2024-09-16T09:25:03.717310+0000","last_became_peered":"2024-09-16T09:25:03.717310+0000","last_unstale":"2024-09-16T09:32:07.417779+0000","last_undegraded":"2024-09-16T09:32:07.417779+0000","last_fullsized":"2024-09-16T09:32:07.417779+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:29:32.485084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:29:32.485084+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":343,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.427748+0000","last_change":"2024-09-16T09:31:59.376923+0000","last_active":"2024-09-16T09:32:07.427748+0000","last_peered":"2024-09-16T09:32:07.427748+0000","last_clean":"2024-09-16T09:32:07.427748+0000","last_became_active":"2024-09-16T09:31:09.272848+0000","last_became_peered":"2024-09-16T09:31:09.272848+0000","last_unstale":"2024-09-16T09:32:07.427748+0000","last_undegraded":"2024-09-16T09:32:07.427748+0000","last_fullsized":"2024-09-16T09:32:07.427748+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:31:59.376749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:31:59.376749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:13.111415+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"187'11","reported_seq":243,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.419330+0000","last_change":"2024-09-16T09:31:09.311243+0000","last_active":"2024-09-16T09:32:07.419330+0000","last_peered":"2024-09-16T09:32:07.419330+0000","last_clean":"2024-09-16T09:32:07.419330+0000","last_became_active":"2024-09-16T09:31:09.277688+0000","last_became_peered":"2024-09-16T09:31:09.277688+0000","last_unstale":"2024-09-16T09:32:07.419330+0000","last_undegraded":"2024-09-16T09:32:07.419330+0000","last_fullsized":"2024-09-16T09:32:07.419330+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:28:53.714413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:28:53.714413+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":316,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.413302+0000","last_change":"2024-09-16T09:29:49.474440+0000","last_active":"2024-09-16T09:32:07.413302+0000","last_peered":"2024-09-16T09:32:07.413302+0000","last_clean":"2024-09-16T09:32:07.413302+0000","last_became_active":"2024-09-16T09:25:07.718810+0000","last_became_peered":"2024-09-16T09:25:07.718810+0000","last_unstale":"2024-09-16T09:32:07.413302+0000","last_undegraded":"2024-09-16T09:32:07.413302+0000","last_fullsized":"2024-09-16T09:32:07.413302+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:49.474391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:49.474391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.11","version":"73'79","reported_seq":195,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.428544+0000","last_change":"2024-09-16T09:31:09.272938+0000","last_active":"2024-09-16T09:32:07.428544+0000","last_peered":"2024-09-16T09:32:07.428544+0000","last_clean":"2024-09-16T09:32:07.428544+0000","last_became_active":"2024-09-16T09:31:09.272208+0000","last_became_peered":"2024-09-16T09:31:09.272208+0000","last_unstale":"2024-09-16T09:32:07.428544+0000","last_undegraded":"2024-09-16T09:32:07.428544+0000","last_fullsized":"2024-09-16T09:32:07.428544+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:30:21.651575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:30:21.651575+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.14","version":"66'10","reported_seq":191,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.425939+0000","last_change":"2024-09-16T09:28:56.825675+0000","last_active":"2024-09-16T09:32:07.425939+0000","last_peered":"2024-09-16T09:32:07.425939+0000","last_clean":"2024-09-16T09:32:07.425939+0000","last_became_active":"2024-09-16T09:28:52.979059+0000","last_became_peered":"2024-09-16T09:28:52.979059+0000","last_unstale":"2024-09-16T09:32:07.425939+0000","last_undegraded":"2024-09-16T09:32:07.425939+0000","last_fullsized":"2024-09-16T09:32:07.425939+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-16T09:28:56.825476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:56.825476+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":14,"num_read_kb":9,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":320,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.418059+0000","last_change":"2024-09-16T09:29:56.021413+0000","last_active":"2024-09-16T09:32:07.418059+0000","last_peered":"2024-09-16T09:32:07.418059+0000","last_clean":"2024-09-16T09:32:07.418059+0000","last_became_active":"2024-09-16T09:25:01.707513+0000","last_became_peered":"2024-09-16T09:25:01.707513+0000","last_unstale":"2024-09-16T09:32:07.418059+0000","last_undegraded":"2024-09-16T09:32:07.418059+0000","last_fullsized":"2024-09-16T09:32:07.418059+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:56.021340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:56.021340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"66'8","reported_seq":413,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.417603+0000","last_change":"2024-09-16T09:29:48.485781+0000","last_active":"2024-09-16T09:32:07.417603+0000","last_peered":"2024-09-16T09:32:07.417603+0000","last_clean":"2024-09-16T09:32:07.417603+0000","last_became_active":"2024-09-16T09:25:05.713833+0000","last_became_peered":"2024-09-16T09:25:05.713833+0000","last_unstale":"2024-09-16T09:32:07.417603+0000","last_undegraded":"2024-09-16T09:32:07.417603+0000","last_fullsized":"2024-09-16T09:32:07.417603+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:29:48.485727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.485727+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":330,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.427856+0000","last_change":"2024-09-16T09:32:00.343429+0000","last_active":"2024-09-16T09:32:07.427856+0000","last_peered":"2024-09-16T09:32:07.427856+0000","last_clean":"2024-09-16T09:32:07.427856+0000","last_became_active":"2024-09-16T09:31:09.275168+0000","last_became_peered":"2024-09-16T09:31:09.275168+0000","last_unstale":"2024-09-16T09:32:07.427856+0000","last_undegraded":"2024-09-16T09:32:07.427856+0000","last_fullsized":"2024-09-16T09:32:07.427856+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:00.343356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:32:00.343356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:23.568990+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"66'11","reported_seq":339,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.413641+0000","last_change":"2024-09-16T09:26:33.302713+0000","last_active":"2024-09-16T09:32:07.413641+0000","last_peered":"2024-09-16T09:32:07.413641+0000","last_clean":"2024-09-16T09:32:07.413641+0000","last_became_active":"2024-09-16T09:26:33.291265+0000","last_became_peered":"2024-09-16T09:26:33.291265+0000","last_unstale":"2024-09-16T09:32:07.413641+0000","last_undegraded":"2024-09-16T09:32:07.413641+0000","last_fullsized":"2024-09-16T09:32:07.413641+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":329,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.414372+0000","last_change":"2024-09-16T09:29:51.517112+0000","last_active":"2024-09-16T09:32:07.414372+0000","last_peered":"2024-09-16T09:32:07.414372+0000","last_clean":"2024-09-16T09:32:07.414372+0000","last_became_active":"2024-09-16T09:25:01.705158+0000","last_became_peered":"2024-09-16T09:25:01.705158+0000","last_unstale":"2024-09-16T09:32:07.414372+0000","last_undegraded":"2024-09-16T09:32:07.414372+0000","last_fullsized":"2024-09-16T09:32:07.414372+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:51.517054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:51.517054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":335,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.424758+0000","last_change":"2024-09-16T09:31:09.266820+0000","last_active":"2024-09-16T09:32:07.424758+0000","last_peered":"2024-09-16T09:32:07.424758+0000","last_clean":"2024-09-16T09:32:07.424758+0000","last_became_active":"2024-09-16T09:31:09.266326+0000","last_became_peered":"2024-09-16T09:31:09.266326+0000","last_unstale":"2024-09-16T09:32:07.424758+0000","last_undegraded":"2024-09-16T09:32:07.424758+0000","last_fullsized":"2024-09-16T09:32:07.424758+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:15.910824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:15.910824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":163,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.428173+0000","last_change":"2024-09-16T09:31:09.277239+0000","last_active":"2024-09-16T09:32:07.428173+0000","last_peered":"2024-09-16T09:32:07.428173+0000","last_clean":"2024-09-16T09:32:07.428173+0000","last_became_active":"2024-09-16T09:31:09.276042+0000","last_became_peered":"2024-09-16T09:31:09.276042+0000","last_unstale":"2024-09-16T09:32:07.428173+0000","last_undegraded":"2024-09-16T09:32:07.428173+0000","last_fullsized":"2024-09-16T09:32:07.428173+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:48.147759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:48.147759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:14.230973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"66'9","reported_seq":325,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.425093+0000","last_change":"2024-09-16T09:26:31.216495+0000","last_active":"2024-09-16T09:32:07.425093+0000","last_peered":"2024-09-16T09:32:07.425093+0000","last_clean":"2024-09-16T09:32:07.425093+0000","last_became_active":"2024-09-16T09:25:03.705807+0000","last_became_peered":"2024-09-16T09:25:03.705807+0000","last_unstale":"2024-09-16T09:32:07.425093+0000","last_undegraded":"2024-09-16T09:32:07.425093+0000","last_fullsized":"2024-09-16T09:32:07.425093+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:26:31.216298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:26:31.216298+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":311,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.420255+0000","last_change":"2024-09-16T09:26:23.796308+0000","last_active":"2024-09-16T09:32:07.420255+0000","last_peered":"2024-09-16T09:32:07.420255+0000","last_clean":"2024-09-16T09:32:07.420255+0000","last_became_active":"2024-09-16T09:25:01.706762+0000","last_became_peered":"2024-09-16T09:25:01.706762+0000","last_unstale":"2024-09-16T09:32:07.420255+0000","last_undegraded":"2024-09-16T09:32:07.420255+0000","last_fullsized":"2024-09-16T09:32:07.420255+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:23.796231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:23.796231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":148,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.427157+0000","last_change":"2024-09-16T09:28:52.983144+0000","last_active":"2024-09-16T09:32:07.427157+0000","last_peered":"2024-09-16T09:32:07.427157+0000","last_clean":"2024-09-16T09:32:07.427157+0000","last_became_active":"2024-09-16T09:28:52.982865+0000","last_became_peered":"2024-09-16T09:28:52.982865+0000","last_unstale":"2024-09-16T09:32:07.427157+0000","last_undegraded":"2024-09-16T09:32:07.427157+0000","last_fullsized":"2024-09-16T09:32:07.427157+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:44.241942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:44.241942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:09.215097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":309,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.426769+0000","last_change":"2024-09-16T09:28:52.552567+0000","last_active":"2024-09-16T09:32:07.426769+0000","last_peered":"2024-09-16T09:32:07.426769+0000","last_clean":"2024-09-16T09:32:07.426769+0000","last_became_active":"2024-09-16T09:25:07.715301+0000","last_became_peered":"2024-09-16T09:25:07.715301+0000","last_unstale":"2024-09-16T09:32:07.426769+0000","last_undegraded":"2024-09-16T09:32:07.426769+0000","last_fullsized":"2024-09-16T09:32:07.426769+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:52.552495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:28:52.552495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:45.269179+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.14","version":"73'79","reported_seq":106,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.422057+0000","last_change":"2024-09-16T09:31:47.253568+0000","last_active":"2024-09-16T09:32:07.422057+0000","last_peered":"2024-09-16T09:32:07.422057+0000","last_clean":"2024-09-16T09:32:07.422057+0000","last_became_active":"2024-09-16T09:30:10.700866+0000","last_became_peered":"2024-09-16T09:30:10.700866+0000","last_unstale":"2024-09-16T09:32:07.422057+0000","last_undegraded":"2024-09-16T09:32:07.422057+0000","last_fullsized":"2024-09-16T09:32:07.422057+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:53.916505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:53.916505+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:32:13.898748+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109227,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"66'11","reported_seq":195,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.418421+0000","last_change":"2024-09-16T09:28:55.643505+0000","last_active":"2024-09-16T09:32:07.418421+0000","last_peered":"2024-09-16T09:32:07.418421+0000","last_clean":"2024-09-16T09:32:07.418421+0000","last_became_active":"2024-09-16T09:28:52.971132+0000","last_became_peered":"2024-09-16T09:28:52.971132+0000","last_unstale":"2024-09-16T09:32:07.418421+0000","last_undegraded":"2024-09-16T09:32:07.418421+0000","last_fullsized":"2024-09-16T09:32:07.418421+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-16T09:28:55.643448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:55.643448+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":19,"num_read_kb":12,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":319,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.416645+0000","last_change":"2024-09-16T09:27:46.175159+0000","last_active":"2024-09-16T09:32:07.416645+0000","last_peered":"2024-09-16T09:32:07.416645+0000","last_clean":"2024-09-16T09:32:07.416645+0000","last_became_active":"2024-09-16T09:25:01.700733+0000","last_became_peered":"2024-09-16T09:25:01.700733+0000","last_unstale":"2024-09-16T09:32:07.416645+0000","last_undegraded":"2024-09-16T09:32:07.416645+0000","last_fullsized":"2024-09-16T09:32:07.416645+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:46.175093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:27:46.175093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":312,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.418701+0000","last_change":"2024-09-16T09:29:25.017736+0000","last_active":"2024-09-16T09:32:07.418701+0000","last_peered":"2024-09-16T09:32:07.418701+0000","last_clean":"2024-09-16T09:32:07.418701+0000","last_became_active":"2024-09-16T09:25:05.713479+0000","last_became_peered":"2024-09-16T09:25:05.713479+0000","last_unstale":"2024-09-16T09:32:07.418701+0000","last_undegraded":"2024-09-16T09:32:07.418701+0000","last_fullsized":"2024-09-16T09:32:07.418701+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:25.017610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:25.017610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":171,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.427534+0000","last_change":"2024-09-16T09:31:09.275121+0000","last_active":"2024-09-16T09:32:07.427534+0000","last_peered":"2024-09-16T09:32:07.427534+0000","last_clean":"2024-09-16T09:32:07.427534+0000","last_became_active":"2024-09-16T09:31:09.274797+0000","last_became_peered":"2024-09-16T09:31:09.274797+0000","last_unstale":"2024-09-16T09:32:07.427534+0000","last_undegraded":"2024-09-16T09:32:07.427534+0000","last_fullsized":"2024-09-16T09:32:07.427534+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:54.589258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:28:54.589258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:48.251310+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"66'4","reported_seq":52,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.423189+0000","last_change":"2024-09-16T09:31:09.401626+0000","last_active":"2024-09-16T09:32:07.423189+0000","last_peered":"2024-09-16T09:32:07.423189+0000","last_clean":"2024-09-16T09:32:07.423189+0000","last_became_active":"2024-09-16T09:31:09.273682+0000","last_became_peered":"2024-09-16T09:31:09.273682+0000","last_unstale":"2024-09-16T09:32:07.423189+0000","last_undegraded":"2024-09-16T09:32:07.423189+0000","last_fullsized":"2024-09-16T09:32:07.423189+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'4","last_scrub_stamp":"2024-09-16T09:26:27.473740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:26:27.473740+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":342,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.427705+0000","last_change":"2024-09-16T09:31:30.235761+0000","last_active":"2024-09-16T09:32:07.427705+0000","last_peered":"2024-09-16T09:32:07.427705+0000","last_clean":"2024-09-16T09:32:07.427705+0000","last_became_active":"2024-09-16T09:31:09.274719+0000","last_became_peered":"2024-09-16T09:31:09.274719+0000","last_unstale":"2024-09-16T09:32:07.427705+0000","last_undegraded":"2024-09-16T09:32:07.427705+0000","last_fullsized":"2024-09-16T09:32:07.427705+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:31:30.235692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:31:30.235692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:49.736157+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":329,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.533628+0000","last_change":"2024-09-16T09:32:07.533575+0000","last_active":"2024-09-16T09:32:07.533628+0000","last_peered":"2024-09-16T09:32:07.533628+0000","last_clean":"2024-09-16T09:32:07.533628+0000","last_became_active":"2024-09-16T09:25:05.712545+0000","last_became_peered":"2024-09-16T09:25:05.712545+0000","last_unstale":"2024-09-16T09:32:07.533628+0000","last_undegraded":"2024-09-16T09:32:07.533628+0000","last_fullsized":"2024-09-16T09:32:07.533628+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:07.533529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:32:07.533529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:28.807212+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":307,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.417454+0000","last_change":"2024-09-16T09:29:27.012810+0000","last_active":"2024-09-16T09:32:07.417454+0000","last_peered":"2024-09-16T09:32:07.417454+0000","last_clean":"2024-09-16T09:32:07.417454+0000","last_became_active":"2024-09-16T09:25:07.720117+0000","last_became_peered":"2024-09-16T09:25:07.720117+0000","last_unstale":"2024-09-16T09:32:07.417454+0000","last_undegraded":"2024-09-16T09:32:07.417454+0000","last_fullsized":"2024-09-16T09:32:07.417454+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:27.012756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:27.012756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":308,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.425359+0000","last_change":"2024-09-16T09:29:19.888825+0000","last_active":"2024-09-16T09:32:07.425359+0000","last_peered":"2024-09-16T09:32:07.425359+0000","last_clean":"2024-09-16T09:32:07.425359+0000","last_became_active":"2024-09-16T09:25:07.712594+0000","last_became_peered":"2024-09-16T09:25:07.712594+0000","last_unstale":"2024-09-16T09:32:07.425359+0000","last_undegraded":"2024-09-16T09:32:07.425359+0000","last_fullsized":"2024-09-16T09:32:07.425359+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:19.888759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:19.888759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":170,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.427305+0000","last_change":"2024-09-16T09:31:26.211625+0000","last_active":"2024-09-16T09:32:07.427305+0000","last_peered":"2024-09-16T09:32:07.427305+0000","last_clean":"2024-09-16T09:32:07.427305+0000","last_became_active":"2024-09-16T09:31:09.271662+0000","last_became_peered":"2024-09-16T09:31:09.271662+0000","last_unstale":"2024-09-16T09:32:07.427305+0000","last_undegraded":"2024-09-16T09:32:07.427305+0000","last_fullsized":"2024-09-16T09:32:07.427305+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:31:26.211363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:31:26.211363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:32.873753+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"66'11","reported_seq":336,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.472023+0000","last_change":"2024-09-16T09:27:57.486459+0000","last_active":"2024-09-16T09:32:07.472023+0000","last_peered":"2024-09-16T09:32:07.472023+0000","last_clean":"2024-09-16T09:32:07.472023+0000","last_became_active":"2024-09-16T09:25:03.707769+0000","last_became_peered":"2024-09-16T09:25:03.707769+0000","last_unstale":"2024-09-16T09:32:07.472023+0000","last_undegraded":"2024-09-16T09:32:07.472023+0000","last_fullsized":"2024-09-16T09:32:07.472023+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-16T09:27:57.486381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:57.486381+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:16.375833+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":305,"reported_epoch":204,"state":"active+clean","last_fresh":"2024-09-16T09:32:07.428109+0000","last_change":"2024-09-16T09:27:57.640962+0000","last_active":"2024-09-16T09:32:07.428109+0000","last_peered":"2024-09-16T09:32:07.428109+0000","last_clean":"2024-09-16T09:32:07.428109+0000","last_became_active":"2024-09-16T09:25:05.711952+0000","last_became_peered":"2024-09-16T09:25:05.711952+0000","last_unstale":"2024-09-16T09:32:07.428109+0000","last_undegraded":"2024-09-16T09:32:07.428109+0000","last_fullsized":"2024-09-16T09:32:07.428109+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:57.640756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:57.640756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:21.189003+0000","scrub_duration":44,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":16,"stat_sum":{"num_bytes":5142085656,"num_objects":78463,"num_object_clones":0,"num_object_copies":235389,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3829,"num_objects_unfound":0,"num_objects_dirty":78463,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":170723,"num_write_kb":7973889,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10880,"num_bytes_recovered":713031680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16143233024,"data_stored":16143220808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":77310,"ondisk_log_size":77310,"up":48,"acting":46,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":70,"ondisk_log_size":70,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":691,"num_read_kb":448,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":126,"num_bytes_recovered":3026,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":395,"ondisk_log_size":395,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1986,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":420,"num_read_kb":415,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":21,"stat_sum":{"num_bytes":806935,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":21,"num_read_kb":40,"num_write":21,"num_write_kb":296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":960396,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1659,"ondisk_log_size":1659,"up":63,"acting":63,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266778,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1801172,"kb_used_data":1581188,"kb_used_omap":1,"kb_used_meta":219966,"kb_avail":91943980,"statfs":{"total":95995035648,"available":94150635520,"internally_reserved":0,"allocated":1619136512,"data_stored":1618848067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":225245642},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495711,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1734792,"kb_used_data":1581364,"kb_used_omap":1,"kb_used_meta":153342,"kb_avail":92010360,"statfs":{"total":95995035648,"available":94218608640,"internally_reserved":0,"allocated":1619316736,"data_stored":1619051815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":157022662},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724646,"num_pgs":64,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2758936,"kb_used_data":2502852,"kb_used_omap":1,"kb_used_meta":256062,"kb_avail":90986216,"statfs":{"total":95995035648,"available":93169885184,"internally_reserved":0,"allocated":2562920448,"data_stored":2562640180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":262207942},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,0,0,4],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953579,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2073748,"kb_used_data":1880132,"kb_used_omap":1,"kb_used_meta":193598,"kb_avail":91671404,"statfs":{"total":95995035648,"available":93871517696,"internally_reserved":0,"allocated":1925255168,"data_stored":1924966763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":198244806},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149809,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1951216,"kb_used_data":1635420,"kb_used_omap":1,"kb_used_meta":315774,"kb_avail":91793936,"statfs":{"total":95995035648,"available":93996990464,"internally_reserved":0,"allocated":1674670080,"data_stored":1674366191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":323353031},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411446,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2524304,"kb_used_data":2244604,"kb_used_omap":1,"kb_used_meta":279678,"kb_avail":91220848,"statfs":{"total":95995035648,"available":93410148352,"internally_reserved":0,"allocated":2298474496,"data_stored":2298201883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":286390725},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574973,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2504428,"kb_used_data":2216284,"kb_used_omap":1,"kb_used_meta":287998,"kb_avail":91240724,"statfs":{"total":95995035648,"available":93430501376,"internally_reserved":0,"allocated":2269474816,"data_stored":2269161705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":294910405},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,1,6,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738499,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2467668,"kb_used_data":2169924,"kb_used_omap":1,"kb_used_meta":297726,"kb_avail":91277484,"statfs":{"total":95995035648,"available":93468143616,"internally_reserved":0,"allocated":2222002176,"data_stored":2221713973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":304871877},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2211184640,"data_stored":2211184640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2252341248,"data_stored":2252341248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2297499648,"data_stored":2297495576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1681522688,"data_stored":1681522688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1909456896,"data_stored":1909456896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":2561150976,"data_stored":2561146904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1616707584,"data_stored":1616707584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1613369344,"data_stored":1613365272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T09:32:12.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:12.737 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:12.737 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:32:12.841 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:32:12.881 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:12.881 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:12.881 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:32:12.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:12 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/2681353372' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:32:12.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:12 smithi084 ceph-mon[32629]: 1.5 scrub starts 2024-09-16T09:32:12.983 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:32:13.003 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:13.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:13.004 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:32:13.105 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:32:13.139 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:13.139 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:13.140 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:32:13.188 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:32:13.240 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:32:13.275 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:13.276 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:13.276 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:32:13.377 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:32:13.410 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:32:12 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:32:13.412 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:13.413 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:13.413 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:32:13.514 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:32:13.543 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-16T09:32:13.544 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:13.544 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:13.545 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:32:13.645 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:32:13.675 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:13.676 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:13.676 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:32:13.713 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:13 smithi070 ceph-mon[30388]: pgmap v335: 168 pgs: 1 peering, 2 active+recovering+undersized+remapped, 165 active+clean; 4.8 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 715 B/s rd, 0 B/s wr, 746 op/s; 3829/235989 objects misplaced (1.623%); 7.4 MiB/s, 117 objects/s recovering 2024-09-16T09:32:13.714 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:13 smithi070 ceph-mon[30388]: 1.2 scrub starts 2024-09-16T09:32:13.714 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:13 smithi070 ceph-mon[30388]: 1.1 scrub starts 2024-09-16T09:32:13.714 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:13 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:32:13.714 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:13 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:32:13.714 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:13 smithi070 ceph-mon[30388]: 1.e scrub starts 2024-09-16T09:32:13.714 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:13 smithi070 ceph-mon[40691]: pgmap v335: 168 pgs: 1 peering, 2 active+recovering+undersized+remapped, 165 active+clean; 4.8 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 715 B/s rd, 0 B/s wr, 746 op/s; 3829/235989 objects misplaced (1.623%); 7.4 MiB/s, 117 objects/s recovering 2024-09-16T09:32:13.714 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:13 smithi070 ceph-mon[40691]: 1.2 scrub starts 2024-09-16T09:32:13.714 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:13 smithi070 ceph-mon[40691]: 1.1 scrub starts 2024-09-16T09:32:13.714 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:13 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:32:13.714 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:13 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:32:13.714 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:13 smithi070 ceph-mon[40691]: 1.e scrub starts 2024-09-16T09:32:13.777 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:32:13.804 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:13.804 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:13.805 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:32:13.906 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:32:13.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:13 smithi084 ceph-mon[32629]: pgmap v335: 168 pgs: 1 peering, 2 active+recovering+undersized+remapped, 165 active+clean; 4.8 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 715 B/s rd, 0 B/s wr, 746 op/s; 3829/235989 objects misplaced (1.623%); 7.4 MiB/s, 117 objects/s recovering 2024-09-16T09:32:13.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:13 smithi084 ceph-mon[32629]: 1.2 scrub starts 2024-09-16T09:32:13.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:13 smithi084 ceph-mon[32629]: 1.1 scrub starts 2024-09-16T09:32:13.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:13 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:32:13.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:13 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:32:13.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:13 smithi084 ceph-mon[32629]: 1.e scrub starts 2024-09-16T09:32:13.945 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:13.946 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:13.946 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:32:13.999 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T09:32:13.999 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:32:14.048 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:32:14.084 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:14.084 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:14.085 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:32:14.185 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:32:14.221 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:14.222 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:14.222 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:32:14.323 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:32:14.360 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:14.361 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:14.361 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:32:14.462 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:32:14.494 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:14.494 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:14.495 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:32:14.595 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:32:14.629 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:14.629 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:14.630 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:32:14.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:14 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:32:14.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:14 smithi070 ceph-mon[30388]: 1.0 scrub starts 2024-09-16T09:32:14.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:14 smithi070 ceph-mon[30388]: osdmap e206: 8 total, 8 up, 8 in 2024-09-16T09:32:14.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:14 smithi070 ceph-mon[30388]: 1.10 scrub starts 2024-09-16T09:32:14.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:14 smithi070 ceph-mon[30388]: 1.f scrub starts 2024-09-16T09:32:14.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:14 smithi070 ceph-mon[30388]: pgmap v337: 168 pgs: 3 active+clean+scrubbing, 2 active+recovering+undersized+remapped, 163 active+clean; 4.5 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1.73k op/s; 3307/219675 objects misplaced (1.505%); 14 MiB/s, 219 objects/s recovering 2024-09-16T09:32:14.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:14 smithi070 ceph-mon[30388]: 1.14 scrub starts 2024-09-16T09:32:14.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:14 smithi070 ceph-mon[30388]: 1.b scrub starts 2024-09-16T09:32:14.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:14 smithi070 ceph-mon[30388]: Health check update: Degraded data redundancy: 2 pgs undersized (PG_DEGRADED) 2024-09-16T09:32:14.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:14 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:32:14.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:14 smithi070 ceph-mon[40691]: 1.0 scrub starts 2024-09-16T09:32:14.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:14 smithi070 ceph-mon[40691]: osdmap e206: 8 total, 8 up, 8 in 2024-09-16T09:32:14.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:14 smithi070 ceph-mon[40691]: 1.10 scrub starts 2024-09-16T09:32:14.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:14 smithi070 ceph-mon[40691]: 1.f scrub starts 2024-09-16T09:32:14.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:14 smithi070 ceph-mon[40691]: pgmap v337: 168 pgs: 3 active+clean+scrubbing, 2 active+recovering+undersized+remapped, 163 active+clean; 4.5 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1.73k op/s; 3307/219675 objects misplaced (1.505%); 14 MiB/s, 219 objects/s recovering 2024-09-16T09:32:14.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:14 smithi070 ceph-mon[40691]: 1.14 scrub starts 2024-09-16T09:32:14.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:14 smithi070 ceph-mon[40691]: 1.b scrub starts 2024-09-16T09:32:14.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:14 smithi070 ceph-mon[40691]: Health check update: Degraded data redundancy: 2 pgs undersized (PG_DEGRADED) 2024-09-16T09:32:14.731 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:32:14.764 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:14.765 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:14.765 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:32:14.866 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:32:14.879 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.0 dump_historic_ops 2024-09-16T09:32:14.891 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:14.892 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:14.893 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:32:14.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:14 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:32:14.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:14 smithi084 ceph-mon[32629]: 1.0 scrub starts 2024-09-16T09:32:14.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:14 smithi084 ceph-mon[32629]: osdmap e206: 8 total, 8 up, 8 in 2024-09-16T09:32:14.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:14 smithi084 ceph-mon[32629]: 1.10 scrub starts 2024-09-16T09:32:14.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:14 smithi084 ceph-mon[32629]: 1.f scrub starts 2024-09-16T09:32:14.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:14 smithi084 ceph-mon[32629]: pgmap v337: 168 pgs: 3 active+clean+scrubbing, 2 active+recovering+undersized+remapped, 163 active+clean; 4.5 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1.73k op/s; 3307/219675 objects misplaced (1.505%); 14 MiB/s, 219 objects/s recovering 2024-09-16T09:32:14.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:14 smithi084 ceph-mon[32629]: 1.14 scrub starts 2024-09-16T09:32:14.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:14 smithi084 ceph-mon[32629]: 1.b scrub starts 2024-09-16T09:32:14.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:14 smithi084 ceph-mon[32629]: Health check update: Degraded data redundancy: 2 pgs undersized (PG_DEGRADED) 2024-09-16T09:32:14.935 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:32:14.994 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:32:15.035 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:15.036 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:15.036 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:32:15.138 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:32:15.173 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:15.174 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:15.174 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:32:15.190 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T09:32:15.275 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:32:15.321 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:15.322 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:15.322 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:32:15.423 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:32:15.471 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:15.472 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:15.472 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:32:15.573 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:32:15.615 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:15.616 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:15.616 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:32:15.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:15 smithi070 ceph-mon[30388]: 1.d scrub starts 2024-09-16T09:32:15.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:15 smithi070 ceph-mon[30388]: 1.12 scrub starts 2024-09-16T09:32:15.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:15 smithi070 ceph-mon[40691]: 1.d scrub starts 2024-09-16T09:32:15.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:15 smithi070 ceph-mon[40691]: 1.12 scrub starts 2024-09-16T09:32:15.717 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:32:15.737 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-16T09:32:15.745 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:15.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:15.747 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:32:15.848 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:32:15.877 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:15.878 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:15.878 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:32:15.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:15 smithi084 ceph-mon[32629]: 1.d scrub starts 2024-09-16T09:32:15.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:15 smithi084 ceph-mon[32629]: 1.12 scrub starts 2024-09-16T09:32:15.979 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:32:16.046 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:16.047 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:16.047 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:32:16.148 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:32:16.174 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:16.175 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:16.175 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:32:16.215 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:32:16.228 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:32:16.233 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:32:16.275 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:32:16.333 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:16.334 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:16.334 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:32:16.435 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:32:16.456 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:16.457 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:16.457 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:32:16.481 INFO:teuthology.orchestra.run.smithi070.stderr:nodeep-scrub is set 2024-09-16T09:32:16.558 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:32:16.587 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:16.588 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:16.588 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:32:16.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:16 smithi070 ceph-mon[30388]: pgmap v338: 168 pgs: 3 active+clean+scrubbing, 2 active+recovering+undersized+remapped, 163 active+clean; 4.5 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.26k op/s; 3307/219675 objects misplaced (1.505%); 11 MiB/s, 170 objects/s recovering 2024-09-16T09:32:16.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:16 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3752096580' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:32:16.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:16 smithi070 ceph-mon[40691]: pgmap v338: 168 pgs: 3 active+clean+scrubbing, 2 active+recovering+undersized+remapped, 163 active+clean; 4.5 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.26k op/s; 3307/219675 objects misplaced (1.505%); 11 MiB/s, 170 objects/s recovering 2024-09-16T09:32:16.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:16 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3752096580' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:32:16.689 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:32:16.722 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:16.722 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:16.723 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:32:16.824 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:32:16.861 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:16.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:16.862 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:32:16.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:16 smithi084 ceph-mon[32629]: pgmap v338: 168 pgs: 3 active+clean+scrubbing, 2 active+recovering+undersized+remapped, 163 active+clean; 4.5 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.26k op/s; 3307/219675 objects misplaced (1.505%); 11 MiB/s, 170 objects/s recovering 2024-09-16T09:32:16.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:16 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3752096580' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:32:16.963 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:32:16.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:16.997 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:16.997 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:32:17.098 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:32:17.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:17.139 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:17.139 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:32:17.240 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:32:17.271 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:17.272 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:17.272 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:32:17.373 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:32:17.411 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:17.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:17.412 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:32:17.513 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:32:17.548 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:17.549 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:17.549 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:32:17.650 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:32:17.685 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:17.685 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:17.686 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:32:17.777 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:17 smithi070 ceph-mon[30388]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:32:17.778 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:17 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3752096580' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-16T09:32:17.778 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:17 smithi070 ceph-mon[30388]: osdmap e207: 8 total, 8 up, 8 in 2024-09-16T09:32:17.778 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:17 smithi070 ceph-mon[40691]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:32:17.778 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:17 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3752096580' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-16T09:32:17.778 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:17 smithi070 ceph-mon[40691]: osdmap e207: 8 total, 8 up, 8 in 2024-09-16T09:32:17.786 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:32:17.814 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:17.815 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:17.816 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:32:17.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:17 smithi084 ceph-mon[32629]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:32:17.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:17 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3752096580' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-16T09:32:17.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:17 smithi084 ceph-mon[32629]: osdmap e207: 8 total, 8 up, 8 in 2024-09-16T09:32:17.917 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:32:17.954 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:17.955 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:17.955 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:32:17.963 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:32:17.963 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:32:17.996 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:32:17.996 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:32:18.029 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:32:18.030 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:32:18.056 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:32:18.061 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:32:18.061 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:32:18.090 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:18.091 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:18.091 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:32:18.096 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:32:18.096 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:32:18.132 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:32:18.133 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:32:18.162 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:32:18.163 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:32:18.192 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:32:18.198 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:32:18.199 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:32:18.218 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:18.219 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:18.220 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:32:18.320 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:32:18.347 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:18.348 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:18.348 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:32:18.449 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:32:18.476 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:18.476 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:18.477 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:32:18.577 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:32:18.605 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:18.605 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:18.605 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:32:18.706 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:32:18.729 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T09:32:18.729 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 20, 2024-09-16T09:32:18.729 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 600, 2024-09-16T09:32:18.729 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [ 2024-09-16T09:32:18.729 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.729 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.24338.0:1 3.12 3:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-09-16T09:32:18.729 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:25:01.537527+0000", 2024-09-16T09:32:18.729 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 437.18686403100003, 2024-09-16T09:32:18.729 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.165654407, 2024-09-16T09:32:18.729 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:32:18.729 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:32:18.730 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T09:32:18.730 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:32:18.730 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.24338", 2024-09-16T09:32:18.730 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/4143256096", 2024-09-16T09:32:18.730 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 1 2024-09-16T09:32:18.730 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.730 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:32:18.730 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.730 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:32:18.730 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:25:01.537527+0000", 2024-09-16T09:32:18.730 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:32:18.730 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.730 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.731 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:32:18.731 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:25:01.537527+0000", 2024-09-16T09:32:18.731 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:32:18.731 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.731 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.731 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:32:18.731 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:25:01.537532+0000", 2024-09-16T09:32:18.731 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.4390000000000002e-06 2024-09-16T09:32:18.731 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.731 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.731 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:32:18.731 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:25:01.537538+0000", 2024-09-16T09:32:18.731 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1109999999999996e-06 2024-09-16T09:32:18.731 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.732 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.732 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:32:18.732 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:25:01.537547+0000", 2024-09-16T09:32:18.732 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.5820000000000008e-06 2024-09-16T09:32:18.732 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.732 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.732 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:32:18.732 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:25:01.537557+0000", 2024-09-16T09:32:18.732 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.5980000000000003e-06 2024-09-16T09:32:18.732 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.732 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.732 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:32:18.732 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:25:01.537612+0000", 2024-09-16T09:32:18.733 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5380000000000002e-05 2024-09-16T09:32:18.733 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.733 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.733 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:32:18.733 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:25:01.537620+0000", 2024-09-16T09:32:18.733 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6389999999999992e-06 2024-09-16T09:32:18.733 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.733 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.733 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:32:18.733 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:25:01.703120+0000", 2024-09-16T09:32:18.733 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.165499545 2024-09-16T09:32:18.733 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.733 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.734 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:32:18.734 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:25:01.703182+0000", 2024-09-16T09:32:18.734 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.2113000000000007e-05 2024-09-16T09:32:18.734 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:32:18.734 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:32:18.734 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:32:18.734 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.734 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.734 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.24698.0:60232 7.4 7:2f837c0e:::benchmark_data_smithi070_78438_object60231:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-16T09:32:18.734 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:26:44.343549+0000", 2024-09-16T09:32:18.734 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 334.38084258599997, 2024-09-16T09:32:18.734 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.135718961, 2024-09-16T09:32:18.734 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:32:18.734 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:32:18.735 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:32:18.735 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:32:18.735 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.24698", 2024-09-16T09:32:18.735 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:32:18.735 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 60232 2024-09-16T09:32:18.735 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.735 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:32:18.735 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.735 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:32:18.735 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.343549+0000", 2024-09-16T09:32:18.735 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:32:18.735 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.735 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.735 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:32:18.736 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.343549+0000", 2024-09-16T09:32:18.736 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:32:18.736 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.736 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.736 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:32:18.736 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.343549+0000", 2024-09-16T09:32:18.736 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.7899999999999998e-07 2024-09-16T09:32:18.736 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.736 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.736 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:32:18.736 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.343568+0000", 2024-09-16T09:32:18.736 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.8346999999999999e-05 2024-09-16T09:32:18.736 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.737 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.737 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:32:18.737 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.343577+0000", 2024-09-16T09:32:18.737 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.8729999999999996e-06 2024-09-16T09:32:18.737 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.737 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.737 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:32:18.737 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.343583+0000", 2024-09-16T09:32:18.737 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.3589999999999996e-06 2024-09-16T09:32:18.737 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.737 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.737 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:32:18.737 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.345880+0000", 2024-09-16T09:32:18.737 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0022970149999999999 2024-09-16T09:32:18.738 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.738 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.738 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:32:18.738 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.345985+0000", 2024-09-16T09:32:18.738 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000105142 2024-09-16T09:32:18.738 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.738 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.738 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,4", 2024-09-16T09:32:18.738 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.346124+0000", 2024-09-16T09:32:18.738 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00013885899999999999 2024-09-16T09:32:18.738 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.738 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.738 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:32:18.738 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.348832+0000", 2024-09-16T09:32:18.738 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0027077260000000001 2024-09-16T09:32:18.739 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.739 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.739 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:32:18.739 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.351646+0000", 2024-09-16T09:32:18.739 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0028142200000000001 2024-09-16T09:32:18.739 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.739 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.739 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:32:18.739 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.479170+0000", 2024-09-16T09:32:18.739 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.12752422399999999 2024-09-16T09:32:18.739 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.739 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.739 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:32:18.739 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.479221+0000", 2024-09-16T09:32:18.740 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.1267999999999998e-05 2024-09-16T09:32:18.740 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.740 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.740 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:32:18.740 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.479268+0000", 2024-09-16T09:32:18.740 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.6248999999999997e-05 2024-09-16T09:32:18.740 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:32:18.740 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:32:18.740 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:32:18.740 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.740 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.740 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.24698.0:60246 7.d e98/82 7:b1444fdd:::benchmark_data_smithi070_78438_object60245:head v 98'3796, mlcod=98'3794)", 2024-09-16T09:32:18.740 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:26:44.352790+0000", 2024-09-16T09:32:18.740 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 334.37160152500002, 2024-09-16T09:32:18.741 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.135640757, 2024-09-16T09:32:18.741 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:32:18.741 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:32:18.741 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:32:18.741 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:32:18.741 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.741 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:32:18.741 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.352790+0000", 2024-09-16T09:32:18.741 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:32:18.741 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.741 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.741 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:32:18.741 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.352790+0000", 2024-09-16T09:32:18.742 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:32:18.742 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.742 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.742 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:32:18.742 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.352790+0000", 2024-09-16T09:32:18.742 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.7399999999999998e-07 2024-09-16T09:32:18.742 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.742 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.742 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:32:18.742 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.355226+0000", 2024-09-16T09:32:18.742 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0024358269999999998 2024-09-16T09:32:18.742 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.742 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.742 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:32:18.742 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.355236+0000", 2024-09-16T09:32:18.743 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0325e-05 2024-09-16T09:32:18.743 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.743 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.744 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:32:18.744 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.355244+0000", 2024-09-16T09:32:18.744 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.1400000000000002e-06 2024-09-16T09:32:18.744 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.744 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.744 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:32:18.744 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.355470+0000", 2024-09-16T09:32:18.744 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00022637699999999999 2024-09-16T09:32:18.744 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.744 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.745 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:32:18.745 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.479298+0000", 2024-09-16T09:32:18.745 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.123828542 2024-09-16T09:32:18.745 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.745 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.745 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:32:18.745 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.488386+0000", 2024-09-16T09:32:18.745 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0090874719999999992 2024-09-16T09:32:18.745 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.746 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.746 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:32:18.746 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.488431+0000", 2024-09-16T09:32:18.746 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.46e-05 2024-09-16T09:32:18.746 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:32:18.746 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:32:18.746 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:32:18.746 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.746 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.746 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.24698.0:60249 7.6 7:6d87f461:::benchmark_data_smithi070_78438_object60248:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-16T09:32:18.746 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:26:44.353156+0000", 2024-09-16T09:32:18.746 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 334.371235571, 2024-09-16T09:32:18.746 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.139856962, 2024-09-16T09:32:18.746 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:32:18.747 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:32:18.747 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:32:18.747 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:32:18.747 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.24698", 2024-09-16T09:32:18.747 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:32:18.747 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 60249 2024-09-16T09:32:18.747 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.747 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:32:18.747 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.747 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:32:18.747 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.353156+0000", 2024-09-16T09:32:18.747 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:32:18.747 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.747 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.748 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:32:18.748 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.353156+0000", 2024-09-16T09:32:18.748 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:32:18.748 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.748 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.748 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:32:18.748 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.353157+0000", 2024-09-16T09:32:18.748 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.3600000000000002e-07 2024-09-16T09:32:18.748 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.748 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.748 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:32:18.748 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.353177+0000", 2024-09-16T09:32:18.748 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0285e-05 2024-09-16T09:32:18.749 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.749 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.749 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:32:18.749 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.353186+0000", 2024-09-16T09:32:18.749 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.3139999999999993e-06 2024-09-16T09:32:18.749 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.749 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.749 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:32:18.749 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.353193+0000", 2024-09-16T09:32:18.749 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.5089999999999999e-06 2024-09-16T09:32:18.749 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.749 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.749 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:32:18.749 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.355179+0000", 2024-09-16T09:32:18.749 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0019857379999999999 2024-09-16T09:32:18.750 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.750 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.750 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:32:18.750 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.355294+0000", 2024-09-16T09:32:18.750 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000115375 2024-09-16T09:32:18.750 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.750 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.750 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 3,7", 2024-09-16T09:32:18.750 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.355429+0000", 2024-09-16T09:32:18.750 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00013529500000000001 2024-09-16T09:32:18.750 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.750 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.750 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:32:18.750 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.482445+0000", 2024-09-16T09:32:18.751 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.12701626799999999 2024-09-16T09:32:18.751 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.751 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.751 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:32:18.751 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.488065+0000", 2024-09-16T09:32:18.751 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0056195510000000004 2024-09-16T09:32:18.751 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.751 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.751 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:32:18.751 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.492938+0000", 2024-09-16T09:32:18.751 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0048733789999999997 2024-09-16T09:32:18.751 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.751 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.752 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:32:18.752 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.492978+0000", 2024-09-16T09:32:18.752 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9425000000000003e-05 2024-09-16T09:32:18.752 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.752 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.752 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:32:18.752 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.493013+0000", 2024-09-16T09:32:18.752 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4887e-05 2024-09-16T09:32:18.752 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:32:18.752 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:32:18.752 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:32:18.752 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.752 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.752 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.24698.0:60251 7.6 7:6833f76d:::benchmark_data_smithi070_78438_object60250:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-16T09:32:18.753 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:26:44.355188+0000", 2024-09-16T09:32:18.753 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 334.36920371399998, 2024-09-16T09:32:18.753 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.138060128, 2024-09-16T09:32:18.753 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:32:18.753 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:32:18.753 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:32:18.753 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:32:18.753 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.24698", 2024-09-16T09:32:18.753 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:32:18.753 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 60251 2024-09-16T09:32:18.753 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.753 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:32:18.753 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.753 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:32:18.754 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.355188+0000", 2024-09-16T09:32:18.754 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:32:18.754 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.754 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.754 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:32:18.754 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.355188+0000", 2024-09-16T09:32:18.754 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:32:18.754 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.754 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.754 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:32:18.754 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.355189+0000", 2024-09-16T09:32:18.754 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0759999999999999e-06 2024-09-16T09:32:18.754 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.754 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.755 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:32:18.755 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.355208+0000", 2024-09-16T09:32:18.755 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9485000000000001e-05 2024-09-16T09:32:18.755 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.755 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.755 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:32:18.755 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.355217+0000", 2024-09-16T09:32:18.755 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.9840000000000007e-06 2024-09-16T09:32:18.755 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.755 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.755 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:32:18.755 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.355224+0000", 2024-09-16T09:32:18.755 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.8399999999999997e-06 2024-09-16T09:32:18.755 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.756 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.756 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:32:18.756 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.482940+0000", 2024-09-16T09:32:18.756 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.12771575900000001 2024-09-16T09:32:18.756 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.756 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.756 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:32:18.756 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.483092+0000", 2024-09-16T09:32:18.756 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00015273500000000001 2024-09-16T09:32:18.756 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.756 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.756 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 3,7", 2024-09-16T09:32:18.756 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.483214+0000", 2024-09-16T09:32:18.756 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000121345 2024-09-16T09:32:18.757 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.757 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.757 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:32:18.757 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.488034+0000", 2024-09-16T09:32:18.757 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.004820202 2024-09-16T09:32:18.757 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.757 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.757 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:32:18.757 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.488093+0000", 2024-09-16T09:32:18.757 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.9089000000000003e-05 2024-09-16T09:32:18.757 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.757 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.757 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:32:18.758 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.493019+0000", 2024-09-16T09:32:18.758 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0049259489999999998 2024-09-16T09:32:18.758 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.758 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.758 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:32:18.758 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.493097+0000", 2024-09-16T09:32:18.758 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.8178999999999998e-05 2024-09-16T09:32:18.758 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.758 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.758 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:32:18.758 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:44.493248+0000", 2024-09-16T09:32:18.758 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00015048500000000001 2024-09-16T09:32:18.758 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:32:18.758 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:32:18.759 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:32:18.759 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.759 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.759 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.24698.0:124192 7.e e118/82 7:7f7040ae:::benchmark_data_smithi070_78438_object124191:head v 118'7729, mlcod=118'7724)", 2024-09-16T09:32:18.759 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:27:41.119969+0000", 2024-09-16T09:32:18.759 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 277.604422677, 2024-09-16T09:32:18.759 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.13654496399999999, 2024-09-16T09:32:18.759 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:32:18.759 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:32:18.759 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:32:18.759 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:32:18.759 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.759 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:32:18.759 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.119969+0000", 2024-09-16T09:32:18.759 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:32:18.760 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.760 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.760 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:32:18.760 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.119969+0000", 2024-09-16T09:32:18.760 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:32:18.760 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.760 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.760 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:32:18.760 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.119970+0000", 2024-09-16T09:32:18.760 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.9000000000000005e-07 2024-09-16T09:32:18.760 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.760 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.760 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:32:18.760 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.119995+0000", 2024-09-16T09:32:18.761 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.5851e-05 2024-09-16T09:32:18.761 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.761 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.761 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:32:18.761 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.120007+0000", 2024-09-16T09:32:18.761 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1534e-05 2024-09-16T09:32:18.761 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.761 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.761 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:32:18.761 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.120014+0000", 2024-09-16T09:32:18.761 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.8140000000000003e-06 2024-09-16T09:32:18.761 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.761 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.761 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:32:18.762 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.122059+0000", 2024-09-16T09:32:18.762 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.002044985 2024-09-16T09:32:18.762 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.762 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.762 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:32:18.762 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.122079+0000", 2024-09-16T09:32:18.762 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9746e-05 2024-09-16T09:32:18.762 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.762 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.762 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:32:18.762 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.256331+0000", 2024-09-16T09:32:18.762 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.13425289400000001 2024-09-16T09:32:18.762 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.762 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.763 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:32:18.763 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.256514+0000", 2024-09-16T09:32:18.763 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00018215000000000001 2024-09-16T09:32:18.763 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:32:18.763 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:32:18.763 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:32:18.763 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.763 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.763 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.24698.0:124197 7.4 7:263ea75f:::benchmark_data_smithi070_78438_object124196:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-16T09:32:18.763 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:27:41.122635+0000", 2024-09-16T09:32:18.763 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 277.60175636100001, 2024-09-16T09:32:18.763 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.13497726600000001, 2024-09-16T09:32:18.763 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:32:18.763 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:32:18.764 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:32:18.764 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:32:18.764 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.24698", 2024-09-16T09:32:18.764 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:32:18.764 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 124197 2024-09-16T09:32:18.764 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.764 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:32:18.764 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.764 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:32:18.764 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.122635+0000", 2024-09-16T09:32:18.764 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:32:18.764 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.764 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.765 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:32:18.765 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.122635+0000", 2024-09-16T09:32:18.765 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:32:18.765 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.765 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.765 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:32:18.765 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.122636+0000", 2024-09-16T09:32:18.765 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2109999999999999e-06 2024-09-16T09:32:18.765 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.765 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.765 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:32:18.765 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.122657+0000", 2024-09-16T09:32:18.765 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.1111000000000002e-05 2024-09-16T09:32:18.765 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.766 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.766 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:32:18.766 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.122665+0000", 2024-09-16T09:32:18.766 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.8269999999999998e-06 2024-09-16T09:32:18.766 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.766 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.766 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:32:18.766 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.122671+0000", 2024-09-16T09:32:18.766 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.6679999999999998e-06 2024-09-16T09:32:18.766 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.766 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.766 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:32:18.766 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.122799+0000", 2024-09-16T09:32:18.766 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00012808000000000001 2024-09-16T09:32:18.766 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.767 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.767 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:32:18.767 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.122919+0000", 2024-09-16T09:32:18.767 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00012026799999999999 2024-09-16T09:32:18.767 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.767 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.767 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,4", 2024-09-16T09:32:18.767 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.123075+0000", 2024-09-16T09:32:18.767 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00015614599999999999 2024-09-16T09:32:18.767 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.767 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.767 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:32:18.767 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.126042+0000", 2024-09-16T09:32:18.767 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.002966621 2024-09-16T09:32:18.768 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.768 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.768 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:32:18.768 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.257476+0000", 2024-09-16T09:32:18.768 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.13143418000000001 2024-09-16T09:32:18.768 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.768 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.768 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:32:18.768 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.257511+0000", 2024-09-16T09:32:18.768 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4891999999999997e-05 2024-09-16T09:32:18.768 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.768 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.768 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:32:18.768 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.257564+0000", 2024-09-16T09:32:18.769 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.2617999999999997e-05 2024-09-16T09:32:18.769 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.769 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.769 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:32:18.769 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.257612+0000", 2024-09-16T09:32:18.769 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.8643999999999997e-05 2024-09-16T09:32:18.769 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:32:18.769 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:32:18.769 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:32:18.769 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.769 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.769 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.24698.0:124201 7.9 7:983fd109:::benchmark_data_smithi070_78438_object124200:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-16T09:32:18.769 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:27:41.126614+0000", 2024-09-16T09:32:18.769 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 277.59777681399999, 2024-09-16T09:32:18.770 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.13937075600000001, 2024-09-16T09:32:18.770 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:32:18.770 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:32:18.770 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:32:18.770 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:32:18.770 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.24698", 2024-09-16T09:32:18.770 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:32:18.770 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 124201 2024-09-16T09:32:18.772 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.772 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:32:18.772 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.772 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:32:18.772 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.126614+0000", 2024-09-16T09:32:18.773 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:32:18.773 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.773 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.773 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:32:18.773 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.126614+0000", 2024-09-16T09:32:18.773 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:32:18.773 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.773 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.773 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:32:18.773 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.126616+0000", 2024-09-16T09:32:18.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:18.774 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:18.774 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:32:18.775 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.111e-06 2024-09-16T09:32:18.775 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.775 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.775 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:32:18.775 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.126631+0000", 2024-09-16T09:32:18.775 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5794000000000002e-05 2024-09-16T09:32:18.775 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.775 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.775 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:32:18.775 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.126641+0000", 2024-09-16T09:32:18.776 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.4210000000000001e-06 2024-09-16T09:32:18.776 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.776 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.776 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:32:18.776 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.126646+0000", 2024-09-16T09:32:18.776 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.1390000000000003e-06 2024-09-16T09:32:18.776 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.776 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.776 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:32:18.776 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.126692+0000", 2024-09-16T09:32:18.776 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.6221999999999999e-05 2024-09-16T09:32:18.776 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.776 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.777 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:32:18.777 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.126809+0000", 2024-09-16T09:32:18.777 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000116993 2024-09-16T09:32:18.777 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.777 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.777 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,4", 2024-09-16T09:32:18.777 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.126948+0000", 2024-09-16T09:32:18.777 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00013902700000000001 2024-09-16T09:32:18.777 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.777 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.777 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:32:18.777 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.259431+0000", 2024-09-16T09:32:18.777 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.13248237500000001 2024-09-16T09:32:18.777 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.778 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.778 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:32:18.778 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.259490+0000", 2024-09-16T09:32:18.778 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.9830000000000001e-05 2024-09-16T09:32:18.778 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.778 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.778 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:32:18.778 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.265895+0000", 2024-09-16T09:32:18.778 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0064041790000000003 2024-09-16T09:32:18.778 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.778 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.778 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:32:18.778 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.265945+0000", 2024-09-16T09:32:18.778 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0835999999999999e-05 2024-09-16T09:32:18.778 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.779 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.779 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:32:18.779 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.265985+0000", 2024-09-16T09:32:18.779 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9829000000000001e-05 2024-09-16T09:32:18.779 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:32:18.779 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:32:18.779 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:32:18.779 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.779 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.779 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.24698.0:124206 7.6 7:6733724f:::benchmark_data_smithi070_78438_object124205:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-16T09:32:18.779 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:27:41.126830+0000", 2024-09-16T09:32:18.779 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 277.59756156100002, 2024-09-16T09:32:18.779 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.14047876400000001, 2024-09-16T09:32:18.779 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:32:18.780 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:32:18.780 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:32:18.780 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:32:18.780 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.24698", 2024-09-16T09:32:18.780 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:32:18.780 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 124206 2024-09-16T09:32:18.780 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.780 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:32:18.780 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.780 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:32:18.780 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.126830+0000", 2024-09-16T09:32:18.780 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:32:18.780 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.781 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.781 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:32:18.781 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.126830+0000", 2024-09-16T09:32:18.781 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:32:18.781 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.781 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.781 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:32:18.781 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.126831+0000", 2024-09-16T09:32:18.781 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.7700000000000003e-07 2024-09-16T09:32:18.781 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.781 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.781 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:32:18.781 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.126851+0000", 2024-09-16T09:32:18.782 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0069e-05 2024-09-16T09:32:18.782 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.782 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.782 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:32:18.782 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.126860+0000", 2024-09-16T09:32:18.782 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.5559999999999996e-06 2024-09-16T09:32:18.782 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.782 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.782 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:32:18.782 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.126867+0000", 2024-09-16T09:32:18.782 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.0600000000000002e-06 2024-09-16T09:32:18.782 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.782 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.782 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:32:18.783 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.126902+0000", 2024-09-16T09:32:18.783 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4652e-05 2024-09-16T09:32:18.783 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.783 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.783 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:32:18.783 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.126974+0000", 2024-09-16T09:32:18.783 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.1738999999999999e-05 2024-09-16T09:32:18.783 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.783 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.783 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 3,7", 2024-09-16T09:32:18.783 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.256433+0000", 2024-09-16T09:32:18.783 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.12945948600000001 2024-09-16T09:32:18.783 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.783 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.783 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:32:18.784 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.266075+0000", 2024-09-16T09:32:18.784 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0096417329999999996 2024-09-16T09:32:18.784 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.784 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.784 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:32:18.784 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.266816+0000", 2024-09-16T09:32:18.784 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00074122599999999999 2024-09-16T09:32:18.784 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.784 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.784 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:32:18.784 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.267216+0000", 2024-09-16T09:32:18.784 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00039957000000000002 2024-09-16T09:32:18.784 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.785 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.785 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:32:18.785 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.267268+0000", 2024-09-16T09:32:18.785 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.2166999999999997e-05 2024-09-16T09:32:18.785 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.785 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.785 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:32:18.785 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.267309+0000", 2024-09-16T09:32:18.785 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0629e-05 2024-09-16T09:32:18.785 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:32:18.785 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:32:18.785 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:32:18.785 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.785 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.786 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.24698.0:124207 7.f 7:f239dbf0:::benchmark_data_smithi070_78438_object124206:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-16T09:32:18.786 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:27:41.126880+0000", 2024-09-16T09:32:18.786 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 277.59751098100003, 2024-09-16T09:32:18.786 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.14893615199999999, 2024-09-16T09:32:18.786 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:32:18.786 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:32:18.786 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:32:18.786 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:32:18.786 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.24698", 2024-09-16T09:32:18.786 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:32:18.786 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 124207 2024-09-16T09:32:18.786 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.786 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:32:18.786 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.786 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:32:18.787 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.126880+0000", 2024-09-16T09:32:18.787 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:32:18.787 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.787 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.787 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:32:18.787 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.126880+0000", 2024-09-16T09:32:18.787 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:32:18.787 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.787 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.787 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:32:18.787 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.126881+0000", 2024-09-16T09:32:18.787 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.9900000000000001e-07 2024-09-16T09:32:18.787 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.787 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.788 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:32:18.788 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.126900+0000", 2024-09-16T09:32:18.788 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.8794999999999999e-05 2024-09-16T09:32:18.788 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.788 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.788 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:32:18.788 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.126907+0000", 2024-09-16T09:32:18.788 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.4739999999999999e-06 2024-09-16T09:32:18.788 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.788 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.788 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:32:18.788 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.126912+0000", 2024-09-16T09:32:18.788 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0780000000000003e-06 2024-09-16T09:32:18.788 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.789 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.789 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:32:18.789 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.126955+0000", 2024-09-16T09:32:18.789 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2762000000000002e-05 2024-09-16T09:32:18.789 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.789 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.789 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:32:18.789 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.127061+0000", 2024-09-16T09:32:18.789 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000106092 2024-09-16T09:32:18.789 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.789 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.789 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 3,7", 2024-09-16T09:32:18.789 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.258383+0000", 2024-09-16T09:32:18.789 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.13132201099999999 2024-09-16T09:32:18.790 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.790 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.790 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:32:18.790 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.270348+0000", 2024-09-16T09:32:18.790 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.011965379 2024-09-16T09:32:18.790 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.790 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.790 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:32:18.790 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.274865+0000", 2024-09-16T09:32:18.790 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0045163240000000004 2024-09-16T09:32:18.790 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.790 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.790 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:32:18.790 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.275717+0000", 2024-09-16T09:32:18.791 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00085265700000000005 2024-09-16T09:32:18.791 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.791 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.791 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:32:18.791 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.275773+0000", 2024-09-16T09:32:18.791 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5497000000000001e-05 2024-09-16T09:32:18.791 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.791 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.791 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:32:18.791 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.275816+0000", 2024-09-16T09:32:18.791 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.3584000000000001e-05 2024-09-16T09:32:18.791 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:32:18.791 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:32:18.791 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:32:18.792 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.792 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.792 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.24698.0:124213 7.9 7:975b8d0d:::benchmark_data_smithi070_78438_object124212:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-09-16T09:32:18.792 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:27:41.132188+0000", 2024-09-16T09:32:18.792 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 277.592203079, 2024-09-16T09:32:18.792 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.13387122000000001, 2024-09-16T09:32:18.792 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:32:18.792 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:32:18.792 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:32:18.792 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:32:18.792 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.24698", 2024-09-16T09:32:18.792 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:32:18.792 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 124213 2024-09-16T09:32:18.792 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.793 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:32:18.793 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.793 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:32:18.793 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.132188+0000", 2024-09-16T09:32:18.793 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:32:18.793 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.793 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.793 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:32:18.793 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.132188+0000", 2024-09-16T09:32:18.793 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:32:18.793 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.793 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.793 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:32:18.793 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.132190+0000", 2024-09-16T09:32:18.794 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6929999999999999e-06 2024-09-16T09:32:18.794 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.794 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.794 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:32:18.794 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.132212+0000", 2024-09-16T09:32:18.794 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.1968e-05 2024-09-16T09:32:18.794 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.794 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.794 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:32:18.794 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.259714+0000", 2024-09-16T09:32:18.794 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.12750203800000001 2024-09-16T09:32:18.794 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.794 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.794 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:32:18.795 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.259725+0000", 2024-09-16T09:32:18.795 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1455e-05 2024-09-16T09:32:18.795 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.795 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.795 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:32:18.795 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.259849+0000", 2024-09-16T09:32:18.795 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00012351300000000001 2024-09-16T09:32:18.795 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.795 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.795 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:32:18.795 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.259974+0000", 2024-09-16T09:32:18.795 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000125184 2024-09-16T09:32:18.795 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.795 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.796 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,4", 2024-09-16T09:32:18.796 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.260109+0000", 2024-09-16T09:32:18.796 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00013443 2024-09-16T09:32:18.796 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.796 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.796 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:32:18.796 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.263475+0000", 2024-09-16T09:32:18.796 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0033660750000000001 2024-09-16T09:32:18.796 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.796 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.796 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:32:18.796 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.265772+0000", 2024-09-16T09:32:18.796 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0022974699999999998 2024-09-16T09:32:18.796 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.797 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.797 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:32:18.797 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.265992+0000", 2024-09-16T09:32:18.797 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000220392 2024-09-16T09:32:18.797 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.797 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.797 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:32:18.797 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.266031+0000", 2024-09-16T09:32:18.797 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9044000000000001e-05 2024-09-16T09:32:18.797 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.797 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.797 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:32:18.797 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:27:41.266059+0000", 2024-09-16T09:32:18.797 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.7957999999999999e-05 2024-09-16T09:32:18.797 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:32:18.798 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:32:18.798 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:32:18.798 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.798 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.798 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.24698.0:203418 7.4 7:263e3061:::benchmark_data_smithi070_78438_object77287:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135)", 2024-09-16T09:32:18.798 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:28:51.959895+0000", 2024-09-16T09:32:18.798 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 206.76449630100001, 2024-09-16T09:32:18.798 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.18330342699999999, 2024-09-16T09:32:18.798 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:32:18.798 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:32:18.798 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:32:18.798 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:32:18.798 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.24698", 2024-09-16T09:32:18.798 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:32:18.799 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 203418 2024-09-16T09:32:18.799 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.799 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:32:18.799 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.799 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:32:18.799 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.959895+0000", 2024-09-16T09:32:18.799 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:32:18.799 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.799 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.799 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:32:18.799 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.959895+0000", 2024-09-16T09:32:18.799 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:32:18.799 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.799 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.799 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:32:18.800 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.959896+0000", 2024-09-16T09:32:18.800 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.372e-06 2024-09-16T09:32:18.800 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.800 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.800 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:32:18.800 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.959900+0000", 2024-09-16T09:32:18.800 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.219e-06 2024-09-16T09:32:18.800 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.800 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.800 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:32:18.800 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.959908+0000", 2024-09-16T09:32:18.800 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.9500000000000001e-06 2024-09-16T09:32:18.800 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.800 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.800 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:32:18.801 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.959913+0000", 2024-09-16T09:32:18.801 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.614e-06 2024-09-16T09:32:18.801 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.801 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.801 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:32:18.801 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.959985+0000", 2024-09-16T09:32:18.801 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.2107999999999995e-05 2024-09-16T09:32:18.801 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.801 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.801 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:32:18.801 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.960137+0000", 2024-09-16T09:32:18.801 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00015132500000000001 2024-09-16T09:32:18.801 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.801 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.802 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,4", 2024-09-16T09:32:18.802 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.961154+0000", 2024-09-16T09:32:18.802 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0010178120000000001 2024-09-16T09:32:18.802 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.802 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.802 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:32:18.802 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.970624+0000", 2024-09-16T09:32:18.802 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0094698930000000001 2024-09-16T09:32:18.802 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.802 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.802 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:32:18.802 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.980402+0000", 2024-09-16T09:32:18.802 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0097777750000000007 2024-09-16T09:32:18.802 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.803 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.803 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:32:18.803 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.143109+0000", 2024-09-16T09:32:18.803 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.162707141 2024-09-16T09:32:18.803 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.803 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.803 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:32:18.803 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.143160+0000", 2024-09-16T09:32:18.803 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0945000000000001e-05 2024-09-16T09:32:18.803 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.803 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.803 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:32:18.803 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.143198+0000", 2024-09-16T09:32:18.803 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8272999999999998e-05 2024-09-16T09:32:18.804 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:32:18.804 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:32:18.804 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:32:18.804 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.804 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.804 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.24698.0:203419 7.6 7:6733c1b2:::benchmark_data_smithi070_78438_object77288:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135)", 2024-09-16T09:32:18.804 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:28:51.959930+0000", 2024-09-16T09:32:18.804 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 206.764460958, 2024-09-16T09:32:18.804 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.16291588900000001, 2024-09-16T09:32:18.804 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:32:18.804 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:32:18.804 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:32:18.804 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:32:18.804 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.24698", 2024-09-16T09:32:18.805 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:32:18.805 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 203419 2024-09-16T09:32:18.805 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.805 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:32:18.805 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.805 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:32:18.805 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.959930+0000", 2024-09-16T09:32:18.805 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:32:18.805 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.805 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.805 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:32:18.805 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.959930+0000", 2024-09-16T09:32:18.805 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:32:18.805 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.806 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.806 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:32:18.806 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.959931+0000", 2024-09-16T09:32:18.806 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6699999999999999e-07 2024-09-16T09:32:18.806 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.806 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.806 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:32:18.806 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.959933+0000", 2024-09-16T09:32:18.806 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.95e-06 2024-09-16T09:32:18.806 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.807 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.807 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:32:18.807 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.959938+0000", 2024-09-16T09:32:18.807 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.6010000000000002e-06 2024-09-16T09:32:18.807 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.807 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.807 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:32:18.807 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.959943+0000", 2024-09-16T09:32:18.807 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.3320000000000002e-06 2024-09-16T09:32:18.807 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.807 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.807 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:32:18.807 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.960136+0000", 2024-09-16T09:32:18.807 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000193439 2024-09-16T09:32:18.808 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.808 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.808 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:32:18.808 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.960367+0000", 2024-09-16T09:32:18.808 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000231134 2024-09-16T09:32:18.808 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.808 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.808 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 3,7", 2024-09-16T09:32:18.808 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.960487+0000", 2024-09-16T09:32:18.808 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000120111 2024-09-16T09:32:18.808 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.808 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.808 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:32:18.808 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.963773+0000", 2024-09-16T09:32:18.809 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.003285587 2024-09-16T09:32:18.809 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.809 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.809 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:32:18.809 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.987222+0000", 2024-09-16T09:32:18.809 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.023448900000000002 2024-09-16T09:32:18.809 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.809 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.809 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:32:18.809 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.122785+0000", 2024-09-16T09:32:18.809 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.13556333200000001 2024-09-16T09:32:18.809 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.809 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.810 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:32:18.810 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.122817+0000", 2024-09-16T09:32:18.810 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2375000000000001e-05 2024-09-16T09:32:18.810 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.810 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.810 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:32:18.810 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.122846+0000", 2024-09-16T09:32:18.810 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.8761000000000001e-05 2024-09-16T09:32:18.810 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:32:18.810 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:32:18.810 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:32:18.810 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.810 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.810 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.24698.0:203421 7.4 7:243428a8:::benchmark_data_smithi070_78438_object77290:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135)", 2024-09-16T09:32:18.810 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:28:51.963412+0000", 2024-09-16T09:32:18.811 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 206.760979077, 2024-09-16T09:32:18.811 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.17988844700000001, 2024-09-16T09:32:18.811 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:32:18.811 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:32:18.811 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:32:18.811 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:32:18.811 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.24698", 2024-09-16T09:32:18.811 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:32:18.811 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 203421 2024-09-16T09:32:18.811 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.811 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:32:18.811 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.811 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:32:18.811 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.963412+0000", 2024-09-16T09:32:18.812 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:32:18.812 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.812 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.812 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:32:18.812 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.963412+0000", 2024-09-16T09:32:18.812 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:32:18.812 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.812 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.812 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:32:18.812 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.963414+0000", 2024-09-16T09:32:18.812 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.668e-06 2024-09-16T09:32:18.812 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.812 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.812 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:32:18.813 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.963420+0000", 2024-09-16T09:32:18.813 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.0510000000000002e-06 2024-09-16T09:32:18.813 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.813 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.813 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:32:18.813 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.963435+0000", 2024-09-16T09:32:18.813 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5495e-05 2024-09-16T09:32:18.813 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.813 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.813 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:32:18.813 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.963444+0000", 2024-09-16T09:32:18.813 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.1249999999999993e-06 2024-09-16T09:32:18.813 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.814 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.814 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:32:18.814 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.964127+0000", 2024-09-16T09:32:18.814 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00068378000000000004 2024-09-16T09:32:18.814 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.814 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.814 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:32:18.814 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.964240+0000", 2024-09-16T09:32:18.814 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000112333 2024-09-16T09:32:18.814 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.814 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.814 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,4", 2024-09-16T09:32:18.814 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.964346+0000", 2024-09-16T09:32:18.814 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000106087 2024-09-16T09:32:18.815 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.815 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.815 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:32:18.815 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.970630+0000", 2024-09-16T09:32:18.815 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0062838479999999999 2024-09-16T09:32:18.815 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.815 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.815 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:32:18.815 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.980509+0000", 2024-09-16T09:32:18.815 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0098798240000000006 2024-09-16T09:32:18.815 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.815 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.815 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:32:18.815 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.143232+0000", 2024-09-16T09:32:18.816 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.16272270899999999 2024-09-16T09:32:18.816 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.816 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.816 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:32:18.817 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.143271+0000", 2024-09-16T09:32:18.817 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8822000000000002e-05 2024-09-16T09:32:18.817 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.817 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.817 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:32:18.817 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.143301+0000", 2024-09-16T09:32:18.817 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.9705000000000001e-05 2024-09-16T09:32:18.817 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:32:18.817 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:32:18.817 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:32:18.817 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.817 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.817 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.24698.0:203423 7.4 7:27039506:::benchmark_data_smithi070_78438_object77292:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135)", 2024-09-16T09:32:18.818 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:28:51.963462+0000", 2024-09-16T09:32:18.818 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 206.760929151, 2024-09-16T09:32:18.818 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.17994733700000001, 2024-09-16T09:32:18.818 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:32:18.818 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:32:18.818 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:32:18.818 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:32:18.818 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.24698", 2024-09-16T09:32:18.818 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:32:18.818 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 203423 2024-09-16T09:32:18.818 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.818 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:32:18.818 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.818 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:32:18.819 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.963462+0000", 2024-09-16T09:32:18.819 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:32:18.819 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.819 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.819 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:32:18.819 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.963462+0000", 2024-09-16T09:32:18.819 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:32:18.819 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.819 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.819 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:32:18.819 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.963463+0000", 2024-09-16T09:32:18.819 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.6899999999999997e-07 2024-09-16T09:32:18.819 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.819 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.820 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:32:18.820 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.963466+0000", 2024-09-16T09:32:18.820 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4089999999999999e-06 2024-09-16T09:32:18.820 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.820 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.820 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:32:18.820 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.963473+0000", 2024-09-16T09:32:18.820 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.1280000000000002e-06 2024-09-16T09:32:18.820 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.820 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.820 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:32:18.820 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.963479+0000", 2024-09-16T09:32:18.820 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7950000000000004e-06 2024-09-16T09:32:18.821 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.821 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.821 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:32:18.821 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.967265+0000", 2024-09-16T09:32:18.821 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.003785928 2024-09-16T09:32:18.821 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.821 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.821 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:32:18.821 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.967393+0000", 2024-09-16T09:32:18.821 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00012783099999999999 2024-09-16T09:32:18.821 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.821 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.821 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,4", 2024-09-16T09:32:18.822 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.967512+0000", 2024-09-16T09:32:18.822 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000119065 2024-09-16T09:32:18.822 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.822 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.822 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:32:18.822 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.970725+0000", 2024-09-16T09:32:18.822 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0032128130000000001 2024-09-16T09:32:18.822 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.822 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.822 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:32:18.822 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.980551+0000", 2024-09-16T09:32:18.822 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0098265179999999994 2024-09-16T09:32:18.822 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.822 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.823 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:32:18.823 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.143332+0000", 2024-09-16T09:32:18.823 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.162780654 2024-09-16T09:32:18.823 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.823 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.823 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:32:18.823 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.143370+0000", 2024-09-16T09:32:18.823 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7917999999999998e-05 2024-09-16T09:32:18.823 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.823 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.823 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:32:18.823 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.143409+0000", 2024-09-16T09:32:18.823 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9609e-05 2024-09-16T09:32:18.823 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:32:18.824 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:32:18.824 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:32:18.824 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.824 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.824 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.24698.0:203426 7.6 7:66bd1b62:::benchmark_data_smithi070_78438_object77295:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135)", 2024-09-16T09:32:18.824 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:28:51.963487+0000", 2024-09-16T09:32:18.824 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 206.76090424399999, 2024-09-16T09:32:18.824 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.159441261, 2024-09-16T09:32:18.824 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:32:18.824 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:32:18.824 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:32:18.824 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:32:18.824 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.24698", 2024-09-16T09:32:18.825 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:32:18.825 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 203426 2024-09-16T09:32:18.825 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.825 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:32:18.825 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.825 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:32:18.825 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.963487+0000", 2024-09-16T09:32:18.825 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:32:18.825 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.825 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.825 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:32:18.825 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.963487+0000", 2024-09-16T09:32:18.825 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:32:18.825 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.825 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.826 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:32:18.826 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.963487+0000", 2024-09-16T09:32:18.826 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2899999999999999e-07 2024-09-16T09:32:18.826 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.826 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.826 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:32:18.826 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.963490+0000", 2024-09-16T09:32:18.826 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.2620000000000001e-06 2024-09-16T09:32:18.826 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.826 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.826 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:32:18.826 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.963495+0000", 2024-09-16T09:32:18.826 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.2240000000000003e-06 2024-09-16T09:32:18.826 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.827 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.827 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:32:18.827 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.963499+0000", 2024-09-16T09:32:18.827 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2629999999999997e-06 2024-09-16T09:32:18.827 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.827 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.827 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:32:18.827 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.963804+0000", 2024-09-16T09:32:18.827 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00030513399999999998 2024-09-16T09:32:18.827 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.827 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.827 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:32:18.827 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.968002+0000", 2024-09-16T09:32:18.827 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0041977179999999996 2024-09-16T09:32:18.827 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.828 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.828 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 3,7", 2024-09-16T09:32:18.828 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.968109+0000", 2024-09-16T09:32:18.828 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000106727 2024-09-16T09:32:18.828 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.828 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.828 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:32:18.828 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.987985+0000", 2024-09-16T09:32:18.828 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.019875995 2024-09-16T09:32:18.828 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.828 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.828 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:32:18.828 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.989233+0000", 2024-09-16T09:32:18.828 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001247883 2024-09-16T09:32:18.829 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.829 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.829 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:32:18.829 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.122873+0000", 2024-09-16T09:32:18.829 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.133640222 2024-09-16T09:32:18.829 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.829 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.829 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:32:18.829 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.122906+0000", 2024-09-16T09:32:18.829 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3269000000000003e-05 2024-09-16T09:32:18.829 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.829 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.829 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:32:18.829 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.122928+0000", 2024-09-16T09:32:18.829 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.2135000000000001e-05 2024-09-16T09:32:18.830 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:32:18.830 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:32:18.830 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:32:18.830 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.830 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.830 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.24698.0:203428 7.f 7:fc13f757:::benchmark_data_smithi070_78438_object77297:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135)", 2024-09-16T09:32:18.830 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:28:51.969082+0000", 2024-09-16T09:32:18.830 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 206.75530925499999, 2024-09-16T09:32:18.830 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.15646602200000001, 2024-09-16T09:32:18.830 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:32:18.830 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:32:18.830 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:32:18.830 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:32:18.830 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.24698", 2024-09-16T09:32:18.831 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:32:18.831 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 203428 2024-09-16T09:32:18.831 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.831 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:32:18.831 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.831 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:32:18.831 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.969082+0000", 2024-09-16T09:32:18.831 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:32:18.831 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.831 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.831 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:32:18.831 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.969082+0000", 2024-09-16T09:32:18.831 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:32:18.831 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.832 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.832 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:32:18.832 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.969083+0000", 2024-09-16T09:32:18.832 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3009999999999999e-06 2024-09-16T09:32:18.832 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.832 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.832 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:32:18.832 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.969089+0000", 2024-09-16T09:32:18.832 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.2000000000000002e-06 2024-09-16T09:32:18.832 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.832 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.832 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:32:18.832 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.969099+0000", 2024-09-16T09:32:18.832 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0703000000000001e-05 2024-09-16T09:32:18.833 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.833 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.833 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:32:18.833 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.969106+0000", 2024-09-16T09:32:18.833 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.9299999999999997e-06 2024-09-16T09:32:18.833 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.833 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.833 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:32:18.833 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.969432+0000", 2024-09-16T09:32:18.833 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00032618600000000001 2024-09-16T09:32:18.833 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.833 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.833 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:32:18.833 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.969546+0000", 2024-09-16T09:32:18.834 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000113579 2024-09-16T09:32:18.834 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.834 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.834 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 3,7", 2024-09-16T09:32:18.834 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.969650+0000", 2024-09-16T09:32:18.834 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000103798 2024-09-16T09:32:18.834 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.834 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.834 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:32:18.834 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.986851+0000", 2024-09-16T09:32:18.834 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.017201569999999999 2024-09-16T09:32:18.834 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.834 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.834 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:32:18.835 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.987148+0000", 2024-09-16T09:32:18.835 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00029701899999999998 2024-09-16T09:32:18.835 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.835 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.835 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:32:18.835 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.125433+0000", 2024-09-16T09:32:18.835 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.13828417900000001 2024-09-16T09:32:18.835 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.835 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.835 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:32:18.835 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.125497+0000", 2024-09-16T09:32:18.835 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.4459000000000001e-05 2024-09-16T09:32:18.835 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.835 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.836 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:32:18.836 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.125548+0000", 2024-09-16T09:32:18.836 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.1097999999999998e-05 2024-09-16T09:32:18.836 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:32:18.836 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:32:18.836 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:32:18.836 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.836 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.836 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.24698.0:203431 7.f 7:f4be5ae6:::benchmark_data_smithi070_78438_object77300:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135)", 2024-09-16T09:32:18.836 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:28:51.971668+0000", 2024-09-16T09:32:18.836 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 206.752723011, 2024-09-16T09:32:18.836 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.15400379, 2024-09-16T09:32:18.836 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:32:18.836 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:32:18.837 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:32:18.837 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:32:18.837 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.24698", 2024-09-16T09:32:18.837 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:32:18.837 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 203431 2024-09-16T09:32:18.837 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.837 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:32:18.837 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.837 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:32:18.837 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.971668+0000", 2024-09-16T09:32:18.837 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:32:18.837 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.837 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.837 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:32:18.838 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.971668+0000", 2024-09-16T09:32:18.838 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:32:18.838 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.838 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.838 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:32:18.838 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.971670+0000", 2024-09-16T09:32:18.838 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5030000000000001e-06 2024-09-16T09:32:18.838 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.838 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.838 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:32:18.838 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.971674+0000", 2024-09-16T09:32:18.838 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.6280000000000004e-06 2024-09-16T09:32:18.838 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.838 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.839 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:32:18.839 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.971686+0000", 2024-09-16T09:32:18.839 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1171e-05 2024-09-16T09:32:18.839 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.839 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.839 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:32:18.839 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.971693+0000", 2024-09-16T09:32:18.839 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.7910000000000005e-06 2024-09-16T09:32:18.839 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.839 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.839 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:32:18.839 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.981076+0000", 2024-09-16T09:32:18.839 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0093824809999999998 2024-09-16T09:32:18.840 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.840 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.840 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:32:18.840 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.981186+0000", 2024-09-16T09:32:18.840 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00011014 2024-09-16T09:32:18.840 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.840 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.840 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 3,7", 2024-09-16T09:32:18.840 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.981301+0000", 2024-09-16T09:32:18.840 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000114521 2024-09-16T09:32:18.840 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.840 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.840 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:32:18.841 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.986856+0000", 2024-09-16T09:32:18.841 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0055554200000000001 2024-09-16T09:32:18.841 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.841 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.841 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:32:18.841 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.987168+0000", 2024-09-16T09:32:18.841 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00031244999999999998 2024-09-16T09:32:18.841 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.841 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.841 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:32:18.841 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.125596+0000", 2024-09-16T09:32:18.841 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.13842763 2024-09-16T09:32:18.841 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.841 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.841 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:32:18.842 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.125646+0000", 2024-09-16T09:32:18.842 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.9725999999999998e-05 2024-09-16T09:32:18.842 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.842 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.842 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:32:18.842 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.125672+0000", 2024-09-16T09:32:18.842 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.6329000000000002e-05 2024-09-16T09:32:18.842 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:32:18.842 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:32:18.842 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:32:18.842 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.842 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.842 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.24698.0:203436 7.4 7:2eae91c1:::benchmark_data_smithi070_78438_object77305:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136)", 2024-09-16T09:32:18.843 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:28:51.987647+0000", 2024-09-16T09:32:18.843 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 206.73674442199999, 2024-09-16T09:32:18.843 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.16453166499999999, 2024-09-16T09:32:18.843 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:32:18.843 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:32:18.843 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:32:18.843 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:32:18.843 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.24698", 2024-09-16T09:32:18.843 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:32:18.843 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 203436 2024-09-16T09:32:18.843 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.843 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:32:18.843 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.843 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:32:18.844 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.987647+0000", 2024-09-16T09:32:18.844 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:32:18.844 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.844 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.844 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:32:18.844 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.987647+0000", 2024-09-16T09:32:18.844 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:32:18.844 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.844 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.844 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:32:18.844 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.987648+0000", 2024-09-16T09:32:18.844 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.395e-06 2024-09-16T09:32:18.844 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.845 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.845 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:32:18.845 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.987653+0000", 2024-09-16T09:32:18.845 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.9989999999999999e-06 2024-09-16T09:32:18.845 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.845 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.845 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:32:18.845 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.987665+0000", 2024-09-16T09:32:18.845 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2075e-05 2024-09-16T09:32:18.845 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.845 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.845 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:32:18.845 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.987673+0000", 2024-09-16T09:32:18.845 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.9200000000000004e-06 2024-09-16T09:32:18.846 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.846 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.846 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:32:18.846 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.987721+0000", 2024-09-16T09:32:18.846 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.7926999999999997e-05 2024-09-16T09:32:18.846 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.846 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.846 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:32:18.846 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.987841+0000", 2024-09-16T09:32:18.846 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000120093 2024-09-16T09:32:18.846 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.846 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.846 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,4", 2024-09-16T09:32:18.846 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.987961+0000", 2024-09-16T09:32:18.847 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000119492 2024-09-16T09:32:18.847 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.847 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.847 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:32:18.847 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.992755+0000", 2024-09-16T09:32:18.847 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.004793917 2024-09-16T09:32:18.847 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.847 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.847 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:32:18.847 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.992788+0000", 2024-09-16T09:32:18.847 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3201999999999998e-05 2024-09-16T09:32:18.847 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.847 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.848 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:32:18.848 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.152086+0000", 2024-09-16T09:32:18.848 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.15929815 2024-09-16T09:32:18.848 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.848 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.848 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:32:18.848 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.152139+0000", 2024-09-16T09:32:18.848 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.308e-05 2024-09-16T09:32:18.848 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.848 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.848 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:32:18.848 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.152179+0000", 2024-09-16T09:32:18.848 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9415000000000001e-05 2024-09-16T09:32:18.849 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:32:18.849 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:32:18.849 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:32:18.849 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.849 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.849 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.24698.0:203438 7.4 7:26bbec67:::benchmark_data_smithi070_78438_object77307:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136)", 2024-09-16T09:32:18.849 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:28:51.988341+0000", 2024-09-16T09:32:18.849 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 206.736050092, 2024-09-16T09:32:18.849 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.16393644399999999, 2024-09-16T09:32:18.849 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:32:18.849 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:32:18.849 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:32:18.849 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:32:18.849 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.24698", 2024-09-16T09:32:18.850 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:32:18.850 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 203438 2024-09-16T09:32:18.850 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.850 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:32:18.850 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.850 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:32:18.850 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.988341+0000", 2024-09-16T09:32:18.850 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:32:18.850 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.850 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.850 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:32:18.850 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.988341+0000", 2024-09-16T09:32:18.850 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:32:18.851 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.851 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.851 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:32:18.851 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.988342+0000", 2024-09-16T09:32:18.851 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.7300000000000005e-07 2024-09-16T09:32:18.851 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.851 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.851 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:32:18.851 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.988346+0000", 2024-09-16T09:32:18.851 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.3290000000000004e-06 2024-09-16T09:32:18.851 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.851 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.851 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:32:18.851 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.988356+0000", 2024-09-16T09:32:18.852 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.6269999999999995e-06 2024-09-16T09:32:18.852 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.852 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.852 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:32:18.852 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.988363+0000", 2024-09-16T09:32:18.852 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.5729999999999996e-06 2024-09-16T09:32:18.852 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.852 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.852 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:32:18.852 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.991029+0000", 2024-09-16T09:32:18.852 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0026668719999999998 2024-09-16T09:32:18.852 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.852 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.853 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:32:18.853 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.991106+0000", 2024-09-16T09:32:18.853 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6378999999999995e-05 2024-09-16T09:32:18.853 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.853 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.853 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,4", 2024-09-16T09:32:18.853 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.991175+0000", 2024-09-16T09:32:18.853 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.9617999999999997e-05 2024-09-16T09:32:18.853 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.853 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.853 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:32:18.853 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.992781+0000", 2024-09-16T09:32:18.853 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001605504 2024-09-16T09:32:18.853 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.854 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.854 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:32:18.854 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.992811+0000", 2024-09-16T09:32:18.854 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.0108999999999999e-05 2024-09-16T09:32:18.854 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.854 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.854 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:32:18.854 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.152211+0000", 2024-09-16T09:32:18.854 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.159399813 2024-09-16T09:32:18.854 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.854 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.854 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:32:18.854 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.152249+0000", 2024-09-16T09:32:18.854 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7799000000000002e-05 2024-09-16T09:32:18.855 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:18.855 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:18.855 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:32:18.855 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.152278+0000", 2024-09-16T09:32:18.855 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.8948000000000001e-05 2024-09-16T09:32:18.855 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:32:18.855 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:32:18.855 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:32:18.855 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:32:18.855 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:32:18.855 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T09:32:18.856 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:18 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:32:18.856 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:18 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:32:18.856 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:18 smithi070 ceph-mon[30388]: pgmap v340: 168 pgs: 2 active+clean+scrubbing, 2 active+recovering+undersized+remapped, 164 active+clean; 4.4 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 774 op/s; 3307/217419 objects misplaced (1.521%); 4.1 MiB/s, 65 objects/s recovering 2024-09-16T09:32:18.856 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:18 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:32:18.856 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:18 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:32:18.856 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:18 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2204845719' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:32:18.856 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:18 smithi070 ceph-mon[30388]: osdmap e208: 8 total, 8 up, 8 in 2024-09-16T09:32:18.856 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:18 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:32:18.856 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:18 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:32:18.857 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:18 smithi070 ceph-mon[40691]: pgmap v340: 168 pgs: 2 active+clean+scrubbing, 2 active+recovering+undersized+remapped, 164 active+clean; 4.4 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 774 op/s; 3307/217419 objects misplaced (1.521%); 4.1 MiB/s, 65 objects/s recovering 2024-09-16T09:32:18.857 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:18 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:32:18.857 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:18 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:32:18.857 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:18 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/2204845719' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:32:18.857 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:18 smithi070 ceph-mon[40691]: osdmap e208: 8 total, 8 up, 8 in 2024-09-16T09:32:18.874 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:32:18.915 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:18.915 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:18.916 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:32:19.016 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:32:19.050 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:19.050 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:19.051 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:32:19.151 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:32:19.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:18 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:32:19.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:18 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:32:19.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:18 smithi084 ceph-mon[32629]: pgmap v340: 168 pgs: 2 active+clean+scrubbing, 2 active+recovering+undersized+remapped, 164 active+clean; 4.4 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 774 op/s; 3307/217419 objects misplaced (1.521%); 4.1 MiB/s, 65 objects/s recovering 2024-09-16T09:32:19.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:18 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:32:19.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:18 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:32:19.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:18 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/2204845719' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:32:19.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:18 smithi084 ceph-mon[32629]: osdmap e208: 8 total, 8 up, 8 in 2024-09-16T09:32:19.194 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:19.195 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:19.195 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:32:19.296 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:32:19.326 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:32:19.329 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2024-09-16T09:32:19.341 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:19.341 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:19.341 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:32:19.442 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:32:19.479 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:19.479 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:19.480 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:32:19.580 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:32:19.619 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:19.620 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:19.620 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:32:19.721 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:32:19.751 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:19.751 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:19.752 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:32:19.852 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:32:19.887 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:19.888 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:19.888 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:32:19.989 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:32:20.022 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:20.023 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:20.023 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:32:20.124 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:32:20.154 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:20.154 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:20.155 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:32:20.255 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:32:20.291 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:20.292 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:20.292 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:32:20.393 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:32:20.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:20 smithi070 ceph-mon[30388]: from='client.15432 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:32:20.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:20 smithi070 ceph-mon[40691]: from='client.15432 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:32:20.422 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:20.422 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:20.423 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:32:20.524 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:32:20.561 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:20.561 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:20.562 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:32:20.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:20 smithi084 ceph-mon[32629]: from='client.15432 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:32:20.662 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:32:20.698 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:20.698 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:20.699 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:32:20.799 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:32:20.833 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:20.834 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:20.834 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:32:20.935 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:32:20.970 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:20.970 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:20.971 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:32:21.071 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:32:21.104 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:21.105 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:21.105 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:32:21.206 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:32:21.239 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:21.239 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:21.240 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:32:21.341 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:32:21.374 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:21.375 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:21.375 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:32:21.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:21 smithi070 ceph-mon[30388]: pgmap v342: 168 pgs: 2 active+recovering+undersized+remapped, 166 active+clean; 4.1 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 826 B/s rd, 0 B/s wr, 957 op/s; 2923/201885 objects misplaced (1.448%); 3.9 MiB/s, 61 objects/s recovering 2024-09-16T09:32:21.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:21 smithi070 ceph-mon[40691]: pgmap v342: 168 pgs: 2 active+recovering+undersized+remapped, 166 active+clean; 4.1 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 826 B/s rd, 0 B/s wr, 957 op/s; 2923/201885 objects misplaced (1.448%); 3.9 MiB/s, 61 objects/s recovering 2024-09-16T09:32:21.476 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:32:21.508 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:21.508 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:21.509 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:32:21.544 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:21 smithi084 ceph-mon[32629]: pgmap v342: 168 pgs: 2 active+recovering+undersized+remapped, 166 active+clean; 4.1 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 826 B/s rd, 0 B/s wr, 957 op/s; 2923/201885 objects misplaced (1.448%); 3.9 MiB/s, 61 objects/s recovering 2024-09-16T09:32:21.609 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:32:21.629 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.1 dump_ops_in_flight 2024-09-16T09:32:21.642 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:21.643 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:21.643 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:32:21.744 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:32:21.797 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:21.797 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:21.798 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:32:21.899 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:32:21.929 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:21.930 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:21.930 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:32:22.031 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:32:22.071 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:22.071 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:22.071 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:32:22.172 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:32:22.197 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T09:32:22.208 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:22.209 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:22.209 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:32:22.238 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:32:21 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:32:21] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:32:22.310 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:32:22.354 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:22.354 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:22.355 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:32:22.456 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:32:22.488 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:22.489 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:22.489 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:32:22.590 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:32:22.624 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:22.624 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:22.625 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:32:22.725 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:32:22.763 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:22.764 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:22.764 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:32:22.865 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:32:22.904 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:22.905 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:22.905 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:32:23.006 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:32:23.040 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":341,"stamp":"2024-09-16T09:32:18.483261+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4397885482,"num_objects":67295,"num_object_clones":0,"num_object_copies":201885,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2923,"num_objects_unfound":0,"num_objects_dirty":67295,"num_whiteouts":0,"num_read":1167,"num_read_kb":938,"num_write":182548,"num_write_kb":7974230,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11924,"num_bytes_recovered":773372704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79918,"ondisk_log_size":79918,"up":504,"acting":502,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":506,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":15485144,"kb_used_data":13423816,"kb_used_omap":12,"kb_used_meta":2061043,"kb_avail":734476072,"statfs":{"total":767960285184,"available":752103497728,"internally_reserved":0,"allocated":13745987584,"data_stored":13743574769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":2110508594},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-745013248,"num_objects":-11368,"num_object_clones":0,"num_object_copies":-34104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-906,"num_objects_unfound":0,"num_objects_dirty":-11368,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":11368,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":906,"num_bytes_recovered":59375616,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.853769"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":335,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.496520+0000","last_change":"2024-09-16T09:30:03.848439+0000","last_active":"2024-09-16T09:32:16.496520+0000","last_peered":"2024-09-16T09:32:16.496520+0000","last_clean":"2024-09-16T09:32:16.496520+0000","last_became_active":"2024-09-16T09:25:01.709108+0000","last_became_peered":"2024-09-16T09:25:01.709108+0000","last_unstale":"2024-09-16T09:32:16.496520+0000","last_undegraded":"2024-09-16T09:32:16.496520+0000","last_fullsized":"2024-09-16T09:32:16.496520+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:30:03.848354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:30:03.848354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"66'9","reported_seq":339,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.487505+0000","last_change":"2024-09-16T09:28:52.689373+0000","last_active":"2024-09-16T09:32:16.487505+0000","last_peered":"2024-09-16T09:32:16.487505+0000","last_clean":"2024-09-16T09:32:16.487505+0000","last_became_active":"2024-09-16T09:25:03.714635+0000","last_became_peered":"2024-09-16T09:25:03.714635+0000","last_unstale":"2024-09-16T09:32:16.487505+0000","last_undegraded":"2024-09-16T09:32:16.487505+0000","last_fullsized":"2024-09-16T09:32:16.487505+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:28:52.689304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:52.689304+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":326,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.498322+0000","last_change":"2024-09-16T09:29:17.910367+0000","last_active":"2024-09-16T09:32:16.498322+0000","last_peered":"2024-09-16T09:32:16.498322+0000","last_clean":"2024-09-16T09:32:16.498322+0000","last_became_active":"2024-09-16T09:25:05.715645+0000","last_became_peered":"2024-09-16T09:25:05.715645+0000","last_unstale":"2024-09-16T09:32:16.498322+0000","last_undegraded":"2024-09-16T09:32:16.498322+0000","last_fullsized":"2024-09-16T09:32:16.498322+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:17.910305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:17.910305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":306,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.488122+0000","last_change":"2024-09-16T09:27:11.565306+0000","last_active":"2024-09-16T09:32:16.488122+0000","last_peered":"2024-09-16T09:32:16.488122+0000","last_clean":"2024-09-16T09:32:16.488122+0000","last_became_active":"2024-09-16T09:25:07.715442+0000","last_became_peered":"2024-09-16T09:25:07.715442+0000","last_unstale":"2024-09-16T09:32:16.488122+0000","last_undegraded":"2024-09-16T09:32:16.488122+0000","last_fullsized":"2024-09-16T09:32:16.488122+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:11.565220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:11.565220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":52,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.494044+0000","last_change":"2024-09-16T09:31:09.269700+0000","last_active":"2024-09-16T09:32:16.494044+0000","last_peered":"2024-09-16T09:32:16.494044+0000","last_clean":"2024-09-16T09:32:16.494044+0000","last_became_active":"2024-09-16T09:31:09.269342+0000","last_became_peered":"2024-09-16T09:31:09.269342+0000","last_unstale":"2024-09-16T09:32:16.494044+0000","last_undegraded":"2024-09-16T09:32:16.494044+0000","last_fullsized":"2024-09-16T09:32:16.494044+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:00.068570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:28:00.068570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":192,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.492613+0000","last_change":"2024-09-16T09:31:56.507595+0000","last_active":"2024-09-16T09:32:16.492613+0000","last_peered":"2024-09-16T09:32:16.492613+0000","last_clean":"2024-09-16T09:32:16.492613+0000","last_became_active":"2024-09-16T09:31:09.275136+0000","last_became_peered":"2024-09-16T09:31:09.275136+0000","last_unstale":"2024-09-16T09:32:16.492613+0000","last_undegraded":"2024-09-16T09:32:16.492613+0000","last_fullsized":"2024-09-16T09:32:16.492613+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:31:56.507528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:31:56.507528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:08.491873+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.19","version":"66'15","reported_seq":253,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.502704+0000","last_change":"2024-09-16T09:31:09.302903+0000","last_active":"2024-09-16T09:32:16.502704+0000","last_peered":"2024-09-16T09:32:16.502704+0000","last_clean":"2024-09-16T09:32:16.502704+0000","last_became_active":"2024-09-16T09:31:09.268750+0000","last_became_peered":"2024-09-16T09:31:09.268750+0000","last_unstale":"2024-09-16T09:32:16.502704+0000","last_undegraded":"2024-09-16T09:32:16.502704+0000","last_fullsized":"2024-09-16T09:32:16.502704+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-16T09:29:57.968162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:29:57.968162+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":326,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.487379+0000","last_change":"2024-09-16T09:29:15.628676+0000","last_active":"2024-09-16T09:32:16.487379+0000","last_peered":"2024-09-16T09:32:16.487379+0000","last_clean":"2024-09-16T09:32:16.487379+0000","last_became_active":"2024-09-16T09:25:05.717236+0000","last_became_peered":"2024-09-16T09:25:05.717236+0000","last_unstale":"2024-09-16T09:32:16.487379+0000","last_undegraded":"2024-09-16T09:32:16.487379+0000","last_fullsized":"2024-09-16T09:32:16.487379+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:15.628467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:15.628467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":165,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.498911+0000","last_change":"2024-09-16T09:28:34.839909+0000","last_active":"2024-09-16T09:32:16.498911+0000","last_peered":"2024-09-16T09:32:16.498911+0000","last_clean":"2024-09-16T09:32:16.498911+0000","last_became_active":"2024-09-16T09:28:34.839435+0000","last_became_peered":"2024-09-16T09:28:34.839435+0000","last_unstale":"2024-09-16T09:32:16.498911+0000","last_undegraded":"2024-09-16T09:32:16.498911+0000","last_fullsized":"2024-09-16T09:32:16.498911+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:01.038400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:28:01.038400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"66'9","reported_seq":338,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.488357+0000","last_change":"2024-09-16T09:29:28.622647+0000","last_active":"2024-09-16T09:32:16.488357+0000","last_peered":"2024-09-16T09:32:16.488357+0000","last_clean":"2024-09-16T09:32:16.488357+0000","last_became_active":"2024-09-16T09:25:03.709147+0000","last_became_peered":"2024-09-16T09:25:03.709147+0000","last_unstale":"2024-09-16T09:32:16.488357+0000","last_undegraded":"2024-09-16T09:32:16.488357+0000","last_fullsized":"2024-09-16T09:32:16.488357+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:29:28.622558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:29:28.622558+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":309,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.491017+0000","last_change":"2024-09-16T09:27:59.080369+0000","last_active":"2024-09-16T09:32:16.491017+0000","last_peered":"2024-09-16T09:32:16.491017+0000","last_clean":"2024-09-16T09:32:16.491017+0000","last_became_active":"2024-09-16T09:25:05.718134+0000","last_became_peered":"2024-09-16T09:25:05.718134+0000","last_unstale":"2024-09-16T09:32:16.491017+0000","last_undegraded":"2024-09-16T09:32:16.491017+0000","last_fullsized":"2024-09-16T09:32:16.491017+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:59.080307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:59.080307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"66'1","reported_seq":323,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.495501+0000","last_change":"2024-09-16T09:31:58.367903+0000","last_active":"2024-09-16T09:32:16.495501+0000","last_peered":"2024-09-16T09:32:16.495501+0000","last_clean":"2024-09-16T09:32:16.495501+0000","last_became_active":"2024-09-16T09:25:07.716558+0000","last_became_peered":"2024-09-16T09:25:07.716558+0000","last_unstale":"2024-09-16T09:32:16.495501+0000","last_undegraded":"2024-09-16T09:32:16.495501+0000","last_fullsized":"2024-09-16T09:32:16.495501+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-16T09:31:58.367837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:31:58.367837+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:20.121824+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":317,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.491868+0000","last_change":"2024-09-16T09:27:56.191264+0000","last_active":"2024-09-16T09:32:16.491868+0000","last_peered":"2024-09-16T09:32:16.491868+0000","last_clean":"2024-09-16T09:32:16.491868+0000","last_became_active":"2024-09-16T09:25:01.696939+0000","last_became_peered":"2024-09-16T09:25:01.696939+0000","last_unstale":"2024-09-16T09:32:16.491868+0000","last_undegraded":"2024-09-16T09:32:16.491868+0000","last_fullsized":"2024-09-16T09:32:16.491868+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:56.191220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:27:56.191220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"66'5","reported_seq":324,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.487951+0000","last_change":"2024-09-16T09:27:05.694733+0000","last_active":"2024-09-16T09:32:16.487951+0000","last_peered":"2024-09-16T09:32:16.487951+0000","last_clean":"2024-09-16T09:32:16.487951+0000","last_became_active":"2024-09-16T09:25:03.716136+0000","last_became_peered":"2024-09-16T09:25:03.716136+0000","last_unstale":"2024-09-16T09:32:16.487951+0000","last_undegraded":"2024-09-16T09:32:16.487951+0000","last_fullsized":"2024-09-16T09:32:16.487951+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-16T09:27:05.694650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:05.694650+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":317,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.497209+0000","last_change":"2024-09-16T09:28:52.746046+0000","last_active":"2024-09-16T09:32:16.497209+0000","last_peered":"2024-09-16T09:32:16.497209+0000","last_clean":"2024-09-16T09:32:16.497209+0000","last_became_active":"2024-09-16T09:25:05.718281+0000","last_became_peered":"2024-09-16T09:25:05.718281+0000","last_unstale":"2024-09-16T09:32:16.497209+0000","last_undegraded":"2024-09-16T09:32:16.497209+0000","last_fullsized":"2024-09-16T09:32:16.497209+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:52.745999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:28:52.745999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":335,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.490087+0000","last_change":"2024-09-16T09:31:09.276016+0000","last_active":"2024-09-16T09:32:16.490087+0000","last_peered":"2024-09-16T09:32:16.490087+0000","last_clean":"2024-09-16T09:32:16.490087+0000","last_became_active":"2024-09-16T09:31:09.275656+0000","last_became_peered":"2024-09-16T09:31:09.275656+0000","last_unstale":"2024-09-16T09:32:16.490087+0000","last_undegraded":"2024-09-16T09:32:16.490087+0000","last_fullsized":"2024-09-16T09:32:16.490087+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:50.468134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:50.468134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":323,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.487794+0000","last_change":"2024-09-16T09:29:56.605555+0000","last_active":"2024-09-16T09:32:16.487794+0000","last_peered":"2024-09-16T09:32:16.487794+0000","last_clean":"2024-09-16T09:32:16.487794+0000","last_became_active":"2024-09-16T09:25:07.720793+0000","last_became_peered":"2024-09-16T09:25:07.720793+0000","last_unstale":"2024-09-16T09:32:16.487794+0000","last_undegraded":"2024-09-16T09:32:16.487794+0000","last_fullsized":"2024-09-16T09:32:16.487794+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:56.605488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:56.605488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":74,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.491945+0000","last_change":"2024-09-16T09:30:39.202178+0000","last_active":"2024-09-16T09:32:16.491945+0000","last_peered":"2024-09-16T09:32:16.491945+0000","last_clean":"2024-09-16T09:32:16.491945+0000","last_became_active":"2024-09-16T09:30:39.201824+0000","last_became_peered":"2024-09-16T09:30:39.201824+0000","last_unstale":"2024-09-16T09:32:16.491945+0000","last_undegraded":"2024-09-16T09:32:16.491945+0000","last_fullsized":"2024-09-16T09:32:16.491945+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:50.912979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:50.912979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:20.431823+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"66'15","reported_seq":401,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.503721+0000","last_change":"2024-09-16T09:31:09.302018+0000","last_active":"2024-09-16T09:32:16.503721+0000","last_peered":"2024-09-16T09:32:16.503721+0000","last_clean":"2024-09-16T09:32:16.503721+0000","last_became_active":"2024-09-16T09:31:09.263534+0000","last_became_peered":"2024-09-16T09:31:09.263534+0000","last_unstale":"2024-09-16T09:32:16.503721+0000","last_undegraded":"2024-09-16T09:32:16.503721+0000","last_fullsized":"2024-09-16T09:32:16.503721+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-16T09:30:21.932604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:30:21.932604+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":318,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.499188+0000","last_change":"2024-09-16T09:29:55.875536+0000","last_active":"2024-09-16T09:32:16.499188+0000","last_peered":"2024-09-16T09:32:16.499188+0000","last_clean":"2024-09-16T09:32:16.499188+0000","last_became_active":"2024-09-16T09:25:05.713929+0000","last_became_peered":"2024-09-16T09:25:05.713929+0000","last_unstale":"2024-09-16T09:32:16.499188+0000","last_undegraded":"2024-09-16T09:32:16.499188+0000","last_fullsized":"2024-09-16T09:32:16.499188+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:55.875480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:55.875480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":171,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.492585+0000","last_change":"2024-09-16T09:30:24.412733+0000","last_active":"2024-09-16T09:32:16.492585+0000","last_peered":"2024-09-16T09:32:16.492585+0000","last_clean":"2024-09-16T09:32:16.492585+0000","last_became_active":"2024-09-16T09:28:52.974255+0000","last_became_peered":"2024-09-16T09:28:52.974255+0000","last_unstale":"2024-09-16T09:32:16.492585+0000","last_undegraded":"2024-09-16T09:32:16.492585+0000","last_fullsized":"2024-09-16T09:32:16.492585+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:30:24.412571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:30:24.412571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:49.424390+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":334,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.488444+0000","last_change":"2024-09-16T09:30:03.628882+0000","last_active":"2024-09-16T09:32:16.488444+0000","last_peered":"2024-09-16T09:32:16.488444+0000","last_clean":"2024-09-16T09:32:16.488444+0000","last_became_active":"2024-09-16T09:25:01.706491+0000","last_became_peered":"2024-09-16T09:25:01.706491+0000","last_unstale":"2024-09-16T09:32:16.488444+0000","last_undegraded":"2024-09-16T09:32:16.488444+0000","last_fullsized":"2024-09-16T09:32:16.488444+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:30:03.628812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:30:03.628812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"66'12","reported_seq":235,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.499358+0000","last_change":"2024-09-16T09:31:09.286673+0000","last_active":"2024-09-16T09:32:16.499358+0000","last_peered":"2024-09-16T09:32:16.499358+0000","last_clean":"2024-09-16T09:32:16.499358+0000","last_became_active":"2024-09-16T09:31:09.268588+0000","last_became_peered":"2024-09-16T09:31:09.268588+0000","last_unstale":"2024-09-16T09:32:16.499358+0000","last_undegraded":"2024-09-16T09:32:16.499358+0000","last_fullsized":"2024-09-16T09:32:16.499358+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-16T09:27:01.700139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:01.700139+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":24,"num_read_kb":15,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":310,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.488555+0000","last_change":"2024-09-16T09:27:00.816500+0000","last_active":"2024-09-16T09:32:16.488555+0000","last_peered":"2024-09-16T09:32:16.488555+0000","last_clean":"2024-09-16T09:32:16.488555+0000","last_became_active":"2024-09-16T09:25:05.710093+0000","last_became_peered":"2024-09-16T09:25:05.710093+0000","last_unstale":"2024-09-16T09:32:16.488555+0000","last_undegraded":"2024-09-16T09:32:16.488555+0000","last_fullsized":"2024-09-16T09:32:16.488555+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:00.816424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:00.816424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":314,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.497422+0000","last_change":"2024-09-16T09:29:29.918433+0000","last_active":"2024-09-16T09:32:16.497422+0000","last_peered":"2024-09-16T09:32:16.497422+0000","last_clean":"2024-09-16T09:32:16.497422+0000","last_became_active":"2024-09-16T09:25:07.719380+0000","last_became_peered":"2024-09-16T09:25:07.719380+0000","last_unstale":"2024-09-16T09:32:16.497422+0000","last_undegraded":"2024-09-16T09:32:16.497422+0000","last_fullsized":"2024-09-16T09:32:16.497422+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-16T09:29:29.918368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:29.918368+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":335,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.498575+0000","last_change":"2024-09-16T09:30:02.843932+0000","last_active":"2024-09-16T09:32:16.498575+0000","last_peered":"2024-09-16T09:32:16.498575+0000","last_clean":"2024-09-16T09:32:16.498575+0000","last_became_active":"2024-09-16T09:25:01.699545+0000","last_became_peered":"2024-09-16T09:25:01.699545+0000","last_unstale":"2024-09-16T09:32:16.498575+0000","last_undegraded":"2024-09-16T09:32:16.498575+0000","last_fullsized":"2024-09-16T09:32:16.498575+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:30:02.843856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:30:02.843856+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"66'10","reported_seq":376,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.496539+0000","last_change":"2024-09-16T09:31:09.328513+0000","last_active":"2024-09-16T09:32:16.496539+0000","last_peered":"2024-09-16T09:32:16.496539+0000","last_clean":"2024-09-16T09:32:16.496539+0000","last_became_active":"2024-09-16T09:31:09.275050+0000","last_became_peered":"2024-09-16T09:31:09.275050+0000","last_unstale":"2024-09-16T09:32:16.496539+0000","last_undegraded":"2024-09-16T09:32:16.496539+0000","last_fullsized":"2024-09-16T09:32:16.496539+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-16T09:27:43.556357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:43.556357+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"66'8","reported_seq":434,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.492708+0000","last_change":"2024-09-16T09:30:31.392771+0000","last_active":"2024-09-16T09:32:16.492708+0000","last_peered":"2024-09-16T09:32:16.492708+0000","last_clean":"2024-09-16T09:32:16.492708+0000","last_became_active":"2024-09-16T09:25:05.712516+0000","last_became_peered":"2024-09-16T09:25:05.712516+0000","last_unstale":"2024-09-16T09:32:16.492708+0000","last_undegraded":"2024-09-16T09:32:16.492708+0000","last_fullsized":"2024-09-16T09:32:16.492708+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:30:31.392399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:30:31.392399+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:44.469073+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"207'11474","reported_seq":11596,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:17.908890+0000","last_change":"2024-09-16T09:29:49.449011+0000","last_active":"2024-09-16T09:32:17.908890+0000","last_peered":"2024-09-16T09:32:17.908890+0000","last_clean":"2024-09-16T09:32:17.908890+0000","last_became_active":"2024-09-16T09:29:49.448473+0000","last_became_peered":"2024-09-16T09:29:49.448473+0000","last_unstale":"2024-09-16T09:32:17.908890+0000","last_undegraded":"2024-09-16T09:32:17.908890+0000","last_fullsized":"2024-09-16T09:32:17.908890+0000","mapping_epoch":152,"log_start":"187'6900","ondisk_log_start":"187'6900","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":4574,"log_dups_size":0,"ondisk_log_size":4574,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":271581184,"num_objects":4144,"num_object_clones":0,"num_object_copies":12432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4144,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11474,"num_write_kb":499776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.a","version":"73'79","reported_seq":257,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.489998+0000","last_change":"2024-09-16T09:29:18.498887+0000","last_active":"2024-09-16T09:32:16.489998+0000","last_peered":"2024-09-16T09:32:16.489998+0000","last_clean":"2024-09-16T09:32:16.489998+0000","last_became_active":"2024-09-16T09:26:35.127352+0000","last_became_peered":"2024-09-16T09:26:35.127352+0000","last_unstale":"2024-09-16T09:32:16.489998+0000","last_undegraded":"2024-09-16T09:32:16.489998+0000","last_fullsized":"2024-09-16T09:32:16.489998+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:18.498766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:18.498766+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"66'15","reported_seq":266,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.511158+0000","last_change":"2024-09-16T09:28:28.065781+0000","last_active":"2024-09-16T09:32:16.511158+0000","last_peered":"2024-09-16T09:32:16.511158+0000","last_clean":"2024-09-16T09:32:16.511158+0000","last_became_active":"2024-09-16T09:26:33.295672+0000","last_became_peered":"2024-09-16T09:26:33.295672+0000","last_unstale":"2024-09-16T09:32:16.511158+0000","last_undegraded":"2024-09-16T09:32:16.511158+0000","last_fullsized":"2024-09-16T09:32:16.511158+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-16T09:28:28.065699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:28.065699+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":52,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.493153+0000","last_change":"2024-09-16T09:31:09.276858+0000","last_active":"2024-09-16T09:32:16.493153+0000","last_peered":"2024-09-16T09:32:16.493153+0000","last_clean":"2024-09-16T09:32:16.493153+0000","last_became_active":"2024-09-16T09:31:09.274534+0000","last_became_peered":"2024-09-16T09:31:09.274534+0000","last_unstale":"2024-09-16T09:32:16.493153+0000","last_undegraded":"2024-09-16T09:32:16.493153+0000","last_fullsized":"2024-09-16T09:32:16.493153+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:22.091537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:28:22.091537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"66'8","reported_seq":399,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.488215+0000","last_change":"2024-09-16T09:25:05.714128+0000","last_active":"2024-09-16T09:32:16.488215+0000","last_peered":"2024-09-16T09:32:16.488215+0000","last_clean":"2024-09-16T09:32:16.488215+0000","last_became_active":"2024-09-16T09:25:05.713813+0000","last_became_peered":"2024-09-16T09:25:05.713813+0000","last_unstale":"2024-09-16T09:32:16.488215+0000","last_undegraded":"2024-09-16T09:32:16.488215+0000","last_fullsized":"2024-09-16T09:32:16.488215+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":314,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.490504+0000","last_change":"2024-09-16T09:29:24.537802+0000","last_active":"2024-09-16T09:32:16.490504+0000","last_peered":"2024-09-16T09:32:16.490504+0000","last_clean":"2024-09-16T09:32:16.490504+0000","last_became_active":"2024-09-16T09:25:07.717344+0000","last_became_peered":"2024-09-16T09:25:07.717344+0000","last_unstale":"2024-09-16T09:32:16.490504+0000","last_undegraded":"2024-09-16T09:32:16.490504+0000","last_fullsized":"2024-09-16T09:32:16.490504+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:24.537731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:24.537731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.c","version":"207'11391","reported_seq":11571,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.151940+0000","last_change":"2024-09-16T09:32:07.676294+0000","last_active":"2024-09-16T09:32:18.151940+0000","last_peered":"2024-09-16T09:32:18.151940+0000","last_clean":"2024-09-16T09:32:18.151940+0000","last_became_active":"2024-09-16T09:29:50.007935+0000","last_became_peered":"2024-09-16T09:29:50.007935+0000","last_unstale":"2024-09-16T09:32:18.151940+0000","last_undegraded":"2024-09-16T09:32:18.151940+0000","last_fullsized":"2024-09-16T09:32:18.151940+0000","mapping_epoch":152,"log_start":"183'6300","ondisk_log_start":"183'6300","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"204'10667","last_scrub_stamp":"2024-09-16T09:32:07.676020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:32:07.676020+0000","objects_scrubbed":4920,"log_size":5091,"log_dups_size":0,"ondisk_log_size":5091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:18.894513+0000","scrub_duration":1433,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":272433152,"num_objects":4157,"num_object_clones":0,"num_object_copies":12471,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4157,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11391,"num_write_kb":497536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"73'79","reported_seq":289,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.488306+0000","last_change":"2024-09-16T09:32:13.446353+0000","last_active":"2024-09-16T09:32:16.488306+0000","last_peered":"2024-09-16T09:32:16.488306+0000","last_clean":"2024-09-16T09:32:16.488306+0000","last_became_active":"2024-09-16T09:30:08.763683+0000","last_became_peered":"2024-09-16T09:30:08.763683+0000","last_unstale":"2024-09-16T09:32:16.488306+0000","last_undegraded":"2024-09-16T09:32:16.488306+0000","last_fullsized":"2024-09-16T09:32:16.488306+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:58.568841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:58.568841+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:32:41.868241+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"66'18","reported_seq":264,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.498510+0000","last_change":"2024-09-16T09:31:09.407700+0000","last_active":"2024-09-16T09:32:16.498510+0000","last_peered":"2024-09-16T09:32:16.498510+0000","last_clean":"2024-09-16T09:32:16.498510+0000","last_became_active":"2024-09-16T09:31:09.270434+0000","last_became_peered":"2024-09-16T09:31:09.270434+0000","last_unstale":"2024-09-16T09:32:16.498510+0000","last_undegraded":"2024-09-16T09:32:16.498510+0000","last_fullsized":"2024-09-16T09:32:16.498510+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'18","last_scrub_stamp":"2024-09-16T09:28:54.699778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:54.699778+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":33,"num_read_kb":21,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":162,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.497777+0000","last_change":"2024-09-16T09:28:58.705169+0000","last_active":"2024-09-16T09:32:16.497777+0000","last_peered":"2024-09-16T09:32:16.497777+0000","last_clean":"2024-09-16T09:32:16.497777+0000","last_became_active":"2024-09-16T09:28:52.983543+0000","last_became_peered":"2024-09-16T09:28:52.983543+0000","last_unstale":"2024-09-16T09:32:16.497777+0000","last_undegraded":"2024-09-16T09:32:16.497777+0000","last_fullsized":"2024-09-16T09:32:16.497777+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:58.705097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:28:58.705097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":341,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.487425+0000","last_change":"2024-09-16T09:31:09.269234+0000","last_active":"2024-09-16T09:32:16.487425+0000","last_peered":"2024-09-16T09:32:16.487425+0000","last_clean":"2024-09-16T09:32:16.487425+0000","last_became_active":"2024-09-16T09:31:09.268864+0000","last_became_peered":"2024-09-16T09:31:09.268864+0000","last_unstale":"2024-09-16T09:32:16.487425+0000","last_undegraded":"2024-09-16T09:32:16.487425+0000","last_fullsized":"2024-09-16T09:32:16.487425+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:50.482240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:50.482240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":321,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.488272+0000","last_change":"2024-09-16T09:31:09.269842+0000","last_active":"2024-09-16T09:32:16.488272+0000","last_peered":"2024-09-16T09:32:16.488272+0000","last_clean":"2024-09-16T09:32:16.488272+0000","last_became_active":"2024-09-16T09:31:09.269433+0000","last_became_peered":"2024-09-16T09:31:09.269433+0000","last_unstale":"2024-09-16T09:32:16.488272+0000","last_undegraded":"2024-09-16T09:32:16.488272+0000","last_fullsized":"2024-09-16T09:32:16.488272+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:13.548608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:13.548608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.d","version":"207'11391","reported_seq":11512,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:17.945402+0000","last_change":"2024-09-16T09:29:50.010860+0000","last_active":"2024-09-16T09:32:17.945402+0000","last_peered":"2024-09-16T09:32:17.945402+0000","last_clean":"2024-09-16T09:32:17.945402+0000","last_became_active":"2024-09-16T09:29:50.010421+0000","last_became_peered":"2024-09-16T09:29:50.010421+0000","last_unstale":"2024-09-16T09:32:17.945402+0000","last_undegraded":"2024-09-16T09:32:17.945402+0000","last_fullsized":"2024-09-16T09:32:17.945402+0000","mapping_epoch":152,"log_start":"187'6700","ondisk_log_start":"187'6700","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":4691,"log_dups_size":0,"ondisk_log_size":4691,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":273547288,"num_objects":4175,"num_object_clones":0,"num_object_copies":12525,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4175,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11391,"num_write_kb":498049,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,2],"acting":[5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.4","version":"73'79","reported_seq":52,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.494841+0000","last_change":"2024-09-16T09:31:09.275499+0000","last_active":"2024-09-16T09:32:16.494841+0000","last_peered":"2024-09-16T09:32:16.494841+0000","last_clean":"2024-09-16T09:32:16.494841+0000","last_became_active":"2024-09-16T09:31:09.275139+0000","last_became_peered":"2024-09-16T09:31:09.275139+0000","last_unstale":"2024-09-16T09:32:16.494841+0000","last_undegraded":"2024-09-16T09:32:16.494841+0000","last_fullsized":"2024-09-16T09:32:16.494841+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:30:22.438378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:30:22.438378+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"66'14","reported_seq":344,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.487509+0000","last_change":"2024-09-16T09:28:32.595064+0000","last_active":"2024-09-16T09:32:16.487509+0000","last_peered":"2024-09-16T09:32:16.487509+0000","last_clean":"2024-09-16T09:32:16.487509+0000","last_became_active":"2024-09-16T09:25:03.708830+0000","last_became_peered":"2024-09-16T09:25:03.708830+0000","last_unstale":"2024-09-16T09:32:16.487509+0000","last_undegraded":"2024-09-16T09:32:16.487509+0000","last_fullsized":"2024-09-16T09:32:16.487509+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'14","last_scrub_stamp":"2024-09-16T09:28:32.594988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:32.594988+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":344,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.496769+0000","last_change":"2024-09-16T09:31:57.373606+0000","last_active":"2024-09-16T09:32:16.496769+0000","last_peered":"2024-09-16T09:32:16.496769+0000","last_clean":"2024-09-16T09:32:16.496769+0000","last_became_active":"2024-09-16T09:25:01.702992+0000","last_became_peered":"2024-09-16T09:25:01.702992+0000","last_unstale":"2024-09-16T09:32:16.496769+0000","last_undegraded":"2024-09-16T09:32:16.496769+0000","last_fullsized":"2024-09-16T09:32:16.496769+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:31:57.373544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:31:57.373544+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:24.193683+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"187'11","reported_seq":251,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.490781+0000","last_change":"2024-09-16T09:31:09.397758+0000","last_active":"2024-09-16T09:32:16.490781+0000","last_peered":"2024-09-16T09:32:16.490781+0000","last_clean":"2024-09-16T09:32:16.490781+0000","last_became_active":"2024-09-16T09:31:09.269173+0000","last_became_peered":"2024-09-16T09:31:09.269173+0000","last_unstale":"2024-09-16T09:32:16.490781+0000","last_undegraded":"2024-09-16T09:32:16.490781+0000","last_fullsized":"2024-09-16T09:32:16.490781+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'9","last_scrub_stamp":"2024-09-16T09:29:21.898533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:21.898533+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":306,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.497149+0000","last_change":"2024-09-16T09:26:40.699888+0000","last_active":"2024-09-16T09:32:16.497149+0000","last_peered":"2024-09-16T09:32:16.497149+0000","last_clean":"2024-09-16T09:32:16.497149+0000","last_became_active":"2024-09-16T09:25:07.722017+0000","last_became_peered":"2024-09-16T09:25:07.722017+0000","last_unstale":"2024-09-16T09:32:16.497149+0000","last_undegraded":"2024-09-16T09:32:16.497149+0000","last_fullsized":"2024-09-16T09:32:16.497149+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:40.699813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:40.699813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.e","version":"207'11467","reported_seq":11589,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.161732+0000","last_change":"2024-09-16T09:29:49.448081+0000","last_active":"2024-09-16T09:32:18.161732+0000","last_peered":"2024-09-16T09:32:18.161732+0000","last_clean":"2024-09-16T09:32:18.161732+0000","last_became_active":"2024-09-16T09:29:49.447767+0000","last_became_peered":"2024-09-16T09:29:49.447767+0000","last_unstale":"2024-09-16T09:32:18.161732+0000","last_undegraded":"2024-09-16T09:32:18.161732+0000","last_fullsized":"2024-09-16T09:32:18.161732+0000","mapping_epoch":152,"log_start":"183'6300","ondisk_log_start":"183'6300","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":5167,"log_dups_size":0,"ondisk_log_size":5167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":271253504,"num_objects":4139,"num_object_clones":0,"num_object_copies":12417,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4139,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11467,"num_write_kb":499392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"73'79","reported_seq":375,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.492536+0000","last_change":"2024-09-16T09:31:09.275413+0000","last_active":"2024-09-16T09:32:16.492536+0000","last_peered":"2024-09-16T09:32:16.492536+0000","last_clean":"2024-09-16T09:32:16.492536+0000","last_became_active":"2024-09-16T09:31:09.274707+0000","last_became_peered":"2024-09-16T09:31:09.274707+0000","last_unstale":"2024-09-16T09:32:16.492536+0000","last_undegraded":"2024-09-16T09:32:16.492536+0000","last_fullsized":"2024-09-16T09:32:16.492536+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:30:23.455388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:30:23.455388+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:41.421930+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"66'10","reported_seq":346,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.498667+0000","last_change":"2024-09-16T09:30:04.848306+0000","last_active":"2024-09-16T09:32:16.498667+0000","last_peered":"2024-09-16T09:32:16.498667+0000","last_clean":"2024-09-16T09:32:16.498667+0000","last_became_active":"2024-09-16T09:25:03.712985+0000","last_became_peered":"2024-09-16T09:25:03.712985+0000","last_unstale":"2024-09-16T09:32:16.498667+0000","last_undegraded":"2024-09-16T09:32:16.498667+0000","last_fullsized":"2024-09-16T09:32:16.498667+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-16T09:30:04.848234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:30:04.848234+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":332,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.490950+0000","last_change":"2024-09-16T09:31:09.272049+0000","last_active":"2024-09-16T09:32:16.490950+0000","last_peered":"2024-09-16T09:32:16.490950+0000","last_clean":"2024-09-16T09:32:16.490950+0000","last_became_active":"2024-09-16T09:31:09.270619+0000","last_became_peered":"2024-09-16T09:31:09.270619+0000","last_unstale":"2024-09-16T09:32:16.490950+0000","last_undegraded":"2024-09-16T09:32:16.490950+0000","last_fullsized":"2024-09-16T09:32:16.490950+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:25.086322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:25.086322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"66'8","reported_seq":415,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.495806+0000","last_change":"2024-09-16T09:28:57.628615+0000","last_active":"2024-09-16T09:32:16.495806+0000","last_peered":"2024-09-16T09:32:16.495806+0000","last_clean":"2024-09-16T09:32:16.495806+0000","last_became_active":"2024-09-16T09:25:05.714507+0000","last_became_peered":"2024-09-16T09:25:05.714507+0000","last_unstale":"2024-09-16T09:32:16.495806+0000","last_undegraded":"2024-09-16T09:32:16.495806+0000","last_fullsized":"2024-09-16T09:32:16.495806+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:28:57.628543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:28:57.628543+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:51.253847+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":329,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.495639+0000","last_change":"2024-09-16T09:31:09.283816+0000","last_active":"2024-09-16T09:32:16.495639+0000","last_peered":"2024-09-16T09:32:16.495639+0000","last_clean":"2024-09-16T09:32:16.495639+0000","last_became_active":"2024-09-16T09:31:09.283192+0000","last_became_peered":"2024-09-16T09:31:09.283192+0000","last_unstale":"2024-09-16T09:32:16.495639+0000","last_undegraded":"2024-09-16T09:32:16.495639+0000","last_fullsized":"2024-09-16T09:32:16.495639+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:22.563395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:28:22.563395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:22.152175+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.f","version":"207'11423","reported_seq":11545,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:17.858084+0000","last_change":"2024-09-16T09:29:49.838498+0000","last_active":"2024-09-16T09:32:17.858084+0000","last_peered":"2024-09-16T09:32:17.858084+0000","last_clean":"2024-09-16T09:32:17.858084+0000","last_became_active":"2024-09-16T09:29:49.838146+0000","last_became_peered":"2024-09-16T09:29:49.838146+0000","last_unstale":"2024-09-16T09:32:17.858084+0000","last_undegraded":"2024-09-16T09:32:17.858084+0000","last_fullsized":"2024-09-16T09:32:17.858084+0000","mapping_epoch":152,"log_start":"180'5400","ondisk_log_start":"180'5400","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":6023,"log_dups_size":0,"ondisk_log_size":6023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":283312128,"num_objects":4323,"num_object_clones":0,"num_object_copies":12969,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4323,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11423,"num_write_kb":503872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"73'79","reported_seq":356,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.492472+0000","last_change":"2024-09-16T09:31:09.274975+0000","last_active":"2024-09-16T09:32:16.492472+0000","last_peered":"2024-09-16T09:32:16.492472+0000","last_clean":"2024-09-16T09:32:16.492472+0000","last_became_active":"2024-09-16T09:31:09.274565+0000","last_became_peered":"2024-09-16T09:31:09.274565+0000","last_unstale":"2024-09-16T09:32:16.492472+0000","last_undegraded":"2024-09-16T09:32:16.492472+0000","last_fullsized":"2024-09-16T09:32:16.492472+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:30:22.415051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:30:22.415051+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:52.385618+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"66'19","reported_seq":359,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.495986+0000","last_change":"2024-09-16T09:31:35.249952+0000","last_active":"2024-09-16T09:32:16.495986+0000","last_peered":"2024-09-16T09:32:16.495986+0000","last_clean":"2024-09-16T09:32:16.495986+0000","last_became_active":"2024-09-16T09:25:03.714797+0000","last_became_peered":"2024-09-16T09:25:03.714797+0000","last_unstale":"2024-09-16T09:32:16.495986+0000","last_undegraded":"2024-09-16T09:32:16.495986+0000","last_fullsized":"2024-09-16T09:32:16.495986+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-16T09:31:35.249864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:31:35.249864+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:40.632025+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":335,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.498818+0000","last_change":"2024-09-16T09:29:16.888357+0000","last_active":"2024-09-16T09:32:16.498818+0000","last_peered":"2024-09-16T09:32:16.498818+0000","last_clean":"2024-09-16T09:32:16.498818+0000","last_became_active":"2024-09-16T09:25:01.706806+0000","last_became_peered":"2024-09-16T09:25:01.706806+0000","last_unstale":"2024-09-16T09:32:16.498818+0000","last_undegraded":"2024-09-16T09:32:16.498818+0000","last_fullsized":"2024-09-16T09:32:16.498818+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:16.888236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:16.888236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":335,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.491783+0000","last_change":"2024-09-16T09:31:55.456669+0000","last_active":"2024-09-16T09:32:16.491783+0000","last_peered":"2024-09-16T09:32:16.491783+0000","last_clean":"2024-09-16T09:32:16.491783+0000","last_became_active":"2024-09-16T09:25:05.708663+0000","last_became_peered":"2024-09-16T09:25:05.708663+0000","last_unstale":"2024-09-16T09:32:16.491783+0000","last_undegraded":"2024-09-16T09:32:16.491783+0000","last_fullsized":"2024-09-16T09:32:16.491783+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:31:55.456592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:31:55.456592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:56.006473+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":307,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.498227+0000","last_change":"2024-09-16T09:26:37.282798+0000","last_active":"2024-09-16T09:32:16.498227+0000","last_peered":"2024-09-16T09:32:16.498227+0000","last_clean":"2024-09-16T09:32:16.498227+0000","last_became_active":"2024-09-16T09:25:07.719267+0000","last_became_peered":"2024-09-16T09:25:07.719267+0000","last_unstale":"2024-09-16T09:32:16.498227+0000","last_undegraded":"2024-09-16T09:32:16.498227+0000","last_fullsized":"2024-09-16T09:32:16.498227+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:37.282735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:37.282735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.8","version":"207'11554","reported_seq":17756,"reported_epoch":207,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-16T09:32:17.937541+0000","last_change":"2024-09-16T09:31:09.438251+0000","last_active":"2024-09-16T09:32:17.937541+0000","last_peered":"2024-09-16T09:32:17.937541+0000","last_clean":"2024-09-16T09:30:20.474651+0000","last_became_active":"2024-09-16T09:31:09.409873+0000","last_became_peered":"2024-09-16T09:31:09.409873+0000","last_unstale":"2024-09-16T09:32:17.937541+0000","last_undegraded":"2024-09-16T09:32:17.937541+0000","last_fullsized":"2024-09-16T09:31:09.256524+0000","mapping_epoch":186,"log_start":"187'6800","ondisk_log_start":"187'6800","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":4754,"log_dups_size":0,"ondisk_log_size":4754,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":273154048,"num_objects":4168,"num_object_clones":0,"num_object_copies":12504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1421,"num_objects_unfound":0,"num_objects_dirty":4168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11554,"num_write_kb":503104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3035,"num_bytes_recovered":198901760,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,7],"avail_no_missing":["5","7"],"object_location_counts":[{"shards":"0,5,7","objects":1421},{"shards":"5,7","objects":2747}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1","version":"73'79","reported_seq":281,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.489941+0000","last_change":"2024-09-16T09:32:13.444142+0000","last_active":"2024-09-16T09:32:16.489941+0000","last_peered":"2024-09-16T09:32:16.489941+0000","last_clean":"2024-09-16T09:32:16.489941+0000","last_became_active":"2024-09-16T09:30:08.766350+0000","last_became_peered":"2024-09-16T09:30:08.766350+0000","last_unstale":"2024-09-16T09:32:16.489941+0000","last_undegraded":"2024-09-16T09:32:16.489941+0000","last_fullsized":"2024-09-16T09:32:16.489941+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:15.460179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:15.460179+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:32:42.562412+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"187'30","reported_seq":517,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.498729+0000","last_change":"2024-09-16T09:31:09.323281+0000","last_active":"2024-09-16T09:32:16.498729+0000","last_peered":"2024-09-16T09:32:16.498729+0000","last_clean":"2024-09-16T09:32:16.498729+0000","last_became_active":"2024-09-16T09:31:09.278839+0000","last_became_peered":"2024-09-16T09:31:09.278839+0000","last_unstale":"2024-09-16T09:32:16.498729+0000","last_undegraded":"2024-09-16T09:32:16.498729+0000","last_fullsized":"2024-09-16T09:32:16.498729+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'28","last_scrub_stamp":"2024-09-16T09:29:18.937623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:29:18.937623+0000","objects_scrubbed":10,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":327,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.490182+0000","last_change":"2024-09-16T09:29:26.637070+0000","last_active":"2024-09-16T09:32:16.490182+0000","last_peered":"2024-09-16T09:32:16.490182+0000","last_clean":"2024-09-16T09:32:16.490182+0000","last_became_active":"2024-09-16T09:25:01.703076+0000","last_became_peered":"2024-09-16T09:25:01.703076+0000","last_unstale":"2024-09-16T09:32:16.490182+0000","last_undegraded":"2024-09-16T09:32:16.490182+0000","last_fullsized":"2024-09-16T09:32:16.490182+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:26.637027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:26.637027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":336,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.490685+0000","last_change":"2024-09-16T09:29:29.486284+0000","last_active":"2024-09-16T09:32:16.490685+0000","last_peered":"2024-09-16T09:32:16.490685+0000","last_clean":"2024-09-16T09:32:16.490685+0000","last_became_active":"2024-09-16T09:25:00.696237+0000","last_became_peered":"2024-09-16T09:25:00.696237+0000","last_unstale":"2024-09-16T09:32:16.490685+0000","last_undegraded":"2024-09-16T09:32:16.490685+0000","last_fullsized":"2024-09-16T09:32:16.490685+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-16T09:29:29.486188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:29:29.486188+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.0025137430000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":303,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.496452+0000","last_change":"2024-09-16T09:25:05.718319+0000","last_active":"2024-09-16T09:32:16.496452+0000","last_peered":"2024-09-16T09:32:16.496452+0000","last_clean":"2024-09-16T09:32:16.496452+0000","last_became_active":"2024-09-16T09:25:05.717591+0000","last_became_peered":"2024-09-16T09:25:05.717591+0000","last_unstale":"2024-09-16T09:32:16.496452+0000","last_undegraded":"2024-09-16T09:32:16.496452+0000","last_fullsized":"2024-09-16T09:32:16.496452+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:11.173707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":52,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.494282+0000","last_change":"2024-09-16T09:31:09.272967+0000","last_active":"2024-09-16T09:32:16.494282+0000","last_peered":"2024-09-16T09:32:16.494282+0000","last_clean":"2024-09-16T09:32:16.494282+0000","last_became_active":"2024-09-16T09:31:09.272609+0000","last_became_peered":"2024-09-16T09:31:09.272609+0000","last_unstale":"2024-09-16T09:32:16.494282+0000","last_undegraded":"2024-09-16T09:32:16.494282+0000","last_fullsized":"2024-09-16T09:32:16.494282+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:45.201503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:45.201503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"207'11374","reported_seq":22616,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:17.765010+0000","last_change":"2024-09-16T09:32:08.439285+0000","last_active":"2024-09-16T09:32:17.765010+0000","last_peered":"2024-09-16T09:32:17.765010+0000","last_clean":"2024-09-16T09:32:17.765010+0000","last_became_active":"2024-09-16T09:32:08.422751+0000","last_became_peered":"2024-09-16T09:32:08.422751+0000","last_unstale":"2024-09-16T09:32:17.765010+0000","last_undegraded":"2024-09-16T09:32:17.765010+0000","last_fullsized":"2024-09-16T09:32:17.765010+0000","mapping_epoch":204,"log_start":"189'7100","ondisk_log_start":"189'7100","created":152,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":4274,"log_dups_size":0,"ondisk_log_size":4274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":272629760,"num_objects":4160,"num_object_clones":0,"num_object_copies":12480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4160,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11374,"num_write_kb":497088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5551,"num_bytes_recovered":363790336,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.2","version":"73'79","reported_seq":406,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.496980+0000","last_change":"2024-09-16T09:32:13.450912+0000","last_active":"2024-09-16T09:32:16.496980+0000","last_peered":"2024-09-16T09:32:16.496980+0000","last_clean":"2024-09-16T09:32:16.496980+0000","last_became_active":"2024-09-16T09:30:08.765943+0000","last_became_peered":"2024-09-16T09:30:08.765943+0000","last_unstale":"2024-09-16T09:32:16.496980+0000","last_undegraded":"2024-09-16T09:32:16.496980+0000","last_fullsized":"2024-09-16T09:32:16.496980+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:26:35.745837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:26:35.745837+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:32:42.371213+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"66'13","reported_seq":350,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.507737+0000","last_change":"2024-09-16T09:27:44.156057+0000","last_active":"2024-09-16T09:32:16.507737+0000","last_peered":"2024-09-16T09:32:16.507737+0000","last_clean":"2024-09-16T09:32:16.507737+0000","last_became_active":"2024-09-16T09:25:03.713241+0000","last_became_peered":"2024-09-16T09:25:03.713241+0000","last_unstale":"2024-09-16T09:32:16.507737+0000","last_undegraded":"2024-09-16T09:32:16.507737+0000","last_fullsized":"2024-09-16T09:32:16.507737+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'13","last_scrub_stamp":"2024-09-16T09:27:44.155969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:44.155969+0000","objects_scrubbed":5,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":29,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":327,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.491085+0000","last_change":"2024-09-16T09:29:50.827103+0000","last_active":"2024-09-16T09:32:16.491085+0000","last_peered":"2024-09-16T09:32:16.491085+0000","last_clean":"2024-09-16T09:32:16.491085+0000","last_became_active":"2024-09-16T09:25:01.706828+0000","last_became_peered":"2024-09-16T09:25:01.706828+0000","last_unstale":"2024-09-16T09:32:16.491085+0000","last_undegraded":"2024-09-16T09:32:16.491085+0000","last_fullsized":"2024-09-16T09:32:16.491085+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:50.827057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:50.827057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":358,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.502804+0000","last_change":"2024-09-16T09:31:09.281163+0000","last_active":"2024-09-16T09:32:16.502804+0000","last_peered":"2024-09-16T09:32:16.502804+0000","last_clean":"2024-09-16T09:32:16.502804+0000","last_became_active":"2024-09-16T09:31:09.266338+0000","last_became_peered":"2024-09-16T09:31:09.266338+0000","last_unstale":"2024-09-16T09:32:16.502804+0000","last_undegraded":"2024-09-16T09:32:16.502804+0000","last_fullsized":"2024-09-16T09:32:16.502804+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:29:16.104209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:29:16.104209+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0.00055022299999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":327,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.503359+0000","last_change":"2024-09-16T09:29:00.169141+0000","last_active":"2024-09-16T09:32:16.503359+0000","last_peered":"2024-09-16T09:32:16.503359+0000","last_clean":"2024-09-16T09:32:16.503359+0000","last_became_active":"2024-09-16T09:25:05.711539+0000","last_became_peered":"2024-09-16T09:25:05.711539+0000","last_unstale":"2024-09-16T09:32:16.503359+0000","last_undegraded":"2024-09-16T09:32:16.503359+0000","last_fullsized":"2024-09-16T09:32:16.503359+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:00.169077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:00.169077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":314,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.497879+0000","last_change":"2024-09-16T09:29:22.968971+0000","last_active":"2024-09-16T09:32:16.497879+0000","last_peered":"2024-09-16T09:32:16.497879+0000","last_clean":"2024-09-16T09:32:16.497879+0000","last_became_active":"2024-09-16T09:25:07.723282+0000","last_became_peered":"2024-09-16T09:25:07.723282+0000","last_unstale":"2024-09-16T09:32:16.497879+0000","last_undegraded":"2024-09-16T09:32:16.497879+0000","last_fullsized":"2024-09-16T09:32:16.497879+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:22.968908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:22.968908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"207'11373","reported_seq":11495,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:17.891789+0000","last_change":"2024-09-16T09:29:49.448898+0000","last_active":"2024-09-16T09:32:17.891789+0000","last_peered":"2024-09-16T09:32:17.891789+0000","last_clean":"2024-09-16T09:32:17.891789+0000","last_became_active":"2024-09-16T09:29:49.448628+0000","last_became_peered":"2024-09-16T09:29:49.448628+0000","last_unstale":"2024-09-16T09:32:17.891789+0000","last_undegraded":"2024-09-16T09:32:17.891789+0000","last_fullsized":"2024-09-16T09:32:17.891789+0000","mapping_epoch":152,"log_start":"187'6800","ondisk_log_start":"187'6800","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":4573,"log_dups_size":0,"ondisk_log_size":4573,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":279248896,"num_objects":4261,"num_object_clones":0,"num_object_copies":12783,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4261,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11373,"num_write_kb":500288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"73'79","reported_seq":288,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.501229+0000","last_change":"2024-09-16T09:31:40.212968+0000","last_active":"2024-09-16T09:32:16.501229+0000","last_peered":"2024-09-16T09:32:16.501229+0000","last_clean":"2024-09-16T09:32:16.501229+0000","last_became_active":"2024-09-16T09:30:08.877110+0000","last_became_peered":"2024-09-16T09:30:08.877110+0000","last_unstale":"2024-09-16T09:32:16.501229+0000","last_undegraded":"2024-09-16T09:32:16.501229+0000","last_fullsized":"2024-09-16T09:32:16.501229+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:31:40.212889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:31:40.212889+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:45.708324+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"66'12","reported_seq":333,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.497642+0000","last_change":"2024-09-16T09:29:02.713335+0000","last_active":"2024-09-16T09:32:16.497642+0000","last_peered":"2024-09-16T09:32:16.497642+0000","last_clean":"2024-09-16T09:32:16.497642+0000","last_became_active":"2024-09-16T09:25:03.715315+0000","last_became_peered":"2024-09-16T09:25:03.715315+0000","last_unstale":"2024-09-16T09:32:16.497642+0000","last_undegraded":"2024-09-16T09:32:16.497642+0000","last_fullsized":"2024-09-16T09:32:16.497642+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-16T09:29:02.713259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:29:02.713259+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:43.220661+0000","scrub_duration":26,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"59'1","reported_seq":350,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.499721+0000","last_change":"2024-09-16T09:31:09.286309+0000","last_active":"2024-09-16T09:32:16.499721+0000","last_peered":"2024-09-16T09:32:16.499721+0000","last_clean":"2024-09-16T09:32:16.499721+0000","last_became_active":"2024-09-16T09:31:09.269627+0000","last_became_peered":"2024-09-16T09:31:09.269627+0000","last_unstale":"2024-09-16T09:32:16.499721+0000","last_undegraded":"2024-09-16T09:32:16.499721+0000","last_fullsized":"2024-09-16T09:32:16.499721+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:27:00.666673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:27:00.666673+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":28,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1986,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"67'5","reported_seq":767,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.498586+0000","last_change":"2024-09-16T09:29:19.994791+0000","last_active":"2024-09-16T09:32:16.498586+0000","last_peered":"2024-09-16T09:32:16.498586+0000","last_clean":"2024-09-16T09:32:16.498586+0000","last_became_active":"2024-09-16T09:25:00.696546+0000","last_became_peered":"2024-09-16T09:25:00.696546+0000","last_unstale":"2024-09-16T09:32:16.498586+0000","last_undegraded":"2024-09-16T09:32:16.498586+0000","last_fullsized":"2024-09-16T09:32:16.498586+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'5","last_scrub_stamp":"2024-09-16T09:29:19.994716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:29:19.994716+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0.0059770420000000001,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":430,"num_read_kb":425,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":310,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.490880+0000","last_change":"2024-09-16T09:27:07.337170+0000","last_active":"2024-09-16T09:32:16.490880+0000","last_peered":"2024-09-16T09:32:16.490880+0000","last_clean":"2024-09-16T09:32:16.490880+0000","last_became_active":"2024-09-16T09:25:05.714704+0000","last_became_peered":"2024-09-16T09:25:05.714704+0000","last_unstale":"2024-09-16T09:32:16.490880+0000","last_undegraded":"2024-09-16T09:32:16.490880+0000","last_fullsized":"2024-09-16T09:32:16.490880+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:07.337124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:07.337124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":321,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.498350+0000","last_change":"2024-09-16T09:31:09.271453+0000","last_active":"2024-09-16T09:32:16.498350+0000","last_peered":"2024-09-16T09:32:16.498350+0000","last_clean":"2024-09-16T09:32:16.498350+0000","last_became_active":"2024-09-16T09:31:09.271111+0000","last_became_peered":"2024-09-16T09:31:09.271111+0000","last_unstale":"2024-09-16T09:32:16.498350+0000","last_undegraded":"2024-09-16T09:32:16.498350+0000","last_fullsized":"2024-09-16T09:32:16.498350+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:24.231167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:24.231167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"207'11297","reported_seq":11484,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:17.594261+0000","last_change":"2024-09-16T09:30:56.023710+0000","last_active":"2024-09-16T09:32:17.594261+0000","last_peered":"2024-09-16T09:32:17.594261+0000","last_clean":"2024-09-16T09:32:17.594261+0000","last_became_active":"2024-09-16T09:29:49.450643+0000","last_became_peered":"2024-09-16T09:29:49.450643+0000","last_unstale":"2024-09-16T09:32:17.594261+0000","last_undegraded":"2024-09-16T09:32:17.594261+0000","last_fullsized":"2024-09-16T09:32:17.594261+0000","mapping_epoch":152,"log_start":"182'6000","ondisk_log_start":"182'6000","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"181'5780","last_scrub_stamp":"2024-09-16T09:30:56.023622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:30:56.023622+0000","objects_scrubbed":5716,"log_size":5297,"log_dups_size":0,"ondisk_log_size":5297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:14.532550+0000","scrub_duration":1526,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":279642112,"num_objects":4267,"num_object_clones":0,"num_object_copies":12801,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4267,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11297,"num_write_kb":498048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"66'16","reported_seq":441,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.492448+0000","last_change":"2024-09-16T09:32:01.564577+0000","last_active":"2024-09-16T09:32:16.492448+0000","last_peered":"2024-09-16T09:32:16.492448+0000","last_clean":"2024-09-16T09:32:16.492448+0000","last_became_active":"2024-09-16T09:25:03.714118+0000","last_became_peered":"2024-09-16T09:25:03.714118+0000","last_unstale":"2024-09-16T09:32:16.492448+0000","last_undegraded":"2024-09-16T09:32:16.492448+0000","last_fullsized":"2024-09-16T09:32:16.492448+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'16","last_scrub_stamp":"2024-09-16T09:32:01.564489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:32:01.564489+0000","objects_scrubbed":8,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:10.625142+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":168,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.491679+0000","last_change":"2024-09-16T09:31:09.270603+0000","last_active":"2024-09-16T09:32:16.491679+0000","last_peered":"2024-09-16T09:32:16.491679+0000","last_clean":"2024-09-16T09:32:16.491679+0000","last_became_active":"2024-09-16T09:31:09.270196+0000","last_became_peered":"2024-09-16T09:31:09.270196+0000","last_unstale":"2024-09-16T09:32:16.491679+0000","last_undegraded":"2024-09-16T09:32:16.491679+0000","last_fullsized":"2024-09-16T09:32:16.491679+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:49.195331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:27:49.195331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"73'79","reported_seq":406,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.498622+0000","last_change":"2024-09-16T09:32:13.453628+0000","last_active":"2024-09-16T09:32:16.498622+0000","last_peered":"2024-09-16T09:32:16.498622+0000","last_clean":"2024-09-16T09:32:16.498622+0000","last_became_active":"2024-09-16T09:30:08.870394+0000","last_became_peered":"2024-09-16T09:30:08.870394+0000","last_unstale":"2024-09-16T09:32:16.498622+0000","last_undegraded":"2024-09-16T09:32:16.498622+0000","last_fullsized":"2024-09-16T09:32:16.498622+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:02.723794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:02.723794+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:32:43.420510+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":310,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.497512+0000","last_change":"2024-09-16T09:26:39.699133+0000","last_active":"2024-09-16T09:32:16.497512+0000","last_peered":"2024-09-16T09:32:16.497512+0000","last_clean":"2024-09-16T09:32:16.497512+0000","last_became_active":"2024-09-16T09:25:05.714341+0000","last_became_peered":"2024-09-16T09:25:05.714341+0000","last_unstale":"2024-09-16T09:32:16.497512+0000","last_undegraded":"2024-09-16T09:32:16.497512+0000","last_fullsized":"2024-09-16T09:32:16.497512+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:39.699059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:26:39.699059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":314,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.490648+0000","last_change":"2024-09-16T09:29:00.243528+0000","last_active":"2024-09-16T09:32:16.490648+0000","last_peered":"2024-09-16T09:32:16.490648+0000","last_clean":"2024-09-16T09:32:16.490648+0000","last_became_active":"2024-09-16T09:25:07.714281+0000","last_became_peered":"2024-09-16T09:25:07.714281+0000","last_unstale":"2024-09-16T09:32:16.490648+0000","last_undegraded":"2024-09-16T09:32:16.490648+0000","last_fullsized":"2024-09-16T09:32:16.490648+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:00.243482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:00.243482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.b","version":"73'79","reported_seq":291,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.500592+0000","last_change":"2024-09-16T09:32:16.500592+0000","last_active":"2024-09-16T09:32:16.500592+0000","last_peered":"2024-09-16T09:32:16.500592+0000","last_clean":"2024-09-16T09:32:16.500592+0000","last_became_active":"2024-09-16T09:30:10.784688+0000","last_became_peered":"2024-09-16T09:30:10.784688+0000","last_unstale":"2024-09-16T09:32:16.500592+0000","last_undegraded":"2024-09-16T09:32:16.500592+0000","last_fullsized":"2024-09-16T09:32:16.500592+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:30:03.887571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:30:03.887571+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:32:44.156390+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,5],"acting":[1,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2","version":"207'11142","reported_seq":11262,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:17.793746+0000","last_change":"2024-09-16T09:29:49.445324+0000","last_active":"2024-09-16T09:32:17.793746+0000","last_peered":"2024-09-16T09:32:17.793746+0000","last_clean":"2024-09-16T09:32:17.793746+0000","last_became_active":"2024-09-16T09:29:49.445022+0000","last_became_peered":"2024-09-16T09:29:49.445022+0000","last_unstale":"2024-09-16T09:32:17.793746+0000","last_undegraded":"2024-09-16T09:32:17.793746+0000","last_fullsized":"2024-09-16T09:32:17.793746+0000","mapping_epoch":152,"log_start":"187'6900","ondisk_log_start":"187'6900","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":4242,"log_dups_size":0,"ondisk_log_size":4242,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":268304384,"num_objects":4094,"num_object_clones":0,"num_object_copies":12282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4094,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11142,"num_write_kb":487552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"66'11","reported_seq":353,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.488142+0000","last_change":"2024-09-16T09:30:05.646701+0000","last_active":"2024-09-16T09:32:16.488142+0000","last_peered":"2024-09-16T09:32:16.488142+0000","last_clean":"2024-09-16T09:32:16.488142+0000","last_became_active":"2024-09-16T09:26:33.287399+0000","last_became_peered":"2024-09-16T09:26:33.287399+0000","last_unstale":"2024-09-16T09:32:16.488142+0000","last_undegraded":"2024-09-16T09:32:16.488142+0000","last_fullsized":"2024-09-16T09:32:16.488142+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-16T09:30:05.646500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:30:05.646500+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":311,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.487877+0000","last_change":"2024-09-16T09:25:01.708568+0000","last_active":"2024-09-16T09:32:16.487877+0000","last_peered":"2024-09-16T09:32:16.487877+0000","last_clean":"2024-09-16T09:32:16.487877+0000","last_became_active":"2024-09-16T09:25:01.708270+0000","last_became_peered":"2024-09-16T09:25:01.708270+0000","last_unstale":"2024-09-16T09:32:16.487877+0000","last_undegraded":"2024-09-16T09:32:16.487877+0000","last_fullsized":"2024-09-16T09:32:16.487877+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:10.865036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":310,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.490507+0000","last_change":"2024-09-16T09:26:34.171190+0000","last_active":"2024-09-16T09:32:16.490507+0000","last_peered":"2024-09-16T09:32:16.490507+0000","last_clean":"2024-09-16T09:32:16.490507+0000","last_became_active":"2024-09-16T09:25:05.711964+0000","last_became_peered":"2024-09-16T09:25:05.711964+0000","last_unstale":"2024-09-16T09:32:16.490507+0000","last_undegraded":"2024-09-16T09:32:16.490507+0000","last_fullsized":"2024-09-16T09:32:16.490507+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:34.171139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:26:34.171139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":193,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.492286+0000","last_change":"2024-09-16T09:32:00.526741+0000","last_active":"2024-09-16T09:32:16.492286+0000","last_peered":"2024-09-16T09:32:16.492286+0000","last_clean":"2024-09-16T09:32:16.492286+0000","last_became_active":"2024-09-16T09:31:09.278618+0000","last_became_peered":"2024-09-16T09:31:09.278618+0000","last_unstale":"2024-09-16T09:32:16.492286+0000","last_undegraded":"2024-09-16T09:32:16.492286+0000","last_fullsized":"2024-09-16T09:32:16.492286+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:00.526674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:32:00.526674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:25.768941+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1","version":"207'11059","reported_seq":11180,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:17.895429+0000","last_change":"2024-09-16T09:29:49.449796+0000","last_active":"2024-09-16T09:32:17.895429+0000","last_peered":"2024-09-16T09:32:17.895429+0000","last_clean":"2024-09-16T09:32:17.895429+0000","last_became_active":"2024-09-16T09:29:49.449475+0000","last_became_peered":"2024-09-16T09:29:49.449475+0000","last_unstale":"2024-09-16T09:32:17.895429+0000","last_undegraded":"2024-09-16T09:32:17.895429+0000","last_fullsized":"2024-09-16T09:32:17.895429+0000","mapping_epoch":152,"log_start":"185'6500","ondisk_log_start":"185'6500","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":4559,"log_dups_size":0,"ondisk_log_size":4559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":275185664,"num_objects":4199,"num_object_clones":0,"num_object_copies":12597,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4199,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11059,"num_write_kb":488256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.8","version":"73'79","reported_seq":289,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.495668+0000","last_change":"2024-09-16T09:32:07.068820+0000","last_active":"2024-09-16T09:32:16.495668+0000","last_peered":"2024-09-16T09:32:16.495668+0000","last_clean":"2024-09-16T09:32:16.495668+0000","last_became_active":"2024-09-16T09:31:09.406638+0000","last_became_peered":"2024-09-16T09:31:09.406638+0000","last_unstale":"2024-09-16T09:32:16.495668+0000","last_undegraded":"2024-09-16T09:32:16.495668+0000","last_fullsized":"2024-09-16T09:32:16.495668+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:17.482769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:17.482769+0000","objects_scrubbed":1,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262688,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":525376,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"66'17","reported_seq":359,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.488154+0000","last_change":"2024-09-16T09:30:04.636115+0000","last_active":"2024-09-16T09:32:16.488154+0000","last_peered":"2024-09-16T09:32:16.488154+0000","last_clean":"2024-09-16T09:32:16.488154+0000","last_became_active":"2024-09-16T09:25:03.713358+0000","last_became_peered":"2024-09-16T09:25:03.713358+0000","last_unstale":"2024-09-16T09:32:16.488154+0000","last_undegraded":"2024-09-16T09:32:16.488154+0000","last_fullsized":"2024-09-16T09:32:16.488154+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'17","last_scrub_stamp":"2024-09-16T09:30:04.636039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:30:04.636039+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":319,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.491984+0000","last_change":"2024-09-16T09:26:04.104188+0000","last_active":"2024-09-16T09:32:16.491984+0000","last_peered":"2024-09-16T09:32:16.491984+0000","last_clean":"2024-09-16T09:32:16.491984+0000","last_became_active":"2024-09-16T09:25:01.703355+0000","last_became_peered":"2024-09-16T09:25:01.703355+0000","last_unstale":"2024-09-16T09:32:16.491984+0000","last_undegraded":"2024-09-16T09:32:16.491984+0000","last_fullsized":"2024-09-16T09:32:16.491984+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:04.104130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:04.104130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:48.432563+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":327,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.498470+0000","last_change":"2024-09-16T09:29:58.839132+0000","last_active":"2024-09-16T09:32:16.498470+0000","last_peered":"2024-09-16T09:32:16.498470+0000","last_clean":"2024-09-16T09:32:16.498470+0000","last_became_active":"2024-09-16T09:25:05.716900+0000","last_became_peered":"2024-09-16T09:25:05.716900+0000","last_unstale":"2024-09-16T09:32:16.498470+0000","last_undegraded":"2024-09-16T09:32:16.498470+0000","last_fullsized":"2024-09-16T09:32:16.498470+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:58.839051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:58.839051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":337,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.511471+0000","last_change":"2024-09-16T09:31:09.276869+0000","last_active":"2024-09-16T09:32:16.511471+0000","last_peered":"2024-09-16T09:32:16.511471+0000","last_clean":"2024-09-16T09:32:16.511471+0000","last_became_active":"2024-09-16T09:31:09.276275+0000","last_became_peered":"2024-09-16T09:31:09.276275+0000","last_unstale":"2024-09-16T09:32:16.511471+0000","last_undegraded":"2024-09-16T09:32:16.511471+0000","last_fullsized":"2024-09-16T09:32:16.511471+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:30:03.946025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:30:03.946025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.0","version":"207'11385","reported_seq":17916,"reported_epoch":207,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-16T09:32:17.946581+0000","last_change":"2024-09-16T09:31:09.476408+0000","last_active":"2024-09-16T09:32:17.946581+0000","last_peered":"2024-09-16T09:32:17.946581+0000","last_clean":"2024-09-16T09:30:20.486290+0000","last_became_active":"2024-09-16T09:31:09.399514+0000","last_became_peered":"2024-09-16T09:31:09.399514+0000","last_unstale":"2024-09-16T09:32:17.946581+0000","last_undegraded":"2024-09-16T09:32:17.946581+0000","last_fullsized":"2024-09-16T09:31:09.254572+0000","mapping_epoch":186,"log_start":"184'6600","ondisk_log_start":"184'6600","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":4785,"log_dups_size":0,"ondisk_log_size":4785,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":272039936,"num_objects":4151,"num_object_clones":0,"num_object_copies":12453,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1502,"num_objects_unfound":0,"num_objects_dirty":4151,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11385,"num_write_kb":497152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3200,"num_bytes_recovered":209715200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,1],"avail_no_missing":["5","1"],"object_location_counts":[{"shards":"1,4,5","objects":1502},{"shards":"1,5","objects":2649}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.9","version":"73'79","reported_seq":390,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.497049+0000","last_change":"2024-09-16T09:29:19.013948+0000","last_active":"2024-09-16T09:32:16.497049+0000","last_peered":"2024-09-16T09:32:16.497049+0000","last_clean":"2024-09-16T09:32:16.497049+0000","last_became_active":"2024-09-16T09:26:35.126066+0000","last_became_peered":"2024-09-16T09:26:35.126066+0000","last_unstale":"2024-09-16T09:32:16.497049+0000","last_undegraded":"2024-09-16T09:32:16.497049+0000","last_fullsized":"2024-09-16T09:32:16.497049+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:19.013865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:19.013865+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"66'10","reported_seq":322,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.488634+0000","last_change":"2024-09-16T09:25:03.708144+0000","last_active":"2024-09-16T09:32:16.488634+0000","last_peered":"2024-09-16T09:32:16.488634+0000","last_clean":"2024-09-16T09:32:16.488634+0000","last_became_active":"2024-09-16T09:25:03.707723+0000","last_became_peered":"2024-09-16T09:25:03.707723+0000","last_unstale":"2024-09-16T09:32:16.488634+0000","last_undegraded":"2024-09-16T09:32:16.488634+0000","last_fullsized":"2024-09-16T09:32:16.488634+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:13.930015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":184,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.496155+0000","last_change":"2024-09-16T09:31:32.253397+0000","last_active":"2024-09-16T09:32:16.496155+0000","last_peered":"2024-09-16T09:32:16.496155+0000","last_clean":"2024-09-16T09:32:16.496155+0000","last_became_active":"2024-09-16T09:31:09.288825+0000","last_became_peered":"2024-09-16T09:31:09.288825+0000","last_unstale":"2024-09-16T09:32:16.496155+0000","last_undegraded":"2024-09-16T09:32:16.496155+0000","last_fullsized":"2024-09-16T09:32:16.496155+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:31:32.253306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:31:32.253306+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:45.682494+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":426,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.504347+0000","last_change":"2024-09-16T09:29:29.866520+0000","last_active":"2024-09-16T09:32:16.504347+0000","last_peered":"2024-09-16T09:32:16.504347+0000","last_clean":"2024-09-16T09:32:16.504347+0000","last_became_active":"2024-09-16T09:25:05.711590+0000","last_became_peered":"2024-09-16T09:25:05.711590+0000","last_unstale":"2024-09-16T09:32:16.504347+0000","last_undegraded":"2024-09-16T09:32:16.504347+0000","last_fullsized":"2024-09-16T09:32:16.504347+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:29:29.866460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:29.866460+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":323,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.487675+0000","last_change":"2024-09-16T09:29:02.667332+0000","last_active":"2024-09-16T09:32:16.487675+0000","last_peered":"2024-09-16T09:32:16.487675+0000","last_clean":"2024-09-16T09:32:16.487675+0000","last_became_active":"2024-09-16T09:25:07.718958+0000","last_became_peered":"2024-09-16T09:25:07.718958+0000","last_unstale":"2024-09-16T09:32:16.487675+0000","last_undegraded":"2024-09-16T09:32:16.487675+0000","last_fullsized":"2024-09-16T09:32:16.487675+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:02.667259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:02.667259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.e","version":"73'79","reported_seq":115,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.487572+0000","last_change":"2024-09-16T09:32:13.445947+0000","last_active":"2024-09-16T09:32:16.487572+0000","last_peered":"2024-09-16T09:32:16.487572+0000","last_clean":"2024-09-16T09:32:16.487572+0000","last_became_active":"2024-09-16T09:30:10.705072+0000","last_became_peered":"2024-09-16T09:30:10.705072+0000","last_unstale":"2024-09-16T09:32:16.487572+0000","last_undegraded":"2024-09-16T09:32:16.487572+0000","last_fullsized":"2024-09-16T09:32:16.487572+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:26:35.170229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:26:35.170229+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:32:42.891843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26835,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":10,"num_write":5,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":26835,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,3],"acting":[4,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.7","version":"207'11285","reported_seq":11407,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:17.903912+0000","last_change":"2024-09-16T09:29:49.839740+0000","last_active":"2024-09-16T09:32:17.903912+0000","last_peered":"2024-09-16T09:32:17.903912+0000","last_clean":"2024-09-16T09:32:17.903912+0000","last_became_active":"2024-09-16T09:29:49.839437+0000","last_became_peered":"2024-09-16T09:29:49.839437+0000","last_unstale":"2024-09-16T09:32:17.903912+0000","last_undegraded":"2024-09-16T09:32:17.903912+0000","last_fullsized":"2024-09-16T09:32:17.903912+0000","mapping_epoch":152,"log_start":"187'6800","ondisk_log_start":"187'6800","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":4485,"log_dups_size":0,"ondisk_log_size":4485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":272564224,"num_objects":4159,"num_object_clones":0,"num_object_copies":12477,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4159,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11285,"num_write_kb":494208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"66'9","reported_seq":347,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.495900+0000","last_change":"2024-09-16T09:31:31.211495+0000","last_active":"2024-09-16T09:32:16.495900+0000","last_peered":"2024-09-16T09:32:16.495900+0000","last_clean":"2024-09-16T09:32:16.495900+0000","last_became_active":"2024-09-16T09:25:03.712230+0000","last_became_peered":"2024-09-16T09:25:03.712230+0000","last_unstale":"2024-09-16T09:32:16.495900+0000","last_undegraded":"2024-09-16T09:32:16.495900+0000","last_fullsized":"2024-09-16T09:32:16.495900+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:31:31.211420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:31:31.211420+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:36.499658+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":332,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.492038+0000","last_change":"2024-09-16T09:31:09.277413+0000","last_active":"2024-09-16T09:32:16.492038+0000","last_peered":"2024-09-16T09:32:16.492038+0000","last_clean":"2024-09-16T09:32:16.492038+0000","last_became_active":"2024-09-16T09:31:09.276516+0000","last_became_peered":"2024-09-16T09:31:09.276516+0000","last_unstale":"2024-09-16T09:32:16.492038+0000","last_undegraded":"2024-09-16T09:32:16.492038+0000","last_fullsized":"2024-09-16T09:32:16.492038+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:26.061106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:26.061106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":333,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.503088+0000","last_change":"2024-09-16T09:31:09.275219+0000","last_active":"2024-09-16T09:32:16.503088+0000","last_peered":"2024-09-16T09:32:16.503088+0000","last_clean":"2024-09-16T09:32:16.503088+0000","last_became_active":"2024-09-16T09:31:09.274702+0000","last_became_peered":"2024-09-16T09:31:09.274702+0000","last_unstale":"2024-09-16T09:32:16.503088+0000","last_undegraded":"2024-09-16T09:32:16.503088+0000","last_fullsized":"2024-09-16T09:32:16.503088+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:57.220068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:28:57.220068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":33,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":323,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.499287+0000","last_change":"2024-09-16T09:31:27.185957+0000","last_active":"2024-09-16T09:32:16.499287+0000","last_peered":"2024-09-16T09:32:16.499287+0000","last_clean":"2024-09-16T09:32:16.499287+0000","last_became_active":"2024-09-16T09:25:07.718778+0000","last_became_peered":"2024-09-16T09:25:07.718778+0000","last_unstale":"2024-09-16T09:32:16.499287+0000","last_undegraded":"2024-09-16T09:32:16.499287+0000","last_fullsized":"2024-09-16T09:32:16.499287+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:31:27.185801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:31:27.185801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:39.017009+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.f","version":"73'79","reported_seq":367,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.492685+0000","last_change":"2024-09-16T09:32:16.492685+0000","last_active":"2024-09-16T09:32:16.492685+0000","last_peered":"2024-09-16T09:32:16.492685+0000","last_clean":"2024-09-16T09:32:16.492685+0000","last_became_active":"2024-09-16T09:30:10.693477+0000","last_became_peered":"2024-09-16T09:30:10.693477+0000","last_unstale":"2024-09-16T09:32:16.492685+0000","last_undegraded":"2024-09-16T09:32:16.492685+0000","last_fullsized":"2024-09-16T09:32:16.492685+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:17.534429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:17.534429+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:32:43.568934+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.6","version":"207'11545","reported_seq":11667,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.131037+0000","last_change":"2024-09-16T09:29:50.008888+0000","last_active":"2024-09-16T09:32:18.131037+0000","last_peered":"2024-09-16T09:32:18.131037+0000","last_clean":"2024-09-16T09:32:18.131037+0000","last_became_active":"2024-09-16T09:29:50.008656+0000","last_became_peered":"2024-09-16T09:29:50.008656+0000","last_unstale":"2024-09-16T09:32:18.131037+0000","last_undegraded":"2024-09-16T09:32:18.131037+0000","last_fullsized":"2024-09-16T09:32:18.131037+0000","mapping_epoch":152,"log_start":"183'6400","ondisk_log_start":"183'6400","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":5145,"log_dups_size":0,"ondisk_log_size":5145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":280297472,"num_objects":4277,"num_object_clones":0,"num_object_copies":12831,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4277,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11545,"num_write_kb":506304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"66'19","reported_seq":354,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.491865+0000","last_change":"2024-09-16T09:29:02.554357+0000","last_active":"2024-09-16T09:32:16.491865+0000","last_peered":"2024-09-16T09:32:16.491865+0000","last_clean":"2024-09-16T09:32:16.491865+0000","last_became_active":"2024-09-16T09:25:03.712783+0000","last_became_peered":"2024-09-16T09:25:03.712783+0000","last_unstale":"2024-09-16T09:32:16.491865+0000","last_undegraded":"2024-09-16T09:32:16.491865+0000","last_fullsized":"2024-09-16T09:32:16.491865+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-16T09:29:02.554274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:29:02.554274+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:17.371584+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"59'1","reported_seq":332,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.498071+0000","last_change":"2024-09-16T09:29:51.025505+0000","last_active":"2024-09-16T09:32:16.498071+0000","last_peered":"2024-09-16T09:32:16.498071+0000","last_clean":"2024-09-16T09:32:16.498071+0000","last_became_active":"2024-09-16T09:25:01.708069+0000","last_became_peered":"2024-09-16T09:25:01.708069+0000","last_unstale":"2024-09-16T09:32:16.498071+0000","last_undegraded":"2024-09-16T09:32:16.498071+0000","last_fullsized":"2024-09-16T09:32:16.498071+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:29:51.025444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:51.025444+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":311,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.502550+0000","last_change":"2024-09-16T09:27:11.067059+0000","last_active":"2024-09-16T09:32:16.502550+0000","last_peered":"2024-09-16T09:32:16.502550+0000","last_clean":"2024-09-16T09:32:16.502550+0000","last_became_active":"2024-09-16T09:25:05.709264+0000","last_became_peered":"2024-09-16T09:25:05.709264+0000","last_unstale":"2024-09-16T09:32:16.502550+0000","last_undegraded":"2024-09-16T09:32:16.502550+0000","last_fullsized":"2024-09-16T09:32:16.502550+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:11.066986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:11.066986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":306,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.498616+0000","last_change":"2024-09-16T09:27:25.865107+0000","last_active":"2024-09-16T09:32:16.498616+0000","last_peered":"2024-09-16T09:32:16.498616+0000","last_clean":"2024-09-16T09:32:16.498616+0000","last_became_active":"2024-09-16T09:25:07.718750+0000","last_became_peered":"2024-09-16T09:25:07.718750+0000","last_unstale":"2024-09-16T09:32:16.498616+0000","last_undegraded":"2024-09-16T09:32:16.498616+0000","last_fullsized":"2024-09-16T09:32:16.498616+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:25.865030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:25.865030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.c","version":"73'79","reported_seq":56,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.494973+0000","last_change":"2024-09-16T09:31:09.290919+0000","last_active":"2024-09-16T09:32:16.494973+0000","last_peered":"2024-09-16T09:32:16.494973+0000","last_clean":"2024-09-16T09:32:16.494973+0000","last_became_active":"2024-09-16T09:31:09.273770+0000","last_became_peered":"2024-09-16T09:31:09.273770+0000","last_unstale":"2024-09-16T09:32:16.494973+0000","last_undegraded":"2024-09-16T09:32:16.494973+0000","last_fullsized":"2024-09-16T09:32:16.494973+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:30:22.614462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:30:22.614462+0000","objects_scrubbed":1,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":327680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.5","version":"207'11465","reported_seq":11660,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:17.589153+0000","last_change":"2024-09-16T09:31:06.121217+0000","last_active":"2024-09-16T09:32:17.589153+0000","last_peered":"2024-09-16T09:32:17.589153+0000","last_clean":"2024-09-16T09:32:17.589153+0000","last_became_active":"2024-09-16T09:29:50.009013+0000","last_became_peered":"2024-09-16T09:29:50.009013+0000","last_unstale":"2024-09-16T09:32:17.589153+0000","last_undegraded":"2024-09-16T09:32:17.589153+0000","last_fullsized":"2024-09-16T09:32:17.589153+0000","mapping_epoch":152,"log_start":"182'6113","ondisk_log_start":"182'6113","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"184'6587","last_scrub_stamp":"2024-09-16T09:31:06.120990+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:31:06.120990+0000","objects_scrubbed":6523,"log_size":5352,"log_dups_size":0,"ondisk_log_size":5352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:20.260788+0000","scrub_duration":1690,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":280428544,"num_objects":4279,"num_object_clones":0,"num_object_copies":12837,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4279,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11465,"num_write_kb":503808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"66'12","reported_seq":384,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.487696+0000","last_change":"2024-09-16T09:31:09.296915+0000","last_active":"2024-09-16T09:32:16.487696+0000","last_peered":"2024-09-16T09:32:16.487696+0000","last_clean":"2024-09-16T09:32:16.487696+0000","last_became_active":"2024-09-16T09:31:09.269047+0000","last_became_peered":"2024-09-16T09:31:09.269047+0000","last_unstale":"2024-09-16T09:32:16.487696+0000","last_undegraded":"2024-09-16T09:32:16.487696+0000","last_fullsized":"2024-09-16T09:32:16.487696+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-16T09:30:21.728668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:30:21.728668+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":326,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.498005+0000","last_change":"2024-09-16T09:29:53.075288+0000","last_active":"2024-09-16T09:32:16.498005+0000","last_peered":"2024-09-16T09:32:16.498005+0000","last_clean":"2024-09-16T09:32:16.498005+0000","last_became_active":"2024-09-16T09:25:01.707528+0000","last_became_peered":"2024-09-16T09:25:01.707528+0000","last_unstale":"2024-09-16T09:32:16.498005+0000","last_undegraded":"2024-09-16T09:32:16.498005+0000","last_fullsized":"2024-09-16T09:32:16.498005+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:53.075232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:53.075232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":319,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.502714+0000","last_change":"2024-09-16T09:27:49.965628+0000","last_active":"2024-09-16T09:32:16.502714+0000","last_peered":"2024-09-16T09:32:16.502714+0000","last_clean":"2024-09-16T09:32:16.502714+0000","last_became_active":"2024-09-16T09:25:05.715253+0000","last_became_peered":"2024-09-16T09:25:05.715253+0000","last_unstale":"2024-09-16T09:32:16.502714+0000","last_undegraded":"2024-09-16T09:32:16.502714+0000","last_fullsized":"2024-09-16T09:32:16.502714+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:49.965424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:49.965424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":307,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.494264+0000","last_change":"2024-09-16T09:27:56.059755+0000","last_active":"2024-09-16T09:32:16.494264+0000","last_peered":"2024-09-16T09:32:16.494264+0000","last_clean":"2024-09-16T09:32:16.494264+0000","last_became_active":"2024-09-16T09:25:07.718899+0000","last_became_peered":"2024-09-16T09:25:07.718899+0000","last_unstale":"2024-09-16T09:32:16.494264+0000","last_undegraded":"2024-09-16T09:32:16.494264+0000","last_fullsized":"2024-09-16T09:32:16.494264+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:56.059687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:56.059687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.d","version":"73'79","reported_seq":115,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.497595+0000","last_change":"2024-09-16T09:32:16.497595+0000","last_active":"2024-09-16T09:32:16.497595+0000","last_peered":"2024-09-16T09:32:16.497595+0000","last_clean":"2024-09-16T09:32:16.497595+0000","last_became_active":"2024-09-16T09:30:10.709958+0000","last_became_peered":"2024-09-16T09:30:10.709958+0000","last_unstale":"2024-09-16T09:32:16.497595+0000","last_undegraded":"2024-09-16T09:32:16.497595+0000","last_fullsized":"2024-09-16T09:32:16.497595+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:58.568841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:58.568841+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:32:44.442311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26835,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":10,"num_write":5,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":26835,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.4","version":"207'11466","reported_seq":11588,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:17.881059+0000","last_change":"2024-09-16T09:29:49.444857+0000","last_active":"2024-09-16T09:32:17.881059+0000","last_peered":"2024-09-16T09:32:17.881059+0000","last_clean":"2024-09-16T09:32:17.881059+0000","last_became_active":"2024-09-16T09:29:49.444515+0000","last_became_peered":"2024-09-16T09:29:49.444515+0000","last_unstale":"2024-09-16T09:32:17.881059+0000","last_undegraded":"2024-09-16T09:32:17.881059+0000","last_fullsized":"2024-09-16T09:32:17.881059+0000","mapping_epoch":152,"log_start":"187'6700","ondisk_log_start":"187'6700","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":4766,"log_dups_size":0,"ondisk_log_size":4766,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":271450112,"num_objects":4142,"num_object_clones":0,"num_object_copies":12426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4142,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11466,"num_write_kb":499456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"66'15","reported_seq":339,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.495729+0000","last_change":"2024-09-16T09:26:33.138828+0000","last_active":"2024-09-16T09:32:16.495729+0000","last_peered":"2024-09-16T09:32:16.495729+0000","last_clean":"2024-09-16T09:32:16.495729+0000","last_became_active":"2024-09-16T09:25:03.715733+0000","last_became_peered":"2024-09-16T09:25:03.715733+0000","last_unstale":"2024-09-16T09:32:16.495729+0000","last_undegraded":"2024-09-16T09:32:16.495729+0000","last_fullsized":"2024-09-16T09:32:16.495729+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-16T09:26:33.138603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:26:33.138603+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":330,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.497830+0000","last_change":"2024-09-16T09:26:26.705847+0000","last_active":"2024-09-16T09:32:16.497830+0000","last_peered":"2024-09-16T09:32:16.497830+0000","last_clean":"2024-09-16T09:32:16.497830+0000","last_became_active":"2024-09-16T09:25:01.707372+0000","last_became_peered":"2024-09-16T09:25:01.707372+0000","last_unstale":"2024-09-16T09:32:16.497830+0000","last_undegraded":"2024-09-16T09:32:16.497830+0000","last_fullsized":"2024-09-16T09:32:16.497830+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-16T09:26:26.705773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:26.705773+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"66'8","reported_seq":422,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.497068+0000","last_change":"2024-09-16T09:30:05.138362+0000","last_active":"2024-09-16T09:32:16.497068+0000","last_peered":"2024-09-16T09:32:16.497068+0000","last_clean":"2024-09-16T09:32:16.497068+0000","last_became_active":"2024-09-16T09:25:05.716189+0000","last_became_peered":"2024-09-16T09:25:05.716189+0000","last_unstale":"2024-09-16T09:32:16.497068+0000","last_undegraded":"2024-09-16T09:32:16.497068+0000","last_fullsized":"2024-09-16T09:32:16.497068+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:30:05.138288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:30:05.138288+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":306,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.490187+0000","last_change":"2024-09-16T09:27:20.219376+0000","last_active":"2024-09-16T09:32:16.490187+0000","last_peered":"2024-09-16T09:32:16.490187+0000","last_clean":"2024-09-16T09:32:16.490187+0000","last_became_active":"2024-09-16T09:25:07.717213+0000","last_became_peered":"2024-09-16T09:25:07.717213+0000","last_unstale":"2024-09-16T09:32:16.490187+0000","last_undegraded":"2024-09-16T09:32:16.490187+0000","last_fullsized":"2024-09-16T09:32:16.490187+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:20.219333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:20.219333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.12","version":"73'79","reported_seq":115,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.490336+0000","last_change":"2024-09-16T09:32:16.490336+0000","last_active":"2024-09-16T09:32:16.490336+0000","last_peered":"2024-09-16T09:32:16.490336+0000","last_clean":"2024-09-16T09:32:16.490336+0000","last_became_active":"2024-09-16T09:30:10.731413+0000","last_became_peered":"2024-09-16T09:30:10.731413+0000","last_unstale":"2024-09-16T09:32:16.490336+0000","last_undegraded":"2024-09-16T09:32:16.490336+0000","last_fullsized":"2024-09-16T09:32:16.490336+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:26:35.745837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:26:35.745837+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:32:44.557232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26835,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":10,"num_write":6,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":26835,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":319,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.491551+0000","last_change":"2024-09-16T09:26:33.139405+0000","last_active":"2024-09-16T09:32:16.491551+0000","last_peered":"2024-09-16T09:32:16.491551+0000","last_clean":"2024-09-16T09:32:16.491551+0000","last_became_active":"2024-09-16T09:25:01.703202+0000","last_became_peered":"2024-09-16T09:25:01.703202+0000","last_unstale":"2024-09-16T09:32:16.491551+0000","last_undegraded":"2024-09-16T09:32:16.491551+0000","last_fullsized":"2024-09-16T09:32:16.491551+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:33.139360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:33.139360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:10.581208+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"66'6","reported_seq":60,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.496362+0000","last_change":"2024-09-16T09:31:09.297514+0000","last_active":"2024-09-16T09:32:16.496362+0000","last_peered":"2024-09-16T09:32:16.496362+0000","last_clean":"2024-09-16T09:32:16.496362+0000","last_became_active":"2024-09-16T09:31:09.270685+0000","last_became_peered":"2024-09-16T09:31:09.270685+0000","last_unstale":"2024-09-16T09:32:16.496362+0000","last_undegraded":"2024-09-16T09:32:16.496362+0000","last_fullsized":"2024-09-16T09:32:16.496362+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'6","last_scrub_stamp":"2024-09-16T09:26:23.458794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:26:23.458794+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":8,"num_read_kb":5,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":317,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.491714+0000","last_change":"2024-09-16T09:29:31.496374+0000","last_active":"2024-09-16T09:32:16.491714+0000","last_peered":"2024-09-16T09:32:16.491714+0000","last_clean":"2024-09-16T09:32:16.491714+0000","last_became_active":"2024-09-16T09:25:05.715616+0000","last_became_peered":"2024-09-16T09:25:05.715616+0000","last_unstale":"2024-09-16T09:32:16.491714+0000","last_undegraded":"2024-09-16T09:32:16.491714+0000","last_fullsized":"2024-09-16T09:32:16.491714+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:31.496304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:31.496304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":305,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.497797+0000","last_change":"2024-09-16T09:26:28.733381+0000","last_active":"2024-09-16T09:32:16.497797+0000","last_peered":"2024-09-16T09:32:16.497797+0000","last_clean":"2024-09-16T09:32:16.497797+0000","last_became_active":"2024-09-16T09:25:07.721059+0000","last_became_peered":"2024-09-16T09:25:07.721059+0000","last_unstale":"2024-09-16T09:32:16.497797+0000","last_undegraded":"2024-09-16T09:32:16.497797+0000","last_fullsized":"2024-09-16T09:32:16.497797+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:28.733324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:28.733324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.13","version":"73'79","reported_seq":301,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.500929+0000","last_change":"2024-09-16T09:31:09.288677+0000","last_active":"2024-09-16T09:32:16.500929+0000","last_peered":"2024-09-16T09:32:16.500929+0000","last_clean":"2024-09-16T09:32:16.500929+0000","last_became_active":"2024-09-16T09:31:09.288233+0000","last_became_peered":"2024-09-16T09:31:09.288233+0000","last_unstale":"2024-09-16T09:32:16.500929+0000","last_undegraded":"2024-09-16T09:32:16.500929+0000","last_fullsized":"2024-09-16T09:32:16.500929+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:30:21.714369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:30:21.714369+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.16","version":"66'9","reported_seq":389,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.496722+0000","last_change":"2024-09-16T09:32:01.319481+0000","last_active":"2024-09-16T09:32:16.496722+0000","last_peered":"2024-09-16T09:32:16.496722+0000","last_clean":"2024-09-16T09:32:16.496722+0000","last_became_active":"2024-09-16T09:31:09.273498+0000","last_became_peered":"2024-09-16T09:31:09.273498+0000","last_unstale":"2024-09-16T09:32:16.496722+0000","last_undegraded":"2024-09-16T09:32:16.496722+0000","last_fullsized":"2024-09-16T09:32:16.496722+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:32:01.319390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:32:01.319390+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:29.275572+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":326,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.497212+0000","last_change":"2024-09-16T09:28:50.698331+0000","last_active":"2024-09-16T09:32:16.497212+0000","last_peered":"2024-09-16T09:32:16.497212+0000","last_clean":"2024-09-16T09:32:16.497212+0000","last_became_active":"2024-09-16T09:25:01.706491+0000","last_became_peered":"2024-09-16T09:25:01.706491+0000","last_unstale":"2024-09-16T09:32:16.497212+0000","last_undegraded":"2024-09-16T09:32:16.497212+0000","last_fullsized":"2024-09-16T09:32:16.497212+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:50.698288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:28:50.698288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":176,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.507510+0000","last_change":"2024-09-16T09:31:09.271558+0000","last_active":"2024-09-16T09:32:16.507510+0000","last_peered":"2024-09-16T09:32:16.507510+0000","last_clean":"2024-09-16T09:32:16.507510+0000","last_became_active":"2024-09-16T09:31:09.271223+0000","last_became_peered":"2024-09-16T09:31:09.271223+0000","last_unstale":"2024-09-16T09:32:16.507510+0000","last_undegraded":"2024-09-16T09:32:16.507510+0000","last_fullsized":"2024-09-16T09:32:16.507510+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:53.810580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:28:53.810580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":307,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.503370+0000","last_change":"2024-09-16T09:26:35.866582+0000","last_active":"2024-09-16T09:32:16.503370+0000","last_peered":"2024-09-16T09:32:16.503370+0000","last_clean":"2024-09-16T09:32:16.503370+0000","last_became_active":"2024-09-16T09:25:07.722050+0000","last_became_peered":"2024-09-16T09:25:07.722050+0000","last_unstale":"2024-09-16T09:32:16.503370+0000","last_undegraded":"2024-09-16T09:32:16.503370+0000","last_fullsized":"2024-09-16T09:32:16.503370+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:35.866512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:35.866512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.10","version":"73'79","reported_seq":115,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.491732+0000","last_change":"2024-09-16T09:32:16.491732+0000","last_active":"2024-09-16T09:32:16.491732+0000","last_peered":"2024-09-16T09:32:16.491732+0000","last_clean":"2024-09-16T09:32:16.491732+0000","last_became_active":"2024-09-16T09:30:10.692770+0000","last_became_peered":"2024-09-16T09:30:10.692770+0000","last_unstale":"2024-09-16T09:32:16.491732+0000","last_undegraded":"2024-09-16T09:32:16.491732+0000","last_fullsized":"2024-09-16T09:32:16.491732+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:02.723794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:02.723794+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:32:43.525182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26835,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":10,"num_write":5,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":26835,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"66'9","reported_seq":338,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.490879+0000","last_change":"2024-09-16T09:29:32.485165+0000","last_active":"2024-09-16T09:32:16.490879+0000","last_peered":"2024-09-16T09:32:16.490879+0000","last_clean":"2024-09-16T09:32:16.490879+0000","last_became_active":"2024-09-16T09:25:03.717310+0000","last_became_peered":"2024-09-16T09:25:03.717310+0000","last_unstale":"2024-09-16T09:32:16.490879+0000","last_undegraded":"2024-09-16T09:32:16.490879+0000","last_fullsized":"2024-09-16T09:32:16.490879+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:29:32.485084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:29:32.485084+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":349,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.496480+0000","last_change":"2024-09-16T09:31:59.376923+0000","last_active":"2024-09-16T09:32:16.496480+0000","last_peered":"2024-09-16T09:32:16.496480+0000","last_clean":"2024-09-16T09:32:16.496480+0000","last_became_active":"2024-09-16T09:31:09.272848+0000","last_became_peered":"2024-09-16T09:31:09.272848+0000","last_unstale":"2024-09-16T09:32:16.496480+0000","last_undegraded":"2024-09-16T09:32:16.496480+0000","last_fullsized":"2024-09-16T09:32:16.496480+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:31:59.376749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:31:59.376749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:13.111415+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"187'11","reported_seq":251,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.497498+0000","last_change":"2024-09-16T09:31:09.311243+0000","last_active":"2024-09-16T09:32:16.497498+0000","last_peered":"2024-09-16T09:32:16.497498+0000","last_clean":"2024-09-16T09:32:16.497498+0000","last_became_active":"2024-09-16T09:31:09.277688+0000","last_became_peered":"2024-09-16T09:31:09.277688+0000","last_unstale":"2024-09-16T09:32:16.497498+0000","last_undegraded":"2024-09-16T09:32:16.497498+0000","last_fullsized":"2024-09-16T09:32:16.497498+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:28:53.714413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:28:53.714413+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":322,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.488173+0000","last_change":"2024-09-16T09:29:49.474440+0000","last_active":"2024-09-16T09:32:16.488173+0000","last_peered":"2024-09-16T09:32:16.488173+0000","last_clean":"2024-09-16T09:32:16.488173+0000","last_became_active":"2024-09-16T09:25:07.718810+0000","last_became_peered":"2024-09-16T09:25:07.718810+0000","last_unstale":"2024-09-16T09:32:16.488173+0000","last_undegraded":"2024-09-16T09:32:16.488173+0000","last_fullsized":"2024-09-16T09:32:16.488173+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:49.474391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:49.474391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.11","version":"73'79","reported_seq":201,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.500139+0000","last_change":"2024-09-16T09:31:09.272938+0000","last_active":"2024-09-16T09:32:16.500139+0000","last_peered":"2024-09-16T09:32:16.500139+0000","last_clean":"2024-09-16T09:32:16.500139+0000","last_became_active":"2024-09-16T09:31:09.272208+0000","last_became_peered":"2024-09-16T09:31:09.272208+0000","last_unstale":"2024-09-16T09:32:16.500139+0000","last_undegraded":"2024-09-16T09:32:16.500139+0000","last_fullsized":"2024-09-16T09:32:16.500139+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:30:21.651575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:30:21.651575+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.14","version":"66'10","reported_seq":197,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.498603+0000","last_change":"2024-09-16T09:28:56.825675+0000","last_active":"2024-09-16T09:32:16.498603+0000","last_peered":"2024-09-16T09:32:16.498603+0000","last_clean":"2024-09-16T09:32:16.498603+0000","last_became_active":"2024-09-16T09:28:52.979059+0000","last_became_peered":"2024-09-16T09:28:52.979059+0000","last_unstale":"2024-09-16T09:32:16.498603+0000","last_undegraded":"2024-09-16T09:32:16.498603+0000","last_fullsized":"2024-09-16T09:32:16.498603+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-16T09:28:56.825476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:56.825476+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":14,"num_read_kb":9,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":326,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.497373+0000","last_change":"2024-09-16T09:29:56.021413+0000","last_active":"2024-09-16T09:32:16.497373+0000","last_peered":"2024-09-16T09:32:16.497373+0000","last_clean":"2024-09-16T09:32:16.497373+0000","last_became_active":"2024-09-16T09:25:01.707513+0000","last_became_peered":"2024-09-16T09:25:01.707513+0000","last_unstale":"2024-09-16T09:32:16.497373+0000","last_undegraded":"2024-09-16T09:32:16.497373+0000","last_fullsized":"2024-09-16T09:32:16.497373+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:56.021340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:56.021340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"66'8","reported_seq":421,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.490694+0000","last_change":"2024-09-16T09:29:48.485781+0000","last_active":"2024-09-16T09:32:16.490694+0000","last_peered":"2024-09-16T09:32:16.490694+0000","last_clean":"2024-09-16T09:32:16.490694+0000","last_became_active":"2024-09-16T09:25:05.713833+0000","last_became_peered":"2024-09-16T09:25:05.713833+0000","last_unstale":"2024-09-16T09:32:16.490694+0000","last_undegraded":"2024-09-16T09:32:16.490694+0000","last_fullsized":"2024-09-16T09:32:16.490694+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:29:48.485727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.485727+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":336,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.496105+0000","last_change":"2024-09-16T09:32:00.343429+0000","last_active":"2024-09-16T09:32:16.496105+0000","last_peered":"2024-09-16T09:32:16.496105+0000","last_clean":"2024-09-16T09:32:16.496105+0000","last_became_active":"2024-09-16T09:31:09.275168+0000","last_became_peered":"2024-09-16T09:31:09.275168+0000","last_unstale":"2024-09-16T09:32:16.496105+0000","last_undegraded":"2024-09-16T09:32:16.496105+0000","last_fullsized":"2024-09-16T09:32:16.496105+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:00.343356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:32:00.343356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:23.568990+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"66'11","reported_seq":345,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.488041+0000","last_change":"2024-09-16T09:26:33.302713+0000","last_active":"2024-09-16T09:32:16.488041+0000","last_peered":"2024-09-16T09:32:16.488041+0000","last_clean":"2024-09-16T09:32:16.488041+0000","last_became_active":"2024-09-16T09:26:33.291265+0000","last_became_peered":"2024-09-16T09:26:33.291265+0000","last_unstale":"2024-09-16T09:32:16.488041+0000","last_undegraded":"2024-09-16T09:32:16.488041+0000","last_fullsized":"2024-09-16T09:32:16.488041+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:08.864166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":335,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.488721+0000","last_change":"2024-09-16T09:29:51.517112+0000","last_active":"2024-09-16T09:32:16.488721+0000","last_peered":"2024-09-16T09:32:16.488721+0000","last_clean":"2024-09-16T09:32:16.488721+0000","last_became_active":"2024-09-16T09:25:01.705158+0000","last_became_peered":"2024-09-16T09:25:01.705158+0000","last_unstale":"2024-09-16T09:32:16.488721+0000","last_undegraded":"2024-09-16T09:32:16.488721+0000","last_fullsized":"2024-09-16T09:32:16.488721+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:51.517054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:51.517054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":341,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.498444+0000","last_change":"2024-09-16T09:31:09.266820+0000","last_active":"2024-09-16T09:32:16.498444+0000","last_peered":"2024-09-16T09:32:16.498444+0000","last_clean":"2024-09-16T09:32:16.498444+0000","last_became_active":"2024-09-16T09:31:09.266326+0000","last_became_peered":"2024-09-16T09:31:09.266326+0000","last_unstale":"2024-09-16T09:32:16.498444+0000","last_undegraded":"2024-09-16T09:32:16.498444+0000","last_fullsized":"2024-09-16T09:32:16.498444+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:15.910824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:15.910824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":169,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.499124+0000","last_change":"2024-09-16T09:31:09.277239+0000","last_active":"2024-09-16T09:32:16.499124+0000","last_peered":"2024-09-16T09:32:16.499124+0000","last_clean":"2024-09-16T09:32:16.499124+0000","last_became_active":"2024-09-16T09:31:09.276042+0000","last_became_peered":"2024-09-16T09:31:09.276042+0000","last_unstale":"2024-09-16T09:32:16.499124+0000","last_undegraded":"2024-09-16T09:32:16.499124+0000","last_fullsized":"2024-09-16T09:32:16.499124+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:48.147759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:48.147759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"66'9","reported_seq":331,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.498784+0000","last_change":"2024-09-16T09:26:31.216495+0000","last_active":"2024-09-16T09:32:16.498784+0000","last_peered":"2024-09-16T09:32:16.498784+0000","last_clean":"2024-09-16T09:32:16.498784+0000","last_became_active":"2024-09-16T09:25:03.705807+0000","last_became_peered":"2024-09-16T09:25:03.705807+0000","last_unstale":"2024-09-16T09:32:16.498784+0000","last_undegraded":"2024-09-16T09:32:16.498784+0000","last_fullsized":"2024-09-16T09:32:16.498784+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:26:31.216298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:26:31.216298+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":318,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.498013+0000","last_change":"2024-09-16T09:26:23.796308+0000","last_active":"2024-09-16T09:32:16.498013+0000","last_peered":"2024-09-16T09:32:16.498013+0000","last_clean":"2024-09-16T09:32:16.498013+0000","last_became_active":"2024-09-16T09:25:01.706762+0000","last_became_peered":"2024-09-16T09:25:01.706762+0000","last_unstale":"2024-09-16T09:32:16.498013+0000","last_undegraded":"2024-09-16T09:32:16.498013+0000","last_fullsized":"2024-09-16T09:32:16.498013+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:23.796231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:23.796231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":154,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.495684+0000","last_change":"2024-09-16T09:28:52.983144+0000","last_active":"2024-09-16T09:32:16.495684+0000","last_peered":"2024-09-16T09:32:16.495684+0000","last_clean":"2024-09-16T09:32:16.495684+0000","last_became_active":"2024-09-16T09:28:52.982865+0000","last_became_peered":"2024-09-16T09:28:52.982865+0000","last_unstale":"2024-09-16T09:32:16.495684+0000","last_undegraded":"2024-09-16T09:32:16.495684+0000","last_fullsized":"2024-09-16T09:32:16.495684+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:44.241942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:44.241942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":315,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.495291+0000","last_change":"2024-09-16T09:28:52.552567+0000","last_active":"2024-09-16T09:32:16.495291+0000","last_peered":"2024-09-16T09:32:16.495291+0000","last_clean":"2024-09-16T09:32:16.495291+0000","last_became_active":"2024-09-16T09:25:07.715301+0000","last_became_peered":"2024-09-16T09:25:07.715301+0000","last_unstale":"2024-09-16T09:32:16.495291+0000","last_undegraded":"2024-09-16T09:32:16.495291+0000","last_fullsized":"2024-09-16T09:32:16.495291+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:52.552495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:28:52.552495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:45.269179+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.14","version":"73'79","reported_seq":115,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.503276+0000","last_change":"2024-09-16T09:32:16.503276+0000","last_active":"2024-09-16T09:32:16.503276+0000","last_peered":"2024-09-16T09:32:16.503276+0000","last_clean":"2024-09-16T09:32:16.503276+0000","last_became_active":"2024-09-16T09:30:10.700866+0000","last_became_peered":"2024-09-16T09:30:10.700866+0000","last_unstale":"2024-09-16T09:32:16.503276+0000","last_undegraded":"2024-09-16T09:32:16.503276+0000","last_fullsized":"2024-09-16T09:32:16.503276+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:53.916505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:53.916505+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:32:43.945820+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109227,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"66'11","reported_seq":201,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.497121+0000","last_change":"2024-09-16T09:28:55.643505+0000","last_active":"2024-09-16T09:32:16.497121+0000","last_peered":"2024-09-16T09:32:16.497121+0000","last_clean":"2024-09-16T09:32:16.497121+0000","last_became_active":"2024-09-16T09:28:52.971132+0000","last_became_peered":"2024-09-16T09:28:52.971132+0000","last_unstale":"2024-09-16T09:32:16.497121+0000","last_undegraded":"2024-09-16T09:32:16.497121+0000","last_fullsized":"2024-09-16T09:32:16.497121+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-16T09:28:55.643448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:55.643448+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":19,"num_read_kb":12,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":326,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.491833+0000","last_change":"2024-09-16T09:27:46.175159+0000","last_active":"2024-09-16T09:32:16.491833+0000","last_peered":"2024-09-16T09:32:16.491833+0000","last_clean":"2024-09-16T09:32:16.491833+0000","last_became_active":"2024-09-16T09:25:01.700733+0000","last_became_peered":"2024-09-16T09:25:01.700733+0000","last_unstale":"2024-09-16T09:32:16.491833+0000","last_undegraded":"2024-09-16T09:32:16.491833+0000","last_fullsized":"2024-09-16T09:32:16.491833+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:46.175093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:27:46.175093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":318,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.498332+0000","last_change":"2024-09-16T09:29:25.017736+0000","last_active":"2024-09-16T09:32:16.498332+0000","last_peered":"2024-09-16T09:32:16.498332+0000","last_clean":"2024-09-16T09:32:16.498332+0000","last_became_active":"2024-09-16T09:25:05.713479+0000","last_became_peered":"2024-09-16T09:25:05.713479+0000","last_unstale":"2024-09-16T09:32:16.498332+0000","last_undegraded":"2024-09-16T09:32:16.498332+0000","last_fullsized":"2024-09-16T09:32:16.498332+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:25.017610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:25.017610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":177,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.496075+0000","last_change":"2024-09-16T09:31:09.275121+0000","last_active":"2024-09-16T09:32:16.496075+0000","last_peered":"2024-09-16T09:32:16.496075+0000","last_clean":"2024-09-16T09:32:16.496075+0000","last_became_active":"2024-09-16T09:31:09.274797+0000","last_became_peered":"2024-09-16T09:31:09.274797+0000","last_unstale":"2024-09-16T09:32:16.496075+0000","last_undegraded":"2024-09-16T09:32:16.496075+0000","last_fullsized":"2024-09-16T09:32:16.496075+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:54.589258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:28:54.589258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:48.251310+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"66'4","reported_seq":58,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.493054+0000","last_change":"2024-09-16T09:31:09.401626+0000","last_active":"2024-09-16T09:32:16.493054+0000","last_peered":"2024-09-16T09:32:16.493054+0000","last_clean":"2024-09-16T09:32:16.493054+0000","last_became_active":"2024-09-16T09:31:09.273682+0000","last_became_peered":"2024-09-16T09:31:09.273682+0000","last_unstale":"2024-09-16T09:32:16.493054+0000","last_undegraded":"2024-09-16T09:32:16.493054+0000","last_fullsized":"2024-09-16T09:32:16.493054+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'4","last_scrub_stamp":"2024-09-16T09:26:27.473740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:26:27.473740+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":349,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.496632+0000","last_change":"2024-09-16T09:31:30.235761+0000","last_active":"2024-09-16T09:32:16.496632+0000","last_peered":"2024-09-16T09:32:16.496632+0000","last_clean":"2024-09-16T09:32:16.496632+0000","last_became_active":"2024-09-16T09:31:09.274719+0000","last_became_peered":"2024-09-16T09:31:09.274719+0000","last_unstale":"2024-09-16T09:32:16.496632+0000","last_undegraded":"2024-09-16T09:32:16.496632+0000","last_fullsized":"2024-09-16T09:32:16.496632+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:31:30.235692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:31:30.235692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:49.736157+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":335,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.491593+0000","last_change":"2024-09-16T09:32:07.533575+0000","last_active":"2024-09-16T09:32:16.491593+0000","last_peered":"2024-09-16T09:32:16.491593+0000","last_clean":"2024-09-16T09:32:16.491593+0000","last_became_active":"2024-09-16T09:25:05.712545+0000","last_became_peered":"2024-09-16T09:25:05.712545+0000","last_unstale":"2024-09-16T09:32:16.491593+0000","last_undegraded":"2024-09-16T09:32:16.491593+0000","last_fullsized":"2024-09-16T09:32:16.491593+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:07.533529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:32:07.533529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:28.807212+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":314,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.497002+0000","last_change":"2024-09-16T09:29:27.012810+0000","last_active":"2024-09-16T09:32:16.497002+0000","last_peered":"2024-09-16T09:32:16.497002+0000","last_clean":"2024-09-16T09:32:16.497002+0000","last_became_active":"2024-09-16T09:25:07.720117+0000","last_became_peered":"2024-09-16T09:25:07.720117+0000","last_unstale":"2024-09-16T09:32:16.497002+0000","last_undegraded":"2024-09-16T09:32:16.497002+0000","last_fullsized":"2024-09-16T09:32:16.497002+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:27.012756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:27.012756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":314,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.498983+0000","last_change":"2024-09-16T09:29:19.888825+0000","last_active":"2024-09-16T09:32:16.498983+0000","last_peered":"2024-09-16T09:32:16.498983+0000","last_clean":"2024-09-16T09:32:16.498983+0000","last_became_active":"2024-09-16T09:25:07.712594+0000","last_became_peered":"2024-09-16T09:25:07.712594+0000","last_unstale":"2024-09-16T09:32:16.498983+0000","last_undegraded":"2024-09-16T09:32:16.498983+0000","last_fullsized":"2024-09-16T09:32:16.498983+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:19.888759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:19.888759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":176,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.495806+0000","last_change":"2024-09-16T09:31:26.211625+0000","last_active":"2024-09-16T09:32:16.495806+0000","last_peered":"2024-09-16T09:32:16.495806+0000","last_clean":"2024-09-16T09:32:16.495806+0000","last_became_active":"2024-09-16T09:31:09.271662+0000","last_became_peered":"2024-09-16T09:31:09.271662+0000","last_unstale":"2024-09-16T09:32:16.495806+0000","last_undegraded":"2024-09-16T09:32:16.495806+0000","last_fullsized":"2024-09-16T09:32:16.495806+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:31:26.211363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:31:26.211363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:32.873753+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"66'11","reported_seq":342,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.492735+0000","last_change":"2024-09-16T09:27:57.486459+0000","last_active":"2024-09-16T09:32:16.492735+0000","last_peered":"2024-09-16T09:32:16.492735+0000","last_clean":"2024-09-16T09:32:16.492735+0000","last_became_active":"2024-09-16T09:25:03.707769+0000","last_became_peered":"2024-09-16T09:25:03.707769+0000","last_unstale":"2024-09-16T09:32:16.492735+0000","last_undegraded":"2024-09-16T09:32:16.492735+0000","last_fullsized":"2024-09-16T09:32:16.492735+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-16T09:27:57.486381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:57.486381+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":311,"reported_epoch":207,"state":"active+clean","last_fresh":"2024-09-16T09:32:16.496456+0000","last_change":"2024-09-16T09:27:57.640962+0000","last_active":"2024-09-16T09:32:16.496456+0000","last_peered":"2024-09-16T09:32:16.496456+0000","last_clean":"2024-09-16T09:32:16.496456+0000","last_became_active":"2024-09-16T09:25:05.711952+0000","last_became_peered":"2024-09-16T09:25:05.711952+0000","last_unstale":"2024-09-16T09:32:16.496456+0000","last_undegraded":"2024-09-16T09:32:16.496456+0000","last_fullsized":"2024-09-16T09:32:16.496456+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:57.640756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:57.640756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:21.189003+0000","scrub_duration":44,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":16,"stat_sum":{"num_bytes":4397072408,"num_objects":67095,"num_object_clones":0,"num_object_copies":201285,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2923,"num_objects_unfound":0,"num_objects_dirty":67095,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":182091,"num_write_kb":7973889,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11786,"num_bytes_recovered":772407296,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":13606137856,"data_stored":13606125640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":77778,"ondisk_log_size":77778,"up":48,"acting":46,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":70,"ondisk_log_size":70,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":691,"num_read_kb":448,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":126,"num_bytes_recovered":3026,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":395,"ondisk_log_size":395,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1986,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":430,"num_read_kb":425,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":21,"stat_sum":{"num_bytes":806935,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":21,"num_read_kb":40,"num_write":21,"num_write_kb":296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":960396,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1659,"ondisk_log_size":1659,"up":63,"acting":63,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266780,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1580356,"kb_used_data":1360308,"kb_used_omap":1,"kb_used_meta":219966,"kb_avail":92164796,"statfs":{"total":95995035648,"available":94376751104,"internally_reserved":0,"allocated":1392955392,"data_stored":1392650814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":225245642},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495713,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1546540,"kb_used_data":1362648,"kb_used_omap":1,"kb_used_meta":183870,"kb_avail":92198612,"statfs":{"total":95995035648,"available":94411378688,"internally_reserved":0,"allocated":1395351552,"data_stored":1395074524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":188283334},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724648,"num_pgs":64,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2425020,"kb_used_data":2150888,"kb_used_omap":1,"kb_used_meta":274110,"kb_avail":91320132,"statfs":{"total":95995035648,"available":93511815168,"internally_reserved":0,"allocated":2202509312,"data_stored":2202216937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":280689094},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953581,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1846600,"kb_used_data":1657140,"kb_used_omap":1,"kb_used_meta":189438,"kb_avail":91898552,"statfs":{"total":95995035648,"available":94104117248,"internally_reserved":0,"allocated":1696911360,"data_stored":1696606822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":193984966},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149811,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1743568,"kb_used_data":1427776,"kb_used_omap":1,"kb_used_meta":315774,"kb_avail":92001584,"statfs":{"total":95995035648,"available":94209622016,"internally_reserved":0,"allocated":1462042624,"data_stored":1461726628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":323353031},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411448,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1924852,"kb_used_data":1645152,"kb_used_omap":1,"kb_used_meta":279678,"kb_avail":91820300,"statfs":{"total":95995035648,"available":94023987200,"internally_reserved":0,"allocated":1684635648,"data_stored":1684350928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":286390725},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574975,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2191772,"kb_used_data":1903756,"kb_used_omap":1,"kb_used_meta":287998,"kb_avail":91553380,"statfs":{"total":95995035648,"available":93750661120,"internally_reserved":0,"allocated":1949446144,"data_stored":1949116900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":294910405},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738501,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2226436,"kb_used_data":1916148,"kb_used_omap":1,"kb_used_meta":310206,"kb_avail":91518716,"statfs":{"total":95995035648,"available":93715165184,"internally_reserved":0,"allocated":1962135552,"data_stored":1961831216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":317651397},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1933639680,"data_stored":1933639680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1925906432,"data_stored":1925906432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1656754176,"data_stored":1656750104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1451622400,"data_stored":1451622400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1673068544,"data_stored":1673068544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":2186285056,"data_stored":2186280984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1389821952,"data_stored":1389821952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1389039616,"data_stored":1389035544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T09:32:23.047 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph status --format=json 2024-09-16T09:32:23.054 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:23.055 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:23.055 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:32:23.058 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:32:23.156 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:32:23.160 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:32:22 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:32:23.200 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:23.201 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:23.202 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:32:23.302 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:32:23.350 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:23.351 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:23.351 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:32:23.452 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:32:23.495 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:23.495 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:23.495 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:32:23.596 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:23 smithi070 ceph-mon[30388]: pgmap v343: 168 pgs: 2 active+recovering+undersized+remapped, 166 active+clean; 4.1 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 741 op/s; 2923/201885 objects misplaced (1.448%); 3.0 MiB/s, 47 objects/s recovering 2024-09-16T09:32:23.596 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:23 smithi070 ceph-mon[40691]: pgmap v343: 168 pgs: 2 active+recovering+undersized+remapped, 166 active+clean; 4.1 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 741 op/s; 2923/201885 objects misplaced (1.448%); 3.0 MiB/s, 47 objects/s recovering 2024-09-16T09:32:23.596 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:32:23.637 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:23.637 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:23.638 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:32:23.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:23 smithi084 ceph-mon[32629]: pgmap v343: 168 pgs: 2 active+recovering+undersized+remapped, 166 active+clean; 4.1 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 741 op/s; 2923/201885 objects misplaced (1.448%); 3.0 MiB/s, 47 objects/s recovering 2024-09-16T09:32:23.722 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:32:23.739 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:32:23.767 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T09:32:23.767 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [ 2024-09-16T09:32:23.767 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:23.767 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.25064.0:188889 8.7 8:eaa58f01:::benchmark_data_smithi070_107919_object64294:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e208)", 2024-09-16T09:32:23.767 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:32:23.755584+0000", 2024-09-16T09:32:23.767 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.0093361720000000002, 2024-09-16T09:32:23.767 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0093669410000000002, 2024-09-16T09:32:23.768 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:32:23.768 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:32:23.768 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "waiting for sub ops", 2024-09-16T09:32:23.768 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:32:23.768 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.25064", 2024-09-16T09:32:23.768 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/2578256439", 2024-09-16T09:32:23.768 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 188889 2024-09-16T09:32:23.768 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:23.768 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:32:23.768 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:23.768 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:32:23.768 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:23.755584+0000", 2024-09-16T09:32:23.768 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:32:23.769 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:23.769 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:23.769 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:32:23.769 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:23.755584+0000", 2024-09-16T09:32:23.769 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:32:23.769 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:23.769 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:23.769 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:32:23.769 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:23.755585+0000", 2024-09-16T09:32:23.770 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3069999999999999e-06 2024-09-16T09:32:23.770 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:23.770 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:23.770 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:32:23.770 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:23.755589+0000", 2024-09-16T09:32:23.770 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.641e-06 2024-09-16T09:32:23.771 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:23.771 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:23.771 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:32:23.771 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:23.755597+0000", 2024-09-16T09:32:23.771 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.4449999999999996e-06 2024-09-16T09:32:23.771 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:23.771 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:23.771 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:32:23.771 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:23.755602+0000", 2024-09-16T09:32:23.771 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.276e-06 2024-09-16T09:32:23.771 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:23.771 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:23.771 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:32:23.772 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:23.755686+0000", 2024-09-16T09:32:23.772 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.3461999999999995e-05 2024-09-16T09:32:23.772 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:23.772 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:23.772 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:32:23.772 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:23.755813+0000", 2024-09-16T09:32:23.772 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00012758299999999999 2024-09-16T09:32:23.772 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:23.772 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:23.772 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 3,6", 2024-09-16T09:32:23.772 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:23.755933+0000", 2024-09-16T09:32:23.772 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00011993999999999999 2024-09-16T09:32:23.772 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:23.772 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:23.773 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:32:23.773 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:23.760209+0000", 2024-09-16T09:32:23.773 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0042756410000000002 2024-09-16T09:32:23.773 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:23.773 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:23.773 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:32:23.773 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:23.760251+0000", 2024-09-16T09:32:23.773 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2407999999999998e-05 2024-09-16T09:32:23.773 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:32:23.773 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:32:23.773 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:32:23.774 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:23.774 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:23.774 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.25064.0:188892 8.1 8:82f91cb7:::benchmark_data_smithi070_107919_object64297:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e208)", 2024-09-16T09:32:23.774 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:32:23.760050+0000", 2024-09-16T09:32:23.774 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.0048695090000000002, 2024-09-16T09:32:23.774 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0049773480000000004, 2024-09-16T09:32:23.774 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:32:23.774 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:32:23.774 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "waiting for sub ops", 2024-09-16T09:32:23.774 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:32:23.774 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.25064", 2024-09-16T09:32:23.774 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/2578256439", 2024-09-16T09:32:23.774 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 188892 2024-09-16T09:32:23.774 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:23.775 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:32:23.775 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:23.775 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:32:23.775 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:23.760050+0000", 2024-09-16T09:32:23.775 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:32:23.775 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:23.775 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:23.775 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:32:23.775 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:23.760050+0000", 2024-09-16T09:32:23.775 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:32:23.775 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:23.775 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:23.775 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:32:23.775 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:23.760052+0000", 2024-09-16T09:32:23.775 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.457e-06 2024-09-16T09:32:23.776 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:23.776 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:23.776 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:32:23.776 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:23.760057+0000", 2024-09-16T09:32:23.776 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.7620000000000003e-06 2024-09-16T09:32:23.776 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:23.776 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:23.776 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:32:23.776 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:23.760069+0000", 2024-09-16T09:32:23.776 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2242000000000001e-05 2024-09-16T09:32:23.776 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:23.776 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:23.776 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:32:23.776 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:23.760076+0000", 2024-09-16T09:32:23.777 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3860000000000001e-06 2024-09-16T09:32:23.777 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:23.777 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:23.777 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:32:23.777 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:23.760134+0000", 2024-09-16T09:32:23.777 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.77e-05 2024-09-16T09:32:23.777 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:23.777 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:23.777 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:32:23.777 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:23.760263+0000", 2024-09-16T09:32:23.777 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00012863600000000001 2024-09-16T09:32:23.777 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:23.777 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:23.777 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 0,2", 2024-09-16T09:32:23.778 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:23.760377+0000", 2024-09-16T09:32:23.778 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000114728 2024-09-16T09:32:23.778 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:23.778 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:23.778 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:32:23.778 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:23.762791+0000", 2024-09-16T09:32:23.778 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0024134320000000001 2024-09-16T09:32:23.778 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:32:23.778 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:32:23.778 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:32:23.778 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:23.778 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:23.778 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.25064.0:188899 8.a 8:5feb651b:::benchmark_data_smithi070_107919_object64304:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e208)", 2024-09-16T09:32:23.779 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:32:23.763084+0000", 2024-09-16T09:32:23.779 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.001836133, 2024-09-16T09:32:23.779 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0020064750000000002, 2024-09-16T09:32:23.779 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:32:23.779 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:32:23.779 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "waiting for sub ops", 2024-09-16T09:32:23.779 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:32:23.779 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.25064", 2024-09-16T09:32:23.779 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/2578256439", 2024-09-16T09:32:23.779 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 188899 2024-09-16T09:32:23.779 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:23.779 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:32:23.779 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:23.779 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:32:23.780 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:23.763084+0000", 2024-09-16T09:32:23.780 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:32:23.780 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:23.780 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:23.780 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:32:23.780 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:23.763084+0000", 2024-09-16T09:32:23.780 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:32:23.780 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:23.780 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:23.780 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:32:23.780 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:23.763085+0000", 2024-09-16T09:32:23.780 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6780000000000001e-06 2024-09-16T09:32:23.780 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:23.780 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:23.780 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:32:23.781 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:23.763092+0000", 2024-09-16T09:32:23.781 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.3709999999999996e-06 2024-09-16T09:32:23.781 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:23.781 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:23.781 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:32:23.781 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:23.763108+0000", 2024-09-16T09:32:23.781 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6660999999999999e-05 2024-09-16T09:32:23.781 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:23.781 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:23.781 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:32:23.781 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:23.763117+0000", 2024-09-16T09:32:23.781 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.5799999999999992e-06 2024-09-16T09:32:23.781 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:23.781 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:23.782 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:32:23.782 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:23.763179+0000", 2024-09-16T09:32:23.782 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.2234000000000001e-05 2024-09-16T09:32:23.782 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:23.782 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:23.782 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:32:23.782 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:23.763304+0000", 2024-09-16T09:32:23.782 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00012454999999999999 2024-09-16T09:32:23.782 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:23.782 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:23.782 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 2,5", 2024-09-16T09:32:23.782 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:23.763429+0000", 2024-09-16T09:32:23.782 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00012527600000000001 2024-09-16T09:32:23.782 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:32:23.783 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:32:23.783 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:32:23.783 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:32:23.783 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-16T09:32:23.783 INFO:teuthology.orchestra.run.smithi070.stdout: "num_ops": 3 2024-09-16T09:32:23.783 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T09:32:23.784 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:23.784 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:23.784 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:32:23.885 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:32:23.922 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:23.923 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:23.924 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:32:24.024 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:32:24.059 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:24.060 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:24.060 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:32:24.126 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:32:24.129 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd unset noscrub 2024-09-16T09:32:24.161 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:32:24.189 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:24.190 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:24.190 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:32:24.291 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:32:24.328 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:24.329 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:24.329 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:32:24.423 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:24 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:32:24.425 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:24 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:32:24.430 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:32:24.458 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:24.458 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:24.459 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:32:24.559 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:32:24.598 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:24.598 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:24.599 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:32:24.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:24 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:32:24.699 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:32:24.750 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:24.751 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:24.751 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:32:24.852 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:32:24.882 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:24.883 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:24.883 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:32:24.984 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:32:25.016 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:25.016 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:25.016 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:32:25.076 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:32:25.076 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:32:25.077 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:32:25.077 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:32:25.077 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:32:25.077 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:32:25.077 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:32:25.077 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:32:25.077 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:32:25.077 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:32:25.097 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:32:25.097 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:32:25.117 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:32:25.145 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:32:25.145 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:32:25.150 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:25.151 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:25.151 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:32:25.179 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:32:25.179 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:32:25.252 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:32:25.291 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:25.292 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:25.292 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:32:25.393 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:32:25.420 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:25.420 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:25.421 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:32:25.467 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:32:25.467 INFO:teuthology.orchestra.run.smithi070.stdout:{"fsid":"a1321834-740c-11ef-bceb-c7b262605968","health":{"status":"HEALTH_WARN","checks":{"OSDMAP_FLAGS":{"severity":"HEALTH_WARN","summary":{"message":"noscrub,nodeep-scrub flag(s) set","count":20},"muted":false},"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 2 pgs undersized","count":2},"muted":false}},"mutes":[]},"election_epoch":14,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":692,"monmap":{"epoch":3,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":208,"num_osds":8,"num_up_osds":8,"osd_up_since":1726478691,"num_in_osds":8,"osd_in_since":1726478668,"num_remapped_pgs":2},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":166},{"state_name":"active+recovering+undersized+remapped","count":2}],"num_pgs":168,"num_pools":7,"num_objects":61445,"data_bytes":4014499882,"bytes_used":14746492928,"bytes_avail":753213792256,"bytes_total":767960285184,"misplaced_objects":2526,"misplaced_total":184335,"misplaced_ratio":0.013703311904955651,"recovering_objects_per_sec":97,"recovering_bytes_per_sec":6394895,"recovering_keys_per_sec":0,"num_objects_recovered":781,"num_bytes_recovered":51183616,"num_keys_recovered":0,"read_bytes_sec":1279,"write_bytes_sec":0,"read_op_per_sec":1,"write_op_per_sec":1471},"fsmap":{"epoch":1,"btime":"2024-09-16T09:19:14:121324+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":1,"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"services":{"dashboard":"https://172.21.15.70:8443/","prometheus":"http://172.21.15.70:9283/"}},"servicemap":{"epoch":14,"modified":"2024-09-16T09:26:33.498407+0000","services":{"rgw":{"daemons":{"summary":"","14475":{"start_epoch":12,"start_stamp":"2024-09-16T09:25:08.794234+0000","gid":14475,"addr":"172.21.15.70:0/1049757559","metadata":{"arch":"x86_64","ceph_release":"squid","ceph_version":"ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev)","ceph_version_short":"19.3.0-4917-g8293d73f","container_hostname":"smithi070","container_image":"quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce","cpu":"Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz","distro":"centos","distro_description":"CentOS Stream 9","distro_version":"9","frontend_config#0":"beast port=80","frontend_type#0":"beast","hostname":"smithi070","id":"foo.a","kernel_description":"#1 SMP PREEMPT_DYNAMIC Thu Sep 5 07:54:07 UTC 2024","kernel_version":"5.14.0-505.el9.x86_64","mem_swap_kb":"0","mem_total_kb":"32487212","num_handles":"1","os":"Linux","pid":"7","realm_id":"","realm_name":"","zone_id":"09633d1c-04b1-424a-bf26-5fa0f61f30f8","zone_name":"default","zonegroup_id":"c6682a31-37e4-465e-b890-750b49769dfa","zonegroup_name":"default"},"task_status":{}}}}}},"progress_events":{}} 2024-09-16T09:32:25.522 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:32:25.553 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:25.554 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:25.554 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:32:25.651 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:25 smithi070 ceph-mon[30388]: pgmap v344: 168 pgs: 2 active+recovering+undersized+remapped, 166 active+clean; 3.7 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.47k op/s; 2526/184335 objects misplaced (1.370%); 6.1 MiB/s, 97 objects/s recovering 2024-09-16T09:32:25.651 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:25 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/863154568' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:32:25.652 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:25 smithi070 ceph-mon[40691]: pgmap v344: 168 pgs: 2 active+recovering+undersized+remapped, 166 active+clean; 3.7 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.47k op/s; 2526/184335 objects misplaced (1.370%); 6.1 MiB/s, 97 objects/s recovering 2024-09-16T09:32:25.652 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:25 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/863154568' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:32:25.655 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:32:25.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:25 smithi084 ceph-mon[32629]: pgmap v344: 168 pgs: 2 active+recovering+undersized+remapped, 166 active+clean; 3.7 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.47k op/s; 2526/184335 objects misplaced (1.370%); 6.1 MiB/s, 97 objects/s recovering 2024-09-16T09:32:25.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:25 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/863154568' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:32:25.685 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:25.686 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:25.686 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:32:25.703 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.1 dump_blocked_ops 2024-09-16T09:32:25.787 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:32:25.833 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:25.834 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:25.834 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:32:25.935 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:32:25.977 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:25.978 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:25.978 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:32:26.079 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:32:26.114 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:26.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:26.115 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:32:26.216 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:32:26.244 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:26.245 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:26.245 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:32:26.346 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:32:26.369 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:26.369 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:26.369 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:32:26.470 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:32:26.496 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:26.497 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:26.498 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:32:26.599 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:32:26.630 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:26.631 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:26.631 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:32:26.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:26 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1865347652' entity='client.admin' cmd={"prefix": "status", "format": "json"} : dispatch 2024-09-16T09:32:26.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:26 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1865347652' entity='client.admin' cmd={"prefix": "status", "format": "json"} : dispatch 2024-09-16T09:32:26.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:26 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1865347652' entity='client.admin' cmd={"prefix": "status", "format": "json"} : dispatch 2024-09-16T09:32:26.733 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:32:26.770 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:26.771 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:26.772 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:32:26.873 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:32:26.895 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:26.896 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:26.896 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:32:26.997 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:32:27.024 INFO:tasks.cephadm.ceph_manager.ceph:making progress, resetting timeout 2024-09-16T09:32:27.025 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-16T09:32:27.026 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:27.026 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:27.027 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:32:27.128 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:32:27.184 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:27.184 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:27.185 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:32:27.285 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:32:27.292 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:27 smithi070 ceph-mon[30388]: pgmap v345: 168 pgs: 2 active+recovering+undersized+remapped, 166 active+clean; 3.7 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.29k op/s; 2526/184335 objects misplaced (1.370%); 5.3 MiB/s, 85 objects/s recovering 2024-09-16T09:32:27.333 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:27.334 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:27.334 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:32:27.435 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:32:27.477 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:27.477 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:27.478 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:32:27.578 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:32:27.616 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:27.617 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:27.617 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:32:27.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:27 smithi084 ceph-mon[32629]: pgmap v345: 168 pgs: 2 active+recovering+undersized+remapped, 166 active+clean; 3.7 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.29k op/s; 2526/184335 objects misplaced (1.370%); 5.3 MiB/s, 85 objects/s recovering 2024-09-16T09:32:27.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:27 smithi070 ceph-mon[40691]: pgmap v345: 168 pgs: 2 active+recovering+undersized+remapped, 166 active+clean; 3.7 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.29k op/s; 2526/184335 objects misplaced (1.370%); 5.3 MiB/s, 85 objects/s recovering 2024-09-16T09:32:27.718 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:32:27.748 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:27.748 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:27.748 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:32:27.817 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:32:27.849 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:32:27.850 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:32:27.872 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:32:27.915 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:27.915 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:27.916 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:32:28.017 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:32:28.046 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:28.047 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:28.047 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:32:28.148 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:32:28.186 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:28.187 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:28.187 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:32:28.288 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:32:28.322 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:28.323 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:28.323 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:32:28.424 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:32:28.460 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:28.461 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:28.461 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:32:28.562 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:32:28.605 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:28.606 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:28.606 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:32:28.707 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:32:28.739 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:28.739 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:28.740 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:32:28.840 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:32:28.882 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:28.882 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:28.882 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:32:28.983 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:32:29.010 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:29.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:29.011 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:32:29.112 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:32:29.160 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:29.160 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:29.161 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:32:29.261 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:32:29.272 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:32:29.275 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2024-09-16T09:32:29.310 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:29.311 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:29.311 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:32:29.318 INFO:teuthology.orchestra.run.smithi070.stderr:noscrub is unset 2024-09-16T09:32:29.412 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:32:29.448 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:29.448 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:29.449 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:32:29.524 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:29 smithi070 ceph-mon[30388]: pgmap v346: 168 pgs: 2 active+recovering+undersized+remapped, 166 active+clean; 3.7 GiB data, 14 GiB used, 702 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.18k op/s; 2526/182103 objects misplaced (1.387%); 4.9 MiB/s, 78 objects/s recovering 2024-09-16T09:32:29.524 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:29 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:32:29.524 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:29 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:32:29.524 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:29 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3272570139' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:32:29.524 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:29 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:32:29.524 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:29 smithi070 ceph-mon[40691]: pgmap v346: 168 pgs: 2 active+recovering+undersized+remapped, 166 active+clean; 3.7 GiB data, 14 GiB used, 702 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.18k op/s; 2526/182103 objects misplaced (1.387%); 4.9 MiB/s, 78 objects/s recovering 2024-09-16T09:32:29.524 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:29 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:32:29.524 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:29 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:32:29.524 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:29 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3272570139' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:32:29.525 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:29 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:32:29.549 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:32:29.581 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:29.582 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:29.582 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:32:29.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:29 smithi084 ceph-mon[32629]: pgmap v346: 168 pgs: 2 active+recovering+undersized+remapped, 166 active+clean; 3.7 GiB data, 14 GiB used, 702 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.18k op/s; 2526/182103 objects misplaced (1.387%); 4.9 MiB/s, 78 objects/s recovering 2024-09-16T09:32:29.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:29 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:32:29.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:29 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:32:29.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:29 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3272570139' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:32:29.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:29 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:32:29.683 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:32:29.721 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:29.721 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:29.722 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:32:29.791 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T09:32:29.823 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:32:29.849 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:29.849 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:29.850 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:32:29.950 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:32:29.980 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:29.982 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:29.982 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:32:30.083 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:32:30.124 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T09:32:30.125 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:32:30.135 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:30.135 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:30.135 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:32:30.237 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:32:30.267 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:30.268 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:30.268 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:32:30.369 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:32:30.403 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:30.404 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:30.404 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:32:30.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:30 smithi070 ceph-mon[30388]: from='client.25394 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:32:30.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:30 smithi070 ceph-mon[30388]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:32:30.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:30 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-16T09:32:30.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:30 smithi070 ceph-mon[30388]: osdmap e209: 8 total, 8 up, 8 in 2024-09-16T09:32:30.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:30 smithi070 ceph-mon[40691]: from='client.25394 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:32:30.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:30 smithi070 ceph-mon[40691]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:32:30.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:30 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-16T09:32:30.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:30 smithi070 ceph-mon[40691]: osdmap e209: 8 total, 8 up, 8 in 2024-09-16T09:32:30.505 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:32:30.544 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:30.544 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:30.545 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:32:30.646 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:32:30.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:30 smithi084 ceph-mon[32629]: from='client.25394 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:32:30.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:30 smithi084 ceph-mon[32629]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:32:30.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:30 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-16T09:32:30.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:30 smithi084 ceph-mon[32629]: osdmap e209: 8 total, 8 up, 8 in 2024-09-16T09:32:30.677 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:30.678 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:30.678 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:32:30.779 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:32:30.821 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:30.822 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:30.822 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:32:30.923 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:32:30.962 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:30.962 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:30.963 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:32:31.063 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:32:31.095 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:31.096 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:31.096 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:32:31.197 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:32:31.232 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:31.233 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:31.233 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:32:31.334 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:32:31.363 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:31.364 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:31.364 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:32:31.390 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.1 dump_historic_ops 2024-09-16T09:32:31.465 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:32:31.502 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:31.503 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:31.504 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:32:31.605 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:32:31.632 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:31.632 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:31.633 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:32:31.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:31 smithi084 ceph-mon[32629]: pgmap v348: 168 pgs: 2 active+recovering+undersized+remapped, 166 active+clean; 3.4 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.18k op/s; 2090/166434 objects misplaced (1.256%); 5.2 MiB/s, 83 objects/s recovering 2024-09-16T09:32:31.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:31 smithi084 ceph-mon[32629]: 4.1e scrub starts 2024-09-16T09:32:31.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:31 smithi084 ceph-mon[32629]: 4.1e scrub ok 2024-09-16T09:32:31.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:31 smithi070 ceph-mon[30388]: pgmap v348: 168 pgs: 2 active+recovering+undersized+remapped, 166 active+clean; 3.4 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.18k op/s; 2090/166434 objects misplaced (1.256%); 5.2 MiB/s, 83 objects/s recovering 2024-09-16T09:32:31.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:31 smithi070 ceph-mon[30388]: 4.1e scrub starts 2024-09-16T09:32:31.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:31 smithi070 ceph-mon[30388]: 4.1e scrub ok 2024-09-16T09:32:31.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:31 smithi070 ceph-mon[40691]: pgmap v348: 168 pgs: 2 active+recovering+undersized+remapped, 166 active+clean; 3.4 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.18k op/s; 2090/166434 objects misplaced (1.256%); 5.2 MiB/s, 83 objects/s recovering 2024-09-16T09:32:31.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:31 smithi070 ceph-mon[40691]: 4.1e scrub starts 2024-09-16T09:32:31.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:31 smithi070 ceph-mon[40691]: 4.1e scrub ok 2024-09-16T09:32:31.734 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:32:31.766 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:31.767 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:31.767 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:32:31.868 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:32:31.913 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":346,"stamp":"2024-09-16T09:32:27.635639+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3965741098,"num_objects":60701,"num_object_clones":0,"num_object_copies":182103,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2526,"num_objects_unfound":0,"num_objects_dirty":60701,"num_whiteouts":0,"num_read":1177,"num_read_kb":948,"num_write":189142,"num_write_kb":7974230,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12321,"num_bytes_recovered":799390496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79912,"ondisk_log_size":79912,"up":504,"acting":502,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":506,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":14211112,"kb_used_data":12091560,"kb_used_omap":12,"kb_used_meta":2119347,"kb_avail":735750104,"statfs":{"total":767960285184,"available":753408106496,"internally_reserved":0,"allocated":12381757440,"data_stored":12379312585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":2170211890},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,7,9,4,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-771489792,"num_objects":-11772,"num_object_clones":0,"num_object_copies":-35316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-781,"num_objects_unfound":0,"num_objects_dirty":-11772,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":11772,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":781,"num_bytes_recovered":51183616,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.004706"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":337,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.495220+0000","last_change":"2024-09-16T09:30:03.848439+0000","last_active":"2024-09-16T09:32:18.495220+0000","last_peered":"2024-09-16T09:32:18.495220+0000","last_clean":"2024-09-16T09:32:18.495220+0000","last_became_active":"2024-09-16T09:25:01.709108+0000","last_became_peered":"2024-09-16T09:25:01.709108+0000","last_unstale":"2024-09-16T09:32:18.495220+0000","last_undegraded":"2024-09-16T09:32:18.495220+0000","last_fullsized":"2024-09-16T09:32:18.495220+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:30:03.848354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:30:03.848354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"66'9","reported_seq":341,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.484942+0000","last_change":"2024-09-16T09:28:52.689373+0000","last_active":"2024-09-16T09:32:18.484942+0000","last_peered":"2024-09-16T09:32:18.484942+0000","last_clean":"2024-09-16T09:32:18.484942+0000","last_became_active":"2024-09-16T09:25:03.714635+0000","last_became_peered":"2024-09-16T09:25:03.714635+0000","last_unstale":"2024-09-16T09:32:18.484942+0000","last_undegraded":"2024-09-16T09:32:18.484942+0000","last_fullsized":"2024-09-16T09:32:18.484942+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:28:52.689304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:52.689304+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":328,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.491672+0000","last_change":"2024-09-16T09:29:17.910367+0000","last_active":"2024-09-16T09:32:18.491672+0000","last_peered":"2024-09-16T09:32:18.491672+0000","last_clean":"2024-09-16T09:32:18.491672+0000","last_became_active":"2024-09-16T09:25:05.715645+0000","last_became_peered":"2024-09-16T09:25:05.715645+0000","last_unstale":"2024-09-16T09:32:18.491672+0000","last_undegraded":"2024-09-16T09:32:18.491672+0000","last_fullsized":"2024-09-16T09:32:18.491672+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:17.910305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:17.910305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":308,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.486234+0000","last_change":"2024-09-16T09:27:11.565306+0000","last_active":"2024-09-16T09:32:18.486234+0000","last_peered":"2024-09-16T09:32:18.486234+0000","last_clean":"2024-09-16T09:32:18.486234+0000","last_became_active":"2024-09-16T09:25:07.715442+0000","last_became_peered":"2024-09-16T09:25:07.715442+0000","last_unstale":"2024-09-16T09:32:18.486234+0000","last_undegraded":"2024-09-16T09:32:18.486234+0000","last_fullsized":"2024-09-16T09:32:18.486234+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:11.565220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:11.565220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":54,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.494910+0000","last_change":"2024-09-16T09:31:09.269700+0000","last_active":"2024-09-16T09:32:18.494910+0000","last_peered":"2024-09-16T09:32:18.494910+0000","last_clean":"2024-09-16T09:32:18.494910+0000","last_became_active":"2024-09-16T09:31:09.269342+0000","last_became_peered":"2024-09-16T09:31:09.269342+0000","last_unstale":"2024-09-16T09:32:18.494910+0000","last_undegraded":"2024-09-16T09:32:18.494910+0000","last_fullsized":"2024-09-16T09:32:18.494910+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:00.068570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:28:00.068570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":194,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.498345+0000","last_change":"2024-09-16T09:31:56.507595+0000","last_active":"2024-09-16T09:32:18.498345+0000","last_peered":"2024-09-16T09:32:18.498345+0000","last_clean":"2024-09-16T09:32:18.498345+0000","last_became_active":"2024-09-16T09:31:09.275136+0000","last_became_peered":"2024-09-16T09:31:09.275136+0000","last_unstale":"2024-09-16T09:32:18.498345+0000","last_undegraded":"2024-09-16T09:32:18.498345+0000","last_fullsized":"2024-09-16T09:32:18.498345+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:31:56.507528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:31:56.507528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:08.491873+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.19","version":"66'15","reported_seq":255,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.499043+0000","last_change":"2024-09-16T09:31:09.302903+0000","last_active":"2024-09-16T09:32:18.499043+0000","last_peered":"2024-09-16T09:32:18.499043+0000","last_clean":"2024-09-16T09:32:18.499043+0000","last_became_active":"2024-09-16T09:31:09.268750+0000","last_became_peered":"2024-09-16T09:31:09.268750+0000","last_unstale":"2024-09-16T09:32:18.499043+0000","last_undegraded":"2024-09-16T09:32:18.499043+0000","last_fullsized":"2024-09-16T09:32:18.499043+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-16T09:29:57.968162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:29:57.968162+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":329,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.484889+0000","last_change":"2024-09-16T09:29:15.628676+0000","last_active":"2024-09-16T09:32:18.484889+0000","last_peered":"2024-09-16T09:32:18.484889+0000","last_clean":"2024-09-16T09:32:18.484889+0000","last_became_active":"2024-09-16T09:25:05.717236+0000","last_became_peered":"2024-09-16T09:25:05.717236+0000","last_unstale":"2024-09-16T09:32:18.484889+0000","last_undegraded":"2024-09-16T09:32:18.484889+0000","last_fullsized":"2024-09-16T09:32:18.484889+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:15.628467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:15.628467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":167,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.492091+0000","last_change":"2024-09-16T09:28:34.839909+0000","last_active":"2024-09-16T09:32:18.492091+0000","last_peered":"2024-09-16T09:32:18.492091+0000","last_clean":"2024-09-16T09:32:18.492091+0000","last_became_active":"2024-09-16T09:28:34.839435+0000","last_became_peered":"2024-09-16T09:28:34.839435+0000","last_unstale":"2024-09-16T09:32:18.492091+0000","last_undegraded":"2024-09-16T09:32:18.492091+0000","last_fullsized":"2024-09-16T09:32:18.492091+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:01.038400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:28:01.038400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"66'9","reported_seq":340,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.487911+0000","last_change":"2024-09-16T09:29:28.622647+0000","last_active":"2024-09-16T09:32:18.487911+0000","last_peered":"2024-09-16T09:32:18.487911+0000","last_clean":"2024-09-16T09:32:18.487911+0000","last_became_active":"2024-09-16T09:25:03.709147+0000","last_became_peered":"2024-09-16T09:25:03.709147+0000","last_unstale":"2024-09-16T09:32:18.487911+0000","last_undegraded":"2024-09-16T09:32:18.487911+0000","last_fullsized":"2024-09-16T09:32:18.487911+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:29:28.622558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:29:28.622558+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":311,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.492974+0000","last_change":"2024-09-16T09:27:59.080369+0000","last_active":"2024-09-16T09:32:18.492974+0000","last_peered":"2024-09-16T09:32:18.492974+0000","last_clean":"2024-09-16T09:32:18.492974+0000","last_became_active":"2024-09-16T09:25:05.718134+0000","last_became_peered":"2024-09-16T09:25:05.718134+0000","last_unstale":"2024-09-16T09:32:18.492974+0000","last_undegraded":"2024-09-16T09:32:18.492974+0000","last_fullsized":"2024-09-16T09:32:18.492974+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:59.080307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:59.080307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"66'1","reported_seq":326,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.493247+0000","last_change":"2024-09-16T09:31:58.367903+0000","last_active":"2024-09-16T09:32:18.493247+0000","last_peered":"2024-09-16T09:32:18.493247+0000","last_clean":"2024-09-16T09:32:18.493247+0000","last_became_active":"2024-09-16T09:25:07.716558+0000","last_became_peered":"2024-09-16T09:25:07.716558+0000","last_unstale":"2024-09-16T09:32:18.493247+0000","last_undegraded":"2024-09-16T09:32:18.493247+0000","last_fullsized":"2024-09-16T09:32:18.493247+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-16T09:31:58.367837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:31:58.367837+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:20.121824+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":319,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.493196+0000","last_change":"2024-09-16T09:27:56.191264+0000","last_active":"2024-09-16T09:32:18.493196+0000","last_peered":"2024-09-16T09:32:18.493196+0000","last_clean":"2024-09-16T09:32:18.493196+0000","last_became_active":"2024-09-16T09:25:01.696939+0000","last_became_peered":"2024-09-16T09:25:01.696939+0000","last_unstale":"2024-09-16T09:32:18.493196+0000","last_undegraded":"2024-09-16T09:32:18.493196+0000","last_fullsized":"2024-09-16T09:32:18.493196+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:56.191220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:27:56.191220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"66'5","reported_seq":326,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.485824+0000","last_change":"2024-09-16T09:27:05.694733+0000","last_active":"2024-09-16T09:32:18.485824+0000","last_peered":"2024-09-16T09:32:18.485824+0000","last_clean":"2024-09-16T09:32:18.485824+0000","last_became_active":"2024-09-16T09:25:03.716136+0000","last_became_peered":"2024-09-16T09:25:03.716136+0000","last_unstale":"2024-09-16T09:32:18.485824+0000","last_undegraded":"2024-09-16T09:32:18.485824+0000","last_fullsized":"2024-09-16T09:32:18.485824+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-16T09:27:05.694650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:05.694650+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":319,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.497139+0000","last_change":"2024-09-16T09:28:52.746046+0000","last_active":"2024-09-16T09:32:18.497139+0000","last_peered":"2024-09-16T09:32:18.497139+0000","last_clean":"2024-09-16T09:32:18.497139+0000","last_became_active":"2024-09-16T09:25:05.718281+0000","last_became_peered":"2024-09-16T09:25:05.718281+0000","last_unstale":"2024-09-16T09:32:18.497139+0000","last_undegraded":"2024-09-16T09:32:18.497139+0000","last_fullsized":"2024-09-16T09:32:18.497139+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:52.745999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:28:52.745999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":337,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.492730+0000","last_change":"2024-09-16T09:31:09.276016+0000","last_active":"2024-09-16T09:32:18.492730+0000","last_peered":"2024-09-16T09:32:18.492730+0000","last_clean":"2024-09-16T09:32:18.492730+0000","last_became_active":"2024-09-16T09:31:09.275656+0000","last_became_peered":"2024-09-16T09:31:09.275656+0000","last_unstale":"2024-09-16T09:32:18.492730+0000","last_undegraded":"2024-09-16T09:32:18.492730+0000","last_fullsized":"2024-09-16T09:32:18.492730+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:50.468134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:50.468134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":325,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.485488+0000","last_change":"2024-09-16T09:29:56.605555+0000","last_active":"2024-09-16T09:32:18.485488+0000","last_peered":"2024-09-16T09:32:18.485488+0000","last_clean":"2024-09-16T09:32:18.485488+0000","last_became_active":"2024-09-16T09:25:07.720793+0000","last_became_peered":"2024-09-16T09:25:07.720793+0000","last_unstale":"2024-09-16T09:32:18.485488+0000","last_undegraded":"2024-09-16T09:32:18.485488+0000","last_fullsized":"2024-09-16T09:32:18.485488+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:56.605488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:56.605488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":76,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.497187+0000","last_change":"2024-09-16T09:30:39.202178+0000","last_active":"2024-09-16T09:32:18.497187+0000","last_peered":"2024-09-16T09:32:18.497187+0000","last_clean":"2024-09-16T09:32:18.497187+0000","last_became_active":"2024-09-16T09:30:39.201824+0000","last_became_peered":"2024-09-16T09:30:39.201824+0000","last_unstale":"2024-09-16T09:32:18.497187+0000","last_undegraded":"2024-09-16T09:32:18.497187+0000","last_fullsized":"2024-09-16T09:32:18.497187+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:50.912979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:50.912979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:20.431823+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"66'15","reported_seq":403,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.500216+0000","last_change":"2024-09-16T09:31:09.302018+0000","last_active":"2024-09-16T09:32:18.500216+0000","last_peered":"2024-09-16T09:32:18.500216+0000","last_clean":"2024-09-16T09:32:18.500216+0000","last_became_active":"2024-09-16T09:31:09.263534+0000","last_became_peered":"2024-09-16T09:31:09.263534+0000","last_unstale":"2024-09-16T09:32:18.500216+0000","last_undegraded":"2024-09-16T09:32:18.500216+0000","last_fullsized":"2024-09-16T09:32:18.500216+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-16T09:30:21.932604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:30:21.932604+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":320,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.492433+0000","last_change":"2024-09-16T09:29:55.875536+0000","last_active":"2024-09-16T09:32:18.492433+0000","last_peered":"2024-09-16T09:32:18.492433+0000","last_clean":"2024-09-16T09:32:18.492433+0000","last_became_active":"2024-09-16T09:25:05.713929+0000","last_became_peered":"2024-09-16T09:25:05.713929+0000","last_unstale":"2024-09-16T09:32:18.492433+0000","last_undegraded":"2024-09-16T09:32:18.492433+0000","last_fullsized":"2024-09-16T09:32:18.492433+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:55.875480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:55.875480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":173,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.498498+0000","last_change":"2024-09-16T09:30:24.412733+0000","last_active":"2024-09-16T09:32:18.498498+0000","last_peered":"2024-09-16T09:32:18.498498+0000","last_clean":"2024-09-16T09:32:18.498498+0000","last_became_active":"2024-09-16T09:28:52.974255+0000","last_became_peered":"2024-09-16T09:28:52.974255+0000","last_unstale":"2024-09-16T09:32:18.498498+0000","last_undegraded":"2024-09-16T09:32:18.498498+0000","last_fullsized":"2024-09-16T09:32:18.498498+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:30:24.412571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:30:24.412571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:49.424390+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":336,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.488061+0000","last_change":"2024-09-16T09:30:03.628882+0000","last_active":"2024-09-16T09:32:18.488061+0000","last_peered":"2024-09-16T09:32:18.488061+0000","last_clean":"2024-09-16T09:32:18.488061+0000","last_became_active":"2024-09-16T09:25:01.706491+0000","last_became_peered":"2024-09-16T09:25:01.706491+0000","last_unstale":"2024-09-16T09:32:18.488061+0000","last_undegraded":"2024-09-16T09:32:18.488061+0000","last_fullsized":"2024-09-16T09:32:18.488061+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:30:03.628812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:30:03.628812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"66'12","reported_seq":237,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.496025+0000","last_change":"2024-09-16T09:31:09.286673+0000","last_active":"2024-09-16T09:32:18.496025+0000","last_peered":"2024-09-16T09:32:18.496025+0000","last_clean":"2024-09-16T09:32:18.496025+0000","last_became_active":"2024-09-16T09:31:09.268588+0000","last_became_peered":"2024-09-16T09:31:09.268588+0000","last_unstale":"2024-09-16T09:32:18.496025+0000","last_undegraded":"2024-09-16T09:32:18.496025+0000","last_fullsized":"2024-09-16T09:32:18.496025+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-16T09:27:01.700139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:01.700139+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":24,"num_read_kb":15,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":312,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.485992+0000","last_change":"2024-09-16T09:27:00.816500+0000","last_active":"2024-09-16T09:32:18.485992+0000","last_peered":"2024-09-16T09:32:18.485992+0000","last_clean":"2024-09-16T09:32:18.485992+0000","last_became_active":"2024-09-16T09:25:05.710093+0000","last_became_peered":"2024-09-16T09:25:05.710093+0000","last_unstale":"2024-09-16T09:32:18.485992+0000","last_undegraded":"2024-09-16T09:32:18.485992+0000","last_fullsized":"2024-09-16T09:32:18.485992+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:00.816424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:00.816424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":316,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.497396+0000","last_change":"2024-09-16T09:29:29.918433+0000","last_active":"2024-09-16T09:32:18.497396+0000","last_peered":"2024-09-16T09:32:18.497396+0000","last_clean":"2024-09-16T09:32:18.497396+0000","last_became_active":"2024-09-16T09:25:07.719380+0000","last_became_peered":"2024-09-16T09:25:07.719380+0000","last_unstale":"2024-09-16T09:32:18.497396+0000","last_undegraded":"2024-09-16T09:32:18.497396+0000","last_fullsized":"2024-09-16T09:32:18.497396+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-16T09:29:29.918368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:29.918368+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":338,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.491836+0000","last_change":"2024-09-16T09:30:02.843932+0000","last_active":"2024-09-16T09:32:18.491836+0000","last_peered":"2024-09-16T09:32:18.491836+0000","last_clean":"2024-09-16T09:32:18.491836+0000","last_became_active":"2024-09-16T09:25:01.699545+0000","last_became_peered":"2024-09-16T09:25:01.699545+0000","last_unstale":"2024-09-16T09:32:18.491836+0000","last_undegraded":"2024-09-16T09:32:18.491836+0000","last_fullsized":"2024-09-16T09:32:18.491836+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:30:02.843856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:30:02.843856+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"66'10","reported_seq":378,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.494664+0000","last_change":"2024-09-16T09:31:09.328513+0000","last_active":"2024-09-16T09:32:18.494664+0000","last_peered":"2024-09-16T09:32:18.494664+0000","last_clean":"2024-09-16T09:32:18.494664+0000","last_became_active":"2024-09-16T09:31:09.275050+0000","last_became_peered":"2024-09-16T09:31:09.275050+0000","last_unstale":"2024-09-16T09:32:18.494664+0000","last_undegraded":"2024-09-16T09:32:18.494664+0000","last_fullsized":"2024-09-16T09:32:18.494664+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-16T09:27:43.556357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:43.556357+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"66'8","reported_seq":438,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:21.848528+0000","last_change":"2024-09-16T09:30:31.392771+0000","last_active":"2024-09-16T09:32:21.848528+0000","last_peered":"2024-09-16T09:32:21.848528+0000","last_clean":"2024-09-16T09:32:21.848528+0000","last_became_active":"2024-09-16T09:25:05.712516+0000","last_became_peered":"2024-09-16T09:25:05.712516+0000","last_unstale":"2024-09-16T09:32:21.848528+0000","last_undegraded":"2024-09-16T09:32:21.848528+0000","last_fullsized":"2024-09-16T09:32:21.848528+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:30:31.392399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:30:31.392399+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:44.469073+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"208'11832","reported_seq":11956,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:22.920666+0000","last_change":"2024-09-16T09:29:49.449011+0000","last_active":"2024-09-16T09:32:22.920666+0000","last_peered":"2024-09-16T09:32:22.920666+0000","last_clean":"2024-09-16T09:32:22.920666+0000","last_became_active":"2024-09-16T09:29:49.448473+0000","last_became_peered":"2024-09-16T09:29:49.448473+0000","last_unstale":"2024-09-16T09:32:22.920666+0000","last_undegraded":"2024-09-16T09:32:22.920666+0000","last_fullsized":"2024-09-16T09:32:22.920666+0000","mapping_epoch":152,"log_start":"190'7300","ondisk_log_start":"190'7300","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":4532,"log_dups_size":0,"ondisk_log_size":4532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":248119296,"num_objects":3786,"num_object_clones":0,"num_object_copies":11358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3786,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11832,"num_write_kb":499776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.a","version":"73'79","reported_seq":259,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.492636+0000","last_change":"2024-09-16T09:29:18.498887+0000","last_active":"2024-09-16T09:32:18.492636+0000","last_peered":"2024-09-16T09:32:18.492636+0000","last_clean":"2024-09-16T09:32:18.492636+0000","last_became_active":"2024-09-16T09:26:35.127352+0000","last_became_peered":"2024-09-16T09:26:35.127352+0000","last_unstale":"2024-09-16T09:32:18.492636+0000","last_undegraded":"2024-09-16T09:32:18.492636+0000","last_fullsized":"2024-09-16T09:32:18.492636+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:18.498766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:18.498766+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"66'15","reported_seq":268,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.497901+0000","last_change":"2024-09-16T09:28:28.065781+0000","last_active":"2024-09-16T09:32:18.497901+0000","last_peered":"2024-09-16T09:32:18.497901+0000","last_clean":"2024-09-16T09:32:18.497901+0000","last_became_active":"2024-09-16T09:26:33.295672+0000","last_became_peered":"2024-09-16T09:26:33.295672+0000","last_unstale":"2024-09-16T09:32:18.497901+0000","last_undegraded":"2024-09-16T09:32:18.497901+0000","last_fullsized":"2024-09-16T09:32:18.497901+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-16T09:28:28.065699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:28.065699+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":54,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.489268+0000","last_change":"2024-09-16T09:31:09.276858+0000","last_active":"2024-09-16T09:32:18.489268+0000","last_peered":"2024-09-16T09:32:18.489268+0000","last_clean":"2024-09-16T09:32:18.489268+0000","last_became_active":"2024-09-16T09:31:09.274534+0000","last_became_peered":"2024-09-16T09:31:09.274534+0000","last_unstale":"2024-09-16T09:32:18.489268+0000","last_undegraded":"2024-09-16T09:32:18.489268+0000","last_fullsized":"2024-09-16T09:32:18.489268+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:22.091537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:28:22.091537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"66'8","reported_seq":402,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.485607+0000","last_change":"2024-09-16T09:25:05.714128+0000","last_active":"2024-09-16T09:32:18.485607+0000","last_peered":"2024-09-16T09:32:18.485607+0000","last_clean":"2024-09-16T09:32:18.485607+0000","last_became_active":"2024-09-16T09:25:05.713813+0000","last_became_peered":"2024-09-16T09:25:05.713813+0000","last_unstale":"2024-09-16T09:32:18.485607+0000","last_undegraded":"2024-09-16T09:32:18.485607+0000","last_fullsized":"2024-09-16T09:32:18.485607+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:17.970742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":316,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.493271+0000","last_change":"2024-09-16T09:29:24.537802+0000","last_active":"2024-09-16T09:32:18.493271+0000","last_peered":"2024-09-16T09:32:18.493271+0000","last_clean":"2024-09-16T09:32:18.493271+0000","last_became_active":"2024-09-16T09:25:07.717344+0000","last_became_peered":"2024-09-16T09:25:07.717344+0000","last_unstale":"2024-09-16T09:32:18.493271+0000","last_undegraded":"2024-09-16T09:32:18.493271+0000","last_fullsized":"2024-09-16T09:32:18.493271+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:24.537731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:24.537731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.c","version":"208'11753","reported_seq":11935,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:23.154566+0000","last_change":"2024-09-16T09:32:07.676294+0000","last_active":"2024-09-16T09:32:23.154566+0000","last_peered":"2024-09-16T09:32:23.154566+0000","last_clean":"2024-09-16T09:32:23.154566+0000","last_became_active":"2024-09-16T09:29:50.007935+0000","last_became_peered":"2024-09-16T09:29:50.007935+0000","last_unstale":"2024-09-16T09:32:23.154566+0000","last_undegraded":"2024-09-16T09:32:23.154566+0000","last_fullsized":"2024-09-16T09:32:23.154566+0000","mapping_epoch":152,"log_start":"184'6600","ondisk_log_start":"184'6600","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"204'10667","last_scrub_stamp":"2024-09-16T09:32:07.676020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:32:07.676020+0000","objects_scrubbed":4920,"log_size":5153,"log_dups_size":0,"ondisk_log_size":5153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:18.894513+0000","scrub_duration":1433,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":248709120,"num_objects":3795,"num_object_clones":0,"num_object_copies":11385,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3795,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11753,"num_write_kb":497536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"73'79","reported_seq":291,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.486224+0000","last_change":"2024-09-16T09:32:13.446353+0000","last_active":"2024-09-16T09:32:18.486224+0000","last_peered":"2024-09-16T09:32:18.486224+0000","last_clean":"2024-09-16T09:32:18.486224+0000","last_became_active":"2024-09-16T09:30:08.763683+0000","last_became_peered":"2024-09-16T09:30:08.763683+0000","last_unstale":"2024-09-16T09:32:18.486224+0000","last_undegraded":"2024-09-16T09:32:18.486224+0000","last_fullsized":"2024-09-16T09:32:18.486224+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:58.568841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:58.568841+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:32:41.868241+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"66'18","reported_seq":266,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.501318+0000","last_change":"2024-09-16T09:31:09.407700+0000","last_active":"2024-09-16T09:32:18.501318+0000","last_peered":"2024-09-16T09:32:18.501318+0000","last_clean":"2024-09-16T09:32:18.501318+0000","last_became_active":"2024-09-16T09:31:09.270434+0000","last_became_peered":"2024-09-16T09:31:09.270434+0000","last_unstale":"2024-09-16T09:32:18.501318+0000","last_undegraded":"2024-09-16T09:32:18.501318+0000","last_fullsized":"2024-09-16T09:32:18.501318+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'18","last_scrub_stamp":"2024-09-16T09:28:54.699778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:54.699778+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":33,"num_read_kb":21,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":164,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.497728+0000","last_change":"2024-09-16T09:28:58.705169+0000","last_active":"2024-09-16T09:32:18.497728+0000","last_peered":"2024-09-16T09:32:18.497728+0000","last_clean":"2024-09-16T09:32:18.497728+0000","last_became_active":"2024-09-16T09:28:52.983543+0000","last_became_peered":"2024-09-16T09:28:52.983543+0000","last_unstale":"2024-09-16T09:32:18.497728+0000","last_undegraded":"2024-09-16T09:32:18.497728+0000","last_fullsized":"2024-09-16T09:32:18.497728+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:58.705097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:28:58.705097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":343,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.485133+0000","last_change":"2024-09-16T09:31:09.269234+0000","last_active":"2024-09-16T09:32:18.485133+0000","last_peered":"2024-09-16T09:32:18.485133+0000","last_clean":"2024-09-16T09:32:18.485133+0000","last_became_active":"2024-09-16T09:31:09.268864+0000","last_became_peered":"2024-09-16T09:31:09.268864+0000","last_unstale":"2024-09-16T09:32:18.485133+0000","last_undegraded":"2024-09-16T09:32:18.485133+0000","last_fullsized":"2024-09-16T09:32:18.485133+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:50.482240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:50.482240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":323,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.487909+0000","last_change":"2024-09-16T09:31:09.269842+0000","last_active":"2024-09-16T09:32:18.487909+0000","last_peered":"2024-09-16T09:32:18.487909+0000","last_clean":"2024-09-16T09:32:18.487909+0000","last_became_active":"2024-09-16T09:31:09.269433+0000","last_became_peered":"2024-09-16T09:31:09.269433+0000","last_unstale":"2024-09-16T09:32:18.487909+0000","last_undegraded":"2024-09-16T09:32:18.487909+0000","last_fullsized":"2024-09-16T09:32:18.487909+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:13.548608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:13.548608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.d","version":"208'11745","reported_seq":11870,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:22.950781+0000","last_change":"2024-09-16T09:29:50.010860+0000","last_active":"2024-09-16T09:32:22.950781+0000","last_peered":"2024-09-16T09:32:22.950781+0000","last_clean":"2024-09-16T09:32:22.950781+0000","last_became_active":"2024-09-16T09:29:50.010421+0000","last_became_peered":"2024-09-16T09:29:50.010421+0000","last_unstale":"2024-09-16T09:32:22.950781+0000","last_undegraded":"2024-09-16T09:32:22.950781+0000","last_fullsized":"2024-09-16T09:32:22.950781+0000","mapping_epoch":152,"log_start":"188'7000","ondisk_log_start":"188'7000","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":4745,"log_dups_size":0,"ondisk_log_size":4745,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":250347544,"num_objects":3821,"num_object_clones":0,"num_object_copies":11463,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3821,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11745,"num_write_kb":498049,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,2],"acting":[5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.4","version":"73'79","reported_seq":54,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.492811+0000","last_change":"2024-09-16T09:31:09.275499+0000","last_active":"2024-09-16T09:32:18.492811+0000","last_peered":"2024-09-16T09:32:18.492811+0000","last_clean":"2024-09-16T09:32:18.492811+0000","last_became_active":"2024-09-16T09:31:09.275139+0000","last_became_peered":"2024-09-16T09:31:09.275139+0000","last_unstale":"2024-09-16T09:32:18.492811+0000","last_undegraded":"2024-09-16T09:32:18.492811+0000","last_fullsized":"2024-09-16T09:32:18.492811+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:30:22.438378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:30:22.438378+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"66'14","reported_seq":346,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.485207+0000","last_change":"2024-09-16T09:28:32.595064+0000","last_active":"2024-09-16T09:32:18.485207+0000","last_peered":"2024-09-16T09:32:18.485207+0000","last_clean":"2024-09-16T09:32:18.485207+0000","last_became_active":"2024-09-16T09:25:03.708830+0000","last_became_peered":"2024-09-16T09:25:03.708830+0000","last_unstale":"2024-09-16T09:32:18.485207+0000","last_undegraded":"2024-09-16T09:32:18.485207+0000","last_fullsized":"2024-09-16T09:32:18.485207+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'14","last_scrub_stamp":"2024-09-16T09:28:32.594988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:32.594988+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":346,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.494927+0000","last_change":"2024-09-16T09:31:57.373606+0000","last_active":"2024-09-16T09:32:18.494927+0000","last_peered":"2024-09-16T09:32:18.494927+0000","last_clean":"2024-09-16T09:32:18.494927+0000","last_became_active":"2024-09-16T09:25:01.702992+0000","last_became_peered":"2024-09-16T09:25:01.702992+0000","last_unstale":"2024-09-16T09:32:18.494927+0000","last_undegraded":"2024-09-16T09:32:18.494927+0000","last_fullsized":"2024-09-16T09:32:18.494927+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:31:57.373544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:31:57.373544+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:24.193683+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"187'11","reported_seq":254,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.496081+0000","last_change":"2024-09-16T09:31:09.397758+0000","last_active":"2024-09-16T09:32:18.496081+0000","last_peered":"2024-09-16T09:32:18.496081+0000","last_clean":"2024-09-16T09:32:18.496081+0000","last_became_active":"2024-09-16T09:31:09.269173+0000","last_became_peered":"2024-09-16T09:31:09.269173+0000","last_unstale":"2024-09-16T09:32:18.496081+0000","last_undegraded":"2024-09-16T09:32:18.496081+0000","last_fullsized":"2024-09-16T09:32:18.496081+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'9","last_scrub_stamp":"2024-09-16T09:29:21.898533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:21.898533+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":308,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.496845+0000","last_change":"2024-09-16T09:26:40.699888+0000","last_active":"2024-09-16T09:32:18.496845+0000","last_peered":"2024-09-16T09:32:18.496845+0000","last_clean":"2024-09-16T09:32:18.496845+0000","last_became_active":"2024-09-16T09:25:07.722017+0000","last_became_peered":"2024-09-16T09:25:07.722017+0000","last_unstale":"2024-09-16T09:32:18.496845+0000","last_undegraded":"2024-09-16T09:32:18.496845+0000","last_fullsized":"2024-09-16T09:32:18.496845+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:40.699813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:40.699813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.e","version":"208'11833","reported_seq":11957,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:23.148556+0000","last_change":"2024-09-16T09:29:49.448081+0000","last_active":"2024-09-16T09:32:23.148556+0000","last_peered":"2024-09-16T09:32:23.148556+0000","last_clean":"2024-09-16T09:32:23.148556+0000","last_became_active":"2024-09-16T09:29:49.447767+0000","last_became_peered":"2024-09-16T09:29:49.447767+0000","last_unstale":"2024-09-16T09:32:23.148556+0000","last_undegraded":"2024-09-16T09:32:23.148556+0000","last_fullsized":"2024-09-16T09:32:23.148556+0000","mapping_epoch":152,"log_start":"187'6700","ondisk_log_start":"187'6700","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":5133,"log_dups_size":0,"ondisk_log_size":5133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":247267328,"num_objects":3773,"num_object_clones":0,"num_object_copies":11319,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3773,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11833,"num_write_kb":499392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"73'79","reported_seq":377,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.498459+0000","last_change":"2024-09-16T09:31:09.275413+0000","last_active":"2024-09-16T09:32:18.498459+0000","last_peered":"2024-09-16T09:32:18.498459+0000","last_clean":"2024-09-16T09:32:18.498459+0000","last_became_active":"2024-09-16T09:31:09.274707+0000","last_became_peered":"2024-09-16T09:31:09.274707+0000","last_unstale":"2024-09-16T09:32:18.498459+0000","last_undegraded":"2024-09-16T09:32:18.498459+0000","last_fullsized":"2024-09-16T09:32:18.498459+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:30:23.455388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:30:23.455388+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:41.421930+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"66'10","reported_seq":348,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.498493+0000","last_change":"2024-09-16T09:30:04.848306+0000","last_active":"2024-09-16T09:32:18.498493+0000","last_peered":"2024-09-16T09:32:18.498493+0000","last_clean":"2024-09-16T09:32:18.498493+0000","last_became_active":"2024-09-16T09:25:03.712985+0000","last_became_peered":"2024-09-16T09:25:03.712985+0000","last_unstale":"2024-09-16T09:32:18.498493+0000","last_undegraded":"2024-09-16T09:32:18.498493+0000","last_fullsized":"2024-09-16T09:32:18.498493+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-16T09:30:04.848234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:30:04.848234+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":334,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.493494+0000","last_change":"2024-09-16T09:31:09.272049+0000","last_active":"2024-09-16T09:32:18.493494+0000","last_peered":"2024-09-16T09:32:18.493494+0000","last_clean":"2024-09-16T09:32:18.493494+0000","last_became_active":"2024-09-16T09:31:09.270619+0000","last_became_peered":"2024-09-16T09:31:09.270619+0000","last_unstale":"2024-09-16T09:32:18.493494+0000","last_undegraded":"2024-09-16T09:32:18.493494+0000","last_fullsized":"2024-09-16T09:32:18.493494+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:25.086322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:25.086322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"66'8","reported_seq":418,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.494228+0000","last_change":"2024-09-16T09:28:57.628615+0000","last_active":"2024-09-16T09:32:18.494228+0000","last_peered":"2024-09-16T09:32:18.494228+0000","last_clean":"2024-09-16T09:32:18.494228+0000","last_became_active":"2024-09-16T09:25:05.714507+0000","last_became_peered":"2024-09-16T09:25:05.714507+0000","last_unstale":"2024-09-16T09:32:18.494228+0000","last_undegraded":"2024-09-16T09:32:18.494228+0000","last_fullsized":"2024-09-16T09:32:18.494228+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:28:57.628543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:28:57.628543+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:51.253847+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":331,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.493346+0000","last_change":"2024-09-16T09:31:09.283816+0000","last_active":"2024-09-16T09:32:18.493346+0000","last_peered":"2024-09-16T09:32:18.493346+0000","last_clean":"2024-09-16T09:32:18.493346+0000","last_became_active":"2024-09-16T09:31:09.283192+0000","last_became_peered":"2024-09-16T09:31:09.283192+0000","last_unstale":"2024-09-16T09:32:18.493346+0000","last_undegraded":"2024-09-16T09:32:18.493346+0000","last_fullsized":"2024-09-16T09:32:18.493346+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:22.563395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:28:22.563395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:22.152175+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.f","version":"208'11788","reported_seq":11912,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:22.824570+0000","last_change":"2024-09-16T09:29:49.838498+0000","last_active":"2024-09-16T09:32:22.824570+0000","last_peered":"2024-09-16T09:32:22.824570+0000","last_clean":"2024-09-16T09:32:22.824570+0000","last_became_active":"2024-09-16T09:29:49.838146+0000","last_became_peered":"2024-09-16T09:29:49.838146+0000","last_unstale":"2024-09-16T09:32:22.824570+0000","last_undegraded":"2024-09-16T09:32:22.824570+0000","last_fullsized":"2024-09-16T09:32:22.824570+0000","mapping_epoch":152,"log_start":"180'5700","ondisk_log_start":"180'5700","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":6088,"log_dups_size":0,"ondisk_log_size":6088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":259391488,"num_objects":3958,"num_object_clones":0,"num_object_copies":11874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3958,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11788,"num_write_kb":503872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"73'79","reported_seq":358,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.498142+0000","last_change":"2024-09-16T09:31:09.274975+0000","last_active":"2024-09-16T09:32:18.498142+0000","last_peered":"2024-09-16T09:32:18.498142+0000","last_clean":"2024-09-16T09:32:18.498142+0000","last_became_active":"2024-09-16T09:31:09.274565+0000","last_became_peered":"2024-09-16T09:31:09.274565+0000","last_unstale":"2024-09-16T09:32:18.498142+0000","last_undegraded":"2024-09-16T09:32:18.498142+0000","last_fullsized":"2024-09-16T09:32:18.498142+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:30:22.415051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:30:22.415051+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:52.385618+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"66'19","reported_seq":361,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.494425+0000","last_change":"2024-09-16T09:31:35.249952+0000","last_active":"2024-09-16T09:32:18.494425+0000","last_peered":"2024-09-16T09:32:18.494425+0000","last_clean":"2024-09-16T09:32:18.494425+0000","last_became_active":"2024-09-16T09:25:03.714797+0000","last_became_peered":"2024-09-16T09:25:03.714797+0000","last_unstale":"2024-09-16T09:32:18.494425+0000","last_undegraded":"2024-09-16T09:32:18.494425+0000","last_fullsized":"2024-09-16T09:32:18.494425+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-16T09:31:35.249864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:31:35.249864+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:40.632025+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":337,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.491994+0000","last_change":"2024-09-16T09:29:16.888357+0000","last_active":"2024-09-16T09:32:18.491994+0000","last_peered":"2024-09-16T09:32:18.491994+0000","last_clean":"2024-09-16T09:32:18.491994+0000","last_became_active":"2024-09-16T09:25:01.706806+0000","last_became_peered":"2024-09-16T09:25:01.706806+0000","last_unstale":"2024-09-16T09:32:18.491994+0000","last_undegraded":"2024-09-16T09:32:18.491994+0000","last_fullsized":"2024-09-16T09:32:18.491994+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:16.888236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:16.888236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":337,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.497387+0000","last_change":"2024-09-16T09:31:55.456669+0000","last_active":"2024-09-16T09:32:18.497387+0000","last_peered":"2024-09-16T09:32:18.497387+0000","last_clean":"2024-09-16T09:32:18.497387+0000","last_became_active":"2024-09-16T09:25:05.708663+0000","last_became_peered":"2024-09-16T09:25:05.708663+0000","last_unstale":"2024-09-16T09:32:18.497387+0000","last_undegraded":"2024-09-16T09:32:18.497387+0000","last_fullsized":"2024-09-16T09:32:18.497387+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:31:55.456592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:31:55.456592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:56.006473+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":309,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.491594+0000","last_change":"2024-09-16T09:26:37.282798+0000","last_active":"2024-09-16T09:32:18.491594+0000","last_peered":"2024-09-16T09:32:18.491594+0000","last_clean":"2024-09-16T09:32:18.491594+0000","last_became_active":"2024-09-16T09:25:07.719267+0000","last_became_peered":"2024-09-16T09:25:07.719267+0000","last_unstale":"2024-09-16T09:32:18.491594+0000","last_undegraded":"2024-09-16T09:32:18.491594+0000","last_fullsized":"2024-09-16T09:32:18.491594+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:37.282735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:37.282735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.8","version":"208'11910","reported_seq":18529,"reported_epoch":208,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-16T09:32:22.942445+0000","last_change":"2024-09-16T09:31:09.438251+0000","last_active":"2024-09-16T09:32:22.942445+0000","last_peered":"2024-09-16T09:32:22.942445+0000","last_clean":"2024-09-16T09:30:20.474651+0000","last_became_active":"2024-09-16T09:31:09.409873+0000","last_became_peered":"2024-09-16T09:31:09.409873+0000","last_unstale":"2024-09-16T09:32:22.942445+0000","last_undegraded":"2024-09-16T09:32:22.942445+0000","last_fullsized":"2024-09-16T09:31:09.256524+0000","mapping_epoch":186,"log_start":"189'7200","ondisk_log_start":"189'7200","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":4710,"log_dups_size":0,"ondisk_log_size":4710,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":249823232,"num_objects":3812,"num_object_clones":0,"num_object_copies":11436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1214,"num_objects_unfound":0,"num_objects_dirty":3812,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11910,"num_write_kb":503104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3242,"num_bytes_recovered":212467712,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,7],"avail_no_missing":["5","7"],"object_location_counts":[{"shards":"0,5,7","objects":1214},{"shards":"5,7","objects":2598}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1","version":"73'79","reported_seq":283,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.492632+0000","last_change":"2024-09-16T09:32:13.444142+0000","last_active":"2024-09-16T09:32:18.492632+0000","last_peered":"2024-09-16T09:32:18.492632+0000","last_clean":"2024-09-16T09:32:18.492632+0000","last_became_active":"2024-09-16T09:30:08.766350+0000","last_became_peered":"2024-09-16T09:30:08.766350+0000","last_unstale":"2024-09-16T09:32:18.492632+0000","last_undegraded":"2024-09-16T09:32:18.492632+0000","last_fullsized":"2024-09-16T09:32:18.492632+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:15.460179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:15.460179+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:32:42.562412+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"187'30","reported_seq":520,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.491887+0000","last_change":"2024-09-16T09:31:09.323281+0000","last_active":"2024-09-16T09:32:18.491887+0000","last_peered":"2024-09-16T09:32:18.491887+0000","last_clean":"2024-09-16T09:32:18.491887+0000","last_became_active":"2024-09-16T09:31:09.278839+0000","last_became_peered":"2024-09-16T09:31:09.278839+0000","last_unstale":"2024-09-16T09:32:18.491887+0000","last_undegraded":"2024-09-16T09:32:18.491887+0000","last_fullsized":"2024-09-16T09:32:18.491887+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'28","last_scrub_stamp":"2024-09-16T09:29:18.937623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:29:18.937623+0000","objects_scrubbed":10,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":329,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.488428+0000","last_change":"2024-09-16T09:29:26.637070+0000","last_active":"2024-09-16T09:32:18.488428+0000","last_peered":"2024-09-16T09:32:18.488428+0000","last_clean":"2024-09-16T09:32:18.488428+0000","last_became_active":"2024-09-16T09:25:01.703076+0000","last_became_peered":"2024-09-16T09:25:01.703076+0000","last_unstale":"2024-09-16T09:32:18.488428+0000","last_undegraded":"2024-09-16T09:32:18.488428+0000","last_fullsized":"2024-09-16T09:32:18.488428+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:26.637027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:26.637027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":338,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.493217+0000","last_change":"2024-09-16T09:29:29.486284+0000","last_active":"2024-09-16T09:32:18.493217+0000","last_peered":"2024-09-16T09:32:18.493217+0000","last_clean":"2024-09-16T09:32:18.493217+0000","last_became_active":"2024-09-16T09:25:00.696237+0000","last_became_peered":"2024-09-16T09:25:00.696237+0000","last_unstale":"2024-09-16T09:32:18.493217+0000","last_undegraded":"2024-09-16T09:32:18.493217+0000","last_fullsized":"2024-09-16T09:32:18.493217+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-16T09:29:29.486188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:29:29.486188+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.0025137430000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":305,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.495122+0000","last_change":"2024-09-16T09:25:05.718319+0000","last_active":"2024-09-16T09:32:18.495122+0000","last_peered":"2024-09-16T09:32:18.495122+0000","last_clean":"2024-09-16T09:32:18.495122+0000","last_became_active":"2024-09-16T09:25:05.717591+0000","last_became_peered":"2024-09-16T09:25:05.717591+0000","last_unstale":"2024-09-16T09:32:18.495122+0000","last_undegraded":"2024-09-16T09:32:18.495122+0000","last_fullsized":"2024-09-16T09:32:18.495122+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:11.173707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":54,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.492067+0000","last_change":"2024-09-16T09:31:09.272967+0000","last_active":"2024-09-16T09:32:18.492067+0000","last_peered":"2024-09-16T09:32:18.492067+0000","last_clean":"2024-09-16T09:32:18.492067+0000","last_became_active":"2024-09-16T09:31:09.272609+0000","last_became_peered":"2024-09-16T09:31:09.272609+0000","last_unstale":"2024-09-16T09:32:18.492067+0000","last_undegraded":"2024-09-16T09:32:18.492067+0000","last_fullsized":"2024-09-16T09:32:18.492067+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:45.201503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:45.201503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"208'11723","reported_seq":22967,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:22.772452+0000","last_change":"2024-09-16T09:32:08.439285+0000","last_active":"2024-09-16T09:32:22.772452+0000","last_peered":"2024-09-16T09:32:22.772452+0000","last_clean":"2024-09-16T09:32:22.772452+0000","last_became_active":"2024-09-16T09:32:08.422751+0000","last_became_peered":"2024-09-16T09:32:08.422751+0000","last_unstale":"2024-09-16T09:32:22.772452+0000","last_undegraded":"2024-09-16T09:32:22.772452+0000","last_fullsized":"2024-09-16T09:32:22.772452+0000","mapping_epoch":204,"log_start":"191'7400","ondisk_log_start":"191'7400","created":152,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":4323,"log_dups_size":0,"ondisk_log_size":4323,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":249757696,"num_objects":3811,"num_object_clones":0,"num_object_copies":11433,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3811,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11723,"num_write_kb":497088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5551,"num_bytes_recovered":363790336,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.2","version":"73'79","reported_seq":408,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.496814+0000","last_change":"2024-09-16T09:32:13.450912+0000","last_active":"2024-09-16T09:32:18.496814+0000","last_peered":"2024-09-16T09:32:18.496814+0000","last_clean":"2024-09-16T09:32:18.496814+0000","last_became_active":"2024-09-16T09:30:08.765943+0000","last_became_peered":"2024-09-16T09:30:08.765943+0000","last_unstale":"2024-09-16T09:32:18.496814+0000","last_undegraded":"2024-09-16T09:32:18.496814+0000","last_fullsized":"2024-09-16T09:32:18.496814+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:26:35.745837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:26:35.745837+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:32:42.371213+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"66'13","reported_seq":352,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.496795+0000","last_change":"2024-09-16T09:27:44.156057+0000","last_active":"2024-09-16T09:32:18.496795+0000","last_peered":"2024-09-16T09:32:18.496795+0000","last_clean":"2024-09-16T09:32:18.496795+0000","last_became_active":"2024-09-16T09:25:03.713241+0000","last_became_peered":"2024-09-16T09:25:03.713241+0000","last_unstale":"2024-09-16T09:32:18.496795+0000","last_undegraded":"2024-09-16T09:32:18.496795+0000","last_fullsized":"2024-09-16T09:32:18.496795+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'13","last_scrub_stamp":"2024-09-16T09:27:44.155969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:44.155969+0000","objects_scrubbed":5,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":29,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":329,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.496436+0000","last_change":"2024-09-16T09:29:50.827103+0000","last_active":"2024-09-16T09:32:18.496436+0000","last_peered":"2024-09-16T09:32:18.496436+0000","last_clean":"2024-09-16T09:32:18.496436+0000","last_became_active":"2024-09-16T09:25:01.706828+0000","last_became_peered":"2024-09-16T09:25:01.706828+0000","last_unstale":"2024-09-16T09:32:18.496436+0000","last_undegraded":"2024-09-16T09:32:18.496436+0000","last_fullsized":"2024-09-16T09:32:18.496436+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:50.827057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:50.827057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":360,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.499149+0000","last_change":"2024-09-16T09:31:09.281163+0000","last_active":"2024-09-16T09:32:18.499149+0000","last_peered":"2024-09-16T09:32:18.499149+0000","last_clean":"2024-09-16T09:32:18.499149+0000","last_became_active":"2024-09-16T09:31:09.266338+0000","last_became_peered":"2024-09-16T09:31:09.266338+0000","last_unstale":"2024-09-16T09:32:18.499149+0000","last_undegraded":"2024-09-16T09:32:18.499149+0000","last_fullsized":"2024-09-16T09:32:18.499149+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:29:16.104209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:29:16.104209+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0.00055022299999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":329,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.498525+0000","last_change":"2024-09-16T09:29:00.169141+0000","last_active":"2024-09-16T09:32:18.498525+0000","last_peered":"2024-09-16T09:32:18.498525+0000","last_clean":"2024-09-16T09:32:18.498525+0000","last_became_active":"2024-09-16T09:25:05.711539+0000","last_became_peered":"2024-09-16T09:25:05.711539+0000","last_unstale":"2024-09-16T09:32:18.498525+0000","last_undegraded":"2024-09-16T09:32:18.498525+0000","last_fullsized":"2024-09-16T09:32:18.498525+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:00.169077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:00.169077+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":316,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.498180+0000","last_change":"2024-09-16T09:29:22.968971+0000","last_active":"2024-09-16T09:32:18.498180+0000","last_peered":"2024-09-16T09:32:18.498180+0000","last_clean":"2024-09-16T09:32:18.498180+0000","last_became_active":"2024-09-16T09:25:07.723282+0000","last_became_peered":"2024-09-16T09:25:07.723282+0000","last_unstale":"2024-09-16T09:32:18.498180+0000","last_undegraded":"2024-09-16T09:32:18.498180+0000","last_fullsized":"2024-09-16T09:32:18.498180+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:22.968908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:22.968908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"208'11743","reported_seq":11867,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:22.908001+0000","last_change":"2024-09-16T09:29:49.448898+0000","last_active":"2024-09-16T09:32:22.908001+0000","last_peered":"2024-09-16T09:32:22.908001+0000","last_clean":"2024-09-16T09:32:22.908001+0000","last_became_active":"2024-09-16T09:29:49.448628+0000","last_became_peered":"2024-09-16T09:29:49.448628+0000","last_unstale":"2024-09-16T09:32:22.908001+0000","last_undegraded":"2024-09-16T09:32:22.908001+0000","last_fullsized":"2024-09-16T09:32:22.908001+0000","mapping_epoch":152,"log_start":"189'7200","ondisk_log_start":"189'7200","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":4543,"log_dups_size":0,"ondisk_log_size":4543,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":255000576,"num_objects":3891,"num_object_clones":0,"num_object_copies":11673,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3891,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11743,"num_write_kb":500288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"73'79","reported_seq":290,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.496640+0000","last_change":"2024-09-16T09:31:40.212968+0000","last_active":"2024-09-16T09:32:18.496640+0000","last_peered":"2024-09-16T09:32:18.496640+0000","last_clean":"2024-09-16T09:32:18.496640+0000","last_became_active":"2024-09-16T09:30:08.877110+0000","last_became_peered":"2024-09-16T09:30:08.877110+0000","last_unstale":"2024-09-16T09:32:18.496640+0000","last_undegraded":"2024-09-16T09:32:18.496640+0000","last_fullsized":"2024-09-16T09:32:18.496640+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:31:40.212889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:31:40.212889+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:45.708324+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"66'12","reported_seq":335,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.494754+0000","last_change":"2024-09-16T09:29:02.713335+0000","last_active":"2024-09-16T09:32:18.494754+0000","last_peered":"2024-09-16T09:32:18.494754+0000","last_clean":"2024-09-16T09:32:18.494754+0000","last_became_active":"2024-09-16T09:25:03.715315+0000","last_became_peered":"2024-09-16T09:25:03.715315+0000","last_unstale":"2024-09-16T09:32:18.494754+0000","last_undegraded":"2024-09-16T09:32:18.494754+0000","last_fullsized":"2024-09-16T09:32:18.494754+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-16T09:29:02.713259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:29:02.713259+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:43.220661+0000","scrub_duration":26,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"59'1","reported_seq":352,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.494227+0000","last_change":"2024-09-16T09:31:09.286309+0000","last_active":"2024-09-16T09:32:18.494227+0000","last_peered":"2024-09-16T09:32:18.494227+0000","last_clean":"2024-09-16T09:32:18.494227+0000","last_became_active":"2024-09-16T09:31:09.269627+0000","last_became_peered":"2024-09-16T09:31:09.269627+0000","last_unstale":"2024-09-16T09:32:18.494227+0000","last_undegraded":"2024-09-16T09:32:18.494227+0000","last_fullsized":"2024-09-16T09:32:18.494227+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:27:00.666673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:27:00.666673+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":28,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1986,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"67'5","reported_seq":779,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:26.153145+0000","last_change":"2024-09-16T09:29:19.994791+0000","last_active":"2024-09-16T09:32:26.153145+0000","last_peered":"2024-09-16T09:32:26.153145+0000","last_clean":"2024-09-16T09:32:26.153145+0000","last_became_active":"2024-09-16T09:25:00.696546+0000","last_became_peered":"2024-09-16T09:25:00.696546+0000","last_unstale":"2024-09-16T09:32:26.153145+0000","last_undegraded":"2024-09-16T09:32:26.153145+0000","last_fullsized":"2024-09-16T09:32:26.153145+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'5","last_scrub_stamp":"2024-09-16T09:29:19.994716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:29:19.994716+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0.0059770420000000001,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":440,"num_read_kb":435,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":312,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.493451+0000","last_change":"2024-09-16T09:27:07.337170+0000","last_active":"2024-09-16T09:32:18.493451+0000","last_peered":"2024-09-16T09:32:18.493451+0000","last_clean":"2024-09-16T09:32:18.493451+0000","last_became_active":"2024-09-16T09:25:05.714704+0000","last_became_peered":"2024-09-16T09:25:05.714704+0000","last_unstale":"2024-09-16T09:32:18.493451+0000","last_undegraded":"2024-09-16T09:32:18.493451+0000","last_fullsized":"2024-09-16T09:32:18.493451+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:07.337124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:07.337124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":323,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.491509+0000","last_change":"2024-09-16T09:31:09.271453+0000","last_active":"2024-09-16T09:32:18.491509+0000","last_peered":"2024-09-16T09:32:18.491509+0000","last_clean":"2024-09-16T09:32:18.491509+0000","last_became_active":"2024-09-16T09:31:09.271111+0000","last_became_peered":"2024-09-16T09:31:09.271111+0000","last_unstale":"2024-09-16T09:32:18.491509+0000","last_undegraded":"2024-09-16T09:32:18.491509+0000","last_fullsized":"2024-09-16T09:32:18.491509+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:24.231167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:24.231167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"208'12026","reported_seq":12215,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:27.599787+0000","last_change":"2024-09-16T09:30:56.023710+0000","last_active":"2024-09-16T09:32:27.599787+0000","last_peered":"2024-09-16T09:32:27.599787+0000","last_clean":"2024-09-16T09:32:27.599787+0000","last_became_active":"2024-09-16T09:29:49.450643+0000","last_became_peered":"2024-09-16T09:29:49.450643+0000","last_unstale":"2024-09-16T09:32:27.599787+0000","last_undegraded":"2024-09-16T09:32:27.599787+0000","last_fullsized":"2024-09-16T09:32:27.599787+0000","mapping_epoch":152,"log_start":"187'6700","ondisk_log_start":"187'6700","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"181'5780","last_scrub_stamp":"2024-09-16T09:30:56.023622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:30:56.023622+0000","objects_scrubbed":5716,"log_size":5326,"log_dups_size":0,"ondisk_log_size":5326,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:14.532550+0000","scrub_duration":1526,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":231866368,"num_objects":3538,"num_object_clones":0,"num_object_copies":10614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3538,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12026,"num_write_kb":498048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"66'16","reported_seq":445,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:21.849100+0000","last_change":"2024-09-16T09:32:01.564577+0000","last_active":"2024-09-16T09:32:21.849100+0000","last_peered":"2024-09-16T09:32:21.849100+0000","last_clean":"2024-09-16T09:32:21.849100+0000","last_became_active":"2024-09-16T09:25:03.714118+0000","last_became_peered":"2024-09-16T09:25:03.714118+0000","last_unstale":"2024-09-16T09:32:21.849100+0000","last_undegraded":"2024-09-16T09:32:21.849100+0000","last_fullsized":"2024-09-16T09:32:21.849100+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'16","last_scrub_stamp":"2024-09-16T09:32:01.564489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:32:01.564489+0000","objects_scrubbed":8,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:10.625142+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":170,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.493253+0000","last_change":"2024-09-16T09:31:09.270603+0000","last_active":"2024-09-16T09:32:18.493253+0000","last_peered":"2024-09-16T09:32:18.493253+0000","last_clean":"2024-09-16T09:32:18.493253+0000","last_became_active":"2024-09-16T09:31:09.270196+0000","last_became_peered":"2024-09-16T09:31:09.270196+0000","last_unstale":"2024-09-16T09:32:18.493253+0000","last_undegraded":"2024-09-16T09:32:18.493253+0000","last_fullsized":"2024-09-16T09:32:18.493253+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:49.195331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:27:49.195331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"73'79","reported_seq":408,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.501567+0000","last_change":"2024-09-16T09:32:13.453628+0000","last_active":"2024-09-16T09:32:18.501567+0000","last_peered":"2024-09-16T09:32:18.501567+0000","last_clean":"2024-09-16T09:32:18.501567+0000","last_became_active":"2024-09-16T09:30:08.870394+0000","last_became_peered":"2024-09-16T09:30:08.870394+0000","last_unstale":"2024-09-16T09:32:18.501567+0000","last_undegraded":"2024-09-16T09:32:18.501567+0000","last_fullsized":"2024-09-16T09:32:18.501567+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:02.723794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:02.723794+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:32:43.420510+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":312,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.497640+0000","last_change":"2024-09-16T09:26:39.699133+0000","last_active":"2024-09-16T09:32:18.497640+0000","last_peered":"2024-09-16T09:32:18.497640+0000","last_clean":"2024-09-16T09:32:18.497640+0000","last_became_active":"2024-09-16T09:25:05.714341+0000","last_became_peered":"2024-09-16T09:25:05.714341+0000","last_unstale":"2024-09-16T09:32:18.497640+0000","last_undegraded":"2024-09-16T09:32:18.497640+0000","last_fullsized":"2024-09-16T09:32:18.497640+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:39.699059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:26:39.699059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":316,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.493344+0000","last_change":"2024-09-16T09:29:00.243528+0000","last_active":"2024-09-16T09:32:18.493344+0000","last_peered":"2024-09-16T09:32:18.493344+0000","last_clean":"2024-09-16T09:32:18.493344+0000","last_became_active":"2024-09-16T09:25:07.714281+0000","last_became_peered":"2024-09-16T09:25:07.714281+0000","last_unstale":"2024-09-16T09:32:18.493344+0000","last_undegraded":"2024-09-16T09:32:18.493344+0000","last_fullsized":"2024-09-16T09:32:18.493344+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:00.243482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:00.243482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.b","version":"73'79","reported_seq":294,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.491932+0000","last_change":"2024-09-16T09:32:16.500592+0000","last_active":"2024-09-16T09:32:18.491932+0000","last_peered":"2024-09-16T09:32:18.491932+0000","last_clean":"2024-09-16T09:32:18.491932+0000","last_became_active":"2024-09-16T09:30:10.784688+0000","last_became_peered":"2024-09-16T09:30:10.784688+0000","last_unstale":"2024-09-16T09:32:18.491932+0000","last_undegraded":"2024-09-16T09:32:18.491932+0000","last_fullsized":"2024-09-16T09:32:18.491932+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:30:03.887571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:30:03.887571+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:32:44.156390+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,5],"acting":[1,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2","version":"208'11528","reported_seq":11652,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:22.809190+0000","last_change":"2024-09-16T09:29:49.445324+0000","last_active":"2024-09-16T09:32:22.809190+0000","last_peered":"2024-09-16T09:32:22.809190+0000","last_clean":"2024-09-16T09:32:22.809190+0000","last_became_active":"2024-09-16T09:29:49.445022+0000","last_became_peered":"2024-09-16T09:29:49.445022+0000","last_unstale":"2024-09-16T09:32:22.809190+0000","last_undegraded":"2024-09-16T09:32:22.809190+0000","last_fullsized":"2024-09-16T09:32:22.809190+0000","mapping_epoch":152,"log_start":"192'7300","ondisk_log_start":"192'7300","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":4228,"log_dups_size":0,"ondisk_log_size":4228,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":243007488,"num_objects":3708,"num_object_clones":0,"num_object_copies":11124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3708,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11528,"num_write_kb":487552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"66'11","reported_seq":355,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.485605+0000","last_change":"2024-09-16T09:30:05.646701+0000","last_active":"2024-09-16T09:32:18.485605+0000","last_peered":"2024-09-16T09:32:18.485605+0000","last_clean":"2024-09-16T09:32:18.485605+0000","last_became_active":"2024-09-16T09:26:33.287399+0000","last_became_peered":"2024-09-16T09:26:33.287399+0000","last_unstale":"2024-09-16T09:32:18.485605+0000","last_undegraded":"2024-09-16T09:32:18.485605+0000","last_fullsized":"2024-09-16T09:32:18.485605+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-16T09:30:05.646500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:30:05.646500+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":313,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.485609+0000","last_change":"2024-09-16T09:25:01.708568+0000","last_active":"2024-09-16T09:32:18.485609+0000","last_peered":"2024-09-16T09:32:18.485609+0000","last_clean":"2024-09-16T09:32:18.485609+0000","last_became_active":"2024-09-16T09:25:01.708270+0000","last_became_peered":"2024-09-16T09:25:01.708270+0000","last_unstale":"2024-09-16T09:32:18.485609+0000","last_undegraded":"2024-09-16T09:32:18.485609+0000","last_fullsized":"2024-09-16T09:32:18.485609+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:10.865036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":312,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.493223+0000","last_change":"2024-09-16T09:26:34.171190+0000","last_active":"2024-09-16T09:32:18.493223+0000","last_peered":"2024-09-16T09:32:18.493223+0000","last_clean":"2024-09-16T09:32:18.493223+0000","last_became_active":"2024-09-16T09:25:05.711964+0000","last_became_peered":"2024-09-16T09:25:05.711964+0000","last_unstale":"2024-09-16T09:32:18.493223+0000","last_undegraded":"2024-09-16T09:32:18.493223+0000","last_fullsized":"2024-09-16T09:32:18.493223+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:34.171139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:26:34.171139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":195,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.497418+0000","last_change":"2024-09-16T09:32:00.526741+0000","last_active":"2024-09-16T09:32:18.497418+0000","last_peered":"2024-09-16T09:32:18.497418+0000","last_clean":"2024-09-16T09:32:18.497418+0000","last_became_active":"2024-09-16T09:31:09.278618+0000","last_became_peered":"2024-09-16T09:31:09.278618+0000","last_unstale":"2024-09-16T09:32:18.497418+0000","last_undegraded":"2024-09-16T09:32:18.497418+0000","last_fullsized":"2024-09-16T09:32:18.497418+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:00.526674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:32:00.526674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:25.768941+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1","version":"208'11425","reported_seq":11549,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:22.893310+0000","last_change":"2024-09-16T09:29:49.449796+0000","last_active":"2024-09-16T09:32:22.893310+0000","last_peered":"2024-09-16T09:32:22.893310+0000","last_clean":"2024-09-16T09:32:22.893310+0000","last_became_active":"2024-09-16T09:29:49.449475+0000","last_became_peered":"2024-09-16T09:29:49.449475+0000","last_unstale":"2024-09-16T09:32:22.893310+0000","last_undegraded":"2024-09-16T09:32:22.893310+0000","last_fullsized":"2024-09-16T09:32:22.893310+0000","mapping_epoch":152,"log_start":"188'6900","ondisk_log_start":"188'6900","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":4525,"log_dups_size":0,"ondisk_log_size":4525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":251199488,"num_objects":3833,"num_object_clones":0,"num_object_copies":11499,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3833,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11425,"num_write_kb":488256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.8","version":"73'79","reported_seq":291,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.495163+0000","last_change":"2024-09-16T09:32:07.068820+0000","last_active":"2024-09-16T09:32:18.495163+0000","last_peered":"2024-09-16T09:32:18.495163+0000","last_clean":"2024-09-16T09:32:18.495163+0000","last_became_active":"2024-09-16T09:31:09.406638+0000","last_became_peered":"2024-09-16T09:31:09.406638+0000","last_unstale":"2024-09-16T09:32:18.495163+0000","last_undegraded":"2024-09-16T09:32:18.495163+0000","last_fullsized":"2024-09-16T09:32:18.495163+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:17.482769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:17.482769+0000","objects_scrubbed":1,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262688,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":525376,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"66'17","reported_seq":361,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.486088+0000","last_change":"2024-09-16T09:30:04.636115+0000","last_active":"2024-09-16T09:32:18.486088+0000","last_peered":"2024-09-16T09:32:18.486088+0000","last_clean":"2024-09-16T09:32:18.486088+0000","last_became_active":"2024-09-16T09:25:03.713358+0000","last_became_peered":"2024-09-16T09:25:03.713358+0000","last_unstale":"2024-09-16T09:32:18.486088+0000","last_undegraded":"2024-09-16T09:32:18.486088+0000","last_fullsized":"2024-09-16T09:32:18.486088+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'17","last_scrub_stamp":"2024-09-16T09:30:04.636039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:30:04.636039+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":321,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.497679+0000","last_change":"2024-09-16T09:26:04.104188+0000","last_active":"2024-09-16T09:32:18.497679+0000","last_peered":"2024-09-16T09:32:18.497679+0000","last_clean":"2024-09-16T09:32:18.497679+0000","last_became_active":"2024-09-16T09:25:01.703355+0000","last_became_peered":"2024-09-16T09:25:01.703355+0000","last_unstale":"2024-09-16T09:32:18.497679+0000","last_undegraded":"2024-09-16T09:32:18.497679+0000","last_fullsized":"2024-09-16T09:32:18.497679+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:04.104130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:04.104130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:48.432563+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":329,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.491607+0000","last_change":"2024-09-16T09:29:58.839132+0000","last_active":"2024-09-16T09:32:18.491607+0000","last_peered":"2024-09-16T09:32:18.491607+0000","last_clean":"2024-09-16T09:32:18.491607+0000","last_became_active":"2024-09-16T09:25:05.716900+0000","last_became_peered":"2024-09-16T09:25:05.716900+0000","last_unstale":"2024-09-16T09:32:18.491607+0000","last_undegraded":"2024-09-16T09:32:18.491607+0000","last_fullsized":"2024-09-16T09:32:18.491607+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:58.839051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:58.839051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":339,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.498052+0000","last_change":"2024-09-16T09:31:09.276869+0000","last_active":"2024-09-16T09:32:18.498052+0000","last_peered":"2024-09-16T09:32:18.498052+0000","last_clean":"2024-09-16T09:32:18.498052+0000","last_became_active":"2024-09-16T09:31:09.276275+0000","last_became_peered":"2024-09-16T09:31:09.276275+0000","last_unstale":"2024-09-16T09:32:18.498052+0000","last_undegraded":"2024-09-16T09:32:18.498052+0000","last_fullsized":"2024-09-16T09:32:18.498052+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:30:03.946025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:30:03.946025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.0","version":"208'11732","reported_seq":18647,"reported_epoch":208,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-16T09:32:22.930954+0000","last_change":"2024-09-16T09:31:09.476408+0000","last_active":"2024-09-16T09:32:22.930954+0000","last_peered":"2024-09-16T09:32:22.930954+0000","last_clean":"2024-09-16T09:30:20.486290+0000","last_became_active":"2024-09-16T09:31:09.399514+0000","last_became_peered":"2024-09-16T09:31:09.399514+0000","last_unstale":"2024-09-16T09:32:22.930954+0000","last_undegraded":"2024-09-16T09:32:22.930954+0000","last_fullsized":"2024-09-16T09:31:09.254572+0000","mapping_epoch":186,"log_start":"187'7000","ondisk_log_start":"187'7000","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":4732,"log_dups_size":0,"ondisk_log_size":4732,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":249298944,"num_objects":3804,"num_object_clones":0,"num_object_copies":11412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1312,"num_objects_unfound":0,"num_objects_dirty":3804,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11732,"num_write_kb":497152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3390,"num_bytes_recovered":222167040,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,1],"avail_no_missing":["5","1"],"object_location_counts":[{"shards":"1,4,5","objects":1312},{"shards":"1,5","objects":2492}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.9","version":"73'79","reported_seq":392,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.496992+0000","last_change":"2024-09-16T09:29:19.013948+0000","last_active":"2024-09-16T09:32:18.496992+0000","last_peered":"2024-09-16T09:32:18.496992+0000","last_clean":"2024-09-16T09:32:18.496992+0000","last_became_active":"2024-09-16T09:26:35.126066+0000","last_became_peered":"2024-09-16T09:26:35.126066+0000","last_unstale":"2024-09-16T09:32:18.496992+0000","last_undegraded":"2024-09-16T09:32:18.496992+0000","last_fullsized":"2024-09-16T09:32:18.496992+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:19.013865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:19.013865+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"66'10","reported_seq":324,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.486132+0000","last_change":"2024-09-16T09:25:03.708144+0000","last_active":"2024-09-16T09:32:18.486132+0000","last_peered":"2024-09-16T09:32:18.486132+0000","last_clean":"2024-09-16T09:32:18.486132+0000","last_became_active":"2024-09-16T09:25:03.707723+0000","last_became_peered":"2024-09-16T09:25:03.707723+0000","last_unstale":"2024-09-16T09:32:18.486132+0000","last_undegraded":"2024-09-16T09:32:18.486132+0000","last_fullsized":"2024-09-16T09:32:18.486132+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:13.930015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":186,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.494600+0000","last_change":"2024-09-16T09:31:32.253397+0000","last_active":"2024-09-16T09:32:18.494600+0000","last_peered":"2024-09-16T09:32:18.494600+0000","last_clean":"2024-09-16T09:32:18.494600+0000","last_became_active":"2024-09-16T09:31:09.288825+0000","last_became_peered":"2024-09-16T09:31:09.288825+0000","last_unstale":"2024-09-16T09:32:18.494600+0000","last_undegraded":"2024-09-16T09:32:18.494600+0000","last_fullsized":"2024-09-16T09:32:18.494600+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:31:32.253306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:31:32.253306+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:45.682494+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":429,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.498428+0000","last_change":"2024-09-16T09:29:29.866520+0000","last_active":"2024-09-16T09:32:18.498428+0000","last_peered":"2024-09-16T09:32:18.498428+0000","last_clean":"2024-09-16T09:32:18.498428+0000","last_became_active":"2024-09-16T09:25:05.711590+0000","last_became_peered":"2024-09-16T09:25:05.711590+0000","last_unstale":"2024-09-16T09:32:18.498428+0000","last_undegraded":"2024-09-16T09:32:18.498428+0000","last_fullsized":"2024-09-16T09:32:18.498428+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:29:29.866460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:29.866460+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":325,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.485407+0000","last_change":"2024-09-16T09:29:02.667332+0000","last_active":"2024-09-16T09:32:18.485407+0000","last_peered":"2024-09-16T09:32:18.485407+0000","last_clean":"2024-09-16T09:32:18.485407+0000","last_became_active":"2024-09-16T09:25:07.718958+0000","last_became_peered":"2024-09-16T09:25:07.718958+0000","last_unstale":"2024-09-16T09:32:18.485407+0000","last_undegraded":"2024-09-16T09:32:18.485407+0000","last_fullsized":"2024-09-16T09:32:18.485407+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:02.667259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:02.667259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.e","version":"73'79","reported_seq":117,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.485232+0000","last_change":"2024-09-16T09:32:13.445947+0000","last_active":"2024-09-16T09:32:18.485232+0000","last_peered":"2024-09-16T09:32:18.485232+0000","last_clean":"2024-09-16T09:32:18.485232+0000","last_became_active":"2024-09-16T09:30:10.705072+0000","last_became_peered":"2024-09-16T09:30:10.705072+0000","last_unstale":"2024-09-16T09:32:18.485232+0000","last_undegraded":"2024-09-16T09:32:18.485232+0000","last_fullsized":"2024-09-16T09:32:18.485232+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:26:35.170229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:26:35.170229+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:32:42.891843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26835,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":10,"num_write":5,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":26835,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,3],"acting":[4,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.7","version":"208'11659","reported_seq":11783,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:22.899657+0000","last_change":"2024-09-16T09:29:49.839740+0000","last_active":"2024-09-16T09:32:22.899657+0000","last_peered":"2024-09-16T09:32:22.899657+0000","last_clean":"2024-09-16T09:32:22.899657+0000","last_became_active":"2024-09-16T09:29:49.839437+0000","last_became_peered":"2024-09-16T09:29:49.839437+0000","last_unstale":"2024-09-16T09:32:22.899657+0000","last_undegraded":"2024-09-16T09:32:22.899657+0000","last_fullsized":"2024-09-16T09:32:22.899657+0000","mapping_epoch":152,"log_start":"189'7100","ondisk_log_start":"189'7100","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":4559,"log_dups_size":0,"ondisk_log_size":4559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":248053760,"num_objects":3785,"num_object_clones":0,"num_object_copies":11355,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3785,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11659,"num_write_kb":494208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"66'9","reported_seq":349,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.494327+0000","last_change":"2024-09-16T09:31:31.211495+0000","last_active":"2024-09-16T09:32:18.494327+0000","last_peered":"2024-09-16T09:32:18.494327+0000","last_clean":"2024-09-16T09:32:18.494327+0000","last_became_active":"2024-09-16T09:25:03.712230+0000","last_became_peered":"2024-09-16T09:25:03.712230+0000","last_unstale":"2024-09-16T09:32:18.494327+0000","last_undegraded":"2024-09-16T09:32:18.494327+0000","last_fullsized":"2024-09-16T09:32:18.494327+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:31:31.211420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:31:31.211420+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:36.499658+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":334,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.493306+0000","last_change":"2024-09-16T09:31:09.277413+0000","last_active":"2024-09-16T09:32:18.493306+0000","last_peered":"2024-09-16T09:32:18.493306+0000","last_clean":"2024-09-16T09:32:18.493306+0000","last_became_active":"2024-09-16T09:31:09.276516+0000","last_became_peered":"2024-09-16T09:31:09.276516+0000","last_unstale":"2024-09-16T09:32:18.493306+0000","last_undegraded":"2024-09-16T09:32:18.493306+0000","last_fullsized":"2024-09-16T09:32:18.493306+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:26.061106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:26.061106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":335,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.498797+0000","last_change":"2024-09-16T09:31:09.275219+0000","last_active":"2024-09-16T09:32:18.498797+0000","last_peered":"2024-09-16T09:32:18.498797+0000","last_clean":"2024-09-16T09:32:18.498797+0000","last_became_active":"2024-09-16T09:31:09.274702+0000","last_became_peered":"2024-09-16T09:31:09.274702+0000","last_unstale":"2024-09-16T09:32:18.498797+0000","last_undegraded":"2024-09-16T09:32:18.498797+0000","last_fullsized":"2024-09-16T09:32:18.498797+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:57.220068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:28:57.220068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":33,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":325,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.494148+0000","last_change":"2024-09-16T09:31:27.185957+0000","last_active":"2024-09-16T09:32:18.494148+0000","last_peered":"2024-09-16T09:32:18.494148+0000","last_clean":"2024-09-16T09:32:18.494148+0000","last_became_active":"2024-09-16T09:25:07.718778+0000","last_became_peered":"2024-09-16T09:25:07.718778+0000","last_unstale":"2024-09-16T09:32:18.494148+0000","last_undegraded":"2024-09-16T09:32:18.494148+0000","last_fullsized":"2024-09-16T09:32:18.494148+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:31:27.185801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:31:27.185801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:39.017009+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.f","version":"73'79","reported_seq":370,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.498659+0000","last_change":"2024-09-16T09:32:16.492685+0000","last_active":"2024-09-16T09:32:18.498659+0000","last_peered":"2024-09-16T09:32:18.498659+0000","last_clean":"2024-09-16T09:32:18.498659+0000","last_became_active":"2024-09-16T09:30:10.693477+0000","last_became_peered":"2024-09-16T09:30:10.693477+0000","last_unstale":"2024-09-16T09:32:18.498659+0000","last_undegraded":"2024-09-16T09:32:18.498659+0000","last_fullsized":"2024-09-16T09:32:18.498659+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:17.534429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:17.534429+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:32:43.568934+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.6","version":"208'11930","reported_seq":12054,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:23.167273+0000","last_change":"2024-09-16T09:29:50.008888+0000","last_active":"2024-09-16T09:32:23.167273+0000","last_peered":"2024-09-16T09:32:23.167273+0000","last_clean":"2024-09-16T09:32:23.167273+0000","last_became_active":"2024-09-16T09:29:50.008656+0000","last_became_peered":"2024-09-16T09:29:50.008656+0000","last_unstale":"2024-09-16T09:32:23.167273+0000","last_undegraded":"2024-09-16T09:32:23.167273+0000","last_fullsized":"2024-09-16T09:32:23.167273+0000","mapping_epoch":152,"log_start":"185'6800","ondisk_log_start":"185'6800","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":5130,"log_dups_size":0,"ondisk_log_size":5130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":255066112,"num_objects":3892,"num_object_clones":0,"num_object_copies":11676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3892,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11930,"num_write_kb":506304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"66'19","reported_seq":356,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.497478+0000","last_change":"2024-09-16T09:29:02.554357+0000","last_active":"2024-09-16T09:32:18.497478+0000","last_peered":"2024-09-16T09:32:18.497478+0000","last_clean":"2024-09-16T09:32:18.497478+0000","last_became_active":"2024-09-16T09:25:03.712783+0000","last_became_peered":"2024-09-16T09:25:03.712783+0000","last_unstale":"2024-09-16T09:32:18.497478+0000","last_undegraded":"2024-09-16T09:32:18.497478+0000","last_fullsized":"2024-09-16T09:32:18.497478+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-16T09:29:02.554274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:29:02.554274+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"59'1","reported_seq":334,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.498556+0000","last_change":"2024-09-16T09:29:51.025505+0000","last_active":"2024-09-16T09:32:18.498556+0000","last_peered":"2024-09-16T09:32:18.498556+0000","last_clean":"2024-09-16T09:32:18.498556+0000","last_became_active":"2024-09-16T09:25:01.708069+0000","last_became_peered":"2024-09-16T09:25:01.708069+0000","last_unstale":"2024-09-16T09:32:18.498556+0000","last_undegraded":"2024-09-16T09:32:18.498556+0000","last_fullsized":"2024-09-16T09:32:18.498556+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:29:51.025444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:51.025444+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":313,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.498126+0000","last_change":"2024-09-16T09:27:11.067059+0000","last_active":"2024-09-16T09:32:18.498126+0000","last_peered":"2024-09-16T09:32:18.498126+0000","last_clean":"2024-09-16T09:32:18.498126+0000","last_became_active":"2024-09-16T09:25:05.709264+0000","last_became_peered":"2024-09-16T09:25:05.709264+0000","last_unstale":"2024-09-16T09:32:18.498126+0000","last_undegraded":"2024-09-16T09:32:18.498126+0000","last_fullsized":"2024-09-16T09:32:18.498126+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:11.066986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:11.066986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":308,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.501708+0000","last_change":"2024-09-16T09:27:25.865107+0000","last_active":"2024-09-16T09:32:18.501708+0000","last_peered":"2024-09-16T09:32:18.501708+0000","last_clean":"2024-09-16T09:32:18.501708+0000","last_became_active":"2024-09-16T09:25:07.718750+0000","last_became_peered":"2024-09-16T09:25:07.718750+0000","last_unstale":"2024-09-16T09:32:18.501708+0000","last_undegraded":"2024-09-16T09:32:18.501708+0000","last_fullsized":"2024-09-16T09:32:18.501708+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:25.865030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:25.865030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.c","version":"73'79","reported_seq":58,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.493144+0000","last_change":"2024-09-16T09:31:09.290919+0000","last_active":"2024-09-16T09:32:18.493144+0000","last_peered":"2024-09-16T09:32:18.493144+0000","last_clean":"2024-09-16T09:32:18.493144+0000","last_became_active":"2024-09-16T09:31:09.273770+0000","last_became_peered":"2024-09-16T09:31:09.273770+0000","last_unstale":"2024-09-16T09:32:18.493144+0000","last_undegraded":"2024-09-16T09:32:18.493144+0000","last_fullsized":"2024-09-16T09:32:18.493144+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:30:22.614462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:30:22.614462+0000","objects_scrubbed":1,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":327680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.5","version":"208'12209","reported_seq":12406,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:27.617996+0000","last_change":"2024-09-16T09:31:06.121217+0000","last_active":"2024-09-16T09:32:27.617996+0000","last_peered":"2024-09-16T09:32:27.617996+0000","last_clean":"2024-09-16T09:32:27.617996+0000","last_became_active":"2024-09-16T09:29:50.009013+0000","last_became_peered":"2024-09-16T09:29:50.009013+0000","last_unstale":"2024-09-16T09:32:27.617996+0000","last_undegraded":"2024-09-16T09:32:27.617996+0000","last_fullsized":"2024-09-16T09:32:27.617996+0000","mapping_epoch":152,"log_start":"187'6913","ondisk_log_start":"187'6913","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"184'6587","last_scrub_stamp":"2024-09-16T09:31:06.120990+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:31:06.120990+0000","objects_scrubbed":6523,"log_size":5296,"log_dups_size":0,"ondisk_log_size":5296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:25.588988+0000","scrub_duration":1690,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":231669760,"num_objects":3535,"num_object_clones":0,"num_object_copies":10605,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3535,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12209,"num_write_kb":503808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"66'12","reported_seq":386,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.485476+0000","last_change":"2024-09-16T09:31:09.296915+0000","last_active":"2024-09-16T09:32:18.485476+0000","last_peered":"2024-09-16T09:32:18.485476+0000","last_clean":"2024-09-16T09:32:18.485476+0000","last_became_active":"2024-09-16T09:31:09.269047+0000","last_became_peered":"2024-09-16T09:31:09.269047+0000","last_unstale":"2024-09-16T09:32:18.485476+0000","last_undegraded":"2024-09-16T09:32:18.485476+0000","last_fullsized":"2024-09-16T09:32:18.485476+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-16T09:30:21.728668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:30:21.728668+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":328,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.497892+0000","last_change":"2024-09-16T09:29:53.075288+0000","last_active":"2024-09-16T09:32:18.497892+0000","last_peered":"2024-09-16T09:32:18.497892+0000","last_clean":"2024-09-16T09:32:18.497892+0000","last_became_active":"2024-09-16T09:25:01.707528+0000","last_became_peered":"2024-09-16T09:25:01.707528+0000","last_unstale":"2024-09-16T09:32:18.497892+0000","last_undegraded":"2024-09-16T09:32:18.497892+0000","last_fullsized":"2024-09-16T09:32:18.497892+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:53.075232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:53.075232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":321,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.497918+0000","last_change":"2024-09-16T09:27:49.965628+0000","last_active":"2024-09-16T09:32:18.497918+0000","last_peered":"2024-09-16T09:32:18.497918+0000","last_clean":"2024-09-16T09:32:18.497918+0000","last_became_active":"2024-09-16T09:25:05.715253+0000","last_became_peered":"2024-09-16T09:25:05.715253+0000","last_unstale":"2024-09-16T09:32:18.497918+0000","last_undegraded":"2024-09-16T09:32:18.497918+0000","last_fullsized":"2024-09-16T09:32:18.497918+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:49.965424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:49.965424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":309,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.495047+0000","last_change":"2024-09-16T09:27:56.059755+0000","last_active":"2024-09-16T09:32:18.495047+0000","last_peered":"2024-09-16T09:32:18.495047+0000","last_clean":"2024-09-16T09:32:18.495047+0000","last_became_active":"2024-09-16T09:25:07.718899+0000","last_became_peered":"2024-09-16T09:25:07.718899+0000","last_unstale":"2024-09-16T09:32:18.495047+0000","last_undegraded":"2024-09-16T09:32:18.495047+0000","last_fullsized":"2024-09-16T09:32:18.495047+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:56.059687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:56.059687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.d","version":"73'79","reported_seq":118,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.496997+0000","last_change":"2024-09-16T09:32:16.497595+0000","last_active":"2024-09-16T09:32:18.496997+0000","last_peered":"2024-09-16T09:32:18.496997+0000","last_clean":"2024-09-16T09:32:18.496997+0000","last_became_active":"2024-09-16T09:30:10.709958+0000","last_became_peered":"2024-09-16T09:30:10.709958+0000","last_unstale":"2024-09-16T09:32:18.496997+0000","last_undegraded":"2024-09-16T09:32:18.496997+0000","last_fullsized":"2024-09-16T09:32:18.496997+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:58.568841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:58.568841+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:32:44.442311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26835,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":10,"num_write":5,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":26835,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.4","version":"208'11849","reported_seq":11973,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:22.955442+0000","last_change":"2024-09-16T09:29:49.444857+0000","last_active":"2024-09-16T09:32:22.955442+0000","last_peered":"2024-09-16T09:32:22.955442+0000","last_clean":"2024-09-16T09:32:22.955442+0000","last_became_active":"2024-09-16T09:29:49.444515+0000","last_became_peered":"2024-09-16T09:29:49.444515+0000","last_unstale":"2024-09-16T09:32:22.955442+0000","last_undegraded":"2024-09-16T09:32:22.955442+0000","last_fullsized":"2024-09-16T09:32:22.955442+0000","mapping_epoch":152,"log_start":"189'7100","ondisk_log_start":"189'7100","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":4749,"log_dups_size":0,"ondisk_log_size":4749,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":246349824,"num_objects":3759,"num_object_clones":0,"num_object_copies":11277,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3759,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11849,"num_write_kb":499456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"66'15","reported_seq":341,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.495235+0000","last_change":"2024-09-16T09:26:33.138828+0000","last_active":"2024-09-16T09:32:18.495235+0000","last_peered":"2024-09-16T09:32:18.495235+0000","last_clean":"2024-09-16T09:32:18.495235+0000","last_became_active":"2024-09-16T09:25:03.715733+0000","last_became_peered":"2024-09-16T09:25:03.715733+0000","last_unstale":"2024-09-16T09:32:18.495235+0000","last_undegraded":"2024-09-16T09:32:18.495235+0000","last_fullsized":"2024-09-16T09:32:18.495235+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-16T09:26:33.138603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:26:33.138603+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":332,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.497821+0000","last_change":"2024-09-16T09:26:26.705847+0000","last_active":"2024-09-16T09:32:18.497821+0000","last_peered":"2024-09-16T09:32:18.497821+0000","last_clean":"2024-09-16T09:32:18.497821+0000","last_became_active":"2024-09-16T09:25:01.707372+0000","last_became_peered":"2024-09-16T09:25:01.707372+0000","last_unstale":"2024-09-16T09:32:18.497821+0000","last_undegraded":"2024-09-16T09:32:18.497821+0000","last_fullsized":"2024-09-16T09:32:18.497821+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-16T09:26:26.705773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:26.705773+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"66'8","reported_seq":426,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:21.848177+0000","last_change":"2024-09-16T09:30:05.138362+0000","last_active":"2024-09-16T09:32:21.848177+0000","last_peered":"2024-09-16T09:32:21.848177+0000","last_clean":"2024-09-16T09:32:21.848177+0000","last_became_active":"2024-09-16T09:25:05.716189+0000","last_became_peered":"2024-09-16T09:25:05.716189+0000","last_unstale":"2024-09-16T09:32:21.848177+0000","last_undegraded":"2024-09-16T09:32:21.848177+0000","last_fullsized":"2024-09-16T09:32:21.848177+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:30:05.138288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:30:05.138288+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":308,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.492772+0000","last_change":"2024-09-16T09:27:20.219376+0000","last_active":"2024-09-16T09:32:18.492772+0000","last_peered":"2024-09-16T09:32:18.492772+0000","last_clean":"2024-09-16T09:32:18.492772+0000","last_became_active":"2024-09-16T09:25:07.717213+0000","last_became_peered":"2024-09-16T09:25:07.717213+0000","last_unstale":"2024-09-16T09:32:18.492772+0000","last_undegraded":"2024-09-16T09:32:18.492772+0000","last_fullsized":"2024-09-16T09:32:18.492772+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:20.219333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:20.219333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.12","version":"73'79","reported_seq":117,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.493067+0000","last_change":"2024-09-16T09:32:16.490336+0000","last_active":"2024-09-16T09:32:18.493067+0000","last_peered":"2024-09-16T09:32:18.493067+0000","last_clean":"2024-09-16T09:32:18.493067+0000","last_became_active":"2024-09-16T09:30:10.731413+0000","last_became_peered":"2024-09-16T09:30:10.731413+0000","last_unstale":"2024-09-16T09:32:18.493067+0000","last_undegraded":"2024-09-16T09:32:18.493067+0000","last_fullsized":"2024-09-16T09:32:18.493067+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:26:35.745837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:26:35.745837+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:32:44.557232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26835,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":10,"num_write":6,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":26835,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":321,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.499059+0000","last_change":"2024-09-16T09:26:33.139405+0000","last_active":"2024-09-16T09:32:18.499059+0000","last_peered":"2024-09-16T09:32:18.499059+0000","last_clean":"2024-09-16T09:32:18.499059+0000","last_became_active":"2024-09-16T09:25:01.703202+0000","last_became_peered":"2024-09-16T09:25:01.703202+0000","last_unstale":"2024-09-16T09:32:18.499059+0000","last_undegraded":"2024-09-16T09:32:18.499059+0000","last_fullsized":"2024-09-16T09:32:18.499059+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:33.139360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:33.139360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:10.581208+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"66'6","reported_seq":62,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.506505+0000","last_change":"2024-09-16T09:31:09.297514+0000","last_active":"2024-09-16T09:32:18.506505+0000","last_peered":"2024-09-16T09:32:18.506505+0000","last_clean":"2024-09-16T09:32:18.506505+0000","last_became_active":"2024-09-16T09:31:09.270685+0000","last_became_peered":"2024-09-16T09:31:09.270685+0000","last_unstale":"2024-09-16T09:32:18.506505+0000","last_undegraded":"2024-09-16T09:32:18.506505+0000","last_fullsized":"2024-09-16T09:32:18.506505+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'6","last_scrub_stamp":"2024-09-16T09:26:23.458794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:26:23.458794+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":8,"num_read_kb":5,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":320,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.493188+0000","last_change":"2024-09-16T09:29:31.496374+0000","last_active":"2024-09-16T09:32:18.493188+0000","last_peered":"2024-09-16T09:32:18.493188+0000","last_clean":"2024-09-16T09:32:18.493188+0000","last_became_active":"2024-09-16T09:25:05.715616+0000","last_became_peered":"2024-09-16T09:25:05.715616+0000","last_unstale":"2024-09-16T09:32:18.493188+0000","last_undegraded":"2024-09-16T09:32:18.493188+0000","last_fullsized":"2024-09-16T09:32:18.493188+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:31.496304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:31.496304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":308,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.498138+0000","last_change":"2024-09-16T09:26:28.733381+0000","last_active":"2024-09-16T09:32:18.498138+0000","last_peered":"2024-09-16T09:32:18.498138+0000","last_clean":"2024-09-16T09:32:18.498138+0000","last_became_active":"2024-09-16T09:25:07.721059+0000","last_became_peered":"2024-09-16T09:25:07.721059+0000","last_unstale":"2024-09-16T09:32:18.498138+0000","last_undegraded":"2024-09-16T09:32:18.498138+0000","last_fullsized":"2024-09-16T09:32:18.498138+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:28.733324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:28.733324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.13","version":"73'79","reported_seq":303,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.496877+0000","last_change":"2024-09-16T09:31:09.288677+0000","last_active":"2024-09-16T09:32:18.496877+0000","last_peered":"2024-09-16T09:32:18.496877+0000","last_clean":"2024-09-16T09:32:18.496877+0000","last_became_active":"2024-09-16T09:31:09.288233+0000","last_became_peered":"2024-09-16T09:31:09.288233+0000","last_unstale":"2024-09-16T09:32:18.496877+0000","last_undegraded":"2024-09-16T09:32:18.496877+0000","last_fullsized":"2024-09-16T09:32:18.496877+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:30:21.714369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:30:21.714369+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.16","version":"66'9","reported_seq":391,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.494838+0000","last_change":"2024-09-16T09:32:01.319481+0000","last_active":"2024-09-16T09:32:18.494838+0000","last_peered":"2024-09-16T09:32:18.494838+0000","last_clean":"2024-09-16T09:32:18.494838+0000","last_became_active":"2024-09-16T09:31:09.273498+0000","last_became_peered":"2024-09-16T09:31:09.273498+0000","last_unstale":"2024-09-16T09:32:18.494838+0000","last_undegraded":"2024-09-16T09:32:18.494838+0000","last_fullsized":"2024-09-16T09:32:18.494838+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:32:01.319390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:32:01.319390+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:29.275572+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":328,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.497131+0000","last_change":"2024-09-16T09:28:50.698331+0000","last_active":"2024-09-16T09:32:18.497131+0000","last_peered":"2024-09-16T09:32:18.497131+0000","last_clean":"2024-09-16T09:32:18.497131+0000","last_became_active":"2024-09-16T09:25:01.706491+0000","last_became_peered":"2024-09-16T09:25:01.706491+0000","last_unstale":"2024-09-16T09:32:18.497131+0000","last_undegraded":"2024-09-16T09:32:18.497131+0000","last_fullsized":"2024-09-16T09:32:18.497131+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:50.698288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:28:50.698288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":178,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.496715+0000","last_change":"2024-09-16T09:31:09.271558+0000","last_active":"2024-09-16T09:32:18.496715+0000","last_peered":"2024-09-16T09:32:18.496715+0000","last_clean":"2024-09-16T09:32:18.496715+0000","last_became_active":"2024-09-16T09:31:09.271223+0000","last_became_peered":"2024-09-16T09:31:09.271223+0000","last_unstale":"2024-09-16T09:32:18.496715+0000","last_undegraded":"2024-09-16T09:32:18.496715+0000","last_fullsized":"2024-09-16T09:32:18.496715+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:53.810580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:28:53.810580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":309,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.499865+0000","last_change":"2024-09-16T09:26:35.866582+0000","last_active":"2024-09-16T09:32:18.499865+0000","last_peered":"2024-09-16T09:32:18.499865+0000","last_clean":"2024-09-16T09:32:18.499865+0000","last_became_active":"2024-09-16T09:25:07.722050+0000","last_became_peered":"2024-09-16T09:25:07.722050+0000","last_unstale":"2024-09-16T09:32:18.499865+0000","last_undegraded":"2024-09-16T09:32:18.499865+0000","last_fullsized":"2024-09-16T09:32:18.499865+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:35.866512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:35.866512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:17.938610+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.10","version":"73'79","reported_seq":117,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.492544+0000","last_change":"2024-09-16T09:32:16.491732+0000","last_active":"2024-09-16T09:32:18.492544+0000","last_peered":"2024-09-16T09:32:18.492544+0000","last_clean":"2024-09-16T09:32:18.492544+0000","last_became_active":"2024-09-16T09:30:10.692770+0000","last_became_peered":"2024-09-16T09:30:10.692770+0000","last_unstale":"2024-09-16T09:32:18.492544+0000","last_undegraded":"2024-09-16T09:32:18.492544+0000","last_fullsized":"2024-09-16T09:32:18.492544+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:02.723794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:02.723794+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:32:43.525182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26835,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":10,"num_write":5,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":26835,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"66'9","reported_seq":340,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.493487+0000","last_change":"2024-09-16T09:29:32.485165+0000","last_active":"2024-09-16T09:32:18.493487+0000","last_peered":"2024-09-16T09:32:18.493487+0000","last_clean":"2024-09-16T09:32:18.493487+0000","last_became_active":"2024-09-16T09:25:03.717310+0000","last_became_peered":"2024-09-16T09:25:03.717310+0000","last_unstale":"2024-09-16T09:32:18.493487+0000","last_undegraded":"2024-09-16T09:32:18.493487+0000","last_fullsized":"2024-09-16T09:32:18.493487+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:29:32.485084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:29:32.485084+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":351,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.494017+0000","last_change":"2024-09-16T09:31:59.376923+0000","last_active":"2024-09-16T09:32:18.494017+0000","last_peered":"2024-09-16T09:32:18.494017+0000","last_clean":"2024-09-16T09:32:18.494017+0000","last_became_active":"2024-09-16T09:31:09.272848+0000","last_became_peered":"2024-09-16T09:31:09.272848+0000","last_unstale":"2024-09-16T09:32:18.494017+0000","last_undegraded":"2024-09-16T09:32:18.494017+0000","last_fullsized":"2024-09-16T09:32:18.494017+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:31:59.376749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:31:59.376749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:13.111415+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"187'11","reported_seq":255,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:21.849628+0000","last_change":"2024-09-16T09:31:09.311243+0000","last_active":"2024-09-16T09:32:21.849628+0000","last_peered":"2024-09-16T09:32:21.849628+0000","last_clean":"2024-09-16T09:32:21.849628+0000","last_became_active":"2024-09-16T09:31:09.277688+0000","last_became_peered":"2024-09-16T09:31:09.277688+0000","last_unstale":"2024-09-16T09:32:21.849628+0000","last_undegraded":"2024-09-16T09:32:21.849628+0000","last_fullsized":"2024-09-16T09:32:21.849628+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:28:53.714413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:28:53.714413+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":325,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.485232+0000","last_change":"2024-09-16T09:29:49.474440+0000","last_active":"2024-09-16T09:32:18.485232+0000","last_peered":"2024-09-16T09:32:18.485232+0000","last_clean":"2024-09-16T09:32:18.485232+0000","last_became_active":"2024-09-16T09:25:07.718810+0000","last_became_peered":"2024-09-16T09:25:07.718810+0000","last_unstale":"2024-09-16T09:32:18.485232+0000","last_undegraded":"2024-09-16T09:32:18.485232+0000","last_fullsized":"2024-09-16T09:32:18.485232+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:49.474391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:49.474391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.11","version":"73'79","reported_seq":203,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.494314+0000","last_change":"2024-09-16T09:31:09.272938+0000","last_active":"2024-09-16T09:32:18.494314+0000","last_peered":"2024-09-16T09:32:18.494314+0000","last_clean":"2024-09-16T09:32:18.494314+0000","last_became_active":"2024-09-16T09:31:09.272208+0000","last_became_peered":"2024-09-16T09:31:09.272208+0000","last_unstale":"2024-09-16T09:32:18.494314+0000","last_undegraded":"2024-09-16T09:32:18.494314+0000","last_fullsized":"2024-09-16T09:32:18.494314+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:30:21.651575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:30:21.651575+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.14","version":"66'10","reported_seq":199,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.491748+0000","last_change":"2024-09-16T09:28:56.825675+0000","last_active":"2024-09-16T09:32:18.491748+0000","last_peered":"2024-09-16T09:32:18.491748+0000","last_clean":"2024-09-16T09:32:18.491748+0000","last_became_active":"2024-09-16T09:28:52.979059+0000","last_became_peered":"2024-09-16T09:28:52.979059+0000","last_unstale":"2024-09-16T09:32:18.491748+0000","last_undegraded":"2024-09-16T09:32:18.491748+0000","last_fullsized":"2024-09-16T09:32:18.491748+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-16T09:28:56.825476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:56.825476+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":14,"num_read_kb":9,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":328,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.497188+0000","last_change":"2024-09-16T09:29:56.021413+0000","last_active":"2024-09-16T09:32:18.497188+0000","last_peered":"2024-09-16T09:32:18.497188+0000","last_clean":"2024-09-16T09:32:18.497188+0000","last_became_active":"2024-09-16T09:25:01.707513+0000","last_became_peered":"2024-09-16T09:25:01.707513+0000","last_unstale":"2024-09-16T09:32:18.497188+0000","last_undegraded":"2024-09-16T09:32:18.497188+0000","last_fullsized":"2024-09-16T09:32:18.497188+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:56.021340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:56.021340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"66'8","reported_seq":425,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:21.849472+0000","last_change":"2024-09-16T09:29:48.485781+0000","last_active":"2024-09-16T09:32:21.849472+0000","last_peered":"2024-09-16T09:32:21.849472+0000","last_clean":"2024-09-16T09:32:21.849472+0000","last_became_active":"2024-09-16T09:25:05.713833+0000","last_became_peered":"2024-09-16T09:25:05.713833+0000","last_unstale":"2024-09-16T09:32:21.849472+0000","last_undegraded":"2024-09-16T09:32:21.849472+0000","last_fullsized":"2024-09-16T09:32:21.849472+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:29:48.485727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.485727+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":339,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.494369+0000","last_change":"2024-09-16T09:32:00.343429+0000","last_active":"2024-09-16T09:32:18.494369+0000","last_peered":"2024-09-16T09:32:18.494369+0000","last_clean":"2024-09-16T09:32:18.494369+0000","last_became_active":"2024-09-16T09:31:09.275168+0000","last_became_peered":"2024-09-16T09:31:09.275168+0000","last_unstale":"2024-09-16T09:32:18.494369+0000","last_undegraded":"2024-09-16T09:32:18.494369+0000","last_fullsized":"2024-09-16T09:32:18.494369+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:00.343356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:32:00.343356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:23.568990+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"66'11","reported_seq":347,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.485926+0000","last_change":"2024-09-16T09:26:33.302713+0000","last_active":"2024-09-16T09:32:18.485926+0000","last_peered":"2024-09-16T09:32:18.485926+0000","last_clean":"2024-09-16T09:32:18.485926+0000","last_became_active":"2024-09-16T09:26:33.291265+0000","last_became_peered":"2024-09-16T09:26:33.291265+0000","last_unstale":"2024-09-16T09:32:18.485926+0000","last_undegraded":"2024-09-16T09:32:18.485926+0000","last_fullsized":"2024-09-16T09:32:18.485926+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:08.864166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":337,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.486324+0000","last_change":"2024-09-16T09:29:51.517112+0000","last_active":"2024-09-16T09:32:18.486324+0000","last_peered":"2024-09-16T09:32:18.486324+0000","last_clean":"2024-09-16T09:32:18.486324+0000","last_became_active":"2024-09-16T09:25:01.705158+0000","last_became_peered":"2024-09-16T09:25:01.705158+0000","last_unstale":"2024-09-16T09:32:18.486324+0000","last_undegraded":"2024-09-16T09:32:18.486324+0000","last_fullsized":"2024-09-16T09:32:18.486324+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:51.517054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:51.517054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":343,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.498370+0000","last_change":"2024-09-16T09:31:09.266820+0000","last_active":"2024-09-16T09:32:18.498370+0000","last_peered":"2024-09-16T09:32:18.498370+0000","last_clean":"2024-09-16T09:32:18.498370+0000","last_became_active":"2024-09-16T09:31:09.266326+0000","last_became_peered":"2024-09-16T09:31:09.266326+0000","last_unstale":"2024-09-16T09:32:18.498370+0000","last_undegraded":"2024-09-16T09:32:18.498370+0000","last_fullsized":"2024-09-16T09:32:18.498370+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:15.910824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:15.910824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":171,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.494099+0000","last_change":"2024-09-16T09:31:09.277239+0000","last_active":"2024-09-16T09:32:18.494099+0000","last_peered":"2024-09-16T09:32:18.494099+0000","last_clean":"2024-09-16T09:32:18.494099+0000","last_became_active":"2024-09-16T09:31:09.276042+0000","last_became_peered":"2024-09-16T09:31:09.276042+0000","last_unstale":"2024-09-16T09:32:18.494099+0000","last_undegraded":"2024-09-16T09:32:18.494099+0000","last_fullsized":"2024-09-16T09:32:18.494099+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:48.147759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:48.147759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"66'9","reported_seq":333,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.498587+0000","last_change":"2024-09-16T09:26:31.216495+0000","last_active":"2024-09-16T09:32:18.498587+0000","last_peered":"2024-09-16T09:32:18.498587+0000","last_clean":"2024-09-16T09:32:18.498587+0000","last_became_active":"2024-09-16T09:25:03.705807+0000","last_became_peered":"2024-09-16T09:25:03.705807+0000","last_unstale":"2024-09-16T09:32:18.498587+0000","last_undegraded":"2024-09-16T09:32:18.498587+0000","last_fullsized":"2024-09-16T09:32:18.498587+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:26:31.216298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:26:31.216298+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":320,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.498406+0000","last_change":"2024-09-16T09:26:23.796308+0000","last_active":"2024-09-16T09:32:18.498406+0000","last_peered":"2024-09-16T09:32:18.498406+0000","last_clean":"2024-09-16T09:32:18.498406+0000","last_became_active":"2024-09-16T09:25:01.706762+0000","last_became_peered":"2024-09-16T09:25:01.706762+0000","last_unstale":"2024-09-16T09:32:18.498406+0000","last_undegraded":"2024-09-16T09:32:18.498406+0000","last_fullsized":"2024-09-16T09:32:18.498406+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:23.796231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:23.796231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":156,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.494097+0000","last_change":"2024-09-16T09:28:52.983144+0000","last_active":"2024-09-16T09:32:18.494097+0000","last_peered":"2024-09-16T09:32:18.494097+0000","last_clean":"2024-09-16T09:32:18.494097+0000","last_became_active":"2024-09-16T09:28:52.982865+0000","last_became_peered":"2024-09-16T09:28:52.982865+0000","last_unstale":"2024-09-16T09:32:18.494097+0000","last_undegraded":"2024-09-16T09:32:18.494097+0000","last_fullsized":"2024-09-16T09:32:18.494097+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:44.241942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:44.241942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":317,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.492985+0000","last_change":"2024-09-16T09:28:52.552567+0000","last_active":"2024-09-16T09:32:18.492985+0000","last_peered":"2024-09-16T09:32:18.492985+0000","last_clean":"2024-09-16T09:32:18.492985+0000","last_became_active":"2024-09-16T09:25:07.715301+0000","last_became_peered":"2024-09-16T09:25:07.715301+0000","last_unstale":"2024-09-16T09:32:18.492985+0000","last_undegraded":"2024-09-16T09:32:18.492985+0000","last_fullsized":"2024-09-16T09:32:18.492985+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:52.552495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:28:52.552495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:45.269179+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.14","version":"73'79","reported_seq":117,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.499527+0000","last_change":"2024-09-16T09:32:16.503276+0000","last_active":"2024-09-16T09:32:18.499527+0000","last_peered":"2024-09-16T09:32:18.499527+0000","last_clean":"2024-09-16T09:32:18.499527+0000","last_became_active":"2024-09-16T09:30:10.700866+0000","last_became_peered":"2024-09-16T09:30:10.700866+0000","last_unstale":"2024-09-16T09:32:18.499527+0000","last_undegraded":"2024-09-16T09:32:18.499527+0000","last_fullsized":"2024-09-16T09:32:18.499527+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:53.916505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:53.916505+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:32:43.945820+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109227,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"66'11","reported_seq":203,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.497039+0000","last_change":"2024-09-16T09:28:55.643505+0000","last_active":"2024-09-16T09:32:18.497039+0000","last_peered":"2024-09-16T09:32:18.497039+0000","last_clean":"2024-09-16T09:32:18.497039+0000","last_became_active":"2024-09-16T09:28:52.971132+0000","last_became_peered":"2024-09-16T09:28:52.971132+0000","last_unstale":"2024-09-16T09:32:18.497039+0000","last_undegraded":"2024-09-16T09:32:18.497039+0000","last_fullsized":"2024-09-16T09:32:18.497039+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-16T09:28:55.643448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:55.643448+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":19,"num_read_kb":12,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":328,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.493234+0000","last_change":"2024-09-16T09:27:46.175159+0000","last_active":"2024-09-16T09:32:18.493234+0000","last_peered":"2024-09-16T09:32:18.493234+0000","last_clean":"2024-09-16T09:32:18.493234+0000","last_became_active":"2024-09-16T09:25:01.700733+0000","last_became_peered":"2024-09-16T09:25:01.700733+0000","last_unstale":"2024-09-16T09:32:18.493234+0000","last_undegraded":"2024-09-16T09:32:18.493234+0000","last_fullsized":"2024-09-16T09:32:18.493234+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:46.175093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:27:46.175093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":320,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.501125+0000","last_change":"2024-09-16T09:29:25.017736+0000","last_active":"2024-09-16T09:32:18.501125+0000","last_peered":"2024-09-16T09:32:18.501125+0000","last_clean":"2024-09-16T09:32:18.501125+0000","last_became_active":"2024-09-16T09:25:05.713479+0000","last_became_peered":"2024-09-16T09:25:05.713479+0000","last_unstale":"2024-09-16T09:32:18.501125+0000","last_undegraded":"2024-09-16T09:32:18.501125+0000","last_fullsized":"2024-09-16T09:32:18.501125+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:25.017610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:25.017610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":179,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.494517+0000","last_change":"2024-09-16T09:31:09.275121+0000","last_active":"2024-09-16T09:32:18.494517+0000","last_peered":"2024-09-16T09:32:18.494517+0000","last_clean":"2024-09-16T09:32:18.494517+0000","last_became_active":"2024-09-16T09:31:09.274797+0000","last_became_peered":"2024-09-16T09:31:09.274797+0000","last_unstale":"2024-09-16T09:32:18.494517+0000","last_undegraded":"2024-09-16T09:32:18.494517+0000","last_fullsized":"2024-09-16T09:32:18.494517+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:54.589258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:28:54.589258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:48.251310+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"66'4","reported_seq":60,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.489179+0000","last_change":"2024-09-16T09:31:09.401626+0000","last_active":"2024-09-16T09:32:18.489179+0000","last_peered":"2024-09-16T09:32:18.489179+0000","last_clean":"2024-09-16T09:32:18.489179+0000","last_became_active":"2024-09-16T09:31:09.273682+0000","last_became_peered":"2024-09-16T09:31:09.273682+0000","last_unstale":"2024-09-16T09:32:18.489179+0000","last_undegraded":"2024-09-16T09:32:18.489179+0000","last_fullsized":"2024-09-16T09:32:18.489179+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'4","last_scrub_stamp":"2024-09-16T09:26:27.473740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:26:27.473740+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":351,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.495340+0000","last_change":"2024-09-16T09:31:30.235761+0000","last_active":"2024-09-16T09:32:18.495340+0000","last_peered":"2024-09-16T09:32:18.495340+0000","last_clean":"2024-09-16T09:32:18.495340+0000","last_became_active":"2024-09-16T09:31:09.274719+0000","last_became_peered":"2024-09-16T09:31:09.274719+0000","last_unstale":"2024-09-16T09:32:18.495340+0000","last_undegraded":"2024-09-16T09:32:18.495340+0000","last_fullsized":"2024-09-16T09:32:18.495340+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:31:30.235692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:31:30.235692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:49.736157+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":337,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.498158+0000","last_change":"2024-09-16T09:32:07.533575+0000","last_active":"2024-09-16T09:32:18.498158+0000","last_peered":"2024-09-16T09:32:18.498158+0000","last_clean":"2024-09-16T09:32:18.498158+0000","last_became_active":"2024-09-16T09:25:05.712545+0000","last_became_peered":"2024-09-16T09:25:05.712545+0000","last_unstale":"2024-09-16T09:32:18.498158+0000","last_undegraded":"2024-09-16T09:32:18.498158+0000","last_fullsized":"2024-09-16T09:32:18.498158+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:07.533529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:32:07.533529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:28.807212+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":316,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.496925+0000","last_change":"2024-09-16T09:29:27.012810+0000","last_active":"2024-09-16T09:32:18.496925+0000","last_peered":"2024-09-16T09:32:18.496925+0000","last_clean":"2024-09-16T09:32:18.496925+0000","last_became_active":"2024-09-16T09:25:07.720117+0000","last_became_peered":"2024-09-16T09:25:07.720117+0000","last_unstale":"2024-09-16T09:32:18.496925+0000","last_undegraded":"2024-09-16T09:32:18.496925+0000","last_fullsized":"2024-09-16T09:32:18.496925+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:27.012756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:27.012756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":316,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.492171+0000","last_change":"2024-09-16T09:29:19.888825+0000","last_active":"2024-09-16T09:32:18.492171+0000","last_peered":"2024-09-16T09:32:18.492171+0000","last_clean":"2024-09-16T09:32:18.492171+0000","last_became_active":"2024-09-16T09:25:07.712594+0000","last_became_peered":"2024-09-16T09:25:07.712594+0000","last_unstale":"2024-09-16T09:32:18.492171+0000","last_undegraded":"2024-09-16T09:32:18.492171+0000","last_fullsized":"2024-09-16T09:32:18.492171+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:19.888759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:19.888759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":178,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.493532+0000","last_change":"2024-09-16T09:31:26.211625+0000","last_active":"2024-09-16T09:32:18.493532+0000","last_peered":"2024-09-16T09:32:18.493532+0000","last_clean":"2024-09-16T09:32:18.493532+0000","last_became_active":"2024-09-16T09:31:09.271662+0000","last_became_peered":"2024-09-16T09:31:09.271662+0000","last_unstale":"2024-09-16T09:32:18.493532+0000","last_undegraded":"2024-09-16T09:32:18.493532+0000","last_fullsized":"2024-09-16T09:32:18.493532+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:31:26.211363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:31:26.211363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:32.873753+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"66'11","reported_seq":344,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.498694+0000","last_change":"2024-09-16T09:27:57.486459+0000","last_active":"2024-09-16T09:32:18.498694+0000","last_peered":"2024-09-16T09:32:18.498694+0000","last_clean":"2024-09-16T09:32:18.498694+0000","last_became_active":"2024-09-16T09:25:03.707769+0000","last_became_peered":"2024-09-16T09:25:03.707769+0000","last_unstale":"2024-09-16T09:32:18.498694+0000","last_undegraded":"2024-09-16T09:32:18.498694+0000","last_fullsized":"2024-09-16T09:32:18.498694+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-16T09:27:57.486381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:57.486381+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:17.472949+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":313,"reported_epoch":208,"state":"active+clean","last_fresh":"2024-09-16T09:32:18.494584+0000","last_change":"2024-09-16T09:27:57.640962+0000","last_active":"2024-09-16T09:32:18.494584+0000","last_peered":"2024-09-16T09:32:18.494584+0000","last_clean":"2024-09-16T09:32:18.494584+0000","last_became_active":"2024-09-16T09:25:05.711952+0000","last_became_peered":"2024-09-16T09:25:05.711952+0000","last_unstale":"2024-09-16T09:32:18.494584+0000","last_undegraded":"2024-09-16T09:32:18.494584+0000","last_fullsized":"2024-09-16T09:32:18.494584+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:57.640756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:57.640756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:21.189003+0000","scrub_duration":44,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":16,"stat_sum":{"num_bytes":3964928024,"num_objects":60501,"num_object_clones":0,"num_object_copies":181503,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2526,"num_objects_unfound":0,"num_objects_dirty":60501,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":188685,"num_write_kb":7973889,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12183,"num_bytes_recovered":798425088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12242399232,"data_stored":12242387016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":77772,"ondisk_log_size":77772,"up":48,"acting":46,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":70,"ondisk_log_size":70,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":691,"num_read_kb":448,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":126,"num_bytes_recovered":3026,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":395,"ondisk_log_size":395,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1986,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":440,"num_read_kb":435,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":21,"stat_sum":{"num_bytes":806935,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":21,"num_read_kb":40,"num_write":21,"num_write_kb":296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":960396,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1659,"ondisk_log_size":1659,"up":63,"acting":63,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266782,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1375824,"kb_used_data":1123904,"kb_used_omap":1,"kb_used_meta":251902,"kb_avail":92369328,"statfs":{"total":95995035648,"available":94586191872,"internally_reserved":0,"allocated":1150877696,"data_stored":1150569113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":257948106},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495715,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1325940,"kb_used_data":1124004,"kb_used_omap":1,"kb_used_meta":201918,"kb_avail":92419212,"statfs":{"total":95995035648,"available":94637273088,"internally_reserved":0,"allocated":1150980096,"data_stored":1150699063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":206764486},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724649,"num_pgs":64,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2241220,"kb_used_data":1971252,"kb_used_omap":1,"kb_used_meta":269950,"kb_avail":91503932,"statfs":{"total":95995035648,"available":93700026368,"internally_reserved":0,"allocated":2018562048,"data_stored":2018265668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":276429254},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,4],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953582,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1737616,"kb_used_data":1539840,"kb_used_omap":1,"kb_used_meta":197758,"kb_avail":92007536,"statfs":{"total":95995035648,"available":94215716864,"internally_reserved":0,"allocated":1576796160,"data_stored":1576487617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":202504646},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149812,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1639964,"kb_used_data":1311692,"kb_used_omap":1,"kb_used_meta":328254,"kb_avail":92105188,"statfs":{"total":95995035648,"available":94315712512,"internally_reserved":0,"allocated":1343172608,"data_stored":1342852607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":336132551},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411449,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1791292,"kb_used_data":1503212,"kb_used_omap":1,"kb_used_meta":287998,"kb_avail":91953860,"statfs":{"total":95995035648,"available":94160752640,"internally_reserved":0,"allocated":1539289088,"data_stored":1539000363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":294910405},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,1,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574976,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2023144,"kb_used_data":1743448,"kb_used_omap":1,"kb_used_meta":279678,"kb_avail":91722008,"statfs":{"total":95995035648,"available":93923336192,"internally_reserved":0,"allocated":1785290752,"data_stored":1784957503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":286390725},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2],"upper_bound":8},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738502,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2076112,"kb_used_data":1774208,"kb_used_omap":1,"kb_used_meta":301886,"kb_avail":91669040,"statfs":{"total":95995035648,"available":93869096960,"internally_reserved":0,"allocated":1816788992,"data_stored":1816480651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":309131717},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1789001728,"data_stored":1789001728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1760034816,"data_stored":1760034816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1512574976,"data_stored":1512570904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1336344576,"data_stored":1336344576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1551564800,"data_stored":1551564800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1995640832,"data_stored":1995636760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1148059648,"data_stored":1148059648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1149177856,"data_stored":1149173784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T09:32:31.926 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:32:31.926 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:31.926 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:31.927 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:32:32.027 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:32:32.071 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:32.071 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:32.071 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:32:32.129 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:32:31 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:32:31] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:32:32.172 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:32:32.210 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:32.211 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:32.211 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:32:32.312 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:32:32.333 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:32 smithi084 ceph-mon[32629]: 4.1b scrub starts 2024-09-16T09:32:32.333 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:32 smithi084 ceph-mon[32629]: 4.1b scrub ok 2024-09-16T09:32:32.333 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:32 smithi084 ceph-mon[32629]: 5.1e scrub starts 2024-09-16T09:32:32.333 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:32 smithi084 ceph-mon[32629]: 5.1e scrub ok 2024-09-16T09:32:32.340 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:32.341 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:32.341 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:32:32.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:32 smithi070 ceph-mon[30388]: 4.1b scrub starts 2024-09-16T09:32:32.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:32 smithi070 ceph-mon[30388]: 4.1b scrub ok 2024-09-16T09:32:32.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:32 smithi070 ceph-mon[30388]: 5.1e scrub starts 2024-09-16T09:32:32.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:32 smithi070 ceph-mon[30388]: 5.1e scrub ok 2024-09-16T09:32:32.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:32 smithi070 ceph-mon[40691]: 4.1b scrub starts 2024-09-16T09:32:32.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:32 smithi070 ceph-mon[40691]: 4.1b scrub ok 2024-09-16T09:32:32.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:32 smithi070 ceph-mon[40691]: 5.1e scrub starts 2024-09-16T09:32:32.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:32 smithi070 ceph-mon[40691]: 5.1e scrub ok 2024-09-16T09:32:32.442 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:32:32.474 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:32.474 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:32.474 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:32:32.575 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:32:32.607 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:32.607 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:32.607 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:32:32.658 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-16T09:32:32.708 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:32:32.737 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:32.738 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:32.738 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:32:32.839 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:32:32.880 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:32.882 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:32.882 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:32:32.983 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:32:33.037 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:33.037 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:33.038 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:32:33.139 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:32:33.160 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:32:32 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:32:33.170 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:33.170 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:33.172 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:32:33.273 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:32:33.308 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:33.308 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:33.309 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:32:33.410 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:32:33.417 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:33 smithi070 ceph-mon[40691]: pgmap v349: 168 pgs: 2 active+recovering+undersized+remapped, 166 active+clean; 3.4 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.18k op/s; 2090/166434 objects misplaced (1.256%); 5.2 MiB/s, 83 objects/s recovering 2024-09-16T09:32:33.417 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:33 smithi070 ceph-mon[40691]: 5.6 scrub starts 2024-09-16T09:32:33.417 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:33 smithi070 ceph-mon[40691]: 5.6 scrub ok 2024-09-16T09:32:33.417 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:33 smithi070 ceph-mon[40691]: 6.2 scrub starts 2024-09-16T09:32:33.417 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:33 smithi070 ceph-mon[40691]: 6.2 scrub ok 2024-09-16T09:32:33.417 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:33 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:32:33.417 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:33 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3280708340' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:32:33.418 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:33 smithi070 ceph-mon[30388]: pgmap v349: 168 pgs: 2 active+recovering+undersized+remapped, 166 active+clean; 3.4 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.18k op/s; 2090/166434 objects misplaced (1.256%); 5.2 MiB/s, 83 objects/s recovering 2024-09-16T09:32:33.418 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:33 smithi070 ceph-mon[30388]: 5.6 scrub starts 2024-09-16T09:32:33.418 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:33 smithi070 ceph-mon[30388]: 5.6 scrub ok 2024-09-16T09:32:33.418 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:33 smithi070 ceph-mon[30388]: 6.2 scrub starts 2024-09-16T09:32:33.418 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:33 smithi070 ceph-mon[30388]: 6.2 scrub ok 2024-09-16T09:32:33.418 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:33 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:32:33.418 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:33 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3280708340' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:32:33.433 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:33.433 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:33.434 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:32:33.535 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:32:33.570 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:33.571 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:33.571 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:32:33.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:33 smithi084 ceph-mon[32629]: pgmap v349: 168 pgs: 2 active+recovering+undersized+remapped, 166 active+clean; 3.4 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.18k op/s; 2090/166434 objects misplaced (1.256%); 5.2 MiB/s, 83 objects/s recovering 2024-09-16T09:32:33.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:33 smithi084 ceph-mon[32629]: 5.6 scrub starts 2024-09-16T09:32:33.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:33 smithi084 ceph-mon[32629]: 5.6 scrub ok 2024-09-16T09:32:33.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:33 smithi084 ceph-mon[32629]: 6.2 scrub starts 2024-09-16T09:32:33.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:33 smithi084 ceph-mon[32629]: 6.2 scrub ok 2024-09-16T09:32:33.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:33 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:32:33.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:33 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3280708340' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:32:33.672 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:32:33.707 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:33.708 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:33.708 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:32:33.809 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:32:33.845 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:33.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:33.847 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:32:33.919 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:32:33.919 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:32:33.919 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:32:33.919 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:32:33.919 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:32:33.919 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:32:33.920 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:32:33.920 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:32:33.920 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:32:33.920 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:32:33.920 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:32:33.920 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:32:33.920 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:32:33.921 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:32:33.921 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:32:33.921 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:32:33.948 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:32:33.987 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:33.988 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:33.988 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:32:34.089 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:32:34.119 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:34.120 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:34.120 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:32:34.125 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:32:34.221 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:32:34.267 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:34.268 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:34.268 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:32:34.369 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:32:34.406 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:34.406 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:34.407 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:32:34.507 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:32:34.546 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:34.548 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:34.548 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:32:34.650 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:32:34.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:34 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:32:34.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:34 smithi084 ceph-mon[32629]: 6.0 scrub starts 2024-09-16T09:32:34.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:34 smithi084 ceph-mon[32629]: 6.0 scrub ok 2024-09-16T09:32:34.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:34 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:32:34.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:34 smithi070 ceph-mon[30388]: 6.0 scrub starts 2024-09-16T09:32:34.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:34 smithi070 ceph-mon[30388]: 6.0 scrub ok 2024-09-16T09:32:34.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:34 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:32:34.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:34 smithi070 ceph-mon[40691]: 6.0 scrub starts 2024-09-16T09:32:34.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:34 smithi070 ceph-mon[40691]: 6.0 scrub ok 2024-09-16T09:32:34.678 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:34.679 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:34.680 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:32:34.780 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:32:34.818 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:34.819 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:34.819 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:32:34.920 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:32:34.921 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-16T09:32:34.965 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:34.965 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:34.966 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:32:35.067 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:32:35.096 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:35.096 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:35.097 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:32:35.198 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:32:35.263 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:35.263 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:35.264 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:32:35.340 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T09:32:35.340 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 20, 2024-09-16T09:32:35.340 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 600, 2024-09-16T09:32:35.340 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [ 2024-09-16T09:32:35.340 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.340 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.24698.0:27463 7.b 7:daab829c:::benchmark_data_smithi070_78438_object27462:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T09:32:35.340 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:26:23.719843+0000", 2024-09-16T09:32:35.341 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 371.61468613199997, 2024-09-16T09:32:35.341 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.073101114999999994, 2024-09-16T09:32:35.341 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:32:35.341 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:32:35.341 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:32:35.341 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:32:35.341 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.24698", 2024-09-16T09:32:35.341 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:32:35.341 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 27463 2024-09-16T09:32:35.341 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.341 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:32:35.341 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.341 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:32:35.342 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:23.719843+0000", 2024-09-16T09:32:35.342 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:32:35.342 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.342 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.342 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:32:35.342 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:23.719843+0000", 2024-09-16T09:32:35.342 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:32:35.342 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.342 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.342 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:32:35.342 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:23.719844+0000", 2024-09-16T09:32:35.342 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.579e-06 2024-09-16T09:32:35.342 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.343 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.343 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:32:35.343 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:23.719867+0000", 2024-09-16T09:32:35.343 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.2972e-05 2024-09-16T09:32:35.343 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.343 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.343 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:32:35.343 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:23.719884+0000", 2024-09-16T09:32:35.343 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7190999999999999e-05 2024-09-16T09:32:35.343 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.343 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.343 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:32:35.343 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:23.719892+0000", 2024-09-16T09:32:35.344 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.7309999999999993e-06 2024-09-16T09:32:35.344 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.344 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.344 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:32:35.344 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:23.719950+0000", 2024-09-16T09:32:35.344 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7933000000000003e-05 2024-09-16T09:32:35.344 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.344 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.344 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:32:35.344 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:23.720065+0000", 2024-09-16T09:32:35.344 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00011503 2024-09-16T09:32:35.344 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.345 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.345 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 5,6", 2024-09-16T09:32:35.345 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:23.720191+0000", 2024-09-16T09:32:35.345 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000125396 2024-09-16T09:32:35.345 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.345 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.345 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:32:35.345 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:23.721401+0000", 2024-09-16T09:32:35.345 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001210668 2024-09-16T09:32:35.345 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.345 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.345 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:32:35.345 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:23.721589+0000", 2024-09-16T09:32:35.345 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00018812399999999999 2024-09-16T09:32:35.346 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.346 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.346 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:32:35.346 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:23.792843+0000", 2024-09-16T09:32:35.346 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.071254182999999999 2024-09-16T09:32:35.346 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.346 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.346 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:32:35.346 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:23.792901+0000", 2024-09-16T09:32:35.346 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7116e-05 2024-09-16T09:32:35.346 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.346 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.346 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:32:35.346 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:26:23.792944+0000", 2024-09-16T09:32:35.347 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.3192000000000003e-05 2024-09-16T09:32:35.347 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:32:35.347 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:32:35.347 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:32:35.347 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.347 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.347 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.24698.0:203418 7.4 e135/82 7:263e3061:::benchmark_data_smithi070_78438_object77287:head v 135'12525, mlcod=135'12523)", 2024-09-16T09:32:35.347 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:28:51.971069+0000", 2024-09-16T09:32:35.347 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 223.363459775, 2024-09-16T09:32:35.347 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.171156426, 2024-09-16T09:32:35.347 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:32:35.347 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:32:35.348 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:32:35.348 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:32:35.348 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.348 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:32:35.348 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.971069+0000", 2024-09-16T09:32:35.348 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:32:35.348 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.348 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.348 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:32:35.348 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.971069+0000", 2024-09-16T09:32:35.348 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:32:35.348 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.348 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.349 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:32:35.349 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.971071+0000", 2024-09-16T09:32:35.349 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5430000000000001e-06 2024-09-16T09:32:35.349 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.349 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.349 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:32:35.349 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.971079+0000", 2024-09-16T09:32:35.349 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.191e-06 2024-09-16T09:32:35.349 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.349 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.349 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:32:35.349 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.971098+0000", 2024-09-16T09:32:35.349 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9249000000000002e-05 2024-09-16T09:32:35.350 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.350 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.350 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:32:35.350 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.971106+0000", 2024-09-16T09:32:35.350 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.3089999999999996e-06 2024-09-16T09:32:35.350 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.350 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.350 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:32:35.350 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.971774+0000", 2024-09-16T09:32:35.350 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00066768999999999997 2024-09-16T09:32:35.350 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.350 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.350 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:32:35.350 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.971794+0000", 2024-09-16T09:32:35.351 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9658000000000001e-05 2024-09-16T09:32:35.351 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.351 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.351 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:32:35.351 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.142182+0000", 2024-09-16T09:32:35.351 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.17038833 2024-09-16T09:32:35.351 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.351 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.351 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:32:35.351 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.142225+0000", 2024-09-16T09:32:35.351 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.3455999999999999e-05 2024-09-16T09:32:35.351 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:32:35.351 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:32:35.352 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:32:35.352 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.352 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.352 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.24698.0:203421 7.4 e135/82 7:243428a8:::benchmark_data_smithi070_78438_object77290:head v 135'12526, mlcod=135'12523)", 2024-09-16T09:32:35.352 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:28:51.971129+0000", 2024-09-16T09:32:35.352 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 223.363399423, 2024-09-16T09:32:35.352 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.17129213099999999, 2024-09-16T09:32:35.352 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:32:35.352 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:32:35.352 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:32:35.352 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:32:35.352 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.352 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:32:35.353 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.971129+0000", 2024-09-16T09:32:35.353 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:32:35.353 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.353 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.353 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:32:35.353 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.971129+0000", 2024-09-16T09:32:35.353 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:32:35.353 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.353 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.353 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:32:35.353 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.971130+0000", 2024-09-16T09:32:35.353 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.0800000000000001e-07 2024-09-16T09:32:35.353 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.353 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.354 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:32:35.354 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.971133+0000", 2024-09-16T09:32:35.354 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1709999999999998e-06 2024-09-16T09:32:35.354 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.354 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.354 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:32:35.354 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.971138+0000", 2024-09-16T09:32:35.354 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5480000000000001e-06 2024-09-16T09:32:35.354 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.354 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.354 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:32:35.354 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.971142+0000", 2024-09-16T09:32:35.354 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9720000000000003e-06 2024-09-16T09:32:35.355 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.355 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.355 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:32:35.355 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.981033+0000", 2024-09-16T09:32:35.355 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0098909940000000002 2024-09-16T09:32:35.355 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.355 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.355 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:32:35.355 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.981096+0000", 2024-09-16T09:32:35.355 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.2539000000000006e-05 2024-09-16T09:32:35.355 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.355 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.355 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:32:35.355 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.142341+0000", 2024-09-16T09:32:35.356 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.16124539900000001 2024-09-16T09:32:35.356 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.356 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.356 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:32:35.356 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.142422+0000", 2024-09-16T09:32:35.356 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.0199999999999998e-05 2024-09-16T09:32:35.356 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:32:35.356 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:32:35.356 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:32:35.356 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.356 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.356 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.24698.0:203423 7.4 e135/82 7:27039506:::benchmark_data_smithi070_78438_object77292:head v 135'12527, mlcod=135'12523)", 2024-09-16T09:32:35.356 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:28:51.971148+0000", 2024-09-16T09:32:35.356 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 223.36338056899999, 2024-09-16T09:32:35.357 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.17133706500000001, 2024-09-16T09:32:35.357 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:32:35.357 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:32:35.357 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:32:35.357 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:32:35.357 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.357 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:32:35.357 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.971148+0000", 2024-09-16T09:32:35.357 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:32:35.357 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.357 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.357 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:32:35.357 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.971148+0000", 2024-09-16T09:32:35.358 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:32:35.358 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.358 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.358 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:32:35.358 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.971148+0000", 2024-09-16T09:32:35.358 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.74e-07 2024-09-16T09:32:35.358 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.358 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.358 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:32:35.358 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.971156+0000", 2024-09-16T09:32:35.358 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.2609999999999996e-06 2024-09-16T09:32:35.358 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.358 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.358 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:32:35.359 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.971161+0000", 2024-09-16T09:32:35.359 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.8609999999999997e-06 2024-09-16T09:32:35.359 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.359 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.359 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:32:35.359 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.971164+0000", 2024-09-16T09:32:35.359 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4400000000000001e-06 2024-09-16T09:32:35.359 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.359 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.359 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:32:35.359 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.986901+0000", 2024-09-16T09:32:35.359 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.01573751 2024-09-16T09:32:35.359 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.359 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.359 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:32:35.360 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.986922+0000", 2024-09-16T09:32:35.360 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0542000000000001e-05 2024-09-16T09:32:35.360 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.360 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.360 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:32:35.360 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.142424+0000", 2024-09-16T09:32:35.360 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.155502057 2024-09-16T09:32:35.360 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.360 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.360 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:32:35.360 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.142485+0000", 2024-09-16T09:32:35.360 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.122e-05 2024-09-16T09:32:35.360 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:32:35.360 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:32:35.361 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:32:35.361 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.361 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.361 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.24698.0:203436 7.4 e136/82 7:2eae91c1:::benchmark_data_smithi070_78438_object77305:head v 136'12528, mlcod=135'12523)", 2024-09-16T09:32:35.361 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:28:51.988508+0000", 2024-09-16T09:32:35.361 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 223.34602114500001, 2024-09-16T09:32:35.361 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.16337481500000001, 2024-09-16T09:32:35.361 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:32:35.361 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:32:35.362 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:32:35.362 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:32:35.362 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.362 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:32:35.362 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.988508+0000", 2024-09-16T09:32:35.362 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:32:35.362 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.362 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.362 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:32:35.362 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.988508+0000", 2024-09-16T09:32:35.362 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:32:35.362 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.362 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.363 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:32:35.363 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.988509+0000", 2024-09-16T09:32:35.363 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.02e-06 2024-09-16T09:32:35.363 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.363 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.363 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:32:35.363 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.988515+0000", 2024-09-16T09:32:35.363 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.9950000000000002e-06 2024-09-16T09:32:35.363 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.363 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.363 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:32:35.363 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.116690+0000", 2024-09-16T09:32:35.363 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.12817552700000001 2024-09-16T09:32:35.363 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.364 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.364 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:32:35.364 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.116707+0000", 2024-09-16T09:32:35.364 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6578e-05 2024-09-16T09:32:35.364 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.364 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.364 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:32:35.364 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.142844+0000", 2024-09-16T09:32:35.364 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.026136994 2024-09-16T09:32:35.364 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.364 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.364 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:32:35.365 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.142862+0000", 2024-09-16T09:32:35.365 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.8063999999999998e-05 2024-09-16T09:32:35.365 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.365 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.365 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:32:35.365 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.151828+0000", 2024-09-16T09:32:35.365 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0089659010000000001 2024-09-16T09:32:35.365 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.365 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.365 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:32:35.365 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.151882+0000", 2024-09-16T09:32:35.365 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4735999999999999e-05 2024-09-16T09:32:35.365 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:32:35.366 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:32:35.366 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:32:35.366 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.366 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.366 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.24698.0:215333 7.b 7:dde3f973:::benchmark_data_smithi070_78438_object89202:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e140)", 2024-09-16T09:32:35.366 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:29:03.353923+0000", 2024-09-16T09:32:35.366 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 211.980606147, 2024-09-16T09:32:35.366 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.076247334, 2024-09-16T09:32:35.366 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:32:35.366 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:32:35.366 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:32:35.366 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:32:35.366 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.24698", 2024-09-16T09:32:35.367 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:32:35.367 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 215333 2024-09-16T09:32:35.367 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.367 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:32:35.367 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.367 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:32:35.367 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:03.353923+0000", 2024-09-16T09:32:35.367 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:32:35.367 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.367 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.367 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:32:35.367 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:03.353923+0000", 2024-09-16T09:32:35.367 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:32:35.368 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.368 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.368 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:32:35.368 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:03.353924+0000", 2024-09-16T09:32:35.368 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.342e-06 2024-09-16T09:32:35.368 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.368 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.368 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:32:35.368 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:03.353928+0000", 2024-09-16T09:32:35.368 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2989999999999998e-06 2024-09-16T09:32:35.368 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.368 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.368 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:32:35.368 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:03.353939+0000", 2024-09-16T09:32:35.369 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0635999999999999e-05 2024-09-16T09:32:35.369 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.369 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.369 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:32:35.369 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:03.353946+0000", 2024-09-16T09:32:35.369 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.0829999999999998e-06 2024-09-16T09:32:35.369 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.369 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.369 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:32:35.369 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:03.357049+0000", 2024-09-16T09:32:35.369 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:32:35.370 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0031032120000000002 2024-09-16T09:32:35.371 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.371 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.371 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:32:35.371 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:03.357172+0000", 2024-09-16T09:32:35.371 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000123082 2024-09-16T09:32:35.371 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.371 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.371 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 5,6", 2024-09-16T09:32:35.371 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:03.357283+0000", 2024-09-16T09:32:35.371 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00011045899999999999 2024-09-16T09:32:35.371 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.371 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.371 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:32:35.372 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:03.366648+0000", 2024-09-16T09:32:35.372 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0093656050000000008 2024-09-16T09:32:35.372 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.372 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.372 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:32:35.372 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:03.418188+0000", 2024-09-16T09:32:35.372 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.051539809999999998 2024-09-16T09:32:35.372 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.372 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.372 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:32:35.372 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:03.430072+0000", 2024-09-16T09:32:35.372 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.011884209 2024-09-16T09:32:35.372 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.373 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.373 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:32:35.373 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:03.430123+0000", 2024-09-16T09:32:35.373 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.1069000000000002e-05 2024-09-16T09:32:35.373 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.373 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.373 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:32:35.373 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:03.430170+0000", 2024-09-16T09:32:35.373 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.6527999999999999e-05 2024-09-16T09:32:35.374 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:32:35.374 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:32:35.374 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:32:35.374 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.374 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.374 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.24698.0:215334 7.b 7:d8f1e772:::benchmark_data_smithi070_78438_object89203:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e140)", 2024-09-16T09:32:35.374 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:29:03.356011+0000", 2024-09-16T09:32:35.374 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 211.978517436, 2024-09-16T09:32:35.374 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.074268952999999999, 2024-09-16T09:32:35.374 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:32:35.374 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:32:35.374 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:32:35.374 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:32:35.375 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.24698", 2024-09-16T09:32:35.375 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:32:35.375 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 215334 2024-09-16T09:32:35.375 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.375 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:32:35.375 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.375 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:32:35.375 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:03.356011+0000", 2024-09-16T09:32:35.375 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:32:35.375 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.375 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.375 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:32:35.376 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:03.356011+0000", 2024-09-16T09:32:35.376 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:32:35.376 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.376 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.376 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:32:35.376 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:03.356013+0000", 2024-09-16T09:32:35.376 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.4869999999999999e-06 2024-09-16T09:32:35.376 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.376 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.376 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:32:35.376 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:03.356019+0000", 2024-09-16T09:32:35.376 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.6790000000000002e-06 2024-09-16T09:32:35.376 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.377 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.377 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:32:35.377 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:03.356030+0000", 2024-09-16T09:32:35.377 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1773000000000001e-05 2024-09-16T09:32:35.377 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.377 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.377 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:32:35.377 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:03.356038+0000", 2024-09-16T09:32:35.377 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.4769999999999997e-06 2024-09-16T09:32:35.377 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.377 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.377 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:32:35.377 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:03.363371+0000", 2024-09-16T09:32:35.377 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0073330260000000003 2024-09-16T09:32:35.378 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.378 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.378 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:32:35.378 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:03.363505+0000", 2024-09-16T09:32:35.378 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00013444600000000001 2024-09-16T09:32:35.378 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.378 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.378 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 5,6", 2024-09-16T09:32:35.378 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:03.363622+0000", 2024-09-16T09:32:35.378 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00011678199999999999 2024-09-16T09:32:35.378 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.378 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.378 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:32:35.378 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:03.366658+0000", 2024-09-16T09:32:35.379 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.003036398 2024-09-16T09:32:35.379 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.379 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.379 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:32:35.379 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:03.424445+0000", 2024-09-16T09:32:35.379 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.057786718000000001 2024-09-16T09:32:35.379 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.379 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.379 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:32:35.379 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:03.430207+0000", 2024-09-16T09:32:35.379 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0057623090000000002 2024-09-16T09:32:35.379 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.379 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.379 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:32:35.379 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:03.430249+0000", 2024-09-16T09:32:35.380 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.1499999999999999e-05 2024-09-16T09:32:35.380 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.380 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.380 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:32:35.380 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:03.430280+0000", 2024-09-16T09:32:35.380 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1358e-05 2024-09-16T09:32:35.380 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:32:35.380 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:32:35.380 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:32:35.380 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.380 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.380 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.24698.0:215744 7.b 7:dbbf7f4e:::benchmark_data_smithi070_78438_object89613:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e141)", 2024-09-16T09:32:35.380 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:29:03.752855+0000", 2024-09-16T09:32:35.381 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 211.581673569, 2024-09-16T09:32:35.381 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.17614168299999999, 2024-09-16T09:32:35.381 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:32:35.381 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:32:35.381 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:32:35.381 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:32:35.381 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.24698", 2024-09-16T09:32:35.381 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:32:35.381 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 215744 2024-09-16T09:32:35.381 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.381 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:32:35.381 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.381 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:32:35.382 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:03.752855+0000", 2024-09-16T09:32:35.382 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:32:35.382 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.382 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.382 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:32:35.382 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:03.752855+0000", 2024-09-16T09:32:35.382 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:32:35.382 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.382 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.382 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:32:35.382 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:03.752857+0000", 2024-09-16T09:32:35.382 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.739e-06 2024-09-16T09:32:35.382 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.383 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.383 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:32:35.383 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:03.752862+0000", 2024-09-16T09:32:35.383 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.7700000000000001e-06 2024-09-16T09:32:35.383 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.383 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.383 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:32:35.383 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:03.752875+0000", 2024-09-16T09:32:35.383 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3475000000000001e-05 2024-09-16T09:32:35.383 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.383 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.383 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:32:35.383 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:03.752882+0000", 2024-09-16T09:32:35.383 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.1040000000000001e-06 2024-09-16T09:32:35.384 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.384 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.384 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:32:35.384 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:03.752942+0000", 2024-09-16T09:32:35.384 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.9731999999999997e-05 2024-09-16T09:32:35.384 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.384 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.384 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:32:35.384 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:03.753064+0000", 2024-09-16T09:32:35.384 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00012189300000000001 2024-09-16T09:32:35.384 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.384 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.384 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 5,6", 2024-09-16T09:32:35.385 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:03.753160+0000", 2024-09-16T09:32:35.385 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.6152999999999999e-05 2024-09-16T09:32:35.385 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.385 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.385 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:32:35.385 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:03.755179+0000", 2024-09-16T09:32:35.385 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.002019285 2024-09-16T09:32:35.385 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.385 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.385 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:32:35.385 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:03.755234+0000", 2024-09-16T09:32:35.385 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4404999999999998e-05 2024-09-16T09:32:35.385 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.386 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.386 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:32:35.386 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:03.928884+0000", 2024-09-16T09:32:35.386 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.17365044800000001 2024-09-16T09:32:35.386 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.386 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.386 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:32:35.386 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:03.928945+0000", 2024-09-16T09:32:35.386 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.0371000000000002e-05 2024-09-16T09:32:35.386 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.386 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.386 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:32:35.386 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:03.928997+0000", 2024-09-16T09:32:35.386 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.2308000000000001e-05 2024-09-16T09:32:35.387 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:32:35.387 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:32:35.387 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:32:35.387 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.387 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.387 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.24698.0:247021 7.0 e146/82 7:011fa108:::benchmark_data_smithi070_78438_object120890:head v 146'15501, mlcod=146'15498)", 2024-09-16T09:32:35.387 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:29:32.630477+0000", 2024-09-16T09:32:35.387 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 182.70405207900001, 2024-09-16T09:32:35.387 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.084335054000000007, 2024-09-16T09:32:35.387 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:32:35.387 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:32:35.387 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:32:35.387 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:32:35.388 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.388 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:32:35.388 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:32.630477+0000", 2024-09-16T09:32:35.388 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:32:35.388 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.388 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.388 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:32:35.388 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:32.630477+0000", 2024-09-16T09:32:35.388 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:32:35.388 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.388 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.388 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:32:35.388 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:32.630479+0000", 2024-09-16T09:32:35.389 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9089999999999998e-06 2024-09-16T09:32:35.389 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.389 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.389 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:32:35.389 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:32.630488+0000", 2024-09-16T09:32:35.389 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.3789999999999995e-06 2024-09-16T09:32:35.389 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.389 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.389 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:32:35.389 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:32.630509+0000", 2024-09-16T09:32:35.389 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.048e-05 2024-09-16T09:32:35.389 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.389 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.389 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:32:35.390 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:32.630517+0000", 2024-09-16T09:32:35.390 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.9600000000000006e-06 2024-09-16T09:32:35.390 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.390 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.390 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:32:35.390 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:32.633683+0000", 2024-09-16T09:32:35.390 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0031652210000000002 2024-09-16T09:32:35.390 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.390 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.390 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:32:35.390 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:32.633702+0000", 2024-09-16T09:32:35.390 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9059000000000002e-05 2024-09-16T09:32:35.390 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.390 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.391 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:32:35.391 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:32.714769+0000", 2024-09-16T09:32:35.391 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.081067024000000001 2024-09-16T09:32:35.391 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.391 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.391 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:32:35.391 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:32.714812+0000", 2024-09-16T09:32:35.391 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.3022000000000002e-05 2024-09-16T09:32:35.391 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:32:35.391 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:32:35.391 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:32:35.391 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.391 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.391 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.24698.0:247027 7.0 e146/82 7:081b7ec9:::benchmark_data_smithi070_78438_object120896:head v 146'15502, mlcod=146'15499)", 2024-09-16T09:32:35.391 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:29:32.632454+0000", 2024-09-16T09:32:35.392 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 182.702075025, 2024-09-16T09:32:35.392 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.082387588999999997, 2024-09-16T09:32:35.392 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:32:35.392 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:32:35.392 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:32:35.392 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:32:35.392 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.392 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:32:35.392 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:32.632454+0000", 2024-09-16T09:32:35.392 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:32:35.392 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.392 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.393 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:32:35.393 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:32.632454+0000", 2024-09-16T09:32:35.393 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:32:35.393 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.393 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.393 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:32:35.393 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:32.632455+0000", 2024-09-16T09:32:35.393 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.1500000000000003e-07 2024-09-16T09:32:35.393 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.393 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.393 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:32:35.393 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:32.632462+0000", 2024-09-16T09:32:35.393 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.1509999999999998e-06 2024-09-16T09:32:35.393 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.394 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.394 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:32:35.394 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:32.632476+0000", 2024-09-16T09:32:35.394 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.466e-05 2024-09-16T09:32:35.394 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.394 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.394 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:32:35.394 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:32.632484+0000", 2024-09-16T09:32:35.394 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.892e-06 2024-09-16T09:32:35.394 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.394 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.394 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:32:35.394 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:32.711686+0000", 2024-09-16T09:32:35.395 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.079201808999999998 2024-09-16T09:32:35.395 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.395 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.395 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:32:35.395 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:32.711704+0000", 2024-09-16T09:32:35.395 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7618e-05 2024-09-16T09:32:35.395 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.395 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.395 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:32:35.395 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:32.714814+0000", 2024-09-16T09:32:35.395 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0031107249999999999 2024-09-16T09:32:35.395 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.395 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.395 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:32:35.396 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:29:32.714841+0000", 2024-09-16T09:32:35.396 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.6919000000000001e-05 2024-09-16T09:32:35.396 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:32:35.396 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:32:35.396 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:32:35.396 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.396 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.396 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.25064.0:69528 8.1 8:8d3f553b:::benchmark_data_smithi070_107919_object69527:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e174)", 2024-09-16T09:32:35.396 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:30:36.638875+0000", 2024-09-16T09:32:35.396 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 118.69565408699999, 2024-09-16T09:32:35.396 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.091207852000000006, 2024-09-16T09:32:35.396 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:32:35.396 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:32:35.397 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:32:35.397 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:32:35.397 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.25064", 2024-09-16T09:32:35.397 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/2578256439", 2024-09-16T09:32:35.397 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 69528 2024-09-16T09:32:35.397 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.397 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:32:35.397 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.397 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:32:35.397 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:30:36.638875+0000", 2024-09-16T09:32:35.397 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:32:35.397 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.397 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.397 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:32:35.398 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:30:36.638875+0000", 2024-09-16T09:32:35.398 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:32:35.398 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.398 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.398 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:32:35.398 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:30:36.638877+0000", 2024-09-16T09:32:35.398 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.148e-06 2024-09-16T09:32:35.398 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.398 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.398 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:32:35.398 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:30:36.638902+0000", 2024-09-16T09:32:35.398 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.5326000000000001e-05 2024-09-16T09:32:35.398 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.398 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.399 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:32:35.399 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:30:36.638917+0000", 2024-09-16T09:32:35.399 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.4848000000000001e-05 2024-09-16T09:32:35.399 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.399 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.399 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:32:35.399 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:30:36.638926+0000", 2024-09-16T09:32:35.399 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.4950000000000008e-06 2024-09-16T09:32:35.399 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.399 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.399 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:32:35.399 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:30:36.638986+0000", 2024-09-16T09:32:35.399 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.0551999999999999e-05 2024-09-16T09:32:35.399 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.400 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.400 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:32:35.400 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:30:36.639109+0000", 2024-09-16T09:32:35.400 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000122454 2024-09-16T09:32:35.400 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.400 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.400 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 0,2", 2024-09-16T09:32:35.400 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:30:36.639256+0000", 2024-09-16T09:32:35.400 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00014719599999999999 2024-09-16T09:32:35.400 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.400 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.400 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:32:35.400 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:30:36.677738+0000", 2024-09-16T09:32:35.400 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.038482220999999997 2024-09-16T09:32:35.401 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.401 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.401 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:32:35.401 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:30:36.677752+0000", 2024-09-16T09:32:35.401 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.4406000000000001e-05 2024-09-16T09:32:35.401 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.401 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.401 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:32:35.401 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:30:36.729711+0000", 2024-09-16T09:32:35.401 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.051958670999999998 2024-09-16T09:32:35.401 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.401 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.401 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:32:35.401 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:30:36.730037+0000", 2024-09-16T09:32:35.402 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00032638300000000002 2024-09-16T09:32:35.402 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.402 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.402 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:32:35.402 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:30:36.730083+0000", 2024-09-16T09:32:35.402 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5151999999999997e-05 2024-09-16T09:32:35.402 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:32:35.402 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:32:35.402 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:32:35.402 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.402 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.402 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.25064.0:126633 8.a 8:5aaae853:::benchmark_data_smithi070_107919_object2038:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e194)", 2024-09-16T09:32:35.402 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:31:27.787616+0000", 2024-09-16T09:32:35.402 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 67.546912974999998, 2024-09-16T09:32:35.403 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.073978759000000005, 2024-09-16T09:32:35.403 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:32:35.403 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:32:35.403 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:32:35.403 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:32:35.403 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.25064", 2024-09-16T09:32:35.403 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/2578256439", 2024-09-16T09:32:35.403 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 126633 2024-09-16T09:32:35.403 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.403 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:32:35.403 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.403 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:32:35.403 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:31:27.787616+0000", 2024-09-16T09:32:35.404 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:32:35.404 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.404 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.404 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:32:35.404 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:31:27.787616+0000", 2024-09-16T09:32:35.404 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:32:35.404 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.404 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.404 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:32:35.404 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:31:27.787617+0000", 2024-09-16T09:32:35.404 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.485e-06 2024-09-16T09:32:35.404 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.404 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.404 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:32:35.404 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:31:27.787623+0000", 2024-09-16T09:32:35.405 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.429e-06 2024-09-16T09:32:35.405 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.405 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.405 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:32:35.405 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:31:27.787638+0000", 2024-09-16T09:32:35.405 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5257e-05 2024-09-16T09:32:35.405 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.405 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.405 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:32:35.405 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:31:27.787645+0000", 2024-09-16T09:32:35.405 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3039999999999999e-06 2024-09-16T09:32:35.405 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.405 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.406 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:32:35.406 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:31:27.787699+0000", 2024-09-16T09:32:35.406 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.3934999999999998e-05 2024-09-16T09:32:35.406 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.406 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.406 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:32:35.406 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:31:27.787825+0000", 2024-09-16T09:32:35.406 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00012589900000000001 2024-09-16T09:32:35.406 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.407 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.407 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 2,5", 2024-09-16T09:32:35.407 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:31:27.787944+0000", 2024-09-16T09:32:35.407 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000118924 2024-09-16T09:32:35.407 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.407 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.407 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:32:35.407 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:31:27.792800+0000", 2024-09-16T09:32:35.407 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0048556600000000004 2024-09-16T09:32:35.407 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.408 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.408 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:32:35.408 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:31:27.792813+0000", 2024-09-16T09:32:35.408 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2855e-05 2024-09-16T09:32:35.408 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.408 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.408 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:32:35.408 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:31:27.861477+0000", 2024-09-16T09:32:35.408 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.068664402999999999 2024-09-16T09:32:35.408 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.408 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.408 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:32:35.409 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:31:27.861542+0000", 2024-09-16T09:32:35.409 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.5029000000000004e-05 2024-09-16T09:32:35.409 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.409 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.409 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:32:35.409 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:31:27.861595+0000", 2024-09-16T09:32:35.409 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.2578999999999999e-05 2024-09-16T09:32:35.409 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:32:35.409 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:32:35.409 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:32:35.409 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.409 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.410 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.25064.0:173629 8.a 8:563f426d:::benchmark_data_smithi070_107919_object49034:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e205)", 2024-09-16T09:32:35.410 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:32:10.836618+0000", 2024-09-16T09:32:35.410 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 24.497911002999999, 2024-09-16T09:32:35.410 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.090518149000000006, 2024-09-16T09:32:35.410 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:32:35.410 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:32:35.410 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:32:35.410 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:32:35.410 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.25064", 2024-09-16T09:32:35.410 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/2578256439", 2024-09-16T09:32:35.410 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 173629 2024-09-16T09:32:35.410 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.410 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:32:35.410 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.411 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:32:35.411 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.836618+0000", 2024-09-16T09:32:35.411 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:32:35.411 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.411 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.411 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:32:35.411 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.836618+0000", 2024-09-16T09:32:35.411 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:32:35.411 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.411 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.411 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:32:35.411 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.836620+0000", 2024-09-16T09:32:35.411 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7409999999999999e-06 2024-09-16T09:32:35.412 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.412 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.412 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:32:35.412 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.836625+0000", 2024-09-16T09:32:35.412 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.3029999999999999e-06 2024-09-16T09:32:35.412 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.412 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.412 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:32:35.412 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.836636+0000", 2024-09-16T09:32:35.412 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1613999999999999e-05 2024-09-16T09:32:35.412 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.412 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.412 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:32:35.412 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.836644+0000", 2024-09-16T09:32:35.413 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.5299999999999999e-06 2024-09-16T09:32:35.413 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.413 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.413 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:32:35.413 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.836701+0000", 2024-09-16T09:32:35.413 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.6997999999999999e-05 2024-09-16T09:32:35.413 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.413 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.413 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:32:35.413 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.836801+0000", 2024-09-16T09:32:35.413 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000100462 2024-09-16T09:32:35.413 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.413 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.413 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 2,5", 2024-09-16T09:32:35.414 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.836900+0000", 2024-09-16T09:32:35.414 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.8827999999999996e-05 2024-09-16T09:32:35.414 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.414 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.414 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:32:35.415 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:32:35.415 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.839346+0000", 2024-09-16T09:32:35.416 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.002445816 2024-09-16T09:32:35.416 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.416 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.416 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:32:35.416 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.840160+0000", 2024-09-16T09:32:35.416 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00081422799999999996 2024-09-16T09:32:35.416 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.416 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.416 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:32:35.416 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.927028+0000", 2024-09-16T09:32:35.416 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.086867510999999994 2024-09-16T09:32:35.416 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.416 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.417 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:32:35.417 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.927074+0000", 2024-09-16T09:32:35.417 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.6202000000000003e-05 2024-09-16T09:32:35.417 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.417 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.417 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:32:35.417 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.927136+0000", 2024-09-16T09:32:35.417 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1915999999999995e-05 2024-09-16T09:32:35.417 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:32:35.417 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:32:35.417 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:32:35.417 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.417 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.418 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.25064.0:173635 8.a 8:560400da:::benchmark_data_smithi070_107919_object49040:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e205)", 2024-09-16T09:32:35.418 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:32:10.840092+0000", 2024-09-16T09:32:35.418 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 24.494436956000001, 2024-09-16T09:32:35.418 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.087223118000000002, 2024-09-16T09:32:35.418 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:32:35.418 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:32:35.418 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:32:35.418 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:32:35.418 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.25064", 2024-09-16T09:32:35.418 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/2578256439", 2024-09-16T09:32:35.418 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 173635 2024-09-16T09:32:35.418 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.418 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:32:35.419 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.419 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:32:35.419 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.840092+0000", 2024-09-16T09:32:35.419 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:32:35.419 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.419 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.419 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:32:35.419 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.840092+0000", 2024-09-16T09:32:35.419 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:32:35.419 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.419 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.419 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:32:35.419 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.840093+0000", 2024-09-16T09:32:35.419 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.438e-06 2024-09-16T09:32:35.419 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.420 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.420 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:32:35.420 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.840098+0000", 2024-09-16T09:32:35.420 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0880000000000002e-06 2024-09-16T09:32:35.420 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.420 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.420 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:32:35.420 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.840110+0000", 2024-09-16T09:32:35.420 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1207e-05 2024-09-16T09:32:35.420 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.420 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.420 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:32:35.420 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.840117+0000", 2024-09-16T09:32:35.421 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.9789999999999996e-06 2024-09-16T09:32:35.421 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.421 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.421 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:32:35.421 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.840257+0000", 2024-09-16T09:32:35.421 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00014021199999999999 2024-09-16T09:32:35.421 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.421 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.421 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:32:35.421 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.840418+0000", 2024-09-16T09:32:35.421 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00016090599999999999 2024-09-16T09:32:35.421 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.421 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.422 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 2,5", 2024-09-16T09:32:35.422 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.840520+0000", 2024-09-16T09:32:35.422 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000101821 2024-09-16T09:32:35.422 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.422 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.422 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:32:35.422 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.845303+0000", 2024-09-16T09:32:35.422 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0047839390000000001 2024-09-16T09:32:35.422 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.422 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.422 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:32:35.422 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.845326+0000", 2024-09-16T09:32:35.422 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.2983999999999999e-05 2024-09-16T09:32:35.423 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.423 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.423 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:32:35.423 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.927176+0000", 2024-09-16T09:32:35.423 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.081849476000000004 2024-09-16T09:32:35.423 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.423 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.423 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:32:35.423 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.927218+0000", 2024-09-16T09:32:35.423 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.1912999999999997e-05 2024-09-16T09:32:35.423 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.423 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.423 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:32:35.424 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.927315+0000", 2024-09-16T09:32:35.424 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.7155000000000001e-05 2024-09-16T09:32:35.424 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:32:35.424 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:32:35.424 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:32:35.424 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.424 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.424 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.25064.0:173637 8.a 8:5031dbaa:::benchmark_data_smithi070_107919_object49042:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e205)", 2024-09-16T09:32:35.424 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:32:10.840274+0000", 2024-09-16T09:32:35.424 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 24.494254727000001, 2024-09-16T09:32:35.424 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.087221243000000004, 2024-09-16T09:32:35.424 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:32:35.424 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:32:35.424 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:32:35.425 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:32:35.425 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.25064", 2024-09-16T09:32:35.425 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/2578256439", 2024-09-16T09:32:35.425 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 173637 2024-09-16T09:32:35.425 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.425 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:32:35.425 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.425 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:32:35.425 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.840274+0000", 2024-09-16T09:32:35.425 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:32:35.425 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.425 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.425 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:32:35.425 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.840274+0000", 2024-09-16T09:32:35.426 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:32:35.426 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.426 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.426 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:32:35.426 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.840275+0000", 2024-09-16T09:32:35.426 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.2500000000000005e-07 2024-09-16T09:32:35.426 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.426 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.426 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:32:35.426 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.840277+0000", 2024-09-16T09:32:35.426 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.7070000000000001e-06 2024-09-16T09:32:35.426 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.426 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.427 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:32:35.427 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.840284+0000", 2024-09-16T09:32:35.427 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.2169999999999999e-06 2024-09-16T09:32:35.427 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.427 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.427 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:32:35.427 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.840289+0000", 2024-09-16T09:32:35.427 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.3639999999999998e-06 2024-09-16T09:32:35.427 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.427 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.427 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:32:35.427 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.842775+0000", 2024-09-16T09:32:35.427 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0024862389999999999 2024-09-16T09:32:35.427 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.428 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.428 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:32:35.428 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.842869+0000", 2024-09-16T09:32:35.428 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.3789000000000006e-05 2024-09-16T09:32:35.428 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.428 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.428 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 2,5", 2024-09-16T09:32:35.428 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.842966+0000", 2024-09-16T09:32:35.428 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.6586e-05 2024-09-16T09:32:35.428 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.428 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.428 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:32:35.428 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.845331+0000", 2024-09-16T09:32:35.428 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0023649500000000002 2024-09-16T09:32:35.429 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.429 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.429 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:32:35.429 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.848046+0000", 2024-09-16T09:32:35.429 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0027154039999999998 2024-09-16T09:32:35.429 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.430 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.430 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:32:35.430 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.927351+0000", 2024-09-16T09:32:35.430 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.079304843999999999 2024-09-16T09:32:35.430 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.430 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.430 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:32:35.430 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.927398+0000", 2024-09-16T09:32:35.430 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.7092000000000002e-05 2024-09-16T09:32:35.430 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.430 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.430 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:32:35.430 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.927495+0000", 2024-09-16T09:32:35.431 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.7425999999999999e-05 2024-09-16T09:32:35.431 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:32:35.431 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:32:35.431 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:32:35.431 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.431 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.431 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.25064.0:173651 8.f e205/152 8:fdf6843a:::benchmark_data_smithi070_107919_object49056:head v 205'10925, mlcod=205'10923)", 2024-09-16T09:32:35.431 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:32:10.849677+0000", 2024-09-16T09:32:35.431 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 24.484852302, 2024-09-16T09:32:35.431 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.083982714999999999, 2024-09-16T09:32:35.431 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:32:35.431 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:32:35.431 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:32:35.432 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:32:35.432 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.432 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:32:35.432 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.849677+0000", 2024-09-16T09:32:35.432 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:32:35.432 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.432 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.432 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:32:35.432 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.849677+0000", 2024-09-16T09:32:35.432 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:32:35.432 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.432 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.432 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:32:35.432 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.849677+0000", 2024-09-16T09:32:35.433 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.4900000000000005e-07 2024-09-16T09:32:35.433 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.433 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.433 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:32:35.433 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.849683+0000", 2024-09-16T09:32:35.433 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.6670000000000001e-06 2024-09-16T09:32:35.433 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.433 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.433 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:32:35.433 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.849693+0000", 2024-09-16T09:32:35.433 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0236e-05 2024-09-16T09:32:35.433 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.433 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.433 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:32:35.434 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.849699+0000", 2024-09-16T09:32:35.434 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.93e-06 2024-09-16T09:32:35.434 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.434 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.434 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:32:35.434 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.849763+0000", 2024-09-16T09:32:35.434 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.4117999999999998e-05 2024-09-16T09:32:35.434 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.434 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.434 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:32:35.434 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.849781+0000", 2024-09-16T09:32:35.434 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7354999999999999e-05 2024-09-16T09:32:35.434 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.435 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.435 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:32:35.435 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.933585+0000", 2024-09-16T09:32:35.435 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.083804672999999996 2024-09-16T09:32:35.435 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.435 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.435 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:32:35.435 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.933659+0000", 2024-09-16T09:32:35.435 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3986999999999995e-05 2024-09-16T09:32:35.435 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:32:35.435 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:32:35.435 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:32:35.435 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.435 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.435 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.25064.0:173653 8.a 8:593f9096:::benchmark_data_smithi070_107919_object49058:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e205)", 2024-09-16T09:32:35.436 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:32:10.850513+0000", 2024-09-16T09:32:35.436 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 24.484015722999999, 2024-09-16T09:32:35.436 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.077552876000000007, 2024-09-16T09:32:35.436 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:32:35.436 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:32:35.436 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:32:35.436 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:32:35.436 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.25064", 2024-09-16T09:32:35.436 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/2578256439", 2024-09-16T09:32:35.436 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 173653 2024-09-16T09:32:35.436 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.436 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:32:35.436 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.437 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:32:35.437 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.850513+0000", 2024-09-16T09:32:35.437 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:32:35.437 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.437 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.437 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:32:35.437 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.850513+0000", 2024-09-16T09:32:35.437 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:32:35.437 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.437 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.437 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:32:35.437 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.850514+0000", 2024-09-16T09:32:35.437 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.345e-06 2024-09-16T09:32:35.437 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.438 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.438 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:32:35.438 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.850519+0000", 2024-09-16T09:32:35.438 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.8629999999999999e-06 2024-09-16T09:32:35.438 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.438 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.438 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:32:35.438 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.850530+0000", 2024-09-16T09:32:35.438 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.114e-05 2024-09-16T09:32:35.438 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.438 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.438 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:32:35.438 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.850537+0000", 2024-09-16T09:32:35.438 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.8170000000000001e-06 2024-09-16T09:32:35.439 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.439 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.439 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:32:35.439 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.850590+0000", 2024-09-16T09:32:35.439 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.2981e-05 2024-09-16T09:32:35.439 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.439 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.439 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:32:35.439 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.850693+0000", 2024-09-16T09:32:35.439 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000102502 2024-09-16T09:32:35.439 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.439 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.439 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 2,5", 2024-09-16T09:32:35.439 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.850794+0000", 2024-09-16T09:32:35.440 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00010134 2024-09-16T09:32:35.440 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.440 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.440 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:32:35.440 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.926995+0000", 2024-09-16T09:32:35.440 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.076200636000000002 2024-09-16T09:32:35.440 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.440 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.440 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:32:35.440 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.927528+0000", 2024-09-16T09:32:35.440 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00053309500000000003 2024-09-16T09:32:35.440 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.440 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.440 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:32:35.441 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.927763+0000", 2024-09-16T09:32:35.441 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00023547499999999999 2024-09-16T09:32:35.441 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.441 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.441 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:32:35.441 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.928013+0000", 2024-09-16T09:32:35.441 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000249872 2024-09-16T09:32:35.441 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.441 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.441 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:32:35.441 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.928066+0000", 2024-09-16T09:32:35.441 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.2809999999999998e-05 2024-09-16T09:32:35.441 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:32:35.441 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:32:35.442 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:32:35.442 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.442 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.442 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.25064.0:173655 8.7 8:ed4dd2c5:::benchmark_data_smithi070_107919_object49060:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e205)", 2024-09-16T09:32:35.442 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:32:10.852286+0000", 2024-09-16T09:32:35.442 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 24.482242885000002, 2024-09-16T09:32:35.442 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.081484780000000007, 2024-09-16T09:32:35.442 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:32:35.442 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:32:35.442 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:32:35.442 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:32:35.442 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.25064", 2024-09-16T09:32:35.442 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/2578256439", 2024-09-16T09:32:35.442 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 173655 2024-09-16T09:32:35.443 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.443 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:32:35.443 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.443 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:32:35.443 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.852286+0000", 2024-09-16T09:32:35.443 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:32:35.443 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.443 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.443 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:32:35.443 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.852286+0000", 2024-09-16T09:32:35.443 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:32:35.443 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.443 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.444 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:32:35.444 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.852287+0000", 2024-09-16T09:32:35.444 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2899999999999999e-06 2024-09-16T09:32:35.444 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.444 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.444 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:32:35.444 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.852292+0000", 2024-09-16T09:32:35.444 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.079e-06 2024-09-16T09:32:35.444 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.444 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.444 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:32:35.444 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.852304+0000", 2024-09-16T09:32:35.444 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1807999999999999e-05 2024-09-16T09:32:35.444 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.445 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.445 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:32:35.445 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.852311+0000", 2024-09-16T09:32:35.445 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.1709999999999996e-06 2024-09-16T09:32:35.445 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.445 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.445 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:32:35.445 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.926364+0000", 2024-09-16T09:32:35.445 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.074053038000000002 2024-09-16T09:32:35.445 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.445 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.445 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:32:35.445 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.926507+0000", 2024-09-16T09:32:35.445 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000142492 2024-09-16T09:32:35.446 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.446 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.446 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 3,6", 2024-09-16T09:32:35.446 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.926632+0000", 2024-09-16T09:32:35.446 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00012537799999999999 2024-09-16T09:32:35.446 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.446 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.446 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:32:35.446 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.931320+0000", 2024-09-16T09:32:35.446 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0046874259999999997 2024-09-16T09:32:35.446 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.446 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.446 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:32:35.446 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.933619+0000", 2024-09-16T09:32:35.447 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0022990720000000001 2024-09-16T09:32:35.447 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.447 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.447 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:32:35.447 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.933697+0000", 2024-09-16T09:32:35.447 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.873e-05 2024-09-16T09:32:35.447 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.447 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.447 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:32:35.447 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.933736+0000", 2024-09-16T09:32:35.447 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8319999999999999e-05 2024-09-16T09:32:35.447 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.447 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.447 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:32:35.448 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.933771+0000", 2024-09-16T09:32:35.448 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4975999999999999e-05 2024-09-16T09:32:35.448 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:32:35.448 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:32:35.448 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:32:35.448 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.448 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.448 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.25064.0:173657 8.3 8:c6dc8808:::benchmark_data_smithi070_107919_object49062:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e205)", 2024-09-16T09:32:35.448 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:32:10.853258+0000", 2024-09-16T09:32:35.448 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 24.481270617, 2024-09-16T09:32:35.448 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.079963668000000002, 2024-09-16T09:32:35.448 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:32:35.448 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:32:35.449 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:32:35.449 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:32:35.449 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.25064", 2024-09-16T09:32:35.449 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/2578256439", 2024-09-16T09:32:35.449 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 173657 2024-09-16T09:32:35.449 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.449 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:32:35.449 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.449 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:32:35.449 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.853258+0000", 2024-09-16T09:32:35.449 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:32:35.449 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.449 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.449 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:32:35.450 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.853258+0000", 2024-09-16T09:32:35.450 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:32:35.450 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.450 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.450 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:32:35.450 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.853259+0000", 2024-09-16T09:32:35.450 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.3600000000000002e-07 2024-09-16T09:32:35.450 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.450 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.450 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:32:35.450 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.853264+0000", 2024-09-16T09:32:35.450 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.403e-06 2024-09-16T09:32:35.450 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.450 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.451 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:32:35.451 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.926423+0000", 2024-09-16T09:32:35.451 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.073159735000000004 2024-09-16T09:32:35.451 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.451 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.451 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:32:35.451 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.926435+0000", 2024-09-16T09:32:35.451 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1905999999999999e-05 2024-09-16T09:32:35.451 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.451 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.451 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:32:35.451 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.927539+0000", 2024-09-16T09:32:35.451 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0011033639999999999 2024-09-16T09:32:35.451 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.452 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.452 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:32:35.452 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.927707+0000", 2024-09-16T09:32:35.452 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000168832 2024-09-16T09:32:35.452 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.452 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.452 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 5,6", 2024-09-16T09:32:35.452 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.927806+0000", 2024-09-16T09:32:35.452 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.8616000000000006e-05 2024-09-16T09:32:35.452 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.452 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.452 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:32:35.452 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.932932+0000", 2024-09-16T09:32:35.452 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.005126467 2024-09-16T09:32:35.453 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.453 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.453 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:32:35.453 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.932943+0000", 2024-09-16T09:32:35.453 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0968000000000001e-05 2024-09-16T09:32:35.453 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.453 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.453 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:32:35.453 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.932971+0000", 2024-09-16T09:32:35.453 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.7546999999999999e-05 2024-09-16T09:32:35.453 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.453 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.453 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:32:35.453 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.933179+0000", 2024-09-16T09:32:35.454 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00020783600000000001 2024-09-16T09:32:35.454 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.454 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.454 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:32:35.454 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.933222+0000", 2024-09-16T09:32:35.454 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.3058e-05 2024-09-16T09:32:35.454 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:32:35.454 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:32:35.454 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:32:35.454 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.454 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.454 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.25064.0:173660 8.b e205/204 8:d12c32a2:::benchmark_data_smithi070_107919_object49065:head v 205'10853, mlcod=205'10851)", 2024-09-16T09:32:35.454 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:32:10.855799+0000", 2024-09-16T09:32:35.454 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 24.478729626, 2024-09-16T09:32:35.454 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.082555492999999994, 2024-09-16T09:32:35.455 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:32:35.455 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:32:35.455 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:32:35.455 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:32:35.455 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.455 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:32:35.455 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.855799+0000", 2024-09-16T09:32:35.455 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:32:35.455 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.455 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.455 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:32:35.455 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.855799+0000", 2024-09-16T09:32:35.455 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:32:35.455 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.456 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.456 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:32:35.456 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.855800+0000", 2024-09-16T09:32:35.456 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.3699999999999999e-07 2024-09-16T09:32:35.456 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.456 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.456 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:32:35.456 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.855807+0000", 2024-09-16T09:32:35.456 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.6420000000000001e-06 2024-09-16T09:32:35.456 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.456 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.456 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:32:35.456 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.928229+0000", 2024-09-16T09:32:35.457 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.072421869999999999 2024-09-16T09:32:35.457 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.457 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.457 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:32:35.457 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.928237+0000", 2024-09-16T09:32:35.457 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.7760000000000003e-06 2024-09-16T09:32:35.457 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.457 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.457 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:32:35.457 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.928357+0000", 2024-09-16T09:32:35.457 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00011969200000000001 2024-09-16T09:32:35.457 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.457 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.458 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:32:35.458 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.928380+0000", 2024-09-16T09:32:35.458 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3113e-05 2024-09-16T09:32:35.458 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.458 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.458 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:32:35.458 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.938294+0000", 2024-09-16T09:32:35.458 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0099135179999999996 2024-09-16T09:32:35.458 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:35.458 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:35.458 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:32:35.458 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:10.938355+0000", 2024-09-16T09:32:35.458 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1044999999999996e-05 2024-09-16T09:32:35.458 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:32:35.459 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:32:35.459 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:32:35.459 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:32:35.459 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:32:35.459 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T09:32:35.460 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:35.460 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:35.461 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:32:35.561 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:32:35.596 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:35.597 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:35.597 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:32:35.624 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:32:35.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:35 smithi084 ceph-mon[32629]: pgmap v350: 168 pgs: 2 active+recovering+undersized+remapped, 166 active+clean; 3.0 GiB data, 12 GiB used, 704 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.21k op/s; 1602/147966 objects misplaced (1.083%); 5.8 MiB/s, 92 objects/s recovering 2024-09-16T09:32:35.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:35 smithi070 ceph-mon[30388]: pgmap v350: 168 pgs: 2 active+recovering+undersized+remapped, 166 active+clean; 3.0 GiB data, 12 GiB used, 704 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.21k op/s; 1602/147966 objects misplaced (1.083%); 5.8 MiB/s, 92 objects/s recovering 2024-09-16T09:32:35.670 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:35 smithi070 ceph-mon[40691]: pgmap v350: 168 pgs: 2 active+recovering+undersized+remapped, 166 active+clean; 3.0 GiB data, 12 GiB used, 704 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.21k op/s; 1602/147966 objects misplaced (1.083%); 5.8 MiB/s, 92 objects/s recovering 2024-09-16T09:32:35.698 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:32:35.727 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:35.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:35.728 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:32:35.829 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:32:35.919 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:35.921 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:35.921 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:32:36.022 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:32:36.059 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:36.060 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:36.060 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:32:36.161 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:32:36.198 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:36.199 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:36.199 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:32:36.300 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:32:36.313 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.2 dump_ops_in_flight 2024-09-16T09:32:36.346 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:36.347 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:36.348 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:32:36.449 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:32:36.489 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:36.489 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:36.489 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:32:36.543 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:36 smithi070 ceph-mon[30388]: 8.e scrub starts 2024-09-16T09:32:36.544 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:36 smithi070 ceph-mon[30388]: 4.1 scrub starts 2024-09-16T09:32:36.544 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:36 smithi070 ceph-mon[30388]: 4.1 scrub ok 2024-09-16T09:32:36.546 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:36 smithi070 ceph-mon[40691]: 8.e scrub starts 2024-09-16T09:32:36.546 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:36 smithi070 ceph-mon[40691]: 4.1 scrub starts 2024-09-16T09:32:36.546 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:36 smithi070 ceph-mon[40691]: 4.1 scrub ok 2024-09-16T09:32:36.590 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:32:36.637 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:36.638 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:36.638 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:32:36.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:36 smithi084 ceph-mon[32629]: 8.e scrub starts 2024-09-16T09:32:36.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:36 smithi084 ceph-mon[32629]: 4.1 scrub starts 2024-09-16T09:32:36.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:36 smithi084 ceph-mon[32629]: 4.1 scrub ok 2024-09-16T09:32:36.739 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:32:36.752 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T09:32:36.768 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:36.769 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:36.769 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:32:36.870 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:32:36.917 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:36.917 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:36.918 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:32:37.018 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:32:37.065 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:37.065 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:37.066 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:32:37.166 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:32:37.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:37.206 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:37.207 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:32:37.308 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:32:37.336 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:37.337 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:37.337 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:32:37.438 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:32:37.471 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:37.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:37.472 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:32:37.534 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:37 smithi070 ceph-mon[30388]: 8.6 scrub starts 2024-09-16T09:32:37.534 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:37 smithi070 ceph-mon[30388]: 8.e scrub ok 2024-09-16T09:32:37.534 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:37 smithi070 ceph-mon[30388]: pgmap v351: 168 pgs: 2 active+recovering+undersized+remapped, 166 active+clean; 3.0 GiB data, 12 GiB used, 704 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.21k op/s; 1602/147966 objects misplaced (1.083%); 5.8 MiB/s, 92 objects/s recovering 2024-09-16T09:32:37.534 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:37 smithi070 ceph-mon[40691]: 8.6 scrub starts 2024-09-16T09:32:37.534 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:37 smithi070 ceph-mon[40691]: 8.e scrub ok 2024-09-16T09:32:37.534 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:37 smithi070 ceph-mon[40691]: pgmap v351: 168 pgs: 2 active+recovering+undersized+remapped, 166 active+clean; 3.0 GiB data, 12 GiB used, 704 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.21k op/s; 1602/147966 objects misplaced (1.083%); 5.8 MiB/s, 92 objects/s recovering 2024-09-16T09:32:37.573 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:32:37.609 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:37.610 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:37.610 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:32:37.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:37 smithi084 ceph-mon[32629]: 8.6 scrub starts 2024-09-16T09:32:37.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:37 smithi084 ceph-mon[32629]: 8.e scrub ok 2024-09-16T09:32:37.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:37 smithi084 ceph-mon[32629]: pgmap v351: 168 pgs: 2 active+recovering+undersized+remapped, 166 active+clean; 3.0 GiB data, 12 GiB used, 704 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.21k op/s; 1602/147966 objects misplaced (1.083%); 5.8 MiB/s, 92 objects/s recovering 2024-09-16T09:32:37.710 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:32:37.756 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:37.757 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:37.757 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:32:37.858 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:32:37.886 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:32:37.893 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:32:37.950 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:32:37.950 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:37.951 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:37.952 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:32:37.958 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2024-09-16T09:32:38.053 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:32:38.107 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:38.108 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:38.108 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:32:38.209 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:32:38.239 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:38.239 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:38.240 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:32:38.340 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:32:38.376 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:38.376 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:38.377 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:32:38.396 INFO:teuthology.orchestra.run.smithi070.stderr:nodeep-scrub is unset 2024-09-16T09:32:38.478 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:32:38.503 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:38.503 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:38.504 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:32:38.604 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:32:38.635 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:38.635 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:38.636 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:32:38.642 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:38 smithi070 ceph-mon[30388]: 8.6 scrub ok 2024-09-16T09:32:38.642 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:38 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/474313500' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:32:38.643 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:38 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:32:38.643 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:38 smithi070 ceph-mon[40691]: 8.6 scrub ok 2024-09-16T09:32:38.643 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:38 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/474313500' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:32:38.643 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:38 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:32:38.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:38 smithi084 ceph-mon[32629]: 8.6 scrub ok 2024-09-16T09:32:38.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:38 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/474313500' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:32:38.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:38 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:32:38.737 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:32:38.778 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:38.779 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:38.779 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:32:38.880 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:32:38.913 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:38.913 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:38.914 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:32:39.014 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:32:39.045 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:39.046 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:39.046 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:32:39.147 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:32:39.188 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:39.189 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:39.189 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:32:39.290 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:32:39.319 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:39.319 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:39.320 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:32:39.398 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:39 smithi070 ceph-mon[30388]: pgmap v352: 168 pgs: 2 active+recovering+undersized+remapped, 166 active+clean; 3.0 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.21k op/s; 1602/145668 objects misplaced (1.100%); 5.8 MiB/s, 92 objects/s recovering 2024-09-16T09:32:39.398 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:39 smithi070 ceph-mon[30388]: from='client.15456 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:32:39.398 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:39 smithi070 ceph-mon[30388]: 4.18 scrub starts 2024-09-16T09:32:39.398 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:39 smithi070 ceph-mon[30388]: 4.18 scrub ok 2024-09-16T09:32:39.398 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:39 smithi070 ceph-mon[30388]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-16T09:32:39.398 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:39 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-16T09:32:39.398 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:39 smithi070 ceph-mon[30388]: osdmap e210: 8 total, 8 up, 8 in 2024-09-16T09:32:39.398 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:39 smithi070 ceph-mon[40691]: pgmap v352: 168 pgs: 2 active+recovering+undersized+remapped, 166 active+clean; 3.0 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.21k op/s; 1602/145668 objects misplaced (1.100%); 5.8 MiB/s, 92 objects/s recovering 2024-09-16T09:32:39.398 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:39 smithi070 ceph-mon[40691]: from='client.15456 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:32:39.398 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:39 smithi070 ceph-mon[40691]: 4.18 scrub starts 2024-09-16T09:32:39.399 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:39 smithi070 ceph-mon[40691]: 4.18 scrub ok 2024-09-16T09:32:39.399 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:39 smithi070 ceph-mon[40691]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-16T09:32:39.399 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:39 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-16T09:32:39.399 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:39 smithi070 ceph-mon[40691]: osdmap e210: 8 total, 8 up, 8 in 2024-09-16T09:32:39.420 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:32:39.449 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:39.450 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:39.450 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:32:39.551 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:32:39.561 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T09:32:39.562 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [ 2024-09-16T09:32:39.562 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:39.562 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.25064.0:207887 8.f 8:fe57046d:::benchmark_data_smithi070_107919_object83292:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e211)", 2024-09-16T09:32:39.562 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:32:39.554821+0000", 2024-09-16T09:32:39.562 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.0049477569999999997, 2024-09-16T09:32:39.562 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0049787110000000002, 2024-09-16T09:32:39.562 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:32:39.562 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:32:39.562 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "waiting for sub ops", 2024-09-16T09:32:39.562 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:32:39.562 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.25064", 2024-09-16T09:32:39.562 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/2578256439", 2024-09-16T09:32:39.562 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 207887 2024-09-16T09:32:39.563 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:39.563 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:32:39.563 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:39.563 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:32:39.563 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:39.554821+0000", 2024-09-16T09:32:39.563 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:32:39.563 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:39.563 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:39.563 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:32:39.563 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:39.554821+0000", 2024-09-16T09:32:39.563 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:32:39.563 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:39.563 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:39.564 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:32:39.564 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:39.554822+0000", 2024-09-16T09:32:39.564 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6670000000000001e-06 2024-09-16T09:32:39.564 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:39.564 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:39.564 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:32:39.564 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:39.554828+0000", 2024-09-16T09:32:39.564 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.2469999999999999e-06 2024-09-16T09:32:39.564 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:39.564 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:39.564 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:32:39.564 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:39.554841+0000", 2024-09-16T09:32:39.565 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3533000000000001e-05 2024-09-16T09:32:39.565 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:39.565 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:39.565 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:32:39.565 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:39.554857+0000", 2024-09-16T09:32:39.565 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6106000000000001e-05 2024-09-16T09:32:39.565 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:39.565 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:39.565 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:32:39.565 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:39.554919+0000", 2024-09-16T09:32:39.565 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1443000000000001e-05 2024-09-16T09:32:39.566 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:39.566 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:39.566 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:32:39.566 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:39.555022+0000", 2024-09-16T09:32:39.566 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000103322 2024-09-16T09:32:39.566 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:39.566 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:39.566 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,3", 2024-09-16T09:32:39.566 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:39.555120+0000", 2024-09-16T09:32:39.566 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.8113000000000001e-05 2024-09-16T09:32:39.566 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:32:39.566 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:32:39.566 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:32:39.566 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:39.567 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:39.567 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.25064.0:207888 8.f 8:f69b45e6:::benchmark_data_smithi070_107919_object83293:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e211)", 2024-09-16T09:32:39.567 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:32:39.557284+0000", 2024-09-16T09:32:39.567 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.0024843299999999999, 2024-09-16T09:32:39.567 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0025806570000000001, 2024-09-16T09:32:39.567 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:32:39.567 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:32:39.567 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "waiting for sub ops", 2024-09-16T09:32:39.567 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:32:39.567 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.25064", 2024-09-16T09:32:39.567 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/2578256439", 2024-09-16T09:32:39.567 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 207888 2024-09-16T09:32:39.567 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:39.568 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:32:39.568 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:39.568 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:32:39.568 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:39.557284+0000", 2024-09-16T09:32:39.568 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:32:39.568 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:39.568 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:39.568 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:32:39.568 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:39.557284+0000", 2024-09-16T09:32:39.568 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:32:39.569 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:39.569 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:39.569 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:32:39.569 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:39.557286+0000", 2024-09-16T09:32:39.569 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.72e-06 2024-09-16T09:32:39.569 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:39.569 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:39.569 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:32:39.569 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:39.557291+0000", 2024-09-16T09:32:39.569 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.361e-06 2024-09-16T09:32:39.569 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:39.570 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:39.570 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:32:39.570 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:39.557304+0000", 2024-09-16T09:32:39.570 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3067999999999999e-05 2024-09-16T09:32:39.570 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:39.570 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:39.570 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:32:39.570 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:39.557312+0000", 2024-09-16T09:32:39.570 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6680000000000001e-06 2024-09-16T09:32:39.570 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:39.570 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:39.570 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:32:39.571 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:39.559100+0000", 2024-09-16T09:32:39.571 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0017882729999999999 2024-09-16T09:32:39.571 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:39.571 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:39.571 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:32:39.571 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:39.559220+0000", 2024-09-16T09:32:39.571 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00011985900000000001 2024-09-16T09:32:39.571 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:39.571 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:39.571 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,3", 2024-09-16T09:32:39.571 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:39.559332+0000", 2024-09-16T09:32:39.571 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000111734 2024-09-16T09:32:39.571 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:32:39.572 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:32:39.572 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:32:39.572 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:39.572 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:39.572 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.25064.0:207875 8.e e211/152 8:71214d79:::benchmark_data_smithi070_107919_object83280:head v 211'13065, mlcod=211'13063)", 2024-09-16T09:32:39.572 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:32:39.557781+0000", 2024-09-16T09:32:39.572 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.001986916, 2024-09-16T09:32:39.572 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0021408849999999999, 2024-09-16T09:32:39.572 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:32:39.572 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:32:39.572 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T09:32:39.572 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:32:39.572 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:39.573 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:32:39.573 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:39.557781+0000", 2024-09-16T09:32:39.573 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:32:39.573 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:39.573 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:39.573 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:32:39.573 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:39.557781+0000", 2024-09-16T09:32:39.573 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:32:39.573 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:39.573 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:39.573 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:32:39.573 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:39.557782+0000", 2024-09-16T09:32:39.573 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.7999999999999993e-07 2024-09-16T09:32:39.574 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:39.574 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:39.574 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:32:39.574 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:39.557789+0000", 2024-09-16T09:32:39.574 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.5910000000000001e-06 2024-09-16T09:32:39.574 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:39.574 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:39.574 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:32:39.574 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:39.557803+0000", 2024-09-16T09:32:39.574 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3723999999999999e-05 2024-09-16T09:32:39.574 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:39.574 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:39.574 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:32:39.574 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:39.557809+0000", 2024-09-16T09:32:39.575 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1979999999999998e-06 2024-09-16T09:32:39.575 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:39.575 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:39.575 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:32:39.575 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:39.557923+0000", 2024-09-16T09:32:39.575 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000113813 2024-09-16T09:32:39.575 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:39.575 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:39.575 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:32:39.575 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:39.557939+0000", 2024-09-16T09:32:39.575 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6470999999999999e-05 2024-09-16T09:32:39.575 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:32:39.575 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:32:39.575 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:32:39.576 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:32:39.576 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-16T09:32:39.576 INFO:teuthology.orchestra.run.smithi070.stdout: "num_ops": 3 2024-09-16T09:32:39.576 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T09:32:39.583 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:39.584 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:39.584 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:32:39.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:39 smithi084 ceph-mon[32629]: pgmap v352: 168 pgs: 2 active+recovering+undersized+remapped, 166 active+clean; 3.0 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.21k op/s; 1602/145668 objects misplaced (1.100%); 5.8 MiB/s, 92 objects/s recovering 2024-09-16T09:32:39.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:39 smithi084 ceph-mon[32629]: from='client.15456 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:32:39.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:39 smithi084 ceph-mon[32629]: 4.18 scrub starts 2024-09-16T09:32:39.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:39 smithi084 ceph-mon[32629]: 4.18 scrub ok 2024-09-16T09:32:39.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:39 smithi084 ceph-mon[32629]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-16T09:32:39.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:39 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-16T09:32:39.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:39 smithi084 ceph-mon[32629]: osdmap e210: 8 total, 8 up, 8 in 2024-09-16T09:32:39.685 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:32:39.716 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:39.717 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:39.717 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:32:39.818 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:32:39.861 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:39.861 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:39.862 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:32:39.963 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:32:39.991 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:39.992 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:39.992 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:32:40.093 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:32:40.142 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:40.143 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:40.143 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:32:40.244 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:32:40.276 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:40.277 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:40.277 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:32:40.378 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:32:40.411 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:40.412 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:40.412 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:32:40.513 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:32:40.530 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.2 dump_blocked_ops 2024-09-16T09:32:40.539 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:40.540 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:40.541 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:32:40.584 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:32:40.584 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:32:40.584 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:32:40.585 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:32:40.585 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:32:40.585 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:32:40.585 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:32:40.585 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:32:40.586 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:32:40.586 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:32:40.586 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:32:40.586 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:32:40.586 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:32:40.586 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:32:40.586 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:32:40.586 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:32:40.641 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:32:40.662 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:40 smithi084 ceph-mon[32629]: 1.11 scrub starts 2024-09-16T09:32:40.662 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:40 smithi084 ceph-mon[32629]: 1.11 scrub ok 2024-09-16T09:32:40.662 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:40 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:32:40.662 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:40 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:32:40.662 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:40 smithi084 ceph-mon[32629]: 6.e scrub starts 2024-09-16T09:32:40.662 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:40 smithi084 ceph-mon[32629]: 6.e scrub ok 2024-09-16T09:32:40.662 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:40 smithi084 ceph-mon[32629]: 3.1f scrub starts 2024-09-16T09:32:40.662 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:40 smithi084 ceph-mon[32629]: osdmap e211: 8 total, 8 up, 8 in 2024-09-16T09:32:40.662 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:40 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/2245397424' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:32:40.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:40 smithi070 ceph-mon[30388]: 1.11 scrub starts 2024-09-16T09:32:40.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:40 smithi070 ceph-mon[30388]: 1.11 scrub ok 2024-09-16T09:32:40.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:40 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:32:40.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:40 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:32:40.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:40 smithi070 ceph-mon[30388]: 6.e scrub starts 2024-09-16T09:32:40.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:40 smithi070 ceph-mon[30388]: 6.e scrub ok 2024-09-16T09:32:40.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:40 smithi070 ceph-mon[30388]: 3.1f scrub starts 2024-09-16T09:32:40.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:40 smithi070 ceph-mon[30388]: osdmap e211: 8 total, 8 up, 8 in 2024-09-16T09:32:40.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:40 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2245397424' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:32:40.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:40 smithi070 ceph-mon[40691]: 1.11 scrub starts 2024-09-16T09:32:40.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:40 smithi070 ceph-mon[40691]: 1.11 scrub ok 2024-09-16T09:32:40.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:40 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:32:40.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:40 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:32:40.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:40 smithi070 ceph-mon[40691]: 6.e scrub starts 2024-09-16T09:32:40.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:40 smithi070 ceph-mon[40691]: 6.e scrub ok 2024-09-16T09:32:40.674 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:40 smithi070 ceph-mon[40691]: 3.1f scrub starts 2024-09-16T09:32:40.674 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:40 smithi070 ceph-mon[40691]: osdmap e211: 8 total, 8 up, 8 in 2024-09-16T09:32:40.674 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:40 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/2245397424' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:32:40.675 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:40.676 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:40.676 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:32:40.777 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:32:40.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:40.820 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:40.820 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:32:40.921 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:32:40.966 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:40.967 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:40.968 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:32:41.068 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:32:41.107 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:41.107 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:41.107 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:32:41.208 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:32:41.241 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:41.241 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:41.242 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:32:41.308 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":352,"stamp":"2024-09-16T09:32:37.641443+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3169806378,"num_objects":48556,"num_object_clones":0,"num_object_copies":145668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1602,"num_objects_unfound":0,"num_objects_dirty":48556,"num_whiteouts":0,"num_read":1187,"num_read_kb":958,"num_write":201287,"num_write_kb":7974230,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13245,"num_bytes_recovered":859945760,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79757,"ondisk_log_size":79757,"up":504,"acting":502,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":506,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":12015440,"kb_used_data":9815240,"kb_used_omap":12,"kb_used_meta":2199859,"kb_avail":737945776,"statfs":{"total":767960285184,"available":755656474624,"internally_reserved":0,"allocated":10050805760,"data_stored":10048328865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":2252656178},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,4,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-795934720,"num_objects":-12145,"num_object_clones":0,"num_object_copies":-36435,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-924,"num_objects_unfound":0,"num_objects_dirty":-12145,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":12145,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":924,"num_bytes_recovered":60555264,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.005803"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":339,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.330251+0000","last_change":"2024-09-16T09:30:03.848439+0000","last_active":"2024-09-16T09:32:29.330251+0000","last_peered":"2024-09-16T09:32:29.330251+0000","last_clean":"2024-09-16T09:32:29.330251+0000","last_became_active":"2024-09-16T09:25:01.709108+0000","last_became_peered":"2024-09-16T09:25:01.709108+0000","last_unstale":"2024-09-16T09:32:29.330251+0000","last_undegraded":"2024-09-16T09:32:29.330251+0000","last_fullsized":"2024-09-16T09:32:29.330251+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:30:03.848354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:30:03.848354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"66'9","reported_seq":343,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.314422+0000","last_change":"2024-09-16T09:28:52.689373+0000","last_active":"2024-09-16T09:32:29.314422+0000","last_peered":"2024-09-16T09:32:29.314422+0000","last_clean":"2024-09-16T09:32:29.314422+0000","last_became_active":"2024-09-16T09:25:03.714635+0000","last_became_peered":"2024-09-16T09:25:03.714635+0000","last_unstale":"2024-09-16T09:32:29.314422+0000","last_undegraded":"2024-09-16T09:32:29.314422+0000","last_fullsized":"2024-09-16T09:32:29.314422+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:28:52.689304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:52.689304+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":331,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.317159+0000","last_change":"2024-09-16T09:29:17.910367+0000","last_active":"2024-09-16T09:32:29.317159+0000","last_peered":"2024-09-16T09:32:29.317159+0000","last_clean":"2024-09-16T09:32:29.317159+0000","last_became_active":"2024-09-16T09:25:05.715645+0000","last_became_peered":"2024-09-16T09:25:05.715645+0000","last_unstale":"2024-09-16T09:32:29.317159+0000","last_undegraded":"2024-09-16T09:32:29.317159+0000","last_fullsized":"2024-09-16T09:32:29.317159+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:17.910305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:17.910305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":310,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.315410+0000","last_change":"2024-09-16T09:27:11.565306+0000","last_active":"2024-09-16T09:32:29.315410+0000","last_peered":"2024-09-16T09:32:29.315410+0000","last_clean":"2024-09-16T09:32:29.315410+0000","last_became_active":"2024-09-16T09:25:07.715442+0000","last_became_peered":"2024-09-16T09:25:07.715442+0000","last_unstale":"2024-09-16T09:32:29.315410+0000","last_undegraded":"2024-09-16T09:32:29.315410+0000","last_fullsized":"2024-09-16T09:32:29.315410+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:11.565220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:11.565220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:25.985557+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":56,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.325798+0000","last_change":"2024-09-16T09:31:09.269700+0000","last_active":"2024-09-16T09:32:29.325798+0000","last_peered":"2024-09-16T09:32:29.325798+0000","last_clean":"2024-09-16T09:32:29.325798+0000","last_became_active":"2024-09-16T09:31:09.269342+0000","last_became_peered":"2024-09-16T09:31:09.269342+0000","last_unstale":"2024-09-16T09:32:29.325798+0000","last_undegraded":"2024-09-16T09:32:29.325798+0000","last_fullsized":"2024-09-16T09:32:29.325798+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:00.068570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:28:00.068570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":196,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.321201+0000","last_change":"2024-09-16T09:31:56.507595+0000","last_active":"2024-09-16T09:32:29.321201+0000","last_peered":"2024-09-16T09:32:29.321201+0000","last_clean":"2024-09-16T09:32:29.321201+0000","last_became_active":"2024-09-16T09:31:09.275136+0000","last_became_peered":"2024-09-16T09:31:09.275136+0000","last_unstale":"2024-09-16T09:32:29.321201+0000","last_undegraded":"2024-09-16T09:32:29.321201+0000","last_fullsized":"2024-09-16T09:32:29.321201+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:31:56.507528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:31:56.507528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:08.491873+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.19","version":"66'15","reported_seq":257,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.325132+0000","last_change":"2024-09-16T09:31:09.302903+0000","last_active":"2024-09-16T09:32:29.325132+0000","last_peered":"2024-09-16T09:32:29.325132+0000","last_clean":"2024-09-16T09:32:29.325132+0000","last_became_active":"2024-09-16T09:31:09.268750+0000","last_became_peered":"2024-09-16T09:31:09.268750+0000","last_unstale":"2024-09-16T09:32:29.325132+0000","last_undegraded":"2024-09-16T09:32:29.325132+0000","last_fullsized":"2024-09-16T09:32:29.325132+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-16T09:29:57.968162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:29:57.968162+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":331,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.314342+0000","last_change":"2024-09-16T09:29:15.628676+0000","last_active":"2024-09-16T09:32:29.314342+0000","last_peered":"2024-09-16T09:32:29.314342+0000","last_clean":"2024-09-16T09:32:29.314342+0000","last_became_active":"2024-09-16T09:25:05.717236+0000","last_became_peered":"2024-09-16T09:25:05.717236+0000","last_unstale":"2024-09-16T09:32:29.314342+0000","last_undegraded":"2024-09-16T09:32:29.314342+0000","last_fullsized":"2024-09-16T09:32:29.314342+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:15.628467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:15.628467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":169,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.317152+0000","last_change":"2024-09-16T09:28:34.839909+0000","last_active":"2024-09-16T09:32:29.317152+0000","last_peered":"2024-09-16T09:32:29.317152+0000","last_clean":"2024-09-16T09:32:29.317152+0000","last_became_active":"2024-09-16T09:28:34.839435+0000","last_became_peered":"2024-09-16T09:28:34.839435+0000","last_unstale":"2024-09-16T09:32:29.317152+0000","last_undegraded":"2024-09-16T09:32:29.317152+0000","last_fullsized":"2024-09-16T09:32:29.317152+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:01.038400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:28:01.038400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"66'9","reported_seq":343,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.315712+0000","last_change":"2024-09-16T09:29:28.622647+0000","last_active":"2024-09-16T09:32:29.315712+0000","last_peered":"2024-09-16T09:32:29.315712+0000","last_clean":"2024-09-16T09:32:29.315712+0000","last_became_active":"2024-09-16T09:25:03.709147+0000","last_became_peered":"2024-09-16T09:25:03.709147+0000","last_unstale":"2024-09-16T09:32:29.315712+0000","last_undegraded":"2024-09-16T09:32:29.315712+0000","last_fullsized":"2024-09-16T09:32:29.315712+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:29:28.622558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:29:28.622558+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":313,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.319490+0000","last_change":"2024-09-16T09:27:59.080369+0000","last_active":"2024-09-16T09:32:29.319490+0000","last_peered":"2024-09-16T09:32:29.319490+0000","last_clean":"2024-09-16T09:32:29.319490+0000","last_became_active":"2024-09-16T09:25:05.718134+0000","last_became_peered":"2024-09-16T09:25:05.718134+0000","last_unstale":"2024-09-16T09:32:29.319490+0000","last_undegraded":"2024-09-16T09:32:29.319490+0000","last_fullsized":"2024-09-16T09:32:29.319490+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:59.080307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:59.080307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"66'1","reported_seq":328,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.324827+0000","last_change":"2024-09-16T09:31:58.367903+0000","last_active":"2024-09-16T09:32:29.324827+0000","last_peered":"2024-09-16T09:32:29.324827+0000","last_clean":"2024-09-16T09:32:29.324827+0000","last_became_active":"2024-09-16T09:25:07.716558+0000","last_became_peered":"2024-09-16T09:25:07.716558+0000","last_unstale":"2024-09-16T09:32:29.324827+0000","last_undegraded":"2024-09-16T09:32:29.324827+0000","last_fullsized":"2024-09-16T09:32:29.324827+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-16T09:31:58.367837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:31:58.367837+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:20.121824+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":322,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.320947+0000","last_change":"2024-09-16T09:27:56.191264+0000","last_active":"2024-09-16T09:32:29.320947+0000","last_peered":"2024-09-16T09:32:29.320947+0000","last_clean":"2024-09-16T09:32:29.320947+0000","last_became_active":"2024-09-16T09:25:01.696939+0000","last_became_peered":"2024-09-16T09:25:01.696939+0000","last_unstale":"2024-09-16T09:32:29.320947+0000","last_undegraded":"2024-09-16T09:32:29.320947+0000","last_fullsized":"2024-09-16T09:32:29.320947+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:56.191220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:27:56.191220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"66'5","reported_seq":337,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:31.024598+0000","last_change":"2024-09-16T09:32:31.024521+0000","last_active":"2024-09-16T09:32:31.024598+0000","last_peered":"2024-09-16T09:32:31.024598+0000","last_clean":"2024-09-16T09:32:31.024598+0000","last_became_active":"2024-09-16T09:25:03.716136+0000","last_became_peered":"2024-09-16T09:25:03.716136+0000","last_unstale":"2024-09-16T09:32:31.024598+0000","last_undegraded":"2024-09-16T09:32:31.024598+0000","last_fullsized":"2024-09-16T09:32:31.024598+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-16T09:32:31.024450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:32:31.024450+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:41.830431+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":322,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.323497+0000","last_change":"2024-09-16T09:28:52.746046+0000","last_active":"2024-09-16T09:32:29.323497+0000","last_peered":"2024-09-16T09:32:29.323497+0000","last_clean":"2024-09-16T09:32:29.323497+0000","last_became_active":"2024-09-16T09:25:05.718281+0000","last_became_peered":"2024-09-16T09:25:05.718281+0000","last_unstale":"2024-09-16T09:32:29.323497+0000","last_undegraded":"2024-09-16T09:32:29.323497+0000","last_fullsized":"2024-09-16T09:32:29.323497+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:52.745999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:28:52.745999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":340,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.319654+0000","last_change":"2024-09-16T09:31:09.276016+0000","last_active":"2024-09-16T09:32:29.319654+0000","last_peered":"2024-09-16T09:32:29.319654+0000","last_clean":"2024-09-16T09:32:29.319654+0000","last_became_active":"2024-09-16T09:31:09.275656+0000","last_became_peered":"2024-09-16T09:31:09.275656+0000","last_unstale":"2024-09-16T09:32:29.319654+0000","last_undegraded":"2024-09-16T09:32:29.319654+0000","last_fullsized":"2024-09-16T09:32:29.319654+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:50.468134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:50.468134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":327,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.316037+0000","last_change":"2024-09-16T09:29:56.605555+0000","last_active":"2024-09-16T09:32:29.316037+0000","last_peered":"2024-09-16T09:32:29.316037+0000","last_clean":"2024-09-16T09:32:29.316037+0000","last_became_active":"2024-09-16T09:25:07.720793+0000","last_became_peered":"2024-09-16T09:25:07.720793+0000","last_unstale":"2024-09-16T09:32:29.316037+0000","last_undegraded":"2024-09-16T09:32:29.316037+0000","last_fullsized":"2024-09-16T09:32:29.316037+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:56.605488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:56.605488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":78,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.319840+0000","last_change":"2024-09-16T09:30:39.202178+0000","last_active":"2024-09-16T09:32:29.319840+0000","last_peered":"2024-09-16T09:32:29.319840+0000","last_clean":"2024-09-16T09:32:29.319840+0000","last_became_active":"2024-09-16T09:30:39.201824+0000","last_became_peered":"2024-09-16T09:30:39.201824+0000","last_unstale":"2024-09-16T09:32:29.319840+0000","last_undegraded":"2024-09-16T09:32:29.319840+0000","last_fullsized":"2024-09-16T09:32:29.319840+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:50.912979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:50.912979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:23.532853+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"66'15","reported_seq":405,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.328758+0000","last_change":"2024-09-16T09:31:09.302018+0000","last_active":"2024-09-16T09:32:29.328758+0000","last_peered":"2024-09-16T09:32:29.328758+0000","last_clean":"2024-09-16T09:32:29.328758+0000","last_became_active":"2024-09-16T09:31:09.263534+0000","last_became_peered":"2024-09-16T09:31:09.263534+0000","last_unstale":"2024-09-16T09:32:29.328758+0000","last_undegraded":"2024-09-16T09:32:29.328758+0000","last_fullsized":"2024-09-16T09:32:29.328758+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-16T09:30:21.932604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:30:21.932604+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":322,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.318220+0000","last_change":"2024-09-16T09:29:55.875536+0000","last_active":"2024-09-16T09:32:29.318220+0000","last_peered":"2024-09-16T09:32:29.318220+0000","last_clean":"2024-09-16T09:32:29.318220+0000","last_became_active":"2024-09-16T09:25:05.713929+0000","last_became_peered":"2024-09-16T09:25:05.713929+0000","last_unstale":"2024-09-16T09:32:29.318220+0000","last_undegraded":"2024-09-16T09:32:29.318220+0000","last_fullsized":"2024-09-16T09:32:29.318220+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:55.875480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:55.875480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":175,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.320980+0000","last_change":"2024-09-16T09:30:24.412733+0000","last_active":"2024-09-16T09:32:29.320980+0000","last_peered":"2024-09-16T09:32:29.320980+0000","last_clean":"2024-09-16T09:32:29.320980+0000","last_became_active":"2024-09-16T09:28:52.974255+0000","last_became_peered":"2024-09-16T09:28:52.974255+0000","last_unstale":"2024-09-16T09:32:29.320980+0000","last_undegraded":"2024-09-16T09:32:29.320980+0000","last_fullsized":"2024-09-16T09:32:29.320980+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:30:24.412571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:30:24.412571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:49.424390+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":339,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.315856+0000","last_change":"2024-09-16T09:30:03.628882+0000","last_active":"2024-09-16T09:32:29.315856+0000","last_peered":"2024-09-16T09:32:29.315856+0000","last_clean":"2024-09-16T09:32:29.315856+0000","last_became_active":"2024-09-16T09:25:01.706491+0000","last_became_peered":"2024-09-16T09:25:01.706491+0000","last_unstale":"2024-09-16T09:32:29.315856+0000","last_undegraded":"2024-09-16T09:32:29.315856+0000","last_fullsized":"2024-09-16T09:32:29.315856+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:30:03.628812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:30:03.628812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"66'12","reported_seq":239,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.318436+0000","last_change":"2024-09-16T09:31:09.286673+0000","last_active":"2024-09-16T09:32:29.318436+0000","last_peered":"2024-09-16T09:32:29.318436+0000","last_clean":"2024-09-16T09:32:29.318436+0000","last_became_active":"2024-09-16T09:31:09.268588+0000","last_became_peered":"2024-09-16T09:31:09.268588+0000","last_unstale":"2024-09-16T09:32:29.318436+0000","last_undegraded":"2024-09-16T09:32:29.318436+0000","last_fullsized":"2024-09-16T09:32:29.318436+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-16T09:27:01.700139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:01.700139+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":24,"num_read_kb":15,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":314,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.315787+0000","last_change":"2024-09-16T09:27:00.816500+0000","last_active":"2024-09-16T09:32:29.315787+0000","last_peered":"2024-09-16T09:32:29.315787+0000","last_clean":"2024-09-16T09:32:29.315787+0000","last_became_active":"2024-09-16T09:25:05.710093+0000","last_became_peered":"2024-09-16T09:25:05.710093+0000","last_unstale":"2024-09-16T09:32:29.315787+0000","last_undegraded":"2024-09-16T09:32:29.315787+0000","last_fullsized":"2024-09-16T09:32:29.315787+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:00.816424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:00.816424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:28.982331+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":318,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.322753+0000","last_change":"2024-09-16T09:29:29.918433+0000","last_active":"2024-09-16T09:32:29.322753+0000","last_peered":"2024-09-16T09:32:29.322753+0000","last_clean":"2024-09-16T09:32:29.322753+0000","last_became_active":"2024-09-16T09:25:07.719380+0000","last_became_peered":"2024-09-16T09:25:07.719380+0000","last_unstale":"2024-09-16T09:32:29.322753+0000","last_undegraded":"2024-09-16T09:32:29.322753+0000","last_fullsized":"2024-09-16T09:32:29.322753+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-16T09:29:29.918368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:29.918368+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":340,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.317303+0000","last_change":"2024-09-16T09:30:02.843932+0000","last_active":"2024-09-16T09:32:29.317303+0000","last_peered":"2024-09-16T09:32:29.317303+0000","last_clean":"2024-09-16T09:32:29.317303+0000","last_became_active":"2024-09-16T09:25:01.699545+0000","last_became_peered":"2024-09-16T09:25:01.699545+0000","last_unstale":"2024-09-16T09:32:29.317303+0000","last_undegraded":"2024-09-16T09:32:29.317303+0000","last_fullsized":"2024-09-16T09:32:29.317303+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:30:02.843856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:30:02.843856+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"66'10","reported_seq":388,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:30.286077+0000","last_change":"2024-09-16T09:32:30.286077+0000","last_active":"2024-09-16T09:32:30.286077+0000","last_peered":"2024-09-16T09:32:30.286077+0000","last_clean":"2024-09-16T09:32:30.286077+0000","last_became_active":"2024-09-16T09:31:09.275050+0000","last_became_peered":"2024-09-16T09:31:09.275050+0000","last_unstale":"2024-09-16T09:32:30.286077+0000","last_undegraded":"2024-09-16T09:32:30.286077+0000","last_fullsized":"2024-09-16T09:32:30.286077+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-16T09:32:30.284919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:32:30.284919+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:30.659184+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"66'8","reported_seq":442,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:31.850017+0000","last_change":"2024-09-16T09:30:31.392771+0000","last_active":"2024-09-16T09:32:31.850017+0000","last_peered":"2024-09-16T09:32:31.850017+0000","last_clean":"2024-09-16T09:32:31.850017+0000","last_became_active":"2024-09-16T09:25:05.712516+0000","last_became_peered":"2024-09-16T09:25:05.712516+0000","last_unstale":"2024-09-16T09:32:31.850017+0000","last_undegraded":"2024-09-16T09:32:31.850017+0000","last_fullsized":"2024-09-16T09:32:31.850017+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:30:31.392399+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:30:31.392399+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:44.469073+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"209'12556","reported_seq":12682,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:32.943082+0000","last_change":"2024-09-16T09:29:49.449011+0000","last_active":"2024-09-16T09:32:32.943082+0000","last_peered":"2024-09-16T09:32:32.943082+0000","last_clean":"2024-09-16T09:32:32.943082+0000","last_became_active":"2024-09-16T09:29:49.448473+0000","last_became_peered":"2024-09-16T09:29:49.448473+0000","last_unstale":"2024-09-16T09:32:32.943082+0000","last_undegraded":"2024-09-16T09:32:32.943082+0000","last_fullsized":"2024-09-16T09:32:32.943082+0000","mapping_epoch":152,"log_start":"194'8000","ondisk_log_start":"194'8000","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":4556,"log_dups_size":0,"ondisk_log_size":4556,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":200671232,"num_objects":3062,"num_object_clones":0,"num_object_copies":9186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3062,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12556,"num_write_kb":499776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.a","version":"73'79","reported_seq":261,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.319511+0000","last_change":"2024-09-16T09:29:18.498887+0000","last_active":"2024-09-16T09:32:29.319511+0000","last_peered":"2024-09-16T09:32:29.319511+0000","last_clean":"2024-09-16T09:32:29.319511+0000","last_became_active":"2024-09-16T09:26:35.127352+0000","last_became_peered":"2024-09-16T09:26:35.127352+0000","last_unstale":"2024-09-16T09:32:29.319511+0000","last_undegraded":"2024-09-16T09:32:29.319511+0000","last_fullsized":"2024-09-16T09:32:29.319511+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:18.498766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:18.498766+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"66'15","reported_seq":270,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.325918+0000","last_change":"2024-09-16T09:28:28.065781+0000","last_active":"2024-09-16T09:32:29.325918+0000","last_peered":"2024-09-16T09:32:29.325918+0000","last_clean":"2024-09-16T09:32:29.325918+0000","last_became_active":"2024-09-16T09:26:33.295672+0000","last_became_peered":"2024-09-16T09:26:33.295672+0000","last_unstale":"2024-09-16T09:32:29.325918+0000","last_undegraded":"2024-09-16T09:32:29.325918+0000","last_fullsized":"2024-09-16T09:32:29.325918+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-16T09:28:28.065699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:28.065699+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:23.961200+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":56,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.335603+0000","last_change":"2024-09-16T09:31:09.276858+0000","last_active":"2024-09-16T09:32:29.335603+0000","last_peered":"2024-09-16T09:32:29.335603+0000","last_clean":"2024-09-16T09:32:29.335603+0000","last_became_active":"2024-09-16T09:31:09.274534+0000","last_became_peered":"2024-09-16T09:31:09.274534+0000","last_unstale":"2024-09-16T09:32:29.335603+0000","last_undegraded":"2024-09-16T09:32:29.335603+0000","last_fullsized":"2024-09-16T09:32:29.335603+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:22.091537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:28:22.091537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"66'8","reported_seq":406,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.316172+0000","last_change":"2024-09-16T09:25:05.714128+0000","last_active":"2024-09-16T09:32:29.316172+0000","last_peered":"2024-09-16T09:32:29.316172+0000","last_clean":"2024-09-16T09:32:29.316172+0000","last_became_active":"2024-09-16T09:25:05.713813+0000","last_became_peered":"2024-09-16T09:25:05.713813+0000","last_unstale":"2024-09-16T09:32:29.316172+0000","last_undegraded":"2024-09-16T09:32:29.316172+0000","last_fullsized":"2024-09-16T09:32:29.316172+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:17.970742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":318,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.319405+0000","last_change":"2024-09-16T09:29:24.537802+0000","last_active":"2024-09-16T09:32:29.319405+0000","last_peered":"2024-09-16T09:32:29.319405+0000","last_clean":"2024-09-16T09:32:29.319405+0000","last_became_active":"2024-09-16T09:25:07.717344+0000","last_became_peered":"2024-09-16T09:25:07.717344+0000","last_unstale":"2024-09-16T09:32:29.319405+0000","last_undegraded":"2024-09-16T09:32:29.319405+0000","last_fullsized":"2024-09-16T09:32:29.319405+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:24.537731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:24.537731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.c","version":"209'12479","reported_seq":12663,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:33.197723+0000","last_change":"2024-09-16T09:32:07.676294+0000","last_active":"2024-09-16T09:32:33.197723+0000","last_peered":"2024-09-16T09:32:33.197723+0000","last_clean":"2024-09-16T09:32:33.197723+0000","last_became_active":"2024-09-16T09:29:50.007935+0000","last_became_peered":"2024-09-16T09:29:50.007935+0000","last_unstale":"2024-09-16T09:32:33.197723+0000","last_undegraded":"2024-09-16T09:32:33.197723+0000","last_fullsized":"2024-09-16T09:32:33.197723+0000","mapping_epoch":152,"log_start":"190'7300","ondisk_log_start":"190'7300","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"204'10667","last_scrub_stamp":"2024-09-16T09:32:07.676020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:32:07.676020+0000","objects_scrubbed":4920,"log_size":5179,"log_dups_size":0,"ondisk_log_size":5179,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:18.894513+0000","scrub_duration":1433,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":201129984,"num_objects":3069,"num_object_clones":0,"num_object_copies":9207,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3069,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12479,"num_write_kb":497536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"73'79","reported_seq":293,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.316366+0000","last_change":"2024-09-16T09:32:13.446353+0000","last_active":"2024-09-16T09:32:29.316366+0000","last_peered":"2024-09-16T09:32:29.316366+0000","last_clean":"2024-09-16T09:32:29.316366+0000","last_became_active":"2024-09-16T09:30:08.763683+0000","last_became_peered":"2024-09-16T09:30:08.763683+0000","last_unstale":"2024-09-16T09:32:29.316366+0000","last_undegraded":"2024-09-16T09:32:29.316366+0000","last_fullsized":"2024-09-16T09:32:29.316366+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:58.568841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:58.568841+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:32:41.868241+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"66'18","reported_seq":268,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.322644+0000","last_change":"2024-09-16T09:31:09.407700+0000","last_active":"2024-09-16T09:32:29.322644+0000","last_peered":"2024-09-16T09:32:29.322644+0000","last_clean":"2024-09-16T09:32:29.322644+0000","last_became_active":"2024-09-16T09:31:09.270434+0000","last_became_peered":"2024-09-16T09:31:09.270434+0000","last_unstale":"2024-09-16T09:32:29.322644+0000","last_undegraded":"2024-09-16T09:32:29.322644+0000","last_fullsized":"2024-09-16T09:32:29.322644+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'18","last_scrub_stamp":"2024-09-16T09:28:54.699778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:54.699778+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":33,"num_read_kb":21,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":166,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.323324+0000","last_change":"2024-09-16T09:28:58.705169+0000","last_active":"2024-09-16T09:32:29.323324+0000","last_peered":"2024-09-16T09:32:29.323324+0000","last_clean":"2024-09-16T09:32:29.323324+0000","last_became_active":"2024-09-16T09:28:52.983543+0000","last_became_peered":"2024-09-16T09:28:52.983543+0000","last_unstale":"2024-09-16T09:32:29.323324+0000","last_undegraded":"2024-09-16T09:32:29.323324+0000","last_fullsized":"2024-09-16T09:32:29.323324+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:58.705097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:28:58.705097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":345,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.314600+0000","last_change":"2024-09-16T09:31:09.269234+0000","last_active":"2024-09-16T09:32:29.314600+0000","last_peered":"2024-09-16T09:32:29.314600+0000","last_clean":"2024-09-16T09:32:29.314600+0000","last_became_active":"2024-09-16T09:31:09.268864+0000","last_became_peered":"2024-09-16T09:31:09.268864+0000","last_unstale":"2024-09-16T09:32:29.314600+0000","last_undegraded":"2024-09-16T09:32:29.314600+0000","last_fullsized":"2024-09-16T09:32:29.314600+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:50.482240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:50.482240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":333,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:32.060608+0000","last_change":"2024-09-16T09:32:32.060608+0000","last_active":"2024-09-16T09:32:32.060608+0000","last_peered":"2024-09-16T09:32:32.060608+0000","last_clean":"2024-09-16T09:32:32.060608+0000","last_became_active":"2024-09-16T09:31:09.269433+0000","last_became_peered":"2024-09-16T09:31:09.269433+0000","last_unstale":"2024-09-16T09:32:32.060608+0000","last_undegraded":"2024-09-16T09:32:32.060608+0000","last_fullsized":"2024-09-16T09:32:32.060608+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:32.060564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:32:32.060564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:48.641239+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.d","version":"209'12494","reported_seq":12620,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:32.976671+0000","last_change":"2024-09-16T09:29:50.010860+0000","last_active":"2024-09-16T09:32:32.976671+0000","last_peered":"2024-09-16T09:32:32.976671+0000","last_clean":"2024-09-16T09:32:32.976671+0000","last_became_active":"2024-09-16T09:29:50.010421+0000","last_became_peered":"2024-09-16T09:29:50.010421+0000","last_unstale":"2024-09-16T09:32:32.976671+0000","last_undegraded":"2024-09-16T09:32:32.976671+0000","last_fullsized":"2024-09-16T09:32:32.976671+0000","mapping_epoch":152,"log_start":"193'7800","ondisk_log_start":"193'7800","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":4694,"log_dups_size":0,"ondisk_log_size":4694,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":201261080,"num_objects":3072,"num_object_clones":0,"num_object_copies":9216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3072,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12494,"num_write_kb":498049,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,2],"acting":[5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.4","version":"73'79","reported_seq":56,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.326623+0000","last_change":"2024-09-16T09:31:09.275499+0000","last_active":"2024-09-16T09:32:29.326623+0000","last_peered":"2024-09-16T09:32:29.326623+0000","last_clean":"2024-09-16T09:32:29.326623+0000","last_became_active":"2024-09-16T09:31:09.275139+0000","last_became_peered":"2024-09-16T09:31:09.275139+0000","last_unstale":"2024-09-16T09:32:29.326623+0000","last_undegraded":"2024-09-16T09:32:29.326623+0000","last_fullsized":"2024-09-16T09:32:29.326623+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:30:22.438378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:30:22.438378+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"66'14","reported_seq":348,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.314673+0000","last_change":"2024-09-16T09:28:32.595064+0000","last_active":"2024-09-16T09:32:29.314673+0000","last_peered":"2024-09-16T09:32:29.314673+0000","last_clean":"2024-09-16T09:32:29.314673+0000","last_became_active":"2024-09-16T09:25:03.708830+0000","last_became_peered":"2024-09-16T09:25:03.708830+0000","last_unstale":"2024-09-16T09:32:29.314673+0000","last_undegraded":"2024-09-16T09:32:29.314673+0000","last_fullsized":"2024-09-16T09:32:29.314673+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'14","last_scrub_stamp":"2024-09-16T09:28:32.594988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:32.594988+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":348,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.329393+0000","last_change":"2024-09-16T09:31:57.373606+0000","last_active":"2024-09-16T09:32:29.329393+0000","last_peered":"2024-09-16T09:32:29.329393+0000","last_clean":"2024-09-16T09:32:29.329393+0000","last_became_active":"2024-09-16T09:25:01.702992+0000","last_became_peered":"2024-09-16T09:25:01.702992+0000","last_unstale":"2024-09-16T09:32:29.329393+0000","last_undegraded":"2024-09-16T09:32:29.329393+0000","last_fullsized":"2024-09-16T09:32:29.329393+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:31:57.373544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:31:57.373544+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:24.193683+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"187'11","reported_seq":258,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.319144+0000","last_change":"2024-09-16T09:31:09.397758+0000","last_active":"2024-09-16T09:32:29.319144+0000","last_peered":"2024-09-16T09:32:29.319144+0000","last_clean":"2024-09-16T09:32:29.319144+0000","last_became_active":"2024-09-16T09:31:09.269173+0000","last_became_peered":"2024-09-16T09:31:09.269173+0000","last_unstale":"2024-09-16T09:32:29.319144+0000","last_undegraded":"2024-09-16T09:32:29.319144+0000","last_fullsized":"2024-09-16T09:32:29.319144+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'9","last_scrub_stamp":"2024-09-16T09:29:21.898533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:21.898533+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":310,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.323032+0000","last_change":"2024-09-16T09:26:40.699888+0000","last_active":"2024-09-16T09:32:29.323032+0000","last_peered":"2024-09-16T09:32:29.323032+0000","last_clean":"2024-09-16T09:32:29.323032+0000","last_became_active":"2024-09-16T09:25:07.722017+0000","last_became_peered":"2024-09-16T09:25:07.722017+0000","last_unstale":"2024-09-16T09:32:29.323032+0000","last_undegraded":"2024-09-16T09:32:29.323032+0000","last_fullsized":"2024-09-16T09:32:29.323032+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:40.699813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:40.699813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.e","version":"209'12592","reported_seq":12714,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:33.184390+0000","last_change":"2024-09-16T09:29:49.448081+0000","last_active":"2024-09-16T09:32:33.184390+0000","last_peered":"2024-09-16T09:32:33.184390+0000","last_clean":"2024-09-16T09:32:33.184390+0000","last_became_active":"2024-09-16T09:29:49.447767+0000","last_became_peered":"2024-09-16T09:29:49.447767+0000","last_unstale":"2024-09-16T09:32:33.184390+0000","last_undegraded":"2024-09-16T09:32:33.184390+0000","last_fullsized":"2024-09-16T09:32:33.184390+0000","mapping_epoch":152,"log_start":"192'7500","ondisk_log_start":"192'7500","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":5092,"log_dups_size":0,"ondisk_log_size":5092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":197525504,"num_objects":3014,"num_object_clones":0,"num_object_copies":9042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3014,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12592,"num_write_kb":499392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"73'79","reported_seq":379,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.320881+0000","last_change":"2024-09-16T09:31:09.275413+0000","last_active":"2024-09-16T09:32:29.320881+0000","last_peered":"2024-09-16T09:32:29.320881+0000","last_clean":"2024-09-16T09:32:29.320881+0000","last_became_active":"2024-09-16T09:31:09.274707+0000","last_became_peered":"2024-09-16T09:31:09.274707+0000","last_unstale":"2024-09-16T09:32:29.320881+0000","last_undegraded":"2024-09-16T09:32:29.320881+0000","last_fullsized":"2024-09-16T09:32:29.320881+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:30:23.455388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:30:23.455388+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:41.421930+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"66'10","reported_seq":350,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.317325+0000","last_change":"2024-09-16T09:30:04.848306+0000","last_active":"2024-09-16T09:32:29.317325+0000","last_peered":"2024-09-16T09:32:29.317325+0000","last_clean":"2024-09-16T09:32:29.317325+0000","last_became_active":"2024-09-16T09:25:03.712985+0000","last_became_peered":"2024-09-16T09:25:03.712985+0000","last_unstale":"2024-09-16T09:32:29.317325+0000","last_undegraded":"2024-09-16T09:32:29.317325+0000","last_fullsized":"2024-09-16T09:32:29.317325+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-16T09:30:04.848234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:30:04.848234+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":336,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.321525+0000","last_change":"2024-09-16T09:31:09.272049+0000","last_active":"2024-09-16T09:32:29.321525+0000","last_peered":"2024-09-16T09:32:29.321525+0000","last_clean":"2024-09-16T09:32:29.321525+0000","last_became_active":"2024-09-16T09:31:09.270619+0000","last_became_peered":"2024-09-16T09:31:09.270619+0000","last_unstale":"2024-09-16T09:32:29.321525+0000","last_undegraded":"2024-09-16T09:32:29.321525+0000","last_fullsized":"2024-09-16T09:32:29.321525+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:25.086322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:25.086322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"66'8","reported_seq":422,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.326916+0000","last_change":"2024-09-16T09:28:57.628615+0000","last_active":"2024-09-16T09:32:29.326916+0000","last_peered":"2024-09-16T09:32:29.326916+0000","last_clean":"2024-09-16T09:32:29.326916+0000","last_became_active":"2024-09-16T09:25:05.714507+0000","last_became_peered":"2024-09-16T09:25:05.714507+0000","last_unstale":"2024-09-16T09:32:29.326916+0000","last_undegraded":"2024-09-16T09:32:29.326916+0000","last_fullsized":"2024-09-16T09:32:29.326916+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:28:57.628543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:28:57.628543+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:51.253847+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":333,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.324969+0000","last_change":"2024-09-16T09:31:09.283816+0000","last_active":"2024-09-16T09:32:29.324969+0000","last_peered":"2024-09-16T09:32:29.324969+0000","last_clean":"2024-09-16T09:32:29.324969+0000","last_became_active":"2024-09-16T09:31:09.283192+0000","last_became_peered":"2024-09-16T09:31:09.283192+0000","last_unstale":"2024-09-16T09:32:29.324969+0000","last_undegraded":"2024-09-16T09:32:29.324969+0000","last_fullsized":"2024-09-16T09:32:29.324969+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:22.563395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:28:22.563395+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.f","version":"209'12568","reported_seq":12694,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:32.866042+0000","last_change":"2024-09-16T09:29:49.838498+0000","last_active":"2024-09-16T09:32:32.866042+0000","last_peered":"2024-09-16T09:32:32.866042+0000","last_clean":"2024-09-16T09:32:32.866042+0000","last_became_active":"2024-09-16T09:29:49.838146+0000","last_became_peered":"2024-09-16T09:29:49.838146+0000","last_unstale":"2024-09-16T09:32:32.866042+0000","last_undegraded":"2024-09-16T09:32:32.866042+0000","last_fullsized":"2024-09-16T09:32:32.866042+0000","mapping_epoch":152,"log_start":"184'6500","ondisk_log_start":"184'6500","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":6068,"log_dups_size":0,"ondisk_log_size":6068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":208273408,"num_objects":3178,"num_object_clones":0,"num_object_copies":9534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3178,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12568,"num_write_kb":503872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"73'79","reported_seq":360,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.320985+0000","last_change":"2024-09-16T09:31:09.274975+0000","last_active":"2024-09-16T09:32:29.320985+0000","last_peered":"2024-09-16T09:32:29.320985+0000","last_clean":"2024-09-16T09:32:29.320985+0000","last_became_active":"2024-09-16T09:31:09.274565+0000","last_became_peered":"2024-09-16T09:31:09.274565+0000","last_unstale":"2024-09-16T09:32:29.320985+0000","last_undegraded":"2024-09-16T09:32:29.320985+0000","last_fullsized":"2024-09-16T09:32:29.320985+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:30:22.415051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:30:22.415051+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:52.385618+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"66'19","reported_seq":363,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.327093+0000","last_change":"2024-09-16T09:31:35.249952+0000","last_active":"2024-09-16T09:32:29.327093+0000","last_peered":"2024-09-16T09:32:29.327093+0000","last_clean":"2024-09-16T09:32:29.327093+0000","last_became_active":"2024-09-16T09:25:03.714797+0000","last_became_peered":"2024-09-16T09:25:03.714797+0000","last_unstale":"2024-09-16T09:32:29.327093+0000","last_undegraded":"2024-09-16T09:32:29.327093+0000","last_fullsized":"2024-09-16T09:32:29.327093+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-16T09:31:35.249864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:31:35.249864+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:40.632025+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":339,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.318867+0000","last_change":"2024-09-16T09:29:16.888357+0000","last_active":"2024-09-16T09:32:29.318867+0000","last_peered":"2024-09-16T09:32:29.318867+0000","last_clean":"2024-09-16T09:32:29.318867+0000","last_became_active":"2024-09-16T09:25:01.706806+0000","last_became_peered":"2024-09-16T09:25:01.706806+0000","last_unstale":"2024-09-16T09:32:29.318867+0000","last_undegraded":"2024-09-16T09:32:29.318867+0000","last_fullsized":"2024-09-16T09:32:29.318867+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:16.888236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:16.888236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":339,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.320017+0000","last_change":"2024-09-16T09:31:55.456669+0000","last_active":"2024-09-16T09:32:29.320017+0000","last_peered":"2024-09-16T09:32:29.320017+0000","last_clean":"2024-09-16T09:32:29.320017+0000","last_became_active":"2024-09-16T09:25:05.708663+0000","last_became_peered":"2024-09-16T09:25:05.708663+0000","last_unstale":"2024-09-16T09:32:29.320017+0000","last_undegraded":"2024-09-16T09:32:29.320017+0000","last_fullsized":"2024-09-16T09:32:29.320017+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:31:55.456592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:31:55.456592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:56.006473+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":311,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.317068+0000","last_change":"2024-09-16T09:26:37.282798+0000","last_active":"2024-09-16T09:32:29.317068+0000","last_peered":"2024-09-16T09:32:29.317068+0000","last_clean":"2024-09-16T09:32:29.317068+0000","last_became_active":"2024-09-16T09:25:07.719267+0000","last_became_peered":"2024-09-16T09:25:07.719267+0000","last_unstale":"2024-09-16T09:32:29.317068+0000","last_undegraded":"2024-09-16T09:32:29.317068+0000","last_fullsized":"2024-09-16T09:32:29.317068+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:37.282735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:37.282735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.8","version":"209'12641","reported_seq":20213,"reported_epoch":209,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-16T09:32:32.993229+0000","last_change":"2024-09-16T09:31:09.438251+0000","last_active":"2024-09-16T09:32:32.993229+0000","last_peered":"2024-09-16T09:32:32.993229+0000","last_clean":"2024-09-16T09:30:20.474651+0000","last_became_active":"2024-09-16T09:31:09.409873+0000","last_became_peered":"2024-09-16T09:31:09.409873+0000","last_unstale":"2024-09-16T09:32:32.993229+0000","last_undegraded":"2024-09-16T09:32:32.993229+0000","last_fullsized":"2024-09-16T09:31:09.256524+0000","mapping_epoch":186,"log_start":"193'7900","ondisk_log_start":"193'7900","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":4741,"log_dups_size":0,"ondisk_log_size":4741,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":201916416,"num_objects":3081,"num_object_clones":0,"num_object_copies":9243,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":738,"num_objects_unfound":0,"num_objects_dirty":3081,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12641,"num_write_kb":503104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3718,"num_bytes_recovered":243662848,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,7],"avail_no_missing":["5","7"],"object_location_counts":[{"shards":"0,5,7","objects":738},{"shards":"5,7","objects":2343}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1","version":"73'79","reported_seq":285,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.319293+0000","last_change":"2024-09-16T09:32:13.444142+0000","last_active":"2024-09-16T09:32:29.319293+0000","last_peered":"2024-09-16T09:32:29.319293+0000","last_clean":"2024-09-16T09:32:29.319293+0000","last_became_active":"2024-09-16T09:30:08.766350+0000","last_became_peered":"2024-09-16T09:30:08.766350+0000","last_unstale":"2024-09-16T09:32:29.319293+0000","last_undegraded":"2024-09-16T09:32:29.319293+0000","last_fullsized":"2024-09-16T09:32:29.319293+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:15.460179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:15.460179+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:32:42.562412+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"187'30","reported_seq":524,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.318786+0000","last_change":"2024-09-16T09:31:09.323281+0000","last_active":"2024-09-16T09:32:29.318786+0000","last_peered":"2024-09-16T09:32:29.318786+0000","last_clean":"2024-09-16T09:32:29.318786+0000","last_became_active":"2024-09-16T09:31:09.278839+0000","last_became_peered":"2024-09-16T09:31:09.278839+0000","last_unstale":"2024-09-16T09:32:29.318786+0000","last_undegraded":"2024-09-16T09:32:29.318786+0000","last_fullsized":"2024-09-16T09:32:29.318786+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'28","last_scrub_stamp":"2024-09-16T09:29:18.937623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:29:18.937623+0000","objects_scrubbed":10,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":331,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.316414+0000","last_change":"2024-09-16T09:29:26.637070+0000","last_active":"2024-09-16T09:32:29.316414+0000","last_peered":"2024-09-16T09:32:29.316414+0000","last_clean":"2024-09-16T09:32:29.316414+0000","last_became_active":"2024-09-16T09:25:01.703076+0000","last_became_peered":"2024-09-16T09:25:01.703076+0000","last_unstale":"2024-09-16T09:32:29.316414+0000","last_undegraded":"2024-09-16T09:32:29.316414+0000","last_fullsized":"2024-09-16T09:32:29.316414+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:26.637027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:26.637027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":340,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.321074+0000","last_change":"2024-09-16T09:29:29.486284+0000","last_active":"2024-09-16T09:32:29.321074+0000","last_peered":"2024-09-16T09:32:29.321074+0000","last_clean":"2024-09-16T09:32:29.321074+0000","last_became_active":"2024-09-16T09:25:00.696237+0000","last_became_peered":"2024-09-16T09:25:00.696237+0000","last_unstale":"2024-09-16T09:32:29.321074+0000","last_undegraded":"2024-09-16T09:32:29.321074+0000","last_fullsized":"2024-09-16T09:32:29.321074+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-16T09:29:29.486188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:29:29.486188+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.0025137430000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":307,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.327600+0000","last_change":"2024-09-16T09:25:05.718319+0000","last_active":"2024-09-16T09:32:29.327600+0000","last_peered":"2024-09-16T09:32:29.327600+0000","last_clean":"2024-09-16T09:32:29.327600+0000","last_became_active":"2024-09-16T09:25:05.717591+0000","last_became_peered":"2024-09-16T09:25:05.717591+0000","last_unstale":"2024-09-16T09:32:29.327600+0000","last_undegraded":"2024-09-16T09:32:29.327600+0000","last_fullsized":"2024-09-16T09:32:29.327600+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:11.173707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":56,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.329280+0000","last_change":"2024-09-16T09:31:09.272967+0000","last_active":"2024-09-16T09:32:29.329280+0000","last_peered":"2024-09-16T09:32:29.329280+0000","last_clean":"2024-09-16T09:32:29.329280+0000","last_became_active":"2024-09-16T09:31:09.272609+0000","last_became_peered":"2024-09-16T09:31:09.272609+0000","last_unstale":"2024-09-16T09:32:29.329280+0000","last_undegraded":"2024-09-16T09:32:29.329280+0000","last_fullsized":"2024-09-16T09:32:29.329280+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:45.201503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:45.201503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"209'12456","reported_seq":23702,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:32.802768+0000","last_change":"2024-09-16T09:32:08.439285+0000","last_active":"2024-09-16T09:32:32.802768+0000","last_peered":"2024-09-16T09:32:32.802768+0000","last_clean":"2024-09-16T09:32:32.802768+0000","last_became_active":"2024-09-16T09:32:08.422751+0000","last_became_peered":"2024-09-16T09:32:08.422751+0000","last_unstale":"2024-09-16T09:32:32.802768+0000","last_undegraded":"2024-09-16T09:32:32.802768+0000","last_fullsized":"2024-09-16T09:32:32.802768+0000","mapping_epoch":204,"log_start":"195'8200","ondisk_log_start":"195'8200","created":152,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":4256,"log_dups_size":0,"ondisk_log_size":4256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":201719808,"num_objects":3078,"num_object_clones":0,"num_object_copies":9234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3078,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12456,"num_write_kb":497088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5551,"num_bytes_recovered":363790336,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.2","version":"73'79","reported_seq":410,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.322654+0000","last_change":"2024-09-16T09:32:13.450912+0000","last_active":"2024-09-16T09:32:29.322654+0000","last_peered":"2024-09-16T09:32:29.322654+0000","last_clean":"2024-09-16T09:32:29.322654+0000","last_became_active":"2024-09-16T09:30:08.765943+0000","last_became_peered":"2024-09-16T09:30:08.765943+0000","last_unstale":"2024-09-16T09:32:29.322654+0000","last_undegraded":"2024-09-16T09:32:29.322654+0000","last_fullsized":"2024-09-16T09:32:29.322654+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:26:35.745837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:26:35.745837+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:32:42.371213+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"66'13","reported_seq":354,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.318118+0000","last_change":"2024-09-16T09:27:44.156057+0000","last_active":"2024-09-16T09:32:29.318118+0000","last_peered":"2024-09-16T09:32:29.318118+0000","last_clean":"2024-09-16T09:32:29.318118+0000","last_became_active":"2024-09-16T09:25:03.713241+0000","last_became_peered":"2024-09-16T09:25:03.713241+0000","last_unstale":"2024-09-16T09:32:29.318118+0000","last_undegraded":"2024-09-16T09:32:29.318118+0000","last_fullsized":"2024-09-16T09:32:29.318118+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'13","last_scrub_stamp":"2024-09-16T09:27:44.155969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:44.155969+0000","objects_scrubbed":5,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":29,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":331,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.319435+0000","last_change":"2024-09-16T09:29:50.827103+0000","last_active":"2024-09-16T09:32:29.319435+0000","last_peered":"2024-09-16T09:32:29.319435+0000","last_clean":"2024-09-16T09:32:29.319435+0000","last_became_active":"2024-09-16T09:25:01.706828+0000","last_became_peered":"2024-09-16T09:25:01.706828+0000","last_unstale":"2024-09-16T09:32:29.319435+0000","last_undegraded":"2024-09-16T09:32:29.319435+0000","last_fullsized":"2024-09-16T09:32:29.319435+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:50.827057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:50.827057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":362,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.325409+0000","last_change":"2024-09-16T09:31:09.281163+0000","last_active":"2024-09-16T09:32:29.325409+0000","last_peered":"2024-09-16T09:32:29.325409+0000","last_clean":"2024-09-16T09:32:29.325409+0000","last_became_active":"2024-09-16T09:31:09.266338+0000","last_became_peered":"2024-09-16T09:31:09.266338+0000","last_unstale":"2024-09-16T09:32:29.325409+0000","last_undegraded":"2024-09-16T09:32:29.325409+0000","last_fullsized":"2024-09-16T09:32:29.325409+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:29:16.104209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:29:16.104209+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0.00055022299999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":339,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:32.029895+0000","last_change":"2024-09-16T09:32:32.029895+0000","last_active":"2024-09-16T09:32:32.029895+0000","last_peered":"2024-09-16T09:32:32.029895+0000","last_clean":"2024-09-16T09:32:32.029895+0000","last_became_active":"2024-09-16T09:25:05.711539+0000","last_became_peered":"2024-09-16T09:25:05.711539+0000","last_unstale":"2024-09-16T09:32:32.029895+0000","last_undegraded":"2024-09-16T09:32:32.029895+0000","last_fullsized":"2024-09-16T09:32:32.029895+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:32.029829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:32:32.029829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:36.831098+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":318,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.323210+0000","last_change":"2024-09-16T09:29:22.968971+0000","last_active":"2024-09-16T09:32:29.323210+0000","last_peered":"2024-09-16T09:32:29.323210+0000","last_clean":"2024-09-16T09:32:29.323210+0000","last_became_active":"2024-09-16T09:25:07.723282+0000","last_became_peered":"2024-09-16T09:25:07.723282+0000","last_unstale":"2024-09-16T09:32:29.323210+0000","last_undegraded":"2024-09-16T09:32:29.323210+0000","last_fullsized":"2024-09-16T09:32:29.323210+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:22.968908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:22.968908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"209'12544","reported_seq":12670,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:32.911657+0000","last_change":"2024-09-16T09:29:49.448898+0000","last_active":"2024-09-16T09:32:32.911657+0000","last_peered":"2024-09-16T09:32:32.911657+0000","last_clean":"2024-09-16T09:32:32.911657+0000","last_became_active":"2024-09-16T09:29:49.448628+0000","last_became_peered":"2024-09-16T09:29:49.448628+0000","last_unstale":"2024-09-16T09:32:32.911657+0000","last_undegraded":"2024-09-16T09:32:32.911657+0000","last_fullsized":"2024-09-16T09:32:32.911657+0000","mapping_epoch":152,"log_start":"194'8000","ondisk_log_start":"194'8000","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":4544,"log_dups_size":0,"ondisk_log_size":4544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":202506240,"num_objects":3090,"num_object_clones":0,"num_object_copies":9270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3090,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12544,"num_write_kb":500288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"73'79","reported_seq":292,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.319891+0000","last_change":"2024-09-16T09:31:40.212968+0000","last_active":"2024-09-16T09:32:29.319891+0000","last_peered":"2024-09-16T09:32:29.319891+0000","last_clean":"2024-09-16T09:32:29.319891+0000","last_became_active":"2024-09-16T09:30:08.877110+0000","last_became_peered":"2024-09-16T09:30:08.877110+0000","last_unstale":"2024-09-16T09:32:29.319891+0000","last_undegraded":"2024-09-16T09:32:29.319891+0000","last_fullsized":"2024-09-16T09:32:29.319891+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:31:40.212889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:31:40.212889+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:45.708324+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"66'12","reported_seq":337,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.329253+0000","last_change":"2024-09-16T09:29:02.713335+0000","last_active":"2024-09-16T09:32:29.329253+0000","last_peered":"2024-09-16T09:32:29.329253+0000","last_clean":"2024-09-16T09:32:29.329253+0000","last_became_active":"2024-09-16T09:25:03.715315+0000","last_became_peered":"2024-09-16T09:25:03.715315+0000","last_unstale":"2024-09-16T09:32:29.329253+0000","last_undegraded":"2024-09-16T09:32:29.329253+0000","last_fullsized":"2024-09-16T09:32:29.329253+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-16T09:29:02.713259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:29:02.713259+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:43.220661+0000","scrub_duration":26,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"59'1","reported_seq":354,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.328773+0000","last_change":"2024-09-16T09:31:09.286309+0000","last_active":"2024-09-16T09:32:29.328773+0000","last_peered":"2024-09-16T09:32:29.328773+0000","last_clean":"2024-09-16T09:32:29.328773+0000","last_became_active":"2024-09-16T09:31:09.269627+0000","last_became_peered":"2024-09-16T09:31:09.269627+0000","last_unstale":"2024-09-16T09:32:29.328773+0000","last_undegraded":"2024-09-16T09:32:29.328773+0000","last_fullsized":"2024-09-16T09:32:29.328773+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:27:00.666673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:27:00.666673+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:21.192318+0000","scrub_duration":28,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1986,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"67'5","reported_seq":791,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:36.166190+0000","last_change":"2024-09-16T09:29:19.994791+0000","last_active":"2024-09-16T09:32:36.166190+0000","last_peered":"2024-09-16T09:32:36.166190+0000","last_clean":"2024-09-16T09:32:36.166190+0000","last_became_active":"2024-09-16T09:25:00.696546+0000","last_became_peered":"2024-09-16T09:25:00.696546+0000","last_unstale":"2024-09-16T09:32:36.166190+0000","last_undegraded":"2024-09-16T09:32:36.166190+0000","last_fullsized":"2024-09-16T09:32:36.166190+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'5","last_scrub_stamp":"2024-09-16T09:29:19.994716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:29:19.994716+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0.0059770420000000001,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":450,"num_read_kb":445,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":314,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.321340+0000","last_change":"2024-09-16T09:27:07.337170+0000","last_active":"2024-09-16T09:32:29.321340+0000","last_peered":"2024-09-16T09:32:29.321340+0000","last_clean":"2024-09-16T09:32:29.321340+0000","last_became_active":"2024-09-16T09:25:05.714704+0000","last_became_peered":"2024-09-16T09:25:05.714704+0000","last_unstale":"2024-09-16T09:32:29.321340+0000","last_undegraded":"2024-09-16T09:32:29.321340+0000","last_fullsized":"2024-09-16T09:32:29.321340+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:07.337124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:07.337124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":325,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.316982+0000","last_change":"2024-09-16T09:31:09.271453+0000","last_active":"2024-09-16T09:32:29.316982+0000","last_peered":"2024-09-16T09:32:29.316982+0000","last_clean":"2024-09-16T09:32:29.316982+0000","last_became_active":"2024-09-16T09:31:09.271111+0000","last_became_peered":"2024-09-16T09:31:09.271111+0000","last_unstale":"2024-09-16T09:32:29.316982+0000","last_undegraded":"2024-09-16T09:32:29.316982+0000","last_fullsized":"2024-09-16T09:32:29.316982+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:24.231167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:24.231167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"209'12772","reported_seq":12963,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:37.615809+0000","last_change":"2024-09-16T09:30:56.023710+0000","last_active":"2024-09-16T09:32:37.615809+0000","last_peered":"2024-09-16T09:32:37.615809+0000","last_clean":"2024-09-16T09:32:37.615809+0000","last_became_active":"2024-09-16T09:29:49.450643+0000","last_became_peered":"2024-09-16T09:29:49.450643+0000","last_unstale":"2024-09-16T09:32:37.615809+0000","last_undegraded":"2024-09-16T09:32:37.615809+0000","last_fullsized":"2024-09-16T09:32:37.615809+0000","mapping_epoch":152,"log_start":"192'7500","ondisk_log_start":"192'7500","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"181'5780","last_scrub_stamp":"2024-09-16T09:30:56.023622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:30:56.023622+0000","objects_scrubbed":5716,"log_size":5272,"log_dups_size":0,"ondisk_log_size":5272,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:14.532550+0000","scrub_duration":1526,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":182976512,"num_objects":2792,"num_object_clones":0,"num_object_copies":8376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2792,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12772,"num_write_kb":498048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"66'16","reported_seq":449,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:31.850099+0000","last_change":"2024-09-16T09:32:01.564577+0000","last_active":"2024-09-16T09:32:31.850099+0000","last_peered":"2024-09-16T09:32:31.850099+0000","last_clean":"2024-09-16T09:32:31.850099+0000","last_became_active":"2024-09-16T09:25:03.714118+0000","last_became_peered":"2024-09-16T09:25:03.714118+0000","last_unstale":"2024-09-16T09:32:31.850099+0000","last_undegraded":"2024-09-16T09:32:31.850099+0000","last_fullsized":"2024-09-16T09:32:31.850099+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'16","last_scrub_stamp":"2024-09-16T09:32:01.564489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:32:01.564489+0000","objects_scrubbed":8,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:10.625142+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":172,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.320985+0000","last_change":"2024-09-16T09:31:09.270603+0000","last_active":"2024-09-16T09:32:29.320985+0000","last_peered":"2024-09-16T09:32:29.320985+0000","last_clean":"2024-09-16T09:32:29.320985+0000","last_became_active":"2024-09-16T09:31:09.270196+0000","last_became_peered":"2024-09-16T09:31:09.270196+0000","last_unstale":"2024-09-16T09:32:29.320985+0000","last_undegraded":"2024-09-16T09:32:29.320985+0000","last_fullsized":"2024-09-16T09:32:29.320985+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:49.195331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:27:49.195331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"73'79","reported_seq":410,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.322980+0000","last_change":"2024-09-16T09:32:13.453628+0000","last_active":"2024-09-16T09:32:29.322980+0000","last_peered":"2024-09-16T09:32:29.322980+0000","last_clean":"2024-09-16T09:32:29.322980+0000","last_became_active":"2024-09-16T09:30:08.870394+0000","last_became_peered":"2024-09-16T09:30:08.870394+0000","last_unstale":"2024-09-16T09:32:29.322980+0000","last_undegraded":"2024-09-16T09:32:29.322980+0000","last_fullsized":"2024-09-16T09:32:29.322980+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:02.723794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:02.723794+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:32:43.420510+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":314,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.322997+0000","last_change":"2024-09-16T09:26:39.699133+0000","last_active":"2024-09-16T09:32:29.322997+0000","last_peered":"2024-09-16T09:32:29.322997+0000","last_clean":"2024-09-16T09:32:29.322997+0000","last_became_active":"2024-09-16T09:25:05.714341+0000","last_became_peered":"2024-09-16T09:25:05.714341+0000","last_unstale":"2024-09-16T09:32:29.322997+0000","last_undegraded":"2024-09-16T09:32:29.322997+0000","last_fullsized":"2024-09-16T09:32:29.322997+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:39.699059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:26:39.699059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":318,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.320639+0000","last_change":"2024-09-16T09:29:00.243528+0000","last_active":"2024-09-16T09:32:29.320639+0000","last_peered":"2024-09-16T09:32:29.320639+0000","last_clean":"2024-09-16T09:32:29.320639+0000","last_became_active":"2024-09-16T09:25:07.714281+0000","last_became_peered":"2024-09-16T09:25:07.714281+0000","last_unstale":"2024-09-16T09:32:29.320639+0000","last_undegraded":"2024-09-16T09:32:29.320639+0000","last_fullsized":"2024-09-16T09:32:29.320639+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:00.243482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:00.243482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.b","version":"73'79","reported_seq":296,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.319448+0000","last_change":"2024-09-16T09:32:16.500592+0000","last_active":"2024-09-16T09:32:29.319448+0000","last_peered":"2024-09-16T09:32:29.319448+0000","last_clean":"2024-09-16T09:32:29.319448+0000","last_became_active":"2024-09-16T09:30:10.784688+0000","last_became_peered":"2024-09-16T09:30:10.784688+0000","last_unstale":"2024-09-16T09:32:29.319448+0000","last_undegraded":"2024-09-16T09:32:29.319448+0000","last_fullsized":"2024-09-16T09:32:29.319448+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:30:03.887571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:30:03.887571+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:32:44.156390+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,5],"acting":[1,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2","version":"209'12297","reported_seq":12421,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:32.799480+0000","last_change":"2024-09-16T09:29:49.445324+0000","last_active":"2024-09-16T09:32:32.799480+0000","last_peered":"2024-09-16T09:32:32.799480+0000","last_clean":"2024-09-16T09:32:32.799480+0000","last_became_active":"2024-09-16T09:29:49.445022+0000","last_became_peered":"2024-09-16T09:29:49.445022+0000","last_unstale":"2024-09-16T09:32:32.799480+0000","last_undegraded":"2024-09-16T09:32:32.799480+0000","last_fullsized":"2024-09-16T09:32:32.799480+0000","mapping_epoch":152,"log_start":"195'8000","ondisk_log_start":"195'8000","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":4297,"log_dups_size":0,"ondisk_log_size":4297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":192610304,"num_objects":2939,"num_object_clones":0,"num_object_copies":8817,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2939,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12297,"num_write_kb":487552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"66'11","reported_seq":357,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.316219+0000","last_change":"2024-09-16T09:30:05.646701+0000","last_active":"2024-09-16T09:32:29.316219+0000","last_peered":"2024-09-16T09:32:29.316219+0000","last_clean":"2024-09-16T09:32:29.316219+0000","last_became_active":"2024-09-16T09:26:33.287399+0000","last_became_peered":"2024-09-16T09:26:33.287399+0000","last_unstale":"2024-09-16T09:32:29.316219+0000","last_undegraded":"2024-09-16T09:32:29.316219+0000","last_fullsized":"2024-09-16T09:32:29.316219+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-16T09:30:05.646500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:30:05.646500+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":315,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.314983+0000","last_change":"2024-09-16T09:25:01.708568+0000","last_active":"2024-09-16T09:32:29.314983+0000","last_peered":"2024-09-16T09:32:29.314983+0000","last_clean":"2024-09-16T09:32:29.314983+0000","last_became_active":"2024-09-16T09:25:01.708270+0000","last_became_peered":"2024-09-16T09:25:01.708270+0000","last_unstale":"2024-09-16T09:32:29.314983+0000","last_undegraded":"2024-09-16T09:32:29.314983+0000","last_fullsized":"2024-09-16T09:32:29.314983+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:10.865036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":314,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.320420+0000","last_change":"2024-09-16T09:26:34.171190+0000","last_active":"2024-09-16T09:32:29.320420+0000","last_peered":"2024-09-16T09:32:29.320420+0000","last_clean":"2024-09-16T09:32:29.320420+0000","last_became_active":"2024-09-16T09:25:05.711964+0000","last_became_peered":"2024-09-16T09:25:05.711964+0000","last_unstale":"2024-09-16T09:32:29.320420+0000","last_undegraded":"2024-09-16T09:32:29.320420+0000","last_fullsized":"2024-09-16T09:32:29.320420+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:34.171139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:26:34.171139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":197,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.320826+0000","last_change":"2024-09-16T09:32:00.526741+0000","last_active":"2024-09-16T09:32:29.320826+0000","last_peered":"2024-09-16T09:32:29.320826+0000","last_clean":"2024-09-16T09:32:29.320826+0000","last_became_active":"2024-09-16T09:31:09.278618+0000","last_became_peered":"2024-09-16T09:31:09.278618+0000","last_unstale":"2024-09-16T09:32:29.320826+0000","last_undegraded":"2024-09-16T09:32:29.320826+0000","last_fullsized":"2024-09-16T09:32:29.320826+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:00.526674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:32:00.526674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:25.768941+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1","version":"209'12222","reported_seq":12348,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:32.939423+0000","last_change":"2024-09-16T09:29:49.449796+0000","last_active":"2024-09-16T09:32:32.939423+0000","last_peered":"2024-09-16T09:32:32.939423+0000","last_clean":"2024-09-16T09:32:32.939423+0000","last_became_active":"2024-09-16T09:29:49.449475+0000","last_became_peered":"2024-09-16T09:29:49.449475+0000","last_unstale":"2024-09-16T09:32:32.939423+0000","last_undegraded":"2024-09-16T09:32:32.939423+0000","last_fullsized":"2024-09-16T09:32:32.939423+0000","mapping_epoch":152,"log_start":"193'7700","ondisk_log_start":"193'7700","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":4522,"log_dups_size":0,"ondisk_log_size":4522,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":198967296,"num_objects":3036,"num_object_clones":0,"num_object_copies":9108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3036,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12222,"num_write_kb":488256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.8","version":"73'79","reported_seq":293,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.322492+0000","last_change":"2024-09-16T09:32:07.068820+0000","last_active":"2024-09-16T09:32:29.322492+0000","last_peered":"2024-09-16T09:32:29.322492+0000","last_clean":"2024-09-16T09:32:29.322492+0000","last_became_active":"2024-09-16T09:31:09.406638+0000","last_became_peered":"2024-09-16T09:31:09.406638+0000","last_unstale":"2024-09-16T09:32:29.322492+0000","last_undegraded":"2024-09-16T09:32:29.322492+0000","last_fullsized":"2024-09-16T09:32:29.322492+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:17.482769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:17.482769+0000","objects_scrubbed":1,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262688,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":525376,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"66'17","reported_seq":363,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.316231+0000","last_change":"2024-09-16T09:30:04.636115+0000","last_active":"2024-09-16T09:32:29.316231+0000","last_peered":"2024-09-16T09:32:29.316231+0000","last_clean":"2024-09-16T09:32:29.316231+0000","last_became_active":"2024-09-16T09:25:03.713358+0000","last_became_peered":"2024-09-16T09:25:03.713358+0000","last_unstale":"2024-09-16T09:32:29.316231+0000","last_undegraded":"2024-09-16T09:32:29.316231+0000","last_fullsized":"2024-09-16T09:32:29.316231+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'17","last_scrub_stamp":"2024-09-16T09:30:04.636039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:30:04.636039+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":323,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.320321+0000","last_change":"2024-09-16T09:26:04.104188+0000","last_active":"2024-09-16T09:32:29.320321+0000","last_peered":"2024-09-16T09:32:29.320321+0000","last_clean":"2024-09-16T09:32:29.320321+0000","last_became_active":"2024-09-16T09:25:01.703355+0000","last_became_peered":"2024-09-16T09:25:01.703355+0000","last_unstale":"2024-09-16T09:32:29.320321+0000","last_undegraded":"2024-09-16T09:32:29.320321+0000","last_fullsized":"2024-09-16T09:32:29.320321+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:04.104130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:04.104130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:48.432563+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":331,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.318554+0000","last_change":"2024-09-16T09:29:58.839132+0000","last_active":"2024-09-16T09:32:29.318554+0000","last_peered":"2024-09-16T09:32:29.318554+0000","last_clean":"2024-09-16T09:32:29.318554+0000","last_became_active":"2024-09-16T09:25:05.716900+0000","last_became_peered":"2024-09-16T09:25:05.716900+0000","last_unstale":"2024-09-16T09:32:29.318554+0000","last_undegraded":"2024-09-16T09:32:29.318554+0000","last_fullsized":"2024-09-16T09:32:29.318554+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:58.839051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:58.839051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":341,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.326163+0000","last_change":"2024-09-16T09:31:09.276869+0000","last_active":"2024-09-16T09:32:29.326163+0000","last_peered":"2024-09-16T09:32:29.326163+0000","last_clean":"2024-09-16T09:32:29.326163+0000","last_became_active":"2024-09-16T09:31:09.276275+0000","last_became_peered":"2024-09-16T09:31:09.276275+0000","last_unstale":"2024-09-16T09:32:29.326163+0000","last_undegraded":"2024-09-16T09:32:29.326163+0000","last_fullsized":"2024-09-16T09:32:29.326163+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:30:03.946025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:30:03.946025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.0","version":"209'12496","reported_seq":20308,"reported_epoch":209,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-16T09:32:32.995302+0000","last_change":"2024-09-16T09:31:09.476408+0000","last_active":"2024-09-16T09:32:32.995302+0000","last_peered":"2024-09-16T09:32:32.995302+0000","last_clean":"2024-09-16T09:30:20.486290+0000","last_became_active":"2024-09-16T09:31:09.399514+0000","last_became_peered":"2024-09-16T09:31:09.399514+0000","last_unstale":"2024-09-16T09:32:32.995302+0000","last_undegraded":"2024-09-16T09:32:32.995302+0000","last_fullsized":"2024-09-16T09:31:09.254572+0000","mapping_epoch":186,"log_start":"193'7800","ondisk_log_start":"193'7800","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":4696,"log_dups_size":0,"ondisk_log_size":4696,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":199229440,"num_objects":3040,"num_object_clones":0,"num_object_copies":9120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":864,"num_objects_unfound":0,"num_objects_dirty":3040,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12496,"num_write_kb":497152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3838,"num_bytes_recovered":251527168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,1],"avail_no_missing":["5","1"],"object_location_counts":[{"shards":"1,4,5","objects":864},{"shards":"1,5","objects":2176}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.9","version":"73'79","reported_seq":394,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.323775+0000","last_change":"2024-09-16T09:29:19.013948+0000","last_active":"2024-09-16T09:32:29.323775+0000","last_peered":"2024-09-16T09:32:29.323775+0000","last_clean":"2024-09-16T09:32:29.323775+0000","last_became_active":"2024-09-16T09:26:35.126066+0000","last_became_peered":"2024-09-16T09:26:35.126066+0000","last_unstale":"2024-09-16T09:32:29.323775+0000","last_undegraded":"2024-09-16T09:32:29.323775+0000","last_fullsized":"2024-09-16T09:32:29.323775+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:19.013865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:19.013865+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"66'10","reported_seq":326,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.315945+0000","last_change":"2024-09-16T09:25:03.708144+0000","last_active":"2024-09-16T09:32:29.315945+0000","last_peered":"2024-09-16T09:32:29.315945+0000","last_clean":"2024-09-16T09:32:29.315945+0000","last_became_active":"2024-09-16T09:25:03.707723+0000","last_became_peered":"2024-09-16T09:25:03.707723+0000","last_unstale":"2024-09-16T09:32:29.315945+0000","last_undegraded":"2024-09-16T09:32:29.315945+0000","last_fullsized":"2024-09-16T09:32:29.315945+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:13.930015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":188,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.327260+0000","last_change":"2024-09-16T09:31:32.253397+0000","last_active":"2024-09-16T09:32:29.327260+0000","last_peered":"2024-09-16T09:32:29.327260+0000","last_clean":"2024-09-16T09:32:29.327260+0000","last_became_active":"2024-09-16T09:31:09.288825+0000","last_became_peered":"2024-09-16T09:31:09.288825+0000","last_unstale":"2024-09-16T09:32:29.327260+0000","last_undegraded":"2024-09-16T09:32:29.327260+0000","last_fullsized":"2024-09-16T09:32:29.327260+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:31:32.253306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:31:32.253306+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:45.682494+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":433,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.331791+0000","last_change":"2024-09-16T09:29:29.866520+0000","last_active":"2024-09-16T09:32:29.331791+0000","last_peered":"2024-09-16T09:32:29.331791+0000","last_clean":"2024-09-16T09:32:29.331791+0000","last_became_active":"2024-09-16T09:25:05.711590+0000","last_became_peered":"2024-09-16T09:25:05.711590+0000","last_unstale":"2024-09-16T09:32:29.331791+0000","last_undegraded":"2024-09-16T09:32:29.331791+0000","last_fullsized":"2024-09-16T09:32:29.331791+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:29:29.866460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:29.866460+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":327,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.315688+0000","last_change":"2024-09-16T09:29:02.667332+0000","last_active":"2024-09-16T09:32:29.315688+0000","last_peered":"2024-09-16T09:32:29.315688+0000","last_clean":"2024-09-16T09:32:29.315688+0000","last_became_active":"2024-09-16T09:25:07.718958+0000","last_became_peered":"2024-09-16T09:25:07.718958+0000","last_unstale":"2024-09-16T09:32:29.315688+0000","last_undegraded":"2024-09-16T09:32:29.315688+0000","last_fullsized":"2024-09-16T09:32:29.315688+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:02.667259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:02.667259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.e","version":"73'79","reported_seq":120,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.315520+0000","last_change":"2024-09-16T09:32:13.445947+0000","last_active":"2024-09-16T09:32:29.315520+0000","last_peered":"2024-09-16T09:32:29.315520+0000","last_clean":"2024-09-16T09:32:29.315520+0000","last_became_active":"2024-09-16T09:30:10.705072+0000","last_became_peered":"2024-09-16T09:30:10.705072+0000","last_unstale":"2024-09-16T09:32:29.315520+0000","last_undegraded":"2024-09-16T09:32:29.315520+0000","last_fullsized":"2024-09-16T09:32:29.315520+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:26:35.170229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:26:35.170229+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:32:42.891843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26835,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":10,"num_write":5,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":26835,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,3],"acting":[4,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.7","version":"209'12417","reported_seq":12543,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:32.932275+0000","last_change":"2024-09-16T09:29:49.839740+0000","last_active":"2024-09-16T09:32:32.932275+0000","last_peered":"2024-09-16T09:32:32.932275+0000","last_clean":"2024-09-16T09:32:32.932275+0000","last_became_active":"2024-09-16T09:29:49.839437+0000","last_became_peered":"2024-09-16T09:29:49.839437+0000","last_unstale":"2024-09-16T09:32:32.932275+0000","last_undegraded":"2024-09-16T09:32:32.932275+0000","last_fullsized":"2024-09-16T09:32:32.932275+0000","mapping_epoch":152,"log_start":"194'7900","ondisk_log_start":"194'7900","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":4517,"log_dups_size":0,"ondisk_log_size":4517,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":198377472,"num_objects":3027,"num_object_clones":0,"num_object_copies":9081,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3027,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12417,"num_write_kb":494208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"66'9","reported_seq":351,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.327009+0000","last_change":"2024-09-16T09:31:31.211495+0000","last_active":"2024-09-16T09:32:29.327009+0000","last_peered":"2024-09-16T09:32:29.327009+0000","last_clean":"2024-09-16T09:32:29.327009+0000","last_became_active":"2024-09-16T09:25:03.712230+0000","last_became_peered":"2024-09-16T09:25:03.712230+0000","last_unstale":"2024-09-16T09:32:29.327009+0000","last_undegraded":"2024-09-16T09:32:29.327009+0000","last_fullsized":"2024-09-16T09:32:29.327009+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:31:31.211420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:31:31.211420+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:36.499658+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":336,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.321055+0000","last_change":"2024-09-16T09:31:09.277413+0000","last_active":"2024-09-16T09:32:29.321055+0000","last_peered":"2024-09-16T09:32:29.321055+0000","last_clean":"2024-09-16T09:32:29.321055+0000","last_became_active":"2024-09-16T09:31:09.276516+0000","last_became_peered":"2024-09-16T09:31:09.276516+0000","last_unstale":"2024-09-16T09:32:29.321055+0000","last_undegraded":"2024-09-16T09:32:29.321055+0000","last_fullsized":"2024-09-16T09:32:29.321055+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:26.061106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:26.061106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":337,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.330110+0000","last_change":"2024-09-16T09:31:09.275219+0000","last_active":"2024-09-16T09:32:29.330110+0000","last_peered":"2024-09-16T09:32:29.330110+0000","last_clean":"2024-09-16T09:32:29.330110+0000","last_became_active":"2024-09-16T09:31:09.274702+0000","last_became_peered":"2024-09-16T09:31:09.274702+0000","last_unstale":"2024-09-16T09:32:29.330110+0000","last_undegraded":"2024-09-16T09:32:29.330110+0000","last_fullsized":"2024-09-16T09:32:29.330110+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:57.220068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:28:57.220068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:26.950321+0000","scrub_duration":33,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":327,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.328797+0000","last_change":"2024-09-16T09:31:27.185957+0000","last_active":"2024-09-16T09:32:29.328797+0000","last_peered":"2024-09-16T09:32:29.328797+0000","last_clean":"2024-09-16T09:32:29.328797+0000","last_became_active":"2024-09-16T09:25:07.718778+0000","last_became_peered":"2024-09-16T09:25:07.718778+0000","last_unstale":"2024-09-16T09:32:29.328797+0000","last_undegraded":"2024-09-16T09:32:29.328797+0000","last_fullsized":"2024-09-16T09:32:29.328797+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:31:27.185801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:31:27.185801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:39.017009+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.f","version":"73'79","reported_seq":372,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.321231+0000","last_change":"2024-09-16T09:32:16.492685+0000","last_active":"2024-09-16T09:32:29.321231+0000","last_peered":"2024-09-16T09:32:29.321231+0000","last_clean":"2024-09-16T09:32:29.321231+0000","last_became_active":"2024-09-16T09:30:10.693477+0000","last_became_peered":"2024-09-16T09:30:10.693477+0000","last_unstale":"2024-09-16T09:32:29.321231+0000","last_undegraded":"2024-09-16T09:32:29.321231+0000","last_fullsized":"2024-09-16T09:32:29.321231+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:17.534429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:17.534429+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:32:43.568934+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.6","version":"209'12680","reported_seq":12806,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:33.163782+0000","last_change":"2024-09-16T09:29:50.008888+0000","last_active":"2024-09-16T09:32:33.163782+0000","last_peered":"2024-09-16T09:32:33.163782+0000","last_clean":"2024-09-16T09:32:33.163782+0000","last_became_active":"2024-09-16T09:29:50.008656+0000","last_became_peered":"2024-09-16T09:29:50.008656+0000","last_unstale":"2024-09-16T09:32:33.163782+0000","last_undegraded":"2024-09-16T09:32:33.163782+0000","last_fullsized":"2024-09-16T09:32:33.163782+0000","mapping_epoch":152,"log_start":"191'7500","ondisk_log_start":"191'7500","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":5180,"log_dups_size":0,"ondisk_log_size":5180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":205914112,"num_objects":3142,"num_object_clones":0,"num_object_copies":9426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3142,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12680,"num_write_kb":506304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"66'19","reported_seq":358,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.320106+0000","last_change":"2024-09-16T09:29:02.554357+0000","last_active":"2024-09-16T09:32:29.320106+0000","last_peered":"2024-09-16T09:32:29.320106+0000","last_clean":"2024-09-16T09:32:29.320106+0000","last_became_active":"2024-09-16T09:25:03.712783+0000","last_became_peered":"2024-09-16T09:25:03.712783+0000","last_unstale":"2024-09-16T09:32:29.320106+0000","last_undegraded":"2024-09-16T09:32:29.320106+0000","last_fullsized":"2024-09-16T09:32:29.320106+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-16T09:29:02.554274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:29:02.554274+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:21.554123+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"59'1","reported_seq":336,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.323467+0000","last_change":"2024-09-16T09:29:51.025505+0000","last_active":"2024-09-16T09:32:29.323467+0000","last_peered":"2024-09-16T09:32:29.323467+0000","last_clean":"2024-09-16T09:32:29.323467+0000","last_became_active":"2024-09-16T09:25:01.708069+0000","last_became_peered":"2024-09-16T09:25:01.708069+0000","last_unstale":"2024-09-16T09:32:29.323467+0000","last_undegraded":"2024-09-16T09:32:29.323467+0000","last_fullsized":"2024-09-16T09:32:29.323467+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:29:51.025444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:51.025444+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":315,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.325346+0000","last_change":"2024-09-16T09:27:11.067059+0000","last_active":"2024-09-16T09:32:29.325346+0000","last_peered":"2024-09-16T09:32:29.325346+0000","last_clean":"2024-09-16T09:32:29.325346+0000","last_became_active":"2024-09-16T09:25:05.709264+0000","last_became_peered":"2024-09-16T09:25:05.709264+0000","last_unstale":"2024-09-16T09:32:29.325346+0000","last_undegraded":"2024-09-16T09:32:29.325346+0000","last_fullsized":"2024-09-16T09:32:29.325346+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:11.066986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:11.066986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:18.935169+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":310,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.322891+0000","last_change":"2024-09-16T09:27:25.865107+0000","last_active":"2024-09-16T09:32:29.322891+0000","last_peered":"2024-09-16T09:32:29.322891+0000","last_clean":"2024-09-16T09:32:29.322891+0000","last_became_active":"2024-09-16T09:25:07.718750+0000","last_became_peered":"2024-09-16T09:25:07.718750+0000","last_unstale":"2024-09-16T09:32:29.322891+0000","last_undegraded":"2024-09-16T09:32:29.322891+0000","last_fullsized":"2024-09-16T09:32:29.322891+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:25.865030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:25.865030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.c","version":"73'79","reported_seq":60,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.326849+0000","last_change":"2024-09-16T09:31:09.290919+0000","last_active":"2024-09-16T09:32:29.326849+0000","last_peered":"2024-09-16T09:32:29.326849+0000","last_clean":"2024-09-16T09:32:29.326849+0000","last_became_active":"2024-09-16T09:31:09.273770+0000","last_became_peered":"2024-09-16T09:31:09.273770+0000","last_unstale":"2024-09-16T09:32:29.326849+0000","last_undegraded":"2024-09-16T09:32:29.326849+0000","last_fullsized":"2024-09-16T09:32:29.326849+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:30:22.614462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:30:22.614462+0000","objects_scrubbed":1,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":327680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.5","version":"209'13016","reported_seq":13215,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:37.609113+0000","last_change":"2024-09-16T09:31:06.121217+0000","last_active":"2024-09-16T09:32:37.609113+0000","last_peered":"2024-09-16T09:32:37.609113+0000","last_clean":"2024-09-16T09:32:37.609113+0000","last_became_active":"2024-09-16T09:29:50.009013+0000","last_became_peered":"2024-09-16T09:29:50.009013+0000","last_unstale":"2024-09-16T09:32:37.609113+0000","last_undegraded":"2024-09-16T09:32:37.609113+0000","last_fullsized":"2024-09-16T09:32:37.609113+0000","mapping_epoch":152,"log_start":"192'7713","ondisk_log_start":"192'7713","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"184'6587","last_scrub_stamp":"2024-09-16T09:31:06.120990+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:31:06.120990+0000","objects_scrubbed":6523,"log_size":5303,"log_dups_size":0,"ondisk_log_size":5303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:25.588988+0000","scrub_duration":1690,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":178782208,"num_objects":2728,"num_object_clones":0,"num_object_copies":8184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2728,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13016,"num_write_kb":503808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"66'12","reported_seq":388,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.314839+0000","last_change":"2024-09-16T09:31:09.296915+0000","last_active":"2024-09-16T09:32:29.314839+0000","last_peered":"2024-09-16T09:32:29.314839+0000","last_clean":"2024-09-16T09:32:29.314839+0000","last_became_active":"2024-09-16T09:31:09.269047+0000","last_became_peered":"2024-09-16T09:31:09.269047+0000","last_unstale":"2024-09-16T09:32:29.314839+0000","last_undegraded":"2024-09-16T09:32:29.314839+0000","last_fullsized":"2024-09-16T09:32:29.314839+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-16T09:30:21.728668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:30:21.728668+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":330,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.324585+0000","last_change":"2024-09-16T09:29:53.075288+0000","last_active":"2024-09-16T09:32:29.324585+0000","last_peered":"2024-09-16T09:32:29.324585+0000","last_clean":"2024-09-16T09:32:29.324585+0000","last_became_active":"2024-09-16T09:25:01.707528+0000","last_became_peered":"2024-09-16T09:25:01.707528+0000","last_unstale":"2024-09-16T09:32:29.324585+0000","last_undegraded":"2024-09-16T09:32:29.324585+0000","last_fullsized":"2024-09-16T09:32:29.324585+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:53.075232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:53.075232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":323,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.325112+0000","last_change":"2024-09-16T09:27:49.965628+0000","last_active":"2024-09-16T09:32:29.325112+0000","last_peered":"2024-09-16T09:32:29.325112+0000","last_clean":"2024-09-16T09:32:29.325112+0000","last_became_active":"2024-09-16T09:25:05.715253+0000","last_became_peered":"2024-09-16T09:25:05.715253+0000","last_unstale":"2024-09-16T09:32:29.325112+0000","last_undegraded":"2024-09-16T09:32:29.325112+0000","last_fullsized":"2024-09-16T09:32:29.325112+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:49.965424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:49.965424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:20.990618+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":311,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.325943+0000","last_change":"2024-09-16T09:27:56.059755+0000","last_active":"2024-09-16T09:32:29.325943+0000","last_peered":"2024-09-16T09:32:29.325943+0000","last_clean":"2024-09-16T09:32:29.325943+0000","last_became_active":"2024-09-16T09:25:07.718899+0000","last_became_peered":"2024-09-16T09:25:07.718899+0000","last_unstale":"2024-09-16T09:32:29.325943+0000","last_undegraded":"2024-09-16T09:32:29.325943+0000","last_fullsized":"2024-09-16T09:32:29.325943+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:56.059687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:56.059687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.d","version":"73'79","reported_seq":120,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.322953+0000","last_change":"2024-09-16T09:32:16.497595+0000","last_active":"2024-09-16T09:32:29.322953+0000","last_peered":"2024-09-16T09:32:29.322953+0000","last_clean":"2024-09-16T09:32:29.322953+0000","last_became_active":"2024-09-16T09:30:10.709958+0000","last_became_peered":"2024-09-16T09:30:10.709958+0000","last_unstale":"2024-09-16T09:32:29.322953+0000","last_undegraded":"2024-09-16T09:32:29.322953+0000","last_fullsized":"2024-09-16T09:32:29.322953+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:58.568841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:58.568841+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:32:44.442311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26835,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":10,"num_write":5,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":26835,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.4","version":"209'12600","reported_seq":12726,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:32.975177+0000","last_change":"2024-09-16T09:29:49.444857+0000","last_active":"2024-09-16T09:32:32.975177+0000","last_peered":"2024-09-16T09:32:32.975177+0000","last_clean":"2024-09-16T09:32:32.975177+0000","last_became_active":"2024-09-16T09:29:49.444515+0000","last_became_peered":"2024-09-16T09:29:49.444515+0000","last_unstale":"2024-09-16T09:32:32.975177+0000","last_undegraded":"2024-09-16T09:32:32.975177+0000","last_fullsized":"2024-09-16T09:32:32.975177+0000","mapping_epoch":152,"log_start":"193'7900","ondisk_log_start":"193'7900","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":4700,"log_dups_size":0,"ondisk_log_size":4700,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":197132288,"num_objects":3008,"num_object_clones":0,"num_object_copies":9024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3008,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12600,"num_write_kb":499456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"66'15","reported_seq":343,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.322582+0000","last_change":"2024-09-16T09:26:33.138828+0000","last_active":"2024-09-16T09:32:29.322582+0000","last_peered":"2024-09-16T09:32:29.322582+0000","last_clean":"2024-09-16T09:32:29.322582+0000","last_became_active":"2024-09-16T09:25:03.715733+0000","last_became_peered":"2024-09-16T09:25:03.715733+0000","last_unstale":"2024-09-16T09:32:29.322582+0000","last_undegraded":"2024-09-16T09:32:29.322582+0000","last_fullsized":"2024-09-16T09:32:29.322582+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-16T09:26:33.138603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:26:33.138603+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":334,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.323413+0000","last_change":"2024-09-16T09:26:26.705847+0000","last_active":"2024-09-16T09:32:29.323413+0000","last_peered":"2024-09-16T09:32:29.323413+0000","last_clean":"2024-09-16T09:32:29.323413+0000","last_became_active":"2024-09-16T09:25:01.707372+0000","last_became_peered":"2024-09-16T09:25:01.707372+0000","last_unstale":"2024-09-16T09:32:29.323413+0000","last_undegraded":"2024-09-16T09:32:29.323413+0000","last_fullsized":"2024-09-16T09:32:29.323413+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-16T09:26:26.705773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:26.705773+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"66'8","reported_seq":430,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:31.850132+0000","last_change":"2024-09-16T09:30:05.138362+0000","last_active":"2024-09-16T09:32:31.850132+0000","last_peered":"2024-09-16T09:32:31.850132+0000","last_clean":"2024-09-16T09:32:31.850132+0000","last_became_active":"2024-09-16T09:25:05.716189+0000","last_became_peered":"2024-09-16T09:25:05.716189+0000","last_unstale":"2024-09-16T09:32:31.850132+0000","last_undegraded":"2024-09-16T09:32:31.850132+0000","last_fullsized":"2024-09-16T09:32:31.850132+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:30:05.138288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:30:05.138288+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":310,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.320639+0000","last_change":"2024-09-16T09:27:20.219376+0000","last_active":"2024-09-16T09:32:29.320639+0000","last_peered":"2024-09-16T09:32:29.320639+0000","last_clean":"2024-09-16T09:32:29.320639+0000","last_became_active":"2024-09-16T09:25:07.717213+0000","last_became_peered":"2024-09-16T09:25:07.717213+0000","last_unstale":"2024-09-16T09:32:29.320639+0000","last_undegraded":"2024-09-16T09:32:29.320639+0000","last_fullsized":"2024-09-16T09:32:29.320639+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:20.219333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:20.219333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.12","version":"73'79","reported_seq":120,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.320871+0000","last_change":"2024-09-16T09:32:16.490336+0000","last_active":"2024-09-16T09:32:29.320871+0000","last_peered":"2024-09-16T09:32:29.320871+0000","last_clean":"2024-09-16T09:32:29.320871+0000","last_became_active":"2024-09-16T09:30:10.731413+0000","last_became_peered":"2024-09-16T09:30:10.731413+0000","last_unstale":"2024-09-16T09:32:29.320871+0000","last_undegraded":"2024-09-16T09:32:29.320871+0000","last_fullsized":"2024-09-16T09:32:29.320871+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:26:35.745837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:26:35.745837+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:32:44.557232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26835,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":10,"num_write":6,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":26835,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":323,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.319836+0000","last_change":"2024-09-16T09:26:33.139405+0000","last_active":"2024-09-16T09:32:29.319836+0000","last_peered":"2024-09-16T09:32:29.319836+0000","last_clean":"2024-09-16T09:32:29.319836+0000","last_became_active":"2024-09-16T09:25:01.703202+0000","last_became_peered":"2024-09-16T09:25:01.703202+0000","last_unstale":"2024-09-16T09:32:29.319836+0000","last_undegraded":"2024-09-16T09:32:29.319836+0000","last_fullsized":"2024-09-16T09:32:29.319836+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:33.139360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:33.139360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:10.581208+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"66'6","reported_seq":64,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.334564+0000","last_change":"2024-09-16T09:31:09.297514+0000","last_active":"2024-09-16T09:32:29.334564+0000","last_peered":"2024-09-16T09:32:29.334564+0000","last_clean":"2024-09-16T09:32:29.334564+0000","last_became_active":"2024-09-16T09:31:09.270685+0000","last_became_peered":"2024-09-16T09:31:09.270685+0000","last_unstale":"2024-09-16T09:32:29.334564+0000","last_undegraded":"2024-09-16T09:32:29.334564+0000","last_fullsized":"2024-09-16T09:32:29.334564+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'6","last_scrub_stamp":"2024-09-16T09:26:23.458794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:26:23.458794+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":8,"num_read_kb":5,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":322,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.319994+0000","last_change":"2024-09-16T09:29:31.496374+0000","last_active":"2024-09-16T09:32:29.319994+0000","last_peered":"2024-09-16T09:32:29.319994+0000","last_clean":"2024-09-16T09:32:29.319994+0000","last_became_active":"2024-09-16T09:25:05.715616+0000","last_became_peered":"2024-09-16T09:25:05.715616+0000","last_unstale":"2024-09-16T09:32:29.319994+0000","last_undegraded":"2024-09-16T09:32:29.319994+0000","last_fullsized":"2024-09-16T09:32:29.319994+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:31.496304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:31.496304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":310,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.323203+0000","last_change":"2024-09-16T09:26:28.733381+0000","last_active":"2024-09-16T09:32:29.323203+0000","last_peered":"2024-09-16T09:32:29.323203+0000","last_clean":"2024-09-16T09:32:29.323203+0000","last_became_active":"2024-09-16T09:25:07.721059+0000","last_became_peered":"2024-09-16T09:25:07.721059+0000","last_unstale":"2024-09-16T09:32:29.323203+0000","last_undegraded":"2024-09-16T09:32:29.323203+0000","last_fullsized":"2024-09-16T09:32:29.323203+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:28.733324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:28.733324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.13","version":"73'79","reported_seq":306,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.319761+0000","last_change":"2024-09-16T09:31:09.288677+0000","last_active":"2024-09-16T09:32:29.319761+0000","last_peered":"2024-09-16T09:32:29.319761+0000","last_clean":"2024-09-16T09:32:29.319761+0000","last_became_active":"2024-09-16T09:31:09.288233+0000","last_became_peered":"2024-09-16T09:31:09.288233+0000","last_unstale":"2024-09-16T09:32:29.319761+0000","last_undegraded":"2024-09-16T09:32:29.319761+0000","last_fullsized":"2024-09-16T09:32:29.319761+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:30:21.714369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:30:21.714369+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.16","version":"66'9","reported_seq":393,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.329341+0000","last_change":"2024-09-16T09:32:01.319481+0000","last_active":"2024-09-16T09:32:29.329341+0000","last_peered":"2024-09-16T09:32:29.329341+0000","last_clean":"2024-09-16T09:32:29.329341+0000","last_became_active":"2024-09-16T09:31:09.273498+0000","last_became_peered":"2024-09-16T09:31:09.273498+0000","last_unstale":"2024-09-16T09:32:29.329341+0000","last_undegraded":"2024-09-16T09:32:29.329341+0000","last_fullsized":"2024-09-16T09:32:29.329341+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:32:01.319390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:32:01.319390+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:29.275572+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":330,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.323990+0000","last_change":"2024-09-16T09:28:50.698331+0000","last_active":"2024-09-16T09:32:29.323990+0000","last_peered":"2024-09-16T09:32:29.323990+0000","last_clean":"2024-09-16T09:32:29.323990+0000","last_became_active":"2024-09-16T09:25:01.706491+0000","last_became_peered":"2024-09-16T09:25:01.706491+0000","last_unstale":"2024-09-16T09:32:29.323990+0000","last_undegraded":"2024-09-16T09:32:29.323990+0000","last_fullsized":"2024-09-16T09:32:29.323990+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:50.698288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:28:50.698288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":180,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.317920+0000","last_change":"2024-09-16T09:31:09.271558+0000","last_active":"2024-09-16T09:32:29.317920+0000","last_peered":"2024-09-16T09:32:29.317920+0000","last_clean":"2024-09-16T09:32:29.317920+0000","last_became_active":"2024-09-16T09:31:09.271223+0000","last_became_peered":"2024-09-16T09:31:09.271223+0000","last_unstale":"2024-09-16T09:32:29.317920+0000","last_undegraded":"2024-09-16T09:32:29.317920+0000","last_fullsized":"2024-09-16T09:32:29.317920+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:53.810580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:28:53.810580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":311,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.327703+0000","last_change":"2024-09-16T09:26:35.866582+0000","last_active":"2024-09-16T09:32:29.327703+0000","last_peered":"2024-09-16T09:32:29.327703+0000","last_clean":"2024-09-16T09:32:29.327703+0000","last_became_active":"2024-09-16T09:25:07.722050+0000","last_became_peered":"2024-09-16T09:25:07.722050+0000","last_unstale":"2024-09-16T09:32:29.327703+0000","last_undegraded":"2024-09-16T09:32:29.327703+0000","last_fullsized":"2024-09-16T09:32:29.327703+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:35.866512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:35.866512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:17.938610+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.10","version":"73'79","reported_seq":120,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.319054+0000","last_change":"2024-09-16T09:32:16.491732+0000","last_active":"2024-09-16T09:32:29.319054+0000","last_peered":"2024-09-16T09:32:29.319054+0000","last_clean":"2024-09-16T09:32:29.319054+0000","last_became_active":"2024-09-16T09:30:10.692770+0000","last_became_peered":"2024-09-16T09:30:10.692770+0000","last_unstale":"2024-09-16T09:32:29.319054+0000","last_undegraded":"2024-09-16T09:32:29.319054+0000","last_fullsized":"2024-09-16T09:32:29.319054+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:02.723794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:02.723794+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:32:43.525182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26835,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":10,"num_write":5,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":26835,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"66'9","reported_seq":342,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.321444+0000","last_change":"2024-09-16T09:29:32.485165+0000","last_active":"2024-09-16T09:32:29.321444+0000","last_peered":"2024-09-16T09:32:29.321444+0000","last_clean":"2024-09-16T09:32:29.321444+0000","last_became_active":"2024-09-16T09:25:03.717310+0000","last_became_peered":"2024-09-16T09:25:03.717310+0000","last_unstale":"2024-09-16T09:32:29.321444+0000","last_undegraded":"2024-09-16T09:32:29.321444+0000","last_fullsized":"2024-09-16T09:32:29.321444+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:29:32.485084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:29:32.485084+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":353,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.328220+0000","last_change":"2024-09-16T09:31:59.376923+0000","last_active":"2024-09-16T09:32:29.328220+0000","last_peered":"2024-09-16T09:32:29.328220+0000","last_clean":"2024-09-16T09:32:29.328220+0000","last_became_active":"2024-09-16T09:31:09.272848+0000","last_became_peered":"2024-09-16T09:31:09.272848+0000","last_unstale":"2024-09-16T09:32:29.328220+0000","last_undegraded":"2024-09-16T09:32:29.328220+0000","last_fullsized":"2024-09-16T09:32:29.328220+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:31:59.376749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:31:59.376749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:13.111415+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"187'11","reported_seq":259,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:31.850128+0000","last_change":"2024-09-16T09:31:09.311243+0000","last_active":"2024-09-16T09:32:31.850128+0000","last_peered":"2024-09-16T09:32:31.850128+0000","last_clean":"2024-09-16T09:32:31.850128+0000","last_became_active":"2024-09-16T09:31:09.277688+0000","last_became_peered":"2024-09-16T09:31:09.277688+0000","last_unstale":"2024-09-16T09:32:31.850128+0000","last_undegraded":"2024-09-16T09:32:31.850128+0000","last_fullsized":"2024-09-16T09:32:31.850128+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:28:53.714413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:28:53.714413+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":327,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.315848+0000","last_change":"2024-09-16T09:29:49.474440+0000","last_active":"2024-09-16T09:32:29.315848+0000","last_peered":"2024-09-16T09:32:29.315848+0000","last_clean":"2024-09-16T09:32:29.315848+0000","last_became_active":"2024-09-16T09:25:07.718810+0000","last_became_peered":"2024-09-16T09:25:07.718810+0000","last_unstale":"2024-09-16T09:32:29.315848+0000","last_undegraded":"2024-09-16T09:32:29.315848+0000","last_fullsized":"2024-09-16T09:32:29.315848+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:49.474391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:49.474391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.11","version":"73'79","reported_seq":206,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.328949+0000","last_change":"2024-09-16T09:31:09.272938+0000","last_active":"2024-09-16T09:32:29.328949+0000","last_peered":"2024-09-16T09:32:29.328949+0000","last_clean":"2024-09-16T09:32:29.328949+0000","last_became_active":"2024-09-16T09:31:09.272208+0000","last_became_peered":"2024-09-16T09:31:09.272208+0000","last_unstale":"2024-09-16T09:32:29.328949+0000","last_undegraded":"2024-09-16T09:32:29.328949+0000","last_fullsized":"2024-09-16T09:32:29.328949+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:30:21.651575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:30:21.651575+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.14","version":"66'10","reported_seq":201,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.318674+0000","last_change":"2024-09-16T09:28:56.825675+0000","last_active":"2024-09-16T09:32:29.318674+0000","last_peered":"2024-09-16T09:32:29.318674+0000","last_clean":"2024-09-16T09:32:29.318674+0000","last_became_active":"2024-09-16T09:28:52.979059+0000","last_became_peered":"2024-09-16T09:28:52.979059+0000","last_unstale":"2024-09-16T09:32:29.318674+0000","last_undegraded":"2024-09-16T09:32:29.318674+0000","last_fullsized":"2024-09-16T09:32:29.318674+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-16T09:28:56.825476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:56.825476+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":14,"num_read_kb":9,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":330,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.323322+0000","last_change":"2024-09-16T09:29:56.021413+0000","last_active":"2024-09-16T09:32:29.323322+0000","last_peered":"2024-09-16T09:32:29.323322+0000","last_clean":"2024-09-16T09:32:29.323322+0000","last_became_active":"2024-09-16T09:25:01.707513+0000","last_became_peered":"2024-09-16T09:25:01.707513+0000","last_unstale":"2024-09-16T09:32:29.323322+0000","last_undegraded":"2024-09-16T09:32:29.323322+0000","last_fullsized":"2024-09-16T09:32:29.323322+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:56.021340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:56.021340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"66'8","reported_seq":429,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.321280+0000","last_change":"2024-09-16T09:29:48.485781+0000","last_active":"2024-09-16T09:32:29.321280+0000","last_peered":"2024-09-16T09:32:29.321280+0000","last_clean":"2024-09-16T09:32:29.321280+0000","last_became_active":"2024-09-16T09:25:05.713833+0000","last_became_peered":"2024-09-16T09:25:05.713833+0000","last_unstale":"2024-09-16T09:32:29.321280+0000","last_undegraded":"2024-09-16T09:32:29.321280+0000","last_fullsized":"2024-09-16T09:32:29.321280+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:29:48.485727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.485727+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":341,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.328152+0000","last_change":"2024-09-16T09:32:00.343429+0000","last_active":"2024-09-16T09:32:29.328152+0000","last_peered":"2024-09-16T09:32:29.328152+0000","last_clean":"2024-09-16T09:32:29.328152+0000","last_became_active":"2024-09-16T09:31:09.275168+0000","last_became_peered":"2024-09-16T09:31:09.275168+0000","last_unstale":"2024-09-16T09:32:29.328152+0000","last_undegraded":"2024-09-16T09:32:29.328152+0000","last_fullsized":"2024-09-16T09:32:29.328152+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:00.343356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:32:00.343356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:23.568990+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"66'11","reported_seq":349,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.315881+0000","last_change":"2024-09-16T09:26:33.302713+0000","last_active":"2024-09-16T09:32:29.315881+0000","last_peered":"2024-09-16T09:32:29.315881+0000","last_clean":"2024-09-16T09:32:29.315881+0000","last_became_active":"2024-09-16T09:26:33.291265+0000","last_became_peered":"2024-09-16T09:26:33.291265+0000","last_unstale":"2024-09-16T09:32:29.315881+0000","last_undegraded":"2024-09-16T09:32:29.315881+0000","last_fullsized":"2024-09-16T09:32:29.315881+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:08.864166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":339,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.316724+0000","last_change":"2024-09-16T09:29:51.517112+0000","last_active":"2024-09-16T09:32:29.316724+0000","last_peered":"2024-09-16T09:32:29.316724+0000","last_clean":"2024-09-16T09:32:29.316724+0000","last_became_active":"2024-09-16T09:25:01.705158+0000","last_became_peered":"2024-09-16T09:25:01.705158+0000","last_unstale":"2024-09-16T09:32:29.316724+0000","last_undegraded":"2024-09-16T09:32:29.316724+0000","last_fullsized":"2024-09-16T09:32:29.316724+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:51.517054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:51.517054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":345,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.317124+0000","last_change":"2024-09-16T09:31:09.266820+0000","last_active":"2024-09-16T09:32:29.317124+0000","last_peered":"2024-09-16T09:32:29.317124+0000","last_clean":"2024-09-16T09:32:29.317124+0000","last_became_active":"2024-09-16T09:31:09.266326+0000","last_became_peered":"2024-09-16T09:31:09.266326+0000","last_unstale":"2024-09-16T09:32:29.317124+0000","last_undegraded":"2024-09-16T09:32:29.317124+0000","last_fullsized":"2024-09-16T09:32:29.317124+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:15.910824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:15.910824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":173,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.328531+0000","last_change":"2024-09-16T09:31:09.277239+0000","last_active":"2024-09-16T09:32:29.328531+0000","last_peered":"2024-09-16T09:32:29.328531+0000","last_clean":"2024-09-16T09:32:29.328531+0000","last_became_active":"2024-09-16T09:31:09.276042+0000","last_became_peered":"2024-09-16T09:31:09.276042+0000","last_unstale":"2024-09-16T09:32:29.328531+0000","last_undegraded":"2024-09-16T09:32:29.328531+0000","last_fullsized":"2024-09-16T09:32:29.328531+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:48.147759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:48.147759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:29.306846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"66'9","reported_seq":335,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.317422+0000","last_change":"2024-09-16T09:26:31.216495+0000","last_active":"2024-09-16T09:32:29.317422+0000","last_peered":"2024-09-16T09:32:29.317422+0000","last_clean":"2024-09-16T09:32:29.317422+0000","last_became_active":"2024-09-16T09:25:03.705807+0000","last_became_peered":"2024-09-16T09:25:03.705807+0000","last_unstale":"2024-09-16T09:32:29.317422+0000","last_undegraded":"2024-09-16T09:32:29.317422+0000","last_fullsized":"2024-09-16T09:32:29.317422+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:26:31.216298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:26:31.216298+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":322,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.323373+0000","last_change":"2024-09-16T09:26:23.796308+0000","last_active":"2024-09-16T09:32:29.323373+0000","last_peered":"2024-09-16T09:32:29.323373+0000","last_clean":"2024-09-16T09:32:29.323373+0000","last_became_active":"2024-09-16T09:25:01.706762+0000","last_became_peered":"2024-09-16T09:25:01.706762+0000","last_unstale":"2024-09-16T09:32:29.323373+0000","last_undegraded":"2024-09-16T09:32:29.323373+0000","last_fullsized":"2024-09-16T09:32:29.323373+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:23.796231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:23.796231+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":158,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.326790+0000","last_change":"2024-09-16T09:28:52.983144+0000","last_active":"2024-09-16T09:32:29.326790+0000","last_peered":"2024-09-16T09:32:29.326790+0000","last_clean":"2024-09-16T09:32:29.326790+0000","last_became_active":"2024-09-16T09:28:52.982865+0000","last_became_peered":"2024-09-16T09:28:52.982865+0000","last_unstale":"2024-09-16T09:32:29.326790+0000","last_undegraded":"2024-09-16T09:32:29.326790+0000","last_fullsized":"2024-09-16T09:32:29.326790+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:44.241942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:44.241942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:28.290345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":319,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.324601+0000","last_change":"2024-09-16T09:28:52.552567+0000","last_active":"2024-09-16T09:32:29.324601+0000","last_peered":"2024-09-16T09:32:29.324601+0000","last_clean":"2024-09-16T09:32:29.324601+0000","last_became_active":"2024-09-16T09:25:07.715301+0000","last_became_peered":"2024-09-16T09:25:07.715301+0000","last_unstale":"2024-09-16T09:32:29.324601+0000","last_undegraded":"2024-09-16T09:32:29.324601+0000","last_fullsized":"2024-09-16T09:32:29.324601+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:52.552495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:28:52.552495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:45.269179+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.14","version":"73'79","reported_seq":120,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.327654+0000","last_change":"2024-09-16T09:32:16.503276+0000","last_active":"2024-09-16T09:32:29.327654+0000","last_peered":"2024-09-16T09:32:29.327654+0000","last_clean":"2024-09-16T09:32:29.327654+0000","last_became_active":"2024-09-16T09:30:10.700866+0000","last_became_peered":"2024-09-16T09:30:10.700866+0000","last_unstale":"2024-09-16T09:32:29.327654+0000","last_undegraded":"2024-09-16T09:32:29.327654+0000","last_fullsized":"2024-09-16T09:32:29.327654+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:53.916505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:53.916505+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:32:43.945820+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109227,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"66'11","reported_seq":205,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.323855+0000","last_change":"2024-09-16T09:28:55.643505+0000","last_active":"2024-09-16T09:32:29.323855+0000","last_peered":"2024-09-16T09:32:29.323855+0000","last_clean":"2024-09-16T09:32:29.323855+0000","last_became_active":"2024-09-16T09:28:52.971132+0000","last_became_peered":"2024-09-16T09:28:52.971132+0000","last_unstale":"2024-09-16T09:32:29.323855+0000","last_undegraded":"2024-09-16T09:32:29.323855+0000","last_fullsized":"2024-09-16T09:32:29.323855+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-16T09:28:55.643448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:55.643448+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":19,"num_read_kb":12,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":330,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.320109+0000","last_change":"2024-09-16T09:27:46.175159+0000","last_active":"2024-09-16T09:32:29.320109+0000","last_peered":"2024-09-16T09:32:29.320109+0000","last_clean":"2024-09-16T09:32:29.320109+0000","last_became_active":"2024-09-16T09:25:01.700733+0000","last_became_peered":"2024-09-16T09:25:01.700733+0000","last_unstale":"2024-09-16T09:32:29.320109+0000","last_undegraded":"2024-09-16T09:32:29.320109+0000","last_fullsized":"2024-09-16T09:32:29.320109+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:46.175093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:27:46.175093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":322,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.322373+0000","last_change":"2024-09-16T09:29:25.017736+0000","last_active":"2024-09-16T09:32:29.322373+0000","last_peered":"2024-09-16T09:32:29.322373+0000","last_clean":"2024-09-16T09:32:29.322373+0000","last_became_active":"2024-09-16T09:25:05.713479+0000","last_became_peered":"2024-09-16T09:25:05.713479+0000","last_unstale":"2024-09-16T09:32:29.322373+0000","last_undegraded":"2024-09-16T09:32:29.322373+0000","last_fullsized":"2024-09-16T09:32:29.322373+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:25.017610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:25.017610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":181,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.327179+0000","last_change":"2024-09-16T09:31:09.275121+0000","last_active":"2024-09-16T09:32:29.327179+0000","last_peered":"2024-09-16T09:32:29.327179+0000","last_clean":"2024-09-16T09:32:29.327179+0000","last_became_active":"2024-09-16T09:31:09.274797+0000","last_became_peered":"2024-09-16T09:31:09.274797+0000","last_unstale":"2024-09-16T09:32:29.327179+0000","last_undegraded":"2024-09-16T09:32:29.327179+0000","last_fullsized":"2024-09-16T09:32:29.327179+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:54.589258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:28:54.589258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:48.251310+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"66'4","reported_seq":62,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.335518+0000","last_change":"2024-09-16T09:31:09.401626+0000","last_active":"2024-09-16T09:32:29.335518+0000","last_peered":"2024-09-16T09:32:29.335518+0000","last_clean":"2024-09-16T09:32:29.335518+0000","last_became_active":"2024-09-16T09:31:09.273682+0000","last_became_peered":"2024-09-16T09:31:09.273682+0000","last_unstale":"2024-09-16T09:32:29.335518+0000","last_undegraded":"2024-09-16T09:32:29.335518+0000","last_fullsized":"2024-09-16T09:32:29.335518+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'4","last_scrub_stamp":"2024-09-16T09:26:27.473740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:26:27.473740+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":353,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.330335+0000","last_change":"2024-09-16T09:31:30.235761+0000","last_active":"2024-09-16T09:32:29.330335+0000","last_peered":"2024-09-16T09:32:29.330335+0000","last_clean":"2024-09-16T09:32:29.330335+0000","last_became_active":"2024-09-16T09:31:09.274719+0000","last_became_peered":"2024-09-16T09:31:09.274719+0000","last_unstale":"2024-09-16T09:32:29.330335+0000","last_undegraded":"2024-09-16T09:32:29.330335+0000","last_fullsized":"2024-09-16T09:32:29.330335+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:31:30.235692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:31:30.235692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:49.736157+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":339,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.320169+0000","last_change":"2024-09-16T09:32:07.533575+0000","last_active":"2024-09-16T09:32:29.320169+0000","last_peered":"2024-09-16T09:32:29.320169+0000","last_clean":"2024-09-16T09:32:29.320169+0000","last_became_active":"2024-09-16T09:25:05.712545+0000","last_became_peered":"2024-09-16T09:25:05.712545+0000","last_unstale":"2024-09-16T09:32:29.320169+0000","last_undegraded":"2024-09-16T09:32:29.320169+0000","last_fullsized":"2024-09-16T09:32:29.320169+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:07.533529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:32:07.533529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:28.807212+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":318,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.322763+0000","last_change":"2024-09-16T09:29:27.012810+0000","last_active":"2024-09-16T09:32:29.322763+0000","last_peered":"2024-09-16T09:32:29.322763+0000","last_clean":"2024-09-16T09:32:29.322763+0000","last_became_active":"2024-09-16T09:25:07.720117+0000","last_became_peered":"2024-09-16T09:25:07.720117+0000","last_unstale":"2024-09-16T09:32:29.322763+0000","last_undegraded":"2024-09-16T09:32:29.322763+0000","last_fullsized":"2024-09-16T09:32:29.322763+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:27.012756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:27.012756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":318,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.317221+0000","last_change":"2024-09-16T09:29:19.888825+0000","last_active":"2024-09-16T09:32:29.317221+0000","last_peered":"2024-09-16T09:32:29.317221+0000","last_clean":"2024-09-16T09:32:29.317221+0000","last_became_active":"2024-09-16T09:25:07.712594+0000","last_became_peered":"2024-09-16T09:25:07.712594+0000","last_unstale":"2024-09-16T09:32:29.317221+0000","last_undegraded":"2024-09-16T09:32:29.317221+0000","last_fullsized":"2024-09-16T09:32:29.317221+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:19.888759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:19.888759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":180,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.326387+0000","last_change":"2024-09-16T09:31:26.211625+0000","last_active":"2024-09-16T09:32:29.326387+0000","last_peered":"2024-09-16T09:32:29.326387+0000","last_clean":"2024-09-16T09:32:29.326387+0000","last_became_active":"2024-09-16T09:31:09.271662+0000","last_became_peered":"2024-09-16T09:31:09.271662+0000","last_unstale":"2024-09-16T09:32:29.326387+0000","last_undegraded":"2024-09-16T09:32:29.326387+0000","last_fullsized":"2024-09-16T09:32:29.326387+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:31:26.211363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:31:26.211363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:32.873753+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"66'11","reported_seq":346,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:29.321272+0000","last_change":"2024-09-16T09:27:57.486459+0000","last_active":"2024-09-16T09:32:29.321272+0000","last_peered":"2024-09-16T09:32:29.321272+0000","last_clean":"2024-09-16T09:32:29.321272+0000","last_became_active":"2024-09-16T09:25:03.707769+0000","last_became_peered":"2024-09-16T09:25:03.707769+0000","last_unstale":"2024-09-16T09:32:29.321272+0000","last_undegraded":"2024-09-16T09:32:29.321272+0000","last_fullsized":"2024-09-16T09:32:29.321272+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-16T09:27:57.486381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:57.486381+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:17.472949+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":323,"reported_epoch":209,"state":"active+clean","last_fresh":"2024-09-16T09:32:31.325543+0000","last_change":"2024-09-16T09:32:31.325543+0000","last_active":"2024-09-16T09:32:31.325543+0000","last_peered":"2024-09-16T09:32:31.325543+0000","last_clean":"2024-09-16T09:32:31.325543+0000","last_became_active":"2024-09-16T09:25:05.711952+0000","last_became_peered":"2024-09-16T09:25:05.711952+0000","last_unstale":"2024-09-16T09:32:31.325543+0000","last_undegraded":"2024-09-16T09:32:31.325543+0000","last_fullsized":"2024-09-16T09:32:31.325543+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:31.325346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:32:31.325346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:54.289339+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":16,"stat_sum":{"num_bytes":3168993304,"num_objects":48356,"num_object_clones":0,"num_object_copies":145068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1602,"num_objects_unfound":0,"num_objects_dirty":48356,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":200830,"num_write_kb":7973889,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13107,"num_bytes_recovered":858980352,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":9908400128,"data_stored":9908387912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":77617,"ondisk_log_size":77617,"up":48,"acting":46,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":70,"ondisk_log_size":70,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":691,"num_read_kb":448,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":126,"num_bytes_recovered":3026,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":395,"ondisk_log_size":395,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1986,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":450,"num_read_kb":445,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":21,"stat_sum":{"num_bytes":806935,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":21,"num_read_kb":40,"num_write":21,"num_write_kb":296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":960396,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1659,"ondisk_log_size":1659,"up":63,"acting":63,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266784,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1131676,"kb_used_data":879756,"kb_used_omap":1,"kb_used_meta":251902,"kb_avail":92613476,"statfs":{"total":95995035648,"available":94836199424,"internally_reserved":0,"allocated":900870144,"data_stored":900557556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":257948106},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495717,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1079872,"kb_used_data":882096,"kb_used_omap":1,"kb_used_meta":197758,"kb_avail":92665280,"statfs":{"total":95995035648,"available":94889246720,"internally_reserved":0,"allocated":903266304,"data_stored":902981266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":202504646},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724651,"num_pgs":64,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1862864,"kb_used_data":1584512,"kb_used_omap":1,"kb_used_meta":278270,"kb_avail":91882288,"statfs":{"total":95995035648,"available":94087462912,"internally_reserved":0,"allocated":1622540288,"data_stored":1622239903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":284948934},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953584,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1509472,"kb_used_data":1293644,"kb_used_omap":1,"kb_used_meta":215806,"kb_avail":92235680,"statfs":{"total":95995035648,"available":94449336320,"internally_reserved":0,"allocated":1324691456,"data_stored":1324378908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":220985798},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149814,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1447144,"kb_used_data":1082776,"kb_used_omap":1,"kb_used_meta":364350,"kb_avail":92298008,"statfs":{"total":95995035648,"available":94513160192,"internally_reserved":0,"allocated":1108762624,"data_stored":1108438618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":373094855},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411451,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1497544,"kb_used_data":1209528,"kb_used_omap":1,"kb_used_meta":287998,"kb_avail":92247608,"statfs":{"total":95995035648,"available":94461550592,"internally_reserved":0,"allocated":1238556672,"data_stored":1238263942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":294910405},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574978,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1701172,"kb_used_data":1399140,"kb_used_omap":1,"kb_used_meta":301886,"kb_avail":92043980,"statfs":{"total":95995035648,"available":94253035520,"internally_reserved":0,"allocated":1432719360,"data_stored":1432382106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":309131717},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738504,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1785696,"kb_used_data":1483788,"kb_used_omap":1,"kb_used_meta":301886,"kb_avail":91959456,"statfs":{"total":95995035648,"available":94166482944,"internally_reserved":0,"allocated":1519398912,"data_stored":1519086566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":309131717},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1488846848,"data_stored":1488846848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1409024000,"data_stored":1409024000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1209536512,"data_stored":1209532440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1101791232,"data_stored":1101791232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1304297472,"data_stored":1304297472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1599606784,"data_stored":1599602712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":897187840,"data_stored":897187840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":898109440,"data_stored":898105368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T09:32:41.315 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph status --format=json 2024-09-16T09:32:41.342 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:32:41.376 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:41.377 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:41.377 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:32:41.478 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:32:41.523 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:41.523 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:41.524 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:32:41.624 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:32:41.658 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:41.659 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:41.659 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:32:41.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:41 smithi084 ceph-mon[32629]: 3.1f scrub ok 2024-09-16T09:32:41.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:41 smithi084 ceph-mon[32629]: pgmap v355: 168 pgs: 2 active+recovering+undersized+remapped, 166 active+clean; 2.6 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.50k op/s; 1050/130407 objects misplaced (0.805%); 8.1 MiB/s, 129 objects/s recovering 2024-09-16T09:32:41.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:41 smithi084 ceph-mon[32629]: 2.1 scrub starts 2024-09-16T09:32:41.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:41 smithi084 ceph-mon[32629]: 2.1 scrub ok 2024-09-16T09:32:41.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:41 smithi070 ceph-mon[30388]: 3.1f scrub ok 2024-09-16T09:32:41.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:41 smithi070 ceph-mon[30388]: pgmap v355: 168 pgs: 2 active+recovering+undersized+remapped, 166 active+clean; 2.6 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.50k op/s; 1050/130407 objects misplaced (0.805%); 8.1 MiB/s, 129 objects/s recovering 2024-09-16T09:32:41.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:41 smithi070 ceph-mon[30388]: 2.1 scrub starts 2024-09-16T09:32:41.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:41 smithi070 ceph-mon[30388]: 2.1 scrub ok 2024-09-16T09:32:41.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:41 smithi070 ceph-mon[40691]: 3.1f scrub ok 2024-09-16T09:32:41.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:41 smithi070 ceph-mon[40691]: pgmap v355: 168 pgs: 2 active+recovering+undersized+remapped, 166 active+clean; 2.6 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.50k op/s; 1050/130407 objects misplaced (0.805%); 8.1 MiB/s, 129 objects/s recovering 2024-09-16T09:32:41.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:41 smithi070 ceph-mon[40691]: 2.1 scrub starts 2024-09-16T09:32:41.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:41 smithi070 ceph-mon[40691]: 2.1 scrub ok 2024-09-16T09:32:41.760 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:32:41.789 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:41.790 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:41.791 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:32:41.892 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:32:41.927 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:41.928 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:41.928 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:32:42.029 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:32:42.061 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:42.062 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:42.062 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:32:42.163 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:32:42.170 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:32:41 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:32:41] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:32:42.200 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:42.201 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:42.201 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:32:42.302 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:32:42.336 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:42.336 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:42.337 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:32:42.411 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:32:42.434 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:32:42.438 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:32:42.486 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:42.487 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:42.487 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:32:42.588 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:32:42.625 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:42.626 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:42.626 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:32:42.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:42 smithi084 ceph-mon[32629]: 3.18 scrub starts 2024-09-16T09:32:42.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:42 smithi084 ceph-mon[32629]: 3.18 scrub ok 2024-09-16T09:32:42.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:42 smithi070 ceph-mon[30388]: 3.18 scrub starts 2024-09-16T09:32:42.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:42 smithi070 ceph-mon[30388]: 3.18 scrub ok 2024-09-16T09:32:42.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:42 smithi070 ceph-mon[40691]: 3.18 scrub starts 2024-09-16T09:32:42.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:42 smithi070 ceph-mon[40691]: 3.18 scrub ok 2024-09-16T09:32:42.727 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:32:42.759 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:42.759 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:42.760 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:32:42.860 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:32:42.893 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:42.894 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:42.894 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:32:42.995 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:32:43.028 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:43.029 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:43.030 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:32:43.130 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:32:43.162 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:43.162 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:43.163 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:32:43.264 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:32:43.299 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:43.299 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:43.300 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:32:43.396 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:32:42 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:32:43.400 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:32:43.434 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:43.435 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:43.435 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:32:43.536 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:32:43.577 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:43.577 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:43.577 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:32:43.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:43 smithi084 ceph-mon[32629]: pgmap v356: 168 pgs: 2 active+recovering+undersized+remapped, 166 active+clean; 2.6 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 731 op/s; 1050/130407 objects misplaced (0.805%); 4.3 MiB/s, 68 objects/s recovering 2024-09-16T09:32:43.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:43 smithi084 ceph-mon[32629]: 1.5 scrub starts 2024-09-16T09:32:43.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:43 smithi084 ceph-mon[32629]: 1.5 scrub ok 2024-09-16T09:32:43.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:43 smithi084 ceph-mon[32629]: 5.d scrub starts 2024-09-16T09:32:43.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:43 smithi084 ceph-mon[32629]: 5.d scrub ok 2024-09-16T09:32:43.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:43 smithi084 ceph-mon[32629]: 1.2 scrub starts 2024-09-16T09:32:43.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:43 smithi084 ceph-mon[32629]: 1.2 scrub ok 2024-09-16T09:32:43.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:43 smithi070 ceph-mon[30388]: pgmap v356: 168 pgs: 2 active+recovering+undersized+remapped, 166 active+clean; 2.6 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 731 op/s; 1050/130407 objects misplaced (0.805%); 4.3 MiB/s, 68 objects/s recovering 2024-09-16T09:32:43.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:43 smithi070 ceph-mon[30388]: 1.5 scrub starts 2024-09-16T09:32:43.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:43 smithi070 ceph-mon[30388]: 1.5 scrub ok 2024-09-16T09:32:43.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:43 smithi070 ceph-mon[30388]: 5.d scrub starts 2024-09-16T09:32:43.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:43 smithi070 ceph-mon[30388]: 5.d scrub ok 2024-09-16T09:32:43.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:43 smithi070 ceph-mon[30388]: 1.2 scrub starts 2024-09-16T09:32:43.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:43 smithi070 ceph-mon[30388]: 1.2 scrub ok 2024-09-16T09:32:43.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:43 smithi070 ceph-mon[40691]: pgmap v356: 168 pgs: 2 active+recovering+undersized+remapped, 166 active+clean; 2.6 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 731 op/s; 1050/130407 objects misplaced (0.805%); 4.3 MiB/s, 68 objects/s recovering 2024-09-16T09:32:43.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:43 smithi070 ceph-mon[40691]: 1.5 scrub starts 2024-09-16T09:32:43.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:43 smithi070 ceph-mon[40691]: 1.5 scrub ok 2024-09-16T09:32:43.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:43 smithi070 ceph-mon[40691]: 5.d scrub starts 2024-09-16T09:32:43.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:43 smithi070 ceph-mon[40691]: 5.d scrub ok 2024-09-16T09:32:43.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:43 smithi070 ceph-mon[40691]: 1.2 scrub starts 2024-09-16T09:32:43.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:43 smithi070 ceph-mon[40691]: 1.2 scrub ok 2024-09-16T09:32:43.678 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:32:43.716 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:43.717 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:43.717 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:32:43.757 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T09:32:43.757 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:32:43.818 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:32:43.838 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd set noscrub 2024-09-16T09:32:43.847 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:43.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:43.849 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:32:43.949 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:32:43.976 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:43.977 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:43.977 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:32:44.078 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:32:44.103 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:44.104 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:44.104 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:32:44.205 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:32:44.229 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:44.230 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:44.230 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:32:44.331 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:32:44.357 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:44.358 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:44.358 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:32:44.391 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T09:32:44.459 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:32:44.504 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:44.505 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:44.505 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:32:44.606 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:32:44.631 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:44.632 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:44.632 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:32:44.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:44 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:32:44.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:44 smithi084 ceph-mon[32629]: 1.e scrub starts 2024-09-16T09:32:44.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:44 smithi084 ceph-mon[32629]: 1.e scrub ok 2024-09-16T09:32:44.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:44 smithi084 ceph-mon[32629]: 1.1 scrub starts 2024-09-16T09:32:44.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:44 smithi084 ceph-mon[32629]: 1.1 scrub ok 2024-09-16T09:32:44.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:44 smithi084 ceph-mon[32629]: 1.0 scrub starts 2024-09-16T09:32:44.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:44 smithi084 ceph-mon[32629]: 1.0 scrub ok 2024-09-16T09:32:44.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:44 smithi084 ceph-mon[32629]: 1.14 scrub starts 2024-09-16T09:32:44.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:44 smithi084 ceph-mon[32629]: 1.14 scrub ok 2024-09-16T09:32:44.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:44 smithi084 ceph-mon[32629]: 1.b scrub starts 2024-09-16T09:32:44.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:44 smithi084 ceph-mon[32629]: 1.b scrub ok 2024-09-16T09:32:44.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:44 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:32:44.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:44 smithi070 ceph-mon[30388]: 1.e scrub starts 2024-09-16T09:32:44.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:44 smithi070 ceph-mon[30388]: 1.e scrub ok 2024-09-16T09:32:44.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:44 smithi070 ceph-mon[30388]: 1.1 scrub starts 2024-09-16T09:32:44.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:44 smithi070 ceph-mon[30388]: 1.1 scrub ok 2024-09-16T09:32:44.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:44 smithi070 ceph-mon[30388]: 1.0 scrub starts 2024-09-16T09:32:44.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:44 smithi070 ceph-mon[30388]: 1.0 scrub ok 2024-09-16T09:32:44.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:44 smithi070 ceph-mon[30388]: 1.14 scrub starts 2024-09-16T09:32:44.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:44 smithi070 ceph-mon[30388]: 1.14 scrub ok 2024-09-16T09:32:44.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:44 smithi070 ceph-mon[30388]: 1.b scrub starts 2024-09-16T09:32:44.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:44 smithi070 ceph-mon[30388]: 1.b scrub ok 2024-09-16T09:32:44.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:44 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:32:44.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:44 smithi070 ceph-mon[40691]: 1.e scrub starts 2024-09-16T09:32:44.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:44 smithi070 ceph-mon[40691]: 1.e scrub ok 2024-09-16T09:32:44.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:44 smithi070 ceph-mon[40691]: 1.1 scrub starts 2024-09-16T09:32:44.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:44 smithi070 ceph-mon[40691]: 1.1 scrub ok 2024-09-16T09:32:44.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:44 smithi070 ceph-mon[40691]: 1.0 scrub starts 2024-09-16T09:32:44.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:44 smithi070 ceph-mon[40691]: 1.0 scrub ok 2024-09-16T09:32:44.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:44 smithi070 ceph-mon[40691]: 1.14 scrub starts 2024-09-16T09:32:44.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:44 smithi070 ceph-mon[40691]: 1.14 scrub ok 2024-09-16T09:32:44.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:44 smithi070 ceph-mon[40691]: 1.b scrub starts 2024-09-16T09:32:44.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:44 smithi070 ceph-mon[40691]: 1.b scrub ok 2024-09-16T09:32:44.734 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:32:44.773 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:44.774 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:44.774 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:32:44.875 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:32:44.906 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:44.906 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:44.907 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:32:45.008 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:32:45.042 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:45.042 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:45.043 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:32:45.144 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:32:45.180 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:45.181 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:45.182 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:32:45.282 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:32:45.317 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:45.318 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:45.318 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:32:45.419 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:32:45.455 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:45.456 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:45.456 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:32:45.557 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:32:45.578 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:45.579 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:45.579 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:32:45.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:45 smithi070 ceph-mon[40691]: pgmap v357: 168 pgs: 2 active+recovering+undersized+remapped, 166 active+clean; 2.3 GiB data, 9.5 GiB used, 706 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.52k op/s; 441/111552 objects misplaced (0.395%); 9.1 MiB/s, 145 objects/s recovering 2024-09-16T09:32:45.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:45 smithi070 ceph-mon[40691]: 5.18 scrub starts 2024-09-16T09:32:45.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:45 smithi070 ceph-mon[40691]: 5.18 scrub ok 2024-09-16T09:32:45.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:45 smithi070 ceph-mon[40691]: 1.10 scrub starts 2024-09-16T09:32:45.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:45 smithi070 ceph-mon[40691]: 1.10 scrub ok 2024-09-16T09:32:45.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:45 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1760301128' entity='client.admin' cmd={"prefix": "status", "format": "json"} : dispatch 2024-09-16T09:32:45.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:45 smithi070 ceph-mon[40691]: 1.f scrub starts 2024-09-16T09:32:45.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:45 smithi070 ceph-mon[40691]: 1.f scrub ok 2024-09-16T09:32:45.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:45 smithi070 ceph-mon[40691]: 1.d scrub starts 2024-09-16T09:32:45.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:45 smithi070 ceph-mon[40691]: 1.d scrub ok 2024-09-16T09:32:45.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:45 smithi070 ceph-mon[40691]: 4.19 scrub starts 2024-09-16T09:32:45.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:45 smithi070 ceph-mon[40691]: 4.19 scrub ok 2024-09-16T09:32:45.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:45 smithi070 ceph-mon[30388]: pgmap v357: 168 pgs: 2 active+recovering+undersized+remapped, 166 active+clean; 2.3 GiB data, 9.5 GiB used, 706 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.52k op/s; 441/111552 objects misplaced (0.395%); 9.1 MiB/s, 145 objects/s recovering 2024-09-16T09:32:45.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:45 smithi070 ceph-mon[30388]: 5.18 scrub starts 2024-09-16T09:32:45.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:45 smithi070 ceph-mon[30388]: 5.18 scrub ok 2024-09-16T09:32:45.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:45 smithi070 ceph-mon[30388]: 1.10 scrub starts 2024-09-16T09:32:45.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:45 smithi070 ceph-mon[30388]: 1.10 scrub ok 2024-09-16T09:32:45.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:45 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1760301128' entity='client.admin' cmd={"prefix": "status", "format": "json"} : dispatch 2024-09-16T09:32:45.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:45 smithi070 ceph-mon[30388]: 1.f scrub starts 2024-09-16T09:32:45.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:45 smithi070 ceph-mon[30388]: 1.f scrub ok 2024-09-16T09:32:45.673 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:45 smithi070 ceph-mon[30388]: 1.d scrub starts 2024-09-16T09:32:45.673 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:45 smithi070 ceph-mon[30388]: 1.d scrub ok 2024-09-16T09:32:45.673 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:45 smithi070 ceph-mon[30388]: 4.19 scrub starts 2024-09-16T09:32:45.673 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:45 smithi070 ceph-mon[30388]: 4.19 scrub ok 2024-09-16T09:32:45.680 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:32:45.701 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:45.702 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:45.703 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:32:45.803 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:32:45.825 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:45.825 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:45.826 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:32:45.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:45 smithi084 ceph-mon[32629]: pgmap v357: 168 pgs: 2 active+recovering+undersized+remapped, 166 active+clean; 2.3 GiB data, 9.5 GiB used, 706 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.52k op/s; 441/111552 objects misplaced (0.395%); 9.1 MiB/s, 145 objects/s recovering 2024-09-16T09:32:45.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:45 smithi084 ceph-mon[32629]: 5.18 scrub starts 2024-09-16T09:32:45.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:45 smithi084 ceph-mon[32629]: 5.18 scrub ok 2024-09-16T09:32:45.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:45 smithi084 ceph-mon[32629]: 1.10 scrub starts 2024-09-16T09:32:45.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:45 smithi084 ceph-mon[32629]: 1.10 scrub ok 2024-09-16T09:32:45.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:45 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1760301128' entity='client.admin' cmd={"prefix": "status", "format": "json"} : dispatch 2024-09-16T09:32:45.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:45 smithi084 ceph-mon[32629]: 1.f scrub starts 2024-09-16T09:32:45.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:45 smithi084 ceph-mon[32629]: 1.f scrub ok 2024-09-16T09:32:45.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:45 smithi084 ceph-mon[32629]: 1.d scrub starts 2024-09-16T09:32:45.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:45 smithi084 ceph-mon[32629]: 1.d scrub ok 2024-09-16T09:32:45.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:45 smithi084 ceph-mon[32629]: 4.19 scrub starts 2024-09-16T09:32:45.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:45 smithi084 ceph-mon[32629]: 4.19 scrub ok 2024-09-16T09:32:45.926 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:32:45.947 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:45.948 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:45.948 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:32:46.049 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:32:46.071 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:46.072 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:46.072 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:32:46.173 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:32:46.194 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:46.195 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:46.195 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:32:46.255 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.2 dump_historic_ops 2024-09-16T09:32:46.296 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:32:46.319 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:46.319 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:46.320 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:32:46.359 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:32:46.359 INFO:teuthology.orchestra.run.smithi070.stdout:{"fsid":"a1321834-740c-11ef-bceb-c7b262605968","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 2 pgs undersized","count":2},"muted":false}},"mutes":[]},"election_epoch":14,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":711,"monmap":{"epoch":3,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":211,"num_osds":8,"num_up_osds":8,"osd_up_since":1726478691,"num_in_osds":8,"osd_in_since":1726478668,"num_remapped_pgs":2},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":166},{"state_name":"active+recovering+undersized+remapped","count":2}],"num_pgs":168,"num_pools":7,"num_objects":37184,"data_bytes":2424530986,"bytes_used":10216415232,"bytes_avail":757743869952,"bytes_total":767960285184,"misplaced_objects":441,"misplaced_total":111552,"misplaced_ratio":0.0039533132530120479,"recovering_objects_per_sec":145,"recovering_bytes_per_sec":9505348,"recovering_keys_per_sec":0,"num_objects_recovered":1161,"num_bytes_recovered":76087296,"num_keys_recovered":0,"read_bytes_sec":1279,"write_bytes_sec":0,"read_op_per_sec":1,"write_op_per_sec":1516},"fsmap":{"epoch":1,"btime":"2024-09-16T09:19:14:121324+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":1,"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"services":{"dashboard":"https://172.21.15.70:8443/","prometheus":"http://172.21.15.70:9283/"}},"servicemap":{"epoch":14,"modified":"2024-09-16T09:26:33.498407+0000","services":{"rgw":{"daemons":{"summary":"","14475":{"start_epoch":12,"start_stamp":"2024-09-16T09:25:08.794234+0000","gid":14475,"addr":"172.21.15.70:0/1049757559","metadata":{"arch":"x86_64","ceph_release":"squid","ceph_version":"ceph version 19.3.0-4917-g8293d73f (8293d73f8690540e843a81caec373f9cc29cf705) squid (dev)","ceph_version_short":"19.3.0-4917-g8293d73f","container_hostname":"smithi070","container_image":"quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce","cpu":"Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz","distro":"centos","distro_description":"CentOS Stream 9","distro_version":"9","frontend_config#0":"beast port=80","frontend_type#0":"beast","hostname":"smithi070","id":"foo.a","kernel_description":"#1 SMP PREEMPT_DYNAMIC Thu Sep 5 07:54:07 UTC 2024","kernel_version":"5.14.0-505.el9.x86_64","mem_swap_kb":"0","mem_total_kb":"32487212","num_handles":"1","os":"Linux","pid":"7","realm_id":"","realm_name":"","zone_id":"09633d1c-04b1-424a-bf26-5fa0f61f30f8","zone_name":"default","zonegroup_id":"c6682a31-37e4-465e-b890-750b49769dfa","zonegroup_name":"default"},"task_status":{}}}}}},"progress_events":{}} 2024-09-16T09:32:46.420 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:32:46.447 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:46.447 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:46.448 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:32:46.549 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:32:46.575 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:46.576 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:46.576 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:32:46.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:46 smithi070 ceph-mon[30388]: 1.12 scrub starts 2024-09-16T09:32:46.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:46 smithi070 ceph-mon[30388]: 1.12 scrub ok 2024-09-16T09:32:46.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:46 smithi070 ceph-mon[30388]: osdmap e212: 8 total, 8 up, 8 in 2024-09-16T09:32:46.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:46 smithi070 ceph-mon[30388]: 1.7 scrub starts 2024-09-16T09:32:46.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:46 smithi070 ceph-mon[30388]: 1.7 scrub ok 2024-09-16T09:32:46.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:46 smithi070 ceph-mon[30388]: 6.4 scrub starts 2024-09-16T09:32:46.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:46 smithi070 ceph-mon[30388]: 6.4 scrub ok 2024-09-16T09:32:46.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:46 smithi070 ceph-mon[40691]: 1.12 scrub starts 2024-09-16T09:32:46.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:46 smithi070 ceph-mon[40691]: 1.12 scrub ok 2024-09-16T09:32:46.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:46 smithi070 ceph-mon[40691]: osdmap e212: 8 total, 8 up, 8 in 2024-09-16T09:32:46.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:46 smithi070 ceph-mon[40691]: 1.7 scrub starts 2024-09-16T09:32:46.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:46 smithi070 ceph-mon[40691]: 1.7 scrub ok 2024-09-16T09:32:46.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:46 smithi070 ceph-mon[40691]: 6.4 scrub starts 2024-09-16T09:32:46.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:46 smithi070 ceph-mon[40691]: 6.4 scrub ok 2024-09-16T09:32:46.677 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:32:46.703 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:46.703 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:46.704 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:32:46.805 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:32:46.828 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:46.829 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:46.829 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:32:46.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:46 smithi084 ceph-mon[32629]: 1.12 scrub starts 2024-09-16T09:32:46.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:46 smithi084 ceph-mon[32629]: 1.12 scrub ok 2024-09-16T09:32:46.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:46 smithi084 ceph-mon[32629]: osdmap e212: 8 total, 8 up, 8 in 2024-09-16T09:32:46.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:46 smithi084 ceph-mon[32629]: 1.7 scrub starts 2024-09-16T09:32:46.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:46 smithi084 ceph-mon[32629]: 1.7 scrub ok 2024-09-16T09:32:46.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:46 smithi084 ceph-mon[32629]: 6.4 scrub starts 2024-09-16T09:32:46.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:46 smithi084 ceph-mon[32629]: 6.4 scrub ok 2024-09-16T09:32:46.930 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:32:46.952 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:46.952 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:46.953 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:32:47.000 INFO:tasks.cephadm.ceph_manager.ceph:making progress, resetting timeout 2024-09-16T09:32:47.001 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-16T09:32:47.053 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:32:47.060 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:32:47.069 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:32:47.077 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:47.077 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:47.078 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:32:47.179 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:32:47.202 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:47.202 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:47.203 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:32:47.304 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:32:47.326 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:47.327 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:47.328 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:32:47.429 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:32:47.455 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:47.456 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:47.456 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:32:47.557 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:32:47.602 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:47.603 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:47.603 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:32:47.617 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:47 smithi070 ceph-mon[30388]: pgmap v359: 168 pgs: 2 active+recovering+undersized+remapped, 166 active+clean; 2.3 GiB data, 9.5 GiB used, 706 GiB / 715 GiB avail; 705 B/s rd, 0 B/s wr, 867 op/s; 441/111552 objects misplaced (0.395%); 5.2 MiB/s, 83 objects/s recovering 2024-09-16T09:32:47.617 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:47 smithi070 ceph-mon[30388]: 3.5 scrub starts 2024-09-16T09:32:47.618 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:47 smithi070 ceph-mon[30388]: 3.5 scrub ok 2024-09-16T09:32:47.618 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:47 smithi070 ceph-mon[30388]: osdmap e213: 8 total, 8 up, 8 in 2024-09-16T09:32:47.618 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:47 smithi070 ceph-mon[30388]: 5.1f scrub starts 2024-09-16T09:32:47.618 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:47 smithi070 ceph-mon[30388]: 5.1f scrub ok 2024-09-16T09:32:47.618 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:47 smithi070 ceph-mon[30388]: 8.f scrub starts 2024-09-16T09:32:47.618 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:47 smithi070 ceph-mon[30388]: 4.12 scrub starts 2024-09-16T09:32:47.618 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:47 smithi070 ceph-mon[30388]: 4.12 scrub ok 2024-09-16T09:32:47.618 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:47 smithi070 ceph-mon[40691]: pgmap v359: 168 pgs: 2 active+recovering+undersized+remapped, 166 active+clean; 2.3 GiB data, 9.5 GiB used, 706 GiB / 715 GiB avail; 705 B/s rd, 0 B/s wr, 867 op/s; 441/111552 objects misplaced (0.395%); 5.2 MiB/s, 83 objects/s recovering 2024-09-16T09:32:47.618 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:47 smithi070 ceph-mon[40691]: 3.5 scrub starts 2024-09-16T09:32:47.618 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:47 smithi070 ceph-mon[40691]: 3.5 scrub ok 2024-09-16T09:32:47.618 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:47 smithi070 ceph-mon[40691]: osdmap e213: 8 total, 8 up, 8 in 2024-09-16T09:32:47.618 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:47 smithi070 ceph-mon[40691]: 5.1f scrub starts 2024-09-16T09:32:47.619 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:47 smithi070 ceph-mon[40691]: 5.1f scrub ok 2024-09-16T09:32:47.619 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:47 smithi070 ceph-mon[40691]: 8.f scrub starts 2024-09-16T09:32:47.619 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:47 smithi070 ceph-mon[40691]: 4.12 scrub starts 2024-09-16T09:32:47.619 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:47 smithi070 ceph-mon[40691]: 4.12 scrub ok 2024-09-16T09:32:47.704 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:32:47.745 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:47.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:47.746 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:32:47.847 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:32:47.881 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:47.881 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:47.881 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:32:47.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:47 smithi084 ceph-mon[32629]: pgmap v359: 168 pgs: 2 active+recovering+undersized+remapped, 166 active+clean; 2.3 GiB data, 9.5 GiB used, 706 GiB / 715 GiB avail; 705 B/s rd, 0 B/s wr, 867 op/s; 441/111552 objects misplaced (0.395%); 5.2 MiB/s, 83 objects/s recovering 2024-09-16T09:32:47.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:47 smithi084 ceph-mon[32629]: 3.5 scrub starts 2024-09-16T09:32:47.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:47 smithi084 ceph-mon[32629]: 3.5 scrub ok 2024-09-16T09:32:47.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:47 smithi084 ceph-mon[32629]: osdmap e213: 8 total, 8 up, 8 in 2024-09-16T09:32:47.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:47 smithi084 ceph-mon[32629]: 5.1f scrub starts 2024-09-16T09:32:47.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:47 smithi084 ceph-mon[32629]: 5.1f scrub ok 2024-09-16T09:32:47.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:47 smithi084 ceph-mon[32629]: 8.f scrub starts 2024-09-16T09:32:47.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:47 smithi084 ceph-mon[32629]: 4.12 scrub starts 2024-09-16T09:32:47.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:47 smithi084 ceph-mon[32629]: 4.12 scrub ok 2024-09-16T09:32:47.982 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:32:48.022 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:48.022 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:48.023 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:32:48.123 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:32:48.151 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:48.152 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:48.152 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:32:48.253 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:32:48.290 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:48.290 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:48.291 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:32:48.391 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:32:48.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:48.435 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:48.435 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:32:48.473 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T09:32:48.473 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:32:48.536 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:32:48.576 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:48.577 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:48.577 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:32:48.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:48 smithi070 ceph-mon[30388]: osdmap e214: 8 total, 8 up, 8 in 2024-09-16T09:32:48.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:48 smithi070 ceph-mon[30388]: 8.f scrub ok 2024-09-16T09:32:48.673 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:48 smithi070 ceph-mon[30388]: 3.15 scrub starts 2024-09-16T09:32:48.673 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:48 smithi070 ceph-mon[30388]: 3.15 scrub ok 2024-09-16T09:32:48.673 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:48 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:32:48.673 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:48 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:32:48.673 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:48 smithi070 ceph-mon[30388]: 4.17 scrub starts 2024-09-16T09:32:48.673 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:48 smithi070 ceph-mon[30388]: 6.f scrub starts 2024-09-16T09:32:48.673 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:48 smithi070 ceph-mon[30388]: 6.f scrub ok 2024-09-16T09:32:48.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:48 smithi070 ceph-mon[40691]: osdmap e214: 8 total, 8 up, 8 in 2024-09-16T09:32:48.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:48 smithi070 ceph-mon[40691]: 8.f scrub ok 2024-09-16T09:32:48.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:48 smithi070 ceph-mon[40691]: 3.15 scrub starts 2024-09-16T09:32:48.674 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:48 smithi070 ceph-mon[40691]: 3.15 scrub ok 2024-09-16T09:32:48.674 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:48 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:32:48.674 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:48 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:32:48.674 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:48 smithi070 ceph-mon[40691]: 4.17 scrub starts 2024-09-16T09:32:48.674 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:48 smithi070 ceph-mon[40691]: 6.f scrub starts 2024-09-16T09:32:48.674 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:48 smithi070 ceph-mon[40691]: 6.f scrub ok 2024-09-16T09:32:48.678 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:32:48.711 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:48.712 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:48.712 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:32:48.729 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:32:48.813 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:32:48.838 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:48.839 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:48.839 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:32:48.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:48 smithi084 ceph-mon[32629]: osdmap e214: 8 total, 8 up, 8 in 2024-09-16T09:32:48.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:48 smithi084 ceph-mon[32629]: 8.f scrub ok 2024-09-16T09:32:48.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:48 smithi084 ceph-mon[32629]: 3.15 scrub starts 2024-09-16T09:32:48.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:48 smithi084 ceph-mon[32629]: 3.15 scrub ok 2024-09-16T09:32:48.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:48 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:32:48.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:48 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:32:48.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:48 smithi084 ceph-mon[32629]: 4.17 scrub starts 2024-09-16T09:32:48.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:48 smithi084 ceph-mon[32629]: 6.f scrub starts 2024-09-16T09:32:48.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:48 smithi084 ceph-mon[32629]: 6.f scrub ok 2024-09-16T09:32:48.940 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:32:48.980 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:48.981 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:48.981 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:32:49.082 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:32:49.124 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:49.125 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:49.125 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:32:49.226 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:32:49.265 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:49.265 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:49.266 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:32:49.367 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:32:49.398 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:49.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:49.399 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:32:49.448 INFO:teuthology.orchestra.run.smithi070.stderr:noscrub is set 2024-09-16T09:32:49.500 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:32:49.527 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:49.527 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:49.528 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:32:49.628 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:32:49.640 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.3 dump_ops_in_flight 2024-09-16T09:32:49.670 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:49.671 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:49.671 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:32:49.705 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:49 smithi070 ceph-mon[30388]: pgmap v362: 168 pgs: 2 active+recovering+undersized+remapped, 166 active+clean; 2.2 GiB data, 9.4 GiB used, 706 GiB / 715 GiB avail; 0 B/s rd, 0 B/s wr, 1.13k op/s; 441/109995 objects misplaced (0.401%); 6.3 MiB/s, 101 objects/s recovering 2024-09-16T09:32:49.705 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:49 smithi070 ceph-mon[30388]: 4.17 scrub ok 2024-09-16T09:32:49.705 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:49 smithi070 ceph-mon[30388]: 4.1a scrub starts 2024-09-16T09:32:49.705 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:49 smithi070 ceph-mon[30388]: 4.1a scrub ok 2024-09-16T09:32:49.705 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:49 smithi070 ceph-mon[30388]: 4.6 scrub starts 2024-09-16T09:32:49.705 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:49 smithi070 ceph-mon[30388]: 4.6 scrub ok 2024-09-16T09:32:49.706 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:49 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1645082407' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:32:49.706 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:49 smithi070 ceph-mon[30388]: 4.10 scrub starts 2024-09-16T09:32:49.706 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:49 smithi070 ceph-mon[30388]: 4.1c scrub starts 2024-09-16T09:32:49.706 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:49 smithi070 ceph-mon[30388]: 4.1c scrub ok 2024-09-16T09:32:49.706 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:49 smithi070 ceph-mon[40691]: pgmap v362: 168 pgs: 2 active+recovering+undersized+remapped, 166 active+clean; 2.2 GiB data, 9.4 GiB used, 706 GiB / 715 GiB avail; 0 B/s rd, 0 B/s wr, 1.13k op/s; 441/109995 objects misplaced (0.401%); 6.3 MiB/s, 101 objects/s recovering 2024-09-16T09:32:49.706 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:49 smithi070 ceph-mon[40691]: 4.17 scrub ok 2024-09-16T09:32:49.706 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:49 smithi070 ceph-mon[40691]: 4.1a scrub starts 2024-09-16T09:32:49.706 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:49 smithi070 ceph-mon[40691]: 4.1a scrub ok 2024-09-16T09:32:49.706 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:49 smithi070 ceph-mon[40691]: 4.6 scrub starts 2024-09-16T09:32:49.706 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:49 smithi070 ceph-mon[40691]: 4.6 scrub ok 2024-09-16T09:32:49.706 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:49 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1645082407' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:32:49.707 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:49 smithi070 ceph-mon[40691]: 4.10 scrub starts 2024-09-16T09:32:49.707 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:49 smithi070 ceph-mon[40691]: 4.1c scrub starts 2024-09-16T09:32:49.707 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:49 smithi070 ceph-mon[40691]: 4.1c scrub ok 2024-09-16T09:32:49.772 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:32:49.797 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:49.798 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:49.798 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:32:49.878 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:32:49.899 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:32:49.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:49 smithi084 ceph-mon[32629]: pgmap v362: 168 pgs: 2 active+recovering+undersized+remapped, 166 active+clean; 2.2 GiB data, 9.4 GiB used, 706 GiB / 715 GiB avail; 0 B/s rd, 0 B/s wr, 1.13k op/s; 441/109995 objects misplaced (0.401%); 6.3 MiB/s, 101 objects/s recovering 2024-09-16T09:32:49.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:49 smithi084 ceph-mon[32629]: 4.17 scrub ok 2024-09-16T09:32:49.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:49 smithi084 ceph-mon[32629]: 4.1a scrub starts 2024-09-16T09:32:49.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:49 smithi084 ceph-mon[32629]: 4.1a scrub ok 2024-09-16T09:32:49.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:49 smithi084 ceph-mon[32629]: 4.6 scrub starts 2024-09-16T09:32:49.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:49 smithi084 ceph-mon[32629]: 4.6 scrub ok 2024-09-16T09:32:49.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:49 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1645082407' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:32:49.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:49 smithi084 ceph-mon[32629]: 4.10 scrub starts 2024-09-16T09:32:49.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:49 smithi084 ceph-mon[32629]: 4.1c scrub starts 2024-09-16T09:32:49.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:49 smithi084 ceph-mon[32629]: 4.1c scrub ok 2024-09-16T09:32:49.922 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:49.923 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:49.923 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:32:50.024 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:32:50.056 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:50.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:50.057 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:32:50.158 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:32:50.204 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:50.205 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:50.205 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:32:50.306 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:32:50.345 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:50.346 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:50.346 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:32:50.447 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:32:50.492 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:50.492 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:50.493 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:32:50.541 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:50 smithi070 ceph-mon[30388]: 4.10 scrub ok 2024-09-16T09:32:50.541 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:50 smithi070 ceph-mon[30388]: 3.f scrub starts 2024-09-16T09:32:50.541 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:50 smithi070 ceph-mon[30388]: 3.f scrub ok 2024-09-16T09:32:50.542 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:50 smithi070 ceph-mon[30388]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:32:50.542 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:50 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1645082407' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-16T09:32:50.542 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:50 smithi070 ceph-mon[30388]: osdmap e215: 8 total, 8 up, 8 in 2024-09-16T09:32:50.542 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:50 smithi070 ceph-mon[30388]: pgmap v364: 168 pgs: 168 active+clean; 2.0 GiB data, 8.9 GiB used, 706 GiB / 715 GiB avail; 0 B/s wr, 946 op/s; 6.5 MiB/s, 104 objects/s recovering 2024-09-16T09:32:50.542 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:50 smithi070 ceph-mon[40691]: 4.10 scrub ok 2024-09-16T09:32:50.542 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:50 smithi070 ceph-mon[40691]: 3.f scrub starts 2024-09-16T09:32:50.542 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:50 smithi070 ceph-mon[40691]: 3.f scrub ok 2024-09-16T09:32:50.542 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:50 smithi070 ceph-mon[40691]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:32:50.542 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:50 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1645082407' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-16T09:32:50.542 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:50 smithi070 ceph-mon[40691]: osdmap e215: 8 total, 8 up, 8 in 2024-09-16T09:32:50.542 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:50 smithi070 ceph-mon[40691]: pgmap v364: 168 pgs: 168 active+clean; 2.0 GiB data, 8.9 GiB used, 706 GiB / 715 GiB avail; 0 B/s wr, 946 op/s; 6.5 MiB/s, 104 objects/s recovering 2024-09-16T09:32:50.593 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:32:50.629 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:50.629 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:50.629 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:32:50.730 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:32:50.766 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:50.767 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:50.767 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:32:50.811 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:50 smithi084 ceph-mon[32629]: 4.10 scrub ok 2024-09-16T09:32:50.811 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:50 smithi084 ceph-mon[32629]: 3.f scrub starts 2024-09-16T09:32:50.811 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:50 smithi084 ceph-mon[32629]: 3.f scrub ok 2024-09-16T09:32:50.811 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:50 smithi084 ceph-mon[32629]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:32:50.811 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:50 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1645082407' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-16T09:32:50.811 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:50 smithi084 ceph-mon[32629]: osdmap e215: 8 total, 8 up, 8 in 2024-09-16T09:32:50.811 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:50 smithi084 ceph-mon[32629]: pgmap v364: 168 pgs: 168 active+clean; 2.0 GiB data, 8.9 GiB used, 706 GiB / 715 GiB avail; 0 B/s wr, 946 op/s; 6.5 MiB/s, 104 objects/s recovering 2024-09-16T09:32:50.868 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:32:50.902 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:50.903 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:50.903 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:32:51.004 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:32:51.041 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:51.041 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:51.041 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:32:51.142 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:32:51.177 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:51.177 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:51.177 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:32:51.278 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:32:51.307 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:51.308 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:51.308 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:32:51.409 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:32:51.410 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:32:51.462 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:51.463 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:51.463 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:32:51.464 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:32:51.465 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:32:51.518 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:32:51.519 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:32:51.541 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:51 smithi084 ceph-mon[32629]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2 pgs undersized) 2024-09-16T09:32:51.541 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:51 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/864583854' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:32:51.550 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:32:51.551 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:32:51.551 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:32:51.560 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2024-09-16T09:32:51.564 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:32:51.586 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:32:51.587 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:32:51.624 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:32:51.624 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:32:51.635 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:51.635 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:51.635 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:32:51.661 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:32:51.661 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:32:51.662 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:51 smithi070 ceph-mon[30388]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2 pgs undersized) 2024-09-16T09:32:51.662 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:51 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/864583854' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:32:51.662 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:51 smithi070 ceph-mon[40691]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2 pgs undersized) 2024-09-16T09:32:51.662 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:51 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/864583854' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:32:51.695 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:32:51.696 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:32:51.735 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:32:51.736 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:32:51.736 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:32:51.769 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:51.770 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:51.770 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:32:51.871 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:32:51.910 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:51.910 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:51.911 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:32:52.011 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:32:52.042 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:52.043 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:52.043 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:32:52.144 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:32:52.183 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:52.184 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:52.184 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:32:52.208 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:32:51 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:32:51] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:32:52.252 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T09:32:52.253 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [ 2024-09-16T09:32:52.253 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:52.253 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.25064.0:221354 8.f e215/152 8:fddb9c2e:::benchmark_data_smithi070_107919_object96759:head v 215'13980, mlcod=215'13978)", 2024-09-16T09:32:52.253 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:32:52.248504+0000", 2024-09-16T09:32:52.253 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.0024335440000000002, 2024-09-16T09:32:52.253 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.002460271, 2024-09-16T09:32:52.253 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:32:52.253 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:32:52.253 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T09:32:52.253 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:32:52.253 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:52.254 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:32:52.254 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:52.248504+0000", 2024-09-16T09:32:52.254 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:32:52.254 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:52.254 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:52.254 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:32:52.254 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:52.248504+0000", 2024-09-16T09:32:52.254 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:32:52.254 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:52.254 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:52.254 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:32:52.254 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:52.248505+0000", 2024-09-16T09:32:52.254 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.8000000000000005e-07 2024-09-16T09:32:52.255 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:52.255 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:52.255 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:32:52.255 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:52.248511+0000", 2024-09-16T09:32:52.255 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.246e-06 2024-09-16T09:32:52.255 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:52.255 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:52.255 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:32:52.255 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:52.248521+0000", 2024-09-16T09:32:52.255 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.927e-06 2024-09-16T09:32:52.255 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:52.255 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:52.255 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:32:52.255 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:52.248527+0000", 2024-09-16T09:32:52.256 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.9279999999999998e-06 2024-09-16T09:32:52.256 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:52.256 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:52.256 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:32:52.256 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:52.248636+0000", 2024-09-16T09:32:52.256 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000109203 2024-09-16T09:32:52.256 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:52.256 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:52.256 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:32:52.256 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:52.248654+0000", 2024-09-16T09:32:52.256 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7322000000000001e-05 2024-09-16T09:32:52.256 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:32:52.256 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:32:52.256 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:32:52.257 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:52.257 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:52.257 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.25064.0:221360 8.7 e215/152 8:e5cad466:::benchmark_data_smithi070_107919_object96765:head v 215'13686, mlcod=215'13684)", 2024-09-16T09:32:52.257 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:32:52.248554+0000", 2024-09-16T09:32:52.257 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.002383909, 2024-09-16T09:32:52.257 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.002465503, 2024-09-16T09:32:52.257 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:32:52.257 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:32:52.257 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T09:32:52.257 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:32:52.257 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:52.257 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:32:52.257 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:52.248554+0000", 2024-09-16T09:32:52.258 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:32:52.258 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:52.258 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:52.258 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:32:52.258 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:52.248554+0000", 2024-09-16T09:32:52.258 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:32:52.258 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:52.258 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:52.258 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:32:52.258 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:52.248554+0000", 2024-09-16T09:32:52.258 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1699999999999999e-07 2024-09-16T09:32:52.258 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:52.258 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:52.258 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:32:52.259 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:52.248558+0000", 2024-09-16T09:32:52.259 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4070000000000002e-06 2024-09-16T09:32:52.259 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:52.259 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:52.259 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:32:52.259 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:52.248564+0000", 2024-09-16T09:32:52.259 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.2580000000000001e-06 2024-09-16T09:32:52.259 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:52.259 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:52.259 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:32:52.259 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:52.248568+0000", 2024-09-16T09:32:52.259 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.1609999999999996e-06 2024-09-16T09:32:52.259 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:52.259 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:52.259 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:32:52.260 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:52.248681+0000", 2024-09-16T09:32:52.260 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000112836 2024-09-16T09:32:52.260 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:32:52.260 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:32:52.260 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:32:52.260 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:32:52.248699+0000", 2024-09-16T09:32:52.260 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.8037999999999999e-05 2024-09-16T09:32:52.260 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:32:52.260 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:32:52.260 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:32:52.260 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:32:52.260 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-16T09:32:52.260 INFO:teuthology.orchestra.run.smithi070.stdout: "num_ops": 2 2024-09-16T09:32:52.260 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T09:32:52.285 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:32:52.322 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:52.323 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:52.323 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:32:52.366 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-16T09:32:52.424 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:32:52.455 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:52.456 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:52.456 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:32:52.557 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:32:52.589 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:52.590 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:52.590 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:32:52.691 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:32:52.726 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:52.727 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:52.727 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:32:52.828 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:32:52.856 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:52 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:32:52.857 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:52 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:32:52.857 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:52 smithi070 ceph-mon[30388]: from='client.25424 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:32:52.857 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:52 smithi070 ceph-mon[30388]: pgmap v365: 168 pgs: 168 active+clean; 2.0 GiB data, 8.9 GiB used, 706 GiB / 715 GiB avail; 0 B/s wr, 666 op/s; 4.6 MiB/s, 73 objects/s recovering 2024-09-16T09:32:52.857 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:52 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:32:52.857 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:52 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:32:52.857 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:52 smithi070 ceph-mon[40691]: from='client.25424 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:32:52.857 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:52 smithi070 ceph-mon[40691]: pgmap v365: 168 pgs: 168 active+clean; 2.0 GiB data, 8.9 GiB used, 706 GiB / 715 GiB avail; 0 B/s wr, 666 op/s; 4.6 MiB/s, 73 objects/s recovering 2024-09-16T09:32:52.869 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:52.870 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:52.870 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:32:52.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:52 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:32:52.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:52 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:32:52.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:52 smithi084 ceph-mon[32629]: from='client.25424 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:32:52.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:52 smithi084 ceph-mon[32629]: pgmap v365: 168 pgs: 168 active+clean; 2.0 GiB data, 8.9 GiB used, 706 GiB / 715 GiB avail; 0 B/s wr, 666 op/s; 4.6 MiB/s, 73 objects/s recovering 2024-09-16T09:32:52.971 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:32:53.005 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:53.006 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:53.007 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:32:53.107 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:32:53.138 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:53.139 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:53.139 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:32:53.240 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:32:53.276 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:53.277 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:53.277 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:32:53.378 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:32:53.398 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:53.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:53.399 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:32:53.410 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:32:52 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:32:53.501 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:32:53.534 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:53.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:53.535 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:32:53.636 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:32:53.668 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:53.668 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:53.668 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:32:53.769 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:32:53.803 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:53.803 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:53.804 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:32:53.905 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:32:53.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:53 smithi084 ceph-mon[32629]: osdmap e216: 8 total, 8 up, 8 in 2024-09-16T09:32:53.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:53 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:32:53.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:53 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:32:53.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:53 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:32:53.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:53 smithi070 ceph-mon[30388]: osdmap e216: 8 total, 8 up, 8 in 2024-09-16T09:32:53.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:53 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:32:53.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:53 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:32:53.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:53 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:32:53.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:53 smithi070 ceph-mon[40691]: osdmap e216: 8 total, 8 up, 8 in 2024-09-16T09:32:53.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:53 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:32:53.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:53 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:32:53.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:53 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:32:53.932 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:53.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:53.933 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:32:54.034 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:32:54.067 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:54.068 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:54.068 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:32:54.169 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:32:54.199 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:54.199 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:54.200 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:32:54.301 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:32:54.332 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:54.333 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:54.333 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:32:54.434 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:32:54.455 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:54.456 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:54.456 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:32:54.557 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:32:54.591 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:54.592 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:54.592 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:32:54.693 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:32:54.729 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:54.729 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:54.730 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:32:54.831 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:32:54.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:54.862 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:54.862 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:32:54.910 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T09:32:54.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:54 smithi084 ceph-mon[32629]: osdmap e217: 8 total, 8 up, 8 in 2024-09-16T09:32:54.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:54 smithi084 ceph-mon[32629]: pgmap v368: 168 pgs: 168 active+clean; 1.6 GiB data, 7.8 GiB used, 707 GiB / 715 GiB avail; 0 B/s wr, 1.59k op/s; 4.6 MiB/s, 73 objects/s recovering 2024-09-16T09:32:54.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:54 smithi084 ceph-mon[32629]: osdmap e218: 8 total, 8 up, 8 in 2024-09-16T09:32:54.920 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:54 smithi070 ceph-mon[40691]: osdmap e217: 8 total, 8 up, 8 in 2024-09-16T09:32:54.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:54 smithi070 ceph-mon[40691]: pgmap v368: 168 pgs: 168 active+clean; 1.6 GiB data, 7.8 GiB used, 707 GiB / 715 GiB avail; 0 B/s wr, 1.59k op/s; 4.6 MiB/s, 73 objects/s recovering 2024-09-16T09:32:54.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:54 smithi070 ceph-mon[40691]: osdmap e218: 8 total, 8 up, 8 in 2024-09-16T09:32:54.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:54 smithi070 ceph-mon[30388]: osdmap e217: 8 total, 8 up, 8 in 2024-09-16T09:32:54.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:54 smithi070 ceph-mon[30388]: pgmap v368: 168 pgs: 168 active+clean; 1.6 GiB data, 7.8 GiB used, 707 GiB / 715 GiB avail; 0 B/s wr, 1.59k op/s; 4.6 MiB/s, 73 objects/s recovering 2024-09-16T09:32:54.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:54 smithi070 ceph-mon[30388]: osdmap e218: 8 total, 8 up, 8 in 2024-09-16T09:32:54.963 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:32:55.010 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:55.011 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:55.011 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:32:55.112 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:32:55.133 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:55.133 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:55.134 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:32:55.235 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:32:55.274 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:55.275 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:55.275 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:32:55.376 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:32:55.411 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:55.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:55.412 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:32:55.513 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:32:55.541 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:55.542 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:55.542 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:32:55.643 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:32:55.683 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:55.684 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:55.684 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:32:55.785 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:32:55.823 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:55.824 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:55.824 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:32:55.925 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:32:55.960 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:55.961 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:55.961 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:32:56.049 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":364,"stamp":"2024-09-16T09:32:49.645299+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2160794163,"num_objects":33162,"num_object_clones":0,"num_object_copies":99486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33162,"num_whiteouts":0,"num_read":1176,"num_read_kb":928,"num_write":216659,"num_write_kb":7973934,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14846,"num_bytes_recovered":964827092,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79750,"ondisk_log_size":79750,"up":504,"acting":504,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":506,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":9301040,"kb_used_data":7034296,"kb_used_omap":12,"kb_used_meta":2266483,"kb_avail":740660176,"statfs":{"total":767960285184,"available":758436020224,"internally_reserved":0,"allocated":7203119104,"data_stored":7200496197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":2320879154},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-263736823,"num_objects":-4022,"num_object_clones":0,"num_object_copies":-12066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-441,"num_objects_unfound":0,"num_objects_dirty":-4022,"num_whiteouts":0,"num_read":-16,"num_read_kb":-35,"num_write":4000,"num_write_kb":-296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":440,"num_bytes_recovered":28794036,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.224622"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":357,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.470020+0000","last_change":"2024-09-16T09:32:39.384174+0000","last_active":"2024-09-16T09:32:47.470020+0000","last_peered":"2024-09-16T09:32:47.470020+0000","last_clean":"2024-09-16T09:32:47.470020+0000","last_became_active":"2024-09-16T09:25:01.709108+0000","last_became_peered":"2024-09-16T09:25:01.709108+0000","last_unstale":"2024-09-16T09:32:47.470020+0000","last_undegraded":"2024-09-16T09:32:47.470020+0000","last_fullsized":"2024-09-16T09:32:47.470020+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:39.384104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:32:39.384104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:41.986629+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"66'9","reported_seq":361,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.463341+0000","last_change":"2024-09-16T09:32:38.074301+0000","last_active":"2024-09-16T09:32:47.463341+0000","last_peered":"2024-09-16T09:32:47.463341+0000","last_clean":"2024-09-16T09:32:47.463341+0000","last_became_active":"2024-09-16T09:25:03.714635+0000","last_became_peered":"2024-09-16T09:25:03.714635+0000","last_unstale":"2024-09-16T09:32:47.463341+0000","last_undegraded":"2024-09-16T09:32:47.463341+0000","last_fullsized":"2024-09-16T09:32:47.463341+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:32:38.074231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:32:38.074231+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:04.491224+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":341,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.433637+0000","last_change":"2024-09-16T09:29:17.910367+0000","last_active":"2024-09-16T09:32:47.433637+0000","last_peered":"2024-09-16T09:32:47.433637+0000","last_clean":"2024-09-16T09:32:47.433637+0000","last_became_active":"2024-09-16T09:25:05.715645+0000","last_became_peered":"2024-09-16T09:25:05.715645+0000","last_unstale":"2024-09-16T09:32:47.433637+0000","last_undegraded":"2024-09-16T09:32:47.433637+0000","last_fullsized":"2024-09-16T09:32:47.433637+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:17.910305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:17.910305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":321,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.463526+0000","last_change":"2024-09-16T09:27:11.565306+0000","last_active":"2024-09-16T09:32:47.463526+0000","last_peered":"2024-09-16T09:32:47.463526+0000","last_clean":"2024-09-16T09:32:47.463526+0000","last_became_active":"2024-09-16T09:25:07.715442+0000","last_became_peered":"2024-09-16T09:25:07.715442+0000","last_unstale":"2024-09-16T09:32:47.463526+0000","last_undegraded":"2024-09-16T09:32:47.463526+0000","last_fullsized":"2024-09-16T09:32:47.463526+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:11.565220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:11.565220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:25.985557+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":66,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.430144+0000","last_change":"2024-09-16T09:31:09.269700+0000","last_active":"2024-09-16T09:32:47.430144+0000","last_peered":"2024-09-16T09:32:47.430144+0000","last_clean":"2024-09-16T09:32:47.430144+0000","last_became_active":"2024-09-16T09:31:09.269342+0000","last_became_peered":"2024-09-16T09:31:09.269342+0000","last_unstale":"2024-09-16T09:32:47.430144+0000","last_undegraded":"2024-09-16T09:32:47.430144+0000","last_fullsized":"2024-09-16T09:32:47.430144+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:00.068570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:28:00.068570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":206,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.468431+0000","last_change":"2024-09-16T09:31:56.507595+0000","last_active":"2024-09-16T09:32:47.468431+0000","last_peered":"2024-09-16T09:32:47.468431+0000","last_clean":"2024-09-16T09:32:47.468431+0000","last_became_active":"2024-09-16T09:31:09.275136+0000","last_became_peered":"2024-09-16T09:31:09.275136+0000","last_unstale":"2024-09-16T09:32:47.468431+0000","last_undegraded":"2024-09-16T09:32:47.468431+0000","last_fullsized":"2024-09-16T09:32:47.468431+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:31:56.507528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:31:56.507528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:08.491873+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.19","version":"66'15","reported_seq":275,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.431645+0000","last_change":"2024-09-16T09:32:45.194666+0000","last_active":"2024-09-16T09:32:47.431645+0000","last_peered":"2024-09-16T09:32:47.431645+0000","last_clean":"2024-09-16T09:32:47.431645+0000","last_became_active":"2024-09-16T09:31:09.268750+0000","last_became_peered":"2024-09-16T09:31:09.268750+0000","last_unstale":"2024-09-16T09:32:47.431645+0000","last_undegraded":"2024-09-16T09:32:47.431645+0000","last_fullsized":"2024-09-16T09:32:47.431645+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-16T09:32:45.194581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:32:45.194581+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:09.284179+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":349,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.463287+0000","last_change":"2024-09-16T09:32:44.114475+0000","last_active":"2024-09-16T09:32:47.463287+0000","last_peered":"2024-09-16T09:32:47.463287+0000","last_clean":"2024-09-16T09:32:47.463287+0000","last_became_active":"2024-09-16T09:25:05.717236+0000","last_became_peered":"2024-09-16T09:25:05.717236+0000","last_unstale":"2024-09-16T09:32:47.463287+0000","last_undegraded":"2024-09-16T09:32:47.463287+0000","last_fullsized":"2024-09-16T09:32:47.463287+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:44.114407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:32:44.114407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:59.985952+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":179,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.433883+0000","last_change":"2024-09-16T09:28:34.839909+0000","last_active":"2024-09-16T09:32:47.433883+0000","last_peered":"2024-09-16T09:32:47.433883+0000","last_clean":"2024-09-16T09:32:47.433883+0000","last_became_active":"2024-09-16T09:28:34.839435+0000","last_became_peered":"2024-09-16T09:28:34.839435+0000","last_unstale":"2024-09-16T09:32:47.433883+0000","last_undegraded":"2024-09-16T09:32:47.433883+0000","last_fullsized":"2024-09-16T09:32:47.433883+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:01.038400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:28:01.038400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"66'9","reported_seq":353,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.464164+0000","last_change":"2024-09-16T09:29:28.622647+0000","last_active":"2024-09-16T09:32:47.464164+0000","last_peered":"2024-09-16T09:32:47.464164+0000","last_clean":"2024-09-16T09:32:47.464164+0000","last_became_active":"2024-09-16T09:25:03.709147+0000","last_became_peered":"2024-09-16T09:25:03.709147+0000","last_unstale":"2024-09-16T09:32:47.464164+0000","last_undegraded":"2024-09-16T09:32:47.464164+0000","last_fullsized":"2024-09-16T09:32:47.464164+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:29:28.622558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:29:28.622558+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":324,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.431991+0000","last_change":"2024-09-16T09:27:59.080369+0000","last_active":"2024-09-16T09:32:47.431991+0000","last_peered":"2024-09-16T09:32:47.431991+0000","last_clean":"2024-09-16T09:32:47.431991+0000","last_became_active":"2024-09-16T09:25:05.718134+0000","last_became_peered":"2024-09-16T09:25:05.718134+0000","last_unstale":"2024-09-16T09:32:47.431991+0000","last_undegraded":"2024-09-16T09:32:47.431991+0000","last_fullsized":"2024-09-16T09:32:47.431991+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:59.080307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:59.080307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"66'1","reported_seq":338,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.468106+0000","last_change":"2024-09-16T09:31:58.367903+0000","last_active":"2024-09-16T09:32:47.468106+0000","last_peered":"2024-09-16T09:32:47.468106+0000","last_clean":"2024-09-16T09:32:47.468106+0000","last_became_active":"2024-09-16T09:25:07.716558+0000","last_became_peered":"2024-09-16T09:25:07.716558+0000","last_unstale":"2024-09-16T09:32:47.468106+0000","last_undegraded":"2024-09-16T09:32:47.468106+0000","last_fullsized":"2024-09-16T09:32:47.468106+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-16T09:31:58.367837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:31:58.367837+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:20.121824+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":332,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.432300+0000","last_change":"2024-09-16T09:27:56.191264+0000","last_active":"2024-09-16T09:32:47.432300+0000","last_peered":"2024-09-16T09:32:47.432300+0000","last_clean":"2024-09-16T09:32:47.432300+0000","last_became_active":"2024-09-16T09:25:01.696939+0000","last_became_peered":"2024-09-16T09:25:01.696939+0000","last_unstale":"2024-09-16T09:32:47.432300+0000","last_undegraded":"2024-09-16T09:32:47.432300+0000","last_fullsized":"2024-09-16T09:32:47.432300+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:56.191220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:27:56.191220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"66'5","reported_seq":347,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.463552+0000","last_change":"2024-09-16T09:32:31.024521+0000","last_active":"2024-09-16T09:32:47.463552+0000","last_peered":"2024-09-16T09:32:47.463552+0000","last_clean":"2024-09-16T09:32:47.463552+0000","last_became_active":"2024-09-16T09:25:03.716136+0000","last_became_peered":"2024-09-16T09:25:03.716136+0000","last_unstale":"2024-09-16T09:32:47.463552+0000","last_undegraded":"2024-09-16T09:32:47.463552+0000","last_fullsized":"2024-09-16T09:32:47.463552+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-16T09:32:31.024450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:32:31.024450+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:41.830431+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":332,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.430018+0000","last_change":"2024-09-16T09:28:52.746046+0000","last_active":"2024-09-16T09:32:47.430018+0000","last_peered":"2024-09-16T09:32:47.430018+0000","last_clean":"2024-09-16T09:32:47.430018+0000","last_became_active":"2024-09-16T09:25:05.718281+0000","last_became_peered":"2024-09-16T09:25:05.718281+0000","last_unstale":"2024-09-16T09:32:47.430018+0000","last_undegraded":"2024-09-16T09:32:47.430018+0000","last_fullsized":"2024-09-16T09:32:47.430018+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:52.745999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:28:52.745999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":350,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.432862+0000","last_change":"2024-09-16T09:31:09.276016+0000","last_active":"2024-09-16T09:32:47.432862+0000","last_peered":"2024-09-16T09:32:47.432862+0000","last_clean":"2024-09-16T09:32:47.432862+0000","last_became_active":"2024-09-16T09:31:09.275656+0000","last_became_peered":"2024-09-16T09:31:09.275656+0000","last_unstale":"2024-09-16T09:32:47.432862+0000","last_undegraded":"2024-09-16T09:32:47.432862+0000","last_fullsized":"2024-09-16T09:32:47.432862+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:50.468134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:50.468134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":337,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.463923+0000","last_change":"2024-09-16T09:29:56.605555+0000","last_active":"2024-09-16T09:32:47.463923+0000","last_peered":"2024-09-16T09:32:47.463923+0000","last_clean":"2024-09-16T09:32:47.463923+0000","last_became_active":"2024-09-16T09:25:07.720793+0000","last_became_peered":"2024-09-16T09:25:07.720793+0000","last_unstale":"2024-09-16T09:32:47.463923+0000","last_undegraded":"2024-09-16T09:32:47.463923+0000","last_fullsized":"2024-09-16T09:32:47.463923+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:56.605488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:56.605488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":88,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.467807+0000","last_change":"2024-09-16T09:30:39.202178+0000","last_active":"2024-09-16T09:32:47.467807+0000","last_peered":"2024-09-16T09:32:47.467807+0000","last_clean":"2024-09-16T09:32:47.467807+0000","last_became_active":"2024-09-16T09:30:39.201824+0000","last_became_peered":"2024-09-16T09:30:39.201824+0000","last_unstale":"2024-09-16T09:32:47.467807+0000","last_undegraded":"2024-09-16T09:32:47.467807+0000","last_fullsized":"2024-09-16T09:32:47.467807+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:50.912979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:50.912979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:23.532853+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"66'15","reported_seq":416,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.432202+0000","last_change":"2024-09-16T09:31:09.302018+0000","last_active":"2024-09-16T09:32:47.432202+0000","last_peered":"2024-09-16T09:32:47.432202+0000","last_clean":"2024-09-16T09:32:47.432202+0000","last_became_active":"2024-09-16T09:31:09.263534+0000","last_became_peered":"2024-09-16T09:31:09.263534+0000","last_unstale":"2024-09-16T09:32:47.432202+0000","last_undegraded":"2024-09-16T09:32:47.432202+0000","last_fullsized":"2024-09-16T09:32:47.432202+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-16T09:30:21.932604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:30:21.932604+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":333,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.434048+0000","last_change":"2024-09-16T09:29:55.875536+0000","last_active":"2024-09-16T09:32:47.434048+0000","last_peered":"2024-09-16T09:32:47.434048+0000","last_clean":"2024-09-16T09:32:47.434048+0000","last_became_active":"2024-09-16T09:25:05.713929+0000","last_became_peered":"2024-09-16T09:25:05.713929+0000","last_unstale":"2024-09-16T09:32:47.434048+0000","last_undegraded":"2024-09-16T09:32:47.434048+0000","last_fullsized":"2024-09-16T09:32:47.434048+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:55.875480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:55.875480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":185,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.468716+0000","last_change":"2024-09-16T09:30:24.412733+0000","last_active":"2024-09-16T09:32:47.468716+0000","last_peered":"2024-09-16T09:32:47.468716+0000","last_clean":"2024-09-16T09:32:47.468716+0000","last_became_active":"2024-09-16T09:28:52.974255+0000","last_became_peered":"2024-09-16T09:28:52.974255+0000","last_unstale":"2024-09-16T09:32:47.468716+0000","last_undegraded":"2024-09-16T09:32:47.468716+0000","last_fullsized":"2024-09-16T09:32:47.468716+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:30:24.412571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:30:24.412571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:49.424390+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":349,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.464661+0000","last_change":"2024-09-16T09:30:03.628882+0000","last_active":"2024-09-16T09:32:47.464661+0000","last_peered":"2024-09-16T09:32:47.464661+0000","last_clean":"2024-09-16T09:32:47.464661+0000","last_became_active":"2024-09-16T09:25:01.706491+0000","last_became_peered":"2024-09-16T09:25:01.706491+0000","last_unstale":"2024-09-16T09:32:47.464661+0000","last_undegraded":"2024-09-16T09:32:47.464661+0000","last_fullsized":"2024-09-16T09:32:47.464661+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:30:03.628812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:30:03.628812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"66'12","reported_seq":249,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.434246+0000","last_change":"2024-09-16T09:31:09.286673+0000","last_active":"2024-09-16T09:32:47.434246+0000","last_peered":"2024-09-16T09:32:47.434246+0000","last_clean":"2024-09-16T09:32:47.434246+0000","last_became_active":"2024-09-16T09:31:09.268588+0000","last_became_peered":"2024-09-16T09:31:09.268588+0000","last_unstale":"2024-09-16T09:32:47.434246+0000","last_undegraded":"2024-09-16T09:32:47.434246+0000","last_fullsized":"2024-09-16T09:32:47.434246+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-16T09:27:01.700139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:01.700139+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":24,"num_read_kb":15,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":325,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.464082+0000","last_change":"2024-09-16T09:27:00.816500+0000","last_active":"2024-09-16T09:32:47.464082+0000","last_peered":"2024-09-16T09:32:47.464082+0000","last_clean":"2024-09-16T09:32:47.464082+0000","last_became_active":"2024-09-16T09:25:05.710093+0000","last_became_peered":"2024-09-16T09:25:05.710093+0000","last_unstale":"2024-09-16T09:32:47.464082+0000","last_undegraded":"2024-09-16T09:32:47.464082+0000","last_fullsized":"2024-09-16T09:32:47.464082+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:00.816424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:00.816424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:28.982331+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":329,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.430221+0000","last_change":"2024-09-16T09:29:29.918433+0000","last_active":"2024-09-16T09:32:47.430221+0000","last_peered":"2024-09-16T09:32:47.430221+0000","last_clean":"2024-09-16T09:32:47.430221+0000","last_became_active":"2024-09-16T09:25:07.719380+0000","last_became_peered":"2024-09-16T09:25:07.719380+0000","last_unstale":"2024-09-16T09:32:47.430221+0000","last_undegraded":"2024-09-16T09:32:47.430221+0000","last_fullsized":"2024-09-16T09:32:47.430221+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-16T09:29:29.918368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:29.918368+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":350,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.433742+0000","last_change":"2024-09-16T09:30:02.843932+0000","last_active":"2024-09-16T09:32:47.433742+0000","last_peered":"2024-09-16T09:32:47.433742+0000","last_clean":"2024-09-16T09:32:47.433742+0000","last_became_active":"2024-09-16T09:25:01.699545+0000","last_became_peered":"2024-09-16T09:25:01.699545+0000","last_unstale":"2024-09-16T09:32:47.433742+0000","last_undegraded":"2024-09-16T09:32:47.433742+0000","last_fullsized":"2024-09-16T09:32:47.433742+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:30:02.843856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:30:02.843856+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"66'10","reported_seq":398,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.470566+0000","last_change":"2024-09-16T09:32:30.286077+0000","last_active":"2024-09-16T09:32:47.470566+0000","last_peered":"2024-09-16T09:32:47.470566+0000","last_clean":"2024-09-16T09:32:47.470566+0000","last_became_active":"2024-09-16T09:31:09.275050+0000","last_became_peered":"2024-09-16T09:31:09.275050+0000","last_unstale":"2024-09-16T09:32:47.470566+0000","last_undegraded":"2024-09-16T09:32:47.470566+0000","last_fullsized":"2024-09-16T09:32:47.470566+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-16T09:32:30.284919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:32:30.284919+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:30.659184+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"66'8","reported_seq":462,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.468862+0000","last_change":"2024-09-16T09:32:46.533273+0000","last_active":"2024-09-16T09:32:47.468862+0000","last_peered":"2024-09-16T09:32:47.468862+0000","last_clean":"2024-09-16T09:32:47.468862+0000","last_became_active":"2024-09-16T09:25:05.712516+0000","last_became_peered":"2024-09-16T09:25:05.712516+0000","last_unstale":"2024-09-16T09:32:47.468862+0000","last_undegraded":"2024-09-16T09:32:47.468862+0000","last_fullsized":"2024-09-16T09:32:47.468862+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:32:46.533204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:32:46.533204+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:58.006839+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"214'13548","reported_seq":13684,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.951026+0000","last_change":"2024-09-16T09:29:49.449011+0000","last_active":"2024-09-16T09:32:47.951026+0000","last_peered":"2024-09-16T09:32:47.951026+0000","last_clean":"2024-09-16T09:32:47.951026+0000","last_became_active":"2024-09-16T09:29:49.448473+0000","last_became_peered":"2024-09-16T09:29:49.448473+0000","last_unstale":"2024-09-16T09:32:47.951026+0000","last_undegraded":"2024-09-16T09:32:47.951026+0000","last_fullsized":"2024-09-16T09:32:47.951026+0000","mapping_epoch":152,"log_start":"197'9000","ondisk_log_start":"197'9000","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":4548,"log_dups_size":0,"ondisk_log_size":4548,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":135659520,"num_objects":2070,"num_object_clones":0,"num_object_copies":6210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2070,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13548,"num_write_kb":499776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.a","version":"73'79","reported_seq":271,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.431672+0000","last_change":"2024-09-16T09:29:18.498887+0000","last_active":"2024-09-16T09:32:47.431672+0000","last_peered":"2024-09-16T09:32:47.431672+0000","last_clean":"2024-09-16T09:32:47.431672+0000","last_became_active":"2024-09-16T09:26:35.127352+0000","last_became_peered":"2024-09-16T09:26:35.127352+0000","last_unstale":"2024-09-16T09:32:47.431672+0000","last_undegraded":"2024-09-16T09:32:47.431672+0000","last_fullsized":"2024-09-16T09:32:47.431672+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:18.498766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:18.498766+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"66'15","reported_seq":280,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.438073+0000","last_change":"2024-09-16T09:28:28.065781+0000","last_active":"2024-09-16T09:32:47.438073+0000","last_peered":"2024-09-16T09:32:47.438073+0000","last_clean":"2024-09-16T09:32:47.438073+0000","last_became_active":"2024-09-16T09:26:33.295672+0000","last_became_peered":"2024-09-16T09:26:33.295672+0000","last_unstale":"2024-09-16T09:32:47.438073+0000","last_undegraded":"2024-09-16T09:32:47.438073+0000","last_fullsized":"2024-09-16T09:32:47.438073+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-16T09:28:28.065699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:28.065699+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:23.961200+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":66,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.429459+0000","last_change":"2024-09-16T09:31:09.276858+0000","last_active":"2024-09-16T09:32:47.429459+0000","last_peered":"2024-09-16T09:32:47.429459+0000","last_clean":"2024-09-16T09:32:47.429459+0000","last_became_active":"2024-09-16T09:31:09.274534+0000","last_became_peered":"2024-09-16T09:31:09.274534+0000","last_unstale":"2024-09-16T09:32:47.429459+0000","last_undegraded":"2024-09-16T09:32:47.429459+0000","last_fullsized":"2024-09-16T09:32:47.429459+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:22.091537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:28:22.091537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"66'8","reported_seq":419,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.464056+0000","last_change":"2024-09-16T09:25:05.714128+0000","last_active":"2024-09-16T09:32:47.464056+0000","last_peered":"2024-09-16T09:32:47.464056+0000","last_clean":"2024-09-16T09:32:47.464056+0000","last_became_active":"2024-09-16T09:25:05.713813+0000","last_became_peered":"2024-09-16T09:25:05.713813+0000","last_unstale":"2024-09-16T09:32:47.464056+0000","last_undegraded":"2024-09-16T09:32:47.464056+0000","last_fullsized":"2024-09-16T09:32:47.464056+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:17.970742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":328,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.432015+0000","last_change":"2024-09-16T09:29:24.537802+0000","last_active":"2024-09-16T09:32:47.432015+0000","last_peered":"2024-09-16T09:32:47.432015+0000","last_clean":"2024-09-16T09:32:47.432015+0000","last_became_active":"2024-09-16T09:25:07.717344+0000","last_became_peered":"2024-09-16T09:25:07.717344+0000","last_unstale":"2024-09-16T09:32:47.432015+0000","last_undegraded":"2024-09-16T09:32:47.432015+0000","last_fullsized":"2024-09-16T09:32:47.432015+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:24.537731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:24.537731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.c","version":"214'13535","reported_seq":13727,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:48.180779+0000","last_change":"2024-09-16T09:32:07.676294+0000","last_active":"2024-09-16T09:32:48.180779+0000","last_peered":"2024-09-16T09:32:48.180779+0000","last_clean":"2024-09-16T09:32:48.180779+0000","last_became_active":"2024-09-16T09:29:50.007935+0000","last_became_peered":"2024-09-16T09:29:50.007935+0000","last_unstale":"2024-09-16T09:32:48.180779+0000","last_undegraded":"2024-09-16T09:32:48.180779+0000","last_fullsized":"2024-09-16T09:32:48.180779+0000","mapping_epoch":152,"log_start":"195'8400","ondisk_log_start":"195'8400","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"204'10667","last_scrub_stamp":"2024-09-16T09:32:07.676020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:32:07.676020+0000","objects_scrubbed":4920,"log_size":5135,"log_dups_size":0,"ondisk_log_size":5135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:18.894513+0000","scrub_duration":1433,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131923968,"num_objects":2013,"num_object_clones":0,"num_object_copies":6039,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2013,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13535,"num_write_kb":497536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"73'79","reported_seq":311,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.464674+0000","last_change":"2024-09-16T09:32:42.123139+0000","last_active":"2024-09-16T09:32:47.464674+0000","last_peered":"2024-09-16T09:32:47.464674+0000","last_clean":"2024-09-16T09:32:47.464674+0000","last_became_active":"2024-09-16T09:30:08.763683+0000","last_became_peered":"2024-09-16T09:30:08.763683+0000","last_unstale":"2024-09-16T09:32:47.464674+0000","last_undegraded":"2024-09-16T09:32:47.464674+0000","last_fullsized":"2024-09-16T09:32:47.464674+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:32:42.123067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:32:42.123067+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:10.308483+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"66'18","reported_seq":278,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.429802+0000","last_change":"2024-09-16T09:31:09.407700+0000","last_active":"2024-09-16T09:32:47.429802+0000","last_peered":"2024-09-16T09:32:47.429802+0000","last_clean":"2024-09-16T09:32:47.429802+0000","last_became_active":"2024-09-16T09:31:09.270434+0000","last_became_peered":"2024-09-16T09:31:09.270434+0000","last_unstale":"2024-09-16T09:32:47.429802+0000","last_undegraded":"2024-09-16T09:32:47.429802+0000","last_fullsized":"2024-09-16T09:32:47.429802+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'18","last_scrub_stamp":"2024-09-16T09:28:54.699778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:54.699778+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":33,"num_read_kb":21,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":176,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.430982+0000","last_change":"2024-09-16T09:28:58.705169+0000","last_active":"2024-09-16T09:32:47.430982+0000","last_peered":"2024-09-16T09:32:47.430982+0000","last_clean":"2024-09-16T09:32:47.430982+0000","last_became_active":"2024-09-16T09:28:52.983543+0000","last_became_peered":"2024-09-16T09:28:52.983543+0000","last_unstale":"2024-09-16T09:32:47.430982+0000","last_undegraded":"2024-09-16T09:32:47.430982+0000","last_fullsized":"2024-09-16T09:32:47.430982+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:58.705097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:28:58.705097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":355,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.463236+0000","last_change":"2024-09-16T09:31:09.269234+0000","last_active":"2024-09-16T09:32:47.463236+0000","last_peered":"2024-09-16T09:32:47.463236+0000","last_clean":"2024-09-16T09:32:47.463236+0000","last_became_active":"2024-09-16T09:31:09.268864+0000","last_became_peered":"2024-09-16T09:31:09.268864+0000","last_unstale":"2024-09-16T09:32:47.463236+0000","last_undegraded":"2024-09-16T09:32:47.463236+0000","last_fullsized":"2024-09-16T09:32:47.463236+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:50.482240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:50.482240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":343,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.463761+0000","last_change":"2024-09-16T09:32:32.060608+0000","last_active":"2024-09-16T09:32:47.463761+0000","last_peered":"2024-09-16T09:32:47.463761+0000","last_clean":"2024-09-16T09:32:47.463761+0000","last_became_active":"2024-09-16T09:31:09.269433+0000","last_became_peered":"2024-09-16T09:31:09.269433+0000","last_unstale":"2024-09-16T09:32:47.463761+0000","last_undegraded":"2024-09-16T09:32:47.463761+0000","last_fullsized":"2024-09-16T09:32:47.463761+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:32.060564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:32:32.060564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:48.641239+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.d","version":"214'13459","reported_seq":13595,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:48.014193+0000","last_change":"2024-09-16T09:29:50.010860+0000","last_active":"2024-09-16T09:32:48.014193+0000","last_peered":"2024-09-16T09:32:48.014193+0000","last_clean":"2024-09-16T09:32:48.014193+0000","last_became_active":"2024-09-16T09:29:50.010421+0000","last_became_peered":"2024-09-16T09:29:50.010421+0000","last_unstale":"2024-09-16T09:32:48.014193+0000","last_undegraded":"2024-09-16T09:32:48.014193+0000","last_fullsized":"2024-09-16T09:32:48.014193+0000","mapping_epoch":152,"log_start":"196'8700","ondisk_log_start":"196'8700","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":4759,"log_dups_size":0,"ondisk_log_size":4759,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":138018840,"num_objects":2107,"num_object_clones":0,"num_object_copies":6321,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2107,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13459,"num_write_kb":498049,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,2],"acting":[5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.4","version":"73'79","reported_seq":66,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.430406+0000","last_change":"2024-09-16T09:31:09.275499+0000","last_active":"2024-09-16T09:32:47.430406+0000","last_peered":"2024-09-16T09:32:47.430406+0000","last_clean":"2024-09-16T09:32:47.430406+0000","last_became_active":"2024-09-16T09:31:09.275139+0000","last_became_peered":"2024-09-16T09:31:09.275139+0000","last_unstale":"2024-09-16T09:32:47.430406+0000","last_undegraded":"2024-09-16T09:32:47.430406+0000","last_fullsized":"2024-09-16T09:32:47.430406+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:30:22.438378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:30:22.438378+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"66'14","reported_seq":366,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.463243+0000","last_change":"2024-09-16T09:32:34.997417+0000","last_active":"2024-09-16T09:32:47.463243+0000","last_peered":"2024-09-16T09:32:47.463243+0000","last_clean":"2024-09-16T09:32:47.463243+0000","last_became_active":"2024-09-16T09:25:03.708830+0000","last_became_peered":"2024-09-16T09:25:03.708830+0000","last_unstale":"2024-09-16T09:32:47.463243+0000","last_undegraded":"2024-09-16T09:32:47.463243+0000","last_fullsized":"2024-09-16T09:32:47.463243+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'14","last_scrub_stamp":"2024-09-16T09:32:34.997360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:32:34.997360+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:47.824918+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":358,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.470870+0000","last_change":"2024-09-16T09:31:57.373606+0000","last_active":"2024-09-16T09:32:47.470870+0000","last_peered":"2024-09-16T09:32:47.470870+0000","last_clean":"2024-09-16T09:32:47.470870+0000","last_became_active":"2024-09-16T09:25:01.702992+0000","last_became_peered":"2024-09-16T09:25:01.702992+0000","last_unstale":"2024-09-16T09:32:47.470870+0000","last_undegraded":"2024-09-16T09:32:47.470870+0000","last_fullsized":"2024-09-16T09:32:47.470870+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:31:57.373544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:31:57.373544+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:24.193683+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"187'11","reported_seq":271,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.433690+0000","last_change":"2024-09-16T09:31:09.397758+0000","last_active":"2024-09-16T09:32:47.433690+0000","last_peered":"2024-09-16T09:32:47.433690+0000","last_clean":"2024-09-16T09:32:47.433690+0000","last_became_active":"2024-09-16T09:31:09.269173+0000","last_became_peered":"2024-09-16T09:31:09.269173+0000","last_unstale":"2024-09-16T09:32:47.433690+0000","last_undegraded":"2024-09-16T09:32:47.433690+0000","last_fullsized":"2024-09-16T09:32:47.433690+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'9","last_scrub_stamp":"2024-09-16T09:29:21.898533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:21.898533+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":320,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.429953+0000","last_change":"2024-09-16T09:26:40.699888+0000","last_active":"2024-09-16T09:32:47.429953+0000","last_peered":"2024-09-16T09:32:47.429953+0000","last_clean":"2024-09-16T09:32:47.429953+0000","last_became_active":"2024-09-16T09:25:07.722017+0000","last_became_peered":"2024-09-16T09:25:07.722017+0000","last_unstale":"2024-09-16T09:32:47.429953+0000","last_undegraded":"2024-09-16T09:32:47.429953+0000","last_fullsized":"2024-09-16T09:32:47.429953+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:40.699813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:40.699813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.e","version":"214'13601","reported_seq":13775,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:48.202388+0000","last_change":"2024-09-16T09:32:35.567515+0000","last_active":"2024-09-16T09:32:48.202388+0000","last_peered":"2024-09-16T09:32:48.202388+0000","last_clean":"2024-09-16T09:32:48.202388+0000","last_became_active":"2024-09-16T09:29:49.447767+0000","last_became_peered":"2024-09-16T09:29:49.447767+0000","last_unstale":"2024-09-16T09:32:48.202388+0000","last_undegraded":"2024-09-16T09:32:48.202388+0000","last_fullsized":"2024-09-16T09:32:48.202388+0000","mapping_epoch":152,"log_start":"196'8500","ondisk_log_start":"196'8500","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"209'12756","last_scrub_stamp":"2024-09-16T09:32:35.567430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:32:35.567430+0000","objects_scrubbed":2885,"log_size":5101,"log_dups_size":0,"ondisk_log_size":5101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:51.840038+0000","scrub_duration":1201,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131399680,"num_objects":2005,"num_object_clones":0,"num_object_copies":6015,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2005,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13601,"num_write_kb":499392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"73'79","reported_seq":397,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.468612+0000","last_change":"2024-09-16T09:32:45.555132+0000","last_active":"2024-09-16T09:32:47.468612+0000","last_peered":"2024-09-16T09:32:47.468612+0000","last_clean":"2024-09-16T09:32:47.468612+0000","last_became_active":"2024-09-16T09:31:09.274707+0000","last_became_peered":"2024-09-16T09:31:09.274707+0000","last_unstale":"2024-09-16T09:32:47.468612+0000","last_undegraded":"2024-09-16T09:32:47.468612+0000","last_fullsized":"2024-09-16T09:32:47.468612+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:32:45.555067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:32:45.555067+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:56.478822+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"66'10","reported_seq":360,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.434443+0000","last_change":"2024-09-16T09:30:04.848306+0000","last_active":"2024-09-16T09:32:47.434443+0000","last_peered":"2024-09-16T09:32:47.434443+0000","last_clean":"2024-09-16T09:32:47.434443+0000","last_became_active":"2024-09-16T09:25:03.712985+0000","last_became_peered":"2024-09-16T09:25:03.712985+0000","last_unstale":"2024-09-16T09:32:47.434443+0000","last_undegraded":"2024-09-16T09:32:47.434443+0000","last_fullsized":"2024-09-16T09:32:47.434443+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-16T09:30:04.848234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:30:04.848234+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":354,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.432594+0000","last_change":"2024-09-16T09:32:46.388711+0000","last_active":"2024-09-16T09:32:47.432594+0000","last_peered":"2024-09-16T09:32:47.432594+0000","last_clean":"2024-09-16T09:32:47.432594+0000","last_became_active":"2024-09-16T09:31:09.270619+0000","last_became_peered":"2024-09-16T09:31:09.270619+0000","last_unstale":"2024-09-16T09:32:47.432594+0000","last_undegraded":"2024-09-16T09:32:47.432594+0000","last_fullsized":"2024-09-16T09:32:47.432594+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:46.388651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:32:46.388651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:12.727689+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"66'8","reported_seq":435,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.469856+0000","last_change":"2024-09-16T09:28:57.628615+0000","last_active":"2024-09-16T09:32:47.469856+0000","last_peered":"2024-09-16T09:32:47.469856+0000","last_clean":"2024-09-16T09:32:47.469856+0000","last_became_active":"2024-09-16T09:25:05.714507+0000","last_became_peered":"2024-09-16T09:25:05.714507+0000","last_unstale":"2024-09-16T09:32:47.469856+0000","last_undegraded":"2024-09-16T09:32:47.469856+0000","last_fullsized":"2024-09-16T09:32:47.469856+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:28:57.628543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:28:57.628543+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:51.253847+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":351,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.468177+0000","last_change":"2024-09-16T09:32:33.362233+0000","last_active":"2024-09-16T09:32:47.468177+0000","last_peered":"2024-09-16T09:32:47.468177+0000","last_clean":"2024-09-16T09:32:47.468177+0000","last_became_active":"2024-09-16T09:31:09.283192+0000","last_became_peered":"2024-09-16T09:31:09.283192+0000","last_unstale":"2024-09-16T09:32:47.468177+0000","last_undegraded":"2024-09-16T09:32:47.468177+0000","last_fullsized":"2024-09-16T09:32:47.468177+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:33.362188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:32:33.362188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:40.243325+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.f","version":"214'13631","reported_seq":13795,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.858183+0000","last_change":"2024-09-16T09:32:47.453692+0000","last_active":"2024-09-16T09:32:47.858183+0000","last_peered":"2024-09-16T09:32:47.858183+0000","last_clean":"2024-09-16T09:32:47.858183+0000","last_became_active":"2024-09-16T09:29:49.838146+0000","last_became_peered":"2024-09-16T09:29:49.838146+0000","last_unstale":"2024-09-16T09:32:47.858183+0000","last_undegraded":"2024-09-16T09:32:47.858183+0000","last_fullsized":"2024-09-16T09:32:47.858183+0000","mapping_epoch":152,"log_start":"192'7600","ondisk_log_start":"192'7600","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"213'13600","last_scrub_stamp":"2024-09-16T09:32:47.453582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:32:47.453582+0000","objects_scrubbed":2146,"log_size":6031,"log_dups_size":0,"ondisk_log_size":6031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:16.176992+0000","scrub_duration":239,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":138608640,"num_objects":2115,"num_object_clones":0,"num_object_copies":6345,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2115,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13631,"num_write_kb":503872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"73'79","reported_seq":370,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.468246+0000","last_change":"2024-09-16T09:31:09.274975+0000","last_active":"2024-09-16T09:32:47.468246+0000","last_peered":"2024-09-16T09:32:47.468246+0000","last_clean":"2024-09-16T09:32:47.468246+0000","last_became_active":"2024-09-16T09:31:09.274565+0000","last_became_peered":"2024-09-16T09:31:09.274565+0000","last_unstale":"2024-09-16T09:32:47.468246+0000","last_undegraded":"2024-09-16T09:32:47.468246+0000","last_fullsized":"2024-09-16T09:32:47.468246+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:30:22.415051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:30:22.415051+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:52.385618+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"66'19","reported_seq":373,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.470060+0000","last_change":"2024-09-16T09:31:35.249952+0000","last_active":"2024-09-16T09:32:47.470060+0000","last_peered":"2024-09-16T09:32:47.470060+0000","last_clean":"2024-09-16T09:32:47.470060+0000","last_became_active":"2024-09-16T09:25:03.714797+0000","last_became_peered":"2024-09-16T09:25:03.714797+0000","last_unstale":"2024-09-16T09:32:47.470060+0000","last_undegraded":"2024-09-16T09:32:47.470060+0000","last_fullsized":"2024-09-16T09:32:47.470060+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-16T09:31:35.249864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:31:35.249864+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":349,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.434786+0000","last_change":"2024-09-16T09:29:16.888357+0000","last_active":"2024-09-16T09:32:47.434786+0000","last_peered":"2024-09-16T09:32:47.434786+0000","last_clean":"2024-09-16T09:32:47.434786+0000","last_became_active":"2024-09-16T09:25:01.706806+0000","last_became_peered":"2024-09-16T09:25:01.706806+0000","last_unstale":"2024-09-16T09:32:47.434786+0000","last_undegraded":"2024-09-16T09:32:47.434786+0000","last_fullsized":"2024-09-16T09:32:47.434786+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:16.888236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:16.888236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":349,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.467483+0000","last_change":"2024-09-16T09:31:55.456669+0000","last_active":"2024-09-16T09:32:47.467483+0000","last_peered":"2024-09-16T09:32:47.467483+0000","last_clean":"2024-09-16T09:32:47.467483+0000","last_became_active":"2024-09-16T09:25:05.708663+0000","last_became_peered":"2024-09-16T09:25:05.708663+0000","last_unstale":"2024-09-16T09:32:47.467483+0000","last_undegraded":"2024-09-16T09:32:47.467483+0000","last_fullsized":"2024-09-16T09:32:47.467483+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:31:55.456592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:31:55.456592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:56.006473+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":321,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.433582+0000","last_change":"2024-09-16T09:26:37.282798+0000","last_active":"2024-09-16T09:32:47.433582+0000","last_peered":"2024-09-16T09:32:47.433582+0000","last_clean":"2024-09-16T09:32:47.433582+0000","last_became_active":"2024-09-16T09:25:07.719267+0000","last_became_peered":"2024-09-16T09:25:07.719267+0000","last_unstale":"2024-09-16T09:32:47.433582+0000","last_undegraded":"2024-09-16T09:32:47.433582+0000","last_fullsized":"2024-09-16T09:32:47.433582+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:37.282735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:37.282735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.8","version":"214'13680","reported_seq":22745,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:48.008234+0000","last_change":"2024-09-16T09:32:46.470436+0000","last_active":"2024-09-16T09:32:48.008234+0000","last_peered":"2024-09-16T09:32:48.008234+0000","last_clean":"2024-09-16T09:32:48.008234+0000","last_became_active":"2024-09-16T09:32:46.435851+0000","last_became_peered":"2024-09-16T09:32:46.435851+0000","last_unstale":"2024-09-16T09:32:48.008234+0000","last_undegraded":"2024-09-16T09:32:48.008234+0000","last_fullsized":"2024-09-16T09:32:48.008234+0000","mapping_epoch":212,"log_start":"196'8900","ondisk_log_start":"196'8900","created":152,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":4780,"log_dups_size":0,"ondisk_log_size":4780,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":133824512,"num_objects":2042,"num_object_clones":0,"num_object_copies":6126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2042,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13680,"num_write_kb":503104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4456,"num_bytes_recovered":292028416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1","version":"73'79","reported_seq":303,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.432321+0000","last_change":"2024-09-16T09:32:43.420831+0000","last_active":"2024-09-16T09:32:47.432321+0000","last_peered":"2024-09-16T09:32:47.432321+0000","last_clean":"2024-09-16T09:32:47.432321+0000","last_became_active":"2024-09-16T09:30:08.766350+0000","last_became_peered":"2024-09-16T09:30:08.766350+0000","last_unstale":"2024-09-16T09:32:47.432321+0000","last_undegraded":"2024-09-16T09:32:47.432321+0000","last_fullsized":"2024-09-16T09:32:47.432321+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:32:43.420784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:32:43.420784+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:55.813620+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"187'30","reported_seq":537,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.434651+0000","last_change":"2024-09-16T09:31:09.323281+0000","last_active":"2024-09-16T09:32:47.434651+0000","last_peered":"2024-09-16T09:32:47.434651+0000","last_clean":"2024-09-16T09:32:47.434651+0000","last_became_active":"2024-09-16T09:31:09.278839+0000","last_became_peered":"2024-09-16T09:31:09.278839+0000","last_unstale":"2024-09-16T09:32:47.434651+0000","last_undegraded":"2024-09-16T09:32:47.434651+0000","last_fullsized":"2024-09-16T09:32:47.434651+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'28","last_scrub_stamp":"2024-09-16T09:29:18.937623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:29:18.937623+0000","objects_scrubbed":10,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":341,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.464085+0000","last_change":"2024-09-16T09:29:26.637070+0000","last_active":"2024-09-16T09:32:47.464085+0000","last_peered":"2024-09-16T09:32:47.464085+0000","last_clean":"2024-09-16T09:32:47.464085+0000","last_became_active":"2024-09-16T09:25:01.703076+0000","last_became_peered":"2024-09-16T09:25:01.703076+0000","last_unstale":"2024-09-16T09:32:47.464085+0000","last_undegraded":"2024-09-16T09:32:47.464085+0000","last_fullsized":"2024-09-16T09:32:47.464085+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:26.637027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:26.637027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":350,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.432727+0000","last_change":"2024-09-16T09:29:29.486284+0000","last_active":"2024-09-16T09:32:47.432727+0000","last_peered":"2024-09-16T09:32:47.432727+0000","last_clean":"2024-09-16T09:32:47.432727+0000","last_became_active":"2024-09-16T09:25:00.696237+0000","last_became_peered":"2024-09-16T09:25:00.696237+0000","last_unstale":"2024-09-16T09:32:47.432727+0000","last_undegraded":"2024-09-16T09:32:47.432727+0000","last_fullsized":"2024-09-16T09:32:47.432727+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-16T09:29:29.486188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:29:29.486188+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.0025137430000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":317,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.469655+0000","last_change":"2024-09-16T09:25:05.718319+0000","last_active":"2024-09-16T09:32:47.469655+0000","last_peered":"2024-09-16T09:32:47.469655+0000","last_clean":"2024-09-16T09:32:47.469655+0000","last_became_active":"2024-09-16T09:25:05.717591+0000","last_became_peered":"2024-09-16T09:25:05.717591+0000","last_unstale":"2024-09-16T09:32:47.469655+0000","last_undegraded":"2024-09-16T09:32:47.469655+0000","last_fullsized":"2024-09-16T09:32:47.469655+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:11.173707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":66,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.430618+0000","last_change":"2024-09-16T09:31:09.272967+0000","last_active":"2024-09-16T09:32:47.430618+0000","last_peered":"2024-09-16T09:32:47.430618+0000","last_clean":"2024-09-16T09:32:47.430618+0000","last_became_active":"2024-09-16T09:31:09.272609+0000","last_became_peered":"2024-09-16T09:31:09.272609+0000","last_unstale":"2024-09-16T09:32:47.430618+0000","last_undegraded":"2024-09-16T09:32:47.430618+0000","last_fullsized":"2024-09-16T09:32:47.430618+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:45.201503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:45.201503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"214'13430","reported_seq":24686,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.796111+0000","last_change":"2024-09-16T09:32:08.439285+0000","last_active":"2024-09-16T09:32:47.796111+0000","last_peered":"2024-09-16T09:32:47.796111+0000","last_clean":"2024-09-16T09:32:47.796111+0000","last_became_active":"2024-09-16T09:32:08.422751+0000","last_became_peered":"2024-09-16T09:32:08.422751+0000","last_unstale":"2024-09-16T09:32:47.796111+0000","last_undegraded":"2024-09-16T09:32:47.796111+0000","last_fullsized":"2024-09-16T09:32:47.796111+0000","mapping_epoch":204,"log_start":"197'9200","ondisk_log_start":"197'9200","created":152,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":4230,"log_dups_size":0,"ondisk_log_size":4230,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":137887744,"num_objects":2104,"num_object_clones":0,"num_object_copies":6312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2104,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13430,"num_write_kb":497088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5551,"num_bytes_recovered":363790336,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.2","version":"73'79","reported_seq":429,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.430262+0000","last_change":"2024-09-16T09:32:42.498732+0000","last_active":"2024-09-16T09:32:47.430262+0000","last_peered":"2024-09-16T09:32:47.430262+0000","last_clean":"2024-09-16T09:32:47.430262+0000","last_became_active":"2024-09-16T09:30:08.765943+0000","last_became_peered":"2024-09-16T09:30:08.765943+0000","last_unstale":"2024-09-16T09:32:47.430262+0000","last_undegraded":"2024-09-16T09:32:47.430262+0000","last_fullsized":"2024-09-16T09:32:47.430262+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:32:42.498658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:32:42.498658+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:09.187680+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"66'13","reported_seq":364,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.434975+0000","last_change":"2024-09-16T09:27:44.156057+0000","last_active":"2024-09-16T09:32:47.434975+0000","last_peered":"2024-09-16T09:32:47.434975+0000","last_clean":"2024-09-16T09:32:47.434975+0000","last_became_active":"2024-09-16T09:25:03.713241+0000","last_became_peered":"2024-09-16T09:25:03.713241+0000","last_unstale":"2024-09-16T09:32:47.434975+0000","last_undegraded":"2024-09-16T09:32:47.434975+0000","last_fullsized":"2024-09-16T09:32:47.434975+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'13","last_scrub_stamp":"2024-09-16T09:27:44.155969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:44.155969+0000","objects_scrubbed":5,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":29,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":341,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.434733+0000","last_change":"2024-09-16T09:29:50.827103+0000","last_active":"2024-09-16T09:32:47.434733+0000","last_peered":"2024-09-16T09:32:47.434733+0000","last_clean":"2024-09-16T09:32:47.434733+0000","last_became_active":"2024-09-16T09:25:01.706828+0000","last_became_peered":"2024-09-16T09:25:01.706828+0000","last_unstale":"2024-09-16T09:32:47.434733+0000","last_undegraded":"2024-09-16T09:32:47.434733+0000","last_fullsized":"2024-09-16T09:32:47.434733+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:50.827057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:50.827057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":380,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.431749+0000","last_change":"2024-09-16T09:32:40.165819+0000","last_active":"2024-09-16T09:32:47.431749+0000","last_peered":"2024-09-16T09:32:47.431749+0000","last_clean":"2024-09-16T09:32:47.431749+0000","last_became_active":"2024-09-16T09:31:09.266338+0000","last_became_peered":"2024-09-16T09:31:09.266338+0000","last_unstale":"2024-09-16T09:32:47.431749+0000","last_undegraded":"2024-09-16T09:32:47.431749+0000","last_fullsized":"2024-09-16T09:32:47.431749+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:32:40.165749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:32:40.165749+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:47.551824+0000","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0.00055022299999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":349,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.432925+0000","last_change":"2024-09-16T09:32:32.029895+0000","last_active":"2024-09-16T09:32:47.432925+0000","last_peered":"2024-09-16T09:32:47.432925+0000","last_clean":"2024-09-16T09:32:47.432925+0000","last_became_active":"2024-09-16T09:25:05.711539+0000","last_became_peered":"2024-09-16T09:25:05.711539+0000","last_unstale":"2024-09-16T09:32:47.432925+0000","last_undegraded":"2024-09-16T09:32:47.432925+0000","last_fullsized":"2024-09-16T09:32:47.432925+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:32.029829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:32:32.029829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:36.831098+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":328,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.430577+0000","last_change":"2024-09-16T09:29:22.968971+0000","last_active":"2024-09-16T09:32:47.430577+0000","last_peered":"2024-09-16T09:32:47.430577+0000","last_clean":"2024-09-16T09:32:47.430577+0000","last_became_active":"2024-09-16T09:25:07.723282+0000","last_became_peered":"2024-09-16T09:25:07.723282+0000","last_unstale":"2024-09-16T09:32:47.430577+0000","last_undegraded":"2024-09-16T09:32:47.430577+0000","last_fullsized":"2024-09-16T09:32:47.430577+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:22.968908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:22.968908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"214'13528","reported_seq":13664,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.950931+0000","last_change":"2024-09-16T09:29:49.448898+0000","last_active":"2024-09-16T09:32:47.950931+0000","last_peered":"2024-09-16T09:32:47.950931+0000","last_clean":"2024-09-16T09:32:47.950931+0000","last_became_active":"2024-09-16T09:29:49.448628+0000","last_became_peered":"2024-09-16T09:29:49.448628+0000","last_unstale":"2024-09-16T09:32:47.950931+0000","last_undegraded":"2024-09-16T09:32:47.950931+0000","last_fullsized":"2024-09-16T09:32:47.950931+0000","mapping_epoch":152,"log_start":"197'9000","ondisk_log_start":"197'9000","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":4528,"log_dups_size":0,"ondisk_log_size":4528,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":138018816,"num_objects":2106,"num_object_clones":0,"num_object_copies":6318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2106,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13528,"num_write_kb":500288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"73'79","reported_seq":302,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.434486+0000","last_change":"2024-09-16T09:31:40.212968+0000","last_active":"2024-09-16T09:32:47.434486+0000","last_peered":"2024-09-16T09:32:47.434486+0000","last_clean":"2024-09-16T09:32:47.434486+0000","last_became_active":"2024-09-16T09:30:08.877110+0000","last_became_peered":"2024-09-16T09:30:08.877110+0000","last_unstale":"2024-09-16T09:32:47.434486+0000","last_undegraded":"2024-09-16T09:32:47.434486+0000","last_fullsized":"2024-09-16T09:32:47.434486+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:31:40.212889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:31:40.212889+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"66'12","reported_seq":347,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.470659+0000","last_change":"2024-09-16T09:29:02.713335+0000","last_active":"2024-09-16T09:32:47.470659+0000","last_peered":"2024-09-16T09:32:47.470659+0000","last_clean":"2024-09-16T09:32:47.470659+0000","last_became_active":"2024-09-16T09:25:03.715315+0000","last_became_peered":"2024-09-16T09:25:03.715315+0000","last_unstale":"2024-09-16T09:32:47.470659+0000","last_undegraded":"2024-09-16T09:32:47.470659+0000","last_fullsized":"2024-09-16T09:32:47.470659+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-16T09:29:02.713259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:29:02.713259+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":26,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"59'1","reported_seq":364,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.469227+0000","last_change":"2024-09-16T09:31:09.286309+0000","last_active":"2024-09-16T09:32:47.469227+0000","last_peered":"2024-09-16T09:32:47.469227+0000","last_clean":"2024-09-16T09:32:47.469227+0000","last_became_active":"2024-09-16T09:31:09.269627+0000","last_became_peered":"2024-09-16T09:31:09.269627+0000","last_unstale":"2024-09-16T09:32:47.469227+0000","last_undegraded":"2024-09-16T09:32:47.469227+0000","last_fullsized":"2024-09-16T09:32:47.469227+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:27:00.666673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:27:00.666673+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:21.192318+0000","scrub_duration":28,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1986,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"67'5","reported_seq":811,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.429928+0000","last_change":"2024-09-16T09:29:19.994791+0000","last_active":"2024-09-16T09:32:47.429928+0000","last_peered":"2024-09-16T09:32:47.429928+0000","last_clean":"2024-09-16T09:32:47.429928+0000","last_became_active":"2024-09-16T09:25:00.696546+0000","last_became_peered":"2024-09-16T09:25:00.696546+0000","last_unstale":"2024-09-16T09:32:47.429928+0000","last_undegraded":"2024-09-16T09:32:47.429928+0000","last_fullsized":"2024-09-16T09:32:47.429928+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'5","last_scrub_stamp":"2024-09-16T09:29:19.994716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:29:19.994716+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0.0059770420000000001,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":460,"num_read_kb":455,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":324,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.432935+0000","last_change":"2024-09-16T09:27:07.337170+0000","last_active":"2024-09-16T09:32:47.432935+0000","last_peered":"2024-09-16T09:32:47.432935+0000","last_clean":"2024-09-16T09:32:47.432935+0000","last_became_active":"2024-09-16T09:25:05.714704+0000","last_became_peered":"2024-09-16T09:25:05.714704+0000","last_unstale":"2024-09-16T09:32:47.432935+0000","last_undegraded":"2024-09-16T09:32:47.432935+0000","last_fullsized":"2024-09-16T09:32:47.432935+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:07.337124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:07.337124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":343,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.434331+0000","last_change":"2024-09-16T09:32:46.252747+0000","last_active":"2024-09-16T09:32:47.434331+0000","last_peered":"2024-09-16T09:32:47.434331+0000","last_clean":"2024-09-16T09:32:47.434331+0000","last_became_active":"2024-09-16T09:31:09.271111+0000","last_became_peered":"2024-09-16T09:31:09.271111+0000","last_unstale":"2024-09-16T09:32:47.434331+0000","last_undegraded":"2024-09-16T09:32:47.434331+0000","last_fullsized":"2024-09-16T09:32:47.434331+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:46.252700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:32:46.252700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:01.656018+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"214'13477","reported_seq":13678,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.570876+0000","last_change":"2024-09-16T09:30:56.023710+0000","last_active":"2024-09-16T09:32:47.570876+0000","last_peered":"2024-09-16T09:32:47.570876+0000","last_clean":"2024-09-16T09:32:47.570876+0000","last_became_active":"2024-09-16T09:29:49.450643+0000","last_became_peered":"2024-09-16T09:29:49.450643+0000","last_unstale":"2024-09-16T09:32:47.570876+0000","last_undegraded":"2024-09-16T09:32:47.570876+0000","last_fullsized":"2024-09-16T09:32:47.570876+0000","mapping_epoch":152,"log_start":"195'8200","ondisk_log_start":"195'8200","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"181'5780","last_scrub_stamp":"2024-09-16T09:30:56.023622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:30:56.023622+0000","objects_scrubbed":5716,"log_size":5277,"log_dups_size":0,"ondisk_log_size":5277,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:14.532550+0000","scrub_duration":1526,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":136773632,"num_objects":2087,"num_object_clones":0,"num_object_copies":6261,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2087,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13477,"num_write_kb":498048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"66'16","reported_seq":461,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.468197+0000","last_change":"2024-09-16T09:32:01.564577+0000","last_active":"2024-09-16T09:32:47.468197+0000","last_peered":"2024-09-16T09:32:47.468197+0000","last_clean":"2024-09-16T09:32:47.468197+0000","last_became_active":"2024-09-16T09:25:03.714118+0000","last_became_peered":"2024-09-16T09:25:03.714118+0000","last_unstale":"2024-09-16T09:32:47.468197+0000","last_undegraded":"2024-09-16T09:32:47.468197+0000","last_fullsized":"2024-09-16T09:32:47.468197+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'16","last_scrub_stamp":"2024-09-16T09:32:01.564489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:32:01.564489+0000","objects_scrubbed":8,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:10.625142+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":182,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.432592+0000","last_change":"2024-09-16T09:31:09.270603+0000","last_active":"2024-09-16T09:32:47.432592+0000","last_peered":"2024-09-16T09:32:47.432592+0000","last_clean":"2024-09-16T09:32:47.432592+0000","last_became_active":"2024-09-16T09:31:09.270196+0000","last_became_peered":"2024-09-16T09:31:09.270196+0000","last_unstale":"2024-09-16T09:32:47.432592+0000","last_undegraded":"2024-09-16T09:32:47.432592+0000","last_fullsized":"2024-09-16T09:32:47.432592+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:49.195331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:27:49.195331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"73'79","reported_seq":428,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.430050+0000","last_change":"2024-09-16T09:32:43.535041+0000","last_active":"2024-09-16T09:32:47.430050+0000","last_peered":"2024-09-16T09:32:47.430050+0000","last_clean":"2024-09-16T09:32:47.430050+0000","last_became_active":"2024-09-16T09:30:08.870394+0000","last_became_peered":"2024-09-16T09:30:08.870394+0000","last_unstale":"2024-09-16T09:32:47.430050+0000","last_undegraded":"2024-09-16T09:32:47.430050+0000","last_fullsized":"2024-09-16T09:32:47.430050+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:32:43.534962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:32:43.534962+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:46.508762+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":324,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.430618+0000","last_change":"2024-09-16T09:26:39.699133+0000","last_active":"2024-09-16T09:32:47.430618+0000","last_peered":"2024-09-16T09:32:47.430618+0000","last_clean":"2024-09-16T09:32:47.430618+0000","last_became_active":"2024-09-16T09:25:05.714341+0000","last_became_peered":"2024-09-16T09:25:05.714341+0000","last_unstale":"2024-09-16T09:32:47.430618+0000","last_undegraded":"2024-09-16T09:32:47.430618+0000","last_fullsized":"2024-09-16T09:32:47.430618+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:39.699059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:26:39.699059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":328,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.432804+0000","last_change":"2024-09-16T09:29:00.243528+0000","last_active":"2024-09-16T09:32:47.432804+0000","last_peered":"2024-09-16T09:32:47.432804+0000","last_clean":"2024-09-16T09:32:47.432804+0000","last_became_active":"2024-09-16T09:25:07.714281+0000","last_became_peered":"2024-09-16T09:25:07.714281+0000","last_unstale":"2024-09-16T09:32:47.432804+0000","last_undegraded":"2024-09-16T09:32:47.432804+0000","last_fullsized":"2024-09-16T09:32:47.432804+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:00.243482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:00.243482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.b","version":"73'79","reported_seq":314,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.433850+0000","last_change":"2024-09-16T09:32:44.255969+0000","last_active":"2024-09-16T09:32:47.433850+0000","last_peered":"2024-09-16T09:32:47.433850+0000","last_clean":"2024-09-16T09:32:47.433850+0000","last_became_active":"2024-09-16T09:30:10.784688+0000","last_became_peered":"2024-09-16T09:30:10.784688+0000","last_unstale":"2024-09-16T09:32:47.433850+0000","last_undegraded":"2024-09-16T09:32:47.433850+0000","last_fullsized":"2024-09-16T09:32:47.433850+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:32:44.255693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:32:44.255693+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:57.310805+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,5],"acting":[1,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2","version":"214'13264","reported_seq":13400,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.772280+0000","last_change":"2024-09-16T09:29:49.445324+0000","last_active":"2024-09-16T09:32:47.772280+0000","last_peered":"2024-09-16T09:32:47.772280+0000","last_clean":"2024-09-16T09:32:47.772280+0000","last_became_active":"2024-09-16T09:29:49.445022+0000","last_became_peered":"2024-09-16T09:29:49.445022+0000","last_unstale":"2024-09-16T09:32:47.772280+0000","last_undegraded":"2024-09-16T09:32:47.772280+0000","last_fullsized":"2024-09-16T09:32:47.772280+0000","mapping_epoch":152,"log_start":"197'9000","ondisk_log_start":"197'9000","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":4264,"log_dups_size":0,"ondisk_log_size":4264,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":129236992,"num_objects":1972,"num_object_clones":0,"num_object_copies":5916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1972,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13264,"num_write_kb":487552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"66'11","reported_seq":367,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.464157+0000","last_change":"2024-09-16T09:30:05.646701+0000","last_active":"2024-09-16T09:32:47.464157+0000","last_peered":"2024-09-16T09:32:47.464157+0000","last_clean":"2024-09-16T09:32:47.464157+0000","last_became_active":"2024-09-16T09:26:33.287399+0000","last_became_peered":"2024-09-16T09:26:33.287399+0000","last_unstale":"2024-09-16T09:32:47.464157+0000","last_undegraded":"2024-09-16T09:32:47.464157+0000","last_fullsized":"2024-09-16T09:32:47.464157+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-16T09:30:05.646500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:30:05.646500+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":325,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.463720+0000","last_change":"2024-09-16T09:25:01.708568+0000","last_active":"2024-09-16T09:32:47.463720+0000","last_peered":"2024-09-16T09:32:47.463720+0000","last_clean":"2024-09-16T09:32:47.463720+0000","last_became_active":"2024-09-16T09:25:01.708270+0000","last_became_peered":"2024-09-16T09:25:01.708270+0000","last_unstale":"2024-09-16T09:32:47.463720+0000","last_undegraded":"2024-09-16T09:32:47.463720+0000","last_fullsized":"2024-09-16T09:32:47.463720+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:10.865036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":324,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.432620+0000","last_change":"2024-09-16T09:26:34.171190+0000","last_active":"2024-09-16T09:32:47.432620+0000","last_peered":"2024-09-16T09:32:47.432620+0000","last_clean":"2024-09-16T09:32:47.432620+0000","last_became_active":"2024-09-16T09:25:05.711964+0000","last_became_peered":"2024-09-16T09:25:05.711964+0000","last_unstale":"2024-09-16T09:32:47.432620+0000","last_undegraded":"2024-09-16T09:32:47.432620+0000","last_fullsized":"2024-09-16T09:32:47.432620+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:34.171139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:26:34.171139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":207,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.467893+0000","last_change":"2024-09-16T09:32:00.526741+0000","last_active":"2024-09-16T09:32:47.467893+0000","last_peered":"2024-09-16T09:32:47.467893+0000","last_clean":"2024-09-16T09:32:47.467893+0000","last_became_active":"2024-09-16T09:31:09.278618+0000","last_became_peered":"2024-09-16T09:31:09.278618+0000","last_unstale":"2024-09-16T09:32:47.467893+0000","last_undegraded":"2024-09-16T09:32:47.467893+0000","last_fullsized":"2024-09-16T09:32:47.467893+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:00.526674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:32:00.526674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:25.768941+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1","version":"214'13214","reported_seq":13350,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.930656+0000","last_change":"2024-09-16T09:29:49.449796+0000","last_active":"2024-09-16T09:32:47.930656+0000","last_peered":"2024-09-16T09:32:47.930656+0000","last_clean":"2024-09-16T09:32:47.930656+0000","last_became_active":"2024-09-16T09:29:49.449475+0000","last_became_peered":"2024-09-16T09:29:49.449475+0000","last_unstale":"2024-09-16T09:32:47.930656+0000","last_undegraded":"2024-09-16T09:32:47.930656+0000","last_fullsized":"2024-09-16T09:32:47.930656+0000","mapping_epoch":152,"log_start":"197'8700","ondisk_log_start":"197'8700","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":4514,"log_dups_size":0,"ondisk_log_size":4514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":133955584,"num_objects":2044,"num_object_clones":0,"num_object_copies":6132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2044,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13214,"num_write_kb":488256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.8","version":"73'79","reported_seq":303,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.432220+0000","last_change":"2024-09-16T09:32:07.068820+0000","last_active":"2024-09-16T09:32:47.432220+0000","last_peered":"2024-09-16T09:32:47.432220+0000","last_clean":"2024-09-16T09:32:47.432220+0000","last_became_active":"2024-09-16T09:31:09.406638+0000","last_became_peered":"2024-09-16T09:31:09.406638+0000","last_unstale":"2024-09-16T09:32:47.432220+0000","last_undegraded":"2024-09-16T09:32:47.432220+0000","last_fullsized":"2024-09-16T09:32:47.432220+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:17.482769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:17.482769+0000","objects_scrubbed":1,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262688,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":525376,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"66'17","reported_seq":373,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.464575+0000","last_change":"2024-09-16T09:30:04.636115+0000","last_active":"2024-09-16T09:32:47.464575+0000","last_peered":"2024-09-16T09:32:47.464575+0000","last_clean":"2024-09-16T09:32:47.464575+0000","last_became_active":"2024-09-16T09:25:03.713358+0000","last_became_peered":"2024-09-16T09:25:03.713358+0000","last_unstale":"2024-09-16T09:32:47.464575+0000","last_undegraded":"2024-09-16T09:32:47.464575+0000","last_fullsized":"2024-09-16T09:32:47.464575+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'17","last_scrub_stamp":"2024-09-16T09:30:04.636039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:30:04.636039+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":333,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.467710+0000","last_change":"2024-09-16T09:26:04.104188+0000","last_active":"2024-09-16T09:32:47.467710+0000","last_peered":"2024-09-16T09:32:47.467710+0000","last_clean":"2024-09-16T09:32:47.467710+0000","last_became_active":"2024-09-16T09:25:01.703355+0000","last_became_peered":"2024-09-16T09:25:01.703355+0000","last_unstale":"2024-09-16T09:32:47.467710+0000","last_undegraded":"2024-09-16T09:32:47.467710+0000","last_fullsized":"2024-09-16T09:32:47.467710+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:04.104130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:04.104130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:48.432563+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":341,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.434426+0000","last_change":"2024-09-16T09:29:58.839132+0000","last_active":"2024-09-16T09:32:47.434426+0000","last_peered":"2024-09-16T09:32:47.434426+0000","last_clean":"2024-09-16T09:32:47.434426+0000","last_became_active":"2024-09-16T09:25:05.716900+0000","last_became_peered":"2024-09-16T09:25:05.716900+0000","last_unstale":"2024-09-16T09:32:47.434426+0000","last_undegraded":"2024-09-16T09:32:47.434426+0000","last_fullsized":"2024-09-16T09:32:47.434426+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:58.839051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:58.839051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":351,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.438163+0000","last_change":"2024-09-16T09:31:09.276869+0000","last_active":"2024-09-16T09:32:47.438163+0000","last_peered":"2024-09-16T09:32:47.438163+0000","last_clean":"2024-09-16T09:32:47.438163+0000","last_became_active":"2024-09-16T09:31:09.276275+0000","last_became_peered":"2024-09-16T09:31:09.276275+0000","last_unstale":"2024-09-16T09:32:47.438163+0000","last_undegraded":"2024-09-16T09:32:47.438163+0000","last_fullsized":"2024-09-16T09:32:47.438163+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:30:03.946025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:30:03.946025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.0","version":"214'13524","reported_seq":23083,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:48.004136+0000","last_change":"2024-09-16T09:32:47.458790+0000","last_active":"2024-09-16T09:32:48.004136+0000","last_peered":"2024-09-16T09:32:48.004136+0000","last_clean":"2024-09-16T09:32:48.004136+0000","last_became_active":"2024-09-16T09:32:47.435160+0000","last_became_peered":"2024-09-16T09:32:47.435160+0000","last_unstale":"2024-09-16T09:32:48.004136+0000","last_undegraded":"2024-09-16T09:32:48.004136+0000","last_fullsized":"2024-09-16T09:32:48.004136+0000","mapping_epoch":213,"log_start":"197'8800","ondisk_log_start":"197'8800","created":152,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":4724,"log_dups_size":0,"ondisk_log_size":4724,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131858432,"num_objects":2012,"num_object_clones":0,"num_object_copies":6036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2012,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13524,"num_write_kb":497152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4702,"num_bytes_recovered":308150272,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.9","version":"73'79","reported_seq":404,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.430057+0000","last_change":"2024-09-16T09:29:19.013948+0000","last_active":"2024-09-16T09:32:47.430057+0000","last_peered":"2024-09-16T09:32:47.430057+0000","last_clean":"2024-09-16T09:32:47.430057+0000","last_became_active":"2024-09-16T09:26:35.126066+0000","last_became_peered":"2024-09-16T09:26:35.126066+0000","last_unstale":"2024-09-16T09:32:47.430057+0000","last_undegraded":"2024-09-16T09:32:47.430057+0000","last_fullsized":"2024-09-16T09:32:47.430057+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:19.013865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:19.013865+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"66'10","reported_seq":336,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.464247+0000","last_change":"2024-09-16T09:25:03.708144+0000","last_active":"2024-09-16T09:32:47.464247+0000","last_peered":"2024-09-16T09:32:47.464247+0000","last_clean":"2024-09-16T09:32:47.464247+0000","last_became_active":"2024-09-16T09:25:03.707723+0000","last_became_peered":"2024-09-16T09:25:03.707723+0000","last_unstale":"2024-09-16T09:32:47.464247+0000","last_undegraded":"2024-09-16T09:32:47.464247+0000","last_fullsized":"2024-09-16T09:32:47.464247+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:13.930015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":198,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.470216+0000","last_change":"2024-09-16T09:31:32.253397+0000","last_active":"2024-09-16T09:32:47.470216+0000","last_peered":"2024-09-16T09:32:47.470216+0000","last_clean":"2024-09-16T09:32:47.470216+0000","last_became_active":"2024-09-16T09:31:09.288825+0000","last_became_peered":"2024-09-16T09:31:09.288825+0000","last_unstale":"2024-09-16T09:32:47.470216+0000","last_undegraded":"2024-09-16T09:32:47.470216+0000","last_fullsized":"2024-09-16T09:32:47.470216+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:31:32.253306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:31:32.253306+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":454,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.432047+0000","last_change":"2024-09-16T09:32:42.175655+0000","last_active":"2024-09-16T09:32:47.432047+0000","last_peered":"2024-09-16T09:32:47.432047+0000","last_clean":"2024-09-16T09:32:47.432047+0000","last_became_active":"2024-09-16T09:25:05.711590+0000","last_became_peered":"2024-09-16T09:25:05.711590+0000","last_unstale":"2024-09-16T09:32:47.432047+0000","last_undegraded":"2024-09-16T09:32:47.432047+0000","last_fullsized":"2024-09-16T09:32:47.432047+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:32:42.175528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:32:42.175528+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:44.416122+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":345,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.463888+0000","last_change":"2024-09-16T09:32:39.072795+0000","last_active":"2024-09-16T09:32:47.463888+0000","last_peered":"2024-09-16T09:32:47.463888+0000","last_clean":"2024-09-16T09:32:47.463888+0000","last_became_active":"2024-09-16T09:25:07.718958+0000","last_became_peered":"2024-09-16T09:25:07.718958+0000","last_unstale":"2024-09-16T09:32:47.463888+0000","last_undegraded":"2024-09-16T09:32:47.463888+0000","last_fullsized":"2024-09-16T09:32:47.463888+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:39.072735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:32:39.072735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:41.434371+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.e","version":"73'79","reported_seq":138,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.463767+0000","last_change":"2024-09-16T09:32:43.092027+0000","last_active":"2024-09-16T09:32:47.463767+0000","last_peered":"2024-09-16T09:32:47.463767+0000","last_clean":"2024-09-16T09:32:47.463767+0000","last_became_active":"2024-09-16T09:30:10.705072+0000","last_became_peered":"2024-09-16T09:30:10.705072+0000","last_unstale":"2024-09-16T09:32:47.463767+0000","last_undegraded":"2024-09-16T09:32:47.463767+0000","last_fullsized":"2024-09-16T09:32:47.463767+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:32:43.091969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:32:43.091969+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:45.325012+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,3],"acting":[4,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.7","version":"214'13381","reported_seq":13517,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.946310+0000","last_change":"2024-09-16T09:29:49.839740+0000","last_active":"2024-09-16T09:32:47.946310+0000","last_peered":"2024-09-16T09:32:47.946310+0000","last_clean":"2024-09-16T09:32:47.946310+0000","last_became_active":"2024-09-16T09:29:49.839437+0000","last_became_peered":"2024-09-16T09:29:49.839437+0000","last_unstale":"2024-09-16T09:32:47.946310+0000","last_undegraded":"2024-09-16T09:32:47.946310+0000","last_fullsized":"2024-09-16T09:32:47.946310+0000","mapping_epoch":152,"log_start":"197'8900","ondisk_log_start":"197'8900","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":4481,"log_dups_size":0,"ondisk_log_size":4481,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":135200768,"num_objects":2063,"num_object_clones":0,"num_object_copies":6189,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2063,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13381,"num_write_kb":494208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"66'9","reported_seq":361,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.469982+0000","last_change":"2024-09-16T09:31:31.211495+0000","last_active":"2024-09-16T09:32:47.469982+0000","last_peered":"2024-09-16T09:32:47.469982+0000","last_clean":"2024-09-16T09:32:47.469982+0000","last_became_active":"2024-09-16T09:25:03.712230+0000","last_became_peered":"2024-09-16T09:25:03.712230+0000","last_unstale":"2024-09-16T09:32:47.469982+0000","last_undegraded":"2024-09-16T09:32:47.469982+0000","last_fullsized":"2024-09-16T09:32:47.469982+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:31:31.211420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:31:31.211420+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":346,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.432411+0000","last_change":"2024-09-16T09:31:09.277413+0000","last_active":"2024-09-16T09:32:47.432411+0000","last_peered":"2024-09-16T09:32:47.432411+0000","last_clean":"2024-09-16T09:32:47.432411+0000","last_became_active":"2024-09-16T09:31:09.276516+0000","last_became_peered":"2024-09-16T09:31:09.276516+0000","last_unstale":"2024-09-16T09:32:47.432411+0000","last_undegraded":"2024-09-16T09:32:47.432411+0000","last_fullsized":"2024-09-16T09:32:47.432411+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:26.061106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:26.061106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":347,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.432440+0000","last_change":"2024-09-16T09:31:09.275219+0000","last_active":"2024-09-16T09:32:47.432440+0000","last_peered":"2024-09-16T09:32:47.432440+0000","last_clean":"2024-09-16T09:32:47.432440+0000","last_became_active":"2024-09-16T09:31:09.274702+0000","last_became_peered":"2024-09-16T09:31:09.274702+0000","last_unstale":"2024-09-16T09:32:47.432440+0000","last_undegraded":"2024-09-16T09:32:47.432440+0000","last_fullsized":"2024-09-16T09:32:47.432440+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:57.220068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:28:57.220068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:26.950321+0000","scrub_duration":33,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":337,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.469191+0000","last_change":"2024-09-16T09:31:27.185957+0000","last_active":"2024-09-16T09:32:47.469191+0000","last_peered":"2024-09-16T09:32:47.469191+0000","last_clean":"2024-09-16T09:32:47.469191+0000","last_became_active":"2024-09-16T09:25:07.718778+0000","last_became_peered":"2024-09-16T09:25:07.718778+0000","last_unstale":"2024-09-16T09:32:47.469191+0000","last_undegraded":"2024-09-16T09:32:47.469191+0000","last_fullsized":"2024-09-16T09:32:47.469191+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:31:27.185801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:31:27.185801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.f","version":"73'79","reported_seq":390,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.468979+0000","last_change":"2024-09-16T09:32:44.532983+0000","last_active":"2024-09-16T09:32:47.468979+0000","last_peered":"2024-09-16T09:32:47.468979+0000","last_clean":"2024-09-16T09:32:47.468979+0000","last_became_active":"2024-09-16T09:30:10.693477+0000","last_became_peered":"2024-09-16T09:30:10.693477+0000","last_unstale":"2024-09-16T09:32:47.468979+0000","last_undegraded":"2024-09-16T09:32:47.468979+0000","last_fullsized":"2024-09-16T09:32:47.468979+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:32:44.532922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:32:44.532922+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:02.098440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.6","version":"214'13745","reported_seq":13919,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:48.202218+0000","last_change":"2024-09-16T09:32:36.526082+0000","last_active":"2024-09-16T09:32:48.202218+0000","last_peered":"2024-09-16T09:32:48.202218+0000","last_clean":"2024-09-16T09:32:48.202218+0000","last_became_active":"2024-09-16T09:29:50.008656+0000","last_became_peered":"2024-09-16T09:29:50.008656+0000","last_unstale":"2024-09-16T09:32:48.202218+0000","last_undegraded":"2024-09-16T09:32:48.202218+0000","last_fullsized":"2024-09-16T09:32:48.202218+0000","mapping_epoch":152,"log_start":"195'8600","ondisk_log_start":"195'8600","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"209'12966","last_scrub_stamp":"2024-09-16T09:32:36.526001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:32:36.526001+0000","objects_scrubbed":2905,"log_size":5145,"log_dups_size":0,"ondisk_log_size":5145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:59.711611+0000","scrub_duration":1127,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":136118272,"num_objects":2077,"num_object_clones":0,"num_object_copies":6231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2077,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13745,"num_write_kb":506304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"66'19","reported_seq":368,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.467578+0000","last_change":"2024-09-16T09:29:02.554357+0000","last_active":"2024-09-16T09:32:47.467578+0000","last_peered":"2024-09-16T09:32:47.467578+0000","last_clean":"2024-09-16T09:32:47.467578+0000","last_became_active":"2024-09-16T09:25:03.712783+0000","last_became_peered":"2024-09-16T09:25:03.712783+0000","last_unstale":"2024-09-16T09:32:47.467578+0000","last_undegraded":"2024-09-16T09:32:47.467578+0000","last_fullsized":"2024-09-16T09:32:47.467578+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-16T09:29:02.554274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:29:02.554274+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:21.554123+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"59'1","reported_seq":346,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.430976+0000","last_change":"2024-09-16T09:29:51.025505+0000","last_active":"2024-09-16T09:32:47.430976+0000","last_peered":"2024-09-16T09:32:47.430976+0000","last_clean":"2024-09-16T09:32:47.430976+0000","last_became_active":"2024-09-16T09:25:01.708069+0000","last_became_peered":"2024-09-16T09:25:01.708069+0000","last_unstale":"2024-09-16T09:32:47.430976+0000","last_undegraded":"2024-09-16T09:32:47.430976+0000","last_fullsized":"2024-09-16T09:32:47.430976+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:29:51.025444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:51.025444+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":325,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.431922+0000","last_change":"2024-09-16T09:27:11.067059+0000","last_active":"2024-09-16T09:32:47.431922+0000","last_peered":"2024-09-16T09:32:47.431922+0000","last_clean":"2024-09-16T09:32:47.431922+0000","last_became_active":"2024-09-16T09:25:05.709264+0000","last_became_peered":"2024-09-16T09:25:05.709264+0000","last_unstale":"2024-09-16T09:32:47.431922+0000","last_undegraded":"2024-09-16T09:32:47.431922+0000","last_fullsized":"2024-09-16T09:32:47.431922+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:11.066986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:11.066986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:18.935169+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":320,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.429952+0000","last_change":"2024-09-16T09:27:25.865107+0000","last_active":"2024-09-16T09:32:47.429952+0000","last_peered":"2024-09-16T09:32:47.429952+0000","last_clean":"2024-09-16T09:32:47.429952+0000","last_became_active":"2024-09-16T09:25:07.718750+0000","last_became_peered":"2024-09-16T09:25:07.718750+0000","last_unstale":"2024-09-16T09:32:47.429952+0000","last_undegraded":"2024-09-16T09:32:47.429952+0000","last_fullsized":"2024-09-16T09:32:47.429952+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:25.865030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:25.865030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.c","version":"73'79","reported_seq":70,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.430554+0000","last_change":"2024-09-16T09:31:09.290919+0000","last_active":"2024-09-16T09:32:47.430554+0000","last_peered":"2024-09-16T09:32:47.430554+0000","last_clean":"2024-09-16T09:32:47.430554+0000","last_became_active":"2024-09-16T09:31:09.273770+0000","last_became_peered":"2024-09-16T09:31:09.273770+0000","last_unstale":"2024-09-16T09:32:47.430554+0000","last_undegraded":"2024-09-16T09:32:47.430554+0000","last_fullsized":"2024-09-16T09:32:47.430554+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:30:22.614462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:30:22.614462+0000","objects_scrubbed":1,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":327680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.5","version":"214'13636","reported_seq":13845,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.612577+0000","last_change":"2024-09-16T09:31:06.121217+0000","last_active":"2024-09-16T09:32:47.612577+0000","last_peered":"2024-09-16T09:32:47.612577+0000","last_clean":"2024-09-16T09:32:47.612577+0000","last_became_active":"2024-09-16T09:29:50.009013+0000","last_became_peered":"2024-09-16T09:29:50.009013+0000","last_unstale":"2024-09-16T09:32:47.612577+0000","last_undegraded":"2024-09-16T09:32:47.612577+0000","last_fullsized":"2024-09-16T09:32:47.612577+0000","mapping_epoch":152,"log_start":"195'8313","ondisk_log_start":"195'8313","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"184'6587","last_scrub_stamp":"2024-09-16T09:31:06.120990+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:31:06.120990+0000","objects_scrubbed":6523,"log_size":5323,"log_dups_size":0,"ondisk_log_size":5323,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:25.588988+0000","scrub_duration":1690,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":138149888,"num_objects":2108,"num_object_clones":0,"num_object_copies":6324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2108,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13636,"num_write_kb":503808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"66'12","reported_seq":398,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.463600+0000","last_change":"2024-09-16T09:31:09.296915+0000","last_active":"2024-09-16T09:32:47.463600+0000","last_peered":"2024-09-16T09:32:47.463600+0000","last_clean":"2024-09-16T09:32:47.463600+0000","last_became_active":"2024-09-16T09:31:09.269047+0000","last_became_peered":"2024-09-16T09:31:09.269047+0000","last_unstale":"2024-09-16T09:32:47.463600+0000","last_undegraded":"2024-09-16T09:32:47.463600+0000","last_fullsized":"2024-09-16T09:32:47.463600+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-16T09:30:21.728668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:30:21.728668+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":340,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.431132+0000","last_change":"2024-09-16T09:29:53.075288+0000","last_active":"2024-09-16T09:32:47.431132+0000","last_peered":"2024-09-16T09:32:47.431132+0000","last_clean":"2024-09-16T09:32:47.431132+0000","last_became_active":"2024-09-16T09:25:01.707528+0000","last_became_peered":"2024-09-16T09:25:01.707528+0000","last_unstale":"2024-09-16T09:32:47.431132+0000","last_undegraded":"2024-09-16T09:32:47.431132+0000","last_fullsized":"2024-09-16T09:32:47.431132+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:53.075232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:53.075232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":333,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.431690+0000","last_change":"2024-09-16T09:27:49.965628+0000","last_active":"2024-09-16T09:32:47.431690+0000","last_peered":"2024-09-16T09:32:47.431690+0000","last_clean":"2024-09-16T09:32:47.431690+0000","last_became_active":"2024-09-16T09:25:05.715253+0000","last_became_peered":"2024-09-16T09:25:05.715253+0000","last_unstale":"2024-09-16T09:32:47.431690+0000","last_undegraded":"2024-09-16T09:32:47.431690+0000","last_fullsized":"2024-09-16T09:32:47.431690+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:49.965424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:49.965424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:20.990618+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":321,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.430257+0000","last_change":"2024-09-16T09:27:56.059755+0000","last_active":"2024-09-16T09:32:47.430257+0000","last_peered":"2024-09-16T09:32:47.430257+0000","last_clean":"2024-09-16T09:32:47.430257+0000","last_became_active":"2024-09-16T09:25:07.718899+0000","last_became_peered":"2024-09-16T09:25:07.718899+0000","last_unstale":"2024-09-16T09:32:47.430257+0000","last_undegraded":"2024-09-16T09:32:47.430257+0000","last_fullsized":"2024-09-16T09:32:47.430257+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:56.059687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:56.059687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.d","version":"73'79","reported_seq":138,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.430334+0000","last_change":"2024-09-16T09:32:44.536868+0000","last_active":"2024-09-16T09:32:47.430334+0000","last_peered":"2024-09-16T09:32:47.430334+0000","last_clean":"2024-09-16T09:32:47.430334+0000","last_became_active":"2024-09-16T09:30:10.709958+0000","last_became_peered":"2024-09-16T09:30:10.709958+0000","last_unstale":"2024-09-16T09:32:47.430334+0000","last_undegraded":"2024-09-16T09:32:47.430334+0000","last_fullsized":"2024-09-16T09:32:47.430334+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:32:44.536799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:32:44.536799+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:04.134689+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.4","version":"214'13570","reported_seq":13705,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:48.009605+0000","last_change":"2024-09-16T09:29:49.444857+0000","last_active":"2024-09-16T09:32:48.009605+0000","last_peered":"2024-09-16T09:32:48.009605+0000","last_clean":"2024-09-16T09:32:48.009605+0000","last_became_active":"2024-09-16T09:29:49.444515+0000","last_became_peered":"2024-09-16T09:29:49.444515+0000","last_unstale":"2024-09-16T09:32:48.009605+0000","last_undegraded":"2024-09-16T09:32:48.009605+0000","last_fullsized":"2024-09-16T09:32:48.009605+0000","mapping_epoch":152,"log_start":"196'8800","ondisk_log_start":"196'8800","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":4770,"log_dups_size":0,"ondisk_log_size":4770,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":133562368,"num_objects":2038,"num_object_clones":0,"num_object_copies":6114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2038,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13570,"num_write_kb":499456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"66'15","reported_seq":353,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.432480+0000","last_change":"2024-09-16T09:26:33.138828+0000","last_active":"2024-09-16T09:32:47.432480+0000","last_peered":"2024-09-16T09:32:47.432480+0000","last_clean":"2024-09-16T09:32:47.432480+0000","last_became_active":"2024-09-16T09:25:03.715733+0000","last_became_peered":"2024-09-16T09:25:03.715733+0000","last_unstale":"2024-09-16T09:32:47.432480+0000","last_undegraded":"2024-09-16T09:32:47.432480+0000","last_fullsized":"2024-09-16T09:32:47.432480+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-16T09:26:33.138603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:26:33.138603+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":344,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.430985+0000","last_change":"2024-09-16T09:26:26.705847+0000","last_active":"2024-09-16T09:32:47.430985+0000","last_peered":"2024-09-16T09:32:47.430985+0000","last_clean":"2024-09-16T09:32:47.430985+0000","last_became_active":"2024-09-16T09:25:01.707372+0000","last_became_peered":"2024-09-16T09:25:01.707372+0000","last_unstale":"2024-09-16T09:32:47.430985+0000","last_undegraded":"2024-09-16T09:32:47.430985+0000","last_fullsized":"2024-09-16T09:32:47.430985+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-16T09:26:26.705773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:26.705773+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"66'8","reported_seq":442,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.429851+0000","last_change":"2024-09-16T09:30:05.138362+0000","last_active":"2024-09-16T09:32:47.429851+0000","last_peered":"2024-09-16T09:32:47.429851+0000","last_clean":"2024-09-16T09:32:47.429851+0000","last_became_active":"2024-09-16T09:25:05.716189+0000","last_became_peered":"2024-09-16T09:25:05.716189+0000","last_unstale":"2024-09-16T09:32:47.429851+0000","last_undegraded":"2024-09-16T09:32:47.429851+0000","last_fullsized":"2024-09-16T09:32:47.429851+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:30:05.138288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:30:05.138288+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":320,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.431873+0000","last_change":"2024-09-16T09:27:20.219376+0000","last_active":"2024-09-16T09:32:47.431873+0000","last_peered":"2024-09-16T09:32:47.431873+0000","last_clean":"2024-09-16T09:32:47.431873+0000","last_became_active":"2024-09-16T09:25:07.717213+0000","last_became_peered":"2024-09-16T09:25:07.717213+0000","last_unstale":"2024-09-16T09:32:47.431873+0000","last_undegraded":"2024-09-16T09:32:47.431873+0000","last_fullsized":"2024-09-16T09:32:47.431873+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:20.219333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:20.219333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.12","version":"73'79","reported_seq":138,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.432276+0000","last_change":"2024-09-16T09:32:45.399435+0000","last_active":"2024-09-16T09:32:47.432276+0000","last_peered":"2024-09-16T09:32:47.432276+0000","last_clean":"2024-09-16T09:32:47.432276+0000","last_became_active":"2024-09-16T09:30:10.731413+0000","last_became_peered":"2024-09-16T09:30:10.731413+0000","last_unstale":"2024-09-16T09:32:47.432276+0000","last_undegraded":"2024-09-16T09:32:47.432276+0000","last_fullsized":"2024-09-16T09:32:47.432276+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:32:45.399361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:32:45.399361+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:01.532310+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":333,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.467459+0000","last_change":"2024-09-16T09:26:33.139405+0000","last_active":"2024-09-16T09:32:47.467459+0000","last_peered":"2024-09-16T09:32:47.467459+0000","last_clean":"2024-09-16T09:32:47.467459+0000","last_became_active":"2024-09-16T09:25:01.703202+0000","last_became_peered":"2024-09-16T09:25:01.703202+0000","last_unstale":"2024-09-16T09:32:47.467459+0000","last_undegraded":"2024-09-16T09:32:47.467459+0000","last_fullsized":"2024-09-16T09:32:47.467459+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:33.139360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:33.139360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:10.581208+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"66'6","reported_seq":82,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.851855+0000","last_change":"2024-09-16T09:32:47.851855+0000","last_active":"2024-09-16T09:32:47.851855+0000","last_peered":"2024-09-16T09:32:47.851855+0000","last_clean":"2024-09-16T09:32:47.851855+0000","last_became_active":"2024-09-16T09:31:09.270685+0000","last_became_peered":"2024-09-16T09:31:09.270685+0000","last_unstale":"2024-09-16T09:32:47.851855+0000","last_undegraded":"2024-09-16T09:32:47.851855+0000","last_fullsized":"2024-09-16T09:32:47.851855+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'6","last_scrub_stamp":"2024-09-16T09:32:47.851772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:32:47.851772+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:58.501874+0000","scrub_duration":28,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":8,"num_read_kb":5,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":332,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.432321+0000","last_change":"2024-09-16T09:29:31.496374+0000","last_active":"2024-09-16T09:32:47.432321+0000","last_peered":"2024-09-16T09:32:47.432321+0000","last_clean":"2024-09-16T09:32:47.432321+0000","last_became_active":"2024-09-16T09:25:05.715616+0000","last_became_peered":"2024-09-16T09:25:05.715616+0000","last_unstale":"2024-09-16T09:32:47.432321+0000","last_undegraded":"2024-09-16T09:32:47.432321+0000","last_fullsized":"2024-09-16T09:32:47.432321+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:31.496304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:31.496304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":320,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.430592+0000","last_change":"2024-09-16T09:26:28.733381+0000","last_active":"2024-09-16T09:32:47.430592+0000","last_peered":"2024-09-16T09:32:47.430592+0000","last_clean":"2024-09-16T09:32:47.430592+0000","last_became_active":"2024-09-16T09:25:07.721059+0000","last_became_peered":"2024-09-16T09:25:07.721059+0000","last_unstale":"2024-09-16T09:32:47.430592+0000","last_undegraded":"2024-09-16T09:32:47.430592+0000","last_fullsized":"2024-09-16T09:32:47.430592+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:28.733324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:28.733324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.13","version":"73'79","reported_seq":316,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.434286+0000","last_change":"2024-09-16T09:31:09.288677+0000","last_active":"2024-09-16T09:32:47.434286+0000","last_peered":"2024-09-16T09:32:47.434286+0000","last_clean":"2024-09-16T09:32:47.434286+0000","last_became_active":"2024-09-16T09:31:09.288233+0000","last_became_peered":"2024-09-16T09:31:09.288233+0000","last_unstale":"2024-09-16T09:32:47.434286+0000","last_undegraded":"2024-09-16T09:32:47.434286+0000","last_fullsized":"2024-09-16T09:32:47.434286+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:30:21.714369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:30:21.714369+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.16","version":"66'9","reported_seq":403,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.470774+0000","last_change":"2024-09-16T09:32:01.319481+0000","last_active":"2024-09-16T09:32:47.470774+0000","last_peered":"2024-09-16T09:32:47.470774+0000","last_clean":"2024-09-16T09:32:47.470774+0000","last_became_active":"2024-09-16T09:31:09.273498+0000","last_became_peered":"2024-09-16T09:31:09.273498+0000","last_unstale":"2024-09-16T09:32:47.470774+0000","last_undegraded":"2024-09-16T09:32:47.470774+0000","last_fullsized":"2024-09-16T09:32:47.470774+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:32:01.319390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:32:01.319390+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:29.275572+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":340,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.430218+0000","last_change":"2024-09-16T09:28:50.698331+0000","last_active":"2024-09-16T09:32:47.430218+0000","last_peered":"2024-09-16T09:32:47.430218+0000","last_clean":"2024-09-16T09:32:47.430218+0000","last_became_active":"2024-09-16T09:25:01.706491+0000","last_became_peered":"2024-09-16T09:25:01.706491+0000","last_unstale":"2024-09-16T09:32:47.430218+0000","last_undegraded":"2024-09-16T09:32:47.430218+0000","last_fullsized":"2024-09-16T09:32:47.430218+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:50.698288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:28:50.698288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":190,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.434867+0000","last_change":"2024-09-16T09:31:09.271558+0000","last_active":"2024-09-16T09:32:47.434867+0000","last_peered":"2024-09-16T09:32:47.434867+0000","last_clean":"2024-09-16T09:32:47.434867+0000","last_became_active":"2024-09-16T09:31:09.271223+0000","last_became_peered":"2024-09-16T09:31:09.271223+0000","last_unstale":"2024-09-16T09:32:47.434867+0000","last_undegraded":"2024-09-16T09:32:47.434867+0000","last_fullsized":"2024-09-16T09:32:47.434867+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:53.810580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:28:53.810580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":321,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.431809+0000","last_change":"2024-09-16T09:26:35.866582+0000","last_active":"2024-09-16T09:32:47.431809+0000","last_peered":"2024-09-16T09:32:47.431809+0000","last_clean":"2024-09-16T09:32:47.431809+0000","last_became_active":"2024-09-16T09:25:07.722050+0000","last_became_peered":"2024-09-16T09:25:07.722050+0000","last_unstale":"2024-09-16T09:32:47.431809+0000","last_undegraded":"2024-09-16T09:32:47.431809+0000","last_fullsized":"2024-09-16T09:32:47.431809+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:35.866512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:35.866512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:17.938610+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.10","version":"73'79","reported_seq":138,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.431543+0000","last_change":"2024-09-16T09:32:44.405696+0000","last_active":"2024-09-16T09:32:47.431543+0000","last_peered":"2024-09-16T09:32:47.431543+0000","last_clean":"2024-09-16T09:32:47.431543+0000","last_became_active":"2024-09-16T09:30:10.692770+0000","last_became_peered":"2024-09-16T09:30:10.692770+0000","last_unstale":"2024-09-16T09:32:47.431543+0000","last_undegraded":"2024-09-16T09:32:47.431543+0000","last_fullsized":"2024-09-16T09:32:47.431543+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:32:44.405640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:32:44.405640+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:52.861751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"66'9","reported_seq":352,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.432550+0000","last_change":"2024-09-16T09:29:32.485165+0000","last_active":"2024-09-16T09:32:47.432550+0000","last_peered":"2024-09-16T09:32:47.432550+0000","last_clean":"2024-09-16T09:32:47.432550+0000","last_became_active":"2024-09-16T09:25:03.717310+0000","last_became_peered":"2024-09-16T09:25:03.717310+0000","last_unstale":"2024-09-16T09:32:47.432550+0000","last_undegraded":"2024-09-16T09:32:47.432550+0000","last_fullsized":"2024-09-16T09:32:47.432550+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:29:32.485084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:29:32.485084+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":363,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.469384+0000","last_change":"2024-09-16T09:31:59.376923+0000","last_active":"2024-09-16T09:32:47.469384+0000","last_peered":"2024-09-16T09:32:47.469384+0000","last_clean":"2024-09-16T09:32:47.469384+0000","last_became_active":"2024-09-16T09:31:09.272848+0000","last_became_peered":"2024-09-16T09:31:09.272848+0000","last_unstale":"2024-09-16T09:32:47.469384+0000","last_undegraded":"2024-09-16T09:32:47.469384+0000","last_fullsized":"2024-09-16T09:32:47.469384+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:31:59.376749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:31:59.376749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:13.111415+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"187'11","reported_seq":271,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.430334+0000","last_change":"2024-09-16T09:31:09.311243+0000","last_active":"2024-09-16T09:32:47.430334+0000","last_peered":"2024-09-16T09:32:47.430334+0000","last_clean":"2024-09-16T09:32:47.430334+0000","last_became_active":"2024-09-16T09:31:09.277688+0000","last_became_peered":"2024-09-16T09:31:09.277688+0000","last_unstale":"2024-09-16T09:32:47.430334+0000","last_undegraded":"2024-09-16T09:32:47.430334+0000","last_fullsized":"2024-09-16T09:32:47.430334+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:28:53.714413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:28:53.714413+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":337,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.464049+0000","last_change":"2024-09-16T09:29:49.474440+0000","last_active":"2024-09-16T09:32:47.464049+0000","last_peered":"2024-09-16T09:32:47.464049+0000","last_clean":"2024-09-16T09:32:47.464049+0000","last_became_active":"2024-09-16T09:25:07.718810+0000","last_became_peered":"2024-09-16T09:25:07.718810+0000","last_unstale":"2024-09-16T09:32:47.464049+0000","last_undegraded":"2024-09-16T09:32:47.464049+0000","last_fullsized":"2024-09-16T09:32:47.464049+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:49.474391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:49.474391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.11","version":"73'79","reported_seq":224,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.469284+0000","last_change":"2024-09-16T09:32:38.401888+0000","last_active":"2024-09-16T09:32:47.469284+0000","last_peered":"2024-09-16T09:32:47.469284+0000","last_clean":"2024-09-16T09:32:47.469284+0000","last_became_active":"2024-09-16T09:31:09.272208+0000","last_became_peered":"2024-09-16T09:31:09.272208+0000","last_unstale":"2024-09-16T09:32:47.469284+0000","last_undegraded":"2024-09-16T09:32:47.469284+0000","last_fullsized":"2024-09-16T09:32:47.469284+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:32:38.401720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:32:38.401720+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:46.010237+0000","scrub_duration":32,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.14","version":"66'10","reported_seq":211,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.434556+0000","last_change":"2024-09-16T09:28:56.825675+0000","last_active":"2024-09-16T09:32:47.434556+0000","last_peered":"2024-09-16T09:32:47.434556+0000","last_clean":"2024-09-16T09:32:47.434556+0000","last_became_active":"2024-09-16T09:28:52.979059+0000","last_became_peered":"2024-09-16T09:28:52.979059+0000","last_unstale":"2024-09-16T09:32:47.434556+0000","last_undegraded":"2024-09-16T09:32:47.434556+0000","last_fullsized":"2024-09-16T09:32:47.434556+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-16T09:28:56.825476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:56.825476+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":14,"num_read_kb":9,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":340,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.430273+0000","last_change":"2024-09-16T09:29:56.021413+0000","last_active":"2024-09-16T09:32:47.430273+0000","last_peered":"2024-09-16T09:32:47.430273+0000","last_clean":"2024-09-16T09:32:47.430273+0000","last_became_active":"2024-09-16T09:25:01.707513+0000","last_became_peered":"2024-09-16T09:25:01.707513+0000","last_unstale":"2024-09-16T09:32:47.430273+0000","last_undegraded":"2024-09-16T09:32:47.430273+0000","last_fullsized":"2024-09-16T09:32:47.430273+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:56.021340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:56.021340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"66'8","reported_seq":442,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.432267+0000","last_change":"2024-09-16T09:29:48.485781+0000","last_active":"2024-09-16T09:32:47.432267+0000","last_peered":"2024-09-16T09:32:47.432267+0000","last_clean":"2024-09-16T09:32:47.432267+0000","last_became_active":"2024-09-16T09:25:05.713833+0000","last_became_peered":"2024-09-16T09:25:05.713833+0000","last_unstale":"2024-09-16T09:32:47.432267+0000","last_undegraded":"2024-09-16T09:32:47.432267+0000","last_fullsized":"2024-09-16T09:32:47.432267+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:29:48.485727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.485727+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":351,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.470366+0000","last_change":"2024-09-16T09:32:00.343429+0000","last_active":"2024-09-16T09:32:47.470366+0000","last_peered":"2024-09-16T09:32:47.470366+0000","last_clean":"2024-09-16T09:32:47.470366+0000","last_became_active":"2024-09-16T09:31:09.275168+0000","last_became_peered":"2024-09-16T09:31:09.275168+0000","last_unstale":"2024-09-16T09:32:47.470366+0000","last_undegraded":"2024-09-16T09:32:47.470366+0000","last_fullsized":"2024-09-16T09:32:47.470366+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:00.343356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:32:00.343356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:23.568990+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"66'11","reported_seq":359,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.463663+0000","last_change":"2024-09-16T09:26:33.302713+0000","last_active":"2024-09-16T09:32:47.463663+0000","last_peered":"2024-09-16T09:32:47.463663+0000","last_clean":"2024-09-16T09:32:47.463663+0000","last_became_active":"2024-09-16T09:26:33.291265+0000","last_became_peered":"2024-09-16T09:26:33.291265+0000","last_unstale":"2024-09-16T09:32:47.463663+0000","last_undegraded":"2024-09-16T09:32:47.463663+0000","last_fullsized":"2024-09-16T09:32:47.463663+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:08.864166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":349,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.464511+0000","last_change":"2024-09-16T09:29:51.517112+0000","last_active":"2024-09-16T09:32:47.464511+0000","last_peered":"2024-09-16T09:32:47.464511+0000","last_clean":"2024-09-16T09:32:47.464511+0000","last_became_active":"2024-09-16T09:25:01.705158+0000","last_became_peered":"2024-09-16T09:25:01.705158+0000","last_unstale":"2024-09-16T09:32:47.464511+0000","last_undegraded":"2024-09-16T09:32:47.464511+0000","last_fullsized":"2024-09-16T09:32:47.464511+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:51.517054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:51.517054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":355,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.434227+0000","last_change":"2024-09-16T09:31:09.266820+0000","last_active":"2024-09-16T09:32:47.434227+0000","last_peered":"2024-09-16T09:32:47.434227+0000","last_clean":"2024-09-16T09:32:47.434227+0000","last_became_active":"2024-09-16T09:31:09.266326+0000","last_became_peered":"2024-09-16T09:31:09.266326+0000","last_unstale":"2024-09-16T09:32:47.434227+0000","last_undegraded":"2024-09-16T09:32:47.434227+0000","last_fullsized":"2024-09-16T09:32:47.434227+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:15.910824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:15.910824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":183,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.469130+0000","last_change":"2024-09-16T09:31:09.277239+0000","last_active":"2024-09-16T09:32:47.469130+0000","last_peered":"2024-09-16T09:32:47.469130+0000","last_clean":"2024-09-16T09:32:47.469130+0000","last_became_active":"2024-09-16T09:31:09.276042+0000","last_became_peered":"2024-09-16T09:31:09.276042+0000","last_unstale":"2024-09-16T09:32:47.469130+0000","last_undegraded":"2024-09-16T09:32:47.469130+0000","last_fullsized":"2024-09-16T09:32:47.469130+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:48.147759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:48.147759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:29.306846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"66'9","reported_seq":353,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.434533+0000","last_change":"2024-09-16T09:32:47.247161+0000","last_active":"2024-09-16T09:32:47.434533+0000","last_peered":"2024-09-16T09:32:47.434533+0000","last_clean":"2024-09-16T09:32:47.434533+0000","last_became_active":"2024-09-16T09:25:03.705807+0000","last_became_peered":"2024-09-16T09:25:03.705807+0000","last_unstale":"2024-09-16T09:32:47.434533+0000","last_undegraded":"2024-09-16T09:32:47.434533+0000","last_fullsized":"2024-09-16T09:32:47.434533+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:32:47.247071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:32:47.247071+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:08.145823+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":340,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.502667+0000","last_change":"2024-09-16T09:32:47.502667+0000","last_active":"2024-09-16T09:32:47.502667+0000","last_peered":"2024-09-16T09:32:47.502667+0000","last_clean":"2024-09-16T09:32:47.502667+0000","last_became_active":"2024-09-16T09:25:01.706762+0000","last_became_peered":"2024-09-16T09:25:01.706762+0000","last_unstale":"2024-09-16T09:32:47.502667+0000","last_undegraded":"2024-09-16T09:32:47.502667+0000","last_fullsized":"2024-09-16T09:32:47.502667+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:47.502598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:32:47.502598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:03.669451+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":168,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.469778+0000","last_change":"2024-09-16T09:28:52.983144+0000","last_active":"2024-09-16T09:32:47.469778+0000","last_peered":"2024-09-16T09:32:47.469778+0000","last_clean":"2024-09-16T09:32:47.469778+0000","last_became_active":"2024-09-16T09:28:52.982865+0000","last_became_peered":"2024-09-16T09:28:52.982865+0000","last_unstale":"2024-09-16T09:32:47.469778+0000","last_undegraded":"2024-09-16T09:32:47.469778+0000","last_fullsized":"2024-09-16T09:32:47.469778+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:44.241942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:44.241942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:28.290345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":329,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.467976+0000","last_change":"2024-09-16T09:28:52.552567+0000","last_active":"2024-09-16T09:32:47.467976+0000","last_peered":"2024-09-16T09:32:47.467976+0000","last_clean":"2024-09-16T09:32:47.467976+0000","last_became_active":"2024-09-16T09:25:07.715301+0000","last_became_peered":"2024-09-16T09:25:07.715301+0000","last_unstale":"2024-09-16T09:32:47.467976+0000","last_undegraded":"2024-09-16T09:32:47.467976+0000","last_fullsized":"2024-09-16T09:32:47.467976+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:52.552495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:28:52.552495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.14","version":"73'79","reported_seq":138,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.431706+0000","last_change":"2024-09-16T09:32:44.163264+0000","last_active":"2024-09-16T09:32:47.431706+0000","last_peered":"2024-09-16T09:32:47.431706+0000","last_clean":"2024-09-16T09:32:47.431706+0000","last_became_active":"2024-09-16T09:30:10.700866+0000","last_became_peered":"2024-09-16T09:30:10.700866+0000","last_unstale":"2024-09-16T09:32:47.431706+0000","last_undegraded":"2024-09-16T09:32:47.431706+0000","last_fullsized":"2024-09-16T09:32:47.431706+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:32:44.163191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:32:44.163191+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:13.888037+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"66'11","reported_seq":215,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.430126+0000","last_change":"2024-09-16T09:28:55.643505+0000","last_active":"2024-09-16T09:32:47.430126+0000","last_peered":"2024-09-16T09:32:47.430126+0000","last_clean":"2024-09-16T09:32:47.430126+0000","last_became_active":"2024-09-16T09:28:52.971132+0000","last_became_peered":"2024-09-16T09:28:52.971132+0000","last_unstale":"2024-09-16T09:32:47.430126+0000","last_undegraded":"2024-09-16T09:32:47.430126+0000","last_fullsized":"2024-09-16T09:32:47.430126+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-16T09:28:55.643448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:55.643448+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":19,"num_read_kb":12,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":340,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.432568+0000","last_change":"2024-09-16T09:27:46.175159+0000","last_active":"2024-09-16T09:32:47.432568+0000","last_peered":"2024-09-16T09:32:47.432568+0000","last_clean":"2024-09-16T09:32:47.432568+0000","last_became_active":"2024-09-16T09:25:01.700733+0000","last_became_peered":"2024-09-16T09:25:01.700733+0000","last_unstale":"2024-09-16T09:32:47.432568+0000","last_undegraded":"2024-09-16T09:32:47.432568+0000","last_fullsized":"2024-09-16T09:32:47.432568+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:46.175093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:27:46.175093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":332,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.429616+0000","last_change":"2024-09-16T09:29:25.017736+0000","last_active":"2024-09-16T09:32:47.429616+0000","last_peered":"2024-09-16T09:32:47.429616+0000","last_clean":"2024-09-16T09:32:47.429616+0000","last_became_active":"2024-09-16T09:25:05.713479+0000","last_became_peered":"2024-09-16T09:25:05.713479+0000","last_unstale":"2024-09-16T09:32:47.429616+0000","last_undegraded":"2024-09-16T09:32:47.429616+0000","last_fullsized":"2024-09-16T09:32:47.429616+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:25.017610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:25.017610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":191,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.470140+0000","last_change":"2024-09-16T09:31:09.275121+0000","last_active":"2024-09-16T09:32:47.470140+0000","last_peered":"2024-09-16T09:32:47.470140+0000","last_clean":"2024-09-16T09:32:47.470140+0000","last_became_active":"2024-09-16T09:31:09.274797+0000","last_became_peered":"2024-09-16T09:31:09.274797+0000","last_unstale":"2024-09-16T09:32:47.470140+0000","last_undegraded":"2024-09-16T09:32:47.470140+0000","last_fullsized":"2024-09-16T09:32:47.470140+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:54.589258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:28:54.589258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:48.251310+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"66'4","reported_seq":72,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.429352+0000","last_change":"2024-09-16T09:31:09.401626+0000","last_active":"2024-09-16T09:32:47.429352+0000","last_peered":"2024-09-16T09:32:47.429352+0000","last_clean":"2024-09-16T09:32:47.429352+0000","last_became_active":"2024-09-16T09:31:09.273682+0000","last_became_peered":"2024-09-16T09:31:09.273682+0000","last_unstale":"2024-09-16T09:32:47.429352+0000","last_undegraded":"2024-09-16T09:32:47.429352+0000","last_fullsized":"2024-09-16T09:32:47.429352+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'4","last_scrub_stamp":"2024-09-16T09:26:27.473740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:26:27.473740+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":363,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.470092+0000","last_change":"2024-09-16T09:31:30.235761+0000","last_active":"2024-09-16T09:32:47.470092+0000","last_peered":"2024-09-16T09:32:47.470092+0000","last_clean":"2024-09-16T09:32:47.470092+0000","last_became_active":"2024-09-16T09:31:09.274719+0000","last_became_peered":"2024-09-16T09:31:09.274719+0000","last_unstale":"2024-09-16T09:32:47.470092+0000","last_undegraded":"2024-09-16T09:32:47.470092+0000","last_fullsized":"2024-09-16T09:32:47.470092+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:31:30.235692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:31:30.235692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:49.736157+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":349,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.467577+0000","last_change":"2024-09-16T09:32:07.533575+0000","last_active":"2024-09-16T09:32:47.467577+0000","last_peered":"2024-09-16T09:32:47.467577+0000","last_clean":"2024-09-16T09:32:47.467577+0000","last_became_active":"2024-09-16T09:25:05.712545+0000","last_became_peered":"2024-09-16T09:25:05.712545+0000","last_unstale":"2024-09-16T09:32:47.467577+0000","last_undegraded":"2024-09-16T09:32:47.467577+0000","last_fullsized":"2024-09-16T09:32:47.467577+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:07.533529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:32:07.533529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:28.807212+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":328,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.429860+0000","last_change":"2024-09-16T09:29:27.012810+0000","last_active":"2024-09-16T09:32:47.429860+0000","last_peered":"2024-09-16T09:32:47.429860+0000","last_clean":"2024-09-16T09:32:47.429860+0000","last_became_active":"2024-09-16T09:25:07.720117+0000","last_became_peered":"2024-09-16T09:25:07.720117+0000","last_unstale":"2024-09-16T09:32:47.429860+0000","last_undegraded":"2024-09-16T09:32:47.429860+0000","last_fullsized":"2024-09-16T09:32:47.429860+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:27.012756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:27.012756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":329,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.433840+0000","last_change":"2024-09-16T09:29:19.888825+0000","last_active":"2024-09-16T09:32:47.433840+0000","last_peered":"2024-09-16T09:32:47.433840+0000","last_clean":"2024-09-16T09:32:47.433840+0000","last_became_active":"2024-09-16T09:25:07.712594+0000","last_became_peered":"2024-09-16T09:25:07.712594+0000","last_unstale":"2024-09-16T09:32:47.433840+0000","last_undegraded":"2024-09-16T09:32:47.433840+0000","last_fullsized":"2024-09-16T09:32:47.433840+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:19.888759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:19.888759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":198,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.468264+0000","last_change":"2024-09-16T09:32:40.411546+0000","last_active":"2024-09-16T09:32:47.468264+0000","last_peered":"2024-09-16T09:32:47.468264+0000","last_clean":"2024-09-16T09:32:47.468264+0000","last_became_active":"2024-09-16T09:31:09.271662+0000","last_became_peered":"2024-09-16T09:31:09.271662+0000","last_unstale":"2024-09-16T09:32:47.468264+0000","last_undegraded":"2024-09-16T09:32:47.468264+0000","last_fullsized":"2024-09-16T09:32:47.468264+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:40.411487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:32:40.411487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:51.366030+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"66'11","reported_seq":356,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.469075+0000","last_change":"2024-09-16T09:27:57.486459+0000","last_active":"2024-09-16T09:32:47.469075+0000","last_peered":"2024-09-16T09:32:47.469075+0000","last_clean":"2024-09-16T09:32:47.469075+0000","last_became_active":"2024-09-16T09:25:03.707769+0000","last_became_peered":"2024-09-16T09:25:03.707769+0000","last_unstale":"2024-09-16T09:32:47.469075+0000","last_undegraded":"2024-09-16T09:32:47.469075+0000","last_fullsized":"2024-09-16T09:32:47.469075+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-16T09:27:57.486381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:57.486381+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:17.472949+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":333,"reported_epoch":214,"state":"active+clean","last_fresh":"2024-09-16T09:32:47.470470+0000","last_change":"2024-09-16T09:32:31.325543+0000","last_active":"2024-09-16T09:32:47.470470+0000","last_peered":"2024-09-16T09:32:47.470470+0000","last_clean":"2024-09-16T09:32:47.470470+0000","last_became_active":"2024-09-16T09:25:05.711952+0000","last_became_peered":"2024-09-16T09:25:05.711952+0000","last_unstale":"2024-09-16T09:32:47.470470+0000","last_undegraded":"2024-09-16T09:32:47.470470+0000","last_fullsized":"2024-09-16T09:32:47.470470+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:31.325346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:32:31.325346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:54.289339+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":16,"stat_sum":{"num_bytes":2160197656,"num_objects":32963,"num_object_clones":0,"num_object_copies":98889,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32963,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":216223,"num_write_kb":7973889,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14709,"num_bytes_recovered":963969024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":7080849408,"data_stored":7080837192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":77610,"ondisk_log_size":77610,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":70,"ondisk_log_size":70,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":691,"num_read_kb":448,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":126,"num_bytes_recovered":3026,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":395,"ondisk_log_size":395,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1986,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":460,"num_read_kb":455,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":21,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":853056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1659,"ondisk_log_size":1659,"up":63,"acting":63,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266786,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":925468,"kb_used_data":677704,"kb_used_omap":1,"kb_used_meta":247742,"kb_avail":92819684,"statfs":{"total":95995035648,"available":95047356416,"internally_reserved":0,"allocated":693968896,"data_stored":693636004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":253688266},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495719,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":874504,"kb_used_data":676716,"kb_used_omap":1,"kb_used_meta":197758,"kb_avail":92870648,"statfs":{"total":95995035648,"available":95099543552,"internally_reserved":0,"allocated":692957184,"data_stored":692651842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":202504646},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724654,"num_pgs":64,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1347460,"kb_used_data":1073328,"kb_used_omap":1,"kb_used_meta":274110,"kb_avail":92397692,"statfs":{"total":95995035648,"available":94615236608,"internally_reserved":0,"allocated":1099087872,"data_stored":1098771288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":280689094},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953587,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1242004,"kb_used_data":967868,"kb_used_omap":1,"kb_used_meta":274110,"kb_avail":92503148,"statfs":{"total":95995035648,"available":94723223552,"internally_reserved":0,"allocated":991096832,"data_stored":990768085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":280689094},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149817,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1157604,"kb_used_data":797396,"kb_used_omap":1,"kb_used_meta":360190,"kb_avail":92587548,"statfs":{"total":95995035648,"available":94809649152,"internally_reserved":0,"allocated":816533504,"data_stored":816189194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":368835015},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411454,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1105152,"kb_used_data":812968,"kb_used_omap":1,"kb_used_meta":292158,"kb_avail":92640000,"statfs":{"total":95995035648,"available":94863360000,"internally_reserved":0,"allocated":832479232,"data_stored":832170303,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":299170245},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574981,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1258156,"kb_used_data":947924,"kb_used_omap":1,"kb_used_meta":310206,"kb_avail":92486996,"statfs":{"total":95995035648,"available":94706683904,"internally_reserved":0,"allocated":970674176,"data_stored":970320723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":317651397},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738507,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1390692,"kb_used_data":1080392,"kb_used_omap":1,"kb_used_meta":310206,"kb_avail":92354460,"statfs":{"total":95995035648,"available":94570967040,"internally_reserved":0,"allocated":1106321408,"data_stored":1105988758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":317651397},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1079640064,"data_stored":1079640064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":950140928,"data_stored":950140928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":817500160,"data_stored":817496088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":804192256,"data_stored":804192256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":978911232,"data_stored":978911232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1071714304,"data_stored":1071710232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":688717824,"data_stored":688717824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":690032640,"data_stored":690028568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T09:32:56.062 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:32:56.102 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:56.103 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:56.103 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:32:56.204 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:32:56.231 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:56.232 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:56.232 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:32:56.333 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:32:56.370 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:56.370 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:56.371 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:32:56.472 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:32:56.505 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:56.506 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:56.506 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:32:56.607 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:32:56.651 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:56.652 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:56.653 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:32:56.753 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:32:56.787 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:56.788 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:56.788 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:32:56.889 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:32:56.915 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:56.916 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:56.916 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:32:56.966 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:56 smithi070 ceph-mon[30388]: pgmap v370: 168 pgs: 168 active+clean; 1.6 GiB data, 7.8 GiB used, 707 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.01k op/s 2024-09-16T09:32:56.966 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:32:56.966 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:56 smithi070 ceph-mon[40691]: pgmap v370: 168 pgs: 168 active+clean; 1.6 GiB data, 7.8 GiB used, 707 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.01k op/s 2024-09-16T09:32:56.967 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.3 dump_blocked_ops 2024-09-16T09:32:57.017 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:32:57.018 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:32:57.042 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:57.043 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:57.043 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:32:57.144 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:32:57.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:56 smithi084 ceph-mon[32629]: pgmap v370: 168 pgs: 168 active+clean; 1.6 GiB data, 7.8 GiB used, 707 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.01k op/s 2024-09-16T09:32:57.176 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:57.177 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:57.177 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:32:57.280 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:32:57.329 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:57.330 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:57.330 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:32:57.431 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:32:57.465 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:57.466 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:57.466 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:32:57.568 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:32:57.624 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:57.624 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:57.625 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:32:57.726 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:32:57.756 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:57.756 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:57.757 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:32:57.858 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:32:57.887 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:57.888 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:57.889 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:32:57.994 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:32:58.023 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:58.024 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:58.024 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:32:58.065 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:32:58.065 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:32:58.065 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:32:58.065 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:32:58.111 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:32:58.112 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:32:58.125 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:32:58.145 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:32:58.155 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:32:58.156 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:32:58.196 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:58.197 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:58.197 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:32:58.203 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:32:58.203 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:32:58.241 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:32:58.241 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:32:58.280 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:32:58.280 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:32:58.301 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:32:58.312 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:32:58.312 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:32:58.332 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:58.333 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:58.333 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:32:58.434 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:32:58.464 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:58.465 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:58.465 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:32:58.566 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:32:58.601 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:58.602 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:58.602 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:32:58.703 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:32:58.723 INFO:teuthology.orchestra.run.smithi070.stderr:nodeep-scrub is set 2024-09-16T09:32:58.741 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:58.742 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:58.742 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:32:58.843 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:32:58.881 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:58.883 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:58.884 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:32:58.985 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:32:58.989 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T09:32:58.989 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:32:58.990 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:58 smithi070 ceph-mon[30388]: pgmap v371: 168 pgs: 168 active+clean; 1.6 GiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.16k op/s 2024-09-16T09:32:58.990 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:58 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2148566868' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:32:58.990 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:58 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/754091826' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:32:58.990 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:58 smithi070 ceph-mon[40691]: pgmap v371: 168 pgs: 168 active+clean; 1.6 GiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.16k op/s 2024-09-16T09:32:58.990 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:58 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/2148566868' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:32:58.991 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:58 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/754091826' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:32:59.010 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:59.011 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:59.011 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:32:59.057 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-16T09:32:59.112 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:32:59.152 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:59.152 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:59.153 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:32:59.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:58 smithi084 ceph-mon[32629]: pgmap v371: 168 pgs: 168 active+clean; 1.6 GiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.16k op/s 2024-09-16T09:32:59.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:58 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/2148566868' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:32:59.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:58 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/754091826' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:32:59.254 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:32:59.301 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:59.302 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:59.302 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:32:59.403 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:32:59.438 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:59.439 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:59.439 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:32:59.540 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:32:59.574 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:59.575 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:59.575 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:32:59.676 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:32:59.707 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:32:59.708 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:59.708 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:32:59.760 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:59 smithi070 ceph-mon[30388]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:32:59.761 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:59 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/754091826' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-16T09:32:59.761 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:59 smithi070 ceph-mon[30388]: osdmap e219: 8 total, 8 up, 8 in 2024-09-16T09:32:59.761 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:32:59 smithi070 ceph-mon[30388]: osdmap e220: 8 total, 8 up, 8 in 2024-09-16T09:32:59.761 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:59 smithi070 ceph-mon[40691]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:32:59.761 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:59 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/754091826' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-16T09:32:59.761 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:59 smithi070 ceph-mon[40691]: osdmap e219: 8 total, 8 up, 8 in 2024-09-16T09:32:59.761 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:32:59 smithi070 ceph-mon[40691]: osdmap e220: 8 total, 8 up, 8 in 2024-09-16T09:32:59.809 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:32:59.847 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:59.847 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:59.848 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:32:59.949 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:32:59.979 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:32:59.979 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:32:59.980 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:33:00.080 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:33:00.110 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:00.111 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:00.111 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:33:00.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:59 smithi084 ceph-mon[32629]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:33:00.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:59 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/754091826' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-16T09:33:00.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:59 smithi084 ceph-mon[32629]: osdmap e219: 8 total, 8 up, 8 in 2024-09-16T09:33:00.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:32:59 smithi084 ceph-mon[32629]: osdmap e220: 8 total, 8 up, 8 in 2024-09-16T09:33:00.212 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:33:00.247 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:00.248 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:00.248 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:33:00.349 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:33:00.378 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:00.379 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:00.380 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:33:00.480 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:33:00.518 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:00.519 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:00.519 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:33:00.620 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:33:00.653 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:00.654 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:00.654 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:33:00.754 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:33:00.787 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:00.787 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:00.788 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:33:00.889 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:33:00.919 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:00.919 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:00.920 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:33:01.020 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:33:01.054 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:01.055 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:01.056 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:33:01.098 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.3 dump_historic_ops 2024-09-16T09:33:01.156 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:33:01.190 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:01.190 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:01.191 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:33:01.232 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:33:01.291 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:33:01.317 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:01.318 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:01.318 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:33:01.368 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:01 smithi070 ceph-mon[30388]: pgmap v374: 168 pgs: 168 active+clean; 1.2 GiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.08k op/s 2024-09-16T09:33:01.369 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:01 smithi070 ceph-mon[40691]: pgmap v374: 168 pgs: 168 active+clean; 1.2 GiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.08k op/s 2024-09-16T09:33:01.419 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:33:01.472 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:01.473 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:01.473 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:33:01.574 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:33:01.610 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:01.610 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:01.610 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:33:01.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:01 smithi084 ceph-mon[32629]: pgmap v374: 168 pgs: 168 active+clean; 1.2 GiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.08k op/s 2024-09-16T09:33:01.712 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:33:01.743 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:01.744 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:01.744 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:33:01.763 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T09:33:01.845 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:33:01.876 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:01.877 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:01.877 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:33:01.978 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:33:02.024 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:02.024 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:02.024 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:33:02.055 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:33:01 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:33:01] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:33:02.125 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:33:02.170 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:02.171 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:02.171 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:33:02.249 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:33:02.272 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:33:02.345 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:02.345 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:02.346 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:33:02.446 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:33:02.455 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:33:02.494 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:02.495 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:02.495 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:33:02.576 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:33:02.581 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2024-09-16T09:33:02.596 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:33:02.665 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd unset noscrub 2024-09-16T09:33:02.668 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:02.668 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:02.668 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:33:02.770 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:33:02.819 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:02.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:02.820 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:33:02.921 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:33:02.960 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:02.961 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:02.961 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:33:03.062 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:33:03.098 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:03.098 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:03.099 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:33:03.160 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:33:02 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:33:03.200 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:33:03.234 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:03.234 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:03.235 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:33:03.245 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:03 smithi070 ceph-mon[30388]: pgmap v375: 168 pgs: 168 active+clean; 1.2 GiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 690 B/s rd, 0 B/s wr, 873 op/s 2024-09-16T09:33:03.335 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:33:03.367 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:03.368 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:03.368 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:33:03.469 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:33:03.515 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:03.515 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:03.516 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:33:03.616 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:33:03.648 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:03.649 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:03.650 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:33:03.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:03 smithi084 ceph-mon[32629]: pgmap v375: 168 pgs: 168 active+clean; 1.2 GiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 690 B/s rd, 0 B/s wr, 873 op/s 2024-09-16T09:33:03.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:03 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:33:03.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:03 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:33:03.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:03 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:33:03.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:03 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:33:03.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:03 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:33:03.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:03 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:33:03.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:03 smithi070 ceph-mon[40691]: pgmap v375: 168 pgs: 168 active+clean; 1.2 GiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 690 B/s rd, 0 B/s wr, 873 op/s 2024-09-16T09:33:03.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:03 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:33:03.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:03 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:33:03.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:03 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:33:03.751 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:33:03.780 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:03.780 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:03.781 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:33:03.881 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:33:03.913 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:03.913 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:03.914 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:33:03.930 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":375,"stamp":"2024-09-16T09:33:01.649194+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1338251827,"num_objects":20611,"num_object_clones":0,"num_object_copies":61833,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20611,"num_whiteouts":0,"num_read":1186,"num_read_kb":938,"num_write":229210,"num_write_kb":7973934,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14846,"num_bytes_recovered":964827092,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79801,"ondisk_log_size":79801,"up":504,"acting":504,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":506,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":6941612,"kb_used_data":4628904,"kb_used_omap":12,"kb_used_meta":2312307,"kb_avail":743019604,"statfs":{"total":767960285184,"available":760852074496,"internally_reserved":0,"allocated":4739997696,"data_stored":4737227553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":2367802930},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,7,14,5,8,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-424083456,"num_objects":-6471,"num_object_clones":0,"num_object_copies":-19413,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6471,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":6471,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.417151"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":365,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.235053+0000","last_change":"2024-09-16T09:32:39.384174+0000","last_active":"2024-09-16T09:32:54.235053+0000","last_peered":"2024-09-16T09:32:54.235053+0000","last_clean":"2024-09-16T09:32:54.235053+0000","last_became_active":"2024-09-16T09:25:01.709108+0000","last_became_peered":"2024-09-16T09:25:01.709108+0000","last_unstale":"2024-09-16T09:32:54.235053+0000","last_undegraded":"2024-09-16T09:32:54.235053+0000","last_fullsized":"2024-09-16T09:32:54.235053+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:39.384104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:32:39.384104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:41.986629+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"66'9","reported_seq":369,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.228955+0000","last_change":"2024-09-16T09:32:38.074301+0000","last_active":"2024-09-16T09:32:54.228955+0000","last_peered":"2024-09-16T09:32:54.228955+0000","last_clean":"2024-09-16T09:32:54.228955+0000","last_became_active":"2024-09-16T09:25:03.714635+0000","last_became_peered":"2024-09-16T09:25:03.714635+0000","last_unstale":"2024-09-16T09:32:54.228955+0000","last_undegraded":"2024-09-16T09:32:54.228955+0000","last_fullsized":"2024-09-16T09:32:54.228955+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:32:38.074231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:32:38.074231+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:04.491224+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":349,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.245237+0000","last_change":"2024-09-16T09:29:17.910367+0000","last_active":"2024-09-16T09:32:54.245237+0000","last_peered":"2024-09-16T09:32:54.245237+0000","last_clean":"2024-09-16T09:32:54.245237+0000","last_became_active":"2024-09-16T09:25:05.715645+0000","last_became_peered":"2024-09-16T09:25:05.715645+0000","last_unstale":"2024-09-16T09:32:54.245237+0000","last_undegraded":"2024-09-16T09:32:54.245237+0000","last_fullsized":"2024-09-16T09:32:54.245237+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:17.910305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:17.910305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":329,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.228968+0000","last_change":"2024-09-16T09:27:11.565306+0000","last_active":"2024-09-16T09:32:54.228968+0000","last_peered":"2024-09-16T09:32:54.228968+0000","last_clean":"2024-09-16T09:32:54.228968+0000","last_became_active":"2024-09-16T09:25:07.715442+0000","last_became_peered":"2024-09-16T09:25:07.715442+0000","last_unstale":"2024-09-16T09:32:54.228968+0000","last_undegraded":"2024-09-16T09:32:54.228968+0000","last_fullsized":"2024-09-16T09:32:54.228968+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:11.565220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:11.565220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:25.985557+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":74,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.237764+0000","last_change":"2024-09-16T09:31:09.269700+0000","last_active":"2024-09-16T09:32:54.237764+0000","last_peered":"2024-09-16T09:32:54.237764+0000","last_clean":"2024-09-16T09:32:54.237764+0000","last_became_active":"2024-09-16T09:31:09.269342+0000","last_became_peered":"2024-09-16T09:31:09.269342+0000","last_unstale":"2024-09-16T09:32:54.237764+0000","last_undegraded":"2024-09-16T09:32:54.237764+0000","last_fullsized":"2024-09-16T09:32:54.237764+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:00.068570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:28:00.068570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":215,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.239383+0000","last_change":"2024-09-16T09:31:56.507595+0000","last_active":"2024-09-16T09:32:54.239383+0000","last_peered":"2024-09-16T09:32:54.239383+0000","last_clean":"2024-09-16T09:32:54.239383+0000","last_became_active":"2024-09-16T09:31:09.275136+0000","last_became_peered":"2024-09-16T09:31:09.275136+0000","last_unstale":"2024-09-16T09:32:54.239383+0000","last_undegraded":"2024-09-16T09:32:54.239383+0000","last_fullsized":"2024-09-16T09:32:54.239383+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:31:56.507528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:31:56.507528+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:08.491873+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.19","version":"66'15","reported_seq":283,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.239908+0000","last_change":"2024-09-16T09:32:45.194666+0000","last_active":"2024-09-16T09:32:54.239908+0000","last_peered":"2024-09-16T09:32:54.239908+0000","last_clean":"2024-09-16T09:32:54.239908+0000","last_became_active":"2024-09-16T09:31:09.268750+0000","last_became_peered":"2024-09-16T09:31:09.268750+0000","last_unstale":"2024-09-16T09:32:54.239908+0000","last_undegraded":"2024-09-16T09:32:54.239908+0000","last_fullsized":"2024-09-16T09:32:54.239908+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-16T09:32:45.194581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:32:45.194581+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:09.284179+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":357,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.228859+0000","last_change":"2024-09-16T09:32:44.114475+0000","last_active":"2024-09-16T09:32:54.228859+0000","last_peered":"2024-09-16T09:32:54.228859+0000","last_clean":"2024-09-16T09:32:54.228859+0000","last_became_active":"2024-09-16T09:25:05.717236+0000","last_became_peered":"2024-09-16T09:25:05.717236+0000","last_unstale":"2024-09-16T09:32:54.228859+0000","last_undegraded":"2024-09-16T09:32:54.228859+0000","last_fullsized":"2024-09-16T09:32:54.228859+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:44.114407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:32:44.114407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:59.985952+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":187,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.239133+0000","last_change":"2024-09-16T09:28:34.839909+0000","last_active":"2024-09-16T09:32:54.239133+0000","last_peered":"2024-09-16T09:32:54.239133+0000","last_clean":"2024-09-16T09:32:54.239133+0000","last_became_active":"2024-09-16T09:28:34.839435+0000","last_became_peered":"2024-09-16T09:28:34.839435+0000","last_unstale":"2024-09-16T09:32:54.239133+0000","last_undegraded":"2024-09-16T09:32:54.239133+0000","last_fullsized":"2024-09-16T09:32:54.239133+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:01.038400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:28:01.038400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"66'9","reported_seq":369,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.229265+0000","last_change":"2024-09-16T09:32:48.185484+0000","last_active":"2024-09-16T09:32:54.229265+0000","last_peered":"2024-09-16T09:32:54.229265+0000","last_clean":"2024-09-16T09:32:54.229265+0000","last_became_active":"2024-09-16T09:25:03.709147+0000","last_became_peered":"2024-09-16T09:25:03.709147+0000","last_unstale":"2024-09-16T09:32:54.229265+0000","last_undegraded":"2024-09-16T09:32:54.229265+0000","last_fullsized":"2024-09-16T09:32:54.229265+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:32:48.185380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:32:48.185380+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:13.997808+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":332,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.233434+0000","last_change":"2024-09-16T09:27:59.080369+0000","last_active":"2024-09-16T09:32:54.233434+0000","last_peered":"2024-09-16T09:32:54.233434+0000","last_clean":"2024-09-16T09:32:54.233434+0000","last_became_active":"2024-09-16T09:25:05.718134+0000","last_became_peered":"2024-09-16T09:25:05.718134+0000","last_unstale":"2024-09-16T09:32:54.233434+0000","last_undegraded":"2024-09-16T09:32:54.233434+0000","last_fullsized":"2024-09-16T09:32:54.233434+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:59.080307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:59.080307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"66'1","reported_seq":346,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.234092+0000","last_change":"2024-09-16T09:31:58.367903+0000","last_active":"2024-09-16T09:32:54.234092+0000","last_peered":"2024-09-16T09:32:54.234092+0000","last_clean":"2024-09-16T09:32:54.234092+0000","last_became_active":"2024-09-16T09:25:07.716558+0000","last_became_peered":"2024-09-16T09:25:07.716558+0000","last_unstale":"2024-09-16T09:32:54.234092+0000","last_undegraded":"2024-09-16T09:32:54.234092+0000","last_fullsized":"2024-09-16T09:32:54.234092+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-16T09:31:58.367837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:31:58.367837+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:20.121824+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":340,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.233914+0000","last_change":"2024-09-16T09:27:56.191264+0000","last_active":"2024-09-16T09:32:54.233914+0000","last_peered":"2024-09-16T09:32:54.233914+0000","last_clean":"2024-09-16T09:32:54.233914+0000","last_became_active":"2024-09-16T09:25:01.696939+0000","last_became_peered":"2024-09-16T09:25:01.696939+0000","last_unstale":"2024-09-16T09:32:54.233914+0000","last_undegraded":"2024-09-16T09:32:54.233914+0000","last_fullsized":"2024-09-16T09:32:54.233914+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:56.191220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:27:56.191220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"66'5","reported_seq":355,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.229256+0000","last_change":"2024-09-16T09:32:31.024521+0000","last_active":"2024-09-16T09:32:54.229256+0000","last_peered":"2024-09-16T09:32:54.229256+0000","last_clean":"2024-09-16T09:32:54.229256+0000","last_became_active":"2024-09-16T09:25:03.716136+0000","last_became_peered":"2024-09-16T09:25:03.716136+0000","last_unstale":"2024-09-16T09:32:54.229256+0000","last_undegraded":"2024-09-16T09:32:54.229256+0000","last_fullsized":"2024-09-16T09:32:54.229256+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-16T09:32:31.024450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:32:31.024450+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:41.830431+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":340,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.237011+0000","last_change":"2024-09-16T09:28:52.746046+0000","last_active":"2024-09-16T09:32:54.237011+0000","last_peered":"2024-09-16T09:32:54.237011+0000","last_clean":"2024-09-16T09:32:54.237011+0000","last_became_active":"2024-09-16T09:25:05.718281+0000","last_became_peered":"2024-09-16T09:25:05.718281+0000","last_unstale":"2024-09-16T09:32:54.237011+0000","last_undegraded":"2024-09-16T09:32:54.237011+0000","last_fullsized":"2024-09-16T09:32:54.237011+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:52.745999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:28:52.745999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":358,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.235386+0000","last_change":"2024-09-16T09:31:09.276016+0000","last_active":"2024-09-16T09:32:54.235386+0000","last_peered":"2024-09-16T09:32:54.235386+0000","last_clean":"2024-09-16T09:32:54.235386+0000","last_became_active":"2024-09-16T09:31:09.275656+0000","last_became_peered":"2024-09-16T09:31:09.275656+0000","last_unstale":"2024-09-16T09:32:54.235386+0000","last_undegraded":"2024-09-16T09:32:54.235386+0000","last_fullsized":"2024-09-16T09:32:54.235386+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:50.468134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:50.468134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":345,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.230243+0000","last_change":"2024-09-16T09:29:56.605555+0000","last_active":"2024-09-16T09:32:54.230243+0000","last_peered":"2024-09-16T09:32:54.230243+0000","last_clean":"2024-09-16T09:32:54.230243+0000","last_became_active":"2024-09-16T09:25:07.720793+0000","last_became_peered":"2024-09-16T09:25:07.720793+0000","last_unstale":"2024-09-16T09:32:54.230243+0000","last_undegraded":"2024-09-16T09:32:54.230243+0000","last_fullsized":"2024-09-16T09:32:54.230243+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:56.605488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:56.605488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":96,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.238749+0000","last_change":"2024-09-16T09:30:39.202178+0000","last_active":"2024-09-16T09:32:54.238749+0000","last_peered":"2024-09-16T09:32:54.238749+0000","last_clean":"2024-09-16T09:32:54.238749+0000","last_became_active":"2024-09-16T09:30:39.201824+0000","last_became_peered":"2024-09-16T09:30:39.201824+0000","last_unstale":"2024-09-16T09:32:54.238749+0000","last_undegraded":"2024-09-16T09:32:54.238749+0000","last_fullsized":"2024-09-16T09:32:54.238749+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:50.912979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:50.912979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:23.532853+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"66'15","reported_seq":432,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.242915+0000","last_change":"2024-09-16T09:32:49.207736+0000","last_active":"2024-09-16T09:32:54.242915+0000","last_peered":"2024-09-16T09:32:54.242915+0000","last_clean":"2024-09-16T09:32:54.242915+0000","last_became_active":"2024-09-16T09:31:09.263534+0000","last_became_peered":"2024-09-16T09:31:09.263534+0000","last_unstale":"2024-09-16T09:32:54.242915+0000","last_undegraded":"2024-09-16T09:32:54.242915+0000","last_fullsized":"2024-09-16T09:32:54.242915+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-16T09:32:49.202163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:32:49.202163+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:51.702349+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":341,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.239448+0000","last_change":"2024-09-16T09:29:55.875536+0000","last_active":"2024-09-16T09:32:54.239448+0000","last_peered":"2024-09-16T09:32:54.239448+0000","last_clean":"2024-09-16T09:32:54.239448+0000","last_became_active":"2024-09-16T09:25:05.713929+0000","last_became_peered":"2024-09-16T09:25:05.713929+0000","last_unstale":"2024-09-16T09:32:54.239448+0000","last_undegraded":"2024-09-16T09:32:54.239448+0000","last_fullsized":"2024-09-16T09:32:54.239448+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:55.875480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:55.875480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":193,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.239014+0000","last_change":"2024-09-16T09:30:24.412733+0000","last_active":"2024-09-16T09:32:54.239014+0000","last_peered":"2024-09-16T09:32:54.239014+0000","last_clean":"2024-09-16T09:32:54.239014+0000","last_became_active":"2024-09-16T09:28:52.974255+0000","last_became_peered":"2024-09-16T09:28:52.974255+0000","last_unstale":"2024-09-16T09:32:54.239014+0000","last_undegraded":"2024-09-16T09:32:54.239014+0000","last_fullsized":"2024-09-16T09:32:54.239014+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:30:24.412571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:30:24.412571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":357,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.229421+0000","last_change":"2024-09-16T09:30:03.628882+0000","last_active":"2024-09-16T09:32:54.229421+0000","last_peered":"2024-09-16T09:32:54.229421+0000","last_clean":"2024-09-16T09:32:54.229421+0000","last_became_active":"2024-09-16T09:25:01.706491+0000","last_became_peered":"2024-09-16T09:25:01.706491+0000","last_unstale":"2024-09-16T09:32:54.229421+0000","last_undegraded":"2024-09-16T09:32:54.229421+0000","last_fullsized":"2024-09-16T09:32:54.229421+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:30:03.628812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:30:03.628812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"66'12","reported_seq":257,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.239634+0000","last_change":"2024-09-16T09:31:09.286673+0000","last_active":"2024-09-16T09:32:54.239634+0000","last_peered":"2024-09-16T09:32:54.239634+0000","last_clean":"2024-09-16T09:32:54.239634+0000","last_became_active":"2024-09-16T09:31:09.268588+0000","last_became_peered":"2024-09-16T09:31:09.268588+0000","last_unstale":"2024-09-16T09:32:54.239634+0000","last_undegraded":"2024-09-16T09:32:54.239634+0000","last_fullsized":"2024-09-16T09:32:54.239634+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-16T09:27:01.700139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:01.700139+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":24,"num_read_kb":15,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":333,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.229652+0000","last_change":"2024-09-16T09:27:00.816500+0000","last_active":"2024-09-16T09:32:54.229652+0000","last_peered":"2024-09-16T09:32:54.229652+0000","last_clean":"2024-09-16T09:32:54.229652+0000","last_became_active":"2024-09-16T09:25:05.710093+0000","last_became_peered":"2024-09-16T09:25:05.710093+0000","last_unstale":"2024-09-16T09:32:54.229652+0000","last_undegraded":"2024-09-16T09:32:54.229652+0000","last_fullsized":"2024-09-16T09:32:54.229652+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:00.816424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:00.816424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:28.982331+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":337,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.236974+0000","last_change":"2024-09-16T09:29:29.918433+0000","last_active":"2024-09-16T09:32:54.236974+0000","last_peered":"2024-09-16T09:32:54.236974+0000","last_clean":"2024-09-16T09:32:54.236974+0000","last_became_active":"2024-09-16T09:25:07.719380+0000","last_became_peered":"2024-09-16T09:25:07.719380+0000","last_unstale":"2024-09-16T09:32:54.236974+0000","last_undegraded":"2024-09-16T09:32:54.236974+0000","last_fullsized":"2024-09-16T09:32:54.236974+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-16T09:29:29.918368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:29.918368+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":358,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.245359+0000","last_change":"2024-09-16T09:30:02.843932+0000","last_active":"2024-09-16T09:32:54.245359+0000","last_peered":"2024-09-16T09:32:54.245359+0000","last_clean":"2024-09-16T09:32:54.245359+0000","last_became_active":"2024-09-16T09:25:01.699545+0000","last_became_peered":"2024-09-16T09:25:01.699545+0000","last_unstale":"2024-09-16T09:32:54.245359+0000","last_undegraded":"2024-09-16T09:32:54.245359+0000","last_fullsized":"2024-09-16T09:32:54.245359+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:30:02.843856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:30:02.843856+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"66'10","reported_seq":406,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.234911+0000","last_change":"2024-09-16T09:32:30.286077+0000","last_active":"2024-09-16T09:32:54.234911+0000","last_peered":"2024-09-16T09:32:54.234911+0000","last_clean":"2024-09-16T09:32:54.234911+0000","last_became_active":"2024-09-16T09:31:09.275050+0000","last_became_peered":"2024-09-16T09:31:09.275050+0000","last_unstale":"2024-09-16T09:32:54.234911+0000","last_undegraded":"2024-09-16T09:32:54.234911+0000","last_fullsized":"2024-09-16T09:32:54.234911+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-16T09:32:30.284919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:32:30.284919+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:30.659184+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"66'8","reported_seq":472,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.239173+0000","last_change":"2024-09-16T09:32:46.533273+0000","last_active":"2024-09-16T09:32:54.239173+0000","last_peered":"2024-09-16T09:32:54.239173+0000","last_clean":"2024-09-16T09:32:54.239173+0000","last_became_active":"2024-09-16T09:25:05.712516+0000","last_became_peered":"2024-09-16T09:25:05.712516+0000","last_unstale":"2024-09-16T09:32:54.239173+0000","last_undegraded":"2024-09-16T09:32:54.239173+0000","last_fullsized":"2024-09-16T09:32:54.239173+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:32:46.533204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:32:46.533204+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:58.006839+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"218'14334","reported_seq":14478,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:57.942553+0000","last_change":"2024-09-16T09:29:49.449011+0000","last_active":"2024-09-16T09:32:57.942553+0000","last_peered":"2024-09-16T09:32:57.942553+0000","last_clean":"2024-09-16T09:32:57.942553+0000","last_became_active":"2024-09-16T09:29:49.448473+0000","last_became_peered":"2024-09-16T09:29:49.448473+0000","last_unstale":"2024-09-16T09:32:57.942553+0000","last_undegraded":"2024-09-16T09:32:57.942553+0000","last_fullsized":"2024-09-16T09:32:57.942553+0000","mapping_epoch":152,"log_start":"200'9800","ondisk_log_start":"200'9800","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":4534,"log_dups_size":0,"ondisk_log_size":4534,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":84148224,"num_objects":1284,"num_object_clones":0,"num_object_copies":3852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1284,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14334,"num_write_kb":499776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.a","version":"73'79","reported_seq":279,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.233150+0000","last_change":"2024-09-16T09:29:18.498887+0000","last_active":"2024-09-16T09:32:54.233150+0000","last_peered":"2024-09-16T09:32:54.233150+0000","last_clean":"2024-09-16T09:32:54.233150+0000","last_became_active":"2024-09-16T09:26:35.127352+0000","last_became_peered":"2024-09-16T09:26:35.127352+0000","last_unstale":"2024-09-16T09:32:54.233150+0000","last_undegraded":"2024-09-16T09:32:54.233150+0000","last_fullsized":"2024-09-16T09:32:54.233150+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:18.498766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:18.498766+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"66'15","reported_seq":288,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.241773+0000","last_change":"2024-09-16T09:28:28.065781+0000","last_active":"2024-09-16T09:32:54.241773+0000","last_peered":"2024-09-16T09:32:54.241773+0000","last_clean":"2024-09-16T09:32:54.241773+0000","last_became_active":"2024-09-16T09:26:33.295672+0000","last_became_peered":"2024-09-16T09:26:33.295672+0000","last_unstale":"2024-09-16T09:32:54.241773+0000","last_undegraded":"2024-09-16T09:32:54.241773+0000","last_fullsized":"2024-09-16T09:32:54.241773+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-16T09:28:28.065699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:28.065699+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:23.961200+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":74,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.238276+0000","last_change":"2024-09-16T09:31:09.276858+0000","last_active":"2024-09-16T09:32:54.238276+0000","last_peered":"2024-09-16T09:32:54.238276+0000","last_clean":"2024-09-16T09:32:54.238276+0000","last_became_active":"2024-09-16T09:31:09.274534+0000","last_became_peered":"2024-09-16T09:31:09.274534+0000","last_unstale":"2024-09-16T09:32:54.238276+0000","last_undegraded":"2024-09-16T09:32:54.238276+0000","last_fullsized":"2024-09-16T09:32:54.238276+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:22.091537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:28:22.091537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"66'8","reported_seq":429,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.231482+0000","last_change":"2024-09-16T09:25:05.714128+0000","last_active":"2024-09-16T09:32:54.231482+0000","last_peered":"2024-09-16T09:32:54.231482+0000","last_clean":"2024-09-16T09:32:54.231482+0000","last_became_active":"2024-09-16T09:25:05.713813+0000","last_became_peered":"2024-09-16T09:25:05.713813+0000","last_unstale":"2024-09-16T09:32:54.231482+0000","last_undegraded":"2024-09-16T09:32:54.231482+0000","last_fullsized":"2024-09-16T09:32:54.231482+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:17.970742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":336,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.233589+0000","last_change":"2024-09-16T09:29:24.537802+0000","last_active":"2024-09-16T09:32:54.233589+0000","last_peered":"2024-09-16T09:32:54.233589+0000","last_clean":"2024-09-16T09:32:54.233589+0000","last_became_active":"2024-09-16T09:25:07.717344+0000","last_became_peered":"2024-09-16T09:25:07.717344+0000","last_unstale":"2024-09-16T09:32:54.233589+0000","last_undegraded":"2024-09-16T09:32:54.233589+0000","last_fullsized":"2024-09-16T09:32:54.233589+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:24.537731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:24.537731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.c","version":"218'14320","reported_seq":14522,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:58.237087+0000","last_change":"2024-09-16T09:32:07.676294+0000","last_active":"2024-09-16T09:32:58.237087+0000","last_peered":"2024-09-16T09:32:58.237087+0000","last_clean":"2024-09-16T09:32:58.237087+0000","last_became_active":"2024-09-16T09:29:50.007935+0000","last_became_peered":"2024-09-16T09:29:50.007935+0000","last_unstale":"2024-09-16T09:32:58.237087+0000","last_undegraded":"2024-09-16T09:32:58.237087+0000","last_fullsized":"2024-09-16T09:32:58.237087+0000","mapping_epoch":152,"log_start":"197'9200","ondisk_log_start":"197'9200","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"204'10667","last_scrub_stamp":"2024-09-16T09:32:07.676020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:32:07.676020+0000","objects_scrubbed":4920,"log_size":5120,"log_dups_size":0,"ondisk_log_size":5120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:18.894513+0000","scrub_duration":1433,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":80478208,"num_objects":1228,"num_object_clones":0,"num_object_copies":3684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14320,"num_write_kb":497536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"73'79","reported_seq":320,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.229829+0000","last_change":"2024-09-16T09:32:42.123139+0000","last_active":"2024-09-16T09:32:54.229829+0000","last_peered":"2024-09-16T09:32:54.229829+0000","last_clean":"2024-09-16T09:32:54.229829+0000","last_became_active":"2024-09-16T09:30:08.763683+0000","last_became_peered":"2024-09-16T09:30:08.763683+0000","last_unstale":"2024-09-16T09:32:54.229829+0000","last_undegraded":"2024-09-16T09:32:54.229829+0000","last_fullsized":"2024-09-16T09:32:54.229829+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:32:42.123067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:32:42.123067+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:10.308483+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"66'18","reported_seq":287,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.237318+0000","last_change":"2024-09-16T09:31:09.407700+0000","last_active":"2024-09-16T09:32:54.237318+0000","last_peered":"2024-09-16T09:32:54.237318+0000","last_clean":"2024-09-16T09:32:54.237318+0000","last_became_active":"2024-09-16T09:31:09.270434+0000","last_became_peered":"2024-09-16T09:31:09.270434+0000","last_unstale":"2024-09-16T09:32:54.237318+0000","last_undegraded":"2024-09-16T09:32:54.237318+0000","last_fullsized":"2024-09-16T09:32:54.237318+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'18","last_scrub_stamp":"2024-09-16T09:28:54.699778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:54.699778+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":33,"num_read_kb":21,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":184,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.237622+0000","last_change":"2024-09-16T09:28:58.705169+0000","last_active":"2024-09-16T09:32:54.237622+0000","last_peered":"2024-09-16T09:32:54.237622+0000","last_clean":"2024-09-16T09:32:54.237622+0000","last_became_active":"2024-09-16T09:28:52.983543+0000","last_became_peered":"2024-09-16T09:28:52.983543+0000","last_unstale":"2024-09-16T09:32:54.237622+0000","last_undegraded":"2024-09-16T09:32:54.237622+0000","last_fullsized":"2024-09-16T09:32:54.237622+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:58.705097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:28:58.705097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":363,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.230692+0000","last_change":"2024-09-16T09:31:09.269234+0000","last_active":"2024-09-16T09:32:54.230692+0000","last_peered":"2024-09-16T09:32:54.230692+0000","last_clean":"2024-09-16T09:32:54.230692+0000","last_became_active":"2024-09-16T09:31:09.268864+0000","last_became_peered":"2024-09-16T09:31:09.268864+0000","last_unstale":"2024-09-16T09:32:54.230692+0000","last_undegraded":"2024-09-16T09:32:54.230692+0000","last_fullsized":"2024-09-16T09:32:54.230692+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:50.482240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:50.482240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":351,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.229172+0000","last_change":"2024-09-16T09:32:32.060608+0000","last_active":"2024-09-16T09:32:54.229172+0000","last_peered":"2024-09-16T09:32:54.229172+0000","last_clean":"2024-09-16T09:32:54.229172+0000","last_became_active":"2024-09-16T09:31:09.269433+0000","last_became_peered":"2024-09-16T09:31:09.269433+0000","last_unstale":"2024-09-16T09:32:54.229172+0000","last_undegraded":"2024-09-16T09:32:54.229172+0000","last_fullsized":"2024-09-16T09:32:54.229172+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:32.060564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:32:32.060564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:48.641239+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.d","version":"218'14226","reported_seq":14371,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:57.992729+0000","last_change":"2024-09-16T09:29:50.010860+0000","last_active":"2024-09-16T09:32:57.992729+0000","last_peered":"2024-09-16T09:32:57.992729+0000","last_clean":"2024-09-16T09:32:57.992729+0000","last_became_active":"2024-09-16T09:29:50.010421+0000","last_became_peered":"2024-09-16T09:29:50.010421+0000","last_unstale":"2024-09-16T09:32:57.992729+0000","last_undegraded":"2024-09-16T09:32:57.992729+0000","last_fullsized":"2024-09-16T09:32:57.992729+0000","mapping_epoch":152,"log_start":"199'9500","ondisk_log_start":"199'9500","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":4726,"log_dups_size":0,"ondisk_log_size":4726,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":87752728,"num_objects":1340,"num_object_clones":0,"num_object_copies":4020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1340,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14226,"num_write_kb":498049,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,2],"acting":[5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.4","version":"73'79","reported_seq":74,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.236956+0000","last_change":"2024-09-16T09:31:09.275499+0000","last_active":"2024-09-16T09:32:54.236956+0000","last_peered":"2024-09-16T09:32:54.236956+0000","last_clean":"2024-09-16T09:32:54.236956+0000","last_became_active":"2024-09-16T09:31:09.275139+0000","last_became_peered":"2024-09-16T09:31:09.275139+0000","last_unstale":"2024-09-16T09:32:54.236956+0000","last_undegraded":"2024-09-16T09:32:54.236956+0000","last_fullsized":"2024-09-16T09:32:54.236956+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:30:22.438378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:30:22.438378+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"66'14","reported_seq":374,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.230779+0000","last_change":"2024-09-16T09:32:34.997417+0000","last_active":"2024-09-16T09:32:54.230779+0000","last_peered":"2024-09-16T09:32:54.230779+0000","last_clean":"2024-09-16T09:32:54.230779+0000","last_became_active":"2024-09-16T09:25:03.708830+0000","last_became_peered":"2024-09-16T09:25:03.708830+0000","last_unstale":"2024-09-16T09:32:54.230779+0000","last_undegraded":"2024-09-16T09:32:54.230779+0000","last_fullsized":"2024-09-16T09:32:54.230779+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'14","last_scrub_stamp":"2024-09-16T09:32:34.997360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:32:34.997360+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:47.824918+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":366,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.235143+0000","last_change":"2024-09-16T09:31:57.373606+0000","last_active":"2024-09-16T09:32:54.235143+0000","last_peered":"2024-09-16T09:32:54.235143+0000","last_clean":"2024-09-16T09:32:54.235143+0000","last_became_active":"2024-09-16T09:25:01.702992+0000","last_became_peered":"2024-09-16T09:25:01.702992+0000","last_unstale":"2024-09-16T09:32:54.235143+0000","last_undegraded":"2024-09-16T09:32:54.235143+0000","last_fullsized":"2024-09-16T09:32:54.235143+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:31:57.373544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:31:57.373544+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:24.193683+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"187'11","reported_seq":282,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.239889+0000","last_change":"2024-09-16T09:31:09.397758+0000","last_active":"2024-09-16T09:32:54.239889+0000","last_peered":"2024-09-16T09:32:54.239889+0000","last_clean":"2024-09-16T09:32:54.239889+0000","last_became_active":"2024-09-16T09:31:09.269173+0000","last_became_peered":"2024-09-16T09:31:09.269173+0000","last_unstale":"2024-09-16T09:32:54.239889+0000","last_undegraded":"2024-09-16T09:32:54.239889+0000","last_fullsized":"2024-09-16T09:32:54.239889+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'9","last_scrub_stamp":"2024-09-16T09:29:21.898533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:21.898533+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":328,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.236569+0000","last_change":"2024-09-16T09:26:40.699888+0000","last_active":"2024-09-16T09:32:54.236569+0000","last_peered":"2024-09-16T09:32:54.236569+0000","last_clean":"2024-09-16T09:32:54.236569+0000","last_became_active":"2024-09-16T09:25:07.722017+0000","last_became_peered":"2024-09-16T09:25:07.722017+0000","last_unstale":"2024-09-16T09:32:54.236569+0000","last_undegraded":"2024-09-16T09:32:54.236569+0000","last_fullsized":"2024-09-16T09:32:54.236569+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:40.699813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:40.699813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.e","version":"218'14348","reported_seq":14530,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:58.224119+0000","last_change":"2024-09-16T09:32:35.567515+0000","last_active":"2024-09-16T09:32:58.224119+0000","last_peered":"2024-09-16T09:32:58.224119+0000","last_clean":"2024-09-16T09:32:58.224119+0000","last_became_active":"2024-09-16T09:29:49.447767+0000","last_became_peered":"2024-09-16T09:29:49.447767+0000","last_unstale":"2024-09-16T09:32:58.224119+0000","last_undegraded":"2024-09-16T09:32:58.224119+0000","last_fullsized":"2024-09-16T09:32:58.224119+0000","mapping_epoch":152,"log_start":"197'9200","ondisk_log_start":"197'9200","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"209'12756","last_scrub_stamp":"2024-09-16T09:32:35.567430+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:32:35.567430+0000","objects_scrubbed":2885,"log_size":5148,"log_dups_size":0,"ondisk_log_size":5148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:51.840038+0000","scrub_duration":1201,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":82444288,"num_objects":1258,"num_object_clones":0,"num_object_copies":3774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1258,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14348,"num_write_kb":499392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"73'79","reported_seq":405,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.239008+0000","last_change":"2024-09-16T09:32:45.555132+0000","last_active":"2024-09-16T09:32:54.239008+0000","last_peered":"2024-09-16T09:32:54.239008+0000","last_clean":"2024-09-16T09:32:54.239008+0000","last_became_active":"2024-09-16T09:31:09.274707+0000","last_became_peered":"2024-09-16T09:31:09.274707+0000","last_unstale":"2024-09-16T09:32:54.239008+0000","last_undegraded":"2024-09-16T09:32:54.239008+0000","last_fullsized":"2024-09-16T09:32:54.239008+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:32:45.555067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:32:45.555067+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:56.478822+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"66'10","reported_seq":368,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.238852+0000","last_change":"2024-09-16T09:30:04.848306+0000","last_active":"2024-09-16T09:32:54.238852+0000","last_peered":"2024-09-16T09:32:54.238852+0000","last_clean":"2024-09-16T09:32:54.238852+0000","last_became_active":"2024-09-16T09:25:03.712985+0000","last_became_peered":"2024-09-16T09:25:03.712985+0000","last_unstale":"2024-09-16T09:32:54.238852+0000","last_undegraded":"2024-09-16T09:32:54.238852+0000","last_fullsized":"2024-09-16T09:32:54.238852+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-16T09:30:04.848234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:30:04.848234+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":362,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.234005+0000","last_change":"2024-09-16T09:32:46.388711+0000","last_active":"2024-09-16T09:32:54.234005+0000","last_peered":"2024-09-16T09:32:54.234005+0000","last_clean":"2024-09-16T09:32:54.234005+0000","last_became_active":"2024-09-16T09:31:09.270619+0000","last_became_peered":"2024-09-16T09:31:09.270619+0000","last_unstale":"2024-09-16T09:32:54.234005+0000","last_undegraded":"2024-09-16T09:32:54.234005+0000","last_fullsized":"2024-09-16T09:32:54.234005+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:46.388651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:32:46.388651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:12.727689+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"66'8","reported_seq":445,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.235410+0000","last_change":"2024-09-16T09:28:57.628615+0000","last_active":"2024-09-16T09:32:54.235410+0000","last_peered":"2024-09-16T09:32:54.235410+0000","last_clean":"2024-09-16T09:32:54.235410+0000","last_became_active":"2024-09-16T09:25:05.714507+0000","last_became_peered":"2024-09-16T09:25:05.714507+0000","last_unstale":"2024-09-16T09:32:54.235410+0000","last_undegraded":"2024-09-16T09:32:54.235410+0000","last_fullsized":"2024-09-16T09:32:54.235410+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:28:57.628543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:28:57.628543+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:52.447851+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":359,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.234554+0000","last_change":"2024-09-16T09:32:33.362233+0000","last_active":"2024-09-16T09:32:54.234554+0000","last_peered":"2024-09-16T09:32:54.234554+0000","last_clean":"2024-09-16T09:32:54.234554+0000","last_became_active":"2024-09-16T09:31:09.283192+0000","last_became_peered":"2024-09-16T09:31:09.283192+0000","last_unstale":"2024-09-16T09:32:54.234554+0000","last_undegraded":"2024-09-16T09:32:54.234554+0000","last_fullsized":"2024-09-16T09:32:54.234554+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:33.362188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:32:33.362188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:40.243325+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.f","version":"218'14469","reported_seq":14641,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:57.902243+0000","last_change":"2024-09-16T09:32:47.453692+0000","last_active":"2024-09-16T09:32:57.902243+0000","last_peered":"2024-09-16T09:32:57.902243+0000","last_clean":"2024-09-16T09:32:57.902243+0000","last_became_active":"2024-09-16T09:29:49.838146+0000","last_became_peered":"2024-09-16T09:29:49.838146+0000","last_unstale":"2024-09-16T09:32:57.902243+0000","last_undegraded":"2024-09-16T09:32:57.902243+0000","last_fullsized":"2024-09-16T09:32:57.902243+0000","mapping_epoch":152,"log_start":"195'8400","ondisk_log_start":"195'8400","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"213'13600","last_scrub_stamp":"2024-09-16T09:32:47.453582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:32:47.453582+0000","objects_scrubbed":2146,"log_size":6069,"log_dups_size":0,"ondisk_log_size":6069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:16.176992+0000","scrub_duration":239,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83689472,"num_objects":1277,"num_object_clones":0,"num_object_copies":3831,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1277,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14469,"num_write_kb":503872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1],"acting":[2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"73'79","reported_seq":378,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.239190+0000","last_change":"2024-09-16T09:31:09.274975+0000","last_active":"2024-09-16T09:32:54.239190+0000","last_peered":"2024-09-16T09:32:54.239190+0000","last_clean":"2024-09-16T09:32:54.239190+0000","last_became_active":"2024-09-16T09:31:09.274565+0000","last_became_peered":"2024-09-16T09:31:09.274565+0000","last_unstale":"2024-09-16T09:32:54.239190+0000","last_undegraded":"2024-09-16T09:32:54.239190+0000","last_fullsized":"2024-09-16T09:32:54.239190+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:30:22.415051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:30:22.415051+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"66'19","reported_seq":381,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.235586+0000","last_change":"2024-09-16T09:31:35.249952+0000","last_active":"2024-09-16T09:32:54.235586+0000","last_peered":"2024-09-16T09:32:54.235586+0000","last_clean":"2024-09-16T09:32:54.235586+0000","last_became_active":"2024-09-16T09:25:03.714797+0000","last_became_peered":"2024-09-16T09:25:03.714797+0000","last_unstale":"2024-09-16T09:32:54.235586+0000","last_undegraded":"2024-09-16T09:32:54.235586+0000","last_fullsized":"2024-09-16T09:32:54.235586+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-16T09:31:35.249864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:31:35.249864+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":357,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.239042+0000","last_change":"2024-09-16T09:29:16.888357+0000","last_active":"2024-09-16T09:32:54.239042+0000","last_peered":"2024-09-16T09:32:54.239042+0000","last_clean":"2024-09-16T09:32:54.239042+0000","last_became_active":"2024-09-16T09:25:01.706806+0000","last_became_peered":"2024-09-16T09:25:01.706806+0000","last_unstale":"2024-09-16T09:32:54.239042+0000","last_undegraded":"2024-09-16T09:32:54.239042+0000","last_fullsized":"2024-09-16T09:32:54.239042+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:16.888236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:16.888236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":357,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.238609+0000","last_change":"2024-09-16T09:31:55.456669+0000","last_active":"2024-09-16T09:32:54.238609+0000","last_peered":"2024-09-16T09:32:54.238609+0000","last_clean":"2024-09-16T09:32:54.238609+0000","last_became_active":"2024-09-16T09:25:05.708663+0000","last_became_peered":"2024-09-16T09:25:05.708663+0000","last_unstale":"2024-09-16T09:32:54.238609+0000","last_undegraded":"2024-09-16T09:32:54.238609+0000","last_fullsized":"2024-09-16T09:32:54.238609+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:31:55.456592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:31:55.456592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:32:56.006473+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":329,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.245122+0000","last_change":"2024-09-16T09:26:37.282798+0000","last_active":"2024-09-16T09:32:54.245122+0000","last_peered":"2024-09-16T09:32:54.245122+0000","last_clean":"2024-09-16T09:32:54.245122+0000","last_became_active":"2024-09-16T09:25:07.719267+0000","last_became_peered":"2024-09-16T09:25:07.719267+0000","last_unstale":"2024-09-16T09:32:54.245122+0000","last_undegraded":"2024-09-16T09:32:54.245122+0000","last_fullsized":"2024-09-16T09:32:54.245122+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:37.282735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:37.282735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.8","version":"218'14443","reported_seq":23516,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:58.026536+0000","last_change":"2024-09-16T09:32:46.470436+0000","last_active":"2024-09-16T09:32:58.026536+0000","last_peered":"2024-09-16T09:32:58.026536+0000","last_clean":"2024-09-16T09:32:58.026536+0000","last_became_active":"2024-09-16T09:32:46.435851+0000","last_became_peered":"2024-09-16T09:32:46.435851+0000","last_unstale":"2024-09-16T09:32:58.026536+0000","last_undegraded":"2024-09-16T09:32:58.026536+0000","last_fullsized":"2024-09-16T09:32:58.026536+0000","mapping_epoch":212,"log_start":"199'9700","ondisk_log_start":"199'9700","created":152,"last_epoch_clean":213,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":4743,"log_dups_size":0,"ondisk_log_size":4743,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83820544,"num_objects":1279,"num_object_clones":0,"num_object_copies":3837,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1279,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14443,"num_write_kb":503104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4456,"num_bytes_recovered":292028416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1","version":"73'79","reported_seq":312,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.235218+0000","last_change":"2024-09-16T09:32:43.420831+0000","last_active":"2024-09-16T09:32:54.235218+0000","last_peered":"2024-09-16T09:32:54.235218+0000","last_clean":"2024-09-16T09:32:54.235218+0000","last_became_active":"2024-09-16T09:30:08.766350+0000","last_became_peered":"2024-09-16T09:30:08.766350+0000","last_unstale":"2024-09-16T09:32:54.235218+0000","last_undegraded":"2024-09-16T09:32:54.235218+0000","last_fullsized":"2024-09-16T09:32:54.235218+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:32:43.420784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:32:43.420784+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:55.813620+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"187'30","reported_seq":547,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.238954+0000","last_change":"2024-09-16T09:31:09.323281+0000","last_active":"2024-09-16T09:32:54.238954+0000","last_peered":"2024-09-16T09:32:54.238954+0000","last_clean":"2024-09-16T09:32:54.238954+0000","last_became_active":"2024-09-16T09:31:09.278839+0000","last_became_peered":"2024-09-16T09:31:09.278839+0000","last_unstale":"2024-09-16T09:32:54.238954+0000","last_undegraded":"2024-09-16T09:32:54.238954+0000","last_fullsized":"2024-09-16T09:32:54.238954+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'28","last_scrub_stamp":"2024-09-16T09:29:18.937623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:29:18.937623+0000","objects_scrubbed":10,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":349,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.229754+0000","last_change":"2024-09-16T09:29:26.637070+0000","last_active":"2024-09-16T09:32:54.229754+0000","last_peered":"2024-09-16T09:32:54.229754+0000","last_clean":"2024-09-16T09:32:54.229754+0000","last_became_active":"2024-09-16T09:25:01.703076+0000","last_became_peered":"2024-09-16T09:25:01.703076+0000","last_unstale":"2024-09-16T09:32:54.229754+0000","last_undegraded":"2024-09-16T09:32:54.229754+0000","last_fullsized":"2024-09-16T09:32:54.229754+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:26.637027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:26.637027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:52.239472+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":358,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.234171+0000","last_change":"2024-09-16T09:29:29.486284+0000","last_active":"2024-09-16T09:32:54.234171+0000","last_peered":"2024-09-16T09:32:54.234171+0000","last_clean":"2024-09-16T09:32:54.234171+0000","last_became_active":"2024-09-16T09:25:00.696237+0000","last_became_peered":"2024-09-16T09:25:00.696237+0000","last_unstale":"2024-09-16T09:32:54.234171+0000","last_undegraded":"2024-09-16T09:32:54.234171+0000","last_fullsized":"2024-09-16T09:32:54.234171+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-16T09:29:29.486188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:29:29.486188+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.0025137430000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":325,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.238654+0000","last_change":"2024-09-16T09:25:05.718319+0000","last_active":"2024-09-16T09:32:54.238654+0000","last_peered":"2024-09-16T09:32:54.238654+0000","last_clean":"2024-09-16T09:32:54.238654+0000","last_became_active":"2024-09-16T09:25:05.717591+0000","last_became_peered":"2024-09-16T09:25:05.717591+0000","last_unstale":"2024-09-16T09:32:54.238654+0000","last_undegraded":"2024-09-16T09:32:54.238654+0000","last_fullsized":"2024-09-16T09:32:54.238654+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:11.173707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":74,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.237426+0000","last_change":"2024-09-16T09:31:09.272967+0000","last_active":"2024-09-16T09:32:54.237426+0000","last_peered":"2024-09-16T09:32:54.237426+0000","last_clean":"2024-09-16T09:32:54.237426+0000","last_became_active":"2024-09-16T09:31:09.272609+0000","last_became_peered":"2024-09-16T09:31:09.272609+0000","last_unstale":"2024-09-16T09:32:54.237426+0000","last_undegraded":"2024-09-16T09:32:54.237426+0000","last_fullsized":"2024-09-16T09:32:54.237426+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:45.201503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:45.201503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:52.833583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"218'14221","reported_seq":25485,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:57.813335+0000","last_change":"2024-09-16T09:32:08.439285+0000","last_active":"2024-09-16T09:32:57.813335+0000","last_peered":"2024-09-16T09:32:57.813335+0000","last_clean":"2024-09-16T09:32:57.813335+0000","last_became_active":"2024-09-16T09:32:08.422751+0000","last_became_peered":"2024-09-16T09:32:08.422751+0000","last_unstale":"2024-09-16T09:32:57.813335+0000","last_undegraded":"2024-09-16T09:32:57.813335+0000","last_fullsized":"2024-09-16T09:32:57.813335+0000","mapping_epoch":204,"log_start":"200'9900","ondisk_log_start":"200'9900","created":152,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":4321,"log_dups_size":0,"ondisk_log_size":4321,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:57.213860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":86048768,"num_objects":1313,"num_object_clones":0,"num_object_copies":3939,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1313,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14221,"num_write_kb":497088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5551,"num_bytes_recovered":363790336,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.2","version":"73'79","reported_seq":437,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.236409+0000","last_change":"2024-09-16T09:32:42.498732+0000","last_active":"2024-09-16T09:32:54.236409+0000","last_peered":"2024-09-16T09:32:54.236409+0000","last_clean":"2024-09-16T09:32:54.236409+0000","last_became_active":"2024-09-16T09:30:08.765943+0000","last_became_peered":"2024-09-16T09:30:08.765943+0000","last_unstale":"2024-09-16T09:32:54.236409+0000","last_undegraded":"2024-09-16T09:32:54.236409+0000","last_fullsized":"2024-09-16T09:32:54.236409+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:32:42.498658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:32:42.498658+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:09.187680+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"66'13","reported_seq":372,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.240900+0000","last_change":"2024-09-16T09:27:44.156057+0000","last_active":"2024-09-16T09:32:54.240900+0000","last_peered":"2024-09-16T09:32:54.240900+0000","last_clean":"2024-09-16T09:32:54.240900+0000","last_became_active":"2024-09-16T09:25:03.713241+0000","last_became_peered":"2024-09-16T09:25:03.713241+0000","last_unstale":"2024-09-16T09:32:54.240900+0000","last_undegraded":"2024-09-16T09:32:54.240900+0000","last_fullsized":"2024-09-16T09:32:54.240900+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'13","last_scrub_stamp":"2024-09-16T09:27:44.155969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:44.155969+0000","objects_scrubbed":5,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":29,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":349,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.242614+0000","last_change":"2024-09-16T09:29:50.827103+0000","last_active":"2024-09-16T09:32:54.242614+0000","last_peered":"2024-09-16T09:32:54.242614+0000","last_clean":"2024-09-16T09:32:54.242614+0000","last_became_active":"2024-09-16T09:25:01.706828+0000","last_became_peered":"2024-09-16T09:25:01.706828+0000","last_unstale":"2024-09-16T09:32:54.242614+0000","last_undegraded":"2024-09-16T09:32:54.242614+0000","last_fullsized":"2024-09-16T09:32:54.242614+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:50.827057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:50.827057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":388,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.240005+0000","last_change":"2024-09-16T09:32:40.165819+0000","last_active":"2024-09-16T09:32:54.240005+0000","last_peered":"2024-09-16T09:32:54.240005+0000","last_clean":"2024-09-16T09:32:54.240005+0000","last_became_active":"2024-09-16T09:31:09.266338+0000","last_became_peered":"2024-09-16T09:31:09.266338+0000","last_unstale":"2024-09-16T09:32:54.240005+0000","last_undegraded":"2024-09-16T09:32:54.240005+0000","last_fullsized":"2024-09-16T09:32:54.240005+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:32:40.165749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:32:40.165749+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:47.551824+0000","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0.00055022299999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":357,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.244280+0000","last_change":"2024-09-16T09:32:32.029895+0000","last_active":"2024-09-16T09:32:54.244280+0000","last_peered":"2024-09-16T09:32:54.244280+0000","last_clean":"2024-09-16T09:32:54.244280+0000","last_became_active":"2024-09-16T09:25:05.711539+0000","last_became_peered":"2024-09-16T09:25:05.711539+0000","last_unstale":"2024-09-16T09:32:54.244280+0000","last_undegraded":"2024-09-16T09:32:54.244280+0000","last_fullsized":"2024-09-16T09:32:54.244280+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:32.029829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:32:32.029829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:36.831098+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":336,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.237775+0000","last_change":"2024-09-16T09:29:22.968971+0000","last_active":"2024-09-16T09:32:54.237775+0000","last_peered":"2024-09-16T09:32:54.237775+0000","last_clean":"2024-09-16T09:32:54.237775+0000","last_became_active":"2024-09-16T09:25:07.723282+0000","last_became_peered":"2024-09-16T09:25:07.723282+0000","last_unstale":"2024-09-16T09:32:54.237775+0000","last_undegraded":"2024-09-16T09:32:54.237775+0000","last_fullsized":"2024-09-16T09:32:54.237775+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:22.968908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:22.968908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"218'14353","reported_seq":14496,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:57.934496+0000","last_change":"2024-09-16T09:29:49.448898+0000","last_active":"2024-09-16T09:32:57.934496+0000","last_peered":"2024-09-16T09:32:57.934496+0000","last_clean":"2024-09-16T09:32:57.934496+0000","last_became_active":"2024-09-16T09:29:49.448628+0000","last_became_peered":"2024-09-16T09:29:49.448628+0000","last_unstale":"2024-09-16T09:32:57.934496+0000","last_undegraded":"2024-09-16T09:32:57.934496+0000","last_fullsized":"2024-09-16T09:32:57.934496+0000","mapping_epoch":152,"log_start":"200'9800","ondisk_log_start":"200'9800","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":4553,"log_dups_size":0,"ondisk_log_size":4553,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83951616,"num_objects":1281,"num_object_clones":0,"num_object_copies":3843,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1281,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14353,"num_write_kb":500288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"73'79","reported_seq":311,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.250060+0000","last_change":"2024-09-16T09:31:40.212968+0000","last_active":"2024-09-16T09:32:54.250060+0000","last_peered":"2024-09-16T09:32:54.250060+0000","last_clean":"2024-09-16T09:32:54.250060+0000","last_became_active":"2024-09-16T09:30:08.877110+0000","last_became_peered":"2024-09-16T09:30:08.877110+0000","last_unstale":"2024-09-16T09:32:54.250060+0000","last_undegraded":"2024-09-16T09:32:54.250060+0000","last_fullsized":"2024-09-16T09:32:54.250060+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:31:40.212889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:31:40.212889+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"66'12","reported_seq":363,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.234987+0000","last_change":"2024-09-16T09:32:48.547237+0000","last_active":"2024-09-16T09:32:54.234987+0000","last_peered":"2024-09-16T09:32:54.234987+0000","last_clean":"2024-09-16T09:32:54.234987+0000","last_became_active":"2024-09-16T09:25:03.715315+0000","last_became_peered":"2024-09-16T09:25:03.715315+0000","last_unstale":"2024-09-16T09:32:54.234987+0000","last_undegraded":"2024-09-16T09:32:54.234987+0000","last_fullsized":"2024-09-16T09:32:54.234987+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-16T09:32:48.546869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:32:48.546869+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:14.182805+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"59'1","reported_seq":372,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.237122+0000","last_change":"2024-09-16T09:31:09.286309+0000","last_active":"2024-09-16T09:32:54.237122+0000","last_peered":"2024-09-16T09:32:54.237122+0000","last_clean":"2024-09-16T09:32:54.237122+0000","last_became_active":"2024-09-16T09:31:09.269627+0000","last_became_peered":"2024-09-16T09:31:09.269627+0000","last_unstale":"2024-09-16T09:32:54.237122+0000","last_undegraded":"2024-09-16T09:32:54.237122+0000","last_fullsized":"2024-09-16T09:32:54.237122+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:27:00.666673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:27:00.666673+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:21.192318+0000","scrub_duration":28,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1986,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"67'5","reported_seq":829,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:56.194210+0000","last_change":"2024-09-16T09:29:19.994791+0000","last_active":"2024-09-16T09:32:56.194210+0000","last_peered":"2024-09-16T09:32:56.194210+0000","last_clean":"2024-09-16T09:32:56.194210+0000","last_became_active":"2024-09-16T09:25:00.696546+0000","last_became_peered":"2024-09-16T09:25:00.696546+0000","last_unstale":"2024-09-16T09:32:56.194210+0000","last_undegraded":"2024-09-16T09:32:56.194210+0000","last_fullsized":"2024-09-16T09:32:56.194210+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'5","last_scrub_stamp":"2024-09-16T09:29:19.994716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:29:19.994716+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0.0059770420000000001,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":470,"num_read_kb":465,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":332,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.234290+0000","last_change":"2024-09-16T09:27:07.337170+0000","last_active":"2024-09-16T09:32:54.234290+0000","last_peered":"2024-09-16T09:32:54.234290+0000","last_clean":"2024-09-16T09:32:54.234290+0000","last_became_active":"2024-09-16T09:25:05.714704+0000","last_became_peered":"2024-09-16T09:25:05.714704+0000","last_unstale":"2024-09-16T09:32:54.234290+0000","last_undegraded":"2024-09-16T09:32:54.234290+0000","last_fullsized":"2024-09-16T09:32:54.234290+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:07.337124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:07.337124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":351,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.238617+0000","last_change":"2024-09-16T09:32:46.252747+0000","last_active":"2024-09-16T09:32:54.238617+0000","last_peered":"2024-09-16T09:32:54.238617+0000","last_clean":"2024-09-16T09:32:54.238617+0000","last_became_active":"2024-09-16T09:31:09.271111+0000","last_became_peered":"2024-09-16T09:31:09.271111+0000","last_unstale":"2024-09-16T09:32:54.238617+0000","last_undegraded":"2024-09-16T09:32:54.238617+0000","last_fullsized":"2024-09-16T09:32:54.238617+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:46.252700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:32:46.252700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:01.656018+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"218'14272","reported_seq":14481,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:57.612036+0000","last_change":"2024-09-16T09:30:56.023710+0000","last_active":"2024-09-16T09:32:57.612036+0000","last_peered":"2024-09-16T09:32:57.612036+0000","last_clean":"2024-09-16T09:32:57.612036+0000","last_became_active":"2024-09-16T09:29:49.450643+0000","last_became_peered":"2024-09-16T09:29:49.450643+0000","last_unstale":"2024-09-16T09:32:57.612036+0000","last_undegraded":"2024-09-16T09:32:57.612036+0000","last_fullsized":"2024-09-16T09:32:57.612036+0000","mapping_epoch":152,"log_start":"197'9000","ondisk_log_start":"197'9000","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"181'5780","last_scrub_stamp":"2024-09-16T09:30:56.023622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:30:56.023622+0000","objects_scrubbed":5716,"log_size":5272,"log_dups_size":0,"ondisk_log_size":5272,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:14.532550+0000","scrub_duration":1526,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":84672512,"num_objects":1292,"num_object_clones":0,"num_object_copies":3876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1292,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14272,"num_write_kb":498048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"66'16","reported_seq":471,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.238828+0000","last_change":"2024-09-16T09:32:01.564577+0000","last_active":"2024-09-16T09:32:54.238828+0000","last_peered":"2024-09-16T09:32:54.238828+0000","last_clean":"2024-09-16T09:32:54.238828+0000","last_became_active":"2024-09-16T09:25:03.714118+0000","last_became_peered":"2024-09-16T09:25:03.714118+0000","last_unstale":"2024-09-16T09:32:54.238828+0000","last_undegraded":"2024-09-16T09:32:54.238828+0000","last_fullsized":"2024-09-16T09:32:54.238828+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'16","last_scrub_stamp":"2024-09-16T09:32:01.564489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:32:01.564489+0000","objects_scrubbed":8,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:10.625142+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":190,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.234086+0000","last_change":"2024-09-16T09:31:09.270603+0000","last_active":"2024-09-16T09:32:54.234086+0000","last_peered":"2024-09-16T09:32:54.234086+0000","last_clean":"2024-09-16T09:32:54.234086+0000","last_became_active":"2024-09-16T09:31:09.270196+0000","last_became_peered":"2024-09-16T09:31:09.270196+0000","last_unstale":"2024-09-16T09:32:54.234086+0000","last_undegraded":"2024-09-16T09:32:54.234086+0000","last_fullsized":"2024-09-16T09:32:54.234086+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:49.195331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:27:49.195331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"73'79","reported_seq":437,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.237687+0000","last_change":"2024-09-16T09:32:43.535041+0000","last_active":"2024-09-16T09:32:54.237687+0000","last_peered":"2024-09-16T09:32:54.237687+0000","last_clean":"2024-09-16T09:32:54.237687+0000","last_became_active":"2024-09-16T09:30:08.870394+0000","last_became_peered":"2024-09-16T09:30:08.870394+0000","last_unstale":"2024-09-16T09:32:54.237687+0000","last_undegraded":"2024-09-16T09:32:54.237687+0000","last_fullsized":"2024-09-16T09:32:54.237687+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:32:43.534962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:32:43.534962+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:46.508762+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":332,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.237430+0000","last_change":"2024-09-16T09:26:39.699133+0000","last_active":"2024-09-16T09:32:54.237430+0000","last_peered":"2024-09-16T09:32:54.237430+0000","last_clean":"2024-09-16T09:32:54.237430+0000","last_became_active":"2024-09-16T09:25:05.714341+0000","last_became_peered":"2024-09-16T09:25:05.714341+0000","last_unstale":"2024-09-16T09:32:54.237430+0000","last_undegraded":"2024-09-16T09:32:54.237430+0000","last_fullsized":"2024-09-16T09:32:54.237430+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:39.699059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:26:39.699059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:51.437385+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":336,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.234120+0000","last_change":"2024-09-16T09:29:00.243528+0000","last_active":"2024-09-16T09:32:54.234120+0000","last_peered":"2024-09-16T09:32:54.234120+0000","last_clean":"2024-09-16T09:32:54.234120+0000","last_became_active":"2024-09-16T09:25:07.714281+0000","last_became_peered":"2024-09-16T09:25:07.714281+0000","last_unstale":"2024-09-16T09:32:54.234120+0000","last_undegraded":"2024-09-16T09:32:54.234120+0000","last_fullsized":"2024-09-16T09:32:54.234120+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:00.243482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:00.243482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.b","version":"73'79","reported_seq":322,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.253793+0000","last_change":"2024-09-16T09:32:44.255969+0000","last_active":"2024-09-16T09:32:54.253793+0000","last_peered":"2024-09-16T09:32:54.253793+0000","last_clean":"2024-09-16T09:32:54.253793+0000","last_became_active":"2024-09-16T09:30:10.784688+0000","last_became_peered":"2024-09-16T09:30:10.784688+0000","last_unstale":"2024-09-16T09:32:54.253793+0000","last_undegraded":"2024-09-16T09:32:54.253793+0000","last_fullsized":"2024-09-16T09:32:54.253793+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:32:44.255693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:32:44.255693+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:57.310805+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,5],"acting":[1,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2","version":"218'14031","reported_seq":14175,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:57.815795+0000","last_change":"2024-09-16T09:29:49.445324+0000","last_active":"2024-09-16T09:32:57.815795+0000","last_peered":"2024-09-16T09:32:57.815795+0000","last_clean":"2024-09-16T09:32:57.815795+0000","last_became_active":"2024-09-16T09:29:49.445022+0000","last_became_peered":"2024-09-16T09:29:49.445022+0000","last_unstale":"2024-09-16T09:32:57.815795+0000","last_undegraded":"2024-09-16T09:32:57.815795+0000","last_fullsized":"2024-09-16T09:32:57.815795+0000","mapping_epoch":152,"log_start":"200'9800","ondisk_log_start":"200'9800","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":4231,"log_dups_size":0,"ondisk_log_size":4231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":78970880,"num_objects":1205,"num_object_clones":0,"num_object_copies":3615,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14031,"num_write_kb":487552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"66'11","reported_seq":375,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.231526+0000","last_change":"2024-09-16T09:30:05.646701+0000","last_active":"2024-09-16T09:32:54.231526+0000","last_peered":"2024-09-16T09:32:54.231526+0000","last_clean":"2024-09-16T09:32:54.231526+0000","last_became_active":"2024-09-16T09:26:33.287399+0000","last_became_peered":"2024-09-16T09:26:33.287399+0000","last_unstale":"2024-09-16T09:32:54.231526+0000","last_undegraded":"2024-09-16T09:32:54.231526+0000","last_fullsized":"2024-09-16T09:32:54.231526+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-16T09:30:05.646500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:30:05.646500+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":333,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.231161+0000","last_change":"2024-09-16T09:25:01.708568+0000","last_active":"2024-09-16T09:32:54.231161+0000","last_peered":"2024-09-16T09:32:54.231161+0000","last_clean":"2024-09-16T09:32:54.231161+0000","last_became_active":"2024-09-16T09:25:01.708270+0000","last_became_peered":"2024-09-16T09:25:01.708270+0000","last_unstale":"2024-09-16T09:32:54.231161+0000","last_undegraded":"2024-09-16T09:32:54.231161+0000","last_fullsized":"2024-09-16T09:32:54.231161+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:25:00.689058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:10.865036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":332,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.233985+0000","last_change":"2024-09-16T09:26:34.171190+0000","last_active":"2024-09-16T09:32:54.233985+0000","last_peered":"2024-09-16T09:32:54.233985+0000","last_clean":"2024-09-16T09:32:54.233985+0000","last_became_active":"2024-09-16T09:25:05.711964+0000","last_became_peered":"2024-09-16T09:25:05.711964+0000","last_unstale":"2024-09-16T09:32:54.233985+0000","last_undegraded":"2024-09-16T09:32:54.233985+0000","last_fullsized":"2024-09-16T09:32:54.233985+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:34.171139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:26:34.171139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":215,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.238754+0000","last_change":"2024-09-16T09:32:00.526741+0000","last_active":"2024-09-16T09:32:54.238754+0000","last_peered":"2024-09-16T09:32:54.238754+0000","last_clean":"2024-09-16T09:32:54.238754+0000","last_became_active":"2024-09-16T09:31:09.278618+0000","last_became_peered":"2024-09-16T09:31:09.278618+0000","last_unstale":"2024-09-16T09:32:54.238754+0000","last_undegraded":"2024-09-16T09:32:54.238754+0000","last_fullsized":"2024-09-16T09:32:54.238754+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:00.526674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:32:00.526674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:25.768941+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1","version":"218'13983","reported_seq":14126,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:57.960225+0000","last_change":"2024-09-16T09:29:49.449796+0000","last_active":"2024-09-16T09:32:57.960225+0000","last_peered":"2024-09-16T09:32:57.960225+0000","last_clean":"2024-09-16T09:32:57.960225+0000","last_became_active":"2024-09-16T09:29:49.449475+0000","last_became_peered":"2024-09-16T09:29:49.449475+0000","last_unstale":"2024-09-16T09:32:57.960225+0000","last_undegraded":"2024-09-16T09:32:57.960225+0000","last_fullsized":"2024-09-16T09:32:57.960225+0000","mapping_epoch":152,"log_start":"200'9500","ondisk_log_start":"200'9500","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":4483,"log_dups_size":0,"ondisk_log_size":4483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83558400,"num_objects":1275,"num_object_clones":0,"num_object_copies":3825,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1275,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13983,"num_write_kb":488256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.8","version":"73'79","reported_seq":311,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.233164+0000","last_change":"2024-09-16T09:32:07.068820+0000","last_active":"2024-09-16T09:32:54.233164+0000","last_peered":"2024-09-16T09:32:54.233164+0000","last_clean":"2024-09-16T09:32:54.233164+0000","last_became_active":"2024-09-16T09:31:09.406638+0000","last_became_peered":"2024-09-16T09:31:09.406638+0000","last_unstale":"2024-09-16T09:32:54.233164+0000","last_undegraded":"2024-09-16T09:32:54.233164+0000","last_fullsized":"2024-09-16T09:32:54.233164+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:17.482769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:17.482769+0000","objects_scrubbed":1,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262688,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":525376,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"66'17","reported_seq":381,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.229691+0000","last_change":"2024-09-16T09:30:04.636115+0000","last_active":"2024-09-16T09:32:54.229691+0000","last_peered":"2024-09-16T09:32:54.229691+0000","last_clean":"2024-09-16T09:32:54.229691+0000","last_became_active":"2024-09-16T09:25:03.713358+0000","last_became_peered":"2024-09-16T09:25:03.713358+0000","last_unstale":"2024-09-16T09:32:54.229691+0000","last_undegraded":"2024-09-16T09:32:54.229691+0000","last_fullsized":"2024-09-16T09:32:54.229691+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'17","last_scrub_stamp":"2024-09-16T09:30:04.636039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:30:04.636039+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":341,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.238932+0000","last_change":"2024-09-16T09:26:04.104188+0000","last_active":"2024-09-16T09:32:54.238932+0000","last_peered":"2024-09-16T09:32:54.238932+0000","last_clean":"2024-09-16T09:32:54.238932+0000","last_became_active":"2024-09-16T09:25:01.703355+0000","last_became_peered":"2024-09-16T09:25:01.703355+0000","last_unstale":"2024-09-16T09:32:54.238932+0000","last_undegraded":"2024-09-16T09:32:54.238932+0000","last_fullsized":"2024-09-16T09:32:54.238932+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:04.104130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:04.104130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:53.552807+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":349,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.238706+0000","last_change":"2024-09-16T09:29:58.839132+0000","last_active":"2024-09-16T09:32:54.238706+0000","last_peered":"2024-09-16T09:32:54.238706+0000","last_clean":"2024-09-16T09:32:54.238706+0000","last_became_active":"2024-09-16T09:25:05.716900+0000","last_became_peered":"2024-09-16T09:25:05.716900+0000","last_unstale":"2024-09-16T09:32:54.238706+0000","last_undegraded":"2024-09-16T09:32:54.238706+0000","last_fullsized":"2024-09-16T09:32:54.238706+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:58.839051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:58.839051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":367,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.242090+0000","last_change":"2024-09-16T09:32:48.260039+0000","last_active":"2024-09-16T09:32:54.242090+0000","last_peered":"2024-09-16T09:32:54.242090+0000","last_clean":"2024-09-16T09:32:54.242090+0000","last_became_active":"2024-09-16T09:31:09.276275+0000","last_became_peered":"2024-09-16T09:31:09.276275+0000","last_unstale":"2024-09-16T09:32:54.242090+0000","last_undegraded":"2024-09-16T09:32:54.242090+0000","last_fullsized":"2024-09-16T09:32:54.242090+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:48.259971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:32:48.259971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:48.391021+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.0","version":"218'14311","reported_seq":23878,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:57.996219+0000","last_change":"2024-09-16T09:32:47.458790+0000","last_active":"2024-09-16T09:32:57.996219+0000","last_peered":"2024-09-16T09:32:57.996219+0000","last_clean":"2024-09-16T09:32:57.996219+0000","last_became_active":"2024-09-16T09:32:47.435160+0000","last_became_peered":"2024-09-16T09:32:47.435160+0000","last_unstale":"2024-09-16T09:32:57.996219+0000","last_undegraded":"2024-09-16T09:32:57.996219+0000","last_fullsized":"2024-09-16T09:32:57.996219+0000","mapping_epoch":213,"log_start":"199'9600","ondisk_log_start":"199'9600","created":152,"last_epoch_clean":214,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":4711,"log_dups_size":0,"ondisk_log_size":4711,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":80281600,"num_objects":1225,"num_object_clones":0,"num_object_copies":3675,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1225,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14311,"num_write_kb":497152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4702,"num_bytes_recovered":308150272,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.9","version":"73'79","reported_seq":412,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.236169+0000","last_change":"2024-09-16T09:29:19.013948+0000","last_active":"2024-09-16T09:32:54.236169+0000","last_peered":"2024-09-16T09:32:54.236169+0000","last_clean":"2024-09-16T09:32:54.236169+0000","last_became_active":"2024-09-16T09:26:35.126066+0000","last_became_peered":"2024-09-16T09:26:35.126066+0000","last_unstale":"2024-09-16T09:32:54.236169+0000","last_undegraded":"2024-09-16T09:32:54.236169+0000","last_fullsized":"2024-09-16T09:32:54.236169+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:19.013865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:19.013865+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"66'10","reported_seq":344,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.229787+0000","last_change":"2024-09-16T09:25:03.708144+0000","last_active":"2024-09-16T09:32:54.229787+0000","last_peered":"2024-09-16T09:32:54.229787+0000","last_clean":"2024-09-16T09:32:54.229787+0000","last_became_active":"2024-09-16T09:25:03.707723+0000","last_became_peered":"2024-09-16T09:25:03.707723+0000","last_unstale":"2024-09-16T09:32:54.229787+0000","last_undegraded":"2024-09-16T09:32:54.229787+0000","last_fullsized":"2024-09-16T09:32:54.229787+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:13.930015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":206,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.235742+0000","last_change":"2024-09-16T09:31:32.253397+0000","last_active":"2024-09-16T09:32:54.235742+0000","last_peered":"2024-09-16T09:32:54.235742+0000","last_clean":"2024-09-16T09:32:54.235742+0000","last_became_active":"2024-09-16T09:31:09.288825+0000","last_became_peered":"2024-09-16T09:31:09.288825+0000","last_unstale":"2024-09-16T09:32:54.235742+0000","last_undegraded":"2024-09-16T09:32:54.235742+0000","last_fullsized":"2024-09-16T09:32:54.235742+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:31:32.253306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:31:32.253306+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"66'8","reported_seq":464,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.243471+0000","last_change":"2024-09-16T09:32:42.175655+0000","last_active":"2024-09-16T09:32:54.243471+0000","last_peered":"2024-09-16T09:32:54.243471+0000","last_clean":"2024-09-16T09:32:54.243471+0000","last_became_active":"2024-09-16T09:25:05.711590+0000","last_became_peered":"2024-09-16T09:25:05.711590+0000","last_unstale":"2024-09-16T09:32:54.243471+0000","last_undegraded":"2024-09-16T09:32:54.243471+0000","last_fullsized":"2024-09-16T09:32:54.243471+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:32:42.175528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:32:42.175528+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:44.416122+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":353,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.230184+0000","last_change":"2024-09-16T09:32:39.072795+0000","last_active":"2024-09-16T09:32:54.230184+0000","last_peered":"2024-09-16T09:32:54.230184+0000","last_clean":"2024-09-16T09:32:54.230184+0000","last_became_active":"2024-09-16T09:25:07.718958+0000","last_became_peered":"2024-09-16T09:25:07.718958+0000","last_unstale":"2024-09-16T09:32:54.230184+0000","last_undegraded":"2024-09-16T09:32:54.230184+0000","last_fullsized":"2024-09-16T09:32:54.230184+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:39.072735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:32:39.072735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:41.434371+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.e","version":"73'79","reported_seq":146,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.229187+0000","last_change":"2024-09-16T09:32:43.092027+0000","last_active":"2024-09-16T09:32:54.229187+0000","last_peered":"2024-09-16T09:32:54.229187+0000","last_clean":"2024-09-16T09:32:54.229187+0000","last_became_active":"2024-09-16T09:30:10.705072+0000","last_became_peered":"2024-09-16T09:30:10.705072+0000","last_unstale":"2024-09-16T09:32:54.229187+0000","last_undegraded":"2024-09-16T09:32:54.229187+0000","last_fullsized":"2024-09-16T09:32:54.229187+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:32:43.091969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:32:43.091969+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:45.325012+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,3],"acting":[4,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.7","version":"218'14150","reported_seq":14294,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:57.941073+0000","last_change":"2024-09-16T09:29:49.839740+0000","last_active":"2024-09-16T09:32:57.941073+0000","last_peered":"2024-09-16T09:32:57.941073+0000","last_clean":"2024-09-16T09:32:57.941073+0000","last_became_active":"2024-09-16T09:29:49.839437+0000","last_became_peered":"2024-09-16T09:29:49.839437+0000","last_unstale":"2024-09-16T09:32:57.941073+0000","last_undegraded":"2024-09-16T09:32:57.941073+0000","last_fullsized":"2024-09-16T09:32:57.941073+0000","mapping_epoch":152,"log_start":"200'9600","ondisk_log_start":"200'9600","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":4550,"log_dups_size":0,"ondisk_log_size":4550,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":84803584,"num_objects":1294,"num_object_clones":0,"num_object_copies":3882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1294,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14150,"num_write_kb":494208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"66'9","reported_seq":369,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.235510+0000","last_change":"2024-09-16T09:31:31.211495+0000","last_active":"2024-09-16T09:32:54.235510+0000","last_peered":"2024-09-16T09:32:54.235510+0000","last_clean":"2024-09-16T09:32:54.235510+0000","last_became_active":"2024-09-16T09:25:03.712230+0000","last_became_peered":"2024-09-16T09:25:03.712230+0000","last_unstale":"2024-09-16T09:32:54.235510+0000","last_undegraded":"2024-09-16T09:32:54.235510+0000","last_fullsized":"2024-09-16T09:32:54.235510+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:31:31.211420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:31:31.211420+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:53.471702+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":354,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.234037+0000","last_change":"2024-09-16T09:31:09.277413+0000","last_active":"2024-09-16T09:32:54.234037+0000","last_peered":"2024-09-16T09:32:54.234037+0000","last_clean":"2024-09-16T09:32:54.234037+0000","last_became_active":"2024-09-16T09:31:09.276516+0000","last_became_peered":"2024-09-16T09:31:09.276516+0000","last_unstale":"2024-09-16T09:32:54.234037+0000","last_undegraded":"2024-09-16T09:32:54.234037+0000","last_fullsized":"2024-09-16T09:32:54.234037+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:26.061106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:26.061106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":355,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.239922+0000","last_change":"2024-09-16T09:31:09.275219+0000","last_active":"2024-09-16T09:32:54.239922+0000","last_peered":"2024-09-16T09:32:54.239922+0000","last_clean":"2024-09-16T09:32:54.239922+0000","last_became_active":"2024-09-16T09:31:09.274702+0000","last_became_peered":"2024-09-16T09:31:09.274702+0000","last_unstale":"2024-09-16T09:32:54.239922+0000","last_undegraded":"2024-09-16T09:32:54.239922+0000","last_fullsized":"2024-09-16T09:32:54.239922+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:57.220068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:28:57.220068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:26.950321+0000","scrub_duration":33,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":345,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.237036+0000","last_change":"2024-09-16T09:31:27.185957+0000","last_active":"2024-09-16T09:32:54.237036+0000","last_peered":"2024-09-16T09:32:54.237036+0000","last_clean":"2024-09-16T09:32:54.237036+0000","last_became_active":"2024-09-16T09:25:07.718778+0000","last_became_peered":"2024-09-16T09:25:07.718778+0000","last_unstale":"2024-09-16T09:32:54.237036+0000","last_undegraded":"2024-09-16T09:32:54.237036+0000","last_fullsized":"2024-09-16T09:32:54.237036+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:31:27.185801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:31:27.185801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.f","version":"73'79","reported_seq":398,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.239231+0000","last_change":"2024-09-16T09:32:44.532983+0000","last_active":"2024-09-16T09:32:54.239231+0000","last_peered":"2024-09-16T09:32:54.239231+0000","last_clean":"2024-09-16T09:32:54.239231+0000","last_became_active":"2024-09-16T09:30:10.693477+0000","last_became_peered":"2024-09-16T09:30:10.693477+0000","last_unstale":"2024-09-16T09:32:54.239231+0000","last_undegraded":"2024-09-16T09:32:54.239231+0000","last_fullsized":"2024-09-16T09:32:54.239231+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:32:44.532922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:32:44.532922+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:02.098440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.6","version":"218'14566","reported_seq":14748,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:58.212006+0000","last_change":"2024-09-16T09:32:36.526082+0000","last_active":"2024-09-16T09:32:58.212006+0000","last_peered":"2024-09-16T09:32:58.212006+0000","last_clean":"2024-09-16T09:32:58.212006+0000","last_became_active":"2024-09-16T09:29:50.008656+0000","last_became_peered":"2024-09-16T09:29:50.008656+0000","last_unstale":"2024-09-16T09:32:58.212006+0000","last_undegraded":"2024-09-16T09:32:58.212006+0000","last_fullsized":"2024-09-16T09:32:58.212006+0000","mapping_epoch":152,"log_start":"197'9400","ondisk_log_start":"197'9400","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"209'12966","last_scrub_stamp":"2024-09-16T09:32:36.526001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:32:36.526001+0000","objects_scrubbed":2905,"log_size":5166,"log_dups_size":0,"ondisk_log_size":5166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:59.711611+0000","scrub_duration":1127,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":82313216,"num_objects":1256,"num_object_clones":0,"num_object_copies":3768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1256,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14566,"num_write_kb":506304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"66'19","reported_seq":376,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.238720+0000","last_change":"2024-09-16T09:29:02.554357+0000","last_active":"2024-09-16T09:32:54.238720+0000","last_peered":"2024-09-16T09:32:54.238720+0000","last_clean":"2024-09-16T09:32:54.238720+0000","last_became_active":"2024-09-16T09:25:03.712783+0000","last_became_peered":"2024-09-16T09:25:03.712783+0000","last_unstale":"2024-09-16T09:32:54.238720+0000","last_undegraded":"2024-09-16T09:32:54.238720+0000","last_fullsized":"2024-09-16T09:32:54.238720+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-16T09:29:02.554274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:29:02.554274+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:21.554123+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"59'1","reported_seq":354,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.238115+0000","last_change":"2024-09-16T09:29:51.025505+0000","last_active":"2024-09-16T09:32:54.238115+0000","last_peered":"2024-09-16T09:32:54.238115+0000","last_clean":"2024-09-16T09:32:54.238115+0000","last_became_active":"2024-09-16T09:25:01.708069+0000","last_became_peered":"2024-09-16T09:25:01.708069+0000","last_unstale":"2024-09-16T09:32:54.238115+0000","last_undegraded":"2024-09-16T09:32:54.238115+0000","last_fullsized":"2024-09-16T09:32:54.238115+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:29:51.025444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:51.025444+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":333,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.244694+0000","last_change":"2024-09-16T09:27:11.067059+0000","last_active":"2024-09-16T09:32:54.244694+0000","last_peered":"2024-09-16T09:32:54.244694+0000","last_clean":"2024-09-16T09:32:54.244694+0000","last_became_active":"2024-09-16T09:25:05.709264+0000","last_became_peered":"2024-09-16T09:25:05.709264+0000","last_unstale":"2024-09-16T09:32:54.244694+0000","last_undegraded":"2024-09-16T09:32:54.244694+0000","last_fullsized":"2024-09-16T09:32:54.244694+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:11.066986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:11.066986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:18.935169+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":328,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.237555+0000","last_change":"2024-09-16T09:27:25.865107+0000","last_active":"2024-09-16T09:32:54.237555+0000","last_peered":"2024-09-16T09:32:54.237555+0000","last_clean":"2024-09-16T09:32:54.237555+0000","last_became_active":"2024-09-16T09:25:07.718750+0000","last_became_peered":"2024-09-16T09:25:07.718750+0000","last_unstale":"2024-09-16T09:32:54.237555+0000","last_undegraded":"2024-09-16T09:32:54.237555+0000","last_fullsized":"2024-09-16T09:32:54.237555+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:25.865030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:25.865030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.c","version":"73'79","reported_seq":78,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.237116+0000","last_change":"2024-09-16T09:31:09.290919+0000","last_active":"2024-09-16T09:32:54.237116+0000","last_peered":"2024-09-16T09:32:54.237116+0000","last_clean":"2024-09-16T09:32:54.237116+0000","last_became_active":"2024-09-16T09:31:09.273770+0000","last_became_peered":"2024-09-16T09:31:09.273770+0000","last_unstale":"2024-09-16T09:32:54.237116+0000","last_undegraded":"2024-09-16T09:32:54.237116+0000","last_fullsized":"2024-09-16T09:32:54.237116+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:30:22.614462+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:30:22.614462+0000","objects_scrubbed":1,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":327680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.5","version":"218'14425","reported_seq":14642,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:57.598315+0000","last_change":"2024-09-16T09:31:06.121217+0000","last_active":"2024-09-16T09:32:57.598315+0000","last_peered":"2024-09-16T09:32:57.598315+0000","last_clean":"2024-09-16T09:32:57.598315+0000","last_became_active":"2024-09-16T09:29:50.009013+0000","last_became_peered":"2024-09-16T09:29:50.009013+0000","last_unstale":"2024-09-16T09:32:57.598315+0000","last_undegraded":"2024-09-16T09:32:57.598315+0000","last_fullsized":"2024-09-16T09:32:57.598315+0000","mapping_epoch":152,"log_start":"197'9113","ondisk_log_start":"197'9113","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"184'6587","last_scrub_stamp":"2024-09-16T09:31:06.120990+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:31:06.120990+0000","objects_scrubbed":6523,"log_size":5312,"log_dups_size":0,"ondisk_log_size":5312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:25.588988+0000","scrub_duration":1690,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":86441984,"num_objects":1319,"num_object_clones":0,"num_object_copies":3957,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1319,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14425,"num_write_kb":503808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"66'12","reported_seq":406,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.231021+0000","last_change":"2024-09-16T09:31:09.296915+0000","last_active":"2024-09-16T09:32:54.231021+0000","last_peered":"2024-09-16T09:32:54.231021+0000","last_clean":"2024-09-16T09:32:54.231021+0000","last_became_active":"2024-09-16T09:31:09.269047+0000","last_became_peered":"2024-09-16T09:31:09.269047+0000","last_unstale":"2024-09-16T09:32:54.231021+0000","last_undegraded":"2024-09-16T09:32:54.231021+0000","last_fullsized":"2024-09-16T09:32:54.231021+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-16T09:30:21.728668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:30:21.728668+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":348,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.237935+0000","last_change":"2024-09-16T09:29:53.075288+0000","last_active":"2024-09-16T09:32:54.237935+0000","last_peered":"2024-09-16T09:32:54.237935+0000","last_clean":"2024-09-16T09:32:54.237935+0000","last_became_active":"2024-09-16T09:25:01.707528+0000","last_became_peered":"2024-09-16T09:25:01.707528+0000","last_unstale":"2024-09-16T09:32:54.237935+0000","last_undegraded":"2024-09-16T09:32:54.237935+0000","last_fullsized":"2024-09-16T09:32:54.237935+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:53.075232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:53.075232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":341,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.239348+0000","last_change":"2024-09-16T09:27:49.965628+0000","last_active":"2024-09-16T09:32:54.239348+0000","last_peered":"2024-09-16T09:32:54.239348+0000","last_clean":"2024-09-16T09:32:54.239348+0000","last_became_active":"2024-09-16T09:25:05.715253+0000","last_became_peered":"2024-09-16T09:25:05.715253+0000","last_unstale":"2024-09-16T09:32:54.239348+0000","last_undegraded":"2024-09-16T09:32:54.239348+0000","last_fullsized":"2024-09-16T09:32:54.239348+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:49.965424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:49.965424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:20.990618+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":329,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.240962+0000","last_change":"2024-09-16T09:27:56.059755+0000","last_active":"2024-09-16T09:32:54.240962+0000","last_peered":"2024-09-16T09:32:54.240962+0000","last_clean":"2024-09-16T09:32:54.240962+0000","last_became_active":"2024-09-16T09:25:07.718899+0000","last_became_peered":"2024-09-16T09:25:07.718899+0000","last_unstale":"2024-09-16T09:32:54.240962+0000","last_undegraded":"2024-09-16T09:32:54.240962+0000","last_fullsized":"2024-09-16T09:32:54.240962+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:56.059687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:56.059687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.d","version":"73'79","reported_seq":146,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.236409+0000","last_change":"2024-09-16T09:32:44.536868+0000","last_active":"2024-09-16T09:32:54.236409+0000","last_peered":"2024-09-16T09:32:54.236409+0000","last_clean":"2024-09-16T09:32:54.236409+0000","last_became_active":"2024-09-16T09:30:10.709958+0000","last_became_peered":"2024-09-16T09:30:10.709958+0000","last_unstale":"2024-09-16T09:32:54.236409+0000","last_undegraded":"2024-09-16T09:32:54.236409+0000","last_fullsized":"2024-09-16T09:32:54.236409+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:32:44.536799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:32:44.536799+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:04.134689+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.4","version":"218'14322","reported_seq":14466,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:58.000414+0000","last_change":"2024-09-16T09:29:49.444857+0000","last_active":"2024-09-16T09:32:58.000414+0000","last_peered":"2024-09-16T09:32:58.000414+0000","last_clean":"2024-09-16T09:32:58.000414+0000","last_became_active":"2024-09-16T09:29:49.444515+0000","last_became_peered":"2024-09-16T09:29:49.444515+0000","last_unstale":"2024-09-16T09:32:58.000414+0000","last_undegraded":"2024-09-16T09:32:58.000414+0000","last_fullsized":"2024-09-16T09:32:58.000414+0000","mapping_epoch":152,"log_start":"199'9600","ondisk_log_start":"199'9600","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:29:48.436141+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.436141+0000","objects_scrubbed":0,"log_size":4722,"log_dups_size":0,"ondisk_log_size":4722,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":84279296,"num_objects":1286,"num_object_clones":0,"num_object_copies":3858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1286,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14322,"num_write_kb":499456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"66'15","reported_seq":361,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.233348+0000","last_change":"2024-09-16T09:26:33.138828+0000","last_active":"2024-09-16T09:32:54.233348+0000","last_peered":"2024-09-16T09:32:54.233348+0000","last_clean":"2024-09-16T09:32:54.233348+0000","last_became_active":"2024-09-16T09:25:03.715733+0000","last_became_peered":"2024-09-16T09:25:03.715733+0000","last_unstale":"2024-09-16T09:32:54.233348+0000","last_undegraded":"2024-09-16T09:32:54.233348+0000","last_fullsized":"2024-09-16T09:32:54.233348+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-16T09:26:33.138603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:26:33.138603+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":360,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.237703+0000","last_change":"2024-09-16T09:32:49.441194+0000","last_active":"2024-09-16T09:32:54.237703+0000","last_peered":"2024-09-16T09:32:54.237703+0000","last_clean":"2024-09-16T09:32:54.237703+0000","last_became_active":"2024-09-16T09:25:01.707372+0000","last_became_peered":"2024-09-16T09:25:01.707372+0000","last_unstale":"2024-09-16T09:32:54.237703+0000","last_undegraded":"2024-09-16T09:32:54.237703+0000","last_fullsized":"2024-09-16T09:32:54.237703+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-16T09:32:49.441122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:32:49.441122+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:17.431142+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"66'8","reported_seq":452,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.236125+0000","last_change":"2024-09-16T09:30:05.138362+0000","last_active":"2024-09-16T09:32:54.236125+0000","last_peered":"2024-09-16T09:32:54.236125+0000","last_clean":"2024-09-16T09:32:54.236125+0000","last_became_active":"2024-09-16T09:25:05.716189+0000","last_became_peered":"2024-09-16T09:25:05.716189+0000","last_unstale":"2024-09-16T09:32:54.236125+0000","last_undegraded":"2024-09-16T09:32:54.236125+0000","last_fullsized":"2024-09-16T09:32:54.236125+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:30:05.138288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:30:05.138288+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":328,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.233373+0000","last_change":"2024-09-16T09:27:20.219376+0000","last_active":"2024-09-16T09:32:54.233373+0000","last_peered":"2024-09-16T09:32:54.233373+0000","last_clean":"2024-09-16T09:32:54.233373+0000","last_became_active":"2024-09-16T09:25:07.717213+0000","last_became_peered":"2024-09-16T09:25:07.717213+0000","last_unstale":"2024-09-16T09:32:54.233373+0000","last_undegraded":"2024-09-16T09:32:54.233373+0000","last_fullsized":"2024-09-16T09:32:54.233373+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:20.219333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:20.219333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.12","version":"73'79","reported_seq":146,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.233621+0000","last_change":"2024-09-16T09:32:45.399435+0000","last_active":"2024-09-16T09:32:54.233621+0000","last_peered":"2024-09-16T09:32:54.233621+0000","last_clean":"2024-09-16T09:32:54.233621+0000","last_became_active":"2024-09-16T09:30:10.731413+0000","last_became_peered":"2024-09-16T09:30:10.731413+0000","last_unstale":"2024-09-16T09:32:54.233621+0000","last_undegraded":"2024-09-16T09:32:54.233621+0000","last_fullsized":"2024-09-16T09:32:54.233621+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:32:45.399361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:32:45.399361+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:01.532310+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":341,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.237448+0000","last_change":"2024-09-16T09:26:33.139405+0000","last_active":"2024-09-16T09:32:54.237448+0000","last_peered":"2024-09-16T09:32:54.237448+0000","last_clean":"2024-09-16T09:32:54.237448+0000","last_became_active":"2024-09-16T09:25:01.703202+0000","last_became_peered":"2024-09-16T09:25:01.703202+0000","last_unstale":"2024-09-16T09:32:54.237448+0000","last_undegraded":"2024-09-16T09:32:54.237448+0000","last_fullsized":"2024-09-16T09:32:54.237448+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:33.139360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:33.139360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:10.581208+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"66'6","reported_seq":90,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.237633+0000","last_change":"2024-09-16T09:32:47.851855+0000","last_active":"2024-09-16T09:32:54.237633+0000","last_peered":"2024-09-16T09:32:54.237633+0000","last_clean":"2024-09-16T09:32:54.237633+0000","last_became_active":"2024-09-16T09:31:09.270685+0000","last_became_peered":"2024-09-16T09:31:09.270685+0000","last_unstale":"2024-09-16T09:32:54.237633+0000","last_undegraded":"2024-09-16T09:32:54.237633+0000","last_fullsized":"2024-09-16T09:32:54.237633+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'6","last_scrub_stamp":"2024-09-16T09:32:47.851772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:32:47.851772+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:58.501874+0000","scrub_duration":28,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":8,"num_read_kb":5,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":340,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.233524+0000","last_change":"2024-09-16T09:29:31.496374+0000","last_active":"2024-09-16T09:32:54.233524+0000","last_peered":"2024-09-16T09:32:54.233524+0000","last_clean":"2024-09-16T09:32:54.233524+0000","last_became_active":"2024-09-16T09:25:05.715616+0000","last_became_peered":"2024-09-16T09:25:05.715616+0000","last_unstale":"2024-09-16T09:32:54.233524+0000","last_undegraded":"2024-09-16T09:32:54.233524+0000","last_fullsized":"2024-09-16T09:32:54.233524+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:31.496304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:31.496304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":328,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.237690+0000","last_change":"2024-09-16T09:26:28.733381+0000","last_active":"2024-09-16T09:32:54.237690+0000","last_peered":"2024-09-16T09:32:54.237690+0000","last_clean":"2024-09-16T09:32:54.237690+0000","last_became_active":"2024-09-16T09:25:07.721059+0000","last_became_peered":"2024-09-16T09:25:07.721059+0000","last_unstale":"2024-09-16T09:32:54.237690+0000","last_undegraded":"2024-09-16T09:32:54.237690+0000","last_fullsized":"2024-09-16T09:32:54.237690+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:28.733324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:28.733324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:52.397089+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.13","version":"73'79","reported_seq":324,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.249918+0000","last_change":"2024-09-16T09:31:09.288677+0000","last_active":"2024-09-16T09:32:54.249918+0000","last_peered":"2024-09-16T09:32:54.249918+0000","last_clean":"2024-09-16T09:32:54.249918+0000","last_became_active":"2024-09-16T09:31:09.288233+0000","last_became_peered":"2024-09-16T09:31:09.288233+0000","last_unstale":"2024-09-16T09:32:54.249918+0000","last_undegraded":"2024-09-16T09:32:54.249918+0000","last_fullsized":"2024-09-16T09:32:54.249918+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:30:21.714369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:30:21.714369+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.16","version":"66'9","reported_seq":411,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.235064+0000","last_change":"2024-09-16T09:32:01.319481+0000","last_active":"2024-09-16T09:32:54.235064+0000","last_peered":"2024-09-16T09:32:54.235064+0000","last_clean":"2024-09-16T09:32:54.235064+0000","last_became_active":"2024-09-16T09:31:09.273498+0000","last_became_peered":"2024-09-16T09:31:09.273498+0000","last_unstale":"2024-09-16T09:32:54.235064+0000","last_undegraded":"2024-09-16T09:32:54.235064+0000","last_fullsized":"2024-09-16T09:32:54.235064+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:32:01.319390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:32:01.319390+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:29.275572+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":348,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.236574+0000","last_change":"2024-09-16T09:28:50.698331+0000","last_active":"2024-09-16T09:32:54.236574+0000","last_peered":"2024-09-16T09:32:54.236574+0000","last_clean":"2024-09-16T09:32:54.236574+0000","last_became_active":"2024-09-16T09:25:01.706491+0000","last_became_peered":"2024-09-16T09:25:01.706491+0000","last_unstale":"2024-09-16T09:32:54.236574+0000","last_undegraded":"2024-09-16T09:32:54.236574+0000","last_fullsized":"2024-09-16T09:32:54.236574+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:50.698288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:28:50.698288+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":198,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.240747+0000","last_change":"2024-09-16T09:31:09.271558+0000","last_active":"2024-09-16T09:32:54.240747+0000","last_peered":"2024-09-16T09:32:54.240747+0000","last_clean":"2024-09-16T09:32:54.240747+0000","last_became_active":"2024-09-16T09:31:09.271223+0000","last_became_peered":"2024-09-16T09:31:09.271223+0000","last_unstale":"2024-09-16T09:32:54.240747+0000","last_undegraded":"2024-09-16T09:32:54.240747+0000","last_fullsized":"2024-09-16T09:32:54.240747+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:53.810580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:28:53.810580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":329,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.241508+0000","last_change":"2024-09-16T09:26:35.866582+0000","last_active":"2024-09-16T09:32:54.241508+0000","last_peered":"2024-09-16T09:32:54.241508+0000","last_clean":"2024-09-16T09:32:54.241508+0000","last_became_active":"2024-09-16T09:25:07.722050+0000","last_became_peered":"2024-09-16T09:25:07.722050+0000","last_unstale":"2024-09-16T09:32:54.241508+0000","last_undegraded":"2024-09-16T09:32:54.241508+0000","last_fullsized":"2024-09-16T09:32:54.241508+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:35.866512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:35.866512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:17.938610+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.10","version":"73'79","reported_seq":146,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.233041+0000","last_change":"2024-09-16T09:32:44.405696+0000","last_active":"2024-09-16T09:32:54.233041+0000","last_peered":"2024-09-16T09:32:54.233041+0000","last_clean":"2024-09-16T09:32:54.233041+0000","last_became_active":"2024-09-16T09:30:10.692770+0000","last_became_peered":"2024-09-16T09:30:10.692770+0000","last_unstale":"2024-09-16T09:32:54.233041+0000","last_undegraded":"2024-09-16T09:32:54.233041+0000","last_fullsized":"2024-09-16T09:32:54.233041+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:32:44.405640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:32:44.405640+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:52.861751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"66'9","reported_seq":360,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.233921+0000","last_change":"2024-09-16T09:29:32.485165+0000","last_active":"2024-09-16T09:32:54.233921+0000","last_peered":"2024-09-16T09:32:54.233921+0000","last_clean":"2024-09-16T09:32:54.233921+0000","last_became_active":"2024-09-16T09:25:03.717310+0000","last_became_peered":"2024-09-16T09:25:03.717310+0000","last_unstale":"2024-09-16T09:32:54.233921+0000","last_undegraded":"2024-09-16T09:32:54.233921+0000","last_fullsized":"2024-09-16T09:32:54.233921+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:29:32.485084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:29:32.485084+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":371,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.236087+0000","last_change":"2024-09-16T09:31:59.376923+0000","last_active":"2024-09-16T09:32:54.236087+0000","last_peered":"2024-09-16T09:32:54.236087+0000","last_clean":"2024-09-16T09:32:54.236087+0000","last_became_active":"2024-09-16T09:31:09.272848+0000","last_became_peered":"2024-09-16T09:31:09.272848+0000","last_unstale":"2024-09-16T09:32:54.236087+0000","last_undegraded":"2024-09-16T09:32:54.236087+0000","last_fullsized":"2024-09-16T09:32:54.236087+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:31:59.376749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:31:59.376749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:13.111415+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"187'11","reported_seq":281,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.237385+0000","last_change":"2024-09-16T09:31:09.311243+0000","last_active":"2024-09-16T09:32:54.237385+0000","last_peered":"2024-09-16T09:32:54.237385+0000","last_clean":"2024-09-16T09:32:54.237385+0000","last_became_active":"2024-09-16T09:31:09.277688+0000","last_became_peered":"2024-09-16T09:31:09.277688+0000","last_unstale":"2024-09-16T09:32:54.237385+0000","last_undegraded":"2024-09-16T09:32:54.237385+0000","last_fullsized":"2024-09-16T09:32:54.237385+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:28:53.714413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:28:53.714413+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":345,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.229224+0000","last_change":"2024-09-16T09:29:49.474440+0000","last_active":"2024-09-16T09:32:54.229224+0000","last_peered":"2024-09-16T09:32:54.229224+0000","last_clean":"2024-09-16T09:32:54.229224+0000","last_became_active":"2024-09-16T09:25:07.718810+0000","last_became_peered":"2024-09-16T09:25:07.718810+0000","last_unstale":"2024-09-16T09:32:54.229224+0000","last_undegraded":"2024-09-16T09:32:54.229224+0000","last_fullsized":"2024-09-16T09:32:54.229224+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:49.474391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:49.474391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.11","version":"73'79","reported_seq":232,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.237207+0000","last_change":"2024-09-16T09:32:38.401888+0000","last_active":"2024-09-16T09:32:54.237207+0000","last_peered":"2024-09-16T09:32:54.237207+0000","last_clean":"2024-09-16T09:32:54.237207+0000","last_became_active":"2024-09-16T09:31:09.272208+0000","last_became_peered":"2024-09-16T09:31:09.272208+0000","last_unstale":"2024-09-16T09:32:54.237207+0000","last_undegraded":"2024-09-16T09:32:54.237207+0000","last_fullsized":"2024-09-16T09:32:54.237207+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:32:38.401720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:32:38.401720+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:46.010237+0000","scrub_duration":32,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.14","version":"66'10","reported_seq":219,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.238834+0000","last_change":"2024-09-16T09:28:56.825675+0000","last_active":"2024-09-16T09:32:54.238834+0000","last_peered":"2024-09-16T09:32:54.238834+0000","last_clean":"2024-09-16T09:32:54.238834+0000","last_became_active":"2024-09-16T09:28:52.979059+0000","last_became_peered":"2024-09-16T09:28:52.979059+0000","last_unstale":"2024-09-16T09:32:54.238834+0000","last_undegraded":"2024-09-16T09:32:54.238834+0000","last_fullsized":"2024-09-16T09:32:54.238834+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-16T09:28:56.825476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:56.825476+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":14,"num_read_kb":9,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":348,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.236831+0000","last_change":"2024-09-16T09:29:56.021413+0000","last_active":"2024-09-16T09:32:54.236831+0000","last_peered":"2024-09-16T09:32:54.236831+0000","last_clean":"2024-09-16T09:32:54.236831+0000","last_became_active":"2024-09-16T09:25:01.707513+0000","last_became_peered":"2024-09-16T09:25:01.707513+0000","last_unstale":"2024-09-16T09:32:54.236831+0000","last_undegraded":"2024-09-16T09:32:54.236831+0000","last_fullsized":"2024-09-16T09:32:54.236831+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:56.021340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:56.021340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"66'8","reported_seq":452,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.233763+0000","last_change":"2024-09-16T09:29:48.485781+0000","last_active":"2024-09-16T09:32:54.233763+0000","last_peered":"2024-09-16T09:32:54.233763+0000","last_clean":"2024-09-16T09:32:54.233763+0000","last_became_active":"2024-09-16T09:25:05.713833+0000","last_became_peered":"2024-09-16T09:25:05.713833+0000","last_unstale":"2024-09-16T09:32:54.233763+0000","last_undegraded":"2024-09-16T09:32:54.233763+0000","last_fullsized":"2024-09-16T09:32:54.233763+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:29:48.485727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.485727+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":359,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.234579+0000","last_change":"2024-09-16T09:32:00.343429+0000","last_active":"2024-09-16T09:32:54.234579+0000","last_peered":"2024-09-16T09:32:54.234579+0000","last_clean":"2024-09-16T09:32:54.234579+0000","last_became_active":"2024-09-16T09:31:09.275168+0000","last_became_peered":"2024-09-16T09:31:09.275168+0000","last_unstale":"2024-09-16T09:32:54.234579+0000","last_undegraded":"2024-09-16T09:32:54.234579+0000","last_fullsized":"2024-09-16T09:32:54.234579+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:00.343356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:32:00.343356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:23.568990+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"66'11","reported_seq":367,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.229350+0000","last_change":"2024-09-16T09:26:33.302713+0000","last_active":"2024-09-16T09:32:54.229350+0000","last_peered":"2024-09-16T09:32:54.229350+0000","last_clean":"2024-09-16T09:32:54.229350+0000","last_became_active":"2024-09-16T09:26:33.291265+0000","last_became_peered":"2024-09-16T09:26:33.291265+0000","last_unstale":"2024-09-16T09:32:54.229350+0000","last_undegraded":"2024-09-16T09:32:54.229350+0000","last_fullsized":"2024-09-16T09:32:54.229350+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:25:02.691904+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:08.864166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":357,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.230665+0000","last_change":"2024-09-16T09:29:51.517112+0000","last_active":"2024-09-16T09:32:54.230665+0000","last_peered":"2024-09-16T09:32:54.230665+0000","last_clean":"2024-09-16T09:32:54.230665+0000","last_became_active":"2024-09-16T09:25:01.705158+0000","last_became_peered":"2024-09-16T09:25:01.705158+0000","last_unstale":"2024-09-16T09:32:54.230665+0000","last_undegraded":"2024-09-16T09:32:54.230665+0000","last_fullsized":"2024-09-16T09:32:54.230665+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:51.517054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:51.517054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":363,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.238668+0000","last_change":"2024-09-16T09:31:09.266820+0000","last_active":"2024-09-16T09:32:54.238668+0000","last_peered":"2024-09-16T09:32:54.238668+0000","last_clean":"2024-09-16T09:32:54.238668+0000","last_became_active":"2024-09-16T09:31:09.266326+0000","last_became_peered":"2024-09-16T09:31:09.266326+0000","last_unstale":"2024-09-16T09:32:54.238668+0000","last_undegraded":"2024-09-16T09:32:54.238668+0000","last_fullsized":"2024-09-16T09:32:54.238668+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:15.910824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:15.910824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":191,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.236945+0000","last_change":"2024-09-16T09:31:09.277239+0000","last_active":"2024-09-16T09:32:54.236945+0000","last_peered":"2024-09-16T09:32:54.236945+0000","last_clean":"2024-09-16T09:32:54.236945+0000","last_became_active":"2024-09-16T09:31:09.276042+0000","last_became_peered":"2024-09-16T09:31:09.276042+0000","last_unstale":"2024-09-16T09:32:54.236945+0000","last_undegraded":"2024-09-16T09:32:54.236945+0000","last_fullsized":"2024-09-16T09:32:54.236945+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:48.147759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:48.147759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:29.306846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"66'9","reported_seq":361,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.238940+0000","last_change":"2024-09-16T09:32:47.247161+0000","last_active":"2024-09-16T09:32:54.238940+0000","last_peered":"2024-09-16T09:32:54.238940+0000","last_clean":"2024-09-16T09:32:54.238940+0000","last_became_active":"2024-09-16T09:25:03.705807+0000","last_became_peered":"2024-09-16T09:25:03.705807+0000","last_unstale":"2024-09-16T09:32:54.238940+0000","last_undegraded":"2024-09-16T09:32:54.238940+0000","last_fullsized":"2024-09-16T09:32:54.238940+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:32:47.247071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:32:47.247071+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:08.145823+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":348,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.238026+0000","last_change":"2024-09-16T09:32:47.502667+0000","last_active":"2024-09-16T09:32:54.238026+0000","last_peered":"2024-09-16T09:32:54.238026+0000","last_clean":"2024-09-16T09:32:54.238026+0000","last_became_active":"2024-09-16T09:25:01.706762+0000","last_became_peered":"2024-09-16T09:25:01.706762+0000","last_unstale":"2024-09-16T09:32:54.238026+0000","last_undegraded":"2024-09-16T09:32:54.238026+0000","last_fullsized":"2024-09-16T09:32:54.238026+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:47.502598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:32:47.502598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:03.669451+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":176,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.235236+0000","last_change":"2024-09-16T09:28:52.983144+0000","last_active":"2024-09-16T09:32:54.235236+0000","last_peered":"2024-09-16T09:32:54.235236+0000","last_clean":"2024-09-16T09:32:54.235236+0000","last_became_active":"2024-09-16T09:28:52.982865+0000","last_became_peered":"2024-09-16T09:28:52.982865+0000","last_unstale":"2024-09-16T09:32:54.235236+0000","last_undegraded":"2024-09-16T09:32:54.235236+0000","last_fullsized":"2024-09-16T09:32:54.235236+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:44.241942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:44.241942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:28.290345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":337,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.233818+0000","last_change":"2024-09-16T09:28:52.552567+0000","last_active":"2024-09-16T09:32:54.233818+0000","last_peered":"2024-09-16T09:32:54.233818+0000","last_clean":"2024-09-16T09:32:54.233818+0000","last_became_active":"2024-09-16T09:25:07.715301+0000","last_became_peered":"2024-09-16T09:25:07.715301+0000","last_unstale":"2024-09-16T09:32:54.233818+0000","last_undegraded":"2024-09-16T09:32:54.233818+0000","last_fullsized":"2024-09-16T09:32:54.233818+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:52.552495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:28:52.552495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:50.499212+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.14","version":"73'79","reported_seq":146,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.241415+0000","last_change":"2024-09-16T09:32:44.163264+0000","last_active":"2024-09-16T09:32:54.241415+0000","last_peered":"2024-09-16T09:32:54.241415+0000","last_clean":"2024-09-16T09:32:54.241415+0000","last_became_active":"2024-09-16T09:30:10.700866+0000","last_became_peered":"2024-09-16T09:30:10.700866+0000","last_unstale":"2024-09-16T09:32:54.241415+0000","last_undegraded":"2024-09-16T09:32:54.241415+0000","last_fullsized":"2024-09-16T09:32:54.241415+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:32:44.163191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:32:44.163191+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:13.888037+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"66'11","reported_seq":223,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.236442+0000","last_change":"2024-09-16T09:28:55.643505+0000","last_active":"2024-09-16T09:32:54.236442+0000","last_peered":"2024-09-16T09:32:54.236442+0000","last_clean":"2024-09-16T09:32:54.236442+0000","last_became_active":"2024-09-16T09:28:52.971132+0000","last_became_peered":"2024-09-16T09:28:52.971132+0000","last_unstale":"2024-09-16T09:32:54.236442+0000","last_undegraded":"2024-09-16T09:32:54.236442+0000","last_fullsized":"2024-09-16T09:32:54.236442+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-16T09:28:55.643448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:55.643448+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":19,"num_read_kb":12,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":348,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.233765+0000","last_change":"2024-09-16T09:27:46.175159+0000","last_active":"2024-09-16T09:32:54.233765+0000","last_peered":"2024-09-16T09:32:54.233765+0000","last_clean":"2024-09-16T09:32:54.233765+0000","last_became_active":"2024-09-16T09:25:01.700733+0000","last_became_peered":"2024-09-16T09:25:01.700733+0000","last_unstale":"2024-09-16T09:32:54.233765+0000","last_undegraded":"2024-09-16T09:32:54.233765+0000","last_fullsized":"2024-09-16T09:32:54.233765+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:46.175093+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:27:46.175093+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":340,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.237045+0000","last_change":"2024-09-16T09:29:25.017736+0000","last_active":"2024-09-16T09:32:54.237045+0000","last_peered":"2024-09-16T09:32:54.237045+0000","last_clean":"2024-09-16T09:32:54.237045+0000","last_became_active":"2024-09-16T09:25:05.713479+0000","last_became_peered":"2024-09-16T09:25:05.713479+0000","last_unstale":"2024-09-16T09:32:54.237045+0000","last_undegraded":"2024-09-16T09:32:54.237045+0000","last_fullsized":"2024-09-16T09:32:54.237045+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:25.017610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:25.017610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":199,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.235665+0000","last_change":"2024-09-16T09:31:09.275121+0000","last_active":"2024-09-16T09:32:54.235665+0000","last_peered":"2024-09-16T09:32:54.235665+0000","last_clean":"2024-09-16T09:32:54.235665+0000","last_became_active":"2024-09-16T09:31:09.274797+0000","last_became_peered":"2024-09-16T09:31:09.274797+0000","last_unstale":"2024-09-16T09:32:54.235665+0000","last_undegraded":"2024-09-16T09:32:54.235665+0000","last_fullsized":"2024-09-16T09:32:54.235665+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:54.589258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:28:54.589258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:51.457590+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"66'4","reported_seq":88,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.238199+0000","last_change":"2024-09-16T09:32:48.847615+0000","last_active":"2024-09-16T09:32:54.238199+0000","last_peered":"2024-09-16T09:32:54.238199+0000","last_clean":"2024-09-16T09:32:54.238199+0000","last_became_active":"2024-09-16T09:31:09.273682+0000","last_became_peered":"2024-09-16T09:31:09.273682+0000","last_unstale":"2024-09-16T09:32:54.238199+0000","last_undegraded":"2024-09-16T09:32:54.238199+0000","last_fullsized":"2024-09-16T09:32:54.238199+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'4","last_scrub_stamp":"2024-09-16T09:32:48.847542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:32:48.847542+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:52.504702+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":371,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.235126+0000","last_change":"2024-09-16T09:31:30.235761+0000","last_active":"2024-09-16T09:32:54.235126+0000","last_peered":"2024-09-16T09:32:54.235126+0000","last_clean":"2024-09-16T09:32:54.235126+0000","last_became_active":"2024-09-16T09:31:09.274719+0000","last_became_peered":"2024-09-16T09:31:09.274719+0000","last_unstale":"2024-09-16T09:32:54.235126+0000","last_undegraded":"2024-09-16T09:32:54.235126+0000","last_fullsized":"2024-09-16T09:32:54.235126+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:31:30.235692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:31:30.235692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":357,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.238496+0000","last_change":"2024-09-16T09:32:07.533575+0000","last_active":"2024-09-16T09:32:54.238496+0000","last_peered":"2024-09-16T09:32:54.238496+0000","last_clean":"2024-09-16T09:32:54.238496+0000","last_became_active":"2024-09-16T09:25:05.712545+0000","last_became_peered":"2024-09-16T09:25:05.712545+0000","last_unstale":"2024-09-16T09:32:54.238496+0000","last_undegraded":"2024-09-16T09:32:54.238496+0000","last_fullsized":"2024-09-16T09:32:54.238496+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:07.533529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:32:07.533529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:28.807212+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":336,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.236932+0000","last_change":"2024-09-16T09:29:27.012810+0000","last_active":"2024-09-16T09:32:54.236932+0000","last_peered":"2024-09-16T09:32:54.236932+0000","last_clean":"2024-09-16T09:32:54.236932+0000","last_became_active":"2024-09-16T09:25:07.720117+0000","last_became_peered":"2024-09-16T09:25:07.720117+0000","last_unstale":"2024-09-16T09:32:54.236932+0000","last_undegraded":"2024-09-16T09:32:54.236932+0000","last_fullsized":"2024-09-16T09:32:54.236932+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:27.012756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:27.012756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":337,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.239211+0000","last_change":"2024-09-16T09:29:19.888825+0000","last_active":"2024-09-16T09:32:54.239211+0000","last_peered":"2024-09-16T09:32:54.239211+0000","last_clean":"2024-09-16T09:32:54.239211+0000","last_became_active":"2024-09-16T09:25:07.712594+0000","last_became_peered":"2024-09-16T09:25:07.712594+0000","last_unstale":"2024-09-16T09:32:54.239211+0000","last_undegraded":"2024-09-16T09:32:54.239211+0000","last_fullsized":"2024-09-16T09:32:54.239211+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:19.888759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:19.888759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":206,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.234740+0000","last_change":"2024-09-16T09:32:40.411546+0000","last_active":"2024-09-16T09:32:54.234740+0000","last_peered":"2024-09-16T09:32:54.234740+0000","last_clean":"2024-09-16T09:32:54.234740+0000","last_became_active":"2024-09-16T09:31:09.271662+0000","last_became_peered":"2024-09-16T09:31:09.271662+0000","last_unstale":"2024-09-16T09:32:54.234740+0000","last_undegraded":"2024-09-16T09:32:54.234740+0000","last_fullsized":"2024-09-16T09:32:54.234740+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:40.411487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:32:40.411487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:51.366030+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"66'11","reported_seq":364,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.239265+0000","last_change":"2024-09-16T09:27:57.486459+0000","last_active":"2024-09-16T09:32:54.239265+0000","last_peered":"2024-09-16T09:32:54.239265+0000","last_clean":"2024-09-16T09:32:54.239265+0000","last_became_active":"2024-09-16T09:25:03.707769+0000","last_became_peered":"2024-09-16T09:25:03.707769+0000","last_unstale":"2024-09-16T09:32:54.239265+0000","last_undegraded":"2024-09-16T09:32:54.239265+0000","last_fullsized":"2024-09-16T09:32:54.239265+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-16T09:27:57.486381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:57.486381+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:17.472949+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":341,"reported_epoch":218,"state":"active+clean","last_fresh":"2024-09-16T09:32:54.234831+0000","last_change":"2024-09-16T09:32:31.325543+0000","last_active":"2024-09-16T09:32:54.234831+0000","last_peered":"2024-09-16T09:32:54.234831+0000","last_clean":"2024-09-16T09:32:54.234831+0000","last_became_active":"2024-09-16T09:25:05.711952+0000","last_became_peered":"2024-09-16T09:25:05.711952+0000","last_unstale":"2024-09-16T09:32:54.234831+0000","last_undegraded":"2024-09-16T09:32:54.234831+0000","last_fullsized":"2024-09-16T09:32:54.234831+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:31.325346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:32:31.325346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:54.289339+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":16,"stat_sum":{"num_bytes":1337655320,"num_objects":20412,"num_object_clones":0,"num_object_copies":61236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20412,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":228774,"num_write_kb":7973889,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14709,"num_bytes_recovered":963969024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4576194560,"data_stored":4576182344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":77661,"ondisk_log_size":77661,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":70,"ondisk_log_size":70,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":691,"num_read_kb":448,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":126,"num_bytes_recovered":3026,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":395,"ondisk_log_size":395,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1986,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":470,"num_read_kb":465,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":21,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":853056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1659,"ondisk_log_size":1659,"up":63,"acting":63,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266788,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":674248,"kb_used_data":426424,"kb_used_omap":1,"kb_used_meta":247742,"kb_avail":93070904,"statfs":{"total":95995035648,"available":95304605696,"internally_reserved":0,"allocated":436658176,"data_stored":436308932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":253688266},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495721,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":639664,"kb_used_data":446044,"kb_used_omap":1,"kb_used_meta":193598,"kb_avail":93105488,"statfs":{"total":95995035648,"available":95340019712,"internally_reserved":0,"allocated":456749056,"data_stored":456427362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":198244806},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724656,"num_pgs":64,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":962940,"kb_used_data":679084,"kb_used_omap":1,"kb_used_meta":283838,"kb_avail":92782212,"statfs":{"total":95995035648,"available":95008985088,"internally_reserved":0,"allocated":695382016,"data_stored":695044975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":290650566},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,5,2,5],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953589,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":977864,"kb_used_data":707896,"kb_used_omap":1,"kb_used_meta":269950,"kb_avail":92767288,"statfs":{"total":95995035648,"available":94993702912,"internally_reserved":0,"allocated":724885504,"data_stored":724536300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":276429254},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,1,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149819,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":854164,"kb_used_data":493828,"kb_used_omap":1,"kb_used_meta":360190,"kb_avail":92890988,"statfs":{"total":95995035648,"available":95120371712,"internally_reserved":0,"allocated":505679872,"data_stored":505319210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":368835015},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411456,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":831284,"kb_used_data":516836,"kb_used_omap":1,"kb_used_meta":314366,"kb_avail":92913868,"statfs":{"total":95995035648,"available":95143800832,"internally_reserved":0,"allocated":529240064,"data_stored":528910678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":321911237},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574983,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":907680,"kb_used_data":593296,"kb_used_omap":1,"kb_used_meta":314366,"kb_avail":92837472,"statfs":{"total":95995035648,"available":95065571328,"internally_reserved":0,"allocated":607535104,"data_stored":607161194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":321911237},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,4,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738509,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1093768,"kb_used_data":765496,"kb_used_omap":1,"kb_used_meta":328254,"kb_avail":92651384,"statfs":{"total":95995035648,"available":94875017216,"internally_reserved":0,"allocated":783867904,"data_stored":783518902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":336132549},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,2,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":755302400,"data_stored":755302400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":585629696,"data_stored":585629696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":508497920,"data_stored":508493848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":495058944,"data_stored":495058944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":706281472,"data_stored":706281472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":664473600,"data_stored":664469528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":429916160,"data_stored":429916160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":431034368,"data_stored":431030296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T09:33:03.937 INFO:tasks.cephadm.ceph_manager.ceph:recovered! 2024-09-16T09:33:03.938 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T09:33:03.938 INFO:tasks.thrashosds.thrasher:inject_pause on osd.5 2024-09-16T09:33:03.938 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-16T09:33:03.938 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-16T09:33:03.938 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.5 config set filestore_inject_stall 3 2024-09-16T09:33:04.015 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:33:04.043 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:04.044 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:04.044 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:33:04.145 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:33:04.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:04.206 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:04.206 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:33:04.307 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:33:04.311 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:33:04.313 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:33:04.313 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:33:04.345 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:04.346 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:04.346 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:33:04.349 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T09:33:04.349 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:33:04.360 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:33:04.361 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:33:04.400 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:33:04.400 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:33:04.436 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:33:04.436 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:33:04.448 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:33:04.470 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:33:04.471 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:33:04.478 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:04.479 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:04.480 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:33:04.501 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:33:04.502 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:33:04.502 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:04 smithi070 ceph-mon[30388]: from='client.25201 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:33:04.502 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:04 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:33:04.502 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:04 smithi070 ceph-mon[30388]: osdmap e221: 8 total, 8 up, 8 in 2024-09-16T09:33:04.502 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:04 smithi070 ceph-mon[40691]: from='client.25201 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:33:04.503 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:04 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:33:04.503 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:04 smithi070 ceph-mon[40691]: osdmap e221: 8 total, 8 up, 8 in 2024-09-16T09:33:04.532 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:04 smithi084 ceph-mon[32629]: from='client.25201 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:33:04.533 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:04 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:33:04.533 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:04 smithi084 ceph-mon[32629]: osdmap e221: 8 total, 8 up, 8 in 2024-09-16T09:33:04.534 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:33:04.535 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:33:04.570 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:33:04.571 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:33:04.580 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:33:04.605 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:04.605 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:04.605 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:33:04.706 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:33:04.738 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:04.739 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:04.739 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:33:04.840 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:33:04.882 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:04.882 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:04.882 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:33:04.983 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:33:05.005 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-09-16T09:33:05.005 INFO:teuthology.orchestra.run.smithi084.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-16T09:33:05.006 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-09-16T09:33:05.023 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:05.023 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:05.024 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:33:05.125 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:33:05.160 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:05.161 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:05.161 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:33:05.262 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:33:05.299 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:05.299 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:05.300 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:33:05.400 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:33:05.438 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:05.438 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:05.439 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:33:05.540 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:33:05.576 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:05.576 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:05.577 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:33:05.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:05 smithi084 ceph-mon[32629]: pgmap v376: 168 pgs: 168 active+clean; 870 MiB data, 5.4 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.62k op/s 2024-09-16T09:33:05.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:05 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1400383938' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:33:05.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:05 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:33:05.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:05 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:33:05.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:05 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:33:05.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:05 smithi070 ceph-mon[30388]: pgmap v376: 168 pgs: 168 active+clean; 870 MiB data, 5.4 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.62k op/s 2024-09-16T09:33:05.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:05 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1400383938' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:33:05.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:05 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:33:05.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:05 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:33:05.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:05 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:33:05.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:05 smithi070 ceph-mon[40691]: pgmap v376: 168 pgs: 168 active+clean; 870 MiB data, 5.4 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.62k op/s 2024-09-16T09:33:05.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:05 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1400383938' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:33:05.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:05 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:33:05.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:05 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:33:05.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:05 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:33:05.678 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:33:05.712 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:05.713 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:05.713 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:33:05.814 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:33:05.847 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:05.847 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:05.848 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:33:05.948 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:33:06.001 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:06.002 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:06.002 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:33:06.103 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:33:06.138 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:06.139 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:06.139 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:33:06.240 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:33:06.273 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:06.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:06.274 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:33:06.375 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:33:06.409 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:06.410 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:06.410 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:33:06.446 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:33:06.460 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.4 dump_ops_in_flight 2024-09-16T09:33:06.511 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:33:06.554 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:06.555 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:06.555 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:33:06.656 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:33:06.695 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:06.695 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:06.695 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:33:06.796 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:33:06.836 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:06.836 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:06.837 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:33:06.868 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:33:06.938 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:33:06.967 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:06.967 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:06.967 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:33:07.068 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:33:07.109 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:07.110 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:07.110 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:33:07.211 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:33:07.253 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:07.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:07.254 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:33:07.355 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:33:07.391 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:07.392 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:07.392 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:33:07.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:07 smithi070 ceph-mon[30388]: pgmap v378: 168 pgs: 168 active+clean; 870 MiB data, 5.4 GiB used, 710 GiB / 715 GiB avail; 739 B/s rd, 0 B/s wr, 940 op/s 2024-09-16T09:33:07.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:07 smithi070 ceph-mon[40691]: pgmap v378: 168 pgs: 168 active+clean; 870 MiB data, 5.4 GiB used, 710 GiB / 715 GiB avail; 739 B/s rd, 0 B/s wr, 940 op/s 2024-09-16T09:33:07.493 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:33:07.515 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:07 smithi084 ceph-mon[32629]: pgmap v378: 168 pgs: 168 active+clean; 870 MiB data, 5.4 GiB used, 710 GiB / 715 GiB avail; 739 B/s rd, 0 B/s wr, 940 op/s 2024-09-16T09:33:07.516 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-09-16T09:33:07.516 INFO:teuthology.orchestra.run.smithi084.stdout: "ops": [ 2024-09-16T09:33:07.516 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:07.516 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.25064.0:241169 8.2 8:45ba1433:::benchmark_data_smithi070_107919_object116574:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e221)", 2024-09-16T09:33:07.516 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:33:07.492806+0000", 2024-09-16T09:33:07.517 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 0.020849375, 2024-09-16T09:33:07.517 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.020879108, 2024-09-16T09:33:07.517 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:33:07.517 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:33:07.517 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "waiting for sub ops", 2024-09-16T09:33:07.517 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:33:07.517 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.25064", 2024-09-16T09:33:07.517 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/2578256439", 2024-09-16T09:33:07.517 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 241169 2024-09-16T09:33:07.517 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:07.517 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:33:07.517 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:07.517 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:33:07.517 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:33:07.492806+0000", 2024-09-16T09:33:07.518 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:33:07.518 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:07.518 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:07.518 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:33:07.518 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:33:07.492806+0000", 2024-09-16T09:33:07.518 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:33:07.518 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:07.518 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:07.518 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:33:07.518 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:33:07.492806+0000", 2024-09-16T09:33:07.518 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.5499999999999998e-07 2024-09-16T09:33:07.518 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:07.518 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:07.519 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:33:07.519 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:33:07.492809+0000", 2024-09-16T09:33:07.519 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.6120000000000001e-06 2024-09-16T09:33:07.519 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:07.519 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:07.519 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:33:07.519 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:33:07.492818+0000", 2024-09-16T09:33:07.519 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.5470000000000002e-06 2024-09-16T09:33:07.519 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:07.519 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:07.519 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:33:07.519 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:33:07.492824+0000", 2024-09-16T09:33:07.519 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.1449999999999999e-06 2024-09-16T09:33:07.519 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:07.520 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:07.520 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:33:07.520 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:33:07.495251+0000", 2024-09-16T09:33:07.520 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0024272059999999999 2024-09-16T09:33:07.520 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:07.520 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:07.520 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:33:07.520 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:33:07.495360+0000", 2024-09-16T09:33:07.520 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00010875200000000001 2024-09-16T09:33:07.520 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:07.520 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:07.520 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,5", 2024-09-16T09:33:07.520 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:33:07.495467+0000", 2024-09-16T09:33:07.521 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000107191 2024-09-16T09:33:07.521 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:07.521 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:07.521 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:33:07.521 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:33:07.500498+0000", 2024-09-16T09:33:07.521 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0050315100000000003 2024-09-16T09:33:07.521 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:07.521 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:07.521 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:33:07.521 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:33:07.501516+0000", 2024-09-16T09:33:07.522 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0010175609999999999 2024-09-16T09:33:07.522 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:33:07.522 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:33:07.522 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:33:07.522 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:07.522 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:07.522 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.25064.0:241170 8.2 8:4eec9c18:::benchmark_data_smithi070_107919_object116575:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e221)", 2024-09-16T09:33:07.522 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:33:07.493002+0000", 2024-09-16T09:33:07.522 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 0.020653273, 2024-09-16T09:33:07.522 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.020757947999999998, 2024-09-16T09:33:07.522 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:33:07.522 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:33:07.523 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "waiting for sub ops", 2024-09-16T09:33:07.523 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:33:07.523 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.25064", 2024-09-16T09:33:07.523 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/2578256439", 2024-09-16T09:33:07.523 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 241170 2024-09-16T09:33:07.523 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:07.523 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:33:07.523 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:07.523 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:33:07.523 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:33:07.493002+0000", 2024-09-16T09:33:07.523 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:33:07.523 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:07.523 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:07.524 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:33:07.524 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:33:07.493002+0000", 2024-09-16T09:33:07.524 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:33:07.524 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:07.524 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:07.524 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:33:07.524 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:33:07.493003+0000", 2024-09-16T09:33:07.524 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.4900000000000004e-07 2024-09-16T09:33:07.524 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:07.524 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:07.524 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:33:07.524 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:33:07.493007+0000", 2024-09-16T09:33:07.524 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.6e-06 2024-09-16T09:33:07.524 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:07.525 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:07.525 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:33:07.525 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:33:07.493018+0000", 2024-09-16T09:33:07.525 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0872e-05 2024-09-16T09:33:07.525 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:07.525 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:07.525 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:33:07.525 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:33:07.493025+0000", 2024-09-16T09:33:07.525 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.8109999999999997e-06 2024-09-16T09:33:07.525 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:07.525 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:07.525 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:33:07.525 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:33:07.497691+0000", 2024-09-16T09:33:07.525 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0046664719999999996 2024-09-16T09:33:07.526 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:07.526 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:07.526 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:33:07.526 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:33:07.497836+0000", 2024-09-16T09:33:07.526 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00014405099999999999 2024-09-16T09:33:07.526 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:07.526 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:07.526 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,5", 2024-09-16T09:33:07.526 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:33:07.497951+0000", 2024-09-16T09:33:07.526 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00011591600000000001 2024-09-16T09:33:07.526 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:07.526 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:07.526 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:33:07.526 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:33:07.500503+0000", 2024-09-16T09:33:07.527 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0025513210000000001 2024-09-16T09:33:07.527 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:07.527 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:07.527 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:33:07.527 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:33:07.507019+0000", 2024-09-16T09:33:07.527 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.006516422 2024-09-16T09:33:07.527 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:33:07.527 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:33:07.527 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:33:07.527 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:07.527 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:07.527 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.25064.0:241179 8.2 8:4a57e4ac:::benchmark_data_smithi070_107919_object116584:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e221)", 2024-09-16T09:33:07.527 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:33:07.501922+0000", 2024-09-16T09:33:07.527 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 0.011732634, 2024-09-16T09:33:07.528 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.011904292, 2024-09-16T09:33:07.528 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:33:07.528 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:33:07.528 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "waiting for sub ops", 2024-09-16T09:33:07.528 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:33:07.528 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.25064", 2024-09-16T09:33:07.528 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/2578256439", 2024-09-16T09:33:07.528 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 241179 2024-09-16T09:33:07.528 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:07.528 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:33:07.528 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:07.528 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:33:07.528 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:33:07.501922+0000", 2024-09-16T09:33:07.529 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:33:07.529 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:07.529 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:07.529 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:33:07.529 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:33:07.501922+0000", 2024-09-16T09:33:07.529 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:33:07.529 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:07.529 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:07.529 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:33:07.529 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:33:07.501924+0000", 2024-09-16T09:33:07.529 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.288e-06 2024-09-16T09:33:07.529 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:07.529 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:07.529 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:33:07.530 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:33:07.501929+0000", 2024-09-16T09:33:07.530 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.0429999999999998e-06 2024-09-16T09:33:07.530 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:07.530 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:07.530 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:33:07.530 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:33:07.501939+0000", 2024-09-16T09:33:07.530 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0291e-05 2024-09-16T09:33:07.530 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:07.530 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:07.530 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:33:07.530 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:33:07.501947+0000", 2024-09-16T09:33:07.530 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.4989999999999997e-06 2024-09-16T09:33:07.530 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:07.530 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:07.531 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:33:07.531 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:33:07.501997+0000", 2024-09-16T09:33:07.531 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.0204999999999998e-05 2024-09-16T09:33:07.531 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:07.531 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:07.531 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:33:07.531 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:33:07.502117+0000", 2024-09-16T09:33:07.531 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000120728 2024-09-16T09:33:07.531 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:07.531 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:07.531 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,5", 2024-09-16T09:33:07.531 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:33:07.502228+0000", 2024-09-16T09:33:07.532 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000110158 2024-09-16T09:33:07.532 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:07.532 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:07.532 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:33:07.532 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:33:07.505481+0000", 2024-09-16T09:33:07.532 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0032537600000000001 2024-09-16T09:33:07.532 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:07.532 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:07.532 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:33:07.532 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:33:07.507061+0000", 2024-09-16T09:33:07.532 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0015792390000000001 2024-09-16T09:33:07.533 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:33:07.533 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:33:07.533 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:33:07.533 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:33:07.533 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-09-16T09:33:07.533 INFO:teuthology.orchestra.run.smithi084.stdout: "num_ops": 3 2024-09-16T09:33:07.533 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-09-16T09:33:07.534 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:07.534 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:07.534 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:33:07.635 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:33:07.668 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:07.669 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:07.669 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:33:07.770 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:33:07.805 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:07.806 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:07.806 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:33:07.907 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:33:07.914 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T09:33:07.942 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:07.943 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:07.943 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:33:08.044 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:33:08.074 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:08.075 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:08.076 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:33:08.176 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:33:08.209 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:08.209 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:08.210 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:33:08.275 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.4 dump_blocked_ops 2024-09-16T09:33:08.290 INFO:teuthology.orchestra.run.smithi070.stderr:noscrub is unset 2024-09-16T09:33:08.311 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:33:08.351 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:08.351 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:08.352 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:33:08.452 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:33:08.487 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:08.487 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:08.488 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:33:08.589 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:33:08.625 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:08.626 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:08.626 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:33:08.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:08 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3731963866' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:33:08.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:08 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:33:08.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:08 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3731963866' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:33:08.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:08 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:33:08.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:08 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3731963866' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:33:08.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:08 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:33:08.691 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:33:08.727 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:33:08.760 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:08.761 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:08.761 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:33:08.862 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:33:08.897 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:08.898 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:08.898 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:33:08.999 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:33:09.037 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:09.038 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:09.038 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:33:09.125 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:33:09.139 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:33:09.189 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:09.189 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:09.190 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:33:09.290 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:33:09.326 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:09.326 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:09.326 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:33:09.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:09 smithi084 ceph-mon[32629]: pgmap v379: 168 pgs: 168 active+clean; 818 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 870 op/s 2024-09-16T09:33:09.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:09 smithi084 ceph-mon[32629]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:33:09.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:09 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-16T09:33:09.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:09 smithi084 ceph-mon[32629]: osdmap e222: 8 total, 8 up, 8 in 2024-09-16T09:33:09.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:09 smithi084 ceph-mon[32629]: 3.1e scrub starts 2024-09-16T09:33:09.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:09 smithi084 ceph-mon[32629]: 3.1e scrub ok 2024-09-16T09:33:09.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:09 smithi084 ceph-mon[32629]: osdmap e223: 8 total, 8 up, 8 in 2024-09-16T09:33:09.413 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:09 smithi070 ceph-mon[30388]: pgmap v379: 168 pgs: 168 active+clean; 818 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 870 op/s 2024-09-16T09:33:09.413 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:09 smithi070 ceph-mon[30388]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:33:09.413 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:09 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-16T09:33:09.413 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:09 smithi070 ceph-mon[30388]: osdmap e222: 8 total, 8 up, 8 in 2024-09-16T09:33:09.413 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:09 smithi070 ceph-mon[30388]: 3.1e scrub starts 2024-09-16T09:33:09.413 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:09 smithi070 ceph-mon[30388]: 3.1e scrub ok 2024-09-16T09:33:09.413 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:09 smithi070 ceph-mon[30388]: osdmap e223: 8 total, 8 up, 8 in 2024-09-16T09:33:09.414 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:09 smithi070 ceph-mon[40691]: pgmap v379: 168 pgs: 168 active+clean; 818 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 870 op/s 2024-09-16T09:33:09.414 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:09 smithi070 ceph-mon[40691]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:33:09.414 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:09 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-16T09:33:09.414 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:09 smithi070 ceph-mon[40691]: osdmap e222: 8 total, 8 up, 8 in 2024-09-16T09:33:09.414 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:09 smithi070 ceph-mon[40691]: 3.1e scrub starts 2024-09-16T09:33:09.414 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:09 smithi070 ceph-mon[40691]: 3.1e scrub ok 2024-09-16T09:33:09.414 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:09 smithi070 ceph-mon[40691]: osdmap e223: 8 total, 8 up, 8 in 2024-09-16T09:33:09.428 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:33:09.470 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:09.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:09.471 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:33:09.572 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:33:09.604 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-09-16T09:33:09.604 INFO:teuthology.orchestra.run.smithi084.stdout: "ops": [], 2024-09-16T09:33:09.604 INFO:teuthology.orchestra.run.smithi084.stdout: "complaint_time": 30, 2024-09-16T09:33:09.604 INFO:teuthology.orchestra.run.smithi084.stdout: "num_blocked_ops": 0 2024-09-16T09:33:09.604 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-09-16T09:33:09.606 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:09.607 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:09.607 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:33:09.708 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:33:09.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:09.750 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:09.750 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:33:09.752 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:33:09.753 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:33:09.786 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:33:09.786 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:33:09.814 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:33:09.815 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:33:09.846 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:33:09.846 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:33:09.851 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:33:09.881 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:33:09.882 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:33:09.883 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:09.883 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:09.884 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:33:09.917 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:33:09.917 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:33:09.948 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:33:09.948 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:33:09.980 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:33:09.980 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:33:09.984 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:33:10.012 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:10.013 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:10.013 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:33:10.114 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:33:10.147 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:10.148 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:10.148 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:33:10.249 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:33:10.285 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:10.286 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:10.286 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:33:10.387 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:33:10.400 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.4 dump_historic_ops 2024-09-16T09:33:10.423 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:10.424 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:10.424 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:33:10.525 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:33:10.556 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:10.557 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:10.557 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:33:10.658 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:33:10.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:10 smithi084 ceph-mon[32629]: 5.12 scrub starts 2024-09-16T09:33:10.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:10 smithi084 ceph-mon[32629]: 5.12 scrub ok 2024-09-16T09:33:10.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:10 smithi084 ceph-mon[32629]: 1.c scrub starts 2024-09-16T09:33:10.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:10 smithi084 ceph-mon[32629]: 1.c scrub ok 2024-09-16T09:33:10.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:10 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3308021947' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:33:10.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:10 smithi070 ceph-mon[40691]: 5.12 scrub starts 2024-09-16T09:33:10.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:10 smithi070 ceph-mon[40691]: 5.12 scrub ok 2024-09-16T09:33:10.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:10 smithi070 ceph-mon[40691]: 1.c scrub starts 2024-09-16T09:33:10.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:10 smithi070 ceph-mon[40691]: 1.c scrub ok 2024-09-16T09:33:10.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:10 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3308021947' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:33:10.673 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:10 smithi070 ceph-mon[30388]: 5.12 scrub starts 2024-09-16T09:33:10.673 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:10 smithi070 ceph-mon[30388]: 5.12 scrub ok 2024-09-16T09:33:10.673 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:10 smithi070 ceph-mon[30388]: 1.c scrub starts 2024-09-16T09:33:10.673 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:10 smithi070 ceph-mon[30388]: 1.c scrub ok 2024-09-16T09:33:10.673 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:10 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3308021947' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:33:10.691 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:10.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:10.693 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:33:10.793 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:33:10.821 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:33:10.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:10.835 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:10.835 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:33:10.900 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T09:33:10.900 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T09:33:10.900 INFO:tasks.thrashosds.thrasher:inject_pause on osd.3 2024-09-16T09:33:10.900 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-16T09:33:10.900 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-16T09:33:10.901 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.3 config set filestore_inject_stall 3 2024-09-16T09:33:10.936 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:33:10.993 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:10.993 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:10.994 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:33:11.088 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-16T09:33:11.095 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:33:11.194 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:11.194 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:11.194 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:33:11.295 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:33:11.345 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:11.345 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:11.346 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:33:11.350 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:33:11.423 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:11 smithi070 ceph-mon[30388]: 5.0 scrub starts 2024-09-16T09:33:11.423 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:11 smithi070 ceph-mon[30388]: 5.0 scrub ok 2024-09-16T09:33:11.423 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:11 smithi070 ceph-mon[30388]: pgmap v382: 168 pgs: 168 active+clean; 460 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 1.09k op/s 2024-09-16T09:33:11.424 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:11 smithi070 ceph-mon[40691]: 5.0 scrub starts 2024-09-16T09:33:11.424 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:11 smithi070 ceph-mon[40691]: 5.0 scrub ok 2024-09-16T09:33:11.424 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:11 smithi070 ceph-mon[40691]: pgmap v382: 168 pgs: 168 active+clean; 460 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 1.09k op/s 2024-09-16T09:33:11.447 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:33:11.486 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:11.487 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:11.487 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:33:11.585 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:11 smithi084 ceph-mon[32629]: 5.0 scrub starts 2024-09-16T09:33:11.585 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:11 smithi084 ceph-mon[32629]: 5.0 scrub ok 2024-09-16T09:33:11.585 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:11 smithi084 ceph-mon[32629]: pgmap v382: 168 pgs: 168 active+clean; 460 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 1.09k op/s 2024-09-16T09:33:11.588 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:33:11.624 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:11.625 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:11.625 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:33:11.635 INFO:teuthology.orchestra.run.smithi084.stdout:ERROR: (22) Invalid argument 2024-09-16T09:33:11.635 INFO:teuthology.orchestra.run.smithi084.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:33:11.726 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:33:11.753 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:11.754 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:11.754 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:33:11.855 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:33:11.886 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:11.887 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:11.888 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:33:11.988 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:33:12.028 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:12.029 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:12.029 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:33:12.130 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:33:12.162 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:12.162 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:12.163 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:33:12.252 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T09:33:12.252 INFO:teuthology.orchestra.run.smithi070.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-16T09:33:12.252 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T09:33:12.253 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:33:11 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:33:11] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:33:12.263 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:33:12.303 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:12.303 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:12.304 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:33:12.404 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:33:12.414 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.5 dump_ops_in_flight 2024-09-16T09:33:12.441 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:12.442 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:12.442 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:33:12.543 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:33:12.584 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:12.585 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:12.585 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:33:12.649 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:12 smithi070 ceph-mon[30388]: 3.b scrub starts 2024-09-16T09:33:12.649 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:12 smithi070 ceph-mon[30388]: 3.b scrub ok 2024-09-16T09:33:12.649 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:12 smithi070 ceph-mon[40691]: 3.b scrub starts 2024-09-16T09:33:12.649 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:12 smithi070 ceph-mon[40691]: 3.b scrub ok 2024-09-16T09:33:12.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:12 smithi084 ceph-mon[32629]: 3.b scrub starts 2024-09-16T09:33:12.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:12 smithi084 ceph-mon[32629]: 3.b scrub ok 2024-09-16T09:33:12.687 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:33:12.721 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:12.721 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:12.722 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:33:12.822 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:33:12.829 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:33:12.865 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:12.865 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:12.865 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:33:12.966 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:33:13.001 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:13.002 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:13.002 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:33:13.103 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:33:13.142 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:13.143 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:13.143 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:33:13.164 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T09:33:13.244 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:33:13.246 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:33:13.300 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:33:12 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:33:13.332 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:13.332 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:13.333 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:33:13.389 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:13 smithi070 ceph-mon[30388]: 3.4 scrub starts 2024-09-16T09:33:13.389 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:13 smithi070 ceph-mon[30388]: 3.4 scrub ok 2024-09-16T09:33:13.389 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:13 smithi070 ceph-mon[30388]: pgmap v383: 168 pgs: 168 active+clean; 460 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 689 B/s rd, 0 B/s wr, 884 op/s 2024-09-16T09:33:13.389 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:13 smithi070 ceph-mon[30388]: 3.2 scrub starts 2024-09-16T09:33:13.389 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:13 smithi070 ceph-mon[30388]: 3.2 scrub ok 2024-09-16T09:33:13.390 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:13 smithi070 ceph-mon[40691]: 3.4 scrub starts 2024-09-16T09:33:13.390 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:13 smithi070 ceph-mon[40691]: 3.4 scrub ok 2024-09-16T09:33:13.390 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:13 smithi070 ceph-mon[40691]: pgmap v383: 168 pgs: 168 active+clean; 460 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 689 B/s rd, 0 B/s wr, 884 op/s 2024-09-16T09:33:13.390 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:13 smithi070 ceph-mon[40691]: 3.2 scrub starts 2024-09-16T09:33:13.390 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:13 smithi070 ceph-mon[40691]: 3.2 scrub ok 2024-09-16T09:33:13.434 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:33:13.470 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:13.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:13.471 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:33:13.526 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Removed 124592 objects 2024-09-16T09:33:13.526 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Clean up completed and total clean up time :107.556 2024-09-16T09:33:13.573 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:33:13.593 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:13.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:13.594 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:33:13.621 INFO:teuthology.orchestra.run.smithi084.stdout:ERROR: (22) Invalid argument 2024-09-16T09:33:13.621 INFO:teuthology.orchestra.run.smithi084.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:33:13.621 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:13 smithi084 ceph-mon[32629]: 3.4 scrub starts 2024-09-16T09:33:13.621 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:13 smithi084 ceph-mon[32629]: 3.4 scrub ok 2024-09-16T09:33:13.621 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:13 smithi084 ceph-mon[32629]: pgmap v383: 168 pgs: 168 active+clean; 460 MiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 689 B/s rd, 0 B/s wr, 884 op/s 2024-09-16T09:33:13.621 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:13 smithi084 ceph-mon[32629]: 3.2 scrub starts 2024-09-16T09:33:13.621 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:13 smithi084 ceph-mon[32629]: 3.2 scrub ok 2024-09-16T09:33:13.695 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:33:13.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:13.717 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:13.718 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:33:13.818 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:33:13.840 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:13.841 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:13.841 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:33:13.852 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:33:13.942 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:33:13.963 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:13.964 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:13.964 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:33:14.065 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:33:14.087 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:14.087 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:14.088 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:33:14.188 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:33:14.209 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:14.210 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:14.211 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:33:14.260 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.5 dump_blocked_ops 2024-09-16T09:33:14.304 INFO:teuthology.orchestra.run.smithi070.stderr:nodeep-scrub is unset 2024-09-16T09:33:14.311 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:33:14.334 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:14.335 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:14.336 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:33:14.336 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:33:14.336 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:33:14.353 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:33:14.353 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:33:14.369 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:33:14.369 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:33:14.385 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:33:14.385 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:33:14.401 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:33:14.402 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:33:14.418 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:33:14.418 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:33:14.435 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:33:14.435 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:33:14.437 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:33:14.451 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:33:14.451 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:33:14.463 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:14.464 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:14.464 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:33:14.508 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:33:14.539 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:14 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:33:14.539 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:14 smithi070 ceph-mon[30388]: 3.11 scrub starts 2024-09-16T09:33:14.540 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:14 smithi070 ceph-mon[30388]: 3.11 scrub ok 2024-09-16T09:33:14.540 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:14 smithi070 ceph-mon[30388]: 5.19 scrub starts 2024-09-16T09:33:14.540 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:14 smithi070 ceph-mon[30388]: 5.19 scrub ok 2024-09-16T09:33:14.540 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:14 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/117892835' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:33:14.540 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:14 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:33:14.540 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:14 smithi070 ceph-mon[40691]: 3.11 scrub starts 2024-09-16T09:33:14.540 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:14 smithi070 ceph-mon[40691]: 3.11 scrub ok 2024-09-16T09:33:14.540 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:14 smithi070 ceph-mon[40691]: 5.19 scrub starts 2024-09-16T09:33:14.540 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:14 smithi070 ceph-mon[40691]: 5.19 scrub ok 2024-09-16T09:33:14.540 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:14 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/117892835' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:33:14.565 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:33:14.586 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:14.587 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:14.587 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:33:14.629 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:14 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:33:14.629 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:14 smithi084 ceph-mon[32629]: 3.11 scrub starts 2024-09-16T09:33:14.629 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:14 smithi084 ceph-mon[32629]: 3.11 scrub ok 2024-09-16T09:33:14.629 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:14 smithi084 ceph-mon[32629]: 5.19 scrub starts 2024-09-16T09:33:14.629 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:14 smithi084 ceph-mon[32629]: 5.19 scrub ok 2024-09-16T09:33:14.629 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:14 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/117892835' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:33:14.688 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:33:14.709 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:14.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:14.710 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:33:14.811 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:33:14.833 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:14.833 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:14.834 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:33:14.935 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:33:14.956 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:14.957 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:14.957 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:33:15.054 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-09-16T09:33:15.054 INFO:teuthology.orchestra.run.smithi084.stdout: "ops": [], 2024-09-16T09:33:15.054 INFO:teuthology.orchestra.run.smithi084.stdout: "complaint_time": 30, 2024-09-16T09:33:15.054 INFO:teuthology.orchestra.run.smithi084.stdout: "num_blocked_ops": 0 2024-09-16T09:33:15.054 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-09-16T09:33:15.058 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:33:15.080 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:15.081 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:15.081 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:33:15.182 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:33:15.204 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:15.204 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:15.205 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:33:15.305 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:33:15.327 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:15.328 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:15.328 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:33:15.429 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:33:15.450 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:15.451 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:15.452 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:33:15.552 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:33:15.553 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:15 smithi070 ceph-mon[40691]: pgmap v384: 168 pgs: 168 active+clean; 42 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.66k op/s 2024-09-16T09:33:15.553 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:15 smithi070 ceph-mon[40691]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-16T09:33:15.554 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:15 smithi070 ceph-mon[40691]: Cluster is now healthy 2024-09-16T09:33:15.554 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:15 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/117892835' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-16T09:33:15.554 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:15 smithi070 ceph-mon[40691]: osdmap e224: 8 total, 8 up, 8 in 2024-09-16T09:33:15.554 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:15 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3516054245' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:33:15.554 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:15 smithi070 ceph-mon[40691]: 3.10 scrub starts 2024-09-16T09:33:15.554 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:15 smithi070 ceph-mon[40691]: 3.10 scrub ok 2024-09-16T09:33:15.554 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:15 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:33:15.554 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:15 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:33:15.554 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:15 smithi070 ceph-mon[30388]: pgmap v384: 168 pgs: 168 active+clean; 42 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.66k op/s 2024-09-16T09:33:15.555 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:15 smithi070 ceph-mon[30388]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-16T09:33:15.555 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:15 smithi070 ceph-mon[30388]: Cluster is now healthy 2024-09-16T09:33:15.555 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:15 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/117892835' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-16T09:33:15.555 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:15 smithi070 ceph-mon[30388]: osdmap e224: 8 total, 8 up, 8 in 2024-09-16T09:33:15.555 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:15 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3516054245' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:33:15.555 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:15 smithi070 ceph-mon[30388]: 3.10 scrub starts 2024-09-16T09:33:15.555 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:15 smithi070 ceph-mon[30388]: 3.10 scrub ok 2024-09-16T09:33:15.555 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:15 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:33:15.555 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:15 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:33:15.576 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:15.576 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:15.577 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:33:15.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:15 smithi084 ceph-mon[32629]: pgmap v384: 168 pgs: 168 active+clean; 42 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.66k op/s 2024-09-16T09:33:15.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:15 smithi084 ceph-mon[32629]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-16T09:33:15.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:15 smithi084 ceph-mon[32629]: Cluster is now healthy 2024-09-16T09:33:15.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:15 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/117892835' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-16T09:33:15.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:15 smithi084 ceph-mon[32629]: osdmap e224: 8 total, 8 up, 8 in 2024-09-16T09:33:15.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:15 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3516054245' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:33:15.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:15 smithi084 ceph-mon[32629]: 3.10 scrub starts 2024-09-16T09:33:15.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:15 smithi084 ceph-mon[32629]: 3.10 scrub ok 2024-09-16T09:33:15.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:15 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:33:15.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:15 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:33:15.677 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:33:15.699 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:15.699 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:15.700 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:33:15.752 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.5 dump_historic_ops 2024-09-16T09:33:15.800 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:33:15.822 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:15.822 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:15.823 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:33:15.924 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:33:15.947 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:15.947 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:15.948 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:33:16.002 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:33:16.048 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:33:16.072 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:16.072 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:16.073 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:33:16.173 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:33:16.195 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:16.196 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:16.196 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:33:16.297 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:33:16.321 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:16.322 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:16.322 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:33:16.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:16 smithi084 ceph-mon[32629]: osdmap e225: 8 total, 8 up, 8 in 2024-09-16T09:33:16.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:16 smithi084 ceph-mon[32629]: 8.9 scrub starts 2024-09-16T09:33:16.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:16 smithi084 ceph-mon[32629]: 8.9 scrub ok 2024-09-16T09:33:16.423 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:33:16.445 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:16.446 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:16.446 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:33:16.548 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:33:16.572 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:16.572 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:16.573 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:33:16.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:16 smithi070 ceph-mon[40691]: osdmap e225: 8 total, 8 up, 8 in 2024-09-16T09:33:16.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:16 smithi070 ceph-mon[40691]: 8.9 scrub starts 2024-09-16T09:33:16.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:16 smithi070 ceph-mon[40691]: 8.9 scrub ok 2024-09-16T09:33:16.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:16 smithi070 ceph-mon[30388]: osdmap e225: 8 total, 8 up, 8 in 2024-09-16T09:33:16.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:16 smithi070 ceph-mon[30388]: 8.9 scrub starts 2024-09-16T09:33:16.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:16 smithi070 ceph-mon[30388]: 8.9 scrub ok 2024-09-16T09:33:16.674 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:33:16.696 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:16.696 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:16.697 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:33:16.797 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:33:16.821 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:16.822 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:16.822 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:33:16.923 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:33:16.947 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:16.948 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:16.949 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:33:17.049 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:33:17.073 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:17.073 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:17.074 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:33:17.175 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:33:17.200 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:17.201 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:17.201 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:33:17.241 INFO:tasks.cephadm.ceph_manager.ceph:removing pool_name unique_pool_1 2024-09-16T09:33:17.241 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd pool rm unique_pool_1 unique_pool_1 --yes-i-really-really-mean-it 2024-09-16T09:33:17.288 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T09:33:17.302 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:33:17.329 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-09-16T09:33:17.330 INFO:teuthology.orchestra.run.smithi084.stdout: "size": 20, 2024-09-16T09:33:17.330 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 600, 2024-09-16T09:33:17.330 INFO:teuthology.orchestra.run.smithi084.stdout: "ops": [ 2024-09-16T09:33:17.330 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.330 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "MOSDPGPushReply(8.0 168/167 [PushReplyOp(8:08f3f0b5:::benchmark_data_smithi070_107919_object491:head)])", 2024-09-16T09:33:17.330 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:30:23.824987+0000", 2024-09-16T09:33:17.330 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 172.69102333399999, 2024-09-16T09:33:17.330 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.13437887400000001, 2024-09-16T09:33:17.330 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:33:17.330 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:33:17.330 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "reached pg", 2024-09-16T09:33:17.330 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:33:17.331 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.331 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:33:17.331 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:23.824987+0000", 2024-09-16T09:33:17.331 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:33:17.331 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.331 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.331 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:33:17.331 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:23.824987+0000", 2024-09-16T09:33:17.331 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:33:17.331 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.331 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.331 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:33:17.331 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:23.824988+0000", 2024-09-16T09:33:17.331 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.5900000000000005e-07 2024-09-16T09:33:17.332 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.332 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.332 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:33:17.332 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:23.824994+0000", 2024-09-16T09:33:17.332 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.088e-06 2024-09-16T09:33:17.332 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.332 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.332 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:33:17.332 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:23.825004+0000", 2024-09-16T09:33:17.332 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0492999999999999e-05 2024-09-16T09:33:17.332 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.332 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.332 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:33:17.333 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:23.825012+0000", 2024-09-16T09:33:17.333 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.277e-06 2024-09-16T09:33:17.333 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.333 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.333 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:33:17.333 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:23.953117+0000", 2024-09-16T09:33:17.333 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.12810587600000001 2024-09-16T09:33:17.333 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.333 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.333 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:33:17.333 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:23.959366+0000", 2024-09-16T09:33:17.333 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0062481810000000002 2024-09-16T09:33:17.333 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:33:17.333 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:33:17.334 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:33:17.334 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.334 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.334 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "MOSDPGPushReply(8.8 168/167 [PushReplyOp(8:11a4176e:::benchmark_data_smithi070_107919_object218:head)])", 2024-09-16T09:33:17.334 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:30:23.878467+0000", 2024-09-16T09:33:17.334 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 172.637542728, 2024-09-16T09:33:17.334 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.12165545899999999, 2024-09-16T09:33:17.334 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:33:17.334 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:33:17.335 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "reached pg", 2024-09-16T09:33:17.335 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:33:17.335 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.335 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:33:17.335 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:23.878467+0000", 2024-09-16T09:33:17.335 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:33:17.335 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.335 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.335 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:33:17.335 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:23.878467+0000", 2024-09-16T09:33:17.335 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:33:17.335 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.335 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.335 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:33:17.336 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:23.878468+0000", 2024-09-16T09:33:17.336 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.0100000000000005e-07 2024-09-16T09:33:17.336 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.336 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.336 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:33:17.336 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:23.878473+0000", 2024-09-16T09:33:17.336 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.6460000000000001e-06 2024-09-16T09:33:17.336 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.336 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.336 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:33:17.337 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:23.878480+0000", 2024-09-16T09:33:17.337 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.6480000000000003e-06 2024-09-16T09:33:17.337 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.337 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.337 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:33:17.337 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:23.878486+0000", 2024-09-16T09:33:17.337 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.7810000000000002e-06 2024-09-16T09:33:17.337 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.337 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.337 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:33:17.337 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:23.995434+0000", 2024-09-16T09:33:17.337 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.11694816800000001 2024-09-16T09:33:17.337 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.338 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.338 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:33:17.338 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:24.000123+0000", 2024-09-16T09:33:17.338 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0046886150000000001 2024-09-16T09:33:17.338 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:33:17.338 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:33:17.338 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:33:17.338 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.338 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.338 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "MOSDPGPushReply(8.8 169/167 [PushReplyOp(8:156d5b64:::benchmark_data_smithi070_107919_object361:head)])", 2024-09-16T09:33:17.338 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:30:24.607597+0000", 2024-09-16T09:33:17.338 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 171.908413166, 2024-09-16T09:33:17.338 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.116072945, 2024-09-16T09:33:17.339 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:33:17.339 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:33:17.339 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "reached pg", 2024-09-16T09:33:17.339 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:33:17.339 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.339 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:33:17.339 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:24.607597+0000", 2024-09-16T09:33:17.339 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:33:17.339 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.339 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.339 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:33:17.339 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:24.607597+0000", 2024-09-16T09:33:17.339 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:33:17.340 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.340 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.340 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:33:17.340 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:24.607598+0000", 2024-09-16T09:33:17.340 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.5099999999999999e-07 2024-09-16T09:33:17.340 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.340 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.340 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:33:17.340 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:24.607603+0000", 2024-09-16T09:33:17.341 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.4859999999999997e-06 2024-09-16T09:33:17.341 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.341 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.341 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:33:17.341 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:24.607613+0000", 2024-09-16T09:33:17.341 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0022e-05 2024-09-16T09:33:17.341 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.341 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.341 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:33:17.341 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:24.607621+0000", 2024-09-16T09:33:17.341 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.7230000000000004e-06 2024-09-16T09:33:17.342 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.342 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.342 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:33:17.342 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:24.719485+0000", 2024-09-16T09:33:17.342 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.111864667 2024-09-16T09:33:17.342 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.342 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.342 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:33:17.342 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:24.723670+0000", 2024-09-16T09:33:17.342 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0041843959999999999 2024-09-16T09:33:17.342 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:33:17.342 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:33:17.342 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:33:17.342 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.343 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.343 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "MOSDPGPushReply(8.8 169/167 [PushReplyOp(8:10968269:::benchmark_data_smithi070_107919_object493:head)])", 2024-09-16T09:33:17.343 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:30:25.294720+0000", 2024-09-16T09:33:17.343 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 171.22129036199999, 2024-09-16T09:33:17.343 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.110359677, 2024-09-16T09:33:17.343 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:33:17.343 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:33:17.343 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "reached pg", 2024-09-16T09:33:17.343 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:33:17.343 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.343 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:33:17.343 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:25.294720+0000", 2024-09-16T09:33:17.343 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:33:17.343 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.343 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.344 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:33:17.344 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:25.294720+0000", 2024-09-16T09:33:17.344 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:33:17.344 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.344 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.344 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:33:17.344 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:25.294720+0000", 2024-09-16T09:33:17.344 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.0199999999999998e-07 2024-09-16T09:33:17.345 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.345 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.345 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:33:17.345 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:25.294724+0000", 2024-09-16T09:33:17.345 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.8890000000000004e-06 2024-09-16T09:33:17.345 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.345 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.345 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:33:17.345 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:25.294731+0000", 2024-09-16T09:33:17.345 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.6649999999999997e-06 2024-09-16T09:33:17.345 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.345 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.345 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:33:17.346 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:25.294736+0000", 2024-09-16T09:33:17.346 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.4979999999999997e-06 2024-09-16T09:33:17.346 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.346 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.346 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:33:17.346 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:25.399626+0000", 2024-09-16T09:33:17.346 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.104890238 2024-09-16T09:33:17.346 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.346 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.346 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:33:17.346 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:25.405079+0000", 2024-09-16T09:33:17.346 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0054530849999999999 2024-09-16T09:33:17.346 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:33:17.347 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:33:17.347 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:33:17.347 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.347 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.347 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "MOSDPGPushReply(8.0 171/167 [PushReplyOp(8:032c0fa5:::benchmark_data_smithi070_107919_object1896:head)])", 2024-09-16T09:33:17.347 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:30:27.623384+0000", 2024-09-16T09:33:17.347 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 168.892625958, 2024-09-16T09:33:17.347 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.104204386, 2024-09-16T09:33:17.347 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:33:17.347 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:33:17.347 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "reached pg", 2024-09-16T09:33:17.347 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:33:17.347 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.347 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:33:17.348 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:27.623384+0000", 2024-09-16T09:33:17.348 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:33:17.348 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.348 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.348 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:33:17.348 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:27.623384+0000", 2024-09-16T09:33:17.348 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:33:17.348 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.348 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.348 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:33:17.348 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:27.623384+0000", 2024-09-16T09:33:17.348 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.5999999999999999e-07 2024-09-16T09:33:17.348 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.349 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.349 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:33:17.349 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:27.623388+0000", 2024-09-16T09:33:17.349 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.5980000000000001e-06 2024-09-16T09:33:17.349 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.349 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.349 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:33:17.349 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:27.623395+0000", 2024-09-16T09:33:17.349 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.9909999999999996e-06 2024-09-16T09:33:17.349 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.349 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.350 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:33:17.350 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:27.623401+0000", 2024-09-16T09:33:17.350 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.959e-06 2024-09-16T09:33:17.350 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.350 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.350 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:33:17.350 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:27.714155+0000", 2024-09-16T09:33:17.350 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.090754192999999997 2024-09-16T09:33:17.350 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.350 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.350 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:33:17.350 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:27.727588+0000", 2024-09-16T09:33:17.350 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.013433285 2024-09-16T09:33:17.350 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:33:17.351 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:33:17.351 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:33:17.351 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.351 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.351 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "MOSDPGPushReply(8.0 171/167 [PushReplyOp(8:03e7bdc2:::benchmark_data_smithi070_107919_object2053:head)])", 2024-09-16T09:33:17.351 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:30:28.027268+0000", 2024-09-16T09:33:17.351 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 168.488741723, 2024-09-16T09:33:17.351 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.14247446499999999, 2024-09-16T09:33:17.351 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:33:17.351 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:33:17.351 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "reached pg", 2024-09-16T09:33:17.351 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:33:17.351 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.352 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:33:17.352 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.027268+0000", 2024-09-16T09:33:17.352 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:33:17.352 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.352 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.352 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:33:17.352 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.027268+0000", 2024-09-16T09:33:17.352 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:33:17.352 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.352 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.352 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:33:17.352 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.027269+0000", 2024-09-16T09:33:17.352 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.5800000000000001e-07 2024-09-16T09:33:17.352 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.353 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.353 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:33:17.353 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.027273+0000", 2024-09-16T09:33:17.353 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.3900000000000003e-06 2024-09-16T09:33:17.353 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.353 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.353 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:33:17.353 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.027281+0000", 2024-09-16T09:33:17.353 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.9740000000000002e-06 2024-09-16T09:33:17.354 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.354 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.354 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:33:17.354 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.027288+0000", 2024-09-16T09:33:17.354 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.8689999999999998e-06 2024-09-16T09:33:17.354 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.354 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.355 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:33:17.355 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.165751+0000", 2024-09-16T09:33:17.355 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.13846350599999999 2024-09-16T09:33:17.355 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.355 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.355 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:33:17.355 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.169743+0000", 2024-09-16T09:33:17.355 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0039914679999999998 2024-09-16T09:33:17.356 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:33:17.356 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:33:17.356 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:33:17.356 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.356 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.356 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.25064.0:59358 8.8 8:119768b4:::benchmark_data_smithi070_107919_object59357:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e171)", 2024-09-16T09:33:17.357 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:30:28.069811+0000", 2024-09-16T09:33:17.357 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 168.44619897499999, 2024-09-16T09:33:17.357 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.097315094000000005, 2024-09-16T09:33:17.357 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:33:17.357 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:33:17.357 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:33:17.357 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:33:17.358 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.25064", 2024-09-16T09:33:17.358 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/2578256439", 2024-09-16T09:33:17.358 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 59358 2024-09-16T09:33:17.358 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.358 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:33:17.358 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.358 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:33:17.358 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.069811+0000", 2024-09-16T09:33:17.359 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:33:17.359 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.359 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.359 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:33:17.359 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.069811+0000", 2024-09-16T09:33:17.359 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:33:17.359 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.359 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.359 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:33:17.359 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.069813+0000", 2024-09-16T09:33:17.359 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.641e-06 2024-09-16T09:33:17.360 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.360 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.360 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:33:17.360 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.069845+0000", 2024-09-16T09:33:17.360 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.1977999999999998e-05 2024-09-16T09:33:17.360 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.360 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.360 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:33:17.360 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.069859+0000", 2024-09-16T09:33:17.360 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.4438e-05 2024-09-16T09:33:17.360 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.360 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.360 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:33:17.361 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.069865+0000", 2024-09-16T09:33:17.361 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.7409999999999998e-06 2024-09-16T09:33:17.361 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.361 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.361 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:33:17.361 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.126331+0000", 2024-09-16T09:33:17.361 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.056465818000000001 2024-09-16T09:33:17.361 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.361 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.361 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:33:17.361 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.126477+0000", 2024-09-16T09:33:17.361 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000146179 2024-09-16T09:33:17.361 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.361 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.362 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,7", 2024-09-16T09:33:17.362 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.126622+0000", 2024-09-16T09:33:17.362 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00014503899999999999 2024-09-16T09:33:17.362 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.362 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.362 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:33:17.363 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.130879+0000", 2024-09-16T09:33:17.363 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0042567880000000001 2024-09-16T09:33:17.364 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.364 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.364 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:33:17.364 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.137098+0000", 2024-09-16T09:33:17.364 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.006219423 2024-09-16T09:33:17.364 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.364 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.364 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:33:17.364 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.158728+0000", 2024-09-16T09:33:17.364 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.02162967 2024-09-16T09:33:17.364 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.364 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.364 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:33:17.364 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.158785+0000", 2024-09-16T09:33:17.365 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.7704000000000002e-05 2024-09-16T09:33:17.365 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.365 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.365 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:33:17.365 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.167126+0000", 2024-09-16T09:33:17.365 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0083406750000000005 2024-09-16T09:33:17.365 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:33:17.365 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:33:17.365 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:33:17.366 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.366 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.366 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.25064.0:59370 8.8 8:17355324:::benchmark_data_smithi070_107919_object59369:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e171)", 2024-09-16T09:33:17.366 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:30:28.078337+0000", 2024-09-16T09:33:17.366 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 168.43767288399999, 2024-09-16T09:33:17.366 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.104820753, 2024-09-16T09:33:17.366 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:33:17.366 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:33:17.366 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:33:17.366 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:33:17.366 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.25064", 2024-09-16T09:33:17.366 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/2578256439", 2024-09-16T09:33:17.366 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 59370 2024-09-16T09:33:17.366 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.367 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:33:17.367 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.367 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:33:17.367 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.078337+0000", 2024-09-16T09:33:17.367 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:33:17.367 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.367 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.367 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:33:17.367 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.078337+0000", 2024-09-16T09:33:17.367 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:33:17.367 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.367 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.367 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:33:17.367 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.078338+0000", 2024-09-16T09:33:17.368 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.054e-06 2024-09-16T09:33:17.368 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.368 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.368 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:33:17.368 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.078394+0000", 2024-09-16T09:33:17.368 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.6239999999999997e-05 2024-09-16T09:33:17.368 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.368 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.368 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:33:17.368 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.078406+0000", 2024-09-16T09:33:17.368 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1309e-05 2024-09-16T09:33:17.368 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.368 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.368 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:33:17.369 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.078414+0000", 2024-09-16T09:33:17.369 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.5339999999999999e-06 2024-09-16T09:33:17.369 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.369 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.369 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:33:17.369 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.137262+0000", 2024-09-16T09:33:17.369 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.058847305000000003 2024-09-16T09:33:17.369 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.369 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.369 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:33:17.369 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.137457+0000", 2024-09-16T09:33:17.369 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000195613 2024-09-16T09:33:17.369 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.370 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.370 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,7", 2024-09-16T09:33:17.370 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.137628+0000", 2024-09-16T09:33:17.370 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00017081999999999999 2024-09-16T09:33:17.370 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.370 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.370 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:33:17.370 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.146675+0000", 2024-09-16T09:33:17.370 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0090471179999999998 2024-09-16T09:33:17.370 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.370 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.370 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:33:17.370 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.146722+0000", 2024-09-16T09:33:17.370 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.7058000000000002e-05 2024-09-16T09:33:17.371 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.371 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.371 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:33:17.371 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.179468+0000", 2024-09-16T09:33:17.371 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.032746022 2024-09-16T09:33:17.371 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.371 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.371 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:33:17.371 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.179514+0000", 2024-09-16T09:33:17.371 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.6162999999999998e-05 2024-09-16T09:33:17.371 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.371 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.371 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:33:17.372 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.183158+0000", 2024-09-16T09:33:17.372 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0036435170000000002 2024-09-16T09:33:17.372 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:33:17.372 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:33:17.372 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:33:17.372 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.372 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.372 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.25064.0:59373 8.8 8:14d238bf:::benchmark_data_smithi070_107919_object59372:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e171)", 2024-09-16T09:33:17.372 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:30:28.080847+0000", 2024-09-16T09:33:17.372 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 168.435162577, 2024-09-16T09:33:17.372 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.105629741, 2024-09-16T09:33:17.372 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:33:17.373 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:33:17.373 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:33:17.373 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:33:17.373 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.25064", 2024-09-16T09:33:17.373 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/2578256439", 2024-09-16T09:33:17.373 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 59373 2024-09-16T09:33:17.374 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.374 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:33:17.374 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.374 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:33:17.374 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.080847+0000", 2024-09-16T09:33:17.374 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:33:17.374 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.374 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.374 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:33:17.374 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.080847+0000", 2024-09-16T09:33:17.374 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:33:17.374 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.374 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.375 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:33:17.375 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.080849+0000", 2024-09-16T09:33:17.375 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0979999999999999e-06 2024-09-16T09:33:17.375 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.375 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.375 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:33:17.375 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.080878+0000", 2024-09-16T09:33:17.375 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.9929000000000001e-05 2024-09-16T09:33:17.375 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.375 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.375 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:33:17.375 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.080889+0000", 2024-09-16T09:33:17.376 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0549e-05 2024-09-16T09:33:17.376 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.376 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.376 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:33:17.376 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.080896+0000", 2024-09-16T09:33:17.376 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.7649999999999996e-06 2024-09-16T09:33:17.376 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.376 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.376 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:33:17.376 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.139771+0000", 2024-09-16T09:33:17.376 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.058875234999999998 2024-09-16T09:33:17.376 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.376 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.377 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:33:17.377 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.140006+0000", 2024-09-16T09:33:17.377 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000235233 2024-09-16T09:33:17.377 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.377 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.377 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,7", 2024-09-16T09:33:17.377 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.140160+0000", 2024-09-16T09:33:17.377 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000153494 2024-09-16T09:33:17.377 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.377 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.377 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:33:17.377 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.155408+0000", 2024-09-16T09:33:17.377 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.015248177999999999 2024-09-16T09:33:17.378 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.378 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.378 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:33:17.378 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.155520+0000", 2024-09-16T09:33:17.378 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00011243699999999999 2024-09-16T09:33:17.378 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.378 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.378 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:33:17.378 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.183169+0000", 2024-09-16T09:33:17.378 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.027648546 2024-09-16T09:33:17.378 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.378 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.378 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:33:17.378 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.183216+0000", 2024-09-16T09:33:17.379 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.7154000000000003e-05 2024-09-16T09:33:17.379 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.379 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.379 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:33:17.379 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:28.186477+0000", 2024-09-16T09:33:17.379 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0032611229999999999 2024-09-16T09:33:17.379 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:33:17.379 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:33:17.379 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:33:17.379 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.379 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.379 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.25064.0:69521 8.8 8:118957f9:::benchmark_data_smithi070_107919_object69520:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e174)", 2024-09-16T09:33:17.379 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:30:36.635667+0000", 2024-09-16T09:33:17.380 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 159.880342947, 2024-09-16T09:33:17.380 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.10586423, 2024-09-16T09:33:17.380 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:33:17.380 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:33:17.380 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:33:17.380 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:33:17.380 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.25064", 2024-09-16T09:33:17.380 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/2578256439", 2024-09-16T09:33:17.380 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 69521 2024-09-16T09:33:17.380 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.380 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:33:17.380 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.380 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:33:17.381 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.635667+0000", 2024-09-16T09:33:17.381 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:33:17.381 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.381 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.381 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:33:17.381 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.635667+0000", 2024-09-16T09:33:17.381 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:33:17.381 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.381 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.381 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:33:17.381 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.635669+0000", 2024-09-16T09:33:17.381 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.823e-06 2024-09-16T09:33:17.381 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.382 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.382 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:33:17.382 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.635697+0000", 2024-09-16T09:33:17.382 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.7994999999999999e-05 2024-09-16T09:33:17.382 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.382 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.382 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:33:17.382 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.635938+0000", 2024-09-16T09:33:17.382 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00024088899999999999 2024-09-16T09:33:17.383 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.383 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.383 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:33:17.383 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.635946+0000", 2024-09-16T09:33:17.383 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.7619999999999995e-06 2024-09-16T09:33:17.383 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.383 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.383 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:33:17.384 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.636579+0000", 2024-09-16T09:33:17.384 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00063343600000000003 2024-09-16T09:33:17.384 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.384 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.384 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:33:17.384 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.636752+0000", 2024-09-16T09:33:17.384 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00017257700000000001 2024-09-16T09:33:17.384 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.384 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.385 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,7", 2024-09-16T09:33:17.385 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.636946+0000", 2024-09-16T09:33:17.385 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00019409599999999999 2024-09-16T09:33:17.385 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.385 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.385 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:33:17.385 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.645412+0000", 2024-09-16T09:33:17.385 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0084667370000000002 2024-09-16T09:33:17.386 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.386 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.386 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:33:17.386 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.645426+0000", 2024-09-16T09:33:17.386 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.3464e-05 2024-09-16T09:33:17.386 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.386 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.386 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:33:17.387 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.737605+0000", 2024-09-16T09:33:17.387 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.092179443999999999 2024-09-16T09:33:17.387 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.387 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.387 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:33:17.387 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.737701+0000", 2024-09-16T09:33:17.387 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.5650999999999996e-05 2024-09-16T09:33:17.387 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.387 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.387 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:33:17.387 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.741531+0000", 2024-09-16T09:33:17.387 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.003830356 2024-09-16T09:33:17.387 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:33:17.387 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:33:17.388 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:33:17.388 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.388 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.388 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.25064.0:69527 8.8 8:1bdf553a:::benchmark_data_smithi070_107919_object69526:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e174)", 2024-09-16T09:33:17.388 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:30:36.637575+0000", 2024-09-16T09:33:17.388 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 159.878434838, 2024-09-16T09:33:17.388 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.106121862, 2024-09-16T09:33:17.388 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:33:17.388 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:33:17.388 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:33:17.388 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:33:17.388 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.25064", 2024-09-16T09:33:17.388 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/2578256439", 2024-09-16T09:33:17.388 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 69527 2024-09-16T09:33:17.389 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.389 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:33:17.389 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.389 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:33:17.389 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.637575+0000", 2024-09-16T09:33:17.389 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:33:17.389 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.389 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.389 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:33:17.389 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.637575+0000", 2024-09-16T09:33:17.389 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:33:17.389 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.389 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.389 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:33:17.390 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.637576+0000", 2024-09-16T09:33:17.390 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.7700000000000004e-07 2024-09-16T09:33:17.390 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.390 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.390 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:33:17.390 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.637608+0000", 2024-09-16T09:33:17.390 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.2075e-05 2024-09-16T09:33:17.390 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.390 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.390 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:33:17.390 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.637614+0000", 2024-09-16T09:33:17.390 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.3690000000000003e-06 2024-09-16T09:33:17.390 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.390 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.390 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:33:17.391 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.637620+0000", 2024-09-16T09:33:17.391 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.6010000000000002e-06 2024-09-16T09:33:17.391 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.391 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.391 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:33:17.391 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.641975+0000", 2024-09-16T09:33:17.391 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0043553430000000002 2024-09-16T09:33:17.391 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.391 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.391 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:33:17.391 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.642107+0000", 2024-09-16T09:33:17.391 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00013171100000000001 2024-09-16T09:33:17.391 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.391 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.392 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,7", 2024-09-16T09:33:17.392 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.642260+0000", 2024-09-16T09:33:17.392 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00015355499999999999 2024-09-16T09:33:17.392 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.392 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.392 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:33:17.392 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.649121+0000", 2024-09-16T09:33:17.392 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0068601160000000003 2024-09-16T09:33:17.392 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.392 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.392 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:33:17.392 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.649142+0000", 2024-09-16T09:33:17.392 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.1352000000000002e-05 2024-09-16T09:33:17.392 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.393 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.393 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:33:17.393 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.741591+0000", 2024-09-16T09:33:17.393 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.092448942000000006 2024-09-16T09:33:17.393 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.393 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.393 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:33:17.393 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.741649+0000", 2024-09-16T09:33:17.393 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.7769000000000002e-05 2024-09-16T09:33:17.393 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.393 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.393 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:33:17.393 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:36.743697+0000", 2024-09-16T09:33:17.394 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0020484520000000001 2024-09-16T09:33:17.394 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:33:17.394 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:33:17.394 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:33:17.394 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.394 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.394 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "MOSDPGPushReply(8.8 174/167 [PushReplyOp(8:1c8c4107:::benchmark_data_smithi070_107919_object4388:head)])", 2024-09-16T09:33:17.394 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:30:37.857431+0000", 2024-09-16T09:33:17.394 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 158.65857919999999, 2024-09-16T09:33:17.394 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.115708227, 2024-09-16T09:33:17.394 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:33:17.394 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:33:17.394 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "reached pg", 2024-09-16T09:33:17.394 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:33:17.395 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.395 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:33:17.395 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:37.857431+0000", 2024-09-16T09:33:17.395 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:33:17.395 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.395 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.395 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:33:17.395 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:37.857431+0000", 2024-09-16T09:33:17.395 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:33:17.395 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.395 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.395 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:33:17.395 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:37.857431+0000", 2024-09-16T09:33:17.396 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.3000000000000002e-07 2024-09-16T09:33:17.396 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.396 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.396 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:33:17.396 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:37.857436+0000", 2024-09-16T09:33:17.396 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.4680000000000003e-06 2024-09-16T09:33:17.396 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.396 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.396 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:33:17.396 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:37.857444+0000", 2024-09-16T09:33:17.396 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.9799999999999998e-06 2024-09-16T09:33:17.396 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.397 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.397 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:33:17.397 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:37.857450+0000", 2024-09-16T09:33:17.397 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.0129999999999999e-06 2024-09-16T09:33:17.397 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.397 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.397 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:33:17.397 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:37.967635+0000", 2024-09-16T09:33:17.397 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.110185139 2024-09-16T09:33:17.397 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.397 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.397 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:33:17.397 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:37.973139+0000", 2024-09-16T09:33:17.398 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.005504297 2024-09-16T09:33:17.398 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:33:17.398 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:33:17.398 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:33:17.398 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.398 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.398 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "MOSDPGPushReply(8.0 176/167 [PushReplyOp(8:0296ea1e:::benchmark_data_smithi070_107919_object6825:head)])", 2024-09-16T09:33:17.398 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:30:39.823571+0000", 2024-09-16T09:33:17.398 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 156.692438874, 2024-09-16T09:33:17.398 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.13149123800000001, 2024-09-16T09:33:17.398 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:33:17.398 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:33:17.398 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "reached pg", 2024-09-16T09:33:17.398 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:33:17.399 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.399 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:33:17.399 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:39.823571+0000", 2024-09-16T09:33:17.399 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:33:17.399 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.399 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.399 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:33:17.399 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:39.823571+0000", 2024-09-16T09:33:17.399 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:33:17.399 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.399 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.399 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:33:17.399 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:39.823572+0000", 2024-09-16T09:33:17.399 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.066e-06 2024-09-16T09:33:17.400 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.400 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.400 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:33:17.400 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:39.823578+0000", 2024-09-16T09:33:17.400 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.2650000000000002e-06 2024-09-16T09:33:17.400 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.400 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.400 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:33:17.400 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:39.823592+0000", 2024-09-16T09:33:17.400 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.3444999999999999e-05 2024-09-16T09:33:17.400 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.400 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.400 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:33:17.400 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:39.823600+0000", 2024-09-16T09:33:17.401 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.7829999999999999e-06 2024-09-16T09:33:17.401 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.401 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.401 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:33:17.401 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:39.951069+0000", 2024-09-16T09:33:17.401 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.127468785 2024-09-16T09:33:17.401 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.401 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.401 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:33:17.401 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:39.955062+0000", 2024-09-16T09:33:17.401 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0039938939999999996 2024-09-16T09:33:17.401 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:33:17.401 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:33:17.401 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:33:17.402 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.402 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.402 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "MOSDPGPushReply(8.0 180/167 [PushReplyOp(8:0cd6d5e7:::benchmark_data_smithi070_107919_object12497:head)])", 2024-09-16T09:33:17.402 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:30:53.252304+0000", 2024-09-16T09:33:17.402 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 143.26370579600001, 2024-09-16T09:33:17.402 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.13780235099999999, 2024-09-16T09:33:17.402 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:33:17.402 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:33:17.402 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "reached pg", 2024-09-16T09:33:17.403 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:33:17.403 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.403 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:33:17.403 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:53.252304+0000", 2024-09-16T09:33:17.403 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:33:17.403 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.403 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.403 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:33:17.404 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:53.252304+0000", 2024-09-16T09:33:17.404 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:33:17.404 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.404 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.404 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:33:17.404 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:53.252305+0000", 2024-09-16T09:33:17.404 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.6000000000000004e-07 2024-09-16T09:33:17.404 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.405 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.405 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:33:17.405 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:53.252310+0000", 2024-09-16T09:33:17.405 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.8210000000000001e-06 2024-09-16T09:33:17.405 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.405 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.405 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:33:17.405 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:53.252317+0000", 2024-09-16T09:33:17.406 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.5410000000000003e-06 2024-09-16T09:33:17.406 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.406 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.406 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:33:17.406 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:53.252324+0000", 2024-09-16T09:33:17.406 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.3790000000000003e-06 2024-09-16T09:33:17.406 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.406 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.406 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:33:17.406 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:53.387129+0000", 2024-09-16T09:33:17.407 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.13480577499999999 2024-09-16T09:33:17.407 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.407 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.407 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:33:17.407 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:53.390107+0000", 2024-09-16T09:33:17.407 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0029772750000000001 2024-09-16T09:33:17.407 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:33:17.407 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:33:17.407 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:33:17.407 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.407 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.407 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "MOSDPGPushReply(8.8 180/167 [PushReplyOp(8:13aee2bf:::benchmark_data_smithi070_107919_object10150:head)])", 2024-09-16T09:33:17.407 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:30:53.264356+0000", 2024-09-16T09:33:17.407 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 143.25165358300001, 2024-09-16T09:33:17.407 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.15374286400000001, 2024-09-16T09:33:17.408 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:33:17.408 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:33:17.408 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "reached pg", 2024-09-16T09:33:17.408 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:33:17.408 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.408 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:33:17.408 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:53.264356+0000", 2024-09-16T09:33:17.408 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:33:17.408 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.408 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.408 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:33:17.408 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:53.264356+0000", 2024-09-16T09:33:17.408 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:33:17.409 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.409 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.409 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:33:17.409 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:53.264357+0000", 2024-09-16T09:33:17.409 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.7800000000000001e-07 2024-09-16T09:33:17.409 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.409 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.409 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:33:17.409 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:53.264363+0000", 2024-09-16T09:33:17.409 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.5629999999999999e-06 2024-09-16T09:33:17.409 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.409 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.409 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:33:17.410 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:53.264372+0000", 2024-09-16T09:33:17.410 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.3670000000000003e-06 2024-09-16T09:33:17.410 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.410 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.410 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:33:17.410 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:53.264380+0000", 2024-09-16T09:33:17.410 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.371e-06 2024-09-16T09:33:17.410 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.410 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.410 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:33:17.410 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:53.411592+0000", 2024-09-16T09:33:17.410 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.14721152800000001 2024-09-16T09:33:17.411 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.411 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.411 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:33:17.411 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:53.418099+0000", 2024-09-16T09:33:17.411 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0065073570000000001 2024-09-16T09:33:17.411 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:33:17.411 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:33:17.411 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:33:17.411 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.411 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.411 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "MOSDPGPushReply(8.0 180/167 [PushReplyOp(8:014384d9:::benchmark_data_smithi070_107919_object12876:head)])", 2024-09-16T09:33:17.411 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:30:54.077469+0000", 2024-09-16T09:33:17.411 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 142.43854126599999, 2024-09-16T09:33:17.411 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.10240925200000001, 2024-09-16T09:33:17.412 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:33:17.412 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:33:17.412 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "reached pg", 2024-09-16T09:33:17.412 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:33:17.412 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.412 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:33:17.412 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:54.077469+0000", 2024-09-16T09:33:17.412 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:33:17.412 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.412 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.412 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:33:17.412 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:54.077469+0000", 2024-09-16T09:33:17.413 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:33:17.413 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.413 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.413 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:33:17.413 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:54.077469+0000", 2024-09-16T09:33:17.413 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.0999999999999998e-07 2024-09-16T09:33:17.413 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.413 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.413 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:33:17.413 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:54.077476+0000", 2024-09-16T09:33:17.413 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.0340000000000002e-06 2024-09-16T09:33:17.413 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.414 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.414 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:33:17.414 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:54.077486+0000", 2024-09-16T09:33:17.414 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0925e-05 2024-09-16T09:33:17.414 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.414 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.414 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:33:17.414 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:54.077494+0000", 2024-09-16T09:33:17.414 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.0990000000000001e-06 2024-09-16T09:33:17.414 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.414 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.414 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:33:17.414 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:54.177117+0000", 2024-09-16T09:33:17.415 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.099623664000000001 2024-09-16T09:33:17.415 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.415 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.415 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:33:17.415 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:30:54.179878+0000", 2024-09-16T09:33:17.415 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0027608200000000002 2024-09-16T09:33:17.415 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:33:17.415 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:33:17.415 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:33:17.415 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.415 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.415 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.25064.0:106991 8.8 8:1dd7c7a4:::benchmark_data_smithi070_107919_object106990:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e186)", 2024-09-16T09:33:17.416 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:31:08.373667+0000", 2024-09-16T09:33:17.416 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 128.14234344100001, 2024-09-16T09:33:17.416 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1058546549999999, 2024-09-16T09:33:17.416 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:33:17.416 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:33:17.416 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:33:17.416 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:33:17.416 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.25064", 2024-09-16T09:33:17.416 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/2578256439", 2024-09-16T09:33:17.416 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 106991 2024-09-16T09:33:17.416 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.416 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:33:17.417 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.417 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:33:17.417 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:08.373667+0000", 2024-09-16T09:33:17.417 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:33:17.417 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.417 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.417 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:33:17.417 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:08.373667+0000", 2024-09-16T09:33:17.417 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:33:17.417 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.417 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.417 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:33:17.417 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:08.373667+0000", 2024-09-16T09:33:17.417 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.47e-07 2024-09-16T09:33:17.418 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.418 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.418 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:33:17.418 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:08.373710+0000", 2024-09-16T09:33:17.418 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.2564000000000001e-05 2024-09-16T09:33:17.418 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.418 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.418 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:33:17.418 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:08.373721+0000", 2024-09-16T09:33:17.418 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1045e-05 2024-09-16T09:33:17.418 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.418 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.418 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:33:17.419 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:08.373727+0000", 2024-09-16T09:33:17.419 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.5160000000000002e-06 2024-09-16T09:33:17.419 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.419 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.419 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:33:17.419 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:08.373779+0000", 2024-09-16T09:33:17.419 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.2237000000000001e-05 2024-09-16T09:33:17.419 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.419 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.419 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:33:17.419 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:08.373786+0000", 2024-09-16T09:33:17.419 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.1389999999999998e-06 2024-09-16T09:33:17.419 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.419 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.420 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:33:17.420 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:09.430374+0000", 2024-09-16T09:33:17.420 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0565881580000001 2024-09-16T09:33:17.420 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.420 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.420 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:33:17.420 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:09.430493+0000", 2024-09-16T09:33:17.420 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000118845 2024-09-16T09:33:17.421 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.421 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.421 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 3,7", 2024-09-16T09:33:17.421 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:09.430648+0000", 2024-09-16T09:33:17.421 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000155477 2024-09-16T09:33:17.421 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.421 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.421 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:33:17.421 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:09.436479+0000", 2024-09-16T09:33:17.422 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0058305120000000004 2024-09-16T09:33:17.422 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.422 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.422 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:33:17.422 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:09.436530+0000", 2024-09-16T09:33:17.422 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.1523999999999997e-05 2024-09-16T09:33:17.422 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.422 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.422 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:33:17.423 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:09.476247+0000", 2024-09-16T09:33:17.423 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.039716695000000003 2024-09-16T09:33:17.423 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.423 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.423 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:33:17.423 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:09.476349+0000", 2024-09-16T09:33:17.423 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000102117 2024-09-16T09:33:17.423 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.424 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.424 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:33:17.424 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:09.479521+0000", 2024-09-16T09:33:17.424 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0031719790000000001 2024-09-16T09:33:17.424 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:33:17.424 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:33:17.425 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:33:17.425 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.425 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.425 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "MOSDPGPushReply(8.0 187/186 [PushReplyOp(8:04425f91:::benchmark_data_smithi070_107919_object52626:head)])", 2024-09-16T09:33:17.425 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:31:10.232177+0000", 2024-09-16T09:33:17.425 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 126.283833235, 2024-09-16T09:33:17.425 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.13291494300000001, 2024-09-16T09:33:17.425 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:33:17.425 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:33:17.425 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "reached pg", 2024-09-16T09:33:17.425 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:33:17.425 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.425 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:33:17.425 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:10.232177+0000", 2024-09-16T09:33:17.426 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:33:17.426 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.426 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.426 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:33:17.426 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:10.232177+0000", 2024-09-16T09:33:17.426 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:33:17.426 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.426 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.426 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:33:17.426 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:10.232177+0000", 2024-09-16T09:33:17.426 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.2700000000000001e-07 2024-09-16T09:33:17.426 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.426 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.426 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:33:17.427 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:10.232179+0000", 2024-09-16T09:33:17.427 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.2340000000000001e-06 2024-09-16T09:33:17.427 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.427 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.427 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:33:17.427 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:10.232186+0000", 2024-09-16T09:33:17.427 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.3980000000000004e-06 2024-09-16T09:33:17.427 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.427 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.427 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:33:17.427 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:10.232190+0000", 2024-09-16T09:33:17.427 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.5990000000000004e-06 2024-09-16T09:33:17.427 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.427 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.427 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:33:17.428 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:10.358728+0000", 2024-09-16T09:33:17.428 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.12653795800000001 2024-09-16T09:33:17.428 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.428 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.428 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:33:17.428 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:10.365092+0000", 2024-09-16T09:33:17.428 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0063635269999999999 2024-09-16T09:33:17.428 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:33:17.428 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:33:17.428 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:33:17.428 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.428 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.428 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "MOSDPGPushReply(8.0 200/186 [PushReplyOp(8:09abd725:::benchmark_data_smithi070_107919_object69214:head)])", 2024-09-16T09:33:17.429 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:31:54.372690+0000", 2024-09-16T09:33:17.429 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 82.143320000000003, 2024-09-16T09:33:17.429 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.158632041, 2024-09-16T09:33:17.429 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:33:17.429 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:33:17.429 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "reached pg", 2024-09-16T09:33:17.429 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:33:17.429 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.429 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:33:17.429 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:54.372690+0000", 2024-09-16T09:33:17.429 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:33:17.429 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.429 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.429 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:33:17.430 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:54.372690+0000", 2024-09-16T09:33:17.430 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:33:17.430 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.430 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.430 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:33:17.430 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:54.372691+0000", 2024-09-16T09:33:17.430 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.47e-07 2024-09-16T09:33:17.430 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.430 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.430 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:33:17.430 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:54.372697+0000", 2024-09-16T09:33:17.430 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.7209999999999999e-06 2024-09-16T09:33:17.430 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.430 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.431 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:33:17.431 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:54.372708+0000", 2024-09-16T09:33:17.431 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1161e-05 2024-09-16T09:33:17.431 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.431 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.431 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:33:17.431 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:54.372716+0000", 2024-09-16T09:33:17.431 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.9740000000000002e-06 2024-09-16T09:33:17.431 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.431 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.431 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:33:17.431 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:54.527807+0000", 2024-09-16T09:33:17.431 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.155091599 2024-09-16T09:33:17.431 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.432 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.432 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:33:17.432 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:54.531322+0000", 2024-09-16T09:33:17.432 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0035147389999999998 2024-09-16T09:33:17.432 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:33:17.432 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:33:17.432 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:33:17.432 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.432 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.432 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "MOSDPGPushReply(8.8 200/186 [PushReplyOp(8:1f7da52e:::benchmark_data_smithi070_107919_object69579:head)])", 2024-09-16T09:33:17.432 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:31:54.402786+0000", 2024-09-16T09:33:17.432 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 82.113223765000001, 2024-09-16T09:33:17.433 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.138351786, 2024-09-16T09:33:17.433 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:33:17.433 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:33:17.433 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "reached pg", 2024-09-16T09:33:17.433 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:33:17.433 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.433 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:33:17.433 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:54.402786+0000", 2024-09-16T09:33:17.433 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:33:17.433 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.433 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.433 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:33:17.434 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:54.402786+0000", 2024-09-16T09:33:17.434 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:33:17.434 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.434 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.434 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:33:17.434 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:54.402787+0000", 2024-09-16T09:33:17.434 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.4300000000000003e-07 2024-09-16T09:33:17.434 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.434 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.434 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:33:17.434 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:54.402793+0000", 2024-09-16T09:33:17.434 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.9050000000000002e-06 2024-09-16T09:33:17.434 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.434 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.435 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:33:17.435 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:54.402804+0000", 2024-09-16T09:33:17.435 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1268e-05 2024-09-16T09:33:17.435 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.435 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.435 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:33:17.435 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:54.402811+0000", 2024-09-16T09:33:17.435 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.7360000000000004e-06 2024-09-16T09:33:17.435 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.435 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.435 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:33:17.435 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:54.536526+0000", 2024-09-16T09:33:17.435 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.13371528899999999 2024-09-16T09:33:17.436 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:17.436 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:17.436 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:33:17.436 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:54.541138+0000", 2024-09-16T09:33:17.436 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0046119450000000001 2024-09-16T09:33:17.436 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:33:17.436 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:33:17.436 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:33:17.436 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:33:17.436 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:33:17.436 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-09-16T09:33:17.460 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:17.460 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:17.460 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:33:17.514 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:33:17.561 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:33:17.585 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:17.585 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:17.586 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:33:17.636 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:17 smithi070 ceph-mon[40691]: pgmap v387: 168 pgs: 168 active+clean; 42 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 797 B/s rd, 0 B/s wr, 1.04k op/s 2024-09-16T09:33:17.636 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:17 smithi070 ceph-mon[40691]: 4.5 scrub starts 2024-09-16T09:33:17.637 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:17 smithi070 ceph-mon[40691]: 4.5 scrub ok 2024-09-16T09:33:17.637 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:17 smithi070 ceph-mon[30388]: pgmap v387: 168 pgs: 168 active+clean; 42 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 797 B/s rd, 0 B/s wr, 1.04k op/s 2024-09-16T09:33:17.637 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:17 smithi070 ceph-mon[30388]: 4.5 scrub starts 2024-09-16T09:33:17.637 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:17 smithi070 ceph-mon[30388]: 4.5 scrub ok 2024-09-16T09:33:17.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:17 smithi084 ceph-mon[32629]: pgmap v387: 168 pgs: 168 active+clean; 42 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 797 B/s rd, 0 B/s wr, 1.04k op/s 2024-09-16T09:33:17.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:17 smithi084 ceph-mon[32629]: 4.5 scrub starts 2024-09-16T09:33:17.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:17 smithi084 ceph-mon[32629]: 4.5 scrub ok 2024-09-16T09:33:17.687 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:33:17.711 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:17.712 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:17.712 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:33:17.813 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:33:17.835 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:17.836 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:17.836 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:33:17.937 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:33:17.959 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:17.959 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:17.960 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:33:18.060 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:33:18.086 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:18.087 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:18.088 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:33:18.161 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T09:33:18.161 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T09:33:18.162 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-16T09:33:18.162 INFO:tasks.thrashosds.thrasher:Setting osd 2 primary_affinity to 0.181029 2024-09-16T09:33:18.162 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd primary-affinity 2 0.18102941315944643 2024-09-16T09:33:18.190 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:33:18.203 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd set noscrub 2024-09-16T09:33:18.228 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:18.229 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:18.230 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:33:18.332 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:33:18.355 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:18.355 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:18.356 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:33:18.456 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:33:18.478 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:18.478 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:18.479 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:33:18.565 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:18 smithi070 ceph-mon[30388]: 5.5 scrub starts 2024-09-16T09:33:18.565 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:18 smithi070 ceph-mon[30388]: 5.5 scrub ok 2024-09-16T09:33:18.565 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:18 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:33:18.565 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:18 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:33:18.566 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:18 smithi070 ceph-mon[40691]: 5.5 scrub starts 2024-09-16T09:33:18.566 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:18 smithi070 ceph-mon[40691]: 5.5 scrub ok 2024-09-16T09:33:18.566 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:18 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:33:18.566 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:18 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:33:18.579 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:33:18.604 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:18.605 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:18.605 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:33:18.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:18 smithi084 ceph-mon[32629]: 5.5 scrub starts 2024-09-16T09:33:18.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:18 smithi084 ceph-mon[32629]: 5.5 scrub ok 2024-09-16T09:33:18.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:18 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:33:18.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:18 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:33:18.706 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:33:18.706 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:33:18.734 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:18.735 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:18.735 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:33:18.836 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:33:18.859 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:18.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:18.860 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:33:18.949 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.6 dump_ops_in_flight 2024-09-16T09:33:18.961 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:33:18.983 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:18.984 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:18.984 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:33:19.043 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:33:19.085 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:33:19.106 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:19.107 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:19.107 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:33:19.198 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:33:19.208 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:33:19.231 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:19.231 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:19.231 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:33:19.327 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'unique_pool_1' removed 2024-09-16T09:33:19.332 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:33:19.391 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:19.391 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:19.392 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:33:19.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:19 smithi070 ceph-mon[30388]: pgmap v388: 168 pgs: 168 active+clean; 42 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 837 op/s 2024-09-16T09:33:19.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:19 smithi070 ceph-mon[30388]: 3.16 scrub starts 2024-09-16T09:33:19.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:19 smithi070 ceph-mon[30388]: 3.16 scrub ok 2024-09-16T09:33:19.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:19 smithi070 ceph-mon[30388]: 6.1d scrub starts 2024-09-16T09:33:19.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:19 smithi070 ceph-mon[30388]: 6.1d scrub ok 2024-09-16T09:33:19.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:19 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3244925085' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "unique_pool_1", "pool2": "unique_pool_1", "yes_i_really_really_mean_it": true} : dispatch 2024-09-16T09:33:19.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:19 smithi070 ceph-mon[40691]: pgmap v388: 168 pgs: 168 active+clean; 42 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 837 op/s 2024-09-16T09:33:19.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:19 smithi070 ceph-mon[40691]: 3.16 scrub starts 2024-09-16T09:33:19.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:19 smithi070 ceph-mon[40691]: 3.16 scrub ok 2024-09-16T09:33:19.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:19 smithi070 ceph-mon[40691]: 6.1d scrub starts 2024-09-16T09:33:19.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:19 smithi070 ceph-mon[40691]: 6.1d scrub ok 2024-09-16T09:33:19.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:19 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3244925085' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "unique_pool_1", "pool2": "unique_pool_1", "yes_i_really_really_mean_it": true} : dispatch 2024-09-16T09:33:19.492 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:33:19.515 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:19.516 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:19.516 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:33:19.617 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:33:19.624 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:19 smithi084 ceph-mon[32629]: pgmap v388: 168 pgs: 168 active+clean; 42 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 837 op/s 2024-09-16T09:33:19.624 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:19 smithi084 ceph-mon[32629]: 3.16 scrub starts 2024-09-16T09:33:19.624 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:19 smithi084 ceph-mon[32629]: 3.16 scrub ok 2024-09-16T09:33:19.624 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:19 smithi084 ceph-mon[32629]: 6.1d scrub starts 2024-09-16T09:33:19.624 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:19 smithi084 ceph-mon[32629]: 6.1d scrub ok 2024-09-16T09:33:19.624 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:19 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3244925085' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "unique_pool_1", "pool2": "unique_pool_1", "yes_i_really_really_mean_it": true} : dispatch 2024-09-16T09:33:19.638 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:19.638 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:19.639 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:33:19.731 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-09-16T09:33:19.731 INFO:teuthology.orchestra.run.smithi084.stdout: "ops": [], 2024-09-16T09:33:19.732 INFO:teuthology.orchestra.run.smithi084.stdout: "num_ops": 0 2024-09-16T09:33:19.732 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-09-16T09:33:19.739 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:33:19.759 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:19.760 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:19.760 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:33:19.863 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:33:19.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:19.885 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:19.885 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:33:19.986 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:33:20.009 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:20.010 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:20.010 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:33:20.111 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:33:20.133 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:20.133 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:20.134 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:33:20.235 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:33:20.258 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:20.258 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:20.259 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:33:20.262 INFO:teuthology.task.full_sequential:In full_sequential, running task radosbench... 2024-09-16T09:33:20.263 INFO:tasks.radosbench:Beginning radosbench... 2024-09-16T09:33:20.263 INFO:tasks.cephadm.ceph_manager.ceph:creating pool_name unique_pool_2 2024-09-16T09:33:20.263 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd pool create unique_pool_2 16 2024-09-16T09:33:20.359 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:33:20.370 INFO:teuthology.orchestra.run.smithi070.stderr:set osd.2 primary-affinity to 0.181029 (8118632) 2024-09-16T09:33:20.382 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:20.383 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:20.384 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:33:20.484 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:33:20.507 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:20.507 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:20.508 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:33:20.609 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:33:20.631 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:20.632 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:20.632 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:33:20.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:20 smithi084 ceph-mon[32629]: 3.17 scrub starts 2024-09-16T09:33:20.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:20 smithi084 ceph-mon[32629]: 3.17 scrub ok 2024-09-16T09:33:20.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:20 smithi084 ceph-mon[32629]: 4.13 scrub starts 2024-09-16T09:33:20.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:20 smithi084 ceph-mon[32629]: 4.13 scrub ok 2024-09-16T09:33:20.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:20 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3244925085' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_1", "pool2": "unique_pool_1", "yes_i_really_really_mean_it": true}]': finished 2024-09-16T09:33:20.662 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:20 smithi084 ceph-mon[32629]: 3.1c scrub starts 2024-09-16T09:33:20.662 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:20 smithi084 ceph-mon[32629]: 3.1c scrub ok 2024-09-16T09:33:20.662 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:20 smithi084 ceph-mon[32629]: osdmap e226: 8 total, 8 up, 8 in 2024-09-16T09:33:20.662 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:20 smithi084 ceph-mon[32629]: 4.1f scrub starts 2024-09-16T09:33:20.662 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:20 smithi084 ceph-mon[32629]: 4.1f scrub ok 2024-09-16T09:33:20.662 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:20 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/410578494' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 2, "weight": 0.18102941315944643} : dispatch 2024-09-16T09:33:20.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:20 smithi070 ceph-mon[40691]: 3.17 scrub starts 2024-09-16T09:33:20.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:20 smithi070 ceph-mon[40691]: 3.17 scrub ok 2024-09-16T09:33:20.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:20 smithi070 ceph-mon[40691]: 4.13 scrub starts 2024-09-16T09:33:20.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:20 smithi070 ceph-mon[40691]: 4.13 scrub ok 2024-09-16T09:33:20.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:20 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3244925085' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_1", "pool2": "unique_pool_1", "yes_i_really_really_mean_it": true}]': finished 2024-09-16T09:33:20.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:20 smithi070 ceph-mon[40691]: 3.1c scrub starts 2024-09-16T09:33:20.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:20 smithi070 ceph-mon[40691]: 3.1c scrub ok 2024-09-16T09:33:20.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:20 smithi070 ceph-mon[40691]: osdmap e226: 8 total, 8 up, 8 in 2024-09-16T09:33:20.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:20 smithi070 ceph-mon[40691]: 4.1f scrub starts 2024-09-16T09:33:20.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:20 smithi070 ceph-mon[40691]: 4.1f scrub ok 2024-09-16T09:33:20.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:20 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/410578494' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 2, "weight": 0.18102941315944643} : dispatch 2024-09-16T09:33:20.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:20 smithi070 ceph-mon[30388]: 3.17 scrub starts 2024-09-16T09:33:20.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:20 smithi070 ceph-mon[30388]: 3.17 scrub ok 2024-09-16T09:33:20.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:20 smithi070 ceph-mon[30388]: 4.13 scrub starts 2024-09-16T09:33:20.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:20 smithi070 ceph-mon[30388]: 4.13 scrub ok 2024-09-16T09:33:20.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:20 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3244925085' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_1", "pool2": "unique_pool_1", "yes_i_really_really_mean_it": true}]': finished 2024-09-16T09:33:20.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:20 smithi070 ceph-mon[30388]: 3.1c scrub starts 2024-09-16T09:33:20.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:20 smithi070 ceph-mon[30388]: 3.1c scrub ok 2024-09-16T09:33:20.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:20 smithi070 ceph-mon[30388]: osdmap e226: 8 total, 8 up, 8 in 2024-09-16T09:33:20.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:20 smithi070 ceph-mon[30388]: 4.1f scrub starts 2024-09-16T09:33:20.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:20 smithi070 ceph-mon[30388]: 4.1f scrub ok 2024-09-16T09:33:20.673 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:20 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/410578494' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 2, "weight": 0.18102941315944643} : dispatch 2024-09-16T09:33:20.733 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:33:20.755 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:20.756 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:20.756 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:33:20.841 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.6 dump_blocked_ops 2024-09-16T09:33:20.857 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:33:20.903 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:20.904 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:20.904 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:33:21.006 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:33:21.027 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:21.028 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:21.028 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:33:21.129 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:33:21.133 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:33:21.151 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:21.152 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:21.152 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:33:21.253 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:33:21.274 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:21.275 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:21.275 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:33:21.376 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:33:21.397 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:21.397 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:21.398 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:33:21.498 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:33:21.523 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:21.523 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:21.524 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:33:21.612 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:21 smithi084 ceph-mon[32629]: 6.14 scrub starts 2024-09-16T09:33:21.612 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:21 smithi084 ceph-mon[32629]: 6.14 scrub ok 2024-09-16T09:33:21.612 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:21 smithi084 ceph-mon[32629]: pgmap v390: 152 pgs: 152 active+clean; 583 KiB data, 2.7 GiB used, 713 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-09-16T09:33:21.612 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:21 smithi084 ceph-mon[32629]: 5.14 scrub starts 2024-09-16T09:33:21.612 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:21 smithi084 ceph-mon[32629]: 5.14 scrub ok 2024-09-16T09:33:21.612 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:21 smithi084 ceph-mon[32629]: 4.4 scrub starts 2024-09-16T09:33:21.612 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:21 smithi084 ceph-mon[32629]: 4.4 scrub ok 2024-09-16T09:33:21.613 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:21 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/410578494' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 2, "weight": 0.18102941315944643}]': finished 2024-09-16T09:33:21.613 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:21 smithi084 ceph-mon[32629]: osdmap e227: 8 total, 8 up, 8 in 2024-09-16T09:33:21.613 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:21 smithi084 ceph-mon[32629]: osdmap e228: 8 total, 8 up, 8 in 2024-09-16T09:33:21.615 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-09-16T09:33:21.615 INFO:teuthology.orchestra.run.smithi084.stdout: "ops": [], 2024-09-16T09:33:21.615 INFO:teuthology.orchestra.run.smithi084.stdout: "complaint_time": 30, 2024-09-16T09:33:21.615 INFO:teuthology.orchestra.run.smithi084.stdout: "num_blocked_ops": 0 2024-09-16T09:33:21.615 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-09-16T09:33:21.625 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:33:21.647 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:21.648 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:21.648 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:33:21.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:21 smithi070 ceph-mon[30388]: 6.14 scrub starts 2024-09-16T09:33:21.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:21 smithi070 ceph-mon[30388]: 6.14 scrub ok 2024-09-16T09:33:21.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:21 smithi070 ceph-mon[30388]: pgmap v390: 152 pgs: 152 active+clean; 583 KiB data, 2.7 GiB used, 713 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-09-16T09:33:21.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:21 smithi070 ceph-mon[30388]: 5.14 scrub starts 2024-09-16T09:33:21.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:21 smithi070 ceph-mon[30388]: 5.14 scrub ok 2024-09-16T09:33:21.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:21 smithi070 ceph-mon[30388]: 4.4 scrub starts 2024-09-16T09:33:21.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:21 smithi070 ceph-mon[30388]: 4.4 scrub ok 2024-09-16T09:33:21.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:21 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/410578494' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 2, "weight": 0.18102941315944643}]': finished 2024-09-16T09:33:21.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:21 smithi070 ceph-mon[30388]: osdmap e227: 8 total, 8 up, 8 in 2024-09-16T09:33:21.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:21 smithi070 ceph-mon[30388]: osdmap e228: 8 total, 8 up, 8 in 2024-09-16T09:33:21.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:21 smithi070 ceph-mon[40691]: 6.14 scrub starts 2024-09-16T09:33:21.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:21 smithi070 ceph-mon[40691]: 6.14 scrub ok 2024-09-16T09:33:21.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:21 smithi070 ceph-mon[40691]: pgmap v390: 152 pgs: 152 active+clean; 583 KiB data, 2.7 GiB used, 713 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-09-16T09:33:21.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:21 smithi070 ceph-mon[40691]: 5.14 scrub starts 2024-09-16T09:33:21.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:21 smithi070 ceph-mon[40691]: 5.14 scrub ok 2024-09-16T09:33:21.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:21 smithi070 ceph-mon[40691]: 4.4 scrub starts 2024-09-16T09:33:21.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:21 smithi070 ceph-mon[40691]: 4.4 scrub ok 2024-09-16T09:33:21.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:21 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/410578494' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 2, "weight": 0.18102941315944643}]': finished 2024-09-16T09:33:21.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:21 smithi070 ceph-mon[40691]: osdmap e227: 8 total, 8 up, 8 in 2024-09-16T09:33:21.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:21 smithi070 ceph-mon[40691]: osdmap e228: 8 total, 8 up, 8 in 2024-09-16T09:33:21.749 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:33:21.770 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:21.771 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:21.771 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:33:21.868 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:33:21.872 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:33:21.894 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:21.895 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:21.895 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:33:21.996 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:33:22.018 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:22.019 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:22.019 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:33:22.120 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:33:22.123 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:33:22.124 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:33:22.124 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:33:21 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:33:21] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:33:22.143 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:33:22.143 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:33:22.144 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:22.144 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:22.144 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:33:22.161 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:33:22.161 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:33:22.177 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:33:22.177 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:33:22.193 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:33:22.193 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:33:22.210 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:33:22.210 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:33:22.226 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:33:22.227 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:33:22.239 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.6 dump_historic_ops 2024-09-16T09:33:22.242 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:33:22.243 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:33:22.245 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:33:22.303 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:22.304 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:22.305 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:33:22.386 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:22 smithi070 ceph-mon[30388]: 5.b scrub starts 2024-09-16T09:33:22.386 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:22 smithi070 ceph-mon[30388]: 5.b scrub ok 2024-09-16T09:33:22.386 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:22 smithi070 ceph-mon[30388]: 3.12 scrub starts 2024-09-16T09:33:22.386 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:22 smithi070 ceph-mon[30388]: 3.12 scrub ok 2024-09-16T09:33:22.387 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:22 smithi070 ceph-mon[30388]: 4.0 scrub starts 2024-09-16T09:33:22.387 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:22 smithi070 ceph-mon[30388]: 4.0 scrub ok 2024-09-16T09:33:22.387 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:22 smithi070 ceph-mon[30388]: 5.1b scrub starts 2024-09-16T09:33:22.387 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:22 smithi070 ceph-mon[30388]: 5.1b scrub ok 2024-09-16T09:33:22.387 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:22 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3779165226' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:33:22.387 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:22 smithi070 ceph-mon[40691]: 5.b scrub starts 2024-09-16T09:33:22.387 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:22 smithi070 ceph-mon[40691]: 5.b scrub ok 2024-09-16T09:33:22.387 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:22 smithi070 ceph-mon[40691]: 3.12 scrub starts 2024-09-16T09:33:22.387 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:22 smithi070 ceph-mon[40691]: 3.12 scrub ok 2024-09-16T09:33:22.387 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:22 smithi070 ceph-mon[40691]: 4.0 scrub starts 2024-09-16T09:33:22.388 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:22 smithi070 ceph-mon[40691]: 4.0 scrub ok 2024-09-16T09:33:22.388 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:22 smithi070 ceph-mon[40691]: 5.1b scrub starts 2024-09-16T09:33:22.388 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:22 smithi070 ceph-mon[40691]: 5.1b scrub ok 2024-09-16T09:33:22.388 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:22 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3779165226' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:33:22.408 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:33:22.431 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:22.431 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:22.431 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:33:22.495 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:33:22.532 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:33:22.554 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:22.554 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:22.554 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:33:22.616 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:22 smithi084 ceph-mon[32629]: 5.b scrub starts 2024-09-16T09:33:22.616 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:22 smithi084 ceph-mon[32629]: 5.b scrub ok 2024-09-16T09:33:22.616 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:22 smithi084 ceph-mon[32629]: 3.12 scrub starts 2024-09-16T09:33:22.616 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:22 smithi084 ceph-mon[32629]: 3.12 scrub ok 2024-09-16T09:33:22.617 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:22 smithi084 ceph-mon[32629]: 4.0 scrub starts 2024-09-16T09:33:22.617 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:22 smithi084 ceph-mon[32629]: 4.0 scrub ok 2024-09-16T09:33:22.617 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:22 smithi084 ceph-mon[32629]: 5.1b scrub starts 2024-09-16T09:33:22.617 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:22 smithi084 ceph-mon[32629]: 5.1b scrub ok 2024-09-16T09:33:22.617 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:22 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3779165226' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:33:22.656 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:33:22.677 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:22.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:22.678 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:33:22.779 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:33:22.801 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:22.802 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:22.802 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:33:22.903 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:33:22.925 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:22.925 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:22.925 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:33:23.026 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:33:23.031 INFO:teuthology.orchestra.run.smithi084.stdout:ERROR: (22) Invalid argument 2024-09-16T09:33:23.031 INFO:teuthology.orchestra.run.smithi084.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:33:23.047 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:23.048 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:23.048 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:33:23.149 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:33:23.170 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:23.171 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:23.171 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:33:23.247 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:33:23.272 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:33:23.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:23.295 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:23.295 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:33:23.345 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:33:23 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:33:23.396 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:33:23.418 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:23.418 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:23.419 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:33:23.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:23 smithi070 ceph-mon[30388]: pgmap v393: 152 pgs: 152 active+clean; 583 KiB data, 2.7 GiB used, 713 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-09-16T09:33:23.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:23 smithi070 ceph-mon[30388]: 3.1 scrub starts 2024-09-16T09:33:23.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:23 smithi070 ceph-mon[30388]: 3.1 scrub ok 2024-09-16T09:33:23.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:23 smithi070 ceph-mon[30388]: 3.0 scrub starts 2024-09-16T09:33:23.492 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:23 smithi070 ceph-mon[30388]: 3.0 scrub ok 2024-09-16T09:33:23.493 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:23 smithi070 ceph-mon[30388]: 4.a scrub starts 2024-09-16T09:33:23.493 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:23 smithi070 ceph-mon[30388]: 4.a scrub ok 2024-09-16T09:33:23.493 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:23 smithi070 ceph-mon[40691]: pgmap v393: 152 pgs: 152 active+clean; 583 KiB data, 2.7 GiB used, 713 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-09-16T09:33:23.493 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:23 smithi070 ceph-mon[40691]: 3.1 scrub starts 2024-09-16T09:33:23.493 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:23 smithi070 ceph-mon[40691]: 3.1 scrub ok 2024-09-16T09:33:23.493 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:23 smithi070 ceph-mon[40691]: 3.0 scrub starts 2024-09-16T09:33:23.493 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:23 smithi070 ceph-mon[40691]: 3.0 scrub ok 2024-09-16T09:33:23.494 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:23 smithi070 ceph-mon[40691]: 4.a scrub starts 2024-09-16T09:33:23.494 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:23 smithi070 ceph-mon[40691]: 4.a scrub ok 2024-09-16T09:33:23.519 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:33:23.540 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:23.540 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:23.541 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:33:23.642 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:33:23.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:23 smithi084 ceph-mon[32629]: pgmap v393: 152 pgs: 152 active+clean; 583 KiB data, 2.7 GiB used, 713 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-09-16T09:33:23.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:23 smithi084 ceph-mon[32629]: 3.1 scrub starts 2024-09-16T09:33:23.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:23 smithi084 ceph-mon[32629]: 3.1 scrub ok 2024-09-16T09:33:23.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:23 smithi084 ceph-mon[32629]: 3.0 scrub starts 2024-09-16T09:33:23.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:23 smithi084 ceph-mon[32629]: 3.0 scrub ok 2024-09-16T09:33:23.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:23 smithi084 ceph-mon[32629]: 4.a scrub starts 2024-09-16T09:33:23.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:23 smithi084 ceph-mon[32629]: 4.a scrub ok 2024-09-16T09:33:23.664 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:23.664 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:23.665 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:33:23.738 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.7 dump_ops_in_flight 2024-09-16T09:33:23.766 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:33:23.802 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:23.803 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:23.803 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:33:23.904 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:33:23.926 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:23.926 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:23.927 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:33:23.988 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:33:24.028 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:33:24.049 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:24.049 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:24.050 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:33:24.151 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:33:24.173 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:24.174 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:24.174 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:33:24.233 INFO:teuthology.orchestra.run.smithi070.stderr:noscrub is set 2024-09-16T09:33:24.237 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'unique_pool_2' created 2024-09-16T09:33:24.275 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:33:24.296 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:24.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:24.297 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:33:24.398 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:33:24.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:24 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:33:24.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:24 smithi084 ceph-mon[32629]: 3.9 scrub starts 2024-09-16T09:33:24.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:24 smithi084 ceph-mon[32629]: 3.9 scrub ok 2024-09-16T09:33:24.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:24 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/476695366' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:33:24.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:24 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3762756908' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "unique_pool_2", "pg_num": 16} : dispatch 2024-09-16T09:33:24.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:24 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "unique_pool_2", "pg_num": 16} : dispatch 2024-09-16T09:33:24.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:24 smithi084 ceph-mon[32629]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:33:24.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:24 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/476695366' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-16T09:33:24.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:24 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_2", "pg_num": 16}]': finished 2024-09-16T09:33:24.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:24 smithi084 ceph-mon[32629]: osdmap e229: 8 total, 8 up, 8 in 2024-09-16T09:33:24.420 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:24.421 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:24.421 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:33:24.463 INFO:teuthology.orchestra.run.smithi084.stdout:ERROR: (22) Invalid argument 2024-09-16T09:33:24.463 INFO:teuthology.orchestra.run.smithi084.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:33:24.522 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:33:24.544 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:24.545 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:24.545 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:33:24.646 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:33:24.669 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:24.670 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:24.671 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:33:24.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:24 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:33:24.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:24 smithi070 ceph-mon[30388]: 3.9 scrub starts 2024-09-16T09:33:24.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:24 smithi070 ceph-mon[30388]: 3.9 scrub ok 2024-09-16T09:33:24.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:24 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/476695366' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:33:24.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:24 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3762756908' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "unique_pool_2", "pg_num": 16} : dispatch 2024-09-16T09:33:24.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:24 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "unique_pool_2", "pg_num": 16} : dispatch 2024-09-16T09:33:24.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:24 smithi070 ceph-mon[30388]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:33:24.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:24 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/476695366' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-16T09:33:24.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:24 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_2", "pg_num": 16}]': finished 2024-09-16T09:33:24.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:24 smithi070 ceph-mon[30388]: osdmap e229: 8 total, 8 up, 8 in 2024-09-16T09:33:24.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:24 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:33:24.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:24 smithi070 ceph-mon[40691]: 3.9 scrub starts 2024-09-16T09:33:24.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:24 smithi070 ceph-mon[40691]: 3.9 scrub ok 2024-09-16T09:33:24.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:24 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/476695366' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:33:24.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:24 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3762756908' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "unique_pool_2", "pg_num": 16} : dispatch 2024-09-16T09:33:24.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:24 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "unique_pool_2", "pg_num": 16} : dispatch 2024-09-16T09:33:24.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:24 smithi070 ceph-mon[40691]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:33:24.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:24 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/476695366' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-16T09:33:24.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:24 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_2", "pg_num": 16}]': finished 2024-09-16T09:33:24.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:24 smithi070 ceph-mon[40691]: osdmap e229: 8 total, 8 up, 8 in 2024-09-16T09:33:24.771 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:33:24.794 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:24.794 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:24.795 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:33:24.895 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:33:24.917 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:24.918 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:24.918 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:33:25.001 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T09:33:25.019 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:33:25.042 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:25.043 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:25.043 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:33:25.144 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:33:25.165 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:25.166 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:25.166 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:33:25.233 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.7 dump_blocked_ops 2024-09-16T09:33:25.267 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:33:25.298 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:25.298 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:25.299 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:33:25.399 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:33:25.421 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:25.421 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:25.422 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:33:25.488 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:33:25.522 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:33:25.544 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:25.545 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:25.545 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:33:25.608 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:25 smithi084 ceph-mon[32629]: 5.8 scrub starts 2024-09-16T09:33:25.609 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:25 smithi084 ceph-mon[32629]: 5.8 scrub ok 2024-09-16T09:33:25.609 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:25 smithi084 ceph-mon[32629]: pgmap v394: 152 pgs: 152 active+clean; 583 KiB data, 2.7 GiB used, 713 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-09-16T09:33:25.609 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:25 smithi084 ceph-mon[32629]: 4.c scrub starts 2024-09-16T09:33:25.609 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:25 smithi084 ceph-mon[32629]: 4.c scrub ok 2024-09-16T09:33:25.609 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:25 smithi084 ceph-mon[32629]: osdmap e230: 8 total, 8 up, 8 in 2024-09-16T09:33:25.646 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:33:25.667 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:25.667 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:25.668 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:33:25.670 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:25 smithi070 ceph-mon[40691]: 5.8 scrub starts 2024-09-16T09:33:25.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:25 smithi070 ceph-mon[40691]: 5.8 scrub ok 2024-09-16T09:33:25.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:25 smithi070 ceph-mon[40691]: pgmap v394: 152 pgs: 152 active+clean; 583 KiB data, 2.7 GiB used, 713 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-09-16T09:33:25.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:25 smithi070 ceph-mon[40691]: 4.c scrub starts 2024-09-16T09:33:25.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:25 smithi070 ceph-mon[40691]: 4.c scrub ok 2024-09-16T09:33:25.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:25 smithi070 ceph-mon[40691]: osdmap e230: 8 total, 8 up, 8 in 2024-09-16T09:33:25.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:25 smithi070 ceph-mon[30388]: 5.8 scrub starts 2024-09-16T09:33:25.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:25 smithi070 ceph-mon[30388]: 5.8 scrub ok 2024-09-16T09:33:25.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:25 smithi070 ceph-mon[30388]: pgmap v394: 152 pgs: 152 active+clean; 583 KiB data, 2.7 GiB used, 713 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-09-16T09:33:25.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:25 smithi070 ceph-mon[30388]: 4.c scrub starts 2024-09-16T09:33:25.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:25 smithi070 ceph-mon[30388]: 4.c scrub ok 2024-09-16T09:33:25.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:25 smithi070 ceph-mon[30388]: osdmap e230: 8 total, 8 up, 8 in 2024-09-16T09:33:25.769 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:33:25.792 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:25.792 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:25.792 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:33:25.893 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:33:25.916 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:25.917 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:25.917 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:33:25.922 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd pool application enable unique_pool_2 rados --yes-i-really-mean-it || true 2024-09-16T09:33:25.926 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:33:26.018 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:33:26.040 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:26.041 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:26.041 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:33:26.142 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:33:26.164 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:26.164 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:26.165 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:33:26.246 INFO:teuthology.orchestra.run.smithi084.stdout:ERROR: (22) Invalid argument 2024-09-16T09:33:26.246 INFO:teuthology.orchestra.run.smithi084.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:33:26.265 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:33:26.287 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:26.288 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:26.288 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:33:26.389 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:33:26.414 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:26.414 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:26.415 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:33:26.427 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:33:26.427 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:33:26.447 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:33:26.449 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:33:26.449 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:33:26.466 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:33:26.466 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:33:26.486 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:33:26.486 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:33:26.503 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:33:26.503 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:33:26.515 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:33:26.519 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:33:26.519 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:33:26.536 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:33:26.537 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:33:26.537 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:26.538 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:26.538 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:33:26.552 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:33:26.553 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:33:26.639 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:33:26.660 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:26.661 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:26.661 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:33:26.762 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:33:26.785 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:26.785 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:26.786 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:33:26.825 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T09:33:26.825 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T09:33:26.826 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-16T09:33:26.826 INFO:tasks.thrashosds.thrasher:Setting osd 1 primary_affinity to 0.000000 2024-09-16T09:33:26.826 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd primary-affinity 1 0 2024-09-16T09:33:26.887 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:33:26.894 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.7 dump_historic_ops 2024-09-16T09:33:26.908 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:26.909 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:26.909 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:33:27.011 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:33:27.032 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:27.033 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:27.033 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:33:27.134 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:33:27.154 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:33:27.156 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:27.156 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:27.157 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:33:27.258 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:33:27.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:27.282 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:27.282 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:33:27.383 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:33:27.405 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:27.405 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:27.406 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:33:27.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:27 smithi070 ceph-mon[30388]: pgmap v397: 168 pgs: 16 unknown, 152 active+clean; 583 KiB data, 2.7 GiB used, 713 GiB / 715 GiB avail; 768 B/s rd, 0 op/s 2024-09-16T09:33:27.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:27 smithi070 ceph-mon[30388]: osdmap e231: 8 total, 8 up, 8 in 2024-09-16T09:33:27.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:27 smithi070 ceph-mon[30388]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T09:33:27.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:27 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/4282268307' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:33:27.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:27 smithi070 ceph-mon[40691]: pgmap v397: 168 pgs: 16 unknown, 152 active+clean; 583 KiB data, 2.7 GiB used, 713 GiB / 715 GiB avail; 768 B/s rd, 0 op/s 2024-09-16T09:33:27.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:27 smithi070 ceph-mon[40691]: osdmap e231: 8 total, 8 up, 8 in 2024-09-16T09:33:27.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:27 smithi070 ceph-mon[40691]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T09:33:27.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:27 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/4282268307' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:33:27.507 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:33:27.520 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:27 smithi084 ceph-mon[32629]: pgmap v397: 168 pgs: 16 unknown, 152 active+clean; 583 KiB data, 2.7 GiB used, 713 GiB / 715 GiB avail; 768 B/s rd, 0 op/s 2024-09-16T09:33:27.521 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:27 smithi084 ceph-mon[32629]: osdmap e231: 8 total, 8 up, 8 in 2024-09-16T09:33:27.521 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:27 smithi084 ceph-mon[32629]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T09:33:27.521 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:27 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/4282268307' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:33:27.528 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:27.529 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:27.529 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:33:27.622 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-09-16T09:33:27.622 INFO:teuthology.orchestra.run.smithi084.stdout: "size": 20, 2024-09-16T09:33:27.623 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 600, 2024-09-16T09:33:27.623 INFO:teuthology.orchestra.run.smithi084.stdout: "ops": [ 2024-09-16T09:33:27.623 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.623 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:22924 7.5 7:a7d40976:::benchmark_data_smithi070_78438_object22923:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T09:33:27.623 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:26:21.580769+0000", 2024-09-16T09:33:27.623 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 426.03950323999999, 2024-09-16T09:33:27.623 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.079602870000000006, 2024-09-16T09:33:27.623 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:33:27.623 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:33:27.623 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:33:27.623 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:33:27.623 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:33:27.623 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:33:27.624 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 22924 2024-09-16T09:33:27.624 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.624 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:33:27.624 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.624 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:33:27.624 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.580769+0000", 2024-09-16T09:33:27.624 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:33:27.624 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.624 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.624 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:33:27.624 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.580769+0000", 2024-09-16T09:33:27.624 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:33:27.625 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.625 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.625 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:33:27.625 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.580770+0000", 2024-09-16T09:33:27.625 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.3200000000000001e-06 2024-09-16T09:33:27.625 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.625 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.625 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:33:27.625 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.580848+0000", 2024-09-16T09:33:27.625 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.7514000000000004e-05 2024-09-16T09:33:27.625 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.625 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.625 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:33:27.626 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.580867+0000", 2024-09-16T09:33:27.626 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.8782000000000001e-05 2024-09-16T09:33:27.626 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.626 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.626 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:33:27.626 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.580874+0000", 2024-09-16T09:33:27.626 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.1269999999999997e-06 2024-09-16T09:33:27.626 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.626 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.626 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:33:27.626 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.580941+0000", 2024-09-16T09:33:27.626 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.7010999999999999e-05 2024-09-16T09:33:27.626 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.627 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.627 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:33:27.627 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.581054+0000", 2024-09-16T09:33:27.627 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000113509 2024-09-16T09:33:27.627 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.627 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.627 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,2", 2024-09-16T09:33:27.627 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.581176+0000", 2024-09-16T09:33:27.627 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000122187 2024-09-16T09:33:27.627 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.627 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.627 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:33:27.627 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.582818+0000", 2024-09-16T09:33:27.627 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0016419200000000001 2024-09-16T09:33:27.627 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.628 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.628 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:33:27.628 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.583304+0000", 2024-09-16T09:33:27.628 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00048596099999999998 2024-09-16T09:33:27.628 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.628 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.628 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:33:27.628 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.660281+0000", 2024-09-16T09:33:27.628 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.076976634000000002 2024-09-16T09:33:27.628 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.628 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.628 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:33:27.628 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.660322+0000", 2024-09-16T09:33:27.629 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.1125999999999999e-05 2024-09-16T09:33:27.629 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.629 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.629 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:33:27.629 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.660372+0000", 2024-09-16T09:33:27.629 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.9778999999999999e-05 2024-09-16T09:33:27.629 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:33:27.630 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:33:27.630 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:33:27.630 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.630 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.630 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:23476 7.5 7:ab69eea0:::benchmark_data_smithi070_78438_object23475:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T09:33:27.630 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:26:21.880371+0000", 2024-09-16T09:33:27.630 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 425.73990090699999, 2024-09-16T09:33:27.630 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.069588092000000004, 2024-09-16T09:33:27.630 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:33:27.630 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:33:27.630 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:33:27.630 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:33:27.631 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:33:27.631 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:33:27.631 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 23476 2024-09-16T09:33:27.631 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.631 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:33:27.631 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.631 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:33:27.631 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.880371+0000", 2024-09-16T09:33:27.631 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:33:27.631 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.631 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.631 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:33:27.631 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.880371+0000", 2024-09-16T09:33:27.631 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:33:27.632 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.632 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.632 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:33:27.632 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.880372+0000", 2024-09-16T09:33:27.632 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.4799999999999997e-07 2024-09-16T09:33:27.632 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.632 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.632 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:33:27.632 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.880461+0000", 2024-09-16T09:33:27.632 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.9232000000000002e-05 2024-09-16T09:33:27.632 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.632 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.633 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:33:27.633 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.880473+0000", 2024-09-16T09:33:27.633 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1687e-05 2024-09-16T09:33:27.633 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.633 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.633 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:33:27.633 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.880480+0000", 2024-09-16T09:33:27.633 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.3000000000000004e-06 2024-09-16T09:33:27.633 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.633 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.633 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:33:27.633 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.881388+0000", 2024-09-16T09:33:27.633 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00090752200000000002 2024-09-16T09:33:27.633 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.634 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.634 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:33:27.634 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.881520+0000", 2024-09-16T09:33:27.634 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00013243400000000001 2024-09-16T09:33:27.634 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.634 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.634 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,2", 2024-09-16T09:33:27.634 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.881649+0000", 2024-09-16T09:33:27.634 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00012842299999999999 2024-09-16T09:33:27.634 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.634 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.634 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:33:27.634 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.882506+0000", 2024-09-16T09:33:27.634 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000857047 2024-09-16T09:33:27.635 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.635 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.635 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:33:27.635 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.884941+0000", 2024-09-16T09:33:27.635 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0024354149999999998 2024-09-16T09:33:27.635 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.635 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.635 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:33:27.635 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.949871+0000", 2024-09-16T09:33:27.635 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.064929929999999997 2024-09-16T09:33:27.635 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.635 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.635 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:33:27.635 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.949921+0000", 2024-09-16T09:33:27.635 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.9660999999999998e-05 2024-09-16T09:33:27.636 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.636 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.636 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:33:27.636 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.949959+0000", 2024-09-16T09:33:27.636 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.8692999999999997e-05 2024-09-16T09:33:27.636 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:33:27.636 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:33:27.636 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:33:27.636 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.636 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.636 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:23481 7.5 7:a58daa7e:::benchmark_data_smithi070_78438_object23480:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T09:33:27.636 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:26:21.883877+0000", 2024-09-16T09:33:27.636 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 425.73639471899997, 2024-09-16T09:33:27.637 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.067196770000000003, 2024-09-16T09:33:27.637 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:33:27.637 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:33:27.637 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:33:27.639 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:33:27.639 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:33:27.640 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:33:27.640 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:33:27.640 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 23481 2024-09-16T09:33:27.640 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.640 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:33:27.641 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.641 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:33:27.641 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.883877+0000", 2024-09-16T09:33:27.641 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:33:27.641 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.641 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.641 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:33:27.641 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.883877+0000", 2024-09-16T09:33:27.641 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:33:27.641 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.641 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.641 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:33:27.641 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.883879+0000", 2024-09-16T09:33:27.642 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.144e-06 2024-09-16T09:33:27.642 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.642 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.642 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:33:27.642 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.883931+0000", 2024-09-16T09:33:27.642 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.2123000000000002e-05 2024-09-16T09:33:27.642 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.642 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.642 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:33:27.642 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.883943+0000", 2024-09-16T09:33:27.642 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.2323999999999999e-05 2024-09-16T09:33:27.642 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.642 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.643 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:33:27.643 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.883950+0000", 2024-09-16T09:33:27.643 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.3900000000000004e-06 2024-09-16T09:33:27.643 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.643 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.643 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:33:27.643 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.884006+0000", 2024-09-16T09:33:27.643 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.5387999999999999e-05 2024-09-16T09:33:27.643 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.643 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.643 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:33:27.643 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.884117+0000", 2024-09-16T09:33:27.643 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000111461 2024-09-16T09:33:27.643 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.643 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.643 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,2", 2024-09-16T09:33:27.644 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.884259+0000", 2024-09-16T09:33:27.644 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000141686 2024-09-16T09:33:27.644 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.644 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.644 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:33:27.644 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.885063+0000", 2024-09-16T09:33:27.644 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00080437399999999995 2024-09-16T09:33:27.644 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.644 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.644 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:33:27.644 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.886156+0000", 2024-09-16T09:33:27.645 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0010922130000000001 2024-09-16T09:33:27.645 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.645 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.645 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:33:27.645 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.950978+0000", 2024-09-16T09:33:27.645 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.064822639000000001 2024-09-16T09:33:27.645 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.645 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.645 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:33:27.645 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.951036+0000", 2024-09-16T09:33:27.645 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.7666e-05 2024-09-16T09:33:27.645 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.646 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.646 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:33:27.646 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:21.951074+0000", 2024-09-16T09:33:27.646 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.8362000000000003e-05 2024-09-16T09:33:27.646 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:33:27.646 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:33:27.646 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:33:27.646 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.646 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.646 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_repop(client.24698.0:27466 7.6 e89/82 7:60c44c8b:::benchmark_data_smithi070_78438_object27465:head v 89'1726, mlcod=89'1723)", 2024-09-16T09:33:27.646 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:26:23.722364+0000", 2024-09-16T09:33:27.646 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 423.897908352, 2024-09-16T09:33:27.646 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.071550814000000004, 2024-09-16T09:33:27.647 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:33:27.647 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:33:27.647 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:33:27.647 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:33:27.647 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.647 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:33:27.647 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.722364+0000", 2024-09-16T09:33:27.647 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:33:27.647 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.647 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.647 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:33:27.647 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.722364+0000", 2024-09-16T09:33:27.647 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:33:27.648 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.648 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.648 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:33:27.648 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.722364+0000", 2024-09-16T09:33:27.648 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.0699999999999998e-07 2024-09-16T09:33:27.648 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.648 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.648 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:33:27.648 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.722426+0000", 2024-09-16T09:33:27.648 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.1278999999999994e-05 2024-09-16T09:33:27.648 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.648 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.648 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:33:27.648 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.722432+0000", 2024-09-16T09:33:27.649 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.003e-06 2024-09-16T09:33:27.649 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.649 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.649 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:33:27.649 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.722435+0000", 2024-09-16T09:33:27.649 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.4010000000000001e-06 2024-09-16T09:33:27.649 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.649 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.649 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:33:27.649 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.722461+0000", 2024-09-16T09:33:27.649 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.5894999999999999e-05 2024-09-16T09:33:27.649 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.649 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.650 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:33:27.650 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.722471+0000", 2024-09-16T09:33:27.650 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0251000000000001e-05 2024-09-16T09:33:27.650 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.650 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.650 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:33:27.650 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.793863+0000", 2024-09-16T09:33:27.650 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.071392294999999995 2024-09-16T09:33:27.650 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.651 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.651 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:33:27.651 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.793915+0000", 2024-09-16T09:33:27.651 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.1282999999999997e-05 2024-09-16T09:33:27.651 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:33:27.651 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:33:27.651 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:33:27.651 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.651 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.651 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_repop(client.24698.0:27468 7.8 e89/82 7:1de713de:::benchmark_data_smithi070_78438_object27467:head v 89'1721, mlcod=89'1718)", 2024-09-16T09:33:27.651 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:26:23.722887+0000", 2024-09-16T09:33:27.651 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 423.897385332, 2024-09-16T09:33:27.651 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.069612414999999997, 2024-09-16T09:33:27.652 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:33:27.652 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:33:27.652 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:33:27.652 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:33:27.652 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.652 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:33:27.652 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.722887+0000", 2024-09-16T09:33:27.652 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:33:27.652 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.652 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.652 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:33:27.652 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.722887+0000", 2024-09-16T09:33:27.652 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:33:27.652 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.653 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.653 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:33:27.653 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.722887+0000", 2024-09-16T09:33:27.653 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.0100000000000005e-07 2024-09-16T09:33:27.653 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.653 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.653 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:33:27.653 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.722949+0000", 2024-09-16T09:33:27.653 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.1298999999999997e-05 2024-09-16T09:33:27.653 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.653 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.653 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:33:27.653 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.790454+0000", 2024-09-16T09:33:27.654 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.067505342999999995 2024-09-16T09:33:27.654 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.654 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.654 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:33:27.654 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.790465+0000", 2024-09-16T09:33:27.654 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1242e-05 2024-09-16T09:33:27.654 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.654 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.654 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:33:27.654 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.790558+0000", 2024-09-16T09:33:27.654 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.3227999999999996e-05 2024-09-16T09:33:27.654 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.654 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.654 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:33:27.654 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.790578+0000", 2024-09-16T09:33:27.655 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.9114999999999999e-05 2024-09-16T09:33:27.655 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.655 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.655 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:33:27.655 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.792380+0000", 2024-09-16T09:33:27.655 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0018023329999999999 2024-09-16T09:33:27.655 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.655 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.655 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:33:27.655 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.792499+0000", 2024-09-16T09:33:27.655 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000119354 2024-09-16T09:33:27.655 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:33:27.655 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:33:27.656 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:33:27.656 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.656 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.656 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:27473 7.5 7:a272c321:::benchmark_data_smithi070_78438_object27472:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-16T09:33:27.656 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:26:23.723821+0000", 2024-09-16T09:33:27.656 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 423.89645074100002, 2024-09-16T09:33:27.656 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.073668624000000002, 2024-09-16T09:33:27.656 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:33:27.656 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:33:27.656 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:33:27.656 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:33:27.656 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:33:27.656 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:33:27.656 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 27473 2024-09-16T09:33:27.657 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.657 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:33:27.657 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.657 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:33:27.657 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.723821+0000", 2024-09-16T09:33:27.657 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:33:27.657 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.657 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.657 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:33:27.657 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.723821+0000", 2024-09-16T09:33:27.657 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:33:27.657 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.658 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.658 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:33:27.658 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.723823+0000", 2024-09-16T09:33:27.658 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1999999999999999e-06 2024-09-16T09:33:27.658 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.658 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.658 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:33:27.658 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.723884+0000", 2024-09-16T09:33:27.659 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.0971000000000003e-05 2024-09-16T09:33:27.659 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.659 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.659 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:33:27.659 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.790196+0000", 2024-09-16T09:33:27.659 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.066312372999999994 2024-09-16T09:33:27.659 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.659 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.659 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:33:27.659 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.790213+0000", 2024-09-16T09:33:27.659 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.6974e-05 2024-09-16T09:33:27.659 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.659 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.659 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:33:27.659 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.790559+0000", 2024-09-16T09:33:27.660 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00034600599999999999 2024-09-16T09:33:27.660 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.660 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.660 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:33:27.660 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.790669+0000", 2024-09-16T09:33:27.660 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000110276 2024-09-16T09:33:27.660 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.660 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.661 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,2", 2024-09-16T09:33:27.661 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.790788+0000", 2024-09-16T09:33:27.661 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000119096 2024-09-16T09:33:27.661 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.661 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.661 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:33:27.661 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.793102+0000", 2024-09-16T09:33:27.661 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0023135030000000002 2024-09-16T09:33:27.661 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.661 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.661 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:33:27.662 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.793337+0000", 2024-09-16T09:33:27.662 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000235481 2024-09-16T09:33:27.662 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.662 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.662 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:33:27.662 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.797381+0000", 2024-09-16T09:33:27.662 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0040435699999999998 2024-09-16T09:33:27.662 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.662 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.663 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:33:27.663 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.797446+0000", 2024-09-16T09:33:27.663 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.4759999999999997e-05 2024-09-16T09:33:27.663 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.663 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.663 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:33:27.663 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:23.797490+0000", 2024-09-16T09:33:27.663 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.4413999999999999e-05 2024-09-16T09:33:27.663 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:33:27.663 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:33:27.664 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:33:27.664 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.664 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.664 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_repop(client.24698.0:28810 7.6 e90/82 7:63b6347d:::benchmark_data_smithi070_78438_object28809:head v 90'1809, mlcod=90'1805)", 2024-09-16T09:33:27.664 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:26:24.482328+0000", 2024-09-16T09:33:27.664 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 423.13794390300001, 2024-09-16T09:33:27.664 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.063714949000000007, 2024-09-16T09:33:27.664 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:33:27.664 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:33:27.664 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:33:27.664 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:33:27.664 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.664 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:33:27.664 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.482328+0000", 2024-09-16T09:33:27.665 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:33:27.665 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.665 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.665 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:33:27.665 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.482328+0000", 2024-09-16T09:33:27.665 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:33:27.665 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.665 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.665 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:33:27.665 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.482329+0000", 2024-09-16T09:33:27.665 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.0200000000000002e-07 2024-09-16T09:33:27.665 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.665 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.665 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:33:27.666 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.482394+0000", 2024-09-16T09:33:27.666 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.4706e-05 2024-09-16T09:33:27.666 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.666 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.666 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:33:27.666 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.543399+0000", 2024-09-16T09:33:27.666 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.061005750999999997 2024-09-16T09:33:27.666 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.666 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.666 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:33:27.666 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.543411+0000", 2024-09-16T09:33:27.666 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1687e-05 2024-09-16T09:33:27.666 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.666 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.667 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:33:27.667 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.543539+0000", 2024-09-16T09:33:27.667 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000128298 2024-09-16T09:33:27.667 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.667 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.667 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:33:27.667 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.543558+0000", 2024-09-16T09:33:27.667 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.8323999999999999e-05 2024-09-16T09:33:27.667 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.667 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.667 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:33:27.668 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.546006+0000", 2024-09-16T09:33:27.668 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0024483180000000001 2024-09-16T09:33:27.668 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.668 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.668 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:33:27.668 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:24.546043+0000", 2024-09-16T09:33:27.668 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.7262999999999999e-05 2024-09-16T09:33:27.668 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:33:27.668 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:33:27.668 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:33:27.668 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.669 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.669 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:62033 7.5 7:a5dfc1ea:::benchmark_data_smithi070_78438_object62032:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-16T09:33:27.669 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:26:45.811217+0000", 2024-09-16T09:33:27.669 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 401.80905537699999, 2024-09-16T09:33:27.669 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.071578969000000006, 2024-09-16T09:33:27.669 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:33:27.669 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:33:27.669 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:33:27.669 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:33:27.669 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:33:27.669 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:33:27.669 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 62033 2024-09-16T09:33:27.669 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.669 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:33:27.670 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.670 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:33:27.670 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.811217+0000", 2024-09-16T09:33:27.670 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:33:27.670 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.670 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.670 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:33:27.670 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.811217+0000", 2024-09-16T09:33:27.670 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:33:27.670 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.670 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.670 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:33:27.670 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.811218+0000", 2024-09-16T09:33:27.670 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.5290000000000001e-06 2024-09-16T09:33:27.671 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.671 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.671 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:33:27.671 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.811262+0000", 2024-09-16T09:33:27.671 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.3850999999999997e-05 2024-09-16T09:33:27.671 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.671 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.671 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:33:27.671 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.811278+0000", 2024-09-16T09:33:27.671 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.5753999999999999e-05 2024-09-16T09:33:27.672 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.672 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.672 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:33:27.672 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.811286+0000", 2024-09-16T09:33:27.672 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.7519999999999996e-06 2024-09-16T09:33:27.672 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.672 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.672 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:33:27.672 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.811333+0000", 2024-09-16T09:33:27.672 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.7203000000000002e-05 2024-09-16T09:33:27.672 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.672 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.672 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:33:27.673 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.811456+0000", 2024-09-16T09:33:27.673 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00012353900000000001 2024-09-16T09:33:27.673 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.673 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.673 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,2", 2024-09-16T09:33:27.673 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.811644+0000", 2024-09-16T09:33:27.673 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00018799100000000001 2024-09-16T09:33:27.673 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.673 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.673 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:33:27.673 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.814035+0000", 2024-09-16T09:33:27.673 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0023902009999999998 2024-09-16T09:33:27.673 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.673 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.674 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:33:27.674 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.815646+0000", 2024-09-16T09:33:27.674 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0016112399999999999 2024-09-16T09:33:27.674 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.674 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.674 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:33:27.674 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.882683+0000", 2024-09-16T09:33:27.674 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.067037332000000005 2024-09-16T09:33:27.674 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.674 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.674 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:33:27.674 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.882753+0000", 2024-09-16T09:33:27.674 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.9534000000000002e-05 2024-09-16T09:33:27.674 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.674 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.675 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:33:27.675 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.882796+0000", 2024-09-16T09:33:27.675 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.3043000000000001e-05 2024-09-16T09:33:27.675 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:33:27.675 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:33:27.675 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:33:27.675 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.675 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.676 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:62044 7.5 7:aee12d22:::benchmark_data_smithi070_78438_object62043:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-16T09:33:27.676 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:26:45.819031+0000", 2024-09-16T09:33:27.676 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 401.80124151699999, 2024-09-16T09:33:27.676 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.063892420000000005, 2024-09-16T09:33:27.676 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:33:27.676 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:33:27.676 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:33:27.677 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:33:27.677 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:33:27.677 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:33:27.677 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 62044 2024-09-16T09:33:27.678 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.678 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:33:27.678 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.678 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:33:27.678 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.819031+0000", 2024-09-16T09:33:27.678 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:33:27.678 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.678 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.678 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:33:27.678 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.819031+0000", 2024-09-16T09:33:27.678 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:33:27.678 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.678 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.679 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:33:27.679 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.819032+0000", 2024-09-16T09:33:27.679 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.3120000000000001e-06 2024-09-16T09:33:27.679 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.679 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.679 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:33:27.679 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.819071+0000", 2024-09-16T09:33:27.679 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.8599000000000001e-05 2024-09-16T09:33:27.679 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.679 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.679 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:33:27.679 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.819081+0000", 2024-09-16T09:33:27.680 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0812000000000001e-05 2024-09-16T09:33:27.680 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.680 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.680 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:33:27.680 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.819089+0000", 2024-09-16T09:33:27.680 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.8089999999999993e-06 2024-09-16T09:33:27.680 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.680 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.680 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:33:27.680 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.819138+0000", 2024-09-16T09:33:27.680 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.8361e-05 2024-09-16T09:33:27.680 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.680 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.680 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:33:27.681 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.819248+0000", 2024-09-16T09:33:27.681 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000110819 2024-09-16T09:33:27.681 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.681 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.681 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,2", 2024-09-16T09:33:27.681 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.819399+0000", 2024-09-16T09:33:27.681 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000150674 2024-09-16T09:33:27.681 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.681 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.681 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:33:27.681 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.820979+0000", 2024-09-16T09:33:27.681 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0015803589999999999 2024-09-16T09:33:27.681 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.682 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.682 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:33:27.682 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.821238+0000", 2024-09-16T09:33:27.682 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00025873699999999999 2024-09-16T09:33:27.682 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.682 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.682 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:33:27.682 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.882832+0000", 2024-09-16T09:33:27.682 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.061593649 2024-09-16T09:33:27.682 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.682 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.682 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:33:27.683 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.882879+0000", 2024-09-16T09:33:27.683 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.6879000000000003e-05 2024-09-16T09:33:27.683 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.683 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.683 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:33:27.683 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:45.882923+0000", 2024-09-16T09:33:27.683 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.4409999999999997e-05 2024-09-16T09:33:27.683 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:33:27.683 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:33:27.683 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:33:27.683 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.683 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.683 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:63392 7.5 7:af2804a3:::benchmark_data_smithi070_78438_object63391:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-09-16T09:33:27.684 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:26:46.805497+0000", 2024-09-16T09:33:27.684 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 400.814775387, 2024-09-16T09:33:27.684 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.064734542000000006, 2024-09-16T09:33:27.684 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:33:27.684 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:33:27.684 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:33:27.684 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:33:27.684 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:33:27.684 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:33:27.684 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 63392 2024-09-16T09:33:27.684 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.684 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:33:27.684 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.684 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:33:27.685 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:46.805497+0000", 2024-09-16T09:33:27.685 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:33:27.685 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.685 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.685 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:33:27.685 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:46.805497+0000", 2024-09-16T09:33:27.685 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:33:27.685 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.685 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.685 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:33:27.685 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:46.805498+0000", 2024-09-16T09:33:27.685 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.6789999999999999e-06 2024-09-16T09:33:27.685 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.685 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.686 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:33:27.686 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:46.805542+0000", 2024-09-16T09:33:27.686 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.3123e-05 2024-09-16T09:33:27.686 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.686 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.686 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:33:27.686 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:46.805555+0000", 2024-09-16T09:33:27.686 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.3777e-05 2024-09-16T09:33:27.686 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.686 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.686 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:33:27.686 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:46.805563+0000", 2024-09-16T09:33:27.686 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.4290000000000004e-06 2024-09-16T09:33:27.686 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.686 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.687 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:33:27.687 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:46.805618+0000", 2024-09-16T09:33:27.687 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.5559000000000001e-05 2024-09-16T09:33:27.687 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.687 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.687 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:33:27.687 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:46.805705+0000", 2024-09-16T09:33:27.687 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.7059999999999997e-05 2024-09-16T09:33:27.687 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.687 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.687 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,2", 2024-09-16T09:33:27.687 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:46.805786+0000", 2024-09-16T09:33:27.687 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.0076999999999999e-05 2024-09-16T09:33:27.688 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.688 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.688 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:33:27.688 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:46.807389+0000", 2024-09-16T09:33:27.688 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0016034160000000001 2024-09-16T09:33:27.688 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.688 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.688 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:33:27.688 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:46.807761+0000", 2024-09-16T09:33:27.688 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00037185799999999998 2024-09-16T09:33:27.688 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.688 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.688 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:33:27.689 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:46.870158+0000", 2024-09-16T09:33:27.689 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.062397059999999997 2024-09-16T09:33:27.689 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.689 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.689 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:33:27.689 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:46.870193+0000", 2024-09-16T09:33:27.689 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.4777000000000003e-05 2024-09-16T09:33:27.689 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.689 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.689 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:33:27.689 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:26:46.870231+0000", 2024-09-16T09:33:27.689 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.8726999999999997e-05 2024-09-16T09:33:27.689 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:33:27.689 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:33:27.690 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:33:27.690 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.690 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.690 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:202401 7.5 7:a2fb40f6:::benchmark_data_smithi070_78438_object76270:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135)", 2024-09-16T09:33:27.690 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:28:50.720099+0000", 2024-09-16T09:33:27.690 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 276.900172881, 2024-09-16T09:33:27.690 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.089978474000000003, 2024-09-16T09:33:27.690 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:33:27.690 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:33:27.690 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:33:27.690 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:33:27.690 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:33:27.690 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:33:27.690 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 202401 2024-09-16T09:33:27.691 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.691 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:33:27.691 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.691 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:33:27.691 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.720099+0000", 2024-09-16T09:33:27.691 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:33:27.691 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.691 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.691 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:33:27.691 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.720099+0000", 2024-09-16T09:33:27.691 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:33:27.691 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.691 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.691 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:33:27.692 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.720101+0000", 2024-09-16T09:33:27.692 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.5889999999999999e-06 2024-09-16T09:33:27.692 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.692 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.692 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:33:27.692 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.720107+0000", 2024-09-16T09:33:27.692 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.6520000000000003e-06 2024-09-16T09:33:27.692 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.692 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.692 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:33:27.692 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.720117+0000", 2024-09-16T09:33:27.692 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0254999999999999e-05 2024-09-16T09:33:27.692 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.693 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.693 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:33:27.693 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.720122+0000", 2024-09-16T09:33:27.693 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.2580000000000003e-06 2024-09-16T09:33:27.693 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.693 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.693 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:33:27.693 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.720178+0000", 2024-09-16T09:33:27.693 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.5501000000000003e-05 2024-09-16T09:33:27.693 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.693 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.693 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:33:27.693 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.720290+0000", 2024-09-16T09:33:27.693 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000112837 2024-09-16T09:33:27.694 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.694 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.694 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,2", 2024-09-16T09:33:27.694 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.720394+0000", 2024-09-16T09:33:27.694 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000103613 2024-09-16T09:33:27.694 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.694 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.694 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:33:27.694 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.722666+0000", 2024-09-16T09:33:27.694 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0022723800000000001 2024-09-16T09:33:27.694 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.694 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.694 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:33:27.694 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.725231+0000", 2024-09-16T09:33:27.695 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0025650759999999999 2024-09-16T09:33:27.695 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.695 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.695 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:33:27.695 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.809799+0000", 2024-09-16T09:33:27.695 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.084568014999999996 2024-09-16T09:33:27.695 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.695 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.695 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:33:27.695 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.810033+0000", 2024-09-16T09:33:27.695 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00023360000000000001 2024-09-16T09:33:27.695 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.695 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.695 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:33:27.696 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.810078+0000", 2024-09-16T09:33:27.696 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.4697999999999998e-05 2024-09-16T09:33:27.696 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:33:27.696 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:33:27.696 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:33:27.696 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.696 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.696 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:202408 7.5 7:a7ac5683:::benchmark_data_smithi070_78438_object76277:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135)", 2024-09-16T09:33:27.696 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:28:50.725540+0000", 2024-09-16T09:33:27.696 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 276.89473253400001, 2024-09-16T09:33:27.696 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.090820330000000005, 2024-09-16T09:33:27.696 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:33:27.696 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:33:27.696 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:33:27.697 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:33:27.697 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:33:27.697 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:33:27.697 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 202408 2024-09-16T09:33:27.697 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.697 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:33:27.697 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.697 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:33:27.697 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.725540+0000", 2024-09-16T09:33:27.697 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:33:27.697 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.697 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.697 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:33:27.697 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.725540+0000", 2024-09-16T09:33:27.697 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:33:27.698 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.698 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.698 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:33:27.698 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.725541+0000", 2024-09-16T09:33:27.698 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.5069999999999999e-06 2024-09-16T09:33:27.698 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.698 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.698 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:33:27.698 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.725546+0000", 2024-09-16T09:33:27.698 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.113e-06 2024-09-16T09:33:27.698 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.698 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.698 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:33:27.698 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.725558+0000", 2024-09-16T09:33:27.698 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.2164000000000001e-05 2024-09-16T09:33:27.699 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.699 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.699 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:33:27.699 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.725565+0000", 2024-09-16T09:33:27.699 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.8850000000000001e-06 2024-09-16T09:33:27.699 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.699 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.699 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:33:27.699 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.725617+0000", 2024-09-16T09:33:27.699 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.1848000000000002e-05 2024-09-16T09:33:27.699 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.699 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.699 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:33:27.699 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.725728+0000", 2024-09-16T09:33:27.700 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00011048300000000001 2024-09-16T09:33:27.700 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.700 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.700 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,2", 2024-09-16T09:33:27.700 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.725839+0000", 2024-09-16T09:33:27.700 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00011105200000000001 2024-09-16T09:33:27.700 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.700 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.700 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:33:27.702 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:27.702 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.810121+0000", 2024-09-16T09:33:27.702 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.084282652 2024-09-16T09:33:27.702 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.702 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.702 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:33:27.703 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.816197+0000", 2024-09-16T09:33:27.703 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0060759309999999997 2024-09-16T09:33:27.703 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.703 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.703 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:33:27.703 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.816253+0000", 2024-09-16T09:33:27.703 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.5912999999999998e-05 2024-09-16T09:33:27.703 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.703 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.703 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:33:27.703 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.816308+0000", 2024-09-16T09:33:27.703 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.4511e-05 2024-09-16T09:33:27.703 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.703 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.704 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:33:27.704 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.816360+0000", 2024-09-16T09:33:27.704 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.2271000000000001e-05 2024-09-16T09:33:27.704 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:33:27.704 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:33:27.704 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:33:27.704 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.704 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.704 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24424.0:218 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch (1) in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e134)", 2024-09-16T09:33:27.704 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:28:50.729011+0000", 2024-09-16T09:33:27.704 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 276.89126140600001, 2024-09-16T09:33:27.704 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.078508791999999994, 2024-09-16T09:33:27.704 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:33:27.704 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:33:27.705 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "started", 2024-09-16T09:33:27.705 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:33:27.705 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24424", 2024-09-16T09:33:27.705 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.84:0/1181633158", 2024-09-16T09:33:27.705 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 218 2024-09-16T09:33:27.705 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.705 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:33:27.705 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.705 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:33:27.705 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.729011+0000", 2024-09-16T09:33:27.705 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:33:27.705 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.705 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.705 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:33:27.705 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.729011+0000", 2024-09-16T09:33:27.706 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:33:27.706 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.706 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.706 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:33:27.706 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.729012+0000", 2024-09-16T09:33:27.706 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.6270000000000001e-06 2024-09-16T09:33:27.706 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.706 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.706 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:33:27.706 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.729019+0000", 2024-09-16T09:33:27.706 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.8889999999999996e-06 2024-09-16T09:33:27.706 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.706 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.706 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:33:27.706 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.806924+0000", 2024-09-16T09:33:27.706 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.077904235000000002 2024-09-16T09:33:27.707 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.707 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.707 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:33:27.707 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.806944+0000", 2024-09-16T09:33:27.707 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.039e-05 2024-09-16T09:33:27.707 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.707 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.707 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:33:27.707 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.807345+0000", 2024-09-16T09:33:27.707 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00040116499999999999 2024-09-16T09:33:27.707 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.707 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.707 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:33:27.707 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.807432+0000", 2024-09-16T09:33:27.707 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.6515999999999996e-05 2024-09-16T09:33:27.708 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.708 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.708 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:33:27.708 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.807520+0000", 2024-09-16T09:33:27.708 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.797e-05 2024-09-16T09:33:27.708 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:33:27.708 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:33:27.708 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:33:27.708 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.708 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.708 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_repop_reply(client.24698.0:202401 7.5 e135/82 ondisk, result = 0)", 2024-09-16T09:33:27.708 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:28:50.730604+0000", 2024-09-16T09:33:27.708 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 276.88966827600001, 2024-09-16T09:33:27.709 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.079479822000000006, 2024-09-16T09:33:27.709 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:33:27.709 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:33:27.709 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "started", 2024-09-16T09:33:27.709 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:33:27.709 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.709 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:33:27.709 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.730604+0000", 2024-09-16T09:33:27.709 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:33:27.709 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.709 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.709 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:33:27.709 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.730604+0000", 2024-09-16T09:33:27.709 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:33:27.709 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.710 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.710 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:33:27.710 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.730605+0000", 2024-09-16T09:33:27.710 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.0999999999999997e-07 2024-09-16T09:33:27.710 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.710 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.710 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:33:27.710 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.730609+0000", 2024-09-16T09:33:27.710 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.9929999999999997e-06 2024-09-16T09:33:27.710 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.710 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.710 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:33:27.710 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.806968+0000", 2024-09-16T09:33:27.710 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.076359716999999994 2024-09-16T09:33:27.711 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.711 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.711 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:33:27.711 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.806980+0000", 2024-09-16T09:33:27.711 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1551e-05 2024-09-16T09:33:27.711 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.711 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.711 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:33:27.711 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.809786+0000", 2024-09-16T09:33:27.711 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0028062339999999999 2024-09-16T09:33:27.711 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.712 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.712 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:33:27.712 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.809794+0000", 2024-09-16T09:33:27.712 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.4630000000000004e-06 2024-09-16T09:33:27.712 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.712 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.712 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:33:27.712 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.810084+0000", 2024-09-16T09:33:27.712 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00029005399999999998 2024-09-16T09:33:27.712 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:33:27.713 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:33:27.713 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:33:27.713 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.713 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.713 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:202411 7.5 7:a7b52676:::benchmark_data_smithi070_78438_object76280:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135)", 2024-09-16T09:33:27.713 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:28:50.732821+0000", 2024-09-16T09:33:27.713 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 276.88745124100001, 2024-09-16T09:33:27.713 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.10429561900000001, 2024-09-16T09:33:27.713 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:33:27.713 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:33:27.713 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:33:27.713 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:33:27.714 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:33:27.714 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:33:27.714 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 202411 2024-09-16T09:33:27.714 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.714 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:33:27.714 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.714 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:33:27.714 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.732821+0000", 2024-09-16T09:33:27.714 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:33:27.714 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.714 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.714 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:33:27.714 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.732821+0000", 2024-09-16T09:33:27.715 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:33:27.715 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.715 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.715 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:33:27.715 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.732822+0000", 2024-09-16T09:33:27.715 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.46e-06 2024-09-16T09:33:27.715 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.715 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.715 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:33:27.715 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.732828+0000", 2024-09-16T09:33:27.715 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.4240000000000001e-06 2024-09-16T09:33:27.715 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.715 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.715 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:33:27.716 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.806990+0000", 2024-09-16T09:33:27.716 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.074162464999999997 2024-09-16T09:33:27.716 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.716 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.716 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:33:27.716 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.806998+0000", 2024-09-16T09:33:27.716 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.0369999999999995e-06 2024-09-16T09:33:27.716 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.716 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.716 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:33:27.716 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.810149+0000", 2024-09-16T09:33:27.716 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0031503339999999999 2024-09-16T09:33:27.716 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.717 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.717 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:33:27.717 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.810253+0000", 2024-09-16T09:33:27.717 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00010388200000000001 2024-09-16T09:33:27.717 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.717 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.717 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,2", 2024-09-16T09:33:27.717 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.810356+0000", 2024-09-16T09:33:27.717 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000103319 2024-09-16T09:33:27.717 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.717 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.717 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:33:27.717 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.816207+0000", 2024-09-16T09:33:27.718 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.005850984 2024-09-16T09:33:27.718 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.718 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.718 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:33:27.718 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.828515+0000", 2024-09-16T09:33:27.718 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.012307904 2024-09-16T09:33:27.718 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.718 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.718 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:33:27.718 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.837029+0000", 2024-09-16T09:33:27.718 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0085145419999999999 2024-09-16T09:33:27.718 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.718 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.718 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:33:27.719 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.837071+0000", 2024-09-16T09:33:27.719 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.1718999999999998e-05 2024-09-16T09:33:27.719 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.719 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.719 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:33:27.719 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:50.837117+0000", 2024-09-16T09:33:27.719 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.5549e-05 2024-09-16T09:33:27.719 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:33:27.719 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:33:27.719 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:33:27.719 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.719 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.719 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:204901 7.5 7:af219aea:::benchmark_data_smithi070_78438_object78770:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137)", 2024-09-16T09:33:27.719 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:28:54.030373+0000", 2024-09-16T09:33:27.720 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 273.58989957099999, 2024-09-16T09:33:27.720 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.077126564999999994, 2024-09-16T09:33:27.720 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:33:27.720 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:33:27.720 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:33:27.720 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:33:27.720 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:33:27.720 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:33:27.720 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 204901 2024-09-16T09:33:27.720 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.720 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:33:27.720 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.720 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:33:27.721 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.030373+0000", 2024-09-16T09:33:27.721 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:33:27.721 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.721 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.721 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:33:27.721 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.030373+0000", 2024-09-16T09:33:27.721 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:33:27.721 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.721 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.721 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:33:27.721 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.030374+0000", 2024-09-16T09:33:27.721 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.389e-06 2024-09-16T09:33:27.721 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.721 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.722 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:33:27.722 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.030378+0000", 2024-09-16T09:33:27.722 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.1679999999999997e-06 2024-09-16T09:33:27.722 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.722 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.722 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:33:27.722 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.030389+0000", 2024-09-16T09:33:27.722 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.097e-05 2024-09-16T09:33:27.722 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.722 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.722 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:33:27.722 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.030395+0000", 2024-09-16T09:33:27.722 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.229e-06 2024-09-16T09:33:27.723 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.723 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.723 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:33:27.723 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.030446+0000", 2024-09-16T09:33:27.723 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.0757999999999998e-05 2024-09-16T09:33:27.723 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.723 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.723 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:33:27.723 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.030553+0000", 2024-09-16T09:33:27.723 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00010718900000000001 2024-09-16T09:33:27.723 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.723 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.723 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,2", 2024-09-16T09:33:27.724 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.030664+0000", 2024-09-16T09:33:27.724 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00011086600000000001 2024-09-16T09:33:27.724 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.724 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.724 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:33:27.724 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.033351+0000", 2024-09-16T09:33:27.724 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.002686875 2024-09-16T09:33:27.724 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.724 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.724 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:33:27.724 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.033716+0000", 2024-09-16T09:33:27.724 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00036531200000000002 2024-09-16T09:33:27.724 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.725 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.725 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:33:27.725 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.107394+0000", 2024-09-16T09:33:27.725 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.073677379000000001 2024-09-16T09:33:27.725 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.725 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.725 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:33:27.725 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.107450+0000", 2024-09-16T09:33:27.725 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.6351999999999999e-05 2024-09-16T09:33:27.725 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.725 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.725 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:33:27.725 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.107499+0000", 2024-09-16T09:33:27.725 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.9078e-05 2024-09-16T09:33:27.726 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:33:27.726 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:33:27.726 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:33:27.726 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.726 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.726 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:204910 7.5 7:a16392ce:::benchmark_data_smithi070_78438_object78779:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e137)", 2024-09-16T09:33:27.726 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:28:54.036796+0000", 2024-09-16T09:33:27.726 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 273.58347609399999, 2024-09-16T09:33:27.726 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.070834338999999996, 2024-09-16T09:33:27.726 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:33:27.726 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:33:27.726 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:33:27.726 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:33:27.726 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:33:27.727 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:33:27.727 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 204910 2024-09-16T09:33:27.727 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.727 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:33:27.727 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.727 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:33:27.727 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.036796+0000", 2024-09-16T09:33:27.727 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:33:27.727 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.727 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.727 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:33:27.727 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.036796+0000", 2024-09-16T09:33:27.727 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:33:27.727 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.728 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.728 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:33:27.728 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.036798+0000", 2024-09-16T09:33:27.728 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.649e-06 2024-09-16T09:33:27.728 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.728 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.728 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:33:27.728 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.036805+0000", 2024-09-16T09:33:27.728 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.0840000000000003e-06 2024-09-16T09:33:27.728 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.728 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.728 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:33:27.728 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.036824+0000", 2024-09-16T09:33:27.728 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.9318e-05 2024-09-16T09:33:27.729 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.729 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.729 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:33:27.729 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.036832+0000", 2024-09-16T09:33:27.729 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.4000000000000003e-06 2024-09-16T09:33:27.729 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.729 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.729 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:33:27.729 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.036897+0000", 2024-09-16T09:33:27.729 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.5238000000000001e-05 2024-09-16T09:33:27.729 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.729 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.729 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:33:27.729 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.037029+0000", 2024-09-16T09:33:27.730 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00013190399999999999 2024-09-16T09:33:27.730 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.730 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.730 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,2", 2024-09-16T09:33:27.730 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.037149+0000", 2024-09-16T09:33:27.730 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000119833 2024-09-16T09:33:27.730 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.730 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.730 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:33:27.730 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.039175+0000", 2024-09-16T09:33:27.730 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0020263389999999998 2024-09-16T09:33:27.730 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.731 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.731 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:33:27.731 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.106199+0000", 2024-09-16T09:33:27.731 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.067023694999999994 2024-09-16T09:33:27.731 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.731 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.731 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:33:27.731 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.107546+0000", 2024-09-16T09:33:27.731 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.001347902 2024-09-16T09:33:27.731 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.731 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.731 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:33:27.731 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.107593+0000", 2024-09-16T09:33:27.731 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.6736000000000001e-05 2024-09-16T09:33:27.732 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.732 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.732 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:33:27.732 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:28:54.107630+0000", 2024-09-16T09:33:27.732 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.7240999999999998e-05 2024-09-16T09:33:27.732 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:33:27.732 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:33:27.732 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:33:27.732 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.732 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.732 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:215340 7.5 7:a4f699d2:::benchmark_data_smithi070_78438_object89209:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e141)", 2024-09-16T09:33:27.732 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:29:03.365151+0000", 2024-09-16T09:33:27.732 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 264.25512127299999, 2024-09-16T09:33:27.732 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.068201946999999999, 2024-09-16T09:33:27.733 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:33:27.733 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:33:27.733 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:33:27.733 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:33:27.733 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:33:27.733 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:33:27.733 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 215340 2024-09-16T09:33:27.733 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.733 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:33:27.733 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.733 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:33:27.733 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.365151+0000", 2024-09-16T09:33:27.733 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:33:27.733 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.734 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.734 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:33:27.734 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.365151+0000", 2024-09-16T09:33:27.734 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:33:27.734 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.734 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.734 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:33:27.734 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.365153+0000", 2024-09-16T09:33:27.734 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.0540000000000002e-06 2024-09-16T09:33:27.734 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.734 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.734 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:33:27.734 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.365162+0000", 2024-09-16T09:33:27.734 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.38e-06 2024-09-16T09:33:27.734 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.735 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.735 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:33:27.735 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.365186+0000", 2024-09-16T09:33:27.735 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.3986e-05 2024-09-16T09:33:27.735 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.735 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.735 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:33:27.735 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.365200+0000", 2024-09-16T09:33:27.735 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.3586e-05 2024-09-16T09:33:27.735 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.735 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.735 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:33:27.735 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.365312+0000", 2024-09-16T09:33:27.735 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000112422 2024-09-16T09:33:27.736 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.736 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.736 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op must wait for map", 2024-09-16T09:33:27.736 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.365322+0000", 2024-09-16T09:33:27.736 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.5289999999999997e-06 2024-09-16T09:33:27.736 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.736 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.736 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:33:27.736 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.422102+0000", 2024-09-16T09:33:27.736 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.056780023999999998 2024-09-16T09:33:27.736 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.736 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.736 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:33:27.736 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.422202+0000", 2024-09-16T09:33:27.737 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.9617999999999994e-05 2024-09-16T09:33:27.737 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.737 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.737 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,2", 2024-09-16T09:33:27.737 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.422578+0000", 2024-09-16T09:33:27.737 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00037663600000000002 2024-09-16T09:33:27.737 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.737 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.737 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:33:27.737 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.431303+0000", 2024-09-16T09:33:27.737 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0087248670000000007 2024-09-16T09:33:27.737 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.737 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.737 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:33:27.737 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.431404+0000", 2024-09-16T09:33:27.738 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00010138500000000001 2024-09-16T09:33:27.738 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.738 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.738 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:33:27.738 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.433215+0000", 2024-09-16T09:33:27.738 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0018102979999999999 2024-09-16T09:33:27.738 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.738 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.738 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:33:27.738 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.433306+0000", 2024-09-16T09:33:27.738 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.1774999999999995e-05 2024-09-16T09:33:27.738 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.738 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.738 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:33:27.738 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.433353+0000", 2024-09-16T09:33:27.738 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.6387000000000002e-05 2024-09-16T09:33:27.739 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:33:27.739 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:33:27.739 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:33:27.739 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.739 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.739 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:215342 7.5 7:a84e8ca2:::benchmark_data_smithi070_78438_object89211:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e141)", 2024-09-16T09:33:27.739 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:29:03.366631+0000", 2024-09-16T09:33:27.739 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 264.253640695, 2024-09-16T09:33:27.739 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.074197923999999998, 2024-09-16T09:33:27.739 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:33:27.739 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:33:27.739 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:33:27.739 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:33:27.739 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:33:27.739 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:33:27.740 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 215342 2024-09-16T09:33:27.740 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.740 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:33:27.740 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.740 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:33:27.740 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.366631+0000", 2024-09-16T09:33:27.740 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:33:27.740 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.740 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.740 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:33:27.740 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.366631+0000", 2024-09-16T09:33:27.740 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:33:27.740 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.740 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.740 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:33:27.741 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.366633+0000", 2024-09-16T09:33:27.741 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.2839999999999999e-06 2024-09-16T09:33:27.741 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.741 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.741 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:33:27.741 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.366640+0000", 2024-09-16T09:33:27.741 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.6329999999999996e-06 2024-09-16T09:33:27.741 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.741 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.741 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:33:27.741 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.420402+0000", 2024-09-16T09:33:27.741 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.053761807000000002 2024-09-16T09:33:27.741 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.741 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.742 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:33:27.742 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.420414+0000", 2024-09-16T09:33:27.742 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1885000000000001e-05 2024-09-16T09:33:27.742 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.742 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.742 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:33:27.742 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.426991+0000", 2024-09-16T09:33:27.742 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0065772399999999998 2024-09-16T09:33:27.742 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.742 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.742 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:33:27.742 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.427091+0000", 2024-09-16T09:33:27.742 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.9407999999999994e-05 2024-09-16T09:33:27.742 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.743 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.743 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,2", 2024-09-16T09:33:27.743 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.427202+0000", 2024-09-16T09:33:27.743 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000111298 2024-09-16T09:33:27.743 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.743 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.743 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:33:27.743 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.433361+0000", 2024-09-16T09:33:27.743 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0061589390000000004 2024-09-16T09:33:27.743 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.743 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.743 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:33:27.743 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.437028+0000", 2024-09-16T09:33:27.743 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0036673130000000002 2024-09-16T09:33:27.744 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.744 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.744 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:33:27.744 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.440723+0000", 2024-09-16T09:33:27.744 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0036951480000000001 2024-09-16T09:33:27.744 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.744 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.744 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:33:27.744 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.440791+0000", 2024-09-16T09:33:27.744 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.7496000000000006e-05 2024-09-16T09:33:27.744 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.744 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.744 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:33:27.745 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:03.440829+0000", 2024-09-16T09:33:27.745 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.8473000000000002e-05 2024-09-16T09:33:27.745 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:33:27.745 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:33:27.745 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:33:27.745 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.745 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.745 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.24698.0:247035 7.5 7:aba136a1:::benchmark_data_smithi070_78438_object120904:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e147)", 2024-09-16T09:33:27.745 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:29:32.642597+0000", 2024-09-16T09:33:27.745 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 234.97767558199999, 2024-09-16T09:33:27.745 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.089290376000000005, 2024-09-16T09:33:27.745 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:33:27.745 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:33:27.745 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:33:27.745 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:33:27.746 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.24698", 2024-09-16T09:33:27.746 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:33:27.746 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 247035 2024-09-16T09:33:27.746 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.746 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:33:27.746 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.746 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:33:27.746 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:32.642597+0000", 2024-09-16T09:33:27.746 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:33:27.746 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.746 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.746 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:33:27.746 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:32.642597+0000", 2024-09-16T09:33:27.746 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:33:27.746 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.746 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.747 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:33:27.747 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:32.642598+0000", 2024-09-16T09:33:27.747 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.511e-06 2024-09-16T09:33:27.747 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.747 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.747 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:33:27.747 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:32.642604+0000", 2024-09-16T09:33:27.747 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.6080000000000003e-06 2024-09-16T09:33:27.747 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.747 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.747 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:33:27.747 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:32.642616+0000", 2024-09-16T09:33:27.747 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.2532e-05 2024-09-16T09:33:27.747 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.747 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.748 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:33:27.748 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:32.642622+0000", 2024-09-16T09:33:27.748 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.8869999999999997e-06 2024-09-16T09:33:27.748 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.748 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.748 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:33:27.748 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:32.642686+0000", 2024-09-16T09:33:27.748 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.3368999999999996e-05 2024-09-16T09:33:27.748 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.748 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.748 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op must wait for map", 2024-09-16T09:33:27.748 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:32.642693+0000", 2024-09-16T09:33:27.748 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.2219999999999996e-06 2024-09-16T09:33:27.748 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.748 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.748 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:33:27.749 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:32.644822+0000", 2024-09-16T09:33:27.749 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0021290290000000002 2024-09-16T09:33:27.749 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.749 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.749 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:33:27.749 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:32.644912+0000", 2024-09-16T09:33:27.749 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.0234000000000004e-05 2024-09-16T09:33:27.749 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.749 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.749 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,2", 2024-09-16T09:33:27.749 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:32.644981+0000", 2024-09-16T09:33:27.749 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.8956999999999997e-05 2024-09-16T09:33:27.749 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.749 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.750 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:33:27.750 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:32.647401+0000", 2024-09-16T09:33:27.750 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0024197620000000002 2024-09-16T09:33:27.750 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.750 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.750 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:33:27.750 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:32.723230+0000", 2024-09-16T09:33:27.750 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.075829222000000002 2024-09-16T09:33:27.750 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.750 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.750 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:33:27.750 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:32.731807+0000", 2024-09-16T09:33:27.750 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.008576719 2024-09-16T09:33:27.750 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.751 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.751 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:33:27.751 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:32.731856+0000", 2024-09-16T09:33:27.751 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.9051999999999997e-05 2024-09-16T09:33:27.751 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:33:27.751 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:33:27.751 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:33:27.751 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:29:32.731887+0000", 2024-09-16T09:33:27.751 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.1272000000000001e-05 2024-09-16T09:33:27.751 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:33:27.751 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:33:27.751 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:33:27.751 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:33:27.751 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:33:27.752 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-09-16T09:33:27.752 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:27.752 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:33:27.753 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-16T09:33:27.853 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:33:27.875 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:27.875 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:27.876 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:33:27.977 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:33:28.000 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:28.001 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:28.001 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:33:28.044 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:33:28.045 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:33:28.102 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:33:28.124 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:28.124 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:28.125 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:33:28.226 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:33:28.241 INFO:teuthology.orchestra.run.smithi070.stderr:enabled application 'rados' on pool 'unique_pool_2' 2024-09-16T09:33:28.250 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:28.250 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:28.250 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:33:28.351 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:33:28.374 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:28.374 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:28.374 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:33:28.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:28 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1157793202' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "unique_pool_2", "app": "rados", "yes_i_really_mean_it": true} : dispatch 2024-09-16T09:33:28.476 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:33:28.497 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:28.498 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:28.499 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:33:28.576 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:28 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1157793202' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "unique_pool_2", "app": "rados", "yes_i_really_mean_it": true} : dispatch 2024-09-16T09:33:28.576 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:28 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1157793202' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "unique_pool_2", "app": "rados", "yes_i_really_mean_it": true} : dispatch 2024-09-16T09:33:28.600 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:33:28.615 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.0 dump_ops_in_flight 2024-09-16T09:33:28.621 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:28.622 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:28.622 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:33:28.723 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:33:28.745 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:28.745 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:28.745 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:33:28.846 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:33:28.868 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:28.869 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:28.869 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:33:28.970 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:33:28.993 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:28.993 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:28.994 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:33:29.094 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:33:29.116 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:29.117 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:29.117 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:33:29.218 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:33:29.234 INFO:teuthology.orchestra.run.smithi070.stderr:set osd.1 primary-affinity to 0 (802) 2024-09-16T09:33:29.242 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:29.244 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:29.244 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:33:29.345 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:33:29.367 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:29.368 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:29.369 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:33:29.470 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:33:29.491 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:29.491 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:29.492 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:33:29.593 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:33:29.615 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:29.615 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:29.616 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:33:29.625 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:29 smithi070 ceph-mon[30388]: pgmap v399: 168 pgs: 13 unknown, 155 active+clean; 583 KiB data, 2.7 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-16T09:33:29.625 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:29 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1157793202' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_2", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-16T09:33:29.625 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:29 smithi070 ceph-mon[30388]: osdmap e232: 8 total, 8 up, 8 in 2024-09-16T09:33:29.625 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:29 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1327013648' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 1, "weight": 0.0} : dispatch 2024-09-16T09:33:29.626 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:29 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1327013648' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 1, "weight": 0.0}]': finished 2024-09-16T09:33:29.626 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:29 smithi070 ceph-mon[30388]: osdmap e233: 8 total, 8 up, 8 in 2024-09-16T09:33:29.626 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:29 smithi070 ceph-mon[40691]: pgmap v399: 168 pgs: 13 unknown, 155 active+clean; 583 KiB data, 2.7 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-16T09:33:29.626 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:29 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1157793202' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_2", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-16T09:33:29.626 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:29 smithi070 ceph-mon[40691]: osdmap e232: 8 total, 8 up, 8 in 2024-09-16T09:33:29.626 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:29 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1327013648' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 1, "weight": 0.0} : dispatch 2024-09-16T09:33:29.626 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:29 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1327013648' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 1, "weight": 0.0}]': finished 2024-09-16T09:33:29.627 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:29 smithi070 ceph-mon[40691]: osdmap e233: 8 total, 8 up, 8 in 2024-09-16T09:33:29.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:29 smithi084 ceph-mon[32629]: pgmap v399: 168 pgs: 13 unknown, 155 active+clean; 583 KiB data, 2.7 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-16T09:33:29.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:29 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1157793202' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_2", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-16T09:33:29.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:29 smithi084 ceph-mon[32629]: osdmap e232: 8 total, 8 up, 8 in 2024-09-16T09:33:29.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:29 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1327013648' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 1, "weight": 0.0} : dispatch 2024-09-16T09:33:29.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:29 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1327013648' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 1, "weight": 0.0}]': finished 2024-09-16T09:33:29.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:29 smithi084 ceph-mon[32629]: osdmap e233: 8 total, 8 up, 8 in 2024-09-16T09:33:29.717 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:33:29.739 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:29.739 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:29.739 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:33:29.840 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:33:29.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:29.862 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:29.862 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:33:29.963 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:33:29.986 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:29.987 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:29.987 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:33:30.006 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T09:33:30.088 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:33:30.113 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:30.114 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:30.114 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:33:30.215 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:33:30.237 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:30.238 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:30.238 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:33:30.243 INFO:teuthology.orchestra.run.smithi070.stderr:nodeep-scrub is set 2024-09-16T09:33:30.340 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:33:30.362 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:30.363 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:30.363 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:33:30.464 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:33:30.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:30.488 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:30.489 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:33:30.590 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:33:30.611 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:30.612 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:30.612 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:33:30.653 DEBUG:teuthology.orchestra.run.smithi070:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.0 -b 65536 --object-size 65536 -p unique_pool_2 bench 90 write' 2024-09-16T09:33:30.655 INFO:tasks.radosbench:joining radosbench (timing out after 3000s) 2024-09-16T09:33:30.655 INFO:teuthology.orchestra.run:waiting for 3000 2024-09-16T09:33:30.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:30 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:33:30.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:30 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:33:30.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:30 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/2512614725' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:33:30.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:30 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:33:30.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:30 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:33:30.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:30 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:33:30.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:30 smithi084 ceph-mon[32629]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:33:30.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:30 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-16T09:33:30.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:30 smithi084 ceph-mon[32629]: osdmap e234: 8 total, 8 up, 8 in 2024-09-16T09:33:30.670 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:30 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:33:30.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:30 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:33:30.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:30 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/2512614725' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:33:30.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:30 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:33:30.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:30 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:33:30.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:30 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:33:30.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:30 smithi070 ceph-mon[40691]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:33:30.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:30 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-16T09:33:30.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:30 smithi070 ceph-mon[40691]: osdmap e234: 8 total, 8 up, 8 in 2024-09-16T09:33:30.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:30 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:33:30.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:30 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:33:30.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:30 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2512614725' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:33:30.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:30 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:33:30.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:30 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:33:30.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:30 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:33:30.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:30 smithi070 ceph-mon[30388]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:33:30.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:30 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-16T09:33:30.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:30 smithi070 ceph-mon[30388]: osdmap e234: 8 total, 8 up, 8 in 2024-09-16T09:33:30.705 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:hints = 1 2024-09-16T09:33:30.705 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Maintaining 16 concurrent writes of 65536 bytes to objects of size 65536 for up to 90 seconds or 0 objects 2024-09-16T09:33:30.705 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Object prefix: benchmark_data_smithi070_137000 2024-09-16T09:33:30.705 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-16T09:33:30.705 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 0 0 0 0 0 0 - 0 2024-09-16T09:33:30.712 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:33:30.757 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:30.758 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:30.758 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:33:30.859 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:33:30.896 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:30.897 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:30.897 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:33:30.998 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:33:31.033 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:31.034 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:31.034 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:33:31.135 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:33:31.167 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:31.168 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:31.168 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:33:31.212 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:33:31.269 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:33:31.276 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:33:31.305 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:31.306 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:31.306 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:33:31.407 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:33:31.412 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:31 smithi070 ceph-mon[30388]: pgmap v402: 168 pgs: 168 active+clean; 583 KiB data, 2.7 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-16T09:33:31.412 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:31 smithi070 ceph-mon[30388]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-16T09:33:31.412 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:31 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:33:31.412 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:31 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:33:31.412 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:31 smithi070 ceph-mon[30388]: 1.3 scrub starts 2024-09-16T09:33:31.413 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:31 smithi070 ceph-mon[40691]: pgmap v402: 168 pgs: 168 active+clean; 583 KiB data, 2.7 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-16T09:33:31.413 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:31 smithi070 ceph-mon[40691]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-16T09:33:31.413 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:31 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:33:31.413 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:31 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:33:31.413 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:31 smithi070 ceph-mon[40691]: 1.3 scrub starts 2024-09-16T09:33:31.446 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:31.446 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:31.447 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:33:31.547 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:33:31.581 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:31.582 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:31.582 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:33:31.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:31 smithi084 ceph-mon[32629]: pgmap v402: 168 pgs: 168 active+clean; 583 KiB data, 2.7 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-16T09:33:31.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:31 smithi084 ceph-mon[32629]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-16T09:33:31.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:31 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:33:31.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:31 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:33:31.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:31 smithi084 ceph-mon[32629]: 1.3 scrub starts 2024-09-16T09:33:31.683 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:33:31.705 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 1 16 3801 3785 236.549 236.562 0.0044149 0.00421217 2024-09-16T09:33:31.719 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:31.720 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:31.720 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:33:31.821 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:33:31.843 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:31.938 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:31.939 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:33:32.039 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:33:32.075 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:32.075 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:32.075 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:33:32.176 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:33:32.210 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:32.210 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:32.211 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:33:32.268 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:33:31 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:33:31] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:33:32.311 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:33:32.350 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T09:33:32.351 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [ 2024-09-16T09:33:32.351 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:33:32.351 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:5914 9.9 9:96d465a7:::benchmark_data_smithi070_137000_object5913:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e235)", 2024-09-16T09:33:32.351 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:33:32.336541+0000", 2024-09-16T09:33:32.351 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.0081650820000000006, 2024-09-16T09:33:32.351 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0081949589999999999, 2024-09-16T09:33:32.352 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:33:32.352 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:33:32.352 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "waiting for sub ops", 2024-09-16T09:33:32.352 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:33:32.352 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:33:32.352 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:33:32.353 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 5914 2024-09-16T09:33:32.353 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:33:32.353 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:33:32.353 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:33:32.353 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:33:32.353 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:32.336541+0000", 2024-09-16T09:33:32.354 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:33:32.354 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:33:32.354 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:33:32.354 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:33:32.354 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:32.336541+0000", 2024-09-16T09:33:32.355 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:33:32.355 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:33:32.355 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:33:32.355 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:33:32.355 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:32.336542+0000", 2024-09-16T09:33:32.355 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.540000000000001e-07 2024-09-16T09:33:32.355 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:33:32.355 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:33:32.355 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:33:32.355 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:32.336574+0000", 2024-09-16T09:33:32.355 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2104999999999998e-05 2024-09-16T09:33:32.356 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:33:32.356 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:33:32.356 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:33:32.356 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:32.336585+0000", 2024-09-16T09:33:32.356 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0876999999999999e-05 2024-09-16T09:33:32.356 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:33:32.356 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:33:32.356 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:33:32.356 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:32.336592+0000", 2024-09-16T09:33:32.356 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.7209999999999997e-06 2024-09-16T09:33:32.356 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:33:32.356 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:33:32.357 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:33:32.357 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:32.337752+0000", 2024-09-16T09:33:32.357 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001159626 2024-09-16T09:33:32.357 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:33:32.357 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:33:32.357 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:33:32.357 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:32.337853+0000", 2024-09-16T09:33:32.357 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000101026 2024-09-16T09:33:32.357 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:33:32.357 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:33:32.357 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 5,7", 2024-09-16T09:33:32.357 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:32.337964+0000", 2024-09-16T09:33:32.357 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000110979 2024-09-16T09:33:32.358 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:33:32.358 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:33:32.358 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:33:32.358 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:32.339077+0000", 2024-09-16T09:33:32.358 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001113597 2024-09-16T09:33:32.358 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:33:32.358 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:33:32.358 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:33:32.358 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:32.341190+0000", 2024-09-16T09:33:32.358 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.002113116 2024-09-16T09:33:32.358 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:33:32.358 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:33:32.358 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:33:32.358 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:33:32.359 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:33:32.359 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:5922 9.5 9:ac373495:::benchmark_data_smithi070_137000_object5921:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e235)", 2024-09-16T09:33:32.359 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:33:32.338971+0000", 2024-09-16T09:33:32.359 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.0057352610000000002, 2024-09-16T09:33:32.359 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0058481430000000001, 2024-09-16T09:33:32.359 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:33:32.359 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:33:32.359 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "waiting for sub ops", 2024-09-16T09:33:32.359 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:33:32.359 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:33:32.359 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:33:32.359 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 5922 2024-09-16T09:33:32.359 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:33:32.360 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:33:32.360 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:33:32.360 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:33:32.360 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:32.338971+0000", 2024-09-16T09:33:32.360 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:33:32.360 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:33:32.360 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:33:32.360 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:33:32.360 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:32.338971+0000", 2024-09-16T09:33:32.360 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:33:32.360 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:33:32.360 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:33:32.361 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:33:32.361 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:32.338972+0000", 2024-09-16T09:33:32.361 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.1699999999999997e-07 2024-09-16T09:33:32.361 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:33:32.361 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:33:32.361 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:33:32.361 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:32.338993+0000", 2024-09-16T09:33:32.361 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.1141e-05 2024-09-16T09:33:32.361 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:33:32.361 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:33:32.362 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:33:32.362 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:32.339002+0000", 2024-09-16T09:33:32.362 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.7220000000000005e-06 2024-09-16T09:33:32.362 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:33:32.362 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:33:32.362 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:33:32.362 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:32.339008+0000", 2024-09-16T09:33:32.362 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1040000000000003e-06 2024-09-16T09:33:32.362 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:33:32.362 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:33:32.362 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:33:32.362 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:32.339131+0000", 2024-09-16T09:33:32.362 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00012349299999999999 2024-09-16T09:33:32.362 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:33:32.363 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:33:32.363 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:33:32.363 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:32.339241+0000", 2024-09-16T09:33:32.363 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000109113 2024-09-16T09:33:32.363 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:33:32.363 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:33:32.363 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 5,7", 2024-09-16T09:33:32.363 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:32.339384+0000", 2024-09-16T09:33:32.363 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00014304699999999999 2024-09-16T09:33:32.363 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:33:32.363 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:33:32.363 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:33:32.363 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:32.340171+0000", 2024-09-16T09:33:32.364 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00078739999999999995 2024-09-16T09:33:32.364 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:33:32.364 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:33:32.364 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:33:32.364 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:33:32.364 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:33:32.364 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:5937 9.9 9:92d5ef5b:::benchmark_data_smithi070_137000_object5936:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e235)", 2024-09-16T09:33:32.364 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:33:32.344122+0000", 2024-09-16T09:33:32.364 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.00058422599999999997, 2024-09-16T09:33:32.364 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00076326599999999997, 2024-09-16T09:33:32.365 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:33:32.365 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:33:32.365 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "waiting for sub ops", 2024-09-16T09:33:32.365 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:33:32.365 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:33:32.365 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:33:32.365 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 5937 2024-09-16T09:33:32.365 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:33:32.365 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:33:32.365 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:33:32.365 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:33:32.365 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:32.344122+0000", 2024-09-16T09:33:32.366 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:33:32.366 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:33:32.366 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:33:32.366 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:33:32.366 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:32.344122+0000", 2024-09-16T09:33:32.366 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:33:32.366 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:33:32.366 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:33:32.366 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:33:32.366 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:32.344124+0000", 2024-09-16T09:33:32.366 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.392e-06 2024-09-16T09:33:32.366 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:33:32.366 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:33:32.366 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:33:32.367 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:32.344142+0000", 2024-09-16T09:33:32.367 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.8763e-05 2024-09-16T09:33:32.367 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:33:32.367 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:33:32.367 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:33:32.367 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:32.344156+0000", 2024-09-16T09:33:32.367 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3215e-05 2024-09-16T09:33:32.367 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:33:32.367 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:33:32.367 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:33:32.367 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:32.344161+0000", 2024-09-16T09:33:32.367 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.339e-06 2024-09-16T09:33:32.367 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:33:32.367 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:33:32.368 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:33:32.368 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:32.344208+0000", 2024-09-16T09:33:32.368 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.7289000000000001e-05 2024-09-16T09:33:32.368 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:33:32.368 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:33:32.368 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:33:32.368 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:32.344347+0000", 2024-09-16T09:33:32.368 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000138632 2024-09-16T09:33:32.368 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:33:32.368 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:33:32.368 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 5,7", 2024-09-16T09:33:32.369 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:32.344492+0000", 2024-09-16T09:33:32.369 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000145236 2024-09-16T09:33:32.369 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:33:32.369 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:33:32.369 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:33:32.369 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:33:32.370 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:33:32.370 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:5938 9.b 9:deae5969:::benchmark_data_smithi070_137000_object5937:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e235)", 2024-09-16T09:33:32.370 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:33:32.344719+0000", 2024-09-16T09:33:32.370 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 4294967295.9999871, 2024-09-16T09:33:32.370 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000226221, 2024-09-16T09:33:32.370 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:33:32.370 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:33:32.371 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T09:33:32.371 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:33:32.371 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:33:32.371 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:33:32.371 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 5938 2024-09-16T09:33:32.371 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:33:32.371 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:33:32.371 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:33:32.371 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:33:32.372 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:32.344719+0000", 2024-09-16T09:33:32.372 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:33:32.372 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:33:32.372 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:33:32.372 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:33:32.372 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:32.344719+0000", 2024-09-16T09:33:32.373 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:33:32.373 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:33:32.373 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:33:32.373 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:33:32.373 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:32.344720+0000", 2024-09-16T09:33:32.374 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.0500000000000002e-07 2024-09-16T09:33:32.374 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:33:32.374 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:33:32.374 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:33:32.374 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:32.344750+0000", 2024-09-16T09:33:32.374 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.0088000000000001e-05 2024-09-16T09:33:32.375 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:33:32.375 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:33:32.375 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:33:32.375 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:32.344760+0000", 2024-09-16T09:33:32.375 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.5000000000000005e-06 2024-09-16T09:33:32.375 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:33:32.375 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:33:32.375 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:33:32.375 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:32.344765+0000", 2024-09-16T09:33:32.375 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4219999999999999e-06 2024-09-16T09:33:32.375 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:33:32.376 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:33:32.376 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:33:32.376 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:32.344815+0000", 2024-09-16T09:33:32.376 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.9341000000000001e-05 2024-09-16T09:33:32.376 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:33:32.376 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:33:32.376 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:33:32.376 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:32.344902+0000", 2024-09-16T09:33:32.376 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.6986999999999999e-05 2024-09-16T09:33:32.376 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:33:32.376 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:33:32.376 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:33:32.376 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:33:32.376 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-16T09:33:32.377 INFO:teuthology.orchestra.run.smithi070.stdout: "num_ops": 4 2024-09-16T09:33:32.377 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T09:33:32.378 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:32.378 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:32.378 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:33:32.479 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:33:32.518 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:32.518 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:32.519 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:33:32.620 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:33:32.657 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:32.658 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:32.659 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:33:32.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:32 smithi084 ceph-mon[32629]: 1.b scrub starts 2024-09-16T09:33:32.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:32 smithi084 ceph-mon[32629]: osdmap e235: 8 total, 8 up, 8 in 2024-09-16T09:33:32.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:32 smithi070 ceph-mon[30388]: 1.b scrub starts 2024-09-16T09:33:32.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:32 smithi070 ceph-mon[30388]: osdmap e235: 8 total, 8 up, 8 in 2024-09-16T09:33:32.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:32 smithi070 ceph-mon[40691]: 1.b scrub starts 2024-09-16T09:33:32.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:32 smithi070 ceph-mon[40691]: osdmap e235: 8 total, 8 up, 8 in 2024-09-16T09:33:32.705 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 2 16 7187 7171 224.078 211.625 0.00393513 0.00445005 2024-09-16T09:33:32.759 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:33:32.800 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:32.800 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:32.801 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:33:32.881 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:33:32.882 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:33:32.902 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:33:32.914 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:33:32.914 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:33:32.937 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:32.938 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:32.938 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:33:32.944 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:33:32.944 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:33:32.977 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:33:32.977 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:33:33.012 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:33:33.012 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:33:33.039 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:33:33.049 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:33:33.050 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:33:33.075 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:33.076 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:33.076 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:33:33.081 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:33:33.081 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:33:33.114 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:33:33.114 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:33:33.167 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd unset noscrub 2024-09-16T09:33:33.177 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:33:33.211 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:33.212 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:33.212 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:33:33.281 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:33:33 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:33:33.313 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:33:33.351 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:33.351 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:33.352 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:33:33.453 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:33:33.463 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:33 smithi070 ceph-mon[30388]: pgmap v405: 168 pgs: 168 active+clean; 583 KiB data, 2.7 GiB used, 713 GiB / 715 GiB avail 2024-09-16T09:33:33.463 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:33 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:33:33.463 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:33 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:33:33.463 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:33 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2541673331' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:33:33.463 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:33 smithi070 ceph-mon[40691]: pgmap v405: 168 pgs: 168 active+clean; 583 KiB data, 2.7 GiB used, 713 GiB / 715 GiB avail 2024-09-16T09:33:33.463 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:33 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:33:33.463 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:33 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:33:33.463 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:33 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/2541673331' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:33:33.496 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.0 dump_blocked_ops 2024-09-16T09:33:33.511 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:33.511 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:33.512 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:33:33.613 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:33:33.646 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:33.647 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:33.647 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:33:33.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:33 smithi084 ceph-mon[32629]: pgmap v405: 168 pgs: 168 active+clean; 583 KiB data, 2.7 GiB used, 713 GiB / 715 GiB avail 2024-09-16T09:33:33.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:33 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:33:33.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:33 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:33:33.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:33 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/2541673331' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:33:33.705 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 3 16 10377 10361 215.829 199.375 0.00530792 0.00462262 2024-09-16T09:33:33.749 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:33:33.778 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:33.779 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:33.779 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:33:33.880 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:33:33.917 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:33.918 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:33.918 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:33:34.019 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:33:34.054 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:34.055 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:34.055 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:33:34.156 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:33:34.192 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:34.194 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:34.195 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:33:34.296 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:33:34.334 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:34.334 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:34.335 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:33:34.436 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:33:34.470 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:34.470 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:34.471 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:33:34.572 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:33:34.608 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:34.608 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:34.609 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:33:34.639 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:34 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:33:34.641 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:34 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:33:34.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:34 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:33:34.705 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 4 15 13348 13333 208.29 185.75 0.00379196 0.00479453 2024-09-16T09:33:34.710 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:33:34.736 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:33:34.740 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:34.741 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:34.741 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:33:34.842 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:33:34.876 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:34.876 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:34.877 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:33:34.978 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:33:35.012 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:35.012 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:35.013 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:33:35.114 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:33:35.152 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:35.152 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:35.153 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:33:35.254 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:33:35.292 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:35.292 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:35.293 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:33:35.303 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:33:35.393 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:33:35.426 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T09:33:35.426 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T09:33:35.427 INFO:tasks.thrashosds.thrasher:Growing pool unique_pool_2 2024-09-16T09:33:35.427 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-16T09:33:35.432 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:35.433 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:35.433 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:33:35.440 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:35 smithi070 ceph-mon[30388]: pgmap v406: 168 pgs: 168 active+clean; 498 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.7 KiB/s rd, 92 MiB/s wr, 1.47k op/s; 3.2 KiB/s, 0 objects/s recovering 2024-09-16T09:33:35.443 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:35 smithi070 ceph-mon[40691]: pgmap v406: 168 pgs: 168 active+clean; 498 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.7 KiB/s rd, 92 MiB/s wr, 1.47k op/s; 3.2 KiB/s, 0 objects/s recovering 2024-09-16T09:33:35.534 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:33:35.564 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:35.565 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:35.565 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:33:35.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:35 smithi084 ceph-mon[32629]: pgmap v406: 168 pgs: 168 active+clean; 498 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.7 KiB/s rd, 92 MiB/s wr, 1.47k op/s; 3.2 KiB/s, 0 objects/s recovering 2024-09-16T09:33:35.666 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:33:35.706 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:35.707 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 5 16 16137 16121 201.475 174.25 0.00640144 0.00495766 2024-09-16T09:33:35.707 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:35.708 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:33:35.809 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:33:35.847 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:35.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:35.848 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:33:35.949 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:33:35.976 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T09:33:35.976 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:33:35.980 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:35.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:35.981 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:33:36.082 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:33:36.117 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:36.118 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:36.118 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:33:36.219 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:33:36.256 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:36.257 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:36.257 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:33:36.307 INFO:teuthology.orchestra.run.smithi070.stderr:noscrub is unset 2024-09-16T09:33:36.357 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:33:36.394 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:36.395 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:36.395 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:33:36.496 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:33:36.533 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:36.534 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:36.534 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:33:36.635 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:33:36.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:36 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1026447917' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:33:36.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:36 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:33:36.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:36 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1026447917' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:33:36.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:36 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:33:36.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:36 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1026447917' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:33:36.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:36 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:33:36.671 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:36.672 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:36.673 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:33:36.702 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T09:33:36.706 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 6 15 19005 18990 197.779 179.312 0.00350737 0.00505087 2024-09-16T09:33:36.774 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:33:36.806 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:36.807 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:36.807 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:33:36.818 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.0 dump_historic_ops 2024-09-16T09:33:36.908 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:33:36.940 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:36.941 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:36.941 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:33:37.042 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:33:37.088 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:37.089 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:37.089 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:33:37.189 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:33:37.232 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:37.232 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:37.232 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:33:37.333 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:33:37.368 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:37.369 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:37.369 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:33:37.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:37 smithi070 ceph-mon[30388]: pgmap v407: 168 pgs: 168 active+clean; 498 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.4 KiB/s rd, 77 MiB/s wr, 1.24k op/s; 2.7 KiB/s, 0 objects/s recovering 2024-09-16T09:33:37.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:37 smithi070 ceph-mon[30388]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:33:37.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:37 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-16T09:33:37.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:37 smithi070 ceph-mon[30388]: osdmap e236: 8 total, 8 up, 8 in 2024-09-16T09:33:37.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:37 smithi070 ceph-mon[40691]: pgmap v407: 168 pgs: 168 active+clean; 498 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.4 KiB/s rd, 77 MiB/s wr, 1.24k op/s; 2.7 KiB/s, 0 objects/s recovering 2024-09-16T09:33:37.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:37 smithi070 ceph-mon[40691]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:33:37.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:37 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-16T09:33:37.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:37 smithi070 ceph-mon[40691]: osdmap e236: 8 total, 8 up, 8 in 2024-09-16T09:33:37.470 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:33:37.505 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:37.506 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:37.506 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:33:37.607 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:33:37.643 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:37.644 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:37.644 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:33:37.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:37 smithi084 ceph-mon[32629]: pgmap v407: 168 pgs: 168 active+clean; 498 MiB data, 3.8 GiB used, 711 GiB / 715 GiB avail; 1.4 KiB/s rd, 77 MiB/s wr, 1.24k op/s; 2.7 KiB/s, 0 objects/s recovering 2024-09-16T09:33:37.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:37 smithi084 ceph-mon[32629]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:33:37.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:37 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-16T09:33:37.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:37 smithi084 ceph-mon[32629]: osdmap e236: 8 total, 8 up, 8 in 2024-09-16T09:33:37.706 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 7 16 21695 21679 193.532 168.062 0.00694436 0.00516116 2024-09-16T09:33:37.745 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:33:37.771 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:37.772 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:37.772 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:33:37.873 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:33:37.909 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:37.910 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:37.910 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:33:38.011 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:33:38.044 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:38.045 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:38.045 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:33:38.146 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:33:38.167 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:38.168 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:38.168 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:33:38.269 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:33:38.303 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:38.304 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:38.304 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:33:38.405 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:33:38.440 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:38.440 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:38.441 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:33:38.542 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:33:38.582 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:38.582 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:38.582 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:33:38.634 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:33:38.634 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:33:38.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:38 smithi084 ceph-mon[32629]: 3.1d scrub starts 2024-09-16T09:33:38.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:38 smithi084 ceph-mon[32629]: 3.1d scrub ok 2024-09-16T09:33:38.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:38 smithi084 ceph-mon[32629]: 2.0 scrub starts 2024-09-16T09:33:38.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:38 smithi084 ceph-mon[32629]: 2.0 scrub ok 2024-09-16T09:33:38.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:38 smithi084 ceph-mon[32629]: 5.11 scrub starts 2024-09-16T09:33:38.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:38 smithi084 ceph-mon[32629]: 5.11 scrub ok 2024-09-16T09:33:38.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:38 smithi070 ceph-mon[30388]: 3.1d scrub starts 2024-09-16T09:33:38.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:38 smithi070 ceph-mon[30388]: 3.1d scrub ok 2024-09-16T09:33:38.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:38 smithi070 ceph-mon[30388]: 2.0 scrub starts 2024-09-16T09:33:38.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:38 smithi070 ceph-mon[30388]: 2.0 scrub ok 2024-09-16T09:33:38.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:38 smithi070 ceph-mon[30388]: 5.11 scrub starts 2024-09-16T09:33:38.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:38 smithi070 ceph-mon[30388]: 5.11 scrub ok 2024-09-16T09:33:38.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:38 smithi070 ceph-mon[40691]: 3.1d scrub starts 2024-09-16T09:33:38.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:38 smithi070 ceph-mon[40691]: 3.1d scrub ok 2024-09-16T09:33:38.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:38 smithi070 ceph-mon[40691]: 2.0 scrub starts 2024-09-16T09:33:38.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:38 smithi070 ceph-mon[40691]: 2.0 scrub ok 2024-09-16T09:33:38.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:38 smithi070 ceph-mon[40691]: 5.11 scrub starts 2024-09-16T09:33:38.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:38 smithi070 ceph-mon[40691]: 5.11 scrub ok 2024-09-16T09:33:38.684 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:33:38.706 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 8 16 23837 23821 186.074 133.875 0.0102943 0.0053694 2024-09-16T09:33:38.716 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:38.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:38.717 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:33:38.818 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:33:38.850 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:38.851 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:38.852 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:33:38.953 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:33:38.987 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:38.988 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:38.988 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:33:39.089 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:33:39.126 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:39.127 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:39.128 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:33:39.229 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:33:39.282 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:39.282 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:39.283 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:33:39.383 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:33:39.417 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:39.417 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:39.418 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:33:39.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:39 smithi070 ceph-mon[30388]: pgmap v409: 168 pgs: 168 active+clean; 780 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 1.9 KiB/s rd, 105 MiB/s wr, 1.68k op/s; 2.3 KiB/s, 0 objects/s recovering 2024-09-16T09:33:39.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:39 smithi070 ceph-mon[30388]: 5.1c scrub starts 2024-09-16T09:33:39.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:39 smithi070 ceph-mon[30388]: 5.1c scrub ok 2024-09-16T09:33:39.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:39 smithi070 ceph-mon[30388]: 5.10 scrub starts 2024-09-16T09:33:39.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:39 smithi070 ceph-mon[30388]: 5.10 scrub ok 2024-09-16T09:33:39.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:39 smithi070 ceph-mon[30388]: 2.2 scrub starts 2024-09-16T09:33:39.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:39 smithi070 ceph-mon[30388]: 2.2 scrub ok 2024-09-16T09:33:39.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:39 smithi070 ceph-mon[30388]: osdmap e237: 8 total, 8 up, 8 in 2024-09-16T09:33:39.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:39 smithi070 ceph-mon[40691]: pgmap v409: 168 pgs: 168 active+clean; 780 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 1.9 KiB/s rd, 105 MiB/s wr, 1.68k op/s; 2.3 KiB/s, 0 objects/s recovering 2024-09-16T09:33:39.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:39 smithi070 ceph-mon[40691]: 5.1c scrub starts 2024-09-16T09:33:39.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:39 smithi070 ceph-mon[40691]: 5.1c scrub ok 2024-09-16T09:33:39.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:39 smithi070 ceph-mon[40691]: 5.10 scrub starts 2024-09-16T09:33:39.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:39 smithi070 ceph-mon[40691]: 5.10 scrub ok 2024-09-16T09:33:39.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:39 smithi070 ceph-mon[40691]: 2.2 scrub starts 2024-09-16T09:33:39.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:39 smithi070 ceph-mon[40691]: 2.2 scrub ok 2024-09-16T09:33:39.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:39 smithi070 ceph-mon[40691]: osdmap e237: 8 total, 8 up, 8 in 2024-09-16T09:33:39.518 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:33:39.554 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:39.555 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:39.555 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:33:39.586 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:33:39.656 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:33:39.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:39 smithi084 ceph-mon[32629]: pgmap v409: 168 pgs: 168 active+clean; 780 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 1.9 KiB/s rd, 105 MiB/s wr, 1.68k op/s; 2.3 KiB/s, 0 objects/s recovering 2024-09-16T09:33:39.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:39 smithi084 ceph-mon[32629]: 5.1c scrub starts 2024-09-16T09:33:39.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:39 smithi084 ceph-mon[32629]: 5.1c scrub ok 2024-09-16T09:33:39.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:39 smithi084 ceph-mon[32629]: 5.10 scrub starts 2024-09-16T09:33:39.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:39 smithi084 ceph-mon[32629]: 5.10 scrub ok 2024-09-16T09:33:39.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:39 smithi084 ceph-mon[32629]: 2.2 scrub starts 2024-09-16T09:33:39.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:39 smithi084 ceph-mon[32629]: 2.2 scrub ok 2024-09-16T09:33:39.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:39 smithi084 ceph-mon[32629]: osdmap e237: 8 total, 8 up, 8 in 2024-09-16T09:33:39.693 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:39.694 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:39.694 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:33:39.706 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 9 16 26355 26339 182.884 157.375 0.00814797 0.00546263 2024-09-16T09:33:39.795 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:33:39.828 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:39.829 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:39.829 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:33:39.930 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:33:39.964 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:39.965 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:39.965 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:33:40.066 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:33:40.102 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:40.102 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:40.103 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:33:40.203 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:33:40.228 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:40.228 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:40.229 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:33:40.329 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:33:40.366 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:40.367 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:40.367 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:33:40.464 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T09:33:40.465 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:33:40.468 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:33:40.502 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:40.503 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:40.503 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:33:40.528 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-16T09:33:40.605 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:33:40.645 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:40.646 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:40.646 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:33:40.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:40 smithi084 ceph-mon[32629]: 6.18 scrub starts 2024-09-16T09:33:40.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:40 smithi084 ceph-mon[32629]: 6.18 scrub ok 2024-09-16T09:33:40.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:40 smithi084 ceph-mon[32629]: 4.1d scrub starts 2024-09-16T09:33:40.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:40 smithi084 ceph-mon[32629]: 4.1d scrub ok 2024-09-16T09:33:40.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:40 smithi084 ceph-mon[32629]: 6.5 scrub starts 2024-09-16T09:33:40.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:40 smithi084 ceph-mon[32629]: 6.5 scrub ok 2024-09-16T09:33:40.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:40 smithi084 ceph-mon[32629]: 5.15 scrub starts 2024-09-16T09:33:40.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:40 smithi084 ceph-mon[32629]: 5.15 scrub ok 2024-09-16T09:33:40.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:40 smithi070 ceph-mon[40691]: 6.18 scrub starts 2024-09-16T09:33:40.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:40 smithi070 ceph-mon[40691]: 6.18 scrub ok 2024-09-16T09:33:40.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:40 smithi070 ceph-mon[40691]: 4.1d scrub starts 2024-09-16T09:33:40.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:40 smithi070 ceph-mon[40691]: 4.1d scrub ok 2024-09-16T09:33:40.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:40 smithi070 ceph-mon[40691]: 6.5 scrub starts 2024-09-16T09:33:40.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:40 smithi070 ceph-mon[40691]: 6.5 scrub ok 2024-09-16T09:33:40.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:40 smithi070 ceph-mon[40691]: 5.15 scrub starts 2024-09-16T09:33:40.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:40 smithi070 ceph-mon[40691]: 5.15 scrub ok 2024-09-16T09:33:40.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:40 smithi070 ceph-mon[30388]: 6.18 scrub starts 2024-09-16T09:33:40.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:40 smithi070 ceph-mon[30388]: 6.18 scrub ok 2024-09-16T09:33:40.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:40 smithi070 ceph-mon[30388]: 4.1d scrub starts 2024-09-16T09:33:40.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:40 smithi070 ceph-mon[30388]: 4.1d scrub ok 2024-09-16T09:33:40.673 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:40 smithi070 ceph-mon[30388]: 6.5 scrub starts 2024-09-16T09:33:40.673 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:40 smithi070 ceph-mon[30388]: 6.5 scrub ok 2024-09-16T09:33:40.673 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:40 smithi070 ceph-mon[30388]: 5.15 scrub starts 2024-09-16T09:33:40.673 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:40 smithi070 ceph-mon[30388]: 5.15 scrub ok 2024-09-16T09:33:40.706 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 10 16 28533 28517 178.207 136.125 0.0113927 0.0056069 2024-09-16T09:33:40.748 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:33:40.779 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:40.779 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:40.779 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:33:40.880 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:33:40.916 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:40.917 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:40.917 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:33:41.018 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:33:41.051 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:41.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:41.052 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:33:41.153 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:33:41.192 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:41.192 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:41.193 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:33:41.293 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:33:41.328 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:41.329 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:41.329 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:33:41.430 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:33:41.465 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:41.466 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:41.466 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:33:41.567 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:33:41.604 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:41.605 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:41.605 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:33:41.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:41 smithi084 ceph-mon[32629]: pgmap v411: 168 pgs: 168 active+clean; 1.4 GiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 1.7 KiB/s rd, 173 MiB/s wr, 2.77k op/s; 2.2 KiB/s, 0 objects/s recovering 2024-09-16T09:33:41.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:41 smithi084 ceph-mon[32629]: 5.a scrub starts 2024-09-16T09:33:41.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:41 smithi084 ceph-mon[32629]: 5.a scrub ok 2024-09-16T09:33:41.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:41 smithi070 ceph-mon[30388]: pgmap v411: 168 pgs: 168 active+clean; 1.4 GiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 1.7 KiB/s rd, 173 MiB/s wr, 2.77k op/s; 2.2 KiB/s, 0 objects/s recovering 2024-09-16T09:33:41.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:41 smithi070 ceph-mon[30388]: 5.a scrub starts 2024-09-16T09:33:41.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:41 smithi070 ceph-mon[30388]: 5.a scrub ok 2024-09-16T09:33:41.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:41 smithi070 ceph-mon[40691]: pgmap v411: 168 pgs: 168 active+clean; 1.4 GiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 1.7 KiB/s rd, 173 MiB/s wr, 2.77k op/s; 2.2 KiB/s, 0 objects/s recovering 2024-09-16T09:33:41.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:41 smithi070 ceph-mon[40691]: 5.a scrub starts 2024-09-16T09:33:41.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:41 smithi070 ceph-mon[40691]: 5.a scrub ok 2024-09-16T09:33:41.706 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:33:41.707 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 11 16 30895 30879 175.425 147.625 0.00415053 0.0056954 2024-09-16T09:33:41.742 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:41.743 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:41.744 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:33:41.844 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:33:41.881 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:41.882 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:41.882 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:33:41.983 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:33:42.016 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:42.017 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:42.017 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:33:42.118 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:33:42.153 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:42.154 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:42.154 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:33:42.239 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:33:41 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:33:41] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:33:42.255 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:33:42.288 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:42.289 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:42.290 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:33:42.390 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:33:42.425 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:42.426 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:42.426 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:33:42.527 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:33:42.554 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:42.555 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:42.555 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:33:42.577 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:42 smithi070 ceph-mon[30388]: 5.6 scrub starts 2024-09-16T09:33:42.577 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:42 smithi070 ceph-mon[30388]: 5.6 scrub ok 2024-09-16T09:33:42.577 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:42 smithi070 ceph-mon[30388]: 6.12 scrub starts 2024-09-16T09:33:42.577 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:42 smithi070 ceph-mon[30388]: 6.12 scrub ok 2024-09-16T09:33:42.577 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:42 smithi070 ceph-mon[30388]: 4.2 scrub starts 2024-09-16T09:33:42.578 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:42 smithi070 ceph-mon[30388]: 4.2 scrub ok 2024-09-16T09:33:42.578 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:42 smithi070 ceph-mon[40691]: 5.6 scrub starts 2024-09-16T09:33:42.578 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:42 smithi070 ceph-mon[40691]: 5.6 scrub ok 2024-09-16T09:33:42.578 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:42 smithi070 ceph-mon[40691]: 6.12 scrub starts 2024-09-16T09:33:42.578 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:42 smithi070 ceph-mon[40691]: 6.12 scrub ok 2024-09-16T09:33:42.579 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:42 smithi070 ceph-mon[40691]: 4.2 scrub starts 2024-09-16T09:33:42.579 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:42 smithi070 ceph-mon[40691]: 4.2 scrub ok 2024-09-16T09:33:42.650 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:33:42.651 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:33:42.656 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:33:42.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:42 smithi084 ceph-mon[32629]: 5.6 scrub starts 2024-09-16T09:33:42.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:42 smithi084 ceph-mon[32629]: 5.6 scrub ok 2024-09-16T09:33:42.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:42 smithi084 ceph-mon[32629]: 6.12 scrub starts 2024-09-16T09:33:42.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:42 smithi084 ceph-mon[32629]: 6.12 scrub ok 2024-09-16T09:33:42.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:42 smithi084 ceph-mon[32629]: 4.2 scrub starts 2024-09-16T09:33:42.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:42 smithi084 ceph-mon[32629]: 4.2 scrub ok 2024-09-16T09:33:42.678 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:33:42.679 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:33:42.686 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:42.687 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:42.688 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:33:42.704 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:33:42.704 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:33:42.706 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 12 16 33051 33035 172.035 134.75 0.00423087 0.00580673 2024-09-16T09:33:42.742 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:33:42.743 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:33:42.776 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:33:42.776 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:33:42.789 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:33:42.811 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:33:42.811 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:33:42.826 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:42.827 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:42.827 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:33:42.845 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:33:42.846 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:33:42.878 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:33:42.879 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:33:42.892 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:33:42.896 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2024-09-16T09:33:42.929 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:33:42.973 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:42.974 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:42.974 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:33:43.075 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:33:43.110 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:43.110 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:43.111 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:33:43.211 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:33:43.248 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:43.249 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:43.249 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:33:43.350 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:33:43.355 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:33:43 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:33:43.392 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:43.393 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:43.394 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:33:43.407 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.1 dump_ops_in_flight 2024-09-16T09:33:43.435 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:43 smithi070 ceph-mon[30388]: pgmap v412: 168 pgs: 168 active+clean; 1.4 GiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 639 B/s rd, 111 MiB/s wr, 1.78k op/s 2024-09-16T09:33:43.435 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:43 smithi070 ceph-mon[30388]: 3.d scrub starts 2024-09-16T09:33:43.435 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:43 smithi070 ceph-mon[30388]: 3.d scrub ok 2024-09-16T09:33:43.435 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:43 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/616823634' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:33:43.436 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:43 smithi070 ceph-mon[40691]: pgmap v412: 168 pgs: 168 active+clean; 1.4 GiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 639 B/s rd, 111 MiB/s wr, 1.78k op/s 2024-09-16T09:33:43.436 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:43 smithi070 ceph-mon[40691]: 3.d scrub starts 2024-09-16T09:33:43.436 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:43 smithi070 ceph-mon[40691]: 3.d scrub ok 2024-09-16T09:33:43.436 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:43 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/616823634' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:33:43.495 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:33:43.546 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:43.547 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:43.547 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:33:43.648 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:33:43.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:43 smithi084 ceph-mon[32629]: pgmap v412: 168 pgs: 168 active+clean; 1.4 GiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 639 B/s rd, 111 MiB/s wr, 1.78k op/s 2024-09-16T09:33:43.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:43 smithi084 ceph-mon[32629]: 3.d scrub starts 2024-09-16T09:33:43.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:43 smithi084 ceph-mon[32629]: 3.d scrub ok 2024-09-16T09:33:43.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:43 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/616823634' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:33:43.682 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:43.683 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:43.683 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:33:43.706 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 13 16 35133 35117 168.81 130.125 0.00877919 0.00591836 2024-09-16T09:33:43.784 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:33:43.823 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:43.823 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:43.823 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:33:43.924 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:33:43.960 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:43.961 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:43.961 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:33:44.062 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:33:44.101 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:44.101 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:44.102 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:33:44.202 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:33:44.241 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:44.242 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:44.242 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:33:44.343 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:33:44.376 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:44.376 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:44.377 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:33:44.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:44 smithi070 ceph-mon[30388]: 6.16 scrub starts 2024-09-16T09:33:44.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:44 smithi070 ceph-mon[30388]: 6.16 scrub ok 2024-09-16T09:33:44.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:44 smithi070 ceph-mon[30388]: from='client.25532 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:33:44.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:44 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:33:44.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:44 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:33:44.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:44 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:33:44.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:44 smithi070 ceph-mon[30388]: 5.1d scrub starts 2024-09-16T09:33:44.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:44 smithi070 ceph-mon[30388]: 5.1d scrub ok 2024-09-16T09:33:44.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:44 smithi070 ceph-mon[30388]: osdmap e238: 8 total, 8 up, 8 in 2024-09-16T09:33:44.422 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:44 smithi070 ceph-mon[30388]: osdmap e239: 8 total, 8 up, 8 in 2024-09-16T09:33:44.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:44 smithi070 ceph-mon[40691]: 6.16 scrub starts 2024-09-16T09:33:44.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:44 smithi070 ceph-mon[40691]: 6.16 scrub ok 2024-09-16T09:33:44.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:44 smithi070 ceph-mon[40691]: from='client.25532 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:33:44.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:44 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:33:44.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:44 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:33:44.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:44 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:33:44.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:44 smithi070 ceph-mon[40691]: 5.1d scrub starts 2024-09-16T09:33:44.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:44 smithi070 ceph-mon[40691]: 5.1d scrub ok 2024-09-16T09:33:44.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:44 smithi070 ceph-mon[40691]: osdmap e238: 8 total, 8 up, 8 in 2024-09-16T09:33:44.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:44 smithi070 ceph-mon[40691]: osdmap e239: 8 total, 8 up, 8 in 2024-09-16T09:33:44.478 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:33:44.514 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:44.515 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:44.516 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:33:44.617 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:33:44.652 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:44.653 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:44.653 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:33:44.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:44 smithi084 ceph-mon[32629]: 6.16 scrub starts 2024-09-16T09:33:44.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:44 smithi084 ceph-mon[32629]: 6.16 scrub ok 2024-09-16T09:33:44.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:44 smithi084 ceph-mon[32629]: from='client.25532 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:33:44.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:44 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:33:44.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:44 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:33:44.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:44 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:33:44.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:44 smithi084 ceph-mon[32629]: 5.1d scrub starts 2024-09-16T09:33:44.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:44 smithi084 ceph-mon[32629]: 5.1d scrub ok 2024-09-16T09:33:44.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:44 smithi084 ceph-mon[32629]: osdmap e238: 8 total, 8 up, 8 in 2024-09-16T09:33:44.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:44 smithi084 ceph-mon[32629]: osdmap e239: 8 total, 8 up, 8 in 2024-09-16T09:33:44.707 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 14 16 37201 37185 165.983 129.25 0.00366567 0.00601985 2024-09-16T09:33:44.754 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:33:44.787 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:44.787 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:44.788 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:33:44.866 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":412,"stamp":"2024-09-16T09:33:41.660745+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1454596093,"num_objects":22385,"num_object_clones":0,"num_object_copies":67155,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22385,"num_whiteouts":0,"num_read":1225,"num_read_kb":980,"num_write":22625,"num_write_kb":1419998,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":109,"num_bytes_recovered":875298,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":24328,"ondisk_log_size":24328,"up":504,"acting":504,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":506,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":6844056,"kb_used_data":4426044,"kb_used_omap":12,"kb_used_meta":2417843,"kb_avail":743117160,"statfs":{"total":767960285184,"available":760951971840,"internally_reserved":0,"allocated":4532269056,"data_stored":4528891193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":2475871794},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":931987456,"num_objects":14221,"num_object_clones":0,"num_object_copies":42663,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14221,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":14221,"num_write_kb":910144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002748"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":402,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.310257+0000","last_change":"2024-09-16T09:32:39.384174+0000","last_active":"2024-09-16T09:33:36.310257+0000","last_peered":"2024-09-16T09:33:36.310257+0000","last_clean":"2024-09-16T09:33:36.310257+0000","last_became_active":"2024-09-16T09:25:01.709108+0000","last_became_peered":"2024-09-16T09:25:01.709108+0000","last_unstale":"2024-09-16T09:33:36.310257+0000","last_undegraded":"2024-09-16T09:33:36.310257+0000","last_fullsized":"2024-09-16T09:33:36.310257+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:39.384104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:32:39.384104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:41.986629+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"66'9","reported_seq":405,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.303319+0000","last_change":"2024-09-16T09:32:38.074301+0000","last_active":"2024-09-16T09:33:36.303319+0000","last_peered":"2024-09-16T09:33:36.303319+0000","last_clean":"2024-09-16T09:33:36.303319+0000","last_became_active":"2024-09-16T09:25:03.714635+0000","last_became_peered":"2024-09-16T09:25:03.714635+0000","last_unstale":"2024-09-16T09:33:36.303319+0000","last_undegraded":"2024-09-16T09:33:36.303319+0000","last_fullsized":"2024-09-16T09:33:36.303319+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:32:38.074231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:32:38.074231+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:04.491224+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":17,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.307418+0000","last_change":"2024-09-16T09:33:30.242561+0000","last_active":"2024-09-16T09:33:36.307418+0000","last_peered":"2024-09-16T09:33:36.307418+0000","last_clean":"2024-09-16T09:33:36.307418+0000","last_became_active":"2024-09-16T09:33:30.242238+0000","last_became_peered":"2024-09-16T09:33:30.242238+0000","last_unstale":"2024-09-16T09:33:36.307418+0000","last_undegraded":"2024-09-16T09:33:36.307418+0000","last_fullsized":"2024-09-16T09:33:36.307418+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:13.426830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:13.426830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:30.513867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":365,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.303179+0000","last_change":"2024-09-16T09:27:11.565306+0000","last_active":"2024-09-16T09:33:36.303179+0000","last_peered":"2024-09-16T09:33:36.303179+0000","last_clean":"2024-09-16T09:33:36.303179+0000","last_became_active":"2024-09-16T09:25:07.715442+0000","last_became_peered":"2024-09-16T09:25:07.715442+0000","last_unstale":"2024-09-16T09:33:36.303179+0000","last_undegraded":"2024-09-16T09:33:36.303179+0000","last_fullsized":"2024-09-16T09:33:36.303179+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:11.565220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:11.565220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:36.247989+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":110,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.305656+0000","last_change":"2024-09-16T09:31:09.269700+0000","last_active":"2024-09-16T09:33:36.305656+0000","last_peered":"2024-09-16T09:33:36.305656+0000","last_clean":"2024-09-16T09:33:36.305656+0000","last_became_active":"2024-09-16T09:31:09.269342+0000","last_became_peered":"2024-09-16T09:31:09.269342+0000","last_unstale":"2024-09-16T09:33:36.305656+0000","last_undegraded":"2024-09-16T09:33:36.305656+0000","last_fullsized":"2024-09-16T09:33:36.305656+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:00.068570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:28:00.068570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:56.792873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":259,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.312495+0000","last_change":"2024-09-16T09:33:08.816231+0000","last_active":"2024-09-16T09:33:36.312495+0000","last_peered":"2024-09-16T09:33:36.312495+0000","last_clean":"2024-09-16T09:33:36.312495+0000","last_became_active":"2024-09-16T09:31:09.275136+0000","last_became_peered":"2024-09-16T09:31:09.275136+0000","last_unstale":"2024-09-16T09:33:36.312495+0000","last_undegraded":"2024-09-16T09:33:36.312495+0000","last_fullsized":"2024-09-16T09:33:36.312495+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:08.816174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:08.816174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:26.614782+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.19","version":"66'15","reported_seq":74,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.313327+0000","last_change":"2024-09-16T09:33:21.337711+0000","last_active":"2024-09-16T09:33:36.313327+0000","last_peered":"2024-09-16T09:33:36.313327+0000","last_clean":"2024-09-16T09:33:36.313327+0000","last_became_active":"2024-09-16T09:33:21.337284+0000","last_became_peered":"2024-09-16T09:33:21.337284+0000","last_unstale":"2024-09-16T09:33:36.313327+0000","last_undegraded":"2024-09-16T09:33:36.313327+0000","last_fullsized":"2024-09-16T09:33:36.313327+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-16T09:32:45.194581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:32:45.194581+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:08.573888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":393,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.303237+0000","last_change":"2024-09-16T09:32:44.114475+0000","last_active":"2024-09-16T09:33:36.303237+0000","last_peered":"2024-09-16T09:33:36.303237+0000","last_clean":"2024-09-16T09:33:36.303237+0000","last_became_active":"2024-09-16T09:25:05.717236+0000","last_became_peered":"2024-09-16T09:25:05.717236+0000","last_unstale":"2024-09-16T09:33:36.303237+0000","last_undegraded":"2024-09-16T09:33:36.303237+0000","last_fullsized":"2024-09-16T09:33:36.303237+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:44.114407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:32:44.114407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:59.985952+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":26,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.655594+0000","last_change":"2024-09-16T09:33:36.655594+0000","last_active":"2024-09-16T09:33:36.655594+0000","last_peered":"2024-09-16T09:33:36.655594+0000","last_clean":"2024-09-16T09:33:36.655594+0000","last_became_active":"2024-09-16T09:33:30.247733+0000","last_became_peered":"2024-09-16T09:33:30.247733+0000","last_unstale":"2024-09-16T09:33:36.655594+0000","last_undegraded":"2024-09-16T09:33:36.655594+0000","last_fullsized":"2024-09-16T09:33:36.655594+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:36.655236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:36.655236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:57.856840+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1a","version":"66'9","reported_seq":405,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.303346+0000","last_change":"2024-09-16T09:32:48.185484+0000","last_active":"2024-09-16T09:33:36.303346+0000","last_peered":"2024-09-16T09:33:36.303346+0000","last_clean":"2024-09-16T09:33:36.303346+0000","last_became_active":"2024-09-16T09:25:03.709147+0000","last_became_peered":"2024-09-16T09:25:03.709147+0000","last_unstale":"2024-09-16T09:33:36.303346+0000","last_undegraded":"2024-09-16T09:33:36.303346+0000","last_fullsized":"2024-09-16T09:33:36.303346+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:32:48.185380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:32:48.185380+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:13.997808+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":376,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.307444+0000","last_change":"2024-09-16T09:33:21.376175+0000","last_active":"2024-09-16T09:33:36.307444+0000","last_peered":"2024-09-16T09:33:36.307444+0000","last_clean":"2024-09-16T09:33:36.307444+0000","last_became_active":"2024-09-16T09:25:05.718134+0000","last_became_peered":"2024-09-16T09:25:05.718134+0000","last_unstale":"2024-09-16T09:33:36.307444+0000","last_undegraded":"2024-09-16T09:33:36.307444+0000","last_fullsized":"2024-09-16T09:33:36.307444+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:21.376128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:21.376128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:23.947055+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"66'1","reported_seq":382,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.311553+0000","last_change":"2024-09-16T09:31:58.367903+0000","last_active":"2024-09-16T09:33:36.311553+0000","last_peered":"2024-09-16T09:33:36.311553+0000","last_clean":"2024-09-16T09:33:36.311553+0000","last_became_active":"2024-09-16T09:25:07.716558+0000","last_became_peered":"2024-09-16T09:25:07.716558+0000","last_unstale":"2024-09-16T09:33:36.311553+0000","last_undegraded":"2024-09-16T09:33:36.311553+0000","last_fullsized":"2024-09-16T09:33:36.311553+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-16T09:31:58.367837+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:31:58.367837+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":384,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.307594+0000","last_change":"2024-09-16T09:33:19.326765+0000","last_active":"2024-09-16T09:33:36.307594+0000","last_peered":"2024-09-16T09:33:36.307594+0000","last_clean":"2024-09-16T09:33:36.307594+0000","last_became_active":"2024-09-16T09:25:01.696939+0000","last_became_peered":"2024-09-16T09:25:01.696939+0000","last_unstale":"2024-09-16T09:33:36.307594+0000","last_undegraded":"2024-09-16T09:33:36.307594+0000","last_fullsized":"2024-09-16T09:33:36.307594+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:19.326695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:19.326695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:39.073939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"66'5","reported_seq":391,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.303700+0000","last_change":"2024-09-16T09:32:31.024521+0000","last_active":"2024-09-16T09:33:36.303700+0000","last_peered":"2024-09-16T09:33:36.303700+0000","last_clean":"2024-09-16T09:33:36.303700+0000","last_became_active":"2024-09-16T09:25:03.716136+0000","last_became_peered":"2024-09-16T09:25:03.716136+0000","last_unstale":"2024-09-16T09:33:36.303700+0000","last_undegraded":"2024-09-16T09:33:36.303700+0000","last_fullsized":"2024-09-16T09:33:36.303700+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-16T09:32:31.024450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:32:31.024450+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:41.830431+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":376,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.308345+0000","last_change":"2024-09-16T09:28:52.746046+0000","last_active":"2024-09-16T09:33:36.308345+0000","last_peered":"2024-09-16T09:33:36.308345+0000","last_clean":"2024-09-16T09:33:36.308345+0000","last_became_active":"2024-09-16T09:25:05.718281+0000","last_became_peered":"2024-09-16T09:25:05.718281+0000","last_unstale":"2024-09-16T09:33:36.308345+0000","last_undegraded":"2024-09-16T09:33:36.308345+0000","last_fullsized":"2024-09-16T09:33:36.308345+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:52.745999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:28:52.745999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:25.324117+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":394,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.307422+0000","last_change":"2024-09-16T09:31:09.276016+0000","last_active":"2024-09-16T09:33:36.307422+0000","last_peered":"2024-09-16T09:33:36.307422+0000","last_clean":"2024-09-16T09:33:36.307422+0000","last_became_active":"2024-09-16T09:31:09.275656+0000","last_became_peered":"2024-09-16T09:31:09.275656+0000","last_unstale":"2024-09-16T09:33:36.307422+0000","last_undegraded":"2024-09-16T09:33:36.307422+0000","last_fullsized":"2024-09-16T09:33:36.307422+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:50.468134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:50.468134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":382,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.305227+0000","last_change":"2024-09-16T09:29:56.605555+0000","last_active":"2024-09-16T09:33:36.305227+0000","last_peered":"2024-09-16T09:33:36.305227+0000","last_clean":"2024-09-16T09:33:36.305227+0000","last_became_active":"2024-09-16T09:25:07.720793+0000","last_became_peered":"2024-09-16T09:25:07.720793+0000","last_unstale":"2024-09-16T09:33:36.305227+0000","last_undegraded":"2024-09-16T09:33:36.305227+0000","last_fullsized":"2024-09-16T09:33:36.305227+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:56.605488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:56.605488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":133,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.311231+0000","last_change":"2024-09-16T09:30:39.202178+0000","last_active":"2024-09-16T09:33:36.311231+0000","last_peered":"2024-09-16T09:33:36.311231+0000","last_clean":"2024-09-16T09:33:36.311231+0000","last_became_active":"2024-09-16T09:30:39.201824+0000","last_became_peered":"2024-09-16T09:30:39.201824+0000","last_unstale":"2024-09-16T09:33:36.311231+0000","last_undegraded":"2024-09-16T09:33:36.311231+0000","last_fullsized":"2024-09-16T09:33:36.311231+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:50.912979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:50.912979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:25.849444+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"66'15","reported_seq":463,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.308973+0000","last_change":"2024-09-16T09:33:30.250162+0000","last_active":"2024-09-16T09:33:36.308973+0000","last_peered":"2024-09-16T09:33:36.308973+0000","last_clean":"2024-09-16T09:33:36.308973+0000","last_became_active":"2024-09-16T09:33:30.249798+0000","last_became_peered":"2024-09-16T09:33:30.249798+0000","last_unstale":"2024-09-16T09:33:36.308973+0000","last_undegraded":"2024-09-16T09:33:36.308973+0000","last_fullsized":"2024-09-16T09:33:36.308973+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-16T09:32:49.202163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:32:49.202163+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:14.257976+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":17,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.304247+0000","last_change":"2024-09-16T09:33:30.251888+0000","last_active":"2024-09-16T09:33:36.304247+0000","last_peered":"2024-09-16T09:33:36.304247+0000","last_clean":"2024-09-16T09:33:36.304247+0000","last_became_active":"2024-09-16T09:33:30.251587+0000","last_became_peered":"2024-09-16T09:33:30.251587+0000","last_unstale":"2024-09-16T09:33:36.304247+0000","last_undegraded":"2024-09-16T09:33:36.304247+0000","last_fullsized":"2024-09-16T09:33:36.304247+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:55.875480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:55.875480+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,0],"acting":[4,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":230,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.312211+0000","last_change":"2024-09-16T09:30:24.412733+0000","last_active":"2024-09-16T09:33:36.312211+0000","last_peered":"2024-09-16T09:33:36.312211+0000","last_clean":"2024-09-16T09:33:36.312211+0000","last_became_active":"2024-09-16T09:28:52.974255+0000","last_became_peered":"2024-09-16T09:28:52.974255+0000","last_unstale":"2024-09-16T09:33:36.312211+0000","last_undegraded":"2024-09-16T09:33:36.312211+0000","last_fullsized":"2024-09-16T09:33:36.312211+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:30:24.412571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:30:24.412571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:54.602091+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":393,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.303433+0000","last_change":"2024-09-16T09:30:03.628882+0000","last_active":"2024-09-16T09:33:36.303433+0000","last_peered":"2024-09-16T09:33:36.303433+0000","last_clean":"2024-09-16T09:33:36.303433+0000","last_became_active":"2024-09-16T09:25:01.706491+0000","last_became_peered":"2024-09-16T09:25:01.706491+0000","last_unstale":"2024-09-16T09:33:36.303433+0000","last_undegraded":"2024-09-16T09:33:36.303433+0000","last_fullsized":"2024-09-16T09:33:36.303433+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:30:03.628812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:30:03.628812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"66'12","reported_seq":60,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.304349+0000","last_change":"2024-09-16T09:33:30.252042+0000","last_active":"2024-09-16T09:33:36.304349+0000","last_peered":"2024-09-16T09:33:36.304349+0000","last_clean":"2024-09-16T09:33:36.304349+0000","last_became_active":"2024-09-16T09:33:30.251626+0000","last_became_peered":"2024-09-16T09:33:30.251626+0000","last_unstale":"2024-09-16T09:33:36.304349+0000","last_undegraded":"2024-09-16T09:33:36.304349+0000","last_fullsized":"2024-09-16T09:33:36.304349+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-16T09:27:01.700139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:01.700139+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":24,"num_read_kb":15,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":369,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.303345+0000","last_change":"2024-09-16T09:27:00.816500+0000","last_active":"2024-09-16T09:33:36.303345+0000","last_peered":"2024-09-16T09:33:36.303345+0000","last_clean":"2024-09-16T09:33:36.303345+0000","last_became_active":"2024-09-16T09:25:05.710093+0000","last_became_peered":"2024-09-16T09:25:05.710093+0000","last_unstale":"2024-09-16T09:33:36.303345+0000","last_undegraded":"2024-09-16T09:33:36.303345+0000","last_fullsized":"2024-09-16T09:33:36.303345+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:00.816424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:00.816424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":373,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.307845+0000","last_change":"2024-09-16T09:29:29.918433+0000","last_active":"2024-09-16T09:33:36.307845+0000","last_peered":"2024-09-16T09:33:36.307845+0000","last_clean":"2024-09-16T09:33:36.307845+0000","last_became_active":"2024-09-16T09:25:07.719380+0000","last_became_peered":"2024-09-16T09:25:07.719380+0000","last_unstale":"2024-09-16T09:33:36.307845+0000","last_undegraded":"2024-09-16T09:33:36.307845+0000","last_fullsized":"2024-09-16T09:33:36.307845+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-16T09:29:29.918368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:29.918368+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":26,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.305144+0000","last_change":"2024-09-16T09:33:30.245515+0000","last_active":"2024-09-16T09:33:36.305144+0000","last_peered":"2024-09-16T09:33:36.305144+0000","last_clean":"2024-09-16T09:33:36.305144+0000","last_became_active":"2024-09-16T09:33:30.245048+0000","last_became_peered":"2024-09-16T09:33:30.245048+0000","last_unstale":"2024-09-16T09:33:36.305144+0000","last_undegraded":"2024-09-16T09:33:36.305144+0000","last_fullsized":"2024-09-16T09:33:36.305144+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:30:02.843856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:30:02.843856+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:30.982173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"66'10","reported_seq":443,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.310176+0000","last_change":"2024-09-16T09:32:30.286077+0000","last_active":"2024-09-16T09:33:36.310176+0000","last_peered":"2024-09-16T09:33:36.310176+0000","last_clean":"2024-09-16T09:33:36.310176+0000","last_became_active":"2024-09-16T09:31:09.275050+0000","last_became_peered":"2024-09-16T09:31:09.275050+0000","last_unstale":"2024-09-16T09:33:36.310176+0000","last_undegraded":"2024-09-16T09:33:36.310176+0000","last_fullsized":"2024-09-16T09:33:36.310176+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-16T09:32:30.284919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:32:30.284919+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"66'8","reported_seq":517,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.312372+0000","last_change":"2024-09-16T09:32:46.533273+0000","last_active":"2024-09-16T09:33:36.312372+0000","last_peered":"2024-09-16T09:33:36.312372+0000","last_clean":"2024-09-16T09:33:36.312372+0000","last_became_active":"2024-09-16T09:25:05.712516+0000","last_became_peered":"2024-09-16T09:25:05.712516+0000","last_unstale":"2024-09-16T09:33:36.312372+0000","last_undegraded":"2024-09-16T09:33:36.312372+0000","last_fullsized":"2024-09-16T09:33:36.312372+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:32:46.533204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:32:46.533204+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:58.006839+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2","version":"236'1401","reported_seq":1418,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:38.054242+0000","last_change":"2024-09-16T09:33:30.249696+0000","last_active":"2024-09-16T09:33:38.054242+0000","last_peered":"2024-09-16T09:33:38.054242+0000","last_clean":"2024-09-16T09:33:38.054242+0000","last_became_active":"2024-09-16T09:33:30.249413+0000","last_became_peered":"2024-09-16T09:33:30.249413+0000","last_unstale":"2024-09-16T09:33:38.054242+0000","last_undegraded":"2024-09-16T09:33:38.054242+0000","last_fullsized":"2024-09-16T09:33:38.054242+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:33:24.230486+0000","objects_scrubbed":0,"log_size":1401,"log_dups_size":0,"ondisk_log_size":1401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:50.556085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":91815936,"num_objects":1401,"num_object_clones":0,"num_object_copies":4203,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1401,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1401,"num_write_kb":89664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.a","version":"73'79","reported_seq":316,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.309629+0000","last_change":"2024-09-16T09:29:18.498887+0000","last_active":"2024-09-16T09:33:36.309629+0000","last_peered":"2024-09-16T09:33:36.309629+0000","last_clean":"2024-09-16T09:33:36.309629+0000","last_became_active":"2024-09-16T09:26:35.127352+0000","last_became_peered":"2024-09-16T09:26:35.127352+0000","last_unstale":"2024-09-16T09:33:36.309629+0000","last_undegraded":"2024-09-16T09:33:36.309629+0000","last_fullsized":"2024-09-16T09:33:36.309629+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:18.498766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:18.498766+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:26.432871+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"66'15","reported_seq":325,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.307827+0000","last_change":"2024-09-16T09:28:28.065781+0000","last_active":"2024-09-16T09:33:36.307827+0000","last_peered":"2024-09-16T09:33:36.307827+0000","last_clean":"2024-09-16T09:33:36.307827+0000","last_became_active":"2024-09-16T09:26:33.295672+0000","last_became_peered":"2024-09-16T09:26:33.295672+0000","last_unstale":"2024-09-16T09:33:36.307827+0000","last_undegraded":"2024-09-16T09:33:36.307827+0000","last_fullsized":"2024-09-16T09:33:36.307827+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-16T09:28:28.065699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:28.065699+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:26.532283+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":111,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.305467+0000","last_change":"2024-09-16T09:31:09.276858+0000","last_active":"2024-09-16T09:33:36.305467+0000","last_peered":"2024-09-16T09:33:36.305467+0000","last_clean":"2024-09-16T09:33:36.305467+0000","last_became_active":"2024-09-16T09:31:09.274534+0000","last_became_peered":"2024-09-16T09:31:09.274534+0000","last_unstale":"2024-09-16T09:33:36.305467+0000","last_undegraded":"2024-09-16T09:33:36.305467+0000","last_fullsized":"2024-09-16T09:33:36.305467+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:22.091537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:28:22.091537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:57.822748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"66'8","reported_seq":473,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.305300+0000","last_change":"2024-09-16T09:25:05.714128+0000","last_active":"2024-09-16T09:33:36.305300+0000","last_peered":"2024-09-16T09:33:36.305300+0000","last_clean":"2024-09-16T09:33:36.305300+0000","last_became_active":"2024-09-16T09:25:05.713813+0000","last_became_peered":"2024-09-16T09:25:05.713813+0000","last_unstale":"2024-09-16T09:33:36.305300+0000","last_undegraded":"2024-09-16T09:33:36.305300+0000","last_fullsized":"2024-09-16T09:33:36.305300+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:30.117178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":372,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.309115+0000","last_change":"2024-09-16T09:29:24.537802+0000","last_active":"2024-09-16T09:33:36.309115+0000","last_peered":"2024-09-16T09:33:36.309115+0000","last_clean":"2024-09-16T09:33:36.309115+0000","last_became_active":"2024-09-16T09:25:07.717344+0000","last_became_peered":"2024-09-16T09:25:07.717344+0000","last_unstale":"2024-09-16T09:33:36.309115+0000","last_undegraded":"2024-09-16T09:33:36.309115+0000","last_fullsized":"2024-09-16T09:33:36.309115+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:24.537731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:24.537731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:29.409081+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"236'1299","reported_seq":1322,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:37.619845+0000","last_change":"2024-09-16T09:33:25.247861+0000","last_active":"2024-09-16T09:33:37.619845+0000","last_peered":"2024-09-16T09:33:37.619845+0000","last_clean":"2024-09-16T09:33:37.619845+0000","last_became_active":"2024-09-16T09:33:25.247575+0000","last_became_peered":"2024-09-16T09:33:25.247575+0000","last_unstale":"2024-09-16T09:33:37.619845+0000","last_undegraded":"2024-09-16T09:33:37.619845+0000","last_fullsized":"2024-09-16T09:33:37.619845+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:33:24.230486+0000","objects_scrubbed":0,"log_size":1299,"log_dups_size":0,"ondisk_log_size":1299,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:53.056343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":85131264,"num_objects":1299,"num_object_clones":0,"num_object_copies":3897,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1299,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1299,"num_write_kb":83136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,5],"acting":[7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.5","version":"73'79","reported_seq":356,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.304695+0000","last_change":"2024-09-16T09:32:42.123139+0000","last_active":"2024-09-16T09:33:36.304695+0000","last_peered":"2024-09-16T09:33:36.304695+0000","last_clean":"2024-09-16T09:33:36.304695+0000","last_became_active":"2024-09-16T09:30:08.763683+0000","last_became_peered":"2024-09-16T09:30:08.763683+0000","last_unstale":"2024-09-16T09:33:36.304695+0000","last_undegraded":"2024-09-16T09:33:36.304695+0000","last_fullsized":"2024-09-16T09:33:36.304695+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:32:42.123067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:32:42.123067+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:10.308483+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"66'18","reported_seq":331,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.306964+0000","last_change":"2024-09-16T09:33:21.333996+0000","last_active":"2024-09-16T09:33:36.306964+0000","last_peered":"2024-09-16T09:33:36.306964+0000","last_clean":"2024-09-16T09:33:36.306964+0000","last_became_active":"2024-09-16T09:31:09.270434+0000","last_became_peered":"2024-09-16T09:31:09.270434+0000","last_unstale":"2024-09-16T09:33:36.306964+0000","last_undegraded":"2024-09-16T09:33:36.306964+0000","last_fullsized":"2024-09-16T09:33:36.306964+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'18","last_scrub_stamp":"2024-09-16T09:33:21.333937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:33:21.333937+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:42.320549+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":33,"num_read_kb":21,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":221,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.307944+0000","last_change":"2024-09-16T09:28:58.705169+0000","last_active":"2024-09-16T09:33:36.307944+0000","last_peered":"2024-09-16T09:33:36.307944+0000","last_clean":"2024-09-16T09:33:36.307944+0000","last_became_active":"2024-09-16T09:28:52.983543+0000","last_became_peered":"2024-09-16T09:28:52.983543+0000","last_unstale":"2024-09-16T09:33:36.307944+0000","last_undegraded":"2024-09-16T09:33:36.307944+0000","last_fullsized":"2024-09-16T09:33:36.307944+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:58.705097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:28:58.705097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:24.341605+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":400,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.304587+0000","last_change":"2024-09-16T09:31:09.269234+0000","last_active":"2024-09-16T09:33:36.304587+0000","last_peered":"2024-09-16T09:33:36.304587+0000","last_clean":"2024-09-16T09:33:36.304587+0000","last_became_active":"2024-09-16T09:31:09.268864+0000","last_became_peered":"2024-09-16T09:31:09.268864+0000","last_unstale":"2024-09-16T09:33:36.304587+0000","last_undegraded":"2024-09-16T09:33:36.304587+0000","last_fullsized":"2024-09-16T09:33:36.304587+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:50.482240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:50.482240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":388,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.303315+0000","last_change":"2024-09-16T09:32:32.060608+0000","last_active":"2024-09-16T09:33:36.303315+0000","last_peered":"2024-09-16T09:33:36.303315+0000","last_clean":"2024-09-16T09:33:36.303315+0000","last_became_active":"2024-09-16T09:31:09.269433+0000","last_became_peered":"2024-09-16T09:31:09.269433+0000","last_unstale":"2024-09-16T09:33:36.303315+0000","last_undegraded":"2024-09-16T09:33:36.303315+0000","last_fullsized":"2024-09-16T09:33:36.303315+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:32.060564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:32:32.060564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:48.641239+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.c","version":"236'1364","reported_seq":1389,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:37.828830+0000","last_change":"2024-09-16T09:33:25.371514+0000","last_active":"2024-09-16T09:33:37.828830+0000","last_peered":"2024-09-16T09:33:37.828830+0000","last_clean":"2024-09-16T09:33:37.828830+0000","last_became_active":"2024-09-16T09:33:25.371230+0000","last_became_peered":"2024-09-16T09:33:25.371230+0000","last_unstale":"2024-09-16T09:33:37.828830+0000","last_undegraded":"2024-09-16T09:33:37.828830+0000","last_fullsized":"2024-09-16T09:33:37.828830+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:33:24.230486+0000","objects_scrubbed":0,"log_size":1364,"log_dups_size":0,"ondisk_log_size":1364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:32.640270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":89391104,"num_objects":1364,"num_object_clones":0,"num_object_copies":4092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1364,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1364,"num_write_kb":87296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"73'79","reported_seq":111,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.307718+0000","last_change":"2024-09-16T09:31:09.275499+0000","last_active":"2024-09-16T09:33:36.307718+0000","last_peered":"2024-09-16T09:33:36.307718+0000","last_clean":"2024-09-16T09:33:36.307718+0000","last_became_active":"2024-09-16T09:31:09.275139+0000","last_became_peered":"2024-09-16T09:31:09.275139+0000","last_unstale":"2024-09-16T09:33:36.307718+0000","last_undegraded":"2024-09-16T09:33:36.307718+0000","last_fullsized":"2024-09-16T09:33:36.307718+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:30:22.438378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:30:22.438378+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:06.886988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"66'14","reported_seq":411,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.304684+0000","last_change":"2024-09-16T09:32:34.997417+0000","last_active":"2024-09-16T09:33:36.304684+0000","last_peered":"2024-09-16T09:33:36.304684+0000","last_clean":"2024-09-16T09:33:36.304684+0000","last_became_active":"2024-09-16T09:25:03.708830+0000","last_became_peered":"2024-09-16T09:25:03.708830+0000","last_unstale":"2024-09-16T09:33:36.304684+0000","last_undegraded":"2024-09-16T09:33:36.304684+0000","last_fullsized":"2024-09-16T09:33:36.304684+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'14","last_scrub_stamp":"2024-09-16T09:32:34.997360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:32:34.997360+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:47.824918+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":402,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.310359+0000","last_change":"2024-09-16T09:31:57.373606+0000","last_active":"2024-09-16T09:33:36.310359+0000","last_peered":"2024-09-16T09:33:36.310359+0000","last_clean":"2024-09-16T09:33:36.310359+0000","last_became_active":"2024-09-16T09:25:01.702992+0000","last_became_peered":"2024-09-16T09:25:01.702992+0000","last_unstale":"2024-09-16T09:33:36.310359+0000","last_undegraded":"2024-09-16T09:33:36.310359+0000","last_fullsized":"2024-09-16T09:33:36.310359+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:31:57.373544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:31:57.373544+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"235'12","reported_seq":45,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.303191+0000","last_change":"2024-09-16T09:33:30.251570+0000","last_active":"2024-09-16T09:33:36.303191+0000","last_peered":"2024-09-16T09:33:36.303191+0000","last_clean":"2024-09-16T09:33:36.303191+0000","last_became_active":"2024-09-16T09:33:30.251047+0000","last_became_peered":"2024-09-16T09:33:30.251047+0000","last_unstale":"2024-09-16T09:33:36.303191+0000","last_undegraded":"2024-09-16T09:33:36.303191+0000","last_fullsized":"2024-09-16T09:33:36.303191+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"187'11","last_scrub_stamp":"2024-09-16T09:33:09.417852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:09.417852+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:28.382638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":365,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.308521+0000","last_change":"2024-09-16T09:26:40.699888+0000","last_active":"2024-09-16T09:33:36.308521+0000","last_peered":"2024-09-16T09:33:36.308521+0000","last_clean":"2024-09-16T09:33:36.308521+0000","last_became_active":"2024-09-16T09:25:07.722017+0000","last_became_peered":"2024-09-16T09:25:07.722017+0000","last_unstale":"2024-09-16T09:33:36.308521+0000","last_undegraded":"2024-09-16T09:33:36.308521+0000","last_fullsized":"2024-09-16T09:33:36.308521+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:40.699813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:40.699813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:55.373609+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"236'1433","reported_seq":1458,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:38.040585+0000","last_change":"2024-09-16T09:33:25.372320+0000","last_active":"2024-09-16T09:33:38.040585+0000","last_peered":"2024-09-16T09:33:38.040585+0000","last_clean":"2024-09-16T09:33:38.040585+0000","last_became_active":"2024-09-16T09:33:25.372005+0000","last_became_peered":"2024-09-16T09:33:25.372005+0000","last_unstale":"2024-09-16T09:33:38.040585+0000","last_undegraded":"2024-09-16T09:33:38.040585+0000","last_fullsized":"2024-09-16T09:33:38.040585+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:33:24.230486+0000","objects_scrubbed":0,"log_size":1433,"log_dups_size":0,"ondisk_log_size":1433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:42.452737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":93913088,"num_objects":1433,"num_object_clones":0,"num_object_copies":4299,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1433,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1433,"num_write_kb":91712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.7","version":"73'79","reported_seq":442,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.312122+0000","last_change":"2024-09-16T09:32:45.555132+0000","last_active":"2024-09-16T09:33:36.312122+0000","last_peered":"2024-09-16T09:33:36.312122+0000","last_clean":"2024-09-16T09:33:36.312122+0000","last_became_active":"2024-09-16T09:31:09.274707+0000","last_became_peered":"2024-09-16T09:31:09.274707+0000","last_unstale":"2024-09-16T09:33:36.312122+0000","last_undegraded":"2024-09-16T09:33:36.312122+0000","last_fullsized":"2024-09-16T09:33:36.312122+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:32:45.555067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:32:45.555067+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:56.478822+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"66'10","reported_seq":52,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.310499+0000","last_change":"2024-09-16T09:33:30.248928+0000","last_active":"2024-09-16T09:33:36.310499+0000","last_peered":"2024-09-16T09:33:36.310499+0000","last_clean":"2024-09-16T09:33:36.310499+0000","last_became_active":"2024-09-16T09:33:30.248453+0000","last_became_peered":"2024-09-16T09:33:30.248453+0000","last_unstale":"2024-09-16T09:33:36.310499+0000","last_undegraded":"2024-09-16T09:33:36.310499+0000","last_fullsized":"2024-09-16T09:33:36.310499+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-16T09:30:04.848234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:30:04.848234+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":14,"num_read_kb":9,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":399,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.309545+0000","last_change":"2024-09-16T09:32:46.388711+0000","last_active":"2024-09-16T09:33:36.309545+0000","last_peered":"2024-09-16T09:33:36.309545+0000","last_clean":"2024-09-16T09:33:36.309545+0000","last_became_active":"2024-09-16T09:31:09.270619+0000","last_became_peered":"2024-09-16T09:31:09.270619+0000","last_unstale":"2024-09-16T09:33:36.309545+0000","last_undegraded":"2024-09-16T09:33:36.309545+0000","last_fullsized":"2024-09-16T09:33:36.309545+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:46.388651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:32:46.388651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:12.727689+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"66'8","reported_seq":490,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.311506+0000","last_change":"2024-09-16T09:28:57.628615+0000","last_active":"2024-09-16T09:33:36.311506+0000","last_peered":"2024-09-16T09:33:36.311506+0000","last_clean":"2024-09-16T09:33:36.311506+0000","last_became_active":"2024-09-16T09:25:05.714507+0000","last_became_peered":"2024-09-16T09:25:05.714507+0000","last_unstale":"2024-09-16T09:33:36.311506+0000","last_undegraded":"2024-09-16T09:33:36.311506+0000","last_fullsized":"2024-09-16T09:33:36.311506+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:28:57.628543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:28:57.628543+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:52.447851+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":396,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.312525+0000","last_change":"2024-09-16T09:32:33.362233+0000","last_active":"2024-09-16T09:33:36.312525+0000","last_peered":"2024-09-16T09:33:36.312525+0000","last_clean":"2024-09-16T09:33:36.312525+0000","last_became_active":"2024-09-16T09:31:09.283192+0000","last_became_peered":"2024-09-16T09:31:09.283192+0000","last_unstale":"2024-09-16T09:33:36.312525+0000","last_undegraded":"2024-09-16T09:33:36.312525+0000","last_fullsized":"2024-09-16T09:33:36.312525+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:33.362188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:32:33.362188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:40.243325+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.e","version":"236'1361","reported_seq":1386,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:37.617886+0000","last_change":"2024-09-16T09:33:25.247961+0000","last_active":"2024-09-16T09:33:37.617886+0000","last_peered":"2024-09-16T09:33:37.617886+0000","last_clean":"2024-09-16T09:33:37.617886+0000","last_became_active":"2024-09-16T09:33:25.247683+0000","last_became_peered":"2024-09-16T09:33:25.247683+0000","last_unstale":"2024-09-16T09:33:37.617886+0000","last_undegraded":"2024-09-16T09:33:37.617886+0000","last_fullsized":"2024-09-16T09:33:37.617886+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:33:24.230486+0000","objects_scrubbed":0,"log_size":1361,"log_dups_size":0,"ondisk_log_size":1361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:38.326992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":89194496,"num_objects":1361,"num_object_clones":0,"num_object_copies":4083,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1361,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1361,"num_write_kb":87104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.6","version":"73'79","reported_seq":415,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.311612+0000","last_change":"2024-09-16T09:31:09.274975+0000","last_active":"2024-09-16T09:33:36.311612+0000","last_peered":"2024-09-16T09:33:36.311612+0000","last_clean":"2024-09-16T09:33:36.311612+0000","last_became_active":"2024-09-16T09:31:09.274565+0000","last_became_peered":"2024-09-16T09:31:09.274565+0000","last_unstale":"2024-09-16T09:33:36.311612+0000","last_undegraded":"2024-09-16T09:33:36.311612+0000","last_fullsized":"2024-09-16T09:33:36.311612+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:30:22.415051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:30:22.415051+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:57.648747+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"66'19","reported_seq":418,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.311791+0000","last_change":"2024-09-16T09:31:35.249952+0000","last_active":"2024-09-16T09:33:36.311791+0000","last_peered":"2024-09-16T09:33:36.311791+0000","last_clean":"2024-09-16T09:33:36.311791+0000","last_became_active":"2024-09-16T09:25:03.714797+0000","last_became_peered":"2024-09-16T09:25:03.714797+0000","last_unstale":"2024-09-16T09:33:36.311791+0000","last_undegraded":"2024-09-16T09:33:36.311791+0000","last_fullsized":"2024-09-16T09:33:36.311791+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-16T09:31:35.249864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:31:35.249864+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:02.472785+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.307440+0000","last_change":"2024-09-16T09:33:30.247311+0000","last_active":"2024-09-16T09:33:36.307440+0000","last_peered":"2024-09-16T09:33:36.307440+0000","last_clean":"2024-09-16T09:33:36.307440+0000","last_became_active":"2024-09-16T09:33:30.246996+0000","last_became_peered":"2024-09-16T09:33:30.246996+0000","last_unstale":"2024-09-16T09:33:36.307440+0000","last_undegraded":"2024-09-16T09:33:36.307440+0000","last_fullsized":"2024-09-16T09:33:36.307440+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:11.407616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:11.407616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:40.581284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,2],"acting":[5,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":394,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.311260+0000","last_change":"2024-09-16T09:31:55.456669+0000","last_active":"2024-09-16T09:33:36.311260+0000","last_peered":"2024-09-16T09:33:36.311260+0000","last_clean":"2024-09-16T09:33:36.311260+0000","last_became_active":"2024-09-16T09:25:05.708663+0000","last_became_peered":"2024-09-16T09:25:05.708663+0000","last_unstale":"2024-09-16T09:33:36.311260+0000","last_undegraded":"2024-09-16T09:33:36.311260+0000","last_fullsized":"2024-09-16T09:33:36.311260+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:31:55.456592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:31:55.456592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:58.609844+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":17,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.310691+0000","last_change":"2024-09-16T09:33:30.250470+0000","last_active":"2024-09-16T09:33:36.310691+0000","last_peered":"2024-09-16T09:33:36.310691+0000","last_clean":"2024-09-16T09:33:36.310691+0000","last_became_active":"2024-09-16T09:33:30.250178+0000","last_became_peered":"2024-09-16T09:33:30.250178+0000","last_unstale":"2024-09-16T09:33:36.310691+0000","last_undegraded":"2024-09-16T09:33:36.310691+0000","last_fullsized":"2024-09-16T09:33:36.310691+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:37.282735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:37.282735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:34.877926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,2],"acting":[6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"236'1451","reported_seq":1476,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:38.254796+0000","last_change":"2024-09-16T09:33:25.243740+0000","last_active":"2024-09-16T09:33:38.254796+0000","last_peered":"2024-09-16T09:33:38.254796+0000","last_clean":"2024-09-16T09:33:38.254796+0000","last_became_active":"2024-09-16T09:33:25.243446+0000","last_became_peered":"2024-09-16T09:33:25.243446+0000","last_unstale":"2024-09-16T09:33:38.254796+0000","last_undegraded":"2024-09-16T09:33:38.254796+0000","last_fullsized":"2024-09-16T09:33:38.254796+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:33:24.230486+0000","objects_scrubbed":0,"log_size":1451,"log_dups_size":0,"ondisk_log_size":1451,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:42.248915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":95092736,"num_objects":1451,"num_object_clones":0,"num_object_copies":4353,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1451,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1451,"num_write_kb":92864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,5],"acting":[0,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"73'79","reported_seq":348,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.307314+0000","last_change":"2024-09-16T09:32:43.420831+0000","last_active":"2024-09-16T09:33:36.307314+0000","last_peered":"2024-09-16T09:33:36.307314+0000","last_clean":"2024-09-16T09:33:36.307314+0000","last_became_active":"2024-09-16T09:30:08.766350+0000","last_became_peered":"2024-09-16T09:30:08.766350+0000","last_unstale":"2024-09-16T09:33:36.307314+0000","last_undegraded":"2024-09-16T09:33:36.307314+0000","last_fullsized":"2024-09-16T09:33:36.307314+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:32:43.420784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:32:43.420784+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:55.813620+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"187'30","reported_seq":599,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.310151+0000","last_change":"2024-09-16T09:33:20.302059+0000","last_active":"2024-09-16T09:33:36.310151+0000","last_peered":"2024-09-16T09:33:36.310151+0000","last_clean":"2024-09-16T09:33:36.310151+0000","last_became_active":"2024-09-16T09:31:09.278839+0000","last_became_peered":"2024-09-16T09:31:09.278839+0000","last_unstale":"2024-09-16T09:33:36.310151+0000","last_undegraded":"2024-09-16T09:33:36.310151+0000","last_fullsized":"2024-09-16T09:33:36.310151+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"187'30","last_scrub_stamp":"2024-09-16T09:33:20.301991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:33:20.301991+0000","objects_scrubbed":10,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:24.584409+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":386,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.304051+0000","last_change":"2024-09-16T09:29:26.637070+0000","last_active":"2024-09-16T09:33:36.304051+0000","last_peered":"2024-09-16T09:33:36.304051+0000","last_clean":"2024-09-16T09:33:36.304051+0000","last_became_active":"2024-09-16T09:25:01.703076+0000","last_became_peered":"2024-09-16T09:25:01.703076+0000","last_unstale":"2024-09-16T09:33:36.304051+0000","last_undegraded":"2024-09-16T09:33:36.304051+0000","last_fullsized":"2024-09-16T09:33:36.304051+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:26.637027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:26.637027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:52.239472+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":395,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.311179+0000","last_change":"2024-09-16T09:29:29.486284+0000","last_active":"2024-09-16T09:33:36.311179+0000","last_peered":"2024-09-16T09:33:36.311179+0000","last_clean":"2024-09-16T09:33:36.311179+0000","last_became_active":"2024-09-16T09:25:00.696237+0000","last_became_peered":"2024-09-16T09:25:00.696237+0000","last_unstale":"2024-09-16T09:33:36.311179+0000","last_undegraded":"2024-09-16T09:33:36.311179+0000","last_fullsized":"2024-09-16T09:33:36.311179+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-16T09:29:29.486188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:29:29.486188+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.0025137430000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":370,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.310840+0000","last_change":"2024-09-16T09:33:16.666704+0000","last_active":"2024-09-16T09:33:36.310840+0000","last_peered":"2024-09-16T09:33:36.310840+0000","last_clean":"2024-09-16T09:33:36.310840+0000","last_became_active":"2024-09-16T09:25:05.717591+0000","last_became_peered":"2024-09-16T09:25:05.717591+0000","last_unstale":"2024-09-16T09:33:36.310840+0000","last_undegraded":"2024-09-16T09:33:36.310840+0000","last_fullsized":"2024-09-16T09:33:36.310840+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:16.666648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:16.666648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:27.059630+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":111,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.306982+0000","last_change":"2024-09-16T09:31:09.272967+0000","last_active":"2024-09-16T09:33:36.306982+0000","last_peered":"2024-09-16T09:33:36.306982+0000","last_clean":"2024-09-16T09:33:36.306982+0000","last_became_active":"2024-09-16T09:31:09.272609+0000","last_became_peered":"2024-09-16T09:31:09.272609+0000","last_unstale":"2024-09-16T09:33:36.306982+0000","last_undegraded":"2024-09-16T09:33:36.306982+0000","last_fullsized":"2024-09-16T09:33:36.306982+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:45.201503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:45.201503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:52.833583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.a","version":"236'1389","reported_seq":1413,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:38.043879+0000","last_change":"2024-09-16T09:33:25.244002+0000","last_active":"2024-09-16T09:33:38.043879+0000","last_peered":"2024-09-16T09:33:38.043879+0000","last_clean":"2024-09-16T09:33:38.043879+0000","last_became_active":"2024-09-16T09:33:25.243701+0000","last_became_peered":"2024-09-16T09:33:25.243701+0000","last_unstale":"2024-09-16T09:33:38.043879+0000","last_undegraded":"2024-09-16T09:33:38.043879+0000","last_fullsized":"2024-09-16T09:33:38.043879+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:33:24.230486+0000","objects_scrubbed":0,"log_size":1389,"log_dups_size":0,"ondisk_log_size":1389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:51.963330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":91029504,"num_objects":1389,"num_object_clones":0,"num_object_copies":4167,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1389,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1389,"num_write_kb":88896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"73'79","reported_seq":473,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.308226+0000","last_change":"2024-09-16T09:32:42.498732+0000","last_active":"2024-09-16T09:33:36.308226+0000","last_peered":"2024-09-16T09:33:36.308226+0000","last_clean":"2024-09-16T09:33:36.308226+0000","last_became_active":"2024-09-16T09:30:08.765943+0000","last_became_peered":"2024-09-16T09:30:08.765943+0000","last_unstale":"2024-09-16T09:33:36.308226+0000","last_undegraded":"2024-09-16T09:33:36.308226+0000","last_fullsized":"2024-09-16T09:33:36.308226+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:32:42.498658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:32:42.498658+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:09.187680+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"66'13","reported_seq":64,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.310004+0000","last_change":"2024-09-16T09:33:30.250219+0000","last_active":"2024-09-16T09:33:36.310004+0000","last_peered":"2024-09-16T09:33:36.310004+0000","last_clean":"2024-09-16T09:33:36.310004+0000","last_became_active":"2024-09-16T09:33:30.249864+0000","last_became_peered":"2024-09-16T09:33:30.249864+0000","last_unstale":"2024-09-16T09:33:36.310004+0000","last_undegraded":"2024-09-16T09:33:36.310004+0000","last_fullsized":"2024-09-16T09:33:36.310004+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'13","last_scrub_stamp":"2024-09-16T09:27:44.155969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:44.155969+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:30.437129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":18,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.311108+0000","last_change":"2024-09-16T09:33:30.248260+0000","last_active":"2024-09-16T09:33:36.311108+0000","last_peered":"2024-09-16T09:33:36.311108+0000","last_clean":"2024-09-16T09:33:36.311108+0000","last_became_active":"2024-09-16T09:33:30.247596+0000","last_became_peered":"2024-09-16T09:33:30.247596+0000","last_unstale":"2024-09-16T09:33:36.311108+0000","last_undegraded":"2024-09-16T09:33:36.311108+0000","last_fullsized":"2024-09-16T09:33:36.311108+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:22.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:22.331302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:45.174192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,2],"acting":[6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":41,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.313423+0000","last_change":"2024-09-16T09:33:21.338149+0000","last_active":"2024-09-16T09:33:36.313423+0000","last_peered":"2024-09-16T09:33:36.313423+0000","last_clean":"2024-09-16T09:33:36.313423+0000","last_became_active":"2024-09-16T09:33:21.337177+0000","last_became_peered":"2024-09-16T09:33:21.337177+0000","last_unstale":"2024-09-16T09:33:36.313423+0000","last_undegraded":"2024-09-16T09:33:36.313423+0000","last_fullsized":"2024-09-16T09:33:36.313423+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:32:40.165749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:32:40.165749+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:45.947620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":393,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.307988+0000","last_change":"2024-09-16T09:32:32.029895+0000","last_active":"2024-09-16T09:33:36.307988+0000","last_peered":"2024-09-16T09:33:36.307988+0000","last_clean":"2024-09-16T09:33:36.307988+0000","last_became_active":"2024-09-16T09:25:05.711539+0000","last_became_peered":"2024-09-16T09:25:05.711539+0000","last_unstale":"2024-09-16T09:33:36.307988+0000","last_undegraded":"2024-09-16T09:33:36.307988+0000","last_fullsized":"2024-09-16T09:33:36.307988+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:32.029829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:32:32.029829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:36.831098+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":373,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.308098+0000","last_change":"2024-09-16T09:29:22.968971+0000","last_active":"2024-09-16T09:33:36.308098+0000","last_peered":"2024-09-16T09:33:36.308098+0000","last_clean":"2024-09-16T09:33:36.308098+0000","last_became_active":"2024-09-16T09:25:07.723282+0000","last_became_peered":"2024-09-16T09:25:07.723282+0000","last_unstale":"2024-09-16T09:33:36.308098+0000","last_undegraded":"2024-09-16T09:33:36.308098+0000","last_fullsized":"2024-09-16T09:33:36.308098+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:22.968908+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:22.968908+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.b","version":"236'1376","reported_seq":1401,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:38.243496+0000","last_change":"2024-09-16T09:33:25.245108+0000","last_active":"2024-09-16T09:33:38.243496+0000","last_peered":"2024-09-16T09:33:38.243496+0000","last_clean":"2024-09-16T09:33:38.243496+0000","last_became_active":"2024-09-16T09:33:25.244819+0000","last_became_peered":"2024-09-16T09:33:25.244819+0000","last_unstale":"2024-09-16T09:33:38.243496+0000","last_undegraded":"2024-09-16T09:33:38.243496+0000","last_fullsized":"2024-09-16T09:33:38.243496+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:33:24.230486+0000","objects_scrubbed":0,"log_size":1376,"log_dups_size":0,"ondisk_log_size":1376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:35.857241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90177536,"num_objects":1376,"num_object_clones":0,"num_object_copies":4128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1376,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1376,"num_write_kb":88064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"73'79","reported_seq":95,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.311369+0000","last_change":"2024-09-16T09:33:31.273689+0000","last_active":"2024-09-16T09:33:36.311369+0000","last_peered":"2024-09-16T09:33:36.311369+0000","last_clean":"2024-09-16T09:33:36.311369+0000","last_became_active":"2024-09-16T09:33:30.249309+0000","last_became_peered":"2024-09-16T09:33:30.249309+0000","last_unstale":"2024-09-16T09:33:36.311369+0000","last_undegraded":"2024-09-16T09:33:36.311369+0000","last_fullsized":"2024-09-16T09:33:36.311369+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:31:40.212889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:31:40.212889+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:34:00.887782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17890,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":6,"num_write":3,"num_write_kb":49,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":17890,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.6","version":"66'12","reported_seq":399,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.310261+0000","last_change":"2024-09-16T09:32:48.547237+0000","last_active":"2024-09-16T09:33:36.310261+0000","last_peered":"2024-09-16T09:33:36.310261+0000","last_clean":"2024-09-16T09:33:36.310261+0000","last_became_active":"2024-09-16T09:25:03.715315+0000","last_became_peered":"2024-09-16T09:25:03.715315+0000","last_unstale":"2024-09-16T09:33:36.310261+0000","last_undegraded":"2024-09-16T09:33:36.310261+0000","last_fullsized":"2024-09-16T09:33:36.310261+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-16T09:32:48.546869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:32:48.546869+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:14.182805+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"59'1","reported_seq":417,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.313132+0000","last_change":"2024-09-16T09:33:21.682893+0000","last_active":"2024-09-16T09:33:36.313132+0000","last_peered":"2024-09-16T09:33:36.313132+0000","last_clean":"2024-09-16T09:33:36.313132+0000","last_became_active":"2024-09-16T09:31:09.269627+0000","last_became_peered":"2024-09-16T09:31:09.269627+0000","last_unstale":"2024-09-16T09:33:36.313132+0000","last_undegraded":"2024-09-16T09:33:36.313132+0000","last_fullsized":"2024-09-16T09:33:36.313132+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:33:21.682840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:21.682840+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:27.930048+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1986,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"67'5","reported_seq":914,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:37.358839+0000","last_change":"2024-09-16T09:33:37.358839+0000","last_active":"2024-09-16T09:33:37.358839+0000","last_peered":"2024-09-16T09:33:37.358839+0000","last_clean":"2024-09-16T09:33:37.358839+0000","last_became_active":"2024-09-16T09:25:00.696546+0000","last_became_peered":"2024-09-16T09:25:00.696546+0000","last_unstale":"2024-09-16T09:33:37.358839+0000","last_undegraded":"2024-09-16T09:33:37.358839+0000","last_fullsized":"2024-09-16T09:33:37.358839+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'5","last_scrub_stamp":"2024-09-16T09:33:37.358741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:33:37.358741+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:37.945861+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0.0059770420000000001,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":510,"num_read_kb":505,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":368,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.310096+0000","last_change":"2024-09-16T09:27:07.337170+0000","last_active":"2024-09-16T09:33:36.310096+0000","last_peered":"2024-09-16T09:33:36.310096+0000","last_clean":"2024-09-16T09:33:36.310096+0000","last_became_active":"2024-09-16T09:25:05.714704+0000","last_became_peered":"2024-09-16T09:25:05.714704+0000","last_unstale":"2024-09-16T09:33:36.310096+0000","last_undegraded":"2024-09-16T09:33:36.310096+0000","last_fullsized":"2024-09-16T09:33:36.310096+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:07.337124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:07.337124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:58.250683+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":287,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.307676+0000","last_change":"2024-09-16T09:33:30.246166+0000","last_active":"2024-09-16T09:33:36.307676+0000","last_peered":"2024-09-16T09:33:36.307676+0000","last_clean":"2024-09-16T09:33:36.307676+0000","last_became_active":"2024-09-16T09:33:30.245852+0000","last_became_peered":"2024-09-16T09:33:30.245852+0000","last_unstale":"2024-09-16T09:33:36.307676+0000","last_undegraded":"2024-09-16T09:33:36.307676+0000","last_fullsized":"2024-09-16T09:33:36.307676+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:46.252700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:32:46.252700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:56.206379+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"236'1456","reported_seq":1481,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:38.247534+0000","last_change":"2024-09-16T09:33:25.371361+0000","last_active":"2024-09-16T09:33:38.247534+0000","last_peered":"2024-09-16T09:33:38.247534+0000","last_clean":"2024-09-16T09:33:38.247534+0000","last_became_active":"2024-09-16T09:33:25.371058+0000","last_became_peered":"2024-09-16T09:33:25.371058+0000","last_unstale":"2024-09-16T09:33:38.247534+0000","last_undegraded":"2024-09-16T09:33:38.247534+0000","last_fullsized":"2024-09-16T09:33:38.247534+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:33:24.230486+0000","objects_scrubbed":0,"log_size":1456,"log_dups_size":0,"ondisk_log_size":1456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:53.759272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":95420416,"num_objects":1456,"num_object_clones":0,"num_object_copies":4368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1456,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1456,"num_write_kb":93184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"66'16","reported_seq":524,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.312180+0000","last_change":"2024-09-16T09:33:16.891850+0000","last_active":"2024-09-16T09:33:36.312180+0000","last_peered":"2024-09-16T09:33:36.312180+0000","last_clean":"2024-09-16T09:33:36.312180+0000","last_became_active":"2024-09-16T09:25:03.714118+0000","last_became_peered":"2024-09-16T09:25:03.714118+0000","last_unstale":"2024-09-16T09:33:36.312180+0000","last_undegraded":"2024-09-16T09:33:36.312180+0000","last_fullsized":"2024-09-16T09:33:36.312180+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'16","last_scrub_stamp":"2024-09-16T09:33:16.891761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:33:16.891761+0000","objects_scrubbed":8,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:44.643462+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":235,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.310576+0000","last_change":"2024-09-16T09:33:12.171507+0000","last_active":"2024-09-16T09:33:36.310576+0000","last_peered":"2024-09-16T09:33:36.310576+0000","last_clean":"2024-09-16T09:33:36.310576+0000","last_became_active":"2024-09-16T09:31:09.270196+0000","last_became_peered":"2024-09-16T09:31:09.270196+0000","last_unstale":"2024-09-16T09:33:36.310576+0000","last_undegraded":"2024-09-16T09:33:36.310576+0000","last_fullsized":"2024-09-16T09:33:36.310576+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:12.171439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:12.171439+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:21.191504+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"73'79","reported_seq":473,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.307283+0000","last_change":"2024-09-16T09:32:43.535041+0000","last_active":"2024-09-16T09:33:36.307283+0000","last_peered":"2024-09-16T09:33:36.307283+0000","last_clean":"2024-09-16T09:33:36.307283+0000","last_became_active":"2024-09-16T09:30:08.870394+0000","last_became_peered":"2024-09-16T09:30:08.870394+0000","last_unstale":"2024-09-16T09:33:36.307283+0000","last_undegraded":"2024-09-16T09:33:36.307283+0000","last_fullsized":"2024-09-16T09:33:36.307283+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:32:43.534962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:32:43.534962+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:46.508762+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":369,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.308044+0000","last_change":"2024-09-16T09:26:39.699133+0000","last_active":"2024-09-16T09:33:36.308044+0000","last_peered":"2024-09-16T09:33:36.308044+0000","last_clean":"2024-09-16T09:33:36.308044+0000","last_became_active":"2024-09-16T09:25:05.714341+0000","last_became_peered":"2024-09-16T09:25:05.714341+0000","last_unstale":"2024-09-16T09:33:36.308044+0000","last_undegraded":"2024-09-16T09:33:36.308044+0000","last_fullsized":"2024-09-16T09:33:36.308044+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:39.699059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:26:39.699059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:51.437385+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":372,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.309726+0000","last_change":"2024-09-16T09:29:00.243528+0000","last_active":"2024-09-16T09:33:36.309726+0000","last_peered":"2024-09-16T09:33:36.309726+0000","last_clean":"2024-09-16T09:33:36.309726+0000","last_became_active":"2024-09-16T09:25:07.714281+0000","last_became_peered":"2024-09-16T09:25:07.714281+0000","last_unstale":"2024-09-16T09:33:36.309726+0000","last_undegraded":"2024-09-16T09:33:36.309726+0000","last_fullsized":"2024-09-16T09:33:36.309726+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:00.243482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:00.243482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:25.391783+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3","version":"236'1291","reported_seq":1316,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:37.604933+0000","last_change":"2024-09-16T09:33:25.371134+0000","last_active":"2024-09-16T09:33:37.604933+0000","last_peered":"2024-09-16T09:33:37.604933+0000","last_clean":"2024-09-16T09:33:37.604933+0000","last_became_active":"2024-09-16T09:33:25.370696+0000","last_became_peered":"2024-09-16T09:33:25.370696+0000","last_unstale":"2024-09-16T09:33:37.604933+0000","last_undegraded":"2024-09-16T09:33:37.604933+0000","last_fullsized":"2024-09-16T09:33:37.604933+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:33:24.230486+0000","objects_scrubbed":0,"log_size":1291,"log_dups_size":0,"ondisk_log_size":1291,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:26.261810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":84606976,"num_objects":1291,"num_object_clones":0,"num_object_copies":3873,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1291,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1291,"num_write_kb":82624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.b","version":"73'79","reported_seq":198,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.310984+0000","last_change":"2024-09-16T09:33:31.278730+0000","last_active":"2024-09-16T09:33:36.310984+0000","last_peered":"2024-09-16T09:33:36.310984+0000","last_clean":"2024-09-16T09:33:36.310984+0000","last_became_active":"2024-09-16T09:33:30.251933+0000","last_became_peered":"2024-09-16T09:33:30.251933+0000","last_unstale":"2024-09-16T09:33:36.310984+0000","last_undegraded":"2024-09-16T09:33:36.310984+0000","last_fullsized":"2024-09-16T09:33:36.310984+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:32:44.255693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:32:44.255693+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:34:00.640407+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"66'11","reported_seq":411,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.305321+0000","last_change":"2024-09-16T09:30:05.646701+0000","last_active":"2024-09-16T09:33:36.305321+0000","last_peered":"2024-09-16T09:33:36.305321+0000","last_clean":"2024-09-16T09:33:36.305321+0000","last_became_active":"2024-09-16T09:26:33.287399+0000","last_became_peered":"2024-09-16T09:26:33.287399+0000","last_unstale":"2024-09-16T09:33:36.305321+0000","last_undegraded":"2024-09-16T09:33:36.305321+0000","last_fullsized":"2024-09-16T09:33:36.305321+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-16T09:30:05.646500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:30:05.646500+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":377,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.304993+0000","last_change":"2024-09-16T09:33:23.200213+0000","last_active":"2024-09-16T09:33:36.304993+0000","last_peered":"2024-09-16T09:33:36.304993+0000","last_clean":"2024-09-16T09:33:36.304993+0000","last_became_active":"2024-09-16T09:25:01.708270+0000","last_became_peered":"2024-09-16T09:25:01.708270+0000","last_unstale":"2024-09-16T09:33:36.304993+0000","last_undegraded":"2024-09-16T09:33:36.304993+0000","last_fullsized":"2024-09-16T09:33:36.304993+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:23.200166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:23.200166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:26.623629+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":368,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.309512+0000","last_change":"2024-09-16T09:26:34.171190+0000","last_active":"2024-09-16T09:33:36.309512+0000","last_peered":"2024-09-16T09:33:36.309512+0000","last_clean":"2024-09-16T09:33:36.309512+0000","last_became_active":"2024-09-16T09:25:05.711964+0000","last_became_peered":"2024-09-16T09:25:05.711964+0000","last_unstale":"2024-09-16T09:33:36.309512+0000","last_undegraded":"2024-09-16T09:33:36.309512+0000","last_fullsized":"2024-09-16T09:33:36.309512+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:34.171139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:26:34.171139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:54.294668+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":252,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.311041+0000","last_change":"2024-09-16T09:32:00.526741+0000","last_active":"2024-09-16T09:33:36.311041+0000","last_peered":"2024-09-16T09:33:36.311041+0000","last_clean":"2024-09-16T09:33:36.311041+0000","last_became_active":"2024-09-16T09:31:09.278618+0000","last_became_peered":"2024-09-16T09:31:09.278618+0000","last_unstale":"2024-09-16T09:33:36.311041+0000","last_undegraded":"2024-09-16T09:33:36.311041+0000","last_fullsized":"2024-09-16T09:33:36.311041+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:00.526674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:32:00.526674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:35.895550+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.0","version":"236'1422","reported_seq":1439,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:37.650984+0000","last_change":"2024-09-16T09:33:30.247937+0000","last_active":"2024-09-16T09:33:37.650984+0000","last_peered":"2024-09-16T09:33:37.650984+0000","last_clean":"2024-09-16T09:33:37.650984+0000","last_became_active":"2024-09-16T09:33:30.247510+0000","last_became_peered":"2024-09-16T09:33:30.247510+0000","last_unstale":"2024-09-16T09:33:37.650984+0000","last_undegraded":"2024-09-16T09:33:37.650984+0000","last_fullsized":"2024-09-16T09:33:37.650984+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:33:24.230486+0000","objects_scrubbed":0,"log_size":1422,"log_dups_size":0,"ondisk_log_size":1422,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:48.926627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":93192192,"num_objects":1422,"num_object_clones":0,"num_object_copies":4266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1422,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1422,"num_write_kb":91008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,2],"acting":[6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.8","version":"73'79","reported_seq":348,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.306548+0000","last_change":"2024-09-16T09:32:07.068820+0000","last_active":"2024-09-16T09:33:36.306548+0000","last_peered":"2024-09-16T09:33:36.306548+0000","last_clean":"2024-09-16T09:33:36.306548+0000","last_became_active":"2024-09-16T09:31:09.406638+0000","last_became_peered":"2024-09-16T09:31:09.406638+0000","last_unstale":"2024-09-16T09:33:36.306548+0000","last_undegraded":"2024-09-16T09:33:36.306548+0000","last_fullsized":"2024-09-16T09:33:36.306548+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:17.482769+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:17.482769+0000","objects_scrubbed":1,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:32.366885+0000","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262688,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":525376,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"66'17","reported_seq":417,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.304540+0000","last_change":"2024-09-16T09:30:04.636115+0000","last_active":"2024-09-16T09:33:36.304540+0000","last_peered":"2024-09-16T09:33:36.304540+0000","last_clean":"2024-09-16T09:33:36.304540+0000","last_became_active":"2024-09-16T09:25:03.713358+0000","last_became_peered":"2024-09-16T09:25:03.713358+0000","last_unstale":"2024-09-16T09:33:36.304540+0000","last_undegraded":"2024-09-16T09:33:36.304540+0000","last_fullsized":"2024-09-16T09:33:36.304540+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'17","last_scrub_stamp":"2024-09-16T09:30:04.636039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:30:04.636039+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":377,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.311585+0000","last_change":"2024-09-16T09:26:04.104188+0000","last_active":"2024-09-16T09:33:36.311585+0000","last_peered":"2024-09-16T09:33:36.311585+0000","last_clean":"2024-09-16T09:33:36.311585+0000","last_became_active":"2024-09-16T09:25:01.703355+0000","last_became_peered":"2024-09-16T09:25:01.703355+0000","last_unstale":"2024-09-16T09:33:36.311585+0000","last_undegraded":"2024-09-16T09:33:36.311585+0000","last_fullsized":"2024-09-16T09:33:36.311585+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:04.104130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:04.104130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:53.552807+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":17,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.303266+0000","last_change":"2024-09-16T09:33:30.252040+0000","last_active":"2024-09-16T09:33:36.303266+0000","last_peered":"2024-09-16T09:33:36.303266+0000","last_clean":"2024-09-16T09:33:36.303266+0000","last_became_active":"2024-09-16T09:33:30.251186+0000","last_became_peered":"2024-09-16T09:33:30.251186+0000","last_unstale":"2024-09-16T09:33:36.303266+0000","last_undegraded":"2024-09-16T09:33:36.303266+0000","last_fullsized":"2024-09-16T09:33:36.303266+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:58.839051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:58.839051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,0],"acting":[4,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":315,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.306871+0000","last_change":"2024-09-16T09:33:21.337606+0000","last_active":"2024-09-16T09:33:36.306871+0000","last_peered":"2024-09-16T09:33:36.306871+0000","last_clean":"2024-09-16T09:33:36.306871+0000","last_became_active":"2024-09-16T09:33:21.337291+0000","last_became_peered":"2024-09-16T09:33:21.337291+0000","last_unstale":"2024-09-16T09:33:36.306871+0000","last_undegraded":"2024-09-16T09:33:36.306871+0000","last_fullsized":"2024-09-16T09:33:36.306871+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:48.259971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:32:48.259971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:12.154060+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4],"acting":[3,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1","version":"236'1304","reported_seq":1329,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:37.638993+0000","last_change":"2024-09-16T09:33:25.371181+0000","last_active":"2024-09-16T09:33:37.638993+0000","last_peered":"2024-09-16T09:33:37.638993+0000","last_clean":"2024-09-16T09:33:37.638993+0000","last_became_active":"2024-09-16T09:33:25.370650+0000","last_became_peered":"2024-09-16T09:33:25.370650+0000","last_unstale":"2024-09-16T09:33:37.638993+0000","last_undegraded":"2024-09-16T09:33:37.638993+0000","last_fullsized":"2024-09-16T09:33:37.638993+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:33:24.230486+0000","objects_scrubbed":0,"log_size":1304,"log_dups_size":0,"ondisk_log_size":1304,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:52.064640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":85458944,"num_objects":1304,"num_object_clones":0,"num_object_copies":3912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1304,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1304,"num_write_kb":83456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.9","version":"73'79","reported_seq":449,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.309080+0000","last_change":"2024-09-16T09:29:19.013948+0000","last_active":"2024-09-16T09:33:36.309080+0000","last_peered":"2024-09-16T09:33:36.309080+0000","last_clean":"2024-09-16T09:33:36.309080+0000","last_became_active":"2024-09-16T09:26:35.126066+0000","last_became_peered":"2024-09-16T09:26:35.126066+0000","last_unstale":"2024-09-16T09:33:36.309080+0000","last_undegraded":"2024-09-16T09:33:36.309080+0000","last_fullsized":"2024-09-16T09:33:36.309080+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:19.013865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:19.013865+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:29.335540+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"66'10","reported_seq":388,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.303484+0000","last_change":"2024-09-16T09:33:24.154567+0000","last_active":"2024-09-16T09:33:36.303484+0000","last_peered":"2024-09-16T09:33:36.303484+0000","last_clean":"2024-09-16T09:33:36.303484+0000","last_became_active":"2024-09-16T09:25:03.707723+0000","last_became_peered":"2024-09-16T09:25:03.707723+0000","last_unstale":"2024-09-16T09:33:36.303484+0000","last_undegraded":"2024-09-16T09:33:36.303484+0000","last_fullsized":"2024-09-16T09:33:36.303484+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-16T09:33:24.154507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:33:24.154507+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:26.179777+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":251,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.311984+0000","last_change":"2024-09-16T09:33:10.528181+0000","last_active":"2024-09-16T09:33:36.311984+0000","last_peered":"2024-09-16T09:33:36.311984+0000","last_clean":"2024-09-16T09:33:36.311984+0000","last_became_active":"2024-09-16T09:31:09.288825+0000","last_became_peered":"2024-09-16T09:31:09.288825+0000","last_unstale":"2024-09-16T09:33:36.311984+0000","last_undegraded":"2024-09-16T09:33:36.311984+0000","last_fullsized":"2024-09-16T09:33:36.311984+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:10.528115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:10.528115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:16.119672+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"228'9","reported_seq":62,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.308400+0000","last_change":"2024-09-16T09:33:21.337623+0000","last_active":"2024-09-16T09:33:36.308400+0000","last_peered":"2024-09-16T09:33:36.308400+0000","last_clean":"2024-09-16T09:33:36.308400+0000","last_became_active":"2024-09-16T09:33:21.337213+0000","last_became_peered":"2024-09-16T09:33:21.337213+0000","last_unstale":"2024-09-16T09:33:36.308400+0000","last_undegraded":"2024-09-16T09:33:36.308400+0000","last_fullsized":"2024-09-16T09:33:36.308400+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:32:42.175528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:32:42.175528+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:08.446570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":389,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.305122+0000","last_change":"2024-09-16T09:32:39.072795+0000","last_active":"2024-09-16T09:33:36.305122+0000","last_peered":"2024-09-16T09:33:36.305122+0000","last_clean":"2024-09-16T09:33:36.305122+0000","last_became_active":"2024-09-16T09:25:07.718958+0000","last_became_peered":"2024-09-16T09:25:07.718958+0000","last_unstale":"2024-09-16T09:33:36.305122+0000","last_undegraded":"2024-09-16T09:33:36.305122+0000","last_fullsized":"2024-09-16T09:33:36.305122+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:39.072735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:32:39.072735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:41.434371+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.6","version":"236'1441","reported_seq":1466,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:38.039535+0000","last_change":"2024-09-16T09:33:25.241567+0000","last_active":"2024-09-16T09:33:38.039535+0000","last_peered":"2024-09-16T09:33:38.039535+0000","last_clean":"2024-09-16T09:33:38.039535+0000","last_became_active":"2024-09-16T09:33:25.241285+0000","last_became_peered":"2024-09-16T09:33:25.241285+0000","last_unstale":"2024-09-16T09:33:38.039535+0000","last_undegraded":"2024-09-16T09:33:38.039535+0000","last_fullsized":"2024-09-16T09:33:38.039535+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:33:24.230486+0000","objects_scrubbed":0,"log_size":1441,"log_dups_size":0,"ondisk_log_size":1441,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:33.124194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":94437376,"num_objects":1441,"num_object_clones":0,"num_object_copies":4323,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1441,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1441,"num_write_kb":92224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.e","version":"73'79","reported_seq":182,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.305029+0000","last_change":"2024-09-16T09:32:43.092027+0000","last_active":"2024-09-16T09:33:36.305029+0000","last_peered":"2024-09-16T09:33:36.305029+0000","last_clean":"2024-09-16T09:33:36.305029+0000","last_became_active":"2024-09-16T09:30:10.705072+0000","last_became_peered":"2024-09-16T09:30:10.705072+0000","last_unstale":"2024-09-16T09:33:36.305029+0000","last_undegraded":"2024-09-16T09:33:36.305029+0000","last_fullsized":"2024-09-16T09:33:36.305029+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:32:43.091969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:32:43.091969+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:45.325012+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,3],"acting":[4,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.b","version":"66'9","reported_seq":405,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.311728+0000","last_change":"2024-09-16T09:31:31.211495+0000","last_active":"2024-09-16T09:33:36.311728+0000","last_peered":"2024-09-16T09:33:36.311728+0000","last_clean":"2024-09-16T09:33:36.311728+0000","last_became_active":"2024-09-16T09:25:03.712230+0000","last_became_peered":"2024-09-16T09:25:03.712230+0000","last_unstale":"2024-09-16T09:33:36.311728+0000","last_undegraded":"2024-09-16T09:33:36.311728+0000","last_fullsized":"2024-09-16T09:33:36.311728+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:31:31.211420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:31:31.211420+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:53.471702+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":390,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.307756+0000","last_change":"2024-09-16T09:31:09.277413+0000","last_active":"2024-09-16T09:33:36.307756+0000","last_peered":"2024-09-16T09:33:36.307756+0000","last_clean":"2024-09-16T09:33:36.307756+0000","last_became_active":"2024-09-16T09:31:09.276516+0000","last_became_peered":"2024-09-16T09:31:09.276516+0000","last_unstale":"2024-09-16T09:33:36.307756+0000","last_undegraded":"2024-09-16T09:33:36.307756+0000","last_fullsized":"2024-09-16T09:33:36.307756+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:26.061106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:26.061106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:56.294039+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":311,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.303257+0000","last_change":"2024-09-16T09:33:21.337626+0000","last_active":"2024-09-16T09:33:36.303257+0000","last_peered":"2024-09-16T09:33:36.303257+0000","last_clean":"2024-09-16T09:33:36.303257+0000","last_became_active":"2024-09-16T09:33:21.337254+0000","last_became_peered":"2024-09-16T09:33:21.337254+0000","last_unstale":"2024-09-16T09:33:36.303257+0000","last_undegraded":"2024-09-16T09:33:36.303257+0000","last_fullsized":"2024-09-16T09:33:36.303257+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:57.220068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:28:57.220068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":33,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3],"acting":[4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":381,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.313037+0000","last_change":"2024-09-16T09:31:27.185957+0000","last_active":"2024-09-16T09:33:36.313037+0000","last_peered":"2024-09-16T09:33:36.313037+0000","last_clean":"2024-09-16T09:33:36.313037+0000","last_became_active":"2024-09-16T09:25:07.718778+0000","last_became_peered":"2024-09-16T09:25:07.718778+0000","last_unstale":"2024-09-16T09:33:36.313037+0000","last_undegraded":"2024-09-16T09:33:36.313037+0000","last_fullsized":"2024-09-16T09:33:36.313037+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:31:27.185801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:31:27.185801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:56.383827+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"236'1370","reported_seq":1394,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:38.038301+0000","last_change":"2024-09-16T09:33:25.371644+0000","last_active":"2024-09-16T09:33:38.038301+0000","last_peered":"2024-09-16T09:33:38.038301+0000","last_clean":"2024-09-16T09:33:38.038301+0000","last_became_active":"2024-09-16T09:33:25.371306+0000","last_became_peered":"2024-09-16T09:33:25.371306+0000","last_unstale":"2024-09-16T09:33:38.038301+0000","last_undegraded":"2024-09-16T09:33:38.038301+0000","last_fullsized":"2024-09-16T09:33:38.038301+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:33:24.230486+0000","objects_scrubbed":0,"log_size":1370,"log_dups_size":0,"ondisk_log_size":1370,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:48.319436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":89784320,"num_objects":1370,"num_object_clones":0,"num_object_copies":4110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1370,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1370,"num_write_kb":87680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.f","version":"73'79","reported_seq":434,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.312414+0000","last_change":"2024-09-16T09:32:44.532983+0000","last_active":"2024-09-16T09:33:36.312414+0000","last_peered":"2024-09-16T09:33:36.312414+0000","last_clean":"2024-09-16T09:33:36.312414+0000","last_became_active":"2024-09-16T09:30:10.693477+0000","last_became_peered":"2024-09-16T09:30:10.693477+0000","last_unstale":"2024-09-16T09:33:36.312414+0000","last_undegraded":"2024-09-16T09:33:36.312414+0000","last_fullsized":"2024-09-16T09:33:36.312414+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:32:44.532922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:32:44.532922+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:02.098440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"66'19","reported_seq":420,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.311346+0000","last_change":"2024-09-16T09:33:22.905808+0000","last_active":"2024-09-16T09:33:36.311346+0000","last_peered":"2024-09-16T09:33:36.311346+0000","last_clean":"2024-09-16T09:33:36.311346+0000","last_became_active":"2024-09-16T09:25:03.712783+0000","last_became_peered":"2024-09-16T09:25:03.712783+0000","last_unstale":"2024-09-16T09:33:36.311346+0000","last_undegraded":"2024-09-16T09:33:36.311346+0000","last_fullsized":"2024-09-16T09:33:36.311346+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-16T09:33:22.905752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:33:22.905752+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:50.313518+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"59'1","reported_seq":390,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.308993+0000","last_change":"2024-09-16T09:29:51.025505+0000","last_active":"2024-09-16T09:33:36.308993+0000","last_peered":"2024-09-16T09:33:36.308993+0000","last_clean":"2024-09-16T09:33:36.308993+0000","last_became_active":"2024-09-16T09:25:01.708069+0000","last_became_peered":"2024-09-16T09:25:01.708069+0000","last_unstale":"2024-09-16T09:33:36.308993+0000","last_undegraded":"2024-09-16T09:33:36.308993+0000","last_fullsized":"2024-09-16T09:33:36.308993+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:29:51.025444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:51.025444+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":377,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.307322+0000","last_change":"2024-09-16T09:33:20.436777+0000","last_active":"2024-09-16T09:33:36.307322+0000","last_peered":"2024-09-16T09:33:36.307322+0000","last_clean":"2024-09-16T09:33:36.307322+0000","last_became_active":"2024-09-16T09:25:05.709264+0000","last_became_peered":"2024-09-16T09:25:05.709264+0000","last_unstale":"2024-09-16T09:33:36.307322+0000","last_undegraded":"2024-09-16T09:33:36.307322+0000","last_fullsized":"2024-09-16T09:33:36.307322+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:20.436728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:20.436728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:34.162333+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":364,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.307194+0000","last_change":"2024-09-16T09:27:25.865107+0000","last_active":"2024-09-16T09:33:36.307194+0000","last_peered":"2024-09-16T09:33:36.307194+0000","last_clean":"2024-09-16T09:33:36.307194+0000","last_became_active":"2024-09-16T09:25:07.718750+0000","last_became_peered":"2024-09-16T09:25:07.718750+0000","last_unstale":"2024-09-16T09:33:36.307194+0000","last_undegraded":"2024-09-16T09:33:36.307194+0000","last_fullsized":"2024-09-16T09:33:36.307194+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:25.865030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:25.865030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:00.396994+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.4","version":"236'1399","reported_seq":1416,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:37.840809+0000","last_change":"2024-09-16T09:33:30.250819+0000","last_active":"2024-09-16T09:33:37.840809+0000","last_peered":"2024-09-16T09:33:37.840809+0000","last_clean":"2024-09-16T09:33:37.840809+0000","last_became_active":"2024-09-16T09:33:30.250449+0000","last_became_peered":"2024-09-16T09:33:30.250449+0000","last_unstale":"2024-09-16T09:33:37.840809+0000","last_undegraded":"2024-09-16T09:33:37.840809+0000","last_fullsized":"2024-09-16T09:33:37.840809+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:33:24.230486+0000","objects_scrubbed":0,"log_size":1399,"log_dups_size":0,"ondisk_log_size":1399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:25.628498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":91684864,"num_objects":1399,"num_object_clones":0,"num_object_copies":4197,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1399,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1399,"num_write_kb":89536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.c","version":"73'79","reported_seq":123,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.307920+0000","last_change":"2024-09-16T09:33:08.949107+0000","last_active":"2024-09-16T09:33:36.307920+0000","last_peered":"2024-09-16T09:33:36.307920+0000","last_clean":"2024-09-16T09:33:36.307920+0000","last_became_active":"2024-09-16T09:31:09.273770+0000","last_became_peered":"2024-09-16T09:31:09.273770+0000","last_unstale":"2024-09-16T09:33:36.307920+0000","last_undegraded":"2024-09-16T09:33:36.307920+0000","last_fullsized":"2024-09-16T09:33:36.307920+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:33:08.949051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:33:08.949051+0000","objects_scrubbed":1,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:16.558324+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":327680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"66'12","reported_seq":442,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.304857+0000","last_change":"2024-09-16T09:31:09.296915+0000","last_active":"2024-09-16T09:33:36.304857+0000","last_peered":"2024-09-16T09:33:36.304857+0000","last_clean":"2024-09-16T09:33:36.304857+0000","last_became_active":"2024-09-16T09:31:09.269047+0000","last_became_peered":"2024-09-16T09:31:09.269047+0000","last_unstale":"2024-09-16T09:33:36.304857+0000","last_undegraded":"2024-09-16T09:33:36.304857+0000","last_fullsized":"2024-09-16T09:33:36.304857+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-16T09:30:21.728668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:30:21.728668+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":384,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.308231+0000","last_change":"2024-09-16T09:29:53.075288+0000","last_active":"2024-09-16T09:33:36.308231+0000","last_peered":"2024-09-16T09:33:36.308231+0000","last_clean":"2024-09-16T09:33:36.308231+0000","last_became_active":"2024-09-16T09:25:01.707528+0000","last_became_peered":"2024-09-16T09:25:01.707528+0000","last_unstale":"2024-09-16T09:33:36.308231+0000","last_undegraded":"2024-09-16T09:33:36.308231+0000","last_fullsized":"2024-09-16T09:33:36.308231+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:53.075232+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:53.075232+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":37,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.311469+0000","last_change":"2024-09-16T09:33:23.643136+0000","last_active":"2024-09-16T09:33:36.311469+0000","last_peered":"2024-09-16T09:33:36.311469+0000","last_clean":"2024-09-16T09:33:36.311469+0000","last_became_active":"2024-09-16T09:33:21.337309+0000","last_became_peered":"2024-09-16T09:33:21.337309+0000","last_unstale":"2024-09-16T09:33:36.311469+0000","last_undegraded":"2024-09-16T09:33:36.311469+0000","last_fullsized":"2024-09-16T09:33:36.311469+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:23.643089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:23.643089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:38.322328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":365,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.305813+0000","last_change":"2024-09-16T09:27:56.059755+0000","last_active":"2024-09-16T09:33:36.305813+0000","last_peered":"2024-09-16T09:33:36.305813+0000","last_clean":"2024-09-16T09:33:36.305813+0000","last_became_active":"2024-09-16T09:25:07.718899+0000","last_became_peered":"2024-09-16T09:25:07.718899+0000","last_unstale":"2024-09-16T09:33:36.305813+0000","last_undegraded":"2024-09-16T09:33:36.305813+0000","last_fullsized":"2024-09-16T09:33:36.305813+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:56.059687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:56.059687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:54.840679+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.5","version":"236'1429","reported_seq":1453,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:38.257594+0000","last_change":"2024-09-16T09:33:25.245337+0000","last_active":"2024-09-16T09:33:38.257594+0000","last_peered":"2024-09-16T09:33:38.257594+0000","last_clean":"2024-09-16T09:33:38.257594+0000","last_became_active":"2024-09-16T09:33:25.245013+0000","last_became_peered":"2024-09-16T09:33:25.245013+0000","last_unstale":"2024-09-16T09:33:38.257594+0000","last_undegraded":"2024-09-16T09:33:38.257594+0000","last_fullsized":"2024-09-16T09:33:38.257594+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:33:24.230486+0000","objects_scrubbed":0,"log_size":1429,"log_dups_size":0,"ondisk_log_size":1429,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:31.104356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":93650944,"num_objects":1429,"num_object_clones":0,"num_object_copies":4287,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1429,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1429,"num_write_kb":91456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.d","version":"73'79","reported_seq":182,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.307913+0000","last_change":"2024-09-16T09:32:44.536868+0000","last_active":"2024-09-16T09:33:36.307913+0000","last_peered":"2024-09-16T09:33:36.307913+0000","last_clean":"2024-09-16T09:33:36.307913+0000","last_became_active":"2024-09-16T09:30:10.709958+0000","last_became_peered":"2024-09-16T09:30:10.709958+0000","last_unstale":"2024-09-16T09:33:36.307913+0000","last_undegraded":"2024-09-16T09:33:36.307913+0000","last_fullsized":"2024-09-16T09:33:36.307913+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:32:44.536799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:32:44.536799+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:04.134689+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"66'15","reported_seq":397,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.306641+0000","last_change":"2024-09-16T09:26:33.138828+0000","last_active":"2024-09-16T09:33:36.306641+0000","last_peered":"2024-09-16T09:33:36.306641+0000","last_clean":"2024-09-16T09:33:36.306641+0000","last_became_active":"2024-09-16T09:25:03.715733+0000","last_became_peered":"2024-09-16T09:25:03.715733+0000","last_unstale":"2024-09-16T09:33:36.306641+0000","last_undegraded":"2024-09-16T09:33:36.306641+0000","last_fullsized":"2024-09-16T09:33:36.306641+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-16T09:26:33.138603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:26:33.138603+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:55.316881+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":396,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.308038+0000","last_change":"2024-09-16T09:32:49.441194+0000","last_active":"2024-09-16T09:33:36.308038+0000","last_peered":"2024-09-16T09:33:36.308038+0000","last_clean":"2024-09-16T09:33:36.308038+0000","last_became_active":"2024-09-16T09:25:01.707372+0000","last_became_peered":"2024-09-16T09:25:01.707372+0000","last_unstale":"2024-09-16T09:33:36.308038+0000","last_undegraded":"2024-09-16T09:33:36.308038+0000","last_fullsized":"2024-09-16T09:33:36.308038+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-16T09:32:49.441122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:32:49.441122+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:17.431142+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"66'8","reported_seq":496,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.308972+0000","last_change":"2024-09-16T09:30:05.138362+0000","last_active":"2024-09-16T09:33:36.308972+0000","last_peered":"2024-09-16T09:33:36.308972+0000","last_clean":"2024-09-16T09:33:36.308972+0000","last_became_active":"2024-09-16T09:25:05.716189+0000","last_became_peered":"2024-09-16T09:25:05.716189+0000","last_unstale":"2024-09-16T09:33:36.308972+0000","last_undegraded":"2024-09-16T09:33:36.308972+0000","last_fullsized":"2024-09-16T09:33:36.308972+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:30:05.138288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:30:05.138288+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":364,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.309785+0000","last_change":"2024-09-16T09:27:20.219376+0000","last_active":"2024-09-16T09:33:36.309785+0000","last_peered":"2024-09-16T09:33:36.309785+0000","last_clean":"2024-09-16T09:33:36.309785+0000","last_became_active":"2024-09-16T09:25:07.717213+0000","last_became_peered":"2024-09-16T09:25:07.717213+0000","last_unstale":"2024-09-16T09:33:36.309785+0000","last_undegraded":"2024-09-16T09:33:36.309785+0000","last_fullsized":"2024-09-16T09:33:36.309785+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:20.219333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:20.219333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:03.172918+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.12","version":"73'79","reported_seq":182,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.310402+0000","last_change":"2024-09-16T09:32:45.399435+0000","last_active":"2024-09-16T09:33:36.310402+0000","last_peered":"2024-09-16T09:33:36.310402+0000","last_clean":"2024-09-16T09:33:36.310402+0000","last_became_active":"2024-09-16T09:30:10.731413+0000","last_became_peered":"2024-09-16T09:30:10.731413+0000","last_unstale":"2024-09-16T09:33:36.310402+0000","last_undegraded":"2024-09-16T09:33:36.310402+0000","last_fullsized":"2024-09-16T09:33:36.310402+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:32:45.399361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:32:45.399361+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:01.532310+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":385,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.311197+0000","last_change":"2024-09-16T09:33:14.845398+0000","last_active":"2024-09-16T09:33:36.311197+0000","last_peered":"2024-09-16T09:33:36.311197+0000","last_clean":"2024-09-16T09:33:36.311197+0000","last_became_active":"2024-09-16T09:25:01.703202+0000","last_became_peered":"2024-09-16T09:25:01.703202+0000","last_unstale":"2024-09-16T09:33:36.311197+0000","last_undegraded":"2024-09-16T09:33:36.311197+0000","last_fullsized":"2024-09-16T09:33:36.311197+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:14.845355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:14.845355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:44.659429+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"66'6","reported_seq":127,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.307691+0000","last_change":"2024-09-16T09:32:47.851855+0000","last_active":"2024-09-16T09:33:36.307691+0000","last_peered":"2024-09-16T09:33:36.307691+0000","last_clean":"2024-09-16T09:33:36.307691+0000","last_became_active":"2024-09-16T09:31:09.270685+0000","last_became_peered":"2024-09-16T09:31:09.270685+0000","last_unstale":"2024-09-16T09:33:36.307691+0000","last_undegraded":"2024-09-16T09:33:36.307691+0000","last_fullsized":"2024-09-16T09:33:36.307691+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'6","last_scrub_stamp":"2024-09-16T09:32:47.851772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:32:47.851772+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:58.501874+0000","scrub_duration":28,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":8,"num_read_kb":5,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":376,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.309070+0000","last_change":"2024-09-16T09:29:31.496374+0000","last_active":"2024-09-16T09:33:36.309070+0000","last_peered":"2024-09-16T09:33:36.309070+0000","last_clean":"2024-09-16T09:33:36.309070+0000","last_became_active":"2024-09-16T09:25:05.715616+0000","last_became_peered":"2024-09-16T09:25:05.715616+0000","last_unstale":"2024-09-16T09:33:36.309070+0000","last_undegraded":"2024-09-16T09:33:36.309070+0000","last_fullsized":"2024-09-16T09:33:36.309070+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:31.496304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:31.496304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:35.386546+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":364,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.308011+0000","last_change":"2024-09-16T09:26:28.733381+0000","last_active":"2024-09-16T09:33:36.308011+0000","last_peered":"2024-09-16T09:33:36.308011+0000","last_clean":"2024-09-16T09:33:36.308011+0000","last_became_active":"2024-09-16T09:25:07.721059+0000","last_became_peered":"2024-09-16T09:25:07.721059+0000","last_unstale":"2024-09-16T09:33:36.308011+0000","last_undegraded":"2024-09-16T09:33:36.308011+0000","last_fullsized":"2024-09-16T09:33:36.308011+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:28.733324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:28.733324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:52.397089+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.13","version":"73'79","reported_seq":268,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.306000+0000","last_change":"2024-09-16T09:33:30.248826+0000","last_active":"2024-09-16T09:33:36.306000+0000","last_peered":"2024-09-16T09:33:36.306000+0000","last_clean":"2024-09-16T09:33:36.306000+0000","last_became_active":"2024-09-16T09:33:30.248497+0000","last_became_peered":"2024-09-16T09:33:30.248497+0000","last_unstale":"2024-09-16T09:33:36.306000+0000","last_undegraded":"2024-09-16T09:33:36.306000+0000","last_fullsized":"2024-09-16T09:33:36.306000+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:30:21.714369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:30:21.714369+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:34.907627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"66'9","reported_seq":447,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.310271+0000","last_change":"2024-09-16T09:32:01.319481+0000","last_active":"2024-09-16T09:33:36.310271+0000","last_peered":"2024-09-16T09:33:36.310271+0000","last_clean":"2024-09-16T09:33:36.310271+0000","last_became_active":"2024-09-16T09:31:09.273498+0000","last_became_peered":"2024-09-16T09:31:09.273498+0000","last_unstale":"2024-09-16T09:33:36.310271+0000","last_undegraded":"2024-09-16T09:33:36.310271+0000","last_fullsized":"2024-09-16T09:33:36.310271+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:32:01.319390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:32:01.319390+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":392,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.309322+0000","last_change":"2024-09-16T09:33:13.312434+0000","last_active":"2024-09-16T09:33:36.309322+0000","last_peered":"2024-09-16T09:33:36.309322+0000","last_clean":"2024-09-16T09:33:36.309322+0000","last_became_active":"2024-09-16T09:25:01.706491+0000","last_became_peered":"2024-09-16T09:25:01.706491+0000","last_unstale":"2024-09-16T09:33:36.309322+0000","last_undegraded":"2024-09-16T09:33:36.309322+0000","last_fullsized":"2024-09-16T09:33:36.309322+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:13.312344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:13.312344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:40.656583+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":142,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.308248+0000","last_change":"2024-09-16T09:33:30.249293+0000","last_active":"2024-09-16T09:33:36.308248+0000","last_peered":"2024-09-16T09:33:36.308248+0000","last_clean":"2024-09-16T09:33:36.308248+0000","last_became_active":"2024-09-16T09:33:30.248992+0000","last_became_peered":"2024-09-16T09:33:30.248992+0000","last_unstale":"2024-09-16T09:33:36.308248+0000","last_undegraded":"2024-09-16T09:33:36.308248+0000","last_fullsized":"2024-09-16T09:33:36.308248+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:53.810580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:28:53.810580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:35.392050+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":373,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.308639+0000","last_change":"2024-09-16T09:33:19.444075+0000","last_active":"2024-09-16T09:33:36.308639+0000","last_peered":"2024-09-16T09:33:36.308639+0000","last_clean":"2024-09-16T09:33:36.308639+0000","last_became_active":"2024-09-16T09:25:07.722050+0000","last_became_peered":"2024-09-16T09:25:07.722050+0000","last_unstale":"2024-09-16T09:33:36.308639+0000","last_undegraded":"2024-09-16T09:33:36.308639+0000","last_fullsized":"2024-09-16T09:33:36.308639+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:19.443984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:33:19.443984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:26.906274+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.10","version":"73'79","reported_seq":182,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.306510+0000","last_change":"2024-09-16T09:32:44.405696+0000","last_active":"2024-09-16T09:33:36.306510+0000","last_peered":"2024-09-16T09:33:36.306510+0000","last_clean":"2024-09-16T09:33:36.306510+0000","last_became_active":"2024-09-16T09:30:10.692770+0000","last_became_peered":"2024-09-16T09:30:10.692770+0000","last_unstale":"2024-09-16T09:33:36.306510+0000","last_undegraded":"2024-09-16T09:33:36.306510+0000","last_fullsized":"2024-09-16T09:33:36.306510+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:32:44.405640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:32:44.405640+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:52.861751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"66'9","reported_seq":396,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.309543+0000","last_change":"2024-09-16T09:29:32.485165+0000","last_active":"2024-09-16T09:33:36.309543+0000","last_peered":"2024-09-16T09:33:36.309543+0000","last_clean":"2024-09-16T09:33:36.309543+0000","last_became_active":"2024-09-16T09:25:03.717310+0000","last_became_peered":"2024-09-16T09:25:03.717310+0000","last_unstale":"2024-09-16T09:33:36.309543+0000","last_undegraded":"2024-09-16T09:33:36.309543+0000","last_fullsized":"2024-09-16T09:33:36.309543+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:29:32.485084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:29:32.485084+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:28.458607+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":415,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.309926+0000","last_change":"2024-09-16T09:33:20.639405+0000","last_active":"2024-09-16T09:33:36.309926+0000","last_peered":"2024-09-16T09:33:36.309926+0000","last_clean":"2024-09-16T09:33:36.309926+0000","last_became_active":"2024-09-16T09:31:09.272848+0000","last_became_peered":"2024-09-16T09:31:09.272848+0000","last_unstale":"2024-09-16T09:33:36.309926+0000","last_undegraded":"2024-09-16T09:33:36.309926+0000","last_fullsized":"2024-09-16T09:33:36.309926+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:20.639350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:20.639350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:37.405814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"187'11","reported_seq":334,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.307996+0000","last_change":"2024-09-16T09:33:20.296801+0000","last_active":"2024-09-16T09:33:36.307996+0000","last_peered":"2024-09-16T09:33:36.307996+0000","last_clean":"2024-09-16T09:33:36.307996+0000","last_became_active":"2024-09-16T09:31:09.277688+0000","last_became_peered":"2024-09-16T09:31:09.277688+0000","last_unstale":"2024-09-16T09:33:36.307996+0000","last_undegraded":"2024-09-16T09:33:36.307996+0000","last_fullsized":"2024-09-16T09:33:36.307996+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"187'11","last_scrub_stamp":"2024-09-16T09:33:20.296746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:20.296746+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:20.919886+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":381,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.304759+0000","last_change":"2024-09-16T09:29:49.474440+0000","last_active":"2024-09-16T09:33:36.304759+0000","last_peered":"2024-09-16T09:33:36.304759+0000","last_clean":"2024-09-16T09:33:36.304759+0000","last_became_active":"2024-09-16T09:25:07.718810+0000","last_became_peered":"2024-09-16T09:25:07.718810+0000","last_unstale":"2024-09-16T09:33:36.304759+0000","last_undegraded":"2024-09-16T09:33:36.304759+0000","last_fullsized":"2024-09-16T09:33:36.304759+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:49.474391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:49.474391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:56.215068+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.11","version":"73'79","reported_seq":268,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.313226+0000","last_change":"2024-09-16T09:32:38.401888+0000","last_active":"2024-09-16T09:33:36.313226+0000","last_peered":"2024-09-16T09:33:36.313226+0000","last_clean":"2024-09-16T09:33:36.313226+0000","last_became_active":"2024-09-16T09:31:09.272208+0000","last_became_peered":"2024-09-16T09:31:09.272208+0000","last_unstale":"2024-09-16T09:33:36.313226+0000","last_undegraded":"2024-09-16T09:33:36.313226+0000","last_fullsized":"2024-09-16T09:33:36.313226+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:32:38.401720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:32:38.401720+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:46.010237+0000","scrub_duration":32,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.14","version":"66'10","reported_seq":52,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.308126+0000","last_change":"2024-09-16T09:33:30.250101+0000","last_active":"2024-09-16T09:33:36.308126+0000","last_peered":"2024-09-16T09:33:36.308126+0000","last_clean":"2024-09-16T09:33:36.308126+0000","last_became_active":"2024-09-16T09:33:30.249769+0000","last_became_peered":"2024-09-16T09:33:30.249769+0000","last_unstale":"2024-09-16T09:33:36.308126+0000","last_undegraded":"2024-09-16T09:33:36.308126+0000","last_fullsized":"2024-09-16T09:33:36.308126+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-16T09:28:56.825476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:56.825476+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:33.375036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":14,"num_read_kb":9,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":384,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.308942+0000","last_change":"2024-09-16T09:29:56.021413+0000","last_active":"2024-09-16T09:33:36.308942+0000","last_peered":"2024-09-16T09:33:36.308942+0000","last_clean":"2024-09-16T09:33:36.308942+0000","last_became_active":"2024-09-16T09:25:01.707513+0000","last_became_peered":"2024-09-16T09:25:01.707513+0000","last_unstale":"2024-09-16T09:33:36.308942+0000","last_undegraded":"2024-09-16T09:33:36.308942+0000","last_fullsized":"2024-09-16T09:33:36.308942+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:56.021340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:56.021340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"66'8","reported_seq":496,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.309337+0000","last_change":"2024-09-16T09:29:48.485781+0000","last_active":"2024-09-16T09:33:36.309337+0000","last_peered":"2024-09-16T09:33:36.309337+0000","last_clean":"2024-09-16T09:33:36.309337+0000","last_became_active":"2024-09-16T09:25:05.713833+0000","last_became_peered":"2024-09-16T09:25:05.713833+0000","last_unstale":"2024-09-16T09:33:36.309337+0000","last_undegraded":"2024-09-16T09:33:36.309337+0000","last_fullsized":"2024-09-16T09:33:36.309337+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:29:48.485727+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:48.485727+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":395,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.310174+0000","last_change":"2024-09-16T09:32:00.343429+0000","last_active":"2024-09-16T09:33:36.310174+0000","last_peered":"2024-09-16T09:33:36.310174+0000","last_clean":"2024-09-16T09:33:36.310174+0000","last_became_active":"2024-09-16T09:31:09.275168+0000","last_became_peered":"2024-09-16T09:31:09.275168+0000","last_unstale":"2024-09-16T09:33:36.310174+0000","last_undegraded":"2024-09-16T09:33:36.310174+0000","last_fullsized":"2024-09-16T09:33:36.310174+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:00.343356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:32:00.343356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"66'11","reported_seq":411,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.303801+0000","last_change":"2024-09-16T09:33:19.084118+0000","last_active":"2024-09-16T09:33:36.303801+0000","last_peered":"2024-09-16T09:33:36.303801+0000","last_clean":"2024-09-16T09:33:36.303801+0000","last_became_active":"2024-09-16T09:26:33.291265+0000","last_became_peered":"2024-09-16T09:26:33.291265+0000","last_unstale":"2024-09-16T09:33:36.303801+0000","last_undegraded":"2024-09-16T09:33:36.303801+0000","last_fullsized":"2024-09-16T09:33:36.303801+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-16T09:33:19.084063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:33:19.084063+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:32.590346+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":393,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.303792+0000","last_change":"2024-09-16T09:29:51.517112+0000","last_active":"2024-09-16T09:33:36.303792+0000","last_peered":"2024-09-16T09:33:36.303792+0000","last_clean":"2024-09-16T09:33:36.303792+0000","last_became_active":"2024-09-16T09:25:01.705158+0000","last_became_peered":"2024-09-16T09:25:01.705158+0000","last_unstale":"2024-09-16T09:33:36.303792+0000","last_undegraded":"2024-09-16T09:33:36.303792+0000","last_fullsized":"2024-09-16T09:33:36.303792+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:51.517054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:51.517054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":307,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.310160+0000","last_change":"2024-09-16T09:33:30.243564+0000","last_active":"2024-09-16T09:33:36.310160+0000","last_peered":"2024-09-16T09:33:36.310160+0000","last_clean":"2024-09-16T09:33:36.310160+0000","last_became_active":"2024-09-16T09:33:30.242922+0000","last_became_peered":"2024-09-16T09:33:30.242922+0000","last_unstale":"2024-09-16T09:33:36.310160+0000","last_undegraded":"2024-09-16T09:33:36.310160+0000","last_fullsized":"2024-09-16T09:33:36.310160+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:08.394830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:08.394830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:34.938859+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":228,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.314164+0000","last_change":"2024-09-16T09:31:09.277239+0000","last_active":"2024-09-16T09:33:36.314164+0000","last_peered":"2024-09-16T09:33:36.314164+0000","last_clean":"2024-09-16T09:33:36.314164+0000","last_became_active":"2024-09-16T09:31:09.276042+0000","last_became_peered":"2024-09-16T09:31:09.276042+0000","last_unstale":"2024-09-16T09:33:36.314164+0000","last_undegraded":"2024-09-16T09:33:36.314164+0000","last_fullsized":"2024-09-16T09:33:36.314164+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:48.147759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:48.147759+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:32.663605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"66'9","reported_seq":53,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.311465+0000","last_change":"2024-09-16T09:33:30.248198+0000","last_active":"2024-09-16T09:33:36.311465+0000","last_peered":"2024-09-16T09:33:36.311465+0000","last_clean":"2024-09-16T09:33:36.311465+0000","last_became_active":"2024-09-16T09:33:30.247738+0000","last_became_peered":"2024-09-16T09:33:30.247738+0000","last_unstale":"2024-09-16T09:33:36.311465+0000","last_undegraded":"2024-09-16T09:33:36.311465+0000","last_fullsized":"2024-09-16T09:33:36.311465+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:32:47.247071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:32:47.247071+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:12.503023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":10,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,2],"acting":[6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":384,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.308829+0000","last_change":"2024-09-16T09:32:47.502667+0000","last_active":"2024-09-16T09:33:36.308829+0000","last_peered":"2024-09-16T09:33:36.308829+0000","last_clean":"2024-09-16T09:33:36.308829+0000","last_became_active":"2024-09-16T09:25:01.706762+0000","last_became_peered":"2024-09-16T09:25:01.706762+0000","last_unstale":"2024-09-16T09:33:36.308829+0000","last_undegraded":"2024-09-16T09:33:36.308829+0000","last_fullsized":"2024-09-16T09:33:36.308829+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:47.502598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:32:47.502598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:03.669451+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":213,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.311148+0000","last_change":"2024-09-16T09:28:52.983144+0000","last_active":"2024-09-16T09:33:36.311148+0000","last_peered":"2024-09-16T09:33:36.311148+0000","last_clean":"2024-09-16T09:33:36.311148+0000","last_became_active":"2024-09-16T09:28:52.982865+0000","last_became_peered":"2024-09-16T09:28:52.982865+0000","last_unstale":"2024-09-16T09:33:36.311148+0000","last_undegraded":"2024-09-16T09:33:36.311148+0000","last_fullsized":"2024-09-16T09:33:36.311148+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:44.241942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:44.241942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:31.671032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":373,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.311318+0000","last_change":"2024-09-16T09:28:52.552567+0000","last_active":"2024-09-16T09:33:36.311318+0000","last_peered":"2024-09-16T09:33:36.311318+0000","last_clean":"2024-09-16T09:33:36.311318+0000","last_became_active":"2024-09-16T09:25:07.715301+0000","last_became_peered":"2024-09-16T09:25:07.715301+0000","last_unstale":"2024-09-16T09:33:36.311318+0000","last_undegraded":"2024-09-16T09:33:36.311318+0000","last_fullsized":"2024-09-16T09:33:36.311318+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:52.552495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:28:52.552495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:50.499212+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.14","version":"73'79","reported_seq":182,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.308477+0000","last_change":"2024-09-16T09:32:44.163264+0000","last_active":"2024-09-16T09:33:36.308477+0000","last_peered":"2024-09-16T09:33:36.308477+0000","last_clean":"2024-09-16T09:33:36.308477+0000","last_became_active":"2024-09-16T09:30:10.700866+0000","last_became_peered":"2024-09-16T09:30:10.700866+0000","last_unstale":"2024-09-16T09:33:36.308477+0000","last_undegraded":"2024-09-16T09:33:36.308477+0000","last_fullsized":"2024-09-16T09:33:36.308477+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:32:44.163191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:32:44.163191+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:13.888037+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"66'11","reported_seq":260,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.309189+0000","last_change":"2024-09-16T09:28:55.643505+0000","last_active":"2024-09-16T09:33:36.309189+0000","last_peered":"2024-09-16T09:33:36.309189+0000","last_clean":"2024-09-16T09:33:36.309189+0000","last_became_active":"2024-09-16T09:28:52.971132+0000","last_became_peered":"2024-09-16T09:28:52.971132+0000","last_unstale":"2024-09-16T09:33:36.309189+0000","last_undegraded":"2024-09-16T09:33:36.309189+0000","last_fullsized":"2024-09-16T09:33:36.309189+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-16T09:28:55.643448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:55.643448+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:27.339945+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":19,"num_read_kb":12,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":392,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.309204+0000","last_change":"2024-09-16T09:33:18.297533+0000","last_active":"2024-09-16T09:33:36.309204+0000","last_peered":"2024-09-16T09:33:36.309204+0000","last_clean":"2024-09-16T09:33:36.309204+0000","last_became_active":"2024-09-16T09:25:01.700733+0000","last_became_peered":"2024-09-16T09:25:01.700733+0000","last_unstale":"2024-09-16T09:33:36.309204+0000","last_undegraded":"2024-09-16T09:33:36.309204+0000","last_fullsized":"2024-09-16T09:33:36.309204+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:18.297451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:18.297451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:44.022852+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":376,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.306271+0000","last_change":"2024-09-16T09:29:25.017736+0000","last_active":"2024-09-16T09:33:36.306271+0000","last_peered":"2024-09-16T09:33:36.306271+0000","last_clean":"2024-09-16T09:33:36.306271+0000","last_became_active":"2024-09-16T09:25:05.713479+0000","last_became_peered":"2024-09-16T09:25:05.713479+0000","last_unstale":"2024-09-16T09:33:36.306271+0000","last_undegraded":"2024-09-16T09:33:36.306271+0000","last_fullsized":"2024-09-16T09:33:36.306271+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:25.017610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:25.017610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":236,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.312155+0000","last_change":"2024-09-16T09:31:09.275121+0000","last_active":"2024-09-16T09:33:36.312155+0000","last_peered":"2024-09-16T09:33:36.312155+0000","last_clean":"2024-09-16T09:33:36.312155+0000","last_became_active":"2024-09-16T09:31:09.274797+0000","last_became_peered":"2024-09-16T09:31:09.274797+0000","last_unstale":"2024-09-16T09:33:36.312155+0000","last_undegraded":"2024-09-16T09:33:36.312155+0000","last_fullsized":"2024-09-16T09:33:36.312155+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:54.589258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:28:54.589258+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:51.457590+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"66'4","reported_seq":125,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.305341+0000","last_change":"2024-09-16T09:32:48.847615+0000","last_active":"2024-09-16T09:33:36.305341+0000","last_peered":"2024-09-16T09:33:36.305341+0000","last_clean":"2024-09-16T09:33:36.305341+0000","last_became_active":"2024-09-16T09:31:09.273682+0000","last_became_peered":"2024-09-16T09:31:09.273682+0000","last_unstale":"2024-09-16T09:33:36.305341+0000","last_undegraded":"2024-09-16T09:33:36.305341+0000","last_fullsized":"2024-09-16T09:33:36.305341+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'4","last_scrub_stamp":"2024-09-16T09:32:48.847542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:32:48.847542+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:52.504702+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":415,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.310622+0000","last_change":"2024-09-16T09:33:18.641115+0000","last_active":"2024-09-16T09:33:36.310622+0000","last_peered":"2024-09-16T09:33:36.310622+0000","last_clean":"2024-09-16T09:33:36.310622+0000","last_became_active":"2024-09-16T09:31:09.274719+0000","last_became_peered":"2024-09-16T09:31:09.274719+0000","last_unstale":"2024-09-16T09:33:36.310622+0000","last_undegraded":"2024-09-16T09:33:36.310622+0000","last_fullsized":"2024-09-16T09:33:36.310622+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:18.641069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:18.641069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:36.841748+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":393,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.310791+0000","last_change":"2024-09-16T09:32:07.533575+0000","last_active":"2024-09-16T09:33:36.310791+0000","last_peered":"2024-09-16T09:33:36.310791+0000","last_clean":"2024-09-16T09:33:36.310791+0000","last_became_active":"2024-09-16T09:25:05.712545+0000","last_became_peered":"2024-09-16T09:25:05.712545+0000","last_unstale":"2024-09-16T09:33:36.310791+0000","last_undegraded":"2024-09-16T09:33:36.310791+0000","last_fullsized":"2024-09-16T09:33:36.310791+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:07.533529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:32:07.533529+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":372,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.308837+0000","last_change":"2024-09-16T09:29:27.012810+0000","last_active":"2024-09-16T09:33:36.308837+0000","last_peered":"2024-09-16T09:33:36.308837+0000","last_clean":"2024-09-16T09:33:36.308837+0000","last_became_active":"2024-09-16T09:25:07.720117+0000","last_became_peered":"2024-09-16T09:25:07.720117+0000","last_unstale":"2024-09-16T09:33:36.308837+0000","last_undegraded":"2024-09-16T09:33:36.308837+0000","last_fullsized":"2024-09-16T09:33:36.308837+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:27.012756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:27.012756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":17,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.309299+0000","last_change":"2024-09-16T09:33:30.248335+0000","last_active":"2024-09-16T09:33:36.309299+0000","last_peered":"2024-09-16T09:33:36.309299+0000","last_clean":"2024-09-16T09:33:36.309299+0000","last_became_active":"2024-09-16T09:33:30.248036+0000","last_became_peered":"2024-09-16T09:33:30.248036+0000","last_unstale":"2024-09-16T09:33:36.309299+0000","last_undegraded":"2024-09-16T09:33:36.309299+0000","last_fullsized":"2024-09-16T09:33:36.309299+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:18.339152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:33:18.339152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:27.531804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":242,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.312737+0000","last_change":"2024-09-16T09:32:40.411546+0000","last_active":"2024-09-16T09:33:36.312737+0000","last_peered":"2024-09-16T09:33:36.312737+0000","last_clean":"2024-09-16T09:33:36.312737+0000","last_became_active":"2024-09-16T09:31:09.271662+0000","last_became_peered":"2024-09-16T09:31:09.271662+0000","last_unstale":"2024-09-16T09:33:36.312737+0000","last_undegraded":"2024-09-16T09:33:36.312737+0000","last_fullsized":"2024-09-16T09:33:36.312737+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:40.411487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:32:40.411487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:51.366030+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"66'11","reported_seq":409,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.312479+0000","last_change":"2024-09-16T09:33:19.843216+0000","last_active":"2024-09-16T09:33:36.312479+0000","last_peered":"2024-09-16T09:33:36.312479+0000","last_clean":"2024-09-16T09:33:36.312479+0000","last_became_active":"2024-09-16T09:25:03.707769+0000","last_became_peered":"2024-09-16T09:25:03.707769+0000","last_unstale":"2024-09-16T09:33:36.312479+0000","last_undegraded":"2024-09-16T09:33:36.312479+0000","last_fullsized":"2024-09-16T09:33:36.312479+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-16T09:33:19.843154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:33:19.843154+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:46.597409+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":378,"reported_epoch":236,"state":"active+clean","last_fresh":"2024-09-16T09:33:36.310084+0000","last_change":"2024-09-16T09:32:31.325543+0000","last_active":"2024-09-16T09:33:36.310084+0000","last_peered":"2024-09-16T09:33:36.310084+0000","last_clean":"2024-09-16T09:33:36.310084+0000","last_became_active":"2024-09-16T09:25:05.711952+0000","last_became_peered":"2024-09-16T09:25:05.711952+0000","last_unstale":"2024-09-16T09:33:36.310084+0000","last_undegraded":"2024-09-16T09:33:36.310084+0000","last_fullsized":"2024-09-16T09:33:36.310084+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:31.325346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:32:31.325346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:33:54.289339+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":16,"stat_sum":{"num_bytes":1453981696,"num_objects":22186,"num_object_clones":0,"num_object_copies":66558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22186,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22186,"num_write_kb":1419904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4364763136,"data_stored":4364763136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22186,"ondisk_log_size":22186,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":688,"num_read_kb":445,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":102,"num_bytes_recovered":2366,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":395,"ondisk_log_size":395,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1986,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":510,"num_read_kb":505,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":21,"stat_sum":{"num_bytes":608258,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":6,"num_write":3,"num_write_kb":49,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":870946,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1659,"ondisk_log_size":1659,"up":63,"acting":63,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266796,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":998052,"kb_used_data":736400,"kb_used_omap":1,"kb_used_meta":261630,"kb_avail":92747100,"statfs":{"total":95995035648,"available":94973030400,"internally_reserved":0,"allocated":754073600,"data_stored":753650936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":267909578},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495729,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":493836,"kb_used_data":237752,"kb_used_omap":1,"kb_used_meta":256062,"kb_avail":93251316,"statfs":{"total":95995035648,"available":95489347584,"internally_reserved":0,"allocated":243458048,"data_stored":243058862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":262207942},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724664,"num_pgs":65,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1056532,"kb_used_data":746308,"kb_used_omap":1,"kb_used_meta":310206,"kb_avail":92688620,"statfs":{"total":95995035648,"available":94913146880,"internally_reserved":0,"allocated":764219392,"data_stored":763808931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":317651398},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953597,"num_pgs":69,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":717544,"kb_used_data":455892,"kb_used_omap":1,"kb_used_meta":261630,"kb_avail":93027608,"statfs":{"total":95995035648,"available":95260270592,"internally_reserved":0,"allocated":466833408,"data_stored":466406712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":267909574},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149827,"num_pgs":73,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1084468,"kb_used_data":710368,"kb_used_omap":1,"kb_used_meta":374078,"kb_avail":92660684,"statfs":{"total":95995035648,"available":94884540416,"internally_reserved":0,"allocated":727416832,"data_stored":726978678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":383056327},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411464,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":565712,"kb_used_data":255484,"kb_used_omap":1,"kb_used_meta":310206,"kb_avail":93179440,"statfs":{"total":95995035648,"available":95415746560,"internally_reserved":0,"allocated":261615616,"data_stored":261212810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":317651397},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574991,"num_pgs":69,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1065088,"kb_used_data":749292,"kb_used_omap":1,"kb_used_meta":315774,"kb_avail":92680064,"statfs":{"total":95995035648,"available":94904385536,"internally_reserved":0,"allocated":767275008,"data_stored":766823606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":323353029},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738517,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":862824,"kb_used_data":534548,"kb_used_omap":1,"kb_used_meta":328254,"kb_avail":92882328,"statfs":{"total":95995035648,"available":95111503872,"internally_reserved":0,"allocated":547377152,"data_stored":546950658,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":336132549},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":374472704,"data_stored":374472704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":830734336,"data_stored":830734336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":278265856,"data_stored":278265856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":734265344,"data_stored":734265344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":271646720,"data_stored":271646720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":828506112,"data_stored":828506112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":265224192,"data_stored":265224192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":781647872,"data_stored":781647872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T09:33:44.874 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2024-09-16T09:33:44.875 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd pool set unique_pool_2 pg_num 26 2024-09-16T09:33:44.888 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:33:44.949 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:33:44.969 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:33:44.986 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:44.987 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:44.988 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:33:45.089 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:33:45.122 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:45.122 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:45.123 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:33:45.223 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:33:45.271 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:45.271 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:45.271 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:33:45.372 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:33:45.408 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:45.409 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:45.409 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:33:45.510 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:33:45.546 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:45.547 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:45.547 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:33:45.630 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:45 smithi070 ceph-mon[30388]: pgmap v414: 168 pgs: 168 active+clean; 2.0 GiB data, 8.7 GiB used, 707 GiB / 715 GiB avail; 1.4 KiB/s rd, 217 MiB/s wr, 3.48k op/s 2024-09-16T09:33:45.630 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:45 smithi070 ceph-mon[30388]: 3.6 scrub starts 2024-09-16T09:33:45.630 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:45 smithi070 ceph-mon[30388]: 3.6 scrub ok 2024-09-16T09:33:45.630 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:45 smithi070 ceph-mon[30388]: 5.c scrub starts 2024-09-16T09:33:45.630 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:45 smithi070 ceph-mon[30388]: 5.c scrub ok 2024-09-16T09:33:45.630 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:45 smithi070 ceph-mon[40691]: pgmap v414: 168 pgs: 168 active+clean; 2.0 GiB data, 8.7 GiB used, 707 GiB / 715 GiB avail; 1.4 KiB/s rd, 217 MiB/s wr, 3.48k op/s 2024-09-16T09:33:45.630 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:45 smithi070 ceph-mon[40691]: 3.6 scrub starts 2024-09-16T09:33:45.630 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:45 smithi070 ceph-mon[40691]: 3.6 scrub ok 2024-09-16T09:33:45.630 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:45 smithi070 ceph-mon[40691]: 5.c scrub starts 2024-09-16T09:33:45.630 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:45 smithi070 ceph-mon[40691]: 5.c scrub ok 2024-09-16T09:33:45.649 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:33:45.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:45 smithi084 ceph-mon[32629]: pgmap v414: 168 pgs: 168 active+clean; 2.0 GiB data, 8.7 GiB used, 707 GiB / 715 GiB avail; 1.4 KiB/s rd, 217 MiB/s wr, 3.48k op/s 2024-09-16T09:33:45.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:45 smithi084 ceph-mon[32629]: 3.6 scrub starts 2024-09-16T09:33:45.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:45 smithi084 ceph-mon[32629]: 3.6 scrub ok 2024-09-16T09:33:45.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:45 smithi084 ceph-mon[32629]: 5.c scrub starts 2024-09-16T09:33:45.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:45 smithi084 ceph-mon[32629]: 5.c scrub ok 2024-09-16T09:33:45.681 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:45.682 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:45.682 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:33:45.707 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 15 16 39163 39147 163.092 122.625 0.0075451 0.00612622 2024-09-16T09:33:45.783 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:33:45.827 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:45.828 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:45.828 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:33:45.929 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:33:45.961 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:45.962 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:45.962 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:33:46.033 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T09:33:46.034 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [ 2024-09-16T09:33:46.034 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:33:46.034 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.15588.0:39737 9.2 e239/233 9:4342143f:::benchmark_data_smithi070_137000_object39736:head v 239'2467, mlcod=239'2463)", 2024-09-16T09:33:46.034 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:33:46.028734+0000", 2024-09-16T09:33:46.034 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.002410499, 2024-09-16T09:33:46.034 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0024394270000000001, 2024-09-16T09:33:46.034 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:33:46.034 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:33:46.034 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T09:33:46.034 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:33:46.034 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:33:46.034 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:33:46.034 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:46.028734+0000", 2024-09-16T09:33:46.034 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:33:46.035 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:33:46.035 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:33:46.035 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:33:46.035 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:46.028734+0000", 2024-09-16T09:33:46.035 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:33:46.035 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:33:46.035 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:33:46.035 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:33:46.035 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:46.028734+0000", 2024-09-16T09:33:46.035 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.8800000000000001e-07 2024-09-16T09:33:46.035 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:33:46.036 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:33:46.036 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:33:46.036 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:46.028758+0000", 2024-09-16T09:33:46.036 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3459e-05 2024-09-16T09:33:46.036 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:33:46.036 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:33:46.036 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:33:46.036 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:46.028769+0000", 2024-09-16T09:33:46.036 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0716999999999999e-05 2024-09-16T09:33:46.036 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:33:46.036 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:33:46.037 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:33:46.037 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:46.028775+0000", 2024-09-16T09:33:46.038 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.5889999999999999e-06 2024-09-16T09:33:46.038 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:33:46.038 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:33:46.038 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:33:46.038 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:46.029881+0000", 2024-09-16T09:33:46.038 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001105883 2024-09-16T09:33:46.038 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:33:46.038 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:33:46.038 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:33:46.038 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:46.029898+0000", 2024-09-16T09:33:46.038 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6629e-05 2024-09-16T09:33:46.038 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:33:46.038 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:33:46.039 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:33:46.039 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:33:46.039 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-16T09:33:46.039 INFO:teuthology.orchestra.run.smithi070.stdout: "num_ops": 1 2024-09-16T09:33:46.039 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T09:33:46.062 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:33:46.093 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:46.094 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:46.094 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:33:46.175 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T09:33:46.195 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:33:46.261 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:46.262 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:46.262 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:33:46.363 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:33:46.391 INFO:teuthology.orchestra.run.smithi070.stderr:nodeep-scrub is unset 2024-09-16T09:33:46.396 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:46.397 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:46.397 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:33:46.498 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:33:46.532 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:46.533 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:46.533 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:33:46.634 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:33:46.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:46 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3267016955' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:33:46.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:46 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:33:46.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:46 smithi084 ceph-mon[32629]: 6.1c scrub starts 2024-09-16T09:33:46.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:46 smithi084 ceph-mon[32629]: 6.1c scrub ok 2024-09-16T09:33:46.666 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:46.667 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:46.667 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:33:46.673 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:46 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3267016955' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:33:46.673 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:46 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:33:46.674 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:46 smithi070 ceph-mon[30388]: 6.1c scrub starts 2024-09-16T09:33:46.674 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:46 smithi070 ceph-mon[30388]: 6.1c scrub ok 2024-09-16T09:33:46.674 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:46 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3267016955' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:33:46.674 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:46 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:33:46.674 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:46 smithi070 ceph-mon[40691]: 6.1c scrub starts 2024-09-16T09:33:46.674 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:46 smithi070 ceph-mon[40691]: 6.1c scrub ok 2024-09-16T09:33:46.707 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 16 16 40822 40806 159.379 103.688 0.00354816 0.00626928 2024-09-16T09:33:46.768 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:33:46.806 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:46.807 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:46.807 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:33:46.908 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:33:46.946 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:46.946 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:46.946 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:33:47.047 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:33:47.084 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:47.084 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:47.085 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:33:47.185 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:33:47.226 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:47.346 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:47.346 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:33:47.447 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:33:47.483 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:47.484 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:47.484 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:33:47.585 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:33:47.612 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:47 smithi070 ceph-mon[30388]: pgmap v416: 168 pgs: 168 active+clean; 2.0 GiB data, 8.7 GiB used, 707 GiB / 715 GiB avail; 797 B/s rd, 111 MiB/s wr, 1.77k op/s 2024-09-16T09:33:47.612 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:47 smithi070 ceph-mon[30388]: 6.19 scrub starts 2024-09-16T09:33:47.612 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:47 smithi070 ceph-mon[30388]: 6.19 scrub ok 2024-09-16T09:33:47.612 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:47 smithi070 ceph-mon[30388]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-16T09:33:47.613 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:47 smithi070 ceph-mon[30388]: Cluster is now healthy 2024-09-16T09:33:47.613 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:47 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-16T09:33:47.613 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:47 smithi070 ceph-mon[30388]: osdmap e240: 8 total, 8 up, 8 in 2024-09-16T09:33:47.613 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:47 smithi070 ceph-mon[40691]: pgmap v416: 168 pgs: 168 active+clean; 2.0 GiB data, 8.7 GiB used, 707 GiB / 715 GiB avail; 797 B/s rd, 111 MiB/s wr, 1.77k op/s 2024-09-16T09:33:47.613 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:47 smithi070 ceph-mon[40691]: 6.19 scrub starts 2024-09-16T09:33:47.613 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:47 smithi070 ceph-mon[40691]: 6.19 scrub ok 2024-09-16T09:33:47.613 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:47 smithi070 ceph-mon[40691]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-16T09:33:47.613 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:47 smithi070 ceph-mon[40691]: Cluster is now healthy 2024-09-16T09:33:47.613 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:47 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-16T09:33:47.613 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:47 smithi070 ceph-mon[40691]: osdmap e240: 8 total, 8 up, 8 in 2024-09-16T09:33:47.625 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:47.625 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:47.626 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:33:47.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:47 smithi084 ceph-mon[32629]: pgmap v416: 168 pgs: 168 active+clean; 2.0 GiB data, 8.7 GiB used, 707 GiB / 715 GiB avail; 797 B/s rd, 111 MiB/s wr, 1.77k op/s 2024-09-16T09:33:47.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:47 smithi084 ceph-mon[32629]: 6.19 scrub starts 2024-09-16T09:33:47.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:47 smithi084 ceph-mon[32629]: 6.19 scrub ok 2024-09-16T09:33:47.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:47 smithi084 ceph-mon[32629]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-16T09:33:47.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:47 smithi084 ceph-mon[32629]: Cluster is now healthy 2024-09-16T09:33:47.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:47 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-16T09:33:47.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:47 smithi084 ceph-mon[32629]: osdmap e240: 8 total, 8 up, 8 in 2024-09-16T09:33:47.707 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 17 16 42881 42865 157.573 128.688 0.00818627 0.00634052 2024-09-16T09:33:47.726 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:33:47.763 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:47.764 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:47.764 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:33:47.865 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:33:47.901 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:47.902 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:47.902 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:33:48.003 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:33:48.037 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:48.038 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:48.039 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:33:48.140 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:33:48.174 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:48.174 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:48.175 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:33:48.276 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:33:48.309 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:48.312 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:48.313 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:33:48.413 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:33:48.449 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:48.449 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:48.450 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:33:48.506 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:33:48.550 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:33:48.578 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.1 dump_blocked_ops 2024-09-16T09:33:48.579 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:48.579 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:48.580 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:33:48.663 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:33:48.680 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:33:48.709 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 18 16 44772 44756 155.384 118.188 0.0171096 0.0064293 2024-09-16T09:33:48.738 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:48.739 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:48.739 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:33:48.839 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:33:48.871 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:48.871 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:48.872 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:33:48.973 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:33:49.007 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:49.007 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:49.008 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:33:49.099 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:48 smithi070 ceph-mon[30388]: pgmap v418: 168 pgs: 168 active+clean; 2.2 GiB data, 9.0 GiB used, 706 GiB / 715 GiB avail; 1.7 KiB/s rd, 138 MiB/s wr, 2.21k op/s 2024-09-16T09:33:49.099 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:48 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:33:49.099 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:48 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:33:49.099 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:48 smithi070 ceph-mon[30388]: 3.e scrub starts 2024-09-16T09:33:49.099 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:48 smithi070 ceph-mon[30388]: 3.e scrub ok 2024-09-16T09:33:49.101 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:48 smithi070 ceph-mon[40691]: pgmap v418: 168 pgs: 168 active+clean; 2.2 GiB data, 9.0 GiB used, 706 GiB / 715 GiB avail; 1.7 KiB/s rd, 138 MiB/s wr, 2.21k op/s 2024-09-16T09:33:49.101 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:48 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:33:49.101 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:48 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:33:49.101 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:48 smithi070 ceph-mon[40691]: 3.e scrub starts 2024-09-16T09:33:49.101 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:48 smithi070 ceph-mon[40691]: 3.e scrub ok 2024-09-16T09:33:49.108 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:33:49.138 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:49.139 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:49.140 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:33:49.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:48 smithi084 ceph-mon[32629]: pgmap v418: 168 pgs: 168 active+clean; 2.2 GiB data, 9.0 GiB used, 706 GiB / 715 GiB avail; 1.7 KiB/s rd, 138 MiB/s wr, 2.21k op/s 2024-09-16T09:33:49.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:48 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:33:49.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:48 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:33:49.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:48 smithi084 ceph-mon[32629]: 3.e scrub starts 2024-09-16T09:33:49.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:48 smithi084 ceph-mon[32629]: 3.e scrub ok 2024-09-16T09:33:49.241 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:33:49.282 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:49.282 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:49.283 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:33:49.384 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:33:49.428 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:49.428 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:49.429 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:33:49.529 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:33:49.562 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:49.563 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:49.563 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:33:49.664 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:33:49.693 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:49.693 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:49.694 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:33:49.712 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 19 16 46386 46370 152.506 100.875 0.0206402 0.00655141 2024-09-16T09:33:49.795 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:33:49.835 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:49.836 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:49.836 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:33:49.937 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:33:49.947 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:33:49.984 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:49.985 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:49.985 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:33:49.993 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:33:49.993 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:33:50.038 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:49 smithi070 ceph-mon[30388]: 6.1e scrub starts 2024-09-16T09:33:50.039 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:49 smithi070 ceph-mon[30388]: 6.1e scrub ok 2024-09-16T09:33:50.039 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:49 smithi070 ceph-mon[30388]: 4.16 scrub starts 2024-09-16T09:33:50.039 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:49 smithi070 ceph-mon[30388]: 4.16 scrub ok 2024-09-16T09:33:50.039 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:49 smithi070 ceph-mon[30388]: osdmap e241: 8 total, 8 up, 8 in 2024-09-16T09:33:50.039 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:33:50.039 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:33:50.040 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:49 smithi070 ceph-mon[40691]: 6.1e scrub starts 2024-09-16T09:33:50.040 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:49 smithi070 ceph-mon[40691]: 6.1e scrub ok 2024-09-16T09:33:50.040 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:49 smithi070 ceph-mon[40691]: 4.16 scrub starts 2024-09-16T09:33:50.040 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:49 smithi070 ceph-mon[40691]: 4.16 scrub ok 2024-09-16T09:33:50.040 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:49 smithi070 ceph-mon[40691]: osdmap e241: 8 total, 8 up, 8 in 2024-09-16T09:33:50.075 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:33:50.076 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:33:50.086 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:33:50.094 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd set noscrub 2024-09-16T09:33:50.114 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:33:50.114 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:33:50.123 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:50.124 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:50.124 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:33:50.149 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:33:50.149 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:33:50.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:49 smithi084 ceph-mon[32629]: 6.1e scrub starts 2024-09-16T09:33:50.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:49 smithi084 ceph-mon[32629]: 6.1e scrub ok 2024-09-16T09:33:50.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:49 smithi084 ceph-mon[32629]: 4.16 scrub starts 2024-09-16T09:33:50.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:49 smithi084 ceph-mon[32629]: 4.16 scrub ok 2024-09-16T09:33:50.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:49 smithi084 ceph-mon[32629]: osdmap e241: 8 total, 8 up, 8 in 2024-09-16T09:33:50.183 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:33:50.183 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:33:50.219 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:33:50.219 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:33:50.225 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:33:50.252 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:33:50.252 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:33:50.254 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:50.255 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:50.256 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:33:50.356 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:33:50.394 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:50.394 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:50.395 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:33:50.495 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:33:50.529 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:50.530 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:50.531 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:33:50.632 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:33:50.670 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:50.671 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:50.671 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:33:50.708 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:2024-09-16T09:33:50.708053+0000 min lat: 0.00127131 max lat: 0.123754 avg lat: 0.00669138 2024-09-16T09:33:50.708 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-16T09:33:50.708 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 20 15 47811 47796 149.336 89.125 0.00744225 0.00669138 2024-09-16T09:33:50.772 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:33:50.825 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T09:33:50.825 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:33:50.828 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:50.828 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:50.828 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:33:50.893 INFO:teuthology.orchestra.run.smithi070.stderr:set pool 9 pg_num to 26 2024-09-16T09:33:50.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:50 smithi084 ceph-mon[32629]: pgmap v420: 168 pgs: 168 active+clean; 2.6 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 852 B/s rd, 103 MiB/s wr, 1.64k op/s 2024-09-16T09:33:50.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:50 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/38769394' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:33:50.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:50 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/2371368966' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num", "val": "26"} : dispatch 2024-09-16T09:33:50.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:50 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num", "val": "26"} : dispatch 2024-09-16T09:33:50.930 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:33:50.965 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:50.966 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:50.967 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:33:51.067 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:33:51.095 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:50 smithi070 ceph-mon[30388]: pgmap v420: 168 pgs: 168 active+clean; 2.6 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 852 B/s rd, 103 MiB/s wr, 1.64k op/s 2024-09-16T09:33:51.095 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:50 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/38769394' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:33:51.095 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:50 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2371368966' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num", "val": "26"} : dispatch 2024-09-16T09:33:51.095 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:50 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num", "val": "26"} : dispatch 2024-09-16T09:33:51.095 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:50 smithi070 ceph-mon[40691]: pgmap v420: 168 pgs: 168 active+clean; 2.6 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 852 B/s rd, 103 MiB/s wr, 1.64k op/s 2024-09-16T09:33:51.095 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:50 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/38769394' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:33:51.096 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:50 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/2371368966' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num", "val": "26"} : dispatch 2024-09-16T09:33:51.096 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:50 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num", "val": "26"} : dispatch 2024-09-16T09:33:51.104 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:51.104 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:51.105 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:33:51.205 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:33:51.244 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:51.244 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:51.244 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:33:51.318 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:33:51.346 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:33:51.385 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:51.385 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:51.386 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:33:51.487 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:33:51.521 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:51.522 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:51.522 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:33:51.623 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:33:51.661 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:51.661 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:51.662 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:33:51.708 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 21 16 49540 49524 147.368 108 0.0116732 0.00678087 2024-09-16T09:33:51.762 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:33:51.808 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:51.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:51.809 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:33:51.910 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:33:51.963 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:51.964 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:51.965 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:33:52.065 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:33:52.086 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:52.087 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:52.087 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:33:52.151 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:51 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num", "val": "26"}]': finished 2024-09-16T09:33:52.151 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:51 smithi084 ceph-mon[32629]: osdmap e242: 8 total, 8 up, 8 in 2024-09-16T09:33:52.151 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:51 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "26"} : dispatch 2024-09-16T09:33:52.152 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:51 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "26"} : dispatch 2024-09-16T09:33:52.170 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:33:51 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:33:51] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:33:52.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:51 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num", "val": "26"}]': finished 2024-09-16T09:33:52.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:51 smithi070 ceph-mon[30388]: osdmap e242: 8 total, 8 up, 8 in 2024-09-16T09:33:52.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:51 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "26"} : dispatch 2024-09-16T09:33:52.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:51 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "26"} : dispatch 2024-09-16T09:33:52.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:51 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num", "val": "26"}]': finished 2024-09-16T09:33:52.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:51 smithi070 ceph-mon[40691]: osdmap e242: 8 total, 8 up, 8 in 2024-09-16T09:33:52.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:51 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "26"} : dispatch 2024-09-16T09:33:52.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:51 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "26"} : dispatch 2024-09-16T09:33:52.188 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:33:52.210 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:52.211 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:52.212 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:33:52.312 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:33:52.334 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:52.335 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:52.335 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:33:52.436 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:33:52.457 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:52.458 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:52.459 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:33:52.560 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:33:52.585 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:52.586 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:52.586 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:33:52.687 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:33:52.709 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 22 16 49872 49856 141.612 20.75 0.00418509 0.0068077 2024-09-16T09:33:52.713 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:52.713 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:52.714 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:33:52.814 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:33:52.840 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:52.841 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:52.841 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:33:52.925 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.1 dump_historic_ops 2024-09-16T09:33:52.942 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:33:52.974 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:52.974 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:52.975 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:33:53.076 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:33:53.104 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:53.105 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:53.106 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:33:53.160 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:33:53 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:33:53.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:52 smithi084 ceph-mon[32629]: pgmap v422: 168 pgs: 168 active+clean; 2.6 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 852 B/s rd, 103 MiB/s wr, 1.64k op/s 2024-09-16T09:33:53.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:52 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "26"}]': finished 2024-09-16T09:33:53.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:52 smithi084 ceph-mon[32629]: osdmap e243: 8 total, 8 up, 8 in 2024-09-16T09:33:53.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:52 smithi070 ceph-mon[30388]: pgmap v422: 168 pgs: 168 active+clean; 2.6 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 852 B/s rd, 103 MiB/s wr, 1.64k op/s 2024-09-16T09:33:53.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:52 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "26"}]': finished 2024-09-16T09:33:53.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:52 smithi070 ceph-mon[30388]: osdmap e243: 8 total, 8 up, 8 in 2024-09-16T09:33:53.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:52 smithi070 ceph-mon[40691]: pgmap v422: 168 pgs: 168 active+clean; 2.6 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 852 B/s rd, 103 MiB/s wr, 1.64k op/s 2024-09-16T09:33:53.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:52 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "26"}]': finished 2024-09-16T09:33:53.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:52 smithi070 ceph-mon[40691]: osdmap e243: 8 total, 8 up, 8 in 2024-09-16T09:33:53.207 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:33:53.236 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:53.237 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:53.237 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:33:53.338 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:33:53.374 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:53.375 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:53.375 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:33:53.383 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:33:53.476 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:33:53.510 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:53.511 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:53.511 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:33:53.612 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:33:53.644 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:53.644 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:53.644 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:33:53.709 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 23 16 51159 51143 138.952 80.4375 0.0039752 0.00718988 2024-09-16T09:33:53.745 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:33:53.780 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:53.780 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:53.781 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:33:53.881 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:33:53.907 INFO:teuthology.orchestra.run.smithi070.stderr:noscrub is set 2024-09-16T09:33:53.918 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:53.919 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:53.919 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:33:54.020 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:33:54.041 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:54.042 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:54.042 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:33:54.143 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:33:54.164 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:54.165 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:54.165 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:33:54.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:53 smithi070 ceph-mon[40691]: 5.4 scrub starts 2024-09-16T09:33:54.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:53 smithi070 ceph-mon[40691]: 5.4 scrub ok 2024-09-16T09:33:54.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:53 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:33:54.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:53 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:33:54.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:53 smithi070 ceph-mon[40691]: osdmap e244: 8 total, 8 up, 8 in 2024-09-16T09:33:54.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:53 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:33:54.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:53 smithi070 ceph-mon[40691]: 9.2 scrub starts 2024-09-16T09:33:54.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:53 smithi070 ceph-mon[40691]: 9.2 scrub ok 2024-09-16T09:33:54.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:53 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1560383964' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:33:54.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:53 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:33:54.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:53 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num_actual", "val": "26"} : dispatch 2024-09-16T09:33:54.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:53 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num_actual", "val": "26"} : dispatch 2024-09-16T09:33:54.172 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:53 smithi070 ceph-mon[30388]: 5.4 scrub starts 2024-09-16T09:33:54.172 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:53 smithi070 ceph-mon[30388]: 5.4 scrub ok 2024-09-16T09:33:54.172 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:53 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:33:54.172 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:53 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:33:54.172 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:53 smithi070 ceph-mon[30388]: osdmap e244: 8 total, 8 up, 8 in 2024-09-16T09:33:54.172 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:53 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:33:54.172 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:53 smithi070 ceph-mon[30388]: 9.2 scrub starts 2024-09-16T09:33:54.172 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:53 smithi070 ceph-mon[30388]: 9.2 scrub ok 2024-09-16T09:33:54.172 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:53 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1560383964' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:33:54.172 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:53 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:33:54.172 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:53 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num_actual", "val": "26"} : dispatch 2024-09-16T09:33:54.172 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:53 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num_actual", "val": "26"} : dispatch 2024-09-16T09:33:54.203 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T09:33:54.266 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:33:54.281 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T09:33:54.281 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:33:54.290 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:54.290 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:54.291 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:33:54.392 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:33:54.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:53 smithi084 ceph-mon[32629]: 5.4 scrub starts 2024-09-16T09:33:54.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:53 smithi084 ceph-mon[32629]: 5.4 scrub ok 2024-09-16T09:33:54.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:53 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:33:54.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:53 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:33:54.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:53 smithi084 ceph-mon[32629]: osdmap e244: 8 total, 8 up, 8 in 2024-09-16T09:33:54.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:53 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:33:54.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:53 smithi084 ceph-mon[32629]: 9.2 scrub starts 2024-09-16T09:33:54.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:53 smithi084 ceph-mon[32629]: 9.2 scrub ok 2024-09-16T09:33:54.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:53 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1560383964' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:33:54.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:53 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:33:54.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:53 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num_actual", "val": "26"} : dispatch 2024-09-16T09:33:54.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:53 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num_actual", "val": "26"} : dispatch 2024-09-16T09:33:54.413 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:54.413 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:54.414 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:33:54.514 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:33:54.537 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:54.537 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:54.537 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:33:54.638 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:33:54.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:54.665 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:54.665 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:33:54.709 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 24 16 51629 51613 134.387 29.375 0.00193512 0.00721537 2024-09-16T09:33:54.766 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:33:54.789 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:54.789 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:54.790 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:33:54.891 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:33:54.913 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:54.913 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:54.914 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:33:55.014 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:33:55.040 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:55.041 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:55.041 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:33:55.142 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:33:55.164 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:55.165 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:55.165 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:33:55.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:54 smithi070 ceph-mon[30388]: 9.4 scrub starts 2024-09-16T09:33:55.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:54 smithi070 ceph-mon[30388]: 9.3 scrub starts 2024-09-16T09:33:55.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:54 smithi070 ceph-mon[30388]: 9.7 scrub starts 2024-09-16T09:33:55.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:54 smithi070 ceph-mon[30388]: 9.4 scrub ok 2024-09-16T09:33:55.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:54 smithi070 ceph-mon[30388]: 9.9 scrub starts 2024-09-16T09:33:55.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:54 smithi070 ceph-mon[30388]: pgmap v425: 178 pgs: 1 active+clean+scrubbing, 4 peering, 4 unknown, 169 active+clean; 3.1 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1.1 KiB/s rd, 95 MiB/s wr, 1.52k op/s; 72 KiB/s, 1 objects/s recovering 2024-09-16T09:33:55.172 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:54 smithi070 ceph-mon[30388]: 9.3 scrub ok 2024-09-16T09:33:55.172 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:54 smithi070 ceph-mon[30388]: 9.7 scrub ok 2024-09-16T09:33:55.172 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:54 smithi070 ceph-mon[30388]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:33:55.172 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:54 smithi070 ceph-mon[30388]: 9.1 scrub starts 2024-09-16T09:33:55.172 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:54 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-16T09:33:55.172 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:54 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num_actual", "val": "26"}]': finished 2024-09-16T09:33:55.172 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:54 smithi070 ceph-mon[30388]: osdmap e245: 8 total, 8 up, 8 in 2024-09-16T09:33:55.172 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:54 smithi070 ceph-mon[30388]: osdmap e246: 8 total, 8 up, 8 in 2024-09-16T09:33:55.173 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:54 smithi070 ceph-mon[40691]: 9.4 scrub starts 2024-09-16T09:33:55.173 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:54 smithi070 ceph-mon[40691]: 9.3 scrub starts 2024-09-16T09:33:55.173 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:54 smithi070 ceph-mon[40691]: 9.7 scrub starts 2024-09-16T09:33:55.173 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:54 smithi070 ceph-mon[40691]: 9.4 scrub ok 2024-09-16T09:33:55.173 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:54 smithi070 ceph-mon[40691]: 9.9 scrub starts 2024-09-16T09:33:55.173 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:54 smithi070 ceph-mon[40691]: pgmap v425: 178 pgs: 1 active+clean+scrubbing, 4 peering, 4 unknown, 169 active+clean; 3.1 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1.1 KiB/s rd, 95 MiB/s wr, 1.52k op/s; 72 KiB/s, 1 objects/s recovering 2024-09-16T09:33:55.173 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:54 smithi070 ceph-mon[40691]: 9.3 scrub ok 2024-09-16T09:33:55.173 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:54 smithi070 ceph-mon[40691]: 9.7 scrub ok 2024-09-16T09:33:55.173 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:54 smithi070 ceph-mon[40691]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:33:55.173 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:54 smithi070 ceph-mon[40691]: 9.1 scrub starts 2024-09-16T09:33:55.173 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:54 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-16T09:33:55.174 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:54 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num_actual", "val": "26"}]': finished 2024-09-16T09:33:55.174 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:54 smithi070 ceph-mon[40691]: osdmap e245: 8 total, 8 up, 8 in 2024-09-16T09:33:55.174 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:54 smithi070 ceph-mon[40691]: osdmap e246: 8 total, 8 up, 8 in 2024-09-16T09:33:55.266 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:33:55.306 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:55.307 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:55.307 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:33:55.408 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:33:55.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:54 smithi084 ceph-mon[32629]: 9.4 scrub starts 2024-09-16T09:33:55.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:54 smithi084 ceph-mon[32629]: 9.3 scrub starts 2024-09-16T09:33:55.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:54 smithi084 ceph-mon[32629]: 9.7 scrub starts 2024-09-16T09:33:55.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:54 smithi084 ceph-mon[32629]: 9.4 scrub ok 2024-09-16T09:33:55.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:54 smithi084 ceph-mon[32629]: 9.9 scrub starts 2024-09-16T09:33:55.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:54 smithi084 ceph-mon[32629]: pgmap v425: 178 pgs: 1 active+clean+scrubbing, 4 peering, 4 unknown, 169 active+clean; 3.1 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1.1 KiB/s rd, 95 MiB/s wr, 1.52k op/s; 72 KiB/s, 1 objects/s recovering 2024-09-16T09:33:55.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:54 smithi084 ceph-mon[32629]: 9.3 scrub ok 2024-09-16T09:33:55.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:54 smithi084 ceph-mon[32629]: 9.7 scrub ok 2024-09-16T09:33:55.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:54 smithi084 ceph-mon[32629]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:33:55.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:54 smithi084 ceph-mon[32629]: 9.1 scrub starts 2024-09-16T09:33:55.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:54 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-16T09:33:55.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:54 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num_actual", "val": "26"}]': finished 2024-09-16T09:33:55.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:54 smithi084 ceph-mon[32629]: osdmap e245: 8 total, 8 up, 8 in 2024-09-16T09:33:55.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:54 smithi084 ceph-mon[32629]: osdmap e246: 8 total, 8 up, 8 in 2024-09-16T09:33:55.446 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:55.446 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:55.446 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:33:55.547 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:33:55.594 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:55.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:55.595 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:33:55.696 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:33:55.709 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 25 16 52025 52009 130.002 24.75 0.011603 0.00768685 2024-09-16T09:33:55.735 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:55.735 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:55.736 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:33:55.837 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:33:55.878 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:55.879 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:55.879 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:33:55.980 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:33:56.017 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:56.018 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:56.018 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:33:56.119 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:33:56.157 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:56.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:56.158 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:33:56.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:55 smithi070 ceph-mon[30388]: 9.11 scrub starts 2024-09-16T09:33:56.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:55 smithi070 ceph-mon[30388]: 9.14 scrub starts 2024-09-16T09:33:56.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:55 smithi070 ceph-mon[30388]: 9.6 scrub starts 2024-09-16T09:33:56.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:55 smithi070 ceph-mon[30388]: 9.5 scrub starts 2024-09-16T09:33:56.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:55 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:33:56.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:55 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:33:56.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:55 smithi070 ceph-mon[30388]: 9.0 scrub starts 2024-09-16T09:33:56.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:55 smithi070 ceph-mon[30388]: osdmap e247: 8 total, 8 up, 8 in 2024-09-16T09:33:56.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:55 smithi070 ceph-mon[40691]: 9.11 scrub starts 2024-09-16T09:33:56.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:55 smithi070 ceph-mon[40691]: 9.14 scrub starts 2024-09-16T09:33:56.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:55 smithi070 ceph-mon[40691]: 9.6 scrub starts 2024-09-16T09:33:56.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:55 smithi070 ceph-mon[40691]: 9.5 scrub starts 2024-09-16T09:33:56.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:55 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:33:56.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:55 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:33:56.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:55 smithi070 ceph-mon[40691]: 9.0 scrub starts 2024-09-16T09:33:56.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:55 smithi070 ceph-mon[40691]: osdmap e247: 8 total, 8 up, 8 in 2024-09-16T09:33:56.259 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:33:56.301 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:56.301 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:56.302 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:33:56.402 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:33:56.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:55 smithi084 ceph-mon[32629]: 9.11 scrub starts 2024-09-16T09:33:56.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:55 smithi084 ceph-mon[32629]: 9.14 scrub starts 2024-09-16T09:33:56.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:55 smithi084 ceph-mon[32629]: 9.6 scrub starts 2024-09-16T09:33:56.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:55 smithi084 ceph-mon[32629]: 9.5 scrub starts 2024-09-16T09:33:56.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:55 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:33:56.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:55 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:33:56.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:55 smithi084 ceph-mon[32629]: 9.0 scrub starts 2024-09-16T09:33:56.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:55 smithi084 ceph-mon[32629]: osdmap e247: 8 total, 8 up, 8 in 2024-09-16T09:33:56.445 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:56.446 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:56.446 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:33:56.547 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:33:56.585 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:56.586 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:56.586 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:33:56.687 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:33:56.709 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 26 16 53360 53344 128.211 83.4375 0.0130781 0.00779325 2024-09-16T09:33:56.726 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:56.727 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:56.727 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:33:56.784 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T09:33:56.784 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T09:33:56.784 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '110', 'type': 'reweight-by-utilization'} 2024-09-16T09:33:56.784 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd reweight-by-utilization 110 0.05 2024-09-16T09:33:56.813 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.2 dump_ops_in_flight 2024-09-16T09:33:56.828 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:33:56.837 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:33:56.862 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:56.863 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:56.863 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:33:56.964 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:33:56.998 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:56.998 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:56.999 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:33:57.100 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:33:57.136 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:57.137 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:57.137 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:33:57.238 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:33:57.274 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:57.275 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:57.275 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:33:57.376 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:33:57.399 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:57.400 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:57.400 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:33:57.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:57 smithi070 ceph-mon[30388]: pgmap v429: 178 pgs: 1 active+clean+scrubbing, 4 peering, 4 unknown, 169 active+clean; 3.1 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1.3 KiB/s rd, 111 MiB/s wr, 1.78k op/s; 85 KiB/s, 1 objects/s recovering 2024-09-16T09:33:57.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:57 smithi070 ceph-mon[30388]: 9.8 scrub starts 2024-09-16T09:33:57.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:57 smithi070 ceph-mon[40691]: pgmap v429: 178 pgs: 1 active+clean+scrubbing, 4 peering, 4 unknown, 169 active+clean; 3.1 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1.3 KiB/s rd, 111 MiB/s wr, 1.78k op/s; 85 KiB/s, 1 objects/s recovering 2024-09-16T09:33:57.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:57 smithi070 ceph-mon[40691]: 9.8 scrub starts 2024-09-16T09:33:57.501 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:33:57.540 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:57.541 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:57.541 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:33:57.642 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:33:57.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:57 smithi084 ceph-mon[32629]: pgmap v429: 178 pgs: 1 active+clean+scrubbing, 4 peering, 4 unknown, 169 active+clean; 3.1 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1.3 KiB/s rd, 111 MiB/s wr, 1.78k op/s; 85 KiB/s, 1 objects/s recovering 2024-09-16T09:33:57.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:57 smithi084 ceph-mon[32629]: 9.8 scrub starts 2024-09-16T09:33:57.678 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:57.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:57.679 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:33:57.709 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 27 16 54469 54453 126.029 69.3125 0.0148978 0.00792875 2024-09-16T09:33:57.779 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:33:57.808 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:33:57.809 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:33:57.829 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:57.830 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:57.830 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:33:57.856 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:33:57.857 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:33:57.882 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:33:57.892 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:33:57.892 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:33:57.900 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:33:57.931 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:33:57.932 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:33:57.932 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:33:57.963 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:33:57.964 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:33:57.987 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:57.988 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:57.988 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:33:57.997 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:33:57.997 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:33:58.028 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:33:58.029 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:33:58.067 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:33:58.068 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:33:58.089 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:33:58.122 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:58.123 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:58.124 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:33:58.224 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:33:58.258 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:58.259 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:58.259 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:33:58.266 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-16T09:33:58.360 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:33:58.395 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:58.396 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:58.397 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:33:58.497 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:33:58.537 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:58.537 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:58.538 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:33:58.569 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:58 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/527807842' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:33:58.569 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:58 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/527807842' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:33:58.638 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:33:58.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:58 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/527807842' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:33:58.676 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:58.677 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:58.677 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:33:58.709 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 28 16 55587 55571 124.024 69.875 0.00955044 0.00805739 2024-09-16T09:33:58.778 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:33:58.818 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:58.818 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:58.819 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:33:58.919 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:33:58.962 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:58.962 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:58.963 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:33:59.063 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:33:59.101 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:59.101 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:59.102 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:33:59.203 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:33:59.243 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:59.243 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:59.244 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:33:59.328 INFO:teuthology.orchestra.run.smithi070.stdout:moved 12 / 534 (2.24719%) 2024-09-16T09:33:59.328 INFO:teuthology.orchestra.run.smithi070.stdout:avg 66.75 2024-09-16T09:33:59.328 INFO:teuthology.orchestra.run.smithi070.stdout:stddev 9.73075 -> 9.79477 (expected baseline 7.6424) 2024-09-16T09:33:59.328 INFO:teuthology.orchestra.run.smithi070.stdout:min osd.2 with 48 -> 48 pgs (0.719101 -> 0.719101 * mean) 2024-09-16T09:33:59.328 INFO:teuthology.orchestra.run.smithi070.stdout:max osd.3 with 77 -> 77 pgs (1.15356 -> 1.15356 * mean) 2024-09-16T09:33:59.328 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:33:59.329 INFO:teuthology.orchestra.run.smithi070.stdout:oload 110 2024-09-16T09:33:59.329 INFO:teuthology.orchestra.run.smithi070.stdout:max_change 0.05 2024-09-16T09:33:59.329 INFO:teuthology.orchestra.run.smithi070.stdout:max_change_osds 4 2024-09-16T09:33:59.329 INFO:teuthology.orchestra.run.smithi070.stdout:average_utilization 0.0171 2024-09-16T09:33:59.329 INFO:teuthology.orchestra.run.smithi070.stdout:overload_utilization 0.0188 2024-09-16T09:33:59.329 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7 weight 1.0000 -> 0.9500 2024-09-16T09:33:59.329 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1 weight 1.0000 -> 0.9500 2024-09-16T09:33:59.329 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5 weight 1.0000 -> 0.9500 2024-09-16T09:33:59.329 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3 weight 1.0000 -> 0.9500 2024-09-16T09:33:59.345 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:33:59.377 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:59.377 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:59.380 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:33:59.481 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:33:59.505 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:59.505 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:59.506 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:33:59.606 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:33:59.634 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:59.635 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:59.635 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:33:59.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:59 smithi084 ceph-mon[32629]: pgmap v430: 178 pgs: 3 active+recovering+undersized+remapped, 1 active+clean+scrubbing, 1 peering, 2 unknown, 171 active+clean; 3.2 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1.0 KiB/s rd, 47 MiB/s wr, 755 op/s; 9019/155601 objects misplaced (5.796%); 4.7 MiB/s, 75 objects/s recovering 2024-09-16T09:33:59.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:59 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd reweightn", "weights": "{'1':'62260','3':'62260','5':'62260','7':'62260'}"} : dispatch 2024-09-16T09:33:59.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:33:59 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "osd reweightn", "weights": "{'1':'62260','3':'62260','5':'62260','7':'62260'}"} : dispatch 2024-09-16T09:33:59.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:59 smithi070 ceph-mon[30388]: pgmap v430: 178 pgs: 3 active+recovering+undersized+remapped, 1 active+clean+scrubbing, 1 peering, 2 unknown, 171 active+clean; 3.2 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1.0 KiB/s rd, 47 MiB/s wr, 755 op/s; 9019/155601 objects misplaced (5.796%); 4.7 MiB/s, 75 objects/s recovering 2024-09-16T09:33:59.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:59 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd reweightn", "weights": "{'1':'62260','3':'62260','5':'62260','7':'62260'}"} : dispatch 2024-09-16T09:33:59.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:33:59 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "osd reweightn", "weights": "{'1':'62260','3':'62260','5':'62260','7':'62260'}"} : dispatch 2024-09-16T09:33:59.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:59 smithi070 ceph-mon[40691]: pgmap v430: 178 pgs: 3 active+recovering+undersized+remapped, 1 active+clean+scrubbing, 1 peering, 2 unknown, 171 active+clean; 3.2 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1.0 KiB/s rd, 47 MiB/s wr, 755 op/s; 9019/155601 objects misplaced (5.796%); 4.7 MiB/s, 75 objects/s recovering 2024-09-16T09:33:59.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:59 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd reweightn", "weights": "{'1':'62260','3':'62260','5':'62260','7':'62260'}"} : dispatch 2024-09-16T09:33:59.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:33:59 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "osd reweightn", "weights": "{'1':'62260','3':'62260','5':'62260','7':'62260'}"} : dispatch 2024-09-16T09:33:59.709 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 29 16 56365 56349 121.424 48.625 0.0933495 0.00813692 2024-09-16T09:33:59.736 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:33:59.765 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:33:59.765 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:59.766 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:33:59.867 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:33:59.893 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:33:59.894 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:33:59.894 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:33:59.995 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:34:00.022 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:00.023 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:00.023 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:34:00.124 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:34:00.151 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:00.152 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:00.152 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:34:00.253 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:34:00.283 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:00.284 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:00.284 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:34:00.385 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:34:00.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:00.412 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:00.412 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:34:00.513 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:34:00.541 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:00.542 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:00.542 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:34:00.640 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:00 smithi070 ceph-mon[30388]: from='client.15636 -' entity='client.admin' cmd=[{"prefix": "osd reweight-by-utilization", "oload": 110, "max_change": 0.05, "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:34:00.640 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:00 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd reweightn", "weights": "{'1':'62260','3':'62260','5':'62260','7':'62260'}"}]': finished 2024-09-16T09:34:00.640 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:00 smithi070 ceph-mon[30388]: osdmap e248: 8 total, 8 up, 8 in 2024-09-16T09:34:00.641 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:00 smithi070 ceph-mon[40691]: from='client.15636 -' entity='client.admin' cmd=[{"prefix": "osd reweight-by-utilization", "oload": 110, "max_change": 0.05, "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:34:00.641 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:00 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd reweightn", "weights": "{'1':'62260','3':'62260','5':'62260','7':'62260'}"}]': finished 2024-09-16T09:34:00.641 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:00 smithi070 ceph-mon[40691]: osdmap e248: 8 total, 8 up, 8 in 2024-09-16T09:34:00.643 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:34:00.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:00 smithi084 ceph-mon[32629]: from='client.15636 -' entity='client.admin' cmd=[{"prefix": "osd reweight-by-utilization", "oload": 110, "max_change": 0.05, "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:34:00.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:00 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd reweightn", "weights": "{'1':'62260','3':'62260','5':'62260','7':'62260'}"}]': finished 2024-09-16T09:34:00.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:00 smithi084 ceph-mon[32629]: osdmap e248: 8 total, 8 up, 8 in 2024-09-16T09:34:00.674 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:00.675 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:00.675 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:34:00.710 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 30 16 56365 56349 117.375 0 - 0.00813692 2024-09-16T09:34:00.776 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:34:00.807 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:00.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:00.808 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:34:00.909 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:34:00.938 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:00.938 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:00.939 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:34:01.040 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:34:01.067 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:01.068 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:01.069 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:34:01.107 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T09:34:01.107 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T09:34:01.108 INFO:teuthology.orchestra.run.smithi070.stdout: "num_ops": 0 2024-09-16T09:34:01.108 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T09:34:01.169 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:34:01.198 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:01.198 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:01.199 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:34:01.300 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:34:01.321 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T09:34:01.329 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:01.329 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:01.330 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:34:01.333 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:01 smithi070 ceph-mon[30388]: pgmap v432: 178 pgs: 1 active+recovery_wait+undersized+degraded+remapped, 7 active+recovering+undersized+remapped, 170 active+clean; 3.3 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 888 B/s rd, 0 op/s; 1587/164661 objects degraded (0.964%); 22562/164661 objects misplaced (13.702%); 9.6 MiB/s, 153 objects/s recovering 2024-09-16T09:34:01.333 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:01 smithi070 ceph-mon[30388]: Health check failed: Degraded data redundancy: 1587/164661 objects degraded (0.964%), 1 pg degraded (PG_DEGRADED) 2024-09-16T09:34:01.333 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:01 smithi070 ceph-mon[30388]: osdmap e249: 8 total, 8 up, 8 in 2024-09-16T09:34:01.333 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:01 smithi070 ceph-mon[30388]: 1.8 scrub starts 2024-09-16T09:34:01.431 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:34:01.460 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:01.460 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:01.461 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:34:01.562 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:34:01.594 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:01.595 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:01.596 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:34:01.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:01 smithi084 ceph-mon[32629]: pgmap v432: 178 pgs: 1 active+recovery_wait+undersized+degraded+remapped, 7 active+recovering+undersized+remapped, 170 active+clean; 3.3 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 888 B/s rd, 0 op/s; 1587/164661 objects degraded (0.964%); 22562/164661 objects misplaced (13.702%); 9.6 MiB/s, 153 objects/s recovering 2024-09-16T09:34:01.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:01 smithi084 ceph-mon[32629]: Health check failed: Degraded data redundancy: 1587/164661 objects degraded (0.964%), 1 pg degraded (PG_DEGRADED) 2024-09-16T09:34:01.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:01 smithi084 ceph-mon[32629]: osdmap e249: 8 total, 8 up, 8 in 2024-09-16T09:34:01.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:01 smithi084 ceph-mon[32629]: 1.8 scrub starts 2024-09-16T09:34:01.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:01 smithi070 ceph-mon[40691]: pgmap v432: 178 pgs: 1 active+recovery_wait+undersized+degraded+remapped, 7 active+recovering+undersized+remapped, 170 active+clean; 3.3 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 888 B/s rd, 0 op/s; 1587/164661 objects degraded (0.964%); 22562/164661 objects misplaced (13.702%); 9.6 MiB/s, 153 objects/s recovering 2024-09-16T09:34:01.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:01 smithi070 ceph-mon[40691]: Health check failed: Degraded data redundancy: 1587/164661 objects degraded (0.964%), 1 pg degraded (PG_DEGRADED) 2024-09-16T09:34:01.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:01 smithi070 ceph-mon[40691]: osdmap e249: 8 total, 8 up, 8 in 2024-09-16T09:34:01.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:01 smithi070 ceph-mon[40691]: 1.8 scrub starts 2024-09-16T09:34:01.696 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:34:01.710 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 31 16 56657 56641 114.178 9.125 0.0151976 0.00875257 2024-09-16T09:34:01.734 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:01.735 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:01.735 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:34:01.826 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:34:01.836 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:34:01.871 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:01.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:01.873 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:34:01.889 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.2 dump_blocked_ops 2024-09-16T09:34:01.973 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:34:02.029 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:02.030 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:02.030 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:34:02.131 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:34:02.171 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:34:01 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:34:01] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:34:02.172 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:02.173 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:02.173 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:34:02.274 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:34:02.313 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:02.314 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:02.314 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:34:02.415 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:34:02.449 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:02.450 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:02.450 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:34:02.551 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:34:02.554 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '110', 'type': 'test-reweight-by-utilization'} 2024-09-16T09:34:02.555 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd test-reweight-by-utilization 110 0.05 2024-09-16T09:34:02.591 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:02.591 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:02.592 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:34:02.594 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:02 smithi070 ceph-mon[30388]: 1.b scrub starts 2024-09-16T09:34:02.594 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:02 smithi070 ceph-mon[30388]: osdmap e250: 8 total, 8 up, 8 in 2024-09-16T09:34:02.594 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:02 smithi070 ceph-mon[30388]: 1.3 scrub starts 2024-09-16T09:34:02.594 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:02 smithi070 ceph-mon[40691]: 1.b scrub starts 2024-09-16T09:34:02.595 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:02 smithi070 ceph-mon[40691]: osdmap e250: 8 total, 8 up, 8 in 2024-09-16T09:34:02.595 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:02 smithi070 ceph-mon[40691]: 1.3 scrub starts 2024-09-16T09:34:02.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:02 smithi084 ceph-mon[32629]: 1.b scrub starts 2024-09-16T09:34:02.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:02 smithi084 ceph-mon[32629]: osdmap e250: 8 total, 8 up, 8 in 2024-09-16T09:34:02.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:02 smithi084 ceph-mon[32629]: 1.3 scrub starts 2024-09-16T09:34:02.693 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:34:02.710 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 32 16 57796 57780 112.834 71.1875 0.00757874 0.0088558 2024-09-16T09:34:02.735 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:02.735 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:02.736 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:34:02.837 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:34:02.882 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:02.883 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:02.883 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:34:02.984 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:34:03.027 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:03.029 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:03.029 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:34:03.130 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:34:03.170 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:03.170 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:03.170 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:34:03.271 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:34:03.310 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:03.311 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:03.311 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:34:03.340 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:34:03 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:34:03.412 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:34:03.456 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:03.457 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:03.457 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:34:03.558 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:34:03.592 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:03.592 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:03.593 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:34:03.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:03 smithi084 ceph-mon[32629]: pgmap v435: 178 pgs: 1 active+recovery_wait+undersized+degraded+remapped, 7 active+recovering+undersized+remapped, 170 active+clean; 3.3 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 1587/164661 objects degraded (0.964%); 22562/164661 objects misplaced (13.702%); 9.2 MiB/s, 147 objects/s recovering 2024-09-16T09:34:03.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:03 smithi084 ceph-mon[32629]: 1.2 scrub starts 2024-09-16T09:34:03.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:03 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:34:03.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:03 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:34:03.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:03 smithi070 ceph-mon[30388]: pgmap v435: 178 pgs: 1 active+recovery_wait+undersized+degraded+remapped, 7 active+recovering+undersized+remapped, 170 active+clean; 3.3 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 1587/164661 objects degraded (0.964%); 22562/164661 objects misplaced (13.702%); 9.2 MiB/s, 147 objects/s recovering 2024-09-16T09:34:03.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:03 smithi070 ceph-mon[30388]: 1.2 scrub starts 2024-09-16T09:34:03.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:03 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:34:03.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:03 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:34:03.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:03 smithi070 ceph-mon[40691]: pgmap v435: 178 pgs: 1 active+recovery_wait+undersized+degraded+remapped, 7 active+recovering+undersized+remapped, 170 active+clean; 3.3 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 1587/164661 objects degraded (0.964%); 22562/164661 objects misplaced (13.702%); 9.2 MiB/s, 147 objects/s recovering 2024-09-16T09:34:03.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:03 smithi070 ceph-mon[40691]: 1.2 scrub starts 2024-09-16T09:34:03.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:03 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:34:03.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:03 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:34:03.694 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:34:03.710 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 33 16 58930 58914 111.562 70.875 0.0128574 0.00895757 2024-09-16T09:34:03.734 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:03.735 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:03.735 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:34:03.836 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:34:03.858 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:03.858 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:03.859 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:34:03.959 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:34:03.998 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:03.999 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:03.999 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:34:04.100 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:34:04.131 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:04.131 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:04.131 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:34:04.232 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:34:04.273 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:04.274 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:04.274 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:34:04.375 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:34:04.412 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:04.413 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:04.413 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:34:04.514 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:34:04.551 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:04.552 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:04.552 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:34:04.652 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:34:04.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:04 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:34:04.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:04 smithi084 ceph-mon[32629]: osdmap e251: 8 total, 8 up, 8 in 2024-09-16T09:34:04.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:04 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:34:04.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:04 smithi070 ceph-mon[30388]: osdmap e251: 8 total, 8 up, 8 in 2024-09-16T09:34:04.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:04 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:34:04.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:04 smithi070 ceph-mon[40691]: osdmap e251: 8 total, 8 up, 8 in 2024-09-16T09:34:04.691 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:04.691 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:04.692 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:34:04.710 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 34 16 59962 59946 110.178 64.5 0.0234449 0.00906213 2024-09-16T09:34:04.792 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:34:04.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:04.835 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:04.835 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:34:04.936 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:34:04.984 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:04.985 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:04.985 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:34:05.085 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:34:05.126 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:05.127 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:05.127 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:34:05.228 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:34:05.268 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:05.269 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:05.269 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:34:05.370 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:34:05.416 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:05.416 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:05.417 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:34:05.422 INFO:teuthology.orchestra.run.smithi070.stderr:nodeep-scrub is set 2024-09-16T09:34:05.518 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:34:05.560 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:05.561 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:05.561 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:34:05.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:05 smithi084 ceph-mon[32629]: pgmap v436: 178 pgs: 2 active+recovery_wait+degraded, 1 active+clean+scrubbing, 4 active+recovery_wait+undersized+degraded+remapped, 7 active+recovering+undersized+remapped, 164 active+clean; 3.5 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 5.8 KiB/s rd, 17 MiB/s wr, 274 op/s; 10424/174828 objects degraded (5.962%); 17080/174828 objects misplaced (9.770%); 36 MiB/s, 575 objects/s recovering 2024-09-16T09:34:05.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:05 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/859878526' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:34:05.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:05 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:34:05.661 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:34:05.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:05 smithi070 ceph-mon[30388]: pgmap v436: 178 pgs: 2 active+recovery_wait+degraded, 1 active+clean+scrubbing, 4 active+recovery_wait+undersized+degraded+remapped, 7 active+recovering+undersized+remapped, 164 active+clean; 3.5 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 5.8 KiB/s rd, 17 MiB/s wr, 274 op/s; 10424/174828 objects degraded (5.962%); 17080/174828 objects misplaced (9.770%); 36 MiB/s, 575 objects/s recovering 2024-09-16T09:34:05.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:05 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/859878526' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:34:05.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:05 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:34:05.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:05 smithi070 ceph-mon[40691]: pgmap v436: 178 pgs: 2 active+recovery_wait+degraded, 1 active+clean+scrubbing, 4 active+recovery_wait+undersized+degraded+remapped, 7 active+recovering+undersized+remapped, 164 active+clean; 3.5 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 5.8 KiB/s rd, 17 MiB/s wr, 274 op/s; 10424/174828 objects degraded (5.962%); 17080/174828 objects misplaced (9.770%); 36 MiB/s, 575 objects/s recovering 2024-09-16T09:34:05.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:05 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/859878526' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:34:05.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:05 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:34:05.699 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:05.700 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:05.700 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:34:05.710 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 35 16 61060 61044 108.991 68.625 0.0108098 0.009169 2024-09-16T09:34:05.801 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:34:05.840 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:05.841 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:05.841 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:34:05.942 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:34:05.983 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:05.984 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:05.984 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:34:06.085 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:34:06.125 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:06.126 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:06.126 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:34:06.227 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:34:06.270 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:06.271 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:06.272 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:34:06.372 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:34:06.411 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:06.411 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:06.412 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:34:06.513 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:34:06.552 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:06.552 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:06.552 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:34:06.653 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:34:06.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:06 smithi084 ceph-mon[32629]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:34:06.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:06 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/859878526' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-16T09:34:06.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:06 smithi084 ceph-mon[32629]: osdmap e252: 8 total, 8 up, 8 in 2024-09-16T09:34:06.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:06 smithi070 ceph-mon[30388]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:34:06.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:06 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/859878526' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-16T09:34:06.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:06 smithi070 ceph-mon[30388]: osdmap e252: 8 total, 8 up, 8 in 2024-09-16T09:34:06.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:06 smithi070 ceph-mon[40691]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:34:06.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:06 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/859878526' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-16T09:34:06.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:06 smithi070 ceph-mon[40691]: osdmap e252: 8 total, 8 up, 8 in 2024-09-16T09:34:06.689 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:06.690 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:06.690 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:34:06.710 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 36 16 62157 62141 107.867 68.5625 0.00909906 0.00925898 2024-09-16T09:34:06.791 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:34:06.826 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:06.826 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:06.827 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:34:06.928 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:34:06.967 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:06.968 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:06.969 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:34:07.070 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:34:07.109 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:07.110 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:07.110 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:34:07.211 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:34:07.252 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:07.252 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:07.252 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:34:07.353 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:34:07.393 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:07.394 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:07.394 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:34:07.495 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:34:07.534 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:07.534 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:07.535 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:34:07.635 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:34:07.662 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:07 smithi084 ceph-mon[32629]: pgmap v439: 178 pgs: 2 active+recovery_wait+degraded, 1 active+clean+scrubbing, 4 active+recovery_wait+undersized+degraded+remapped, 7 active+recovering+undersized+remapped, 164 active+clean; 3.5 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 6.5 KiB/s rd, 40 MiB/s wr, 639 op/s; 10424/174828 objects degraded (5.962%); 17080/174828 objects misplaced (9.770%); 34 MiB/s, 546 objects/s recovering 2024-09-16T09:34:07.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:07 smithi070 ceph-mon[30388]: pgmap v439: 178 pgs: 2 active+recovery_wait+degraded, 1 active+clean+scrubbing, 4 active+recovery_wait+undersized+degraded+remapped, 7 active+recovering+undersized+remapped, 164 active+clean; 3.5 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 6.5 KiB/s rd, 40 MiB/s wr, 639 op/s; 10424/174828 objects degraded (5.962%); 17080/174828 objects misplaced (9.770%); 34 MiB/s, 546 objects/s recovering 2024-09-16T09:34:07.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:07 smithi070 ceph-mon[40691]: pgmap v439: 178 pgs: 2 active+recovery_wait+degraded, 1 active+clean+scrubbing, 4 active+recovery_wait+undersized+degraded+remapped, 7 active+recovering+undersized+remapped, 164 active+clean; 3.5 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 6.5 KiB/s rd, 40 MiB/s wr, 639 op/s; 10424/174828 objects degraded (5.962%); 17080/174828 objects misplaced (9.770%); 34 MiB/s, 546 objects/s recovering 2024-09-16T09:34:07.677 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:07.704 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:07.705 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:34:07.710 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 37 16 63216 63200 106.741 66.1875 0.0215699 0.00936337 2024-09-16T09:34:07.748 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:34:07.805 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:34:07.843 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:07.843 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:07.844 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:34:07.944 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:34:07.982 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:07.982 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:07.983 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:34:08.084 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:34:08.123 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:08.123 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:08.124 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:34:08.225 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:34:08.269 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:08.270 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:08.270 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:34:08.371 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:34:08.407 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:08.408 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:08.408 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:34:08.509 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:34:08.549 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:08.549 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:08.549 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:34:08.650 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:34:08.692 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:08.693 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:08.694 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:34:08.711 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 38 16 64357 64341 105.808 71.3125 0.00879777 0.00944618 2024-09-16T09:34:08.794 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:34:08.832 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:08.833 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:08.833 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:34:08.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:08 smithi084 ceph-mon[32629]: Health check update: Degraded data redundancy: 10424/175941 objects degraded (5.925%), 6 pgs degraded (PG_DEGRADED) 2024-09-16T09:34:08.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:08 smithi070 ceph-mon[30388]: Health check update: Degraded data redundancy: 10424/175941 objects degraded (5.925%), 6 pgs degraded (PG_DEGRADED) 2024-09-16T09:34:08.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:08 smithi070 ceph-mon[40691]: Health check update: Degraded data redundancy: 10424/175941 objects degraded (5.925%), 6 pgs degraded (PG_DEGRADED) 2024-09-16T09:34:08.934 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:34:08.971 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:08.972 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:08.973 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:34:09.074 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:34:09.114 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:09.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:09.115 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:34:09.216 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:34:09.240 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:09.241 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:09.241 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:34:09.342 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:34:09.378 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:09.379 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:09.380 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:34:09.480 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:34:09.505 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:09.506 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:09.506 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:34:09.607 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:34:09.643 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:09.643 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:09.644 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:34:09.711 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 39 16 65402 65386 104.77 65.3125 0.0105576 0.00953849 2024-09-16T09:34:09.744 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:34:09.781 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:09.781 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:09.782 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:34:09.883 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:34:09.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:09 smithi084 ceph-mon[32629]: pgmap v440: 178 pgs: 2 active+recovery_wait+degraded, 4 active+recovery_wait+undersized+degraded+remapped, 7 active+recovering+undersized+remapped, 165 active+clean; 3.6 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 5.5 KiB/s rd, 37 MiB/s wr, 597 op/s; 10424/175941 objects degraded (5.925%); 17080/175941 objects misplaced (9.708%); 29 MiB/s, 460 objects/s recovering 2024-09-16T09:34:09.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:09 smithi084 ceph-mon[32629]: osdmap e253: 8 total, 8 up, 8 in 2024-09-16T09:34:09.920 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:09.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:09 smithi070 ceph-mon[30388]: pgmap v440: 178 pgs: 2 active+recovery_wait+degraded, 4 active+recovery_wait+undersized+degraded+remapped, 7 active+recovering+undersized+remapped, 165 active+clean; 3.6 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 5.5 KiB/s rd, 37 MiB/s wr, 597 op/s; 10424/175941 objects degraded (5.925%); 17080/175941 objects misplaced (9.708%); 29 MiB/s, 460 objects/s recovering 2024-09-16T09:34:09.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:09 smithi070 ceph-mon[30388]: osdmap e253: 8 total, 8 up, 8 in 2024-09-16T09:34:09.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:09 smithi070 ceph-mon[40691]: pgmap v440: 178 pgs: 2 active+recovery_wait+degraded, 4 active+recovery_wait+undersized+degraded+remapped, 7 active+recovering+undersized+remapped, 165 active+clean; 3.6 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 5.5 KiB/s rd, 37 MiB/s wr, 597 op/s; 10424/175941 objects degraded (5.925%); 17080/175941 objects misplaced (9.708%); 29 MiB/s, 460 objects/s recovering 2024-09-16T09:34:09.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:09 smithi070 ceph-mon[40691]: osdmap e253: 8 total, 8 up, 8 in 2024-09-16T09:34:09.922 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:09.922 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:34:10.023 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:34:10.056 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:10.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:10.057 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:34:10.157 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:34:10.197 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:10.197 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:10.198 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:34:10.298 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:34:10.325 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:10.326 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:10.326 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:34:10.427 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:34:10.463 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:10.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:10.465 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:34:10.565 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:34:10.593 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:10.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:10.595 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:34:10.696 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:34:10.710 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:2024-09-16T09:34:10.710328+0000 min lat: 0.00127131 max lat: 2.16104 avg lat: 0.00963126 2024-09-16T09:34:10.711 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-16T09:34:10.711 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 40 16 66435 66419 103.765 64.5625 0.0313398 0.00963126 2024-09-16T09:34:10.734 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:10.734 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:10.735 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:34:10.836 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:34:10.857 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:10.858 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:10.858 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:34:10.959 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:34:10.996 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:10.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:10.997 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:34:11.098 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:34:11.132 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:11.132 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:11.132 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:34:11.233 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:34:11.273 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:11.274 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:11.274 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:34:11.375 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:34:11.416 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:11.417 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:11.417 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:34:11.518 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:34:11.551 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:11.552 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:11.552 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:34:11.653 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:34:11.684 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:11.685 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:11.686 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:34:11.711 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 41 16 67304 67288 102.559 54.3125 0.00596868 0.00970851 2024-09-16T09:34:11.786 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:34:11.812 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:11.813 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:11.813 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:34:11.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:11 smithi084 ceph-mon[32629]: pgmap v442: 178 pgs: 2 active+recovery_wait+degraded, 4 active+recovery_wait+undersized+degraded+remapped, 7 active+recovering+undersized+remapped, 165 active+clean; 3.9 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 852 B/s rd, 56 MiB/s wr, 896 op/s; 10424/190959 objects degraded (5.459%); 12990/190959 objects misplaced (6.803%); 23 MiB/s, 363 objects/s recovering 2024-09-16T09:34:11.915 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:34:11.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:11 smithi070 ceph-mon[30388]: pgmap v442: 178 pgs: 2 active+recovery_wait+degraded, 4 active+recovery_wait+undersized+degraded+remapped, 7 active+recovering+undersized+remapped, 165 active+clean; 3.9 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 852 B/s rd, 56 MiB/s wr, 896 op/s; 10424/190959 objects degraded (5.459%); 12990/190959 objects misplaced (6.803%); 23 MiB/s, 363 objects/s recovering 2024-09-16T09:34:11.920 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:11 smithi070 ceph-mon[40691]: pgmap v442: 178 pgs: 2 active+recovery_wait+degraded, 4 active+recovery_wait+undersized+degraded+remapped, 7 active+recovering+undersized+remapped, 165 active+clean; 3.9 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 852 B/s rd, 56 MiB/s wr, 896 op/s; 10424/190959 objects degraded (5.459%); 12990/190959 objects misplaced (6.803%); 23 MiB/s, 363 objects/s recovering 2024-09-16T09:34:11.945 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:11.946 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:11.946 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:34:12.047 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:34:12.078 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:12.078 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:12.079 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:34:12.180 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:34:12.209 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:12.210 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:12.210 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:34:12.311 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:34:12.336 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:12.337 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:12.337 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:34:12.421 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:34:11 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:34:11] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:34:12.438 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:34:12.479 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:12.479 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:12.480 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:34:12.581 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:34:12.619 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:12.620 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:12.620 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:34:12.711 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 42 15 67609 67594 100.572 19.125 0.00629547 0.00993841 2024-09-16T09:34:12.721 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:34:12.750 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:12.751 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:12.751 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:34:12.825 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:12 smithi070 ceph-mon[30388]: osdmap e254: 8 total, 8 up, 8 in 2024-09-16T09:34:12.826 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:12 smithi070 ceph-mon[40691]: osdmap e254: 8 total, 8 up, 8 in 2024-09-16T09:34:12.852 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:34:12.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:12 smithi084 ceph-mon[32629]: osdmap e254: 8 total, 8 up, 8 in 2024-09-16T09:34:12.921 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:12.922 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:12.922 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:34:13.023 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:34:13.056 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:13.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:13.058 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:34:13.158 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:34:13.193 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:13.194 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:13.195 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:34:13.295 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:34:13.329 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:13.329 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:13.330 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:34:13.410 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:34:13 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:34:13.430 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:34:13.478 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:13.478 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:13.479 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:34:13.579 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:34:13.620 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:13.620 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:13.621 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:34:13.711 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 43 16 68711 68695 99.8335 68.8125 0.0111762 0.0100099 2024-09-16T09:34:13.721 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:34:13.760 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:13.760 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:13.761 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:34:13.861 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:34:13.898 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:13.899 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:13.899 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:34:13.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:13 smithi084 ceph-mon[32629]: pgmap v444: 178 pgs: 2 active+recovery_wait+degraded, 4 active+recovery_wait+undersized+degraded+remapped, 7 active+recovering+undersized+remapped, 165 active+clean; 3.9 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 811 B/s rd, 53 MiB/s wr, 853 op/s; 10424/190959 objects degraded (5.459%); 12990/190959 objects misplaced (6.803%); 22 MiB/s, 345 objects/s recovering 2024-09-16T09:34:13.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:13 smithi084 ceph-mon[32629]: osdmap e255: 8 total, 8 up, 8 in 2024-09-16T09:34:13.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:13 smithi084 ceph-mon[32629]: 9.16 scrub starts 2024-09-16T09:34:13.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:13 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:34:13.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:13 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:34:13.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:13 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:34:13.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:13 smithi070 ceph-mon[30388]: pgmap v444: 178 pgs: 2 active+recovery_wait+degraded, 4 active+recovery_wait+undersized+degraded+remapped, 7 active+recovering+undersized+remapped, 165 active+clean; 3.9 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 811 B/s rd, 53 MiB/s wr, 853 op/s; 10424/190959 objects degraded (5.459%); 12990/190959 objects misplaced (6.803%); 22 MiB/s, 345 objects/s recovering 2024-09-16T09:34:13.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:13 smithi070 ceph-mon[30388]: osdmap e255: 8 total, 8 up, 8 in 2024-09-16T09:34:13.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:13 smithi070 ceph-mon[30388]: 9.16 scrub starts 2024-09-16T09:34:13.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:13 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:34:13.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:13 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:34:13.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:13 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:34:13.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:13 smithi070 ceph-mon[40691]: pgmap v444: 178 pgs: 2 active+recovery_wait+degraded, 4 active+recovery_wait+undersized+degraded+remapped, 7 active+recovering+undersized+remapped, 165 active+clean; 3.9 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 811 B/s rd, 53 MiB/s wr, 853 op/s; 10424/190959 objects degraded (5.459%); 12990/190959 objects misplaced (6.803%); 22 MiB/s, 345 objects/s recovering 2024-09-16T09:34:13.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:13 smithi070 ceph-mon[40691]: osdmap e255: 8 total, 8 up, 8 in 2024-09-16T09:34:13.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:13 smithi070 ceph-mon[40691]: 9.16 scrub starts 2024-09-16T09:34:13.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:13 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:34:13.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:13 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:34:13.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:13 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:34:14.001 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:34:14.037 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:14.037 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:14.038 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:34:14.086 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:34:14.086 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:34:14.139 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:34:14.171 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:14.172 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:14.172 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:34:14.273 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:34:14.309 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:14.310 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:14.311 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:34:14.411 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:34:14.452 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:14.452 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:14.453 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:34:14.554 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:34:14.593 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:14.593 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:14.594 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:34:14.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:14 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:34:14.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:14 smithi070 ceph-mon[30388]: 9.13 scrub starts 2024-09-16T09:34:14.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:14 smithi070 ceph-mon[30388]: Health check update: Degraded data redundancy: 10424/190959 objects degraded (5.459%), 6 pgs degraded (PG_DEGRADED) 2024-09-16T09:34:14.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:14 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:34:14.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:14 smithi070 ceph-mon[40691]: 9.13 scrub starts 2024-09-16T09:34:14.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:14 smithi070 ceph-mon[40691]: Health check update: Degraded data redundancy: 10424/190959 objects degraded (5.459%), 6 pgs degraded (PG_DEGRADED) 2024-09-16T09:34:14.695 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:34:14.711 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 44 16 69865 69849 99.2036 72.125 0.00262641 0.0100592 2024-09-16T09:34:14.719 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:14.719 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:14.720 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:34:14.821 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:34:14.862 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:14.863 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:14.864 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:34:14.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:14 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:34:14.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:14 smithi084 ceph-mon[32629]: 9.13 scrub starts 2024-09-16T09:34:14.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:14 smithi084 ceph-mon[32629]: Health check update: Degraded data redundancy: 10424/190959 objects degraded (5.459%), 6 pgs degraded (PG_DEGRADED) 2024-09-16T09:34:14.965 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:34:14.995 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:14.995 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:14.995 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:34:15.096 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:34:15.118 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:15.118 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:15.118 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:34:15.219 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:34:15.261 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:15.262 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:15.262 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:34:15.363 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:34:15.397 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:15.398 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:15.399 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:34:15.499 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:34:15.536 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:15.537 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:15.537 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:34:15.638 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:34:15.677 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:15.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:15.678 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:34:15.711 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 45 15 70905 70890 98.4449 65.0625 0.00750226 0.0101467 2024-09-16T09:34:15.778 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:34:15.800 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:15.800 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:15.801 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:34:15.902 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:34:15.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:15 smithi084 ceph-mon[32629]: pgmap v446: 178 pgs: 2 active+recovery_wait+degraded, 3 active+recovery_wait+undersized+degraded+remapped, 6 active+recovering+undersized+remapped, 167 active+clean; 4.1 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.7 KiB/s rd, 98 MiB/s wr, 1.57k op/s; 8837/204153 objects degraded (4.329%); 9151/204153 objects misplaced (4.482%); 51 MiB/s, 820 objects/s recovering 2024-09-16T09:34:15.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:15 smithi070 ceph-mon[30388]: pgmap v446: 178 pgs: 2 active+recovery_wait+degraded, 3 active+recovery_wait+undersized+degraded+remapped, 6 active+recovering+undersized+remapped, 167 active+clean; 4.1 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.7 KiB/s rd, 98 MiB/s wr, 1.57k op/s; 8837/204153 objects degraded (4.329%); 9151/204153 objects misplaced (4.482%); 51 MiB/s, 820 objects/s recovering 2024-09-16T09:34:15.920 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:15 smithi070 ceph-mon[40691]: pgmap v446: 178 pgs: 2 active+recovery_wait+degraded, 3 active+recovery_wait+undersized+degraded+remapped, 6 active+recovering+undersized+remapped, 167 active+clean; 4.1 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.7 KiB/s rd, 98 MiB/s wr, 1.57k op/s; 8837/204153 objects degraded (4.329%); 9151/204153 objects misplaced (4.482%); 51 MiB/s, 820 objects/s recovering 2024-09-16T09:34:15.931 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:15.932 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:15.932 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:34:16.033 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:34:16.043 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd unset noscrub 2024-09-16T09:34:16.065 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:16.065 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:16.066 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:34:16.167 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:34:16.199 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:16.199 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:16.200 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:34:16.301 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:34:16.322 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:16.323 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:16.323 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:34:16.424 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:34:16.449 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:16.450 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:16.450 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:34:16.551 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:34:16.594 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:16.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:16.595 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:34:16.696 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:34:16.711 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 46 16 71304 71288 96.8456 24.875 0.0151593 0.0102976 2024-09-16T09:34:16.717 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:16.718 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:16.718 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:34:16.819 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:34:16.861 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:16.861 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:16.861 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:34:16.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:16 smithi084 ceph-mon[32629]: osdmap e256: 8 total, 8 up, 8 in 2024-09-16T09:34:16.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:16 smithi084 ceph-mon[32629]: pgmap v448: 178 pgs: 2 active+recovery_wait+degraded, 3 active+recovery_wait+undersized+degraded+remapped, 6 active+recovering+undersized+remapped, 167 active+clean; 4.1 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 853 B/s rd, 46 MiB/s wr, 733 op/s; 8837/204153 objects degraded (4.329%); 9151/204153 objects misplaced (4.482%); 29 MiB/s, 456 objects/s recovering 2024-09-16T09:34:16.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:16 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:34:16.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:16 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:34:16.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:16 smithi070 ceph-mon[30388]: osdmap e256: 8 total, 8 up, 8 in 2024-09-16T09:34:16.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:16 smithi070 ceph-mon[30388]: pgmap v448: 178 pgs: 2 active+recovery_wait+degraded, 3 active+recovery_wait+undersized+degraded+remapped, 6 active+recovering+undersized+remapped, 167 active+clean; 4.1 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 853 B/s rd, 46 MiB/s wr, 733 op/s; 8837/204153 objects degraded (4.329%); 9151/204153 objects misplaced (4.482%); 29 MiB/s, 456 objects/s recovering 2024-09-16T09:34:16.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:16 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:34:16.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:16 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:34:16.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:16 smithi070 ceph-mon[40691]: osdmap e256: 8 total, 8 up, 8 in 2024-09-16T09:34:16.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:16 smithi070 ceph-mon[40691]: pgmap v448: 178 pgs: 2 active+recovery_wait+degraded, 3 active+recovery_wait+undersized+degraded+remapped, 6 active+recovering+undersized+remapped, 167 active+clean; 4.1 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 853 B/s rd, 46 MiB/s wr, 733 op/s; 8837/204153 objects degraded (4.329%); 9151/204153 objects misplaced (4.482%); 29 MiB/s, 456 objects/s recovering 2024-09-16T09:34:16.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:16 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:34:16.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:16 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:34:16.962 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:34:16.995 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:16.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:16.997 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:34:17.097 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:34:17.133 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:17.134 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:17.134 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:34:17.236 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:34:17.270 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:17.271 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:17.271 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:34:17.372 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:34:17.407 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:17.408 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:17.409 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:34:17.509 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:34:17.532 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:17.533 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:17.533 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:34:17.634 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:34:17.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:17 smithi070 ceph-mon[30388]: osdmap e257: 8 total, 8 up, 8 in 2024-09-16T09:34:17.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:17 smithi070 ceph-mon[40691]: osdmap e257: 8 total, 8 up, 8 in 2024-09-16T09:34:17.671 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:17.672 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:17.672 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:34:17.712 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 47 16 72306 72290 96.1157 62.625 0.00585317 0.0103978 2024-09-16T09:34:17.773 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:34:17.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:17.808 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:17.809 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:34:17.909 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:34:17.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:17 smithi084 ceph-mon[32629]: osdmap e257: 8 total, 8 up, 8 in 2024-09-16T09:34:17.932 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:17.933 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:17.933 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:34:18.034 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:34:18.066 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:18.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:18.067 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:34:18.168 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:34:18.200 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:18.200 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:18.201 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:34:18.301 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:34:18.322 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:18.323 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:18.323 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:34:18.424 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:34:18.454 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:18.454 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:18.455 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:34:18.556 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:34:18.593 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:18.594 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:18.594 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:34:18.695 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:34:18.712 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 48 16 72829 72813 94.7943 32.6875 0.00239162 0.0105421 2024-09-16T09:34:18.722 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:18.722 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:18.723 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:34:18.823 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:34:18.865 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:18.866 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:18.866 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:34:18.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:18 smithi084 ceph-mon[32629]: 9.12 scrub starts 2024-09-16T09:34:18.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:18 smithi084 ceph-mon[32629]: osdmap e258: 8 total, 8 up, 8 in 2024-09-16T09:34:18.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:18 smithi084 ceph-mon[32629]: pgmap v451: 178 pgs: 1 active+recovery_wait+degraded, 1 creating+peering, 3 active+recovery_wait+undersized+degraded+remapped, 5 active+recovering+undersized+remapped, 168 active+clean; 4.2 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 978 B/s rd, 58 MiB/s wr, 926 op/s; 8833/205488 objects degraded (4.299%); 8427/205488 objects misplaced (4.101%); 37 MiB/s, 593 objects/s recovering 2024-09-16T09:34:18.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:18 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:34:18.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:18 smithi070 ceph-mon[30388]: 9.12 scrub starts 2024-09-16T09:34:18.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:18 smithi070 ceph-mon[30388]: osdmap e258: 8 total, 8 up, 8 in 2024-09-16T09:34:18.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:18 smithi070 ceph-mon[30388]: pgmap v451: 178 pgs: 1 active+recovery_wait+degraded, 1 creating+peering, 3 active+recovery_wait+undersized+degraded+remapped, 5 active+recovering+undersized+remapped, 168 active+clean; 4.2 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 978 B/s rd, 58 MiB/s wr, 926 op/s; 8833/205488 objects degraded (4.299%); 8427/205488 objects misplaced (4.101%); 37 MiB/s, 593 objects/s recovering 2024-09-16T09:34:18.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:18 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:34:18.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:18 smithi070 ceph-mon[40691]: 9.12 scrub starts 2024-09-16T09:34:18.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:18 smithi070 ceph-mon[40691]: osdmap e258: 8 total, 8 up, 8 in 2024-09-16T09:34:18.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:18 smithi070 ceph-mon[40691]: pgmap v451: 178 pgs: 1 active+recovery_wait+degraded, 1 creating+peering, 3 active+recovery_wait+undersized+degraded+remapped, 5 active+recovering+undersized+remapped, 168 active+clean; 4.2 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 978 B/s rd, 58 MiB/s wr, 926 op/s; 8833/205488 objects degraded (4.299%); 8427/205488 objects misplaced (4.101%); 37 MiB/s, 593 objects/s recovering 2024-09-16T09:34:18.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:18 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:34:18.967 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:34:18.993 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:18.993 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:18.994 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:34:19.095 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:34:19.128 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:19.129 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:19.129 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:34:19.230 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:34:19.265 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:19.265 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:19.266 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:34:19.367 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:34:19.401 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:19.401 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:19.401 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:34:19.502 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:34:19.537 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:19.538 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:19.538 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:34:19.639 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:34:19.675 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:19.676 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:19.676 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:34:19.712 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 49 15 73916 73901 94.2474 68 0.0253114 0.010605 2024-09-16T09:34:19.777 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:34:19.798 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:19.799 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:19.799 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:34:19.900 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:34:19.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:19 smithi084 ceph-mon[32629]: osdmap e259: 8 total, 8 up, 8 in 2024-09-16T09:34:19.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:19 smithi084 ceph-mon[32629]: 9.18 scrub starts 2024-09-16T09:34:19.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:19 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3406782969' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:34:19.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:19 smithi084 ceph-mon[32629]: Health check update: Degraded data redundancy: 8833/205488 objects degraded (4.299%), 4 pgs degraded (PG_DEGRADED) 2024-09-16T09:34:19.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:19 smithi070 ceph-mon[30388]: osdmap e259: 8 total, 8 up, 8 in 2024-09-16T09:34:19.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:19 smithi070 ceph-mon[30388]: 9.18 scrub starts 2024-09-16T09:34:19.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:19 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3406782969' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:34:19.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:19 smithi070 ceph-mon[30388]: Health check update: Degraded data redundancy: 8833/205488 objects degraded (4.299%), 4 pgs degraded (PG_DEGRADED) 2024-09-16T09:34:19.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:19 smithi070 ceph-mon[40691]: osdmap e259: 8 total, 8 up, 8 in 2024-09-16T09:34:19.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:19 smithi070 ceph-mon[40691]: 9.18 scrub starts 2024-09-16T09:34:19.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:19 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3406782969' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:34:19.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:19 smithi070 ceph-mon[40691]: Health check update: Degraded data redundancy: 8833/205488 objects degraded (4.299%), 4 pgs degraded (PG_DEGRADED) 2024-09-16T09:34:19.923 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:19.923 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:19.924 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:34:20.024 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:34:20.064 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:20.065 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:20.065 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:34:20.166 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:34:20.209 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:20.209 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:20.210 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:34:20.310 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:34:20.350 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:20.351 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:34:20.351 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:34:20.351 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:34:20.351 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:34:20.351 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:34:20.351 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:34:20.351 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:34:20.352 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:34:20.352 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:34:20.352 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:34:20.352 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:34:20.352 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:34:20.352 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:34:20.352 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:34:20.352 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:20.353 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:34:20.353 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:34:20.353 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:34:20.454 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:34:20.490 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:20.491 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:20.492 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:34:20.592 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:34:20.622 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:20.623 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:20.623 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:34:20.712 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 50 16 74920 74904 93.6161 62.6875 0.00253687 0.0106382 2024-09-16T09:34:20.724 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:34:20.745 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:20.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:20.747 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:34:20.847 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:34:20.873 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:20.874 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:20.874 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:34:20.975 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:34:21.005 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:21.005 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:21.006 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:34:21.106 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:34:21.131 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:21.132 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:21.132 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:34:21.233 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:34:21.257 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:21.258 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:21.258 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:34:21.359 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:34:21.391 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:21.391 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:21.392 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:34:21.493 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:34:21.526 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:21.527 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:21.527 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:34:21.628 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:34:21.652 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:21.652 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:21.653 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:34:21.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:21 smithi084 ceph-mon[32629]: pgmap v453: 178 pgs: 1 creating+peering, 1 active+recovery_wait+undersized+degraded+remapped, 6 active+recovering+undersized+remapped, 170 active+clean; 4.4 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 68 MiB/s wr, 1.10k op/s; 3540/217923 objects degraded (1.624%); 10539/217923 objects misplaced (4.836%); 29 MiB/s, 462 objects/s recovering 2024-09-16T09:34:21.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:21 smithi084 ceph-mon[32629]: osdmap e260: 8 total, 8 up, 8 in 2024-09-16T09:34:21.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:21 smithi070 ceph-mon[30388]: pgmap v453: 178 pgs: 1 creating+peering, 1 active+recovery_wait+undersized+degraded+remapped, 6 active+recovering+undersized+remapped, 170 active+clean; 4.4 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 68 MiB/s wr, 1.10k op/s; 3540/217923 objects degraded (1.624%); 10539/217923 objects misplaced (4.836%); 29 MiB/s, 462 objects/s recovering 2024-09-16T09:34:21.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:21 smithi070 ceph-mon[30388]: osdmap e260: 8 total, 8 up, 8 in 2024-09-16T09:34:21.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:21 smithi070 ceph-mon[40691]: pgmap v453: 178 pgs: 1 creating+peering, 1 active+recovery_wait+undersized+degraded+remapped, 6 active+recovering+undersized+remapped, 170 active+clean; 4.4 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 68 MiB/s wr, 1.10k op/s; 3540/217923 objects degraded (1.624%); 10539/217923 objects misplaced (4.836%); 29 MiB/s, 462 objects/s recovering 2024-09-16T09:34:21.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:21 smithi070 ceph-mon[40691]: osdmap e260: 8 total, 8 up, 8 in 2024-09-16T09:34:21.712 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 51 16 75491 75475 92.4803 35.6875 0.011205 0.010808 2024-09-16T09:34:21.754 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:34:21.793 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:21.794 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:21.794 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:34:21.895 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:34:21.933 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:21.934 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:21.935 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:34:22.036 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:34:22.070 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:22.071 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:22.071 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:34:22.172 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:34:22.209 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:22.210 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:22.210 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:34:22.267 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:34:21 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:34:21] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:34:22.311 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:34:22.345 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:22.345 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:22.346 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:34:22.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:22 smithi084 ceph-mon[32629]: osdmap e261: 8 total, 8 up, 8 in 2024-09-16T09:34:22.446 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:34:22.480 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:22.481 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:22.482 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:34:22.582 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:34:22.618 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:22.618 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:22.619 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:34:22.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:22 smithi070 ceph-mon[30388]: osdmap e261: 8 total, 8 up, 8 in 2024-09-16T09:34:22.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:22 smithi070 ceph-mon[40691]: osdmap e261: 8 total, 8 up, 8 in 2024-09-16T09:34:22.712 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 52 16 76686 76670 92.138 74.6875 0.00392246 0.010848 2024-09-16T09:34:22.720 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:34:22.743 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:22.743 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:22.744 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:34:22.845 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:34:22.888 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:22.889 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:22.889 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:34:22.990 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:34:23.026 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:23.026 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:23.027 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:34:23.127 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:34:23.166 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:23.167 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:23.168 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:34:23.268 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:34:23.306 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:23.307 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:23.308 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:34:23.409 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:34:23.410 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:34:23 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:34:23.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:23 smithi084 ceph-mon[32629]: 9.17 scrub starts 2024-09-16T09:34:23.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:23 smithi084 ceph-mon[32629]: pgmap v456: 178 pgs: 1 creating+peering, 1 active+recovery_wait+undersized+degraded+remapped, 6 active+recovering+undersized+remapped, 170 active+clean; 4.4 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 70 MiB/s wr, 1.11k op/s; 3540/217923 objects degraded (1.624%); 10539/217923 objects misplaced (4.836%); 29 MiB/s, 469 objects/s recovering 2024-09-16T09:34:23.444 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:23.445 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:23.445 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:34:23.546 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:34:23.583 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:23.583 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:23.584 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:34:23.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:23 smithi070 ceph-mon[30388]: 9.17 scrub starts 2024-09-16T09:34:23.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:23 smithi070 ceph-mon[30388]: pgmap v456: 178 pgs: 1 creating+peering, 1 active+recovery_wait+undersized+degraded+remapped, 6 active+recovering+undersized+remapped, 170 active+clean; 4.4 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 70 MiB/s wr, 1.11k op/s; 3540/217923 objects degraded (1.624%); 10539/217923 objects misplaced (4.836%); 29 MiB/s, 469 objects/s recovering 2024-09-16T09:34:23.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:23 smithi070 ceph-mon[40691]: 9.17 scrub starts 2024-09-16T09:34:23.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:23 smithi070 ceph-mon[40691]: pgmap v456: 178 pgs: 1 creating+peering, 1 active+recovery_wait+undersized+degraded+remapped, 6 active+recovering+undersized+remapped, 170 active+clean; 4.4 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 70 MiB/s wr, 1.11k op/s; 3540/217923 objects degraded (1.624%); 10539/217923 objects misplaced (4.836%); 29 MiB/s, 469 objects/s recovering 2024-09-16T09:34:23.684 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:34:23.712 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 53 16 77797 77781 91.7096 69.4375 0.00667492 0.0108981 2024-09-16T09:34:23.723 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:23.724 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:23.724 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:34:23.825 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:34:23.862 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:23.863 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:23.863 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:34:23.964 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:34:24.002 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:24.002 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:24.003 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:34:24.104 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:34:24.136 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:24.136 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:24.137 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:34:24.237 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:34:24.272 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:24.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:24.273 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:34:24.374 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:34:24.411 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:24.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:24.412 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:34:24.513 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:34:24.523 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T09:34:24.523 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:34:24.524 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:24 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:34:24.524 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:24 smithi070 ceph-mon[30388]: Health check update: Degraded data redundancy: 3540/217923 objects degraded (1.624%), 1 pg degraded (PG_DEGRADED) 2024-09-16T09:34:24.524 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:24 smithi070 ceph-mon[30388]: osdmap e262: 8 total, 8 up, 8 in 2024-09-16T09:34:24.524 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:24 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:34:24.524 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:24 smithi070 ceph-mon[40691]: Health check update: Degraded data redundancy: 3540/217923 objects degraded (1.624%), 1 pg degraded (PG_DEGRADED) 2024-09-16T09:34:24.524 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:24 smithi070 ceph-mon[40691]: osdmap e262: 8 total, 8 up, 8 in 2024-09-16T09:34:24.539 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:24.540 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:24.540 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:34:24.641 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:34:24.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:24 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:34:24.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:24 smithi084 ceph-mon[32629]: Health check update: Degraded data redundancy: 3540/217923 objects degraded (1.624%), 1 pg degraded (PG_DEGRADED) 2024-09-16T09:34:24.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:24 smithi084 ceph-mon[32629]: osdmap e262: 8 total, 8 up, 8 in 2024-09-16T09:34:24.677 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:24.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:24.679 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:34:24.712 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 54 16 78944 78928 91.3387 71.6875 0.00264901 0.0109001 2024-09-16T09:34:24.779 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:34:24.804 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:24.805 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:24.805 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:34:24.906 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:34:24.930 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:24.931 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:24.931 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:34:25.032 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:34:25.056 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:25.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:25.057 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:34:25.158 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:34:25.186 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:25.186 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:25.187 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:34:25.287 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:34:25.308 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:34:25 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: 2024-09-16T09:34:25.014+0000 7fc9f961c640 -1 mgr.server reply reply (3) No such process no change 2024-09-16T09:34:25.331 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:25.332 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:25.332 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:34:25.433 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:34:25.467 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:25.468 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:25.468 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:34:25.569 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:34:25.606 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:25.607 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:25.607 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:34:25.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:25 smithi084 ceph-mon[32629]: pgmap v457: 178 pgs: 6 active+recovering+undersized+remapped, 172 active+clean; 4.7 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.7 KiB/s rd, 90 MiB/s wr, 1.44k op/s; 10298/231372 objects misplaced (4.451%); 38 MiB/s, 609 objects/s recovering 2024-09-16T09:34:25.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:25 smithi084 ceph-mon[32629]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 3540/217923 objects degraded (1.624%), 1 pg degraded) 2024-09-16T09:34:25.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:25 smithi084 ceph-mon[32629]: 9.6 scrub starts 2024-09-16T09:34:25.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:25 smithi084 ceph-mon[32629]: 9.11 scrub starts 2024-09-16T09:34:25.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:25 smithi084 ceph-mon[32629]: 9.14 scrub starts 2024-09-16T09:34:25.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:25 smithi084 ceph-mon[32629]: 9.1 scrub starts 2024-09-16T09:34:25.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:25 smithi084 ceph-mon[32629]: osdmap e263: 8 total, 8 up, 8 in 2024-09-16T09:34:25.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:25 smithi070 ceph-mon[30388]: pgmap v457: 178 pgs: 6 active+recovering+undersized+remapped, 172 active+clean; 4.7 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.7 KiB/s rd, 90 MiB/s wr, 1.44k op/s; 10298/231372 objects misplaced (4.451%); 38 MiB/s, 609 objects/s recovering 2024-09-16T09:34:25.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:25 smithi070 ceph-mon[30388]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 3540/217923 objects degraded (1.624%), 1 pg degraded) 2024-09-16T09:34:25.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:25 smithi070 ceph-mon[30388]: 9.6 scrub starts 2024-09-16T09:34:25.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:25 smithi070 ceph-mon[30388]: 9.11 scrub starts 2024-09-16T09:34:25.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:25 smithi070 ceph-mon[30388]: 9.14 scrub starts 2024-09-16T09:34:25.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:25 smithi070 ceph-mon[30388]: 9.1 scrub starts 2024-09-16T09:34:25.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:25 smithi070 ceph-mon[30388]: osdmap e263: 8 total, 8 up, 8 in 2024-09-16T09:34:25.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:25 smithi070 ceph-mon[40691]: pgmap v457: 178 pgs: 6 active+recovering+undersized+remapped, 172 active+clean; 4.7 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.7 KiB/s rd, 90 MiB/s wr, 1.44k op/s; 10298/231372 objects misplaced (4.451%); 38 MiB/s, 609 objects/s recovering 2024-09-16T09:34:25.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:25 smithi070 ceph-mon[40691]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 3540/217923 objects degraded (1.624%), 1 pg degraded) 2024-09-16T09:34:25.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:25 smithi070 ceph-mon[40691]: 9.6 scrub starts 2024-09-16T09:34:25.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:25 smithi070 ceph-mon[40691]: 9.11 scrub starts 2024-09-16T09:34:25.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:25 smithi070 ceph-mon[40691]: 9.14 scrub starts 2024-09-16T09:34:25.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:25 smithi070 ceph-mon[40691]: 9.1 scrub starts 2024-09-16T09:34:25.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:25 smithi070 ceph-mon[40691]: osdmap e263: 8 total, 8 up, 8 in 2024-09-16T09:34:25.708 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:34:25.712 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 55 16 79567 79551 90.386 38.9375 0.00728923 0.0110582 2024-09-16T09:34:25.746 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:25.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:25.747 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:34:25.847 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:34:25.884 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:25.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:25.885 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:34:25.986 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:34:26.023 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:26.023 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:26.024 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:34:26.125 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:34:26.162 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:26.162 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:26.163 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:34:26.263 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:34:26.298 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:26.298 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:26.299 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:34:26.399 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:34:26.438 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:26.439 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:26.439 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:34:26.540 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:34:26.586 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:26.586 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:26.586 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:34:26.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:26 smithi084 ceph-mon[32629]: 9.9 scrub starts 2024-09-16T09:34:26.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:26 smithi084 ceph-mon[32629]: from='client.15651 -' entity='client.admin' cmd=[{"prefix": "osd test-reweight-by-utilization", "oload": 110, "max_change": 0.05, "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:34:26.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:26 smithi084 ceph-mon[32629]: 9.15 scrub starts 2024-09-16T09:34:26.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:26 smithi084 ceph-mon[32629]: 9.0 scrub starts 2024-09-16T09:34:26.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:26 smithi070 ceph-mon[30388]: 9.9 scrub starts 2024-09-16T09:34:26.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:26 smithi070 ceph-mon[30388]: from='client.15651 -' entity='client.admin' cmd=[{"prefix": "osd test-reweight-by-utilization", "oload": 110, "max_change": 0.05, "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:34:26.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:26 smithi070 ceph-mon[30388]: 9.15 scrub starts 2024-09-16T09:34:26.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:26 smithi070 ceph-mon[30388]: 9.0 scrub starts 2024-09-16T09:34:26.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:26 smithi070 ceph-mon[40691]: 9.9 scrub starts 2024-09-16T09:34:26.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:26 smithi070 ceph-mon[40691]: from='client.15651 -' entity='client.admin' cmd=[{"prefix": "osd test-reweight-by-utilization", "oload": 110, "max_change": 0.05, "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:34:26.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:26 smithi070 ceph-mon[40691]: 9.15 scrub starts 2024-09-16T09:34:26.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:26 smithi070 ceph-mon[40691]: 9.0 scrub starts 2024-09-16T09:34:26.687 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:34:26.712 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 56 16 80889 80873 90.2473 82.625 0.00438102 0.0110761 2024-09-16T09:34:26.720 INFO:teuthology.orchestra.run.smithi070.stdout:moved 9 / 534 (1.68539%) 2024-09-16T09:34:26.720 INFO:teuthology.orchestra.run.smithi070.stdout:avg 66.75 2024-09-16T09:34:26.720 INFO:teuthology.orchestra.run.smithi070.stdout:stddev 9.79477 -> 8.04285 (expected baseline 7.6424) 2024-09-16T09:34:26.720 INFO:teuthology.orchestra.run.smithi070.stdout:min osd.2 with 48 -> 52 pgs (0.719101 -> 0.779026 * mean) 2024-09-16T09:34:26.720 INFO:teuthology.orchestra.run.smithi070.stdout:max osd.7 with 79 -> 75 pgs (1.18352 -> 1.1236 * mean) 2024-09-16T09:34:26.721 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:34:26.721 INFO:teuthology.orchestra.run.smithi070.stdout:oload 110 2024-09-16T09:34:26.721 INFO:teuthology.orchestra.run.smithi070.stdout:max_change 0.05 2024-09-16T09:34:26.721 INFO:teuthology.orchestra.run.smithi070.stdout:max_change_osds 4 2024-09-16T09:34:26.721 INFO:teuthology.orchestra.run.smithi070.stdout:average_utilization 0.0251 2024-09-16T09:34:26.721 INFO:teuthology.orchestra.run.smithi070.stdout:overload_utilization 0.0276 2024-09-16T09:34:26.721 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7 weight 0.9500 -> 0.9000 2024-09-16T09:34:26.721 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1 weight 0.9500 -> 0.9000 2024-09-16T09:34:26.721 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5 weight 0.9500 -> 0.9000 2024-09-16T09:34:26.721 INFO:teuthology.orchestra.run.smithi070.stderr:no change 2024-09-16T09:34:26.722 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:26.723 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:26.724 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:34:26.824 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:34:26.865 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:26.865 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:26.865 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:34:26.966 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:34:27.006 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:27.007 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:27.007 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:34:27.108 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:34:27.145 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:27.146 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:27.146 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:34:27.247 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:34:27.284 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:27.285 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:27.285 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:34:27.381 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T09:34:27.386 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:34:27.419 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:27.420 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:27.421 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:34:27.521 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:34:27.535 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.2 dump_historic_ops 2024-09-16T09:34:27.572 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:27.573 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:27.574 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:34:27.608 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:27 smithi070 ceph-mon[30388]: pgmap v460: 178 pgs: 6 active+recovering+undersized+remapped, 172 active+clean; 4.7 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 944 B/s rd, 52 MiB/s wr, 828 op/s; 10298/231372 objects misplaced (4.451%); 24 MiB/s, 385 objects/s recovering 2024-09-16T09:34:27.608 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:27 smithi070 ceph-mon[40691]: pgmap v460: 178 pgs: 6 active+recovering+undersized+remapped, 172 active+clean; 4.7 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 944 B/s rd, 52 MiB/s wr, 828 op/s; 10298/231372 objects misplaced (4.451%); 24 MiB/s, 385 objects/s recovering 2024-09-16T09:34:27.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:27 smithi084 ceph-mon[32629]: pgmap v460: 178 pgs: 6 active+recovering+undersized+remapped, 172 active+clean; 4.7 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 944 B/s rd, 52 MiB/s wr, 828 op/s; 10298/231372 objects misplaced (4.451%); 24 MiB/s, 385 objects/s recovering 2024-09-16T09:34:27.675 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:34:27.714 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 57 16 82197 82181 90.0956 81.75 0.00630633 0.0110722 2024-09-16T09:34:27.716 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:27.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:27.718 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:34:27.819 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:34:27.850 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:27.851 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:27.851 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:34:27.952 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:34:27.979 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:27.980 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:27.980 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:34:28.044 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '110', 'type': 'test-reweight-by-utilization'} 2024-09-16T09:34:28.044 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd test-reweight-by-utilization 110 3.0 2024-09-16T09:34:28.082 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:34:28.139 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:28.140 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:28.140 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:34:28.160 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:34:28.160 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:34:28.163 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:34:28.241 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:34:28.264 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:28.264 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:28.265 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:34:28.365 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:34:28.408 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:28.409 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:28.409 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:34:28.510 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:34:28.550 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:28.551 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:28.551 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:34:28.652 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:34:28.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:28 smithi084 ceph-mon[32629]: osdmap e264: 8 total, 8 up, 8 in 2024-09-16T09:34:28.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:28 smithi084 ceph-mon[32629]: osdmap e265: 8 total, 8 up, 8 in 2024-09-16T09:34:28.661 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:28 smithi070 ceph-mon[40691]: osdmap e264: 8 total, 8 up, 8 in 2024-09-16T09:34:28.661 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:28 smithi070 ceph-mon[40691]: osdmap e265: 8 total, 8 up, 8 in 2024-09-16T09:34:28.662 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:28 smithi070 ceph-mon[30388]: osdmap e264: 8 total, 8 up, 8 in 2024-09-16T09:34:28.662 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:28 smithi070 ceph-mon[30388]: osdmap e265: 8 total, 8 up, 8 in 2024-09-16T09:34:28.696 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:28.697 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:28.697 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:34:28.714 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 58 16 82975 82959 89.3806 48.625 0.0118772 0.011183 2024-09-16T09:34:28.798 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:34:28.846 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:28.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:28.847 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:34:28.948 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:34:28.989 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:28.990 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:28.991 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:34:29.091 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:34:29.125 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:29.125 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:29.126 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:34:29.226 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:34:29.260 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:29.261 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:29.261 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:34:29.362 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:34:29.410 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:29.410 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:29.411 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:34:29.511 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:34:29.545 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:29.546 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:29.546 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:34:29.647 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:34:29.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:29 smithi084 ceph-mon[32629]: pgmap v462: 178 pgs: 6 active+recovering+undersized+remapped, 172 active+clean; 4.7 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 853 B/s rd, 47 MiB/s wr, 747 op/s; 10298/231372 objects misplaced (4.451%); 22 MiB/s, 347 objects/s recovering 2024-09-16T09:34:29.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:29 smithi084 ceph-mon[32629]: 9.10 scrub starts 2024-09-16T09:34:29.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:29 smithi070 ceph-mon[30388]: pgmap v462: 178 pgs: 6 active+recovering+undersized+remapped, 172 active+clean; 4.7 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 853 B/s rd, 47 MiB/s wr, 747 op/s; 10298/231372 objects misplaced (4.451%); 22 MiB/s, 347 objects/s recovering 2024-09-16T09:34:29.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:29 smithi070 ceph-mon[30388]: 9.10 scrub starts 2024-09-16T09:34:29.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:29 smithi070 ceph-mon[40691]: pgmap v462: 178 pgs: 6 active+recovering+undersized+remapped, 172 active+clean; 4.7 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 853 B/s rd, 47 MiB/s wr, 747 op/s; 10298/231372 objects misplaced (4.451%); 22 MiB/s, 347 objects/s recovering 2024-09-16T09:34:29.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:29 smithi070 ceph-mon[40691]: 9.10 scrub starts 2024-09-16T09:34:29.683 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:29.683 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:29.684 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:34:29.715 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 59 16 84326 84310 89.2967 84.4375 0.00825138 0.0111932 2024-09-16T09:34:29.784 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:34:29.822 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:29.823 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:29.823 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:34:29.924 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:34:29.965 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:29.965 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:29.966 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:34:30.037 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T09:34:30.037 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:34:30.067 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:34:30.100 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:30.101 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:30.101 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:34:30.202 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:34:30.244 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:30.244 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:30.245 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:34:30.345 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:34:30.379 INFO:teuthology.orchestra.run.smithi070.stderr:noscrub is unset 2024-09-16T09:34:30.391 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:30.391 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:30.392 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:34:30.493 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:34:30.526 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:30.526 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:30.527 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:34:30.627 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:34:30.658 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:30 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/919828102' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:34:30.659 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:30 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:34:30.659 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:30 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1130655931' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:34:30.659 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:30.660 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:30.660 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:34:30.660 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:30 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/919828102' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:34:30.660 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:30 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:34:30.660 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:30 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1130655931' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:34:30.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:30 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/919828102' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:34:30.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:30 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:34:30.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:30 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1130655931' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:34:30.715 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:2024-09-16T09:34:30.714383+0000 min lat: 0.00127131 max lat: 2.16104 avg lat: 0.0111821 2024-09-16T09:34:30.715 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-16T09:34:30.715 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 60 16 85573 85557 89.1073 77.9375 0.00702262 0.0111821 2024-09-16T09:34:30.761 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:34:30.761 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:34:30.761 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:34:30.761 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:34:30.761 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:34:30.761 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:34:30.761 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:34:30.761 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:34:30.762 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:34:30.762 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:34:30.762 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:34:30.762 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:34:30.762 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:34:30.762 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:34:30.762 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:34:30.762 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:34:30.763 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:34:30.786 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:30.786 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:30.787 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:34:30.888 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:34:30.910 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:30.910 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:30.911 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:34:31.011 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:34:31.033 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:31.034 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:31.034 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:34:31.135 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:34:31.158 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:31.159 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:31.159 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:34:31.260 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:34:31.284 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:31.284 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:31.285 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:34:31.385 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:34:31.407 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:31.408 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:31.409 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:34:31.509 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:34:31.542 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:31.542 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:31.543 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:34:31.643 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:34:31.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:31 smithi084 ceph-mon[32629]: pgmap v464: 178 pgs: 1 creating+peering, 4 active+recovering+undersized+remapped, 173 active+clean; 5.0 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 944 B/s rd, 63 MiB/s wr, 1.00k op/s; 6361/247680 objects misplaced (2.568%); 26 MiB/s, 411 objects/s recovering 2024-09-16T09:34:31.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:31 smithi084 ceph-mon[32629]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:34:31.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:31 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-16T09:34:31.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:31 smithi084 ceph-mon[32629]: osdmap e266: 8 total, 8 up, 8 in 2024-09-16T09:34:31.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:31 smithi084 ceph-mon[32629]: 6.b scrub starts 2024-09-16T09:34:31.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:31 smithi084 ceph-mon[32629]: 6.b scrub ok 2024-09-16T09:34:31.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:31 smithi084 ceph-mon[32629]: 1.8 scrub starts 2024-09-16T09:34:31.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:31 smithi084 ceph-mon[32629]: 1.8 scrub ok 2024-09-16T09:34:31.661 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:31 smithi070 ceph-mon[30388]: pgmap v464: 178 pgs: 1 creating+peering, 4 active+recovering+undersized+remapped, 173 active+clean; 5.0 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 944 B/s rd, 63 MiB/s wr, 1.00k op/s; 6361/247680 objects misplaced (2.568%); 26 MiB/s, 411 objects/s recovering 2024-09-16T09:34:31.661 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:31 smithi070 ceph-mon[30388]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:34:31.661 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:31 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-16T09:34:31.661 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:31 smithi070 ceph-mon[30388]: osdmap e266: 8 total, 8 up, 8 in 2024-09-16T09:34:31.661 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:31 smithi070 ceph-mon[30388]: 6.b scrub starts 2024-09-16T09:34:31.661 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:31 smithi070 ceph-mon[30388]: 6.b scrub ok 2024-09-16T09:34:31.661 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:31 smithi070 ceph-mon[30388]: 1.8 scrub starts 2024-09-16T09:34:31.661 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:31 smithi070 ceph-mon[30388]: 1.8 scrub ok 2024-09-16T09:34:31.662 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:31 smithi070 ceph-mon[40691]: pgmap v464: 178 pgs: 1 creating+peering, 4 active+recovering+undersized+remapped, 173 active+clean; 5.0 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 944 B/s rd, 63 MiB/s wr, 1.00k op/s; 6361/247680 objects misplaced (2.568%); 26 MiB/s, 411 objects/s recovering 2024-09-16T09:34:31.662 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:31 smithi070 ceph-mon[40691]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:34:31.662 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:31 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-16T09:34:31.662 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:31 smithi070 ceph-mon[40691]: osdmap e266: 8 total, 8 up, 8 in 2024-09-16T09:34:31.662 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:31 smithi070 ceph-mon[40691]: 6.b scrub starts 2024-09-16T09:34:31.662 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:31 smithi070 ceph-mon[40691]: 6.b scrub ok 2024-09-16T09:34:31.662 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:31 smithi070 ceph-mon[40691]: 1.8 scrub starts 2024-09-16T09:34:31.662 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:31 smithi070 ceph-mon[40691]: 1.8 scrub ok 2024-09-16T09:34:31.674 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:31.675 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:31.676 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:34:31.714 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 61 16 85936 85920 88.0185 22.6875 0.0116864 0.0113319 2024-09-16T09:34:31.777 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:34:31.805 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:31.806 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:31.807 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:34:31.907 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:34:31.934 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:31.935 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:31.936 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:34:32.037 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:34:32.060 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:32.061 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:32.061 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:34:32.162 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:34:32.184 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:32.184 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:32.185 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:34:32.286 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:34:32.309 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:32.309 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:32.310 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:34:32.371 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:34:31 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:34:31] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:34:32.411 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:34:32.442 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:32.443 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:32.443 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:34:32.491 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:34:32.539 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.3 dump_ops_in_flight 2024-09-16T09:34:32.544 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:34:32.618 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:32.619 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:32.619 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:34:32.629 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:32 smithi070 ceph-mon[30388]: osdmap e267: 8 total, 8 up, 8 in 2024-09-16T09:34:32.630 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:32 smithi070 ceph-mon[30388]: 9.19 scrub starts 2024-09-16T09:34:32.630 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:32 smithi070 ceph-mon[30388]: 9.19 scrub ok 2024-09-16T09:34:32.630 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:32 smithi070 ceph-mon[30388]: 1.3 scrub starts 2024-09-16T09:34:32.630 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:32 smithi070 ceph-mon[30388]: 1.3 scrub ok 2024-09-16T09:34:32.630 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:32 smithi070 ceph-mon[40691]: osdmap e267: 8 total, 8 up, 8 in 2024-09-16T09:34:32.630 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:32 smithi070 ceph-mon[40691]: 9.19 scrub starts 2024-09-16T09:34:32.630 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:32 smithi070 ceph-mon[40691]: 9.19 scrub ok 2024-09-16T09:34:32.630 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:32 smithi070 ceph-mon[40691]: 1.3 scrub starts 2024-09-16T09:34:32.630 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:32 smithi070 ceph-mon[40691]: 1.3 scrub ok 2024-09-16T09:34:32.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:32 smithi084 ceph-mon[32629]: osdmap e267: 8 total, 8 up, 8 in 2024-09-16T09:34:32.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:32 smithi084 ceph-mon[32629]: 9.19 scrub starts 2024-09-16T09:34:32.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:32 smithi084 ceph-mon[32629]: 9.19 scrub ok 2024-09-16T09:34:32.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:32 smithi084 ceph-mon[32629]: 1.3 scrub starts 2024-09-16T09:34:32.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:32 smithi084 ceph-mon[32629]: 1.3 scrub ok 2024-09-16T09:34:32.715 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 62 16 86626 86610 87.2944 43.125 0.00709133 0.0114508 2024-09-16T09:34:32.720 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:34:32.761 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:32.762 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:32.762 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:34:32.863 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:34:32.898 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:32.899 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:32.899 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:34:33.000 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:34:33.040 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:33.041 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:33.041 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:34:33.142 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:34:33.160 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:34:33 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:34:33.180 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:33.181 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:33.182 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:34:33.191 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:34:33.282 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:34:33.318 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:33.319 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:33.319 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:34:33.420 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:34:33.459 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:33.460 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:33.460 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:34:33.465 INFO:teuthology.orchestra.run.smithi070.stdout:moved 68 / 534 (12.7341%) 2024-09-16T09:34:33.465 INFO:teuthology.orchestra.run.smithi070.stdout:avg 66.75 2024-09-16T09:34:33.466 INFO:teuthology.orchestra.run.smithi070.stdout:stddev 9.79477 -> 12.4574 (expected baseline 7.6424) 2024-09-16T09:34:33.466 INFO:teuthology.orchestra.run.smithi070.stdout:min osd.2 with 48 -> 67 pgs (0.719101 -> 1.00375 * mean) 2024-09-16T09:34:33.466 INFO:teuthology.orchestra.run.smithi070.stdout:max osd.7 with 79 -> 46 pgs (1.18352 -> 0.689139 * mean) 2024-09-16T09:34:33.466 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:34:33.466 INFO:teuthology.orchestra.run.smithi070.stdout:oload 110 2024-09-16T09:34:33.466 INFO:teuthology.orchestra.run.smithi070.stdout:max_change 3 2024-09-16T09:34:33.466 INFO:teuthology.orchestra.run.smithi070.stdout:max_change_osds 4 2024-09-16T09:34:33.466 INFO:teuthology.orchestra.run.smithi070.stdout:average_utilization 0.0269 2024-09-16T09:34:33.466 INFO:teuthology.orchestra.run.smithi070.stdout:overload_utilization 0.0295 2024-09-16T09:34:33.466 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1 weight 0.9500 -> 0.6613 2024-09-16T09:34:33.466 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7 weight 0.9500 -> 0.6643 2024-09-16T09:34:33.466 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5 weight 0.9500 -> 0.7109 2024-09-16T09:34:33.467 INFO:teuthology.orchestra.run.smithi070.stderr:no change 2024-09-16T09:34:33.550 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-16T09:34:33.561 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:34:33.594 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:34:33 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: 2024-09-16T09:34:33.463+0000 7fc9f961c640 -1 mgr.server reply reply (3) No such process no change 2024-09-16T09:34:33.594 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:33 smithi070 ceph-mon[30388]: pgmap v467: 178 pgs: 1 creating+peering, 4 active+recovering+undersized+remapped, 173 active+clean; 5.0 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1.1 KiB/s rd, 78 MiB/s wr, 1.25k op/s; 6361/247680 objects misplaced (2.568%); 32 MiB/s, 512 objects/s recovering 2024-09-16T09:34:33.594 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:33 smithi070 ceph-mon[30388]: 1.b scrub starts 2024-09-16T09:34:33.594 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:33 smithi070 ceph-mon[30388]: 1.b scrub ok 2024-09-16T09:34:33.594 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:33 smithi070 ceph-mon[30388]: 6.14 scrub starts 2024-09-16T09:34:33.595 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:33 smithi070 ceph-mon[30388]: 6.14 scrub ok 2024-09-16T09:34:33.595 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:33 smithi070 ceph-mon[30388]: osdmap e268: 8 total, 8 up, 8 in 2024-09-16T09:34:33.595 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:33 smithi070 ceph-mon[30388]: 3.a scrub starts 2024-09-16T09:34:33.595 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:33 smithi070 ceph-mon[30388]: 3.a scrub ok 2024-09-16T09:34:33.595 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:33 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:34:33.595 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:33 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:34:33.595 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:33 smithi070 ceph-mon[40691]: pgmap v467: 178 pgs: 1 creating+peering, 4 active+recovering+undersized+remapped, 173 active+clean; 5.0 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1.1 KiB/s rd, 78 MiB/s wr, 1.25k op/s; 6361/247680 objects misplaced (2.568%); 32 MiB/s, 512 objects/s recovering 2024-09-16T09:34:33.595 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:33 smithi070 ceph-mon[40691]: 1.b scrub starts 2024-09-16T09:34:33.595 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:33 smithi070 ceph-mon[40691]: 1.b scrub ok 2024-09-16T09:34:33.595 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:33 smithi070 ceph-mon[40691]: 6.14 scrub starts 2024-09-16T09:34:33.595 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:33 smithi070 ceph-mon[40691]: 6.14 scrub ok 2024-09-16T09:34:33.596 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:33 smithi070 ceph-mon[40691]: osdmap e268: 8 total, 8 up, 8 in 2024-09-16T09:34:33.596 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:33 smithi070 ceph-mon[40691]: 3.a scrub starts 2024-09-16T09:34:33.596 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:33 smithi070 ceph-mon[40691]: 3.a scrub ok 2024-09-16T09:34:33.596 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:33 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:34:33.596 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:33 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:34:33.636 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:33.636 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:33.637 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:34:33.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:33 smithi084 ceph-mon[32629]: pgmap v467: 178 pgs: 1 creating+peering, 4 active+recovering+undersized+remapped, 173 active+clean; 5.0 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1.1 KiB/s rd, 78 MiB/s wr, 1.25k op/s; 6361/247680 objects misplaced (2.568%); 32 MiB/s, 512 objects/s recovering 2024-09-16T09:34:33.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:33 smithi084 ceph-mon[32629]: 1.b scrub starts 2024-09-16T09:34:33.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:33 smithi084 ceph-mon[32629]: 1.b scrub ok 2024-09-16T09:34:33.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:33 smithi084 ceph-mon[32629]: 6.14 scrub starts 2024-09-16T09:34:33.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:33 smithi084 ceph-mon[32629]: 6.14 scrub ok 2024-09-16T09:34:33.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:33 smithi084 ceph-mon[32629]: osdmap e268: 8 total, 8 up, 8 in 2024-09-16T09:34:33.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:33 smithi084 ceph-mon[32629]: 3.a scrub starts 2024-09-16T09:34:33.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:33 smithi084 ceph-mon[32629]: 3.a scrub ok 2024-09-16T09:34:33.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:33 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:34:33.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:33 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:34:33.715 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 63 15 88048 88033 87.3204 88.9375 0.00510443 0.0114475 2024-09-16T09:34:33.737 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:34:33.783 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:33.784 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:33.784 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:34:33.885 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:34:33.921 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:33.922 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:33.922 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:34:34.010 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T09:34:34.020 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T09:34:34.021 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [ 2024-09-16T09:34:34.021 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:34.021 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.15588.0:88438 9.c e268/229 9:3e8ee585:::benchmark_data_smithi070_137000_object88437:head v 268'5455, mlcod=268'5453)", 2024-09-16T09:34:34.021 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:34:34.017363+0000", 2024-09-16T09:34:34.021 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.0011448910000000001, 2024-09-16T09:34:34.021 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001170241, 2024-09-16T09:34:34.021 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:34:34.021 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:34:34.021 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T09:34:34.021 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:34:34.021 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:34.021 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:34:34.021 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:34.017363+0000", 2024-09-16T09:34:34.022 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:34:34.022 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:34.022 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:34.022 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:34:34.022 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:34.017363+0000", 2024-09-16T09:34:34.022 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:34:34.022 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:34.022 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:34.022 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:34:34.022 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:34.017364+0000", 2024-09-16T09:34:34.022 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.1399999999999996e-07 2024-09-16T09:34:34.022 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:34.022 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:34.022 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:34:34.023 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:34.017418+0000", 2024-09-16T09:34:34.023 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4367999999999999e-05 2024-09-16T09:34:34.023 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:34.023 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:34.023 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:34:34.023 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:34.017433+0000", 2024-09-16T09:34:34.023 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.4301000000000001e-05 2024-09-16T09:34:34.023 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:34.023 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:34.023 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:34:34.023 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:34.017441+0000", 2024-09-16T09:34:34.023 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.5779999999999999e-06 2024-09-16T09:34:34.023 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:34.024 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:34.024 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:34:34.024 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:34.017497+0000", 2024-09-16T09:34:34.024 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5621000000000002e-05 2024-09-16T09:34:34.024 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:34.024 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:34.024 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:34:34.024 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:34.017519+0000", 2024-09-16T09:34:34.024 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.2192e-05 2024-09-16T09:34:34.024 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:34:34.024 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:34:34.024 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:34:34.024 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:34.025 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:34.025 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.15588.0:88439 9.f e268/229 9:ff03d9a6:::benchmark_data_smithi070_137000_object88438:head v 268'5584, mlcod=268'5582)", 2024-09-16T09:34:34.025 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:34:34.018054+0000", 2024-09-16T09:34:34.025 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.00045415100000000002, 2024-09-16T09:34:34.025 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00053178600000000002, 2024-09-16T09:34:34.025 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:34:34.026 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:34:34.026 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T09:34:34.026 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:34:34.026 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:34.026 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:34:34.026 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:34.018054+0000", 2024-09-16T09:34:34.026 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:34:34.026 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:34.026 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:34.026 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:34:34.026 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:34.018054+0000", 2024-09-16T09:34:34.027 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:34:34.027 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:34.027 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:34.027 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:34:34.027 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:34.018055+0000", 2024-09-16T09:34:34.027 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.7499999999999999e-07 2024-09-16T09:34:34.027 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:34.027 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:34.027 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:34:34.027 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:34.018095+0000", 2024-09-16T09:34:34.027 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0575999999999999e-05 2024-09-16T09:34:34.027 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:34.027 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:34.027 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:34:34.028 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:34.018109+0000", 2024-09-16T09:34:34.028 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3619e-05 2024-09-16T09:34:34.028 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:34.028 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:34.028 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:34:34.028 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:34.018115+0000", 2024-09-16T09:34:34.028 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.4810000000000003e-06 2024-09-16T09:34:34.028 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:34.028 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:34.028 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:34:34.028 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:34.018176+0000", 2024-09-16T09:34:34.028 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.0279999999999999e-05 2024-09-16T09:34:34.028 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:34.028 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:34.029 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:34:34.029 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:34.018192+0000", 2024-09-16T09:34:34.029 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6606e-05 2024-09-16T09:34:34.029 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:34:34.029 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:34:34.029 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:34:34.029 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:34:34.029 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-16T09:34:34.029 INFO:teuthology.orchestra.run.smithi070.stdout: "num_ops": 2 2024-09-16T09:34:34.029 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T09:34:34.030 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:34:34.069 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:34.070 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:34.070 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:34:34.171 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:34:34.210 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:34.210 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:34.211 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:34:34.311 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:34:34.352 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:34.352 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:34.353 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:34:34.456 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:34:34.487 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:34.488 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:34.488 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:34:34.589 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:34:34.629 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '1000', 'type': 'reweight-by-utilization'} 2024-09-16T09:34:34.629 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd reweight-by-utilization 1000 3.0 2024-09-16T09:34:34.630 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:34.631 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:34.631 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:34:34.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:34 smithi084 ceph-mon[32629]: 1.2 scrub starts 2024-09-16T09:34:34.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:34 smithi084 ceph-mon[32629]: 1.2 scrub ok 2024-09-16T09:34:34.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:34 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:34:34.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:34 smithi084 ceph-mon[32629]: 6.1b scrub starts 2024-09-16T09:34:34.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:34 smithi084 ceph-mon[32629]: 6.1b scrub ok 2024-09-16T09:34:34.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:34 smithi084 ceph-mon[32629]: from='client.25601 -' entity='client.admin' cmd=[{"prefix": "osd test-reweight-by-utilization", "oload": 110, "max_change": 3.0, "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:34:34.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:34 smithi084 ceph-mon[32629]: 3.1b scrub starts 2024-09-16T09:34:34.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:34 smithi084 ceph-mon[32629]: 3.1b scrub ok 2024-09-16T09:34:34.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:34 smithi084 ceph-mon[32629]: osdmap e269: 8 total, 8 up, 8 in 2024-09-16T09:34:34.667 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:34 smithi070 ceph-mon[30388]: 1.2 scrub starts 2024-09-16T09:34:34.667 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:34 smithi070 ceph-mon[30388]: 1.2 scrub ok 2024-09-16T09:34:34.667 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:34 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:34:34.668 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:34 smithi070 ceph-mon[30388]: 6.1b scrub starts 2024-09-16T09:34:34.668 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:34 smithi070 ceph-mon[30388]: 6.1b scrub ok 2024-09-16T09:34:34.668 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:34 smithi070 ceph-mon[30388]: from='client.25601 -' entity='client.admin' cmd=[{"prefix": "osd test-reweight-by-utilization", "oload": 110, "max_change": 3.0, "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:34:34.668 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:34 smithi070 ceph-mon[30388]: 3.1b scrub starts 2024-09-16T09:34:34.668 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:34 smithi070 ceph-mon[30388]: 3.1b scrub ok 2024-09-16T09:34:34.668 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:34 smithi070 ceph-mon[30388]: osdmap e269: 8 total, 8 up, 8 in 2024-09-16T09:34:34.669 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:34 smithi070 ceph-mon[40691]: 1.2 scrub starts 2024-09-16T09:34:34.669 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:34 smithi070 ceph-mon[40691]: 1.2 scrub ok 2024-09-16T09:34:34.669 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:34 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:34:34.669 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:34 smithi070 ceph-mon[40691]: 6.1b scrub starts 2024-09-16T09:34:34.669 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:34 smithi070 ceph-mon[40691]: 6.1b scrub ok 2024-09-16T09:34:34.669 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:34 smithi070 ceph-mon[40691]: from='client.25601 -' entity='client.admin' cmd=[{"prefix": "osd test-reweight-by-utilization", "oload": 110, "max_change": 3.0, "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:34:34.669 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:34 smithi070 ceph-mon[40691]: 3.1b scrub starts 2024-09-16T09:34:34.669 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:34 smithi070 ceph-mon[40691]: 3.1b scrub ok 2024-09-16T09:34:34.670 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:34 smithi070 ceph-mon[40691]: osdmap e269: 8 total, 8 up, 8 in 2024-09-16T09:34:34.715 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 64 16 89413 89397 87.2879 85.25 0.00464069 0.0114517 2024-09-16T09:34:34.732 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:34:34.770 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:34.770 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:34.771 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:34:34.871 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:34:34.912 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:34.913 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:34.914 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:34:35.014 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:34:35.048 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:35.049 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:35.049 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:34:35.150 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:34:35.181 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:35.182 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:35.182 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:34:35.283 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:34:35.322 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:35.323 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:35.323 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:34:35.424 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:34:35.460 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:35.461 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:35.461 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:34:35.562 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:34:35.595 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:35.596 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:35.596 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:34:35.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:35 smithi084 ceph-mon[32629]: pgmap v469: 178 pgs: 2 active+recovering+undersized+remapped, 176 active+clean; 5.3 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 34 MiB/s wr, 537 op/s; 3962/261441 objects misplaced (1.515%); 20 MiB/s, 314 objects/s recovering 2024-09-16T09:34:35.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:35 smithi084 ceph-mon[32629]: 9.5 scrub starts 2024-09-16T09:34:35.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:35 smithi084 ceph-mon[32629]: 9.5 scrub ok 2024-09-16T09:34:35.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:35 smithi070 ceph-mon[30388]: pgmap v469: 178 pgs: 2 active+recovering+undersized+remapped, 176 active+clean; 5.3 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 34 MiB/s wr, 537 op/s; 3962/261441 objects misplaced (1.515%); 20 MiB/s, 314 objects/s recovering 2024-09-16T09:34:35.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:35 smithi070 ceph-mon[30388]: 9.5 scrub starts 2024-09-16T09:34:35.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:35 smithi070 ceph-mon[30388]: 9.5 scrub ok 2024-09-16T09:34:35.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:35 smithi070 ceph-mon[40691]: pgmap v469: 178 pgs: 2 active+recovering+undersized+remapped, 176 active+clean; 5.3 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 34 MiB/s wr, 537 op/s; 3962/261441 objects misplaced (1.515%); 20 MiB/s, 314 objects/s recovering 2024-09-16T09:34:35.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:35 smithi070 ceph-mon[40691]: 9.5 scrub starts 2024-09-16T09:34:35.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:35 smithi070 ceph-mon[40691]: 9.5 scrub ok 2024-09-16T09:34:35.697 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:34:35.715 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 65 16 90874 90858 87.3496 91.3125 0.00893184 0.0114433 2024-09-16T09:34:35.729 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:35.730 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:35.730 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:34:35.831 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:34:35.871 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:35.871 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:35.872 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:34:35.972 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:34:35.977 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.3 dump_blocked_ops 2024-09-16T09:34:35.995 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:34:36.025 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:36.025 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:36.025 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:34:36.126 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:34:36.162 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:36.163 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:36.163 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:34:36.264 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:34:36.305 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:36.305 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:36.306 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:34:36.317 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:34:36.407 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:34:36.446 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:36.447 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:36.447 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:34:36.548 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:34:36.592 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:36.593 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:36.594 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:34:36.694 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:34:36.715 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 66 16 92284 92268 87.3613 88.125 0.0150354 0.011442 2024-09-16T09:34:36.731 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:36.731 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:36.732 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:34:36.833 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:34:36.869 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:36.870 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:36.870 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:34:36.955 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:34:36.972 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:34:37.007 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:37.007 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:37.008 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:34:37.106 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:34:37.109 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:34:37.140 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:37.141 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:37.141 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:34:37.242 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:34:37.278 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:37.279 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:37.279 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:34:37.354 INFO:teuthology.orchestra.run.smithi070.stdout:moved 0 / 534 (0%) 2024-09-16T09:34:37.354 INFO:teuthology.orchestra.run.smithi070.stdout:avg 66.75 2024-09-16T09:34:37.354 INFO:teuthology.orchestra.run.smithi070.stdout:stddev 9.79477 -> 9.79477 (expected baseline 7.6424) 2024-09-16T09:34:37.354 INFO:teuthology.orchestra.run.smithi070.stdout:min osd.2 with 48 -> 48 pgs (0.719101 -> 0.719101 * mean) 2024-09-16T09:34:37.354 INFO:teuthology.orchestra.run.smithi070.stdout:max osd.7 with 79 -> 79 pgs (1.18352 -> 1.18352 * mean) 2024-09-16T09:34:37.355 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:34:37.355 INFO:teuthology.orchestra.run.smithi070.stdout:oload 1000 2024-09-16T09:34:37.355 INFO:teuthology.orchestra.run.smithi070.stdout:max_change 3 2024-09-16T09:34:37.355 INFO:teuthology.orchestra.run.smithi070.stdout:max_change_osds 4 2024-09-16T09:34:37.355 INFO:teuthology.orchestra.run.smithi070.stdout:average_utilization 0.0277 2024-09-16T09:34:37.355 INFO:teuthology.orchestra.run.smithi070.stdout:overload_utilization 0.2774 2024-09-16T09:34:37.355 INFO:teuthology.orchestra.run.smithi070.stderr:no change 2024-09-16T09:34:37.380 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:34:37.408 INFO:teuthology.orchestra.run.smithi070.stderr:nodeep-scrub is unset 2024-09-16T09:34:37.415 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:37.416 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:37.416 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:34:37.517 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:34:37.554 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:37.555 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:37.556 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:34:37.656 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:34:37.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:37 smithi084 ceph-mon[32629]: pgmap v471: 178 pgs: 2 active+recovering+undersized+remapped, 176 active+clean; 5.3 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 23 MiB/s wr, 363 op/s; 3962/261441 objects misplaced (1.515%); 957 KiB/s, 15 objects/s recovering 2024-09-16T09:34:37.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:37 smithi084 ceph-mon[32629]: 6.1 scrub starts 2024-09-16T09:34:37.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:37 smithi084 ceph-mon[32629]: 6.1 scrub ok 2024-09-16T09:34:37.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:37 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/118965944' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:34:37.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:37 smithi070 ceph-mon[30388]: pgmap v471: 178 pgs: 2 active+recovering+undersized+remapped, 176 active+clean; 5.3 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 23 MiB/s wr, 363 op/s; 3962/261441 objects misplaced (1.515%); 957 KiB/s, 15 objects/s recovering 2024-09-16T09:34:37.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:37 smithi070 ceph-mon[30388]: 6.1 scrub starts 2024-09-16T09:34:37.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:37 smithi070 ceph-mon[30388]: 6.1 scrub ok 2024-09-16T09:34:37.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:37 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/118965944' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:34:37.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:37 smithi070 ceph-mon[40691]: pgmap v471: 178 pgs: 2 active+recovering+undersized+remapped, 176 active+clean; 5.3 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 23 MiB/s wr, 363 op/s; 3962/261441 objects misplaced (1.515%); 957 KiB/s, 15 objects/s recovering 2024-09-16T09:34:37.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:37 smithi070 ceph-mon[40691]: 6.1 scrub starts 2024-09-16T09:34:37.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:37 smithi070 ceph-mon[40691]: 6.1 scrub ok 2024-09-16T09:34:37.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:37 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/118965944' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:34:37.690 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:37.691 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:37.692 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:34:37.715 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 67 16 93646 93630 87.3278 85.125 0.00853985 0.0114463 2024-09-16T09:34:37.793 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:34:37.838 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:37.839 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:37.839 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:34:37.940 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:34:37.977 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:37.978 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:37.978 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:34:38.007 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:34:38.007 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:34:38.007 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:34:38.007 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:34:38.021 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:34:38.022 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:34:38.052 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:34:38.052 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:34:38.079 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:34:38.085 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:34:38.085 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:34:38.112 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:38.113 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:38.113 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:34:38.116 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:34:38.117 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:34:38.149 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:34:38.149 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:34:38.185 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:34:38.185 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:34:38.214 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:34:38.249 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:38.249 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:38.250 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:34:38.350 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:34:38.390 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:38.390 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:38.390 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:34:38.491 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:34:38.524 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:38.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:38.525 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:34:38.613 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:38 smithi070 ceph-mon[30388]: from='client.15672 -' entity='client.admin' cmd=[{"prefix": "osd reweight-by-utilization", "oload": 1000, "max_change": 3.0, "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:34:38.613 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:38 smithi070 ceph-mon[30388]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-16T09:34:38.613 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:38 smithi070 ceph-mon[30388]: Cluster is now healthy 2024-09-16T09:34:38.613 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:38 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/118965944' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-16T09:34:38.613 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:38 smithi070 ceph-mon[30388]: osdmap e270: 8 total, 8 up, 8 in 2024-09-16T09:34:38.614 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:38 smithi070 ceph-mon[30388]: 6.1f scrub starts 2024-09-16T09:34:38.614 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:38 smithi070 ceph-mon[30388]: 6.1f scrub ok 2024-09-16T09:34:38.614 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:38 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/692655066' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:34:38.614 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:38 smithi070 ceph-mon[40691]: from='client.15672 -' entity='client.admin' cmd=[{"prefix": "osd reweight-by-utilization", "oload": 1000, "max_change": 3.0, "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:34:38.614 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:38 smithi070 ceph-mon[40691]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-16T09:34:38.614 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:38 smithi070 ceph-mon[40691]: Cluster is now healthy 2024-09-16T09:34:38.614 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:38 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/118965944' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-16T09:34:38.614 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:38 smithi070 ceph-mon[40691]: osdmap e270: 8 total, 8 up, 8 in 2024-09-16T09:34:38.614 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:38 smithi070 ceph-mon[40691]: 6.1f scrub starts 2024-09-16T09:34:38.614 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:38 smithi070 ceph-mon[40691]: 6.1f scrub ok 2024-09-16T09:34:38.614 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:38 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/692655066' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:34:38.626 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:34:38.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:38 smithi084 ceph-mon[32629]: from='client.15672 -' entity='client.admin' cmd=[{"prefix": "osd reweight-by-utilization", "oload": 1000, "max_change": 3.0, "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:34:38.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:38 smithi084 ceph-mon[32629]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-16T09:34:38.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:38 smithi084 ceph-mon[32629]: Cluster is now healthy 2024-09-16T09:34:38.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:38 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/118965944' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-16T09:34:38.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:38 smithi084 ceph-mon[32629]: osdmap e270: 8 total, 8 up, 8 in 2024-09-16T09:34:38.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:38 smithi084 ceph-mon[32629]: 6.1f scrub starts 2024-09-16T09:34:38.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:38 smithi084 ceph-mon[32629]: 6.1f scrub ok 2024-09-16T09:34:38.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:38 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/692655066' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:34:38.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:38.665 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:38.665 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:34:38.715 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 68 16 95150 95134 87.4258 94 0.0219733 0.0114334 2024-09-16T09:34:38.766 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:34:38.773 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T09:34:38.773 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:34:38.807 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:38.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:38.808 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:34:38.909 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:34:38.937 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:38.938 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:38.938 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:34:39.039 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:34:39.076 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:39.077 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:39.077 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:34:39.178 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:34:39.218 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:39.219 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:39.219 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:34:39.320 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:34:39.359 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:39.360 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:39.360 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:34:39.460 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:34:39.494 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:39.495 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:39.495 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:34:39.596 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:34:39.633 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:39.633 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:39.634 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:34:39.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:39 smithi070 ceph-mon[30388]: pgmap v473: 178 pgs: 2 active+recovering+undersized+remapped, 176 active+clean; 5.3 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 27 MiB/s wr, 428 op/s; 3962/263346 objects misplaced (1.504%); 848 KiB/s, 13 objects/s recovering 2024-09-16T09:34:39.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:39 smithi070 ceph-mon[30388]: 5.b scrub starts 2024-09-16T09:34:39.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:39 smithi070 ceph-mon[30388]: 5.b scrub ok 2024-09-16T09:34:39.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:39 smithi070 ceph-mon[30388]: osdmap e271: 8 total, 8 up, 8 in 2024-09-16T09:34:39.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:39 smithi070 ceph-mon[40691]: pgmap v473: 178 pgs: 2 active+recovering+undersized+remapped, 176 active+clean; 5.3 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 27 MiB/s wr, 428 op/s; 3962/263346 objects misplaced (1.504%); 848 KiB/s, 13 objects/s recovering 2024-09-16T09:34:39.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:39 smithi070 ceph-mon[40691]: 5.b scrub starts 2024-09-16T09:34:39.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:39 smithi070 ceph-mon[40691]: 5.b scrub ok 2024-09-16T09:34:39.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:39 smithi070 ceph-mon[40691]: osdmap e271: 8 total, 8 up, 8 in 2024-09-16T09:34:39.715 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 69 16 96539 96523 87.4168 86.8125 0.008011 0.0114343 2024-09-16T09:34:39.734 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:34:39.775 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:39.776 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:39.776 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:34:39.877 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:34:39.908 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:39.908 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:39.909 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:34:39.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:39 smithi084 ceph-mon[32629]: pgmap v473: 178 pgs: 2 active+recovering+undersized+remapped, 176 active+clean; 5.3 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 27 MiB/s wr, 428 op/s; 3962/263346 objects misplaced (1.504%); 848 KiB/s, 13 objects/s recovering 2024-09-16T09:34:39.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:39 smithi084 ceph-mon[32629]: 5.b scrub starts 2024-09-16T09:34:39.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:39 smithi084 ceph-mon[32629]: 5.b scrub ok 2024-09-16T09:34:39.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:39 smithi084 ceph-mon[32629]: osdmap e271: 8 total, 8 up, 8 in 2024-09-16T09:34:40.010 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:34:40.044 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:40.045 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:40.045 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:34:40.146 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:34:40.187 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:40.187 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:40.188 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:34:40.288 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:34:40.327 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:40.327 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:40.328 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:34:40.428 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:34:40.459 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:40.459 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:40.459 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:34:40.560 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:34:40.585 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:40.585 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:40.586 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:34:40.687 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:34:40.708 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:40.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:40.710 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:34:40.715 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 70 15 97901 97886 87.3848 85.1875 0.00283693 0.0114131 2024-09-16T09:34:40.810 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:34:40.832 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:40.832 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:40.833 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:34:40.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:40 smithi084 ceph-mon[32629]: 3.8 scrub starts 2024-09-16T09:34:40.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:40 smithi084 ceph-mon[32629]: 3.8 scrub ok 2024-09-16T09:34:40.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:40 smithi070 ceph-mon[30388]: 3.8 scrub starts 2024-09-16T09:34:40.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:40 smithi070 ceph-mon[30388]: 3.8 scrub ok 2024-09-16T09:34:40.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:40 smithi070 ceph-mon[40691]: 3.8 scrub starts 2024-09-16T09:34:40.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:40 smithi070 ceph-mon[40691]: 3.8 scrub ok 2024-09-16T09:34:40.933 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:34:40.955 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:40.955 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:40.956 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:34:41.056 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:34:41.077 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:41.078 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:41.079 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:34:41.179 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:34:41.200 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:41.201 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:41.201 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:34:41.273 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.3 dump_historic_ops 2024-09-16T09:34:41.302 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:34:41.324 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:41.324 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:41.325 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:34:41.405 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T09:34:41.426 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:34:41.451 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:41.452 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:41.452 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:34:41.553 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:34:41.593 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:41.594 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:41.594 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:34:41.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:41 smithi070 ceph-mon[30388]: pgmap v475: 178 pgs: 2 active+recovering+undersized+remapped, 176 active+clean; 5.7 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 853 B/s rd, 45 MiB/s wr, 720 op/s; 2789/282666 objects misplaced (0.987%) 2024-09-16T09:34:41.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:41 smithi070 ceph-mon[30388]: osdmap e272: 8 total, 8 up, 8 in 2024-09-16T09:34:41.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:41 smithi070 ceph-mon[40691]: pgmap v475: 178 pgs: 2 active+recovering+undersized+remapped, 176 active+clean; 5.7 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 853 B/s rd, 45 MiB/s wr, 720 op/s; 2789/282666 objects misplaced (0.987%) 2024-09-16T09:34:41.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:41 smithi070 ceph-mon[40691]: osdmap e272: 8 total, 8 up, 8 in 2024-09-16T09:34:41.694 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:34:41.715 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 71 16 98303 98287 86.507 25.0625 0.00707323 0.0115549 2024-09-16T09:34:41.727 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:41.728 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:41.728 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:34:41.795 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '1000', 'type': 'test-reweight-by-utilization'} 2024-09-16T09:34:41.796 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd test-reweight-by-utilization 1000 3.0 2024-09-16T09:34:41.829 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:34:41.866 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:41.867 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:41.867 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:34:41.877 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:34:41.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:41 smithi084 ceph-mon[32629]: pgmap v475: 178 pgs: 2 active+recovering+undersized+remapped, 176 active+clean; 5.7 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 853 B/s rd, 45 MiB/s wr, 720 op/s; 2789/282666 objects misplaced (0.987%) 2024-09-16T09:34:41.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:41 smithi084 ceph-mon[32629]: osdmap e272: 8 total, 8 up, 8 in 2024-09-16T09:34:41.969 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:34:42.003 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:42.003 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:42.004 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:34:42.015 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:34:42.022 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:34:41 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:34:41] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:34:42.105 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:34:42.151 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:42.152 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:42.152 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:34:42.253 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:34:42.293 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:42.294 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:42.294 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:34:42.395 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:34:42.430 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:42.431 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:42.431 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:34:42.532 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:34:42.565 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:42.566 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:42.566 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:34:42.620 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd set noscrub 2024-09-16T09:34:42.639 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:42 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:34:42.639 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:42 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:34:42.639 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:42 smithi070 ceph-mon[30388]: 3.19 scrub starts 2024-09-16T09:34:42.639 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:42 smithi070 ceph-mon[30388]: 3.19 scrub ok 2024-09-16T09:34:42.639 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:42 smithi070 ceph-mon[30388]: osdmap e273: 8 total, 8 up, 8 in 2024-09-16T09:34:42.639 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:42 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:34:42.640 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:42 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:34:42.640 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:42 smithi070 ceph-mon[40691]: 3.19 scrub starts 2024-09-16T09:34:42.640 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:42 smithi070 ceph-mon[40691]: 3.19 scrub ok 2024-09-16T09:34:42.640 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:42 smithi070 ceph-mon[40691]: osdmap e273: 8 total, 8 up, 8 in 2024-09-16T09:34:42.668 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:34:42.716 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 72 16 99681 99665 86.5016 86.125 0.0112275 0.0115558 2024-09-16T09:34:42.718 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:34:42.718 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:42.719 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:42.719 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:34:42.725 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:34:42.725 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:34:42.766 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:34:42.767 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:34:42.797 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:34:42.798 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:34:42.820 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:34:42.829 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:34:42.829 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:34:42.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:42.858 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:42.858 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:34:42.862 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:34:42.862 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:34:42.902 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:34:42.902 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:34:42.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:42 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:34:42.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:42 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:34:42.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:42 smithi084 ceph-mon[32629]: 3.19 scrub starts 2024-09-16T09:34:42.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:42 smithi084 ceph-mon[32629]: 3.19 scrub ok 2024-09-16T09:34:42.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:42 smithi084 ceph-mon[32629]: osdmap e273: 8 total, 8 up, 8 in 2024-09-16T09:34:42.935 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:34:42.935 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:34:42.959 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:34:42.963 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:34:42.963 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:34:42.997 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:42.998 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:42.998 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:34:43.099 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:34:43.130 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:43.131 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:43.131 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:34:43.232 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:34:43.265 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:43.265 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:43.265 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:34:43.366 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:34:43.403 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:43.404 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:43.404 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:34:43.410 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:34:43 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:34:43.496 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T09:34:43.497 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 20, 2024-09-16T09:34:43.497 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 600, 2024-09-16T09:34:43.497 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [ 2024-09-16T09:34:43.497 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.497 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.24698.0:203420 7.a 7:52851f87:::benchmark_data_smithi070_78438_object77289:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135)", 2024-09-16T09:34:43.497 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:28:51.960186+0000", 2024-09-16T09:34:43.497 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 351.53028343699998, 2024-09-16T09:34:43.497 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.16906342799999999, 2024-09-16T09:34:43.497 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:34:43.497 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:34:43.497 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "waiting for sub ops", 2024-09-16T09:34:43.497 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:34:43.498 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.24698", 2024-09-16T09:34:43.498 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:34:43.498 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 203420 2024-09-16T09:34:43.498 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.498 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:34:43.498 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.498 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:34:43.498 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.960186+0000", 2024-09-16T09:34:43.498 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:34:43.498 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.498 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.499 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:34:43.499 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.960186+0000", 2024-09-16T09:34:43.499 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:34:43.499 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.499 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.499 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:34:43.499 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.960187+0000", 2024-09-16T09:34:43.499 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.4589999999999999e-06 2024-09-16T09:34:43.499 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.499 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.499 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:34:43.499 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.960192+0000", 2024-09-16T09:34:43.499 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.4449999999999998e-06 2024-09-16T09:34:43.499 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.500 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.500 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:34:43.500 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.960201+0000", 2024-09-16T09:34:43.500 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.5289999999999997e-06 2024-09-16T09:34:43.500 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.500 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.500 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:34:43.500 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.960208+0000", 2024-09-16T09:34:43.500 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.6499999999999999e-06 2024-09-16T09:34:43.500 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.500 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.500 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:34:43.501 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.962224+0000", 2024-09-16T09:34:43.501 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0020158580000000001 2024-09-16T09:34:43.501 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.501 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.501 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:34:43.501 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.962328+0000", 2024-09-16T09:34:43.501 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000103984 2024-09-16T09:34:43.501 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.501 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.501 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 2,5", 2024-09-16T09:34:43.501 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.962437+0000", 2024-09-16T09:34:43.501 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000108879 2024-09-16T09:34:43.502 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.502 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.502 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:34:43.502 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.967574+0000", 2024-09-16T09:34:43.502 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0051370909999999999 2024-09-16T09:34:43.502 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.502 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.502 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:34:43.502 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.129249+0000", 2024-09-16T09:34:43.502 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.16167553300000001 2024-09-16T09:34:43.502 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:34:43.502 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:34:43.502 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:34:43.502 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.503 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.503 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.24698.0:203422 7.a 7:5a4cd988:::benchmark_data_smithi070_78438_object77291:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135)", 2024-09-16T09:34:43.503 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:28:51.966795+0000", 2024-09-16T09:34:43.503 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 351.52367421299999, 2024-09-16T09:34:43.503 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.16245894399999999, 2024-09-16T09:34:43.503 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:34:43.503 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:34:43.503 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "waiting for sub ops", 2024-09-16T09:34:43.503 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:34:43.503 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.24698", 2024-09-16T09:34:43.503 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:34:43.503 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 203422 2024-09-16T09:34:43.503 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.503 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:34:43.504 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.504 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:34:43.504 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.966795+0000", 2024-09-16T09:34:43.504 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:34:43.504 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.504 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.504 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:34:43.504 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.966795+0000", 2024-09-16T09:34:43.504 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:34:43.504 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.504 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.504 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:34:43.504 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.966797+0000", 2024-09-16T09:34:43.505 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.243e-06 2024-09-16T09:34:43.505 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.505 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.505 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:34:43.505 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.966805+0000", 2024-09-16T09:34:43.505 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.7649999999999993e-06 2024-09-16T09:34:43.505 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.505 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.505 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:34:43.505 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.966819+0000", 2024-09-16T09:34:43.505 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3859000000000001e-05 2024-09-16T09:34:43.505 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.505 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.505 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:34:43.506 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.966829+0000", 2024-09-16T09:34:43.506 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.5070000000000006e-06 2024-09-16T09:34:43.506 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.506 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.506 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:34:43.506 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.967612+0000", 2024-09-16T09:34:43.506 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00078302500000000002 2024-09-16T09:34:43.506 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.506 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.506 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:34:43.506 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.967723+0000", 2024-09-16T09:34:43.507 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00011114400000000001 2024-09-16T09:34:43.507 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.507 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.507 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 2,5", 2024-09-16T09:34:43.507 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.968124+0000", 2024-09-16T09:34:43.507 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00040120299999999999 2024-09-16T09:34:43.507 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.507 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.507 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:34:43.507 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.129254+0000", 2024-09-16T09:34:43.507 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.161130198 2024-09-16T09:34:43.507 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:34:43.507 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:34:43.507 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:34:43.508 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.508 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.508 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.24698.0:203434 7.2 7:4b27635e:::benchmark_data_smithi070_78438_object77303:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e135)", 2024-09-16T09:34:43.508 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:28:51.985566+0000", 2024-09-16T09:34:43.508 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 351.50490363099999, 2024-09-16T09:34:43.508 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.16551761700000001, 2024-09-16T09:34:43.508 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:34:43.508 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:34:43.508 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:34:43.508 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:34:43.508 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.24698", 2024-09-16T09:34:43.508 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:34:43.508 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 203434 2024-09-16T09:34:43.508 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.509 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:34:43.509 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.509 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:34:43.509 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.985566+0000", 2024-09-16T09:34:43.509 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:34:43.509 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.509 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.509 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:34:43.509 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.985566+0000", 2024-09-16T09:34:43.509 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:34:43.509 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.509 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.509 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:34:43.509 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.985567+0000", 2024-09-16T09:34:43.510 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5689999999999999e-06 2024-09-16T09:34:43.510 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.510 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.510 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:34:43.510 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.985573+0000", 2024-09-16T09:34:43.510 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.339e-06 2024-09-16T09:34:43.510 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.510 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.510 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:34:43.510 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.117695+0000", 2024-09-16T09:34:43.510 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.13212251 2024-09-16T09:34:43.510 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.510 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.510 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:34:43.510 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.117706+0000", 2024-09-16T09:34:43.511 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0431000000000001e-05 2024-09-16T09:34:43.511 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.511 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.511 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:34:43.511 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.130808+0000", 2024-09-16T09:34:43.511 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.013101936999999999 2024-09-16T09:34:43.511 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.511 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.511 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:34:43.512 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.130930+0000", 2024-09-16T09:34:43.512 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00012259099999999999 2024-09-16T09:34:43.512 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.512 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.512 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,2", 2024-09-16T09:34:43.512 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.131054+0000", 2024-09-16T09:34:43.512 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000123695 2024-09-16T09:34:43.512 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.512 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.512 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:34:43.512 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.137688+0000", 2024-09-16T09:34:43.512 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0066342689999999999 2024-09-16T09:34:43.512 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.513 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.513 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:34:43.513 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.137822+0000", 2024-09-16T09:34:43.513 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00013432900000000001 2024-09-16T09:34:43.513 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.513 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.513 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:34:43.513 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.150917+0000", 2024-09-16T09:34:43.513 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.013094252000000001 2024-09-16T09:34:43.513 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.513 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.513 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:34:43.513 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.151020+0000", 2024-09-16T09:34:43.513 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000103112 2024-09-16T09:34:43.514 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.514 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.514 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:34:43.514 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.151083+0000", 2024-09-16T09:34:43.514 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.3583000000000004e-05 2024-09-16T09:34:43.514 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:34:43.514 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:34:43.514 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:34:43.514 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.514 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.514 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:64878 9.2 9:47defa64:::benchmark_data_smithi070_137000_object64877:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e252)", 2024-09-16T09:34:43.514 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:34:09.137162+0000", 2024-09-16T09:34:43.514 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 34.353307723, 2024-09-16T09:34:43.514 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.18651403499999999, 2024-09-16T09:34:43.515 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:34:43.515 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:34:43.515 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:34:43.515 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:34:43.515 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:34:43.515 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:34:43.515 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 64878 2024-09-16T09:34:43.515 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.515 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:34:43.515 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.515 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:34:43.515 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:09.137162+0000", 2024-09-16T09:34:43.515 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:34:43.515 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.515 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.516 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:34:43.516 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:09.137162+0000", 2024-09-16T09:34:43.516 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:34:43.516 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.516 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.516 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:34:43.516 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:09.137163+0000", 2024-09-16T09:34:43.516 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7600000000000001e-06 2024-09-16T09:34:43.516 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.516 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.516 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:34:43.516 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:09.137187+0000", 2024-09-16T09:34:43.516 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3373000000000001e-05 2024-09-16T09:34:43.517 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.517 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.517 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:34:43.517 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:09.137204+0000", 2024-09-16T09:34:43.517 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7354e-05 2024-09-16T09:34:43.517 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.517 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.517 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:34:43.517 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:09.137213+0000", 2024-09-16T09:34:43.517 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.2499999999999995e-06 2024-09-16T09:34:43.517 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.517 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.517 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:34:43.517 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:09.137266+0000", 2024-09-16T09:34:43.518 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.3062000000000001e-05 2024-09-16T09:34:43.518 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.518 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.518 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:34:43.518 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:09.137363+0000", 2024-09-16T09:34:43.518 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.6398999999999996e-05 2024-09-16T09:34:43.518 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.518 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.518 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,2", 2024-09-16T09:34:43.518 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:09.137523+0000", 2024-09-16T09:34:43.518 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000160443 2024-09-16T09:34:43.518 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.518 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.518 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:34:43.519 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:09.138997+0000", 2024-09-16T09:34:43.519 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00147338 2024-09-16T09:34:43.519 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.519 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.519 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:34:43.519 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:09.140348+0000", 2024-09-16T09:34:43.519 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001351725 2024-09-16T09:34:43.519 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.519 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.519 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:34:43.519 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:09.323584+0000", 2024-09-16T09:34:43.519 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.18323550899999999 2024-09-16T09:34:43.519 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.519 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.519 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:34:43.520 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:09.323634+0000", 2024-09-16T09:34:43.520 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.9836000000000002e-05 2024-09-16T09:34:43.520 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.520 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.520 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:34:43.520 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:09.323676+0000", 2024-09-16T09:34:43.520 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.1943999999999997e-05 2024-09-16T09:34:43.520 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:34:43.520 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:34:43.520 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:34:43.520 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.520 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.520 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:85224 9.19 9:9bf6c0c6:::benchmark_data_smithi070_137000_object85223:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e266)", 2024-09-16T09:34:43.520 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:34:30.373671+0000", 2024-09-16T09:34:43.521 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 13.116798749999999, 2024-09-16T09:34:43.521 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0883978459999999, 2024-09-16T09:34:43.521 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:34:43.521 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:34:43.521 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T09:34:43.521 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:34:43.521 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:34:43.521 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:34:43.521 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 85224 2024-09-16T09:34:43.521 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.521 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:34:43.521 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.521 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:34:43.521 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.373671+0000", 2024-09-16T09:34:43.522 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:34:43.522 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.522 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.522 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:34:43.522 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.373671+0000", 2024-09-16T09:34:43.522 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:34:43.522 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.522 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.522 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:34:43.522 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.373673+0000", 2024-09-16T09:34:43.522 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.4600000000000002e-06 2024-09-16T09:34:43.522 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.522 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.522 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:34:43.522 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.373702+0000", 2024-09-16T09:34:43.523 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.8934000000000001e-05 2024-09-16T09:34:43.523 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.523 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.523 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:34:43.523 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.373725+0000", 2024-09-16T09:34:43.523 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3468999999999998e-05 2024-09-16T09:34:43.523 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.523 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.523 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:34:43.523 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.373737+0000", 2024-09-16T09:34:43.523 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1252e-05 2024-09-16T09:34:43.523 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.523 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.523 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:34:43.524 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.386012+0000", 2024-09-16T09:34:43.524 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.012275569 2024-09-16T09:34:43.524 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.524 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.524 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:34:43.524 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.386022+0000", 2024-09-16T09:34:43.524 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.2609999999999999e-06 2024-09-16T09:34:43.524 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.524 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.524 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:34:43.524 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.428618+0000", 2024-09-16T09:34:43.524 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0425969100000001 2024-09-16T09:34:43.524 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.524 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.524 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for missing object", 2024-09-16T09:34:43.525 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.428759+0000", 2024-09-16T09:34:43.525 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00014022899999999999 2024-09-16T09:34:43.525 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:34:43.526 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.526 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.526 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:34:43.526 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.432901+0000", 2024-09-16T09:34:43.526 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0041425020000000002 2024-09-16T09:34:43.527 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.527 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.527 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for degraded object", 2024-09-16T09:34:43.527 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.432957+0000", 2024-09-16T09:34:43.527 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5377999999999997e-05 2024-09-16T09:34:43.527 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.527 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.527 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:34:43.527 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.462010+0000", 2024-09-16T09:34:43.527 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.029052923000000001 2024-09-16T09:34:43.527 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.527 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.527 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:34:43.527 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.462068+0000", 2024-09-16T09:34:43.528 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8959000000000003e-05 2024-09-16T09:34:43.528 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:34:43.528 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:34:43.528 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:34:43.528 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.528 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.528 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:85228 9.19 9:98aa15be:::benchmark_data_smithi070_137000_object85227:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e266)", 2024-09-16T09:34:43.528 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:34:30.376655+0000", 2024-09-16T09:34:43.528 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 13.113813993999999, 2024-09-16T09:34:43.528 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0565689490000001, 2024-09-16T09:34:43.528 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:34:43.528 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:34:43.528 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:34:43.529 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:34:43.529 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:34:43.529 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:34:43.529 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 85228 2024-09-16T09:34:43.529 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.529 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:34:43.529 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.529 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:34:43.529 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.376655+0000", 2024-09-16T09:34:43.529 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:34:43.529 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.529 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.529 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:34:43.529 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.376655+0000", 2024-09-16T09:34:43.530 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:34:43.530 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.530 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.530 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:34:43.530 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.376657+0000", 2024-09-16T09:34:43.530 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.4419999999999999e-06 2024-09-16T09:34:43.530 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.530 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.530 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:34:43.530 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.376680+0000", 2024-09-16T09:34:43.530 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.2923000000000001e-05 2024-09-16T09:34:43.530 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.530 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.530 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:34:43.531 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.376692+0000", 2024-09-16T09:34:43.531 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1861999999999999e-05 2024-09-16T09:34:43.531 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.531 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.531 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:34:43.531 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.376701+0000", 2024-09-16T09:34:43.531 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.0080000000000008e-06 2024-09-16T09:34:43.531 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.531 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.531 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:34:43.531 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.386042+0000", 2024-09-16T09:34:43.531 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0093412960000000007 2024-09-16T09:34:43.531 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.531 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.532 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:34:43.532 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.386048+0000", 2024-09-16T09:34:43.532 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.9170000000000003e-06 2024-09-16T09:34:43.532 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.532 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.532 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:34:43.532 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.429089+0000", 2024-09-16T09:34:43.532 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0430408870000001 2024-09-16T09:34:43.532 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.532 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.532 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:34:43.532 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.429187+0000", 2024-09-16T09:34:43.533 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.8227999999999995e-05 2024-09-16T09:34:43.533 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.533 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.533 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 5,6", 2024-09-16T09:34:43.533 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.429456+0000", 2024-09-16T09:34:43.533 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00026942699999999998 2024-09-16T09:34:43.533 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.533 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.533 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:34:43.533 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.433028+0000", 2024-09-16T09:34:43.533 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0035718199999999999 2024-09-16T09:34:43.533 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.533 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.533 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:34:43.534 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.433093+0000", 2024-09-16T09:34:43.534 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.4393000000000005e-05 2024-09-16T09:34:43.534 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.534 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.534 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:34:43.534 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.433126+0000", 2024-09-16T09:34:43.534 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3083000000000002e-05 2024-09-16T09:34:43.534 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.534 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.534 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:34:43.534 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.433160+0000", 2024-09-16T09:34:43.534 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4746999999999998e-05 2024-09-16T09:34:43.534 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.535 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.535 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:34:43.535 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.433224+0000", 2024-09-16T09:34:43.535 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.3916000000000003e-05 2024-09-16T09:34:43.535 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:34:43.535 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:34:43.535 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:34:43.535 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.535 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.535 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:85232 9.19 9:9ebfcfc9:::benchmark_data_smithi070_137000_object85231:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e266)", 2024-09-16T09:34:43.535 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:34:30.381300+0000", 2024-09-16T09:34:43.535 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 13.109169682999999, 2024-09-16T09:34:43.535 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0521562879999999, 2024-09-16T09:34:43.535 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:34:43.536 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:34:43.536 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:34:43.536 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:34:43.536 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:34:43.536 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:34:43.536 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 85232 2024-09-16T09:34:43.536 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.536 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:34:43.536 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.536 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:34:43.536 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.381300+0000", 2024-09-16T09:34:43.536 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:34:43.536 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.536 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.537 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:34:43.537 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.381300+0000", 2024-09-16T09:34:43.537 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:34:43.537 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.537 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.537 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:34:43.537 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.381301+0000", 2024-09-16T09:34:43.537 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6959999999999999e-06 2024-09-16T09:34:43.537 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.537 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.537 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:34:43.537 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.381325+0000", 2024-09-16T09:34:43.537 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3213999999999998e-05 2024-09-16T09:34:43.537 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.537 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.538 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:34:43.538 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.381338+0000", 2024-09-16T09:34:43.538 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3237999999999999e-05 2024-09-16T09:34:43.538 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.538 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.538 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:34:43.538 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.381346+0000", 2024-09-16T09:34:43.538 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.8390000000000007e-06 2024-09-16T09:34:43.538 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.538 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.538 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:34:43.538 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.386067+0000", 2024-09-16T09:34:43.538 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0047214400000000004 2024-09-16T09:34:43.538 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.539 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.539 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:34:43.539 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.386073+0000", 2024-09-16T09:34:43.539 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5790000000000003e-06 2024-09-16T09:34:43.539 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.539 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.539 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:34:43.539 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.430684+0000", 2024-09-16T09:34:43.539 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.044610824 2024-09-16T09:34:43.539 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.539 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.539 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:34:43.539 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.430767+0000", 2024-09-16T09:34:43.539 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.3483e-05 2024-09-16T09:34:43.539 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.540 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.540 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 5,6", 2024-09-16T09:34:43.540 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.430988+0000", 2024-09-16T09:34:43.540 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00022084900000000001 2024-09-16T09:34:43.540 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.540 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.540 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:34:43.540 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.433032+0000", 2024-09-16T09:34:43.540 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0020444159999999999 2024-09-16T09:34:43.540 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.540 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.540 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:34:43.540 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.433282+0000", 2024-09-16T09:34:43.541 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000250024 2024-09-16T09:34:43.541 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.541 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.541 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:34:43.541 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.433335+0000", 2024-09-16T09:34:43.541 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.3096999999999997e-05 2024-09-16T09:34:43.541 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.541 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.541 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:34:43.541 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.433370+0000", 2024-09-16T09:34:43.541 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4465000000000003e-05 2024-09-16T09:34:43.541 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.541 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.541 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:34:43.542 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.433456+0000", 2024-09-16T09:34:43.542 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.6124000000000004e-05 2024-09-16T09:34:43.542 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:34:43.542 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:34:43.542 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:34:43.542 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.542 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.542 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:85266 9.19 9:9983710d:::benchmark_data_smithi070_137000_object85265:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e266)", 2024-09-16T09:34:43.542 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:34:30.409167+0000", 2024-09-16T09:34:43.542 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 13.081301884, 2024-09-16T09:34:43.542 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.029484174, 2024-09-16T09:34:43.542 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:34:43.542 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:34:43.542 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:34:43.543 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:34:43.543 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:34:43.543 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:34:43.543 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 85266 2024-09-16T09:34:43.543 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.543 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:34:43.543 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.543 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:34:43.543 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.409167+0000", 2024-09-16T09:34:43.543 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:34:43.543 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.543 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.543 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:34:43.543 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.409167+0000", 2024-09-16T09:34:43.544 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:34:43.544 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.544 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.544 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:34:43.544 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.409169+0000", 2024-09-16T09:34:43.544 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6339999999999999e-06 2024-09-16T09:34:43.544 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.544 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.544 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:34:43.544 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.409201+0000", 2024-09-16T09:34:43.544 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2193000000000002e-05 2024-09-16T09:34:43.544 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.544 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.544 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:34:43.545 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.409218+0000", 2024-09-16T09:34:43.545 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6481e-05 2024-09-16T09:34:43.545 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.545 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.545 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:34:43.545 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.409225+0000", 2024-09-16T09:34:43.545 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6469999999999998e-06 2024-09-16T09:34:43.545 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.545 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.545 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:34:43.545 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.410186+0000", 2024-09-16T09:34:43.545 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000960673 2024-09-16T09:34:43.545 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.545 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.546 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:34:43.546 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.410198+0000", 2024-09-16T09:34:43.546 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1993e-05 2024-09-16T09:34:43.546 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.546 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.546 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:34:43.546 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.433567+0000", 2024-09-16T09:34:43.546 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.023368582 2024-09-16T09:34:43.546 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.546 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.546 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:34:43.546 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.433672+0000", 2024-09-16T09:34:43.546 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000105046 2024-09-16T09:34:43.546 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.547 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.547 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 5,6", 2024-09-16T09:34:43.547 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.433833+0000", 2024-09-16T09:34:43.547 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00016096599999999999 2024-09-16T09:34:43.547 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.547 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.547 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:34:43.547 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.438355+0000", 2024-09-16T09:34:43.547 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.004522206 2024-09-16T09:34:43.547 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.547 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.547 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:34:43.547 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.438417+0000", 2024-09-16T09:34:43.547 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1888000000000001e-05 2024-09-16T09:34:43.548 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.548 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.548 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:34:43.548 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.438515+0000", 2024-09-16T09:34:43.548 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.8084999999999993e-05 2024-09-16T09:34:43.548 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.548 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.548 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:34:43.548 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.438550+0000", 2024-09-16T09:34:43.548 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5172000000000001e-05 2024-09-16T09:34:43.548 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.548 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.548 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:34:43.549 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.438652+0000", 2024-09-16T09:34:43.549 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00010160799999999999 2024-09-16T09:34:43.549 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:34:43.549 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:34:43.549 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:34:43.549 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.549 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.549 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:85297 9.19 9:9ca61566:::benchmark_data_smithi070_137000_object85296:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e266)", 2024-09-16T09:34:43.549 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:34:30.437608+0000", 2024-09-16T09:34:43.549 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 13.052861128, 2024-09-16T09:34:43.549 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0012625879999999, 2024-09-16T09:34:43.549 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:34:43.549 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:34:43.549 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:34:43.550 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:34:43.550 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:34:43.550 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:34:43.550 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 85297 2024-09-16T09:34:43.550 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.550 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:34:43.550 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.550 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:34:43.550 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.437608+0000", 2024-09-16T09:34:43.550 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:34:43.550 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.550 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.550 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:34:43.550 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.437608+0000", 2024-09-16T09:34:43.551 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:34:43.551 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.551 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.551 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:34:43.551 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.437610+0000", 2024-09-16T09:34:43.551 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.491e-06 2024-09-16T09:34:43.551 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.551 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.551 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:34:43.551 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.437634+0000", 2024-09-16T09:34:43.551 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3952e-05 2024-09-16T09:34:43.551 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.551 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.551 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:34:43.551 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.437649+0000", 2024-09-16T09:34:43.552 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5109000000000001e-05 2024-09-16T09:34:43.552 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.552 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.552 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:34:43.552 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.437657+0000", 2024-09-16T09:34:43.552 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.8150000000000006e-06 2024-09-16T09:34:43.552 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.552 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.552 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:34:43.552 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.437720+0000", 2024-09-16T09:34:43.552 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.3774000000000003e-05 2024-09-16T09:34:43.552 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.552 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.552 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:34:43.553 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.437770+0000", 2024-09-16T09:34:43.553 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.9468000000000001e-05 2024-09-16T09:34:43.553 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.553 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.553 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:34:43.553 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.434598+0000", 2024-09-16T09:34:43.553 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.99682824999999997 2024-09-16T09:34:43.553 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.553 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.553 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:34:43.553 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.434737+0000", 2024-09-16T09:34:43.553 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00013851299999999999 2024-09-16T09:34:43.553 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.553 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.554 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 5,6", 2024-09-16T09:34:43.554 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.434965+0000", 2024-09-16T09:34:43.554 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000228354 2024-09-16T09:34:43.554 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.554 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.554 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:34:43.554 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.438468+0000", 2024-09-16T09:34:43.554 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0035034340000000001 2024-09-16T09:34:43.554 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.554 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.554 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:34:43.554 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.438476+0000", 2024-09-16T09:34:43.554 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.072e-06 2024-09-16T09:34:43.554 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.555 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.555 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:34:43.555 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.438704+0000", 2024-09-16T09:34:43.555 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00022734400000000001 2024-09-16T09:34:43.555 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.555 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.555 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:34:43.555 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.438762+0000", 2024-09-16T09:34:43.555 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7778000000000001e-05 2024-09-16T09:34:43.555 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.555 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.555 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:34:43.555 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.438871+0000", 2024-09-16T09:34:43.555 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000109234 2024-09-16T09:34:43.556 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:34:43.556 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:34:43.556 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:34:43.556 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.556 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.556 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:85362 9.19 9:9e6a1315:::benchmark_data_smithi070_137000_object85361:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e266)", 2024-09-16T09:34:43.556 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:34:30.478936+0000", 2024-09-16T09:34:43.556 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 13.011533335999999, 2024-09-16T09:34:43.556 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.96017200800000002, 2024-09-16T09:34:43.556 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:34:43.556 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:34:43.556 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:34:43.556 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:34:43.556 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:34:43.557 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:34:43.557 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 85362 2024-09-16T09:34:43.557 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.557 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:34:43.557 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.557 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:34:43.557 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.478936+0000", 2024-09-16T09:34:43.557 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:34:43.557 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.557 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.557 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:34:43.557 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.478936+0000", 2024-09-16T09:34:43.557 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:34:43.557 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.558 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.558 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:34:43.558 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.478938+0000", 2024-09-16T09:34:43.558 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.074e-06 2024-09-16T09:34:43.558 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.558 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.558 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:34:43.558 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.478960+0000", 2024-09-16T09:34:43.558 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.2323999999999998e-05 2024-09-16T09:34:43.558 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.558 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.558 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:34:43.558 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.478975+0000", 2024-09-16T09:34:43.559 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.4656e-05 2024-09-16T09:34:43.559 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.559 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.559 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:34:43.559 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.478983+0000", 2024-09-16T09:34:43.559 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.7959999999999996e-06 2024-09-16T09:34:43.559 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.559 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.559 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:34:43.559 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.479053+0000", 2024-09-16T09:34:43.559 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.0017000000000004e-05 2024-09-16T09:34:43.559 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.559 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.559 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:34:43.560 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.479063+0000", 2024-09-16T09:34:43.560 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0475000000000001e-05 2024-09-16T09:34:43.560 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.560 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.560 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:34:43.560 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.435981+0000", 2024-09-16T09:34:43.560 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.95691784400000002 2024-09-16T09:34:43.560 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.560 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.560 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:34:43.560 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.436119+0000", 2024-09-16T09:34:43.560 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000138102 2024-09-16T09:34:43.560 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.561 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.561 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 5,6", 2024-09-16T09:34:43.561 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.436359+0000", 2024-09-16T09:34:43.561 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00023921200000000001 2024-09-16T09:34:43.561 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.561 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.561 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:34:43.561 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.438480+0000", 2024-09-16T09:34:43.561 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00212174 2024-09-16T09:34:43.561 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.561 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.561 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:34:43.561 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.438923+0000", 2024-09-16T09:34:43.561 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00044283799999999998 2024-09-16T09:34:43.562 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.562 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.562 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:34:43.562 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.438975+0000", 2024-09-16T09:34:43.562 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.1765000000000003e-05 2024-09-16T09:34:43.562 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.562 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.562 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:34:43.562 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.439009+0000", 2024-09-16T09:34:43.562 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3649000000000003e-05 2024-09-16T09:34:43.562 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.562 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.562 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:34:43.562 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.439108+0000", 2024-09-16T09:34:43.563 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.9516000000000001e-05 2024-09-16T09:34:43.563 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:34:43.563 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:34:43.563 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:34:43.563 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.563 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.563 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:85372 9.19 9:9c0a1df9:::benchmark_data_smithi070_137000_object85371:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e266)", 2024-09-16T09:34:43.563 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:34:30.485504+0000", 2024-09-16T09:34:43.563 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 13.004964966999999, 2024-09-16T09:34:43.563 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.95555173800000004, 2024-09-16T09:34:43.563 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:34:43.563 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:34:43.563 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:34:43.564 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:34:43.564 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:34:43.564 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:34:43.564 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 85372 2024-09-16T09:34:43.564 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.564 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:34:43.564 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.564 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:34:43.564 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.485504+0000", 2024-09-16T09:34:43.564 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:34:43.564 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.564 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.565 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:34:43.565 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.485504+0000", 2024-09-16T09:34:43.565 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:34:43.565 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.565 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.565 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:34:43.565 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.485506+0000", 2024-09-16T09:34:43.565 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.849e-06 2024-09-16T09:34:43.565 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.565 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.565 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:34:43.565 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.485539+0000", 2024-09-16T09:34:43.565 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2876999999999997e-05 2024-09-16T09:34:43.565 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.566 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.566 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:34:43.566 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.485555+0000", 2024-09-16T09:34:43.566 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5863999999999999e-05 2024-09-16T09:34:43.566 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.566 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.566 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:34:43.566 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.485564+0000", 2024-09-16T09:34:43.566 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.4970000000000007e-06 2024-09-16T09:34:43.566 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.566 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.566 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:34:43.566 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.485650+0000", 2024-09-16T09:34:43.566 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.5865999999999994e-05 2024-09-16T09:34:43.566 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.567 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.567 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:34:43.567 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.485663+0000", 2024-09-16T09:34:43.567 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2605999999999999e-05 2024-09-16T09:34:43.567 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.567 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.568 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:34:43.568 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.437202+0000", 2024-09-16T09:34:43.568 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.95153927000000005 2024-09-16T09:34:43.568 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.568 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.568 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:34:43.569 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.437307+0000", 2024-09-16T09:34:43.569 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00010431500000000001 2024-09-16T09:34:43.569 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.569 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.569 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 5,6", 2024-09-16T09:34:43.569 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.437536+0000", 2024-09-16T09:34:43.569 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00022945499999999999 2024-09-16T09:34:43.569 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.569 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.569 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:34:43.569 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.440781+0000", 2024-09-16T09:34:43.569 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00324522 2024-09-16T09:34:43.569 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.569 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.570 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:34:43.570 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.440836+0000", 2024-09-16T09:34:43.570 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5237e-05 2024-09-16T09:34:43.570 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.570 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.570 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:34:43.570 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.440895+0000", 2024-09-16T09:34:43.570 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8075000000000002e-05 2024-09-16T09:34:43.570 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.570 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.570 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:34:43.570 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.440945+0000", 2024-09-16T09:34:43.570 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0782999999999998e-05 2024-09-16T09:34:43.570 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.571 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.571 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:34:43.571 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.441056+0000", 2024-09-16T09:34:43.571 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00011082399999999999 2024-09-16T09:34:43.571 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:34:43.571 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:34:43.571 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:34:43.571 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.571 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.571 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:85393 9.19 9:9bf54bf1:::benchmark_data_smithi070_137000_object85392:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e266)", 2024-09-16T09:34:43.571 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:34:30.501393+0000", 2024-09-16T09:34:43.571 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 12.98907631, 2024-09-16T09:34:43.571 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.94266689699999995, 2024-09-16T09:34:43.571 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:34:43.572 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:34:43.572 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:34:43.572 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:34:43.572 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:34:43.572 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:34:43.572 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 85393 2024-09-16T09:34:43.572 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.572 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:34:43.572 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.572 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:34:43.572 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.501393+0000", 2024-09-16T09:34:43.572 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:34:43.572 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.572 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.573 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:34:43.573 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.501393+0000", 2024-09-16T09:34:43.573 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:34:43.573 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.573 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.573 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:34:43.573 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.501395+0000", 2024-09-16T09:34:43.573 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.699e-06 2024-09-16T09:34:43.573 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.573 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.573 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:34:43.573 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.501419+0000", 2024-09-16T09:34:43.573 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3853999999999999e-05 2024-09-16T09:34:43.573 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.574 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.574 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:34:43.574 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.501438+0000", 2024-09-16T09:34:43.574 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9306000000000001e-05 2024-09-16T09:34:43.574 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.574 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.574 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:34:43.574 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.501446+0000", 2024-09-16T09:34:43.574 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.2239999999999995e-06 2024-09-16T09:34:43.574 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.574 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.574 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:34:43.574 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.501515+0000", 2024-09-16T09:34:43.575 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.8873000000000003e-05 2024-09-16T09:34:43.575 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.575 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.575 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:34:43.575 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.501523+0000", 2024-09-16T09:34:43.575 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.5619999999999998e-06 2024-09-16T09:34:43.575 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.575 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.575 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:34:43.575 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.439271+0000", 2024-09-16T09:34:43.575 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.93774797300000001 2024-09-16T09:34:43.575 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.575 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.575 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:34:43.576 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.439369+0000", 2024-09-16T09:34:43.576 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.8064000000000002e-05 2024-09-16T09:34:43.576 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.576 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.576 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 5,6", 2024-09-16T09:34:43.576 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.439624+0000", 2024-09-16T09:34:43.576 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000254944 2024-09-16T09:34:43.576 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.576 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.576 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:34:43.576 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.443757+0000", 2024-09-16T09:34:43.576 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0041332360000000002 2024-09-16T09:34:43.576 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.576 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.577 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:34:43.577 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.443814+0000", 2024-09-16T09:34:43.577 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7192999999999999e-05 2024-09-16T09:34:43.577 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.577 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.577 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:34:43.577 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.443853+0000", 2024-09-16T09:34:43.577 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9413999999999999e-05 2024-09-16T09:34:43.577 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.577 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.577 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:34:43.577 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.443926+0000", 2024-09-16T09:34:43.577 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.2818000000000007e-05 2024-09-16T09:34:43.577 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.578 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.578 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:34:43.578 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.444060+0000", 2024-09-16T09:34:43.578 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00013373700000000001 2024-09-16T09:34:43.578 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:34:43.578 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:34:43.578 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:34:43.578 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.578 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.578 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:85403 9.19 9:9fe05e91:::benchmark_data_smithi070_137000_object85402:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e266)", 2024-09-16T09:34:43.578 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:34:30.507369+0000", 2024-09-16T09:34:43.578 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 12.983100565000001, 2024-09-16T09:34:43.578 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.94047342, 2024-09-16T09:34:43.578 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:34:43.579 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:34:43.579 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:34:43.579 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:34:43.579 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:34:43.579 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:34:43.579 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 85403 2024-09-16T09:34:43.579 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.579 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:34:43.579 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.579 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:34:43.579 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.507369+0000", 2024-09-16T09:34:43.579 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:34:43.579 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.579 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.579 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:34:43.580 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.507369+0000", 2024-09-16T09:34:43.580 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:34:43.580 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.580 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.580 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:34:43.580 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.507371+0000", 2024-09-16T09:34:43.580 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.841e-06 2024-09-16T09:34:43.580 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.580 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.580 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:34:43.580 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.507399+0000", 2024-09-16T09:34:43.580 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.8099999999999999e-05 2024-09-16T09:34:43.580 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.581 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.581 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:34:43.581 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.507416+0000", 2024-09-16T09:34:43.581 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7606999999999999e-05 2024-09-16T09:34:43.581 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.581 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.581 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:34:43.581 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.507541+0000", 2024-09-16T09:34:43.581 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00012422799999999999 2024-09-16T09:34:43.581 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.581 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.581 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:34:43.581 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.508065+0000", 2024-09-16T09:34:43.581 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00052490600000000001 2024-09-16T09:34:43.582 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.582 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.582 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:34:43.582 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.508077+0000", 2024-09-16T09:34:43.582 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1532000000000001e-05 2024-09-16T09:34:43.582 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.582 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.582 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:34:43.582 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.441107+0000", 2024-09-16T09:34:43.582 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.93303015 2024-09-16T09:34:43.582 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.582 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.582 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:34:43.582 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.441234+0000", 2024-09-16T09:34:43.583 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00012694099999999999 2024-09-16T09:34:43.583 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.583 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.583 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 5,6", 2024-09-16T09:34:43.583 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.441499+0000", 2024-09-16T09:34:43.583 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00026452700000000002 2024-09-16T09:34:43.583 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.583 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.583 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:34:43.583 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.444075+0000", 2024-09-16T09:34:43.583 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0025765670000000001 2024-09-16T09:34:43.583 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.583 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.583 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:34:43.584 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.447637+0000", 2024-09-16T09:34:43.584 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.003562164 2024-09-16T09:34:43.584 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.584 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.584 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:34:43.584 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.447676+0000", 2024-09-16T09:34:43.584 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8618000000000002e-05 2024-09-16T09:34:43.584 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.584 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.584 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:34:43.584 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.447730+0000", 2024-09-16T09:34:43.584 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.3699999999999997e-05 2024-09-16T09:34:43.584 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.585 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.585 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:34:43.585 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.447842+0000", 2024-09-16T09:34:43.585 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000112539 2024-09-16T09:34:43.585 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:34:43.585 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:34:43.585 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:34:43.585 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.585 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.585 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:85439 9.19 9:9e006a6c:::benchmark_data_smithi070_137000_object85438:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e266)", 2024-09-16T09:34:43.585 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:34:30.539586+0000", 2024-09-16T09:34:43.585 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 12.950883786, 2024-09-16T09:34:43.585 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.90846788000000001, 2024-09-16T09:34:43.585 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:34:43.586 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:34:43.586 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:34:43.586 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:34:43.586 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:34:43.586 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:34:43.586 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 85439 2024-09-16T09:34:43.586 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.586 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:34:43.586 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.586 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:34:43.586 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.539586+0000", 2024-09-16T09:34:43.586 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:34:43.586 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.586 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.587 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:34:43.587 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.539586+0000", 2024-09-16T09:34:43.587 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:34:43.587 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.587 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.587 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:34:43.587 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.539587+0000", 2024-09-16T09:34:43.587 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.855e-06 2024-09-16T09:34:43.587 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.587 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.587 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:34:43.587 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.539611+0000", 2024-09-16T09:34:43.587 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3862999999999998e-05 2024-09-16T09:34:43.587 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.588 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.588 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:34:43.588 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.539626+0000", 2024-09-16T09:34:43.588 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.4827999999999999e-05 2024-09-16T09:34:43.588 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.588 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.588 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:34:43.588 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.539633+0000", 2024-09-16T09:34:43.588 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.1509999999999998e-06 2024-09-16T09:34:43.588 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.588 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.588 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:34:43.588 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.540101+0000", 2024-09-16T09:34:43.589 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00046737900000000003 2024-09-16T09:34:43.589 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.589 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.589 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:34:43.589 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.540111+0000", 2024-09-16T09:34:43.589 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0175e-05 2024-09-16T09:34:43.589 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.589 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.589 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:34:43.589 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.442534+0000", 2024-09-16T09:34:43.589 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.90242282900000004 2024-09-16T09:34:43.589 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.589 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.589 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:34:43.590 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.442662+0000", 2024-09-16T09:34:43.590 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000127916 2024-09-16T09:34:43.590 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.590 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.590 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 5,6", 2024-09-16T09:34:43.590 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.442912+0000", 2024-09-16T09:34:43.590 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00025042800000000002 2024-09-16T09:34:43.590 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.590 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.590 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:34:43.590 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.445532+0000", 2024-09-16T09:34:43.590 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0026201610000000002 2024-09-16T09:34:43.590 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.590 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.590 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:34:43.591 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.447883+0000", 2024-09-16T09:34:43.591 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0023513309999999999 2024-09-16T09:34:43.591 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.591 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.591 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:34:43.591 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.447918+0000", 2024-09-16T09:34:43.591 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4714e-05 2024-09-16T09:34:43.591 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.591 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.591 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:34:43.591 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.447958+0000", 2024-09-16T09:34:43.591 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0037999999999999e-05 2024-09-16T09:34:43.591 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.591 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.592 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:34:43.592 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.448053+0000", 2024-09-16T09:34:43.592 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.5211999999999996e-05 2024-09-16T09:34:43.592 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:34:43.592 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:34:43.592 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:34:43.592 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.592 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.592 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:85469 9.19 9:9aef5b3d:::benchmark_data_smithi070_137000_object85468:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e266)", 2024-09-16T09:34:43.592 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:34:30.561856+0000", 2024-09-16T09:34:43.592 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 12.928613243000001, 2024-09-16T09:34:43.592 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.89085814100000005, 2024-09-16T09:34:43.592 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:34:43.592 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:34:43.593 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:34:43.593 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:34:43.593 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:34:43.593 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:34:43.593 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 85469 2024-09-16T09:34:43.593 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.593 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:34:43.593 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.593 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:34:43.593 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.561856+0000", 2024-09-16T09:34:43.593 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:34:43.593 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.593 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.593 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:34:43.594 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.561856+0000", 2024-09-16T09:34:43.594 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:34:43.594 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.594 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.594 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:34:43.594 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.561858+0000", 2024-09-16T09:34:43.594 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9240000000000001e-06 2024-09-16T09:34:43.594 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.594 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.594 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:34:43.594 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.561884+0000", 2024-09-16T09:34:43.594 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.5800999999999999e-05 2024-09-16T09:34:43.594 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.594 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.595 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:34:43.595 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.561905+0000", 2024-09-16T09:34:43.595 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0781e-05 2024-09-16T09:34:43.595 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.595 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.595 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:34:43.595 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.561915+0000", 2024-09-16T09:34:43.595 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.9769999999999995e-06 2024-09-16T09:34:43.595 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.595 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.595 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:34:43.595 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.561988+0000", 2024-09-16T09:34:43.595 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3320999999999998e-05 2024-09-16T09:34:43.596 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.596 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.596 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:34:43.596 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.561998+0000", 2024-09-16T09:34:43.596 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0348e-05 2024-09-16T09:34:43.596 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.596 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.596 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:34:43.596 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.444104+0000", 2024-09-16T09:34:43.596 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.88210558999999999 2024-09-16T09:34:43.597 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.597 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.597 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:34:43.597 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.444236+0000", 2024-09-16T09:34:43.597 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000131791 2024-09-16T09:34:43.597 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.597 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.597 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 5,6", 2024-09-16T09:34:43.597 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.444465+0000", 2024-09-16T09:34:43.597 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00022897799999999999 2024-09-16T09:34:43.597 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.597 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.597 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:34:43.597 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.448090+0000", 2024-09-16T09:34:43.598 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0036256389999999999 2024-09-16T09:34:43.598 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.598 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.598 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:34:43.598 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.448128+0000", 2024-09-16T09:34:43.598 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8000999999999998e-05 2024-09-16T09:34:43.598 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.598 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.598 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:34:43.598 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.452599+0000", 2024-09-16T09:34:43.598 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.004471226 2024-09-16T09:34:43.598 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.598 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.598 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:34:43.599 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.452634+0000", 2024-09-16T09:34:43.599 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4146999999999997e-05 2024-09-16T09:34:43.599 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.599 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.599 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:34:43.599 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.452714+0000", 2024-09-16T09:34:43.599 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.0617000000000004e-05 2024-09-16T09:34:43.599 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:34:43.599 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:34:43.599 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:34:43.599 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.599 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.599 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:85476 9.19 9:9a6bb818:::benchmark_data_smithi070_137000_object85475:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e266)", 2024-09-16T09:34:43.600 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:34:30.568168+0000", 2024-09-16T09:34:43.600 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 12.922301517999999, 2024-09-16T09:34:43.600 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.88463710600000001, 2024-09-16T09:34:43.600 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:34:43.600 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:34:43.600 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:34:43.600 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:34:43.600 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:34:43.600 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:34:43.600 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 85476 2024-09-16T09:34:43.600 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.600 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:34:43.600 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.600 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:34:43.601 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.568168+0000", 2024-09-16T09:34:43.601 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:34:43.601 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.601 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.601 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:34:43.601 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.568168+0000", 2024-09-16T09:34:43.601 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:34:43.601 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.601 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.601 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:34:43.601 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.568170+0000", 2024-09-16T09:34:43.601 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.894e-06 2024-09-16T09:34:43.601 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.601 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.602 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:34:43.602 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.568196+0000", 2024-09-16T09:34:43.602 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.6284000000000001e-05 2024-09-16T09:34:43.602 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.602 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.602 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:34:43.602 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.568214+0000", 2024-09-16T09:34:43.602 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.8141000000000001e-05 2024-09-16T09:34:43.602 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.602 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.602 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:34:43.602 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.568223+0000", 2024-09-16T09:34:43.602 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.5960000000000004e-06 2024-09-16T09:34:43.602 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.603 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.603 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:34:43.603 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.568300+0000", 2024-09-16T09:34:43.603 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.7408000000000002e-05 2024-09-16T09:34:43.603 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.603 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.603 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:34:43.603 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.568312+0000", 2024-09-16T09:34:43.603 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1671999999999999e-05 2024-09-16T09:34:43.603 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.603 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.603 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:34:43.603 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.445787+0000", 2024-09-16T09:34:43.603 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.87747500599999995 2024-09-16T09:34:43.604 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.604 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.604 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:34:43.604 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.445938+0000", 2024-09-16T09:34:43.604 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00015162300000000001 2024-09-16T09:34:43.604 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.604 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.604 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 5,6", 2024-09-16T09:34:43.604 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.446279+0000", 2024-09-16T09:34:43.604 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00034082399999999999 2024-09-16T09:34:43.604 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.604 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.604 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:34:43.604 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.448205+0000", 2024-09-16T09:34:43.605 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0019254560000000001 2024-09-16T09:34:43.605 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.605 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.605 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:34:43.605 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.451146+0000", 2024-09-16T09:34:43.605 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0029411289999999998 2024-09-16T09:34:43.605 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.605 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.605 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:34:43.605 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.452725+0000", 2024-09-16T09:34:43.605 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0015790890000000001 2024-09-16T09:34:43.605 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.605 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.605 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:34:43.606 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.452751+0000", 2024-09-16T09:34:43.606 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.6052000000000001e-05 2024-09-16T09:34:43.606 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.606 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.606 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:34:43.606 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.452805+0000", 2024-09-16T09:34:43.606 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.3931999999999998e-05 2024-09-16T09:34:43.606 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:34:43.606 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:34:43.606 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:34:43.606 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.606 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.606 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:85486 9.19 9:98235907:::benchmark_data_smithi070_137000_object85485:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e266)", 2024-09-16T09:34:43.606 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:34:30.578103+0000", 2024-09-16T09:34:43.607 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 12.912366068000001, 2024-09-16T09:34:43.607 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.87747065000000002, 2024-09-16T09:34:43.607 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:34:43.607 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:34:43.607 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:34:43.607 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:34:43.607 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:34:43.607 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:34:43.607 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 85486 2024-09-16T09:34:43.607 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.607 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:34:43.607 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.607 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:34:43.607 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.578103+0000", 2024-09-16T09:34:43.607 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:34:43.608 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.608 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.608 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:34:43.608 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.578103+0000", 2024-09-16T09:34:43.608 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:34:43.608 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.608 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.608 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:34:43.608 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.578105+0000", 2024-09-16T09:34:43.608 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0020000000000001e-06 2024-09-16T09:34:43.608 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.608 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.608 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:34:43.608 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.578133+0000", 2024-09-16T09:34:43.609 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.7464999999999999e-05 2024-09-16T09:34:43.609 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.609 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.609 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:34:43.609 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.578154+0000", 2024-09-16T09:34:43.609 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0874999999999999e-05 2024-09-16T09:34:43.609 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.609 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.609 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:34:43.609 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.578164+0000", 2024-09-16T09:34:43.609 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0208e-05 2024-09-16T09:34:43.609 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.609 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.609 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:34:43.610 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.578247+0000", 2024-09-16T09:34:43.610 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.2780000000000004e-05 2024-09-16T09:34:43.610 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.610 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.610 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:34:43.610 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.578259+0000", 2024-09-16T09:34:43.610 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2328e-05 2024-09-16T09:34:43.610 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.610 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.610 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:34:43.610 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.448234+0000", 2024-09-16T09:34:43.610 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.86997486899999998 2024-09-16T09:34:43.610 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.610 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.611 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:34:43.611 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.448365+0000", 2024-09-16T09:34:43.611 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00013145 2024-09-16T09:34:43.611 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.611 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.611 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 5,6", 2024-09-16T09:34:43.611 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.448681+0000", 2024-09-16T09:34:43.611 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00031555899999999998 2024-09-16T09:34:43.611 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.611 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.611 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:34:43.611 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.452812+0000", 2024-09-16T09:34:43.611 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0041307669999999996 2024-09-16T09:34:43.611 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.612 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.612 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:34:43.612 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.452908+0000", 2024-09-16T09:34:43.612 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.6566999999999999e-05 2024-09-16T09:34:43.612 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.612 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.612 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:34:43.612 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.455236+0000", 2024-09-16T09:34:43.612 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0023281819999999998 2024-09-16T09:34:43.612 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.612 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.612 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:34:43.612 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.455306+0000", 2024-09-16T09:34:43.613 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.9658000000000003e-05 2024-09-16T09:34:43.613 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.613 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.613 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:34:43.613 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.455574+0000", 2024-09-16T09:34:43.613 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00026793999999999999 2024-09-16T09:34:43.613 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:34:43.613 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:34:43.613 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:34:43.613 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.613 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.613 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:85565 9.19 9:987f344a:::benchmark_data_smithi070_137000_object85564:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e266)", 2024-09-16T09:34:43.613 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:34:30.683479+0000", 2024-09-16T09:34:43.614 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 12.806990631, 2024-09-16T09:34:43.614 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.77327836599999999, 2024-09-16T09:34:43.614 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:34:43.614 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:34:43.614 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:34:43.614 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:34:43.614 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:34:43.614 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:34:43.614 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 85565 2024-09-16T09:34:43.614 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.614 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:34:43.614 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.614 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:34:43.614 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.683479+0000", 2024-09-16T09:34:43.615 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:34:43.615 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.615 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.615 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:34:43.615 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.683479+0000", 2024-09-16T09:34:43.615 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:34:43.615 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.615 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.615 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:34:43.615 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.683480+0000", 2024-09-16T09:34:43.615 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.4169999999999999e-06 2024-09-16T09:34:43.615 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.615 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.615 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:34:43.616 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.683500+0000", 2024-09-16T09:34:43.616 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9652000000000001e-05 2024-09-16T09:34:43.616 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.616 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.616 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:34:43.616 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.683511+0000", 2024-09-16T09:34:43.616 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1561e-05 2024-09-16T09:34:43.616 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.616 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.616 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:34:43.616 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.683517+0000", 2024-09-16T09:34:43.616 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7919999999999998e-06 2024-09-16T09:34:43.616 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.617 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.617 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:34:43.617 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.683585+0000", 2024-09-16T09:34:43.617 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.7953999999999993e-05 2024-09-16T09:34:43.617 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.617 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.617 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:34:43.617 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.683594+0000", 2024-09-16T09:34:43.617 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.2669999999999995e-06 2024-09-16T09:34:43.617 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.617 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.617 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:34:43.617 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.449881+0000", 2024-09-16T09:34:43.617 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.76628674200000002 2024-09-16T09:34:43.618 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.618 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.618 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:34:43.618 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.450007+0000", 2024-09-16T09:34:43.618 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00012549499999999999 2024-09-16T09:34:43.618 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.618 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.618 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 5,6", 2024-09-16T09:34:43.618 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.450194+0000", 2024-09-16T09:34:43.618 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00018697900000000001 2024-09-16T09:34:43.618 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.618 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.618 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:34:43.619 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.452942+0000", 2024-09-16T09:34:43.619 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0027483329999999999 2024-09-16T09:34:43.619 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.619 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.619 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:34:43.619 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.452953+0000", 2024-09-16T09:34:43.619 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0868e-05 2024-09-16T09:34:43.619 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.619 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.619 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:34:43.619 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.456605+0000", 2024-09-16T09:34:43.619 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0036523020000000001 2024-09-16T09:34:43.619 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.619 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.619 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:34:43.620 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.456664+0000", 2024-09-16T09:34:43.620 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8504000000000001e-05 2024-09-16T09:34:43.620 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.620 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.620 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:34:43.620 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.456757+0000", 2024-09-16T09:34:43.620 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.3499999999999996e-05 2024-09-16T09:34:43.620 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:34:43.620 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:34:43.620 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:34:43.620 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.620 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.620 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:85568 9.19 9:98328676:::benchmark_data_smithi070_137000_object85567:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e266)", 2024-09-16T09:34:43.621 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:34:30.696918+0000", 2024-09-16T09:34:43.621 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 12.793551556000001, 2024-09-16T09:34:43.621 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.76423191800000001, 2024-09-16T09:34:43.621 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:34:43.621 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:34:43.621 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:34:43.621 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:34:43.621 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:34:43.621 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:34:43.621 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 85568 2024-09-16T09:34:43.621 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.621 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:34:43.621 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.621 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:34:43.622 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.696918+0000", 2024-09-16T09:34:43.622 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:34:43.622 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.622 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.622 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:34:43.622 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.696918+0000", 2024-09-16T09:34:43.622 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:34:43.622 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.622 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.622 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:34:43.622 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.696920+0000", 2024-09-16T09:34:43.622 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0600000000000002e-06 2024-09-16T09:34:43.623 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.623 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.624 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:34:43.624 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.696947+0000", 2024-09-16T09:34:43.624 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.7351999999999998e-05 2024-09-16T09:34:43.624 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.624 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.624 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:34:43.624 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.696967+0000", 2024-09-16T09:34:43.624 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0174999999999999e-05 2024-09-16T09:34:43.624 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.624 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.624 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:34:43.624 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.696978+0000", 2024-09-16T09:34:43.624 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0125999999999999e-05 2024-09-16T09:34:43.625 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.625 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.625 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:34:43.625 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.697039+0000", 2024-09-16T09:34:43.625 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1815000000000004e-05 2024-09-16T09:34:43.625 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.625 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.625 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:34:43.625 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.697047+0000", 2024-09-16T09:34:43.625 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6890000000000004e-06 2024-09-16T09:34:43.625 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.625 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.625 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:34:43.625 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.451181+0000", 2024-09-16T09:34:43.626 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.75413436899999997 2024-09-16T09:34:43.626 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.626 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.626 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:34:43.626 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.451322+0000", 2024-09-16T09:34:43.626 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00014107 2024-09-16T09:34:43.626 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.626 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.626 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 5,6", 2024-09-16T09:34:43.626 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.451599+0000", 2024-09-16T09:34:43.626 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00027628800000000002 2024-09-16T09:34:43.626 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.626 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.626 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:34:43.627 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.455193+0000", 2024-09-16T09:34:43.627 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0035944610000000002 2024-09-16T09:34:43.627 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.627 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.627 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:34:43.627 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.456175+0000", 2024-09-16T09:34:43.627 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00098208999999999992 2024-09-16T09:34:43.627 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.627 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.627 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:34:43.627 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.460855+0000", 2024-09-16T09:34:43.627 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0046800820000000003 2024-09-16T09:34:43.627 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.627 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.628 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:34:43.628 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.460909+0000", 2024-09-16T09:34:43.628 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.346e-05 2024-09-16T09:34:43.628 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.628 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.628 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:34:43.628 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.461150+0000", 2024-09-16T09:34:43.628 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000240881 2024-09-16T09:34:43.628 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:34:43.628 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:34:43.628 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:34:43.628 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.628 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.628 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:85625 9.19 9:9dc68788:::benchmark_data_smithi070_137000_object85624:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e266)", 2024-09-16T09:34:43.629 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:34:30.886964+0000", 2024-09-16T09:34:43.629 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 12.603505023, 2024-09-16T09:34:43.629 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.57467557000000002, 2024-09-16T09:34:43.629 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:34:43.629 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:34:43.629 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:34:43.629 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:34:43.629 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:34:43.629 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:34:43.629 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 85625 2024-09-16T09:34:43.629 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.629 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:34:43.629 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.630 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:34:43.630 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.886964+0000", 2024-09-16T09:34:43.630 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:34:43.630 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.630 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.630 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:34:43.630 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.886964+0000", 2024-09-16T09:34:43.630 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:34:43.630 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.630 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.630 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:34:43.630 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.886966+0000", 2024-09-16T09:34:43.630 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2699999999999999e-06 2024-09-16T09:34:43.630 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.630 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.631 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:34:43.631 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.886983+0000", 2024-09-16T09:34:43.631 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7880000000000002e-05 2024-09-16T09:34:43.631 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.631 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.631 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:34:43.631 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.886995+0000", 2024-09-16T09:34:43.631 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1573000000000001e-05 2024-09-16T09:34:43.631 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.631 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.631 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:34:43.631 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.887000+0000", 2024-09-16T09:34:43.631 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.339e-06 2024-09-16T09:34:43.632 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.632 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.632 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:34:43.632 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.887058+0000", 2024-09-16T09:34:43.632 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7649000000000003e-05 2024-09-16T09:34:43.632 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.632 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.632 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:34:43.632 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.887066+0000", 2024-09-16T09:34:43.632 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.0479999999999999e-06 2024-09-16T09:34:43.632 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.632 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.632 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:34:43.632 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.453001+0000", 2024-09-16T09:34:43.633 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.56593447600000002 2024-09-16T09:34:43.633 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.633 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.633 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:34:43.633 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.453090+0000", 2024-09-16T09:34:43.633 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.9319000000000003e-05 2024-09-16T09:34:43.633 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.633 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.633 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 5,6", 2024-09-16T09:34:43.633 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.453251+0000", 2024-09-16T09:34:43.633 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000161244 2024-09-16T09:34:43.633 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.633 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.634 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:34:43.634 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.455625+0000", 2024-09-16T09:34:43.634 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0023737789999999999 2024-09-16T09:34:43.634 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.634 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.634 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:34:43.634 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.456217+0000", 2024-09-16T09:34:43.634 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00059225900000000004 2024-09-16T09:34:43.634 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.634 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.634 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:34:43.634 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.461229+0000", 2024-09-16T09:34:43.634 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0050117080000000001 2024-09-16T09:34:43.634 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.635 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.635 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:34:43.635 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.461292+0000", 2024-09-16T09:34:43.635 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.2854999999999994e-05 2024-09-16T09:34:43.635 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:34:43.635 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:34:43.635 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:34:43.635 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.461640+0000", 2024-09-16T09:34:43.635 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00034817099999999998 2024-09-16T09:34:43.635 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:34:43.635 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:34:43.635 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:34:43.635 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:34:43.635 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:34:43.636 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T09:34:43.636 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:43 smithi070 ceph-mon[30388]: pgmap v478: 178 pgs: 2 active+recovering+undersized+remapped, 176 active+clean; 5.7 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.2 KiB/s rd, 63 MiB/s wr, 1.01k op/s; 2789/282666 objects misplaced (0.987%) 2024-09-16T09:34:43.636 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:43 smithi070 ceph-mon[30388]: 9.8 scrub starts 2024-09-16T09:34:43.636 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:43 smithi070 ceph-mon[30388]: 1.4 scrub starts 2024-09-16T09:34:43.636 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:43 smithi070 ceph-mon[30388]: 1.4 scrub ok 2024-09-16T09:34:43.636 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:43 smithi070 ceph-mon[30388]: 9.8 scrub ok 2024-09-16T09:34:43.636 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:43 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1921864480' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:34:43.636 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:43 smithi070 ceph-mon[30388]: 1.6 scrub starts 2024-09-16T09:34:43.636 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:43 smithi070 ceph-mon[30388]: 1.6 scrub ok 2024-09-16T09:34:43.637 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:43 smithi070 ceph-mon[40691]: pgmap v478: 178 pgs: 2 active+recovering+undersized+remapped, 176 active+clean; 5.7 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.2 KiB/s rd, 63 MiB/s wr, 1.01k op/s; 2789/282666 objects misplaced (0.987%) 2024-09-16T09:34:43.637 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:43 smithi070 ceph-mon[40691]: 9.8 scrub starts 2024-09-16T09:34:43.637 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:43 smithi070 ceph-mon[40691]: 1.4 scrub starts 2024-09-16T09:34:43.637 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:43 smithi070 ceph-mon[40691]: 1.4 scrub ok 2024-09-16T09:34:43.637 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:43 smithi070 ceph-mon[40691]: 9.8 scrub ok 2024-09-16T09:34:43.637 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:43 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1921864480' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:34:43.637 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:43 smithi070 ceph-mon[40691]: 1.6 scrub starts 2024-09-16T09:34:43.637 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:43 smithi070 ceph-mon[40691]: 1.6 scrub ok 2024-09-16T09:34:43.638 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:43.638 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:43.638 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:34:43.716 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 73 16 101104 101088 86.5349 88.9375 0.0191924 0.0115514 2024-09-16T09:34:43.739 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:34:43.778 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:43.778 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:43.779 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:34:43.880 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:34:43.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:43 smithi084 ceph-mon[32629]: pgmap v478: 178 pgs: 2 active+recovering+undersized+remapped, 176 active+clean; 5.7 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.2 KiB/s rd, 63 MiB/s wr, 1.01k op/s; 2789/282666 objects misplaced (0.987%) 2024-09-16T09:34:43.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:43 smithi084 ceph-mon[32629]: 9.8 scrub starts 2024-09-16T09:34:43.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:43 smithi084 ceph-mon[32629]: 1.4 scrub starts 2024-09-16T09:34:43.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:43 smithi084 ceph-mon[32629]: 1.4 scrub ok 2024-09-16T09:34:43.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:43 smithi084 ceph-mon[32629]: 9.8 scrub ok 2024-09-16T09:34:43.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:43 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1921864480' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:34:43.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:43 smithi084 ceph-mon[32629]: 1.6 scrub starts 2024-09-16T09:34:43.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:43 smithi084 ceph-mon[32629]: 1.6 scrub ok 2024-09-16T09:34:43.911 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:43.912 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:43.912 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:34:44.013 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:34:44.054 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:44.054 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:44.055 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:34:44.156 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:34:44.196 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:44.197 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:44.198 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:34:44.298 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:34:44.335 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:44.336 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:44.336 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:34:44.437 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:34:44.477 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:44.477 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:44.478 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:34:44.579 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:34:44.621 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:44.622 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:44.622 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:34:44.716 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 74 16 102361 102345 86.4271 78.5625 0.0035145 0.0115656 2024-09-16T09:34:44.722 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:34:44.762 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:44.762 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:44.763 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:34:44.774 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:44 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:34:44.775 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:44 smithi070 ceph-mon[30388]: 9.13 scrub starts 2024-09-16T09:34:44.775 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:44 smithi070 ceph-mon[30388]: 9.16 scrub starts 2024-09-16T09:34:44.775 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:44 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:34:44.775 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:44 smithi070 ceph-mon[40691]: 9.13 scrub starts 2024-09-16T09:34:44.775 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:44 smithi070 ceph-mon[40691]: 9.16 scrub starts 2024-09-16T09:34:44.864 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:34:44.885 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:44.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:44.886 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:34:44.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:44 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:34:44.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:44 smithi084 ceph-mon[32629]: 9.13 scrub starts 2024-09-16T09:34:44.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:44 smithi084 ceph-mon[32629]: 9.16 scrub starts 2024-09-16T09:34:44.987 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:34:45.023 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:45.023 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:45.024 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:34:45.125 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:34:45.162 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:45.163 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:45.163 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:34:45.264 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:34:45.305 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:45.306 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:45.306 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:34:45.407 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:34:45.450 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:45.450 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:45.451 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:34:45.552 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:34:45.587 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:45.587 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:45.588 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:34:45.689 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:34:45.716 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 75 16 103508 103492 86.2305 71.6875 0.0086156 0.0115923 2024-09-16T09:34:45.720 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:45.721 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:45.725 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:34:45.826 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:34:45.862 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:45.863 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:45.864 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:34:45.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:45 smithi084 ceph-mon[32629]: pgmap v479: 178 pgs: 1 active+recovering+undersized+remapped, 177 active+clean; 6.1 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.7 KiB/s rd, 69 MiB/s wr, 1.11k op/s; 2095/300834 objects misplaced (0.696%) 2024-09-16T09:34:45.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:45 smithi084 ceph-mon[32629]: 9.16 scrub ok 2024-09-16T09:34:45.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:45 smithi084 ceph-mon[32629]: 9.13 scrub ok 2024-09-16T09:34:45.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:45 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:34:45.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:45 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:34:45.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:45 smithi070 ceph-mon[30388]: pgmap v479: 178 pgs: 1 active+recovering+undersized+remapped, 177 active+clean; 6.1 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.7 KiB/s rd, 69 MiB/s wr, 1.11k op/s; 2095/300834 objects misplaced (0.696%) 2024-09-16T09:34:45.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:45 smithi070 ceph-mon[30388]: 9.16 scrub ok 2024-09-16T09:34:45.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:45 smithi070 ceph-mon[30388]: 9.13 scrub ok 2024-09-16T09:34:45.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:45 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:34:45.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:45 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:34:45.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:45 smithi070 ceph-mon[40691]: pgmap v479: 178 pgs: 1 active+recovering+undersized+remapped, 177 active+clean; 6.1 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.7 KiB/s rd, 69 MiB/s wr, 1.11k op/s; 2095/300834 objects misplaced (0.696%) 2024-09-16T09:34:45.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:45 smithi070 ceph-mon[40691]: 9.16 scrub ok 2024-09-16T09:34:45.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:45 smithi070 ceph-mon[40691]: 9.13 scrub ok 2024-09-16T09:34:45.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:45 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:34:45.923 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:45 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:34:45.965 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:34:46.000 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:46.001 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:46.001 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:34:46.102 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:34:46.143 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:46.144 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:46.144 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:34:46.245 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:34:46.286 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:46.287 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:46.287 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:34:46.388 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:34:46.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:46.428 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:46.428 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:34:46.529 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:34:46.565 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:46.565 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:46.566 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:34:46.599 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T09:34:46.667 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:34:46.704 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:46.705 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:46.705 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:34:46.716 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 76 16 104926 104910 86.2619 88.625 0.0149874 0.0115883 2024-09-16T09:34:46.806 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:34:46.837 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:46.837 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:46.838 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:34:46.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:46 smithi084 ceph-mon[32629]: osdmap e274: 8 total, 8 up, 8 in 2024-09-16T09:34:46.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:46 smithi084 ceph-mon[32629]: pgmap v481: 178 pgs: 1 active+recovering+undersized+remapped, 177 active+clean; 6.1 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 853 B/s rd, 31 MiB/s wr, 497 op/s; 2095/300834 objects misplaced (0.696%) 2024-09-16T09:34:46.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:46 smithi084 ceph-mon[32629]: 5.2 scrub starts 2024-09-16T09:34:46.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:46 smithi084 ceph-mon[32629]: 5.2 scrub ok 2024-09-16T09:34:46.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:46 smithi070 ceph-mon[30388]: osdmap e274: 8 total, 8 up, 8 in 2024-09-16T09:34:46.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:46 smithi070 ceph-mon[30388]: pgmap v481: 178 pgs: 1 active+recovering+undersized+remapped, 177 active+clean; 6.1 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 853 B/s rd, 31 MiB/s wr, 497 op/s; 2095/300834 objects misplaced (0.696%) 2024-09-16T09:34:46.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:46 smithi070 ceph-mon[30388]: 5.2 scrub starts 2024-09-16T09:34:46.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:46 smithi070 ceph-mon[30388]: 5.2 scrub ok 2024-09-16T09:34:46.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:46 smithi070 ceph-mon[40691]: osdmap e274: 8 total, 8 up, 8 in 2024-09-16T09:34:46.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:46 smithi070 ceph-mon[40691]: pgmap v481: 178 pgs: 1 active+recovering+undersized+remapped, 177 active+clean; 6.1 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 853 B/s rd, 31 MiB/s wr, 497 op/s; 2095/300834 objects misplaced (0.696%) 2024-09-16T09:34:46.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:46 smithi070 ceph-mon[40691]: 5.2 scrub starts 2024-09-16T09:34:46.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:46 smithi070 ceph-mon[40691]: 5.2 scrub ok 2024-09-16T09:34:46.938 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:34:46.979 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:46.980 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:46.980 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:34:47.081 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:34:47.104 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:47.104 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:47.104 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:34:47.179 INFO:teuthology.orchestra.run.smithi070.stdout:moved 0 / 534 (0%) 2024-09-16T09:34:47.179 INFO:teuthology.orchestra.run.smithi070.stdout:avg 66.75 2024-09-16T09:34:47.179 INFO:teuthology.orchestra.run.smithi070.stdout:stddev 9.79477 -> 9.79477 (expected baseline 7.6424) 2024-09-16T09:34:47.179 INFO:teuthology.orchestra.run.smithi070.stdout:min osd.2 with 48 -> 48 pgs (0.719101 -> 0.719101 * mean) 2024-09-16T09:34:47.179 INFO:teuthology.orchestra.run.smithi070.stdout:max osd.7 with 79 -> 79 pgs (1.18352 -> 1.18352 * mean) 2024-09-16T09:34:47.179 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:34:47.179 INFO:teuthology.orchestra.run.smithi070.stdout:oload 1000 2024-09-16T09:34:47.179 INFO:teuthology.orchestra.run.smithi070.stdout:max_change 3 2024-09-16T09:34:47.179 INFO:teuthology.orchestra.run.smithi070.stdout:max_change_osds 4 2024-09-16T09:34:47.180 INFO:teuthology.orchestra.run.smithi070.stdout:average_utilization 0.0308 2024-09-16T09:34:47.180 INFO:teuthology.orchestra.run.smithi070.stdout:overload_utilization 0.3080 2024-09-16T09:34:47.180 INFO:teuthology.orchestra.run.smithi070.stderr:no change 2024-09-16T09:34:47.206 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:34:47.242 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:47.243 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:47.243 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:34:47.344 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:34:47.380 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:47.381 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:47.381 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:34:47.482 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:34:47.510 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:47.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:47.511 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:34:47.612 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:34:47.649 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:47.650 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:47.650 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:34:47.716 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 77 16 106184 106168 86.1626 78.625 0.0158511 0.0116006 2024-09-16T09:34:47.751 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:34:47.773 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:47.777 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:47.777 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:34:47.878 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:34:47.907 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:47.907 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:47.908 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:34:47.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:47 smithi084 ceph-mon[32629]: from='client.15684 -' entity='client.admin' cmd=[{"prefix": "osd test-reweight-by-utilization", "oload": 1000, "max_change": 3.0, "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:34:47.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:47 smithi084 ceph-mon[32629]: 4.4 scrub starts 2024-09-16T09:34:47.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:47 smithi084 ceph-mon[32629]: 4.4 scrub ok 2024-09-16T09:34:47.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:47 smithi070 ceph-mon[30388]: from='client.15684 -' entity='client.admin' cmd=[{"prefix": "osd test-reweight-by-utilization", "oload": 1000, "max_change": 3.0, "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:34:47.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:47 smithi070 ceph-mon[30388]: 4.4 scrub starts 2024-09-16T09:34:47.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:47 smithi070 ceph-mon[30388]: 4.4 scrub ok 2024-09-16T09:34:47.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:47 smithi070 ceph-mon[40691]: from='client.15684 -' entity='client.admin' cmd=[{"prefix": "osd test-reweight-by-utilization", "oload": 1000, "max_change": 3.0, "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T09:34:47.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:47 smithi070 ceph-mon[40691]: 4.4 scrub starts 2024-09-16T09:34:47.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:47 smithi070 ceph-mon[40691]: 4.4 scrub ok 2024-09-16T09:34:48.009 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:34:48.047 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:48.048 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:48.048 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:34:48.149 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:34:48.176 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:48.177 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:48.178 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:34:48.278 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:34:48.313 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:48.313 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:48.314 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:34:48.414 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:34:48.447 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:48.447 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:48.447 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:34:48.548 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:34:48.577 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:48.577 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:48.578 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:34:48.679 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:34:48.716 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 78 15 107424 107409 86.0523 77.5625 0.0347659 0.0116158 2024-09-16T09:34:48.720 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:48.721 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:48.721 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:34:48.822 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:34:48.848 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:48 smithi070 ceph-mon[30388]: 5.13 scrub starts 2024-09-16T09:34:48.849 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:48 smithi070 ceph-mon[30388]: 5.13 scrub ok 2024-09-16T09:34:48.849 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:48 smithi070 ceph-mon[30388]: 9.12 scrub starts 2024-09-16T09:34:48.849 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:48 smithi070 ceph-mon[30388]: pgmap v482: 178 pgs: 1 active+recovering+undersized+remapped, 177 active+clean; 6.1 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 705 B/s rd, 31 MiB/s wr, 491 op/s; 2095/302601 objects misplaced (0.692%) 2024-09-16T09:34:48.849 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:48 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:34:48.849 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:48 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:34:48.849 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:48 smithi070 ceph-mon[40691]: 5.13 scrub starts 2024-09-16T09:34:48.849 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:48 smithi070 ceph-mon[40691]: 5.13 scrub ok 2024-09-16T09:34:48.849 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:48 smithi070 ceph-mon[40691]: 9.12 scrub starts 2024-09-16T09:34:48.850 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:48 smithi070 ceph-mon[40691]: pgmap v482: 178 pgs: 1 active+recovering+undersized+remapped, 177 active+clean; 6.1 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 705 B/s rd, 31 MiB/s wr, 491 op/s; 2095/302601 objects misplaced (0.692%) 2024-09-16T09:34:48.850 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:48 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:34:48.850 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:48 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:34:48.867 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:48.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:48.868 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:34:48.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:48 smithi084 ceph-mon[32629]: 5.13 scrub starts 2024-09-16T09:34:48.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:48 smithi084 ceph-mon[32629]: 5.13 scrub ok 2024-09-16T09:34:48.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:48 smithi084 ceph-mon[32629]: 9.12 scrub starts 2024-09-16T09:34:48.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:48 smithi084 ceph-mon[32629]: pgmap v482: 178 pgs: 1 active+recovering+undersized+remapped, 177 active+clean; 6.1 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 705 B/s rd, 31 MiB/s wr, 491 op/s; 2095/302601 objects misplaced (0.692%) 2024-09-16T09:34:48.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:48 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:34:48.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:48 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:34:48.968 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:34:48.994 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:48.995 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:48.995 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:34:49.096 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:34:49.137 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:49.138 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:49.138 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:34:49.239 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:34:49.275 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:49.276 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:49.276 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:34:49.377 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:34:49.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:49.400 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:49.400 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:34:49.501 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:34:49.535 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:49.536 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:49.536 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:34:49.630 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:49 smithi070 ceph-mon[30388]: 1.13 scrub starts 2024-09-16T09:34:49.631 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:49 smithi070 ceph-mon[30388]: 1.13 scrub ok 2024-09-16T09:34:49.631 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:49 smithi070 ceph-mon[30388]: 9.12 scrub ok 2024-09-16T09:34:49.631 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:49 smithi070 ceph-mon[30388]: osdmap e275: 8 total, 8 up, 8 in 2024-09-16T09:34:49.631 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:49 smithi070 ceph-mon[40691]: 1.13 scrub starts 2024-09-16T09:34:49.631 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:49 smithi070 ceph-mon[40691]: 1.13 scrub ok 2024-09-16T09:34:49.631 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:49 smithi070 ceph-mon[40691]: 9.12 scrub ok 2024-09-16T09:34:49.631 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:49 smithi070 ceph-mon[40691]: osdmap e275: 8 total, 8 up, 8 in 2024-09-16T09:34:49.637 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:34:49.661 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:49.662 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:49.662 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:34:49.716 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 79 16 108437 108421 85.7636 63.25 0.00378512 0.0116442 2024-09-16T09:34:49.763 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:34:49.796 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:49.797 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:49.797 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:34:49.898 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:34:49.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:49 smithi084 ceph-mon[32629]: 1.13 scrub starts 2024-09-16T09:34:49.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:49 smithi084 ceph-mon[32629]: 1.13 scrub ok 2024-09-16T09:34:49.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:49 smithi084 ceph-mon[32629]: 9.12 scrub ok 2024-09-16T09:34:49.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:49 smithi084 ceph-mon[32629]: osdmap e275: 8 total, 8 up, 8 in 2024-09-16T09:34:49.930 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:49.931 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:49.931 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:34:50.032 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:34:50.065 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:50.065 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:50.065 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:34:50.166 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:34:50.205 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:50.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:50.206 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:34:50.307 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:34:50.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:50.342 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:50.342 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:34:50.446 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:34:50.476 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:50.477 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:50.477 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:34:50.579 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:34:50.620 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:50.621 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:50.621 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:34:50.717 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:2024-09-16T09:34:50.716157+0000 min lat: 0.00127131 max lat: 2.16104 avg lat: 0.0116702 2024-09-16T09:34:50.717 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-16T09:34:50.717 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 80 16 109664 109648 85.6501 76.6875 0.0089271 0.0116702 2024-09-16T09:34:50.721 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:34:50.742 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:50.743 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:50.743 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:34:50.844 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:34:50.876 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:50.877 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:50.877 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:34:50.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:50 smithi084 ceph-mon[32629]: 6.11 scrub starts 2024-09-16T09:34:50.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:50 smithi084 ceph-mon[32629]: 6.11 scrub ok 2024-09-16T09:34:50.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:50 smithi084 ceph-mon[32629]: pgmap v484: 178 pgs: 1 active+clean+scrubbing, 1 active+recovering+undersized+remapped, 176 active+clean; 6.5 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.2 KiB/s rd, 23 MiB/s wr, 372 op/s; 1879/319896 objects misplaced (0.587%) 2024-09-16T09:34:50.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:50 smithi084 ceph-mon[32629]: 9.18 scrub starts 2024-09-16T09:34:50.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:50 smithi070 ceph-mon[30388]: 6.11 scrub starts 2024-09-16T09:34:50.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:50 smithi070 ceph-mon[30388]: 6.11 scrub ok 2024-09-16T09:34:50.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:50 smithi070 ceph-mon[30388]: pgmap v484: 178 pgs: 1 active+clean+scrubbing, 1 active+recovering+undersized+remapped, 176 active+clean; 6.5 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.2 KiB/s rd, 23 MiB/s wr, 372 op/s; 1879/319896 objects misplaced (0.587%) 2024-09-16T09:34:50.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:50 smithi070 ceph-mon[30388]: 9.18 scrub starts 2024-09-16T09:34:50.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:50 smithi070 ceph-mon[40691]: 6.11 scrub starts 2024-09-16T09:34:50.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:50 smithi070 ceph-mon[40691]: 6.11 scrub ok 2024-09-16T09:34:50.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:50 smithi070 ceph-mon[40691]: pgmap v484: 178 pgs: 1 active+clean+scrubbing, 1 active+recovering+undersized+remapped, 176 active+clean; 6.5 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.2 KiB/s rd, 23 MiB/s wr, 372 op/s; 1879/319896 objects misplaced (0.587%) 2024-09-16T09:34:50.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:50 smithi070 ceph-mon[40691]: 9.18 scrub starts 2024-09-16T09:34:50.979 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:34:51.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:51.008 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:51.009 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:34:51.109 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:34:51.147 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:51.147 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:51.147 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:34:51.248 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:34:51.288 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:51.289 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:51.289 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:34:51.390 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:34:51.418 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:51.419 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:51.420 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:34:51.521 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:34:51.564 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:51.565 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:51.565 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:34:51.666 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:34:51.704 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:51.704 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:51.705 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:34:51.716 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 81 16 111017 111001 85.6365 84.5625 0.0121375 0.011673 2024-09-16T09:34:51.806 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:34:51.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:51.847 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:51.847 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:34:51.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:51 smithi084 ceph-mon[32629]: 4.10 scrub starts 2024-09-16T09:34:51.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:51 smithi084 ceph-mon[32629]: 4.10 scrub ok 2024-09-16T09:34:51.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:51 smithi084 ceph-mon[32629]: 9.18 scrub ok 2024-09-16T09:34:51.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:51 smithi084 ceph-mon[32629]: 6.c scrub starts 2024-09-16T09:34:51.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:51 smithi084 ceph-mon[32629]: 6.c scrub ok 2024-09-16T09:34:51.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:51 smithi070 ceph-mon[30388]: 4.10 scrub starts 2024-09-16T09:34:51.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:51 smithi070 ceph-mon[30388]: 4.10 scrub ok 2024-09-16T09:34:51.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:51 smithi070 ceph-mon[30388]: 9.18 scrub ok 2024-09-16T09:34:51.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:51 smithi070 ceph-mon[30388]: 6.c scrub starts 2024-09-16T09:34:51.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:51 smithi070 ceph-mon[30388]: 6.c scrub ok 2024-09-16T09:34:51.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:51 smithi070 ceph-mon[40691]: 4.10 scrub starts 2024-09-16T09:34:51.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:51 smithi070 ceph-mon[40691]: 4.10 scrub ok 2024-09-16T09:34:51.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:51 smithi070 ceph-mon[40691]: 9.18 scrub ok 2024-09-16T09:34:51.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:51 smithi070 ceph-mon[40691]: 6.c scrub starts 2024-09-16T09:34:51.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:51 smithi070 ceph-mon[40691]: 6.c scrub ok 2024-09-16T09:34:51.948 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:34:51.986 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:51.987 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:51.987 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:34:52.088 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:34:52.116 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:52.117 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:52.117 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:34:52.218 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:34:52.256 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:52.257 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:52.257 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:34:52.358 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:34:52.388 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:52.389 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:52.389 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:34:52.421 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:34:51 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:34:51] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:34:52.492 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:34:52.521 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:52.521 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:52.522 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:34:52.622 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:34:52.664 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:52.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:52.665 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:34:52.717 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 82 16 112380 112364 85.631 85.1875 0.0129719 0.0116728 2024-09-16T09:34:52.766 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:34:52.806 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:52.806 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:52.807 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:34:52.908 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:34:52.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:52 smithi084 ceph-mon[32629]: 6.10 scrub starts 2024-09-16T09:34:52.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:52 smithi084 ceph-mon[32629]: 6.10 scrub ok 2024-09-16T09:34:52.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:52 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:34:52.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:52 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:34:52.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:52 smithi084 ceph-mon[32629]: 4.17 scrub starts 2024-09-16T09:34:52.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:52 smithi084 ceph-mon[32629]: 4.17 scrub ok 2024-09-16T09:34:52.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:52 smithi084 ceph-mon[32629]: osdmap e276: 8 total, 8 up, 8 in 2024-09-16T09:34:52.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:52 smithi084 ceph-mon[32629]: pgmap v486: 178 pgs: 1 active+clean+scrubbing, 1 active+recovering+undersized+remapped, 176 active+clean; 6.5 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 833 B/s rd, 0 op/s; 1879/319896 objects misplaced (0.587%) 2024-09-16T09:34:52.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:52 smithi084 ceph-mon[32629]: 6.13 scrub starts 2024-09-16T09:34:52.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:52 smithi084 ceph-mon[32629]: 6.13 scrub ok 2024-09-16T09:34:52.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:52 smithi084 ceph-mon[32629]: 1.7 scrub starts 2024-09-16T09:34:52.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:52 smithi084 ceph-mon[32629]: 1.7 scrub ok 2024-09-16T09:34:52.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:52 smithi070 ceph-mon[30388]: 6.10 scrub starts 2024-09-16T09:34:52.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:52 smithi070 ceph-mon[30388]: 6.10 scrub ok 2024-09-16T09:34:52.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:52 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:34:52.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:52 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:34:52.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:52 smithi070 ceph-mon[30388]: 4.17 scrub starts 2024-09-16T09:34:52.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:52 smithi070 ceph-mon[30388]: 4.17 scrub ok 2024-09-16T09:34:52.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:52 smithi070 ceph-mon[30388]: osdmap e276: 8 total, 8 up, 8 in 2024-09-16T09:34:52.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:52 smithi070 ceph-mon[30388]: pgmap v486: 178 pgs: 1 active+clean+scrubbing, 1 active+recovering+undersized+remapped, 176 active+clean; 6.5 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 833 B/s rd, 0 op/s; 1879/319896 objects misplaced (0.587%) 2024-09-16T09:34:52.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:52 smithi070 ceph-mon[30388]: 6.13 scrub starts 2024-09-16T09:34:52.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:52 smithi070 ceph-mon[30388]: 6.13 scrub ok 2024-09-16T09:34:52.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:52 smithi070 ceph-mon[30388]: 1.7 scrub starts 2024-09-16T09:34:52.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:52 smithi070 ceph-mon[30388]: 1.7 scrub ok 2024-09-16T09:34:52.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:52 smithi070 ceph-mon[40691]: 6.10 scrub starts 2024-09-16T09:34:52.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:52 smithi070 ceph-mon[40691]: 6.10 scrub ok 2024-09-16T09:34:52.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:52 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:34:52.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:52 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:34:52.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:52 smithi070 ceph-mon[40691]: 4.17 scrub starts 2024-09-16T09:34:52.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:52 smithi070 ceph-mon[40691]: 4.17 scrub ok 2024-09-16T09:34:52.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:52 smithi070 ceph-mon[40691]: osdmap e276: 8 total, 8 up, 8 in 2024-09-16T09:34:52.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:52 smithi070 ceph-mon[40691]: pgmap v486: 178 pgs: 1 active+clean+scrubbing, 1 active+recovering+undersized+remapped, 176 active+clean; 6.5 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 833 B/s rd, 0 op/s; 1879/319896 objects misplaced (0.587%) 2024-09-16T09:34:52.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:52 smithi070 ceph-mon[40691]: 6.13 scrub starts 2024-09-16T09:34:52.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:52 smithi070 ceph-mon[40691]: 6.13 scrub ok 2024-09-16T09:34:52.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:52 smithi070 ceph-mon[40691]: 1.7 scrub starts 2024-09-16T09:34:52.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:52 smithi070 ceph-mon[40691]: 1.7 scrub ok 2024-09-16T09:34:52.945 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:52.945 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:52.946 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:34:53.046 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:34:53.079 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:53.080 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:53.081 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:34:53.139 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.4 dump_ops_in_flight 2024-09-16T09:34:53.182 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:34:53.227 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:53.228 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:53.229 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:34:53.330 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:34:53.374 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:53.375 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:53.375 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:34:53.410 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:34:53 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:34:53.476 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:34:53.509 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:53.510 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:53.510 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:34:53.610 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:34:53.647 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:53.648 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:53.648 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:34:53.717 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 83 16 113582 113566 85.5043 75.125 0.0227699 0.0116908 2024-09-16T09:34:53.749 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:34:53.759 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:34:53.768 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:34:53.810 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:53.810 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:53.811 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:34:53.844 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:34:53.897 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:53 smithi070 ceph-mon[30388]: 9.17 scrub starts 2024-09-16T09:34:53.897 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:53 smithi070 ceph-mon[30388]: 5.1f scrub starts 2024-09-16T09:34:53.897 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:53 smithi070 ceph-mon[30388]: 5.1f scrub ok 2024-09-16T09:34:53.897 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:53 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:34:53.897 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:53 smithi070 ceph-mon[40691]: 9.17 scrub starts 2024-09-16T09:34:53.897 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:53 smithi070 ceph-mon[40691]: 5.1f scrub starts 2024-09-16T09:34:53.897 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:53 smithi070 ceph-mon[40691]: 5.1f scrub ok 2024-09-16T09:34:53.897 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:53 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:34:53.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:53 smithi084 ceph-mon[32629]: 9.17 scrub starts 2024-09-16T09:34:53.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:53 smithi084 ceph-mon[32629]: 5.1f scrub starts 2024-09-16T09:34:53.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:53 smithi084 ceph-mon[32629]: 5.1f scrub ok 2024-09-16T09:34:53.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:53 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:34:53.912 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:34:53.941 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:53.942 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:53.942 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:34:54.043 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:34:54.082 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:54.083 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:54.083 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:34:54.184 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:34:54.227 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:54.227 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:54.228 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:34:54.329 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:34:54.373 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:54.374 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:54.374 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:34:54.475 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:34:54.496 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:54.496 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:54.497 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:34:54.597 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:34:54.619 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:54.620 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:54.620 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:34:54.717 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 84 16 114592 114576 85.2379 63.125 0.00360735 0.0116963 2024-09-16T09:34:54.721 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:34:54.744 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:54.745 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:54.745 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:34:54.805 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:54 smithi070 ceph-mon[30388]: 9.17 scrub ok 2024-09-16T09:34:54.806 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:54 smithi070 ceph-mon[30388]: pgmap v487: 178 pgs: 1 active+clean+scrubbing, 1 active+recovering+undersized+remapped, 176 active+clean; 6.9 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 1583/338577 objects misplaced (0.468%) 2024-09-16T09:34:54.806 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:54 smithi070 ceph-mon[30388]: 1.f scrub starts 2024-09-16T09:34:54.806 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:54 smithi070 ceph-mon[30388]: 1.f scrub ok 2024-09-16T09:34:54.806 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:54 smithi070 ceph-mon[30388]: osdmap e277: 8 total, 8 up, 8 in 2024-09-16T09:34:54.806 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:54 smithi070 ceph-mon[40691]: 9.17 scrub ok 2024-09-16T09:34:54.806 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:54 smithi070 ceph-mon[40691]: pgmap v487: 178 pgs: 1 active+clean+scrubbing, 1 active+recovering+undersized+remapped, 176 active+clean; 6.9 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 1583/338577 objects misplaced (0.468%) 2024-09-16T09:34:54.806 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:54 smithi070 ceph-mon[40691]: 1.f scrub starts 2024-09-16T09:34:54.806 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:54 smithi070 ceph-mon[40691]: 1.f scrub ok 2024-09-16T09:34:54.806 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:54 smithi070 ceph-mon[40691]: osdmap e277: 8 total, 8 up, 8 in 2024-09-16T09:34:54.811 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:34:54.811 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:34:54.824 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:34:54.824 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:34:54.841 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:34:54.842 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:34:54.846 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:34:54.866 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:34:54.866 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:34:54.874 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:54.875 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:54.875 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:34:54.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:54 smithi084 ceph-mon[32629]: 9.17 scrub ok 2024-09-16T09:34:54.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:54 smithi084 ceph-mon[32629]: pgmap v487: 178 pgs: 1 active+clean+scrubbing, 1 active+recovering+undersized+remapped, 176 active+clean; 6.9 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 1583/338577 objects misplaced (0.468%) 2024-09-16T09:34:54.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:54 smithi084 ceph-mon[32629]: 1.f scrub starts 2024-09-16T09:34:54.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:54 smithi084 ceph-mon[32629]: 1.f scrub ok 2024-09-16T09:34:54.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:54 smithi084 ceph-mon[32629]: osdmap e277: 8 total, 8 up, 8 in 2024-09-16T09:34:54.957 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:34:54.957 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:34:54.977 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:34:54.987 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:34:54.987 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:34:55.011 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:34:55.011 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:34:55.012 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:55.013 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:55.013 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:34:55.048 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:34:55.048 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:34:55.113 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:34:55.152 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:55.152 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:55.153 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:34:55.253 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:34:55.279 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:55.280 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:55.280 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:34:55.381 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:34:55.402 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:55.403 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:55.403 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:34:55.504 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:34:55.531 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:55.532 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:55.532 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:34:55.606 INFO:teuthology.orchestra.run.smithi070.stderr:noscrub is set 2024-09-16T09:34:55.633 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:34:55.667 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:55.668 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:55.668 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:34:55.717 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 85 15 115419 115404 84.8438 51.75 0.0322588 0.0117821 2024-09-16T09:34:55.741 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-09-16T09:34:55.741 INFO:teuthology.orchestra.run.smithi084.stdout: "ops": [ 2024-09-16T09:34:55.741 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:34:55.741 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:114578 9.10 9:09286dbd:::benchmark_data_smithi070_137000_object114577:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e277)", 2024-09-16T09:34:55.741 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:34:54.457514+0000", 2024-09-16T09:34:55.742 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 0.085397814000000002, 2024-09-16T09:34:55.742 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.085428678999999993, 2024-09-16T09:34:55.742 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:34:55.742 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:34:55.742 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "waiting for sub ops", 2024-09-16T09:34:55.742 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:34:55.742 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:34:55.742 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:34:55.742 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 114578 2024-09-16T09:34:55.742 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:34:55.743 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:34:55.743 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:34:55.743 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:34:55.743 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:54.457514+0000", 2024-09-16T09:34:55.743 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:34:55.743 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:34:55.743 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:34:55.743 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:34:55.743 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:54.457514+0000", 2024-09-16T09:34:55.743 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:34:55.743 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:34:55.743 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:34:55.743 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:34:55.743 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:54.457525+0000", 2024-09-16T09:34:55.744 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0434999999999999e-05 2024-09-16T09:34:55.744 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:34:55.744 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:34:55.744 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:34:55.744 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:54.457590+0000", 2024-09-16T09:34:55.744 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.4814000000000007e-05 2024-09-16T09:34:55.744 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:34:55.744 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:34:55.744 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:34:55.744 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:54.457605+0000", 2024-09-16T09:34:55.744 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.5029e-05 2024-09-16T09:34:55.744 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:34:55.744 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:34:55.745 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:34:55.745 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:54.457613+0000", 2024-09-16T09:34:55.745 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.4230000000000005e-06 2024-09-16T09:34:55.745 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:34:55.745 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:34:55.745 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:34:55.745 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:54.457677+0000", 2024-09-16T09:34:55.745 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.3851999999999998e-05 2024-09-16T09:34:55.745 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:34:55.745 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:34:55.745 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:34:55.745 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:54.457813+0000", 2024-09-16T09:34:55.745 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00013647799999999999 2024-09-16T09:34:55.746 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:34:55.746 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:34:55.746 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,7", 2024-09-16T09:34:55.746 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:54.458159+0000", 2024-09-16T09:34:55.746 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00034563600000000002 2024-09-16T09:34:55.746 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:34:55.746 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:34:55.746 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:34:55.746 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:54.461186+0000", 2024-09-16T09:34:55.746 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0030270639999999999 2024-09-16T09:34:55.746 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:34:55.746 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:34:55.746 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:34:55.746 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:54.464073+0000", 2024-09-16T09:34:55.747 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0028871249999999999 2024-09-16T09:34:55.747 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:34:55.747 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:34:55.747 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:34:55.747 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:34:55.747 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-09-16T09:34:55.747 INFO:teuthology.orchestra.run.smithi084.stdout: "num_ops": 1 2024-09-16T09:34:55.747 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-09-16T09:34:55.768 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:34:55.789 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:55.790 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:55.790 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:34:55.844 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:55 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3305125209' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:34:55.844 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:55 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1077144386' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:34:55.844 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:55 smithi070 ceph-mon[30388]: 9.14 scrub starts 2024-09-16T09:34:55.844 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:55 smithi070 ceph-mon[30388]: 9.6 scrub starts 2024-09-16T09:34:55.844 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:55 smithi070 ceph-mon[30388]: 9.1 scrub starts 2024-09-16T09:34:55.844 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:55 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3305125209' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:34:55.844 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:55 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1077144386' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:34:55.845 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:55 smithi070 ceph-mon[40691]: 9.14 scrub starts 2024-09-16T09:34:55.845 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:55 smithi070 ceph-mon[40691]: 9.6 scrub starts 2024-09-16T09:34:55.845 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:55 smithi070 ceph-mon[40691]: 9.1 scrub starts 2024-09-16T09:34:55.891 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:34:55.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:55 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3305125209' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:34:55.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:55 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1077144386' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:34:55.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:55 smithi084 ceph-mon[32629]: 9.14 scrub starts 2024-09-16T09:34:55.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:55 smithi084 ceph-mon[32629]: 9.6 scrub starts 2024-09-16T09:34:55.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:55 smithi084 ceph-mon[32629]: 9.1 scrub starts 2024-09-16T09:34:55.931 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:55.932 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:55.932 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:34:56.033 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:34:56.071 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:56.072 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:56.072 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:34:56.173 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:34:56.220 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:56.221 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:56.221 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:34:56.322 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:34:56.356 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:56.356 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:56.357 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:34:56.458 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:34:56.494 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:56.495 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:56.496 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:34:56.597 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:34:56.635 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:56.635 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:56.635 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:34:56.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:56 smithi070 ceph-mon[30388]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:34:56.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:56 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:34:56.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:56 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:34:56.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:56 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3305125209' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-16T09:34:56.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:56 smithi070 ceph-mon[30388]: osdmap e278: 8 total, 8 up, 8 in 2024-09-16T09:34:56.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:56 smithi070 ceph-mon[30388]: 9.11 scrub starts 2024-09-16T09:34:56.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:56 smithi070 ceph-mon[30388]: pgmap v490: 178 pgs: 1 active+clean+scrubbing, 1 active+recovering+undersized+remapped, 176 active+clean; 6.9 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 1583/338577 objects misplaced (0.468%) 2024-09-16T09:34:56.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:56 smithi070 ceph-mon[30388]: 9.9 scrub starts 2024-09-16T09:34:56.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:56 smithi070 ceph-mon[30388]: 9.0 scrub starts 2024-09-16T09:34:56.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:56 smithi070 ceph-mon[40691]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:34:56.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:56 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:34:56.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:56 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:34:56.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:56 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3305125209' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-16T09:34:56.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:56 smithi070 ceph-mon[40691]: osdmap e278: 8 total, 8 up, 8 in 2024-09-16T09:34:56.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:56 smithi070 ceph-mon[40691]: 9.11 scrub starts 2024-09-16T09:34:56.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:56 smithi070 ceph-mon[40691]: pgmap v490: 178 pgs: 1 active+clean+scrubbing, 1 active+recovering+undersized+remapped, 176 active+clean; 6.9 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 1583/338577 objects misplaced (0.468%) 2024-09-16T09:34:56.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:56 smithi070 ceph-mon[40691]: 9.9 scrub starts 2024-09-16T09:34:56.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:56 smithi070 ceph-mon[40691]: 9.0 scrub starts 2024-09-16T09:34:56.717 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 86 15 116651 116636 84.7526 77 0.00911154 0.0117947 2024-09-16T09:34:56.736 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:34:56.769 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:56.769 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:56.770 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:34:56.870 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:34:56.904 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:56.904 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:56.905 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:34:56.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:56 smithi084 ceph-mon[32629]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:34:56.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:56 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:34:56.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:56 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:34:56.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:56 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3305125209' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-16T09:34:56.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:56 smithi084 ceph-mon[32629]: osdmap e278: 8 total, 8 up, 8 in 2024-09-16T09:34:56.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:56 smithi084 ceph-mon[32629]: 9.11 scrub starts 2024-09-16T09:34:56.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:56 smithi084 ceph-mon[32629]: pgmap v490: 178 pgs: 1 active+clean+scrubbing, 1 active+recovering+undersized+remapped, 176 active+clean; 6.9 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 1583/338577 objects misplaced (0.468%) 2024-09-16T09:34:56.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:56 smithi084 ceph-mon[32629]: 9.9 scrub starts 2024-09-16T09:34:56.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:56 smithi084 ceph-mon[32629]: 9.0 scrub starts 2024-09-16T09:34:57.005 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:34:57.037 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:57.038 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:57.038 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:34:57.132 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.4 dump_blocked_ops 2024-09-16T09:34:57.139 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:34:57.166 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:57.167 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:57.167 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:34:57.268 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:34:57.311 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:57.312 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:57.312 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:34:57.413 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:34:57.440 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:57.440 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:57.441 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:34:57.542 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:34:57.613 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:57.614 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:57.615 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:34:57.716 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:34:57.717 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 87 16 118003 117987 84.7488 84.4375 0.00316932 0.0117929 2024-09-16T09:34:57.738 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:57.738 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:57.739 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:34:57.777 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:34:57.839 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:34:57.868 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:57.869 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:57.869 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:34:57.894 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T09:34:57.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:57 smithi084 ceph-mon[32629]: 9.15 scrub starts 2024-09-16T09:34:57.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:57 smithi084 ceph-mon[32629]: osdmap e279: 8 total, 8 up, 8 in 2024-09-16T09:34:57.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:57 smithi070 ceph-mon[30388]: 9.15 scrub starts 2024-09-16T09:34:57.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:57 smithi070 ceph-mon[30388]: osdmap e279: 8 total, 8 up, 8 in 2024-09-16T09:34:57.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:57 smithi070 ceph-mon[40691]: 9.15 scrub starts 2024-09-16T09:34:57.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:57 smithi070 ceph-mon[40691]: osdmap e279: 8 total, 8 up, 8 in 2024-09-16T09:34:57.970 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:34:58.006 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:58.007 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:58.007 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:34:58.108 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:34:58.130 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:58.130 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:58.130 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:34:58.231 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:34:58.267 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:58.268 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:58.268 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:34:58.274 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:34:58.369 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:34:58.409 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:58.409 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:58.410 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:34:58.511 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:34:58.538 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:58.539 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:58.539 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:34:58.640 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:34:58.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:58 smithi070 ceph-mon[30388]: pgmap v492: 178 pgs: 1 active+clean+scrubbing, 1 active+recovering+undersized+remapped, 176 active+clean; 6.9 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 853 B/s rd, 533 KiB/s wr, 9 op/s; 1583/340248 objects misplaced (0.465%) 2024-09-16T09:34:58.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:58 smithi070 ceph-mon[40691]: pgmap v492: 178 pgs: 1 active+clean+scrubbing, 1 active+recovering+undersized+remapped, 176 active+clean; 6.9 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 853 B/s rd, 533 KiB/s wr, 9 op/s; 1583/340248 objects misplaced (0.465%) 2024-09-16T09:34:58.679 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:58.680 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:58.680 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:34:58.717 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 88 16 119295 119279 84.703 80.75 0.00312742 0.0118 2024-09-16T09:34:58.752 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T09:34:58.752 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T09:34:58.752 INFO:tasks.thrashosds.thrasher:Shrinking pool unique_pool_2 2024-09-16T09:34:58.752 INFO:tasks.cephadm.ceph_manager.ceph:contract_pool unique_pool_2 by 10 min 8 2024-09-16T09:34:58.753 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-16T09:34:58.780 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:34:58.839 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:58.839 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:58.840 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:34:58.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:58 smithi084 ceph-mon[32629]: pgmap v492: 178 pgs: 1 active+clean+scrubbing, 1 active+recovering+undersized+remapped, 176 active+clean; 6.9 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 853 B/s rd, 533 KiB/s wr, 9 op/s; 1583/340248 objects misplaced (0.465%) 2024-09-16T09:34:58.941 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:34:58.980 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:58.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:58.981 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:34:58.997 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:34:58.997 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:34:59.032 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:34:59.032 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:34:59.036 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-16T09:34:59.063 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:34:59.064 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:34:59.082 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:34:59.097 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:34:59.097 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:34:59.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:59.138 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:59.138 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:34:59.140 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:34:59.140 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:34:59.140 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:34:59.173 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:34:59.173 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:34:59.200 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:34:59.200 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:34:59.234 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:34:59.235 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:34:59.239 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:34:59.283 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:59.283 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:59.284 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:34:59.385 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:34:59.429 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:59.429 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:59.430 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:34:59.505 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-09-16T09:34:59.505 INFO:teuthology.orchestra.run.smithi084.stdout: "ops": [], 2024-09-16T09:34:59.505 INFO:teuthology.orchestra.run.smithi084.stdout: "complaint_time": 30, 2024-09-16T09:34:59.505 INFO:teuthology.orchestra.run.smithi084.stdout: "num_blocked_ops": 0 2024-09-16T09:34:59.505 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-09-16T09:34:59.531 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:34:59.553 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:59.554 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:59.554 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:34:59.656 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:34:59.693 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:59.694 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:34:59.694 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:34:59.718 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 89 16 120330 120314 84.4781 64.6875 0.00850411 0.011833 2024-09-16T09:34:59.795 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:34:59.839 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:59.839 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:59.840 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:34:59.882 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:59 smithi070 ceph-mon[30388]: 9.10 scrub starts 2024-09-16T09:34:59.882 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:34:59 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3240312781' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:34:59.883 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:59 smithi070 ceph-mon[40691]: 9.10 scrub starts 2024-09-16T09:34:59.883 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:34:59 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3240312781' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:34:59.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:59 smithi084 ceph-mon[32629]: 9.10 scrub starts 2024-09-16T09:34:59.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:34:59 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3240312781' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:34:59.941 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:34:59.975 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:34:59.975 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:34:59.976 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:35:00.077 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:35:00.119 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:00.120 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:00.120 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:35:00.221 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:35:00.258 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:00.259 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:00.259 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:35:00.360 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:35:00.400 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:00.401 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:00.401 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:35:00.502 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:35:00.532 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:00.532 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:00.533 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:35:00.634 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:35:00.667 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:00.668 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:00.668 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:35:00.700 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.4 dump_historic_ops 2024-09-16T09:35:00.718 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Total time run: 90.0098 2024-09-16T09:35:00.718 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Total writes made: 121708 2024-09-16T09:35:00.718 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Write size: 65536 2024-09-16T09:35:00.718 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Object size: 65536 2024-09-16T09:35:00.718 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Bandwidth (MB/sec): 84.5102 2024-09-16T09:35:00.718 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Stddev Bandwidth: 44.3611 2024-09-16T09:35:00.718 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Max bandwidth (MB/sec): 236.562 2024-09-16T09:35:00.718 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Min bandwidth (MB/sec): 0 2024-09-16T09:35:00.718 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Average IOPS: 1352 2024-09-16T09:35:00.718 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Stddev IOPS: 709.778 2024-09-16T09:35:00.718 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Max IOPS: 3785 2024-09-16T09:35:00.718 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Min IOPS: 0 2024-09-16T09:35:00.719 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Average Latency(s): 0.0118289 2024-09-16T09:35:00.719 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Stddev Latency(s): 0.0428032 2024-09-16T09:35:00.719 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Max latency(s): 2.16104 2024-09-16T09:35:00.719 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Min latency(s): 0.00127131 2024-09-16T09:35:00.724 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Cleaning up (deleting benchmark objects) 2024-09-16T09:35:00.769 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:35:00.798 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:00.798 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:00.799 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:35:00.899 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:35:00.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:00 smithi084 ceph-mon[32629]: pgmap v493: 178 pgs: 1 active+recovering+undersized+remapped, 177 active+clean; 7.2 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 853 B/s rd, 21 MiB/s wr, 336 op/s; 1325/355410 objects misplaced (0.373%) 2024-09-16T09:35:00.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:00 smithi084 ceph-mon[32629]: from='client.25649 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:35:00.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:00 smithi070 ceph-mon[30388]: pgmap v493: 178 pgs: 1 active+recovering+undersized+remapped, 177 active+clean; 7.2 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 853 B/s rd, 21 MiB/s wr, 336 op/s; 1325/355410 objects misplaced (0.373%) 2024-09-16T09:35:00.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:00 smithi070 ceph-mon[30388]: from='client.25649 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:35:00.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:00 smithi070 ceph-mon[40691]: pgmap v493: 178 pgs: 1 active+recovering+undersized+remapped, 177 active+clean; 7.2 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 853 B/s rd, 21 MiB/s wr, 336 op/s; 1325/355410 objects misplaced (0.373%) 2024-09-16T09:35:00.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:00 smithi070 ceph-mon[40691]: from='client.25649 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:35:00.947 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:00.948 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:00.948 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:35:01.007 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:35:01.010 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2024-09-16T09:35:01.048 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:35:01.076 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:01.077 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:01.077 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:35:01.130 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:35:01.177 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:35:01.222 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:01.223 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:01.224 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:35:01.325 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:35:01.353 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:01.353 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:01.354 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:35:01.455 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:35:01.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:01.486 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:01.486 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:35:01.587 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:35:01.621 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:01.622 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:01.623 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:35:01.637 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:01 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:35:01.723 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:35:01.749 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:01.750 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:01.750 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:35:01.813 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":493,"stamp":"2024-09-16T09:34:59.686882+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":7751571995,"num_objects":118470,"num_object_clones":0,"num_object_copies":355410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1325,"num_objects_unfound":0,"num_objects_dirty":118470,"num_whiteouts":0,"num_read":1302,"num_read_kb":1054,"num_write":87193,"num_write_kb":5552461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5652,"num_bytes_recovered":362812198,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":132961,"ondisk_log_size":132961,"up":534,"acting":533,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":545,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":26498592,"kb_used_data":23357244,"kb_used_omap":12,"kb_used_meta":3141171,"kb_avail":723462624,"statfs":{"total":767960285184,"available":740825726976,"internally_reserved":0,"allocated":23917817856,"data_stored":23912915801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":3216559666},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,3,1,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":367755264,"num_objects":5611,"num_object_clones":0,"num_object_copies":16833,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-258,"num_objects_unfound":0,"num_objects_dirty":5611,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2012,"num_write_kb":128800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-1363,"num_bytes_recovered":-89325568,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001803"},"pg_stats":[{"pgid":"9.15","version":"279'5261","reported_seq":10363,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:57.792025+0000","last_change":"2024-09-16T09:34:56.601504+0000","last_active":"2024-09-16T09:34:57.792025+0000","last_peered":"2024-09-16T09:34:57.792025+0000","last_clean":"2024-09-16T09:34:57.792025+0000","last_became_active":"2024-09-16T09:34:25.276458+0000","last_became_peered":"2024-09-16T09:34:25.276458+0000","last_unstale":"2024-09-16T09:34:57.792025+0000","last_undegraded":"2024-09-16T09:34:57.792025+0000","last_fullsized":"2024-09-16T09:34:57.792025+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:33:24.230486+0000","objects_scrubbed":0,"log_size":5261,"log_dups_size":0,"ondisk_log_size":5261,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:35:26.591180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":241631232,"num_objects":3687,"num_object_clones":0,"num_object_copies":11061,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3687,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3687,"num_write_kb":235968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1655,"num_bytes_recovered":108462080,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3],"acting":[7,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":488,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.615818+0000","last_change":"2024-09-16T09:32:39.384174+0000","last_active":"2024-09-16T09:34:56.615818+0000","last_peered":"2024-09-16T09:34:56.615818+0000","last_clean":"2024-09-16T09:34:56.615818+0000","last_became_active":"2024-09-16T09:25:01.709108+0000","last_became_peered":"2024-09-16T09:25:01.709108+0000","last_unstale":"2024-09-16T09:34:56.615818+0000","last_undegraded":"2024-09-16T09:34:56.615818+0000","last_fullsized":"2024-09-16T09:34:56.615818+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:39.384104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:32:39.384104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"66'9","reported_seq":492,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.632118+0000","last_change":"2024-09-16T09:32:38.074301+0000","last_active":"2024-09-16T09:34:56.632118+0000","last_peered":"2024-09-16T09:34:56.632118+0000","last_clean":"2024-09-16T09:34:56.632118+0000","last_became_active":"2024-09-16T09:25:03.714635+0000","last_became_peered":"2024-09-16T09:25:03.714635+0000","last_unstale":"2024-09-16T09:34:56.632118+0000","last_undegraded":"2024-09-16T09:34:56.632118+0000","last_fullsized":"2024-09-16T09:34:56.632118+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:32:38.074231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:32:38.074231+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":103,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.631816+0000","last_change":"2024-09-16T09:33:30.242561+0000","last_active":"2024-09-16T09:34:56.631816+0000","last_peered":"2024-09-16T09:34:56.631816+0000","last_clean":"2024-09-16T09:34:56.631816+0000","last_became_active":"2024-09-16T09:33:30.242238+0000","last_became_peered":"2024-09-16T09:33:30.242238+0000","last_unstale":"2024-09-16T09:34:56.631816+0000","last_undegraded":"2024-09-16T09:34:56.631816+0000","last_fullsized":"2024-09-16T09:34:56.631816+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:13.426830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:13.426830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":451,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.632882+0000","last_change":"2024-09-16T09:27:11.565306+0000","last_active":"2024-09-16T09:34:56.632882+0000","last_peered":"2024-09-16T09:34:56.632882+0000","last_clean":"2024-09-16T09:34:56.632882+0000","last_became_active":"2024-09-16T09:25:07.715442+0000","last_became_peered":"2024-09-16T09:25:07.715442+0000","last_unstale":"2024-09-16T09:34:56.632882+0000","last_undegraded":"2024-09-16T09:34:56.632882+0000","last_fullsized":"2024-09-16T09:34:56.632882+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:11.565220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:11.565220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"279'5344","reported_seq":5496,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:57.880649+0000","last_change":"2024-09-16T09:34:55.621991+0000","last_active":"2024-09-16T09:34:57.880649+0000","last_peered":"2024-09-16T09:34:57.880649+0000","last_clean":"2024-09-16T09:34:57.880649+0000","last_became_active":"2024-09-16T09:33:52.932390+0000","last_became_peered":"2024-09-16T09:33:52.932390+0000","last_unstale":"2024-09-16T09:34:57.880649+0000","last_undegraded":"2024-09-16T09:34:57.880649+0000","last_fullsized":"2024-09-16T09:34:57.880649+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:33:24.230486+0000","objects_scrubbed":2949,"log_size":5344,"log_dups_size":0,"ondisk_log_size":5344,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:35:25.616579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":245268480,"num_objects":3743,"num_object_clones":0,"num_object_copies":11229,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3743,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3743,"num_write_kb":239520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":205,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.626971+0000","last_change":"2024-09-16T09:34:33.452784+0000","last_active":"2024-09-16T09:34:56.626971+0000","last_peered":"2024-09-16T09:34:56.626971+0000","last_clean":"2024-09-16T09:34:56.626971+0000","last_became_active":"2024-09-16T09:31:09.269342+0000","last_became_peered":"2024-09-16T09:31:09.269342+0000","last_unstale":"2024-09-16T09:34:56.626971+0000","last_undegraded":"2024-09-16T09:34:56.626971+0000","last_fullsized":"2024-09-16T09:34:56.626971+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:33.452706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:34:33.452706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:35:54.801370+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":346,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.620627+0000","last_change":"2024-09-16T09:33:08.816231+0000","last_active":"2024-09-16T09:34:56.620627+0000","last_peered":"2024-09-16T09:34:56.620627+0000","last_clean":"2024-09-16T09:34:56.620627+0000","last_became_active":"2024-09-16T09:31:09.275136+0000","last_became_peered":"2024-09-16T09:31:09.275136+0000","last_unstale":"2024-09-16T09:34:56.620627+0000","last_undegraded":"2024-09-16T09:34:56.620627+0000","last_fullsized":"2024-09-16T09:34:56.620627+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:08.816174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:08.816174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.19","version":"66'15","reported_seq":160,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.616382+0000","last_change":"2024-09-16T09:33:21.337711+0000","last_active":"2024-09-16T09:34:56.616382+0000","last_peered":"2024-09-16T09:34:56.616382+0000","last_clean":"2024-09-16T09:34:56.616382+0000","last_became_active":"2024-09-16T09:33:21.337284+0000","last_became_peered":"2024-09-16T09:33:21.337284+0000","last_unstale":"2024-09-16T09:34:56.616382+0000","last_undegraded":"2024-09-16T09:34:56.616382+0000","last_fullsized":"2024-09-16T09:34:56.616382+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-16T09:32:45.194581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:32:45.194581+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":480,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.632041+0000","last_change":"2024-09-16T09:32:44.114475+0000","last_active":"2024-09-16T09:34:56.632041+0000","last_peered":"2024-09-16T09:34:56.632041+0000","last_clean":"2024-09-16T09:34:56.632041+0000","last_became_active":"2024-09-16T09:25:05.717236+0000","last_became_peered":"2024-09-16T09:25:05.717236+0000","last_unstale":"2024-09-16T09:34:56.632041+0000","last_undegraded":"2024-09-16T09:34:56.632041+0000","last_fullsized":"2024-09-16T09:34:56.632041+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:44.114407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:32:44.114407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.17","version":"279'5288","reported_seq":10330,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:57.818849+0000","last_change":"2024-09-16T09:34:53.523774+0000","last_active":"2024-09-16T09:34:57.818849+0000","last_peered":"2024-09-16T09:34:57.818849+0000","last_clean":"2024-09-16T09:34:57.818849+0000","last_became_active":"2024-09-16T09:34:21.272254+0000","last_became_peered":"2024-09-16T09:34:21.272254+0000","last_unstale":"2024-09-16T09:34:57.818849+0000","last_undegraded":"2024-09-16T09:34:57.818849+0000","last_fullsized":"2024-09-16T09:34:57.818849+0000","mapping_epoch":260,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":5,"last_scrub":"276'5145","last_scrub_stamp":"2024-09-16T09:34:53.523609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:34:53.523609+0000","objects_scrubbed":3599,"log_size":5288,"log_dups_size":0,"ondisk_log_size":5288,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:36:21.313541+0000","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":246349824,"num_objects":3759,"num_object_clones":0,"num_object_copies":11277,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3759,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":160,"num_write_kb":10240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":112,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.616718+0000","last_change":"2024-09-16T09:33:36.655594+0000","last_active":"2024-09-16T09:34:56.616718+0000","last_peered":"2024-09-16T09:34:56.616718+0000","last_clean":"2024-09-16T09:34:56.616718+0000","last_became_active":"2024-09-16T09:33:30.247733+0000","last_became_peered":"2024-09-16T09:33:30.247733+0000","last_unstale":"2024-09-16T09:34:56.616718+0000","last_undegraded":"2024-09-16T09:34:56.616718+0000","last_fullsized":"2024-09-16T09:34:56.616718+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:36.655236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:36.655236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:57.856840+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1a","version":"66'9","reported_seq":491,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.633156+0000","last_change":"2024-09-16T09:32:48.185484+0000","last_active":"2024-09-16T09:34:56.633156+0000","last_peered":"2024-09-16T09:34:56.633156+0000","last_clean":"2024-09-16T09:34:56.633156+0000","last_became_active":"2024-09-16T09:25:03.709147+0000","last_became_peered":"2024-09-16T09:25:03.709147+0000","last_unstale":"2024-09-16T09:34:56.633156+0000","last_undegraded":"2024-09-16T09:34:56.633156+0000","last_fullsized":"2024-09-16T09:34:56.633156+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:32:48.185380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:32:48.185380+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":462,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.625238+0000","last_change":"2024-09-16T09:33:21.376175+0000","last_active":"2024-09-16T09:34:56.625238+0000","last_peered":"2024-09-16T09:34:56.625238+0000","last_clean":"2024-09-16T09:34:56.625238+0000","last_became_active":"2024-09-16T09:25:05.718134+0000","last_became_peered":"2024-09-16T09:25:05.718134+0000","last_unstale":"2024-09-16T09:34:56.625238+0000","last_undegraded":"2024-09-16T09:34:56.625238+0000","last_fullsized":"2024-09-16T09:34:56.625238+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:21.376128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:21.376128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"66'1","reported_seq":477,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.614691+0000","last_change":"2024-09-16T09:33:38.705001+0000","last_active":"2024-09-16T09:34:56.614691+0000","last_peered":"2024-09-16T09:34:56.614691+0000","last_clean":"2024-09-16T09:34:56.614691+0000","last_became_active":"2024-09-16T09:25:07.716558+0000","last_became_peered":"2024-09-16T09:25:07.716558+0000","last_unstale":"2024-09-16T09:34:56.614691+0000","last_undegraded":"2024-09-16T09:34:56.614691+0000","last_fullsized":"2024-09-16T09:34:56.614691+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-16T09:33:38.704868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:33:38.704868+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:35:05.456548+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"279'5310","reported_seq":8617,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:57.861515+0000","last_change":"2024-09-16T09:34:44.425869+0000","last_active":"2024-09-16T09:34:57.861515+0000","last_peered":"2024-09-16T09:34:57.861515+0000","last_clean":"2024-09-16T09:34:57.861515+0000","last_became_active":"2024-09-16T09:34:12.449671+0000","last_became_peered":"2024-09-16T09:34:12.449671+0000","last_unstale":"2024-09-16T09:34:57.861515+0000","last_undegraded":"2024-09-16T09:34:57.861515+0000","last_fullsized":"2024-09-16T09:34:57.861515+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":5,"last_scrub":"273'4808","last_scrub_stamp":"2024-09-16T09:34:44.425779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:34:44.425779+0000","objects_scrubbed":3181,"log_size":5310,"log_dups_size":0,"ondisk_log_size":5310,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:35:58.356833+0000","scrub_duration":809,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":242286592,"num_objects":3697,"num_object_clones":0,"num_object_copies":11091,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3697,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":516,"num_write_kb":33024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":470,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.623077+0000","last_change":"2024-09-16T09:33:19.326765+0000","last_active":"2024-09-16T09:34:56.623077+0000","last_peered":"2024-09-16T09:34:56.623077+0000","last_clean":"2024-09-16T09:34:56.623077+0000","last_became_active":"2024-09-16T09:25:01.696939+0000","last_became_peered":"2024-09-16T09:25:01.696939+0000","last_unstale":"2024-09-16T09:34:56.623077+0000","last_undegraded":"2024-09-16T09:34:56.623077+0000","last_fullsized":"2024-09-16T09:34:56.623077+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:19.326695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:19.326695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"66'5","reported_seq":477,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.632340+0000","last_change":"2024-09-16T09:32:31.024521+0000","last_active":"2024-09-16T09:34:56.632340+0000","last_peered":"2024-09-16T09:34:56.632340+0000","last_clean":"2024-09-16T09:34:56.632340+0000","last_became_active":"2024-09-16T09:25:03.716136+0000","last_became_peered":"2024-09-16T09:25:03.716136+0000","last_unstale":"2024-09-16T09:34:56.632340+0000","last_undegraded":"2024-09-16T09:34:56.632340+0000","last_fullsized":"2024-09-16T09:34:56.632340+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-16T09:32:31.024450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:32:31.024450+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":462,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.623318+0000","last_change":"2024-09-16T09:28:52.746046+0000","last_active":"2024-09-16T09:34:56.623318+0000","last_peered":"2024-09-16T09:34:56.623318+0000","last_clean":"2024-09-16T09:34:56.623318+0000","last_became_active":"2024-09-16T09:25:05.718281+0000","last_became_peered":"2024-09-16T09:25:05.718281+0000","last_unstale":"2024-09-16T09:34:56.623318+0000","last_undegraded":"2024-09-16T09:34:56.623318+0000","last_fullsized":"2024-09-16T09:34:56.623318+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:52.745999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:28:52.745999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":489,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.631892+0000","last_change":"2024-09-16T09:33:46.366042+0000","last_active":"2024-09-16T09:34:56.631892+0000","last_peered":"2024-09-16T09:34:56.631892+0000","last_clean":"2024-09-16T09:34:56.631892+0000","last_became_active":"2024-09-16T09:31:09.275656+0000","last_became_peered":"2024-09-16T09:31:09.275656+0000","last_unstale":"2024-09-16T09:34:56.631892+0000","last_undegraded":"2024-09-16T09:34:56.631892+0000","last_fullsized":"2024-09-16T09:34:56.631892+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:46.365972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:33:46.365972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:35:13.224630+0000","scrub_duration":73,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"279'5199","reported_seq":5338,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:57.804651+0000","last_change":"2024-09-16T09:34:55.666090+0000","last_active":"2024-09-16T09:34:57.804651+0000","last_peered":"2024-09-16T09:34:57.804651+0000","last_clean":"2024-09-16T09:34:57.804651+0000","last_became_active":"2024-09-16T09:33:54.255470+0000","last_became_peered":"2024-09-16T09:33:54.255470+0000","last_unstale":"2024-09-16T09:34:57.804651+0000","last_undegraded":"2024-09-16T09:34:57.804651+0000","last_fullsized":"2024-09-16T09:34:57.804651+0000","mapping_epoch":245,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":246,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:33:24.230486+0000","objects_scrubbed":0,"log_size":5199,"log_dups_size":0,"ondisk_log_size":5199,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:35:25.619111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":241041408,"num_objects":3678,"num_object_clones":0,"num_object_copies":11034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3678,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3678,"num_write_kb":235392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":476,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.634303+0000","last_change":"2024-09-16T09:33:48.322010+0000","last_active":"2024-09-16T09:34:56.634303+0000","last_peered":"2024-09-16T09:34:56.634303+0000","last_clean":"2024-09-16T09:34:56.634303+0000","last_became_active":"2024-09-16T09:25:07.720793+0000","last_became_peered":"2024-09-16T09:25:07.720793+0000","last_unstale":"2024-09-16T09:34:56.634303+0000","last_undegraded":"2024-09-16T09:34:56.634303+0000","last_fullsized":"2024-09-16T09:34:56.634303+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:48.321895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:33:48.321895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:35:01.426825+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":227,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.620979+0000","last_change":"2024-09-16T09:34:33.866299+0000","last_active":"2024-09-16T09:34:56.620979+0000","last_peered":"2024-09-16T09:34:56.620979+0000","last_clean":"2024-09-16T09:34:56.620979+0000","last_became_active":"2024-09-16T09:30:39.201824+0000","last_became_peered":"2024-09-16T09:30:39.201824+0000","last_unstale":"2024-09-16T09:34:56.620979+0000","last_undegraded":"2024-09-16T09:34:56.620979+0000","last_fullsized":"2024-09-16T09:34:56.620979+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:33.866142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:34:33.866142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:35:56.888572+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"66'15","reported_seq":549,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.621195+0000","last_change":"2024-09-16T09:33:30.250162+0000","last_active":"2024-09-16T09:34:56.621195+0000","last_peered":"2024-09-16T09:34:56.621195+0000","last_clean":"2024-09-16T09:34:56.621195+0000","last_became_active":"2024-09-16T09:33:30.249798+0000","last_became_peered":"2024-09-16T09:33:30.249798+0000","last_unstale":"2024-09-16T09:34:56.621195+0000","last_undegraded":"2024-09-16T09:34:56.621195+0000","last_fullsized":"2024-09-16T09:34:56.621195+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-16T09:32:49.202163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:32:49.202163+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:35:17.443981+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":111,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.633694+0000","last_change":"2024-09-16T09:33:43.326933+0000","last_active":"2024-09-16T09:34:56.633694+0000","last_peered":"2024-09-16T09:34:56.633694+0000","last_clean":"2024-09-16T09:34:56.633694+0000","last_became_active":"2024-09-16T09:33:30.251587+0000","last_became_peered":"2024-09-16T09:33:30.251587+0000","last_unstale":"2024-09-16T09:34:56.633694+0000","last_undegraded":"2024-09-16T09:34:56.633694+0000","last_fullsized":"2024-09-16T09:34:56.633694+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:43.326852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:43.326852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:35:01.711807+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,0],"acting":[4,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.10","version":"279'5216","reported_seq":11923,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:57.900078+0000","last_change":"2024-09-16T09:34:28.725831+0000","last_active":"2024-09-16T09:34:57.900078+0000","last_peered":"2024-09-16T09:34:57.900078+0000","last_clean":"2024-09-16T09:34:57.900078+0000","last_became_active":"2024-09-16T09:34:28.331112+0000","last_became_peered":"2024-09-16T09:34:28.331112+0000","last_unstale":"2024-09-16T09:34:57.900078+0000","last_undegraded":"2024-09-16T09:34:57.900078+0000","last_fullsized":"2024-09-16T09:34:57.900078+0000","mapping_epoch":264,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":265,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:33:24.230486+0000","objects_scrubbed":0,"log_size":5216,"log_dups_size":0,"ondisk_log_size":5216,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:34:58.668827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":238157824,"num_objects":3634,"num_object_clones":0,"num_object_copies":10902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3634,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3634,"num_write_kb":232576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1645,"num_bytes_recovered":107806720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":324,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.620839+0000","last_change":"2024-09-16T09:34:37.804874+0000","last_active":"2024-09-16T09:34:56.620839+0000","last_peered":"2024-09-16T09:34:56.620839+0000","last_clean":"2024-09-16T09:34:56.620839+0000","last_became_active":"2024-09-16T09:28:52.974255+0000","last_became_peered":"2024-09-16T09:28:52.974255+0000","last_unstale":"2024-09-16T09:34:56.620839+0000","last_undegraded":"2024-09-16T09:34:56.620839+0000","last_fullsized":"2024-09-16T09:34:56.620839+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:37.804805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:34:37.804805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:36:05.263108+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":479,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.633272+0000","last_change":"2024-09-16T09:30:03.628882+0000","last_active":"2024-09-16T09:34:56.633272+0000","last_peered":"2024-09-16T09:34:56.633272+0000","last_clean":"2024-09-16T09:34:56.633272+0000","last_became_active":"2024-09-16T09:25:01.706491+0000","last_became_peered":"2024-09-16T09:25:01.706491+0000","last_unstale":"2024-09-16T09:34:56.633272+0000","last_undegraded":"2024-09-16T09:34:56.633272+0000","last_fullsized":"2024-09-16T09:34:56.633272+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:30:03.628812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:30:03.628812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"66'12","reported_seq":154,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.633840+0000","last_change":"2024-09-16T09:33:39.284526+0000","last_active":"2024-09-16T09:34:56.633840+0000","last_peered":"2024-09-16T09:34:56.633840+0000","last_clean":"2024-09-16T09:34:56.633840+0000","last_became_active":"2024-09-16T09:33:30.251626+0000","last_became_peered":"2024-09-16T09:33:30.251626+0000","last_unstale":"2024-09-16T09:34:56.633840+0000","last_undegraded":"2024-09-16T09:34:56.633840+0000","last_fullsized":"2024-09-16T09:34:56.633840+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-16T09:33:39.284451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:33:39.284451+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":24,"num_read_kb":15,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":463,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.634352+0000","last_change":"2024-09-16T09:33:38.238661+0000","last_active":"2024-09-16T09:34:56.634352+0000","last_peered":"2024-09-16T09:34:56.634352+0000","last_clean":"2024-09-16T09:34:56.634352+0000","last_became_active":"2024-09-16T09:25:05.710093+0000","last_became_peered":"2024-09-16T09:25:05.710093+0000","last_unstale":"2024-09-16T09:34:56.634352+0000","last_undegraded":"2024-09-16T09:34:56.634352+0000","last_fullsized":"2024-09-16T09:34:56.634352+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:38.238593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:38.238593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"279'5215","reported_seq":10150,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:57.792018+0000","last_change":"2024-09-16T09:34:44.662632+0000","last_active":"2024-09-16T09:34:57.792018+0000","last_peered":"2024-09-16T09:34:57.792018+0000","last_clean":"2024-09-16T09:34:57.792018+0000","last_became_active":"2024-09-16T09:34:12.451887+0000","last_became_peered":"2024-09-16T09:34:12.451887+0000","last_unstale":"2024-09-16T09:34:57.792018+0000","last_undegraded":"2024-09-16T09:34:57.792018+0000","last_fullsized":"2024-09-16T09:34:57.792018+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":5,"last_scrub":"273'4719","last_scrub_stamp":"2024-09-16T09:34:44.662413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:34:44.662413+0000","objects_scrubbed":3181,"log_size":5215,"log_dups_size":0,"ondisk_log_size":5215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:35:53.412877+0000","scrub_duration":1093,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":242352128,"num_objects":3698,"num_object_clones":0,"num_object_copies":11094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3698,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":517,"num_write_kb":33088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,5],"acting":[7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":467,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.623878+0000","last_change":"2024-09-16T09:33:46.253182+0000","last_active":"2024-09-16T09:34:56.623878+0000","last_peered":"2024-09-16T09:34:56.623878+0000","last_clean":"2024-09-16T09:34:56.623878+0000","last_became_active":"2024-09-16T09:25:07.719380+0000","last_became_peered":"2024-09-16T09:25:07.719380+0000","last_unstale":"2024-09-16T09:34:56.623878+0000","last_undegraded":"2024-09-16T09:34:56.623878+0000","last_fullsized":"2024-09-16T09:34:56.623878+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-16T09:33:46.253114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:33:46.253114+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":120,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.626651+0000","last_change":"2024-09-16T09:34:41.401691+0000","last_active":"2024-09-16T09:34:56.626651+0000","last_peered":"2024-09-16T09:34:56.626651+0000","last_clean":"2024-09-16T09:34:56.626651+0000","last_became_active":"2024-09-16T09:33:30.245048+0000","last_became_peered":"2024-09-16T09:33:30.245048+0000","last_unstale":"2024-09-16T09:34:56.626651+0000","last_undegraded":"2024-09-16T09:34:56.626651+0000","last_fullsized":"2024-09-16T09:34:56.626651+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:34:41.401641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:34:41.401641+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:35:42.139806+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"66'10","reported_seq":529,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.615150+0000","last_change":"2024-09-16T09:32:30.286077+0000","last_active":"2024-09-16T09:34:56.615150+0000","last_peered":"2024-09-16T09:34:56.615150+0000","last_clean":"2024-09-16T09:34:56.615150+0000","last_became_active":"2024-09-16T09:31:09.275050+0000","last_became_peered":"2024-09-16T09:31:09.275050+0000","last_unstale":"2024-09-16T09:34:56.615150+0000","last_undegraded":"2024-09-16T09:34:56.615150+0000","last_fullsized":"2024-09-16T09:34:56.615150+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-16T09:32:30.284919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:32:30.284919+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"66'8","reported_seq":627,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.621007+0000","last_change":"2024-09-16T09:34:52.922123+0000","last_active":"2024-09-16T09:34:56.621007+0000","last_peered":"2024-09-16T09:34:56.621007+0000","last_clean":"2024-09-16T09:34:56.621007+0000","last_became_active":"2024-09-16T09:25:05.712516+0000","last_became_peered":"2024-09-16T09:25:05.712516+0000","last_unstale":"2024-09-16T09:34:56.621007+0000","last_undegraded":"2024-09-16T09:34:56.621007+0000","last_fullsized":"2024-09-16T09:34:56.621007+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:34:52.922047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:34:52.922047+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:36:15.788853+0000","scrub_duration":33,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2","version":"279'5255","reported_seq":5391,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:58.173773+0000","last_change":"2024-09-16T09:33:53.529863+0000","last_active":"2024-09-16T09:34:58.173773+0000","last_peered":"2024-09-16T09:34:58.173773+0000","last_clean":"2024-09-16T09:34:58.173773+0000","last_became_active":"2024-09-16T09:33:52.933886+0000","last_became_peered":"2024-09-16T09:33:52.933886+0000","last_unstale":"2024-09-16T09:34:58.173773+0000","last_undegraded":"2024-09-16T09:34:58.173773+0000","last_fullsized":"2024-09-16T09:34:58.173773+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"244'3174","last_scrub_stamp":"2024-09-16T09:33:53.529712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:33:53.529712+0000","objects_scrubbed":1593,"log_size":5255,"log_dups_size":0,"ondisk_log_size":5255,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:59.644537+0000","scrub_duration":449,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":241303552,"num_objects":3682,"num_object_clones":0,"num_object_copies":11046,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3682,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2089,"num_write_kb":133696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.a","version":"73'79","reported_seq":402,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.624012+0000","last_change":"2024-09-16T09:29:18.498887+0000","last_active":"2024-09-16T09:34:56.624012+0000","last_peered":"2024-09-16T09:34:56.624012+0000","last_clean":"2024-09-16T09:34:56.624012+0000","last_became_active":"2024-09-16T09:26:35.127352+0000","last_became_peered":"2024-09-16T09:26:35.127352+0000","last_unstale":"2024-09-16T09:34:56.624012+0000","last_undegraded":"2024-09-16T09:34:56.624012+0000","last_fullsized":"2024-09-16T09:34:56.624012+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:18.498766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:18.498766+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"66'15","reported_seq":411,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.620857+0000","last_change":"2024-09-16T09:28:28.065781+0000","last_active":"2024-09-16T09:34:56.620857+0000","last_peered":"2024-09-16T09:34:56.620857+0000","last_clean":"2024-09-16T09:34:56.620857+0000","last_became_active":"2024-09-16T09:26:33.295672+0000","last_became_peered":"2024-09-16T09:26:33.295672+0000","last_unstale":"2024-09-16T09:34:56.620857+0000","last_undegraded":"2024-09-16T09:34:56.620857+0000","last_fullsized":"2024-09-16T09:34:56.620857+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-16T09:28:28.065699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:28.065699+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:35:27.260832+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":205,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.626540+0000","last_change":"2024-09-16T09:34:39.420386+0000","last_active":"2024-09-16T09:34:56.626540+0000","last_peered":"2024-09-16T09:34:56.626540+0000","last_clean":"2024-09-16T09:34:56.626540+0000","last_became_active":"2024-09-16T09:31:09.274534+0000","last_became_peered":"2024-09-16T09:31:09.274534+0000","last_unstale":"2024-09-16T09:34:56.626540+0000","last_undegraded":"2024-09-16T09:34:56.626540+0000","last_fullsized":"2024-09-16T09:34:56.626540+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:39.420314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:34:39.420314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:35:58.932088+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"66'8","reported_seq":576,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.634427+0000","last_change":"2024-09-16T09:25:05.714128+0000","last_active":"2024-09-16T09:34:56.634427+0000","last_peered":"2024-09-16T09:34:56.634427+0000","last_clean":"2024-09-16T09:34:56.634427+0000","last_became_active":"2024-09-16T09:25:05.713813+0000","last_became_peered":"2024-09-16T09:25:05.713813+0000","last_unstale":"2024-09-16T09:34:56.634427+0000","last_undegraded":"2024-09-16T09:34:56.634427+0000","last_fullsized":"2024-09-16T09:34:56.634427+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":459,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.628628+0000","last_change":"2024-09-16T09:29:24.537802+0000","last_active":"2024-09-16T09:34:56.628628+0000","last_peered":"2024-09-16T09:34:56.628628+0000","last_clean":"2024-09-16T09:34:56.628628+0000","last_became_active":"2024-09-16T09:25:07.717344+0000","last_became_peered":"2024-09-16T09:25:07.717344+0000","last_unstale":"2024-09-16T09:34:56.628628+0000","last_undegraded":"2024-09-16T09:34:56.628628+0000","last_fullsized":"2024-09-16T09:34:56.628628+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:24.537731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:24.537731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"279'7357","reported_seq":11931,"reported_epoch":279,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-16T09:34:57.833019+0000","last_change":"2024-09-16T09:34:15.286261+0000","last_active":"2024-09-16T09:34:57.833019+0000","last_peered":"2024-09-16T09:34:57.833019+0000","last_clean":"2024-09-16T09:33:59.264728+0000","last_became_active":"2024-09-16T09:34:01.447555+0000","last_became_peered":"2024-09-16T09:34:01.447555+0000","last_unstale":"2024-09-16T09:34:57.833019+0000","last_undegraded":"2024-09-16T09:34:57.833019+0000","last_fullsized":"2024-09-16T09:34:01.323950+0000","mapping_epoch":249,"log_start":"247'3500","ondisk_log_start":"247'3500","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:33:24.230486+0000","objects_scrubbed":0,"log_size":3857,"log_dups_size":0,"ondisk_log_size":3857,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":482148352,"num_objects":7357,"num_object_clones":0,"num_object_copies":22071,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1325,"num_objects_unfound":0,"num_objects_dirty":7357,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7357,"num_write_kb":470848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2229,"num_bytes_recovered":146079744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5],"avail_no_missing":["7","1","5"],"object_location_counts":[{"shards":"1,5,7","objects":1325},{"shards":"5,7","objects":6032}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.5","version":"73'79","reported_seq":443,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.634783+0000","last_change":"2024-09-16T09:32:42.123139+0000","last_active":"2024-09-16T09:34:56.634783+0000","last_peered":"2024-09-16T09:34:56.634783+0000","last_clean":"2024-09-16T09:34:56.634783+0000","last_became_active":"2024-09-16T09:30:08.763683+0000","last_became_peered":"2024-09-16T09:30:08.763683+0000","last_unstale":"2024-09-16T09:34:56.634783+0000","last_undegraded":"2024-09-16T09:34:56.634783+0000","last_fullsized":"2024-09-16T09:34:56.634783+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:32:42.123067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:32:42.123067+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"66'18","reported_seq":418,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.621697+0000","last_change":"2024-09-16T09:33:21.333996+0000","last_active":"2024-09-16T09:34:56.621697+0000","last_peered":"2024-09-16T09:34:56.621697+0000","last_clean":"2024-09-16T09:34:56.621697+0000","last_became_active":"2024-09-16T09:31:09.270434+0000","last_became_peered":"2024-09-16T09:31:09.270434+0000","last_unstale":"2024-09-16T09:34:56.621697+0000","last_undegraded":"2024-09-16T09:34:56.621697+0000","last_fullsized":"2024-09-16T09:34:56.621697+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'18","last_scrub_stamp":"2024-09-16T09:33:21.333937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:33:21.333937+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":33,"num_read_kb":21,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":307,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.624528+0000","last_change":"2024-09-16T09:28:58.705169+0000","last_active":"2024-09-16T09:34:56.624528+0000","last_peered":"2024-09-16T09:34:56.624528+0000","last_clean":"2024-09-16T09:34:56.624528+0000","last_became_active":"2024-09-16T09:28:52.983543+0000","last_became_peered":"2024-09-16T09:28:52.983543+0000","last_unstale":"2024-09-16T09:34:56.624528+0000","last_undegraded":"2024-09-16T09:34:56.624528+0000","last_fullsized":"2024-09-16T09:34:56.624528+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:58.705097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:28:58.705097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":486,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.631446+0000","last_change":"2024-09-16T09:31:09.269234+0000","last_active":"2024-09-16T09:34:56.631446+0000","last_peered":"2024-09-16T09:34:56.631446+0000","last_clean":"2024-09-16T09:34:56.631446+0000","last_became_active":"2024-09-16T09:31:09.268864+0000","last_became_peered":"2024-09-16T09:31:09.268864+0000","last_unstale":"2024-09-16T09:34:56.631446+0000","last_undegraded":"2024-09-16T09:34:56.631446+0000","last_fullsized":"2024-09-16T09:34:56.631446+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:50.482240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:50.482240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":474,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.633084+0000","last_change":"2024-09-16T09:32:32.060608+0000","last_active":"2024-09-16T09:34:56.633084+0000","last_peered":"2024-09-16T09:34:56.633084+0000","last_clean":"2024-09-16T09:34:56.633084+0000","last_became_active":"2024-09-16T09:31:09.269433+0000","last_became_peered":"2024-09-16T09:31:09.269433+0000","last_unstale":"2024-09-16T09:34:56.633084+0000","last_undegraded":"2024-09-16T09:34:56.633084+0000","last_fullsized":"2024-09-16T09:34:56.633084+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:32.060564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:32:32.060564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.c","version":"279'7285","reported_seq":7397,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:57.890244+0000","last_change":"2024-09-16T09:33:25.371514+0000","last_active":"2024-09-16T09:34:57.890244+0000","last_peered":"2024-09-16T09:34:57.890244+0000","last_clean":"2024-09-16T09:34:57.890244+0000","last_became_active":"2024-09-16T09:33:25.371230+0000","last_became_peered":"2024-09-16T09:33:25.371230+0000","last_unstale":"2024-09-16T09:34:57.890244+0000","last_undegraded":"2024-09-16T09:34:57.890244+0000","last_fullsized":"2024-09-16T09:34:57.890244+0000","mapping_epoch":229,"log_start":"242'3100","ondisk_log_start":"242'3100","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:33:24.230486+0000","objects_scrubbed":0,"log_size":4185,"log_dups_size":0,"ondisk_log_size":4185,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":477429760,"num_objects":7285,"num_object_clones":0,"num_object_copies":21855,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7285,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7285,"num_write_kb":466240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"73'79","reported_seq":205,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.627502+0000","last_change":"2024-09-16T09:34:42.362203+0000","last_active":"2024-09-16T09:34:56.627502+0000","last_peered":"2024-09-16T09:34:56.627502+0000","last_clean":"2024-09-16T09:34:56.627502+0000","last_became_active":"2024-09-16T09:31:09.275139+0000","last_became_peered":"2024-09-16T09:31:09.275139+0000","last_unstale":"2024-09-16T09:34:56.627502+0000","last_undegraded":"2024-09-16T09:34:56.627502+0000","last_fullsized":"2024-09-16T09:34:56.627502+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:34:42.362121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:34:42.362121+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:36:07.590504+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"66'14","reported_seq":497,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.631526+0000","last_change":"2024-09-16T09:32:34.997417+0000","last_active":"2024-09-16T09:34:56.631526+0000","last_peered":"2024-09-16T09:34:56.631526+0000","last_clean":"2024-09-16T09:34:56.631526+0000","last_became_active":"2024-09-16T09:25:03.708830+0000","last_became_peered":"2024-09-16T09:25:03.708830+0000","last_unstale":"2024-09-16T09:34:56.631526+0000","last_undegraded":"2024-09-16T09:34:56.631526+0000","last_fullsized":"2024-09-16T09:34:56.631526+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'14","last_scrub_stamp":"2024-09-16T09:32:34.997360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:32:34.997360+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":497,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.615367+0000","last_change":"2024-09-16T09:33:43.678231+0000","last_active":"2024-09-16T09:34:56.615367+0000","last_peered":"2024-09-16T09:34:56.615367+0000","last_clean":"2024-09-16T09:34:56.615367+0000","last_became_active":"2024-09-16T09:25:01.702992+0000","last_became_peered":"2024-09-16T09:25:01.702992+0000","last_unstale":"2024-09-16T09:34:56.615367+0000","last_undegraded":"2024-09-16T09:34:56.615367+0000","last_fullsized":"2024-09-16T09:34:56.615367+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:33:43.678138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:43.678138+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:35:13.034934+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"235'12","reported_seq":147,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.631996+0000","last_change":"2024-09-16T09:33:30.251570+0000","last_active":"2024-09-16T09:34:56.631996+0000","last_peered":"2024-09-16T09:34:56.631996+0000","last_clean":"2024-09-16T09:34:56.631996+0000","last_became_active":"2024-09-16T09:33:30.251047+0000","last_became_peered":"2024-09-16T09:33:30.251047+0000","last_unstale":"2024-09-16T09:34:56.631996+0000","last_undegraded":"2024-09-16T09:34:56.631996+0000","last_fullsized":"2024-09-16T09:34:56.631996+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"187'11","last_scrub_stamp":"2024-09-16T09:33:09.417852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:09.417852+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":451,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.624915+0000","last_change":"2024-09-16T09:26:40.699888+0000","last_active":"2024-09-16T09:34:56.624915+0000","last_peered":"2024-09-16T09:34:56.624915+0000","last_clean":"2024-09-16T09:34:56.624915+0000","last_became_active":"2024-09-16T09:25:07.722017+0000","last_became_peered":"2024-09-16T09:25:07.722017+0000","last_unstale":"2024-09-16T09:34:56.624915+0000","last_undegraded":"2024-09-16T09:34:56.624915+0000","last_fullsized":"2024-09-16T09:34:56.624915+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:40.699813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:40.699813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"279'7499","reported_seq":7613,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:58.049347+0000","last_change":"2024-09-16T09:33:25.372320+0000","last_active":"2024-09-16T09:34:58.049347+0000","last_peered":"2024-09-16T09:34:58.049347+0000","last_clean":"2024-09-16T09:34:58.049347+0000","last_became_active":"2024-09-16T09:33:25.372005+0000","last_became_peered":"2024-09-16T09:33:25.372005+0000","last_unstale":"2024-09-16T09:34:58.049347+0000","last_undegraded":"2024-09-16T09:34:58.049347+0000","last_fullsized":"2024-09-16T09:34:58.049347+0000","mapping_epoch":229,"log_start":"242'3100","ondisk_log_start":"242'3100","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:33:24.230486+0000","objects_scrubbed":0,"log_size":4399,"log_dups_size":0,"ondisk_log_size":4399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":491454464,"num_objects":7499,"num_object_clones":0,"num_object_copies":22497,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7499,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7499,"num_write_kb":479936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.7","version":"73'79","reported_seq":536,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.620720+0000","last_change":"2024-09-16T09:34:51.864131+0000","last_active":"2024-09-16T09:34:56.620720+0000","last_peered":"2024-09-16T09:34:56.620720+0000","last_clean":"2024-09-16T09:34:56.620720+0000","last_became_active":"2024-09-16T09:31:09.274707+0000","last_became_peered":"2024-09-16T09:31:09.274707+0000","last_unstale":"2024-09-16T09:34:56.620720+0000","last_undegraded":"2024-09-16T09:34:56.620720+0000","last_fullsized":"2024-09-16T09:34:56.620720+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:34:51.864036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:34:51.864036+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:36:20.395033+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"66'10","reported_seq":146,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.624412+0000","last_change":"2024-09-16T09:33:41.368102+0000","last_active":"2024-09-16T09:34:56.624412+0000","last_peered":"2024-09-16T09:34:56.624412+0000","last_clean":"2024-09-16T09:34:56.624412+0000","last_became_active":"2024-09-16T09:33:30.248453+0000","last_became_peered":"2024-09-16T09:33:30.248453+0000","last_unstale":"2024-09-16T09:34:56.624412+0000","last_undegraded":"2024-09-16T09:34:56.624412+0000","last_fullsized":"2024-09-16T09:34:56.624412+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-16T09:33:41.368016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:33:41.368016+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:35:10.493830+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":14,"num_read_kb":9,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":485,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.633103+0000","last_change":"2024-09-16T09:32:46.388711+0000","last_active":"2024-09-16T09:34:56.633103+0000","last_peered":"2024-09-16T09:34:56.633103+0000","last_clean":"2024-09-16T09:34:56.633103+0000","last_became_active":"2024-09-16T09:31:09.270619+0000","last_became_peered":"2024-09-16T09:31:09.270619+0000","last_unstale":"2024-09-16T09:34:56.633103+0000","last_undegraded":"2024-09-16T09:34:56.633103+0000","last_fullsized":"2024-09-16T09:34:56.633103+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:46.388651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:32:46.388651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"66'8","reported_seq":592,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.615368+0000","last_change":"2024-09-16T09:28:57.628615+0000","last_active":"2024-09-16T09:34:56.615368+0000","last_peered":"2024-09-16T09:34:56.615368+0000","last_clean":"2024-09-16T09:34:56.615368+0000","last_became_active":"2024-09-16T09:25:05.714507+0000","last_became_peered":"2024-09-16T09:25:05.714507+0000","last_unstale":"2024-09-16T09:34:56.615368+0000","last_undegraded":"2024-09-16T09:34:56.615368+0000","last_fullsized":"2024-09-16T09:34:56.615368+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:28:57.628543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:28:57.628543+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":482,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.614818+0000","last_change":"2024-09-16T09:32:33.362233+0000","last_active":"2024-09-16T09:34:56.614818+0000","last_peered":"2024-09-16T09:34:56.614818+0000","last_clean":"2024-09-16T09:34:56.614818+0000","last_became_active":"2024-09-16T09:31:09.283192+0000","last_became_peered":"2024-09-16T09:31:09.283192+0000","last_unstale":"2024-09-16T09:34:56.614818+0000","last_undegraded":"2024-09-16T09:34:56.614818+0000","last_fullsized":"2024-09-16T09:34:56.614818+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:33.362188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:32:33.362188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.e","version":"279'7446","reported_seq":7559,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:57.844078+0000","last_change":"2024-09-16T09:33:25.247961+0000","last_active":"2024-09-16T09:34:57.844078+0000","last_peered":"2024-09-16T09:34:57.844078+0000","last_clean":"2024-09-16T09:34:57.844078+0000","last_became_active":"2024-09-16T09:33:25.247683+0000","last_became_peered":"2024-09-16T09:33:25.247683+0000","last_unstale":"2024-09-16T09:34:57.844078+0000","last_undegraded":"2024-09-16T09:34:57.844078+0000","last_fullsized":"2024-09-16T09:34:57.844078+0000","mapping_epoch":229,"log_start":"250'3600","ondisk_log_start":"250'3600","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:33:24.230486+0000","objects_scrubbed":0,"log_size":3846,"log_dups_size":0,"ondisk_log_size":3846,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":487981056,"num_objects":7446,"num_object_clones":0,"num_object_copies":22338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7446,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7446,"num_write_kb":476544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.6","version":"73'79","reported_seq":509,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.620447+0000","last_change":"2024-09-16T09:34:42.728232+0000","last_active":"2024-09-16T09:34:56.620447+0000","last_peered":"2024-09-16T09:34:56.620447+0000","last_clean":"2024-09-16T09:34:56.620447+0000","last_became_active":"2024-09-16T09:31:09.274565+0000","last_became_peered":"2024-09-16T09:31:09.274565+0000","last_unstale":"2024-09-16T09:34:56.620447+0000","last_undegraded":"2024-09-16T09:34:56.620447+0000","last_fullsized":"2024-09-16T09:34:56.620447+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:34:42.728143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:34:42.728143+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:35:48.900068+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"66'19","reported_seq":504,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.616047+0000","last_change":"2024-09-16T09:31:35.249952+0000","last_active":"2024-09-16T09:34:56.616047+0000","last_peered":"2024-09-16T09:34:56.616047+0000","last_clean":"2024-09-16T09:34:56.616047+0000","last_became_active":"2024-09-16T09:25:03.714797+0000","last_became_peered":"2024-09-16T09:25:03.714797+0000","last_unstale":"2024-09-16T09:34:56.616047+0000","last_undegraded":"2024-09-16T09:34:56.616047+0000","last_fullsized":"2024-09-16T09:34:56.616047+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-16T09:31:35.249864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:31:35.249864+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":103,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.622948+0000","last_change":"2024-09-16T09:33:30.247311+0000","last_active":"2024-09-16T09:34:56.622948+0000","last_peered":"2024-09-16T09:34:56.622948+0000","last_clean":"2024-09-16T09:34:56.622948+0000","last_became_active":"2024-09-16T09:33:30.246996+0000","last_became_peered":"2024-09-16T09:33:30.246996+0000","last_unstale":"2024-09-16T09:34:56.622948+0000","last_undegraded":"2024-09-16T09:34:56.622948+0000","last_fullsized":"2024-09-16T09:34:56.622948+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:11.407616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:11.407616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,2],"acting":[5,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":488,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.619145+0000","last_change":"2024-09-16T09:34:45.772748+0000","last_active":"2024-09-16T09:34:56.619145+0000","last_peered":"2024-09-16T09:34:56.619145+0000","last_clean":"2024-09-16T09:34:56.619145+0000","last_became_active":"2024-09-16T09:25:05.708663+0000","last_became_peered":"2024-09-16T09:25:05.708663+0000","last_unstale":"2024-09-16T09:34:56.619145+0000","last_undegraded":"2024-09-16T09:34:56.619145+0000","last_fullsized":"2024-09-16T09:34:56.619145+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:45.772674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:34:45.772674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:36:11.743458+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":111,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.619608+0000","last_change":"2024-09-16T09:34:36.806165+0000","last_active":"2024-09-16T09:34:56.619608+0000","last_peered":"2024-09-16T09:34:56.619608+0000","last_clean":"2024-09-16T09:34:56.619608+0000","last_became_active":"2024-09-16T09:33:30.250178+0000","last_became_peered":"2024-09-16T09:33:30.250178+0000","last_unstale":"2024-09-16T09:34:56.619608+0000","last_undegraded":"2024-09-16T09:34:56.619608+0000","last_fullsized":"2024-09-16T09:34:56.619608+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:36.805971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:34:36.805971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:35:58.467636+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,2],"acting":[6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"279'5341","reported_seq":5481,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:58.380045+0000","last_change":"2024-09-16T09:34:55.817762+0000","last_active":"2024-09-16T09:34:58.380045+0000","last_peered":"2024-09-16T09:34:58.380045+0000","last_clean":"2024-09-16T09:34:58.380045+0000","last_became_active":"2024-09-16T09:33:52.910861+0000","last_became_peered":"2024-09-16T09:33:52.910861+0000","last_unstale":"2024-09-16T09:34:58.380045+0000","last_undegraded":"2024-09-16T09:34:58.380045+0000","last_fullsized":"2024-09-16T09:34:58.380045+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:33:24.230486+0000","objects_scrubbed":0,"log_size":5341,"log_dups_size":0,"ondisk_log_size":5341,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:35:25.797096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":248053760,"num_objects":3785,"num_object_clones":0,"num_object_copies":11355,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3785,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3785,"num_write_kb":242240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,5],"acting":[0,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"73'79","reported_seq":435,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.631662+0000","last_change":"2024-09-16T09:32:43.420831+0000","last_active":"2024-09-16T09:34:56.631662+0000","last_peered":"2024-09-16T09:34:56.631662+0000","last_clean":"2024-09-16T09:34:56.631662+0000","last_became_active":"2024-09-16T09:30:08.766350+0000","last_became_peered":"2024-09-16T09:30:08.766350+0000","last_unstale":"2024-09-16T09:34:56.631662+0000","last_undegraded":"2024-09-16T09:34:56.631662+0000","last_fullsized":"2024-09-16T09:34:56.631662+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:32:43.420784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:32:43.420784+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"187'30","reported_seq":709,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.625618+0000","last_change":"2024-09-16T09:34:47.032891+0000","last_active":"2024-09-16T09:34:56.625618+0000","last_peered":"2024-09-16T09:34:56.625618+0000","last_clean":"2024-09-16T09:34:56.625618+0000","last_became_active":"2024-09-16T09:31:09.278839+0000","last_became_peered":"2024-09-16T09:31:09.278839+0000","last_unstale":"2024-09-16T09:34:56.625618+0000","last_undegraded":"2024-09-16T09:34:56.625618+0000","last_fullsized":"2024-09-16T09:34:56.625618+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"187'30","last_scrub_stamp":"2024-09-16T09:34:47.032800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:34:47.032800+0000","objects_scrubbed":10,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:36:03.763093+0000","scrub_duration":26,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":472,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.632673+0000","last_change":"2024-09-16T09:29:26.637070+0000","last_active":"2024-09-16T09:34:56.632673+0000","last_peered":"2024-09-16T09:34:56.632673+0000","last_clean":"2024-09-16T09:34:56.632673+0000","last_became_active":"2024-09-16T09:25:01.703076+0000","last_became_peered":"2024-09-16T09:25:01.703076+0000","last_unstale":"2024-09-16T09:34:56.632673+0000","last_undegraded":"2024-09-16T09:34:56.632673+0000","last_fullsized":"2024-09-16T09:34:56.632673+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:26.637027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:26.637027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":489,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.624638+0000","last_change":"2024-09-16T09:33:38.417375+0000","last_active":"2024-09-16T09:34:56.624638+0000","last_peered":"2024-09-16T09:34:56.624638+0000","last_clean":"2024-09-16T09:34:56.624638+0000","last_became_active":"2024-09-16T09:25:00.696237+0000","last_became_peered":"2024-09-16T09:25:00.696237+0000","last_unstale":"2024-09-16T09:34:56.624638+0000","last_undegraded":"2024-09-16T09:34:56.624638+0000","last_fullsized":"2024-09-16T09:34:56.624638+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-16T09:33:38.417308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:33:38.417308+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:34:57.484310+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.0025137430000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":456,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.616939+0000","last_change":"2024-09-16T09:33:16.666704+0000","last_active":"2024-09-16T09:34:56.616939+0000","last_peered":"2024-09-16T09:34:56.616939+0000","last_clean":"2024-09-16T09:34:56.616939+0000","last_became_active":"2024-09-16T09:25:05.717591+0000","last_became_peered":"2024-09-16T09:25:05.717591+0000","last_unstale":"2024-09-16T09:34:56.616939+0000","last_undegraded":"2024-09-16T09:34:56.616939+0000","last_fullsized":"2024-09-16T09:34:56.616939+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:16.666648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:16.666648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":197,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.627614+0000","last_change":"2024-09-16T09:31:09.272967+0000","last_active":"2024-09-16T09:34:56.627614+0000","last_peered":"2024-09-16T09:34:56.627614+0000","last_clean":"2024-09-16T09:34:56.627614+0000","last_became_active":"2024-09-16T09:31:09.272609+0000","last_became_peered":"2024-09-16T09:31:09.272609+0000","last_unstale":"2024-09-16T09:34:56.627614+0000","last_undegraded":"2024-09-16T09:34:56.627614+0000","last_fullsized":"2024-09-16T09:34:56.627614+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:45.201503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:45.201503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.a","version":"279'7280","reported_seq":7394,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:58.017311+0000","last_change":"2024-09-16T09:33:25.244002+0000","last_active":"2024-09-16T09:34:58.017311+0000","last_peered":"2024-09-16T09:34:58.017311+0000","last_clean":"2024-09-16T09:34:58.017311+0000","last_became_active":"2024-09-16T09:33:25.243701+0000","last_became_peered":"2024-09-16T09:33:25.243701+0000","last_unstale":"2024-09-16T09:34:58.017311+0000","last_undegraded":"2024-09-16T09:34:58.017311+0000","last_fullsized":"2024-09-16T09:34:58.017311+0000","mapping_epoch":229,"log_start":"241'2900","ondisk_log_start":"241'2900","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:33:24.230486+0000","objects_scrubbed":0,"log_size":4380,"log_dups_size":0,"ondisk_log_size":4380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":477102080,"num_objects":7280,"num_object_clones":0,"num_object_copies":21840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7280,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7280,"num_write_kb":465920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"73'79","reported_seq":159,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.614865+0000","last_change":"2024-09-16T09:34:32.717994+0000","last_active":"2024-09-16T09:34:56.614865+0000","last_peered":"2024-09-16T09:34:56.614865+0000","last_clean":"2024-09-16T09:34:56.614865+0000","last_became_active":"2024-09-16T09:34:00.336514+0000","last_became_peered":"2024-09-16T09:34:00.336514+0000","last_unstale":"2024-09-16T09:34:56.614865+0000","last_undegraded":"2024-09-16T09:34:56.614865+0000","last_fullsized":"2024-09-16T09:34:56.614865+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:34:32.717915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:34:32.717915+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:35:59.254478+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"66'13","reported_seq":169,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.623415+0000","last_change":"2024-09-16T09:34:15.319682+0000","last_active":"2024-09-16T09:34:56.623415+0000","last_peered":"2024-09-16T09:34:56.623415+0000","last_clean":"2024-09-16T09:34:56.623415+0000","last_became_active":"2024-09-16T09:34:00.337121+0000","last_became_peered":"2024-09-16T09:34:00.337121+0000","last_unstale":"2024-09-16T09:34:56.623415+0000","last_undegraded":"2024-09-16T09:34:56.623415+0000","last_fullsized":"2024-09-16T09:34:56.623415+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'13","last_scrub_stamp":"2024-09-16T09:27:44.155969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:44.155969+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":18,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":103,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.619373+0000","last_change":"2024-09-16T09:33:30.248260+0000","last_active":"2024-09-16T09:34:56.619373+0000","last_peered":"2024-09-16T09:34:56.619373+0000","last_clean":"2024-09-16T09:34:56.619373+0000","last_became_active":"2024-09-16T09:33:30.247596+0000","last_became_peered":"2024-09-16T09:33:30.247596+0000","last_unstale":"2024-09-16T09:34:56.619373+0000","last_undegraded":"2024-09-16T09:34:56.619373+0000","last_fullsized":"2024-09-16T09:34:56.619373+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:22.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:22.331302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,2],"acting":[6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":127,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.616467+0000","last_change":"2024-09-16T09:33:21.338149+0000","last_active":"2024-09-16T09:34:56.616467+0000","last_peered":"2024-09-16T09:34:56.616467+0000","last_clean":"2024-09-16T09:34:56.616467+0000","last_became_active":"2024-09-16T09:33:21.337177+0000","last_became_peered":"2024-09-16T09:33:21.337177+0000","last_unstale":"2024-09-16T09:34:56.616467+0000","last_undegraded":"2024-09-16T09:34:56.616467+0000","last_fullsized":"2024-09-16T09:34:56.616467+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:32:40.165749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:32:40.165749+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":488,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.622641+0000","last_change":"2024-09-16T09:33:40.418799+0000","last_active":"2024-09-16T09:34:56.622641+0000","last_peered":"2024-09-16T09:34:56.622641+0000","last_clean":"2024-09-16T09:34:56.622641+0000","last_became_active":"2024-09-16T09:25:05.711539+0000","last_became_peered":"2024-09-16T09:25:05.711539+0000","last_unstale":"2024-09-16T09:34:56.622641+0000","last_undegraded":"2024-09-16T09:34:56.622641+0000","last_fullsized":"2024-09-16T09:34:56.622641+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:40.417644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:40.417644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:35:03.494764+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":467,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.626561+0000","last_change":"2024-09-16T09:33:39.313624+0000","last_active":"2024-09-16T09:34:56.626561+0000","last_peered":"2024-09-16T09:34:56.626561+0000","last_clean":"2024-09-16T09:34:56.626561+0000","last_became_active":"2024-09-16T09:25:07.723282+0000","last_became_peered":"2024-09-16T09:25:07.723282+0000","last_unstale":"2024-09-16T09:34:56.626561+0000","last_undegraded":"2024-09-16T09:34:56.626561+0000","last_fullsized":"2024-09-16T09:34:56.626561+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:39.313562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:33:39.313562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.b","version":"279'7281","reported_seq":7393,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:58.449624+0000","last_change":"2024-09-16T09:33:25.245108+0000","last_active":"2024-09-16T09:34:58.449624+0000","last_peered":"2024-09-16T09:34:58.449624+0000","last_clean":"2024-09-16T09:34:58.449624+0000","last_became_active":"2024-09-16T09:33:25.244819+0000","last_became_peered":"2024-09-16T09:33:25.244819+0000","last_unstale":"2024-09-16T09:34:58.449624+0000","last_undegraded":"2024-09-16T09:34:58.449624+0000","last_fullsized":"2024-09-16T09:34:58.449624+0000","mapping_epoch":229,"log_start":"240'2500","ondisk_log_start":"240'2500","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:33:24.230486+0000","objects_scrubbed":0,"log_size":4781,"log_dups_size":0,"ondisk_log_size":4781,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":477167616,"num_objects":7281,"num_object_clones":0,"num_object_copies":21843,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7281,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7281,"num_write_kb":465984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"73'79","reported_seq":192,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.621138+0000","last_change":"2024-09-16T09:34:31.929462+0000","last_active":"2024-09-16T09:34:56.621138+0000","last_peered":"2024-09-16T09:34:56.621138+0000","last_clean":"2024-09-16T09:34:56.621138+0000","last_became_active":"2024-09-16T09:33:30.249309+0000","last_became_peered":"2024-09-16T09:33:30.249309+0000","last_unstale":"2024-09-16T09:34:56.621138+0000","last_undegraded":"2024-09-16T09:34:56.621138+0000","last_fullsized":"2024-09-16T09:34:56.621138+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:34:31.929388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:34:31.929388+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:35:38.514043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.6","version":"66'12","reported_seq":486,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.615228+0000","last_change":"2024-09-16T09:32:48.547237+0000","last_active":"2024-09-16T09:34:56.615228+0000","last_peered":"2024-09-16T09:34:56.615228+0000","last_clean":"2024-09-16T09:34:56.615228+0000","last_became_active":"2024-09-16T09:25:03.715315+0000","last_became_peered":"2024-09-16T09:25:03.715315+0000","last_unstale":"2024-09-16T09:34:56.615228+0000","last_undegraded":"2024-09-16T09:34:56.615228+0000","last_fullsized":"2024-09-16T09:34:56.615228+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-16T09:32:48.546869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:32:48.546869+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"59'1","reported_seq":503,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.615147+0000","last_change":"2024-09-16T09:33:21.682893+0000","last_active":"2024-09-16T09:34:56.615147+0000","last_peered":"2024-09-16T09:34:56.615147+0000","last_clean":"2024-09-16T09:34:56.615147+0000","last_became_active":"2024-09-16T09:31:09.269627+0000","last_became_peered":"2024-09-16T09:31:09.269627+0000","last_unstale":"2024-09-16T09:34:56.615147+0000","last_undegraded":"2024-09-16T09:34:56.615147+0000","last_fullsized":"2024-09-16T09:34:56.615147+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:33:21.682840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:21.682840+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1986,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"67'5","reported_seq":1080,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.621875+0000","last_change":"2024-09-16T09:33:37.358839+0000","last_active":"2024-09-16T09:34:56.621875+0000","last_peered":"2024-09-16T09:34:56.621875+0000","last_clean":"2024-09-16T09:34:56.621875+0000","last_became_active":"2024-09-16T09:25:00.696546+0000","last_became_peered":"2024-09-16T09:25:00.696546+0000","last_unstale":"2024-09-16T09:34:56.621875+0000","last_undegraded":"2024-09-16T09:34:56.621875+0000","last_fullsized":"2024-09-16T09:34:56.621875+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'5","last_scrub_stamp":"2024-09-16T09:33:37.358741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:33:37.358741+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0.0059770420000000001,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":590,"num_read_kb":585,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":455,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.623503+0000","last_change":"2024-09-16T09:27:07.337170+0000","last_active":"2024-09-16T09:34:56.623503+0000","last_peered":"2024-09-16T09:34:56.623503+0000","last_clean":"2024-09-16T09:34:56.623503+0000","last_became_active":"2024-09-16T09:25:05.714704+0000","last_became_peered":"2024-09-16T09:25:05.714704+0000","last_unstale":"2024-09-16T09:34:56.623503+0000","last_undegraded":"2024-09-16T09:34:56.623503+0000","last_fullsized":"2024-09-16T09:34:56.623503+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:07.337124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:07.337124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":373,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.623152+0000","last_change":"2024-09-16T09:33:30.246166+0000","last_active":"2024-09-16T09:34:56.623152+0000","last_peered":"2024-09-16T09:34:56.623152+0000","last_clean":"2024-09-16T09:34:56.623152+0000","last_became_active":"2024-09-16T09:33:30.245852+0000","last_became_peered":"2024-09-16T09:33:30.245852+0000","last_unstale":"2024-09-16T09:34:56.623152+0000","last_undegraded":"2024-09-16T09:34:56.623152+0000","last_fullsized":"2024-09-16T09:34:56.623152+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:46.252700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:32:46.252700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"279'5244","reported_seq":10733,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:58.450878+0000","last_change":"2024-09-16T09:34:42.510673+0000","last_active":"2024-09-16T09:34:58.450878+0000","last_peered":"2024-09-16T09:34:58.450878+0000","last_clean":"2024-09-16T09:34:58.450878+0000","last_became_active":"2024-09-16T09:34:41.432414+0000","last_became_peered":"2024-09-16T09:34:41.432414+0000","last_unstale":"2024-09-16T09:34:58.450878+0000","last_undegraded":"2024-09-16T09:34:58.450878+0000","last_fullsized":"2024-09-16T09:34:58.450878+0000","mapping_epoch":272,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":273,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'4604","last_scrub_stamp":"2024-09-16T09:34:42.510459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:34:42.510459+0000","objects_scrubbed":3078,"log_size":5244,"log_dups_size":0,"ondisk_log_size":5244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:35:45.760515+0000","scrub_duration":697,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":244580352,"num_objects":3732,"num_object_clones":0,"num_object_copies":11196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3732,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":654,"num_write_kb":41856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"66'16","reported_seq":626,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.620284+0000","last_change":"2024-09-16T09:33:16.891850+0000","last_active":"2024-09-16T09:34:56.620284+0000","last_peered":"2024-09-16T09:34:56.620284+0000","last_clean":"2024-09-16T09:34:56.620284+0000","last_became_active":"2024-09-16T09:25:03.714118+0000","last_became_peered":"2024-09-16T09:25:03.714118+0000","last_unstale":"2024-09-16T09:34:56.620284+0000","last_undegraded":"2024-09-16T09:34:56.620284+0000","last_fullsized":"2024-09-16T09:34:56.620284+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'16","last_scrub_stamp":"2024-09-16T09:33:16.891761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:33:16.891761+0000","objects_scrubbed":8,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":321,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.624568+0000","last_change":"2024-09-16T09:33:12.171507+0000","last_active":"2024-09-16T09:34:56.624568+0000","last_peered":"2024-09-16T09:34:56.624568+0000","last_clean":"2024-09-16T09:34:56.624568+0000","last_became_active":"2024-09-16T09:31:09.270196+0000","last_became_peered":"2024-09-16T09:31:09.270196+0000","last_unstale":"2024-09-16T09:34:56.624568+0000","last_undegraded":"2024-09-16T09:34:56.624568+0000","last_fullsized":"2024-09-16T09:34:56.624568+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:12.171439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:12.171439+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"73'79","reported_seq":560,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.622117+0000","last_change":"2024-09-16T09:32:43.535041+0000","last_active":"2024-09-16T09:34:56.622117+0000","last_peered":"2024-09-16T09:34:56.622117+0000","last_clean":"2024-09-16T09:34:56.622117+0000","last_became_active":"2024-09-16T09:30:08.870394+0000","last_became_peered":"2024-09-16T09:30:08.870394+0000","last_unstale":"2024-09-16T09:34:56.622117+0000","last_undegraded":"2024-09-16T09:34:56.622117+0000","last_fullsized":"2024-09-16T09:34:56.622117+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:32:43.534962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:32:43.534962+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":463,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.624075+0000","last_change":"2024-09-16T09:33:52.291748+0000","last_active":"2024-09-16T09:34:56.624075+0000","last_peered":"2024-09-16T09:34:56.624075+0000","last_clean":"2024-09-16T09:34:56.624075+0000","last_became_active":"2024-09-16T09:25:05.714341+0000","last_became_peered":"2024-09-16T09:25:05.714341+0000","last_unstale":"2024-09-16T09:34:56.624075+0000","last_undegraded":"2024-09-16T09:34:56.624075+0000","last_fullsized":"2024-09-16T09:34:56.624075+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:52.291702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:52.291702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:35:05.616010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":459,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.623275+0000","last_change":"2024-09-16T09:29:00.243528+0000","last_active":"2024-09-16T09:34:56.623275+0000","last_peered":"2024-09-16T09:34:56.623275+0000","last_clean":"2024-09-16T09:34:56.623275+0000","last_became_active":"2024-09-16T09:25:07.714281+0000","last_became_peered":"2024-09-16T09:25:07.714281+0000","last_unstale":"2024-09-16T09:34:56.623275+0000","last_undegraded":"2024-09-16T09:34:56.623275+0000","last_fullsized":"2024-09-16T09:34:56.623275+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:00.243482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:00.243482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3","version":"279'5159","reported_seq":5304,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:57.800686+0000","last_change":"2024-09-16T09:33:53.717144+0000","last_active":"2024-09-16T09:34:57.800686+0000","last_peered":"2024-09-16T09:34:57.800686+0000","last_clean":"2024-09-16T09:34:57.800686+0000","last_became_active":"2024-09-16T09:33:52.914057+0000","last_became_peered":"2024-09-16T09:33:52.914057+0000","last_unstale":"2024-09-16T09:34:57.800686+0000","last_undegraded":"2024-09-16T09:34:57.800686+0000","last_fullsized":"2024-09-16T09:34:57.800686+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"244'3096","last_scrub_stamp":"2024-09-16T09:33:53.717081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:33:53.717081+0000","objects_scrubbed":1554,"log_size":5159,"log_dups_size":0,"ondisk_log_size":5159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:35:01.397702+0000","scrub_duration":410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":238026752,"num_objects":3632,"num_object_clones":0,"num_object_copies":10896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3632,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2078,"num_write_kb":132992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.b","version":"73'79","reported_seq":295,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.614731+0000","last_change":"2024-09-16T09:34:31.720971+0000","last_active":"2024-09-16T09:34:56.614731+0000","last_peered":"2024-09-16T09:34:56.614731+0000","last_clean":"2024-09-16T09:34:56.614731+0000","last_became_active":"2024-09-16T09:33:30.251933+0000","last_became_peered":"2024-09-16T09:33:30.251933+0000","last_unstale":"2024-09-16T09:34:56.614731+0000","last_undegraded":"2024-09-16T09:34:56.614731+0000","last_fullsized":"2024-09-16T09:34:56.614731+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:34:31.720787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:34:31.720787+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:35:50.655452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"66'11","reported_seq":517,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.634477+0000","last_change":"2024-09-16T09:34:00.338209+0000","last_active":"2024-09-16T09:34:56.634477+0000","last_peered":"2024-09-16T09:34:56.634477+0000","last_clean":"2024-09-16T09:34:56.634477+0000","last_became_active":"2024-09-16T09:34:00.328533+0000","last_became_peered":"2024-09-16T09:34:00.328533+0000","last_unstale":"2024-09-16T09:34:56.634477+0000","last_undegraded":"2024-09-16T09:34:56.634477+0000","last_fullsized":"2024-09-16T09:34:56.634477+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-16T09:30:05.646500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:30:05.646500+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[4,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":464,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.631862+0000","last_change":"2024-09-16T09:33:23.200213+0000","last_active":"2024-09-16T09:34:56.631862+0000","last_peered":"2024-09-16T09:34:56.631862+0000","last_clean":"2024-09-16T09:34:56.631862+0000","last_became_active":"2024-09-16T09:25:01.708270+0000","last_became_peered":"2024-09-16T09:25:01.708270+0000","last_unstale":"2024-09-16T09:34:56.631862+0000","last_undegraded":"2024-09-16T09:34:56.631862+0000","last_fullsized":"2024-09-16T09:34:56.631862+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:23.200166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:23.200166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":455,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.623055+0000","last_change":"2024-09-16T09:26:34.171190+0000","last_active":"2024-09-16T09:34:56.623055+0000","last_peered":"2024-09-16T09:34:56.623055+0000","last_clean":"2024-09-16T09:34:56.623055+0000","last_became_active":"2024-09-16T09:25:05.711964+0000","last_became_peered":"2024-09-16T09:25:05.711964+0000","last_unstale":"2024-09-16T09:34:56.623055+0000","last_undegraded":"2024-09-16T09:34:56.623055+0000","last_fullsized":"2024-09-16T09:34:56.623055+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:34.171139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:26:34.171139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":346,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.619972+0000","last_change":"2024-09-16T09:34:50.866121+0000","last_active":"2024-09-16T09:34:56.619972+0000","last_peered":"2024-09-16T09:34:56.619972+0000","last_clean":"2024-09-16T09:34:56.619972+0000","last_became_active":"2024-09-16T09:31:09.278618+0000","last_became_peered":"2024-09-16T09:31:09.278618+0000","last_unstale":"2024-09-16T09:34:56.619972+0000","last_undegraded":"2024-09-16T09:34:56.619972+0000","last_fullsized":"2024-09-16T09:34:56.619972+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:50.866032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:34:50.866032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:36:08.203842+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.0","version":"279'5269","reported_seq":5389,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:57.613246+0000","last_change":"2024-09-16T09:34:55.879445+0000","last_active":"2024-09-16T09:34:57.613246+0000","last_peered":"2024-09-16T09:34:57.613246+0000","last_clean":"2024-09-16T09:34:57.613246+0000","last_became_active":"2024-09-16T09:33:52.914803+0000","last_became_peered":"2024-09-16T09:33:52.914803+0000","last_unstale":"2024-09-16T09:34:57.613246+0000","last_undegraded":"2024-09-16T09:34:57.613246+0000","last_fullsized":"2024-09-16T09:34:57.613246+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:33:24.230486+0000","objects_scrubbed":0,"log_size":5269,"log_dups_size":0,"ondisk_log_size":5269,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:35:25.869755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":241631232,"num_objects":3687,"num_object_clones":0,"num_object_copies":11061,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3687,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3687,"num_write_kb":235968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,2],"acting":[6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.8","version":"73'79","reported_seq":87,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.618669+0000","last_change":"2024-09-16T09:34:30.931922+0000","last_active":"2024-09-16T09:34:56.618669+0000","last_peered":"2024-09-16T09:34:56.618669+0000","last_clean":"2024-09-16T09:34:56.618669+0000","last_became_active":"2024-09-16T09:34:00.329241+0000","last_became_peered":"2024-09-16T09:34:00.329241+0000","last_unstale":"2024-09-16T09:34:56.618669+0000","last_undegraded":"2024-09-16T09:34:56.618669+0000","last_fullsized":"2024-09-16T09:34:56.618669+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:34:30.931863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:34:30.931863+0000","objects_scrubbed":1,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:35:51.811083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262688,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,3],"acting":[6,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"66'17","reported_seq":504,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.636875+0000","last_change":"2024-09-16T09:30:04.636115+0000","last_active":"2024-09-16T09:34:56.636875+0000","last_peered":"2024-09-16T09:34:56.636875+0000","last_clean":"2024-09-16T09:34:56.636875+0000","last_became_active":"2024-09-16T09:25:03.713358+0000","last_became_peered":"2024-09-16T09:25:03.713358+0000","last_unstale":"2024-09-16T09:34:56.636875+0000","last_undegraded":"2024-09-16T09:34:56.636875+0000","last_fullsized":"2024-09-16T09:34:56.636875+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'17","last_scrub_stamp":"2024-09-16T09:30:04.636039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:30:04.636039+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":472,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.619476+0000","last_change":"2024-09-16T09:34:32.899965+0000","last_active":"2024-09-16T09:34:56.619476+0000","last_peered":"2024-09-16T09:34:56.619476+0000","last_clean":"2024-09-16T09:34:56.619476+0000","last_became_active":"2024-09-16T09:25:01.703355+0000","last_became_peered":"2024-09-16T09:25:01.703355+0000","last_unstale":"2024-09-16T09:34:56.619476+0000","last_undegraded":"2024-09-16T09:34:56.619476+0000","last_fullsized":"2024-09-16T09:34:56.619476+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:32.899886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:34:32.899886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:35:58.423518+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":111,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.637035+0000","last_change":"2024-09-16T09:33:44.298324+0000","last_active":"2024-09-16T09:34:56.637035+0000","last_peered":"2024-09-16T09:34:56.637035+0000","last_clean":"2024-09-16T09:34:56.637035+0000","last_became_active":"2024-09-16T09:33:30.251186+0000","last_became_peered":"2024-09-16T09:33:30.251186+0000","last_unstale":"2024-09-16T09:34:56.637035+0000","last_undegraded":"2024-09-16T09:34:56.637035+0000","last_fullsized":"2024-09-16T09:34:56.637035+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:44.298246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:44.298246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:35:02.507010+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,0],"acting":[4,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":401,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.627578+0000","last_change":"2024-09-16T09:33:21.337606+0000","last_active":"2024-09-16T09:34:56.627578+0000","last_peered":"2024-09-16T09:34:56.627578+0000","last_clean":"2024-09-16T09:34:56.627578+0000","last_became_active":"2024-09-16T09:33:21.337291+0000","last_became_peered":"2024-09-16T09:33:21.337291+0000","last_unstale":"2024-09-16T09:34:56.627578+0000","last_undegraded":"2024-09-16T09:34:56.627578+0000","last_fullsized":"2024-09-16T09:34:56.627578+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:48.259971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:32:48.259971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4],"acting":[3,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1","version":"279'5203","reported_seq":5356,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:57.676834+0000","last_change":"2024-09-16T09:34:55.644496+0000","last_active":"2024-09-16T09:34:57.676834+0000","last_peered":"2024-09-16T09:34:57.676834+0000","last_clean":"2024-09-16T09:34:57.676834+0000","last_became_active":"2024-09-16T09:33:52.910622+0000","last_became_peered":"2024-09-16T09:33:52.910622+0000","last_unstale":"2024-09-16T09:34:57.676834+0000","last_undegraded":"2024-09-16T09:34:57.676834+0000","last_fullsized":"2024-09-16T09:34:57.676834+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:33:24.230486+0000","objects_scrubbed":2399,"log_size":5203,"log_dups_size":0,"ondisk_log_size":5203,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:35:25.629527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":241303552,"num_objects":3682,"num_object_clones":0,"num_object_copies":11046,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3682,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3682,"num_write_kb":235648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.9","version":"73'79","reported_seq":535,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.623595+0000","last_change":"2024-09-16T09:29:19.013948+0000","last_active":"2024-09-16T09:34:56.623595+0000","last_peered":"2024-09-16T09:34:56.623595+0000","last_clean":"2024-09-16T09:34:56.623595+0000","last_became_active":"2024-09-16T09:26:35.126066+0000","last_became_peered":"2024-09-16T09:26:35.126066+0000","last_unstale":"2024-09-16T09:34:56.623595+0000","last_undegraded":"2024-09-16T09:34:56.623595+0000","last_fullsized":"2024-09-16T09:34:56.623595+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:19.013865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:19.013865+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"66'10","reported_seq":475,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.634507+0000","last_change":"2024-09-16T09:33:24.154567+0000","last_active":"2024-09-16T09:34:56.634507+0000","last_peered":"2024-09-16T09:34:56.634507+0000","last_clean":"2024-09-16T09:34:56.634507+0000","last_became_active":"2024-09-16T09:25:03.707723+0000","last_became_peered":"2024-09-16T09:25:03.707723+0000","last_unstale":"2024-09-16T09:34:56.634507+0000","last_undegraded":"2024-09-16T09:34:56.634507+0000","last_fullsized":"2024-09-16T09:34:56.634507+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-16T09:33:24.154507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:33:24.154507+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":337,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.615613+0000","last_change":"2024-09-16T09:33:10.528181+0000","last_active":"2024-09-16T09:34:56.615613+0000","last_peered":"2024-09-16T09:34:56.615613+0000","last_clean":"2024-09-16T09:34:56.615613+0000","last_became_active":"2024-09-16T09:31:09.288825+0000","last_became_peered":"2024-09-16T09:31:09.288825+0000","last_unstale":"2024-09-16T09:34:56.615613+0000","last_undegraded":"2024-09-16T09:34:56.615613+0000","last_fullsized":"2024-09-16T09:34:56.615613+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:10.528115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:10.528115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"228'9","reported_seq":164,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.626690+0000","last_change":"2024-09-16T09:33:21.337623+0000","last_active":"2024-09-16T09:34:56.626690+0000","last_peered":"2024-09-16T09:34:56.626690+0000","last_clean":"2024-09-16T09:34:56.626690+0000","last_became_active":"2024-09-16T09:33:21.337213+0000","last_became_peered":"2024-09-16T09:33:21.337213+0000","last_unstale":"2024-09-16T09:34:56.626690+0000","last_undegraded":"2024-09-16T09:34:56.626690+0000","last_fullsized":"2024-09-16T09:34:56.626690+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:32:42.175528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:32:42.175528+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":476,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.634221+0000","last_change":"2024-09-16T09:32:39.072795+0000","last_active":"2024-09-16T09:34:56.634221+0000","last_peered":"2024-09-16T09:34:56.634221+0000","last_clean":"2024-09-16T09:34:56.634221+0000","last_became_active":"2024-09-16T09:25:07.718958+0000","last_became_peered":"2024-09-16T09:25:07.718958+0000","last_unstale":"2024-09-16T09:34:56.634221+0000","last_undegraded":"2024-09-16T09:34:56.634221+0000","last_fullsized":"2024-09-16T09:34:56.634221+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:39.072735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:32:39.072735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.6","version":"279'5299","reported_seq":5458,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:58.012298+0000","last_change":"2024-09-16T09:34:55.669695+0000","last_active":"2024-09-16T09:34:58.012298+0000","last_peered":"2024-09-16T09:34:58.012298+0000","last_clean":"2024-09-16T09:34:58.012298+0000","last_became_active":"2024-09-16T09:33:52.928727+0000","last_became_peered":"2024-09-16T09:33:52.928727+0000","last_unstale":"2024-09-16T09:34:58.012298+0000","last_undegraded":"2024-09-16T09:34:58.012298+0000","last_fullsized":"2024-09-16T09:34:58.012298+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:33:24.230486+0000","objects_scrubbed":2649,"log_size":5299,"log_dups_size":0,"ondisk_log_size":5299,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:35:25.617766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":244711424,"num_objects":3734,"num_object_clones":0,"num_object_copies":11202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3734,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3734,"num_write_kb":238976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.e","version":"73'79","reported_seq":269,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.634131+0000","last_change":"2024-09-16T09:32:43.092027+0000","last_active":"2024-09-16T09:34:56.634131+0000","last_peered":"2024-09-16T09:34:56.634131+0000","last_clean":"2024-09-16T09:34:56.634131+0000","last_became_active":"2024-09-16T09:30:10.705072+0000","last_became_peered":"2024-09-16T09:30:10.705072+0000","last_unstale":"2024-09-16T09:34:56.634131+0000","last_undegraded":"2024-09-16T09:34:56.634131+0000","last_fullsized":"2024-09-16T09:34:56.634131+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:32:43.091969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:32:43.091969+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,3],"acting":[4,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.b","version":"66'9","reported_seq":492,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.615437+0000","last_change":"2024-09-16T09:31:31.211495+0000","last_active":"2024-09-16T09:34:56.615437+0000","last_peered":"2024-09-16T09:34:56.615437+0000","last_clean":"2024-09-16T09:34:56.615437+0000","last_became_active":"2024-09-16T09:25:03.712230+0000","last_became_peered":"2024-09-16T09:25:03.712230+0000","last_unstale":"2024-09-16T09:34:56.615437+0000","last_undegraded":"2024-09-16T09:34:56.615437+0000","last_fullsized":"2024-09-16T09:34:56.615437+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:31:31.211420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:31:31.211420+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":477,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.623228+0000","last_change":"2024-09-16T09:31:09.277413+0000","last_active":"2024-09-16T09:34:56.623228+0000","last_peered":"2024-09-16T09:34:56.623228+0000","last_clean":"2024-09-16T09:34:56.623228+0000","last_became_active":"2024-09-16T09:31:09.276516+0000","last_became_peered":"2024-09-16T09:31:09.276516+0000","last_unstale":"2024-09-16T09:34:56.623228+0000","last_undegraded":"2024-09-16T09:34:56.623228+0000","last_fullsized":"2024-09-16T09:34:56.623228+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:26.061106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:26.061106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":405,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.632969+0000","last_change":"2024-09-16T09:33:40.260547+0000","last_active":"2024-09-16T09:34:56.632969+0000","last_peered":"2024-09-16T09:34:56.632969+0000","last_clean":"2024-09-16T09:34:56.632969+0000","last_became_active":"2024-09-16T09:33:21.337254+0000","last_became_peered":"2024-09-16T09:33:21.337254+0000","last_unstale":"2024-09-16T09:34:56.632969+0000","last_undegraded":"2024-09-16T09:34:56.632969+0000","last_fullsized":"2024-09-16T09:34:56.632969+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:40.260500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:40.260500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3],"acting":[4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":468,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.615067+0000","last_change":"2024-09-16T09:31:27.185957+0000","last_active":"2024-09-16T09:34:56.615067+0000","last_peered":"2024-09-16T09:34:56.615067+0000","last_clean":"2024-09-16T09:34:56.615067+0000","last_became_active":"2024-09-16T09:25:07.718778+0000","last_became_peered":"2024-09-16T09:25:07.718778+0000","last_unstale":"2024-09-16T09:34:56.615067+0000","last_undegraded":"2024-09-16T09:34:56.615067+0000","last_fullsized":"2024-09-16T09:34:56.615067+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:31:27.185801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:31:27.185801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"279'5252","reported_seq":5399,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:58.016836+0000","last_change":"2024-09-16T09:33:53.762479+0000","last_active":"2024-09-16T09:34:58.016836+0000","last_peered":"2024-09-16T09:34:58.016836+0000","last_clean":"2024-09-16T09:34:58.016836+0000","last_became_active":"2024-09-16T09:33:52.930653+0000","last_became_peered":"2024-09-16T09:33:52.930653+0000","last_unstale":"2024-09-16T09:34:58.016836+0000","last_undegraded":"2024-09-16T09:34:58.016836+0000","last_fullsized":"2024-09-16T09:34:58.016836+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"244'3138","last_scrub_stamp":"2024-09-16T09:33:53.762280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:33:53.762280+0000","objects_scrubbed":1562,"log_size":5252,"log_dups_size":0,"ondisk_log_size":5252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:35:13.259315+0000","scrub_duration":422,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":241827840,"num_objects":3690,"num_object_clones":0,"num_object_copies":11070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3690,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2128,"num_write_kb":136192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.f","version":"73'79","reported_seq":529,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.621061+0000","last_change":"2024-09-16T09:34:53.866426+0000","last_active":"2024-09-16T09:34:56.621061+0000","last_peered":"2024-09-16T09:34:56.621061+0000","last_clean":"2024-09-16T09:34:56.621061+0000","last_became_active":"2024-09-16T09:30:10.693477+0000","last_became_peered":"2024-09-16T09:30:10.693477+0000","last_unstale":"2024-09-16T09:34:56.621061+0000","last_undegraded":"2024-09-16T09:34:56.621061+0000","last_fullsized":"2024-09-16T09:34:56.621061+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:34:53.866337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:34:53.866337+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:36:03.166374+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"66'19","reported_seq":507,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.619291+0000","last_change":"2024-09-16T09:33:22.905808+0000","last_active":"2024-09-16T09:34:56.619291+0000","last_peered":"2024-09-16T09:34:56.619291+0000","last_clean":"2024-09-16T09:34:56.619291+0000","last_became_active":"2024-09-16T09:25:03.712783+0000","last_became_peered":"2024-09-16T09:25:03.712783+0000","last_unstale":"2024-09-16T09:34:56.619291+0000","last_undegraded":"2024-09-16T09:34:56.619291+0000","last_fullsized":"2024-09-16T09:34:56.619291+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-16T09:33:22.905752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:33:22.905752+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"59'1","reported_seq":485,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.627195+0000","last_change":"2024-09-16T09:33:42.294543+0000","last_active":"2024-09-16T09:34:56.627195+0000","last_peered":"2024-09-16T09:34:56.627195+0000","last_clean":"2024-09-16T09:34:56.627195+0000","last_became_active":"2024-09-16T09:25:01.708069+0000","last_became_peered":"2024-09-16T09:25:01.708069+0000","last_unstale":"2024-09-16T09:34:56.627195+0000","last_undegraded":"2024-09-16T09:34:56.627195+0000","last_fullsized":"2024-09-16T09:34:56.627195+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:33:42.294378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:42.294378+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:35:07.729855+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":479,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.619974+0000","last_change":"2024-09-16T09:34:38.338415+0000","last_active":"2024-09-16T09:34:56.619974+0000","last_peered":"2024-09-16T09:34:56.619974+0000","last_clean":"2024-09-16T09:34:56.619974+0000","last_became_active":"2024-09-16T09:34:00.330050+0000","last_became_peered":"2024-09-16T09:34:00.330050+0000","last_unstale":"2024-09-16T09:34:56.619974+0000","last_undegraded":"2024-09-16T09:34:56.619974+0000","last_fullsized":"2024-09-16T09:34:56.619974+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:38.338329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:34:38.338329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:36:06.360092+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,7],"acting":[2,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":451,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.621992+0000","last_change":"2024-09-16T09:27:25.865107+0000","last_active":"2024-09-16T09:34:56.621992+0000","last_peered":"2024-09-16T09:34:56.621992+0000","last_clean":"2024-09-16T09:34:56.621992+0000","last_became_active":"2024-09-16T09:25:07.718750+0000","last_became_peered":"2024-09-16T09:25:07.718750+0000","last_unstale":"2024-09-16T09:34:56.621992+0000","last_undegraded":"2024-09-16T09:34:56.621992+0000","last_fullsized":"2024-09-16T09:34:56.621992+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:25.865030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:25.865030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.4","version":"279'5375","reported_seq":5512,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:57.869452+0000","last_change":"2024-09-16T09:33:53.661296+0000","last_active":"2024-09-16T09:34:57.869452+0000","last_peered":"2024-09-16T09:34:57.869452+0000","last_clean":"2024-09-16T09:34:57.869452+0000","last_became_active":"2024-09-16T09:33:52.909537+0000","last_became_peered":"2024-09-16T09:33:52.909537+0000","last_unstale":"2024-09-16T09:34:57.869452+0000","last_undegraded":"2024-09-16T09:34:57.869452+0000","last_fullsized":"2024-09-16T09:34:57.869452+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"244'3237","last_scrub_stamp":"2024-09-16T09:33:53.661132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:33:53.661132+0000","objects_scrubbed":1642,"log_size":5375,"log_dups_size":0,"ondisk_log_size":5375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:35:11.863455+0000","scrub_duration":365,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":248250368,"num_objects":3788,"num_object_clones":0,"num_object_copies":11364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3788,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2146,"num_write_kb":137344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.c","version":"73'79","reported_seq":209,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.627397+0000","last_change":"2024-09-16T09:33:08.949107+0000","last_active":"2024-09-16T09:34:56.627397+0000","last_peered":"2024-09-16T09:34:56.627397+0000","last_clean":"2024-09-16T09:34:56.627397+0000","last_became_active":"2024-09-16T09:31:09.273770+0000","last_became_peered":"2024-09-16T09:31:09.273770+0000","last_unstale":"2024-09-16T09:34:56.627397+0000","last_undegraded":"2024-09-16T09:34:56.627397+0000","last_fullsized":"2024-09-16T09:34:56.627397+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:33:08.949051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:33:08.949051+0000","objects_scrubbed":1,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":327680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"66'12","reported_seq":529,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.631720+0000","last_change":"2024-09-16T09:31:09.296915+0000","last_active":"2024-09-16T09:34:56.631720+0000","last_peered":"2024-09-16T09:34:56.631720+0000","last_clean":"2024-09-16T09:34:56.631720+0000","last_became_active":"2024-09-16T09:31:09.269047+0000","last_became_peered":"2024-09-16T09:31:09.269047+0000","last_unstale":"2024-09-16T09:34:56.631720+0000","last_undegraded":"2024-09-16T09:34:56.631720+0000","last_fullsized":"2024-09-16T09:34:56.631720+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-16T09:30:21.728668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:30:21.728668+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":479,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.626094+0000","last_change":"2024-09-16T09:33:48.225867+0000","last_active":"2024-09-16T09:34:56.626094+0000","last_peered":"2024-09-16T09:34:56.626094+0000","last_clean":"2024-09-16T09:34:56.626094+0000","last_became_active":"2024-09-16T09:25:01.707528+0000","last_became_peered":"2024-09-16T09:25:01.707528+0000","last_unstale":"2024-09-16T09:34:56.626094+0000","last_undegraded":"2024-09-16T09:34:56.626094+0000","last_fullsized":"2024-09-16T09:34:56.626094+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:48.225635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:48.225635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:35:17.793189+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":123,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.614611+0000","last_change":"2024-09-16T09:33:23.643136+0000","last_active":"2024-09-16T09:34:56.614611+0000","last_peered":"2024-09-16T09:34:56.614611+0000","last_clean":"2024-09-16T09:34:56.614611+0000","last_became_active":"2024-09-16T09:33:21.337309+0000","last_became_peered":"2024-09-16T09:33:21.337309+0000","last_unstale":"2024-09-16T09:34:56.614611+0000","last_undegraded":"2024-09-16T09:34:56.614611+0000","last_fullsized":"2024-09-16T09:34:56.614611+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:23.643089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:23.643089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":460,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.627061+0000","last_change":"2024-09-16T09:34:30.458186+0000","last_active":"2024-09-16T09:34:56.627061+0000","last_peered":"2024-09-16T09:34:56.627061+0000","last_clean":"2024-09-16T09:34:56.627061+0000","last_became_active":"2024-09-16T09:25:07.718899+0000","last_became_peered":"2024-09-16T09:25:07.718899+0000","last_unstale":"2024-09-16T09:34:56.627061+0000","last_undegraded":"2024-09-16T09:34:56.627061+0000","last_fullsized":"2024-09-16T09:34:56.627061+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:30.458115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:34:30.458115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:35:38.072321+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.5","version":"279'5341","reported_seq":8980,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:58.345560+0000","last_change":"2024-09-16T09:34:34.440150+0000","last_active":"2024-09-16T09:34:58.345560+0000","last_peered":"2024-09-16T09:34:58.345560+0000","last_clean":"2024-09-16T09:34:58.345560+0000","last_became_active":"2024-09-16T09:34:32.381065+0000","last_became_peered":"2024-09-16T09:34:32.381065+0000","last_unstale":"2024-09-16T09:34:58.345560+0000","last_undegraded":"2024-09-16T09:34:58.345560+0000","last_fullsized":"2024-09-16T09:34:58.345560+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'4368","last_scrub_stamp":"2024-09-16T09:34:34.439986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:34:34.439986+0000","objects_scrubbed":2756,"log_size":5341,"log_dups_size":0,"ondisk_log_size":5341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:35:47.700208+0000","scrub_duration":710,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":245104640,"num_objects":3740,"num_object_clones":0,"num_object_copies":11220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3740,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":984,"num_write_kb":62976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.d","version":"73'79","reported_seq":269,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.626364+0000","last_change":"2024-09-16T09:32:44.536868+0000","last_active":"2024-09-16T09:34:56.626364+0000","last_peered":"2024-09-16T09:34:56.626364+0000","last_clean":"2024-09-16T09:34:56.626364+0000","last_became_active":"2024-09-16T09:30:10.709958+0000","last_became_peered":"2024-09-16T09:30:10.709958+0000","last_unstale":"2024-09-16T09:34:56.626364+0000","last_undegraded":"2024-09-16T09:34:56.626364+0000","last_fullsized":"2024-09-16T09:34:56.626364+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:32:44.536799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:32:44.536799+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"66'15","reported_seq":484,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.620797+0000","last_change":"2024-09-16T09:26:33.138828+0000","last_active":"2024-09-16T09:34:56.620797+0000","last_peered":"2024-09-16T09:34:56.620797+0000","last_clean":"2024-09-16T09:34:56.620797+0000","last_became_active":"2024-09-16T09:25:03.715733+0000","last_became_peered":"2024-09-16T09:25:03.715733+0000","last_unstale":"2024-09-16T09:34:56.620797+0000","last_undegraded":"2024-09-16T09:34:56.620797+0000","last_fullsized":"2024-09-16T09:34:56.620797+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-16T09:26:33.138603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:26:33.138603+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":483,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.624612+0000","last_change":"2024-09-16T09:32:49.441194+0000","last_active":"2024-09-16T09:34:56.624612+0000","last_peered":"2024-09-16T09:34:56.624612+0000","last_clean":"2024-09-16T09:34:56.624612+0000","last_became_active":"2024-09-16T09:25:01.707372+0000","last_became_peered":"2024-09-16T09:25:01.707372+0000","last_unstale":"2024-09-16T09:34:56.624612+0000","last_undegraded":"2024-09-16T09:34:56.624612+0000","last_fullsized":"2024-09-16T09:34:56.624612+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-16T09:32:49.441122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:32:49.441122+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"66'8","reported_seq":599,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.623465+0000","last_change":"2024-09-16T09:30:05.138362+0000","last_active":"2024-09-16T09:34:56.623465+0000","last_peered":"2024-09-16T09:34:56.623465+0000","last_clean":"2024-09-16T09:34:56.623465+0000","last_became_active":"2024-09-16T09:25:05.716189+0000","last_became_peered":"2024-09-16T09:25:05.716189+0000","last_unstale":"2024-09-16T09:34:56.623465+0000","last_undegraded":"2024-09-16T09:34:56.623465+0000","last_fullsized":"2024-09-16T09:34:56.623465+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:30:05.138288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:30:05.138288+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":451,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.624150+0000","last_change":"2024-09-16T09:27:20.219376+0000","last_active":"2024-09-16T09:34:56.624150+0000","last_peered":"2024-09-16T09:34:56.624150+0000","last_clean":"2024-09-16T09:34:56.624150+0000","last_became_active":"2024-09-16T09:25:07.717213+0000","last_became_peered":"2024-09-16T09:25:07.717213+0000","last_unstale":"2024-09-16T09:34:56.624150+0000","last_undegraded":"2024-09-16T09:34:56.624150+0000","last_fullsized":"2024-09-16T09:34:56.624150+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:20.219333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:20.219333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.12","version":"73'79","reported_seq":269,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.624333+0000","last_change":"2024-09-16T09:32:45.399435+0000","last_active":"2024-09-16T09:34:56.624333+0000","last_peered":"2024-09-16T09:34:56.624333+0000","last_clean":"2024-09-16T09:34:56.624333+0000","last_became_active":"2024-09-16T09:30:10.731413+0000","last_became_peered":"2024-09-16T09:30:10.731413+0000","last_unstale":"2024-09-16T09:34:56.624333+0000","last_undegraded":"2024-09-16T09:34:56.624333+0000","last_fullsized":"2024-09-16T09:34:56.624333+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:32:45.399361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:32:45.399361+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":479,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.619454+0000","last_change":"2024-09-16T09:34:00.334779+0000","last_active":"2024-09-16T09:34:56.619454+0000","last_peered":"2024-09-16T09:34:56.619454+0000","last_clean":"2024-09-16T09:34:56.619454+0000","last_became_active":"2024-09-16T09:34:00.334361+0000","last_became_peered":"2024-09-16T09:34:00.334361+0000","last_unstale":"2024-09-16T09:34:56.619454+0000","last_undegraded":"2024-09-16T09:34:56.619454+0000","last_fullsized":"2024-09-16T09:34:56.619454+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:14.845355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:14.845355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"66'6","reported_seq":221,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.627637+0000","last_change":"2024-09-16T09:34:51.181994+0000","last_active":"2024-09-16T09:34:56.627637+0000","last_peered":"2024-09-16T09:34:56.627637+0000","last_clean":"2024-09-16T09:34:56.627637+0000","last_became_active":"2024-09-16T09:31:09.270685+0000","last_became_peered":"2024-09-16T09:31:09.270685+0000","last_unstale":"2024-09-16T09:34:56.627637+0000","last_undegraded":"2024-09-16T09:34:56.627637+0000","last_fullsized":"2024-09-16T09:34:56.627637+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'6","last_scrub_stamp":"2024-09-16T09:34:51.181918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:34:51.181918+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:36:17.849346+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":8,"num_read_kb":5,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":463,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.622559+0000","last_change":"2024-09-16T09:29:31.496374+0000","last_active":"2024-09-16T09:34:56.622559+0000","last_peered":"2024-09-16T09:34:56.622559+0000","last_clean":"2024-09-16T09:34:56.622559+0000","last_became_active":"2024-09-16T09:25:05.715616+0000","last_became_peered":"2024-09-16T09:25:05.715616+0000","last_unstale":"2024-09-16T09:34:56.622559+0000","last_undegraded":"2024-09-16T09:34:56.622559+0000","last_fullsized":"2024-09-16T09:34:56.622559+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:31.496304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:31.496304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":451,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.626473+0000","last_change":"2024-09-16T09:26:28.733381+0000","last_active":"2024-09-16T09:34:56.626473+0000","last_peered":"2024-09-16T09:34:56.626473+0000","last_clean":"2024-09-16T09:34:56.626473+0000","last_became_active":"2024-09-16T09:25:07.721059+0000","last_became_peered":"2024-09-16T09:25:07.721059+0000","last_unstale":"2024-09-16T09:34:56.626473+0000","last_undegraded":"2024-09-16T09:34:56.626473+0000","last_fullsized":"2024-09-16T09:34:56.626473+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:28.733324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:28.733324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.13","version":"73'79","reported_seq":362,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.627158+0000","last_change":"2024-09-16T09:34:48.260121+0000","last_active":"2024-09-16T09:34:56.627158+0000","last_peered":"2024-09-16T09:34:56.627158+0000","last_clean":"2024-09-16T09:34:56.627158+0000","last_became_active":"2024-09-16T09:33:30.248497+0000","last_became_peered":"2024-09-16T09:33:30.248497+0000","last_unstale":"2024-09-16T09:34:56.627158+0000","last_undegraded":"2024-09-16T09:34:56.627158+0000","last_fullsized":"2024-09-16T09:34:56.627158+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:34:48.260041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:34:48.260041+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:35:49.647205+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"66'9","reported_seq":542,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.615275+0000","last_change":"2024-09-16T09:33:48.681742+0000","last_active":"2024-09-16T09:34:56.615275+0000","last_peered":"2024-09-16T09:34:56.615275+0000","last_clean":"2024-09-16T09:34:56.615275+0000","last_became_active":"2024-09-16T09:31:09.273498+0000","last_became_peered":"2024-09-16T09:31:09.273498+0000","last_unstale":"2024-09-16T09:34:56.615275+0000","last_undegraded":"2024-09-16T09:34:56.615275+0000","last_fullsized":"2024-09-16T09:34:56.615275+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:33:48.681650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:33:48.681650+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":479,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.623832+0000","last_change":"2024-09-16T09:33:13.312434+0000","last_active":"2024-09-16T09:34:56.623832+0000","last_peered":"2024-09-16T09:34:56.623832+0000","last_clean":"2024-09-16T09:34:56.623832+0000","last_became_active":"2024-09-16T09:25:01.706491+0000","last_became_peered":"2024-09-16T09:25:01.706491+0000","last_unstale":"2024-09-16T09:34:56.623832+0000","last_undegraded":"2024-09-16T09:34:56.623832+0000","last_fullsized":"2024-09-16T09:34:56.623832+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:13.312344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:13.312344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":228,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.623969+0000","last_change":"2024-09-16T09:33:30.249293+0000","last_active":"2024-09-16T09:34:56.623969+0000","last_peered":"2024-09-16T09:34:56.623969+0000","last_clean":"2024-09-16T09:34:56.623969+0000","last_became_active":"2024-09-16T09:33:30.248992+0000","last_became_peered":"2024-09-16T09:33:30.248992+0000","last_unstale":"2024-09-16T09:34:56.623969+0000","last_undegraded":"2024-09-16T09:34:56.623969+0000","last_fullsized":"2024-09-16T09:34:56.623969+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:53.810580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:28:53.810580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":468,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.620322+0000","last_change":"2024-09-16T09:34:32.309307+0000","last_active":"2024-09-16T09:34:56.620322+0000","last_peered":"2024-09-16T09:34:56.620322+0000","last_clean":"2024-09-16T09:34:56.620322+0000","last_became_active":"2024-09-16T09:25:07.722050+0000","last_became_peered":"2024-09-16T09:25:07.722050+0000","last_unstale":"2024-09-16T09:34:56.620322+0000","last_undegraded":"2024-09-16T09:34:56.620322+0000","last_fullsized":"2024-09-16T09:34:56.620322+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:32.309262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:34:32.309262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:35:52.868735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.18","version":"279'5235","reported_seq":10136,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:57.650041+0000","last_change":"2024-09-16T09:34:50.611371+0000","last_active":"2024-09-16T09:34:57.650041+0000","last_peered":"2024-09-16T09:34:57.650041+0000","last_clean":"2024-09-16T09:34:57.650041+0000","last_became_active":"2024-09-16T09:34:18.560651+0000","last_became_peered":"2024-09-16T09:34:18.560651+0000","last_unstale":"2024-09-16T09:34:57.650041+0000","last_undegraded":"2024-09-16T09:34:57.650041+0000","last_fullsized":"2024-09-16T09:34:57.650041+0000","mapping_epoch":258,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":5,"last_scrub":"275'4947","last_scrub_stamp":"2024-09-16T09:34:50.611186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:34:50.611186+0000","objects_scrubbed":3349,"log_size":5235,"log_dups_size":0,"ondisk_log_size":5235,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:36:03.721069+0000","scrub_duration":773,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":239665152,"num_objects":3657,"num_object_clones":0,"num_object_copies":10971,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3657,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":308,"num_write_kb":19712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,5],"acting":[6,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.10","version":"73'79","reported_seq":269,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.620740+0000","last_change":"2024-09-16T09:32:44.405696+0000","last_active":"2024-09-16T09:34:56.620740+0000","last_peered":"2024-09-16T09:34:56.620740+0000","last_clean":"2024-09-16T09:34:56.620740+0000","last_became_active":"2024-09-16T09:30:10.692770+0000","last_became_peered":"2024-09-16T09:30:10.692770+0000","last_unstale":"2024-09-16T09:34:56.620740+0000","last_undegraded":"2024-09-16T09:34:56.620740+0000","last_fullsized":"2024-09-16T09:34:56.620740+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:32:44.405640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:32:44.405640+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"66'9","reported_seq":483,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.633014+0000","last_change":"2024-09-16T09:29:32.485165+0000","last_active":"2024-09-16T09:34:56.633014+0000","last_peered":"2024-09-16T09:34:56.633014+0000","last_clean":"2024-09-16T09:34:56.633014+0000","last_became_active":"2024-09-16T09:25:03.717310+0000","last_became_peered":"2024-09-16T09:25:03.717310+0000","last_unstale":"2024-09-16T09:34:56.633014+0000","last_undegraded":"2024-09-16T09:34:56.633014+0000","last_fullsized":"2024-09-16T09:34:56.633014+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:29:32.485084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:29:32.485084+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":502,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.615057+0000","last_change":"2024-09-16T09:33:20.639405+0000","last_active":"2024-09-16T09:34:56.615057+0000","last_peered":"2024-09-16T09:34:56.615057+0000","last_clean":"2024-09-16T09:34:56.615057+0000","last_became_active":"2024-09-16T09:31:09.272848+0000","last_became_peered":"2024-09-16T09:31:09.272848+0000","last_unstale":"2024-09-16T09:34:56.615057+0000","last_undegraded":"2024-09-16T09:34:56.615057+0000","last_fullsized":"2024-09-16T09:34:56.615057+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:20.639350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:20.639350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"187'11","reported_seq":436,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.624026+0000","last_change":"2024-09-16T09:33:20.296801+0000","last_active":"2024-09-16T09:34:56.624026+0000","last_peered":"2024-09-16T09:34:56.624026+0000","last_clean":"2024-09-16T09:34:56.624026+0000","last_became_active":"2024-09-16T09:31:09.277688+0000","last_became_peered":"2024-09-16T09:31:09.277688+0000","last_unstale":"2024-09-16T09:34:56.624026+0000","last_undegraded":"2024-09-16T09:34:56.624026+0000","last_fullsized":"2024-09-16T09:34:56.624026+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"187'11","last_scrub_stamp":"2024-09-16T09:33:20.296746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:20.296746+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":468,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.633882+0000","last_change":"2024-09-16T09:29:49.474440+0000","last_active":"2024-09-16T09:34:56.633882+0000","last_peered":"2024-09-16T09:34:56.633882+0000","last_clean":"2024-09-16T09:34:56.633882+0000","last_became_active":"2024-09-16T09:25:07.718810+0000","last_became_peered":"2024-09-16T09:25:07.718810+0000","last_unstale":"2024-09-16T09:34:56.633882+0000","last_undegraded":"2024-09-16T09:34:56.633882+0000","last_fullsized":"2024-09-16T09:34:56.633882+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:49.474391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:49.474391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.19","version":"279'5240","reported_seq":10179,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:58.185623+0000","last_change":"2024-09-16T09:34:31.880681+0000","last_active":"2024-09-16T09:34:58.185623+0000","last_peered":"2024-09-16T09:34:58.185623+0000","last_clean":"2024-09-16T09:34:58.185623+0000","last_became_active":"2024-09-16T09:34:31.426697+0000","last_became_peered":"2024-09-16T09:34:31.426697+0000","last_unstale":"2024-09-16T09:34:58.185623+0000","last_undegraded":"2024-09-16T09:34:58.185623+0000","last_fullsized":"2024-09-16T09:34:58.185623+0000","mapping_epoch":266,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":267,"parent":"0.0","parent_split_bits":5,"last_scrub":"267'4219","last_scrub_stamp":"2024-09-16T09:34:31.880547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:34:31.880547+0000","objects_scrubbed":2634,"log_size":5240,"log_dups_size":0,"ondisk_log_size":5240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:35:48.312703+0000","scrub_duration":405,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":239861760,"num_objects":3660,"num_object_clones":0,"num_object_copies":10980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3660,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1026,"num_write_kb":65664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.11","version":"73'79","reported_seq":355,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.616284+0000","last_change":"2024-09-16T09:32:38.401888+0000","last_active":"2024-09-16T09:34:56.616284+0000","last_peered":"2024-09-16T09:34:56.616284+0000","last_clean":"2024-09-16T09:34:56.616284+0000","last_became_active":"2024-09-16T09:31:09.272208+0000","last_became_peered":"2024-09-16T09:31:09.272208+0000","last_unstale":"2024-09-16T09:34:56.616284+0000","last_undegraded":"2024-09-16T09:34:56.616284+0000","last_fullsized":"2024-09-16T09:34:56.616284+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:32:38.401720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:32:38.401720+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":32,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.14","version":"66'10","reported_seq":138,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.624159+0000","last_change":"2024-09-16T09:33:30.250101+0000","last_active":"2024-09-16T09:34:56.624159+0000","last_peered":"2024-09-16T09:34:56.624159+0000","last_clean":"2024-09-16T09:34:56.624159+0000","last_became_active":"2024-09-16T09:33:30.249769+0000","last_became_peered":"2024-09-16T09:33:30.249769+0000","last_unstale":"2024-09-16T09:34:56.624159+0000","last_undegraded":"2024-09-16T09:34:56.624159+0000","last_fullsized":"2024-09-16T09:34:56.624159+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-16T09:28:56.825476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:56.825476+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":14,"num_read_kb":9,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":471,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.625202+0000","last_change":"2024-09-16T09:29:56.021413+0000","last_active":"2024-09-16T09:34:56.625202+0000","last_peered":"2024-09-16T09:34:56.625202+0000","last_clean":"2024-09-16T09:34:56.625202+0000","last_became_active":"2024-09-16T09:25:01.707513+0000","last_became_peered":"2024-09-16T09:25:01.707513+0000","last_unstale":"2024-09-16T09:34:56.625202+0000","last_undegraded":"2024-09-16T09:34:56.625202+0000","last_fullsized":"2024-09-16T09:34:56.625202+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:56.021340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:56.021340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"66'8","reported_seq":607,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.632931+0000","last_change":"2024-09-16T09:33:39.388721+0000","last_active":"2024-09-16T09:34:56.632931+0000","last_peered":"2024-09-16T09:34:56.632931+0000","last_clean":"2024-09-16T09:34:56.632931+0000","last_became_active":"2024-09-16T09:25:05.713833+0000","last_became_peered":"2024-09-16T09:25:05.713833+0000","last_unstale":"2024-09-16T09:34:56.632931+0000","last_undegraded":"2024-09-16T09:34:56.632931+0000","last_fullsized":"2024-09-16T09:34:56.632931+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:33:39.388463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:39.388463+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":490,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.614922+0000","last_change":"2024-09-16T09:33:42.666290+0000","last_active":"2024-09-16T09:34:56.614922+0000","last_peered":"2024-09-16T09:34:56.614922+0000","last_clean":"2024-09-16T09:34:56.614922+0000","last_became_active":"2024-09-16T09:31:09.275168+0000","last_became_peered":"2024-09-16T09:31:09.275168+0000","last_unstale":"2024-09-16T09:34:56.614922+0000","last_undegraded":"2024-09-16T09:34:56.614922+0000","last_fullsized":"2024-09-16T09:34:56.614922+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:42.666200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:33:42.666200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:35:11.658552+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"66'11","reported_seq":498,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.632422+0000","last_change":"2024-09-16T09:33:19.084118+0000","last_active":"2024-09-16T09:34:56.632422+0000","last_peered":"2024-09-16T09:34:56.632422+0000","last_clean":"2024-09-16T09:34:56.632422+0000","last_became_active":"2024-09-16T09:26:33.291265+0000","last_became_peered":"2024-09-16T09:26:33.291265+0000","last_unstale":"2024-09-16T09:34:56.632422+0000","last_undegraded":"2024-09-16T09:34:56.632422+0000","last_fullsized":"2024-09-16T09:34:56.632422+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-16T09:33:19.084063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:33:19.084063+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":480,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.634789+0000","last_change":"2024-09-16T09:29:51.517112+0000","last_active":"2024-09-16T09:34:56.634789+0000","last_peered":"2024-09-16T09:34:56.634789+0000","last_clean":"2024-09-16T09:34:56.634789+0000","last_became_active":"2024-09-16T09:25:01.705158+0000","last_became_peered":"2024-09-16T09:25:01.705158+0000","last_unstale":"2024-09-16T09:34:56.634789+0000","last_undegraded":"2024-09-16T09:34:56.634789+0000","last_fullsized":"2024-09-16T09:34:56.634789+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:51.517054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:51.517054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":400,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.624221+0000","last_change":"2024-09-16T09:34:00.329968+0000","last_active":"2024-09-16T09:34:56.624221+0000","last_peered":"2024-09-16T09:34:56.624221+0000","last_clean":"2024-09-16T09:34:56.624221+0000","last_became_active":"2024-09-16T09:34:00.329642+0000","last_became_peered":"2024-09-16T09:34:00.329642+0000","last_unstale":"2024-09-16T09:34:56.624221+0000","last_undegraded":"2024-09-16T09:34:56.624221+0000","last_fullsized":"2024-09-16T09:34:56.624221+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:08.394830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:08.394830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":322,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.614981+0000","last_change":"2024-09-16T09:34:48.786847+0000","last_active":"2024-09-16T09:34:56.614981+0000","last_peered":"2024-09-16T09:34:56.614981+0000","last_clean":"2024-09-16T09:34:56.614981+0000","last_became_active":"2024-09-16T09:31:09.276042+0000","last_became_peered":"2024-09-16T09:31:09.276042+0000","last_unstale":"2024-09-16T09:34:56.614981+0000","last_undegraded":"2024-09-16T09:34:56.614981+0000","last_fullsized":"2024-09-16T09:34:56.614981+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:48.786785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:34:48.786785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:36:17.728202+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"66'9","reported_seq":156,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.619353+0000","last_change":"2024-09-16T09:34:15.299311+0000","last_active":"2024-09-16T09:34:56.619353+0000","last_peered":"2024-09-16T09:34:56.619353+0000","last_clean":"2024-09-16T09:34:56.619353+0000","last_became_active":"2024-09-16T09:34:00.331466+0000","last_became_peered":"2024-09-16T09:34:00.331466+0000","last_unstale":"2024-09-16T09:34:56.619353+0000","last_undegraded":"2024-09-16T09:34:56.619353+0000","last_fullsized":"2024-09-16T09:34:56.619353+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:32:47.247071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:32:47.247071+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":10,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,0],"acting":[6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":471,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.627052+0000","last_change":"2024-09-16T09:32:47.502667+0000","last_active":"2024-09-16T09:34:56.627052+0000","last_peered":"2024-09-16T09:34:56.627052+0000","last_clean":"2024-09-16T09:34:56.627052+0000","last_became_active":"2024-09-16T09:25:01.706762+0000","last_became_peered":"2024-09-16T09:25:01.706762+0000","last_unstale":"2024-09-16T09:34:56.627052+0000","last_undegraded":"2024-09-16T09:34:56.627052+0000","last_fullsized":"2024-09-16T09:34:56.627052+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:47.502598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:32:47.502598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":307,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.615297+0000","last_change":"2024-09-16T09:34:46.742640+0000","last_active":"2024-09-16T09:34:56.615297+0000","last_peered":"2024-09-16T09:34:56.615297+0000","last_clean":"2024-09-16T09:34:56.615297+0000","last_became_active":"2024-09-16T09:28:52.982865+0000","last_became_peered":"2024-09-16T09:28:52.982865+0000","last_unstale":"2024-09-16T09:34:56.615297+0000","last_undegraded":"2024-09-16T09:34:56.615297+0000","last_fullsized":"2024-09-16T09:34:56.615297+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:46.742562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:34:46.742562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:35:57.573477+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":468,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.614501+0000","last_change":"2024-09-16T09:34:50.779241+0000","last_active":"2024-09-16T09:34:56.614501+0000","last_peered":"2024-09-16T09:34:56.614501+0000","last_clean":"2024-09-16T09:34:56.614501+0000","last_became_active":"2024-09-16T09:25:07.715301+0000","last_became_peered":"2024-09-16T09:25:07.715301+0000","last_unstale":"2024-09-16T09:34:56.614501+0000","last_undegraded":"2024-09-16T09:34:56.614501+0000","last_fullsized":"2024-09-16T09:34:56.614501+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:50.779164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:34:50.779164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:36:11.307448+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.14","version":"73'79","reported_seq":269,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.620235+0000","last_change":"2024-09-16T09:32:44.163264+0000","last_active":"2024-09-16T09:34:56.620235+0000","last_peered":"2024-09-16T09:34:56.620235+0000","last_clean":"2024-09-16T09:34:56.620235+0000","last_became_active":"2024-09-16T09:30:10.700866+0000","last_became_peered":"2024-09-16T09:30:10.700866+0000","last_unstale":"2024-09-16T09:34:56.620235+0000","last_undegraded":"2024-09-16T09:34:56.620235+0000","last_fullsized":"2024-09-16T09:34:56.620235+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:32:44.163191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:32:44.163191+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:35:15.485909+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"66'11","reported_seq":346,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.623691+0000","last_change":"2024-09-16T09:28:55.643505+0000","last_active":"2024-09-16T09:34:56.623691+0000","last_peered":"2024-09-16T09:34:56.623691+0000","last_clean":"2024-09-16T09:34:56.623691+0000","last_became_active":"2024-09-16T09:28:52.971132+0000","last_became_peered":"2024-09-16T09:28:52.971132+0000","last_unstale":"2024-09-16T09:34:56.623691+0000","last_undegraded":"2024-09-16T09:34:56.623691+0000","last_fullsized":"2024-09-16T09:34:56.623691+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-16T09:28:55.643448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:55.643448+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":19,"num_read_kb":12,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":479,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.622692+0000","last_change":"2024-09-16T09:33:18.297533+0000","last_active":"2024-09-16T09:34:56.622692+0000","last_peered":"2024-09-16T09:34:56.622692+0000","last_clean":"2024-09-16T09:34:56.622692+0000","last_became_active":"2024-09-16T09:25:01.700733+0000","last_became_peered":"2024-09-16T09:25:01.700733+0000","last_unstale":"2024-09-16T09:34:56.622692+0000","last_undegraded":"2024-09-16T09:34:56.622692+0000","last_fullsized":"2024-09-16T09:34:56.622692+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:18.297451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:18.297451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":471,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.621334+0000","last_change":"2024-09-16T09:33:38.336281+0000","last_active":"2024-09-16T09:34:56.621334+0000","last_peered":"2024-09-16T09:34:56.621334+0000","last_clean":"2024-09-16T09:34:56.621334+0000","last_became_active":"2024-09-16T09:25:05.713479+0000","last_became_peered":"2024-09-16T09:25:05.713479+0000","last_unstale":"2024-09-16T09:34:56.621334+0000","last_undegraded":"2024-09-16T09:34:56.621334+0000","last_fullsized":"2024-09-16T09:34:56.621334+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:38.336221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:38.336221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":330,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.615536+0000","last_change":"2024-09-16T09:34:51.781047+0000","last_active":"2024-09-16T09:34:56.615536+0000","last_peered":"2024-09-16T09:34:56.615536+0000","last_clean":"2024-09-16T09:34:56.615536+0000","last_became_active":"2024-09-16T09:31:09.274797+0000","last_became_peered":"2024-09-16T09:31:09.274797+0000","last_unstale":"2024-09-16T09:34:56.615536+0000","last_undegraded":"2024-09-16T09:34:56.615536+0000","last_fullsized":"2024-09-16T09:34:56.615536+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:51.780980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:34:51.780980+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:36:10.448717+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"66'4","reported_seq":219,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.626465+0000","last_change":"2024-09-16T09:34:50.206999+0000","last_active":"2024-09-16T09:34:56.626465+0000","last_peered":"2024-09-16T09:34:56.626465+0000","last_clean":"2024-09-16T09:34:56.626465+0000","last_became_active":"2024-09-16T09:31:09.273682+0000","last_became_peered":"2024-09-16T09:31:09.273682+0000","last_unstale":"2024-09-16T09:34:56.626465+0000","last_undegraded":"2024-09-16T09:34:56.626465+0000","last_fullsized":"2024-09-16T09:34:56.626465+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'4","last_scrub_stamp":"2024-09-16T09:34:50.206916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:34:50.206916+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:35:57.708906+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":502,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.615909+0000","last_change":"2024-09-16T09:33:18.641115+0000","last_active":"2024-09-16T09:34:56.615909+0000","last_peered":"2024-09-16T09:34:56.615909+0000","last_clean":"2024-09-16T09:34:56.615909+0000","last_became_active":"2024-09-16T09:31:09.274719+0000","last_became_peered":"2024-09-16T09:31:09.274719+0000","last_unstale":"2024-09-16T09:34:56.615909+0000","last_undegraded":"2024-09-16T09:34:56.615909+0000","last_fullsized":"2024-09-16T09:34:56.615909+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:18.641069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:18.641069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":488,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.619699+0000","last_change":"2024-09-16T09:33:37.941485+0000","last_active":"2024-09-16T09:34:56.619699+0000","last_peered":"2024-09-16T09:34:56.619699+0000","last_clean":"2024-09-16T09:34:56.619699+0000","last_became_active":"2024-09-16T09:25:05.712545+0000","last_became_peered":"2024-09-16T09:25:05.712545+0000","last_unstale":"2024-09-16T09:34:56.619699+0000","last_undegraded":"2024-09-16T09:34:56.619699+0000","last_fullsized":"2024-09-16T09:34:56.619699+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:37.941403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:37.941403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:35:05.596463+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":467,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.622569+0000","last_change":"2024-09-16T09:33:41.323824+0000","last_active":"2024-09-16T09:34:56.622569+0000","last_peered":"2024-09-16T09:34:56.622569+0000","last_clean":"2024-09-16T09:34:56.622569+0000","last_became_active":"2024-09-16T09:25:07.720117+0000","last_became_peered":"2024-09-16T09:25:07.720117+0000","last_unstale":"2024-09-16T09:34:56.622569+0000","last_undegraded":"2024-09-16T09:34:56.622569+0000","last_fullsized":"2024-09-16T09:34:56.622569+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:41.323773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:33:41.323773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.12","version":"279'5325","reported_seq":10389,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:58.072518+0000","last_change":"2024-09-16T09:34:48.381773+0000","last_active":"2024-09-16T09:34:58.072518+0000","last_peered":"2024-09-16T09:34:58.072518+0000","last_clean":"2024-09-16T09:34:58.072518+0000","last_became_active":"2024-09-16T09:34:16.497116+0000","last_became_peered":"2024-09-16T09:34:16.497116+0000","last_unstale":"2024-09-16T09:34:58.072518+0000","last_undegraded":"2024-09-16T09:34:58.072518+0000","last_fullsized":"2024-09-16T09:34:58.072518+0000","mapping_epoch":256,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":5,"last_scrub":"274'4970","last_scrub_stamp":"2024-09-16T09:34:48.381443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:34:48.381443+0000","objects_scrubbed":3385,"log_size":5325,"log_dups_size":0,"ondisk_log_size":5325,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:35:59.958057+0000","scrub_duration":905,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":246284288,"num_objects":3758,"num_object_clones":0,"num_object_copies":11274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3758,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":373,"num_write_kb":23872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":103,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.632835+0000","last_change":"2024-09-16T09:33:30.248335+0000","last_active":"2024-09-16T09:34:56.632835+0000","last_peered":"2024-09-16T09:34:56.632835+0000","last_clean":"2024-09-16T09:34:56.632835+0000","last_became_active":"2024-09-16T09:33:30.248036+0000","last_became_peered":"2024-09-16T09:33:30.248036+0000","last_unstale":"2024-09-16T09:34:56.632835+0000","last_undegraded":"2024-09-16T09:34:56.632835+0000","last_fullsized":"2024-09-16T09:34:56.632835+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:18.339152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:33:18.339152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":329,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.615055+0000","last_change":"2024-09-16T09:32:40.411546+0000","last_active":"2024-09-16T09:34:56.615055+0000","last_peered":"2024-09-16T09:34:56.615055+0000","last_clean":"2024-09-16T09:34:56.615055+0000","last_became_active":"2024-09-16T09:31:09.271662+0000","last_became_peered":"2024-09-16T09:31:09.271662+0000","last_unstale":"2024-09-16T09:34:56.615055+0000","last_undegraded":"2024-09-16T09:34:56.615055+0000","last_fullsized":"2024-09-16T09:34:56.615055+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:40.411487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:32:40.411487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"66'11","reported_seq":495,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.621146+0000","last_change":"2024-09-16T09:33:19.843216+0000","last_active":"2024-09-16T09:34:56.621146+0000","last_peered":"2024-09-16T09:34:56.621146+0000","last_clean":"2024-09-16T09:34:56.621146+0000","last_became_active":"2024-09-16T09:25:03.707769+0000","last_became_peered":"2024-09-16T09:25:03.707769+0000","last_unstale":"2024-09-16T09:34:56.621146+0000","last_undegraded":"2024-09-16T09:34:56.621146+0000","last_fullsized":"2024-09-16T09:34:56.621146+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-16T09:33:19.843154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:33:19.843154+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":464,"reported_epoch":279,"state":"active+clean","last_fresh":"2024-09-16T09:34:56.615058+0000","last_change":"2024-09-16T09:32:31.325543+0000","last_active":"2024-09-16T09:34:56.615058+0000","last_peered":"2024-09-16T09:34:56.615058+0000","last_clean":"2024-09-16T09:34:56.615058+0000","last_became_active":"2024-09-16T09:25:05.711952+0000","last_became_peered":"2024-09-16T09:25:05.711952+0000","last_unstale":"2024-09-16T09:34:56.615058+0000","last_undegraded":"2024-09-16T09:34:56.615058+0000","last_fullsized":"2024-09-16T09:34:56.615058+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:31.325346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:32:31.325346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":26,"stat_sum":{"num_bytes":7750975488,"num_objects":118271,"num_object_clones":0,"num_object_copies":354813,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1325,"num_objects_unfound":0,"num_objects_dirty":118271,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":86757,"num_write_kb":5552416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5531,"num_bytes_recovered":362479616,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":23903141888,"data_stored":23903141888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":130819,"ondisk_log_size":130819,"up":78,"acting":77,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":688,"num_read_kb":445,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":116,"num_bytes_recovered":2916,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":395,"ondisk_log_size":395,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1986,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":590,"num_read_kb":585,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":21,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":327680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1659,"ondisk_log_size":1659,"up":63,"acting":63,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266812,"num_pgs":79,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":4866180,"kb_used_data":4469876,"kb_used_omap":1,"kb_used_meta":396286,"kb_avail":88878972,"statfs":{"total":95995035648,"available":91012067328,"internally_reserved":0,"allocated":4577153024,"data_stored":4576527878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":405797322},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495745,"num_pgs":61,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2200940,"kb_used_data":1903188,"kb_used_omap":1,"kb_used_meta":297726,"kb_avail":91544212,"statfs":{"total":95995035648,"available":93741273088,"internally_reserved":0,"allocated":1948864512,"data_stored":1948271250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":304871878},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724680,"num_pgs":69,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":4336940,"kb_used_data":3948952,"kb_used_omap":1,"kb_used_meta":387966,"kb_avail":89408212,"statfs":{"total":95995035648,"available":91554009088,"internally_reserved":0,"allocated":4043726848,"data_stored":4043129415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":397277638},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,1,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953613,"num_pgs":73,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2669884,"kb_used_data":2331884,"kb_used_omap":1,"kb_used_meta":337982,"kb_avail":91075268,"statfs":{"total":95995035648,"available":93261074432,"internally_reserved":0,"allocated":2387849216,"data_stored":2387231996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":346094022},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149843,"num_pgs":77,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3553364,"kb_used_data":3052924,"kb_used_omap":1,"kb_used_meta":500414,"kb_avail":90191788,"statfs":{"total":95995035648,"available":92356390912,"internally_reserved":0,"allocated":3126194176,"data_stored":3125561512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":512424391},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411480,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1283880,"kb_used_data":945872,"kb_used_omap":1,"kb_used_meta":337982,"kb_avail":92461272,"statfs":{"total":95995035648,"available":94680342528,"internally_reserved":0,"allocated":968572928,"data_stored":967983584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":346094021},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575007,"num_pgs":74,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":4998668,"kb_used_data":4516276,"kb_used_omap":1,"kb_used_meta":482366,"kb_avail":88746484,"statfs":{"total":95995035648,"available":90876399616,"internally_reserved":0,"allocated":4624666624,"data_stored":4624040642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":493943237},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738533,"num_pgs":64,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2588736,"kb_used_data":2188272,"kb_used_omap":1,"kb_used_meta":400446,"kb_avail":91156416,"statfs":{"total":95995035648,"available":93344169984,"internally_reserved":0,"allocated":2240790528,"data_stored":2240169524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":410057157},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2212691968,"data_stored":2212691968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4623302656,"data_stored":4623302656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":972357632,"data_stored":972357632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":3155951616,"data_stored":3155951616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":2335834112,"data_stored":2335834112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4051173376,"data_stored":4051173376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1964638208,"data_stored":1964638208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4587192320,"data_stored":4587192320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T09:35:01.821 INFO:tasks.cephadm.ceph_manager.ceph:decrease pool size by 10 2024-09-16T09:35:01.822 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd pool set unique_pool_2 pg_num 16 2024-09-16T09:35:01.823 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-09-16T09:35:01.823 INFO:teuthology.orchestra.run.smithi084.stdout: "size": 20, 2024-09-16T09:35:01.823 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 600, 2024-09-16T09:35:01.823 INFO:teuthology.orchestra.run.smithi084.stdout: "ops": [ 2024-09-16T09:35:01.823 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.823 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.25064.0:170549 8.b 8:d1a36449:::benchmark_data_smithi070_107919_object45954:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e204)", 2024-09-16T09:35:01.823 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:32:07.438095+0000", 2024-09-16T09:35:01.824 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 174.37257263699999, 2024-09-16T09:35:01.824 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.004960772, 2024-09-16T09:35:01.824 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:35:01.824 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:35:01.824 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:35:01.824 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:35:01.824 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.25064", 2024-09-16T09:35:01.824 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/2578256439", 2024-09-16T09:35:01.824 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 170549 2024-09-16T09:35:01.824 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.824 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:35:01.824 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.824 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:35:01.824 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.438095+0000", 2024-09-16T09:35:01.825 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:35:01.825 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.825 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.825 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:35:01.825 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.438095+0000", 2024-09-16T09:35:01.825 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:35:01.825 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.825 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.825 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:35:01.825 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.438096+0000", 2024-09-16T09:35:01.825 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.7370000000000001e-06 2024-09-16T09:35:01.825 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.825 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.825 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:35:01.825 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.438103+0000", 2024-09-16T09:35:01.826 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.3670000000000002e-06 2024-09-16T09:35:01.826 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.826 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.826 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:35:01.826 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.438119+0000", 2024-09-16T09:35:01.826 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.6084999999999999e-05 2024-09-16T09:35:01.826 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.826 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.826 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:35:01.826 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.438126+0000", 2024-09-16T09:35:01.827 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.7349999999999999e-06 2024-09-16T09:35:01.827 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.827 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.827 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:35:01.827 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.438179+0000", 2024-09-16T09:35:01.827 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.2961000000000003e-05 2024-09-16T09:35:01.827 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.827 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.827 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:35:01.827 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.438187+0000", 2024-09-16T09:35:01.827 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.1890000000000007e-06 2024-09-16T09:35:01.827 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.827 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.827 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:35:01.827 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.436726+0000", 2024-09-16T09:35:01.828 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.99853915599999998 2024-09-16T09:35:01.828 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.828 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.828 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:35:01.828 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.436844+0000", 2024-09-16T09:35:01.828 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000118089 2024-09-16T09:35:01.828 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.828 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.828 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 1,3", 2024-09-16T09:35:01.828 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.436976+0000", 2024-09-16T09:35:01.829 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00013196700000000001 2024-09-16T09:35:01.829 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.829 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.829 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:35:01.829 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.442912+0000", 2024-09-16T09:35:01.829 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0059363970000000004 2024-09-16T09:35:01.829 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.829 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.829 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:35:01.829 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.442954+0000", 2024-09-16T09:35:01.829 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.1761000000000002e-05 2024-09-16T09:35:01.829 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.829 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.829 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:35:01.830 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.442990+0000", 2024-09-16T09:35:01.830 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.6091999999999999e-05 2024-09-16T09:35:01.830 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.830 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.830 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:35:01.830 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.443029+0000", 2024-09-16T09:35:01.830 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.9186000000000001e-05 2024-09-16T09:35:01.830 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.830 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.830 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:35:01.830 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.443055+0000", 2024-09-16T09:35:01.830 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.605e-05 2024-09-16T09:35:01.830 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:35:01.830 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:35:01.830 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:35:01.831 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.831 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.831 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.25064.0:170577 8.b 8:d5966a91:::benchmark_data_smithi070_107919_object45982:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e204)", 2024-09-16T09:35:01.831 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:32:07.478909+0000", 2024-09-16T09:35:01.831 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 174.331757759, 2024-09-16T09:35:01.831 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.96429471700000002, 2024-09-16T09:35:01.831 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:35:01.831 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:35:01.831 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:35:01.832 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:35:01.832 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.25064", 2024-09-16T09:35:01.832 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/2578256439", 2024-09-16T09:35:01.832 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 170577 2024-09-16T09:35:01.832 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.832 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:35:01.832 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.832 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:35:01.832 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.478909+0000", 2024-09-16T09:35:01.832 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:35:01.832 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.832 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.832 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:35:01.832 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.478909+0000", 2024-09-16T09:35:01.832 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:35:01.833 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.833 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.833 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:35:01.833 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.478911+0000", 2024-09-16T09:35:01.833 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.8759999999999999e-06 2024-09-16T09:35:01.833 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.833 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.833 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:35:01.833 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.478918+0000", 2024-09-16T09:35:01.833 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.6590000000000001e-06 2024-09-16T09:35:01.833 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.833 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.833 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:35:01.833 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.478936+0000", 2024-09-16T09:35:01.834 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.827e-05 2024-09-16T09:35:01.834 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.834 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.834 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:35:01.834 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.478945+0000", 2024-09-16T09:35:01.834 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.2139999999999996e-06 2024-09-16T09:35:01.834 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.834 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.834 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:35:01.834 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.479008+0000", 2024-09-16T09:35:01.834 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.3459000000000003e-05 2024-09-16T09:35:01.834 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.834 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.834 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:35:01.834 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.479032+0000", 2024-09-16T09:35:01.835 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.3918e-05 2024-09-16T09:35:01.835 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.835 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.835 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:35:01.835 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.439436+0000", 2024-09-16T09:35:01.835 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.96040375899999997 2024-09-16T09:35:01.835 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.835 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.835 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:35:01.835 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.439545+0000", 2024-09-16T09:35:01.835 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000109576 2024-09-16T09:35:01.835 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.835 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.835 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 1,3", 2024-09-16T09:35:01.835 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.439667+0000", 2024-09-16T09:35:01.836 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00012150999999999999 2024-09-16T09:35:01.836 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.836 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.836 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:35:01.836 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.443079+0000", 2024-09-16T09:35:01.836 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0034118569999999999 2024-09-16T09:35:01.836 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.836 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.836 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:35:01.836 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.443131+0000", 2024-09-16T09:35:01.837 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.2565000000000002e-05 2024-09-16T09:35:01.837 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.837 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.837 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:35:01.837 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.443149+0000", 2024-09-16T09:35:01.837 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.7343999999999999e-05 2024-09-16T09:35:01.837 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.837 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.837 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:35:01.837 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.443177+0000", 2024-09-16T09:35:01.837 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.8654e-05 2024-09-16T09:35:01.837 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.837 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.838 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:35:01.838 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.443204+0000", 2024-09-16T09:35:01.838 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.7056e-05 2024-09-16T09:35:01.838 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:35:01.838 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:35:01.838 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:35:01.838 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.838 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.838 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.25064.0:170589 8.b 8:d7b2d79b:::benchmark_data_smithi070_107919_object45994:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e204)", 2024-09-16T09:35:01.838 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:32:07.486872+0000", 2024-09-16T09:35:01.838 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 174.32379481500001, 2024-09-16T09:35:01.838 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.97349502600000004, 2024-09-16T09:35:01.838 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:35:01.838 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:35:01.839 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:35:01.839 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:35:01.839 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.25064", 2024-09-16T09:35:01.839 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/2578256439", 2024-09-16T09:35:01.839 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 170589 2024-09-16T09:35:01.839 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.839 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:35:01.839 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.839 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:35:01.839 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.486872+0000", 2024-09-16T09:35:01.839 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:35:01.839 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.839 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.839 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:35:01.840 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.486872+0000", 2024-09-16T09:35:01.840 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:35:01.840 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.840 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.840 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:35:01.840 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.486874+0000", 2024-09-16T09:35:01.840 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.6300000000000001e-06 2024-09-16T09:35:01.840 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.840 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.840 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:35:01.840 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.486879+0000", 2024-09-16T09:35:01.840 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.1320000000000002e-06 2024-09-16T09:35:01.840 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.840 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.841 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:35:01.841 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.486895+0000", 2024-09-16T09:35:01.841 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.5719999999999999e-05 2024-09-16T09:35:01.841 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.841 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.841 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:35:01.841 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.486924+0000", 2024-09-16T09:35:01.841 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.8742e-05 2024-09-16T09:35:01.841 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.841 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.841 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:35:01.841 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.486985+0000", 2024-09-16T09:35:01.841 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.1099999999999994e-05 2024-09-16T09:35:01.842 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.842 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.842 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:35:01.842 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.486995+0000", 2024-09-16T09:35:01.842 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0538e-05 2024-09-16T09:35:01.842 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.842 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.842 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:35:01.842 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.441464+0000", 2024-09-16T09:35:01.842 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.95446911700000003 2024-09-16T09:35:01.842 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.842 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.842 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:35:01.842 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.441585+0000", 2024-09-16T09:35:01.842 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000120128 2024-09-16T09:35:01.843 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.843 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.843 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 1,3", 2024-09-16T09:35:01.843 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.441713+0000", 2024-09-16T09:35:01.843 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00012807699999999999 2024-09-16T09:35:01.843 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.843 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.843 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:35:01.843 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.444780+0000", 2024-09-16T09:35:01.843 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0030674370000000001 2024-09-16T09:35:01.843 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.843 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.843 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:35:01.843 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.448814+0000", 2024-09-16T09:35:01.844 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0040338570000000001 2024-09-16T09:35:01.844 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.844 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.844 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:35:01.844 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.460268+0000", 2024-09-16T09:35:01.844 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.011454097 2024-09-16T09:35:01.844 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.844 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.844 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:35:01.844 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.460321+0000", 2024-09-16T09:35:01.844 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.2630000000000003e-05 2024-09-16T09:35:01.844 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.844 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.844 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:35:01.845 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.460367+0000", 2024-09-16T09:35:01.845 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.6820999999999998e-05 2024-09-16T09:35:01.845 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:35:01.845 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:35:01.845 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:35:01.845 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.845 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.845 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.25064.0:170602 8.b 8:d9b50ae1:::benchmark_data_smithi070_107919_object46007:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e204)", 2024-09-16T09:35:01.845 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:32:07.497614+0000", 2024-09-16T09:35:01.845 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 174.313052876, 2024-09-16T09:35:01.845 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.96754656299999997, 2024-09-16T09:35:01.845 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:35:01.845 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:35:01.845 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:35:01.845 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:35:01.846 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.25064", 2024-09-16T09:35:01.846 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/2578256439", 2024-09-16T09:35:01.846 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 170602 2024-09-16T09:35:01.846 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.846 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:35:01.846 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.846 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:35:01.846 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.497614+0000", 2024-09-16T09:35:01.846 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:35:01.846 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.846 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.846 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:35:01.846 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.497614+0000", 2024-09-16T09:35:01.847 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:35:01.847 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.847 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.847 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:35:01.847 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.497616+0000", 2024-09-16T09:35:01.847 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.1050000000000002e-06 2024-09-16T09:35:01.847 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.847 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.847 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:35:01.847 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.497624+0000", 2024-09-16T09:35:01.847 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.0840000000000003e-06 2024-09-16T09:35:01.847 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.847 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.847 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:35:01.848 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.497645+0000", 2024-09-16T09:35:01.848 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.1508000000000001e-05 2024-09-16T09:35:01.848 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.848 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.848 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:35:01.848 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.497655+0000", 2024-09-16T09:35:01.848 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.9879999999999999e-06 2024-09-16T09:35:01.848 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.848 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.848 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:35:01.848 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.497728+0000", 2024-09-16T09:35:01.848 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.2469e-05 2024-09-16T09:35:01.848 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.848 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.848 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:35:01.849 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.497739+0000", 2024-09-16T09:35:01.849 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1053000000000001e-05 2024-09-16T09:35:01.849 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.849 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.849 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:35:01.849 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.443231+0000", 2024-09-16T09:35:01.849 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.94549259399999996 2024-09-16T09:35:01.849 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.849 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.849 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:35:01.849 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.443292+0000", 2024-09-16T09:35:01.849 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.0851999999999999e-05 2024-09-16T09:35:01.849 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.849 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.850 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 1,3", 2024-09-16T09:35:01.850 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.443352+0000", 2024-09-16T09:35:01.850 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.9738999999999998e-05 2024-09-16T09:35:01.850 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.850 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.850 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:35:01.850 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.448824+0000", 2024-09-16T09:35:01.850 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0054719779999999997 2024-09-16T09:35:01.850 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.850 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.850 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:35:01.850 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.451707+0000", 2024-09-16T09:35:01.850 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0028830180000000002 2024-09-16T09:35:01.850 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.850 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.850 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:35:01.851 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.465097+0000", 2024-09-16T09:35:01.851 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.013389961000000001 2024-09-16T09:35:01.851 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.851 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.851 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:35:01.851 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.465131+0000", 2024-09-16T09:35:01.851 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.4198e-05 2024-09-16T09:35:01.851 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.851 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.851 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:35:01.851 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.465161+0000", 2024-09-16T09:35:01.851 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.0015999999999999e-05 2024-09-16T09:35:01.851 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:35:01.851 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:35:01.852 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:35:01.852 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.852 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.852 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.25064.0:170610 8.b 8:daf1a32b:::benchmark_data_smithi070_107919_object46015:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e204)", 2024-09-16T09:35:01.852 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:32:07.504240+0000", 2024-09-16T09:35:01.852 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 174.30642764800001, 2024-09-16T09:35:01.852 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.96107343000000001, 2024-09-16T09:35:01.852 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:35:01.852 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:35:01.852 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:35:01.852 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:35:01.852 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.25064", 2024-09-16T09:35:01.852 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/2578256439", 2024-09-16T09:35:01.852 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 170610 2024-09-16T09:35:01.853 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.853 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:35:01.853 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.853 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:35:01.853 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.504240+0000", 2024-09-16T09:35:01.853 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:35:01.853 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.853 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.853 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:35:01.853 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.504240+0000", 2024-09-16T09:35:01.853 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:35:01.853 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.853 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.853 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:35:01.853 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.504240+0000", 2024-09-16T09:35:01.854 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.5099999999999999e-07 2024-09-16T09:35:01.854 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.854 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.854 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:35:01.854 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.504244+0000", 2024-09-16T09:35:01.854 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.4750000000000002e-06 2024-09-16T09:35:01.854 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.854 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.854 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:35:01.854 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.504253+0000", 2024-09-16T09:35:01.854 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.2110000000000004e-06 2024-09-16T09:35:01.854 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.854 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.854 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:35:01.854 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.504259+0000", 2024-09-16T09:35:01.855 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.003e-06 2024-09-16T09:35:01.855 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.855 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.855 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:35:01.855 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.505205+0000", 2024-09-16T09:35:01.855 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000945793 2024-09-16T09:35:01.855 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.855 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.855 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:35:01.855 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.505221+0000", 2024-09-16T09:35:01.855 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.6079999999999999e-05 2024-09-16T09:35:01.855 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.855 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.855 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:35:01.855 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.444807+0000", 2024-09-16T09:35:01.856 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.93958567500000001 2024-09-16T09:35:01.856 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.856 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.856 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:35:01.856 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.444917+0000", 2024-09-16T09:35:01.856 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000110856 2024-09-16T09:35:01.856 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.856 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.856 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 1,3", 2024-09-16T09:35:01.856 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.445032+0000", 2024-09-16T09:35:01.856 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000114125 2024-09-16T09:35:01.856 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.856 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.856 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:35:01.857 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.451674+0000", 2024-09-16T09:35:01.857 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0066422909999999998 2024-09-16T09:35:01.857 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.857 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.857 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:35:01.857 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.460207+0000", 2024-09-16T09:35:01.857 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0085334610000000009 2024-09-16T09:35:01.857 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.857 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.857 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:35:01.857 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.465215+0000", 2024-09-16T09:35:01.857 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0050073909999999999 2024-09-16T09:35:01.857 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.857 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.857 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:35:01.858 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.465267+0000", 2024-09-16T09:35:01.858 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.2315000000000003e-05 2024-09-16T09:35:01.858 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.858 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.858 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:35:01.858 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.465313+0000", 2024-09-16T09:35:01.858 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.6003e-05 2024-09-16T09:35:01.858 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:35:01.858 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:35:01.858 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:35:01.858 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.858 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.858 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.25064.0:170643 8.b 8:d543baed:::benchmark_data_smithi070_107919_object46048:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e204)", 2024-09-16T09:35:01.858 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:32:07.532952+0000", 2024-09-16T09:35:01.859 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 174.27771486399999, 2024-09-16T09:35:01.859 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.93245734999999996, 2024-09-16T09:35:01.859 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:35:01.859 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:35:01.859 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:35:01.859 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:35:01.859 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.25064", 2024-09-16T09:35:01.859 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/2578256439", 2024-09-16T09:35:01.859 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 170643 2024-09-16T09:35:01.859 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.859 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:35:01.859 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.859 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:35:01.859 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.532952+0000", 2024-09-16T09:35:01.859 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:35:01.860 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.860 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.860 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:35:01.860 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.532952+0000", 2024-09-16T09:35:01.860 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:35:01.860 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.860 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.860 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:35:01.860 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.532954+0000", 2024-09-16T09:35:01.860 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.79e-06 2024-09-16T09:35:01.860 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.860 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.860 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:35:01.860 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.532960+0000", 2024-09-16T09:35:01.861 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.9950000000000002e-06 2024-09-16T09:35:01.861 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.861 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.861 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:35:01.861 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.532976+0000", 2024-09-16T09:35:01.861 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.6243e-05 2024-09-16T09:35:01.861 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.861 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.861 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:35:01.861 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.532984+0000", 2024-09-16T09:35:01.861 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.1740000000000003e-06 2024-09-16T09:35:01.861 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.861 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.861 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:35:01.861 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.533052+0000", 2024-09-16T09:35:01.862 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.8814999999999998e-05 2024-09-16T09:35:01.862 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.862 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.862 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:35:01.862 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.533062+0000", 2024-09-16T09:35:01.862 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.2720000000000003e-06 2024-09-16T09:35:01.862 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.862 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.862 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:35:01.862 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.446896+0000", 2024-09-16T09:35:01.862 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.91383476600000002 2024-09-16T09:35:01.862 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.862 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.862 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:35:01.862 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.447014+0000", 2024-09-16T09:35:01.863 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000117649 2024-09-16T09:35:01.863 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.863 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.863 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 1,3", 2024-09-16T09:35:01.863 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.447092+0000", 2024-09-16T09:35:01.863 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.7888999999999999e-05 2024-09-16T09:35:01.863 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.863 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.863 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:35:01.863 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.452731+0000", 2024-09-16T09:35:01.863 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0056386359999999998 2024-09-16T09:35:01.863 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.863 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.863 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:35:01.863 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.461994+0000", 2024-09-16T09:35:01.864 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.009263149 2024-09-16T09:35:01.864 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.864 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.864 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:35:01.864 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.465360+0000", 2024-09-16T09:35:01.864 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:35:01.866 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0033663159999999998 2024-09-16T09:35:01.866 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.866 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.866 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:35:01.866 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.465388+0000", 2024-09-16T09:35:01.866 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.7433e-05 2024-09-16T09:35:01.866 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.867 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.867 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:35:01.867 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.465410+0000", 2024-09-16T09:35:01.867 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.2223000000000001e-05 2024-09-16T09:35:01.867 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:35:01.867 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:35:01.867 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:35:01.867 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.867 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.867 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.25064.0:170647 8.b 8:dc89519e:::benchmark_data_smithi070_107919_object46052:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e204)", 2024-09-16T09:35:01.867 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:32:07.536170+0000", 2024-09-16T09:35:01.867 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 174.27449688300001, 2024-09-16T09:35:01.867 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.92937628299999997, 2024-09-16T09:35:01.868 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:35:01.868 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:35:01.868 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:35:01.868 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:35:01.868 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.25064", 2024-09-16T09:35:01.868 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/2578256439", 2024-09-16T09:35:01.868 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 170647 2024-09-16T09:35:01.868 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.868 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:35:01.868 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.868 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:35:01.868 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.536170+0000", 2024-09-16T09:35:01.868 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:35:01.869 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.869 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.869 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:35:01.869 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.536170+0000", 2024-09-16T09:35:01.869 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:35:01.869 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.869 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.869 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:35:01.869 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.536172+0000", 2024-09-16T09:35:01.869 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.525e-06 2024-09-16T09:35:01.869 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.869 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.869 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:35:01.869 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.536178+0000", 2024-09-16T09:35:01.870 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.6500000000000001e-06 2024-09-16T09:35:01.870 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.870 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.870 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:35:01.870 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.536192+0000", 2024-09-16T09:35:01.870 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.4374e-05 2024-09-16T09:35:01.870 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.870 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.870 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:35:01.870 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.536199+0000", 2024-09-16T09:35:01.870 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.1289999999999998e-06 2024-09-16T09:35:01.870 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.870 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.871 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:35:01.871 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.536257+0000", 2024-09-16T09:35:01.871 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.8238e-05 2024-09-16T09:35:01.871 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.871 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.871 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:35:01.871 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.536266+0000", 2024-09-16T09:35:01.871 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.7029999999999995e-06 2024-09-16T09:35:01.871 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.871 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.871 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:35:01.871 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.448851+0000", 2024-09-16T09:35:01.871 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.91258534400000002 2024-09-16T09:35:01.871 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.872 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.872 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:35:01.872 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.448940+0000", 2024-09-16T09:35:01.872 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.8317000000000002e-05 2024-09-16T09:35:01.872 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.872 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.872 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 1,3", 2024-09-16T09:35:01.872 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.449011+0000", 2024-09-16T09:35:01.872 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.1390000000000006e-05 2024-09-16T09:35:01.872 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.872 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.872 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:35:01.872 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.452738+0000", 2024-09-16T09:35:01.872 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0037269479999999999 2024-09-16T09:35:01.872 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.873 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.873 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:35:01.873 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.462063+0000", 2024-09-16T09:35:01.873 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0093253020000000006 2024-09-16T09:35:01.873 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.873 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.873 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:35:01.873 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.465456+0000", 2024-09-16T09:35:01.873 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0033928539999999998 2024-09-16T09:35:01.873 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.873 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.873 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:35:01.873 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.465506+0000", 2024-09-16T09:35:01.873 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.0167999999999998e-05 2024-09-16T09:35:01.874 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.874 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.874 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:35:01.874 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.465547+0000", 2024-09-16T09:35:01.874 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.0340999999999999e-05 2024-09-16T09:35:01.874 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:35:01.874 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:35:01.874 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:35:01.874 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.874 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.874 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.25064.0:170683 8.b 8:dd769891:::benchmark_data_smithi070_107919_object46088:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e204)", 2024-09-16T09:35:01.874 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:32:07.569822+0000", 2024-09-16T09:35:01.874 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 174.240844767, 2024-09-16T09:35:01.874 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.89580738500000001, 2024-09-16T09:35:01.874 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:35:01.875 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:35:01.875 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:35:01.875 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:35:01.875 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.25064", 2024-09-16T09:35:01.875 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/2578256439", 2024-09-16T09:35:01.875 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 170683 2024-09-16T09:35:01.875 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.875 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:35:01.875 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.875 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:35:01.875 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.569822+0000", 2024-09-16T09:35:01.875 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:35:01.875 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.875 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.876 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:35:01.876 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.569822+0000", 2024-09-16T09:35:01.876 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:35:01.876 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.876 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.876 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:35:01.876 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.569823+0000", 2024-09-16T09:35:01.876 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.2900000000000004e-07 2024-09-16T09:35:01.876 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.876 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.876 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:35:01.876 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.569826+0000", 2024-09-16T09:35:01.876 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.65e-06 2024-09-16T09:35:01.876 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.876 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.877 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:35:01.877 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.569834+0000", 2024-09-16T09:35:01.877 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.1610000000000003e-06 2024-09-16T09:35:01.877 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.877 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.877 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:35:01.877 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.569839+0000", 2024-09-16T09:35:01.877 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.2480000000000004e-06 2024-09-16T09:35:01.877 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.877 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.877 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:35:01.877 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.569896+0000", 2024-09-16T09:35:01.877 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.7076e-05 2024-09-16T09:35:01.877 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.878 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.878 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:35:01.878 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.569914+0000", 2024-09-16T09:35:01.878 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.8151e-05 2024-09-16T09:35:01.878 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.878 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.878 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:35:01.878 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.450541+0000", 2024-09-16T09:35:01.878 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.88062674299999999 2024-09-16T09:35:01.878 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.878 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.878 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:35:01.878 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.450607+0000", 2024-09-16T09:35:01.878 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.6024000000000003e-05 2024-09-16T09:35:01.878 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.879 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.879 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 1,3", 2024-09-16T09:35:01.879 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.450672+0000", 2024-09-16T09:35:01.879 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.4826999999999994e-05 2024-09-16T09:35:01.879 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.879 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.879 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:35:01.879 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.457678+0000", 2024-09-16T09:35:01.879 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0070063829999999997 2024-09-16T09:35:01.879 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.879 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.879 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:35:01.879 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.463607+0000", 2024-09-16T09:35:01.879 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.005928427 2024-09-16T09:35:01.879 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.880 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.880 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:35:01.880 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.465583+0000", 2024-09-16T09:35:01.880 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0019759019999999999 2024-09-16T09:35:01.880 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.880 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.880 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:35:01.880 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.465610+0000", 2024-09-16T09:35:01.880 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.7498e-05 2024-09-16T09:35:01.880 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.880 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.881 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:35:01.881 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.465630+0000", 2024-09-16T09:35:01.881 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.9766e-05 2024-09-16T09:35:01.881 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:35:01.881 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:35:01.881 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:35:01.881 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.881 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.881 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.25064.0:170703 8.b 8:db7ca685:::benchmark_data_smithi070_107919_object46108:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e204)", 2024-09-16T09:35:01.881 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:32:07.597028+0000", 2024-09-16T09:35:01.881 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 174.21363930800001, 2024-09-16T09:35:01.881 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.87521025299999999, 2024-09-16T09:35:01.881 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:35:01.881 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:35:01.881 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:35:01.882 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:35:01.882 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.25064", 2024-09-16T09:35:01.882 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/2578256439", 2024-09-16T09:35:01.882 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 170703 2024-09-16T09:35:01.882 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.882 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:35:01.882 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.882 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:35:01.882 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.597028+0000", 2024-09-16T09:35:01.882 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:35:01.882 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.882 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.882 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:35:01.882 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.597028+0000", 2024-09-16T09:35:01.883 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:35:01.883 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.883 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.883 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:35:01.883 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.597030+0000", 2024-09-16T09:35:01.883 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.64e-06 2024-09-16T09:35:01.883 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.883 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.883 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:35:01.883 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.597036+0000", 2024-09-16T09:35:01.883 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.7089999999999996e-06 2024-09-16T09:35:01.883 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.883 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.883 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:35:01.884 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.597049+0000", 2024-09-16T09:35:01.884 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.3098000000000001e-05 2024-09-16T09:35:01.884 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.884 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.884 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:35:01.884 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.597058+0000", 2024-09-16T09:35:01.884 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.1110000000000008e-06 2024-09-16T09:35:01.884 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.884 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.884 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:35:01.884 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.597131+0000", 2024-09-16T09:35:01.884 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.3113000000000003e-05 2024-09-16T09:35:01.884 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.884 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.885 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:35:01.885 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.597142+0000", 2024-09-16T09:35:01.885 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1875000000000001e-05 2024-09-16T09:35:01.885 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.885 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.885 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:35:01.885 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.451730+0000", 2024-09-16T09:35:01.885 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.85458758000000001 2024-09-16T09:35:01.885 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.885 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.885 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:35:01.885 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.451807+0000", 2024-09-16T09:35:01.885 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.6444000000000002e-05 2024-09-16T09:35:01.885 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.885 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.886 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 1,3", 2024-09-16T09:35:01.886 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.451879+0000", 2024-09-16T09:35:01.886 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.2731999999999994e-05 2024-09-16T09:35:01.886 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.886 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.886 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:35:01.886 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.460144+0000", 2024-09-16T09:35:01.886 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0082645380000000001 2024-09-16T09:35:01.886 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.886 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.886 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:35:01.886 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.469010+0000", 2024-09-16T09:35:01.886 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0088663939999999997 2024-09-16T09:35:01.886 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.887 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.887 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:35:01.887 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.472167+0000", 2024-09-16T09:35:01.887 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0031565080000000001 2024-09-16T09:35:01.887 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.887 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.887 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:35:01.887 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.472205+0000", 2024-09-16T09:35:01.887 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.8371000000000002e-05 2024-09-16T09:35:01.887 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.887 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.887 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:35:01.887 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.472238+0000", 2024-09-16T09:35:01.887 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.3139999999999998e-05 2024-09-16T09:35:01.887 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:35:01.888 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:35:01.888 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:35:01.888 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.888 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.888 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.25064.0:170704 8.b 8:da444018:::benchmark_data_smithi070_107919_object46109:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e204)", 2024-09-16T09:35:01.888 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:32:07.597673+0000", 2024-09-16T09:35:01.888 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 174.212994235, 2024-09-16T09:35:01.888 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.87687375599999995, 2024-09-16T09:35:01.888 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:35:01.888 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:35:01.888 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:35:01.888 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:35:01.888 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.25064", 2024-09-16T09:35:01.888 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/2578256439", 2024-09-16T09:35:01.889 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 170704 2024-09-16T09:35:01.889 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.889 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:35:01.889 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.889 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:35:01.889 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.597673+0000", 2024-09-16T09:35:01.889 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:35:01.889 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.889 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.889 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:35:01.889 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.597673+0000", 2024-09-16T09:35:01.889 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:35:01.889 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.889 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.889 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:35:01.890 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.597674+0000", 2024-09-16T09:35:01.890 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.6799999999999999e-07 2024-09-16T09:35:01.890 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.890 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.890 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:35:01.890 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.597678+0000", 2024-09-16T09:35:01.890 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.4109999999999998e-06 2024-09-16T09:35:01.890 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.890 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.890 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:35:01.890 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.597693+0000", 2024-09-16T09:35:01.890 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.4775e-05 2024-09-16T09:35:01.890 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.890 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.890 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:35:01.891 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.597701+0000", 2024-09-16T09:35:01.891 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.0330000000000008e-06 2024-09-16T09:35:01.891 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.891 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.891 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:35:01.891 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.597765+0000", 2024-09-16T09:35:01.891 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.3764999999999997e-05 2024-09-16T09:35:01.891 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.891 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.891 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:35:01.891 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.597773+0000", 2024-09-16T09:35:01.891 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.5650000000000002e-06 2024-09-16T09:35:01.891 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.891 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.891 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:35:01.892 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.452768+0000", 2024-09-16T09:35:01.892 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.85499473199999998 2024-09-16T09:35:01.892 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.892 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.892 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:35:01.892 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.452931+0000", 2024-09-16T09:35:01.892 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00016284700000000001 2024-09-16T09:35:01.892 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.892 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.892 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 1,3", 2024-09-16T09:35:01.892 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.453066+0000", 2024-09-16T09:35:01.892 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000135179 2024-09-16T09:35:01.892 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.892 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.893 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:35:01.893 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.460151+0000", 2024-09-16T09:35:01.893 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0070853280000000001 2024-09-16T09:35:01.893 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.893 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.893 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:35:01.893 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.469069+0000", 2024-09-16T09:35:01.893 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0089178130000000001 2024-09-16T09:35:01.893 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.893 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.893 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:35:01.893 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.474461+0000", 2024-09-16T09:35:01.893 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0053922019999999996 2024-09-16T09:35:01.893 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.894 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.894 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:35:01.894 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.474508+0000", 2024-09-16T09:35:01.894 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.6147999999999999e-05 2024-09-16T09:35:01.894 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.894 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.894 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:35:01.894 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.474547+0000", 2024-09-16T09:35:01.894 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.9190000000000003e-05 2024-09-16T09:35:01.894 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:35:01.894 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:35:01.894 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:35:01.894 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.894 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.895 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.25064.0:170710 8.b 8:d492cf9c:::benchmark_data_smithi070_107919_object46115:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e204)", 2024-09-16T09:35:01.895 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:32:07.603557+0000", 2024-09-16T09:35:01.895 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 174.207109844, 2024-09-16T09:35:01.895 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.87113101000000004, 2024-09-16T09:35:01.895 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:35:01.895 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:35:01.895 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:35:01.895 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:35:01.895 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.25064", 2024-09-16T09:35:01.895 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/2578256439", 2024-09-16T09:35:01.895 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 170710 2024-09-16T09:35:01.895 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.895 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:35:01.895 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.895 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:35:01.896 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.603557+0000", 2024-09-16T09:35:01.896 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:35:01.896 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.896 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.896 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:35:01.896 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.603557+0000", 2024-09-16T09:35:01.896 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:35:01.896 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.896 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.896 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:35:01.896 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.603559+0000", 2024-09-16T09:35:01.896 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.6020000000000001e-06 2024-09-16T09:35:01.896 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.896 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.897 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:35:01.897 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.603565+0000", 2024-09-16T09:35:01.897 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.0249999999999999e-06 2024-09-16T09:35:01.897 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.897 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.897 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:35:01.897 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.603578+0000", 2024-09-16T09:35:01.897 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.2574e-05 2024-09-16T09:35:01.897 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.897 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.897 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:35:01.897 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.603585+0000", 2024-09-16T09:35:01.897 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.6809999999999998e-06 2024-09-16T09:35:01.897 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.898 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.898 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:35:01.898 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.603653+0000", 2024-09-16T09:35:01.898 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.7407e-05 2024-09-16T09:35:01.898 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.898 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.898 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:35:01.898 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.603663+0000", 2024-09-16T09:35:01.898 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.9580000000000002e-06 2024-09-16T09:35:01.898 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.898 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.898 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:35:01.898 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.455039+0000", 2024-09-16T09:35:01.898 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.85137605699999996 2024-09-16T09:35:01.899 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.899 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.899 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:35:01.899 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.455153+0000", 2024-09-16T09:35:01.899 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000114546 2024-09-16T09:35:01.899 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.899 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.899 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 1,3", 2024-09-16T09:35:01.899 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.455271+0000", 2024-09-16T09:35:01.899 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00011796 2024-09-16T09:35:01.899 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.899 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.899 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:35:01.899 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.460157+0000", 2024-09-16T09:35:01.899 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0048861909999999998 2024-09-16T09:35:01.900 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.900 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.900 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:35:01.900 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.471870+0000", 2024-09-16T09:35:01.900 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.011713037000000001 2024-09-16T09:35:01.900 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.900 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.900 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:35:01.900 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.474603+0000", 2024-09-16T09:35:01.900 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0027326300000000002 2024-09-16T09:35:01.900 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.900 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.900 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:35:01.900 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.474652+0000", 2024-09-16T09:35:01.900 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.9325e-05 2024-09-16T09:35:01.901 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.901 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.901 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:35:01.901 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.474688+0000", 2024-09-16T09:35:01.901 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.6016999999999998e-05 2024-09-16T09:35:01.901 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:35:01.901 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:35:01.901 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:35:01.901 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.901 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.901 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.25064.0:170714 8.b 8:db66d47b:::benchmark_data_smithi070_107919_object46119:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e204)", 2024-09-16T09:35:01.901 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:32:07.607664+0000", 2024-09-16T09:35:01.901 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 174.203003604, 2024-09-16T09:35:01.901 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.86717594799999997, 2024-09-16T09:35:01.902 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:35:01.902 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:35:01.902 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:35:01.902 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:35:01.902 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.25064", 2024-09-16T09:35:01.902 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/2578256439", 2024-09-16T09:35:01.902 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 170714 2024-09-16T09:35:01.902 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.902 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:35:01.902 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.902 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:35:01.902 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.607664+0000", 2024-09-16T09:35:01.902 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:35:01.902 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.902 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.903 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:35:01.903 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.607664+0000", 2024-09-16T09:35:01.903 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:35:01.903 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.903 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.903 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:35:01.903 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.607665+0000", 2024-09-16T09:35:01.903 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.4479999999999999e-06 2024-09-16T09:35:01.903 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.903 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.903 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:35:01.904 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.607671+0000", 2024-09-16T09:35:01.904 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.4360000000000001e-06 2024-09-16T09:35:01.904 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.904 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.904 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:35:01.904 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.607682+0000", 2024-09-16T09:35:01.904 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1874e-05 2024-09-16T09:35:01.904 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.904 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.904 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:35:01.904 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.607688+0000", 2024-09-16T09:35:01.904 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.8860000000000001e-06 2024-09-16T09:35:01.904 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.904 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.905 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:35:01.905 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.607754+0000", 2024-09-16T09:35:01.905 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.6018000000000004e-05 2024-09-16T09:35:01.905 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.905 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.905 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:35:01.905 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.607765+0000", 2024-09-16T09:35:01.905 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0789e-05 2024-09-16T09:35:01.905 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.905 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.905 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:35:01.905 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.456688+0000", 2024-09-16T09:35:01.905 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.84892325499999999 2024-09-16T09:35:01.905 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.905 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.906 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:35:01.906 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.456757+0000", 2024-09-16T09:35:01.906 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.8257999999999996e-05 2024-09-16T09:35:01.906 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.906 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.906 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 1,3", 2024-09-16T09:35:01.906 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.456823+0000", 2024-09-16T09:35:01.906 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.6606999999999994e-05 2024-09-16T09:35:01.906 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.906 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.906 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:35:01.906 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.460381+0000", 2024-09-16T09:35:01.906 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0035573739999999999 2024-09-16T09:35:01.906 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.907 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.907 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:35:01.907 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.471911+0000", 2024-09-16T09:35:01.907 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.011530383 2024-09-16T09:35:01.907 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.907 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.907 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:35:01.907 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.474737+0000", 2024-09-16T09:35:01.907 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0028262299999999999 2024-09-16T09:35:01.907 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.907 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.907 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:35:01.907 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.474803+0000", 2024-09-16T09:35:01.907 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.5602e-05 2024-09-16T09:35:01.907 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.908 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.908 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:35:01.908 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.474840+0000", 2024-09-16T09:35:01.908 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.6788000000000001e-05 2024-09-16T09:35:01.908 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:35:01.908 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:35:01.908 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:35:01.908 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.908 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.908 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:67145 9.16 9:6eb0471e:::benchmark_data_smithi070_137000_object67144:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e254)", 2024-09-16T09:35:01.908 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:34:11.477245+0000", 2024-09-16T09:35:01.908 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 50.333421858999998, 2024-09-16T09:35:01.908 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.00133326, 2024-09-16T09:35:01.909 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:35:01.909 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:35:01.909 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:35:01.909 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:35:01.909 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:35:01.909 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:35:01.909 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 67145 2024-09-16T09:35:01.909 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.909 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:35:01.909 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.909 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:35:01.909 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.477245+0000", 2024-09-16T09:35:01.909 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:35:01.909 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.909 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.910 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:35:01.910 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.477245+0000", 2024-09-16T09:35:01.910 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:35:01.910 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.910 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.910 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:35:01.910 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.477247+0000", 2024-09-16T09:35:01.910 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.2529999999999999e-06 2024-09-16T09:35:01.910 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.910 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.910 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:35:01.910 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.477323+0000", 2024-09-16T09:35:01.910 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.6688999999999997e-05 2024-09-16T09:35:01.910 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.911 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.911 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:35:01.911 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.477339+0000", 2024-09-16T09:35:01.911 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.5832999999999999e-05 2024-09-16T09:35:01.911 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.911 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.911 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:35:01.911 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.477346+0000", 2024-09-16T09:35:01.911 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.153e-06 2024-09-16T09:35:01.911 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.911 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.911 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:35:01.911 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.477404+0000", 2024-09-16T09:35:01.911 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.8041999999999998e-05 2024-09-16T09:35:01.912 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.912 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.912 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:35:01.912 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.477413+0000", 2024-09-16T09:35:01.912 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.5110000000000003e-06 2024-09-16T09:35:01.912 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.912 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.912 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:35:01.912 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.452134+0000", 2024-09-16T09:35:01.912 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.97472151100000004 2024-09-16T09:35:01.912 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.912 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.912 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:35:01.912 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.452270+0000", 2024-09-16T09:35:01.913 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00013596699999999999 2024-09-16T09:35:01.913 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.913 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.913 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 1,2", 2024-09-16T09:35:01.913 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.452413+0000", 2024-09-16T09:35:01.913 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000142884 2024-09-16T09:35:01.913 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.913 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.913 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:35:01.913 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.458265+0000", 2024-09-16T09:35:01.913 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0058516549999999999 2024-09-16T09:35:01.913 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.913 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.913 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:35:01.914 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.460714+0000", 2024-09-16T09:35:01.914 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0024490620000000001 2024-09-16T09:35:01.914 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.914 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.914 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:35:01.914 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.478484+0000", 2024-09-16T09:35:01.914 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.017769589999999998 2024-09-16T09:35:01.914 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.914 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.914 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:35:01.914 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.478535+0000", 2024-09-16T09:35:01.914 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.1928999999999997e-05 2024-09-16T09:35:01.914 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.916 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.916 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:35:01.916 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.478579+0000", 2024-09-16T09:35:01.916 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.3180999999999999e-05 2024-09-16T09:35:01.916 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:35:01.916 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:35:01.916 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:35:01.916 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.916 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.916 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:67148 9.16 9:6b4987c7:::benchmark_data_smithi070_137000_object67147:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e254)", 2024-09-16T09:35:01.916 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:34:11.477768+0000", 2024-09-16T09:35:01.916 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 50.332899478000002, 2024-09-16T09:35:01.916 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.001389544, 2024-09-16T09:35:01.917 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:35:01.917 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:35:01.917 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:35:01.917 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:35:01.917 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:35:01.917 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:35:01.917 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 67148 2024-09-16T09:35:01.917 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.917 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:35:01.917 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.917 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:35:01.917 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.477768+0000", 2024-09-16T09:35:01.917 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:35:01.917 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.918 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.918 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:35:01.918 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.477768+0000", 2024-09-16T09:35:01.918 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:35:01.918 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.918 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.918 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:35:01.918 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.477769+0000", 2024-09-16T09:35:01.918 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.7000000000000003e-07 2024-09-16T09:35:01.918 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.918 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.918 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:35:01.918 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.477849+0000", 2024-09-16T09:35:01.918 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.0500999999999994e-05 2024-09-16T09:35:01.918 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.919 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.919 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:35:01.919 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.477867+0000", 2024-09-16T09:35:01.919 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.8182e-05 2024-09-16T09:35:01.919 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.919 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.919 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:35:01.919 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.477876+0000", 2024-09-16T09:35:01.919 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.0900000000000005e-06 2024-09-16T09:35:01.919 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.919 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.919 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:35:01.919 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.478146+0000", 2024-09-16T09:35:01.919 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00027060400000000003 2024-09-16T09:35:01.919 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.920 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.920 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:35:01.920 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.478156+0000", 2024-09-16T09:35:01.920 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.7990000000000005e-06 2024-09-16T09:35:01.920 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.920 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.920 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:35:01.920 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.453564+0000", 2024-09-16T09:35:01.920 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.97540761799999998 2024-09-16T09:35:01.920 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.920 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.920 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:35:01.920 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.453683+0000", 2024-09-16T09:35:01.921 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000119882 2024-09-16T09:35:01.921 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.921 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.921 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 1,2", 2024-09-16T09:35:01.922 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.453825+0000", 2024-09-16T09:35:01.922 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000142006 2024-09-16T09:35:01.922 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.922 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.922 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:35:01.922 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.458271+0000", 2024-09-16T09:35:01.922 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0044451899999999999 2024-09-16T09:35:01.923 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.923 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.923 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:35:01.923 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.460761+0000", 2024-09-16T09:35:01.923 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0024901599999999999 2024-09-16T09:35:01.923 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.923 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.923 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:35:01.923 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.479057+0000", 2024-09-16T09:35:01.923 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.018296501 2024-09-16T09:35:01.923 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.923 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.923 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:35:01.924 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.479120+0000", 2024-09-16T09:35:01.924 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.3081000000000001e-05 2024-09-16T09:35:01.924 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.924 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.924 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:35:01.924 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.479157+0000", 2024-09-16T09:35:01.924 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.6959999999999998e-05 2024-09-16T09:35:01.924 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:35:01.924 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:35:01.924 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:35:01.924 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.924 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.924 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:67153 9.16 9:6b621114:::benchmark_data_smithi070_137000_object67152:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e254)", 2024-09-16T09:35:01.925 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:34:11.481611+0000", 2024-09-16T09:35:01.925 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 50.329056143999999, 2024-09-16T09:35:01.925 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.99775295500000005, 2024-09-16T09:35:01.925 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:35:01.925 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:35:01.925 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:35:01.925 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:35:01.925 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:35:01.925 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:35:01.925 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 67153 2024-09-16T09:35:01.925 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.925 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:35:01.925 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.925 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:35:01.925 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.481611+0000", 2024-09-16T09:35:01.926 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:35:01.926 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.926 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.926 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:35:01.926 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.481611+0000", 2024-09-16T09:35:01.926 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:35:01.926 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.926 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.926 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:35:01.926 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.481612+0000", 2024-09-16T09:35:01.926 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.1900000000000001e-07 2024-09-16T09:35:01.926 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.926 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.927 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:35:01.927 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.481650+0000", 2024-09-16T09:35:01.927 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.7489e-05 2024-09-16T09:35:01.927 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.927 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.927 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:35:01.927 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.481659+0000", 2024-09-16T09:35:01.927 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.4909999999999994e-06 2024-09-16T09:35:01.927 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.927 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.927 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:35:01.927 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.481666+0000", 2024-09-16T09:35:01.927 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.9809999999999997e-06 2024-09-16T09:35:01.927 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.928 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.928 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:35:01.928 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.481724+0000", 2024-09-16T09:35:01.928 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.8035000000000003e-05 2024-09-16T09:35:01.928 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.928 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.928 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:35:01.928 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.481733+0000", 2024-09-16T09:35:01.928 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.3489999999999998e-06 2024-09-16T09:35:01.928 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.928 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.928 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:35:01.928 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.455089+0000", 2024-09-16T09:35:01.929 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.97335562499999995 2024-09-16T09:35:01.929 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.929 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.929 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:35:01.929 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.455208+0000", 2024-09-16T09:35:01.929 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000119305 2024-09-16T09:35:01.929 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.929 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.929 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 1,2", 2024-09-16T09:35:01.929 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.455357+0000", 2024-09-16T09:35:01.929 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00014880000000000001 2024-09-16T09:35:01.929 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.929 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.930 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:35:01.930 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.459587+0000", 2024-09-16T09:35:01.930 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0042294289999999998 2024-09-16T09:35:01.930 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.930 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.930 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:35:01.930 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.460813+0000", 2024-09-16T09:35:01.930 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0012265399999999999 2024-09-16T09:35:01.930 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.930 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.930 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:35:01.930 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.479275+0000", 2024-09-16T09:35:01.930 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.018462135000000001 2024-09-16T09:35:01.930 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.931 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.931 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:35:01.931 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.479321+0000", 2024-09-16T09:35:01.931 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.6267000000000003e-05 2024-09-16T09:35:01.931 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.931 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.931 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:35:01.931 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.479364+0000", 2024-09-16T09:35:01.931 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.2589999999999997e-05 2024-09-16T09:35:01.931 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:35:01.931 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:35:01.931 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:35:01.931 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.931 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.931 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:67177 9.16 9:6c8c3aa4:::benchmark_data_smithi070_137000_object67176:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e254)", 2024-09-16T09:35:01.932 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:34:11.503406+0000", 2024-09-16T09:35:01.932 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 50.307261261000001, 2024-09-16T09:35:01.932 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.97608242499999998, 2024-09-16T09:35:01.932 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:35:01.932 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:35:01.932 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:35:01.932 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:35:01.932 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:35:01.932 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:35:01.932 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 67177 2024-09-16T09:35:01.932 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.932 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:35:01.932 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.933 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:35:01.933 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.503406+0000", 2024-09-16T09:35:01.933 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:35:01.933 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.933 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.933 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:35:01.933 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.503406+0000", 2024-09-16T09:35:01.933 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:35:01.933 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.933 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.933 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:35:01.933 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.503406+0000", 2024-09-16T09:35:01.933 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.01e-07 2024-09-16T09:35:01.933 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.933 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.934 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:35:01.934 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.503432+0000", 2024-09-16T09:35:01.934 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.5673e-05 2024-09-16T09:35:01.934 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.934 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.934 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:35:01.934 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.503442+0000", 2024-09-16T09:35:01.934 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.7040000000000006e-06 2024-09-16T09:35:01.934 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.934 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.934 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:35:01.934 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.503445+0000", 2024-09-16T09:35:01.934 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.6339999999999999e-06 2024-09-16T09:35:01.934 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.935 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.935 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:35:01.935 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.503516+0000", 2024-09-16T09:35:01.935 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.0438999999999994e-05 2024-09-16T09:35:01.935 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.935 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.935 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:35:01.935 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.503528+0000", 2024-09-16T09:35:01.935 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.2349e-05 2024-09-16T09:35:01.935 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.935 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.935 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:35:01.935 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.456887+0000", 2024-09-16T09:35:01.935 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.95335896399999998 2024-09-16T09:35:01.935 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.936 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.936 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:35:01.936 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:01 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:35:01.936 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:01 smithi084 ceph-mon[32629]: osdmap e280: 8 total, 8 up, 8 in 2024-09-16T09:35:01.937 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:01 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:35:01.937 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:01 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:35:01.937 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:01 smithi070 ceph-mon[30388]: osdmap e280: 8 total, 8 up, 8 in 2024-09-16T09:35:01.937 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:01 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:35:01.937 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:01 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:35:01.937 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:01 smithi070 ceph-mon[40691]: osdmap e280: 8 total, 8 up, 8 in 2024-09-16T09:35:01.938 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:35:01.938 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.457028+0000", 2024-09-16T09:35:01.938 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00014072600000000001 2024-09-16T09:35:01.938 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.938 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.938 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 1,2", 2024-09-16T09:35:01.938 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.457172+0000", 2024-09-16T09:35:01.938 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000144389 2024-09-16T09:35:01.938 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.938 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.938 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:35:01.939 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.460657+0000", 2024-09-16T09:35:01.939 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0034844250000000002 2024-09-16T09:35:01.939 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.939 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.939 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:35:01.939 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.460862+0000", 2024-09-16T09:35:01.939 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00020545900000000001 2024-09-16T09:35:01.939 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.939 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.939 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:35:01.939 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.479405+0000", 2024-09-16T09:35:01.939 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.018543343 2024-09-16T09:35:01.939 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.939 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.940 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:35:01.940 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.479458+0000", 2024-09-16T09:35:01.940 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.2652999999999999e-05 2024-09-16T09:35:01.940 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.940 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.940 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:35:01.940 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.479488+0000", 2024-09-16T09:35:01.940 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.0266000000000001e-05 2024-09-16T09:35:01.940 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:35:01.940 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:35:01.940 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:35:01.940 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.940 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.941 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:67186 9.16 9:6ddb896c:::benchmark_data_smithi070_137000_object67185:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e254)", 2024-09-16T09:35:01.941 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:34:11.514674+0000", 2024-09-16T09:35:01.941 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 50.295993248999999, 2024-09-16T09:35:01.941 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.96492478400000004, 2024-09-16T09:35:01.941 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:35:01.941 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:35:01.941 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:35:01.941 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:35:01.941 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:35:01.941 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:35:01.941 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 67186 2024-09-16T09:35:01.941 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.941 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:35:01.941 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.942 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:35:01.942 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.514674+0000", 2024-09-16T09:35:01.942 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:35:01.942 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.942 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.942 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:35:01.942 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.514674+0000", 2024-09-16T09:35:01.942 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:35:01.942 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.942 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.942 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:35:01.942 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.514676+0000", 2024-09-16T09:35:01.942 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.9690000000000001e-06 2024-09-16T09:35:01.942 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.943 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.943 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:35:01.943 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.514846+0000", 2024-09-16T09:35:01.943 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00016989199999999999 2024-09-16T09:35:01.943 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.943 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.943 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:35:01.943 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.514862+0000", 2024-09-16T09:35:01.943 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.6317999999999999e-05 2024-09-16T09:35:01.943 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.943 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.943 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:35:01.943 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.514870+0000", 2024-09-16T09:35:01.943 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.1699999999999997e-06 2024-09-16T09:35:01.944 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.944 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.944 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:35:01.944 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.514942+0000", 2024-09-16T09:35:01.944 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.1619999999999995e-05 2024-09-16T09:35:01.944 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.944 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.944 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:35:01.944 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.514952+0000", 2024-09-16T09:35:01.944 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.577e-06 2024-09-16T09:35:01.944 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.944 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.944 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:35:01.944 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.458305+0000", 2024-09-16T09:35:01.945 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.94335340499999998 2024-09-16T09:35:01.945 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.945 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.945 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:35:01.945 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.458454+0000", 2024-09-16T09:35:01.945 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00014879499999999999 2024-09-16T09:35:01.945 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.945 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.945 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 1,2", 2024-09-16T09:35:01.945 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.458547+0000", 2024-09-16T09:35:01.945 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.3602000000000003e-05 2024-09-16T09:35:01.945 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.945 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.945 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:35:01.946 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.460826+0000", 2024-09-16T09:35:01.946 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0022790449999999999 2024-09-16T09:35:01.946 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.946 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.946 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:35:01.946 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.466858+0000", 2024-09-16T09:35:01.946 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0060317620000000004 2024-09-16T09:35:01.946 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.946 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.946 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:35:01.946 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.479526+0000", 2024-09-16T09:35:01.946 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.012668008 2024-09-16T09:35:01.946 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.946 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.946 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:35:01.947 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.479570+0000", 2024-09-16T09:35:01.947 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.3455000000000003e-05 2024-09-16T09:35:01.947 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.947 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.947 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:35:01.947 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.479599+0000", 2024-09-16T09:35:01.947 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.9166000000000002e-05 2024-09-16T09:35:01.947 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:35:01.947 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:35:01.947 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:35:01.947 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.947 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.947 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:67213 9.16 9:6ebe1d56:::benchmark_data_smithi070_137000_object67212:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e254)", 2024-09-16T09:35:01.947 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:34:11.541163+0000", 2024-09-16T09:35:01.947 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 50.269504415, 2024-09-16T09:35:01.948 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.93855440499999998, 2024-09-16T09:35:01.948 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:35:01.948 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:35:01.948 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:35:01.948 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:35:01.948 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:35:01.948 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:35:01.948 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 67213 2024-09-16T09:35:01.948 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.948 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:35:01.948 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.948 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:35:01.948 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.541163+0000", 2024-09-16T09:35:01.948 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:35:01.949 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.949 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.949 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:35:01.949 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.541163+0000", 2024-09-16T09:35:01.949 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:35:01.949 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.949 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.949 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:35:01.949 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.541164+0000", 2024-09-16T09:35:01.949 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.607e-06 2024-09-16T09:35:01.949 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.949 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.949 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:35:01.949 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.541193+0000", 2024-09-16T09:35:01.949 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.8742e-05 2024-09-16T09:35:01.950 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.950 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.950 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:35:01.950 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.541211+0000", 2024-09-16T09:35:01.950 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.8088999999999999e-05 2024-09-16T09:35:01.950 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.950 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.950 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:35:01.950 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.541219+0000", 2024-09-16T09:35:01.950 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.6760000000000007e-06 2024-09-16T09:35:01.950 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.950 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.950 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:35:01.950 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.541282+0000", 2024-09-16T09:35:01.950 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.3074e-05 2024-09-16T09:35:01.951 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.951 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.951 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:35:01.951 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.541294+0000", 2024-09-16T09:35:01.951 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1619e-05 2024-09-16T09:35:01.951 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.951 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.951 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:35:01.951 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.459650+0000", 2024-09-16T09:35:01.951 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.91835631900000003 2024-09-16T09:35:01.951 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.951 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.951 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:35:01.952 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.459767+0000", 2024-09-16T09:35:01.952 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000117439 2024-09-16T09:35:01.952 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.952 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.952 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 1,2", 2024-09-16T09:35:01.952 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.459847+0000", 2024-09-16T09:35:01.952 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.9629999999999995e-05 2024-09-16T09:35:01.952 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.952 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.952 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:35:01.952 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.461953+0000", 2024-09-16T09:35:01.952 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0021063269999999999 2024-09-16T09:35:01.952 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.952 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.953 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:35:01.953 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.466906+0000", 2024-09-16T09:35:01.953 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0049525569999999998 2024-09-16T09:35:01.953 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.953 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.953 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:35:01.953 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.479651+0000", 2024-09-16T09:35:01.953 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.012744734000000001 2024-09-16T09:35:01.953 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.953 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.953 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:35:01.953 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.479681+0000", 2024-09-16T09:35:01.953 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.0688000000000002e-05 2024-09-16T09:35:01.953 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.954 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.954 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:35:01.954 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.479717+0000", 2024-09-16T09:35:01.954 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.5904e-05 2024-09-16T09:35:01.954 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:35:01.954 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:35:01.954 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:35:01.954 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.954 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.954 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:67242 9.16 9:6a225ad4:::benchmark_data_smithi070_137000_object67241:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e254)", 2024-09-16T09:35:01.954 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:34:11.573512+0000", 2024-09-16T09:35:01.954 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 50.237155432000002, 2024-09-16T09:35:01.954 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.90633363499999997, 2024-09-16T09:35:01.954 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:35:01.955 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:35:01.955 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:35:01.955 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:35:01.955 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:35:01.955 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:35:01.955 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 67242 2024-09-16T09:35:01.955 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.955 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:35:01.955 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.955 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:35:01.955 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.573512+0000", 2024-09-16T09:35:01.955 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:35:01.955 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.955 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.956 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:35:01.956 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.573512+0000", 2024-09-16T09:35:01.956 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:35:01.956 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.956 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.956 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:35:01.956 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.573513+0000", 2024-09-16T09:35:01.956 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.4729999999999999e-06 2024-09-16T09:35:01.956 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.956 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.956 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:35:01.956 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.573545+0000", 2024-09-16T09:35:01.956 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.1516000000000001e-05 2024-09-16T09:35:01.956 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.957 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.957 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:35:01.957 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.573560+0000", 2024-09-16T09:35:01.957 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.5218000000000001e-05 2024-09-16T09:35:01.957 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.957 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.957 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:35:01.957 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.573567+0000", 2024-09-16T09:35:01.957 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.4209999999999998e-06 2024-09-16T09:35:01.957 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.957 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.957 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:35:01.957 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.573622+0000", 2024-09-16T09:35:01.957 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.4625e-05 2024-09-16T09:35:01.957 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.958 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.958 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:35:01.958 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.573628+0000", 2024-09-16T09:35:01.958 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.2129999999999996e-06 2024-09-16T09:35:01.958 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.958 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.958 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:35:01.958 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.460896+0000", 2024-09-16T09:35:01.958 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.88726759899999996 2024-09-16T09:35:01.958 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.958 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.958 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:35:01.958 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.461000+0000", 2024-09-16T09:35:01.958 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000103923 2024-09-16T09:35:01.959 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.959 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.959 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 1,2", 2024-09-16T09:35:01.959 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.461143+0000", 2024-09-16T09:35:01.959 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000143635 2024-09-16T09:35:01.959 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.959 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.959 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:35:01.959 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.465556+0000", 2024-09-16T09:35:01.959 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0044125800000000001 2024-09-16T09:35:01.959 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.959 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.959 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:35:01.959 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.466954+0000", 2024-09-16T09:35:01.959 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.001397868 2024-09-16T09:35:01.960 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.960 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.960 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:35:01.960 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.479774+0000", 2024-09-16T09:35:01.960 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.012819841 2024-09-16T09:35:01.960 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.960 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.960 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:35:01.960 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.479815+0000", 2024-09-16T09:35:01.960 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.0818999999999997e-05 2024-09-16T09:35:01.960 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.960 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.960 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:35:01.960 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.479845+0000", 2024-09-16T09:35:01.961 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.0904000000000001e-05 2024-09-16T09:35:01.961 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:35:01.961 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:35:01.961 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:35:01.961 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.961 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.961 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:67253 9.16 9:69d4b77b:::benchmark_data_smithi070_137000_object67252:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e254)", 2024-09-16T09:35:01.961 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:34:11.584445+0000", 2024-09-16T09:35:01.961 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 50.226222579999998, 2024-09-16T09:35:01.961 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.89559586199999996, 2024-09-16T09:35:01.961 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:35:01.961 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:35:01.961 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:35:01.961 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:35:01.962 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:35:01.962 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:35:01.962 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 67253 2024-09-16T09:35:01.962 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.962 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:35:01.962 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.962 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:35:01.962 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.584445+0000", 2024-09-16T09:35:01.962 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:35:01.962 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.962 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.962 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:35:01.962 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.584445+0000", 2024-09-16T09:35:01.962 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:35:01.962 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.963 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.963 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:35:01.963 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.584446+0000", 2024-09-16T09:35:01.963 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.226e-06 2024-09-16T09:35:01.963 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.963 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.963 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:35:01.963 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.584475+0000", 2024-09-16T09:35:01.963 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.9193e-05 2024-09-16T09:35:01.963 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.963 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.963 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:35:01.963 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.584492+0000", 2024-09-16T09:35:01.963 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.6954000000000001e-05 2024-09-16T09:35:01.963 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.964 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.964 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:35:01.964 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.584502+0000", 2024-09-16T09:35:01.964 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.7489999999999993e-06 2024-09-16T09:35:01.964 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.964 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.964 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:35:01.964 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.584570+0000", 2024-09-16T09:35:01.964 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.8324000000000005e-05 2024-09-16T09:35:01.964 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.964 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.964 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:35:01.964 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.584582+0000", 2024-09-16T09:35:01.964 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.234e-05 2024-09-16T09:35:01.965 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.965 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.965 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:35:01.965 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.462001+0000", 2024-09-16T09:35:01.965 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.87741897700000004 2024-09-16T09:35:01.965 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.965 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.965 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:35:01.965 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.462129+0000", 2024-09-16T09:35:01.965 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00012799799999999999 2024-09-16T09:35:01.965 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.965 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.965 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 1,2", 2024-09-16T09:35:01.965 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.462260+0000", 2024-09-16T09:35:01.965 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0001305 2024-09-16T09:35:01.966 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.966 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.966 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:35:01.966 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.465559+0000", 2024-09-16T09:35:01.966 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0032993950000000001 2024-09-16T09:35:01.966 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.966 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.966 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:35:01.966 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.469360+0000", 2024-09-16T09:35:01.966 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0038007319999999998 2024-09-16T09:35:01.966 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.966 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.966 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:35:01.966 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.479926+0000", 2024-09-16T09:35:01.967 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.010566414999999999 2024-09-16T09:35:01.967 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.967 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.967 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:35:01.967 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.479994+0000", 2024-09-16T09:35:01.967 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.7053999999999999e-05 2024-09-16T09:35:01.967 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:01.967 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:01.967 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:35:01.967 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.480041+0000", 2024-09-16T09:35:01.967 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.7005000000000001e-05 2024-09-16T09:35:01.967 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:35:01.967 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:35:01.967 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:35:01.967 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:35:01.968 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:35:01.968 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-09-16T09:35:01.969 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:01.969 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:01.969 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:35:02.070 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:35:02.101 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:02.104 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:02.104 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:35:02.171 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:35:01 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:35:01] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:35:02.205 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:35:02.241 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:02.241 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:02.242 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:35:02.343 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:35:02.374 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:02.374 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:02.375 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:35:02.476 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:35:02.509 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:35:02.512 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:02.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:02.513 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:35:02.614 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:35:02.655 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:02.656 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:02.656 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:35:02.757 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:35:02.792 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:02.792 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:02.793 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:35:02.812 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:02 smithi070 ceph-mon[30388]: pgmap v495: 178 pgs: 1 active+recovering+undersized+remapped, 177 active+clean; 7.2 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 841 B/s rd, 21 MiB/s wr, 331 op/s; 1325/355410 objects misplaced (0.373%) 2024-09-16T09:35:02.813 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:02 smithi070 ceph-mon[40691]: pgmap v495: 178 pgs: 1 active+recovering+undersized+remapped, 177 active+clean; 7.2 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 841 B/s rd, 21 MiB/s wr, 331 op/s; 1325/355410 objects misplaced (0.373%) 2024-09-16T09:35:02.825 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T09:35:02.828 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:02 smithi084 ceph-mon[32629]: pgmap v495: 178 pgs: 1 active+recovering+undersized+remapped, 177 active+clean; 7.2 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 841 B/s rd, 21 MiB/s wr, 331 op/s; 1325/355410 objects misplaced (0.373%) 2024-09-16T09:35:02.880 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.5 dump_ops_in_flight 2024-09-16T09:35:02.893 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:35:02.958 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:02.958 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:02.959 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:35:03.059 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:35:03.102 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:03.103 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:03.103 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:35:03.204 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:35:03.222 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:35:03.245 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:03.246 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:03.246 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:35:03.347 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:35:03.381 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:03.382 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:03.382 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:35:03.383 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:35:03.410 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:35:03 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:35:03.483 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:35:03.531 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:03.532 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:03.532 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:35:03.633 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:35:03.658 INFO:teuthology.orchestra.run.smithi070.stderr:nodeep-scrub is set 2024-09-16T09:35:03.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:03.666 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:03.666 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:35:03.666 INFO:teuthology.orchestra.run.smithi070.stderr:set pool 9 pg_num to 16 2024-09-16T09:35:03.767 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:35:03.804 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:03.805 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:03.805 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:35:03.837 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:03 smithi084 ceph-mon[32629]: Health check failed: Degraded data redundancy: 1 pg undersized (PG_DEGRADED) 2024-09-16T09:35:03.837 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:03 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1287788413' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:35:03.837 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:03 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:35:03.837 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:03 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:35:03.837 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:03 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:35:03.837 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:03 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/4093338967' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num", "val": "16"} : dispatch 2024-09-16T09:35:03.837 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:03 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num", "val": "16"} : dispatch 2024-09-16T09:35:03.906 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:35:03.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:03 smithi070 ceph-mon[30388]: Health check failed: Degraded data redundancy: 1 pg undersized (PG_DEGRADED) 2024-09-16T09:35:03.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:03 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1287788413' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:35:03.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:03 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:35:03.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:03 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:35:03.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:03 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:35:03.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:03 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/4093338967' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num", "val": "16"} : dispatch 2024-09-16T09:35:03.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:03 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num", "val": "16"} : dispatch 2024-09-16T09:35:03.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:03 smithi070 ceph-mon[40691]: Health check failed: Degraded data redundancy: 1 pg undersized (PG_DEGRADED) 2024-09-16T09:35:03.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:03 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1287788413' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:35:03.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:03 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:35:03.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:03 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:35:03.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:03 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:35:03.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:03 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/4093338967' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num", "val": "16"} : dispatch 2024-09-16T09:35:03.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:03 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num", "val": "16"} : dispatch 2024-09-16T09:35:03.945 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:03.946 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:03.946 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:35:04.004 INFO:teuthology.orchestra.run.smithi084.stdout:ERROR: (22) Invalid argument 2024-09-16T09:35:04.004 INFO:teuthology.orchestra.run.smithi084.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:35:04.026 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:35:04.026 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:35:04.026 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:35:04.026 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:35:04.026 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:35:04.026 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:35:04.026 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:35:04.026 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:35:04.026 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:35:04.026 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:35:04.027 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:35:04.027 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:35:04.027 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:35:04.027 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:35:04.045 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:35:04.045 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:35:04.047 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:35:04.084 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:04.085 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:04.085 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:35:04.186 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:35:04.224 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:04.225 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:04.225 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:35:04.326 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:35:04.369 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:04.369 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:04.369 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:35:04.470 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:35:04.508 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:04.508 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:04.509 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:35:04.609 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:35:04.648 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:04.649 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:04.649 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:35:04.750 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:35:04.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:04.774 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:04.774 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:35:04.807 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:04 smithi084 ceph-mon[32629]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:35:04.807 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:04 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1287788413' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-16T09:35:04.807 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:04 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num", "val": "16"}]': finished 2024-09-16T09:35:04.807 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:04 smithi084 ceph-mon[32629]: osdmap e281: 8 total, 8 up, 8 in 2024-09-16T09:35:04.807 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:04 smithi084 ceph-mon[32629]: 9.11 scrub starts 2024-09-16T09:35:04.807 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:04 smithi084 ceph-mon[32629]: pgmap v497: 178 pgs: 1 active+recovering+undersized+remapped, 177 active+clean; 7.2 GiB data, 26 GiB used, 690 GiB / 715 GiB avail; 1.6 KiB/s rd, 47 MiB/s wr, 1.20k op/s; 1053/356061 objects misplaced (0.296%) 2024-09-16T09:35:04.807 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:04 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num_actual", "val": "16"} : dispatch 2024-09-16T09:35:04.807 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:04 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num_actual", "val": "16"} : dispatch 2024-09-16T09:35:04.808 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:04 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1331102180' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:35:04.808 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:04 smithi084 ceph-mon[32629]: 9.10 scrub starts 2024-09-16T09:35:04.808 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:04 smithi084 ceph-mon[32629]: 9.0 scrub starts 2024-09-16T09:35:04.808 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:04 smithi084 ceph-mon[32629]: 9.9 scrub starts 2024-09-16T09:35:04.839 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.5 dump_blocked_ops 2024-09-16T09:35:04.876 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:35:04.901 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:04.901 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:04.902 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:35:04.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:04 smithi070 ceph-mon[40691]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:35:04.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:04 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1287788413' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-16T09:35:04.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:04 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num", "val": "16"}]': finished 2024-09-16T09:35:04.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:04 smithi070 ceph-mon[40691]: osdmap e281: 8 total, 8 up, 8 in 2024-09-16T09:35:04.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:04 smithi070 ceph-mon[40691]: 9.11 scrub starts 2024-09-16T09:35:04.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:04 smithi070 ceph-mon[40691]: pgmap v497: 178 pgs: 1 active+recovering+undersized+remapped, 177 active+clean; 7.2 GiB data, 26 GiB used, 690 GiB / 715 GiB avail; 1.6 KiB/s rd, 47 MiB/s wr, 1.20k op/s; 1053/356061 objects misplaced (0.296%) 2024-09-16T09:35:04.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:04 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num_actual", "val": "16"} : dispatch 2024-09-16T09:35:04.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:04 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num_actual", "val": "16"} : dispatch 2024-09-16T09:35:04.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:04 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1331102180' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:35:04.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:04 smithi070 ceph-mon[40691]: 9.10 scrub starts 2024-09-16T09:35:04.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:04 smithi070 ceph-mon[40691]: 9.0 scrub starts 2024-09-16T09:35:04.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:04 smithi070 ceph-mon[40691]: 9.9 scrub starts 2024-09-16T09:35:04.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:04 smithi070 ceph-mon[30388]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:35:04.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:04 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1287788413' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-16T09:35:04.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:04 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num", "val": "16"}]': finished 2024-09-16T09:35:04.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:04 smithi070 ceph-mon[30388]: osdmap e281: 8 total, 8 up, 8 in 2024-09-16T09:35:04.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:04 smithi070 ceph-mon[30388]: 9.11 scrub starts 2024-09-16T09:35:04.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:04 smithi070 ceph-mon[30388]: pgmap v497: 178 pgs: 1 active+recovering+undersized+remapped, 177 active+clean; 7.2 GiB data, 26 GiB used, 690 GiB / 715 GiB avail; 1.6 KiB/s rd, 47 MiB/s wr, 1.20k op/s; 1053/356061 objects misplaced (0.296%) 2024-09-16T09:35:04.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:04 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num_actual", "val": "16"} : dispatch 2024-09-16T09:35:04.923 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:04 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num_actual", "val": "16"} : dispatch 2024-09-16T09:35:04.923 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:04 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1331102180' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:35:04.923 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:04 smithi070 ceph-mon[30388]: 9.10 scrub starts 2024-09-16T09:35:04.923 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:04 smithi070 ceph-mon[30388]: 9.0 scrub starts 2024-09-16T09:35:04.923 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:04 smithi070 ceph-mon[30388]: 9.9 scrub starts 2024-09-16T09:35:05.002 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:35:05.024 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:05.024 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:05.025 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:35:05.125 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:35:05.146 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:35:05.150 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:05.150 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:05.151 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:35:05.252 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:35:05.274 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:05.275 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:05.275 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:35:05.376 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:35:05.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:05.399 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:05.399 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:35:05.500 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:35:05.522 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:05.522 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:05.523 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:35:05.624 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:35:05.647 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:05.647 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:05.647 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:35:05.748 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:35:05.770 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:05.771 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:05.771 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:35:05.872 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:35:05.894 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:05.894 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:05.895 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:35:05.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:05 smithi070 ceph-mon[30388]: 9.6 scrub starts 2024-09-16T09:35:05.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:05 smithi070 ceph-mon[30388]: 9.15 scrub starts 2024-09-16T09:35:05.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:05 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num_actual", "val": "16"}]': finished 2024-09-16T09:35:05.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:05 smithi070 ceph-mon[30388]: osdmap e282: 8 total, 8 up, 8 in 2024-09-16T09:35:05.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:05 smithi070 ceph-mon[30388]: 9.14 scrub starts 2024-09-16T09:35:05.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:05 smithi070 ceph-mon[30388]: 9.1 scrub starts 2024-09-16T09:35:05.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:05 smithi070 ceph-mon[30388]: 9.9 scrub starts 2024-09-16T09:35:05.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:05 smithi070 ceph-mon[40691]: 9.6 scrub starts 2024-09-16T09:35:05.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:05 smithi070 ceph-mon[40691]: 9.15 scrub starts 2024-09-16T09:35:05.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:05 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num_actual", "val": "16"}]': finished 2024-09-16T09:35:05.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:05 smithi070 ceph-mon[40691]: osdmap e282: 8 total, 8 up, 8 in 2024-09-16T09:35:05.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:05 smithi070 ceph-mon[40691]: 9.14 scrub starts 2024-09-16T09:35:05.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:05 smithi070 ceph-mon[40691]: 9.1 scrub starts 2024-09-16T09:35:05.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:05 smithi070 ceph-mon[40691]: 9.9 scrub starts 2024-09-16T09:35:05.995 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:35:06.017 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:06.018 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:06.018 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:35:06.119 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:35:06.124 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:05 smithi084 ceph-mon[32629]: 9.6 scrub starts 2024-09-16T09:35:06.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:05 smithi084 ceph-mon[32629]: 9.15 scrub starts 2024-09-16T09:35:06.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:05 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num_actual", "val": "16"}]': finished 2024-09-16T09:35:06.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:05 smithi084 ceph-mon[32629]: osdmap e282: 8 total, 8 up, 8 in 2024-09-16T09:35:06.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:05 smithi084 ceph-mon[32629]: 9.14 scrub starts 2024-09-16T09:35:06.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:05 smithi084 ceph-mon[32629]: 9.1 scrub starts 2024-09-16T09:35:06.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:05 smithi084 ceph-mon[32629]: 9.9 scrub starts 2024-09-16T09:35:06.141 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:06.141 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:06.141 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:35:06.242 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:35:06.265 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:06.266 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:06.266 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:35:06.368 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:35:06.390 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:06.390 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:06.390 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:35:06.491 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:35:06.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:06.514 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:06.514 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:35:06.615 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:35:06.636 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:06.637 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:06.637 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:35:06.738 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:35:06.770 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:06.771 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:06.771 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:35:06.872 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:35:06.899 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:06.900 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:06.901 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:35:06.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:06 smithi070 ceph-mon[30388]: 9.6 scrub starts 2024-09-16T09:35:06.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:06 smithi070 ceph-mon[30388]: osdmap e283: 8 total, 8 up, 8 in 2024-09-16T09:35:06.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:06 smithi070 ceph-mon[30388]: pgmap v500: 178 pgs: 1 active+recovering+undersized+remapped, 177 active+clean; 7.2 GiB data, 26 GiB used, 690 GiB / 715 GiB avail; 1.2 KiB/s rd, 41 MiB/s wr, 1.28k op/s; 1053/356061 objects misplaced (0.296%); 3.4 MiB/s, 53 objects/s recovering 2024-09-16T09:35:06.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:06 smithi070 ceph-mon[30388]: 9.0 scrub starts 2024-09-16T09:35:06.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:06 smithi070 ceph-mon[30388]: osdmap e284: 8 total, 8 up, 8 in 2024-09-16T09:35:06.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:06 smithi070 ceph-mon[40691]: 9.6 scrub starts 2024-09-16T09:35:06.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:06 smithi070 ceph-mon[40691]: osdmap e283: 8 total, 8 up, 8 in 2024-09-16T09:35:06.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:06 smithi070 ceph-mon[40691]: pgmap v500: 178 pgs: 1 active+recovering+undersized+remapped, 177 active+clean; 7.2 GiB data, 26 GiB used, 690 GiB / 715 GiB avail; 1.2 KiB/s rd, 41 MiB/s wr, 1.28k op/s; 1053/356061 objects misplaced (0.296%); 3.4 MiB/s, 53 objects/s recovering 2024-09-16T09:35:06.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:06 smithi070 ceph-mon[40691]: 9.0 scrub starts 2024-09-16T09:35:06.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:06 smithi070 ceph-mon[40691]: osdmap e284: 8 total, 8 up, 8 in 2024-09-16T09:35:07.001 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:35:07.029 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:07.029 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:07.029 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:35:07.130 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:35:07.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:06 smithi084 ceph-mon[32629]: 9.6 scrub starts 2024-09-16T09:35:07.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:06 smithi084 ceph-mon[32629]: osdmap e283: 8 total, 8 up, 8 in 2024-09-16T09:35:07.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:06 smithi084 ceph-mon[32629]: pgmap v500: 178 pgs: 1 active+recovering+undersized+remapped, 177 active+clean; 7.2 GiB data, 26 GiB used, 690 GiB / 715 GiB avail; 1.2 KiB/s rd, 41 MiB/s wr, 1.28k op/s; 1053/356061 objects misplaced (0.296%); 3.4 MiB/s, 53 objects/s recovering 2024-09-16T09:35:07.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:06 smithi084 ceph-mon[32629]: 9.0 scrub starts 2024-09-16T09:35:07.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:06 smithi084 ceph-mon[32629]: osdmap e284: 8 total, 8 up, 8 in 2024-09-16T09:35:07.162 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:07.163 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:07.164 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:35:07.264 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:35:07.299 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:07.300 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:07.300 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:35:07.401 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:35:07.441 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:07.441 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:07.442 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:35:07.542 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:35:07.576 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:07.577 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:07.577 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:35:07.678 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:35:07.712 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:07.712 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:07.713 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:35:07.777 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd unset noscrub 2024-09-16T09:35:07.814 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:35:07.845 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:07.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:07.848 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:35:07.869 INFO:teuthology.orchestra.run.smithi084.stdout:ERROR: (22) Invalid argument 2024-09-16T09:35:07.869 INFO:teuthology.orchestra.run.smithi084.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:35:07.949 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:35:07.979 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:07.979 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:07.980 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:35:08.081 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:35:08.134 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:08.134 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:08.135 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:35:08.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:07 smithi084 ceph-mon[32629]: 9.15 starting backfill to osd.0 from (0'0,244'3202] MIN to 281'5565 2024-09-16T09:35:08.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:07 smithi084 ceph-mon[32629]: 9.19 starting backfill to osd.0 from (0'0,265'4192] MIN to 281'5503 2024-09-16T09:35:08.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:07 smithi084 ceph-mon[32629]: 9.16 starting backfill to osd.5 from (0'0,244'3177] MIN to 281'5606 2024-09-16T09:35:08.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:07 smithi084 ceph-mon[32629]: 9.18 starting backfill to osd.0 from (0'0,257'3815] MIN to 281'5505 2024-09-16T09:35:08.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:07 smithi084 ceph-mon[32629]: 9.17 starting backfill to osd.3 from (0'0,0'0] MAX to 281'5553 2024-09-16T09:35:08.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:07 smithi084 ceph-mon[32629]: 9.13 starting backfill to osd.1 from (0'0,0'0] MAX to 281'5467 2024-09-16T09:35:08.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:07 smithi084 ceph-mon[32629]: 9.19 starting backfill to osd.7 from (0'0,0'0] MAX to 281'5503 2024-09-16T09:35:08.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:07 smithi084 ceph-mon[32629]: 9.15 starting backfill to osd.5 from (0'0,244'3202] MIN to 281'5565 2024-09-16T09:35:08.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:07 smithi084 ceph-mon[32629]: 9.13 starting backfill to osd.3 from (0'0,0'0] MAX to 281'5467 2024-09-16T09:35:08.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:07 smithi084 ceph-mon[32629]: 9.17 starting backfill to osd.5 from (0'0,244'3150] MIN to 281'5553 2024-09-16T09:35:08.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:07 smithi084 ceph-mon[32629]: 9.10 starting backfill to osd.6 from (0'0,244'3213] MIN to 281'5546 2024-09-16T09:35:08.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:07 smithi084 ceph-mon[32629]: 9.11 scrub starts 2024-09-16T09:35:08.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:07 smithi070 ceph-mon[30388]: 9.15 starting backfill to osd.0 from (0'0,244'3202] MIN to 281'5565 2024-09-16T09:35:08.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:07 smithi070 ceph-mon[30388]: 9.19 starting backfill to osd.0 from (0'0,265'4192] MIN to 281'5503 2024-09-16T09:35:08.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:07 smithi070 ceph-mon[30388]: 9.16 starting backfill to osd.5 from (0'0,244'3177] MIN to 281'5606 2024-09-16T09:35:08.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:07 smithi070 ceph-mon[30388]: 9.18 starting backfill to osd.0 from (0'0,257'3815] MIN to 281'5505 2024-09-16T09:35:08.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:07 smithi070 ceph-mon[30388]: 9.17 starting backfill to osd.3 from (0'0,0'0] MAX to 281'5553 2024-09-16T09:35:08.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:07 smithi070 ceph-mon[30388]: 9.13 starting backfill to osd.1 from (0'0,0'0] MAX to 281'5467 2024-09-16T09:35:08.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:07 smithi070 ceph-mon[30388]: 9.19 starting backfill to osd.7 from (0'0,0'0] MAX to 281'5503 2024-09-16T09:35:08.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:07 smithi070 ceph-mon[30388]: 9.15 starting backfill to osd.5 from (0'0,244'3202] MIN to 281'5565 2024-09-16T09:35:08.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:07 smithi070 ceph-mon[30388]: 9.13 starting backfill to osd.3 from (0'0,0'0] MAX to 281'5467 2024-09-16T09:35:08.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:07 smithi070 ceph-mon[30388]: 9.17 starting backfill to osd.5 from (0'0,244'3150] MIN to 281'5553 2024-09-16T09:35:08.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:07 smithi070 ceph-mon[30388]: 9.10 starting backfill to osd.6 from (0'0,244'3213] MIN to 281'5546 2024-09-16T09:35:08.172 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:07 smithi070 ceph-mon[30388]: 9.11 scrub starts 2024-09-16T09:35:08.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:07 smithi070 ceph-mon[40691]: 9.15 starting backfill to osd.0 from (0'0,244'3202] MIN to 281'5565 2024-09-16T09:35:08.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:07 smithi070 ceph-mon[40691]: 9.19 starting backfill to osd.0 from (0'0,265'4192] MIN to 281'5503 2024-09-16T09:35:08.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:07 smithi070 ceph-mon[40691]: 9.16 starting backfill to osd.5 from (0'0,244'3177] MIN to 281'5606 2024-09-16T09:35:08.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:07 smithi070 ceph-mon[40691]: 9.18 starting backfill to osd.0 from (0'0,257'3815] MIN to 281'5505 2024-09-16T09:35:08.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:07 smithi070 ceph-mon[40691]: 9.17 starting backfill to osd.3 from (0'0,0'0] MAX to 281'5553 2024-09-16T09:35:08.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:07 smithi070 ceph-mon[40691]: 9.13 starting backfill to osd.1 from (0'0,0'0] MAX to 281'5467 2024-09-16T09:35:08.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:07 smithi070 ceph-mon[40691]: 9.19 starting backfill to osd.7 from (0'0,0'0] MAX to 281'5503 2024-09-16T09:35:08.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:07 smithi070 ceph-mon[40691]: 9.15 starting backfill to osd.5 from (0'0,244'3202] MIN to 281'5565 2024-09-16T09:35:08.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:07 smithi070 ceph-mon[40691]: 9.13 starting backfill to osd.3 from (0'0,0'0] MAX to 281'5467 2024-09-16T09:35:08.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:07 smithi070 ceph-mon[40691]: 9.17 starting backfill to osd.5 from (0'0,244'3150] MIN to 281'5553 2024-09-16T09:35:08.173 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:07 smithi070 ceph-mon[40691]: 9.10 starting backfill to osd.6 from (0'0,244'3213] MIN to 281'5546 2024-09-16T09:35:08.173 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:07 smithi070 ceph-mon[40691]: 9.11 scrub starts 2024-09-16T09:35:08.234 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:35:08.236 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:35:08.283 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:08.284 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:08.284 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:35:08.388 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:35:08.416 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:08.417 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:08.417 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:35:08.518 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:35:08.527 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T09:35:08.544 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:08.545 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:08.545 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:35:08.613 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.5 dump_historic_ops 2024-09-16T09:35:08.647 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:35:08.690 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:08.691 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:08.691 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:35:08.792 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:35:08.819 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:08.819 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:08.820 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:35:08.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:08 smithi084 ceph-mon[32629]: pgmap v502: 178 pgs: 1 active+recovering+undersized+remapped, 177 active+clean; 7.2 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 0 B/s wr, 80 op/s; 1053/355086 objects misplaced (0.297%) 2024-09-16T09:35:08.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:08 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:35:08.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:08 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:35:08.921 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:35:08.949 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:35:08.951 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:08.951 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:08.952 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:35:09.021 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:35:09.053 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:35:09.080 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:09.081 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:09.081 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:35:09.162 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:08 smithi070 ceph-mon[30388]: pgmap v502: 178 pgs: 1 active+recovering+undersized+remapped, 177 active+clean; 7.2 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 0 B/s wr, 80 op/s; 1053/355086 objects misplaced (0.297%) 2024-09-16T09:35:09.162 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:08 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:35:09.162 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:08 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:35:09.162 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:08 smithi070 ceph-mon[40691]: pgmap v502: 178 pgs: 1 active+recovering+undersized+remapped, 177 active+clean; 7.2 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 0 B/s wr, 80 op/s; 1053/355086 objects misplaced (0.297%) 2024-09-16T09:35:09.162 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:08 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:35:09.162 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:08 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:35:09.182 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:35:09.217 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:09.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:09.218 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:35:09.319 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:35:09.349 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:09.349 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:09.350 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:35:09.451 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:35:09.481 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:09.482 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:09.482 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:35:09.501 INFO:teuthology.orchestra.run.smithi084.stdout:ERROR: (22) Invalid argument 2024-09-16T09:35:09.501 INFO:teuthology.orchestra.run.smithi084.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:35:09.583 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:35:09.608 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:09.609 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:09.610 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:35:09.636 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:35:09.637 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:35:09.667 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:35:09.667 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:35:09.695 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:35:09.696 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:35:09.699 INFO:teuthology.orchestra.run.smithi070.stderr:noscrub is unset 2024-09-16T09:35:09.710 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:35:09.736 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:35:09.737 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:35:09.738 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:09 smithi070 ceph-mon[30388]: osdmap e285: 8 total, 8 up, 8 in 2024-09-16T09:35:09.739 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:09 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1674012361' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:35:09.739 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:09 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:35:09.739 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:09 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1690047365' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:35:09.739 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:09 smithi070 ceph-mon[40691]: osdmap e285: 8 total, 8 up, 8 in 2024-09-16T09:35:09.739 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:09 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1674012361' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:35:09.739 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:09 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:35:09.739 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:09 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1690047365' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:35:09.745 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:09.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:09.746 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:35:09.776 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:35:09.776 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:35:09.815 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:35:09.815 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:35:09.818 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T09:35:09.818 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T09:35:09.818 INFO:tasks.thrashosds.thrasher:Shrinking pool unique_pool_2 2024-09-16T09:35:09.818 INFO:tasks.cephadm.ceph_manager.ceph:contract_pool unique_pool_2 by 10 min 8 2024-09-16T09:35:09.818 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-16T09:35:09.847 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:35:09.850 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:35:09.850 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:35:09.874 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:09.874 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:09.875 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:35:09.887 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:35:09.887 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:35:09.976 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:35:10.041 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:10.042 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:10.042 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:35:10.143 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:35:10.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:09 smithi084 ceph-mon[32629]: osdmap e285: 8 total, 8 up, 8 in 2024-09-16T09:35:10.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:09 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1674012361' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:35:10.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:09 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:35:10.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:09 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1690047365' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:35:10.200 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:10.200 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:10.201 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:35:10.304 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:35:10.338 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:10.339 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:10.340 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:35:10.353 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.6 dump_ops_in_flight 2024-09-16T09:35:10.441 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:35:10.483 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:10.484 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:10.484 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:35:10.585 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:35:10.620 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:10.621 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:10.621 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:35:10.722 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:35:10.768 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:10.769 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:10.769 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:35:10.792 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:35:10.870 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:35:10.908 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:10.909 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:10.909 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:35:10.980 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:10 smithi084 ceph-mon[32629]: pgmap v504: 178 pgs: 2 active+remapped+backfilling, 2 active+recovering+undersized+degraded+remapped, 2 active+recovering+undersized+remapped, 3 active+remapped+backfill_toofull, 169 active+clean; 7.0 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 1018 B/s rd, 0 B/s wr, 698 op/s; 9/345543 objects degraded (0.003%); 56978/345543 objects misplaced (16.489%); 17 MiB/s, 270 objects/s recovering 2024-09-16T09:35:10.980 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:10 smithi084 ceph-mon[32629]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:35:10.980 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:10 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-16T09:35:10.980 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:10 smithi084 ceph-mon[32629]: osdmap e286: 8 total, 8 up, 8 in 2024-09-16T09:35:10.981 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:10 smithi084 ceph-mon[32629]: Health check failed: Low space hindering backfill (add storage if this doesn't resolve itself): 3 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-16T09:35:10.981 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:10 smithi084 ceph-mon[32629]: Health check update: Degraded data redundancy: 9/345543 objects degraded (0.003%), 2 pgs degraded, 1 pg undersized (PG_DEGRADED) 2024-09-16T09:35:10.981 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:10 smithi070 ceph-mon[30388]: pgmap v504: 178 pgs: 2 active+remapped+backfilling, 2 active+recovering+undersized+degraded+remapped, 2 active+recovering+undersized+remapped, 3 active+remapped+backfill_toofull, 169 active+clean; 7.0 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 1018 B/s rd, 0 B/s wr, 698 op/s; 9/345543 objects degraded (0.003%); 56978/345543 objects misplaced (16.489%); 17 MiB/s, 270 objects/s recovering 2024-09-16T09:35:10.981 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:10 smithi070 ceph-mon[30388]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:35:10.981 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:10 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-16T09:35:10.982 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:10 smithi070 ceph-mon[30388]: osdmap e286: 8 total, 8 up, 8 in 2024-09-16T09:35:10.982 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:10 smithi070 ceph-mon[30388]: Health check failed: Low space hindering backfill (add storage if this doesn't resolve itself): 3 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-16T09:35:10.982 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:10 smithi070 ceph-mon[30388]: Health check update: Degraded data redundancy: 9/345543 objects degraded (0.003%), 2 pgs degraded, 1 pg undersized (PG_DEGRADED) 2024-09-16T09:35:10.982 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:10 smithi070 ceph-mon[40691]: pgmap v504: 178 pgs: 2 active+remapped+backfilling, 2 active+recovering+undersized+degraded+remapped, 2 active+recovering+undersized+remapped, 3 active+remapped+backfill_toofull, 169 active+clean; 7.0 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 1018 B/s rd, 0 B/s wr, 698 op/s; 9/345543 objects degraded (0.003%); 56978/345543 objects misplaced (16.489%); 17 MiB/s, 270 objects/s recovering 2024-09-16T09:35:10.982 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:10 smithi070 ceph-mon[40691]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:35:10.982 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:10 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-16T09:35:10.982 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:10 smithi070 ceph-mon[40691]: osdmap e286: 8 total, 8 up, 8 in 2024-09-16T09:35:10.982 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:10 smithi070 ceph-mon[40691]: Health check failed: Low space hindering backfill (add storage if this doesn't resolve itself): 3 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-16T09:35:10.982 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:10 smithi070 ceph-mon[40691]: Health check update: Degraded data redundancy: 9/345543 objects degraded (0.003%), 2 pgs degraded, 1 pg undersized (PG_DEGRADED) 2024-09-16T09:35:11.011 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:35:11.044 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:11.044 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:11.045 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:35:11.146 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:35:11.178 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:11.178 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:11.179 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:35:11.279 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:35:11.314 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:11.314 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:11.315 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:35:11.404 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-09-16T09:35:11.404 INFO:teuthology.orchestra.run.smithi084.stdout: "ops": [ 2024-09-16T09:35:11.404 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:11.404 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:130808 9.0 9:00e551a2:::benchmark_data_smithi070_137000_object9097:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e286)", 2024-09-16T09:35:11.404 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:35:11.393470+0000", 2024-09-16T09:35:11.404 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 0.0078126429999999993, 2024-09-16T09:35:11.404 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0078406829999999993, 2024-09-16T09:35:11.404 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:35:11.404 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:35:11.404 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "waiting for sub ops", 2024-09-16T09:35:11.404 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:35:11.405 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:35:11.405 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:35:11.405 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 130808 2024-09-16T09:35:11.405 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:11.405 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:35:11.405 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:11.405 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:35:11.405 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:35:11.393470+0000", 2024-09-16T09:35:11.405 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:35:11.405 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:11.405 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:11.405 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:35:11.405 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:35:11.393470+0000", 2024-09-16T09:35:11.405 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:35:11.406 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:11.406 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:11.406 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:35:11.406 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:35:11.393472+0000", 2024-09-16T09:35:11.406 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.5799999999999999e-06 2024-09-16T09:35:11.406 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:11.406 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:11.406 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:35:11.406 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:35:11.393479+0000", 2024-09-16T09:35:11.406 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.8260000000000004e-06 2024-09-16T09:35:11.406 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:11.406 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:11.406 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:35:11.407 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:35:11.393500+0000", 2024-09-16T09:35:11.407 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.0469e-05 2024-09-16T09:35:11.407 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:11.407 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:11.407 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:35:11.407 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:35:11.393509+0000", 2024-09-16T09:35:11.407 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.1379999999999996e-06 2024-09-16T09:35:11.407 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:11.407 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:11.407 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:35:11.407 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:35:11.393574+0000", 2024-09-16T09:35:11.407 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.5630999999999996e-05 2024-09-16T09:35:11.407 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:11.407 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:11.408 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:35:11.408 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:35:11.393702+0000", 2024-09-16T09:35:11.408 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00012807500000000001 2024-09-16T09:35:11.408 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:11.408 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:11.408 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 1,2", 2024-09-16T09:35:11.408 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:35:11.394084+0000", 2024-09-16T09:35:11.408 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00038146099999999999 2024-09-16T09:35:11.408 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:11.408 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:11.408 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:35:11.408 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:35:11.398495+0000", 2024-09-16T09:35:11.408 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0044116069999999997 2024-09-16T09:35:11.409 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:35:11.409 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:35:11.409 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:35:11.409 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:11.409 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:11.409 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:130811 9.0 9:035c8abe:::benchmark_data_smithi070_137000_object9100:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e286)", 2024-09-16T09:35:11.409 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:35:11.395588+0000", 2024-09-16T09:35:11.409 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 0.0056947100000000004, 2024-09-16T09:35:11.409 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.005799144, 2024-09-16T09:35:11.409 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:35:11.409 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:35:11.409 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "waiting for sub ops", 2024-09-16T09:35:11.409 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:35:11.409 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:35:11.410 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:35:11.410 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 130811 2024-09-16T09:35:11.410 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:11.410 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:35:11.410 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:11.410 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:35:11.410 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:35:11.395588+0000", 2024-09-16T09:35:11.410 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:35:11.410 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:11.410 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:11.410 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:35:11.410 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:35:11.395588+0000", 2024-09-16T09:35:11.410 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:35:11.410 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:11.411 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:11.411 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:35:11.411 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:35:11.395589+0000", 2024-09-16T09:35:11.411 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1540000000000001e-06 2024-09-16T09:35:11.411 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:11.411 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:11.411 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:35:11.411 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:35:11.395594+0000", 2024-09-16T09:35:11.411 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.074e-06 2024-09-16T09:35:11.412 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:11.412 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:11.412 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:35:11.412 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:35:11.395610+0000", 2024-09-16T09:35:11.412 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.6022e-05 2024-09-16T09:35:11.412 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:11.412 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:11.412 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:35:11.412 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:35:11.395618+0000", 2024-09-16T09:35:11.412 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.2579999999999995e-06 2024-09-16T09:35:11.412 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:11.412 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:11.412 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:35:11.413 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:35:11.396386+0000", 2024-09-16T09:35:11.413 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00076806800000000001 2024-09-16T09:35:11.413 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:11.413 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:11.413 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:35:11.413 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:35:11.396499+0000", 2024-09-16T09:35:11.413 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000112617 2024-09-16T09:35:11.414 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:11.414 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:11.414 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 1,2", 2024-09-16T09:35:11.414 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:35:11.396812+0000", 2024-09-16T09:35:11.414 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00031319699999999998 2024-09-16T09:35:11.414 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:11.414 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:11.414 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:35:11.414 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:35:11.398642+0000", 2024-09-16T09:35:11.414 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00183051 2024-09-16T09:35:11.414 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:35:11.414 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:35:11.414 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:35:11.414 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:11.415 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:11.415 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_repop(client.15588.0:130731 9.10 e286/283 9:0b04e84e:::benchmark_data_smithi070_137000_object9020:head v 286'5669, mlcod=235'617)", 2024-09-16T09:35:11.415 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:35:11.398945+0000", 2024-09-16T09:35:11.415 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 0.0023370629999999999, 2024-09-16T09:35:11.415 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0025076170000000002, 2024-09-16T09:35:11.415 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:35:11.415 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:35:11.415 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "started", 2024-09-16T09:35:11.415 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:35:11.415 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:11.415 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:35:11.415 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:35:11.398945+0000", 2024-09-16T09:35:11.415 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:35:11.415 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:11.416 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:11.416 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:35:11.416 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:35:11.398945+0000", 2024-09-16T09:35:11.416 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:35:11.416 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:11.416 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:11.416 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:35:11.416 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:35:11.398947+0000", 2024-09-16T09:35:11.416 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.2869999999999999e-06 2024-09-16T09:35:11.416 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:11.416 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:11.416 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:35:11.417 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:35:11.398953+0000", 2024-09-16T09:35:11.417 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.0630000000000002e-06 2024-09-16T09:35:11.417 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:11.417 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:11.417 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:35:11.417 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:35:11.398965+0000", 2024-09-16T09:35:11.418 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.201e-05 2024-09-16T09:35:11.418 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:11.418 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:11.418 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:35:11.418 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:35:11.398972+0000", 2024-09-16T09:35:11.418 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.9380000000000003e-06 2024-09-16T09:35:11.418 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:11.419 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:11.419 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:35:11.419 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:35:11.399260+0000", 2024-09-16T09:35:11.419 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00028868699999999998 2024-09-16T09:35:11.419 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:11.419 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:11.419 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:35:11.419 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:35:11.399280+0000", 2024-09-16T09:35:11.419 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.9694000000000002e-05 2024-09-16T09:35:11.419 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:35:11.420 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:35:11.420 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:35:11.420 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:11.420 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:11.420 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_repop(client.15588.0:130813 9.19 e286/283 9:9bd150ff:::benchmark_data_smithi070_137000_object9102:head v 286'5618, mlcod=286'5616)", 2024-09-16T09:35:11.420 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:35:11.400911+0000", 2024-09-16T09:35:11.420 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 0.00037134599999999999, 2024-09-16T09:35:11.420 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00059325199999999995, 2024-09-16T09:35:11.420 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:35:11.420 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:35:11.420 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "started", 2024-09-16T09:35:11.421 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:35:11.421 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:11.421 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:35:11.421 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:35:11.400911+0000", 2024-09-16T09:35:11.421 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:35:11.421 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:11.421 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:11.421 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:35:11.421 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:35:11.400911+0000", 2024-09-16T09:35:11.421 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:35:11.421 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:11.422 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:11.422 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:35:11.422 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:35:11.400912+0000", 2024-09-16T09:35:11.422 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.339e-06 2024-09-16T09:35:11.422 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:11.422 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:11.422 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:35:11.422 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:35:11.400920+0000", 2024-09-16T09:35:11.422 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.4830000000000002e-06 2024-09-16T09:35:11.422 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:11.422 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:11.423 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:35:11.423 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:35:11.400937+0000", 2024-09-16T09:35:11.423 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.6744000000000001e-05 2024-09-16T09:35:11.423 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:11.423 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:11.423 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:35:11.423 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:35:11.400945+0000", 2024-09-16T09:35:11.423 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.7329999999999992e-06 2024-09-16T09:35:11.423 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:11.423 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:11.423 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:35:11.423 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:35:11.401027+0000", 2024-09-16T09:35:11.423 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.1398999999999998e-05 2024-09-16T09:35:11.424 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:35:11.424 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:35:11.424 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:35:11.424 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:35:11.401049+0000", 2024-09-16T09:35:11.424 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.2177000000000001e-05 2024-09-16T09:35:11.424 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:35:11.424 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:35:11.424 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:35:11.424 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:35:11.424 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-09-16T09:35:11.424 INFO:teuthology.orchestra.run.smithi084.stdout: "num_ops": 4 2024-09-16T09:35:11.424 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-09-16T09:35:11.425 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:35:11.459 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:11.460 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:11.460 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:35:11.491 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:35:11.560 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:35:11.585 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:11.586 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:11.587 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:35:11.687 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:35:11.714 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:11.715 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:11.715 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:35:11.816 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:35:11.858 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:11.859 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:11.859 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:35:11.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:11 smithi070 ceph-mon[30388]: 5.e scrub starts 2024-09-16T09:35:11.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:11 smithi070 ceph-mon[30388]: 5.e scrub ok 2024-09-16T09:35:11.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:11 smithi070 ceph-mon[40691]: 5.e scrub starts 2024-09-16T09:35:11.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:11 smithi070 ceph-mon[40691]: 5.e scrub ok 2024-09-16T09:35:11.959 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:35:11.992 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:11.993 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:11.993 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:35:12.094 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:35:12.121 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:12.121 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:12.122 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:35:12.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:11 smithi084 ceph-mon[32629]: 5.e scrub starts 2024-09-16T09:35:12.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:11 smithi084 ceph-mon[32629]: 5.e scrub ok 2024-09-16T09:35:12.223 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:35:12.252 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:12.253 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:12.253 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:35:12.354 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:35:12.380 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:12.380 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:12.381 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:35:12.421 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:35:11 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:35:11] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:35:12.463 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:35:12.466 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2024-09-16T09:35:12.481 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:35:12.512 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:12.512 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:12.513 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:35:12.613 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:35:12.640 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:12.641 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:12.641 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:35:12.742 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:35:12.782 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:12.783 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:12.783 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:35:12.856 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T09:35:12.884 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:35:12.914 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:12.914 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:12.915 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:35:13.016 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:35:13.037 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:13.038 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:13.038 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:35:13.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:12 smithi084 ceph-mon[32629]: pgmap v506: 178 pgs: 2 active+remapped+backfilling, 2 active+recovering+undersized+degraded+remapped, 2 active+recovering+undersized+remapped, 3 active+remapped+backfill_toofull, 169 active+clean; 7.0 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 585 op/s; 9/345543 objects degraded (0.003%); 56978/345543 objects misplaced (16.489%); 14 MiB/s, 226 objects/s recovering 2024-09-16T09:35:13.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:12 smithi084 ceph-mon[32629]: 6.1a scrub starts 2024-09-16T09:35:13.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:12 smithi084 ceph-mon[32629]: 6.1a scrub ok 2024-09-16T09:35:13.125 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:12 smithi084 ceph-mon[32629]: from='client.15735 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:35:13.139 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:35:13.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:12 smithi070 ceph-mon[30388]: pgmap v506: 178 pgs: 2 active+remapped+backfilling, 2 active+recovering+undersized+degraded+remapped, 2 active+recovering+undersized+remapped, 3 active+remapped+backfill_toofull, 169 active+clean; 7.0 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 585 op/s; 9/345543 objects degraded (0.003%); 56978/345543 objects misplaced (16.489%); 14 MiB/s, 226 objects/s recovering 2024-09-16T09:35:13.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:12 smithi070 ceph-mon[30388]: 6.1a scrub starts 2024-09-16T09:35:13.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:12 smithi070 ceph-mon[30388]: 6.1a scrub ok 2024-09-16T09:35:13.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:12 smithi070 ceph-mon[30388]: from='client.15735 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:35:13.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:12 smithi070 ceph-mon[40691]: pgmap v506: 178 pgs: 2 active+remapped+backfilling, 2 active+recovering+undersized+degraded+remapped, 2 active+recovering+undersized+remapped, 3 active+remapped+backfill_toofull, 169 active+clean; 7.0 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 585 op/s; 9/345543 objects degraded (0.003%); 56978/345543 objects misplaced (16.489%); 14 MiB/s, 226 objects/s recovering 2024-09-16T09:35:13.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:12 smithi070 ceph-mon[40691]: 6.1a scrub starts 2024-09-16T09:35:13.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:12 smithi070 ceph-mon[40691]: 6.1a scrub ok 2024-09-16T09:35:13.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:12 smithi070 ceph-mon[40691]: from='client.15735 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:35:13.186 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:13.187 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:13.187 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:35:13.288 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:35:13.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:13.322 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:13.322 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:35:13.377 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-16T09:35:13.410 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:35:13 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:35:13.423 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:35:13.461 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:13.462 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:13.462 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:35:13.562 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":506,"stamp":"2024-09-16T09:35:11.690343+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":7535958579,"num_objects":115181,"num_object_clones":0,"num_object_copies":345543,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9,"num_objects_misplaced":56978,"num_objects_unfound":0,"num_objects_dirty":115181,"num_whiteouts":0,"num_read":1313,"num_read_kb":1065,"num_write":97190,"num_write_kb":5767054,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7282,"num_bytes_recovered":469504806,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":135971,"ondisk_log_size":135971,"up":534,"acting":530,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":549,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":26153868,"kb_used_data":22881960,"kb_used_omap":12,"kb_used_meta":3271667,"kb_avail":723807348,"statfs":{"total":767960285184,"available":741178724352,"internally_reserved":0,"allocated":23431127040,"data_stored":23426060777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":3350187570},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-229769216,"num_objects":-3506,"num_object_clones":0,"num_object_copies":-10518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9,"num_objects_misplaced":55925,"num_objects_unfound":0,"num_objects_dirty":-3506,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":3506,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1358,"num_bytes_recovered":88866816,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001481"},"pg_stats":[{"pgid":"9.15","version":"284'5602","reported_seq":11119,"reported_epoch":284,"state":"active+remapped+backfilling","last_fresh":"2024-09-16T09:35:08.094708+0000","last_change":"2024-09-16T09:35:06.857486+0000","last_active":"2024-09-16T09:35:08.094708+0000","last_peered":"2024-09-16T09:35:08.094708+0000","last_clean":"2024-09-16T09:35:04.657830+0000","last_became_active":"2024-09-16T09:35:06.770018+0000","last_became_peered":"2024-09-16T09:35:06.770018+0000","last_unstale":"2024-09-16T09:35:08.094708+0000","last_undegraded":"2024-09-16T09:35:08.094708+0000","last_fullsized":"2024-09-16T09:35:08.094708+0000","mapping_epoch":283,"log_start":"235'419","ondisk_log_start":"235'419","created":243,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:33:24.230486+0000","objects_scrubbed":0,"log_size":5183,"log_dups_size":0,"ondisk_log_size":5183,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":235667456,"num_objects":3596,"num_object_clones":0,"num_object_copies":10788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":6932,"num_objects_unfound":0,"num_objects_dirty":3596,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4028,"num_write_kb":243968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1787,"num_bytes_recovered":117112832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[1,3,7],"avail_no_missing":["1","3","7"],"object_location_counts":[{"shards":"1,3,7","objects":3596}],"blocked_by":[],"up_primary":0,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":498,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.668599+0000","last_change":"2024-09-16T09:32:39.384174+0000","last_active":"2024-09-16T09:35:06.668599+0000","last_peered":"2024-09-16T09:35:06.668599+0000","last_clean":"2024-09-16T09:35:06.668599+0000","last_became_active":"2024-09-16T09:25:01.709108+0000","last_became_peered":"2024-09-16T09:25:01.709108+0000","last_unstale":"2024-09-16T09:35:06.668599+0000","last_undegraded":"2024-09-16T09:35:06.668599+0000","last_fullsized":"2024-09-16T09:35:06.668599+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:39.384104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:32:39.384104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"66'9","reported_seq":502,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.663787+0000","last_change":"2024-09-16T09:32:38.074301+0000","last_active":"2024-09-16T09:35:06.663787+0000","last_peered":"2024-09-16T09:35:06.663787+0000","last_clean":"2024-09-16T09:35:06.663787+0000","last_became_active":"2024-09-16T09:25:03.714635+0000","last_became_peered":"2024-09-16T09:25:03.714635+0000","last_unstale":"2024-09-16T09:35:06.663787+0000","last_undegraded":"2024-09-16T09:35:06.663787+0000","last_fullsized":"2024-09-16T09:35:06.663787+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:32:38.074231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:32:38.074231+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":113,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.668232+0000","last_change":"2024-09-16T09:33:30.242561+0000","last_active":"2024-09-16T09:35:06.668232+0000","last_peered":"2024-09-16T09:35:06.668232+0000","last_clean":"2024-09-16T09:35:06.668232+0000","last_became_active":"2024-09-16T09:33:30.242238+0000","last_became_peered":"2024-09-16T09:33:30.242238+0000","last_unstale":"2024-09-16T09:35:06.668232+0000","last_undegraded":"2024-09-16T09:35:06.668232+0000","last_fullsized":"2024-09-16T09:35:06.668232+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:13.426830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:13.426830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":462,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.663819+0000","last_change":"2024-09-16T09:27:11.565306+0000","last_active":"2024-09-16T09:35:06.663819+0000","last_peered":"2024-09-16T09:35:06.663819+0000","last_clean":"2024-09-16T09:35:06.663819+0000","last_became_active":"2024-09-16T09:25:07.715442+0000","last_became_peered":"2024-09-16T09:25:07.715442+0000","last_unstale":"2024-09-16T09:35:06.663819+0000","last_undegraded":"2024-09-16T09:35:06.663819+0000","last_fullsized":"2024-09-16T09:35:06.663819+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:11.565220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:11.565220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"284'5652","reported_seq":5819,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:07.809091+0000","last_change":"2024-09-16T09:35:05.669999+0000","last_active":"2024-09-16T09:35:07.809091+0000","last_peered":"2024-09-16T09:35:07.809091+0000","last_clean":"2024-09-16T09:35:07.809091+0000","last_became_active":"2024-09-16T09:33:52.932390+0000","last_became_peered":"2024-09-16T09:33:52.932390+0000","last_unstale":"2024-09-16T09:35:07.809091+0000","last_undegraded":"2024-09-16T09:35:07.809091+0000","last_fullsized":"2024-09-16T09:35:07.809091+0000","mapping_epoch":243,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:33:24.230486+0000","objects_scrubbed":0,"log_size":5652,"log_dups_size":0,"ondisk_log_size":5652,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:35:34.853459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":237797376,"num_objects":3629,"num_object_clones":0,"num_object_copies":10887,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3629,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4051,"num_write_kb":245728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":215,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.663846+0000","last_change":"2024-09-16T09:34:33.452784+0000","last_active":"2024-09-16T09:35:06.663846+0000","last_peered":"2024-09-16T09:35:06.663846+0000","last_clean":"2024-09-16T09:35:06.663846+0000","last_became_active":"2024-09-16T09:31:09.269342+0000","last_became_peered":"2024-09-16T09:31:09.269342+0000","last_unstale":"2024-09-16T09:35:06.663846+0000","last_undegraded":"2024-09-16T09:35:06.663846+0000","last_fullsized":"2024-09-16T09:35:06.663846+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:33.452706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:34:33.452706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:35:54.801370+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":356,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.684761+0000","last_change":"2024-09-16T09:33:08.816231+0000","last_active":"2024-09-16T09:35:06.684761+0000","last_peered":"2024-09-16T09:35:06.684761+0000","last_clean":"2024-09-16T09:35:06.684761+0000","last_became_active":"2024-09-16T09:31:09.275136+0000","last_became_peered":"2024-09-16T09:31:09.275136+0000","last_unstale":"2024-09-16T09:35:06.684761+0000","last_undegraded":"2024-09-16T09:35:06.684761+0000","last_fullsized":"2024-09-16T09:35:06.684761+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:08.816174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:08.816174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:36:00.908210+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.19","version":"66'15","reported_seq":170,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.668904+0000","last_change":"2024-09-16T09:33:21.337711+0000","last_active":"2024-09-16T09:35:06.668904+0000","last_peered":"2024-09-16T09:35:06.668904+0000","last_clean":"2024-09-16T09:35:06.668904+0000","last_became_active":"2024-09-16T09:33:21.337284+0000","last_became_peered":"2024-09-16T09:33:21.337284+0000","last_unstale":"2024-09-16T09:35:06.668904+0000","last_undegraded":"2024-09-16T09:35:06.668904+0000","last_fullsized":"2024-09-16T09:35:06.668904+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-16T09:32:45.194581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:32:45.194581+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":490,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.663672+0000","last_change":"2024-09-16T09:32:44.114475+0000","last_active":"2024-09-16T09:35:06.663672+0000","last_peered":"2024-09-16T09:35:06.663672+0000","last_clean":"2024-09-16T09:35:06.663672+0000","last_became_active":"2024-09-16T09:25:05.717236+0000","last_became_peered":"2024-09-16T09:25:05.717236+0000","last_unstale":"2024-09-16T09:35:06.663672+0000","last_undegraded":"2024-09-16T09:35:06.663672+0000","last_fullsized":"2024-09-16T09:35:06.663672+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:44.114407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:32:44.114407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.17","version":"284'5592","reported_seq":11222,"reported_epoch":284,"state":"active+remapped+backfilling","last_fresh":"2024-09-16T09:35:08.565113+0000","last_change":"2024-09-16T09:35:06.854529+0000","last_active":"2024-09-16T09:35:08.565113+0000","last_peered":"2024-09-16T09:35:08.565113+0000","last_clean":"2024-09-16T09:35:04.640571+0000","last_became_active":"2024-09-16T09:35:06.801573+0000","last_became_peered":"2024-09-16T09:35:06.801573+0000","last_unstale":"2024-09-16T09:35:08.565113+0000","last_undegraded":"2024-09-16T09:35:08.565113+0000","last_fullsized":"2024-09-16T09:35:08.565113+0000","mapping_epoch":283,"log_start":"235'446","ondisk_log_start":"235'446","created":243,"last_epoch_clean":261,"parent":"0.0","parent_split_bits":5,"last_scrub":"276'5145","last_scrub_stamp":"2024-09-16T09:34:53.523609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:34:53.523609+0000","objects_scrubbed":3599,"log_size":5146,"log_dups_size":0,"ondisk_log_size":5146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":964,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":239665152,"num_objects":3657,"num_object_clones":0,"num_object_copies":10971,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":6936,"num_objects_unfound":0,"num_objects_dirty":3657,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":464,"num_write_kb":16704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":190,"num_bytes_recovered":12451840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[0,7,1],"avail_no_missing":["0","1","7"],"object_location_counts":[{"shards":"0,1,7","objects":3656},{"shards":"5","objects":1}],"blocked_by":[],"up_primary":5,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":122,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.679183+0000","last_change":"2024-09-16T09:33:36.655594+0000","last_active":"2024-09-16T09:35:06.679183+0000","last_peered":"2024-09-16T09:35:06.679183+0000","last_clean":"2024-09-16T09:35:06.679183+0000","last_became_active":"2024-09-16T09:33:30.247733+0000","last_became_peered":"2024-09-16T09:33:30.247733+0000","last_unstale":"2024-09-16T09:35:06.679183+0000","last_undegraded":"2024-09-16T09:35:06.679183+0000","last_fullsized":"2024-09-16T09:35:06.679183+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:36.655236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:36.655236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1a","version":"66'9","reported_seq":502,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.663942+0000","last_change":"2024-09-16T09:32:48.185484+0000","last_active":"2024-09-16T09:35:06.663942+0000","last_peered":"2024-09-16T09:35:06.663942+0000","last_clean":"2024-09-16T09:35:06.663942+0000","last_became_active":"2024-09-16T09:25:03.709147+0000","last_became_peered":"2024-09-16T09:25:03.709147+0000","last_unstale":"2024-09-16T09:35:06.663942+0000","last_undegraded":"2024-09-16T09:35:06.663942+0000","last_fullsized":"2024-09-16T09:35:06.663942+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:32:48.185380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:32:48.185380+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":473,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.672248+0000","last_change":"2024-09-16T09:33:21.376175+0000","last_active":"2024-09-16T09:35:06.672248+0000","last_peered":"2024-09-16T09:35:06.672248+0000","last_clean":"2024-09-16T09:35:06.672248+0000","last_became_active":"2024-09-16T09:25:05.718134+0000","last_became_peered":"2024-09-16T09:25:05.718134+0000","last_unstale":"2024-09-16T09:35:06.672248+0000","last_undegraded":"2024-09-16T09:35:06.672248+0000","last_fullsized":"2024-09-16T09:35:06.672248+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:21.376128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:21.376128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"66'1","reported_seq":487,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.930361+0000","last_change":"2024-09-16T09:33:38.705001+0000","last_active":"2024-09-16T09:35:06.930361+0000","last_peered":"2024-09-16T09:35:06.930361+0000","last_clean":"2024-09-16T09:35:06.930361+0000","last_became_active":"2024-09-16T09:25:07.716558+0000","last_became_peered":"2024-09-16T09:25:07.716558+0000","last_unstale":"2024-09-16T09:35:06.930361+0000","last_undegraded":"2024-09-16T09:35:06.930361+0000","last_fullsized":"2024-09-16T09:35:06.930361+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-16T09:33:38.704868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:33:38.704868+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"284'5640","reported_seq":8963,"reported_epoch":284,"state":"active+remapped+backfill_toofull","last_fresh":"2024-09-16T09:35:07.967404+0000","last_change":"2024-09-16T09:35:06.764592+0000","last_active":"2024-09-16T09:35:07.967404+0000","last_peered":"2024-09-16T09:35:07.967404+0000","last_clean":"2024-09-16T09:35:04.664094+0000","last_became_active":"2024-09-16T09:35:06.748050+0000","last_became_peered":"2024-09-16T09:35:06.748050+0000","last_unstale":"2024-09-16T09:35:07.967404+0000","last_undegraded":"2024-09-16T09:35:07.967404+0000","last_fullsized":"2024-09-16T09:35:07.967404+0000","mapping_epoch":283,"log_start":"235'259","ondisk_log_start":"235'259","created":243,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":5,"last_scrub":"273'4808","last_scrub_stamp":"2024-09-16T09:34:44.425779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:34:44.425779+0000","objects_scrubbed":3181,"log_size":5381,"log_dups_size":0,"ondisk_log_size":5381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":809,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":236126208,"num_objects":3603,"num_object_clones":0,"num_object_copies":10809,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3603,"num_objects_unfound":0,"num_objects_dirty":3603,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":846,"num_write_kb":40576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[1,2,4],"avail_no_missing":["1","2","4"],"object_location_counts":[{"shards":"1,2,4","objects":3603}],"blocked_by":[],"up_primary":5,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":481,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.667466+0000","last_change":"2024-09-16T09:33:19.326765+0000","last_active":"2024-09-16T09:35:06.667466+0000","last_peered":"2024-09-16T09:35:06.667466+0000","last_clean":"2024-09-16T09:35:06.667466+0000","last_became_active":"2024-09-16T09:25:01.696939+0000","last_became_peered":"2024-09-16T09:25:01.696939+0000","last_unstale":"2024-09-16T09:35:06.667466+0000","last_undegraded":"2024-09-16T09:35:06.667466+0000","last_fullsized":"2024-09-16T09:35:06.667466+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:19.326695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:19.326695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"66'5","reported_seq":488,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.663940+0000","last_change":"2024-09-16T09:32:31.024521+0000","last_active":"2024-09-16T09:35:06.663940+0000","last_peered":"2024-09-16T09:35:06.663940+0000","last_clean":"2024-09-16T09:35:06.663940+0000","last_became_active":"2024-09-16T09:25:03.716136+0000","last_became_peered":"2024-09-16T09:25:03.716136+0000","last_unstale":"2024-09-16T09:35:06.663940+0000","last_undegraded":"2024-09-16T09:35:06.663940+0000","last_fullsized":"2024-09-16T09:35:06.663940+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'5","last_scrub_stamp":"2024-09-16T09:32:31.024450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:32:31.024450+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":473,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.669023+0000","last_change":"2024-09-16T09:28:52.746046+0000","last_active":"2024-09-16T09:35:06.669023+0000","last_peered":"2024-09-16T09:35:06.669023+0000","last_clean":"2024-09-16T09:35:06.669023+0000","last_became_active":"2024-09-16T09:25:05.718281+0000","last_became_peered":"2024-09-16T09:25:05.718281+0000","last_unstale":"2024-09-16T09:35:06.669023+0000","last_undegraded":"2024-09-16T09:35:06.669023+0000","last_fullsized":"2024-09-16T09:35:06.669023+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:52.745999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:28:52.745999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":499,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.668314+0000","last_change":"2024-09-16T09:33:46.366042+0000","last_active":"2024-09-16T09:35:06.668314+0000","last_peered":"2024-09-16T09:35:06.668314+0000","last_clean":"2024-09-16T09:35:06.668314+0000","last_became_active":"2024-09-16T09:31:09.275656+0000","last_became_peered":"2024-09-16T09:31:09.275656+0000","last_unstale":"2024-09-16T09:35:06.668314+0000","last_undegraded":"2024-09-16T09:35:06.668314+0000","last_fullsized":"2024-09-16T09:35:06.668314+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:46.365972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:33:46.365972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:35:13.224630+0000","scrub_duration":73,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"284'5499","reported_seq":5661,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:07.664456+0000","last_change":"2024-09-16T09:35:07.073656+0000","last_active":"2024-09-16T09:35:07.664456+0000","last_peered":"2024-09-16T09:35:07.664456+0000","last_clean":"2024-09-16T09:35:07.664456+0000","last_became_active":"2024-09-16T09:35:05.670682+0000","last_became_peered":"2024-09-16T09:35:05.670682+0000","last_unstale":"2024-09-16T09:35:07.664456+0000","last_undegraded":"2024-09-16T09:35:07.664456+0000","last_fullsized":"2024-09-16T09:35:07.664456+0000","mapping_epoch":282,"log_start":"235'304","ondisk_log_start":"235'304","created":243,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:33:24.230486+0000","objects_scrubbed":0,"log_size":5195,"log_dups_size":0,"ondisk_log_size":5195,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:35:36.992132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":234356736,"num_objects":3576,"num_object_clones":0,"num_object_copies":10728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3576,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3978,"num_write_kb":241728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":486,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.664608+0000","last_change":"2024-09-16T09:33:48.322010+0000","last_active":"2024-09-16T09:35:06.664608+0000","last_peered":"2024-09-16T09:35:06.664608+0000","last_clean":"2024-09-16T09:35:06.664608+0000","last_became_active":"2024-09-16T09:25:07.720793+0000","last_became_peered":"2024-09-16T09:25:07.720793+0000","last_unstale":"2024-09-16T09:35:06.664608+0000","last_undegraded":"2024-09-16T09:35:06.664608+0000","last_fullsized":"2024-09-16T09:35:06.664608+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:48.321895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:33:48.321895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":237,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.683434+0000","last_change":"2024-09-16T09:34:33.866299+0000","last_active":"2024-09-16T09:35:06.683434+0000","last_peered":"2024-09-16T09:35:06.683434+0000","last_clean":"2024-09-16T09:35:06.683434+0000","last_became_active":"2024-09-16T09:30:39.201824+0000","last_became_peered":"2024-09-16T09:30:39.201824+0000","last_unstale":"2024-09-16T09:35:06.683434+0000","last_undegraded":"2024-09-16T09:35:06.683434+0000","last_fullsized":"2024-09-16T09:35:06.683434+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:33.866142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:34:33.866142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:35:56.888572+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"66'15","reported_seq":560,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.691362+0000","last_change":"2024-09-16T09:33:30.250162+0000","last_active":"2024-09-16T09:35:06.691362+0000","last_peered":"2024-09-16T09:35:06.691362+0000","last_clean":"2024-09-16T09:35:06.691362+0000","last_became_active":"2024-09-16T09:33:30.249798+0000","last_became_peered":"2024-09-16T09:33:30.249798+0000","last_unstale":"2024-09-16T09:35:06.691362+0000","last_undegraded":"2024-09-16T09:35:06.691362+0000","last_fullsized":"2024-09-16T09:35:06.691362+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-16T09:32:49.202163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:32:49.202163+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:35:17.443981+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":121,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.664369+0000","last_change":"2024-09-16T09:33:43.326933+0000","last_active":"2024-09-16T09:35:06.664369+0000","last_peered":"2024-09-16T09:35:06.664369+0000","last_clean":"2024-09-16T09:35:06.664369+0000","last_became_active":"2024-09-16T09:33:30.251587+0000","last_became_peered":"2024-09-16T09:33:30.251587+0000","last_unstale":"2024-09-16T09:35:06.664369+0000","last_undegraded":"2024-09-16T09:35:06.664369+0000","last_fullsized":"2024-09-16T09:35:06.664369+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:43.326852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:43.326852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,0],"acting":[4,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.10","version":"284'5592","reported_seq":12467,"reported_epoch":284,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-16T09:35:08.503315+0000","last_change":"2024-09-16T09:35:07.064657+0000","last_active":"2024-09-16T09:35:08.503315+0000","last_peered":"2024-09-16T09:35:08.503315+0000","last_clean":"2024-09-16T09:35:04.646442+0000","last_became_active":"2024-09-16T09:35:06.938011+0000","last_became_peered":"2024-09-16T09:35:06.938011+0000","last_unstale":"2024-09-16T09:35:08.503315+0000","last_undegraded":"2024-09-16T09:35:08.503315+0000","last_fullsized":"2024-09-16T09:35:06.666095+0000","mapping_epoch":283,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":265,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:33:24.230486+0000","objects_scrubbed":0,"log_size":5592,"log_dups_size":0,"ondisk_log_size":5592,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":229638144,"num_objects":3504,"num_object_clones":0,"num_object_copies":10512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":10548,"num_objects_unfound":0,"num_objects_dirty":3504,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4010,"num_write_kb":240448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1694,"num_bytes_recovered":110952448,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,2],"acting":[0,1],"avail_no_missing":["0","4","7"],"object_location_counts":[{"shards":"0,1","objects":-12},{"shards":"0,4,7","objects":3516}],"blocked_by":[],"up_primary":6,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":335,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.684995+0000","last_change":"2024-09-16T09:34:37.804874+0000","last_active":"2024-09-16T09:35:06.684995+0000","last_peered":"2024-09-16T09:35:06.684995+0000","last_clean":"2024-09-16T09:35:06.684995+0000","last_became_active":"2024-09-16T09:28:52.974255+0000","last_became_peered":"2024-09-16T09:28:52.974255+0000","last_unstale":"2024-09-16T09:35:06.684995+0000","last_undegraded":"2024-09-16T09:35:06.684995+0000","last_fullsized":"2024-09-16T09:35:06.684995+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:37.804805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:34:37.804805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:36:05.263108+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":490,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.664104+0000","last_change":"2024-09-16T09:30:03.628882+0000","last_active":"2024-09-16T09:35:06.664104+0000","last_peered":"2024-09-16T09:35:06.664104+0000","last_clean":"2024-09-16T09:35:06.664104+0000","last_became_active":"2024-09-16T09:25:01.706491+0000","last_became_peered":"2024-09-16T09:25:01.706491+0000","last_unstale":"2024-09-16T09:35:06.664104+0000","last_undegraded":"2024-09-16T09:35:06.664104+0000","last_fullsized":"2024-09-16T09:35:06.664104+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:30:03.628812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:30:03.628812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"66'12","reported_seq":164,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.664475+0000","last_change":"2024-09-16T09:33:39.284526+0000","last_active":"2024-09-16T09:35:06.664475+0000","last_peered":"2024-09-16T09:35:06.664475+0000","last_clean":"2024-09-16T09:35:06.664475+0000","last_became_active":"2024-09-16T09:33:30.251626+0000","last_became_peered":"2024-09-16T09:33:30.251626+0000","last_unstale":"2024-09-16T09:35:06.664475+0000","last_undegraded":"2024-09-16T09:35:06.664475+0000","last_fullsized":"2024-09-16T09:35:06.664475+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-16T09:33:39.284451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:33:39.284451+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":24,"num_read_kb":15,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":474,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.664116+0000","last_change":"2024-09-16T09:33:38.238661+0000","last_active":"2024-09-16T09:35:06.664116+0000","last_peered":"2024-09-16T09:35:06.664116+0000","last_clean":"2024-09-16T09:35:06.664116+0000","last_became_active":"2024-09-16T09:25:05.710093+0000","last_became_peered":"2024-09-16T09:25:05.710093+0000","last_unstale":"2024-09-16T09:35:06.664116+0000","last_undegraded":"2024-09-16T09:35:06.664116+0000","last_fullsized":"2024-09-16T09:35:06.664116+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:38.238593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:38.238593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"284'5497","reported_seq":10451,"reported_epoch":284,"state":"active+remapped+backfill_toofull","last_fresh":"2024-09-16T09:35:07.842839+0000","last_change":"2024-09-16T09:35:06.794060+0000","last_active":"2024-09-16T09:35:07.842839+0000","last_peered":"2024-09-16T09:35:07.842839+0000","last_clean":"2024-09-16T09:35:04.649840+0000","last_became_active":"2024-09-16T09:35:06.787372+0000","last_became_peered":"2024-09-16T09:35:06.787372+0000","last_unstale":"2024-09-16T09:35:07.842839+0000","last_undegraded":"2024-09-16T09:35:07.842839+0000","last_fullsized":"2024-09-16T09:35:07.842839+0000","mapping_epoch":283,"log_start":"235'477","ondisk_log_start":"235'477","created":243,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":5,"last_scrub":"273'4719","last_scrub_stamp":"2024-09-16T09:34:44.662413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:34:44.662413+0000","objects_scrubbed":3181,"log_size":5020,"log_dups_size":0,"ondisk_log_size":5020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1093,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":237502464,"num_objects":3624,"num_object_clones":0,"num_object_copies":10872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":7248,"num_objects_unfound":0,"num_objects_dirty":3624,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":799,"num_write_kb":39744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,5,6],"avail_no_missing":["7","5","6"],"object_location_counts":[{"shards":"5,6,7","objects":3624}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":477,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.669363+0000","last_change":"2024-09-16T09:33:46.253182+0000","last_active":"2024-09-16T09:35:06.669363+0000","last_peered":"2024-09-16T09:35:06.669363+0000","last_clean":"2024-09-16T09:35:06.669363+0000","last_became_active":"2024-09-16T09:25:07.719380+0000","last_became_peered":"2024-09-16T09:25:07.719380+0000","last_unstale":"2024-09-16T09:35:06.669363+0000","last_undegraded":"2024-09-16T09:35:06.669363+0000","last_fullsized":"2024-09-16T09:35:06.669363+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-16T09:33:46.253114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:33:46.253114+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":130,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.663825+0000","last_change":"2024-09-16T09:34:41.401691+0000","last_active":"2024-09-16T09:35:06.663825+0000","last_peered":"2024-09-16T09:35:06.663825+0000","last_clean":"2024-09-16T09:35:06.663825+0000","last_became_active":"2024-09-16T09:33:30.245048+0000","last_became_peered":"2024-09-16T09:33:30.245048+0000","last_unstale":"2024-09-16T09:35:06.663825+0000","last_undegraded":"2024-09-16T09:35:06.663825+0000","last_fullsized":"2024-09-16T09:35:06.663825+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:34:41.401641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:34:41.401641+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:35:42.139806+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"66'10","reported_seq":540,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.666850+0000","last_change":"2024-09-16T09:32:30.286077+0000","last_active":"2024-09-16T09:35:06.666850+0000","last_peered":"2024-09-16T09:35:06.666850+0000","last_clean":"2024-09-16T09:35:06.666850+0000","last_became_active":"2024-09-16T09:31:09.275050+0000","last_became_peered":"2024-09-16T09:31:09.275050+0000","last_unstale":"2024-09-16T09:35:06.666850+0000","last_undegraded":"2024-09-16T09:35:06.666850+0000","last_fullsized":"2024-09-16T09:35:06.666850+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-16T09:32:30.284919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:32:30.284919+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"66'8","reported_seq":639,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.685116+0000","last_change":"2024-09-16T09:34:52.922123+0000","last_active":"2024-09-16T09:35:06.685116+0000","last_peered":"2024-09-16T09:35:06.685116+0000","last_clean":"2024-09-16T09:35:06.685116+0000","last_became_active":"2024-09-16T09:25:05.712516+0000","last_became_peered":"2024-09-16T09:25:05.712516+0000","last_unstale":"2024-09-16T09:35:06.685116+0000","last_undegraded":"2024-09-16T09:35:06.685116+0000","last_fullsized":"2024-09-16T09:35:06.685116+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:34:52.922047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:34:52.922047+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:36:15.788853+0000","scrub_duration":33,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2","version":"284'5554","reported_seq":5702,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:08.163686+0000","last_change":"2024-09-16T09:33:53.529863+0000","last_active":"2024-09-16T09:35:08.163686+0000","last_peered":"2024-09-16T09:35:08.163686+0000","last_clean":"2024-09-16T09:35:08.163686+0000","last_became_active":"2024-09-16T09:33:52.933886+0000","last_became_peered":"2024-09-16T09:33:52.933886+0000","last_unstale":"2024-09-16T09:35:08.163686+0000","last_undegraded":"2024-09-16T09:35:08.163686+0000","last_fullsized":"2024-09-16T09:35:08.163686+0000","mapping_epoch":233,"log_start":"235'449","ondisk_log_start":"235'449","created":229,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"244'3174","last_scrub_stamp":"2024-09-16T09:33:53.529712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:33:53.529712+0000","objects_scrubbed":1593,"log_size":5105,"log_dups_size":0,"ondisk_log_size":5105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":449,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":234160128,"num_objects":3573,"num_object_clones":0,"num_object_copies":10719,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3573,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2388,"num_write_kb":139776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.a","version":"73'79","reported_seq":412,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.666972+0000","last_change":"2024-09-16T09:29:18.498887+0000","last_active":"2024-09-16T09:35:06.666972+0000","last_peered":"2024-09-16T09:35:06.666972+0000","last_clean":"2024-09-16T09:35:06.666972+0000","last_became_active":"2024-09-16T09:26:35.127352+0000","last_became_peered":"2024-09-16T09:26:35.127352+0000","last_unstale":"2024-09-16T09:35:06.666972+0000","last_undegraded":"2024-09-16T09:35:06.666972+0000","last_fullsized":"2024-09-16T09:35:06.666972+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:18.498766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:18.498766+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"66'15","reported_seq":421,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.690303+0000","last_change":"2024-09-16T09:28:28.065781+0000","last_active":"2024-09-16T09:35:06.690303+0000","last_peered":"2024-09-16T09:35:06.690303+0000","last_clean":"2024-09-16T09:35:06.690303+0000","last_became_active":"2024-09-16T09:26:33.295672+0000","last_became_peered":"2024-09-16T09:26:33.295672+0000","last_unstale":"2024-09-16T09:35:06.690303+0000","last_undegraded":"2024-09-16T09:35:06.690303+0000","last_fullsized":"2024-09-16T09:35:06.690303+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-16T09:28:28.065699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:28.065699+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:35:27.260832+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":215,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.663659+0000","last_change":"2024-09-16T09:34:39.420386+0000","last_active":"2024-09-16T09:35:06.663659+0000","last_peered":"2024-09-16T09:35:06.663659+0000","last_clean":"2024-09-16T09:35:06.663659+0000","last_became_active":"2024-09-16T09:31:09.274534+0000","last_became_peered":"2024-09-16T09:31:09.274534+0000","last_unstale":"2024-09-16T09:35:06.663659+0000","last_undegraded":"2024-09-16T09:35:06.663659+0000","last_fullsized":"2024-09-16T09:35:06.663659+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:39.420314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:34:39.420314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:35:58.932088+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"66'8","reported_seq":588,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.664609+0000","last_change":"2024-09-16T09:25:05.714128+0000","last_active":"2024-09-16T09:35:06.664609+0000","last_peered":"2024-09-16T09:35:06.664609+0000","last_clean":"2024-09-16T09:35:06.664609+0000","last_became_active":"2024-09-16T09:25:05.713813+0000","last_became_peered":"2024-09-16T09:25:05.713813+0000","last_unstale":"2024-09-16T09:35:06.664609+0000","last_undegraded":"2024-09-16T09:35:06.664609+0000","last_fullsized":"2024-09-16T09:35:06.664609+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:25:04.695480+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":469,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.671279+0000","last_change":"2024-09-16T09:29:24.537802+0000","last_active":"2024-09-16T09:35:06.671279+0000","last_peered":"2024-09-16T09:35:06.671279+0000","last_clean":"2024-09-16T09:35:06.671279+0000","last_became_active":"2024-09-16T09:25:07.717344+0000","last_became_peered":"2024-09-16T09:25:07.717344+0000","last_unstale":"2024-09-16T09:35:06.671279+0000","last_undegraded":"2024-09-16T09:35:06.671279+0000","last_fullsized":"2024-09-16T09:35:06.671279+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:24.537731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:24.537731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"284'7971","reported_seq":14922,"reported_epoch":284,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-16T09:35:07.867313+0000","last_change":"2024-09-16T09:34:15.286261+0000","last_active":"2024-09-16T09:35:07.867313+0000","last_peered":"2024-09-16T09:35:07.867313+0000","last_clean":"2024-09-16T09:33:59.264728+0000","last_became_active":"2024-09-16T09:34:01.447555+0000","last_became_peered":"2024-09-16T09:34:01.447555+0000","last_unstale":"2024-09-16T09:35:07.867313+0000","last_undegraded":"2024-09-16T09:35:07.867313+0000","last_fullsized":"2024-09-16T09:34:01.323950+0000","mapping_epoch":249,"log_start":"253'4200","ondisk_log_start":"253'4200","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:33:24.230486+0000","objects_scrubbed":0,"log_size":3771,"log_dups_size":0,"ondisk_log_size":3771,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":472383512,"num_objects":7209,"num_object_clones":0,"num_object_copies":21627,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":142,"num_objects_unfound":0,"num_objects_dirty":7209,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7971,"num_write_kb":485697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3412,"num_bytes_recovered":223608832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5],"avail_no_missing":["7","1","5"],"object_location_counts":[{"shards":"1,5,7","objects":142},{"shards":"5,7","objects":7067}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.5","version":"73'79","reported_seq":453,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.664752+0000","last_change":"2024-09-16T09:32:42.123139+0000","last_active":"2024-09-16T09:35:06.664752+0000","last_peered":"2024-09-16T09:35:06.664752+0000","last_clean":"2024-09-16T09:35:06.664752+0000","last_became_active":"2024-09-16T09:30:08.763683+0000","last_became_peered":"2024-09-16T09:30:08.763683+0000","last_unstale":"2024-09-16T09:35:06.664752+0000","last_undegraded":"2024-09-16T09:35:06.664752+0000","last_fullsized":"2024-09-16T09:35:06.664752+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:32:42.123067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:32:42.123067+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"66'18","reported_seq":428,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.667698+0000","last_change":"2024-09-16T09:33:21.333996+0000","last_active":"2024-09-16T09:35:06.667698+0000","last_peered":"2024-09-16T09:35:06.667698+0000","last_clean":"2024-09-16T09:35:06.667698+0000","last_became_active":"2024-09-16T09:31:09.270434+0000","last_became_peered":"2024-09-16T09:31:09.270434+0000","last_unstale":"2024-09-16T09:35:06.667698+0000","last_undegraded":"2024-09-16T09:35:06.667698+0000","last_fullsized":"2024-09-16T09:35:06.667698+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'18","last_scrub_stamp":"2024-09-16T09:33:21.333937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:33:21.333937+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":33,"num_read_kb":21,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":317,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.669977+0000","last_change":"2024-09-16T09:28:58.705169+0000","last_active":"2024-09-16T09:35:06.669977+0000","last_peered":"2024-09-16T09:35:06.669977+0000","last_clean":"2024-09-16T09:35:06.669977+0000","last_became_active":"2024-09-16T09:28:52.983543+0000","last_became_peered":"2024-09-16T09:28:52.983543+0000","last_unstale":"2024-09-16T09:35:06.669977+0000","last_undegraded":"2024-09-16T09:35:06.669977+0000","last_fullsized":"2024-09-16T09:35:06.669977+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:58.705097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:28:58.705097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":496,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.663649+0000","last_change":"2024-09-16T09:31:09.269234+0000","last_active":"2024-09-16T09:35:06.663649+0000","last_peered":"2024-09-16T09:35:06.663649+0000","last_clean":"2024-09-16T09:35:06.663649+0000","last_became_active":"2024-09-16T09:31:09.268864+0000","last_became_peered":"2024-09-16T09:31:09.268864+0000","last_unstale":"2024-09-16T09:35:06.663649+0000","last_undegraded":"2024-09-16T09:35:06.663649+0000","last_fullsized":"2024-09-16T09:35:06.663649+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:50.482240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:50.482240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":484,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.664127+0000","last_change":"2024-09-16T09:32:32.060608+0000","last_active":"2024-09-16T09:35:06.664127+0000","last_peered":"2024-09-16T09:35:06.664127+0000","last_clean":"2024-09-16T09:35:06.664127+0000","last_became_active":"2024-09-16T09:31:09.269433+0000","last_became_peered":"2024-09-16T09:31:09.269433+0000","last_unstale":"2024-09-16T09:35:06.664127+0000","last_undegraded":"2024-09-16T09:35:06.664127+0000","last_fullsized":"2024-09-16T09:35:06.664127+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:32.060564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:32:32.060564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.c","version":"284'7906","reported_seq":8032,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:07.887562+0000","last_change":"2024-09-16T09:33:25.371514+0000","last_active":"2024-09-16T09:35:07.887562+0000","last_peered":"2024-09-16T09:35:07.887562+0000","last_clean":"2024-09-16T09:35:07.887562+0000","last_became_active":"2024-09-16T09:33:25.371230+0000","last_became_peered":"2024-09-16T09:33:25.371230+0000","last_unstale":"2024-09-16T09:35:07.887562+0000","last_undegraded":"2024-09-16T09:35:07.887562+0000","last_fullsized":"2024-09-16T09:35:07.887562+0000","mapping_epoch":229,"log_start":"251'3700","ondisk_log_start":"251'3700","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:33:24.230486+0000","objects_scrubbed":0,"log_size":4206,"log_dups_size":0,"ondisk_log_size":4206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":465960960,"num_objects":7110,"num_object_clones":0,"num_object_copies":21330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7110,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7906,"num_write_kb":480512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"73'79","reported_seq":215,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.664692+0000","last_change":"2024-09-16T09:34:42.362203+0000","last_active":"2024-09-16T09:35:06.664692+0000","last_peered":"2024-09-16T09:35:06.664692+0000","last_clean":"2024-09-16T09:35:06.664692+0000","last_became_active":"2024-09-16T09:31:09.275139+0000","last_became_peered":"2024-09-16T09:31:09.275139+0000","last_unstale":"2024-09-16T09:35:06.664692+0000","last_undegraded":"2024-09-16T09:35:06.664692+0000","last_fullsized":"2024-09-16T09:35:06.664692+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:34:42.362121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:34:42.362121+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:36:07.590504+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"66'14","reported_seq":507,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.663706+0000","last_change":"2024-09-16T09:32:34.997417+0000","last_active":"2024-09-16T09:35:06.663706+0000","last_peered":"2024-09-16T09:35:06.663706+0000","last_clean":"2024-09-16T09:35:06.663706+0000","last_became_active":"2024-09-16T09:25:03.708830+0000","last_became_peered":"2024-09-16T09:25:03.708830+0000","last_unstale":"2024-09-16T09:35:06.663706+0000","last_undegraded":"2024-09-16T09:35:06.663706+0000","last_fullsized":"2024-09-16T09:35:06.663706+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'14","last_scrub_stamp":"2024-09-16T09:32:34.997360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:32:34.997360+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":507,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.667095+0000","last_change":"2024-09-16T09:33:43.678231+0000","last_active":"2024-09-16T09:35:06.667095+0000","last_peered":"2024-09-16T09:35:06.667095+0000","last_clean":"2024-09-16T09:35:06.667095+0000","last_became_active":"2024-09-16T09:25:01.702992+0000","last_became_peered":"2024-09-16T09:25:01.702992+0000","last_unstale":"2024-09-16T09:35:06.667095+0000","last_undegraded":"2024-09-16T09:35:06.667095+0000","last_fullsized":"2024-09-16T09:35:06.667095+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:33:43.678138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:43.678138+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:35:13.034934+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"235'12","reported_seq":159,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.663714+0000","last_change":"2024-09-16T09:33:30.251570+0000","last_active":"2024-09-16T09:35:06.663714+0000","last_peered":"2024-09-16T09:35:06.663714+0000","last_clean":"2024-09-16T09:35:06.663714+0000","last_became_active":"2024-09-16T09:33:30.251047+0000","last_became_peered":"2024-09-16T09:33:30.251047+0000","last_unstale":"2024-09-16T09:35:06.663714+0000","last_undegraded":"2024-09-16T09:35:06.663714+0000","last_fullsized":"2024-09-16T09:35:06.663714+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"187'11","last_scrub_stamp":"2024-09-16T09:33:09.417852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:09.417852+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":461,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.683195+0000","last_change":"2024-09-16T09:26:40.699888+0000","last_active":"2024-09-16T09:35:06.683195+0000","last_peered":"2024-09-16T09:35:06.683195+0000","last_clean":"2024-09-16T09:35:06.683195+0000","last_became_active":"2024-09-16T09:25:07.722017+0000","last_became_peered":"2024-09-16T09:25:07.722017+0000","last_unstale":"2024-09-16T09:35:06.683195+0000","last_undegraded":"2024-09-16T09:35:06.683195+0000","last_fullsized":"2024-09-16T09:35:06.683195+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:40.699813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:40.699813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"284'8132","reported_seq":8258,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:08.069424+0000","last_change":"2024-09-16T09:33:25.372320+0000","last_active":"2024-09-16T09:35:08.069424+0000","last_peered":"2024-09-16T09:35:08.069424+0000","last_clean":"2024-09-16T09:35:08.069424+0000","last_became_active":"2024-09-16T09:33:25.372005+0000","last_became_peered":"2024-09-16T09:33:25.372005+0000","last_unstale":"2024-09-16T09:35:08.069424+0000","last_undegraded":"2024-09-16T09:35:08.069424+0000","last_fullsized":"2024-09-16T09:35:08.069424+0000","mapping_epoch":229,"log_start":"250'3700","ondisk_log_start":"250'3700","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:33:24.230486+0000","objects_scrubbed":0,"log_size":4432,"log_dups_size":0,"ondisk_log_size":4432,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":476839936,"num_objects":7276,"num_object_clones":0,"num_object_copies":21828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7276,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8132,"num_write_kb":493056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.7","version":"73'79","reported_seq":546,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.684859+0000","last_change":"2024-09-16T09:34:51.864131+0000","last_active":"2024-09-16T09:35:06.684859+0000","last_peered":"2024-09-16T09:35:06.684859+0000","last_clean":"2024-09-16T09:35:06.684859+0000","last_became_active":"2024-09-16T09:31:09.274707+0000","last_became_peered":"2024-09-16T09:31:09.274707+0000","last_unstale":"2024-09-16T09:35:06.684859+0000","last_undegraded":"2024-09-16T09:35:06.684859+0000","last_fullsized":"2024-09-16T09:35:06.684859+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:34:51.864036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:34:51.864036+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:36:20.395033+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"66'10","reported_seq":156,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.667419+0000","last_change":"2024-09-16T09:33:41.368102+0000","last_active":"2024-09-16T09:35:06.667419+0000","last_peered":"2024-09-16T09:35:06.667419+0000","last_clean":"2024-09-16T09:35:06.667419+0000","last_became_active":"2024-09-16T09:33:30.248453+0000","last_became_peered":"2024-09-16T09:33:30.248453+0000","last_unstale":"2024-09-16T09:35:06.667419+0000","last_undegraded":"2024-09-16T09:35:06.667419+0000","last_fullsized":"2024-09-16T09:35:06.667419+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-16T09:33:41.368016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:33:41.368016+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:35:10.493830+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":14,"num_read_kb":9,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":495,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.671928+0000","last_change":"2024-09-16T09:32:46.388711+0000","last_active":"2024-09-16T09:35:06.671928+0000","last_peered":"2024-09-16T09:35:06.671928+0000","last_clean":"2024-09-16T09:35:06.671928+0000","last_became_active":"2024-09-16T09:31:09.270619+0000","last_became_peered":"2024-09-16T09:31:09.270619+0000","last_unstale":"2024-09-16T09:35:06.671928+0000","last_undegraded":"2024-09-16T09:35:06.671928+0000","last_fullsized":"2024-09-16T09:35:06.671928+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:46.388651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:32:46.388651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"66'8","reported_seq":604,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.666737+0000","last_change":"2024-09-16T09:28:57.628615+0000","last_active":"2024-09-16T09:35:06.666737+0000","last_peered":"2024-09-16T09:35:06.666737+0000","last_clean":"2024-09-16T09:35:06.666737+0000","last_became_active":"2024-09-16T09:25:05.714507+0000","last_became_peered":"2024-09-16T09:25:05.714507+0000","last_unstale":"2024-09-16T09:35:06.666737+0000","last_undegraded":"2024-09-16T09:35:06.666737+0000","last_fullsized":"2024-09-16T09:35:06.666737+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:28:57.628543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:28:57.628543+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:35:57.817677+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":492,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.930467+0000","last_change":"2024-09-16T09:32:33.362233+0000","last_active":"2024-09-16T09:35:06.930467+0000","last_peered":"2024-09-16T09:35:06.930467+0000","last_clean":"2024-09-16T09:35:06.930467+0000","last_became_active":"2024-09-16T09:31:09.283192+0000","last_became_peered":"2024-09-16T09:31:09.283192+0000","last_unstale":"2024-09-16T09:35:06.930467+0000","last_undegraded":"2024-09-16T09:35:06.930467+0000","last_fullsized":"2024-09-16T09:35:06.930467+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:33.362188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:32:33.362188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.e","version":"284'8078","reported_seq":8204,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:07.829023+0000","last_change":"2024-09-16T09:33:25.247961+0000","last_active":"2024-09-16T09:35:07.829023+0000","last_peered":"2024-09-16T09:35:07.829023+0000","last_clean":"2024-09-16T09:35:07.829023+0000","last_became_active":"2024-09-16T09:33:25.247683+0000","last_became_peered":"2024-09-16T09:33:25.247683+0000","last_unstale":"2024-09-16T09:35:07.829023+0000","last_undegraded":"2024-09-16T09:35:07.829023+0000","last_fullsized":"2024-09-16T09:35:07.829023+0000","mapping_epoch":229,"log_start":"255'4300","ondisk_log_start":"255'4300","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:33:24.230486+0000","objects_scrubbed":0,"log_size":3778,"log_dups_size":0,"ondisk_log_size":3778,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":474218496,"num_objects":7236,"num_object_clones":0,"num_object_copies":21708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7236,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8078,"num_write_kb":490048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.6","version":"73'79","reported_seq":519,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.684591+0000","last_change":"2024-09-16T09:34:42.728232+0000","last_active":"2024-09-16T09:35:06.684591+0000","last_peered":"2024-09-16T09:35:06.684591+0000","last_clean":"2024-09-16T09:35:06.684591+0000","last_became_active":"2024-09-16T09:31:09.274565+0000","last_became_peered":"2024-09-16T09:31:09.274565+0000","last_unstale":"2024-09-16T09:35:06.684591+0000","last_undegraded":"2024-09-16T09:35:06.684591+0000","last_fullsized":"2024-09-16T09:35:06.684591+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:34:42.728143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:34:42.728143+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:35:48.900068+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"66'19","reported_seq":514,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.666936+0000","last_change":"2024-09-16T09:31:35.249952+0000","last_active":"2024-09-16T09:35:06.666936+0000","last_peered":"2024-09-16T09:35:06.666936+0000","last_clean":"2024-09-16T09:35:06.666936+0000","last_became_active":"2024-09-16T09:25:03.714797+0000","last_became_peered":"2024-09-16T09:25:03.714797+0000","last_unstale":"2024-09-16T09:35:06.666936+0000","last_undegraded":"2024-09-16T09:35:06.666936+0000","last_fullsized":"2024-09-16T09:35:06.666936+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-16T09:31:35.249864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:31:35.249864+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":113,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.667314+0000","last_change":"2024-09-16T09:33:30.247311+0000","last_active":"2024-09-16T09:35:06.667314+0000","last_peered":"2024-09-16T09:35:06.667314+0000","last_clean":"2024-09-16T09:35:06.667314+0000","last_became_active":"2024-09-16T09:33:30.246996+0000","last_became_peered":"2024-09-16T09:33:30.246996+0000","last_unstale":"2024-09-16T09:35:06.667314+0000","last_undegraded":"2024-09-16T09:35:06.667314+0000","last_fullsized":"2024-09-16T09:35:06.667314+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:11.407616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:11.407616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,2],"acting":[5,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":498,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.683628+0000","last_change":"2024-09-16T09:34:45.772748+0000","last_active":"2024-09-16T09:35:06.683628+0000","last_peered":"2024-09-16T09:35:06.683628+0000","last_clean":"2024-09-16T09:35:06.683628+0000","last_became_active":"2024-09-16T09:25:05.708663+0000","last_became_peered":"2024-09-16T09:25:05.708663+0000","last_unstale":"2024-09-16T09:35:06.683628+0000","last_undegraded":"2024-09-16T09:35:06.683628+0000","last_fullsized":"2024-09-16T09:35:06.683628+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:45.772674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:34:45.772674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:36:11.743458+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":121,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.684009+0000","last_change":"2024-09-16T09:34:36.806165+0000","last_active":"2024-09-16T09:35:06.684009+0000","last_peered":"2024-09-16T09:35:06.684009+0000","last_clean":"2024-09-16T09:35:06.684009+0000","last_became_active":"2024-09-16T09:33:30.250178+0000","last_became_peered":"2024-09-16T09:33:30.250178+0000","last_unstale":"2024-09-16T09:35:06.684009+0000","last_undegraded":"2024-09-16T09:35:06.684009+0000","last_fullsized":"2024-09-16T09:35:06.684009+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:36.805971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:34:36.805971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:35:58.467636+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,2],"acting":[6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"284'5640","reported_seq":5798,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:08.469542+0000","last_change":"2024-09-16T09:35:05.664293+0000","last_active":"2024-09-16T09:35:08.469542+0000","last_peered":"2024-09-16T09:35:08.469542+0000","last_clean":"2024-09-16T09:35:08.469542+0000","last_became_active":"2024-09-16T09:33:52.910861+0000","last_became_peered":"2024-09-16T09:33:52.910861+0000","last_unstale":"2024-09-16T09:35:08.469542+0000","last_undegraded":"2024-09-16T09:35:08.469542+0000","last_fullsized":"2024-09-16T09:35:08.469542+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:33:24.230486+0000","objects_scrubbed":0,"log_size":5640,"log_dups_size":0,"ondisk_log_size":5640,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:35:34.989807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":239599616,"num_objects":3656,"num_object_clones":0,"num_object_copies":10968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3656,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4084,"num_write_kb":247680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,5],"acting":[0,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"73'79","reported_seq":445,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.668070+0000","last_change":"2024-09-16T09:32:43.420831+0000","last_active":"2024-09-16T09:35:06.668070+0000","last_peered":"2024-09-16T09:35:06.668070+0000","last_clean":"2024-09-16T09:35:06.668070+0000","last_became_active":"2024-09-16T09:30:08.766350+0000","last_became_peered":"2024-09-16T09:30:08.766350+0000","last_unstale":"2024-09-16T09:35:06.668070+0000","last_undegraded":"2024-09-16T09:35:06.668070+0000","last_fullsized":"2024-09-16T09:35:06.668070+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:32:43.420784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:32:43.420784+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"187'30","reported_seq":721,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.665003+0000","last_change":"2024-09-16T09:34:47.032891+0000","last_active":"2024-09-16T09:35:06.665003+0000","last_peered":"2024-09-16T09:35:06.665003+0000","last_clean":"2024-09-16T09:35:06.665003+0000","last_became_active":"2024-09-16T09:31:09.278839+0000","last_became_peered":"2024-09-16T09:31:09.278839+0000","last_unstale":"2024-09-16T09:35:06.665003+0000","last_undegraded":"2024-09-16T09:35:06.665003+0000","last_fullsized":"2024-09-16T09:35:06.665003+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"187'30","last_scrub_stamp":"2024-09-16T09:34:47.032800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:34:47.032800+0000","objects_scrubbed":10,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:36:03.763093+0000","scrub_duration":26,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":482,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.664211+0000","last_change":"2024-09-16T09:29:26.637070+0000","last_active":"2024-09-16T09:35:06.664211+0000","last_peered":"2024-09-16T09:35:06.664211+0000","last_clean":"2024-09-16T09:35:06.664211+0000","last_became_active":"2024-09-16T09:25:01.703076+0000","last_became_peered":"2024-09-16T09:25:01.703076+0000","last_unstale":"2024-09-16T09:35:06.664211+0000","last_undegraded":"2024-09-16T09:35:06.664211+0000","last_fullsized":"2024-09-16T09:35:06.664211+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:26.637027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:26.637027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":499,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.667708+0000","last_change":"2024-09-16T09:33:38.417375+0000","last_active":"2024-09-16T09:35:06.667708+0000","last_peered":"2024-09-16T09:35:06.667708+0000","last_clean":"2024-09-16T09:35:06.667708+0000","last_became_active":"2024-09-16T09:25:00.696237+0000","last_became_peered":"2024-09-16T09:25:00.696237+0000","last_unstale":"2024-09-16T09:35:06.667708+0000","last_undegraded":"2024-09-16T09:35:06.667708+0000","last_fullsized":"2024-09-16T09:35:06.667708+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-16T09:33:38.417308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:33:38.417308+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.0025137430000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":466,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.673252+0000","last_change":"2024-09-16T09:33:16.666704+0000","last_active":"2024-09-16T09:35:06.673252+0000","last_peered":"2024-09-16T09:35:06.673252+0000","last_clean":"2024-09-16T09:35:06.673252+0000","last_became_active":"2024-09-16T09:25:05.717591+0000","last_became_peered":"2024-09-16T09:25:05.717591+0000","last_unstale":"2024-09-16T09:35:06.673252+0000","last_undegraded":"2024-09-16T09:35:06.673252+0000","last_fullsized":"2024-09-16T09:35:06.673252+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:16.666648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:16.666648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":207,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.666975+0000","last_change":"2024-09-16T09:31:09.272967+0000","last_active":"2024-09-16T09:35:06.666975+0000","last_peered":"2024-09-16T09:35:06.666975+0000","last_clean":"2024-09-16T09:35:06.666975+0000","last_became_active":"2024-09-16T09:31:09.272609+0000","last_became_peered":"2024-09-16T09:31:09.272609+0000","last_unstale":"2024-09-16T09:35:06.666975+0000","last_undegraded":"2024-09-16T09:35:06.666975+0000","last_fullsized":"2024-09-16T09:35:06.666975+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:45.201503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:45.201503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:36:03.306867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.a","version":"284'7915","reported_seq":8041,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:08.105011+0000","last_change":"2024-09-16T09:33:25.244002+0000","last_active":"2024-09-16T09:35:08.105011+0000","last_peered":"2024-09-16T09:35:08.105011+0000","last_clean":"2024-09-16T09:35:08.105011+0000","last_became_active":"2024-09-16T09:33:25.243701+0000","last_became_peered":"2024-09-16T09:33:25.243701+0000","last_unstale":"2024-09-16T09:35:08.105011+0000","last_undegraded":"2024-09-16T09:35:08.105011+0000","last_fullsized":"2024-09-16T09:35:08.105011+0000","mapping_epoch":229,"log_start":"250'3500","ondisk_log_start":"250'3500","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:33:24.230486+0000","objects_scrubbed":0,"log_size":4415,"log_dups_size":0,"ondisk_log_size":4415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":462880768,"num_objects":7063,"num_object_clones":0,"num_object_copies":21189,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7063,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7915,"num_write_kb":479296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"73'79","reported_seq":169,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.919109+0000","last_change":"2024-09-16T09:34:32.717994+0000","last_active":"2024-09-16T09:35:06.919109+0000","last_peered":"2024-09-16T09:35:06.919109+0000","last_clean":"2024-09-16T09:35:06.919109+0000","last_became_active":"2024-09-16T09:34:00.336514+0000","last_became_peered":"2024-09-16T09:34:00.336514+0000","last_unstale":"2024-09-16T09:35:06.919109+0000","last_undegraded":"2024-09-16T09:35:06.919109+0000","last_fullsized":"2024-09-16T09:35:06.919109+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:34:32.717915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:34:32.717915+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:35:59.254478+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"66'13","reported_seq":179,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.668862+0000","last_change":"2024-09-16T09:34:15.319682+0000","last_active":"2024-09-16T09:35:06.668862+0000","last_peered":"2024-09-16T09:35:06.668862+0000","last_clean":"2024-09-16T09:35:06.668862+0000","last_became_active":"2024-09-16T09:34:00.337121+0000","last_became_peered":"2024-09-16T09:34:00.337121+0000","last_unstale":"2024-09-16T09:35:06.668862+0000","last_undegraded":"2024-09-16T09:35:06.668862+0000","last_fullsized":"2024-09-16T09:35:06.668862+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'13","last_scrub_stamp":"2024-09-16T09:27:44.155969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:44.155969+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":18,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":113,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.684267+0000","last_change":"2024-09-16T09:33:30.248260+0000","last_active":"2024-09-16T09:35:06.684267+0000","last_peered":"2024-09-16T09:35:06.684267+0000","last_clean":"2024-09-16T09:35:06.684267+0000","last_became_active":"2024-09-16T09:33:30.247596+0000","last_became_peered":"2024-09-16T09:33:30.247596+0000","last_unstale":"2024-09-16T09:35:06.684267+0000","last_undegraded":"2024-09-16T09:35:06.684267+0000","last_fullsized":"2024-09-16T09:35:06.684267+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:22.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:22.331302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,2],"acting":[6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":137,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.673070+0000","last_change":"2024-09-16T09:33:21.338149+0000","last_active":"2024-09-16T09:35:06.673070+0000","last_peered":"2024-09-16T09:35:06.673070+0000","last_clean":"2024-09-16T09:35:06.673070+0000","last_became_active":"2024-09-16T09:33:21.337177+0000","last_became_peered":"2024-09-16T09:33:21.337177+0000","last_unstale":"2024-09-16T09:35:06.673070+0000","last_undegraded":"2024-09-16T09:35:06.673070+0000","last_fullsized":"2024-09-16T09:35:06.673070+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:32:40.165749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:32:40.165749+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":498,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.692901+0000","last_change":"2024-09-16T09:33:40.418799+0000","last_active":"2024-09-16T09:35:06.692901+0000","last_peered":"2024-09-16T09:35:06.692901+0000","last_clean":"2024-09-16T09:35:06.692901+0000","last_became_active":"2024-09-16T09:25:05.711539+0000","last_became_peered":"2024-09-16T09:25:05.711539+0000","last_unstale":"2024-09-16T09:35:06.692901+0000","last_undegraded":"2024-09-16T09:35:06.692901+0000","last_fullsized":"2024-09-16T09:35:06.692901+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:40.417644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:40.417644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:36:04.363902+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":477,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.670337+0000","last_change":"2024-09-16T09:33:39.313624+0000","last_active":"2024-09-16T09:35:06.670337+0000","last_peered":"2024-09-16T09:35:06.670337+0000","last_clean":"2024-09-16T09:35:06.670337+0000","last_became_active":"2024-09-16T09:25:07.723282+0000","last_became_peered":"2024-09-16T09:25:07.723282+0000","last_unstale":"2024-09-16T09:35:06.670337+0000","last_undegraded":"2024-09-16T09:35:06.670337+0000","last_fullsized":"2024-09-16T09:35:06.670337+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:39.313562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:33:39.313562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.b","version":"284'7876","reported_seq":8002,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:08.525119+0000","last_change":"2024-09-16T09:33:25.245108+0000","last_active":"2024-09-16T09:35:08.525119+0000","last_peered":"2024-09-16T09:35:08.525119+0000","last_clean":"2024-09-16T09:35:08.525119+0000","last_became_active":"2024-09-16T09:33:25.244819+0000","last_became_peered":"2024-09-16T09:33:25.244819+0000","last_unstale":"2024-09-16T09:35:08.525119+0000","last_undegraded":"2024-09-16T09:35:08.525119+0000","last_fullsized":"2024-09-16T09:35:08.525119+0000","mapping_epoch":229,"log_start":"244'3100","ondisk_log_start":"244'3100","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:33:24.230486+0000","objects_scrubbed":0,"log_size":4776,"log_dups_size":0,"ondisk_log_size":4776,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459014144,"num_objects":7004,"num_object_clones":0,"num_object_copies":21012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7004,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7876,"num_write_kb":476160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"73'79","reported_seq":202,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.683594+0000","last_change":"2024-09-16T09:34:31.929462+0000","last_active":"2024-09-16T09:35:06.683594+0000","last_peered":"2024-09-16T09:35:06.683594+0000","last_clean":"2024-09-16T09:35:06.683594+0000","last_became_active":"2024-09-16T09:33:30.249309+0000","last_became_peered":"2024-09-16T09:33:30.249309+0000","last_unstale":"2024-09-16T09:35:06.683594+0000","last_undegraded":"2024-09-16T09:35:06.683594+0000","last_fullsized":"2024-09-16T09:35:06.683594+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:34:31.929388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:34:31.929388+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:35:38.514043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.6","version":"66'12","reported_seq":496,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.666902+0000","last_change":"2024-09-16T09:32:48.547237+0000","last_active":"2024-09-16T09:35:06.666902+0000","last_peered":"2024-09-16T09:35:06.666902+0000","last_clean":"2024-09-16T09:35:06.666902+0000","last_became_active":"2024-09-16T09:25:03.715315+0000","last_became_peered":"2024-09-16T09:25:03.715315+0000","last_unstale":"2024-09-16T09:35:06.666902+0000","last_undegraded":"2024-09-16T09:35:06.666902+0000","last_fullsized":"2024-09-16T09:35:06.666902+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-16T09:32:48.546869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:32:48.546869+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"59'1","reported_seq":513,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.667595+0000","last_change":"2024-09-16T09:33:21.682893+0000","last_active":"2024-09-16T09:35:06.667595+0000","last_peered":"2024-09-16T09:35:06.667595+0000","last_clean":"2024-09-16T09:35:06.667595+0000","last_became_active":"2024-09-16T09:31:09.269627+0000","last_became_peered":"2024-09-16T09:31:09.269627+0000","last_unstale":"2024-09-16T09:35:06.667595+0000","last_undegraded":"2024-09-16T09:35:06.667595+0000","last_fullsized":"2024-09-16T09:35:06.667595+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:33:21.682840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:21.682840+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1986,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"67'5","reported_seq":1101,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.668303+0000","last_change":"2024-09-16T09:33:37.358839+0000","last_active":"2024-09-16T09:35:06.668303+0000","last_peered":"2024-09-16T09:35:06.668303+0000","last_clean":"2024-09-16T09:35:06.668303+0000","last_became_active":"2024-09-16T09:25:00.696546+0000","last_became_peered":"2024-09-16T09:25:00.696546+0000","last_unstale":"2024-09-16T09:35:06.668303+0000","last_undegraded":"2024-09-16T09:35:06.668303+0000","last_fullsized":"2024-09-16T09:35:06.668303+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'5","last_scrub_stamp":"2024-09-16T09:33:37.358741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:33:37.358741+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0.0059770420000000001,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":600,"num_read_kb":595,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":465,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.668955+0000","last_change":"2024-09-16T09:27:07.337170+0000","last_active":"2024-09-16T09:35:06.668955+0000","last_peered":"2024-09-16T09:35:06.668955+0000","last_clean":"2024-09-16T09:35:06.668955+0000","last_became_active":"2024-09-16T09:25:05.714704+0000","last_became_peered":"2024-09-16T09:25:05.714704+0000","last_unstale":"2024-09-16T09:35:06.668955+0000","last_undegraded":"2024-09-16T09:35:06.668955+0000","last_fullsized":"2024-09-16T09:35:06.668955+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:07.337124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:07.337124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":383,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.667548+0000","last_change":"2024-09-16T09:33:30.246166+0000","last_active":"2024-09-16T09:35:06.667548+0000","last_peered":"2024-09-16T09:35:06.667548+0000","last_clean":"2024-09-16T09:35:06.667548+0000","last_became_active":"2024-09-16T09:33:30.245852+0000","last_became_peered":"2024-09-16T09:33:30.245852+0000","last_unstale":"2024-09-16T09:35:06.667548+0000","last_undegraded":"2024-09-16T09:35:06.667548+0000","last_fullsized":"2024-09-16T09:35:06.667548+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:46.252700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:32:46.252700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"284'5574","reported_seq":11075,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:08.487221+0000","last_change":"2024-09-16T09:34:42.510673+0000","last_active":"2024-09-16T09:35:08.487221+0000","last_peered":"2024-09-16T09:35:08.487221+0000","last_clean":"2024-09-16T09:35:08.487221+0000","last_became_active":"2024-09-16T09:34:41.432414+0000","last_became_peered":"2024-09-16T09:34:41.432414+0000","last_unstale":"2024-09-16T09:35:08.487221+0000","last_undegraded":"2024-09-16T09:35:08.487221+0000","last_fullsized":"2024-09-16T09:35:08.487221+0000","mapping_epoch":272,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":273,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'4604","last_scrub_stamp":"2024-09-16T09:34:42.510459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:34:42.510459+0000","objects_scrubbed":3078,"log_size":5574,"log_dups_size":0,"ondisk_log_size":5574,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:36:02.051070+0000","scrub_duration":697,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":235405312,"num_objects":3592,"num_object_clones":0,"num_object_copies":10776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3592,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":984,"num_write_kb":47936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"66'16","reported_seq":638,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.684440+0000","last_change":"2024-09-16T09:33:16.891850+0000","last_active":"2024-09-16T09:35:06.684440+0000","last_peered":"2024-09-16T09:35:06.684440+0000","last_clean":"2024-09-16T09:35:06.684440+0000","last_became_active":"2024-09-16T09:25:03.714118+0000","last_became_peered":"2024-09-16T09:25:03.714118+0000","last_unstale":"2024-09-16T09:35:06.684440+0000","last_undegraded":"2024-09-16T09:35:06.684440+0000","last_fullsized":"2024-09-16T09:35:06.684440+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'16","last_scrub_stamp":"2024-09-16T09:33:16.891761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:33:16.891761+0000","objects_scrubbed":8,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:36:01.941639+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":332,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.667662+0000","last_change":"2024-09-16T09:33:12.171507+0000","last_active":"2024-09-16T09:35:06.667662+0000","last_peered":"2024-09-16T09:35:06.667662+0000","last_clean":"2024-09-16T09:35:06.667662+0000","last_became_active":"2024-09-16T09:31:09.270196+0000","last_became_peered":"2024-09-16T09:31:09.270196+0000","last_unstale":"2024-09-16T09:35:06.667662+0000","last_undegraded":"2024-09-16T09:35:06.667662+0000","last_fullsized":"2024-09-16T09:35:06.667662+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:12.171439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:12.171439+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"73'79","reported_seq":570,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.668509+0000","last_change":"2024-09-16T09:32:43.535041+0000","last_active":"2024-09-16T09:35:06.668509+0000","last_peered":"2024-09-16T09:35:06.668509+0000","last_clean":"2024-09-16T09:35:06.668509+0000","last_became_active":"2024-09-16T09:30:08.870394+0000","last_became_peered":"2024-09-16T09:30:08.870394+0000","last_unstale":"2024-09-16T09:35:06.668509+0000","last_undegraded":"2024-09-16T09:35:06.668509+0000","last_fullsized":"2024-09-16T09:35:06.668509+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:32:43.534962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:32:43.534962+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":473,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.669462+0000","last_change":"2024-09-16T09:33:52.291748+0000","last_active":"2024-09-16T09:35:06.669462+0000","last_peered":"2024-09-16T09:35:06.669462+0000","last_clean":"2024-09-16T09:35:06.669462+0000","last_became_active":"2024-09-16T09:25:05.714341+0000","last_became_peered":"2024-09-16T09:25:05.714341+0000","last_unstale":"2024-09-16T09:35:06.669462+0000","last_undegraded":"2024-09-16T09:35:06.669462+0000","last_fullsized":"2024-09-16T09:35:06.669462+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:52.291702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:52.291702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":469,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.668730+0000","last_change":"2024-09-16T09:29:00.243528+0000","last_active":"2024-09-16T09:35:06.668730+0000","last_peered":"2024-09-16T09:35:06.668730+0000","last_clean":"2024-09-16T09:35:06.668730+0000","last_became_active":"2024-09-16T09:25:07.714281+0000","last_became_peered":"2024-09-16T09:25:07.714281+0000","last_unstale":"2024-09-16T09:35:06.668730+0000","last_undegraded":"2024-09-16T09:35:06.668730+0000","last_fullsized":"2024-09-16T09:35:06.668730+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:00.243482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:00.243482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3","version":"284'5491","reported_seq":5648,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:07.794754+0000","last_change":"2024-09-16T09:33:53.717144+0000","last_active":"2024-09-16T09:35:07.794754+0000","last_peered":"2024-09-16T09:35:07.794754+0000","last_clean":"2024-09-16T09:35:07.794754+0000","last_became_active":"2024-09-16T09:33:52.914057+0000","last_became_peered":"2024-09-16T09:33:52.914057+0000","last_unstale":"2024-09-16T09:35:07.794754+0000","last_undegraded":"2024-09-16T09:35:07.794754+0000","last_fullsized":"2024-09-16T09:35:07.794754+0000","mapping_epoch":229,"log_start":"235'392","ondisk_log_start":"235'392","created":229,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"244'3096","last_scrub_stamp":"2024-09-16T09:33:53.717081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:33:53.717081+0000","objects_scrubbed":1554,"log_size":5099,"log_dups_size":0,"ondisk_log_size":5099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:35:19.581083+0000","scrub_duration":410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":232128512,"num_objects":3542,"num_object_clones":0,"num_object_copies":10626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3542,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2410,"num_write_kb":140736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.b","version":"73'79","reported_seq":305,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.666456+0000","last_change":"2024-09-16T09:34:31.720971+0000","last_active":"2024-09-16T09:35:06.666456+0000","last_peered":"2024-09-16T09:35:06.666456+0000","last_clean":"2024-09-16T09:35:06.666456+0000","last_became_active":"2024-09-16T09:33:30.251933+0000","last_became_peered":"2024-09-16T09:33:30.251933+0000","last_unstale":"2024-09-16T09:35:06.666456+0000","last_undegraded":"2024-09-16T09:35:06.666456+0000","last_fullsized":"2024-09-16T09:35:06.666456+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:34:31.720787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:34:31.720787+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:35:50.655452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"66'11","reported_seq":527,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.664664+0000","last_change":"2024-09-16T09:34:00.338209+0000","last_active":"2024-09-16T09:35:06.664664+0000","last_peered":"2024-09-16T09:35:06.664664+0000","last_clean":"2024-09-16T09:35:06.664664+0000","last_became_active":"2024-09-16T09:34:00.328533+0000","last_became_peered":"2024-09-16T09:34:00.328533+0000","last_unstale":"2024-09-16T09:35:06.664664+0000","last_undegraded":"2024-09-16T09:35:06.664664+0000","last_fullsized":"2024-09-16T09:35:06.664664+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-16T09:30:05.646500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:30:05.646500+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[4,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":474,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.663971+0000","last_change":"2024-09-16T09:33:23.200213+0000","last_active":"2024-09-16T09:35:06.663971+0000","last_peered":"2024-09-16T09:35:06.663971+0000","last_clean":"2024-09-16T09:35:06.663971+0000","last_became_active":"2024-09-16T09:25:01.708270+0000","last_became_peered":"2024-09-16T09:25:01.708270+0000","last_unstale":"2024-09-16T09:35:06.663971+0000","last_undegraded":"2024-09-16T09:35:06.663971+0000","last_fullsized":"2024-09-16T09:35:06.663971+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:23.200166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:23.200166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":465,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.668512+0000","last_change":"2024-09-16T09:26:34.171190+0000","last_active":"2024-09-16T09:35:06.668512+0000","last_peered":"2024-09-16T09:35:06.668512+0000","last_clean":"2024-09-16T09:35:06.668512+0000","last_became_active":"2024-09-16T09:25:05.711964+0000","last_became_peered":"2024-09-16T09:25:05.711964+0000","last_unstale":"2024-09-16T09:35:06.668512+0000","last_undegraded":"2024-09-16T09:35:06.668512+0000","last_fullsized":"2024-09-16T09:35:06.668512+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:34.171139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:26:34.171139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":356,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.684187+0000","last_change":"2024-09-16T09:34:50.866121+0000","last_active":"2024-09-16T09:35:06.684187+0000","last_peered":"2024-09-16T09:35:06.684187+0000","last_clean":"2024-09-16T09:35:06.684187+0000","last_became_active":"2024-09-16T09:31:09.278618+0000","last_became_peered":"2024-09-16T09:31:09.278618+0000","last_unstale":"2024-09-16T09:35:06.684187+0000","last_undegraded":"2024-09-16T09:35:06.684187+0000","last_fullsized":"2024-09-16T09:35:06.684187+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:50.866032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:34:50.866032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:36:08.203842+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.0","version":"284'5570","reported_seq":5708,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:07.633070+0000","last_change":"2024-09-16T09:35:06.683659+0000","last_active":"2024-09-16T09:35:07.633070+0000","last_peered":"2024-09-16T09:35:07.633070+0000","last_clean":"2024-09-16T09:35:07.633070+0000","last_became_active":"2024-09-16T09:33:52.914803+0000","last_became_peered":"2024-09-16T09:33:52.914803+0000","last_unstale":"2024-09-16T09:35:07.633070+0000","last_undegraded":"2024-09-16T09:35:07.633070+0000","last_fullsized":"2024-09-16T09:35:07.633070+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:33:24.230486+0000","objects_scrubbed":0,"log_size":5570,"log_dups_size":0,"ondisk_log_size":5570,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:35:36.018919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":235536384,"num_objects":3594,"num_object_clones":0,"num_object_copies":10782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3594,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3988,"num_write_kb":242624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,2],"acting":[6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.8","version":"73'79","reported_seq":97,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.685379+0000","last_change":"2024-09-16T09:34:30.931922+0000","last_active":"2024-09-16T09:35:06.685379+0000","last_peered":"2024-09-16T09:35:06.685379+0000","last_clean":"2024-09-16T09:35:06.685379+0000","last_became_active":"2024-09-16T09:34:00.329241+0000","last_became_peered":"2024-09-16T09:34:00.329241+0000","last_unstale":"2024-09-16T09:35:06.685379+0000","last_undegraded":"2024-09-16T09:35:06.685379+0000","last_fullsized":"2024-09-16T09:35:06.685379+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:34:30.931863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:34:30.931863+0000","objects_scrubbed":1,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:35:51.811083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262688,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,3],"acting":[6,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"66'17","reported_seq":514,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.664651+0000","last_change":"2024-09-16T09:30:04.636115+0000","last_active":"2024-09-16T09:35:06.664651+0000","last_peered":"2024-09-16T09:35:06.664651+0000","last_clean":"2024-09-16T09:35:06.664651+0000","last_became_active":"2024-09-16T09:25:03.713358+0000","last_became_peered":"2024-09-16T09:25:03.713358+0000","last_unstale":"2024-09-16T09:35:06.664651+0000","last_undegraded":"2024-09-16T09:35:06.664651+0000","last_fullsized":"2024-09-16T09:35:06.664651+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'17","last_scrub_stamp":"2024-09-16T09:30:04.636039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:30:04.636039+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":482,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.683942+0000","last_change":"2024-09-16T09:34:32.899965+0000","last_active":"2024-09-16T09:35:06.683942+0000","last_peered":"2024-09-16T09:35:06.683942+0000","last_clean":"2024-09-16T09:35:06.683942+0000","last_became_active":"2024-09-16T09:25:01.703355+0000","last_became_peered":"2024-09-16T09:25:01.703355+0000","last_unstale":"2024-09-16T09:35:06.683942+0000","last_undegraded":"2024-09-16T09:35:06.683942+0000","last_fullsized":"2024-09-16T09:35:06.683942+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:32.899886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:34:32.899886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:35:58.423518+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":121,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.664080+0000","last_change":"2024-09-16T09:33:44.298324+0000","last_active":"2024-09-16T09:35:06.664080+0000","last_peered":"2024-09-16T09:35:06.664080+0000","last_clean":"2024-09-16T09:35:06.664080+0000","last_became_active":"2024-09-16T09:33:30.251186+0000","last_became_peered":"2024-09-16T09:33:30.251186+0000","last_unstale":"2024-09-16T09:35:06.664080+0000","last_undegraded":"2024-09-16T09:35:06.664080+0000","last_fullsized":"2024-09-16T09:35:06.664080+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:44.298246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:44.298246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,0],"acting":[4,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":411,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.666427+0000","last_change":"2024-09-16T09:33:21.337606+0000","last_active":"2024-09-16T09:35:06.666427+0000","last_peered":"2024-09-16T09:35:06.666427+0000","last_clean":"2024-09-16T09:35:06.666427+0000","last_became_active":"2024-09-16T09:33:21.337291+0000","last_became_peered":"2024-09-16T09:33:21.337291+0000","last_unstale":"2024-09-16T09:35:06.666427+0000","last_undegraded":"2024-09-16T09:35:06.666427+0000","last_fullsized":"2024-09-16T09:35:06.666427+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:48.259971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:32:48.259971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:36:06.340108+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4],"acting":[3,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1","version":"284'5494","reported_seq":5663,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:07.583905+0000","last_change":"2024-09-16T09:35:05.665756+0000","last_active":"2024-09-16T09:35:07.583905+0000","last_peered":"2024-09-16T09:35:07.583905+0000","last_clean":"2024-09-16T09:35:07.583905+0000","last_became_active":"2024-09-16T09:33:52.910622+0000","last_became_peered":"2024-09-16T09:33:52.910622+0000","last_unstale":"2024-09-16T09:35:07.583905+0000","last_undegraded":"2024-09-16T09:35:07.583905+0000","last_fullsized":"2024-09-16T09:35:07.583905+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:33:24.230486+0000","objects_scrubbed":0,"log_size":5494,"log_dups_size":0,"ondisk_log_size":5494,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:35:34.975804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":235995136,"num_objects":3601,"num_object_clones":0,"num_object_copies":10803,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3601,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3973,"num_write_kb":242368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.9","version":"73'79","reported_seq":545,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.669265+0000","last_change":"2024-09-16T09:29:19.013948+0000","last_active":"2024-09-16T09:35:06.669265+0000","last_peered":"2024-09-16T09:35:06.669265+0000","last_clean":"2024-09-16T09:35:06.669265+0000","last_became_active":"2024-09-16T09:26:35.126066+0000","last_became_peered":"2024-09-16T09:26:35.126066+0000","last_unstale":"2024-09-16T09:35:06.669265+0000","last_undegraded":"2024-09-16T09:35:06.669265+0000","last_fullsized":"2024-09-16T09:35:06.669265+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:19.013865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:19.013865+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"66'10","reported_seq":485,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.664235+0000","last_change":"2024-09-16T09:33:24.154567+0000","last_active":"2024-09-16T09:35:06.664235+0000","last_peered":"2024-09-16T09:35:06.664235+0000","last_clean":"2024-09-16T09:35:06.664235+0000","last_became_active":"2024-09-16T09:25:03.707723+0000","last_became_peered":"2024-09-16T09:25:03.707723+0000","last_unstale":"2024-09-16T09:35:06.664235+0000","last_undegraded":"2024-09-16T09:35:06.664235+0000","last_fullsized":"2024-09-16T09:35:06.664235+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-16T09:33:24.154507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:33:24.154507+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":347,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.667037+0000","last_change":"2024-09-16T09:33:10.528181+0000","last_active":"2024-09-16T09:35:06.667037+0000","last_peered":"2024-09-16T09:35:06.667037+0000","last_clean":"2024-09-16T09:35:06.667037+0000","last_became_active":"2024-09-16T09:31:09.288825+0000","last_became_peered":"2024-09-16T09:31:09.288825+0000","last_unstale":"2024-09-16T09:35:06.667037+0000","last_undegraded":"2024-09-16T09:35:06.667037+0000","last_fullsized":"2024-09-16T09:35:06.667037+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:10.528115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:10.528115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"228'9","reported_seq":176,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.670475+0000","last_change":"2024-09-16T09:33:21.337623+0000","last_active":"2024-09-16T09:35:06.670475+0000","last_peered":"2024-09-16T09:35:06.670475+0000","last_clean":"2024-09-16T09:35:06.670475+0000","last_became_active":"2024-09-16T09:33:21.337213+0000","last_became_peered":"2024-09-16T09:33:21.337213+0000","last_unstale":"2024-09-16T09:35:06.670475+0000","last_undegraded":"2024-09-16T09:35:06.670475+0000","last_fullsized":"2024-09-16T09:35:06.670475+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:32:42.175528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:32:42.175528+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":486,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.664456+0000","last_change":"2024-09-16T09:32:39.072795+0000","last_active":"2024-09-16T09:35:06.664456+0000","last_peered":"2024-09-16T09:35:06.664456+0000","last_clean":"2024-09-16T09:35:06.664456+0000","last_became_active":"2024-09-16T09:25:07.718958+0000","last_became_peered":"2024-09-16T09:25:07.718958+0000","last_unstale":"2024-09-16T09:35:06.664456+0000","last_undegraded":"2024-09-16T09:35:06.664456+0000","last_fullsized":"2024-09-16T09:35:06.664456+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:39.072735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:32:39.072735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.6","version":"284'5615","reported_seq":5792,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:08.029719+0000","last_change":"2024-09-16T09:35:05.663112+0000","last_active":"2024-09-16T09:35:08.029719+0000","last_peered":"2024-09-16T09:35:08.029719+0000","last_clean":"2024-09-16T09:35:08.029719+0000","last_became_active":"2024-09-16T09:33:52.928727+0000","last_became_peered":"2024-09-16T09:33:52.928727+0000","last_unstale":"2024-09-16T09:35:08.029719+0000","last_undegraded":"2024-09-16T09:35:08.029719+0000","last_fullsized":"2024-09-16T09:35:08.029719+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:33:24.230486+0000","objects_scrubbed":0,"log_size":5615,"log_dups_size":0,"ondisk_log_size":5615,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:35:35.589417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":237502464,"num_objects":3624,"num_object_clones":0,"num_object_copies":10872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3624,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4050,"num_write_kb":245568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.e","version":"73'79","reported_seq":279,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.664466+0000","last_change":"2024-09-16T09:32:43.092027+0000","last_active":"2024-09-16T09:35:06.664466+0000","last_peered":"2024-09-16T09:35:06.664466+0000","last_clean":"2024-09-16T09:35:06.664466+0000","last_became_active":"2024-09-16T09:30:10.705072+0000","last_became_peered":"2024-09-16T09:30:10.705072+0000","last_unstale":"2024-09-16T09:35:06.664466+0000","last_undegraded":"2024-09-16T09:35:06.664466+0000","last_fullsized":"2024-09-16T09:35:06.664466+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:32:43.091969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:32:43.091969+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,3],"acting":[4,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.b","version":"66'9","reported_seq":502,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.666912+0000","last_change":"2024-09-16T09:31:31.211495+0000","last_active":"2024-09-16T09:35:06.666912+0000","last_peered":"2024-09-16T09:35:06.666912+0000","last_clean":"2024-09-16T09:35:06.666912+0000","last_became_active":"2024-09-16T09:25:03.712230+0000","last_became_peered":"2024-09-16T09:25:03.712230+0000","last_unstale":"2024-09-16T09:35:06.666912+0000","last_undegraded":"2024-09-16T09:35:06.666912+0000","last_fullsized":"2024-09-16T09:35:06.666912+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:31:31.211420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:31:31.211420+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:36:02.986832+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":487,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.667630+0000","last_change":"2024-09-16T09:31:09.277413+0000","last_active":"2024-09-16T09:35:06.667630+0000","last_peered":"2024-09-16T09:35:06.667630+0000","last_clean":"2024-09-16T09:35:06.667630+0000","last_became_active":"2024-09-16T09:31:09.276516+0000","last_became_peered":"2024-09-16T09:31:09.276516+0000","last_unstale":"2024-09-16T09:35:06.667630+0000","last_undegraded":"2024-09-16T09:35:06.667630+0000","last_fullsized":"2024-09-16T09:35:06.667630+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:26.061106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:26.061106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":415,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.663783+0000","last_change":"2024-09-16T09:33:40.260547+0000","last_active":"2024-09-16T09:35:06.663783+0000","last_peered":"2024-09-16T09:35:06.663783+0000","last_clean":"2024-09-16T09:35:06.663783+0000","last_became_active":"2024-09-16T09:33:21.337254+0000","last_became_peered":"2024-09-16T09:33:21.337254+0000","last_unstale":"2024-09-16T09:35:06.663783+0000","last_undegraded":"2024-09-16T09:35:06.663783+0000","last_fullsized":"2024-09-16T09:35:06.663783+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:40.260500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:40.260500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3],"acting":[4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":478,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.667507+0000","last_change":"2024-09-16T09:31:27.185957+0000","last_active":"2024-09-16T09:35:06.667507+0000","last_peered":"2024-09-16T09:35:06.667507+0000","last_clean":"2024-09-16T09:35:06.667507+0000","last_became_active":"2024-09-16T09:25:07.718778+0000","last_became_peered":"2024-09-16T09:25:07.718778+0000","last_unstale":"2024-09-16T09:35:06.667507+0000","last_undegraded":"2024-09-16T09:35:06.667507+0000","last_fullsized":"2024-09-16T09:35:06.667507+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:31:27.185801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:31:27.185801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"284'5531","reported_seq":5690,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:08.030002+0000","last_change":"2024-09-16T09:33:53.762479+0000","last_active":"2024-09-16T09:35:08.030002+0000","last_peered":"2024-09-16T09:35:08.030002+0000","last_clean":"2024-09-16T09:35:08.030002+0000","last_became_active":"2024-09-16T09:33:52.930653+0000","last_became_peered":"2024-09-16T09:33:52.930653+0000","last_unstale":"2024-09-16T09:35:08.030002+0000","last_undegraded":"2024-09-16T09:35:08.030002+0000","last_fullsized":"2024-09-16T09:35:08.030002+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"244'3138","last_scrub_stamp":"2024-09-16T09:33:53.762280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:33:53.762280+0000","objects_scrubbed":1562,"log_size":5531,"log_dups_size":0,"ondisk_log_size":5531,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":422,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":235732992,"num_objects":3597,"num_object_clones":0,"num_object_copies":10791,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3597,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2407,"num_write_kb":142144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.f","version":"73'79","reported_seq":539,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.685159+0000","last_change":"2024-09-16T09:34:53.866426+0000","last_active":"2024-09-16T09:35:06.685159+0000","last_peered":"2024-09-16T09:35:06.685159+0000","last_clean":"2024-09-16T09:35:06.685159+0000","last_became_active":"2024-09-16T09:30:10.693477+0000","last_became_peered":"2024-09-16T09:30:10.693477+0000","last_unstale":"2024-09-16T09:35:06.685159+0000","last_undegraded":"2024-09-16T09:35:06.685159+0000","last_fullsized":"2024-09-16T09:35:06.685159+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:34:53.866337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:34:53.866337+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:36:03.166374+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"66'19","reported_seq":517,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.683719+0000","last_change":"2024-09-16T09:33:22.905808+0000","last_active":"2024-09-16T09:35:06.683719+0000","last_peered":"2024-09-16T09:35:06.683719+0000","last_clean":"2024-09-16T09:35:06.683719+0000","last_became_active":"2024-09-16T09:25:03.712783+0000","last_became_peered":"2024-09-16T09:25:03.712783+0000","last_unstale":"2024-09-16T09:35:06.683719+0000","last_undegraded":"2024-09-16T09:35:06.683719+0000","last_fullsized":"2024-09-16T09:35:06.683719+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-16T09:33:22.905752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:33:22.905752+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"59'1","reported_seq":495,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.670720+0000","last_change":"2024-09-16T09:33:42.294543+0000","last_active":"2024-09-16T09:35:06.670720+0000","last_peered":"2024-09-16T09:35:06.670720+0000","last_clean":"2024-09-16T09:35:06.670720+0000","last_became_active":"2024-09-16T09:25:01.708069+0000","last_became_peered":"2024-09-16T09:25:01.708069+0000","last_unstale":"2024-09-16T09:35:06.670720+0000","last_undegraded":"2024-09-16T09:35:06.670720+0000","last_fullsized":"2024-09-16T09:35:06.670720+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:33:42.294378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:42.294378+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:35:07.729855+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":489,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.689722+0000","last_change":"2024-09-16T09:34:38.338415+0000","last_active":"2024-09-16T09:35:06.689722+0000","last_peered":"2024-09-16T09:35:06.689722+0000","last_clean":"2024-09-16T09:35:06.689722+0000","last_became_active":"2024-09-16T09:34:00.330050+0000","last_became_peered":"2024-09-16T09:34:00.330050+0000","last_unstale":"2024-09-16T09:35:06.689722+0000","last_undegraded":"2024-09-16T09:35:06.689722+0000","last_fullsized":"2024-09-16T09:35:06.689722+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:38.338329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:34:38.338329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:36:06.360092+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,7],"acting":[2,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":461,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.668411+0000","last_change":"2024-09-16T09:27:25.865107+0000","last_active":"2024-09-16T09:35:06.668411+0000","last_peered":"2024-09-16T09:35:06.668411+0000","last_clean":"2024-09-16T09:35:06.668411+0000","last_became_active":"2024-09-16T09:25:07.718750+0000","last_became_peered":"2024-09-16T09:25:07.718750+0000","last_unstale":"2024-09-16T09:35:06.668411+0000","last_undegraded":"2024-09-16T09:35:06.668411+0000","last_fullsized":"2024-09-16T09:35:06.668411+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:25.865030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:25.865030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.4","version":"284'5711","reported_seq":5860,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:07.856718+0000","last_change":"2024-09-16T09:33:53.661296+0000","last_active":"2024-09-16T09:35:07.856718+0000","last_peered":"2024-09-16T09:35:07.856718+0000","last_clean":"2024-09-16T09:35:07.856718+0000","last_became_active":"2024-09-16T09:33:52.909537+0000","last_became_peered":"2024-09-16T09:33:52.909537+0000","last_unstale":"2024-09-16T09:35:07.856718+0000","last_undegraded":"2024-09-16T09:35:07.856718+0000","last_fullsized":"2024-09-16T09:35:07.856718+0000","mapping_epoch":233,"log_start":"234'23","ondisk_log_start":"234'23","created":229,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"244'3237","last_scrub_stamp":"2024-09-16T09:33:53.661132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:33:53.661132+0000","objects_scrubbed":1642,"log_size":5688,"log_dups_size":0,"ondisk_log_size":5688,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":365,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":241303552,"num_objects":3682,"num_object_clones":0,"num_object_copies":11046,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3682,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2482,"num_write_kb":144704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.c","version":"73'79","reported_seq":219,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.664540+0000","last_change":"2024-09-16T09:33:08.949107+0000","last_active":"2024-09-16T09:35:06.664540+0000","last_peered":"2024-09-16T09:35:06.664540+0000","last_clean":"2024-09-16T09:35:06.664540+0000","last_became_active":"2024-09-16T09:31:09.273770+0000","last_became_peered":"2024-09-16T09:31:09.273770+0000","last_unstale":"2024-09-16T09:35:06.664540+0000","last_undegraded":"2024-09-16T09:35:06.664540+0000","last_fullsized":"2024-09-16T09:35:06.664540+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:33:08.949051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:33:08.949051+0000","objects_scrubbed":1,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":327680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"66'12","reported_seq":539,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.663919+0000","last_change":"2024-09-16T09:31:09.296915+0000","last_active":"2024-09-16T09:35:06.663919+0000","last_peered":"2024-09-16T09:35:06.663919+0000","last_clean":"2024-09-16T09:35:06.663919+0000","last_became_active":"2024-09-16T09:31:09.269047+0000","last_became_peered":"2024-09-16T09:31:09.269047+0000","last_unstale":"2024-09-16T09:35:06.663919+0000","last_undegraded":"2024-09-16T09:35:06.663919+0000","last_fullsized":"2024-09-16T09:35:06.663919+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-16T09:30:21.728668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:30:21.728668+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":489,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.669969+0000","last_change":"2024-09-16T09:33:48.225867+0000","last_active":"2024-09-16T09:35:06.669969+0000","last_peered":"2024-09-16T09:35:06.669969+0000","last_clean":"2024-09-16T09:35:06.669969+0000","last_became_active":"2024-09-16T09:25:01.707528+0000","last_became_peered":"2024-09-16T09:25:01.707528+0000","last_unstale":"2024-09-16T09:35:06.669969+0000","last_undegraded":"2024-09-16T09:35:06.669969+0000","last_fullsized":"2024-09-16T09:35:06.669969+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:48.225635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:48.225635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:35:17.793189+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":133,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.930178+0000","last_change":"2024-09-16T09:33:23.643136+0000","last_active":"2024-09-16T09:35:06.930178+0000","last_peered":"2024-09-16T09:35:06.930178+0000","last_clean":"2024-09-16T09:35:06.930178+0000","last_became_active":"2024-09-16T09:33:21.337309+0000","last_became_peered":"2024-09-16T09:33:21.337309+0000","last_unstale":"2024-09-16T09:35:06.930178+0000","last_undegraded":"2024-09-16T09:35:06.930178+0000","last_fullsized":"2024-09-16T09:35:06.930178+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:23.643089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:23.643089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":470,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.663925+0000","last_change":"2024-09-16T09:34:30.458186+0000","last_active":"2024-09-16T09:35:06.663925+0000","last_peered":"2024-09-16T09:35:06.663925+0000","last_clean":"2024-09-16T09:35:06.663925+0000","last_became_active":"2024-09-16T09:25:07.718899+0000","last_became_peered":"2024-09-16T09:25:07.718899+0000","last_unstale":"2024-09-16T09:35:06.663925+0000","last_undegraded":"2024-09-16T09:35:06.663925+0000","last_fullsized":"2024-09-16T09:35:06.663925+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:30.458115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:34:30.458115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:35:38.072321+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.5","version":"284'5657","reported_seq":9308,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:08.554776+0000","last_change":"2024-09-16T09:34:34.440150+0000","last_active":"2024-09-16T09:35:08.554776+0000","last_peered":"2024-09-16T09:35:08.554776+0000","last_clean":"2024-09-16T09:35:08.554776+0000","last_became_active":"2024-09-16T09:34:32.381065+0000","last_became_peered":"2024-09-16T09:34:32.381065+0000","last_unstale":"2024-09-16T09:35:08.554776+0000","last_undegraded":"2024-09-16T09:35:08.554776+0000","last_fullsized":"2024-09-16T09:35:08.554776+0000","mapping_epoch":267,"log_start":"0'0","ondisk_log_start":"0'0","created":229,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'4368","last_scrub_stamp":"2024-09-16T09:34:34.439986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:34:34.439986+0000","objects_scrubbed":2756,"log_size":5657,"log_dups_size":0,"ondisk_log_size":5657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:35:40.340118+0000","scrub_duration":710,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":238682112,"num_objects":3642,"num_object_clones":0,"num_object_copies":10926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3642,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1300,"num_write_kb":69952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.d","version":"73'79","reported_seq":279,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.670236+0000","last_change":"2024-09-16T09:32:44.536868+0000","last_active":"2024-09-16T09:35:06.670236+0000","last_peered":"2024-09-16T09:35:06.670236+0000","last_clean":"2024-09-16T09:35:06.670236+0000","last_became_active":"2024-09-16T09:30:10.709958+0000","last_became_peered":"2024-09-16T09:30:10.709958+0000","last_unstale":"2024-09-16T09:35:06.670236+0000","last_undegraded":"2024-09-16T09:35:06.670236+0000","last_fullsized":"2024-09-16T09:35:06.670236+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:32:44.536799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:32:44.536799+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"66'15","reported_seq":494,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.668301+0000","last_change":"2024-09-16T09:26:33.138828+0000","last_active":"2024-09-16T09:35:06.668301+0000","last_peered":"2024-09-16T09:35:06.668301+0000","last_clean":"2024-09-16T09:35:06.668301+0000","last_became_active":"2024-09-16T09:25:03.715733+0000","last_became_peered":"2024-09-16T09:25:03.715733+0000","last_unstale":"2024-09-16T09:35:06.668301+0000","last_undegraded":"2024-09-16T09:35:06.668301+0000","last_fullsized":"2024-09-16T09:35:06.668301+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-16T09:26:33.138603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:26:33.138603+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":493,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.670180+0000","last_change":"2024-09-16T09:32:49.441194+0000","last_active":"2024-09-16T09:35:06.670180+0000","last_peered":"2024-09-16T09:35:06.670180+0000","last_clean":"2024-09-16T09:35:06.670180+0000","last_became_active":"2024-09-16T09:25:01.707372+0000","last_became_peered":"2024-09-16T09:25:01.707372+0000","last_unstale":"2024-09-16T09:35:06.670180+0000","last_undegraded":"2024-09-16T09:35:06.670180+0000","last_fullsized":"2024-09-16T09:35:06.670180+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-16T09:32:49.441122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:32:49.441122+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"66'8","reported_seq":611,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.668825+0000","last_change":"2024-09-16T09:30:05.138362+0000","last_active":"2024-09-16T09:35:06.668825+0000","last_peered":"2024-09-16T09:35:06.668825+0000","last_clean":"2024-09-16T09:35:06.668825+0000","last_became_active":"2024-09-16T09:25:05.716189+0000","last_became_peered":"2024-09-16T09:25:05.716189+0000","last_unstale":"2024-09-16T09:35:06.668825+0000","last_undegraded":"2024-09-16T09:35:06.668825+0000","last_fullsized":"2024-09-16T09:35:06.668825+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:30:05.138288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:30:05.138288+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":461,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.667112+0000","last_change":"2024-09-16T09:27:20.219376+0000","last_active":"2024-09-16T09:35:06.667112+0000","last_peered":"2024-09-16T09:35:06.667112+0000","last_clean":"2024-09-16T09:35:06.667112+0000","last_became_active":"2024-09-16T09:25:07.717213+0000","last_became_peered":"2024-09-16T09:25:07.717213+0000","last_unstale":"2024-09-16T09:35:06.667112+0000","last_undegraded":"2024-09-16T09:35:06.667112+0000","last_fullsized":"2024-09-16T09:35:06.667112+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:20.219333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:20.219333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.12","version":"73'79","reported_seq":279,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.667332+0000","last_change":"2024-09-16T09:32:45.399435+0000","last_active":"2024-09-16T09:35:06.667332+0000","last_peered":"2024-09-16T09:35:06.667332+0000","last_clean":"2024-09-16T09:35:06.667332+0000","last_became_active":"2024-09-16T09:30:10.731413+0000","last_became_peered":"2024-09-16T09:30:10.731413+0000","last_unstale":"2024-09-16T09:35:06.667332+0000","last_undegraded":"2024-09-16T09:35:06.667332+0000","last_fullsized":"2024-09-16T09:35:06.667332+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:32:45.399361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:32:45.399361+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":489,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.684349+0000","last_change":"2024-09-16T09:34:00.334779+0000","last_active":"2024-09-16T09:35:06.684349+0000","last_peered":"2024-09-16T09:35:06.684349+0000","last_clean":"2024-09-16T09:35:06.684349+0000","last_became_active":"2024-09-16T09:34:00.334361+0000","last_became_peered":"2024-09-16T09:34:00.334361+0000","last_unstale":"2024-09-16T09:35:06.684349+0000","last_undegraded":"2024-09-16T09:35:06.684349+0000","last_fullsized":"2024-09-16T09:35:06.684349+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:14.845355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:14.845355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:35:57.854802+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"66'6","reported_seq":231,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.666656+0000","last_change":"2024-09-16T09:34:51.181994+0000","last_active":"2024-09-16T09:35:06.666656+0000","last_peered":"2024-09-16T09:35:06.666656+0000","last_clean":"2024-09-16T09:35:06.666656+0000","last_became_active":"2024-09-16T09:31:09.270685+0000","last_became_peered":"2024-09-16T09:31:09.270685+0000","last_unstale":"2024-09-16T09:35:06.666656+0000","last_undegraded":"2024-09-16T09:35:06.666656+0000","last_fullsized":"2024-09-16T09:35:06.666656+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'6","last_scrub_stamp":"2024-09-16T09:34:51.181918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:34:51.181918+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:36:17.849346+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":8,"num_read_kb":5,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":473,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.668067+0000","last_change":"2024-09-16T09:29:31.496374+0000","last_active":"2024-09-16T09:35:06.668067+0000","last_peered":"2024-09-16T09:35:06.668067+0000","last_clean":"2024-09-16T09:35:06.668067+0000","last_became_active":"2024-09-16T09:25:05.715616+0000","last_became_peered":"2024-09-16T09:25:05.715616+0000","last_unstale":"2024-09-16T09:35:06.668067+0000","last_undegraded":"2024-09-16T09:35:06.668067+0000","last_fullsized":"2024-09-16T09:35:06.668067+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:31.496304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:31.496304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":461,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.670333+0000","last_change":"2024-09-16T09:26:28.733381+0000","last_active":"2024-09-16T09:35:06.670333+0000","last_peered":"2024-09-16T09:35:06.670333+0000","last_clean":"2024-09-16T09:35:06.670333+0000","last_became_active":"2024-09-16T09:25:07.721059+0000","last_became_peered":"2024-09-16T09:25:07.721059+0000","last_unstale":"2024-09-16T09:35:06.670333+0000","last_undegraded":"2024-09-16T09:35:06.670333+0000","last_fullsized":"2024-09-16T09:35:06.670333+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:28.733324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:28.733324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.13","version":"73'79","reported_seq":372,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.664050+0000","last_change":"2024-09-16T09:34:48.260121+0000","last_active":"2024-09-16T09:35:06.664050+0000","last_peered":"2024-09-16T09:35:06.664050+0000","last_clean":"2024-09-16T09:35:06.664050+0000","last_became_active":"2024-09-16T09:33:30.248497+0000","last_became_peered":"2024-09-16T09:33:30.248497+0000","last_unstale":"2024-09-16T09:35:06.664050+0000","last_undegraded":"2024-09-16T09:35:06.664050+0000","last_fullsized":"2024-09-16T09:35:06.664050+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:34:48.260041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:34:48.260041+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:35:49.647205+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"66'9","reported_seq":552,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.666999+0000","last_change":"2024-09-16T09:33:48.681742+0000","last_active":"2024-09-16T09:35:06.666999+0000","last_peered":"2024-09-16T09:35:06.666999+0000","last_clean":"2024-09-16T09:35:06.666999+0000","last_became_active":"2024-09-16T09:31:09.273498+0000","last_became_peered":"2024-09-16T09:31:09.273498+0000","last_unstale":"2024-09-16T09:35:06.666999+0000","last_undegraded":"2024-09-16T09:35:06.666999+0000","last_fullsized":"2024-09-16T09:35:06.666999+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:33:48.681650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:33:48.681650+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":489,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.669065+0000","last_change":"2024-09-16T09:33:13.312434+0000","last_active":"2024-09-16T09:35:06.669065+0000","last_peered":"2024-09-16T09:35:06.669065+0000","last_clean":"2024-09-16T09:35:06.669065+0000","last_became_active":"2024-09-16T09:25:01.706491+0000","last_became_peered":"2024-09-16T09:25:01.706491+0000","last_unstale":"2024-09-16T09:35:06.669065+0000","last_undegraded":"2024-09-16T09:35:06.669065+0000","last_fullsized":"2024-09-16T09:35:06.669065+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:13.312344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:13.312344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":238,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.669740+0000","last_change":"2024-09-16T09:33:30.249293+0000","last_active":"2024-09-16T09:35:06.669740+0000","last_peered":"2024-09-16T09:35:06.669740+0000","last_clean":"2024-09-16T09:35:06.669740+0000","last_became_active":"2024-09-16T09:33:30.248992+0000","last_became_peered":"2024-09-16T09:33:30.248992+0000","last_unstale":"2024-09-16T09:35:06.669740+0000","last_undegraded":"2024-09-16T09:35:06.669740+0000","last_fullsized":"2024-09-16T09:35:06.669740+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:53.810580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:28:53.810580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":478,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.690906+0000","last_change":"2024-09-16T09:34:32.309307+0000","last_active":"2024-09-16T09:35:06.690906+0000","last_peered":"2024-09-16T09:35:06.690906+0000","last_clean":"2024-09-16T09:35:06.690906+0000","last_became_active":"2024-09-16T09:25:07.722050+0000","last_became_peered":"2024-09-16T09:25:07.722050+0000","last_unstale":"2024-09-16T09:35:06.690906+0000","last_undegraded":"2024-09-16T09:35:06.690906+0000","last_fullsized":"2024-09-16T09:35:06.690906+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:32.309262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:34:32.309262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:35:52.868735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.18","version":"284'5537","reported_seq":10530,"reported_epoch":284,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T09:35:08.110292+0000","last_change":"2024-09-16T09:35:07.069020+0000","last_active":"2024-09-16T09:35:08.110292+0000","last_peered":"2024-09-16T09:35:08.110292+0000","last_clean":"2024-09-16T09:35:04.658405+0000","last_became_active":"2024-09-16T09:35:07.029937+0000","last_became_peered":"2024-09-16T09:35:07.029937+0000","last_unstale":"2024-09-16T09:35:08.110292+0000","last_undegraded":"2024-09-16T09:35:06.861421+0000","last_fullsized":"2024-09-16T09:35:06.666924+0000","mapping_epoch":283,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":5,"last_scrub":"275'4947","last_scrub_stamp":"2024-09-16T09:34:50.611186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:34:50.611186+0000","objects_scrubbed":3349,"log_size":5537,"log_dups_size":0,"ondisk_log_size":5537,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":773,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":234815488,"num_objects":3583,"num_object_clones":0,"num_object_copies":10749,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6,"num_objects_misplaced":7148,"num_objects_unfound":0,"num_objects_dirty":3583,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":610,"num_write_kb":27008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":2490368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[5,7],"avail_no_missing":["5","6","7"],"object_location_counts":[{"shards":"5,6,7","objects":3571},{"shards":"5,7","objects":6}],"blocked_by":[],"up_primary":0,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.10","version":"73'79","reported_seq":279,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.668120+0000","last_change":"2024-09-16T09:32:44.405696+0000","last_active":"2024-09-16T09:35:06.668120+0000","last_peered":"2024-09-16T09:35:06.668120+0000","last_clean":"2024-09-16T09:35:06.668120+0000","last_became_active":"2024-09-16T09:30:10.692770+0000","last_became_peered":"2024-09-16T09:30:10.692770+0000","last_unstale":"2024-09-16T09:35:06.668120+0000","last_undegraded":"2024-09-16T09:35:06.668120+0000","last_fullsized":"2024-09-16T09:35:06.668120+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:32:44.405640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:32:44.405640+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"66'9","reported_seq":493,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.671916+0000","last_change":"2024-09-16T09:29:32.485165+0000","last_active":"2024-09-16T09:35:06.671916+0000","last_peered":"2024-09-16T09:35:06.671916+0000","last_clean":"2024-09-16T09:35:06.671916+0000","last_became_active":"2024-09-16T09:25:03.717310+0000","last_became_peered":"2024-09-16T09:25:03.717310+0000","last_unstale":"2024-09-16T09:35:06.671916+0000","last_undegraded":"2024-09-16T09:35:06.671916+0000","last_fullsized":"2024-09-16T09:35:06.671916+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:29:32.485084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:29:32.485084+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":512,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.919313+0000","last_change":"2024-09-16T09:33:20.639405+0000","last_active":"2024-09-16T09:35:06.919313+0000","last_peered":"2024-09-16T09:35:06.919313+0000","last_clean":"2024-09-16T09:35:06.919313+0000","last_became_active":"2024-09-16T09:31:09.272848+0000","last_became_peered":"2024-09-16T09:31:09.272848+0000","last_unstale":"2024-09-16T09:35:06.919313+0000","last_undegraded":"2024-09-16T09:35:06.919313+0000","last_fullsized":"2024-09-16T09:35:06.919313+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:20.639350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:20.639350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"187'11","reported_seq":448,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.669434+0000","last_change":"2024-09-16T09:33:20.296801+0000","last_active":"2024-09-16T09:35:06.669434+0000","last_peered":"2024-09-16T09:35:06.669434+0000","last_clean":"2024-09-16T09:35:06.669434+0000","last_became_active":"2024-09-16T09:31:09.277688+0000","last_became_peered":"2024-09-16T09:31:09.277688+0000","last_unstale":"2024-09-16T09:35:06.669434+0000","last_undegraded":"2024-09-16T09:35:06.669434+0000","last_fullsized":"2024-09-16T09:35:06.669434+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"187'11","last_scrub_stamp":"2024-09-16T09:33:20.296746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:20.296746+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":478,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.664634+0000","last_change":"2024-09-16T09:29:49.474440+0000","last_active":"2024-09-16T09:35:06.664634+0000","last_peered":"2024-09-16T09:35:06.664634+0000","last_clean":"2024-09-16T09:35:06.664634+0000","last_became_active":"2024-09-16T09:25:07.718810+0000","last_became_peered":"2024-09-16T09:25:07.718810+0000","last_unstale":"2024-09-16T09:35:06.664634+0000","last_undegraded":"2024-09-16T09:35:06.664634+0000","last_fullsized":"2024-09-16T09:35:06.664634+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:49.474391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:49.474391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.19","version":"284'5543","reported_seq":10500,"reported_epoch":284,"state":"active+remapped+backfill_toofull","last_fresh":"2024-09-16T09:35:07.911026+0000","last_change":"2024-09-16T09:35:06.822727+0000","last_active":"2024-09-16T09:35:07.911026+0000","last_peered":"2024-09-16T09:35:07.911026+0000","last_clean":"2024-09-16T09:35:04.639693+0000","last_became_active":"2024-09-16T09:35:06.797341+0000","last_became_peered":"2024-09-16T09:35:06.797341+0000","last_unstale":"2024-09-16T09:35:07.911026+0000","last_undegraded":"2024-09-16T09:35:07.911026+0000","last_fullsized":"2024-09-16T09:35:07.911026+0000","mapping_epoch":283,"log_start":"235'418","ondisk_log_start":"235'418","created":243,"last_epoch_clean":267,"parent":"0.0","parent_split_bits":5,"last_scrub":"267'4219","last_scrub_stamp":"2024-09-16T09:34:31.880547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:34:31.880547+0000","objects_scrubbed":2634,"log_size":5125,"log_dups_size":0,"ondisk_log_size":5125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":405,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":232194048,"num_objects":3543,"num_object_clones":0,"num_object_copies":10629,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":7086,"num_objects_unfound":0,"num_objects_dirty":3543,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1329,"num_write_kb":71616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,5],"acting":[3,5,6],"avail_no_missing":["3","5","6"],"object_location_counts":[{"shards":"3,5,6","objects":3543}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.11","version":"73'79","reported_seq":365,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.668912+0000","last_change":"2024-09-16T09:32:38.401888+0000","last_active":"2024-09-16T09:35:06.668912+0000","last_peered":"2024-09-16T09:35:06.668912+0000","last_clean":"2024-09-16T09:35:06.668912+0000","last_became_active":"2024-09-16T09:31:09.272208+0000","last_became_peered":"2024-09-16T09:31:09.272208+0000","last_unstale":"2024-09-16T09:35:06.668912+0000","last_undegraded":"2024-09-16T09:35:06.668912+0000","last_fullsized":"2024-09-16T09:35:06.668912+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:32:38.401720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:32:38.401720+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":32,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.14","version":"66'10","reported_seq":148,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.669486+0000","last_change":"2024-09-16T09:33:30.250101+0000","last_active":"2024-09-16T09:35:06.669486+0000","last_peered":"2024-09-16T09:35:06.669486+0000","last_clean":"2024-09-16T09:35:06.669486+0000","last_became_active":"2024-09-16T09:33:30.249769+0000","last_became_peered":"2024-09-16T09:33:30.249769+0000","last_unstale":"2024-09-16T09:35:06.669486+0000","last_undegraded":"2024-09-16T09:35:06.669486+0000","last_fullsized":"2024-09-16T09:35:06.669486+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-16T09:28:56.825476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:56.825476+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":14,"num_read_kb":9,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":481,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.683488+0000","last_change":"2024-09-16T09:29:56.021413+0000","last_active":"2024-09-16T09:35:06.683488+0000","last_peered":"2024-09-16T09:35:06.683488+0000","last_clean":"2024-09-16T09:35:06.683488+0000","last_became_active":"2024-09-16T09:25:01.707513+0000","last_became_peered":"2024-09-16T09:25:01.707513+0000","last_unstale":"2024-09-16T09:35:06.683488+0000","last_undegraded":"2024-09-16T09:35:06.683488+0000","last_fullsized":"2024-09-16T09:35:06.683488+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:56.021340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:56.021340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"66'8","reported_seq":619,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.671432+0000","last_change":"2024-09-16T09:33:39.388721+0000","last_active":"2024-09-16T09:35:06.671432+0000","last_peered":"2024-09-16T09:35:06.671432+0000","last_clean":"2024-09-16T09:35:06.671432+0000","last_became_active":"2024-09-16T09:25:05.713833+0000","last_became_peered":"2024-09-16T09:25:05.713833+0000","last_unstale":"2024-09-16T09:35:06.671432+0000","last_undegraded":"2024-09-16T09:35:06.671432+0000","last_fullsized":"2024-09-16T09:35:06.671432+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:33:39.388463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:39.388463+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":500,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.666560+0000","last_change":"2024-09-16T09:33:42.666290+0000","last_active":"2024-09-16T09:35:06.666560+0000","last_peered":"2024-09-16T09:35:06.666560+0000","last_clean":"2024-09-16T09:35:06.666560+0000","last_became_active":"2024-09-16T09:31:09.275168+0000","last_became_peered":"2024-09-16T09:31:09.275168+0000","last_unstale":"2024-09-16T09:35:06.666560+0000","last_undegraded":"2024-09-16T09:35:06.666560+0000","last_fullsized":"2024-09-16T09:35:06.666560+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:42.666200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:33:42.666200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:35:11.658552+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"66'11","reported_seq":508,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.664045+0000","last_change":"2024-09-16T09:33:19.084118+0000","last_active":"2024-09-16T09:35:06.664045+0000","last_peered":"2024-09-16T09:35:06.664045+0000","last_clean":"2024-09-16T09:35:06.664045+0000","last_became_active":"2024-09-16T09:26:33.291265+0000","last_became_peered":"2024-09-16T09:26:33.291265+0000","last_unstale":"2024-09-16T09:35:06.664045+0000","last_undegraded":"2024-09-16T09:35:06.664045+0000","last_fullsized":"2024-09-16T09:35:06.664045+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-16T09:33:19.084063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:33:19.084063+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":490,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.664397+0000","last_change":"2024-09-16T09:29:51.517112+0000","last_active":"2024-09-16T09:35:06.664397+0000","last_peered":"2024-09-16T09:35:06.664397+0000","last_clean":"2024-09-16T09:35:06.664397+0000","last_became_active":"2024-09-16T09:25:01.705158+0000","last_became_peered":"2024-09-16T09:25:01.705158+0000","last_unstale":"2024-09-16T09:35:06.664397+0000","last_undegraded":"2024-09-16T09:35:06.664397+0000","last_fullsized":"2024-09-16T09:35:06.664397+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:51.517054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:51.517054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":410,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.667202+0000","last_change":"2024-09-16T09:34:00.329968+0000","last_active":"2024-09-16T09:35:06.667202+0000","last_peered":"2024-09-16T09:35:06.667202+0000","last_clean":"2024-09-16T09:35:06.667202+0000","last_became_active":"2024-09-16T09:34:00.329642+0000","last_became_peered":"2024-09-16T09:34:00.329642+0000","last_unstale":"2024-09-16T09:35:06.667202+0000","last_undegraded":"2024-09-16T09:35:06.667202+0000","last_fullsized":"2024-09-16T09:35:06.667202+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:08.394830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:08.394830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":332,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.667408+0000","last_change":"2024-09-16T09:34:48.786847+0000","last_active":"2024-09-16T09:35:06.667408+0000","last_peered":"2024-09-16T09:35:06.667408+0000","last_clean":"2024-09-16T09:35:06.667408+0000","last_became_active":"2024-09-16T09:31:09.276042+0000","last_became_peered":"2024-09-16T09:31:09.276042+0000","last_unstale":"2024-09-16T09:35:06.667408+0000","last_undegraded":"2024-09-16T09:35:06.667408+0000","last_fullsized":"2024-09-16T09:35:06.667408+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:48.786785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:34:48.786785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:36:17.728202+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"66'9","reported_seq":166,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.683853+0000","last_change":"2024-09-16T09:34:15.299311+0000","last_active":"2024-09-16T09:35:06.683853+0000","last_peered":"2024-09-16T09:35:06.683853+0000","last_clean":"2024-09-16T09:35:06.683853+0000","last_became_active":"2024-09-16T09:34:00.331466+0000","last_became_peered":"2024-09-16T09:34:00.331466+0000","last_unstale":"2024-09-16T09:35:06.683853+0000","last_undegraded":"2024-09-16T09:35:06.683853+0000","last_fullsized":"2024-09-16T09:35:06.683853+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:32:47.247071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:32:47.247071+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:35:56.847392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":10,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,0],"acting":[6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":481,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.670600+0000","last_change":"2024-09-16T09:32:47.502667+0000","last_active":"2024-09-16T09:35:06.670600+0000","last_peered":"2024-09-16T09:35:06.670600+0000","last_clean":"2024-09-16T09:35:06.670600+0000","last_became_active":"2024-09-16T09:25:01.706762+0000","last_became_peered":"2024-09-16T09:25:01.706762+0000","last_unstale":"2024-09-16T09:35:06.670600+0000","last_undegraded":"2024-09-16T09:35:06.670600+0000","last_fullsized":"2024-09-16T09:35:06.670600+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:47.502598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:32:47.502598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":317,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.666610+0000","last_change":"2024-09-16T09:34:46.742640+0000","last_active":"2024-09-16T09:35:06.666610+0000","last_peered":"2024-09-16T09:35:06.666610+0000","last_clean":"2024-09-16T09:35:06.666610+0000","last_became_active":"2024-09-16T09:28:52.982865+0000","last_became_peered":"2024-09-16T09:28:52.982865+0000","last_unstale":"2024-09-16T09:35:06.666610+0000","last_undegraded":"2024-09-16T09:35:06.666610+0000","last_fullsized":"2024-09-16T09:35:06.666610+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:46.742562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:34:46.742562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:35:57.573477+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":478,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.930015+0000","last_change":"2024-09-16T09:34:50.779241+0000","last_active":"2024-09-16T09:35:06.930015+0000","last_peered":"2024-09-16T09:35:06.930015+0000","last_clean":"2024-09-16T09:35:06.930015+0000","last_became_active":"2024-09-16T09:25:07.715301+0000","last_became_peered":"2024-09-16T09:25:07.715301+0000","last_unstale":"2024-09-16T09:35:06.930015+0000","last_undegraded":"2024-09-16T09:35:06.930015+0000","last_fullsized":"2024-09-16T09:35:06.930015+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:50.779164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:34:50.779164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:36:11.307448+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.14","version":"73'79","reported_seq":279,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.690810+0000","last_change":"2024-09-16T09:32:44.163264+0000","last_active":"2024-09-16T09:35:06.690810+0000","last_peered":"2024-09-16T09:35:06.690810+0000","last_clean":"2024-09-16T09:35:06.690810+0000","last_became_active":"2024-09-16T09:30:10.700866+0000","last_became_peered":"2024-09-16T09:30:10.700866+0000","last_unstale":"2024-09-16T09:35:06.690810+0000","last_undegraded":"2024-09-16T09:35:06.690810+0000","last_fullsized":"2024-09-16T09:35:06.690810+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:32:44.163191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:32:44.163191+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:35:15.485909+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"66'11","reported_seq":356,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.668919+0000","last_change":"2024-09-16T09:28:55.643505+0000","last_active":"2024-09-16T09:35:06.668919+0000","last_peered":"2024-09-16T09:35:06.668919+0000","last_clean":"2024-09-16T09:35:06.668919+0000","last_became_active":"2024-09-16T09:28:52.971132+0000","last_became_peered":"2024-09-16T09:28:52.971132+0000","last_unstale":"2024-09-16T09:35:06.668919+0000","last_undegraded":"2024-09-16T09:35:06.668919+0000","last_fullsized":"2024-09-16T09:35:06.668919+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-16T09:28:55.643448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:55.643448+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":19,"num_read_kb":12,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":489,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.668209+0000","last_change":"2024-09-16T09:33:18.297533+0000","last_active":"2024-09-16T09:35:06.668209+0000","last_peered":"2024-09-16T09:35:06.668209+0000","last_clean":"2024-09-16T09:35:06.668209+0000","last_became_active":"2024-09-16T09:25:01.700733+0000","last_became_peered":"2024-09-16T09:25:01.700733+0000","last_unstale":"2024-09-16T09:35:06.668209+0000","last_undegraded":"2024-09-16T09:35:06.668209+0000","last_fullsized":"2024-09-16T09:35:06.668209+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:18.297451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:18.297451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":481,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.667187+0000","last_change":"2024-09-16T09:33:38.336281+0000","last_active":"2024-09-16T09:35:06.667187+0000","last_peered":"2024-09-16T09:35:06.667187+0000","last_clean":"2024-09-16T09:35:06.667187+0000","last_became_active":"2024-09-16T09:25:05.713479+0000","last_became_peered":"2024-09-16T09:25:05.713479+0000","last_unstale":"2024-09-16T09:35:06.667187+0000","last_undegraded":"2024-09-16T09:35:06.667187+0000","last_fullsized":"2024-09-16T09:35:06.667187+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:38.336221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:38.336221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":340,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.668320+0000","last_change":"2024-09-16T09:34:51.781047+0000","last_active":"2024-09-16T09:35:06.668320+0000","last_peered":"2024-09-16T09:35:06.668320+0000","last_clean":"2024-09-16T09:35:06.668320+0000","last_became_active":"2024-09-16T09:31:09.274797+0000","last_became_peered":"2024-09-16T09:31:09.274797+0000","last_unstale":"2024-09-16T09:35:06.668320+0000","last_undegraded":"2024-09-16T09:35:06.668320+0000","last_fullsized":"2024-09-16T09:35:06.668320+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:51.780980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:34:51.780980+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:36:10.448717+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"66'4","reported_seq":229,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.663654+0000","last_change":"2024-09-16T09:34:50.206999+0000","last_active":"2024-09-16T09:35:06.663654+0000","last_peered":"2024-09-16T09:35:06.663654+0000","last_clean":"2024-09-16T09:35:06.663654+0000","last_became_active":"2024-09-16T09:31:09.273682+0000","last_became_peered":"2024-09-16T09:31:09.273682+0000","last_unstale":"2024-09-16T09:35:06.663654+0000","last_undegraded":"2024-09-16T09:35:06.663654+0000","last_fullsized":"2024-09-16T09:35:06.663654+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'4","last_scrub_stamp":"2024-09-16T09:34:50.206916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:34:50.206916+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:35:57.708906+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":512,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.668696+0000","last_change":"2024-09-16T09:33:18.641115+0000","last_active":"2024-09-16T09:35:06.668696+0000","last_peered":"2024-09-16T09:35:06.668696+0000","last_clean":"2024-09-16T09:35:06.668696+0000","last_became_active":"2024-09-16T09:31:09.274719+0000","last_became_peered":"2024-09-16T09:31:09.274719+0000","last_unstale":"2024-09-16T09:35:06.668696+0000","last_undegraded":"2024-09-16T09:35:06.668696+0000","last_fullsized":"2024-09-16T09:35:06.668696+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:18.641069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:18.641069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":498,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.683893+0000","last_change":"2024-09-16T09:33:37.941485+0000","last_active":"2024-09-16T09:35:06.683893+0000","last_peered":"2024-09-16T09:35:06.683893+0000","last_clean":"2024-09-16T09:35:06.683893+0000","last_became_active":"2024-09-16T09:25:05.712545+0000","last_became_peered":"2024-09-16T09:25:05.712545+0000","last_unstale":"2024-09-16T09:35:06.683893+0000","last_undegraded":"2024-09-16T09:35:06.683893+0000","last_fullsized":"2024-09-16T09:35:06.683893+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:37.941403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:37.941403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":477,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.668974+0000","last_change":"2024-09-16T09:33:41.323824+0000","last_active":"2024-09-16T09:35:06.668974+0000","last_peered":"2024-09-16T09:35:06.668974+0000","last_clean":"2024-09-16T09:35:06.668974+0000","last_became_active":"2024-09-16T09:25:07.720117+0000","last_became_peered":"2024-09-16T09:25:07.720117+0000","last_unstale":"2024-09-16T09:35:06.668974+0000","last_undegraded":"2024-09-16T09:35:06.668974+0000","last_fullsized":"2024-09-16T09:35:06.668974+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:41.323773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:33:41.323773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.12","version":"284'5647","reported_seq":10837,"reported_epoch":284,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T09:35:08.561583+0000","last_change":"2024-09-16T09:35:07.002674+0000","last_active":"2024-09-16T09:35:08.561583+0000","last_peered":"2024-09-16T09:35:08.561583+0000","last_clean":"2024-09-16T09:35:04.631553+0000","last_became_active":"2024-09-16T09:35:06.928120+0000","last_became_peered":"2024-09-16T09:35:06.928120+0000","last_unstale":"2024-09-16T09:35:08.561583+0000","last_undegraded":"2024-09-16T09:35:06.916590+0000","last_fullsized":"2024-09-16T09:35:06.667188+0000","mapping_epoch":283,"log_start":"0'0","ondisk_log_start":"0'0","created":243,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":5,"last_scrub":"274'4970","last_scrub_stamp":"2024-09-16T09:34:48.381443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:34:48.381443+0000","objects_scrubbed":3385,"log_size":5647,"log_dups_size":0,"ondisk_log_size":5647,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":905,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":240254976,"num_objects":3666,"num_object_clones":0,"num_object_copies":10998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3,"num_objects_misplaced":7335,"num_objects_unfound":0,"num_objects_dirty":3666,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":695,"num_write_kb":31232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37,"num_bytes_recovered":2424832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[0,1],"avail_no_missing":["0","1","5"],"object_location_counts":[{"shards":"0,1","objects":-3},{"shards":"0,1,5","objects":3666}],"blocked_by":[],"up_primary":3,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":113,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.671389+0000","last_change":"2024-09-16T09:33:30.248335+0000","last_active":"2024-09-16T09:35:06.671389+0000","last_peered":"2024-09-16T09:35:06.671389+0000","last_clean":"2024-09-16T09:35:06.671389+0000","last_became_active":"2024-09-16T09:33:30.248036+0000","last_became_peered":"2024-09-16T09:33:30.248036+0000","last_unstale":"2024-09-16T09:35:06.671389+0000","last_undegraded":"2024-09-16T09:35:06.671389+0000","last_fullsized":"2024-09-16T09:35:06.671389+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:18.339152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:33:18.339152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":339,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.930592+0000","last_change":"2024-09-16T09:32:40.411546+0000","last_active":"2024-09-16T09:35:06.930592+0000","last_peered":"2024-09-16T09:35:06.930592+0000","last_clean":"2024-09-16T09:35:06.930592+0000","last_became_active":"2024-09-16T09:31:09.271662+0000","last_became_peered":"2024-09-16T09:31:09.271662+0000","last_unstale":"2024-09-16T09:35:06.930592+0000","last_undegraded":"2024-09-16T09:35:06.930592+0000","last_fullsized":"2024-09-16T09:35:06.930592+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:40.411487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:32:40.411487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"66'11","reported_seq":505,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.685246+0000","last_change":"2024-09-16T09:33:19.843216+0000","last_active":"2024-09-16T09:35:06.685246+0000","last_peered":"2024-09-16T09:35:06.685246+0000","last_clean":"2024-09-16T09:35:06.685246+0000","last_became_active":"2024-09-16T09:25:03.707769+0000","last_became_peered":"2024-09-16T09:25:03.707769+0000","last_unstale":"2024-09-16T09:35:06.685246+0000","last_undegraded":"2024-09-16T09:35:06.685246+0000","last_fullsized":"2024-09-16T09:35:06.685246+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-16T09:33:19.843154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:33:19.843154+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":474,"reported_epoch":284,"state":"active+clean","last_fresh":"2024-09-16T09:35:06.666717+0000","last_change":"2024-09-16T09:32:31.325543+0000","last_active":"2024-09-16T09:35:06.666717+0000","last_peered":"2024-09-16T09:35:06.666717+0000","last_clean":"2024-09-16T09:35:06.666717+0000","last_became_active":"2024-09-16T09:25:05.711952+0000","last_became_peered":"2024-09-16T09:25:05.711952+0000","last_unstale":"2024-09-16T09:35:06.666717+0000","last_undegraded":"2024-09-16T09:35:06.666717+0000","last_fullsized":"2024-09-16T09:35:06.666717+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:31.325346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:32:31.325346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":26,"stat_sum":{"num_bytes":7535362072,"num_objects":114982,"num_object_clones":0,"num_object_copies":344946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9,"num_objects_misplaced":56978,"num_objects_unfound":0,"num_objects_dirty":114982,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":96754,"num_write_kb":5767009,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7161,"num_bytes_recovered":469172224,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":23275581440,"data_stored":23275569224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":133829,"ondisk_log_size":133829,"up":78,"acting":74,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":688,"num_read_kb":445,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":116,"num_bytes_recovered":2916,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":395,"ondisk_log_size":395,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1986,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":600,"num_read_kb":595,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":21,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":327680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1659,"ondisk_log_size":1659,"up":63,"acting":63,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266814,"num_pgs":80,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":4796808,"kb_used_data":4382452,"kb_used_omap":1,"kb_used_meta":414334,"kb_avail":88948344,"statfs":{"total":95995035648,"available":91083104256,"internally_reserved":0,"allocated":4487630848,"data_stored":4486982631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":424278474},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495747,"num_pgs":61,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2182880,"kb_used_data":1876816,"kb_used_omap":1,"kb_used_meta":306046,"kb_avail":91562272,"statfs":{"total":95995035648,"available":93759766528,"internally_reserved":0,"allocated":1921859584,"data_stored":1921247323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":313391558},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724682,"num_pgs":69,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":4252208,"kb_used_data":3828120,"kb_used_omap":1,"kb_used_meta":424062,"kb_avail":89492944,"statfs":{"total":95995035648,"available":91640774656,"internally_reserved":0,"allocated":3919994880,"data_stored":3919374376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":434239942},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953615,"num_pgs":73,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2668544,"kb_used_data":2330540,"kb_used_omap":1,"kb_used_meta":337982,"kb_avail":91076608,"statfs":{"total":95995035648,"available":93262446592,"internally_reserved":0,"allocated":2386472960,"data_stored":2385832669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":346094022},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149845,"num_pgs":80,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3534668,"kb_used_data":3016184,"kb_used_omap":1,"kb_used_meta":518462,"kb_avail":90210484,"statfs":{"total":95995035648,"available":92375535616,"internally_reserved":0,"allocated":3088572416,"data_stored":3087920753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":530905543},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411482,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1273248,"kb_used_data":931084,"kb_used_omap":1,"kb_used_meta":342142,"kb_avail":92471904,"statfs":{"total":95995035648,"available":94691229696,"internally_reserved":0,"allocated":953430016,"data_stored":952821673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":350353861},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575009,"num_pgs":74,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":4943692,"kb_used_data":4425200,"kb_used_omap":1,"kb_used_meta":518462,"kb_avail":88801460,"statfs":{"total":95995035648,"available":90932695040,"internally_reserved":0,"allocated":4531404800,"data_stored":4530759819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":530905541},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738535,"num_pgs":64,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2501820,"kb_used_data":2091564,"kb_used_omap":1,"kb_used_meta":410174,"kb_avail":91243332,"statfs":{"total":95995035648,"available":93433171968,"internally_reserved":0,"allocated":2141761536,"data_stored":2141121533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":420018629},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2117533696,"data_stored":2117533696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4495835136,"data_stored":4495835136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":946077696,"data_stored":946077696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":3068919808,"data_stored":3068919808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":2351370240,"data_stored":2351366168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":3908243456,"data_stored":3908239384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1910439936,"data_stored":1910439936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4477161472,"data_stored":4477157400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T09:35:13.569 INFO:tasks.cephadm.ceph_manager.ceph:would drop below min_pgs, proj 6, currently 16 2024-09-16T09:35:13.569 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:35:13.596 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:13.596 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:13.596 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:35:13.697 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:35:13.743 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:35:13.789 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:13.789 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:13.790 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:35:13.870 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:35:13.890 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:35:13.943 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:13.943 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:13.944 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:35:14.044 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:35:14.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:14.068 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:14.068 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:35:14.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:13 smithi084 ceph-mon[32629]: 3.3 scrub starts 2024-09-16T09:35:14.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:13 smithi084 ceph-mon[32629]: 3.3 scrub ok 2024-09-16T09:35:14.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:13 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:35:14.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:13 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:35:14.169 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:35:14.170 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:13 smithi070 ceph-mon[30388]: 3.3 scrub starts 2024-09-16T09:35:14.170 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:13 smithi070 ceph-mon[30388]: 3.3 scrub ok 2024-09-16T09:35:14.170 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:13 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:35:14.170 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:13 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:35:14.170 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:13 smithi070 ceph-mon[40691]: 3.3 scrub starts 2024-09-16T09:35:14.170 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:13 smithi070 ceph-mon[40691]: 3.3 scrub ok 2024-09-16T09:35:14.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:13 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:35:14.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:13 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:35:14.214 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:14.215 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:14.215 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:35:14.316 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:35:14.350 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:14.350 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:14.351 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:35:14.452 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:35:14.476 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:14.476 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:14.476 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:35:14.577 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:35:14.605 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:14.606 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:14.606 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:35:14.707 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:35:14.725 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:35:14.725 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:35:14.743 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:14.743 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:14.744 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:35:14.762 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:35:14.763 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:35:14.798 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:35:14.798 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:35:14.833 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:35:14.833 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:35:14.845 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:35:14.866 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:35:14.866 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:35:14.877 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:14.878 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:14.878 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:35:14.902 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:35:14.902 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:35:14.938 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:35:14.939 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:35:14.979 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:35:14.991 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:35:14.991 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:35:15.019 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:15.020 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:15.020 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:35:15.121 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:35:15.142 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:15.143 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:15.143 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:35:15.244 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:35:15.277 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:15.277 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:15.278 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:35:15.379 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:35:15.412 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:15.413 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:15.413 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:35:15.514 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:35:15.544 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:15.545 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:15.545 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:35:15.646 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:35:15.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:15 smithi084 ceph-mon[32629]: pgmap v507: 178 pgs: 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 2 active+recovering+undersized+degraded+remapped, 3 active+remapped+backfill_toofull, 170 active+clean; 6.8 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 62 KiB/s rd, 0 B/s wr, 1.17k op/s; 14/333642 objects degraded (0.004%); 52549/333642 objects misplaced (15.750%); 27 MiB/s, 432 objects/s recovering 2024-09-16T09:35:15.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:15 smithi084 ceph-mon[32629]: osdmap e287: 8 total, 8 up, 8 in 2024-09-16T09:35:15.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:15 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/2084485479' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:35:15.668 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:15.669 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:15.669 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:35:15.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:15 smithi070 ceph-mon[30388]: pgmap v507: 178 pgs: 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 2 active+recovering+undersized+degraded+remapped, 3 active+remapped+backfill_toofull, 170 active+clean; 6.8 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 62 KiB/s rd, 0 B/s wr, 1.17k op/s; 14/333642 objects degraded (0.004%); 52549/333642 objects misplaced (15.750%); 27 MiB/s, 432 objects/s recovering 2024-09-16T09:35:15.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:15 smithi070 ceph-mon[30388]: osdmap e287: 8 total, 8 up, 8 in 2024-09-16T09:35:15.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:15 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2084485479' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:35:15.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:15 smithi070 ceph-mon[40691]: pgmap v507: 178 pgs: 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 2 active+recovering+undersized+degraded+remapped, 3 active+remapped+backfill_toofull, 170 active+clean; 6.8 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 62 KiB/s rd, 0 B/s wr, 1.17k op/s; 14/333642 objects degraded (0.004%); 52549/333642 objects misplaced (15.750%); 27 MiB/s, 432 objects/s recovering 2024-09-16T09:35:15.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:15 smithi070 ceph-mon[40691]: osdmap e287: 8 total, 8 up, 8 in 2024-09-16T09:35:15.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:15 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/2084485479' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:35:15.769 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:35:15.811 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:15.811 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:15.812 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:35:15.912 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:35:15.941 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:15.942 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:15.942 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:35:16.044 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:35:16.065 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:16.065 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:16.066 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:35:16.166 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:35:16.194 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:16.195 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:16.195 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:35:16.296 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:35:16.326 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:16.327 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:16.328 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:35:16.428 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:35:16.467 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:16.467 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:16.468 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:35:16.568 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:35:16.598 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:16.599 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:16.599 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:35:16.700 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:35:16.734 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:16.735 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:16.735 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:35:16.836 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:35:16.875 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:16.875 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:16.875 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:35:16.976 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:35:17.003 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:17.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:17.004 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:35:17.105 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:35:17.141 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:17.141 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:17.142 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:35:17.242 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:35:17.267 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:17.268 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:17.268 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:35:17.369 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:35:17.400 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:17.400 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:17.401 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:35:17.502 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:35:17.523 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:17.525 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:17.525 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:35:17.626 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:35:17.647 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:17 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:35:17.647 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:17 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:35:17.647 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:17 smithi084 ceph-mon[32629]: pgmap v509: 178 pgs: 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 2 active+recovering+undersized+degraded+remapped, 3 active+remapped+backfill_toofull, 170 active+clean; 6.8 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 62 KiB/s rd, 0 B/s wr, 669 op/s; 14/333642 objects degraded (0.004%); 52549/333642 objects misplaced (15.750%); 15 MiB/s, 240 objects/s recovering 2024-09-16T09:35:17.647 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:17 smithi084 ceph-mon[32629]: osdmap e288: 8 total, 8 up, 8 in 2024-09-16T09:35:17.647 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:17 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1469729653' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:35:17.647 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:17 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:35:17.659 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:17.660 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:17.660 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:35:17.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:17 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:35:17.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:17 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:35:17.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:17 smithi070 ceph-mon[30388]: pgmap v509: 178 pgs: 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 2 active+recovering+undersized+degraded+remapped, 3 active+remapped+backfill_toofull, 170 active+clean; 6.8 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 62 KiB/s rd, 0 B/s wr, 669 op/s; 14/333642 objects degraded (0.004%); 52549/333642 objects misplaced (15.750%); 15 MiB/s, 240 objects/s recovering 2024-09-16T09:35:17.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:17 smithi070 ceph-mon[30388]: osdmap e288: 8 total, 8 up, 8 in 2024-09-16T09:35:17.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:17 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1469729653' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:35:17.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:17 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:35:17.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:17 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:35:17.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:17 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:35:17.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:17 smithi070 ceph-mon[40691]: pgmap v509: 178 pgs: 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 2 active+recovering+undersized+degraded+remapped, 3 active+remapped+backfill_toofull, 170 active+clean; 6.8 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 62 KiB/s rd, 0 B/s wr, 669 op/s; 14/333642 objects degraded (0.004%); 52549/333642 objects misplaced (15.750%); 15 MiB/s, 240 objects/s recovering 2024-09-16T09:35:17.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:17 smithi070 ceph-mon[40691]: osdmap e288: 8 total, 8 up, 8 in 2024-09-16T09:35:17.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:17 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1469729653' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:35:17.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:17 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:35:17.692 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.6 dump_blocked_ops 2024-09-16T09:35:17.761 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:35:17.784 INFO:teuthology.orchestra.run.smithi070.stderr:nodeep-scrub is unset 2024-09-16T09:35:17.802 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:17.803 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:17.803 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:35:17.904 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:35:17.927 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:17.928 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:17.928 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:35:18.029 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:35:18.064 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:18.065 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:18.065 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:35:18.112 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:35:18.166 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:35:18.214 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:18.215 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:18.215 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:35:18.316 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:35:18.347 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:18.348 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:18.348 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:35:18.449 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:35:18.470 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:18.470 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:18.470 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:35:18.542 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:18 smithi084 ceph-mon[32629]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-16T09:35:18.542 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:18 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-16T09:35:18.542 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:18 smithi084 ceph-mon[32629]: osdmap e289: 8 total, 8 up, 8 in 2024-09-16T09:35:18.542 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:18 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:35:18.542 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:18 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:35:18.542 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:18 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:35:18.542 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:18 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:35:18.542 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:18 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:35:18.570 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T09:35:18.570 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T09:35:18.570 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-16T09:35:18.570 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd dump -f json-pretty 2024-09-16T09:35:18.571 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:35:18.609 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:18 smithi070 ceph-mon[30388]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-16T09:35:18.609 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:18 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-16T09:35:18.609 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:18 smithi070 ceph-mon[30388]: osdmap e289: 8 total, 8 up, 8 in 2024-09-16T09:35:18.609 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:18 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:35:18.609 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:18 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:35:18.610 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:18 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:35:18.610 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:18 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:35:18.610 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:18 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:35:18.610 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:18 smithi070 ceph-mon[40691]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-16T09:35:18.610 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:18 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-16T09:35:18.610 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:18 smithi070 ceph-mon[40691]: osdmap e289: 8 total, 8 up, 8 in 2024-09-16T09:35:18.610 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:18 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:35:18.610 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:18 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:35:18.610 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:18 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:35:18.610 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:18 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:35:18.610 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:18 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:35:18.617 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:18.618 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:18.618 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:35:18.685 INFO:teuthology.orchestra.run.smithi084.stdout:ERROR: (22) Invalid argument 2024-09-16T09:35:18.685 INFO:teuthology.orchestra.run.smithi084.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:35:18.719 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:35:18.762 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:18.763 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:18.763 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:35:18.864 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:35:18.911 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:18.911 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:18.912 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:35:19.013 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:35:19.045 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:19.045 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:19.046 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:35:19.147 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:35:19.198 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:19.199 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:19.199 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:35:19.300 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:35:19.325 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:19.325 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:19.325 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:35:19.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:19 smithi084 ceph-mon[32629]: pgmap v512: 178 pgs: 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 2 active+recovering+undersized+degraded+remapped, 3 active+remapped+backfill_toofull, 170 active+clean; 6.8 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 72 KiB/s rd, 0 B/s wr, 837 op/s; 14/332616 objects degraded (0.004%); 52549/332616 objects misplaced (15.799%); 18 MiB/s, 280 objects/s recovering 2024-09-16T09:35:19.425 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:19 smithi084 ceph-mon[32629]: Health check update: Degraded data redundancy: 14/332616 objects degraded (0.004%), 2 pgs degraded (PG_DEGRADED) 2024-09-16T09:35:19.425 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:19 smithi084 ceph-mon[32629]: 6.17 scrub starts 2024-09-16T09:35:19.425 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:19 smithi084 ceph-mon[32629]: 6.17 scrub ok 2024-09-16T09:35:19.426 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:19 smithi070 ceph-mon[30388]: pgmap v512: 178 pgs: 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 2 active+recovering+undersized+degraded+remapped, 3 active+remapped+backfill_toofull, 170 active+clean; 6.8 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 72 KiB/s rd, 0 B/s wr, 837 op/s; 14/332616 objects degraded (0.004%); 52549/332616 objects misplaced (15.799%); 18 MiB/s, 280 objects/s recovering 2024-09-16T09:35:19.426 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:19 smithi070 ceph-mon[30388]: Health check update: Degraded data redundancy: 14/332616 objects degraded (0.004%), 2 pgs degraded (PG_DEGRADED) 2024-09-16T09:35:19.427 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:19 smithi070 ceph-mon[30388]: 6.17 scrub starts 2024-09-16T09:35:19.427 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:19 smithi070 ceph-mon[30388]: 6.17 scrub ok 2024-09-16T09:35:19.427 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:19 smithi070 ceph-mon[40691]: pgmap v512: 178 pgs: 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 2 active+recovering+undersized+degraded+remapped, 3 active+remapped+backfill_toofull, 170 active+clean; 6.8 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 72 KiB/s rd, 0 B/s wr, 837 op/s; 14/332616 objects degraded (0.004%); 52549/332616 objects misplaced (15.799%); 18 MiB/s, 280 objects/s recovering 2024-09-16T09:35:19.427 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:19 smithi070 ceph-mon[40691]: Health check update: Degraded data redundancy: 14/332616 objects degraded (0.004%), 2 pgs degraded (PG_DEGRADED) 2024-09-16T09:35:19.427 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:19 smithi070 ceph-mon[40691]: 6.17 scrub starts 2024-09-16T09:35:19.427 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:19 smithi070 ceph-mon[40691]: 6.17 scrub ok 2024-09-16T09:35:19.428 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:35:19.469 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:19.470 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:19.470 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:35:19.502 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.6 dump_historic_ops 2024-09-16T09:35:19.571 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:35:19.620 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:19.620 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:19.621 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:35:19.721 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:35:19.758 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:19.759 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:19.760 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:35:19.838 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:35:19.860 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:35:19.905 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:19.905 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:19.905 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:35:20.006 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:35:20.058 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:20.059 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:20.059 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:35:20.160 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:35:20.197 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:20.197 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:20.197 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:35:20.298 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:35:20.320 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:20.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:20.321 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:35:20.422 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:35:20.450 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:20.451 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:20.451 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:35:20.551 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:35:20.588 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:20.589 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:20.589 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:35:20.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:20 smithi084 ceph-mon[32629]: 5.1 scrub starts 2024-09-16T09:35:20.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:20 smithi084 ceph-mon[32629]: 5.1 scrub ok 2024-09-16T09:35:20.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:20 smithi070 ceph-mon[30388]: 5.1 scrub starts 2024-09-16T09:35:20.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:20 smithi070 ceph-mon[30388]: 5.1 scrub ok 2024-09-16T09:35:20.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:20 smithi070 ceph-mon[40691]: 5.1 scrub starts 2024-09-16T09:35:20.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:20 smithi070 ceph-mon[40691]: 5.1 scrub ok 2024-09-16T09:35:20.690 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:35:20.712 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:20.713 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:20.713 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:35:20.814 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:35:20.847 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:20.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:20.848 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:35:20.852 INFO:teuthology.orchestra.run.smithi084.stdout:ERROR: (22) Invalid argument 2024-09-16T09:35:20.852 INFO:teuthology.orchestra.run.smithi084.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:35:20.949 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:35:20.992 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:20.993 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:20.993 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:35:21.094 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:35:21.128 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:21.129 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:21.129 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:35:21.230 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:35:21.257 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:21.257 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:21.258 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:35:21.359 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:35:21.388 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:21.389 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:21.389 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:35:21.490 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:35:21.512 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:21.512 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:21.513 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:35:21.614 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:35:21.652 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:21.653 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:21.653 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:35:21.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:21 smithi084 ceph-mon[32629]: pgmap v513: 178 pgs: 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 2 active+recovering+undersized+degraded+remapped, 3 active+remapped+backfill_toofull, 170 active+clean; 6.6 GiB data, 24 GiB used, 692 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 586 op/s; 31/323088 objects degraded (0.010%); 48966/323088 objects misplaced (15.156%); 13 MiB/s, 215 objects/s recovering 2024-09-16T09:35:21.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:21 smithi070 ceph-mon[30388]: pgmap v513: 178 pgs: 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 2 active+recovering+undersized+degraded+remapped, 3 active+remapped+backfill_toofull, 170 active+clean; 6.6 GiB data, 24 GiB used, 692 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 586 op/s; 31/323088 objects degraded (0.010%); 48966/323088 objects misplaced (15.156%); 13 MiB/s, 215 objects/s recovering 2024-09-16T09:35:21.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:21 smithi070 ceph-mon[40691]: pgmap v513: 178 pgs: 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 2 active+recovering+undersized+degraded+remapped, 3 active+remapped+backfill_toofull, 170 active+clean; 6.6 GiB data, 24 GiB used, 692 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 586 op/s; 31/323088 objects degraded (0.010%); 48966/323088 objects misplaced (15.156%); 13 MiB/s, 215 objects/s recovering 2024-09-16T09:35:21.756 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:35:21.791 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:21.792 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:21.792 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:35:21.858 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.7 dump_ops_in_flight 2024-09-16T09:35:21.893 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:35:21.922 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:21.923 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:21.923 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:35:22.024 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:35:22.062 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:22.063 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:22.063 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:35:22.164 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:35:22.187 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:35:22.201 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:22.202 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:22.203 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:35:22.304 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:35:22.335 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:22.336 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:22.337 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:35:22.421 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:35:21 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:35:21] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:35:22.437 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:35:22.462 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:22.463 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:22.464 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:35:22.564 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:35:22.606 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:22.606 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:22.606 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:35:22.707 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:35:22.739 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:22.740 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:22.740 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:35:22.822 INFO:teuthology.orchestra.run.smithi084.stdout:ERROR: (22) Invalid argument 2024-09-16T09:35:22.822 INFO:teuthology.orchestra.run.smithi084.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:35:22.841 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:35:22.865 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:22.866 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:22.866 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:35:22.967 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:35:23.009 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:23.009 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:23.009 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:35:23.110 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:35:23.132 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:23.133 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:23.133 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:35:23.234 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:35:23.263 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:23.263 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:23.264 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:35:23.365 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:35:23.387 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:23.387 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:23.388 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:35:23.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:23 smithi084 ceph-mon[32629]: pgmap v514: 178 pgs: 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 2 active+recovering+undersized+degraded+remapped, 3 active+remapped+backfill_toofull, 170 active+clean; 6.6 GiB data, 24 GiB used, 692 GiB / 715 GiB avail; 689 B/s rd, 0 B/s wr, 474 op/s; 31/323088 objects degraded (0.010%); 48966/323088 objects misplaced (15.156%); 11 MiB/s, 173 objects/s recovering 2024-09-16T09:35:23.410 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:35:23 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:35:23.489 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:35:23.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:23.513 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:23.513 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:35:23.614 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:35:23.638 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:23.639 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:23.639 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:35:23.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:23 smithi070 ceph-mon[30388]: pgmap v514: 178 pgs: 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 2 active+recovering+undersized+degraded+remapped, 3 active+remapped+backfill_toofull, 170 active+clean; 6.6 GiB data, 24 GiB used, 692 GiB / 715 GiB avail; 689 B/s rd, 0 B/s wr, 474 op/s; 31/323088 objects degraded (0.010%); 48966/323088 objects misplaced (15.156%); 11 MiB/s, 173 objects/s recovering 2024-09-16T09:35:23.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:23 smithi070 ceph-mon[40691]: pgmap v514: 178 pgs: 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 2 active+recovering+undersized+degraded+remapped, 3 active+remapped+backfill_toofull, 170 active+clean; 6.6 GiB data, 24 GiB used, 692 GiB / 715 GiB avail; 689 B/s rd, 0 B/s wr, 474 op/s; 31/323088 objects degraded (0.010%); 48966/323088 objects misplaced (15.156%); 11 MiB/s, 173 objects/s recovering 2024-09-16T09:35:23.740 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:35:23.784 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:23.785 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:23.785 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:35:23.886 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:35:23.909 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:23.910 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:23.910 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:35:24.011 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:35:24.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:24.053 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:24.053 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:35:24.154 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:35:24.177 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:24.178 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:24.178 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:35:24.279 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:35:24.329 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:24.329 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:24.329 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:35:24.430 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:35:24.452 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:24.452 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:24.452 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:35:24.553 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:35:24.594 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:24.595 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:24.595 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:35:24.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:24 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:35:24.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:24 smithi084 ceph-mon[32629]: 3.1a scrub starts 2024-09-16T09:35:24.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:24 smithi084 ceph-mon[32629]: 3.1a scrub ok 2024-09-16T09:35:24.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:24 smithi084 ceph-mon[32629]: Health check update: Degraded data redundancy: 31/323088 objects degraded (0.010%), 2 pgs degraded (PG_DEGRADED) 2024-09-16T09:35:24.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:24 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:35:24.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:24 smithi070 ceph-mon[30388]: 3.1a scrub starts 2024-09-16T09:35:24.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:24 smithi070 ceph-mon[30388]: 3.1a scrub ok 2024-09-16T09:35:24.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:24 smithi070 ceph-mon[30388]: Health check update: Degraded data redundancy: 31/323088 objects degraded (0.010%), 2 pgs degraded (PG_DEGRADED) 2024-09-16T09:35:24.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:24 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:35:24.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:24 smithi070 ceph-mon[40691]: 3.1a scrub starts 2024-09-16T09:35:24.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:24 smithi070 ceph-mon[40691]: 3.1a scrub ok 2024-09-16T09:35:24.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:24 smithi070 ceph-mon[40691]: Health check update: Degraded data redundancy: 31/323088 objects degraded (0.010%), 2 pgs degraded (PG_DEGRADED) 2024-09-16T09:35:24.696 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:35:24.733 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:24.734 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:24.734 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:35:24.835 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:35:24.861 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:24.861 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:24.861 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:35:24.962 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:35:24.997 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:24.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:24.998 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:35:25.098 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:35:25.127 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:25.128 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:25.128 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:35:25.229 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:35:25.263 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:25.264 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:25.264 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:35:25.365 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:35:25.416 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:25.417 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:25.417 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:35:25.518 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:35:25.547 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:25.548 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:25.548 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:35:25.649 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:35:25.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:25 smithi084 ceph-mon[32629]: pgmap v515: 178 pgs: 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 2 active+recovering+undersized+degraded+remapped, 3 active+remapped+backfill_toofull, 170 active+clean; 6.4 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 795 op/s; 31/314577 objects degraded (0.010%); 47219/314577 objects misplaced (15.010%); 14 MiB/s, 225 objects/s recovering 2024-09-16T09:35:25.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:25 smithi084 ceph-mon[32629]: 3.14 scrub starts 2024-09-16T09:35:25.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:25 smithi084 ceph-mon[32629]: 3.14 scrub ok 2024-09-16T09:35:25.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:25 smithi070 ceph-mon[30388]: pgmap v515: 178 pgs: 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 2 active+recovering+undersized+degraded+remapped, 3 active+remapped+backfill_toofull, 170 active+clean; 6.4 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 795 op/s; 31/314577 objects degraded (0.010%); 47219/314577 objects misplaced (15.010%); 14 MiB/s, 225 objects/s recovering 2024-09-16T09:35:25.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:25 smithi070 ceph-mon[30388]: 3.14 scrub starts 2024-09-16T09:35:25.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:25 smithi070 ceph-mon[30388]: 3.14 scrub ok 2024-09-16T09:35:25.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:25 smithi070 ceph-mon[40691]: pgmap v515: 178 pgs: 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 2 active+recovering+undersized+degraded+remapped, 3 active+remapped+backfill_toofull, 170 active+clean; 6.4 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 795 op/s; 31/314577 objects degraded (0.010%); 47219/314577 objects misplaced (15.010%); 14 MiB/s, 225 objects/s recovering 2024-09-16T09:35:25.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:25 smithi070 ceph-mon[40691]: 3.14 scrub starts 2024-09-16T09:35:25.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:25 smithi070 ceph-mon[40691]: 3.14 scrub ok 2024-09-16T09:35:25.684 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:25.685 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:25.685 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:35:25.786 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:35:25.816 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:25.817 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:25.817 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:35:25.918 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:35:25.958 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:25.958 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:25.959 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:35:26.060 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:35:26.093 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:26.093 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:26.093 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:35:26.151 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.7 dump_blocked_ops 2024-09-16T09:35:26.195 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:35:26.223 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:26.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:26.225 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:35:26.326 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:35:26.364 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:26.365 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:26.365 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:35:26.466 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:35:26.504 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:26.505 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:26.505 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:35:26.520 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:35:26.606 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:35:26.639 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:26.640 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:26.640 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:35:26.741 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:35:26.781 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:26.782 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:26.783 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:35:26.883 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:35:26.908 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:26.908 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:26.909 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:35:27.009 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:35:27.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:27.051 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:27.051 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:35:27.152 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:35:27.174 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:27.175 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:27.175 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:35:27.242 INFO:teuthology.orchestra.run.smithi084.stdout:ERROR: (22) Invalid argument 2024-09-16T09:35:27.242 INFO:teuthology.orchestra.run.smithi084.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:35:27.276 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:35:27.314 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:27.314 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:27.315 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:35:27.415 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:35:27.450 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:27.451 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:27.451 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:35:27.552 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:35:27.579 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:27.579 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:27.580 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:35:27.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:27 smithi084 ceph-mon[32629]: pgmap v516: 178 pgs: 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 2 active+recovering+undersized+degraded+remapped, 3 active+remapped+backfill_toofull, 170 active+clean; 6.3 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 773 op/s; 49/311820 objects degraded (0.016%); 45673/311820 objects misplaced (14.647%); 16 MiB/s, 257 objects/s recovering 2024-09-16T09:35:27.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:27 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:35:27.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:27 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:35:27.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:27 smithi070 ceph-mon[30388]: pgmap v516: 178 pgs: 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 2 active+recovering+undersized+degraded+remapped, 3 active+remapped+backfill_toofull, 170 active+clean; 6.3 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 773 op/s; 49/311820 objects degraded (0.016%); 45673/311820 objects misplaced (14.647%); 16 MiB/s, 257 objects/s recovering 2024-09-16T09:35:27.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:27 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:35:27.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:27 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:35:27.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:27 smithi070 ceph-mon[40691]: pgmap v516: 178 pgs: 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 2 active+recovering+undersized+degraded+remapped, 3 active+remapped+backfill_toofull, 170 active+clean; 6.3 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 773 op/s; 49/311820 objects degraded (0.016%); 45673/311820 objects misplaced (14.647%); 16 MiB/s, 257 objects/s recovering 2024-09-16T09:35:27.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:27 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:35:27.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:27 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:35:27.681 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:35:27.709 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:27.710 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:27.710 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:35:27.811 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:35:27.843 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:27.844 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:27.845 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:35:27.945 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:35:27.989 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:27.990 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:27.990 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:35:28.091 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:35:28.127 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:28.128 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:28.129 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:35:28.229 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:35:28.267 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:28.267 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:28.268 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:35:28.369 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:35:28.373 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.7 dump_historic_ops 2024-09-16T09:35:28.396 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:28.397 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:28.397 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:35:28.498 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:35:28.538 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:28.539 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:28.539 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:35:28.640 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:35:28.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:28 smithi084 ceph-mon[32629]: osdmap e290: 8 total, 8 up, 8 in 2024-09-16T09:35:28.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:28 smithi084 ceph-mon[32629]: 4.e scrub starts 2024-09-16T09:35:28.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:28 smithi084 ceph-mon[32629]: 4.e scrub ok 2024-09-16T09:35:28.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:28 smithi070 ceph-mon[30388]: osdmap e290: 8 total, 8 up, 8 in 2024-09-16T09:35:28.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:28 smithi070 ceph-mon[30388]: 4.e scrub starts 2024-09-16T09:35:28.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:28 smithi070 ceph-mon[30388]: 4.e scrub ok 2024-09-16T09:35:28.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:28 smithi070 ceph-mon[40691]: osdmap e290: 8 total, 8 up, 8 in 2024-09-16T09:35:28.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:28 smithi070 ceph-mon[40691]: 4.e scrub starts 2024-09-16T09:35:28.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:28 smithi070 ceph-mon[40691]: 4.e scrub ok 2024-09-16T09:35:28.687 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:28.688 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:28.688 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:35:28.748 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:35:28.789 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:35:28.819 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:28.819 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:28.820 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:35:28.920 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:35:28.950 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:28.950 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:28.951 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:35:29.051 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:35:29.085 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:29.085 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:29.086 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:35:29.187 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:35:29.219 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:29.219 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:29.219 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:35:29.320 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:35:29.341 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:29.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:29.342 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:35:29.443 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:35:29.475 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:29.476 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:29.476 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:35:29.577 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:35:29.618 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:29.619 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:29.619 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:35:29.628 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:29 smithi084 ceph-mon[32629]: pgmap v518: 178 pgs: 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 2 active+recovering+undersized+degraded+remapped, 3 active+remapped+backfill_toofull, 170 active+clean; 6.3 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 730 op/s; 49/310731 objects degraded (0.016%); 45673/310731 objects misplaced (14.699%); 15 MiB/s, 242 objects/s recovering 2024-09-16T09:35:29.628 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:29 smithi084 ceph-mon[32629]: 4.d scrub starts 2024-09-16T09:35:29.628 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:29 smithi084 ceph-mon[32629]: 4.d scrub ok 2024-09-16T09:35:29.628 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:29 smithi084 ceph-mon[32629]: Health check update: Degraded data redundancy: 49/310731 objects degraded (0.016%), 2 pgs degraded (PG_DEGRADED) 2024-09-16T09:35:29.628 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:29 smithi084 ceph-mon[32629]: osdmap e291: 8 total, 8 up, 8 in 2024-09-16T09:35:29.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:29 smithi070 ceph-mon[30388]: pgmap v518: 178 pgs: 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 2 active+recovering+undersized+degraded+remapped, 3 active+remapped+backfill_toofull, 170 active+clean; 6.3 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 730 op/s; 49/310731 objects degraded (0.016%); 45673/310731 objects misplaced (14.699%); 15 MiB/s, 242 objects/s recovering 2024-09-16T09:35:29.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:29 smithi070 ceph-mon[30388]: 4.d scrub starts 2024-09-16T09:35:29.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:29 smithi070 ceph-mon[30388]: 4.d scrub ok 2024-09-16T09:35:29.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:29 smithi070 ceph-mon[30388]: Health check update: Degraded data redundancy: 49/310731 objects degraded (0.016%), 2 pgs degraded (PG_DEGRADED) 2024-09-16T09:35:29.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:29 smithi070 ceph-mon[30388]: osdmap e291: 8 total, 8 up, 8 in 2024-09-16T09:35:29.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:29 smithi070 ceph-mon[40691]: pgmap v518: 178 pgs: 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 2 active+recovering+undersized+degraded+remapped, 3 active+remapped+backfill_toofull, 170 active+clean; 6.3 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 730 op/s; 49/310731 objects degraded (0.016%); 45673/310731 objects misplaced (14.699%); 15 MiB/s, 242 objects/s recovering 2024-09-16T09:35:29.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:29 smithi070 ceph-mon[40691]: 4.d scrub starts 2024-09-16T09:35:29.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:29 smithi070 ceph-mon[40691]: 4.d scrub ok 2024-09-16T09:35:29.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:29 smithi070 ceph-mon[40691]: Health check update: Degraded data redundancy: 49/310731 objects degraded (0.016%), 2 pgs degraded (PG_DEGRADED) 2024-09-16T09:35:29.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:29 smithi070 ceph-mon[40691]: osdmap e291: 8 total, 8 up, 8 in 2024-09-16T09:35:29.720 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:35:29.755 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:29.756 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:29.756 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:35:29.843 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:35:29.857 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:35:29.881 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:29.882 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:29.882 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:35:29.983 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:35:30.027 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:30.028 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:30.028 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:35:30.129 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:35:30.151 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:30.151 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:30.152 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:35:30.252 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:35:30.292 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:30.293 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:30.293 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:35:30.394 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:35:30.423 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:30.423 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:30.424 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:35:30.524 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:35:30.541 INFO:teuthology.orchestra.run.smithi084.stdout:ERROR: (22) Invalid argument 2024-09-16T09:35:30.541 INFO:teuthology.orchestra.run.smithi084.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:35:30.556 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:30.556 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:30.556 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:35:30.657 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:35:30.688 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:30.689 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:30.689 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:35:30.790 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:35:30.821 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:30.822 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:30.822 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:35:30.923 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:35:30.948 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:30.949 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:30.949 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:35:31.050 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:35:31.092 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:31.092 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:31.093 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:35:31.133 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T09:35:31.194 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:35:31.216 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:31.216 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:31.217 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:35:31.318 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:35:31.351 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:31.352 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:31.352 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:35:31.453 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:35:31.477 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:31.477 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:31.478 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:35:31.579 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:35:31.600 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:31.601 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:31.602 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:35:31.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:31 smithi084 ceph-mon[32629]: pgmap v520: 178 pgs: 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 2 active+recovering+undersized+degraded+remapped, 3 active+remapped+backfill_toofull, 170 active+clean; 6.1 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 983 op/s; 80/299511 objects degraded (0.027%); 42115/299511 objects misplaced (14.061%); 18 MiB/s, 294 objects/s recovering 2024-09-16T09:35:31.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:31 smithi070 ceph-mon[30388]: pgmap v520: 178 pgs: 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 2 active+recovering+undersized+degraded+remapped, 3 active+remapped+backfill_toofull, 170 active+clean; 6.1 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 983 op/s; 80/299511 objects degraded (0.027%); 42115/299511 objects misplaced (14.061%); 18 MiB/s, 294 objects/s recovering 2024-09-16T09:35:31.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:31 smithi070 ceph-mon[40691]: pgmap v520: 178 pgs: 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 2 active+recovering+undersized+degraded+remapped, 3 active+remapped+backfill_toofull, 170 active+clean; 6.1 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 983 op/s; 80/299511 objects degraded (0.027%); 42115/299511 objects misplaced (14.061%); 18 MiB/s, 294 objects/s recovering 2024-09-16T09:35:31.702 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:35:31.723 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:31.724 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:31.724 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:35:31.820 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd set noscrub 2024-09-16T09:35:31.825 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:35:31.922 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:31.922 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:31.923 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:35:32.023 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:35:32.045 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:32.045 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:32.046 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:35:32.125 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:35:31 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:35:31] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:35:32.147 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:35:32.177 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:32.177 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:32.178 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:35:32.278 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:35:32.301 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:32.302 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:32.302 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:35:32.403 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:35:32.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:32 smithi070 ceph-mon[30388]: 4.9 scrub starts 2024-09-16T09:35:32.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:32 smithi070 ceph-mon[40691]: 4.9 scrub starts 2024-09-16T09:35:32.425 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:32.425 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:32.426 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:35:32.448 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.0 dump_ops_in_flight 2024-09-16T09:35:32.527 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:35:32.583 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:32.584 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:32.584 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:35:32.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:32 smithi084 ceph-mon[32629]: 4.9 scrub starts 2024-09-16T09:35:32.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:32 smithi084 ceph-mon[32629]: 4.9 scrub ok 2024-09-16T09:35:32.685 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:35:32.712 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:32.713 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:32.713 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:35:32.814 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:35:32.837 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:32.837 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:32.838 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:35:32.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:32 smithi070 ceph-mon[30388]: 4.9 scrub ok 2024-09-16T09:35:32.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:32 smithi070 ceph-mon[40691]: 4.9 scrub ok 2024-09-16T09:35:32.938 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:35:32.961 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:32.961 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:32.961 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:35:33.064 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:35:33.130 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:33.130 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:33.130 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:35:33.231 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:35:33.253 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:33.254 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:33.254 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:35:33.355 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:35:33.397 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:33.398 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:33.398 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:35:33.410 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:35:33 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:35:33.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:33 smithi084 ceph-mon[32629]: pgmap v521: 178 pgs: 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 2 active+recovering+undersized+degraded+remapped, 3 active+remapped+backfill_toofull, 170 active+clean; 6.1 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 628 op/s; 80/299511 objects degraded (0.027%); 42115/299511 objects misplaced (14.061%); 14 MiB/s, 230 objects/s recovering 2024-09-16T09:35:33.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:33 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:35:33.499 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:35:33.525 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:33.526 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:33.526 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:35:33.627 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:35:33.660 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:33.661 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:33.662 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:35:33.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:33 smithi070 ceph-mon[30388]: pgmap v521: 178 pgs: 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 2 active+recovering+undersized+degraded+remapped, 3 active+remapped+backfill_toofull, 170 active+clean; 6.1 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 628 op/s; 80/299511 objects degraded (0.027%); 42115/299511 objects misplaced (14.061%); 14 MiB/s, 230 objects/s recovering 2024-09-16T09:35:33.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:33 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:35:33.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:33 smithi070 ceph-mon[40691]: pgmap v521: 178 pgs: 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 2 active+recovering+undersized+degraded+remapped, 3 active+remapped+backfill_toofull, 170 active+clean; 6.1 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 628 op/s; 80/299511 objects degraded (0.027%); 42115/299511 objects misplaced (14.061%); 14 MiB/s, 230 objects/s recovering 2024-09-16T09:35:33.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:33 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:35:33.762 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:35:33.792 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:33.793 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:33.793 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:35:33.894 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:35:33.947 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:33.947 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:33.948 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:35:34.048 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:35:34.077 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:34.077 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:34.078 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:35:34.178 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:35:34.215 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:34.215 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:34.216 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:35:34.317 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:35:34.348 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:34.349 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:34.350 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:35:34.451 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:35:34.484 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:34.484 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:34.485 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:35:34.487 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:35:34.487 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T09:35:34.487 INFO:teuthology.orchestra.run.smithi070.stdout: "epoch": 291, 2024-09-16T09:35:34.487 INFO:teuthology.orchestra.run.smithi070.stdout: "fsid": "a1321834-740c-11ef-bceb-c7b262605968", 2024-09-16T09:35:34.487 INFO:teuthology.orchestra.run.smithi070.stdout: "created": "2024-09-16T09:19:14.121498+0000", 2024-09-16T09:35:34.487 INFO:teuthology.orchestra.run.smithi070.stdout: "modified": "2024-09-16T09:35:29.275073+0000", 2024-09-16T09:35:34.487 INFO:teuthology.orchestra.run.smithi070.stdout: "last_up_change": "2024-09-16T09:24:51.124496+0000", 2024-09-16T09:35:34.488 INFO:teuthology.orchestra.run.smithi070.stdout: "last_in_change": "2024-09-16T09:24:28.432920+0000", 2024-09-16T09:35:34.488 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-16T09:35:34.488 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_num": 5799936, 2024-09-16T09:35:34.488 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_set": [ 2024-09-16T09:35:34.488 INFO:teuthology.orchestra.run.smithi070.stdout: "pglog_hardlimit", 2024-09-16T09:35:34.488 INFO:teuthology.orchestra.run.smithi070.stdout: "purged_snapdirs", 2024-09-16T09:35:34.488 INFO:teuthology.orchestra.run.smithi070.stdout: "recovery_deletes", 2024-09-16T09:35:34.488 INFO:teuthology.orchestra.run.smithi070.stdout: "sortbitwise" 2024-09-16T09:35:34.488 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-16T09:35:34.489 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_version": 18, 2024-09-16T09:35:34.489 INFO:teuthology.orchestra.run.smithi070.stdout: "full_ratio": 0.94999998807907104, 2024-09-16T09:35:34.489 INFO:teuthology.orchestra.run.smithi070.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-16T09:35:34.489 INFO:teuthology.orchestra.run.smithi070.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-16T09:35:34.489 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_snapshot": "", 2024-09-16T09:35:34.489 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_max": 9, 2024-09-16T09:35:34.489 INFO:teuthology.orchestra.run.smithi070.stdout: "max_osd": 8, 2024-09-16T09:35:34.489 INFO:teuthology.orchestra.run.smithi070.stdout: "require_min_compat_client": "luminous", 2024-09-16T09:35:34.489 INFO:teuthology.orchestra.run.smithi070.stdout: "min_compat_client": "luminous", 2024-09-16T09:35:34.489 INFO:teuthology.orchestra.run.smithi070.stdout: "require_osd_release": "squid", 2024-09-16T09:35:34.489 INFO:teuthology.orchestra.run.smithi070.stdout: "allow_crimson": false, 2024-09-16T09:35:34.490 INFO:teuthology.orchestra.run.smithi070.stdout: "pools": [ 2024-09-16T09:35:34.490 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.490 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 1, 2024-09-16T09:35:34.490 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": ".mgr", 2024-09-16T09:35:34.490 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2024-09-16T09:22:27.617694+0000", 2024-09-16T09:35:34.490 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2024-09-16T09:35:34.490 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2024-09-16T09:35:34.490 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2024-09-16T09:35:34.490 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 3, 2024-09-16T09:35:34.490 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 2, 2024-09-16T09:35:34.490 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2024-09-16T09:35:34.491 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2024-09-16T09:35:34.491 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2024-09-16T09:35:34.491 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T09:35:34.491 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T09:35:34.491 INFO:teuthology.orchestra.run.smithi070.stdout: "is_stretch_pool": false, 2024-09-16T09:35:34.491 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2024-09-16T09:35:34.491 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2024-09-16T09:35:34.491 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 21, 2024-09-16T09:35:34.491 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 21, 2024-09-16T09:35:34.491 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 21, 2024-09-16T09:35:34.492 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 21, 2024-09-16T09:35:34.492 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 21, 2024-09-16T09:35:34.492 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2024-09-16T09:35:34.492 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2024-09-16T09:35:34.492 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2024-09-16T09:35:34.492 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2024-09-16T09:35:34.492 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2024-09-16T09:35:34.492 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2024-09-16T09:35:34.492 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2024-09-16T09:35:34.492 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.493 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "162", 2024-09-16T09:35:34.496 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2024-09-16T09:35:34.497 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T09:35:34.497 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "160", 2024-09-16T09:35:34.497 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2024-09-16T09:35:34.497 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2024-09-16T09:35:34.497 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2024-09-16T09:35:34.497 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2024-09-16T09:35:34.497 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2024-09-16T09:35:34.497 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2024-09-16T09:35:34.497 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2024-09-16T09:35:34.497 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2024-09-16T09:35:34.497 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2024-09-16T09:35:34.498 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2024-09-16T09:35:34.498 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2024-09-16T09:35:34.498 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2024-09-16T09:35:34.498 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2024-09-16T09:35:34.498 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2024-09-16T09:35:34.498 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2024-09-16T09:35:34.498 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T09:35:34.498 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T09:35:34.498 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T09:35:34.498 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2024-09-16T09:35:34.498 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2024-09-16T09:35:34.498 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2024-09-16T09:35:34.499 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2024-09-16T09:35:34.499 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2024-09-16T09:35:34.499 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.499 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2024-09-16T09:35:34.499 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2024-09-16T09:35:34.499 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2024-09-16T09:35:34.499 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2024-09-16T09:35:34.499 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2024-09-16T09:35:34.499 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T09:35:34.499 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2024-09-16T09:35:34.499 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2024-09-16T09:35:34.500 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2024-09-16T09:35:34.500 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2024-09-16T09:35:34.500 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2024-09-16T09:35:34.500 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2024-09-16T09:35:34.500 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_max": 32, 2024-09-16T09:35:34.500 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_min": 1 2024-09-16T09:35:34.500 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.500 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2024-09-16T09:35:34.500 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": {} 2024-09-16T09:35:34.500 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.501 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2024-09-16T09:35:34.501 INFO:teuthology.orchestra.run.smithi070.stdout: "score_type": "Fair distribution", 2024-09-16T09:35:34.501 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 1.6699999570846558, 2024-09-16T09:35:34.501 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 1.6699999570846558, 2024-09-16T09:35:34.501 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1.1399999856948853, 2024-09-16T09:35:34.501 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 1.8999999761581421, 2024-09-16T09:35:34.501 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 1.8999999761581421, 2024-09-16T09:35:34.501 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 0.70719337463378906, 2024-09-16T09:35:34.501 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 0.81000000238418579, 2024-09-16T09:35:34.501 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 0.70999997854232788 2024-09-16T09:35:34.501 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:35:34.501 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.502 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.502 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 2, 2024-09-16T09:35:34.502 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "datapool", 2024-09-16T09:35:34.502 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2024-09-16T09:24:59.169244+0000", 2024-09-16T09:35:34.502 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 8193, 2024-09-16T09:35:34.502 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-16T09:35:34.502 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2024-09-16T09:35:34.502 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 3, 2024-09-16T09:35:34.502 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 2, 2024-09-16T09:35:34.502 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2024-09-16T09:35:34.503 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2024-09-16T09:35:34.503 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2024-09-16T09:35:34.503 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T09:35:34.503 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T09:35:34.503 INFO:teuthology.orchestra.run.smithi070.stdout: "is_stretch_pool": false, 2024-09-16T09:35:34.503 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2024-09-16T09:35:34.503 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2024-09-16T09:35:34.503 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 3, 2024-09-16T09:35:34.503 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 3, 2024-09-16T09:35:34.503 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 3, 2024-09-16T09:35:34.503 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 3, 2024-09-16T09:35:34.504 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 3, 2024-09-16T09:35:34.504 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2024-09-16T09:35:34.504 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2024-09-16T09:35:34.504 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2024-09-16T09:35:34.504 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2024-09-16T09:35:34.504 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2024-09-16T09:35:34.504 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2024-09-16T09:35:34.504 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2024-09-16T09:35:34.504 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.504 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "61", 2024-09-16T09:35:34.504 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2024-09-16T09:35:34.505 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T09:35:34.505 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T09:35:34.505 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2024-09-16T09:35:34.505 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2024-09-16T09:35:34.505 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 2, 2024-09-16T09:35:34.505 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 61, 2024-09-16T09:35:34.505 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2024-09-16T09:35:34.505 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2024-09-16T09:35:34.505 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2024-09-16T09:35:34.505 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2024-09-16T09:35:34.505 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2024-09-16T09:35:34.505 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2024-09-16T09:35:34.506 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2024-09-16T09:35:34.506 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2024-09-16T09:35:34.506 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2024-09-16T09:35:34.506 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2024-09-16T09:35:34.506 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2024-09-16T09:35:34.506 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T09:35:34.506 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T09:35:34.506 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T09:35:34.506 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2024-09-16T09:35:34.506 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2024-09-16T09:35:34.506 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2024-09-16T09:35:34.506 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2024-09-16T09:35:34.507 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2024-09-16T09:35:34.507 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.507 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2024-09-16T09:35:34.507 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2024-09-16T09:35:34.507 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2024-09-16T09:35:34.507 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2024-09-16T09:35:34.507 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2024-09-16T09:35:34.507 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T09:35:34.507 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2024-09-16T09:35:34.507 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2024-09-16T09:35:34.507 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2024-09-16T09:35:34.508 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2024-09-16T09:35:34.508 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2024-09-16T09:35:34.508 INFO:teuthology.orchestra.run.smithi070.stdout: "options": {}, 2024-09-16T09:35:34.508 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2024-09-16T09:35:34.508 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd": {} 2024-09-16T09:35:34.508 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.508 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2024-09-16T09:35:34.508 INFO:teuthology.orchestra.run.smithi070.stdout: "score_type": "Fair distribution", 2024-09-16T09:35:34.508 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 2.3299999237060547, 2024-09-16T09:35:34.508 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 2.3299999237060547, 2024-09-16T09:35:34.508 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1, 2024-09-16T09:35:34.508 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 2.3299999237060547, 2024-09-16T09:35:34.509 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 2.3299999237060547, 2024-09-16T09:35:34.509 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 0.70719337463378906, 2024-09-16T09:35:34.509 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 0.81000000238418579, 2024-09-16T09:35:34.509 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 0.70999997854232788 2024-09-16T09:35:34.509 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:35:34.509 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.509 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.509 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 3, 2024-09-16T09:35:34.509 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": ".rgw.root", 2024-09-16T09:35:34.509 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2024-09-16T09:25:00.056119+0000", 2024-09-16T09:35:34.509 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2024-09-16T09:35:34.509 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2024-09-16T09:35:34.510 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2024-09-16T09:35:34.510 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 3, 2024-09-16T09:35:34.510 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 2, 2024-09-16T09:35:34.510 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2024-09-16T09:35:34.510 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2024-09-16T09:35:34.510 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2024-09-16T09:35:34.510 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T09:35:34.510 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T09:35:34.510 INFO:teuthology.orchestra.run.smithi070.stdout: "is_stretch_pool": false, 2024-09-16T09:35:34.510 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2024-09-16T09:35:34.510 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2024-09-16T09:35:34.510 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 32, 2024-09-16T09:35:34.511 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 32, 2024-09-16T09:35:34.511 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 32, 2024-09-16T09:35:34.511 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 32, 2024-09-16T09:35:34.511 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 32, 2024-09-16T09:35:34.511 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2024-09-16T09:35:34.511 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2024-09-16T09:35:34.511 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2024-09-16T09:35:34.511 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2024-09-16T09:35:34.511 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2024-09-16T09:35:34.511 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2024-09-16T09:35:34.511 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2024-09-16T09:35:34.512 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.512 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "60", 2024-09-16T09:35:34.512 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2024-09-16T09:35:34.512 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T09:35:34.512 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T09:35:34.512 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2024-09-16T09:35:34.512 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2024-09-16T09:35:34.512 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2024-09-16T09:35:34.512 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2024-09-16T09:35:34.512 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2024-09-16T09:35:34.513 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2024-09-16T09:35:34.513 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2024-09-16T09:35:34.513 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2024-09-16T09:35:34.513 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2024-09-16T09:35:34.513 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2024-09-16T09:35:34.513 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2024-09-16T09:35:34.513 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2024-09-16T09:35:34.513 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2024-09-16T09:35:34.513 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2024-09-16T09:35:34.513 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2024-09-16T09:35:34.513 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T09:35:34.513 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T09:35:34.514 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T09:35:34.514 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2024-09-16T09:35:34.514 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2024-09-16T09:35:34.514 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2024-09-16T09:35:34.514 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2024-09-16T09:35:34.514 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2024-09-16T09:35:34.514 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.514 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2024-09-16T09:35:34.514 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2024-09-16T09:35:34.514 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2024-09-16T09:35:34.514 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2024-09-16T09:35:34.515 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2024-09-16T09:35:34.515 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T09:35:34.515 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2024-09-16T09:35:34.515 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2024-09-16T09:35:34.515 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2024-09-16T09:35:34.515 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2024-09-16T09:35:34.515 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2024-09-16T09:35:34.515 INFO:teuthology.orchestra.run.smithi070.stdout: "options": {}, 2024-09-16T09:35:34.515 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2024-09-16T09:35:34.515 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw": {} 2024-09-16T09:35:34.515 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.515 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2024-09-16T09:35:34.516 INFO:teuthology.orchestra.run.smithi070.stdout: "score_type": "Fair distribution", 2024-09-16T09:35:34.516 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 1.75, 2024-09-16T09:35:34.516 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 1.75, 2024-09-16T09:35:34.516 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1.1399999856948853, 2024-09-16T09:35:34.516 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 2, 2024-09-16T09:35:34.516 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 2, 2024-09-16T09:35:34.516 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 0.70719337463378906, 2024-09-16T09:35:34.516 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 0.81000000238418579, 2024-09-16T09:35:34.516 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 0.70999997854232788 2024-09-16T09:35:34.516 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:35:34.517 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.517 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.517 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 4, 2024-09-16T09:35:34.517 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "default.rgw.log", 2024-09-16T09:35:34.517 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2024-09-16T09:25:01.770016+0000", 2024-09-16T09:35:34.517 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2024-09-16T09:35:34.517 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2024-09-16T09:35:34.517 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2024-09-16T09:35:34.517 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 3, 2024-09-16T09:35:34.517 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 2, 2024-09-16T09:35:34.517 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2024-09-16T09:35:34.518 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2024-09-16T09:35:34.518 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2024-09-16T09:35:34.518 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T09:35:34.518 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T09:35:34.518 INFO:teuthology.orchestra.run.smithi070.stdout: "is_stretch_pool": false, 2024-09-16T09:35:34.518 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2024-09-16T09:35:34.518 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2024-09-16T09:35:34.518 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 32, 2024-09-16T09:35:34.518 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 32, 2024-09-16T09:35:34.518 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 32, 2024-09-16T09:35:34.518 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 32, 2024-09-16T09:35:34.519 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 32, 2024-09-16T09:35:34.519 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2024-09-16T09:35:34.519 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2024-09-16T09:35:34.519 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2024-09-16T09:35:34.519 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2024-09-16T09:35:34.519 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2024-09-16T09:35:34.519 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2024-09-16T09:35:34.519 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2024-09-16T09:35:34.519 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.519 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "62", 2024-09-16T09:35:34.519 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2024-09-16T09:35:34.519 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T09:35:34.520 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T09:35:34.520 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2024-09-16T09:35:34.520 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2024-09-16T09:35:34.520 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2024-09-16T09:35:34.520 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2024-09-16T09:35:34.520 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2024-09-16T09:35:34.520 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2024-09-16T09:35:34.520 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2024-09-16T09:35:34.520 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2024-09-16T09:35:34.520 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2024-09-16T09:35:34.520 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2024-09-16T09:35:34.520 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2024-09-16T09:35:34.521 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2024-09-16T09:35:34.521 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2024-09-16T09:35:34.521 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2024-09-16T09:35:34.521 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2024-09-16T09:35:34.521 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T09:35:34.521 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T09:35:34.521 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T09:35:34.521 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2024-09-16T09:35:34.521 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2024-09-16T09:35:34.521 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2024-09-16T09:35:34.521 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2024-09-16T09:35:34.522 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2024-09-16T09:35:34.522 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.522 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2024-09-16T09:35:34.522 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2024-09-16T09:35:34.522 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2024-09-16T09:35:34.522 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2024-09-16T09:35:34.522 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2024-09-16T09:35:34.522 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T09:35:34.522 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2024-09-16T09:35:34.522 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2024-09-16T09:35:34.522 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2024-09-16T09:35:34.523 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2024-09-16T09:35:34.523 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2024-09-16T09:35:34.523 INFO:teuthology.orchestra.run.smithi070.stdout: "options": {}, 2024-09-16T09:35:34.523 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2024-09-16T09:35:34.523 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw": {} 2024-09-16T09:35:34.523 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.523 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2024-09-16T09:35:34.523 INFO:teuthology.orchestra.run.smithi070.stdout: "score_type": "Fair distribution", 2024-09-16T09:35:34.523 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 6.8499999046325684, 2024-09-16T09:35:34.523 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 2.190000057220459, 2024-09-16T09:35:34.524 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1.1399999856948853, 2024-09-16T09:35:34.524 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 7.809999942779541, 2024-09-16T09:35:34.524 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 2.5, 2024-09-16T09:35:34.524 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 0.70719337463378906, 2024-09-16T09:35:34.524 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 0.81000000238418579, 2024-09-16T09:35:34.524 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 0.70999997854232788, 2024-09-16T09:35:34.524 INFO:teuthology.orchestra.run.smithi070.stdout: "error_message": "pool 4 has acting primaries on OSD(s) with primary affinity 0, read balance score is not accurate" 2024-09-16T09:35:34.524 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:35:34.524 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.524 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.524 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 5, 2024-09-16T09:35:34.525 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "default.rgw.control", 2024-09-16T09:35:34.525 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2024-09-16T09:25:03.725555+0000", 2024-09-16T09:35:34.525 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2024-09-16T09:35:34.525 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2024-09-16T09:35:34.525 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2024-09-16T09:35:34.525 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 3, 2024-09-16T09:35:34.525 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 2, 2024-09-16T09:35:34.525 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2024-09-16T09:35:34.525 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2024-09-16T09:35:34.525 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2024-09-16T09:35:34.525 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T09:35:34.526 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T09:35:34.526 INFO:teuthology.orchestra.run.smithi070.stdout: "is_stretch_pool": false, 2024-09-16T09:35:34.526 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2024-09-16T09:35:34.526 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2024-09-16T09:35:34.526 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 32, 2024-09-16T09:35:34.526 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 32, 2024-09-16T09:35:34.526 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 32, 2024-09-16T09:35:34.526 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 32, 2024-09-16T09:35:34.526 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 32, 2024-09-16T09:35:34.526 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2024-09-16T09:35:34.526 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2024-09-16T09:35:34.527 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2024-09-16T09:35:34.527 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2024-09-16T09:35:34.527 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2024-09-16T09:35:34.527 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2024-09-16T09:35:34.527 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2024-09-16T09:35:34.527 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.527 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "64", 2024-09-16T09:35:34.527 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2024-09-16T09:35:34.527 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T09:35:34.527 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T09:35:34.527 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2024-09-16T09:35:34.527 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2024-09-16T09:35:34.528 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2024-09-16T09:35:34.528 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2024-09-16T09:35:34.528 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2024-09-16T09:35:34.528 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2024-09-16T09:35:34.528 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2024-09-16T09:35:34.528 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2024-09-16T09:35:34.528 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2024-09-16T09:35:34.528 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2024-09-16T09:35:34.528 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2024-09-16T09:35:34.528 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2024-09-16T09:35:34.528 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2024-09-16T09:35:34.529 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2024-09-16T09:35:34.529 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2024-09-16T09:35:34.529 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T09:35:34.529 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T09:35:34.529 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T09:35:34.529 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2024-09-16T09:35:34.529 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2024-09-16T09:35:34.529 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2024-09-16T09:35:34.529 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2024-09-16T09:35:34.529 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2024-09-16T09:35:34.529 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.529 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2024-09-16T09:35:34.530 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2024-09-16T09:35:34.530 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2024-09-16T09:35:34.530 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2024-09-16T09:35:34.530 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2024-09-16T09:35:34.530 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T09:35:34.530 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2024-09-16T09:35:34.530 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2024-09-16T09:35:34.530 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2024-09-16T09:35:34.530 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2024-09-16T09:35:34.530 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2024-09-16T09:35:34.530 INFO:teuthology.orchestra.run.smithi070.stdout: "options": {}, 2024-09-16T09:35:34.530 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2024-09-16T09:35:34.531 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw": {} 2024-09-16T09:35:34.531 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.531 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2024-09-16T09:35:34.531 INFO:teuthology.orchestra.run.smithi070.stdout: "score_type": "Fair distribution", 2024-09-16T09:35:34.531 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 1.75, 2024-09-16T09:35:34.531 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 1.75, 2024-09-16T09:35:34.531 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1.1399999856948853, 2024-09-16T09:35:34.531 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 2, 2024-09-16T09:35:34.531 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 2, 2024-09-16T09:35:34.531 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 0.70719337463378906, 2024-09-16T09:35:34.531 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 0.81000000238418579, 2024-09-16T09:35:34.532 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 0.70999997854232788 2024-09-16T09:35:34.532 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:35:34.532 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.532 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.532 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 6, 2024-09-16T09:35:34.532 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "default.rgw.meta", 2024-09-16T09:35:34.532 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2024-09-16T09:25:05.773530+0000", 2024-09-16T09:35:34.532 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2024-09-16T09:35:34.532 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2024-09-16T09:35:34.532 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2024-09-16T09:35:34.532 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 3, 2024-09-16T09:35:34.533 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 2, 2024-09-16T09:35:34.533 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2024-09-16T09:35:34.533 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2024-09-16T09:35:34.533 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2024-09-16T09:35:34.533 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T09:35:34.533 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T09:35:34.533 INFO:teuthology.orchestra.run.smithi070.stdout: "is_stretch_pool": false, 2024-09-16T09:35:34.533 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2024-09-16T09:35:34.533 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2024-09-16T09:35:34.533 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 32, 2024-09-16T09:35:34.533 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 32, 2024-09-16T09:35:34.534 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 32, 2024-09-16T09:35:34.534 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 32, 2024-09-16T09:35:34.534 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 32, 2024-09-16T09:35:34.534 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2024-09-16T09:35:34.534 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2024-09-16T09:35:34.534 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2024-09-16T09:35:34.534 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2024-09-16T09:35:34.534 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2024-09-16T09:35:34.534 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2024-09-16T09:35:34.534 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2024-09-16T09:35:34.535 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.535 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "66", 2024-09-16T09:35:34.535 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2024-09-16T09:35:34.535 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T09:35:34.535 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T09:35:34.535 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2024-09-16T09:35:34.535 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2024-09-16T09:35:34.535 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2024-09-16T09:35:34.535 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2024-09-16T09:35:34.535 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2024-09-16T09:35:34.535 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2024-09-16T09:35:34.536 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2024-09-16T09:35:34.536 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2024-09-16T09:35:34.536 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2024-09-16T09:35:34.536 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2024-09-16T09:35:34.536 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2024-09-16T09:35:34.536 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2024-09-16T09:35:34.536 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2024-09-16T09:35:34.536 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2024-09-16T09:35:34.536 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2024-09-16T09:35:34.536 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T09:35:34.536 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T09:35:34.536 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T09:35:34.537 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2024-09-16T09:35:34.537 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2024-09-16T09:35:34.537 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2024-09-16T09:35:34.537 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2024-09-16T09:35:34.537 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2024-09-16T09:35:34.537 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.537 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2024-09-16T09:35:34.537 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2024-09-16T09:35:34.537 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2024-09-16T09:35:34.537 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2024-09-16T09:35:34.538 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2024-09-16T09:35:34.538 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T09:35:34.538 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2024-09-16T09:35:34.538 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2024-09-16T09:35:34.538 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2024-09-16T09:35:34.538 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2024-09-16T09:35:34.538 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2024-09-16T09:35:34.538 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2024-09-16T09:35:34.538 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_bias": 4 2024-09-16T09:35:34.538 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.538 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2024-09-16T09:35:34.538 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw": {} 2024-09-16T09:35:34.539 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.539 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2024-09-16T09:35:34.539 INFO:teuthology.orchestra.run.smithi070.stdout: "score_type": "Fair distribution", 2024-09-16T09:35:34.539 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 1.5399999618530273, 2024-09-16T09:35:34.539 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 1.5399999618530273, 2024-09-16T09:35:34.539 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1.1399999856948853, 2024-09-16T09:35:34.539 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 1.75, 2024-09-16T09:35:34.539 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 1.75, 2024-09-16T09:35:34.539 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 0.70719337463378906, 2024-09-16T09:35:34.539 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 0.81000000238418579, 2024-09-16T09:35:34.539 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 0.70999997854232788 2024-09-16T09:35:34.540 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:35:34.540 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.540 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.540 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 9, 2024-09-16T09:35:34.540 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "unique_pool_2", 2024-09-16T09:35:34.540 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2024-09-16T09:33:23.906323+0000", 2024-09-16T09:35:34.540 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2024-09-16T09:35:34.540 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2024-09-16T09:35:34.540 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2024-09-16T09:35:34.540 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 3, 2024-09-16T09:35:34.540 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 2, 2024-09-16T09:35:34.541 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2024-09-16T09:35:34.541 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2024-09-16T09:35:34.541 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2024-09-16T09:35:34.541 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T09:35:34.541 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T09:35:34.541 INFO:teuthology.orchestra.run.smithi070.stdout: "is_stretch_pool": false, 2024-09-16T09:35:34.541 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2024-09-16T09:35:34.541 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2024-09-16T09:35:34.541 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 26, 2024-09-16T09:35:34.541 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 16, 2024-09-16T09:35:34.542 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 16, 2024-09-16T09:35:34.542 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 16, 2024-09-16T09:35:34.542 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 26, 2024-09-16T09:35:34.542 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2024-09-16T09:35:34.542 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2024-09-16T09:35:34.542 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2024-09-16T09:35:34.542 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2024-09-16T09:35:34.542 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2024-09-16T09:35:34.542 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2024-09-16T09:35:34.542 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2024-09-16T09:35:34.542 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.542 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "282", 2024-09-16T09:35:34.543 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2024-09-16T09:35:34.543 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T09:35:34.543 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "243", 2024-09-16T09:35:34.543 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2024-09-16T09:35:34.543 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2024-09-16T09:35:34.543 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2024-09-16T09:35:34.543 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2024-09-16T09:35:34.543 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2024-09-16T09:35:34.543 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2024-09-16T09:35:34.543 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2024-09-16T09:35:34.544 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2024-09-16T09:35:34.544 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2024-09-16T09:35:34.544 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2024-09-16T09:35:34.544 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2024-09-16T09:35:34.544 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2024-09-16T09:35:34.544 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2024-09-16T09:35:34.544 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2024-09-16T09:35:34.545 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2024-09-16T09:35:34.545 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T09:35:34.545 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T09:35:34.545 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T09:35:34.545 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2024-09-16T09:35:34.545 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2024-09-16T09:35:34.546 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2024-09-16T09:35:34.546 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2024-09-16T09:35:34.546 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2024-09-16T09:35:34.546 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.546 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2024-09-16T09:35:34.546 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2024-09-16T09:35:34.546 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2024-09-16T09:35:34.546 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2024-09-16T09:35:34.546 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2024-09-16T09:35:34.546 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T09:35:34.546 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2024-09-16T09:35:34.546 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2024-09-16T09:35:34.547 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2024-09-16T09:35:34.547 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2024-09-16T09:35:34.547 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2024-09-16T09:35:34.547 INFO:teuthology.orchestra.run.smithi070.stdout: "options": {}, 2024-09-16T09:35:34.547 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2024-09-16T09:35:34.547 INFO:teuthology.orchestra.run.smithi070.stdout: "rados": {} 2024-09-16T09:35:34.547 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.547 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2024-09-16T09:35:34.547 INFO:teuthology.orchestra.run.smithi070.stdout: "score_type": "Fair distribution", 2024-09-16T09:35:34.547 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 4.3400001525878906, 2024-09-16T09:35:34.547 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 1.8899999856948853, 2024-09-16T09:35:34.548 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1.1399999856948853, 2024-09-16T09:35:34.548 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 4.9499998092651367, 2024-09-16T09:35:34.548 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 2.1500000953674316, 2024-09-16T09:35:34.548 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 0.70719337463378906, 2024-09-16T09:35:34.548 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 0.81000000238418579, 2024-09-16T09:35:34.548 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 0.70999997854232788, 2024-09-16T09:35:34.548 INFO:teuthology.orchestra.run.smithi070.stdout: "error_message": "pool 9 has acting primaries on OSD(s) with primary affinity 0, read balance score is not accurate" 2024-09-16T09:35:34.548 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:35:34.548 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:35:34.548 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-16T09:35:34.548 INFO:teuthology.orchestra.run.smithi070.stdout: "osds": [ 2024-09-16T09:35:34.549 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.549 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 0, 2024-09-16T09:35:34.549 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "7ef288c8-3bf0-4814-8e86-f70a7de041ce", 2024-09-16T09:35:34.549 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2024-09-16T09:35:34.549 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2024-09-16T09:35:34.549 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2024-09-16T09:35:34.549 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2024-09-16T09:35:34.549 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2024-09-16T09:35:34.549 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2024-09-16T09:35:34.549 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 8, 2024-09-16T09:35:34.549 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 283, 2024-09-16T09:35:34.550 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2024-09-16T09:35:34.550 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2024-09-16T09:35:34.550 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2024-09-16T09:35:34.550 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:35:34.550 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.550 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:35:34.550 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6802", 2024-09-16T09:35:34.550 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2013075781 2024-09-16T09:35:34.550 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.550 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.550 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:35:34.550 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6803", 2024-09-16T09:35:34.551 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2013075781 2024-09-16T09:35:34.551 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:35:34.551 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:35:34.551 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.551 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2024-09-16T09:35:34.551 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:35:34.551 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.551 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:35:34.551 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6804", 2024-09-16T09:35:34.551 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2013075781 2024-09-16T09:35:34.551 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.552 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.552 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:35:34.552 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6805", 2024-09-16T09:35:34.552 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2013075781 2024-09-16T09:35:34.552 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:35:34.552 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:35:34.552 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.552 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2024-09-16T09:35:34.552 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:35:34.552 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.552 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:35:34.553 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6808", 2024-09-16T09:35:34.553 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2013075781 2024-09-16T09:35:34.553 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.553 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.553 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:35:34.553 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6809", 2024-09-16T09:35:34.553 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2013075781 2024-09-16T09:35:34.553 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:35:34.553 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:35:34.553 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.553 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2024-09-16T09:35:34.554 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:35:34.554 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.554 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:35:34.554 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6806", 2024-09-16T09:35:34.554 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2013075781 2024-09-16T09:35:34.554 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.554 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.554 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:35:34.554 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6807", 2024-09-16T09:35:34.554 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2013075781 2024-09-16T09:35:34.554 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:35:34.554 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:35:34.554 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.555 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6803/2013075781", 2024-09-16T09:35:34.555 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6805/2013075781", 2024-09-16T09:35:34.555 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6809/2013075781", 2024-09-16T09:35:34.555 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6807/2013075781", 2024-09-16T09:35:34.555 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2024-09-16T09:35:34.555 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2024-09-16T09:35:34.555 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2024-09-16T09:35:34.555 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:35:34.555 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.555 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.555 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 1, 2024-09-16T09:35:34.555 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "474d41d9-295a-450c-8f27-ce487a14678b", 2024-09-16T09:35:34.556 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2024-09-16T09:35:34.556 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2024-09-16T09:35:34.556 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 0.95001220703125, 2024-09-16T09:35:34.556 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 0, 2024-09-16T09:35:34.556 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2024-09-16T09:35:34.556 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2024-09-16T09:35:34.556 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2024-09-16T09:35:34.556 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 283, 2024-09-16T09:35:34.556 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2024-09-16T09:35:34.556 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2024-09-16T09:35:34.556 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2024-09-16T09:35:34.556 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:35:34.557 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.557 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:35:34.557 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6810", 2024-09-16T09:35:34.557 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1670012115 2024-09-16T09:35:34.557 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.557 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.557 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:35:34.557 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6811", 2024-09-16T09:35:34.557 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1670012115 2024-09-16T09:35:34.557 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:35:34.557 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:35:34.558 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.558 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2024-09-16T09:35:34.558 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:35:34.558 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.558 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:35:34.558 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6812", 2024-09-16T09:35:34.558 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1670012115 2024-09-16T09:35:34.558 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.558 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.558 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:35:34.558 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6813", 2024-09-16T09:35:34.559 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1670012115 2024-09-16T09:35:34.559 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:35:34.559 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:35:34.559 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.559 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2024-09-16T09:35:34.559 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:35:34.559 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.559 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:35:34.559 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6816", 2024-09-16T09:35:34.559 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1670012115 2024-09-16T09:35:34.559 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.559 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.560 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:35:34.560 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6817", 2024-09-16T09:35:34.560 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1670012115 2024-09-16T09:35:34.560 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:35:34.560 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:35:34.560 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.560 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2024-09-16T09:35:34.560 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:35:34.560 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.560 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:35:34.560 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6814", 2024-09-16T09:35:34.561 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1670012115 2024-09-16T09:35:34.561 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.561 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.561 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:35:34.561 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6815", 2024-09-16T09:35:34.561 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1670012115 2024-09-16T09:35:34.561 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:35:34.561 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:35:34.561 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.561 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6811/1670012115", 2024-09-16T09:35:34.561 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6813/1670012115", 2024-09-16T09:35:34.562 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6817/1670012115", 2024-09-16T09:35:34.562 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6815/1670012115", 2024-09-16T09:35:34.562 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2024-09-16T09:35:34.562 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2024-09-16T09:35:34.562 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2024-09-16T09:35:34.562 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:35:34.562 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.562 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.562 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 2, 2024-09-16T09:35:34.562 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "5b4ca4e8-cecc-4163-b291-c9209b02c4b7", 2024-09-16T09:35:34.562 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2024-09-16T09:35:34.563 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2024-09-16T09:35:34.563 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2024-09-16T09:35:34.563 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 0.1810150146484375, 2024-09-16T09:35:34.563 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2024-09-16T09:35:34.563 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2024-09-16T09:35:34.563 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 18, 2024-09-16T09:35:34.563 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 255, 2024-09-16T09:35:34.563 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2024-09-16T09:35:34.563 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2024-09-16T09:35:34.563 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2024-09-16T09:35:34.563 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:35:34.564 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.564 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:35:34.564 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6818", 2024-09-16T09:35:34.564 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3601682336 2024-09-16T09:35:34.564 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.564 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.564 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:35:34.564 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6819", 2024-09-16T09:35:34.564 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3601682336 2024-09-16T09:35:34.564 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:35:34.564 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:35:34.564 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.565 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2024-09-16T09:35:34.565 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:35:34.565 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.565 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:35:34.565 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6820", 2024-09-16T09:35:34.565 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3601682336 2024-09-16T09:35:34.565 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.565 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.565 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:35:34.565 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6821", 2024-09-16T09:35:34.565 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3601682336 2024-09-16T09:35:34.565 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:35:34.565 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:35:34.566 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.566 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2024-09-16T09:35:34.566 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:35:34.566 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.566 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:35:34.566 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6824", 2024-09-16T09:35:34.566 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3601682336 2024-09-16T09:35:34.566 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.566 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.566 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:35:34.566 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6825", 2024-09-16T09:35:34.566 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3601682336 2024-09-16T09:35:34.567 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:35:34.567 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:35:34.567 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.567 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2024-09-16T09:35:34.567 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:35:34.567 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.567 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:35:34.567 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6822", 2024-09-16T09:35:34.567 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3601682336 2024-09-16T09:35:34.567 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.567 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.567 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:35:34.568 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6823", 2024-09-16T09:35:34.568 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3601682336 2024-09-16T09:35:34.568 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:35:34.568 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:35:34.568 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.568 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6819/3601682336", 2024-09-16T09:35:34.568 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6821/3601682336", 2024-09-16T09:35:34.568 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6825/3601682336", 2024-09-16T09:35:34.568 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6823/3601682336", 2024-09-16T09:35:34.568 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2024-09-16T09:35:34.568 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2024-09-16T09:35:34.568 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2024-09-16T09:35:34.569 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:35:34.569 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.569 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.569 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 3, 2024-09-16T09:35:34.569 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "ffb8d527-1b58-4ecd-808c-47d4ba1bf387", 2024-09-16T09:35:34.569 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2024-09-16T09:35:34.569 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2024-09-16T09:35:34.569 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 0.95001220703125, 2024-09-16T09:35:34.569 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 0.476531982421875, 2024-09-16T09:35:34.569 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2024-09-16T09:35:34.569 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2024-09-16T09:35:34.569 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 26, 2024-09-16T09:35:34.570 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 283, 2024-09-16T09:35:34.570 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2024-09-16T09:35:34.570 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2024-09-16T09:35:34.570 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2024-09-16T09:35:34.570 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:35:34.570 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.570 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:35:34.570 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6826", 2024-09-16T09:35:34.570 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2038083007 2024-09-16T09:35:34.570 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.570 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.571 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:35:34.571 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6827", 2024-09-16T09:35:34.571 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2038083007 2024-09-16T09:35:34.571 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:35:34.571 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:35:34.571 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.571 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2024-09-16T09:35:34.571 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:35:34.571 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.571 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:35:34.571 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6828", 2024-09-16T09:35:34.571 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2038083007 2024-09-16T09:35:34.572 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.572 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.572 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:35:34.572 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6829", 2024-09-16T09:35:34.572 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2038083007 2024-09-16T09:35:34.572 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:35:34.572 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:35:34.572 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.572 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2024-09-16T09:35:34.572 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:35:34.572 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.573 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:35:34.573 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6832", 2024-09-16T09:35:34.573 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2038083007 2024-09-16T09:35:34.573 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.573 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.573 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:35:34.573 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6833", 2024-09-16T09:35:34.573 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2038083007 2024-09-16T09:35:34.573 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:35:34.573 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:35:34.573 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.574 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2024-09-16T09:35:34.574 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:35:34.574 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.574 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:35:34.574 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6830", 2024-09-16T09:35:34.574 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2038083007 2024-09-16T09:35:34.574 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.574 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.574 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:35:34.574 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6831", 2024-09-16T09:35:34.574 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2038083007 2024-09-16T09:35:34.574 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:35:34.575 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:35:34.575 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.575 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6827/2038083007", 2024-09-16T09:35:34.575 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6829/2038083007", 2024-09-16T09:35:34.575 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6833/2038083007", 2024-09-16T09:35:34.575 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6831/2038083007", 2024-09-16T09:35:34.575 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2024-09-16T09:35:34.575 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2024-09-16T09:35:34.575 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2024-09-16T09:35:34.575 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:35:34.575 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.575 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.576 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 4, 2024-09-16T09:35:34.576 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "a946760b-4d91-4a4b-8e9b-a07a50589211", 2024-09-16T09:35:34.576 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2024-09-16T09:35:34.576 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2024-09-16T09:35:34.576 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2024-09-16T09:35:34.576 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2024-09-16T09:35:34.576 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2024-09-16T09:35:34.576 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2024-09-16T09:35:34.576 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 32, 2024-09-16T09:35:34.576 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 264, 2024-09-16T09:35:34.577 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2024-09-16T09:35:34.577 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2024-09-16T09:35:34.577 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2024-09-16T09:35:34.577 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:35:34.577 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.577 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:35:34.577 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6800", 2024-09-16T09:35:34.577 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2250892710 2024-09-16T09:35:34.577 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.577 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.577 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:35:34.577 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6801", 2024-09-16T09:35:34.578 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2250892710 2024-09-16T09:35:34.578 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:35:34.578 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:35:34.578 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.578 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2024-09-16T09:35:34.578 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:35:34.578 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.578 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:35:34.578 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6802", 2024-09-16T09:35:34.578 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2250892710 2024-09-16T09:35:34.578 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.578 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.579 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:35:34.579 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6803", 2024-09-16T09:35:34.579 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2250892710 2024-09-16T09:35:34.579 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:35:34.579 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:35:34.579 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.579 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2024-09-16T09:35:34.579 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:35:34.579 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.579 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:35:34.579 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6806", 2024-09-16T09:35:34.579 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2250892710 2024-09-16T09:35:34.580 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.580 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.580 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:35:34.580 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6807", 2024-09-16T09:35:34.580 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2250892710 2024-09-16T09:35:34.580 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:35:34.580 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:35:34.580 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.580 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2024-09-16T09:35:34.580 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:35:34.580 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.581 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:35:34.581 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6804", 2024-09-16T09:35:34.581 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2250892710 2024-09-16T09:35:34.581 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.581 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.581 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:35:34.581 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6805", 2024-09-16T09:35:34.581 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2250892710 2024-09-16T09:35:34.581 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:35:34.581 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:35:34.581 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.581 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.84:6801/2250892710", 2024-09-16T09:35:34.582 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.84:6803/2250892710", 2024-09-16T09:35:34.582 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.84:6807/2250892710", 2024-09-16T09:35:34.582 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.84:6805/2250892710", 2024-09-16T09:35:34.582 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2024-09-16T09:35:34.582 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2024-09-16T09:35:34.582 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2024-09-16T09:35:34.582 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:35:34.582 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.582 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.582 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 5, 2024-09-16T09:35:34.583 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "67681760-75a9-4846-946e-a935d42c5e05", 2024-09-16T09:35:34.583 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2024-09-16T09:35:34.583 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2024-09-16T09:35:34.583 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 0.95001220703125, 2024-09-16T09:35:34.583 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2024-09-16T09:35:34.583 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2024-09-16T09:35:34.583 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2024-09-16T09:35:34.583 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 39, 2024-09-16T09:35:34.583 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 283, 2024-09-16T09:35:34.583 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2024-09-16T09:35:34.583 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2024-09-16T09:35:34.583 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2024-09-16T09:35:34.584 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:35:34.584 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.584 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:35:34.584 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6808", 2024-09-16T09:35:34.584 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1803945348 2024-09-16T09:35:34.584 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.584 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.584 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:35:34.584 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6809", 2024-09-16T09:35:34.584 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1803945348 2024-09-16T09:35:34.584 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:35:34.585 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:35:34.585 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.585 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2024-09-16T09:35:34.585 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:35:34.585 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.585 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:35:34.585 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6810", 2024-09-16T09:35:34.585 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1803945348 2024-09-16T09:35:34.585 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.585 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.586 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:35:34.586 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6811", 2024-09-16T09:35:34.586 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1803945348 2024-09-16T09:35:34.586 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:35:34.586 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:35:34.586 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.586 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2024-09-16T09:35:34.586 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:35:34.586 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.586 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:35:34.586 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6814", 2024-09-16T09:35:34.586 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1803945348 2024-09-16T09:35:34.586 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.587 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.587 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:35:34.587 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6815", 2024-09-16T09:35:34.587 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1803945348 2024-09-16T09:35:34.587 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:35:34.587 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:35:34.587 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.587 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2024-09-16T09:35:34.587 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:35:34.587 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.587 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:35:34.588 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6812", 2024-09-16T09:35:34.588 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1803945348 2024-09-16T09:35:34.588 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.588 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.588 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:35:34.588 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6813", 2024-09-16T09:35:34.588 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1803945348 2024-09-16T09:35:34.588 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:35:34.588 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:35:34.588 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.588 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.84:6809/1803945348", 2024-09-16T09:35:34.589 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.84:6811/1803945348", 2024-09-16T09:35:34.589 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.84:6815/1803945348", 2024-09-16T09:35:34.589 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.84:6813/1803945348", 2024-09-16T09:35:34.589 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2024-09-16T09:35:34.589 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2024-09-16T09:35:34.589 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2024-09-16T09:35:34.589 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:35:34.589 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.589 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.589 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 6, 2024-09-16T09:35:34.589 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "7c8f0152-f539-4b7e-8113-91dd7529d3a1", 2024-09-16T09:35:34.590 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2024-09-16T09:35:34.590 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2024-09-16T09:35:34.590 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2024-09-16T09:35:34.590 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2024-09-16T09:35:34.590 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2024-09-16T09:35:34.590 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2024-09-16T09:35:34.590 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 46, 2024-09-16T09:35:34.590 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 282, 2024-09-16T09:35:34.590 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2024-09-16T09:35:34.590 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2024-09-16T09:35:34.590 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2024-09-16T09:35:34.590 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:35:34.591 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.591 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:35:34.591 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6816", 2024-09-16T09:35:34.591 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 610727189 2024-09-16T09:35:34.591 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.591 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.591 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:35:34.591 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6817", 2024-09-16T09:35:34.591 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 610727189 2024-09-16T09:35:34.591 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:35:34.591 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:35:34.591 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.592 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2024-09-16T09:35:34.592 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:35:34.592 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.592 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:35:34.592 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6818", 2024-09-16T09:35:34.592 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 610727189 2024-09-16T09:35:34.592 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.592 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.592 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:35:34.592 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6819", 2024-09-16T09:35:34.592 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 610727189 2024-09-16T09:35:34.593 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:35:34.593 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:35:34.593 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.593 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2024-09-16T09:35:34.593 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:35:34.593 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.593 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:35:34.593 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6822", 2024-09-16T09:35:34.593 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 610727189 2024-09-16T09:35:34.593 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.593 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.593 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:35:34.594 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6823", 2024-09-16T09:35:34.594 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 610727189 2024-09-16T09:35:34.594 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:35:34.594 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:35:34.594 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.594 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2024-09-16T09:35:34.594 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:35:34.594 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.594 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:35:34.594 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6820", 2024-09-16T09:35:34.594 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 610727189 2024-09-16T09:35:34.595 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.595 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.595 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:35:34.595 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6821", 2024-09-16T09:35:34.595 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 610727189 2024-09-16T09:35:34.595 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:35:34.595 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:35:34.595 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.595 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.84:6817/610727189", 2024-09-16T09:35:34.595 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.84:6819/610727189", 2024-09-16T09:35:34.595 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.84:6823/610727189", 2024-09-16T09:35:34.595 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.84:6821/610727189", 2024-09-16T09:35:34.596 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2024-09-16T09:35:34.596 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2024-09-16T09:35:34.596 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2024-09-16T09:35:34.596 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:35:34.596 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.596 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.596 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 7, 2024-09-16T09:35:34.596 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "945efaba-225e-49af-aefd-b4d58ca8093b", 2024-09-16T09:35:34.596 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2024-09-16T09:35:34.596 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2024-09-16T09:35:34.596 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 0.95001220703125, 2024-09-16T09:35:34.597 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2024-09-16T09:35:34.597 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2024-09-16T09:35:34.597 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2024-09-16T09:35:34.597 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 53, 2024-09-16T09:35:34.597 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 285, 2024-09-16T09:35:34.597 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2024-09-16T09:35:34.597 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2024-09-16T09:35:34.597 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2024-09-16T09:35:34.597 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:35:34.597 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.597 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:35:34.598 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6824", 2024-09-16T09:35:34.598 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1002252531 2024-09-16T09:35:34.598 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.598 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.598 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:35:34.598 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6825", 2024-09-16T09:35:34.598 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1002252531 2024-09-16T09:35:34.598 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:35:34.598 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:35:34.598 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.598 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2024-09-16T09:35:34.598 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:35:34.599 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.599 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:35:34.599 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6826", 2024-09-16T09:35:34.599 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1002252531 2024-09-16T09:35:34.599 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.599 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.599 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:35:34.599 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6827", 2024-09-16T09:35:34.599 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1002252531 2024-09-16T09:35:34.599 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:35:34.599 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:35:34.599 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.600 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2024-09-16T09:35:34.600 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:35:34.600 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.600 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:35:34.600 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6830", 2024-09-16T09:35:34.600 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1002252531 2024-09-16T09:35:34.600 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.600 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.600 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:35:34.600 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6831", 2024-09-16T09:35:34.601 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1002252531 2024-09-16T09:35:34.601 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:35:34.601 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:35:34.601 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.601 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2024-09-16T09:35:34.601 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:35:34.601 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.601 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:35:34.601 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6828", 2024-09-16T09:35:34.601 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1002252531 2024-09-16T09:35:34.602 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.602 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.602 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:35:34.602 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6829", 2024-09-16T09:35:34.602 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1002252531 2024-09-16T09:35:34.602 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:35:34.602 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:35:34.602 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.602 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.84:6825/1002252531", 2024-09-16T09:35:34.602 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.84:6827/1002252531", 2024-09-16T09:35:34.602 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.84:6831/1002252531", 2024-09-16T09:35:34.603 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.84:6829/1002252531", 2024-09-16T09:35:34.603 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2024-09-16T09:35:34.603 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2024-09-16T09:35:34.603 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2024-09-16T09:35:34.603 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:35:34.603 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:35:34.603 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-16T09:35:34.603 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_xinfo": [ 2024-09-16T09:35:34.603 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.603 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 0, 2024-09-16T09:35:34.603 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2024-09-16T09:35:34.604 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2024-09-16T09:35:34.604 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2024-09-16T09:35:34.604 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540719139924082687, 2024-09-16T09:35:34.604 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2024-09-16T09:35:34.604 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2024-09-16T09:35:07.849981+0000", 2024-09-16T09:35:34.604 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2024-09-16T09:35:34.604 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.604 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.604 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 1, 2024-09-16T09:35:34.604 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2024-09-16T09:35:34.605 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2024-09-16T09:35:34.605 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2024-09-16T09:35:34.605 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540719139924082687, 2024-09-16T09:35:34.605 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2024-09-16T09:35:34.605 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2024-09-16T09:34:40.462734+0000", 2024-09-16T09:35:34.605 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2024-09-16T09:35:34.605 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.605 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.605 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 2, 2024-09-16T09:35:34.605 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2024-09-16T09:35:34.605 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2024-09-16T09:35:34.606 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2024-09-16T09:35:34.606 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540719139924082687, 2024-09-16T09:35:34.606 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2024-09-16T09:35:34.606 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2024-09-16T09:34:45.327630+0000", 2024-09-16T09:35:34.606 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2024-09-16T09:35:34.606 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.606 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.606 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 3, 2024-09-16T09:35:34.606 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2024-09-16T09:35:34.606 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2024-09-16T09:35:34.606 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2024-09-16T09:35:34.606 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540719139924082687, 2024-09-16T09:35:34.607 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2024-09-16T09:35:34.607 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2024-09-16T09:34:55.598225+0000", 2024-09-16T09:35:34.607 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2024-09-16T09:35:34.607 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.607 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.607 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 4, 2024-09-16T09:35:34.607 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2024-09-16T09:35:34.607 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2024-09-16T09:35:34.607 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2024-09-16T09:35:34.607 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540719139924082687, 2024-09-16T09:35:34.607 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2024-09-16T09:35:34.607 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2024-09-16T09:34:51.095682+0000", 2024-09-16T09:35:34.607 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2024-09-16T09:35:34.608 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.608 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.608 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 5, 2024-09-16T09:35:34.608 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2024-09-16T09:35:34.608 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2024-09-16T09:35:34.608 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2024-09-16T09:35:34.608 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540719139924082687, 2024-09-16T09:35:34.608 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2024-09-16T09:35:34.608 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2024-09-16T09:35:00.308654+0000", 2024-09-16T09:35:34.608 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2024-09-16T09:35:34.608 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.609 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.609 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 6, 2024-09-16T09:35:34.609 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2024-09-16T09:35:34.609 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2024-09-16T09:35:34.609 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2024-09-16T09:35:34.609 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540719139924082687, 2024-09-16T09:35:34.609 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2024-09-16T09:35:34.609 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2024-09-16T09:35:26.990283+0000", 2024-09-16T09:35:34.609 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2024-09-16T09:35:34.609 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.609 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.609 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 7, 2024-09-16T09:35:34.610 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2024-09-16T09:35:34.610 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2024-09-16T09:35:34.610 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2024-09-16T09:35:34.610 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540719139924082687, 2024-09-16T09:35:34.610 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2024-09-16T09:35:34.610 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2024-09-16T09:35:15.569347+0000", 2024-09-16T09:35:34.610 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2024-09-16T09:35:34.610 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:35:34.610 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-16T09:35:34.610 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap": [ 2024-09-16T09:35:34.610 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.610 INFO:teuthology.orchestra.run.smithi070.stdout: "pgid": "3.1d", 2024-09-16T09:35:34.611 INFO:teuthology.orchestra.run.smithi070.stdout: "osds": [ 2024-09-16T09:35:34.611 INFO:teuthology.orchestra.run.smithi070.stdout: 1, 2024-09-16T09:35:34.611 INFO:teuthology.orchestra.run.smithi070.stdout: 0, 2024-09-16T09:35:34.611 INFO:teuthology.orchestra.run.smithi070.stdout: 7 2024-09-16T09:35:34.611 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:35:34.611 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:35:34.611 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-16T09:35:34.611 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap_items": [ 2024-09-16T09:35:34.611 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.611 INFO:teuthology.orchestra.run.smithi070.stdout: "pgid": "3.1b", 2024-09-16T09:35:34.611 INFO:teuthology.orchestra.run.smithi070.stdout: "mappings": [ 2024-09-16T09:35:34.611 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.612 INFO:teuthology.orchestra.run.smithi070.stdout: "from": 0, 2024-09-16T09:35:34.612 INFO:teuthology.orchestra.run.smithi070.stdout: "to": 6 2024-09-16T09:35:34.612 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.612 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.612 INFO:teuthology.orchestra.run.smithi070.stdout: "from": 4, 2024-09-16T09:35:34.612 INFO:teuthology.orchestra.run.smithi070.stdout: "to": 7 2024-09-16T09:35:34.612 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:35:34.612 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:35:34.612 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.612 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.612 INFO:teuthology.orchestra.run.smithi070.stdout: "pgid": "4.f", 2024-09-16T09:35:34.612 INFO:teuthology.orchestra.run.smithi070.stdout: "mappings": [ 2024-09-16T09:35:34.613 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.613 INFO:teuthology.orchestra.run.smithi070.stdout: "from": 1, 2024-09-16T09:35:34.613 INFO:teuthology.orchestra.run.smithi070.stdout: "to": 2 2024-09-16T09:35:34.613 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:35:34.613 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:35:34.613 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.613 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.613 INFO:teuthology.orchestra.run.smithi070.stdout: "pgid": "4.13", 2024-09-16T09:35:34.613 INFO:teuthology.orchestra.run.smithi070.stdout: "mappings": [ 2024-09-16T09:35:34.613 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.613 INFO:teuthology.orchestra.run.smithi070.stdout: "from": 1, 2024-09-16T09:35:34.614 INFO:teuthology.orchestra.run.smithi070.stdout: "to": 5 2024-09-16T09:35:34.614 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:35:34.614 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:35:34.614 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:35:34.614 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-16T09:35:34.614 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap_primaries": [], 2024-09-16T09:35:34.614 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_temp": [ 2024-09-16T09:35:34.614 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.614 INFO:teuthology.orchestra.run.smithi070.stdout: "pgid": "9.10", 2024-09-16T09:35:34.614 INFO:teuthology.orchestra.run.smithi070.stdout: "osds": [ 2024-09-16T09:35:34.614 INFO:teuthology.orchestra.run.smithi070.stdout: 0, 2024-09-16T09:35:34.614 INFO:teuthology.orchestra.run.smithi070.stdout: 1 2024-09-16T09:35:34.615 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:35:34.615 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.615 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.615 INFO:teuthology.orchestra.run.smithi070.stdout: "pgid": "9.12", 2024-09-16T09:35:34.615 INFO:teuthology.orchestra.run.smithi070.stdout: "osds": [ 2024-09-16T09:35:34.615 INFO:teuthology.orchestra.run.smithi070.stdout: 0, 2024-09-16T09:35:34.615 INFO:teuthology.orchestra.run.smithi070.stdout: 1 2024-09-16T09:35:34.615 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:35:34.615 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.615 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.615 INFO:teuthology.orchestra.run.smithi070.stdout: "pgid": "9.13", 2024-09-16T09:35:34.615 INFO:teuthology.orchestra.run.smithi070.stdout: "osds": [ 2024-09-16T09:35:34.616 INFO:teuthology.orchestra.run.smithi070.stdout: 7, 2024-09-16T09:35:34.616 INFO:teuthology.orchestra.run.smithi070.stdout: 5, 2024-09-16T09:35:34.616 INFO:teuthology.orchestra.run.smithi070.stdout: 6 2024-09-16T09:35:34.616 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:35:34.616 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.616 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.616 INFO:teuthology.orchestra.run.smithi070.stdout: "pgid": "9.15", 2024-09-16T09:35:34.616 INFO:teuthology.orchestra.run.smithi070.stdout: "osds": [ 2024-09-16T09:35:34.616 INFO:teuthology.orchestra.run.smithi070.stdout: 1, 2024-09-16T09:35:34.616 INFO:teuthology.orchestra.run.smithi070.stdout: 3, 2024-09-16T09:35:34.616 INFO:teuthology.orchestra.run.smithi070.stdout: 7 2024-09-16T09:35:34.616 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:35:34.617 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.617 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.617 INFO:teuthology.orchestra.run.smithi070.stdout: "pgid": "9.16", 2024-09-16T09:35:34.617 INFO:teuthology.orchestra.run.smithi070.stdout: "osds": [ 2024-09-16T09:35:34.617 INFO:teuthology.orchestra.run.smithi070.stdout: 1, 2024-09-16T09:35:34.617 INFO:teuthology.orchestra.run.smithi070.stdout: 2, 2024-09-16T09:35:34.617 INFO:teuthology.orchestra.run.smithi070.stdout: 4 2024-09-16T09:35:34.617 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:35:34.617 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.617 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.617 INFO:teuthology.orchestra.run.smithi070.stdout: "pgid": "9.17", 2024-09-16T09:35:34.617 INFO:teuthology.orchestra.run.smithi070.stdout: "osds": [ 2024-09-16T09:35:34.618 INFO:teuthology.orchestra.run.smithi070.stdout: 0, 2024-09-16T09:35:34.618 INFO:teuthology.orchestra.run.smithi070.stdout: 7, 2024-09-16T09:35:34.618 INFO:teuthology.orchestra.run.smithi070.stdout: 1 2024-09-16T09:35:34.618 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:35:34.618 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.618 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.618 INFO:teuthology.orchestra.run.smithi070.stdout: "pgid": "9.18", 2024-09-16T09:35:34.618 INFO:teuthology.orchestra.run.smithi070.stdout: "osds": [ 2024-09-16T09:35:34.618 INFO:teuthology.orchestra.run.smithi070.stdout: 5, 2024-09-16T09:35:34.618 INFO:teuthology.orchestra.run.smithi070.stdout: 7 2024-09-16T09:35:34.618 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:35:34.618 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.618 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:35:34.619 INFO:teuthology.orchestra.run.smithi070.stdout: "pgid": "9.19", 2024-09-16T09:35:34.619 INFO:teuthology.orchestra.run.smithi070.stdout: "osds": [ 2024-09-16T09:35:34.619 INFO:teuthology.orchestra.run.smithi070.stdout: 3, 2024-09-16T09:35:34.619 INFO:teuthology.orchestra.run.smithi070.stdout: 5, 2024-09-16T09:35:34.619 INFO:teuthology.orchestra.run.smithi070.stdout: 6 2024-09-16T09:35:34.619 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:35:34.619 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:35:34.619 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-16T09:35:34.619 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_temp": [], 2024-09-16T09:35:34.619 INFO:teuthology.orchestra.run.smithi070.stdout: "blocklist": { 2024-09-16T09:35:34.619 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/125491368": "2024-09-17T09:25:31.465167+0000", 2024-09-16T09:35:34.620 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/3354284852": "2024-09-17T09:19:43.205989+0000", 2024-09-16T09:35:34.620 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/3284625547": "2024-09-17T09:25:31.465167+0000", 2024-09-16T09:35:34.620 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/186565817": "2024-09-17T09:19:43.205989+0000", 2024-09-16T09:35:34.620 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/2213852865": "2024-09-17T09:19:43.205989+0000", 2024-09-16T09:35:34.620 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/1812373760": "2024-09-17T09:20:09.306330+0000", 2024-09-16T09:35:34.620 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:6800/837338094": "2024-09-17T09:19:43.205989+0000", 2024-09-16T09:35:34.620 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:6800/1876661229": "2024-09-17T09:25:31.465167+0000", 2024-09-16T09:35:34.620 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/3351688485": "2024-09-17T09:20:09.306330+0000", 2024-09-16T09:35:34.620 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:6800/3871553358": "2024-09-17T09:20:09.306330+0000", 2024-09-16T09:35:34.620 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/1230482871": "2024-09-17T09:25:31.465167+0000", 2024-09-16T09:35:34.620 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:6801/837338094": "2024-09-17T09:19:43.205989+0000", 2024-09-16T09:35:34.621 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:6801/3871553358": "2024-09-17T09:20:09.306330+0000", 2024-09-16T09:35:34.621 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/1403001486": "2024-09-17T09:20:09.306330+0000", 2024-09-16T09:35:34.621 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/1723643739": "2024-09-17T09:25:31.465167+0000", 2024-09-16T09:35:34.621 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:6801/1876661229": "2024-09-17T09:25:31.465167+0000", 2024-09-16T09:35:34.621 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/3456417396": "2024-09-17T09:25:31.465167+0000" 2024-09-16T09:35:34.621 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.621 INFO:teuthology.orchestra.run.smithi070.stdout: "range_blocklist": {}, 2024-09-16T09:35:34.621 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profiles": { 2024-09-16T09:35:34.621 INFO:teuthology.orchestra.run.smithi070.stdout: "default": { 2024-09-16T09:35:34.621 INFO:teuthology.orchestra.run.smithi070.stdout: "crush-failure-domain": "osd", 2024-09-16T09:35:34.621 INFO:teuthology.orchestra.run.smithi070.stdout: "k": "2", 2024-09-16T09:35:34.622 INFO:teuthology.orchestra.run.smithi070.stdout: "m": "1", 2024-09-16T09:35:34.622 INFO:teuthology.orchestra.run.smithi070.stdout: "plugin": "jerasure", 2024-09-16T09:35:34.622 INFO:teuthology.orchestra.run.smithi070.stdout: "technique": "reed_sol_van" 2024-09-16T09:35:34.622 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:35:34.622 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:35:34.622 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps_queue": [], 2024-09-16T09:35:34.622 INFO:teuthology.orchestra.run.smithi070.stdout: "new_removed_snaps": [], 2024-09-16T09:35:34.622 INFO:teuthology.orchestra.run.smithi070.stdout: "new_purged_snaps": [], 2024-09-16T09:35:34.622 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_node_flags": {}, 2024-09-16T09:35:34.622 INFO:teuthology.orchestra.run.smithi070.stdout: "device_class_flags": {}, 2024-09-16T09:35:34.622 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode": { 2024-09-16T09:35:34.623 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode_enabled": false, 2024-09-16T09:35:34.623 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_bucket_count": 0, 2024-09-16T09:35:34.623 INFO:teuthology.orchestra.run.smithi070.stdout: "degraded_stretch_mode": 0, 2024-09-16T09:35:34.623 INFO:teuthology.orchestra.run.smithi070.stdout: "recovering_stretch_mode": 0, 2024-09-16T09:35:34.623 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode_bucket": 0 2024-09-16T09:35:34.623 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:35:34.623 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T09:35:34.623 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:35:34.658 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:34.658 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:34.659 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:35:34.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:34 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:35:34.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:34 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1883640422' entity='client.admin' cmd={"prefix": "osd dump", "format": "json-pretty"} : dispatch 2024-09-16T09:35:34.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:34 smithi084 ceph-mon[32629]: Health check update: Degraded data redundancy: 80/299511 objects degraded (0.027%), 2 pgs degraded (PG_DEGRADED) 2024-09-16T09:35:34.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:34 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:35:34.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:34 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1883640422' entity='client.admin' cmd={"prefix": "osd dump", "format": "json-pretty"} : dispatch 2024-09-16T09:35:34.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:34 smithi070 ceph-mon[30388]: Health check update: Degraded data redundancy: 80/299511 objects degraded (0.027%), 2 pgs degraded (PG_DEGRADED) 2024-09-16T09:35:34.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:34 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:35:34.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:34 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1883640422' entity='client.admin' cmd={"prefix": "osd dump", "format": "json-pretty"} : dispatch 2024-09-16T09:35:34.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:34 smithi070 ceph-mon[40691]: Health check update: Degraded data redundancy: 80/299511 objects degraded (0.027%), 2 pgs degraded (PG_DEGRADED) 2024-09-16T09:35:34.760 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:35:34.801 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:34.802 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:34.802 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:35:34.903 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:35:34.927 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:34.927 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:34.928 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:35:35.029 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:35:35.068 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:35.069 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:35.070 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:35:35.170 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:35:35.207 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:35.208 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:35.208 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:35:35.309 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:35:35.341 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:35.341 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:35.342 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:35:35.443 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:35:35.484 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:35.485 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:35.485 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:35:35.586 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:35:35.633 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:35.634 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:35.634 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:35:35.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:35 smithi084 ceph-mon[32629]: pgmap v522: 178 pgs: 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 2 active+recovering+undersized+degraded+remapped, 3 active+remapped+backfill_toofull, 170 active+clean; 5.9 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 877 op/s; 80/290784 objects degraded (0.028%); 40127/290784 objects misplaced (13.800%); 15 MiB/s, 234 objects/s recovering 2024-09-16T09:35:35.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:35 smithi084 ceph-mon[32629]: 9.14 scrub starts 2024-09-16T09:35:35.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:35 smithi070 ceph-mon[30388]: pgmap v522: 178 pgs: 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 2 active+recovering+undersized+degraded+remapped, 3 active+remapped+backfill_toofull, 170 active+clean; 5.9 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 877 op/s; 80/290784 objects degraded (0.028%); 40127/290784 objects misplaced (13.800%); 15 MiB/s, 234 objects/s recovering 2024-09-16T09:35:35.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:35 smithi070 ceph-mon[30388]: 9.14 scrub starts 2024-09-16T09:35:35.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:35 smithi070 ceph-mon[40691]: pgmap v522: 178 pgs: 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 2 active+recovering+undersized+degraded+remapped, 3 active+remapped+backfill_toofull, 170 active+clean; 5.9 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 877 op/s; 80/290784 objects degraded (0.028%); 40127/290784 objects misplaced (13.800%); 15 MiB/s, 234 objects/s recovering 2024-09-16T09:35:35.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:35 smithi070 ceph-mon[40691]: 9.14 scrub starts 2024-09-16T09:35:35.736 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:35:35.775 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:35.776 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:35.776 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:35:35.877 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:35:35.904 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:35.904 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:35.905 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:35:36.005 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:35:36.034 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:36.035 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:36.035 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:35:36.136 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:35:36.173 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:36.174 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:36.174 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:35:36.275 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:35:36.314 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:36.315 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:36.315 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:35:36.416 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:35:36.456 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:36.459 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:36.460 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:35:36.561 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:35:36.597 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:36.598 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:36.598 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:35:36.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:36 smithi084 ceph-mon[32629]: 9.9 scrub starts 2024-09-16T09:35:36.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:36 smithi084 ceph-mon[32629]: 9.14 scrub ok 2024-09-16T09:35:36.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:36 smithi084 ceph-mon[32629]: 9.6 scrub starts 2024-09-16T09:35:36.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:36 smithi084 ceph-mon[32629]: 6.e scrub starts 2024-09-16T09:35:36.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:36 smithi084 ceph-mon[32629]: 9.1 scrub starts 2024-09-16T09:35:36.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:36 smithi084 ceph-mon[32629]: 6.e scrub ok 2024-09-16T09:35:36.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:36 smithi070 ceph-mon[30388]: 9.9 scrub starts 2024-09-16T09:35:36.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:36 smithi070 ceph-mon[30388]: 9.14 scrub ok 2024-09-16T09:35:36.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:36 smithi070 ceph-mon[30388]: 9.6 scrub starts 2024-09-16T09:35:36.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:36 smithi070 ceph-mon[30388]: 6.e scrub starts 2024-09-16T09:35:36.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:36 smithi070 ceph-mon[30388]: 9.1 scrub starts 2024-09-16T09:35:36.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:36 smithi070 ceph-mon[30388]: 6.e scrub ok 2024-09-16T09:35:36.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:36 smithi070 ceph-mon[40691]: 9.9 scrub starts 2024-09-16T09:35:36.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:36 smithi070 ceph-mon[40691]: 9.14 scrub ok 2024-09-16T09:35:36.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:36 smithi070 ceph-mon[40691]: 9.6 scrub starts 2024-09-16T09:35:36.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:36 smithi070 ceph-mon[40691]: 6.e scrub starts 2024-09-16T09:35:36.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:36 smithi070 ceph-mon[40691]: 9.1 scrub starts 2024-09-16T09:35:36.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:36 smithi070 ceph-mon[40691]: 6.e scrub ok 2024-09-16T09:35:36.699 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:35:36.728 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:36.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:36.729 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:35:36.829 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:35:36.871 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:36.871 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:36.872 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:35:36.973 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:35:37.015 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:37.016 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:37.016 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:35:37.117 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:35:37.142 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:37.143 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:37.143 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:35:37.244 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:35:37.271 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:37.272 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:37.272 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:35:37.373 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:35:37.399 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:37.400 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:37.400 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:35:37.501 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:35:37.546 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:37.546 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:37.547 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:35:37.648 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:35:37.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:37 smithi084 ceph-mon[32629]: pgmap v523: 178 pgs: 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 2 active+recovering+undersized+degraded+remapped, 3 active+remapped+backfill_toofull, 170 active+clean; 5.8 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 956 op/s; 115/287826 objects degraded (0.040%); 38723/287826 objects misplaced (13.454%); 18 MiB/s, 294 objects/s recovering 2024-09-16T09:35:37.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:37 smithi084 ceph-mon[32629]: 9.9 scrub ok 2024-09-16T09:35:37.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:37 smithi084 ceph-mon[32629]: 9.1 scrub ok 2024-09-16T09:35:37.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:37 smithi084 ceph-mon[32629]: 9.6 scrub ok 2024-09-16T09:35:37.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:37 smithi084 ceph-mon[32629]: 9.0 scrub starts 2024-09-16T09:35:37.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:37 smithi070 ceph-mon[30388]: pgmap v523: 178 pgs: 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 2 active+recovering+undersized+degraded+remapped, 3 active+remapped+backfill_toofull, 170 active+clean; 5.8 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 956 op/s; 115/287826 objects degraded (0.040%); 38723/287826 objects misplaced (13.454%); 18 MiB/s, 294 objects/s recovering 2024-09-16T09:35:37.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:37 smithi070 ceph-mon[30388]: 9.9 scrub ok 2024-09-16T09:35:37.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:37 smithi070 ceph-mon[30388]: 9.1 scrub ok 2024-09-16T09:35:37.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:37 smithi070 ceph-mon[30388]: 9.6 scrub ok 2024-09-16T09:35:37.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:37 smithi070 ceph-mon[30388]: 9.0 scrub starts 2024-09-16T09:35:37.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:37 smithi070 ceph-mon[40691]: pgmap v523: 178 pgs: 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 2 active+recovering+undersized+degraded+remapped, 3 active+remapped+backfill_toofull, 170 active+clean; 5.8 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 956 op/s; 115/287826 objects degraded (0.040%); 38723/287826 objects misplaced (13.454%); 18 MiB/s, 294 objects/s recovering 2024-09-16T09:35:37.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:37 smithi070 ceph-mon[40691]: 9.9 scrub ok 2024-09-16T09:35:37.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:37 smithi070 ceph-mon[40691]: 9.1 scrub ok 2024-09-16T09:35:37.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:37 smithi070 ceph-mon[40691]: 9.6 scrub ok 2024-09-16T09:35:37.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:37 smithi070 ceph-mon[40691]: 9.0 scrub starts 2024-09-16T09:35:37.675 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:37.676 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:37.676 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:35:37.777 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:35:37.803 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:37.803 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:37.804 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:35:37.905 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:35:37.941 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:37.942 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:37.942 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:35:38.043 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:35:38.073 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:38.073 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:38.073 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:35:38.174 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:35:38.214 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:38.215 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:38.215 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:35:38.316 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:35:38.351 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:38.351 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:38.351 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:35:38.452 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:35:38.480 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:38.481 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:38.481 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:35:38.582 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:35:38.611 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:38.611 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:38.612 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:35:38.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:38 smithi070 ceph-mon[30388]: 9.0 scrub ok 2024-09-16T09:35:38.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:38 smithi070 ceph-mon[30388]: 9.11 scrub starts 2024-09-16T09:35:38.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:38 smithi070 ceph-mon[40691]: 9.0 scrub ok 2024-09-16T09:35:38.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:38 smithi070 ceph-mon[40691]: 9.11 scrub starts 2024-09-16T09:35:38.713 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:35:38.718 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:38 smithi084 ceph-mon[32629]: 9.0 scrub ok 2024-09-16T09:35:38.718 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:38 smithi084 ceph-mon[32629]: 9.11 scrub starts 2024-09-16T09:35:38.746 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:38.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:38.747 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:35:38.848 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:35:38.875 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:38.876 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:38.876 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:35:38.977 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:35:39.002 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:39.003 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:39.003 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:35:39.104 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:35:39.132 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:39.132 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:39.133 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:35:39.234 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:35:39.272 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:39.272 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:39.272 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:35:39.373 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:35:39.421 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:39.422 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:39.422 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:35:39.523 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:35:39.551 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:39.552 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:39.553 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:35:39.653 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:35:39.675 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:39.676 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:39.676 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:35:39.777 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:35:39.824 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:39.825 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:39.826 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:35:39.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:39 smithi084 ceph-mon[32629]: pgmap v524: 178 pgs: 1 active+clean+scrubbing, 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 2 active+recovering+undersized+degraded+remapped, 3 active+remapped+backfill_toofull, 169 active+clean; 5.8 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 335 op/s; 115/286749 objects degraded (0.040%); 38723/286749 objects misplaced (13.504%); 15 MiB/s, 245 objects/s recovering 2024-09-16T09:35:39.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:39 smithi084 ceph-mon[32629]: 9.11 scrub ok 2024-09-16T09:35:39.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:39 smithi084 ceph-mon[32629]: Health check update: Degraded data redundancy: 115/286749 objects degraded (0.040%), 2 pgs degraded (PG_DEGRADED) 2024-09-16T09:35:39.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:39 smithi070 ceph-mon[30388]: pgmap v524: 178 pgs: 1 active+clean+scrubbing, 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 2 active+recovering+undersized+degraded+remapped, 3 active+remapped+backfill_toofull, 169 active+clean; 5.8 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 335 op/s; 115/286749 objects degraded (0.040%); 38723/286749 objects misplaced (13.504%); 15 MiB/s, 245 objects/s recovering 2024-09-16T09:35:39.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:39 smithi070 ceph-mon[30388]: 9.11 scrub ok 2024-09-16T09:35:39.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:39 smithi070 ceph-mon[30388]: Health check update: Degraded data redundancy: 115/286749 objects degraded (0.040%), 2 pgs degraded (PG_DEGRADED) 2024-09-16T09:35:39.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:39 smithi070 ceph-mon[40691]: pgmap v524: 178 pgs: 1 active+clean+scrubbing, 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 2 active+recovering+undersized+degraded+remapped, 3 active+remapped+backfill_toofull, 169 active+clean; 5.8 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 335 op/s; 115/286749 objects degraded (0.040%); 38723/286749 objects misplaced (13.504%); 15 MiB/s, 245 objects/s recovering 2024-09-16T09:35:39.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:39 smithi070 ceph-mon[40691]: 9.11 scrub ok 2024-09-16T09:35:39.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:39 smithi070 ceph-mon[40691]: Health check update: Degraded data redundancy: 115/286749 objects degraded (0.040%), 2 pgs degraded (PG_DEGRADED) 2024-09-16T09:35:39.926 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:35:39.949 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:39.949 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:39.950 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:35:40.050 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:35:40.084 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:40.085 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:40.086 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:35:40.187 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:35:40.208 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:40.209 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:40.210 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:35:40.310 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:35:40.345 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:40.345 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:40.346 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:35:40.446 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:35:40.477 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:40.477 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:40.478 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:35:40.579 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:35:40.606 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:40.606 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:40.607 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:35:40.708 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:35:40.738 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:40.739 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:40.739 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:35:40.840 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:35:40.872 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:40.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:40.873 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:35:40.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:40 smithi084 ceph-mon[32629]: 4.1b scrub starts 2024-09-16T09:35:40.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:40 smithi084 ceph-mon[32629]: 4.1b scrub ok 2024-09-16T09:35:40.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:40 smithi070 ceph-mon[30388]: 4.1b scrub starts 2024-09-16T09:35:40.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:40 smithi070 ceph-mon[30388]: 4.1b scrub ok 2024-09-16T09:35:40.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:40 smithi070 ceph-mon[40691]: 4.1b scrub starts 2024-09-16T09:35:40.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:40 smithi070 ceph-mon[40691]: 4.1b scrub ok 2024-09-16T09:35:40.974 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:35:40.999 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:41.000 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:41.000 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:35:41.077 INFO:tasks.thrashosds.thrasher:j is {'epoch': 291, 'fsid': 'a1321834-740c-11ef-bceb-c7b262605968', 'created': '2024-09-16T09:19:14.121498+0000', 'modified': '2024-09-16T09:35:29.275073+0000', 'last_up_change': '2024-09-16T09:24:51.124496+0000', 'last_in_change': '2024-09-16T09:24:28.432920+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 18, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 9, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T09:22:27.617694+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 21, 'pg_placement_num': 21, 'pg_placement_num_target': 21, 'pg_num_target': 21, 'pg_num_pending': 21, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '162', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '160', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6699999570846558, 'score_stable': 1.6699999570846558, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 1.899999976158142, 'raw_score_stable': 1.899999976158142, 'primary_affinity_weighted': 0.7071933746337891, 'average_primary_affinity': 0.8100000023841858, 'average_primary_affinity_weighted': 0.7099999785423279}}, {'pool': 2, 'pool_name': 'datapool', 'create_time': '2024-09-16T09:24:59.169244+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 3, 'pg_placement_num': 3, 'pg_placement_num_target': 3, 'pg_num_target': 3, 'pg_num_pending': 3, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 61, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.3299999237060547, 'score_stable': 2.3299999237060547, 'optimal_score': 1, 'raw_score_acting': 2.3299999237060547, 'raw_score_stable': 2.3299999237060547, 'primary_affinity_weighted': 0.7071933746337891, 'average_primary_affinity': 0.8100000023841858, 'average_primary_affinity_weighted': 0.7099999785423279}}, {'pool': 3, 'pool_name': '.rgw.root', 'create_time': '2024-09-16T09:25:00.056119+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.7071933746337891, 'average_primary_affinity': 0.8100000023841858, 'average_primary_affinity_weighted': 0.7099999785423279}}, {'pool': 4, 'pool_name': 'default.rgw.log', 'create_time': '2024-09-16T09:25:01.770016+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '62', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.849999904632568, 'score_stable': 2.190000057220459, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 7.809999942779541, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 0.7071933746337891, 'average_primary_affinity': 0.8100000023841858, 'average_primary_affinity_weighted': 0.7099999785423279, 'error_message': 'pool 4 has acting primaries on OSD(s) with primary affinity 0, read balance score is not accurate'}}, {'pool': 5, 'pool_name': 'default.rgw.control', 'create_time': '2024-09-16T09:25:03.725555+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.7071933746337891, 'average_primary_affinity': 0.8100000023841858, 'average_primary_affinity_weighted': 0.7099999785423279}}, {'pool': 6, 'pool_name': 'default.rgw.meta', 'create_time': '2024-09-16T09:25:05.773530+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5399999618530273, 'score_stable': 1.5399999618530273, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 0.7071933746337891, 'average_primary_affinity': 0.8100000023841858, 'average_primary_affinity_weighted': 0.7099999785423279}}, {'pool': 9, 'pool_name': 'unique_pool_2', 'create_time': '2024-09-16T09:33:23.906323+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 26, 'pg_placement_num': 16, 'pg_placement_num_target': 16, 'pg_num_target': 16, 'pg_num_pending': 26, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '282', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '243', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rados': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 4.340000152587891, 'score_stable': 1.8899999856948853, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 4.949999809265137, 'raw_score_stable': 2.1500000953674316, 'primary_affinity_weighted': 0.7071933746337891, 'average_primary_affinity': 0.8100000023841858, 'average_primary_affinity_weighted': 0.7099999785423279, 'error_message': 'pool 9 has acting primaries on OSD(s) with primary affinity 0, read balance score is not accurate'}}], 'osds': [{'osd': 0, 'uuid': '7ef288c8-3bf0-4814-8e86-f70a7de041ce', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 8, 'up_thru': 283, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6802', 'nonce': 2013075781}, {'type': 'v1', 'addr': '172.21.15.70:6803', 'nonce': 2013075781}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6804', 'nonce': 2013075781}, {'type': 'v1', 'addr': '172.21.15.70:6805', 'nonce': 2013075781}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6808', 'nonce': 2013075781}, {'type': 'v1', 'addr': '172.21.15.70:6809', 'nonce': 2013075781}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6806', 'nonce': 2013075781}, {'type': 'v1', 'addr': '172.21.15.70:6807', 'nonce': 2013075781}]}, 'public_addr': '172.21.15.70:6803/2013075781', 'cluster_addr': '172.21.15.70:6805/2013075781', 'heartbeat_back_addr': '172.21.15.70:6809/2013075781', 'heartbeat_front_addr': '172.21.15.70:6807/2013075781', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '474d41d9-295a-450c-8f27-ce487a14678b', 'up': 1, 'in': 1, 'weight': 0.95001220703125, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 283, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6810', 'nonce': 1670012115}, {'type': 'v1', 'addr': '172.21.15.70:6811', 'nonce': 1670012115}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6812', 'nonce': 1670012115}, {'type': 'v1', 'addr': '172.21.15.70:6813', 'nonce': 1670012115}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6816', 'nonce': 1670012115}, {'type': 'v1', 'addr': '172.21.15.70:6817', 'nonce': 1670012115}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6814', 'nonce': 1670012115}, {'type': 'v1', 'addr': '172.21.15.70:6815', 'nonce': 1670012115}]}, 'public_addr': '172.21.15.70:6811/1670012115', 'cluster_addr': '172.21.15.70:6813/1670012115', 'heartbeat_back_addr': '172.21.15.70:6817/1670012115', 'heartbeat_front_addr': '172.21.15.70:6815/1670012115', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '5b4ca4e8-cecc-4163-b291-c9209b02c4b7', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.1810150146484375, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 18, 'up_thru': 255, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6818', 'nonce': 3601682336}, {'type': 'v1', 'addr': '172.21.15.70:6819', 'nonce': 3601682336}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6820', 'nonce': 3601682336}, {'type': 'v1', 'addr': '172.21.15.70:6821', 'nonce': 3601682336}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6824', 'nonce': 3601682336}, {'type': 'v1', 'addr': '172.21.15.70:6825', 'nonce': 3601682336}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6822', 'nonce': 3601682336}, {'type': 'v1', 'addr': '172.21.15.70:6823', 'nonce': 3601682336}]}, 'public_addr': '172.21.15.70:6819/3601682336', 'cluster_addr': '172.21.15.70:6821/3601682336', 'heartbeat_back_addr': '172.21.15.70:6825/3601682336', 'heartbeat_front_addr': '172.21.15.70:6823/3601682336', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'ffb8d527-1b58-4ecd-808c-47d4ba1bf387', 'up': 1, 'in': 1, 'weight': 0.95001220703125, 'primary_affinity': 0.476531982421875, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 26, 'up_thru': 283, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6826', 'nonce': 2038083007}, {'type': 'v1', 'addr': '172.21.15.70:6827', 'nonce': 2038083007}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6828', 'nonce': 2038083007}, {'type': 'v1', 'addr': '172.21.15.70:6829', 'nonce': 2038083007}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6832', 'nonce': 2038083007}, {'type': 'v1', 'addr': '172.21.15.70:6833', 'nonce': 2038083007}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6830', 'nonce': 2038083007}, {'type': 'v1', 'addr': '172.21.15.70:6831', 'nonce': 2038083007}]}, 'public_addr': '172.21.15.70:6827/2038083007', 'cluster_addr': '172.21.15.70:6829/2038083007', 'heartbeat_back_addr': '172.21.15.70:6833/2038083007', 'heartbeat_front_addr': '172.21.15.70:6831/2038083007', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'a946760b-4d91-4a4b-8e9b-a07a50589211', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 32, 'up_thru': 264, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6800', 'nonce': 2250892710}, {'type': 'v1', 'addr': '172.21.15.84:6801', 'nonce': 2250892710}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6802', 'nonce': 2250892710}, {'type': 'v1', 'addr': '172.21.15.84:6803', 'nonce': 2250892710}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6806', 'nonce': 2250892710}, {'type': 'v1', 'addr': '172.21.15.84:6807', 'nonce': 2250892710}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6804', 'nonce': 2250892710}, {'type': 'v1', 'addr': '172.21.15.84:6805', 'nonce': 2250892710}]}, 'public_addr': '172.21.15.84:6801/2250892710', 'cluster_addr': '172.21.15.84:6803/2250892710', 'heartbeat_back_addr': '172.21.15.84:6807/2250892710', 'heartbeat_front_addr': '172.21.15.84:6805/2250892710', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '67681760-75a9-4846-946e-a935d42c5e05', 'up': 1, 'in': 1, 'weight': 0.95001220703125, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 39, 'up_thru': 283, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6808', 'nonce': 1803945348}, {'type': 'v1', 'addr': '172.21.15.84:6809', 'nonce': 1803945348}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6810', 'nonce': 1803945348}, {'type': 'v1', 'addr': '172.21.15.84:6811', 'nonce': 1803945348}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6814', 'nonce': 1803945348}, {'type': 'v1', 'addr': '172.21.15.84:6815', 'nonce': 1803945348}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6812', 'nonce': 1803945348}, {'type': 'v1', 'addr': '172.21.15.84:6813', 'nonce': 1803945348}]}, 'public_addr': '172.21.15.84:6809/1803945348', 'cluster_addr': '172.21.15.84:6811/1803945348', 'heartbeat_back_addr': '172.21.15.84:6815/1803945348', 'heartbeat_front_addr': '172.21.15.84:6813/1803945348', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '7c8f0152-f539-4b7e-8113-91dd7529d3a1', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 46, 'up_thru': 282, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6816', 'nonce': 610727189}, {'type': 'v1', 'addr': '172.21.15.84:6817', 'nonce': 610727189}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6818', 'nonce': 610727189}, {'type': 'v1', 'addr': '172.21.15.84:6819', 'nonce': 610727189}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6822', 'nonce': 610727189}, {'type': 'v1', 'addr': '172.21.15.84:6823', 'nonce': 610727189}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6820', 'nonce': 610727189}, {'type': 'v1', 'addr': '172.21.15.84:6821', 'nonce': 610727189}]}, 'public_addr': '172.21.15.84:6817/610727189', 'cluster_addr': '172.21.15.84:6819/610727189', 'heartbeat_back_addr': '172.21.15.84:6823/610727189', 'heartbeat_front_addr': '172.21.15.84:6821/610727189', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '945efaba-225e-49af-aefd-b4d58ca8093b', 'up': 1, 'in': 1, 'weight': 0.95001220703125, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 53, 'up_thru': 285, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6824', 'nonce': 1002252531}, {'type': 'v1', 'addr': '172.21.15.84:6825', 'nonce': 1002252531}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6826', 'nonce': 1002252531}, {'type': 'v1', 'addr': '172.21.15.84:6827', 'nonce': 1002252531}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6830', 'nonce': 1002252531}, {'type': 'v1', 'addr': '172.21.15.84:6831', 'nonce': 1002252531}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6828', 'nonce': 1002252531}, {'type': 'v1', 'addr': '172.21.15.84:6829', 'nonce': 1002252531}]}, 'public_addr': '172.21.15.84:6825/1002252531', 'cluster_addr': '172.21.15.84:6827/1002252531', 'heartbeat_back_addr': '172.21.15.84:6831/1002252531', 'heartbeat_front_addr': '172.21.15.84:6829/1002252531', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T09:35:07.849981+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T09:34:40.462734+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T09:34:45.327630+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T09:34:55.598225+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T09:34:51.095682+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T09:35:00.308654+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T09:35:26.990283+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T09:35:15.569347+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.1d', 'osds': [1, 0, 7]}], 'pg_upmap_items': [{'pgid': '3.1b', 'mappings': [{'from': 0, 'to': 6}, {'from': 4, 'to': 7}]}, {'pgid': '4.f', 'mappings': [{'from': 1, 'to': 2}]}, {'pgid': '4.13', 'mappings': [{'from': 1, 'to': 5}]}], 'pg_upmap_primaries': [], 'pg_temp': [{'pgid': '9.10', 'osds': [0, 1]}, {'pgid': '9.12', 'osds': [0, 1]}, {'pgid': '9.13', 'osds': [7, 5, 6]}, {'pgid': '9.15', 'osds': [1, 3, 7]}, {'pgid': '9.16', 'osds': [1, 2, 4]}, {'pgid': '9.17', 'osds': [0, 7, 1]}, {'pgid': '9.18', 'osds': [5, 7]}, {'pgid': '9.19', 'osds': [3, 5, 6]}], 'primary_temp': [], 'blocklist': {'172.21.15.70:0/125491368': '2024-09-17T09:25:31.465167+0000', '172.21.15.70:0/3354284852': '2024-09-17T09:19:43.205989+0000', '172.21.15.70:0/3284625547': '2024-09-17T09:25:31.465167+0000', '172.21.15.70:0/186565817': '2024-09-17T09:19:43.205989+0000', '172.21.15.70:0/2213852865': '2024-09-17T09:19:43.205989+0000', '172.21.15.70:0/1812373760': '2024-09-17T09:20:09.306330+0000', '172.21.15.70:6800/837338094': '2024-09-17T09:19:43.205989+0000', '172.21.15.70:6800/1876661229': '2024-09-17T09:25:31.465167+0000', '172.21.15.70:0/3351688485': '2024-09-17T09:20:09.306330+0000', '172.21.15.70:6800/3871553358': '2024-09-17T09:20:09.306330+0000', '172.21.15.70:0/1230482871': '2024-09-17T09:25:31.465167+0000', '172.21.15.70:6801/837338094': '2024-09-17T09:19:43.205989+0000', '172.21.15.70:6801/3871553358': '2024-09-17T09:20:09.306330+0000', '172.21.15.70:0/1403001486': '2024-09-17T09:20:09.306330+0000', '172.21.15.70:0/1723643739': '2024-09-17T09:25:31.465167+0000', '172.21.15.70:6801/1876661229': '2024-09-17T09:25:31.465167+0000', '172.21.15.70:0/3456417396': '2024-09-17T09:25:31.465167+0000'}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-16T09:35:41.078 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-16T09:35:41.101 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:35:41.117 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:35:41.121 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:35:41.122 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:41.123 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:41.123 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:35:41.167 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:35:41.224 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:35:41.253 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:41.253 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:41.254 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:35:41.355 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:35:41.383 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:41.384 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:41.384 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:35:41.485 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:35:41.516 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:41.517 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:41.517 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:35:41.618 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:35:41.656 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:41.657 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:41.657 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:35:41.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:41 smithi070 ceph-mon[30388]: pgmap v525: 178 pgs: 1 active+recovering+undersized+remapped, 1 active+clean+scrubbing, 4 active+remapped+backfilling, 2 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_toofull, 169 active+clean; 5.6 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 983 B/s rd, 0 op/s; 141/275781 objects degraded (0.051%); 35068/275781 objects misplaced (12.716%); 16 MiB/s, 253 objects/s recovering 2024-09-16T09:35:41.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:41 smithi070 ceph-mon[30388]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull (PG_BACKFILL_FULL) 2024-09-16T09:35:41.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:41 smithi070 ceph-mon[40691]: pgmap v525: 178 pgs: 1 active+recovering+undersized+remapped, 1 active+clean+scrubbing, 4 active+remapped+backfilling, 2 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_toofull, 169 active+clean; 5.6 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 983 B/s rd, 0 op/s; 141/275781 objects degraded (0.051%); 35068/275781 objects misplaced (12.716%); 16 MiB/s, 253 objects/s recovering 2024-09-16T09:35:41.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:41 smithi070 ceph-mon[40691]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull (PG_BACKFILL_FULL) 2024-09-16T09:35:41.758 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:35:41.779 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:41.779 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:41.780 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:35:41.880 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:35:41.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:41 smithi084 ceph-mon[32629]: pgmap v525: 178 pgs: 1 active+recovering+undersized+remapped, 1 active+clean+scrubbing, 4 active+remapped+backfilling, 2 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_toofull, 169 active+clean; 5.6 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 983 B/s rd, 0 op/s; 141/275781 objects degraded (0.051%); 35068/275781 objects misplaced (12.716%); 16 MiB/s, 253 objects/s recovering 2024-09-16T09:35:41.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:41 smithi084 ceph-mon[32629]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull (PG_BACKFILL_FULL) 2024-09-16T09:35:41.915 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:41.915 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:41.916 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:35:42.016 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:35:42.047 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:42.048 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:42.048 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:35:42.149 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:35:42.171 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:35:41 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:35:41] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:35:42.178 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:42.178 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:42.179 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:35:42.279 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:35:42.308 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:42.309 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:42.309 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:35:42.410 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:35:42.452 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:42.453 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:42.453 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:35:42.554 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:35:42.583 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:42.584 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:42.584 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:35:42.589 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:42 smithi070 ceph-mon[30388]: 1.e scrub starts 2024-09-16T09:35:42.589 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:42 smithi070 ceph-mon[30388]: 1.e scrub ok 2024-09-16T09:35:42.589 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:42 smithi070 ceph-mon[40691]: 1.e scrub starts 2024-09-16T09:35:42.589 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:42 smithi070 ceph-mon[40691]: 1.e scrub ok 2024-09-16T09:35:42.685 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:35:42.727 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:42.727 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:42.727 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:35:42.828 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:35:42.856 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:42.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:42.857 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:35:42.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:42 smithi084 ceph-mon[32629]: 1.e scrub starts 2024-09-16T09:35:42.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:42 smithi084 ceph-mon[32629]: 1.e scrub ok 2024-09-16T09:35:42.958 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:35:42.979 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:42.979 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:42.980 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:35:43.080 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:35:43.107 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:43.107 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:43.108 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:35:43.209 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:35:43.236 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:43.237 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:43.237 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:35:43.338 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:35:43.366 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:43.367 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:43.367 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:35:43.410 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:35:43 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:35:43.468 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:35:43.520 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:43.524 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:43.525 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:35:43.625 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:35:43.655 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:43.656 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:43.656 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:35:43.757 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:35:43.771 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:43 smithi070 ceph-mon[30388]: pgmap v526: 178 pgs: 1 active+recovering+undersized+remapped, 1 active+clean+scrubbing, 4 active+remapped+backfilling, 2 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_toofull, 169 active+clean; 5.6 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 852 B/s rd, 0 op/s; 141/275781 objects degraded (0.051%); 35068/275781 objects misplaced (12.716%); 14 MiB/s, 220 objects/s recovering 2024-09-16T09:35:43.771 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:43 smithi070 ceph-mon[40691]: pgmap v526: 178 pgs: 1 active+recovering+undersized+remapped, 1 active+clean+scrubbing, 4 active+remapped+backfilling, 2 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_toofull, 169 active+clean; 5.6 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 852 B/s rd, 0 op/s; 141/275781 objects degraded (0.051%); 35068/275781 objects misplaced (12.716%); 14 MiB/s, 220 objects/s recovering 2024-09-16T09:35:43.784 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:43.785 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:43.785 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:35:43.886 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:35:43.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:43 smithi084 ceph-mon[32629]: pgmap v526: 178 pgs: 1 active+recovering+undersized+remapped, 1 active+clean+scrubbing, 4 active+remapped+backfilling, 2 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_toofull, 169 active+clean; 5.6 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 852 B/s rd, 0 op/s; 141/275781 objects degraded (0.051%); 35068/275781 objects misplaced (12.716%); 14 MiB/s, 220 objects/s recovering 2024-09-16T09:35:43.910 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:43.911 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:43.911 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:35:44.012 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:35:44.040 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:44.041 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:44.041 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:35:44.142 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:35:44.167 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:44.168 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:44.168 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:35:44.269 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:35:44.295 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:44.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:44.296 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:35:44.396 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:35:44.427 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:44.428 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:44.428 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:35:44.529 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:35:44.565 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:44.565 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:44.566 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:35:44.667 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:35:44.693 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:44.694 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:44.694 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:35:44.795 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:35:44.828 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:44 smithi070 ceph-mon[30388]: 9.15 scrub starts 2024-09-16T09:35:44.828 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:44 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:35:44.828 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:44 smithi070 ceph-mon[30388]: osdmap e292: 8 total, 8 up, 8 in 2024-09-16T09:35:44.828 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:44 smithi070 ceph-mon[30388]: 4.1 scrub starts 2024-09-16T09:35:44.828 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:44 smithi070 ceph-mon[30388]: 4.1 scrub ok 2024-09-16T09:35:44.828 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:44 smithi070 ceph-mon[30388]: Health check update: Degraded data redundancy: 141/275781 objects degraded (0.051%), 2 pgs degraded (PG_DEGRADED) 2024-09-16T09:35:44.828 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:44 smithi070 ceph-mon[30388]: osdmap e293: 8 total, 8 up, 8 in 2024-09-16T09:35:44.829 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:44 smithi070 ceph-mon[40691]: 9.15 scrub starts 2024-09-16T09:35:44.829 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:44 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:35:44.829 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:44 smithi070 ceph-mon[40691]: osdmap e292: 8 total, 8 up, 8 in 2024-09-16T09:35:44.829 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:44 smithi070 ceph-mon[40691]: 4.1 scrub starts 2024-09-16T09:35:44.829 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:44 smithi070 ceph-mon[40691]: 4.1 scrub ok 2024-09-16T09:35:44.829 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:44 smithi070 ceph-mon[40691]: Health check update: Degraded data redundancy: 141/275781 objects degraded (0.051%), 2 pgs degraded (PG_DEGRADED) 2024-09-16T09:35:44.829 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:44 smithi070 ceph-mon[40691]: osdmap e293: 8 total, 8 up, 8 in 2024-09-16T09:35:44.830 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:44.831 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:44.831 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:35:44.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:44 smithi084 ceph-mon[32629]: 9.15 scrub starts 2024-09-16T09:35:44.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:44 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:35:44.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:44 smithi084 ceph-mon[32629]: osdmap e292: 8 total, 8 up, 8 in 2024-09-16T09:35:44.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:44 smithi084 ceph-mon[32629]: 4.1 scrub starts 2024-09-16T09:35:44.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:44 smithi084 ceph-mon[32629]: 4.1 scrub ok 2024-09-16T09:35:44.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:44 smithi084 ceph-mon[32629]: Health check update: Degraded data redundancy: 141/275781 objects degraded (0.051%), 2 pgs degraded (PG_DEGRADED) 2024-09-16T09:35:44.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:44 smithi084 ceph-mon[32629]: osdmap e293: 8 total, 8 up, 8 in 2024-09-16T09:35:44.932 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:35:44.966 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:44.966 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:44.967 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:35:45.068 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:35:45.103 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:45.103 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:45.104 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:35:45.205 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:35:45.231 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:45.232 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:45.232 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:35:45.333 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:35:45.359 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:45.359 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:45.360 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:35:45.461 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:35:45.506 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:45.506 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:45.506 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:35:45.607 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:35:45.634 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:45.634 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:45.635 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:35:45.735 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:35:45.781 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:45.781 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:45.782 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:35:45.841 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:45 smithi070 ceph-mon[30388]: pgmap v528: 178 pgs: 1 active+clean+remapped+scrubbing, 1 active+recovering+undersized+remapped, 3 active+remapped+backfilling, 2 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_toofull, 170 active+clean; 5.4 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s; 141/267504 objects degraded (0.053%); 31606/267504 objects misplaced (11.815%); 23 MiB/s, 370 objects/s recovering 2024-09-16T09:35:45.841 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:45 smithi070 ceph-mon[30388]: osdmap e294: 8 total, 8 up, 8 in 2024-09-16T09:35:45.841 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:45 smithi070 ceph-mon[40691]: pgmap v528: 178 pgs: 1 active+clean+remapped+scrubbing, 1 active+recovering+undersized+remapped, 3 active+remapped+backfilling, 2 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_toofull, 170 active+clean; 5.4 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s; 141/267504 objects degraded (0.053%); 31606/267504 objects misplaced (11.815%); 23 MiB/s, 370 objects/s recovering 2024-09-16T09:35:45.841 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:45 smithi070 ceph-mon[40691]: osdmap e294: 8 total, 8 up, 8 in 2024-09-16T09:35:45.883 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:35:45.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:45 smithi084 ceph-mon[32629]: pgmap v528: 178 pgs: 1 active+clean+remapped+scrubbing, 1 active+recovering+undersized+remapped, 3 active+remapped+backfilling, 2 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_toofull, 170 active+clean; 5.4 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s; 141/267504 objects degraded (0.053%); 31606/267504 objects misplaced (11.815%); 23 MiB/s, 370 objects/s recovering 2024-09-16T09:35:45.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:45 smithi084 ceph-mon[32629]: osdmap e294: 8 total, 8 up, 8 in 2024-09-16T09:35:45.914 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:45.915 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:45.915 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:35:46.016 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:35:46.061 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:46.062 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:46.062 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:35:46.163 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:35:46.191 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:46.192 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:46.193 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:35:46.294 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:35:46.321 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:46.322 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:46.322 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:35:46.423 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:35:46.463 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:46.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:46.464 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:35:46.565 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:35:46.592 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:46.592 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:46.593 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:35:46.693 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:35:46.723 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:46.723 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:46.724 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:35:46.825 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:35:46.868 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:46.868 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:46.869 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:35:46.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:46 smithi084 ceph-mon[32629]: 9.15 scrub starts 2024-09-16T09:35:46.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:46 smithi084 ceph-mon[32629]: 9.17 scrub starts 2024-09-16T09:35:46.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:46 smithi070 ceph-mon[30388]: 9.15 scrub starts 2024-09-16T09:35:46.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:46 smithi070 ceph-mon[30388]: 9.17 scrub starts 2024-09-16T09:35:46.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:46 smithi070 ceph-mon[40691]: 9.15 scrub starts 2024-09-16T09:35:46.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:46 smithi070 ceph-mon[40691]: 9.17 scrub starts 2024-09-16T09:35:46.970 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:35:47.001 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:47.001 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:47.002 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:35:47.102 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:35:47.131 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:47.131 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:47.131 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:35:47.232 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:35:47.271 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:47.272 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:47.272 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:35:47.373 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:35:47.407 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:47.407 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:47.408 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:35:47.509 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:35:47.530 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:47.530 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:47.531 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:35:47.631 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:35:47.672 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:47.673 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:47.673 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:35:47.774 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:35:47.796 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:47.796 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:47.796 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:35:47.897 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:35:47.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:47 smithi084 ceph-mon[32629]: pgmap v531: 178 pgs: 1 active+clean+remapped, 1 peering, 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 2 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_toofull, 170 active+clean; 5.4 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 852 B/s rd, 0 op/s; 157/264939 objects degraded (0.059%); 30048/264939 objects misplaced (11.341%); 25 MiB/s, 403 objects/s recovering 2024-09-16T09:35:47.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:47 smithi084 ceph-mon[32629]: 9.15 scrub ok 2024-09-16T09:35:47.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:47 smithi084 ceph-mon[32629]: 9.17 scrub ok 2024-09-16T09:35:47.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:47 smithi084 ceph-mon[32629]: 6.2 scrub starts 2024-09-16T09:35:47.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:47 smithi084 ceph-mon[32629]: 6.2 scrub ok 2024-09-16T09:35:47.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:47 smithi070 ceph-mon[30388]: pgmap v531: 178 pgs: 1 active+clean+remapped, 1 peering, 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 2 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_toofull, 170 active+clean; 5.4 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 852 B/s rd, 0 op/s; 157/264939 objects degraded (0.059%); 30048/264939 objects misplaced (11.341%); 25 MiB/s, 403 objects/s recovering 2024-09-16T09:35:47.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:47 smithi070 ceph-mon[30388]: 9.15 scrub ok 2024-09-16T09:35:47.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:47 smithi070 ceph-mon[30388]: 9.17 scrub ok 2024-09-16T09:35:47.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:47 smithi070 ceph-mon[30388]: 6.2 scrub starts 2024-09-16T09:35:47.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:47 smithi070 ceph-mon[30388]: 6.2 scrub ok 2024-09-16T09:35:47.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:47 smithi070 ceph-mon[40691]: pgmap v531: 178 pgs: 1 active+clean+remapped, 1 peering, 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 2 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_toofull, 170 active+clean; 5.4 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 852 B/s rd, 0 op/s; 157/264939 objects degraded (0.059%); 30048/264939 objects misplaced (11.341%); 25 MiB/s, 403 objects/s recovering 2024-09-16T09:35:47.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:47 smithi070 ceph-mon[40691]: 9.15 scrub ok 2024-09-16T09:35:47.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:47 smithi070 ceph-mon[40691]: 9.17 scrub ok 2024-09-16T09:35:47.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:47 smithi070 ceph-mon[40691]: 6.2 scrub starts 2024-09-16T09:35:47.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:47 smithi070 ceph-mon[40691]: 6.2 scrub ok 2024-09-16T09:35:47.929 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:47.929 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:47.930 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:35:48.031 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:35:48.060 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:48.060 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:48.061 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:35:48.162 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:35:48.202 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:48.203 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:48.203 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:35:48.304 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:35:48.332 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:48.332 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:48.333 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:35:48.433 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:35:48.468 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:48.469 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:48.469 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:35:48.570 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:35:48.616 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:48.617 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:48.617 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:35:48.718 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:35:48.752 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:48.753 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:48.753 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:35:48.854 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:35:48.866 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:48 smithi070 ceph-mon[30388]: pgmap v532: 178 pgs: 1 active+clean+remapped, 1 peering, 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 2 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_toofull, 170 active+clean; 5.4 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 157/263829 objects degraded (0.060%); 30048/263829 objects misplaced (11.389%); 25 MiB/s, 403 objects/s recovering 2024-09-16T09:35:48.867 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:48 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:35:48.867 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:48 smithi070 ceph-mon[40691]: pgmap v532: 178 pgs: 1 active+clean+remapped, 1 peering, 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 2 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_toofull, 170 active+clean; 5.4 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 157/263829 objects degraded (0.060%); 30048/263829 objects misplaced (11.389%); 25 MiB/s, 403 objects/s recovering 2024-09-16T09:35:48.867 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:48 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:35:48.894 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:48.894 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:48.895 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:35:48.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:48 smithi084 ceph-mon[32629]: pgmap v532: 178 pgs: 1 active+clean+remapped, 1 peering, 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 2 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_toofull, 170 active+clean; 5.4 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 157/263829 objects degraded (0.060%); 30048/263829 objects misplaced (11.389%); 25 MiB/s, 403 objects/s recovering 2024-09-16T09:35:48.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:48 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:35:48.995 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:35:49.037 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:49.038 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:49.038 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:35:49.139 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:35:49.170 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:49.171 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:49.171 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:35:49.272 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:35:49.303 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:49.304 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:49.304 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:35:49.405 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:35:49.443 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:49.444 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:49.444 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:35:49.545 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:35:49.589 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:49.590 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:49.590 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:35:49.691 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:35:49.726 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:49.726 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:49.726 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:35:49.764 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:49 smithi070 ceph-mon[30388]: Health check update: Degraded data redundancy: 157/263829 objects degraded (0.060%), 2 pgs degraded (PG_DEGRADED) 2024-09-16T09:35:49.764 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:49 smithi070 ceph-mon[30388]: osdmap e295: 8 total, 8 up, 8 in 2024-09-16T09:35:49.764 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:49 smithi070 ceph-mon[40691]: Health check update: Degraded data redundancy: 157/263829 objects degraded (0.060%), 2 pgs degraded (PG_DEGRADED) 2024-09-16T09:35:49.765 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:49 smithi070 ceph-mon[40691]: osdmap e295: 8 total, 8 up, 8 in 2024-09-16T09:35:49.827 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:35:49.849 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:49.849 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:49.849 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:35:49.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:49 smithi084 ceph-mon[32629]: Health check update: Degraded data redundancy: 157/263829 objects degraded (0.060%), 2 pgs degraded (PG_DEGRADED) 2024-09-16T09:35:49.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:49 smithi084 ceph-mon[32629]: osdmap e295: 8 total, 8 up, 8 in 2024-09-16T09:35:49.950 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:35:49.988 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:49.988 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:49.989 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:35:50.090 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:35:50.190 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:50.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:50.191 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:35:50.292 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:35:50.321 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:50.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:50.322 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:35:50.422 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:35:50.468 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:50.469 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:50.470 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:35:50.570 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:35:50.612 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:50.612 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:50.613 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:35:50.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:50 smithi070 ceph-mon[30388]: pgmap v534: 178 pgs: 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 2 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_toofull, 172 active+clean; 5.1 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 852 B/s rd, 0 op/s; 157/253656 objects degraded (0.062%); 26388/253656 objects misplaced (10.403%) 2024-09-16T09:35:50.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:50 smithi070 ceph-mon[40691]: pgmap v534: 178 pgs: 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 2 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_toofull, 172 active+clean; 5.1 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 852 B/s rd, 0 op/s; 157/253656 objects degraded (0.062%); 26388/253656 objects misplaced (10.403%) 2024-09-16T09:35:50.714 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:35:50.738 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:50.738 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:50.739 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:35:50.839 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:35:50.868 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:50.868 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:50.869 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:35:50.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:50 smithi084 ceph-mon[32629]: pgmap v534: 178 pgs: 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 2 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_toofull, 172 active+clean; 5.1 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 852 B/s rd, 0 op/s; 157/253656 objects degraded (0.062%); 26388/253656 objects misplaced (10.403%) 2024-09-16T09:35:50.969 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:35:51.006 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:51.007 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:51.008 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:35:51.108 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:35:51.151 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:51.152 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:51.152 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:35:51.163 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T09:35:51.163 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:35:51.253 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:35:51.292 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:51.293 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:51.293 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:35:51.394 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:35:51.422 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:51.423 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:51.423 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:35:51.524 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:35:51.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:51.568 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:51.568 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:35:51.669 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:35:51.699 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:51.699 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:51.700 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:35:51.800 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:35:51.811 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:51 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/435519731' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:35:51.837 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:51.838 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:51.838 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:35:51.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:51 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/435519731' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:35:51.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:51 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/435519731' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:35:51.939 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:35:51.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:51.982 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:51.982 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:35:52.083 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:35:52.124 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:52.125 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:52.125 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:35:52.226 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:35:52.258 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:52.259 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:52.259 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:35:52.360 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:35:52.395 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:52.396 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:52.396 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:35:52.420 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:35:51 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:35:51] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:35:52.497 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:35:52.539 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:52.540 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:52.540 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:35:52.641 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:35:52.664 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:52.664 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:52.665 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:35:52.765 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:35:52.796 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:52.797 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:52.797 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:35:52.837 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:52 smithi084 ceph-mon[32629]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:35:52.837 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:52 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/435519731' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-16T09:35:52.837 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:52 smithi084 ceph-mon[32629]: osdmap e296: 8 total, 8 up, 8 in 2024-09-16T09:35:52.837 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:52 smithi084 ceph-mon[32629]: pgmap v536: 178 pgs: 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 2 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_toofull, 172 active+clean; 5.1 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 796 B/s rd, 0 op/s; 157/253656 objects degraded (0.062%); 26388/253656 objects misplaced (10.403%) 2024-09-16T09:35:52.898 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:35:52.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:52 smithi070 ceph-mon[30388]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:35:52.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:52 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/435519731' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-16T09:35:52.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:52 smithi070 ceph-mon[30388]: osdmap e296: 8 total, 8 up, 8 in 2024-09-16T09:35:52.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:52 smithi070 ceph-mon[30388]: pgmap v536: 178 pgs: 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 2 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_toofull, 172 active+clean; 5.1 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 796 B/s rd, 0 op/s; 157/253656 objects degraded (0.062%); 26388/253656 objects misplaced (10.403%) 2024-09-16T09:35:52.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:52 smithi070 ceph-mon[40691]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:35:52.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:52 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/435519731' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-16T09:35:52.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:52 smithi070 ceph-mon[40691]: osdmap e296: 8 total, 8 up, 8 in 2024-09-16T09:35:52.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:52 smithi070 ceph-mon[40691]: pgmap v536: 178 pgs: 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 2 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_toofull, 172 active+clean; 5.1 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 796 B/s rd, 0 op/s; 157/253656 objects degraded (0.062%); 26388/253656 objects misplaced (10.403%) 2024-09-16T09:35:52.938 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:52.939 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:52.939 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:35:53.040 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:35:53.073 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:53.073 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:53.074 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:35:53.175 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:35:53.205 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:53.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:53.206 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:35:53.307 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:35:53.328 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:53.329 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:53.329 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:35:53.430 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:35:53.475 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:53.476 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:53.476 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:35:53.544 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:53 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:35:53.545 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:53 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/816873316' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:35:53.545 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:35:53 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:35:53.576 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:35:53.628 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:53.628 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:53.628 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:35:53.729 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:35:53.774 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:53.774 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:53.775 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:35:53.875 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:35:53.913 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:53.914 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:53.914 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:35:53.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:53 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:35:53.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:53 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/816873316' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:35:53.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:53 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:35:53.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:53 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/816873316' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:35:53.926 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:35:53.926 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:35:53.926 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:35:53.926 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:35:53.926 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:35:53.927 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:35:53.927 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:35:53.927 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:35:53.927 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:35:53.927 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:35:53.927 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:35:53.927 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:35:53.927 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:35:53.928 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:35:53.928 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:35:53.928 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:35:54.015 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:35:54.047 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:54.048 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:54.048 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:35:54.149 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:35:54.180 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:54.181 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:54.181 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:35:54.282 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:35:54.318 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:54.319 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:54.319 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:35:54.419 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:35:54.454 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:54.454 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:54.455 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:35:54.555 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:35:54.576 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:54.577 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:54.577 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:35:54.678 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:35:54.710 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:54.711 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:54.711 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:35:54.812 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:35:54.836 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:54.837 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:54.837 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:35:54.869 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:54 smithi084 ceph-mon[32629]: pgmap v537: 178 pgs: 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 2 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_toofull, 172 active+clean; 4.9 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 299 op/s; 157/243060 objects degraded (0.065%); 23871/243060 objects misplaced (9.821%) 2024-09-16T09:35:54.869 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:54 smithi084 ceph-mon[32629]: Health check update: Degraded data redundancy: 157/253656 objects degraded (0.062%), 2 pgs degraded (PG_DEGRADED) 2024-09-16T09:35:54.869 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:54 smithi084 ceph-mon[32629]: osdmap e297: 8 total, 8 up, 8 in 2024-09-16T09:35:54.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:54 smithi070 ceph-mon[30388]: pgmap v537: 178 pgs: 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 2 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_toofull, 172 active+clean; 4.9 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 299 op/s; 157/243060 objects degraded (0.065%); 23871/243060 objects misplaced (9.821%) 2024-09-16T09:35:54.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:54 smithi070 ceph-mon[30388]: Health check update: Degraded data redundancy: 157/253656 objects degraded (0.062%), 2 pgs degraded (PG_DEGRADED) 2024-09-16T09:35:54.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:54 smithi070 ceph-mon[30388]: osdmap e297: 8 total, 8 up, 8 in 2024-09-16T09:35:54.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:54 smithi070 ceph-mon[40691]: pgmap v537: 178 pgs: 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 2 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_toofull, 172 active+clean; 4.9 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 299 op/s; 157/243060 objects degraded (0.065%); 23871/243060 objects misplaced (9.821%) 2024-09-16T09:35:54.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:54 smithi070 ceph-mon[40691]: Health check update: Degraded data redundancy: 157/253656 objects degraded (0.062%), 2 pgs degraded (PG_DEGRADED) 2024-09-16T09:35:54.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:54 smithi070 ceph-mon[40691]: osdmap e297: 8 total, 8 up, 8 in 2024-09-16T09:35:54.938 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:35:54.975 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:54.975 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:54.976 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:35:55.076 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:35:55.112 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:55.113 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:55.113 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:35:55.214 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:35:55.258 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:55.259 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:55.260 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:35:55.360 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:35:55.389 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:55.390 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:55.390 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:35:55.491 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:35:55.529 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:55.530 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:55.531 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:35:55.633 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:35:55.654 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:55.655 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:55.655 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:35:55.756 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:35:55.786 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:55.786 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:55.787 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:35:55.887 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:35:55.915 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:55.916 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:55.916 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:35:56.017 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:35:56.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:56.058 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:56.058 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:35:56.159 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:35:56.192 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:56.192 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:56.193 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:35:56.293 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:35:56.322 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:56.323 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:56.323 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:35:56.424 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:35:56.457 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:56.457 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:56.457 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:35:56.558 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:35:56.601 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:56.601 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:56.602 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:35:56.702 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:35:56.729 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:56.730 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:56.730 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:35:56.831 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:35:56.868 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:56.868 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:56.869 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:35:56.970 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:35:56.995 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:56.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:56.997 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:35:57.097 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:35:57.130 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:57.131 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:57.131 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:35:57.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:56 smithi084 ceph-mon[32629]: pgmap v539: 178 pgs: 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 2 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_toofull, 172 active+clean; 4.9 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 796 B/s rd, 0 B/s wr, 726 op/s; 178/239670 objects degraded (0.074%); 23163/239670 objects misplaced (9.665%); 16 MiB/s, 251 objects/s recovering 2024-09-16T09:35:57.170 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:56 smithi070 ceph-mon[30388]: pgmap v539: 178 pgs: 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 2 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_toofull, 172 active+clean; 4.9 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 796 B/s rd, 0 B/s wr, 726 op/s; 178/239670 objects degraded (0.074%); 23163/239670 objects misplaced (9.665%); 16 MiB/s, 251 objects/s recovering 2024-09-16T09:35:57.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:56 smithi070 ceph-mon[40691]: pgmap v539: 178 pgs: 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 2 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_toofull, 172 active+clean; 4.9 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 796 B/s rd, 0 B/s wr, 726 op/s; 178/239670 objects degraded (0.074%); 23163/239670 objects misplaced (9.665%); 16 MiB/s, 251 objects/s recovering 2024-09-16T09:35:57.232 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:35:57.257 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:57.258 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:57.258 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:35:57.359 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:35:57.381 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:57.382 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:57.383 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:35:57.483 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:35:57.505 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:57.506 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:57.506 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:35:57.607 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:35:57.629 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:57.629 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:57.630 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:35:57.730 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:35:57.760 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:57.761 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:57.761 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:35:57.862 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:35:57.899 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:57.900 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:57.900 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:35:58.001 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:35:58.032 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:58.033 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:58.033 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:35:58.134 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:35:58.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:57 smithi084 ceph-mon[32629]: osdmap e298: 8 total, 8 up, 8 in 2024-09-16T09:35:58.164 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:58.165 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:58.165 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:35:58.170 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:57 smithi070 ceph-mon[30388]: osdmap e298: 8 total, 8 up, 8 in 2024-09-16T09:35:58.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:57 smithi070 ceph-mon[40691]: osdmap e298: 8 total, 8 up, 8 in 2024-09-16T09:35:58.265 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:35:58.299 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:58.299 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:58.300 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:35:58.401 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:35:58.443 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:58.443 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:58.444 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:35:58.484 INFO:teuthology.orchestra.run.smithi070.stderr:noscrub is set 2024-09-16T09:35:58.545 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:35:58.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:58.568 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:58.568 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:35:58.669 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:35:58.707 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:58.708 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:58.708 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:35:58.809 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:35:58.859 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:58.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:58.861 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:35:58.961 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:35:58.997 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:58.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:58.998 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:35:59.099 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:35:59.139 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:59.139 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:59.140 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:35:59.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:58 smithi084 ceph-mon[32629]: pgmap v541: 178 pgs: 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 2 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_toofull, 172 active+clean; 4.8 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 829 B/s rd, 0 B/s wr, 814 op/s; 178/238590 objects degraded (0.075%); 23163/238590 objects misplaced (9.708%); 16 MiB/s, 262 objects/s recovering 2024-09-16T09:35:59.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:58 smithi084 ceph-mon[32629]: osdmap e299: 8 total, 8 up, 8 in 2024-09-16T09:35:59.170 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:58 smithi070 ceph-mon[30388]: pgmap v541: 178 pgs: 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 2 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_toofull, 172 active+clean; 4.8 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 829 B/s rd, 0 B/s wr, 814 op/s; 178/238590 objects degraded (0.075%); 23163/238590 objects misplaced (9.708%); 16 MiB/s, 262 objects/s recovering 2024-09-16T09:35:59.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:58 smithi070 ceph-mon[30388]: osdmap e299: 8 total, 8 up, 8 in 2024-09-16T09:35:59.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:58 smithi070 ceph-mon[40691]: pgmap v541: 178 pgs: 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 2 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_toofull, 172 active+clean; 4.8 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 829 B/s rd, 0 B/s wr, 814 op/s; 178/238590 objects degraded (0.075%); 23163/238590 objects misplaced (9.708%); 16 MiB/s, 262 objects/s recovering 2024-09-16T09:35:59.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:58 smithi070 ceph-mon[40691]: osdmap e299: 8 total, 8 up, 8 in 2024-09-16T09:35:59.240 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:35:59.272 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:59.272 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:59.273 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:35:59.374 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:35:59.400 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:59.401 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:59.402 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:35:59.503 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:35:59.534 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:59.535 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:35:59.535 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:35:59.636 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:35:59.674 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:59.675 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:59.676 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:35:59.777 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:35:59.819 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:59.819 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:59.820 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:35:59.921 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:35:59.961 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:35:59.962 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:35:59.962 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:36:00.063 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:36:00.092 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:00.093 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:00.093 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:36:00.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:35:59 smithi084 ceph-mon[32629]: Health check update: Degraded data redundancy: 178/238590 objects degraded (0.075%), 2 pgs degraded (PG_DEGRADED) 2024-09-16T09:36:00.170 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:35:59 smithi070 ceph-mon[30388]: Health check update: Degraded data redundancy: 178/238590 objects degraded (0.075%), 2 pgs degraded (PG_DEGRADED) 2024-09-16T09:36:00.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:35:59 smithi070 ceph-mon[40691]: Health check update: Degraded data redundancy: 178/238590 objects degraded (0.075%), 2 pgs degraded (PG_DEGRADED) 2024-09-16T09:36:00.194 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:36:00.230 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:00.231 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:00.231 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:36:00.332 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:36:00.376 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:00.377 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:00.377 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:36:00.478 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:36:00.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:00.510 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:00.510 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:36:00.611 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:36:00.647 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:00.648 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:00.648 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:36:00.749 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:36:00.790 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:00.790 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:00.790 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:36:00.891 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:36:00.920 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:00.920 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:00.921 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:36:01.021 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:36:01.057 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:01.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:01.058 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:36:01.159 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:36:01.180 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:01.181 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:01.182 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:36:01.282 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:36:01.327 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:01.327 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:01.328 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:36:01.429 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:36:01.472 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:01.473 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:01.473 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:36:01.574 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:36:01.606 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:01.607 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:01.607 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:36:01.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:01 smithi084 ceph-mon[32629]: pgmap v543: 178 pgs: 1 active+remapped+backfilling, 1 active+recovery_wait, 1 active+recovering+undersized+remapped, 2 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_toofull, 172 active+clean; 4.6 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 860 op/s; 196/227574 objects degraded (0.086%); 19742/227574 objects misplaced (8.675%); 20 MiB/s, 316 objects/s recovering 2024-09-16T09:36:01.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:01 smithi070 ceph-mon[30388]: pgmap v543: 178 pgs: 1 active+remapped+backfilling, 1 active+recovery_wait, 1 active+recovering+undersized+remapped, 2 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_toofull, 172 active+clean; 4.6 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 860 op/s; 196/227574 objects degraded (0.086%); 19742/227574 objects misplaced (8.675%); 20 MiB/s, 316 objects/s recovering 2024-09-16T09:36:01.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:01 smithi070 ceph-mon[40691]: pgmap v543: 178 pgs: 1 active+remapped+backfilling, 1 active+recovery_wait, 1 active+recovering+undersized+remapped, 2 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_toofull, 172 active+clean; 4.6 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 860 op/s; 196/227574 objects degraded (0.086%); 19742/227574 objects misplaced (8.675%); 20 MiB/s, 316 objects/s recovering 2024-09-16T09:36:01.708 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:36:01.745 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:01.745 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:01.746 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:36:01.847 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:36:01.871 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:01.871 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:01.872 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:36:01.973 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:36:01.998 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:01.998 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:01.998 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:36:02.099 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:36:02.121 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:02.122 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:02.122 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:36:02.223 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:36:02.244 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:02.245 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:02.245 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:36:02.307 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:36:01 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:36:01] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:36:02.346 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:36:02.374 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:02.377 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:02.377 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:36:02.479 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:36:02.514 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:02.514 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:02.515 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:36:02.615 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:36:02.642 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:02.643 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:02.643 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:36:02.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:02 smithi084 ceph-mon[32629]: osdmap e300: 8 total, 8 up, 8 in 2024-09-16T09:36:02.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:02 smithi070 ceph-mon[30388]: osdmap e300: 8 total, 8 up, 8 in 2024-09-16T09:36:02.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:02 smithi070 ceph-mon[40691]: osdmap e300: 8 total, 8 up, 8 in 2024-09-16T09:36:02.744 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:36:02.779 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:02.779 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:02.780 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:36:02.881 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:36:02.923 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:02.923 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:02.924 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:36:03.025 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:36:03.055 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:03.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:03.057 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:36:03.157 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:36:03.190 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:03.191 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:03.191 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:36:03.292 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:36:03.351 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:03.351 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:03.352 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:36:03.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:03 smithi070 ceph-mon[30388]: pgmap v545: 178 pgs: 1 active+remapped+backfilling, 1 active+recovery_wait, 1 active+recovering+undersized+remapped, 2 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_toofull, 172 active+clean; 4.6 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 672 op/s; 196/227574 objects degraded (0.086%); 19742/227574 objects misplaced (8.675%); 17 MiB/s, 271 objects/s recovering 2024-09-16T09:36:03.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:03 smithi070 ceph-mon[30388]: osdmap e301: 8 total, 8 up, 8 in 2024-09-16T09:36:03.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:03 smithi070 ceph-mon[30388]: 9.19 scrub starts 2024-09-16T09:36:03.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:03 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:36:03.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:03 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:36:03.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:03 smithi070 ceph-mon[40691]: pgmap v545: 178 pgs: 1 active+remapped+backfilling, 1 active+recovery_wait, 1 active+recovering+undersized+remapped, 2 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_toofull, 172 active+clean; 4.6 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 672 op/s; 196/227574 objects degraded (0.086%); 19742/227574 objects misplaced (8.675%); 17 MiB/s, 271 objects/s recovering 2024-09-16T09:36:03.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:03 smithi070 ceph-mon[40691]: osdmap e301: 8 total, 8 up, 8 in 2024-09-16T09:36:03.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:03 smithi070 ceph-mon[40691]: 9.19 scrub starts 2024-09-16T09:36:03.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:03 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:36:03.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:03 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:36:03.453 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:36:03.482 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:03.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:03.483 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:36:03.584 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:36:03.618 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:03.619 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:03.619 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:36:03.660 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:36:03 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:36:03.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:03 smithi084 ceph-mon[32629]: pgmap v545: 178 pgs: 1 active+remapped+backfilling, 1 active+recovery_wait, 1 active+recovering+undersized+remapped, 2 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_toofull, 172 active+clean; 4.6 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 672 op/s; 196/227574 objects degraded (0.086%); 19742/227574 objects misplaced (8.675%); 17 MiB/s, 271 objects/s recovering 2024-09-16T09:36:03.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:03 smithi084 ceph-mon[32629]: osdmap e301: 8 total, 8 up, 8 in 2024-09-16T09:36:03.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:03 smithi084 ceph-mon[32629]: 9.19 scrub starts 2024-09-16T09:36:03.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:03 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:36:03.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:03 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:36:03.720 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:36:03.750 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:03.750 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:03.751 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:36:03.852 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:36:03.887 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:03.888 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:03.888 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:36:03.989 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:36:04.030 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:04.031 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:04.031 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:36:04.133 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:36:04.163 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:04.164 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:04.164 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:36:04.252 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.0 dump_blocked_ops 2024-09-16T09:36:04.265 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:36:04.288 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:04.289 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:04.289 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:36:04.390 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:36:04.417 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:04.418 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:04.418 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:36:04.519 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:36:04.559 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:04.560 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:04.560 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:36:04.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:04 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:36:04.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:04 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:36:04.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:04 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:36:04.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:04 smithi084 ceph-mon[32629]: osdmap e302: 8 total, 8 up, 8 in 2024-09-16T09:36:04.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:04 smithi084 ceph-mon[32629]: Health check update: Degraded data redundancy: 196/227574 objects degraded (0.086%), 2 pgs degraded (PG_DEGRADED) 2024-09-16T09:36:04.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:04 smithi084 ceph-mon[32629]: osdmap e303: 8 total, 8 up, 8 in 2024-09-16T09:36:04.661 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:36:04.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:04 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:36:04.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:04 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:36:04.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:04 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:36:04.673 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:04 smithi070 ceph-mon[30388]: osdmap e302: 8 total, 8 up, 8 in 2024-09-16T09:36:04.673 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:04 smithi070 ceph-mon[30388]: Health check update: Degraded data redundancy: 196/227574 objects degraded (0.086%), 2 pgs degraded (PG_DEGRADED) 2024-09-16T09:36:04.673 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:04 smithi070 ceph-mon[30388]: osdmap e303: 8 total, 8 up, 8 in 2024-09-16T09:36:04.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:04 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:36:04.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:04 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:36:04.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:04 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:36:04.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:04 smithi070 ceph-mon[40691]: osdmap e302: 8 total, 8 up, 8 in 2024-09-16T09:36:04.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:04 smithi070 ceph-mon[40691]: Health check update: Degraded data redundancy: 196/227574 objects degraded (0.086%), 2 pgs degraded (PG_DEGRADED) 2024-09-16T09:36:04.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:04 smithi070 ceph-mon[40691]: osdmap e303: 8 total, 8 up, 8 in 2024-09-16T09:36:04.698 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:04.699 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:04.699 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:36:04.800 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:36:04.833 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:04.833 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:04.833 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:36:04.934 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:36:04.978 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:04.978 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:04.978 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:36:05.079 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:36:05.111 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:05.111 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:05.112 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:36:05.212 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:36:05.241 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:05.242 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:05.242 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:36:05.343 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:36:05.373 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:05.374 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:05.374 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:36:05.475 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:36:05.515 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:05.516 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:05.516 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:36:05.617 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:36:05.651 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:05.652 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:05.652 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:36:05.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:05 smithi084 ceph-mon[32629]: pgmap v547: 178 pgs: 1 active+remapped+backfilling, 1 active+recovery_wait, 1 active+recovering+undersized+remapped, 2 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_toofull, 172 active+clean; 4.4 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.15k op/s; 196/217983 objects degraded (0.090%); 18692/217983 objects misplaced (8.575%); 21 MiB/s, 336 objects/s recovering 2024-09-16T09:36:05.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:05 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:36:05.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:05 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:36:05.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:05 smithi070 ceph-mon[30388]: pgmap v547: 178 pgs: 1 active+remapped+backfilling, 1 active+recovery_wait, 1 active+recovering+undersized+remapped, 2 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_toofull, 172 active+clean; 4.4 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.15k op/s; 196/217983 objects degraded (0.090%); 18692/217983 objects misplaced (8.575%); 21 MiB/s, 336 objects/s recovering 2024-09-16T09:36:05.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:05 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:36:05.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:05 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:36:05.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:05 smithi070 ceph-mon[40691]: pgmap v547: 178 pgs: 1 active+remapped+backfilling, 1 active+recovery_wait, 1 active+recovering+undersized+remapped, 2 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_toofull, 172 active+clean; 4.4 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.15k op/s; 196/217983 objects degraded (0.090%); 18692/217983 objects misplaced (8.575%); 21 MiB/s, 336 objects/s recovering 2024-09-16T09:36:05.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:05 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:36:05.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:05 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:36:05.753 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:36:05.790 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:05.791 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:05.791 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:36:05.892 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:36:05.931 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:05.931 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:05.932 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:36:06.033 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:36:06.070 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:06.071 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:06.071 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:36:06.172 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:36:06.206 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:06.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:06.207 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:36:06.307 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:36:06.340 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:06.340 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:06.341 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:36:06.442 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:36:06.471 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:06.472 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:06.472 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:36:06.573 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:36:06.598 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:06.599 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:06.599 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:36:06.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:06 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "25"} : dispatch 2024-09-16T09:36:06.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:06 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "25"} : dispatch 2024-09-16T09:36:06.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:06 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "25"} : dispatch 2024-09-16T09:36:06.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:06 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "25"} : dispatch 2024-09-16T09:36:06.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:06 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "25"} : dispatch 2024-09-16T09:36:06.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:06 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "25"} : dispatch 2024-09-16T09:36:06.700 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:36:06.729 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:06.730 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:06.730 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:36:06.831 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:36:06.853 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:06.854 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:06.854 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:36:06.955 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:36:06.976 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:06.977 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:06.977 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:36:07.078 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:36:07.102 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:07.103 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:07.104 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:36:07.204 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:36:07.226 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:07.226 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:07.227 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:36:07.327 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:36:07.349 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:07.349 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:07.350 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:36:07.450 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:36:07.472 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:07.473 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:07.473 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:36:07.574 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:36:07.598 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:07.599 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:07.599 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:36:07.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:07 smithi084 ceph-mon[32629]: pgmap v550: 178 pgs: 1 active+recovering+undersized+remapped, 1 active+recovery_wait, 2 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_toofull, 173 active+clean; 4.3 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.02k op/s; 196/214098 objects degraded (0.092%); 17340/214098 objects misplaced (8.099%); 14 MiB/s, 221 objects/s recovering 2024-09-16T09:36:07.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:07 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "25"}]': finished 2024-09-16T09:36:07.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:07 smithi084 ceph-mon[32629]: osdmap e304: 8 total, 8 up, 8 in 2024-09-16T09:36:07.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:07 smithi070 ceph-mon[30388]: pgmap v550: 178 pgs: 1 active+recovering+undersized+remapped, 1 active+recovery_wait, 2 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_toofull, 173 active+clean; 4.3 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.02k op/s; 196/214098 objects degraded (0.092%); 17340/214098 objects misplaced (8.099%); 14 MiB/s, 221 objects/s recovering 2024-09-16T09:36:07.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:07 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "25"}]': finished 2024-09-16T09:36:07.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:07 smithi070 ceph-mon[30388]: osdmap e304: 8 total, 8 up, 8 in 2024-09-16T09:36:07.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:07 smithi070 ceph-mon[40691]: pgmap v550: 178 pgs: 1 active+recovering+undersized+remapped, 1 active+recovery_wait, 2 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_toofull, 173 active+clean; 4.3 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.02k op/s; 196/214098 objects degraded (0.092%); 17340/214098 objects misplaced (8.099%); 14 MiB/s, 221 objects/s recovering 2024-09-16T09:36:07.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:07 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "25"}]': finished 2024-09-16T09:36:07.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:07 smithi070 ceph-mon[40691]: osdmap e304: 8 total, 8 up, 8 in 2024-09-16T09:36:07.700 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:36:07.725 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:07.726 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:07.726 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:36:07.827 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:36:07.849 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:07.850 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:07.850 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:36:07.951 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:36:07.975 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:07.976 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:07.976 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:36:08.077 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:36:08.099 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:08.099 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:08.100 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:36:08.200 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:36:08.222 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:08.222 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:08.223 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:36:08.323 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:36:08.345 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:08.346 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:08.347 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:36:08.447 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:36:08.470 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:08.471 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:08.471 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:36:08.572 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:36:08.594 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:08.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:08.595 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:36:08.651 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:08 smithi070 ceph-mon[30388]: osdmap e305: 8 total, 8 up, 8 in 2024-09-16T09:36:08.651 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:08 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:36:08.651 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:08 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:36:08.651 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:08 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:36:08.652 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:08 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:36:08.652 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:08 smithi070 ceph-mon[40691]: osdmap e305: 8 total, 8 up, 8 in 2024-09-16T09:36:08.652 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:08 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:36:08.652 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:08 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:36:08.652 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:08 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:36:08.652 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:08 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:36:08.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:08 smithi084 ceph-mon[32629]: osdmap e305: 8 total, 8 up, 8 in 2024-09-16T09:36:08.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:08 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:36:08.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:08 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:36:08.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:08 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:36:08.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:08 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:36:08.696 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:36:08.717 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:08.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:08.718 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:36:08.818 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:36:08.841 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:08.841 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:08.842 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:36:08.942 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:36:08.967 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:08.968 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:08.969 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:36:09.069 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:36:09.094 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:09.095 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:09.095 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:36:09.196 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:36:09.221 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:09.222 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:09.222 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:36:09.323 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:36:09.345 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:09.345 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:09.346 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:36:09.447 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:36:09.486 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:09.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:09.487 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:36:09.588 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:36:09.626 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:09.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:09.627 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:36:09.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:09 smithi084 ceph-mon[32629]: pgmap v553: 178 pgs: 1 active+recovering+undersized+remapped, 1 active+recovery_wait, 2 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_toofull, 173 active+clean; 4.3 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 0 B/s wr, 421 op/s; 196/212928 objects degraded (0.092%); 17340/212928 objects misplaced (8.144%); 9.2 MiB/s, 146 objects/s recovering 2024-09-16T09:36:09.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:09 smithi084 ceph-mon[32629]: osdmap e306: 8 total, 8 up, 8 in 2024-09-16T09:36:09.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:09 smithi084 ceph-mon[32629]: Health check update: Degraded data redundancy: 196/212928 objects degraded (0.092%), 2 pgs degraded, 3 pgs undersized (PG_DEGRADED) 2024-09-16T09:36:09.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:09 smithi084 ceph-mon[32629]: osdmap e307: 8 total, 8 up, 8 in 2024-09-16T09:36:09.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:09 smithi070 ceph-mon[30388]: pgmap v553: 178 pgs: 1 active+recovering+undersized+remapped, 1 active+recovery_wait, 2 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_toofull, 173 active+clean; 4.3 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 0 B/s wr, 421 op/s; 196/212928 objects degraded (0.092%); 17340/212928 objects misplaced (8.144%); 9.2 MiB/s, 146 objects/s recovering 2024-09-16T09:36:09.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:09 smithi070 ceph-mon[30388]: osdmap e306: 8 total, 8 up, 8 in 2024-09-16T09:36:09.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:09 smithi070 ceph-mon[30388]: Health check update: Degraded data redundancy: 196/212928 objects degraded (0.092%), 2 pgs degraded, 3 pgs undersized (PG_DEGRADED) 2024-09-16T09:36:09.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:09 smithi070 ceph-mon[30388]: osdmap e307: 8 total, 8 up, 8 in 2024-09-16T09:36:09.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:09 smithi070 ceph-mon[40691]: pgmap v553: 178 pgs: 1 active+recovering+undersized+remapped, 1 active+recovery_wait, 2 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_toofull, 173 active+clean; 4.3 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 0 B/s wr, 421 op/s; 196/212928 objects degraded (0.092%); 17340/212928 objects misplaced (8.144%); 9.2 MiB/s, 146 objects/s recovering 2024-09-16T09:36:09.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:09 smithi070 ceph-mon[40691]: osdmap e306: 8 total, 8 up, 8 in 2024-09-16T09:36:09.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:09 smithi070 ceph-mon[40691]: Health check update: Degraded data redundancy: 196/212928 objects degraded (0.092%), 2 pgs degraded, 3 pgs undersized (PG_DEGRADED) 2024-09-16T09:36:09.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:09 smithi070 ceph-mon[40691]: osdmap e307: 8 total, 8 up, 8 in 2024-09-16T09:36:09.728 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:36:09.785 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:09.786 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:09.786 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:36:09.887 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:36:09.912 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:09.913 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:09.913 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:36:10.014 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:36:10.055 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:10.056 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:10.056 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:36:10.157 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:36:10.160 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:36:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=sqlstore.transactions t=2024-09-16T09:36:09.842729496Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-16T09:36:10.161 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:36:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=cleanup t=2024-09-16T09:36:09.848547396Z level=info msg="Completed cleanup jobs" duration=12.97063ms 2024-09-16T09:36:10.161 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:36:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=sqlstore.transactions t=2024-09-16T09:36:09.857084316Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=1 code="database is locked" 2024-09-16T09:36:10.161 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:36:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=plugins.update.checker t=2024-09-16T09:36:09.974007696Z level=info msg="Update check succeeded" duration=60.895776ms 2024-09-16T09:36:10.161 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:36:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=grafana.update.checker t=2024-09-16T09:36:09.990993053Z level=info msg="Update check succeeded" duration=69.490644ms 2024-09-16T09:36:10.189 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:10.189 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:10.189 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:36:10.291 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:36:10.321 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:10.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:10.322 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:36:10.423 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:36:10.447 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:10.448 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:10.448 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:36:10.549 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:36:10.590 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:10.591 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:10.591 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:36:10.692 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:36:10.725 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:10.726 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:10.726 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:36:10.827 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:36:10.851 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:36:10.854 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:10.854 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:10.854 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:36:10.955 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:36:10.986 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:10.987 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:10.987 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:36:11.088 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:36:11.127 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:11.127 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:11.128 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:36:11.138 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:36:11.229 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:36:11.260 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:11.261 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:11.261 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:36:11.362 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:36:11.385 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:11.385 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:11.385 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:36:11.486 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:36:11.514 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:11.515 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:11.515 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:36:11.616 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:36:11.643 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:11.644 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:11.644 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:36:11.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:11 smithi084 ceph-mon[32629]: pgmap v556: 177 pgs: 1 peering, 1 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+undersized+remapped, 1 active+recovery_wait, 2 active+recovering+undersized+degraded+remapped, 171 active+clean; 4.1 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 923 op/s; 205/203025 objects degraded (0.101%); 14464/203025 objects misplaced (7.124%); 18 MiB/s, 288 objects/s recovering 2024-09-16T09:36:11.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:11 smithi084 ceph-mon[32629]: 9.16 scrub starts 2024-09-16T09:36:11.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:11 smithi070 ceph-mon[30388]: pgmap v556: 177 pgs: 1 peering, 1 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+undersized+remapped, 1 active+recovery_wait, 2 active+recovering+undersized+degraded+remapped, 171 active+clean; 4.1 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 923 op/s; 205/203025 objects degraded (0.101%); 14464/203025 objects misplaced (7.124%); 18 MiB/s, 288 objects/s recovering 2024-09-16T09:36:11.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:11 smithi070 ceph-mon[30388]: 9.16 scrub starts 2024-09-16T09:36:11.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:11 smithi070 ceph-mon[40691]: pgmap v556: 177 pgs: 1 peering, 1 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+undersized+remapped, 1 active+recovery_wait, 2 active+recovering+undersized+degraded+remapped, 171 active+clean; 4.1 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 923 op/s; 205/203025 objects degraded (0.101%); 14464/203025 objects misplaced (7.124%); 18 MiB/s, 288 objects/s recovering 2024-09-16T09:36:11.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:11 smithi070 ceph-mon[40691]: 9.16 scrub starts 2024-09-16T09:36:11.745 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:36:11.775 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:11.776 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:11.776 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:36:11.788 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-16T09:36:11.878 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:36:11.915 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:11.916 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:11.916 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:36:12.017 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:36:12.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:12.052 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:12.052 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:36:12.117 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:36:11 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:36:11] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:36:12.154 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:36:12.175 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:12.176 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:12.176 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:36:12.277 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:36:12.299 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:12.299 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:12.300 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:36:12.401 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:36:12.428 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:12.429 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:12.432 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:36:12.533 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:36:12.566 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:12.566 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:12.567 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:36:12.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:12 smithi084 ceph-mon[32629]: osdmap e308: 8 total, 8 up, 8 in 2024-09-16T09:36:12.668 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:36:12.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:12 smithi070 ceph-mon[30388]: osdmap e308: 8 total, 8 up, 8 in 2024-09-16T09:36:12.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:12 smithi070 ceph-mon[40691]: osdmap e308: 8 total, 8 up, 8 in 2024-09-16T09:36:12.704 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:12.705 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:12.705 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:36:12.806 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:36:12.842 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:12.842 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:12.843 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:36:12.924 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T09:36:12.943 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:36:12.970 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:12.971 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:12.971 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:36:13.072 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:36:13.097 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:13.097 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:13.098 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:36:13.198 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:36:13.231 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:13.232 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:13.233 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:36:13.333 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:36:13.389 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:13.390 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:13.390 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:36:13.491 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:36:13.521 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:13.522 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:13.522 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:36:13.623 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:36:13.639 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:13 smithi070 ceph-mon[30388]: pgmap v558: 177 pgs: 1 peering, 1 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+undersized+remapped, 1 active+recovery_wait, 2 active+recovering+undersized+degraded+remapped, 171 active+clean; 4.1 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 845 op/s; 205/203025 objects degraded (0.101%); 14464/203025 objects misplaced (7.124%); 17 MiB/s, 264 objects/s recovering 2024-09-16T09:36:13.639 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:13 smithi070 ceph-mon[30388]: osdmap e309: 8 total, 8 up, 8 in 2024-09-16T09:36:13.640 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:13 smithi070 ceph-mon[40691]: pgmap v558: 177 pgs: 1 peering, 1 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+undersized+remapped, 1 active+recovery_wait, 2 active+recovering+undersized+degraded+remapped, 171 active+clean; 4.1 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 845 op/s; 205/203025 objects degraded (0.101%); 14464/203025 objects misplaced (7.124%); 17 MiB/s, 264 objects/s recovering 2024-09-16T09:36:13.640 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:13 smithi070 ceph-mon[40691]: osdmap e309: 8 total, 8 up, 8 in 2024-09-16T09:36:13.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:13 smithi084 ceph-mon[32629]: pgmap v558: 177 pgs: 1 peering, 1 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+undersized+remapped, 1 active+recovery_wait, 2 active+recovering+undersized+degraded+remapped, 171 active+clean; 4.1 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 845 op/s; 205/203025 objects degraded (0.101%); 14464/203025 objects misplaced (7.124%); 17 MiB/s, 264 objects/s recovering 2024-09-16T09:36:13.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:13 smithi084 ceph-mon[32629]: osdmap e309: 8 total, 8 up, 8 in 2024-09-16T09:36:13.660 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:36:13 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:36:13.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:13.668 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:13.668 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:36:13.769 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:36:13.795 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:13.796 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:13.796 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:36:13.836 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:36:13.840 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2024-09-16T09:36:13.897 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:36:13.939 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:13.940 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:13.940 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:36:14.041 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:36:14.070 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:14.071 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:14.072 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:36:14.173 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:36:14.221 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:14.221 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:14.222 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:36:14.322 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:36:14.356 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:14.356 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:14.356 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:36:14.457 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:36:14.489 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:14.490 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:14.490 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:36:14.572 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T09:36:14.572 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T09:36:14.572 INFO:teuthology.orchestra.run.smithi070.stdout: "complaint_time": 30, 2024-09-16T09:36:14.572 INFO:teuthology.orchestra.run.smithi070.stdout: "num_blocked_ops": 0 2024-09-16T09:36:14.572 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T09:36:14.591 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:36:14.634 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:14.636 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:14.636 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:36:14.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:14 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:36:14.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:14 smithi084 ceph-mon[32629]: Health check update: Degraded data redundancy: 205/203025 objects degraded (0.101%), 2 pgs degraded, 3 pgs undersized (PG_DEGRADED) 2024-09-16T09:36:14.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:14 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:36:14.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:14 smithi070 ceph-mon[30388]: Health check update: Degraded data redundancy: 205/203025 objects degraded (0.101%), 2 pgs degraded, 3 pgs undersized (PG_DEGRADED) 2024-09-16T09:36:14.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:14 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:36:14.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:14 smithi070 ceph-mon[40691]: Health check update: Degraded data redundancy: 205/203025 objects degraded (0.101%), 2 pgs degraded, 3 pgs undersized (PG_DEGRADED) 2024-09-16T09:36:14.737 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:36:14.774 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:14.775 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:14.775 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:36:14.876 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:36:14.912 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:14.913 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:14.913 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:36:15.014 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:36:15.047 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:15.047 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:15.048 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:36:15.149 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:36:15.185 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:15.185 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:15.186 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:36:15.287 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:36:15.326 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:15.326 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:15.327 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:36:15.427 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:36:15.444 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:36:15.484 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:15.484 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:15.485 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:36:15.585 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:36:15.619 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:15.619 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:15.620 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:36:15.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:15 smithi084 ceph-mon[32629]: pgmap v560: 177 pgs: 2 active+remapped+backfilling, 1 peering, 1 active+recovering+undersized+remapped, 1 active+recovering+undersized+degraded+remapped, 172 active+clean; 4.0 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 954 B/s rd, 13 MiB/s wr, 1.11k op/s; 199/195570 objects degraded (0.102%); 14571/195570 objects misplaced (7.451%); 45 MiB/s, 757 objects/s recovering 2024-09-16T09:36:15.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:15 smithi084 ceph-mon[32629]: from='client.25444 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:36:15.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:15 smithi084 ceph-mon[32629]: Health check cleared: PG_BACKFILL_FULL (was: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull) 2024-09-16T09:36:15.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:15 smithi070 ceph-mon[30388]: pgmap v560: 177 pgs: 2 active+remapped+backfilling, 1 peering, 1 active+recovering+undersized+remapped, 1 active+recovering+undersized+degraded+remapped, 172 active+clean; 4.0 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 954 B/s rd, 13 MiB/s wr, 1.11k op/s; 199/195570 objects degraded (0.102%); 14571/195570 objects misplaced (7.451%); 45 MiB/s, 757 objects/s recovering 2024-09-16T09:36:15.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:15 smithi070 ceph-mon[30388]: from='client.25444 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:36:15.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:15 smithi070 ceph-mon[30388]: Health check cleared: PG_BACKFILL_FULL (was: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull) 2024-09-16T09:36:15.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:15 smithi070 ceph-mon[40691]: pgmap v560: 177 pgs: 2 active+remapped+backfilling, 1 peering, 1 active+recovering+undersized+remapped, 1 active+recovering+undersized+degraded+remapped, 172 active+clean; 4.0 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 954 B/s rd, 13 MiB/s wr, 1.11k op/s; 199/195570 objects degraded (0.102%); 14571/195570 objects misplaced (7.451%); 45 MiB/s, 757 objects/s recovering 2024-09-16T09:36:15.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:15 smithi070 ceph-mon[40691]: from='client.25444 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:36:15.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:15 smithi070 ceph-mon[40691]: Health check cleared: PG_BACKFILL_FULL (was: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull) 2024-09-16T09:36:15.721 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:36:15.746 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:15.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:15.747 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:36:15.847 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:36:15.887 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:15.888 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:15.888 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:36:15.989 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:36:16.033 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:16.034 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:16.034 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:36:16.135 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:36:16.163 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:16.164 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:16.164 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:36:16.265 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:36:16.293 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:16.294 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:16.294 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:36:16.395 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:36:16.429 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:16.430 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:16.430 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:36:16.531 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:36:16.556 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:16.557 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:16.557 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:36:16.585 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.0 dump_historic_ops 2024-09-16T09:36:16.658 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:36:16.689 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:16.690 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:16.690 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:36:16.791 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:36:16.816 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:16.816 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:16.817 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:36:16.918 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:36:16.974 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:16.975 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:16.975 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:36:17.041 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":560,"stamp":"2024-09-16T09:36:13.721725+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4259781171,"num_objects":65190,"num_object_clones":0,"num_object_copies":195570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":199,"num_objects_misplaced":14571,"num_objects_unfound":0,"num_objects_dirty":65190,"num_whiteouts":0,"num_read":1807,"num_read_kb":1559,"num_write":114310,"num_write_kb":4057390,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21388,"num_bytes_recovered":1383273254,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":116826,"ondisk_log_size":116826,"up":531,"acting":529,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":544,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":17292536,"kb_used_data":13594392,"kb_used_omap":12,"kb_used_meta":3697843,"kb_avail":732668680,"statfs":{"total":767960285184,"available":750252728320,"internally_reserved":0,"allocated":13920657408,"data_stored":13914824412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":3786591794},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,3,3,1,1,0,0,1,1,14],"upper_bound":2048},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-70778880,"num_objects":-1080,"num_object_clones":0,"num_object_copies":-3240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3,"num_objects_misplaced":-1229,"num_objects_unfound":0,"num_objects_dirty":-1080,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5952,"num_write_kb":71616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4064,"num_bytes_recovered":255787008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.366366"},"pg_stats":[{"pgid":"9.15","version":"304'7114","reported_seq":21307,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-09-16T09:36:08.361736+0000","last_change":"2024-09-16T09:35:46.279050+0000","last_active":"2024-09-16T09:36:08.361736+0000","last_peered":"2024-09-16T09:36:08.361736+0000","last_clean":"2024-09-16T09:36:08.361736+0000","last_became_active":"2024-09-16T09:35:44.293269+0000","last_became_peered":"2024-09-16T09:35:44.293269+0000","last_unstale":"2024-09-16T09:36:08.361736+0000","last_undegraded":"2024-09-16T09:36:08.361736+0000","last_fullsized":"2024-09-16T09:36:08.361736+0000","mapping_epoch":292,"log_start":"237'1901","ondisk_log_start":"237'1901","created":243,"last_epoch_clean":293,"parent":"0.0","parent_split_bits":5,"last_scrub":"294'6521","last_scrub_stamp":"2024-09-16T09:35:46.278931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:35:46.278931+0000","objects_scrubbed":2721,"log_size":5213,"log_dups_size":0,"ondisk_log_size":5213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:36:51.040937+0000","scrub_duration":1070,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":138346496,"num_objects":2111,"num_object_clones":0,"num_object_copies":6333,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2111,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":610,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":543,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-09-16T09:36:08.363007+0000","last_change":"2024-09-16T09:32:39.384174+0000","last_active":"2024-09-16T09:36:08.363007+0000","last_peered":"2024-09-16T09:36:08.363007+0000","last_clean":"2024-09-16T09:36:08.363007+0000","last_became_active":"2024-09-16T09:25:01.709108+0000","last_became_peered":"2024-09-16T09:25:01.709108+0000","last_unstale":"2024-09-16T09:36:08.363007+0000","last_undegraded":"2024-09-16T09:36:08.363007+0000","last_fullsized":"2024-09-16T09:36:08.363007+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:39.384104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:32:39.384104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"285'15","reported_seq":561,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.371670+0000","last_change":"2024-09-16T09:32:38.074301+0000","last_active":"2024-09-16T09:36:12.371670+0000","last_peered":"2024-09-16T09:36:12.371670+0000","last_clean":"2024-09-16T09:36:12.371670+0000","last_became_active":"2024-09-16T09:25:03.714635+0000","last_became_peered":"2024-09-16T09:25:03.714635+0000","last_unstale":"2024-09-16T09:36:12.371670+0000","last_undegraded":"2024-09-16T09:36:12.371670+0000","last_fullsized":"2024-09-16T09:36:12.371670+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:32:38.074231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:32:38.074231+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:36:56.911701+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":164,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.413736+0000","last_change":"2024-09-16T09:33:30.242561+0000","last_active":"2024-09-16T09:36:12.413736+0000","last_peered":"2024-09-16T09:36:12.413736+0000","last_clean":"2024-09-16T09:36:12.413736+0000","last_became_active":"2024-09-16T09:33:30.242238+0000","last_became_peered":"2024-09-16T09:33:30.242238+0000","last_unstale":"2024-09-16T09:36:12.413736+0000","last_undegraded":"2024-09-16T09:36:12.413736+0000","last_fullsized":"2024-09-16T09:36:12.413736+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:13.426830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:13.426830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":520,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.371750+0000","last_change":"2024-09-16T09:35:11.876996+0000","last_active":"2024-09-16T09:36:12.371750+0000","last_peered":"2024-09-16T09:36:12.371750+0000","last_clean":"2024-09-16T09:36:12.371750+0000","last_became_active":"2024-09-16T09:25:07.715442+0000","last_became_peered":"2024-09-16T09:25:07.715442+0000","last_unstale":"2024-09-16T09:36:12.371750+0000","last_undegraded":"2024-09-16T09:36:12.371750+0000","last_fullsized":"2024-09-16T09:36:12.371750+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:35:11.876955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:35:11.876955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:36:41.829050+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"309'7282","reported_seq":7540,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.912798+0000","last_change":"2024-09-16T09:35:35.736229+0000","last_active":"2024-09-16T09:36:12.912798+0000","last_peered":"2024-09-16T09:36:12.912798+0000","last_clean":"2024-09-16T09:36:12.912798+0000","last_became_active":"2024-09-16T09:33:52.932390+0000","last_became_peered":"2024-09-16T09:33:52.932390+0000","last_unstale":"2024-09-16T09:36:12.912798+0000","last_undegraded":"2024-09-16T09:36:12.912798+0000","last_fullsized":"2024-09-16T09:36:12.912798+0000","mapping_epoch":243,"log_start":"241'3010","ondisk_log_start":"241'3010","created":243,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":5,"last_scrub":"291'6339","last_scrub_stamp":"2024-09-16T09:35:35.736110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:35:35.736110+0000","objects_scrubbed":2932,"log_size":4272,"log_dups_size":0,"ondisk_log_size":4272,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:36:52.157097+0000","scrub_duration":832,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":130023424,"num_objects":1984,"num_object_clones":0,"num_object_copies":5952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1984,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":948,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":266,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.405315+0000","last_change":"2024-09-16T09:34:33.452784+0000","last_active":"2024-09-16T09:36:12.405315+0000","last_peered":"2024-09-16T09:36:12.405315+0000","last_clean":"2024-09-16T09:36:12.405315+0000","last_became_active":"2024-09-16T09:31:09.269342+0000","last_became_peered":"2024-09-16T09:31:09.269342+0000","last_unstale":"2024-09-16T09:36:12.405315+0000","last_undegraded":"2024-09-16T09:36:12.405315+0000","last_fullsized":"2024-09-16T09:36:12.405315+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:33.452706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:34:33.452706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":406,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.445173+0000","last_change":"2024-09-16T09:33:08.816231+0000","last_active":"2024-09-16T09:36:12.445173+0000","last_peered":"2024-09-16T09:36:12.445173+0000","last_clean":"2024-09-16T09:36:12.445173+0000","last_became_active":"2024-09-16T09:31:09.275136+0000","last_became_peered":"2024-09-16T09:31:09.275136+0000","last_unstale":"2024-09-16T09:36:12.445173+0000","last_undegraded":"2024-09-16T09:36:12.445173+0000","last_fullsized":"2024-09-16T09:36:12.445173+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:08.816174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:08.816174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.19","version":"285'27","reported_seq":233,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-09-16T09:36:08.362231+0000","last_change":"2024-09-16T09:33:21.337711+0000","last_active":"2024-09-16T09:36:08.362231+0000","last_peered":"2024-09-16T09:36:08.362231+0000","last_clean":"2024-09-16T09:36:08.362231+0000","last_became_active":"2024-09-16T09:33:21.337284+0000","last_became_peered":"2024-09-16T09:33:21.337284+0000","last_unstale":"2024-09-16T09:36:08.362231+0000","last_undegraded":"2024-09-16T09:36:08.362231+0000","last_fullsized":"2024-09-16T09:36:08.362231+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-16T09:32:45.194581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:32:45.194581+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":540,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.371565+0000","last_change":"2024-09-16T09:32:44.114475+0000","last_active":"2024-09-16T09:36:12.371565+0000","last_peered":"2024-09-16T09:36:12.371565+0000","last_clean":"2024-09-16T09:36:12.371565+0000","last_became_active":"2024-09-16T09:25:05.717236+0000","last_became_peered":"2024-09-16T09:25:05.717236+0000","last_unstale":"2024-09-16T09:36:12.371565+0000","last_undegraded":"2024-09-16T09:36:12.371565+0000","last_fullsized":"2024-09-16T09:36:12.371565+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:44.114407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:32:44.114407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:36:54.864438+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.17","version":"309'7210","reported_seq":21949,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:13.085327+0000","last_change":"2024-09-16T09:35:46.875017+0000","last_active":"2024-09-16T09:36:13.085327+0000","last_peered":"2024-09-16T09:36:13.085327+0000","last_clean":"2024-09-16T09:36:13.085327+0000","last_became_active":"2024-09-16T09:35:45.351627+0000","last_became_peered":"2024-09-16T09:35:45.351627+0000","last_unstale":"2024-09-16T09:36:13.085327+0000","last_undegraded":"2024-09-16T09:36:13.085327+0000","last_fullsized":"2024-09-16T09:36:13.085327+0000","mapping_epoch":293,"log_start":"239'2356","ondisk_log_start":"239'2356","created":243,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":5,"last_scrub":"294'6528","last_scrub_stamp":"2024-09-16T09:35:46.874786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:35:46.874786+0000","objects_scrubbed":2733,"log_size":4854,"log_dups_size":0,"ondisk_log_size":4854,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:37:07.393823+0000","scrub_duration":985,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":133627904,"num_objects":2039,"num_object_clones":0,"num_object_copies":6117,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2039,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2082,"num_write_kb":16704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3233,"num_bytes_recovered":211877888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":167,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-09-16T09:36:08.362035+0000","last_change":"2024-09-16T09:33:36.655594+0000","last_active":"2024-09-16T09:36:08.362035+0000","last_peered":"2024-09-16T09:36:08.362035+0000","last_clean":"2024-09-16T09:36:08.362035+0000","last_became_active":"2024-09-16T09:33:30.247733+0000","last_became_peered":"2024-09-16T09:33:30.247733+0000","last_unstale":"2024-09-16T09:36:08.362035+0000","last_undegraded":"2024-09-16T09:36:08.362035+0000","last_fullsized":"2024-09-16T09:36:08.362035+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:36.655236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:36.655236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1a","version":"285'15","reported_seq":561,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.371953+0000","last_change":"2024-09-16T09:32:48.185484+0000","last_active":"2024-09-16T09:36:12.371953+0000","last_peered":"2024-09-16T09:36:12.371953+0000","last_clean":"2024-09-16T09:36:12.371953+0000","last_became_active":"2024-09-16T09:25:03.709147+0000","last_became_peered":"2024-09-16T09:25:03.709147+0000","last_unstale":"2024-09-16T09:36:12.371953+0000","last_undegraded":"2024-09-16T09:36:12.371953+0000","last_fullsized":"2024-09-16T09:36:12.371953+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:32:48.185380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:32:48.185380+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:36:59.829704+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":523,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.414461+0000","last_change":"2024-09-16T09:33:21.376175+0000","last_active":"2024-09-16T09:36:12.414461+0000","last_peered":"2024-09-16T09:36:12.414461+0000","last_clean":"2024-09-16T09:36:12.414461+0000","last_became_active":"2024-09-16T09:25:05.718134+0000","last_became_peered":"2024-09-16T09:25:05.718134+0000","last_unstale":"2024-09-16T09:36:12.414461+0000","last_undegraded":"2024-09-16T09:36:12.414461+0000","last_fullsized":"2024-09-16T09:36:12.414461+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:21.376128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:21.376128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"66'1","reported_seq":531,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-09-16T09:36:08.361908+0000","last_change":"2024-09-16T09:33:38.705001+0000","last_active":"2024-09-16T09:36:08.361908+0000","last_peered":"2024-09-16T09:36:08.361908+0000","last_clean":"2024-09-16T09:36:08.361908+0000","last_became_active":"2024-09-16T09:25:07.716558+0000","last_became_peered":"2024-09-16T09:25:07.716558+0000","last_unstale":"2024-09-16T09:36:08.361908+0000","last_undegraded":"2024-09-16T09:36:08.361908+0000","last_fullsized":"2024-09-16T09:36:08.361908+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-16T09:33:38.704868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:33:38.704868+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"309'7203","reported_seq":16019,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:13.115653+0000","last_change":"2024-09-16T09:36:11.008150+0000","last_active":"2024-09-16T09:36:13.115653+0000","last_peered":"2024-09-16T09:36:13.115653+0000","last_clean":"2024-09-16T09:36:13.115653+0000","last_became_active":"2024-09-16T09:35:57.832106+0000","last_became_peered":"2024-09-16T09:35:57.832106+0000","last_unstale":"2024-09-16T09:36:13.115653+0000","last_undegraded":"2024-09-16T09:36:13.115653+0000","last_fullsized":"2024-09-16T09:36:13.115653+0000","mapping_epoch":298,"log_start":"239'2465","ondisk_log_start":"239'2465","created":243,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":5,"last_scrub":"273'4808","last_scrub_stamp":"2024-09-16T09:34:44.425779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:34:44.425779+0000","objects_scrubbed":0,"log_size":4738,"log_dups_size":0,"ondisk_log_size":4738,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:36:40.914935+0000","scrub_duration":809,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":133693440,"num_objects":2040,"num_object_clones":0,"num_object_copies":6120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2040,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2409,"num_write_kb":40576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2712,"num_bytes_recovered":177602560,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":531,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.414089+0000","last_change":"2024-09-16T09:33:19.326765+0000","last_active":"2024-09-16T09:36:12.414089+0000","last_peered":"2024-09-16T09:36:12.414089+0000","last_clean":"2024-09-16T09:36:12.414089+0000","last_became_active":"2024-09-16T09:25:01.696939+0000","last_became_peered":"2024-09-16T09:25:01.696939+0000","last_unstale":"2024-09-16T09:36:12.414089+0000","last_undegraded":"2024-09-16T09:36:12.414089+0000","last_fullsized":"2024-09-16T09:36:12.414089+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:19.326695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:19.326695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"285'7","reported_seq":549,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.371963+0000","last_change":"2024-09-16T09:35:39.975915+0000","last_active":"2024-09-16T09:36:12.371963+0000","last_peered":"2024-09-16T09:36:12.371963+0000","last_clean":"2024-09-16T09:36:12.371963+0000","last_became_active":"2024-09-16T09:25:03.716136+0000","last_became_peered":"2024-09-16T09:25:03.716136+0000","last_unstale":"2024-09-16T09:36:12.371963+0000","last_undegraded":"2024-09-16T09:36:12.371963+0000","last_fullsized":"2024-09-16T09:36:12.371963+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'7","last_scrub_stamp":"2024-09-16T09:35:39.975840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:35:39.975840+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:36:45.686422+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":10,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":523,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.408526+0000","last_change":"2024-09-16T09:28:52.746046+0000","last_active":"2024-09-16T09:36:12.408526+0000","last_peered":"2024-09-16T09:36:12.408526+0000","last_clean":"2024-09-16T09:36:12.408526+0000","last_became_active":"2024-09-16T09:25:05.718281+0000","last_became_peered":"2024-09-16T09:25:05.718281+0000","last_unstale":"2024-09-16T09:36:12.408526+0000","last_undegraded":"2024-09-16T09:36:12.408526+0000","last_fullsized":"2024-09-16T09:36:12.408526+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:52.745999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:28:52.745999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":549,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.413774+0000","last_change":"2024-09-16T09:33:46.366042+0000","last_active":"2024-09-16T09:36:12.413774+0000","last_peered":"2024-09-16T09:36:12.413774+0000","last_clean":"2024-09-16T09:36:12.413774+0000","last_became_active":"2024-09-16T09:31:09.275656+0000","last_became_peered":"2024-09-16T09:31:09.275656+0000","last_unstale":"2024-09-16T09:36:12.413774+0000","last_undegraded":"2024-09-16T09:36:12.413774+0000","last_fullsized":"2024-09-16T09:36:12.413774+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:46.365972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:33:46.365972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":73,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"309'7046","reported_seq":7298,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.684513+0000","last_change":"2024-09-16T09:35:38.622217+0000","last_active":"2024-09-16T09:36:12.684513+0000","last_peered":"2024-09-16T09:36:12.684513+0000","last_clean":"2024-09-16T09:36:12.684513+0000","last_became_active":"2024-09-16T09:35:05.670682+0000","last_became_peered":"2024-09-16T09:35:05.670682+0000","last_unstale":"2024-09-16T09:36:12.684513+0000","last_undegraded":"2024-09-16T09:36:12.684513+0000","last_fullsized":"2024-09-16T09:36:12.684513+0000","mapping_epoch":282,"log_start":"236'1195","ondisk_log_start":"236'1195","created":243,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":5,"last_scrub":"291'6207","last_scrub_stamp":"2024-09-16T09:35:38.621911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:35:38.621911+0000","objects_scrubbed":2873,"log_size":5851,"log_dups_size":0,"ondisk_log_size":5851,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:36:43.013543+0000","scrub_duration":752,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":132644864,"num_objects":2024,"num_object_clones":0,"num_object_copies":6072,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2024,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":849,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":537,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.372292+0000","last_change":"2024-09-16T09:33:48.322010+0000","last_active":"2024-09-16T09:36:12.372292+0000","last_peered":"2024-09-16T09:36:12.372292+0000","last_clean":"2024-09-16T09:36:12.372292+0000","last_became_active":"2024-09-16T09:25:07.720793+0000","last_became_peered":"2024-09-16T09:25:07.720793+0000","last_unstale":"2024-09-16T09:36:12.372292+0000","last_undegraded":"2024-09-16T09:36:12.372292+0000","last_fullsized":"2024-09-16T09:36:12.372292+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:48.321895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:33:48.321895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":288,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.445088+0000","last_change":"2024-09-16T09:34:33.866299+0000","last_active":"2024-09-16T09:36:12.445088+0000","last_peered":"2024-09-16T09:36:12.445088+0000","last_clean":"2024-09-16T09:36:12.445088+0000","last_became_active":"2024-09-16T09:30:39.201824+0000","last_became_peered":"2024-09-16T09:30:39.201824+0000","last_unstale":"2024-09-16T09:36:12.445088+0000","last_undegraded":"2024-09-16T09:36:12.445088+0000","last_fullsized":"2024-09-16T09:36:12.445088+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:33.866142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:34:33.866142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"285'27","reported_seq":628,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.433540+0000","last_change":"2024-09-16T09:33:30.250162+0000","last_active":"2024-09-16T09:36:12.433540+0000","last_peered":"2024-09-16T09:36:12.433540+0000","last_clean":"2024-09-16T09:36:12.433540+0000","last_became_active":"2024-09-16T09:33:30.249798+0000","last_became_peered":"2024-09-16T09:33:30.249798+0000","last_unstale":"2024-09-16T09:36:12.433540+0000","last_undegraded":"2024-09-16T09:36:12.433540+0000","last_fullsized":"2024-09-16T09:36:12.433540+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-16T09:32:49.202163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:32:49.202163+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":172,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.372257+0000","last_change":"2024-09-16T09:33:43.326933+0000","last_active":"2024-09-16T09:36:12.372257+0000","last_peered":"2024-09-16T09:36:12.372257+0000","last_clean":"2024-09-16T09:36:12.372257+0000","last_became_active":"2024-09-16T09:33:30.251587+0000","last_became_peered":"2024-09-16T09:33:30.251587+0000","last_unstale":"2024-09-16T09:36:12.372257+0000","last_undegraded":"2024-09-16T09:36:12.372257+0000","last_fullsized":"2024-09-16T09:36:12.372257+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:43.326852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:43.326852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,0],"acting":[4,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.10","version":"304'7062","reported_seq":19114,"reported_epoch":306,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-16T09:36:09.047621+0000","last_change":"2024-09-16T09:35:07.064657+0000","last_active":"2024-09-16T09:36:09.047621+0000","last_peered":"2024-09-16T09:36:09.047621+0000","last_clean":"2024-09-16T09:35:04.646442+0000","last_became_active":"2024-09-16T09:35:06.938011+0000","last_became_peered":"2024-09-16T09:35:06.938011+0000","last_unstale":"2024-09-16T09:36:09.047621+0000","last_undegraded":"2024-09-16T09:36:09.047621+0000","last_fullsized":"2024-09-16T09:35:06.666095+0000","mapping_epoch":283,"log_start":"236'1430","ondisk_log_start":"236'1430","created":243,"last_epoch_clean":265,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:33:24.230486+0000","objects_scrubbed":0,"log_size":5632,"log_dups_size":0,"ondisk_log_size":5632,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":133300224,"num_objects":2034,"num_object_clones":0,"num_object_copies":6102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":5415,"num_objects_unfound":0,"num_objects_dirty":2034,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5480,"num_write_kb":240448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3405,"num_bytes_recovered":223084544,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,2],"acting":[0,1],"avail_no_missing":["0","4","7"],"object_location_counts":[{"shards":"0,1","objects":229},{"shards":"0,4,7","objects":1805}],"blocked_by":[],"up_primary":6,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":385,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.445548+0000","last_change":"2024-09-16T09:34:37.804874+0000","last_active":"2024-09-16T09:36:12.445548+0000","last_peered":"2024-09-16T09:36:12.445548+0000","last_clean":"2024-09-16T09:36:12.445548+0000","last_became_active":"2024-09-16T09:28:52.974255+0000","last_became_peered":"2024-09-16T09:28:52.974255+0000","last_unstale":"2024-09-16T09:36:12.445548+0000","last_undegraded":"2024-09-16T09:36:12.445548+0000","last_fullsized":"2024-09-16T09:36:12.445548+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:37.804805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:34:37.804805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":548,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.372111+0000","last_change":"2024-09-16T09:35:23.751992+0000","last_active":"2024-09-16T09:36:12.372111+0000","last_peered":"2024-09-16T09:36:12.372111+0000","last_clean":"2024-09-16T09:36:12.372111+0000","last_became_active":"2024-09-16T09:25:01.706491+0000","last_became_peered":"2024-09-16T09:25:01.706491+0000","last_unstale":"2024-09-16T09:36:12.372111+0000","last_undegraded":"2024-09-16T09:36:12.372111+0000","last_fullsized":"2024-09-16T09:36:12.372111+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:35:23.751946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:35:23.751946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:36:52.091018+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"285'18","reported_seq":224,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.372470+0000","last_change":"2024-09-16T09:33:39.284526+0000","last_active":"2024-09-16T09:36:12.372470+0000","last_peered":"2024-09-16T09:36:12.372470+0000","last_clean":"2024-09-16T09:36:12.372470+0000","last_became_active":"2024-09-16T09:33:30.251626+0000","last_became_peered":"2024-09-16T09:33:30.251626+0000","last_unstale":"2024-09-16T09:36:12.372470+0000","last_undegraded":"2024-09-16T09:36:12.372470+0000","last_fullsized":"2024-09-16T09:36:12.372470+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-16T09:33:39.284451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:33:39.284451+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":33,"num_read_kb":24,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":524,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.372177+0000","last_change":"2024-09-16T09:33:38.238661+0000","last_active":"2024-09-16T09:36:12.372177+0000","last_peered":"2024-09-16T09:36:12.372177+0000","last_clean":"2024-09-16T09:36:12.372177+0000","last_became_active":"2024-09-16T09:25:05.710093+0000","last_became_peered":"2024-09-16T09:25:05.710093+0000","last_unstale":"2024-09-16T09:36:12.372177+0000","last_undegraded":"2024-09-16T09:36:12.372177+0000","last_fullsized":"2024-09-16T09:36:12.372177+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:38.238593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:38.238593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"309'7093","reported_seq":13581,"reported_epoch":309,"state":"active+remapped+backfilling","last_fresh":"2024-09-16T09:36:12.882494+0000","last_change":"2024-09-16T09:36:10.740488+0000","last_active":"2024-09-16T09:36:12.882494+0000","last_peered":"2024-09-16T09:36:12.882494+0000","last_clean":"2024-09-16T09:35:04.649840+0000","last_became_active":"2024-09-16T09:35:06.787372+0000","last_became_peered":"2024-09-16T09:35:06.787372+0000","last_unstale":"2024-09-16T09:36:12.882494+0000","last_undegraded":"2024-09-16T09:36:12.882494+0000","last_fullsized":"2024-09-16T09:36:12.882494+0000","mapping_epoch":283,"log_start":"247'3164","ondisk_log_start":"247'3164","created":243,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":5,"last_scrub":"273'4719","last_scrub_stamp":"2024-09-16T09:34:44.662413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:34:44.662413+0000","objects_scrubbed":3181,"log_size":3929,"log_dups_size":0,"ondisk_log_size":3929,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1093,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":132907008,"num_objects":2028,"num_object_clones":0,"num_object_copies":6084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3082,"num_objects_unfound":0,"num_objects_dirty":2028,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2395,"num_write_kb":39744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":492,"num_bytes_recovered":32243712,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,5,6],"avail_no_missing":["7","5","6"],"object_location_counts":[{"shards":"5,6,7","objects":2028}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":528,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.408762+0000","last_change":"2024-09-16T09:33:46.253182+0000","last_active":"2024-09-16T09:36:12.408762+0000","last_peered":"2024-09-16T09:36:12.408762+0000","last_clean":"2024-09-16T09:36:12.408762+0000","last_became_active":"2024-09-16T09:25:07.719380+0000","last_became_peered":"2024-09-16T09:25:07.719380+0000","last_unstale":"2024-09-16T09:36:12.408762+0000","last_undegraded":"2024-09-16T09:36:12.408762+0000","last_fullsized":"2024-09-16T09:36:12.408762+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-16T09:33:46.253114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:33:46.253114+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":181,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.404332+0000","last_change":"2024-09-16T09:34:41.401691+0000","last_active":"2024-09-16T09:36:12.404332+0000","last_peered":"2024-09-16T09:36:12.404332+0000","last_clean":"2024-09-16T09:36:12.404332+0000","last_became_active":"2024-09-16T09:33:30.245048+0000","last_became_peered":"2024-09-16T09:33:30.245048+0000","last_unstale":"2024-09-16T09:36:12.404332+0000","last_undegraded":"2024-09-16T09:36:12.404332+0000","last_fullsized":"2024-09-16T09:36:12.404332+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:34:41.401641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:34:41.401641+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"285'20","reported_seq":599,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-09-16T09:36:08.362534+0000","last_change":"2024-09-16T09:32:30.286077+0000","last_active":"2024-09-16T09:36:08.362534+0000","last_peered":"2024-09-16T09:36:08.362534+0000","last_clean":"2024-09-16T09:36:08.362534+0000","last_became_active":"2024-09-16T09:31:09.275050+0000","last_became_peered":"2024-09-16T09:31:09.275050+0000","last_unstale":"2024-09-16T09:36:08.362534+0000","last_undegraded":"2024-09-16T09:36:08.362534+0000","last_fullsized":"2024-09-16T09:36:08.362534+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-16T09:32:30.284919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:32:30.284919+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"66'8","reported_seq":703,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.445611+0000","last_change":"2024-09-16T09:34:52.922123+0000","last_active":"2024-09-16T09:36:12.445611+0000","last_peered":"2024-09-16T09:36:12.445611+0000","last_clean":"2024-09-16T09:36:12.445611+0000","last_became_active":"2024-09-16T09:25:05.712516+0000","last_became_peered":"2024-09-16T09:25:05.712516+0000","last_unstale":"2024-09-16T09:36:12.445611+0000","last_undegraded":"2024-09-16T09:36:12.445611+0000","last_fullsized":"2024-09-16T09:36:12.445611+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:34:52.922047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:34:52.922047+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:36:15.788853+0000","scrub_duration":33,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2","version":"309'7155","reported_seq":7356,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:13.269078+0000","last_change":"2024-09-16T09:33:53.529863+0000","last_active":"2024-09-16T09:36:13.269078+0000","last_peered":"2024-09-16T09:36:13.269078+0000","last_clean":"2024-09-16T09:36:13.269078+0000","last_became_active":"2024-09-16T09:33:52.933886+0000","last_became_peered":"2024-09-16T09:33:52.933886+0000","last_unstale":"2024-09-16T09:36:13.269078+0000","last_undegraded":"2024-09-16T09:36:13.269078+0000","last_fullsized":"2024-09-16T09:36:13.269078+0000","mapping_epoch":233,"log_start":"247'3226","ondisk_log_start":"247'3226","created":229,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"244'3174","last_scrub_stamp":"2024-09-16T09:33:53.529712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:33:53.529712+0000","objects_scrubbed":1593,"log_size":3929,"log_dups_size":0,"ondisk_log_size":3929,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":449,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":129236992,"num_objects":1972,"num_object_clones":0,"num_object_copies":5916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1972,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3989,"num_write_kb":139776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.a","version":"73'79","reported_seq":463,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.413716+0000","last_change":"2024-09-16T09:29:18.498887+0000","last_active":"2024-09-16T09:36:12.413716+0000","last_peered":"2024-09-16T09:36:12.413716+0000","last_clean":"2024-09-16T09:36:12.413716+0000","last_became_active":"2024-09-16T09:26:35.127352+0000","last_became_peered":"2024-09-16T09:26:35.127352+0000","last_unstale":"2024-09-16T09:36:12.413716+0000","last_undegraded":"2024-09-16T09:36:12.413716+0000","last_fullsized":"2024-09-16T09:36:12.413716+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:18.498766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:18.498766+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"285'27","reported_seq":490,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.433565+0000","last_change":"2024-09-16T09:28:28.065781+0000","last_active":"2024-09-16T09:36:12.433565+0000","last_peered":"2024-09-16T09:36:12.433565+0000","last_clean":"2024-09-16T09:36:12.433565+0000","last_became_active":"2024-09-16T09:26:33.295672+0000","last_became_peered":"2024-09-16T09:26:33.295672+0000","last_unstale":"2024-09-16T09:36:12.433565+0000","last_undegraded":"2024-09-16T09:36:12.433565+0000","last_fullsized":"2024-09-16T09:36:12.433565+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-16T09:28:28.065699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:28.065699+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":266,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.404767+0000","last_change":"2024-09-16T09:34:39.420386+0000","last_active":"2024-09-16T09:36:12.404767+0000","last_peered":"2024-09-16T09:36:12.404767+0000","last_clean":"2024-09-16T09:36:12.404767+0000","last_became_active":"2024-09-16T09:31:09.274534+0000","last_became_peered":"2024-09-16T09:31:09.274534+0000","last_unstale":"2024-09-16T09:36:12.404767+0000","last_undegraded":"2024-09-16T09:36:12.404767+0000","last_fullsized":"2024-09-16T09:36:12.404767+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:39.420314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:34:39.420314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"66'8","reported_seq":659,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.372386+0000","last_change":"2024-09-16T09:35:10.915838+0000","last_active":"2024-09-16T09:36:12.372386+0000","last_peered":"2024-09-16T09:36:12.372386+0000","last_clean":"2024-09-16T09:36:12.372386+0000","last_became_active":"2024-09-16T09:25:05.713813+0000","last_became_peered":"2024-09-16T09:25:05.713813+0000","last_unstale":"2024-09-16T09:36:12.372386+0000","last_undegraded":"2024-09-16T09:36:12.372386+0000","last_fullsized":"2024-09-16T09:36:12.372386+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:35:10.915780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:35:10.915780+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:36:23.797357+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":519,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.414008+0000","last_change":"2024-09-16T09:29:24.537802+0000","last_active":"2024-09-16T09:36:12.414008+0000","last_peered":"2024-09-16T09:36:12.414008+0000","last_clean":"2024-09-16T09:36:12.414008+0000","last_became_active":"2024-09-16T09:25:07.717344+0000","last_became_peered":"2024-09-16T09:25:07.717344+0000","last_unstale":"2024-09-16T09:36:12.414008+0000","last_undegraded":"2024-09-16T09:36:12.414008+0000","last_fullsized":"2024-09-16T09:36:12.414008+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:24.537731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:24.537731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"309'11231","reported_seq":18527,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.859916+0000","last_change":"2024-09-16T09:35:09.710620+0000","last_active":"2024-09-16T09:36:12.859916+0000","last_peered":"2024-09-16T09:36:12.859916+0000","last_clean":"2024-09-16T09:36:12.859916+0000","last_became_active":"2024-09-16T09:35:09.702658+0000","last_became_peered":"2024-09-16T09:35:09.702658+0000","last_unstale":"2024-09-16T09:36:12.859916+0000","last_undegraded":"2024-09-16T09:36:12.859916+0000","last_fullsized":"2024-09-16T09:36:12.859916+0000","mapping_epoch":285,"log_start":"279'7300","ondisk_log_start":"279'7300","created":229,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:33:24.230486+0000","objects_scrubbed":0,"log_size":3931,"log_dups_size":0,"ondisk_log_size":3931,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":258736152,"num_objects":3949,"num_object_clones":0,"num_object_copies":11847,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3949,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11231,"num_write_kb":485697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3554,"num_bytes_recovered":232914944,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.5","version":"73'79","reported_seq":503,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.372881+0000","last_change":"2024-09-16T09:32:42.123139+0000","last_active":"2024-09-16T09:36:12.372881+0000","last_peered":"2024-09-16T09:36:12.372881+0000","last_clean":"2024-09-16T09:36:12.372881+0000","last_became_active":"2024-09-16T09:30:08.763683+0000","last_became_peered":"2024-09-16T09:30:08.763683+0000","last_unstale":"2024-09-16T09:36:12.372881+0000","last_undegraded":"2024-09-16T09:36:12.372881+0000","last_fullsized":"2024-09-16T09:36:12.372881+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:32:42.123067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:32:42.123067+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:36:57.873747+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"285'30","reported_seq":496,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.412678+0000","last_change":"2024-09-16T09:33:21.333996+0000","last_active":"2024-09-16T09:36:12.412678+0000","last_peered":"2024-09-16T09:36:12.412678+0000","last_clean":"2024-09-16T09:36:12.412678+0000","last_became_active":"2024-09-16T09:31:09.270434+0000","last_became_peered":"2024-09-16T09:31:09.270434+0000","last_unstale":"2024-09-16T09:36:12.412678+0000","last_undegraded":"2024-09-16T09:36:12.412678+0000","last_fullsized":"2024-09-16T09:36:12.412678+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'18","last_scrub_stamp":"2024-09-16T09:33:21.333937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:33:21.333937+0000","objects_scrubbed":8,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":51,"num_read_kb":39,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":368,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.410045+0000","last_change":"2024-09-16T09:28:58.705169+0000","last_active":"2024-09-16T09:36:12.410045+0000","last_peered":"2024-09-16T09:36:12.410045+0000","last_clean":"2024-09-16T09:36:12.410045+0000","last_became_active":"2024-09-16T09:28:52.983543+0000","last_became_peered":"2024-09-16T09:28:52.983543+0000","last_unstale":"2024-09-16T09:36:12.410045+0000","last_undegraded":"2024-09-16T09:36:12.410045+0000","last_fullsized":"2024-09-16T09:36:12.410045+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:58.705097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:28:58.705097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":555,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.371612+0000","last_change":"2024-09-16T09:35:19.822713+0000","last_active":"2024-09-16T09:36:12.371612+0000","last_peered":"2024-09-16T09:36:12.371612+0000","last_clean":"2024-09-16T09:36:12.371612+0000","last_became_active":"2024-09-16T09:31:09.268864+0000","last_became_peered":"2024-09-16T09:31:09.268864+0000","last_unstale":"2024-09-16T09:36:12.371612+0000","last_undegraded":"2024-09-16T09:36:12.371612+0000","last_fullsized":"2024-09-16T09:36:12.371612+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:35:19.822635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:35:19.822635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:36:49.698950+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":543,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.371875+0000","last_change":"2024-09-16T09:35:46.933987+0000","last_active":"2024-09-16T09:36:12.371875+0000","last_peered":"2024-09-16T09:36:12.371875+0000","last_clean":"2024-09-16T09:36:12.371875+0000","last_became_active":"2024-09-16T09:31:09.269433+0000","last_became_peered":"2024-09-16T09:31:09.269433+0000","last_unstale":"2024-09-16T09:36:12.371875+0000","last_undegraded":"2024-09-16T09:36:12.371875+0000","last_fullsized":"2024-09-16T09:36:12.371875+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:35:46.933912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:35:46.933912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:37:05.460732+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.c","version":"309'11064","reported_seq":11242,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.877098+0000","last_change":"2024-09-16T09:33:25.371514+0000","last_active":"2024-09-16T09:36:12.877098+0000","last_peered":"2024-09-16T09:36:12.877098+0000","last_clean":"2024-09-16T09:36:12.877098+0000","last_became_active":"2024-09-16T09:33:25.371230+0000","last_became_peered":"2024-09-16T09:33:25.371230+0000","last_unstale":"2024-09-16T09:36:12.877098+0000","last_undegraded":"2024-09-16T09:36:12.877098+0000","last_fullsized":"2024-09-16T09:36:12.877098+0000","mapping_epoch":229,"log_start":"275'6800","ondisk_log_start":"275'6800","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:33:24.230486+0000","objects_scrubbed":0,"log_size":4264,"log_dups_size":0,"ondisk_log_size":4264,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:37:12.807241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":258998272,"num_objects":3952,"num_object_clones":0,"num_object_copies":11856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3952,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11064,"num_write_kb":480512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"73'79","reported_seq":266,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.405930+0000","last_change":"2024-09-16T09:34:42.362203+0000","last_active":"2024-09-16T09:36:12.405930+0000","last_peered":"2024-09-16T09:36:12.405930+0000","last_clean":"2024-09-16T09:36:12.405930+0000","last_became_active":"2024-09-16T09:31:09.275139+0000","last_became_peered":"2024-09-16T09:31:09.275139+0000","last_unstale":"2024-09-16T09:36:12.405930+0000","last_undegraded":"2024-09-16T09:36:12.405930+0000","last_fullsized":"2024-09-16T09:36:12.405930+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:34:42.362121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:34:42.362121+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"285'28","reported_seq":587,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.371621+0000","last_change":"2024-09-16T09:35:43.914378+0000","last_active":"2024-09-16T09:36:12.371621+0000","last_peered":"2024-09-16T09:36:12.371621+0000","last_clean":"2024-09-16T09:36:12.371621+0000","last_became_active":"2024-09-16T09:25:03.708830+0000","last_became_peered":"2024-09-16T09:25:03.708830+0000","last_unstale":"2024-09-16T09:36:12.371621+0000","last_undegraded":"2024-09-16T09:36:12.371621+0000","last_fullsized":"2024-09-16T09:36:12.371621+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'28","last_scrub_stamp":"2024-09-16T09:35:43.914312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:35:43.914312+0000","objects_scrubbed":7,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:36:55.765222+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":552,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-09-16T09:36:08.362717+0000","last_change":"2024-09-16T09:33:43.678231+0000","last_active":"2024-09-16T09:36:08.362717+0000","last_peered":"2024-09-16T09:36:08.362717+0000","last_clean":"2024-09-16T09:36:08.362717+0000","last_became_active":"2024-09-16T09:25:01.702992+0000","last_became_peered":"2024-09-16T09:25:01.702992+0000","last_unstale":"2024-09-16T09:36:08.362717+0000","last_undegraded":"2024-09-16T09:36:08.362717+0000","last_fullsized":"2024-09-16T09:36:08.362717+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:33:43.678138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:43.678138+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"235'12","reported_seq":223,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.371559+0000","last_change":"2024-09-16T09:33:30.251570+0000","last_active":"2024-09-16T09:36:12.371559+0000","last_peered":"2024-09-16T09:36:12.371559+0000","last_clean":"2024-09-16T09:36:12.371559+0000","last_became_active":"2024-09-16T09:33:30.251047+0000","last_became_peered":"2024-09-16T09:33:30.251047+0000","last_unstale":"2024-09-16T09:36:12.371559+0000","last_undegraded":"2024-09-16T09:36:12.371559+0000","last_fullsized":"2024-09-16T09:36:12.371559+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"187'11","last_scrub_stamp":"2024-09-16T09:33:09.417852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:09.417852+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:37:07.764604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":512,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.408915+0000","last_change":"2024-09-16T09:26:40.699888+0000","last_active":"2024-09-16T09:36:12.408915+0000","last_peered":"2024-09-16T09:36:12.408915+0000","last_clean":"2024-09-16T09:36:12.408915+0000","last_became_active":"2024-09-16T09:25:07.722017+0000","last_became_peered":"2024-09-16T09:25:07.722017+0000","last_unstale":"2024-09-16T09:36:12.408915+0000","last_undegraded":"2024-09-16T09:36:12.408915+0000","last_fullsized":"2024-09-16T09:36:12.408915+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:40.699813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:40.699813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"309'11442","reported_seq":11621,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:13.108701+0000","last_change":"2024-09-16T09:33:25.372320+0000","last_active":"2024-09-16T09:36:13.108701+0000","last_peered":"2024-09-16T09:36:13.108701+0000","last_clean":"2024-09-16T09:36:13.108701+0000","last_became_active":"2024-09-16T09:33:25.372005+0000","last_became_peered":"2024-09-16T09:33:25.372005+0000","last_unstale":"2024-09-16T09:36:13.108701+0000","last_undegraded":"2024-09-16T09:36:13.108701+0000","last_fullsized":"2024-09-16T09:36:13.108701+0000","mapping_epoch":229,"log_start":"275'6900","ondisk_log_start":"275'6900","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:33:24.230486+0000","objects_scrubbed":0,"log_size":4542,"log_dups_size":0,"ondisk_log_size":4542,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":259915776,"num_objects":3966,"num_object_clones":0,"num_object_copies":11898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3966,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11442,"num_write_kb":493056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.7","version":"73'79","reported_seq":597,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.445021+0000","last_change":"2024-09-16T09:34:51.864131+0000","last_active":"2024-09-16T09:36:12.445021+0000","last_peered":"2024-09-16T09:36:12.445021+0000","last_clean":"2024-09-16T09:36:12.445021+0000","last_became_active":"2024-09-16T09:31:09.274707+0000","last_became_peered":"2024-09-16T09:31:09.274707+0000","last_unstale":"2024-09-16T09:36:12.445021+0000","last_undegraded":"2024-09-16T09:36:12.445021+0000","last_fullsized":"2024-09-16T09:36:12.445021+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:34:51.864036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:34:51.864036+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:36:20.395033+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"285'20","reported_seq":222,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.414012+0000","last_change":"2024-09-16T09:33:41.368102+0000","last_active":"2024-09-16T09:36:12.414012+0000","last_peered":"2024-09-16T09:36:12.414012+0000","last_clean":"2024-09-16T09:36:12.414012+0000","last_became_active":"2024-09-16T09:33:30.248453+0000","last_became_peered":"2024-09-16T09:33:30.248453+0000","last_unstale":"2024-09-16T09:36:12.414012+0000","last_undegraded":"2024-09-16T09:36:12.414012+0000","last_fullsized":"2024-09-16T09:36:12.414012+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-16T09:33:41.368016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:33:41.368016+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":546,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.414402+0000","last_change":"2024-09-16T09:32:46.388711+0000","last_active":"2024-09-16T09:36:12.414402+0000","last_peered":"2024-09-16T09:36:12.414402+0000","last_clean":"2024-09-16T09:36:12.414402+0000","last_became_active":"2024-09-16T09:31:09.270619+0000","last_became_peered":"2024-09-16T09:31:09.270619+0000","last_unstale":"2024-09-16T09:36:12.414402+0000","last_undegraded":"2024-09-16T09:36:12.414402+0000","last_fullsized":"2024-09-16T09:36:12.414402+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:46.388651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:32:46.388651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"66'8","reported_seq":661,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-09-16T09:36:08.363176+0000","last_change":"2024-09-16T09:28:57.628615+0000","last_active":"2024-09-16T09:36:08.363176+0000","last_peered":"2024-09-16T09:36:08.363176+0000","last_clean":"2024-09-16T09:36:08.363176+0000","last_became_active":"2024-09-16T09:25:05.714507+0000","last_became_peered":"2024-09-16T09:25:05.714507+0000","last_unstale":"2024-09-16T09:36:08.363176+0000","last_undegraded":"2024-09-16T09:36:08.363176+0000","last_fullsized":"2024-09-16T09:36:08.363176+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:28:57.628543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:28:57.628543+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":537,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-09-16T09:36:08.362000+0000","last_change":"2024-09-16T09:32:33.362233+0000","last_active":"2024-09-16T09:36:08.362000+0000","last_peered":"2024-09-16T09:36:08.362000+0000","last_clean":"2024-09-16T09:36:08.362000+0000","last_became_active":"2024-09-16T09:31:09.283192+0000","last_became_peered":"2024-09-16T09:31:09.283192+0000","last_unstale":"2024-09-16T09:36:08.362000+0000","last_undegraded":"2024-09-16T09:36:08.362000+0000","last_fullsized":"2024-09-16T09:36:08.362000+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:33.362188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:32:33.362188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.e","version":"309'11270","reported_seq":11448,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.871379+0000","last_change":"2024-09-16T09:33:25.247961+0000","last_active":"2024-09-16T09:36:12.871379+0000","last_peered":"2024-09-16T09:36:12.871379+0000","last_clean":"2024-09-16T09:36:12.871379+0000","last_became_active":"2024-09-16T09:33:25.247683+0000","last_became_peered":"2024-09-16T09:33:25.247683+0000","last_unstale":"2024-09-16T09:36:12.871379+0000","last_undegraded":"2024-09-16T09:36:12.871379+0000","last_fullsized":"2024-09-16T09:36:12.871379+0000","mapping_epoch":229,"log_start":"279'7400","ondisk_log_start":"279'7400","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:33:24.230486+0000","objects_scrubbed":0,"log_size":3870,"log_dups_size":0,"ondisk_log_size":3870,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":265027584,"num_objects":4044,"num_object_clones":0,"num_object_copies":12132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4044,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11270,"num_write_kb":490048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.6","version":"73'79","reported_seq":570,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.445021+0000","last_change":"2024-09-16T09:34:42.728232+0000","last_active":"2024-09-16T09:36:12.445021+0000","last_peered":"2024-09-16T09:36:12.445021+0000","last_clean":"2024-09-16T09:36:12.445021+0000","last_became_active":"2024-09-16T09:31:09.274565+0000","last_became_peered":"2024-09-16T09:31:09.274565+0000","last_unstale":"2024-09-16T09:36:12.445021+0000","last_undegraded":"2024-09-16T09:36:12.445021+0000","last_fullsized":"2024-09-16T09:36:12.445021+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:34:42.728143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:34:42.728143+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"285'29","reported_seq":574,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-09-16T09:36:08.363323+0000","last_change":"2024-09-16T09:31:35.249952+0000","last_active":"2024-09-16T09:36:08.363323+0000","last_peered":"2024-09-16T09:36:08.363323+0000","last_clean":"2024-09-16T09:36:08.363323+0000","last_became_active":"2024-09-16T09:25:03.714797+0000","last_became_peered":"2024-09-16T09:25:03.714797+0000","last_unstale":"2024-09-16T09:36:08.363323+0000","last_undegraded":"2024-09-16T09:36:08.363323+0000","last_fullsized":"2024-09-16T09:36:08.363323+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-16T09:31:35.249864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:31:35.249864+0000","objects_scrubbed":8,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":54,"num_read_kb":40,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":164,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.414014+0000","last_change":"2024-09-16T09:33:30.247311+0000","last_active":"2024-09-16T09:36:12.414014+0000","last_peered":"2024-09-16T09:36:12.414014+0000","last_clean":"2024-09-16T09:36:12.414014+0000","last_became_active":"2024-09-16T09:33:30.246996+0000","last_became_peered":"2024-09-16T09:33:30.246996+0000","last_unstale":"2024-09-16T09:36:12.414014+0000","last_undegraded":"2024-09-16T09:36:12.414014+0000","last_fullsized":"2024-09-16T09:36:12.414014+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:11.407616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:11.407616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,2],"acting":[5,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":549,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.444512+0000","last_change":"2024-09-16T09:34:45.772748+0000","last_active":"2024-09-16T09:36:12.444512+0000","last_peered":"2024-09-16T09:36:12.444512+0000","last_clean":"2024-09-16T09:36:12.444512+0000","last_became_active":"2024-09-16T09:25:05.708663+0000","last_became_peered":"2024-09-16T09:25:05.708663+0000","last_unstale":"2024-09-16T09:36:12.444512+0000","last_undegraded":"2024-09-16T09:36:12.444512+0000","last_fullsized":"2024-09-16T09:36:12.444512+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:45.772674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:34:45.772674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":172,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.444437+0000","last_change":"2024-09-16T09:34:36.806165+0000","last_active":"2024-09-16T09:36:12.444437+0000","last_peered":"2024-09-16T09:36:12.444437+0000","last_clean":"2024-09-16T09:36:12.444437+0000","last_became_active":"2024-09-16T09:33:30.250178+0000","last_became_peered":"2024-09-16T09:33:30.250178+0000","last_unstale":"2024-09-16T09:36:12.444437+0000","last_undegraded":"2024-09-16T09:36:12.444437+0000","last_fullsized":"2024-09-16T09:36:12.444437+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:36.805971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:34:36.805971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,2],"acting":[6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"303'7145","reported_seq":7392,"reported_epoch":305,"state":"peering","last_fresh":"2024-09-16T09:36:07.440282+0000","last_change":"2024-09-16T09:36:08.416123+0000","last_active":"2024-09-16T09:36:06.349804+0000","last_peered":"2024-09-16T09:36:07.440282+0000","last_clean":"2024-09-16T09:36:07.440282+0000","last_became_active":"2024-09-16T09:33:52.910861+0000","last_became_peered":"2024-09-16T09:36:07.359144+0000","last_unstale":"2024-09-16T09:36:07.440282+0000","last_undegraded":"2024-09-16T09:36:07.440282+0000","last_fullsized":"2024-09-16T09:36:07.440282+0000","mapping_epoch":229,"log_start":"303'7145","ondisk_log_start":"303'7145","created":229,"last_epoch_clean":305,"parent":"0.0","parent_split_bits":0,"last_scrub":"291'6329","last_scrub_stamp":"2024-09-16T09:35:36.136797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:35:36.136797+0000","objects_scrubbed":2998,"log_size":0,"log_dups_size":6000,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":920,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":280494080,"num_objects":4280,"num_object_clones":0,"num_object_copies":12840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4280,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3590,"num_write_kb":71616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2504,"num_bytes_recovered":164102144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,5],"acting":[0,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5,7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"73'79","reported_seq":495,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.413673+0000","last_change":"2024-09-16T09:32:43.420831+0000","last_active":"2024-09-16T09:36:12.413673+0000","last_peered":"2024-09-16T09:36:12.413673+0000","last_clean":"2024-09-16T09:36:12.413673+0000","last_became_active":"2024-09-16T09:30:08.766350+0000","last_became_peered":"2024-09-16T09:30:08.766350+0000","last_unstale":"2024-09-16T09:36:12.413673+0000","last_undegraded":"2024-09-16T09:36:12.413673+0000","last_fullsized":"2024-09-16T09:36:12.413673+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:32:43.420784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:32:43.420784+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"285'42","reported_seq":803,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.407921+0000","last_change":"2024-09-16T09:34:47.032891+0000","last_active":"2024-09-16T09:36:12.407921+0000","last_peered":"2024-09-16T09:36:12.407921+0000","last_clean":"2024-09-16T09:36:12.407921+0000","last_became_active":"2024-09-16T09:31:09.278839+0000","last_became_peered":"2024-09-16T09:31:09.278839+0000","last_unstale":"2024-09-16T09:36:12.407921+0000","last_undegraded":"2024-09-16T09:36:12.407921+0000","last_fullsized":"2024-09-16T09:36:12.407921+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"187'30","last_scrub_stamp":"2024-09-16T09:34:47.032800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:34:47.032800+0000","objects_scrubbed":10,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":26,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":66,"num_read_kb":51,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":541,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.372174+0000","last_change":"2024-09-16T09:35:12.897657+0000","last_active":"2024-09-16T09:36:12.372174+0000","last_peered":"2024-09-16T09:36:12.372174+0000","last_clean":"2024-09-16T09:36:12.372174+0000","last_became_active":"2024-09-16T09:25:01.703076+0000","last_became_peered":"2024-09-16T09:25:01.703076+0000","last_unstale":"2024-09-16T09:36:12.372174+0000","last_undegraded":"2024-09-16T09:36:12.372174+0000","last_fullsized":"2024-09-16T09:36:12.372174+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:35:12.897588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:35:12.897588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:36:29.683584+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":550,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.414208+0000","last_change":"2024-09-16T09:33:38.417375+0000","last_active":"2024-09-16T09:36:12.414208+0000","last_peered":"2024-09-16T09:36:12.414208+0000","last_clean":"2024-09-16T09:36:12.414208+0000","last_became_active":"2024-09-16T09:25:00.696237+0000","last_became_peered":"2024-09-16T09:25:00.696237+0000","last_unstale":"2024-09-16T09:36:12.414208+0000","last_undegraded":"2024-09-16T09:36:12.414208+0000","last_fullsized":"2024-09-16T09:36:12.414208+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-16T09:33:38.417308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:33:38.417308+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.0025137430000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":511,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-09-16T09:36:08.362233+0000","last_change":"2024-09-16T09:33:16.666704+0000","last_active":"2024-09-16T09:36:08.362233+0000","last_peered":"2024-09-16T09:36:08.362233+0000","last_clean":"2024-09-16T09:36:08.362233+0000","last_became_active":"2024-09-16T09:25:05.717591+0000","last_became_peered":"2024-09-16T09:25:05.717591+0000","last_unstale":"2024-09-16T09:36:08.362233+0000","last_undegraded":"2024-09-16T09:36:08.362233+0000","last_fullsized":"2024-09-16T09:36:08.362233+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:16.666648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:16.666648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":258,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.406810+0000","last_change":"2024-09-16T09:31:09.272967+0000","last_active":"2024-09-16T09:36:12.406810+0000","last_peered":"2024-09-16T09:36:12.406810+0000","last_clean":"2024-09-16T09:36:12.406810+0000","last_became_active":"2024-09-16T09:31:09.272609+0000","last_became_peered":"2024-09-16T09:31:09.272609+0000","last_unstale":"2024-09-16T09:36:12.406810+0000","last_undegraded":"2024-09-16T09:36:12.406810+0000","last_fullsized":"2024-09-16T09:36:12.406810+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:45.201503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:45.201503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.a","version":"309'11052","reported_seq":11231,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:13.135118+0000","last_change":"2024-09-16T09:33:25.244002+0000","last_active":"2024-09-16T09:36:13.135118+0000","last_peered":"2024-09-16T09:36:13.135118+0000","last_clean":"2024-09-16T09:36:13.135118+0000","last_became_active":"2024-09-16T09:33:25.243701+0000","last_became_peered":"2024-09-16T09:33:25.243701+0000","last_unstale":"2024-09-16T09:36:13.135118+0000","last_undegraded":"2024-09-16T09:36:13.135118+0000","last_fullsized":"2024-09-16T09:36:13.135118+0000","mapping_epoch":229,"log_start":"274'6500","ondisk_log_start":"274'6500","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:33:24.230486+0000","objects_scrubbed":0,"log_size":4552,"log_dups_size":0,"ondisk_log_size":4552,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":257294336,"num_objects":3926,"num_object_clones":0,"num_object_copies":11778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3926,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11052,"num_write_kb":479296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"73'79","reported_seq":214,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-09-16T09:36:08.364623+0000","last_change":"2024-09-16T09:34:32.717994+0000","last_active":"2024-09-16T09:36:08.364623+0000","last_peered":"2024-09-16T09:36:08.364623+0000","last_clean":"2024-09-16T09:36:08.364623+0000","last_became_active":"2024-09-16T09:34:00.336514+0000","last_became_peered":"2024-09-16T09:34:00.336514+0000","last_unstale":"2024-09-16T09:36:08.364623+0000","last_undegraded":"2024-09-16T09:36:08.364623+0000","last_fullsized":"2024-09-16T09:36:08.364623+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:34:32.717915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:34:32.717915+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"285'17","reported_seq":236,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.414866+0000","last_change":"2024-09-16T09:34:15.319682+0000","last_active":"2024-09-16T09:36:12.414866+0000","last_peered":"2024-09-16T09:36:12.414866+0000","last_clean":"2024-09-16T09:36:12.414866+0000","last_became_active":"2024-09-16T09:34:00.337121+0000","last_became_peered":"2024-09-16T09:34:00.337121+0000","last_unstale":"2024-09-16T09:36:12.414866+0000","last_undegraded":"2024-09-16T09:36:12.414866+0000","last_fullsized":"2024-09-16T09:36:12.414866+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'13","last_scrub_stamp":"2024-09-16T09:27:44.155969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:44.155969+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":35,"num_read_kb":24,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":164,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.444876+0000","last_change":"2024-09-16T09:33:30.248260+0000","last_active":"2024-09-16T09:36:12.444876+0000","last_peered":"2024-09-16T09:36:12.444876+0000","last_clean":"2024-09-16T09:36:12.444876+0000","last_became_active":"2024-09-16T09:33:30.247596+0000","last_became_peered":"2024-09-16T09:33:30.247596+0000","last_unstale":"2024-09-16T09:36:12.444876+0000","last_undegraded":"2024-09-16T09:36:12.444876+0000","last_fullsized":"2024-09-16T09:36:12.444876+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:22.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:22.331302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,2],"acting":[6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":182,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-09-16T09:36:08.362312+0000","last_change":"2024-09-16T09:33:21.338149+0000","last_active":"2024-09-16T09:36:08.362312+0000","last_peered":"2024-09-16T09:36:08.362312+0000","last_clean":"2024-09-16T09:36:08.362312+0000","last_became_active":"2024-09-16T09:33:21.337177+0000","last_became_peered":"2024-09-16T09:33:21.337177+0000","last_unstale":"2024-09-16T09:36:08.362312+0000","last_undegraded":"2024-09-16T09:36:08.362312+0000","last_fullsized":"2024-09-16T09:36:08.362312+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:32:40.165749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:32:40.165749+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":549,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.433608+0000","last_change":"2024-09-16T09:33:40.418799+0000","last_active":"2024-09-16T09:36:12.433608+0000","last_peered":"2024-09-16T09:36:12.433608+0000","last_clean":"2024-09-16T09:36:12.433608+0000","last_became_active":"2024-09-16T09:25:05.711539+0000","last_became_peered":"2024-09-16T09:25:05.711539+0000","last_unstale":"2024-09-16T09:36:12.433608+0000","last_undegraded":"2024-09-16T09:36:12.433608+0000","last_fullsized":"2024-09-16T09:36:12.433608+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:40.417644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:40.417644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":528,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.409026+0000","last_change":"2024-09-16T09:33:39.313624+0000","last_active":"2024-09-16T09:36:12.409026+0000","last_peered":"2024-09-16T09:36:12.409026+0000","last_clean":"2024-09-16T09:36:12.409026+0000","last_became_active":"2024-09-16T09:25:07.723282+0000","last_became_peered":"2024-09-16T09:25:07.723282+0000","last_unstale":"2024-09-16T09:36:12.409026+0000","last_undegraded":"2024-09-16T09:36:12.409026+0000","last_fullsized":"2024-09-16T09:36:12.409026+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:39.313562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:33:39.313562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.b","version":"304'10745","reported_seq":10918,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-09-16T09:36:08.363322+0000","last_change":"2024-09-16T09:33:25.245108+0000","last_active":"2024-09-16T09:36:08.363322+0000","last_peered":"2024-09-16T09:36:08.363322+0000","last_clean":"2024-09-16T09:36:08.363322+0000","last_became_active":"2024-09-16T09:33:25.244819+0000","last_became_peered":"2024-09-16T09:33:25.244819+0000","last_unstale":"2024-09-16T09:36:08.363322+0000","last_undegraded":"2024-09-16T09:36:08.363322+0000","last_fullsized":"2024-09-16T09:36:08.363322+0000","mapping_epoch":229,"log_start":"271'5900","ondisk_log_start":"271'5900","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:33:24.230486+0000","objects_scrubbed":0,"log_size":4845,"log_dups_size":0,"ondisk_log_size":4845,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":270991360,"num_objects":4135,"num_object_clones":0,"num_object_copies":12405,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4135,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10745,"num_write_kb":476160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"73'79","reported_seq":253,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.444716+0000","last_change":"2024-09-16T09:34:31.929462+0000","last_active":"2024-09-16T09:36:12.444716+0000","last_peered":"2024-09-16T09:36:12.444716+0000","last_clean":"2024-09-16T09:36:12.444716+0000","last_became_active":"2024-09-16T09:33:30.249309+0000","last_became_peered":"2024-09-16T09:33:30.249309+0000","last_unstale":"2024-09-16T09:36:12.444716+0000","last_undegraded":"2024-09-16T09:36:12.444716+0000","last_fullsized":"2024-09-16T09:36:12.444716+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:34:31.929388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:34:31.929388+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.6","version":"285'24","reported_seq":559,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-09-16T09:36:08.362614+0000","last_change":"2024-09-16T09:32:48.547237+0000","last_active":"2024-09-16T09:36:08.362614+0000","last_peered":"2024-09-16T09:36:08.362614+0000","last_clean":"2024-09-16T09:36:08.362614+0000","last_became_active":"2024-09-16T09:25:03.715315+0000","last_became_peered":"2024-09-16T09:25:03.715315+0000","last_unstale":"2024-09-16T09:36:08.362614+0000","last_undegraded":"2024-09-16T09:36:08.362614+0000","last_fullsized":"2024-09-16T09:36:08.362614+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-16T09:32:48.546869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:32:48.546869+0000","objects_scrubbed":6,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":36,"num_read_kb":30,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"59'1","reported_seq":558,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-09-16T09:36:08.361431+0000","last_change":"2024-09-16T09:33:21.682893+0000","last_active":"2024-09-16T09:36:08.361431+0000","last_peered":"2024-09-16T09:36:08.361431+0000","last_clean":"2024-09-16T09:36:08.361431+0000","last_became_active":"2024-09-16T09:31:09.269627+0000","last_became_peered":"2024-09-16T09:31:09.269627+0000","last_unstale":"2024-09-16T09:36:08.361431+0000","last_undegraded":"2024-09-16T09:36:08.361431+0000","last_fullsized":"2024-09-16T09:36:08.361431+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:33:21.682840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:21.682840+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1986,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"67'5","reported_seq":1216,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.412815+0000","last_change":"2024-09-16T09:33:37.358839+0000","last_active":"2024-09-16T09:36:12.412815+0000","last_peered":"2024-09-16T09:36:12.412815+0000","last_clean":"2024-09-16T09:36:12.412815+0000","last_became_active":"2024-09-16T09:25:00.696546+0000","last_became_peered":"2024-09-16T09:25:00.696546+0000","last_unstale":"2024-09-16T09:36:12.412815+0000","last_undegraded":"2024-09-16T09:36:12.412815+0000","last_fullsized":"2024-09-16T09:36:12.412815+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'5","last_scrub_stamp":"2024-09-16T09:33:37.358741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:33:37.358741+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0.0059770420000000001,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":665,"num_read_kb":660,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":516,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.414870+0000","last_change":"2024-09-16T09:27:07.337170+0000","last_active":"2024-09-16T09:36:12.414870+0000","last_peered":"2024-09-16T09:36:12.414870+0000","last_clean":"2024-09-16T09:36:12.414870+0000","last_became_active":"2024-09-16T09:25:05.714704+0000","last_became_peered":"2024-09-16T09:25:05.714704+0000","last_unstale":"2024-09-16T09:36:12.414870+0000","last_undegraded":"2024-09-16T09:36:12.414870+0000","last_fullsized":"2024-09-16T09:36:12.414870+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:07.337124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:07.337124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":434,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.414202+0000","last_change":"2024-09-16T09:33:30.246166+0000","last_active":"2024-09-16T09:36:12.414202+0000","last_peered":"2024-09-16T09:36:12.414202+0000","last_clean":"2024-09-16T09:36:12.414202+0000","last_became_active":"2024-09-16T09:33:30.245852+0000","last_became_peered":"2024-09-16T09:33:30.245852+0000","last_unstale":"2024-09-16T09:36:12.414202+0000","last_undegraded":"2024-09-16T09:36:12.414202+0000","last_fullsized":"2024-09-16T09:36:12.414202+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:46.252700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:32:46.252700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"304'7046","reported_seq":12594,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-09-16T09:36:08.362237+0000","last_change":"2024-09-16T09:34:42.510673+0000","last_active":"2024-09-16T09:36:08.362237+0000","last_peered":"2024-09-16T09:36:08.362237+0000","last_clean":"2024-09-16T09:36:08.362237+0000","last_became_active":"2024-09-16T09:34:41.432414+0000","last_became_peered":"2024-09-16T09:34:41.432414+0000","last_unstale":"2024-09-16T09:36:08.362237+0000","last_undegraded":"2024-09-16T09:36:08.362237+0000","last_fullsized":"2024-09-16T09:36:08.362237+0000","mapping_epoch":272,"log_start":"236'1526","ondisk_log_start":"236'1526","created":229,"last_epoch_clean":273,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'4604","last_scrub_stamp":"2024-09-16T09:34:42.510459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:34:42.510459+0000","objects_scrubbed":3078,"log_size":5520,"log_dups_size":0,"ondisk_log_size":5520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":697,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":138936320,"num_objects":2120,"num_object_clones":0,"num_object_copies":6360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2120,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2456,"num_write_kb":47936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"285'28","reported_seq":720,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.445322+0000","last_change":"2024-09-16T09:33:16.891850+0000","last_active":"2024-09-16T09:36:12.445322+0000","last_peered":"2024-09-16T09:36:12.445322+0000","last_clean":"2024-09-16T09:36:12.445322+0000","last_became_active":"2024-09-16T09:25:03.714118+0000","last_became_peered":"2024-09-16T09:25:03.714118+0000","last_unstale":"2024-09-16T09:36:12.445322+0000","last_undegraded":"2024-09-16T09:36:12.445322+0000","last_fullsized":"2024-09-16T09:36:12.445322+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'16","last_scrub_stamp":"2024-09-16T09:33:16.891761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:33:16.891761+0000","objects_scrubbed":8,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":43,"num_read_kb":33,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":382,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.414131+0000","last_change":"2024-09-16T09:33:12.171507+0000","last_active":"2024-09-16T09:36:12.414131+0000","last_peered":"2024-09-16T09:36:12.414131+0000","last_clean":"2024-09-16T09:36:12.414131+0000","last_became_active":"2024-09-16T09:31:09.270196+0000","last_became_peered":"2024-09-16T09:31:09.270196+0000","last_unstale":"2024-09-16T09:36:12.414131+0000","last_undegraded":"2024-09-16T09:36:12.414131+0000","last_fullsized":"2024-09-16T09:36:12.414131+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:12.171439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:12.171439+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"73'79","reported_seq":620,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.412850+0000","last_change":"2024-09-16T09:32:43.535041+0000","last_active":"2024-09-16T09:36:12.412850+0000","last_peered":"2024-09-16T09:36:12.412850+0000","last_clean":"2024-09-16T09:36:12.412850+0000","last_became_active":"2024-09-16T09:30:08.870394+0000","last_became_peered":"2024-09-16T09:30:08.870394+0000","last_unstale":"2024-09-16T09:36:12.412850+0000","last_undegraded":"2024-09-16T09:36:12.412850+0000","last_fullsized":"2024-09-16T09:36:12.412850+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:32:43.534962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:32:43.534962+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":524,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.408872+0000","last_change":"2024-09-16T09:33:52.291748+0000","last_active":"2024-09-16T09:36:12.408872+0000","last_peered":"2024-09-16T09:36:12.408872+0000","last_clean":"2024-09-16T09:36:12.408872+0000","last_became_active":"2024-09-16T09:25:05.714341+0000","last_became_peered":"2024-09-16T09:25:05.714341+0000","last_unstale":"2024-09-16T09:36:12.408872+0000","last_undegraded":"2024-09-16T09:36:12.408872+0000","last_fullsized":"2024-09-16T09:36:12.408872+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:52.291702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:52.291702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":520,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.414818+0000","last_change":"2024-09-16T09:29:00.243528+0000","last_active":"2024-09-16T09:36:12.414818+0000","last_peered":"2024-09-16T09:36:12.414818+0000","last_clean":"2024-09-16T09:36:12.414818+0000","last_became_active":"2024-09-16T09:25:07.714281+0000","last_became_peered":"2024-09-16T09:25:07.714281+0000","last_unstale":"2024-09-16T09:36:12.414818+0000","last_undegraded":"2024-09-16T09:36:12.414818+0000","last_fullsized":"2024-09-16T09:36:12.414818+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:00.243482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:00.243482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3","version":"309'7055","reported_seq":7265,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.831026+0000","last_change":"2024-09-16T09:33:53.717144+0000","last_active":"2024-09-16T09:36:12.831026+0000","last_peered":"2024-09-16T09:36:12.831026+0000","last_clean":"2024-09-16T09:36:12.831026+0000","last_became_active":"2024-09-16T09:33:52.914057+0000","last_became_peered":"2024-09-16T09:33:52.914057+0000","last_unstale":"2024-09-16T09:36:12.831026+0000","last_undegraded":"2024-09-16T09:36:12.831026+0000","last_fullsized":"2024-09-16T09:36:12.831026+0000","mapping_epoch":229,"log_start":"247'3156","ondisk_log_start":"247'3156","created":229,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"244'3096","last_scrub_stamp":"2024-09-16T09:33:53.717081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:33:53.717081+0000","objects_scrubbed":1554,"log_size":3899,"log_dups_size":0,"ondisk_log_size":3899,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":129630208,"num_objects":1978,"num_object_clones":0,"num_object_copies":5934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1978,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3974,"num_write_kb":140736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.b","version":"73'79","reported_seq":350,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-09-16T09:36:08.361426+0000","last_change":"2024-09-16T09:34:31.720971+0000","last_active":"2024-09-16T09:36:08.361426+0000","last_peered":"2024-09-16T09:36:08.361426+0000","last_clean":"2024-09-16T09:36:08.361426+0000","last_became_active":"2024-09-16T09:33:30.251933+0000","last_became_peered":"2024-09-16T09:33:30.251933+0000","last_unstale":"2024-09-16T09:36:08.361426+0000","last_undegraded":"2024-09-16T09:36:08.361426+0000","last_fullsized":"2024-09-16T09:36:08.361426+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:34:31.720787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:34:31.720787+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"285'19","reported_seq":597,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.372391+0000","last_change":"2024-09-16T09:35:27.813299+0000","last_active":"2024-09-16T09:36:12.372391+0000","last_peered":"2024-09-16T09:36:12.372391+0000","last_clean":"2024-09-16T09:36:12.372391+0000","last_became_active":"2024-09-16T09:34:00.328533+0000","last_became_peered":"2024-09-16T09:34:00.328533+0000","last_unstale":"2024-09-16T09:36:12.372391+0000","last_undegraded":"2024-09-16T09:36:12.372391+0000","last_fullsized":"2024-09-16T09:36:12.372391+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'19","last_scrub_stamp":"2024-09-16T09:35:27.813206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:35:27.813206+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:36:47.784558+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[4,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":525,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.371948+0000","last_change":"2024-09-16T09:33:23.200213+0000","last_active":"2024-09-16T09:36:12.371948+0000","last_peered":"2024-09-16T09:36:12.371948+0000","last_clean":"2024-09-16T09:36:12.371948+0000","last_became_active":"2024-09-16T09:25:01.708270+0000","last_became_peered":"2024-09-16T09:25:01.708270+0000","last_unstale":"2024-09-16T09:36:12.371948+0000","last_undegraded":"2024-09-16T09:36:12.371948+0000","last_fullsized":"2024-09-16T09:36:12.371948+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:23.200166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:23.200166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:37:04.837585+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":515,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.414710+0000","last_change":"2024-09-16T09:26:34.171190+0000","last_active":"2024-09-16T09:36:12.414710+0000","last_peered":"2024-09-16T09:36:12.414710+0000","last_clean":"2024-09-16T09:36:12.414710+0000","last_became_active":"2024-09-16T09:25:05.711964+0000","last_became_peered":"2024-09-16T09:25:05.711964+0000","last_unstale":"2024-09-16T09:36:12.414710+0000","last_undegraded":"2024-09-16T09:36:12.414710+0000","last_fullsized":"2024-09-16T09:36:12.414710+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:34.171139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:26:34.171139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":407,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.444947+0000","last_change":"2024-09-16T09:34:50.866121+0000","last_active":"2024-09-16T09:36:12.444947+0000","last_peered":"2024-09-16T09:36:12.444947+0000","last_clean":"2024-09-16T09:36:12.444947+0000","last_became_active":"2024-09-16T09:31:09.278618+0000","last_became_peered":"2024-09-16T09:31:09.278618+0000","last_unstale":"2024-09-16T09:36:12.444947+0000","last_undegraded":"2024-09-16T09:36:12.444947+0000","last_fullsized":"2024-09-16T09:36:12.444947+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:50.866032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:34:50.866032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.0","version":"309'7185","reported_seq":7413,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.663069+0000","last_change":"2024-09-16T09:35:37.795621+0000","last_active":"2024-09-16T09:36:12.663069+0000","last_peered":"2024-09-16T09:36:12.663069+0000","last_clean":"2024-09-16T09:36:12.663069+0000","last_became_active":"2024-09-16T09:33:52.914803+0000","last_became_peered":"2024-09-16T09:33:52.914803+0000","last_unstale":"2024-09-16T09:36:12.663069+0000","last_undegraded":"2024-09-16T09:36:12.663069+0000","last_fullsized":"2024-09-16T09:36:12.663069+0000","mapping_epoch":233,"log_start":"236'1373","ondisk_log_start":"236'1373","created":229,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"291'6312","last_scrub_stamp":"2024-09-16T09:35:37.795459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:35:37.795459+0000","objects_scrubbed":2847,"log_size":5812,"log_dups_size":0,"ondisk_log_size":5812,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:36:56.511051+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":128778240,"num_objects":1965,"num_object_clones":0,"num_object_copies":5895,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1965,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":882,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,2],"acting":[6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.8","version":"73'79","reported_seq":148,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.444230+0000","last_change":"2024-09-16T09:34:30.931922+0000","last_active":"2024-09-16T09:36:12.444230+0000","last_peered":"2024-09-16T09:36:12.444230+0000","last_clean":"2024-09-16T09:36:12.444230+0000","last_became_active":"2024-09-16T09:34:00.329241+0000","last_became_peered":"2024-09-16T09:34:00.329241+0000","last_unstale":"2024-09-16T09:36:12.444230+0000","last_undegraded":"2024-09-16T09:36:12.444230+0000","last_fullsized":"2024-09-16T09:36:12.444230+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:34:30.931863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:34:30.931863+0000","objects_scrubbed":1,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262688,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,3],"acting":[6,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"285'31","reported_seq":593,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.372783+0000","last_change":"2024-09-16T09:35:28.861520+0000","last_active":"2024-09-16T09:36:12.372783+0000","last_peered":"2024-09-16T09:36:12.372783+0000","last_clean":"2024-09-16T09:36:12.372783+0000","last_became_active":"2024-09-16T09:25:03.713358+0000","last_became_peered":"2024-09-16T09:25:03.713358+0000","last_unstale":"2024-09-16T09:36:12.372783+0000","last_undegraded":"2024-09-16T09:36:12.372783+0000","last_fullsized":"2024-09-16T09:36:12.372783+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'31","last_scrub_stamp":"2024-09-16T09:35:28.861433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:35:28.861433+0000","objects_scrubbed":8,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:36:48.852231+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":50,"num_read_kb":40,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":533,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.444689+0000","last_change":"2024-09-16T09:34:32.899965+0000","last_active":"2024-09-16T09:36:12.444689+0000","last_peered":"2024-09-16T09:36:12.444689+0000","last_clean":"2024-09-16T09:36:12.444689+0000","last_became_active":"2024-09-16T09:25:01.703355+0000","last_became_peered":"2024-09-16T09:25:01.703355+0000","last_unstale":"2024-09-16T09:36:12.444689+0000","last_undegraded":"2024-09-16T09:36:12.444689+0000","last_fullsized":"2024-09-16T09:36:12.444689+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:32.899886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:34:32.899886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":172,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.372065+0000","last_change":"2024-09-16T09:33:44.298324+0000","last_active":"2024-09-16T09:36:12.372065+0000","last_peered":"2024-09-16T09:36:12.372065+0000","last_clean":"2024-09-16T09:36:12.372065+0000","last_became_active":"2024-09-16T09:33:30.251186+0000","last_became_peered":"2024-09-16T09:33:30.251186+0000","last_unstale":"2024-09-16T09:36:12.372065+0000","last_undegraded":"2024-09-16T09:36:12.372065+0000","last_fullsized":"2024-09-16T09:36:12.372065+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:44.298246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:44.298246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,0],"acting":[4,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":462,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.406802+0000","last_change":"2024-09-16T09:33:21.337606+0000","last_active":"2024-09-16T09:36:12.406802+0000","last_peered":"2024-09-16T09:36:12.406802+0000","last_clean":"2024-09-16T09:36:12.406802+0000","last_became_active":"2024-09-16T09:33:21.337291+0000","last_became_peered":"2024-09-16T09:33:21.337291+0000","last_unstale":"2024-09-16T09:36:12.406802+0000","last_undegraded":"2024-09-16T09:36:12.406802+0000","last_fullsized":"2024-09-16T09:36:12.406802+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:48.259971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:32:48.259971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4],"acting":[3,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1","version":"309'7053","reported_seq":7313,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.677989+0000","last_change":"2024-09-16T09:35:36.575967+0000","last_active":"2024-09-16T09:36:12.677989+0000","last_peered":"2024-09-16T09:36:12.677989+0000","last_clean":"2024-09-16T09:36:12.677989+0000","last_became_active":"2024-09-16T09:33:52.910622+0000","last_became_peered":"2024-09-16T09:33:52.910622+0000","last_unstale":"2024-09-16T09:36:12.677989+0000","last_undegraded":"2024-09-16T09:36:12.677989+0000","last_fullsized":"2024-09-16T09:36:12.677989+0000","mapping_epoch":229,"log_start":"236'1245","ondisk_log_start":"236'1245","created":229,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"291'6189","last_scrub_stamp":"2024-09-16T09:35:36.575666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:35:36.575666+0000","objects_scrubbed":2923,"log_size":5808,"log_dups_size":0,"ondisk_log_size":5808,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:37:06.425386+0000","scrub_duration":620,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":134152192,"num_objects":2047,"num_object_clones":0,"num_object_copies":6141,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2047,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":876,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.9","version":"73'79","reported_seq":596,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.408578+0000","last_change":"2024-09-16T09:29:19.013948+0000","last_active":"2024-09-16T09:36:12.408578+0000","last_peered":"2024-09-16T09:36:12.408578+0000","last_clean":"2024-09-16T09:36:12.408578+0000","last_became_active":"2024-09-16T09:26:35.126066+0000","last_became_peered":"2024-09-16T09:26:35.126066+0000","last_unstale":"2024-09-16T09:36:12.408578+0000","last_undegraded":"2024-09-16T09:36:12.408578+0000","last_fullsized":"2024-09-16T09:36:12.408578+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:19.013865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:19.013865+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"285'20","reported_seq":550,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.372242+0000","last_change":"2024-09-16T09:33:24.154567+0000","last_active":"2024-09-16T09:36:12.372242+0000","last_peered":"2024-09-16T09:36:12.372242+0000","last_clean":"2024-09-16T09:36:12.372242+0000","last_became_active":"2024-09-16T09:25:03.707723+0000","last_became_peered":"2024-09-16T09:25:03.707723+0000","last_unstale":"2024-09-16T09:36:12.372242+0000","last_undegraded":"2024-09-16T09:36:12.372242+0000","last_fullsized":"2024-09-16T09:36:12.372242+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-16T09:33:24.154507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:33:24.154507+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:37:00.827564+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":392,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-09-16T09:36:08.363446+0000","last_change":"2024-09-16T09:33:10.528181+0000","last_active":"2024-09-16T09:36:08.363446+0000","last_peered":"2024-09-16T09:36:08.363446+0000","last_clean":"2024-09-16T09:36:08.363446+0000","last_became_active":"2024-09-16T09:31:09.288825+0000","last_became_peered":"2024-09-16T09:31:09.288825+0000","last_unstale":"2024-09-16T09:36:08.363446+0000","last_undegraded":"2024-09-16T09:36:08.363446+0000","last_fullsized":"2024-09-16T09:36:08.363446+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:10.528115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:10.528115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"228'9","reported_seq":240,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.409058+0000","last_change":"2024-09-16T09:33:21.337623+0000","last_active":"2024-09-16T09:36:12.409058+0000","last_peered":"2024-09-16T09:36:12.409058+0000","last_clean":"2024-09-16T09:36:12.409058+0000","last_became_active":"2024-09-16T09:33:21.337213+0000","last_became_peered":"2024-09-16T09:33:21.337213+0000","last_unstale":"2024-09-16T09:36:12.409058+0000","last_undegraded":"2024-09-16T09:36:12.409058+0000","last_fullsized":"2024-09-16T09:36:12.409058+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:32:42.175528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:32:42.175528+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":544,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.372297+0000","last_change":"2024-09-16T09:35:36.061013+0000","last_active":"2024-09-16T09:36:12.372297+0000","last_peered":"2024-09-16T09:36:12.372297+0000","last_clean":"2024-09-16T09:36:12.372297+0000","last_became_active":"2024-09-16T09:25:07.718958+0000","last_became_peered":"2024-09-16T09:25:07.718958+0000","last_unstale":"2024-09-16T09:36:12.372297+0000","last_undegraded":"2024-09-16T09:36:12.372297+0000","last_fullsized":"2024-09-16T09:36:12.372297+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:35:36.060942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:35:36.060942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:37:03.402548+0000","scrub_duration":157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.6","version":"309'7268","reported_seq":7535,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.967074+0000","last_change":"2024-09-16T09:35:36.698052+0000","last_active":"2024-09-16T09:36:12.967074+0000","last_peered":"2024-09-16T09:36:12.967074+0000","last_clean":"2024-09-16T09:36:12.967074+0000","last_became_active":"2024-09-16T09:33:52.928727+0000","last_became_peered":"2024-09-16T09:33:52.928727+0000","last_unstale":"2024-09-16T09:36:12.967074+0000","last_undegraded":"2024-09-16T09:36:12.967074+0000","last_fullsized":"2024-09-16T09:36:12.967074+0000","mapping_epoch":229,"log_start":"239'2474","ondisk_log_start":"239'2474","created":229,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"291'6321","last_scrub_stamp":"2024-09-16T09:35:36.697883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:35:36.697883+0000","objects_scrubbed":2977,"log_size":4794,"log_dups_size":0,"ondisk_log_size":4794,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:37:04.581052+0000","scrub_duration":949,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":132317184,"num_objects":2019,"num_object_clones":0,"num_object_copies":6057,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2019,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":958,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.e","version":"73'79","reported_seq":337,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.372174+0000","last_change":"2024-09-16T09:35:41.979877+0000","last_active":"2024-09-16T09:36:12.372174+0000","last_peered":"2024-09-16T09:36:12.372174+0000","last_clean":"2024-09-16T09:36:12.372174+0000","last_became_active":"2024-09-16T09:30:10.705072+0000","last_became_peered":"2024-09-16T09:30:10.705072+0000","last_unstale":"2024-09-16T09:36:12.372174+0000","last_undegraded":"2024-09-16T09:36:12.372174+0000","last_fullsized":"2024-09-16T09:36:12.372174+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:35:41.979814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:35:41.979814+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:36:59.731534+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,3],"acting":[4,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.b","version":"285'15","reported_seq":555,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-09-16T09:36:08.363265+0000","last_change":"2024-09-16T09:31:31.211495+0000","last_active":"2024-09-16T09:36:08.363265+0000","last_peered":"2024-09-16T09:36:08.363265+0000","last_clean":"2024-09-16T09:36:08.363265+0000","last_became_active":"2024-09-16T09:25:03.712230+0000","last_became_peered":"2024-09-16T09:25:03.712230+0000","last_unstale":"2024-09-16T09:36:08.363265+0000","last_undegraded":"2024-09-16T09:36:08.363265+0000","last_fullsized":"2024-09-16T09:36:08.363265+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:31:31.211420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:31:31.211420+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":537,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.414238+0000","last_change":"2024-09-16T09:31:09.277413+0000","last_active":"2024-09-16T09:36:12.414238+0000","last_peered":"2024-09-16T09:36:12.414238+0000","last_clean":"2024-09-16T09:36:12.414238+0000","last_became_active":"2024-09-16T09:31:09.276516+0000","last_became_peered":"2024-09-16T09:31:09.276516+0000","last_unstale":"2024-09-16T09:36:12.414238+0000","last_undegraded":"2024-09-16T09:36:12.414238+0000","last_fullsized":"2024-09-16T09:36:12.414238+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:26.061106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:26.061106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":466,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.371765+0000","last_change":"2024-09-16T09:33:40.260547+0000","last_active":"2024-09-16T09:36:12.371765+0000","last_peered":"2024-09-16T09:36:12.371765+0000","last_clean":"2024-09-16T09:36:12.371765+0000","last_became_active":"2024-09-16T09:33:21.337254+0000","last_became_peered":"2024-09-16T09:33:21.337254+0000","last_unstale":"2024-09-16T09:36:12.371765+0000","last_undegraded":"2024-09-16T09:36:12.371765+0000","last_fullsized":"2024-09-16T09:36:12.371765+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:40.260500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:40.260500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3],"acting":[4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":523,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-09-16T09:36:08.361283+0000","last_change":"2024-09-16T09:31:27.185957+0000","last_active":"2024-09-16T09:36:08.361283+0000","last_peered":"2024-09-16T09:36:08.361283+0000","last_clean":"2024-09-16T09:36:08.361283+0000","last_became_active":"2024-09-16T09:25:07.718778+0000","last_became_peered":"2024-09-16T09:25:07.718778+0000","last_unstale":"2024-09-16T09:36:08.361283+0000","last_undegraded":"2024-09-16T09:36:08.361283+0000","last_fullsized":"2024-09-16T09:36:08.361283+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:31:27.185801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:31:27.185801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"309'7124","reported_seq":7336,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:13.040557+0000","last_change":"2024-09-16T09:33:53.762479+0000","last_active":"2024-09-16T09:36:13.040557+0000","last_peered":"2024-09-16T09:36:13.040557+0000","last_clean":"2024-09-16T09:36:13.040557+0000","last_became_active":"2024-09-16T09:33:52.930653+0000","last_became_peered":"2024-09-16T09:33:52.930653+0000","last_unstale":"2024-09-16T09:36:13.040557+0000","last_undegraded":"2024-09-16T09:36:13.040557+0000","last_fullsized":"2024-09-16T09:36:13.040557+0000","mapping_epoch":229,"log_start":"238'2222","ondisk_log_start":"238'2222","created":229,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"244'3138","last_scrub_stamp":"2024-09-16T09:33:53.762280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:33:53.762280+0000","objects_scrubbed":1562,"log_size":4902,"log_dups_size":0,"ondisk_log_size":4902,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":422,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131334144,"num_objects":2004,"num_object_clones":0,"num_object_copies":6012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2004,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4000,"num_write_kb":142144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.f","version":"73'79","reported_seq":589,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.445637+0000","last_change":"2024-09-16T09:34:53.866426+0000","last_active":"2024-09-16T09:36:12.445637+0000","last_peered":"2024-09-16T09:36:12.445637+0000","last_clean":"2024-09-16T09:36:12.445637+0000","last_became_active":"2024-09-16T09:30:10.693477+0000","last_became_peered":"2024-09-16T09:30:10.693477+0000","last_unstale":"2024-09-16T09:36:12.445637+0000","last_undegraded":"2024-09-16T09:36:12.445637+0000","last_fullsized":"2024-09-16T09:36:12.445637+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:34:53.866337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:34:53.866337+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"285'35","reported_seq":592,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.444558+0000","last_change":"2024-09-16T09:33:22.905808+0000","last_active":"2024-09-16T09:36:12.444558+0000","last_peered":"2024-09-16T09:36:12.444558+0000","last_clean":"2024-09-16T09:36:12.444558+0000","last_became_active":"2024-09-16T09:25:03.712783+0000","last_became_peered":"2024-09-16T09:25:03.712783+0000","last_unstale":"2024-09-16T09:36:12.444558+0000","last_undegraded":"2024-09-16T09:36:12.444558+0000","last_fullsized":"2024-09-16T09:36:12.444558+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-16T09:33:22.905752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:33:22.905752+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":56,"num_read_kb":45,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"59'1","reported_seq":545,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.409372+0000","last_change":"2024-09-16T09:33:42.294543+0000","last_active":"2024-09-16T09:36:12.409372+0000","last_peered":"2024-09-16T09:36:12.409372+0000","last_clean":"2024-09-16T09:36:12.409372+0000","last_became_active":"2024-09-16T09:25:01.708069+0000","last_became_peered":"2024-09-16T09:25:01.708069+0000","last_unstale":"2024-09-16T09:36:12.409372+0000","last_undegraded":"2024-09-16T09:36:12.409372+0000","last_fullsized":"2024-09-16T09:36:12.409372+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:33:42.294378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:42.294378+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":539,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.433160+0000","last_change":"2024-09-16T09:34:38.338415+0000","last_active":"2024-09-16T09:36:12.433160+0000","last_peered":"2024-09-16T09:36:12.433160+0000","last_clean":"2024-09-16T09:36:12.433160+0000","last_became_active":"2024-09-16T09:34:00.330050+0000","last_became_peered":"2024-09-16T09:34:00.330050+0000","last_unstale":"2024-09-16T09:36:12.433160+0000","last_undegraded":"2024-09-16T09:36:12.433160+0000","last_fullsized":"2024-09-16T09:36:12.433160+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:38.338329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:34:38.338329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,7],"acting":[2,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":512,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.412809+0000","last_change":"2024-09-16T09:27:25.865107+0000","last_active":"2024-09-16T09:36:12.412809+0000","last_peered":"2024-09-16T09:36:12.412809+0000","last_clean":"2024-09-16T09:36:12.412809+0000","last_became_active":"2024-09-16T09:25:07.718750+0000","last_became_peered":"2024-09-16T09:25:07.718750+0000","last_unstale":"2024-09-16T09:36:12.412809+0000","last_undegraded":"2024-09-16T09:36:12.412809+0000","last_fullsized":"2024-09-16T09:36:12.412809+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:25.865030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:25.865030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.4","version":"309'7353","reported_seq":7555,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.898080+0000","last_change":"2024-09-16T09:33:53.661296+0000","last_active":"2024-09-16T09:36:12.898080+0000","last_peered":"2024-09-16T09:36:12.898080+0000","last_clean":"2024-09-16T09:36:12.898080+0000","last_became_active":"2024-09-16T09:33:52.909537+0000","last_became_peered":"2024-09-16T09:33:52.909537+0000","last_unstale":"2024-09-16T09:36:12.898080+0000","last_undegraded":"2024-09-16T09:36:12.898080+0000","last_fullsized":"2024-09-16T09:36:12.898080+0000","mapping_epoch":233,"log_start":"242'3106","ondisk_log_start":"242'3106","created":229,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"244'3237","last_scrub_stamp":"2024-09-16T09:33:53.661132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:33:53.661132+0000","objects_scrubbed":1642,"log_size":4247,"log_dups_size":0,"ondisk_log_size":4247,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":365,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":133693440,"num_objects":2040,"num_object_clones":0,"num_object_copies":6120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2040,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4124,"num_write_kb":144704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.c","version":"73'79","reported_seq":270,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.405789+0000","last_change":"2024-09-16T09:33:08.949107+0000","last_active":"2024-09-16T09:36:12.405789+0000","last_peered":"2024-09-16T09:36:12.405789+0000","last_clean":"2024-09-16T09:36:12.405789+0000","last_became_active":"2024-09-16T09:31:09.273770+0000","last_became_peered":"2024-09-16T09:31:09.273770+0000","last_unstale":"2024-09-16T09:36:12.405789+0000","last_undegraded":"2024-09-16T09:36:12.405789+0000","last_fullsized":"2024-09-16T09:36:12.405789+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:33:08.949051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:33:08.949051+0000","objects_scrubbed":1,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":327680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"285'18","reported_seq":607,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.371745+0000","last_change":"2024-09-16T09:35:31.869240+0000","last_active":"2024-09-16T09:36:12.371745+0000","last_peered":"2024-09-16T09:36:12.371745+0000","last_clean":"2024-09-16T09:36:12.371745+0000","last_became_active":"2024-09-16T09:31:09.269047+0000","last_became_peered":"2024-09-16T09:31:09.269047+0000","last_unstale":"2024-09-16T09:36:12.371745+0000","last_undegraded":"2024-09-16T09:36:12.371745+0000","last_fullsized":"2024-09-16T09:36:12.371745+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'18","last_scrub_stamp":"2024-09-16T09:35:31.869148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:35:31.869148+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:36:38.427638+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":539,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.409541+0000","last_change":"2024-09-16T09:33:48.225867+0000","last_active":"2024-09-16T09:36:12.409541+0000","last_peered":"2024-09-16T09:36:12.409541+0000","last_clean":"2024-09-16T09:36:12.409541+0000","last_became_active":"2024-09-16T09:25:01.707528+0000","last_became_peered":"2024-09-16T09:25:01.707528+0000","last_unstale":"2024-09-16T09:36:12.409541+0000","last_undegraded":"2024-09-16T09:36:12.409541+0000","last_fullsized":"2024-09-16T09:36:12.409541+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:48.225635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:48.225635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":178,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-09-16T09:36:08.361785+0000","last_change":"2024-09-16T09:33:23.643136+0000","last_active":"2024-09-16T09:36:08.361785+0000","last_peered":"2024-09-16T09:36:08.361785+0000","last_clean":"2024-09-16T09:36:08.361785+0000","last_became_active":"2024-09-16T09:33:21.337309+0000","last_became_peered":"2024-09-16T09:33:21.337309+0000","last_unstale":"2024-09-16T09:36:08.361785+0000","last_undegraded":"2024-09-16T09:36:08.361785+0000","last_fullsized":"2024-09-16T09:36:08.361785+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:23.643089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:23.643089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":520,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.405496+0000","last_change":"2024-09-16T09:34:30.458186+0000","last_active":"2024-09-16T09:36:12.405496+0000","last_peered":"2024-09-16T09:36:12.405496+0000","last_clean":"2024-09-16T09:36:12.405496+0000","last_became_active":"2024-09-16T09:25:07.718899+0000","last_became_peered":"2024-09-16T09:25:07.718899+0000","last_unstale":"2024-09-16T09:36:12.405496+0000","last_undegraded":"2024-09-16T09:36:12.405496+0000","last_fullsized":"2024-09-16T09:36:12.405496+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:30.458115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:34:30.458115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.5","version":"304'7117","reported_seq":10815,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-09-16T09:36:08.361918+0000","last_change":"2024-09-16T09:34:34.440150+0000","last_active":"2024-09-16T09:36:08.361918+0000","last_peered":"2024-09-16T09:36:08.361918+0000","last_clean":"2024-09-16T09:36:08.361918+0000","last_became_active":"2024-09-16T09:34:32.381065+0000","last_became_peered":"2024-09-16T09:34:32.381065+0000","last_unstale":"2024-09-16T09:36:08.361918+0000","last_undegraded":"2024-09-16T09:36:08.361918+0000","last_fullsized":"2024-09-16T09:36:08.361918+0000","mapping_epoch":267,"log_start":"236'1521","ondisk_log_start":"236'1521","created":229,"last_epoch_clean":268,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'4368","last_scrub_stamp":"2024-09-16T09:34:34.439986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:34:34.439986+0000","objects_scrubbed":2756,"log_size":5596,"log_dups_size":0,"ondisk_log_size":5596,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":710,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":142999552,"num_objects":2182,"num_object_clones":0,"num_object_copies":6546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2182,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2760,"num_write_kb":69952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.d","version":"73'79","reported_seq":329,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.408898+0000","last_change":"2024-09-16T09:32:44.536868+0000","last_active":"2024-09-16T09:36:12.408898+0000","last_peered":"2024-09-16T09:36:12.408898+0000","last_clean":"2024-09-16T09:36:12.408898+0000","last_became_active":"2024-09-16T09:30:10.709958+0000","last_became_peered":"2024-09-16T09:30:10.709958+0000","last_unstale":"2024-09-16T09:36:12.408898+0000","last_undegraded":"2024-09-16T09:36:12.408898+0000","last_fullsized":"2024-09-16T09:36:12.408898+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:32:44.536799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:32:44.536799+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"285'27","reported_seq":563,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.413895+0000","last_change":"2024-09-16T09:26:33.138828+0000","last_active":"2024-09-16T09:36:12.413895+0000","last_peered":"2024-09-16T09:36:12.413895+0000","last_clean":"2024-09-16T09:36:12.413895+0000","last_became_active":"2024-09-16T09:25:03.715733+0000","last_became_peered":"2024-09-16T09:25:03.715733+0000","last_unstale":"2024-09-16T09:36:12.413895+0000","last_undegraded":"2024-09-16T09:36:12.413895+0000","last_fullsized":"2024-09-16T09:36:12.413895+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-16T09:26:33.138603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:26:33.138603+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":543,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.410108+0000","last_change":"2024-09-16T09:32:49.441194+0000","last_active":"2024-09-16T09:36:12.410108+0000","last_peered":"2024-09-16T09:36:12.410108+0000","last_clean":"2024-09-16T09:36:12.410108+0000","last_became_active":"2024-09-16T09:25:01.707372+0000","last_became_peered":"2024-09-16T09:25:01.707372+0000","last_unstale":"2024-09-16T09:36:12.410108+0000","last_undegraded":"2024-09-16T09:36:12.410108+0000","last_fullsized":"2024-09-16T09:36:12.410108+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-16T09:32:49.441122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:32:49.441122+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"66'8","reported_seq":675,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.408476+0000","last_change":"2024-09-16T09:30:05.138362+0000","last_active":"2024-09-16T09:36:12.408476+0000","last_peered":"2024-09-16T09:36:12.408476+0000","last_clean":"2024-09-16T09:36:12.408476+0000","last_became_active":"2024-09-16T09:25:05.716189+0000","last_became_peered":"2024-09-16T09:25:05.716189+0000","last_unstale":"2024-09-16T09:36:12.408476+0000","last_undegraded":"2024-09-16T09:36:12.408476+0000","last_fullsized":"2024-09-16T09:36:12.408476+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:30:05.138288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:30:05.138288+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":512,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.413837+0000","last_change":"2024-09-16T09:27:20.219376+0000","last_active":"2024-09-16T09:36:12.413837+0000","last_peered":"2024-09-16T09:36:12.413837+0000","last_clean":"2024-09-16T09:36:12.413837+0000","last_became_active":"2024-09-16T09:25:07.717213+0000","last_became_peered":"2024-09-16T09:25:07.717213+0000","last_unstale":"2024-09-16T09:36:12.413837+0000","last_undegraded":"2024-09-16T09:36:12.413837+0000","last_fullsized":"2024-09-16T09:36:12.413837+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:20.219333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:20.219333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.12","version":"73'79","reported_seq":329,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.413964+0000","last_change":"2024-09-16T09:32:45.399435+0000","last_active":"2024-09-16T09:36:12.413964+0000","last_peered":"2024-09-16T09:36:12.413964+0000","last_clean":"2024-09-16T09:36:12.413964+0000","last_became_active":"2024-09-16T09:30:10.731413+0000","last_became_peered":"2024-09-16T09:30:10.731413+0000","last_unstale":"2024-09-16T09:36:12.413964+0000","last_undegraded":"2024-09-16T09:36:12.413964+0000","last_fullsized":"2024-09-16T09:36:12.413964+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:32:45.399361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:32:45.399361+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":539,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.444978+0000","last_change":"2024-09-16T09:34:00.334779+0000","last_active":"2024-09-16T09:36:12.444978+0000","last_peered":"2024-09-16T09:36:12.444978+0000","last_clean":"2024-09-16T09:36:12.444978+0000","last_became_active":"2024-09-16T09:34:00.334361+0000","last_became_peered":"2024-09-16T09:34:00.334361+0000","last_unstale":"2024-09-16T09:36:12.444978+0000","last_undegraded":"2024-09-16T09:36:12.444978+0000","last_fullsized":"2024-09-16T09:36:12.444978+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:14.845355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:14.845355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"285'12","reported_seq":291,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.407023+0000","last_change":"2024-09-16T09:34:51.181994+0000","last_active":"2024-09-16T09:36:12.407023+0000","last_peered":"2024-09-16T09:36:12.407023+0000","last_clean":"2024-09-16T09:36:12.407023+0000","last_became_active":"2024-09-16T09:31:09.270685+0000","last_became_peered":"2024-09-16T09:31:09.270685+0000","last_unstale":"2024-09-16T09:36:12.407023+0000","last_undegraded":"2024-09-16T09:36:12.407023+0000","last_fullsized":"2024-09-16T09:36:12.407023+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'6","last_scrub_stamp":"2024-09-16T09:34:51.181918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:34:51.181918+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:36:17.849346+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":17,"num_read_kb":14,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":523,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.414551+0000","last_change":"2024-09-16T09:29:31.496374+0000","last_active":"2024-09-16T09:36:12.414551+0000","last_peered":"2024-09-16T09:36:12.414551+0000","last_clean":"2024-09-16T09:36:12.414551+0000","last_became_active":"2024-09-16T09:25:05.715616+0000","last_became_peered":"2024-09-16T09:25:05.715616+0000","last_unstale":"2024-09-16T09:36:12.414551+0000","last_undegraded":"2024-09-16T09:36:12.414551+0000","last_fullsized":"2024-09-16T09:36:12.414551+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:31.496304+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:29:31.496304+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":511,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.408868+0000","last_change":"2024-09-16T09:26:28.733381+0000","last_active":"2024-09-16T09:36:12.408868+0000","last_peered":"2024-09-16T09:36:12.408868+0000","last_clean":"2024-09-16T09:36:12.408868+0000","last_became_active":"2024-09-16T09:25:07.721059+0000","last_became_peered":"2024-09-16T09:25:07.721059+0000","last_unstale":"2024-09-16T09:36:12.408868+0000","last_undegraded":"2024-09-16T09:36:12.408868+0000","last_fullsized":"2024-09-16T09:36:12.408868+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:28.733324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:26:28.733324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.13","version":"73'79","reported_seq":423,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.405637+0000","last_change":"2024-09-16T09:34:48.260121+0000","last_active":"2024-09-16T09:36:12.405637+0000","last_peered":"2024-09-16T09:36:12.405637+0000","last_clean":"2024-09-16T09:36:12.405637+0000","last_became_active":"2024-09-16T09:33:30.248497+0000","last_became_peered":"2024-09-16T09:33:30.248497+0000","last_unstale":"2024-09-16T09:36:12.405637+0000","last_undegraded":"2024-09-16T09:36:12.405637+0000","last_fullsized":"2024-09-16T09:36:12.405637+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:34:48.260041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:34:48.260041+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"285'15","reported_seq":605,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-09-16T09:36:08.362800+0000","last_change":"2024-09-16T09:33:48.681742+0000","last_active":"2024-09-16T09:36:08.362800+0000","last_peered":"2024-09-16T09:36:08.362800+0000","last_clean":"2024-09-16T09:36:08.362800+0000","last_became_active":"2024-09-16T09:31:09.273498+0000","last_became_peered":"2024-09-16T09:31:09.273498+0000","last_unstale":"2024-09-16T09:36:08.362800+0000","last_undegraded":"2024-09-16T09:36:08.362800+0000","last_fullsized":"2024-09-16T09:36:08.362800+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:33:48.681650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:33:48.681650+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":539,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.408683+0000","last_change":"2024-09-16T09:33:13.312434+0000","last_active":"2024-09-16T09:36:12.408683+0000","last_peered":"2024-09-16T09:36:12.408683+0000","last_clean":"2024-09-16T09:36:12.408683+0000","last_became_active":"2024-09-16T09:25:01.706491+0000","last_became_peered":"2024-09-16T09:25:01.706491+0000","last_unstale":"2024-09-16T09:36:12.408683+0000","last_undegraded":"2024-09-16T09:36:12.408683+0000","last_fullsized":"2024-09-16T09:36:12.408683+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:13.312344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:13.312344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":289,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.409529+0000","last_change":"2024-09-16T09:33:30.249293+0000","last_active":"2024-09-16T09:36:12.409529+0000","last_peered":"2024-09-16T09:36:12.409529+0000","last_clean":"2024-09-16T09:36:12.409529+0000","last_became_active":"2024-09-16T09:33:30.248992+0000","last_became_peered":"2024-09-16T09:33:30.248992+0000","last_unstale":"2024-09-16T09:36:12.409529+0000","last_undegraded":"2024-09-16T09:36:12.409529+0000","last_fullsized":"2024-09-16T09:36:12.409529+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:53.810580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:28:53.810580+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":528,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.433266+0000","last_change":"2024-09-16T09:34:32.309307+0000","last_active":"2024-09-16T09:36:12.433266+0000","last_peered":"2024-09-16T09:36:12.433266+0000","last_clean":"2024-09-16T09:36:12.433266+0000","last_became_active":"2024-09-16T09:25:07.722050+0000","last_became_peered":"2024-09-16T09:25:07.722050+0000","last_unstale":"2024-09-16T09:36:12.433266+0000","last_undegraded":"2024-09-16T09:36:12.433266+0000","last_fullsized":"2024-09-16T09:36:12.433266+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:32.309262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:34:32.309262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.18","version":"309'7121","reported_seq":19750,"reported_epoch":309,"state":"active+remapped+backfilling","last_fresh":"2024-09-16T09:36:12.881024+0000","last_change":"2024-09-16T09:36:12.459574+0000","last_active":"2024-09-16T09:36:12.881024+0000","last_peered":"2024-09-16T09:36:12.881024+0000","last_clean":"2024-09-16T09:35:04.658405+0000","last_became_active":"2024-09-16T09:36:12.425212+0000","last_became_peered":"2024-09-16T09:36:12.425212+0000","last_unstale":"2024-09-16T09:36:12.881024+0000","last_undegraded":"2024-09-16T09:36:12.881024+0000","last_fullsized":"2024-09-16T09:36:12.881024+0000","mapping_epoch":308,"log_start":"247'3233","ondisk_log_start":"247'3233","created":243,"last_epoch_clean":259,"parent":"0.0","parent_split_bits":5,"last_scrub":"275'4947","last_scrub_stamp":"2024-09-16T09:34:50.611186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:34:50.611186+0000","objects_scrubbed":3349,"log_size":3888,"log_dups_size":0,"ondisk_log_size":3888,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":773,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131006464,"num_objects":1999,"num_object_clones":0,"num_object_copies":5997,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1943,"num_objects_unfound":0,"num_objects_dirty":1999,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2194,"num_write_kb":27008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3826,"num_bytes_recovered":240123904,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[7,1,5],"avail_no_missing":["7","1","5","6"],"object_location_counts":[{"shards":"1,5,7","objects":1999}],"blocked_by":[],"up_primary":0,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.10","version":"73'79","reported_seq":329,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.413769+0000","last_change":"2024-09-16T09:32:44.405696+0000","last_active":"2024-09-16T09:36:12.413769+0000","last_peered":"2024-09-16T09:36:12.413769+0000","last_clean":"2024-09-16T09:36:12.413769+0000","last_became_active":"2024-09-16T09:30:10.692770+0000","last_became_peered":"2024-09-16T09:30:10.692770+0000","last_unstale":"2024-09-16T09:36:12.413769+0000","last_undegraded":"2024-09-16T09:36:12.413769+0000","last_fullsized":"2024-09-16T09:36:12.413769+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:32:44.405640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:32:44.405640+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"285'15","reported_seq":552,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.414371+0000","last_change":"2024-09-16T09:29:32.485165+0000","last_active":"2024-09-16T09:36:12.414371+0000","last_peered":"2024-09-16T09:36:12.414371+0000","last_clean":"2024-09-16T09:36:12.414371+0000","last_became_active":"2024-09-16T09:25:03.717310+0000","last_became_peered":"2024-09-16T09:25:03.717310+0000","last_unstale":"2024-09-16T09:36:12.414371+0000","last_undegraded":"2024-09-16T09:36:12.414371+0000","last_fullsized":"2024-09-16T09:36:12.414371+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:29:32.485084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:29:32.485084+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":556,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-09-16T09:36:08.364828+0000","last_change":"2024-09-16T09:33:20.639405+0000","last_active":"2024-09-16T09:36:08.364828+0000","last_peered":"2024-09-16T09:36:08.364828+0000","last_clean":"2024-09-16T09:36:08.364828+0000","last_became_active":"2024-09-16T09:31:09.272848+0000","last_became_peered":"2024-09-16T09:31:09.272848+0000","last_unstale":"2024-09-16T09:36:08.364828+0000","last_undegraded":"2024-09-16T09:36:08.364828+0000","last_fullsized":"2024-09-16T09:36:08.364828+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:20.639350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:20.639350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"187'11","reported_seq":512,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.408802+0000","last_change":"2024-09-16T09:33:20.296801+0000","last_active":"2024-09-16T09:36:12.408802+0000","last_peered":"2024-09-16T09:36:12.408802+0000","last_clean":"2024-09-16T09:36:12.408802+0000","last_became_active":"2024-09-16T09:31:09.277688+0000","last_became_peered":"2024-09-16T09:31:09.277688+0000","last_unstale":"2024-09-16T09:36:12.408802+0000","last_undegraded":"2024-09-16T09:36:12.408802+0000","last_fullsized":"2024-09-16T09:36:12.408802+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"187'11","last_scrub_stamp":"2024-09-16T09:33:20.296746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:20.296746+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":536,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.372498+0000","last_change":"2024-09-16T09:35:18.812189+0000","last_active":"2024-09-16T09:36:12.372498+0000","last_peered":"2024-09-16T09:36:12.372498+0000","last_clean":"2024-09-16T09:36:12.372498+0000","last_became_active":"2024-09-16T09:25:07.718810+0000","last_became_peered":"2024-09-16T09:25:07.718810+0000","last_unstale":"2024-09-16T09:36:12.372498+0000","last_undegraded":"2024-09-16T09:36:12.372498+0000","last_fullsized":"2024-09-16T09:36:12.372498+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:35:18.812145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:35:18.812145+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:36:38.242580+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.11","version":"73'79","reported_seq":409,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-09-16T09:36:08.362143+0000","last_change":"2024-09-16T09:32:38.401888+0000","last_active":"2024-09-16T09:36:08.362143+0000","last_peered":"2024-09-16T09:36:08.362143+0000","last_clean":"2024-09-16T09:36:08.362143+0000","last_became_active":"2024-09-16T09:31:09.272208+0000","last_became_peered":"2024-09-16T09:31:09.272208+0000","last_unstale":"2024-09-16T09:36:08.362143+0000","last_undegraded":"2024-09-16T09:36:08.362143+0000","last_fullsized":"2024-09-16T09:36:08.362143+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:32:38.401720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:32:38.401720+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":32,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.14","version":"285'20","reported_seq":214,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.408924+0000","last_change":"2024-09-16T09:33:30.250101+0000","last_active":"2024-09-16T09:36:12.408924+0000","last_peered":"2024-09-16T09:36:12.408924+0000","last_clean":"2024-09-16T09:36:12.408924+0000","last_became_active":"2024-09-16T09:33:30.249769+0000","last_became_peered":"2024-09-16T09:33:30.249769+0000","last_unstale":"2024-09-16T09:36:12.408924+0000","last_undegraded":"2024-09-16T09:36:12.408924+0000","last_fullsized":"2024-09-16T09:36:12.408924+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-16T09:28:56.825476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:56.825476+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":531,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.409133+0000","last_change":"2024-09-16T09:29:56.021413+0000","last_active":"2024-09-16T09:36:12.409133+0000","last_peered":"2024-09-16T09:36:12.409133+0000","last_clean":"2024-09-16T09:36:12.409133+0000","last_became_active":"2024-09-16T09:25:01.707513+0000","last_became_peered":"2024-09-16T09:25:01.707513+0000","last_unstale":"2024-09-16T09:36:12.409133+0000","last_undegraded":"2024-09-16T09:36:12.409133+0000","last_fullsized":"2024-09-16T09:36:12.409133+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:56.021340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:56.021340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"66'8","reported_seq":682,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.414176+0000","last_change":"2024-09-16T09:33:39.388721+0000","last_active":"2024-09-16T09:36:12.414176+0000","last_peered":"2024-09-16T09:36:12.414176+0000","last_clean":"2024-09-16T09:36:12.414176+0000","last_became_active":"2024-09-16T09:25:05.713833+0000","last_became_peered":"2024-09-16T09:25:05.713833+0000","last_unstale":"2024-09-16T09:36:12.414176+0000","last_undegraded":"2024-09-16T09:36:12.414176+0000","last_fullsized":"2024-09-16T09:36:12.414176+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:33:39.388463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:39.388463+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":544,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-09-16T09:36:08.362378+0000","last_change":"2024-09-16T09:33:42.666290+0000","last_active":"2024-09-16T09:36:08.362378+0000","last_peered":"2024-09-16T09:36:08.362378+0000","last_clean":"2024-09-16T09:36:08.362378+0000","last_became_active":"2024-09-16T09:31:09.275168+0000","last_became_peered":"2024-09-16T09:31:09.275168+0000","last_unstale":"2024-09-16T09:36:08.362378+0000","last_undegraded":"2024-09-16T09:36:08.362378+0000","last_fullsized":"2024-09-16T09:36:08.362378+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:42.666200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:33:42.666200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"285'19","reported_seq":570,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.371978+0000","last_change":"2024-09-16T09:33:19.084118+0000","last_active":"2024-09-16T09:36:12.371978+0000","last_peered":"2024-09-16T09:36:12.371978+0000","last_clean":"2024-09-16T09:36:12.371978+0000","last_became_active":"2024-09-16T09:26:33.291265+0000","last_became_peered":"2024-09-16T09:26:33.291265+0000","last_unstale":"2024-09-16T09:36:12.371978+0000","last_undegraded":"2024-09-16T09:36:12.371978+0000","last_fullsized":"2024-09-16T09:36:12.371978+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-16T09:33:19.084063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:33:19.084063+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":548,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.372406+0000","last_change":"2024-09-16T09:35:24.722539+0000","last_active":"2024-09-16T09:36:12.372406+0000","last_peered":"2024-09-16T09:36:12.372406+0000","last_clean":"2024-09-16T09:36:12.372406+0000","last_became_active":"2024-09-16T09:25:01.705158+0000","last_became_peered":"2024-09-16T09:25:01.705158+0000","last_unstale":"2024-09-16T09:36:12.372406+0000","last_undegraded":"2024-09-16T09:36:12.372406+0000","last_fullsized":"2024-09-16T09:36:12.372406+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:35:24.722476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:35:24.722476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:36:39.731563+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":461,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.413862+0000","last_change":"2024-09-16T09:34:00.329968+0000","last_active":"2024-09-16T09:36:12.413862+0000","last_peered":"2024-09-16T09:36:12.413862+0000","last_clean":"2024-09-16T09:36:12.413862+0000","last_became_active":"2024-09-16T09:34:00.329642+0000","last_became_peered":"2024-09-16T09:34:00.329642+0000","last_unstale":"2024-09-16T09:36:12.413862+0000","last_undegraded":"2024-09-16T09:36:12.413862+0000","last_fullsized":"2024-09-16T09:36:12.413862+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:08.394830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:08.394830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":377,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-09-16T09:36:08.361251+0000","last_change":"2024-09-16T09:34:48.786847+0000","last_active":"2024-09-16T09:36:08.361251+0000","last_peered":"2024-09-16T09:36:08.361251+0000","last_clean":"2024-09-16T09:36:08.361251+0000","last_became_active":"2024-09-16T09:31:09.276042+0000","last_became_peered":"2024-09-16T09:31:09.276042+0000","last_unstale":"2024-09-16T09:36:08.361251+0000","last_undegraded":"2024-09-16T09:36:08.361251+0000","last_fullsized":"2024-09-16T09:36:08.361251+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:48.786785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:34:48.786785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:36:17.728202+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"285'15","reported_seq":226,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.444682+0000","last_change":"2024-09-16T09:34:15.299311+0000","last_active":"2024-09-16T09:36:12.444682+0000","last_peered":"2024-09-16T09:36:12.444682+0000","last_clean":"2024-09-16T09:36:12.444682+0000","last_became_active":"2024-09-16T09:34:00.331466+0000","last_became_peered":"2024-09-16T09:34:00.331466+0000","last_unstale":"2024-09-16T09:36:12.444682+0000","last_undegraded":"2024-09-16T09:36:12.444682+0000","last_fullsized":"2024-09-16T09:36:12.444682+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:32:47.247071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:32:47.247071+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,0],"acting":[6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":531,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.409277+0000","last_change":"2024-09-16T09:32:47.502667+0000","last_active":"2024-09-16T09:36:12.409277+0000","last_peered":"2024-09-16T09:36:12.409277+0000","last_clean":"2024-09-16T09:36:12.409277+0000","last_became_active":"2024-09-16T09:25:01.706762+0000","last_became_peered":"2024-09-16T09:25:01.706762+0000","last_unstale":"2024-09-16T09:36:12.409277+0000","last_undegraded":"2024-09-16T09:36:12.409277+0000","last_fullsized":"2024-09-16T09:36:12.409277+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:47.502598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:32:47.502598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":362,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-09-16T09:36:08.363174+0000","last_change":"2024-09-16T09:34:46.742640+0000","last_active":"2024-09-16T09:36:08.363174+0000","last_peered":"2024-09-16T09:36:08.363174+0000","last_clean":"2024-09-16T09:36:08.363174+0000","last_became_active":"2024-09-16T09:28:52.982865+0000","last_became_peered":"2024-09-16T09:28:52.982865+0000","last_unstale":"2024-09-16T09:36:08.363174+0000","last_undegraded":"2024-09-16T09:36:08.363174+0000","last_fullsized":"2024-09-16T09:36:08.363174+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:46.742562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:34:46.742562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":522,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-09-16T09:36:08.361665+0000","last_change":"2024-09-16T09:34:50.779241+0000","last_active":"2024-09-16T09:36:08.361665+0000","last_peered":"2024-09-16T09:36:08.361665+0000","last_clean":"2024-09-16T09:36:08.361665+0000","last_became_active":"2024-09-16T09:25:07.715301+0000","last_became_peered":"2024-09-16T09:25:07.715301+0000","last_unstale":"2024-09-16T09:36:08.361665+0000","last_undegraded":"2024-09-16T09:36:08.361665+0000","last_fullsized":"2024-09-16T09:36:08.361665+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:50.779164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:34:50.779164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:36:11.307448+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.14","version":"73'79","reported_seq":329,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.433166+0000","last_change":"2024-09-16T09:32:44.163264+0000","last_active":"2024-09-16T09:36:12.433166+0000","last_peered":"2024-09-16T09:36:12.433166+0000","last_clean":"2024-09-16T09:36:12.433166+0000","last_became_active":"2024-09-16T09:30:10.700866+0000","last_became_peered":"2024-09-16T09:30:10.700866+0000","last_unstale":"2024-09-16T09:36:12.433166+0000","last_undegraded":"2024-09-16T09:36:12.433166+0000","last_fullsized":"2024-09-16T09:36:12.433166+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:32:44.163191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:32:44.163191+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"285'19","reported_seq":419,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.408587+0000","last_change":"2024-09-16T09:28:55.643505+0000","last_active":"2024-09-16T09:36:12.408587+0000","last_peered":"2024-09-16T09:36:12.408587+0000","last_clean":"2024-09-16T09:36:12.408587+0000","last_became_active":"2024-09-16T09:28:52.971132+0000","last_became_peered":"2024-09-16T09:28:52.971132+0000","last_unstale":"2024-09-16T09:36:12.408587+0000","last_undegraded":"2024-09-16T09:36:12.408587+0000","last_fullsized":"2024-09-16T09:36:12.408587+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-16T09:28:55.643448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:55.643448+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":539,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.414675+0000","last_change":"2024-09-16T09:33:18.297533+0000","last_active":"2024-09-16T09:36:12.414675+0000","last_peered":"2024-09-16T09:36:12.414675+0000","last_clean":"2024-09-16T09:36:12.414675+0000","last_became_active":"2024-09-16T09:25:01.700733+0000","last_became_peered":"2024-09-16T09:25:01.700733+0000","last_unstale":"2024-09-16T09:36:12.414675+0000","last_undegraded":"2024-09-16T09:36:12.414675+0000","last_fullsized":"2024-09-16T09:36:12.414675+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:18.297451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:18.297451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":531,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.408238+0000","last_change":"2024-09-16T09:33:38.336281+0000","last_active":"2024-09-16T09:36:12.408238+0000","last_peered":"2024-09-16T09:36:12.408238+0000","last_clean":"2024-09-16T09:36:12.408238+0000","last_became_active":"2024-09-16T09:25:05.713479+0000","last_became_peered":"2024-09-16T09:25:05.713479+0000","last_unstale":"2024-09-16T09:36:12.408238+0000","last_undegraded":"2024-09-16T09:36:12.408238+0000","last_fullsized":"2024-09-16T09:36:12.408238+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:38.336221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:38.336221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":385,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-09-16T09:36:08.363391+0000","last_change":"2024-09-16T09:34:51.781047+0000","last_active":"2024-09-16T09:36:08.363391+0000","last_peered":"2024-09-16T09:36:08.363391+0000","last_clean":"2024-09-16T09:36:08.363391+0000","last_became_active":"2024-09-16T09:31:09.274797+0000","last_became_peered":"2024-09-16T09:31:09.274797+0000","last_unstale":"2024-09-16T09:36:08.363391+0000","last_undegraded":"2024-09-16T09:36:08.363391+0000","last_fullsized":"2024-09-16T09:36:08.363391+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:51.780980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:34:51.780980+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:36:10.448717+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"285'8","reported_seq":286,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.404674+0000","last_change":"2024-09-16T09:34:50.206999+0000","last_active":"2024-09-16T09:36:12.404674+0000","last_peered":"2024-09-16T09:36:12.404674+0000","last_clean":"2024-09-16T09:36:12.404674+0000","last_became_active":"2024-09-16T09:31:09.273682+0000","last_became_peered":"2024-09-16T09:31:09.273682+0000","last_unstale":"2024-09-16T09:36:12.404674+0000","last_undegraded":"2024-09-16T09:36:12.404674+0000","last_fullsized":"2024-09-16T09:36:12.404674+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'4","last_scrub_stamp":"2024-09-16T09:34:50.206916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:34:50.206916+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":9,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":556,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-09-16T09:36:08.363079+0000","last_change":"2024-09-16T09:33:18.641115+0000","last_active":"2024-09-16T09:36:08.363079+0000","last_peered":"2024-09-16T09:36:08.363079+0000","last_clean":"2024-09-16T09:36:08.363079+0000","last_became_active":"2024-09-16T09:31:09.274719+0000","last_became_peered":"2024-09-16T09:31:09.274719+0000","last_unstale":"2024-09-16T09:36:08.363079+0000","last_undegraded":"2024-09-16T09:36:08.363079+0000","last_fullsized":"2024-09-16T09:36:08.363079+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:18.641069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:18.641069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":548,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.444476+0000","last_change":"2024-09-16T09:33:37.941485+0000","last_active":"2024-09-16T09:36:12.444476+0000","last_peered":"2024-09-16T09:36:12.444476+0000","last_clean":"2024-09-16T09:36:12.444476+0000","last_became_active":"2024-09-16T09:25:05.712545+0000","last_became_peered":"2024-09-16T09:25:05.712545+0000","last_unstale":"2024-09-16T09:36:12.444476+0000","last_undegraded":"2024-09-16T09:36:12.444476+0000","last_fullsized":"2024-09-16T09:36:12.444476+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:37.941403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:37.941403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":527,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.408491+0000","last_change":"2024-09-16T09:33:41.323824+0000","last_active":"2024-09-16T09:36:12.408491+0000","last_peered":"2024-09-16T09:36:12.408491+0000","last_clean":"2024-09-16T09:36:12.408491+0000","last_became_active":"2024-09-16T09:25:07.720117+0000","last_became_peered":"2024-09-16T09:25:07.720117+0000","last_unstale":"2024-09-16T09:36:12.408491+0000","last_undegraded":"2024-09-16T09:36:12.408491+0000","last_fullsized":"2024-09-16T09:36:12.408491+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:41.323773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:33:41.323773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.12","version":"304'7160","reported_seq":16906,"reported_epoch":306,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-16T09:36:09.054837+0000","last_change":"2024-09-16T09:35:07.002674+0000","last_active":"2024-09-16T09:36:09.054837+0000","last_peered":"2024-09-16T09:36:09.054837+0000","last_clean":"2024-09-16T09:35:04.631553+0000","last_became_active":"2024-09-16T09:35:06.928120+0000","last_became_peered":"2024-09-16T09:35:06.928120+0000","last_unstale":"2024-09-16T09:36:09.054837+0000","last_undegraded":"2024-09-16T09:35:06.916590+0000","last_fullsized":"2024-09-16T09:35:06.667188+0000","mapping_epoch":283,"log_start":"237'1650","ondisk_log_start":"237'1650","created":243,"last_epoch_clean":257,"parent":"0.0","parent_split_bits":5,"last_scrub":"274'4970","last_scrub_stamp":"2024-09-16T09:34:48.381443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:34:48.381443+0000","objects_scrubbed":3385,"log_size":5510,"log_dups_size":0,"ondisk_log_size":5510,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":905,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":141099008,"num_objects":2153,"num_object_clones":0,"num_object_copies":6459,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":199,"num_objects_misplaced":4131,"num_objects_unfound":0,"num_objects_dirty":2153,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2208,"num_write_kb":31232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1541,"num_bytes_recovered":100990976,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[0,1],"avail_no_missing":["0","1","5"],"object_location_counts":[{"shards":"0,1","objects":-12},{"shards":"0,1,5","objects":1966}],"blocked_by":[],"up_primary":3,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":164,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.414202+0000","last_change":"2024-09-16T09:33:30.248335+0000","last_active":"2024-09-16T09:36:12.414202+0000","last_peered":"2024-09-16T09:36:12.414202+0000","last_clean":"2024-09-16T09:36:12.414202+0000","last_became_active":"2024-09-16T09:33:30.248036+0000","last_became_peered":"2024-09-16T09:33:30.248036+0000","last_unstale":"2024-09-16T09:36:12.414202+0000","last_undegraded":"2024-09-16T09:36:12.414202+0000","last_fullsized":"2024-09-16T09:36:12.414202+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:18.339152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:33:18.339152+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":384,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-09-16T09:36:08.362378+0000","last_change":"2024-09-16T09:32:40.411546+0000","last_active":"2024-09-16T09:36:08.362378+0000","last_peered":"2024-09-16T09:36:08.362378+0000","last_clean":"2024-09-16T09:36:08.362378+0000","last_became_active":"2024-09-16T09:31:09.271662+0000","last_became_peered":"2024-09-16T09:31:09.271662+0000","last_unstale":"2024-09-16T09:36:08.362378+0000","last_undegraded":"2024-09-16T09:36:08.362378+0000","last_fullsized":"2024-09-16T09:36:08.362378+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:40.411487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:32:40.411487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"285'19","reported_seq":568,"reported_epoch":309,"state":"active+clean","last_fresh":"2024-09-16T09:36:12.445705+0000","last_change":"2024-09-16T09:33:19.843216+0000","last_active":"2024-09-16T09:36:12.445705+0000","last_peered":"2024-09-16T09:36:12.445705+0000","last_clean":"2024-09-16T09:36:12.445705+0000","last_became_active":"2024-09-16T09:25:03.707769+0000","last_became_peered":"2024-09-16T09:25:03.707769+0000","last_unstale":"2024-09-16T09:36:12.445705+0000","last_undegraded":"2024-09-16T09:36:12.445705+0000","last_fullsized":"2024-09-16T09:36:12.445705+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-16T09:33:19.843154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:33:19.843154+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":519,"reported_epoch":306,"state":"active+clean","last_fresh":"2024-09-16T09:36:08.362461+0000","last_change":"2024-09-16T09:32:31.325543+0000","last_active":"2024-09-16T09:36:08.362461+0000","last_peered":"2024-09-16T09:36:08.362461+0000","last_clean":"2024-09-16T09:36:08.362461+0000","last_became_active":"2024-09-16T09:25:05.711952+0000","last_became_peered":"2024-09-16T09:25:05.711952+0000","last_unstale":"2024-09-16T09:36:08.362461+0000","last_undegraded":"2024-09-16T09:36:08.362461+0000","last_fullsized":"2024-09-16T09:36:08.362461+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:31.325346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:32:31.325346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":25,"stat_sum":{"num_bytes":4259184664,"num_objects":64991,"num_object_clones":0,"num_object_copies":194973,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":199,"num_objects_misplaced":14571,"num_objects_unfound":0,"num_objects_dirty":64991,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":113588,"num_write_kb":4057345,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21267,"num_bytes_recovered":1382940672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":13855305728,"data_stored":13855293512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":114398,"ondisk_log_size":114398,"up":75,"acting":73,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":1117,"num_read_kb":874,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":116,"num_bytes_recovered":2916,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":681,"ondisk_log_size":681,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1986,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":665,"num_read_kb":660,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":21,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":327680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1659,"ondisk_log_size":1659,"up":63,"acting":63,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266827,"num_pgs":78,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3013872,"kb_used_data":2545376,"kb_used_omap":1,"kb_used_meta":468478,"kb_avail":90731280,"statfs":{"total":95995035648,"available":92908830720,"internally_reserved":0,"allocated":2606465024,"data_stored":2605718964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":479721930},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,1,1,14],"upper_bound":2048},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495760,"num_pgs":61,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1514756,"kb_used_data":1172592,"kb_used_omap":1,"kb_used_meta":342142,"kb_avail":92230396,"statfs":{"total":95995035648,"available":94443925504,"internally_reserved":0,"allocated":1200734208,"data_stored":1200028114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":350353862},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724695,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3106776,"kb_used_data":2548036,"kb_used_omap":1,"kb_used_meta":558718,"kb_avail":90638376,"statfs":{"total":95995035648,"available":92813697024,"internally_reserved":0,"allocated":2609188864,"data_stored":2608470517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":572127686},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,3,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953628,"num_pgs":72,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1786728,"kb_used_data":1404312,"kb_used_omap":1,"kb_used_meta":382398,"kb_avail":91958424,"statfs":{"total":95995035648,"available":94165426176,"internally_reserved":0,"allocated":1438015488,"data_stored":1437277354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":391576006},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149858,"num_pgs":78,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2146492,"kb_used_data":1591780,"kb_used_omap":1,"kb_used_meta":554558,"kb_avail":91598660,"statfs":{"total":95995035648,"available":93797027840,"internally_reserved":0,"allocated":1629982720,"data_stored":1629233214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":567867847},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,3],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411495,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":931152,"kb_used_data":548728,"kb_used_omap":1,"kb_used_meta":382398,"kb_avail":92814000,"statfs":{"total":95995035648,"available":95041536000,"internally_reserved":0,"allocated":561897472,"data_stored":561191286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":391576005},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575022,"num_pgs":75,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2910576,"kb_used_data":2355996,"kb_used_omap":1,"kb_used_meta":554558,"kb_avail":90834576,"statfs":{"total":95995035648,"available":93014605824,"internally_reserved":0,"allocated":2412539904,"data_stored":2411797080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":567867845},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738547,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1882184,"kb_used_data":1427572,"kb_used_omap":1,"kb_used_meta":454590,"kb_avail":91862968,"statfs":{"total":95995035648,"available":94067679232,"internally_reserved":0,"allocated":1461833728,"data_stored":1461107883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":465500613},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1459486720,"data_stored":1459486720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2404843520,"data_stored":2404843520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":557383680,"data_stored":557383680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1613758464,"data_stored":1613758464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1433145344,"data_stored":1433141272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":2598572032,"data_stored":2598567960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1191444480,"data_stored":1191444480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":2596671488,"data_stored":2596667416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T09:36:17.049 INFO:tasks.thrashosds.thrasher:Setting 4.b to [0, 5, 1] 2024-09-16T09:36:17.049 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '4.b', '0', '5', '1'] 2024-09-16T09:36:17.049 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd pg-upmap 4.b 0 5 1 2024-09-16T09:36:17.051 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:36:17.076 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:36:17.104 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:17.105 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:17.105 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:36:17.206 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:36:17.240 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:17.241 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:17.241 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:36:17.342 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:36:17.368 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:17.369 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:17.369 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:36:17.470 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:36:17.504 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:17.504 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:17.505 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:36:17.605 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:36:17.617 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:17 smithi070 ceph-mon[30388]: pgmap v561: 177 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 173 active+clean; 3.9 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 796 B/s rd, 0 B/s wr, 596 op/s; 199/191538 objects degraded (0.104%); 13648/191538 objects misplaced (7.125%); 12 MiB/s, 218 objects/s recovering 2024-09-16T09:36:17.617 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:17 smithi070 ceph-mon[40691]: pgmap v561: 177 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 173 active+clean; 3.9 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 796 B/s rd, 0 B/s wr, 596 op/s; 199/191538 objects degraded (0.104%); 13648/191538 objects misplaced (7.125%); 12 MiB/s, 218 objects/s recovering 2024-09-16T09:36:17.636 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:17.637 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:17.637 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:36:17.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:17 smithi084 ceph-mon[32629]: pgmap v561: 177 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 173 active+clean; 3.9 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 796 B/s rd, 0 B/s wr, 596 op/s; 199/191538 objects degraded (0.104%); 13648/191538 objects misplaced (7.125%); 12 MiB/s, 218 objects/s recovering 2024-09-16T09:36:17.738 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:36:17.769 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:17.770 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:17.770 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:36:17.870 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:36:17.927 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:17.927 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:17.927 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:36:17.956 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:36:17.956 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:36:18.001 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:36:18.001 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:36:18.029 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:36:18.032 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:36:18.032 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:36:18.058 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:18.059 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:18.059 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:36:18.077 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:36:18.077 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:36:18.111 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:36:18.121 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:36:18.121 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:36:18.160 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:36:18.173 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:36:18.174 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:36:18.185 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:18.186 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:18.186 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:36:18.221 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:36:18.221 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:36:18.256 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:36:18.256 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:36:18.287 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:36:18.341 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:18.342 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:18.342 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:36:18.442 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:36:18.472 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:18 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:36:18.472 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:18 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:36:18.472 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:18 smithi070 ceph-mon[30388]: osdmap e310: 8 total, 8 up, 8 in 2024-09-16T09:36:18.472 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:18 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/297607894' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:36:18.472 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:18 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:36:18.473 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:18 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:36:18.473 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:18 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1356626028' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:36:18.473 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:18 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:36:18.473 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:18 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:36:18.473 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:18 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:36:18.473 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:18 smithi070 ceph-mon[40691]: osdmap e310: 8 total, 8 up, 8 in 2024-09-16T09:36:18.473 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:18 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/297607894' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:36:18.473 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:18 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:36:18.473 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:18 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:36:18.473 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:18 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1356626028' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:36:18.473 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:18 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:36:18.488 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:18.489 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:18.490 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:36:18.590 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:36:18.629 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:18.630 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:18.631 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:36:18.732 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:36:18.783 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:18.784 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:18.784 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:36:18.885 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:36:18.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:18 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:36:18.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:18 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:36:18.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:18 smithi084 ceph-mon[32629]: osdmap e310: 8 total, 8 up, 8 in 2024-09-16T09:36:18.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:18 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/297607894' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:36:18.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:18 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:36:18.912 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:18 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:36:18.912 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:18 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1356626028' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:36:18.912 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:18 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:36:18.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:18.937 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:18.937 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:36:19.038 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:36:19.078 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:19.078 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:19.079 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:36:19.162 INFO:teuthology.orchestra.run.smithi070.stderr:nodeep-scrub is set 2024-09-16T09:36:19.179 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:36:19.217 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:19.217 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:19.217 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:36:19.318 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:36:19.354 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:19.355 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:19.355 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:36:19.456 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:36:19.489 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:19.490 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:19.490 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:36:19.591 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:36:19.617 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:19.618 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:19.618 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:36:19.697 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:19 smithi070 ceph-mon[30388]: pgmap v563: 177 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 173 active+clean; 3.9 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 804 B/s rd, 0 B/s wr, 691 op/s; 199/189846 objects degraded (0.105%); 13648/189846 objects misplaced (7.189%); 12 MiB/s, 220 objects/s recovering 2024-09-16T09:36:19.697 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:19 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:36:19.697 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:19 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-16T09:36:19.697 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:19 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-16T09:36:19.697 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:19 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-16T09:36:19.697 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:19 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-16T09:36:19.697 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:19 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-16T09:36:19.698 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:19 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-16T09:36:19.698 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:19 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-16T09:36:19.698 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:19 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-16T09:36:19.698 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:19 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:36:19.698 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:19 smithi070 ceph-mon[30388]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:36:19.698 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:19 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-16T09:36:19.698 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:19 smithi070 ceph-mon[30388]: osdmap e311: 8 total, 8 up, 8 in 2024-09-16T09:36:19.698 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:19 smithi070 ceph-mon[30388]: Health check update: Degraded data redundancy: 199/189846 objects degraded (0.105%), 1 pg degraded, 2 pgs undersized (PG_DEGRADED) 2024-09-16T09:36:19.698 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:19 smithi070 ceph-mon[40691]: pgmap v563: 177 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 173 active+clean; 3.9 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 804 B/s rd, 0 B/s wr, 691 op/s; 199/189846 objects degraded (0.105%); 13648/189846 objects misplaced (7.189%); 12 MiB/s, 220 objects/s recovering 2024-09-16T09:36:19.699 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:19 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:36:19.699 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:19 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-16T09:36:19.699 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:19 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-16T09:36:19.699 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:19 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-16T09:36:19.699 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:19 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-16T09:36:19.699 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:19 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-16T09:36:19.699 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:19 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-16T09:36:19.699 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:19 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-16T09:36:19.699 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:19 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-16T09:36:19.699 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:19 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:36:19.699 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:19 smithi070 ceph-mon[40691]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:36:19.699 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:19 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-16T09:36:19.699 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:19 smithi070 ceph-mon[40691]: osdmap e311: 8 total, 8 up, 8 in 2024-09-16T09:36:19.699 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:19 smithi070 ceph-mon[40691]: Health check update: Degraded data redundancy: 199/189846 objects degraded (0.105%), 1 pg degraded, 2 pgs undersized (PG_DEGRADED) 2024-09-16T09:36:19.719 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:36:19.739 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T09:36:19.739 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:36:19.743 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:19.744 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:19.744 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:36:19.845 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:36:19.873 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:19.873 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:19.874 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:36:19.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:19 smithi084 ceph-mon[32629]: pgmap v563: 177 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 173 active+clean; 3.9 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 804 B/s rd, 0 B/s wr, 691 op/s; 199/189846 objects degraded (0.105%); 13648/189846 objects misplaced (7.189%); 12 MiB/s, 220 objects/s recovering 2024-09-16T09:36:19.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:19 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:36:19.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:19 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-16T09:36:19.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:19 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-16T09:36:19.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:19 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-16T09:36:19.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:19 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-16T09:36:19.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:19 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-16T09:36:19.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:19 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-16T09:36:19.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:19 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-16T09:36:19.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:19 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-16T09:36:19.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:19 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:36:19.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:19 smithi084 ceph-mon[32629]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:36:19.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:19 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-16T09:36:19.912 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:19 smithi084 ceph-mon[32629]: osdmap e311: 8 total, 8 up, 8 in 2024-09-16T09:36:19.912 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:19 smithi084 ceph-mon[32629]: Health check update: Degraded data redundancy: 199/189846 objects degraded (0.105%), 1 pg degraded, 2 pgs undersized (PG_DEGRADED) 2024-09-16T09:36:19.974 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:36:20.000 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:20.001 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:20.001 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:36:20.102 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:36:20.128 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:20.129 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:20.129 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:36:20.230 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:36:20.257 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:20.258 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:20.258 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:36:20.359 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:36:20.401 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:20.402 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:20.402 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:36:20.503 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:36:20.539 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:20.540 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:20.540 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:36:20.641 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:36:20.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:20 smithi070 ceph-mon[30388]: Adjusting osd_memory_target on smithi084 to 3248M 2024-09-16T09:36:20.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:20 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-16T09:36:20.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:20 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-16T09:36:20.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:20 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-16T09:36:20.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:20 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-16T09:36:20.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:20 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-16T09:36:20.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:20 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-16T09:36:20.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:20 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-16T09:36:20.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:20 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-16T09:36:20.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:20 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:36:20.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:20 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:36:20.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:20 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:36:20.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:20 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:36:20.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:20 smithi070 ceph-mon[40691]: Adjusting osd_memory_target on smithi084 to 3248M 2024-09-16T09:36:20.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:20 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-16T09:36:20.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:20 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-16T09:36:20.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:20 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-16T09:36:20.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:20 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-16T09:36:20.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:20 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-16T09:36:20.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:20 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-16T09:36:20.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:20 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-16T09:36:20.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:20 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-16T09:36:20.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:20 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:36:20.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:20 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:36:20.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:20 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:36:20.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:20 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:36:20.674 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:20.674 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:20.674 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:36:20.770 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.1 dump_ops_in_flight 2024-09-16T09:36:20.775 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:36:20.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:20.985 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:20.985 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:36:20.986 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:20 smithi084 ceph-mon[32629]: Adjusting osd_memory_target on smithi084 to 3248M 2024-09-16T09:36:20.986 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:20 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-16T09:36:20.986 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:20 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-16T09:36:20.986 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:20 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-16T09:36:20.986 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:20 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-16T09:36:20.986 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:20 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-16T09:36:20.986 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:20 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-16T09:36:20.986 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:20 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-16T09:36:20.986 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:20 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-16T09:36:20.986 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:20 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:36:20.986 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:20 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:36:20.986 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:20 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:36:20.987 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:20 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:36:21.086 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:36:21.128 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:21.129 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:21.129 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:36:21.230 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:36:21.260 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:21.261 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:21.261 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:36:21.362 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:36:21.398 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:21.398 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:21.399 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:36:21.500 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:36:21.531 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:21.531 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:21.532 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:36:21.633 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:36:21.674 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:21.674 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:21.675 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:36:21.776 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:36:21.809 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:21.810 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:21.811 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:36:21.811 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:21 smithi084 ceph-mon[32629]: pgmap v565: 177 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 173 active+clean; 3.6 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1.27k op/s; 199/175086 objects degraded (0.114%); 11137/175086 objects misplaced (6.361%); 20 MiB/s, 346 objects/s recovering 2024-09-16T09:36:21.811 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:21 smithi084 ceph-mon[32629]: Adjusting osd_memory_target on smithi070 to 3248M 2024-09-16T09:36:21.911 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:36:21.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:21 smithi070 ceph-mon[30388]: pgmap v565: 177 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 173 active+clean; 3.6 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1.27k op/s; 199/175086 objects degraded (0.114%); 11137/175086 objects misplaced (6.361%); 20 MiB/s, 346 objects/s recovering 2024-09-16T09:36:21.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:21 smithi070 ceph-mon[30388]: Adjusting osd_memory_target on smithi070 to 3248M 2024-09-16T09:36:21.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:21 smithi070 ceph-mon[40691]: pgmap v565: 177 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 173 active+clean; 3.6 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1.27k op/s; 199/175086 objects degraded (0.114%); 11137/175086 objects misplaced (6.361%); 20 MiB/s, 346 objects/s recovering 2024-09-16T09:36:21.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:21 smithi070 ceph-mon[40691]: Adjusting osd_memory_target on smithi070 to 3248M 2024-09-16T09:36:21.941 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:21.942 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:21.942 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:36:22.043 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:36:22.080 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:22.081 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:22.082 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:36:22.183 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:36:22.209 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:22.210 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:22.210 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:36:22.311 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:36:22.344 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:22.345 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:22.345 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:36:22.421 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:36:21 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:36:21] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:36:22.445 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:36:22.489 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:22.489 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:22.489 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:36:22.590 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:36:22.628 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:22.628 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:22.628 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:36:22.729 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:36:22.767 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:22.767 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:22.768 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:36:22.770 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T09:36:22.869 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:36:22.931 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:22.932 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:22.932 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:36:23.033 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:36:23.056 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:23.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:23.057 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:36:23.158 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:36:23.189 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:23.189 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:23.190 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:36:23.290 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:36:23.322 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:23.323 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:23.323 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:36:23.424 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:36:23.457 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:23.458 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:23.458 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:36:23.460 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:36:23 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:36:23.460 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:23 smithi084 ceph-mon[32629]: pgmap v566: 177 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 173 active+clean; 3.6 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 853 op/s; 199/175086 objects degraded (0.114%); 11137/175086 objects misplaced (6.361%); 12 MiB/s, 189 objects/s recovering 2024-09-16T09:36:23.460 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:23 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:36:23.460 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:23 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:36:23.559 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:36:23.578 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:36:23.584 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:36:23.587 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:23.588 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:23.588 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:36:23.689 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:36:23.711 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:23.711 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:23.712 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:36:23.715 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:23 smithi070 ceph-mon[30388]: pgmap v566: 177 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 173 active+clean; 3.6 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 853 op/s; 199/175086 objects degraded (0.114%); 11137/175086 objects misplaced (6.361%); 12 MiB/s, 189 objects/s recovering 2024-09-16T09:36:23.715 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:23 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:36:23.715 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:23 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:36:23.715 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:23 smithi070 ceph-mon[40691]: pgmap v566: 177 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 173 active+clean; 3.6 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 853 op/s; 199/175086 objects degraded (0.114%); 11137/175086 objects misplaced (6.361%); 12 MiB/s, 189 objects/s recovering 2024-09-16T09:36:23.716 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:23 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:36:23.716 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:23 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:36:23.813 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:36:23.821 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:36:23.840 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:23.841 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:23.841 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:36:23.942 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:36:23.992 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:23.992 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:23.992 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:36:24.093 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:36:24.132 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:24.132 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:24.133 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:36:24.234 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:36:24.258 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:24.259 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:24.260 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:36:24.361 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:36:24.402 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:24.403 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:24.403 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:36:24.504 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:36:24.542 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:24.542 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:24.543 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:36:24.644 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:36:24.681 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:24.682 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:24.682 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:36:24.783 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:36:24.816 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:24.817 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:24.817 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:36:24.821 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:24 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:36:24.821 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:24 smithi084 ceph-mon[32629]: osdmap e312: 8 total, 8 up, 8 in 2024-09-16T09:36:24.821 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:24 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:36:24.821 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:24 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:36:24.821 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:24 smithi084 ceph-mon[32629]: Health check update: Degraded data redundancy: 199/175086 objects degraded (0.114%), 1 pg degraded, 2 pgs undersized (PG_DEGRADED) 2024-09-16T09:36:24.821 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:24 smithi084 ceph-mon[32629]: osdmap e313: 8 total, 8 up, 8 in 2024-09-16T09:36:24.918 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:36:24.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:24 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:36:24.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:24 smithi070 ceph-mon[30388]: osdmap e312: 8 total, 8 up, 8 in 2024-09-16T09:36:24.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:24 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:36:24.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:24 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:36:24.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:24 smithi070 ceph-mon[30388]: Health check update: Degraded data redundancy: 199/175086 objects degraded (0.114%), 1 pg degraded, 2 pgs undersized (PG_DEGRADED) 2024-09-16T09:36:24.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:24 smithi070 ceph-mon[30388]: osdmap e313: 8 total, 8 up, 8 in 2024-09-16T09:36:24.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:24 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:36:24.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:24 smithi070 ceph-mon[40691]: osdmap e312: 8 total, 8 up, 8 in 2024-09-16T09:36:24.923 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:24 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:36:24.923 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:24 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:36:24.923 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:24 smithi070 ceph-mon[40691]: Health check update: Degraded data redundancy: 199/175086 objects degraded (0.114%), 1 pg degraded, 2 pgs undersized (PG_DEGRADED) 2024-09-16T09:36:24.923 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:24 smithi070 ceph-mon[40691]: osdmap e313: 8 total, 8 up, 8 in 2024-09-16T09:36:24.956 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:24.957 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:24.957 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:36:25.058 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:36:25.089 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:25.090 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:25.090 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:36:25.191 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:36:25.223 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:25.224 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:25.224 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:36:25.324 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:36:25.350 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:25.350 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:25.351 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:36:25.451 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:36:25.483 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:25.484 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:25.484 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:36:25.524 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd unset noscrub 2024-09-16T09:36:25.585 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:36:25.606 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:25.607 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:25.607 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:36:25.708 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:36:25.744 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:25.745 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:25.745 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:36:25.846 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:36:25.875 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:25.876 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:25.877 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:36:25.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:25 smithi084 ceph-mon[32629]: pgmap v568: 177 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 173 active+clean; 3.3 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1.49k op/s; 199/163299 objects degraded (0.122%); 9557/163299 objects misplaced (5.852%); 20 MiB/s, 313 objects/s recovering 2024-09-16T09:36:25.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:25 smithi070 ceph-mon[30388]: pgmap v568: 177 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 173 active+clean; 3.3 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1.49k op/s; 199/163299 objects degraded (0.122%); 9557/163299 objects misplaced (5.852%); 20 MiB/s, 313 objects/s recovering 2024-09-16T09:36:25.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:25 smithi070 ceph-mon[40691]: pgmap v568: 177 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 173 active+clean; 3.3 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1.49k op/s; 199/163299 objects degraded (0.122%); 9557/163299 objects misplaced (5.852%); 20 MiB/s, 313 objects/s recovering 2024-09-16T09:36:25.977 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:36:26.008 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:26.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:26.009 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:36:26.110 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:36:26.144 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:26.144 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:26.144 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:36:26.245 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:36:26.283 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:26.284 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:26.284 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:36:26.385 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:36:26.420 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:26.421 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:26.421 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:36:26.522 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:36:26.552 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:26.553 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:26.553 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:36:26.654 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:36:26.682 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:26.683 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:26.683 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:36:26.785 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:36:26.812 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:26.812 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:26.813 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:36:26.913 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:36:26.952 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:26.952 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:26.953 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:36:27.054 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:36:27.085 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:27.085 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:27.085 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:36:27.186 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:36:27.207 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:27.208 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:27.208 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:36:27.309 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:36:27.334 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:27.335 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:27.335 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:36:27.436 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:36:27.460 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:27.461 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:27.461 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:36:27.562 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:36:27.599 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:27.599 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:27.600 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:36:27.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:27 smithi070 ceph-mon[30388]: pgmap v570: 177 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 173 active+clean; 3.2 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 779 B/s rd, 0 B/s wr, 816 op/s; 199/159003 objects degraded (0.125%); 8758/159003 objects misplaced (5.508%); 11 MiB/s, 175 objects/s recovering 2024-09-16T09:36:27.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:27 smithi070 ceph-mon[30388]: osdmap e314: 8 total, 8 up, 8 in 2024-09-16T09:36:27.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:27 smithi070 ceph-mon[40691]: pgmap v570: 177 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 173 active+clean; 3.2 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 779 B/s rd, 0 B/s wr, 816 op/s; 199/159003 objects degraded (0.125%); 8758/159003 objects misplaced (5.508%); 11 MiB/s, 175 objects/s recovering 2024-09-16T09:36:27.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:27 smithi070 ceph-mon[40691]: osdmap e314: 8 total, 8 up, 8 in 2024-09-16T09:36:27.701 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:36:27.729 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:27.730 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:27.730 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:36:27.831 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:36:27.853 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:27.854 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:27.854 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:36:27.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:27 smithi084 ceph-mon[32629]: pgmap v570: 177 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 173 active+clean; 3.2 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 779 B/s rd, 0 B/s wr, 816 op/s; 199/159003 objects degraded (0.125%); 8758/159003 objects misplaced (5.508%); 11 MiB/s, 175 objects/s recovering 2024-09-16T09:36:27.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:27 smithi084 ceph-mon[32629]: osdmap e314: 8 total, 8 up, 8 in 2024-09-16T09:36:27.956 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:36:27.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:27.997 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:27.997 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:36:28.098 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:36:28.126 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:28.126 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:28.127 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:36:28.228 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:36:28.258 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:28.258 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:28.259 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:36:28.359 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:36:28.390 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:28.391 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:28.391 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:36:28.492 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:36:28.520 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:28.521 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:28.522 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:36:28.622 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:36:28.668 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:28.669 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:28.669 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:36:28.771 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:36:28.799 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:28 smithi070 ceph-mon[30388]: osdmap e315: 8 total, 8 up, 8 in 2024-09-16T09:36:28.800 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:28 smithi070 ceph-mon[30388]: pgmap v573: 177 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 173 active+clean; 3.2 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 0 B/s wr, 446 op/s; 199/157611 objects degraded (0.126%); 8758/157611 objects misplaced (5.557%); 4.7 MiB/s, 75 objects/s recovering 2024-09-16T09:36:28.800 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:28 smithi070 ceph-mon[40691]: osdmap e315: 8 total, 8 up, 8 in 2024-09-16T09:36:28.801 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:28 smithi070 ceph-mon[40691]: pgmap v573: 177 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 173 active+clean; 3.2 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 0 B/s wr, 446 op/s; 199/157611 objects degraded (0.126%); 8758/157611 objects misplaced (5.557%); 4.7 MiB/s, 75 objects/s recovering 2024-09-16T09:36:28.810 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:28.811 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:28.811 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:36:28.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:28 smithi084 ceph-mon[32629]: osdmap e315: 8 total, 8 up, 8 in 2024-09-16T09:36:28.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:28 smithi084 ceph-mon[32629]: pgmap v573: 177 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+recovering+undersized+remapped, 2 active+remapped+backfilling, 173 active+clean; 3.2 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 0 B/s wr, 446 op/s; 199/157611 objects degraded (0.126%); 8758/157611 objects misplaced (5.557%); 4.7 MiB/s, 75 objects/s recovering 2024-09-16T09:36:28.913 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:36:28.939 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:28.940 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:28.941 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:36:29.026 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T09:36:29.027 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:36:29.041 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:36:29.076 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:29.077 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:29.077 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:36:29.178 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:36:29.205 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:29.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:29.206 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:36:29.307 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:36:29.335 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:29.336 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:29.336 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:36:29.437 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:36:29.470 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:29.470 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:29.471 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:36:29.571 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:36:29.598 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:29.599 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:29.599 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:36:29.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:29 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3305594654' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:36:29.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:29 smithi070 ceph-mon[30388]: Health check update: Degraded data redundancy: 199/157611 objects degraded (0.126%), 1 pg degraded, 2 pgs undersized (PG_DEGRADED) 2024-09-16T09:36:29.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:29 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3305594654' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:36:29.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:29 smithi070 ceph-mon[40691]: Health check update: Degraded data redundancy: 199/157611 objects degraded (0.126%), 1 pg degraded, 2 pgs undersized (PG_DEGRADED) 2024-09-16T09:36:29.700 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:36:29.727 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:29.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:29.728 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:36:29.764 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:36:29.765 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:36:29.765 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:36:29.765 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:36:29.765 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:36:29.765 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:36:29.766 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:36:29.766 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:36:29.766 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:36:29.766 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:36:29.766 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:36:29.766 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:36:29.766 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:36:29.767 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:36:29.767 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:36:29.767 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:36:29.829 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:36:29.859 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:29.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:29.860 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:36:29.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:29 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3305594654' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:36:29.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:29 smithi084 ceph-mon[32629]: Health check update: Degraded data redundancy: 199/157611 objects degraded (0.126%), 1 pg degraded, 2 pgs undersized (PG_DEGRADED) 2024-09-16T09:36:29.961 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:36:29.992 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:29.992 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:29.993 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:36:30.094 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:36:30.131 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:30.132 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:30.132 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:36:30.233 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:36:30.264 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:30.265 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:30.265 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:36:30.366 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:36:30.399 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:30.400 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:30.402 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:36:30.503 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:36:30.532 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:30.533 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:30.533 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:36:30.549 INFO:teuthology.orchestra.run.smithi070.stderr:set 4.b pg_upmap mapping to [0,5,1] 2024-09-16T09:36:30.634 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:36:30.673 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:30.673 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:30.673 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:36:30.774 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:36:30.810 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:30.810 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:30.811 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:36:30.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:30 smithi084 ceph-mon[32629]: pgmap v574: 177 pgs: 1 active+remapped+backfilling, 1 active+recovering+undersized+degraded+remapped, 1 active+recovering+undersized+remapped, 174 active+clean; 2.9 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.05k op/s; 199/144411 objects degraded (0.138%); 6663/144411 objects misplaced (4.614%); 15 MiB/s, 232 objects/s recovering 2024-09-16T09:36:30.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:30 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:36:30.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:30 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:36:30.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:30 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/4011729206' entity='client.admin' cmd={"prefix": "osd pg-upmap", "pgid": "4.b", "id": [0, 5, 1]} : dispatch 2024-09-16T09:36:30.912 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:36:30.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:30 smithi070 ceph-mon[30388]: pgmap v574: 177 pgs: 1 active+remapped+backfilling, 1 active+recovering+undersized+degraded+remapped, 1 active+recovering+undersized+remapped, 174 active+clean; 2.9 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.05k op/s; 199/144411 objects degraded (0.138%); 6663/144411 objects misplaced (4.614%); 15 MiB/s, 232 objects/s recovering 2024-09-16T09:36:30.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:30 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:36:30.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:30 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:36:30.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:30 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/4011729206' entity='client.admin' cmd={"prefix": "osd pg-upmap", "pgid": "4.b", "id": [0, 5, 1]} : dispatch 2024-09-16T09:36:30.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:30 smithi070 ceph-mon[40691]: pgmap v574: 177 pgs: 1 active+remapped+backfilling, 1 active+recovering+undersized+degraded+remapped, 1 active+recovering+undersized+remapped, 174 active+clean; 2.9 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.05k op/s; 199/144411 objects degraded (0.138%); 6663/144411 objects misplaced (4.614%); 15 MiB/s, 232 objects/s recovering 2024-09-16T09:36:30.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:30 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:36:30.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:30 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:36:30.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:30 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/4011729206' entity='client.admin' cmd={"prefix": "osd pg-upmap", "pgid": "4.b", "id": [0, 5, 1]} : dispatch 2024-09-16T09:36:30.943 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:30.944 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:30.944 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:36:31.045 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:36:31.080 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:31.081 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:31.081 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:36:31.182 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:36:31.229 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:31.230 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:31.231 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:36:31.331 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:36:31.362 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:31.363 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:31.363 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:36:31.404 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.1 dump_blocked_ops 2024-09-16T09:36:31.464 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:36:31.506 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:31.507 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:31.507 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:36:31.608 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:36:31.630 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:31.631 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:31.631 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:36:31.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:31 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/4011729206' entity='client.admin' cmd='[{"prefix": "osd pg-upmap", "pgid": "4.b", "id": [0, 5, 1]}]': finished 2024-09-16T09:36:31.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:31 smithi070 ceph-mon[30388]: osdmap e316: 8 total, 8 up, 8 in 2024-09-16T09:36:31.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:31 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/4011729206' entity='client.admin' cmd='[{"prefix": "osd pg-upmap", "pgid": "4.b", "id": [0, 5, 1]}]': finished 2024-09-16T09:36:31.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:31 smithi070 ceph-mon[40691]: osdmap e316: 8 total, 8 up, 8 in 2024-09-16T09:36:31.731 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:36:31.777 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:31.777 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:31.777 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:36:31.878 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:36:31.908 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:31.908 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:31.909 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:36:31.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:31 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/4011729206' entity='client.admin' cmd='[{"prefix": "osd pg-upmap", "pgid": "4.b", "id": [0, 5, 1]}]': finished 2024-09-16T09:36:31.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:31 smithi084 ceph-mon[32629]: osdmap e316: 8 total, 8 up, 8 in 2024-09-16T09:36:32.010 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:36:32.047 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:32.048 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:32.048 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:36:32.149 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:36:32.182 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:36:31 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:36:31] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:36:32.183 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:32.183 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:32.183 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:36:32.284 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:36:32.313 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:32.313 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:32.314 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:36:32.415 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:36:32.445 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:32.446 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:32.447 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:36:32.547 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:36:32.579 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T09:36:32.586 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:32.587 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:32.587 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:36:32.688 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:36:32.722 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:32.723 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:32.723 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:36:32.824 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:36:32.851 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:32.851 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:32.852 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:36:32.878 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:32 smithi070 ceph-mon[30388]: osdmap e317: 8 total, 8 up, 8 in 2024-09-16T09:36:32.878 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:32 smithi070 ceph-mon[30388]: pgmap v577: 177 pgs: 1 active+remapped+backfilling, 1 active+recovering+undersized+degraded+remapped, 1 active+recovering+undersized+remapped, 174 active+clean; 2.9 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 982 B/s rd, 0 B/s wr, 934 op/s; 199/144411 objects degraded (0.138%); 6663/144411 objects misplaced (4.614%); 13 MiB/s, 206 objects/s recovering 2024-09-16T09:36:32.878 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:32 smithi070 ceph-mon[40691]: osdmap e317: 8 total, 8 up, 8 in 2024-09-16T09:36:32.878 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:32 smithi070 ceph-mon[40691]: pgmap v577: 177 pgs: 1 active+remapped+backfilling, 1 active+recovering+undersized+degraded+remapped, 1 active+recovering+undersized+remapped, 174 active+clean; 2.9 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 982 B/s rd, 0 B/s wr, 934 op/s; 199/144411 objects degraded (0.138%); 6663/144411 objects misplaced (4.614%); 13 MiB/s, 206 objects/s recovering 2024-09-16T09:36:32.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:32 smithi084 ceph-mon[32629]: osdmap e317: 8 total, 8 up, 8 in 2024-09-16T09:36:32.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:32 smithi084 ceph-mon[32629]: pgmap v577: 177 pgs: 1 active+remapped+backfilling, 1 active+recovering+undersized+degraded+remapped, 1 active+recovering+undersized+remapped, 174 active+clean; 2.9 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 982 B/s rd, 0 B/s wr, 934 op/s; 199/144411 objects degraded (0.138%); 6663/144411 objects misplaced (4.614%); 13 MiB/s, 206 objects/s recovering 2024-09-16T09:36:32.953 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:36:32.976 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:32.976 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:32.977 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:36:33.078 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:36:33.099 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:33.099 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:33.100 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:36:33.200 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:36:33.232 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:33.233 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:33.233 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:36:33.334 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:36:33.370 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:33.370 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:33.371 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:36:33.472 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:36:33.503 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:33.504 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:33.504 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:36:33.570 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:36:33 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:36:33.606 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:36:33.640 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:33.640 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:33.641 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:36:33.723 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:36:33.741 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:36:33.772 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:33.774 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:33.774 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:36:33.875 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:36:33.878 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:33 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:36:33.878 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:33 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:36:33.880 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:33 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:36:33.880 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:33 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:36:33.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:33 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:36:33.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:33 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:36:33.924 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:33.924 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:33.925 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:36:34.026 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:36:34.066 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:34.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:34.067 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:36:34.168 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:36:34.205 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:34.205 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:34.206 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:36:34.306 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:36:34.366 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:34.367 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:34.367 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:36:34.468 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:36:34.504 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:36:34.521 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:34.522 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:34.522 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:36:34.592 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:36:34.623 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:36:34.653 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:34.653 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:34.654 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:36:34.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:34 smithi070 ceph-mon[30388]: pgmap v578: 177 pgs: 1 active+remapped+backfilling, 1 active+recovering+undersized+degraded+remapped, 1 active+recovering+undersized+remapped, 174 active+clean; 2.7 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1.47k op/s; 199/131556 objects degraded (0.151%); 6309/131556 objects misplaced (4.796%); 14 MiB/s, 222 objects/s recovering 2024-09-16T09:36:34.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:34 smithi070 ceph-mon[30388]: Health check update: Degraded data redundancy: 199/144411 objects degraded (0.138%), 1 pg degraded, 2 pgs undersized (PG_DEGRADED) 2024-09-16T09:36:34.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:34 smithi070 ceph-mon[40691]: pgmap v578: 177 pgs: 1 active+remapped+backfilling, 1 active+recovering+undersized+degraded+remapped, 1 active+recovering+undersized+remapped, 174 active+clean; 2.7 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1.47k op/s; 199/131556 objects degraded (0.151%); 6309/131556 objects misplaced (4.796%); 14 MiB/s, 222 objects/s recovering 2024-09-16T09:36:34.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:34 smithi070 ceph-mon[40691]: Health check update: Degraded data redundancy: 199/144411 objects degraded (0.138%), 1 pg degraded, 2 pgs undersized (PG_DEGRADED) 2024-09-16T09:36:34.755 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:36:34.784 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:34.785 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:34.785 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:36:34.886 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:36:34.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:34 smithi084 ceph-mon[32629]: pgmap v578: 177 pgs: 1 active+remapped+backfilling, 1 active+recovering+undersized+degraded+remapped, 1 active+recovering+undersized+remapped, 174 active+clean; 2.7 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1.47k op/s; 199/131556 objects degraded (0.151%); 6309/131556 objects misplaced (4.796%); 14 MiB/s, 222 objects/s recovering 2024-09-16T09:36:34.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:34 smithi084 ceph-mon[32629]: Health check update: Degraded data redundancy: 199/144411 objects degraded (0.138%), 1 pg degraded, 2 pgs undersized (PG_DEGRADED) 2024-09-16T09:36:34.917 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:34.917 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:34.918 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:36:35.018 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:36:35.048 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:35.049 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:35.049 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:36:35.150 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:36:35.177 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:35.178 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:35.178 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:36:35.279 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:36:35.318 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:35.319 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:35.319 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:36:35.420 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:36:35.457 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:36:35.457 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:36:35.469 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:35.469 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:35.469 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:36:35.490 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:36:35.490 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:36:35.536 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:36:35.537 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:36:35.570 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:36:35.582 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:36:35.583 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:36:35.591 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T09:36:35.591 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:36:35.598 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:35.599 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:35.599 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:36:35.606 INFO:teuthology.orchestra.run.smithi070.stderr:noscrub is unset 2024-09-16T09:36:35.632 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:36:35.632 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:36:35.676 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:36:35.677 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:36:35.700 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:36:35.709 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:36:35.709 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:36:35.726 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:35.727 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:35.727 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:36:35.743 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:36:35.743 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:36:35.800 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:35 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/595352861' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:36:35.800 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:35 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:36:35.800 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:35 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/4188347563' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:36:35.800 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:35 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/595352861' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:36:35.801 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:35 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:36:35.801 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:35 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/4188347563' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:36:35.828 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:36:35.853 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:35.854 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:35.855 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:36:35.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:35 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/595352861' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:36:35.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:35 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:36:35.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:35 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/4188347563' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:36:35.956 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:36:35.985 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:35.986 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:35.986 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:36:36.087 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:36:36.110 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:36.111 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:36.111 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:36:36.212 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:36:36.236 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:36.237 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:36.237 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:36:36.338 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:36:36.361 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:36.362 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:36.362 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:36:36.463 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:36:36.485 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:36.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:36.487 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:36:36.587 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:36:36.614 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:36.615 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:36.615 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:36:36.716 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:36:36.752 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:36.753 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:36.753 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:36:36.796 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:36 smithi070 ceph-mon[30388]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:36:36.796 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:36 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-16T09:36:36.797 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:36 smithi070 ceph-mon[30388]: osdmap e318: 8 total, 8 up, 8 in 2024-09-16T09:36:36.797 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:36 smithi070 ceph-mon[30388]: 3.3 scrub starts 2024-09-16T09:36:36.797 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:36 smithi070 ceph-mon[30388]: 3.3 scrub ok 2024-09-16T09:36:36.797 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:36 smithi070 ceph-mon[30388]: pgmap v580: 177 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+recovering+undersized+remapped, 1 active+remapped+backfilling, 174 active+clean; 2.6 GiB data, 13 GiB used, 703 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 982 op/s; 199/126738 objects degraded (0.157%); 5399/126738 objects misplaced (4.260%); 7.0 MiB/s, 112 objects/s recovering 2024-09-16T09:36:36.797 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:36 smithi070 ceph-mon[40691]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:36:36.797 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:36 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-16T09:36:36.797 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:36 smithi070 ceph-mon[40691]: osdmap e318: 8 total, 8 up, 8 in 2024-09-16T09:36:36.797 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:36 smithi070 ceph-mon[40691]: 3.3 scrub starts 2024-09-16T09:36:36.797 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:36 smithi070 ceph-mon[40691]: 3.3 scrub ok 2024-09-16T09:36:36.797 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:36 smithi070 ceph-mon[40691]: pgmap v580: 177 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+recovering+undersized+remapped, 1 active+remapped+backfilling, 174 active+clean; 2.6 GiB data, 13 GiB used, 703 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 982 op/s; 199/126738 objects degraded (0.157%); 5399/126738 objects misplaced (4.260%); 7.0 MiB/s, 112 objects/s recovering 2024-09-16T09:36:36.855 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:36:36.883 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:36.883 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:36.884 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:36:36.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:36 smithi084 ceph-mon[32629]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:36:36.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:36 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-16T09:36:36.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:36 smithi084 ceph-mon[32629]: osdmap e318: 8 total, 8 up, 8 in 2024-09-16T09:36:36.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:36 smithi084 ceph-mon[32629]: 3.3 scrub starts 2024-09-16T09:36:36.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:36 smithi084 ceph-mon[32629]: 3.3 scrub ok 2024-09-16T09:36:36.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:36 smithi084 ceph-mon[32629]: pgmap v580: 177 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+recovering+undersized+remapped, 1 active+remapped+backfilling, 174 active+clean; 2.6 GiB data, 13 GiB used, 703 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 982 op/s; 199/126738 objects degraded (0.157%); 5399/126738 objects misplaced (4.260%); 7.0 MiB/s, 112 objects/s recovering 2024-09-16T09:36:36.984 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:36:37.023 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:37.024 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:37.024 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:36:37.125 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:36:37.160 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:37.161 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:37.162 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:36:37.217 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.1 dump_historic_ops 2024-09-16T09:36:37.262 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:36:37.293 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:37.293 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:37.293 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:36:37.394 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:36:37.426 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:37.426 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:37.427 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:36:37.527 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:36:37.557 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:37.557 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:37.558 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:36:37.658 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:36:37.685 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:37.686 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:37.686 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:36:37.786 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:36:37.815 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:37.816 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:37.816 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:36:37.842 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:36:37.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:37 smithi084 ceph-mon[32629]: osdmap e319: 8 total, 8 up, 8 in 2024-09-16T09:36:37.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:37 smithi084 ceph-mon[32629]: 6.d scrub starts 2024-09-16T09:36:37.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:37 smithi084 ceph-mon[32629]: 6.d scrub ok 2024-09-16T09:36:37.917 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:36:37.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:37 smithi070 ceph-mon[30388]: osdmap e319: 8 total, 8 up, 8 in 2024-09-16T09:36:37.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:37 smithi070 ceph-mon[30388]: 6.d scrub starts 2024-09-16T09:36:37.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:37 smithi070 ceph-mon[30388]: 6.d scrub ok 2024-09-16T09:36:37.923 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:37 smithi070 ceph-mon[40691]: osdmap e319: 8 total, 8 up, 8 in 2024-09-16T09:36:37.923 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:37 smithi070 ceph-mon[40691]: 6.d scrub starts 2024-09-16T09:36:37.923 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:37 smithi070 ceph-mon[40691]: 6.d scrub ok 2024-09-16T09:36:37.948 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:37.948 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:37.949 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:36:38.049 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:36:38.092 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:38.092 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:38.093 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:36:38.194 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:36:38.224 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:38.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:38.225 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:36:38.326 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:36:38.365 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:38.365 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:38.366 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:36:38.466 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:36:38.492 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T09:36:38.505 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:38.506 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:38.506 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:36:38.607 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:36:38.639 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:38.640 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:38.640 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:36:38.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:38 smithi070 ceph-mon[30388]: 9.18 scrub starts 2024-09-16T09:36:38.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:38 smithi070 ceph-mon[30388]: 6.15 scrub starts 2024-09-16T09:36:38.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:38 smithi070 ceph-mon[30388]: 6.15 scrub ok 2024-09-16T09:36:38.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:38 smithi070 ceph-mon[30388]: pgmap v582: 177 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+recovering+undersized+remapped, 1 active+remapped+backfilling, 174 active+clean; 2.5 GiB data, 13 GiB used, 703 GiB / 715 GiB avail; 830 B/s rd, 0 B/s wr, 1.03k op/s; 199/125373 objects degraded (0.159%); 5399/125373 objects misplaced (4.306%); 6.8 MiB/s, 109 objects/s recovering 2024-09-16T09:36:38.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:38 smithi070 ceph-mon[30388]: 5.16 scrub starts 2024-09-16T09:36:38.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:38 smithi070 ceph-mon[30388]: 5.16 scrub ok 2024-09-16T09:36:38.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:38 smithi070 ceph-mon[40691]: 9.18 scrub starts 2024-09-16T09:36:38.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:38 smithi070 ceph-mon[40691]: 6.15 scrub starts 2024-09-16T09:36:38.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:38 smithi070 ceph-mon[40691]: 6.15 scrub ok 2024-09-16T09:36:38.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:38 smithi070 ceph-mon[40691]: pgmap v582: 177 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+recovering+undersized+remapped, 1 active+remapped+backfilling, 174 active+clean; 2.5 GiB data, 13 GiB used, 703 GiB / 715 GiB avail; 830 B/s rd, 0 B/s wr, 1.03k op/s; 199/125373 objects degraded (0.159%); 5399/125373 objects misplaced (4.306%); 6.8 MiB/s, 109 objects/s recovering 2024-09-16T09:36:38.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:38 smithi070 ceph-mon[40691]: 5.16 scrub starts 2024-09-16T09:36:38.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:38 smithi070 ceph-mon[40691]: 5.16 scrub ok 2024-09-16T09:36:38.741 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:36:38.776 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T09:36:38.776 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T09:36:38.776 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-16T09:36:38.777 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd dump -f json-pretty 2024-09-16T09:36:38.796 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:38.796 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:38.797 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:36:38.898 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:36:38.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:38 smithi084 ceph-mon[32629]: 9.18 scrub starts 2024-09-16T09:36:38.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:38 smithi084 ceph-mon[32629]: 6.15 scrub starts 2024-09-16T09:36:38.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:38 smithi084 ceph-mon[32629]: 6.15 scrub ok 2024-09-16T09:36:38.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:38 smithi084 ceph-mon[32629]: pgmap v582: 177 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+recovering+undersized+remapped, 1 active+remapped+backfilling, 174 active+clean; 2.5 GiB data, 13 GiB used, 703 GiB / 715 GiB avail; 830 B/s rd, 0 B/s wr, 1.03k op/s; 199/125373 objects degraded (0.159%); 5399/125373 objects misplaced (4.306%); 6.8 MiB/s, 109 objects/s recovering 2024-09-16T09:36:38.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:38 smithi084 ceph-mon[32629]: 5.16 scrub starts 2024-09-16T09:36:38.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:38 smithi084 ceph-mon[32629]: 5.16 scrub ok 2024-09-16T09:36:38.948 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:38.949 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:38.949 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:36:39.050 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:36:39.077 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:39.077 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:39.078 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:36:39.178 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:36:39.212 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:39.213 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:39.213 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:36:39.314 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:36:39.345 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:39.346 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:39.346 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:36:39.447 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:36:39.479 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:39.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:39.480 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:36:39.581 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:36:39.617 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:39.617 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:39.618 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:36:39.719 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:36:39.736 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-16T09:36:39.746 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:39.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:39.747 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:36:39.758 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T09:36:39.758 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:36:39.848 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:36:39.879 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:39.880 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:39.880 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:36:39.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:39 smithi084 ceph-mon[32629]: 9.18 scrub ok 2024-09-16T09:36:39.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:39 smithi084 ceph-mon[32629]: Health check update: Degraded data redundancy: 199/125373 objects degraded (0.159%), 1 pg degraded, 2 pgs undersized (PG_DEGRADED) 2024-09-16T09:36:39.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:39 smithi070 ceph-mon[30388]: 9.18 scrub ok 2024-09-16T09:36:39.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:39 smithi070 ceph-mon[30388]: Health check update: Degraded data redundancy: 199/125373 objects degraded (0.159%), 1 pg degraded, 2 pgs undersized (PG_DEGRADED) 2024-09-16T09:36:39.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:39 smithi070 ceph-mon[40691]: 9.18 scrub ok 2024-09-16T09:36:39.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:39 smithi070 ceph-mon[40691]: Health check update: Degraded data redundancy: 199/125373 objects degraded (0.159%), 1 pg degraded, 2 pgs undersized (PG_DEGRADED) 2024-09-16T09:36:39.981 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:36:40.009 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:40.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:40.010 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:36:40.111 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:36:40.155 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:40.155 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:40.156 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:36:40.256 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:36:40.292 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:40.292 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:40.293 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:36:40.394 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:36:40.420 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:40.421 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:40.421 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:36:40.522 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:36:40.558 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:40.558 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:40.559 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:36:40.659 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:36:40.686 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:40.687 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:40.687 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:36:40.788 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:36:40.814 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:40.815 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:40.815 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:36:40.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:40 smithi084 ceph-mon[32629]: pgmap v583: 177 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+recovering+undersized+remapped, 175 active+clean; 2.2 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.39k op/s; 199/111018 objects degraded (0.179%); 3866/111018 objects misplaced (3.482%); 11 MiB/s, 177 objects/s recovering 2024-09-16T09:36:40.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:40 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "24"} : dispatch 2024-09-16T09:36:40.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:40 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "24"} : dispatch 2024-09-16T09:36:40.916 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:36:40.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:40 smithi070 ceph-mon[30388]: pgmap v583: 177 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+recovering+undersized+remapped, 175 active+clean; 2.2 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.39k op/s; 199/111018 objects degraded (0.179%); 3866/111018 objects misplaced (3.482%); 11 MiB/s, 177 objects/s recovering 2024-09-16T09:36:40.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:40 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "24"} : dispatch 2024-09-16T09:36:40.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:40 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "24"} : dispatch 2024-09-16T09:36:40.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:40 smithi070 ceph-mon[40691]: pgmap v583: 177 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+recovering+undersized+remapped, 175 active+clean; 2.2 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.39k op/s; 199/111018 objects degraded (0.179%); 3866/111018 objects misplaced (3.482%); 11 MiB/s, 177 objects/s recovering 2024-09-16T09:36:40.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:40 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "24"} : dispatch 2024-09-16T09:36:40.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:40 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "24"} : dispatch 2024-09-16T09:36:40.941 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:40.942 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:40.942 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:36:41.043 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:36:41.064 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:41.065 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:41.065 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:36:41.166 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:36:41.192 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:41.192 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:41.193 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:36:41.293 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:36:41.316 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:41.316 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:41.317 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:36:41.418 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:36:41.439 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:41.439 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:41.440 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:36:41.540 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:36:41.562 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:41.563 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:41.563 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:36:41.664 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:36:41.689 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:41.690 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:41.690 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:36:41.791 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:36:41.812 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:41.813 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:41.813 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:36:41.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:41 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "24"}]': finished 2024-09-16T09:36:41.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:41 smithi084 ceph-mon[32629]: osdmap e320: 8 total, 8 up, 8 in 2024-09-16T09:36:41.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:41 smithi084 ceph-mon[32629]: 1.10 scrub starts 2024-09-16T09:36:41.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:41 smithi084 ceph-mon[32629]: 1.10 scrub ok 2024-09-16T09:36:41.914 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:36:41.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:41 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "24"}]': finished 2024-09-16T09:36:41.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:41 smithi070 ceph-mon[30388]: osdmap e320: 8 total, 8 up, 8 in 2024-09-16T09:36:41.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:41 smithi070 ceph-mon[30388]: 1.10 scrub starts 2024-09-16T09:36:41.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:41 smithi070 ceph-mon[30388]: 1.10 scrub ok 2024-09-16T09:36:41.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:41 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "24"}]': finished 2024-09-16T09:36:41.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:41 smithi070 ceph-mon[40691]: osdmap e320: 8 total, 8 up, 8 in 2024-09-16T09:36:41.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:41 smithi070 ceph-mon[40691]: 1.10 scrub starts 2024-09-16T09:36:41.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:41 smithi070 ceph-mon[40691]: 1.10 scrub ok 2024-09-16T09:36:41.940 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:41.940 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:41.941 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:36:42.042 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:36:42.066 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:42.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:42.068 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:36:42.169 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:36:42.190 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:42.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:42.191 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:36:42.292 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:36:42.316 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:42.317 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:42.317 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:36:42.418 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:36:42.420 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:36:41 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:36:41] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:36:42.441 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:42.441 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:42.442 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:36:42.542 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:36:42.567 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:42.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:42.568 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:36:42.669 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:36:42.697 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:42.698 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:42.698 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:36:42.799 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:36:42.827 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:42.828 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:42.828 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:36:42.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:42 smithi084 ceph-mon[32629]: osdmap e321: 8 total, 8 up, 8 in 2024-09-16T09:36:42.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:42 smithi084 ceph-mon[32629]: 9.c scrub starts 2024-09-16T09:36:42.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:42 smithi084 ceph-mon[32629]: pgmap v586: 177 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+recovering+undersized+remapped, 175 active+clean; 2.2 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 873 op/s; 199/111018 objects degraded (0.179%); 3866/111018 objects misplaced (3.482%); 7.8 MiB/s, 124 objects/s recovering 2024-09-16T09:36:42.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:42 smithi084 ceph-mon[32629]: 9.c scrub ok 2024-09-16T09:36:42.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:42 smithi070 ceph-mon[30388]: osdmap e321: 8 total, 8 up, 8 in 2024-09-16T09:36:42.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:42 smithi070 ceph-mon[30388]: 9.c scrub starts 2024-09-16T09:36:42.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:42 smithi070 ceph-mon[30388]: pgmap v586: 177 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+recovering+undersized+remapped, 175 active+clean; 2.2 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 873 op/s; 199/111018 objects degraded (0.179%); 3866/111018 objects misplaced (3.482%); 7.8 MiB/s, 124 objects/s recovering 2024-09-16T09:36:42.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:42 smithi070 ceph-mon[30388]: 9.c scrub ok 2024-09-16T09:36:42.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:42 smithi070 ceph-mon[40691]: osdmap e321: 8 total, 8 up, 8 in 2024-09-16T09:36:42.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:42 smithi070 ceph-mon[40691]: 9.c scrub starts 2024-09-16T09:36:42.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:42 smithi070 ceph-mon[40691]: pgmap v586: 177 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+recovering+undersized+remapped, 175 active+clean; 2.2 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 873 op/s; 199/111018 objects degraded (0.179%); 3866/111018 objects misplaced (3.482%); 7.8 MiB/s, 124 objects/s recovering 2024-09-16T09:36:42.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:42 smithi070 ceph-mon[40691]: 9.c scrub ok 2024-09-16T09:36:42.929 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:36:42.960 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:42.960 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:42.960 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:36:43.061 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:36:43.087 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:43.088 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:43.088 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:36:43.189 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:36:43.215 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:43.216 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:43.217 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:36:43.317 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:36:43.340 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:43.340 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:43.341 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:36:43.441 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:36:43.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:43.464 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:43.464 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:36:43.565 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:36:43.590 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:43.591 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:43.591 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:36:43.641 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:36:43 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:36:43.692 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:36:43.725 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:43.726 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:43.726 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:36:43.827 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:36:43.857 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:43.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:43.858 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:36:43.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:43 smithi084 ceph-mon[32629]: 1.1 scrub starts 2024-09-16T09:36:43.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:43 smithi084 ceph-mon[32629]: 1.1 scrub ok 2024-09-16T09:36:43.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:43 smithi084 ceph-mon[32629]: osdmap e322: 8 total, 8 up, 8 in 2024-09-16T09:36:43.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:43 smithi084 ceph-mon[32629]: 6.4 scrub starts 2024-09-16T09:36:43.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:43 smithi084 ceph-mon[32629]: 6.4 scrub ok 2024-09-16T09:36:43.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:43 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:36:43.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:43 smithi070 ceph-mon[30388]: 1.1 scrub starts 2024-09-16T09:36:43.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:43 smithi070 ceph-mon[30388]: 1.1 scrub ok 2024-09-16T09:36:43.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:43 smithi070 ceph-mon[30388]: osdmap e322: 8 total, 8 up, 8 in 2024-09-16T09:36:43.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:43 smithi070 ceph-mon[30388]: 6.4 scrub starts 2024-09-16T09:36:43.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:43 smithi070 ceph-mon[30388]: 6.4 scrub ok 2024-09-16T09:36:43.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:43 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:36:43.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:43 smithi070 ceph-mon[40691]: 1.1 scrub starts 2024-09-16T09:36:43.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:43 smithi070 ceph-mon[40691]: 1.1 scrub ok 2024-09-16T09:36:43.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:43 smithi070 ceph-mon[40691]: osdmap e322: 8 total, 8 up, 8 in 2024-09-16T09:36:43.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:43 smithi070 ceph-mon[40691]: 6.4 scrub starts 2024-09-16T09:36:43.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:43 smithi070 ceph-mon[40691]: 6.4 scrub ok 2024-09-16T09:36:43.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:43 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:36:43.959 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:36:43.988 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:43.989 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:43.989 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:36:44.090 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:36:44.118 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:44.118 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:44.119 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:36:44.219 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:36:44.257 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:44.258 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:44.258 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:36:44.359 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:36:44.393 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:44.394 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:44.394 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:36:44.495 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:36:44.528 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:44.529 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:44.529 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:36:44.630 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:36:44.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:44.666 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:44.666 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:36:44.767 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:36:44.792 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:44.793 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:44.793 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:36:44.894 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:36:44.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:44 smithi070 ceph-mon[30388]: osdmap e323: 8 total, 8 up, 8 in 2024-09-16T09:36:44.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:44 smithi070 ceph-mon[30388]: pgmap v589: 176 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+recovering+undersized+remapped, 174 active+clean; 2.0 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 199/100494 objects degraded (0.198%); 3866/100494 objects misplaced (3.847%) 2024-09-16T09:36:44.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:44 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "23"} : dispatch 2024-09-16T09:36:44.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:44 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "23"} : dispatch 2024-09-16T09:36:44.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:44 smithi070 ceph-mon[30388]: 1.12 scrub starts 2024-09-16T09:36:44.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:44 smithi070 ceph-mon[30388]: 1.12 scrub ok 2024-09-16T09:36:44.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:44 smithi070 ceph-mon[30388]: Health check update: Degraded data redundancy: 199/111018 objects degraded (0.179%), 1 pg degraded, 2 pgs undersized (PG_DEGRADED) 2024-09-16T09:36:44.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:44 smithi070 ceph-mon[40691]: osdmap e323: 8 total, 8 up, 8 in 2024-09-16T09:36:44.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:44 smithi070 ceph-mon[40691]: pgmap v589: 176 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+recovering+undersized+remapped, 174 active+clean; 2.0 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 199/100494 objects degraded (0.198%); 3866/100494 objects misplaced (3.847%) 2024-09-16T09:36:44.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:44 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "23"} : dispatch 2024-09-16T09:36:44.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:44 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "23"} : dispatch 2024-09-16T09:36:44.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:44 smithi070 ceph-mon[40691]: 1.12 scrub starts 2024-09-16T09:36:44.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:44 smithi070 ceph-mon[40691]: 1.12 scrub ok 2024-09-16T09:36:44.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:44 smithi070 ceph-mon[40691]: Health check update: Degraded data redundancy: 199/111018 objects degraded (0.179%), 1 pg degraded, 2 pgs undersized (PG_DEGRADED) 2024-09-16T09:36:44.923 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:44.923 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:44.924 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:36:45.024 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:36:45.049 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:45.050 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:45.050 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:36:45.151 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:36:45.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:44 smithi084 ceph-mon[32629]: osdmap e323: 8 total, 8 up, 8 in 2024-09-16T09:36:45.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:44 smithi084 ceph-mon[32629]: pgmap v589: 176 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+recovering+undersized+remapped, 174 active+clean; 2.0 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 199/100494 objects degraded (0.198%); 3866/100494 objects misplaced (3.847%) 2024-09-16T09:36:45.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:44 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "23"} : dispatch 2024-09-16T09:36:45.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:44 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "23"} : dispatch 2024-09-16T09:36:45.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:44 smithi084 ceph-mon[32629]: 1.12 scrub starts 2024-09-16T09:36:45.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:44 smithi084 ceph-mon[32629]: 1.12 scrub ok 2024-09-16T09:36:45.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:44 smithi084 ceph-mon[32629]: Health check update: Degraded data redundancy: 199/111018 objects degraded (0.179%), 1 pg degraded, 2 pgs undersized (PG_DEGRADED) 2024-09-16T09:36:45.177 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:45.178 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:45.178 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:36:45.279 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:36:45.300 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:45.301 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:45.301 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:36:45.402 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:36:45.423 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:45.424 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:45.424 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:36:45.525 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:36:45.546 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:45.547 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:45.547 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:36:45.648 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:36:45.675 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:45.675 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:45.676 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:36:45.777 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:36:45.801 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:45.802 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:45.802 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:36:45.903 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:36:45.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:45 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "23"}]': finished 2024-09-16T09:36:45.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:45 smithi070 ceph-mon[30388]: osdmap e324: 8 total, 8 up, 8 in 2024-09-16T09:36:45.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:45 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "23"}]': finished 2024-09-16T09:36:45.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:45 smithi070 ceph-mon[40691]: osdmap e324: 8 total, 8 up, 8 in 2024-09-16T09:36:45.925 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:45.925 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:45.926 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:36:46.026 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:36:46.049 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:46.050 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:46.050 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:36:46.151 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:36:46.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:45 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "23"}]': finished 2024-09-16T09:36:46.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:45 smithi084 ceph-mon[32629]: osdmap e324: 8 total, 8 up, 8 in 2024-09-16T09:36:46.173 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:46.174 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:46.174 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:36:46.275 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:36:46.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:46.298 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:46.298 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:36:46.399 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:36:46.417 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.2 dump_ops_in_flight 2024-09-16T09:36:46.419 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:36:46.421 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:46.421 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:46.422 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:36:46.523 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:36:46.546 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:46.547 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:46.548 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:36:46.649 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:36:46.685 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:46.685 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:46.686 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:36:46.786 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:36:46.809 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:46.810 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:46.810 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:36:46.911 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:36:46.932 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:46.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:46.933 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:36:47.035 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:36:47.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:47.057 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:47.057 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:36:47.158 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:36:47.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:46 smithi084 ceph-mon[32629]: 9.4 scrub starts 2024-09-16T09:36:47.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:46 smithi084 ceph-mon[32629]: osdmap e325: 8 total, 8 up, 8 in 2024-09-16T09:36:47.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:46 smithi084 ceph-mon[32629]: pgmap v592: 176 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+recovering+undersized+remapped, 174 active+clean; 2.0 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 828 op/s; 199/101088 objects degraded (0.197%); 884/101088 objects misplaced (0.874%); 19 MiB/s, 297 objects/s recovering 2024-09-16T09:36:47.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:46 smithi084 ceph-mon[32629]: 9.4 scrub ok 2024-09-16T09:36:47.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:46 smithi084 ceph-mon[32629]: osdmap e326: 8 total, 8 up, 8 in 2024-09-16T09:36:47.170 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:46 smithi070 ceph-mon[30388]: 9.4 scrub starts 2024-09-16T09:36:47.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:46 smithi070 ceph-mon[30388]: osdmap e325: 8 total, 8 up, 8 in 2024-09-16T09:36:47.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:46 smithi070 ceph-mon[30388]: pgmap v592: 176 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+recovering+undersized+remapped, 174 active+clean; 2.0 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 828 op/s; 199/101088 objects degraded (0.197%); 884/101088 objects misplaced (0.874%); 19 MiB/s, 297 objects/s recovering 2024-09-16T09:36:47.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:46 smithi070 ceph-mon[30388]: 9.4 scrub ok 2024-09-16T09:36:47.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:46 smithi070 ceph-mon[30388]: osdmap e326: 8 total, 8 up, 8 in 2024-09-16T09:36:47.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:46 smithi070 ceph-mon[40691]: 9.4 scrub starts 2024-09-16T09:36:47.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:46 smithi070 ceph-mon[40691]: osdmap e325: 8 total, 8 up, 8 in 2024-09-16T09:36:47.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:46 smithi070 ceph-mon[40691]: pgmap v592: 176 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+recovering+undersized+remapped, 174 active+clean; 2.0 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 828 op/s; 199/101088 objects degraded (0.197%); 884/101088 objects misplaced (0.874%); 19 MiB/s, 297 objects/s recovering 2024-09-16T09:36:47.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:46 smithi070 ceph-mon[40691]: 9.4 scrub ok 2024-09-16T09:36:47.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:46 smithi070 ceph-mon[40691]: osdmap e326: 8 total, 8 up, 8 in 2024-09-16T09:36:47.180 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:47.181 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:47.181 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:36:47.282 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:36:47.305 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:47.305 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:47.306 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:36:47.406 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:36:47.430 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:47.431 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:47.431 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:36:47.532 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:36:47.554 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:47.555 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:47.555 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:36:47.656 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:36:47.677 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:47.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:47.678 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:36:47.779 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:36:47.809 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:47.810 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:47.810 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:36:47.911 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:36:47.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:47 smithi070 ceph-mon[30388]: 9.10 continuing backfill to osd.6 from (247'3284,324'8179] 9:089a27e8:::benchmark_data_smithi070_137000_object107484:head to 324'8179 2024-09-16T09:36:47.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:47 smithi070 ceph-mon[30388]: 3.5 scrub starts 2024-09-16T09:36:47.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:47 smithi070 ceph-mon[30388]: 3.5 scrub ok 2024-09-16T09:36:47.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:47 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/813279349' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:36:47.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:47 smithi070 ceph-mon[40691]: 9.10 continuing backfill to osd.6 from (247'3284,324'8179] 9:089a27e8:::benchmark_data_smithi070_137000_object107484:head to 324'8179 2024-09-16T09:36:47.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:47 smithi070 ceph-mon[40691]: 3.5 scrub starts 2024-09-16T09:36:47.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:47 smithi070 ceph-mon[40691]: 3.5 scrub ok 2024-09-16T09:36:47.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:47 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/813279349' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:36:47.935 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:47.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:47.937 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:36:48.037 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:36:48.060 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:48.061 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:48.061 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:36:48.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:47 smithi084 ceph-mon[32629]: 9.10 continuing backfill to osd.6 from (247'3284,324'8179] 9:089a27e8:::benchmark_data_smithi070_137000_object107484:head to 324'8179 2024-09-16T09:36:48.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:47 smithi084 ceph-mon[32629]: 3.5 scrub starts 2024-09-16T09:36:48.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:47 smithi084 ceph-mon[32629]: 3.5 scrub ok 2024-09-16T09:36:48.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:47 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/813279349' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:36:48.162 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:36:48.183 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:48.183 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:48.184 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:36:48.285 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:36:48.307 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:48.308 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:48.308 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:36:48.409 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:36:48.430 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:48.431 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:48.431 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:36:48.439 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:36:48.439 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:36:48.440 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:36:48.440 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:36:48.440 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:36:48.440 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:36:48.440 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:36:48.440 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:36:48.441 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:36:48.441 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:36:48.441 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:36:48.441 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:36:48.441 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:36:48.441 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:36:48.441 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:36:48.441 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:36:48.532 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:36:48.554 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:48.554 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:48.554 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:36:48.655 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:36:48.677 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:48.677 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:48.678 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:36:48.779 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:36:48.814 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:48.815 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:48.815 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:36:48.916 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:36:48.944 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:48.945 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:48.945 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:36:49.046 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:36:49.064 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:48 smithi070 ceph-mon[30388]: 4.f scrub starts 2024-09-16T09:36:49.064 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:48 smithi070 ceph-mon[30388]: 4.f scrub ok 2024-09-16T09:36:49.064 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:48 smithi070 ceph-mon[30388]: 6.3 scrub starts 2024-09-16T09:36:49.064 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:48 smithi070 ceph-mon[30388]: 6.3 scrub ok 2024-09-16T09:36:49.065 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:48 smithi070 ceph-mon[30388]: osdmap e327: 8 total, 8 up, 8 in 2024-09-16T09:36:49.065 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:48 smithi070 ceph-mon[30388]: pgmap v595: 175 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+recovering+undersized+remapped, 173 active+clean; 2.0 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.5 MiB/s wr, 275 op/s; 199/97407 objects degraded (0.204%); 884/97407 objects misplaced (0.908%); 49 MiB/s, 826 objects/s recovering 2024-09-16T09:36:49.065 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:48 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "22"} : dispatch 2024-09-16T09:36:49.065 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:48 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "22"} : dispatch 2024-09-16T09:36:49.065 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:48 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:36:49.065 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:48 smithi070 ceph-mon[40691]: 4.f scrub starts 2024-09-16T09:36:49.065 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:48 smithi070 ceph-mon[40691]: 4.f scrub ok 2024-09-16T09:36:49.065 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:48 smithi070 ceph-mon[40691]: 6.3 scrub starts 2024-09-16T09:36:49.065 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:48 smithi070 ceph-mon[40691]: 6.3 scrub ok 2024-09-16T09:36:49.065 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:48 smithi070 ceph-mon[40691]: osdmap e327: 8 total, 8 up, 8 in 2024-09-16T09:36:49.065 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:48 smithi070 ceph-mon[40691]: pgmap v595: 175 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+recovering+undersized+remapped, 173 active+clean; 2.0 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.5 MiB/s wr, 275 op/s; 199/97407 objects degraded (0.204%); 884/97407 objects misplaced (0.908%); 49 MiB/s, 826 objects/s recovering 2024-09-16T09:36:49.066 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:48 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "22"} : dispatch 2024-09-16T09:36:49.066 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:48 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "22"} : dispatch 2024-09-16T09:36:49.066 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:48 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:36:49.067 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:49.068 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:49.068 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:36:49.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:48 smithi084 ceph-mon[32629]: 4.f scrub starts 2024-09-16T09:36:49.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:48 smithi084 ceph-mon[32629]: 4.f scrub ok 2024-09-16T09:36:49.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:48 smithi084 ceph-mon[32629]: 6.3 scrub starts 2024-09-16T09:36:49.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:48 smithi084 ceph-mon[32629]: 6.3 scrub ok 2024-09-16T09:36:49.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:48 smithi084 ceph-mon[32629]: osdmap e327: 8 total, 8 up, 8 in 2024-09-16T09:36:49.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:48 smithi084 ceph-mon[32629]: pgmap v595: 175 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+recovering+undersized+remapped, 173 active+clean; 2.0 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 2.5 MiB/s wr, 275 op/s; 199/97407 objects degraded (0.204%); 884/97407 objects misplaced (0.908%); 49 MiB/s, 826 objects/s recovering 2024-09-16T09:36:49.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:48 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "22"} : dispatch 2024-09-16T09:36:49.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:48 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "22"} : dispatch 2024-09-16T09:36:49.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:48 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:36:49.168 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:36:49.190 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:49.190 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:49.191 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:36:49.292 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:36:49.316 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:49.316 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:49.317 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:36:49.346 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:36:49.348 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:36:49.418 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:36:49.439 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:49.440 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:49.440 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:36:49.541 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:36:49.563 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:49.564 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:49.564 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:36:49.665 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:36:49.686 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:49.687 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:49.687 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:36:49.788 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:36:49.811 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:49.812 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:49.812 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:36:49.913 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:36:49.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:49 smithi070 ceph-mon[30388]: 1.14 scrub starts 2024-09-16T09:36:49.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:49 smithi070 ceph-mon[30388]: 1.14 scrub ok 2024-09-16T09:36:49.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:49 smithi070 ceph-mon[30388]: 6.8 scrub starts 2024-09-16T09:36:49.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:49 smithi070 ceph-mon[30388]: 6.8 scrub ok 2024-09-16T09:36:49.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:49 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "22"}]': finished 2024-09-16T09:36:49.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:49 smithi070 ceph-mon[30388]: osdmap e328: 8 total, 8 up, 8 in 2024-09-16T09:36:49.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:49 smithi070 ceph-mon[30388]: Health check update: Degraded data redundancy: 199/97407 objects degraded (0.204%), 1 pg degraded, 2 pgs undersized (PG_DEGRADED) 2024-09-16T09:36:49.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:49 smithi070 ceph-mon[30388]: osdmap e329: 8 total, 8 up, 8 in 2024-09-16T09:36:49.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:49 smithi070 ceph-mon[40691]: 1.14 scrub starts 2024-09-16T09:36:49.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:49 smithi070 ceph-mon[40691]: 1.14 scrub ok 2024-09-16T09:36:49.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:49 smithi070 ceph-mon[40691]: 6.8 scrub starts 2024-09-16T09:36:49.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:49 smithi070 ceph-mon[40691]: 6.8 scrub ok 2024-09-16T09:36:49.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:49 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "22"}]': finished 2024-09-16T09:36:49.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:49 smithi070 ceph-mon[40691]: osdmap e328: 8 total, 8 up, 8 in 2024-09-16T09:36:49.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:49 smithi070 ceph-mon[40691]: Health check update: Degraded data redundancy: 199/97407 objects degraded (0.204%), 1 pg degraded, 2 pgs undersized (PG_DEGRADED) 2024-09-16T09:36:49.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:49 smithi070 ceph-mon[40691]: osdmap e329: 8 total, 8 up, 8 in 2024-09-16T09:36:49.935 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:49.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:49.936 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:36:50.037 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:36:50.058 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:50.059 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:50.059 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:36:50.160 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:36:50.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:49 smithi084 ceph-mon[32629]: 1.14 scrub starts 2024-09-16T09:36:50.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:49 smithi084 ceph-mon[32629]: 1.14 scrub ok 2024-09-16T09:36:50.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:49 smithi084 ceph-mon[32629]: 6.8 scrub starts 2024-09-16T09:36:50.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:49 smithi084 ceph-mon[32629]: 6.8 scrub ok 2024-09-16T09:36:50.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:49 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "22"}]': finished 2024-09-16T09:36:50.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:49 smithi084 ceph-mon[32629]: osdmap e328: 8 total, 8 up, 8 in 2024-09-16T09:36:50.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:49 smithi084 ceph-mon[32629]: Health check update: Degraded data redundancy: 199/97407 objects degraded (0.204%), 1 pg degraded, 2 pgs undersized (PG_DEGRADED) 2024-09-16T09:36:50.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:49 smithi084 ceph-mon[32629]: osdmap e329: 8 total, 8 up, 8 in 2024-09-16T09:36:50.182 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:50.182 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:50.183 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:36:50.283 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:36:50.305 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:50.306 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:50.306 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:36:50.407 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:36:50.421 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:36:50.428 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:50.429 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:50.429 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:36:50.530 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:36:50.552 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:50.552 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:50.556 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:36:50.657 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:36:50.678 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:50.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:50.678 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:36:50.779 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:36:50.802 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:50.802 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:50.803 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:36:50.904 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:36:50.920 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:50 smithi070 ceph-mon[40691]: 4.4 scrub starts 2024-09-16T09:36:50.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:50 smithi070 ceph-mon[40691]: 4.4 scrub ok 2024-09-16T09:36:50.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:50 smithi070 ceph-mon[40691]: 6.17 scrub starts 2024-09-16T09:36:50.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:50 smithi070 ceph-mon[40691]: 6.17 scrub ok 2024-09-16T09:36:50.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:50 smithi070 ceph-mon[40691]: pgmap v598: 175 pgs: 1 creating+peering, 1 active+remapped+backfilling, 173 active+clean; 1.9 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 406 op/s; 67/96225 objects misplaced (0.070%); 17 MiB/s, 358 objects/s recovering 2024-09-16T09:36:50.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:50 smithi070 ceph-mon[40691]: 9.10 scrub starts 2024-09-16T09:36:50.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:50 smithi070 ceph-mon[40691]: 9.10 scrub ok 2024-09-16T09:36:50.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:50 smithi070 ceph-mon[40691]: 3.2 scrub starts 2024-09-16T09:36:50.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:50 smithi070 ceph-mon[40691]: 3.2 scrub ok 2024-09-16T09:36:50.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:50 smithi070 ceph-mon[40691]: osdmap e330: 8 total, 8 up, 8 in 2024-09-16T09:36:50.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:50 smithi070 ceph-mon[30388]: 4.4 scrub starts 2024-09-16T09:36:50.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:50 smithi070 ceph-mon[30388]: 4.4 scrub ok 2024-09-16T09:36:50.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:50 smithi070 ceph-mon[30388]: 6.17 scrub starts 2024-09-16T09:36:50.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:50 smithi070 ceph-mon[30388]: 6.17 scrub ok 2024-09-16T09:36:50.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:50 smithi070 ceph-mon[30388]: pgmap v598: 175 pgs: 1 creating+peering, 1 active+remapped+backfilling, 173 active+clean; 1.9 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 406 op/s; 67/96225 objects misplaced (0.070%); 17 MiB/s, 358 objects/s recovering 2024-09-16T09:36:50.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:50 smithi070 ceph-mon[30388]: 9.10 scrub starts 2024-09-16T09:36:50.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:50 smithi070 ceph-mon[30388]: 9.10 scrub ok 2024-09-16T09:36:50.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:50 smithi070 ceph-mon[30388]: 3.2 scrub starts 2024-09-16T09:36:50.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:50 smithi070 ceph-mon[30388]: 3.2 scrub ok 2024-09-16T09:36:50.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:50 smithi070 ceph-mon[30388]: osdmap e330: 8 total, 8 up, 8 in 2024-09-16T09:36:50.926 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:50.926 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:50.927 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:36:51.028 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:36:51.048 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:51.049 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:51.049 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:36:51.061 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:50 smithi084 ceph-mon[32629]: 4.4 scrub starts 2024-09-16T09:36:51.061 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:50 smithi084 ceph-mon[32629]: 4.4 scrub ok 2024-09-16T09:36:51.062 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:50 smithi084 ceph-mon[32629]: 6.17 scrub starts 2024-09-16T09:36:51.062 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:50 smithi084 ceph-mon[32629]: 6.17 scrub ok 2024-09-16T09:36:51.062 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:50 smithi084 ceph-mon[32629]: pgmap v598: 175 pgs: 1 creating+peering, 1 active+remapped+backfilling, 173 active+clean; 1.9 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 406 op/s; 67/96225 objects misplaced (0.070%); 17 MiB/s, 358 objects/s recovering 2024-09-16T09:36:51.062 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:50 smithi084 ceph-mon[32629]: 9.10 scrub starts 2024-09-16T09:36:51.062 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:50 smithi084 ceph-mon[32629]: 9.10 scrub ok 2024-09-16T09:36:51.062 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:50 smithi084 ceph-mon[32629]: 3.2 scrub starts 2024-09-16T09:36:51.062 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:50 smithi084 ceph-mon[32629]: 3.2 scrub ok 2024-09-16T09:36:51.062 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:50 smithi084 ceph-mon[32629]: osdmap e330: 8 total, 8 up, 8 in 2024-09-16T09:36:51.150 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:36:51.170 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:51.171 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:51.172 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:36:51.272 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:36:51.294 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:51.294 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:51.295 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:36:51.396 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:36:51.431 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:51.432 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:51.432 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:36:51.456 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:36:51.458 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T09:36:51.458 INFO:teuthology.orchestra.run.smithi070.stdout: "epoch": 331, 2024-09-16T09:36:51.458 INFO:teuthology.orchestra.run.smithi070.stdout: "fsid": "a1321834-740c-11ef-bceb-c7b262605968", 2024-09-16T09:36:51.458 INFO:teuthology.orchestra.run.smithi070.stdout: "created": "2024-09-16T09:19:14.121498+0000", 2024-09-16T09:36:51.458 INFO:teuthology.orchestra.run.smithi070.stdout: "modified": "2024-09-16T09:36:51.309118+0000", 2024-09-16T09:36:51.458 INFO:teuthology.orchestra.run.smithi070.stdout: "last_up_change": "2024-09-16T09:24:51.124496+0000", 2024-09-16T09:36:51.458 INFO:teuthology.orchestra.run.smithi070.stdout: "last_in_change": "2024-09-16T09:24:28.432920+0000", 2024-09-16T09:36:51.458 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-16T09:36:51.459 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_num": 5804032, 2024-09-16T09:36:51.459 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_set": [ 2024-09-16T09:36:51.459 INFO:teuthology.orchestra.run.smithi070.stdout: "nodeep-scrub", 2024-09-16T09:36:51.459 INFO:teuthology.orchestra.run.smithi070.stdout: "pglog_hardlimit", 2024-09-16T09:36:51.459 INFO:teuthology.orchestra.run.smithi070.stdout: "purged_snapdirs", 2024-09-16T09:36:51.459 INFO:teuthology.orchestra.run.smithi070.stdout: "recovery_deletes", 2024-09-16T09:36:51.459 INFO:teuthology.orchestra.run.smithi070.stdout: "sortbitwise" 2024-09-16T09:36:51.459 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-16T09:36:51.459 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_version": 18, 2024-09-16T09:36:51.459 INFO:teuthology.orchestra.run.smithi070.stdout: "full_ratio": 0.94999998807907104, 2024-09-16T09:36:51.459 INFO:teuthology.orchestra.run.smithi070.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-16T09:36:51.459 INFO:teuthology.orchestra.run.smithi070.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-16T09:36:51.459 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_snapshot": "", 2024-09-16T09:36:51.460 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_max": 9, 2024-09-16T09:36:51.460 INFO:teuthology.orchestra.run.smithi070.stdout: "max_osd": 8, 2024-09-16T09:36:51.460 INFO:teuthology.orchestra.run.smithi070.stdout: "require_min_compat_client": "luminous", 2024-09-16T09:36:51.460 INFO:teuthology.orchestra.run.smithi070.stdout: "min_compat_client": "luminous", 2024-09-16T09:36:51.460 INFO:teuthology.orchestra.run.smithi070.stdout: "require_osd_release": "squid", 2024-09-16T09:36:51.460 INFO:teuthology.orchestra.run.smithi070.stdout: "allow_crimson": false, 2024-09-16T09:36:51.460 INFO:teuthology.orchestra.run.smithi070.stdout: "pools": [ 2024-09-16T09:36:51.460 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:36:51.460 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 1, 2024-09-16T09:36:51.460 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": ".mgr", 2024-09-16T09:36:51.460 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2024-09-16T09:22:27.617694+0000", 2024-09-16T09:36:51.460 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2024-09-16T09:36:51.460 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2024-09-16T09:36:51.461 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2024-09-16T09:36:51.461 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 3, 2024-09-16T09:36:51.461 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 2, 2024-09-16T09:36:51.461 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2024-09-16T09:36:51.461 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2024-09-16T09:36:51.461 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2024-09-16T09:36:51.461 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T09:36:51.461 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T09:36:51.461 INFO:teuthology.orchestra.run.smithi070.stdout: "is_stretch_pool": false, 2024-09-16T09:36:51.461 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2024-09-16T09:36:51.461 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2024-09-16T09:36:51.461 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 21, 2024-09-16T09:36:51.462 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 21, 2024-09-16T09:36:51.462 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 21, 2024-09-16T09:36:51.462 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 21, 2024-09-16T09:36:51.462 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 21, 2024-09-16T09:36:51.462 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2024-09-16T09:36:51.462 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2024-09-16T09:36:51.462 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2024-09-16T09:36:51.462 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2024-09-16T09:36:51.462 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2024-09-16T09:36:51.462 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2024-09-16T09:36:51.462 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2024-09-16T09:36:51.462 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.462 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "162", 2024-09-16T09:36:51.463 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2024-09-16T09:36:51.463 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T09:36:51.463 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "160", 2024-09-16T09:36:51.463 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2024-09-16T09:36:51.463 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2024-09-16T09:36:51.463 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2024-09-16T09:36:51.463 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2024-09-16T09:36:51.463 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2024-09-16T09:36:51.463 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2024-09-16T09:36:51.463 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2024-09-16T09:36:51.463 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2024-09-16T09:36:51.463 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2024-09-16T09:36:51.463 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2024-09-16T09:36:51.464 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2024-09-16T09:36:51.464 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2024-09-16T09:36:51.464 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2024-09-16T09:36:51.464 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2024-09-16T09:36:51.464 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2024-09-16T09:36:51.464 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T09:36:51.464 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T09:36:51.464 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T09:36:51.464 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2024-09-16T09:36:51.464 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2024-09-16T09:36:51.464 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2024-09-16T09:36:51.464 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2024-09-16T09:36:51.465 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2024-09-16T09:36:51.465 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.465 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2024-09-16T09:36:51.465 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2024-09-16T09:36:51.465 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2024-09-16T09:36:51.465 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2024-09-16T09:36:51.465 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2024-09-16T09:36:51.465 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T09:36:51.465 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2024-09-16T09:36:51.465 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2024-09-16T09:36:51.465 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2024-09-16T09:36:51.465 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2024-09-16T09:36:51.465 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2024-09-16T09:36:51.466 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2024-09-16T09:36:51.466 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_max": 32, 2024-09-16T09:36:51.466 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_min": 1 2024-09-16T09:36:51.466 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.466 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2024-09-16T09:36:51.466 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": {} 2024-09-16T09:36:51.466 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.466 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2024-09-16T09:36:51.466 INFO:teuthology.orchestra.run.smithi070.stdout: "score_type": "Fair distribution", 2024-09-16T09:36:51.466 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 1.6699999570846558, 2024-09-16T09:36:51.466 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 1.6699999570846558, 2024-09-16T09:36:51.466 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1.1399999856948853, 2024-09-16T09:36:51.466 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 1.8999999761581421, 2024-09-16T09:36:51.467 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 1.8999999761581421, 2024-09-16T09:36:51.467 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 0.70719337463378906, 2024-09-16T09:36:51.467 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 0.81000000238418579, 2024-09-16T09:36:51.467 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 0.70999997854232788 2024-09-16T09:36:51.467 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:36:51.467 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.467 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:36:51.467 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 2, 2024-09-16T09:36:51.467 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "datapool", 2024-09-16T09:36:51.467 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2024-09-16T09:24:59.169244+0000", 2024-09-16T09:36:51.467 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 8193, 2024-09-16T09:36:51.467 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-16T09:36:51.467 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2024-09-16T09:36:51.468 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 3, 2024-09-16T09:36:51.468 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 2, 2024-09-16T09:36:51.468 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2024-09-16T09:36:51.468 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2024-09-16T09:36:51.468 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2024-09-16T09:36:51.468 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T09:36:51.468 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T09:36:51.468 INFO:teuthology.orchestra.run.smithi070.stdout: "is_stretch_pool": false, 2024-09-16T09:36:51.468 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2024-09-16T09:36:51.468 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2024-09-16T09:36:51.468 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 3, 2024-09-16T09:36:51.468 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 3, 2024-09-16T09:36:51.468 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 3, 2024-09-16T09:36:51.469 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 3, 2024-09-16T09:36:51.469 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 3, 2024-09-16T09:36:51.469 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2024-09-16T09:36:51.469 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2024-09-16T09:36:51.469 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2024-09-16T09:36:51.469 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2024-09-16T09:36:51.469 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2024-09-16T09:36:51.469 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2024-09-16T09:36:51.469 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2024-09-16T09:36:51.469 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.469 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "61", 2024-09-16T09:36:51.469 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2024-09-16T09:36:51.469 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T09:36:51.470 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T09:36:51.470 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2024-09-16T09:36:51.470 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2024-09-16T09:36:51.470 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 2, 2024-09-16T09:36:51.470 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 61, 2024-09-16T09:36:51.470 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2024-09-16T09:36:51.470 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2024-09-16T09:36:51.470 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2024-09-16T09:36:51.470 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2024-09-16T09:36:51.470 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2024-09-16T09:36:51.470 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2024-09-16T09:36:51.470 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2024-09-16T09:36:51.470 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2024-09-16T09:36:51.470 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2024-09-16T09:36:51.470 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2024-09-16T09:36:51.471 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2024-09-16T09:36:51.471 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T09:36:51.471 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T09:36:51.471 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T09:36:51.471 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2024-09-16T09:36:51.471 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2024-09-16T09:36:51.471 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2024-09-16T09:36:51.471 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2024-09-16T09:36:51.471 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2024-09-16T09:36:51.471 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.471 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2024-09-16T09:36:51.471 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2024-09-16T09:36:51.471 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2024-09-16T09:36:51.471 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2024-09-16T09:36:51.472 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2024-09-16T09:36:51.472 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T09:36:51.472 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2024-09-16T09:36:51.472 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2024-09-16T09:36:51.472 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2024-09-16T09:36:51.472 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2024-09-16T09:36:51.472 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2024-09-16T09:36:51.472 INFO:teuthology.orchestra.run.smithi070.stdout: "options": {}, 2024-09-16T09:36:51.472 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2024-09-16T09:36:51.472 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd": {} 2024-09-16T09:36:51.472 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.472 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2024-09-16T09:36:51.472 INFO:teuthology.orchestra.run.smithi070.stdout: "score_type": "Fair distribution", 2024-09-16T09:36:51.473 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 2.3299999237060547, 2024-09-16T09:36:51.473 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 2.3299999237060547, 2024-09-16T09:36:51.473 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1, 2024-09-16T09:36:51.473 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 2.3299999237060547, 2024-09-16T09:36:51.473 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 2.3299999237060547, 2024-09-16T09:36:51.473 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 0.70719337463378906, 2024-09-16T09:36:51.473 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 0.81000000238418579, 2024-09-16T09:36:51.473 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 0.70999997854232788 2024-09-16T09:36:51.473 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:36:51.473 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.473 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:36:51.473 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 3, 2024-09-16T09:36:51.473 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": ".rgw.root", 2024-09-16T09:36:51.474 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2024-09-16T09:25:00.056119+0000", 2024-09-16T09:36:51.474 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2024-09-16T09:36:51.474 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2024-09-16T09:36:51.474 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2024-09-16T09:36:51.474 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 3, 2024-09-16T09:36:51.474 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 2, 2024-09-16T09:36:51.474 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2024-09-16T09:36:51.474 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2024-09-16T09:36:51.474 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2024-09-16T09:36:51.474 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T09:36:51.474 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T09:36:51.474 INFO:teuthology.orchestra.run.smithi070.stdout: "is_stretch_pool": false, 2024-09-16T09:36:51.474 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2024-09-16T09:36:51.474 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2024-09-16T09:36:51.475 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 32, 2024-09-16T09:36:51.475 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 32, 2024-09-16T09:36:51.475 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 32, 2024-09-16T09:36:51.475 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 32, 2024-09-16T09:36:51.475 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 32, 2024-09-16T09:36:51.475 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2024-09-16T09:36:51.475 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2024-09-16T09:36:51.476 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2024-09-16T09:36:51.476 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2024-09-16T09:36:51.476 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2024-09-16T09:36:51.476 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2024-09-16T09:36:51.476 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2024-09-16T09:36:51.476 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.476 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "60", 2024-09-16T09:36:51.476 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2024-09-16T09:36:51.476 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T09:36:51.476 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T09:36:51.476 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2024-09-16T09:36:51.477 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2024-09-16T09:36:51.477 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2024-09-16T09:36:51.477 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2024-09-16T09:36:51.477 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2024-09-16T09:36:51.477 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2024-09-16T09:36:51.477 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2024-09-16T09:36:51.477 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2024-09-16T09:36:51.477 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2024-09-16T09:36:51.477 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2024-09-16T09:36:51.477 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2024-09-16T09:36:51.477 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2024-09-16T09:36:51.477 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2024-09-16T09:36:51.477 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2024-09-16T09:36:51.478 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2024-09-16T09:36:51.478 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T09:36:51.478 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T09:36:51.478 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T09:36:51.478 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2024-09-16T09:36:51.478 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2024-09-16T09:36:51.478 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2024-09-16T09:36:51.478 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2024-09-16T09:36:51.478 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2024-09-16T09:36:51.478 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.478 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2024-09-16T09:36:51.478 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2024-09-16T09:36:51.478 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2024-09-16T09:36:51.479 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2024-09-16T09:36:51.479 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2024-09-16T09:36:51.479 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T09:36:51.479 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2024-09-16T09:36:51.479 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2024-09-16T09:36:51.479 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2024-09-16T09:36:51.479 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2024-09-16T09:36:51.479 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2024-09-16T09:36:51.479 INFO:teuthology.orchestra.run.smithi070.stdout: "options": {}, 2024-09-16T09:36:51.479 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2024-09-16T09:36:51.479 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw": {} 2024-09-16T09:36:51.479 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.479 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2024-09-16T09:36:51.480 INFO:teuthology.orchestra.run.smithi070.stdout: "score_type": "Fair distribution", 2024-09-16T09:36:51.480 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 1.75, 2024-09-16T09:36:51.480 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 1.75, 2024-09-16T09:36:51.480 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1.1399999856948853, 2024-09-16T09:36:51.480 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 2, 2024-09-16T09:36:51.480 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 2, 2024-09-16T09:36:51.480 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 0.70719337463378906, 2024-09-16T09:36:51.480 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 0.81000000238418579, 2024-09-16T09:36:51.480 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 0.70999997854232788 2024-09-16T09:36:51.480 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:36:51.480 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.480 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:36:51.480 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 4, 2024-09-16T09:36:51.481 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "default.rgw.log", 2024-09-16T09:36:51.481 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2024-09-16T09:25:01.770016+0000", 2024-09-16T09:36:51.481 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2024-09-16T09:36:51.481 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2024-09-16T09:36:51.481 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2024-09-16T09:36:51.481 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 3, 2024-09-16T09:36:51.481 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 2, 2024-09-16T09:36:51.481 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2024-09-16T09:36:51.481 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2024-09-16T09:36:51.481 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2024-09-16T09:36:51.481 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T09:36:51.481 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T09:36:51.481 INFO:teuthology.orchestra.run.smithi070.stdout: "is_stretch_pool": false, 2024-09-16T09:36:51.482 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2024-09-16T09:36:51.482 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2024-09-16T09:36:51.482 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 32, 2024-09-16T09:36:51.482 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 32, 2024-09-16T09:36:51.482 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 32, 2024-09-16T09:36:51.482 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 32, 2024-09-16T09:36:51.482 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 32, 2024-09-16T09:36:51.482 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2024-09-16T09:36:51.482 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2024-09-16T09:36:51.482 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2024-09-16T09:36:51.482 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2024-09-16T09:36:51.482 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2024-09-16T09:36:51.482 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2024-09-16T09:36:51.483 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2024-09-16T09:36:51.483 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.483 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "62", 2024-09-16T09:36:51.483 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2024-09-16T09:36:51.483 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T09:36:51.483 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T09:36:51.483 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2024-09-16T09:36:51.483 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2024-09-16T09:36:51.483 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2024-09-16T09:36:51.483 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2024-09-16T09:36:51.483 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2024-09-16T09:36:51.483 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2024-09-16T09:36:51.483 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2024-09-16T09:36:51.484 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2024-09-16T09:36:51.484 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2024-09-16T09:36:51.484 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2024-09-16T09:36:51.484 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2024-09-16T09:36:51.484 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2024-09-16T09:36:51.484 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2024-09-16T09:36:51.484 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2024-09-16T09:36:51.484 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2024-09-16T09:36:51.484 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T09:36:51.484 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T09:36:51.484 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T09:36:51.484 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2024-09-16T09:36:51.484 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2024-09-16T09:36:51.485 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2024-09-16T09:36:51.485 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2024-09-16T09:36:51.485 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2024-09-16T09:36:51.485 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.485 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2024-09-16T09:36:51.485 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2024-09-16T09:36:51.485 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2024-09-16T09:36:51.485 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2024-09-16T09:36:51.485 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2024-09-16T09:36:51.485 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T09:36:51.485 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2024-09-16T09:36:51.485 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2024-09-16T09:36:51.486 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2024-09-16T09:36:51.486 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2024-09-16T09:36:51.486 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2024-09-16T09:36:51.486 INFO:teuthology.orchestra.run.smithi070.stdout: "options": {}, 2024-09-16T09:36:51.486 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2024-09-16T09:36:51.486 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw": {} 2024-09-16T09:36:51.486 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.486 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2024-09-16T09:36:51.486 INFO:teuthology.orchestra.run.smithi070.stdout: "score_type": "Fair distribution", 2024-09-16T09:36:51.486 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 6.8499999046325684, 2024-09-16T09:36:51.486 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 2.190000057220459, 2024-09-16T09:36:51.486 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1.1399999856948853, 2024-09-16T09:36:51.486 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 7.809999942779541, 2024-09-16T09:36:51.486 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 2.5, 2024-09-16T09:36:51.487 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 0.70719337463378906, 2024-09-16T09:36:51.487 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 0.81000000238418579, 2024-09-16T09:36:51.487 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 0.70999997854232788, 2024-09-16T09:36:51.487 INFO:teuthology.orchestra.run.smithi070.stdout: "error_message": "pool 4 has acting primaries on OSD(s) with primary affinity 0, read balance score is not accurate" 2024-09-16T09:36:51.487 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:36:51.487 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.487 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:36:51.487 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 5, 2024-09-16T09:36:51.487 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "default.rgw.control", 2024-09-16T09:36:51.487 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2024-09-16T09:25:03.725555+0000", 2024-09-16T09:36:51.487 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2024-09-16T09:36:51.487 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2024-09-16T09:36:51.487 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2024-09-16T09:36:51.488 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 3, 2024-09-16T09:36:51.488 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 2, 2024-09-16T09:36:51.488 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2024-09-16T09:36:51.488 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2024-09-16T09:36:51.488 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2024-09-16T09:36:51.488 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T09:36:51.488 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T09:36:51.488 INFO:teuthology.orchestra.run.smithi070.stdout: "is_stretch_pool": false, 2024-09-16T09:36:51.488 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2024-09-16T09:36:51.488 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2024-09-16T09:36:51.488 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 32, 2024-09-16T09:36:51.488 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 32, 2024-09-16T09:36:51.488 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 32, 2024-09-16T09:36:51.488 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 32, 2024-09-16T09:36:51.488 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 32, 2024-09-16T09:36:51.489 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2024-09-16T09:36:51.489 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2024-09-16T09:36:51.489 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2024-09-16T09:36:51.489 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2024-09-16T09:36:51.489 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2024-09-16T09:36:51.489 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2024-09-16T09:36:51.489 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2024-09-16T09:36:51.489 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.489 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "64", 2024-09-16T09:36:51.489 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2024-09-16T09:36:51.489 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T09:36:51.489 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T09:36:51.489 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2024-09-16T09:36:51.489 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2024-09-16T09:36:51.490 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2024-09-16T09:36:51.490 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2024-09-16T09:36:51.490 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2024-09-16T09:36:51.490 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2024-09-16T09:36:51.490 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2024-09-16T09:36:51.490 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2024-09-16T09:36:51.490 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2024-09-16T09:36:51.490 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2024-09-16T09:36:51.490 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2024-09-16T09:36:51.490 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2024-09-16T09:36:51.490 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2024-09-16T09:36:51.490 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2024-09-16T09:36:51.490 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2024-09-16T09:36:51.490 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T09:36:51.490 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T09:36:51.491 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T09:36:51.491 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2024-09-16T09:36:51.491 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2024-09-16T09:36:51.491 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2024-09-16T09:36:51.491 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2024-09-16T09:36:51.491 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2024-09-16T09:36:51.491 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.491 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2024-09-16T09:36:51.491 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2024-09-16T09:36:51.491 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2024-09-16T09:36:51.491 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2024-09-16T09:36:51.491 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2024-09-16T09:36:51.491 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T09:36:51.492 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2024-09-16T09:36:51.492 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2024-09-16T09:36:51.492 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2024-09-16T09:36:51.492 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2024-09-16T09:36:51.492 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2024-09-16T09:36:51.492 INFO:teuthology.orchestra.run.smithi070.stdout: "options": {}, 2024-09-16T09:36:51.492 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2024-09-16T09:36:51.492 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw": {} 2024-09-16T09:36:51.492 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.492 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2024-09-16T09:36:51.492 INFO:teuthology.orchestra.run.smithi070.stdout: "score_type": "Fair distribution", 2024-09-16T09:36:51.492 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 1.75, 2024-09-16T09:36:51.492 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 1.75, 2024-09-16T09:36:51.493 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1.1399999856948853, 2024-09-16T09:36:51.493 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 2, 2024-09-16T09:36:51.493 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 2, 2024-09-16T09:36:51.493 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 0.70719337463378906, 2024-09-16T09:36:51.493 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 0.81000000238418579, 2024-09-16T09:36:51.493 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 0.70999997854232788 2024-09-16T09:36:51.493 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:36:51.493 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.493 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:36:51.493 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 6, 2024-09-16T09:36:51.493 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "default.rgw.meta", 2024-09-16T09:36:51.493 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2024-09-16T09:25:05.773530+0000", 2024-09-16T09:36:51.493 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2024-09-16T09:36:51.494 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2024-09-16T09:36:51.494 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2024-09-16T09:36:51.494 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 3, 2024-09-16T09:36:51.494 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 2, 2024-09-16T09:36:51.494 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2024-09-16T09:36:51.494 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2024-09-16T09:36:51.494 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2024-09-16T09:36:51.494 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T09:36:51.494 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T09:36:51.494 INFO:teuthology.orchestra.run.smithi070.stdout: "is_stretch_pool": false, 2024-09-16T09:36:51.494 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2024-09-16T09:36:51.494 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2024-09-16T09:36:51.494 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 32, 2024-09-16T09:36:51.495 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 32, 2024-09-16T09:36:51.495 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 32, 2024-09-16T09:36:51.495 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 32, 2024-09-16T09:36:51.495 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 32, 2024-09-16T09:36:51.495 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2024-09-16T09:36:51.495 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2024-09-16T09:36:51.495 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2024-09-16T09:36:51.495 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2024-09-16T09:36:51.495 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2024-09-16T09:36:51.495 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2024-09-16T09:36:51.495 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2024-09-16T09:36:51.495 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.496 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "66", 2024-09-16T09:36:51.496 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2024-09-16T09:36:51.496 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T09:36:51.496 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T09:36:51.496 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2024-09-16T09:36:51.496 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2024-09-16T09:36:51.496 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2024-09-16T09:36:51.496 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2024-09-16T09:36:51.496 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2024-09-16T09:36:51.496 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2024-09-16T09:36:51.496 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2024-09-16T09:36:51.496 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2024-09-16T09:36:51.496 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2024-09-16T09:36:51.496 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2024-09-16T09:36:51.497 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2024-09-16T09:36:51.497 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2024-09-16T09:36:51.497 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2024-09-16T09:36:51.497 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2024-09-16T09:36:51.497 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2024-09-16T09:36:51.497 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T09:36:51.497 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T09:36:51.497 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T09:36:51.497 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2024-09-16T09:36:51.497 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2024-09-16T09:36:51.497 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2024-09-16T09:36:51.497 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2024-09-16T09:36:51.497 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2024-09-16T09:36:51.498 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.498 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2024-09-16T09:36:51.498 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2024-09-16T09:36:51.498 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2024-09-16T09:36:51.498 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2024-09-16T09:36:51.498 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2024-09-16T09:36:51.498 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T09:36:51.498 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2024-09-16T09:36:51.498 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2024-09-16T09:36:51.498 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2024-09-16T09:36:51.498 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2024-09-16T09:36:51.498 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2024-09-16T09:36:51.498 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2024-09-16T09:36:51.498 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_bias": 4 2024-09-16T09:36:51.499 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.499 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2024-09-16T09:36:51.499 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw": {} 2024-09-16T09:36:51.499 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.499 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2024-09-16T09:36:51.499 INFO:teuthology.orchestra.run.smithi070.stdout: "score_type": "Fair distribution", 2024-09-16T09:36:51.499 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 1.5399999618530273, 2024-09-16T09:36:51.499 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 1.5399999618530273, 2024-09-16T09:36:51.499 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1.1399999856948853, 2024-09-16T09:36:51.499 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 1.75, 2024-09-16T09:36:51.499 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 1.75, 2024-09-16T09:36:51.499 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 0.70719337463378906, 2024-09-16T09:36:51.499 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 0.81000000238418579, 2024-09-16T09:36:51.499 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 0.70999997854232788 2024-09-16T09:36:51.500 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:36:51.500 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.500 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:36:51.500 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 9, 2024-09-16T09:36:51.500 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "unique_pool_2", 2024-09-16T09:36:51.500 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2024-09-16T09:33:23.906323+0000", 2024-09-16T09:36:51.500 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2024-09-16T09:36:51.500 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2024-09-16T09:36:51.500 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2024-09-16T09:36:51.500 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 3, 2024-09-16T09:36:51.500 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 2, 2024-09-16T09:36:51.500 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2024-09-16T09:36:51.500 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2024-09-16T09:36:51.501 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2024-09-16T09:36:51.501 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T09:36:51.501 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T09:36:51.501 INFO:teuthology.orchestra.run.smithi070.stdout: "is_stretch_pool": false, 2024-09-16T09:36:51.501 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2024-09-16T09:36:51.501 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2024-09-16T09:36:51.501 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 22, 2024-09-16T09:36:51.501 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 16, 2024-09-16T09:36:51.501 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 16, 2024-09-16T09:36:51.501 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 16, 2024-09-16T09:36:51.501 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 22, 2024-09-16T09:36:51.501 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2024-09-16T09:36:51.501 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "9.16", 2024-09-16T09:36:51.501 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 330, 2024-09-16T09:36:51.502 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 329, 2024-09-16T09:36:51.502 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 329, 2024-09-16T09:36:51.502 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "327'8255", 2024-09-16T09:36:51.502 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "327'8239" 2024-09-16T09:36:51.502 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.502 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "330", 2024-09-16T09:36:51.502 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2024-09-16T09:36:51.502 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "330", 2024-09-16T09:36:51.502 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "328", 2024-09-16T09:36:51.502 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2024-09-16T09:36:51.502 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2024-09-16T09:36:51.502 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2024-09-16T09:36:51.502 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2024-09-16T09:36:51.503 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2024-09-16T09:36:51.503 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2024-09-16T09:36:51.503 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2024-09-16T09:36:51.503 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2024-09-16T09:36:51.503 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2024-09-16T09:36:51.503 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2024-09-16T09:36:51.503 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2024-09-16T09:36:51.503 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2024-09-16T09:36:51.503 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2024-09-16T09:36:51.503 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2024-09-16T09:36:51.503 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2024-09-16T09:36:51.503 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T09:36:51.503 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T09:36:51.504 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T09:36:51.504 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2024-09-16T09:36:51.504 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2024-09-16T09:36:51.504 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2024-09-16T09:36:51.504 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2024-09-16T09:36:51.504 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2024-09-16T09:36:51.504 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.504 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2024-09-16T09:36:51.504 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2024-09-16T09:36:51.504 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2024-09-16T09:36:51.504 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2024-09-16T09:36:51.504 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2024-09-16T09:36:51.504 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T09:36:51.505 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2024-09-16T09:36:51.505 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2024-09-16T09:36:51.505 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2024-09-16T09:36:51.505 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2024-09-16T09:36:51.505 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2024-09-16T09:36:51.505 INFO:teuthology.orchestra.run.smithi070.stdout: "options": {}, 2024-09-16T09:36:51.505 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2024-09-16T09:36:51.505 INFO:teuthology.orchestra.run.smithi070.stdout: "rados": {} 2024-09-16T09:36:51.505 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.505 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2024-09-16T09:36:51.505 INFO:teuthology.orchestra.run.smithi070.stdout: "score_type": "Fair distribution", 2024-09-16T09:36:51.505 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 1.6000000238418579, 2024-09-16T09:36:51.505 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 1.6000000238418579, 2024-09-16T09:36:51.506 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1.1399999856948853, 2024-09-16T09:36:51.506 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 1.8200000524520874, 2024-09-16T09:36:51.506 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 1.8200000524520874, 2024-09-16T09:36:51.506 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 0.70719337463378906, 2024-09-16T09:36:51.506 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 0.81000000238418579, 2024-09-16T09:36:51.506 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 0.70999997854232788 2024-09-16T09:36:51.506 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:36:51.506 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:36:51.506 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-16T09:36:51.506 INFO:teuthology.orchestra.run.smithi070.stdout: "osds": [ 2024-09-16T09:36:51.506 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:36:51.506 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 0, 2024-09-16T09:36:51.506 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "7ef288c8-3bf0-4814-8e86-f70a7de041ce", 2024-09-16T09:36:51.506 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2024-09-16T09:36:51.507 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2024-09-16T09:36:51.507 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2024-09-16T09:36:51.507 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2024-09-16T09:36:51.507 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2024-09-16T09:36:51.507 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2024-09-16T09:36:51.507 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 8, 2024-09-16T09:36:51.507 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 326, 2024-09-16T09:36:51.507 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2024-09-16T09:36:51.507 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2024-09-16T09:36:51.507 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2024-09-16T09:36:51.507 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:36:51.507 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:36:51.507 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:36:51.507 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6802", 2024-09-16T09:36:51.507 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2013075781 2024-09-16T09:36:51.507 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.508 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:36:51.508 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:36:51.508 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6803", 2024-09-16T09:36:51.508 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2013075781 2024-09-16T09:36:51.508 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:36:51.508 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:36:51.508 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.508 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2024-09-16T09:36:51.508 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:36:51.508 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:36:51.508 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:36:51.508 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6804", 2024-09-16T09:36:51.508 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2013075781 2024-09-16T09:36:51.509 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.509 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:36:51.509 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:36:51.509 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6805", 2024-09-16T09:36:51.509 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2013075781 2024-09-16T09:36:51.509 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:36:51.510 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:36:51.510 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.510 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2024-09-16T09:36:51.510 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:36:51.510 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:36:51.510 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:36:51.510 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6808", 2024-09-16T09:36:51.510 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2013075781 2024-09-16T09:36:51.510 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.510 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:36:51.511 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:36:51.511 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6809", 2024-09-16T09:36:51.511 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2013075781 2024-09-16T09:36:51.511 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:36:51.511 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:36:51.511 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.511 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2024-09-16T09:36:51.511 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:36:51.511 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:36:51.511 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:36:51.511 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6806", 2024-09-16T09:36:51.511 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2013075781 2024-09-16T09:36:51.511 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.511 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:36:51.511 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:36:51.512 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6807", 2024-09-16T09:36:51.512 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2013075781 2024-09-16T09:36:51.512 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:36:51.512 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:36:51.512 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.512 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6803/2013075781", 2024-09-16T09:36:51.512 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6805/2013075781", 2024-09-16T09:36:51.512 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6809/2013075781", 2024-09-16T09:36:51.512 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6807/2013075781", 2024-09-16T09:36:51.512 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2024-09-16T09:36:51.512 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2024-09-16T09:36:51.512 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2024-09-16T09:36:51.512 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:36:51.513 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.513 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:36:51.513 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 1, 2024-09-16T09:36:51.513 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "474d41d9-295a-450c-8f27-ce487a14678b", 2024-09-16T09:36:51.513 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2024-09-16T09:36:51.513 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2024-09-16T09:36:51.513 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 0.95001220703125, 2024-09-16T09:36:51.513 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 0, 2024-09-16T09:36:51.513 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2024-09-16T09:36:51.513 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2024-09-16T09:36:51.514 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2024-09-16T09:36:51.514 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 327, 2024-09-16T09:36:51.514 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2024-09-16T09:36:51.514 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2024-09-16T09:36:51.514 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2024-09-16T09:36:51.514 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:36:51.514 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:36:51.514 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:36:51.514 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6810", 2024-09-16T09:36:51.514 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1670012115 2024-09-16T09:36:51.514 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.514 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:36:51.515 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:36:51.515 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6811", 2024-09-16T09:36:51.515 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1670012115 2024-09-16T09:36:51.515 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:36:51.515 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:36:51.515 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.515 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2024-09-16T09:36:51.515 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:36:51.515 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:36:51.516 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:36:51.516 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6812", 2024-09-16T09:36:51.516 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1670012115 2024-09-16T09:36:51.516 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.516 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:36:51.516 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:36:51.516 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6813", 2024-09-16T09:36:51.516 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1670012115 2024-09-16T09:36:51.516 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:36:51.516 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:36:51.516 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.516 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2024-09-16T09:36:51.516 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:36:51.517 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:36:51.517 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:36:51.517 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6816", 2024-09-16T09:36:51.517 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1670012115 2024-09-16T09:36:51.517 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.517 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:36:51.517 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:36:51.517 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6817", 2024-09-16T09:36:51.517 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1670012115 2024-09-16T09:36:51.517 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:36:51.517 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:36:51.517 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.517 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2024-09-16T09:36:51.518 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:36:51.518 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:36:51.518 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:36:51.518 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6814", 2024-09-16T09:36:51.518 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1670012115 2024-09-16T09:36:51.518 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.518 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:36:51.518 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:36:51.518 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6815", 2024-09-16T09:36:51.518 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1670012115 2024-09-16T09:36:51.518 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:36:51.518 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:36:51.518 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.519 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6811/1670012115", 2024-09-16T09:36:51.519 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6813/1670012115", 2024-09-16T09:36:51.519 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6817/1670012115", 2024-09-16T09:36:51.519 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6815/1670012115", 2024-09-16T09:36:51.519 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2024-09-16T09:36:51.519 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2024-09-16T09:36:51.519 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2024-09-16T09:36:51.519 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:36:51.520 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.520 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:36:51.520 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 2, 2024-09-16T09:36:51.520 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "5b4ca4e8-cecc-4163-b291-c9209b02c4b7", 2024-09-16T09:36:51.520 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2024-09-16T09:36:51.520 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2024-09-16T09:36:51.520 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2024-09-16T09:36:51.520 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 0.1810150146484375, 2024-09-16T09:36:51.520 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2024-09-16T09:36:51.520 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2024-09-16T09:36:51.520 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 18, 2024-09-16T09:36:51.520 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 255, 2024-09-16T09:36:51.521 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2024-09-16T09:36:51.521 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2024-09-16T09:36:51.521 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2024-09-16T09:36:51.521 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:36:51.521 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:36:51.521 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:36:51.521 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6818", 2024-09-16T09:36:51.521 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3601682336 2024-09-16T09:36:51.631 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.631 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:36:51.631 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:36:51.631 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6819", 2024-09-16T09:36:51.631 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3601682336 2024-09-16T09:36:51.631 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:36:51.631 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:36:51.632 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.632 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2024-09-16T09:36:51.632 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:36:51.632 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:36:51.632 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:36:51.632 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6820", 2024-09-16T09:36:51.632 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3601682336 2024-09-16T09:36:51.632 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.632 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:36:51.632 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:36:51.632 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6821", 2024-09-16T09:36:51.632 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3601682336 2024-09-16T09:36:51.632 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:36:51.632 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:36:51.633 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.633 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2024-09-16T09:36:51.633 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:36:51.633 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:36:51.633 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:36:51.633 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6824", 2024-09-16T09:36:51.633 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3601682336 2024-09-16T09:36:51.633 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.633 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:36:51.633 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:36:51.633 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6825", 2024-09-16T09:36:51.633 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3601682336 2024-09-16T09:36:51.633 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:36:51.633 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:36:51.634 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.634 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2024-09-16T09:36:51.634 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:36:51.634 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:36:51.634 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:36:51.634 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6822", 2024-09-16T09:36:51.634 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3601682336 2024-09-16T09:36:51.634 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.634 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:36:51.634 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:36:51.634 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6823", 2024-09-16T09:36:51.634 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3601682336 2024-09-16T09:36:51.634 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:36:51.634 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:36:51.635 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.635 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6819/3601682336", 2024-09-16T09:36:51.635 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6821/3601682336", 2024-09-16T09:36:51.635 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6825/3601682336", 2024-09-16T09:36:51.635 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6823/3601682336", 2024-09-16T09:36:51.635 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2024-09-16T09:36:51.635 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2024-09-16T09:36:51.635 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2024-09-16T09:36:51.635 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:36:51.635 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.635 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:36:51.635 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 3, 2024-09-16T09:36:51.635 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "ffb8d527-1b58-4ecd-808c-47d4ba1bf387", 2024-09-16T09:36:51.636 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2024-09-16T09:36:51.636 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2024-09-16T09:36:51.636 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 0.95001220703125, 2024-09-16T09:36:51.636 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 0.476531982421875, 2024-09-16T09:36:51.636 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2024-09-16T09:36:51.636 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2024-09-16T09:36:51.636 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 26, 2024-09-16T09:36:51.636 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 327, 2024-09-16T09:36:51.636 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2024-09-16T09:36:51.636 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2024-09-16T09:36:51.636 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2024-09-16T09:36:51.636 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:36:51.636 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:36:51.636 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:36:51.637 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6826", 2024-09-16T09:36:51.637 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2038083007 2024-09-16T09:36:51.637 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.637 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:36:51.637 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:36:51.637 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6827", 2024-09-16T09:36:51.637 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2038083007 2024-09-16T09:36:51.637 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:36:51.637 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:36:51.637 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.638 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2024-09-16T09:36:51.638 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:36:51.638 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:36:51.638 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:36:51.638 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6828", 2024-09-16T09:36:51.638 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2038083007 2024-09-16T09:36:51.638 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.638 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:36:51.638 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:36:51.638 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6829", 2024-09-16T09:36:51.638 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2038083007 2024-09-16T09:36:51.638 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:36:51.638 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:36:51.639 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.639 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2024-09-16T09:36:51.639 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:36:51.639 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:36:51.639 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:36:51.639 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6832", 2024-09-16T09:36:51.639 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2038083007 2024-09-16T09:36:51.639 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.639 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:36:51.639 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:36:51.639 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6833", 2024-09-16T09:36:51.639 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2038083007 2024-09-16T09:36:51.639 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:36:51.639 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:36:51.639 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.640 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2024-09-16T09:36:51.640 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:36:51.640 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:36:51.640 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:36:51.640 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6830", 2024-09-16T09:36:51.640 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2038083007 2024-09-16T09:36:51.640 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.640 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:36:51.640 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:36:51.640 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6831", 2024-09-16T09:36:51.640 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2038083007 2024-09-16T09:36:51.640 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:36:51.640 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:36:51.641 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.641 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6827/2038083007", 2024-09-16T09:36:51.641 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6829/2038083007", 2024-09-16T09:36:51.641 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6833/2038083007", 2024-09-16T09:36:51.641 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6831/2038083007", 2024-09-16T09:36:51.641 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2024-09-16T09:36:51.641 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2024-09-16T09:36:51.641 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2024-09-16T09:36:51.641 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:36:51.641 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.641 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:36:51.641 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 4, 2024-09-16T09:36:51.641 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "a946760b-4d91-4a4b-8e9b-a07a50589211", 2024-09-16T09:36:51.642 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2024-09-16T09:36:51.642 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2024-09-16T09:36:51.642 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2024-09-16T09:36:51.642 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2024-09-16T09:36:51.642 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2024-09-16T09:36:51.642 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2024-09-16T09:36:51.642 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 32, 2024-09-16T09:36:51.642 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 264, 2024-09-16T09:36:51.642 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2024-09-16T09:36:51.642 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2024-09-16T09:36:51.642 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2024-09-16T09:36:51.642 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:36:51.642 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:36:51.642 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:36:51.642 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6800", 2024-09-16T09:36:51.643 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2250892710 2024-09-16T09:36:51.643 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.643 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:36:51.643 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:36:51.643 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6801", 2024-09-16T09:36:51.643 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2250892710 2024-09-16T09:36:51.643 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:36:51.643 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:36:51.643 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.643 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2024-09-16T09:36:51.643 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:36:51.643 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:36:51.643 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:36:51.644 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6802", 2024-09-16T09:36:51.644 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2250892710 2024-09-16T09:36:51.644 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.644 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:36:51.644 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:36:51.644 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6803", 2024-09-16T09:36:51.644 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2250892710 2024-09-16T09:36:51.644 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:36:51.644 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:36:51.644 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.644 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2024-09-16T09:36:51.644 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:36:51.644 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:36:51.645 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:36:51.645 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6806", 2024-09-16T09:36:51.645 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2250892710 2024-09-16T09:36:51.645 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.645 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:36:51.645 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:36:51.645 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6807", 2024-09-16T09:36:51.645 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2250892710 2024-09-16T09:36:51.645 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:36:51.645 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:36:51.645 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.645 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2024-09-16T09:36:51.645 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:36:51.645 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:36:51.646 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:36:51.646 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6804", 2024-09-16T09:36:51.646 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2250892710 2024-09-16T09:36:51.646 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.646 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:36:51.646 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:36:51.646 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6805", 2024-09-16T09:36:51.646 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2250892710 2024-09-16T09:36:51.646 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:36:51.646 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:36:51.646 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.646 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.84:6801/2250892710", 2024-09-16T09:36:51.646 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.84:6803/2250892710", 2024-09-16T09:36:51.646 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.84:6807/2250892710", 2024-09-16T09:36:51.647 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.84:6805/2250892710", 2024-09-16T09:36:51.647 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2024-09-16T09:36:51.647 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2024-09-16T09:36:51.647 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2024-09-16T09:36:51.647 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:36:51.647 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.647 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:36:51.647 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 5, 2024-09-16T09:36:51.647 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "67681760-75a9-4846-946e-a935d42c5e05", 2024-09-16T09:36:51.647 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2024-09-16T09:36:51.647 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2024-09-16T09:36:51.647 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 0.95001220703125, 2024-09-16T09:36:51.647 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2024-09-16T09:36:51.647 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2024-09-16T09:36:51.648 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2024-09-16T09:36:51.648 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 39, 2024-09-16T09:36:51.648 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 330, 2024-09-16T09:36:51.648 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2024-09-16T09:36:51.648 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2024-09-16T09:36:51.648 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2024-09-16T09:36:51.648 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:36:51.648 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:36:51.648 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:36:51.648 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6808", 2024-09-16T09:36:51.648 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1803945348 2024-09-16T09:36:51.648 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.648 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:36:51.649 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:36:51.649 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6809", 2024-09-16T09:36:51.649 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1803945348 2024-09-16T09:36:51.649 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:36:51.649 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:36:51.649 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.649 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2024-09-16T09:36:51.649 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:36:51.649 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:36:51.649 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:36:51.649 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6810", 2024-09-16T09:36:51.649 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1803945348 2024-09-16T09:36:51.649 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.649 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:36:51.650 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:36:51.650 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6811", 2024-09-16T09:36:51.650 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1803945348 2024-09-16T09:36:51.650 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:36:51.650 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:36:51.650 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.650 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2024-09-16T09:36:51.650 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:36:51.650 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:36:51.650 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:36:51.650 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6814", 2024-09-16T09:36:51.650 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1803945348 2024-09-16T09:36:51.650 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.650 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:36:51.651 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:36:51.651 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6815", 2024-09-16T09:36:51.651 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1803945348 2024-09-16T09:36:51.651 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:36:51.651 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:36:51.651 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.651 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2024-09-16T09:36:51.651 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:36:51.651 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:36:51.651 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:36:51.651 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6812", 2024-09-16T09:36:51.651 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1803945348 2024-09-16T09:36:51.651 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.651 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:36:51.651 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:36:51.652 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6813", 2024-09-16T09:36:51.652 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1803945348 2024-09-16T09:36:51.652 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:36:51.652 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:36:51.652 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.652 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.84:6809/1803945348", 2024-09-16T09:36:51.652 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.84:6811/1803945348", 2024-09-16T09:36:51.652 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.84:6815/1803945348", 2024-09-16T09:36:51.652 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.84:6813/1803945348", 2024-09-16T09:36:51.652 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2024-09-16T09:36:51.652 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2024-09-16T09:36:51.652 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2024-09-16T09:36:51.652 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:36:51.653 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.653 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:36:51.653 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 6, 2024-09-16T09:36:51.653 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "7c8f0152-f539-4b7e-8113-91dd7529d3a1", 2024-09-16T09:36:51.653 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2024-09-16T09:36:51.653 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2024-09-16T09:36:51.653 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2024-09-16T09:36:51.653 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2024-09-16T09:36:51.653 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2024-09-16T09:36:51.653 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2024-09-16T09:36:51.653 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 46, 2024-09-16T09:36:51.653 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 328, 2024-09-16T09:36:51.653 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2024-09-16T09:36:51.653 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2024-09-16T09:36:51.654 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2024-09-16T09:36:51.654 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:36:51.654 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:36:51.654 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:36:51.654 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6816", 2024-09-16T09:36:51.654 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 610727189 2024-09-16T09:36:51.654 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.654 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:36:51.654 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:36:51.654 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6817", 2024-09-16T09:36:51.654 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 610727189 2024-09-16T09:36:51.654 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:36:51.654 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:36:51.654 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.654 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2024-09-16T09:36:51.655 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:36:51.655 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:36:51.655 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:36:51.655 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6818", 2024-09-16T09:36:51.655 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 610727189 2024-09-16T09:36:51.655 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.655 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:36:51.655 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:36:51.655 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6819", 2024-09-16T09:36:51.655 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 610727189 2024-09-16T09:36:51.655 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:36:51.655 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:36:51.655 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.655 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2024-09-16T09:36:51.656 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:36:51.656 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:36:51.656 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:36:51.656 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6822", 2024-09-16T09:36:51.656 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 610727189 2024-09-16T09:36:51.656 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.656 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:36:51.656 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:36:51.656 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6823", 2024-09-16T09:36:51.656 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 610727189 2024-09-16T09:36:51.656 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:36:51.656 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:36:51.656 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.656 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2024-09-16T09:36:51.657 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:36:51.657 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:36:51.657 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:36:51.657 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6820", 2024-09-16T09:36:51.657 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 610727189 2024-09-16T09:36:51.657 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.657 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:36:51.657 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:36:51.657 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6821", 2024-09-16T09:36:51.657 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 610727189 2024-09-16T09:36:51.657 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:36:51.657 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:36:51.657 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.658 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.84:6817/610727189", 2024-09-16T09:36:51.658 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.84:6819/610727189", 2024-09-16T09:36:51.658 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:36:51.659 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.84:6823/610727189", 2024-09-16T09:36:51.659 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.84:6821/610727189", 2024-09-16T09:36:51.659 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2024-09-16T09:36:51.659 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2024-09-16T09:36:51.659 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2024-09-16T09:36:51.659 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:36:51.659 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.660 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:36:51.660 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 7, 2024-09-16T09:36:51.660 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "945efaba-225e-49af-aefd-b4d58ca8093b", 2024-09-16T09:36:51.660 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2024-09-16T09:36:51.660 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2024-09-16T09:36:51.660 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 0.95001220703125, 2024-09-16T09:36:51.660 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2024-09-16T09:36:51.660 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2024-09-16T09:36:51.660 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2024-09-16T09:36:51.660 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 53, 2024-09-16T09:36:51.660 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 317, 2024-09-16T09:36:51.660 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2024-09-16T09:36:51.661 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2024-09-16T09:36:51.661 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2024-09-16T09:36:51.661 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:36:51.661 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:36:51.661 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:36:51.661 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6824", 2024-09-16T09:36:51.661 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1002252531 2024-09-16T09:36:51.661 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.661 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:36:51.661 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:36:51.661 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6825", 2024-09-16T09:36:51.661 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1002252531 2024-09-16T09:36:51.661 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:36:51.661 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:36:51.662 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.662 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2024-09-16T09:36:51.662 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:36:51.662 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:36:51.662 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:36:51.662 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6826", 2024-09-16T09:36:51.662 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1002252531 2024-09-16T09:36:51.662 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.662 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:36:51.662 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:36:51.662 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6827", 2024-09-16T09:36:51.662 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1002252531 2024-09-16T09:36:51.662 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:36:51.662 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:36:51.663 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.663 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2024-09-16T09:36:51.663 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:36:51.663 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:36:51.663 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:36:51.663 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6830", 2024-09-16T09:36:51.663 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1002252531 2024-09-16T09:36:51.663 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.663 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:36:51.663 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:36:51.663 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6831", 2024-09-16T09:36:51.663 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1002252531 2024-09-16T09:36:51.663 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:36:51.663 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:36:51.664 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.664 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2024-09-16T09:36:51.664 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:36:51.664 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:36:51.664 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:36:51.664 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6828", 2024-09-16T09:36:51.664 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1002252531 2024-09-16T09:36:51.664 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.664 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:36:51.664 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:36:51.664 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6829", 2024-09-16T09:36:51.664 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1002252531 2024-09-16T09:36:51.664 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:36:51.665 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:36:51.665 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.665 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.84:6825/1002252531", 2024-09-16T09:36:51.665 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.84:6827/1002252531", 2024-09-16T09:36:51.665 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.84:6831/1002252531", 2024-09-16T09:36:51.665 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.84:6829/1002252531", 2024-09-16T09:36:51.665 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2024-09-16T09:36:51.665 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2024-09-16T09:36:51.665 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2024-09-16T09:36:51.665 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:36:51.665 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:36:51.665 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-16T09:36:51.665 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_xinfo": [ 2024-09-16T09:36:51.665 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:36:51.666 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 0, 2024-09-16T09:36:51.666 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2024-09-16T09:36:51.666 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2024-09-16T09:36:51.666 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2024-09-16T09:36:51.666 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540719139924082687, 2024-09-16T09:36:51.666 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2024-09-16T09:36:51.666 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2024-09-16T09:36:22.587298+0000", 2024-09-16T09:36:51.666 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2024-09-16T09:36:51.666 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.666 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:36:51.666 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 1, 2024-09-16T09:36:51.666 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2024-09-16T09:36:51.666 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2024-09-16T09:36:51.666 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2024-09-16T09:36:51.667 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540719139924082687, 2024-09-16T09:36:51.667 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2024-09-16T09:36:51.667 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2024-09-16T09:36:03.094481+0000", 2024-09-16T09:36:51.667 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2024-09-16T09:36:51.667 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.667 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:36:51.667 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 2, 2024-09-16T09:36:51.667 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2024-09-16T09:36:51.667 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2024-09-16T09:36:51.667 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2024-09-16T09:36:51.667 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540719139924082687, 2024-09-16T09:36:51.667 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2024-09-16T09:36:51.667 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2024-09-16T09:36:03.981460+0000", 2024-09-16T09:36:51.668 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2024-09-16T09:36:51.668 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.668 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:36:51.668 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 3, 2024-09-16T09:36:51.668 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2024-09-16T09:36:51.668 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2024-09-16T09:36:51.668 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2024-09-16T09:36:51.668 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540719139924082687, 2024-09-16T09:36:51.668 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2024-09-16T09:36:51.668 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2024-09-16T09:36:07.542484+0000", 2024-09-16T09:36:51.668 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2024-09-16T09:36:51.668 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.668 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:36:51.668 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 4, 2024-09-16T09:36:51.669 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2024-09-16T09:36:51.669 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2024-09-16T09:36:51.669 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2024-09-16T09:36:51.669 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540719139924082687, 2024-09-16T09:36:51.669 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2024-09-16T09:36:51.669 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2024-09-16T09:36:07.833203+0000", 2024-09-16T09:36:51.669 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2024-09-16T09:36:51.669 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.669 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:36:51.669 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 5, 2024-09-16T09:36:51.669 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2024-09-16T09:36:51.669 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2024-09-16T09:36:51.669 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2024-09-16T09:36:51.669 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540719139924082687, 2024-09-16T09:36:51.670 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2024-09-16T09:36:51.670 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2024-09-16T09:36:23.873408+0000", 2024-09-16T09:36:51.670 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2024-09-16T09:36:51.670 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.670 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:36:51.670 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 6, 2024-09-16T09:36:51.670 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2024-09-16T09:36:51.670 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2024-09-16T09:36:51.670 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2024-09-16T09:36:51.670 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540719139924082687, 2024-09-16T09:36:51.670 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2024-09-16T09:36:51.670 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2024-09-16T09:36:29.981660+0000", 2024-09-16T09:36:51.670 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2024-09-16T09:36:51.670 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.671 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:36:51.671 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 7, 2024-09-16T09:36:51.671 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2024-09-16T09:36:51.671 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2024-09-16T09:36:51.671 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2024-09-16T09:36:51.671 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540719139924082687, 2024-09-16T09:36:51.671 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2024-09-16T09:36:51.671 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2024-09-16T09:36:16.693988+0000", 2024-09-16T09:36:51.671 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2024-09-16T09:36:51.671 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:36:51.671 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-16T09:36:51.671 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap": [ 2024-09-16T09:36:51.671 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:36:51.671 INFO:teuthology.orchestra.run.smithi070.stdout: "pgid": "3.1d", 2024-09-16T09:36:51.672 INFO:teuthology.orchestra.run.smithi070.stdout: "osds": [ 2024-09-16T09:36:51.672 INFO:teuthology.orchestra.run.smithi070.stdout: 1, 2024-09-16T09:36:51.672 INFO:teuthology.orchestra.run.smithi070.stdout: 0, 2024-09-16T09:36:51.672 INFO:teuthology.orchestra.run.smithi070.stdout: 7 2024-09-16T09:36:51.672 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:36:51.672 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.672 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:36:51.672 INFO:teuthology.orchestra.run.smithi070.stdout: "pgid": "4.b", 2024-09-16T09:36:51.672 INFO:teuthology.orchestra.run.smithi070.stdout: "osds": [ 2024-09-16T09:36:51.672 INFO:teuthology.orchestra.run.smithi070.stdout: 0, 2024-09-16T09:36:51.672 INFO:teuthology.orchestra.run.smithi070.stdout: 5, 2024-09-16T09:36:51.672 INFO:teuthology.orchestra.run.smithi070.stdout: 1 2024-09-16T09:36:51.672 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:36:51.672 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:36:51.673 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-16T09:36:51.673 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap_items": [ 2024-09-16T09:36:51.673 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:36:51.673 INFO:teuthology.orchestra.run.smithi070.stdout: "pgid": "3.1b", 2024-09-16T09:36:51.673 INFO:teuthology.orchestra.run.smithi070.stdout: "mappings": [ 2024-09-16T09:36:51.673 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:36:51.673 INFO:teuthology.orchestra.run.smithi070.stdout: "from": 0, 2024-09-16T09:36:51.673 INFO:teuthology.orchestra.run.smithi070.stdout: "to": 6 2024-09-16T09:36:51.673 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.673 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:36:51.673 INFO:teuthology.orchestra.run.smithi070.stdout: "from": 4, 2024-09-16T09:36:51.673 INFO:teuthology.orchestra.run.smithi070.stdout: "to": 7 2024-09-16T09:36:51.673 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:36:51.673 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:36:51.674 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.674 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:36:51.674 INFO:teuthology.orchestra.run.smithi070.stdout: "pgid": "4.f", 2024-09-16T09:36:51.674 INFO:teuthology.orchestra.run.smithi070.stdout: "mappings": [ 2024-09-16T09:36:51.674 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:36:51.674 INFO:teuthology.orchestra.run.smithi070.stdout: "from": 1, 2024-09-16T09:36:51.674 INFO:teuthology.orchestra.run.smithi070.stdout: "to": 2 2024-09-16T09:36:51.674 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:36:51.674 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:36:51.674 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.674 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:36:51.674 INFO:teuthology.orchestra.run.smithi070.stdout: "pgid": "4.13", 2024-09-16T09:36:51.674 INFO:teuthology.orchestra.run.smithi070.stdout: "mappings": [ 2024-09-16T09:36:51.674 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:36:51.674 INFO:teuthology.orchestra.run.smithi070.stdout: "from": 1, 2024-09-16T09:36:51.675 INFO:teuthology.orchestra.run.smithi070.stdout: "to": 5 2024-09-16T09:36:51.675 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:36:51.675 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:36:51.675 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:36:51.675 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-16T09:36:51.675 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap_primaries": [], 2024-09-16T09:36:51.675 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_temp": [], 2024-09-16T09:36:51.675 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_temp": [], 2024-09-16T09:36:51.675 INFO:teuthology.orchestra.run.smithi070.stdout: "blocklist": { 2024-09-16T09:36:51.675 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/125491368": "2024-09-17T09:25:31.465167+0000", 2024-09-16T09:36:51.675 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/3354284852": "2024-09-17T09:19:43.205989+0000", 2024-09-16T09:36:51.675 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/3284625547": "2024-09-17T09:25:31.465167+0000", 2024-09-16T09:36:51.675 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/186565817": "2024-09-17T09:19:43.205989+0000", 2024-09-16T09:36:51.675 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/2213852865": "2024-09-17T09:19:43.205989+0000", 2024-09-16T09:36:51.676 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/1812373760": "2024-09-17T09:20:09.306330+0000", 2024-09-16T09:36:51.676 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:6800/837338094": "2024-09-17T09:19:43.205989+0000", 2024-09-16T09:36:51.676 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:6800/1876661229": "2024-09-17T09:25:31.465167+0000", 2024-09-16T09:36:51.676 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/3351688485": "2024-09-17T09:20:09.306330+0000", 2024-09-16T09:36:51.676 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:6800/3871553358": "2024-09-17T09:20:09.306330+0000", 2024-09-16T09:36:51.676 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/1230482871": "2024-09-17T09:25:31.465167+0000", 2024-09-16T09:36:51.676 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:6801/837338094": "2024-09-17T09:19:43.205989+0000", 2024-09-16T09:36:51.676 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:6801/3871553358": "2024-09-17T09:20:09.306330+0000", 2024-09-16T09:36:51.676 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/1403001486": "2024-09-17T09:20:09.306330+0000", 2024-09-16T09:36:51.676 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/1723643739": "2024-09-17T09:25:31.465167+0000", 2024-09-16T09:36:51.676 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:6801/1876661229": "2024-09-17T09:25:31.465167+0000", 2024-09-16T09:36:51.676 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/3456417396": "2024-09-17T09:25:31.465167+0000" 2024-09-16T09:36:51.676 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.676 INFO:teuthology.orchestra.run.smithi070.stdout: "range_blocklist": {}, 2024-09-16T09:36:51.677 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profiles": { 2024-09-16T09:36:51.677 INFO:teuthology.orchestra.run.smithi070.stdout: "default": { 2024-09-16T09:36:51.677 INFO:teuthology.orchestra.run.smithi070.stdout: "crush-failure-domain": "osd", 2024-09-16T09:36:51.677 INFO:teuthology.orchestra.run.smithi070.stdout: "k": "2", 2024-09-16T09:36:51.677 INFO:teuthology.orchestra.run.smithi070.stdout: "m": "1", 2024-09-16T09:36:51.677 INFO:teuthology.orchestra.run.smithi070.stdout: "plugin": "jerasure", 2024-09-16T09:36:51.677 INFO:teuthology.orchestra.run.smithi070.stdout: "technique": "reed_sol_van" 2024-09-16T09:36:51.677 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:36:51.677 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:36:51.677 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps_queue": [], 2024-09-16T09:36:51.677 INFO:teuthology.orchestra.run.smithi070.stdout: "new_removed_snaps": [], 2024-09-16T09:36:51.677 INFO:teuthology.orchestra.run.smithi070.stdout: "new_purged_snaps": [], 2024-09-16T09:36:51.677 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_node_flags": {}, 2024-09-16T09:36:51.677 INFO:teuthology.orchestra.run.smithi070.stdout: "device_class_flags": {}, 2024-09-16T09:36:51.678 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode": { 2024-09-16T09:36:51.678 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode_enabled": false, 2024-09-16T09:36:51.678 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_bucket_count": 0, 2024-09-16T09:36:51.678 INFO:teuthology.orchestra.run.smithi070.stdout: "degraded_stretch_mode": 0, 2024-09-16T09:36:51.678 INFO:teuthology.orchestra.run.smithi070.stdout: "recovering_stretch_mode": 0, 2024-09-16T09:36:51.678 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode_bucket": 0 2024-09-16T09:36:51.678 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:36:51.678 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T09:36:51.717 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:51.718 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:51.718 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:36:51.758 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:51 smithi070 ceph-mon[30388]: 4.9 scrub starts 2024-09-16T09:36:51.758 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:51 smithi070 ceph-mon[30388]: 4.9 scrub ok 2024-09-16T09:36:51.758 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:51 smithi070 ceph-mon[30388]: 3.7 scrub starts 2024-09-16T09:36:51.759 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:51 smithi070 ceph-mon[30388]: 3.7 scrub ok 2024-09-16T09:36:51.759 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:51 smithi070 ceph-mon[30388]: 6.6 scrub starts 2024-09-16T09:36:51.759 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:51 smithi070 ceph-mon[30388]: 6.6 scrub ok 2024-09-16T09:36:51.759 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:51 smithi070 ceph-mon[30388]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 199/97407 objects degraded (0.204%), 1 pg degraded, 2 pgs undersized) 2024-09-16T09:36:51.759 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:51 smithi070 ceph-mon[30388]: 5.1b scrub starts 2024-09-16T09:36:51.759 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:51 smithi070 ceph-mon[30388]: 5.1b scrub ok 2024-09-16T09:36:51.759 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:51 smithi070 ceph-mon[30388]: osdmap e331: 8 total, 8 up, 8 in 2024-09-16T09:36:51.759 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:51 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/4171855427' entity='client.admin' cmd={"prefix": "osd dump", "format": "json-pretty"} : dispatch 2024-09-16T09:36:51.759 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:51 smithi070 ceph-mon[40691]: 4.9 scrub starts 2024-09-16T09:36:51.760 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:51 smithi070 ceph-mon[40691]: 4.9 scrub ok 2024-09-16T09:36:51.760 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:51 smithi070 ceph-mon[40691]: 3.7 scrub starts 2024-09-16T09:36:51.760 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:51 smithi070 ceph-mon[40691]: 3.7 scrub ok 2024-09-16T09:36:51.760 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:51 smithi070 ceph-mon[40691]: 6.6 scrub starts 2024-09-16T09:36:51.760 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:51 smithi070 ceph-mon[40691]: 6.6 scrub ok 2024-09-16T09:36:51.760 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:51 smithi070 ceph-mon[40691]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 199/97407 objects degraded (0.204%), 1 pg degraded, 2 pgs undersized) 2024-09-16T09:36:51.760 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:51 smithi070 ceph-mon[40691]: 5.1b scrub starts 2024-09-16T09:36:51.760 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:51 smithi070 ceph-mon[40691]: 5.1b scrub ok 2024-09-16T09:36:51.760 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:51 smithi070 ceph-mon[40691]: osdmap e331: 8 total, 8 up, 8 in 2024-09-16T09:36:51.760 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:51 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/4171855427' entity='client.admin' cmd={"prefix": "osd dump", "format": "json-pretty"} : dispatch 2024-09-16T09:36:51.819 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:36:51.853 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:51.854 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:51.854 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:36:51.955 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:36:51.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:51.998 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:51.998 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:36:52.099 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:36:52.136 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:52.136 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:52.137 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:36:52.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:51 smithi084 ceph-mon[32629]: 4.9 scrub starts 2024-09-16T09:36:52.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:51 smithi084 ceph-mon[32629]: 4.9 scrub ok 2024-09-16T09:36:52.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:51 smithi084 ceph-mon[32629]: 3.7 scrub starts 2024-09-16T09:36:52.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:51 smithi084 ceph-mon[32629]: 3.7 scrub ok 2024-09-16T09:36:52.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:51 smithi084 ceph-mon[32629]: 6.6 scrub starts 2024-09-16T09:36:52.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:51 smithi084 ceph-mon[32629]: 6.6 scrub ok 2024-09-16T09:36:52.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:51 smithi084 ceph-mon[32629]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 199/97407 objects degraded (0.204%), 1 pg degraded, 2 pgs undersized) 2024-09-16T09:36:52.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:51 smithi084 ceph-mon[32629]: 5.1b scrub starts 2024-09-16T09:36:52.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:51 smithi084 ceph-mon[32629]: 5.1b scrub ok 2024-09-16T09:36:52.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:51 smithi084 ceph-mon[32629]: osdmap e331: 8 total, 8 up, 8 in 2024-09-16T09:36:52.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:51 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/4171855427' entity='client.admin' cmd={"prefix": "osd dump", "format": "json-pretty"} : dispatch 2024-09-16T09:36:52.237 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:36:52.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:52.273 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:52.273 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:36:52.335 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:36:51 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:36:51] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:36:52.375 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:36:52.375 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T09:36:52.408 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:52.409 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:52.409 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:36:52.510 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:36:52.552 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:52.553 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:52.553 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:36:52.654 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:36:52.695 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:52.695 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:52.696 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:36:52.797 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:36:52.834 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:52.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:52.835 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:36:52.935 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:36:52.956 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:52.956 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:52.957 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:36:53.058 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:36:53.078 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:53.079 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:53.079 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:36:53.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:52 smithi084 ceph-mon[32629]: 9.14 scrub starts 2024-09-16T09:36:53.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:52 smithi084 ceph-mon[32629]: 6.f scrub starts 2024-09-16T09:36:53.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:52 smithi084 ceph-mon[32629]: 6.f scrub ok 2024-09-16T09:36:53.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:52 smithi084 ceph-mon[32629]: pgmap v601: 174 pgs: 1 creating+peering, 1 active+remapped+backfilling, 172 active+clean; 1.9 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 264 op/s; 67/93261 objects misplaced (0.072%); 25 MiB/s, 489 objects/s recovering 2024-09-16T09:36:53.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:52 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "21"} : dispatch 2024-09-16T09:36:53.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:52 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "21"} : dispatch 2024-09-16T09:36:53.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:52 smithi084 ceph-mon[32629]: 9.14 scrub ok 2024-09-16T09:36:53.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:52 smithi084 ceph-mon[32629]: 6.1d scrub starts 2024-09-16T09:36:53.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:52 smithi084 ceph-mon[32629]: 6.1d scrub ok 2024-09-16T09:36:53.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:52 smithi070 ceph-mon[30388]: 9.14 scrub starts 2024-09-16T09:36:53.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:52 smithi070 ceph-mon[30388]: 6.f scrub starts 2024-09-16T09:36:53.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:52 smithi070 ceph-mon[30388]: 6.f scrub ok 2024-09-16T09:36:53.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:52 smithi070 ceph-mon[30388]: pgmap v601: 174 pgs: 1 creating+peering, 1 active+remapped+backfilling, 172 active+clean; 1.9 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 264 op/s; 67/93261 objects misplaced (0.072%); 25 MiB/s, 489 objects/s recovering 2024-09-16T09:36:53.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:52 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "21"} : dispatch 2024-09-16T09:36:53.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:52 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "21"} : dispatch 2024-09-16T09:36:53.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:52 smithi070 ceph-mon[30388]: 9.14 scrub ok 2024-09-16T09:36:53.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:52 smithi070 ceph-mon[30388]: 6.1d scrub starts 2024-09-16T09:36:53.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:52 smithi070 ceph-mon[30388]: 6.1d scrub ok 2024-09-16T09:36:53.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:52 smithi070 ceph-mon[40691]: 9.14 scrub starts 2024-09-16T09:36:53.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:52 smithi070 ceph-mon[40691]: 6.f scrub starts 2024-09-16T09:36:53.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:52 smithi070 ceph-mon[40691]: 6.f scrub ok 2024-09-16T09:36:53.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:52 smithi070 ceph-mon[40691]: pgmap v601: 174 pgs: 1 creating+peering, 1 active+remapped+backfilling, 172 active+clean; 1.9 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 264 op/s; 67/93261 objects misplaced (0.072%); 25 MiB/s, 489 objects/s recovering 2024-09-16T09:36:53.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:52 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "21"} : dispatch 2024-09-16T09:36:53.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:52 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "21"} : dispatch 2024-09-16T09:36:53.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:52 smithi070 ceph-mon[40691]: 9.14 scrub ok 2024-09-16T09:36:53.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:52 smithi070 ceph-mon[40691]: 6.1d scrub starts 2024-09-16T09:36:53.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:52 smithi070 ceph-mon[40691]: 6.1d scrub ok 2024-09-16T09:36:53.179 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:36:53.200 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:53.201 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:53.201 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:36:53.302 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:36:53.324 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:53.324 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:53.324 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:36:53.425 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:36:53.446 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:53.447 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:53.448 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:36:53.548 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:36:53.570 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:53.570 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:53.570 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:36:53.651 INFO:tasks.thrashosds.thrasher:j is {'epoch': 331, 'fsid': 'a1321834-740c-11ef-bceb-c7b262605968', 'created': '2024-09-16T09:19:14.121498+0000', 'modified': '2024-09-16T09:36:51.309118+0000', 'last_up_change': '2024-09-16T09:24:51.124496+0000', 'last_in_change': '2024-09-16T09:24:28.432920+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 18, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 9, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T09:22:27.617694+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 21, 'pg_placement_num': 21, 'pg_placement_num_target': 21, 'pg_num_target': 21, 'pg_num_pending': 21, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '162', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '160', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6699999570846558, 'score_stable': 1.6699999570846558, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 1.899999976158142, 'raw_score_stable': 1.899999976158142, 'primary_affinity_weighted': 0.7071933746337891, 'average_primary_affinity': 0.8100000023841858, 'average_primary_affinity_weighted': 0.7099999785423279}}, {'pool': 2, 'pool_name': 'datapool', 'create_time': '2024-09-16T09:24:59.169244+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 3, 'pg_placement_num': 3, 'pg_placement_num_target': 3, 'pg_num_target': 3, 'pg_num_pending': 3, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 61, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.3299999237060547, 'score_stable': 2.3299999237060547, 'optimal_score': 1, 'raw_score_acting': 2.3299999237060547, 'raw_score_stable': 2.3299999237060547, 'primary_affinity_weighted': 0.7071933746337891, 'average_primary_affinity': 0.8100000023841858, 'average_primary_affinity_weighted': 0.7099999785423279}}, {'pool': 3, 'pool_name': '.rgw.root', 'create_time': '2024-09-16T09:25:00.056119+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.7071933746337891, 'average_primary_affinity': 0.8100000023841858, 'average_primary_affinity_weighted': 0.7099999785423279}}, {'pool': 4, 'pool_name': 'default.rgw.log', 'create_time': '2024-09-16T09:25:01.770016+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '62', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.849999904632568, 'score_stable': 2.190000057220459, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 7.809999942779541, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 0.7071933746337891, 'average_primary_affinity': 0.8100000023841858, 'average_primary_affinity_weighted': 0.7099999785423279, 'error_message': 'pool 4 has acting primaries on OSD(s) with primary affinity 0, read balance score is not accurate'}}, {'pool': 5, 'pool_name': 'default.rgw.control', 'create_time': '2024-09-16T09:25:03.725555+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.7071933746337891, 'average_primary_affinity': 0.8100000023841858, 'average_primary_affinity_weighted': 0.7099999785423279}}, {'pool': 6, 'pool_name': 'default.rgw.meta', 'create_time': '2024-09-16T09:25:05.773530+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5399999618530273, 'score_stable': 1.5399999618530273, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 0.7071933746337891, 'average_primary_affinity': 0.8100000023841858, 'average_primary_affinity_weighted': 0.7099999785423279}}, {'pool': 9, 'pool_name': 'unique_pool_2', 'create_time': '2024-09-16T09:33:23.906323+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 22, 'pg_placement_num': 16, 'pg_placement_num_target': 16, 'pg_num_target': 16, 'pg_num_pending': 22, 'last_pg_merge_meta': {'source_pgid': '9.16', 'ready_epoch': 330, 'last_epoch_started': 329, 'last_epoch_clean': 329, 'source_version': "327'8255", 'target_version': "327'8239"}, 'last_change': '330', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '330', 'last_force_op_resend_preluminous': '328', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rados': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.600000023841858, 'score_stable': 1.600000023841858, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 1.8200000524520874, 'raw_score_stable': 1.8200000524520874, 'primary_affinity_weighted': 0.7071933746337891, 'average_primary_affinity': 0.8100000023841858, 'average_primary_affinity_weighted': 0.7099999785423279}}], 'osds': [{'osd': 0, 'uuid': '7ef288c8-3bf0-4814-8e86-f70a7de041ce', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 8, 'up_thru': 326, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6802', 'nonce': 2013075781}, {'type': 'v1', 'addr': '172.21.15.70:6803', 'nonce': 2013075781}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6804', 'nonce': 2013075781}, {'type': 'v1', 'addr': '172.21.15.70:6805', 'nonce': 2013075781}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6808', 'nonce': 2013075781}, {'type': 'v1', 'addr': '172.21.15.70:6809', 'nonce': 2013075781}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6806', 'nonce': 2013075781}, {'type': 'v1', 'addr': '172.21.15.70:6807', 'nonce': 2013075781}]}, 'public_addr': '172.21.15.70:6803/2013075781', 'cluster_addr': '172.21.15.70:6805/2013075781', 'heartbeat_back_addr': '172.21.15.70:6809/2013075781', 'heartbeat_front_addr': '172.21.15.70:6807/2013075781', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '474d41d9-295a-450c-8f27-ce487a14678b', 'up': 1, 'in': 1, 'weight': 0.95001220703125, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 327, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6810', 'nonce': 1670012115}, {'type': 'v1', 'addr': '172.21.15.70:6811', 'nonce': 1670012115}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6812', 'nonce': 1670012115}, {'type': 'v1', 'addr': '172.21.15.70:6813', 'nonce': 1670012115}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6816', 'nonce': 1670012115}, {'type': 'v1', 'addr': '172.21.15.70:6817', 'nonce': 1670012115}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6814', 'nonce': 1670012115}, {'type': 'v1', 'addr': '172.21.15.70:6815', 'nonce': 1670012115}]}, 'public_addr': '172.21.15.70:6811/1670012115', 'cluster_addr': '172.21.15.70:6813/1670012115', 'heartbeat_back_addr': '172.21.15.70:6817/1670012115', 'heartbeat_front_addr': '172.21.15.70:6815/1670012115', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '5b4ca4e8-cecc-4163-b291-c9209b02c4b7', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.1810150146484375, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 18, 'up_thru': 255, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6818', 'nonce': 3601682336}, {'type': 'v1', 'addr': '172.21.15.70:6819', 'nonce': 3601682336}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6820', 'nonce': 3601682336}, {'type': 'v1', 'addr': '172.21.15.70:6821', 'nonce': 3601682336}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6824', 'nonce': 3601682336}, {'type': 'v1', 'addr': '172.21.15.70:6825', 'nonce': 3601682336}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6822', 'nonce': 3601682336}, {'type': 'v1', 'addr': '172.21.15.70:6823', 'nonce': 3601682336}]}, 'public_addr': '172.21.15.70:6819/3601682336', 'cluster_addr': '172.21.15.70:6821/3601682336', 'heartbeat_back_addr': '172.21.15.70:6825/3601682336', 'heartbeat_front_addr': '172.21.15.70:6823/3601682336', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'ffb8d527-1b58-4ecd-808c-47d4ba1bf387', 'up': 1, 'in': 1, 'weight': 0.95001220703125, 'primary_affinity': 0.476531982421875, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 26, 'up_thru': 327, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6826', 'nonce': 2038083007}, {'type': 'v1', 'addr': '172.21.15.70:6827', 'nonce': 2038083007}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6828', 'nonce': 2038083007}, {'type': 'v1', 'addr': '172.21.15.70:6829', 'nonce': 2038083007}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6832', 'nonce': 2038083007}, {'type': 'v1', 'addr': '172.21.15.70:6833', 'nonce': 2038083007}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6830', 'nonce': 2038083007}, {'type': 'v1', 'addr': '172.21.15.70:6831', 'nonce': 2038083007}]}, 'public_addr': '172.21.15.70:6827/2038083007', 'cluster_addr': '172.21.15.70:6829/2038083007', 'heartbeat_back_addr': '172.21.15.70:6833/2038083007', 'heartbeat_front_addr': '172.21.15.70:6831/2038083007', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'a946760b-4d91-4a4b-8e9b-a07a50589211', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 32, 'up_thru': 264, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6800', 'nonce': 2250892710}, {'type': 'v1', 'addr': '172.21.15.84:6801', 'nonce': 2250892710}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6802', 'nonce': 2250892710}, {'type': 'v1', 'addr': '172.21.15.84:6803', 'nonce': 2250892710}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6806', 'nonce': 2250892710}, {'type': 'v1', 'addr': '172.21.15.84:6807', 'nonce': 2250892710}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6804', 'nonce': 2250892710}, {'type': 'v1', 'addr': '172.21.15.84:6805', 'nonce': 2250892710}]}, 'public_addr': '172.21.15.84:6801/2250892710', 'cluster_addr': '172.21.15.84:6803/2250892710', 'heartbeat_back_addr': '172.21.15.84:6807/2250892710', 'heartbeat_front_addr': '172.21.15.84:6805/2250892710', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '67681760-75a9-4846-946e-a935d42c5e05', 'up': 1, 'in': 1, 'weight': 0.95001220703125, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 39, 'up_thru': 330, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6808', 'nonce': 1803945348}, {'type': 'v1', 'addr': '172.21.15.84:6809', 'nonce': 1803945348}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6810', 'nonce': 1803945348}, {'type': 'v1', 'addr': '172.21.15.84:6811', 'nonce': 1803945348}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6814', 'nonce': 1803945348}, {'type': 'v1', 'addr': '172.21.15.84:6815', 'nonce': 1803945348}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6812', 'nonce': 1803945348}, {'type': 'v1', 'addr': '172.21.15.84:6813', 'nonce': 1803945348}]}, 'public_addr': '172.21.15.84:6809/1803945348', 'cluster_addr': '172.21.15.84:6811/1803945348', 'heartbeat_back_addr': '172.21.15.84:6815/1803945348', 'heartbeat_front_addr': '172.21.15.84:6813/1803945348', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '7c8f0152-f539-4b7e-8113-91dd7529d3a1', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 46, 'up_thru': 328, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6816', 'nonce': 610727189}, {'type': 'v1', 'addr': '172.21.15.84:6817', 'nonce': 610727189}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6818', 'nonce': 610727189}, {'type': 'v1', 'addr': '172.21.15.84:6819', 'nonce': 610727189}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6822', 'nonce': 610727189}, {'type': 'v1', 'addr': '172.21.15.84:6823', 'nonce': 610727189}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6820', 'nonce': 610727189}, {'type': 'v1', 'addr': '172.21.15.84:6821', 'nonce': 610727189}]}, 'public_addr': '172.21.15.84:6817/610727189', 'cluster_addr': '172.21.15.84:6819/610727189', 'heartbeat_back_addr': '172.21.15.84:6823/610727189', 'heartbeat_front_addr': '172.21.15.84:6821/610727189', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '945efaba-225e-49af-aefd-b4d58ca8093b', 'up': 1, 'in': 1, 'weight': 0.95001220703125, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 53, 'up_thru': 317, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6824', 'nonce': 1002252531}, {'type': 'v1', 'addr': '172.21.15.84:6825', 'nonce': 1002252531}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6826', 'nonce': 1002252531}, {'type': 'v1', 'addr': '172.21.15.84:6827', 'nonce': 1002252531}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6830', 'nonce': 1002252531}, {'type': 'v1', 'addr': '172.21.15.84:6831', 'nonce': 1002252531}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6828', 'nonce': 1002252531}, {'type': 'v1', 'addr': '172.21.15.84:6829', 'nonce': 1002252531}]}, 'public_addr': '172.21.15.84:6825/1002252531', 'cluster_addr': '172.21.15.84:6827/1002252531', 'heartbeat_back_addr': '172.21.15.84:6831/1002252531', 'heartbeat_front_addr': '172.21.15.84:6829/1002252531', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T09:36:22.587298+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T09:36:03.094481+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T09:36:03.981460+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T09:36:07.542484+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T09:36:07.833203+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T09:36:23.873408+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T09:36:29.981660+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T09:36:16.693988+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.1d', 'osds': [1, 0, 7]}, {'pgid': '4.b', 'osds': [0, 5, 1]}], 'pg_upmap_items': [{'pgid': '3.1b', 'mappings': [{'from': 0, 'to': 6}, {'from': 4, 'to': 7}]}, {'pgid': '4.f', 'mappings': [{'from': 1, 'to': 2}]}, {'pgid': '4.13', 'mappings': [{'from': 1, 'to': 5}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {'172.21.15.70:0/125491368': '2024-09-17T09:25:31.465167+0000', '172.21.15.70:0/3354284852': '2024-09-17T09:19:43.205989+0000', '172.21.15.70:0/3284625547': '2024-09-17T09:25:31.465167+0000', '172.21.15.70:0/186565817': '2024-09-17T09:19:43.205989+0000', '172.21.15.70:0/2213852865': '2024-09-17T09:19:43.205989+0000', '172.21.15.70:0/1812373760': '2024-09-17T09:20:09.306330+0000', '172.21.15.70:6800/837338094': '2024-09-17T09:19:43.205989+0000', '172.21.15.70:6800/1876661229': '2024-09-17T09:25:31.465167+0000', '172.21.15.70:0/3351688485': '2024-09-17T09:20:09.306330+0000', '172.21.15.70:6800/3871553358': '2024-09-17T09:20:09.306330+0000', '172.21.15.70:0/1230482871': '2024-09-17T09:25:31.465167+0000', '172.21.15.70:6801/837338094': '2024-09-17T09:19:43.205989+0000', '172.21.15.70:6801/3871553358': '2024-09-17T09:20:09.306330+0000', '172.21.15.70:0/1403001486': '2024-09-17T09:20:09.306330+0000', '172.21.15.70:0/1723643739': '2024-09-17T09:25:31.465167+0000', '172.21.15.70:6801/1876661229': '2024-09-17T09:25:31.465167+0000', '172.21.15.70:0/3456417396': '2024-09-17T09:25:31.465167+0000'}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-16T09:36:53.651 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-16T09:36:53.660 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:36:53 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:36:53.671 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:36:53.715 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:53.716 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:53.716 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:36:53.817 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:36:53.838 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:53.839 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:53.839 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:36:53.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:53 smithi070 ceph-mon[30388]: 4.14 scrub starts 2024-09-16T09:36:53.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:53 smithi070 ceph-mon[30388]: 4.14 scrub ok 2024-09-16T09:36:53.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:53 smithi070 ceph-mon[30388]: 5.3 scrub starts 2024-09-16T09:36:53.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:53 smithi070 ceph-mon[30388]: 5.3 scrub ok 2024-09-16T09:36:53.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:53 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "21"}]': finished 2024-09-16T09:36:53.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:53 smithi070 ceph-mon[30388]: osdmap e332: 8 total, 8 up, 8 in 2024-09-16T09:36:53.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:53 smithi070 ceph-mon[30388]: 5.19 scrub starts 2024-09-16T09:36:53.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:53 smithi070 ceph-mon[30388]: 5.19 scrub ok 2024-09-16T09:36:53.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:53 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:36:53.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:53 smithi070 ceph-mon[40691]: 4.14 scrub starts 2024-09-16T09:36:53.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:53 smithi070 ceph-mon[40691]: 4.14 scrub ok 2024-09-16T09:36:53.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:53 smithi070 ceph-mon[40691]: 5.3 scrub starts 2024-09-16T09:36:53.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:53 smithi070 ceph-mon[40691]: 5.3 scrub ok 2024-09-16T09:36:53.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:53 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "21"}]': finished 2024-09-16T09:36:53.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:53 smithi070 ceph-mon[40691]: osdmap e332: 8 total, 8 up, 8 in 2024-09-16T09:36:53.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:53 smithi070 ceph-mon[40691]: 5.19 scrub starts 2024-09-16T09:36:53.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:53 smithi070 ceph-mon[40691]: 5.19 scrub ok 2024-09-16T09:36:53.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:53 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:36:53.940 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:36:53.961 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:53.962 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:53.962 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:36:54.063 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:36:54.084 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:54.084 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:54.085 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:36:54.099 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T09:36:54.099 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T09:36:54.099 INFO:teuthology.orchestra.run.smithi070.stdout: "num_ops": 0 2024-09-16T09:36:54.099 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T09:36:54.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:53 smithi084 ceph-mon[32629]: 4.14 scrub starts 2024-09-16T09:36:54.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:53 smithi084 ceph-mon[32629]: 4.14 scrub ok 2024-09-16T09:36:54.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:53 smithi084 ceph-mon[32629]: 5.3 scrub starts 2024-09-16T09:36:54.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:53 smithi084 ceph-mon[32629]: 5.3 scrub ok 2024-09-16T09:36:54.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:53 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "21"}]': finished 2024-09-16T09:36:54.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:53 smithi084 ceph-mon[32629]: osdmap e332: 8 total, 8 up, 8 in 2024-09-16T09:36:54.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:53 smithi084 ceph-mon[32629]: 5.19 scrub starts 2024-09-16T09:36:54.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:53 smithi084 ceph-mon[32629]: 5.19 scrub ok 2024-09-16T09:36:54.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:53 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:36:54.186 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:36:54.207 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:54.208 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:54.208 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:36:54.309 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:36:54.334 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:54.335 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:54.335 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:36:54.340 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:36:54.436 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:36:54.457 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:54.458 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:54.458 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:36:54.559 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:36:54.580 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:54.581 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:54.581 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:36:54.682 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:36:54.704 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:54.704 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:54.705 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:36:54.734 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:54 smithi070 ceph-mon[30388]: 6.9 scrub starts 2024-09-16T09:36:54.734 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:54 smithi070 ceph-mon[30388]: 6.9 scrub ok 2024-09-16T09:36:54.734 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:54 smithi070 ceph-mon[30388]: 1.c scrub starts 2024-09-16T09:36:54.735 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:54 smithi070 ceph-mon[30388]: 1.c scrub ok 2024-09-16T09:36:54.735 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:54 smithi070 ceph-mon[30388]: osdmap e333: 8 total, 8 up, 8 in 2024-09-16T09:36:54.735 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:54 smithi070 ceph-mon[30388]: pgmap v604: 174 pgs: 174 active+clean; 1.8 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:36:54.735 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:54 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1137970998' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:36:54.735 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:54 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:36:54.738 INFO:teuthology.orchestra.run.smithi070.stderr:nodeep-scrub is unset 2024-09-16T09:36:54.806 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:36:54.827 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:54.828 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:54.828 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:36:54.907 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.2 dump_blocked_ops 2024-09-16T09:36:54.930 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:36:54.971 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:54.972 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:54.972 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:36:55.073 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:36:55.095 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:55.096 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:55.096 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:36:55.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:54 smithi084 ceph-mon[32629]: 6.9 scrub starts 2024-09-16T09:36:55.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:54 smithi084 ceph-mon[32629]: 6.9 scrub ok 2024-09-16T09:36:55.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:54 smithi084 ceph-mon[32629]: 1.c scrub starts 2024-09-16T09:36:55.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:54 smithi084 ceph-mon[32629]: 1.c scrub ok 2024-09-16T09:36:55.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:54 smithi084 ceph-mon[32629]: osdmap e333: 8 total, 8 up, 8 in 2024-09-16T09:36:55.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:54 smithi084 ceph-mon[32629]: pgmap v604: 174 pgs: 174 active+clean; 1.8 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:36:55.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:54 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1137970998' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:36:55.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:54 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:36:55.170 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:54 smithi070 ceph-mon[40691]: 6.9 scrub starts 2024-09-16T09:36:55.170 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:54 smithi070 ceph-mon[40691]: 6.9 scrub ok 2024-09-16T09:36:55.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:54 smithi070 ceph-mon[40691]: 1.c scrub starts 2024-09-16T09:36:55.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:54 smithi070 ceph-mon[40691]: 1.c scrub ok 2024-09-16T09:36:55.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:54 smithi070 ceph-mon[40691]: osdmap e333: 8 total, 8 up, 8 in 2024-09-16T09:36:55.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:54 smithi070 ceph-mon[40691]: pgmap v604: 174 pgs: 174 active+clean; 1.8 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:36:55.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:54 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1137970998' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:36:55.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:54 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:36:55.197 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:36:55.219 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:55.219 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:55.220 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:36:55.321 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:36:55.341 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:55.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:55.342 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:36:55.443 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:36:55.464 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:55.465 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:55.466 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:36:55.567 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:36:55.593 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:55.593 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:55.594 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:36:55.612 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:36:55.694 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:36:55.721 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:55.721 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:55.722 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:36:55.823 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:36:55.851 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:55 smithi070 ceph-mon[30388]: 4.1c scrub starts 2024-09-16T09:36:55.851 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:55 smithi070 ceph-mon[30388]: 4.1c scrub ok 2024-09-16T09:36:55.851 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:55 smithi070 ceph-mon[30388]: 5.17 scrub starts 2024-09-16T09:36:55.851 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:55 smithi070 ceph-mon[30388]: 5.17 scrub ok 2024-09-16T09:36:55.851 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:55 smithi070 ceph-mon[30388]: 4.3 scrub starts 2024-09-16T09:36:55.851 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:55 smithi070 ceph-mon[30388]: 4.3 scrub ok 2024-09-16T09:36:55.851 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:55 smithi070 ceph-mon[30388]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-16T09:36:55.851 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:55 smithi070 ceph-mon[30388]: Cluster is now healthy 2024-09-16T09:36:55.851 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:55 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-16T09:36:55.851 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:55 smithi070 ceph-mon[30388]: osdmap e334: 8 total, 8 up, 8 in 2024-09-16T09:36:55.852 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:55 smithi070 ceph-mon[40691]: 4.1c scrub starts 2024-09-16T09:36:55.852 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:55 smithi070 ceph-mon[40691]: 4.1c scrub ok 2024-09-16T09:36:55.852 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:55 smithi070 ceph-mon[40691]: 5.17 scrub starts 2024-09-16T09:36:55.852 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:55 smithi070 ceph-mon[40691]: 5.17 scrub ok 2024-09-16T09:36:55.852 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:55 smithi070 ceph-mon[40691]: 4.3 scrub starts 2024-09-16T09:36:55.852 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:55 smithi070 ceph-mon[40691]: 4.3 scrub ok 2024-09-16T09:36:55.852 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:55 smithi070 ceph-mon[40691]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-16T09:36:55.852 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:55 smithi070 ceph-mon[40691]: Cluster is now healthy 2024-09-16T09:36:55.852 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:55 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-16T09:36:55.852 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:55 smithi070 ceph-mon[40691]: osdmap e334: 8 total, 8 up, 8 in 2024-09-16T09:36:55.858 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:55.859 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:55.859 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:36:55.960 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:36:55.997 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:55.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:55.998 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:36:56.099 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:36:56.137 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:56.140 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:56.140 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:36:56.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:55 smithi084 ceph-mon[32629]: 4.1c scrub starts 2024-09-16T09:36:56.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:55 smithi084 ceph-mon[32629]: 4.1c scrub ok 2024-09-16T09:36:56.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:55 smithi084 ceph-mon[32629]: 5.17 scrub starts 2024-09-16T09:36:56.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:55 smithi084 ceph-mon[32629]: 5.17 scrub ok 2024-09-16T09:36:56.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:55 smithi084 ceph-mon[32629]: 4.3 scrub starts 2024-09-16T09:36:56.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:55 smithi084 ceph-mon[32629]: 4.3 scrub ok 2024-09-16T09:36:56.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:55 smithi084 ceph-mon[32629]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-16T09:36:56.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:55 smithi084 ceph-mon[32629]: Cluster is now healthy 2024-09-16T09:36:56.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:55 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-16T09:36:56.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:55 smithi084 ceph-mon[32629]: osdmap e334: 8 total, 8 up, 8 in 2024-09-16T09:36:56.242 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:36:56.272 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:56.272 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:56.272 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:36:56.373 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:36:56.408 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:56.409 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:56.409 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:36:56.510 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:36:56.539 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:56.540 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:56.540 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:36:56.582 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T09:36:56.582 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:36:56.626 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:36:56.627 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:36:56.627 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:36:56.627 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:36:56.627 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:36:56.627 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:36:56.627 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:36:56.628 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:36:56.628 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:36:56.628 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:36:56.640 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:36:56.659 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:36:56.660 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:36:56.677 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:56.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:56.678 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:36:56.696 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:36:56.696 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:36:56.730 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:36:56.730 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:36:56.779 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:36:56.815 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:56.816 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:56.816 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:36:56.917 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:36:56.942 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:56.942 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:56.943 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:36:57.043 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:36:57.064 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:36:57.065 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:57.065 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:57.065 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:36:57.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:56 smithi084 ceph-mon[32629]: 5.6 scrub starts 2024-09-16T09:36:57.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:56 smithi084 ceph-mon[32629]: 5.6 scrub ok 2024-09-16T09:36:57.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:56 smithi084 ceph-mon[32629]: 4.b scrub starts 2024-09-16T09:36:57.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:56 smithi084 ceph-mon[32629]: 4.b scrub ok 2024-09-16T09:36:57.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:56 smithi084 ceph-mon[32629]: pgmap v606: 173 pgs: 1 clean+premerge+peered, 172 active+clean; 1.7 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-16T09:36:57.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:56 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "20"} : dispatch 2024-09-16T09:36:57.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:56 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "20"} : dispatch 2024-09-16T09:36:57.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:56 smithi084 ceph-mon[32629]: osdmap e335: 8 total, 8 up, 8 in 2024-09-16T09:36:57.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:56 smithi084 ceph-mon[32629]: 9.2 scrub starts 2024-09-16T09:36:57.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:56 smithi084 ceph-mon[32629]: 4.12 scrub starts 2024-09-16T09:36:57.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:56 smithi084 ceph-mon[32629]: 4.12 scrub ok 2024-09-16T09:36:57.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:56 smithi084 ceph-mon[32629]: 9.2 scrub ok 2024-09-16T09:36:57.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:56 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/4235603019' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:36:57.166 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:36:57.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:56 smithi070 ceph-mon[30388]: 5.6 scrub starts 2024-09-16T09:36:57.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:56 smithi070 ceph-mon[30388]: 5.6 scrub ok 2024-09-16T09:36:57.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:56 smithi070 ceph-mon[30388]: 4.b scrub starts 2024-09-16T09:36:57.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:56 smithi070 ceph-mon[30388]: 4.b scrub ok 2024-09-16T09:36:57.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:56 smithi070 ceph-mon[30388]: pgmap v606: 173 pgs: 1 clean+premerge+peered, 172 active+clean; 1.7 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-16T09:36:57.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:56 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "20"} : dispatch 2024-09-16T09:36:57.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:56 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "20"} : dispatch 2024-09-16T09:36:57.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:56 smithi070 ceph-mon[30388]: osdmap e335: 8 total, 8 up, 8 in 2024-09-16T09:36:57.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:56 smithi070 ceph-mon[30388]: 9.2 scrub starts 2024-09-16T09:36:57.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:56 smithi070 ceph-mon[30388]: 4.12 scrub starts 2024-09-16T09:36:57.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:56 smithi070 ceph-mon[30388]: 4.12 scrub ok 2024-09-16T09:36:57.172 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:56 smithi070 ceph-mon[30388]: 9.2 scrub ok 2024-09-16T09:36:57.172 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:56 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/4235603019' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:36:57.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:56 smithi070 ceph-mon[40691]: 5.6 scrub starts 2024-09-16T09:36:57.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:56 smithi070 ceph-mon[40691]: 5.6 scrub ok 2024-09-16T09:36:57.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:56 smithi070 ceph-mon[40691]: 4.b scrub starts 2024-09-16T09:36:57.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:56 smithi070 ceph-mon[40691]: 4.b scrub ok 2024-09-16T09:36:57.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:56 smithi070 ceph-mon[40691]: pgmap v606: 173 pgs: 1 clean+premerge+peered, 172 active+clean; 1.7 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-16T09:36:57.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:56 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "20"} : dispatch 2024-09-16T09:36:57.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:56 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "20"} : dispatch 2024-09-16T09:36:57.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:56 smithi070 ceph-mon[40691]: osdmap e335: 8 total, 8 up, 8 in 2024-09-16T09:36:57.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:56 smithi070 ceph-mon[40691]: 9.2 scrub starts 2024-09-16T09:36:57.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:56 smithi070 ceph-mon[40691]: 4.12 scrub starts 2024-09-16T09:36:57.173 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:56 smithi070 ceph-mon[40691]: 4.12 scrub ok 2024-09-16T09:36:57.173 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:56 smithi070 ceph-mon[40691]: 9.2 scrub ok 2024-09-16T09:36:57.173 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:56 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/4235603019' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:36:57.187 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:57.188 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:57.189 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:36:57.289 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:36:57.311 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:57.311 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:57.312 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:36:57.413 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:36:57.434 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:57.435 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:57.435 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:36:57.536 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd set noscrub 2024-09-16T09:36:57.536 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:36:57.559 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:57.560 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:57.560 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:36:57.642 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.2 dump_historic_ops 2024-09-16T09:36:57.661 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:36:57.706 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:57.706 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:57.707 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:36:57.807 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:36:57.829 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:57.829 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:57.830 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:36:57.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:57 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "20"}]': finished 2024-09-16T09:36:57.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:57 smithi070 ceph-mon[30388]: osdmap e336: 8 total, 8 up, 8 in 2024-09-16T09:36:57.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:57 smithi070 ceph-mon[30388]: 9.a scrub starts 2024-09-16T09:36:57.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:57 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "20"}]': finished 2024-09-16T09:36:57.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:57 smithi070 ceph-mon[40691]: osdmap e336: 8 total, 8 up, 8 in 2024-09-16T09:36:57.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:57 smithi070 ceph-mon[40691]: 9.a scrub starts 2024-09-16T09:36:57.930 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:36:57.953 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:57.953 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:57.954 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:36:58.055 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:36:58.077 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:58.078 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:58.078 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:36:58.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:57 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "20"}]': finished 2024-09-16T09:36:58.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:57 smithi084 ceph-mon[32629]: osdmap e336: 8 total, 8 up, 8 in 2024-09-16T09:36:58.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:57 smithi084 ceph-mon[32629]: 9.a scrub starts 2024-09-16T09:36:58.179 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:36:58.200 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:58.201 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:58.201 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:36:58.302 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:36:58.324 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:58.324 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:58.325 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:36:58.426 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:36:58.447 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:58.447 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:58.448 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:36:58.549 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:36:58.571 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:58.571 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:58.572 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:36:58.656 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:36:58.658 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:36:58.673 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:36:58.695 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:58.695 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:58.696 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:36:58.797 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:36:58.815 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:36:58.817 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2024-09-16T09:36:58.822 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:58.823 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:58.823 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:36:58.924 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:36:58.945 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:58.945 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:58.946 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:36:59.046 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:36:59.050 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:58 smithi070 ceph-mon[30388]: 9.a scrub ok 2024-09-16T09:36:59.050 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:58 smithi070 ceph-mon[30388]: pgmap v609: 173 pgs: 1 clean+premerge+peered, 172 active+clean; 1.7 GiB data, 10 GiB used, 705 GiB / 715 GiB avail 2024-09-16T09:36:59.050 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:58 smithi070 ceph-mon[30388]: osdmap e337: 8 total, 8 up, 8 in 2024-09-16T09:36:59.050 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:58 smithi070 ceph-mon[40691]: 9.a scrub ok 2024-09-16T09:36:59.050 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:58 smithi070 ceph-mon[40691]: pgmap v609: 173 pgs: 1 clean+premerge+peered, 172 active+clean; 1.7 GiB data, 10 GiB used, 705 GiB / 715 GiB avail 2024-09-16T09:36:59.050 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:58 smithi070 ceph-mon[40691]: osdmap e337: 8 total, 8 up, 8 in 2024-09-16T09:36:59.068 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:59.069 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:59.069 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:36:59.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:58 smithi084 ceph-mon[32629]: 9.a scrub ok 2024-09-16T09:36:59.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:58 smithi084 ceph-mon[32629]: pgmap v609: 173 pgs: 1 clean+premerge+peered, 172 active+clean; 1.7 GiB data, 10 GiB used, 705 GiB / 715 GiB avail 2024-09-16T09:36:59.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:58 smithi084 ceph-mon[32629]: osdmap e337: 8 total, 8 up, 8 in 2024-09-16T09:36:59.169 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:36:59.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:59.191 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:59.192 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:36:59.292 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:36:59.314 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:59.315 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:59.315 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:36:59.416 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:36:59.456 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:59.456 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:59.456 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:36:59.557 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:36:59.589 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:59.590 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:59.590 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:36:59.691 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:36:59.724 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:59.725 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:36:59.725 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:36:59.826 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:36:59.861 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:36:59.861 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:36:59.862 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:36:59.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:59 smithi070 ceph-mon[30388]: 6.14 scrub starts 2024-09-16T09:36:59.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:59 smithi070 ceph-mon[30388]: 6.14 scrub ok 2024-09-16T09:36:59.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:59 smithi070 ceph-mon[30388]: osdmap e338: 8 total, 8 up, 8 in 2024-09-16T09:36:59.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:59 smithi070 ceph-mon[30388]: from='client.25760 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:36:59.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:59 smithi070 ceph-mon[30388]: osdmap e339: 8 total, 8 up, 8 in 2024-09-16T09:36:59.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:59 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "19"} : dispatch 2024-09-16T09:36:59.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:36:59 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "19"} : dispatch 2024-09-16T09:36:59.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:59 smithi070 ceph-mon[40691]: 6.14 scrub starts 2024-09-16T09:36:59.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:59 smithi070 ceph-mon[40691]: 6.14 scrub ok 2024-09-16T09:36:59.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:59 smithi070 ceph-mon[40691]: osdmap e338: 8 total, 8 up, 8 in 2024-09-16T09:36:59.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:59 smithi070 ceph-mon[40691]: from='client.25760 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:36:59.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:59 smithi070 ceph-mon[40691]: osdmap e339: 8 total, 8 up, 8 in 2024-09-16T09:36:59.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:59 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "19"} : dispatch 2024-09-16T09:36:59.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:36:59 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "19"} : dispatch 2024-09-16T09:36:59.962 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:37:00.008 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:00.009 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:00.009 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:37:00.110 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:37:00.139 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:00.140 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:00.140 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:37:00.160 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T09:37:00.160 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:37:00.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:59 smithi084 ceph-mon[32629]: 6.14 scrub starts 2024-09-16T09:37:00.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:59 smithi084 ceph-mon[32629]: 6.14 scrub ok 2024-09-16T09:37:00.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:59 smithi084 ceph-mon[32629]: osdmap e338: 8 total, 8 up, 8 in 2024-09-16T09:37:00.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:59 smithi084 ceph-mon[32629]: from='client.25760 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:37:00.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:59 smithi084 ceph-mon[32629]: osdmap e339: 8 total, 8 up, 8 in 2024-09-16T09:37:00.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:59 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "19"} : dispatch 2024-09-16T09:37:00.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:36:59 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "19"} : dispatch 2024-09-16T09:37:00.184 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T09:37:00.241 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:37:00.284 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:00.285 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:00.285 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:37:00.320 INFO:teuthology.orchestra.run.smithi070.stderr:noscrub is set 2024-09-16T09:37:00.387 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:37:00.416 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:00.417 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:00.417 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:37:00.518 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:37:00.540 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:00.541 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:00.542 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:37:00.642 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:37:00.664 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:00.664 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:00.665 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:37:00.765 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:37:00.787 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:00.788 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:00.788 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:37:00.889 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:37:00.897 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:00 smithi070 ceph-mon[30388]: 5.b scrub starts 2024-09-16T09:37:00.897 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:00 smithi070 ceph-mon[30388]: 5.b scrub ok 2024-09-16T09:37:00.897 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:00 smithi070 ceph-mon[30388]: 6.b scrub starts 2024-09-16T09:37:00.897 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:00 smithi070 ceph-mon[30388]: 6.b scrub ok 2024-09-16T09:37:00.898 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:00 smithi070 ceph-mon[30388]: pgmap v613: 172 pgs: 1 clean+premerge+peered, 171 active+clean; 1.6 GiB data, 9.6 GiB used, 706 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 457 op/s 2024-09-16T09:37:00.898 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:00 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1757875044' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:37:00.898 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:00 smithi070 ceph-mon[30388]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:37:00.898 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:00 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "19"}]': finished 2024-09-16T09:37:00.898 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:00 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1757875044' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-16T09:37:00.898 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:00 smithi070 ceph-mon[30388]: osdmap e340: 8 total, 8 up, 8 in 2024-09-16T09:37:00.898 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:00 smithi070 ceph-mon[40691]: 5.b scrub starts 2024-09-16T09:37:00.899 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:00 smithi070 ceph-mon[40691]: 5.b scrub ok 2024-09-16T09:37:00.899 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:00 smithi070 ceph-mon[40691]: 6.b scrub starts 2024-09-16T09:37:00.899 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:00 smithi070 ceph-mon[40691]: 6.b scrub ok 2024-09-16T09:37:00.899 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:00 smithi070 ceph-mon[40691]: pgmap v613: 172 pgs: 1 clean+premerge+peered, 171 active+clean; 1.6 GiB data, 9.6 GiB used, 706 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 457 op/s 2024-09-16T09:37:00.899 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:00 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1757875044' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:37:00.899 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:00 smithi070 ceph-mon[40691]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:37:00.899 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:00 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "19"}]': finished 2024-09-16T09:37:00.899 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:00 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1757875044' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-16T09:37:00.899 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:00 smithi070 ceph-mon[40691]: osdmap e340: 8 total, 8 up, 8 in 2024-09-16T09:37:00.911 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:00.911 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:00.912 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:37:01.012 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:37:01.034 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:01.034 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:01.035 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:37:01.136 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:37:01.157 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:01.157 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:01.158 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:37:01.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:00 smithi084 ceph-mon[32629]: 5.b scrub starts 2024-09-16T09:37:01.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:00 smithi084 ceph-mon[32629]: 5.b scrub ok 2024-09-16T09:37:01.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:00 smithi084 ceph-mon[32629]: 6.b scrub starts 2024-09-16T09:37:01.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:00 smithi084 ceph-mon[32629]: 6.b scrub ok 2024-09-16T09:37:01.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:00 smithi084 ceph-mon[32629]: pgmap v613: 172 pgs: 1 clean+premerge+peered, 171 active+clean; 1.6 GiB data, 9.6 GiB used, 706 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 457 op/s 2024-09-16T09:37:01.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:00 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1757875044' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:37:01.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:00 smithi084 ceph-mon[32629]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:37:01.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:00 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "19"}]': finished 2024-09-16T09:37:01.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:00 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1757875044' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-16T09:37:01.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:00 smithi084 ceph-mon[32629]: osdmap e340: 8 total, 8 up, 8 in 2024-09-16T09:37:01.259 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:37:01.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:01.281 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:01.282 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:37:01.299 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":611,"stamp":"2024-09-16T09:36:58.765393+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1697716787,"num_objects":26096,"num_object_clones":0,"num_object_copies":78288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26096,"num_whiteouts":0,"num_read":1852,"num_read_kb":1604,"num_write":141159,"num_write_kb":3816942,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23212,"num_bytes_recovered":1478562708,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76844,"ondisk_log_size":76844,"up":516,"acting":516,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":531,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":10149516,"kb_used_data":6275152,"kb_used_omap":12,"kb_used_meta":3874163,"kb_avail":739811700,"statfs":{"total":767960285184,"available":757567180800,"internally_reserved":0,"allocated":6425755648,"data_stored":6422511987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":3967143474},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,4,2,1,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-157679616,"num_objects":-2406,"num_object_clones":0,"num_object_copies":-7218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-2406,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":-526,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.031883"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":597,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-09-16T09:36:53.733971+0000","last_change":"2024-09-16T09:32:39.384174+0000","last_active":"2024-09-16T09:36:53.733971+0000","last_peered":"2024-09-16T09:36:53.733971+0000","last_clean":"2024-09-16T09:36:53.733971+0000","last_became_active":"2024-09-16T09:25:01.709108+0000","last_became_peered":"2024-09-16T09:25:01.709108+0000","last_unstale":"2024-09-16T09:36:53.733971+0000","last_undegraded":"2024-09-16T09:36:53.733971+0000","last_fullsized":"2024-09-16T09:36:53.733971+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:39.384104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:32:39.384104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"285'15","reported_seq":617,"reported_epoch":337,"state":"active+clean","last_fresh":"2024-09-16T09:36:57.763882+0000","last_change":"2024-09-16T09:32:38.074301+0000","last_active":"2024-09-16T09:36:57.763882+0000","last_peered":"2024-09-16T09:36:57.763882+0000","last_clean":"2024-09-16T09:36:57.763882+0000","last_became_active":"2024-09-16T09:25:03.714635+0000","last_became_peered":"2024-09-16T09:25:03.714635+0000","last_unstale":"2024-09-16T09:36:57.763882+0000","last_undegraded":"2024-09-16T09:36:57.763882+0000","last_fullsized":"2024-09-16T09:36:57.763882+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:32:38.074231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:32:38.074231+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":228,"reported_epoch":337,"state":"active+clean","last_fresh":"2024-09-16T09:36:58.027859+0000","last_change":"2024-09-16T09:36:53.098680+0000","last_active":"2024-09-16T09:36:58.027859+0000","last_peered":"2024-09-16T09:36:58.027859+0000","last_clean":"2024-09-16T09:36:58.027859+0000","last_became_active":"2024-09-16T09:33:30.242238+0000","last_became_peered":"2024-09-16T09:33:30.242238+0000","last_unstale":"2024-09-16T09:36:58.027859+0000","last_undegraded":"2024-09-16T09:36:58.027859+0000","last_fullsized":"2024-09-16T09:36:58.027859+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:53.098635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:36:53.098635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:03.340158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":576,"reported_epoch":337,"state":"active+clean","last_fresh":"2024-09-16T09:36:57.764366+0000","last_change":"2024-09-16T09:35:11.876996+0000","last_active":"2024-09-16T09:36:57.764366+0000","last_peered":"2024-09-16T09:36:57.764366+0000","last_clean":"2024-09-16T09:36:57.764366+0000","last_became_active":"2024-09-16T09:25:07.715442+0000","last_became_peered":"2024-09-16T09:25:07.715442+0000","last_unstale":"2024-09-16T09:36:57.764366+0000","last_undegraded":"2024-09-16T09:36:57.764366+0000","last_fullsized":"2024-09-16T09:36:57.764366+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:35:11.876955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:35:11.876955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":322,"reported_epoch":337,"state":"active+clean","last_fresh":"2024-09-16T09:36:57.760496+0000","last_change":"2024-09-16T09:34:33.452784+0000","last_active":"2024-09-16T09:36:57.760496+0000","last_peered":"2024-09-16T09:36:57.760496+0000","last_clean":"2024-09-16T09:36:57.760496+0000","last_became_active":"2024-09-16T09:31:09.269342+0000","last_became_peered":"2024-09-16T09:31:09.269342+0000","last_unstale":"2024-09-16T09:36:57.760496+0000","last_undegraded":"2024-09-16T09:36:57.760496+0000","last_fullsized":"2024-09-16T09:36:57.760496+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:33.452706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:34:33.452706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":461,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-16T09:36:56.769660+0000","last_change":"2024-09-16T09:33:08.816231+0000","last_active":"2024-09-16T09:36:56.769660+0000","last_peered":"2024-09-16T09:36:56.769660+0000","last_clean":"2024-09-16T09:36:56.769660+0000","last_became_active":"2024-09-16T09:31:09.275136+0000","last_became_peered":"2024-09-16T09:31:09.275136+0000","last_unstale":"2024-09-16T09:36:56.769660+0000","last_undegraded":"2024-09-16T09:36:56.769660+0000","last_fullsized":"2024-09-16T09:36:56.769660+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:08.816174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:08.816174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.19","version":"285'27","reported_seq":287,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-09-16T09:36:53.734246+0000","last_change":"2024-09-16T09:33:21.337711+0000","last_active":"2024-09-16T09:36:53.734246+0000","last_peered":"2024-09-16T09:36:53.734246+0000","last_clean":"2024-09-16T09:36:53.734246+0000","last_became_active":"2024-09-16T09:33:21.337284+0000","last_became_peered":"2024-09-16T09:33:21.337284+0000","last_unstale":"2024-09-16T09:36:53.734246+0000","last_undegraded":"2024-09-16T09:36:53.734246+0000","last_fullsized":"2024-09-16T09:36:53.734246+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-16T09:32:45.194581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:32:45.194581+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":596,"reported_epoch":337,"state":"active+clean","last_fresh":"2024-09-16T09:36:57.763758+0000","last_change":"2024-09-16T09:32:44.114475+0000","last_active":"2024-09-16T09:36:57.763758+0000","last_peered":"2024-09-16T09:36:57.763758+0000","last_clean":"2024-09-16T09:36:57.763758+0000","last_became_active":"2024-09-16T09:25:05.717236+0000","last_became_peered":"2024-09-16T09:25:05.717236+0000","last_unstale":"2024-09-16T09:36:57.763758+0000","last_undegraded":"2024-09-16T09:36:57.763758+0000","last_fullsized":"2024-09-16T09:36:57.763758+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:44.114407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:32:44.114407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":221,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-09-16T09:36:53.733721+0000","last_change":"2024-09-16T09:33:36.655594+0000","last_active":"2024-09-16T09:36:53.733721+0000","last_peered":"2024-09-16T09:36:53.733721+0000","last_clean":"2024-09-16T09:36:53.733721+0000","last_became_active":"2024-09-16T09:33:30.247733+0000","last_became_peered":"2024-09-16T09:33:30.247733+0000","last_unstale":"2024-09-16T09:36:53.733721+0000","last_undegraded":"2024-09-16T09:36:53.733721+0000","last_fullsized":"2024-09-16T09:36:53.733721+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:36.655236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:36.655236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1a","version":"285'15","reported_seq":617,"reported_epoch":337,"state":"active+clean","last_fresh":"2024-09-16T09:36:57.764646+0000","last_change":"2024-09-16T09:32:48.185484+0000","last_active":"2024-09-16T09:36:57.764646+0000","last_peered":"2024-09-16T09:36:57.764646+0000","last_clean":"2024-09-16T09:36:57.764646+0000","last_became_active":"2024-09-16T09:25:03.709147+0000","last_became_peered":"2024-09-16T09:25:03.709147+0000","last_unstale":"2024-09-16T09:36:57.764646+0000","last_undegraded":"2024-09-16T09:36:57.764646+0000","last_fullsized":"2024-09-16T09:36:57.764646+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:32:48.185380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:32:48.185380+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:36:59.829704+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":587,"reported_epoch":337,"state":"active+clean","last_fresh":"2024-09-16T09:36:58.028185+0000","last_change":"2024-09-16T09:36:51.134364+0000","last_active":"2024-09-16T09:36:58.028185+0000","last_peered":"2024-09-16T09:36:58.028185+0000","last_clean":"2024-09-16T09:36:58.028185+0000","last_became_active":"2024-09-16T09:25:05.718134+0000","last_became_peered":"2024-09-16T09:25:05.718134+0000","last_unstale":"2024-09-16T09:36:58.028185+0000","last_undegraded":"2024-09-16T09:36:58.028185+0000","last_fullsized":"2024-09-16T09:36:58.028185+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:51.134314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:36:51.134314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:37:54.019613+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"66'1","reported_seq":585,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-09-16T09:36:53.732827+0000","last_change":"2024-09-16T09:33:38.705001+0000","last_active":"2024-09-16T09:36:53.732827+0000","last_peered":"2024-09-16T09:36:53.732827+0000","last_clean":"2024-09-16T09:36:53.732827+0000","last_became_active":"2024-09-16T09:25:07.716558+0000","last_became_peered":"2024-09-16T09:25:07.716558+0000","last_unstale":"2024-09-16T09:36:53.732827+0000","last_undegraded":"2024-09-16T09:36:53.732827+0000","last_fullsized":"2024-09-16T09:36:53.732827+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-16T09:33:38.704868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:33:38.704868+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":587,"reported_epoch":337,"state":"active+clean","last_fresh":"2024-09-16T09:36:58.028418+0000","last_change":"2024-09-16T09:33:19.326765+0000","last_active":"2024-09-16T09:36:58.028418+0000","last_peered":"2024-09-16T09:36:58.028418+0000","last_clean":"2024-09-16T09:36:58.028418+0000","last_became_active":"2024-09-16T09:25:01.696939+0000","last_became_peered":"2024-09-16T09:25:01.696939+0000","last_unstale":"2024-09-16T09:36:58.028418+0000","last_undegraded":"2024-09-16T09:36:58.028418+0000","last_fullsized":"2024-09-16T09:36:58.028418+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:19.326695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:19.326695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"285'7","reported_seq":605,"reported_epoch":337,"state":"active+clean","last_fresh":"2024-09-16T09:36:57.764106+0000","last_change":"2024-09-16T09:35:39.975915+0000","last_active":"2024-09-16T09:36:57.764106+0000","last_peered":"2024-09-16T09:36:57.764106+0000","last_clean":"2024-09-16T09:36:57.764106+0000","last_became_active":"2024-09-16T09:25:03.716136+0000","last_became_peered":"2024-09-16T09:25:03.716136+0000","last_unstale":"2024-09-16T09:36:57.764106+0000","last_undegraded":"2024-09-16T09:36:57.764106+0000","last_fullsized":"2024-09-16T09:36:57.764106+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'7","last_scrub_stamp":"2024-09-16T09:35:39.975840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:35:39.975840+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":10,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":577,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-16T09:36:56.761030+0000","last_change":"2024-09-16T09:28:52.746046+0000","last_active":"2024-09-16T09:36:56.761030+0000","last_peered":"2024-09-16T09:36:56.761030+0000","last_clean":"2024-09-16T09:36:56.761030+0000","last_became_active":"2024-09-16T09:25:05.718281+0000","last_became_peered":"2024-09-16T09:25:05.718281+0000","last_unstale":"2024-09-16T09:36:56.761030+0000","last_undegraded":"2024-09-16T09:36:56.761030+0000","last_fullsized":"2024-09-16T09:36:56.761030+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:52.745999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:28:52.745999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":605,"reported_epoch":337,"state":"active+clean","last_fresh":"2024-09-16T09:36:58.027886+0000","last_change":"2024-09-16T09:33:46.366042+0000","last_active":"2024-09-16T09:36:58.027886+0000","last_peered":"2024-09-16T09:36:58.027886+0000","last_clean":"2024-09-16T09:36:58.027886+0000","last_became_active":"2024-09-16T09:31:09.275656+0000","last_became_peered":"2024-09-16T09:31:09.275656+0000","last_unstale":"2024-09-16T09:36:58.027886+0000","last_undegraded":"2024-09-16T09:36:58.027886+0000","last_fullsized":"2024-09-16T09:36:58.027886+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:46.365972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:33:46.365972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":73,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"336'8235","reported_seq":8550,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-16T09:36:56.895100+0000","last_change":"2024-09-16T09:35:38.622217+0000","last_active":"2024-09-16T09:36:56.895100+0000","last_peered":"2024-09-16T09:36:56.895100+0000","last_clean":"2024-09-16T09:36:56.895100+0000","last_became_active":"2024-09-16T09:35:05.670682+0000","last_became_peered":"2024-09-16T09:35:05.670682+0000","last_unstale":"2024-09-16T09:36:56.895100+0000","last_undegraded":"2024-09-16T09:36:56.895100+0000","last_fullsized":"2024-09-16T09:36:56.895100+0000","mapping_epoch":282,"log_start":"248'3240","ondisk_log_start":"248'3240","created":243,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":5,"last_scrub":"291'6207","last_scrub_stamp":"2024-09-16T09:35:38.621911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:35:38.621911+0000","objects_scrubbed":2873,"log_size":4995,"log_dups_size":0,"ondisk_log_size":4995,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":752,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54722560,"num_objects":835,"num_object_clones":0,"num_object_copies":2505,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":835,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2038,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":593,"reported_epoch":337,"state":"active+clean","last_fresh":"2024-09-16T09:36:57.765025+0000","last_change":"2024-09-16T09:33:48.322010+0000","last_active":"2024-09-16T09:36:57.765025+0000","last_peered":"2024-09-16T09:36:57.765025+0000","last_clean":"2024-09-16T09:36:57.765025+0000","last_became_active":"2024-09-16T09:25:07.720793+0000","last_became_peered":"2024-09-16T09:25:07.720793+0000","last_unstale":"2024-09-16T09:36:57.765025+0000","last_undegraded":"2024-09-16T09:36:57.765025+0000","last_fullsized":"2024-09-16T09:36:57.765025+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:48.321895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:33:48.321895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:37:26.677312+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":342,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-16T09:36:56.770094+0000","last_change":"2024-09-16T09:34:33.866299+0000","last_active":"2024-09-16T09:36:56.770094+0000","last_peered":"2024-09-16T09:36:56.770094+0000","last_clean":"2024-09-16T09:36:56.770094+0000","last_became_active":"2024-09-16T09:30:39.201824+0000","last_became_peered":"2024-09-16T09:30:39.201824+0000","last_unstale":"2024-09-16T09:36:56.770094+0000","last_undegraded":"2024-09-16T09:36:56.770094+0000","last_fullsized":"2024-09-16T09:36:56.770094+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:33.866142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:34:33.866142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"285'27","reported_seq":690,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-16T09:36:56.763230+0000","last_change":"2024-09-16T09:36:54.484021+0000","last_active":"2024-09-16T09:36:56.763230+0000","last_peered":"2024-09-16T09:36:56.763230+0000","last_clean":"2024-09-16T09:36:56.763230+0000","last_became_active":"2024-09-16T09:33:30.249798+0000","last_became_peered":"2024-09-16T09:33:30.249798+0000","last_unstale":"2024-09-16T09:36:56.763230+0000","last_undegraded":"2024-09-16T09:36:56.763230+0000","last_fullsized":"2024-09-16T09:36:56.763230+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'27","last_scrub_stamp":"2024-09-16T09:36:54.483966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:36:54.483966+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:05.562379+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":228,"reported_epoch":337,"state":"active+clean","last_fresh":"2024-09-16T09:36:57.764673+0000","last_change":"2024-09-16T09:33:43.326933+0000","last_active":"2024-09-16T09:36:57.764673+0000","last_peered":"2024-09-16T09:36:57.764673+0000","last_clean":"2024-09-16T09:36:57.764673+0000","last_became_active":"2024-09-16T09:33:30.251587+0000","last_became_peered":"2024-09-16T09:33:30.251587+0000","last_unstale":"2024-09-16T09:36:57.764673+0000","last_undegraded":"2024-09-16T09:36:57.764673+0000","last_fullsized":"2024-09-16T09:36:57.764673+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:43.326852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:43.326852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:37:29.639718+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,0],"acting":[4,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.10","version":"336'8344","reported_seq":27590,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-16T09:36:56.920002+0000","last_change":"2024-09-16T09:36:50.039921+0000","last_active":"2024-09-16T09:36:56.920002+0000","last_peered":"2024-09-16T09:36:56.920002+0000","last_clean":"2024-09-16T09:36:56.920002+0000","last_became_active":"2024-09-16T09:36:49.314768+0000","last_became_peered":"2024-09-16T09:36:49.314768+0000","last_unstale":"2024-09-16T09:36:56.920002+0000","last_undegraded":"2024-09-16T09:36:56.920002+0000","last_fullsized":"2024-09-16T09:36:56.920002+0000","mapping_epoch":328,"log_start":"263'4125","ondisk_log_start":"263'4125","created":243,"last_epoch_clean":329,"parent":"0.0","parent_split_bits":5,"last_scrub":"329'8188","last_scrub_stamp":"2024-09-16T09:36:50.039825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:36:50.039825+0000","objects_scrubbed":922,"log_size":4219,"log_dups_size":0,"ondisk_log_size":4219,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:37:54.291700+0000","scrub_duration":104,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50200576,"num_objects":766,"num_object_clones":0,"num_object_copies":2298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":766,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":156,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,2],"acting":[6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":439,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-16T09:36:56.769857+0000","last_change":"2024-09-16T09:34:37.804874+0000","last_active":"2024-09-16T09:36:56.769857+0000","last_peered":"2024-09-16T09:36:56.769857+0000","last_clean":"2024-09-16T09:36:56.769857+0000","last_became_active":"2024-09-16T09:28:52.974255+0000","last_became_peered":"2024-09-16T09:28:52.974255+0000","last_unstale":"2024-09-16T09:36:56.769857+0000","last_undegraded":"2024-09-16T09:36:56.769857+0000","last_fullsized":"2024-09-16T09:36:56.769857+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:37.804805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:34:37.804805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":604,"reported_epoch":337,"state":"active+clean","last_fresh":"2024-09-16T09:36:57.764755+0000","last_change":"2024-09-16T09:35:23.751992+0000","last_active":"2024-09-16T09:36:57.764755+0000","last_peered":"2024-09-16T09:36:57.764755+0000","last_clean":"2024-09-16T09:36:57.764755+0000","last_became_active":"2024-09-16T09:25:01.706491+0000","last_became_peered":"2024-09-16T09:25:01.706491+0000","last_unstale":"2024-09-16T09:36:57.764755+0000","last_undegraded":"2024-09-16T09:36:57.764755+0000","last_fullsized":"2024-09-16T09:36:57.764755+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:35:23.751946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:35:23.751946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"285'18","reported_seq":280,"reported_epoch":337,"state":"active+clean","last_fresh":"2024-09-16T09:36:57.764812+0000","last_change":"2024-09-16T09:33:39.284526+0000","last_active":"2024-09-16T09:36:57.764812+0000","last_peered":"2024-09-16T09:36:57.764812+0000","last_clean":"2024-09-16T09:36:57.764812+0000","last_became_active":"2024-09-16T09:33:30.251626+0000","last_became_peered":"2024-09-16T09:33:30.251626+0000","last_unstale":"2024-09-16T09:36:57.764812+0000","last_undegraded":"2024-09-16T09:36:57.764812+0000","last_fullsized":"2024-09-16T09:36:57.764812+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-16T09:33:39.284451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:33:39.284451+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:37:14.798648+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":33,"num_read_kb":24,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":580,"reported_epoch":337,"state":"active+clean","last_fresh":"2024-09-16T09:36:57.764490+0000","last_change":"2024-09-16T09:33:38.238661+0000","last_active":"2024-09-16T09:36:57.764490+0000","last_peered":"2024-09-16T09:36:57.764490+0000","last_clean":"2024-09-16T09:36:57.764490+0000","last_became_active":"2024-09-16T09:25:05.710093+0000","last_became_peered":"2024-09-16T09:25:05.710093+0000","last_unstale":"2024-09-16T09:36:57.764490+0000","last_undegraded":"2024-09-16T09:36:57.764490+0000","last_fullsized":"2024-09-16T09:36:57.764490+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:38.238593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:38.238593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:37:22.765467+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"336'8277","reported_seq":18946,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-16T09:36:56.822603+0000","last_change":"2024-09-16T09:36:27.532320+0000","last_active":"2024-09-16T09:36:56.822603+0000","last_peered":"2024-09-16T09:36:56.822603+0000","last_clean":"2024-09-16T09:36:56.822603+0000","last_became_active":"2024-09-16T09:36:27.519720+0000","last_became_peered":"2024-09-16T09:36:27.519720+0000","last_unstale":"2024-09-16T09:36:56.822603+0000","last_undegraded":"2024-09-16T09:36:56.822603+0000","last_fullsized":"2024-09-16T09:36:56.822603+0000","mapping_epoch":314,"log_start":"268'4264","ondisk_log_start":"268'4264","created":243,"last_epoch_clean":315,"parent":"0.0","parent_split_bits":5,"last_scrub":"273'4719","last_scrub_stamp":"2024-09-16T09:34:44.662413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:34:44.662413+0000","objects_scrubbed":3181,"log_size":4013,"log_dups_size":0,"ondisk_log_size":4013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"queued for scrub","scrub_duration":1093,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":55312384,"num_objects":844,"num_object_clones":0,"num_object_copies":2532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":844,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3579,"num_write_kb":39744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1861,"num_bytes_recovered":121962496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":582,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-16T09:36:56.761456+0000","last_change":"2024-09-16T09:33:46.253182+0000","last_active":"2024-09-16T09:36:56.761456+0000","last_peered":"2024-09-16T09:36:56.761456+0000","last_clean":"2024-09-16T09:36:56.761456+0000","last_became_active":"2024-09-16T09:25:07.719380+0000","last_became_peered":"2024-09-16T09:25:07.719380+0000","last_unstale":"2024-09-16T09:36:56.761456+0000","last_undegraded":"2024-09-16T09:36:56.761456+0000","last_fullsized":"2024-09-16T09:36:56.761456+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-16T09:33:46.253114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:33:46.253114+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":237,"reported_epoch":337,"state":"active+clean","last_fresh":"2024-09-16T09:36:57.761789+0000","last_change":"2024-09-16T09:34:41.401691+0000","last_active":"2024-09-16T09:36:57.761789+0000","last_peered":"2024-09-16T09:36:57.761789+0000","last_clean":"2024-09-16T09:36:57.761789+0000","last_became_active":"2024-09-16T09:33:30.245048+0000","last_became_peered":"2024-09-16T09:33:30.245048+0000","last_unstale":"2024-09-16T09:36:57.761789+0000","last_undegraded":"2024-09-16T09:36:57.761789+0000","last_fullsized":"2024-09-16T09:36:57.761789+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:34:41.401641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:34:41.401641+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"285'20","reported_seq":653,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-09-16T09:36:53.733494+0000","last_change":"2024-09-16T09:32:30.286077+0000","last_active":"2024-09-16T09:36:53.733494+0000","last_peered":"2024-09-16T09:36:53.733494+0000","last_clean":"2024-09-16T09:36:53.733494+0000","last_became_active":"2024-09-16T09:31:09.275050+0000","last_became_peered":"2024-09-16T09:31:09.275050+0000","last_unstale":"2024-09-16T09:36:53.733494+0000","last_undegraded":"2024-09-16T09:36:53.733494+0000","last_fullsized":"2024-09-16T09:36:53.733494+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-16T09:32:30.284919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:32:30.284919+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"66'8","reported_seq":766,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-16T09:36:56.770032+0000","last_change":"2024-09-16T09:34:52.922123+0000","last_active":"2024-09-16T09:36:56.770032+0000","last_peered":"2024-09-16T09:36:56.770032+0000","last_clean":"2024-09-16T09:36:56.770032+0000","last_became_active":"2024-09-16T09:25:05.712516+0000","last_became_peered":"2024-09-16T09:25:05.712516+0000","last_unstale":"2024-09-16T09:36:56.770032+0000","last_undegraded":"2024-09-16T09:36:56.770032+0000","last_fullsized":"2024-09-16T09:36:56.770032+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:34:52.922047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:34:52.922047+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":33,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2","version":"336'8272","reported_seq":8555,"reported_epoch":337,"state":"active+clean","last_fresh":"2024-09-16T09:36:57.760255+0000","last_change":"2024-09-16T09:36:56.054414+0000","last_active":"2024-09-16T09:36:57.760255+0000","last_peered":"2024-09-16T09:36:57.760255+0000","last_clean":"2024-09-16T09:36:57.760255+0000","last_became_active":"2024-09-16T09:33:52.933886+0000","last_became_peered":"2024-09-16T09:33:52.933886+0000","last_unstale":"2024-09-16T09:36:57.760255+0000","last_undegraded":"2024-09-16T09:36:57.760255+0000","last_fullsized":"2024-09-16T09:36:57.760255+0000","mapping_epoch":233,"log_start":"269'4326","ondisk_log_start":"269'4326","created":229,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"335'8228","last_scrub_stamp":"2024-09-16T09:36:56.054247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:36:56.054247+0000","objects_scrubbed":907,"log_size":3946,"log_dups_size":0,"ondisk_log_size":3946,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:13.015343+0000","scrub_duration":287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56033280,"num_objects":855,"num_object_clones":0,"num_object_copies":2565,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":855,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5106,"num_write_kb":139776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.a","version":"73'79","reported_seq":519,"reported_epoch":337,"state":"active+clean","last_fresh":"2024-09-16T09:36:58.027822+0000","last_change":"2024-09-16T09:29:18.498887+0000","last_active":"2024-09-16T09:36:58.027822+0000","last_peered":"2024-09-16T09:36:58.027822+0000","last_clean":"2024-09-16T09:36:58.027822+0000","last_became_active":"2024-09-16T09:26:35.127352+0000","last_became_peered":"2024-09-16T09:26:35.127352+0000","last_unstale":"2024-09-16T09:36:58.027822+0000","last_undegraded":"2024-09-16T09:36:58.027822+0000","last_fullsized":"2024-09-16T09:36:58.027822+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:18.498766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:18.498766+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:37:33.100091+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"285'27","reported_seq":552,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-16T09:36:56.762706+0000","last_change":"2024-09-16T09:36:47.588508+0000","last_active":"2024-09-16T09:36:56.762706+0000","last_peered":"2024-09-16T09:36:56.762706+0000","last_clean":"2024-09-16T09:36:56.762706+0000","last_became_active":"2024-09-16T09:26:33.295672+0000","last_became_peered":"2024-09-16T09:26:33.295672+0000","last_unstale":"2024-09-16T09:36:56.762706+0000","last_undegraded":"2024-09-16T09:36:56.762706+0000","last_fullsized":"2024-09-16T09:36:56.762706+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'27","last_scrub_stamp":"2024-09-16T09:36:47.588449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:36:47.588449+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:14.427263+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":322,"reported_epoch":337,"state":"active+clean","last_fresh":"2024-09-16T09:36:57.759997+0000","last_change":"2024-09-16T09:34:39.420386+0000","last_active":"2024-09-16T09:36:57.759997+0000","last_peered":"2024-09-16T09:36:57.759997+0000","last_clean":"2024-09-16T09:36:57.759997+0000","last_became_active":"2024-09-16T09:31:09.274534+0000","last_became_peered":"2024-09-16T09:31:09.274534+0000","last_unstale":"2024-09-16T09:36:57.759997+0000","last_undegraded":"2024-09-16T09:36:57.759997+0000","last_fullsized":"2024-09-16T09:36:57.759997+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:39.420314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:34:39.420314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"66'8","reported_seq":725,"reported_epoch":337,"state":"active+clean","last_fresh":"2024-09-16T09:36:57.765015+0000","last_change":"2024-09-16T09:35:10.915838+0000","last_active":"2024-09-16T09:36:57.765015+0000","last_peered":"2024-09-16T09:36:57.765015+0000","last_clean":"2024-09-16T09:36:57.765015+0000","last_became_active":"2024-09-16T09:25:05.713813+0000","last_became_peered":"2024-09-16T09:25:05.713813+0000","last_unstale":"2024-09-16T09:36:57.765015+0000","last_undegraded":"2024-09-16T09:36:57.765015+0000","last_fullsized":"2024-09-16T09:36:57.765015+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:35:10.915780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:35:10.915780+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:37:32.658273+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":584,"reported_epoch":337,"state":"active+clean","last_fresh":"2024-09-16T09:36:58.028307+0000","last_change":"2024-09-16T09:36:37.152446+0000","last_active":"2024-09-16T09:36:58.028307+0000","last_peered":"2024-09-16T09:36:58.028307+0000","last_clean":"2024-09-16T09:36:58.028307+0000","last_became_active":"2024-09-16T09:25:07.717344+0000","last_became_peered":"2024-09-16T09:25:07.717344+0000","last_unstale":"2024-09-16T09:36:58.028307+0000","last_undegraded":"2024-09-16T09:36:58.028307+0000","last_fullsized":"2024-09-16T09:36:58.028307+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:37.152377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:36:37.152377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:37:38.943155+0000","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"336'13505","reported_seq":20864,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-16T09:36:56.903588+0000","last_change":"2024-09-16T09:35:09.710620+0000","last_active":"2024-09-16T09:36:56.903588+0000","last_peered":"2024-09-16T09:36:56.903588+0000","last_clean":"2024-09-16T09:36:56.903588+0000","last_became_active":"2024-09-16T09:35:09.702658+0000","last_became_peered":"2024-09-16T09:35:09.702658+0000","last_unstale":"2024-09-16T09:36:56.903588+0000","last_undegraded":"2024-09-16T09:36:56.903588+0000","last_fullsized":"2024-09-16T09:36:56.903588+0000","mapping_epoch":285,"log_start":"291'9500","ondisk_log_start":"291'9500","created":229,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:33:24.230486+0000","objects_scrubbed":0,"log_size":4005,"log_dups_size":0,"ondisk_log_size":4005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109707288,"num_objects":1675,"num_object_clones":0,"num_object_copies":5025,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1675,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13505,"num_write_kb":485697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3554,"num_bytes_recovered":232914944,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.5","version":"73'79","reported_seq":559,"reported_epoch":337,"state":"active+clean","last_fresh":"2024-09-16T09:36:57.765165+0000","last_change":"2024-09-16T09:32:42.123139+0000","last_active":"2024-09-16T09:36:57.765165+0000","last_peered":"2024-09-16T09:36:57.765165+0000","last_clean":"2024-09-16T09:36:57.765165+0000","last_became_active":"2024-09-16T09:30:08.763683+0000","last_became_peered":"2024-09-16T09:30:08.763683+0000","last_unstale":"2024-09-16T09:36:57.765165+0000","last_undegraded":"2024-09-16T09:36:57.765165+0000","last_fullsized":"2024-09-16T09:36:57.765165+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:32:42.123067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:32:42.123067+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:36:57.873747+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"285'30","reported_seq":551,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-16T09:36:56.761693+0000","last_change":"2024-09-16T09:33:21.333996+0000","last_active":"2024-09-16T09:36:56.761693+0000","last_peered":"2024-09-16T09:36:56.761693+0000","last_clean":"2024-09-16T09:36:56.761693+0000","last_became_active":"2024-09-16T09:31:09.270434+0000","last_became_peered":"2024-09-16T09:31:09.270434+0000","last_unstale":"2024-09-16T09:36:56.761693+0000","last_undegraded":"2024-09-16T09:36:56.761693+0000","last_fullsized":"2024-09-16T09:36:56.761693+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'18","last_scrub_stamp":"2024-09-16T09:33:21.333937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:33:21.333937+0000","objects_scrubbed":8,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":51,"num_read_kb":39,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":430,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-16T09:36:56.761587+0000","last_change":"2024-09-16T09:36:50.622852+0000","last_active":"2024-09-16T09:36:56.761587+0000","last_peered":"2024-09-16T09:36:56.761587+0000","last_clean":"2024-09-16T09:36:56.761587+0000","last_became_active":"2024-09-16T09:28:52.983543+0000","last_became_peered":"2024-09-16T09:28:52.983543+0000","last_unstale":"2024-09-16T09:36:56.761587+0000","last_undegraded":"2024-09-16T09:36:56.761587+0000","last_fullsized":"2024-09-16T09:36:56.761587+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:50.622807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:36:50.622807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:37:54.700623+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":611,"reported_epoch":337,"state":"active+clean","last_fresh":"2024-09-16T09:36:57.763943+0000","last_change":"2024-09-16T09:35:19.822713+0000","last_active":"2024-09-16T09:36:57.763943+0000","last_peered":"2024-09-16T09:36:57.763943+0000","last_clean":"2024-09-16T09:36:57.763943+0000","last_became_active":"2024-09-16T09:31:09.268864+0000","last_became_peered":"2024-09-16T09:31:09.268864+0000","last_unstale":"2024-09-16T09:36:57.763943+0000","last_undegraded":"2024-09-16T09:36:57.763943+0000","last_fullsized":"2024-09-16T09:36:57.763943+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:35:19.822635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:35:19.822635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":599,"reported_epoch":337,"state":"active+clean","last_fresh":"2024-09-16T09:36:57.764626+0000","last_change":"2024-09-16T09:35:46.933987+0000","last_active":"2024-09-16T09:36:57.764626+0000","last_peered":"2024-09-16T09:36:57.764626+0000","last_clean":"2024-09-16T09:36:57.764626+0000","last_became_active":"2024-09-16T09:31:09.269433+0000","last_became_peered":"2024-09-16T09:31:09.269433+0000","last_unstale":"2024-09-16T09:36:57.764626+0000","last_undegraded":"2024-09-16T09:36:57.764626+0000","last_fullsized":"2024-09-16T09:36:57.764626+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:35:46.933912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:35:46.933912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:37:05.460732+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.c","version":"336'13331","reported_seq":13603,"reported_epoch":337,"state":"active+clean","last_fresh":"2024-09-16T09:36:57.764650+0000","last_change":"2024-09-16T09:36:41.907258+0000","last_active":"2024-09-16T09:36:57.764650+0000","last_peered":"2024-09-16T09:36:57.764650+0000","last_clean":"2024-09-16T09:36:57.764650+0000","last_became_active":"2024-09-16T09:33:25.371230+0000","last_became_peered":"2024-09-16T09:33:25.371230+0000","last_unstale":"2024-09-16T09:36:57.764650+0000","last_undegraded":"2024-09-16T09:36:57.764650+0000","last_fullsized":"2024-09-16T09:36:57.764650+0000","mapping_epoch":229,"log_start":"291'9100","ondisk_log_start":"291'9100","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"320'12934","last_scrub_stamp":"2024-09-16T09:36:41.907153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:36:41.907153+0000","objects_scrubbed":2082,"log_size":4231,"log_dups_size":0,"ondisk_log_size":4231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:07.793630+0000","scrub_duration":229,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110428160,"num_objects":1685,"num_object_clones":0,"num_object_copies":5055,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1685,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13331,"num_write_kb":480512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"73'79","reported_seq":322,"reported_epoch":337,"state":"active+clean","last_fresh":"2024-09-16T09:36:57.762507+0000","last_change":"2024-09-16T09:34:42.362203+0000","last_active":"2024-09-16T09:36:57.762507+0000","last_peered":"2024-09-16T09:36:57.762507+0000","last_clean":"2024-09-16T09:36:57.762507+0000","last_became_active":"2024-09-16T09:31:09.275139+0000","last_became_peered":"2024-09-16T09:31:09.275139+0000","last_unstale":"2024-09-16T09:36:57.762507+0000","last_undegraded":"2024-09-16T09:36:57.762507+0000","last_fullsized":"2024-09-16T09:36:57.762507+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:34:42.362121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:34:42.362121+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"285'28","reported_seq":643,"reported_epoch":337,"state":"active+clean","last_fresh":"2024-09-16T09:36:57.764073+0000","last_change":"2024-09-16T09:35:43.914378+0000","last_active":"2024-09-16T09:36:57.764073+0000","last_peered":"2024-09-16T09:36:57.764073+0000","last_clean":"2024-09-16T09:36:57.764073+0000","last_became_active":"2024-09-16T09:25:03.708830+0000","last_became_peered":"2024-09-16T09:25:03.708830+0000","last_unstale":"2024-09-16T09:36:57.764073+0000","last_undegraded":"2024-09-16T09:36:57.764073+0000","last_fullsized":"2024-09-16T09:36:57.764073+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'28","last_scrub_stamp":"2024-09-16T09:35:43.914312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:35:43.914312+0000","objects_scrubbed":7,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":606,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-09-16T09:36:53.733768+0000","last_change":"2024-09-16T09:33:43.678231+0000","last_active":"2024-09-16T09:36:53.733768+0000","last_peered":"2024-09-16T09:36:53.733768+0000","last_clean":"2024-09-16T09:36:53.733768+0000","last_became_active":"2024-09-16T09:25:01.702992+0000","last_became_peered":"2024-09-16T09:25:01.702992+0000","last_unstale":"2024-09-16T09:36:53.733768+0000","last_undegraded":"2024-09-16T09:36:53.733768+0000","last_fullsized":"2024-09-16T09:36:53.733768+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:33:43.678138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:43.678138+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"235'12","reported_seq":288,"reported_epoch":337,"state":"active+clean","last_fresh":"2024-09-16T09:36:57.763758+0000","last_change":"2024-09-16T09:33:30.251570+0000","last_active":"2024-09-16T09:36:57.763758+0000","last_peered":"2024-09-16T09:36:57.763758+0000","last_clean":"2024-09-16T09:36:57.763758+0000","last_became_active":"2024-09-16T09:33:30.251047+0000","last_became_peered":"2024-09-16T09:33:30.251047+0000","last_unstale":"2024-09-16T09:36:57.763758+0000","last_undegraded":"2024-09-16T09:36:57.763758+0000","last_fullsized":"2024-09-16T09:36:57.763758+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"187'11","last_scrub_stamp":"2024-09-16T09:33:09.417852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:09.417852+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:37:07.764604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":574,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-16T09:36:56.761443+0000","last_change":"2024-09-16T09:36:47.605655+0000","last_active":"2024-09-16T09:36:56.761443+0000","last_peered":"2024-09-16T09:36:56.761443+0000","last_clean":"2024-09-16T09:36:56.761443+0000","last_became_active":"2024-09-16T09:25:07.722017+0000","last_became_peered":"2024-09-16T09:25:07.722017+0000","last_unstale":"2024-09-16T09:36:56.761443+0000","last_undegraded":"2024-09-16T09:36:56.761443+0000","last_fullsized":"2024-09-16T09:36:56.761443+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:47.605615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:36:47.605615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:08.852140+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"336'13676","reported_seq":13920,"reported_epoch":337,"state":"active+clean","last_fresh":"2024-09-16T09:36:58.028253+0000","last_change":"2024-09-16T09:33:25.372320+0000","last_active":"2024-09-16T09:36:58.028253+0000","last_peered":"2024-09-16T09:36:58.028253+0000","last_clean":"2024-09-16T09:36:58.028253+0000","last_became_active":"2024-09-16T09:33:25.372005+0000","last_became_peered":"2024-09-16T09:33:25.372005+0000","last_unstale":"2024-09-16T09:36:58.028253+0000","last_undegraded":"2024-09-16T09:36:58.028253+0000","last_fullsized":"2024-09-16T09:36:58.028253+0000","mapping_epoch":229,"log_start":"289'9012","ondisk_log_start":"289'9012","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:33:24.230486+0000","objects_scrubbed":0,"log_size":4664,"log_dups_size":0,"ondisk_log_size":4664,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":113508352,"num_objects":1732,"num_object_clones":0,"num_object_copies":5196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1732,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13676,"num_write_kb":493056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.7","version":"73'79","reported_seq":651,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-16T09:36:56.769761+0000","last_change":"2024-09-16T09:34:51.864131+0000","last_active":"2024-09-16T09:36:56.769761+0000","last_peered":"2024-09-16T09:36:56.769761+0000","last_clean":"2024-09-16T09:36:56.769761+0000","last_became_active":"2024-09-16T09:31:09.274707+0000","last_became_peered":"2024-09-16T09:31:09.274707+0000","last_unstale":"2024-09-16T09:36:56.769761+0000","last_undegraded":"2024-09-16T09:36:56.769761+0000","last_fullsized":"2024-09-16T09:36:56.769761+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:34:51.864036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:34:51.864036+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"285'20","reported_seq":278,"reported_epoch":337,"state":"active+clean","last_fresh":"2024-09-16T09:36:58.028223+0000","last_change":"2024-09-16T09:33:41.368102+0000","last_active":"2024-09-16T09:36:58.028223+0000","last_peered":"2024-09-16T09:36:58.028223+0000","last_clean":"2024-09-16T09:36:58.028223+0000","last_became_active":"2024-09-16T09:33:30.248453+0000","last_became_peered":"2024-09-16T09:33:30.248453+0000","last_unstale":"2024-09-16T09:36:58.028223+0000","last_undegraded":"2024-09-16T09:36:58.028223+0000","last_fullsized":"2024-09-16T09:36:58.028223+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-16T09:33:41.368016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:33:41.368016+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":610,"reported_epoch":337,"state":"active+clean","last_fresh":"2024-09-16T09:36:58.028627+0000","last_change":"2024-09-16T09:36:47.095703+0000","last_active":"2024-09-16T09:36:58.028627+0000","last_peered":"2024-09-16T09:36:58.028627+0000","last_clean":"2024-09-16T09:36:58.028627+0000","last_became_active":"2024-09-16T09:31:09.270619+0000","last_became_peered":"2024-09-16T09:31:09.270619+0000","last_unstale":"2024-09-16T09:36:58.028627+0000","last_undegraded":"2024-09-16T09:36:58.028627+0000","last_fullsized":"2024-09-16T09:36:58.028627+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:47.095658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:36:47.095658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:02.698363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"66'8","reported_seq":732,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-09-16T09:36:53.733460+0000","last_change":"2024-09-16T09:36:52.689167+0000","last_active":"2024-09-16T09:36:53.733460+0000","last_peered":"2024-09-16T09:36:53.733460+0000","last_clean":"2024-09-16T09:36:53.733460+0000","last_became_active":"2024-09-16T09:25:05.714507+0000","last_became_peered":"2024-09-16T09:25:05.714507+0000","last_unstale":"2024-09-16T09:36:53.733460+0000","last_undegraded":"2024-09-16T09:36:53.733460+0000","last_fullsized":"2024-09-16T09:36:53.733460+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:36:52.689091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:36:52.689091+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:06.275939+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":591,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-09-16T09:36:53.733161+0000","last_change":"2024-09-16T09:32:33.362233+0000","last_active":"2024-09-16T09:36:53.733161+0000","last_peered":"2024-09-16T09:36:53.733161+0000","last_clean":"2024-09-16T09:36:53.733161+0000","last_became_active":"2024-09-16T09:31:09.283192+0000","last_became_peered":"2024-09-16T09:31:09.283192+0000","last_unstale":"2024-09-16T09:36:53.733161+0000","last_undegraded":"2024-09-16T09:36:53.733161+0000","last_fullsized":"2024-09-16T09:36:53.733161+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:33.362188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:32:33.362188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.e","version":"336'13667","reported_seq":13909,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-16T09:36:56.923362+0000","last_change":"2024-09-16T09:33:25.247961+0000","last_active":"2024-09-16T09:36:56.923362+0000","last_peered":"2024-09-16T09:36:56.923362+0000","last_clean":"2024-09-16T09:36:56.923362+0000","last_became_active":"2024-09-16T09:33:25.247683+0000","last_became_peered":"2024-09-16T09:33:25.247683+0000","last_unstale":"2024-09-16T09:36:56.923362+0000","last_undegraded":"2024-09-16T09:36:56.923362+0000","last_fullsized":"2024-09-16T09:36:56.923362+0000","mapping_epoch":229,"log_start":"291'9600","ondisk_log_start":"291'9600","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:33:24.230486+0000","objects_scrubbed":0,"log_size":4067,"log_dups_size":0,"ondisk_log_size":4067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":107937792,"num_objects":1647,"num_object_clones":0,"num_object_copies":4941,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1647,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13667,"num_write_kb":490048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.6","version":"73'79","reported_seq":624,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-16T09:36:56.770505+0000","last_change":"2024-09-16T09:34:42.728232+0000","last_active":"2024-09-16T09:36:56.770505+0000","last_peered":"2024-09-16T09:36:56.770505+0000","last_clean":"2024-09-16T09:36:56.770505+0000","last_became_active":"2024-09-16T09:31:09.274565+0000","last_became_peered":"2024-09-16T09:31:09.274565+0000","last_unstale":"2024-09-16T09:36:56.770505+0000","last_undegraded":"2024-09-16T09:36:56.770505+0000","last_fullsized":"2024-09-16T09:36:56.770505+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:34:42.728143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:34:42.728143+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"285'29","reported_seq":628,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-09-16T09:36:53.733776+0000","last_change":"2024-09-16T09:31:35.249952+0000","last_active":"2024-09-16T09:36:53.733776+0000","last_peered":"2024-09-16T09:36:53.733776+0000","last_clean":"2024-09-16T09:36:53.733776+0000","last_became_active":"2024-09-16T09:25:03.714797+0000","last_became_peered":"2024-09-16T09:25:03.714797+0000","last_unstale":"2024-09-16T09:36:53.733776+0000","last_undegraded":"2024-09-16T09:36:53.733776+0000","last_fullsized":"2024-09-16T09:36:53.733776+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-16T09:31:35.249864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:31:35.249864+0000","objects_scrubbed":8,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":54,"num_read_kb":40,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":220,"reported_epoch":337,"state":"active+clean","last_fresh":"2024-09-16T09:36:58.028130+0000","last_change":"2024-09-16T09:33:30.247311+0000","last_active":"2024-09-16T09:36:58.028130+0000","last_peered":"2024-09-16T09:36:58.028130+0000","last_clean":"2024-09-16T09:36:58.028130+0000","last_became_active":"2024-09-16T09:33:30.246996+0000","last_became_peered":"2024-09-16T09:33:30.246996+0000","last_unstale":"2024-09-16T09:36:58.028130+0000","last_undegraded":"2024-09-16T09:36:58.028130+0000","last_fullsized":"2024-09-16T09:36:58.028130+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:11.407616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:11.407616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,2],"acting":[5,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":603,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-16T09:36:56.769433+0000","last_change":"2024-09-16T09:34:45.772748+0000","last_active":"2024-09-16T09:36:56.769433+0000","last_peered":"2024-09-16T09:36:56.769433+0000","last_clean":"2024-09-16T09:36:56.769433+0000","last_became_active":"2024-09-16T09:25:05.708663+0000","last_became_peered":"2024-09-16T09:25:05.708663+0000","last_unstale":"2024-09-16T09:36:56.769433+0000","last_undegraded":"2024-09-16T09:36:56.769433+0000","last_fullsized":"2024-09-16T09:36:56.769433+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:45.772674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:34:45.772674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":226,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-16T09:36:56.770502+0000","last_change":"2024-09-16T09:34:36.806165+0000","last_active":"2024-09-16T09:36:56.770502+0000","last_peered":"2024-09-16T09:36:56.770502+0000","last_clean":"2024-09-16T09:36:56.770502+0000","last_became_active":"2024-09-16T09:33:30.250178+0000","last_became_peered":"2024-09-16T09:33:30.250178+0000","last_unstale":"2024-09-16T09:36:56.770502+0000","last_undegraded":"2024-09-16T09:36:56.770502+0000","last_fullsized":"2024-09-16T09:36:56.770502+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:36.805971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:34:36.805971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,2],"acting":[6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"332'9571","reported_seq":9887,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-09-16T09:36:53.732833+0000","last_change":"2024-09-16T09:36:09.351656+0000","last_active":"2024-09-16T09:36:53.732833+0000","last_peered":"2024-09-16T09:36:53.732833+0000","last_clean":"2024-09-16T09:36:53.732833+0000","last_became_active":"2024-09-16T09:36:09.349892+0000","last_became_peered":"2024-09-16T09:36:09.349892+0000","last_unstale":"2024-09-16T09:36:53.732833+0000","last_undegraded":"2024-09-16T09:36:53.732833+0000","last_fullsized":"2024-09-16T09:36:53.732833+0000","mapping_epoch":229,"log_start":"303'7145","ondisk_log_start":"303'7145","created":243,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"291'6329","last_scrub_stamp":"2024-09-16T09:35:36.136797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:35:36.136797+0000","objects_scrubbed":2998,"log_size":2426,"log_dups_size":6000,"ondisk_log_size":2426,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":920,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":121503744,"num_objects":1854,"num_object_clones":0,"num_object_copies":5562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1854,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6016,"num_write_kb":71616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2504,"num_bytes_recovered":164102144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,5],"acting":[0,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"73'79","reported_seq":559,"reported_epoch":337,"state":"active+clean","last_fresh":"2024-09-16T09:36:58.027787+0000","last_change":"2024-09-16T09:36:42.191578+0000","last_active":"2024-09-16T09:36:58.027787+0000","last_peered":"2024-09-16T09:36:58.027787+0000","last_clean":"2024-09-16T09:36:58.027787+0000","last_became_active":"2024-09-16T09:30:08.766350+0000","last_became_peered":"2024-09-16T09:30:08.766350+0000","last_unstale":"2024-09-16T09:36:58.027787+0000","last_undegraded":"2024-09-16T09:36:58.027787+0000","last_fullsized":"2024-09-16T09:36:58.027787+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:36:42.191521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:36:42.191521+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:08.048449+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"285'42","reported_seq":876,"reported_epoch":337,"state":"active+clean","last_fresh":"2024-09-16T09:36:57.764170+0000","last_change":"2024-09-16T09:36:49.100039+0000","last_active":"2024-09-16T09:36:57.764170+0000","last_peered":"2024-09-16T09:36:57.764170+0000","last_clean":"2024-09-16T09:36:57.764170+0000","last_became_active":"2024-09-16T09:31:09.278839+0000","last_became_peered":"2024-09-16T09:31:09.278839+0000","last_unstale":"2024-09-16T09:36:57.764170+0000","last_undegraded":"2024-09-16T09:36:57.764170+0000","last_fullsized":"2024-09-16T09:36:57.764170+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'42","last_scrub_stamp":"2024-09-16T09:36:49.099960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:36:49.099960+0000","objects_scrubbed":10,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:37:50.536835+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":66,"num_read_kb":51,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":605,"reported_epoch":337,"state":"active+clean","last_fresh":"2024-09-16T09:36:57.764509+0000","last_change":"2024-09-16T09:36:35.691463+0000","last_active":"2024-09-16T09:36:57.764509+0000","last_peered":"2024-09-16T09:36:57.764509+0000","last_clean":"2024-09-16T09:36:57.764509+0000","last_became_active":"2024-09-16T09:25:01.703076+0000","last_became_peered":"2024-09-16T09:25:01.703076+0000","last_unstale":"2024-09-16T09:36:57.764509+0000","last_undegraded":"2024-09-16T09:36:57.764509+0000","last_fullsized":"2024-09-16T09:36:57.764509+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:35.691415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:36:35.691415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:37:38.143566+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":606,"reported_epoch":337,"state":"active+clean","last_fresh":"2024-09-16T09:36:58.028449+0000","last_change":"2024-09-16T09:33:38.417375+0000","last_active":"2024-09-16T09:36:58.028449+0000","last_peered":"2024-09-16T09:36:58.028449+0000","last_clean":"2024-09-16T09:36:58.028449+0000","last_became_active":"2024-09-16T09:25:00.696237+0000","last_became_peered":"2024-09-16T09:25:00.696237+0000","last_unstale":"2024-09-16T09:36:58.028449+0000","last_undegraded":"2024-09-16T09:36:58.028449+0000","last_fullsized":"2024-09-16T09:36:58.028449+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-16T09:33:38.417308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:33:38.417308+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.0025137430000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":565,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-09-16T09:36:53.734043+0000","last_change":"2024-09-16T09:33:16.666704+0000","last_active":"2024-09-16T09:36:53.734043+0000","last_peered":"2024-09-16T09:36:53.734043+0000","last_clean":"2024-09-16T09:36:53.734043+0000","last_became_active":"2024-09-16T09:25:05.717591+0000","last_became_peered":"2024-09-16T09:25:05.717591+0000","last_unstale":"2024-09-16T09:36:53.734043+0000","last_undegraded":"2024-09-16T09:36:53.734043+0000","last_fullsized":"2024-09-16T09:36:53.734043+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:16.666648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:16.666648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":322,"reported_epoch":337,"state":"active+clean","last_fresh":"2024-09-16T09:36:57.761244+0000","last_change":"2024-09-16T09:36:50.667709+0000","last_active":"2024-09-16T09:36:57.761244+0000","last_peered":"2024-09-16T09:36:57.761244+0000","last_clean":"2024-09-16T09:36:57.761244+0000","last_became_active":"2024-09-16T09:31:09.272609+0000","last_became_peered":"2024-09-16T09:31:09.272609+0000","last_unstale":"2024-09-16T09:36:57.761244+0000","last_undegraded":"2024-09-16T09:36:57.761244+0000","last_fullsized":"2024-09-16T09:36:57.761244+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:50.667661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:36:50.667661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:09.567715+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.a","version":"336'13288","reported_seq":13556,"reported_epoch":337,"state":"active+clean","last_fresh":"2024-09-16T09:36:58.027792+0000","last_change":"2024-09-16T09:36:57.195191+0000","last_active":"2024-09-16T09:36:58.027792+0000","last_peered":"2024-09-16T09:36:58.027792+0000","last_clean":"2024-09-16T09:36:58.027792+0000","last_became_active":"2024-09-16T09:33:25.243701+0000","last_became_peered":"2024-09-16T09:33:25.243701+0000","last_unstale":"2024-09-16T09:36:58.027792+0000","last_undegraded":"2024-09-16T09:36:58.027792+0000","last_fullsized":"2024-09-16T09:36:58.027792+0000","mapping_epoch":229,"log_start":"289'8607","ondisk_log_start":"289'8607","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"336'13288","last_scrub_stamp":"2024-09-16T09:36:57.195079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:36:57.195079+0000","objects_scrubbed":1690,"log_size":4681,"log_dups_size":0,"ondisk_log_size":4681,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:16.536196+0000","scrub_duration":184,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110755840,"num_objects":1690,"num_object_clones":0,"num_object_copies":5070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1690,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13288,"num_write_kb":479296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"73'79","reported_seq":268,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-09-16T09:36:53.733012+0000","last_change":"2024-09-16T09:34:32.717994+0000","last_active":"2024-09-16T09:36:53.733012+0000","last_peered":"2024-09-16T09:36:53.733012+0000","last_clean":"2024-09-16T09:36:53.733012+0000","last_became_active":"2024-09-16T09:34:00.336514+0000","last_became_peered":"2024-09-16T09:34:00.336514+0000","last_unstale":"2024-09-16T09:36:53.733012+0000","last_undegraded":"2024-09-16T09:36:53.733012+0000","last_fullsized":"2024-09-16T09:36:53.733012+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:34:32.717915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:34:32.717915+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"285'17","reported_seq":292,"reported_epoch":337,"state":"active+clean","last_fresh":"2024-09-16T09:36:58.028857+0000","last_change":"2024-09-16T09:34:15.319682+0000","last_active":"2024-09-16T09:36:58.028857+0000","last_peered":"2024-09-16T09:36:58.028857+0000","last_clean":"2024-09-16T09:36:58.028857+0000","last_became_active":"2024-09-16T09:34:00.337121+0000","last_became_peered":"2024-09-16T09:34:00.337121+0000","last_unstale":"2024-09-16T09:36:58.028857+0000","last_undegraded":"2024-09-16T09:36:58.028857+0000","last_fullsized":"2024-09-16T09:36:58.028857+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'13","last_scrub_stamp":"2024-09-16T09:27:44.155969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:44.155969+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:37:26.933332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":35,"num_read_kb":24,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":218,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-16T09:36:56.769141+0000","last_change":"2024-09-16T09:33:30.248260+0000","last_active":"2024-09-16T09:36:56.769141+0000","last_peered":"2024-09-16T09:36:56.769141+0000","last_clean":"2024-09-16T09:36:56.769141+0000","last_became_active":"2024-09-16T09:33:30.247596+0000","last_became_peered":"2024-09-16T09:33:30.247596+0000","last_unstale":"2024-09-16T09:36:56.769141+0000","last_undegraded":"2024-09-16T09:36:56.769141+0000","last_fullsized":"2024-09-16T09:36:56.769141+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:22.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:22.331302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,2],"acting":[6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":236,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-09-16T09:36:53.734354+0000","last_change":"2024-09-16T09:33:21.338149+0000","last_active":"2024-09-16T09:36:53.734354+0000","last_peered":"2024-09-16T09:36:53.734354+0000","last_clean":"2024-09-16T09:36:53.734354+0000","last_became_active":"2024-09-16T09:33:21.337177+0000","last_became_peered":"2024-09-16T09:33:21.337177+0000","last_unstale":"2024-09-16T09:36:53.734354+0000","last_undegraded":"2024-09-16T09:36:53.734354+0000","last_fullsized":"2024-09-16T09:36:53.734354+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:32:40.165749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:32:40.165749+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":611,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-16T09:36:56.763977+0000","last_change":"2024-09-16T09:36:55.530225+0000","last_active":"2024-09-16T09:36:56.763977+0000","last_peered":"2024-09-16T09:36:56.763977+0000","last_clean":"2024-09-16T09:36:56.763977+0000","last_became_active":"2024-09-16T09:25:05.711539+0000","last_became_peered":"2024-09-16T09:25:05.711539+0000","last_unstale":"2024-09-16T09:36:56.763977+0000","last_undegraded":"2024-09-16T09:36:56.763977+0000","last_fullsized":"2024-09-16T09:36:56.763977+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:55.530175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:36:55.530175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:18.377901+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":582,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-16T09:36:56.761218+0000","last_change":"2024-09-16T09:33:39.313624+0000","last_active":"2024-09-16T09:36:56.761218+0000","last_peered":"2024-09-16T09:36:56.761218+0000","last_clean":"2024-09-16T09:36:56.761218+0000","last_became_active":"2024-09-16T09:25:07.723282+0000","last_became_peered":"2024-09-16T09:25:07.723282+0000","last_unstale":"2024-09-16T09:36:56.761218+0000","last_undegraded":"2024-09-16T09:36:56.761218+0000","last_fullsized":"2024-09-16T09:36:56.761218+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:39.313562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:33:39.313562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.b","version":"332'13061","reported_seq":13295,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-09-16T09:36:53.733594+0000","last_change":"2024-09-16T09:33:25.245108+0000","last_active":"2024-09-16T09:36:53.733594+0000","last_peered":"2024-09-16T09:36:53.733594+0000","last_clean":"2024-09-16T09:36:53.733594+0000","last_became_active":"2024-09-16T09:33:25.244819+0000","last_became_peered":"2024-09-16T09:33:25.244819+0000","last_unstale":"2024-09-16T09:36:53.733594+0000","last_undegraded":"2024-09-16T09:36:53.733594+0000","last_fullsized":"2024-09-16T09:36:53.733594+0000","mapping_epoch":229,"log_start":"286'8100","ondisk_log_start":"286'8100","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:33:24.230486+0000","objects_scrubbed":0,"log_size":4961,"log_dups_size":0,"ondisk_log_size":4961,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":119209984,"num_objects":1819,"num_object_clones":0,"num_object_copies":5457,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1819,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13061,"num_write_kb":476160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"73'79","reported_seq":307,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-16T09:36:56.770264+0000","last_change":"2024-09-16T09:34:31.929462+0000","last_active":"2024-09-16T09:36:56.770264+0000","last_peered":"2024-09-16T09:36:56.770264+0000","last_clean":"2024-09-16T09:36:56.770264+0000","last_became_active":"2024-09-16T09:33:30.249309+0000","last_became_peered":"2024-09-16T09:33:30.249309+0000","last_unstale":"2024-09-16T09:36:56.770264+0000","last_undegraded":"2024-09-16T09:36:56.770264+0000","last_fullsized":"2024-09-16T09:36:56.770264+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:34:31.929388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:34:31.929388+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.6","version":"285'24","reported_seq":613,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-09-16T09:36:53.733587+0000","last_change":"2024-09-16T09:32:48.547237+0000","last_active":"2024-09-16T09:36:53.733587+0000","last_peered":"2024-09-16T09:36:53.733587+0000","last_clean":"2024-09-16T09:36:53.733587+0000","last_became_active":"2024-09-16T09:25:03.715315+0000","last_became_peered":"2024-09-16T09:25:03.715315+0000","last_unstale":"2024-09-16T09:36:53.733587+0000","last_undegraded":"2024-09-16T09:36:53.733587+0000","last_fullsized":"2024-09-16T09:36:53.733587+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-16T09:32:48.546869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:32:48.546869+0000","objects_scrubbed":6,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":36,"num_read_kb":30,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"59'1","reported_seq":612,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-09-16T09:36:53.733162+0000","last_change":"2024-09-16T09:33:21.682893+0000","last_active":"2024-09-16T09:36:53.733162+0000","last_peered":"2024-09-16T09:36:53.733162+0000","last_clean":"2024-09-16T09:36:53.733162+0000","last_became_active":"2024-09-16T09:31:09.269627+0000","last_became_peered":"2024-09-16T09:31:09.269627+0000","last_unstale":"2024-09-16T09:36:53.733162+0000","last_undegraded":"2024-09-16T09:36:53.733162+0000","last_fullsized":"2024-09-16T09:36:53.733162+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:33:21.682840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:21.682840+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1986,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"67'5","reported_seq":1315,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-16T09:36:56.767732+0000","last_change":"2024-09-16T09:33:37.358839+0000","last_active":"2024-09-16T09:36:56.767732+0000","last_peered":"2024-09-16T09:36:56.767732+0000","last_clean":"2024-09-16T09:36:56.767732+0000","last_became_active":"2024-09-16T09:25:00.696546+0000","last_became_peered":"2024-09-16T09:25:00.696546+0000","last_unstale":"2024-09-16T09:36:56.767732+0000","last_undegraded":"2024-09-16T09:36:56.767732+0000","last_fullsized":"2024-09-16T09:36:56.767732+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'5","last_scrub_stamp":"2024-09-16T09:33:37.358741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:33:37.358741+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0.0059770420000000001,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":710,"num_read_kb":705,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":572,"reported_epoch":337,"state":"active+clean","last_fresh":"2024-09-16T09:36:58.027617+0000","last_change":"2024-09-16T09:27:07.337170+0000","last_active":"2024-09-16T09:36:58.027617+0000","last_peered":"2024-09-16T09:36:58.027617+0000","last_clean":"2024-09-16T09:36:58.027617+0000","last_became_active":"2024-09-16T09:25:05.714704+0000","last_became_peered":"2024-09-16T09:25:05.714704+0000","last_unstale":"2024-09-16T09:36:58.027617+0000","last_undegraded":"2024-09-16T09:36:58.027617+0000","last_fullsized":"2024-09-16T09:36:58.027617+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:07.337124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:07.337124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:37:19.897485+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":498,"reported_epoch":337,"state":"active+clean","last_fresh":"2024-09-16T09:36:58.028242+0000","last_change":"2024-09-16T09:36:43.150813+0000","last_active":"2024-09-16T09:36:58.028242+0000","last_peered":"2024-09-16T09:36:58.028242+0000","last_clean":"2024-09-16T09:36:58.028242+0000","last_became_active":"2024-09-16T09:33:30.245852+0000","last_became_peered":"2024-09-16T09:33:30.245852+0000","last_unstale":"2024-09-16T09:36:58.028242+0000","last_undegraded":"2024-09-16T09:36:58.028242+0000","last_fullsized":"2024-09-16T09:36:58.028242+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:43.150764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:36:43.150764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:37:48.439707+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"332'8365","reported_seq":13986,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-09-16T09:36:53.733242+0000","last_change":"2024-09-16T09:36:43.655321+0000","last_active":"2024-09-16T09:36:53.733242+0000","last_peered":"2024-09-16T09:36:53.733242+0000","last_clean":"2024-09-16T09:36:53.733242+0000","last_became_active":"2024-09-16T09:36:43.654159+0000","last_became_peered":"2024-09-16T09:36:43.654159+0000","last_unstale":"2024-09-16T09:36:53.733242+0000","last_undegraded":"2024-09-16T09:36:53.733242+0000","last_fullsized":"2024-09-16T09:36:53.733242+0000","mapping_epoch":272,"log_start":"319'8082","ondisk_log_start":"319'8082","created":243,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"275'4947","last_scrub_stamp":"2024-09-16T09:34:50.611186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:34:50.611186+0000","objects_scrubbed":3078,"log_size":283,"log_dups_size":6000,"ondisk_log_size":283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":697,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123011072,"num_objects":1877,"num_object_clones":0,"num_object_copies":5631,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1877,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6892,"num_write_kb":74944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5349,"num_bytes_recovered":339935232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"285'28","reported_seq":783,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-16T09:36:56.770342+0000","last_change":"2024-09-16T09:33:16.891850+0000","last_active":"2024-09-16T09:36:56.770342+0000","last_peered":"2024-09-16T09:36:56.770342+0000","last_clean":"2024-09-16T09:36:56.770342+0000","last_became_active":"2024-09-16T09:25:03.714118+0000","last_became_peered":"2024-09-16T09:25:03.714118+0000","last_unstale":"2024-09-16T09:36:56.770342+0000","last_undegraded":"2024-09-16T09:36:56.770342+0000","last_fullsized":"2024-09-16T09:36:56.770342+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'16","last_scrub_stamp":"2024-09-16T09:33:16.891761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:33:16.891761+0000","objects_scrubbed":8,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":43,"num_read_kb":33,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":446,"reported_epoch":337,"state":"active+clean","last_fresh":"2024-09-16T09:36:58.028375+0000","last_change":"2024-09-16T09:36:50.114910+0000","last_active":"2024-09-16T09:36:58.028375+0000","last_peered":"2024-09-16T09:36:58.028375+0000","last_clean":"2024-09-16T09:36:58.028375+0000","last_became_active":"2024-09-16T09:31:09.270196+0000","last_became_peered":"2024-09-16T09:31:09.270196+0000","last_unstale":"2024-09-16T09:36:58.028375+0000","last_undegraded":"2024-09-16T09:36:58.028375+0000","last_fullsized":"2024-09-16T09:36:58.028375+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:50.114861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:36:50.114861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:13.001323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"73'79","reported_seq":675,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-16T09:36:56.761966+0000","last_change":"2024-09-16T09:32:43.535041+0000","last_active":"2024-09-16T09:36:56.761966+0000","last_peered":"2024-09-16T09:36:56.761966+0000","last_clean":"2024-09-16T09:36:56.761966+0000","last_became_active":"2024-09-16T09:30:08.870394+0000","last_became_peered":"2024-09-16T09:30:08.870394+0000","last_unstale":"2024-09-16T09:36:56.761966+0000","last_undegraded":"2024-09-16T09:36:56.761966+0000","last_fullsized":"2024-09-16T09:36:56.761966+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:32:43.534962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:32:43.534962+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":578,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-16T09:36:56.761827+0000","last_change":"2024-09-16T09:33:52.291748+0000","last_active":"2024-09-16T09:36:56.761827+0000","last_peered":"2024-09-16T09:36:56.761827+0000","last_clean":"2024-09-16T09:36:56.761827+0000","last_became_active":"2024-09-16T09:25:05.714341+0000","last_became_peered":"2024-09-16T09:25:05.714341+0000","last_unstale":"2024-09-16T09:36:56.761827+0000","last_undegraded":"2024-09-16T09:36:56.761827+0000","last_fullsized":"2024-09-16T09:36:56.761827+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:52.291702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:52.291702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":576,"reported_epoch":337,"state":"active+clean","last_fresh":"2024-09-16T09:36:58.028792+0000","last_change":"2024-09-16T09:29:00.243528+0000","last_active":"2024-09-16T09:36:58.028792+0000","last_peered":"2024-09-16T09:36:58.028792+0000","last_clean":"2024-09-16T09:36:58.028792+0000","last_became_active":"2024-09-16T09:25:07.714281+0000","last_became_peered":"2024-09-16T09:25:07.714281+0000","last_unstale":"2024-09-16T09:36:58.028792+0000","last_undegraded":"2024-09-16T09:36:58.028792+0000","last_fullsized":"2024-09-16T09:36:58.028792+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:00.243482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:00.243482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:37:32.082677+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3","version":"336'8196","reported_seq":8469,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-16T09:36:56.907335+0000","last_change":"2024-09-16T09:33:53.717144+0000","last_active":"2024-09-16T09:36:56.907335+0000","last_peered":"2024-09-16T09:36:56.907335+0000","last_clean":"2024-09-16T09:36:56.907335+0000","last_became_active":"2024-09-16T09:33:52.914057+0000","last_became_peered":"2024-09-16T09:33:52.914057+0000","last_unstale":"2024-09-16T09:36:56.907335+0000","last_undegraded":"2024-09-16T09:36:56.907335+0000","last_fullsized":"2024-09-16T09:36:56.907335+0000","mapping_epoch":229,"log_start":"265'4156","ondisk_log_start":"265'4156","created":229,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"244'3096","last_scrub_stamp":"2024-09-16T09:33:53.717081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:33:53.717081+0000","objects_scrubbed":1554,"log_size":4040,"log_dups_size":0,"ondisk_log_size":4040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54853632,"num_objects":837,"num_object_clones":0,"num_object_copies":2511,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":837,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5115,"num_write_kb":140736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.b","version":"73'79","reported_seq":404,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-09-16T09:36:53.733224+0000","last_change":"2024-09-16T09:34:31.720971+0000","last_active":"2024-09-16T09:36:53.733224+0000","last_peered":"2024-09-16T09:36:53.733224+0000","last_clean":"2024-09-16T09:36:53.733224+0000","last_became_active":"2024-09-16T09:33:30.251933+0000","last_became_peered":"2024-09-16T09:33:30.251933+0000","last_unstale":"2024-09-16T09:36:53.733224+0000","last_undegraded":"2024-09-16T09:36:53.733224+0000","last_fullsized":"2024-09-16T09:36:53.733224+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:34:31.720787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:34:31.720787+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"285'19","reported_seq":654,"reported_epoch":337,"state":"active+clean","last_fresh":"2024-09-16T09:36:57.765117+0000","last_change":"2024-09-16T09:35:27.813299+0000","last_active":"2024-09-16T09:36:57.765117+0000","last_peered":"2024-09-16T09:36:57.765117+0000","last_clean":"2024-09-16T09:36:57.765117+0000","last_became_active":"2024-09-16T09:34:00.328533+0000","last_became_peered":"2024-09-16T09:34:00.328533+0000","last_unstale":"2024-09-16T09:36:57.765117+0000","last_undegraded":"2024-09-16T09:36:57.765117+0000","last_fullsized":"2024-09-16T09:36:57.765117+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'19","last_scrub_stamp":"2024-09-16T09:35:27.813206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:35:27.813206+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[4,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":581,"reported_epoch":337,"state":"active+clean","last_fresh":"2024-09-16T09:36:57.764313+0000","last_change":"2024-09-16T09:33:23.200213+0000","last_active":"2024-09-16T09:36:57.764313+0000","last_peered":"2024-09-16T09:36:57.764313+0000","last_clean":"2024-09-16T09:36:57.764313+0000","last_became_active":"2024-09-16T09:25:01.708270+0000","last_became_peered":"2024-09-16T09:25:01.708270+0000","last_unstale":"2024-09-16T09:36:57.764313+0000","last_undegraded":"2024-09-16T09:36:57.764313+0000","last_fullsized":"2024-09-16T09:36:57.764313+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:23.200166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:23.200166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:37:04.837585+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":572,"reported_epoch":337,"state":"active+clean","last_fresh":"2024-09-16T09:36:58.027632+0000","last_change":"2024-09-16T09:26:34.171190+0000","last_active":"2024-09-16T09:36:58.027632+0000","last_peered":"2024-09-16T09:36:58.027632+0000","last_clean":"2024-09-16T09:36:58.027632+0000","last_became_active":"2024-09-16T09:25:05.711964+0000","last_became_peered":"2024-09-16T09:25:05.711964+0000","last_unstale":"2024-09-16T09:36:58.027632+0000","last_undegraded":"2024-09-16T09:36:58.027632+0000","last_fullsized":"2024-09-16T09:36:58.027632+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:34.171139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:26:34.171139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:37:14.922914+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":461,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-16T09:36:56.769925+0000","last_change":"2024-09-16T09:34:50.866121+0000","last_active":"2024-09-16T09:36:56.769925+0000","last_peered":"2024-09-16T09:36:56.769925+0000","last_clean":"2024-09-16T09:36:56.769925+0000","last_became_active":"2024-09-16T09:31:09.278618+0000","last_became_peered":"2024-09-16T09:31:09.278618+0000","last_unstale":"2024-09-16T09:36:56.769925+0000","last_undegraded":"2024-09-16T09:36:56.769925+0000","last_fullsized":"2024-09-16T09:36:56.769925+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:50.866032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:34:50.866032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.0","version":"336'8322","reported_seq":8613,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-16T09:36:56.925705+0000","last_change":"2024-09-16T09:35:37.795621+0000","last_active":"2024-09-16T09:36:56.925705+0000","last_peered":"2024-09-16T09:36:56.925705+0000","last_clean":"2024-09-16T09:36:56.925705+0000","last_became_active":"2024-09-16T09:33:52.914803+0000","last_became_peered":"2024-09-16T09:33:52.914803+0000","last_unstale":"2024-09-16T09:36:56.925705+0000","last_undegraded":"2024-09-16T09:36:56.925705+0000","last_fullsized":"2024-09-16T09:36:56.925705+0000","mapping_epoch":233,"log_start":"247'3323","ondisk_log_start":"247'3323","created":229,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"291'6312","last_scrub_stamp":"2024-09-16T09:35:37.795459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:35:37.795459+0000","objects_scrubbed":2847,"log_size":4999,"log_dups_size":0,"ondisk_log_size":4999,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54263808,"num_objects":828,"num_object_clones":0,"num_object_copies":2484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":828,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2019,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,2],"acting":[6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.8","version":"73'79","reported_seq":202,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-16T09:36:56.769627+0000","last_change":"2024-09-16T09:34:30.931922+0000","last_active":"2024-09-16T09:36:56.769627+0000","last_peered":"2024-09-16T09:36:56.769627+0000","last_clean":"2024-09-16T09:36:56.769627+0000","last_became_active":"2024-09-16T09:34:00.329241+0000","last_became_peered":"2024-09-16T09:34:00.329241+0000","last_unstale":"2024-09-16T09:36:56.769627+0000","last_undegraded":"2024-09-16T09:36:56.769627+0000","last_fullsized":"2024-09-16T09:36:56.769627+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:34:30.931863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:34:30.931863+0000","objects_scrubbed":1,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262688,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,3],"acting":[6,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"285'31","reported_seq":650,"reported_epoch":337,"state":"active+clean","last_fresh":"2024-09-16T09:36:57.764969+0000","last_change":"2024-09-16T09:35:28.861520+0000","last_active":"2024-09-16T09:36:57.764969+0000","last_peered":"2024-09-16T09:36:57.764969+0000","last_clean":"2024-09-16T09:36:57.764969+0000","last_became_active":"2024-09-16T09:25:03.713358+0000","last_became_peered":"2024-09-16T09:25:03.713358+0000","last_unstale":"2024-09-16T09:36:57.764969+0000","last_undegraded":"2024-09-16T09:36:57.764969+0000","last_fullsized":"2024-09-16T09:36:57.764969+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'31","last_scrub_stamp":"2024-09-16T09:35:28.861433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:35:28.861433+0000","objects_scrubbed":8,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":50,"num_read_kb":40,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":587,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-16T09:36:56.769780+0000","last_change":"2024-09-16T09:34:32.899965+0000","last_active":"2024-09-16T09:36:56.769780+0000","last_peered":"2024-09-16T09:36:56.769780+0000","last_clean":"2024-09-16T09:36:56.769780+0000","last_became_active":"2024-09-16T09:25:01.703355+0000","last_became_peered":"2024-09-16T09:25:01.703355+0000","last_unstale":"2024-09-16T09:36:56.769780+0000","last_undegraded":"2024-09-16T09:36:56.769780+0000","last_fullsized":"2024-09-16T09:36:56.769780+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:32.899886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:34:32.899886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":228,"reported_epoch":337,"state":"active+clean","last_fresh":"2024-09-16T09:36:57.764357+0000","last_change":"2024-09-16T09:33:44.298324+0000","last_active":"2024-09-16T09:36:57.764357+0000","last_peered":"2024-09-16T09:36:57.764357+0000","last_clean":"2024-09-16T09:36:57.764357+0000","last_became_active":"2024-09-16T09:33:30.251186+0000","last_became_peered":"2024-09-16T09:33:30.251186+0000","last_unstale":"2024-09-16T09:36:57.764357+0000","last_undegraded":"2024-09-16T09:36:57.764357+0000","last_fullsized":"2024-09-16T09:36:57.764357+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:44.298246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:44.298246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:37:30.644758+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,0],"acting":[4,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":526,"reported_epoch":337,"state":"active+clean","last_fresh":"2024-09-16T09:36:57.760550+0000","last_change":"2024-09-16T09:36:51.640537+0000","last_active":"2024-09-16T09:36:57.760550+0000","last_peered":"2024-09-16T09:36:57.760550+0000","last_clean":"2024-09-16T09:36:57.760550+0000","last_became_active":"2024-09-16T09:33:21.337291+0000","last_became_peered":"2024-09-16T09:33:21.337291+0000","last_unstale":"2024-09-16T09:36:57.760550+0000","last_undegraded":"2024-09-16T09:36:57.760550+0000","last_fullsized":"2024-09-16T09:36:57.760550+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:51.640472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:36:51.640472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:06.647486+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4],"acting":[3,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1","version":"336'8259","reported_seq":8582,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-16T09:36:56.890460+0000","last_change":"2024-09-16T09:35:36.575967+0000","last_active":"2024-09-16T09:36:56.890460+0000","last_peered":"2024-09-16T09:36:56.890460+0000","last_clean":"2024-09-16T09:36:56.890460+0000","last_became_active":"2024-09-16T09:33:52.910622+0000","last_became_peered":"2024-09-16T09:33:52.910622+0000","last_unstale":"2024-09-16T09:36:56.890460+0000","last_undegraded":"2024-09-16T09:36:56.890460+0000","last_fullsized":"2024-09-16T09:36:56.890460+0000","mapping_epoch":229,"log_start":"250'3307","ondisk_log_start":"250'3307","created":229,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"291'6189","last_scrub_stamp":"2024-09-16T09:35:36.575666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:35:36.575666+0000","objects_scrubbed":2923,"log_size":4952,"log_dups_size":0,"ondisk_log_size":4952,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":620,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":55115776,"num_objects":841,"num_object_clones":0,"num_object_copies":2523,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":841,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2082,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.9","version":"73'79","reported_seq":650,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-16T09:36:56.761917+0000","last_change":"2024-09-16T09:29:19.013948+0000","last_active":"2024-09-16T09:36:56.761917+0000","last_peered":"2024-09-16T09:36:56.761917+0000","last_clean":"2024-09-16T09:36:56.761917+0000","last_became_active":"2024-09-16T09:26:35.126066+0000","last_became_peered":"2024-09-16T09:26:35.126066+0000","last_unstale":"2024-09-16T09:36:56.761917+0000","last_undegraded":"2024-09-16T09:36:56.761917+0000","last_fullsized":"2024-09-16T09:36:56.761917+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:19.013865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:19.013865+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"285'20","reported_seq":607,"reported_epoch":337,"state":"active+clean","last_fresh":"2024-09-16T09:36:57.764546+0000","last_change":"2024-09-16T09:33:24.154567+0000","last_active":"2024-09-16T09:36:57.764546+0000","last_peered":"2024-09-16T09:36:57.764546+0000","last_clean":"2024-09-16T09:36:57.764546+0000","last_became_active":"2024-09-16T09:25:03.707723+0000","last_became_peered":"2024-09-16T09:25:03.707723+0000","last_unstale":"2024-09-16T09:36:57.764546+0000","last_undegraded":"2024-09-16T09:36:57.764546+0000","last_fullsized":"2024-09-16T09:36:57.764546+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-16T09:33:24.154507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:33:24.154507+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:37:00.827564+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":446,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-09-16T09:36:53.733962+0000","last_change":"2024-09-16T09:33:10.528181+0000","last_active":"2024-09-16T09:36:53.733962+0000","last_peered":"2024-09-16T09:36:53.733962+0000","last_clean":"2024-09-16T09:36:53.733962+0000","last_became_active":"2024-09-16T09:31:09.288825+0000","last_became_peered":"2024-09-16T09:31:09.288825+0000","last_unstale":"2024-09-16T09:36:53.733962+0000","last_undegraded":"2024-09-16T09:36:53.733962+0000","last_fullsized":"2024-09-16T09:36:53.733962+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:10.528115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:10.528115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"228'9","reported_seq":303,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-16T09:36:56.761350+0000","last_change":"2024-09-16T09:33:21.337623+0000","last_active":"2024-09-16T09:36:56.761350+0000","last_peered":"2024-09-16T09:36:56.761350+0000","last_clean":"2024-09-16T09:36:56.761350+0000","last_became_active":"2024-09-16T09:33:21.337213+0000","last_became_peered":"2024-09-16T09:33:21.337213+0000","last_unstale":"2024-09-16T09:36:56.761350+0000","last_undegraded":"2024-09-16T09:36:56.761350+0000","last_fullsized":"2024-09-16T09:36:56.761350+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:32:42.175528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:32:42.175528+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":601,"reported_epoch":337,"state":"active+clean","last_fresh":"2024-09-16T09:36:57.764895+0000","last_change":"2024-09-16T09:35:36.061013+0000","last_active":"2024-09-16T09:36:57.764895+0000","last_peered":"2024-09-16T09:36:57.764895+0000","last_clean":"2024-09-16T09:36:57.764895+0000","last_became_active":"2024-09-16T09:25:07.718958+0000","last_became_peered":"2024-09-16T09:25:07.718958+0000","last_unstale":"2024-09-16T09:36:57.764895+0000","last_undegraded":"2024-09-16T09:36:57.764895+0000","last_fullsized":"2024-09-16T09:36:57.764895+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:35:36.060942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:35:36.060942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:37:03.402548+0000","scrub_duration":157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.6","version":"336'8542","reported_seq":8870,"reported_epoch":337,"state":"active+clean","last_fresh":"2024-09-16T09:36:58.028243+0000","last_change":"2024-09-16T09:36:51.317058+0000","last_active":"2024-09-16T09:36:58.028243+0000","last_peered":"2024-09-16T09:36:58.028243+0000","last_clean":"2024-09-16T09:36:58.028243+0000","last_became_active":"2024-09-16T09:36:51.316286+0000","last_became_peered":"2024-09-16T09:36:51.316286+0000","last_unstale":"2024-09-16T09:36:58.028243+0000","last_undegraded":"2024-09-16T09:36:58.028243+0000","last_fullsized":"2024-09-16T09:36:58.028243+0000","mapping_epoch":229,"log_start":"327'8255","ondisk_log_start":"327'8255","created":243,"last_epoch_clean":331,"parent":"0.0","parent_split_bits":0,"last_scrub":"291'6321","last_scrub_stamp":"2024-09-16T09:35:36.697883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:35:36.697883+0000","objects_scrubbed":2977,"log_size":287,"log_dups_size":6000,"ondisk_log_size":287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":949,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":114622464,"num_objects":1749,"num_object_clones":0,"num_object_copies":5247,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1749,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5677,"num_write_kb":40576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2712,"num_bytes_recovered":177602560,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.e","version":"73'79","reported_seq":394,"reported_epoch":337,"state":"active+clean","last_fresh":"2024-09-16T09:36:57.764955+0000","last_change":"2024-09-16T09:35:41.979877+0000","last_active":"2024-09-16T09:36:57.764955+0000","last_peered":"2024-09-16T09:36:57.764955+0000","last_clean":"2024-09-16T09:36:57.764955+0000","last_became_active":"2024-09-16T09:30:10.705072+0000","last_became_peered":"2024-09-16T09:30:10.705072+0000","last_unstale":"2024-09-16T09:36:57.764955+0000","last_undegraded":"2024-09-16T09:36:57.764955+0000","last_fullsized":"2024-09-16T09:36:57.764955+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:35:41.979814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:35:41.979814+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:36:59.731534+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,3],"acting":[4,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.b","version":"285'15","reported_seq":627,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-09-16T09:36:53.733682+0000","last_change":"2024-09-16T09:36:31.653003+0000","last_active":"2024-09-16T09:36:53.733682+0000","last_peered":"2024-09-16T09:36:53.733682+0000","last_clean":"2024-09-16T09:36:53.733682+0000","last_became_active":"2024-09-16T09:36:31.590202+0000","last_became_peered":"2024-09-16T09:36:31.590202+0000","last_unstale":"2024-09-16T09:36:53.733682+0000","last_undegraded":"2024-09-16T09:36:53.733682+0000","last_fullsized":"2024-09-16T09:36:53.733682+0000","mapping_epoch":316,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":317,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:31:31.211420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:31:31.211420+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":594,"reported_epoch":337,"state":"active+clean","last_fresh":"2024-09-16T09:36:58.028330+0000","last_change":"2024-09-16T09:31:09.277413+0000","last_active":"2024-09-16T09:36:58.028330+0000","last_peered":"2024-09-16T09:36:58.028330+0000","last_clean":"2024-09-16T09:36:58.028330+0000","last_became_active":"2024-09-16T09:31:09.276516+0000","last_became_peered":"2024-09-16T09:31:09.276516+0000","last_unstale":"2024-09-16T09:36:58.028330+0000","last_undegraded":"2024-09-16T09:36:58.028330+0000","last_fullsized":"2024-09-16T09:36:58.028330+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:26.061106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:26.061106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:37:17.890836+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":522,"reported_epoch":337,"state":"active+clean","last_fresh":"2024-09-16T09:36:57.764497+0000","last_change":"2024-09-16T09:33:40.260547+0000","last_active":"2024-09-16T09:36:57.764497+0000","last_peered":"2024-09-16T09:36:57.764497+0000","last_clean":"2024-09-16T09:36:57.764497+0000","last_became_active":"2024-09-16T09:33:21.337254+0000","last_became_peered":"2024-09-16T09:33:21.337254+0000","last_unstale":"2024-09-16T09:36:57.764497+0000","last_undegraded":"2024-09-16T09:36:57.764497+0000","last_fullsized":"2024-09-16T09:36:57.764497+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:40.260500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:40.260500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:37:24.718816+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3],"acting":[4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":585,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-09-16T09:36:53.733070+0000","last_change":"2024-09-16T09:36:53.647305+0000","last_active":"2024-09-16T09:36:53.733070+0000","last_peered":"2024-09-16T09:36:53.733070+0000","last_clean":"2024-09-16T09:36:53.733070+0000","last_became_active":"2024-09-16T09:25:07.718778+0000","last_became_peered":"2024-09-16T09:25:07.718778+0000","last_unstale":"2024-09-16T09:36:53.733070+0000","last_undegraded":"2024-09-16T09:36:53.733070+0000","last_fullsized":"2024-09-16T09:36:53.733070+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:53.647256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:36:53.647256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:19.369534+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"336'8562","reported_seq":8781,"reported_epoch":337,"state":"active+clean","last_fresh":"2024-09-16T09:36:58.028701+0000","last_change":"2024-09-16T09:36:47.704186+0000","last_active":"2024-09-16T09:36:58.028701+0000","last_peered":"2024-09-16T09:36:58.028701+0000","last_clean":"2024-09-16T09:36:58.028701+0000","last_became_active":"2024-09-16T09:36:47.703235+0000","last_became_peered":"2024-09-16T09:36:47.703235+0000","last_unstale":"2024-09-16T09:36:58.028701+0000","last_undegraded":"2024-09-16T09:36:58.028701+0000","last_fullsized":"2024-09-16T09:36:58.028701+0000","mapping_epoch":229,"log_start":"323'8223","ondisk_log_start":"323'8223","created":243,"last_epoch_clean":327,"parent":"0.0","parent_split_bits":0,"last_scrub":"276'5145","last_scrub_stamp":"2024-09-16T09:34:53.523609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:34:53.523609+0000","objects_scrubbed":1562,"log_size":339,"log_dups_size":6000,"ondisk_log_size":339,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":422,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109117440,"num_objects":1665,"num_object_clones":0,"num_object_copies":4995,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1665,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8460,"num_write_kb":158848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3234,"num_bytes_recovered":211877888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.f","version":"73'79","reported_seq":644,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-16T09:36:56.770128+0000","last_change":"2024-09-16T09:34:53.866426+0000","last_active":"2024-09-16T09:36:56.770128+0000","last_peered":"2024-09-16T09:36:56.770128+0000","last_clean":"2024-09-16T09:36:56.770128+0000","last_became_active":"2024-09-16T09:30:10.693477+0000","last_became_peered":"2024-09-16T09:30:10.693477+0000","last_unstale":"2024-09-16T09:36:56.770128+0000","last_undegraded":"2024-09-16T09:36:56.770128+0000","last_fullsized":"2024-09-16T09:36:56.770128+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:34:53.866337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:34:53.866337+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"285'35","reported_seq":646,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-16T09:36:56.769527+0000","last_change":"2024-09-16T09:33:22.905808+0000","last_active":"2024-09-16T09:36:56.769527+0000","last_peered":"2024-09-16T09:36:56.769527+0000","last_clean":"2024-09-16T09:36:56.769527+0000","last_became_active":"2024-09-16T09:25:03.712783+0000","last_became_peered":"2024-09-16T09:25:03.712783+0000","last_unstale":"2024-09-16T09:36:56.769527+0000","last_undegraded":"2024-09-16T09:36:56.769527+0000","last_fullsized":"2024-09-16T09:36:56.769527+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-16T09:33:22.905752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:33:22.905752+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":56,"num_read_kb":45,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"59'1","reported_seq":600,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-16T09:36:56.761645+0000","last_change":"2024-09-16T09:33:42.294543+0000","last_active":"2024-09-16T09:36:56.761645+0000","last_peered":"2024-09-16T09:36:56.761645+0000","last_clean":"2024-09-16T09:36:56.761645+0000","last_became_active":"2024-09-16T09:25:01.708069+0000","last_became_peered":"2024-09-16T09:25:01.708069+0000","last_unstale":"2024-09-16T09:36:56.761645+0000","last_undegraded":"2024-09-16T09:36:56.761645+0000","last_fullsized":"2024-09-16T09:36:56.761645+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:33:42.294378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:42.294378+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":594,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-16T09:36:56.761380+0000","last_change":"2024-09-16T09:34:38.338415+0000","last_active":"2024-09-16T09:36:56.761380+0000","last_peered":"2024-09-16T09:36:56.761380+0000","last_clean":"2024-09-16T09:36:56.761380+0000","last_became_active":"2024-09-16T09:34:00.330050+0000","last_became_peered":"2024-09-16T09:34:00.330050+0000","last_unstale":"2024-09-16T09:36:56.761380+0000","last_undegraded":"2024-09-16T09:36:56.761380+0000","last_fullsized":"2024-09-16T09:36:56.761380+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:38.338329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:34:38.338329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,7],"acting":[2,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":574,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-16T09:36:56.762111+0000","last_change":"2024-09-16T09:36:48.649739+0000","last_active":"2024-09-16T09:36:56.762111+0000","last_peered":"2024-09-16T09:36:56.762111+0000","last_clean":"2024-09-16T09:36:56.762111+0000","last_became_active":"2024-09-16T09:25:07.718750+0000","last_became_peered":"2024-09-16T09:25:07.718750+0000","last_unstale":"2024-09-16T09:36:56.762111+0000","last_undegraded":"2024-09-16T09:36:56.762111+0000","last_fullsized":"2024-09-16T09:36:56.762111+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:48.649696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:36:48.649696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:18.071944+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.4","version":"335'8519","reported_seq":8810,"reported_epoch":337,"state":"clean+premerge+peered","last_fresh":"2024-09-16T09:36:57.788642+0000","last_change":"2024-09-16T09:36:57.788530+0000","last_active":"2024-09-16T09:36:56.759490+0000","last_peered":"2024-09-16T09:36:57.788642+0000","last_clean":"2024-09-16T09:36:57.788642+0000","last_became_active":"2024-09-16T09:33:52.909537+0000","last_became_peered":"2024-09-16T09:36:57.765426+0000","last_unstale":"2024-09-16T09:36:57.788642+0000","last_undegraded":"2024-09-16T09:36:57.788642+0000","last_fullsized":"2024-09-16T09:36:57.788642+0000","mapping_epoch":233,"log_start":"263'4206","ondisk_log_start":"263'4206","created":229,"last_epoch_clean":337,"parent":"0.0","parent_split_bits":0,"last_scrub":"324'8386","last_scrub_stamp":"2024-09-16T09:36:45.776828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:36:45.776828+0000","objects_scrubbed":1007,"log_size":4313,"log_dups_size":0,"ondisk_log_size":4313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:37:55.932168+0000","scrub_duration":170,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":57278464,"num_objects":874,"num_object_clones":0,"num_object_copies":2622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":874,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5290,"num_write_kb":144704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.c","version":"73'79","reported_seq":334,"reported_epoch":337,"state":"active+clean","last_fresh":"2024-09-16T09:36:57.762303+0000","last_change":"2024-09-16T09:36:53.697797+0000","last_active":"2024-09-16T09:36:57.762303+0000","last_peered":"2024-09-16T09:36:57.762303+0000","last_clean":"2024-09-16T09:36:57.762303+0000","last_became_active":"2024-09-16T09:31:09.273770+0000","last_became_peered":"2024-09-16T09:31:09.273770+0000","last_unstale":"2024-09-16T09:36:57.762303+0000","last_undegraded":"2024-09-16T09:36:57.762303+0000","last_fullsized":"2024-09-16T09:36:57.762303+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:36:53.697742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:36:53.697742+0000","objects_scrubbed":1,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:37:57.508460+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":327680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"285'18","reported_seq":671,"reported_epoch":337,"state":"active+clean","last_fresh":"2024-09-16T09:36:57.764166+0000","last_change":"2024-09-16T09:36:50.603724+0000","last_active":"2024-09-16T09:36:57.764166+0000","last_peered":"2024-09-16T09:36:57.764166+0000","last_clean":"2024-09-16T09:36:57.764166+0000","last_became_active":"2024-09-16T09:31:09.269047+0000","last_became_peered":"2024-09-16T09:31:09.269047+0000","last_unstale":"2024-09-16T09:36:57.764166+0000","last_undegraded":"2024-09-16T09:36:57.764166+0000","last_fullsized":"2024-09-16T09:36:57.764166+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'18","last_scrub_stamp":"2024-09-16T09:36:50.603660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:36:50.603660+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:15.970176+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":594,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-16T09:36:56.764298+0000","last_change":"2024-09-16T09:33:48.225867+0000","last_active":"2024-09-16T09:36:56.764298+0000","last_peered":"2024-09-16T09:36:56.764298+0000","last_clean":"2024-09-16T09:36:56.764298+0000","last_became_active":"2024-09-16T09:25:01.707528+0000","last_became_peered":"2024-09-16T09:25:01.707528+0000","last_unstale":"2024-09-16T09:36:56.764298+0000","last_undegraded":"2024-09-16T09:36:56.764298+0000","last_fullsized":"2024-09-16T09:36:56.764298+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:48.225635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:48.225635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":232,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-09-16T09:36:53.732733+0000","last_change":"2024-09-16T09:33:23.643136+0000","last_active":"2024-09-16T09:36:53.732733+0000","last_peered":"2024-09-16T09:36:53.732733+0000","last_clean":"2024-09-16T09:36:53.732733+0000","last_became_active":"2024-09-16T09:33:21.337309+0000","last_became_peered":"2024-09-16T09:33:21.337309+0000","last_unstale":"2024-09-16T09:36:53.732733+0000","last_undegraded":"2024-09-16T09:36:53.732733+0000","last_fullsized":"2024-09-16T09:36:53.732733+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:23.643089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:23.643089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":577,"reported_epoch":337,"state":"active+clean","last_fresh":"2024-09-16T09:36:57.760642+0000","last_change":"2024-09-16T09:34:30.458186+0000","last_active":"2024-09-16T09:36:57.760642+0000","last_peered":"2024-09-16T09:36:57.760642+0000","last_clean":"2024-09-16T09:36:57.760642+0000","last_became_active":"2024-09-16T09:25:07.718899+0000","last_became_peered":"2024-09-16T09:25:07.718899+0000","last_unstale":"2024-09-16T09:36:57.760642+0000","last_undegraded":"2024-09-16T09:36:57.760642+0000","last_fullsized":"2024-09-16T09:36:57.760642+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:30.458115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:34:30.458115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.5","version":"331'8327","reported_seq":12092,"reported_epoch":333,"state":"clean+premerge+peered","last_fresh":"2024-09-16T09:36:53.758581+0000","last_change":"2024-09-16T09:36:53.758466+0000","last_active":"2024-09-16T09:36:52.736119+0000","last_peered":"2024-09-16T09:36:53.758581+0000","last_clean":"2024-09-16T09:36:53.758581+0000","last_became_active":"2024-09-16T09:34:32.381065+0000","last_became_peered":"2024-09-16T09:36:53.735990+0000","last_unstale":"2024-09-16T09:36:53.758581+0000","last_undegraded":"2024-09-16T09:36:53.758581+0000","last_fullsized":"2024-09-16T09:36:53.758581+0000","mapping_epoch":267,"log_start":"247'3310","ondisk_log_start":"247'3310","created":229,"last_epoch_clean":333,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'4368","last_scrub_stamp":"2024-09-16T09:34:34.439986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:34:34.439986+0000","objects_scrubbed":2756,"log_size":5017,"log_dups_size":0,"ondisk_log_size":5017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":710,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63700992,"num_objects":972,"num_object_clones":0,"num_object_copies":2916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":972,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3970,"num_write_kb":69952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.d","version":"73'79","reported_seq":384,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-16T09:36:56.761039+0000","last_change":"2024-09-16T09:32:44.536868+0000","last_active":"2024-09-16T09:36:56.761039+0000","last_peered":"2024-09-16T09:36:56.761039+0000","last_clean":"2024-09-16T09:36:56.761039+0000","last_became_active":"2024-09-16T09:30:10.709958+0000","last_became_peered":"2024-09-16T09:30:10.709958+0000","last_unstale":"2024-09-16T09:36:56.761039+0000","last_undegraded":"2024-09-16T09:36:56.761039+0000","last_fullsized":"2024-09-16T09:36:56.761039+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:32:44.536799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:32:44.536799+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"285'27","reported_seq":619,"reported_epoch":337,"state":"active+clean","last_fresh":"2024-09-16T09:36:58.027790+0000","last_change":"2024-09-16T09:26:33.138828+0000","last_active":"2024-09-16T09:36:58.027790+0000","last_peered":"2024-09-16T09:36:58.027790+0000","last_clean":"2024-09-16T09:36:58.027790+0000","last_became_active":"2024-09-16T09:25:03.715733+0000","last_became_peered":"2024-09-16T09:25:03.715733+0000","last_unstale":"2024-09-16T09:36:58.027790+0000","last_undegraded":"2024-09-16T09:36:58.027790+0000","last_fullsized":"2024-09-16T09:36:58.027790+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-16T09:26:33.138603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:26:33.138603+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:37:16.922468+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":598,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-16T09:36:56.761726+0000","last_change":"2024-09-16T09:32:49.441194+0000","last_active":"2024-09-16T09:36:56.761726+0000","last_peered":"2024-09-16T09:36:56.761726+0000","last_clean":"2024-09-16T09:36:56.761726+0000","last_became_active":"2024-09-16T09:25:01.707372+0000","last_became_peered":"2024-09-16T09:25:01.707372+0000","last_unstale":"2024-09-16T09:36:56.761726+0000","last_undegraded":"2024-09-16T09:36:56.761726+0000","last_fullsized":"2024-09-16T09:36:56.761726+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-16T09:32:49.441122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:32:49.441122+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"66'8","reported_seq":738,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-16T09:36:56.761771+0000","last_change":"2024-09-16T09:30:05.138362+0000","last_active":"2024-09-16T09:36:56.761771+0000","last_peered":"2024-09-16T09:36:56.761771+0000","last_clean":"2024-09-16T09:36:56.761771+0000","last_became_active":"2024-09-16T09:25:05.716189+0000","last_became_peered":"2024-09-16T09:25:05.716189+0000","last_unstale":"2024-09-16T09:36:56.761771+0000","last_undegraded":"2024-09-16T09:36:56.761771+0000","last_fullsized":"2024-09-16T09:36:56.761771+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:30:05.138288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:30:05.138288+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":568,"reported_epoch":337,"state":"active+clean","last_fresh":"2024-09-16T09:36:58.027949+0000","last_change":"2024-09-16T09:27:20.219376+0000","last_active":"2024-09-16T09:36:58.027949+0000","last_peered":"2024-09-16T09:36:58.027949+0000","last_clean":"2024-09-16T09:36:58.027949+0000","last_became_active":"2024-09-16T09:25:07.717213+0000","last_became_peered":"2024-09-16T09:25:07.717213+0000","last_unstale":"2024-09-16T09:36:58.027949+0000","last_undegraded":"2024-09-16T09:36:58.027949+0000","last_fullsized":"2024-09-16T09:36:58.027949+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:20.219333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:20.219333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:37:23.878316+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.12","version":"73'79","reported_seq":393,"reported_epoch":337,"state":"active+clean","last_fresh":"2024-09-16T09:36:58.028121+0000","last_change":"2024-09-16T09:36:44.171217+0000","last_active":"2024-09-16T09:36:58.028121+0000","last_peered":"2024-09-16T09:36:58.028121+0000","last_clean":"2024-09-16T09:36:58.028121+0000","last_became_active":"2024-09-16T09:30:10.731413+0000","last_became_peered":"2024-09-16T09:30:10.731413+0000","last_unstale":"2024-09-16T09:36:58.028121+0000","last_undegraded":"2024-09-16T09:36:58.028121+0000","last_fullsized":"2024-09-16T09:36:58.028121+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:36:44.171151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:36:44.171151+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:37:50.857154+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":594,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-16T09:36:56.769121+0000","last_change":"2024-09-16T09:34:00.334779+0000","last_active":"2024-09-16T09:36:56.769121+0000","last_peered":"2024-09-16T09:36:56.769121+0000","last_clean":"2024-09-16T09:36:56.769121+0000","last_became_active":"2024-09-16T09:34:00.334361+0000","last_became_peered":"2024-09-16T09:34:00.334361+0000","last_unstale":"2024-09-16T09:36:56.769121+0000","last_undegraded":"2024-09-16T09:36:56.769121+0000","last_fullsized":"2024-09-16T09:36:56.769121+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:14.845355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:14.845355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"285'12","reported_seq":347,"reported_epoch":337,"state":"active+clean","last_fresh":"2024-09-16T09:36:57.762085+0000","last_change":"2024-09-16T09:34:51.181994+0000","last_active":"2024-09-16T09:36:57.762085+0000","last_peered":"2024-09-16T09:36:57.762085+0000","last_clean":"2024-09-16T09:36:57.762085+0000","last_became_active":"2024-09-16T09:31:09.270685+0000","last_became_peered":"2024-09-16T09:31:09.270685+0000","last_unstale":"2024-09-16T09:36:57.762085+0000","last_undegraded":"2024-09-16T09:36:57.762085+0000","last_fullsized":"2024-09-16T09:36:57.762085+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'6","last_scrub_stamp":"2024-09-16T09:34:51.181918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:34:51.181918+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":17,"num_read_kb":14,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":587,"reported_epoch":337,"state":"active+clean","last_fresh":"2024-09-16T09:36:58.028466+0000","last_change":"2024-09-16T09:36:38.129449+0000","last_active":"2024-09-16T09:36:58.028466+0000","last_peered":"2024-09-16T09:36:58.028466+0000","last_clean":"2024-09-16T09:36:58.028466+0000","last_became_active":"2024-09-16T09:25:05.715616+0000","last_became_peered":"2024-09-16T09:25:05.715616+0000","last_unstale":"2024-09-16T09:36:58.028466+0000","last_undegraded":"2024-09-16T09:36:58.028466+0000","last_fullsized":"2024-09-16T09:36:58.028466+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:38.129314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:36:38.129314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:37:44.283058+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":573,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-16T09:36:56.761092+0000","last_change":"2024-09-16T09:36:37.715166+0000","last_active":"2024-09-16T09:36:56.761092+0000","last_peered":"2024-09-16T09:36:56.761092+0000","last_clean":"2024-09-16T09:36:56.761092+0000","last_became_active":"2024-09-16T09:25:07.721059+0000","last_became_peered":"2024-09-16T09:25:07.721059+0000","last_unstale":"2024-09-16T09:36:56.761092+0000","last_undegraded":"2024-09-16T09:36:56.761092+0000","last_fullsized":"2024-09-16T09:36:56.761092+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:37.715114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:36:37.715114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:37:58.590391+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.13","version":"73'79","reported_seq":479,"reported_epoch":337,"state":"active+clean","last_fresh":"2024-09-16T09:36:57.760773+0000","last_change":"2024-09-16T09:34:48.260121+0000","last_active":"2024-09-16T09:36:57.760773+0000","last_peered":"2024-09-16T09:36:57.760773+0000","last_clean":"2024-09-16T09:36:57.760773+0000","last_became_active":"2024-09-16T09:33:30.248497+0000","last_became_peered":"2024-09-16T09:33:30.248497+0000","last_unstale":"2024-09-16T09:36:57.760773+0000","last_undegraded":"2024-09-16T09:36:57.760773+0000","last_fullsized":"2024-09-16T09:36:57.760773+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:34:48.260041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:34:48.260041+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"285'15","reported_seq":659,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-09-16T09:36:53.733676+0000","last_change":"2024-09-16T09:33:48.681742+0000","last_active":"2024-09-16T09:36:53.733676+0000","last_peered":"2024-09-16T09:36:53.733676+0000","last_clean":"2024-09-16T09:36:53.733676+0000","last_became_active":"2024-09-16T09:31:09.273498+0000","last_became_peered":"2024-09-16T09:31:09.273498+0000","last_unstale":"2024-09-16T09:36:53.733676+0000","last_undegraded":"2024-09-16T09:36:53.733676+0000","last_fullsized":"2024-09-16T09:36:53.733676+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:33:48.681650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:33:48.681650+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":594,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-16T09:36:56.762075+0000","last_change":"2024-09-16T09:33:13.312434+0000","last_active":"2024-09-16T09:36:56.762075+0000","last_peered":"2024-09-16T09:36:56.762075+0000","last_clean":"2024-09-16T09:36:56.762075+0000","last_became_active":"2024-09-16T09:25:01.706491+0000","last_became_peered":"2024-09-16T09:25:01.706491+0000","last_unstale":"2024-09-16T09:36:56.762075+0000","last_undegraded":"2024-09-16T09:36:56.762075+0000","last_fullsized":"2024-09-16T09:36:56.762075+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:13.312344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:13.312344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":351,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-16T09:36:56.761336+0000","last_change":"2024-09-16T09:36:54.596000+0000","last_active":"2024-09-16T09:36:56.761336+0000","last_peered":"2024-09-16T09:36:56.761336+0000","last_clean":"2024-09-16T09:36:56.761336+0000","last_became_active":"2024-09-16T09:33:30.248992+0000","last_became_peered":"2024-09-16T09:33:30.248992+0000","last_unstale":"2024-09-16T09:36:56.761336+0000","last_undegraded":"2024-09-16T09:36:56.761336+0000","last_fullsized":"2024-09-16T09:36:56.761336+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:54.595924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:36:54.595924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:10.833233+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":583,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-16T09:36:56.761616+0000","last_change":"2024-09-16T09:34:32.309307+0000","last_active":"2024-09-16T09:36:56.761616+0000","last_peered":"2024-09-16T09:36:56.761616+0000","last_clean":"2024-09-16T09:36:56.761616+0000","last_became_active":"2024-09-16T09:25:07.722050+0000","last_became_peered":"2024-09-16T09:25:07.722050+0000","last_unstale":"2024-09-16T09:36:56.761616+0000","last_undegraded":"2024-09-16T09:36:56.761616+0000","last_fullsized":"2024-09-16T09:36:56.761616+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:32.309262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:34:32.309262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.10","version":"73'79","reported_seq":393,"reported_epoch":337,"state":"active+clean","last_fresh":"2024-09-16T09:36:58.027666+0000","last_change":"2024-09-16T09:36:41.169117+0000","last_active":"2024-09-16T09:36:58.027666+0000","last_peered":"2024-09-16T09:36:58.027666+0000","last_clean":"2024-09-16T09:36:58.027666+0000","last_became_active":"2024-09-16T09:30:10.692770+0000","last_became_peered":"2024-09-16T09:30:10.692770+0000","last_unstale":"2024-09-16T09:36:58.027666+0000","last_undegraded":"2024-09-16T09:36:58.027666+0000","last_fullsized":"2024-09-16T09:36:58.027666+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:36:41.169069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:36:41.169069+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:08.440462+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"285'15","reported_seq":608,"reported_epoch":337,"state":"active+clean","last_fresh":"2024-09-16T09:36:58.028608+0000","last_change":"2024-09-16T09:29:32.485165+0000","last_active":"2024-09-16T09:36:58.028608+0000","last_peered":"2024-09-16T09:36:58.028608+0000","last_clean":"2024-09-16T09:36:58.028608+0000","last_became_active":"2024-09-16T09:25:03.717310+0000","last_became_peered":"2024-09-16T09:25:03.717310+0000","last_unstale":"2024-09-16T09:36:58.028608+0000","last_undegraded":"2024-09-16T09:36:58.028608+0000","last_fullsized":"2024-09-16T09:36:58.028608+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:29:32.485084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:29:32.485084+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:37:35.087553+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":611,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-09-16T09:36:53.733289+0000","last_change":"2024-09-16T09:33:20.639405+0000","last_active":"2024-09-16T09:36:53.733289+0000","last_peered":"2024-09-16T09:36:53.733289+0000","last_clean":"2024-09-16T09:36:53.733289+0000","last_became_active":"2024-09-16T09:31:09.272848+0000","last_became_peered":"2024-09-16T09:31:09.272848+0000","last_unstale":"2024-09-16T09:36:53.733289+0000","last_undegraded":"2024-09-16T09:36:53.733289+0000","last_fullsized":"2024-09-16T09:36:53.733289+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:20.639350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:20.639350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"187'11","reported_seq":575,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-16T09:36:56.761594+0000","last_change":"2024-09-16T09:33:20.296801+0000","last_active":"2024-09-16T09:36:56.761594+0000","last_peered":"2024-09-16T09:36:56.761594+0000","last_clean":"2024-09-16T09:36:56.761594+0000","last_became_active":"2024-09-16T09:31:09.277688+0000","last_became_peered":"2024-09-16T09:31:09.277688+0000","last_unstale":"2024-09-16T09:36:56.761594+0000","last_undegraded":"2024-09-16T09:36:56.761594+0000","last_fullsized":"2024-09-16T09:36:56.761594+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"187'11","last_scrub_stamp":"2024-09-16T09:33:20.296746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:20.296746+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":600,"reported_epoch":337,"state":"active+clean","last_fresh":"2024-09-16T09:36:57.765027+0000","last_change":"2024-09-16T09:36:49.628523+0000","last_active":"2024-09-16T09:36:57.765027+0000","last_peered":"2024-09-16T09:36:57.765027+0000","last_clean":"2024-09-16T09:36:57.765027+0000","last_became_active":"2024-09-16T09:25:07.718810+0000","last_became_peered":"2024-09-16T09:25:07.718810+0000","last_unstale":"2024-09-16T09:36:57.765027+0000","last_undegraded":"2024-09-16T09:36:57.765027+0000","last_fullsized":"2024-09-16T09:36:57.765027+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:49.628477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:36:49.628477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:37:58.131251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.11","version":"73'79","reported_seq":463,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-09-16T09:36:53.734119+0000","last_change":"2024-09-16T09:32:38.401888+0000","last_active":"2024-09-16T09:36:53.734119+0000","last_peered":"2024-09-16T09:36:53.734119+0000","last_clean":"2024-09-16T09:36:53.734119+0000","last_became_active":"2024-09-16T09:31:09.272208+0000","last_became_peered":"2024-09-16T09:31:09.272208+0000","last_unstale":"2024-09-16T09:36:53.734119+0000","last_undegraded":"2024-09-16T09:36:53.734119+0000","last_fullsized":"2024-09-16T09:36:53.734119+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:32:38.401720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:32:38.401720+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":32,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.14","version":"285'20","reported_seq":276,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-16T09:36:56.761919+0000","last_change":"2024-09-16T09:36:52.609515+0000","last_active":"2024-09-16T09:36:56.761919+0000","last_peered":"2024-09-16T09:36:56.761919+0000","last_clean":"2024-09-16T09:36:56.761919+0000","last_became_active":"2024-09-16T09:33:30.249769+0000","last_became_peered":"2024-09-16T09:33:30.249769+0000","last_unstale":"2024-09-16T09:36:56.761919+0000","last_undegraded":"2024-09-16T09:36:56.761919+0000","last_fullsized":"2024-09-16T09:36:56.761919+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'20","last_scrub_stamp":"2024-09-16T09:36:52.609440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:36:52.609440+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:37:58.502398+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":586,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-16T09:36:56.761623+0000","last_change":"2024-09-16T09:29:56.021413+0000","last_active":"2024-09-16T09:36:56.761623+0000","last_peered":"2024-09-16T09:36:56.761623+0000","last_clean":"2024-09-16T09:36:56.761623+0000","last_became_active":"2024-09-16T09:25:01.707513+0000","last_became_peered":"2024-09-16T09:25:01.707513+0000","last_unstale":"2024-09-16T09:36:56.761623+0000","last_undegraded":"2024-09-16T09:36:56.761623+0000","last_fullsized":"2024-09-16T09:36:56.761623+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:56.021340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:56.021340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"66'8","reported_seq":747,"reported_epoch":337,"state":"active+clean","last_fresh":"2024-09-16T09:36:58.028402+0000","last_change":"2024-09-16T09:33:39.388721+0000","last_active":"2024-09-16T09:36:58.028402+0000","last_peered":"2024-09-16T09:36:58.028402+0000","last_clean":"2024-09-16T09:36:58.028402+0000","last_became_active":"2024-09-16T09:25:05.713833+0000","last_became_peered":"2024-09-16T09:25:05.713833+0000","last_unstale":"2024-09-16T09:36:58.028402+0000","last_undegraded":"2024-09-16T09:36:58.028402+0000","last_fullsized":"2024-09-16T09:36:58.028402+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:33:39.388463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:39.388463+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":598,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-09-16T09:36:53.733289+0000","last_change":"2024-09-16T09:33:42.666290+0000","last_active":"2024-09-16T09:36:53.733289+0000","last_peered":"2024-09-16T09:36:53.733289+0000","last_clean":"2024-09-16T09:36:53.733289+0000","last_became_active":"2024-09-16T09:31:09.275168+0000","last_became_peered":"2024-09-16T09:31:09.275168+0000","last_unstale":"2024-09-16T09:36:53.733289+0000","last_undegraded":"2024-09-16T09:36:53.733289+0000","last_fullsized":"2024-09-16T09:36:53.733289+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:42.666200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:33:42.666200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"285'19","reported_seq":627,"reported_epoch":337,"state":"active+clean","last_fresh":"2024-09-16T09:36:57.764233+0000","last_change":"2024-09-16T09:33:19.084118+0000","last_active":"2024-09-16T09:36:57.764233+0000","last_peered":"2024-09-16T09:36:57.764233+0000","last_clean":"2024-09-16T09:36:57.764233+0000","last_became_active":"2024-09-16T09:26:33.291265+0000","last_became_peered":"2024-09-16T09:26:33.291265+0000","last_unstale":"2024-09-16T09:36:57.764233+0000","last_undegraded":"2024-09-16T09:36:57.764233+0000","last_fullsized":"2024-09-16T09:36:57.764233+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-16T09:33:19.084063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:33:19.084063+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:37:13.780897+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":605,"reported_epoch":337,"state":"active+clean","last_fresh":"2024-09-16T09:36:57.764827+0000","last_change":"2024-09-16T09:35:24.722539+0000","last_active":"2024-09-16T09:36:57.764827+0000","last_peered":"2024-09-16T09:36:57.764827+0000","last_clean":"2024-09-16T09:36:57.764827+0000","last_became_active":"2024-09-16T09:25:01.705158+0000","last_became_peered":"2024-09-16T09:25:01.705158+0000","last_unstale":"2024-09-16T09:36:57.764827+0000","last_undegraded":"2024-09-16T09:36:57.764827+0000","last_fullsized":"2024-09-16T09:36:57.764827+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:35:24.722476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:35:24.722476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":517,"reported_epoch":337,"state":"active+clean","last_fresh":"2024-09-16T09:36:58.028192+0000","last_change":"2024-09-16T09:34:00.329968+0000","last_active":"2024-09-16T09:36:58.028192+0000","last_peered":"2024-09-16T09:36:58.028192+0000","last_clean":"2024-09-16T09:36:58.028192+0000","last_became_active":"2024-09-16T09:34:00.329642+0000","last_became_peered":"2024-09-16T09:34:00.329642+0000","last_unstale":"2024-09-16T09:36:58.028192+0000","last_undegraded":"2024-09-16T09:36:58.028192+0000","last_fullsized":"2024-09-16T09:36:58.028192+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:08.394830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:08.394830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":431,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-09-16T09:36:53.732795+0000","last_change":"2024-09-16T09:34:48.786847+0000","last_active":"2024-09-16T09:36:53.732795+0000","last_peered":"2024-09-16T09:36:53.732795+0000","last_clean":"2024-09-16T09:36:53.732795+0000","last_became_active":"2024-09-16T09:31:09.276042+0000","last_became_peered":"2024-09-16T09:31:09.276042+0000","last_unstale":"2024-09-16T09:36:53.732795+0000","last_undegraded":"2024-09-16T09:36:53.732795+0000","last_fullsized":"2024-09-16T09:36:53.732795+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:48.786785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:34:48.786785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"285'15","reported_seq":288,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-16T09:36:56.769991+0000","last_change":"2024-09-16T09:36:55.936077+0000","last_active":"2024-09-16T09:36:56.769991+0000","last_peered":"2024-09-16T09:36:56.769991+0000","last_clean":"2024-09-16T09:36:56.769991+0000","last_became_active":"2024-09-16T09:34:00.331466+0000","last_became_peered":"2024-09-16T09:34:00.331466+0000","last_unstale":"2024-09-16T09:36:56.769991+0000","last_undegraded":"2024-09-16T09:36:56.769991+0000","last_fullsized":"2024-09-16T09:36:56.769991+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'15","last_scrub_stamp":"2024-09-16T09:36:55.935996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:36:55.935996+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:24.990467+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,0],"acting":[6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":585,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-16T09:36:56.761551+0000","last_change":"2024-09-16T09:32:47.502667+0000","last_active":"2024-09-16T09:36:56.761551+0000","last_peered":"2024-09-16T09:36:56.761551+0000","last_clean":"2024-09-16T09:36:56.761551+0000","last_became_active":"2024-09-16T09:25:01.706762+0000","last_became_peered":"2024-09-16T09:25:01.706762+0000","last_unstale":"2024-09-16T09:36:56.761551+0000","last_undegraded":"2024-09-16T09:36:56.761551+0000","last_fullsized":"2024-09-16T09:36:56.761551+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:47.502598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:32:47.502598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":416,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-09-16T09:36:53.733371+0000","last_change":"2024-09-16T09:34:46.742640+0000","last_active":"2024-09-16T09:36:53.733371+0000","last_peered":"2024-09-16T09:36:53.733371+0000","last_clean":"2024-09-16T09:36:53.733371+0000","last_became_active":"2024-09-16T09:28:52.982865+0000","last_became_peered":"2024-09-16T09:28:52.982865+0000","last_unstale":"2024-09-16T09:36:53.733371+0000","last_undegraded":"2024-09-16T09:36:53.733371+0000","last_fullsized":"2024-09-16T09:36:53.733371+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:46.742562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:34:46.742562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":577,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-09-16T09:36:53.732720+0000","last_change":"2024-09-16T09:34:50.779241+0000","last_active":"2024-09-16T09:36:53.732720+0000","last_peered":"2024-09-16T09:36:53.732720+0000","last_clean":"2024-09-16T09:36:53.732720+0000","last_became_active":"2024-09-16T09:25:07.715301+0000","last_became_peered":"2024-09-16T09:25:07.715301+0000","last_unstale":"2024-09-16T09:36:53.732720+0000","last_undegraded":"2024-09-16T09:36:53.732720+0000","last_fullsized":"2024-09-16T09:36:53.732720+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:50.779164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:34:50.779164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.14","version":"73'79","reported_seq":391,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-16T09:36:56.761520+0000","last_change":"2024-09-16T09:36:48.597220+0000","last_active":"2024-09-16T09:36:56.761520+0000","last_peered":"2024-09-16T09:36:56.761520+0000","last_clean":"2024-09-16T09:36:56.761520+0000","last_became_active":"2024-09-16T09:30:10.700866+0000","last_became_peered":"2024-09-16T09:30:10.700866+0000","last_unstale":"2024-09-16T09:36:56.761520+0000","last_undegraded":"2024-09-16T09:36:56.761520+0000","last_fullsized":"2024-09-16T09:36:56.761520+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:36:48.597169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:36:48.597169+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:15.020958+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"285'19","reported_seq":473,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-16T09:36:56.761969+0000","last_change":"2024-09-16T09:28:55.643505+0000","last_active":"2024-09-16T09:36:56.761969+0000","last_peered":"2024-09-16T09:36:56.761969+0000","last_clean":"2024-09-16T09:36:56.761969+0000","last_became_active":"2024-09-16T09:28:52.971132+0000","last_became_peered":"2024-09-16T09:28:52.971132+0000","last_unstale":"2024-09-16T09:36:56.761969+0000","last_undegraded":"2024-09-16T09:36:56.761969+0000","last_fullsized":"2024-09-16T09:36:56.761969+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-16T09:28:55.643448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:55.643448+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":595,"reported_epoch":337,"state":"active+clean","last_fresh":"2024-09-16T09:36:58.028552+0000","last_change":"2024-09-16T09:33:18.297533+0000","last_active":"2024-09-16T09:36:58.028552+0000","last_peered":"2024-09-16T09:36:58.028552+0000","last_clean":"2024-09-16T09:36:58.028552+0000","last_became_active":"2024-09-16T09:25:01.700733+0000","last_became_peered":"2024-09-16T09:25:01.700733+0000","last_unstale":"2024-09-16T09:36:58.028552+0000","last_undegraded":"2024-09-16T09:36:58.028552+0000","last_fullsized":"2024-09-16T09:36:58.028552+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:18.297451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:18.297451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":586,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-16T09:36:56.760665+0000","last_change":"2024-09-16T09:33:38.336281+0000","last_active":"2024-09-16T09:36:56.760665+0000","last_peered":"2024-09-16T09:36:56.760665+0000","last_clean":"2024-09-16T09:36:56.760665+0000","last_became_active":"2024-09-16T09:25:05.713479+0000","last_became_peered":"2024-09-16T09:25:05.713479+0000","last_unstale":"2024-09-16T09:36:56.760665+0000","last_undegraded":"2024-09-16T09:36:56.760665+0000","last_fullsized":"2024-09-16T09:36:56.760665+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:38.336221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:38.336221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":439,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-09-16T09:36:53.733870+0000","last_change":"2024-09-16T09:34:51.781047+0000","last_active":"2024-09-16T09:36:53.733870+0000","last_peered":"2024-09-16T09:36:53.733870+0000","last_clean":"2024-09-16T09:36:53.733870+0000","last_became_active":"2024-09-16T09:31:09.274797+0000","last_became_peered":"2024-09-16T09:31:09.274797+0000","last_unstale":"2024-09-16T09:36:53.733870+0000","last_undegraded":"2024-09-16T09:36:53.733870+0000","last_fullsized":"2024-09-16T09:36:53.733870+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:51.780980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:34:51.780980+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"285'8","reported_seq":342,"reported_epoch":337,"state":"active+clean","last_fresh":"2024-09-16T09:36:57.759963+0000","last_change":"2024-09-16T09:34:50.206999+0000","last_active":"2024-09-16T09:36:57.759963+0000","last_peered":"2024-09-16T09:36:57.759963+0000","last_clean":"2024-09-16T09:36:57.759963+0000","last_became_active":"2024-09-16T09:31:09.273682+0000","last_became_peered":"2024-09-16T09:31:09.273682+0000","last_unstale":"2024-09-16T09:36:57.759963+0000","last_undegraded":"2024-09-16T09:36:57.759963+0000","last_fullsized":"2024-09-16T09:36:57.759963+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'4","last_scrub_stamp":"2024-09-16T09:34:50.206916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:34:50.206916+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":9,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":610,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-09-16T09:36:53.733977+0000","last_change":"2024-09-16T09:33:18.641115+0000","last_active":"2024-09-16T09:36:53.733977+0000","last_peered":"2024-09-16T09:36:53.733977+0000","last_clean":"2024-09-16T09:36:53.733977+0000","last_became_active":"2024-09-16T09:31:09.274719+0000","last_became_peered":"2024-09-16T09:31:09.274719+0000","last_unstale":"2024-09-16T09:36:53.733977+0000","last_undegraded":"2024-09-16T09:36:53.733977+0000","last_fullsized":"2024-09-16T09:36:53.733977+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:18.641069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:18.641069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":603,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-16T09:36:56.770587+0000","last_change":"2024-09-16T09:33:37.941485+0000","last_active":"2024-09-16T09:36:56.770587+0000","last_peered":"2024-09-16T09:36:56.770587+0000","last_clean":"2024-09-16T09:36:56.770587+0000","last_became_active":"2024-09-16T09:25:05.712545+0000","last_became_peered":"2024-09-16T09:25:05.712545+0000","last_unstale":"2024-09-16T09:36:56.770587+0000","last_undegraded":"2024-09-16T09:36:56.770587+0000","last_fullsized":"2024-09-16T09:36:56.770587+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:37.941403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:37.941403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":582,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-16T09:36:56.760922+0000","last_change":"2024-09-16T09:33:41.323824+0000","last_active":"2024-09-16T09:36:56.760922+0000","last_peered":"2024-09-16T09:36:56.760922+0000","last_clean":"2024-09-16T09:36:56.760922+0000","last_became_active":"2024-09-16T09:25:07.720117+0000","last_became_peered":"2024-09-16T09:25:07.720117+0000","last_unstale":"2024-09-16T09:36:56.760922+0000","last_undegraded":"2024-09-16T09:36:56.760922+0000","last_fullsized":"2024-09-16T09:36:56.760922+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:41.323773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:33:41.323773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.12","version":"336'8461","reported_seq":23646,"reported_epoch":337,"state":"active+clean","last_fresh":"2024-09-16T09:36:57.760427+0000","last_change":"2024-09-16T09:36:48.755566+0000","last_active":"2024-09-16T09:36:57.760427+0000","last_peered":"2024-09-16T09:36:57.760427+0000","last_clean":"2024-09-16T09:36:57.760427+0000","last_became_active":"2024-09-16T09:36:48.730974+0000","last_became_peered":"2024-09-16T09:36:48.730974+0000","last_unstale":"2024-09-16T09:36:57.760427+0000","last_undegraded":"2024-09-16T09:36:57.760427+0000","last_fullsized":"2024-09-16T09:36:57.760427+0000","mapping_epoch":327,"log_start":"269'4483","ondisk_log_start":"269'4483","created":243,"last_epoch_clean":328,"parent":"0.0","parent_split_bits":5,"last_scrub":"274'4970","last_scrub_stamp":"2024-09-16T09:34:48.381443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:34:48.381443+0000","objects_scrubbed":3385,"log_size":3978,"log_dups_size":0,"ondisk_log_size":3978,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":905,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":55836672,"num_objects":852,"num_object_clones":0,"num_object_copies":2556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":852,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3509,"num_write_kb":31232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3873,"num_bytes_recovered":229834752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":228,"reported_epoch":337,"state":"active+clean","last_fresh":"2024-09-16T09:36:58.028437+0000","last_change":"2024-09-16T09:36:52.091335+0000","last_active":"2024-09-16T09:36:58.028437+0000","last_peered":"2024-09-16T09:36:58.028437+0000","last_clean":"2024-09-16T09:36:58.028437+0000","last_became_active":"2024-09-16T09:33:30.248036+0000","last_became_peered":"2024-09-16T09:33:30.248036+0000","last_unstale":"2024-09-16T09:36:58.028437+0000","last_undegraded":"2024-09-16T09:36:58.028437+0000","last_fullsized":"2024-09-16T09:36:58.028437+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:52.088284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:36:52.088284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:37:59.806406+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":438,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-09-16T09:36:53.734132+0000","last_change":"2024-09-16T09:32:40.411546+0000","last_active":"2024-09-16T09:36:53.734132+0000","last_peered":"2024-09-16T09:36:53.734132+0000","last_clean":"2024-09-16T09:36:53.734132+0000","last_became_active":"2024-09-16T09:31:09.271662+0000","last_became_peered":"2024-09-16T09:31:09.271662+0000","last_unstale":"2024-09-16T09:36:53.734132+0000","last_undegraded":"2024-09-16T09:36:53.734132+0000","last_fullsized":"2024-09-16T09:36:53.734132+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:40.411487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:32:40.411487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"285'19","reported_seq":622,"reported_epoch":336,"state":"active+clean","last_fresh":"2024-09-16T09:36:56.770186+0000","last_change":"2024-09-16T09:33:19.843216+0000","last_active":"2024-09-16T09:36:56.770186+0000","last_peered":"2024-09-16T09:36:56.770186+0000","last_clean":"2024-09-16T09:36:56.770186+0000","last_became_active":"2024-09-16T09:25:03.707769+0000","last_became_peered":"2024-09-16T09:25:03.707769+0000","last_unstale":"2024-09-16T09:36:56.770186+0000","last_undegraded":"2024-09-16T09:36:56.770186+0000","last_fullsized":"2024-09-16T09:36:56.770186+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-16T09:33:19.843154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:33:19.843154+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":573,"reported_epoch":333,"state":"active+clean","last_fresh":"2024-09-16T09:36:53.734080+0000","last_change":"2024-09-16T09:32:31.325543+0000","last_active":"2024-09-16T09:36:53.734080+0000","last_peered":"2024-09-16T09:36:53.734080+0000","last_clean":"2024-09-16T09:36:53.734080+0000","last_became_active":"2024-09-16T09:25:05.711952+0000","last_became_peered":"2024-09-16T09:25:05.711952+0000","last_unstale":"2024-09-16T09:36:53.734080+0000","last_undegraded":"2024-09-16T09:36:53.734080+0000","last_fullsized":"2024-09-16T09:36:53.734080+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:31.325346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:32:31.325346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":20,"stat_sum":{"num_bytes":1697120280,"num_objects":25897,"num_object_clones":0,"num_object_copies":77691,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25897,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":140437,"num_write_kb":3816897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23087,"num_bytes_recovered":1478230016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":6405369856,"data_stored":6405357640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74416,"ondisk_log_size":74416,"up":60,"acting":60,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":1117,"num_read_kb":874,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":120,"num_bytes_recovered":3026,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":681,"ondisk_log_size":681,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1986,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":710,"num_read_kb":705,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":21,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":327680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1659,"ondisk_log_size":1659,"up":63,"acting":63,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266836,"num_pgs":75,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1457112,"kb_used_data":974716,"kb_used_omap":1,"kb_used_meta":482366,"kb_avail":92288040,"statfs":{"total":95995035648,"available":94502952960,"internally_reserved":0,"allocated":998109184,"data_stored":997687851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":493943242},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495769,"num_pgs":61,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":952368,"kb_used_data":600472,"kb_used_omap":1,"kb_used_meta":351870,"kb_avail":92792784,"statfs":{"total":95995035648,"available":95019810816,"internally_reserved":0,"allocated":614883328,"data_stored":614497951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":360315334},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724704,"num_pgs":65,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1743796,"kb_used_data":1175332,"kb_used_omap":1,"kb_used_meta":568446,"kb_avail":92001356,"statfs":{"total":95995035648,"available":94209388544,"internally_reserved":0,"allocated":1203539968,"data_stored":1203142362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":582089158},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953637,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1099468,"kb_used_data":712880,"kb_used_omap":1,"kb_used_meta":386558,"kb_avail":92645684,"statfs":{"total":95995035648,"available":94869180416,"internally_reserved":0,"allocated":729989120,"data_stored":729579699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":395835846},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149867,"num_pgs":77,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1342884,"kb_used_data":761932,"kb_used_omap":1,"kb_used_meta":580926,"kb_avail":92402268,"statfs":{"total":95995035648,"available":94619922432,"internally_reserved":0,"allocated":780218368,"data_stored":779789488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":594868679},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411504,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":738860,"kb_used_data":324500,"kb_used_omap":1,"kb_used_meta":414334,"kb_avail":93006292,"statfs":{"total":95995035648,"available":95238443008,"internally_reserved":0,"allocated":332288000,"data_stored":331906541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":424278469},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575031,"num_pgs":72,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1707736,"kb_used_data":1090692,"kb_used_omap":1,"kb_used_meta":617022,"kb_avail":92037416,"statfs":{"total":95995035648,"available":94246313984,"internally_reserved":0,"allocated":1116868608,"data_stored":1116446410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":631830981},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,1,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738556,"num_pgs":61,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1107292,"kb_used_data":634628,"kb_used_omap":1,"kb_used_meta":472638,"kb_avail":92637860,"statfs":{"total":95995035648,"available":94861168640,"internally_reserved":0,"allocated":649859072,"data_stored":649461685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":483981765},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":644546560,"data_stored":644546560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1115750400,"data_stored":1115750400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":330956800,"data_stored":330956800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":778502144,"data_stored":778502144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":728895488,"data_stored":728891416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1202458624,"data_stored":1202454552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":607256576,"data_stored":607256576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":997003264,"data_stored":996999192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T09:37:01.306 INFO:tasks.thrashosds.thrasher:Setting 2.1 to [3, 0, 2, 1, 4, 6] 2024-09-16T09:37:01.306 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '2.1', '3', '0', '2', '1', '4', '6'] 2024-09-16T09:37:01.307 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd pg-upmap-items 2.1 3 0 2 1 4 6 2024-09-16T09:37:01.382 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:37:01.404 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:01.405 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:01.405 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:37:01.506 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:37:01.527 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:01.528 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:01.528 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:37:01.629 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:37:01.650 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:01.651 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:01.651 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:37:01.752 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:37:01.774 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:01.774 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:01.775 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:37:01.868 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:37:01.875 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:37:01.897 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:01.897 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:01.897 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:37:01.998 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:37:02.020 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:02.020 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:02.021 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:37:02.122 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:37:02.144 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:02.144 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:02.144 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:37:02.170 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:37:01 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:37:01] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:37:02.246 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:37:02.268 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:02.268 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:02.269 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:37:02.310 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.3 dump_ops_in_flight 2024-09-16T09:37:02.335 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:37:02.369 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:37:02.392 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:02.392 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:02.393 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:37:02.494 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:37:02.515 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:02.516 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:02.516 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:37:02.548 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:02 smithi070 ceph-mon[30388]: osdmap e341: 8 total, 8 up, 8 in 2024-09-16T09:37:02.548 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:02 smithi070 ceph-mon[40691]: osdmap e341: 8 total, 8 up, 8 in 2024-09-16T09:37:02.617 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:37:02.638 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:02.639 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:02.639 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:37:02.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:02 smithi084 ceph-mon[32629]: osdmap e341: 8 total, 8 up, 8 in 2024-09-16T09:37:02.741 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:37:02.763 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:02.764 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:02.764 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:37:02.865 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:37:02.886 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:02.887 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:02.887 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:37:02.988 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:37:03.009 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:03.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:03.011 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:37:03.112 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:37:03.134 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:03.134 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:03.134 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:37:03.235 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:37:03.256 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:03.257 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:03.257 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:37:03.340 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:37:03.341 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:37:03.341 INFO:teuthology.orchestra.run.smithi070.stderr:set 2.1 pg_upmap_items mapping to [3->0,2->1,4->6] 2024-09-16T09:37:03.358 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:37:03.365 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:37:03.366 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:37:03.401 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:03.402 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:03.402 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:37:03.405 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:03 smithi070 ceph-mon[40691]: pgmap v616: 172 pgs: 1 clean+premerge+peered, 171 active+clean; 1.6 GiB data, 9.6 GiB used, 706 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 386 op/s 2024-09-16T09:37:03.405 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:03 smithi070 ceph-mon[40691]: osdmap e342: 8 total, 8 up, 8 in 2024-09-16T09:37:03.405 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:03 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:37:03.405 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:03 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:37:03.405 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:03 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/337070291' entity='client.admin' cmd={"prefix": "osd pg-upmap-items", "pgid": "2.1", "id": [3, 0, 2, 1, 4, 6]} : dispatch 2024-09-16T09:37:03.405 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:03 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/732598204' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:37:03.405 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:03 smithi070 ceph-mon[30388]: pgmap v616: 172 pgs: 1 clean+premerge+peered, 171 active+clean; 1.6 GiB data, 9.6 GiB used, 706 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 386 op/s 2024-09-16T09:37:03.405 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:03 smithi070 ceph-mon[30388]: osdmap e342: 8 total, 8 up, 8 in 2024-09-16T09:37:03.406 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:03 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:37:03.406 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:03 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:37:03.406 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:03 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/337070291' entity='client.admin' cmd={"prefix": "osd pg-upmap-items", "pgid": "2.1", "id": [3, 0, 2, 1, 4, 6]} : dispatch 2024-09-16T09:37:03.406 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:03 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/732598204' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:37:03.406 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:37:03.406 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:37:03.437 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:37:03.438 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:37:03.470 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:37:03.470 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:37:03.503 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:37:03.504 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:37:03.505 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:37:03.538 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:03.538 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:37:03.539 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:37:03.539 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:03.540 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:37:03.574 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:37:03.574 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:37:03.640 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:37:03.660 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:37:03 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:37:03.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:03 smithi084 ceph-mon[32629]: pgmap v616: 172 pgs: 1 clean+premerge+peered, 171 active+clean; 1.6 GiB data, 9.6 GiB used, 706 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 386 op/s 2024-09-16T09:37:03.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:03 smithi084 ceph-mon[32629]: osdmap e342: 8 total, 8 up, 8 in 2024-09-16T09:37:03.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:03 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:37:03.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:03 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:37:03.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:03 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/337070291' entity='client.admin' cmd={"prefix": "osd pg-upmap-items", "pgid": "2.1", "id": [3, 0, 2, 1, 4, 6]} : dispatch 2024-09-16T09:37:03.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:03 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/732598204' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:37:03.675 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:03.676 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:03.677 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:37:03.777 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:37:03.811 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:03.812 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:03.812 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:37:03.824 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-16T09:37:03.845 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:37:03.913 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:37:03.951 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:03.951 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:03.952 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:37:04.052 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:37:04.085 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:04.086 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:04.086 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:37:04.187 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:37:04.220 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:04.221 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:04.221 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:37:04.322 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:37:04.362 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:04.363 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:04.363 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:37:04.464 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:37:04.491 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:04.492 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:04.493 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:37:04.593 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:37:04.615 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:04.616 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:04.616 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:37:04.655 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:04 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:37:04.655 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:04 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/337070291' entity='client.admin' cmd='[{"prefix": "osd pg-upmap-items", "pgid": "2.1", "id": [3, 0, 2, 1, 4, 6]}]': finished 2024-09-16T09:37:04.656 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:04 smithi070 ceph-mon[30388]: osdmap e343: 8 total, 8 up, 8 in 2024-09-16T09:37:04.656 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:04 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "18"} : dispatch 2024-09-16T09:37:04.656 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:04 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "18"} : dispatch 2024-09-16T09:37:04.656 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:04 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:37:04.656 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:04 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/337070291' entity='client.admin' cmd='[{"prefix": "osd pg-upmap-items", "pgid": "2.1", "id": [3, 0, 2, 1, 4, 6]}]': finished 2024-09-16T09:37:04.656 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:04 smithi070 ceph-mon[40691]: osdmap e343: 8 total, 8 up, 8 in 2024-09-16T09:37:04.656 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:04 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "18"} : dispatch 2024-09-16T09:37:04.656 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:04 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "18"} : dispatch 2024-09-16T09:37:04.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:04 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:37:04.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:04 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/337070291' entity='client.admin' cmd='[{"prefix": "osd pg-upmap-items", "pgid": "2.1", "id": [3, 0, 2, 1, 4, 6]}]': finished 2024-09-16T09:37:04.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:04 smithi084 ceph-mon[32629]: osdmap e343: 8 total, 8 up, 8 in 2024-09-16T09:37:04.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:04 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "18"} : dispatch 2024-09-16T09:37:04.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:04 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "18"} : dispatch 2024-09-16T09:37:04.717 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:37:04.738 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:04.738 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:04.739 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:37:04.839 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:37:04.861 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:04.861 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:04.862 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:37:04.962 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:37:04.985 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:04.986 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:04.986 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:37:05.087 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:37:05.109 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:05.110 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:05.110 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:37:05.211 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:37:05.233 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:05.234 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:05.234 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:37:05.335 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:37:05.364 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:05.364 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:05.364 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:37:05.465 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:37:05.486 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:05.487 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:05.487 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:37:05.588 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:37:05.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:05 smithi070 ceph-mon[30388]: pgmap v619: 171 pgs: 1 peering, 170 active+clean; 1.6 GiB data, 9.2 GiB used, 706 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 918 op/s; 0 B/s, 0 objects/s recovering 2024-09-16T09:37:05.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:05 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "18"}]': finished 2024-09-16T09:37:05.605 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:05 smithi070 ceph-mon[30388]: osdmap e344: 8 total, 8 up, 8 in 2024-09-16T09:37:05.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:05 smithi070 ceph-mon[30388]: osdmap e345: 8 total, 8 up, 8 in 2024-09-16T09:37:05.606 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:05 smithi070 ceph-mon[40691]: pgmap v619: 171 pgs: 1 peering, 170 active+clean; 1.6 GiB data, 9.2 GiB used, 706 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 918 op/s; 0 B/s, 0 objects/s recovering 2024-09-16T09:37:05.606 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:05 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "18"}]': finished 2024-09-16T09:37:05.606 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:05 smithi070 ceph-mon[40691]: osdmap e344: 8 total, 8 up, 8 in 2024-09-16T09:37:05.606 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:05 smithi070 ceph-mon[40691]: osdmap e345: 8 total, 8 up, 8 in 2024-09-16T09:37:05.610 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:05.610 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:05.611 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:37:05.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:05 smithi084 ceph-mon[32629]: pgmap v619: 171 pgs: 1 peering, 170 active+clean; 1.6 GiB data, 9.2 GiB used, 706 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 918 op/s; 0 B/s, 0 objects/s recovering 2024-09-16T09:37:05.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:05 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "18"}]': finished 2024-09-16T09:37:05.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:05 smithi084 ceph-mon[32629]: osdmap e344: 8 total, 8 up, 8 in 2024-09-16T09:37:05.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:05 smithi084 ceph-mon[32629]: osdmap e345: 8 total, 8 up, 8 in 2024-09-16T09:37:05.711 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:37:05.733 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:05.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:05.734 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:37:05.835 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:37:05.857 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:05.858 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:05.858 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:37:05.959 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:37:05.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:05.981 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:05.981 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:37:06.080 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T09:37:06.081 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T09:37:06.081 INFO:teuthology.orchestra.run.smithi070.stdout: "num_ops": 0 2024-09-16T09:37:06.081 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T09:37:06.082 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:37:06.105 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:06.106 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:06.106 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:37:06.207 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:37:06.230 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:06.231 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:06.231 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:37:06.332 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:37:06.354 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:06.354 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:06.355 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:37:06.455 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:37:06.479 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:06.479 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:06.479 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:37:06.580 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:37:06.602 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:06.603 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:06.603 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:37:06.704 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:37:06.725 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:06.726 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:06.726 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:37:06.827 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:37:06.848 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:06.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:06.849 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:37:06.950 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:37:06.973 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:06.976 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:06.977 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:37:07.078 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:37:07.098 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:07.099 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:07.099 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:37:07.200 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:37:07.222 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:07.223 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:07.223 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:37:07.324 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:37:07.345 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:07.346 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:07.347 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:37:07.447 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:37:07.478 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:07.478 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:07.479 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:37:07.580 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:37:07.611 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:07.611 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:07.612 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:37:07.625 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:07 smithi070 ceph-mon[30388]: pgmap v622: 171 pgs: 2 peering, 169 active+clean; 1.5 GiB data, 9.1 GiB used, 706 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.16k op/s; 0 B/s, 0 objects/s recovering 2024-09-16T09:37:07.625 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:07 smithi070 ceph-mon[30388]: osdmap e346: 8 total, 8 up, 8 in 2024-09-16T09:37:07.626 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:07 smithi070 ceph-mon[40691]: pgmap v622: 171 pgs: 2 peering, 169 active+clean; 1.5 GiB data, 9.1 GiB used, 706 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.16k op/s; 0 B/s, 0 objects/s recovering 2024-09-16T09:37:07.626 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:07 smithi070 ceph-mon[40691]: osdmap e346: 8 total, 8 up, 8 in 2024-09-16T09:37:07.637 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:37:07.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:07 smithi084 ceph-mon[32629]: pgmap v622: 171 pgs: 2 peering, 169 active+clean; 1.5 GiB data, 9.1 GiB used, 706 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.16k op/s; 0 B/s, 0 objects/s recovering 2024-09-16T09:37:07.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:07 smithi084 ceph-mon[32629]: osdmap e346: 8 total, 8 up, 8 in 2024-09-16T09:37:07.677 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.3 dump_blocked_ops 2024-09-16T09:37:07.713 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:37:07.755 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:07.756 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:07.756 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:37:07.857 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:37:07.892 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:07.893 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:07.893 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:37:07.994 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:37:08.031 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:08.031 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:08.032 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:37:08.132 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:37:08.163 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:08.165 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:08.165 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:37:08.266 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:37:08.302 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:08.303 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:08.303 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:37:08.341 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:37:08.404 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:37:08.437 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:08.438 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:08.439 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:37:08.479 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:08 smithi070 ceph-mon[30388]: osdmap e347: 8 total, 8 up, 8 in 2024-09-16T09:37:08.480 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:08 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "17"} : dispatch 2024-09-16T09:37:08.480 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:08 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "17"} : dispatch 2024-09-16T09:37:08.480 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:08 smithi070 ceph-mon[40691]: osdmap e347: 8 total, 8 up, 8 in 2024-09-16T09:37:08.480 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:08 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "17"} : dispatch 2024-09-16T09:37:08.480 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:08 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "17"} : dispatch 2024-09-16T09:37:08.540 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:37:08.562 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:08.562 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:08.563 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:37:08.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:08 smithi084 ceph-mon[32629]: osdmap e347: 8 total, 8 up, 8 in 2024-09-16T09:37:08.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:08 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "17"} : dispatch 2024-09-16T09:37:08.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:08 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "17"} : dispatch 2024-09-16T09:37:08.663 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:37:08.685 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:08.686 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:08.686 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:37:08.787 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:37:08.807 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:08.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:08.809 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:37:08.859 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T09:37:08.860 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T09:37:08.860 INFO:teuthology.orchestra.run.smithi070.stdout: "complaint_time": 30, 2024-09-16T09:37:08.860 INFO:teuthology.orchestra.run.smithi070.stdout: "num_blocked_ops": 0 2024-09-16T09:37:08.860 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T09:37:08.910 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:37:08.930 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:08.931 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:08.931 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:37:09.032 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:37:09.053 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:09.054 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:09.054 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:37:09.059 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T09:37:09.155 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:37:09.177 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:09.177 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:09.178 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:37:09.278 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:37:09.300 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:09.300 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:09.301 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:37:09.317 INFO:teuthology.orchestra.run.smithi070.stderr:nodeep-scrub is set 2024-09-16T09:37:09.401 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:37:09.423 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:09.424 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:09.424 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:37:09.453 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T09:37:09.454 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T09:37:09.454 INFO:tasks.thrashosds.thrasher:Growing pool unique_pool_2 2024-09-16T09:37:09.454 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-16T09:37:09.525 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:37:09.547 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:09.547 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:09.548 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:37:09.649 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:37:09.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:09 smithi084 ceph-mon[32629]: pgmap v625: 170 pgs: 2 peering, 168 active+clean; 1.4 GiB data, 9.0 GiB used, 706 GiB / 715 GiB avail 2024-09-16T09:37:09.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:09 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "17"}]': finished 2024-09-16T09:37:09.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:09 smithi084 ceph-mon[32629]: osdmap e348: 8 total, 8 up, 8 in 2024-09-16T09:37:09.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:09 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/61768082' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:37:09.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:09 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:37:09.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:09 smithi084 ceph-mon[32629]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:37:09.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:09 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-16T09:37:09.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:09 smithi084 ceph-mon[32629]: osdmap e349: 8 total, 8 up, 8 in 2024-09-16T09:37:09.670 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:09.670 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:09.671 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:37:09.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:09 smithi070 ceph-mon[30388]: pgmap v625: 170 pgs: 2 peering, 168 active+clean; 1.4 GiB data, 9.0 GiB used, 706 GiB / 715 GiB avail 2024-09-16T09:37:09.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:09 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "17"}]': finished 2024-09-16T09:37:09.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:09 smithi070 ceph-mon[30388]: osdmap e348: 8 total, 8 up, 8 in 2024-09-16T09:37:09.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:09 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/61768082' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:37:09.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:09 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:37:09.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:09 smithi070 ceph-mon[30388]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:37:09.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:09 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-16T09:37:09.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:09 smithi070 ceph-mon[30388]: osdmap e349: 8 total, 8 up, 8 in 2024-09-16T09:37:09.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:09 smithi070 ceph-mon[40691]: pgmap v625: 170 pgs: 2 peering, 168 active+clean; 1.4 GiB data, 9.0 GiB used, 706 GiB / 715 GiB avail 2024-09-16T09:37:09.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:09 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "17"}]': finished 2024-09-16T09:37:09.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:09 smithi070 ceph-mon[40691]: osdmap e348: 8 total, 8 up, 8 in 2024-09-16T09:37:09.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:09 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/61768082' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:37:09.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:09 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:37:09.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:09 smithi070 ceph-mon[40691]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:37:09.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:09 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-16T09:37:09.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:09 smithi070 ceph-mon[40691]: osdmap e349: 8 total, 8 up, 8 in 2024-09-16T09:37:09.773 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:37:09.794 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:09.795 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:09.795 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:37:09.826 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.3 dump_historic_ops 2024-09-16T09:37:09.896 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:37:09.918 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:09.918 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:09.919 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:37:10.020 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:37:10.042 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:10.042 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:10.043 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:37:10.143 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:37:10.165 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:10.166 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:10.166 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:37:10.267 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:37:10.270 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:37:10.288 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:10.289 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:10.289 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:37:10.302 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:37:10.306 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:37:10.390 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:37:10.413 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:10.414 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:10.414 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:37:10.515 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:37:10.539 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:10.539 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:10.539 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:37:10.640 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:37:10.662 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:10.662 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:10.663 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:37:10.764 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:37:10.785 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:10.786 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:10.786 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:37:10.887 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:37:10.908 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:10.909 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:10.910 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:37:11.010 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:37:11.032 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:11.032 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:11.033 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:37:11.119 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T09:37:11.119 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 20, 2024-09-16T09:37:11.120 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 600, 2024-09-16T09:37:11.120 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [ 2024-09-16T09:37:11.120 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.120 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:85224 9.19 9:9bf6c0c6:::benchmark_data_smithi070_137000_object85223:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e266)", 2024-09-16T09:37:11.120 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:34:30.373671+0000", 2024-09-16T09:37:11.120 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 160.742548059, 2024-09-16T09:37:11.120 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0883978459999999, 2024-09-16T09:37:11.120 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:37:11.120 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:37:11.120 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T09:37:11.120 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:37:11.120 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:37:11.120 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:37:11.121 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 85224 2024-09-16T09:37:11.121 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.121 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:37:11.121 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.121 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:37:11.121 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.373671+0000", 2024-09-16T09:37:11.121 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:37:11.121 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.121 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.121 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:37:11.121 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.373671+0000", 2024-09-16T09:37:11.121 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:37:11.121 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.122 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.122 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:37:11.122 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.373673+0000", 2024-09-16T09:37:11.122 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.4600000000000002e-06 2024-09-16T09:37:11.122 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.122 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.122 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:37:11.122 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.373702+0000", 2024-09-16T09:37:11.122 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.8934000000000001e-05 2024-09-16T09:37:11.122 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.122 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.122 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:37:11.122 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.373725+0000", 2024-09-16T09:37:11.123 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3468999999999998e-05 2024-09-16T09:37:11.123 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.123 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.123 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:37:11.123 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.373737+0000", 2024-09-16T09:37:11.123 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1252e-05 2024-09-16T09:37:11.123 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.123 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.123 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:37:11.123 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.386012+0000", 2024-09-16T09:37:11.123 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.012275569 2024-09-16T09:37:11.123 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.123 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.123 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:37:11.124 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.386022+0000", 2024-09-16T09:37:11.124 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.2609999999999999e-06 2024-09-16T09:37:11.124 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.124 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.124 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:37:11.124 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.428618+0000", 2024-09-16T09:37:11.124 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0425969100000001 2024-09-16T09:37:11.124 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.124 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.124 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for missing object", 2024-09-16T09:37:11.124 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.428759+0000", 2024-09-16T09:37:11.124 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00014022899999999999 2024-09-16T09:37:11.124 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.125 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.125 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:37:11.125 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.432901+0000", 2024-09-16T09:37:11.125 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0041425020000000002 2024-09-16T09:37:11.125 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.125 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.125 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for degraded object", 2024-09-16T09:37:11.125 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.432957+0000", 2024-09-16T09:37:11.125 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5377999999999997e-05 2024-09-16T09:37:11.125 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.125 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.125 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:37:11.125 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.462010+0000", 2024-09-16T09:37:11.125 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.029052923000000001 2024-09-16T09:37:11.126 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.126 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.126 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:37:11.126 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.462068+0000", 2024-09-16T09:37:11.126 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8959000000000003e-05 2024-09-16T09:37:11.126 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:37:11.126 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:37:11.126 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:37:11.126 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.126 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.126 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:85228 9.19 9:98aa15be:::benchmark_data_smithi070_137000_object85227:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e266)", 2024-09-16T09:37:11.126 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:34:30.376655+0000", 2024-09-16T09:37:11.126 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 160.73956330300001, 2024-09-16T09:37:11.126 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0565689490000001, 2024-09-16T09:37:11.127 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:37:11.127 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:37:11.127 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:37:11.127 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:37:11.127 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:37:11.127 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:37:11.127 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 85228 2024-09-16T09:37:11.127 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.127 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:37:11.127 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.127 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:37:11.127 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.376655+0000", 2024-09-16T09:37:11.127 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:37:11.127 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.128 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.128 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:37:11.128 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.376655+0000", 2024-09-16T09:37:11.128 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:37:11.128 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.128 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.128 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:37:11.128 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.376657+0000", 2024-09-16T09:37:11.128 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.4419999999999999e-06 2024-09-16T09:37:11.128 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.128 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.128 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:37:11.128 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.376680+0000", 2024-09-16T09:37:11.128 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.2923000000000001e-05 2024-09-16T09:37:11.128 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.129 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.129 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:37:11.129 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.376692+0000", 2024-09-16T09:37:11.129 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1861999999999999e-05 2024-09-16T09:37:11.129 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.129 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.129 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:37:11.129 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.376701+0000", 2024-09-16T09:37:11.129 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.0080000000000008e-06 2024-09-16T09:37:11.129 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.129 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.129 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:37:11.129 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.386042+0000", 2024-09-16T09:37:11.129 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0093412960000000007 2024-09-16T09:37:11.130 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.130 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.130 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:37:11.130 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.386048+0000", 2024-09-16T09:37:11.130 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.9170000000000003e-06 2024-09-16T09:37:11.130 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.130 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.130 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:37:11.130 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.429089+0000", 2024-09-16T09:37:11.130 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0430408870000001 2024-09-16T09:37:11.130 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.130 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.131 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:37:11.131 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.429187+0000", 2024-09-16T09:37:11.131 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.8227999999999995e-05 2024-09-16T09:37:11.131 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.131 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.131 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 5,6", 2024-09-16T09:37:11.131 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.429456+0000", 2024-09-16T09:37:11.131 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00026942699999999998 2024-09-16T09:37:11.131 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.131 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.131 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:37:11.131 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.433028+0000", 2024-09-16T09:37:11.131 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0035718199999999999 2024-09-16T09:37:11.131 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.131 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.132 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:11.132 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.433093+0000", 2024-09-16T09:37:11.132 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.4393000000000005e-05 2024-09-16T09:37:11.132 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.132 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.132 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:11.132 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.433126+0000", 2024-09-16T09:37:11.132 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3083000000000002e-05 2024-09-16T09:37:11.132 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.132 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.132 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:37:11.132 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.433160+0000", 2024-09-16T09:37:11.132 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4746999999999998e-05 2024-09-16T09:37:11.132 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.133 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.133 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:37:11.133 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.433224+0000", 2024-09-16T09:37:11.133 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.3916000000000003e-05 2024-09-16T09:37:11.133 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:37:11.133 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:37:11.133 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:37:11.133 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.133 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.133 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:85232 9.19 9:9ebfcfc9:::benchmark_data_smithi070_137000_object85231:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e266)", 2024-09-16T09:37:11.133 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:34:30.381300+0000", 2024-09-16T09:37:11.133 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 160.73491899199999, 2024-09-16T09:37:11.133 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0521562879999999, 2024-09-16T09:37:11.134 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:37:11.134 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:37:11.134 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:37:11.134 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:37:11.134 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:37:11.134 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:37:11.134 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 85232 2024-09-16T09:37:11.134 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.134 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:37:11.134 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.134 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:37:11.134 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.381300+0000", 2024-09-16T09:37:11.134 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:37:11.134 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.135 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.135 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:37:11.135 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.381300+0000", 2024-09-16T09:37:11.135 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:37:11.135 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.135 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.135 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:37:11.135 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.381301+0000", 2024-09-16T09:37:11.135 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6959999999999999e-06 2024-09-16T09:37:11.135 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.135 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.135 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:37:11.135 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.381325+0000", 2024-09-16T09:37:11.135 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3213999999999998e-05 2024-09-16T09:37:11.136 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.136 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.136 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:37:11.136 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.381338+0000", 2024-09-16T09:37:11.136 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3237999999999999e-05 2024-09-16T09:37:11.136 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.136 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.136 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:37:11.136 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.381346+0000", 2024-09-16T09:37:11.136 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.8390000000000007e-06 2024-09-16T09:37:11.136 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.136 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.136 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:37:11.136 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.386067+0000", 2024-09-16T09:37:11.137 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0047214400000000004 2024-09-16T09:37:11.137 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.137 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.137 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:37:11.137 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.386073+0000", 2024-09-16T09:37:11.137 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5790000000000003e-06 2024-09-16T09:37:11.137 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.137 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.137 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:37:11.137 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.430684+0000", 2024-09-16T09:37:11.137 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.044610824 2024-09-16T09:37:11.137 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.137 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.137 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:37:11.137 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.430767+0000", 2024-09-16T09:37:11.138 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.3483e-05 2024-09-16T09:37:11.138 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.138 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.138 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 5,6", 2024-09-16T09:37:11.138 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.430988+0000", 2024-09-16T09:37:11.138 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00022084900000000001 2024-09-16T09:37:11.138 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.138 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.138 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:37:11.138 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.433032+0000", 2024-09-16T09:37:11.138 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0020444159999999999 2024-09-16T09:37:11.138 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.138 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.138 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:11.139 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.433282+0000", 2024-09-16T09:37:11.139 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000250024 2024-09-16T09:37:11.139 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.139 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.139 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:11.139 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.433335+0000", 2024-09-16T09:37:11.139 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.3096999999999997e-05 2024-09-16T09:37:11.139 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.139 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.139 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:37:11.139 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.433370+0000", 2024-09-16T09:37:11.139 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4465000000000003e-05 2024-09-16T09:37:11.139 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.139 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.140 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:37:11.140 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.433456+0000", 2024-09-16T09:37:11.140 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.6124000000000004e-05 2024-09-16T09:37:11.140 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:37:11.140 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:37:11.140 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:37:11.140 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.140 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.140 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:85266 9.19 9:9983710d:::benchmark_data_smithi070_137000_object85265:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e266)", 2024-09-16T09:37:11.140 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:34:30.409167+0000", 2024-09-16T09:37:11.140 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 160.70705119300001, 2024-09-16T09:37:11.140 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.029484174, 2024-09-16T09:37:11.140 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:37:11.140 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:37:11.140 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:37:11.141 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:37:11.141 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:37:11.141 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:37:11.141 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 85266 2024-09-16T09:37:11.141 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.141 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:37:11.141 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.141 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:37:11.141 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.409167+0000", 2024-09-16T09:37:11.141 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:37:11.141 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.141 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.141 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:37:11.142 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.409167+0000", 2024-09-16T09:37:11.142 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:37:11.142 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.142 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.142 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:37:11.142 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.409169+0000", 2024-09-16T09:37:11.142 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6339999999999999e-06 2024-09-16T09:37:11.142 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.142 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.142 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:37:11.142 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.409201+0000", 2024-09-16T09:37:11.142 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2193000000000002e-05 2024-09-16T09:37:11.142 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.142 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.143 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:37:11.143 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.409218+0000", 2024-09-16T09:37:11.143 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6481e-05 2024-09-16T09:37:11.143 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.143 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.143 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:37:11.143 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.409225+0000", 2024-09-16T09:37:11.143 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6469999999999998e-06 2024-09-16T09:37:11.143 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.143 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.143 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:37:11.143 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.410186+0000", 2024-09-16T09:37:11.143 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000960673 2024-09-16T09:37:11.143 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.143 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.144 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:37:11.144 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.410198+0000", 2024-09-16T09:37:11.144 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1993e-05 2024-09-16T09:37:11.144 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.144 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.144 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:37:11.144 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.433567+0000", 2024-09-16T09:37:11.144 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.023368582 2024-09-16T09:37:11.144 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.144 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.144 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:37:11.144 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.433672+0000", 2024-09-16T09:37:11.144 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000105046 2024-09-16T09:37:11.144 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.145 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.145 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 5,6", 2024-09-16T09:37:11.145 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.433833+0000", 2024-09-16T09:37:11.145 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00016096599999999999 2024-09-16T09:37:11.145 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.145 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.145 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:37:11.145 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.438355+0000", 2024-09-16T09:37:11.145 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.004522206 2024-09-16T09:37:11.145 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.145 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.145 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:11.145 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.438417+0000", 2024-09-16T09:37:11.145 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1888000000000001e-05 2024-09-16T09:37:11.145 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.146 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.146 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:11.146 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.438515+0000", 2024-09-16T09:37:11.146 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.8084999999999993e-05 2024-09-16T09:37:11.146 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.146 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.146 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:37:11.146 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.438550+0000", 2024-09-16T09:37:11.146 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5172000000000001e-05 2024-09-16T09:37:11.146 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.146 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.146 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:37:11.148 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:37:11.148 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.438652+0000", 2024-09-16T09:37:11.149 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00010160799999999999 2024-09-16T09:37:11.149 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:37:11.149 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:37:11.149 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:37:11.149 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.149 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.149 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:211248 9.12 9:499944f5:::benchmark_data_smithi070_137000_object89537:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e327)", 2024-09-16T09:37:11.149 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:36:47.708810+0000", 2024-09-16T09:37:11.149 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 23.407408486000001, 2024-09-16T09:37:11.149 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0511326670000001, 2024-09-16T09:37:11.149 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:37:11.149 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:37:11.149 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:37:11.150 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:37:11.150 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:37:11.150 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:37:11.150 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 211248 2024-09-16T09:37:11.150 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.150 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:37:11.150 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.150 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:37:11.150 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.708810+0000", 2024-09-16T09:37:11.150 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:37:11.150 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.150 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.150 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:37:11.150 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.708810+0000", 2024-09-16T09:37:11.150 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:37:11.151 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.151 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.151 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:37:11.151 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.708812+0000", 2024-09-16T09:37:11.151 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.4780000000000001e-06 2024-09-16T09:37:11.151 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.151 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.151 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:37:11.151 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.708816+0000", 2024-09-16T09:37:11.151 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.7090000000000001e-06 2024-09-16T09:37:11.151 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.151 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.151 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:37:11.151 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.708827+0000", 2024-09-16T09:37:11.152 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0274e-05 2024-09-16T09:37:11.152 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.152 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.152 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:37:11.152 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.708833+0000", 2024-09-16T09:37:11.152 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.7660000000000001e-06 2024-09-16T09:37:11.152 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.152 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.152 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:37:11.152 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.711366+0000", 2024-09-16T09:37:11.152 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0025325529999999999 2024-09-16T09:37:11.152 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.152 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.152 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:37:11.153 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.711385+0000", 2024-09-16T09:37:11.153 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9368000000000002e-05 2024-09-16T09:37:11.153 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.153 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.153 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:37:11.153 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.748172+0000", 2024-09-16T09:37:11.153 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0367871070000001 2024-09-16T09:37:11.153 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.153 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.153 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:37:11.153 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.748254+0000", 2024-09-16T09:37:11.153 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.1648000000000002e-05 2024-09-16T09:37:11.153 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.153 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.153 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,2", 2024-09-16T09:37:11.154 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.748561+0000", 2024-09-16T09:37:11.154 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00030718299999999999 2024-09-16T09:37:11.154 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.154 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.154 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:11.154 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.755820+0000", 2024-09-16T09:37:11.154 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0072592109999999998 2024-09-16T09:37:11.154 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.154 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.155 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:37:11.155 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.759723+0000", 2024-09-16T09:37:11.155 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0039020249999999999 2024-09-16T09:37:11.155 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.155 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.155 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:11.155 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.759846+0000", 2024-09-16T09:37:11.155 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00012310000000000001 2024-09-16T09:37:11.155 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.155 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.155 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:37:11.155 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.759901+0000", 2024-09-16T09:37:11.155 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5315000000000001e-05 2024-09-16T09:37:11.155 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.156 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.156 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:37:11.156 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.759943+0000", 2024-09-16T09:37:11.156 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.193e-05 2024-09-16T09:37:11.156 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:37:11.156 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:37:11.156 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:37:11.156 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.156 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.156 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:211251 9.12 9:49db1cb8:::benchmark_data_smithi070_137000_object89540:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e327)", 2024-09-16T09:37:11.156 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:36:47.713400+0000", 2024-09-16T09:37:11.156 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 23.402818511, 2024-09-16T09:37:11.156 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0484086290000001, 2024-09-16T09:37:11.156 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:37:11.157 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:37:11.157 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:37:11.157 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:37:11.157 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:37:11.157 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:37:11.157 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 211251 2024-09-16T09:37:11.157 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.157 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:37:11.157 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.157 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:37:11.157 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.713400+0000", 2024-09-16T09:37:11.157 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:37:11.157 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.157 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.157 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:37:11.158 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.713400+0000", 2024-09-16T09:37:11.158 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:37:11.158 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.158 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.158 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:37:11.158 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.713402+0000", 2024-09-16T09:37:11.158 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7439999999999999e-06 2024-09-16T09:37:11.158 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.158 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.158 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:37:11.158 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.713407+0000", 2024-09-16T09:37:11.158 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4260000000000002e-06 2024-09-16T09:37:11.158 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.158 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.158 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:37:11.159 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.713423+0000", 2024-09-16T09:37:11.159 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5974999999999999e-05 2024-09-16T09:37:11.159 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.159 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.159 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:37:11.159 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.713431+0000", 2024-09-16T09:37:11.159 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.5920000000000003e-06 2024-09-16T09:37:11.159 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.159 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.159 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:37:11.159 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.713501+0000", 2024-09-16T09:37:11.159 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.0025999999999997e-05 2024-09-16T09:37:11.159 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.159 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.160 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:37:11.160 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.713510+0000", 2024-09-16T09:37:11.160 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.5089999999999999e-06 2024-09-16T09:37:11.160 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.160 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.160 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:37:11.160 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.755869+0000", 2024-09-16T09:37:11.160 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0423587969999999 2024-09-16T09:37:11.160 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.160 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.160 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:37:11.160 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.755973+0000", 2024-09-16T09:37:11.160 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000103688 2024-09-16T09:37:11.161 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.161 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.161 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,2", 2024-09-16T09:37:11.161 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.756051+0000", 2024-09-16T09:37:11.161 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.8528000000000004e-05 2024-09-16T09:37:11.161 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.161 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.161 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:11.161 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.759988+0000", 2024-09-16T09:37:11.161 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0039360829999999999 2024-09-16T09:37:11.161 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.161 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.161 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:11.161 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.760034+0000", 2024-09-16T09:37:11.161 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.6269e-05 2024-09-16T09:37:11.162 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.162 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.162 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:37:11.162 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.761690+0000", 2024-09-16T09:37:11.162 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0016558860000000001 2024-09-16T09:37:11.162 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.162 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.162 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:37:11.162 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.761768+0000", 2024-09-16T09:37:11.162 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.8317000000000002e-05 2024-09-16T09:37:11.162 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.162 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.162 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:37:11.162 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.761809+0000", 2024-09-16T09:37:11.163 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0788999999999999e-05 2024-09-16T09:37:11.163 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:37:11.163 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:37:11.163 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:37:11.163 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.163 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.163 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:211260 9.12 9:4be11eeb:::benchmark_data_smithi070_137000_object89549:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e327)", 2024-09-16T09:37:11.163 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:36:47.713450+0000", 2024-09-16T09:37:11.163 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 23.402768172999998, 2024-09-16T09:37:11.163 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0500658940000001, 2024-09-16T09:37:11.164 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:37:11.164 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:37:11.164 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:37:11.164 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:37:11.164 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:37:11.164 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:37:11.164 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 211260 2024-09-16T09:37:11.164 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.164 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:37:11.164 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.164 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:37:11.164 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.713450+0000", 2024-09-16T09:37:11.164 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:37:11.164 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.165 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.165 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:37:11.165 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.713450+0000", 2024-09-16T09:37:11.165 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:37:11.165 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.165 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.165 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:37:11.165 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.713451+0000", 2024-09-16T09:37:11.165 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.3000000000000001e-07 2024-09-16T09:37:11.165 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.165 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.165 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:37:11.165 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.713453+0000", 2024-09-16T09:37:11.165 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.4650000000000001e-06 2024-09-16T09:37:11.166 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.166 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.166 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:37:11.166 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.713464+0000", 2024-09-16T09:37:11.166 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0682999999999999e-05 2024-09-16T09:37:11.166 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.166 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.166 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:37:11.166 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.713469+0000", 2024-09-16T09:37:11.166 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.8749999999999999e-06 2024-09-16T09:37:11.166 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.166 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.167 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:37:11.167 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.714719+0000", 2024-09-16T09:37:11.167 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001249586 2024-09-16T09:37:11.167 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.167 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.167 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:37:11.167 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.714728+0000", 2024-09-16T09:37:11.167 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.2599999999999994e-06 2024-09-16T09:37:11.167 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.167 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.167 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:37:11.167 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.757394+0000", 2024-09-16T09:37:11.167 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0426663780000001 2024-09-16T09:37:11.167 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.168 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.168 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:37:11.168 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.757502+0000", 2024-09-16T09:37:11.168 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000107833 2024-09-16T09:37:11.168 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.168 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.168 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,2", 2024-09-16T09:37:11.168 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.757619+0000", 2024-09-16T09:37:11.168 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000117453 2024-09-16T09:37:11.168 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.168 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.168 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:37:11.168 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.761817+0000", 2024-09-16T09:37:11.168 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0041977969999999996 2024-09-16T09:37:11.168 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.169 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.169 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:11.169 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.763289+0000", 2024-09-16T09:37:11.169 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0014714890000000001 2024-09-16T09:37:11.169 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.169 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.169 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:11.169 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.763422+0000", 2024-09-16T09:37:11.169 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00013334600000000001 2024-09-16T09:37:11.169 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.169 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.169 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:37:11.169 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.763477+0000", 2024-09-16T09:37:11.170 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5334000000000002e-05 2024-09-16T09:37:11.170 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.170 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.170 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:37:11.170 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.763516+0000", 2024-09-16T09:37:11.170 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8964999999999997e-05 2024-09-16T09:37:11.170 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:37:11.170 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:37:11.170 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:37:11.170 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.170 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.170 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:211279 9.12 9:4baade64:::benchmark_data_smithi070_137000_object89568:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e327)", 2024-09-16T09:37:11.170 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:36:47.728918+0000", 2024-09-16T09:37:11.171 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 23.387300970999998, 2024-09-16T09:37:11.171 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0347446069999999, 2024-09-16T09:37:11.171 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:37:11.171 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:37:11.171 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:37:11.171 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:37:11.171 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:37:11.171 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:37:11.171 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 211279 2024-09-16T09:37:11.171 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.171 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:37:11.171 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.171 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:37:11.171 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.728918+0000", 2024-09-16T09:37:11.172 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:37:11.172 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.172 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.172 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:37:11.172 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.728918+0000", 2024-09-16T09:37:11.172 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:37:11.172 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.172 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.172 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:37:11.172 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.728919+0000", 2024-09-16T09:37:11.172 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1620000000000001e-06 2024-09-16T09:37:11.172 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.172 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.173 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:37:11.173 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.728923+0000", 2024-09-16T09:37:11.173 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0609999999999997e-06 2024-09-16T09:37:11.173 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.173 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.173 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:37:11.173 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.728936+0000", 2024-09-16T09:37:11.173 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2891000000000001e-05 2024-09-16T09:37:11.173 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.173 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.173 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:37:11.173 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.728943+0000", 2024-09-16T09:37:11.174 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.9650000000000002e-06 2024-09-16T09:37:11.174 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.174 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.174 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:37:11.174 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.729002+0000", 2024-09-16T09:37:11.174 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.9392999999999999e-05 2024-09-16T09:37:11.174 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.174 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.174 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:37:11.174 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.729022+0000", 2024-09-16T09:37:11.174 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9752999999999999e-05 2024-09-16T09:37:11.174 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.174 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.174 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:37:11.175 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.758680+0000", 2024-09-16T09:37:11.175 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.029658484 2024-09-16T09:37:11.175 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.175 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.175 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:37:11.175 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.758761+0000", 2024-09-16T09:37:11.175 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.0319000000000001e-05 2024-09-16T09:37:11.175 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.175 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.175 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,2", 2024-09-16T09:37:11.175 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.758823+0000", 2024-09-16T09:37:11.175 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.2467999999999999e-05 2024-09-16T09:37:11.175 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.176 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.176 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:37:11.176 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.763303+0000", 2024-09-16T09:37:11.176 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0044794739999999998 2024-09-16T09:37:11.176 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.176 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.176 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:11.176 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.763362+0000", 2024-09-16T09:37:11.176 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.9268999999999998e-05 2024-09-16T09:37:11.176 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.176 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.176 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:11.176 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.763574+0000", 2024-09-16T09:37:11.177 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00021250800000000001 2024-09-16T09:37:11.177 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.177 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.177 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:37:11.177 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.763623+0000", 2024-09-16T09:37:11.177 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.8494e-05 2024-09-16T09:37:11.177 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.177 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.177 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:37:11.177 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.763662+0000", 2024-09-16T09:37:11.177 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9366000000000002e-05 2024-09-16T09:37:11.177 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:37:11.177 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:37:11.178 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:37:11.178 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.178 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.178 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:211280 9.12 9:4ff46297:::benchmark_data_smithi070_137000_object89569:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e327)", 2024-09-16T09:37:11.178 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:36:47.730052+0000", 2024-09-16T09:37:11.178 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 23.386166511999999, 2024-09-16T09:37:11.178 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.03381397, 2024-09-16T09:37:11.178 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:37:11.178 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:37:11.178 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:37:11.178 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:37:11.178 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:37:11.179 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:37:11.179 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 211280 2024-09-16T09:37:11.179 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.179 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:37:11.179 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.179 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:37:11.179 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.730052+0000", 2024-09-16T09:37:11.179 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:37:11.179 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.179 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.179 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:37:11.179 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.730052+0000", 2024-09-16T09:37:11.179 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:37:11.180 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.180 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.180 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:37:11.180 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.730053+0000", 2024-09-16T09:37:11.180 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.108e-06 2024-09-16T09:37:11.180 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.180 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.180 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:37:11.180 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.730058+0000", 2024-09-16T09:37:11.180 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.4039999999999997e-06 2024-09-16T09:37:11.180 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.180 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.180 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:37:11.181 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.730109+0000", 2024-09-16T09:37:11.181 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.1700000000000003e-05 2024-09-16T09:37:11.181 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.181 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.181 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:37:11.181 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.730141+0000", 2024-09-16T09:37:11.181 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1483000000000003e-05 2024-09-16T09:37:11.181 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.181 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.181 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:37:11.181 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.730225+0000", 2024-09-16T09:37:11.181 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.3858999999999998e-05 2024-09-16T09:37:11.181 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.181 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.182 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:37:11.182 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.730233+0000", 2024-09-16T09:37:11.182 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.4030000000000007e-06 2024-09-16T09:37:11.182 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.182 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.182 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:37:11.182 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.760090+0000", 2024-09-16T09:37:11.182 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0298566229999999 2024-09-16T09:37:11.182 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.182 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.182 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:37:11.182 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.760195+0000", 2024-09-16T09:37:11.182 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000105503 2024-09-16T09:37:11.182 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.183 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.183 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,2", 2024-09-16T09:37:11.183 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.760353+0000", 2024-09-16T09:37:11.183 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00015804 2024-09-16T09:37:11.183 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.183 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.183 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:37:11.183 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.763526+0000", 2024-09-16T09:37:11.183 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0031731810000000002 2024-09-16T09:37:11.183 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.183 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.183 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:11.183 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.763723+0000", 2024-09-16T09:37:11.183 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00019647100000000001 2024-09-16T09:37:11.184 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.184 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.184 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:11.184 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.763770+0000", 2024-09-16T09:37:11.184 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.7450999999999997e-05 2024-09-16T09:37:11.184 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.184 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.184 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:37:11.184 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.763820+0000", 2024-09-16T09:37:11.184 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.9749000000000001e-05 2024-09-16T09:37:11.184 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.184 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.184 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:37:11.184 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.763866+0000", 2024-09-16T09:37:11.185 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5995000000000003e-05 2024-09-16T09:37:11.185 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:37:11.185 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:37:11.185 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:37:11.185 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.185 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.185 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:211328 9.12 9:487331f4:::benchmark_data_smithi070_137000_object89617:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e327)", 2024-09-16T09:37:11.185 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:36:47.762277+0000", 2024-09-16T09:37:11.185 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 23.353941233, 2024-09-16T09:37:11.185 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.007375701, 2024-09-16T09:37:11.185 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:37:11.185 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:37:11.185 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:37:11.185 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:37:11.186 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:37:11.186 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:37:11.186 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 211328 2024-09-16T09:37:11.186 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.186 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:37:11.186 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.186 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:37:11.186 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.762277+0000", 2024-09-16T09:37:11.186 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:37:11.186 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.186 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.186 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:37:11.186 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.762277+0000", 2024-09-16T09:37:11.186 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:37:11.187 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.187 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.187 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:37:11.187 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.762278+0000", 2024-09-16T09:37:11.187 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.9199999999999999e-07 2024-09-16T09:37:11.187 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.187 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.187 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:37:11.187 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.762283+0000", 2024-09-16T09:37:11.187 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.7190000000000001e-06 2024-09-16T09:37:11.187 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.187 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.187 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:37:11.187 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.762295+0000", 2024-09-16T09:37:11.188 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2009e-05 2024-09-16T09:37:11.188 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.188 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.188 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:37:11.188 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.762306+0000", 2024-09-16T09:37:11.188 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0730000000000001e-05 2024-09-16T09:37:11.188 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.188 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.188 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:37:11.188 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.762359+0000", 2024-09-16T09:37:11.188 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.3554999999999998e-05 2024-09-16T09:37:11.188 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.188 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.188 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:37:11.189 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.762370+0000", 2024-09-16T09:37:11.189 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0219e-05 2024-09-16T09:37:11.189 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.189 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.189 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:37:11.189 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.764182+0000", 2024-09-16T09:37:11.189 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.001812323 2024-09-16T09:37:11.189 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.189 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.189 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:37:11.189 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.764285+0000", 2024-09-16T09:37:11.189 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000102872 2024-09-16T09:37:11.189 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.189 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.190 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,2", 2024-09-16T09:37:11.190 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.764397+0000", 2024-09-16T09:37:11.190 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000111922 2024-09-16T09:37:11.190 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.190 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.190 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:11.190 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.769453+0000", 2024-09-16T09:37:11.190 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0050566739999999997 2024-09-16T09:37:11.190 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.190 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.190 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:11.190 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.769506+0000", 2024-09-16T09:37:11.190 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.2123999999999998e-05 2024-09-16T09:37:11.190 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.190 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.190 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:37:11.191 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.769526+0000", 2024-09-16T09:37:11.191 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0806e-05 2024-09-16T09:37:11.191 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.191 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.191 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:37:11.191 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.769612+0000", 2024-09-16T09:37:11.191 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.5620999999999999e-05 2024-09-16T09:37:11.191 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.191 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.191 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:37:11.191 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.769653+0000", 2024-09-16T09:37:11.191 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.1134999999999998e-05 2024-09-16T09:37:11.191 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:37:11.191 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:37:11.191 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:37:11.191 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.192 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.192 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:211329 9.12 9:4b744b27:::benchmark_data_smithi070_137000_object89618:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e327)", 2024-09-16T09:37:11.192 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:36:47.764514+0000", 2024-09-16T09:37:11.192 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 23.351704520999998, 2024-09-16T09:37:11.192 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0096066159999999, 2024-09-16T09:37:11.192 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:37:11.192 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:37:11.192 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:37:11.192 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:37:11.192 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:37:11.192 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:37:11.192 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 211329 2024-09-16T09:37:11.192 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.193 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:37:11.193 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.193 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:37:11.193 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.764514+0000", 2024-09-16T09:37:11.193 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:37:11.193 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.193 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.193 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:37:11.193 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.764514+0000", 2024-09-16T09:37:11.193 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:37:11.193 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.193 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.193 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:37:11.193 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.764516+0000", 2024-09-16T09:37:11.194 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.4100000000000001e-06 2024-09-16T09:37:11.194 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.194 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.194 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:37:11.194 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.764521+0000", 2024-09-16T09:37:11.194 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.2599999999999996e-06 2024-09-16T09:37:11.194 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.194 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.194 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:37:11.194 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.764531+0000", 2024-09-16T09:37:11.194 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0322e-05 2024-09-16T09:37:11.194 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.194 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.195 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:37:11.195 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.764538+0000", 2024-09-16T09:37:11.195 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.0600000000000002e-06 2024-09-16T09:37:11.195 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.195 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.195 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:37:11.195 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.764618+0000", 2024-09-16T09:37:11.195 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.0177000000000002e-05 2024-09-16T09:37:11.195 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.195 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.195 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:37:11.195 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.764628+0000", 2024-09-16T09:37:11.195 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.5400000000000001e-06 2024-09-16T09:37:11.196 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.196 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.196 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:37:11.196 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.767496+0000", 2024-09-16T09:37:11.196 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0028676009999999 2024-09-16T09:37:11.196 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.196 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.196 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:37:11.196 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.767619+0000", 2024-09-16T09:37:11.196 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00012315699999999999 2024-09-16T09:37:11.196 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.196 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.196 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,2", 2024-09-16T09:37:11.196 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.767750+0000", 2024-09-16T09:37:11.197 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00013136499999999999 2024-09-16T09:37:11.197 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.197 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.197 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:11.197 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.773939+0000", 2024-09-16T09:37:11.197 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0061884619999999996 2024-09-16T09:37:11.197 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.197 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.197 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:37:11.197 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.773955+0000", 2024-09-16T09:37:11.197 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6376999999999999e-05 2024-09-16T09:37:11.197 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.197 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.198 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:11.198 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.774033+0000", 2024-09-16T09:37:11.198 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.8088000000000002e-05 2024-09-16T09:37:11.198 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.198 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.198 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:37:11.198 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.774082+0000", 2024-09-16T09:37:11.198 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.9354999999999998e-05 2024-09-16T09:37:11.198 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.198 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.198 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:37:11.198 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.774121+0000", 2024-09-16T09:37:11.198 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8442000000000002e-05 2024-09-16T09:37:11.198 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:37:11.199 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:37:11.199 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:37:11.199 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.199 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.199 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:220422 9.2 9.55f30b02 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e344)", 2024-09-16T09:37:11.199 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:37:04.372312+0000", 2024-09-16T09:37:11.199 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 6.743907053, 2024-09-16T09:37:11.199 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0049641709999999, 2024-09-16T09:37:11.199 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:37:11.199 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:37:11.199 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T09:37:11.199 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:37:11.200 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:37:11.200 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:37:11.200 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 220422 2024-09-16T09:37:11.200 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.200 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:37:11.200 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.200 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:37:11.200 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:04.372312+0000", 2024-09-16T09:37:11.200 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:37:11.200 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.200 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.200 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:37:11.201 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:04.372312+0000", 2024-09-16T09:37:11.201 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:37:11.201 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.201 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.201 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:37:11.201 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:04.372313+0000", 2024-09-16T09:37:11.201 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6610000000000001e-06 2024-09-16T09:37:11.201 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.201 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.201 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:37:11.201 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:04.372318+0000", 2024-09-16T09:37:11.201 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.9830000000000004e-06 2024-09-16T09:37:11.201 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.201 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.201 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:37:11.202 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:04.372333+0000", 2024-09-16T09:37:11.202 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.4544e-05 2024-09-16T09:37:11.202 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.202 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.202 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:37:11.202 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:04.372341+0000", 2024-09-16T09:37:11.202 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.8650000000000001e-06 2024-09-16T09:37:11.202 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.202 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.202 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:37:11.202 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:04.372416+0000", 2024-09-16T09:37:11.202 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.4883999999999997e-05 2024-09-16T09:37:11.202 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.202 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.203 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:37:11.203 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:04.372426+0000", 2024-09-16T09:37:11.203 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.078e-05 2024-09-16T09:37:11.203 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.203 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.203 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:37:11.203 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:05.377221+0000", 2024-09-16T09:37:11.203 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.004794207 2024-09-16T09:37:11.203 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.203 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.203 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:37:11.203 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:05.377276+0000", 2024-09-16T09:37:11.203 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5247000000000001e-05 2024-09-16T09:37:11.204 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:37:11.204 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:37:11.204 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:37:11.204 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.204 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.204 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:220449 9.2 9:4d2f3ff5:::benchmark_data_smithi070_137000_object98738:head [delete] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e346)", 2024-09-16T09:37:11.204 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:37:06.366595+0000", 2024-09-16T09:37:11.204 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 4.7496236969999996, 2024-09-16T09:37:11.204 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.005813316, 2024-09-16T09:37:11.204 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:37:11.204 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:37:11.204 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:37:11.204 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:37:11.205 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:37:11.205 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:37:11.205 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 220449 2024-09-16T09:37:11.205 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.205 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:37:11.205 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.205 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:37:11.205 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366595+0000", 2024-09-16T09:37:11.205 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:37:11.205 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.205 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.205 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:37:11.205 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366595+0000", 2024-09-16T09:37:11.205 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:37:11.206 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.206 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.206 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:37:11.206 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366596+0000", 2024-09-16T09:37:11.206 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.054e-06 2024-09-16T09:37:11.206 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.206 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.206 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:37:11.206 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366601+0000", 2024-09-16T09:37:11.206 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.7400000000000004e-06 2024-09-16T09:37:11.206 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.206 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.207 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:37:11.207 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366660+0000", 2024-09-16T09:37:11.207 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.9509999999999998e-05 2024-09-16T09:37:11.207 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.207 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.207 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:37:11.207 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366669+0000", 2024-09-16T09:37:11.207 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.0019999999999995e-06 2024-09-16T09:37:11.207 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.207 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.207 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:37:11.207 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.460193+0000", 2024-09-16T09:37:11.207 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.093523464000000001 2024-09-16T09:37:11.207 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.208 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.208 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:37:11.208 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.460206+0000", 2024-09-16T09:37:11.208 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3113e-05 2024-09-16T09:37:11.208 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.208 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.208 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:37:11.208 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.371051+0000", 2024-09-16T09:37:11.208 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.91084521600000001 2024-09-16T09:37:11.208 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.208 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.208 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:37:11.209 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.371151+0000", 2024-09-16T09:37:11.209 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.9918000000000001e-05 2024-09-16T09:37:11.209 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.209 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.209 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,2", 2024-09-16T09:37:11.209 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.371223+0000", 2024-09-16T09:37:11.209 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.2237000000000007e-05 2024-09-16T09:37:11.209 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.209 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.209 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:11.209 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.372257+0000", 2024-09-16T09:37:11.209 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001033422 2024-09-16T09:37:11.209 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.209 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.210 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:37:11.210 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.372276+0000", 2024-09-16T09:37:11.210 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9151999999999999e-05 2024-09-16T09:37:11.210 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.210 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.210 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:11.210 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.372326+0000", 2024-09-16T09:37:11.210 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0175e-05 2024-09-16T09:37:11.210 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.210 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.210 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:37:11.210 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.372384+0000", 2024-09-16T09:37:11.210 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7680999999999999e-05 2024-09-16T09:37:11.210 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.211 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.211 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:37:11.211 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.372408+0000", 2024-09-16T09:37:11.211 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.4632000000000001e-05 2024-09-16T09:37:11.211 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:37:11.211 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:37:11.211 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:37:11.211 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.211 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.211 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:220479 9.2 9:4adb387c:::benchmark_data_smithi070_137000_object98768:head [delete] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e346)", 2024-09-16T09:37:11.211 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:37:06.366688+0000", 2024-09-16T09:37:11.211 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 4.7495308710000002, 2024-09-16T09:37:11.211 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0069119719999999, 2024-09-16T09:37:11.212 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:37:11.212 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:37:11.212 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:37:11.212 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:37:11.212 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:37:11.212 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:37:11.212 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 220479 2024-09-16T09:37:11.212 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.212 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:37:11.212 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.212 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:37:11.212 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366688+0000", 2024-09-16T09:37:11.212 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:37:11.212 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.213 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.213 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:37:11.213 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366688+0000", 2024-09-16T09:37:11.213 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:37:11.213 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.213 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.213 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:37:11.213 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366688+0000", 2024-09-16T09:37:11.213 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.6100000000000001e-07 2024-09-16T09:37:11.213 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.213 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.213 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:37:11.213 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366692+0000", 2024-09-16T09:37:11.213 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2969999999999999e-06 2024-09-16T09:37:11.213 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.214 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.214 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:37:11.214 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366699+0000", 2024-09-16T09:37:11.214 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.5299999999999999e-06 2024-09-16T09:37:11.214 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.214 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.214 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:37:11.214 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366705+0000", 2024-09-16T09:37:11.214 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4489999999999998e-06 2024-09-16T09:37:11.214 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.214 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.214 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:37:11.214 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.460555+0000", 2024-09-16T09:37:11.214 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.093850180000000005 2024-09-16T09:37:11.214 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.215 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.215 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:37:11.215 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.460562+0000", 2024-09-16T09:37:11.215 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.6819999999999997e-06 2024-09-16T09:37:11.215 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.215 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.215 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:37:11.215 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.371896+0000", 2024-09-16T09:37:11.215 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.91133457399999995 2024-09-16T09:37:11.215 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.215 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.215 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:37:11.215 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.371993+0000", 2024-09-16T09:37:11.215 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.6768999999999995e-05 2024-09-16T09:37:11.216 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.216 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.216 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,2", 2024-09-16T09:37:11.216 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.372058+0000", 2024-09-16T09:37:11.216 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.4678999999999995e-05 2024-09-16T09:37:11.216 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.216 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.216 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:37:11.216 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.373407+0000", 2024-09-16T09:37:11.216 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001349056 2024-09-16T09:37:11.216 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.216 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.216 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:11.217 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.373457+0000", 2024-09-16T09:37:11.217 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0291999999999999e-05 2024-09-16T09:37:11.217 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.217 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.217 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:11.217 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.373501+0000", 2024-09-16T09:37:11.217 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.4558999999999998e-05 2024-09-16T09:37:11.217 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.217 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.217 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:37:11.217 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.373558+0000", 2024-09-16T09:37:11.217 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.6956000000000002e-05 2024-09-16T09:37:11.217 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.218 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.218 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:37:11.218 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.373600+0000", 2024-09-16T09:37:11.218 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.1387999999999997e-05 2024-09-16T09:37:11.218 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:37:11.218 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:37:11.218 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:37:11.218 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.218 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.218 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:220549 9.2 9:4ee8c96f:::benchmark_data_smithi070_137000_object98838:head [delete] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e346)", 2024-09-16T09:37:11.218 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:37:06.366728+0000", 2024-09-16T09:37:11.218 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 4.7494903940000004, 2024-09-16T09:37:11.218 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0076284369999999, 2024-09-16T09:37:11.219 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:37:11.219 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:37:11.219 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:37:11.219 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:37:11.219 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:37:11.219 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:37:11.219 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 220549 2024-09-16T09:37:11.219 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.219 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:37:11.219 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.219 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:37:11.219 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366728+0000", 2024-09-16T09:37:11.219 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:37:11.219 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.220 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.220 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:37:11.220 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366728+0000", 2024-09-16T09:37:11.220 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:37:11.220 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.220 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.220 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:37:11.220 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366729+0000", 2024-09-16T09:37:11.220 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8899999999999999e-07 2024-09-16T09:37:11.220 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.220 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.220 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:37:11.220 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366732+0000", 2024-09-16T09:37:11.220 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.717e-06 2024-09-16T09:37:11.221 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.221 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.221 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:37:11.221 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366739+0000", 2024-09-16T09:37:11.221 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.0400000000000004e-06 2024-09-16T09:37:11.221 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.221 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.221 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:37:11.221 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366744+0000", 2024-09-16T09:37:11.221 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4700000000000001e-06 2024-09-16T09:37:11.221 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.221 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.221 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:37:11.221 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.460597+0000", 2024-09-16T09:37:11.221 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.093853013999999998 2024-09-16T09:37:11.222 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.222 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.222 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:37:11.222 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.460606+0000", 2024-09-16T09:37:11.222 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.5490000000000006e-06 2024-09-16T09:37:11.222 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.222 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.222 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:37:11.222 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.372452+0000", 2024-09-16T09:37:11.222 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.91184652300000002 2024-09-16T09:37:11.222 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.222 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.222 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:37:11.223 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.372557+0000", 2024-09-16T09:37:11.223 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000104411 2024-09-16T09:37:11.223 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.223 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.223 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,2", 2024-09-16T09:37:11.223 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.372649+0000", 2024-09-16T09:37:11.223 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.1911999999999997e-05 2024-09-16T09:37:11.223 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.223 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.223 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:11.223 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.373670+0000", 2024-09-16T09:37:11.223 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001021903 2024-09-16T09:37:11.223 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.223 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.223 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:37:11.224 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.373683+0000", 2024-09-16T09:37:11.224 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2655e-05 2024-09-16T09:37:11.224 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.224 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.224 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:11.224 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.374270+0000", 2024-09-16T09:37:11.224 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00058656799999999999 2024-09-16T09:37:11.224 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.224 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.224 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:37:11.224 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.374320+0000", 2024-09-16T09:37:11.224 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0627999999999997e-05 2024-09-16T09:37:11.224 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.224 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.225 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:37:11.225 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.374357+0000", 2024-09-16T09:37:11.225 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6458000000000002e-05 2024-09-16T09:37:11.225 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:37:11.225 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:37:11.225 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:37:11.225 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.225 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.225 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:220550 9.2 9:4bb12f00:::benchmark_data_smithi070_137000_object98839:head [delete] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e346)", 2024-09-16T09:37:11.225 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:37:06.366752+0000", 2024-09-16T09:37:11.225 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 4.7494667929999999, 2024-09-16T09:37:11.225 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.00861194, 2024-09-16T09:37:11.225 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:37:11.225 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:37:11.226 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:37:11.226 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:37:11.226 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:37:11.226 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:37:11.226 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 220550 2024-09-16T09:37:11.226 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.226 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:37:11.226 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.226 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:37:11.226 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366752+0000", 2024-09-16T09:37:11.226 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:37:11.226 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.226 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.226 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:37:11.227 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366752+0000", 2024-09-16T09:37:11.227 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:37:11.227 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.227 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.227 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:37:11.227 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366752+0000", 2024-09-16T09:37:11.227 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.9999999999999999e-07 2024-09-16T09:37:11.227 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.227 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.227 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:37:11.227 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366754+0000", 2024-09-16T09:37:11.227 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9439999999999999e-06 2024-09-16T09:37:11.227 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.227 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.227 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:37:11.228 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366760+0000", 2024-09-16T09:37:11.228 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8529999999999997e-06 2024-09-16T09:37:11.228 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.228 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.228 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:37:11.228 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366765+0000", 2024-09-16T09:37:11.228 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5399999999999997e-06 2024-09-16T09:37:11.228 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.228 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.228 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:37:11.228 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.460637+0000", 2024-09-16T09:37:11.228 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.093872609999999995 2024-09-16T09:37:11.228 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.229 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.229 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:37:11.229 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.460643+0000", 2024-09-16T09:37:11.229 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5160000000000002e-06 2024-09-16T09:37:11.229 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.229 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.229 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:37:11.229 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.372992+0000", 2024-09-16T09:37:11.229 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.91234923700000004 2024-09-16T09:37:11.229 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.229 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.229 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:37:11.229 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.373087+0000", 2024-09-16T09:37:11.229 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.4653000000000003e-05 2024-09-16T09:37:11.229 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.230 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.230 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,2", 2024-09-16T09:37:11.230 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.373154+0000", 2024-09-16T09:37:11.230 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.7946000000000003e-05 2024-09-16T09:37:11.230 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.230 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.230 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:11.230 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.374404+0000", 2024-09-16T09:37:11.230 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0012499570000000001 2024-09-16T09:37:11.230 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.230 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.230 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:11.230 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.374439+0000", 2024-09-16T09:37:11.230 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4578e-05 2024-09-16T09:37:11.231 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.231 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.231 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:37:11.231 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.375280+0000", 2024-09-16T09:37:11.231 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00084091799999999996 2024-09-16T09:37:11.231 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.231 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.231 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:37:11.231 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.375329+0000", 2024-09-16T09:37:11.231 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.9123999999999999e-05 2024-09-16T09:37:11.231 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.231 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.231 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:37:11.231 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.375364+0000", 2024-09-16T09:37:11.231 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4764000000000001e-05 2024-09-16T09:37:11.232 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:37:11.232 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:37:11.232 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:37:11.232 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.232 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.233 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:220634 9.2 9:497a55c3:::benchmark_data_smithi070_137000_object98923:head [delete] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e346)", 2024-09-16T09:37:11.233 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:37:06.366770+0000", 2024-09-16T09:37:11.233 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 4.7494482659999999, 2024-09-16T09:37:11.233 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.010067807, 2024-09-16T09:37:11.234 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:37:11.234 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:37:11.234 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:37:11.234 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:37:11.234 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:37:11.234 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:37:11.234 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 220634 2024-09-16T09:37:11.234 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.234 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:37:11.234 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.234 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:37:11.234 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366770+0000", 2024-09-16T09:37:11.234 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:37:11.234 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.235 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.235 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:37:11.235 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366770+0000", 2024-09-16T09:37:11.235 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:37:11.235 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.235 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.235 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:37:11.235 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366771+0000", 2024-09-16T09:37:11.235 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.8099999999999999e-07 2024-09-16T09:37:11.235 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.235 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.235 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:37:11.235 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366777+0000", 2024-09-16T09:37:11.236 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.6120000000000004e-06 2024-09-16T09:37:11.236 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.236 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.236 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:37:11.236 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366783+0000", 2024-09-16T09:37:11.236 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8089999999999998e-06 2024-09-16T09:37:11.236 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.236 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.236 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:37:11.236 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366788+0000", 2024-09-16T09:37:11.236 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5210000000000004e-06 2024-09-16T09:37:11.236 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.236 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.237 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:37:11.237 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.460677+0000", 2024-09-16T09:37:11.237 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.093889792999999999 2024-09-16T09:37:11.237 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.237 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.237 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:37:11.237 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.460684+0000", 2024-09-16T09:37:11.237 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.8959999999999997e-06 2024-09-16T09:37:11.237 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.237 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.237 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:37:11.237 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.373725+0000", 2024-09-16T09:37:11.237 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.91304026400000005 2024-09-16T09:37:11.237 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.238 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.238 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:37:11.238 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.373825+0000", 2024-09-16T09:37:11.238 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000100238 2024-09-16T09:37:11.238 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.238 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.238 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,2", 2024-09-16T09:37:11.238 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.373937+0000", 2024-09-16T09:37:11.238 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000112669 2024-09-16T09:37:11.238 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.238 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.238 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:37:11.238 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.375849+0000", 2024-09-16T09:37:11.238 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001911966 2024-09-16T09:37:11.239 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.239 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.239 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:11.239 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.375893+0000", 2024-09-16T09:37:11.239 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.3689000000000001e-05 2024-09-16T09:37:11.239 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.239 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.239 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:11.239 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.376754+0000", 2024-09-16T09:37:11.239 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00086106599999999996 2024-09-16T09:37:11.239 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.239 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.239 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:37:11.240 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.376800+0000", 2024-09-16T09:37:11.240 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5964999999999998e-05 2024-09-16T09:37:11.240 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.240 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.240 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:37:11.240 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.376838+0000", 2024-09-16T09:37:11.240 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8037999999999997e-05 2024-09-16T09:37:11.240 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:37:11.240 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:37:11.240 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:37:11.240 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.240 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.240 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:220635 9.2 9:48fa68c0:::benchmark_data_smithi070_137000_object98924:head [delete] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e346)", 2024-09-16T09:37:11.240 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:37:06.366794+0000", 2024-09-16T09:37:11.241 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 4.7494249640000001, 2024-09-16T09:37:11.241 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0102374599999999, 2024-09-16T09:37:11.241 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:37:11.241 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:37:11.241 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:37:11.241 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:37:11.241 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:37:11.241 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:37:11.241 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 220635 2024-09-16T09:37:11.241 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.241 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:37:11.241 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.241 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:37:11.241 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366794+0000", 2024-09-16T09:37:11.242 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:37:11.242 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.242 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.242 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:37:11.242 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366794+0000", 2024-09-16T09:37:11.242 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:37:11.242 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.242 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.242 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:37:11.242 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366794+0000", 2024-09-16T09:37:11.242 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1699999999999999e-07 2024-09-16T09:37:11.242 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.242 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.242 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:37:11.243 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366796+0000", 2024-09-16T09:37:11.243 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.792e-06 2024-09-16T09:37:11.243 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.243 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.243 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:37:11.243 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366801+0000", 2024-09-16T09:37:11.243 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5999999999999997e-06 2024-09-16T09:37:11.243 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.243 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.243 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:37:11.243 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366806+0000", 2024-09-16T09:37:11.243 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.7469999999999996e-06 2024-09-16T09:37:11.243 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.243 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.244 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:37:11.244 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.460714+0000", 2024-09-16T09:37:11.244 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.093907861999999995 2024-09-16T09:37:11.244 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.244 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.244 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:37:11.244 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.460719+0000", 2024-09-16T09:37:11.244 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4070000000000001e-06 2024-09-16T09:37:11.244 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.244 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.244 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:37:11.244 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.374493+0000", 2024-09-16T09:37:11.244 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.91377397100000002 2024-09-16T09:37:11.244 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.244 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.244 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:37:11.245 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.374586+0000", 2024-09-16T09:37:11.245 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.2905000000000006e-05 2024-09-16T09:37:11.245 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.245 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.245 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,2", 2024-09-16T09:37:11.245 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.374680+0000", 2024-09-16T09:37:11.245 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.3769000000000003e-05 2024-09-16T09:37:11.245 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.245 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.245 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:37:11.245 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.375855+0000", 2024-09-16T09:37:11.245 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001175073 2024-09-16T09:37:11.245 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.245 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.245 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:11.246 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.376878+0000", 2024-09-16T09:37:11.246 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0010223820000000001 2024-09-16T09:37:11.246 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.246 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.246 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:11.246 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.376950+0000", 2024-09-16T09:37:11.246 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.1966999999999997e-05 2024-09-16T09:37:11.246 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.246 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.246 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:37:11.246 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.376997+0000", 2024-09-16T09:37:11.246 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.7987e-05 2024-09-16T09:37:11.246 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.246 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.246 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:37:11.247 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.377031+0000", 2024-09-16T09:37:11.247 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3680999999999998e-05 2024-09-16T09:37:11.247 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:37:11.247 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:37:11.247 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:37:11.247 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.247 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.247 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:220638 9.2 9:4df3d7d7:::benchmark_data_smithi070_137000_object98927:head [delete] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e346)", 2024-09-16T09:37:11.247 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:37:06.366812+0000", 2024-09-16T09:37:11.247 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 4.7494068540000001, 2024-09-16T09:37:11.247 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0103380449999999, 2024-09-16T09:37:11.247 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:37:11.247 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:37:11.248 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:37:11.248 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:37:11.248 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:37:11.248 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:37:11.248 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 220638 2024-09-16T09:37:11.248 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.248 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:37:11.248 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.248 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:37:11.248 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366812+0000", 2024-09-16T09:37:11.248 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:37:11.248 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.248 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.249 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:37:11.249 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366812+0000", 2024-09-16T09:37:11.249 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:37:11.249 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.249 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.249 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:37:11.249 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366812+0000", 2024-09-16T09:37:11.249 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.58e-07 2024-09-16T09:37:11.249 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.249 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.249 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:37:11.249 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366814+0000", 2024-09-16T09:37:11.249 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6670000000000001e-06 2024-09-16T09:37:11.250 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.250 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.250 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:37:11.250 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366820+0000", 2024-09-16T09:37:11.250 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.6799999999999998e-06 2024-09-16T09:37:11.250 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.250 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.250 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:37:11.250 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366824+0000", 2024-09-16T09:37:11.250 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.7400000000000004e-06 2024-09-16T09:37:11.250 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.250 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.250 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:37:11.251 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.460752+0000", 2024-09-16T09:37:11.251 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.093927963000000003 2024-09-16T09:37:11.251 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.251 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.251 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:37:11.251 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.460759+0000", 2024-09-16T09:37:11.251 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.7989999999999996e-06 2024-09-16T09:37:11.251 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.251 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.251 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:37:11.251 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.374896+0000", 2024-09-16T09:37:11.251 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.91413692400000002 2024-09-16T09:37:11.251 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.251 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.252 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:37:11.252 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.374996+0000", 2024-09-16T09:37:11.252 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.9656999999999998e-05 2024-09-16T09:37:11.252 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.252 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.252 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,2", 2024-09-16T09:37:11.252 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.375097+0000", 2024-09-16T09:37:11.252 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000101109 2024-09-16T09:37:11.252 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.252 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.252 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:37:11.252 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.376713+0000", 2024-09-16T09:37:11.252 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0016159530000000001 2024-09-16T09:37:11.253 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.253 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.253 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:11.253 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.376914+0000", 2024-09-16T09:37:11.253 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00020110700000000001 2024-09-16T09:37:11.253 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.253 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.253 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:11.253 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.377068+0000", 2024-09-16T09:37:11.253 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00015383500000000001 2024-09-16T09:37:11.253 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.253 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.253 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:37:11.254 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.377115+0000", 2024-09-16T09:37:11.254 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.7604e-05 2024-09-16T09:37:11.254 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.254 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.254 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:37:11.254 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.377150+0000", 2024-09-16T09:37:11.254 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4649e-05 2024-09-16T09:37:11.254 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:37:11.254 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:37:11.254 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:37:11.254 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.254 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.254 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:220657 9.2 9:4cd451a8:::benchmark_data_smithi070_137000_object98946:head [delete] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e346)", 2024-09-16T09:37:11.254 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:37:06.366830+0000", 2024-09-16T09:37:11.254 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 4.7493887140000002, 2024-09-16T09:37:11.255 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.010672051, 2024-09-16T09:37:11.255 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:37:11.255 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:37:11.255 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:37:11.255 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:37:11.255 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:37:11.255 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:37:11.255 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 220657 2024-09-16T09:37:11.255 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.255 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:37:11.255 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.255 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:37:11.255 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366830+0000", 2024-09-16T09:37:11.255 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:37:11.256 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.256 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.256 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:37:11.256 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366830+0000", 2024-09-16T09:37:11.256 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:37:11.256 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.256 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.256 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:37:11.256 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366830+0000", 2024-09-16T09:37:11.256 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.65e-07 2024-09-16T09:37:11.256 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.256 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.256 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:37:11.257 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366832+0000", 2024-09-16T09:37:11.257 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6479999999999999e-06 2024-09-16T09:37:11.257 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.257 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.257 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:37:11.257 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366838+0000", 2024-09-16T09:37:11.257 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.6640000000000003e-06 2024-09-16T09:37:11.257 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.257 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.257 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:37:11.257 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366847+0000", 2024-09-16T09:37:11.257 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.8670000000000006e-06 2024-09-16T09:37:11.257 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.257 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.257 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:37:11.258 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.460788+0000", 2024-09-16T09:37:11.258 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.093940896999999995 2024-09-16T09:37:11.258 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.258 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.258 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:37:11.258 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.460795+0000", 2024-09-16T09:37:11.258 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.5699999999999998e-06 2024-09-16T09:37:11.258 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.258 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.258 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:37:11.258 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.375406+0000", 2024-09-16T09:37:11.258 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.91461130199999996 2024-09-16T09:37:11.258 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.258 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.259 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:37:11.259 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.375587+0000", 2024-09-16T09:37:11.259 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00018084700000000001 2024-09-16T09:37:11.259 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.259 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.259 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,2", 2024-09-16T09:37:11.259 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.375684+0000", 2024-09-16T09:37:11.259 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.7095999999999993e-05 2024-09-16T09:37:11.259 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.259 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.259 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:37:11.259 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.376717+0000", 2024-09-16T09:37:11.259 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0010328729999999999 2024-09-16T09:37:11.259 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.259 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.260 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:11.260 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.377357+0000", 2024-09-16T09:37:11.260 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00063957 2024-09-16T09:37:11.260 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.260 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.260 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:11.260 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.377417+0000", 2024-09-16T09:37:11.260 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.0062999999999997e-05 2024-09-16T09:37:11.260 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.260 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.260 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:37:11.260 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.377466+0000", 2024-09-16T09:37:11.260 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.9407999999999999e-05 2024-09-16T09:37:11.260 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:11.260 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:11.261 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:37:11.261 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.377502+0000", 2024-09-16T09:37:11.261 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5981e-05 2024-09-16T09:37:11.261 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:37:11.261 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:37:11.261 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:37:11.261 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:37:11.261 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:37:11.261 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T09:37:11.262 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:11.262 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:11.262 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:37:11.363 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:37:11.384 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:11.385 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:11.385 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:37:11.393 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:11 smithi070 ceph-mon[30388]: pgmap v628: 170 pgs: 170 active+clean; 1.3 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 851 op/s; 0 B/s, 0 objects/s recovering 2024-09-16T09:37:11.393 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:11 smithi070 ceph-mon[30388]: osdmap e350: 8 total, 8 up, 8 in 2024-09-16T09:37:11.393 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:11 smithi070 ceph-mon[40691]: pgmap v628: 170 pgs: 170 active+clean; 1.3 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 851 op/s; 0 B/s, 0 objects/s recovering 2024-09-16T09:37:11.393 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:11 smithi070 ceph-mon[40691]: osdmap e350: 8 total, 8 up, 8 in 2024-09-16T09:37:11.486 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:37:11.520 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:11.521 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:11.521 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:37:11.605 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:37:11.609 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2024-09-16T09:37:11.623 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:37:11.655 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:11.655 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:11.656 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:37:11.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:11 smithi084 ceph-mon[32629]: pgmap v628: 170 pgs: 170 active+clean; 1.3 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 851 op/s; 0 B/s, 0 objects/s recovering 2024-09-16T09:37:11.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:11 smithi084 ceph-mon[32629]: osdmap e350: 8 total, 8 up, 8 in 2024-09-16T09:37:11.757 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:37:11.788 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:11.788 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:11.789 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:37:11.890 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:37:11.925 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:11.926 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:11.926 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:37:12.027 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:37:12.063 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:12.064 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:12.064 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:37:12.165 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:37:12.170 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:37:11 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:37:11] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:37:12.200 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:12.201 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:12.201 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:37:12.234 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd unset noscrub 2024-09-16T09:37:12.302 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:37:12.339 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:12.340 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:12.340 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:37:12.441 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:37:12.474 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:12.474 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:12.474 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:37:12.575 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:37:12.599 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:12.600 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:12.600 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:37:12.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:12 smithi084 ceph-mon[32629]: osdmap e351: 8 total, 8 up, 8 in 2024-09-16T09:37:12.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:12 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "16"} : dispatch 2024-09-16T09:37:12.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:12 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "16"} : dispatch 2024-09-16T09:37:12.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:12 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:37:12.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:12 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:37:12.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:12 smithi070 ceph-mon[30388]: osdmap e351: 8 total, 8 up, 8 in 2024-09-16T09:37:12.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:12 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "16"} : dispatch 2024-09-16T09:37:12.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:12 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "16"} : dispatch 2024-09-16T09:37:12.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:12 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:37:12.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:12 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:37:12.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:12 smithi070 ceph-mon[40691]: osdmap e351: 8 total, 8 up, 8 in 2024-09-16T09:37:12.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:12 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "16"} : dispatch 2024-09-16T09:37:12.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:12 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "16"} : dispatch 2024-09-16T09:37:12.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:12 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:37:12.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:12 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:37:12.701 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:37:12.722 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:12.723 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:12.723 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:37:12.824 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:37:12.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:12.846 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:12.846 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:37:12.947 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:37:12.969 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:12.969 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:12.970 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:37:13.013 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.4 dump_ops_in_flight 2024-09-16T09:37:13.071 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:37:13.092 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:13.093 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:13.094 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:37:13.194 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:37:13.216 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:13.217 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:13.217 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:37:13.265 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:37:13.319 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:37:13.341 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:13.341 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:13.342 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:37:13.386 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:37:13 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:37:13.443 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:37:13.463 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:13.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:13.464 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:37:13.565 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:37:13.587 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:13.588 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:13.588 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:37:13.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:13 smithi070 ceph-mon[30388]: from='client.15840 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:37:13.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:13 smithi070 ceph-mon[30388]: pgmap v631: 169 pgs: 169 active+clean; 1.3 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 1.2 KiB/s rd, 7.6 MiB/s wr, 1.11k op/s; 55 MiB/s, 968 objects/s recovering 2024-09-16T09:37:13.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:13 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "16"}]': finished 2024-09-16T09:37:13.606 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:13 smithi070 ceph-mon[30388]: osdmap e352: 8 total, 8 up, 8 in 2024-09-16T09:37:13.606 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:13 smithi070 ceph-mon[40691]: from='client.15840 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:37:13.606 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:13 smithi070 ceph-mon[40691]: pgmap v631: 169 pgs: 169 active+clean; 1.3 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 1.2 KiB/s rd, 7.6 MiB/s wr, 1.11k op/s; 55 MiB/s, 968 objects/s recovering 2024-09-16T09:37:13.606 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:13 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "16"}]': finished 2024-09-16T09:37:13.607 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:13 smithi070 ceph-mon[40691]: osdmap e352: 8 total, 8 up, 8 in 2024-09-16T09:37:13.633 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":630,"stamp":"2024-09-16T09:37:11.385458+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1344018995,"num_objects":20699,"num_object_clones":0,"num_object_copies":62097,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20699,"num_whiteouts":0,"num_read":1862,"num_read_kb":1614,"num_write":149336,"num_write_kb":3816942,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23214,"num_bytes_recovered":1478562708,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":50192,"ondisk_log_size":50192,"up":507,"acting":507,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":523,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":8960880,"kb_used_data":5018032,"kb_used_omap":12,"kb_used_meta":3942195,"kb_avail":741000336,"statfs":{"total":767960285184,"available":758784344064,"internally_reserved":0,"allocated":5138464768,"data_stored":5134876121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":4036808242},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,6,1,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-215613440,"num_objects":-3290,"num_object_clones":0,"num_object_copies":-9870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-3290,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":4778,"num_write_kb":31232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3874,"num_bytes_recovered":229834752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.019828"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":627,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-16T09:37:08.377239+0000","last_change":"2024-09-16T09:32:39.384174+0000","last_active":"2024-09-16T09:37:08.377239+0000","last_peered":"2024-09-16T09:37:08.377239+0000","last_clean":"2024-09-16T09:37:08.377239+0000","last_became_active":"2024-09-16T09:25:01.709108+0000","last_became_peered":"2024-09-16T09:25:01.709108+0000","last_unstale":"2024-09-16T09:37:08.377239+0000","last_undegraded":"2024-09-16T09:37:08.377239+0000","last_fullsized":"2024-09-16T09:37:08.377239+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:39.384104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:32:39.384104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"285'15","reported_seq":638,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.375029+0000","last_change":"2024-09-16T09:32:38.074301+0000","last_active":"2024-09-16T09:37:07.375029+0000","last_peered":"2024-09-16T09:37:07.375029+0000","last_clean":"2024-09-16T09:37:07.375029+0000","last_became_active":"2024-09-16T09:25:03.714635+0000","last_became_peered":"2024-09-16T09:25:03.714635+0000","last_unstale":"2024-09-16T09:37:07.375029+0000","last_undegraded":"2024-09-16T09:37:07.375029+0000","last_fullsized":"2024-09-16T09:37:07.375029+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:32:38.074231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:32:38.074231+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:06.474635+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":248,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.366213+0000","last_change":"2024-09-16T09:36:53.098680+0000","last_active":"2024-09-16T09:37:07.366213+0000","last_peered":"2024-09-16T09:37:07.366213+0000","last_clean":"2024-09-16T09:37:07.366213+0000","last_became_active":"2024-09-16T09:33:30.242238+0000","last_became_peered":"2024-09-16T09:33:30.242238+0000","last_unstale":"2024-09-16T09:37:07.366213+0000","last_undegraded":"2024-09-16T09:37:07.366213+0000","last_fullsized":"2024-09-16T09:37:07.366213+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:53.098635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:36:53.098635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:03.340158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":596,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.375115+0000","last_change":"2024-09-16T09:35:11.876996+0000","last_active":"2024-09-16T09:37:07.375115+0000","last_peered":"2024-09-16T09:37:07.375115+0000","last_clean":"2024-09-16T09:37:07.375115+0000","last_became_active":"2024-09-16T09:25:07.715442+0000","last_became_peered":"2024-09-16T09:25:07.715442+0000","last_unstale":"2024-09-16T09:37:07.375115+0000","last_undegraded":"2024-09-16T09:37:07.375115+0000","last_fullsized":"2024-09-16T09:37:07.375115+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:35:11.876955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:35:11.876955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":342,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.368518+0000","last_change":"2024-09-16T09:34:33.452784+0000","last_active":"2024-09-16T09:37:07.368518+0000","last_peered":"2024-09-16T09:37:07.368518+0000","last_clean":"2024-09-16T09:37:07.368518+0000","last_became_active":"2024-09-16T09:31:09.269342+0000","last_became_peered":"2024-09-16T09:31:09.269342+0000","last_unstale":"2024-09-16T09:37:07.368518+0000","last_undegraded":"2024-09-16T09:37:07.368518+0000","last_fullsized":"2024-09-16T09:37:07.368518+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:33.452706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:34:33.452706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":483,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.372261+0000","last_change":"2024-09-16T09:33:08.816231+0000","last_active":"2024-09-16T09:37:07.372261+0000","last_peered":"2024-09-16T09:37:07.372261+0000","last_clean":"2024-09-16T09:37:07.372261+0000","last_became_active":"2024-09-16T09:31:09.275136+0000","last_became_peered":"2024-09-16T09:31:09.275136+0000","last_unstale":"2024-09-16T09:37:07.372261+0000","last_undegraded":"2024-09-16T09:37:07.372261+0000","last_fullsized":"2024-09-16T09:37:07.372261+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:08.816174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:08.816174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:04.941417+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.19","version":"285'27","reported_seq":317,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-16T09:37:08.375962+0000","last_change":"2024-09-16T09:33:21.337711+0000","last_active":"2024-09-16T09:37:08.375962+0000","last_peered":"2024-09-16T09:37:08.375962+0000","last_clean":"2024-09-16T09:37:08.375962+0000","last_became_active":"2024-09-16T09:33:21.337284+0000","last_became_peered":"2024-09-16T09:33:21.337284+0000","last_unstale":"2024-09-16T09:37:08.375962+0000","last_undegraded":"2024-09-16T09:37:08.375962+0000","last_fullsized":"2024-09-16T09:37:08.375962+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-16T09:32:45.194581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:32:45.194581+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":616,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.374859+0000","last_change":"2024-09-16T09:32:44.114475+0000","last_active":"2024-09-16T09:37:07.374859+0000","last_peered":"2024-09-16T09:37:07.374859+0000","last_clean":"2024-09-16T09:37:07.374859+0000","last_became_active":"2024-09-16T09:25:05.717236+0000","last_became_peered":"2024-09-16T09:25:05.717236+0000","last_unstale":"2024-09-16T09:37:07.374859+0000","last_undegraded":"2024-09-16T09:37:07.374859+0000","last_fullsized":"2024-09-16T09:37:07.374859+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:44.114407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:32:44.114407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:02.562703+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":251,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-16T09:37:08.372415+0000","last_change":"2024-09-16T09:33:36.655594+0000","last_active":"2024-09-16T09:37:08.372415+0000","last_peered":"2024-09-16T09:37:08.372415+0000","last_clean":"2024-09-16T09:37:08.372415+0000","last_became_active":"2024-09-16T09:33:30.247733+0000","last_became_peered":"2024-09-16T09:33:30.247733+0000","last_unstale":"2024-09-16T09:37:08.372415+0000","last_undegraded":"2024-09-16T09:37:08.372415+0000","last_fullsized":"2024-09-16T09:37:08.372415+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:36.655236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:36.655236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1a","version":"285'15","reported_seq":637,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.375310+0000","last_change":"2024-09-16T09:32:48.185484+0000","last_active":"2024-09-16T09:37:07.375310+0000","last_peered":"2024-09-16T09:37:07.375310+0000","last_clean":"2024-09-16T09:37:07.375310+0000","last_became_active":"2024-09-16T09:25:03.709147+0000","last_became_peered":"2024-09-16T09:25:03.709147+0000","last_unstale":"2024-09-16T09:37:07.375310+0000","last_undegraded":"2024-09-16T09:37:07.375310+0000","last_fullsized":"2024-09-16T09:37:07.375310+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:32:48.185380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:32:48.185380+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":607,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.366530+0000","last_change":"2024-09-16T09:36:51.134364+0000","last_active":"2024-09-16T09:37:07.366530+0000","last_peered":"2024-09-16T09:37:07.366530+0000","last_clean":"2024-09-16T09:37:07.366530+0000","last_became_active":"2024-09-16T09:25:05.718134+0000","last_became_peered":"2024-09-16T09:25:05.718134+0000","last_unstale":"2024-09-16T09:37:07.366530+0000","last_undegraded":"2024-09-16T09:37:07.366530+0000","last_fullsized":"2024-09-16T09:37:07.366530+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:51.134314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:36:51.134314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:37:54.019613+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"66'1","reported_seq":615,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-16T09:37:08.373664+0000","last_change":"2024-09-16T09:33:38.705001+0000","last_active":"2024-09-16T09:37:08.373664+0000","last_peered":"2024-09-16T09:37:08.373664+0000","last_clean":"2024-09-16T09:37:08.373664+0000","last_became_active":"2024-09-16T09:25:07.716558+0000","last_became_peered":"2024-09-16T09:25:07.716558+0000","last_unstale":"2024-09-16T09:37:08.373664+0000","last_undegraded":"2024-09-16T09:37:08.373664+0000","last_fullsized":"2024-09-16T09:37:08.373664+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-16T09:33:38.704868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:33:38.704868+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":607,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.366495+0000","last_change":"2024-09-16T09:33:19.326765+0000","last_active":"2024-09-16T09:37:07.366495+0000","last_peered":"2024-09-16T09:37:07.366495+0000","last_clean":"2024-09-16T09:37:07.366495+0000","last_became_active":"2024-09-16T09:25:01.696939+0000","last_became_peered":"2024-09-16T09:25:01.696939+0000","last_unstale":"2024-09-16T09:37:07.366495+0000","last_undegraded":"2024-09-16T09:37:07.366495+0000","last_fullsized":"2024-09-16T09:37:07.366495+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:19.326695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:19.326695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"285'7","reported_seq":625,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.375117+0000","last_change":"2024-09-16T09:35:39.975915+0000","last_active":"2024-09-16T09:37:07.375117+0000","last_peered":"2024-09-16T09:37:07.375117+0000","last_clean":"2024-09-16T09:37:07.375117+0000","last_became_active":"2024-09-16T09:25:03.716136+0000","last_became_peered":"2024-09-16T09:25:03.716136+0000","last_unstale":"2024-09-16T09:37:07.375117+0000","last_undegraded":"2024-09-16T09:37:07.375117+0000","last_fullsized":"2024-09-16T09:37:07.375117+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'7","last_scrub_stamp":"2024-09-16T09:35:39.975840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:35:39.975840+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":10,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":599,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.377286+0000","last_change":"2024-09-16T09:28:52.746046+0000","last_active":"2024-09-16T09:37:07.377286+0000","last_peered":"2024-09-16T09:37:07.377286+0000","last_clean":"2024-09-16T09:37:07.377286+0000","last_became_active":"2024-09-16T09:25:05.718281+0000","last_became_peered":"2024-09-16T09:25:05.718281+0000","last_unstale":"2024-09-16T09:37:07.377286+0000","last_undegraded":"2024-09-16T09:37:07.377286+0000","last_fullsized":"2024-09-16T09:37:07.377286+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:52.745999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:28:52.745999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:00.486115+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":625,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.366508+0000","last_change":"2024-09-16T09:33:46.366042+0000","last_active":"2024-09-16T09:37:07.366508+0000","last_peered":"2024-09-16T09:37:07.366508+0000","last_clean":"2024-09-16T09:37:07.366508+0000","last_became_active":"2024-09-16T09:31:09.275656+0000","last_became_peered":"2024-09-16T09:31:09.275656+0000","last_unstale":"2024-09-16T09:37:07.366508+0000","last_undegraded":"2024-09-16T09:37:07.366508+0000","last_fullsized":"2024-09-16T09:37:07.366508+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:46.365972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:33:46.365972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":73,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":613,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.375853+0000","last_change":"2024-09-16T09:33:48.322010+0000","last_active":"2024-09-16T09:37:07.375853+0000","last_peered":"2024-09-16T09:37:07.375853+0000","last_clean":"2024-09-16T09:37:07.375853+0000","last_became_active":"2024-09-16T09:25:07.720793+0000","last_became_peered":"2024-09-16T09:25:07.720793+0000","last_unstale":"2024-09-16T09:37:07.375853+0000","last_undegraded":"2024-09-16T09:37:07.375853+0000","last_fullsized":"2024-09-16T09:37:07.375853+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:48.321895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:33:48.321895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:37:26.677312+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":364,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.372160+0000","last_change":"2024-09-16T09:34:33.866299+0000","last_active":"2024-09-16T09:37:07.372160+0000","last_peered":"2024-09-16T09:37:07.372160+0000","last_clean":"2024-09-16T09:37:07.372160+0000","last_became_active":"2024-09-16T09:30:39.201824+0000","last_became_peered":"2024-09-16T09:30:39.201824+0000","last_unstale":"2024-09-16T09:37:07.372160+0000","last_undegraded":"2024-09-16T09:37:07.372160+0000","last_fullsized":"2024-09-16T09:37:07.372160+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:33.866142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:34:33.866142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"285'27","reported_seq":712,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.367273+0000","last_change":"2024-09-16T09:36:54.484021+0000","last_active":"2024-09-16T09:37:07.367273+0000","last_peered":"2024-09-16T09:37:07.367273+0000","last_clean":"2024-09-16T09:37:07.367273+0000","last_became_active":"2024-09-16T09:33:30.249798+0000","last_became_peered":"2024-09-16T09:33:30.249798+0000","last_unstale":"2024-09-16T09:37:07.367273+0000","last_undegraded":"2024-09-16T09:37:07.367273+0000","last_fullsized":"2024-09-16T09:37:07.367273+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'27","last_scrub_stamp":"2024-09-16T09:36:54.483966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:36:54.483966+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:05.562379+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":248,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.375843+0000","last_change":"2024-09-16T09:33:43.326933+0000","last_active":"2024-09-16T09:37:07.375843+0000","last_peered":"2024-09-16T09:37:07.375843+0000","last_clean":"2024-09-16T09:37:07.375843+0000","last_became_active":"2024-09-16T09:33:30.251587+0000","last_became_peered":"2024-09-16T09:33:30.251587+0000","last_unstale":"2024-09-16T09:37:07.375843+0000","last_undegraded":"2024-09-16T09:37:07.375843+0000","last_fullsized":"2024-09-16T09:37:07.375843+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:43.326852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:43.326852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:37:29.639718+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,0],"acting":[4,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.10","version":"347'8498","reported_seq":27771,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.671093+0000","last_change":"2024-09-16T09:36:50.039921+0000","last_active":"2024-09-16T09:37:07.671093+0000","last_peered":"2024-09-16T09:37:07.671093+0000","last_clean":"2024-09-16T09:37:07.671093+0000","last_became_active":"2024-09-16T09:36:49.314768+0000","last_became_peered":"2024-09-16T09:36:49.314768+0000","last_unstale":"2024-09-16T09:37:07.671093+0000","last_undegraded":"2024-09-16T09:37:07.671093+0000","last_fullsized":"2024-09-16T09:37:07.671093+0000","mapping_epoch":328,"log_start":"263'4125","ondisk_log_start":"263'4125","created":243,"last_epoch_clean":329,"parent":"0.0","parent_split_bits":5,"last_scrub":"329'8188","last_scrub_stamp":"2024-09-16T09:36:50.039825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:36:50.039825+0000","objects_scrubbed":922,"log_size":4373,"log_dups_size":0,"ondisk_log_size":4373,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:14.016676+0000","scrub_duration":104,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40108032,"num_objects":612,"num_object_clones":0,"num_object_copies":1836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":612,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":310,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,2],"acting":[6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":461,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.372330+0000","last_change":"2024-09-16T09:34:37.804874+0000","last_active":"2024-09-16T09:37:07.372330+0000","last_peered":"2024-09-16T09:37:07.372330+0000","last_clean":"2024-09-16T09:37:07.372330+0000","last_became_active":"2024-09-16T09:28:52.974255+0000","last_became_peered":"2024-09-16T09:28:52.974255+0000","last_unstale":"2024-09-16T09:37:07.372330+0000","last_undegraded":"2024-09-16T09:37:07.372330+0000","last_fullsized":"2024-09-16T09:37:07.372330+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:37.804805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:34:37.804805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":624,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.375433+0000","last_change":"2024-09-16T09:35:23.751992+0000","last_active":"2024-09-16T09:37:07.375433+0000","last_peered":"2024-09-16T09:37:07.375433+0000","last_clean":"2024-09-16T09:37:07.375433+0000","last_became_active":"2024-09-16T09:25:01.706491+0000","last_became_peered":"2024-09-16T09:25:01.706491+0000","last_unstale":"2024-09-16T09:37:07.375433+0000","last_undegraded":"2024-09-16T09:37:07.375433+0000","last_fullsized":"2024-09-16T09:37:07.375433+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:35:23.751946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:35:23.751946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"285'18","reported_seq":300,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.375922+0000","last_change":"2024-09-16T09:33:39.284526+0000","last_active":"2024-09-16T09:37:07.375922+0000","last_peered":"2024-09-16T09:37:07.375922+0000","last_clean":"2024-09-16T09:37:07.375922+0000","last_became_active":"2024-09-16T09:33:30.251626+0000","last_became_peered":"2024-09-16T09:33:30.251626+0000","last_unstale":"2024-09-16T09:37:07.375922+0000","last_undegraded":"2024-09-16T09:37:07.375922+0000","last_fullsized":"2024-09-16T09:37:07.375922+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-16T09:33:39.284451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:33:39.284451+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:37:14.798648+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":33,"num_read_kb":24,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":600,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.375302+0000","last_change":"2024-09-16T09:33:38.238661+0000","last_active":"2024-09-16T09:37:07.375302+0000","last_peered":"2024-09-16T09:37:07.375302+0000","last_clean":"2024-09-16T09:37:07.375302+0000","last_became_active":"2024-09-16T09:25:05.710093+0000","last_became_peered":"2024-09-16T09:25:05.710093+0000","last_unstale":"2024-09-16T09:37:07.375302+0000","last_undegraded":"2024-09-16T09:37:07.375302+0000","last_fullsized":"2024-09-16T09:37:07.375302+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:38.238593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:38.238593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:37:22.765467+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":604,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.378652+0000","last_change":"2024-09-16T09:33:46.253182+0000","last_active":"2024-09-16T09:37:07.378652+0000","last_peered":"2024-09-16T09:37:07.378652+0000","last_clean":"2024-09-16T09:37:07.378652+0000","last_became_active":"2024-09-16T09:25:07.719380+0000","last_became_peered":"2024-09-16T09:25:07.719380+0000","last_unstale":"2024-09-16T09:37:07.378652+0000","last_undegraded":"2024-09-16T09:37:07.378652+0000","last_fullsized":"2024-09-16T09:37:07.378652+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-16T09:33:46.253114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:33:46.253114+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":257,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.368225+0000","last_change":"2024-09-16T09:34:41.401691+0000","last_active":"2024-09-16T09:37:07.368225+0000","last_peered":"2024-09-16T09:37:07.368225+0000","last_clean":"2024-09-16T09:37:07.368225+0000","last_became_active":"2024-09-16T09:33:30.245048+0000","last_became_peered":"2024-09-16T09:33:30.245048+0000","last_unstale":"2024-09-16T09:37:07.368225+0000","last_undegraded":"2024-09-16T09:37:07.368225+0000","last_fullsized":"2024-09-16T09:37:07.368225+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:34:41.401641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:34:41.401641+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:01.710748+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"285'20","reported_seq":684,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-16T09:37:08.372624+0000","last_change":"2024-09-16T09:32:30.286077+0000","last_active":"2024-09-16T09:37:08.372624+0000","last_peered":"2024-09-16T09:37:08.372624+0000","last_clean":"2024-09-16T09:37:08.372624+0000","last_became_active":"2024-09-16T09:31:09.275050+0000","last_became_peered":"2024-09-16T09:31:09.275050+0000","last_unstale":"2024-09-16T09:37:08.372624+0000","last_undegraded":"2024-09-16T09:37:08.372624+0000","last_fullsized":"2024-09-16T09:37:08.372624+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-16T09:32:30.284919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:32:30.284919+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:01.547406+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"66'8","reported_seq":790,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.372383+0000","last_change":"2024-09-16T09:34:52.922123+0000","last_active":"2024-09-16T09:37:07.372383+0000","last_peered":"2024-09-16T09:37:07.372383+0000","last_clean":"2024-09-16T09:37:07.372383+0000","last_became_active":"2024-09-16T09:25:05.712516+0000","last_became_peered":"2024-09-16T09:25:05.712516+0000","last_unstale":"2024-09-16T09:37:07.372383+0000","last_undegraded":"2024-09-16T09:37:07.372383+0000","last_fullsized":"2024-09-16T09:37:07.372383+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:34:52.922047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:34:52.922047+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":33,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2","version":"347'8722","reported_seq":8869,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:08.343788+0000","last_change":"2024-09-16T09:37:07.371676+0000","last_active":"2024-09-16T09:37:08.343788+0000","last_peered":"2024-09-16T09:37:08.343788+0000","last_clean":"2024-09-16T09:37:08.343788+0000","last_became_active":"2024-09-16T09:37:07.370603+0000","last_became_peered":"2024-09-16T09:37:07.370603+0000","last_unstale":"2024-09-16T09:37:08.343788+0000","last_undegraded":"2024-09-16T09:37:08.343788+0000","last_fullsized":"2024-09-16T09:37:08.343788+0000","mapping_epoch":233,"log_start":"343'8569","ondisk_log_start":"343'8569","created":243,"last_epoch_clean":347,"parent":"0.0","parent_split_bits":0,"last_scrub":"335'8228","last_scrub_stamp":"2024-09-16T09:36:56.054247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:36:56.054247+0000","objects_scrubbed":907,"log_size":153,"log_dups_size":6000,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:17.408287+0000","scrub_duration":287,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":86638592,"num_objects":1322,"num_object_clones":0,"num_object_copies":3966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1322,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9000,"num_write_kb":171008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3873,"num_bytes_recovered":229834752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.a","version":"73'79","reported_seq":539,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.366256+0000","last_change":"2024-09-16T09:29:18.498887+0000","last_active":"2024-09-16T09:37:07.366256+0000","last_peered":"2024-09-16T09:37:07.366256+0000","last_clean":"2024-09-16T09:37:07.366256+0000","last_became_active":"2024-09-16T09:26:35.127352+0000","last_became_peered":"2024-09-16T09:26:35.127352+0000","last_unstale":"2024-09-16T09:37:07.366256+0000","last_undegraded":"2024-09-16T09:37:07.366256+0000","last_fullsized":"2024-09-16T09:37:07.366256+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:18.498766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:18.498766+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:37:33.100091+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"285'27","reported_seq":574,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.367296+0000","last_change":"2024-09-16T09:36:47.588508+0000","last_active":"2024-09-16T09:37:07.367296+0000","last_peered":"2024-09-16T09:37:07.367296+0000","last_clean":"2024-09-16T09:37:07.367296+0000","last_became_active":"2024-09-16T09:26:33.295672+0000","last_became_peered":"2024-09-16T09:26:33.295672+0000","last_unstale":"2024-09-16T09:37:07.367296+0000","last_undegraded":"2024-09-16T09:37:07.367296+0000","last_fullsized":"2024-09-16T09:37:07.367296+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'27","last_scrub_stamp":"2024-09-16T09:36:47.588449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:36:47.588449+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:14.427263+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":342,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.368095+0000","last_change":"2024-09-16T09:34:39.420386+0000","last_active":"2024-09-16T09:37:07.368095+0000","last_peered":"2024-09-16T09:37:07.368095+0000","last_clean":"2024-09-16T09:37:07.368095+0000","last_became_active":"2024-09-16T09:31:09.274534+0000","last_became_peered":"2024-09-16T09:31:09.274534+0000","last_unstale":"2024-09-16T09:37:07.368095+0000","last_undegraded":"2024-09-16T09:37:07.368095+0000","last_fullsized":"2024-09-16T09:37:07.368095+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:39.420314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:34:39.420314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"66'8","reported_seq":747,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.375948+0000","last_change":"2024-09-16T09:35:10.915838+0000","last_active":"2024-09-16T09:37:07.375948+0000","last_peered":"2024-09-16T09:37:07.375948+0000","last_clean":"2024-09-16T09:37:07.375948+0000","last_became_active":"2024-09-16T09:25:05.713813+0000","last_became_peered":"2024-09-16T09:25:05.713813+0000","last_unstale":"2024-09-16T09:37:07.375948+0000","last_undegraded":"2024-09-16T09:37:07.375948+0000","last_fullsized":"2024-09-16T09:37:07.375948+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:35:10.915780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:35:10.915780+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:37:32.658273+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":604,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.366808+0000","last_change":"2024-09-16T09:36:37.152446+0000","last_active":"2024-09-16T09:37:07.366808+0000","last_peered":"2024-09-16T09:37:07.366808+0000","last_clean":"2024-09-16T09:37:07.366808+0000","last_became_active":"2024-09-16T09:25:07.717344+0000","last_became_peered":"2024-09-16T09:25:07.717344+0000","last_unstale":"2024-09-16T09:37:07.366808+0000","last_undegraded":"2024-09-16T09:37:07.366808+0000","last_fullsized":"2024-09-16T09:37:07.366808+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:37.152377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:36:37.152377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:37:38.943155+0000","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"347'13847","reported_seq":21232,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.896130+0000","last_change":"2024-09-16T09:35:09.710620+0000","last_active":"2024-09-16T09:37:07.896130+0000","last_peered":"2024-09-16T09:37:07.896130+0000","last_clean":"2024-09-16T09:37:07.896130+0000","last_became_active":"2024-09-16T09:35:09.702658+0000","last_became_peered":"2024-09-16T09:35:09.702658+0000","last_unstale":"2024-09-16T09:37:07.896130+0000","last_undegraded":"2024-09-16T09:37:07.896130+0000","last_fullsized":"2024-09-16T09:37:07.896130+0000","mapping_epoch":285,"log_start":"292'9700","ondisk_log_start":"292'9700","created":229,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:33:24.230486+0000","objects_scrubbed":0,"log_size":4147,"log_dups_size":0,"ondisk_log_size":4147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":87293976,"num_objects":1333,"num_object_clones":0,"num_object_copies":3999,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1333,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13847,"num_write_kb":485697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3554,"num_bytes_recovered":232914944,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.5","version":"73'79","reported_seq":580,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.376127+0000","last_change":"2024-09-16T09:32:42.123139+0000","last_active":"2024-09-16T09:37:07.376127+0000","last_peered":"2024-09-16T09:37:07.376127+0000","last_clean":"2024-09-16T09:37:07.376127+0000","last_became_active":"2024-09-16T09:30:08.763683+0000","last_became_peered":"2024-09-16T09:30:08.763683+0000","last_unstale":"2024-09-16T09:37:07.376127+0000","last_undegraded":"2024-09-16T09:37:07.376127+0000","last_fullsized":"2024-09-16T09:37:07.376127+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:32:42.123067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:32:42.123067+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"285'30","reported_seq":573,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.376805+0000","last_change":"2024-09-16T09:33:21.333996+0000","last_active":"2024-09-16T09:37:07.376805+0000","last_peered":"2024-09-16T09:37:07.376805+0000","last_clean":"2024-09-16T09:37:07.376805+0000","last_became_active":"2024-09-16T09:31:09.270434+0000","last_became_peered":"2024-09-16T09:31:09.270434+0000","last_unstale":"2024-09-16T09:37:07.376805+0000","last_undegraded":"2024-09-16T09:37:07.376805+0000","last_fullsized":"2024-09-16T09:37:07.376805+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'18","last_scrub_stamp":"2024-09-16T09:33:21.333937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:33:21.333937+0000","objects_scrubbed":8,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":51,"num_read_kb":39,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":452,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.377730+0000","last_change":"2024-09-16T09:36:50.622852+0000","last_active":"2024-09-16T09:37:07.377730+0000","last_peered":"2024-09-16T09:37:07.377730+0000","last_clean":"2024-09-16T09:37:07.377730+0000","last_became_active":"2024-09-16T09:28:52.983543+0000","last_became_peered":"2024-09-16T09:28:52.983543+0000","last_unstale":"2024-09-16T09:37:07.377730+0000","last_undegraded":"2024-09-16T09:37:07.377730+0000","last_fullsized":"2024-09-16T09:37:07.377730+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:50.622807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:36:50.622807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:37:54.700623+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":631,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.374868+0000","last_change":"2024-09-16T09:35:19.822713+0000","last_active":"2024-09-16T09:37:07.374868+0000","last_peered":"2024-09-16T09:37:07.374868+0000","last_clean":"2024-09-16T09:37:07.374868+0000","last_became_active":"2024-09-16T09:31:09.268864+0000","last_became_peered":"2024-09-16T09:31:09.268864+0000","last_unstale":"2024-09-16T09:37:07.374868+0000","last_undegraded":"2024-09-16T09:37:07.374868+0000","last_fullsized":"2024-09-16T09:37:07.374868+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:35:19.822635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:35:19.822635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":619,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.375299+0000","last_change":"2024-09-16T09:35:46.933987+0000","last_active":"2024-09-16T09:37:07.375299+0000","last_peered":"2024-09-16T09:37:07.375299+0000","last_clean":"2024-09-16T09:37:07.375299+0000","last_became_active":"2024-09-16T09:31:09.269433+0000","last_became_peered":"2024-09-16T09:31:09.269433+0000","last_unstale":"2024-09-16T09:37:07.375299+0000","last_undegraded":"2024-09-16T09:37:07.375299+0000","last_fullsized":"2024-09-16T09:37:07.375299+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:35:46.933912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:35:46.933912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.c","version":"347'13672","reported_seq":13969,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.923203+0000","last_change":"2024-09-16T09:36:41.907258+0000","last_active":"2024-09-16T09:37:07.923203+0000","last_peered":"2024-09-16T09:37:07.923203+0000","last_clean":"2024-09-16T09:37:07.923203+0000","last_became_active":"2024-09-16T09:33:25.371230+0000","last_became_peered":"2024-09-16T09:33:25.371230+0000","last_unstale":"2024-09-16T09:37:07.923203+0000","last_undegraded":"2024-09-16T09:37:07.923203+0000","last_fullsized":"2024-09-16T09:37:07.923203+0000","mapping_epoch":229,"log_start":"291'9300","ondisk_log_start":"291'9300","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"320'12934","last_scrub_stamp":"2024-09-16T09:36:41.907153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:36:41.907153+0000","objects_scrubbed":2082,"log_size":4372,"log_dups_size":0,"ondisk_log_size":4372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:37:48.003426+0000","scrub_duration":229,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080384,"num_objects":1344,"num_object_clones":0,"num_object_copies":4032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1344,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13672,"num_write_kb":480512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"73'79","reported_seq":342,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.368649+0000","last_change":"2024-09-16T09:34:42.362203+0000","last_active":"2024-09-16T09:37:07.368649+0000","last_peered":"2024-09-16T09:37:07.368649+0000","last_clean":"2024-09-16T09:37:07.368649+0000","last_became_active":"2024-09-16T09:31:09.275139+0000","last_became_peered":"2024-09-16T09:31:09.275139+0000","last_unstale":"2024-09-16T09:37:07.368649+0000","last_undegraded":"2024-09-16T09:37:07.368649+0000","last_fullsized":"2024-09-16T09:37:07.368649+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:34:42.362121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:34:42.362121+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"285'28","reported_seq":663,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.374916+0000","last_change":"2024-09-16T09:35:43.914378+0000","last_active":"2024-09-16T09:37:07.374916+0000","last_peered":"2024-09-16T09:37:07.374916+0000","last_clean":"2024-09-16T09:37:07.374916+0000","last_became_active":"2024-09-16T09:25:03.708830+0000","last_became_peered":"2024-09-16T09:25:03.708830+0000","last_unstale":"2024-09-16T09:37:07.374916+0000","last_undegraded":"2024-09-16T09:37:07.374916+0000","last_fullsized":"2024-09-16T09:37:07.374916+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'28","last_scrub_stamp":"2024-09-16T09:35:43.914312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:35:43.914312+0000","objects_scrubbed":7,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":636,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-16T09:37:08.372765+0000","last_change":"2024-09-16T09:33:43.678231+0000","last_active":"2024-09-16T09:37:08.372765+0000","last_peered":"2024-09-16T09:37:08.372765+0000","last_clean":"2024-09-16T09:37:08.372765+0000","last_became_active":"2024-09-16T09:25:01.702992+0000","last_became_peered":"2024-09-16T09:25:01.702992+0000","last_unstale":"2024-09-16T09:37:08.372765+0000","last_undegraded":"2024-09-16T09:37:08.372765+0000","last_fullsized":"2024-09-16T09:37:08.372765+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:33:43.678138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:43.678138+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"235'12","reported_seq":310,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.374854+0000","last_change":"2024-09-16T09:33:30.251570+0000","last_active":"2024-09-16T09:37:07.374854+0000","last_peered":"2024-09-16T09:37:07.374854+0000","last_clean":"2024-09-16T09:37:07.374854+0000","last_became_active":"2024-09-16T09:33:30.251047+0000","last_became_peered":"2024-09-16T09:33:30.251047+0000","last_unstale":"2024-09-16T09:37:07.374854+0000","last_undegraded":"2024-09-16T09:37:07.374854+0000","last_fullsized":"2024-09-16T09:37:07.374854+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"187'11","last_scrub_stamp":"2024-09-16T09:33:09.417852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:09.417852+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:37:07.764604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":596,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.377220+0000","last_change":"2024-09-16T09:36:47.605655+0000","last_active":"2024-09-16T09:37:07.377220+0000","last_peered":"2024-09-16T09:37:07.377220+0000","last_clean":"2024-09-16T09:37:07.377220+0000","last_became_active":"2024-09-16T09:25:07.722017+0000","last_became_peered":"2024-09-16T09:25:07.722017+0000","last_unstale":"2024-09-16T09:37:07.377220+0000","last_undegraded":"2024-09-16T09:37:07.377220+0000","last_fullsized":"2024-09-16T09:37:07.377220+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:47.605615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:36:47.605615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:08.852140+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"347'14061","reported_seq":14329,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:08.182121+0000","last_change":"2024-09-16T09:33:25.372320+0000","last_active":"2024-09-16T09:37:08.182121+0000","last_peered":"2024-09-16T09:37:08.182121+0000","last_clean":"2024-09-16T09:37:08.182121+0000","last_became_active":"2024-09-16T09:33:25.372005+0000","last_became_peered":"2024-09-16T09:33:25.372005+0000","last_unstale":"2024-09-16T09:37:08.182121+0000","last_undegraded":"2024-09-16T09:37:08.182121+0000","last_fullsized":"2024-09-16T09:37:08.182121+0000","mapping_epoch":229,"log_start":"291'9412","ondisk_log_start":"291'9412","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:33:24.230486+0000","objects_scrubbed":0,"log_size":4649,"log_dups_size":0,"ondisk_log_size":4649,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88276992,"num_objects":1347,"num_object_clones":0,"num_object_copies":4041,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1347,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14061,"num_write_kb":493056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.7","version":"73'79","reported_seq":673,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.372267+0000","last_change":"2024-09-16T09:34:51.864131+0000","last_active":"2024-09-16T09:37:07.372267+0000","last_peered":"2024-09-16T09:37:07.372267+0000","last_clean":"2024-09-16T09:37:07.372267+0000","last_became_active":"2024-09-16T09:31:09.274707+0000","last_became_peered":"2024-09-16T09:31:09.274707+0000","last_unstale":"2024-09-16T09:37:07.372267+0000","last_undegraded":"2024-09-16T09:37:07.372267+0000","last_fullsized":"2024-09-16T09:37:07.372267+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:34:51.864036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:34:51.864036+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"285'20","reported_seq":298,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.366533+0000","last_change":"2024-09-16T09:33:41.368102+0000","last_active":"2024-09-16T09:37:07.366533+0000","last_peered":"2024-09-16T09:37:07.366533+0000","last_clean":"2024-09-16T09:37:07.366533+0000","last_became_active":"2024-09-16T09:33:30.248453+0000","last_became_peered":"2024-09-16T09:33:30.248453+0000","last_unstale":"2024-09-16T09:37:07.366533+0000","last_undegraded":"2024-09-16T09:37:07.366533+0000","last_fullsized":"2024-09-16T09:37:07.366533+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-16T09:33:41.368016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:33:41.368016+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":630,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.366912+0000","last_change":"2024-09-16T09:36:47.095703+0000","last_active":"2024-09-16T09:37:07.366912+0000","last_peered":"2024-09-16T09:37:07.366912+0000","last_clean":"2024-09-16T09:37:07.366912+0000","last_became_active":"2024-09-16T09:31:09.270619+0000","last_became_peered":"2024-09-16T09:31:09.270619+0000","last_unstale":"2024-09-16T09:37:07.366912+0000","last_undegraded":"2024-09-16T09:37:07.366912+0000","last_fullsized":"2024-09-16T09:37:07.366912+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:47.095658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:36:47.095658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:02.698363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"66'8","reported_seq":765,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-16T09:37:08.374902+0000","last_change":"2024-09-16T09:36:52.689167+0000","last_active":"2024-09-16T09:37:08.374902+0000","last_peered":"2024-09-16T09:37:08.374902+0000","last_clean":"2024-09-16T09:37:08.374902+0000","last_became_active":"2024-09-16T09:25:05.714507+0000","last_became_peered":"2024-09-16T09:25:05.714507+0000","last_unstale":"2024-09-16T09:37:08.374902+0000","last_undegraded":"2024-09-16T09:37:08.374902+0000","last_fullsized":"2024-09-16T09:37:08.374902+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:36:52.689091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:36:52.689091+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:06.275939+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":621,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-16T09:37:08.373818+0000","last_change":"2024-09-16T09:32:33.362233+0000","last_active":"2024-09-16T09:37:08.373818+0000","last_peered":"2024-09-16T09:37:08.373818+0000","last_clean":"2024-09-16T09:37:08.373818+0000","last_became_active":"2024-09-16T09:31:09.283192+0000","last_became_peered":"2024-09-16T09:31:09.283192+0000","last_unstale":"2024-09-16T09:37:08.373818+0000","last_undegraded":"2024-09-16T09:37:08.373818+0000","last_fullsized":"2024-09-16T09:37:08.373818+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:33.362188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:32:33.362188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:02.526988+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.e","version":"347'13994","reported_seq":14263,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.899023+0000","last_change":"2024-09-16T09:33:25.247961+0000","last_active":"2024-09-16T09:37:07.899023+0000","last_peered":"2024-09-16T09:37:07.899023+0000","last_clean":"2024-09-16T09:37:07.899023+0000","last_became_active":"2024-09-16T09:33:25.247683+0000","last_became_peered":"2024-09-16T09:33:25.247683+0000","last_unstale":"2024-09-16T09:37:07.899023+0000","last_undegraded":"2024-09-16T09:37:07.899023+0000","last_fullsized":"2024-09-16T09:37:07.899023+0000","mapping_epoch":229,"log_start":"294'9900","ondisk_log_start":"294'9900","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:33:24.230486+0000","objects_scrubbed":0,"log_size":4094,"log_dups_size":0,"ondisk_log_size":4094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":86507520,"num_objects":1320,"num_object_clones":0,"num_object_copies":3960,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1320,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13994,"num_write_kb":490048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.6","version":"73'79","reported_seq":646,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.371818+0000","last_change":"2024-09-16T09:34:42.728232+0000","last_active":"2024-09-16T09:37:07.371818+0000","last_peered":"2024-09-16T09:37:07.371818+0000","last_clean":"2024-09-16T09:37:07.371818+0000","last_became_active":"2024-09-16T09:31:09.274565+0000","last_became_peered":"2024-09-16T09:31:09.274565+0000","last_unstale":"2024-09-16T09:37:07.371818+0000","last_undegraded":"2024-09-16T09:37:07.371818+0000","last_fullsized":"2024-09-16T09:37:07.371818+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:34:42.728143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:34:42.728143+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"285'29","reported_seq":666,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-16T09:37:08.375212+0000","last_change":"2024-09-16T09:36:54.604573+0000","last_active":"2024-09-16T09:37:08.375212+0000","last_peered":"2024-09-16T09:37:08.375212+0000","last_clean":"2024-09-16T09:37:08.375212+0000","last_became_active":"2024-09-16T09:25:03.714797+0000","last_became_peered":"2024-09-16T09:25:03.714797+0000","last_unstale":"2024-09-16T09:37:08.375212+0000","last_undegraded":"2024-09-16T09:37:08.375212+0000","last_fullsized":"2024-09-16T09:37:08.375212+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'29","last_scrub_stamp":"2024-09-16T09:36:54.604484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:36:54.604484+0000","objects_scrubbed":8,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:21.529216+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":54,"num_read_kb":40,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":240,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.366399+0000","last_change":"2024-09-16T09:33:30.247311+0000","last_active":"2024-09-16T09:37:07.366399+0000","last_peered":"2024-09-16T09:37:07.366399+0000","last_clean":"2024-09-16T09:37:07.366399+0000","last_became_active":"2024-09-16T09:33:30.246996+0000","last_became_peered":"2024-09-16T09:33:30.246996+0000","last_unstale":"2024-09-16T09:37:07.366399+0000","last_undegraded":"2024-09-16T09:37:07.366399+0000","last_fullsized":"2024-09-16T09:37:07.366399+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:11.407616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:11.407616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,2],"acting":[5,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":625,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.371691+0000","last_change":"2024-09-16T09:34:45.772748+0000","last_active":"2024-09-16T09:37:07.371691+0000","last_peered":"2024-09-16T09:37:07.371691+0000","last_clean":"2024-09-16T09:37:07.371691+0000","last_became_active":"2024-09-16T09:25:05.708663+0000","last_became_peered":"2024-09-16T09:25:05.708663+0000","last_unstale":"2024-09-16T09:37:07.371691+0000","last_undegraded":"2024-09-16T09:37:07.371691+0000","last_fullsized":"2024-09-16T09:37:07.371691+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:45.772674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:34:45.772674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":248,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.371632+0000","last_change":"2024-09-16T09:34:36.806165+0000","last_active":"2024-09-16T09:37:07.371632+0000","last_peered":"2024-09-16T09:37:07.371632+0000","last_clean":"2024-09-16T09:37:07.371632+0000","last_became_active":"2024-09-16T09:33:30.250178+0000","last_became_peered":"2024-09-16T09:33:30.250178+0000","last_unstale":"2024-09-16T09:37:07.371632+0000","last_undegraded":"2024-09-16T09:37:07.371632+0000","last_fullsized":"2024-09-16T09:37:07.371632+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:36.805971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:34:36.805971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,2],"acting":[6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"348'10125","reported_seq":10479,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-16T09:37:08.568221+0000","last_change":"2024-09-16T09:36:09.351656+0000","last_active":"2024-09-16T09:37:08.568221+0000","last_peered":"2024-09-16T09:37:08.568221+0000","last_clean":"2024-09-16T09:37:08.568221+0000","last_became_active":"2024-09-16T09:36:09.349892+0000","last_became_peered":"2024-09-16T09:36:09.349892+0000","last_unstale":"2024-09-16T09:37:08.568221+0000","last_undegraded":"2024-09-16T09:37:08.568221+0000","last_fullsized":"2024-09-16T09:37:08.568221+0000","mapping_epoch":229,"log_start":"303'7145","ondisk_log_start":"303'7145","created":243,"last_epoch_clean":307,"parent":"0.0","parent_split_bits":0,"last_scrub":"291'6329","last_scrub_stamp":"2024-09-16T09:35:36.136797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:35:36.136797+0000","objects_scrubbed":2998,"log_size":2980,"log_dups_size":6000,"ondisk_log_size":2980,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":920,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":85196800,"num_objects":1300,"num_object_clones":0,"num_object_copies":3900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1300,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6570,"num_write_kb":71616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2504,"num_bytes_recovered":164102144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,5],"acting":[0,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"73'79","reported_seq":580,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.366130+0000","last_change":"2024-09-16T09:36:42.191578+0000","last_active":"2024-09-16T09:37:07.366130+0000","last_peered":"2024-09-16T09:37:07.366130+0000","last_clean":"2024-09-16T09:37:07.366130+0000","last_became_active":"2024-09-16T09:30:08.766350+0000","last_became_peered":"2024-09-16T09:30:08.766350+0000","last_unstale":"2024-09-16T09:37:07.366130+0000","last_undegraded":"2024-09-16T09:37:07.366130+0000","last_fullsized":"2024-09-16T09:37:07.366130+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:36:42.191521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:36:42.191521+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:08.048449+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"285'42","reported_seq":898,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.370760+0000","last_change":"2024-09-16T09:36:49.100039+0000","last_active":"2024-09-16T09:37:07.370760+0000","last_peered":"2024-09-16T09:37:07.370760+0000","last_clean":"2024-09-16T09:37:07.370760+0000","last_became_active":"2024-09-16T09:31:09.278839+0000","last_became_peered":"2024-09-16T09:31:09.278839+0000","last_unstale":"2024-09-16T09:37:07.370760+0000","last_undegraded":"2024-09-16T09:37:07.370760+0000","last_fullsized":"2024-09-16T09:37:07.370760+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'42","last_scrub_stamp":"2024-09-16T09:36:49.099960+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:36:49.099960+0000","objects_scrubbed":10,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:37:50.536835+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":66,"num_read_kb":51,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":625,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.375349+0000","last_change":"2024-09-16T09:36:35.691463+0000","last_active":"2024-09-16T09:37:07.375349+0000","last_peered":"2024-09-16T09:37:07.375349+0000","last_clean":"2024-09-16T09:37:07.375349+0000","last_became_active":"2024-09-16T09:25:01.703076+0000","last_became_peered":"2024-09-16T09:25:01.703076+0000","last_unstale":"2024-09-16T09:37:07.375349+0000","last_undegraded":"2024-09-16T09:37:07.375349+0000","last_fullsized":"2024-09-16T09:37:07.375349+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:35.691415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:36:35.691415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:37:38.143566+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":626,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.366657+0000","last_change":"2024-09-16T09:33:38.417375+0000","last_active":"2024-09-16T09:37:07.366657+0000","last_peered":"2024-09-16T09:37:07.366657+0000","last_clean":"2024-09-16T09:37:07.366657+0000","last_became_active":"2024-09-16T09:25:00.696237+0000","last_became_peered":"2024-09-16T09:25:00.696237+0000","last_unstale":"2024-09-16T09:37:07.366657+0000","last_undegraded":"2024-09-16T09:37:07.366657+0000","last_fullsized":"2024-09-16T09:37:07.366657+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-16T09:33:38.417308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:33:38.417308+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.0025137430000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":595,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-16T09:37:08.372657+0000","last_change":"2024-09-16T09:33:16.666704+0000","last_active":"2024-09-16T09:37:08.372657+0000","last_peered":"2024-09-16T09:37:08.372657+0000","last_clean":"2024-09-16T09:37:08.372657+0000","last_became_active":"2024-09-16T09:25:05.717591+0000","last_became_peered":"2024-09-16T09:25:05.717591+0000","last_unstale":"2024-09-16T09:37:08.372657+0000","last_undegraded":"2024-09-16T09:37:08.372657+0000","last_fullsized":"2024-09-16T09:37:08.372657+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:16.666648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:16.666648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":342,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.369218+0000","last_change":"2024-09-16T09:36:50.667709+0000","last_active":"2024-09-16T09:37:07.369218+0000","last_peered":"2024-09-16T09:37:07.369218+0000","last_clean":"2024-09-16T09:37:07.369218+0000","last_became_active":"2024-09-16T09:31:09.272609+0000","last_became_peered":"2024-09-16T09:31:09.272609+0000","last_unstale":"2024-09-16T09:37:07.369218+0000","last_undegraded":"2024-09-16T09:37:07.369218+0000","last_fullsized":"2024-09-16T09:37:07.369218+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:50.667661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:36:50.667661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:09.567715+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.a","version":"347'13663","reported_seq":13956,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:08.156324+0000","last_change":"2024-09-16T09:36:57.195191+0000","last_active":"2024-09-16T09:37:08.156324+0000","last_peered":"2024-09-16T09:37:08.156324+0000","last_clean":"2024-09-16T09:37:08.156324+0000","last_became_active":"2024-09-16T09:33:25.243701+0000","last_became_peered":"2024-09-16T09:33:25.243701+0000","last_unstale":"2024-09-16T09:37:08.156324+0000","last_undegraded":"2024-09-16T09:37:08.156324+0000","last_fullsized":"2024-09-16T09:37:08.156324+0000","mapping_epoch":229,"log_start":"291'9007","ondisk_log_start":"291'9007","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"336'13288","last_scrub_stamp":"2024-09-16T09:36:57.195079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:36:57.195079+0000","objects_scrubbed":1690,"log_size":4656,"log_dups_size":0,"ondisk_log_size":4656,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:19.164762+0000","scrub_duration":184,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":86179840,"num_objects":1315,"num_object_clones":0,"num_object_copies":3945,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1315,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13663,"num_write_kb":479296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"73'79","reported_seq":298,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-16T09:37:08.372174+0000","last_change":"2024-09-16T09:34:32.717994+0000","last_active":"2024-09-16T09:37:08.372174+0000","last_peered":"2024-09-16T09:37:08.372174+0000","last_clean":"2024-09-16T09:37:08.372174+0000","last_became_active":"2024-09-16T09:34:00.336514+0000","last_became_peered":"2024-09-16T09:34:00.336514+0000","last_unstale":"2024-09-16T09:37:08.372174+0000","last_undegraded":"2024-09-16T09:37:08.372174+0000","last_fullsized":"2024-09-16T09:37:08.372174+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:34:32.717915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:34:32.717915+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"285'17","reported_seq":312,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.367016+0000","last_change":"2024-09-16T09:34:15.319682+0000","last_active":"2024-09-16T09:37:07.367016+0000","last_peered":"2024-09-16T09:37:07.367016+0000","last_clean":"2024-09-16T09:37:07.367016+0000","last_became_active":"2024-09-16T09:34:00.337121+0000","last_became_peered":"2024-09-16T09:34:00.337121+0000","last_unstale":"2024-09-16T09:37:07.367016+0000","last_undegraded":"2024-09-16T09:37:07.367016+0000","last_fullsized":"2024-09-16T09:37:07.367016+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'13","last_scrub_stamp":"2024-09-16T09:27:44.155969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:27:44.155969+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:37:26.933332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":35,"num_read_kb":24,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":240,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.371451+0000","last_change":"2024-09-16T09:33:30.248260+0000","last_active":"2024-09-16T09:37:07.371451+0000","last_peered":"2024-09-16T09:37:07.371451+0000","last_clean":"2024-09-16T09:37:07.371451+0000","last_became_active":"2024-09-16T09:33:30.247596+0000","last_became_peered":"2024-09-16T09:33:30.247596+0000","last_unstale":"2024-09-16T09:37:07.371451+0000","last_undegraded":"2024-09-16T09:37:07.371451+0000","last_fullsized":"2024-09-16T09:37:07.371451+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:22.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:22.331302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,2],"acting":[6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":277,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-16T09:37:08.378555+0000","last_change":"2024-09-16T09:37:04.362941+0000","last_active":"2024-09-16T09:37:08.378555+0000","last_peered":"2024-09-16T09:37:08.378555+0000","last_clean":"2024-09-16T09:37:08.378555+0000","last_became_active":"2024-09-16T09:37:04.358985+0000","last_became_peered":"2024-09-16T09:37:04.358985+0000","last_unstale":"2024-09-16T09:37:08.378555+0000","last_undegraded":"2024-09-16T09:37:08.378555+0000","last_fullsized":"2024-09-16T09:37:08.378555+0000","mapping_epoch":343,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":344,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:32:40.165749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:32:40.165749+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":633,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.368169+0000","last_change":"2024-09-16T09:36:55.530225+0000","last_active":"2024-09-16T09:37:07.368169+0000","last_peered":"2024-09-16T09:37:07.368169+0000","last_clean":"2024-09-16T09:37:07.368169+0000","last_became_active":"2024-09-16T09:25:05.711539+0000","last_became_peered":"2024-09-16T09:25:05.711539+0000","last_unstale":"2024-09-16T09:37:07.368169+0000","last_undegraded":"2024-09-16T09:37:07.368169+0000","last_fullsized":"2024-09-16T09:37:07.368169+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:55.530175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:36:55.530175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:18.377901+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":604,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.377311+0000","last_change":"2024-09-16T09:33:39.313624+0000","last_active":"2024-09-16T09:37:07.377311+0000","last_peered":"2024-09-16T09:37:07.377311+0000","last_clean":"2024-09-16T09:37:07.377311+0000","last_became_active":"2024-09-16T09:25:07.723282+0000","last_became_peered":"2024-09-16T09:25:07.723282+0000","last_unstale":"2024-09-16T09:37:07.377311+0000","last_undegraded":"2024-09-16T09:37:07.377311+0000","last_fullsized":"2024-09-16T09:37:07.377311+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:39.313562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:33:39.313562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.b","version":"348'13625","reported_seq":13897,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-16T09:37:08.564866+0000","last_change":"2024-09-16T09:33:25.245108+0000","last_active":"2024-09-16T09:37:08.564866+0000","last_peered":"2024-09-16T09:37:08.564866+0000","last_clean":"2024-09-16T09:37:08.564866+0000","last_became_active":"2024-09-16T09:33:25.244819+0000","last_became_peered":"2024-09-16T09:33:25.244819+0000","last_unstale":"2024-09-16T09:37:08.564866+0000","last_undegraded":"2024-09-16T09:37:08.564866+0000","last_fullsized":"2024-09-16T09:37:08.564866+0000","mapping_epoch":229,"log_start":"289'8600","ondisk_log_start":"289'8600","created":229,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:33:24.230486+0000","objects_scrubbed":0,"log_size":5025,"log_dups_size":0,"ondisk_log_size":5025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":82247680,"num_objects":1255,"num_object_clones":0,"num_object_copies":3765,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1255,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13625,"num_write_kb":476160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"73'79","reported_seq":329,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.371855+0000","last_change":"2024-09-16T09:34:31.929462+0000","last_active":"2024-09-16T09:37:07.371855+0000","last_peered":"2024-09-16T09:37:07.371855+0000","last_clean":"2024-09-16T09:37:07.371855+0000","last_became_active":"2024-09-16T09:33:30.249309+0000","last_became_peered":"2024-09-16T09:33:30.249309+0000","last_unstale":"2024-09-16T09:37:07.371855+0000","last_undegraded":"2024-09-16T09:37:07.371855+0000","last_fullsized":"2024-09-16T09:37:07.371855+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:34:31.929388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:34:31.929388+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.6","version":"285'24","reported_seq":643,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-16T09:37:08.372638+0000","last_change":"2024-09-16T09:32:48.547237+0000","last_active":"2024-09-16T09:37:08.372638+0000","last_peered":"2024-09-16T09:37:08.372638+0000","last_clean":"2024-09-16T09:37:08.372638+0000","last_became_active":"2024-09-16T09:25:03.715315+0000","last_became_peered":"2024-09-16T09:25:03.715315+0000","last_unstale":"2024-09-16T09:37:08.372638+0000","last_undegraded":"2024-09-16T09:37:08.372638+0000","last_fullsized":"2024-09-16T09:37:08.372638+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-16T09:32:48.546869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:32:48.546869+0000","objects_scrubbed":6,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":36,"num_read_kb":30,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"59'1","reported_seq":642,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-16T09:37:08.375777+0000","last_change":"2024-09-16T09:33:21.682893+0000","last_active":"2024-09-16T09:37:08.375777+0000","last_peered":"2024-09-16T09:37:08.375777+0000","last_clean":"2024-09-16T09:37:08.375777+0000","last_became_active":"2024-09-16T09:31:09.269627+0000","last_became_peered":"2024-09-16T09:31:09.269627+0000","last_unstale":"2024-09-16T09:37:08.375777+0000","last_undegraded":"2024-09-16T09:37:08.375777+0000","last_fullsized":"2024-09-16T09:37:08.375777+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:33:21.682840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:21.682840+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1986,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"67'5","reported_seq":1348,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.376979+0000","last_change":"2024-09-16T09:33:37.358839+0000","last_active":"2024-09-16T09:37:07.376979+0000","last_peered":"2024-09-16T09:37:07.376979+0000","last_clean":"2024-09-16T09:37:07.376979+0000","last_became_active":"2024-09-16T09:25:00.696546+0000","last_became_peered":"2024-09-16T09:25:00.696546+0000","last_unstale":"2024-09-16T09:37:07.376979+0000","last_undegraded":"2024-09-16T09:37:07.376979+0000","last_fullsized":"2024-09-16T09:37:07.376979+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'5","last_scrub_stamp":"2024-09-16T09:33:37.358741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:33:37.358741+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0.0059770420000000001,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":720,"num_read_kb":715,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":592,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.367021+0000","last_change":"2024-09-16T09:27:07.337170+0000","last_active":"2024-09-16T09:37:07.367021+0000","last_peered":"2024-09-16T09:37:07.367021+0000","last_clean":"2024-09-16T09:37:07.367021+0000","last_became_active":"2024-09-16T09:25:05.714704+0000","last_became_peered":"2024-09-16T09:25:05.714704+0000","last_unstale":"2024-09-16T09:37:07.367021+0000","last_undegraded":"2024-09-16T09:37:07.367021+0000","last_fullsized":"2024-09-16T09:37:07.367021+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:07.337124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:07.337124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:37:19.897485+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":518,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.366558+0000","last_change":"2024-09-16T09:36:43.150813+0000","last_active":"2024-09-16T09:37:07.366558+0000","last_peered":"2024-09-16T09:37:07.366558+0000","last_clean":"2024-09-16T09:37:07.366558+0000","last_became_active":"2024-09-16T09:33:30.245852+0000","last_became_peered":"2024-09-16T09:33:30.245852+0000","last_unstale":"2024-09-16T09:37:07.366558+0000","last_undegraded":"2024-09-16T09:37:07.366558+0000","last_fullsized":"2024-09-16T09:37:07.366558+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:43.150764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:36:43.150764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:37:48.439707+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"348'8914","reported_seq":14573,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-16T09:37:08.566102+0000","last_change":"2024-09-16T09:36:43.655321+0000","last_active":"2024-09-16T09:37:08.566102+0000","last_peered":"2024-09-16T09:37:08.566102+0000","last_clean":"2024-09-16T09:37:08.566102+0000","last_became_active":"2024-09-16T09:36:43.654159+0000","last_became_peered":"2024-09-16T09:36:43.654159+0000","last_unstale":"2024-09-16T09:37:08.566102+0000","last_undegraded":"2024-09-16T09:37:08.566102+0000","last_fullsized":"2024-09-16T09:37:08.566102+0000","mapping_epoch":272,"log_start":"319'8082","ondisk_log_start":"319'8082","created":243,"last_epoch_clean":323,"parent":"0.0","parent_split_bits":0,"last_scrub":"275'4947","last_scrub_stamp":"2024-09-16T09:34:50.611186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:34:50.611186+0000","objects_scrubbed":3078,"log_size":832,"log_dups_size":6000,"ondisk_log_size":832,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":697,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":87031808,"num_objects":1328,"num_object_clones":0,"num_object_copies":3984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1328,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7441,"num_write_kb":74944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5349,"num_bytes_recovered":339935232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"285'28","reported_seq":807,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.371914+0000","last_change":"2024-09-16T09:33:16.891850+0000","last_active":"2024-09-16T09:37:07.371914+0000","last_peered":"2024-09-16T09:37:07.371914+0000","last_clean":"2024-09-16T09:37:07.371914+0000","last_became_active":"2024-09-16T09:25:03.714118+0000","last_became_peered":"2024-09-16T09:25:03.714118+0000","last_unstale":"2024-09-16T09:37:07.371914+0000","last_undegraded":"2024-09-16T09:37:07.371914+0000","last_fullsized":"2024-09-16T09:37:07.371914+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'16","last_scrub_stamp":"2024-09-16T09:33:16.891761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:33:16.891761+0000","objects_scrubbed":8,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":43,"num_read_kb":33,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":466,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.366575+0000","last_change":"2024-09-16T09:36:50.114910+0000","last_active":"2024-09-16T09:37:07.366575+0000","last_peered":"2024-09-16T09:37:07.366575+0000","last_clean":"2024-09-16T09:37:07.366575+0000","last_became_active":"2024-09-16T09:31:09.270196+0000","last_became_peered":"2024-09-16T09:31:09.270196+0000","last_unstale":"2024-09-16T09:37:07.366575+0000","last_undegraded":"2024-09-16T09:37:07.366575+0000","last_fullsized":"2024-09-16T09:37:07.366575+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:50.114861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:36:50.114861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:13.001323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"73'79","reported_seq":697,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.377041+0000","last_change":"2024-09-16T09:32:43.535041+0000","last_active":"2024-09-16T09:37:07.377041+0000","last_peered":"2024-09-16T09:37:07.377041+0000","last_clean":"2024-09-16T09:37:07.377041+0000","last_became_active":"2024-09-16T09:30:08.870394+0000","last_became_peered":"2024-09-16T09:30:08.870394+0000","last_unstale":"2024-09-16T09:37:07.377041+0000","last_undegraded":"2024-09-16T09:37:07.377041+0000","last_fullsized":"2024-09-16T09:37:07.377041+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:32:43.534962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:32:43.534962+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":600,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.378742+0000","last_change":"2024-09-16T09:33:52.291748+0000","last_active":"2024-09-16T09:37:07.378742+0000","last_peered":"2024-09-16T09:37:07.378742+0000","last_clean":"2024-09-16T09:37:07.378742+0000","last_became_active":"2024-09-16T09:25:05.714341+0000","last_became_peered":"2024-09-16T09:25:05.714341+0000","last_unstale":"2024-09-16T09:37:07.378742+0000","last_undegraded":"2024-09-16T09:37:07.378742+0000","last_fullsized":"2024-09-16T09:37:07.378742+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:52.291702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:52.291702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":596,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.366928+0000","last_change":"2024-09-16T09:29:00.243528+0000","last_active":"2024-09-16T09:37:07.366928+0000","last_peered":"2024-09-16T09:37:07.366928+0000","last_clean":"2024-09-16T09:37:07.366928+0000","last_became_active":"2024-09-16T09:25:07.714281+0000","last_became_peered":"2024-09-16T09:25:07.714281+0000","last_unstale":"2024-09-16T09:37:07.366928+0000","last_undegraded":"2024-09-16T09:37:07.366928+0000","last_fullsized":"2024-09-16T09:37:07.366928+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:00.243482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:29:00.243482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:37:32.082677+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3","version":"347'8555","reported_seq":8776,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.896064+0000","last_change":"2024-09-16T09:37:03.351343+0000","last_active":"2024-09-16T09:37:07.896064+0000","last_peered":"2024-09-16T09:37:07.896064+0000","last_clean":"2024-09-16T09:37:07.896064+0000","last_became_active":"2024-09-16T09:37:03.350847+0000","last_became_peered":"2024-09-16T09:37:03.350847+0000","last_unstale":"2024-09-16T09:37:07.896064+0000","last_undegraded":"2024-09-16T09:37:07.896064+0000","last_fullsized":"2024-09-16T09:37:07.896064+0000","mapping_epoch":229,"log_start":"339'8346","ondisk_log_start":"339'8346","created":243,"last_epoch_clean":343,"parent":"0.0","parent_split_bits":0,"last_scrub":"273'4719","last_scrub_stamp":"2024-09-16T09:34:44.662413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:34:44.662413+0000","objects_scrubbed":1554,"log_size":209,"log_dups_size":6000,"ondisk_log_size":209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":88080384,"num_objects":1344,"num_object_clones":0,"num_object_copies":4032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1344,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9031,"num_write_kb":180480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1862,"num_bytes_recovered":121962496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.b","version":"73'79","reported_seq":434,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-16T09:37:08.374669+0000","last_change":"2024-09-16T09:34:31.720971+0000","last_active":"2024-09-16T09:37:08.374669+0000","last_peered":"2024-09-16T09:37:08.374669+0000","last_clean":"2024-09-16T09:37:08.374669+0000","last_became_active":"2024-09-16T09:33:30.251933+0000","last_became_peered":"2024-09-16T09:33:30.251933+0000","last_unstale":"2024-09-16T09:37:08.374669+0000","last_undegraded":"2024-09-16T09:37:08.374669+0000","last_fullsized":"2024-09-16T09:37:08.374669+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:34:31.720787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:34:31.720787+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"285'19","reported_seq":674,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.376032+0000","last_change":"2024-09-16T09:35:27.813299+0000","last_active":"2024-09-16T09:37:07.376032+0000","last_peered":"2024-09-16T09:37:07.376032+0000","last_clean":"2024-09-16T09:37:07.376032+0000","last_became_active":"2024-09-16T09:34:00.328533+0000","last_became_peered":"2024-09-16T09:34:00.328533+0000","last_unstale":"2024-09-16T09:37:07.376032+0000","last_undegraded":"2024-09-16T09:37:07.376032+0000","last_fullsized":"2024-09-16T09:37:07.376032+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'19","last_scrub_stamp":"2024-09-16T09:35:27.813206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:35:27.813206+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,3],"acting":[4,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":601,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.375144+0000","last_change":"2024-09-16T09:33:23.200213+0000","last_active":"2024-09-16T09:37:07.375144+0000","last_peered":"2024-09-16T09:37:07.375144+0000","last_clean":"2024-09-16T09:37:07.375144+0000","last_became_active":"2024-09-16T09:25:01.708270+0000","last_became_peered":"2024-09-16T09:25:01.708270+0000","last_unstale":"2024-09-16T09:37:07.375144+0000","last_undegraded":"2024-09-16T09:37:07.375144+0000","last_fullsized":"2024-09-16T09:37:07.375144+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:23.200166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:23.200166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":592,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.366807+0000","last_change":"2024-09-16T09:26:34.171190+0000","last_active":"2024-09-16T09:37:07.366807+0000","last_peered":"2024-09-16T09:37:07.366807+0000","last_clean":"2024-09-16T09:37:07.366807+0000","last_became_active":"2024-09-16T09:25:05.711964+0000","last_became_peered":"2024-09-16T09:25:05.711964+0000","last_unstale":"2024-09-16T09:37:07.366807+0000","last_undegraded":"2024-09-16T09:37:07.366807+0000","last_fullsized":"2024-09-16T09:37:07.366807+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:34.171139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:26:34.171139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:37:14.922914+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":483,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.371862+0000","last_change":"2024-09-16T09:34:50.866121+0000","last_active":"2024-09-16T09:37:07.371862+0000","last_peered":"2024-09-16T09:37:07.371862+0000","last_clean":"2024-09-16T09:37:07.371862+0000","last_became_active":"2024-09-16T09:31:09.278618+0000","last_became_peered":"2024-09-16T09:31:09.278618+0000","last_unstale":"2024-09-16T09:37:07.371862+0000","last_undegraded":"2024-09-16T09:37:07.371862+0000","last_fullsized":"2024-09-16T09:37:07.371862+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:50.866032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:34:50.866032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.0","version":"347'8493","reported_seq":8811,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.697110+0000","last_change":"2024-09-16T09:35:37.795621+0000","last_active":"2024-09-16T09:37:07.697110+0000","last_peered":"2024-09-16T09:37:07.697110+0000","last_clean":"2024-09-16T09:37:07.697110+0000","last_became_active":"2024-09-16T09:33:52.914803+0000","last_became_peered":"2024-09-16T09:33:52.914803+0000","last_unstale":"2024-09-16T09:37:07.697110+0000","last_undegraded":"2024-09-16T09:37:07.697110+0000","last_fullsized":"2024-09-16T09:37:07.697110+0000","mapping_epoch":233,"log_start":"251'3523","ondisk_log_start":"251'3523","created":229,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"291'6312","last_scrub_stamp":"2024-09-16T09:35:37.795459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:35:37.795459+0000","objects_scrubbed":2847,"log_size":4970,"log_dups_size":0,"ondisk_log_size":4970,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43057152,"num_objects":657,"num_object_clones":0,"num_object_copies":1971,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":657,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2190,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,2],"acting":[6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.8","version":"73'79","reported_seq":224,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.372132+0000","last_change":"2024-09-16T09:34:30.931922+0000","last_active":"2024-09-16T09:37:07.372132+0000","last_peered":"2024-09-16T09:37:07.372132+0000","last_clean":"2024-09-16T09:37:07.372132+0000","last_became_active":"2024-09-16T09:34:00.329241+0000","last_became_peered":"2024-09-16T09:34:00.329241+0000","last_unstale":"2024-09-16T09:37:07.372132+0000","last_undegraded":"2024-09-16T09:37:07.372132+0000","last_fullsized":"2024-09-16T09:37:07.372132+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:34:30.931863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:34:30.931863+0000","objects_scrubbed":1,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262688,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,3],"acting":[6,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"285'31","reported_seq":670,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.376015+0000","last_change":"2024-09-16T09:35:28.861520+0000","last_active":"2024-09-16T09:37:07.376015+0000","last_peered":"2024-09-16T09:37:07.376015+0000","last_clean":"2024-09-16T09:37:07.376015+0000","last_became_active":"2024-09-16T09:25:03.713358+0000","last_became_peered":"2024-09-16T09:25:03.713358+0000","last_unstale":"2024-09-16T09:37:07.376015+0000","last_undegraded":"2024-09-16T09:37:07.376015+0000","last_fullsized":"2024-09-16T09:37:07.376015+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'31","last_scrub_stamp":"2024-09-16T09:35:28.861433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:35:28.861433+0000","objects_scrubbed":8,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":50,"num_read_kb":40,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":609,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.372043+0000","last_change":"2024-09-16T09:34:32.899965+0000","last_active":"2024-09-16T09:37:07.372043+0000","last_peered":"2024-09-16T09:37:07.372043+0000","last_clean":"2024-09-16T09:37:07.372043+0000","last_became_active":"2024-09-16T09:25:01.703355+0000","last_became_peered":"2024-09-16T09:25:01.703355+0000","last_unstale":"2024-09-16T09:37:07.372043+0000","last_undegraded":"2024-09-16T09:37:07.372043+0000","last_fullsized":"2024-09-16T09:37:07.372043+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:32.899886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:34:32.899886+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":248,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.375174+0000","last_change":"2024-09-16T09:33:44.298324+0000","last_active":"2024-09-16T09:37:07.375174+0000","last_peered":"2024-09-16T09:37:07.375174+0000","last_clean":"2024-09-16T09:37:07.375174+0000","last_became_active":"2024-09-16T09:33:30.251186+0000","last_became_peered":"2024-09-16T09:33:30.251186+0000","last_unstale":"2024-09-16T09:37:07.375174+0000","last_undegraded":"2024-09-16T09:37:07.375174+0000","last_fullsized":"2024-09-16T09:37:07.375174+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:44.298246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:44.298246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:37:30.644758+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,0],"acting":[4,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":546,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.368941+0000","last_change":"2024-09-16T09:36:51.640537+0000","last_active":"2024-09-16T09:37:07.368941+0000","last_peered":"2024-09-16T09:37:07.368941+0000","last_clean":"2024-09-16T09:37:07.368941+0000","last_became_active":"2024-09-16T09:33:21.337291+0000","last_became_peered":"2024-09-16T09:33:21.337291+0000","last_unstale":"2024-09-16T09:37:07.368941+0000","last_undegraded":"2024-09-16T09:37:07.368941+0000","last_fullsized":"2024-09-16T09:37:07.368941+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:51.640472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:36:51.640472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:06.647486+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4],"acting":[3,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1","version":"347'8418","reported_seq":8768,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.701690+0000","last_change":"2024-09-16T09:35:36.575967+0000","last_active":"2024-09-16T09:37:07.701690+0000","last_peered":"2024-09-16T09:37:07.701690+0000","last_clean":"2024-09-16T09:37:07.701690+0000","last_became_active":"2024-09-16T09:33:52.910622+0000","last_became_peered":"2024-09-16T09:33:52.910622+0000","last_unstale":"2024-09-16T09:37:07.701690+0000","last_undegraded":"2024-09-16T09:37:07.701690+0000","last_fullsized":"2024-09-16T09:37:07.701690+0000","mapping_epoch":229,"log_start":"252'3407","ondisk_log_start":"252'3407","created":229,"last_epoch_clean":244,"parent":"0.0","parent_split_bits":0,"last_scrub":"291'6189","last_scrub_stamp":"2024-09-16T09:35:36.575666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:35:36.575666+0000","objects_scrubbed":2923,"log_size":5011,"log_dups_size":0,"ondisk_log_size":5011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":620,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":44695552,"num_objects":682,"num_object_clones":0,"num_object_copies":2046,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":682,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2241,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.9","version":"73'79","reported_seq":672,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.378268+0000","last_change":"2024-09-16T09:29:19.013948+0000","last_active":"2024-09-16T09:37:07.378268+0000","last_peered":"2024-09-16T09:37:07.378268+0000","last_clean":"2024-09-16T09:37:07.378268+0000","last_became_active":"2024-09-16T09:26:35.126066+0000","last_became_peered":"2024-09-16T09:26:35.126066+0000","last_unstale":"2024-09-16T09:37:07.378268+0000","last_undegraded":"2024-09-16T09:37:07.378268+0000","last_fullsized":"2024-09-16T09:37:07.378268+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:19.013865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:19.013865+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:02.493078+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"285'20","reported_seq":627,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.375397+0000","last_change":"2024-09-16T09:33:24.154567+0000","last_active":"2024-09-16T09:37:07.375397+0000","last_peered":"2024-09-16T09:37:07.375397+0000","last_clean":"2024-09-16T09:37:07.375397+0000","last_became_active":"2024-09-16T09:25:03.707723+0000","last_became_peered":"2024-09-16T09:25:03.707723+0000","last_unstale":"2024-09-16T09:37:07.375397+0000","last_undegraded":"2024-09-16T09:37:07.375397+0000","last_fullsized":"2024-09-16T09:37:07.375397+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-16T09:33:24.154507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:33:24.154507+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":476,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-16T09:37:08.375395+0000","last_change":"2024-09-16T09:33:10.528181+0000","last_active":"2024-09-16T09:37:08.375395+0000","last_peered":"2024-09-16T09:37:08.375395+0000","last_clean":"2024-09-16T09:37:08.375395+0000","last_became_active":"2024-09-16T09:31:09.288825+0000","last_became_peered":"2024-09-16T09:31:09.288825+0000","last_unstale":"2024-09-16T09:37:08.375395+0000","last_undegraded":"2024-09-16T09:37:08.375395+0000","last_fullsized":"2024-09-16T09:37:08.375395+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:10.528115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:10.528115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"228'9","reported_seq":327,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.377398+0000","last_change":"2024-09-16T09:33:21.337623+0000","last_active":"2024-09-16T09:37:07.377398+0000","last_peered":"2024-09-16T09:37:07.377398+0000","last_clean":"2024-09-16T09:37:07.377398+0000","last_became_active":"2024-09-16T09:33:21.337213+0000","last_became_peered":"2024-09-16T09:33:21.337213+0000","last_unstale":"2024-09-16T09:37:07.377398+0000","last_undegraded":"2024-09-16T09:37:07.377398+0000","last_fullsized":"2024-09-16T09:37:07.377398+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:32:42.175528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:32:42.175528+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":621,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.375720+0000","last_change":"2024-09-16T09:35:36.061013+0000","last_active":"2024-09-16T09:37:07.375720+0000","last_peered":"2024-09-16T09:37:07.375720+0000","last_clean":"2024-09-16T09:37:07.375720+0000","last_became_active":"2024-09-16T09:25:07.718958+0000","last_became_peered":"2024-09-16T09:25:07.718958+0000","last_unstale":"2024-09-16T09:37:07.375720+0000","last_undegraded":"2024-09-16T09:37:07.375720+0000","last_fullsized":"2024-09-16T09:37:07.375720+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:35:36.060942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:35:36.060942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.6","version":"347'8912","reported_seq":9265,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:08.184845+0000","last_change":"2024-09-16T09:36:51.317058+0000","last_active":"2024-09-16T09:37:08.184845+0000","last_peered":"2024-09-16T09:37:08.184845+0000","last_clean":"2024-09-16T09:37:08.184845+0000","last_became_active":"2024-09-16T09:36:51.316286+0000","last_became_peered":"2024-09-16T09:36:51.316286+0000","last_unstale":"2024-09-16T09:37:08.184845+0000","last_undegraded":"2024-09-16T09:37:08.184845+0000","last_fullsized":"2024-09-16T09:37:08.184845+0000","mapping_epoch":229,"log_start":"327'8255","ondisk_log_start":"327'8255","created":243,"last_epoch_clean":331,"parent":"0.0","parent_split_bits":0,"last_scrub":"291'6321","last_scrub_stamp":"2024-09-16T09:35:36.697883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:35:36.697883+0000","objects_scrubbed":2977,"log_size":657,"log_dups_size":6000,"ondisk_log_size":657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":949,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90374144,"num_objects":1379,"num_object_clones":0,"num_object_copies":4137,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1379,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6047,"num_write_kb":40576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2712,"num_bytes_recovered":177602560,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.e","version":"73'79","reported_seq":414,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.375660+0000","last_change":"2024-09-16T09:35:41.979877+0000","last_active":"2024-09-16T09:37:07.375660+0000","last_peered":"2024-09-16T09:37:07.375660+0000","last_clean":"2024-09-16T09:37:07.375660+0000","last_became_active":"2024-09-16T09:30:10.705072+0000","last_became_peered":"2024-09-16T09:30:10.705072+0000","last_unstale":"2024-09-16T09:37:07.375660+0000","last_undegraded":"2024-09-16T09:37:07.375660+0000","last_fullsized":"2024-09-16T09:37:07.375660+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:35:41.979814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:35:41.979814+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,3],"acting":[4,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.b","version":"285'15","reported_seq":665,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-16T09:37:08.375122+0000","last_change":"2024-09-16T09:36:55.651561+0000","last_active":"2024-09-16T09:37:08.375122+0000","last_peered":"2024-09-16T09:37:08.375122+0000","last_clean":"2024-09-16T09:37:08.375122+0000","last_became_active":"2024-09-16T09:36:31.590202+0000","last_became_peered":"2024-09-16T09:36:31.590202+0000","last_unstale":"2024-09-16T09:37:08.375122+0000","last_undegraded":"2024-09-16T09:37:08.375122+0000","last_fullsized":"2024-09-16T09:37:08.375122+0000","mapping_epoch":316,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":317,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'15","last_scrub_stamp":"2024-09-16T09:36:55.651496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:36:55.651496+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:01.510778+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":614,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.366619+0000","last_change":"2024-09-16T09:31:09.277413+0000","last_active":"2024-09-16T09:37:07.366619+0000","last_peered":"2024-09-16T09:37:07.366619+0000","last_clean":"2024-09-16T09:37:07.366619+0000","last_became_active":"2024-09-16T09:31:09.276516+0000","last_became_peered":"2024-09-16T09:31:09.276516+0000","last_unstale":"2024-09-16T09:37:07.366619+0000","last_undegraded":"2024-09-16T09:37:07.366619+0000","last_fullsized":"2024-09-16T09:37:07.366619+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:26.061106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:26:26.061106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:37:17.890836+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":542,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.375162+0000","last_change":"2024-09-16T09:33:40.260547+0000","last_active":"2024-09-16T09:37:07.375162+0000","last_peered":"2024-09-16T09:37:07.375162+0000","last_clean":"2024-09-16T09:37:07.375162+0000","last_became_active":"2024-09-16T09:33:21.337254+0000","last_became_peered":"2024-09-16T09:33:21.337254+0000","last_unstale":"2024-09-16T09:37:07.375162+0000","last_undegraded":"2024-09-16T09:37:07.375162+0000","last_fullsized":"2024-09-16T09:37:07.375162+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:40.260500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:40.260500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:37:24.718816+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3],"acting":[4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":615,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-16T09:37:08.372729+0000","last_change":"2024-09-16T09:36:53.647305+0000","last_active":"2024-09-16T09:37:08.372729+0000","last_peered":"2024-09-16T09:37:08.372729+0000","last_clean":"2024-09-16T09:37:08.372729+0000","last_became_active":"2024-09-16T09:25:07.718778+0000","last_became_peered":"2024-09-16T09:25:07.718778+0000","last_unstale":"2024-09-16T09:37:08.372729+0000","last_undegraded":"2024-09-16T09:37:08.372729+0000","last_fullsized":"2024-09-16T09:37:08.372729+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:53.647256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:36:53.647256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:19.369534+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"347'8966","reported_seq":9210,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:08.168781+0000","last_change":"2024-09-16T09:36:47.704186+0000","last_active":"2024-09-16T09:37:08.168781+0000","last_peered":"2024-09-16T09:37:08.168781+0000","last_clean":"2024-09-16T09:37:08.168781+0000","last_became_active":"2024-09-16T09:36:47.703235+0000","last_became_peered":"2024-09-16T09:36:47.703235+0000","last_unstale":"2024-09-16T09:37:08.168781+0000","last_undegraded":"2024-09-16T09:37:08.168781+0000","last_fullsized":"2024-09-16T09:37:08.168781+0000","mapping_epoch":229,"log_start":"323'8223","ondisk_log_start":"323'8223","created":243,"last_epoch_clean":327,"parent":"0.0","parent_split_bits":0,"last_scrub":"276'5145","last_scrub_stamp":"2024-09-16T09:34:53.523609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:34:53.523609+0000","objects_scrubbed":1562,"log_size":743,"log_dups_size":6000,"ondisk_log_size":743,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":422,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":82640896,"num_objects":1261,"num_object_clones":0,"num_object_copies":3783,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1261,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8864,"num_write_kb":158848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3234,"num_bytes_recovered":211877888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.f","version":"73'79","reported_seq":666,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.372437+0000","last_change":"2024-09-16T09:34:53.866426+0000","last_active":"2024-09-16T09:37:07.372437+0000","last_peered":"2024-09-16T09:37:07.372437+0000","last_clean":"2024-09-16T09:37:07.372437+0000","last_became_active":"2024-09-16T09:30:10.693477+0000","last_became_peered":"2024-09-16T09:30:10.693477+0000","last_unstale":"2024-09-16T09:37:07.372437+0000","last_undegraded":"2024-09-16T09:37:07.372437+0000","last_fullsized":"2024-09-16T09:37:07.372437+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:34:53.866337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:34:53.866337+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"285'35","reported_seq":668,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.371787+0000","last_change":"2024-09-16T09:33:22.905808+0000","last_active":"2024-09-16T09:37:07.371787+0000","last_peered":"2024-09-16T09:37:07.371787+0000","last_clean":"2024-09-16T09:37:07.371787+0000","last_became_active":"2024-09-16T09:25:03.712783+0000","last_became_peered":"2024-09-16T09:25:03.712783+0000","last_unstale":"2024-09-16T09:37:07.371787+0000","last_undegraded":"2024-09-16T09:37:07.371787+0000","last_fullsized":"2024-09-16T09:37:07.371787+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'19","last_scrub_stamp":"2024-09-16T09:33:22.905752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:33:22.905752+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":56,"num_read_kb":45,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"59'1","reported_seq":622,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.377803+0000","last_change":"2024-09-16T09:33:42.294543+0000","last_active":"2024-09-16T09:37:07.377803+0000","last_peered":"2024-09-16T09:37:07.377803+0000","last_clean":"2024-09-16T09:37:07.377803+0000","last_became_active":"2024-09-16T09:25:01.708069+0000","last_became_peered":"2024-09-16T09:25:01.708069+0000","last_unstale":"2024-09-16T09:37:07.377803+0000","last_undegraded":"2024-09-16T09:37:07.377803+0000","last_fullsized":"2024-09-16T09:37:07.377803+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:33:42.294378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:42.294378+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":624,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.366935+0000","last_change":"2024-09-16T09:36:59.543452+0000","last_active":"2024-09-16T09:37:07.366935+0000","last_peered":"2024-09-16T09:37:07.366935+0000","last_clean":"2024-09-16T09:37:07.366935+0000","last_became_active":"2024-09-16T09:34:00.330050+0000","last_became_peered":"2024-09-16T09:34:00.330050+0000","last_unstale":"2024-09-16T09:37:07.366935+0000","last_undegraded":"2024-09-16T09:37:07.366935+0000","last_fullsized":"2024-09-16T09:37:07.366935+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:59.543358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:36:59.543358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:01.818651+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,7],"acting":[2,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":596,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.377032+0000","last_change":"2024-09-16T09:36:48.649739+0000","last_active":"2024-09-16T09:37:07.377032+0000","last_peered":"2024-09-16T09:37:07.377032+0000","last_clean":"2024-09-16T09:37:07.377032+0000","last_became_active":"2024-09-16T09:25:07.718750+0000","last_became_peered":"2024-09-16T09:25:07.718750+0000","last_unstale":"2024-09-16T09:37:07.377032+0000","last_undegraded":"2024-09-16T09:37:07.377032+0000","last_fullsized":"2024-09-16T09:37:07.377032+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:48.649696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:36:48.649696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:18.071944+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.4","version":"347'8860","reported_seq":9182,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.912843+0000","last_change":"2024-09-16T09:36:59.322547+0000","last_active":"2024-09-16T09:37:07.912843+0000","last_peered":"2024-09-16T09:37:07.912843+0000","last_clean":"2024-09-16T09:37:07.912843+0000","last_became_active":"2024-09-16T09:36:59.321590+0000","last_became_peered":"2024-09-16T09:36:59.321590+0000","last_unstale":"2024-09-16T09:37:07.912843+0000","last_undegraded":"2024-09-16T09:37:07.912843+0000","last_fullsized":"2024-09-16T09:37:07.912843+0000","mapping_epoch":233,"log_start":"335'8519","ondisk_log_start":"335'8519","created":229,"last_epoch_clean":339,"parent":"0.0","parent_split_bits":0,"last_scrub":"324'8386","last_scrub_stamp":"2024-09-16T09:36:45.776828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:36:45.776828+0000","objects_scrubbed":1007,"log_size":341,"log_dups_size":6000,"ondisk_log_size":341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:09.201274+0000","scrub_duration":170,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":91226112,"num_objects":1392,"num_object_clones":0,"num_object_copies":4176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1392,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7704,"num_write_kb":144704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.c","version":"73'79","reported_seq":354,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.368543+0000","last_change":"2024-09-16T09:36:53.697797+0000","last_active":"2024-09-16T09:37:07.368543+0000","last_peered":"2024-09-16T09:37:07.368543+0000","last_clean":"2024-09-16T09:37:07.368543+0000","last_became_active":"2024-09-16T09:31:09.273770+0000","last_became_peered":"2024-09-16T09:31:09.273770+0000","last_unstale":"2024-09-16T09:37:07.368543+0000","last_undegraded":"2024-09-16T09:37:07.368543+0000","last_fullsized":"2024-09-16T09:37:07.368543+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:36:53.697742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:36:53.697742+0000","objects_scrubbed":1,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:37:57.508460+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":327680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"285'18","reported_seq":691,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.375020+0000","last_change":"2024-09-16T09:36:50.603724+0000","last_active":"2024-09-16T09:37:07.375020+0000","last_peered":"2024-09-16T09:37:07.375020+0000","last_clean":"2024-09-16T09:37:07.375020+0000","last_became_active":"2024-09-16T09:31:09.269047+0000","last_became_peered":"2024-09-16T09:31:09.269047+0000","last_unstale":"2024-09-16T09:37:07.375020+0000","last_undegraded":"2024-09-16T09:37:07.375020+0000","last_fullsized":"2024-09-16T09:37:07.375020+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'18","last_scrub_stamp":"2024-09-16T09:36:50.603660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:36:50.603660+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:15.970176+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":616,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.378421+0000","last_change":"2024-09-16T09:33:48.225867+0000","last_active":"2024-09-16T09:37:07.378421+0000","last_peered":"2024-09-16T09:37:07.378421+0000","last_clean":"2024-09-16T09:37:07.378421+0000","last_became_active":"2024-09-16T09:25:01.707528+0000","last_became_peered":"2024-09-16T09:25:01.707528+0000","last_unstale":"2024-09-16T09:37:07.378421+0000","last_undegraded":"2024-09-16T09:37:07.378421+0000","last_fullsized":"2024-09-16T09:37:07.378421+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:48.225635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:48.225635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":262,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-16T09:37:08.374324+0000","last_change":"2024-09-16T09:33:23.643136+0000","last_active":"2024-09-16T09:37:08.374324+0000","last_peered":"2024-09-16T09:37:08.374324+0000","last_clean":"2024-09-16T09:37:08.374324+0000","last_became_active":"2024-09-16T09:33:21.337309+0000","last_became_peered":"2024-09-16T09:33:21.337309+0000","last_unstale":"2024-09-16T09:37:08.374324+0000","last_undegraded":"2024-09-16T09:37:08.374324+0000","last_fullsized":"2024-09-16T09:37:08.374324+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:23.643089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:23.643089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":605,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.368243+0000","last_change":"2024-09-16T09:36:59.713546+0000","last_active":"2024-09-16T09:37:07.368243+0000","last_peered":"2024-09-16T09:37:07.368243+0000","last_clean":"2024-09-16T09:37:07.368243+0000","last_became_active":"2024-09-16T09:25:07.718899+0000","last_became_peered":"2024-09-16T09:25:07.718899+0000","last_unstale":"2024-09-16T09:37:07.368243+0000","last_undegraded":"2024-09-16T09:37:07.368243+0000","last_fullsized":"2024-09-16T09:37:07.368243+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:59.713355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:36:59.713355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:05.628663+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.5","version":"348'8888","reported_seq":12688,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-16T09:37:08.525244+0000","last_change":"2024-09-16T09:36:55.746633+0000","last_active":"2024-09-16T09:37:08.525244+0000","last_peered":"2024-09-16T09:37:08.525244+0000","last_clean":"2024-09-16T09:37:08.525244+0000","last_became_active":"2024-09-16T09:36:55.745616+0000","last_became_peered":"2024-09-16T09:36:55.745616+0000","last_unstale":"2024-09-16T09:37:08.525244+0000","last_undegraded":"2024-09-16T09:37:08.525244+0000","last_fullsized":"2024-09-16T09:37:08.525244+0000","mapping_epoch":267,"log_start":"331'8336","ondisk_log_start":"331'8336","created":229,"last_epoch_clean":335,"parent":"0.0","parent_split_bits":0,"last_scrub":"269'4368","last_scrub_stamp":"2024-09-16T09:34:34.439986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:33:24.230486+0000","last_clean_scrub_stamp":"2024-09-16T09:34:34.439986+0000","objects_scrubbed":2756,"log_size":552,"log_dups_size":6000,"ondisk_log_size":552,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":710,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":85786624,"num_objects":1309,"num_object_clones":0,"num_object_copies":3927,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1309,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6354,"num_write_kb":69952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.d","version":"73'79","reported_seq":406,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.377135+0000","last_change":"2024-09-16T09:32:44.536868+0000","last_active":"2024-09-16T09:37:07.377135+0000","last_peered":"2024-09-16T09:37:07.377135+0000","last_clean":"2024-09-16T09:37:07.377135+0000","last_became_active":"2024-09-16T09:30:10.709958+0000","last_became_peered":"2024-09-16T09:30:10.709958+0000","last_unstale":"2024-09-16T09:37:07.377135+0000","last_undegraded":"2024-09-16T09:37:07.377135+0000","last_fullsized":"2024-09-16T09:37:07.377135+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:32:44.536799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:32:44.536799+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"285'27","reported_seq":639,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.366140+0000","last_change":"2024-09-16T09:26:33.138828+0000","last_active":"2024-09-16T09:37:07.366140+0000","last_peered":"2024-09-16T09:37:07.366140+0000","last_clean":"2024-09-16T09:37:07.366140+0000","last_became_active":"2024-09-16T09:25:03.715733+0000","last_became_peered":"2024-09-16T09:25:03.715733+0000","last_unstale":"2024-09-16T09:37:07.366140+0000","last_undegraded":"2024-09-16T09:37:07.366140+0000","last_fullsized":"2024-09-16T09:37:07.366140+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'15","last_scrub_stamp":"2024-09-16T09:26:33.138603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:26:33.138603+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:37:16.922468+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":620,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.377815+0000","last_change":"2024-09-16T09:32:49.441194+0000","last_active":"2024-09-16T09:37:07.377815+0000","last_peered":"2024-09-16T09:37:07.377815+0000","last_clean":"2024-09-16T09:37:07.377815+0000","last_became_active":"2024-09-16T09:25:01.707372+0000","last_became_peered":"2024-09-16T09:25:01.707372+0000","last_unstale":"2024-09-16T09:37:07.377815+0000","last_undegraded":"2024-09-16T09:37:07.377815+0000","last_fullsized":"2024-09-16T09:37:07.377815+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-16T09:32:49.441122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:32:49.441122+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"66'8","reported_seq":762,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.378125+0000","last_change":"2024-09-16T09:30:05.138362+0000","last_active":"2024-09-16T09:37:07.378125+0000","last_peered":"2024-09-16T09:37:07.378125+0000","last_clean":"2024-09-16T09:37:07.378125+0000","last_became_active":"2024-09-16T09:25:05.716189+0000","last_became_peered":"2024-09-16T09:25:05.716189+0000","last_unstale":"2024-09-16T09:37:07.378125+0000","last_undegraded":"2024-09-16T09:37:07.378125+0000","last_fullsized":"2024-09-16T09:37:07.378125+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:30:05.138288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:30:05.138288+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:05.431628+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":588,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.366370+0000","last_change":"2024-09-16T09:27:20.219376+0000","last_active":"2024-09-16T09:37:07.366370+0000","last_peered":"2024-09-16T09:37:07.366370+0000","last_clean":"2024-09-16T09:37:07.366370+0000","last_became_active":"2024-09-16T09:25:07.717213+0000","last_became_peered":"2024-09-16T09:25:07.717213+0000","last_unstale":"2024-09-16T09:37:07.366370+0000","last_undegraded":"2024-09-16T09:37:07.366370+0000","last_fullsized":"2024-09-16T09:37:07.366370+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:20.219333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:27:20.219333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:37:23.878316+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.12","version":"73'79","reported_seq":413,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.366483+0000","last_change":"2024-09-16T09:36:44.171217+0000","last_active":"2024-09-16T09:37:07.366483+0000","last_peered":"2024-09-16T09:37:07.366483+0000","last_clean":"2024-09-16T09:37:07.366483+0000","last_became_active":"2024-09-16T09:30:10.731413+0000","last_became_peered":"2024-09-16T09:30:10.731413+0000","last_unstale":"2024-09-16T09:37:07.366483+0000","last_undegraded":"2024-09-16T09:37:07.366483+0000","last_fullsized":"2024-09-16T09:37:07.366483+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:36:44.171151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:36:44.171151+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:37:50.857154+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":616,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.371428+0000","last_change":"2024-09-16T09:34:00.334779+0000","last_active":"2024-09-16T09:37:07.371428+0000","last_peered":"2024-09-16T09:37:07.371428+0000","last_clean":"2024-09-16T09:37:07.371428+0000","last_became_active":"2024-09-16T09:34:00.334361+0000","last_became_peered":"2024-09-16T09:34:00.334361+0000","last_unstale":"2024-09-16T09:37:07.371428+0000","last_undegraded":"2024-09-16T09:37:07.371428+0000","last_fullsized":"2024-09-16T09:37:07.371428+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:14.845355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:14.845355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:03.930437+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"285'12","reported_seq":367,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.369075+0000","last_change":"2024-09-16T09:34:51.181994+0000","last_active":"2024-09-16T09:37:07.369075+0000","last_peered":"2024-09-16T09:37:07.369075+0000","last_clean":"2024-09-16T09:37:07.369075+0000","last_became_active":"2024-09-16T09:31:09.270685+0000","last_became_peered":"2024-09-16T09:31:09.270685+0000","last_unstale":"2024-09-16T09:37:07.369075+0000","last_undegraded":"2024-09-16T09:37:07.369075+0000","last_fullsized":"2024-09-16T09:37:07.369075+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'6","last_scrub_stamp":"2024-09-16T09:34:51.181918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:34:51.181918+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":17,"num_read_kb":14,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":607,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.366702+0000","last_change":"2024-09-16T09:36:38.129449+0000","last_active":"2024-09-16T09:37:07.366702+0000","last_peered":"2024-09-16T09:37:07.366702+0000","last_clean":"2024-09-16T09:37:07.366702+0000","last_became_active":"2024-09-16T09:25:05.715616+0000","last_became_peered":"2024-09-16T09:25:05.715616+0000","last_unstale":"2024-09-16T09:37:07.366702+0000","last_undegraded":"2024-09-16T09:37:07.366702+0000","last_fullsized":"2024-09-16T09:37:07.366702+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:38.129314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:36:38.129314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:37:44.283058+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":595,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.377224+0000","last_change":"2024-09-16T09:36:37.715166+0000","last_active":"2024-09-16T09:37:07.377224+0000","last_peered":"2024-09-16T09:37:07.377224+0000","last_clean":"2024-09-16T09:37:07.377224+0000","last_became_active":"2024-09-16T09:25:07.721059+0000","last_became_peered":"2024-09-16T09:25:07.721059+0000","last_unstale":"2024-09-16T09:37:07.377224+0000","last_undegraded":"2024-09-16T09:37:07.377224+0000","last_fullsized":"2024-09-16T09:37:07.377224+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:37.715114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:36:37.715114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:37:58.590391+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.13","version":"73'79","reported_seq":499,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.368425+0000","last_change":"2024-09-16T09:34:48.260121+0000","last_active":"2024-09-16T09:37:07.368425+0000","last_peered":"2024-09-16T09:37:07.368425+0000","last_clean":"2024-09-16T09:37:07.368425+0000","last_became_active":"2024-09-16T09:33:30.248497+0000","last_became_peered":"2024-09-16T09:33:30.248497+0000","last_unstale":"2024-09-16T09:37:07.368425+0000","last_undegraded":"2024-09-16T09:37:07.368425+0000","last_fullsized":"2024-09-16T09:37:07.368425+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:34:48.260041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:34:48.260041+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"285'15","reported_seq":690,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-16T09:37:08.372717+0000","last_change":"2024-09-16T09:33:48.681742+0000","last_active":"2024-09-16T09:37:08.372717+0000","last_peered":"2024-09-16T09:37:08.372717+0000","last_clean":"2024-09-16T09:37:08.372717+0000","last_became_active":"2024-09-16T09:31:09.273498+0000","last_became_peered":"2024-09-16T09:31:09.273498+0000","last_unstale":"2024-09-16T09:37:08.372717+0000","last_undegraded":"2024-09-16T09:37:08.372717+0000","last_fullsized":"2024-09-16T09:37:08.372717+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:33:48.681650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:33:48.681650+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":616,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.378485+0000","last_change":"2024-09-16T09:33:13.312434+0000","last_active":"2024-09-16T09:37:07.378485+0000","last_peered":"2024-09-16T09:37:07.378485+0000","last_clean":"2024-09-16T09:37:07.378485+0000","last_became_active":"2024-09-16T09:25:01.706491+0000","last_became_peered":"2024-09-16T09:25:01.706491+0000","last_unstale":"2024-09-16T09:37:07.378485+0000","last_undegraded":"2024-09-16T09:37:07.378485+0000","last_fullsized":"2024-09-16T09:37:07.378485+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:13.312344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:13.312344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":373,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.377607+0000","last_change":"2024-09-16T09:36:54.596000+0000","last_active":"2024-09-16T09:37:07.377607+0000","last_peered":"2024-09-16T09:37:07.377607+0000","last_clean":"2024-09-16T09:37:07.377607+0000","last_became_active":"2024-09-16T09:33:30.248992+0000","last_became_peered":"2024-09-16T09:33:30.248992+0000","last_unstale":"2024-09-16T09:37:07.377607+0000","last_undegraded":"2024-09-16T09:37:07.377607+0000","last_fullsized":"2024-09-16T09:37:07.377607+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:54.595924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:36:54.595924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:10.833233+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":613,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.366969+0000","last_change":"2024-09-16T09:36:58.536167+0000","last_active":"2024-09-16T09:37:07.366969+0000","last_peered":"2024-09-16T09:37:07.366969+0000","last_clean":"2024-09-16T09:37:07.366969+0000","last_became_active":"2024-09-16T09:25:07.722050+0000","last_became_peered":"2024-09-16T09:25:07.722050+0000","last_unstale":"2024-09-16T09:37:07.366969+0000","last_undegraded":"2024-09-16T09:37:07.366969+0000","last_fullsized":"2024-09-16T09:37:07.366969+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:58.536121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:36:58.536121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:18.611763+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.10","version":"73'79","reported_seq":413,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.366035+0000","last_change":"2024-09-16T09:36:41.169117+0000","last_active":"2024-09-16T09:37:07.366035+0000","last_peered":"2024-09-16T09:37:07.366035+0000","last_clean":"2024-09-16T09:37:07.366035+0000","last_became_active":"2024-09-16T09:30:10.692770+0000","last_became_peered":"2024-09-16T09:30:10.692770+0000","last_unstale":"2024-09-16T09:37:07.366035+0000","last_undegraded":"2024-09-16T09:37:07.366035+0000","last_fullsized":"2024-09-16T09:37:07.366035+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:36:41.169069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:36:41.169069+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:08.440462+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"285'15","reported_seq":629,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.366961+0000","last_change":"2024-09-16T09:29:32.485165+0000","last_active":"2024-09-16T09:37:07.366961+0000","last_peered":"2024-09-16T09:37:07.366961+0000","last_clean":"2024-09-16T09:37:07.366961+0000","last_became_active":"2024-09-16T09:25:03.717310+0000","last_became_peered":"2024-09-16T09:25:03.717310+0000","last_unstale":"2024-09-16T09:37:07.366961+0000","last_undegraded":"2024-09-16T09:37:07.366961+0000","last_fullsized":"2024-09-16T09:37:07.366961+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:29:32.485084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:29:32.485084+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:37:35.087553+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":641,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-16T09:37:08.372451+0000","last_change":"2024-09-16T09:33:20.639405+0000","last_active":"2024-09-16T09:37:08.372451+0000","last_peered":"2024-09-16T09:37:08.372451+0000","last_clean":"2024-09-16T09:37:08.372451+0000","last_became_active":"2024-09-16T09:31:09.272848+0000","last_became_peered":"2024-09-16T09:31:09.272848+0000","last_unstale":"2024-09-16T09:37:08.372451+0000","last_undegraded":"2024-09-16T09:37:08.372451+0000","last_fullsized":"2024-09-16T09:37:08.372451+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:20.639350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:20.639350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"187'11","reported_seq":599,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.378664+0000","last_change":"2024-09-16T09:33:20.296801+0000","last_active":"2024-09-16T09:37:07.378664+0000","last_peered":"2024-09-16T09:37:07.378664+0000","last_clean":"2024-09-16T09:37:07.378664+0000","last_became_active":"2024-09-16T09:31:09.277688+0000","last_became_peered":"2024-09-16T09:31:09.277688+0000","last_unstale":"2024-09-16T09:37:07.378664+0000","last_undegraded":"2024-09-16T09:37:07.378664+0000","last_fullsized":"2024-09-16T09:37:07.378664+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"187'11","last_scrub_stamp":"2024-09-16T09:33:20.296746+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:20.296746+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":620,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.375617+0000","last_change":"2024-09-16T09:36:49.628523+0000","last_active":"2024-09-16T09:37:07.375617+0000","last_peered":"2024-09-16T09:37:07.375617+0000","last_clean":"2024-09-16T09:37:07.375617+0000","last_became_active":"2024-09-16T09:25:07.718810+0000","last_became_peered":"2024-09-16T09:25:07.718810+0000","last_unstale":"2024-09-16T09:37:07.375617+0000","last_undegraded":"2024-09-16T09:37:07.375617+0000","last_fullsized":"2024-09-16T09:37:07.375617+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:49.628477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:36:49.628477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:37:58.131251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.11","version":"73'79","reported_seq":493,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-16T09:37:08.375869+0000","last_change":"2024-09-16T09:32:38.401888+0000","last_active":"2024-09-16T09:37:08.375869+0000","last_peered":"2024-09-16T09:37:08.375869+0000","last_clean":"2024-09-16T09:37:08.375869+0000","last_became_active":"2024-09-16T09:31:09.272208+0000","last_became_peered":"2024-09-16T09:31:09.272208+0000","last_unstale":"2024-09-16T09:37:08.375869+0000","last_undegraded":"2024-09-16T09:37:08.375869+0000","last_fullsized":"2024-09-16T09:37:08.375869+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:32:38.401720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:32:38.401720+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":32,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.14","version":"285'20","reported_seq":298,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.378764+0000","last_change":"2024-09-16T09:36:52.609515+0000","last_active":"2024-09-16T09:37:07.378764+0000","last_peered":"2024-09-16T09:37:07.378764+0000","last_clean":"2024-09-16T09:37:07.378764+0000","last_became_active":"2024-09-16T09:33:30.249769+0000","last_became_peered":"2024-09-16T09:33:30.249769+0000","last_unstale":"2024-09-16T09:37:07.378764+0000","last_undegraded":"2024-09-16T09:37:07.378764+0000","last_fullsized":"2024-09-16T09:37:07.378764+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'20","last_scrub_stamp":"2024-09-16T09:36:52.609440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:36:52.609440+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:37:58.502398+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":608,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.377531+0000","last_change":"2024-09-16T09:29:56.021413+0000","last_active":"2024-09-16T09:37:07.377531+0000","last_peered":"2024-09-16T09:37:07.377531+0000","last_clean":"2024-09-16T09:37:07.377531+0000","last_became_active":"2024-09-16T09:25:01.707513+0000","last_became_peered":"2024-09-16T09:25:01.707513+0000","last_unstale":"2024-09-16T09:37:07.377531+0000","last_undegraded":"2024-09-16T09:37:07.377531+0000","last_fullsized":"2024-09-16T09:37:07.377531+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:56.021340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:56.021340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:03.515625+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"66'8","reported_seq":769,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.366573+0000","last_change":"2024-09-16T09:33:39.388721+0000","last_active":"2024-09-16T09:37:07.366573+0000","last_peered":"2024-09-16T09:37:07.366573+0000","last_clean":"2024-09-16T09:37:07.366573+0000","last_became_active":"2024-09-16T09:25:05.713833+0000","last_became_peered":"2024-09-16T09:25:05.713833+0000","last_unstale":"2024-09-16T09:37:07.366573+0000","last_undegraded":"2024-09-16T09:37:07.366573+0000","last_fullsized":"2024-09-16T09:37:07.366573+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:33:39.388463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:39.388463+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":628,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-16T09:37:08.372436+0000","last_change":"2024-09-16T09:33:42.666290+0000","last_active":"2024-09-16T09:37:08.372436+0000","last_peered":"2024-09-16T09:37:08.372436+0000","last_clean":"2024-09-16T09:37:08.372436+0000","last_became_active":"2024-09-16T09:31:09.275168+0000","last_became_peered":"2024-09-16T09:31:09.275168+0000","last_unstale":"2024-09-16T09:37:08.372436+0000","last_undegraded":"2024-09-16T09:37:08.372436+0000","last_fullsized":"2024-09-16T09:37:08.372436+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:42.666200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:33:42.666200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"285'19","reported_seq":647,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.375182+0000","last_change":"2024-09-16T09:33:19.084118+0000","last_active":"2024-09-16T09:37:07.375182+0000","last_peered":"2024-09-16T09:37:07.375182+0000","last_clean":"2024-09-16T09:37:07.375182+0000","last_became_active":"2024-09-16T09:26:33.291265+0000","last_became_peered":"2024-09-16T09:26:33.291265+0000","last_unstale":"2024-09-16T09:37:07.375182+0000","last_undegraded":"2024-09-16T09:37:07.375182+0000","last_fullsized":"2024-09-16T09:37:07.375182+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-16T09:33:19.084063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:33:19.084063+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:37:13.780897+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":625,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.375868+0000","last_change":"2024-09-16T09:35:24.722539+0000","last_active":"2024-09-16T09:37:07.375868+0000","last_peered":"2024-09-16T09:37:07.375868+0000","last_clean":"2024-09-16T09:37:07.375868+0000","last_became_active":"2024-09-16T09:25:01.705158+0000","last_became_peered":"2024-09-16T09:25:01.705158+0000","last_unstale":"2024-09-16T09:37:07.375868+0000","last_undegraded":"2024-09-16T09:37:07.375868+0000","last_fullsized":"2024-09-16T09:37:07.375868+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:35:24.722476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:35:24.722476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":537,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.366361+0000","last_change":"2024-09-16T09:34:00.329968+0000","last_active":"2024-09-16T09:37:07.366361+0000","last_peered":"2024-09-16T09:37:07.366361+0000","last_clean":"2024-09-16T09:37:07.366361+0000","last_became_active":"2024-09-16T09:34:00.329642+0000","last_became_peered":"2024-09-16T09:34:00.329642+0000","last_unstale":"2024-09-16T09:37:07.366361+0000","last_undegraded":"2024-09-16T09:37:07.366361+0000","last_fullsized":"2024-09-16T09:37:07.366361+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:08.394830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:08.394830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:06.039660+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":461,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-16T09:37:08.372631+0000","last_change":"2024-09-16T09:34:48.786847+0000","last_active":"2024-09-16T09:37:08.372631+0000","last_peered":"2024-09-16T09:37:08.372631+0000","last_clean":"2024-09-16T09:37:08.372631+0000","last_became_active":"2024-09-16T09:31:09.276042+0000","last_became_peered":"2024-09-16T09:31:09.276042+0000","last_unstale":"2024-09-16T09:37:08.372631+0000","last_undegraded":"2024-09-16T09:37:08.372631+0000","last_fullsized":"2024-09-16T09:37:08.372631+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:48.786785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:34:48.786785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"285'15","reported_seq":310,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.371950+0000","last_change":"2024-09-16T09:36:55.936077+0000","last_active":"2024-09-16T09:37:07.371950+0000","last_peered":"2024-09-16T09:37:07.371950+0000","last_clean":"2024-09-16T09:37:07.371950+0000","last_became_active":"2024-09-16T09:34:00.331466+0000","last_became_peered":"2024-09-16T09:34:00.331466+0000","last_unstale":"2024-09-16T09:37:07.371950+0000","last_undegraded":"2024-09-16T09:37:07.371950+0000","last_fullsized":"2024-09-16T09:37:07.371950+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'15","last_scrub_stamp":"2024-09-16T09:36:55.935996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:36:55.935996+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:24.990467+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,0],"acting":[6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":608,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.377749+0000","last_change":"2024-09-16T09:32:47.502667+0000","last_active":"2024-09-16T09:37:07.377749+0000","last_peered":"2024-09-16T09:37:07.377749+0000","last_clean":"2024-09-16T09:37:07.377749+0000","last_became_active":"2024-09-16T09:25:01.706762+0000","last_became_peered":"2024-09-16T09:25:01.706762+0000","last_unstale":"2024-09-16T09:37:07.377749+0000","last_undegraded":"2024-09-16T09:37:07.377749+0000","last_fullsized":"2024-09-16T09:37:07.377749+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:47.502598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:32:47.502598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":446,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-16T09:37:08.374820+0000","last_change":"2024-09-16T09:34:46.742640+0000","last_active":"2024-09-16T09:37:08.374820+0000","last_peered":"2024-09-16T09:37:08.374820+0000","last_clean":"2024-09-16T09:37:08.374820+0000","last_became_active":"2024-09-16T09:28:52.982865+0000","last_became_peered":"2024-09-16T09:28:52.982865+0000","last_unstale":"2024-09-16T09:37:08.374820+0000","last_undegraded":"2024-09-16T09:37:08.374820+0000","last_fullsized":"2024-09-16T09:37:08.374820+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:46.742562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:34:46.742562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":607,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-16T09:37:08.372070+0000","last_change":"2024-09-16T09:34:50.779241+0000","last_active":"2024-09-16T09:37:08.372070+0000","last_peered":"2024-09-16T09:37:08.372070+0000","last_clean":"2024-09-16T09:37:08.372070+0000","last_became_active":"2024-09-16T09:25:07.715301+0000","last_became_peered":"2024-09-16T09:25:07.715301+0000","last_unstale":"2024-09-16T09:37:08.372070+0000","last_undegraded":"2024-09-16T09:37:08.372070+0000","last_fullsized":"2024-09-16T09:37:08.372070+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:50.779164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:34:50.779164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.14","version":"73'79","reported_seq":413,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.366935+0000","last_change":"2024-09-16T09:36:48.597220+0000","last_active":"2024-09-16T09:37:07.366935+0000","last_peered":"2024-09-16T09:37:07.366935+0000","last_clean":"2024-09-16T09:37:07.366935+0000","last_became_active":"2024-09-16T09:30:10.700866+0000","last_became_peered":"2024-09-16T09:30:10.700866+0000","last_unstale":"2024-09-16T09:37:07.366935+0000","last_undegraded":"2024-09-16T09:37:07.366935+0000","last_fullsized":"2024-09-16T09:37:07.366935+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:36:48.597169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:36:48.597169+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:15.020958+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"285'19","reported_seq":495,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.378357+0000","last_change":"2024-09-16T09:28:55.643505+0000","last_active":"2024-09-16T09:37:07.378357+0000","last_peered":"2024-09-16T09:37:07.378357+0000","last_clean":"2024-09-16T09:37:07.378357+0000","last_became_active":"2024-09-16T09:28:52.971132+0000","last_became_peered":"2024-09-16T09:28:52.971132+0000","last_unstale":"2024-09-16T09:37:07.378357+0000","last_undegraded":"2024-09-16T09:37:07.378357+0000","last_fullsized":"2024-09-16T09:37:07.378357+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-16T09:28:55.643448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:55.643448+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:01.504208+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":616,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.366842+0000","last_change":"2024-09-16T09:33:18.297533+0000","last_active":"2024-09-16T09:37:07.366842+0000","last_peered":"2024-09-16T09:37:07.366842+0000","last_clean":"2024-09-16T09:37:07.366842+0000","last_became_active":"2024-09-16T09:25:01.700733+0000","last_became_peered":"2024-09-16T09:25:01.700733+0000","last_unstale":"2024-09-16T09:37:07.366842+0000","last_undegraded":"2024-09-16T09:37:07.366842+0000","last_fullsized":"2024-09-16T09:37:07.366842+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:18.297451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:18.297451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":608,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.376542+0000","last_change":"2024-09-16T09:33:38.336281+0000","last_active":"2024-09-16T09:37:07.376542+0000","last_peered":"2024-09-16T09:37:07.376542+0000","last_clean":"2024-09-16T09:37:07.376542+0000","last_became_active":"2024-09-16T09:25:05.713479+0000","last_became_peered":"2024-09-16T09:25:05.713479+0000","last_unstale":"2024-09-16T09:37:07.376542+0000","last_undegraded":"2024-09-16T09:37:07.376542+0000","last_fullsized":"2024-09-16T09:37:07.376542+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:38.336221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:38.336221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":469,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-16T09:37:08.375300+0000","last_change":"2024-09-16T09:34:51.781047+0000","last_active":"2024-09-16T09:37:08.375300+0000","last_peered":"2024-09-16T09:37:08.375300+0000","last_clean":"2024-09-16T09:37:08.375300+0000","last_became_active":"2024-09-16T09:31:09.274797+0000","last_became_peered":"2024-09-16T09:31:09.274797+0000","last_unstale":"2024-09-16T09:37:08.375300+0000","last_undegraded":"2024-09-16T09:37:08.375300+0000","last_fullsized":"2024-09-16T09:37:08.375300+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:51.780980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:34:51.780980+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"285'8","reported_seq":362,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.368060+0000","last_change":"2024-09-16T09:34:50.206999+0000","last_active":"2024-09-16T09:37:07.368060+0000","last_peered":"2024-09-16T09:37:07.368060+0000","last_clean":"2024-09-16T09:37:07.368060+0000","last_became_active":"2024-09-16T09:31:09.273682+0000","last_became_peered":"2024-09-16T09:31:09.273682+0000","last_unstale":"2024-09-16T09:37:07.368060+0000","last_undegraded":"2024-09-16T09:37:07.368060+0000","last_fullsized":"2024-09-16T09:37:07.368060+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'4","last_scrub_stamp":"2024-09-16T09:34:50.206916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:34:50.206916+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":9,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":641,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-16T09:37:08.377353+0000","last_change":"2024-09-16T09:33:18.641115+0000","last_active":"2024-09-16T09:37:08.377353+0000","last_peered":"2024-09-16T09:37:08.377353+0000","last_clean":"2024-09-16T09:37:08.377353+0000","last_became_active":"2024-09-16T09:31:09.274719+0000","last_became_peered":"2024-09-16T09:31:09.274719+0000","last_unstale":"2024-09-16T09:37:08.377353+0000","last_undegraded":"2024-09-16T09:37:08.377353+0000","last_fullsized":"2024-09-16T09:37:08.377353+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:18.641069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:18.641069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":625,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.371689+0000","last_change":"2024-09-16T09:33:37.941485+0000","last_active":"2024-09-16T09:37:07.371689+0000","last_peered":"2024-09-16T09:37:07.371689+0000","last_clean":"2024-09-16T09:37:07.371689+0000","last_became_active":"2024-09-16T09:25:05.712545+0000","last_became_peered":"2024-09-16T09:25:05.712545+0000","last_unstale":"2024-09-16T09:37:07.371689+0000","last_undegraded":"2024-09-16T09:37:07.371689+0000","last_fullsized":"2024-09-16T09:37:07.371689+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:37.941403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:37.941403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":604,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.377204+0000","last_change":"2024-09-16T09:33:41.323824+0000","last_active":"2024-09-16T09:37:07.377204+0000","last_peered":"2024-09-16T09:37:07.377204+0000","last_clean":"2024-09-16T09:37:07.377204+0000","last_became_active":"2024-09-16T09:25:07.720117+0000","last_became_peered":"2024-09-16T09:25:07.720117+0000","last_unstale":"2024-09-16T09:37:07.377204+0000","last_undegraded":"2024-09-16T09:37:07.377204+0000","last_fullsized":"2024-09-16T09:37:07.377204+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:41.323773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:33:41.323773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":248,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.366671+0000","last_change":"2024-09-16T09:36:52.091335+0000","last_active":"2024-09-16T09:37:07.366671+0000","last_peered":"2024-09-16T09:37:07.366671+0000","last_clean":"2024-09-16T09:37:07.366671+0000","last_became_active":"2024-09-16T09:33:30.248036+0000","last_became_peered":"2024-09-16T09:33:30.248036+0000","last_unstale":"2024-09-16T09:37:07.366671+0000","last_undegraded":"2024-09-16T09:37:07.366671+0000","last_fullsized":"2024-09-16T09:37:07.366671+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:52.088284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:36:52.088284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:37:59.806406+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":468,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-16T09:37:08.374206+0000","last_change":"2024-09-16T09:32:40.411546+0000","last_active":"2024-09-16T09:37:08.374206+0000","last_peered":"2024-09-16T09:37:08.374206+0000","last_clean":"2024-09-16T09:37:08.374206+0000","last_became_active":"2024-09-16T09:31:09.271662+0000","last_became_peered":"2024-09-16T09:31:09.271662+0000","last_unstale":"2024-09-16T09:37:08.374206+0000","last_undegraded":"2024-09-16T09:37:08.374206+0000","last_fullsized":"2024-09-16T09:37:08.374206+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:40.411487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:32:40.411487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"285'19","reported_seq":644,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-16T09:37:07.372436+0000","last_change":"2024-09-16T09:33:19.843216+0000","last_active":"2024-09-16T09:37:07.372436+0000","last_peered":"2024-09-16T09:37:07.372436+0000","last_clean":"2024-09-16T09:37:07.372436+0000","last_became_active":"2024-09-16T09:25:03.707769+0000","last_became_peered":"2024-09-16T09:25:03.707769+0000","last_unstale":"2024-09-16T09:37:07.372436+0000","last_undegraded":"2024-09-16T09:37:07.372436+0000","last_fullsized":"2024-09-16T09:37:07.372436+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-16T09:33:19.843154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:33:19.843154+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":603,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-16T09:37:08.372553+0000","last_change":"2024-09-16T09:32:31.325543+0000","last_active":"2024-09-16T09:37:08.372553+0000","last_peered":"2024-09-16T09:37:08.372553+0000","last_clean":"2024-09-16T09:37:08.372553+0000","last_became_active":"2024-09-16T09:25:05.711952+0000","last_became_peered":"2024-09-16T09:25:05.711952+0000","last_unstale":"2024-09-16T09:37:08.372553+0000","last_undegraded":"2024-09-16T09:37:08.372553+0000","last_fullsized":"2024-09-16T09:37:08.372553+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:31.325346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:32:31.325346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":17,"stat_sum":{"num_bytes":1343422488,"num_objects":20500,"num_object_clones":0,"num_object_copies":61500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20500,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":148614,"num_write_kb":3816897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23088,"num_bytes_recovered":1478230016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4978388992,"data_stored":4978376776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":47764,"ondisk_log_size":47764,"up":51,"acting":51,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":1117,"num_read_kb":874,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":120,"num_bytes_recovered":3026,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":681,"ondisk_log_size":681,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1986,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":720,"num_read_kb":715,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":21,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":327680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1659,"ondisk_log_size":1659,"up":63,"acting":63,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266838,"num_pgs":74,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1326616,"kb_used_data":813632,"kb_used_omap":1,"kb_used_meta":512894,"kb_avail":92418536,"statfs":{"total":95995035648,"available":94636580864,"internally_reserved":0,"allocated":833159168,"data_stored":832694638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":525203914},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495771,"num_pgs":61,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":851676,"kb_used_data":495620,"kb_used_omap":1,"kb_used_meta":356030,"kb_avail":92893476,"statfs":{"total":95995035648,"available":95122919424,"internally_reserved":0,"allocated":507514880,"data_stored":507094094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":364575174},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724706,"num_pgs":65,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1528192,"kb_used_data":951208,"kb_used_omap":1,"kb_used_meta":576766,"kb_avail":92216960,"statfs":{"total":95995035648,"available":94430167040,"internally_reserved":0,"allocated":974036992,"data_stored":973596189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":590608838},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,5,1,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953639,"num_pgs":70,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":974920,"kb_used_data":600820,"kb_used_omap":1,"kb_used_meta":374078,"kb_avail":92770232,"statfs":{"total":95995035648,"available":94996717568,"internally_reserved":0,"allocated":615239680,"data_stored":614787062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":383056326},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149869,"num_pgs":74,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1213536,"kb_used_data":622788,"kb_used_omap":1,"kb_used_meta":590654,"kb_avail":92531616,"statfs":{"total":95995035648,"available":94752374784,"internally_reserved":0,"allocated":637734912,"data_stored":637266936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":604830151},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411506,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":693092,"kb_used_data":274572,"kb_used_omap":1,"kb_used_meta":418494,"kb_avail":93052060,"statfs":{"total":95995035648,"available":95285309440,"internally_reserved":0,"allocated":281161728,"data_stored":280741136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":428538309},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575033,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1480144,"kb_used_data":849084,"kb_used_omap":1,"kb_used_meta":630910,"kb_avail":92265008,"statfs":{"total":95995035648,"available":94479368192,"internally_reserved":0,"allocated":869462016,"data_stored":869000722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":646052293},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738559,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":892704,"kb_used_data":410308,"kb_used_omap":1,"kb_used_meta":482366,"kb_avail":92852448,"statfs":{"total":95995035648,"available":95080906752,"internally_reserved":0,"allocated":420155392,"data_stored":419695344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":493943237},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":416612352,"data_stored":416612352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":861667328,"data_stored":861667328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":262537216,"data_stored":262537216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":596180992,"data_stored":596180992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":600379392,"data_stored":600375320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":956502016,"data_stored":956497944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":490995712,"data_stored":490995712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":793513984,"data_stored":793509912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T09:37:13.640 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2024-09-16T09:37:13.640 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd pool set unique_pool_2 pg_num 26 2024-09-16T09:37:13.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:13 smithi084 ceph-mon[32629]: from='client.15840 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:37:13.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:13 smithi084 ceph-mon[32629]: pgmap v631: 169 pgs: 169 active+clean; 1.3 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 1.2 KiB/s rd, 7.6 MiB/s wr, 1.11k op/s; 55 MiB/s, 968 objects/s recovering 2024-09-16T09:37:13.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:13 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "16"}]': finished 2024-09-16T09:37:13.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:13 smithi084 ceph-mon[32629]: osdmap e352: 8 total, 8 up, 8 in 2024-09-16T09:37:13.689 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:37:13.738 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:13.739 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:13.739 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:37:13.840 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:37:13.843 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:37:13.844 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:37:13.863 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:37:13.864 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:37:13.867 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:13.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:13.868 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:37:13.880 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:37:13.887 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:37:13.888 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:37:13.906 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:37:13.906 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:37:13.924 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:37:13.924 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:37:13.942 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:37:13.942 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:37:13.959 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:37:13.960 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:37:13.968 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:37:13.976 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:37:13.976 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:37:13.994 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:13.994 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:13.995 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:37:14.095 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:37:14.121 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:14.121 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:14.122 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:37:14.223 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:37:14.248 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:14.249 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:14.249 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:37:14.350 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:37:14.375 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:14.375 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:14.376 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:37:14.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:14 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:37:14.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:14 smithi070 ceph-mon[30388]: osdmap e353: 8 total, 8 up, 8 in 2024-09-16T09:37:14.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:14 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3934966519' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:37:14.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:14 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:37:14.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:14 smithi070 ceph-mon[40691]: osdmap e353: 8 total, 8 up, 8 in 2024-09-16T09:37:14.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:14 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3934966519' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:37:14.477 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:37:14.500 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:14.501 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:14.501 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:37:14.602 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:37:14.624 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:14.625 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:14.625 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:37:14.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:14 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:37:14.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:14 smithi084 ceph-mon[32629]: osdmap e353: 8 total, 8 up, 8 in 2024-09-16T09:37:14.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:14 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3934966519' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:37:14.726 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:37:14.748 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:14.748 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:14.749 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:37:14.850 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:37:14.872 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:14.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:14.873 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:37:14.973 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:37:14.995 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:14.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:14.996 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:37:15.097 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:37:15.120 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:15.121 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:15.121 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:37:15.205 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:37:15.221 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:37:15.245 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:15.246 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:15.246 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:37:15.347 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:37:15.372 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:15.372 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:15.373 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:37:15.466 INFO:teuthology.orchestra.run.smithi070.stderr:noscrub is unset 2024-09-16T09:37:15.474 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:37:15.496 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:15.497 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:15.498 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:37:15.599 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:37:15.620 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:15.621 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:15.622 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:37:15.723 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:37:15.755 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:15.756 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:15.756 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:37:15.857 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:37:15.868 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:15 smithi084 ceph-mon[32629]: pgmap v634: 169 pgs: 2 peering, 167 active+clean; 1.1 GiB data, 7.8 GiB used, 707 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 649 op/s 2024-09-16T09:37:15.868 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:15 smithi084 ceph-mon[32629]: osdmap e354: 8 total, 8 up, 8 in 2024-09-16T09:37:15.868 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:15 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/4192158720' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:37:15.895 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:15.896 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:15.896 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:37:15.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:15 smithi070 ceph-mon[30388]: pgmap v634: 169 pgs: 2 peering, 167 active+clean; 1.1 GiB data, 7.8 GiB used, 707 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 649 op/s 2024-09-16T09:37:15.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:15 smithi070 ceph-mon[30388]: osdmap e354: 8 total, 8 up, 8 in 2024-09-16T09:37:15.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:15 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/4192158720' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:37:15.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:15 smithi070 ceph-mon[40691]: pgmap v634: 169 pgs: 2 peering, 167 active+clean; 1.1 GiB data, 7.8 GiB used, 707 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 649 op/s 2024-09-16T09:37:15.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:15 smithi070 ceph-mon[40691]: osdmap e354: 8 total, 8 up, 8 in 2024-09-16T09:37:15.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:15 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/4192158720' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:37:15.997 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:37:16.035 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:16.035 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:16.036 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:37:16.137 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:37:16.173 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:16.174 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:16.174 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:37:16.275 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:37:16.290 INFO:teuthology.orchestra.run.smithi084.stdout:ERROR: (22) Invalid argument 2024-09-16T09:37:16.290 INFO:teuthology.orchestra.run.smithi084.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:37:16.303 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:16.304 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:16.304 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:37:16.406 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:37:16.435 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:16.436 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:16.436 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:37:16.484 INFO:teuthology.orchestra.run.smithi070.stderr:set pool 9 pg_num to 26 2024-09-16T09:37:16.538 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:37:16.567 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:16.567 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:16.568 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:37:16.669 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:37:16.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:16 smithi070 ceph-mon[30388]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:37:16.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:16 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/4192158720' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-16T09:37:16.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:16 smithi070 ceph-mon[30388]: osdmap e355: 8 total, 8 up, 8 in 2024-09-16T09:37:16.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:16 smithi070 ceph-mon[30388]: 1.11 scrub starts 2024-09-16T09:37:16.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:16 smithi070 ceph-mon[30388]: 1.11 scrub ok 2024-09-16T09:37:16.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:16 smithi070 ceph-mon[30388]: 4.10 scrub starts 2024-09-16T09:37:16.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:16 smithi070 ceph-mon[30388]: 4.10 scrub ok 2024-09-16T09:37:16.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:16 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/199781195' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num", "val": "26"} : dispatch 2024-09-16T09:37:16.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:16 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num", "val": "26"} : dispatch 2024-09-16T09:37:16.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:16 smithi070 ceph-mon[40691]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:37:16.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:16 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/4192158720' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-16T09:37:16.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:16 smithi070 ceph-mon[40691]: osdmap e355: 8 total, 8 up, 8 in 2024-09-16T09:37:16.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:16 smithi070 ceph-mon[40691]: 1.11 scrub starts 2024-09-16T09:37:16.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:16 smithi070 ceph-mon[40691]: 1.11 scrub ok 2024-09-16T09:37:16.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:16 smithi070 ceph-mon[40691]: 4.10 scrub starts 2024-09-16T09:37:16.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:16 smithi070 ceph-mon[40691]: 4.10 scrub ok 2024-09-16T09:37:16.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:16 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/199781195' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num", "val": "26"} : dispatch 2024-09-16T09:37:16.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:16 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num", "val": "26"} : dispatch 2024-09-16T09:37:16.704 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:16.704 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:16.705 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:37:16.808 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:37:16.836 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:16.837 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:16.837 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:37:16.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:16 smithi084 ceph-mon[32629]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:37:16.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:16 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/4192158720' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-16T09:37:16.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:16 smithi084 ceph-mon[32629]: osdmap e355: 8 total, 8 up, 8 in 2024-09-16T09:37:16.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:16 smithi084 ceph-mon[32629]: 1.11 scrub starts 2024-09-16T09:37:16.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:16 smithi084 ceph-mon[32629]: 1.11 scrub ok 2024-09-16T09:37:16.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:16 smithi084 ceph-mon[32629]: 4.10 scrub starts 2024-09-16T09:37:16.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:16 smithi084 ceph-mon[32629]: 4.10 scrub ok 2024-09-16T09:37:16.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:16 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/199781195' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num", "val": "26"} : dispatch 2024-09-16T09:37:16.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:16 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num", "val": "26"} : dispatch 2024-09-16T09:37:16.938 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:37:16.973 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:16.974 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:16.974 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:37:17.075 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:37:17.104 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:17.105 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:17.105 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:37:17.138 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T09:37:17.182 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.4 dump_blocked_ops 2024-09-16T09:37:17.206 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:37:17.239 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:17.240 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:17.240 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:37:17.341 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:37:17.388 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:17.389 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:17.389 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:37:17.490 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:37:17.524 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:37:17.532 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:17.532 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:17.533 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:37:17.631 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:37:17.633 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:37:17.666 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:17.667 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:17.667 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:37:17.768 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:37:17.771 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:17 smithi084 ceph-mon[32629]: pgmap v637: 168 pgs: 1 peering, 167 active+clean; 1.1 GiB data, 7.8 GiB used, 707 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.26k op/s 2024-09-16T09:37:17.771 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:17 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num", "val": "26"}]': finished 2024-09-16T09:37:17.771 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:17 smithi084 ceph-mon[32629]: osdmap e356: 8 total, 8 up, 8 in 2024-09-16T09:37:17.772 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:17 smithi084 ceph-mon[32629]: 5.d scrub starts 2024-09-16T09:37:17.772 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:17 smithi084 ceph-mon[32629]: 5.d scrub ok 2024-09-16T09:37:17.772 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:17 smithi084 ceph-mon[32629]: 3.8 scrub starts 2024-09-16T09:37:17.772 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:17 smithi084 ceph-mon[32629]: 3.8 scrub ok 2024-09-16T09:37:17.806 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:17.807 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:17.808 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:37:17.835 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:17 smithi070 ceph-mon[30388]: pgmap v637: 168 pgs: 1 peering, 167 active+clean; 1.1 GiB data, 7.8 GiB used, 707 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.26k op/s 2024-09-16T09:37:17.835 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:17 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num", "val": "26"}]': finished 2024-09-16T09:37:17.836 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:17 smithi070 ceph-mon[30388]: osdmap e356: 8 total, 8 up, 8 in 2024-09-16T09:37:17.836 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:17 smithi070 ceph-mon[30388]: 5.d scrub starts 2024-09-16T09:37:17.836 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:17 smithi070 ceph-mon[30388]: 5.d scrub ok 2024-09-16T09:37:17.836 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:17 smithi070 ceph-mon[30388]: 3.8 scrub starts 2024-09-16T09:37:17.836 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:17 smithi070 ceph-mon[30388]: 3.8 scrub ok 2024-09-16T09:37:17.836 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:17 smithi070 ceph-mon[40691]: pgmap v637: 168 pgs: 1 peering, 167 active+clean; 1.1 GiB data, 7.8 GiB used, 707 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.26k op/s 2024-09-16T09:37:17.836 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:17 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num", "val": "26"}]': finished 2024-09-16T09:37:17.836 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:17 smithi070 ceph-mon[40691]: osdmap e356: 8 total, 8 up, 8 in 2024-09-16T09:37:17.836 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:17 smithi070 ceph-mon[40691]: 5.d scrub starts 2024-09-16T09:37:17.836 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:17 smithi070 ceph-mon[40691]: 5.d scrub ok 2024-09-16T09:37:17.836 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:17 smithi070 ceph-mon[40691]: 3.8 scrub starts 2024-09-16T09:37:17.837 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:17 smithi070 ceph-mon[40691]: 3.8 scrub ok 2024-09-16T09:37:17.908 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:37:17.939 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:17.940 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:17.940 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:37:18.041 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:37:18.076 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:18.076 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:18.077 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:37:18.178 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:37:18.193 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:37:18.194 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:37:18.211 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:18.211 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:18.212 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:37:18.222 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:37:18.223 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:37:18.253 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:37:18.253 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:37:18.281 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:37:18.281 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:37:18.312 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:37:18.312 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:37:18.312 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:37:18.316 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-09-16T09:37:18.316 INFO:teuthology.orchestra.run.smithi084.stdout: "ops": [], 2024-09-16T09:37:18.316 INFO:teuthology.orchestra.run.smithi084.stdout: "complaint_time": 30, 2024-09-16T09:37:18.316 INFO:teuthology.orchestra.run.smithi084.stdout: "num_blocked_ops": 0 2024-09-16T09:37:18.316 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-09-16T09:37:18.350 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:18.351 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:18.351 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:37:18.352 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:37:18.352 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:37:18.381 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-16T09:37:18.390 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:37:18.390 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:37:18.433 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:37:18.433 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:37:18.454 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:37:18.483 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:18.484 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:18.485 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:37:18.585 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:37:18.673 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:18 smithi070 ceph-mon[30388]: 3.f scrub starts 2024-09-16T09:37:18.673 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:18 smithi070 ceph-mon[30388]: 3.f scrub ok 2024-09-16T09:37:18.673 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:18 smithi070 ceph-mon[30388]: 1.4 scrub starts 2024-09-16T09:37:18.673 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:18 smithi070 ceph-mon[30388]: 1.4 scrub ok 2024-09-16T09:37:18.673 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:18 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "26"} : dispatch 2024-09-16T09:37:18.673 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:18 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "26"} : dispatch 2024-09-16T09:37:18.673 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:18 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1083334387' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:37:18.673 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:18 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:37:18.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:18 smithi070 ceph-mon[40691]: 3.f scrub starts 2024-09-16T09:37:18.674 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:18 smithi070 ceph-mon[40691]: 3.f scrub ok 2024-09-16T09:37:18.674 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:18 smithi070 ceph-mon[40691]: 1.4 scrub starts 2024-09-16T09:37:18.674 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:18 smithi070 ceph-mon[40691]: 1.4 scrub ok 2024-09-16T09:37:18.674 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:18 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "26"} : dispatch 2024-09-16T09:37:18.674 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:18 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "26"} : dispatch 2024-09-16T09:37:18.674 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:18 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1083334387' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:37:18.674 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:18 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:37:18.675 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:18.675 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:18.676 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:37:18.777 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:37:18.818 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:18.818 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:18.819 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:37:18.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:18 smithi084 ceph-mon[32629]: 3.f scrub starts 2024-09-16T09:37:18.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:18 smithi084 ceph-mon[32629]: 3.f scrub ok 2024-09-16T09:37:18.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:18 smithi084 ceph-mon[32629]: 1.4 scrub starts 2024-09-16T09:37:18.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:18 smithi084 ceph-mon[32629]: 1.4 scrub ok 2024-09-16T09:37:18.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:18 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "26"} : dispatch 2024-09-16T09:37:18.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:18 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "26"} : dispatch 2024-09-16T09:37:18.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:18 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1083334387' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:37:18.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:18 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:37:18.919 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:37:18.940 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:18.941 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:18.941 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:37:19.042 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:37:19.065 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:19.065 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:19.066 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:37:19.166 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:37:19.188 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:19.189 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:19.189 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:37:19.290 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:37:19.311 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:19.312 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:19.313 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:37:19.413 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:37:19.435 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:19.436 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:19.436 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:37:19.537 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:37:19.562 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:19.563 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:19.563 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:37:19.664 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:37:19.698 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:19.699 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:19.700 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:37:19.800 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:37:19.825 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:19.826 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:19.826 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:37:19.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:19 smithi084 ceph-mon[32629]: pgmap v639: 168 pgs: 168 active+clean; 1.1 GiB data, 7.7 GiB used, 708 GiB / 715 GiB avail; 0 B/s wr, 268 op/s 2024-09-16T09:37:19.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:19 smithi084 ceph-mon[32629]: 3.9 scrub starts 2024-09-16T09:37:19.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:19 smithi084 ceph-mon[32629]: 3.9 scrub ok 2024-09-16T09:37:19.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:19 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "26"}]': finished 2024-09-16T09:37:19.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:19 smithi084 ceph-mon[32629]: osdmap e357: 8 total, 8 up, 8 in 2024-09-16T09:37:19.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:19 smithi084 ceph-mon[32629]: 3.18 scrub starts 2024-09-16T09:37:19.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:19 smithi084 ceph-mon[32629]: 3.18 scrub ok 2024-09-16T09:37:19.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:19 smithi070 ceph-mon[30388]: pgmap v639: 168 pgs: 168 active+clean; 1.1 GiB data, 7.7 GiB used, 708 GiB / 715 GiB avail; 0 B/s wr, 268 op/s 2024-09-16T09:37:19.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:19 smithi070 ceph-mon[30388]: 3.9 scrub starts 2024-09-16T09:37:19.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:19 smithi070 ceph-mon[30388]: 3.9 scrub ok 2024-09-16T09:37:19.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:19 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "26"}]': finished 2024-09-16T09:37:19.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:19 smithi070 ceph-mon[30388]: osdmap e357: 8 total, 8 up, 8 in 2024-09-16T09:37:19.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:19 smithi070 ceph-mon[30388]: 3.18 scrub starts 2024-09-16T09:37:19.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:19 smithi070 ceph-mon[30388]: 3.18 scrub ok 2024-09-16T09:37:19.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:19 smithi070 ceph-mon[40691]: pgmap v639: 168 pgs: 168 active+clean; 1.1 GiB data, 7.7 GiB used, 708 GiB / 715 GiB avail; 0 B/s wr, 268 op/s 2024-09-16T09:37:19.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:19 smithi070 ceph-mon[40691]: 3.9 scrub starts 2024-09-16T09:37:19.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:19 smithi070 ceph-mon[40691]: 3.9 scrub ok 2024-09-16T09:37:19.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:19 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pg_num_actual", "val": "26"}]': finished 2024-09-16T09:37:19.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:19 smithi070 ceph-mon[40691]: osdmap e357: 8 total, 8 up, 8 in 2024-09-16T09:37:19.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:19 smithi070 ceph-mon[40691]: 3.18 scrub starts 2024-09-16T09:37:19.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:19 smithi070 ceph-mon[40691]: 3.18 scrub ok 2024-09-16T09:37:19.927 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:37:19.957 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:19.958 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:19.958 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:37:20.059 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:37:20.094 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:20.095 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:20.095 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:37:20.149 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:37:20.196 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:37:20.229 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:20.229 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:20.230 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:37:20.330 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:37:20.365 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:20.365 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:20.366 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:37:20.466 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:37:20.508 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:20.508 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:20.509 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:37:20.526 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:20 smithi070 ceph-mon[40691]: 5.0 scrub starts 2024-09-16T09:37:20.526 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:20 smithi070 ceph-mon[40691]: 5.0 scrub ok 2024-09-16T09:37:20.527 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:20 smithi070 ceph-mon[40691]: osdmap e358: 8 total, 8 up, 8 in 2024-09-16T09:37:20.527 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:20 smithi070 ceph-mon[30388]: 5.0 scrub starts 2024-09-16T09:37:20.527 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:20 smithi070 ceph-mon[30388]: 5.0 scrub ok 2024-09-16T09:37:20.527 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:20 smithi070 ceph-mon[30388]: osdmap e358: 8 total, 8 up, 8 in 2024-09-16T09:37:20.527 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:20 smithi070 ceph-mon[30388]: 5.14 scrub starts 2024-09-16T09:37:20.527 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:20 smithi070 ceph-mon[30388]: 5.14 scrub ok 2024-09-16T09:37:20.527 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:20 smithi070 ceph-mon[30388]: 9.9 scrub starts 2024-09-16T09:37:20.527 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:20 smithi070 ceph-mon[30388]: 9.2 scrub starts 2024-09-16T09:37:20.527 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:20 smithi070 ceph-mon[30388]: 9.9 scrub ok 2024-09-16T09:37:20.527 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:20 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num_actual", "val": "26"} : dispatch 2024-09-16T09:37:20.527 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:20 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num_actual", "val": "26"} : dispatch 2024-09-16T09:37:20.527 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:20 smithi070 ceph-mon[30388]: 9.2 scrub ok 2024-09-16T09:37:20.528 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:20 smithi070 ceph-mon[30388]: 9.1 scrub starts 2024-09-16T09:37:20.528 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:20 smithi070 ceph-mon[30388]: 9.7 scrub starts 2024-09-16T09:37:20.528 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:20 smithi070 ceph-mon[30388]: 9.7 scrub ok 2024-09-16T09:37:20.528 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:20 smithi070 ceph-mon[30388]: 9.1 scrub ok 2024-09-16T09:37:20.528 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:20 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:37:20.609 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:37:20.631 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:20.631 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:20.632 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:37:20.732 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:37:20.754 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:20.755 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:20.756 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:37:20.778 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:20 smithi070 ceph-mon[40691]: 5.14 scrub starts 2024-09-16T09:37:20.778 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:20 smithi070 ceph-mon[40691]: 5.14 scrub ok 2024-09-16T09:37:20.778 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:20 smithi070 ceph-mon[40691]: 9.9 scrub starts 2024-09-16T09:37:20.778 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:20 smithi070 ceph-mon[40691]: 9.2 scrub starts 2024-09-16T09:37:20.779 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:20 smithi070 ceph-mon[40691]: 9.9 scrub ok 2024-09-16T09:37:20.779 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:20 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num_actual", "val": "26"} : dispatch 2024-09-16T09:37:20.779 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:20 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num_actual", "val": "26"} : dispatch 2024-09-16T09:37:20.779 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:20 smithi070 ceph-mon[40691]: 9.2 scrub ok 2024-09-16T09:37:20.779 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:20 smithi070 ceph-mon[40691]: 9.1 scrub starts 2024-09-16T09:37:20.779 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:20 smithi070 ceph-mon[40691]: 9.7 scrub starts 2024-09-16T09:37:20.779 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:20 smithi070 ceph-mon[40691]: 9.7 scrub ok 2024-09-16T09:37:20.779 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:20 smithi070 ceph-mon[40691]: 9.1 scrub ok 2024-09-16T09:37:20.779 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:20 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:37:20.856 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:37:20.880 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:20.881 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:20.881 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:37:20.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:20 smithi084 ceph-mon[32629]: 5.0 scrub starts 2024-09-16T09:37:20.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:20 smithi084 ceph-mon[32629]: 5.0 scrub ok 2024-09-16T09:37:20.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:20 smithi084 ceph-mon[32629]: osdmap e358: 8 total, 8 up, 8 in 2024-09-16T09:37:20.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:20 smithi084 ceph-mon[32629]: 5.14 scrub starts 2024-09-16T09:37:20.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:20 smithi084 ceph-mon[32629]: 5.14 scrub ok 2024-09-16T09:37:20.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:20 smithi084 ceph-mon[32629]: 9.9 scrub starts 2024-09-16T09:37:20.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:20 smithi084 ceph-mon[32629]: 9.2 scrub starts 2024-09-16T09:37:20.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:20 smithi084 ceph-mon[32629]: 9.9 scrub ok 2024-09-16T09:37:20.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:20 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num_actual", "val": "26"} : dispatch 2024-09-16T09:37:20.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:20 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num_actual", "val": "26"} : dispatch 2024-09-16T09:37:20.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:20 smithi084 ceph-mon[32629]: 9.2 scrub ok 2024-09-16T09:37:20.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:20 smithi084 ceph-mon[32629]: 9.1 scrub starts 2024-09-16T09:37:20.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:20 smithi084 ceph-mon[32629]: 9.7 scrub starts 2024-09-16T09:37:20.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:20 smithi084 ceph-mon[32629]: 9.7 scrub ok 2024-09-16T09:37:20.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:20 smithi084 ceph-mon[32629]: 9.1 scrub ok 2024-09-16T09:37:20.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:20 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:37:20.982 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:37:21.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:21.005 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:21.005 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:37:21.106 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:37:21.114 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.4 dump_historic_ops 2024-09-16T09:37:21.128 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:21.128 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:21.128 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:37:21.230 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:37:21.252 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:21.253 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:21.253 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:37:21.354 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:37:21.376 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:21.377 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:21.378 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:37:21.378 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:37:21.479 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:37:21.500 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:21.501 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:21.501 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:37:21.518 INFO:teuthology.orchestra.run.smithi070.stderr:nodeep-scrub is unset 2024-09-16T09:37:21.603 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:37:21.624 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:21.624 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:21.625 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:37:21.712 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:21 smithi084 ceph-mon[32629]: pgmap v642: 178 pgs: 3 peering, 10 unknown, 165 active+clean; 800 MiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.30k op/s 2024-09-16T09:37:21.713 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:21 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:37:21.713 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:21 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:37:21.713 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:21 smithi084 ceph-mon[32629]: 9.4 scrub starts 2024-09-16T09:37:21.713 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:21 smithi084 ceph-mon[32629]: 9.4 scrub ok 2024-09-16T09:37:21.713 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:21 smithi084 ceph-mon[32629]: 9.3 scrub starts 2024-09-16T09:37:21.713 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:21 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num_actual", "val": "26"}]': finished 2024-09-16T09:37:21.713 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:21 smithi084 ceph-mon[32629]: osdmap e359: 8 total, 8 up, 8 in 2024-09-16T09:37:21.713 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:21 smithi084 ceph-mon[32629]: 9.3 scrub ok 2024-09-16T09:37:21.713 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:21 smithi084 ceph-mon[32629]: 9.8 scrub starts 2024-09-16T09:37:21.713 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:21 smithi084 ceph-mon[32629]: 9.8 scrub ok 2024-09-16T09:37:21.713 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:21 smithi084 ceph-mon[32629]: 4.17 scrub starts 2024-09-16T09:37:21.713 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:21 smithi084 ceph-mon[32629]: 4.17 scrub ok 2024-09-16T09:37:21.713 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:21 smithi084 ceph-mon[32629]: 9.0 scrub starts 2024-09-16T09:37:21.714 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:21 smithi084 ceph-mon[32629]: 9.0 scrub ok 2024-09-16T09:37:21.714 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:21 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/2545459435' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:37:21.714 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:21 smithi084 ceph-mon[32629]: 9.6 scrub starts 2024-09-16T09:37:21.714 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:21 smithi084 ceph-mon[32629]: 9.6 scrub ok 2024-09-16T09:37:21.714 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:21 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:37:21.714 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:21 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:37:21.714 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:21 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:37:21.725 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:37:21.747 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:21.748 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:21.748 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:37:21.849 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:37:21.850 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:21 smithi070 ceph-mon[40691]: pgmap v642: 178 pgs: 3 peering, 10 unknown, 165 active+clean; 800 MiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.30k op/s 2024-09-16T09:37:21.850 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:21 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:37:21.850 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:21 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:37:21.850 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:21 smithi070 ceph-mon[40691]: 9.4 scrub starts 2024-09-16T09:37:21.850 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:21 smithi070 ceph-mon[40691]: 9.4 scrub ok 2024-09-16T09:37:21.850 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:21 smithi070 ceph-mon[40691]: 9.3 scrub starts 2024-09-16T09:37:21.850 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:21 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num_actual", "val": "26"}]': finished 2024-09-16T09:37:21.850 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:21 smithi070 ceph-mon[40691]: osdmap e359: 8 total, 8 up, 8 in 2024-09-16T09:37:21.851 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:21 smithi070 ceph-mon[40691]: 9.3 scrub ok 2024-09-16T09:37:21.851 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:21 smithi070 ceph-mon[40691]: 9.8 scrub starts 2024-09-16T09:37:21.851 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:21 smithi070 ceph-mon[40691]: 9.8 scrub ok 2024-09-16T09:37:21.851 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:21 smithi070 ceph-mon[40691]: 4.17 scrub starts 2024-09-16T09:37:21.851 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:21 smithi070 ceph-mon[40691]: 4.17 scrub ok 2024-09-16T09:37:21.851 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:21 smithi070 ceph-mon[40691]: 9.0 scrub starts 2024-09-16T09:37:21.851 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:21 smithi070 ceph-mon[40691]: 9.0 scrub ok 2024-09-16T09:37:21.851 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:21 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/2545459435' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:37:21.851 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:21 smithi070 ceph-mon[40691]: 9.6 scrub starts 2024-09-16T09:37:21.851 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:21 smithi070 ceph-mon[40691]: 9.6 scrub ok 2024-09-16T09:37:21.851 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:21 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:37:21.851 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:21 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:37:21.851 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:21 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:37:21.852 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:21 smithi070 ceph-mon[30388]: pgmap v642: 178 pgs: 3 peering, 10 unknown, 165 active+clean; 800 MiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.30k op/s 2024-09-16T09:37:21.852 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:21 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:37:21.852 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:21 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:37:21.852 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:21 smithi070 ceph-mon[30388]: 9.4 scrub starts 2024-09-16T09:37:21.852 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:21 smithi070 ceph-mon[30388]: 9.4 scrub ok 2024-09-16T09:37:21.852 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:21 smithi070 ceph-mon[30388]: 9.3 scrub starts 2024-09-16T09:37:21.852 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:21 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_2", "var": "pgp_num_actual", "val": "26"}]': finished 2024-09-16T09:37:21.852 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:21 smithi070 ceph-mon[30388]: osdmap e359: 8 total, 8 up, 8 in 2024-09-16T09:37:21.852 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:21 smithi070 ceph-mon[30388]: 9.3 scrub ok 2024-09-16T09:37:21.853 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:21 smithi070 ceph-mon[30388]: 9.8 scrub starts 2024-09-16T09:37:21.853 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:21 smithi070 ceph-mon[30388]: 9.8 scrub ok 2024-09-16T09:37:21.853 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:21 smithi070 ceph-mon[30388]: 4.17 scrub starts 2024-09-16T09:37:21.853 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:21 smithi070 ceph-mon[30388]: 4.17 scrub ok 2024-09-16T09:37:21.853 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:21 smithi070 ceph-mon[30388]: 9.0 scrub starts 2024-09-16T09:37:21.853 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:21 smithi070 ceph-mon[30388]: 9.0 scrub ok 2024-09-16T09:37:21.853 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:21 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2545459435' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:37:21.853 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:21 smithi070 ceph-mon[30388]: 9.6 scrub starts 2024-09-16T09:37:21.853 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:21 smithi070 ceph-mon[30388]: 9.6 scrub ok 2024-09-16T09:37:21.853 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:21 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:37:21.853 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:21 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:37:21.853 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:21 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:37:21.874 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:21.874 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:21.874 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:37:21.975 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:37:21.996 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:21.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:21.997 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:37:22.010 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T09:37:22.098 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:37:22.119 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:22.120 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:22.120 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:37:22.134 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-09-16T09:37:22.134 INFO:teuthology.orchestra.run.smithi084.stdout: "size": 20, 2024-09-16T09:37:22.134 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 600, 2024-09-16T09:37:22.134 INFO:teuthology.orchestra.run.smithi084.stdout: "ops": [ 2024-09-16T09:37:22.135 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.135 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.25064.0:170549 8.b 8:d1a36449:::benchmark_data_smithi070_107919_object45954:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e204)", 2024-09-16T09:37:22.135 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:32:07.438095+0000", 2024-09-16T09:37:22.135 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 314.69318649399997, 2024-09-16T09:37:22.135 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.004960772, 2024-09-16T09:37:22.135 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:37:22.135 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:37:22.135 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:37:22.135 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:37:22.135 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.25064", 2024-09-16T09:37:22.135 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/2578256439", 2024-09-16T09:37:22.136 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 170549 2024-09-16T09:37:22.136 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.136 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:37:22.136 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.136 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:37:22.136 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.438095+0000", 2024-09-16T09:37:22.136 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:37:22.136 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.136 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.136 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:37:22.136 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.438095+0000", 2024-09-16T09:37:22.137 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:37:22.137 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.137 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.137 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:37:22.137 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.438096+0000", 2024-09-16T09:37:22.137 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.7370000000000001e-06 2024-09-16T09:37:22.138 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.138 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.138 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:37:22.138 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.438103+0000", 2024-09-16T09:37:22.138 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.3670000000000002e-06 2024-09-16T09:37:22.138 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.138 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.138 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:37:22.138 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.438119+0000", 2024-09-16T09:37:22.139 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.6084999999999999e-05 2024-09-16T09:37:22.139 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.139 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.139 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:37:22.139 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.438126+0000", 2024-09-16T09:37:22.139 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.7349999999999999e-06 2024-09-16T09:37:22.139 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.139 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.139 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:37:22.139 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.438179+0000", 2024-09-16T09:37:22.139 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.2961000000000003e-05 2024-09-16T09:37:22.139 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.139 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.139 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:37:22.139 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.438187+0000", 2024-09-16T09:37:22.140 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.1890000000000007e-06 2024-09-16T09:37:22.140 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.140 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.140 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:37:22.140 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.436726+0000", 2024-09-16T09:37:22.140 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.99853915599999998 2024-09-16T09:37:22.140 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.140 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.140 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:37:22.140 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.436844+0000", 2024-09-16T09:37:22.140 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000118089 2024-09-16T09:37:22.140 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.140 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.141 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 1,3", 2024-09-16T09:37:22.141 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.436976+0000", 2024-09-16T09:37:22.141 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00013196700000000001 2024-09-16T09:37:22.141 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.141 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.141 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:37:22.141 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.442912+0000", 2024-09-16T09:37:22.141 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0059363970000000004 2024-09-16T09:37:22.141 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.141 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.141 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:22.141 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.442954+0000", 2024-09-16T09:37:22.141 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.1761000000000002e-05 2024-09-16T09:37:22.141 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.142 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.142 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:22.142 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.442990+0000", 2024-09-16T09:37:22.142 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.6091999999999999e-05 2024-09-16T09:37:22.142 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.142 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.142 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:37:22.142 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.443029+0000", 2024-09-16T09:37:22.142 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.9186000000000001e-05 2024-09-16T09:37:22.142 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.142 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.142 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:37:22.142 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.443055+0000", 2024-09-16T09:37:22.142 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.605e-05 2024-09-16T09:37:22.143 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:37:22.143 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:37:22.143 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:37:22.143 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.143 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.143 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.25064.0:170577 8.b 8:d5966a91:::benchmark_data_smithi070_107919_object45982:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e204)", 2024-09-16T09:37:22.143 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:32:07.478909+0000", 2024-09-16T09:37:22.143 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 314.65237161599998, 2024-09-16T09:37:22.143 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.96429471700000002, 2024-09-16T09:37:22.143 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:37:22.143 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:37:22.143 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:37:22.143 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:37:22.143 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.25064", 2024-09-16T09:37:22.143 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/2578256439", 2024-09-16T09:37:22.144 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 170577 2024-09-16T09:37:22.144 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.144 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:37:22.144 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.144 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:37:22.144 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.478909+0000", 2024-09-16T09:37:22.144 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:37:22.144 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.144 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.144 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:37:22.144 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.478909+0000", 2024-09-16T09:37:22.144 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:37:22.144 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.144 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.145 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:37:22.145 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.478911+0000", 2024-09-16T09:37:22.145 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.8759999999999999e-06 2024-09-16T09:37:22.145 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.145 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.145 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:37:22.145 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.478918+0000", 2024-09-16T09:37:22.145 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.6590000000000001e-06 2024-09-16T09:37:22.145 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.145 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.145 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:37:22.145 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.478936+0000", 2024-09-16T09:37:22.145 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.827e-05 2024-09-16T09:37:22.146 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.146 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.146 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:37:22.146 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.478945+0000", 2024-09-16T09:37:22.146 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.2139999999999996e-06 2024-09-16T09:37:22.146 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.146 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.146 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:37:22.146 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.479008+0000", 2024-09-16T09:37:22.146 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.3459000000000003e-05 2024-09-16T09:37:22.146 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.146 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.147 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:37:22.147 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.479032+0000", 2024-09-16T09:37:22.147 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.3918e-05 2024-09-16T09:37:22.147 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.147 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.147 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:37:22.148 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.439436+0000", 2024-09-16T09:37:22.148 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.96040375899999997 2024-09-16T09:37:22.148 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.148 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.148 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:37:22.148 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.439545+0000", 2024-09-16T09:37:22.149 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000109576 2024-09-16T09:37:22.149 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.149 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.149 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 1,3", 2024-09-16T09:37:22.149 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.439667+0000", 2024-09-16T09:37:22.149 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00012150999999999999 2024-09-16T09:37:22.149 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.150 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.150 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:22.150 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.443079+0000", 2024-09-16T09:37:22.150 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0034118569999999999 2024-09-16T09:37:22.150 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.150 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.150 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:37:22.151 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.443131+0000", 2024-09-16T09:37:22.151 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.2565000000000002e-05 2024-09-16T09:37:22.151 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.151 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.151 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:22.151 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.443149+0000", 2024-09-16T09:37:22.151 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.7343999999999999e-05 2024-09-16T09:37:22.151 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.152 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.152 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:37:22.152 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.443177+0000", 2024-09-16T09:37:22.152 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.8654e-05 2024-09-16T09:37:22.152 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.152 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.153 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:37:22.153 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.443204+0000", 2024-09-16T09:37:22.153 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.7056e-05 2024-09-16T09:37:22.153 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:37:22.153 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:37:22.153 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:37:22.154 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.154 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.154 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.25064.0:170589 8.b 8:d7b2d79b:::benchmark_data_smithi070_107919_object45994:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e204)", 2024-09-16T09:37:22.154 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:32:07.486872+0000", 2024-09-16T09:37:22.154 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 314.644408672, 2024-09-16T09:37:22.155 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.97349502600000004, 2024-09-16T09:37:22.155 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:37:22.155 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:37:22.155 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:37:22.155 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:37:22.155 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.25064", 2024-09-16T09:37:22.155 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/2578256439", 2024-09-16T09:37:22.155 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 170589 2024-09-16T09:37:22.155 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.155 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:37:22.155 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.155 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:37:22.156 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.486872+0000", 2024-09-16T09:37:22.156 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:37:22.156 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.156 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.156 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:37:22.156 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.486872+0000", 2024-09-16T09:37:22.156 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:37:22.156 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.156 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.156 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:37:22.156 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.486874+0000", 2024-09-16T09:37:22.156 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.6300000000000001e-06 2024-09-16T09:37:22.156 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.157 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.157 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:37:22.157 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.486879+0000", 2024-09-16T09:37:22.157 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.1320000000000002e-06 2024-09-16T09:37:22.157 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.157 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.157 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:37:22.157 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.486895+0000", 2024-09-16T09:37:22.157 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.5719999999999999e-05 2024-09-16T09:37:22.157 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.157 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.157 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:37:22.157 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.486924+0000", 2024-09-16T09:37:22.158 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.8742e-05 2024-09-16T09:37:22.158 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.158 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.158 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:37:22.158 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.486985+0000", 2024-09-16T09:37:22.158 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.1099999999999994e-05 2024-09-16T09:37:22.158 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.158 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.158 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:37:22.158 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.486995+0000", 2024-09-16T09:37:22.158 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0538e-05 2024-09-16T09:37:22.158 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.158 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.158 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:37:22.159 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.441464+0000", 2024-09-16T09:37:22.159 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.95446911700000003 2024-09-16T09:37:22.159 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.159 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.159 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:37:22.159 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.441585+0000", 2024-09-16T09:37:22.159 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000120128 2024-09-16T09:37:22.159 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.159 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.159 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 1,3", 2024-09-16T09:37:22.159 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.441713+0000", 2024-09-16T09:37:22.160 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00012807699999999999 2024-09-16T09:37:22.160 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.160 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.160 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:37:22.160 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.444780+0000", 2024-09-16T09:37:22.160 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0030674370000000001 2024-09-16T09:37:22.160 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.160 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.160 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:22.160 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.448814+0000", 2024-09-16T09:37:22.160 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0040338570000000001 2024-09-16T09:37:22.160 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.160 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.161 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:22.161 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.460268+0000", 2024-09-16T09:37:22.161 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.011454097 2024-09-16T09:37:22.161 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.161 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.161 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:37:22.161 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.460321+0000", 2024-09-16T09:37:22.161 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.2630000000000003e-05 2024-09-16T09:37:22.161 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.161 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.161 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:37:22.161 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.460367+0000", 2024-09-16T09:37:22.161 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.6820999999999998e-05 2024-09-16T09:37:22.162 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:37:22.162 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:37:22.162 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:37:22.162 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.162 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.162 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.25064.0:170602 8.b 8:d9b50ae1:::benchmark_data_smithi070_107919_object46007:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e204)", 2024-09-16T09:37:22.162 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:32:07.497614+0000", 2024-09-16T09:37:22.162 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 314.63366673299998, 2024-09-16T09:37:22.162 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.96754656299999997, 2024-09-16T09:37:22.162 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:37:22.162 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:37:22.162 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:37:22.162 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:37:22.162 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.25064", 2024-09-16T09:37:22.163 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/2578256439", 2024-09-16T09:37:22.163 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 170602 2024-09-16T09:37:22.163 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.163 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:37:22.163 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.163 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:37:22.163 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.497614+0000", 2024-09-16T09:37:22.163 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:37:22.163 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.163 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.163 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:37:22.163 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.497614+0000", 2024-09-16T09:37:22.163 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:37:22.163 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.164 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.164 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:37:22.164 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.497616+0000", 2024-09-16T09:37:22.164 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.1050000000000002e-06 2024-09-16T09:37:22.164 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.164 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.164 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:37:22.164 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.497624+0000", 2024-09-16T09:37:22.164 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.0840000000000003e-06 2024-09-16T09:37:22.164 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.164 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.164 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:37:22.164 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.497645+0000", 2024-09-16T09:37:22.164 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.1508000000000001e-05 2024-09-16T09:37:22.164 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.165 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.165 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:37:22.165 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.497655+0000", 2024-09-16T09:37:22.165 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.9879999999999999e-06 2024-09-16T09:37:22.165 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.165 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.165 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:37:22.165 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.497728+0000", 2024-09-16T09:37:22.165 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.2469e-05 2024-09-16T09:37:22.165 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.165 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.165 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:37:22.165 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.497739+0000", 2024-09-16T09:37:22.165 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1053000000000001e-05 2024-09-16T09:37:22.166 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.166 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.166 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:37:22.166 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.443231+0000", 2024-09-16T09:37:22.166 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.94549259399999996 2024-09-16T09:37:22.166 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.166 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.166 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:37:22.166 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.443292+0000", 2024-09-16T09:37:22.166 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.0851999999999999e-05 2024-09-16T09:37:22.166 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.166 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.166 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 1,3", 2024-09-16T09:37:22.166 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.443352+0000", 2024-09-16T09:37:22.166 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.9738999999999998e-05 2024-09-16T09:37:22.167 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.167 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.167 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:37:22.167 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.448824+0000", 2024-09-16T09:37:22.167 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0054719779999999997 2024-09-16T09:37:22.167 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.167 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.167 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:22.167 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.451707+0000", 2024-09-16T09:37:22.167 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0028830180000000002 2024-09-16T09:37:22.167 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.167 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.167 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:22.168 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.465097+0000", 2024-09-16T09:37:22.168 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.013389961000000001 2024-09-16T09:37:22.168 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.168 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.168 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:37:22.169 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.465131+0000", 2024-09-16T09:37:22.169 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.4198e-05 2024-09-16T09:37:22.170 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.170 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.170 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:37:22.170 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.465161+0000", 2024-09-16T09:37:22.170 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.0015999999999999e-05 2024-09-16T09:37:22.170 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:37:22.170 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:37:22.170 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:37:22.170 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.170 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.170 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.25064.0:170610 8.b 8:daf1a32b:::benchmark_data_smithi070_107919_object46015:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e204)", 2024-09-16T09:37:22.170 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:32:07.504240+0000", 2024-09-16T09:37:22.171 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 314.62704150500002, 2024-09-16T09:37:22.171 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.96107343000000001, 2024-09-16T09:37:22.171 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:37:22.171 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:37:22.171 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:37:22.171 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:37:22.171 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.25064", 2024-09-16T09:37:22.171 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/2578256439", 2024-09-16T09:37:22.171 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 170610 2024-09-16T09:37:22.171 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.171 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:37:22.171 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.171 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:37:22.171 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.504240+0000", 2024-09-16T09:37:22.172 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:37:22.172 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.172 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.172 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:37:22.172 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.504240+0000", 2024-09-16T09:37:22.172 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:37:22.172 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.172 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.172 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:37:22.172 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.504240+0000", 2024-09-16T09:37:22.172 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.5099999999999999e-07 2024-09-16T09:37:22.172 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.172 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.172 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:37:22.173 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.504244+0000", 2024-09-16T09:37:22.173 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.4750000000000002e-06 2024-09-16T09:37:22.173 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.173 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.173 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:37:22.173 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.504253+0000", 2024-09-16T09:37:22.173 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.2110000000000004e-06 2024-09-16T09:37:22.173 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.173 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.173 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:37:22.173 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.504259+0000", 2024-09-16T09:37:22.173 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.003e-06 2024-09-16T09:37:22.174 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.174 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.174 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:37:22.174 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.505205+0000", 2024-09-16T09:37:22.174 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000945793 2024-09-16T09:37:22.174 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.174 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.174 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:37:22.174 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.505221+0000", 2024-09-16T09:37:22.174 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.6079999999999999e-05 2024-09-16T09:37:22.174 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.174 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.174 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:37:22.174 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.444807+0000", 2024-09-16T09:37:22.175 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.93958567500000001 2024-09-16T09:37:22.175 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.175 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.175 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:37:22.175 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.444917+0000", 2024-09-16T09:37:22.175 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000110856 2024-09-16T09:37:22.175 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.175 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.175 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 1,3", 2024-09-16T09:37:22.175 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.445032+0000", 2024-09-16T09:37:22.175 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000114125 2024-09-16T09:37:22.175 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.175 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.175 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:37:22.176 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.451674+0000", 2024-09-16T09:37:22.176 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0066422909999999998 2024-09-16T09:37:22.176 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.176 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.176 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:22.176 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.460207+0000", 2024-09-16T09:37:22.176 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0085334610000000009 2024-09-16T09:37:22.176 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.176 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.176 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:22.176 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.465215+0000", 2024-09-16T09:37:22.176 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0050073909999999999 2024-09-16T09:37:22.176 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.176 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.177 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:37:22.177 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.465267+0000", 2024-09-16T09:37:22.177 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.2315000000000003e-05 2024-09-16T09:37:22.177 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.177 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.177 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:37:22.177 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.465313+0000", 2024-09-16T09:37:22.177 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.6003e-05 2024-09-16T09:37:22.177 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:37:22.177 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:37:22.177 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:37:22.177 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.177 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.178 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.25064.0:170643 8.b 8:d543baed:::benchmark_data_smithi070_107919_object46048:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e204)", 2024-09-16T09:37:22.178 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:32:07.532952+0000", 2024-09-16T09:37:22.178 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 314.59832872099997, 2024-09-16T09:37:22.178 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.93245734999999996, 2024-09-16T09:37:22.178 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:37:22.178 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:37:22.178 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:37:22.178 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:37:22.178 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.25064", 2024-09-16T09:37:22.178 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/2578256439", 2024-09-16T09:37:22.178 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 170643 2024-09-16T09:37:22.178 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.179 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:37:22.179 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.179 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:37:22.179 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.532952+0000", 2024-09-16T09:37:22.179 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:37:22.179 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.179 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.179 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:37:22.179 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.532952+0000", 2024-09-16T09:37:22.179 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:37:22.179 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.179 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.179 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:37:22.179 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.532954+0000", 2024-09-16T09:37:22.180 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.79e-06 2024-09-16T09:37:22.180 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.180 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.180 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:37:22.180 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.532960+0000", 2024-09-16T09:37:22.180 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.9950000000000002e-06 2024-09-16T09:37:22.180 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.180 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.180 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:37:22.180 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.532976+0000", 2024-09-16T09:37:22.180 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.6243e-05 2024-09-16T09:37:22.180 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.180 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.181 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:37:22.181 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.532984+0000", 2024-09-16T09:37:22.181 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.1740000000000003e-06 2024-09-16T09:37:22.181 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.181 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.181 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:37:22.181 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.533052+0000", 2024-09-16T09:37:22.181 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.8814999999999998e-05 2024-09-16T09:37:22.181 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.181 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.181 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:37:22.181 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.533062+0000", 2024-09-16T09:37:22.181 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.2720000000000003e-06 2024-09-16T09:37:22.181 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.182 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.182 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:37:22.182 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.446896+0000", 2024-09-16T09:37:22.182 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.91383476600000002 2024-09-16T09:37:22.182 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.182 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.182 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:37:22.182 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.447014+0000", 2024-09-16T09:37:22.182 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000117649 2024-09-16T09:37:22.182 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.182 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.182 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 1,3", 2024-09-16T09:37:22.182 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.447092+0000", 2024-09-16T09:37:22.182 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.7888999999999999e-05 2024-09-16T09:37:22.182 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.183 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.183 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:37:22.183 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.452731+0000", 2024-09-16T09:37:22.183 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0056386359999999998 2024-09-16T09:37:22.183 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.183 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.183 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:22.184 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.461994+0000", 2024-09-16T09:37:22.184 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.009263149 2024-09-16T09:37:22.184 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.184 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.184 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:22.184 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.465360+0000", 2024-09-16T09:37:22.184 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0033663159999999998 2024-09-16T09:37:22.184 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.184 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.184 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:37:22.184 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.465388+0000", 2024-09-16T09:37:22.184 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.7433e-05 2024-09-16T09:37:22.185 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.185 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.185 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:37:22.185 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.465410+0000", 2024-09-16T09:37:22.185 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.2223000000000001e-05 2024-09-16T09:37:22.185 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:37:22.185 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:37:22.185 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:37:22.185 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.185 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.185 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.25064.0:170647 8.b 8:dc89519e:::benchmark_data_smithi070_107919_object46052:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e204)", 2024-09-16T09:37:22.185 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:32:07.536170+0000", 2024-09-16T09:37:22.185 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 314.59511074, 2024-09-16T09:37:22.186 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.92937628299999997, 2024-09-16T09:37:22.186 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:37:22.186 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:37:22.186 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:37:22.186 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:37:22.186 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.25064", 2024-09-16T09:37:22.186 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/2578256439", 2024-09-16T09:37:22.186 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 170647 2024-09-16T09:37:22.186 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.186 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:37:22.186 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.186 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:37:22.186 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.536170+0000", 2024-09-16T09:37:22.187 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:37:22.187 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.187 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.187 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:37:22.187 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.536170+0000", 2024-09-16T09:37:22.187 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:37:22.187 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.187 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.187 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:37:22.187 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.536172+0000", 2024-09-16T09:37:22.187 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.525e-06 2024-09-16T09:37:22.187 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.187 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.187 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:37:22.188 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.536178+0000", 2024-09-16T09:37:22.188 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.6500000000000001e-06 2024-09-16T09:37:22.188 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.188 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.188 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:37:22.188 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.536192+0000", 2024-09-16T09:37:22.188 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.4374e-05 2024-09-16T09:37:22.188 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.188 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.188 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:37:22.188 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.536199+0000", 2024-09-16T09:37:22.188 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.1289999999999998e-06 2024-09-16T09:37:22.188 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.189 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.189 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:37:22.189 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.536257+0000", 2024-09-16T09:37:22.189 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.8238e-05 2024-09-16T09:37:22.189 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.189 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.189 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:37:22.189 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.536266+0000", 2024-09-16T09:37:22.189 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.7029999999999995e-06 2024-09-16T09:37:22.189 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.189 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.189 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:37:22.189 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.448851+0000", 2024-09-16T09:37:22.190 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.91258534400000002 2024-09-16T09:37:22.190 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.190 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.190 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:37:22.190 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.448940+0000", 2024-09-16T09:37:22.190 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.8317000000000002e-05 2024-09-16T09:37:22.190 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.190 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.190 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 1,3", 2024-09-16T09:37:22.190 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.449011+0000", 2024-09-16T09:37:22.190 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.1390000000000006e-05 2024-09-16T09:37:22.190 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.190 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.190 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:37:22.191 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.452738+0000", 2024-09-16T09:37:22.191 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0037269479999999999 2024-09-16T09:37:22.191 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.191 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.191 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:22.191 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.462063+0000", 2024-09-16T09:37:22.191 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0093253020000000006 2024-09-16T09:37:22.191 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.191 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.191 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:22.191 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.465456+0000", 2024-09-16T09:37:22.191 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0033928539999999998 2024-09-16T09:37:22.191 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.191 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.191 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:37:22.191 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.465506+0000", 2024-09-16T09:37:22.192 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.0167999999999998e-05 2024-09-16T09:37:22.192 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.192 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.192 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:37:22.192 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.465547+0000", 2024-09-16T09:37:22.192 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.0340999999999999e-05 2024-09-16T09:37:22.192 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:37:22.192 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:37:22.192 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:37:22.192 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.192 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.192 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.25064.0:170683 8.b 8:dd769891:::benchmark_data_smithi070_107919_object46088:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e204)", 2024-09-16T09:37:22.192 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:32:07.569822+0000", 2024-09-16T09:37:22.192 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 314.56145862400001, 2024-09-16T09:37:22.193 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.89580738500000001, 2024-09-16T09:37:22.193 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:37:22.193 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:37:22.193 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:37:22.193 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:37:22.193 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.25064", 2024-09-16T09:37:22.193 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/2578256439", 2024-09-16T09:37:22.193 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 170683 2024-09-16T09:37:22.193 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.193 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:37:22.194 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.194 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:37:22.194 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.569822+0000", 2024-09-16T09:37:22.194 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:37:22.194 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.194 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.194 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:37:22.194 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.569822+0000", 2024-09-16T09:37:22.194 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:37:22.194 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.194 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.194 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:37:22.194 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.569823+0000", 2024-09-16T09:37:22.194 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.2900000000000004e-07 2024-09-16T09:37:22.194 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.195 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.195 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:37:22.195 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.569826+0000", 2024-09-16T09:37:22.195 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.65e-06 2024-09-16T09:37:22.195 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.195 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.195 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:37:22.195 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.569834+0000", 2024-09-16T09:37:22.195 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.1610000000000003e-06 2024-09-16T09:37:22.195 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.195 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.195 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:37:22.195 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.569839+0000", 2024-09-16T09:37:22.196 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.2480000000000004e-06 2024-09-16T09:37:22.196 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.196 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.196 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:37:22.196 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.569896+0000", 2024-09-16T09:37:22.196 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.7076e-05 2024-09-16T09:37:22.196 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.196 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.196 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:37:22.196 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.569914+0000", 2024-09-16T09:37:22.196 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.8151e-05 2024-09-16T09:37:22.196 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.196 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.196 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:37:22.197 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.450541+0000", 2024-09-16T09:37:22.197 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.88062674299999999 2024-09-16T09:37:22.197 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.197 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.197 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:37:22.197 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.450607+0000", 2024-09-16T09:37:22.197 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.6024000000000003e-05 2024-09-16T09:37:22.197 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.197 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.197 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 1,3", 2024-09-16T09:37:22.197 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.450672+0000", 2024-09-16T09:37:22.197 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.4826999999999994e-05 2024-09-16T09:37:22.197 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.197 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.198 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:37:22.198 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.457678+0000", 2024-09-16T09:37:22.198 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0070063829999999997 2024-09-16T09:37:22.198 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.198 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.198 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:22.198 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.463607+0000", 2024-09-16T09:37:22.198 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.005928427 2024-09-16T09:37:22.198 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.198 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.198 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:22.198 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.465583+0000", 2024-09-16T09:37:22.198 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0019759019999999999 2024-09-16T09:37:22.199 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.199 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.199 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:37:22.199 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.465610+0000", 2024-09-16T09:37:22.199 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.7498e-05 2024-09-16T09:37:22.199 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.199 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.199 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:37:22.199 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.465630+0000", 2024-09-16T09:37:22.199 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.9766e-05 2024-09-16T09:37:22.199 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:37:22.199 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:37:22.199 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:37:22.199 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.200 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.200 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:67145 9.16 9:6eb0471e:::benchmark_data_smithi070_137000_object67144:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e254)", 2024-09-16T09:37:22.200 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:34:11.477245+0000", 2024-09-16T09:37:22.200 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 190.65403571600001, 2024-09-16T09:37:22.200 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.00133326, 2024-09-16T09:37:22.200 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:37:22.200 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:37:22.200 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:37:22.200 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:37:22.200 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:37:22.200 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:37:22.200 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 67145 2024-09-16T09:37:22.200 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.200 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:37:22.200 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.201 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:37:22.201 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.477245+0000", 2024-09-16T09:37:22.201 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:37:22.201 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.201 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.201 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:37:22.201 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.477245+0000", 2024-09-16T09:37:22.201 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:37:22.201 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.201 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.201 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:37:22.201 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.477247+0000", 2024-09-16T09:37:22.201 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.2529999999999999e-06 2024-09-16T09:37:22.202 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.202 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.202 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:37:22.202 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.477323+0000", 2024-09-16T09:37:22.202 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.6688999999999997e-05 2024-09-16T09:37:22.202 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.202 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.202 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:37:22.202 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.477339+0000", 2024-09-16T09:37:22.202 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.5832999999999999e-05 2024-09-16T09:37:22.202 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.202 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.202 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:37:22.202 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.477346+0000", 2024-09-16T09:37:22.202 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.153e-06 2024-09-16T09:37:22.203 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.203 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.203 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:37:22.203 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.477404+0000", 2024-09-16T09:37:22.203 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.8041999999999998e-05 2024-09-16T09:37:22.203 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.203 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.203 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:37:22.203 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.477413+0000", 2024-09-16T09:37:22.203 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.5110000000000003e-06 2024-09-16T09:37:22.203 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.203 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.203 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:37:22.204 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.452134+0000", 2024-09-16T09:37:22.204 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.97472151100000004 2024-09-16T09:37:22.204 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.204 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.204 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:37:22.204 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.452270+0000", 2024-09-16T09:37:22.204 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00013596699999999999 2024-09-16T09:37:22.204 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.204 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.204 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 1,2", 2024-09-16T09:37:22.204 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.452413+0000", 2024-09-16T09:37:22.204 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000142884 2024-09-16T09:37:22.204 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.205 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.205 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:37:22.205 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.458265+0000", 2024-09-16T09:37:22.205 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0058516549999999999 2024-09-16T09:37:22.205 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.205 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.205 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:22.205 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.460714+0000", 2024-09-16T09:37:22.205 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0024490620000000001 2024-09-16T09:37:22.205 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.205 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.205 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:22.205 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.478484+0000", 2024-09-16T09:37:22.205 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.017769589999999998 2024-09-16T09:37:22.206 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.206 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.206 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:37:22.206 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.478535+0000", 2024-09-16T09:37:22.206 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.1928999999999997e-05 2024-09-16T09:37:22.206 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.206 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.206 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:37:22.206 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.478579+0000", 2024-09-16T09:37:22.206 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.3180999999999999e-05 2024-09-16T09:37:22.206 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:37:22.206 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:37:22.206 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:37:22.206 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.206 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.206 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:67148 9.16 9:6b4987c7:::benchmark_data_smithi070_137000_object67147:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e254)", 2024-09-16T09:37:22.207 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:34:11.477768+0000", 2024-09-16T09:37:22.207 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 190.65351333500001, 2024-09-16T09:37:22.207 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.001389544, 2024-09-16T09:37:22.207 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:37:22.207 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:37:22.207 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:37:22.207 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:37:22.207 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:37:22.207 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:37:22.207 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 67148 2024-09-16T09:37:22.207 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.207 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:37:22.207 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.208 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:37:22.208 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.477768+0000", 2024-09-16T09:37:22.208 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:37:22.208 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.208 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.208 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:37:22.208 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.477768+0000", 2024-09-16T09:37:22.208 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:37:22.208 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.208 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.208 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:37:22.209 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.477769+0000", 2024-09-16T09:37:22.209 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.7000000000000003e-07 2024-09-16T09:37:22.209 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.209 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.209 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:37:22.209 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.477849+0000", 2024-09-16T09:37:22.209 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.0500999999999994e-05 2024-09-16T09:37:22.209 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.209 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.209 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:37:22.209 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.477867+0000", 2024-09-16T09:37:22.209 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.8182e-05 2024-09-16T09:37:22.209 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.209 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.209 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:37:22.209 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.477876+0000", 2024-09-16T09:37:22.210 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.0900000000000005e-06 2024-09-16T09:37:22.210 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.210 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.210 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:37:22.210 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.478146+0000", 2024-09-16T09:37:22.210 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00027060400000000003 2024-09-16T09:37:22.210 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.210 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.210 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:37:22.210 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.478156+0000", 2024-09-16T09:37:22.210 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.7990000000000005e-06 2024-09-16T09:37:22.210 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.210 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.210 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:37:22.210 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.453564+0000", 2024-09-16T09:37:22.211 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.97540761799999998 2024-09-16T09:37:22.211 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.211 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.211 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:37:22.211 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.453683+0000", 2024-09-16T09:37:22.211 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000119882 2024-09-16T09:37:22.211 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.211 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.211 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 1,2", 2024-09-16T09:37:22.211 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.453825+0000", 2024-09-16T09:37:22.211 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000142006 2024-09-16T09:37:22.211 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.211 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.211 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:37:22.211 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.458271+0000", 2024-09-16T09:37:22.211 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0044451899999999999 2024-09-16T09:37:22.212 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.212 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.212 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:22.212 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.460761+0000", 2024-09-16T09:37:22.212 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0024901599999999999 2024-09-16T09:37:22.212 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.212 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.212 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:22.212 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.479057+0000", 2024-09-16T09:37:22.212 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.018296501 2024-09-16T09:37:22.212 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.212 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.212 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:37:22.212 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.479120+0000", 2024-09-16T09:37:22.213 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.3081000000000001e-05 2024-09-16T09:37:22.213 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.213 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.213 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:37:22.213 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.479157+0000", 2024-09-16T09:37:22.213 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.6959999999999998e-05 2024-09-16T09:37:22.213 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:37:22.213 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:37:22.213 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:37:22.213 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.214 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.214 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:67153 9.16 9:6b621114:::benchmark_data_smithi070_137000_object67152:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e254)", 2024-09-16T09:37:22.214 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:34:11.481611+0000", 2024-09-16T09:37:22.214 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 190.649670001, 2024-09-16T09:37:22.214 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.99775295500000005, 2024-09-16T09:37:22.214 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:37:22.214 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:37:22.214 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:37:22.214 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:37:22.214 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:37:22.214 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:37:22.214 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 67153 2024-09-16T09:37:22.214 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.214 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:37:22.214 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.215 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:37:22.215 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.481611+0000", 2024-09-16T09:37:22.215 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:37:22.215 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.215 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.215 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:37:22.215 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.481611+0000", 2024-09-16T09:37:22.215 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:37:22.215 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.215 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.215 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:37:22.215 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.481612+0000", 2024-09-16T09:37:22.215 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.1900000000000001e-07 2024-09-16T09:37:22.215 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.216 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.216 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:37:22.216 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.481650+0000", 2024-09-16T09:37:22.216 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.7489e-05 2024-09-16T09:37:22.216 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.216 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.216 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:37:22.216 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.481659+0000", 2024-09-16T09:37:22.216 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.4909999999999994e-06 2024-09-16T09:37:22.216 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.216 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.216 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:37:22.216 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.481666+0000", 2024-09-16T09:37:22.216 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.9809999999999997e-06 2024-09-16T09:37:22.216 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.217 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.217 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:37:22.217 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.481724+0000", 2024-09-16T09:37:22.217 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.8035000000000003e-05 2024-09-16T09:37:22.217 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.217 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.217 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:37:22.217 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.481733+0000", 2024-09-16T09:37:22.217 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.3489999999999998e-06 2024-09-16T09:37:22.217 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.217 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.217 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:37:22.217 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.455089+0000", 2024-09-16T09:37:22.217 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.97335562499999995 2024-09-16T09:37:22.218 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.218 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.218 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:37:22.218 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.455208+0000", 2024-09-16T09:37:22.218 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000119305 2024-09-16T09:37:22.218 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.218 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.218 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 1,2", 2024-09-16T09:37:22.218 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.455357+0000", 2024-09-16T09:37:22.218 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00014880000000000001 2024-09-16T09:37:22.218 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.218 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.218 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:37:22.218 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.459587+0000", 2024-09-16T09:37:22.219 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0042294289999999998 2024-09-16T09:37:22.219 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.219 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.219 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:22.219 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.460813+0000", 2024-09-16T09:37:22.219 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0012265399999999999 2024-09-16T09:37:22.219 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.219 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.219 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:22.219 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.479275+0000", 2024-09-16T09:37:22.219 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.018462135000000001 2024-09-16T09:37:22.219 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.219 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.219 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:37:22.219 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.479321+0000", 2024-09-16T09:37:22.220 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.6267000000000003e-05 2024-09-16T09:37:22.220 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.220 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.220 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:37:22.220 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.479364+0000", 2024-09-16T09:37:22.220 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.2589999999999997e-05 2024-09-16T09:37:22.220 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:37:22.220 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:37:22.220 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:37:22.220 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.220 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.220 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:67177 9.16 9:6c8c3aa4:::benchmark_data_smithi070_137000_object67176:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e254)", 2024-09-16T09:37:22.220 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:34:11.503406+0000", 2024-09-16T09:37:22.220 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 190.62787511799999, 2024-09-16T09:37:22.221 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.97608242499999998, 2024-09-16T09:37:22.221 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:37:22.221 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:37:22.221 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:37:22.221 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:37:22.221 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:37:22.221 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:37:22.221 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 67177 2024-09-16T09:37:22.221 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.221 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:37:22.221 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.221 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:37:22.221 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.503406+0000", 2024-09-16T09:37:22.221 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:37:22.221 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.222 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.222 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:37:22.222 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.503406+0000", 2024-09-16T09:37:22.222 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:37:22.222 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.222 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.222 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:37:22.222 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.503406+0000", 2024-09-16T09:37:22.222 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.01e-07 2024-09-16T09:37:22.222 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.222 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.223 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:37:22.223 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.503432+0000", 2024-09-16T09:37:22.223 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.5673e-05 2024-09-16T09:37:22.223 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.223 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.223 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:37:22.223 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.503442+0000", 2024-09-16T09:37:22.223 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.7040000000000006e-06 2024-09-16T09:37:22.223 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.223 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.223 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:37:22.223 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.503445+0000", 2024-09-16T09:37:22.223 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.6339999999999999e-06 2024-09-16T09:37:22.223 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.224 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.224 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:37:22.224 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.503516+0000", 2024-09-16T09:37:22.224 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.0438999999999994e-05 2024-09-16T09:37:22.224 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.224 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.224 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:37:22.224 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.503528+0000", 2024-09-16T09:37:22.224 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.2349e-05 2024-09-16T09:37:22.224 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.224 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.224 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:37:22.224 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.456887+0000", 2024-09-16T09:37:22.225 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.95335896399999998 2024-09-16T09:37:22.225 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.225 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.225 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:37:22.225 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.457028+0000", 2024-09-16T09:37:22.225 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00014072600000000001 2024-09-16T09:37:22.225 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.225 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.225 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 1,2", 2024-09-16T09:37:22.225 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.457172+0000", 2024-09-16T09:37:22.225 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000144389 2024-09-16T09:37:22.225 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.225 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.225 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:37:22.226 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.460657+0000", 2024-09-16T09:37:22.226 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0034844250000000002 2024-09-16T09:37:22.226 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.226 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.226 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:22.226 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.460862+0000", 2024-09-16T09:37:22.226 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00020545900000000001 2024-09-16T09:37:22.226 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.226 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.226 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:22.226 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.479405+0000", 2024-09-16T09:37:22.226 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.018543343 2024-09-16T09:37:22.226 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.226 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.227 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:37:22.227 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.479458+0000", 2024-09-16T09:37:22.227 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.2652999999999999e-05 2024-09-16T09:37:22.227 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.227 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.227 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:37:22.227 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.479488+0000", 2024-09-16T09:37:22.227 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.0266000000000001e-05 2024-09-16T09:37:22.227 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:37:22.227 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:37:22.227 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:37:22.227 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.227 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.227 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:67186 9.16 9:6ddb896c:::benchmark_data_smithi070_137000_object67185:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e254)", 2024-09-16T09:37:22.227 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:34:11.514674+0000", 2024-09-16T09:37:22.228 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 190.616607106, 2024-09-16T09:37:22.228 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.96492478400000004, 2024-09-16T09:37:22.228 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:37:22.228 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:37:22.228 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:37:22.228 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:37:22.228 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:37:22.228 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:37:22.228 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 67186 2024-09-16T09:37:22.228 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.228 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:37:22.228 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.228 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:37:22.228 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.514674+0000", 2024-09-16T09:37:22.229 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:37:22.229 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.229 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.229 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:37:22.229 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.514674+0000", 2024-09-16T09:37:22.229 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:37:22.229 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.229 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.229 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:37:22.229 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.514676+0000", 2024-09-16T09:37:22.229 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.9690000000000001e-06 2024-09-16T09:37:22.230 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.230 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.230 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:37:22.230 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.514846+0000", 2024-09-16T09:37:22.230 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00016989199999999999 2024-09-16T09:37:22.230 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.230 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.230 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:37:22.230 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.514862+0000", 2024-09-16T09:37:22.230 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.6317999999999999e-05 2024-09-16T09:37:22.230 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.231 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.231 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:37:22.231 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.514870+0000", 2024-09-16T09:37:22.231 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.1699999999999997e-06 2024-09-16T09:37:22.231 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.231 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.231 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:37:22.231 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.514942+0000", 2024-09-16T09:37:22.231 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.1619999999999995e-05 2024-09-16T09:37:22.231 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.231 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.231 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:37:22.231 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.514952+0000", 2024-09-16T09:37:22.232 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.577e-06 2024-09-16T09:37:22.232 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.232 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.232 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:37:22.232 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.458305+0000", 2024-09-16T09:37:22.232 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.94335340499999998 2024-09-16T09:37:22.232 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.232 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.232 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:37:22.232 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.458454+0000", 2024-09-16T09:37:22.232 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00014879499999999999 2024-09-16T09:37:22.232 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.232 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.232 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 1,2", 2024-09-16T09:37:22.233 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.458547+0000", 2024-09-16T09:37:22.233 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.3602000000000003e-05 2024-09-16T09:37:22.233 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.233 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.233 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:37:22.233 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.460826+0000", 2024-09-16T09:37:22.233 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0022790449999999999 2024-09-16T09:37:22.233 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.233 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.233 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:22.233 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.466858+0000", 2024-09-16T09:37:22.233 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0060317620000000004 2024-09-16T09:37:22.233 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.233 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.234 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:22.234 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.479526+0000", 2024-09-16T09:37:22.234 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.012668008 2024-09-16T09:37:22.234 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.234 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.234 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:37:22.234 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.479570+0000", 2024-09-16T09:37:22.234 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.3455000000000003e-05 2024-09-16T09:37:22.234 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.234 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.234 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:37:22.234 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.479599+0000", 2024-09-16T09:37:22.234 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.9166000000000002e-05 2024-09-16T09:37:22.234 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:37:22.235 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:37:22.235 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:37:22.235 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.235 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.235 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:67213 9.16 9:6ebe1d56:::benchmark_data_smithi070_137000_object67212:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e254)", 2024-09-16T09:37:22.235 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:34:11.541163+0000", 2024-09-16T09:37:22.235 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 190.59011827200001, 2024-09-16T09:37:22.235 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.93855440499999998, 2024-09-16T09:37:22.235 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:37:22.235 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:37:22.235 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:37:22.235 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:37:22.235 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:37:22.235 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:37:22.236 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 67213 2024-09-16T09:37:22.236 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.236 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:37:22.236 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.236 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:37:22.236 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.541163+0000", 2024-09-16T09:37:22.236 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:37:22.236 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.236 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.236 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:37:22.236 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.541163+0000", 2024-09-16T09:37:22.236 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:37:22.236 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.236 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.237 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:37:22.237 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.541164+0000", 2024-09-16T09:37:22.237 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.607e-06 2024-09-16T09:37:22.237 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.237 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.237 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:37:22.237 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.541193+0000", 2024-09-16T09:37:22.237 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.8742e-05 2024-09-16T09:37:22.237 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.237 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.237 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:37:22.237 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.541211+0000", 2024-09-16T09:37:22.237 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.8088999999999999e-05 2024-09-16T09:37:22.238 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.238 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.238 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:37:22.238 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.541219+0000", 2024-09-16T09:37:22.238 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.6760000000000007e-06 2024-09-16T09:37:22.238 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.238 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.238 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:37:22.238 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.541282+0000", 2024-09-16T09:37:22.238 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.3074e-05 2024-09-16T09:37:22.238 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.238 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.238 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:37:22.238 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.541294+0000", 2024-09-16T09:37:22.238 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1619e-05 2024-09-16T09:37:22.239 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.239 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.239 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:37:22.239 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.459650+0000", 2024-09-16T09:37:22.239 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.91835631900000003 2024-09-16T09:37:22.239 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.239 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.239 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:37:22.239 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.459767+0000", 2024-09-16T09:37:22.239 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000117439 2024-09-16T09:37:22.239 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.239 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.240 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 1,2", 2024-09-16T09:37:22.240 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.459847+0000", 2024-09-16T09:37:22.240 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.9629999999999995e-05 2024-09-16T09:37:22.240 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.240 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.240 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:37:22.240 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.461953+0000", 2024-09-16T09:37:22.240 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0021063269999999999 2024-09-16T09:37:22.240 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.240 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.240 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:22.240 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.466906+0000", 2024-09-16T09:37:22.240 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0049525569999999998 2024-09-16T09:37:22.241 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.241 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.241 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:22.241 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.479651+0000", 2024-09-16T09:37:22.241 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.012744734000000001 2024-09-16T09:37:22.241 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.241 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.241 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:37:22.241 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.479681+0000", 2024-09-16T09:37:22.241 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.0688000000000002e-05 2024-09-16T09:37:22.241 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.241 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.241 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:37:22.241 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.479717+0000", 2024-09-16T09:37:22.241 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.5904e-05 2024-09-16T09:37:22.242 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:37:22.242 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:37:22.242 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:37:22.242 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.242 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.242 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:67242 9.16 9:6a225ad4:::benchmark_data_smithi070_137000_object67241:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e254)", 2024-09-16T09:37:22.242 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:34:11.573512+0000", 2024-09-16T09:37:22.242 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 190.55776928899999, 2024-09-16T09:37:22.242 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.90633363499999997, 2024-09-16T09:37:22.242 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:37:22.242 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:37:22.242 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:37:22.242 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:37:22.242 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:37:22.242 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:37:22.243 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 67242 2024-09-16T09:37:22.243 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.243 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:37:22.243 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.243 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:37:22.243 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.573512+0000", 2024-09-16T09:37:22.243 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:37:22.243 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.243 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.243 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:37:22.243 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.573512+0000", 2024-09-16T09:37:22.243 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:37:22.243 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.243 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.243 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:37:22.244 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.573513+0000", 2024-09-16T09:37:22.244 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.4729999999999999e-06 2024-09-16T09:37:22.244 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.244 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.244 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:37:22.244 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.573545+0000", 2024-09-16T09:37:22.244 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.1516000000000001e-05 2024-09-16T09:37:22.244 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.244 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.244 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:37:22.244 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.573560+0000", 2024-09-16T09:37:22.244 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.5218000000000001e-05 2024-09-16T09:37:22.244 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.244 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.244 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:37:22.429 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.573567+0000", 2024-09-16T09:37:22.429 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.4209999999999998e-06 2024-09-16T09:37:22.429 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.429 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.429 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:37:22.430 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.573622+0000", 2024-09-16T09:37:22.430 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.4625e-05 2024-09-16T09:37:22.430 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.430 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.430 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:37:22.430 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.573628+0000", 2024-09-16T09:37:22.430 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.2129999999999996e-06 2024-09-16T09:37:22.430 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.430 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.430 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:37:22.430 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.460896+0000", 2024-09-16T09:37:22.431 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.88726759899999996 2024-09-16T09:37:22.431 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.431 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.431 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:37:22.431 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.461000+0000", 2024-09-16T09:37:22.431 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000103923 2024-09-16T09:37:22.431 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.431 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.431 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 1,2", 2024-09-16T09:37:22.431 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.461143+0000", 2024-09-16T09:37:22.432 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000143635 2024-09-16T09:37:22.432 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.432 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.432 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:37:22.432 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.465556+0000", 2024-09-16T09:37:22.432 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0044125800000000001 2024-09-16T09:37:22.432 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.432 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.432 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:22.432 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.466954+0000", 2024-09-16T09:37:22.433 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.001397868 2024-09-16T09:37:22.433 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.433 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.433 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:22.433 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.479774+0000", 2024-09-16T09:37:22.433 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.012819841 2024-09-16T09:37:22.433 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.433 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.433 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:37:22.433 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.479815+0000", 2024-09-16T09:37:22.433 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.0818999999999997e-05 2024-09-16T09:37:22.433 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.433 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.433 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:37:22.434 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.479845+0000", 2024-09-16T09:37:22.434 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.0904000000000001e-05 2024-09-16T09:37:22.434 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:37:22.434 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:37:22.434 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:37:22.434 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.434 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.434 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:231311 9.14 9:2a6a4bc0:::benchmark_data_smithi070_137000_object109600:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e357)", 2024-09-16T09:37:22.434 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:37:18.512297+0000", 2024-09-16T09:37:22.434 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 3.6189837140000001, 2024-09-16T09:37:22.434 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0604506849999999, 2024-09-16T09:37:22.434 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:37:22.434 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:37:22.434 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "reached pg", 2024-09-16T09:37:22.435 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:37:22.435 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:37:22.435 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:37:22.435 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 231311 2024-09-16T09:37:22.435 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.435 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:37:22.435 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.435 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:37:22.435 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.512297+0000", 2024-09-16T09:37:22.435 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:37:22.435 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.435 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.435 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:37:22.435 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.512297+0000", 2024-09-16T09:37:22.436 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:37:22.436 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.436 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.436 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:37:22.436 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.512300+0000", 2024-09-16T09:37:22.436 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.3499999999999999e-06 2024-09-16T09:37:22.436 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.436 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.436 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:37:22.436 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.512306+0000", 2024-09-16T09:37:22.436 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.246e-06 2024-09-16T09:37:22.436 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.436 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.436 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:37:22.437 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.512327+0000", 2024-09-16T09:37:22.437 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.0866e-05 2024-09-16T09:37:22.437 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.437 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.437 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:37:22.437 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.512337+0000", 2024-09-16T09:37:22.437 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.7780000000000002e-06 2024-09-16T09:37:22.437 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.437 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.437 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:37:22.437 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.633134+0000", 2024-09-16T09:37:22.437 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.120797824 2024-09-16T09:37:22.437 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.438 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.438 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:37:22.438 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.633143+0000", 2024-09-16T09:37:22.438 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.6379999999999993e-06 2024-09-16T09:37:22.438 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.438 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.438 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:37:22.438 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.572704+0000", 2024-09-16T09:37:22.439 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.93956075500000003 2024-09-16T09:37:22.439 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.439 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.439 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:37:22.439 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.572748+0000", 2024-09-16T09:37:22.439 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.4227999999999997e-05 2024-09-16T09:37:22.439 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:37:22.439 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:37:22.439 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:37:22.439 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.439 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.439 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:231323 9.4 9:248ba587:::benchmark_data_smithi070_137000_object109612:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e357)", 2024-09-16T09:37:22.440 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:37:18.512353+0000", 2024-09-16T09:37:22.440 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 3.618928199, 2024-09-16T09:37:22.440 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.057531448, 2024-09-16T09:37:22.440 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:37:22.440 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:37:22.440 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:37:22.440 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:37:22.440 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:37:22.440 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:37:22.440 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 231323 2024-09-16T09:37:22.440 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.440 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:37:22.440 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.441 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:37:22.441 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.512353+0000", 2024-09-16T09:37:22.441 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:37:22.441 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.441 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.441 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:37:22.441 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.512353+0000", 2024-09-16T09:37:22.441 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:37:22.441 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.441 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.441 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:37:22.441 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.512353+0000", 2024-09-16T09:37:22.441 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.2500000000000001e-07 2024-09-16T09:37:22.441 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.442 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.442 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:37:22.442 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.512356+0000", 2024-09-16T09:37:22.442 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.3539999999999998e-06 2024-09-16T09:37:22.442 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.442 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.442 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:37:22.442 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.512366+0000", 2024-09-16T09:37:22.442 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.9310000000000003e-06 2024-09-16T09:37:22.442 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.442 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.442 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:37:22.442 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.512371+0000", 2024-09-16T09:37:22.443 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.2850000000000002e-06 2024-09-16T09:37:22.443 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.443 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.443 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:37:22.443 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.633167+0000", 2024-09-16T09:37:22.443 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.12079590699999999 2024-09-16T09:37:22.443 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.443 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.443 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:37:22.443 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.633174+0000", 2024-09-16T09:37:22.443 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.2359999999999998e-06 2024-09-16T09:37:22.443 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.443 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.443 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:37:22.443 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.552690+0000", 2024-09-16T09:37:22.444 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.91951564699999999 2024-09-16T09:37:22.444 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.444 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.444 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:37:22.444 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.552776+0000", 2024-09-16T09:37:22.444 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.6244999999999998e-05 2024-09-16T09:37:22.444 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.444 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.444 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 1,3", 2024-09-16T09:37:22.444 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.552844+0000", 2024-09-16T09:37:22.444 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.7615999999999998e-05 2024-09-16T09:37:22.444 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.444 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.445 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:37:22.445 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.555525+0000", 2024-09-16T09:37:22.445 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0026812189999999999 2024-09-16T09:37:22.445 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.445 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.445 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:22.445 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.568683+0000", 2024-09-16T09:37:22.445 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.013158433000000001 2024-09-16T09:37:22.445 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.445 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.445 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:22.445 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.569788+0000", 2024-09-16T09:37:22.445 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00110486 2024-09-16T09:37:22.445 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.446 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.446 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:37:22.446 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.569844+0000", 2024-09-16T09:37:22.446 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.5894999999999999e-05 2024-09-16T09:37:22.446 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.446 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.446 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:37:22.446 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.569884+0000", 2024-09-16T09:37:22.446 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.0395000000000002e-05 2024-09-16T09:37:22.446 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:37:22.446 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:37:22.446 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:37:22.446 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.446 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.446 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:232744 9.16 9.f8cf6db6 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e359)", 2024-09-16T09:37:22.447 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:37:20.541411+0000", 2024-09-16T09:37:22.447 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 1.589870042, 2024-09-16T09:37:22.447 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.97886637600000004, 2024-09-16T09:37:22.447 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:37:22.447 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:37:22.447 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "reached pg", 2024-09-16T09:37:22.447 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:37:22.447 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:37:22.447 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:37:22.447 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 232744 2024-09-16T09:37:22.447 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.447 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:37:22.447 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.448 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:37:22.448 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.541411+0000", 2024-09-16T09:37:22.448 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:37:22.448 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.448 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.448 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:37:22.448 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.541411+0000", 2024-09-16T09:37:22.448 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:37:22.448 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.448 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.449 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:37:22.449 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.541413+0000", 2024-09-16T09:37:22.449 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.0899999999999999e-06 2024-09-16T09:37:22.449 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.449 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.449 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:37:22.449 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.541421+0000", 2024-09-16T09:37:22.449 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.5059999999999998e-06 2024-09-16T09:37:22.449 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.449 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.449 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:37:22.449 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.541444+0000", 2024-09-16T09:37:22.450 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.3776000000000001e-05 2024-09-16T09:37:22.450 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.450 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.450 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:37:22.450 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.541455+0000", 2024-09-16T09:37:22.450 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0285999999999999e-05 2024-09-16T09:37:22.451 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.451 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.451 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:37:22.451 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.542060+0000", 2024-09-16T09:37:22.451 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00060544400000000001 2024-09-16T09:37:22.451 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.451 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.451 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:37:22.452 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.542073+0000", 2024-09-16T09:37:22.452 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.2604e-05 2024-09-16T09:37:22.452 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.452 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.452 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:37:22.452 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:21.520267+0000", 2024-09-16T09:37:22.452 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.97819373099999996 2024-09-16T09:37:22.452 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.452 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.452 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:37:22.452 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:21.520277+0000", 2024-09-16T09:37:22.452 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0939e-05 2024-09-16T09:37:22.453 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:37:22.453 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:37:22.453 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:37:22.453 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.453 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.453 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:232750 9.10 9.b4158f10 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e359)", 2024-09-16T09:37:22.453 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:37:20.544912+0000", 2024-09-16T09:37:22.453 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 1.5863686299999999, 2024-09-16T09:37:22.453 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.97432714600000003, 2024-09-16T09:37:22.453 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:37:22.453 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:37:22.453 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "reached pg", 2024-09-16T09:37:22.453 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:37:22.453 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:37:22.454 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:37:22.454 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 232750 2024-09-16T09:37:22.454 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.454 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:37:22.454 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.454 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:37:22.454 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.544912+0000", 2024-09-16T09:37:22.454 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:37:22.454 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.454 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.454 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:37:22.454 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:37:22.455 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T09:37:22.455 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd reweight 0 1 2024-09-16T09:37:22.456 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:37:21 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:37:21] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:37:22.456 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.544912+0000", 2024-09-16T09:37:22.456 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:37:22.456 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.456 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.456 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:37:22.457 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.544913+0000", 2024-09-16T09:37:22.457 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.6899999999999996e-07 2024-09-16T09:37:22.457 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.457 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.457 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:37:22.457 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.544918+0000", 2024-09-16T09:37:22.457 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.8609999999999997e-06 2024-09-16T09:37:22.457 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.457 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.457 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:37:22.457 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.544936+0000", 2024-09-16T09:37:22.457 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.7781000000000001e-05 2024-09-16T09:37:22.457 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.458 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.458 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:37:22.458 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.544945+0000", 2024-09-16T09:37:22.458 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.5480000000000002e-06 2024-09-16T09:37:22.458 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.458 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.458 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:37:22.458 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.545003+0000", 2024-09-16T09:37:22.458 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.8072000000000002e-05 2024-09-16T09:37:22.458 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.458 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.458 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:37:22.458 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.545013+0000", 2024-09-16T09:37:22.458 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.049e-05 2024-09-16T09:37:22.459 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.459 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.459 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:37:22.459 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:21.519228+0000", 2024-09-16T09:37:22.459 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.97421449199999999 2024-09-16T09:37:22.459 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.459 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.459 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:37:22.459 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:21.519240+0000", 2024-09-16T09:37:22.459 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1933000000000001e-05 2024-09-16T09:37:22.459 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:37:22.459 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:37:22.459 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:37:22.459 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.460 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.460 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:232762 9.10 9.b5cb3930 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e359)", 2024-09-16T09:37:22.460 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:37:20.565162+0000", 2024-09-16T09:37:22.460 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 1.566119568, 2024-09-16T09:37:22.460 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.95411015799999999, 2024-09-16T09:37:22.460 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:37:22.460 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:37:22.460 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "reached pg", 2024-09-16T09:37:22.460 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:37:22.460 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:37:22.460 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:37:22.460 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 232762 2024-09-16T09:37:22.460 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.460 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:37:22.461 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.461 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:37:22.461 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.565162+0000", 2024-09-16T09:37:22.461 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:37:22.461 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.461 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.461 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:37:22.461 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.565162+0000", 2024-09-16T09:37:22.461 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:37:22.461 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.461 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.461 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:37:22.461 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.565163+0000", 2024-09-16T09:37:22.462 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.6700000000000002e-07 2024-09-16T09:37:22.462 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.462 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.462 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:37:22.462 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.565167+0000", 2024-09-16T09:37:22.462 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.0929999999999996e-06 2024-09-16T09:37:22.462 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.462 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.462 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:37:22.462 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.565178+0000", 2024-09-16T09:37:22.462 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1055e-05 2024-09-16T09:37:22.462 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.462 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.462 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:37:22.463 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.565183+0000", 2024-09-16T09:37:22.463 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.8489999999999997e-06 2024-09-16T09:37:22.463 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.463 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.463 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:37:22.463 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.565240+0000", 2024-09-16T09:37:22.463 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.7416000000000001e-05 2024-09-16T09:37:22.463 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.463 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.463 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:37:22.463 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.565246+0000", 2024-09-16T09:37:22.463 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.2949999999999999e-06 2024-09-16T09:37:22.463 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.463 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.464 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:37:22.464 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:21.519264+0000", 2024-09-16T09:37:22.464 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.95401728699999999 2024-09-16T09:37:22.464 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:22.464 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:22.464 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:37:22.464 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:21.519272+0000", 2024-09-16T09:37:22.464 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.1960000000000008e-06 2024-09-16T09:37:22.464 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:37:22.464 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:37:22.464 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:37:22.464 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:37:22.464 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:37:22.464 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-09-16T09:37:22.465 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:37:22.527 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:22.528 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:22.528 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:37:22.629 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:37:22.665 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:22.666 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:22.666 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:37:22.767 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:37:22.779 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:22 smithi070 ceph-mon[40691]: 9.14 scrub starts 2024-09-16T09:37:22.779 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:22 smithi070 ceph-mon[40691]: 9.14 scrub ok 2024-09-16T09:37:22.779 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:22 smithi070 ceph-mon[40691]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-16T09:37:22.779 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:22 smithi070 ceph-mon[40691]: Cluster is now healthy 2024-09-16T09:37:22.779 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:22 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/2545459435' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-16T09:37:22.779 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:22 smithi070 ceph-mon[40691]: osdmap e360: 8 total, 8 up, 8 in 2024-09-16T09:37:22.779 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:22 smithi070 ceph-mon[40691]: 9.5 scrub starts 2024-09-16T09:37:22.780 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:22 smithi070 ceph-mon[40691]: 9.5 scrub ok 2024-09-16T09:37:22.780 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:22 smithi070 ceph-mon[40691]: pgmap v645: 178 pgs: 3 peering, 10 unknown, 165 active+clean; 800 MiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.16k op/s 2024-09-16T09:37:22.780 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:22 smithi070 ceph-mon[40691]: 3.0 scrub starts 2024-09-16T09:37:22.780 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:22 smithi070 ceph-mon[40691]: 3.0 scrub ok 2024-09-16T09:37:22.780 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:22 smithi070 ceph-mon[30388]: 9.14 scrub starts 2024-09-16T09:37:22.780 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:22 smithi070 ceph-mon[30388]: 9.14 scrub ok 2024-09-16T09:37:22.780 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:22 smithi070 ceph-mon[30388]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-16T09:37:22.780 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:22 smithi070 ceph-mon[30388]: Cluster is now healthy 2024-09-16T09:37:22.780 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:22 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2545459435' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-16T09:37:22.781 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:22 smithi070 ceph-mon[30388]: osdmap e360: 8 total, 8 up, 8 in 2024-09-16T09:37:22.781 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:22 smithi070 ceph-mon[30388]: 9.5 scrub starts 2024-09-16T09:37:22.781 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:22 smithi070 ceph-mon[30388]: 9.5 scrub ok 2024-09-16T09:37:22.781 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:22 smithi070 ceph-mon[30388]: pgmap v645: 178 pgs: 3 peering, 10 unknown, 165 active+clean; 800 MiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.16k op/s 2024-09-16T09:37:22.781 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:22 smithi070 ceph-mon[30388]: 3.0 scrub starts 2024-09-16T09:37:22.781 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:22 smithi070 ceph-mon[30388]: 3.0 scrub ok 2024-09-16T09:37:22.803 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:22.804 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:22.804 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:37:22.905 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:37:22.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:22 smithi084 ceph-mon[32629]: 9.14 scrub starts 2024-09-16T09:37:22.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:22 smithi084 ceph-mon[32629]: 9.14 scrub ok 2024-09-16T09:37:22.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:22 smithi084 ceph-mon[32629]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-16T09:37:22.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:22 smithi084 ceph-mon[32629]: Cluster is now healthy 2024-09-16T09:37:22.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:22 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/2545459435' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-16T09:37:22.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:22 smithi084 ceph-mon[32629]: osdmap e360: 8 total, 8 up, 8 in 2024-09-16T09:37:22.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:22 smithi084 ceph-mon[32629]: 9.5 scrub starts 2024-09-16T09:37:22.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:22 smithi084 ceph-mon[32629]: 9.5 scrub ok 2024-09-16T09:37:22.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:22 smithi084 ceph-mon[32629]: pgmap v645: 178 pgs: 3 peering, 10 unknown, 165 active+clean; 800 MiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.16k op/s 2024-09-16T09:37:22.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:22 smithi084 ceph-mon[32629]: 3.0 scrub starts 2024-09-16T09:37:22.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:22 smithi084 ceph-mon[32629]: 3.0 scrub ok 2024-09-16T09:37:22.938 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:22.939 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:22.939 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:37:22.970 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:37:22.970 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:37:23.010 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:37:23.010 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:37:23.014 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:37:23.040 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:37:23.047 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:37:23.048 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:37:23.078 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:23.079 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:23.079 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:37:23.082 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:37:23.083 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:37:23.114 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:37:23.115 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:37:23.147 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:37:23.148 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:37:23.180 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:37:23.195 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:37:23.195 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:37:23.202 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.5 dump_ops_in_flight 2024-09-16T09:37:23.212 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:23.212 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:23.213 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:37:23.232 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:37:23.232 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:37:23.314 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:37:23.357 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:23.357 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:23.358 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:37:23.459 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:37:23.493 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:23.494 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:23.495 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:37:23.533 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:37:23 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:37:23.595 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:37:23.620 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:23 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:37:23.620 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:23 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:37:23.620 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:23 smithi070 ceph-mon[30388]: 9.11 scrub starts 2024-09-16T09:37:23.620 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:23 smithi070 ceph-mon[30388]: 9.11 scrub ok 2024-09-16T09:37:23.620 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:23 smithi070 ceph-mon[30388]: 9.17 starting backfill to osd.0 from (292'6477,292'6477] MIN to 358'9538 2024-09-16T09:37:23.620 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:23 smithi070 ceph-mon[30388]: osdmap e361: 8 total, 8 up, 8 in 2024-09-16T09:37:23.620 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:23 smithi070 ceph-mon[30388]: 9.10 starting backfill to osd.0 from (327'8186,327'8186] MIN to 358'8966 2024-09-16T09:37:23.621 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:23 smithi070 ceph-mon[30388]: 9.12 starting backfill to osd.0 from (324'8279,324'8279] MIN to 358'9339 2024-09-16T09:37:23.621 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:23 smithi070 ceph-mon[30388]: 9.16 starting backfill to osd.4 from (297'6808,297'6808] MIN to 358'9584 2024-09-16T09:37:23.621 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:23 smithi070 ceph-mon[30388]: 9.19 starting backfill to osd.3 from (0'0,0'0] MAX to 358'10704 2024-09-16T09:37:23.621 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:23 smithi070 ceph-mon[30388]: 9.18 starting backfill to osd.5 from (317'7885,317'7885] MIN to 358'9498 2024-09-16T09:37:23.621 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:23 smithi070 ceph-mon[30388]: 9.15 starting backfill to osd.3 from (0'0,0'0] MAX to 358'9437 2024-09-16T09:37:23.621 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:23 smithi070 ceph-mon[30388]: 9.17 starting backfill to osd.1 from (0'0,0'0] MAX to 358'9538 2024-09-16T09:37:23.621 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:23 smithi070 ceph-mon[30388]: 9.19 starting backfill to osd.6 from (299'6841,299'6841] MIN to 358'10704 2024-09-16T09:37:23.621 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:23 smithi070 ceph-mon[30388]: 9.10 starting backfill to osd.4 from (281'5546,281'5546] MIN to 358'8966 2024-09-16T09:37:23.621 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:23 smithi070 ceph-mon[30388]: 9.13 starting backfill to osd.5 from (313'7550,313'7550] MIN to 358'9183 2024-09-16T09:37:23.621 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:23 smithi070 ceph-mon[30388]: 9.18 starting backfill to osd.6 from (281'5505,281'5505] MIN to 358'9498 2024-09-16T09:37:23.621 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:23 smithi070 ceph-mon[30388]: 9.13 starting backfill to osd.6 from (313'7550,313'7550] MIN to 358'9183 2024-09-16T09:37:23.621 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:23 smithi070 ceph-mon[30388]: 9.15 starting backfill to osd.7 from (0'0,0'0] MAX to 358'9437 2024-09-16T09:37:23.621 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:23 smithi070 ceph-mon[30388]: 9.12 starting backfill to osd.5 from (281'5607,281'5607] MIN to 358'9339 2024-09-16T09:37:23.621 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:23 smithi070 ceph-mon[30388]: 9.10 starting backfill to osd.7 from (0'0,0'0] MAX to 358'8966 2024-09-16T09:37:23.622 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:23 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1863643931' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:37:23.622 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:23 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:37:23.622 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:23 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:37:23.622 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:23 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:37:23.622 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:23 smithi070 ceph-mon[40691]: 9.11 scrub starts 2024-09-16T09:37:23.622 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:23 smithi070 ceph-mon[40691]: 9.11 scrub ok 2024-09-16T09:37:23.622 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:23 smithi070 ceph-mon[40691]: 9.17 starting backfill to osd.0 from (292'6477,292'6477] MIN to 358'9538 2024-09-16T09:37:23.622 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:23 smithi070 ceph-mon[40691]: osdmap e361: 8 total, 8 up, 8 in 2024-09-16T09:37:23.622 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:23 smithi070 ceph-mon[40691]: 9.10 starting backfill to osd.0 from (327'8186,327'8186] MIN to 358'8966 2024-09-16T09:37:23.622 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:23 smithi070 ceph-mon[40691]: 9.12 starting backfill to osd.0 from (324'8279,324'8279] MIN to 358'9339 2024-09-16T09:37:23.623 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:23 smithi070 ceph-mon[40691]: 9.16 starting backfill to osd.4 from (297'6808,297'6808] MIN to 358'9584 2024-09-16T09:37:23.623 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:23 smithi070 ceph-mon[40691]: 9.19 starting backfill to osd.3 from (0'0,0'0] MAX to 358'10704 2024-09-16T09:37:23.623 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:23 smithi070 ceph-mon[40691]: 9.18 starting backfill to osd.5 from (317'7885,317'7885] MIN to 358'9498 2024-09-16T09:37:23.623 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:23 smithi070 ceph-mon[40691]: 9.15 starting backfill to osd.3 from (0'0,0'0] MAX to 358'9437 2024-09-16T09:37:23.623 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:23 smithi070 ceph-mon[40691]: 9.17 starting backfill to osd.1 from (0'0,0'0] MAX to 358'9538 2024-09-16T09:37:23.623 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:23 smithi070 ceph-mon[40691]: 9.19 starting backfill to osd.6 from (299'6841,299'6841] MIN to 358'10704 2024-09-16T09:37:23.623 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:23 smithi070 ceph-mon[40691]: 9.10 starting backfill to osd.4 from (281'5546,281'5546] MIN to 358'8966 2024-09-16T09:37:23.623 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:23 smithi070 ceph-mon[40691]: 9.13 starting backfill to osd.5 from (313'7550,313'7550] MIN to 358'9183 2024-09-16T09:37:23.623 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:23 smithi070 ceph-mon[40691]: 9.18 starting backfill to osd.6 from (281'5505,281'5505] MIN to 358'9498 2024-09-16T09:37:23.623 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:23 smithi070 ceph-mon[40691]: 9.13 starting backfill to osd.6 from (313'7550,313'7550] MIN to 358'9183 2024-09-16T09:37:23.623 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:23 smithi070 ceph-mon[40691]: 9.15 starting backfill to osd.7 from (0'0,0'0] MAX to 358'9437 2024-09-16T09:37:23.623 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:23 smithi070 ceph-mon[40691]: 9.12 starting backfill to osd.5 from (281'5607,281'5607] MIN to 358'9339 2024-09-16T09:37:23.623 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:23 smithi070 ceph-mon[40691]: 9.10 starting backfill to osd.7 from (0'0,0'0] MAX to 358'8966 2024-09-16T09:37:23.623 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:23 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1863643931' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:37:23.624 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:23 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:37:23.637 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:23.639 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:23.639 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:37:23.656 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:37:23.741 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:37:23.774 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:23.775 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:23.775 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:37:23.795 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:23 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:37:23.796 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:23 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:37:23.796 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:23 smithi084 ceph-mon[32629]: 9.11 scrub starts 2024-09-16T09:37:23.796 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:23 smithi084 ceph-mon[32629]: 9.11 scrub ok 2024-09-16T09:37:23.796 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:23 smithi084 ceph-mon[32629]: 9.17 starting backfill to osd.0 from (292'6477,292'6477] MIN to 358'9538 2024-09-16T09:37:23.796 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:23 smithi084 ceph-mon[32629]: osdmap e361: 8 total, 8 up, 8 in 2024-09-16T09:37:23.796 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:23 smithi084 ceph-mon[32629]: 9.10 starting backfill to osd.0 from (327'8186,327'8186] MIN to 358'8966 2024-09-16T09:37:23.796 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:23 smithi084 ceph-mon[32629]: 9.12 starting backfill to osd.0 from (324'8279,324'8279] MIN to 358'9339 2024-09-16T09:37:23.796 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:23 smithi084 ceph-mon[32629]: 9.16 starting backfill to osd.4 from (297'6808,297'6808] MIN to 358'9584 2024-09-16T09:37:23.796 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:23 smithi084 ceph-mon[32629]: 9.19 starting backfill to osd.3 from (0'0,0'0] MAX to 358'10704 2024-09-16T09:37:23.796 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:23 smithi084 ceph-mon[32629]: 9.18 starting backfill to osd.5 from (317'7885,317'7885] MIN to 358'9498 2024-09-16T09:37:23.797 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:23 smithi084 ceph-mon[32629]: 9.15 starting backfill to osd.3 from (0'0,0'0] MAX to 358'9437 2024-09-16T09:37:23.797 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:23 smithi084 ceph-mon[32629]: 9.17 starting backfill to osd.1 from (0'0,0'0] MAX to 358'9538 2024-09-16T09:37:23.797 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:23 smithi084 ceph-mon[32629]: 9.19 starting backfill to osd.6 from (299'6841,299'6841] MIN to 358'10704 2024-09-16T09:37:23.797 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:23 smithi084 ceph-mon[32629]: 9.10 starting backfill to osd.4 from (281'5546,281'5546] MIN to 358'8966 2024-09-16T09:37:23.797 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:23 smithi084 ceph-mon[32629]: 9.13 starting backfill to osd.5 from (313'7550,313'7550] MIN to 358'9183 2024-09-16T09:37:23.797 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:23 smithi084 ceph-mon[32629]: 9.18 starting backfill to osd.6 from (281'5505,281'5505] MIN to 358'9498 2024-09-16T09:37:23.797 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:23 smithi084 ceph-mon[32629]: 9.13 starting backfill to osd.6 from (313'7550,313'7550] MIN to 358'9183 2024-09-16T09:37:23.797 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:23 smithi084 ceph-mon[32629]: 9.15 starting backfill to osd.7 from (0'0,0'0] MAX to 358'9437 2024-09-16T09:37:23.797 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:23 smithi084 ceph-mon[32629]: 9.12 starting backfill to osd.5 from (281'5607,281'5607] MIN to 358'9339 2024-09-16T09:37:23.797 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:23 smithi084 ceph-mon[32629]: 9.10 starting backfill to osd.7 from (0'0,0'0] MAX to 358'8966 2024-09-16T09:37:23.797 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:23 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1863643931' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:37:23.798 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:23 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:37:23.876 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:37:23.913 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:23.913 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:23.914 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:37:24.015 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:37:24.054 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:24.055 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:24.055 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:37:24.156 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:37:24.189 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:24.190 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:24.190 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:37:24.291 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:37:24.318 INFO:teuthology.orchestra.run.smithi084.stdout:ERROR: (22) Invalid argument 2024-09-16T09:37:24.318 INFO:teuthology.orchestra.run.smithi084.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:37:24.325 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:24.325 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:24.326 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:37:24.427 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:37:24.464 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:24.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:24.465 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:37:24.466 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd set noscrub 2024-09-16T09:37:24.542 INFO:teuthology.orchestra.run.smithi070.stderr:reweighted osd.0 to 1 (10000) 2024-09-16T09:37:24.565 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:37:24.611 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:24.612 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:24.612 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:37:24.713 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:37:24.742 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:24.742 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:24.743 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:37:24.808 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:24 smithi084 ceph-mon[32629]: pgmap v647: 178 pgs: 3 active+remapped+backfilling, 3 peering, 3 unknown, 2 active+remapped+backfill_toofull, 167 active+clean; 657 MiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 2674/32112 objects misplaced (8.327%) 2024-09-16T09:37:24.809 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:24 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3479283208' entity='client.admin' cmd={"prefix": "osd reweight", "id": 0, "weight": 1.0} : dispatch 2024-09-16T09:37:24.844 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:37:24.866 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:24.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:24.867 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:37:24.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:24 smithi070 ceph-mon[30388]: pgmap v647: 178 pgs: 3 active+remapped+backfilling, 3 peering, 3 unknown, 2 active+remapped+backfill_toofull, 167 active+clean; 657 MiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 2674/32112 objects misplaced (8.327%) 2024-09-16T09:37:24.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:24 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3479283208' entity='client.admin' cmd={"prefix": "osd reweight", "id": 0, "weight": 1.0} : dispatch 2024-09-16T09:37:24.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:24 smithi070 ceph-mon[40691]: pgmap v647: 178 pgs: 3 active+remapped+backfilling, 3 peering, 3 unknown, 2 active+remapped+backfill_toofull, 167 active+clean; 657 MiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 2674/32112 objects misplaced (8.327%) 2024-09-16T09:37:24.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:24 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3479283208' entity='client.admin' cmd={"prefix": "osd reweight", "id": 0, "weight": 1.0} : dispatch 2024-09-16T09:37:24.968 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:37:24.992 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:24.992 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:24.993 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:37:25.093 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:37:25.115 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:25.116 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:25.116 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:37:25.171 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.5 dump_blocked_ops 2024-09-16T09:37:25.217 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:37:25.238 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:25.239 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:25.239 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:37:25.340 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:37:25.362 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:25.362 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:25.363 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:37:25.440 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:37:25.464 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:37:25.490 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:25.490 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:25.491 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:37:25.591 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:37:25.623 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:25.624 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:25.625 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:37:25.726 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:37:25.747 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:25.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:25.749 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:37:25.796 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:25 smithi084 ceph-mon[32629]: Health check failed: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-16T09:37:25.796 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:25 smithi084 ceph-mon[32629]: 9.19 scrub starts 2024-09-16T09:37:25.796 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:25 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3479283208' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-16T09:37:25.797 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:25 smithi084 ceph-mon[32629]: osdmap e362: 8 total, 8 up, 8 in 2024-09-16T09:37:25.797 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:25 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:37:25.797 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:25 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:37:25.850 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:37:25.873 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:25.874 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:25.874 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:37:25.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:25 smithi070 ceph-mon[30388]: Health check failed: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-16T09:37:25.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:25 smithi070 ceph-mon[30388]: 9.19 scrub starts 2024-09-16T09:37:25.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:25 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3479283208' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-16T09:37:25.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:25 smithi070 ceph-mon[30388]: osdmap e362: 8 total, 8 up, 8 in 2024-09-16T09:37:25.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:25 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:37:25.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:25 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:37:25.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:25 smithi070 ceph-mon[40691]: Health check failed: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-16T09:37:25.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:25 smithi070 ceph-mon[40691]: 9.19 scrub starts 2024-09-16T09:37:25.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:25 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3479283208' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-16T09:37:25.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:25 smithi070 ceph-mon[40691]: osdmap e362: 8 total, 8 up, 8 in 2024-09-16T09:37:25.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:25 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:37:25.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:25 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:37:25.975 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:37:25.997 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:25.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:25.998 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:37:26.047 INFO:teuthology.orchestra.run.smithi084.stdout:ERROR: (22) Invalid argument 2024-09-16T09:37:26.047 INFO:teuthology.orchestra.run.smithi084.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:37:26.098 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:37:26.120 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:26.121 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:26.121 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:37:26.222 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:37:26.244 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:26.245 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:26.245 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:37:26.346 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:37:26.370 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:26.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:26.371 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:37:26.472 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:37:26.497 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:26.497 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:26.498 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:37:26.598 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:37:26.621 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:26.621 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:26.622 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:37:26.652 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:26 smithi084 ceph-mon[32629]: 9.10 scrub starts 2024-09-16T09:37:26.652 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:26 smithi084 ceph-mon[32629]: 9.10 scrub ok 2024-09-16T09:37:26.652 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:26 smithi084 ceph-mon[32629]: 9.13 scrub starts 2024-09-16T09:37:26.652 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:26 smithi084 ceph-mon[32629]: 9.13 scrub ok 2024-09-16T09:37:26.652 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:26 smithi084 ceph-mon[32629]: osdmap e363: 8 total, 8 up, 8 in 2024-09-16T09:37:26.652 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:26 smithi084 ceph-mon[32629]: pgmap v650: 178 pgs: 5 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 170 active+clean; 598 MiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 3536/29268 objects misplaced (12.081%) 2024-09-16T09:37:26.652 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:26 smithi084 ceph-mon[32629]: 9.19 scrub starts 2024-09-16T09:37:26.652 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:26 smithi084 ceph-mon[32629]: 9.19 scrub ok 2024-09-16T09:37:26.652 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:26 smithi084 ceph-mon[32629]: 4.1f scrub starts 2024-09-16T09:37:26.653 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:26 smithi084 ceph-mon[32629]: 4.1f scrub ok 2024-09-16T09:37:26.688 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.5 dump_historic_ops 2024-09-16T09:37:26.723 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:37:26.757 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:26.757 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:26.758 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:37:26.858 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:37:26.892 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:26.893 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:26.893 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:37:26.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:26 smithi070 ceph-mon[30388]: 9.10 scrub starts 2024-09-16T09:37:26.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:26 smithi070 ceph-mon[30388]: 9.10 scrub ok 2024-09-16T09:37:26.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:26 smithi070 ceph-mon[30388]: 9.13 scrub starts 2024-09-16T09:37:26.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:26 smithi070 ceph-mon[30388]: 9.13 scrub ok 2024-09-16T09:37:26.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:26 smithi070 ceph-mon[30388]: osdmap e363: 8 total, 8 up, 8 in 2024-09-16T09:37:26.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:26 smithi070 ceph-mon[30388]: pgmap v650: 178 pgs: 5 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 170 active+clean; 598 MiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 3536/29268 objects misplaced (12.081%) 2024-09-16T09:37:26.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:26 smithi070 ceph-mon[30388]: 9.19 scrub starts 2024-09-16T09:37:26.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:26 smithi070 ceph-mon[30388]: 9.19 scrub ok 2024-09-16T09:37:26.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:26 smithi070 ceph-mon[30388]: 4.1f scrub starts 2024-09-16T09:37:26.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:26 smithi070 ceph-mon[30388]: 4.1f scrub ok 2024-09-16T09:37:26.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:26 smithi070 ceph-mon[40691]: 9.10 scrub starts 2024-09-16T09:37:26.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:26 smithi070 ceph-mon[40691]: 9.10 scrub ok 2024-09-16T09:37:26.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:26 smithi070 ceph-mon[40691]: 9.13 scrub starts 2024-09-16T09:37:26.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:26 smithi070 ceph-mon[40691]: 9.13 scrub ok 2024-09-16T09:37:26.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:26 smithi070 ceph-mon[40691]: osdmap e363: 8 total, 8 up, 8 in 2024-09-16T09:37:26.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:26 smithi070 ceph-mon[40691]: pgmap v650: 178 pgs: 5 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 170 active+clean; 598 MiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 3536/29268 objects misplaced (12.081%) 2024-09-16T09:37:26.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:26 smithi070 ceph-mon[40691]: 9.19 scrub starts 2024-09-16T09:37:26.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:26 smithi070 ceph-mon[40691]: 9.19 scrub ok 2024-09-16T09:37:26.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:26 smithi070 ceph-mon[40691]: 4.1f scrub starts 2024-09-16T09:37:26.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:26 smithi070 ceph-mon[40691]: 4.1f scrub ok 2024-09-16T09:37:26.994 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:37:27.022 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:27.022 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:27.023 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:37:27.123 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:37:27.127 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:37:27.151 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:27.151 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:27.152 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:37:27.253 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:37:27.280 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:27.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:27.281 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:37:27.382 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:37:27.411 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:27.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:27.413 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:37:27.513 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:37:27.546 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:27.546 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:27.546 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:37:27.551 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:27 smithi084 ceph-mon[32629]: 9.16 scrub starts 2024-09-16T09:37:27.551 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:27 smithi084 ceph-mon[32629]: 9.16 scrub ok 2024-09-16T09:37:27.551 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:27 smithi084 ceph-mon[32629]: osdmap e364: 8 total, 8 up, 8 in 2024-09-16T09:37:27.646 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:37:27.681 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:27.681 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:27.682 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:37:27.766 INFO:teuthology.orchestra.run.smithi084.stdout:ERROR: (22) Invalid argument 2024-09-16T09:37:27.767 INFO:teuthology.orchestra.run.smithi084.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:37:27.783 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:37:27.814 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:27.815 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:27.816 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:37:27.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:27 smithi084 ceph-mon[32629]: 5.1e scrub starts 2024-09-16T09:37:27.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:27 smithi084 ceph-mon[32629]: 5.1e scrub ok 2024-09-16T09:37:27.917 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:37:27.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:27 smithi070 ceph-mon[30388]: 9.16 scrub starts 2024-09-16T09:37:27.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:27 smithi070 ceph-mon[30388]: 9.16 scrub ok 2024-09-16T09:37:27.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:27 smithi070 ceph-mon[30388]: osdmap e364: 8 total, 8 up, 8 in 2024-09-16T09:37:27.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:27 smithi070 ceph-mon[30388]: 5.1e scrub starts 2024-09-16T09:37:27.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:27 smithi070 ceph-mon[30388]: 5.1e scrub ok 2024-09-16T09:37:27.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:27 smithi070 ceph-mon[40691]: 9.16 scrub starts 2024-09-16T09:37:27.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:27 smithi070 ceph-mon[40691]: 9.16 scrub ok 2024-09-16T09:37:27.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:27 smithi070 ceph-mon[40691]: osdmap e364: 8 total, 8 up, 8 in 2024-09-16T09:37:27.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:27 smithi070 ceph-mon[40691]: 5.1e scrub starts 2024-09-16T09:37:27.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:27 smithi070 ceph-mon[40691]: 5.1e scrub ok 2024-09-16T09:37:27.946 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:27.947 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:27.947 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:37:28.048 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:37:28.078 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:28.079 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:28.079 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:37:28.180 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:37:28.212 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:28.213 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:28.213 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:37:28.314 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:37:28.341 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:28.342 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:28.342 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:37:28.443 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:37:28.471 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:28.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:28.472 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:37:28.548 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.6 dump_ops_in_flight 2024-09-16T09:37:28.572 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:37:28.599 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:28.599 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:28.600 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:37:28.700 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:37:28.728 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:28.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:28.729 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:37:28.830 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:37:28.874 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:28.874 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:28.875 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:37:28.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:28 smithi084 ceph-mon[32629]: 9.10 scrub starts 2024-09-16T09:37:28.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:28 smithi084 ceph-mon[32629]: 9.10 scrub ok 2024-09-16T09:37:28.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:28 smithi084 ceph-mon[32629]: 9.15 scrub starts 2024-09-16T09:37:28.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:28 smithi084 ceph-mon[32629]: 9.15 scrub ok 2024-09-16T09:37:28.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:28 smithi084 ceph-mon[32629]: pgmap v652: 178 pgs: 5 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 170 active+clean; 577 MiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 981 B/s rd, 0 op/s; 3536/28257 objects misplaced (12.514%) 2024-09-16T09:37:28.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:28 smithi084 ceph-mon[32629]: 4.a scrub starts 2024-09-16T09:37:28.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:28 smithi084 ceph-mon[32629]: 4.a scrub ok 2024-09-16T09:37:28.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:28 smithi070 ceph-mon[30388]: 9.10 scrub starts 2024-09-16T09:37:28.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:28 smithi070 ceph-mon[30388]: 9.10 scrub ok 2024-09-16T09:37:28.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:28 smithi070 ceph-mon[30388]: 9.15 scrub starts 2024-09-16T09:37:28.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:28 smithi070 ceph-mon[30388]: 9.15 scrub ok 2024-09-16T09:37:28.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:28 smithi070 ceph-mon[30388]: pgmap v652: 178 pgs: 5 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 170 active+clean; 577 MiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 981 B/s rd, 0 op/s; 3536/28257 objects misplaced (12.514%) 2024-09-16T09:37:28.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:28 smithi070 ceph-mon[30388]: 4.a scrub starts 2024-09-16T09:37:28.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:28 smithi070 ceph-mon[30388]: 4.a scrub ok 2024-09-16T09:37:28.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:28 smithi070 ceph-mon[40691]: 9.10 scrub starts 2024-09-16T09:37:28.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:28 smithi070 ceph-mon[40691]: 9.10 scrub ok 2024-09-16T09:37:28.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:28 smithi070 ceph-mon[40691]: 9.15 scrub starts 2024-09-16T09:37:28.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:28 smithi070 ceph-mon[40691]: 9.15 scrub ok 2024-09-16T09:37:28.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:28 smithi070 ceph-mon[40691]: pgmap v652: 178 pgs: 5 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 170 active+clean; 577 MiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 981 B/s rd, 0 op/s; 3536/28257 objects misplaced (12.514%) 2024-09-16T09:37:28.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:28 smithi070 ceph-mon[40691]: 4.a scrub starts 2024-09-16T09:37:28.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:28 smithi070 ceph-mon[40691]: 4.a scrub ok 2024-09-16T09:37:28.967 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:37:28.975 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:37:29.003 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:29.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:29.004 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:37:29.105 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:37:29.148 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:29.149 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:29.149 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:37:29.250 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:37:29.284 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:29.285 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:29.286 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:37:29.386 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:37:29.423 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:29.424 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:29.424 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:37:29.525 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:37:29.567 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:29.568 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:29.568 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:37:29.595 INFO:teuthology.orchestra.run.smithi084.stdout:ERROR: (22) Invalid argument 2024-09-16T09:37:29.595 INFO:teuthology.orchestra.run.smithi084.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:37:29.668 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:37:29.705 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:29.706 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:29.706 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:37:29.807 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:37:29.842 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:29.843 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:29.843 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:37:29.944 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:37:29.981 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:29.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:29.982 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:37:30.082 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:37:30.121 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd reweight 1 1 2024-09-16T09:37:30.135 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:30.136 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:30.136 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:37:30.238 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:37:30.272 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:30.272 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:30.273 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:37:30.374 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:37:30.391 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.6 dump_blocked_ops 2024-09-16T09:37:30.407 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:37:30.422 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:30.423 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:30.424 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:37:30.477 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Removed 121708 objects 2024-09-16T09:37:30.477 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Clean up completed and total clean up time :149.752 2024-09-16T09:37:30.525 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:37:30.546 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:30.546 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:30.547 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:37:30.596 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:30 smithi070 ceph-mon[30388]: osdmap e365: 8 total, 8 up, 8 in 2024-09-16T09:37:30.596 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:30 smithi070 ceph-mon[30388]: 4.19 scrub starts 2024-09-16T09:37:30.596 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:30 smithi070 ceph-mon[30388]: 4.19 scrub ok 2024-09-16T09:37:30.596 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:30 smithi070 ceph-mon[30388]: 4.8 scrub starts 2024-09-16T09:37:30.596 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:30 smithi070 ceph-mon[30388]: 4.8 scrub ok 2024-09-16T09:37:30.597 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:30 smithi070 ceph-mon[40691]: osdmap e365: 8 total, 8 up, 8 in 2024-09-16T09:37:30.597 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:30 smithi070 ceph-mon[40691]: 4.19 scrub starts 2024-09-16T09:37:30.597 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:30 smithi070 ceph-mon[40691]: 4.19 scrub ok 2024-09-16T09:37:30.597 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:30 smithi070 ceph-mon[40691]: 4.8 scrub starts 2024-09-16T09:37:30.597 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:30 smithi070 ceph-mon[40691]: 4.8 scrub ok 2024-09-16T09:37:30.647 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:37:30.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:30 smithi084 ceph-mon[32629]: osdmap e365: 8 total, 8 up, 8 in 2024-09-16T09:37:30.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:30 smithi084 ceph-mon[32629]: 4.19 scrub starts 2024-09-16T09:37:30.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:30 smithi084 ceph-mon[32629]: 4.19 scrub ok 2024-09-16T09:37:30.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:30 smithi084 ceph-mon[32629]: 4.8 scrub starts 2024-09-16T09:37:30.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:30 smithi084 ceph-mon[32629]: 4.8 scrub ok 2024-09-16T09:37:30.666 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:37:30.668 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:30.669 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:30.669 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:37:30.770 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:37:30.791 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:30.791 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:30.792 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:37:30.893 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:37:30.914 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:30.915 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:30.916 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:37:31.016 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:37:31.018 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:37:31.039 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:31.040 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:31.040 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:37:31.141 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:37:31.144 INFO:teuthology.orchestra.run.smithi084.stdout:ERROR: (22) Invalid argument 2024-09-16T09:37:31.144 INFO:teuthology.orchestra.run.smithi084.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:37:31.166 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:31.167 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:31.167 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:37:31.268 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:37:31.290 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:31.290 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:31.291 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:37:31.328 INFO:teuthology.orchestra.run.smithi070.stderr:noscrub is set 2024-09-16T09:37:31.392 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:37:31.413 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:31.414 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:31.414 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:37:31.420 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:31 smithi070 ceph-mon[40691]: pgmap v654: 178 pgs: 3 active+remapped+backfill_toofull, 175 active+clean; 259 MiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 984 B/s rd, 0 op/s; 656/13020 objects misplaced (5.038%) 2024-09-16T09:37:31.420 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:31 smithi070 ceph-mon[40691]: 3.c scrub starts 2024-09-16T09:37:31.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:31 smithi070 ceph-mon[40691]: 3.c scrub ok 2024-09-16T09:37:31.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:31 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/4233888721' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:37:31.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:31 smithi070 ceph-mon[30388]: pgmap v654: 178 pgs: 3 active+remapped+backfill_toofull, 175 active+clean; 259 MiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 984 B/s rd, 0 op/s; 656/13020 objects misplaced (5.038%) 2024-09-16T09:37:31.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:31 smithi070 ceph-mon[30388]: 3.c scrub starts 2024-09-16T09:37:31.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:31 smithi070 ceph-mon[30388]: 3.c scrub ok 2024-09-16T09:37:31.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:31 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/4233888721' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:37:31.515 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:37:31.537 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:31.537 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:31.538 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:37:31.638 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:37:31.660 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:31.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:31 smithi084 ceph-mon[32629]: pgmap v654: 178 pgs: 3 active+remapped+backfill_toofull, 175 active+clean; 259 MiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 984 B/s rd, 0 op/s; 656/13020 objects misplaced (5.038%) 2024-09-16T09:37:31.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:31 smithi084 ceph-mon[32629]: 3.c scrub starts 2024-09-16T09:37:31.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:31 smithi084 ceph-mon[32629]: 3.c scrub ok 2024-09-16T09:37:31.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:31 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/4233888721' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:37:31.661 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:31.661 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:37:31.762 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:37:31.784 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:31.784 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:31.785 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:37:31.800 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.6 dump_historic_ops 2024-09-16T09:37:31.886 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:37:31.909 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:31.909 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:31.909 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:37:32.010 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:37:32.031 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:32.032 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:32.032 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:37:32.085 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:37:32.133 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:37:32.154 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:32.154 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:32.155 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:37:32.171 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:37:31 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:37:31] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:37:32.255 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:37:32.278 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:32.278 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:32.278 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:37:32.335 INFO:teuthology.orchestra.run.smithi070.stderr:reweighted osd.1 to 1 (10000) 2024-09-16T09:37:32.380 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:37:32.397 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T09:37:32.402 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:32.402 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:32.402 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:37:32.493 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:32 smithi084 ceph-mon[32629]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:37:32.493 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:32 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/4233888721' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-16T09:37:32.493 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:32 smithi084 ceph-mon[32629]: osdmap e366: 8 total, 8 up, 8 in 2024-09-16T09:37:32.493 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:32 smithi084 ceph-mon[32629]: 4.13 scrub starts 2024-09-16T09:37:32.493 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:32 smithi084 ceph-mon[32629]: 4.13 scrub ok 2024-09-16T09:37:32.493 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:32 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/572803857' entity='client.admin' cmd={"prefix": "osd reweight", "id": 1, "weight": 1.0} : dispatch 2024-09-16T09:37:32.503 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:37:32.525 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:32.526 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:32.526 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:37:32.600 INFO:teuthology.orchestra.run.smithi084.stdout:ERROR: (22) Invalid argument 2024-09-16T09:37:32.600 INFO:teuthology.orchestra.run.smithi084.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:37:32.627 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:37:32.648 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:32.649 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:32.649 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:37:32.662 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:32 smithi070 ceph-mon[30388]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:37:32.662 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:32 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/4233888721' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-16T09:37:32.662 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:32 smithi070 ceph-mon[30388]: osdmap e366: 8 total, 8 up, 8 in 2024-09-16T09:37:32.662 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:32 smithi070 ceph-mon[30388]: 4.13 scrub starts 2024-09-16T09:37:32.662 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:32 smithi070 ceph-mon[30388]: 4.13 scrub ok 2024-09-16T09:37:32.662 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:32 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/572803857' entity='client.admin' cmd={"prefix": "osd reweight", "id": 1, "weight": 1.0} : dispatch 2024-09-16T09:37:32.662 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:32 smithi070 ceph-mon[40691]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:37:32.663 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:32 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/4233888721' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-16T09:37:32.663 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:32 smithi070 ceph-mon[40691]: osdmap e366: 8 total, 8 up, 8 in 2024-09-16T09:37:32.663 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:32 smithi070 ceph-mon[40691]: 4.13 scrub starts 2024-09-16T09:37:32.663 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:32 smithi070 ceph-mon[40691]: 4.13 scrub ok 2024-09-16T09:37:32.663 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:32 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/572803857' entity='client.admin' cmd={"prefix": "osd reweight", "id": 1, "weight": 1.0} : dispatch 2024-09-16T09:37:32.751 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:37:32.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:32.774 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:32.774 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:37:32.875 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:37:32.896 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:32.897 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:32.897 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:37:32.998 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:37:33.016 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd reweight 2 1 2024-09-16T09:37:33.020 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:33.020 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:33.021 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:37:33.089 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:37:33.122 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:37:33.142 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:33.143 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:33.143 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:37:33.225 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.7 dump_ops_in_flight 2024-09-16T09:37:33.243 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:37:33.266 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:33.267 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:33.267 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:37:33.368 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:37:33.389 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:33.389 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:33.390 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:37:33.481 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:37:33.491 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:37:33.500 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:33 smithi070 ceph-mon[30388]: pgmap v656: 178 pgs: 3 active+remapped+backfill_toofull, 175 active+clean; 259 MiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 656/13020 objects misplaced (5.038%) 2024-09-16T09:37:33.500 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:33 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/572803857' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-16T09:37:33.500 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:33 smithi070 ceph-mon[30388]: osdmap e367: 8 total, 8 up, 8 in 2024-09-16T09:37:33.501 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:33 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:37:33.501 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:33 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:37:33.501 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:33 smithi070 ceph-mon[30388]: osdmap e368: 8 total, 8 up, 8 in 2024-09-16T09:37:33.501 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:33 smithi070 ceph-mon[40691]: pgmap v656: 178 pgs: 3 active+remapped+backfill_toofull, 175 active+clean; 259 MiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 656/13020 objects misplaced (5.038%) 2024-09-16T09:37:33.501 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:33 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/572803857' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-16T09:37:33.501 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:33 smithi070 ceph-mon[40691]: osdmap e367: 8 total, 8 up, 8 in 2024-09-16T09:37:33.501 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:33 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:37:33.501 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:33 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:37:33.501 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:33 smithi070 ceph-mon[40691]: osdmap e368: 8 total, 8 up, 8 in 2024-09-16T09:37:33.512 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:33.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:33.513 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:37:33.602 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:37:33 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:37:33.602 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:33 smithi084 ceph-mon[32629]: pgmap v656: 178 pgs: 3 active+remapped+backfill_toofull, 175 active+clean; 259 MiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 656/13020 objects misplaced (5.038%) 2024-09-16T09:37:33.602 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:33 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/572803857' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-16T09:37:33.602 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:33 smithi084 ceph-mon[32629]: osdmap e367: 8 total, 8 up, 8 in 2024-09-16T09:37:33.602 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:33 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:37:33.602 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:33 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:37:33.603 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:33 smithi084 ceph-mon[32629]: osdmap e368: 8 total, 8 up, 8 in 2024-09-16T09:37:33.608 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:37:33.608 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:37:33.614 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:37:33.619 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:37:33.620 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:37:33.635 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:33.635 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:33.636 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:37:33.636 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:37:33.636 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:37:33.653 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:37:33.653 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:37:33.676 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:37:33.676 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:37:33.694 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:37:33.698 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:37:33.698 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:37:33.725 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:37:33.725 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:37:33.736 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:37:33.747 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:37:33.747 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:37:33.759 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:33.760 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:33.760 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:37:33.861 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:37:33.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:33.885 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:33.886 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:37:33.974 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-09-16T09:37:33.975 INFO:teuthology.orchestra.run.smithi084.stdout: "ops": [], 2024-09-16T09:37:33.975 INFO:teuthology.orchestra.run.smithi084.stdout: "num_ops": 0 2024-09-16T09:37:33.975 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-09-16T09:37:33.987 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:37:34.008 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:34.009 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:34.009 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:37:34.110 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:37:34.131 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:34.132 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:34.132 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:37:34.233 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:37:34.254 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:34.255 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:34.255 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:37:34.281 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-16T09:37:34.357 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:37:34.378 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:34.379 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:34.379 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:37:34.480 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:37:34.500 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:34.501 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:34.501 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:37:34.602 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:37:34.623 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:34.624 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:34.624 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:37:34.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:34 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:37:34.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:34 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/2442611531' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:37:34.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:34 smithi084 ceph-mon[32629]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 3 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-16T09:37:34.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:34 smithi084 ceph-mon[32629]: osdmap e369: 8 total, 8 up, 8 in 2024-09-16T09:37:34.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:34 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:37:34.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:34 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2442611531' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:37:34.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:34 smithi070 ceph-mon[30388]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 3 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-16T09:37:34.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:34 smithi070 ceph-mon[30388]: osdmap e369: 8 total, 8 up, 8 in 2024-09-16T09:37:34.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:34 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:37:34.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:34 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/2442611531' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:37:34.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:34 smithi070 ceph-mon[40691]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 3 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-16T09:37:34.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:34 smithi070 ceph-mon[40691]: osdmap e369: 8 total, 8 up, 8 in 2024-09-16T09:37:34.710 INFO:tasks.cephadm.ceph_manager.ceph:removing pool_name unique_pool_2 2024-09-16T09:37:34.710 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd pool rm unique_pool_2 unique_pool_2 --yes-i-really-really-mean-it 2024-09-16T09:37:34.725 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:37:34.746 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:34.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:34.747 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:37:34.798 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.7 dump_blocked_ops 2024-09-16T09:37:34.847 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:37:34.869 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:34.869 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:34.870 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:37:34.970 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:37:34.992 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:34.993 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:34.993 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:37:35.046 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:37:35.094 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:37:35.115 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:35.116 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:35.116 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:37:35.217 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:37:35.238 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:35.239 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:35.239 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:37:35.301 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:37:35.302 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:37:35.340 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:37:35.365 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:35.365 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:35.366 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:37:35.466 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:37:35.488 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:35.489 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:35.489 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:37:35.491 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:35 smithi084 ceph-mon[32629]: pgmap v659: 178 pgs: 4 peering, 3 active+remapped+backfill_toofull, 171 active+clean; 32 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 829 op/s; 156/2088 objects misplaced (7.471%) 2024-09-16T09:37:35.491 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:35 smithi084 ceph-mon[32629]: 9.d starting backfill to osd.1 from (0'0,0'0] MAX to 365'15180 2024-09-16T09:37:35.590 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:37:35.591 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:35 smithi070 ceph-mon[30388]: pgmap v659: 178 pgs: 4 peering, 3 active+remapped+backfill_toofull, 171 active+clean; 32 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 829 op/s; 156/2088 objects misplaced (7.471%) 2024-09-16T09:37:35.591 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:35 smithi070 ceph-mon[30388]: 9.d starting backfill to osd.1 from (0'0,0'0] MAX to 365'15180 2024-09-16T09:37:35.592 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:35 smithi070 ceph-mon[40691]: pgmap v659: 178 pgs: 4 peering, 3 active+remapped+backfill_toofull, 171 active+clean; 32 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 829 op/s; 156/2088 objects misplaced (7.471%) 2024-09-16T09:37:35.592 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:35 smithi070 ceph-mon[40691]: 9.d starting backfill to osd.1 from (0'0,0'0] MAX to 365'15180 2024-09-16T09:37:35.599 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-09-16T09:37:35.599 INFO:teuthology.orchestra.run.smithi084.stdout: "ops": [], 2024-09-16T09:37:35.599 INFO:teuthology.orchestra.run.smithi084.stdout: "complaint_time": 30, 2024-09-16T09:37:35.599 INFO:teuthology.orchestra.run.smithi084.stdout: "num_blocked_ops": 0 2024-09-16T09:37:35.599 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-09-16T09:37:35.612 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:35.612 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:35.613 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:37:35.713 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:37:35.735 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:35.735 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:35.736 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:37:35.836 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:37:35.858 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:35.859 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:35.859 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:37:35.960 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:37:35.982 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:35.982 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:35.983 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:37:36.083 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:37:36.105 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:36.105 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:36.106 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:37:36.206 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:37:36.227 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:36.228 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:36.229 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:37:36.241 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.7 dump_historic_ops 2024-09-16T09:37:36.329 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:37:36.352 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:36.352 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:36.353 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:37:36.357 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'unique_pool_2' removed 2024-09-16T09:37:36.363 INFO:teuthology.orchestra.run.smithi070.stderr:reweighted osd.2 to 1 (10000) 2024-09-16T09:37:36.454 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:37:36.475 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:36.476 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:36.476 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:37:36.519 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:37:36.577 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:37:36.600 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:36.600 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:36.601 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:37:36.639 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:36 smithi084 ceph-mon[32629]: osdmap e370: 8 total, 8 up, 8 in 2024-09-16T09:37:36.639 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:36 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/850532253' entity='client.admin' cmd={"prefix": "osd reweight", "id": 2, "weight": 1.0} : dispatch 2024-09-16T09:37:36.639 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:36 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 2, "weight": 1.0} : dispatch 2024-09-16T09:37:36.639 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:36 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/843350003' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "unique_pool_2", "pool2": "unique_pool_2", "yes_i_really_really_mean_it": true} : dispatch 2024-09-16T09:37:36.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:36 smithi070 ceph-mon[30388]: osdmap e370: 8 total, 8 up, 8 in 2024-09-16T09:37:36.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:36 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/850532253' entity='client.admin' cmd={"prefix": "osd reweight", "id": 2, "weight": 1.0} : dispatch 2024-09-16T09:37:36.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:36 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 2, "weight": 1.0} : dispatch 2024-09-16T09:37:36.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:36 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/843350003' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "unique_pool_2", "pool2": "unique_pool_2", "yes_i_really_really_mean_it": true} : dispatch 2024-09-16T09:37:36.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:36 smithi070 ceph-mon[40691]: osdmap e370: 8 total, 8 up, 8 in 2024-09-16T09:37:36.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:36 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/850532253' entity='client.admin' cmd={"prefix": "osd reweight", "id": 2, "weight": 1.0} : dispatch 2024-09-16T09:37:36.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:36 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 2, "weight": 1.0} : dispatch 2024-09-16T09:37:36.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:36 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/843350003' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "unique_pool_2", "pool2": "unique_pool_2", "yes_i_really_really_mean_it": true} : dispatch 2024-09-16T09:37:36.702 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:37:36.724 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:36.725 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:36.725 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:37:36.826 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:37:36.856 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:36.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:36.857 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:37:36.958 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:37:36.980 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:36.980 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:36.981 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:37:37.015 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T09:37:37.068 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-09-16T09:37:37.068 INFO:teuthology.orchestra.run.smithi084.stdout: "size": 20, 2024-09-16T09:37:37.068 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 600, 2024-09-16T09:37:37.068 INFO:teuthology.orchestra.run.smithi084.stdout: "ops": [ 2024-09-16T09:37:37.068 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.068 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:56275 9.d 9:b3de4b96:::benchmark_data_smithi070_137000_object56274:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e247)", 2024-09-16T09:37:37.068 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:33:59.304262+0000", 2024-09-16T09:37:37.068 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 217.76086470499999, 2024-09-16T09:37:37.069 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.1508694620000002, 2024-09-16T09:37:37.069 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:37:37.069 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:37:37.069 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "reached pg", 2024-09-16T09:37:37.069 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:37:37.069 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:37:37.069 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:37:37.069 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 56275 2024-09-16T09:37:37.069 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.069 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:37:37.069 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.069 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:37:37.069 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:33:59.304262+0000", 2024-09-16T09:37:37.070 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:37:37.070 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.070 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.070 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:37:37.070 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:33:59.304262+0000", 2024-09-16T09:37:37.070 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:37:37.070 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.070 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.070 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:37:37.070 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:33:59.304264+0000", 2024-09-16T09:37:37.070 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.4529999999999999e-06 2024-09-16T09:37:37.070 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.070 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.070 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:37:37.071 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:33:59.304319+0000", 2024-09-16T09:37:37.071 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.5575999999999998e-05 2024-09-16T09:37:37.071 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.071 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.071 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:37:37.071 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:33:59.304331+0000", 2024-09-16T09:37:37.071 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1892999999999999e-05 2024-09-16T09:37:37.071 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.071 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.071 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:37:37.071 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:33:59.304337+0000", 2024-09-16T09:37:37.071 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.891e-06 2024-09-16T09:37:37.071 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.071 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.072 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:37:37.072 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:33:59.304389+0000", 2024-09-16T09:37:37.072 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.2015000000000003e-05 2024-09-16T09:37:37.072 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.072 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.072 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:37:37.072 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:33:59.304495+0000", 2024-09-16T09:37:37.072 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000106168 2024-09-16T09:37:37.072 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.072 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.072 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 1,5", 2024-09-16T09:37:37.072 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:33:59.304646+0000", 2024-09-16T09:37:37.072 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00015057300000000001 2024-09-16T09:37:37.073 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.073 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.073 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:37:37.073 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:33:59.306508+0000", 2024-09-16T09:37:37.073 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0018619280000000001 2024-09-16T09:37:37.073 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.073 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.073 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:37.073 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:33:59.306755+0000", 2024-09-16T09:37:37.073 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00024750999999999999 2024-09-16T09:37:37.073 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.073 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.073 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:37:37.073 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:33:59.319589+0000", 2024-09-16T09:37:37.074 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.012834287 2024-09-16T09:37:37.074 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.074 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.074 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:37:37.074 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:33:59.319600+0000", 2024-09-16T09:37:37.074 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0966e-05 2024-09-16T09:37:37.074 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.074 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.074 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:37:37.074 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:00.335957+0000", 2024-09-16T09:37:37.074 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.016356378 2024-09-16T09:37:37.074 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.074 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.075 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:37:37.075 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:00.335972+0000", 2024-09-16T09:37:37.075 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.5605000000000001e-05 2024-09-16T09:37:37.075 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.075 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.075 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:37:37.075 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:01.455037+0000", 2024-09-16T09:37:37.075 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1190647410000001 2024-09-16T09:37:37.075 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.075 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.075 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:37:37.075 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:01.455132+0000", 2024-09-16T09:37:37.075 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.4477999999999999e-05 2024-09-16T09:37:37.075 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:37:37.076 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:37:37.076 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:37:37.076 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.076 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.076 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:56279 9.d 9:b7adec88:::benchmark_data_smithi070_137000_object56278:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e247)", 2024-09-16T09:37:37.076 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:33:59.308343+0000", 2024-09-16T09:37:37.076 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 217.756783988, 2024-09-16T09:37:37.076 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.150991936, 2024-09-16T09:37:37.076 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:37:37.076 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:37:37.076 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "reached pg", 2024-09-16T09:37:37.076 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:37:37.076 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:37:37.076 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:37:37.077 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 56279 2024-09-16T09:37:37.077 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.077 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:37:37.077 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.077 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:37:37.077 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:33:59.308343+0000", 2024-09-16T09:37:37.077 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:37:37.077 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.077 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.077 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:37:37.077 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:33:59.308343+0000", 2024-09-16T09:37:37.077 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:37:37.077 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.077 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.078 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:37:37.078 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:33:59.308344+0000", 2024-09-16T09:37:37.078 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.2619999999999999e-06 2024-09-16T09:37:37.078 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.078 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.078 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:37:37.078 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:33:59.308448+0000", 2024-09-16T09:37:37.078 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000103414 2024-09-16T09:37:37.078 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.078 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.078 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:37:37.078 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:33:59.308461+0000", 2024-09-16T09:37:37.078 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.3373999999999999e-05 2024-09-16T09:37:37.078 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.079 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.079 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:37:37.079 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:33:59.308467+0000", 2024-09-16T09:37:37.079 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.3709999999999996e-06 2024-09-16T09:37:37.079 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.079 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.079 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:37:37.079 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:33:59.308516+0000", 2024-09-16T09:37:37.079 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.8668000000000002e-05 2024-09-16T09:37:37.079 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.079 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.079 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:37:37.079 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:33:59.308652+0000", 2024-09-16T09:37:37.079 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000136317 2024-09-16T09:37:37.080 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.080 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.080 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 1,5", 2024-09-16T09:37:37.080 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:33:59.308889+0000", 2024-09-16T09:37:37.080 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000237123 2024-09-16T09:37:37.080 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.080 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.080 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:37:37.080 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:33:59.310982+0000", 2024-09-16T09:37:37.080 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0020922940000000002 2024-09-16T09:37:37.080 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.080 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.080 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:37.081 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:33:59.311284+0000", 2024-09-16T09:37:37.081 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00030216299999999999 2024-09-16T09:37:37.081 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.081 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.081 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:37:37.081 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:33:59.319633+0000", 2024-09-16T09:37:37.081 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0083495989999999992 2024-09-16T09:37:37.081 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.081 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.081 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:37:37.081 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:33:59.319642+0000", 2024-09-16T09:37:37.081 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.3850000000000002e-06 2024-09-16T09:37:37.081 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.082 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.082 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:37:37.082 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:00.336011+0000", 2024-09-16T09:37:37.082 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.016368658 2024-09-16T09:37:37.082 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.082 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.082 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:37:37.082 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:00.336018+0000", 2024-09-16T09:37:37.082 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.7759999999999997e-06 2024-09-16T09:37:37.082 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.082 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.082 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:37:37.083 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:01.459194+0000", 2024-09-16T09:37:37.083 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.123176188 2024-09-16T09:37:37.083 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.083 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.083 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:37:37.083 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:01.459335+0000", 2024-09-16T09:37:37.083 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00014034400000000001 2024-09-16T09:37:37.083 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:37:37.083 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:37:37.083 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:37:37.083 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.083 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.084 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:56304 9.d 9:b6eef410:::benchmark_data_smithi070_137000_object56303:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e248)", 2024-09-16T09:37:37.084 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:33:59.353610+0000", 2024-09-16T09:37:37.084 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 217.711517305, 2024-09-16T09:37:37.084 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.127432819, 2024-09-16T09:37:37.084 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:37:37.084 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:37:37.084 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:37:37.084 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:37:37.084 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:37:37.084 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:37:37.084 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 56304 2024-09-16T09:37:37.084 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.084 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:37:37.084 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.085 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:37:37.085 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:33:59.353610+0000", 2024-09-16T09:37:37.085 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:37:37.085 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.085 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.085 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:37:37.085 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:33:59.353610+0000", 2024-09-16T09:37:37.085 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:37:37.085 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.085 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.085 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:37:37.085 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:33:59.353611+0000", 2024-09-16T09:37:37.085 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.418e-06 2024-09-16T09:37:37.086 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.086 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.086 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:37:37.086 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:33:59.353644+0000", 2024-09-16T09:37:37.086 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.2861000000000003e-05 2024-09-16T09:37:37.086 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.086 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.086 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:37:37.086 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:33:59.353653+0000", 2024-09-16T09:37:37.086 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.3810000000000005e-06 2024-09-16T09:37:37.086 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.086 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.086 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:37:37.087 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:33:59.353659+0000", 2024-09-16T09:37:37.087 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.1120000000000001e-06 2024-09-16T09:37:37.087 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.087 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.087 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:37:37.087 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:33:59.353718+0000", 2024-09-16T09:37:37.087 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.8208999999999998e-05 2024-09-16T09:37:37.087 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.087 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.087 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:37:37.087 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:33:59.353728+0000", 2024-09-16T09:37:37.087 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0437e-05 2024-09-16T09:37:37.087 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.087 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.087 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:37:37.088 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:00.336042+0000", 2024-09-16T09:37:37.088 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.98231433700000004 2024-09-16T09:37:37.088 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.088 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.088 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:37:37.088 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:00.336049+0000", 2024-09-16T09:37:37.088 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.2910000000000004e-06 2024-09-16T09:37:37.088 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.088 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.088 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:37:37.088 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:01.460301+0000", 2024-09-16T09:37:37.089 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1242522509999999 2024-09-16T09:37:37.089 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.089 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.089 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:37:37.089 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:01.460571+0000", 2024-09-16T09:37:37.089 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00027019700000000002 2024-09-16T09:37:37.089 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.089 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.089 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 4,5", 2024-09-16T09:37:37.089 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:01.460749+0000", 2024-09-16T09:37:37.089 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00017809100000000001 2024-09-16T09:37:37.090 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.090 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.090 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:37:37.090 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:01.471784+0000", 2024-09-16T09:37:37.090 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.011034805999999999 2024-09-16T09:37:37.090 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.090 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.090 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:37.090 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:01.471825+0000", 2024-09-16T09:37:37.090 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.0655000000000003e-05 2024-09-16T09:37:37.090 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.090 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.090 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:37.090 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:01.477471+0000", 2024-09-16T09:37:37.091 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0056467519999999997 2024-09-16T09:37:37.091 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.091 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.091 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:37:37.091 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:01.477582+0000", 2024-09-16T09:37:37.091 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000110305 2024-09-16T09:37:37.091 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.091 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.091 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:37:37.091 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:01.481042+0000", 2024-09-16T09:37:37.091 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.003460716 2024-09-16T09:37:37.091 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:37:37.091 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:37:37.092 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:37:37.092 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.092 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.092 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:56356 9.d 9:bb0a0400:::benchmark_data_smithi070_137000_object56355:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e248)", 2024-09-16T09:37:37.092 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:33:59.410378+0000", 2024-09-16T09:37:37.092 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 217.65474924200001, 2024-09-16T09:37:37.092 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.0922890729999999, 2024-09-16T09:37:37.092 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:37:37.092 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:37:37.092 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:37:37.092 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:37:37.092 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:37:37.092 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:37:37.092 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 56356 2024-09-16T09:37:37.093 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.093 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:37:37.093 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.093 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:37:37.093 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:33:59.410378+0000", 2024-09-16T09:37:37.093 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:37:37.093 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.093 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.093 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:37:37.093 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:33:59.410378+0000", 2024-09-16T09:37:37.093 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:37:37.093 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.093 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.093 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:37:37.094 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:33:59.410378+0000", 2024-09-16T09:37:37.094 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.4e-07 2024-09-16T09:37:37.094 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.094 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.094 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:37:37.094 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:33:59.410453+0000", 2024-09-16T09:37:37.094 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.5018999999999994e-05 2024-09-16T09:37:37.094 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.094 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.094 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:37:37.094 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:33:59.410465+0000", 2024-09-16T09:37:37.094 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1313e-05 2024-09-16T09:37:37.094 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.094 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.095 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:37:37.095 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:33:59.410472+0000", 2024-09-16T09:37:37.095 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.1799999999999999e-06 2024-09-16T09:37:37.095 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.095 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.095 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:37:37.095 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:33:59.410793+0000", 2024-09-16T09:37:37.095 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00032136799999999999 2024-09-16T09:37:37.095 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.095 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.095 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:37:37.095 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:33:59.410804+0000", 2024-09-16T09:37:37.095 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0511e-05 2024-09-16T09:37:37.095 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.096 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.096 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:37:37.096 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:00.336071+0000", 2024-09-16T09:37:37.096 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.92526685099999995 2024-09-16T09:37:37.096 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.096 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.096 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:37:37.096 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:00.336077+0000", 2024-09-16T09:37:37.096 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.1829999999999999e-06 2024-09-16T09:37:37.096 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:37:37.098 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.098 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.098 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:37:37.098 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:01.463210+0000", 2024-09-16T09:37:37.098 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1271330019999999 2024-09-16T09:37:37.098 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.098 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.098 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:37:37.098 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:01.463374+0000", 2024-09-16T09:37:37.098 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00016363200000000001 2024-09-16T09:37:37.098 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.099 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.099 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 4,5", 2024-09-16T09:37:37.099 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:01.463574+0000", 2024-09-16T09:37:37.099 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00020077300000000001 2024-09-16T09:37:37.099 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.099 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.099 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:37.099 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:01.471865+0000", 2024-09-16T09:37:37.099 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0082902319999999998 2024-09-16T09:37:37.099 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.099 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.099 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:37:37.099 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:01.485312+0000", 2024-09-16T09:37:37.099 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.013447578999999999 2024-09-16T09:37:37.100 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.100 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.100 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:37.100 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:01.496463+0000", 2024-09-16T09:37:37.100 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.011150721000000001 2024-09-16T09:37:37.100 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.100 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.100 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:37:37.100 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:01.496525+0000", 2024-09-16T09:37:37.100 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.1667000000000004e-05 2024-09-16T09:37:37.100 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.100 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.100 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:37:37.101 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:01.502667+0000", 2024-09-16T09:37:37.101 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0061422020000000003 2024-09-16T09:37:37.101 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:37:37.101 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:37:37.101 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:37:37.101 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.101 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.101 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:56361 9.d 9:ba416558:::benchmark_data_smithi070_137000_object56360:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e248)", 2024-09-16T09:37:37.101 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:33:59.417939+0000", 2024-09-16T09:37:37.101 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 217.64718763799999, 2024-09-16T09:37:37.101 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.0871587570000001, 2024-09-16T09:37:37.101 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:37:37.101 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:37:37.101 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:37:37.102 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:37:37.102 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:37:37.102 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:37:37.102 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 56361 2024-09-16T09:37:37.102 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.102 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:37:37.102 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.102 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:37:37.102 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:33:59.417939+0000", 2024-09-16T09:37:37.102 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:37:37.102 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.102 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.102 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:37:37.102 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:33:59.417939+0000", 2024-09-16T09:37:37.103 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:37:37.103 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.103 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.103 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:37:37.103 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:33:59.417941+0000", 2024-09-16T09:37:37.103 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.5579999999999999e-06 2024-09-16T09:37:37.103 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.103 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.103 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:37:37.103 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:33:59.417989+0000", 2024-09-16T09:37:37.103 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.8208999999999999e-05 2024-09-16T09:37:37.103 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.103 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.104 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:37:37.104 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:33:59.418003+0000", 2024-09-16T09:37:37.104 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.3569e-05 2024-09-16T09:37:37.104 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.104 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.104 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:37:37.104 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:33:59.418010+0000", 2024-09-16T09:37:37.104 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.2169999999999997e-06 2024-09-16T09:37:37.104 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.104 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.104 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:37:37.104 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:33:59.418070+0000", 2024-09-16T09:37:37.104 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.0293e-05 2024-09-16T09:37:37.105 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.105 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.105 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:37:37.105 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:33:59.418081+0000", 2024-09-16T09:37:37.105 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0721e-05 2024-09-16T09:37:37.105 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.105 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.105 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:37:37.105 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:00.336099+0000", 2024-09-16T09:37:37.105 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.91801782499999995 2024-09-16T09:37:37.105 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.105 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.105 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:37:37.106 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:00.336105+0000", 2024-09-16T09:37:37.106 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.2269999999999998e-06 2024-09-16T09:37:37.106 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.106 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.106 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:37:37.106 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:01.466945+0000", 2024-09-16T09:37:37.106 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.130840012 2024-09-16T09:37:37.106 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.106 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.106 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:37:37.106 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:01.467113+0000", 2024-09-16T09:37:37.106 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000167837 2024-09-16T09:37:37.106 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.106 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.107 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 4,5", 2024-09-16T09:37:37.107 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:01.467277+0000", 2024-09-16T09:37:37.107 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000164523 2024-09-16T09:37:37.107 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.107 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.107 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:37.107 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:01.481101+0000", 2024-09-16T09:37:37.107 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.013823419 2024-09-16T09:37:37.107 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.107 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.107 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:37:37.107 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:01.485318+0000", 2024-09-16T09:37:37.108 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0042174580000000003 2024-09-16T09:37:37.108 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.108 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.108 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:37.108 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:01.502723+0000", 2024-09-16T09:37:37.108 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.017405331999999999 2024-09-16T09:37:37.108 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.108 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.108 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:37:37.108 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:01.502835+0000", 2024-09-16T09:37:37.108 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000111507 2024-09-16T09:37:37.108 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.108 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.108 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:37:37.109 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:01.505098+0000", 2024-09-16T09:37:37.109 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00226305 2024-09-16T09:37:37.109 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:37:37.109 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:37:37.109 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:37:37.109 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.109 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.109 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:56363 9.d 9:b24a00c9:::benchmark_data_smithi070_137000_object56362:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e248)", 2024-09-16T09:37:37.109 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:33:59.421272+0000", 2024-09-16T09:37:37.109 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 217.643855113, 2024-09-16T09:37:37.109 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.0854374610000002, 2024-09-16T09:37:37.109 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:37:37.109 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:37:37.110 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:37:37.110 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:37:37.110 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:37:37.110 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:37:37.110 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 56363 2024-09-16T09:37:37.110 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.110 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:37:37.110 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.110 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:37:37.110 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:33:59.421272+0000", 2024-09-16T09:37:37.110 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:37:37.110 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.110 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.110 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:37:37.111 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:33:59.421272+0000", 2024-09-16T09:37:37.111 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:37:37.111 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.111 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.111 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:37:37.111 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:33:59.421274+0000", 2024-09-16T09:37:37.111 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.787e-06 2024-09-16T09:37:37.111 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.111 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.111 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:37:37.111 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:33:59.421310+0000", 2024-09-16T09:37:37.111 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.6127000000000002e-05 2024-09-16T09:37:37.111 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.112 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.112 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:37:37.112 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:33:59.421321+0000", 2024-09-16T09:37:37.112 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1737e-05 2024-09-16T09:37:37.112 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.112 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.112 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:37:37.112 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:33:59.421329+0000", 2024-09-16T09:37:37.112 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.3930000000000002e-06 2024-09-16T09:37:37.112 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.112 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.112 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:37:37.112 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:33:59.421403+0000", 2024-09-16T09:37:37.113 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.3769000000000005e-05 2024-09-16T09:37:37.113 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.113 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.113 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:37:37.113 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:33:59.421411+0000", 2024-09-16T09:37:37.113 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.4370000000000007e-06 2024-09-16T09:37:37.113 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.113 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.113 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:37:37.113 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:00.336127+0000", 2024-09-16T09:37:37.113 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.91471611200000003 2024-09-16T09:37:37.113 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.113 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.113 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:37:37.114 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:00.336133+0000", 2024-09-16T09:37:37.114 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.1120000000000001e-06 2024-09-16T09:37:37.114 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.114 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.114 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:37:37.114 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:01.471893+0000", 2024-09-16T09:37:37.114 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.13575994 2024-09-16T09:37:37.114 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.114 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.114 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:37:37.114 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:01.472021+0000", 2024-09-16T09:37:37.114 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00012820099999999999 2024-09-16T09:37:37.115 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.115 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.115 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 4,5", 2024-09-16T09:37:37.115 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:01.472165+0000", 2024-09-16T09:37:37.115 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00014317299999999999 2024-09-16T09:37:37.115 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.115 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.115 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:37.115 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:01.481147+0000", 2024-09-16T09:37:37.115 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0089819710000000001 2024-09-16T09:37:37.115 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.115 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.115 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:37:37.116 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:01.485324+0000", 2024-09-16T09:37:37.116 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0041773449999999998 2024-09-16T09:37:37.116 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.116 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.116 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:37.116 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:01.505161+0000", 2024-09-16T09:37:37.116 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.019837104000000001 2024-09-16T09:37:37.116 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.116 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.116 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:37:37.116 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:01.505222+0000", 2024-09-16T09:37:37.116 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.1030999999999999e-05 2024-09-16T09:37:37.116 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.116 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.117 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:37:37.117 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:01.506709+0000", 2024-09-16T09:37:37.117 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0014872220000000001 2024-09-16T09:37:37.117 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:37:37.117 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:37:37.117 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:37:37.117 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.117 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.117 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:56365 9.d 9:bd4977cf:::benchmark_data_smithi070_137000_object56364:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e248)", 2024-09-16T09:37:37.117 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:33:59.516549+0000", 2024-09-16T09:37:37.117 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 217.54857765899999, 2024-09-16T09:37:37.117 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.991296089, 2024-09-16T09:37:37.117 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:37:37.118 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:37:37.118 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:37:37.118 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:37:37.118 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:37:37.118 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:37:37.118 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 56365 2024-09-16T09:37:37.118 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.118 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:37:37.118 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.118 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:37:37.118 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:33:59.516549+0000", 2024-09-16T09:37:37.118 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:37:37.118 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.119 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.119 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:37:37.119 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:33:59.516549+0000", 2024-09-16T09:37:37.119 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:37:37.119 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.119 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.119 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:37:37.119 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:33:59.516551+0000", 2024-09-16T09:37:37.119 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.889e-06 2024-09-16T09:37:37.119 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.119 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.119 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:37:37.119 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:33:59.516606+0000", 2024-09-16T09:37:37.119 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.4926999999999998e-05 2024-09-16T09:37:37.120 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.120 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.120 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:37:37.120 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:33:59.516616+0000", 2024-09-16T09:37:37.120 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.6590000000000002e-06 2024-09-16T09:37:37.120 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.120 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.120 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:37:37.120 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:33:59.516622+0000", 2024-09-16T09:37:37.120 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.2389999999999999e-06 2024-09-16T09:37:37.120 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.120 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.120 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:37:37.121 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:33:59.516669+0000", 2024-09-16T09:37:37.121 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.7290999999999998e-05 2024-09-16T09:37:37.121 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.121 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.121 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:37:37.121 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:33:59.516676+0000", 2024-09-16T09:37:37.121 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.1400000000000002e-06 2024-09-16T09:37:37.121 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.121 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.121 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:37:37.122 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:00.336551+0000", 2024-09-16T09:37:37.122 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.81987438000000001 2024-09-16T09:37:37.122 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.122 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.122 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:37:37.122 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:00.336559+0000", 2024-09-16T09:37:37.122 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.2819999999999996e-06 2024-09-16T09:37:37.122 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.122 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.122 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:37:37.122 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:01.474595+0000", 2024-09-16T09:37:37.122 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1380363039999999 2024-09-16T09:37:37.122 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.122 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.123 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:37:37.123 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:01.474724+0000", 2024-09-16T09:37:37.123 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00012868800000000001 2024-09-16T09:37:37.123 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.123 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.123 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 4,5", 2024-09-16T09:37:37.123 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:01.474873+0000", 2024-09-16T09:37:37.123 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00014924400000000001 2024-09-16T09:37:37.123 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.123 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.123 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:37.123 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:01.481214+0000", 2024-09-16T09:37:37.123 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0063412160000000002 2024-09-16T09:37:37.124 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.124 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.124 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:37:37.124 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:01.487653+0000", 2024-09-16T09:37:37.124 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0064380399999999999 2024-09-16T09:37:37.124 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.124 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.124 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:37.124 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:01.506771+0000", 2024-09-16T09:37:37.124 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.019118953000000001 2024-09-16T09:37:37.124 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.124 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.124 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:37:37.124 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:01.506832+0000", 2024-09-16T09:37:37.125 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.0545999999999999e-05 2024-09-16T09:37:37.125 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.125 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.125 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:37:37.125 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:01.507845+0000", 2024-09-16T09:37:37.125 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0010132909999999999 2024-09-16T09:37:37.125 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:37:37.125 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:37:37.125 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:37:37.125 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.125 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.125 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:178341 9.18 9:1b73fd02:::benchmark_data_smithi070_137000_object56630:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e308)", 2024-09-16T09:37:37.125 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:36:11.365831+0000", 2024-09-16T09:37:37.125 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 85.69929544, 2024-09-16T09:37:37.126 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.068998525, 2024-09-16T09:37:37.126 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:37:37.126 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:37:37.126 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "reached pg", 2024-09-16T09:37:37.126 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:37:37.126 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:37:37.126 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:37:37.126 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 178341 2024-09-16T09:37:37.126 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.126 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:37:37.126 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.126 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:37:37.126 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:11.365831+0000", 2024-09-16T09:37:37.127 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:37:37.127 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.127 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.127 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:37:37.127 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:11.365831+0000", 2024-09-16T09:37:37.127 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:37:37.127 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.127 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.127 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:37:37.127 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:11.365833+0000", 2024-09-16T09:37:37.127 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.658e-06 2024-09-16T09:37:37.127 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.127 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.127 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:37:37.128 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:11.365838+0000", 2024-09-16T09:37:37.128 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.933e-06 2024-09-16T09:37:37.128 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.128 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.128 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:37:37.128 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:11.365852+0000", 2024-09-16T09:37:37.128 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.3536000000000001e-05 2024-09-16T09:37:37.128 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.128 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.128 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:37:37.128 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:11.365860+0000", 2024-09-16T09:37:37.128 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.3159999999999997e-06 2024-09-16T09:37:37.129 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.129 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.129 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:37:37.129 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:11.365924+0000", 2024-09-16T09:37:37.129 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.4473000000000004e-05 2024-09-16T09:37:37.129 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.129 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.129 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op must wait for map", 2024-09-16T09:37:37.129 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:11.365932+0000", 2024-09-16T09:37:37.129 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.3810000000000001e-06 2024-09-16T09:37:37.129 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.129 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.129 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:37:37.130 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:11.367323+0000", 2024-09-16T09:37:37.130 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.001391352 2024-09-16T09:37:37.130 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.130 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.130 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:37:37.130 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:11.367332+0000", 2024-09-16T09:37:37.130 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.9169999999999995e-06 2024-09-16T09:37:37.130 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.130 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.130 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:37:37.130 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:12.434755+0000", 2024-09-16T09:37:37.130 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.067423271 2024-09-16T09:37:37.130 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.130 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.131 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:37:37.131 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:12.434830+0000", 2024-09-16T09:37:37.131 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.4688e-05 2024-09-16T09:37:37.131 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:37:37.131 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:37:37.131 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:37:37.131 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.131 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.131 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:178375 9.18 9:1e6d3e73:::benchmark_data_smithi070_137000_object56664:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e308)", 2024-09-16T09:37:37.131 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:36:11.411935+0000", 2024-09-16T09:37:37.131 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 85.653192161000007, 2024-09-16T09:37:37.131 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0356727299999999, 2024-09-16T09:37:37.131 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:37:37.131 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:37:37.132 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:37:37.132 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:37:37.132 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:37:37.132 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:37:37.132 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 178375 2024-09-16T09:37:37.132 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.132 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:37:37.132 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.132 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:37:37.132 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:11.411935+0000", 2024-09-16T09:37:37.132 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:37:37.132 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.132 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.133 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:37:37.133 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:11.411935+0000", 2024-09-16T09:37:37.133 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:37:37.133 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.133 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.133 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:37:37.133 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:11.411936+0000", 2024-09-16T09:37:37.133 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0780000000000001e-06 2024-09-16T09:37:37.133 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.133 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.133 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:37:37.133 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:11.411940+0000", 2024-09-16T09:37:37.133 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.3819999999999997e-06 2024-09-16T09:37:37.134 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.134 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.134 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:37:37.134 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:11.411950+0000", 2024-09-16T09:37:37.134 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.8570000000000007e-06 2024-09-16T09:37:37.134 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.134 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.134 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:37:37.134 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:11.411955+0000", 2024-09-16T09:37:37.134 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.3179999999999997e-06 2024-09-16T09:37:37.134 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.134 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.134 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:37:37.134 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:11.412011+0000", 2024-09-16T09:37:37.135 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.5760999999999997e-05 2024-09-16T09:37:37.135 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.135 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.135 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:37:37.135 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:11.412020+0000", 2024-09-16T09:37:37.135 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.6999999999999997e-06 2024-09-16T09:37:37.135 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.135 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.135 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:37:37.135 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:12.435136+0000", 2024-09-16T09:37:37.135 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0231163080000001 2024-09-16T09:37:37.135 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.136 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.136 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:37:37.136 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:12.435258+0000", 2024-09-16T09:37:37.136 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00012221000000000001 2024-09-16T09:37:37.136 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.136 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.136 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,1,5", 2024-09-16T09:37:37.136 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:12.435568+0000", 2024-09-16T09:37:37.136 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00030966499999999998 2024-09-16T09:37:37.136 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.136 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.136 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:37.136 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:12.443060+0000", 2024-09-16T09:37:37.136 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0074922180000000001 2024-09-16T09:37:37.137 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.137 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.137 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:37.137 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:12.443306+0000", 2024-09-16T09:37:37.137 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00024587699999999998 2024-09-16T09:37:37.137 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.137 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.137 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:37:37.137 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:12.443323+0000", 2024-09-16T09:37:37.137 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.7405e-05 2024-09-16T09:37:37.137 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.137 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.137 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:37.138 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:12.447432+0000", 2024-09-16T09:37:37.138 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0041085410000000003 2024-09-16T09:37:37.138 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.138 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.138 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:37:37.138 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:12.447486+0000", 2024-09-16T09:37:37.138 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.4119999999999997e-05 2024-09-16T09:37:37.138 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.138 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.138 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:37:37.138 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:12.447607+0000", 2024-09-16T09:37:37.138 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00012129 2024-09-16T09:37:37.138 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:37:37.138 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:37:37.139 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:37:37.139 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.139 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.139 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:178422 9.18 9:186d9728:::benchmark_data_smithi070_137000_object56711:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e308)", 2024-09-16T09:37:37.139 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:36:11.448738+0000", 2024-09-16T09:37:37.139 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 85.616388975999996, 2024-09-16T09:37:37.139 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0107649409999999, 2024-09-16T09:37:37.139 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:37:37.139 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:37:37.139 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:37:37.139 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:37:37.139 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:37:37.139 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:37:37.140 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 178422 2024-09-16T09:37:37.140 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.140 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:37:37.140 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.140 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:37:37.140 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:11.448738+0000", 2024-09-16T09:37:37.140 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:37:37.140 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.140 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.140 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:37:37.140 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:11.448738+0000", 2024-09-16T09:37:37.140 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:37:37.140 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.141 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.141 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:37:37.141 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:11.448739+0000", 2024-09-16T09:37:37.141 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.7199999999999998e-07 2024-09-16T09:37:37.141 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.141 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.141 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:37:37.141 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:11.448743+0000", 2024-09-16T09:37:37.141 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.8480000000000003e-06 2024-09-16T09:37:37.141 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.141 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.141 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:37:37.141 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:11.448753+0000", 2024-09-16T09:37:37.141 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0066e-05 2024-09-16T09:37:37.142 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.142 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.142 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:37:37.142 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:11.448759+0000", 2024-09-16T09:37:37.142 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.5679999999999996e-06 2024-09-16T09:37:37.142 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.142 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.142 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:37:37.142 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:11.448820+0000", 2024-09-16T09:37:37.142 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.1198999999999995e-05 2024-09-16T09:37:37.142 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.142 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.142 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:37:37.143 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:11.448828+0000", 2024-09-16T09:37:37.143 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.6680000000000001e-06 2024-09-16T09:37:37.143 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.143 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.143 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:37:37.143 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:12.441890+0000", 2024-09-16T09:37:37.143 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.99306225000000004 2024-09-16T09:37:37.143 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.143 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.143 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:37:37.143 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:12.441981+0000", 2024-09-16T09:37:37.143 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.0457999999999993e-05 2024-09-16T09:37:37.143 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.144 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.144 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,1,5", 2024-09-16T09:37:37.144 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:12.442059+0000", 2024-09-16T09:37:37.144 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.8590000000000005e-05 2024-09-16T09:37:37.144 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.144 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.144 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:37.144 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:12.447287+0000", 2024-09-16T09:37:37.144 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0052277319999999997 2024-09-16T09:37:37.144 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.144 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.144 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:37:37.144 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:12.447302+0000", 2024-09-16T09:37:37.145 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.4858000000000001e-05 2024-09-16T09:37:37.145 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.145 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.145 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:37.145 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:12.447350+0000", 2024-09-16T09:37:37.145 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.8189000000000002e-05 2024-09-16T09:37:37.145 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.145 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.145 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:37.145 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:12.459284+0000", 2024-09-16T09:37:37.145 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.011933918999999999 2024-09-16T09:37:37.145 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.145 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.146 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:37:37.146 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:12.459368+0000", 2024-09-16T09:37:37.146 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.4075999999999999e-05 2024-09-16T09:37:37.146 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.146 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.146 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:37:37.146 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:12.459503+0000", 2024-09-16T09:37:37.146 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00013474799999999999 2024-09-16T09:37:37.146 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:37:37.146 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:37:37.146 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:37:37.146 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.146 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.147 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:178442 9.18 9:1acb8e56:::benchmark_data_smithi070_137000_object56731:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e308)", 2024-09-16T09:37:37.147 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:36:11.462079+0000", 2024-09-16T09:37:37.147 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 85.603047501000006, 2024-09-16T09:37:37.147 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.99786588799999998, 2024-09-16T09:37:37.147 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:37:37.147 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:37:37.147 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:37:37.147 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:37:37.147 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:37:37.147 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:37:37.147 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 178442 2024-09-16T09:37:37.147 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.147 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:37:37.147 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.148 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:37:37.148 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:11.462079+0000", 2024-09-16T09:37:37.148 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:37:37.148 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.148 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.148 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:37:37.148 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:11.462079+0000", 2024-09-16T09:37:37.148 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:37:37.148 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.148 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.148 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:37:37.148 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:11.462080+0000", 2024-09-16T09:37:37.148 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.64e-07 2024-09-16T09:37:37.148 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.149 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.149 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:37:37.149 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:11.462085+0000", 2024-09-16T09:37:37.149 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.1779999999999996e-06 2024-09-16T09:37:37.149 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.149 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.149 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:37:37.149 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:11.462098+0000", 2024-09-16T09:37:37.149 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.3536999999999999e-05 2024-09-16T09:37:37.149 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.149 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.149 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:37:37.149 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:11.462105+0000", 2024-09-16T09:37:37.150 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.08e-06 2024-09-16T09:37:37.150 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.150 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.150 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:37:37.150 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:11.462178+0000", 2024-09-16T09:37:37.150 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.2868999999999996e-05 2024-09-16T09:37:37.150 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.150 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.150 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:37:37.150 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:11.462189+0000", 2024-09-16T09:37:37.150 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0903e-05 2024-09-16T09:37:37.150 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.150 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.151 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:37:37.151 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:12.443377+0000", 2024-09-16T09:37:37.151 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.98118798299999999 2024-09-16T09:37:37.151 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.151 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.151 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:37:37.151 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:12.443489+0000", 2024-09-16T09:37:37.151 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00011176200000000001 2024-09-16T09:37:37.151 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.151 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.151 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,1,5", 2024-09-16T09:37:37.151 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:12.443623+0000", 2024-09-16T09:37:37.151 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000134284 2024-09-16T09:37:37.152 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.152 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.152 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:37:37.152 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:12.447400+0000", 2024-09-16T09:37:37.152 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.003777032 2024-09-16T09:37:37.152 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.152 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.152 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:37.152 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:12.447667+0000", 2024-09-16T09:37:37.152 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00026657800000000001 2024-09-16T09:37:37.152 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.152 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.152 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:37.152 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:12.451300+0000", 2024-09-16T09:37:37.153 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0036331760000000001 2024-09-16T09:37:37.153 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.153 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.153 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:37.153 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:12.459782+0000", 2024-09-16T09:37:37.153 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0084822370000000001 2024-09-16T09:37:37.153 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.153 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.153 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:37:37.153 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:12.459834+0000", 2024-09-16T09:37:37.153 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.2064000000000002e-05 2024-09-16T09:37:37.153 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.153 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.153 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:37:37.154 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:12.459945+0000", 2024-09-16T09:37:37.154 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000111241 2024-09-16T09:37:37.154 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:37:37.154 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:37:37.154 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:37:37.154 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.154 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.154 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:231309 9.13 9:ca1e125e:::benchmark_data_smithi070_137000_object109598:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e357)", 2024-09-16T09:37:37.154 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:37:18.512402+0000", 2024-09-16T09:37:37.154 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 18.552725005999999, 2024-09-16T09:37:37.154 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.044116117, 2024-09-16T09:37:37.154 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:37:37.154 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:37:37.155 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "reached pg", 2024-09-16T09:37:37.155 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:37:37.155 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:37:37.155 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:37:37.155 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 231309 2024-09-16T09:37:37.155 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.155 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:37:37.155 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.155 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:37:37.155 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.512402+0000", 2024-09-16T09:37:37.155 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:37:37.155 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.155 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.156 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:37:37.156 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.512402+0000", 2024-09-16T09:37:37.156 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:37:37.156 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.156 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.156 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:37:37.156 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.512404+0000", 2024-09-16T09:37:37.156 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.7e-06 2024-09-16T09:37:37.156 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.156 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.156 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:37:37.156 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.512410+0000", 2024-09-16T09:37:37.156 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.7630000000000003e-06 2024-09-16T09:37:37.156 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.157 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.157 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:37:37.157 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.512426+0000", 2024-09-16T09:37:37.157 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.5945000000000001e-05 2024-09-16T09:37:37.157 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.157 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.157 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:37:37.157 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.512435+0000", 2024-09-16T09:37:37.157 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.7879999999999995e-06 2024-09-16T09:37:37.157 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.157 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.157 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:37:37.157 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.663153+0000", 2024-09-16T09:37:37.158 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.15071828800000001 2024-09-16T09:37:37.158 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.158 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.158 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:37:37.158 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.663161+0000", 2024-09-16T09:37:37.158 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.1329999999999999e-06 2024-09-16T09:37:37.158 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.158 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.158 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:37:37.158 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.556446+0000", 2024-09-16T09:37:37.158 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.89328482799999998 2024-09-16T09:37:37.158 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.158 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.159 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:37:37.159 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.556518+0000", 2024-09-16T09:37:37.159 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.1672000000000001e-05 2024-09-16T09:37:37.159 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:37:37.159 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:37:37.159 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:37:37.159 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.159 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.159 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:231324 9.3 9:c4e2d072:::benchmark_data_smithi070_137000_object109613:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e357)", 2024-09-16T09:37:37.159 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:37:18.512452+0000", 2024-09-16T09:37:37.159 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 18.552674463999999, 2024-09-16T09:37:37.159 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0379426430000001, 2024-09-16T09:37:37.159 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:37:37.159 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:37:37.159 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:37:37.160 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:37:37.160 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:37:37.160 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:37:37.160 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 231324 2024-09-16T09:37:37.160 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.160 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:37:37.160 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.160 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:37:37.160 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.512452+0000", 2024-09-16T09:37:37.160 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:37:37.160 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.160 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.161 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:37:37.161 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.512452+0000", 2024-09-16T09:37:37.161 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:37:37.161 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.161 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.161 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:37:37.161 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.512453+0000", 2024-09-16T09:37:37.161 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.9999999999999998e-07 2024-09-16T09:37:37.161 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.161 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.161 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:37:37.161 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.512455+0000", 2024-09-16T09:37:37.161 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.4059999999999999e-06 2024-09-16T09:37:37.162 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.162 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.162 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:37:37.162 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.512462+0000", 2024-09-16T09:37:37.162 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.3110000000000002e-06 2024-09-16T09:37:37.162 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.162 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.162 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:37:37.162 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.512466+0000", 2024-09-16T09:37:37.162 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.7960000000000003e-06 2024-09-16T09:37:37.162 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.162 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.162 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:37:37.162 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.663387+0000", 2024-09-16T09:37:37.163 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.15092061200000001 2024-09-16T09:37:37.163 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.163 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.163 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:37:37.163 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.663395+0000", 2024-09-16T09:37:37.163 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.3909999999999998e-06 2024-09-16T09:37:37.163 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.163 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.163 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:37:37.163 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.522320+0000", 2024-09-16T09:37:37.163 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.85892479399999999 2024-09-16T09:37:37.163 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.163 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.164 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:37:37.164 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.522445+0000", 2024-09-16T09:37:37.164 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000124745 2024-09-16T09:37:37.164 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.164 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.164 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 1,3", 2024-09-16T09:37:37.164 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.522561+0000", 2024-09-16T09:37:37.164 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000116464 2024-09-16T09:37:37.164 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.164 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.164 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:37:37.164 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.526065+0000", 2024-09-16T09:37:37.164 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0035035439999999999 2024-09-16T09:37:37.165 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.165 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.165 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:37.165 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.540946+0000", 2024-09-16T09:37:37.165 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.014881116 2024-09-16T09:37:37.165 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.165 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.165 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:37.165 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.550318+0000", 2024-09-16T09:37:37.165 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0093716570000000003 2024-09-16T09:37:37.165 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.165 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.165 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:37:37.166 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.550353+0000", 2024-09-16T09:37:37.166 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.5729999999999998e-05 2024-09-16T09:37:37.166 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.166 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.166 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:37:37.166 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.550395+0000", 2024-09-16T09:37:37.166 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.1576999999999999e-05 2024-09-16T09:37:37.166 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:37:37.166 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:37:37.166 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:37:37.166 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.166 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.166 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:232725 9.13 9:cc864dd3:::benchmark_data_smithi070_137000_object111014:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e358)", 2024-09-16T09:37:37.166 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:37:20.523206+0000", 2024-09-16T09:37:37.166 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 16.541920803, 2024-09-16T09:37:37.167 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.1218705099999999, 2024-09-16T09:37:37.167 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:37:37.167 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:37:37.167 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "reached pg", 2024-09-16T09:37:37.167 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:37:37.167 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:37:37.167 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:37:37.167 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 232725 2024-09-16T09:37:37.167 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.167 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:37:37.167 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.167 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:37:37.167 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.523206+0000", 2024-09-16T09:37:37.167 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:37:37.168 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.168 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.168 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:37:37.168 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.523206+0000", 2024-09-16T09:37:37.168 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:37:37.168 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.168 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.168 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:37:37.168 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.523208+0000", 2024-09-16T09:37:37.168 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.7680000000000001e-06 2024-09-16T09:37:37.168 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.168 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.168 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:37:37.169 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.523214+0000", 2024-09-16T09:37:37.169 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.8649999999999998e-06 2024-09-16T09:37:37.169 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.169 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.169 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:37:37.170 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.523227+0000", 2024-09-16T09:37:37.170 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.3233e-05 2024-09-16T09:37:37.170 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.170 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.170 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:37:37.170 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.523236+0000", 2024-09-16T09:37:37.170 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.7250000000000003e-06 2024-09-16T09:37:37.171 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.171 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.171 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:37:37.171 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.524818+0000", 2024-09-16T09:37:37.171 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0015818780000000001 2024-09-16T09:37:37.171 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.171 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.171 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:37:37.171 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.524977+0000", 2024-09-16T09:37:37.171 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00015932699999999999 2024-09-16T09:37:37.171 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.171 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.171 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 1,3", 2024-09-16T09:37:37.171 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.525071+0000", 2024-09-16T09:37:37.172 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.3791999999999999e-05 2024-09-16T09:37:37.172 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.172 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.172 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:37:37.172 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.529527+0000", 2024-09-16T09:37:37.172 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0044562040000000001 2024-09-16T09:37:37.172 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.172 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.172 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:37:37.172 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.536411+0000", 2024-09-16T09:37:37.172 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0068840899999999998 2024-09-16T09:37:37.172 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.172 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.172 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:37:37.173 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.536420+0000", 2024-09-16T09:37:37.173 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.4590000000000004e-06 2024-09-16T09:37:37.173 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.173 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.173 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:37:37.173 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:21.526404+0000", 2024-09-16T09:37:37.173 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.98998363 2024-09-16T09:37:37.173 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.173 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.173 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:37:37.173 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:21.526416+0000", 2024-09-16T09:37:37.173 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.2149e-05 2024-09-16T09:37:37.173 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.173 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.173 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:37:37.174 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:22.637791+0000", 2024-09-16T09:37:37.174 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1113750929999999 2024-09-16T09:37:37.174 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.174 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.174 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for missing object", 2024-09-16T09:37:37.174 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:22.637932+0000", 2024-09-16T09:37:37.174 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00014046800000000001 2024-09-16T09:37:37.174 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.174 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.174 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:37:37.174 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:22.645001+0000", 2024-09-16T09:37:37.174 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0070696379999999996 2024-09-16T09:37:37.174 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.174 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.175 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:37:37.175 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:22.645077+0000", 2024-09-16T09:37:37.175 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.5191000000000006e-05 2024-09-16T09:37:37.175 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:37:37.175 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:37:37.175 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:37:37.175 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.175 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.175 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:232732 9.17 9:ebb5d972:::benchmark_data_smithi070_137000_object111021:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e359)", 2024-09-16T09:37:37.175 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:37:20.529143+0000", 2024-09-16T09:37:37.175 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 16.535984161999998, 2024-09-16T09:37:37.175 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.0767027910000002, 2024-09-16T09:37:37.175 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:37:37.176 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:37:37.176 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "reached pg", 2024-09-16T09:37:37.176 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:37:37.176 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:37:37.176 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:37:37.176 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 232732 2024-09-16T09:37:37.176 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.176 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:37:37.176 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.176 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:37:37.176 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.529143+0000", 2024-09-16T09:37:37.176 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:37:37.176 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.176 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.177 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:37:37.177 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.529143+0000", 2024-09-16T09:37:37.177 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:37:37.177 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.177 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.177 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:37:37.177 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.529144+0000", 2024-09-16T09:37:37.177 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.0500000000000002e-07 2024-09-16T09:37:37.177 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.177 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.177 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:37:37.177 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.529147+0000", 2024-09-16T09:37:37.177 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.0450000000000001e-06 2024-09-16T09:37:37.177 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.178 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.178 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:37:37.178 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.529154+0000", 2024-09-16T09:37:37.178 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.6359999999999994e-06 2024-09-16T09:37:37.178 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.178 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.178 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:37:37.178 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.529160+0000", 2024-09-16T09:37:37.178 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.6330000000000001e-06 2024-09-16T09:37:37.178 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.178 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.178 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:37:37.178 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.529214+0000", 2024-09-16T09:37:37.178 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.4246000000000002e-05 2024-09-16T09:37:37.178 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.179 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.179 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op must wait for map", 2024-09-16T09:37:37.179 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.529221+0000", 2024-09-16T09:37:37.179 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.4529999999999999e-06 2024-09-16T09:37:37.179 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.179 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.179 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:37:37.179 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.536419+0000", 2024-09-16T09:37:37.179 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0071983660000000003 2024-09-16T09:37:37.179 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.179 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.179 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:37:37.179 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.536428+0000", 2024-09-16T09:37:37.179 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.8789999999999992e-06 2024-09-16T09:37:37.179 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.180 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.180 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:37:37.180 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:21.528591+0000", 2024-09-16T09:37:37.180 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.99216334100000003 2024-09-16T09:37:37.180 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.180 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.180 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:37:37.180 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:21.528600+0000", 2024-09-16T09:37:37.180 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.3400000000000004e-06 2024-09-16T09:37:37.180 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.180 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.180 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:37:37.180 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:22.603425+0000", 2024-09-16T09:37:37.180 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0748247 2024-09-16T09:37:37.180 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.181 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.181 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for missing object", 2024-09-16T09:37:37.181 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:22.603534+0000", 2024-09-16T09:37:37.181 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000109141 2024-09-16T09:37:37.181 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.181 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.181 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:37:37.181 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:22.605775+0000", 2024-09-16T09:37:37.181 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0022410849999999999 2024-09-16T09:37:37.181 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.181 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.181 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:37:37.181 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:22.605846+0000", 2024-09-16T09:37:37.182 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.0121000000000002e-05 2024-09-16T09:37:37.182 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:37:37.182 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:37:37.182 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:37:37.182 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.182 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.182 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:232737 9.15 9.c37a9f75 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e359)", 2024-09-16T09:37:37.182 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:37:20.538677+0000", 2024-09-16T09:37:37.182 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 16.526449483, 2024-09-16T09:37:37.182 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.98906841300000004, 2024-09-16T09:37:37.182 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:37:37.182 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:37:37.182 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "reached pg", 2024-09-16T09:37:37.182 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:37:37.183 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:37:37.183 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:37:37.183 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 232737 2024-09-16T09:37:37.183 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.183 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:37:37.183 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.183 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:37:37.183 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.538677+0000", 2024-09-16T09:37:37.183 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:37:37.183 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.183 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.183 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:37:37.183 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.538677+0000", 2024-09-16T09:37:37.183 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:37:37.184 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.184 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.184 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:37:37.184 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.538679+0000", 2024-09-16T09:37:37.184 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.341e-06 2024-09-16T09:37:37.184 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.184 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.184 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:37:37.184 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.538682+0000", 2024-09-16T09:37:37.184 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.4790000000000001e-06 2024-09-16T09:37:37.184 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.184 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.184 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:37:37.185 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.538691+0000", 2024-09-16T09:37:37.185 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.3850000000000002e-06 2024-09-16T09:37:37.185 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.185 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.185 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:37:37.185 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.538697+0000", 2024-09-16T09:37:37.185 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.5710000000000003e-06 2024-09-16T09:37:37.185 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.185 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.185 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:37:37.185 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.538774+0000", 2024-09-16T09:37:37.185 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.7281999999999996e-05 2024-09-16T09:37:37.185 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.185 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.186 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:37:37.186 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.538781+0000", 2024-09-16T09:37:37.186 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.7059999999999998e-06 2024-09-16T09:37:37.186 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.186 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.186 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:37:37.186 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:21.527737+0000", 2024-09-16T09:37:37.186 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.98895597499999999 2024-09-16T09:37:37.186 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.186 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.186 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:37:37.186 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:21.527746+0000", 2024-09-16T09:37:37.186 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.6740000000000003e-06 2024-09-16T09:37:37.187 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:37:37.187 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:37:37.187 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:37:37.187 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.187 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.187 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:232745 9.13 9:ca005211:::benchmark_data_smithi070_137000_object111034:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e359)", 2024-09-16T09:37:37.187 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:37:20.542452+0000", 2024-09-16T09:37:37.187 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 16.522675192000001, 2024-09-16T09:37:37.187 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.1016708319999999, 2024-09-16T09:37:37.187 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:37:37.187 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:37:37.187 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:37:37.187 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:37:37.188 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:37:37.188 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:37:37.188 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 232745 2024-09-16T09:37:37.188 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.188 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:37:37.188 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.188 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:37:37.188 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.542452+0000", 2024-09-16T09:37:37.188 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:37:37.188 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.188 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.188 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:37:37.188 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.542452+0000", 2024-09-16T09:37:37.188 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:37:37.189 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.189 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.189 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:37:37.189 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.542453+0000", 2024-09-16T09:37:37.189 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.2589999999999999e-06 2024-09-16T09:37:37.189 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.189 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.189 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:37:37.189 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.542456+0000", 2024-09-16T09:37:37.189 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.022e-06 2024-09-16T09:37:37.189 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.189 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.189 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:37:37.190 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.542464+0000", 2024-09-16T09:37:37.190 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.8480000000000001e-06 2024-09-16T09:37:37.190 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.190 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.190 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:37:37.190 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.542469+0000", 2024-09-16T09:37:37.190 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.1159999999999998e-06 2024-09-16T09:37:37.190 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.190 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.190 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:37:37.190 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.542519+0000", 2024-09-16T09:37:37.190 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.0204999999999998e-05 2024-09-16T09:37:37.190 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.190 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.191 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:37:37.191 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.542529+0000", 2024-09-16T09:37:37.191 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.5540000000000003e-06 2024-09-16T09:37:37.191 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.191 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.191 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:37:37.191 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:21.527708+0000", 2024-09-16T09:37:37.191 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.98517922700000005 2024-09-16T09:37:37.191 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.191 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.191 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:37:37.191 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:21.527716+0000", 2024-09-16T09:37:37.191 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.4600000000000003e-06 2024-09-16T09:37:37.192 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.192 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.192 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:37:37.192 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:22.638211+0000", 2024-09-16T09:37:37.192 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.110494358 2024-09-16T09:37:37.192 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.192 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.192 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:37:37.192 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:22.638332+0000", 2024-09-16T09:37:37.192 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000120921 2024-09-16T09:37:37.192 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.192 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.192 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 1,3,5,6", 2024-09-16T09:37:37.192 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:22.638679+0000", 2024-09-16T09:37:37.193 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000347573 2024-09-16T09:37:37.193 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.193 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.193 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:37:37.193 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:22.639748+0000", 2024-09-16T09:37:37.193 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.001068544 2024-09-16T09:37:37.193 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.193 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.193 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:37.193 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:22.640935+0000", 2024-09-16T09:37:37.193 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0011872040000000001 2024-09-16T09:37:37.193 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.193 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.193 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:37.194 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:22.642759+0000", 2024-09-16T09:37:37.194 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.001823964 2024-09-16T09:37:37.194 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.194 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.194 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:37.194 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:22.643538+0000", 2024-09-16T09:37:37.194 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00077939100000000005 2024-09-16T09:37:37.194 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.194 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.194 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:37.194 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:22.643927+0000", 2024-09-16T09:37:37.194 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000388279 2024-09-16T09:37:37.194 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.194 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.195 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:37:37.195 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:22.643997+0000", 2024-09-16T09:37:37.195 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.0118999999999997e-05 2024-09-16T09:37:37.195 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.195 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.195 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:37:37.195 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:22.644123+0000", 2024-09-16T09:37:37.195 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000125788 2024-09-16T09:37:37.195 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:37:37.195 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:37:37.195 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:37:37.195 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.195 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.195 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:232746 9.13 9:ca02ffd2:::benchmark_data_smithi070_137000_object111035:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e359)", 2024-09-16T09:37:37.196 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:37:20.543482+0000", 2024-09-16T09:37:37.196 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 16.521645122999999, 2024-09-16T09:37:37.196 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.1008667710000002, 2024-09-16T09:37:37.196 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:37:37.196 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:37:37.196 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:37:37.196 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:37:37.196 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:37:37.196 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:37:37.196 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 232746 2024-09-16T09:37:37.196 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.196 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:37:37.196 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.196 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:37:37.197 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.543482+0000", 2024-09-16T09:37:37.197 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:37:37.197 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.197 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.197 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:37:37.197 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.543482+0000", 2024-09-16T09:37:37.197 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:37:37.197 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.197 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.197 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:37:37.197 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.543483+0000", 2024-09-16T09:37:37.197 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.3699999999999999e-07 2024-09-16T09:37:37.197 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.197 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.198 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:37:37.198 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.543486+0000", 2024-09-16T09:37:37.198 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.1259999999999998e-06 2024-09-16T09:37:37.198 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.198 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.198 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:37:37.198 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.543494+0000", 2024-09-16T09:37:37.198 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.5490000000000006e-06 2024-09-16T09:37:37.198 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.198 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.198 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:37:37.198 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.543500+0000", 2024-09-16T09:37:37.198 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.7119999999999997e-06 2024-09-16T09:37:37.199 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.199 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.199 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:37:37.199 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.543607+0000", 2024-09-16T09:37:37.199 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000106889 2024-09-16T09:37:37.199 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.199 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.199 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:37:37.199 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.543616+0000", 2024-09-16T09:37:37.199 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.8810000000000002e-06 2024-09-16T09:37:37.199 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.199 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.199 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:37:37.199 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:21.527778+0000", 2024-09-16T09:37:37.200 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.98416193200000002 2024-09-16T09:37:37.200 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.200 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.200 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:37:37.200 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:21.527786+0000", 2024-09-16T09:37:37.200 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.8839999999999994e-06 2024-09-16T09:37:37.200 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.200 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.200 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:37:37.200 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:22.639169+0000", 2024-09-16T09:37:37.200 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1113836070000001 2024-09-16T09:37:37.200 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.200 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.201 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:37:37.201 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:22.639282+0000", 2024-09-16T09:37:37.201 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000113257 2024-09-16T09:37:37.201 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.201 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.201 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 1,3,5,6", 2024-09-16T09:37:37.201 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:22.639397+0000", 2024-09-16T09:37:37.201 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00011454400000000001 2024-09-16T09:37:37.201 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.201 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.201 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:37:37.201 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:22.640484+0000", 2024-09-16T09:37:37.201 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00108669 2024-09-16T09:37:37.201 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.202 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.202 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:37.202 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:22.642805+0000", 2024-09-16T09:37:37.202 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0023214920000000001 2024-09-16T09:37:37.202 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.202 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.202 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:37.202 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:22.643348+0000", 2024-09-16T09:37:37.202 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00054297099999999995 2024-09-16T09:37:37.202 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.202 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.202 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:37.202 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:22.643580+0000", 2024-09-16T09:37:37.203 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00023155699999999999 2024-09-16T09:37:37.203 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.203 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.203 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:37.203 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:22.644170+0000", 2024-09-16T09:37:37.203 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00059059100000000001 2024-09-16T09:37:37.203 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.203 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.203 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:37:37.203 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:22.644221+0000", 2024-09-16T09:37:37.203 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.0899000000000002e-05 2024-09-16T09:37:37.203 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.203 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.203 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:37:37.204 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:22.644349+0000", 2024-09-16T09:37:37.204 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000127353 2024-09-16T09:37:37.204 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:37:37.204 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:37:37.204 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:37:37.204 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.204 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.204 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:232751 9.13 9:cb4950ef:::benchmark_data_smithi070_137000_object111040:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e359)", 2024-09-16T09:37:37.204 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:37:20.546747+0000", 2024-09-16T09:37:37.204 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 16.518380242999999, 2024-09-16T09:37:37.204 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.0978242109999998, 2024-09-16T09:37:37.204 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:37:37.204 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:37:37.205 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:37:37.205 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:37:37.205 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:37:37.205 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:37:37.205 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 232751 2024-09-16T09:37:37.205 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.205 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:37:37.205 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.205 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:37:37.205 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.546747+0000", 2024-09-16T09:37:37.205 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:37:37.205 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.205 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.205 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:37:37.206 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.546747+0000", 2024-09-16T09:37:37.206 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:37:37.206 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.206 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.206 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:37:37.206 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.546748+0000", 2024-09-16T09:37:37.206 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.5060000000000001e-06 2024-09-16T09:37:37.206 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.206 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.206 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:37:37.206 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.546753+0000", 2024-09-16T09:37:37.206 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.9690000000000002e-06 2024-09-16T09:37:37.207 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.207 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.207 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:37:37.207 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.546766+0000", 2024-09-16T09:37:37.207 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.3193999999999999e-05 2024-09-16T09:37:37.207 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.207 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.207 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:37:37.207 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.546774+0000", 2024-09-16T09:37:37.207 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.5460000000000003e-06 2024-09-16T09:37:37.207 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.207 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.207 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:37:37.207 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.546892+0000", 2024-09-16T09:37:37.208 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00011846899999999999 2024-09-16T09:37:37.208 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.208 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.208 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:37:37.208 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.546912+0000", 2024-09-16T09:37:37.208 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.9236999999999999e-05 2024-09-16T09:37:37.208 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.208 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.208 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:37:37.208 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:21.527821+0000", 2024-09-16T09:37:37.208 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.98090930899999995 2024-09-16T09:37:37.208 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.208 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.209 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:37:37.209 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:21.527829+0000", 2024-09-16T09:37:37.209 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.9049999999999997e-06 2024-09-16T09:37:37.209 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.209 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.209 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:37:37.209 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:22.639776+0000", 2024-09-16T09:37:37.209 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.111947405 2024-09-16T09:37:37.209 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.209 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.209 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:37:37.209 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:22.639884+0000", 2024-09-16T09:37:37.209 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00010733799999999999 2024-09-16T09:37:37.209 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.210 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.210 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 1,3,5,6", 2024-09-16T09:37:37.210 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:22.640001+0000", 2024-09-16T09:37:37.210 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000117915 2024-09-16T09:37:37.210 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.210 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.210 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:37:37.210 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:22.640707+0000", 2024-09-16T09:37:37.210 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00070530899999999999 2024-09-16T09:37:37.210 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.210 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.210 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:37.210 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:22.643487+0000", 2024-09-16T09:37:37.210 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.002780542 2024-09-16T09:37:37.211 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.211 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.211 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:37.211 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:22.643617+0000", 2024-09-16T09:37:37.211 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00012976400000000001 2024-09-16T09:37:37.211 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.211 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.211 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:37.211 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:22.643657+0000", 2024-09-16T09:37:37.211 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.0129999999999997e-05 2024-09-16T09:37:37.211 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.211 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.211 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:37.212 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:22.644392+0000", 2024-09-16T09:37:37.212 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00073488000000000004 2024-09-16T09:37:37.212 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.212 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.212 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:37:37.212 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:22.644449+0000", 2024-09-16T09:37:37.212 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.7259999999999997e-05 2024-09-16T09:37:37.212 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.212 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.212 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:37:37.212 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:22.644571+0000", 2024-09-16T09:37:37.212 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000121533 2024-09-16T09:37:37.212 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:37:37.212 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:37:37.213 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:37:37.213 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.213 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.213 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:232753 9.11 9:88059982:::benchmark_data_smithi070_137000_object111042:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e359)", 2024-09-16T09:37:37.213 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:37:20.548314+0000", 2024-09-16T09:37:37.213 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 16.516812628, 2024-09-16T09:37:37.213 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.98918844399999994, 2024-09-16T09:37:37.213 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:37:37.213 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:37:37.213 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:37:37.213 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:37:37.213 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:37:37.213 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:37:37.213 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 232753 2024-09-16T09:37:37.213 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.214 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:37:37.214 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.214 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:37:37.214 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.548314+0000", 2024-09-16T09:37:37.214 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:37:37.214 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.214 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.214 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:37:37.214 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.548314+0000", 2024-09-16T09:37:37.214 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:37:37.214 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.214 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.214 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:37:37.214 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.548315+0000", 2024-09-16T09:37:37.215 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.091e-06 2024-09-16T09:37:37.215 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.215 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.215 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:37:37.215 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.548320+0000", 2024-09-16T09:37:37.215 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.5739999999999997e-06 2024-09-16T09:37:37.215 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.215 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.215 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:37:37.215 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.548332+0000", 2024-09-16T09:37:37.215 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.2e-05 2024-09-16T09:37:37.215 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.215 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.215 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:37:37.216 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.548339+0000", 2024-09-16T09:37:37.216 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.878e-06 2024-09-16T09:37:37.216 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.216 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.216 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:37:37.216 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.548397+0000", 2024-09-16T09:37:37.216 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.8480999999999998e-05 2024-09-16T09:37:37.216 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.216 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.216 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:37:37.216 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.548406+0000", 2024-09-16T09:37:37.216 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.1050000000000001e-06 2024-09-16T09:37:37.216 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.216 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.216 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:37:37.217 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:21.530598+0000", 2024-09-16T09:37:37.217 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.98219196499999994 2024-09-16T09:37:37.217 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.217 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.217 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:37:37.217 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:21.530681+0000", 2024-09-16T09:37:37.217 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.2694999999999993e-05 2024-09-16T09:37:37.217 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.217 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.217 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 3,6", 2024-09-16T09:37:37.217 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:21.530750+0000", 2024-09-16T09:37:37.217 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.8931999999999997e-05 2024-09-16T09:37:37.217 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.218 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.218 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:37:37.218 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:21.531394+0000", 2024-09-16T09:37:37.218 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00064393100000000004 2024-09-16T09:37:37.218 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.218 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.218 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:37.218 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:21.531517+0000", 2024-09-16T09:37:37.218 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00012291400000000001 2024-09-16T09:37:37.218 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.218 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.218 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:37.218 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:21.537450+0000", 2024-09-16T09:37:37.218 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0059329969999999998 2024-09-16T09:37:37.219 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.219 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.219 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:37:37.219 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:21.537480+0000", 2024-09-16T09:37:37.219 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.0577000000000002e-05 2024-09-16T09:37:37.219 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:37:37.219 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:37:37.219 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:37:37.219 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:21.537503+0000", 2024-09-16T09:37:37.219 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.2303999999999999e-05 2024-09-16T09:37:37.219 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:37:37.219 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:37:37.219 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:37:37.219 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:37:37.220 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:37:37.220 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-09-16T09:37:37.221 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:37.221 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:37.221 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:37:37.322 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:37:37.344 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:37.345 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:37.345 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:37:37.362 INFO:teuthology.orchestra.run.smithi070.stderr:nodeep-scrub is set 2024-09-16T09:37:37.446 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:37:37.467 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:37.468 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:37.468 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:37:37.569 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd reweight 3 1 2024-09-16T09:37:37.569 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:37:37.591 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:37.591 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:37.592 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:37:37.656 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:37 smithi084 ceph-mon[32629]: pgmap v662: 178 pgs: 4 peering, 3 active+remapped+backfill_toofull, 171 active+clean; 807 KiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.04k op/s; 8/609 objects misplaced (1.314%) 2024-09-16T09:37:37.656 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:37 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-16T09:37:37.656 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:37 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/843350003' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_2", "pool2": "unique_pool_2", "yes_i_really_really_mean_it": true}]': finished 2024-09-16T09:37:37.656 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:37 smithi084 ceph-mon[32629]: osdmap e371: 8 total, 8 up, 8 in 2024-09-16T09:37:37.656 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:37 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3424318320' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:37:37.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:37 smithi070 ceph-mon[30388]: pgmap v662: 178 pgs: 4 peering, 3 active+remapped+backfill_toofull, 171 active+clean; 807 KiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.04k op/s; 8/609 objects misplaced (1.314%) 2024-09-16T09:37:37.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:37 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-16T09:37:37.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:37 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/843350003' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_2", "pool2": "unique_pool_2", "yes_i_really_really_mean_it": true}]': finished 2024-09-16T09:37:37.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:37 smithi070 ceph-mon[30388]: osdmap e371: 8 total, 8 up, 8 in 2024-09-16T09:37:37.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:37 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3424318320' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:37:37.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:37 smithi070 ceph-mon[40691]: pgmap v662: 178 pgs: 4 peering, 3 active+remapped+backfill_toofull, 171 active+clean; 807 KiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.04k op/s; 8/609 objects misplaced (1.314%) 2024-09-16T09:37:37.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:37 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-16T09:37:37.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:37 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/843350003' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_2", "pool2": "unique_pool_2", "yes_i_really_really_mean_it": true}]': finished 2024-09-16T09:37:37.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:37 smithi070 ceph-mon[40691]: osdmap e371: 8 total, 8 up, 8 in 2024-09-16T09:37:37.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:37 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3424318320' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:37:37.693 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:37:37.714 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:37.715 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:37.715 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:37:37.816 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:37:37.838 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:37.838 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:37.839 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:37:37.939 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:37:37.961 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:37.961 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:37.962 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:37:38.058 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.0 dump_ops_in_flight 2024-09-16T09:37:38.062 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:37:38.122 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:38.122 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:38.123 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:37:38.223 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:37:38.245 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:38.246 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:38.247 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:37:38.347 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:37:38.369 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:38.370 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:38.370 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:37:38.471 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:37:38.494 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:38.495 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:38.496 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:37:38.583 INFO:teuthology.task.full_sequential:In full_sequential, running task radosbench... 2024-09-16T09:37:38.584 INFO:tasks.radosbench:Beginning radosbench... 2024-09-16T09:37:38.584 INFO:tasks.cephadm.ceph_manager.ceph:creating pool_name unique_pool_3 2024-09-16T09:37:38.584 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd pool create unique_pool_3 16 2024-09-16T09:37:38.597 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:37:38.648 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:38.648 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:38.649 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:37:38.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:38 smithi084 ceph-mon[32629]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:37:38.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:38 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3424318320' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-16T09:37:38.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:38 smithi084 ceph-mon[32629]: osdmap e372: 8 total, 8 up, 8 in 2024-09-16T09:37:38.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:38 smithi070 ceph-mon[30388]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:37:38.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:38 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3424318320' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-16T09:37:38.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:38 smithi070 ceph-mon[30388]: osdmap e372: 8 total, 8 up, 8 in 2024-09-16T09:37:38.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:38 smithi070 ceph-mon[40691]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:37:38.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:38 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3424318320' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-16T09:37:38.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:38 smithi070 ceph-mon[40691]: osdmap e372: 8 total, 8 up, 8 in 2024-09-16T09:37:38.749 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:37:38.773 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:38.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:38.774 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:37:38.874 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:37:38.896 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:38.896 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:38.897 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:37:38.969 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:37:38.998 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:37:39.022 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:39.023 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:39.023 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:37:39.124 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:37:39.145 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:39.146 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:39.146 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:37:39.243 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:37:39.247 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:37:39.273 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:39.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:39.273 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:37:39.374 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:37:39.396 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:39.396 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:39.397 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:37:39.497 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:37:39.519 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:39.519 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:39.520 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:37:39.559 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:39 smithi070 ceph-mon[30388]: pgmap v665: 152 pgs: 2 peering, 150 active+clean; 583 KiB data, 4.3 GiB used, 711 GiB / 715 GiB avail 2024-09-16T09:37:39.559 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:39 smithi070 ceph-mon[30388]: Health check cleared: PG_BACKFILL_FULL (was: Low space hindering backfill (add storage if this doesn't resolve itself): 3 pgs backfill_toofull) 2024-09-16T09:37:39.559 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:39 smithi070 ceph-mon[30388]: osdmap e373: 8 total, 8 up, 8 in 2024-09-16T09:37:39.559 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:39 smithi070 ceph-mon[40691]: pgmap v665: 152 pgs: 2 peering, 150 active+clean; 583 KiB data, 4.3 GiB used, 711 GiB / 715 GiB avail 2024-09-16T09:37:39.560 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:39 smithi070 ceph-mon[40691]: Health check cleared: PG_BACKFILL_FULL (was: Low space hindering backfill (add storage if this doesn't resolve itself): 3 pgs backfill_toofull) 2024-09-16T09:37:39.560 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:39 smithi070 ceph-mon[40691]: osdmap e373: 8 total, 8 up, 8 in 2024-09-16T09:37:39.620 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:37:39.642 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:39.642 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:39.643 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:37:39.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:39 smithi084 ceph-mon[32629]: pgmap v665: 152 pgs: 2 peering, 150 active+clean; 583 KiB data, 4.3 GiB used, 711 GiB / 715 GiB avail 2024-09-16T09:37:39.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:39 smithi084 ceph-mon[32629]: Health check cleared: PG_BACKFILL_FULL (was: Low space hindering backfill (add storage if this doesn't resolve itself): 3 pgs backfill_toofull) 2024-09-16T09:37:39.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:39 smithi084 ceph-mon[32629]: osdmap e373: 8 total, 8 up, 8 in 2024-09-16T09:37:39.744 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:37:39.765 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:39.766 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:39.766 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:37:39.807 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:37:39.808 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:37:39.824 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:37:39.825 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:37:39.840 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:37:39.841 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:37:39.857 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:37:39.857 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:37:39.867 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:37:39.874 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:37:39.874 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:37:39.887 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:39.888 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:39.888 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:37:39.891 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:37:39.891 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:37:39.908 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:37:39.908 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:37:39.925 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:37:39.925 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:37:39.989 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:37:40.010 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:40.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:40.011 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:37:40.111 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:37:40.133 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:40.134 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:40.134 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:37:40.235 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:37:40.256 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:40.257 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:40.257 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:37:40.358 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:37:40.367 INFO:teuthology.orchestra.run.smithi070.stderr:reweighted osd.3 to 1 (10000) 2024-09-16T09:37:40.368 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:37:40.371 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:37:40.388 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:40.389 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:40.389 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:37:40.490 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:37:40.516 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:40.516 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:40.517 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:37:40.617 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:37:40.639 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:40.640 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:40.640 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:37:40.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:40 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/639470759' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:37:40.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:40 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/673739193' entity='client.admin' cmd={"prefix": "osd reweight", "id": 3, "weight": 1.0} : dispatch 2024-09-16T09:37:40.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:40 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 3, "weight": 1.0} : dispatch 2024-09-16T09:37:40.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:40 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/639470759' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:37:40.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:40 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/673739193' entity='client.admin' cmd={"prefix": "osd reweight", "id": 3, "weight": 1.0} : dispatch 2024-09-16T09:37:40.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:40 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 3, "weight": 1.0} : dispatch 2024-09-16T09:37:40.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:40 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/639470759' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:37:40.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:40 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/673739193' entity='client.admin' cmd={"prefix": "osd reweight", "id": 3, "weight": 1.0} : dispatch 2024-09-16T09:37:40.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:40 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 3, "weight": 1.0} : dispatch 2024-09-16T09:37:40.741 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:37:40.763 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:40.763 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:40.764 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:37:40.865 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:37:40.886 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:40.887 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:40.887 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:37:40.966 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd unset noscrub 2024-09-16T09:37:40.988 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:37:41.015 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:41.015 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:41.015 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:37:41.116 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:37:41.141 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:41.141 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:41.142 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:37:41.243 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:37:41.264 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:41.265 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:41.265 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:37:41.366 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:37:41.393 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:41.393 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:41.394 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:37:41.494 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:37:41.520 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:41.521 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:41.521 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:37:41.587 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:41 smithi070 ceph-mon[30388]: pgmap v667: 152 pgs: 152 active+clean; 583 KiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 932 B/s rd, 0 op/s 2024-09-16T09:37:41.588 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:41 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-16T09:37:41.588 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:41 smithi070 ceph-mon[30388]: osdmap e374: 8 total, 8 up, 8 in 2024-09-16T09:37:41.588 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:41 smithi070 ceph-mon[40691]: pgmap v667: 152 pgs: 152 active+clean; 583 KiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 932 B/s rd, 0 op/s 2024-09-16T09:37:41.588 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:41 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-16T09:37:41.588 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:41 smithi070 ceph-mon[40691]: osdmap e374: 8 total, 8 up, 8 in 2024-09-16T09:37:41.621 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:37:41.645 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:41.645 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:41.646 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:37:41.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:41 smithi084 ceph-mon[32629]: pgmap v667: 152 pgs: 152 active+clean; 583 KiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 932 B/s rd, 0 op/s 2024-09-16T09:37:41.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:41 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-16T09:37:41.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:41 smithi084 ceph-mon[32629]: osdmap e374: 8 total, 8 up, 8 in 2024-09-16T09:37:41.747 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:37:41.769 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:41.770 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:41.770 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:37:41.855 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T09:37:41.855 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:37:41.871 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:37:41.894 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:41.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:41.895 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:37:41.996 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:37:42.020 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:42.021 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:42.021 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:37:42.122 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:37:42.144 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:42.144 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:42.145 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:37:42.171 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:37:41 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:37:41] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:37:42.245 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:37:42.267 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:42.268 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:42.268 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:37:42.369 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:37:42.391 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:42.391 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:42.391 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:37:42.492 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:37:42.494 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:42 smithi070 ceph-mon[30388]: osdmap e375: 8 total, 8 up, 8 in 2024-09-16T09:37:42.494 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:42 smithi070 ceph-mon[40691]: osdmap e375: 8 total, 8 up, 8 in 2024-09-16T09:37:42.519 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:42.520 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:42.520 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:37:42.521 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd reweight 4 1 2024-09-16T09:37:42.621 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:37:42.643 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:42.644 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:42.644 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:37:42.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:42 smithi084 ceph-mon[32629]: osdmap e375: 8 total, 8 up, 8 in 2024-09-16T09:37:42.745 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:37:42.772 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:42.772 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:42.773 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:37:42.874 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:37:42.895 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:42.896 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:42.896 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:37:42.994 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.0 dump_blocked_ops 2024-09-16T09:37:42.997 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:37:43.022 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:43.023 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:43.023 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:37:43.124 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:37:43.147 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:43.147 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:43.148 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:37:43.157 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T09:37:43.248 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:37:43.270 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:43.271 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:43.271 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:37:43.334 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:37:43.341 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:37:43.373 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:37:43.386 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:37:43.397 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:43.397 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:43.398 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:37:43.499 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:37:43.521 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:43.522 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:43.522 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:37:43.623 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:37:43.639 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:43 smithi070 ceph-mon[40691]: pgmap v670: 152 pgs: 152 active+clean; 583 KiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 125 B/s, 3 objects/s recovering 2024-09-16T09:37:43.639 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:43 smithi070 ceph-mon[30388]: pgmap v670: 152 pgs: 152 active+clean; 583 KiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 125 B/s, 3 objects/s recovering 2024-09-16T09:37:43.647 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:43.647 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:43.648 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:37:43.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:43 smithi084 ceph-mon[32629]: pgmap v670: 152 pgs: 152 active+clean; 583 KiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 125 B/s, 3 objects/s recovering 2024-09-16T09:37:43.660 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:37:43 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:37:43.749 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:37:43.774 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:43.774 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:43.774 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:37:43.875 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:37:43.897 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:43.898 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:43.898 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:37:43.999 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:37:44.020 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:44.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:44.022 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:37:44.122 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:37:44.148 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:44.149 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:44.149 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:37:44.250 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:37:44.276 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:44.277 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:44.277 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:37:44.378 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:37:44.389 INFO:teuthology.orchestra.run.smithi070.stderr:reweighted osd.4 to 1 (10000) 2024-09-16T09:37:44.390 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'unique_pool_3' created 2024-09-16T09:37:44.409 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:44.410 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:44.411 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:37:44.511 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:37:44.533 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:44.534 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:44.534 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:37:44.635 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:37:44.656 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:44.656 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:44.657 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:37:44.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:44 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:37:44.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:44 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/867182341' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "unique_pool_3", "pg_num": 16} : dispatch 2024-09-16T09:37:44.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:44 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "unique_pool_3", "pg_num": 16} : dispatch 2024-09-16T09:37:44.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:44 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1682305734' entity='client.admin' cmd={"prefix": "osd reweight", "id": 4, "weight": 1.0} : dispatch 2024-09-16T09:37:44.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:44 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 4, "weight": 1.0} : dispatch 2024-09-16T09:37:44.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:44 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:37:44.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:44 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/867182341' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "unique_pool_3", "pg_num": 16} : dispatch 2024-09-16T09:37:44.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:44 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "unique_pool_3", "pg_num": 16} : dispatch 2024-09-16T09:37:44.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:44 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1682305734' entity='client.admin' cmd={"prefix": "osd reweight", "id": 4, "weight": 1.0} : dispatch 2024-09-16T09:37:44.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:44 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 4, "weight": 1.0} : dispatch 2024-09-16T09:37:44.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:44 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:37:44.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:44 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/867182341' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "unique_pool_3", "pg_num": 16} : dispatch 2024-09-16T09:37:44.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:44 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "unique_pool_3", "pg_num": 16} : dispatch 2024-09-16T09:37:44.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:44 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1682305734' entity='client.admin' cmd={"prefix": "osd reweight", "id": 4, "weight": 1.0} : dispatch 2024-09-16T09:37:44.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:44 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 4, "weight": 1.0} : dispatch 2024-09-16T09:37:44.758 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:37:44.780 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:44.781 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:44.781 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:37:44.882 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:37:44.904 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:44.904 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:44.905 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:37:44.986 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T09:37:44.986 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:37:45.006 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:37:45.027 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:45.028 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:45.028 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:37:45.129 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:37:45.155 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:45.156 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:45.156 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:37:45.257 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:37:45.280 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:45.281 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:45.281 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:37:45.382 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:37:45.389 INFO:teuthology.orchestra.run.smithi070.stderr:noscrub is unset 2024-09-16T09:37:45.411 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:45.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:45.412 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:37:45.513 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:37:45.538 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:45.538 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:45.539 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:37:45.640 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:37:45.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:45 smithi084 ceph-mon[32629]: pgmap v671: 152 pgs: 152 active+clean; 583 KiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s; 73 B/s, 1 objects/s recovering 2024-09-16T09:37:45.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:45 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_3", "pg_num": 16}]': finished 2024-09-16T09:37:45.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:45 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-16T09:37:45.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:45 smithi084 ceph-mon[32629]: osdmap e376: 8 total, 8 up, 8 in 2024-09-16T09:37:45.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:45 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3917409207' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:37:45.664 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:45.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:45.665 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:37:45.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:45 smithi070 ceph-mon[30388]: pgmap v671: 152 pgs: 152 active+clean; 583 KiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s; 73 B/s, 1 objects/s recovering 2024-09-16T09:37:45.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:45 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_3", "pg_num": 16}]': finished 2024-09-16T09:37:45.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:45 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-16T09:37:45.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:45 smithi070 ceph-mon[30388]: osdmap e376: 8 total, 8 up, 8 in 2024-09-16T09:37:45.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:45 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3917409207' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:37:45.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:45 smithi070 ceph-mon[40691]: pgmap v671: 152 pgs: 152 active+clean; 583 KiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s; 73 B/s, 1 objects/s recovering 2024-09-16T09:37:45.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:45 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_3", "pg_num": 16}]': finished 2024-09-16T09:37:45.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:45 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-16T09:37:45.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:45 smithi070 ceph-mon[40691]: osdmap e376: 8 total, 8 up, 8 in 2024-09-16T09:37:45.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:45 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3917409207' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:37:45.766 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:37:45.790 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:45.790 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:45.791 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:37:45.802 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.0 dump_historic_ops 2024-09-16T09:37:45.892 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:37:45.916 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:45.917 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:45.917 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:37:46.018 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:37:46.039 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:46.039 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:46.040 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:37:46.141 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:37:46.162 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:46.163 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:46.163 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:37:46.264 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:37:46.285 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:46.286 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:46.286 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:37:46.387 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:37:46.409 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:46.410 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:46.410 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:37:46.511 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:37:46.537 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:46.538 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:46.538 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:37:46.589 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:46 smithi070 ceph-mon[30388]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:37:46.589 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:46 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3917409207' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-16T09:37:46.589 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:46 smithi070 ceph-mon[30388]: osdmap e377: 8 total, 8 up, 8 in 2024-09-16T09:37:46.589 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:46 smithi070 ceph-mon[30388]: 6.5 scrub starts 2024-09-16T09:37:46.589 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:46 smithi070 ceph-mon[30388]: 6.5 scrub ok 2024-09-16T09:37:46.589 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:46 smithi070 ceph-mon[40691]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:37:46.590 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:46 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3917409207' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-16T09:37:46.590 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:46 smithi070 ceph-mon[40691]: osdmap e377: 8 total, 8 up, 8 in 2024-09-16T09:37:46.590 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:46 smithi070 ceph-mon[40691]: 6.5 scrub starts 2024-09-16T09:37:46.590 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:46 smithi070 ceph-mon[40691]: 6.5 scrub ok 2024-09-16T09:37:46.614 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd pool application enable unique_pool_3 rados --yes-i-really-mean-it || true 2024-09-16T09:37:46.638 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:37:46.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:46 smithi084 ceph-mon[32629]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:37:46.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:46 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3917409207' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-16T09:37:46.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:46 smithi084 ceph-mon[32629]: osdmap e377: 8 total, 8 up, 8 in 2024-09-16T09:37:46.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:46 smithi084 ceph-mon[32629]: 6.5 scrub starts 2024-09-16T09:37:46.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:46 smithi084 ceph-mon[32629]: 6.5 scrub ok 2024-09-16T09:37:46.679 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:46.679 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:46.680 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:37:46.780 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:37:46.802 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:46.803 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:46.803 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:37:46.904 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:37:46.930 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:46.931 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:46.931 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:37:47.032 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:37:47.054 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:47.055 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:47.055 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:37:47.156 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:37:47.177 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:47.177 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:47.178 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:37:47.279 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:37:47.304 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:47.305 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:47.305 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:37:47.406 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:37:47.431 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:47.432 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:47.432 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:37:47.533 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:37:47.555 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:47.555 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:47.555 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:37:47.656 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:37:47.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:47 smithi084 ceph-mon[32629]: pgmap v674: 168 pgs: 16 unknown, 152 active+clean; 583 KiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 950 B/s rd, 0 op/s 2024-09-16T09:37:47.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:47 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:37:47.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:47 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:37:47.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:47 smithi084 ceph-mon[32629]: 5.c scrub starts 2024-09-16T09:37:47.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:47 smithi084 ceph-mon[32629]: 5.c scrub ok 2024-09-16T09:37:47.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:47 smithi084 ceph-mon[32629]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T09:37:47.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:47 smithi084 ceph-mon[32629]: osdmap e378: 8 total, 8 up, 8 in 2024-09-16T09:37:47.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:47 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:37:47.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:47 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:37:47.670 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:47 smithi070 ceph-mon[40691]: pgmap v674: 168 pgs: 16 unknown, 152 active+clean; 583 KiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 950 B/s rd, 0 op/s 2024-09-16T09:37:47.670 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:47 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:37:47.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:47 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:37:47.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:47 smithi070 ceph-mon[40691]: 5.c scrub starts 2024-09-16T09:37:47.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:47 smithi070 ceph-mon[40691]: 5.c scrub ok 2024-09-16T09:37:47.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:47 smithi070 ceph-mon[40691]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T09:37:47.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:47 smithi070 ceph-mon[40691]: osdmap e378: 8 total, 8 up, 8 in 2024-09-16T09:37:47.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:47 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:37:47.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:47 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:37:47.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:47 smithi070 ceph-mon[30388]: pgmap v674: 168 pgs: 16 unknown, 152 active+clean; 583 KiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 950 B/s rd, 0 op/s 2024-09-16T09:37:47.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:47 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:37:47.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:47 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:37:47.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:47 smithi070 ceph-mon[30388]: 5.c scrub starts 2024-09-16T09:37:47.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:47 smithi070 ceph-mon[30388]: 5.c scrub ok 2024-09-16T09:37:47.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:47 smithi070 ceph-mon[30388]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T09:37:47.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:47 smithi070 ceph-mon[30388]: osdmap e378: 8 total, 8 up, 8 in 2024-09-16T09:37:47.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:47 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:37:47.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:47 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:37:47.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:47.678 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:47.679 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:37:47.705 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd reweight 5 1 2024-09-16T09:37:47.741 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:37:47.741 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:37:47.745 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:37:47.779 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:37:47.804 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:47.805 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:47.805 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:37:47.906 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:37:47.931 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:47.932 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:47.932 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:37:48.033 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:37:48.056 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:48.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:48.057 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:37:48.158 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:37:48.180 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:48.180 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:48.181 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:37:48.281 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:37:48.303 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:48.303 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:48.304 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:37:48.405 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:37:48.426 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:48.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:48.427 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:37:48.528 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:37:48.557 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:48.557 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:48.557 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:37:48.558 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:37:48.558 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:37:48.559 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:48 smithi070 ceph-mon[40691]: 6.a scrub starts 2024-09-16T09:37:48.559 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:48 smithi070 ceph-mon[40691]: 6.a scrub ok 2024-09-16T09:37:48.559 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:48 smithi070 ceph-mon[40691]: osdmap e379: 8 total, 8 up, 8 in 2024-09-16T09:37:48.559 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:48 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:37:48.559 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:48 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:37:48.559 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:48 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:37:48.559 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:48 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:37:48.559 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:48 smithi070 ceph-mon[30388]: 6.a scrub starts 2024-09-16T09:37:48.559 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:48 smithi070 ceph-mon[30388]: 6.a scrub ok 2024-09-16T09:37:48.559 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:48 smithi070 ceph-mon[30388]: osdmap e379: 8 total, 8 up, 8 in 2024-09-16T09:37:48.559 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:48 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:37:48.560 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:48 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:37:48.560 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:48 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:37:48.560 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:48 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:37:48.576 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:37:48.576 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:37:48.593 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:37:48.593 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:37:48.609 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:37:48.609 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:37:48.626 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:37:48.626 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:37:48.643 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:37:48.644 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:37:48.658 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:37:48.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:48 smithi084 ceph-mon[32629]: 6.a scrub starts 2024-09-16T09:37:48.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:48 smithi084 ceph-mon[32629]: 6.a scrub ok 2024-09-16T09:37:48.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:48 smithi084 ceph-mon[32629]: osdmap e379: 8 total, 8 up, 8 in 2024-09-16T09:37:48.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:48 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:37:48.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:48 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:37:48.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:48 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:37:48.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:48 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:37:48.661 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:37:48.662 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:37:48.680 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:37:48.680 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:37:48.680 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:48.681 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:48.681 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:37:48.782 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:37:48.804 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:48.805 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:48.805 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:37:48.906 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:37:48.927 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:48.928 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:48.928 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:37:49.029 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:37:49.050 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:49.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:49.051 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:37:49.063 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-16T09:37:49.152 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:37:49.174 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:49.175 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:49.175 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:37:49.276 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:37:49.302 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:49.303 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:49.303 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:37:49.404 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:37:49.426 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:49.427 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:49.427 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:37:49.528 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:37:49.554 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:49.554 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:49.554 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:37:49.656 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:37:49.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:49 smithi084 ceph-mon[32629]: pgmap v677: 168 pgs: 10 unknown, 158 active+clean; 583 KiB data, 4.3 GiB used, 711 GiB / 715 GiB avail 2024-09-16T09:37:49.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:49 smithi084 ceph-mon[32629]: osdmap e380: 8 total, 8 up, 8 in 2024-09-16T09:37:49.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:49 smithi084 ceph-mon[32629]: 6.1c scrub starts 2024-09-16T09:37:49.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:49 smithi084 ceph-mon[32629]: 6.1c scrub ok 2024-09-16T09:37:49.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:49 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/462278507' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:37:49.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:49 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:37:49.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:49 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:37:49.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:49 smithi084 ceph-mon[32629]: osdmap e381: 8 total, 8 up, 8 in 2024-09-16T09:37:49.670 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:49 smithi070 ceph-mon[40691]: pgmap v677: 168 pgs: 10 unknown, 158 active+clean; 583 KiB data, 4.3 GiB used, 711 GiB / 715 GiB avail 2024-09-16T09:37:49.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:49 smithi070 ceph-mon[40691]: osdmap e380: 8 total, 8 up, 8 in 2024-09-16T09:37:49.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:49 smithi070 ceph-mon[40691]: 6.1c scrub starts 2024-09-16T09:37:49.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:49 smithi070 ceph-mon[40691]: 6.1c scrub ok 2024-09-16T09:37:49.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:49 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/462278507' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:37:49.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:49 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:37:49.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:49 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:37:49.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:49 smithi070 ceph-mon[40691]: osdmap e381: 8 total, 8 up, 8 in 2024-09-16T09:37:49.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:49 smithi070 ceph-mon[30388]: pgmap v677: 168 pgs: 10 unknown, 158 active+clean; 583 KiB data, 4.3 GiB used, 711 GiB / 715 GiB avail 2024-09-16T09:37:49.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:49 smithi070 ceph-mon[30388]: osdmap e380: 8 total, 8 up, 8 in 2024-09-16T09:37:49.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:49 smithi070 ceph-mon[30388]: 6.1c scrub starts 2024-09-16T09:37:49.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:49 smithi070 ceph-mon[30388]: 6.1c scrub ok 2024-09-16T09:37:49.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:49 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/462278507' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:37:49.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:49 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:37:49.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:49 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:37:49.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:49 smithi070 ceph-mon[30388]: osdmap e381: 8 total, 8 up, 8 in 2024-09-16T09:37:49.677 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:49.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:49.678 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:37:49.780 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:37:49.807 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:49.807 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:49.808 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:37:49.909 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:37:49.932 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:49.932 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:49.933 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:37:49.941 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T09:37:49.941 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 20, 2024-09-16T09:37:49.942 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 600, 2024-09-16T09:37:49.942 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [ 2024-09-16T09:37:49.942 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.942 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:56268 9.8 9:1675f6ad:::benchmark_data_smithi070_137000_object56267:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e247)", 2024-09-16T09:37:49.942 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:33:59.290075+0000", 2024-09-16T09:37:49.942 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 230.49611854400001, 2024-09-16T09:37:49.942 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.1582197380000001, 2024-09-16T09:37:49.942 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:37:49.942 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:37:49.942 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T09:37:49.942 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:37:49.942 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:37:49.942 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:37:49.943 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 56268 2024-09-16T09:37:49.943 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.943 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:37:49.943 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.943 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:37:49.943 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.290075+0000", 2024-09-16T09:37:49.943 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:37:49.943 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.943 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.943 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:37:49.943 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.290075+0000", 2024-09-16T09:37:49.943 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:37:49.943 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.943 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.944 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:37:49.944 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.290076+0000", 2024-09-16T09:37:49.944 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.655e-06 2024-09-16T09:37:49.944 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.944 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.944 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:37:49.944 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.290105+0000", 2024-09-16T09:37:49.944 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.9164000000000001e-05 2024-09-16T09:37:49.944 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.944 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.944 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:37:49.944 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.290119+0000", 2024-09-16T09:37:49.944 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.4029000000000001e-05 2024-09-16T09:37:49.945 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.945 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.945 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:37:49.945 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.290128+0000", 2024-09-16T09:37:49.945 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.7509999999999997e-06 2024-09-16T09:37:49.945 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.945 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.945 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:37:49.945 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.294353+0000", 2024-09-16T09:37:49.946 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.004224953 2024-09-16T09:37:49.946 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.946 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.946 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:37:49.946 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.294478+0000", 2024-09-16T09:37:49.946 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00012500599999999999 2024-09-16T09:37:49.946 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.946 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.946 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 3,5", 2024-09-16T09:37:49.946 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.294612+0000", 2024-09-16T09:37:49.946 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000133566 2024-09-16T09:37:49.946 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.946 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.946 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:49.947 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.304980+0000", 2024-09-16T09:37:49.947 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.010368376 2024-09-16T09:37:49.947 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.947 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.947 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:49.947 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.309978+0000", 2024-09-16T09:37:49.947 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.004998176 2024-09-16T09:37:49.947 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.947 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.948 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:37:49.948 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.324183+0000", 2024-09-16T09:37:49.948 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.014205044 2024-09-16T09:37:49.948 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.948 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.948 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:37:49.948 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.324200+0000", 2024-09-16T09:37:49.948 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6597999999999999e-05 2024-09-16T09:37:49.948 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.948 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.948 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:37:49.948 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:00.329393+0000", 2024-09-16T09:37:49.948 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0051930309999999 2024-09-16T09:37:49.949 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.949 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.949 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:37:49.949 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:00.329404+0000", 2024-09-16T09:37:49.949 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1428e-05 2024-09-16T09:37:49.949 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.949 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.949 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:37:49.949 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.443818+0000", 2024-09-16T09:37:49.950 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1144139829999999 2024-09-16T09:37:49.950 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.950 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.950 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for degraded object", 2024-09-16T09:37:49.950 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.444263+0000", 2024-09-16T09:37:49.950 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000444785 2024-09-16T09:37:49.950 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.950 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.950 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:37:49.950 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.448204+0000", 2024-09-16T09:37:49.950 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0039411899999999998 2024-09-16T09:37:49.950 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.950 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.950 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:37:49.951 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.448294+0000", 2024-09-16T09:37:49.951 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.0002999999999999e-05 2024-09-16T09:37:49.951 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:37:49.951 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:37:49.951 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:37:49.951 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.951 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.951 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:56272 9.8 9:16bf4ef5:::benchmark_data_smithi070_137000_object56271:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e247)", 2024-09-16T09:37:49.951 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:33:59.294216+0000", 2024-09-16T09:37:49.952 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 230.491977152, 2024-09-16T09:37:49.952 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.1564348999999998, 2024-09-16T09:37:49.952 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:37:49.952 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:37:49.952 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T09:37:49.952 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:37:49.953 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:37:49.953 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:37:49.953 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 56272 2024-09-16T09:37:49.953 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.953 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:37:49.953 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.953 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:37:49.954 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.294216+0000", 2024-09-16T09:37:49.954 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:37:49.954 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.954 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.954 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:37:49.954 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.294216+0000", 2024-09-16T09:37:49.954 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:37:49.955 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.955 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.955 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:37:49.955 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.294216+0000", 2024-09-16T09:37:49.955 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.7999999999999996e-07 2024-09-16T09:37:49.955 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.955 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.956 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:37:49.956 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.294235+0000", 2024-09-16T09:37:49.956 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.821e-05 2024-09-16T09:37:49.956 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.956 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.956 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:37:49.956 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.294241+0000", 2024-09-16T09:37:49.956 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.3929999999999996e-06 2024-09-16T09:37:49.957 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.957 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.957 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:37:49.957 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.294246+0000", 2024-09-16T09:37:49.957 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0640000000000001e-06 2024-09-16T09:37:49.957 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.958 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.958 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:37:49.958 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.305514+0000", 2024-09-16T09:37:49.958 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.01126776 2024-09-16T09:37:49.958 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.958 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.958 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:37:49.959 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.307083+0000", 2024-09-16T09:37:49.959 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0015691730000000001 2024-09-16T09:37:49.959 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.959 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.959 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 3,5", 2024-09-16T09:37:49.959 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.307248+0000", 2024-09-16T09:37:49.960 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000164897 2024-09-16T09:37:49.960 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.960 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.960 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:49.960 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.310953+0000", 2024-09-16T09:37:49.960 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.003705205 2024-09-16T09:37:49.960 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.960 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.961 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:49.961 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.316630+0000", 2024-09-16T09:37:49.961 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0056768859999999999 2024-09-16T09:37:49.961 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.961 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.961 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:37:49.961 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.324233+0000", 2024-09-16T09:37:49.961 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0076027580000000003 2024-09-16T09:37:49.961 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.961 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.961 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:37:49.961 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.324240+0000", 2024-09-16T09:37:49.961 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.4070000000000004e-06 2024-09-16T09:37:49.962 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.962 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.962 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:37:49.962 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:00.329438+0000", 2024-09-16T09:37:49.962 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0051972819999999 2024-09-16T09:37:49.962 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.962 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.962 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:37:49.962 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:00.329447+0000", 2024-09-16T09:37:49.962 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.3770000000000002e-06 2024-09-16T09:37:49.962 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.962 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.962 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:37:49.962 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.445367+0000", 2024-09-16T09:37:49.963 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.115919661 2024-09-16T09:37:49.963 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.963 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.963 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for degraded object", 2024-09-16T09:37:49.963 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.445830+0000", 2024-09-16T09:37:49.963 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00046355399999999997 2024-09-16T09:37:49.963 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.963 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.963 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:37:49.963 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.450558+0000", 2024-09-16T09:37:49.963 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0047278470000000003 2024-09-16T09:37:49.963 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.963 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.964 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:37:49.964 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.450651+0000", 2024-09-16T09:37:49.964 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.2946000000000001e-05 2024-09-16T09:37:49.964 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:37:49.964 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:37:49.964 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:37:49.964 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.964 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.964 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:56291 9.8 9:154da08f:::benchmark_data_smithi070_137000_object56290:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e248)", 2024-09-16T09:37:49.964 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:33:59.336366+0000", 2024-09-16T09:37:49.964 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 230.44982762800001, 2024-09-16T09:37:49.964 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.1217081750000002, 2024-09-16T09:37:49.964 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:37:49.965 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:37:49.965 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:37:49.965 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:37:49.965 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:37:49.965 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:37:49.965 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 56291 2024-09-16T09:37:49.965 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.965 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:37:49.965 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.965 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:37:49.965 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.336366+0000", 2024-09-16T09:37:49.965 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:37:49.965 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.966 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.966 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:37:49.966 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.336366+0000", 2024-09-16T09:37:49.966 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:37:49.966 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.966 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.966 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:37:49.966 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.336367+0000", 2024-09-16T09:37:49.967 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9e-06 2024-09-16T09:37:49.967 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.967 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.967 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:37:49.967 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.336400+0000", 2024-09-16T09:37:49.967 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2264000000000001e-05 2024-09-16T09:37:49.967 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.967 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.967 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:37:49.967 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.336413+0000", 2024-09-16T09:37:49.967 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3414999999999999e-05 2024-09-16T09:37:49.967 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.967 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.968 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:37:49.968 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.336421+0000", 2024-09-16T09:37:49.968 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.3280000000000006e-06 2024-09-16T09:37:49.968 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.968 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.968 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:37:49.968 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.336954+0000", 2024-09-16T09:37:49.968 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00053217100000000001 2024-09-16T09:37:49.968 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.968 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.969 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:37:49.969 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.336964+0000", 2024-09-16T09:37:49.969 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0788000000000001e-05 2024-09-16T09:37:49.969 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.969 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.969 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:37:49.969 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:00.329472+0000", 2024-09-16T09:37:49.969 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.992507892 2024-09-16T09:37:49.969 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.969 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.970 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:37:49.970 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:00.329479+0000", 2024-09-16T09:37:49.970 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.0949999999999998e-06 2024-09-16T09:37:49.970 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.970 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.970 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:37:49.970 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.450681+0000", 2024-09-16T09:37:49.970 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1212013789999999 2024-09-16T09:37:49.970 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.970 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.970 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:37:49.970 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.450820+0000", 2024-09-16T09:37:49.970 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00013962100000000001 2024-09-16T09:37:49.970 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.971 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.971 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,7", 2024-09-16T09:37:49.971 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.450994+0000", 2024-09-16T09:37:49.971 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00017363099999999999 2024-09-16T09:37:49.971 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.971 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.971 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:49.971 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.456555+0000", 2024-09-16T09:37:49.971 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0055610900000000003 2024-09-16T09:37:49.971 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.971 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.971 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:49.972 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.456690+0000", 2024-09-16T09:37:49.972 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000134957 2024-09-16T09:37:49.972 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.972 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.972 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:37:49.972 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.456706+0000", 2024-09-16T09:37:49.972 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5968999999999999e-05 2024-09-16T09:37:49.972 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.972 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.972 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:37:49.972 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.456755+0000", 2024-09-16T09:37:49.973 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.8538000000000002e-05 2024-09-16T09:37:49.973 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.973 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.973 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:37:49.973 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.458074+0000", 2024-09-16T09:37:49.973 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001319137 2024-09-16T09:37:49.973 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:37:49.973 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:37:49.973 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:37:49.973 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.973 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.973 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:56320 9.8 9:10470d54:::benchmark_data_smithi070_137000_object56319:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e248)", 2024-09-16T09:37:49.974 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:33:59.372799+0000", 2024-09-16T09:37:49.974 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 230.413393899, 2024-09-16T09:37:49.974 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0883279570000002, 2024-09-16T09:37:49.974 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:37:49.974 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:37:49.974 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:37:49.974 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:37:49.974 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:37:49.974 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:37:49.974 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 56320 2024-09-16T09:37:49.974 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.974 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:37:49.974 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.974 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:37:49.975 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.372799+0000", 2024-09-16T09:37:49.975 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:37:49.975 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.975 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.975 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:37:49.975 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.372799+0000", 2024-09-16T09:37:49.975 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:37:49.975 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.975 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.975 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:37:49.975 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.372801+0000", 2024-09-16T09:37:49.975 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.269e-06 2024-09-16T09:37:49.975 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.975 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.976 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:37:49.976 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.372824+0000", 2024-09-16T09:37:49.976 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3258999999999999e-05 2024-09-16T09:37:49.976 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.976 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.976 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:37:49.976 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.372840+0000", 2024-09-16T09:37:49.976 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5809999999999999e-05 2024-09-16T09:37:49.976 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.976 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.976 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:37:49.976 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.372847+0000", 2024-09-16T09:37:49.977 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.7139999999999993e-06 2024-09-16T09:37:49.977 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.977 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.977 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:37:49.977 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.373173+0000", 2024-09-16T09:37:49.977 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00032524399999999998 2024-09-16T09:37:49.977 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.977 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.977 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:37:49.977 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.373179+0000", 2024-09-16T09:37:49.977 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.5309999999999998e-06 2024-09-16T09:37:49.977 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.977 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.978 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:37:49.978 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:00.329503+0000", 2024-09-16T09:37:49.978 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.95632355099999999 2024-09-16T09:37:49.978 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.978 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.978 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:37:49.978 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:00.329510+0000", 2024-09-16T09:37:49.978 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.0759999999999997e-06 2024-09-16T09:37:49.978 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.978 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.978 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:37:49.978 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.452119+0000", 2024-09-16T09:37:49.978 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.122609137 2024-09-16T09:37:49.979 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.979 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.979 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:37:49.979 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.452252+0000", 2024-09-16T09:37:49.979 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00013358599999999999 2024-09-16T09:37:49.979 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.979 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.979 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,7", 2024-09-16T09:37:49.979 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.452410+0000", 2024-09-16T09:37:49.979 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000157508 2024-09-16T09:37:49.979 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.979 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.979 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:37:49.979 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.458086+0000", 2024-09-16T09:37:49.980 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.005675785 2024-09-16T09:37:49.980 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.980 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.980 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:49.980 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.459588+0000", 2024-09-16T09:37:49.980 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0015021780000000001 2024-09-16T09:37:49.980 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.980 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.980 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:49.980 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.459644+0000", 2024-09-16T09:37:49.980 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.588e-05 2024-09-16T09:37:49.980 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.980 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.981 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:37:49.981 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.459704+0000", 2024-09-16T09:37:49.981 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.0590000000000001e-05 2024-09-16T09:37:49.981 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.981 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.981 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:37:49.981 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.461127+0000", 2024-09-16T09:37:49.981 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001422839 2024-09-16T09:37:49.984 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:37:49.984 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:37:49.984 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:37:49.984 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.984 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.984 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:56331 9.8 9:15f0826d:::benchmark_data_smithi070_137000_object56330:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e248)", 2024-09-16T09:37:49.984 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:33:59.382251+0000", 2024-09-16T09:37:49.984 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 230.403942379, 2024-09-16T09:37:49.984 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0852117350000001, 2024-09-16T09:37:49.985 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:37:49.985 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:37:49.985 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:37:49.985 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:37:49.985 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:37:49.985 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:37:49.985 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 56331 2024-09-16T09:37:49.985 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.986 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:37:49.986 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.986 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:37:49.986 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.382251+0000", 2024-09-16T09:37:49.986 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:37:49.986 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.986 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.986 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:37:49.986 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.382251+0000", 2024-09-16T09:37:49.986 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:37:49.986 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.986 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.986 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:37:49.987 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.382253+0000", 2024-09-16T09:37:49.987 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.84e-06 2024-09-16T09:37:49.987 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.987 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.987 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:37:49.987 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.382275+0000", 2024-09-16T09:37:49.987 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.2112000000000001e-05 2024-09-16T09:37:49.987 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.987 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.987 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:37:49.987 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.382287+0000", 2024-09-16T09:37:49.987 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2593e-05 2024-09-16T09:37:49.987 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.988 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.988 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:37:49.988 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.382295+0000", 2024-09-16T09:37:49.988 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6040000000000004e-06 2024-09-16T09:37:49.988 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.988 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.988 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:37:49.988 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.382349+0000", 2024-09-16T09:37:49.988 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4032e-05 2024-09-16T09:37:49.988 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.988 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.988 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:37:49.989 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.382358+0000", 2024-09-16T09:37:49.989 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.789e-06 2024-09-16T09:37:49.989 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.989 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.989 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:37:49.989 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:00.329532+0000", 2024-09-16T09:37:49.989 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.94717437299999996 2024-09-16T09:37:49.989 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.989 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.989 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:37:49.989 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:00.329539+0000", 2024-09-16T09:37:49.989 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.9999999999999999e-06 2024-09-16T09:37:49.989 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.990 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.990 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:37:49.990 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.453657+0000", 2024-09-16T09:37:49.990 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1241178030000001 2024-09-16T09:37:49.990 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.990 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.990 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:37:49.990 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.453811+0000", 2024-09-16T09:37:49.990 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00015423800000000001 2024-09-16T09:37:49.990 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.990 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.990 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,7", 2024-09-16T09:37:49.990 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.453990+0000", 2024-09-16T09:37:49.990 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00017888499999999999 2024-09-16T09:37:49.991 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.991 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.991 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:49.991 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.465277+0000", 2024-09-16T09:37:49.991 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.011287346 2024-09-16T09:37:49.991 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.991 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.991 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:49.991 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.465324+0000", 2024-09-16T09:37:49.991 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.6168999999999998e-05 2024-09-16T09:37:49.991 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.991 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.991 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:37:49.991 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.466075+0000", 2024-09-16T09:37:49.992 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00075194300000000003 2024-09-16T09:37:49.992 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.992 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.992 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:37:49.992 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.466124+0000", 2024-09-16T09:37:49.992 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.8591000000000003e-05 2024-09-16T09:37:49.992 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.992 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.992 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:37:49.992 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.467463+0000", 2024-09-16T09:37:49.992 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0013384169999999999 2024-09-16T09:37:49.992 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:37:49.992 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:37:49.993 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:37:49.993 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.993 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.993 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:56335 9.8 9:17f26bc5:::benchmark_data_smithi070_137000_object56334:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e248)", 2024-09-16T09:37:49.993 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:33:59.387512+0000", 2024-09-16T09:37:49.993 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 230.398681205, 2024-09-16T09:37:49.993 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0818823069999999, 2024-09-16T09:37:49.993 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:37:49.993 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:37:49.993 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:37:49.993 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:37:49.993 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:37:49.993 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:37:49.994 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 56335 2024-09-16T09:37:49.994 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.994 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:37:49.994 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.994 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:37:49.994 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.387512+0000", 2024-09-16T09:37:49.994 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:37:49.994 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.994 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.994 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:37:49.994 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.387512+0000", 2024-09-16T09:37:49.994 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:37:49.995 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.995 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.995 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:37:49.995 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.387513+0000", 2024-09-16T09:37:49.995 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3009999999999999e-06 2024-09-16T09:37:49.995 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.995 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.995 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:37:49.995 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.387541+0000", 2024-09-16T09:37:49.995 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.7489000000000001e-05 2024-09-16T09:37:49.995 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.995 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.995 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:37:49.996 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.387552+0000", 2024-09-16T09:37:49.996 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1311000000000001e-05 2024-09-16T09:37:49.996 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.996 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.996 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:37:49.996 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.387560+0000", 2024-09-16T09:37:49.996 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6090000000000003e-06 2024-09-16T09:37:49.996 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.996 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.996 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:37:49.996 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.387621+0000", 2024-09-16T09:37:49.996 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1826000000000001e-05 2024-09-16T09:37:49.996 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.997 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.997 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:37:49.997 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.389396+0000", 2024-09-16T09:37:49.997 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0017745560000000001 2024-09-16T09:37:49.997 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.997 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.997 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:37:49.997 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:00.329562+0000", 2024-09-16T09:37:49.997 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.94016562599999998 2024-09-16T09:37:49.997 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.997 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.997 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:37:49.997 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:00.329569+0000", 2024-09-16T09:37:49.998 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3270000000000003e-06 2024-09-16T09:37:49.998 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.998 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.998 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:37:49.998 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.455069+0000", 2024-09-16T09:37:49.998 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1255000989999999 2024-09-16T09:37:49.998 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.998 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.998 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:37:49.998 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.455197+0000", 2024-09-16T09:37:49.998 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00012834399999999999 2024-09-16T09:37:49.998 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.998 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.998 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,7", 2024-09-16T09:37:49.999 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.455353+0000", 2024-09-16T09:37:49.999 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000155308 2024-09-16T09:37:49.999 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.999 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.999 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:49.999 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.465366+0000", 2024-09-16T09:37:49.999 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.01001286 2024-09-16T09:37:49.999 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.999 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:49.999 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:49.999 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.465415+0000", 2024-09-16T09:37:49.999 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.8909000000000002e-05 2024-09-16T09:37:49.999 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:49.999 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.000 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:37:50.000 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.467474+0000", 2024-09-16T09:37:50.000 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0020590360000000002 2024-09-16T09:37:50.000 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.000 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.000 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:37:50.000 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.467527+0000", 2024-09-16T09:37:50.000 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.3908e-05 2024-09-16T09:37:50.000 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.000 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.000 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:37:50.000 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.469394+0000", 2024-09-16T09:37:50.000 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001866798 2024-09-16T09:37:50.001 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:37:50.001 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:37:50.001 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:37:50.001 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.001 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.001 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:56337 9.5 9:a00652f1:::benchmark_data_smithi070_137000_object56336:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e248)", 2024-09-16T09:37:50.001 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:33:59.390215+0000", 2024-09-16T09:37:50.002 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 230.39597846999999, 2024-09-16T09:37:50.002 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0362062170000002, 2024-09-16T09:37:50.002 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:37:50.002 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:37:50.002 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:37:50.002 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:37:50.002 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:37:50.002 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:37:50.002 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 56337 2024-09-16T09:37:50.003 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.003 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:37:50.003 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.003 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:37:50.003 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.390215+0000", 2024-09-16T09:37:50.003 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:37:50.003 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.003 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.003 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:37:50.003 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.390215+0000", 2024-09-16T09:37:50.003 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:37:50.003 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.003 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.003 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:37:50.004 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.390216+0000", 2024-09-16T09:37:50.004 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.4789999999999999e-06 2024-09-16T09:37:50.004 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.004 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.004 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:37:50.004 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.390239+0000", 2024-09-16T09:37:50.004 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.2495000000000001e-05 2024-09-16T09:37:50.004 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.004 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.004 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:37:50.004 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.390250+0000", 2024-09-16T09:37:50.004 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.114e-05 2024-09-16T09:37:50.004 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.004 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.005 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:37:50.005 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.390256+0000", 2024-09-16T09:37:50.005 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.3040000000000001e-06 2024-09-16T09:37:50.005 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.005 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.005 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:37:50.005 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.390312+0000", 2024-09-16T09:37:50.005 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5562000000000001e-05 2024-09-16T09:37:50.005 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.005 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.005 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:37:50.005 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.390321+0000", 2024-09-16T09:37:50.005 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.9090000000000006e-06 2024-09-16T09:37:50.005 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.006 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.006 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:37:50.006 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:00.325539+0000", 2024-09-16T09:37:50.006 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.93521816300000005 2024-09-16T09:37:50.006 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.006 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.006 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:37:50.006 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:00.325551+0000", 2024-09-16T09:37:50.006 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2512e-05 2024-09-16T09:37:50.006 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.006 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.006 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:37:50.006 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.421298+0000", 2024-09-16T09:37:50.006 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.095746436 2024-09-16T09:37:50.007 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.007 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.007 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:37:50.007 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.421439+0000", 2024-09-16T09:37:50.007 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000141519 2024-09-16T09:37:50.007 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.007 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.007 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,5", 2024-09-16T09:37:50.007 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.421596+0000", 2024-09-16T09:37:50.007 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00015686199999999999 2024-09-16T09:37:50.007 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.008 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.008 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:37:50.008 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.424192+0000", 2024-09-16T09:37:50.008 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0025960250000000001 2024-09-16T09:37:50.008 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.008 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.008 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:50.008 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.424254+0000", 2024-09-16T09:37:50.008 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1910999999999997e-05 2024-09-16T09:37:50.008 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.008 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.008 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:50.008 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.425261+0000", 2024-09-16T09:37:50.009 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001006514 2024-09-16T09:37:50.009 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.009 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.009 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:37:50.009 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.425323+0000", 2024-09-16T09:37:50.009 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.2603999999999999e-05 2024-09-16T09:37:50.009 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.009 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.009 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:37:50.009 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.426421+0000", 2024-09-16T09:37:50.009 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0010977820000000001 2024-09-16T09:37:50.009 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:37:50.009 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:37:50.010 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:37:50.010 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.010 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.010 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:56346 9.8 9:12468c50:::benchmark_data_smithi070_137000_object56345:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e248)", 2024-09-16T09:37:50.010 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:33:59.398468+0000", 2024-09-16T09:37:50.010 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 230.38772469899999, 2024-09-16T09:37:50.010 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0731202789999998, 2024-09-16T09:37:50.010 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:37:50.010 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:37:50.010 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:37:50.010 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:37:50.010 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:37:50.010 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:37:50.011 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 56346 2024-09-16T09:37:50.011 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.011 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:37:50.011 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.011 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:37:50.011 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.398468+0000", 2024-09-16T09:37:50.011 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:37:50.011 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.011 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.011 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:37:50.011 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.398468+0000", 2024-09-16T09:37:50.011 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:37:50.011 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.011 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.011 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:37:50.012 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.398470+0000", 2024-09-16T09:37:50.012 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.4330000000000001e-06 2024-09-16T09:37:50.012 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.012 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.012 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:37:50.012 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.398492+0000", 2024-09-16T09:37:50.012 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.1801e-05 2024-09-16T09:37:50.012 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.012 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.012 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:37:50.012 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.398502+0000", 2024-09-16T09:37:50.012 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0226e-05 2024-09-16T09:37:50.012 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.013 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.013 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:37:50.013 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.398508+0000", 2024-09-16T09:37:50.013 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1369999999999999e-06 2024-09-16T09:37:50.013 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.013 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.013 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:37:50.013 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.400785+0000", 2024-09-16T09:37:50.013 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0022771190000000002 2024-09-16T09:37:50.013 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.013 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.014 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:37:50.014 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.400795+0000", 2024-09-16T09:37:50.014 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.4870000000000008e-06 2024-09-16T09:37:50.014 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.014 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.014 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:37:50.014 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:00.329592+0000", 2024-09-16T09:37:50.014 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.92879704500000004 2024-09-16T09:37:50.014 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.014 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.014 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:37:50.014 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:00.329599+0000", 2024-09-16T09:37:50.014 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.0700000000000001e-06 2024-09-16T09:37:50.015 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.015 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.015 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:37:50.015 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.458122+0000", 2024-09-16T09:37:50.015 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1285228519999999 2024-09-16T09:37:50.015 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.015 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.015 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:37:50.015 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.458231+0000", 2024-09-16T09:37:50.015 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000109332 2024-09-16T09:37:50.015 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.015 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.015 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,7", 2024-09-16T09:37:50.015 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.458441+0000", 2024-09-16T09:37:50.016 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00021033299999999999 2024-09-16T09:37:50.016 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.016 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.016 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:50.016 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.469462+0000", 2024-09-16T09:37:50.016 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.011020914999999999 2024-09-16T09:37:50.016 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.016 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.016 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:37:50.016 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.469499+0000", 2024-09-16T09:37:50.016 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6566999999999997e-05 2024-09-16T09:37:50.016 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.016 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.017 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:50.017 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.469801+0000", 2024-09-16T09:37:50.017 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00030174599999999998 2024-09-16T09:37:50.017 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.017 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.017 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:37:50.017 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.470011+0000", 2024-09-16T09:37:50.017 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000210595 2024-09-16T09:37:50.017 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.017 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.017 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:37:50.017 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.471589+0000", 2024-09-16T09:37:50.017 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001577621 2024-09-16T09:37:50.017 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:37:50.018 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:37:50.018 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:37:50.018 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.018 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.018 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:56348 9.8 9:151f704f:::benchmark_data_smithi070_137000_object56347:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e248)", 2024-09-16T09:37:50.018 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:33:59.401792+0000", 2024-09-16T09:37:50.018 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 230.38440130399999, 2024-09-16T09:37:50.018 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0809194390000001, 2024-09-16T09:37:50.018 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:37:50.018 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:37:50.018 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:37:50.018 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:37:50.018 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:37:50.018 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:37:50.019 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 56348 2024-09-16T09:37:50.019 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.019 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:37:50.019 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.019 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:37:50.019 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.401792+0000", 2024-09-16T09:37:50.019 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:37:50.019 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.019 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.019 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:37:50.019 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.401792+0000", 2024-09-16T09:37:50.019 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:37:50.019 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.019 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.019 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:37:50.020 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.401794+0000", 2024-09-16T09:37:50.020 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.657e-06 2024-09-16T09:37:50.020 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.020 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.020 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:37:50.020 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.401819+0000", 2024-09-16T09:37:50.020 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.5366000000000001e-05 2024-09-16T09:37:50.020 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.020 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.020 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:37:50.020 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.401831+0000", 2024-09-16T09:37:50.020 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2116000000000001e-05 2024-09-16T09:37:50.020 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.021 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.021 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:37:50.021 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.401837+0000", 2024-09-16T09:37:50.021 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.2049999999999999e-06 2024-09-16T09:37:50.021 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.021 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.021 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:37:50.021 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.401966+0000", 2024-09-16T09:37:50.021 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00012920400000000001 2024-09-16T09:37:50.021 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.021 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.021 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:37:50.021 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.401974+0000", 2024-09-16T09:37:50.021 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3320000000000003e-06 2024-09-16T09:37:50.022 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.022 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.022 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:37:50.022 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:00.329622+0000", 2024-09-16T09:37:50.022 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.92764788399999998 2024-09-16T09:37:50.022 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.022 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.022 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:37:50.022 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:00.329629+0000", 2024-09-16T09:37:50.023 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.9800000000000001e-06 2024-09-16T09:37:50.023 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.023 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.023 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:37:50.023 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.461167+0000", 2024-09-16T09:37:50.023 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1315382869999999 2024-09-16T09:37:50.024 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.024 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.024 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:37:50.024 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.461323+0000", 2024-09-16T09:37:50.024 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000156477 2024-09-16T09:37:50.024 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.024 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.024 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,7", 2024-09-16T09:37:50.024 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.461496+0000", 2024-09-16T09:37:50.024 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00017256800000000001 2024-09-16T09:37:50.024 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.024 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.024 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:37:50.025 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.469507+0000", 2024-09-16T09:37:50.025 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0080107310000000001 2024-09-16T09:37:50.025 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.025 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.025 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:50.025 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.469586+0000", 2024-09-16T09:37:50.025 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.9714000000000003e-05 2024-09-16T09:37:50.025 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.025 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.025 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:50.025 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.480994+0000", 2024-09-16T09:37:50.025 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.011407782 2024-09-16T09:37:50.025 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.026 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.026 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:37:50.026 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.481093+0000", 2024-09-16T09:37:50.026 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.8571000000000002e-05 2024-09-16T09:37:50.026 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.026 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.026 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:37:50.026 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.482711+0000", 2024-09-16T09:37:50.026 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001618565 2024-09-16T09:37:50.026 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:37:50.026 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:37:50.026 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:37:50.026 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.026 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.027 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:175691 9.9 9.b751ed49 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e304)", 2024-09-16T09:37:50.027 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:36:06.341011+0000", 2024-09-16T09:37:50.027 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 103.445181925, 2024-09-16T09:37:50.027 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0983883910000001, 2024-09-16T09:37:50.027 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:37:50.027 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:37:50.027 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T09:37:50.027 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:37:50.027 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:37:50.027 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:37:50.027 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 175691 2024-09-16T09:37:50.027 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.027 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:37:50.027 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.028 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:37:50.028 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:06.341011+0000", 2024-09-16T09:37:50.028 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:37:50.028 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.028 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.028 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:37:50.028 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:06.341011+0000", 2024-09-16T09:37:50.028 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:37:50.028 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.028 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.028 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:37:50.029 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:06.341013+0000", 2024-09-16T09:37:50.029 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.4139999999999999e-06 2024-09-16T09:37:50.029 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.029 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.029 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:37:50.029 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:06.341017+0000", 2024-09-16T09:37:50.029 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.8289999999999998e-06 2024-09-16T09:37:50.029 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.029 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.029 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:37:50.029 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:06.341032+0000", 2024-09-16T09:37:50.030 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.4956999999999999e-05 2024-09-16T09:37:50.030 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.030 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.030 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:37:50.030 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:06.341040+0000", 2024-09-16T09:37:50.030 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6290000000000001e-06 2024-09-16T09:37:50.030 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.030 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.030 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:37:50.030 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:06.341110+0000", 2024-09-16T09:37:50.030 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.9534000000000002e-05 2024-09-16T09:37:50.030 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.030 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.030 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op must wait for map", 2024-09-16T09:37:50.031 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:06.341116+0000", 2024-09-16T09:37:50.031 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.2090000000000002e-06 2024-09-16T09:37:50.031 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.031 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.031 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:37:50.031 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:06.350340+0000", 2024-09-16T09:37:50.031 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0092238479999999998 2024-09-16T09:37:50.031 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.031 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.031 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:37:50.031 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:06.350350+0000", 2024-09-16T09:37:50.031 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0281e-05 2024-09-16T09:37:50.031 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.031 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.032 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:37:50.032 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:07.439357+0000", 2024-09-16T09:37:50.032 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.08900679 2024-09-16T09:37:50.032 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.032 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.032 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:37:50.032 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:07.439400+0000", 2024-09-16T09:37:50.032 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2899999999999999e-05 2024-09-16T09:37:50.032 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:37:50.032 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:37:50.032 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:37:50.032 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.032 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.033 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:175720 9.19 9.74aa9ef9 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e304)", 2024-09-16T09:37:50.033 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:36:06.379575+0000", 2024-09-16T09:37:50.033 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 103.40661786, 2024-09-16T09:37:50.033 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0607223160000001, 2024-09-16T09:37:50.033 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:37:50.033 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:37:50.033 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T09:37:50.033 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:37:50.033 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:37:50.033 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:37:50.033 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 175720 2024-09-16T09:37:50.033 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.033 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:37:50.033 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.034 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:37:50.034 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:06.379575+0000", 2024-09-16T09:37:50.034 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:37:50.034 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.034 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.034 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:37:50.034 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:06.379575+0000", 2024-09-16T09:37:50.034 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:37:50.034 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.034 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.034 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:37:50.034 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:06.379576+0000", 2024-09-16T09:37:50.034 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.0800000000000003e-07 2024-09-16T09:37:50.034 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.035 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.035 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:37:50.035 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:06.379580+0000", 2024-09-16T09:37:50.035 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9210000000000002e-06 2024-09-16T09:37:50.035 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.035 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.035 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:37:50.035 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:06.379592+0000", 2024-09-16T09:37:50.035 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.184e-05 2024-09-16T09:37:50.035 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.035 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.035 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:37:50.035 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:06.379600+0000", 2024-09-16T09:37:50.035 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.8059999999999995e-06 2024-09-16T09:37:50.036 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.036 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.036 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:37:50.036 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:06.379666+0000", 2024-09-16T09:37:50.036 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.6243999999999998e-05 2024-09-16T09:37:50.036 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.036 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.036 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:37:50.036 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:06.379676+0000", 2024-09-16T09:37:50.036 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0413e-05 2024-09-16T09:37:50.036 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.036 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.036 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:37:50.037 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:07.440247+0000", 2024-09-16T09:37:50.037 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.060571003 2024-09-16T09:37:50.037 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.037 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.037 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:37:50.037 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:07.440298+0000", 2024-09-16T09:37:50.037 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0180999999999999e-05 2024-09-16T09:37:50.037 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:37:50.037 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:37:50.037 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:37:50.037 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.038 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.038 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:175736 9.9 9.2dd87e89 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e304)", 2024-09-16T09:37:50.038 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:36:06.395074+0000", 2024-09-16T09:37:50.038 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 103.391119324, 2024-09-16T09:37:50.038 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.044973742, 2024-09-16T09:37:50.038 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:37:50.038 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:37:50.038 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T09:37:50.038 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:37:50.038 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:37:50.038 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:37:50.038 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 175736 2024-09-16T09:37:50.038 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.039 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:37:50.039 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.039 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:37:50.039 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:06.395074+0000", 2024-09-16T09:37:50.039 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:37:50.039 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.039 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.039 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:37:50.039 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:06.395074+0000", 2024-09-16T09:37:50.039 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:37:50.039 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.039 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.039 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:37:50.040 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:06.395074+0000", 2024-09-16T09:37:50.040 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.2900000000000004e-07 2024-09-16T09:37:50.040 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.040 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.040 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:37:50.040 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:06.395077+0000", 2024-09-16T09:37:50.040 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.6129999999999998e-06 2024-09-16T09:37:50.040 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.040 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.040 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:37:50.041 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:06.395084+0000", 2024-09-16T09:37:50.041 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.8190000000000003e-06 2024-09-16T09:37:50.041 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.041 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.041 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:37:50.041 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:06.395089+0000", 2024-09-16T09:37:50.041 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.6189999999999999e-06 2024-09-16T09:37:50.041 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.042 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.042 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:37:50.042 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:06.395228+0000", 2024-09-16T09:37:50.042 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00013882099999999999 2024-09-16T09:37:50.042 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.042 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.042 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:37:50.042 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:06.395239+0000", 2024-09-16T09:37:50.042 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0645000000000001e-05 2024-09-16T09:37:50.042 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.042 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.042 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:37:50.042 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:07.440039+0000", 2024-09-16T09:37:50.042 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.044799668 2024-09-16T09:37:50.043 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.043 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.043 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:37:50.043 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:07.440048+0000", 2024-09-16T09:37:50.043 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.0280000000000006e-06 2024-09-16T09:37:50.043 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:37:50.043 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:37:50.043 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:37:50.043 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.043 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.043 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:231317 9.15 9:aaf7d9df:::benchmark_data_smithi070_137000_object109606:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e357)", 2024-09-16T09:37:50.043 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:37:18.512274+0000", 2024-09-16T09:37:50.044 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 31.273919192000001, 2024-09-16T09:37:50.044 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0402452229999999, 2024-09-16T09:37:50.044 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:37:50.044 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:37:50.044 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T09:37:50.044 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:37:50.044 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:37:50.044 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:37:50.044 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 231317 2024-09-16T09:37:50.044 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.044 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:37:50.044 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.045 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:37:50.045 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:18.512274+0000", 2024-09-16T09:37:50.045 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:37:50.045 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.045 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.045 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:37:50.045 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:18.512274+0000", 2024-09-16T09:37:50.045 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:37:50.045 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.045 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.046 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:37:50.046 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:18.512276+0000", 2024-09-16T09:37:50.046 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3140000000000002e-06 2024-09-16T09:37:50.046 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.046 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.046 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:37:50.046 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:18.512284+0000", 2024-09-16T09:37:50.046 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.106e-06 2024-09-16T09:37:50.046 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.046 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.047 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:37:50.047 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:18.512304+0000", 2024-09-16T09:37:50.047 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0038999999999999e-05 2024-09-16T09:37:50.047 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.047 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.047 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:37:50.047 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:18.512315+0000", 2024-09-16T09:37:50.047 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0587e-05 2024-09-16T09:37:50.047 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.047 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.047 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:37:50.047 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:18.625285+0000", 2024-09-16T09:37:50.048 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.112970271 2024-09-16T09:37:50.048 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.048 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.048 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:37:50.048 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:18.625293+0000", 2024-09-16T09:37:50.048 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6499999999999996e-06 2024-09-16T09:37:50.048 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.048 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.048 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:37:50.048 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:19.552302+0000", 2024-09-16T09:37:50.049 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.92700903000000001 2024-09-16T09:37:50.049 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.049 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.049 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:37:50.049 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:19.552519+0000", 2024-09-16T09:37:50.049 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000217226 2024-09-16T09:37:50.049 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:37:50.049 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:37:50.049 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:37:50.049 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.049 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.050 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:231330 9.19 9:9bc810af:::benchmark_data_smithi070_137000_object109619:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e357)", 2024-09-16T09:37:50.050 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:37:18.512335+0000", 2024-09-16T09:37:50.050 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 31.273858505, 2024-09-16T09:37:50.050 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.058821751, 2024-09-16T09:37:50.050 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:37:50.050 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:37:50.050 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:37:50.050 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:37:50.050 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:37:50.050 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:37:50.051 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 231330 2024-09-16T09:37:50.051 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.051 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:37:50.051 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.051 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:37:50.051 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:18.512335+0000", 2024-09-16T09:37:50.051 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:37:50.051 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.051 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.051 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:37:50.051 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:18.512335+0000", 2024-09-16T09:37:50.051 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:37:50.051 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.052 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.052 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:37:50.052 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:18.512335+0000", 2024-09-16T09:37:50.052 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0900000000000002e-07 2024-09-16T09:37:50.052 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.052 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.052 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:37:50.052 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:18.512338+0000", 2024-09-16T09:37:50.052 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.4530000000000001e-06 2024-09-16T09:37:50.052 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.052 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.052 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:37:50.052 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:18.512345+0000", 2024-09-16T09:37:50.053 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.9129999999999997e-06 2024-09-16T09:37:50.053 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.053 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.053 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:37:50.053 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:18.512350+0000", 2024-09-16T09:37:50.053 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.344e-06 2024-09-16T09:37:50.053 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.053 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.053 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:37:50.053 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:18.785156+0000", 2024-09-16T09:37:50.053 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.27280600599999999 2024-09-16T09:37:50.053 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.053 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.053 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:37:50.054 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:18.785164+0000", 2024-09-16T09:37:50.054 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.568e-06 2024-09-16T09:37:50.054 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.054 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.054 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:37:50.054 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:19.554018+0000", 2024-09-16T09:37:50.054 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.76885353300000003 2024-09-16T09:37:50.054 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.054 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.054 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:37:50.054 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:19.554177+0000", 2024-09-16T09:37:50.054 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000158548 2024-09-16T09:37:50.054 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.055 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.055 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 5,7", 2024-09-16T09:37:50.055 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:19.554311+0000", 2024-09-16T09:37:50.055 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00013476499999999999 2024-09-16T09:37:50.055 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.055 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.055 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:37:50.055 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:19.555602+0000", 2024-09-16T09:37:50.055 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001290975 2024-09-16T09:37:50.055 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.055 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.055 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:50.056 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:19.556689+0000", 2024-09-16T09:37:50.056 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0010867889999999999 2024-09-16T09:37:50.056 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.056 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.056 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:50.056 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:19.571062+0000", 2024-09-16T09:37:50.056 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.014372818000000001 2024-09-16T09:37:50.056 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.056 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.056 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:37:50.057 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:19.571116+0000", 2024-09-16T09:37:50.057 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.3925000000000003e-05 2024-09-16T09:37:50.057 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.057 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.057 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:37:50.057 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:19.571156+0000", 2024-09-16T09:37:50.057 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0605000000000002e-05 2024-09-16T09:37:50.057 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:37:50.057 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:37:50.057 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:37:50.057 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.057 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.058 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:232724 9.19 9:9df51a8e:::benchmark_data_smithi070_137000_object111013:head [delete] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e360)", 2024-09-16T09:37:50.058 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:37:21.522489+0000", 2024-09-16T09:37:50.058 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 28.263704543999999, 2024-09-16T09:37:50.058 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.123121356, 2024-09-16T09:37:50.058 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:37:50.058 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:37:50.058 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T09:37:50.058 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:37:50.058 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:37:50.058 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:37:50.059 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 232724 2024-09-16T09:37:50.059 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.059 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:37:50.059 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.059 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:37:50.059 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.522489+0000", 2024-09-16T09:37:50.059 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:37:50.059 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.059 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.059 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:37:50.059 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.522489+0000", 2024-09-16T09:37:50.059 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:37:50.059 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.059 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.060 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:37:50.060 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.522491+0000", 2024-09-16T09:37:50.060 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.2869999999999999e-06 2024-09-16T09:37:50.060 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.060 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.060 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:37:50.060 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.522499+0000", 2024-09-16T09:37:50.060 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.1009999999999992e-06 2024-09-16T09:37:50.060 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.060 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.060 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:37:50.060 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.522521+0000", 2024-09-16T09:37:50.061 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.2373000000000001e-05 2024-09-16T09:37:50.061 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.061 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.061 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:37:50.061 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.522533+0000", 2024-09-16T09:37:50.061 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1231e-05 2024-09-16T09:37:50.061 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.061 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.061 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:37:50.061 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.522754+0000", 2024-09-16T09:37:50.061 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00022174900000000001 2024-09-16T09:37:50.061 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.061 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.062 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op must wait for map", 2024-09-16T09:37:50.062 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.522760+0000", 2024-09-16T09:37:50.062 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.2379999999999997e-06 2024-09-16T09:37:50.062 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.062 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.062 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:37:50.062 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.528487+0000", 2024-09-16T09:37:50.062 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.005727195 2024-09-16T09:37:50.062 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.062 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.062 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:37:50.062 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.528499+0000", 2024-09-16T09:37:50.062 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2656999999999999e-05 2024-09-16T09:37:50.062 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.063 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.063 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:37:50.063 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.645538+0000", 2024-09-16T09:37:50.063 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1170389300000001 2024-09-16T09:37:50.063 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.063 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.063 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:37:50.063 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.645610+0000", 2024-09-16T09:37:50.063 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.1594999999999995e-05 2024-09-16T09:37:50.063 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:37:50.063 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:37:50.063 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:37:50.063 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.063 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.064 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:232730 9.19 9:99b28832:::benchmark_data_smithi070_137000_object111019:head [delete] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e360)", 2024-09-16T09:37:50.064 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:37:21.522561+0000", 2024-09-16T09:37:50.064 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 28.263632305000002, 2024-09-16T09:37:50.064 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.129695954, 2024-09-16T09:37:50.064 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:37:50.064 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:37:50.064 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:37:50.064 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:37:50.064 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:37:50.064 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:37:50.064 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 232730 2024-09-16T09:37:50.064 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.064 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:37:50.065 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.065 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:37:50.065 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.522561+0000", 2024-09-16T09:37:50.065 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:37:50.065 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.065 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.065 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:37:50.065 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.522561+0000", 2024-09-16T09:37:50.065 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:37:50.065 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.065 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.065 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:37:50.065 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.522561+0000", 2024-09-16T09:37:50.066 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8699999999999995e-07 2024-09-16T09:37:50.066 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.066 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.066 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:37:50.066 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.522565+0000", 2024-09-16T09:37:50.066 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3790000000000002e-06 2024-09-16T09:37:50.066 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.066 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.066 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:37:50.066 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.522573+0000", 2024-09-16T09:37:50.067 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.3799999999999994e-06 2024-09-16T09:37:50.067 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.067 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.067 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:37:50.067 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.522579+0000", 2024-09-16T09:37:50.067 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1099999999999999e-06 2024-09-16T09:37:50.067 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.067 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.067 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:37:50.067 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.522780+0000", 2024-09-16T09:37:50.067 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00020084399999999999 2024-09-16T09:37:50.068 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.068 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.068 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting_for_map not empty", 2024-09-16T09:37:50.068 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.522785+0000", 2024-09-16T09:37:50.068 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5750000000000002e-06 2024-09-16T09:37:50.068 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.068 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.068 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:37:50.068 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.528530+0000", 2024-09-16T09:37:50.068 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0057457580000000001 2024-09-16T09:37:50.069 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.069 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.069 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:37:50.069 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.528537+0000", 2024-09-16T09:37:50.069 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.0780000000000001e-06 2024-09-16T09:37:50.069 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.069 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.069 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:37:50.069 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.645817+0000", 2024-09-16T09:37:50.069 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.117280482 2024-09-16T09:37:50.070 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.070 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.070 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:37:50.070 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.645920+0000", 2024-09-16T09:37:50.070 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00010247999999999999 2024-09-16T09:37:50.070 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.070 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.070 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 3,5,6,7", 2024-09-16T09:37:50.070 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.646661+0000", 2024-09-16T09:37:50.071 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00074141599999999995 2024-09-16T09:37:50.071 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.071 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.071 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:37:50.071 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.649533+0000", 2024-09-16T09:37:50.071 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.002872281 2024-09-16T09:37:50.071 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.071 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.071 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:50.071 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.651288+0000", 2024-09-16T09:37:50.071 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0017543979999999999 2024-09-16T09:37:50.071 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.072 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.072 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:50.072 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.651495+0000", 2024-09-16T09:37:50.072 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00020767 2024-09-16T09:37:50.072 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.072 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.072 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:50.072 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.651661+0000", 2024-09-16T09:37:50.072 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00016588900000000001 2024-09-16T09:37:50.072 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.073 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.073 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:50.073 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.652067+0000", 2024-09-16T09:37:50.073 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000405735 2024-09-16T09:37:50.073 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.073 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.073 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:37:50.073 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.652124+0000", 2024-09-16T09:37:50.073 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7484000000000001e-05 2024-09-16T09:37:50.074 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.074 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.074 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:37:50.074 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.652257+0000", 2024-09-16T09:37:50.074 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00013240800000000001 2024-09-16T09:37:50.074 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:37:50.074 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:37:50.074 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:37:50.074 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.074 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.074 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:232734 9.18 9:1cfe7a7c:::benchmark_data_smithi070_137000_object111023:head [delete] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e360)", 2024-09-16T09:37:50.074 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:37:21.522588+0000", 2024-09-16T09:37:50.074 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 28.263604855000001, 2024-09-16T09:37:50.074 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.128298018, 2024-09-16T09:37:50.075 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:37:50.075 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:37:50.075 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:37:50.075 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:37:50.075 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:37:50.075 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:37:50.075 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 232734 2024-09-16T09:37:50.075 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.075 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:37:50.075 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.075 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:37:50.075 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.522588+0000", 2024-09-16T09:37:50.075 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:37:50.075 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.076 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.076 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:37:50.076 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.522588+0000", 2024-09-16T09:37:50.076 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:37:50.076 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.076 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.076 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:37:50.076 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.522589+0000", 2024-09-16T09:37:50.076 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9799999999999999e-07 2024-09-16T09:37:50.076 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.076 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.076 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:37:50.076 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.522591+0000", 2024-09-16T09:37:50.077 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.1799999999999999e-06 2024-09-16T09:37:50.077 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.077 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.077 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:37:50.077 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.522599+0000", 2024-09-16T09:37:50.077 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.388e-06 2024-09-16T09:37:50.077 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.077 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.077 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:37:50.077 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.522605+0000", 2024-09-16T09:37:50.077 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4330000000000003e-06 2024-09-16T09:37:50.077 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.077 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.078 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:37:50.078 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.522717+0000", 2024-09-16T09:37:50.078 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000112147 2024-09-16T09:37:50.078 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.078 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.078 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op must wait for map", 2024-09-16T09:37:50.078 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.522725+0000", 2024-09-16T09:37:50.078 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.259e-06 2024-09-16T09:37:50.078 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.078 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.078 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:37:50.078 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.529317+0000", 2024-09-16T09:37:50.078 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0065921959999999998 2024-09-16T09:37:50.078 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.079 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.079 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:37:50.079 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.529326+0000", 2024-09-16T09:37:50.079 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.738e-06 2024-09-16T09:37:50.079 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.079 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.079 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:37:50.079 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.642752+0000", 2024-09-16T09:37:50.079 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.113426391 2024-09-16T09:37:50.079 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.080 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.080 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:37:50.080 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.642982+0000", 2024-09-16T09:37:50.080 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00022971299999999999 2024-09-16T09:37:50.080 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.080 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.080 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,5,6,7", 2024-09-16T09:37:50.080 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.643102+0000", 2024-09-16T09:37:50.080 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00011989000000000001 2024-09-16T09:37:50.080 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.080 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.080 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:37:50.080 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.650395+0000", 2024-09-16T09:37:50.081 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0072925999999999998 2024-09-16T09:37:50.081 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.081 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.081 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:50.081 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.650604+0000", 2024-09-16T09:37:50.081 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000209803 2024-09-16T09:37:50.081 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.081 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.081 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:50.081 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.650647+0000", 2024-09-16T09:37:50.081 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2106e-05 2024-09-16T09:37:50.081 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.081 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.081 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:50.082 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.650683+0000", 2024-09-16T09:37:50.082 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6665999999999998e-05 2024-09-16T09:37:50.082 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.082 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.082 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:50.082 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.650722+0000", 2024-09-16T09:37:50.082 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.858e-05 2024-09-16T09:37:50.082 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.082 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.082 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:37:50.082 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.650778+0000", 2024-09-16T09:37:50.082 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.6115000000000001e-05 2024-09-16T09:37:50.082 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.082 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.082 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:37:50.083 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.650886+0000", 2024-09-16T09:37:50.083 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000108415 2024-09-16T09:37:50.083 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:37:50.083 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:37:50.083 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:37:50.083 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.083 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.083 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:232737 9.15 9:aef95ec3:::benchmark_data_smithi070_137000_object111026:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e360)", 2024-09-16T09:37:50.083 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:37:21.522620+0000", 2024-09-16T09:37:50.083 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 28.263572946, 2024-09-16T09:37:50.083 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0972297040000001, 2024-09-16T09:37:50.083 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:37:50.083 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:37:50.083 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:37:50.084 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:37:50.084 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:37:50.084 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:37:50.084 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 232737 2024-09-16T09:37:50.084 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.084 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:37:50.084 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.084 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:37:50.084 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.522620+0000", 2024-09-16T09:37:50.084 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:37:50.085 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.085 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.085 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:37:50.085 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.522620+0000", 2024-09-16T09:37:50.085 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:37:50.085 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.086 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.086 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:37:50.086 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.522621+0000", 2024-09-16T09:37:50.086 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.3799999999999997e-07 2024-09-16T09:37:50.086 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.087 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.087 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:37:50.087 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.522623+0000", 2024-09-16T09:37:50.087 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.2819999999999999e-06 2024-09-16T09:37:50.087 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.087 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.087 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:37:50.088 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.522630+0000", 2024-09-16T09:37:50.088 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.8990000000000004e-06 2024-09-16T09:37:50.088 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.088 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.088 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:37:50.088 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.522635+0000", 2024-09-16T09:37:50.088 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.9949999999999996e-06 2024-09-16T09:37:50.088 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.088 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.088 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:37:50.088 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.523000+0000", 2024-09-16T09:37:50.089 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00036526899999999999 2024-09-16T09:37:50.089 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.089 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.089 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op must wait for map", 2024-09-16T09:37:50.089 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.523007+0000", 2024-09-16T09:37:50.089 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.7950000000000002e-06 2024-09-16T09:37:50.089 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.089 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.089 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:37:50.089 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.532786+0000", 2024-09-16T09:37:50.089 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0097794809999999996 2024-09-16T09:37:50.090 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.090 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.090 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:37:50.090 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.532795+0000", 2024-09-16T09:37:50.090 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.7930000000000003e-06 2024-09-16T09:37:50.090 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.090 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.090 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:37:50.090 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.602050+0000", 2024-09-16T09:37:50.091 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.069254801 2024-09-16T09:37:50.091 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.091 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.091 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:37:50.091 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.602294+0000", 2024-09-16T09:37:50.091 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00024363299999999999 2024-09-16T09:37:50.091 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.091 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.092 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,3,5,7", 2024-09-16T09:37:50.092 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.602437+0000", 2024-09-16T09:37:50.092 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000143201 2024-09-16T09:37:50.092 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.092 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.092 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:50.092 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.603959+0000", 2024-09-16T09:37:50.092 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001521969 2024-09-16T09:37:50.093 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.093 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.093 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:50.093 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.604003+0000", 2024-09-16T09:37:50.093 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.4097000000000001e-05 2024-09-16T09:37:50.093 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.093 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.093 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:50.094 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.604403+0000", 2024-09-16T09:37:50.094 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00039965200000000002 2024-09-16T09:37:50.094 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.094 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.094 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:50.094 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.619638+0000", 2024-09-16T09:37:50.094 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.01523505 2024-09-16T09:37:50.094 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.095 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.095 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:37:50.095 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.619676+0000", 2024-09-16T09:37:50.095 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8154000000000001e-05 2024-09-16T09:37:50.095 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.095 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.095 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:37:50.095 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.619714+0000", 2024-09-16T09:37:50.096 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7957999999999998e-05 2024-09-16T09:37:50.096 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.096 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.096 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:37:50.096 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.619850+0000", 2024-09-16T09:37:50.096 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00013613699999999999 2024-09-16T09:37:50.096 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:37:50.096 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:37:50.096 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:37:50.097 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.097 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.097 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:232755 9.19 9:9c3a2274:::benchmark_data_smithi070_137000_object111044:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e360)", 2024-09-16T09:37:50.097 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:37:21.522649+0000", 2024-09-16T09:37:50.097 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 28.263544016000001, 2024-09-16T09:37:50.097 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.141048657, 2024-09-16T09:37:50.097 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:37:50.097 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:37:50.098 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:37:50.098 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:37:50.098 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:37:50.098 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:37:50.098 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 232755 2024-09-16T09:37:50.098 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.098 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:37:50.098 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.099 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:37:50.099 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.522649+0000", 2024-09-16T09:37:50.099 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:37:50.099 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.099 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.099 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:37:50.099 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.522649+0000", 2024-09-16T09:37:50.099 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:37:50.099 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.100 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.100 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:37:50.100 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.522650+0000", 2024-09-16T09:37:50.100 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9999999999999998e-07 2024-09-16T09:37:50.100 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.100 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.100 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:37:50.100 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.522652+0000", 2024-09-16T09:37:50.101 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0980000000000001e-06 2024-09-16T09:37:50.101 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.101 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.101 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:37:50.101 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.522659+0000", 2024-09-16T09:37:50.101 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.8990000000000004e-06 2024-09-16T09:37:50.101 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.101 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.101 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:37:50.102 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.522664+0000", 2024-09-16T09:37:50.102 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.169e-06 2024-09-16T09:37:50.102 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.102 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.102 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:37:50.102 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.522805+0000", 2024-09-16T09:37:50.102 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00014155999999999999 2024-09-16T09:37:50.102 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.102 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.103 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting_for_map not empty", 2024-09-16T09:37:50.103 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.522810+0000", 2024-09-16T09:37:50.103 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2429999999999999e-06 2024-09-16T09:37:50.103 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.103 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.103 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:37:50.103 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.528559+0000", 2024-09-16T09:37:50.103 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0057496800000000001 2024-09-16T09:37:50.103 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.104 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.104 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:37:50.104 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.528565+0000", 2024-09-16T09:37:50.104 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.054e-06 2024-09-16T09:37:50.104 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.104 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.104 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:37:50.104 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.648010+0000", 2024-09-16T09:37:50.104 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1194449900000001 2024-09-16T09:37:50.105 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.105 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.105 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:37:50.105 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.648126+0000", 2024-09-16T09:37:50.105 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00011561900000000001 2024-09-16T09:37:50.105 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.105 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.105 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 3,5,6,7", 2024-09-16T09:37:50.105 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.648246+0000", 2024-09-16T09:37:50.105 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00012061399999999999 2024-09-16T09:37:50.105 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.105 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.106 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:37:50.106 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.649539+0000", 2024-09-16T09:37:50.106 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001292587 2024-09-16T09:37:50.106 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.106 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.106 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:50.106 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.651534+0000", 2024-09-16T09:37:50.106 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001994699 2024-09-16T09:37:50.106 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.106 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.106 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:50.107 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.651708+0000", 2024-09-16T09:37:50.107 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000174298 2024-09-16T09:37:50.107 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.107 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.107 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:50.107 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.651819+0000", 2024-09-16T09:37:50.107 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00011064400000000001 2024-09-16T09:37:50.107 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.107 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.107 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:50.108 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.663512+0000", 2024-09-16T09:37:50.108 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.011693301 2024-09-16T09:37:50.108 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.108 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.108 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:37:50.108 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.663579+0000", 2024-09-16T09:37:50.108 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.6544999999999994e-05 2024-09-16T09:37:50.108 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.108 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.108 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:37:50.108 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.663698+0000", 2024-09-16T09:37:50.109 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000119257 2024-09-16T09:37:50.109 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:37:50.109 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:37:50.109 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:37:50.109 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.109 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.109 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:232771 9.15 9:aa0978e0:::benchmark_data_smithi070_137000_object111060:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e360)", 2024-09-16T09:37:50.109 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:37:21.557486+0000", 2024-09-16T09:37:50.109 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 28.228707048, 2024-09-16T09:37:50.109 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0625502760000001, 2024-09-16T09:37:50.109 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:37:50.109 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:37:50.110 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:37:50.110 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:37:50.110 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:37:50.110 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:37:50.110 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 232771 2024-09-16T09:37:50.110 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.110 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:37:50.110 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.110 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:37:50.110 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.557486+0000", 2024-09-16T09:37:50.110 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:37:50.110 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.110 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.110 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:37:50.111 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.557486+0000", 2024-09-16T09:37:50.111 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:37:50.111 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.111 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.111 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:37:50.111 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.557488+0000", 2024-09-16T09:37:50.111 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.733e-06 2024-09-16T09:37:50.111 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.111 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.111 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:37:50.111 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.557495+0000", 2024-09-16T09:37:50.111 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.5800000000000003e-06 2024-09-16T09:37:50.111 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.111 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.112 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:37:50.112 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.557514+0000", 2024-09-16T09:37:50.112 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.8436000000000001e-05 2024-09-16T09:37:50.112 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.112 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.112 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:37:50.112 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.557521+0000", 2024-09-16T09:37:50.112 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.2899999999999997e-06 2024-09-16T09:37:50.112 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.112 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.112 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:37:50.112 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.557584+0000", 2024-09-16T09:37:50.112 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.3122999999999999e-05 2024-09-16T09:37:50.113 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.113 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.113 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:37:50.113 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.557596+0000", 2024-09-16T09:37:50.113 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1474000000000001e-05 2024-09-16T09:37:50.113 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.113 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.113 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:37:50.113 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.603185+0000", 2024-09-16T09:37:50.113 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.045589487 2024-09-16T09:37:50.114 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.114 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.114 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:37:50.114 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.603270+0000", 2024-09-16T09:37:50.114 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.4728999999999995e-05 2024-09-16T09:37:50.114 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.114 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.114 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,3,5,7", 2024-09-16T09:37:50.114 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.603399+0000", 2024-09-16T09:37:50.114 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00012893599999999999 2024-09-16T09:37:50.114 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.114 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.114 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:50.114 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.604344+0000", 2024-09-16T09:37:50.115 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00094472899999999995 2024-09-16T09:37:50.115 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.115 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.115 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:50.115 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.604641+0000", 2024-09-16T09:37:50.115 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00029712100000000002 2024-09-16T09:37:50.115 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.115 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.115 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:50.115 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.605004+0000", 2024-09-16T09:37:50.115 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00036339199999999998 2024-09-16T09:37:50.115 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.115 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.115 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:37:50.116 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.619669+0000", 2024-09-16T09:37:50.116 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.014664877999999999 2024-09-16T09:37:50.116 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.116 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.116 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:37:50.116 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.619860+0000", 2024-09-16T09:37:50.116 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000191223 2024-09-16T09:37:50.116 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.116 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.116 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:37:50.116 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.619914+0000", 2024-09-16T09:37:50.116 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4163999999999998e-05 2024-09-16T09:37:50.116 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:37:50.116 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:37:50.116 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:37:50.117 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.620036+0000", 2024-09-16T09:37:50.117 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000121981 2024-09-16T09:37:50.117 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:37:50.117 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:37:50.117 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:37:50.117 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:37:50.117 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:37:50.117 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T09:37:50.117 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:37:50.144 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:50.144 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:50.145 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:37:50.245 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:37:50.272 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:50.272 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:50.273 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:37:50.374 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:37:50.395 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:50.396 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:50.396 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:37:50.497 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:37:50.518 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:50.519 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:50.519 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:37:50.620 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:37:50.646 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:50.647 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:50.647 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:37:50.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:50 smithi084 ceph-mon[32629]: 4.7 scrub starts 2024-09-16T09:37:50.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:50 smithi084 ceph-mon[32629]: 4.7 scrub ok 2024-09-16T09:37:50.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:50 smithi084 ceph-mon[32629]: 5.e scrub starts 2024-09-16T09:37:50.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:50 smithi084 ceph-mon[32629]: 5.e scrub ok 2024-09-16T09:37:50.670 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:50 smithi070 ceph-mon[40691]: 4.7 scrub starts 2024-09-16T09:37:50.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:50 smithi070 ceph-mon[40691]: 4.7 scrub ok 2024-09-16T09:37:50.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:50 smithi070 ceph-mon[40691]: 5.e scrub starts 2024-09-16T09:37:50.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:50 smithi070 ceph-mon[40691]: 5.e scrub ok 2024-09-16T09:37:50.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:50 smithi070 ceph-mon[30388]: 4.7 scrub starts 2024-09-16T09:37:50.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:50 smithi070 ceph-mon[30388]: 4.7 scrub ok 2024-09-16T09:37:50.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:50 smithi070 ceph-mon[30388]: 5.e scrub starts 2024-09-16T09:37:50.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:50 smithi070 ceph-mon[30388]: 5.e scrub ok 2024-09-16T09:37:50.749 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:37:50.773 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:50.774 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:50.774 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:37:50.875 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:37:50.899 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:50.900 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:50.900 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:37:50.943 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.1 dump_ops_in_flight 2024-09-16T09:37:51.001 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:37:51.023 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:51.023 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:51.024 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:37:51.124 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:37:51.146 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:51.146 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:51.146 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:37:51.247 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:37:51.270 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:51.270 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:51.271 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:37:51.304 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:37:51.345 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:37:51.372 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:37:51.393 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:51.394 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:51.394 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:37:51.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:51 smithi070 ceph-mon[40691]: pgmap v680: 168 pgs: 168 active+clean; 583 KiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:37:51.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:51 smithi070 ceph-mon[40691]: 6.7 scrub starts 2024-09-16T09:37:51.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:51 smithi070 ceph-mon[40691]: 6.7 scrub ok 2024-09-16T09:37:51.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:51 smithi070 ceph-mon[40691]: 3.12 scrub starts 2024-09-16T09:37:51.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:51 smithi070 ceph-mon[40691]: 3.12 scrub ok 2024-09-16T09:37:51.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:51 smithi070 ceph-mon[40691]: 6.12 scrub starts 2024-09-16T09:37:51.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:51 smithi070 ceph-mon[40691]: 6.12 scrub ok 2024-09-16T09:37:51.422 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:51 smithi070 ceph-mon[30388]: pgmap v680: 168 pgs: 168 active+clean; 583 KiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:37:51.422 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:51 smithi070 ceph-mon[30388]: 6.7 scrub starts 2024-09-16T09:37:51.422 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:51 smithi070 ceph-mon[30388]: 6.7 scrub ok 2024-09-16T09:37:51.422 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:51 smithi070 ceph-mon[30388]: 3.12 scrub starts 2024-09-16T09:37:51.422 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:51 smithi070 ceph-mon[30388]: 3.12 scrub ok 2024-09-16T09:37:51.422 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:51 smithi070 ceph-mon[30388]: 6.12 scrub starts 2024-09-16T09:37:51.422 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:51 smithi070 ceph-mon[30388]: 6.12 scrub ok 2024-09-16T09:37:51.496 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:37:51.522 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:51.523 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:51.523 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:37:51.588 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T09:37:51.624 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:37:51.649 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:51.649 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:51.650 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:37:51.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:51 smithi084 ceph-mon[32629]: pgmap v680: 168 pgs: 168 active+clean; 583 KiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:37:51.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:51 smithi084 ceph-mon[32629]: 6.7 scrub starts 2024-09-16T09:37:51.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:51 smithi084 ceph-mon[32629]: 6.7 scrub ok 2024-09-16T09:37:51.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:51 smithi084 ceph-mon[32629]: 3.12 scrub starts 2024-09-16T09:37:51.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:51 smithi084 ceph-mon[32629]: 3.12 scrub ok 2024-09-16T09:37:51.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:51 smithi084 ceph-mon[32629]: 6.12 scrub starts 2024-09-16T09:37:51.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:51 smithi084 ceph-mon[32629]: 6.12 scrub ok 2024-09-16T09:37:51.751 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:37:51.772 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:51.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:51.773 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:37:51.874 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:37:51.896 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:51.897 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:51.897 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:37:51.955 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:37:51 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:37:51] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:37:51.997 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:37:52.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:52.019 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:52.019 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:37:52.069 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:37:52.121 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:37:52.142 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:52.143 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:52.143 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:37:52.244 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:37:52.265 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:52.266 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:52.266 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:37:52.367 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:37:52.368 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:37:52.388 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:52.389 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:52.389 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:37:52.414 INFO:teuthology.orchestra.run.smithi070.stderr:reweighted osd.5 to 1 (10000) 2024-09-16T09:37:52.418 INFO:teuthology.orchestra.run.smithi070.stderr:enabled application 'rados' on pool 'unique_pool_3' 2024-09-16T09:37:52.489 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:52 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3664736293' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "unique_pool_3", "app": "rados", "yes_i_really_mean_it": true} : dispatch 2024-09-16T09:37:52.489 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:52 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "unique_pool_3", "app": "rados", "yes_i_really_mean_it": true} : dispatch 2024-09-16T09:37:52.489 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:52 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3121466495' entity='client.admin' cmd={"prefix": "osd reweight", "id": 5, "weight": 1.0} : dispatch 2024-09-16T09:37:52.489 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:52 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3664736293' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "unique_pool_3", "app": "rados", "yes_i_really_mean_it": true} : dispatch 2024-09-16T09:37:52.489 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:52 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "unique_pool_3", "app": "rados", "yes_i_really_mean_it": true} : dispatch 2024-09-16T09:37:52.489 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:52 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3121466495' entity='client.admin' cmd={"prefix": "osd reweight", "id": 5, "weight": 1.0} : dispatch 2024-09-16T09:37:52.489 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:37:52.512 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:52.512 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:52.513 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:37:52.613 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:37:52.636 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:52.636 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:52.637 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:37:52.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:52 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3664736293' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "unique_pool_3", "app": "rados", "yes_i_really_mean_it": true} : dispatch 2024-09-16T09:37:52.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:52 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "unique_pool_3", "app": "rados", "yes_i_really_mean_it": true} : dispatch 2024-09-16T09:37:52.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:52 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3121466495' entity='client.admin' cmd={"prefix": "osd reweight", "id": 5, "weight": 1.0} : dispatch 2024-09-16T09:37:52.738 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:37:52.760 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:52.760 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:52.760 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:37:52.861 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:37:52.882 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:52.883 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:52.884 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:37:52.984 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:37:53.131 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:53.131 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:53.132 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:37:53.232 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:37:53.249 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T09:37:53.249 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T09:37:53.249 INFO:teuthology.orchestra.run.smithi070.stdout: "num_ops": 0 2024-09-16T09:37:53.249 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T09:37:53.254 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:53.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:53.255 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:37:53.355 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:37:53.377 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:53.378 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:53.378 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:37:53.418 INFO:teuthology.orchestra.run.smithi070.stderr:nodeep-scrub is unset 2024-09-16T09:37:53.479 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:37:53.500 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:53.501 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:53.501 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:37:53.602 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:37:53.624 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:53.624 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:53.625 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:37:53.660 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:37:53 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:37:53.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:53 smithi084 ceph-mon[32629]: pgmap v681: 168 pgs: 168 active+clean; 583 KiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 955 B/s rd, 0 op/s 2024-09-16T09:37:53.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:53 smithi084 ceph-mon[32629]: 1.a scrub starts 2024-09-16T09:37:53.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:53 smithi084 ceph-mon[32629]: 1.a scrub ok 2024-09-16T09:37:53.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:53 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_3", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-16T09:37:53.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:53 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3121466495' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-16T09:37:53.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:53 smithi084 ceph-mon[32629]: osdmap e382: 8 total, 8 up, 8 in 2024-09-16T09:37:53.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:53 smithi084 ceph-mon[32629]: 5.4 scrub starts 2024-09-16T09:37:53.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:53 smithi084 ceph-mon[32629]: 5.4 scrub ok 2024-09-16T09:37:53.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:53 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1300372309' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:37:53.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:53 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:37:53.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:53 smithi084 ceph-mon[32629]: 3.a scrub starts 2024-09-16T09:37:53.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:53 smithi084 ceph-mon[32629]: 3.a scrub ok 2024-09-16T09:37:53.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:53 smithi084 ceph-mon[32629]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-16T09:37:53.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:53 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-16T09:37:53.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:53 smithi070 ceph-mon[30388]: pgmap v681: 168 pgs: 168 active+clean; 583 KiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 955 B/s rd, 0 op/s 2024-09-16T09:37:53.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:53 smithi070 ceph-mon[30388]: 1.a scrub starts 2024-09-16T09:37:53.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:53 smithi070 ceph-mon[30388]: 1.a scrub ok 2024-09-16T09:37:53.673 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:53 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_3", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-16T09:37:53.673 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:53 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3121466495' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-16T09:37:53.673 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:53 smithi070 ceph-mon[30388]: osdmap e382: 8 total, 8 up, 8 in 2024-09-16T09:37:53.673 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:53 smithi070 ceph-mon[30388]: 5.4 scrub starts 2024-09-16T09:37:53.673 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:53 smithi070 ceph-mon[30388]: 5.4 scrub ok 2024-09-16T09:37:53.673 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:53 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1300372309' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:37:53.673 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:53 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:37:53.673 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:53 smithi070 ceph-mon[30388]: 3.a scrub starts 2024-09-16T09:37:53.673 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:53 smithi070 ceph-mon[30388]: 3.a scrub ok 2024-09-16T09:37:53.673 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:53 smithi070 ceph-mon[30388]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-16T09:37:53.673 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:53 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-16T09:37:53.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:53 smithi070 ceph-mon[40691]: pgmap v681: 168 pgs: 168 active+clean; 583 KiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 955 B/s rd, 0 op/s 2024-09-16T09:37:53.674 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:53 smithi070 ceph-mon[40691]: 1.a scrub starts 2024-09-16T09:37:53.674 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:53 smithi070 ceph-mon[40691]: 1.a scrub ok 2024-09-16T09:37:53.674 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:53 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_3", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-16T09:37:53.674 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:53 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3121466495' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-16T09:37:53.674 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:53 smithi070 ceph-mon[40691]: osdmap e382: 8 total, 8 up, 8 in 2024-09-16T09:37:53.674 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:53 smithi070 ceph-mon[40691]: 5.4 scrub starts 2024-09-16T09:37:53.674 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:53 smithi070 ceph-mon[40691]: 5.4 scrub ok 2024-09-16T09:37:53.674 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:53 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1300372309' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:37:53.674 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:53 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:37:53.674 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:53 smithi070 ceph-mon[40691]: 3.a scrub starts 2024-09-16T09:37:53.674 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:53 smithi070 ceph-mon[40691]: 3.a scrub ok 2024-09-16T09:37:53.674 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:53 smithi070 ceph-mon[40691]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-16T09:37:53.674 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:53 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-16T09:37:53.726 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:37:53.748 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:53.748 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:53.748 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:37:53.849 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:37:53.871 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:53.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:53.873 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:37:53.974 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:37:53.995 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:53.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:53.996 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:37:54.097 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:37:54.120 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:54.120 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:54.120 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:37:54.221 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:37:54.243 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:54.244 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:54.244 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:37:54.346 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:37:54.367 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:54.367 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:54.368 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:37:54.468 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:37:54.490 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:54.491 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:54.491 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:37:54.557 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:54 smithi070 ceph-mon[30388]: 3.14 scrub starts 2024-09-16T09:37:54.557 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:54 smithi070 ceph-mon[30388]: 3.14 scrub ok 2024-09-16T09:37:54.558 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:54 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:37:54.558 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:54 smithi070 ceph-mon[30388]: osdmap e383: 8 total, 8 up, 8 in 2024-09-16T09:37:54.558 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:54 smithi070 ceph-mon[30388]: 1.8 scrub starts 2024-09-16T09:37:54.558 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:54 smithi070 ceph-mon[30388]: 1.8 scrub ok 2024-09-16T09:37:54.558 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:54 smithi070 ceph-mon[40691]: 3.14 scrub starts 2024-09-16T09:37:54.558 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:54 smithi070 ceph-mon[40691]: 3.14 scrub ok 2024-09-16T09:37:54.558 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:54 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:37:54.558 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:54 smithi070 ceph-mon[40691]: osdmap e383: 8 total, 8 up, 8 in 2024-09-16T09:37:54.558 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:54 smithi070 ceph-mon[40691]: 1.8 scrub starts 2024-09-16T09:37:54.558 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:54 smithi070 ceph-mon[40691]: 1.8 scrub ok 2024-09-16T09:37:54.592 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:37:54.614 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:54.615 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:54.615 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:37:54.716 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:37:54.737 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:54.738 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:54.738 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:37:54.839 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:37:54.862 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:54.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:54.863 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:37:54.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:54 smithi084 ceph-mon[32629]: 3.14 scrub starts 2024-09-16T09:37:54.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:54 smithi084 ceph-mon[32629]: 3.14 scrub ok 2024-09-16T09:37:54.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:54 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:37:54.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:54 smithi084 ceph-mon[32629]: osdmap e383: 8 total, 8 up, 8 in 2024-09-16T09:37:54.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:54 smithi084 ceph-mon[32629]: 1.8 scrub starts 2024-09-16T09:37:54.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:54 smithi084 ceph-mon[32629]: 1.8 scrub ok 2024-09-16T09:37:54.928 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd reweight 6 1 2024-09-16T09:37:54.964 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:37:54.992 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:54.992 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:54.993 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:37:55.094 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:37:55.115 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:55.116 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:55.116 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:37:55.217 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:37:55.239 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:55.240 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:55.240 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:37:55.341 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:37:55.363 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:55.364 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:55.364 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:37:55.465 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:37:55.486 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:55.487 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:55.487 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:37:55.588 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:37:55.593 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:37:55.593 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:37:55.610 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:55.610 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:37:55.611 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:37:55.611 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:55.611 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:37:55.626 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:37:55.626 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:37:55.642 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:37:55.642 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:37:55.659 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:37:55.659 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:37:55.659 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:55 smithi070 ceph-mon[30388]: pgmap v684: 168 pgs: 3 peering, 165 active+clean; 378 KiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 4.7 KiB/s rd, 28 KiB/s wr, 4 op/s; 9.8 KiB/s, 0 objects/s recovering 2024-09-16T09:37:55.659 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:55 smithi070 ceph-mon[30388]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-09-16T09:37:55.659 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:55 smithi070 ceph-mon[30388]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-16T09:37:55.660 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:55 smithi070 ceph-mon[40691]: pgmap v684: 168 pgs: 3 peering, 165 active+clean; 378 KiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 4.7 KiB/s rd, 28 KiB/s wr, 4 op/s; 9.8 KiB/s, 0 objects/s recovering 2024-09-16T09:37:55.660 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:55 smithi070 ceph-mon[40691]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-09-16T09:37:55.660 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:55 smithi070 ceph-mon[40691]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-16T09:37:55.675 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:37:55.675 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:37:55.692 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:37:55.692 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:37:55.708 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:37:55.709 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:37:55.712 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:37:55.733 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:55.734 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:55.734 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:37:55.835 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:37:55.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:55.857 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:55.857 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:37:55.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:55 smithi084 ceph-mon[32629]: pgmap v684: 168 pgs: 3 peering, 165 active+clean; 378 KiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 4.7 KiB/s rd, 28 KiB/s wr, 4 op/s; 9.8 KiB/s, 0 objects/s recovering 2024-09-16T09:37:55.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:55 smithi084 ceph-mon[32629]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-09-16T09:37:55.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:55 smithi084 ceph-mon[32629]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-16T09:37:55.958 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:37:55.980 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:55.980 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:55.981 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:37:56.081 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:37:56.103 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:56.103 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:56.104 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:37:56.204 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:37:56.226 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:56.226 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:56.227 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:37:56.259 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd set noscrub 2024-09-16T09:37:56.277 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.1 dump_blocked_ops 2024-09-16T09:37:56.328 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:37:56.382 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:56.383 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:56.383 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:37:56.421 DEBUG:teuthology.orchestra.run.smithi070:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.0 -b 65536 --object-size 65536 -p unique_pool_3 bench 90 write' 2024-09-16T09:37:56.426 INFO:tasks.radosbench:joining radosbench (timing out after 3000s) 2024-09-16T09:37:56.426 INFO:teuthology.orchestra.run:waiting for 3000 2024-09-16T09:37:56.485 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:37:56.506 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:56.507 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:56.507 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:37:56.532 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:hints = 1 2024-09-16T09:37:56.532 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Maintaining 16 concurrent writes of 65536 bytes to objects of size 65536 for up to 90 seconds or 0 objects 2024-09-16T09:37:56.532 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Object prefix: benchmark_data_smithi070_166744 2024-09-16T09:37:56.532 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-16T09:37:56.532 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 0 0 0 0 0 0 - 0 2024-09-16T09:37:56.608 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:37:56.643 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:56.643 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:56.644 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:37:56.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:56 smithi070 ceph-mon[30388]: 4.15 scrub starts 2024-09-16T09:37:56.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:56 smithi070 ceph-mon[30388]: 4.15 scrub ok 2024-09-16T09:37:56.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:56 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3084785221' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:37:56.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:56 smithi070 ceph-mon[30388]: 3.10 scrub starts 2024-09-16T09:37:56.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:56 smithi070 ceph-mon[30388]: 3.10 scrub ok 2024-09-16T09:37:56.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:56 smithi070 ceph-mon[40691]: 4.15 scrub starts 2024-09-16T09:37:56.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:56 smithi070 ceph-mon[40691]: 4.15 scrub ok 2024-09-16T09:37:56.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:56 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3084785221' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:37:56.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:56 smithi070 ceph-mon[40691]: 3.10 scrub starts 2024-09-16T09:37:56.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:56 smithi070 ceph-mon[40691]: 3.10 scrub ok 2024-09-16T09:37:56.748 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:37:56.781 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:56.782 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:56.782 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:37:56.844 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:37:56.883 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:37:56.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:56 smithi084 ceph-mon[32629]: 4.15 scrub starts 2024-09-16T09:37:56.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:56 smithi084 ceph-mon[32629]: 4.15 scrub ok 2024-09-16T09:37:56.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:56 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3084785221' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:37:56.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:56 smithi084 ceph-mon[32629]: 3.10 scrub starts 2024-09-16T09:37:56.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:56 smithi084 ceph-mon[32629]: 3.10 scrub ok 2024-09-16T09:37:56.930 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:56.931 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:56.931 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:37:57.032 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:37:57.067 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:57.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:57.068 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:37:57.169 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:37:57.213 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:57.213 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:57.214 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:37:57.314 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:37:57.353 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:57.354 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:57.354 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:37:57.455 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:37:57.490 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:57.491 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:57.491 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:37:57.534 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 1 16 3701 3685 229.776 230.312 0.00393129 0.00433381 2024-09-16T09:37:57.544 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:37:57.544 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:37:57.592 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:37:57.627 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:57.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:57.628 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:37:57.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:57 smithi070 ceph-mon[30388]: pgmap v685: 168 pgs: 3 peering, 165 active+clean; 378 KiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 3.9 KiB/s rd, 23 KiB/s wr, 3 op/s; 8.2 KiB/s, 0 objects/s recovering 2024-09-16T09:37:57.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:57 smithi070 ceph-mon[30388]: 6.1a scrub starts 2024-09-16T09:37:57.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:57 smithi070 ceph-mon[30388]: 6.1a scrub ok 2024-09-16T09:37:57.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:57 smithi070 ceph-mon[30388]: 3.d scrub starts 2024-09-16T09:37:57.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:57 smithi070 ceph-mon[30388]: 3.d scrub ok 2024-09-16T09:37:57.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:57 smithi070 ceph-mon[30388]: 6.1 scrub starts 2024-09-16T09:37:57.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:57 smithi070 ceph-mon[30388]: 6.1 scrub ok 2024-09-16T09:37:57.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:57 smithi070 ceph-mon[40691]: pgmap v685: 168 pgs: 3 peering, 165 active+clean; 378 KiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 3.9 KiB/s rd, 23 KiB/s wr, 3 op/s; 8.2 KiB/s, 0 objects/s recovering 2024-09-16T09:37:57.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:57 smithi070 ceph-mon[40691]: 6.1a scrub starts 2024-09-16T09:37:57.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:57 smithi070 ceph-mon[40691]: 6.1a scrub ok 2024-09-16T09:37:57.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:57 smithi070 ceph-mon[40691]: 3.d scrub starts 2024-09-16T09:37:57.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:57 smithi070 ceph-mon[40691]: 3.d scrub ok 2024-09-16T09:37:57.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:57 smithi070 ceph-mon[40691]: 6.1 scrub starts 2024-09-16T09:37:57.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:57 smithi070 ceph-mon[40691]: 6.1 scrub ok 2024-09-16T09:37:57.729 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:37:57.766 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:57.766 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:57.767 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:37:57.868 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:37:57.906 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:57.907 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:57.907 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:37:57.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:57 smithi084 ceph-mon[32629]: pgmap v685: 168 pgs: 3 peering, 165 active+clean; 378 KiB data, 4.2 GiB used, 711 GiB / 715 GiB avail; 3.9 KiB/s rd, 23 KiB/s wr, 3 op/s; 8.2 KiB/s, 0 objects/s recovering 2024-09-16T09:37:57.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:57 smithi084 ceph-mon[32629]: 6.1a scrub starts 2024-09-16T09:37:57.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:57 smithi084 ceph-mon[32629]: 6.1a scrub ok 2024-09-16T09:37:57.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:57 smithi084 ceph-mon[32629]: 3.d scrub starts 2024-09-16T09:37:57.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:57 smithi084 ceph-mon[32629]: 3.d scrub ok 2024-09-16T09:37:57.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:57 smithi084 ceph-mon[32629]: 6.1 scrub starts 2024-09-16T09:37:57.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:57 smithi084 ceph-mon[32629]: 6.1 scrub ok 2024-09-16T09:37:58.008 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:37:58.043 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:58.044 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:58.044 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:37:58.145 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:37:58.181 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:58.182 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:58.182 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:37:58.192 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T09:37:58.192 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:37:58.283 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:37:58.318 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:58.318 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:58.319 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:37:58.420 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:37:58.443 INFO:teuthology.orchestra.run.smithi070.stderr:noscrub is set 2024-09-16T09:37:58.456 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:58.456 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:58.456 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:37:58.535 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 2 16 7107 7091 221.325 212.875 0.00260126 0.00450873 2024-09-16T09:37:58.557 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:37:58.595 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:58.596 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:58.596 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:37:58.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:58 smithi070 ceph-mon[30388]: 4.4 scrub starts 2024-09-16T09:37:58.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:58 smithi070 ceph-mon[30388]: 4.4 scrub ok 2024-09-16T09:37:58.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:58 smithi070 ceph-mon[30388]: 5.8 scrub starts 2024-09-16T09:37:58.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:58 smithi070 ceph-mon[30388]: 5.8 scrub ok 2024-09-16T09:37:58.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:58 smithi070 ceph-mon[30388]: 3.e scrub starts 2024-09-16T09:37:58.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:58 smithi070 ceph-mon[30388]: 3.e scrub ok 2024-09-16T09:37:58.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:58 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3879816545' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:37:58.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:58 smithi070 ceph-mon[40691]: 4.4 scrub starts 2024-09-16T09:37:58.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:58 smithi070 ceph-mon[40691]: 4.4 scrub ok 2024-09-16T09:37:58.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:58 smithi070 ceph-mon[40691]: 5.8 scrub starts 2024-09-16T09:37:58.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:58 smithi070 ceph-mon[40691]: 5.8 scrub ok 2024-09-16T09:37:58.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:58 smithi070 ceph-mon[40691]: 3.e scrub starts 2024-09-16T09:37:58.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:58 smithi070 ceph-mon[40691]: 3.e scrub ok 2024-09-16T09:37:58.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:58 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3879816545' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:37:58.697 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:37:58.729 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:58.729 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:58.730 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:37:58.817 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T09:37:58.831 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:37:58.901 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:58.902 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:58.902 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:37:58.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:58 smithi084 ceph-mon[32629]: 4.4 scrub starts 2024-09-16T09:37:58.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:58 smithi084 ceph-mon[32629]: 4.4 scrub ok 2024-09-16T09:37:58.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:58 smithi084 ceph-mon[32629]: 5.8 scrub starts 2024-09-16T09:37:58.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:58 smithi084 ceph-mon[32629]: 5.8 scrub ok 2024-09-16T09:37:58.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:58 smithi084 ceph-mon[32629]: 3.e scrub starts 2024-09-16T09:37:58.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:58 smithi084 ceph-mon[32629]: 3.e scrub ok 2024-09-16T09:37:58.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:58 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3879816545' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:37:59.003 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:37:59.045 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:59.045 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:59.046 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:37:59.147 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:37:59.181 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:59.182 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:59.182 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:37:59.283 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:37:59.320 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:59.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:59.321 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:37:59.422 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:37:59.456 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:59.457 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:59.458 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:37:59.535 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 3 16 10152 10136 210.991 190.312 0.00338974 0.0047324 2024-09-16T09:37:59.558 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:37:59.562 INFO:teuthology.orchestra.run.smithi070.stderr:reweighted osd.6 to 1 (10000) 2024-09-16T09:37:59.594 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:59.595 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:59.595 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:37:59.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:59 smithi070 ceph-mon[30388]: pgmap v686: 168 pgs: 2 peering, 166 active+clean; 103 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 3.1 KiB/s rd, 13 MiB/s wr, 208 op/s; 6.5 KiB/s, 0 objects/s recovering 2024-09-16T09:37:59.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:59 smithi070 ceph-mon[30388]: 5.12 scrub starts 2024-09-16T09:37:59.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:59 smithi070 ceph-mon[30388]: 5.12 scrub ok 2024-09-16T09:37:59.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:59 smithi070 ceph-mon[30388]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:37:59.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:59 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3879816545' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-16T09:37:59.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:59 smithi070 ceph-mon[30388]: osdmap e384: 8 total, 8 up, 8 in 2024-09-16T09:37:59.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:59 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/865736403' entity='client.admin' cmd={"prefix": "osd reweight", "id": 6, "weight": 1.0} : dispatch 2024-09-16T09:37:59.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:59 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/865736403' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-16T09:37:59.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:37:59 smithi070 ceph-mon[30388]: osdmap e385: 8 total, 8 up, 8 in 2024-09-16T09:37:59.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:59 smithi070 ceph-mon[40691]: pgmap v686: 168 pgs: 2 peering, 166 active+clean; 103 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 3.1 KiB/s rd, 13 MiB/s wr, 208 op/s; 6.5 KiB/s, 0 objects/s recovering 2024-09-16T09:37:59.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:59 smithi070 ceph-mon[40691]: 5.12 scrub starts 2024-09-16T09:37:59.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:59 smithi070 ceph-mon[40691]: 5.12 scrub ok 2024-09-16T09:37:59.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:59 smithi070 ceph-mon[40691]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:37:59.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:59 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3879816545' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-16T09:37:59.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:59 smithi070 ceph-mon[40691]: osdmap e384: 8 total, 8 up, 8 in 2024-09-16T09:37:59.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:59 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/865736403' entity='client.admin' cmd={"prefix": "osd reweight", "id": 6, "weight": 1.0} : dispatch 2024-09-16T09:37:59.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:59 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/865736403' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-16T09:37:59.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:37:59 smithi070 ceph-mon[40691]: osdmap e385: 8 total, 8 up, 8 in 2024-09-16T09:37:59.696 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:37:59.730 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:37:59.731 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:59.731 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:37:59.832 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:37:59.867 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:37:59.868 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:37:59.868 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:37:59.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:59 smithi084 ceph-mon[32629]: pgmap v686: 168 pgs: 2 peering, 166 active+clean; 103 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 3.1 KiB/s rd, 13 MiB/s wr, 208 op/s; 6.5 KiB/s, 0 objects/s recovering 2024-09-16T09:37:59.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:59 smithi084 ceph-mon[32629]: 5.12 scrub starts 2024-09-16T09:37:59.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:59 smithi084 ceph-mon[32629]: 5.12 scrub ok 2024-09-16T09:37:59.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:59 smithi084 ceph-mon[32629]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:37:59.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:59 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3879816545' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-16T09:37:59.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:59 smithi084 ceph-mon[32629]: osdmap e384: 8 total, 8 up, 8 in 2024-09-16T09:37:59.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:59 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/865736403' entity='client.admin' cmd={"prefix": "osd reweight", "id": 6, "weight": 1.0} : dispatch 2024-09-16T09:37:59.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:59 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/865736403' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-16T09:37:59.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:37:59 smithi084 ceph-mon[32629]: osdmap e385: 8 total, 8 up, 8 in 2024-09-16T09:37:59.970 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:38:00.001 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:00.002 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:00.003 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:38:00.103 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:38:00.137 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:00.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:00.138 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:38:00.240 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:38:00.275 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:00.275 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:00.276 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:38:00.377 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:38:00.400 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.1 dump_historic_ops 2024-09-16T09:38:00.413 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:00.413 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:00.414 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:38:00.515 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:38:00.535 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 4 16 13335 13319 207.975 198.938 0.00672136 0.0048016 2024-09-16T09:38:00.553 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:00.553 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:00.554 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:38:00.655 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:38:00.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:00 smithi070 ceph-mon[30388]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-09-16T09:38:00.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:00 smithi070 ceph-mon[40691]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-09-16T09:38:00.696 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:00.697 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:00.697 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:38:00.798 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:38:00.855 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:00.855 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:00.856 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:38:00.857 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd reweight 7 1 2024-09-16T09:38:00.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:00 smithi084 ceph-mon[32629]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-09-16T09:38:00.956 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:38:00.998 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:00.999 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:00.999 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:38:01.100 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:38:01.134 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:01.134 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:01.135 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:38:01.236 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:38:01.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:01.274 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:01.274 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:38:01.374 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:38:01.412 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:38:01.422 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:01.423 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:01.423 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:38:01.424 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:38:01.478 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:38:01.524 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:38:01.535 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 5 16 16138 16122 201.414 175.188 0.0077017 0.0049576 2024-09-16T09:38:01.558 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:01.559 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:01.559 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:38:01.569 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:01 smithi070 ceph-mon[30388]: pgmap v689: 168 pgs: 168 active+clean; 359 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 56 MiB/s wr, 902 op/s 2024-09-16T09:38:01.569 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:01 smithi070 ceph-mon[40691]: pgmap v689: 168 pgs: 168 active+clean; 359 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 56 MiB/s wr, 902 op/s 2024-09-16T09:38:01.662 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:38:01.697 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:01.698 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:01.698 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:38:01.799 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:38:01.836 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:01.837 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:01.838 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:38:01.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:01 smithi084 ceph-mon[32629]: pgmap v689: 168 pgs: 168 active+clean; 359 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 56 MiB/s wr, 902 op/s 2024-09-16T09:38:01.938 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:38:01.973 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:01.974 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:01.974 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:38:02.075 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:38:02.107 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:02.108 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:02.108 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:38:02.134 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:38:01 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:38:01] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:38:02.210 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:38:02.246 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:02.247 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:02.247 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:38:02.310 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T09:38:02.310 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:38:02.348 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:38:02.385 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:02.386 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:02.386 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:38:02.487 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:38:02.523 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:02.523 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:02.524 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:38:02.535 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 6 16 18966 18950 197.302 176.75 0.00476457 0.00506315 2024-09-16T09:38:02.624 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:38:02.666 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:02.667 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:02.667 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:38:02.768 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:38:02.807 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:02.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:02.808 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:38:02.909 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:38:02.943 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:02.943 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:02.944 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:38:03.044 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:38:03.071 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:03.071 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:03.072 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:38:03.173 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:38:03.207 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:03.208 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:03.208 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:38:03.292 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.2 dump_ops_in_flight 2024-09-16T09:38:03.309 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:38:03.372 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:03.372 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:03.372 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:38:03.380 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-16T09:38:03.473 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:38:03.509 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:03.509 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:03.509 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:38:03.536 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 7 16 21448 21432 191.277 155.125 0.00428769 0.00522219 2024-09-16T09:38:03.611 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:38:03.660 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:03.661 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:03.661 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:38:03.662 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:38:03 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:38:03.662 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:03 smithi084 ceph-mon[32629]: pgmap v690: 168 pgs: 168 active+clean; 359 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 45 MiB/s wr, 714 op/s 2024-09-16T09:38:03.663 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:03 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3346730007' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:38:03.663 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:03 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:38:03.663 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:03 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:38:03.762 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:38:03.793 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:03.794 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:03.794 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:38:03.895 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:38:03.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:03 smithi070 ceph-mon[40691]: pgmap v690: 168 pgs: 168 active+clean; 359 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 45 MiB/s wr, 714 op/s 2024-09-16T09:38:03.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:03 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3346730007' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:38:03.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:03 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:38:03.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:03 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:38:03.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:03 smithi070 ceph-mon[30388]: pgmap v690: 168 pgs: 168 active+clean; 359 MiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 45 MiB/s wr, 714 op/s 2024-09-16T09:38:03.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:03 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3346730007' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:38:03.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:03 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:38:03.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:03 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:38:03.932 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:03.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:03.933 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:38:04.034 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:38:04.059 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:04.060 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:04.061 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:38:04.105 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:38:04.105 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:38:04.105 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:38:04.105 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:38:04.105 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:38:04.105 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:38:04.105 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:38:04.105 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:38:04.105 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:38:04.106 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:38:04.106 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:38:04.106 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:38:04.106 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:38:04.106 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:38:04.106 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:38:04.106 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:38:04.161 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:38:04.188 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:04.188 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:04.189 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:38:04.289 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:38:04.329 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:04.329 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:04.330 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:38:04.430 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:38:04.456 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:04.456 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:04.457 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:38:04.535 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 8 16 23719 23703 185.11 141.938 0.003802 0.00539704 2024-09-16T09:38:04.558 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:38:04.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:04.594 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:04.595 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:38:04.695 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:38:04.730 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:04.731 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:04.731 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:38:04.832 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:38:04.866 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:04.867 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:04.867 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:38:04.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:04 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:38:04.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:04 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:38:04.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:04 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:38:04.968 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:38:05.008 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:05.009 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:05.009 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:38:05.110 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:38:05.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:05.149 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:05.149 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:38:05.250 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:38:05.271 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:05.272 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:05.272 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:38:05.373 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:38:05.394 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:05.395 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:05.395 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:38:05.496 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:38:05.517 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:05.517 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:05.518 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:38:05.535 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 9 16 25427 25411 176.404 106.75 0.00180845 0.00562069 2024-09-16T09:38:05.619 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:38:05.651 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:05.652 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:05.652 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:38:05.753 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:38:05.787 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:05.788 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:05.788 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:38:05.889 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:38:05.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:05 smithi084 ceph-mon[32629]: pgmap v691: 168 pgs: 168 active+clean; 1.1 GiB data, 7.3 GiB used, 708 GiB / 715 GiB avail; 139 MiB/s wr, 2.23k op/s 2024-09-16T09:38:05.916 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:05.917 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:05.917 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:38:05.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:05 smithi070 ceph-mon[30388]: pgmap v691: 168 pgs: 168 active+clean; 1.1 GiB data, 7.3 GiB used, 708 GiB / 715 GiB avail; 139 MiB/s wr, 2.23k op/s 2024-09-16T09:38:05.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:05 smithi070 ceph-mon[40691]: pgmap v691: 168 pgs: 168 active+clean; 1.1 GiB data, 7.3 GiB used, 708 GiB / 715 GiB avail; 139 MiB/s wr, 2.23k op/s 2024-09-16T09:38:06.018 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:38:06.040 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:06.040 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:06.041 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:38:06.142 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:38:06.163 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:06.163 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:06.164 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:38:06.265 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:38:06.301 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:06.301 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:06.302 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:38:06.403 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:38:06.438 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:06.439 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:06.439 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:38:06.535 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 10 16 27177 27161 169.702 109.375 0.00698685 0.0058874 2024-09-16T09:38:06.540 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:38:06.576 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:06.576 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:06.577 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:38:06.677 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:38:06.707 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:06.707 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:06.708 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:38:06.809 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:38:06.845 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:06.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:06.846 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:38:06.947 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:38:06.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:06.981 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:06.982 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:38:07.082 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:38:07.114 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:07.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:07.115 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:38:07.216 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:38:07.249 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:07.250 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:07.250 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:38:07.351 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:38:07.389 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:07.390 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:07.390 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:38:07.491 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:38:07.511 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:07.512 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:07.513 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:38:07.535 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 11 16 29131 29115 165.377 122.125 0.0051003 0.00602195 2024-09-16T09:38:07.613 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:38:07.647 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:07.647 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:07.648 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:38:07.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:07 smithi070 ceph-mon[30388]: pgmap v692: 168 pgs: 168 active+clean; 1.3 GiB data, 7.8 GiB used, 707 GiB / 715 GiB avail; 147 MiB/s wr, 2.35k op/s 2024-09-16T09:38:07.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:07 smithi070 ceph-mon[40691]: pgmap v692: 168 pgs: 168 active+clean; 1.3 GiB data, 7.8 GiB used, 707 GiB / 715 GiB avail; 147 MiB/s wr, 2.35k op/s 2024-09-16T09:38:07.748 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:38:07.784 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:07.784 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:07.785 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:38:07.885 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:38:07.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:07 smithi084 ceph-mon[32629]: pgmap v692: 168 pgs: 168 active+clean; 1.3 GiB data, 7.8 GiB used, 707 GiB / 715 GiB avail; 147 MiB/s wr, 2.35k op/s 2024-09-16T09:38:07.917 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:07.918 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:07.918 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:38:08.019 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:38:08.054 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:08.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:08.056 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:38:08.157 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:38:08.191 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:08.192 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:08.192 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:38:08.293 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:38:08.331 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:08.332 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:08.332 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:38:08.433 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:38:08.468 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:08.468 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:08.469 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:38:08.536 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 12 16 31350 31334 163.149 138.688 0.00409384 0.00612241 2024-09-16T09:38:08.570 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:38:08.612 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:08.613 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:08.613 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:38:08.714 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:38:08.727 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:38:08.740 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:38:08.745 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:08.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:08.746 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:38:08.847 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:38:08.881 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:08.882 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:08.882 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:38:08.983 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:38:09.018 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:09.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:09.019 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:38:09.120 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:38:09.152 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:09.152 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:09.153 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:38:09.254 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:38:09.293 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:09.294 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:09.294 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:38:09.395 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:38:09.431 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:09.432 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:09.432 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:38:09.519 INFO:teuthology.orchestra.run.smithi070.stderr:reweighted osd.7 to 1 (10000) 2024-09-16T09:38:09.532 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:38:09.536 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 13 16 33429 33413 160.594 129.938 0.0180252 0.00622059 2024-09-16T09:38:09.565 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:09.565 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:09.566 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:38:09.669 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:38:09.690 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:09.691 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:09.691 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:38:09.792 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:38:09.814 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:09.815 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:09.815 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:38:09.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:09 smithi084 ceph-mon[32629]: pgmap v693: 168 pgs: 168 active+clean; 1.5 GiB data, 8.1 GiB used, 707 GiB / 715 GiB avail; 549 B/s rd, 136 MiB/s wr, 2.17k op/s 2024-09-16T09:38:09.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:09 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1340326135' entity='client.admin' cmd={"prefix": "osd reweight", "id": 7, "weight": 1.0} : dispatch 2024-09-16T09:38:09.916 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:38:09.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:09 smithi070 ceph-mon[30388]: pgmap v693: 168 pgs: 168 active+clean; 1.5 GiB data, 8.1 GiB used, 707 GiB / 715 GiB avail; 549 B/s rd, 136 MiB/s wr, 2.17k op/s 2024-09-16T09:38:09.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:09 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1340326135' entity='client.admin' cmd={"prefix": "osd reweight", "id": 7, "weight": 1.0} : dispatch 2024-09-16T09:38:09.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:09 smithi070 ceph-mon[40691]: pgmap v693: 168 pgs: 168 active+clean; 1.5 GiB data, 8.1 GiB used, 707 GiB / 715 GiB avail; 549 B/s rd, 136 MiB/s wr, 2.17k op/s 2024-09-16T09:38:09.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:09 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1340326135' entity='client.admin' cmd={"prefix": "osd reweight", "id": 7, "weight": 1.0} : dispatch 2024-09-16T09:38:09.949 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:09.950 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:09.950 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:38:10.051 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:38:10.087 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:10.088 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:10.088 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:38:10.189 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:38:10.226 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:10.226 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:10.227 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:38:10.328 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:38:10.367 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:10.368 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:10.368 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:38:10.406 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T09:38:10.469 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:38:10.512 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:10.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:10.513 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:38:10.537 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 14 15 34946 34931 155.9 94.875 0.0100003 0.00640909 2024-09-16T09:38:10.614 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:38:10.635 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:10.636 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:10.636 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:38:10.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:10 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1340326135' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-16T09:38:10.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:10 smithi070 ceph-mon[30388]: osdmap e386: 8 total, 8 up, 8 in 2024-09-16T09:38:10.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:10 smithi070 ceph-mon[30388]: pgmap v695: 168 pgs: 168 active+clean; 1.9 GiB data, 9.7 GiB used, 706 GiB / 715 GiB avail; 1023 B/s rd, 155 MiB/s wr, 2.48k op/s 2024-09-16T09:38:10.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:10 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1340326135' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-16T09:38:10.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:10 smithi070 ceph-mon[40691]: osdmap e386: 8 total, 8 up, 8 in 2024-09-16T09:38:10.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:10 smithi070 ceph-mon[40691]: pgmap v695: 168 pgs: 168 active+clean; 1.9 GiB data, 9.7 GiB used, 706 GiB / 715 GiB avail; 1023 B/s rd, 155 MiB/s wr, 2.48k op/s 2024-09-16T09:38:10.737 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:38:10.774 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:10.774 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:10.775 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:38:10.875 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:38:10.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:10 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1340326135' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-16T09:38:10.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:10 smithi084 ceph-mon[32629]: osdmap e386: 8 total, 8 up, 8 in 2024-09-16T09:38:10.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:10 smithi084 ceph-mon[32629]: pgmap v695: 168 pgs: 168 active+clean; 1.9 GiB data, 9.7 GiB used, 706 GiB / 715 GiB avail; 1023 B/s rd, 155 MiB/s wr, 2.48k op/s 2024-09-16T09:38:10.911 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:10.912 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:10.912 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:38:11.013 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:38:11.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:11.051 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:11.052 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:38:11.152 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:38:11.183 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T09:38:11.183 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T09:38:11.183 INFO:teuthology.orchestra.run.smithi070.stdout: "num_ops": 0 2024-09-16T09:38:11.183 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T09:38:11.195 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:11.196 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:11.196 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:38:11.297 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:38:11.331 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:11.332 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:11.332 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:38:11.433 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:38:11.468 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:11.469 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:11.469 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:38:11.536 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 15 16 36775 36759 153.123 114.25 0.00543166 0.00652583 2024-09-16T09:38:11.549 INFO:teuthology.orchestra.run.smithi070.stderr:nodeep-scrub is set 2024-09-16T09:38:11.570 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:38:11.602 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:11.603 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:11.603 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:38:11.704 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:38:11.737 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:11.738 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:11.739 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:38:11.840 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:38:11.877 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:11.878 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:11.878 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:38:11.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:11 smithi084 ceph-mon[32629]: osdmap e387: 8 total, 8 up, 8 in 2024-09-16T09:38:11.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:11 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1120493250' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:38:11.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:11 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:38:11.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:11 smithi070 ceph-mon[30388]: osdmap e387: 8 total, 8 up, 8 in 2024-09-16T09:38:11.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:11 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1120493250' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:38:11.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:11 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:38:11.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:11 smithi070 ceph-mon[40691]: osdmap e387: 8 total, 8 up, 8 in 2024-09-16T09:38:11.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:11 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1120493250' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:38:11.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:11 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:38:11.979 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:38:12.011 INFO:tasks.cephadm.ceph_manager.ceph:waiting for recovery to complete 2024-09-16T09:38:12.012 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-16T09:38:12.023 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:12.024 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:12.024 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:38:12.126 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:38:12.168 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:12.169 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:12.169 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:38:12.231 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:38:11 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:38:11] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:38:12.270 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:38:12.307 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:12.307 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:12.308 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:38:12.409 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:38:12.451 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:12.452 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:12.452 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:38:12.537 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 16 16 38851 38835 151.655 129.75 0.0071226 0.00658855 2024-09-16T09:38:12.553 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:38:12.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:12.595 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:12.595 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:38:12.696 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:38:12.731 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:12.731 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:12.732 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:38:12.833 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:38:12.873 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:12.873 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:12.874 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:38:12.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:12 smithi084 ceph-mon[32629]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:38:12.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:12 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-16T09:38:12.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:12 smithi084 ceph-mon[32629]: osdmap e388: 8 total, 8 up, 8 in 2024-09-16T09:38:12.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:12 smithi084 ceph-mon[32629]: pgmap v698: 168 pgs: 168 active+clean; 1.9 GiB data, 9.7 GiB used, 706 GiB / 715 GiB avail; 853 B/s rd, 105 MiB/s wr, 1.67k op/s 2024-09-16T09:38:12.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:12 smithi070 ceph-mon[30388]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:38:12.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:12 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-16T09:38:12.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:12 smithi070 ceph-mon[30388]: osdmap e388: 8 total, 8 up, 8 in 2024-09-16T09:38:12.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:12 smithi070 ceph-mon[30388]: pgmap v698: 168 pgs: 168 active+clean; 1.9 GiB data, 9.7 GiB used, 706 GiB / 715 GiB avail; 853 B/s rd, 105 MiB/s wr, 1.67k op/s 2024-09-16T09:38:12.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:12 smithi070 ceph-mon[40691]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:38:12.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:12 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-16T09:38:12.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:12 smithi070 ceph-mon[40691]: osdmap e388: 8 total, 8 up, 8 in 2024-09-16T09:38:12.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:12 smithi070 ceph-mon[40691]: pgmap v698: 168 pgs: 168 active+clean; 1.9 GiB data, 9.7 GiB used, 706 GiB / 715 GiB avail; 853 B/s rd, 105 MiB/s wr, 1.67k op/s 2024-09-16T09:38:12.975 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:38:13.021 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.2 dump_blocked_ops 2024-09-16T09:38:13.024 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:13.025 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:13.025 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:38:13.055 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:38:13.127 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:38:13.168 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:13.169 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:13.169 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:38:13.270 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:38:13.317 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:13.318 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:13.318 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:38:13.419 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:38:13.453 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:13.453 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:13.453 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:38:13.537 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 17 16 40856 40840 150.102 125.312 0.0115378 0.00665564 2024-09-16T09:38:13.554 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:38:13.557 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:38:13.569 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:13 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:38:13.569 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:38:13 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:38:13.582 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:13.583 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:13.583 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:38:13.684 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:38:13.716 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:13.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:13.718 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:38:13.804 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:13 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:38:13.804 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:13 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:38:13.819 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:38:13.851 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:13.852 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:13.853 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:38:13.953 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:38:13.987 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:13.987 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:13.987 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:38:14.088 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:38:14.124 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:14.126 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:14.126 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:38:14.227 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:38:14.266 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:14.267 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:14.268 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:38:14.368 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:38:14.407 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:14.408 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:14.409 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:38:14.450 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:38:14.450 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:38:14.480 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:38:14.480 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:38:14.509 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:38:14.518 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:38:14.518 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:38:14.537 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 18 16 42666 42650 148.048 113.125 0.00715016 0.00674963 2024-09-16T09:38:14.547 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:14.548 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:14.548 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:38:14.550 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:38:14.550 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:38:14.584 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:38:14.584 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:38:14.619 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:38:14.619 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:38:14.649 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:38:14.654 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:38:14.655 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:38:14.685 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:14.686 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:14.686 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:38:14.692 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:38:14.692 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:38:14.787 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:38:14.818 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:14.819 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:14.819 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:38:14.920 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:38:14.955 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:14.956 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:14.956 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:38:15.057 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:38:15.093 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:15.094 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:15.094 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:38:15.195 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:38:15.228 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:15.229 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:15.229 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:38:15.330 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:38:15.364 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:15.365 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:15.365 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:38:15.466 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:38:15.501 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:15.502 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:15.502 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:38:15.537 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 19 16 44520 44504 146.354 115.875 0.00513985 0.0068276 2024-09-16T09:38:15.576 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd unset noscrub 2024-09-16T09:38:15.596 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:15 smithi070 ceph-mon[30388]: pgmap v699: 168 pgs: 168 active+clean; 2.3 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.7 KiB/s rd, 147 MiB/s wr, 2.36k op/s 2024-09-16T09:38:15.596 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:15 smithi070 ceph-mon[30388]: osdmap e389: 8 total, 8 up, 8 in 2024-09-16T09:38:15.596 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:15 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3635830096' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:38:15.596 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:15 smithi070 ceph-mon[40691]: pgmap v699: 168 pgs: 168 active+clean; 2.3 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.7 KiB/s rd, 147 MiB/s wr, 2.36k op/s 2024-09-16T09:38:15.597 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:15 smithi070 ceph-mon[40691]: osdmap e389: 8 total, 8 up, 8 in 2024-09-16T09:38:15.597 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:15 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3635830096' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:38:15.603 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:38:15.646 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:15.647 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:15.647 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:38:15.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:15 smithi084 ceph-mon[32629]: pgmap v699: 168 pgs: 168 active+clean; 2.3 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.7 KiB/s rd, 147 MiB/s wr, 2.36k op/s 2024-09-16T09:38:15.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:15 smithi084 ceph-mon[32629]: osdmap e389: 8 total, 8 up, 8 in 2024-09-16T09:38:15.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:15 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3635830096' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:38:15.748 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:38:15.797 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:15.798 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:15.799 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:38:15.899 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:38:15.929 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:15.930 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:15.931 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:38:16.031 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:38:16.069 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:16.069 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:16.069 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:38:16.118 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:38:16.122 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2024-09-16T09:38:16.171 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:38:16.208 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:16.209 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:16.209 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:38:16.310 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:38:16.348 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:16.348 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:16.349 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:38:16.450 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:38:16.490 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:16.490 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:16.490 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:38:16.493 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:38:16.538 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:2024-09-16T09:38:16.538166+0000 min lat: 0.00121703 max lat: 0.282667 avg lat: 0.00691643 2024-09-16T09:38:16.539 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-16T09:38:16.539 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 20 16 46264 46248 144.481 109 0.00688685 0.00691643 2024-09-16T09:38:16.592 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:38:16.625 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:16.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:16.627 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:38:16.727 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:38:16.770 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:16.770 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:16.771 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:38:16.871 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:38:16.909 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:16.910 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:16.910 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:38:17.011 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:38:17.044 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:17.045 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:17.045 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:38:17.146 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:38:17.173 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:17.174 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:17.175 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:38:17.275 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:38:17.308 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:17.309 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:17.309 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:38:17.410 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:38:17.447 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:17.448 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:17.448 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:38:17.538 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 21 16 48059 48043 142.943 112.188 0.00560364 0.00699083 2024-09-16T09:38:17.549 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:38:17.584 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:17.585 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:17.586 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:38:17.656 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:17 smithi070 ceph-mon[30388]: pgmap v701: 168 pgs: 168 active+clean; 2.4 GiB data, 12 GiB used, 704 GiB / 715 GiB avail; 852 B/s rd, 100 MiB/s wr, 1.60k op/s 2024-09-16T09:38:17.656 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:17 smithi070 ceph-mon[30388]: from='client.16005 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:38:17.656 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:17 smithi070 ceph-mon[40691]: pgmap v701: 168 pgs: 168 active+clean; 2.4 GiB data, 12 GiB used, 704 GiB / 715 GiB avail; 852 B/s rd, 100 MiB/s wr, 1.60k op/s 2024-09-16T09:38:17.657 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:17 smithi070 ceph-mon[40691]: from='client.16005 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:38:17.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:17 smithi084 ceph-mon[32629]: pgmap v701: 168 pgs: 168 active+clean; 2.4 GiB data, 12 GiB used, 704 GiB / 715 GiB avail; 852 B/s rd, 100 MiB/s wr, 1.60k op/s 2024-09-16T09:38:17.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:17 smithi084 ceph-mon[32629]: from='client.16005 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:38:17.687 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:38:17.720 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:17.721 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:17.721 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:38:17.822 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:38:17.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:17.861 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:17.861 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:38:17.962 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:38:17.997 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:17.998 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:17.999 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:38:18.100 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:38:18.137 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:18.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:18.138 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:38:18.146 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T09:38:18.146 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:38:18.239 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:38:18.272 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:18.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:18.273 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:38:18.374 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:38:18.412 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:18.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:18.413 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:38:18.514 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:38:18.539 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 22 16 49881 49865 141.621 113.875 0.00818404 0.00705605 2024-09-16T09:38:18.549 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:18.550 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:18.551 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:38:18.584 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":701,"stamp":"2024-09-16T09:38:15.757951+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2628721179,"num_objects":40301,"num_object_clones":0,"num_object_copies":120903,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40301,"num_whiteouts":0,"num_read":1926,"num_read_kb":1678,"num_write":40824,"num_write_kb":2566573,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":140,"num_bytes_recovered":333242,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42530,"ondisk_log_size":42530,"up":504,"acting":504,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":506,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":12069860,"kb_used_data":7738748,"kb_used_omap":12,"kb_used_meta":4330931,"kb_avail":737891356,"statfs":{"total":767960285184,"available":755600748544,"internally_reserved":0,"allocated":7924477952,"data_stored":7919499385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":4434873906},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,3,6,1,0,0,0,1,4],"upper_bound":512},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":628424704,"num_objects":9589,"num_object_clones":0,"num_object_copies":28767,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9589,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":9589,"num_write_kb":613696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.003204"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":708,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.558727+0000","last_change":"2024-09-16T09:32:39.384174+0000","last_active":"2024-09-16T09:38:11.558727+0000","last_peered":"2024-09-16T09:38:11.558727+0000","last_clean":"2024-09-16T09:38:11.558727+0000","last_became_active":"2024-09-16T09:25:01.709108+0000","last_became_peered":"2024-09-16T09:25:01.709108+0000","last_unstale":"2024-09-16T09:38:11.558727+0000","last_undegraded":"2024-09-16T09:38:11.558727+0000","last_fullsized":"2024-09-16T09:38:11.558727+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:39.384104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:32:39.384104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"285'15","reported_seq":720,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.549468+0000","last_change":"2024-09-16T09:32:38.074301+0000","last_active":"2024-09-16T09:38:11.549468+0000","last_peered":"2024-09-16T09:38:11.549468+0000","last_clean":"2024-09-16T09:38:11.549468+0000","last_became_active":"2024-09-16T09:25:03.714635+0000","last_became_peered":"2024-09-16T09:25:03.714635+0000","last_unstale":"2024-09-16T09:38:11.549468+0000","last_undegraded":"2024-09-16T09:38:11.549468+0000","last_fullsized":"2024-09-16T09:38:11.549468+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:32:38.074231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:32:38.074231+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:07.303929+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":330,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.549583+0000","last_change":"2024-09-16T09:36:53.098680+0000","last_active":"2024-09-16T09:38:11.549583+0000","last_peered":"2024-09-16T09:38:11.549583+0000","last_clean":"2024-09-16T09:38:11.549583+0000","last_became_active":"2024-09-16T09:33:30.242238+0000","last_became_peered":"2024-09-16T09:33:30.242238+0000","last_unstale":"2024-09-16T09:38:11.549583+0000","last_undegraded":"2024-09-16T09:38:11.549583+0000","last_fullsized":"2024-09-16T09:38:11.549583+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:53.098635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:36:53.098635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":687,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.548396+0000","last_change":"2024-09-16T09:37:56.314518+0000","last_active":"2024-09-16T09:38:11.548396+0000","last_peered":"2024-09-16T09:38:11.548396+0000","last_clean":"2024-09-16T09:38:11.548396+0000","last_became_active":"2024-09-16T09:25:07.715442+0000","last_became_peered":"2024-09-16T09:25:07.715442+0000","last_unstale":"2024-09-16T09:38:11.548396+0000","last_undegraded":"2024-09-16T09:38:11.548396+0000","last_fullsized":"2024-09-16T09:38:11.548396+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:56.314468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:37:56.314468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:04.475570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":424,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.554353+0000","last_change":"2024-09-16T09:34:33.452784+0000","last_active":"2024-09-16T09:38:11.554353+0000","last_peered":"2024-09-16T09:38:11.554353+0000","last_clean":"2024-09-16T09:38:11.554353+0000","last_became_active":"2024-09-16T09:31:09.269342+0000","last_became_peered":"2024-09-16T09:31:09.269342+0000","last_unstale":"2024-09-16T09:38:11.554353+0000","last_undegraded":"2024-09-16T09:38:11.554353+0000","last_fullsized":"2024-09-16T09:38:11.554353+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:33.452706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:34:33.452706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":565,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.552512+0000","last_change":"2024-09-16T09:33:08.816231+0000","last_active":"2024-09-16T09:38:11.552512+0000","last_peered":"2024-09-16T09:38:11.552512+0000","last_clean":"2024-09-16T09:38:11.552512+0000","last_became_active":"2024-09-16T09:31:09.275136+0000","last_became_peered":"2024-09-16T09:31:09.275136+0000","last_unstale":"2024-09-16T09:38:11.552512+0000","last_undegraded":"2024-09-16T09:38:11.552512+0000","last_fullsized":"2024-09-16T09:38:11.552512+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:08.816174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:08.816174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:08.989282+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.19","version":"285'27","reported_seq":406,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.559212+0000","last_change":"2024-09-16T09:37:29.683657+0000","last_active":"2024-09-16T09:38:11.559212+0000","last_peered":"2024-09-16T09:38:11.559212+0000","last_clean":"2024-09-16T09:38:11.559212+0000","last_became_active":"2024-09-16T09:33:21.337284+0000","last_became_peered":"2024-09-16T09:33:21.337284+0000","last_unstale":"2024-09-16T09:38:11.559212+0000","last_undegraded":"2024-09-16T09:38:11.559212+0000","last_fullsized":"2024-09-16T09:38:11.559212+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'27","last_scrub_stamp":"2024-09-16T09:37:29.683568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:37:29.683568+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:35.791977+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":699,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.549398+0000","last_change":"2024-09-16T09:32:44.114475+0000","last_active":"2024-09-16T09:38:11.549398+0000","last_peered":"2024-09-16T09:38:11.549398+0000","last_clean":"2024-09-16T09:38:11.549398+0000","last_became_active":"2024-09-16T09:25:05.717236+0000","last_became_peered":"2024-09-16T09:25:05.717236+0000","last_unstale":"2024-09-16T09:38:11.549398+0000","last_undegraded":"2024-09-16T09:38:11.549398+0000","last_fullsized":"2024-09-16T09:38:11.549398+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:44.114407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:32:44.114407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:03.327373+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":331,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.560468+0000","last_change":"2024-09-16T09:33:36.655594+0000","last_active":"2024-09-16T09:38:11.560468+0000","last_peered":"2024-09-16T09:38:11.560468+0000","last_clean":"2024-09-16T09:38:11.560468+0000","last_became_active":"2024-09-16T09:33:30.247733+0000","last_became_peered":"2024-09-16T09:33:30.247733+0000","last_unstale":"2024-09-16T09:38:11.560468+0000","last_undegraded":"2024-09-16T09:38:11.560468+0000","last_fullsized":"2024-09-16T09:38:11.560468+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:36.655236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:36.655236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:00.446453+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1a","version":"285'15","reported_seq":720,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.548733+0000","last_change":"2024-09-16T09:32:48.185484+0000","last_active":"2024-09-16T09:38:11.548733+0000","last_peered":"2024-09-16T09:38:11.548733+0000","last_clean":"2024-09-16T09:38:11.548733+0000","last_became_active":"2024-09-16T09:25:03.709147+0000","last_became_peered":"2024-09-16T09:25:03.709147+0000","last_unstale":"2024-09-16T09:38:11.548733+0000","last_undegraded":"2024-09-16T09:38:11.548733+0000","last_fullsized":"2024-09-16T09:38:11.548733+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:32:48.185380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:32:48.185380+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":690,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.548644+0000","last_change":"2024-09-16T09:36:51.134364+0000","last_active":"2024-09-16T09:38:11.548644+0000","last_peered":"2024-09-16T09:38:11.548644+0000","last_clean":"2024-09-16T09:38:11.548644+0000","last_became_active":"2024-09-16T09:25:05.718134+0000","last_became_peered":"2024-09-16T09:25:05.718134+0000","last_unstale":"2024-09-16T09:38:11.548644+0000","last_undegraded":"2024-09-16T09:38:11.548644+0000","last_fullsized":"2024-09-16T09:38:11.548644+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:51.134314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:36:51.134314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"66'1","reported_seq":696,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.559050+0000","last_change":"2024-09-16T09:33:38.705001+0000","last_active":"2024-09-16T09:38:11.559050+0000","last_peered":"2024-09-16T09:38:11.559050+0000","last_clean":"2024-09-16T09:38:11.559050+0000","last_became_active":"2024-09-16T09:25:07.716558+0000","last_became_peered":"2024-09-16T09:25:07.716558+0000","last_unstale":"2024-09-16T09:38:11.559050+0000","last_undegraded":"2024-09-16T09:38:11.559050+0000","last_fullsized":"2024-09-16T09:38:11.559050+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-16T09:33:38.704868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:33:38.704868+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":690,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.549248+0000","last_change":"2024-09-16T09:33:19.326765+0000","last_active":"2024-09-16T09:38:11.549248+0000","last_peered":"2024-09-16T09:38:11.549248+0000","last_clean":"2024-09-16T09:38:11.549248+0000","last_became_active":"2024-09-16T09:25:01.696939+0000","last_became_peered":"2024-09-16T09:25:01.696939+0000","last_unstale":"2024-09-16T09:38:11.549248+0000","last_undegraded":"2024-09-16T09:38:11.549248+0000","last_fullsized":"2024-09-16T09:38:11.549248+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:19.326695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:19.326695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:11.230887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"285'7","reported_seq":708,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.546352+0000","last_change":"2024-09-16T09:35:39.975915+0000","last_active":"2024-09-16T09:38:11.546352+0000","last_peered":"2024-09-16T09:38:11.546352+0000","last_clean":"2024-09-16T09:38:11.546352+0000","last_became_active":"2024-09-16T09:25:03.716136+0000","last_became_peered":"2024-09-16T09:25:03.716136+0000","last_unstale":"2024-09-16T09:38:11.546352+0000","last_undegraded":"2024-09-16T09:38:11.546352+0000","last_fullsized":"2024-09-16T09:38:11.546352+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'7","last_scrub_stamp":"2024-09-16T09:35:39.975840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:35:39.975840+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:00.297013+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":10,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":681,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.557797+0000","last_change":"2024-09-16T09:28:52.746046+0000","last_active":"2024-09-16T09:38:11.557797+0000","last_peered":"2024-09-16T09:38:11.557797+0000","last_clean":"2024-09-16T09:38:11.557797+0000","last_became_active":"2024-09-16T09:25:05.718281+0000","last_became_peered":"2024-09-16T09:25:05.718281+0000","last_unstale":"2024-09-16T09:38:11.557797+0000","last_undegraded":"2024-09-16T09:38:11.557797+0000","last_fullsized":"2024-09-16T09:38:11.557797+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:52.745999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:28:52.745999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:04.420096+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":708,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.549674+0000","last_change":"2024-09-16T09:33:46.366042+0000","last_active":"2024-09-16T09:38:11.549674+0000","last_peered":"2024-09-16T09:38:11.549674+0000","last_clean":"2024-09-16T09:38:11.549674+0000","last_became_active":"2024-09-16T09:31:09.275656+0000","last_became_peered":"2024-09-16T09:31:09.275656+0000","last_unstale":"2024-09-16T09:38:11.549674+0000","last_undegraded":"2024-09-16T09:38:11.549674+0000","last_fullsized":"2024-09-16T09:38:11.549674+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:46.365972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:33:46.365972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":73,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":696,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.548060+0000","last_change":"2024-09-16T09:33:48.322010+0000","last_active":"2024-09-16T09:38:11.548060+0000","last_peered":"2024-09-16T09:38:11.548060+0000","last_clean":"2024-09-16T09:38:11.548060+0000","last_became_active":"2024-09-16T09:25:07.720793+0000","last_became_peered":"2024-09-16T09:25:07.720793+0000","last_unstale":"2024-09-16T09:38:11.548060+0000","last_undegraded":"2024-09-16T09:38:11.548060+0000","last_fullsized":"2024-09-16T09:38:11.548060+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:48.321895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:33:48.321895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:41.370700+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":447,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.552077+0000","last_change":"2024-09-16T09:34:33.866299+0000","last_active":"2024-09-16T09:38:11.552077+0000","last_peered":"2024-09-16T09:38:11.552077+0000","last_clean":"2024-09-16T09:38:11.552077+0000","last_became_active":"2024-09-16T09:30:39.201824+0000","last_became_peered":"2024-09-16T09:30:39.201824+0000","last_unstale":"2024-09-16T09:38:11.552077+0000","last_undegraded":"2024-09-16T09:38:11.552077+0000","last_fullsized":"2024-09-16T09:38:11.552077+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:33.866142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:34:33.866142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:40.929489+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"285'27","reported_seq":795,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.553512+0000","last_change":"2024-09-16T09:36:54.484021+0000","last_active":"2024-09-16T09:38:11.553512+0000","last_peered":"2024-09-16T09:38:11.553512+0000","last_clean":"2024-09-16T09:38:11.553512+0000","last_became_active":"2024-09-16T09:33:30.249798+0000","last_became_peered":"2024-09-16T09:33:30.249798+0000","last_unstale":"2024-09-16T09:38:11.553512+0000","last_undegraded":"2024-09-16T09:38:11.553512+0000","last_fullsized":"2024-09-16T09:38:11.553512+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'27","last_scrub_stamp":"2024-09-16T09:36:54.483966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:36:54.483966+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:07.794557+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":330,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.547814+0000","last_change":"2024-09-16T09:33:43.326933+0000","last_active":"2024-09-16T09:38:11.547814+0000","last_peered":"2024-09-16T09:38:11.547814+0000","last_clean":"2024-09-16T09:38:11.547814+0000","last_became_active":"2024-09-16T09:33:30.251587+0000","last_became_peered":"2024-09-16T09:33:30.251587+0000","last_unstale":"2024-09-16T09:38:11.547814+0000","last_undegraded":"2024-09-16T09:38:11.547814+0000","last_fullsized":"2024-09-16T09:38:11.547814+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:43.326852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:43.326852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:42.368973+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,0],"acting":[4,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":544,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.552691+0000","last_change":"2024-09-16T09:34:37.804874+0000","last_active":"2024-09-16T09:38:11.552691+0000","last_peered":"2024-09-16T09:38:11.552691+0000","last_clean":"2024-09-16T09:38:11.552691+0000","last_became_active":"2024-09-16T09:28:52.974255+0000","last_became_peered":"2024-09-16T09:28:52.974255+0000","last_unstale":"2024-09-16T09:38:11.552691+0000","last_undegraded":"2024-09-16T09:38:11.552691+0000","last_fullsized":"2024-09-16T09:38:11.552691+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:37.804805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:34:37.804805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:00.917819+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":707,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.548870+0000","last_change":"2024-09-16T09:35:23.751992+0000","last_active":"2024-09-16T09:38:11.548870+0000","last_peered":"2024-09-16T09:38:11.548870+0000","last_clean":"2024-09-16T09:38:11.548870+0000","last_became_active":"2024-09-16T09:25:01.706491+0000","last_became_peered":"2024-09-16T09:25:01.706491+0000","last_unstale":"2024-09-16T09:38:11.548870+0000","last_undegraded":"2024-09-16T09:38:11.548870+0000","last_fullsized":"2024-09-16T09:38:11.548870+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:35:23.751946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:35:23.751946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"285'18","reported_seq":382,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.547966+0000","last_change":"2024-09-16T09:33:39.284526+0000","last_active":"2024-09-16T09:38:11.547966+0000","last_peered":"2024-09-16T09:38:11.547966+0000","last_clean":"2024-09-16T09:38:11.547966+0000","last_became_active":"2024-09-16T09:33:30.251626+0000","last_became_peered":"2024-09-16T09:33:30.251626+0000","last_unstale":"2024-09-16T09:38:11.547966+0000","last_undegraded":"2024-09-16T09:38:11.547966+0000","last_fullsized":"2024-09-16T09:38:11.547966+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-16T09:33:39.284451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:33:39.284451+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:34.368282+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":33,"num_read_kb":24,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":683,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.546674+0000","last_change":"2024-09-16T09:33:38.238661+0000","last_active":"2024-09-16T09:38:11.546674+0000","last_peered":"2024-09-16T09:38:11.546674+0000","last_clean":"2024-09-16T09:38:11.546674+0000","last_became_active":"2024-09-16T09:25:05.710093+0000","last_became_peered":"2024-09-16T09:25:05.710093+0000","last_unstale":"2024-09-16T09:38:11.546674+0000","last_undegraded":"2024-09-16T09:38:11.546674+0000","last_fullsized":"2024-09-16T09:38:11.546674+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:38.238593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:38.238593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:36.342171+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":694,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.558126+0000","last_change":"2024-09-16T09:37:48.412421+0000","last_active":"2024-09-16T09:38:11.558126+0000","last_peered":"2024-09-16T09:38:11.558126+0000","last_clean":"2024-09-16T09:38:11.558126+0000","last_became_active":"2024-09-16T09:25:07.719380+0000","last_became_peered":"2024-09-16T09:25:07.719380+0000","last_unstale":"2024-09-16T09:38:11.558126+0000","last_undegraded":"2024-09-16T09:38:11.558126+0000","last_fullsized":"2024-09-16T09:38:11.558126+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-16T09:37:48.411054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:37:48.411054+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:01.043235+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":339,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.551111+0000","last_change":"2024-09-16T09:34:41.401691+0000","last_active":"2024-09-16T09:38:11.551111+0000","last_peered":"2024-09-16T09:38:11.551111+0000","last_clean":"2024-09-16T09:38:11.551111+0000","last_became_active":"2024-09-16T09:33:30.245048+0000","last_became_peered":"2024-09-16T09:33:30.245048+0000","last_unstale":"2024-09-16T09:38:11.551111+0000","last_undegraded":"2024-09-16T09:38:11.551111+0000","last_fullsized":"2024-09-16T09:38:11.551111+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:34:41.401641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:34:41.401641+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:03.993235+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"285'20","reported_seq":764,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.559127+0000","last_change":"2024-09-16T09:32:30.286077+0000","last_active":"2024-09-16T09:38:11.559127+0000","last_peered":"2024-09-16T09:38:11.559127+0000","last_clean":"2024-09-16T09:38:11.559127+0000","last_became_active":"2024-09-16T09:31:09.275050+0000","last_became_peered":"2024-09-16T09:31:09.275050+0000","last_unstale":"2024-09-16T09:38:11.559127+0000","last_undegraded":"2024-09-16T09:38:11.559127+0000","last_fullsized":"2024-09-16T09:38:11.559127+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-16T09:32:30.284919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:32:30.284919+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:02.456876+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"66'8","reported_seq":886,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.552856+0000","last_change":"2024-09-16T09:34:52.922123+0000","last_active":"2024-09-16T09:38:11.552856+0000","last_peered":"2024-09-16T09:38:11.552856+0000","last_clean":"2024-09-16T09:38:11.552856+0000","last_became_active":"2024-09-16T09:25:05.712516+0000","last_became_peered":"2024-09-16T09:25:05.712516+0000","last_unstale":"2024-09-16T09:38:11.552856+0000","last_undegraded":"2024-09-16T09:38:11.552856+0000","last_fullsized":"2024-09-16T09:38:11.552856+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:34:52.922047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:34:52.922047+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":33,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1","version":"388'2485","reported_seq":2520,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:12.953437+0000","last_change":"2024-09-16T09:37:45.401338+0000","last_active":"2024-09-16T09:38:12.953437+0000","last_peered":"2024-09-16T09:38:12.953437+0000","last_clean":"2024-09-16T09:38:12.953437+0000","last_became_active":"2024-09-16T09:37:45.400966+0000","last_became_peered":"2024-09-16T09:37:45.400966+0000","last_unstale":"2024-09-16T09:38:12.953437+0000","last_undegraded":"2024-09-16T09:38:12.953437+0000","last_fullsized":"2024-09-16T09:38:12.953437+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_clean_scrub_stamp":"2024-09-16T09:37:44.378497+0000","objects_scrubbed":0,"log_size":2485,"log_dups_size":0,"ondisk_log_size":2485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:52.315503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":162856960,"num_objects":2485,"num_object_clones":0,"num_object_copies":7455,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2485,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2485,"num_write_kb":159040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.a","version":"73'79","reported_seq":630,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.550770+0000","last_change":"2024-09-16T09:37:52.203798+0000","last_active":"2024-09-16T09:38:11.550770+0000","last_peered":"2024-09-16T09:38:11.550770+0000","last_clean":"2024-09-16T09:38:11.550770+0000","last_became_active":"2024-09-16T09:26:35.127352+0000","last_became_peered":"2024-09-16T09:26:35.127352+0000","last_unstale":"2024-09-16T09:38:11.550770+0000","last_undegraded":"2024-09-16T09:38:11.550770+0000","last_fullsized":"2024-09-16T09:38:11.550770+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:37:52.203752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:37:52.203752+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:59.150165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"285'27","reported_seq":657,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.557781+0000","last_change":"2024-09-16T09:36:47.588508+0000","last_active":"2024-09-16T09:38:11.557781+0000","last_peered":"2024-09-16T09:38:11.557781+0000","last_clean":"2024-09-16T09:38:11.557781+0000","last_became_active":"2024-09-16T09:26:33.295672+0000","last_became_peered":"2024-09-16T09:26:33.295672+0000","last_unstale":"2024-09-16T09:38:11.557781+0000","last_undegraded":"2024-09-16T09:38:11.557781+0000","last_fullsized":"2024-09-16T09:38:11.557781+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'27","last_scrub_stamp":"2024-09-16T09:36:47.588449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:36:47.588449+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:14.427263+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":433,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.550921+0000","last_change":"2024-09-16T09:37:16.625626+0000","last_active":"2024-09-16T09:38:11.550921+0000","last_peered":"2024-09-16T09:38:11.550921+0000","last_clean":"2024-09-16T09:38:11.550921+0000","last_became_active":"2024-09-16T09:31:09.274534+0000","last_became_peered":"2024-09-16T09:31:09.274534+0000","last_unstale":"2024-09-16T09:38:11.550921+0000","last_undegraded":"2024-09-16T09:38:11.550921+0000","last_fullsized":"2024-09-16T09:38:11.550921+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:16.625556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:37:16.625556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:26.929216+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"66'8","reported_seq":850,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.548539+0000","last_change":"2024-09-16T09:37:49.336009+0000","last_active":"2024-09-16T09:38:11.548539+0000","last_peered":"2024-09-16T09:38:11.548539+0000","last_clean":"2024-09-16T09:38:11.548539+0000","last_became_active":"2024-09-16T09:25:05.713813+0000","last_became_peered":"2024-09-16T09:25:05.713813+0000","last_unstale":"2024-09-16T09:38:11.548539+0000","last_undegraded":"2024-09-16T09:38:11.548539+0000","last_fullsized":"2024-09-16T09:38:11.548539+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:37:49.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:37:49.335939+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:08.862747+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":686,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.548813+0000","last_change":"2024-09-16T09:36:37.152446+0000","last_active":"2024-09-16T09:38:11.548813+0000","last_peered":"2024-09-16T09:38:11.548813+0000","last_clean":"2024-09-16T09:38:11.548813+0000","last_became_active":"2024-09-16T09:25:07.717344+0000","last_became_peered":"2024-09-16T09:25:07.717344+0000","last_unstale":"2024-09-16T09:38:11.548813+0000","last_undegraded":"2024-09-16T09:38:11.548813+0000","last_fullsized":"2024-09-16T09:38:11.548813+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:37.152377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:36:37.152377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.e","version":"388'2529","reported_seq":2564,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:12.961679+0000","last_change":"2024-09-16T09:37:45.464349+0000","last_active":"2024-09-16T09:38:12.961679+0000","last_peered":"2024-09-16T09:38:12.961679+0000","last_clean":"2024-09-16T09:38:12.961679+0000","last_became_active":"2024-09-16T09:37:45.463961+0000","last_became_peered":"2024-09-16T09:37:45.463961+0000","last_unstale":"2024-09-16T09:38:12.961679+0000","last_undegraded":"2024-09-16T09:38:12.961679+0000","last_fullsized":"2024-09-16T09:38:12.961679+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_clean_scrub_stamp":"2024-09-16T09:37:44.378497+0000","objects_scrubbed":0,"log_size":2529,"log_dups_size":0,"ondisk_log_size":2529,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:49.201675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":165740544,"num_objects":2529,"num_object_clones":0,"num_object_copies":7587,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2529,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2529,"num_write_kb":161856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"73'79","reported_seq":662,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.548298+0000","last_change":"2024-09-16T09:32:42.123139+0000","last_active":"2024-09-16T09:38:11.548298+0000","last_peered":"2024-09-16T09:38:11.548298+0000","last_clean":"2024-09-16T09:38:11.548298+0000","last_became_active":"2024-09-16T09:30:08.763683+0000","last_became_peered":"2024-09-16T09:30:08.763683+0000","last_unstale":"2024-09-16T09:38:11.548298+0000","last_undegraded":"2024-09-16T09:38:11.548298+0000","last_fullsized":"2024-09-16T09:38:11.548298+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:32:42.123067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:32:42.123067+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:09.339965+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"285'30","reported_seq":654,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.557921+0000","last_change":"2024-09-16T09:33:21.333996+0000","last_active":"2024-09-16T09:38:11.557921+0000","last_peered":"2024-09-16T09:38:11.557921+0000","last_clean":"2024-09-16T09:38:11.557921+0000","last_became_active":"2024-09-16T09:31:09.270434+0000","last_became_peered":"2024-09-16T09:31:09.270434+0000","last_unstale":"2024-09-16T09:38:11.557921+0000","last_undegraded":"2024-09-16T09:38:11.557921+0000","last_fullsized":"2024-09-16T09:38:11.557921+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'18","last_scrub_stamp":"2024-09-16T09:33:21.333937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:33:21.333937+0000","objects_scrubbed":8,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:38.469118+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":51,"num_read_kb":39,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":534,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.558627+0000","last_change":"2024-09-16T09:36:50.622852+0000","last_active":"2024-09-16T09:38:11.558627+0000","last_peered":"2024-09-16T09:38:11.558627+0000","last_clean":"2024-09-16T09:38:11.558627+0000","last_became_active":"2024-09-16T09:28:52.983543+0000","last_became_peered":"2024-09-16T09:28:52.983543+0000","last_unstale":"2024-09-16T09:38:11.558627+0000","last_undegraded":"2024-09-16T09:38:11.558627+0000","last_fullsized":"2024-09-16T09:38:11.558627+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:50.622807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:36:50.622807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:59.525944+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":714,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.547938+0000","last_change":"2024-09-16T09:35:19.822713+0000","last_active":"2024-09-16T09:38:11.547938+0000","last_peered":"2024-09-16T09:38:11.547938+0000","last_clean":"2024-09-16T09:38:11.547938+0000","last_became_active":"2024-09-16T09:31:09.268864+0000","last_became_peered":"2024-09-16T09:31:09.268864+0000","last_unstale":"2024-09-16T09:38:11.547938+0000","last_undegraded":"2024-09-16T09:38:11.547938+0000","last_fullsized":"2024-09-16T09:38:11.547938+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:35:19.822635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:35:19.822635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:11.341259+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":702,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.548652+0000","last_change":"2024-09-16T09:35:46.933987+0000","last_active":"2024-09-16T09:38:11.548652+0000","last_peered":"2024-09-16T09:38:11.548652+0000","last_clean":"2024-09-16T09:38:11.548652+0000","last_became_active":"2024-09-16T09:31:09.269433+0000","last_became_peered":"2024-09-16T09:31:09.269433+0000","last_unstale":"2024-09-16T09:38:11.548652+0000","last_undegraded":"2024-09-16T09:38:11.548652+0000","last_fullsized":"2024-09-16T09:38:11.548652+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:35:46.933912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:35:46.933912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"388'2446","reported_seq":2480,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:12.747439+0000","last_change":"2024-09-16T09:37:45.463119+0000","last_active":"2024-09-16T09:38:12.747439+0000","last_peered":"2024-09-16T09:38:12.747439+0000","last_clean":"2024-09-16T09:38:12.747439+0000","last_became_active":"2024-09-16T09:37:45.462743+0000","last_became_peered":"2024-09-16T09:37:45.462743+0000","last_unstale":"2024-09-16T09:38:12.747439+0000","last_undegraded":"2024-09-16T09:38:12.747439+0000","last_fullsized":"2024-09-16T09:38:12.747439+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_clean_scrub_stamp":"2024-09-16T09:37:44.378497+0000","objects_scrubbed":0,"log_size":2446,"log_dups_size":0,"ondisk_log_size":2446,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:54.300226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":160301056,"num_objects":2446,"num_object_clones":0,"num_object_copies":7338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2446,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2446,"num_write_kb":156544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.4","version":"73'79","reported_seq":433,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.552237+0000","last_change":"2024-09-16T09:37:17.664174+0000","last_active":"2024-09-16T09:38:11.552237+0000","last_peered":"2024-09-16T09:38:11.552237+0000","last_clean":"2024-09-16T09:38:11.552237+0000","last_became_active":"2024-09-16T09:31:09.275139+0000","last_became_peered":"2024-09-16T09:31:09.275139+0000","last_unstale":"2024-09-16T09:38:11.552237+0000","last_undegraded":"2024-09-16T09:38:11.552237+0000","last_fullsized":"2024-09-16T09:38:11.552237+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:37:17.664105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:37:17.664105+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:38.043254+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"285'28","reported_seq":746,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.548023+0000","last_change":"2024-09-16T09:35:43.914378+0000","last_active":"2024-09-16T09:38:11.548023+0000","last_peered":"2024-09-16T09:38:11.548023+0000","last_clean":"2024-09-16T09:38:11.548023+0000","last_became_active":"2024-09-16T09:25:03.708830+0000","last_became_peered":"2024-09-16T09:25:03.708830+0000","last_unstale":"2024-09-16T09:38:11.548023+0000","last_undegraded":"2024-09-16T09:38:11.548023+0000","last_fullsized":"2024-09-16T09:38:11.548023+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'28","last_scrub_stamp":"2024-09-16T09:35:43.914312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:35:43.914312+0000","objects_scrubbed":7,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":717,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.559446+0000","last_change":"2024-09-16T09:33:43.678231+0000","last_active":"2024-09-16T09:38:11.559446+0000","last_peered":"2024-09-16T09:38:11.559446+0000","last_clean":"2024-09-16T09:38:11.559446+0000","last_became_active":"2024-09-16T09:25:01.702992+0000","last_became_peered":"2024-09-16T09:25:01.702992+0000","last_unstale":"2024-09-16T09:38:11.559446+0000","last_undegraded":"2024-09-16T09:38:11.559446+0000","last_fullsized":"2024-09-16T09:38:11.559446+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:33:43.678138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:43.678138+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"235'12","reported_seq":414,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.549356+0000","last_change":"2024-09-16T09:37:19.380567+0000","last_active":"2024-09-16T09:38:11.549356+0000","last_peered":"2024-09-16T09:38:11.549356+0000","last_clean":"2024-09-16T09:38:11.549356+0000","last_became_active":"2024-09-16T09:33:30.251047+0000","last_became_peered":"2024-09-16T09:33:30.251047+0000","last_unstale":"2024-09-16T09:38:11.549356+0000","last_undegraded":"2024-09-16T09:38:11.549356+0000","last_fullsized":"2024-09-16T09:38:11.549356+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"235'12","last_scrub_stamp":"2024-09-16T09:37:19.380504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:37:19.380504+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:37.307298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":678,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.557887+0000","last_change":"2024-09-16T09:36:47.605655+0000","last_active":"2024-09-16T09:38:11.557887+0000","last_peered":"2024-09-16T09:38:11.557887+0000","last_clean":"2024-09-16T09:38:11.557887+0000","last_became_active":"2024-09-16T09:25:07.722017+0000","last_became_peered":"2024-09-16T09:25:07.722017+0000","last_unstale":"2024-09-16T09:38:11.557887+0000","last_undegraded":"2024-09-16T09:38:11.557887+0000","last_fullsized":"2024-09-16T09:38:11.557887+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:47.605615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:36:47.605615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.c","version":"388'2433","reported_seq":2467,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:12.922926+0000","last_change":"2024-09-16T09:37:45.401117+0000","last_active":"2024-09-16T09:38:12.922926+0000","last_peered":"2024-09-16T09:38:12.922926+0000","last_clean":"2024-09-16T09:38:12.922926+0000","last_became_active":"2024-09-16T09:37:45.400778+0000","last_became_peered":"2024-09-16T09:37:45.400778+0000","last_unstale":"2024-09-16T09:38:12.922926+0000","last_undegraded":"2024-09-16T09:38:12.922926+0000","last_fullsized":"2024-09-16T09:38:12.922926+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_clean_scrub_stamp":"2024-09-16T09:37:44.378497+0000","objects_scrubbed":0,"log_size":2433,"log_dups_size":0,"ondisk_log_size":2433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:49.375813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":159449088,"num_objects":2433,"num_object_clones":0,"num_object_copies":7299,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2433,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2433,"num_write_kb":155712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"73'79","reported_seq":756,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.552372+0000","last_change":"2024-09-16T09:34:51.864131+0000","last_active":"2024-09-16T09:38:11.552372+0000","last_peered":"2024-09-16T09:38:11.552372+0000","last_clean":"2024-09-16T09:38:11.552372+0000","last_became_active":"2024-09-16T09:31:09.274707+0000","last_became_peered":"2024-09-16T09:31:09.274707+0000","last_unstale":"2024-09-16T09:38:11.552372+0000","last_undegraded":"2024-09-16T09:38:11.552372+0000","last_fullsized":"2024-09-16T09:38:11.552372+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:34:51.864036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:34:51.864036+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"285'20","reported_seq":381,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.551080+0000","last_change":"2024-09-16T09:33:41.368102+0000","last_active":"2024-09-16T09:38:11.551080+0000","last_peered":"2024-09-16T09:38:11.551080+0000","last_clean":"2024-09-16T09:38:11.551080+0000","last_became_active":"2024-09-16T09:33:30.248453+0000","last_became_peered":"2024-09-16T09:33:30.248453+0000","last_unstale":"2024-09-16T09:38:11.551080+0000","last_undegraded":"2024-09-16T09:38:11.551080+0000","last_fullsized":"2024-09-16T09:38:11.551080+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-16T09:33:41.368016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:33:41.368016+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":713,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.549163+0000","last_change":"2024-09-16T09:36:47.095703+0000","last_active":"2024-09-16T09:38:11.549163+0000","last_peered":"2024-09-16T09:38:11.549163+0000","last_clean":"2024-09-16T09:38:11.549163+0000","last_became_active":"2024-09-16T09:31:09.270619+0000","last_became_peered":"2024-09-16T09:31:09.270619+0000","last_unstale":"2024-09-16T09:38:11.549163+0000","last_undegraded":"2024-09-16T09:38:11.549163+0000","last_fullsized":"2024-09-16T09:38:11.549163+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:47.095658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:36:47.095658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"66'8","reported_seq":859,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.559951+0000","last_change":"2024-09-16T09:36:52.689167+0000","last_active":"2024-09-16T09:38:11.559951+0000","last_peered":"2024-09-16T09:38:11.559951+0000","last_clean":"2024-09-16T09:38:11.559951+0000","last_became_active":"2024-09-16T09:25:05.714507+0000","last_became_peered":"2024-09-16T09:25:05.714507+0000","last_unstale":"2024-09-16T09:38:11.559951+0000","last_undegraded":"2024-09-16T09:38:11.559951+0000","last_fullsized":"2024-09-16T09:38:11.559951+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:36:52.689091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:36:52.689091+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":702,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.559206+0000","last_change":"2024-09-16T09:32:33.362233+0000","last_active":"2024-09-16T09:38:11.559206+0000","last_peered":"2024-09-16T09:38:11.559206+0000","last_clean":"2024-09-16T09:38:11.559206+0000","last_became_active":"2024-09-16T09:31:09.283192+0000","last_became_peered":"2024-09-16T09:31:09.283192+0000","last_unstale":"2024-09-16T09:38:11.559206+0000","last_undegraded":"2024-09-16T09:38:11.559206+0000","last_fullsized":"2024-09-16T09:38:11.559206+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:33.362188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:32:33.362188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.d","version":"388'2453","reported_seq":2488,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:12.748378+0000","last_change":"2024-09-16T09:37:45.405475+0000","last_active":"2024-09-16T09:38:12.748378+0000","last_peered":"2024-09-16T09:38:12.748378+0000","last_clean":"2024-09-16T09:38:12.748378+0000","last_became_active":"2024-09-16T09:37:45.405149+0000","last_became_peered":"2024-09-16T09:37:45.405149+0000","last_unstale":"2024-09-16T09:38:12.748378+0000","last_undegraded":"2024-09-16T09:38:12.748378+0000","last_fullsized":"2024-09-16T09:38:12.748378+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_clean_scrub_stamp":"2024-09-16T09:37:44.378497+0000","objects_scrubbed":0,"log_size":2453,"log_dups_size":0,"ondisk_log_size":2453,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:48.823494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":160759808,"num_objects":2453,"num_object_clones":0,"num_object_copies":7359,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2453,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2453,"num_write_kb":156992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"73'79","reported_seq":729,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.552414+0000","last_change":"2024-09-16T09:34:42.728232+0000","last_active":"2024-09-16T09:38:11.552414+0000","last_peered":"2024-09-16T09:38:11.552414+0000","last_clean":"2024-09-16T09:38:11.552414+0000","last_became_active":"2024-09-16T09:31:09.274565+0000","last_became_peered":"2024-09-16T09:31:09.274565+0000","last_unstale":"2024-09-16T09:38:11.552414+0000","last_undegraded":"2024-09-16T09:38:11.552414+0000","last_fullsized":"2024-09-16T09:38:11.552414+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:34:42.728143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:34:42.728143+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:35.968055+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"285'29","reported_seq":747,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.558861+0000","last_change":"2024-09-16T09:36:54.604573+0000","last_active":"2024-09-16T09:38:11.558861+0000","last_peered":"2024-09-16T09:38:11.558861+0000","last_clean":"2024-09-16T09:38:11.558861+0000","last_became_active":"2024-09-16T09:25:03.714797+0000","last_became_peered":"2024-09-16T09:25:03.714797+0000","last_unstale":"2024-09-16T09:38:11.558861+0000","last_undegraded":"2024-09-16T09:38:11.558861+0000","last_fullsized":"2024-09-16T09:38:11.558861+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'29","last_scrub_stamp":"2024-09-16T09:36:54.604484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:36:54.604484+0000","objects_scrubbed":8,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:21.529216+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":54,"num_read_kb":40,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":323,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.549163+0000","last_change":"2024-09-16T09:33:30.247311+0000","last_active":"2024-09-16T09:38:11.549163+0000","last_peered":"2024-09-16T09:38:11.549163+0000","last_clean":"2024-09-16T09:38:11.549163+0000","last_became_active":"2024-09-16T09:33:30.246996+0000","last_became_peered":"2024-09-16T09:33:30.246996+0000","last_unstale":"2024-09-16T09:38:11.549163+0000","last_undegraded":"2024-09-16T09:38:11.549163+0000","last_fullsized":"2024-09-16T09:38:11.549163+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:11.407616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:11.407616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,2],"acting":[5,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":708,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.553833+0000","last_change":"2024-09-16T09:34:45.772748+0000","last_active":"2024-09-16T09:38:11.553833+0000","last_peered":"2024-09-16T09:38:11.553833+0000","last_clean":"2024-09-16T09:38:11.553833+0000","last_became_active":"2024-09-16T09:25:05.708663+0000","last_became_peered":"2024-09-16T09:25:05.708663+0000","last_unstale":"2024-09-16T09:38:11.553833+0000","last_undegraded":"2024-09-16T09:38:11.553833+0000","last_fullsized":"2024-09-16T09:38:11.553833+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:45.772674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:34:45.772674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":339,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.551667+0000","last_change":"2024-09-16T09:37:56.882475+0000","last_active":"2024-09-16T09:38:11.551667+0000","last_peered":"2024-09-16T09:38:11.551667+0000","last_clean":"2024-09-16T09:38:11.551667+0000","last_became_active":"2024-09-16T09:33:30.250178+0000","last_became_peered":"2024-09-16T09:33:30.250178+0000","last_unstale":"2024-09-16T09:38:11.551667+0000","last_undegraded":"2024-09-16T09:38:11.551667+0000","last_fullsized":"2024-09-16T09:38:11.551667+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:56.882326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:37:56.882326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:02.807091+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,2],"acting":[6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.a","version":"388'2489","reported_seq":2524,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:12.728032+0000","last_change":"2024-09-16T09:37:45.404280+0000","last_active":"2024-09-16T09:38:12.728032+0000","last_peered":"2024-09-16T09:38:12.728032+0000","last_clean":"2024-09-16T09:38:12.728032+0000","last_became_active":"2024-09-16T09:37:45.404021+0000","last_became_peered":"2024-09-16T09:37:45.404021+0000","last_unstale":"2024-09-16T09:38:12.728032+0000","last_undegraded":"2024-09-16T09:38:12.728032+0000","last_fullsized":"2024-09-16T09:38:12.728032+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_clean_scrub_stamp":"2024-09-16T09:37:44.378497+0000","objects_scrubbed":0,"log_size":2489,"log_dups_size":0,"ondisk_log_size":2489,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:08.182634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":163119104,"num_objects":2489,"num_object_clones":0,"num_object_copies":7467,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2489,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2489,"num_write_kb":159296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"73'79","reported_seq":662,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.549441+0000","last_change":"2024-09-16T09:36:42.191578+0000","last_active":"2024-09-16T09:38:11.549441+0000","last_peered":"2024-09-16T09:38:11.549441+0000","last_clean":"2024-09-16T09:38:11.549441+0000","last_became_active":"2024-09-16T09:30:08.766350+0000","last_became_peered":"2024-09-16T09:30:08.766350+0000","last_unstale":"2024-09-16T09:38:11.549441+0000","last_undegraded":"2024-09-16T09:38:11.549441+0000","last_fullsized":"2024-09-16T09:38:11.549441+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:36:42.191521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:36:42.191521+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"285'42","reported_seq":1002,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.547335+0000","last_change":"2024-09-16T09:37:57.407386+0000","last_active":"2024-09-16T09:38:11.547335+0000","last_peered":"2024-09-16T09:38:11.547335+0000","last_clean":"2024-09-16T09:38:11.547335+0000","last_became_active":"2024-09-16T09:31:09.278839+0000","last_became_peered":"2024-09-16T09:31:09.278839+0000","last_unstale":"2024-09-16T09:38:11.547335+0000","last_undegraded":"2024-09-16T09:38:11.547335+0000","last_fullsized":"2024-09-16T09:38:11.547335+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'42","last_scrub_stamp":"2024-09-16T09:37:57.407297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:37:57.407297+0000","objects_scrubbed":10,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:23.815599+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":66,"num_read_kb":51,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":708,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.546657+0000","last_change":"2024-09-16T09:36:35.691463+0000","last_active":"2024-09-16T09:38:11.546657+0000","last_peered":"2024-09-16T09:38:11.546657+0000","last_clean":"2024-09-16T09:38:11.546657+0000","last_became_active":"2024-09-16T09:25:01.703076+0000","last_became_peered":"2024-09-16T09:25:01.703076+0000","last_unstale":"2024-09-16T09:38:11.546657+0000","last_undegraded":"2024-09-16T09:38:11.546657+0000","last_fullsized":"2024-09-16T09:38:11.546657+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:35.691415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:36:35.691415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":709,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.551267+0000","last_change":"2024-09-16T09:33:38.417375+0000","last_active":"2024-09-16T09:38:11.551267+0000","last_peered":"2024-09-16T09:38:11.551267+0000","last_clean":"2024-09-16T09:38:11.551267+0000","last_became_active":"2024-09-16T09:25:00.696237+0000","last_became_peered":"2024-09-16T09:25:00.696237+0000","last_unstale":"2024-09-16T09:38:11.551267+0000","last_undegraded":"2024-09-16T09:38:11.551267+0000","last_fullsized":"2024-09-16T09:38:11.551267+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-16T09:33:38.417308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:33:38.417308+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.0025137430000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":676,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.560964+0000","last_change":"2024-09-16T09:33:16.666704+0000","last_active":"2024-09-16T09:38:11.560964+0000","last_peered":"2024-09-16T09:38:11.560964+0000","last_clean":"2024-09-16T09:38:11.560964+0000","last_became_active":"2024-09-16T09:25:05.717591+0000","last_became_peered":"2024-09-16T09:25:05.717591+0000","last_unstale":"2024-09-16T09:38:11.560964+0000","last_undegraded":"2024-09-16T09:38:11.560964+0000","last_fullsized":"2024-09-16T09:38:11.560964+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:16.666648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:16.666648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:38.618334+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":425,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.552412+0000","last_change":"2024-09-16T09:36:50.667709+0000","last_active":"2024-09-16T09:38:11.552412+0000","last_peered":"2024-09-16T09:38:11.552412+0000","last_clean":"2024-09-16T09:38:11.552412+0000","last_became_active":"2024-09-16T09:31:09.272609+0000","last_became_peered":"2024-09-16T09:31:09.272609+0000","last_unstale":"2024-09-16T09:38:11.552412+0000","last_undegraded":"2024-09-16T09:38:11.552412+0000","last_fullsized":"2024-09-16T09:38:11.552412+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:50.667661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:36:50.667661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"388'2507","reported_seq":2542,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:13.220356+0000","last_change":"2024-09-16T09:37:45.463232+0000","last_active":"2024-09-16T09:38:13.220356+0000","last_peered":"2024-09-16T09:38:13.220356+0000","last_clean":"2024-09-16T09:38:13.220356+0000","last_became_active":"2024-09-16T09:37:45.462868+0000","last_became_peered":"2024-09-16T09:37:45.462868+0000","last_unstale":"2024-09-16T09:38:13.220356+0000","last_undegraded":"2024-09-16T09:38:13.220356+0000","last_fullsized":"2024-09-16T09:38:13.220356+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_clean_scrub_stamp":"2024-09-16T09:37:44.378497+0000","objects_scrubbed":0,"log_size":2507,"log_dups_size":0,"ondisk_log_size":2507,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:55.563655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":164298752,"num_objects":2507,"num_object_clones":0,"num_object_copies":7521,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2507,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2507,"num_write_kb":160448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"73'79","reported_seq":14,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.557829+0000","last_change":"2024-09-16T09:38:10.556681+0000","last_active":"2024-09-16T09:38:11.557829+0000","last_peered":"2024-09-16T09:38:11.557829+0000","last_clean":"2024-09-16T09:38:11.557829+0000","last_became_active":"2024-09-16T09:38:10.556320+0000","last_became_peered":"2024-09-16T09:38:10.556320+0000","last_unstale":"2024-09-16T09:38:11.557829+0000","last_undegraded":"2024-09-16T09:38:11.557829+0000","last_fullsized":"2024-09-16T09:38:11.557829+0000","mapping_epoch":386,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":387,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:34:32.717915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:34:32.717915+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"285'17","reported_seq":422,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.551147+0000","last_change":"2024-09-16T09:37:49.246464+0000","last_active":"2024-09-16T09:38:11.551147+0000","last_peered":"2024-09-16T09:38:11.551147+0000","last_clean":"2024-09-16T09:38:11.551147+0000","last_became_active":"2024-09-16T09:37:33.349009+0000","last_became_peered":"2024-09-16T09:37:33.349009+0000","last_unstale":"2024-09-16T09:38:11.551147+0000","last_undegraded":"2024-09-16T09:38:11.551147+0000","last_fullsized":"2024-09-16T09:38:11.551147+0000","mapping_epoch":367,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":368,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'17","last_scrub_stamp":"2024-09-16T09:37:49.246379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:37:49.246379+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:56.883815+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":35,"num_read_kb":24,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":331,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.551492+0000","last_change":"2024-09-16T09:37:21.946685+0000","last_active":"2024-09-16T09:38:11.551492+0000","last_peered":"2024-09-16T09:38:11.551492+0000","last_clean":"2024-09-16T09:38:11.551492+0000","last_became_active":"2024-09-16T09:33:30.247596+0000","last_became_peered":"2024-09-16T09:33:30.247596+0000","last_unstale":"2024-09-16T09:38:11.551492+0000","last_undegraded":"2024-09-16T09:38:11.551492+0000","last_fullsized":"2024-09-16T09:38:11.551492+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:21.946641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:37:21.946641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:25.048262+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,2],"acting":[6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":358,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.559271+0000","last_change":"2024-09-16T09:37:04.362941+0000","last_active":"2024-09-16T09:38:11.559271+0000","last_peered":"2024-09-16T09:38:11.559271+0000","last_clean":"2024-09-16T09:38:11.559271+0000","last_became_active":"2024-09-16T09:37:04.358985+0000","last_became_peered":"2024-09-16T09:37:04.358985+0000","last_unstale":"2024-09-16T09:38:11.559271+0000","last_undegraded":"2024-09-16T09:38:11.559271+0000","last_fullsized":"2024-09-16T09:38:11.559271+0000","mapping_epoch":343,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":344,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:32:40.165749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:32:40.165749+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:12.613598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":716,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.553573+0000","last_change":"2024-09-16T09:36:55.530225+0000","last_active":"2024-09-16T09:38:11.553573+0000","last_peered":"2024-09-16T09:38:11.553573+0000","last_clean":"2024-09-16T09:38:11.553573+0000","last_became_active":"2024-09-16T09:25:05.711539+0000","last_became_peered":"2024-09-16T09:25:05.711539+0000","last_unstale":"2024-09-16T09:38:11.553573+0000","last_undegraded":"2024-09-16T09:38:11.553573+0000","last_fullsized":"2024-09-16T09:38:11.553573+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:55.530175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:36:55.530175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:18.377901+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":694,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.558380+0000","last_change":"2024-09-16T09:37:45.450074+0000","last_active":"2024-09-16T09:38:11.558380+0000","last_peered":"2024-09-16T09:38:11.558380+0000","last_clean":"2024-09-16T09:38:11.558380+0000","last_became_active":"2024-09-16T09:25:07.723282+0000","last_became_peered":"2024-09-16T09:25:07.723282+0000","last_unstale":"2024-09-16T09:38:11.558380+0000","last_undegraded":"2024-09-16T09:38:11.558380+0000","last_fullsized":"2024-09-16T09:38:11.558380+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:45.450022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:37:45.450022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:15.116366+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"388'2648","reported_seq":2683,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.196279+0000","last_change":"2024-09-16T09:37:45.464193+0000","last_active":"2024-09-16T09:38:14.196279+0000","last_peered":"2024-09-16T09:38:14.196279+0000","last_clean":"2024-09-16T09:38:14.196279+0000","last_became_active":"2024-09-16T09:37:45.462754+0000","last_became_peered":"2024-09-16T09:37:45.462754+0000","last_unstale":"2024-09-16T09:38:14.196279+0000","last_undegraded":"2024-09-16T09:38:14.196279+0000","last_fullsized":"2024-09-16T09:38:14.196279+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_clean_scrub_stamp":"2024-09-16T09:37:44.378497+0000","objects_scrubbed":0,"log_size":2648,"log_dups_size":0,"ondisk_log_size":2648,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:46.604408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":173539328,"num_objects":2648,"num_object_clones":0,"num_object_copies":7944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2648,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2648,"num_write_kb":169472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,7],"acting":[0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"73'79","reported_seq":412,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.552055+0000","last_change":"2024-09-16T09:34:31.929462+0000","last_active":"2024-09-16T09:38:11.552055+0000","last_peered":"2024-09-16T09:38:11.552055+0000","last_clean":"2024-09-16T09:38:11.552055+0000","last_became_active":"2024-09-16T09:33:30.249309+0000","last_became_peered":"2024-09-16T09:33:30.249309+0000","last_unstale":"2024-09-16T09:38:11.552055+0000","last_undegraded":"2024-09-16T09:38:11.552055+0000","last_fullsized":"2024-09-16T09:38:11.552055+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:34:31.929388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:34:31.929388+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:34.931244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.6","version":"285'24","reported_seq":724,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.559221+0000","last_change":"2024-09-16T09:32:48.547237+0000","last_active":"2024-09-16T09:38:11.559221+0000","last_peered":"2024-09-16T09:38:11.559221+0000","last_clean":"2024-09-16T09:38:11.559221+0000","last_became_active":"2024-09-16T09:25:03.715315+0000","last_became_peered":"2024-09-16T09:25:03.715315+0000","last_unstale":"2024-09-16T09:38:11.559221+0000","last_undegraded":"2024-09-16T09:38:11.559221+0000","last_fullsized":"2024-09-16T09:38:11.559221+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-16T09:32:48.546869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:32:48.546869+0000","objects_scrubbed":6,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:32.692965+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":36,"num_read_kb":30,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"59'1","reported_seq":723,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.558785+0000","last_change":"2024-09-16T09:33:21.682893+0000","last_active":"2024-09-16T09:38:11.558785+0000","last_peered":"2024-09-16T09:38:11.558785+0000","last_clean":"2024-09-16T09:38:11.558785+0000","last_became_active":"2024-09-16T09:31:09.269627+0000","last_became_peered":"2024-09-16T09:31:09.269627+0000","last_unstale":"2024-09-16T09:38:11.558785+0000","last_undegraded":"2024-09-16T09:38:11.558785+0000","last_fullsized":"2024-09-16T09:38:11.558785+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:33:21.682840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:21.682840+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:42.553527+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1986,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"67'5","reported_seq":1494,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.879287+0000","last_change":"2024-09-16T09:33:37.358839+0000","last_active":"2024-09-16T09:38:11.879287+0000","last_peered":"2024-09-16T09:38:11.879287+0000","last_clean":"2024-09-16T09:38:11.879287+0000","last_became_active":"2024-09-16T09:25:00.696546+0000","last_became_peered":"2024-09-16T09:25:00.696546+0000","last_unstale":"2024-09-16T09:38:11.879287+0000","last_undegraded":"2024-09-16T09:38:11.879287+0000","last_fullsized":"2024-09-16T09:38:11.879287+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'5","last_scrub_stamp":"2024-09-16T09:33:37.358741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:33:37.358741+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:33.496609+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0.0059770420000000001,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":785,"num_read_kb":780,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":675,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.551239+0000","last_change":"2024-09-16T09:27:07.337170+0000","last_active":"2024-09-16T09:38:11.551239+0000","last_peered":"2024-09-16T09:38:11.551239+0000","last_clean":"2024-09-16T09:38:11.551239+0000","last_became_active":"2024-09-16T09:25:05.714704+0000","last_became_peered":"2024-09-16T09:25:05.714704+0000","last_unstale":"2024-09-16T09:38:11.551239+0000","last_undegraded":"2024-09-16T09:38:11.551239+0000","last_fullsized":"2024-09-16T09:38:11.551239+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:07.337124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:07.337124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:38.140085+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":601,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.549326+0000","last_change":"2024-09-16T09:36:43.150813+0000","last_active":"2024-09-16T09:38:11.549326+0000","last_peered":"2024-09-16T09:38:11.549326+0000","last_clean":"2024-09-16T09:38:11.549326+0000","last_became_active":"2024-09-16T09:33:30.245852+0000","last_became_peered":"2024-09-16T09:33:30.245852+0000","last_unstale":"2024-09-16T09:38:11.549326+0000","last_undegraded":"2024-09-16T09:38:11.549326+0000","last_fullsized":"2024-09-16T09:38:11.549326+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:43.150764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:36:43.150764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"388'2463","reported_seq":2498,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:12.738129+0000","last_change":"2024-09-16T09:37:45.406191+0000","last_active":"2024-09-16T09:38:12.738129+0000","last_peered":"2024-09-16T09:38:12.738129+0000","last_clean":"2024-09-16T09:38:12.738129+0000","last_became_active":"2024-09-16T09:37:45.405552+0000","last_became_peered":"2024-09-16T09:37:45.405552+0000","last_unstale":"2024-09-16T09:38:12.738129+0000","last_undegraded":"2024-09-16T09:38:12.738129+0000","last_fullsized":"2024-09-16T09:38:12.738129+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_clean_scrub_stamp":"2024-09-16T09:37:44.378497+0000","objects_scrubbed":0,"log_size":2463,"log_dups_size":0,"ondisk_log_size":2463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:55.714372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":161415168,"num_objects":2463,"num_object_clones":0,"num_object_copies":7389,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2463,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2463,"num_write_kb":157632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"285'28","reported_seq":903,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.552651+0000","last_change":"2024-09-16T09:33:16.891850+0000","last_active":"2024-09-16T09:38:11.552651+0000","last_peered":"2024-09-16T09:38:11.552651+0000","last_clean":"2024-09-16T09:38:11.552651+0000","last_became_active":"2024-09-16T09:25:03.714118+0000","last_became_peered":"2024-09-16T09:25:03.714118+0000","last_unstale":"2024-09-16T09:38:11.552651+0000","last_undegraded":"2024-09-16T09:38:11.552651+0000","last_fullsized":"2024-09-16T09:38:11.552651+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'16","last_scrub_stamp":"2024-09-16T09:33:16.891761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:33:16.891761+0000","objects_scrubbed":8,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:11.950479+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":43,"num_read_kb":33,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":549,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.551175+0000","last_change":"2024-09-16T09:36:50.114910+0000","last_active":"2024-09-16T09:38:11.551175+0000","last_peered":"2024-09-16T09:38:11.551175+0000","last_clean":"2024-09-16T09:38:11.551175+0000","last_became_active":"2024-09-16T09:31:09.270196+0000","last_became_peered":"2024-09-16T09:31:09.270196+0000","last_unstale":"2024-09-16T09:38:11.551175+0000","last_undegraded":"2024-09-16T09:38:11.551175+0000","last_fullsized":"2024-09-16T09:38:11.551175+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:50.114861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:36:50.114861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:13.001323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"73'79","reported_seq":778,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.558288+0000","last_change":"2024-09-16T09:32:43.535041+0000","last_active":"2024-09-16T09:38:11.558288+0000","last_peered":"2024-09-16T09:38:11.558288+0000","last_clean":"2024-09-16T09:38:11.558288+0000","last_became_active":"2024-09-16T09:30:08.870394+0000","last_became_peered":"2024-09-16T09:30:08.870394+0000","last_unstale":"2024-09-16T09:38:11.558288+0000","last_undegraded":"2024-09-16T09:38:11.558288+0000","last_fullsized":"2024-09-16T09:38:11.558288+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:32:43.534962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:32:43.534962+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":690,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.558235+0000","last_change":"2024-09-16T09:37:52.517698+0000","last_active":"2024-09-16T09:38:11.558235+0000","last_peered":"2024-09-16T09:38:11.558235+0000","last_clean":"2024-09-16T09:38:11.558235+0000","last_became_active":"2024-09-16T09:25:05.714341+0000","last_became_peered":"2024-09-16T09:25:05.714341+0000","last_unstale":"2024-09-16T09:38:11.558235+0000","last_undegraded":"2024-09-16T09:38:11.558235+0000","last_fullsized":"2024-09-16T09:38:11.558235+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:52.517652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:37:52.517652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:20.774965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":687,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.548702+0000","last_change":"2024-09-16T09:37:50.218655+0000","last_active":"2024-09-16T09:38:11.548702+0000","last_peered":"2024-09-16T09:38:11.548702+0000","last_clean":"2024-09-16T09:38:11.548702+0000","last_became_active":"2024-09-16T09:25:07.714281+0000","last_became_peered":"2024-09-16T09:25:07.714281+0000","last_unstale":"2024-09-16T09:38:11.548702+0000","last_undegraded":"2024-09-16T09:38:11.548702+0000","last_fullsized":"2024-09-16T09:38:11.548702+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:50.218594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:37:50.218594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:20.077058+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.0","version":"388'2369","reported_seq":2404,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:12.749199+0000","last_change":"2024-09-16T09:37:45.407072+0000","last_active":"2024-09-16T09:38:12.749199+0000","last_peered":"2024-09-16T09:38:12.749199+0000","last_clean":"2024-09-16T09:38:12.749199+0000","last_became_active":"2024-09-16T09:37:45.406736+0000","last_became_peered":"2024-09-16T09:37:45.406736+0000","last_unstale":"2024-09-16T09:38:12.749199+0000","last_undegraded":"2024-09-16T09:38:12.749199+0000","last_fullsized":"2024-09-16T09:38:12.749199+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_clean_scrub_stamp":"2024-09-16T09:37:44.378497+0000","objects_scrubbed":0,"log_size":2369,"log_dups_size":0,"ondisk_log_size":2369,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:03.181256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":155254784,"num_objects":2369,"num_object_clones":0,"num_object_copies":7107,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2369,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2369,"num_write_kb":151616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.b","version":"73'79","reported_seq":515,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.558457+0000","last_change":"2024-09-16T09:34:31.720971+0000","last_active":"2024-09-16T09:38:11.558457+0000","last_peered":"2024-09-16T09:38:11.558457+0000","last_clean":"2024-09-16T09:38:11.558457+0000","last_became_active":"2024-09-16T09:33:30.251933+0000","last_became_peered":"2024-09-16T09:33:30.251933+0000","last_unstale":"2024-09-16T09:38:11.558457+0000","last_undegraded":"2024-09-16T09:38:11.558457+0000","last_fullsized":"2024-09-16T09:38:11.558457+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:34:31.720787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:34:31.720787+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"285'19","reported_seq":775,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.548585+0000","last_change":"2024-09-16T09:37:33.355562+0000","last_active":"2024-09-16T09:38:11.548585+0000","last_peered":"2024-09-16T09:38:11.548585+0000","last_clean":"2024-09-16T09:38:11.548585+0000","last_became_active":"2024-09-16T09:37:33.348433+0000","last_became_peered":"2024-09-16T09:37:33.348433+0000","last_unstale":"2024-09-16T09:38:11.548585+0000","last_undegraded":"2024-09-16T09:38:11.548585+0000","last_fullsized":"2024-09-16T09:38:11.548585+0000","mapping_epoch":367,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":368,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'19","last_scrub_stamp":"2024-09-16T09:35:27.813206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:35:27.813206+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":692,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.548337+0000","last_change":"2024-09-16T09:37:18.392256+0000","last_active":"2024-09-16T09:38:11.548337+0000","last_peered":"2024-09-16T09:38:11.548337+0000","last_clean":"2024-09-16T09:38:11.548337+0000","last_became_active":"2024-09-16T09:25:01.708270+0000","last_became_peered":"2024-09-16T09:25:01.708270+0000","last_unstale":"2024-09-16T09:38:11.548337+0000","last_undegraded":"2024-09-16T09:38:11.548337+0000","last_fullsized":"2024-09-16T09:38:11.548337+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:18.392084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:37:18.392084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:33.733932+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":674,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.548562+0000","last_change":"2024-09-16T09:26:34.171190+0000","last_active":"2024-09-16T09:38:11.548562+0000","last_peered":"2024-09-16T09:38:11.548562+0000","last_clean":"2024-09-16T09:38:11.548562+0000","last_became_active":"2024-09-16T09:25:05.711964+0000","last_became_peered":"2024-09-16T09:25:05.711964+0000","last_unstale":"2024-09-16T09:38:11.548562+0000","last_undegraded":"2024-09-16T09:38:11.548562+0000","last_fullsized":"2024-09-16T09:38:11.548562+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:34.171139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:26:34.171139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:15.084115+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":566,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.552302+0000","last_change":"2024-09-16T09:34:50.866121+0000","last_active":"2024-09-16T09:38:11.552302+0000","last_peered":"2024-09-16T09:38:11.552302+0000","last_clean":"2024-09-16T09:38:11.552302+0000","last_became_active":"2024-09-16T09:31:09.278618+0000","last_became_peered":"2024-09-16T09:31:09.278618+0000","last_unstale":"2024-09-16T09:38:11.552302+0000","last_undegraded":"2024-09-16T09:38:11.552302+0000","last_fullsized":"2024-09-16T09:38:11.552302+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:50.866032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:34:50.866032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:09.987998+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3","version":"388'2574","reported_seq":2609,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:13.237492+0000","last_change":"2024-09-16T09:37:45.463216+0000","last_active":"2024-09-16T09:38:13.237492+0000","last_peered":"2024-09-16T09:38:13.237492+0000","last_clean":"2024-09-16T09:38:13.237492+0000","last_became_active":"2024-09-16T09:37:45.462708+0000","last_became_peered":"2024-09-16T09:37:45.462708+0000","last_unstale":"2024-09-16T09:38:13.237492+0000","last_undegraded":"2024-09-16T09:38:13.237492+0000","last_fullsized":"2024-09-16T09:38:13.237492+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_clean_scrub_stamp":"2024-09-16T09:37:44.378497+0000","objects_scrubbed":0,"log_size":2574,"log_dups_size":0,"ondisk_log_size":2574,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:05.295932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":168689664,"num_objects":2574,"num_object_clones":0,"num_object_copies":7722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2574,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2574,"num_write_kb":164736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.8","version":"73'79","reported_seq":34,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.548519+0000","last_change":"2024-09-16T09:37:54.183161+0000","last_active":"2024-09-16T09:38:11.548519+0000","last_peered":"2024-09-16T09:38:11.548519+0000","last_clean":"2024-09-16T09:38:11.548519+0000","last_became_active":"2024-09-16T09:37:53.418691+0000","last_became_peered":"2024-09-16T09:37:53.418691+0000","last_unstale":"2024-09-16T09:38:11.548519+0000","last_undegraded":"2024-09-16T09:38:11.548519+0000","last_fullsized":"2024-09-16T09:38:11.548519+0000","mapping_epoch":382,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":383,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:37:54.183107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:37:54.183107+0000","objects_scrubbed":1,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:09.320645+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262688,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"285'31","reported_seq":752,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.548163+0000","last_change":"2024-09-16T09:35:28.861520+0000","last_active":"2024-09-16T09:38:11.548163+0000","last_peered":"2024-09-16T09:38:11.548163+0000","last_clean":"2024-09-16T09:38:11.548163+0000","last_became_active":"2024-09-16T09:25:03.713358+0000","last_became_peered":"2024-09-16T09:25:03.713358+0000","last_unstale":"2024-09-16T09:38:11.548163+0000","last_undegraded":"2024-09-16T09:38:11.548163+0000","last_fullsized":"2024-09-16T09:38:11.548163+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'31","last_scrub_stamp":"2024-09-16T09:35:28.861433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:35:28.861433+0000","objects_scrubbed":8,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:08.335954+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":50,"num_read_kb":40,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":700,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.553976+0000","last_change":"2024-09-16T09:37:52.902619+0000","last_active":"2024-09-16T09:38:11.553976+0000","last_peered":"2024-09-16T09:38:11.553976+0000","last_clean":"2024-09-16T09:38:11.553976+0000","last_became_active":"2024-09-16T09:25:01.703355+0000","last_became_peered":"2024-09-16T09:25:01.703355+0000","last_unstale":"2024-09-16T09:38:11.553976+0000","last_undegraded":"2024-09-16T09:38:11.553976+0000","last_fullsized":"2024-09-16T09:38:11.553976+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:52.902556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:37:52.902556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:04.411673+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":339,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.546588+0000","last_change":"2024-09-16T09:37:46.378016+0000","last_active":"2024-09-16T09:38:11.546588+0000","last_peered":"2024-09-16T09:38:11.546588+0000","last_clean":"2024-09-16T09:38:11.546588+0000","last_became_active":"2024-09-16T09:33:30.251186+0000","last_became_peered":"2024-09-16T09:33:30.251186+0000","last_unstale":"2024-09-16T09:38:11.546588+0000","last_undegraded":"2024-09-16T09:38:11.546588+0000","last_fullsized":"2024-09-16T09:38:11.546588+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:46.377951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:37:46.377951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:54.699212+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,0],"acting":[4,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":629,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.550685+0000","last_change":"2024-09-16T09:36:51.640537+0000","last_active":"2024-09-16T09:38:11.550685+0000","last_peered":"2024-09-16T09:38:11.550685+0000","last_clean":"2024-09-16T09:38:11.550685+0000","last_became_active":"2024-09-16T09:33:21.337291+0000","last_became_peered":"2024-09-16T09:33:21.337291+0000","last_unstale":"2024-09-16T09:38:11.550685+0000","last_undegraded":"2024-09-16T09:38:11.550685+0000","last_fullsized":"2024-09-16T09:38:11.550685+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:51.640472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:36:51.640472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4],"acting":[3,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"388'2505","reported_seq":2540,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:12.950841+0000","last_change":"2024-09-16T09:37:45.402477+0000","last_active":"2024-09-16T09:38:12.950841+0000","last_peered":"2024-09-16T09:38:12.950841+0000","last_clean":"2024-09-16T09:38:12.950841+0000","last_became_active":"2024-09-16T09:37:45.401847+0000","last_became_peered":"2024-09-16T09:37:45.401847+0000","last_unstale":"2024-09-16T09:38:12.950841+0000","last_undegraded":"2024-09-16T09:38:12.950841+0000","last_fullsized":"2024-09-16T09:38:12.950841+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_clean_scrub_stamp":"2024-09-16T09:37:44.378497+0000","objects_scrubbed":0,"log_size":2505,"log_dups_size":0,"ondisk_log_size":2505,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:05.829127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":164167680,"num_objects":2505,"num_object_clones":0,"num_object_copies":7515,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2505,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2505,"num_write_kb":160320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,0],"acting":[4,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.9","version":"73'79","reported_seq":754,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.558643+0000","last_change":"2024-09-16T09:29:19.013948+0000","last_active":"2024-09-16T09:38:11.558643+0000","last_peered":"2024-09-16T09:38:11.558643+0000","last_clean":"2024-09-16T09:38:11.558643+0000","last_became_active":"2024-09-16T09:26:35.126066+0000","last_became_peered":"2024-09-16T09:26:35.126066+0000","last_unstale":"2024-09-16T09:38:11.558643+0000","last_undegraded":"2024-09-16T09:38:11.558643+0000","last_fullsized":"2024-09-16T09:38:11.558643+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:19.013865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:19.013865+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:10.452624+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"285'20","reported_seq":709,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.546820+0000","last_change":"2024-09-16T09:33:24.154567+0000","last_active":"2024-09-16T09:38:11.546820+0000","last_peered":"2024-09-16T09:38:11.546820+0000","last_clean":"2024-09-16T09:38:11.546820+0000","last_became_active":"2024-09-16T09:25:03.707723+0000","last_became_peered":"2024-09-16T09:25:03.707723+0000","last_unstale":"2024-09-16T09:38:11.546820+0000","last_undegraded":"2024-09-16T09:38:11.546820+0000","last_fullsized":"2024-09-16T09:38:11.546820+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-16T09:33:24.154507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:33:24.154507+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:13.467647+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":557,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.559086+0000","last_change":"2024-09-16T09:33:10.528181+0000","last_active":"2024-09-16T09:38:11.559086+0000","last_peered":"2024-09-16T09:38:11.559086+0000","last_clean":"2024-09-16T09:38:11.559086+0000","last_became_active":"2024-09-16T09:31:09.288825+0000","last_became_peered":"2024-09-16T09:31:09.288825+0000","last_unstale":"2024-09-16T09:38:11.559086+0000","last_undegraded":"2024-09-16T09:38:11.559086+0000","last_fullsized":"2024-09-16T09:38:11.559086+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:10.528115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:10.528115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:34.688091+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"228'9","reported_seq":430,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.558523+0000","last_change":"2024-09-16T09:37:16.515947+0000","last_active":"2024-09-16T09:38:11.558523+0000","last_peered":"2024-09-16T09:38:11.558523+0000","last_clean":"2024-09-16T09:38:11.558523+0000","last_became_active":"2024-09-16T09:33:21.337213+0000","last_became_peered":"2024-09-16T09:33:21.337213+0000","last_unstale":"2024-09-16T09:38:11.558523+0000","last_undegraded":"2024-09-16T09:38:11.558523+0000","last_fullsized":"2024-09-16T09:38:11.558523+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"228'9","last_scrub_stamp":"2024-09-16T09:37:16.514010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:37:16.514010+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:17.192669+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":703,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.547982+0000","last_change":"2024-09-16T09:35:36.061013+0000","last_active":"2024-09-16T09:38:11.547982+0000","last_peered":"2024-09-16T09:38:11.547982+0000","last_clean":"2024-09-16T09:38:11.547982+0000","last_became_active":"2024-09-16T09:25:07.718958+0000","last_became_peered":"2024-09-16T09:25:07.718958+0000","last_unstale":"2024-09-16T09:38:11.547982+0000","last_undegraded":"2024-09-16T09:38:11.547982+0000","last_fullsized":"2024-09-16T09:38:11.547982+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:35:36.060942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:35:36.060942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.5","version":"388'2635","reported_seq":2670,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.200408+0000","last_change":"2024-09-16T09:37:45.463538+0000","last_active":"2024-09-16T09:38:14.200408+0000","last_peered":"2024-09-16T09:38:14.200408+0000","last_clean":"2024-09-16T09:38:14.200408+0000","last_became_active":"2024-09-16T09:37:45.462778+0000","last_became_peered":"2024-09-16T09:37:45.462778+0000","last_unstale":"2024-09-16T09:38:14.200408+0000","last_undegraded":"2024-09-16T09:38:14.200408+0000","last_fullsized":"2024-09-16T09:38:14.200408+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_clean_scrub_stamp":"2024-09-16T09:37:44.378497+0000","objects_scrubbed":0,"log_size":2635,"log_dups_size":0,"ondisk_log_size":2635,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:09.642344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":172687360,"num_objects":2635,"num_object_clones":0,"num_object_copies":7905,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2635,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2635,"num_write_kb":168640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.e","version":"73'79","reported_seq":496,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.547882+0000","last_change":"2024-09-16T09:35:41.979877+0000","last_active":"2024-09-16T09:38:11.547882+0000","last_peered":"2024-09-16T09:38:11.547882+0000","last_clean":"2024-09-16T09:38:11.547882+0000","last_became_active":"2024-09-16T09:30:10.705072+0000","last_became_peered":"2024-09-16T09:30:10.705072+0000","last_unstale":"2024-09-16T09:38:11.547882+0000","last_undegraded":"2024-09-16T09:38:11.547882+0000","last_fullsized":"2024-09-16T09:38:11.547882+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:35:41.979814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:35:41.979814+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,3],"acting":[4,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.b","version":"285'15","reported_seq":745,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.558765+0000","last_change":"2024-09-16T09:36:55.651561+0000","last_active":"2024-09-16T09:38:11.558765+0000","last_peered":"2024-09-16T09:38:11.558765+0000","last_clean":"2024-09-16T09:38:11.558765+0000","last_became_active":"2024-09-16T09:36:31.590202+0000","last_became_peered":"2024-09-16T09:36:31.590202+0000","last_unstale":"2024-09-16T09:38:11.558765+0000","last_undegraded":"2024-09-16T09:38:11.558765+0000","last_fullsized":"2024-09-16T09:38:11.558765+0000","mapping_epoch":316,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":317,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'15","last_scrub_stamp":"2024-09-16T09:36:55.651496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:36:55.651496+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":704,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.549403+0000","last_change":"2024-09-16T09:37:31.033631+0000","last_active":"2024-09-16T09:38:11.549403+0000","last_peered":"2024-09-16T09:38:11.549403+0000","last_clean":"2024-09-16T09:38:11.549403+0000","last_became_active":"2024-09-16T09:31:09.276516+0000","last_became_peered":"2024-09-16T09:31:09.276516+0000","last_unstale":"2024-09-16T09:38:11.549403+0000","last_undegraded":"2024-09-16T09:38:11.549403+0000","last_fullsized":"2024-09-16T09:38:11.549403+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:31.033577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:37:31.033577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:49.512309+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":625,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.548521+0000","last_change":"2024-09-16T09:33:40.260547+0000","last_active":"2024-09-16T09:38:11.548521+0000","last_peered":"2024-09-16T09:38:11.548521+0000","last_clean":"2024-09-16T09:38:11.548521+0000","last_became_active":"2024-09-16T09:33:21.337254+0000","last_became_peered":"2024-09-16T09:33:21.337254+0000","last_unstale":"2024-09-16T09:38:11.548521+0000","last_undegraded":"2024-09-16T09:38:11.548521+0000","last_fullsized":"2024-09-16T09:38:11.548521+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:40.260500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:40.260500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:39.339396+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3],"acting":[4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":696,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.558704+0000","last_change":"2024-09-16T09:36:53.647305+0000","last_active":"2024-09-16T09:38:11.558704+0000","last_peered":"2024-09-16T09:38:11.558704+0000","last_clean":"2024-09-16T09:38:11.558704+0000","last_became_active":"2024-09-16T09:25:07.718778+0000","last_became_peered":"2024-09-16T09:25:07.718778+0000","last_unstale":"2024-09-16T09:38:11.558704+0000","last_undegraded":"2024-09-16T09:38:11.558704+0000","last_fullsized":"2024-09-16T09:38:11.558704+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:53.647256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:36:53.647256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:19.369534+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"388'2668","reported_seq":2703,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.191036+0000","last_change":"2024-09-16T09:37:45.400129+0000","last_active":"2024-09-16T09:38:14.191036+0000","last_peered":"2024-09-16T09:38:14.191036+0000","last_clean":"2024-09-16T09:38:14.191036+0000","last_became_active":"2024-09-16T09:37:45.399844+0000","last_became_peered":"2024-09-16T09:37:45.399844+0000","last_unstale":"2024-09-16T09:38:14.191036+0000","last_undegraded":"2024-09-16T09:38:14.191036+0000","last_fullsized":"2024-09-16T09:38:14.191036+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_clean_scrub_stamp":"2024-09-16T09:37:44.378497+0000","objects_scrubbed":0,"log_size":2668,"log_dups_size":0,"ondisk_log_size":2668,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:51.633388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":174850048,"num_objects":2668,"num_object_clones":0,"num_object_copies":8004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2668,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2668,"num_write_kb":170752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.f","version":"73'79","reported_seq":748,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.552850+0000","last_change":"2024-09-16T09:34:53.866426+0000","last_active":"2024-09-16T09:38:11.552850+0000","last_peered":"2024-09-16T09:38:11.552850+0000","last_clean":"2024-09-16T09:38:11.552850+0000","last_became_active":"2024-09-16T09:30:10.693477+0000","last_became_peered":"2024-09-16T09:30:10.693477+0000","last_unstale":"2024-09-16T09:38:11.552850+0000","last_undegraded":"2024-09-16T09:38:11.552850+0000","last_fullsized":"2024-09-16T09:38:11.552850+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:34:53.866337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:34:53.866337+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:59.878482+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"285'35","reported_seq":759,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.553823+0000","last_change":"2024-09-16T09:37:28.078195+0000","last_active":"2024-09-16T09:38:11.553823+0000","last_peered":"2024-09-16T09:38:11.553823+0000","last_clean":"2024-09-16T09:38:11.553823+0000","last_became_active":"2024-09-16T09:25:03.712783+0000","last_became_peered":"2024-09-16T09:25:03.712783+0000","last_unstale":"2024-09-16T09:38:11.553823+0000","last_undegraded":"2024-09-16T09:38:11.553823+0000","last_fullsized":"2024-09-16T09:38:11.553823+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'35","last_scrub_stamp":"2024-09-16T09:37:28.077841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:37:28.077841+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:52.547070+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":56,"num_read_kb":45,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"59'1","reported_seq":711,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.558789+0000","last_change":"2024-09-16T09:37:56.528872+0000","last_active":"2024-09-16T09:38:11.558789+0000","last_peered":"2024-09-16T09:38:11.558789+0000","last_clean":"2024-09-16T09:38:11.558789+0000","last_became_active":"2024-09-16T09:25:01.708069+0000","last_became_peered":"2024-09-16T09:25:01.708069+0000","last_unstale":"2024-09-16T09:38:11.558789+0000","last_undegraded":"2024-09-16T09:38:11.558789+0000","last_fullsized":"2024-09-16T09:38:11.558789+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:37:56.528796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:37:56.528796+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:57.114353+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":713,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.554526+0000","last_change":"2024-09-16T09:37:53.518635+0000","last_active":"2024-09-16T09:38:11.554526+0000","last_peered":"2024-09-16T09:38:11.554526+0000","last_clean":"2024-09-16T09:38:11.554526+0000","last_became_active":"2024-09-16T09:37:53.518284+0000","last_became_peered":"2024-09-16T09:37:53.518284+0000","last_unstale":"2024-09-16T09:38:11.554526+0000","last_undegraded":"2024-09-16T09:38:11.554526+0000","last_fullsized":"2024-09-16T09:38:11.554526+0000","mapping_epoch":382,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":383,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:59.543358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:36:59.543358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:24.211506+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":678,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.558237+0000","last_change":"2024-09-16T09:36:48.649739+0000","last_active":"2024-09-16T09:38:11.558237+0000","last_peered":"2024-09-16T09:38:11.558237+0000","last_clean":"2024-09-16T09:38:11.558237+0000","last_became_active":"2024-09-16T09:25:07.718750+0000","last_became_peered":"2024-09-16T09:25:07.718750+0000","last_unstale":"2024-09-16T09:38:11.558237+0000","last_undegraded":"2024-09-16T09:38:11.558237+0000","last_fullsized":"2024-09-16T09:38:11.558237+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:48.649696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:36:48.649696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:18.071944+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.7","version":"388'2448","reported_seq":2483,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:12.732312+0000","last_change":"2024-09-16T09:37:45.404688+0000","last_active":"2024-09-16T09:38:12.732312+0000","last_peered":"2024-09-16T09:38:12.732312+0000","last_clean":"2024-09-16T09:38:12.732312+0000","last_became_active":"2024-09-16T09:37:45.404354+0000","last_became_peered":"2024-09-16T09:37:45.404354+0000","last_unstale":"2024-09-16T09:38:12.732312+0000","last_undegraded":"2024-09-16T09:38:12.732312+0000","last_fullsized":"2024-09-16T09:38:12.732312+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_clean_scrub_stamp":"2024-09-16T09:37:44.378497+0000","objects_scrubbed":0,"log_size":2448,"log_dups_size":0,"ondisk_log_size":2448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:13.592270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":160432128,"num_objects":2448,"num_object_clones":0,"num_object_copies":7344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2448,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2448,"num_write_kb":156672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.c","version":"73'79","reported_seq":437,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.552142+0000","last_change":"2024-09-16T09:36:53.697797+0000","last_active":"2024-09-16T09:38:11.552142+0000","last_peered":"2024-09-16T09:38:11.552142+0000","last_clean":"2024-09-16T09:38:11.552142+0000","last_became_active":"2024-09-16T09:31:09.273770+0000","last_became_peered":"2024-09-16T09:31:09.273770+0000","last_unstale":"2024-09-16T09:38:11.552142+0000","last_undegraded":"2024-09-16T09:38:11.552142+0000","last_fullsized":"2024-09-16T09:38:11.552142+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:36:53.697742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:36:53.697742+0000","objects_scrubbed":1,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:00.926871+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":327680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"285'18","reported_seq":774,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.548184+0000","last_change":"2024-09-16T09:36:50.603724+0000","last_active":"2024-09-16T09:38:11.548184+0000","last_peered":"2024-09-16T09:38:11.548184+0000","last_clean":"2024-09-16T09:38:11.548184+0000","last_became_active":"2024-09-16T09:31:09.269047+0000","last_became_peered":"2024-09-16T09:31:09.269047+0000","last_unstale":"2024-09-16T09:38:11.548184+0000","last_undegraded":"2024-09-16T09:38:11.548184+0000","last_fullsized":"2024-09-16T09:38:11.548184+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'18","last_scrub_stamp":"2024-09-16T09:36:50.603660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:36:50.603660+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:15.970176+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":705,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.558984+0000","last_change":"2024-09-16T09:37:57.529616+0000","last_active":"2024-09-16T09:38:11.558984+0000","last_peered":"2024-09-16T09:38:11.558984+0000","last_clean":"2024-09-16T09:38:11.558984+0000","last_became_active":"2024-09-16T09:25:01.707528+0000","last_became_peered":"2024-09-16T09:25:01.707528+0000","last_unstale":"2024-09-16T09:38:11.558984+0000","last_undegraded":"2024-09-16T09:38:11.558984+0000","last_fullsized":"2024-09-16T09:38:11.558984+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:57.529554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:37:57.529554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:05.467113+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":351,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.559794+0000","last_change":"2024-09-16T09:37:57.449473+0000","last_active":"2024-09-16T09:38:11.559794+0000","last_peered":"2024-09-16T09:38:11.559794+0000","last_clean":"2024-09-16T09:38:11.559794+0000","last_became_active":"2024-09-16T09:33:21.337309+0000","last_became_peered":"2024-09-16T09:33:21.337309+0000","last_unstale":"2024-09-16T09:38:11.559794+0000","last_undegraded":"2024-09-16T09:38:11.559794+0000","last_fullsized":"2024-09-16T09:38:11.559794+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:57.449105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:37:57.449105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:12.289276+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":687,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.555644+0000","last_change":"2024-09-16T09:36:59.713546+0000","last_active":"2024-09-16T09:38:11.555644+0000","last_peered":"2024-09-16T09:38:11.555644+0000","last_clean":"2024-09-16T09:38:11.555644+0000","last_became_active":"2024-09-16T09:25:07.718899+0000","last_became_peered":"2024-09-16T09:25:07.718899+0000","last_unstale":"2024-09-16T09:38:11.555644+0000","last_undegraded":"2024-09-16T09:38:11.555644+0000","last_fullsized":"2024-09-16T09:38:11.555644+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:59.713355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:36:59.713355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.6","version":"388'2450","reported_seq":2485,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:13.220339+0000","last_change":"2024-09-16T09:37:45.404587+0000","last_active":"2024-09-16T09:38:13.220339+0000","last_peered":"2024-09-16T09:38:13.220339+0000","last_clean":"2024-09-16T09:38:13.220339+0000","last_became_active":"2024-09-16T09:37:45.404263+0000","last_became_peered":"2024-09-16T09:37:45.404263+0000","last_unstale":"2024-09-16T09:38:13.220339+0000","last_undegraded":"2024-09-16T09:38:13.220339+0000","last_fullsized":"2024-09-16T09:38:13.220339+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_clean_scrub_stamp":"2024-09-16T09:37:44.378497+0000","objects_scrubbed":0,"log_size":2450,"log_dups_size":0,"ondisk_log_size":2450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:58.611017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":160563200,"num_objects":2450,"num_object_clones":0,"num_object_copies":7350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2450,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2450,"num_write_kb":156800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.d","version":"73'79","reported_seq":487,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.558244+0000","last_change":"2024-09-16T09:32:44.536868+0000","last_active":"2024-09-16T09:38:11.558244+0000","last_peered":"2024-09-16T09:38:11.558244+0000","last_clean":"2024-09-16T09:38:11.558244+0000","last_became_active":"2024-09-16T09:30:10.709958+0000","last_became_peered":"2024-09-16T09:30:10.709958+0000","last_unstale":"2024-09-16T09:38:11.558244+0000","last_undegraded":"2024-09-16T09:38:11.558244+0000","last_fullsized":"2024-09-16T09:38:11.558244+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:32:44.536799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:32:44.536799+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:14.489819+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"285'27","reported_seq":730,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.549000+0000","last_change":"2024-09-16T09:37:30.081518+0000","last_active":"2024-09-16T09:38:11.549000+0000","last_peered":"2024-09-16T09:38:11.549000+0000","last_clean":"2024-09-16T09:38:11.549000+0000","last_became_active":"2024-09-16T09:25:03.715733+0000","last_became_peered":"2024-09-16T09:25:03.715733+0000","last_unstale":"2024-09-16T09:38:11.549000+0000","last_undegraded":"2024-09-16T09:38:11.549000+0000","last_fullsized":"2024-09-16T09:38:11.549000+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'27","last_scrub_stamp":"2024-09-16T09:37:30.081447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:37:30.081447+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:45.797304+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":709,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.558711+0000","last_change":"2024-09-16T09:37:17.544132+0000","last_active":"2024-09-16T09:38:11.558711+0000","last_peered":"2024-09-16T09:38:11.558711+0000","last_clean":"2024-09-16T09:38:11.558711+0000","last_became_active":"2024-09-16T09:25:01.707372+0000","last_became_peered":"2024-09-16T09:25:01.707372+0000","last_unstale":"2024-09-16T09:38:11.558711+0000","last_undegraded":"2024-09-16T09:38:11.558711+0000","last_fullsized":"2024-09-16T09:38:11.558711+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-16T09:37:17.544069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:37:17.544069+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:23.672102+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"66'8","reported_seq":857,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.558590+0000","last_change":"2024-09-16T09:30:05.138362+0000","last_active":"2024-09-16T09:38:11.558590+0000","last_peered":"2024-09-16T09:38:11.558590+0000","last_clean":"2024-09-16T09:38:11.558590+0000","last_became_active":"2024-09-16T09:25:05.716189+0000","last_became_peered":"2024-09-16T09:25:05.716189+0000","last_unstale":"2024-09-16T09:38:11.558590+0000","last_undegraded":"2024-09-16T09:38:11.558590+0000","last_fullsized":"2024-09-16T09:38:11.558590+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:30:05.138288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:30:05.138288+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":679,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.550786+0000","last_change":"2024-09-16T09:37:47.244099+0000","last_active":"2024-09-16T09:38:11.550786+0000","last_peered":"2024-09-16T09:38:11.550786+0000","last_clean":"2024-09-16T09:38:11.550786+0000","last_became_active":"2024-09-16T09:25:07.717213+0000","last_became_peered":"2024-09-16T09:25:07.717213+0000","last_unstale":"2024-09-16T09:38:11.550786+0000","last_undegraded":"2024-09-16T09:38:11.550786+0000","last_fullsized":"2024-09-16T09:38:11.550786+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:47.244032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:37:47.244032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:48.590206+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.12","version":"73'79","reported_seq":496,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.551061+0000","last_change":"2024-09-16T09:36:44.171217+0000","last_active":"2024-09-16T09:38:11.551061+0000","last_peered":"2024-09-16T09:38:11.551061+0000","last_clean":"2024-09-16T09:38:11.551061+0000","last_became_active":"2024-09-16T09:30:10.731413+0000","last_became_peered":"2024-09-16T09:30:10.731413+0000","last_unstale":"2024-09-16T09:38:11.551061+0000","last_undegraded":"2024-09-16T09:38:11.551061+0000","last_fullsized":"2024-09-16T09:38:11.551061+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:36:44.171151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:36:44.171151+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":713,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.551310+0000","last_change":"2024-09-16T09:37:55.863661+0000","last_active":"2024-09-16T09:38:11.551310+0000","last_peered":"2024-09-16T09:38:11.551310+0000","last_clean":"2024-09-16T09:38:11.551310+0000","last_became_active":"2024-09-16T09:37:53.518354+0000","last_became_peered":"2024-09-16T09:37:53.518354+0000","last_unstale":"2024-09-16T09:38:11.551310+0000","last_undegraded":"2024-09-16T09:38:11.551310+0000","last_fullsized":"2024-09-16T09:38:11.551310+0000","mapping_epoch":382,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":383,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:55.863613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:37:55.863613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:01.033155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"285'12","reported_seq":457,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.554348+0000","last_change":"2024-09-16T09:37:20.749151+0000","last_active":"2024-09-16T09:38:11.554348+0000","last_peered":"2024-09-16T09:38:11.554348+0000","last_clean":"2024-09-16T09:38:11.554348+0000","last_became_active":"2024-09-16T09:31:09.270685+0000","last_became_peered":"2024-09-16T09:31:09.270685+0000","last_unstale":"2024-09-16T09:38:11.554348+0000","last_undegraded":"2024-09-16T09:38:11.554348+0000","last_fullsized":"2024-09-16T09:38:11.554348+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'12","last_scrub_stamp":"2024-09-16T09:37:20.749101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:37:20.749101+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:32.592645+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":17,"num_read_kb":14,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":690,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.550810+0000","last_change":"2024-09-16T09:36:38.129449+0000","last_active":"2024-09-16T09:38:11.550810+0000","last_peered":"2024-09-16T09:38:11.550810+0000","last_clean":"2024-09-16T09:38:11.550810+0000","last_became_active":"2024-09-16T09:25:05.715616+0000","last_became_peered":"2024-09-16T09:25:05.715616+0000","last_unstale":"2024-09-16T09:38:11.550810+0000","last_undegraded":"2024-09-16T09:38:11.550810+0000","last_fullsized":"2024-09-16T09:38:11.550810+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:38.129314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:36:38.129314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":677,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.558659+0000","last_change":"2024-09-16T09:36:37.715166+0000","last_active":"2024-09-16T09:38:11.558659+0000","last_peered":"2024-09-16T09:38:11.558659+0000","last_clean":"2024-09-16T09:38:11.558659+0000","last_became_active":"2024-09-16T09:25:07.721059+0000","last_became_peered":"2024-09-16T09:25:07.721059+0000","last_unstale":"2024-09-16T09:38:11.558659+0000","last_undegraded":"2024-09-16T09:38:11.558659+0000","last_fullsized":"2024-09-16T09:38:11.558659+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:37.715114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:36:37.715114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.13","version":"73'79","reported_seq":582,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.555898+0000","last_change":"2024-09-16T09:34:48.260121+0000","last_active":"2024-09-16T09:38:11.555898+0000","last_peered":"2024-09-16T09:38:11.555898+0000","last_clean":"2024-09-16T09:38:11.555898+0000","last_became_active":"2024-09-16T09:33:30.248497+0000","last_became_peered":"2024-09-16T09:33:30.248497+0000","last_unstale":"2024-09-16T09:38:11.555898+0000","last_undegraded":"2024-09-16T09:38:11.555898+0000","last_fullsized":"2024-09-16T09:38:11.555898+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:34:48.260041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:34:48.260041+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:11.022422+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"285'15","reported_seq":770,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.559344+0000","last_change":"2024-09-16T09:33:48.681742+0000","last_active":"2024-09-16T09:38:11.559344+0000","last_peered":"2024-09-16T09:38:11.559344+0000","last_clean":"2024-09-16T09:38:11.559344+0000","last_became_active":"2024-09-16T09:31:09.273498+0000","last_became_peered":"2024-09-16T09:31:09.273498+0000","last_unstale":"2024-09-16T09:38:11.559344+0000","last_undegraded":"2024-09-16T09:38:11.559344+0000","last_fullsized":"2024-09-16T09:38:11.559344+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:33:48.681650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:33:48.681650+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:59.423578+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":697,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.558825+0000","last_change":"2024-09-16T09:33:13.312434+0000","last_active":"2024-09-16T09:38:11.558825+0000","last_peered":"2024-09-16T09:38:11.558825+0000","last_clean":"2024-09-16T09:38:11.558825+0000","last_became_active":"2024-09-16T09:25:01.706491+0000","last_became_peered":"2024-09-16T09:25:01.706491+0000","last_unstale":"2024-09-16T09:38:11.558825+0000","last_undegraded":"2024-09-16T09:38:11.558825+0000","last_fullsized":"2024-09-16T09:38:11.558825+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:13.312344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:13.312344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:37.450175+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":454,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.558418+0000","last_change":"2024-09-16T09:36:54.596000+0000","last_active":"2024-09-16T09:38:11.558418+0000","last_peered":"2024-09-16T09:38:11.558418+0000","last_clean":"2024-09-16T09:38:11.558418+0000","last_became_active":"2024-09-16T09:33:30.248992+0000","last_became_peered":"2024-09-16T09:33:30.248992+0000","last_unstale":"2024-09-16T09:38:11.558418+0000","last_undegraded":"2024-09-16T09:38:11.558418+0000","last_fullsized":"2024-09-16T09:38:11.558418+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:54.595924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:36:54.595924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":695,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.552610+0000","last_change":"2024-09-16T09:36:58.536167+0000","last_active":"2024-09-16T09:38:11.552610+0000","last_peered":"2024-09-16T09:38:11.552610+0000","last_clean":"2024-09-16T09:38:11.552610+0000","last_became_active":"2024-09-16T09:25:07.722050+0000","last_became_peered":"2024-09-16T09:25:07.722050+0000","last_unstale":"2024-09-16T09:38:11.552610+0000","last_undegraded":"2024-09-16T09:38:11.552610+0000","last_fullsized":"2024-09-16T09:38:11.552610+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:58.536121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:36:58.536121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:18.611763+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.10","version":"73'79","reported_seq":496,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.548416+0000","last_change":"2024-09-16T09:36:41.169117+0000","last_active":"2024-09-16T09:38:11.548416+0000","last_peered":"2024-09-16T09:38:11.548416+0000","last_clean":"2024-09-16T09:38:11.548416+0000","last_became_active":"2024-09-16T09:30:10.692770+0000","last_became_peered":"2024-09-16T09:30:10.692770+0000","last_unstale":"2024-09-16T09:38:11.548416+0000","last_undegraded":"2024-09-16T09:38:11.548416+0000","last_fullsized":"2024-09-16T09:38:11.548416+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:36:41.169069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:36:41.169069+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"285'15","reported_seq":719,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.549528+0000","last_change":"2024-09-16T09:37:55.203919+0000","last_active":"2024-09-16T09:38:11.549528+0000","last_peered":"2024-09-16T09:38:11.549528+0000","last_clean":"2024-09-16T09:38:11.549528+0000","last_became_active":"2024-09-16T09:25:03.717310+0000","last_became_peered":"2024-09-16T09:25:03.717310+0000","last_unstale":"2024-09-16T09:38:11.549528+0000","last_undegraded":"2024-09-16T09:38:11.549528+0000","last_fullsized":"2024-09-16T09:38:11.549528+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'15","last_scrub_stamp":"2024-09-16T09:37:55.203856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:37:55.203856+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:23.098469+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":729,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.565883+0000","last_change":"2024-09-16T09:37:50.471532+0000","last_active":"2024-09-16T09:38:11.565883+0000","last_peered":"2024-09-16T09:38:11.565883+0000","last_clean":"2024-09-16T09:38:11.565883+0000","last_became_active":"2024-09-16T09:31:09.272848+0000","last_became_peered":"2024-09-16T09:31:09.272848+0000","last_unstale":"2024-09-16T09:38:11.565883+0000","last_undegraded":"2024-09-16T09:38:11.565883+0000","last_fullsized":"2024-09-16T09:38:11.565883+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:50.471486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:37:50.471486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:09.546185+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"187'11","reported_seq":702,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.913170+0000","last_change":"2024-09-16T09:37:19.540982+0000","last_active":"2024-09-16T09:38:11.913170+0000","last_peered":"2024-09-16T09:38:11.913170+0000","last_clean":"2024-09-16T09:38:11.913170+0000","last_became_active":"2024-09-16T09:31:09.277688+0000","last_became_peered":"2024-09-16T09:31:09.277688+0000","last_unstale":"2024-09-16T09:38:11.913170+0000","last_undegraded":"2024-09-16T09:38:11.913170+0000","last_fullsized":"2024-09-16T09:38:11.913170+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"187'11","last_scrub_stamp":"2024-09-16T09:37:19.540886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:37:19.540886+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:47.772515+0000","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":703,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.546897+0000","last_change":"2024-09-16T09:36:49.628523+0000","last_active":"2024-09-16T09:38:11.546897+0000","last_peered":"2024-09-16T09:38:11.546897+0000","last_clean":"2024-09-16T09:38:11.546897+0000","last_became_active":"2024-09-16T09:25:07.718810+0000","last_became_peered":"2024-09-16T09:25:07.718810+0000","last_unstale":"2024-09-16T09:38:11.546897+0000","last_undegraded":"2024-09-16T09:38:11.546897+0000","last_fullsized":"2024-09-16T09:38:11.546897+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:49.628477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:36:49.628477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.11","version":"73'79","reported_seq":582,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.558874+0000","last_change":"2024-09-16T09:37:15.623104+0000","last_active":"2024-09-16T09:38:11.558874+0000","last_peered":"2024-09-16T09:38:11.558874+0000","last_clean":"2024-09-16T09:38:11.558874+0000","last_became_active":"2024-09-16T09:31:09.272208+0000","last_became_peered":"2024-09-16T09:31:09.272208+0000","last_unstale":"2024-09-16T09:38:11.558874+0000","last_undegraded":"2024-09-16T09:38:11.558874+0000","last_fullsized":"2024-09-16T09:38:11.558874+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:37:15.623041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:37:15.623041+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:16.219169+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.14","version":"285'20","reported_seq":380,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.558353+0000","last_change":"2024-09-16T09:36:52.609515+0000","last_active":"2024-09-16T09:38:11.558353+0000","last_peered":"2024-09-16T09:38:11.558353+0000","last_clean":"2024-09-16T09:38:11.558353+0000","last_became_active":"2024-09-16T09:33:30.249769+0000","last_became_peered":"2024-09-16T09:33:30.249769+0000","last_unstale":"2024-09-16T09:38:11.558353+0000","last_undegraded":"2024-09-16T09:38:11.558353+0000","last_fullsized":"2024-09-16T09:38:11.558353+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'20","last_scrub_stamp":"2024-09-16T09:36:52.609440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:36:52.609440+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":689,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.558022+0000","last_change":"2024-09-16T09:29:56.021413+0000","last_active":"2024-09-16T09:38:11.558022+0000","last_peered":"2024-09-16T09:38:11.558022+0000","last_clean":"2024-09-16T09:38:11.558022+0000","last_became_active":"2024-09-16T09:25:01.707513+0000","last_became_peered":"2024-09-16T09:25:01.707513+0000","last_unstale":"2024-09-16T09:38:11.558022+0000","last_undegraded":"2024-09-16T09:38:11.558022+0000","last_fullsized":"2024-09-16T09:38:11.558022+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:56.021340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:56.021340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"66'8","reported_seq":865,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.549024+0000","last_change":"2024-09-16T09:33:39.388721+0000","last_active":"2024-09-16T09:38:11.549024+0000","last_peered":"2024-09-16T09:38:11.549024+0000","last_clean":"2024-09-16T09:38:11.549024+0000","last_became_active":"2024-09-16T09:25:05.713833+0000","last_became_peered":"2024-09-16T09:25:05.713833+0000","last_unstale":"2024-09-16T09:38:11.549024+0000","last_undegraded":"2024-09-16T09:38:11.549024+0000","last_fullsized":"2024-09-16T09:38:11.549024+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:33:39.388463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:39.388463+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":709,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.558985+0000","last_change":"2024-09-16T09:33:42.666290+0000","last_active":"2024-09-16T09:38:11.558985+0000","last_peered":"2024-09-16T09:38:11.558985+0000","last_clean":"2024-09-16T09:38:11.558985+0000","last_became_active":"2024-09-16T09:31:09.275168+0000","last_became_peered":"2024-09-16T09:31:09.275168+0000","last_unstale":"2024-09-16T09:38:11.558985+0000","last_undegraded":"2024-09-16T09:38:11.558985+0000","last_fullsized":"2024-09-16T09:38:11.558985+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:42.666200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:33:42.666200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"285'19","reported_seq":737,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.546442+0000","last_change":"2024-09-16T09:37:31.362985+0000","last_active":"2024-09-16T09:38:11.546442+0000","last_peered":"2024-09-16T09:38:11.546442+0000","last_clean":"2024-09-16T09:38:11.546442+0000","last_became_active":"2024-09-16T09:26:33.291265+0000","last_became_peered":"2024-09-16T09:26:33.291265+0000","last_unstale":"2024-09-16T09:38:11.546442+0000","last_undegraded":"2024-09-16T09:38:11.546442+0000","last_fullsized":"2024-09-16T09:38:11.546442+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'19","last_scrub_stamp":"2024-09-16T09:37:31.362927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:37:31.362927+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:37.215450+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":715,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.547069+0000","last_change":"2024-09-16T09:37:53.239593+0000","last_active":"2024-09-16T09:38:11.547069+0000","last_peered":"2024-09-16T09:38:11.547069+0000","last_clean":"2024-09-16T09:38:11.547069+0000","last_became_active":"2024-09-16T09:25:01.705158+0000","last_became_peered":"2024-09-16T09:25:01.705158+0000","last_unstale":"2024-09-16T09:38:11.547069+0000","last_undegraded":"2024-09-16T09:38:11.547069+0000","last_fullsized":"2024-09-16T09:38:11.547069+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:53.239547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:37:53.239547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:02.480253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":635,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.550873+0000","last_change":"2024-09-16T09:37:58.252056+0000","last_active":"2024-09-16T09:38:11.550873+0000","last_peered":"2024-09-16T09:38:11.550873+0000","last_clean":"2024-09-16T09:38:11.550873+0000","last_became_active":"2024-09-16T09:37:41.515278+0000","last_became_peered":"2024-09-16T09:37:41.515278+0000","last_unstale":"2024-09-16T09:38:11.550873+0000","last_undegraded":"2024-09-16T09:38:11.550873+0000","last_fullsized":"2024-09-16T09:38:11.550873+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:58.251994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:37:58.251994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:15.408415+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":542,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.558647+0000","last_change":"2024-09-16T09:34:48.786847+0000","last_active":"2024-09-16T09:38:11.558647+0000","last_peered":"2024-09-16T09:38:11.558647+0000","last_clean":"2024-09-16T09:38:11.558647+0000","last_became_active":"2024-09-16T09:31:09.276042+0000","last_became_peered":"2024-09-16T09:31:09.276042+0000","last_unstale":"2024-09-16T09:38:11.558647+0000","last_undegraded":"2024-09-16T09:38:11.558647+0000","last_fullsized":"2024-09-16T09:38:11.558647+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:48.786785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:34:48.786785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"285'15","reported_seq":410,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.553982+0000","last_change":"2024-09-16T09:37:33.354025+0000","last_active":"2024-09-16T09:38:11.553982+0000","last_peered":"2024-09-16T09:38:11.553982+0000","last_clean":"2024-09-16T09:38:11.553982+0000","last_became_active":"2024-09-16T09:37:33.348223+0000","last_became_peered":"2024-09-16T09:37:33.348223+0000","last_unstale":"2024-09-16T09:38:11.553982+0000","last_undegraded":"2024-09-16T09:38:11.553982+0000","last_fullsized":"2024-09-16T09:38:11.553982+0000","mapping_epoch":367,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":368,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'15","last_scrub_stamp":"2024-09-16T09:36:55.935996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:36:55.935996+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,2],"acting":[6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":689,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.558701+0000","last_change":"2024-09-16T09:32:47.502667+0000","last_active":"2024-09-16T09:38:11.558701+0000","last_peered":"2024-09-16T09:38:11.558701+0000","last_clean":"2024-09-16T09:38:11.558701+0000","last_became_active":"2024-09-16T09:25:01.706762+0000","last_became_peered":"2024-09-16T09:25:01.706762+0000","last_unstale":"2024-09-16T09:38:11.558701+0000","last_undegraded":"2024-09-16T09:38:11.558701+0000","last_fullsized":"2024-09-16T09:38:11.558701+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:47.502598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:32:47.502598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":527,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.558592+0000","last_change":"2024-09-16T09:34:46.742640+0000","last_active":"2024-09-16T09:38:11.558592+0000","last_peered":"2024-09-16T09:38:11.558592+0000","last_clean":"2024-09-16T09:38:11.558592+0000","last_became_active":"2024-09-16T09:28:52.982865+0000","last_became_peered":"2024-09-16T09:28:52.982865+0000","last_unstale":"2024-09-16T09:38:11.558592+0000","last_undegraded":"2024-09-16T09:38:11.558592+0000","last_fullsized":"2024-09-16T09:38:11.558592+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:46.742562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:34:46.742562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":687,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.558794+0000","last_change":"2024-09-16T09:34:50.779241+0000","last_active":"2024-09-16T09:38:11.558794+0000","last_peered":"2024-09-16T09:38:11.558794+0000","last_clean":"2024-09-16T09:38:11.558794+0000","last_became_active":"2024-09-16T09:25:07.715301+0000","last_became_peered":"2024-09-16T09:25:07.715301+0000","last_unstale":"2024-09-16T09:38:11.558794+0000","last_undegraded":"2024-09-16T09:38:11.558794+0000","last_fullsized":"2024-09-16T09:38:11.558794+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:50.779164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:34:50.779164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.14","version":"73'79","reported_seq":496,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.552498+0000","last_change":"2024-09-16T09:36:48.597220+0000","last_active":"2024-09-16T09:38:11.552498+0000","last_peered":"2024-09-16T09:38:11.552498+0000","last_clean":"2024-09-16T09:38:11.552498+0000","last_became_active":"2024-09-16T09:30:10.700866+0000","last_became_peered":"2024-09-16T09:30:10.700866+0000","last_unstale":"2024-09-16T09:38:11.552498+0000","last_undegraded":"2024-09-16T09:38:11.552498+0000","last_fullsized":"2024-09-16T09:38:11.552498+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:36:48.597169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:36:48.597169+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:15.020958+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"285'19","reported_seq":577,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.558758+0000","last_change":"2024-09-16T09:28:55.643505+0000","last_active":"2024-09-16T09:38:11.558758+0000","last_peered":"2024-09-16T09:38:11.558758+0000","last_clean":"2024-09-16T09:38:11.558758+0000","last_became_active":"2024-09-16T09:28:52.971132+0000","last_became_peered":"2024-09-16T09:28:52.971132+0000","last_unstale":"2024-09-16T09:38:11.558758+0000","last_undegraded":"2024-09-16T09:38:11.558758+0000","last_fullsized":"2024-09-16T09:38:11.558758+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-16T09:28:55.643448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:55.643448+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:07.375359+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":698,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.550953+0000","last_change":"2024-09-16T09:33:18.297533+0000","last_active":"2024-09-16T09:38:11.550953+0000","last_peered":"2024-09-16T09:38:11.550953+0000","last_clean":"2024-09-16T09:38:11.550953+0000","last_became_active":"2024-09-16T09:25:01.700733+0000","last_became_peered":"2024-09-16T09:25:01.700733+0000","last_unstale":"2024-09-16T09:38:11.550953+0000","last_undegraded":"2024-09-16T09:38:11.550953+0000","last_fullsized":"2024-09-16T09:38:11.550953+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:18.297451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:18.297451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":689,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.557642+0000","last_change":"2024-09-16T09:33:38.336281+0000","last_active":"2024-09-16T09:38:11.557642+0000","last_peered":"2024-09-16T09:38:11.557642+0000","last_clean":"2024-09-16T09:38:11.557642+0000","last_became_active":"2024-09-16T09:25:05.713479+0000","last_became_peered":"2024-09-16T09:25:05.713479+0000","last_unstale":"2024-09-16T09:38:11.557642+0000","last_undegraded":"2024-09-16T09:38:11.557642+0000","last_fullsized":"2024-09-16T09:38:11.557642+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:38.336221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:38.336221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:36.450450+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":550,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.559006+0000","last_change":"2024-09-16T09:34:51.781047+0000","last_active":"2024-09-16T09:38:11.559006+0000","last_peered":"2024-09-16T09:38:11.559006+0000","last_clean":"2024-09-16T09:38:11.559006+0000","last_became_active":"2024-09-16T09:31:09.274797+0000","last_became_peered":"2024-09-16T09:31:09.274797+0000","last_unstale":"2024-09-16T09:38:11.559006+0000","last_undegraded":"2024-09-16T09:38:11.559006+0000","last_fullsized":"2024-09-16T09:38:11.559006+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:51.780980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:34:51.780980+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"285'8","reported_seq":453,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.550871+0000","last_change":"2024-09-16T09:37:15.672673+0000","last_active":"2024-09-16T09:38:11.550871+0000","last_peered":"2024-09-16T09:38:11.550871+0000","last_clean":"2024-09-16T09:38:11.550871+0000","last_became_active":"2024-09-16T09:31:09.273682+0000","last_became_peered":"2024-09-16T09:31:09.273682+0000","last_unstale":"2024-09-16T09:38:11.550871+0000","last_undegraded":"2024-09-16T09:38:11.550871+0000","last_fullsized":"2024-09-16T09:38:11.550871+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'8","last_scrub_stamp":"2024-09-16T09:37:15.672596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:37:15.672596+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:20.990359+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":9,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":721,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.558815+0000","last_change":"2024-09-16T09:33:18.641115+0000","last_active":"2024-09-16T09:38:11.558815+0000","last_peered":"2024-09-16T09:38:11.558815+0000","last_clean":"2024-09-16T09:38:11.558815+0000","last_became_active":"2024-09-16T09:31:09.274719+0000","last_became_peered":"2024-09-16T09:31:09.274719+0000","last_unstale":"2024-09-16T09:38:11.558815+0000","last_undegraded":"2024-09-16T09:38:11.558815+0000","last_fullsized":"2024-09-16T09:38:11.558815+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:18.641069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:18.641069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:44.551604+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":707,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.551685+0000","last_change":"2024-09-16T09:33:37.941485+0000","last_active":"2024-09-16T09:38:11.551685+0000","last_peered":"2024-09-16T09:38:11.551685+0000","last_clean":"2024-09-16T09:38:11.551685+0000","last_became_active":"2024-09-16T09:25:05.712545+0000","last_became_peered":"2024-09-16T09:25:05.712545+0000","last_unstale":"2024-09-16T09:38:11.551685+0000","last_undegraded":"2024-09-16T09:38:11.551685+0000","last_fullsized":"2024-09-16T09:38:11.551685+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:37.941403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:37.941403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:33.011670+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":693,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.557704+0000","last_change":"2024-09-16T09:37:50.502460+0000","last_active":"2024-09-16T09:38:11.557704+0000","last_peered":"2024-09-16T09:38:11.557704+0000","last_clean":"2024-09-16T09:38:11.557704+0000","last_became_active":"2024-09-16T09:25:07.720117+0000","last_became_peered":"2024-09-16T09:25:07.720117+0000","last_unstale":"2024-09-16T09:38:11.557704+0000","last_undegraded":"2024-09-16T09:38:11.557704+0000","last_fullsized":"2024-09-16T09:38:11.557704+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:50.502398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:37:50.502398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:13.329570+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":330,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.549070+0000","last_change":"2024-09-16T09:36:52.091335+0000","last_active":"2024-09-16T09:38:11.549070+0000","last_peered":"2024-09-16T09:38:11.549070+0000","last_clean":"2024-09-16T09:38:11.549070+0000","last_became_active":"2024-09-16T09:33:30.248036+0000","last_became_peered":"2024-09-16T09:33:30.248036+0000","last_unstale":"2024-09-16T09:38:11.549070+0000","last_undegraded":"2024-09-16T09:38:11.549070+0000","last_fullsized":"2024-09-16T09:38:11.549070+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:52.088284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:36:52.088284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":556,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.559573+0000","last_change":"2024-09-16T09:37:18.825035+0000","last_active":"2024-09-16T09:38:11.559573+0000","last_peered":"2024-09-16T09:38:11.559573+0000","last_clean":"2024-09-16T09:38:11.559573+0000","last_became_active":"2024-09-16T09:31:09.271662+0000","last_became_peered":"2024-09-16T09:31:09.271662+0000","last_unstale":"2024-09-16T09:38:11.559573+0000","last_undegraded":"2024-09-16T09:38:11.559573+0000","last_fullsized":"2024-09-16T09:38:11.559573+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:18.824937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:37:18.824937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:29.541577+0000","scrub_duration":79,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"285'19","reported_seq":735,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.552925+0000","last_change":"2024-09-16T09:37:25.995663+0000","last_active":"2024-09-16T09:38:11.552925+0000","last_peered":"2024-09-16T09:38:11.552925+0000","last_clean":"2024-09-16T09:38:11.552925+0000","last_became_active":"2024-09-16T09:25:03.707769+0000","last_became_peered":"2024-09-16T09:25:03.707769+0000","last_unstale":"2024-09-16T09:38:11.552925+0000","last_undegraded":"2024-09-16T09:38:11.552925+0000","last_fullsized":"2024-09-16T09:38:11.552925+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'19","last_scrub_stamp":"2024-09-16T09:37:25.995607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:37:25.995607+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:34.249870+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":692,"reported_epoch":388,"state":"active+clean","last_fresh":"2024-09-16T09:38:11.559085+0000","last_change":"2024-09-16T09:37:26.646784+0000","last_active":"2024-09-16T09:38:11.559085+0000","last_peered":"2024-09-16T09:38:11.559085+0000","last_clean":"2024-09-16T09:38:11.559085+0000","last_became_active":"2024-09-16T09:25:05.711952+0000","last_became_peered":"2024-09-16T09:25:05.711952+0000","last_unstale":"2024-09-16T09:38:11.559085+0000","last_undegraded":"2024-09-16T09:38:11.559085+0000","last_fullsized":"2024-09-16T09:38:11.559085+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:26.646686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:37:26.646686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:54.298242+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":16,"stat_sum":{"num_bytes":2628124672,"num_objects":40102,"num_object_clones":0,"num_object_copies":120306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40102,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":40102,"num_write_kb":2566528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":7944404992,"data_stored":7944404992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":40102,"ondisk_log_size":40102,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":1117,"num_read_kb":874,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":134,"num_bytes_recovered":3576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":681,"ondisk_log_size":681,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1986,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":785,"num_read_kb":780,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":21,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":327680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1659,"ondisk_log_size":1659,"up":63,"acting":63,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266851,"num_pgs":66,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1187904,"kb_used_data":619432,"kb_used_omap":1,"kb_used_meta":568446,"kb_avail":92557248,"statfs":{"total":95995035648,"available":94778621952,"internally_reserved":0,"allocated":634298368,"data_stored":633675616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":582089162},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495784,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1530280,"kb_used_data":1142288,"kb_used_omap":1,"kb_used_meta":387966,"kb_avail":92214872,"statfs":{"total":95995035648,"available":94428028928,"internally_reserved":0,"allocated":1169702912,"data_stored":1169103638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":397277638},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,2,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724719,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1495420,"kb_used_data":878368,"kb_used_omap":1,"kb_used_meta":617022,"kb_avail":92249732,"statfs":{"total":95995035648,"available":94463725568,"internally_reserved":0,"allocated":899448832,"data_stored":898834297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":631830982},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,1,4],"upper_bound":512},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953652,"num_pgs":72,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1662272,"kb_used_data":1225704,"kb_used_omap":1,"kb_used_meta":436542,"kb_avail":92082880,"statfs":{"total":95995035648,"available":94292869120,"internally_reserved":0,"allocated":1255120896,"data_stored":1254498098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":447019462},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149882,"num_pgs":70,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1440904,"kb_used_data":805816,"kb_used_omap":1,"kb_used_meta":635070,"kb_avail":92304248,"statfs":{"total":95995035648,"available":94519549952,"internally_reserved":0,"allocated":825155584,"data_stored":824517342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":650312135},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411519,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1402532,"kb_used_data":960400,"kb_used_omap":1,"kb_used_meta":442110,"kb_avail":92342620,"statfs":{"total":95995035648,"available":94558842880,"internally_reserved":0,"allocated":983449600,"data_stored":982850692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":452721093},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575046,"num_pgs":65,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1147800,"kb_used_data":484936,"kb_used_omap":1,"kb_used_meta":662846,"kb_avail":92597352,"statfs":{"total":95995035648,"available":94819688448,"internally_reserved":0,"allocated":496574464,"data_stored":495918988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":678754757},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738572,"num_pgs":62,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2202748,"kb_used_data":1621804,"kb_used_omap":1,"kb_used_meta":580926,"kb_avail":91542404,"statfs":{"total":95995035648,"available":93739421696,"internally_reserved":0,"allocated":1660727296,"data_stored":1660100714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":594868677},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1728053248,"data_stored":1728053248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":496435200,"data_stored":496435200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":986841088,"data_stored":986841088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":839974912,"data_stored":839974912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1295712256,"data_stored":1295712256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":825163776,"data_stored":825163776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1125056512,"data_stored":1125056512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":647168000,"data_stored":647168000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T09:38:18.593 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-16T09:38:18.614 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:18 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:38:18.615 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:18 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:38:18.651 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:38:18.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:18 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:38:18.673 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:18.673 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:18.673 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:38:18.774 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:38:18.812 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:18.813 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:18.813 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:38:18.839 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:38:18.914 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:38:18.948 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:18.948 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:18.949 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:38:19.050 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:38:19.081 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:19.083 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:19.083 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:38:19.184 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:38:19.218 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:19.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:19.219 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:38:19.320 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:38:19.359 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:19.359 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:19.359 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:38:19.424 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.2 dump_historic_ops 2024-09-16T09:38:19.460 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:38:19.494 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:19.496 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:19.497 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:38:19.539 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 23 16 51360 51344 139.483 92.4375 0.00971779 0.00716413 2024-09-16T09:38:19.562 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T09:38:19.597 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:38:19.609 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:19 smithi070 ceph-mon[30388]: pgmap v702: 168 pgs: 168 active+clean; 2.7 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 709 B/s rd, 113 MiB/s wr, 1.80k op/s 2024-09-16T09:38:19.609 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:19 smithi070 ceph-mon[40691]: pgmap v702: 168 pgs: 168 active+clean; 2.7 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 709 B/s rd, 113 MiB/s wr, 1.80k op/s 2024-09-16T09:38:19.634 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:19.635 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:19.635 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:38:19.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:19 smithi084 ceph-mon[32629]: pgmap v702: 168 pgs: 168 active+clean; 2.7 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 709 B/s rd, 113 MiB/s wr, 1.80k op/s 2024-09-16T09:38:19.737 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:38:19.777 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:19.777 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:19.778 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:38:19.878 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:38:19.908 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:19.908 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:19.909 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:38:20.009 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:38:20.031 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:20.032 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:20.032 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:38:20.057 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:38:20.133 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:38:20.180 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:20.181 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:20.182 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:38:20.282 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:38:20.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:20.320 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:20.320 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:38:20.351 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:38:20.380 INFO:teuthology.orchestra.run.smithi070.stderr:noscrub is unset 2024-09-16T09:38:20.421 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:38:20.457 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:20.457 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:20.458 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:38:20.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:20 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/445588777' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:38:20.491 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:20 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:38:20.491 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:20 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/445588777' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:38:20.491 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:20 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:38:20.539 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 24 16 52590 52574 136.874 76.875 0.00375875 0.00730083 2024-09-16T09:38:20.558 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:38:20.591 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:20.592 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:20.592 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:38:20.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:20 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/445588777' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:38:20.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:20 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:38:20.693 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:38:20.731 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:20.732 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:20.732 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:38:20.833 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:38:20.868 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:20.869 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:20.869 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:38:20.970 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:38:21.008 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:21.009 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:21.009 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:38:21.110 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:38:21.136 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:21.137 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:21.137 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:38:21.238 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:38:21.273 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:21.274 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:21.274 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:38:21.375 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:38:21.399 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:21.400 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:21.400 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:38:21.501 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:38:21.535 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:21.536 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:21.536 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:38:21.539 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 25 16 54300 54284 135.674 106.875 0.00685298 0.00736576 2024-09-16T09:38:21.636 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:38:21.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:21 smithi084 ceph-mon[32629]: pgmap v703: 168 pgs: 168 active+clean; 3.0 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.2 KiB/s rd, 142 MiB/s wr, 2.27k op/s 2024-09-16T09:38:21.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:21 smithi084 ceph-mon[32629]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:38:21.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:21 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-16T09:38:21.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:21 smithi084 ceph-mon[32629]: osdmap e390: 8 total, 8 up, 8 in 2024-09-16T09:38:21.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:21 smithi084 ceph-mon[32629]: 6.b scrub starts 2024-09-16T09:38:21.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:21 smithi084 ceph-mon[32629]: 6.b scrub ok 2024-09-16T09:38:21.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:21 smithi070 ceph-mon[30388]: pgmap v703: 168 pgs: 168 active+clean; 3.0 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.2 KiB/s rd, 142 MiB/s wr, 2.27k op/s 2024-09-16T09:38:21.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:21 smithi070 ceph-mon[30388]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:38:21.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:21 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-16T09:38:21.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:21 smithi070 ceph-mon[30388]: osdmap e390: 8 total, 8 up, 8 in 2024-09-16T09:38:21.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:21 smithi070 ceph-mon[30388]: 6.b scrub starts 2024-09-16T09:38:21.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:21 smithi070 ceph-mon[30388]: 6.b scrub ok 2024-09-16T09:38:21.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:21 smithi070 ceph-mon[40691]: pgmap v703: 168 pgs: 168 active+clean; 3.0 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.2 KiB/s rd, 142 MiB/s wr, 2.27k op/s 2024-09-16T09:38:21.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:21 smithi070 ceph-mon[40691]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:38:21.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:21 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-16T09:38:21.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:21 smithi070 ceph-mon[40691]: osdmap e390: 8 total, 8 up, 8 in 2024-09-16T09:38:21.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:21 smithi070 ceph-mon[40691]: 6.b scrub starts 2024-09-16T09:38:21.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:21 smithi070 ceph-mon[40691]: 6.b scrub ok 2024-09-16T09:38:21.680 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:21.681 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:21.681 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:38:21.782 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:38:21.823 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:21.824 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:21.824 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:38:21.925 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:38:21.966 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:21.967 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:21.967 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:38:22.001 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:38:22.004 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2024-09-16T09:38:22.028 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:38:21 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:38:21] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:38:22.068 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:38:22.119 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:22.120 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:22.120 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:38:22.221 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:38:22.255 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:22.255 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:22.255 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:38:22.356 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:38:22.389 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:22.390 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:22.390 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:38:22.491 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:38:22.532 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:22.533 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:22.533 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:38:22.539 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 26 16 55678 55662 133.769 86.125 0.00978556 0.00747055 2024-09-16T09:38:22.634 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:38:22.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:22 smithi084 ceph-mon[32629]: 3.16 scrub starts 2024-09-16T09:38:22.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:22 smithi084 ceph-mon[32629]: 3.16 scrub ok 2024-09-16T09:38:22.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:22 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:38:22.670 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:22.670 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:22.671 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:38:22.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:22 smithi070 ceph-mon[30388]: 3.16 scrub starts 2024-09-16T09:38:22.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:22 smithi070 ceph-mon[30388]: 3.16 scrub ok 2024-09-16T09:38:22.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:22 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:38:22.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:22 smithi070 ceph-mon[40691]: 3.16 scrub starts 2024-09-16T09:38:22.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:22 smithi070 ceph-mon[40691]: 3.16 scrub ok 2024-09-16T09:38:22.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:22 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:38:22.771 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:38:22.811 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:22.811 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:22.811 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:38:22.860 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":705,"stamp":"2024-09-16T09:38:21.760949+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3220118043,"num_objects":49325,"num_object_clones":0,"num_object_copies":147975,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49325,"num_whiteouts":0,"num_read":1931,"num_read_kb":1683,"num_write":49848,"num_write_kb":3144109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":140,"num_bytes_recovered":333242,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":51554,"ondisk_log_size":51554,"up":504,"acting":504,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":506,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":13838876,"kb_used_data":9460636,"kb_used_omap":12,"kb_used_meta":4378099,"kb_avail":736122340,"statfs":{"total":767960285184,"available":753789276160,"internally_reserved":0,"allocated":9687691264,"data_stored":9682680121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":4483173938},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,2,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":706609152,"num_objects":10782,"num_object_clones":0,"num_object_copies":32346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10782,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":10782,"num_write_kb":690048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.005514"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":710,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.349356+0000","last_change":"2024-09-16T09:32:39.384174+0000","last_active":"2024-09-16T09:38:14.349356+0000","last_peered":"2024-09-16T09:38:14.349356+0000","last_clean":"2024-09-16T09:38:14.349356+0000","last_became_active":"2024-09-16T09:25:01.709108+0000","last_became_peered":"2024-09-16T09:25:01.709108+0000","last_unstale":"2024-09-16T09:38:14.349356+0000","last_undegraded":"2024-09-16T09:38:14.349356+0000","last_fullsized":"2024-09-16T09:38:14.349356+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:39.384104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:32:39.384104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"285'15","reported_seq":722,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.338324+0000","last_change":"2024-09-16T09:32:38.074301+0000","last_active":"2024-09-16T09:38:14.338324+0000","last_peered":"2024-09-16T09:38:14.338324+0000","last_clean":"2024-09-16T09:38:14.338324+0000","last_became_active":"2024-09-16T09:25:03.714635+0000","last_became_peered":"2024-09-16T09:25:03.714635+0000","last_unstale":"2024-09-16T09:38:14.338324+0000","last_undegraded":"2024-09-16T09:38:14.338324+0000","last_fullsized":"2024-09-16T09:38:14.338324+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:32:38.074231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:32:38.074231+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:07.303929+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":332,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.336887+0000","last_change":"2024-09-16T09:36:53.098680+0000","last_active":"2024-09-16T09:38:14.336887+0000","last_peered":"2024-09-16T09:38:14.336887+0000","last_clean":"2024-09-16T09:38:14.336887+0000","last_became_active":"2024-09-16T09:33:30.242238+0000","last_became_peered":"2024-09-16T09:33:30.242238+0000","last_unstale":"2024-09-16T09:38:14.336887+0000","last_undegraded":"2024-09-16T09:38:14.336887+0000","last_fullsized":"2024-09-16T09:38:14.336887+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:53.098635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:36:53.098635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":689,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.338518+0000","last_change":"2024-09-16T09:37:56.314518+0000","last_active":"2024-09-16T09:38:14.338518+0000","last_peered":"2024-09-16T09:38:14.338518+0000","last_clean":"2024-09-16T09:38:14.338518+0000","last_became_active":"2024-09-16T09:25:07.715442+0000","last_became_peered":"2024-09-16T09:25:07.715442+0000","last_unstale":"2024-09-16T09:38:14.338518+0000","last_undegraded":"2024-09-16T09:38:14.338518+0000","last_fullsized":"2024-09-16T09:38:14.338518+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:56.314468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:37:56.314468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:04.475570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":426,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.344655+0000","last_change":"2024-09-16T09:34:33.452784+0000","last_active":"2024-09-16T09:38:14.344655+0000","last_peered":"2024-09-16T09:38:14.344655+0000","last_clean":"2024-09-16T09:38:14.344655+0000","last_became_active":"2024-09-16T09:31:09.269342+0000","last_became_peered":"2024-09-16T09:31:09.269342+0000","last_unstale":"2024-09-16T09:38:14.344655+0000","last_undegraded":"2024-09-16T09:38:14.344655+0000","last_fullsized":"2024-09-16T09:38:14.344655+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:33.452706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:34:33.452706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":567,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.341402+0000","last_change":"2024-09-16T09:33:08.816231+0000","last_active":"2024-09-16T09:38:14.341402+0000","last_peered":"2024-09-16T09:38:14.341402+0000","last_clean":"2024-09-16T09:38:14.341402+0000","last_became_active":"2024-09-16T09:31:09.275136+0000","last_became_peered":"2024-09-16T09:31:09.275136+0000","last_unstale":"2024-09-16T09:38:14.341402+0000","last_undegraded":"2024-09-16T09:38:14.341402+0000","last_fullsized":"2024-09-16T09:38:14.341402+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:08.816174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:08.816174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:08.989282+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.19","version":"285'27","reported_seq":408,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.348505+0000","last_change":"2024-09-16T09:37:29.683657+0000","last_active":"2024-09-16T09:38:14.348505+0000","last_peered":"2024-09-16T09:38:14.348505+0000","last_clean":"2024-09-16T09:38:14.348505+0000","last_became_active":"2024-09-16T09:33:21.337284+0000","last_became_peered":"2024-09-16T09:33:21.337284+0000","last_unstale":"2024-09-16T09:38:14.348505+0000","last_undegraded":"2024-09-16T09:38:14.348505+0000","last_fullsized":"2024-09-16T09:38:14.348505+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'27","last_scrub_stamp":"2024-09-16T09:37:29.683568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:37:29.683568+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:35.791977+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":701,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.338225+0000","last_change":"2024-09-16T09:32:44.114475+0000","last_active":"2024-09-16T09:38:14.338225+0000","last_peered":"2024-09-16T09:38:14.338225+0000","last_clean":"2024-09-16T09:38:14.338225+0000","last_became_active":"2024-09-16T09:25:05.717236+0000","last_became_peered":"2024-09-16T09:25:05.717236+0000","last_unstale":"2024-09-16T09:38:14.338225+0000","last_undegraded":"2024-09-16T09:38:14.338225+0000","last_fullsized":"2024-09-16T09:38:14.338225+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:44.114407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:32:44.114407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:03.327373+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":333,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.350963+0000","last_change":"2024-09-16T09:33:36.655594+0000","last_active":"2024-09-16T09:38:14.350963+0000","last_peered":"2024-09-16T09:38:14.350963+0000","last_clean":"2024-09-16T09:38:14.350963+0000","last_became_active":"2024-09-16T09:33:30.247733+0000","last_became_peered":"2024-09-16T09:33:30.247733+0000","last_unstale":"2024-09-16T09:38:14.350963+0000","last_undegraded":"2024-09-16T09:38:14.350963+0000","last_fullsized":"2024-09-16T09:38:14.350963+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:36.655236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:36.655236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:00.446453+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1a","version":"285'15","reported_seq":722,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.338677+0000","last_change":"2024-09-16T09:32:48.185484+0000","last_active":"2024-09-16T09:38:14.338677+0000","last_peered":"2024-09-16T09:38:14.338677+0000","last_clean":"2024-09-16T09:38:14.338677+0000","last_became_active":"2024-09-16T09:25:03.709147+0000","last_became_peered":"2024-09-16T09:25:03.709147+0000","last_unstale":"2024-09-16T09:38:14.338677+0000","last_undegraded":"2024-09-16T09:38:14.338677+0000","last_fullsized":"2024-09-16T09:38:14.338677+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:32:48.185380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:32:48.185380+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:12.337650+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":692,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.337340+0000","last_change":"2024-09-16T09:36:51.134364+0000","last_active":"2024-09-16T09:38:14.337340+0000","last_peered":"2024-09-16T09:38:14.337340+0000","last_clean":"2024-09-16T09:38:14.337340+0000","last_became_active":"2024-09-16T09:25:05.718134+0000","last_became_peered":"2024-09-16T09:25:05.718134+0000","last_unstale":"2024-09-16T09:38:14.337340+0000","last_undegraded":"2024-09-16T09:38:14.337340+0000","last_fullsized":"2024-09-16T09:38:14.337340+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:51.134314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:36:51.134314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"66'1","reported_seq":698,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.348002+0000","last_change":"2024-09-16T09:33:38.705001+0000","last_active":"2024-09-16T09:38:14.348002+0000","last_peered":"2024-09-16T09:38:14.348002+0000","last_clean":"2024-09-16T09:38:14.348002+0000","last_became_active":"2024-09-16T09:25:07.716558+0000","last_became_peered":"2024-09-16T09:25:07.716558+0000","last_unstale":"2024-09-16T09:38:14.348002+0000","last_undegraded":"2024-09-16T09:38:14.348002+0000","last_fullsized":"2024-09-16T09:38:14.348002+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-16T09:33:38.704868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:33:38.704868+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":692,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.337455+0000","last_change":"2024-09-16T09:33:19.326765+0000","last_active":"2024-09-16T09:38:14.337455+0000","last_peered":"2024-09-16T09:38:14.337455+0000","last_clean":"2024-09-16T09:38:14.337455+0000","last_became_active":"2024-09-16T09:25:01.696939+0000","last_became_peered":"2024-09-16T09:25:01.696939+0000","last_unstale":"2024-09-16T09:38:14.337455+0000","last_undegraded":"2024-09-16T09:38:14.337455+0000","last_fullsized":"2024-09-16T09:38:14.337455+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:19.326695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:19.326695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:11.230887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"285'7","reported_seq":710,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.338470+0000","last_change":"2024-09-16T09:35:39.975915+0000","last_active":"2024-09-16T09:38:14.338470+0000","last_peered":"2024-09-16T09:38:14.338470+0000","last_clean":"2024-09-16T09:38:14.338470+0000","last_became_active":"2024-09-16T09:25:03.716136+0000","last_became_peered":"2024-09-16T09:25:03.716136+0000","last_unstale":"2024-09-16T09:38:14.338470+0000","last_undegraded":"2024-09-16T09:38:14.338470+0000","last_fullsized":"2024-09-16T09:38:14.338470+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'7","last_scrub_stamp":"2024-09-16T09:35:39.975840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:35:39.975840+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:00.297013+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":10,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":683,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.353182+0000","last_change":"2024-09-16T09:28:52.746046+0000","last_active":"2024-09-16T09:38:14.353182+0000","last_peered":"2024-09-16T09:38:14.353182+0000","last_clean":"2024-09-16T09:38:14.353182+0000","last_became_active":"2024-09-16T09:25:05.718281+0000","last_became_peered":"2024-09-16T09:25:05.718281+0000","last_unstale":"2024-09-16T09:38:14.353182+0000","last_undegraded":"2024-09-16T09:38:14.353182+0000","last_fullsized":"2024-09-16T09:38:14.353182+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:52.745999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:28:52.745999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:04.420096+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":710,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.336981+0000","last_change":"2024-09-16T09:33:46.366042+0000","last_active":"2024-09-16T09:38:14.336981+0000","last_peered":"2024-09-16T09:38:14.336981+0000","last_clean":"2024-09-16T09:38:14.336981+0000","last_became_active":"2024-09-16T09:31:09.275656+0000","last_became_peered":"2024-09-16T09:31:09.275656+0000","last_unstale":"2024-09-16T09:38:14.336981+0000","last_undegraded":"2024-09-16T09:38:14.336981+0000","last_fullsized":"2024-09-16T09:38:14.336981+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:46.365972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:33:46.365972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":73,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":698,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.339322+0000","last_change":"2024-09-16T09:33:48.322010+0000","last_active":"2024-09-16T09:38:14.339322+0000","last_peered":"2024-09-16T09:38:14.339322+0000","last_clean":"2024-09-16T09:38:14.339322+0000","last_became_active":"2024-09-16T09:25:07.720793+0000","last_became_peered":"2024-09-16T09:25:07.720793+0000","last_unstale":"2024-09-16T09:38:14.339322+0000","last_undegraded":"2024-09-16T09:38:14.339322+0000","last_fullsized":"2024-09-16T09:38:14.339322+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:48.321895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:33:48.321895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:41.370700+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":449,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.341285+0000","last_change":"2024-09-16T09:34:33.866299+0000","last_active":"2024-09-16T09:38:14.341285+0000","last_peered":"2024-09-16T09:38:14.341285+0000","last_clean":"2024-09-16T09:38:14.341285+0000","last_became_active":"2024-09-16T09:30:39.201824+0000","last_became_peered":"2024-09-16T09:30:39.201824+0000","last_unstale":"2024-09-16T09:38:14.341285+0000","last_undegraded":"2024-09-16T09:38:14.341285+0000","last_fullsized":"2024-09-16T09:38:14.341285+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:33.866142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:34:33.866142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:40.929489+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"285'27","reported_seq":797,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.344110+0000","last_change":"2024-09-16T09:36:54.484021+0000","last_active":"2024-09-16T09:38:14.344110+0000","last_peered":"2024-09-16T09:38:14.344110+0000","last_clean":"2024-09-16T09:38:14.344110+0000","last_became_active":"2024-09-16T09:33:30.249798+0000","last_became_peered":"2024-09-16T09:33:30.249798+0000","last_unstale":"2024-09-16T09:38:14.344110+0000","last_undegraded":"2024-09-16T09:38:14.344110+0000","last_fullsized":"2024-09-16T09:38:14.344110+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'27","last_scrub_stamp":"2024-09-16T09:36:54.483966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:36:54.483966+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:07.794557+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":332,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.339039+0000","last_change":"2024-09-16T09:33:43.326933+0000","last_active":"2024-09-16T09:38:14.339039+0000","last_peered":"2024-09-16T09:38:14.339039+0000","last_clean":"2024-09-16T09:38:14.339039+0000","last_became_active":"2024-09-16T09:33:30.251587+0000","last_became_peered":"2024-09-16T09:33:30.251587+0000","last_unstale":"2024-09-16T09:38:14.339039+0000","last_undegraded":"2024-09-16T09:38:14.339039+0000","last_fullsized":"2024-09-16T09:38:14.339039+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:43.326852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:43.326852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:42.368973+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,0],"acting":[4,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":546,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.341351+0000","last_change":"2024-09-16T09:34:37.804874+0000","last_active":"2024-09-16T09:38:14.341351+0000","last_peered":"2024-09-16T09:38:14.341351+0000","last_clean":"2024-09-16T09:38:14.341351+0000","last_became_active":"2024-09-16T09:28:52.974255+0000","last_became_peered":"2024-09-16T09:28:52.974255+0000","last_unstale":"2024-09-16T09:38:14.341351+0000","last_undegraded":"2024-09-16T09:38:14.341351+0000","last_fullsized":"2024-09-16T09:38:14.341351+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:37.804805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:34:37.804805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:00.917819+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":709,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.338786+0000","last_change":"2024-09-16T09:35:23.751992+0000","last_active":"2024-09-16T09:38:14.338786+0000","last_peered":"2024-09-16T09:38:14.338786+0000","last_clean":"2024-09-16T09:38:14.338786+0000","last_became_active":"2024-09-16T09:25:01.706491+0000","last_became_peered":"2024-09-16T09:25:01.706491+0000","last_unstale":"2024-09-16T09:38:14.338786+0000","last_undegraded":"2024-09-16T09:38:14.338786+0000","last_fullsized":"2024-09-16T09:38:14.338786+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:35:23.751946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:35:23.751946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"285'18","reported_seq":384,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.339130+0000","last_change":"2024-09-16T09:33:39.284526+0000","last_active":"2024-09-16T09:38:14.339130+0000","last_peered":"2024-09-16T09:38:14.339130+0000","last_clean":"2024-09-16T09:38:14.339130+0000","last_became_active":"2024-09-16T09:33:30.251626+0000","last_became_peered":"2024-09-16T09:33:30.251626+0000","last_unstale":"2024-09-16T09:38:14.339130+0000","last_undegraded":"2024-09-16T09:38:14.339130+0000","last_fullsized":"2024-09-16T09:38:14.339130+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-16T09:33:39.284451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:33:39.284451+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:34.368282+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":33,"num_read_kb":24,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":685,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.339202+0000","last_change":"2024-09-16T09:33:38.238661+0000","last_active":"2024-09-16T09:38:14.339202+0000","last_peered":"2024-09-16T09:38:14.339202+0000","last_clean":"2024-09-16T09:38:14.339202+0000","last_became_active":"2024-09-16T09:25:05.710093+0000","last_became_peered":"2024-09-16T09:25:05.710093+0000","last_unstale":"2024-09-16T09:38:14.339202+0000","last_undegraded":"2024-09-16T09:38:14.339202+0000","last_fullsized":"2024-09-16T09:38:14.339202+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:38.238593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:38.238593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:36.342171+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":696,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.353727+0000","last_change":"2024-09-16T09:37:48.412421+0000","last_active":"2024-09-16T09:38:14.353727+0000","last_peered":"2024-09-16T09:38:14.353727+0000","last_clean":"2024-09-16T09:38:14.353727+0000","last_became_active":"2024-09-16T09:25:07.719380+0000","last_became_peered":"2024-09-16T09:25:07.719380+0000","last_unstale":"2024-09-16T09:38:14.353727+0000","last_undegraded":"2024-09-16T09:38:14.353727+0000","last_fullsized":"2024-09-16T09:38:14.353727+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-16T09:37:48.411054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:37:48.411054+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:01.043235+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":341,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.341957+0000","last_change":"2024-09-16T09:34:41.401691+0000","last_active":"2024-09-16T09:38:14.341957+0000","last_peered":"2024-09-16T09:38:14.341957+0000","last_clean":"2024-09-16T09:38:14.341957+0000","last_became_active":"2024-09-16T09:33:30.245048+0000","last_became_peered":"2024-09-16T09:33:30.245048+0000","last_unstale":"2024-09-16T09:38:14.341957+0000","last_undegraded":"2024-09-16T09:38:14.341957+0000","last_fullsized":"2024-09-16T09:38:14.341957+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:34:41.401641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:34:41.401641+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:03.993235+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"285'20","reported_seq":766,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.351164+0000","last_change":"2024-09-16T09:32:30.286077+0000","last_active":"2024-09-16T09:38:14.351164+0000","last_peered":"2024-09-16T09:38:14.351164+0000","last_clean":"2024-09-16T09:38:14.351164+0000","last_became_active":"2024-09-16T09:31:09.275050+0000","last_became_peered":"2024-09-16T09:31:09.275050+0000","last_unstale":"2024-09-16T09:38:14.351164+0000","last_undegraded":"2024-09-16T09:38:14.351164+0000","last_fullsized":"2024-09-16T09:38:14.351164+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-16T09:32:30.284919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:32:30.284919+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:02.456876+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"66'8","reported_seq":889,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.341638+0000","last_change":"2024-09-16T09:34:52.922123+0000","last_active":"2024-09-16T09:38:14.341638+0000","last_peered":"2024-09-16T09:38:14.341638+0000","last_clean":"2024-09-16T09:38:14.341638+0000","last_became_active":"2024-09-16T09:25:05.712516+0000","last_became_peered":"2024-09-16T09:25:05.712516+0000","last_unstale":"2024-09-16T09:38:14.341638+0000","last_undegraded":"2024-09-16T09:38:14.341638+0000","last_fullsized":"2024-09-16T09:38:14.341638+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:34:52.922047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:34:52.922047+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":33,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1","version":"389'3069","reported_seq":3106,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:17.969638+0000","last_change":"2024-09-16T09:37:45.401338+0000","last_active":"2024-09-16T09:38:17.969638+0000","last_peered":"2024-09-16T09:38:17.969638+0000","last_clean":"2024-09-16T09:38:17.969638+0000","last_became_active":"2024-09-16T09:37:45.400966+0000","last_became_peered":"2024-09-16T09:37:45.400966+0000","last_unstale":"2024-09-16T09:38:17.969638+0000","last_undegraded":"2024-09-16T09:38:17.969638+0000","last_fullsized":"2024-09-16T09:38:17.969638+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_clean_scrub_stamp":"2024-09-16T09:37:44.378497+0000","objects_scrubbed":0,"log_size":3069,"log_dups_size":0,"ondisk_log_size":3069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:52.315503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":201129984,"num_objects":3069,"num_object_clones":0,"num_object_copies":9207,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3069,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3069,"num_write_kb":196416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.a","version":"73'79","reported_seq":632,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.336933+0000","last_change":"2024-09-16T09:37:52.203798+0000","last_active":"2024-09-16T09:38:14.336933+0000","last_peered":"2024-09-16T09:38:14.336933+0000","last_clean":"2024-09-16T09:38:14.336933+0000","last_became_active":"2024-09-16T09:26:35.127352+0000","last_became_peered":"2024-09-16T09:26:35.127352+0000","last_unstale":"2024-09-16T09:38:14.336933+0000","last_undegraded":"2024-09-16T09:38:14.336933+0000","last_fullsized":"2024-09-16T09:38:14.336933+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:37:52.203752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:37:52.203752+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:59.150165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"285'27","reported_seq":659,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.346779+0000","last_change":"2024-09-16T09:36:47.588508+0000","last_active":"2024-09-16T09:38:14.346779+0000","last_peered":"2024-09-16T09:38:14.346779+0000","last_clean":"2024-09-16T09:38:14.346779+0000","last_became_active":"2024-09-16T09:26:33.295672+0000","last_became_peered":"2024-09-16T09:26:33.295672+0000","last_unstale":"2024-09-16T09:38:14.346779+0000","last_undegraded":"2024-09-16T09:38:14.346779+0000","last_fullsized":"2024-09-16T09:38:14.346779+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'27","last_scrub_stamp":"2024-09-16T09:36:47.588449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:36:47.588449+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:14.427263+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":435,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.342119+0000","last_change":"2024-09-16T09:37:16.625626+0000","last_active":"2024-09-16T09:38:14.342119+0000","last_peered":"2024-09-16T09:38:14.342119+0000","last_clean":"2024-09-16T09:38:14.342119+0000","last_became_active":"2024-09-16T09:31:09.274534+0000","last_became_peered":"2024-09-16T09:31:09.274534+0000","last_unstale":"2024-09-16T09:38:14.342119+0000","last_undegraded":"2024-09-16T09:38:14.342119+0000","last_fullsized":"2024-09-16T09:38:14.342119+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:16.625556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:37:16.625556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:26.929216+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"66'8","reported_seq":853,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.339422+0000","last_change":"2024-09-16T09:37:49.336009+0000","last_active":"2024-09-16T09:38:14.339422+0000","last_peered":"2024-09-16T09:38:14.339422+0000","last_clean":"2024-09-16T09:38:14.339422+0000","last_became_active":"2024-09-16T09:25:05.713813+0000","last_became_peered":"2024-09-16T09:25:05.713813+0000","last_unstale":"2024-09-16T09:38:14.339422+0000","last_undegraded":"2024-09-16T09:38:14.339422+0000","last_fullsized":"2024-09-16T09:38:14.339422+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:37:49.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:37:49.335939+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:08.862747+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":688,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.337643+0000","last_change":"2024-09-16T09:36:37.152446+0000","last_active":"2024-09-16T09:38:14.337643+0000","last_peered":"2024-09-16T09:38:14.337643+0000","last_clean":"2024-09-16T09:38:14.337643+0000","last_became_active":"2024-09-16T09:25:07.717344+0000","last_became_peered":"2024-09-16T09:25:07.717344+0000","last_unstale":"2024-09-16T09:38:14.337643+0000","last_undegraded":"2024-09-16T09:38:14.337643+0000","last_fullsized":"2024-09-16T09:38:14.337643+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:37.152377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:36:37.152377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.e","version":"389'3084","reported_seq":3121,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:17.937778+0000","last_change":"2024-09-16T09:37:45.464349+0000","last_active":"2024-09-16T09:38:17.937778+0000","last_peered":"2024-09-16T09:38:17.937778+0000","last_clean":"2024-09-16T09:38:17.937778+0000","last_became_active":"2024-09-16T09:37:45.463961+0000","last_became_peered":"2024-09-16T09:37:45.463961+0000","last_unstale":"2024-09-16T09:38:17.937778+0000","last_undegraded":"2024-09-16T09:38:17.937778+0000","last_fullsized":"2024-09-16T09:38:17.937778+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_clean_scrub_stamp":"2024-09-16T09:37:44.378497+0000","objects_scrubbed":0,"log_size":3084,"log_dups_size":0,"ondisk_log_size":3084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:49.201675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":202113024,"num_objects":3084,"num_object_clones":0,"num_object_copies":9252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3084,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3084,"num_write_kb":197376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"73'79","reported_seq":664,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.339488+0000","last_change":"2024-09-16T09:32:42.123139+0000","last_active":"2024-09-16T09:38:14.339488+0000","last_peered":"2024-09-16T09:38:14.339488+0000","last_clean":"2024-09-16T09:38:14.339488+0000","last_became_active":"2024-09-16T09:30:08.763683+0000","last_became_peered":"2024-09-16T09:30:08.763683+0000","last_unstale":"2024-09-16T09:38:14.339488+0000","last_undegraded":"2024-09-16T09:38:14.339488+0000","last_fullsized":"2024-09-16T09:38:14.339488+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:32:42.123067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:32:42.123067+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:09.339965+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"285'30","reported_seq":656,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.353358+0000","last_change":"2024-09-16T09:33:21.333996+0000","last_active":"2024-09-16T09:38:14.353358+0000","last_peered":"2024-09-16T09:38:14.353358+0000","last_clean":"2024-09-16T09:38:14.353358+0000","last_became_active":"2024-09-16T09:31:09.270434+0000","last_became_peered":"2024-09-16T09:31:09.270434+0000","last_unstale":"2024-09-16T09:38:14.353358+0000","last_undegraded":"2024-09-16T09:38:14.353358+0000","last_fullsized":"2024-09-16T09:38:14.353358+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'18","last_scrub_stamp":"2024-09-16T09:33:21.333937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:33:21.333937+0000","objects_scrubbed":8,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:38.469118+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":51,"num_read_kb":39,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":536,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.355059+0000","last_change":"2024-09-16T09:36:50.622852+0000","last_active":"2024-09-16T09:38:14.355059+0000","last_peered":"2024-09-16T09:38:14.355059+0000","last_clean":"2024-09-16T09:38:14.355059+0000","last_became_active":"2024-09-16T09:28:52.983543+0000","last_became_peered":"2024-09-16T09:28:52.983543+0000","last_unstale":"2024-09-16T09:38:14.355059+0000","last_undegraded":"2024-09-16T09:38:14.355059+0000","last_fullsized":"2024-09-16T09:38:14.355059+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:50.622807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:36:50.622807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:59.525944+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":716,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.338433+0000","last_change":"2024-09-16T09:35:19.822713+0000","last_active":"2024-09-16T09:38:14.338433+0000","last_peered":"2024-09-16T09:38:14.338433+0000","last_clean":"2024-09-16T09:38:14.338433+0000","last_became_active":"2024-09-16T09:31:09.268864+0000","last_became_peered":"2024-09-16T09:31:09.268864+0000","last_unstale":"2024-09-16T09:38:14.338433+0000","last_undegraded":"2024-09-16T09:38:14.338433+0000","last_fullsized":"2024-09-16T09:38:14.338433+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:35:19.822635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:35:19.822635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:11.341259+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":704,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.338633+0000","last_change":"2024-09-16T09:35:46.933987+0000","last_active":"2024-09-16T09:38:14.338633+0000","last_peered":"2024-09-16T09:38:14.338633+0000","last_clean":"2024-09-16T09:38:14.338633+0000","last_became_active":"2024-09-16T09:31:09.269433+0000","last_became_peered":"2024-09-16T09:31:09.269433+0000","last_unstale":"2024-09-16T09:38:14.338633+0000","last_undegraded":"2024-09-16T09:38:14.338633+0000","last_fullsized":"2024-09-16T09:38:14.338633+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:35:46.933912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:35:46.933912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"389'3004","reported_seq":3041,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:17.744669+0000","last_change":"2024-09-16T09:37:45.463119+0000","last_active":"2024-09-16T09:38:17.744669+0000","last_peered":"2024-09-16T09:38:17.744669+0000","last_clean":"2024-09-16T09:38:17.744669+0000","last_became_active":"2024-09-16T09:37:45.462743+0000","last_became_peered":"2024-09-16T09:37:45.462743+0000","last_unstale":"2024-09-16T09:38:17.744669+0000","last_undegraded":"2024-09-16T09:38:17.744669+0000","last_fullsized":"2024-09-16T09:38:17.744669+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_clean_scrub_stamp":"2024-09-16T09:37:44.378497+0000","objects_scrubbed":0,"log_size":3004,"log_dups_size":0,"ondisk_log_size":3004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:54.300226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":196870144,"num_objects":3004,"num_object_clones":0,"num_object_copies":9012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3004,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3004,"num_write_kb":192256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.4","version":"73'79","reported_seq":435,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.341652+0000","last_change":"2024-09-16T09:37:17.664174+0000","last_active":"2024-09-16T09:38:14.341652+0000","last_peered":"2024-09-16T09:38:14.341652+0000","last_clean":"2024-09-16T09:38:14.341652+0000","last_became_active":"2024-09-16T09:31:09.275139+0000","last_became_peered":"2024-09-16T09:31:09.275139+0000","last_unstale":"2024-09-16T09:38:14.341652+0000","last_undegraded":"2024-09-16T09:38:14.341652+0000","last_fullsized":"2024-09-16T09:38:14.341652+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:37:17.664105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:37:17.664105+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:38.043254+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"285'28","reported_seq":748,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.338519+0000","last_change":"2024-09-16T09:35:43.914378+0000","last_active":"2024-09-16T09:38:14.338519+0000","last_peered":"2024-09-16T09:38:14.338519+0000","last_clean":"2024-09-16T09:38:14.338519+0000","last_became_active":"2024-09-16T09:25:03.708830+0000","last_became_peered":"2024-09-16T09:25:03.708830+0000","last_unstale":"2024-09-16T09:38:14.338519+0000","last_undegraded":"2024-09-16T09:38:14.338519+0000","last_fullsized":"2024-09-16T09:38:14.338519+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'28","last_scrub_stamp":"2024-09-16T09:35:43.914312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:35:43.914312+0000","objects_scrubbed":7,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":719,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.351520+0000","last_change":"2024-09-16T09:33:43.678231+0000","last_active":"2024-09-16T09:38:14.351520+0000","last_peered":"2024-09-16T09:38:14.351520+0000","last_clean":"2024-09-16T09:38:14.351520+0000","last_became_active":"2024-09-16T09:25:01.702992+0000","last_became_peered":"2024-09-16T09:25:01.702992+0000","last_unstale":"2024-09-16T09:38:14.351520+0000","last_undegraded":"2024-09-16T09:38:14.351520+0000","last_fullsized":"2024-09-16T09:38:14.351520+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:33:43.678138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:43.678138+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"235'12","reported_seq":417,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.338282+0000","last_change":"2024-09-16T09:37:19.380567+0000","last_active":"2024-09-16T09:38:14.338282+0000","last_peered":"2024-09-16T09:38:14.338282+0000","last_clean":"2024-09-16T09:38:14.338282+0000","last_became_active":"2024-09-16T09:33:30.251047+0000","last_became_peered":"2024-09-16T09:33:30.251047+0000","last_unstale":"2024-09-16T09:38:14.338282+0000","last_undegraded":"2024-09-16T09:38:14.338282+0000","last_fullsized":"2024-09-16T09:38:14.338282+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"235'12","last_scrub_stamp":"2024-09-16T09:37:19.380504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:37:19.380504+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:37.307298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":680,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.353320+0000","last_change":"2024-09-16T09:36:47.605655+0000","last_active":"2024-09-16T09:38:14.353320+0000","last_peered":"2024-09-16T09:38:14.353320+0000","last_clean":"2024-09-16T09:38:14.353320+0000","last_became_active":"2024-09-16T09:25:07.722017+0000","last_became_peered":"2024-09-16T09:25:07.722017+0000","last_unstale":"2024-09-16T09:38:14.353320+0000","last_undegraded":"2024-09-16T09:38:14.353320+0000","last_fullsized":"2024-09-16T09:38:14.353320+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:47.605615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:36:47.605615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.c","version":"389'3014","reported_seq":3050,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:17.890710+0000","last_change":"2024-09-16T09:37:45.401117+0000","last_active":"2024-09-16T09:38:17.890710+0000","last_peered":"2024-09-16T09:38:17.890710+0000","last_clean":"2024-09-16T09:38:17.890710+0000","last_became_active":"2024-09-16T09:37:45.400778+0000","last_became_peered":"2024-09-16T09:37:45.400778+0000","last_unstale":"2024-09-16T09:38:17.890710+0000","last_undegraded":"2024-09-16T09:38:17.890710+0000","last_fullsized":"2024-09-16T09:38:17.890710+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_clean_scrub_stamp":"2024-09-16T09:37:44.378497+0000","objects_scrubbed":0,"log_size":3014,"log_dups_size":0,"ondisk_log_size":3014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:49.375813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":197525504,"num_objects":3014,"num_object_clones":0,"num_object_copies":9042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3014,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3014,"num_write_kb":192896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"73'79","reported_seq":758,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.341267+0000","last_change":"2024-09-16T09:34:51.864131+0000","last_active":"2024-09-16T09:38:14.341267+0000","last_peered":"2024-09-16T09:38:14.341267+0000","last_clean":"2024-09-16T09:38:14.341267+0000","last_became_active":"2024-09-16T09:31:09.274707+0000","last_became_peered":"2024-09-16T09:31:09.274707+0000","last_unstale":"2024-09-16T09:38:14.341267+0000","last_undegraded":"2024-09-16T09:38:14.341267+0000","last_fullsized":"2024-09-16T09:38:14.341267+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:34:51.864036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:34:51.864036+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"285'20","reported_seq":383,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.337460+0000","last_change":"2024-09-16T09:33:41.368102+0000","last_active":"2024-09-16T09:38:14.337460+0000","last_peered":"2024-09-16T09:38:14.337460+0000","last_clean":"2024-09-16T09:38:14.337460+0000","last_became_active":"2024-09-16T09:33:30.248453+0000","last_became_peered":"2024-09-16T09:33:30.248453+0000","last_unstale":"2024-09-16T09:38:14.337460+0000","last_undegraded":"2024-09-16T09:38:14.337460+0000","last_fullsized":"2024-09-16T09:38:14.337460+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-16T09:33:41.368016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:33:41.368016+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":715,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.338124+0000","last_change":"2024-09-16T09:36:47.095703+0000","last_active":"2024-09-16T09:38:14.338124+0000","last_peered":"2024-09-16T09:38:14.338124+0000","last_clean":"2024-09-16T09:38:14.338124+0000","last_became_active":"2024-09-16T09:31:09.270619+0000","last_became_peered":"2024-09-16T09:31:09.270619+0000","last_unstale":"2024-09-16T09:38:14.338124+0000","last_undegraded":"2024-09-16T09:38:14.338124+0000","last_fullsized":"2024-09-16T09:38:14.338124+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:47.095658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:36:47.095658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"66'8","reported_seq":862,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.349043+0000","last_change":"2024-09-16T09:36:52.689167+0000","last_active":"2024-09-16T09:38:14.349043+0000","last_peered":"2024-09-16T09:38:14.349043+0000","last_clean":"2024-09-16T09:38:14.349043+0000","last_became_active":"2024-09-16T09:25:05.714507+0000","last_became_peered":"2024-09-16T09:25:05.714507+0000","last_unstale":"2024-09-16T09:38:14.349043+0000","last_undegraded":"2024-09-16T09:38:14.349043+0000","last_fullsized":"2024-09-16T09:38:14.349043+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:36:52.689091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:36:52.689091+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":704,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.348140+0000","last_change":"2024-09-16T09:32:33.362233+0000","last_active":"2024-09-16T09:38:14.348140+0000","last_peered":"2024-09-16T09:38:14.348140+0000","last_clean":"2024-09-16T09:38:14.348140+0000","last_became_active":"2024-09-16T09:31:09.283192+0000","last_became_peered":"2024-09-16T09:31:09.283192+0000","last_unstale":"2024-09-16T09:38:14.348140+0000","last_undegraded":"2024-09-16T09:38:14.348140+0000","last_fullsized":"2024-09-16T09:38:14.348140+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:33.362188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:32:33.362188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.d","version":"389'2991","reported_seq":3028,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:17.708448+0000","last_change":"2024-09-16T09:37:45.405475+0000","last_active":"2024-09-16T09:38:17.708448+0000","last_peered":"2024-09-16T09:38:17.708448+0000","last_clean":"2024-09-16T09:38:17.708448+0000","last_became_active":"2024-09-16T09:37:45.405149+0000","last_became_peered":"2024-09-16T09:37:45.405149+0000","last_unstale":"2024-09-16T09:38:17.708448+0000","last_undegraded":"2024-09-16T09:38:17.708448+0000","last_fullsized":"2024-09-16T09:38:17.708448+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_clean_scrub_stamp":"2024-09-16T09:37:44.378497+0000","objects_scrubbed":0,"log_size":2991,"log_dups_size":0,"ondisk_log_size":2991,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:48.823494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":196018176,"num_objects":2991,"num_object_clones":0,"num_object_copies":8973,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2991,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2991,"num_write_kb":191424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"73'79","reported_seq":731,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.341050+0000","last_change":"2024-09-16T09:34:42.728232+0000","last_active":"2024-09-16T09:38:14.341050+0000","last_peered":"2024-09-16T09:38:14.341050+0000","last_clean":"2024-09-16T09:38:14.341050+0000","last_became_active":"2024-09-16T09:31:09.274565+0000","last_became_peered":"2024-09-16T09:31:09.274565+0000","last_unstale":"2024-09-16T09:38:14.341050+0000","last_undegraded":"2024-09-16T09:38:14.341050+0000","last_fullsized":"2024-09-16T09:38:14.341050+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:34:42.728143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:34:42.728143+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:35.968055+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"285'29","reported_seq":749,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.349385+0000","last_change":"2024-09-16T09:36:54.604573+0000","last_active":"2024-09-16T09:38:14.349385+0000","last_peered":"2024-09-16T09:38:14.349385+0000","last_clean":"2024-09-16T09:38:14.349385+0000","last_became_active":"2024-09-16T09:25:03.714797+0000","last_became_peered":"2024-09-16T09:25:03.714797+0000","last_unstale":"2024-09-16T09:38:14.349385+0000","last_undegraded":"2024-09-16T09:38:14.349385+0000","last_fullsized":"2024-09-16T09:38:14.349385+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'29","last_scrub_stamp":"2024-09-16T09:36:54.604484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:36:54.604484+0000","objects_scrubbed":8,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:21.529216+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":54,"num_read_kb":40,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":325,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.337311+0000","last_change":"2024-09-16T09:33:30.247311+0000","last_active":"2024-09-16T09:38:14.337311+0000","last_peered":"2024-09-16T09:38:14.337311+0000","last_clean":"2024-09-16T09:38:14.337311+0000","last_became_active":"2024-09-16T09:33:30.246996+0000","last_became_peered":"2024-09-16T09:33:30.246996+0000","last_unstale":"2024-09-16T09:38:14.337311+0000","last_undegraded":"2024-09-16T09:38:14.337311+0000","last_fullsized":"2024-09-16T09:38:14.337311+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:11.407616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:11.407616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,2],"acting":[5,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":710,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.341914+0000","last_change":"2024-09-16T09:34:45.772748+0000","last_active":"2024-09-16T09:38:14.341914+0000","last_peered":"2024-09-16T09:38:14.341914+0000","last_clean":"2024-09-16T09:38:14.341914+0000","last_became_active":"2024-09-16T09:25:05.708663+0000","last_became_peered":"2024-09-16T09:25:05.708663+0000","last_unstale":"2024-09-16T09:38:14.341914+0000","last_undegraded":"2024-09-16T09:38:14.341914+0000","last_fullsized":"2024-09-16T09:38:14.341914+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:45.772674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:34:45.772674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":341,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.341544+0000","last_change":"2024-09-16T09:37:56.882475+0000","last_active":"2024-09-16T09:38:14.341544+0000","last_peered":"2024-09-16T09:38:14.341544+0000","last_clean":"2024-09-16T09:38:14.341544+0000","last_became_active":"2024-09-16T09:33:30.250178+0000","last_became_peered":"2024-09-16T09:33:30.250178+0000","last_unstale":"2024-09-16T09:38:14.341544+0000","last_undegraded":"2024-09-16T09:38:14.341544+0000","last_fullsized":"2024-09-16T09:38:14.341544+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:56.882326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:37:56.882326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:02.807091+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,2],"acting":[6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.a","version":"389'3087","reported_seq":3124,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:17.734263+0000","last_change":"2024-09-16T09:37:45.404280+0000","last_active":"2024-09-16T09:38:17.734263+0000","last_peered":"2024-09-16T09:38:17.734263+0000","last_clean":"2024-09-16T09:38:17.734263+0000","last_became_active":"2024-09-16T09:37:45.404021+0000","last_became_peered":"2024-09-16T09:37:45.404021+0000","last_unstale":"2024-09-16T09:38:17.734263+0000","last_undegraded":"2024-09-16T09:38:17.734263+0000","last_fullsized":"2024-09-16T09:38:17.734263+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_clean_scrub_stamp":"2024-09-16T09:37:44.378497+0000","objects_scrubbed":0,"log_size":3087,"log_dups_size":0,"ondisk_log_size":3087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:08.182634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":202309632,"num_objects":3087,"num_object_clones":0,"num_object_copies":9261,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3087,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3087,"num_write_kb":197568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"73'79","reported_seq":664,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.336792+0000","last_change":"2024-09-16T09:36:42.191578+0000","last_active":"2024-09-16T09:38:14.336792+0000","last_peered":"2024-09-16T09:38:14.336792+0000","last_clean":"2024-09-16T09:38:14.336792+0000","last_became_active":"2024-09-16T09:30:08.766350+0000","last_became_peered":"2024-09-16T09:30:08.766350+0000","last_unstale":"2024-09-16T09:38:14.336792+0000","last_undegraded":"2024-09-16T09:38:14.336792+0000","last_fullsized":"2024-09-16T09:38:14.336792+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:36:42.191521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:36:42.191521+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"285'42","reported_seq":1005,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.339415+0000","last_change":"2024-09-16T09:37:57.407386+0000","last_active":"2024-09-16T09:38:14.339415+0000","last_peered":"2024-09-16T09:38:14.339415+0000","last_clean":"2024-09-16T09:38:14.339415+0000","last_became_active":"2024-09-16T09:31:09.278839+0000","last_became_peered":"2024-09-16T09:31:09.278839+0000","last_unstale":"2024-09-16T09:38:14.339415+0000","last_undegraded":"2024-09-16T09:38:14.339415+0000","last_fullsized":"2024-09-16T09:38:14.339415+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'42","last_scrub_stamp":"2024-09-16T09:37:57.407297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:37:57.407297+0000","objects_scrubbed":10,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:23.815599+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":66,"num_read_kb":51,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":710,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.338844+0000","last_change":"2024-09-16T09:36:35.691463+0000","last_active":"2024-09-16T09:38:14.338844+0000","last_peered":"2024-09-16T09:38:14.338844+0000","last_clean":"2024-09-16T09:38:14.338844+0000","last_became_active":"2024-09-16T09:25:01.703076+0000","last_became_peered":"2024-09-16T09:25:01.703076+0000","last_unstale":"2024-09-16T09:38:14.338844+0000","last_undegraded":"2024-09-16T09:38:14.338844+0000","last_fullsized":"2024-09-16T09:38:14.338844+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:35.691415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:36:35.691415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":711,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.337740+0000","last_change":"2024-09-16T09:33:38.417375+0000","last_active":"2024-09-16T09:38:14.337740+0000","last_peered":"2024-09-16T09:38:14.337740+0000","last_clean":"2024-09-16T09:38:14.337740+0000","last_became_active":"2024-09-16T09:25:00.696237+0000","last_became_peered":"2024-09-16T09:25:00.696237+0000","last_unstale":"2024-09-16T09:38:14.337740+0000","last_undegraded":"2024-09-16T09:38:14.337740+0000","last_fullsized":"2024-09-16T09:38:14.337740+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-16T09:33:38.417308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:33:38.417308+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.0025137430000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":678,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.352561+0000","last_change":"2024-09-16T09:33:16.666704+0000","last_active":"2024-09-16T09:38:14.352561+0000","last_peered":"2024-09-16T09:38:14.352561+0000","last_clean":"2024-09-16T09:38:14.352561+0000","last_became_active":"2024-09-16T09:25:05.717591+0000","last_became_peered":"2024-09-16T09:25:05.717591+0000","last_unstale":"2024-09-16T09:38:14.352561+0000","last_undegraded":"2024-09-16T09:38:14.352561+0000","last_fullsized":"2024-09-16T09:38:14.352561+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:16.666648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:16.666648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:38.618334+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":427,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.343208+0000","last_change":"2024-09-16T09:36:50.667709+0000","last_active":"2024-09-16T09:38:14.343208+0000","last_peered":"2024-09-16T09:38:14.343208+0000","last_clean":"2024-09-16T09:38:14.343208+0000","last_became_active":"2024-09-16T09:31:09.272609+0000","last_became_peered":"2024-09-16T09:31:09.272609+0000","last_unstale":"2024-09-16T09:38:14.343208+0000","last_undegraded":"2024-09-16T09:38:14.343208+0000","last_fullsized":"2024-09-16T09:38:14.343208+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:50.667661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:36:50.667661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"389'3124","reported_seq":3161,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:18.234186+0000","last_change":"2024-09-16T09:37:45.463232+0000","last_active":"2024-09-16T09:38:18.234186+0000","last_peered":"2024-09-16T09:38:18.234186+0000","last_clean":"2024-09-16T09:38:18.234186+0000","last_became_active":"2024-09-16T09:37:45.462868+0000","last_became_peered":"2024-09-16T09:37:45.462868+0000","last_unstale":"2024-09-16T09:38:18.234186+0000","last_undegraded":"2024-09-16T09:38:18.234186+0000","last_fullsized":"2024-09-16T09:38:18.234186+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_clean_scrub_stamp":"2024-09-16T09:37:44.378497+0000","objects_scrubbed":0,"log_size":3124,"log_dups_size":0,"ondisk_log_size":3124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:55.563655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":204734464,"num_objects":3124,"num_object_clones":0,"num_object_copies":9372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3124,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3124,"num_write_kb":199936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"73'79","reported_seq":16,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.353092+0000","last_change":"2024-09-16T09:38:10.556681+0000","last_active":"2024-09-16T09:38:14.353092+0000","last_peered":"2024-09-16T09:38:14.353092+0000","last_clean":"2024-09-16T09:38:14.353092+0000","last_became_active":"2024-09-16T09:38:10.556320+0000","last_became_peered":"2024-09-16T09:38:10.556320+0000","last_unstale":"2024-09-16T09:38:14.353092+0000","last_undegraded":"2024-09-16T09:38:14.353092+0000","last_fullsized":"2024-09-16T09:38:14.353092+0000","mapping_epoch":386,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":387,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:34:32.717915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:34:32.717915+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"285'17","reported_seq":424,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.337766+0000","last_change":"2024-09-16T09:37:49.246464+0000","last_active":"2024-09-16T09:38:14.337766+0000","last_peered":"2024-09-16T09:38:14.337766+0000","last_clean":"2024-09-16T09:38:14.337766+0000","last_became_active":"2024-09-16T09:37:33.349009+0000","last_became_peered":"2024-09-16T09:37:33.349009+0000","last_unstale":"2024-09-16T09:38:14.337766+0000","last_undegraded":"2024-09-16T09:38:14.337766+0000","last_fullsized":"2024-09-16T09:38:14.337766+0000","mapping_epoch":367,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":368,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'17","last_scrub_stamp":"2024-09-16T09:37:49.246379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:37:49.246379+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:56.883815+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":35,"num_read_kb":24,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":333,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.341739+0000","last_change":"2024-09-16T09:37:21.946685+0000","last_active":"2024-09-16T09:38:14.341739+0000","last_peered":"2024-09-16T09:38:14.341739+0000","last_clean":"2024-09-16T09:38:14.341739+0000","last_became_active":"2024-09-16T09:33:30.247596+0000","last_became_peered":"2024-09-16T09:33:30.247596+0000","last_unstale":"2024-09-16T09:38:14.341739+0000","last_undegraded":"2024-09-16T09:38:14.341739+0000","last_fullsized":"2024-09-16T09:38:14.341739+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:21.946641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:37:21.946641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:25.048262+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,2],"acting":[6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":360,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.348589+0000","last_change":"2024-09-16T09:37:04.362941+0000","last_active":"2024-09-16T09:38:14.348589+0000","last_peered":"2024-09-16T09:38:14.348589+0000","last_clean":"2024-09-16T09:38:14.348589+0000","last_became_active":"2024-09-16T09:37:04.358985+0000","last_became_peered":"2024-09-16T09:37:04.358985+0000","last_unstale":"2024-09-16T09:38:14.348589+0000","last_undegraded":"2024-09-16T09:38:14.348589+0000","last_fullsized":"2024-09-16T09:38:14.348589+0000","mapping_epoch":343,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":344,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:32:40.165749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:32:40.165749+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":718,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.344099+0000","last_change":"2024-09-16T09:36:55.530225+0000","last_active":"2024-09-16T09:38:14.344099+0000","last_peered":"2024-09-16T09:38:14.344099+0000","last_clean":"2024-09-16T09:38:14.344099+0000","last_became_active":"2024-09-16T09:25:05.711539+0000","last_became_peered":"2024-09-16T09:25:05.711539+0000","last_unstale":"2024-09-16T09:38:14.344099+0000","last_undegraded":"2024-09-16T09:38:14.344099+0000","last_fullsized":"2024-09-16T09:38:14.344099+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:55.530175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:36:55.530175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:18.377901+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":696,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.355099+0000","last_change":"2024-09-16T09:37:45.450074+0000","last_active":"2024-09-16T09:38:14.355099+0000","last_peered":"2024-09-16T09:38:14.355099+0000","last_clean":"2024-09-16T09:38:14.355099+0000","last_became_active":"2024-09-16T09:25:07.723282+0000","last_became_peered":"2024-09-16T09:25:07.723282+0000","last_unstale":"2024-09-16T09:38:14.355099+0000","last_undegraded":"2024-09-16T09:38:14.355099+0000","last_fullsized":"2024-09-16T09:38:14.355099+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:45.450022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:37:45.450022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:15.116366+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"389'3142","reported_seq":3179,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:19.222476+0000","last_change":"2024-09-16T09:37:45.464193+0000","last_active":"2024-09-16T09:38:19.222476+0000","last_peered":"2024-09-16T09:38:19.222476+0000","last_clean":"2024-09-16T09:38:19.222476+0000","last_became_active":"2024-09-16T09:37:45.462754+0000","last_became_peered":"2024-09-16T09:37:45.462754+0000","last_unstale":"2024-09-16T09:38:19.222476+0000","last_undegraded":"2024-09-16T09:38:19.222476+0000","last_fullsized":"2024-09-16T09:38:19.222476+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_clean_scrub_stamp":"2024-09-16T09:37:44.378497+0000","objects_scrubbed":0,"log_size":3142,"log_dups_size":0,"ondisk_log_size":3142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:46.604408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":205914112,"num_objects":3142,"num_object_clones":0,"num_object_copies":9426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3142,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3142,"num_write_kb":201088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,7],"acting":[0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"73'79","reported_seq":414,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.341381+0000","last_change":"2024-09-16T09:34:31.929462+0000","last_active":"2024-09-16T09:38:14.341381+0000","last_peered":"2024-09-16T09:38:14.341381+0000","last_clean":"2024-09-16T09:38:14.341381+0000","last_became_active":"2024-09-16T09:33:30.249309+0000","last_became_peered":"2024-09-16T09:33:30.249309+0000","last_unstale":"2024-09-16T09:38:14.341381+0000","last_undegraded":"2024-09-16T09:38:14.341381+0000","last_fullsized":"2024-09-16T09:38:14.341381+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:34:31.929388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:34:31.929388+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:34.931244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.6","version":"285'24","reported_seq":726,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.351303+0000","last_change":"2024-09-16T09:32:48.547237+0000","last_active":"2024-09-16T09:38:14.351303+0000","last_peered":"2024-09-16T09:38:14.351303+0000","last_clean":"2024-09-16T09:38:14.351303+0000","last_became_active":"2024-09-16T09:25:03.715315+0000","last_became_peered":"2024-09-16T09:25:03.715315+0000","last_unstale":"2024-09-16T09:38:14.351303+0000","last_undegraded":"2024-09-16T09:38:14.351303+0000","last_fullsized":"2024-09-16T09:38:14.351303+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-16T09:32:48.546869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:32:48.546869+0000","objects_scrubbed":6,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:32.692965+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":36,"num_read_kb":30,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"59'1","reported_seq":725,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.348319+0000","last_change":"2024-09-16T09:33:21.682893+0000","last_active":"2024-09-16T09:38:14.348319+0000","last_peered":"2024-09-16T09:38:14.348319+0000","last_clean":"2024-09-16T09:38:14.348319+0000","last_became_active":"2024-09-16T09:31:09.269627+0000","last_became_peered":"2024-09-16T09:31:09.269627+0000","last_unstale":"2024-09-16T09:38:14.348319+0000","last_undegraded":"2024-09-16T09:38:14.348319+0000","last_fullsized":"2024-09-16T09:38:14.348319+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:33:21.682840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:21.682840+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:42.553527+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1986,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"67'5","reported_seq":1501,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:16.885603+0000","last_change":"2024-09-16T09:33:37.358839+0000","last_active":"2024-09-16T09:38:16.885603+0000","last_peered":"2024-09-16T09:38:16.885603+0000","last_clean":"2024-09-16T09:38:16.885603+0000","last_became_active":"2024-09-16T09:25:00.696546+0000","last_became_peered":"2024-09-16T09:25:00.696546+0000","last_unstale":"2024-09-16T09:38:16.885603+0000","last_undegraded":"2024-09-16T09:38:16.885603+0000","last_fullsized":"2024-09-16T09:38:16.885603+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'5","last_scrub_stamp":"2024-09-16T09:33:37.358741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:33:37.358741+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:33.496609+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0.0059770420000000001,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":790,"num_read_kb":785,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":677,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.337870+0000","last_change":"2024-09-16T09:27:07.337170+0000","last_active":"2024-09-16T09:38:14.337870+0000","last_peered":"2024-09-16T09:38:14.337870+0000","last_clean":"2024-09-16T09:38:14.337870+0000","last_became_active":"2024-09-16T09:25:05.714704+0000","last_became_peered":"2024-09-16T09:25:05.714704+0000","last_unstale":"2024-09-16T09:38:14.337870+0000","last_undegraded":"2024-09-16T09:38:14.337870+0000","last_fullsized":"2024-09-16T09:38:14.337870+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:07.337124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:07.337124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:38.140085+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":603,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.337533+0000","last_change":"2024-09-16T09:36:43.150813+0000","last_active":"2024-09-16T09:38:14.337533+0000","last_peered":"2024-09-16T09:38:14.337533+0000","last_clean":"2024-09-16T09:38:14.337533+0000","last_became_active":"2024-09-16T09:33:30.245852+0000","last_became_peered":"2024-09-16T09:33:30.245852+0000","last_unstale":"2024-09-16T09:38:14.337533+0000","last_undegraded":"2024-09-16T09:38:14.337533+0000","last_fullsized":"2024-09-16T09:38:14.337533+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:43.150764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:36:43.150764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"389'3026","reported_seq":3063,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:17.727093+0000","last_change":"2024-09-16T09:37:45.406191+0000","last_active":"2024-09-16T09:38:17.727093+0000","last_peered":"2024-09-16T09:38:17.727093+0000","last_clean":"2024-09-16T09:38:17.727093+0000","last_became_active":"2024-09-16T09:37:45.405552+0000","last_became_peered":"2024-09-16T09:37:45.405552+0000","last_unstale":"2024-09-16T09:38:17.727093+0000","last_undegraded":"2024-09-16T09:38:17.727093+0000","last_fullsized":"2024-09-16T09:38:17.727093+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_clean_scrub_stamp":"2024-09-16T09:37:44.378497+0000","objects_scrubbed":0,"log_size":3026,"log_dups_size":0,"ondisk_log_size":3026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:55.714372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":198311936,"num_objects":3026,"num_object_clones":0,"num_object_copies":9078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3026,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3026,"num_write_kb":193664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"285'28","reported_seq":906,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.341265+0000","last_change":"2024-09-16T09:33:16.891850+0000","last_active":"2024-09-16T09:38:14.341265+0000","last_peered":"2024-09-16T09:38:14.341265+0000","last_clean":"2024-09-16T09:38:14.341265+0000","last_became_active":"2024-09-16T09:25:03.714118+0000","last_became_peered":"2024-09-16T09:25:03.714118+0000","last_unstale":"2024-09-16T09:38:14.341265+0000","last_undegraded":"2024-09-16T09:38:14.341265+0000","last_fullsized":"2024-09-16T09:38:14.341265+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'16","last_scrub_stamp":"2024-09-16T09:33:16.891761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:33:16.891761+0000","objects_scrubbed":8,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:12.972702+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":43,"num_read_kb":33,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":551,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.337644+0000","last_change":"2024-09-16T09:36:50.114910+0000","last_active":"2024-09-16T09:38:14.337644+0000","last_peered":"2024-09-16T09:38:14.337644+0000","last_clean":"2024-09-16T09:38:14.337644+0000","last_became_active":"2024-09-16T09:31:09.270196+0000","last_became_peered":"2024-09-16T09:31:09.270196+0000","last_unstale":"2024-09-16T09:38:14.337644+0000","last_undegraded":"2024-09-16T09:38:14.337644+0000","last_fullsized":"2024-09-16T09:38:14.337644+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:50.114861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:36:50.114861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"73'79","reported_seq":780,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.353683+0000","last_change":"2024-09-16T09:32:43.535041+0000","last_active":"2024-09-16T09:38:14.353683+0000","last_peered":"2024-09-16T09:38:14.353683+0000","last_clean":"2024-09-16T09:38:14.353683+0000","last_became_active":"2024-09-16T09:30:08.870394+0000","last_became_peered":"2024-09-16T09:30:08.870394+0000","last_unstale":"2024-09-16T09:38:14.353683+0000","last_undegraded":"2024-09-16T09:38:14.353683+0000","last_fullsized":"2024-09-16T09:38:14.353683+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:32:43.534962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:32:43.534962+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":692,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.353850+0000","last_change":"2024-09-16T09:37:52.517698+0000","last_active":"2024-09-16T09:38:14.353850+0000","last_peered":"2024-09-16T09:38:14.353850+0000","last_clean":"2024-09-16T09:38:14.353850+0000","last_became_active":"2024-09-16T09:25:05.714341+0000","last_became_peered":"2024-09-16T09:25:05.714341+0000","last_unstale":"2024-09-16T09:38:14.353850+0000","last_undegraded":"2024-09-16T09:38:14.353850+0000","last_fullsized":"2024-09-16T09:38:14.353850+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:52.517652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:37:52.517652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:20.774965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":689,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.337664+0000","last_change":"2024-09-16T09:37:50.218655+0000","last_active":"2024-09-16T09:38:14.337664+0000","last_peered":"2024-09-16T09:38:14.337664+0000","last_clean":"2024-09-16T09:38:14.337664+0000","last_became_active":"2024-09-16T09:25:07.714281+0000","last_became_peered":"2024-09-16T09:25:07.714281+0000","last_unstale":"2024-09-16T09:38:14.337664+0000","last_undegraded":"2024-09-16T09:38:14.337664+0000","last_fullsized":"2024-09-16T09:38:14.337664+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:50.218594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:37:50.218594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:20.077058+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.0","version":"389'2935","reported_seq":2972,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:17.729568+0000","last_change":"2024-09-16T09:37:45.407072+0000","last_active":"2024-09-16T09:38:17.729568+0000","last_peered":"2024-09-16T09:38:17.729568+0000","last_clean":"2024-09-16T09:38:17.729568+0000","last_became_active":"2024-09-16T09:37:45.406736+0000","last_became_peered":"2024-09-16T09:37:45.406736+0000","last_unstale":"2024-09-16T09:38:17.729568+0000","last_undegraded":"2024-09-16T09:38:17.729568+0000","last_fullsized":"2024-09-16T09:38:17.729568+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_clean_scrub_stamp":"2024-09-16T09:37:44.378497+0000","objects_scrubbed":0,"log_size":2935,"log_dups_size":0,"ondisk_log_size":2935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:03.181256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":192348160,"num_objects":2935,"num_object_clones":0,"num_object_copies":8805,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2935,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2935,"num_write_kb":187840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.b","version":"73'79","reported_seq":517,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.348731+0000","last_change":"2024-09-16T09:34:31.720971+0000","last_active":"2024-09-16T09:38:14.348731+0000","last_peered":"2024-09-16T09:38:14.348731+0000","last_clean":"2024-09-16T09:38:14.348731+0000","last_became_active":"2024-09-16T09:33:30.251933+0000","last_became_peered":"2024-09-16T09:33:30.251933+0000","last_unstale":"2024-09-16T09:38:14.348731+0000","last_undegraded":"2024-09-16T09:38:14.348731+0000","last_fullsized":"2024-09-16T09:38:14.348731+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:34:31.720787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:34:31.720787+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"285'19","reported_seq":777,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.339401+0000","last_change":"2024-09-16T09:37:33.355562+0000","last_active":"2024-09-16T09:38:14.339401+0000","last_peered":"2024-09-16T09:38:14.339401+0000","last_clean":"2024-09-16T09:38:14.339401+0000","last_became_active":"2024-09-16T09:37:33.348433+0000","last_became_peered":"2024-09-16T09:37:33.348433+0000","last_unstale":"2024-09-16T09:38:14.339401+0000","last_undegraded":"2024-09-16T09:38:14.339401+0000","last_fullsized":"2024-09-16T09:38:14.339401+0000","mapping_epoch":367,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":368,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'19","last_scrub_stamp":"2024-09-16T09:35:27.813206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:35:27.813206+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:13.318653+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":694,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.339041+0000","last_change":"2024-09-16T09:37:18.392256+0000","last_active":"2024-09-16T09:38:14.339041+0000","last_peered":"2024-09-16T09:38:14.339041+0000","last_clean":"2024-09-16T09:38:14.339041+0000","last_became_active":"2024-09-16T09:25:01.708270+0000","last_became_peered":"2024-09-16T09:25:01.708270+0000","last_unstale":"2024-09-16T09:38:14.339041+0000","last_undegraded":"2024-09-16T09:38:14.339041+0000","last_fullsized":"2024-09-16T09:38:14.339041+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:18.392084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:37:18.392084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:33.733932+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":676,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.337551+0000","last_change":"2024-09-16T09:26:34.171190+0000","last_active":"2024-09-16T09:38:14.337551+0000","last_peered":"2024-09-16T09:38:14.337551+0000","last_clean":"2024-09-16T09:38:14.337551+0000","last_became_active":"2024-09-16T09:25:05.711964+0000","last_became_peered":"2024-09-16T09:25:05.711964+0000","last_unstale":"2024-09-16T09:38:14.337551+0000","last_undegraded":"2024-09-16T09:38:14.337551+0000","last_fullsized":"2024-09-16T09:38:14.337551+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:34.171139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:26:34.171139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:15.084115+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":568,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.342035+0000","last_change":"2024-09-16T09:34:50.866121+0000","last_active":"2024-09-16T09:38:14.342035+0000","last_peered":"2024-09-16T09:38:14.342035+0000","last_clean":"2024-09-16T09:38:14.342035+0000","last_became_active":"2024-09-16T09:31:09.278618+0000","last_became_peered":"2024-09-16T09:31:09.278618+0000","last_unstale":"2024-09-16T09:38:14.342035+0000","last_undegraded":"2024-09-16T09:38:14.342035+0000","last_fullsized":"2024-09-16T09:38:14.342035+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:50.866032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:34:50.866032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:09.987998+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3","version":"389'3152","reported_seq":3189,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:18.195590+0000","last_change":"2024-09-16T09:37:45.463216+0000","last_active":"2024-09-16T09:38:18.195590+0000","last_peered":"2024-09-16T09:38:18.195590+0000","last_clean":"2024-09-16T09:38:18.195590+0000","last_became_active":"2024-09-16T09:37:45.462708+0000","last_became_peered":"2024-09-16T09:37:45.462708+0000","last_unstale":"2024-09-16T09:38:18.195590+0000","last_undegraded":"2024-09-16T09:38:18.195590+0000","last_fullsized":"2024-09-16T09:38:18.195590+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_clean_scrub_stamp":"2024-09-16T09:37:44.378497+0000","objects_scrubbed":0,"log_size":3152,"log_dups_size":0,"ondisk_log_size":3152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:05.295932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":206569472,"num_objects":3152,"num_object_clones":0,"num_object_copies":9456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3152,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3152,"num_write_kb":201728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.8","version":"73'79","reported_seq":36,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.336665+0000","last_change":"2024-09-16T09:37:54.183161+0000","last_active":"2024-09-16T09:38:14.336665+0000","last_peered":"2024-09-16T09:38:14.336665+0000","last_clean":"2024-09-16T09:38:14.336665+0000","last_became_active":"2024-09-16T09:37:53.418691+0000","last_became_peered":"2024-09-16T09:37:53.418691+0000","last_unstale":"2024-09-16T09:38:14.336665+0000","last_undegraded":"2024-09-16T09:38:14.336665+0000","last_fullsized":"2024-09-16T09:38:14.336665+0000","mapping_epoch":382,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":383,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:37:54.183107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:37:54.183107+0000","objects_scrubbed":1,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:09.320645+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262688,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"285'31","reported_seq":754,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.339239+0000","last_change":"2024-09-16T09:35:28.861520+0000","last_active":"2024-09-16T09:38:14.339239+0000","last_peered":"2024-09-16T09:38:14.339239+0000","last_clean":"2024-09-16T09:38:14.339239+0000","last_became_active":"2024-09-16T09:25:03.713358+0000","last_became_peered":"2024-09-16T09:25:03.713358+0000","last_unstale":"2024-09-16T09:38:14.339239+0000","last_undegraded":"2024-09-16T09:38:14.339239+0000","last_fullsized":"2024-09-16T09:38:14.339239+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'31","last_scrub_stamp":"2024-09-16T09:35:28.861433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:35:28.861433+0000","objects_scrubbed":8,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:08.335954+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":50,"num_read_kb":40,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":702,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.342043+0000","last_change":"2024-09-16T09:37:52.902619+0000","last_active":"2024-09-16T09:38:14.342043+0000","last_peered":"2024-09-16T09:38:14.342043+0000","last_clean":"2024-09-16T09:38:14.342043+0000","last_became_active":"2024-09-16T09:25:01.703355+0000","last_became_peered":"2024-09-16T09:25:01.703355+0000","last_unstale":"2024-09-16T09:38:14.342043+0000","last_undegraded":"2024-09-16T09:38:14.342043+0000","last_fullsized":"2024-09-16T09:38:14.342043+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:52.902556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:37:52.902556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:04.411673+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":341,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.338964+0000","last_change":"2024-09-16T09:37:46.378016+0000","last_active":"2024-09-16T09:38:14.338964+0000","last_peered":"2024-09-16T09:38:14.338964+0000","last_clean":"2024-09-16T09:38:14.338964+0000","last_became_active":"2024-09-16T09:33:30.251186+0000","last_became_peered":"2024-09-16T09:33:30.251186+0000","last_unstale":"2024-09-16T09:38:14.338964+0000","last_undegraded":"2024-09-16T09:38:14.338964+0000","last_fullsized":"2024-09-16T09:38:14.338964+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:46.377951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:37:46.377951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:54.699212+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,0],"acting":[4,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":631,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.343666+0000","last_change":"2024-09-16T09:36:51.640537+0000","last_active":"2024-09-16T09:38:14.343666+0000","last_peered":"2024-09-16T09:38:14.343666+0000","last_clean":"2024-09-16T09:38:14.343666+0000","last_became_active":"2024-09-16T09:33:21.337291+0000","last_became_peered":"2024-09-16T09:33:21.337291+0000","last_unstale":"2024-09-16T09:38:14.343666+0000","last_undegraded":"2024-09-16T09:38:14.343666+0000","last_fullsized":"2024-09-16T09:38:14.343666+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:51.640472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:36:51.640472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4],"acting":[3,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"389'3070","reported_seq":3107,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:17.966579+0000","last_change":"2024-09-16T09:37:45.402477+0000","last_active":"2024-09-16T09:38:17.966579+0000","last_peered":"2024-09-16T09:38:17.966579+0000","last_clean":"2024-09-16T09:38:17.966579+0000","last_became_active":"2024-09-16T09:37:45.401847+0000","last_became_peered":"2024-09-16T09:37:45.401847+0000","last_unstale":"2024-09-16T09:38:17.966579+0000","last_undegraded":"2024-09-16T09:38:17.966579+0000","last_fullsized":"2024-09-16T09:38:17.966579+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_clean_scrub_stamp":"2024-09-16T09:37:44.378497+0000","objects_scrubbed":0,"log_size":3070,"log_dups_size":0,"ondisk_log_size":3070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:05.829127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":201195520,"num_objects":3070,"num_object_clones":0,"num_object_copies":9210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3070,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3070,"num_write_kb":196480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,0],"acting":[4,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.9","version":"73'79","reported_seq":756,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.353243+0000","last_change":"2024-09-16T09:29:19.013948+0000","last_active":"2024-09-16T09:38:14.353243+0000","last_peered":"2024-09-16T09:38:14.353243+0000","last_clean":"2024-09-16T09:38:14.353243+0000","last_became_active":"2024-09-16T09:26:35.126066+0000","last_became_peered":"2024-09-16T09:26:35.126066+0000","last_unstale":"2024-09-16T09:38:14.353243+0000","last_undegraded":"2024-09-16T09:38:14.353243+0000","last_fullsized":"2024-09-16T09:38:14.353243+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:19.013865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:19.013865+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:10.452624+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"285'20","reported_seq":711,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.339368+0000","last_change":"2024-09-16T09:33:24.154567+0000","last_active":"2024-09-16T09:38:14.339368+0000","last_peered":"2024-09-16T09:38:14.339368+0000","last_clean":"2024-09-16T09:38:14.339368+0000","last_became_active":"2024-09-16T09:25:03.707723+0000","last_became_peered":"2024-09-16T09:25:03.707723+0000","last_unstale":"2024-09-16T09:38:14.339368+0000","last_undegraded":"2024-09-16T09:38:14.339368+0000","last_fullsized":"2024-09-16T09:38:14.339368+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-16T09:33:24.154507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:33:24.154507+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:14.304180+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":559,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.349560+0000","last_change":"2024-09-16T09:33:10.528181+0000","last_active":"2024-09-16T09:38:14.349560+0000","last_peered":"2024-09-16T09:38:14.349560+0000","last_clean":"2024-09-16T09:38:14.349560+0000","last_became_active":"2024-09-16T09:31:09.288825+0000","last_became_peered":"2024-09-16T09:31:09.288825+0000","last_unstale":"2024-09-16T09:38:14.349560+0000","last_undegraded":"2024-09-16T09:38:14.349560+0000","last_fullsized":"2024-09-16T09:38:14.349560+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:10.528115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:10.528115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:34.688091+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"228'9","reported_seq":433,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.355246+0000","last_change":"2024-09-16T09:37:16.515947+0000","last_active":"2024-09-16T09:38:14.355246+0000","last_peered":"2024-09-16T09:38:14.355246+0000","last_clean":"2024-09-16T09:38:14.355246+0000","last_became_active":"2024-09-16T09:33:21.337213+0000","last_became_peered":"2024-09-16T09:33:21.337213+0000","last_unstale":"2024-09-16T09:38:14.355246+0000","last_undegraded":"2024-09-16T09:38:14.355246+0000","last_fullsized":"2024-09-16T09:38:14.355246+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"228'9","last_scrub_stamp":"2024-09-16T09:37:16.514010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:37:16.514010+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:17.192669+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":705,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.339296+0000","last_change":"2024-09-16T09:35:36.061013+0000","last_active":"2024-09-16T09:38:14.339296+0000","last_peered":"2024-09-16T09:38:14.339296+0000","last_clean":"2024-09-16T09:38:14.339296+0000","last_became_active":"2024-09-16T09:25:07.718958+0000","last_became_peered":"2024-09-16T09:25:07.718958+0000","last_unstale":"2024-09-16T09:38:14.339296+0000","last_undegraded":"2024-09-16T09:38:14.339296+0000","last_fullsized":"2024-09-16T09:38:14.339296+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:35:36.060942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:35:36.060942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.5","version":"389'3164","reported_seq":3200,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:19.217812+0000","last_change":"2024-09-16T09:37:45.463538+0000","last_active":"2024-09-16T09:38:19.217812+0000","last_peered":"2024-09-16T09:38:19.217812+0000","last_clean":"2024-09-16T09:38:19.217812+0000","last_became_active":"2024-09-16T09:37:45.462778+0000","last_became_peered":"2024-09-16T09:37:45.462778+0000","last_unstale":"2024-09-16T09:38:19.217812+0000","last_undegraded":"2024-09-16T09:38:19.217812+0000","last_fullsized":"2024-09-16T09:38:19.217812+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_clean_scrub_stamp":"2024-09-16T09:37:44.378497+0000","objects_scrubbed":0,"log_size":3164,"log_dups_size":0,"ondisk_log_size":3164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:09.642344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":207355904,"num_objects":3164,"num_object_clones":0,"num_object_copies":9492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3164,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3164,"num_write_kb":202496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.e","version":"73'79","reported_seq":498,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.339207+0000","last_change":"2024-09-16T09:35:41.979877+0000","last_active":"2024-09-16T09:38:14.339207+0000","last_peered":"2024-09-16T09:38:14.339207+0000","last_clean":"2024-09-16T09:38:14.339207+0000","last_became_active":"2024-09-16T09:30:10.705072+0000","last_became_peered":"2024-09-16T09:30:10.705072+0000","last_unstale":"2024-09-16T09:38:14.339207+0000","last_undegraded":"2024-09-16T09:38:14.339207+0000","last_fullsized":"2024-09-16T09:38:14.339207+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:35:41.979814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:35:41.979814+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,3],"acting":[4,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.b","version":"285'15","reported_seq":747,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.349248+0000","last_change":"2024-09-16T09:36:55.651561+0000","last_active":"2024-09-16T09:38:14.349248+0000","last_peered":"2024-09-16T09:38:14.349248+0000","last_clean":"2024-09-16T09:38:14.349248+0000","last_became_active":"2024-09-16T09:36:31.590202+0000","last_became_peered":"2024-09-16T09:36:31.590202+0000","last_unstale":"2024-09-16T09:38:14.349248+0000","last_undegraded":"2024-09-16T09:38:14.349248+0000","last_fullsized":"2024-09-16T09:38:14.349248+0000","mapping_epoch":316,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":317,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'15","last_scrub_stamp":"2024-09-16T09:36:55.651496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:36:55.651496+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":706,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.337626+0000","last_change":"2024-09-16T09:37:31.033631+0000","last_active":"2024-09-16T09:38:14.337626+0000","last_peered":"2024-09-16T09:38:14.337626+0000","last_clean":"2024-09-16T09:38:14.337626+0000","last_became_active":"2024-09-16T09:31:09.276516+0000","last_became_peered":"2024-09-16T09:31:09.276516+0000","last_unstale":"2024-09-16T09:38:14.337626+0000","last_undegraded":"2024-09-16T09:38:14.337626+0000","last_fullsized":"2024-09-16T09:38:14.337626+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:31.033577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:37:31.033577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:49.512309+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":627,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.338488+0000","last_change":"2024-09-16T09:33:40.260547+0000","last_active":"2024-09-16T09:38:14.338488+0000","last_peered":"2024-09-16T09:38:14.338488+0000","last_clean":"2024-09-16T09:38:14.338488+0000","last_became_active":"2024-09-16T09:33:21.337254+0000","last_became_peered":"2024-09-16T09:33:21.337254+0000","last_unstale":"2024-09-16T09:38:14.338488+0000","last_undegraded":"2024-09-16T09:38:14.338488+0000","last_fullsized":"2024-09-16T09:38:14.338488+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:40.260500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:40.260500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:39.339396+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3],"acting":[4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":698,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.348237+0000","last_change":"2024-09-16T09:36:53.647305+0000","last_active":"2024-09-16T09:38:14.348237+0000","last_peered":"2024-09-16T09:38:14.348237+0000","last_clean":"2024-09-16T09:38:14.348237+0000","last_became_active":"2024-09-16T09:25:07.718778+0000","last_became_peered":"2024-09-16T09:25:07.718778+0000","last_unstale":"2024-09-16T09:38:14.348237+0000","last_undegraded":"2024-09-16T09:38:14.348237+0000","last_fullsized":"2024-09-16T09:38:14.348237+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:53.647256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:36:53.647256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:19.369534+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"389'3222","reported_seq":3259,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:19.188996+0000","last_change":"2024-09-16T09:37:45.400129+0000","last_active":"2024-09-16T09:38:19.188996+0000","last_peered":"2024-09-16T09:38:19.188996+0000","last_clean":"2024-09-16T09:38:19.188996+0000","last_became_active":"2024-09-16T09:37:45.399844+0000","last_became_peered":"2024-09-16T09:37:45.399844+0000","last_unstale":"2024-09-16T09:38:19.188996+0000","last_undegraded":"2024-09-16T09:38:19.188996+0000","last_fullsized":"2024-09-16T09:38:19.188996+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_clean_scrub_stamp":"2024-09-16T09:37:44.378497+0000","objects_scrubbed":0,"log_size":3222,"log_dups_size":0,"ondisk_log_size":3222,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:51.633388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":211156992,"num_objects":3222,"num_object_clones":0,"num_object_copies":9666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3222,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3222,"num_write_kb":206208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.f","version":"73'79","reported_seq":750,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.341574+0000","last_change":"2024-09-16T09:34:53.866426+0000","last_active":"2024-09-16T09:38:14.341574+0000","last_peered":"2024-09-16T09:38:14.341574+0000","last_clean":"2024-09-16T09:38:14.341574+0000","last_became_active":"2024-09-16T09:30:10.693477+0000","last_became_peered":"2024-09-16T09:30:10.693477+0000","last_unstale":"2024-09-16T09:38:14.341574+0000","last_undegraded":"2024-09-16T09:38:14.341574+0000","last_fullsized":"2024-09-16T09:38:14.341574+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:34:53.866337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:34:53.866337+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:59.878482+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"285'35","reported_seq":761,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.341938+0000","last_change":"2024-09-16T09:37:28.078195+0000","last_active":"2024-09-16T09:38:14.341938+0000","last_peered":"2024-09-16T09:38:14.341938+0000","last_clean":"2024-09-16T09:38:14.341938+0000","last_became_active":"2024-09-16T09:25:03.712783+0000","last_became_peered":"2024-09-16T09:25:03.712783+0000","last_unstale":"2024-09-16T09:38:14.341938+0000","last_undegraded":"2024-09-16T09:38:14.341938+0000","last_fullsized":"2024-09-16T09:38:14.341938+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'35","last_scrub_stamp":"2024-09-16T09:37:28.077841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:37:28.077841+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:52.547070+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":56,"num_read_kb":45,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"59'1","reported_seq":713,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.355571+0000","last_change":"2024-09-16T09:37:56.528872+0000","last_active":"2024-09-16T09:38:14.355571+0000","last_peered":"2024-09-16T09:38:14.355571+0000","last_clean":"2024-09-16T09:38:14.355571+0000","last_became_active":"2024-09-16T09:25:01.708069+0000","last_became_peered":"2024-09-16T09:25:01.708069+0000","last_unstale":"2024-09-16T09:38:14.355571+0000","last_undegraded":"2024-09-16T09:38:14.355571+0000","last_fullsized":"2024-09-16T09:38:14.355571+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:37:56.528796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:37:56.528796+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:57.114353+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":715,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.344163+0000","last_change":"2024-09-16T09:37:53.518635+0000","last_active":"2024-09-16T09:38:14.344163+0000","last_peered":"2024-09-16T09:38:14.344163+0000","last_clean":"2024-09-16T09:38:14.344163+0000","last_became_active":"2024-09-16T09:37:53.518284+0000","last_became_peered":"2024-09-16T09:37:53.518284+0000","last_unstale":"2024-09-16T09:38:14.344163+0000","last_undegraded":"2024-09-16T09:38:14.344163+0000","last_fullsized":"2024-09-16T09:38:14.344163+0000","mapping_epoch":382,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":383,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:59.543358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:36:59.543358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:24.211506+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":680,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.353644+0000","last_change":"2024-09-16T09:36:48.649739+0000","last_active":"2024-09-16T09:38:14.353644+0000","last_peered":"2024-09-16T09:38:14.353644+0000","last_clean":"2024-09-16T09:38:14.353644+0000","last_became_active":"2024-09-16T09:25:07.718750+0000","last_became_peered":"2024-09-16T09:25:07.718750+0000","last_unstale":"2024-09-16T09:38:14.353644+0000","last_undegraded":"2024-09-16T09:38:14.353644+0000","last_fullsized":"2024-09-16T09:38:14.353644+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:48.649696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:36:48.649696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:18.071944+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.7","version":"389'3058","reported_seq":3095,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:17.750860+0000","last_change":"2024-09-16T09:37:45.404688+0000","last_active":"2024-09-16T09:38:17.750860+0000","last_peered":"2024-09-16T09:38:17.750860+0000","last_clean":"2024-09-16T09:38:17.750860+0000","last_became_active":"2024-09-16T09:37:45.404354+0000","last_became_peered":"2024-09-16T09:37:45.404354+0000","last_unstale":"2024-09-16T09:38:17.750860+0000","last_undegraded":"2024-09-16T09:38:17.750860+0000","last_fullsized":"2024-09-16T09:38:17.750860+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_clean_scrub_stamp":"2024-09-16T09:37:44.378497+0000","objects_scrubbed":0,"log_size":3058,"log_dups_size":0,"ondisk_log_size":3058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:13.592270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":200409088,"num_objects":3058,"num_object_clones":0,"num_object_copies":9174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3058,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3058,"num_write_kb":195712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.c","version":"73'79","reported_seq":439,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.341548+0000","last_change":"2024-09-16T09:36:53.697797+0000","last_active":"2024-09-16T09:38:14.341548+0000","last_peered":"2024-09-16T09:38:14.341548+0000","last_clean":"2024-09-16T09:38:14.341548+0000","last_became_active":"2024-09-16T09:31:09.273770+0000","last_became_peered":"2024-09-16T09:31:09.273770+0000","last_unstale":"2024-09-16T09:38:14.341548+0000","last_undegraded":"2024-09-16T09:38:14.341548+0000","last_fullsized":"2024-09-16T09:38:14.341548+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:36:53.697742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:36:53.697742+0000","objects_scrubbed":1,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:00.926871+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":327680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"285'18","reported_seq":776,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.338884+0000","last_change":"2024-09-16T09:36:50.603724+0000","last_active":"2024-09-16T09:38:14.338884+0000","last_peered":"2024-09-16T09:38:14.338884+0000","last_clean":"2024-09-16T09:38:14.338884+0000","last_became_active":"2024-09-16T09:31:09.269047+0000","last_became_peered":"2024-09-16T09:31:09.269047+0000","last_unstale":"2024-09-16T09:38:14.338884+0000","last_undegraded":"2024-09-16T09:38:14.338884+0000","last_fullsized":"2024-09-16T09:38:14.338884+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'18","last_scrub_stamp":"2024-09-16T09:36:50.603660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:36:50.603660+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:15.970176+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":707,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.354703+0000","last_change":"2024-09-16T09:37:57.529616+0000","last_active":"2024-09-16T09:38:14.354703+0000","last_peered":"2024-09-16T09:38:14.354703+0000","last_clean":"2024-09-16T09:38:14.354703+0000","last_became_active":"2024-09-16T09:25:01.707528+0000","last_became_peered":"2024-09-16T09:25:01.707528+0000","last_unstale":"2024-09-16T09:38:14.354703+0000","last_undegraded":"2024-09-16T09:38:14.354703+0000","last_fullsized":"2024-09-16T09:38:14.354703+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:57.529554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:37:57.529554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:05.467113+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":353,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.347914+0000","last_change":"2024-09-16T09:37:57.449473+0000","last_active":"2024-09-16T09:38:14.347914+0000","last_peered":"2024-09-16T09:38:14.347914+0000","last_clean":"2024-09-16T09:38:14.347914+0000","last_became_active":"2024-09-16T09:33:21.337309+0000","last_became_peered":"2024-09-16T09:33:21.337309+0000","last_unstale":"2024-09-16T09:38:14.347914+0000","last_undegraded":"2024-09-16T09:38:14.347914+0000","last_fullsized":"2024-09-16T09:38:14.347914+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:57.449105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:37:57.449105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:12.289276+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":689,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.344746+0000","last_change":"2024-09-16T09:36:59.713546+0000","last_active":"2024-09-16T09:38:14.344746+0000","last_peered":"2024-09-16T09:38:14.344746+0000","last_clean":"2024-09-16T09:38:14.344746+0000","last_became_active":"2024-09-16T09:25:07.718899+0000","last_became_peered":"2024-09-16T09:25:07.718899+0000","last_unstale":"2024-09-16T09:38:14.344746+0000","last_undegraded":"2024-09-16T09:38:14.344746+0000","last_fullsized":"2024-09-16T09:38:14.344746+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:59.713355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:36:59.713355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.6","version":"389'2984","reported_seq":3021,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:18.230426+0000","last_change":"2024-09-16T09:37:45.404587+0000","last_active":"2024-09-16T09:38:18.230426+0000","last_peered":"2024-09-16T09:38:18.230426+0000","last_clean":"2024-09-16T09:38:18.230426+0000","last_became_active":"2024-09-16T09:37:45.404263+0000","last_became_peered":"2024-09-16T09:37:45.404263+0000","last_unstale":"2024-09-16T09:38:18.230426+0000","last_undegraded":"2024-09-16T09:38:18.230426+0000","last_fullsized":"2024-09-16T09:38:18.230426+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_clean_scrub_stamp":"2024-09-16T09:37:44.378497+0000","objects_scrubbed":0,"log_size":2984,"log_dups_size":0,"ondisk_log_size":2984,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:58.611017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":195559424,"num_objects":2984,"num_object_clones":0,"num_object_copies":8952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2984,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2984,"num_write_kb":190976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.d","version":"73'79","reported_seq":489,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.353734+0000","last_change":"2024-09-16T09:32:44.536868+0000","last_active":"2024-09-16T09:38:14.353734+0000","last_peered":"2024-09-16T09:38:14.353734+0000","last_clean":"2024-09-16T09:38:14.353734+0000","last_became_active":"2024-09-16T09:30:10.709958+0000","last_became_peered":"2024-09-16T09:30:10.709958+0000","last_unstale":"2024-09-16T09:38:14.353734+0000","last_undegraded":"2024-09-16T09:38:14.353734+0000","last_fullsized":"2024-09-16T09:38:14.353734+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:32:44.536799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:32:44.536799+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:14.489819+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"285'27","reported_seq":732,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.336775+0000","last_change":"2024-09-16T09:37:30.081518+0000","last_active":"2024-09-16T09:38:14.336775+0000","last_peered":"2024-09-16T09:38:14.336775+0000","last_clean":"2024-09-16T09:38:14.336775+0000","last_became_active":"2024-09-16T09:25:03.715733+0000","last_became_peered":"2024-09-16T09:25:03.715733+0000","last_unstale":"2024-09-16T09:38:14.336775+0000","last_undegraded":"2024-09-16T09:38:14.336775+0000","last_fullsized":"2024-09-16T09:38:14.336775+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'27","last_scrub_stamp":"2024-09-16T09:37:30.081447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:37:30.081447+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:45.797304+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":711,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.355136+0000","last_change":"2024-09-16T09:37:17.544132+0000","last_active":"2024-09-16T09:38:14.355136+0000","last_peered":"2024-09-16T09:38:14.355136+0000","last_clean":"2024-09-16T09:38:14.355136+0000","last_became_active":"2024-09-16T09:25:01.707372+0000","last_became_peered":"2024-09-16T09:25:01.707372+0000","last_unstale":"2024-09-16T09:38:14.355136+0000","last_undegraded":"2024-09-16T09:38:14.355136+0000","last_fullsized":"2024-09-16T09:38:14.355136+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-16T09:37:17.544069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:37:17.544069+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:23.672102+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"66'8","reported_seq":860,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.353264+0000","last_change":"2024-09-16T09:30:05.138362+0000","last_active":"2024-09-16T09:38:14.353264+0000","last_peered":"2024-09-16T09:38:14.353264+0000","last_clean":"2024-09-16T09:38:14.353264+0000","last_became_active":"2024-09-16T09:25:05.716189+0000","last_became_peered":"2024-09-16T09:25:05.716189+0000","last_unstale":"2024-09-16T09:38:14.353264+0000","last_undegraded":"2024-09-16T09:38:14.353264+0000","last_fullsized":"2024-09-16T09:38:14.353264+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:30:05.138288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:30:05.138288+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:14.462129+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":681,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.337085+0000","last_change":"2024-09-16T09:37:47.244099+0000","last_active":"2024-09-16T09:38:14.337085+0000","last_peered":"2024-09-16T09:38:14.337085+0000","last_clean":"2024-09-16T09:38:14.337085+0000","last_became_active":"2024-09-16T09:25:07.717213+0000","last_became_peered":"2024-09-16T09:25:07.717213+0000","last_unstale":"2024-09-16T09:38:14.337085+0000","last_undegraded":"2024-09-16T09:38:14.337085+0000","last_fullsized":"2024-09-16T09:38:14.337085+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:47.244032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:37:47.244032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:48.590206+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.12","version":"73'79","reported_seq":498,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.337320+0000","last_change":"2024-09-16T09:36:44.171217+0000","last_active":"2024-09-16T09:38:14.337320+0000","last_peered":"2024-09-16T09:38:14.337320+0000","last_clean":"2024-09-16T09:38:14.337320+0000","last_became_active":"2024-09-16T09:30:10.731413+0000","last_became_peered":"2024-09-16T09:30:10.731413+0000","last_unstale":"2024-09-16T09:38:14.337320+0000","last_undegraded":"2024-09-16T09:38:14.337320+0000","last_fullsized":"2024-09-16T09:38:14.337320+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:36:44.171151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:36:44.171151+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":715,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.341653+0000","last_change":"2024-09-16T09:37:55.863661+0000","last_active":"2024-09-16T09:38:14.341653+0000","last_peered":"2024-09-16T09:38:14.341653+0000","last_clean":"2024-09-16T09:38:14.341653+0000","last_became_active":"2024-09-16T09:37:53.518354+0000","last_became_peered":"2024-09-16T09:37:53.518354+0000","last_unstale":"2024-09-16T09:38:14.341653+0000","last_undegraded":"2024-09-16T09:38:14.341653+0000","last_fullsized":"2024-09-16T09:38:14.341653+0000","mapping_epoch":382,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":383,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:55.863613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:37:55.863613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:01.033155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"285'12","reported_seq":459,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.344089+0000","last_change":"2024-09-16T09:37:20.749151+0000","last_active":"2024-09-16T09:38:14.344089+0000","last_peered":"2024-09-16T09:38:14.344089+0000","last_clean":"2024-09-16T09:38:14.344089+0000","last_became_active":"2024-09-16T09:31:09.270685+0000","last_became_peered":"2024-09-16T09:31:09.270685+0000","last_unstale":"2024-09-16T09:38:14.344089+0000","last_undegraded":"2024-09-16T09:38:14.344089+0000","last_fullsized":"2024-09-16T09:38:14.344089+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'12","last_scrub_stamp":"2024-09-16T09:37:20.749101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:37:20.749101+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:32.592645+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":17,"num_read_kb":14,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":692,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.337358+0000","last_change":"2024-09-16T09:36:38.129449+0000","last_active":"2024-09-16T09:38:14.337358+0000","last_peered":"2024-09-16T09:38:14.337358+0000","last_clean":"2024-09-16T09:38:14.337358+0000","last_became_active":"2024-09-16T09:25:05.715616+0000","last_became_peered":"2024-09-16T09:25:05.715616+0000","last_unstale":"2024-09-16T09:38:14.337358+0000","last_undegraded":"2024-09-16T09:38:14.337358+0000","last_fullsized":"2024-09-16T09:38:14.337358+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:38.129314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:36:38.129314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":679,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.355015+0000","last_change":"2024-09-16T09:36:37.715166+0000","last_active":"2024-09-16T09:38:14.355015+0000","last_peered":"2024-09-16T09:38:14.355015+0000","last_clean":"2024-09-16T09:38:14.355015+0000","last_became_active":"2024-09-16T09:25:07.721059+0000","last_became_peered":"2024-09-16T09:25:07.721059+0000","last_unstale":"2024-09-16T09:38:14.355015+0000","last_undegraded":"2024-09-16T09:38:14.355015+0000","last_fullsized":"2024-09-16T09:38:14.355015+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:37.715114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:36:37.715114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.13","version":"73'79","reported_seq":584,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.344931+0000","last_change":"2024-09-16T09:34:48.260121+0000","last_active":"2024-09-16T09:38:14.344931+0000","last_peered":"2024-09-16T09:38:14.344931+0000","last_clean":"2024-09-16T09:38:14.344931+0000","last_became_active":"2024-09-16T09:33:30.248497+0000","last_became_peered":"2024-09-16T09:33:30.248497+0000","last_unstale":"2024-09-16T09:38:14.344931+0000","last_undegraded":"2024-09-16T09:38:14.344931+0000","last_fullsized":"2024-09-16T09:38:14.344931+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:34:48.260041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:34:48.260041+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:11.022422+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"285'15","reported_seq":772,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.351425+0000","last_change":"2024-09-16T09:33:48.681742+0000","last_active":"2024-09-16T09:38:14.351425+0000","last_peered":"2024-09-16T09:38:14.351425+0000","last_clean":"2024-09-16T09:38:14.351425+0000","last_became_active":"2024-09-16T09:31:09.273498+0000","last_became_peered":"2024-09-16T09:31:09.273498+0000","last_unstale":"2024-09-16T09:38:14.351425+0000","last_undegraded":"2024-09-16T09:38:14.351425+0000","last_fullsized":"2024-09-16T09:38:14.351425+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:33:48.681650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:33:48.681650+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:59.423578+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":699,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.353513+0000","last_change":"2024-09-16T09:33:13.312434+0000","last_active":"2024-09-16T09:38:14.353513+0000","last_peered":"2024-09-16T09:38:14.353513+0000","last_clean":"2024-09-16T09:38:14.353513+0000","last_became_active":"2024-09-16T09:25:01.706491+0000","last_became_peered":"2024-09-16T09:25:01.706491+0000","last_unstale":"2024-09-16T09:38:14.353513+0000","last_undegraded":"2024-09-16T09:38:14.353513+0000","last_fullsized":"2024-09-16T09:38:14.353513+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:13.312344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:13.312344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:37.450175+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":456,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.354805+0000","last_change":"2024-09-16T09:36:54.596000+0000","last_active":"2024-09-16T09:38:14.354805+0000","last_peered":"2024-09-16T09:38:14.354805+0000","last_clean":"2024-09-16T09:38:14.354805+0000","last_became_active":"2024-09-16T09:33:30.248992+0000","last_became_peered":"2024-09-16T09:33:30.248992+0000","last_unstale":"2024-09-16T09:38:14.354805+0000","last_undegraded":"2024-09-16T09:38:14.354805+0000","last_fullsized":"2024-09-16T09:38:14.354805+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:54.595924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:36:54.595924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":697,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.343713+0000","last_change":"2024-09-16T09:36:58.536167+0000","last_active":"2024-09-16T09:38:14.343713+0000","last_peered":"2024-09-16T09:38:14.343713+0000","last_clean":"2024-09-16T09:38:14.343713+0000","last_became_active":"2024-09-16T09:25:07.722050+0000","last_became_peered":"2024-09-16T09:25:07.722050+0000","last_unstale":"2024-09-16T09:38:14.343713+0000","last_undegraded":"2024-09-16T09:38:14.343713+0000","last_fullsized":"2024-09-16T09:38:14.343713+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:58.536121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:36:58.536121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:18.611763+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.10","version":"73'79","reported_seq":498,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.336609+0000","last_change":"2024-09-16T09:36:41.169117+0000","last_active":"2024-09-16T09:38:14.336609+0000","last_peered":"2024-09-16T09:38:14.336609+0000","last_clean":"2024-09-16T09:38:14.336609+0000","last_became_active":"2024-09-16T09:30:10.692770+0000","last_became_peered":"2024-09-16T09:30:10.692770+0000","last_unstale":"2024-09-16T09:38:14.336609+0000","last_undegraded":"2024-09-16T09:38:14.336609+0000","last_fullsized":"2024-09-16T09:38:14.336609+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:36:41.169069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:36:41.169069+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"285'15","reported_seq":721,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.338062+0000","last_change":"2024-09-16T09:37:55.203919+0000","last_active":"2024-09-16T09:38:14.338062+0000","last_peered":"2024-09-16T09:38:14.338062+0000","last_clean":"2024-09-16T09:38:14.338062+0000","last_became_active":"2024-09-16T09:25:03.717310+0000","last_became_peered":"2024-09-16T09:25:03.717310+0000","last_unstale":"2024-09-16T09:38:14.338062+0000","last_undegraded":"2024-09-16T09:38:14.338062+0000","last_fullsized":"2024-09-16T09:38:14.338062+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'15","last_scrub_stamp":"2024-09-16T09:37:55.203856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:37:55.203856+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:23.098469+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":731,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.351209+0000","last_change":"2024-09-16T09:37:50.471532+0000","last_active":"2024-09-16T09:38:14.351209+0000","last_peered":"2024-09-16T09:38:14.351209+0000","last_clean":"2024-09-16T09:38:14.351209+0000","last_became_active":"2024-09-16T09:31:09.272848+0000","last_became_peered":"2024-09-16T09:31:09.272848+0000","last_unstale":"2024-09-16T09:38:14.351209+0000","last_undegraded":"2024-09-16T09:38:14.351209+0000","last_fullsized":"2024-09-16T09:38:14.351209+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:50.471486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:37:50.471486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:09.546185+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"187'11","reported_seq":705,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.353872+0000","last_change":"2024-09-16T09:37:19.540982+0000","last_active":"2024-09-16T09:38:14.353872+0000","last_peered":"2024-09-16T09:38:14.353872+0000","last_clean":"2024-09-16T09:38:14.353872+0000","last_became_active":"2024-09-16T09:31:09.277688+0000","last_became_peered":"2024-09-16T09:31:09.277688+0000","last_unstale":"2024-09-16T09:38:14.353872+0000","last_undegraded":"2024-09-16T09:38:14.353872+0000","last_fullsized":"2024-09-16T09:38:14.353872+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"187'11","last_scrub_stamp":"2024-09-16T09:37:19.540886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:37:19.540886+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:47.772515+0000","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":705,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.339573+0000","last_change":"2024-09-16T09:36:49.628523+0000","last_active":"2024-09-16T09:38:14.339573+0000","last_peered":"2024-09-16T09:38:14.339573+0000","last_clean":"2024-09-16T09:38:14.339573+0000","last_became_active":"2024-09-16T09:25:07.718810+0000","last_became_peered":"2024-09-16T09:25:07.718810+0000","last_unstale":"2024-09-16T09:38:14.339573+0000","last_undegraded":"2024-09-16T09:38:14.339573+0000","last_fullsized":"2024-09-16T09:38:14.339573+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:49.628477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:36:49.628477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.11","version":"73'79","reported_seq":584,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.348418+0000","last_change":"2024-09-16T09:37:15.623104+0000","last_active":"2024-09-16T09:38:14.348418+0000","last_peered":"2024-09-16T09:38:14.348418+0000","last_clean":"2024-09-16T09:38:14.348418+0000","last_became_active":"2024-09-16T09:31:09.272208+0000","last_became_peered":"2024-09-16T09:31:09.272208+0000","last_unstale":"2024-09-16T09:38:14.348418+0000","last_undegraded":"2024-09-16T09:38:14.348418+0000","last_fullsized":"2024-09-16T09:38:14.348418+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:37:15.623041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:37:15.623041+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:16.219169+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.14","version":"285'20","reported_seq":382,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.354193+0000","last_change":"2024-09-16T09:36:52.609515+0000","last_active":"2024-09-16T09:38:14.354193+0000","last_peered":"2024-09-16T09:38:14.354193+0000","last_clean":"2024-09-16T09:38:14.354193+0000","last_became_active":"2024-09-16T09:33:30.249769+0000","last_became_peered":"2024-09-16T09:33:30.249769+0000","last_unstale":"2024-09-16T09:38:14.354193+0000","last_undegraded":"2024-09-16T09:38:14.354193+0000","last_fullsized":"2024-09-16T09:38:14.354193+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'20","last_scrub_stamp":"2024-09-16T09:36:52.609440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:36:52.609440+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":691,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.353552+0000","last_change":"2024-09-16T09:29:56.021413+0000","last_active":"2024-09-16T09:38:14.353552+0000","last_peered":"2024-09-16T09:38:14.353552+0000","last_clean":"2024-09-16T09:38:14.353552+0000","last_became_active":"2024-09-16T09:25:01.707513+0000","last_became_peered":"2024-09-16T09:25:01.707513+0000","last_unstale":"2024-09-16T09:38:14.353552+0000","last_undegraded":"2024-09-16T09:38:14.353552+0000","last_fullsized":"2024-09-16T09:38:14.353552+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:56.021340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:56.021340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:13.464027+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"66'8","reported_seq":868,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.337872+0000","last_change":"2024-09-16T09:33:39.388721+0000","last_active":"2024-09-16T09:38:14.337872+0000","last_peered":"2024-09-16T09:38:14.337872+0000","last_clean":"2024-09-16T09:38:14.337872+0000","last_became_active":"2024-09-16T09:25:05.713833+0000","last_became_peered":"2024-09-16T09:25:05.713833+0000","last_unstale":"2024-09-16T09:38:14.337872+0000","last_undegraded":"2024-09-16T09:38:14.337872+0000","last_fullsized":"2024-09-16T09:38:14.337872+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:33:39.388463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:39.388463+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":711,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.350558+0000","last_change":"2024-09-16T09:33:42.666290+0000","last_active":"2024-09-16T09:38:14.350558+0000","last_peered":"2024-09-16T09:38:14.350558+0000","last_clean":"2024-09-16T09:38:14.350558+0000","last_became_active":"2024-09-16T09:31:09.275168+0000","last_became_peered":"2024-09-16T09:31:09.275168+0000","last_unstale":"2024-09-16T09:38:14.350558+0000","last_undegraded":"2024-09-16T09:38:14.350558+0000","last_fullsized":"2024-09-16T09:38:14.350558+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:42.666200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:33:42.666200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"285'19","reported_seq":739,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.338642+0000","last_change":"2024-09-16T09:37:31.362985+0000","last_active":"2024-09-16T09:38:14.338642+0000","last_peered":"2024-09-16T09:38:14.338642+0000","last_clean":"2024-09-16T09:38:14.338642+0000","last_became_active":"2024-09-16T09:26:33.291265+0000","last_became_peered":"2024-09-16T09:26:33.291265+0000","last_unstale":"2024-09-16T09:38:14.338642+0000","last_undegraded":"2024-09-16T09:38:14.338642+0000","last_fullsized":"2024-09-16T09:38:14.338642+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'19","last_scrub_stamp":"2024-09-16T09:37:31.362927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:37:31.362927+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:37.215450+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":717,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.339755+0000","last_change":"2024-09-16T09:37:53.239593+0000","last_active":"2024-09-16T09:38:14.339755+0000","last_peered":"2024-09-16T09:38:14.339755+0000","last_clean":"2024-09-16T09:38:14.339755+0000","last_became_active":"2024-09-16T09:25:01.705158+0000","last_became_peered":"2024-09-16T09:25:01.705158+0000","last_unstale":"2024-09-16T09:38:14.339755+0000","last_undegraded":"2024-09-16T09:38:14.339755+0000","last_fullsized":"2024-09-16T09:38:14.339755+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:53.239547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:37:53.239547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:02.480253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":637,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.337170+0000","last_change":"2024-09-16T09:37:58.252056+0000","last_active":"2024-09-16T09:38:14.337170+0000","last_peered":"2024-09-16T09:38:14.337170+0000","last_clean":"2024-09-16T09:38:14.337170+0000","last_became_active":"2024-09-16T09:37:41.515278+0000","last_became_peered":"2024-09-16T09:37:41.515278+0000","last_unstale":"2024-09-16T09:38:14.337170+0000","last_undegraded":"2024-09-16T09:38:14.337170+0000","last_fullsized":"2024-09-16T09:38:14.337170+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:58.251994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:37:58.251994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:15.408415+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":544,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.348143+0000","last_change":"2024-09-16T09:34:48.786847+0000","last_active":"2024-09-16T09:38:14.348143+0000","last_peered":"2024-09-16T09:38:14.348143+0000","last_clean":"2024-09-16T09:38:14.348143+0000","last_became_active":"2024-09-16T09:31:09.276042+0000","last_became_peered":"2024-09-16T09:31:09.276042+0000","last_unstale":"2024-09-16T09:38:14.348143+0000","last_undegraded":"2024-09-16T09:38:14.348143+0000","last_fullsized":"2024-09-16T09:38:14.348143+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:48.786785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:34:48.786785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"285'15","reported_seq":412,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.341996+0000","last_change":"2024-09-16T09:37:33.354025+0000","last_active":"2024-09-16T09:38:14.341996+0000","last_peered":"2024-09-16T09:38:14.341996+0000","last_clean":"2024-09-16T09:38:14.341996+0000","last_became_active":"2024-09-16T09:37:33.348223+0000","last_became_peered":"2024-09-16T09:37:33.348223+0000","last_unstale":"2024-09-16T09:38:14.341996+0000","last_undegraded":"2024-09-16T09:38:14.341996+0000","last_fullsized":"2024-09-16T09:38:14.341996+0000","mapping_epoch":367,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":368,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'15","last_scrub_stamp":"2024-09-16T09:36:55.935996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:36:55.935996+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,2],"acting":[6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":691,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.355413+0000","last_change":"2024-09-16T09:32:47.502667+0000","last_active":"2024-09-16T09:38:14.355413+0000","last_peered":"2024-09-16T09:38:14.355413+0000","last_clean":"2024-09-16T09:38:14.355413+0000","last_became_active":"2024-09-16T09:25:01.706762+0000","last_became_peered":"2024-09-16T09:25:01.706762+0000","last_unstale":"2024-09-16T09:38:14.355413+0000","last_undegraded":"2024-09-16T09:38:14.355413+0000","last_fullsized":"2024-09-16T09:38:14.355413+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:47.502598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:32:47.502598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":529,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.348863+0000","last_change":"2024-09-16T09:34:46.742640+0000","last_active":"2024-09-16T09:38:14.348863+0000","last_peered":"2024-09-16T09:38:14.348863+0000","last_clean":"2024-09-16T09:38:14.348863+0000","last_became_active":"2024-09-16T09:28:52.982865+0000","last_became_peered":"2024-09-16T09:28:52.982865+0000","last_unstale":"2024-09-16T09:38:14.348863+0000","last_undegraded":"2024-09-16T09:38:14.348863+0000","last_fullsized":"2024-09-16T09:38:14.348863+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:46.742562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:34:46.742562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":689,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.347804+0000","last_change":"2024-09-16T09:34:50.779241+0000","last_active":"2024-09-16T09:38:14.347804+0000","last_peered":"2024-09-16T09:38:14.347804+0000","last_clean":"2024-09-16T09:38:14.347804+0000","last_became_active":"2024-09-16T09:25:07.715301+0000","last_became_peered":"2024-09-16T09:25:07.715301+0000","last_unstale":"2024-09-16T09:38:14.347804+0000","last_undegraded":"2024-09-16T09:38:14.347804+0000","last_fullsized":"2024-09-16T09:38:14.347804+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:50.779164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:34:50.779164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.14","version":"73'79","reported_seq":498,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.343605+0000","last_change":"2024-09-16T09:36:48.597220+0000","last_active":"2024-09-16T09:38:14.343605+0000","last_peered":"2024-09-16T09:38:14.343605+0000","last_clean":"2024-09-16T09:38:14.343605+0000","last_became_active":"2024-09-16T09:30:10.700866+0000","last_became_peered":"2024-09-16T09:30:10.700866+0000","last_unstale":"2024-09-16T09:38:14.343605+0000","last_undegraded":"2024-09-16T09:38:14.343605+0000","last_fullsized":"2024-09-16T09:38:14.343605+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:36:48.597169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:36:48.597169+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:15.020958+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"285'19","reported_seq":579,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.353321+0000","last_change":"2024-09-16T09:28:55.643505+0000","last_active":"2024-09-16T09:38:14.353321+0000","last_peered":"2024-09-16T09:38:14.353321+0000","last_clean":"2024-09-16T09:38:14.353321+0000","last_became_active":"2024-09-16T09:28:52.971132+0000","last_became_peered":"2024-09-16T09:28:52.971132+0000","last_unstale":"2024-09-16T09:38:14.353321+0000","last_undegraded":"2024-09-16T09:38:14.353321+0000","last_fullsized":"2024-09-16T09:38:14.353321+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-16T09:28:55.643448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:55.643448+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:07.375359+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":700,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.337491+0000","last_change":"2024-09-16T09:33:18.297533+0000","last_active":"2024-09-16T09:38:14.337491+0000","last_peered":"2024-09-16T09:38:14.337491+0000","last_clean":"2024-09-16T09:38:14.337491+0000","last_became_active":"2024-09-16T09:25:01.700733+0000","last_became_peered":"2024-09-16T09:25:01.700733+0000","last_unstale":"2024-09-16T09:38:14.337491+0000","last_undegraded":"2024-09-16T09:38:14.337491+0000","last_fullsized":"2024-09-16T09:38:14.337491+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:18.297451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:18.297451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":691,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.352952+0000","last_change":"2024-09-16T09:33:38.336281+0000","last_active":"2024-09-16T09:38:14.352952+0000","last_peered":"2024-09-16T09:38:14.352952+0000","last_clean":"2024-09-16T09:38:14.352952+0000","last_became_active":"2024-09-16T09:25:05.713479+0000","last_became_peered":"2024-09-16T09:25:05.713479+0000","last_unstale":"2024-09-16T09:38:14.352952+0000","last_undegraded":"2024-09-16T09:38:14.352952+0000","last_fullsized":"2024-09-16T09:38:14.352952+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:38.336221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:38.336221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:36.450450+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":552,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.349473+0000","last_change":"2024-09-16T09:34:51.781047+0000","last_active":"2024-09-16T09:38:14.349473+0000","last_peered":"2024-09-16T09:38:14.349473+0000","last_clean":"2024-09-16T09:38:14.349473+0000","last_became_active":"2024-09-16T09:31:09.274797+0000","last_became_peered":"2024-09-16T09:31:09.274797+0000","last_unstale":"2024-09-16T09:38:14.349473+0000","last_undegraded":"2024-09-16T09:38:14.349473+0000","last_fullsized":"2024-09-16T09:38:14.349473+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:51.780980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:34:51.780980+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"285'8","reported_seq":455,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.342025+0000","last_change":"2024-09-16T09:37:15.672673+0000","last_active":"2024-09-16T09:38:14.342025+0000","last_peered":"2024-09-16T09:38:14.342025+0000","last_clean":"2024-09-16T09:38:14.342025+0000","last_became_active":"2024-09-16T09:31:09.273682+0000","last_became_peered":"2024-09-16T09:31:09.273682+0000","last_unstale":"2024-09-16T09:38:14.342025+0000","last_undegraded":"2024-09-16T09:38:14.342025+0000","last_fullsized":"2024-09-16T09:38:14.342025+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'8","last_scrub_stamp":"2024-09-16T09:37:15.672596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:37:15.672596+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:20.990359+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":9,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":723,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.349443+0000","last_change":"2024-09-16T09:33:18.641115+0000","last_active":"2024-09-16T09:38:14.349443+0000","last_peered":"2024-09-16T09:38:14.349443+0000","last_clean":"2024-09-16T09:38:14.349443+0000","last_became_active":"2024-09-16T09:31:09.274719+0000","last_became_peered":"2024-09-16T09:31:09.274719+0000","last_unstale":"2024-09-16T09:38:14.349443+0000","last_undegraded":"2024-09-16T09:38:14.349443+0000","last_fullsized":"2024-09-16T09:38:14.349443+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:18.641069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:18.641069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:44.551604+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":709,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.341632+0000","last_change":"2024-09-16T09:33:37.941485+0000","last_active":"2024-09-16T09:38:14.341632+0000","last_peered":"2024-09-16T09:38:14.341632+0000","last_clean":"2024-09-16T09:38:14.341632+0000","last_became_active":"2024-09-16T09:25:05.712545+0000","last_became_peered":"2024-09-16T09:25:05.712545+0000","last_unstale":"2024-09-16T09:38:14.341632+0000","last_undegraded":"2024-09-16T09:38:14.341632+0000","last_fullsized":"2024-09-16T09:38:14.341632+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:37.941403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:37.941403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:33.011670+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":695,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.353162+0000","last_change":"2024-09-16T09:37:50.502460+0000","last_active":"2024-09-16T09:38:14.353162+0000","last_peered":"2024-09-16T09:38:14.353162+0000","last_clean":"2024-09-16T09:38:14.353162+0000","last_became_active":"2024-09-16T09:25:07.720117+0000","last_became_peered":"2024-09-16T09:25:07.720117+0000","last_unstale":"2024-09-16T09:38:14.353162+0000","last_undegraded":"2024-09-16T09:38:14.353162+0000","last_fullsized":"2024-09-16T09:38:14.353162+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:50.502398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:37:50.502398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:13.329570+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":332,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.337925+0000","last_change":"2024-09-16T09:36:52.091335+0000","last_active":"2024-09-16T09:38:14.337925+0000","last_peered":"2024-09-16T09:38:14.337925+0000","last_clean":"2024-09-16T09:38:14.337925+0000","last_became_active":"2024-09-16T09:33:30.248036+0000","last_became_peered":"2024-09-16T09:33:30.248036+0000","last_unstale":"2024-09-16T09:38:14.337925+0000","last_undegraded":"2024-09-16T09:38:14.337925+0000","last_fullsized":"2024-09-16T09:38:14.337925+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:52.088284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:36:52.088284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":558,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.348434+0000","last_change":"2024-09-16T09:37:18.825035+0000","last_active":"2024-09-16T09:38:14.348434+0000","last_peered":"2024-09-16T09:38:14.348434+0000","last_clean":"2024-09-16T09:38:14.348434+0000","last_became_active":"2024-09-16T09:31:09.271662+0000","last_became_peered":"2024-09-16T09:31:09.271662+0000","last_unstale":"2024-09-16T09:38:14.348434+0000","last_undegraded":"2024-09-16T09:38:14.348434+0000","last_fullsized":"2024-09-16T09:38:14.348434+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:18.824937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:37:18.824937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:29.541577+0000","scrub_duration":79,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"285'19","reported_seq":737,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.341691+0000","last_change":"2024-09-16T09:37:25.995663+0000","last_active":"2024-09-16T09:38:14.341691+0000","last_peered":"2024-09-16T09:38:14.341691+0000","last_clean":"2024-09-16T09:38:14.341691+0000","last_became_active":"2024-09-16T09:25:03.707769+0000","last_became_peered":"2024-09-16T09:25:03.707769+0000","last_unstale":"2024-09-16T09:38:14.341691+0000","last_undegraded":"2024-09-16T09:38:14.341691+0000","last_fullsized":"2024-09-16T09:38:14.341691+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'19","last_scrub_stamp":"2024-09-16T09:37:25.995607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:37:25.995607+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:34.249870+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":694,"reported_epoch":389,"state":"active+clean","last_fresh":"2024-09-16T09:38:14.350837+0000","last_change":"2024-09-16T09:37:26.646784+0000","last_active":"2024-09-16T09:38:14.350837+0000","last_peered":"2024-09-16T09:38:14.350837+0000","last_clean":"2024-09-16T09:38:14.350837+0000","last_became_active":"2024-09-16T09:25:05.711952+0000","last_became_peered":"2024-09-16T09:25:05.711952+0000","last_unstale":"2024-09-16T09:38:14.350837+0000","last_undegraded":"2024-09-16T09:38:14.350837+0000","last_fullsized":"2024-09-16T09:38:14.350837+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:26.646686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:37:26.646686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:54.298242+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":16,"stat_sum":{"num_bytes":3219521536,"num_objects":49126,"num_object_clones":0,"num_object_copies":147378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49126,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49126,"num_write_kb":3144064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":9715449856,"data_stored":9715449856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":49126,"ondisk_log_size":49126,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":1117,"num_read_kb":874,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":134,"num_bytes_recovered":3576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":681,"ondisk_log_size":681,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1986,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":790,"num_read_kb":785,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":21,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":327680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1659,"ondisk_log_size":1659,"up":63,"acting":63,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266852,"num_pgs":66,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1343876,"kb_used_data":767092,"kb_used_omap":1,"kb_used_meta":576766,"kb_avail":92401276,"statfs":{"total":95995035648,"available":94618906624,"internally_reserved":0,"allocated":785502208,"data_stored":784875384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":590608842},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495785,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1794348,"kb_used_data":1393884,"kb_used_omap":1,"kb_used_meta":400446,"kb_avail":91950804,"statfs":{"total":95995035648,"available":94157623296,"internally_reserved":0,"allocated":1427337216,"data_stored":1426733870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":410057158},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,2,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724720,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1708604,"kb_used_data":1095724,"kb_used_omap":1,"kb_used_meta":612862,"kb_avail":92036548,"statfs":{"total":95995035648,"available":94245425152,"internally_reserved":0,"allocated":1122021376,"data_stored":1121402769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":627571142},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953653,"num_pgs":72,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1943684,"kb_used_data":1507124,"kb_used_omap":1,"kb_used_meta":436542,"kb_avail":91801468,"statfs":{"total":95995035648,"available":94004703232,"internally_reserved":0,"allocated":1543294976,"data_stored":1542668106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":447019462},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149883,"num_pgs":70,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1639636,"kb_used_data":990660,"kb_used_omap":1,"kb_used_meta":648958,"kb_avail":92105516,"statfs":{"total":95995035648,"available":94316048384,"internally_reserved":0,"allocated":1014435840,"data_stored":1013793526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":664533447},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411520,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1591404,"kb_used_data":1136796,"kb_used_omap":1,"kb_used_meta":454590,"kb_avail":92153748,"statfs":{"total":95995035648,"available":94365437952,"internally_reserved":0,"allocated":1164079104,"data_stored":1163476124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":465500613},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575047,"num_pgs":65,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1258276,"kb_used_data":591252,"kb_used_omap":1,"kb_used_meta":667006,"kb_avail":92486876,"statfs":{"total":95995035648,"available":94706561024,"internally_reserved":0,"allocated":605442048,"data_stored":604782500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":683014597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738573,"num_pgs":62,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2559048,"kb_used_data":1978104,"kb_used_omap":1,"kb_used_meta":580926,"kb_avail":91186104,"statfs":{"total":95995035648,"available":93374570496,"internally_reserved":0,"allocated":2025578496,"data_stored":2024947842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":594868677},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2077556736,"data_stored":2077556736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":606142464,"data_stored":606142464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1209729024,"data_stored":1209729024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1026031616,"data_stored":1026031616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1591607296,"data_stored":1591607296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1017446400,"data_stored":1017446400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1387331584,"data_stored":1387331584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":799604736,"data_stored":799604736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T09:38:22.867 INFO:tasks.cephadm.ceph_manager.ceph:recovered! 2024-09-16T09:38:22.867 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T09:38:22.867 INFO:tasks.thrashosds.thrasher:Growing pool unique_pool_3 2024-09-16T09:38:22.867 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-16T09:38:22.912 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:38:22.957 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:22.957 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:22.957 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:38:23.058 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:38:23.096 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:23.097 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:23.098 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:38:23.198 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:38:23.219 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:23.219 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:23.220 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:38:23.320 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:38:23.356 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:23.357 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:23.358 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:38:23.459 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:38:23.495 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:23.496 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:23.497 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:38:23.497 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:23 smithi070 ceph-mon[30388]: pgmap v705: 168 pgs: 168 active+clean; 3.0 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 639 B/s rd, 84 MiB/s wr, 1.35k op/s 2024-09-16T09:38:23.498 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:23 smithi070 ceph-mon[30388]: from='client.16017 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:38:23.498 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:23 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:38:23.498 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:23 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:38:23.498 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:23 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:38:23.498 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:23 smithi070 ceph-mon[40691]: pgmap v705: 168 pgs: 168 active+clean; 3.0 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 639 B/s rd, 84 MiB/s wr, 1.35k op/s 2024-09-16T09:38:23.499 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:23 smithi070 ceph-mon[40691]: from='client.16017 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:38:23.499 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:23 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:38:23.499 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:23 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:38:23.499 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:23 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:38:23.541 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 27 16 57118 57102 132.138 90 0.0119646 0.00756322 2024-09-16T09:38:23.598 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:38:23.638 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:23.639 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:23.640 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:38:23.660 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:38:23 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:38:23.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:23 smithi084 ceph-mon[32629]: pgmap v705: 168 pgs: 168 active+clean; 3.0 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 639 B/s rd, 84 MiB/s wr, 1.35k op/s 2024-09-16T09:38:23.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:23 smithi084 ceph-mon[32629]: from='client.16017 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:38:23.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:23 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:38:23.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:23 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:38:23.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:23 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:38:23.740 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:38:23.780 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:23.780 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:23.781 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:38:23.863 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:38:23.864 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:38:23.881 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:38:23.898 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:38:23.903 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:38:23.903 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:38:23.909 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:38:23.914 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:23.915 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:23.915 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:38:23.936 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:38:23.936 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:38:23.967 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:38:23.967 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:38:24.000 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:38:24.000 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:38:24.016 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:38:24.028 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:38:24.029 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:38:24.058 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:24.059 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:24.059 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:38:24.064 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:38:24.064 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:38:24.097 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:38:24.098 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:38:24.160 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:38:24.196 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:24.197 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:24.197 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:38:24.298 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:38:24.339 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:24.340 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:24.340 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:38:24.441 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:38:24.476 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:24.477 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:24.477 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:38:24.541 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 28 16 58619 58603 130.769 93.8125 0.00796283 0.00764172 2024-09-16T09:38:24.578 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:38:24.618 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:24.619 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:24.619 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:38:24.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:24 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:38:24.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:24 smithi084 ceph-mon[32629]: 2.1 scrub starts 2024-09-16T09:38:24.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:24 smithi084 ceph-mon[32629]: 2.1 scrub ok 2024-09-16T09:38:24.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:24 smithi084 ceph-mon[32629]: 1.2 scrub starts 2024-09-16T09:38:24.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:24 smithi084 ceph-mon[32629]: 1.2 scrub ok 2024-09-16T09:38:24.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:24 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3330001458' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:38:24.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:24 smithi084 ceph-mon[32629]: 6.f scrub starts 2024-09-16T09:38:24.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:24 smithi084 ceph-mon[32629]: 6.f scrub ok 2024-09-16T09:38:24.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:24 smithi084 ceph-mon[32629]: 1.7 scrub starts 2024-09-16T09:38:24.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:24 smithi084 ceph-mon[32629]: 1.7 scrub ok 2024-09-16T09:38:24.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:24 smithi084 ceph-mon[32629]: 5.15 scrub starts 2024-09-16T09:38:24.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:24 smithi084 ceph-mon[32629]: 5.15 scrub ok 2024-09-16T09:38:24.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:24 smithi084 ceph-mon[32629]: osdmap e391: 8 total, 8 up, 8 in 2024-09-16T09:38:24.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:24 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:38:24.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:24 smithi070 ceph-mon[30388]: 2.1 scrub starts 2024-09-16T09:38:24.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:24 smithi070 ceph-mon[30388]: 2.1 scrub ok 2024-09-16T09:38:24.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:24 smithi070 ceph-mon[30388]: 1.2 scrub starts 2024-09-16T09:38:24.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:24 smithi070 ceph-mon[30388]: 1.2 scrub ok 2024-09-16T09:38:24.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:24 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3330001458' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:38:24.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:24 smithi070 ceph-mon[30388]: 6.f scrub starts 2024-09-16T09:38:24.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:24 smithi070 ceph-mon[30388]: 6.f scrub ok 2024-09-16T09:38:24.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:24 smithi070 ceph-mon[30388]: 1.7 scrub starts 2024-09-16T09:38:24.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:24 smithi070 ceph-mon[30388]: 1.7 scrub ok 2024-09-16T09:38:24.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:24 smithi070 ceph-mon[30388]: 5.15 scrub starts 2024-09-16T09:38:24.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:24 smithi070 ceph-mon[30388]: 5.15 scrub ok 2024-09-16T09:38:24.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:24 smithi070 ceph-mon[30388]: osdmap e391: 8 total, 8 up, 8 in 2024-09-16T09:38:24.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:24 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:38:24.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:24 smithi070 ceph-mon[40691]: 2.1 scrub starts 2024-09-16T09:38:24.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:24 smithi070 ceph-mon[40691]: 2.1 scrub ok 2024-09-16T09:38:24.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:24 smithi070 ceph-mon[40691]: 1.2 scrub starts 2024-09-16T09:38:24.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:24 smithi070 ceph-mon[40691]: 1.2 scrub ok 2024-09-16T09:38:24.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:24 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3330001458' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:38:24.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:24 smithi070 ceph-mon[40691]: 6.f scrub starts 2024-09-16T09:38:24.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:24 smithi070 ceph-mon[40691]: 6.f scrub ok 2024-09-16T09:38:24.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:24 smithi070 ceph-mon[40691]: 1.7 scrub starts 2024-09-16T09:38:24.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:24 smithi070 ceph-mon[40691]: 1.7 scrub ok 2024-09-16T09:38:24.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:24 smithi070 ceph-mon[40691]: 5.15 scrub starts 2024-09-16T09:38:24.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:24 smithi070 ceph-mon[40691]: 5.15 scrub ok 2024-09-16T09:38:24.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:24 smithi070 ceph-mon[40691]: osdmap e391: 8 total, 8 up, 8 in 2024-09-16T09:38:24.696 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T09:38:24.697 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 20, 2024-09-16T09:38:24.697 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 600, 2024-09-16T09:38:24.697 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [ 2024-09-16T09:38:24.697 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.697 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.24698.0:203420 7.a 7:52851f87:::benchmark_data_smithi070_78438_object77289:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e136)", 2024-09-16T09:38:24.697 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:28:51.981780+0000", 2024-09-16T09:38:24.697 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 572.70964956, 2024-09-16T09:38:24.697 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.020255924, 2024-09-16T09:38:24.698 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:38:24.698 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:38:24.698 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T09:38:24.698 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:38:24.698 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.24698", 2024-09-16T09:38:24.698 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:38:24.698 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 203420 2024-09-16T09:38:24.698 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.698 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:38:24.698 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.698 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:38:24.698 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.981780+0000", 2024-09-16T09:38:24.699 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:38:24.699 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.699 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.699 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:38:24.699 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.981780+0000", 2024-09-16T09:38:24.699 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:38:24.699 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.700 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.700 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:38:24.700 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.981782+0000", 2024-09-16T09:38:24.700 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5400000000000001e-06 2024-09-16T09:38:24.700 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.700 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.700 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:38:24.701 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.981788+0000", 2024-09-16T09:38:24.701 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.0129999999999999e-06 2024-09-16T09:38:24.701 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.701 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.701 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:38:24.701 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.981802+0000", 2024-09-16T09:38:24.702 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3876000000000001e-05 2024-09-16T09:38:24.702 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.702 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.702 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:38:24.702 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.981810+0000", 2024-09-16T09:38:24.702 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.7709999999999995e-06 2024-09-16T09:38:24.702 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.702 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.702 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:38:24.703 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.982159+0000", 2024-09-16T09:38:24.703 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000348389 2024-09-16T09:38:24.703 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.703 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.703 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op must wait for map", 2024-09-16T09:38:24.703 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.982179+0000", 2024-09-16T09:38:24.703 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0489999999999999e-05 2024-09-16T09:38:24.703 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.704 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.704 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:38:24.704 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.985921+0000", 2024-09-16T09:38:24.704 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0037419480000000001 2024-09-16T09:38:24.704 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.705 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.705 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:38:24.705 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.985932+0000", 2024-09-16T09:38:24.705 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0929e-05 2024-09-16T09:38:24.705 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.705 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.705 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:38:24.706 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.001181+0000", 2024-09-16T09:38:24.706 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0152491459999999 2024-09-16T09:38:24.706 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.706 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.706 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:38:24.706 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.002036+0000", 2024-09-16T09:38:24.706 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00085482199999999998 2024-09-16T09:38:24.706 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:38:24.706 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:38:24.706 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:38:24.707 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.707 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.707 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.24698.0:203422 7.a 7:5a4cd988:::benchmark_data_smithi070_78438_object77291:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e136)", 2024-09-16T09:38:24.707 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:28:51.981838+0000", 2024-09-16T09:38:24.707 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 572.70959189899997, 2024-09-16T09:38:24.708 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.020646688, 2024-09-16T09:38:24.708 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:38:24.708 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:38:24.708 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T09:38:24.708 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:38:24.708 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.24698", 2024-09-16T09:38:24.708 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:38:24.709 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 203422 2024-09-16T09:38:24.709 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.709 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:38:24.709 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.709 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:38:24.709 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.981838+0000", 2024-09-16T09:38:24.709 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:38:24.709 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.709 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.709 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:38:24.709 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.981838+0000", 2024-09-16T09:38:24.709 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:38:24.709 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.709 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.710 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:38:24.710 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.981838+0000", 2024-09-16T09:38:24.710 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0500000000000004e-07 2024-09-16T09:38:24.710 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.710 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.710 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:38:24.710 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.981841+0000", 2024-09-16T09:38:24.710 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.061e-06 2024-09-16T09:38:24.710 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.710 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.710 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:38:24.710 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.981848+0000", 2024-09-16T09:38:24.710 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.8730000000000001e-06 2024-09-16T09:38:24.711 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.711 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.711 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:38:24.711 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.981854+0000", 2024-09-16T09:38:24.711 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.6879999999999996e-06 2024-09-16T09:38:24.711 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.711 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.711 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:38:24.711 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.982493+0000", 2024-09-16T09:38:24.711 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00063907000000000005 2024-09-16T09:38:24.711 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.711 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.711 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting_for_map not empty", 2024-09-16T09:38:24.712 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.982513+0000", 2024-09-16T09:38:24.712 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.965e-05 2024-09-16T09:38:24.712 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.712 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.712 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:38:24.712 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.986082+0000", 2024-09-16T09:38:24.712 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0035691189999999999 2024-09-16T09:38:24.712 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.712 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.712 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:38:24.712 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:51.986091+0000", 2024-09-16T09:38:24.712 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.9099999999999994e-06 2024-09-16T09:38:24.712 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.713 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.713 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:38:24.713 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.002421+0000", 2024-09-16T09:38:24.713 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0163305250000001 2024-09-16T09:38:24.713 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.713 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.713 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:38:24.713 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.002484+0000", 2024-09-16T09:38:24.713 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.3287000000000006e-05 2024-09-16T09:38:24.713 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:38:24.713 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:38:24.713 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:38:24.713 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.713 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.714 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.24698.0:203455 7.a 7:5b32df42:::benchmark_data_smithi070_78438_object77324:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136)", 2024-09-16T09:38:24.714 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:28:52.145758+0000", 2024-09-16T09:38:24.714 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 572.54567138300001, 2024-09-16T09:38:24.714 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.86280117499999998, 2024-09-16T09:38:24.714 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:38:24.714 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:38:24.714 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:38:24.714 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:38:24.714 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.24698", 2024-09-16T09:38:24.714 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:38:24.714 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 203455 2024-09-16T09:38:24.714 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.714 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:38:24.715 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.715 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:38:24.715 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.145758+0000", 2024-09-16T09:38:24.715 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:38:24.715 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.715 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.715 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:38:24.715 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.145758+0000", 2024-09-16T09:38:24.715 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:38:24.715 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.715 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.715 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:38:24.715 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.145760+0000", 2024-09-16T09:38:24.715 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.703e-06 2024-09-16T09:38:24.716 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.716 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.716 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:38:24.716 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.145767+0000", 2024-09-16T09:38:24.716 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.5590000000000002e-06 2024-09-16T09:38:24.716 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.716 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.716 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:38:24.716 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.145785+0000", 2024-09-16T09:38:24.716 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.8501000000000001e-05 2024-09-16T09:38:24.716 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.716 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.716 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:38:24.717 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.145793+0000", 2024-09-16T09:38:24.717 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.2830000000000001e-06 2024-09-16T09:38:24.717 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.717 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.717 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:38:24.717 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.145867+0000", 2024-09-16T09:38:24.717 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3374000000000006e-05 2024-09-16T09:38:24.717 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.717 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.717 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:38:24.717 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.145878+0000", 2024-09-16T09:38:24.717 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0942e-05 2024-09-16T09:38:24.717 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.718 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.718 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:38:24.718 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.002527+0000", 2024-09-16T09:38:24.718 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.85664891300000001 2024-09-16T09:38:24.718 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.718 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.718 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:38:24.718 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.002770+0000", 2024-09-16T09:38:24.718 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000243754 2024-09-16T09:38:24.718 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.718 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.718 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 3,5", 2024-09-16T09:38:24.718 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.002875+0000", 2024-09-16T09:38:24.718 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000105033 2024-09-16T09:38:24.719 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.719 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.719 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:38:24.719 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.008335+0000", 2024-09-16T09:38:24.719 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0054600509999999996 2024-09-16T09:38:24.719 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.719 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.719 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:38:24.719 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.008442+0000", 2024-09-16T09:38:24.720 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00010692 2024-09-16T09:38:24.720 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.720 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.720 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:38:24.720 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.008478+0000", 2024-09-16T09:38:24.720 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5590000000000003e-05 2024-09-16T09:38:24.720 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.720 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.720 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:38:24.720 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.008516+0000", 2024-09-16T09:38:24.720 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8565000000000001e-05 2024-09-16T09:38:24.720 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.720 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.721 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:38:24.721 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.008559+0000", 2024-09-16T09:38:24.721 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2987e-05 2024-09-16T09:38:24.721 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:38:24.721 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:38:24.721 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:38:24.721 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.721 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.721 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.24698.0:203461 7.a 7:5b0a38e6:::benchmark_data_smithi070_78438_object77330:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136)", 2024-09-16T09:38:24.721 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:28:52.150646+0000", 2024-09-16T09:38:24.721 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 572.54078377799999, 2024-09-16T09:38:24.721 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.85954066799999995, 2024-09-16T09:38:24.721 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:38:24.722 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:38:24.722 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:38:24.722 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:38:24.722 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.24698", 2024-09-16T09:38:24.722 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:38:24.722 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 203461 2024-09-16T09:38:24.722 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.722 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:38:24.722 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.722 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:38:24.722 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.150646+0000", 2024-09-16T09:38:24.722 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:38:24.722 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.722 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.723 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:38:24.723 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.150646+0000", 2024-09-16T09:38:24.723 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:38:24.723 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.723 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.723 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:38:24.723 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.150647+0000", 2024-09-16T09:38:24.723 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.1800000000000005e-07 2024-09-16T09:38:24.723 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.723 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.723 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:38:24.723 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.150666+0000", 2024-09-16T09:38:24.723 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.8927e-05 2024-09-16T09:38:24.723 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.724 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.724 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:38:24.724 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.150676+0000", 2024-09-16T09:38:24.724 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0113e-05 2024-09-16T09:38:24.724 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.724 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.724 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:38:24.724 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.150682+0000", 2024-09-16T09:38:24.724 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.3980000000000004e-06 2024-09-16T09:38:24.724 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.724 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.724 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:38:24.724 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.150779+0000", 2024-09-16T09:38:24.725 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.6989000000000003e-05 2024-09-16T09:38:24.725 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.725 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.725 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:38:24.725 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.150788+0000", 2024-09-16T09:38:24.725 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.4409999999999999e-06 2024-09-16T09:38:24.725 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.725 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.725 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:38:24.725 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.004849+0000", 2024-09-16T09:38:24.726 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.85406023600000003 2024-09-16T09:38:24.726 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.726 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.726 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:38:24.726 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.004962+0000", 2024-09-16T09:38:24.726 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00011376599999999999 2024-09-16T09:38:24.726 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.726 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.726 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 3,5", 2024-09-16T09:38:24.726 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.005070+0000", 2024-09-16T09:38:24.726 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000107599 2024-09-16T09:38:24.726 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.726 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.726 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:38:24.727 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.008390+0000", 2024-09-16T09:38:24.727 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0033195080000000001 2024-09-16T09:38:24.727 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.727 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.727 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:38:24.727 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.010026+0000", 2024-09-16T09:38:24.727 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001636167 2024-09-16T09:38:24.727 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.727 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.727 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:38:24.727 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.010073+0000", 2024-09-16T09:38:24.727 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.7425000000000001e-05 2024-09-16T09:38:24.727 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.728 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.728 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:38:24.728 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.010138+0000", 2024-09-16T09:38:24.728 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.4800000000000003e-05 2024-09-16T09:38:24.728 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.728 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.728 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:38:24.728 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.010187+0000", 2024-09-16T09:38:24.728 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.8581000000000001e-05 2024-09-16T09:38:24.728 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:38:24.728 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:38:24.728 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:38:24.729 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.729 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.729 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.24698.0:203478 7.a 7:5aee0767:::benchmark_data_smithi070_78438_object77347:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136)", 2024-09-16T09:38:24.729 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:28:52.164088+0000", 2024-09-16T09:38:24.729 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 572.52734203700004, 2024-09-16T09:38:24.729 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.85042330099999996, 2024-09-16T09:38:24.729 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:38:24.729 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:38:24.729 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:38:24.729 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:38:24.729 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.24698", 2024-09-16T09:38:24.729 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:38:24.729 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 203478 2024-09-16T09:38:24.730 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.730 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:38:24.730 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.730 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:38:24.730 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.164088+0000", 2024-09-16T09:38:24.730 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:38:24.730 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.730 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.730 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:38:24.730 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.164088+0000", 2024-09-16T09:38:24.730 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:38:24.730 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.730 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.731 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:38:24.731 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.164088+0000", 2024-09-16T09:38:24.731 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.9900000000000001e-07 2024-09-16T09:38:24.731 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.731 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.731 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:38:24.731 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:38:24.732 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.164091+0000", 2024-09-16T09:38:24.733 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.5469999999999999e-06 2024-09-16T09:38:24.733 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.733 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.733 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:38:24.733 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.164097+0000", 2024-09-16T09:38:24.733 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.4949999999999996e-06 2024-09-16T09:38:24.733 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.733 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.733 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:38:24.733 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.164103+0000", 2024-09-16T09:38:24.733 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4249999999999997e-06 2024-09-16T09:38:24.733 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.733 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.734 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:38:24.734 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.164297+0000", 2024-09-16T09:38:24.734 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00019394400000000001 2024-09-16T09:38:24.734 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.734 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.734 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:38:24.734 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.164307+0000", 2024-09-16T09:38:24.734 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0781000000000001e-05 2024-09-16T09:38:24.734 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.734 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.734 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:38:24.734 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.006832+0000", 2024-09-16T09:38:24.734 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.84252505899999997 2024-09-16T09:38:24.735 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.735 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.735 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:38:24.735 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.006941+0000", 2024-09-16T09:38:24.735 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00010823300000000001 2024-09-16T09:38:24.735 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.735 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.735 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 3,5", 2024-09-16T09:38:24.735 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.007051+0000", 2024-09-16T09:38:24.735 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000110491 2024-09-16T09:38:24.735 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.735 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.735 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:38:24.736 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.010031+0000", 2024-09-16T09:38:24.736 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0029797629999999999 2024-09-16T09:38:24.736 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.736 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.736 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:38:24.736 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.010244+0000", 2024-09-16T09:38:24.736 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00021353600000000001 2024-09-16T09:38:24.736 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.736 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.736 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:38:24.736 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.014448+0000", 2024-09-16T09:38:24.736 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0042034200000000002 2024-09-16T09:38:24.737 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.737 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.737 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:38:24.737 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.014481+0000", 2024-09-16T09:38:24.737 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3259000000000001e-05 2024-09-16T09:38:24.737 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.737 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.737 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:38:24.737 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.014511+0000", 2024-09-16T09:38:24.737 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.9848999999999998e-05 2024-09-16T09:38:24.737 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:38:24.737 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:38:24.737 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:38:24.737 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.738 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.738 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.24698.0:203490 7.a 7:5b988e00:::benchmark_data_smithi070_78438_object77359:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136)", 2024-09-16T09:38:24.738 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:28:52.173441+0000", 2024-09-16T09:38:24.738 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 572.51798816600001, 2024-09-16T09:38:24.738 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.84119520299999995, 2024-09-16T09:38:24.738 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:38:24.738 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:38:24.738 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:38:24.738 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:38:24.738 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.24698", 2024-09-16T09:38:24.738 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:38:24.738 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 203490 2024-09-16T09:38:24.738 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.739 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:38:24.739 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.739 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:38:24.739 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.173441+0000", 2024-09-16T09:38:24.739 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:38:24.739 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.739 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.739 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:38:24.739 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.173441+0000", 2024-09-16T09:38:24.739 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:38:24.739 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.739 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.740 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:38:24.740 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.173443+0000", 2024-09-16T09:38:24.740 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.426e-06 2024-09-16T09:38:24.740 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.740 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.740 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:38:24.740 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.173449+0000", 2024-09-16T09:38:24.740 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.6210000000000001e-06 2024-09-16T09:38:24.740 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.740 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.740 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:38:24.740 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.173464+0000", 2024-09-16T09:38:24.740 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5287000000000001e-05 2024-09-16T09:38:24.740 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.741 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.741 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:38:24.741 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.173471+0000", 2024-09-16T09:38:24.741 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.2910000000000002e-06 2024-09-16T09:38:24.741 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.741 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.741 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:38:24.741 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.173534+0000", 2024-09-16T09:38:24.741 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.3219000000000006e-05 2024-09-16T09:38:24.741 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.741 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.741 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:38:24.741 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.173547+0000", 2024-09-16T09:38:24.742 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2451e-05 2024-09-16T09:38:24.742 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.742 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.742 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:38:24.742 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.008597+0000", 2024-09-16T09:38:24.742 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.83504982800000005 2024-09-16T09:38:24.742 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.742 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.742 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:38:24.742 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.008699+0000", 2024-09-16T09:38:24.742 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000102577 2024-09-16T09:38:24.742 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.742 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.742 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 3,5", 2024-09-16T09:38:24.743 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.008804+0000", 2024-09-16T09:38:24.743 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000105265 2024-09-16T09:38:24.743 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.743 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.743 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:38:24.743 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.014398+0000", 2024-09-16T09:38:24.743 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0055940340000000003 2024-09-16T09:38:24.743 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.743 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.743 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:38:24.743 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.014414+0000", 2024-09-16T09:38:24.743 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5449999999999999e-05 2024-09-16T09:38:24.743 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.743 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.744 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:38:24.744 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.014556+0000", 2024-09-16T09:38:24.744 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000141786 2024-09-16T09:38:24.744 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.744 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.744 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:38:24.744 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.014601+0000", 2024-09-16T09:38:24.744 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5532999999999999e-05 2024-09-16T09:38:24.744 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.744 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.744 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:38:24.744 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.014637+0000", 2024-09-16T09:38:24.744 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5435000000000002e-05 2024-09-16T09:38:24.745 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:38:24.745 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:38:24.745 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:38:24.745 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.745 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.745 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.24698.0:203508 7.a 7:50f2d387:::benchmark_data_smithi070_78438_object77377:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136)", 2024-09-16T09:38:24.745 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:28:52.190869+0000", 2024-09-16T09:38:24.745 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 572.50056079000001, 2024-09-16T09:38:24.745 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.82400409200000002, 2024-09-16T09:38:24.745 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:38:24.745 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:38:24.745 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:38:24.745 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:38:24.745 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.24698", 2024-09-16T09:38:24.746 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:38:24.746 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 203508 2024-09-16T09:38:24.746 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.746 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:38:24.746 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.746 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:38:24.746 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.190869+0000", 2024-09-16T09:38:24.746 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:38:24.746 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.746 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.746 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:38:24.746 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.190869+0000", 2024-09-16T09:38:24.746 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:38:24.746 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.747 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.747 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:38:24.747 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.190870+0000", 2024-09-16T09:38:24.747 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5290000000000001e-06 2024-09-16T09:38:24.747 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.747 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.747 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:38:24.747 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.190876+0000", 2024-09-16T09:38:24.747 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4389999999999999e-06 2024-09-16T09:38:24.747 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.747 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.747 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:38:24.747 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.190887+0000", 2024-09-16T09:38:24.748 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0847e-05 2024-09-16T09:38:24.748 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.748 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.748 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:38:24.748 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.190893+0000", 2024-09-16T09:38:24.748 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.5100000000000004e-06 2024-09-16T09:38:24.748 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.748 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.748 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:38:24.748 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.191097+0000", 2024-09-16T09:38:24.748 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00020403700000000001 2024-09-16T09:38:24.748 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.748 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.749 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:38:24.749 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.191106+0000", 2024-09-16T09:38:24.749 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.7600000000000008e-06 2024-09-16T09:38:24.749 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.749 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.749 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:38:24.749 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.010277+0000", 2024-09-16T09:38:24.749 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.81917143299999995 2024-09-16T09:38:24.749 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.749 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.749 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:38:24.749 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.010378+0000", 2024-09-16T09:38:24.749 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000100559 2024-09-16T09:38:24.749 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.750 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.750 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 3,5", 2024-09-16T09:38:24.750 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.010474+0000", 2024-09-16T09:38:24.750 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.6433000000000003e-05 2024-09-16T09:38:24.750 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.750 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.750 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:38:24.750 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.014647+0000", 2024-09-16T09:38:24.750 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.004172111 2024-09-16T09:38:24.750 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.750 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.750 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:38:24.750 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.014703+0000", 2024-09-16T09:38:24.751 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.6196999999999997e-05 2024-09-16T09:38:24.751 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.751 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.751 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:38:24.751 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.014771+0000", 2024-09-16T09:38:24.751 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.8649e-05 2024-09-16T09:38:24.751 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.751 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.751 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:38:24.751 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.014833+0000", 2024-09-16T09:38:24.751 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1499000000000002e-05 2024-09-16T09:38:24.751 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.751 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.751 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:38:24.752 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.014873+0000", 2024-09-16T09:38:24.752 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0089000000000002e-05 2024-09-16T09:38:24.752 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:38:24.752 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:38:24.752 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:38:24.752 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.752 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.752 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.24698.0:203532 7.a 7:50322cd6:::benchmark_data_smithi070_78438_object77401:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136)", 2024-09-16T09:38:24.752 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:28:52.208817+0000", 2024-09-16T09:38:24.752 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 572.48261248300003, 2024-09-16T09:38:24.752 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.81510171200000003, 2024-09-16T09:38:24.752 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:38:24.752 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:38:24.753 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:38:24.753 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:38:24.753 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.24698", 2024-09-16T09:38:24.753 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:38:24.753 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 203532 2024-09-16T09:38:24.753 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.753 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:38:24.753 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.753 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:38:24.753 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.208817+0000", 2024-09-16T09:38:24.753 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:38:24.753 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.753 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.754 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:38:24.754 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.208817+0000", 2024-09-16T09:38:24.754 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:38:24.754 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.754 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.754 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:38:24.754 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.208819+0000", 2024-09-16T09:38:24.754 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6869999999999999e-06 2024-09-16T09:38:24.754 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.754 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.754 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:38:24.754 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.208825+0000", 2024-09-16T09:38:24.754 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.0510000000000002e-06 2024-09-16T09:38:24.754 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.755 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.755 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:38:24.755 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.208842+0000", 2024-09-16T09:38:24.755 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7286e-05 2024-09-16T09:38:24.755 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.755 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.755 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:38:24.755 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.208850+0000", 2024-09-16T09:38:24.755 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6669999999999996e-06 2024-09-16T09:38:24.755 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.755 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.755 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:38:24.755 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.208919+0000", 2024-09-16T09:38:24.755 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.9596000000000002e-05 2024-09-16T09:38:24.756 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.756 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.756 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:38:24.756 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.208932+0000", 2024-09-16T09:38:24.756 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2170999999999999e-05 2024-09-16T09:38:24.756 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.756 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.756 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:38:24.756 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.012264+0000", 2024-09-16T09:38:24.756 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.803332245 2024-09-16T09:38:24.756 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.756 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.756 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:38:24.756 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.012380+0000", 2024-09-16T09:38:24.757 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000116592 2024-09-16T09:38:24.757 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.757 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.757 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 3,5", 2024-09-16T09:38:24.757 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.012477+0000", 2024-09-16T09:38:24.757 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.6370000000000001e-05 2024-09-16T09:38:24.757 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.757 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.757 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:38:24.757 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.014652+0000", 2024-09-16T09:38:24.757 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0021752859999999998 2024-09-16T09:38:24.757 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.757 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.758 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:38:24.758 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.014915+0000", 2024-09-16T09:38:24.758 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00026334699999999998 2024-09-16T09:38:24.758 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.758 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.758 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:38:24.758 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.023829+0000", 2024-09-16T09:38:24.758 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0089137379999999992 2024-09-16T09:38:24.758 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.758 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.758 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:38:24.758 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.023879+0000", 2024-09-16T09:38:24.758 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.9595999999999997e-05 2024-09-16T09:38:24.758 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.759 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.759 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:38:24.759 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.023919+0000", 2024-09-16T09:38:24.759 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0080000000000003e-05 2024-09-16T09:38:24.759 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:38:24.759 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:38:24.759 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:38:24.759 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.759 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.759 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.24698.0:203542 7.a 7:54d67e0e:::benchmark_data_smithi070_78438_object77411:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136)", 2024-09-16T09:38:24.759 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:28:52.223327+0000", 2024-09-16T09:38:24.759 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 572.46810264199996, 2024-09-16T09:38:24.759 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.80535596300000001, 2024-09-16T09:38:24.759 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:38:24.760 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:38:24.760 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:38:24.760 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:38:24.760 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.24698", 2024-09-16T09:38:24.760 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:38:24.760 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 203542 2024-09-16T09:38:24.760 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.760 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:38:24.760 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.760 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:38:24.760 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.223327+0000", 2024-09-16T09:38:24.760 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:38:24.760 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.760 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.761 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:38:24.761 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.223327+0000", 2024-09-16T09:38:24.761 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:38:24.761 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.761 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.761 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:38:24.761 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.223328+0000", 2024-09-16T09:38:24.761 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.291e-06 2024-09-16T09:38:24.761 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.761 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.761 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:38:24.761 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.223333+0000", 2024-09-16T09:38:24.761 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.1789999999999999e-06 2024-09-16T09:38:24.762 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.762 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.762 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:38:24.762 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.223347+0000", 2024-09-16T09:38:24.762 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3696999999999999e-05 2024-09-16T09:38:24.762 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.762 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.762 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:38:24.762 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.223355+0000", 2024-09-16T09:38:24.762 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.486e-06 2024-09-16T09:38:24.762 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.762 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.762 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:38:24.762 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.223423+0000", 2024-09-16T09:38:24.763 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.8592999999999999e-05 2024-09-16T09:38:24.763 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.763 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.763 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:38:24.763 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.223433+0000", 2024-09-16T09:38:24.763 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.3219999999999999e-06 2024-09-16T09:38:24.763 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.763 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.763 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:38:24.763 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.014946+0000", 2024-09-16T09:38:24.763 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.79151325800000005 2024-09-16T09:38:24.763 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.763 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.764 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:38:24.764 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.015054+0000", 2024-09-16T09:38:24.764 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00010777600000000001 2024-09-16T09:38:24.764 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.764 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.764 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 3,5", 2024-09-16T09:38:24.764 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.015166+0000", 2024-09-16T09:38:24.764 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000111987 2024-09-16T09:38:24.764 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.764 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.764 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:38:24.764 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.023955+0000", 2024-09-16T09:38:24.764 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0087896369999999995 2024-09-16T09:38:24.765 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.765 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.765 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:38:24.765 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.028597+0000", 2024-09-16T09:38:24.765 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0046413829999999998 2024-09-16T09:38:24.765 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.765 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.765 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:38:24.765 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.028608+0000", 2024-09-16T09:38:24.765 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1442e-05 2024-09-16T09:38:24.765 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.765 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.765 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:38:24.766 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.028642+0000", 2024-09-16T09:38:24.766 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3893000000000002e-05 2024-09-16T09:38:24.766 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.766 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.766 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:38:24.766 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.028683+0000", 2024-09-16T09:38:24.766 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.1019000000000001e-05 2024-09-16T09:38:24.766 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:38:24.766 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:38:24.766 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:38:24.766 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.766 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.766 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.24698.0:203546 7.a 7:5c0c2c2e:::benchmark_data_smithi070_78438_object77415:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136)", 2024-09-16T09:38:24.767 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:28:52.223460+0000", 2024-09-16T09:38:24.767 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 572.46796930400001, 2024-09-16T09:38:24.767 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.80539693000000001, 2024-09-16T09:38:24.767 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:38:24.767 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:38:24.767 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:38:24.767 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:38:24.767 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.24698", 2024-09-16T09:38:24.767 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:38:24.767 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 203546 2024-09-16T09:38:24.767 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.767 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:38:24.767 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.767 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:38:24.768 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.223460+0000", 2024-09-16T09:38:24.768 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:38:24.768 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.768 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.768 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:38:24.768 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.223460+0000", 2024-09-16T09:38:24.768 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:38:24.768 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.768 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.768 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:38:24.768 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.223461+0000", 2024-09-16T09:38:24.768 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0380000000000001e-06 2024-09-16T09:38:24.768 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.769 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.769 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:38:24.769 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.223466+0000", 2024-09-16T09:38:24.769 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.9910000000000002e-06 2024-09-16T09:38:24.769 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.769 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.769 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:38:24.769 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.223480+0000", 2024-09-16T09:38:24.769 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3917000000000001e-05 2024-09-16T09:38:24.769 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.769 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.769 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:38:24.769 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.223487+0000", 2024-09-16T09:38:24.769 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.9840000000000004e-06 2024-09-16T09:38:24.770 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.770 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.770 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:38:24.770 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.224169+0000", 2024-09-16T09:38:24.770 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00068157799999999996 2024-09-16T09:38:24.770 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.770 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.770 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:38:24.770 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.224177+0000", 2024-09-16T09:38:24.770 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.1219999999999995e-06 2024-09-16T09:38:24.770 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.770 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.770 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:38:24.770 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.017181+0000", 2024-09-16T09:38:24.771 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.79300421899999995 2024-09-16T09:38:24.771 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.771 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.771 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:38:24.771 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.017275+0000", 2024-09-16T09:38:24.771 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.3654999999999997e-05 2024-09-16T09:38:24.771 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.771 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.771 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 3,5", 2024-09-16T09:38:24.771 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.017384+0000", 2024-09-16T09:38:24.771 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000108768 2024-09-16T09:38:24.771 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.771 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.771 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:38:24.772 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.028693+0000", 2024-09-16T09:38:24.772 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.011309817999999999 2024-09-16T09:38:24.772 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.772 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.772 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:38:24.772 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.028743+0000", 2024-09-16T09:38:24.772 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.9947000000000001e-05 2024-09-16T09:38:24.772 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.772 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.772 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:38:24.772 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.028784+0000", 2024-09-16T09:38:24.772 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.015e-05 2024-09-16T09:38:24.772 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.773 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.773 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:38:24.773 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.028823+0000", 2024-09-16T09:38:24.773 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9246999999999999e-05 2024-09-16T09:38:24.773 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.773 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.773 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:38:24.773 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.028857+0000", 2024-09-16T09:38:24.773 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4496000000000003e-05 2024-09-16T09:38:24.773 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:38:24.773 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:38:24.773 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:38:24.773 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.774 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.774 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.24698.0:203550 7.a 7:5010ec5d:::benchmark_data_smithi070_78438_object77419:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136)", 2024-09-16T09:38:24.774 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:28:52.230377+0000", 2024-09-16T09:38:24.774 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 572.46105299500005, 2024-09-16T09:38:24.774 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.80332031299999995, 2024-09-16T09:38:24.774 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:38:24.774 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:38:24.774 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:38:24.774 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:38:24.774 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.24698", 2024-09-16T09:38:24.774 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:38:24.774 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 203550 2024-09-16T09:38:24.774 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.774 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:38:24.775 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.775 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:38:24.775 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.230377+0000", 2024-09-16T09:38:24.775 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:38:24.775 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.775 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.775 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:38:24.775 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.230377+0000", 2024-09-16T09:38:24.775 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:38:24.775 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.775 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.775 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:38:24.775 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.230379+0000", 2024-09-16T09:38:24.776 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.148e-06 2024-09-16T09:38:24.776 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.776 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.776 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:38:24.776 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.230387+0000", 2024-09-16T09:38:24.776 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.3319999999999992e-06 2024-09-16T09:38:24.776 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.776 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.776 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:38:24.776 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.230410+0000", 2024-09-16T09:38:24.776 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.2585000000000001e-05 2024-09-16T09:38:24.776 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.776 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.777 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:38:24.777 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.230419+0000", 2024-09-16T09:38:24.777 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.516e-06 2024-09-16T09:38:24.777 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.777 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.777 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:38:24.777 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.230500+0000", 2024-09-16T09:38:24.777 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.0457999999999994e-05 2024-09-16T09:38:24.777 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.777 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.777 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:38:24.777 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.230511+0000", 2024-09-16T09:38:24.777 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1549000000000001e-05 2024-09-16T09:38:24.778 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.778 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.778 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:38:24.778 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.019416+0000", 2024-09-16T09:38:24.778 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.78890451800000005 2024-09-16T09:38:24.778 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.778 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.778 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:38:24.778 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.019508+0000", 2024-09-16T09:38:24.778 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.1932e-05 2024-09-16T09:38:24.778 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.778 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.778 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 3,5", 2024-09-16T09:38:24.778 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.019600+0000", 2024-09-16T09:38:24.779 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.2604999999999999e-05 2024-09-16T09:38:24.779 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.779 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.779 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:38:24.779 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.028698+0000", 2024-09-16T09:38:24.779 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.009097628 2024-09-16T09:38:24.779 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.779 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.779 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:38:24.779 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.033536+0000", 2024-09-16T09:38:24.779 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0048381350000000004 2024-09-16T09:38:24.779 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.779 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.780 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:38:24.780 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.033612+0000", 2024-09-16T09:38:24.780 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6386999999999999e-05 2024-09-16T09:38:24.780 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.780 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.780 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:38:24.780 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.033659+0000", 2024-09-16T09:38:24.780 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.6350000000000002e-05 2024-09-16T09:38:24.780 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.780 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.780 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:38:24.780 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.033697+0000", 2024-09-16T09:38:24.780 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8170000000000002e-05 2024-09-16T09:38:24.781 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:38:24.781 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:38:24.781 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:38:24.781 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.781 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.781 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.24698.0:203558 7.a 7:5a81c49c:::benchmark_data_smithi070_78438_object77427:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136)", 2024-09-16T09:38:24.781 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:28:52.238828+0000", 2024-09-16T09:38:24.781 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 572.45260179700006, 2024-09-16T09:38:24.781 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.794966693, 2024-09-16T09:38:24.781 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:38:24.781 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:38:24.781 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:38:24.781 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:38:24.782 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.24698", 2024-09-16T09:38:24.782 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:38:24.782 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 203558 2024-09-16T09:38:24.782 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.782 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:38:24.782 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.782 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:38:24.782 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.238828+0000", 2024-09-16T09:38:24.782 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:38:24.782 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.782 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.782 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:38:24.782 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.238828+0000", 2024-09-16T09:38:24.782 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:38:24.783 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.783 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.783 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:38:24.783 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.238829+0000", 2024-09-16T09:38:24.783 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1880000000000001e-06 2024-09-16T09:38:24.783 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.783 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.783 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:38:24.783 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.238833+0000", 2024-09-16T09:38:24.783 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.1760000000000003e-06 2024-09-16T09:38:24.783 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.783 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.783 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:38:24.783 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.238844+0000", 2024-09-16T09:38:24.784 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1172000000000001e-05 2024-09-16T09:38:24.784 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.784 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.784 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:38:24.784 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.238852+0000", 2024-09-16T09:38:24.784 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3170000000000004e-06 2024-09-16T09:38:24.784 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.784 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.784 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:38:24.784 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.238911+0000", 2024-09-16T09:38:24.784 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.9190000000000001e-05 2024-09-16T09:38:24.784 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.784 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.785 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:38:24.785 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.238921+0000", 2024-09-16T09:38:24.785 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0129000000000001e-05 2024-09-16T09:38:24.785 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.785 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.785 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:38:24.785 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.021618+0000", 2024-09-16T09:38:24.785 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.78269724100000004 2024-09-16T09:38:24.785 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.785 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.785 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:38:24.785 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.021719+0000", 2024-09-16T09:38:24.785 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00010029100000000001 2024-09-16T09:38:24.785 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.786 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.786 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 3,5", 2024-09-16T09:38:24.786 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.021818+0000", 2024-09-16T09:38:24.786 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.9066000000000003e-05 2024-09-16T09:38:24.786 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.786 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.786 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:38:24.786 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.028703+0000", 2024-09-16T09:38:24.786 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0068851210000000001 2024-09-16T09:38:24.786 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.786 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.786 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:38:24.786 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.033575+0000", 2024-09-16T09:38:24.786 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0048726500000000001 2024-09-16T09:38:24.787 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.787 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.787 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:38:24.787 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.033729+0000", 2024-09-16T09:38:24.787 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00015408800000000001 2024-09-16T09:38:24.787 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.787 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.787 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:38:24.787 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.033766+0000", 2024-09-16T09:38:24.787 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6412999999999998e-05 2024-09-16T09:38:24.787 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.787 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.787 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:38:24.787 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.033795+0000", 2024-09-16T09:38:24.788 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.8651000000000001e-05 2024-09-16T09:38:24.788 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:38:24.788 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:38:24.788 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:38:24.788 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.788 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.788 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.24698.0:203641 7.a 7:5c484e8b:::benchmark_data_smithi070_78438_object77510:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136)", 2024-09-16T09:38:24.788 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:28:52.342560+0000", 2024-09-16T09:38:24.788 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 572.34887007600003, 2024-09-16T09:38:24.788 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.69139150400000005, 2024-09-16T09:38:24.788 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:38:24.788 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:38:24.788 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:38:24.789 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:38:24.789 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.24698", 2024-09-16T09:38:24.789 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:38:24.789 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 203641 2024-09-16T09:38:24.789 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.789 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:38:24.789 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.790 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:38:24.790 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.342560+0000", 2024-09-16T09:38:24.790 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:38:24.791 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.791 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.791 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:38:24.791 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.342560+0000", 2024-09-16T09:38:24.791 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:38:24.791 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.791 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.791 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:38:24.791 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.342561+0000", 2024-09-16T09:38:24.791 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.285e-06 2024-09-16T09:38:24.791 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.791 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.791 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:38:24.792 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.342566+0000", 2024-09-16T09:38:24.792 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.972e-06 2024-09-16T09:38:24.792 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.792 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.792 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:38:24.792 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.342577+0000", 2024-09-16T09:38:24.792 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0961999999999999e-05 2024-09-16T09:38:24.792 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.792 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.792 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:38:24.792 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.342585+0000", 2024-09-16T09:38:24.792 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.7919999999999993e-06 2024-09-16T09:38:24.792 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.793 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.793 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:38:24.793 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.342652+0000", 2024-09-16T09:38:24.793 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.7025000000000003e-05 2024-09-16T09:38:24.793 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.793 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.793 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:38:24.793 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.342663+0000", 2024-09-16T09:38:24.793 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1562999999999999e-05 2024-09-16T09:38:24.793 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.793 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.793 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:38:24.793 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.023982+0000", 2024-09-16T09:38:24.793 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.68131918899999999 2024-09-16T09:38:24.794 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.794 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.794 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:38:24.794 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.024072+0000", 2024-09-16T09:38:24.794 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.9392000000000001e-05 2024-09-16T09:38:24.794 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.794 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.794 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 3,5", 2024-09-16T09:38:24.794 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.024161+0000", 2024-09-16T09:38:24.794 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.9412000000000004e-05 2024-09-16T09:38:24.794 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.794 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.794 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:38:24.794 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.028707+0000", 2024-09-16T09:38:24.795 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.004546161 2024-09-16T09:38:24.795 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.795 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.795 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:38:24.795 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.033826+0000", 2024-09-16T09:38:24.795 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0051183080000000002 2024-09-16T09:38:24.795 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.795 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.795 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:38:24.795 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.033887+0000", 2024-09-16T09:38:24.795 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1198999999999995e-05 2024-09-16T09:38:24.795 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.795 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.796 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:38:24.796 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.033923+0000", 2024-09-16T09:38:24.796 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6049e-05 2024-09-16T09:38:24.796 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.796 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.796 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:38:24.796 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.033951+0000", 2024-09-16T09:38:24.796 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.8195e-05 2024-09-16T09:38:24.796 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:38:24.796 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:38:24.796 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:38:24.796 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.796 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.797 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.24698.0:203652 7.a 7:5453db86:::benchmark_data_smithi070_78438_object77521:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136)", 2024-09-16T09:38:24.797 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:28:52.357854+0000", 2024-09-16T09:38:24.797 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 572.33357526600003, 2024-09-16T09:38:24.797 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.67619149199999995, 2024-09-16T09:38:24.797 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:38:24.797 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:38:24.797 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:38:24.797 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:38:24.797 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.24698", 2024-09-16T09:38:24.797 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:38:24.797 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 203652 2024-09-16T09:38:24.797 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.798 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:38:24.798 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.798 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:38:24.798 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.357854+0000", 2024-09-16T09:38:24.798 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:38:24.798 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.798 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.798 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:38:24.798 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.357854+0000", 2024-09-16T09:38:24.798 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:38:24.798 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.798 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.798 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:38:24.798 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.357855+0000", 2024-09-16T09:38:24.798 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.023e-06 2024-09-16T09:38:24.799 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.799 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.799 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:38:24.799 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.357860+0000", 2024-09-16T09:38:24.799 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2939999999999999e-06 2024-09-16T09:38:24.799 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.799 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.799 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:38:24.799 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.357869+0000", 2024-09-16T09:38:24.799 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.1030000000000008e-06 2024-09-16T09:38:24.799 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.799 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.799 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:38:24.799 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.357874+0000", 2024-09-16T09:38:24.800 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0370000000000002e-06 2024-09-16T09:38:24.800 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.800 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.800 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:38:24.800 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.357930+0000", 2024-09-16T09:38:24.800 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.6468999999999998e-05 2024-09-16T09:38:24.800 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.800 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.800 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:38:24.800 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.357940+0000", 2024-09-16T09:38:24.800 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.5850000000000006e-06 2024-09-16T09:38:24.800 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.800 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.801 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:38:24.801 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.026230+0000", 2024-09-16T09:38:24.801 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.66828984300000005 2024-09-16T09:38:24.801 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.801 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.801 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:38:24.801 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.026333+0000", 2024-09-16T09:38:24.801 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00010361099999999999 2024-09-16T09:38:24.801 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.801 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.801 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 3,5", 2024-09-16T09:38:24.801 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.026438+0000", 2024-09-16T09:38:24.801 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00010440700000000001 2024-09-16T09:38:24.801 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.802 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.802 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:38:24.802 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.033500+0000", 2024-09-16T09:38:24.802 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0070627210000000001 2024-09-16T09:38:24.802 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.802 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.802 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:38:24.802 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.033857+0000", 2024-09-16T09:38:24.802 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00035613700000000002 2024-09-16T09:38:24.802 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.802 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.802 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:38:24.802 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.033982+0000", 2024-09-16T09:38:24.802 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00012542900000000001 2024-09-16T09:38:24.803 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.803 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.803 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:38:24.803 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.034018+0000", 2024-09-16T09:38:24.803 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5757999999999999e-05 2024-09-16T09:38:24.803 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.803 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.803 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:38:24.803 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.034046+0000", 2024-09-16T09:38:24.803 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.8075000000000001e-05 2024-09-16T09:38:24.803 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:38:24.803 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:38:24.803 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:38:24.803 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.803 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.804 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.24698.0:203750 7.a 7:52e49071:::benchmark_data_smithi070_78438_object77619:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136)", 2024-09-16T09:38:24.804 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:28:52.581478+0000", 2024-09-16T09:38:24.804 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 572.10995145100003, 2024-09-16T09:38:24.804 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.45793680799999997, 2024-09-16T09:38:24.804 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:38:24.804 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:38:24.804 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:38:24.804 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:38:24.804 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.24698", 2024-09-16T09:38:24.804 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:38:24.804 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 203750 2024-09-16T09:38:24.804 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.804 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:38:24.805 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.805 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:38:24.805 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.581478+0000", 2024-09-16T09:38:24.805 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:38:24.805 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.805 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.805 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:38:24.805 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.581478+0000", 2024-09-16T09:38:24.805 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:38:24.805 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.805 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.805 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:38:24.805 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.581480+0000", 2024-09-16T09:38:24.806 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.08e-06 2024-09-16T09:38:24.806 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.806 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.806 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:38:24.806 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.581486+0000", 2024-09-16T09:38:24.806 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8279999999999999e-06 2024-09-16T09:38:24.806 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.806 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.806 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:38:24.806 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.581502+0000", 2024-09-16T09:38:24.806 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5852999999999999e-05 2024-09-16T09:38:24.806 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.806 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.807 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:38:24.807 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.581510+0000", 2024-09-16T09:38:24.807 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.0700000000000007e-06 2024-09-16T09:38:24.807 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.807 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.807 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:38:24.807 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.581576+0000", 2024-09-16T09:38:24.807 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.6338000000000001e-05 2024-09-16T09:38:24.807 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.807 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.807 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:38:24.807 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.581587+0000", 2024-09-16T09:38:24.807 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0725e-05 2024-09-16T09:38:24.807 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.807 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.808 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:38:24.808 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.028904+0000", 2024-09-16T09:38:24.808 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.44731681699999998 2024-09-16T09:38:24.808 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.808 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.808 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:38:24.808 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.029003+0000", 2024-09-16T09:38:24.808 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.8913000000000007e-05 2024-09-16T09:38:24.808 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.808 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.808 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 3,5", 2024-09-16T09:38:24.808 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.029098+0000", 2024-09-16T09:38:24.808 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.5378999999999996e-05 2024-09-16T09:38:24.808 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.809 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.809 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:38:24.809 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.033546+0000", 2024-09-16T09:38:24.809 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0044476680000000001 2024-09-16T09:38:24.809 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.809 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.809 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:38:24.809 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.034091+0000", 2024-09-16T09:38:24.809 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00054530099999999999 2024-09-16T09:38:24.809 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.809 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.809 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:38:24.809 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.039290+0000", 2024-09-16T09:38:24.809 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0051983799999999998 2024-09-16T09:38:24.809 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.810 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.810 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:38:24.810 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.039347+0000", 2024-09-16T09:38:24.810 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7201999999999999e-05 2024-09-16T09:38:24.810 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.810 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.810 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:38:24.810 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.039415+0000", 2024-09-16T09:38:24.810 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.8254000000000001e-05 2024-09-16T09:38:24.810 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:38:24.810 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:38:24.810 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:38:24.810 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.810 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.811 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.24698.0:203753 7.a 7:5a1d367b:::benchmark_data_smithi070_78438_object77622:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e136)", 2024-09-16T09:38:24.811 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:28:52.587237+0000", 2024-09-16T09:38:24.811 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 572.10419274000003, 2024-09-16T09:38:24.811 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.45243622999999999, 2024-09-16T09:38:24.811 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:38:24.811 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:38:24.811 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:38:24.811 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:38:24.811 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.24698", 2024-09-16T09:38:24.811 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3536822056", 2024-09-16T09:38:24.811 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 203753 2024-09-16T09:38:24.811 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.812 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:38:24.812 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.812 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:38:24.812 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.587237+0000", 2024-09-16T09:38:24.812 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:38:24.812 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.812 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.812 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:38:24.812 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.587237+0000", 2024-09-16T09:38:24.812 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:38:24.812 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.812 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.812 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:38:24.812 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.587238+0000", 2024-09-16T09:38:24.813 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2139999999999999e-06 2024-09-16T09:38:24.813 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.813 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.813 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:38:24.813 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.587242+0000", 2024-09-16T09:38:24.813 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2259999999999999e-06 2024-09-16T09:38:24.813 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.813 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.813 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:38:24.813 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.587254+0000", 2024-09-16T09:38:24.813 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2069000000000001e-05 2024-09-16T09:38:24.813 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.813 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.813 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:38:24.814 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.587260+0000", 2024-09-16T09:38:24.814 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.835e-06 2024-09-16T09:38:24.814 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.814 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.814 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:38:24.814 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.587318+0000", 2024-09-16T09:38:24.814 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7497000000000002e-05 2024-09-16T09:38:24.814 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.814 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.814 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:38:24.814 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:52.587329+0000", 2024-09-16T09:38:24.814 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1158000000000001e-05 2024-09-16T09:38:24.814 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.815 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.815 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:38:24.815 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.031439+0000", 2024-09-16T09:38:24.815 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.444110216 2024-09-16T09:38:24.815 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.815 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.815 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:38:24.815 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.031537+0000", 2024-09-16T09:38:24.815 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.7628999999999996e-05 2024-09-16T09:38:24.815 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.815 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.815 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 3,5", 2024-09-16T09:38:24.815 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.031628+0000", 2024-09-16T09:38:24.816 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.0997999999999998e-05 2024-09-16T09:38:24.816 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.816 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.816 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:38:24.816 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.039223+0000", 2024-09-16T09:38:24.816 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0075953080000000003 2024-09-16T09:38:24.816 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.816 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.816 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:38:24.816 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.039460+0000", 2024-09-16T09:38:24.816 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00023686599999999999 2024-09-16T09:38:24.816 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.816 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.816 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:38:24.817 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.039544+0000", 2024-09-16T09:38:24.817 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.3762000000000002e-05 2024-09-16T09:38:24.817 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.817 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.817 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:38:24.817 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.039636+0000", 2024-09-16T09:38:24.817 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.1879000000000006e-05 2024-09-16T09:38:24.817 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.817 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.817 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:38:24.817 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:28:53.039673+0000", 2024-09-16T09:38:24.817 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7573000000000001e-05 2024-09-16T09:38:24.817 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:38:24.817 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:38:24.818 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:38:24.818 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.818 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.818 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:64857 9.12 9:4d910f5d:::benchmark_data_smithi070_137000_object64856:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e252)", 2024-09-16T09:38:24.818 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:34:09.120564+0000", 2024-09-16T09:38:24.818 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 255.57086567299999, 2024-09-16T09:38:24.818 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.19693081400000001, 2024-09-16T09:38:24.818 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:38:24.818 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:38:24.818 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:38:24.818 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:38:24.818 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:38:24.818 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:38:24.818 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 64857 2024-09-16T09:38:24.819 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.819 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:38:24.819 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.819 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:38:24.819 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:09.120564+0000", 2024-09-16T09:38:24.819 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:38:24.819 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.819 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.819 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:38:24.819 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:09.120564+0000", 2024-09-16T09:38:24.819 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:38:24.819 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.820 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.820 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:38:24.820 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:09.120566+0000", 2024-09-16T09:38:24.820 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7549999999999999e-06 2024-09-16T09:38:24.820 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.820 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.820 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:38:24.820 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:09.120590+0000", 2024-09-16T09:38:24.820 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3975e-05 2024-09-16T09:38:24.820 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.820 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.820 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:38:24.820 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:09.120603+0000", 2024-09-16T09:38:24.820 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3498e-05 2024-09-16T09:38:24.821 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.821 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.821 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:38:24.821 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:09.120610+0000", 2024-09-16T09:38:24.821 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.2019999999999998e-06 2024-09-16T09:38:24.821 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.821 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.821 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:38:24.821 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:09.120765+0000", 2024-09-16T09:38:24.821 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00015469 2024-09-16T09:38:24.821 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.821 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.821 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:38:24.822 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:09.120908+0000", 2024-09-16T09:38:24.822 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00014259599999999999 2024-09-16T09:38:24.822 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.822 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.822 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 0,1,5", 2024-09-16T09:38:24.822 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:09.121056+0000", 2024-09-16T09:38:24.822 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00014818200000000001 2024-09-16T09:38:24.822 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.822 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.822 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:38:24.822 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:09.122492+0000", 2024-09-16T09:38:24.822 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0014365840000000001 2024-09-16T09:38:24.822 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.823 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.823 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:38:24.823 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:09.125428+0000", 2024-09-16T09:38:24.823 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0029351500000000001 2024-09-16T09:38:24.823 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.823 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.823 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:38:24.823 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:09.129344+0000", 2024-09-16T09:38:24.823 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0039167940000000004 2024-09-16T09:38:24.823 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.823 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.823 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:38:24.823 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:09.316575+0000", 2024-09-16T09:38:24.823 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.187230956 2024-09-16T09:38:24.824 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.824 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.824 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:38:24.824 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:09.316805+0000", 2024-09-16T09:38:24.824 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00022933099999999999 2024-09-16T09:38:24.824 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.824 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.824 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:38:24.824 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:09.317495+0000", 2024-09-16T09:38:24.824 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00069010100000000004 2024-09-16T09:38:24.824 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:38:24.824 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:38:24.824 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:38:24.825 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.825 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.825 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop_reply(client.15588.0:64857 9.12 e252/246 ondisk, result = 0)", 2024-09-16T09:38:24.825 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:34:09.129465+0000", 2024-09-16T09:38:24.825 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 255.56196446199999, 2024-09-16T09:38:24.825 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.18804085700000001, 2024-09-16T09:38:24.825 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:38:24.825 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:38:24.825 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T09:38:24.825 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:38:24.825 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.825 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:38:24.825 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:09.129465+0000", 2024-09-16T09:38:24.826 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:38:24.826 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.826 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.826 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:38:24.826 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:09.129465+0000", 2024-09-16T09:38:24.826 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:38:24.826 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.826 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.826 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:38:24.826 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:09.129466+0000", 2024-09-16T09:38:24.826 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5599999999999995e-07 2024-09-16T09:38:24.826 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.826 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.826 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:38:24.827 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:09.129469+0000", 2024-09-16T09:38:24.827 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3610000000000001e-06 2024-09-16T09:38:24.827 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.827 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.827 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:38:24.827 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:09.129477+0000", 2024-09-16T09:38:24.827 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.3289999999999994e-06 2024-09-16T09:38:24.827 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.827 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.827 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:38:24.827 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:09.129483+0000", 2024-09-16T09:38:24.827 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7599999999999999e-06 2024-09-16T09:38:24.827 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.827 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.828 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:38:24.828 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:09.316560+0000", 2024-09-16T09:38:24.828 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.18707707800000001 2024-09-16T09:38:24.828 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.828 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.828 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:38:24.828 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:09.316569+0000", 2024-09-16T09:38:24.828 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.4810000000000006e-06 2024-09-16T09:38:24.828 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.828 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.828 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:38:24.828 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:09.317506+0000", 2024-09-16T09:38:24.829 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00093729200000000005 2024-09-16T09:38:24.829 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:38:24.829 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:38:24.829 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:38:24.829 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.829 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.829 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.15975.0:33579 10.9 e386/376 10:99683988:::benchmark_data_smithi070_166744_object33578:head v 386'2079, mlcod=386'2076)", 2024-09-16T09:38:24.829 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:38:09.640462+0000", 2024-09-16T09:38:24.829 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 15.050967657999999, 2024-09-16T09:38:24.829 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.28017080700000002, 2024-09-16T09:38:24.829 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:38:24.829 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:38:24.829 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:38:24.829 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:38:24.830 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.830 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:38:24.830 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:38:09.640462+0000", 2024-09-16T09:38:24.830 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:38:24.830 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.830 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.830 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:38:24.830 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:38:09.640462+0000", 2024-09-16T09:38:24.830 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:38:24.830 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.830 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.830 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:38:24.830 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:38:09.640463+0000", 2024-09-16T09:38:24.831 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.5499999999999997e-07 2024-09-16T09:38:24.831 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.831 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.831 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:38:24.831 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:38:09.640526+0000", 2024-09-16T09:38:24.831 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.3503999999999994e-05 2024-09-16T09:38:24.831 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.831 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.831 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:38:24.831 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:38:09.640535+0000", 2024-09-16T09:38:24.831 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.2029999999999998e-06 2024-09-16T09:38:24.831 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.831 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.831 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:38:24.832 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:38:09.640542+0000", 2024-09-16T09:38:24.832 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.5010000000000001e-06 2024-09-16T09:38:24.832 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.832 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.832 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:38:24.832 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:38:09.917405+0000", 2024-09-16T09:38:24.832 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.27686350799999998 2024-09-16T09:38:24.832 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.832 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.832 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:38:24.832 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:38:09.917429+0000", 2024-09-16T09:38:24.832 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.4017000000000001e-05 2024-09-16T09:38:24.832 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.833 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.833 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:38:24.833 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:38:09.920588+0000", 2024-09-16T09:38:24.833 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0031587820000000002 2024-09-16T09:38:24.833 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.833 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.833 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:38:24.833 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:38:09.920633+0000", 2024-09-16T09:38:24.833 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.4536999999999997e-05 2024-09-16T09:38:24.833 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:38:24.833 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:38:24.833 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:38:24.833 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.834 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.834 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.15975.0:33595 10.f e386/376 10:fdf68c83:::benchmark_data_smithi070_166744_object33594:head v 386'2134, mlcod=386'2132)", 2024-09-16T09:38:24.834 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:38:09.647007+0000", 2024-09-16T09:38:24.834 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 15.044422472000001, 2024-09-16T09:38:24.834 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.27538453699999998, 2024-09-16T09:38:24.834 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:38:24.834 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:38:24.834 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:38:24.834 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:38:24.834 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.834 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:38:24.834 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:38:09.647007+0000", 2024-09-16T09:38:24.834 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:38:24.834 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.835 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.835 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:38:24.835 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:38:09.647007+0000", 2024-09-16T09:38:24.835 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:38:24.835 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.835 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.835 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:38:24.835 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:38:09.647008+0000", 2024-09-16T09:38:24.835 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.043e-06 2024-09-16T09:38:24.835 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.835 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.835 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:38:24.835 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:38:09.647046+0000", 2024-09-16T09:38:24.836 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7463999999999999e-05 2024-09-16T09:38:24.836 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.836 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.836 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:38:24.836 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:38:09.917354+0000", 2024-09-16T09:38:24.836 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.27030826499999999 2024-09-16T09:38:24.836 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.836 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.836 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:38:24.836 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:38:09.917365+0000", 2024-09-16T09:38:24.836 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1069e-05 2024-09-16T09:38:24.836 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.836 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.836 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:38:24.836 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:38:09.917423+0000", 2024-09-16T09:38:24.837 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8158000000000001e-05 2024-09-16T09:38:24.837 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.837 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.837 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:38:24.837 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:38:09.917437+0000", 2024-09-16T09:38:24.837 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.4046000000000001e-05 2024-09-16T09:38:24.837 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.837 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.837 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:38:24.837 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:38:09.922337+0000", 2024-09-16T09:38:24.837 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0048999910000000002 2024-09-16T09:38:24.837 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:24.837 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:24.837 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:38:24.838 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:38:09.922392+0000", 2024-09-16T09:38:24.838 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4500999999999999e-05 2024-09-16T09:38:24.838 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:38:24.838 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:38:24.838 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:38:24.838 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:38:24.838 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:38:24.838 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T09:38:24.840 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:24.840 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:24.840 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:38:24.941 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:38:24.984 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:24.985 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:24.986 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:38:25.086 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:38:25.125 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:25.126 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:25.126 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:38:25.227 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:38:25.260 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:25.261 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:25.262 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:38:25.362 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:38:25.400 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:25.401 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:25.402 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:38:25.503 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:38:25.538 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:25.539 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:25.539 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:38:25.541 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 29 16 60073 60057 129.394 90.875 0.0106158 0.00772306 2024-09-16T09:38:25.556 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-16T09:38:25.594 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:25 smithi070 ceph-mon[40691]: pgmap v706: 168 pgs: 168 active+clean; 3.4 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 977 B/s rd, 100 MiB/s wr, 1.60k op/s 2024-09-16T09:38:25.594 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:25 smithi070 ceph-mon[40691]: 6.18 scrub starts 2024-09-16T09:38:25.594 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:25 smithi070 ceph-mon[40691]: 6.18 scrub ok 2024-09-16T09:38:25.594 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:25 smithi070 ceph-mon[40691]: 4.14 scrub starts 2024-09-16T09:38:25.594 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:25 smithi070 ceph-mon[40691]: 4.14 scrub ok 2024-09-16T09:38:25.594 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:25 smithi070 ceph-mon[30388]: pgmap v706: 168 pgs: 168 active+clean; 3.4 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 977 B/s rd, 100 MiB/s wr, 1.60k op/s 2024-09-16T09:38:25.595 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:25 smithi070 ceph-mon[30388]: 6.18 scrub starts 2024-09-16T09:38:25.595 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:25 smithi070 ceph-mon[30388]: 6.18 scrub ok 2024-09-16T09:38:25.595 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:25 smithi070 ceph-mon[30388]: 4.14 scrub starts 2024-09-16T09:38:25.595 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:25 smithi070 ceph-mon[30388]: 4.14 scrub ok 2024-09-16T09:38:25.640 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:38:25.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:25 smithi084 ceph-mon[32629]: pgmap v706: 168 pgs: 168 active+clean; 3.4 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 977 B/s rd, 100 MiB/s wr, 1.60k op/s 2024-09-16T09:38:25.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:25 smithi084 ceph-mon[32629]: 6.18 scrub starts 2024-09-16T09:38:25.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:25 smithi084 ceph-mon[32629]: 6.18 scrub ok 2024-09-16T09:38:25.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:25 smithi084 ceph-mon[32629]: 4.14 scrub starts 2024-09-16T09:38:25.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:25 smithi084 ceph-mon[32629]: 4.14 scrub ok 2024-09-16T09:38:25.703 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:25.704 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:25.704 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:38:25.724 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:38:25.727 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2024-09-16T09:38:25.805 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:38:25.852 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:25.856 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:25.856 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:38:25.957 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:38:25.990 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:25.990 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:25.990 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:38:26.091 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:38:26.123 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:26.124 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:26.125 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:38:26.225 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:38:26.263 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:26.263 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:26.264 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:38:26.364 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:38:26.404 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:26.404 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:26.405 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:38:26.506 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:38:26.538 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:26.539 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:26.539 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:38:26.541 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 30 16 61612 61596 128.287 96.1875 0.0133786 0.00778999 2024-09-16T09:38:26.640 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:38:26.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:26 smithi070 ceph-mon[30388]: 5.b scrub starts 2024-09-16T09:38:26.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:26 smithi070 ceph-mon[30388]: 5.b scrub ok 2024-09-16T09:38:26.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:26 smithi070 ceph-mon[30388]: 6.16 scrub starts 2024-09-16T09:38:26.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:26 smithi070 ceph-mon[30388]: 6.16 scrub ok 2024-09-16T09:38:26.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:26 smithi070 ceph-mon[30388]: 6.6 scrub starts 2024-09-16T09:38:26.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:26 smithi070 ceph-mon[30388]: 6.6 scrub ok 2024-09-16T09:38:26.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:26 smithi070 ceph-mon[40691]: 5.b scrub starts 2024-09-16T09:38:26.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:26 smithi070 ceph-mon[40691]: 5.b scrub ok 2024-09-16T09:38:26.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:26 smithi070 ceph-mon[40691]: 6.16 scrub starts 2024-09-16T09:38:26.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:26 smithi070 ceph-mon[40691]: 6.16 scrub ok 2024-09-16T09:38:26.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:26 smithi070 ceph-mon[40691]: 6.6 scrub starts 2024-09-16T09:38:26.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:26 smithi070 ceph-mon[40691]: 6.6 scrub ok 2024-09-16T09:38:26.676 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:26.677 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:26.678 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:38:26.779 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:38:26.819 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:26.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:26.820 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:38:26.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:26 smithi084 ceph-mon[32629]: 5.b scrub starts 2024-09-16T09:38:26.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:26 smithi084 ceph-mon[32629]: 5.b scrub ok 2024-09-16T09:38:26.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:26 smithi084 ceph-mon[32629]: 6.16 scrub starts 2024-09-16T09:38:26.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:26 smithi084 ceph-mon[32629]: 6.16 scrub ok 2024-09-16T09:38:26.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:26 smithi084 ceph-mon[32629]: 6.6 scrub starts 2024-09-16T09:38:26.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:26 smithi084 ceph-mon[32629]: 6.6 scrub ok 2024-09-16T09:38:26.921 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:38:26.961 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:26.961 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:26.962 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:38:27.007 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.3 dump_ops_in_flight 2024-09-16T09:38:27.062 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:38:27.097 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:27.098 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:27.098 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:38:27.198 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:38:27.233 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:27.233 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:27.234 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:38:27.335 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:38:27.364 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:27.365 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:27.365 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:38:27.466 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:38:27.499 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:27.500 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:27.500 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:38:27.541 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 31 16 63177 63161 127.304 97.8125 0.00754117 0.00785031 2024-09-16T09:38:27.601 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:38:27.637 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:27.638 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:27.638 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:38:27.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:27 smithi070 ceph-mon[30388]: from='client.25982 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:38:27.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:27 smithi070 ceph-mon[30388]: pgmap v708: 168 pgs: 168 active+clean; 3.5 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 1.1 KiB/s rd, 102 MiB/s wr, 1.64k op/s 2024-09-16T09:38:27.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:27 smithi070 ceph-mon[30388]: 6.e scrub starts 2024-09-16T09:38:27.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:27 smithi070 ceph-mon[30388]: 6.e scrub ok 2024-09-16T09:38:27.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:27 smithi070 ceph-mon[30388]: 2.2 scrub starts 2024-09-16T09:38:27.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:27 smithi070 ceph-mon[30388]: 2.2 scrub ok 2024-09-16T09:38:27.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:27 smithi070 ceph-mon[40691]: from='client.25982 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:38:27.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:27 smithi070 ceph-mon[40691]: pgmap v708: 168 pgs: 168 active+clean; 3.5 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 1.1 KiB/s rd, 102 MiB/s wr, 1.64k op/s 2024-09-16T09:38:27.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:27 smithi070 ceph-mon[40691]: 6.e scrub starts 2024-09-16T09:38:27.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:27 smithi070 ceph-mon[40691]: 6.e scrub ok 2024-09-16T09:38:27.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:27 smithi070 ceph-mon[40691]: 2.2 scrub starts 2024-09-16T09:38:27.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:27 smithi070 ceph-mon[40691]: 2.2 scrub ok 2024-09-16T09:38:27.739 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:38:27.783 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:27.783 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:27.784 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:38:27.885 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:38:27.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:27 smithi084 ceph-mon[32629]: from='client.25982 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:38:27.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:27 smithi084 ceph-mon[32629]: pgmap v708: 168 pgs: 168 active+clean; 3.5 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 1.1 KiB/s rd, 102 MiB/s wr, 1.64k op/s 2024-09-16T09:38:27.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:27 smithi084 ceph-mon[32629]: 6.e scrub starts 2024-09-16T09:38:27.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:27 smithi084 ceph-mon[32629]: 6.e scrub ok 2024-09-16T09:38:27.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:27 smithi084 ceph-mon[32629]: 2.2 scrub starts 2024-09-16T09:38:27.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:27 smithi084 ceph-mon[32629]: 2.2 scrub ok 2024-09-16T09:38:27.935 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:27.935 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:27.935 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:38:27.951 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":707,"stamp":"2024-09-16T09:38:24.338950+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3711900187,"num_objects":56829,"num_object_clones":0,"num_object_copies":170487,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56829,"num_whiteouts":0,"num_read":1935,"num_read_kb":1687,"num_write":57352,"num_write_kb":3624365,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":140,"num_bytes_recovered":333242,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":59058,"ondisk_log_size":59058,"up":504,"acting":504,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":506,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":15280840,"kb_used_data":10901180,"kb_used_omap":12,"kb_used_meta":4379507,"kb_avail":734680376,"statfs":{"total":767960285184,"available":752312705024,"internally_reserved":0,"allocated":11162808320,"data_stored":11157764601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":4484615730},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1083179008,"num_objects":16528,"num_object_clones":0,"num_object_copies":49584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16528,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":16528,"num_write_kb":1057792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.580998"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":712,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.370915+0000","last_change":"2024-09-16T09:32:39.384174+0000","last_active":"2024-09-16T09:38:20.370915+0000","last_peered":"2024-09-16T09:38:20.370915+0000","last_clean":"2024-09-16T09:38:20.370915+0000","last_became_active":"2024-09-16T09:25:01.709108+0000","last_became_peered":"2024-09-16T09:25:01.709108+0000","last_unstale":"2024-09-16T09:38:20.370915+0000","last_undegraded":"2024-09-16T09:38:20.370915+0000","last_fullsized":"2024-09-16T09:38:20.370915+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:39.384104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:32:39.384104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:20.331363+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"285'15","reported_seq":724,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.369203+0000","last_change":"2024-09-16T09:32:38.074301+0000","last_active":"2024-09-16T09:38:20.369203+0000","last_peered":"2024-09-16T09:38:20.369203+0000","last_clean":"2024-09-16T09:38:20.369203+0000","last_became_active":"2024-09-16T09:25:03.714635+0000","last_became_peered":"2024-09-16T09:25:03.714635+0000","last_unstale":"2024-09-16T09:38:20.369203+0000","last_undegraded":"2024-09-16T09:38:20.369203+0000","last_fullsized":"2024-09-16T09:38:20.369203+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:32:38.074231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:32:38.074231+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:07.303929+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":335,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.364152+0000","last_change":"2024-09-16T09:36:53.098680+0000","last_active":"2024-09-16T09:38:20.364152+0000","last_peered":"2024-09-16T09:38:20.364152+0000","last_clean":"2024-09-16T09:38:20.364152+0000","last_became_active":"2024-09-16T09:33:30.242238+0000","last_became_peered":"2024-09-16T09:33:30.242238+0000","last_unstale":"2024-09-16T09:38:20.364152+0000","last_undegraded":"2024-09-16T09:38:20.364152+0000","last_fullsized":"2024-09-16T09:38:20.364152+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:53.098635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:36:53.098635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":691,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.367272+0000","last_change":"2024-09-16T09:37:56.314518+0000","last_active":"2024-09-16T09:38:20.367272+0000","last_peered":"2024-09-16T09:38:20.367272+0000","last_clean":"2024-09-16T09:38:20.367272+0000","last_became_active":"2024-09-16T09:25:07.715442+0000","last_became_peered":"2024-09-16T09:25:07.715442+0000","last_unstale":"2024-09-16T09:38:20.367272+0000","last_undegraded":"2024-09-16T09:38:20.367272+0000","last_fullsized":"2024-09-16T09:38:20.367272+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:56.314468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:37:56.314468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:04.475570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":428,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.361277+0000","last_change":"2024-09-16T09:34:33.452784+0000","last_active":"2024-09-16T09:38:20.361277+0000","last_peered":"2024-09-16T09:38:20.361277+0000","last_clean":"2024-09-16T09:38:20.361277+0000","last_became_active":"2024-09-16T09:31:09.269342+0000","last_became_peered":"2024-09-16T09:31:09.269342+0000","last_unstale":"2024-09-16T09:38:20.361277+0000","last_undegraded":"2024-09-16T09:38:20.361277+0000","last_fullsized":"2024-09-16T09:38:20.361277+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:33.452706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:34:33.452706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:15.084990+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":569,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.361066+0000","last_change":"2024-09-16T09:33:08.816231+0000","last_active":"2024-09-16T09:38:20.361066+0000","last_peered":"2024-09-16T09:38:20.361066+0000","last_clean":"2024-09-16T09:38:20.361066+0000","last_became_active":"2024-09-16T09:31:09.275136+0000","last_became_peered":"2024-09-16T09:31:09.275136+0000","last_unstale":"2024-09-16T09:38:20.361066+0000","last_undegraded":"2024-09-16T09:38:20.361066+0000","last_fullsized":"2024-09-16T09:38:20.361066+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:08.816174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:08.816174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:08.989282+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.19","version":"285'27","reported_seq":410,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.373897+0000","last_change":"2024-09-16T09:37:29.683657+0000","last_active":"2024-09-16T09:38:20.373897+0000","last_peered":"2024-09-16T09:38:20.373897+0000","last_clean":"2024-09-16T09:38:20.373897+0000","last_became_active":"2024-09-16T09:33:21.337284+0000","last_became_peered":"2024-09-16T09:33:21.337284+0000","last_unstale":"2024-09-16T09:38:20.373897+0000","last_undegraded":"2024-09-16T09:38:20.373897+0000","last_fullsized":"2024-09-16T09:38:20.373897+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'27","last_scrub_stamp":"2024-09-16T09:37:29.683568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:37:29.683568+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:35.791977+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":703,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.367058+0000","last_change":"2024-09-16T09:32:44.114475+0000","last_active":"2024-09-16T09:38:20.367058+0000","last_peered":"2024-09-16T09:38:20.367058+0000","last_clean":"2024-09-16T09:38:20.367058+0000","last_became_active":"2024-09-16T09:25:05.717236+0000","last_became_peered":"2024-09-16T09:25:05.717236+0000","last_unstale":"2024-09-16T09:38:20.367058+0000","last_undegraded":"2024-09-16T09:38:20.367058+0000","last_fullsized":"2024-09-16T09:38:20.367058+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:44.114407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:32:44.114407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:03.327373+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":335,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.369194+0000","last_change":"2024-09-16T09:33:36.655594+0000","last_active":"2024-09-16T09:38:20.369194+0000","last_peered":"2024-09-16T09:38:20.369194+0000","last_clean":"2024-09-16T09:38:20.369194+0000","last_became_active":"2024-09-16T09:33:30.247733+0000","last_became_peered":"2024-09-16T09:33:30.247733+0000","last_unstale":"2024-09-16T09:38:20.369194+0000","last_undegraded":"2024-09-16T09:38:20.369194+0000","last_fullsized":"2024-09-16T09:38:20.369194+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:36.655236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:36.655236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:00.446453+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1a","version":"285'15","reported_seq":724,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.367590+0000","last_change":"2024-09-16T09:32:48.185484+0000","last_active":"2024-09-16T09:38:20.367590+0000","last_peered":"2024-09-16T09:38:20.367590+0000","last_clean":"2024-09-16T09:38:20.367590+0000","last_became_active":"2024-09-16T09:25:03.709147+0000","last_became_peered":"2024-09-16T09:25:03.709147+0000","last_unstale":"2024-09-16T09:38:20.367590+0000","last_undegraded":"2024-09-16T09:38:20.367590+0000","last_fullsized":"2024-09-16T09:38:20.367590+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:32:48.185380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:32:48.185380+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:12.337650+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":694,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.364422+0000","last_change":"2024-09-16T09:36:51.134364+0000","last_active":"2024-09-16T09:38:20.364422+0000","last_peered":"2024-09-16T09:38:20.364422+0000","last_clean":"2024-09-16T09:38:20.364422+0000","last_became_active":"2024-09-16T09:25:05.718134+0000","last_became_peered":"2024-09-16T09:25:05.718134+0000","last_unstale":"2024-09-16T09:38:20.364422+0000","last_undegraded":"2024-09-16T09:38:20.364422+0000","last_fullsized":"2024-09-16T09:38:20.364422+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:51.134314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:36:51.134314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"66'1","reported_seq":700,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.367955+0000","last_change":"2024-09-16T09:33:38.705001+0000","last_active":"2024-09-16T09:38:20.367955+0000","last_peered":"2024-09-16T09:38:20.367955+0000","last_clean":"2024-09-16T09:38:20.367955+0000","last_became_active":"2024-09-16T09:25:07.716558+0000","last_became_peered":"2024-09-16T09:25:07.716558+0000","last_unstale":"2024-09-16T09:38:20.367955+0000","last_undegraded":"2024-09-16T09:38:20.367955+0000","last_fullsized":"2024-09-16T09:38:20.367955+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-16T09:33:38.704868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:33:38.704868+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":694,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.364240+0000","last_change":"2024-09-16T09:33:19.326765+0000","last_active":"2024-09-16T09:38:20.364240+0000","last_peered":"2024-09-16T09:38:20.364240+0000","last_clean":"2024-09-16T09:38:20.364240+0000","last_became_active":"2024-09-16T09:25:01.696939+0000","last_became_peered":"2024-09-16T09:25:01.696939+0000","last_unstale":"2024-09-16T09:38:20.364240+0000","last_undegraded":"2024-09-16T09:38:20.364240+0000","last_fullsized":"2024-09-16T09:38:20.364240+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:19.326695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:19.326695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:11.230887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"285'7","reported_seq":712,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.367435+0000","last_change":"2024-09-16T09:35:39.975915+0000","last_active":"2024-09-16T09:38:20.367435+0000","last_peered":"2024-09-16T09:38:20.367435+0000","last_clean":"2024-09-16T09:38:20.367435+0000","last_became_active":"2024-09-16T09:25:03.716136+0000","last_became_peered":"2024-09-16T09:25:03.716136+0000","last_unstale":"2024-09-16T09:38:20.367435+0000","last_undegraded":"2024-09-16T09:38:20.367435+0000","last_fullsized":"2024-09-16T09:38:20.367435+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'7","last_scrub_stamp":"2024-09-16T09:35:39.975840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:35:39.975840+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:00.297013+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":10,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":685,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.365395+0000","last_change":"2024-09-16T09:28:52.746046+0000","last_active":"2024-09-16T09:38:20.365395+0000","last_peered":"2024-09-16T09:38:20.365395+0000","last_clean":"2024-09-16T09:38:20.365395+0000","last_became_active":"2024-09-16T09:25:05.718281+0000","last_became_peered":"2024-09-16T09:25:05.718281+0000","last_unstale":"2024-09-16T09:38:20.365395+0000","last_undegraded":"2024-09-16T09:38:20.365395+0000","last_fullsized":"2024-09-16T09:38:20.365395+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:52.745999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:28:52.745999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:04.420096+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":712,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.364201+0000","last_change":"2024-09-16T09:33:46.366042+0000","last_active":"2024-09-16T09:38:20.364201+0000","last_peered":"2024-09-16T09:38:20.364201+0000","last_clean":"2024-09-16T09:38:20.364201+0000","last_became_active":"2024-09-16T09:31:09.275656+0000","last_became_peered":"2024-09-16T09:31:09.275656+0000","last_unstale":"2024-09-16T09:38:20.364201+0000","last_undegraded":"2024-09-16T09:38:20.364201+0000","last_fullsized":"2024-09-16T09:38:20.364201+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:46.365972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:33:46.365972+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":73,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":700,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.369366+0000","last_change":"2024-09-16T09:33:48.322010+0000","last_active":"2024-09-16T09:38:20.369366+0000","last_peered":"2024-09-16T09:38:20.369366+0000","last_clean":"2024-09-16T09:38:20.369366+0000","last_became_active":"2024-09-16T09:25:07.720793+0000","last_became_peered":"2024-09-16T09:25:07.720793+0000","last_unstale":"2024-09-16T09:38:20.369366+0000","last_undegraded":"2024-09-16T09:38:20.369366+0000","last_fullsized":"2024-09-16T09:38:20.369366+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:48.321895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:33:48.321895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:41.370700+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":451,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.360171+0000","last_change":"2024-09-16T09:34:33.866299+0000","last_active":"2024-09-16T09:38:20.360171+0000","last_peered":"2024-09-16T09:38:20.360171+0000","last_clean":"2024-09-16T09:38:20.360171+0000","last_became_active":"2024-09-16T09:30:39.201824+0000","last_became_peered":"2024-09-16T09:30:39.201824+0000","last_unstale":"2024-09-16T09:38:20.360171+0000","last_undegraded":"2024-09-16T09:38:20.360171+0000","last_fullsized":"2024-09-16T09:38:20.360171+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:33.866142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:34:33.866142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:40.929489+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"285'27","reported_seq":799,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.365330+0000","last_change":"2024-09-16T09:36:54.484021+0000","last_active":"2024-09-16T09:38:20.365330+0000","last_peered":"2024-09-16T09:38:20.365330+0000","last_clean":"2024-09-16T09:38:20.365330+0000","last_became_active":"2024-09-16T09:33:30.249798+0000","last_became_peered":"2024-09-16T09:33:30.249798+0000","last_unstale":"2024-09-16T09:38:20.365330+0000","last_undegraded":"2024-09-16T09:38:20.365330+0000","last_fullsized":"2024-09-16T09:38:20.365330+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'27","last_scrub_stamp":"2024-09-16T09:36:54.483966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:36:54.483966+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:07.794557+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":334,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.368245+0000","last_change":"2024-09-16T09:33:43.326933+0000","last_active":"2024-09-16T09:38:20.368245+0000","last_peered":"2024-09-16T09:38:20.368245+0000","last_clean":"2024-09-16T09:38:20.368245+0000","last_became_active":"2024-09-16T09:33:30.251587+0000","last_became_peered":"2024-09-16T09:33:30.251587+0000","last_unstale":"2024-09-16T09:38:20.368245+0000","last_undegraded":"2024-09-16T09:38:20.368245+0000","last_fullsized":"2024-09-16T09:38:20.368245+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:43.326852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:43.326852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:42.368973+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,0],"acting":[4,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":548,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.361318+0000","last_change":"2024-09-16T09:34:37.804874+0000","last_active":"2024-09-16T09:38:20.361318+0000","last_peered":"2024-09-16T09:38:20.361318+0000","last_clean":"2024-09-16T09:38:20.361318+0000","last_became_active":"2024-09-16T09:28:52.974255+0000","last_became_peered":"2024-09-16T09:28:52.974255+0000","last_unstale":"2024-09-16T09:38:20.361318+0000","last_undegraded":"2024-09-16T09:38:20.361318+0000","last_fullsized":"2024-09-16T09:38:20.361318+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:37.804805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:34:37.804805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:00.917819+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":711,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.367726+0000","last_change":"2024-09-16T09:35:23.751992+0000","last_active":"2024-09-16T09:38:20.367726+0000","last_peered":"2024-09-16T09:38:20.367726+0000","last_clean":"2024-09-16T09:38:20.367726+0000","last_became_active":"2024-09-16T09:25:01.706491+0000","last_became_peered":"2024-09-16T09:25:01.706491+0000","last_unstale":"2024-09-16T09:38:20.367726+0000","last_undegraded":"2024-09-16T09:38:20.367726+0000","last_fullsized":"2024-09-16T09:38:20.367726+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:35:23.751946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:35:23.751946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:15.256399+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"285'18","reported_seq":386,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.368371+0000","last_change":"2024-09-16T09:33:39.284526+0000","last_active":"2024-09-16T09:38:20.368371+0000","last_peered":"2024-09-16T09:38:20.368371+0000","last_clean":"2024-09-16T09:38:20.368371+0000","last_became_active":"2024-09-16T09:33:30.251626+0000","last_became_peered":"2024-09-16T09:33:30.251626+0000","last_unstale":"2024-09-16T09:38:20.368371+0000","last_undegraded":"2024-09-16T09:38:20.368371+0000","last_fullsized":"2024-09-16T09:38:20.368371+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-16T09:33:39.284451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:33:39.284451+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:34.368282+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":33,"num_read_kb":24,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":687,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.367837+0000","last_change":"2024-09-16T09:33:38.238661+0000","last_active":"2024-09-16T09:38:20.367837+0000","last_peered":"2024-09-16T09:38:20.367837+0000","last_clean":"2024-09-16T09:38:20.367837+0000","last_became_active":"2024-09-16T09:25:05.710093+0000","last_became_peered":"2024-09-16T09:25:05.710093+0000","last_unstale":"2024-09-16T09:38:20.367837+0000","last_undegraded":"2024-09-16T09:38:20.367837+0000","last_fullsized":"2024-09-16T09:38:20.367837+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:38.238593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:38.238593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:36.342171+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":698,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.365732+0000","last_change":"2024-09-16T09:37:48.412421+0000","last_active":"2024-09-16T09:38:20.365732+0000","last_peered":"2024-09-16T09:38:20.365732+0000","last_clean":"2024-09-16T09:38:20.365732+0000","last_became_active":"2024-09-16T09:25:07.719380+0000","last_became_peered":"2024-09-16T09:25:07.719380+0000","last_unstale":"2024-09-16T09:38:20.365732+0000","last_undegraded":"2024-09-16T09:38:20.365732+0000","last_fullsized":"2024-09-16T09:38:20.365732+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-16T09:37:48.411054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:37:48.411054+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:01.043235+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":344,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.361696+0000","last_change":"2024-09-16T09:34:41.401691+0000","last_active":"2024-09-16T09:38:20.361696+0000","last_peered":"2024-09-16T09:38:20.361696+0000","last_clean":"2024-09-16T09:38:20.361696+0000","last_became_active":"2024-09-16T09:33:30.245048+0000","last_became_peered":"2024-09-16T09:33:30.245048+0000","last_unstale":"2024-09-16T09:38:20.361696+0000","last_undegraded":"2024-09-16T09:38:20.361696+0000","last_fullsized":"2024-09-16T09:38:20.361696+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:34:41.401641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:34:41.401641+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:03.993235+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"285'20","reported_seq":768,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.368450+0000","last_change":"2024-09-16T09:32:30.286077+0000","last_active":"2024-09-16T09:38:20.368450+0000","last_peered":"2024-09-16T09:38:20.368450+0000","last_clean":"2024-09-16T09:38:20.368450+0000","last_became_active":"2024-09-16T09:31:09.275050+0000","last_became_peered":"2024-09-16T09:31:09.275050+0000","last_unstale":"2024-09-16T09:38:20.368450+0000","last_undegraded":"2024-09-16T09:38:20.368450+0000","last_fullsized":"2024-09-16T09:38:20.368450+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-16T09:32:30.284919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:32:30.284919+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:02.456876+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"66'8","reported_seq":892,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.361357+0000","last_change":"2024-09-16T09:34:52.922123+0000","last_active":"2024-09-16T09:38:20.361357+0000","last_peered":"2024-09-16T09:38:20.361357+0000","last_clean":"2024-09-16T09:38:20.361357+0000","last_became_active":"2024-09-16T09:25:05.712516+0000","last_became_peered":"2024-09-16T09:25:05.712516+0000","last_unstale":"2024-09-16T09:38:20.361357+0000","last_undegraded":"2024-09-16T09:38:20.361357+0000","last_fullsized":"2024-09-16T09:38:20.361357+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:34:52.922047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:34:52.922047+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:17.998821+0000","scrub_duration":33,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1","version":"390'3537","reported_seq":3576,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:22.956248+0000","last_change":"2024-09-16T09:37:45.401338+0000","last_active":"2024-09-16T09:38:22.956248+0000","last_peered":"2024-09-16T09:38:22.956248+0000","last_clean":"2024-09-16T09:38:22.956248+0000","last_became_active":"2024-09-16T09:37:45.400966+0000","last_became_peered":"2024-09-16T09:37:45.400966+0000","last_unstale":"2024-09-16T09:38:22.956248+0000","last_undegraded":"2024-09-16T09:38:22.956248+0000","last_fullsized":"2024-09-16T09:38:22.956248+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_clean_scrub_stamp":"2024-09-16T09:37:44.378497+0000","objects_scrubbed":0,"log_size":3537,"log_dups_size":0,"ondisk_log_size":3537,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:52.315503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":231800832,"num_objects":3537,"num_object_clones":0,"num_object_copies":10611,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3537,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3537,"num_write_kb":226368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.a","version":"73'79","reported_seq":634,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.365038+0000","last_change":"2024-09-16T09:37:52.203798+0000","last_active":"2024-09-16T09:38:20.365038+0000","last_peered":"2024-09-16T09:38:20.365038+0000","last_clean":"2024-09-16T09:38:20.365038+0000","last_became_active":"2024-09-16T09:26:35.127352+0000","last_became_peered":"2024-09-16T09:26:35.127352+0000","last_unstale":"2024-09-16T09:38:20.365038+0000","last_undegraded":"2024-09-16T09:38:20.365038+0000","last_fullsized":"2024-09-16T09:38:20.365038+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:37:52.203752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:37:52.203752+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:59.150165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"285'27","reported_seq":661,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.366132+0000","last_change":"2024-09-16T09:36:47.588508+0000","last_active":"2024-09-16T09:38:20.366132+0000","last_peered":"2024-09-16T09:38:20.366132+0000","last_clean":"2024-09-16T09:38:20.366132+0000","last_became_active":"2024-09-16T09:26:33.295672+0000","last_became_peered":"2024-09-16T09:26:33.295672+0000","last_unstale":"2024-09-16T09:38:20.366132+0000","last_undegraded":"2024-09-16T09:38:20.366132+0000","last_fullsized":"2024-09-16T09:38:20.366132+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'27","last_scrub_stamp":"2024-09-16T09:36:47.588449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:36:47.588449+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:14.953315+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":437,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.361449+0000","last_change":"2024-09-16T09:37:16.625626+0000","last_active":"2024-09-16T09:38:20.361449+0000","last_peered":"2024-09-16T09:38:20.361449+0000","last_clean":"2024-09-16T09:38:20.361449+0000","last_became_active":"2024-09-16T09:31:09.274534+0000","last_became_peered":"2024-09-16T09:31:09.274534+0000","last_unstale":"2024-09-16T09:38:20.361449+0000","last_undegraded":"2024-09-16T09:38:20.361449+0000","last_fullsized":"2024-09-16T09:38:20.361449+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:16.625556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:37:16.625556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:26.929216+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"66'8","reported_seq":856,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.369418+0000","last_change":"2024-09-16T09:37:49.336009+0000","last_active":"2024-09-16T09:38:20.369418+0000","last_peered":"2024-09-16T09:38:20.369418+0000","last_clean":"2024-09-16T09:38:20.369418+0000","last_became_active":"2024-09-16T09:25:05.713813+0000","last_became_peered":"2024-09-16T09:25:05.713813+0000","last_unstale":"2024-09-16T09:38:20.369418+0000","last_undegraded":"2024-09-16T09:38:20.369418+0000","last_fullsized":"2024-09-16T09:38:20.369418+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:37:49.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:37:49.335939+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:08.862747+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":690,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.364489+0000","last_change":"2024-09-16T09:36:37.152446+0000","last_active":"2024-09-16T09:38:20.364489+0000","last_peered":"2024-09-16T09:38:20.364489+0000","last_clean":"2024-09-16T09:38:20.364489+0000","last_became_active":"2024-09-16T09:25:07.717344+0000","last_became_peered":"2024-09-16T09:25:07.717344+0000","last_unstale":"2024-09-16T09:38:20.364489+0000","last_undegraded":"2024-09-16T09:38:20.364489+0000","last_fullsized":"2024-09-16T09:38:20.364489+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:37.152377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:36:37.152377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.e","version":"390'3580","reported_seq":3617,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:22.976069+0000","last_change":"2024-09-16T09:37:45.464349+0000","last_active":"2024-09-16T09:38:22.976069+0000","last_peered":"2024-09-16T09:38:22.976069+0000","last_clean":"2024-09-16T09:38:22.976069+0000","last_became_active":"2024-09-16T09:37:45.463961+0000","last_became_peered":"2024-09-16T09:37:45.463961+0000","last_unstale":"2024-09-16T09:38:22.976069+0000","last_undegraded":"2024-09-16T09:38:22.976069+0000","last_fullsized":"2024-09-16T09:38:22.976069+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_clean_scrub_stamp":"2024-09-16T09:37:44.378497+0000","objects_scrubbed":0,"log_size":3580,"log_dups_size":0,"ondisk_log_size":3580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:49.201675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":234618880,"num_objects":3580,"num_object_clones":0,"num_object_copies":10740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3580,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3580,"num_write_kb":229120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"73'79","reported_seq":666,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.369616+0000","last_change":"2024-09-16T09:32:42.123139+0000","last_active":"2024-09-16T09:38:20.369616+0000","last_peered":"2024-09-16T09:38:20.369616+0000","last_clean":"2024-09-16T09:38:20.369616+0000","last_became_active":"2024-09-16T09:30:08.763683+0000","last_became_peered":"2024-09-16T09:30:08.763683+0000","last_unstale":"2024-09-16T09:38:20.369616+0000","last_undegraded":"2024-09-16T09:38:20.369616+0000","last_fullsized":"2024-09-16T09:38:20.369616+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:32:42.123067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:32:42.123067+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:09.339965+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"285'30","reported_seq":658,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.367105+0000","last_change":"2024-09-16T09:33:21.333996+0000","last_active":"2024-09-16T09:38:20.367105+0000","last_peered":"2024-09-16T09:38:20.367105+0000","last_clean":"2024-09-16T09:38:20.367105+0000","last_became_active":"2024-09-16T09:31:09.270434+0000","last_became_peered":"2024-09-16T09:31:09.270434+0000","last_unstale":"2024-09-16T09:38:20.367105+0000","last_undegraded":"2024-09-16T09:38:20.367105+0000","last_fullsized":"2024-09-16T09:38:20.367105+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'18","last_scrub_stamp":"2024-09-16T09:33:21.333937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:33:21.333937+0000","objects_scrubbed":8,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:38.469118+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":51,"num_read_kb":39,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":538,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.367263+0000","last_change":"2024-09-16T09:36:50.622852+0000","last_active":"2024-09-16T09:38:20.367263+0000","last_peered":"2024-09-16T09:38:20.367263+0000","last_clean":"2024-09-16T09:38:20.367263+0000","last_became_active":"2024-09-16T09:28:52.983543+0000","last_became_peered":"2024-09-16T09:28:52.983543+0000","last_unstale":"2024-09-16T09:38:20.367263+0000","last_undegraded":"2024-09-16T09:38:20.367263+0000","last_fullsized":"2024-09-16T09:38:20.367263+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:50.622807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:36:50.622807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:59.525944+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":718,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.369898+0000","last_change":"2024-09-16T09:35:19.822713+0000","last_active":"2024-09-16T09:38:20.369898+0000","last_peered":"2024-09-16T09:38:20.369898+0000","last_clean":"2024-09-16T09:38:20.369898+0000","last_became_active":"2024-09-16T09:31:09.268864+0000","last_became_peered":"2024-09-16T09:31:09.268864+0000","last_unstale":"2024-09-16T09:38:20.369898+0000","last_undegraded":"2024-09-16T09:38:20.369898+0000","last_fullsized":"2024-09-16T09:38:20.369898+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:35:19.822635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:35:19.822635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:11.341259+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":706,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.367502+0000","last_change":"2024-09-16T09:35:46.933987+0000","last_active":"2024-09-16T09:38:20.367502+0000","last_peered":"2024-09-16T09:38:20.367502+0000","last_clean":"2024-09-16T09:38:20.367502+0000","last_became_active":"2024-09-16T09:31:09.269433+0000","last_became_peered":"2024-09-16T09:31:09.269433+0000","last_unstale":"2024-09-16T09:38:20.367502+0000","last_undegraded":"2024-09-16T09:38:20.367502+0000","last_fullsized":"2024-09-16T09:38:20.367502+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:35:46.933912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:35:46.933912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"390'3485","reported_seq":3524,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:22.754768+0000","last_change":"2024-09-16T09:37:45.463119+0000","last_active":"2024-09-16T09:38:22.754768+0000","last_peered":"2024-09-16T09:38:22.754768+0000","last_clean":"2024-09-16T09:38:22.754768+0000","last_became_active":"2024-09-16T09:37:45.462743+0000","last_became_peered":"2024-09-16T09:37:45.462743+0000","last_unstale":"2024-09-16T09:38:22.754768+0000","last_undegraded":"2024-09-16T09:38:22.754768+0000","last_fullsized":"2024-09-16T09:38:22.754768+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_clean_scrub_stamp":"2024-09-16T09:37:44.378497+0000","objects_scrubbed":0,"log_size":3485,"log_dups_size":0,"ondisk_log_size":3485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:54.300226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":228392960,"num_objects":3485,"num_object_clones":0,"num_object_copies":10455,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3485,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3485,"num_write_kb":223040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.4","version":"73'79","reported_seq":437,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.364156+0000","last_change":"2024-09-16T09:37:17.664174+0000","last_active":"2024-09-16T09:38:20.364156+0000","last_peered":"2024-09-16T09:38:20.364156+0000","last_clean":"2024-09-16T09:38:20.364156+0000","last_became_active":"2024-09-16T09:31:09.275139+0000","last_became_peered":"2024-09-16T09:31:09.275139+0000","last_unstale":"2024-09-16T09:38:20.364156+0000","last_undegraded":"2024-09-16T09:38:20.364156+0000","last_fullsized":"2024-09-16T09:38:20.364156+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:37:17.664105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:37:17.664105+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:38.043254+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"285'28","reported_seq":750,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.369977+0000","last_change":"2024-09-16T09:35:43.914378+0000","last_active":"2024-09-16T09:38:20.369977+0000","last_peered":"2024-09-16T09:38:20.369977+0000","last_clean":"2024-09-16T09:38:20.369977+0000","last_became_active":"2024-09-16T09:25:03.708830+0000","last_became_peered":"2024-09-16T09:25:03.708830+0000","last_unstale":"2024-09-16T09:38:20.369977+0000","last_undegraded":"2024-09-16T09:38:20.369977+0000","last_fullsized":"2024-09-16T09:38:20.369977+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'28","last_scrub_stamp":"2024-09-16T09:35:43.914312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:35:43.914312+0000","objects_scrubbed":7,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:19.189121+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":721,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.368712+0000","last_change":"2024-09-16T09:33:43.678231+0000","last_active":"2024-09-16T09:38:20.368712+0000","last_peered":"2024-09-16T09:38:20.368712+0000","last_clean":"2024-09-16T09:38:20.368712+0000","last_became_active":"2024-09-16T09:25:01.702992+0000","last_became_peered":"2024-09-16T09:25:01.702992+0000","last_unstale":"2024-09-16T09:38:20.368712+0000","last_undegraded":"2024-09-16T09:38:20.368712+0000","last_fullsized":"2024-09-16T09:38:20.368712+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:33:43.678138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:43.678138+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"235'12","reported_seq":420,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.367010+0000","last_change":"2024-09-16T09:37:19.380567+0000","last_active":"2024-09-16T09:38:20.367010+0000","last_peered":"2024-09-16T09:38:20.367010+0000","last_clean":"2024-09-16T09:38:20.367010+0000","last_became_active":"2024-09-16T09:33:30.251047+0000","last_became_peered":"2024-09-16T09:33:30.251047+0000","last_unstale":"2024-09-16T09:38:20.367010+0000","last_undegraded":"2024-09-16T09:38:20.367010+0000","last_fullsized":"2024-09-16T09:38:20.367010+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"235'12","last_scrub_stamp":"2024-09-16T09:37:19.380504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:37:19.380504+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:37.307298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":682,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.365596+0000","last_change":"2024-09-16T09:36:47.605655+0000","last_active":"2024-09-16T09:38:20.365596+0000","last_peered":"2024-09-16T09:38:20.365596+0000","last_clean":"2024-09-16T09:38:20.365596+0000","last_became_active":"2024-09-16T09:25:07.722017+0000","last_became_peered":"2024-09-16T09:25:07.722017+0000","last_unstale":"2024-09-16T09:38:20.365596+0000","last_undegraded":"2024-09-16T09:38:20.365596+0000","last_fullsized":"2024-09-16T09:38:20.365596+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:47.605615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:36:47.605615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.c","version":"390'3505","reported_seq":3543,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:22.874606+0000","last_change":"2024-09-16T09:37:45.401117+0000","last_active":"2024-09-16T09:38:22.874606+0000","last_peered":"2024-09-16T09:38:22.874606+0000","last_clean":"2024-09-16T09:38:22.874606+0000","last_became_active":"2024-09-16T09:37:45.400778+0000","last_became_peered":"2024-09-16T09:37:45.400778+0000","last_unstale":"2024-09-16T09:38:22.874606+0000","last_undegraded":"2024-09-16T09:38:22.874606+0000","last_fullsized":"2024-09-16T09:38:22.874606+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_clean_scrub_stamp":"2024-09-16T09:37:44.378497+0000","objects_scrubbed":0,"log_size":3505,"log_dups_size":0,"ondisk_log_size":3505,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:49.375813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":229703680,"num_objects":3505,"num_object_clones":0,"num_object_copies":10515,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3505,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3505,"num_write_kb":224320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"73'79","reported_seq":760,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.361027+0000","last_change":"2024-09-16T09:34:51.864131+0000","last_active":"2024-09-16T09:38:20.361027+0000","last_peered":"2024-09-16T09:38:20.361027+0000","last_clean":"2024-09-16T09:38:20.361027+0000","last_became_active":"2024-09-16T09:31:09.274707+0000","last_became_peered":"2024-09-16T09:31:09.274707+0000","last_unstale":"2024-09-16T09:38:20.361027+0000","last_undegraded":"2024-09-16T09:38:20.361027+0000","last_fullsized":"2024-09-16T09:38:20.361027+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:34:51.864036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:34:51.864036+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"285'20","reported_seq":385,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.365491+0000","last_change":"2024-09-16T09:33:41.368102+0000","last_active":"2024-09-16T09:38:20.365491+0000","last_peered":"2024-09-16T09:38:20.365491+0000","last_clean":"2024-09-16T09:38:20.365491+0000","last_became_active":"2024-09-16T09:33:30.248453+0000","last_became_peered":"2024-09-16T09:33:30.248453+0000","last_unstale":"2024-09-16T09:38:20.365491+0000","last_undegraded":"2024-09-16T09:38:20.365491+0000","last_fullsized":"2024-09-16T09:38:20.365491+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-16T09:33:41.368016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:33:41.368016+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":717,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.364937+0000","last_change":"2024-09-16T09:36:47.095703+0000","last_active":"2024-09-16T09:38:20.364937+0000","last_peered":"2024-09-16T09:38:20.364937+0000","last_clean":"2024-09-16T09:38:20.364937+0000","last_became_active":"2024-09-16T09:31:09.270619+0000","last_became_peered":"2024-09-16T09:31:09.270619+0000","last_unstale":"2024-09-16T09:38:20.364937+0000","last_undegraded":"2024-09-16T09:38:20.364937+0000","last_fullsized":"2024-09-16T09:38:20.364937+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:47.095658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:36:47.095658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"66'8","reported_seq":865,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.369835+0000","last_change":"2024-09-16T09:36:52.689167+0000","last_active":"2024-09-16T09:38:20.369835+0000","last_peered":"2024-09-16T09:38:20.369835+0000","last_clean":"2024-09-16T09:38:20.369835+0000","last_became_active":"2024-09-16T09:25:05.714507+0000","last_became_peered":"2024-09-16T09:25:05.714507+0000","last_unstale":"2024-09-16T09:38:20.369835+0000","last_undegraded":"2024-09-16T09:38:20.369835+0000","last_fullsized":"2024-09-16T09:38:20.369835+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:36:52.689091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:36:52.689091+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":706,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.368100+0000","last_change":"2024-09-16T09:32:33.362233+0000","last_active":"2024-09-16T09:38:20.368100+0000","last_peered":"2024-09-16T09:38:20.368100+0000","last_clean":"2024-09-16T09:38:20.368100+0000","last_became_active":"2024-09-16T09:31:09.283192+0000","last_became_peered":"2024-09-16T09:31:09.283192+0000","last_unstale":"2024-09-16T09:38:20.368100+0000","last_undegraded":"2024-09-16T09:38:20.368100+0000","last_fullsized":"2024-09-16T09:38:20.368100+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:33.362188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:32:33.362188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:18.341863+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.d","version":"390'3458","reported_seq":3497,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:22.734587+0000","last_change":"2024-09-16T09:37:45.405475+0000","last_active":"2024-09-16T09:38:22.734587+0000","last_peered":"2024-09-16T09:38:22.734587+0000","last_clean":"2024-09-16T09:38:22.734587+0000","last_became_active":"2024-09-16T09:37:45.405149+0000","last_became_peered":"2024-09-16T09:37:45.405149+0000","last_unstale":"2024-09-16T09:38:22.734587+0000","last_undegraded":"2024-09-16T09:38:22.734587+0000","last_fullsized":"2024-09-16T09:38:22.734587+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_clean_scrub_stamp":"2024-09-16T09:37:44.378497+0000","objects_scrubbed":0,"log_size":3458,"log_dups_size":0,"ondisk_log_size":3458,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:48.823494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":226623488,"num_objects":3458,"num_object_clones":0,"num_object_copies":10374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3458,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3458,"num_write_kb":221312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"73'79","reported_seq":733,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.360826+0000","last_change":"2024-09-16T09:34:42.728232+0000","last_active":"2024-09-16T09:38:20.360826+0000","last_peered":"2024-09-16T09:38:20.360826+0000","last_clean":"2024-09-16T09:38:20.360826+0000","last_became_active":"2024-09-16T09:31:09.274565+0000","last_became_peered":"2024-09-16T09:31:09.274565+0000","last_unstale":"2024-09-16T09:38:20.360826+0000","last_undegraded":"2024-09-16T09:38:20.360826+0000","last_fullsized":"2024-09-16T09:38:20.360826+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:34:42.728143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:34:42.728143+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:35.968055+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"285'29","reported_seq":751,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.370047+0000","last_change":"2024-09-16T09:36:54.604573+0000","last_active":"2024-09-16T09:38:20.370047+0000","last_peered":"2024-09-16T09:38:20.370047+0000","last_clean":"2024-09-16T09:38:20.370047+0000","last_became_active":"2024-09-16T09:25:03.714797+0000","last_became_peered":"2024-09-16T09:25:03.714797+0000","last_unstale":"2024-09-16T09:38:20.370047+0000","last_undegraded":"2024-09-16T09:38:20.370047+0000","last_fullsized":"2024-09-16T09:38:20.370047+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'29","last_scrub_stamp":"2024-09-16T09:36:54.604484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:36:54.604484+0000","objects_scrubbed":8,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:21.529216+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":54,"num_read_kb":40,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":327,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.364103+0000","last_change":"2024-09-16T09:33:30.247311+0000","last_active":"2024-09-16T09:38:20.364103+0000","last_peered":"2024-09-16T09:38:20.364103+0000","last_clean":"2024-09-16T09:38:20.364103+0000","last_became_active":"2024-09-16T09:33:30.246996+0000","last_became_peered":"2024-09-16T09:33:30.246996+0000","last_unstale":"2024-09-16T09:38:20.364103+0000","last_undegraded":"2024-09-16T09:38:20.364103+0000","last_fullsized":"2024-09-16T09:38:20.364103+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:11.407616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:11.407616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:19.244214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,2],"acting":[5,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":712,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.360342+0000","last_change":"2024-09-16T09:34:45.772748+0000","last_active":"2024-09-16T09:38:20.360342+0000","last_peered":"2024-09-16T09:38:20.360342+0000","last_clean":"2024-09-16T09:38:20.360342+0000","last_became_active":"2024-09-16T09:25:05.708663+0000","last_became_peered":"2024-09-16T09:25:05.708663+0000","last_unstale":"2024-09-16T09:38:20.360342+0000","last_undegraded":"2024-09-16T09:38:20.360342+0000","last_fullsized":"2024-09-16T09:38:20.360342+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:45.772674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:34:45.772674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:17.038064+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":343,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.360090+0000","last_change":"2024-09-16T09:37:56.882475+0000","last_active":"2024-09-16T09:38:20.360090+0000","last_peered":"2024-09-16T09:38:20.360090+0000","last_clean":"2024-09-16T09:38:20.360090+0000","last_became_active":"2024-09-16T09:33:30.250178+0000","last_became_peered":"2024-09-16T09:33:30.250178+0000","last_unstale":"2024-09-16T09:38:20.360090+0000","last_undegraded":"2024-09-16T09:38:20.360090+0000","last_fullsized":"2024-09-16T09:38:20.360090+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:56.882326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:37:56.882326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:02.807091+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,2],"acting":[6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.a","version":"390'3564","reported_seq":3600,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:22.754725+0000","last_change":"2024-09-16T09:37:45.404280+0000","last_active":"2024-09-16T09:38:22.754725+0000","last_peered":"2024-09-16T09:38:22.754725+0000","last_clean":"2024-09-16T09:38:22.754725+0000","last_became_active":"2024-09-16T09:37:45.404021+0000","last_became_peered":"2024-09-16T09:37:45.404021+0000","last_unstale":"2024-09-16T09:38:22.754725+0000","last_undegraded":"2024-09-16T09:38:22.754725+0000","last_fullsized":"2024-09-16T09:38:22.754725+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_clean_scrub_stamp":"2024-09-16T09:37:44.378497+0000","objects_scrubbed":0,"log_size":3564,"log_dups_size":0,"ondisk_log_size":3564,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:08.182634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":233570304,"num_objects":3564,"num_object_clones":0,"num_object_copies":10692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3564,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3564,"num_write_kb":228096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"73'79","reported_seq":666,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.363943+0000","last_change":"2024-09-16T09:36:42.191578+0000","last_active":"2024-09-16T09:38:20.363943+0000","last_peered":"2024-09-16T09:38:20.363943+0000","last_clean":"2024-09-16T09:38:20.363943+0000","last_became_active":"2024-09-16T09:30:08.766350+0000","last_became_peered":"2024-09-16T09:30:08.766350+0000","last_unstale":"2024-09-16T09:38:20.363943+0000","last_undegraded":"2024-09-16T09:38:20.363943+0000","last_fullsized":"2024-09-16T09:38:20.363943+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:36:42.191521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:36:42.191521+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"285'42","reported_seq":1008,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.375083+0000","last_change":"2024-09-16T09:37:57.407386+0000","last_active":"2024-09-16T09:38:20.375083+0000","last_peered":"2024-09-16T09:38:20.375083+0000","last_clean":"2024-09-16T09:38:20.375083+0000","last_became_active":"2024-09-16T09:31:09.278839+0000","last_became_peered":"2024-09-16T09:31:09.278839+0000","last_unstale":"2024-09-16T09:38:20.375083+0000","last_undegraded":"2024-09-16T09:38:20.375083+0000","last_fullsized":"2024-09-16T09:38:20.375083+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'42","last_scrub_stamp":"2024-09-16T09:37:57.407297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:37:57.407297+0000","objects_scrubbed":10,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:23.815599+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":66,"num_read_kb":51,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":712,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.367893+0000","last_change":"2024-09-16T09:36:35.691463+0000","last_active":"2024-09-16T09:38:20.367893+0000","last_peered":"2024-09-16T09:38:20.367893+0000","last_clean":"2024-09-16T09:38:20.367893+0000","last_became_active":"2024-09-16T09:25:01.703076+0000","last_became_peered":"2024-09-16T09:25:01.703076+0000","last_unstale":"2024-09-16T09:38:20.367893+0000","last_undegraded":"2024-09-16T09:38:20.367893+0000","last_fullsized":"2024-09-16T09:38:20.367893+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:35.691415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:36:35.691415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":713,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.365740+0000","last_change":"2024-09-16T09:33:38.417375+0000","last_active":"2024-09-16T09:38:20.365740+0000","last_peered":"2024-09-16T09:38:20.365740+0000","last_clean":"2024-09-16T09:38:20.365740+0000","last_became_active":"2024-09-16T09:25:00.696237+0000","last_became_peered":"2024-09-16T09:25:00.696237+0000","last_unstale":"2024-09-16T09:38:20.365740+0000","last_undegraded":"2024-09-16T09:38:20.365740+0000","last_fullsized":"2024-09-16T09:38:20.365740+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-16T09:33:38.417308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:33:38.417308+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.0025137430000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":680,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.369483+0000","last_change":"2024-09-16T09:33:16.666704+0000","last_active":"2024-09-16T09:38:20.369483+0000","last_peered":"2024-09-16T09:38:20.369483+0000","last_clean":"2024-09-16T09:38:20.369483+0000","last_became_active":"2024-09-16T09:25:05.717591+0000","last_became_peered":"2024-09-16T09:25:05.717591+0000","last_unstale":"2024-09-16T09:38:20.369483+0000","last_undegraded":"2024-09-16T09:38:20.369483+0000","last_fullsized":"2024-09-16T09:38:20.369483+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:16.666648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:16.666648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:38.618334+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":429,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.362658+0000","last_change":"2024-09-16T09:36:50.667709+0000","last_active":"2024-09-16T09:38:20.362658+0000","last_peered":"2024-09-16T09:38:20.362658+0000","last_clean":"2024-09-16T09:38:20.362658+0000","last_became_active":"2024-09-16T09:31:09.272609+0000","last_became_peered":"2024-09-16T09:31:09.272609+0000","last_unstale":"2024-09-16T09:38:20.362658+0000","last_undegraded":"2024-09-16T09:38:20.362658+0000","last_fullsized":"2024-09-16T09:38:20.362658+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:50.667661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:36:50.667661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"390'3554","reported_seq":3593,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:23.240750+0000","last_change":"2024-09-16T09:37:45.463232+0000","last_active":"2024-09-16T09:38:23.240750+0000","last_peered":"2024-09-16T09:38:23.240750+0000","last_clean":"2024-09-16T09:38:23.240750+0000","last_became_active":"2024-09-16T09:37:45.462868+0000","last_became_peered":"2024-09-16T09:37:45.462868+0000","last_unstale":"2024-09-16T09:38:23.240750+0000","last_undegraded":"2024-09-16T09:38:23.240750+0000","last_fullsized":"2024-09-16T09:38:23.240750+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_clean_scrub_stamp":"2024-09-16T09:37:44.378497+0000","objects_scrubbed":0,"log_size":3554,"log_dups_size":0,"ondisk_log_size":3554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:55.563655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":232914944,"num_objects":3554,"num_object_clones":0,"num_object_copies":10662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3554,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3554,"num_write_kb":227456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"73'79","reported_seq":18,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.367436+0000","last_change":"2024-09-16T09:38:10.556681+0000","last_active":"2024-09-16T09:38:20.367436+0000","last_peered":"2024-09-16T09:38:20.367436+0000","last_clean":"2024-09-16T09:38:20.367436+0000","last_became_active":"2024-09-16T09:38:10.556320+0000","last_became_peered":"2024-09-16T09:38:10.556320+0000","last_unstale":"2024-09-16T09:38:20.367436+0000","last_undegraded":"2024-09-16T09:38:20.367436+0000","last_fullsized":"2024-09-16T09:38:20.367436+0000","mapping_epoch":386,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":387,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:34:32.717915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:34:32.717915+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"285'17","reported_seq":426,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.368454+0000","last_change":"2024-09-16T09:37:49.246464+0000","last_active":"2024-09-16T09:38:20.368454+0000","last_peered":"2024-09-16T09:38:20.368454+0000","last_clean":"2024-09-16T09:38:20.368454+0000","last_became_active":"2024-09-16T09:37:33.349009+0000","last_became_peered":"2024-09-16T09:37:33.349009+0000","last_unstale":"2024-09-16T09:38:20.368454+0000","last_undegraded":"2024-09-16T09:38:20.368454+0000","last_fullsized":"2024-09-16T09:38:20.368454+0000","mapping_epoch":367,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":368,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'17","last_scrub_stamp":"2024-09-16T09:37:49.246379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:37:49.246379+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:56.883815+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":35,"num_read_kb":24,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":335,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.359954+0000","last_change":"2024-09-16T09:37:21.946685+0000","last_active":"2024-09-16T09:38:20.359954+0000","last_peered":"2024-09-16T09:38:20.359954+0000","last_clean":"2024-09-16T09:38:20.359954+0000","last_became_active":"2024-09-16T09:33:30.247596+0000","last_became_peered":"2024-09-16T09:33:30.247596+0000","last_unstale":"2024-09-16T09:38:20.359954+0000","last_undegraded":"2024-09-16T09:38:20.359954+0000","last_fullsized":"2024-09-16T09:38:20.359954+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:21.946641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:37:21.946641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:25.048262+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,2],"acting":[6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":370,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:23.416822+0000","last_change":"2024-09-16T09:38:23.416822+0000","last_active":"2024-09-16T09:38:23.416822+0000","last_peered":"2024-09-16T09:38:23.416822+0000","last_clean":"2024-09-16T09:38:23.416822+0000","last_became_active":"2024-09-16T09:37:04.358985+0000","last_became_peered":"2024-09-16T09:37:04.358985+0000","last_unstale":"2024-09-16T09:38:23.416822+0000","last_undegraded":"2024-09-16T09:38:23.416822+0000","last_fullsized":"2024-09-16T09:38:23.416822+0000","mapping_epoch":343,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":344,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:38:23.416756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:38:23.416756+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:50.859015+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":720,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.365244+0000","last_change":"2024-09-16T09:36:55.530225+0000","last_active":"2024-09-16T09:38:20.365244+0000","last_peered":"2024-09-16T09:38:20.365244+0000","last_clean":"2024-09-16T09:38:20.365244+0000","last_became_active":"2024-09-16T09:25:05.711539+0000","last_became_peered":"2024-09-16T09:25:05.711539+0000","last_unstale":"2024-09-16T09:38:20.365244+0000","last_undegraded":"2024-09-16T09:38:20.365244+0000","last_fullsized":"2024-09-16T09:38:20.365244+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:55.530175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:36:55.530175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:19.090568+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":698,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.365950+0000","last_change":"2024-09-16T09:37:45.450074+0000","last_active":"2024-09-16T09:38:20.365950+0000","last_peered":"2024-09-16T09:38:20.365950+0000","last_clean":"2024-09-16T09:38:20.365950+0000","last_became_active":"2024-09-16T09:25:07.723282+0000","last_became_peered":"2024-09-16T09:25:07.723282+0000","last_unstale":"2024-09-16T09:38:20.365950+0000","last_undegraded":"2024-09-16T09:38:20.365950+0000","last_fullsized":"2024-09-16T09:38:20.365950+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:45.450022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:37:45.450022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:15.116366+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"390'3645","reported_seq":3684,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:24.235305+0000","last_change":"2024-09-16T09:37:45.464193+0000","last_active":"2024-09-16T09:38:24.235305+0000","last_peered":"2024-09-16T09:38:24.235305+0000","last_clean":"2024-09-16T09:38:24.235305+0000","last_became_active":"2024-09-16T09:37:45.462754+0000","last_became_peered":"2024-09-16T09:37:45.462754+0000","last_unstale":"2024-09-16T09:38:24.235305+0000","last_undegraded":"2024-09-16T09:38:24.235305+0000","last_fullsized":"2024-09-16T09:38:24.235305+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_clean_scrub_stamp":"2024-09-16T09:37:44.378497+0000","objects_scrubbed":0,"log_size":3645,"log_dups_size":0,"ondisk_log_size":3645,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:46.604408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":238878720,"num_objects":3645,"num_object_clones":0,"num_object_copies":10935,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3645,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3645,"num_write_kb":233280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,7],"acting":[0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"73'79","reported_seq":416,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.360341+0000","last_change":"2024-09-16T09:34:31.929462+0000","last_active":"2024-09-16T09:38:20.360341+0000","last_peered":"2024-09-16T09:38:20.360341+0000","last_clean":"2024-09-16T09:38:20.360341+0000","last_became_active":"2024-09-16T09:33:30.249309+0000","last_became_peered":"2024-09-16T09:33:30.249309+0000","last_unstale":"2024-09-16T09:38:20.360341+0000","last_undegraded":"2024-09-16T09:38:20.360341+0000","last_fullsized":"2024-09-16T09:38:20.360341+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:34:31.929388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:34:31.929388+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:34.931244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.6","version":"285'24","reported_seq":728,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.368538+0000","last_change":"2024-09-16T09:32:48.547237+0000","last_active":"2024-09-16T09:38:20.368538+0000","last_peered":"2024-09-16T09:38:20.368538+0000","last_clean":"2024-09-16T09:38:20.368538+0000","last_became_active":"2024-09-16T09:25:03.715315+0000","last_became_peered":"2024-09-16T09:25:03.715315+0000","last_unstale":"2024-09-16T09:38:20.368538+0000","last_undegraded":"2024-09-16T09:38:20.368538+0000","last_fullsized":"2024-09-16T09:38:20.368538+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-16T09:32:48.546869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:32:48.546869+0000","objects_scrubbed":6,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:32.692965+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":36,"num_read_kb":30,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"59'1","reported_seq":727,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.373705+0000","last_change":"2024-09-16T09:33:21.682893+0000","last_active":"2024-09-16T09:38:20.373705+0000","last_peered":"2024-09-16T09:38:20.373705+0000","last_clean":"2024-09-16T09:38:20.373705+0000","last_became_active":"2024-09-16T09:31:09.269627+0000","last_became_peered":"2024-09-16T09:31:09.269627+0000","last_unstale":"2024-09-16T09:38:20.373705+0000","last_undegraded":"2024-09-16T09:38:20.373705+0000","last_fullsized":"2024-09-16T09:38:20.373705+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:33:21.682840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:21.682840+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:42.553527+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1986,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"67'5","reported_seq":1507,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:21.079215+0000","last_change":"2024-09-16T09:33:37.358839+0000","last_active":"2024-09-16T09:38:21.079215+0000","last_peered":"2024-09-16T09:38:21.079215+0000","last_clean":"2024-09-16T09:38:21.079215+0000","last_became_active":"2024-09-16T09:25:00.696546+0000","last_became_peered":"2024-09-16T09:25:00.696546+0000","last_unstale":"2024-09-16T09:38:21.079215+0000","last_undegraded":"2024-09-16T09:38:21.079215+0000","last_fullsized":"2024-09-16T09:38:21.079215+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'5","last_scrub_stamp":"2024-09-16T09:33:37.358741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:33:37.358741+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:33.496609+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0.0059770420000000001,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":794,"num_read_kb":789,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":679,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.368473+0000","last_change":"2024-09-16T09:27:07.337170+0000","last_active":"2024-09-16T09:38:20.368473+0000","last_peered":"2024-09-16T09:38:20.368473+0000","last_clean":"2024-09-16T09:38:20.368473+0000","last_became_active":"2024-09-16T09:25:05.714704+0000","last_became_peered":"2024-09-16T09:25:05.714704+0000","last_unstale":"2024-09-16T09:38:20.368473+0000","last_undegraded":"2024-09-16T09:38:20.368473+0000","last_fullsized":"2024-09-16T09:38:20.368473+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:07.337124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:07.337124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:38.140085+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":605,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.364317+0000","last_change":"2024-09-16T09:36:43.150813+0000","last_active":"2024-09-16T09:38:20.364317+0000","last_peered":"2024-09-16T09:38:20.364317+0000","last_clean":"2024-09-16T09:38:20.364317+0000","last_became_active":"2024-09-16T09:33:30.245852+0000","last_became_peered":"2024-09-16T09:33:30.245852+0000","last_unstale":"2024-09-16T09:38:20.364317+0000","last_undegraded":"2024-09-16T09:38:20.364317+0000","last_fullsized":"2024-09-16T09:38:20.364317+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:43.150764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:36:43.150764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"390'3491","reported_seq":3530,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:22.758042+0000","last_change":"2024-09-16T09:37:45.406191+0000","last_active":"2024-09-16T09:38:22.758042+0000","last_peered":"2024-09-16T09:38:22.758042+0000","last_clean":"2024-09-16T09:38:22.758042+0000","last_became_active":"2024-09-16T09:37:45.405552+0000","last_became_peered":"2024-09-16T09:37:45.405552+0000","last_unstale":"2024-09-16T09:38:22.758042+0000","last_undegraded":"2024-09-16T09:38:22.758042+0000","last_fullsized":"2024-09-16T09:38:22.758042+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_clean_scrub_stamp":"2024-09-16T09:37:44.378497+0000","objects_scrubbed":0,"log_size":3491,"log_dups_size":0,"ondisk_log_size":3491,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:55.714372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":228786176,"num_objects":3491,"num_object_clones":0,"num_object_copies":10473,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3491,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3491,"num_write_kb":223424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"285'28","reported_seq":909,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.360792+0000","last_change":"2024-09-16T09:33:16.891850+0000","last_active":"2024-09-16T09:38:20.360792+0000","last_peered":"2024-09-16T09:38:20.360792+0000","last_clean":"2024-09-16T09:38:20.360792+0000","last_became_active":"2024-09-16T09:25:03.714118+0000","last_became_peered":"2024-09-16T09:25:03.714118+0000","last_unstale":"2024-09-16T09:38:20.360792+0000","last_undegraded":"2024-09-16T09:38:20.360792+0000","last_fullsized":"2024-09-16T09:38:20.360792+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'16","last_scrub_stamp":"2024-09-16T09:33:16.891761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:33:16.891761+0000","objects_scrubbed":8,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:12.972702+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":43,"num_read_kb":33,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":553,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.365621+0000","last_change":"2024-09-16T09:36:50.114910+0000","last_active":"2024-09-16T09:38:20.365621+0000","last_peered":"2024-09-16T09:38:20.365621+0000","last_clean":"2024-09-16T09:38:20.365621+0000","last_became_active":"2024-09-16T09:31:09.270196+0000","last_became_peered":"2024-09-16T09:31:09.270196+0000","last_unstale":"2024-09-16T09:38:20.365621+0000","last_undegraded":"2024-09-16T09:38:20.365621+0000","last_fullsized":"2024-09-16T09:38:20.365621+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:50.114861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:36:50.114861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"73'79","reported_seq":782,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.367516+0000","last_change":"2024-09-16T09:32:43.535041+0000","last_active":"2024-09-16T09:38:20.367516+0000","last_peered":"2024-09-16T09:38:20.367516+0000","last_clean":"2024-09-16T09:38:20.367516+0000","last_became_active":"2024-09-16T09:30:08.870394+0000","last_became_peered":"2024-09-16T09:30:08.870394+0000","last_unstale":"2024-09-16T09:38:20.367516+0000","last_undegraded":"2024-09-16T09:38:20.367516+0000","last_fullsized":"2024-09-16T09:38:20.367516+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:32:43.534962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:32:43.534962+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:16.454386+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":694,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.365921+0000","last_change":"2024-09-16T09:37:52.517698+0000","last_active":"2024-09-16T09:38:20.365921+0000","last_peered":"2024-09-16T09:38:20.365921+0000","last_clean":"2024-09-16T09:38:20.365921+0000","last_became_active":"2024-09-16T09:25:05.714341+0000","last_became_peered":"2024-09-16T09:25:05.714341+0000","last_unstale":"2024-09-16T09:38:20.365921+0000","last_undegraded":"2024-09-16T09:38:20.365921+0000","last_fullsized":"2024-09-16T09:38:20.365921+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:52.517652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:37:52.517652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:20.774965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":691,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.368371+0000","last_change":"2024-09-16T09:37:50.218655+0000","last_active":"2024-09-16T09:38:20.368371+0000","last_peered":"2024-09-16T09:38:20.368371+0000","last_clean":"2024-09-16T09:38:20.368371+0000","last_became_active":"2024-09-16T09:25:07.714281+0000","last_became_peered":"2024-09-16T09:25:07.714281+0000","last_unstale":"2024-09-16T09:38:20.368371+0000","last_undegraded":"2024-09-16T09:38:20.368371+0000","last_fullsized":"2024-09-16T09:38:20.368371+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:50.218594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:37:50.218594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:20.077058+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.0","version":"390'3374","reported_seq":3413,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:22.755393+0000","last_change":"2024-09-16T09:37:45.407072+0000","last_active":"2024-09-16T09:38:22.755393+0000","last_peered":"2024-09-16T09:38:22.755393+0000","last_clean":"2024-09-16T09:38:22.755393+0000","last_became_active":"2024-09-16T09:37:45.406736+0000","last_became_peered":"2024-09-16T09:37:45.406736+0000","last_unstale":"2024-09-16T09:38:22.755393+0000","last_undegraded":"2024-09-16T09:38:22.755393+0000","last_fullsized":"2024-09-16T09:38:22.755393+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_clean_scrub_stamp":"2024-09-16T09:37:44.378497+0000","objects_scrubbed":0,"log_size":3374,"log_dups_size":0,"ondisk_log_size":3374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:03.181256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":221118464,"num_objects":3374,"num_object_clones":0,"num_object_copies":10122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3374,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3374,"num_write_kb":215936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.b","version":"73'79","reported_seq":519,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.369500+0000","last_change":"2024-09-16T09:34:31.720971+0000","last_active":"2024-09-16T09:38:20.369500+0000","last_peered":"2024-09-16T09:38:20.369500+0000","last_clean":"2024-09-16T09:38:20.369500+0000","last_became_active":"2024-09-16T09:33:30.251933+0000","last_became_peered":"2024-09-16T09:33:30.251933+0000","last_unstale":"2024-09-16T09:38:20.369500+0000","last_undegraded":"2024-09-16T09:38:20.369500+0000","last_fullsized":"2024-09-16T09:38:20.369500+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:34:31.720787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:34:31.720787+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"285'19","reported_seq":779,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.369463+0000","last_change":"2024-09-16T09:37:33.355562+0000","last_active":"2024-09-16T09:38:20.369463+0000","last_peered":"2024-09-16T09:38:20.369463+0000","last_clean":"2024-09-16T09:38:20.369463+0000","last_became_active":"2024-09-16T09:37:33.348433+0000","last_became_peered":"2024-09-16T09:37:33.348433+0000","last_unstale":"2024-09-16T09:38:20.369463+0000","last_undegraded":"2024-09-16T09:38:20.369463+0000","last_fullsized":"2024-09-16T09:38:20.369463+0000","mapping_epoch":367,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":368,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'19","last_scrub_stamp":"2024-09-16T09:35:27.813206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:35:27.813206+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:13.318653+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":696,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.370275+0000","last_change":"2024-09-16T09:37:18.392256+0000","last_active":"2024-09-16T09:38:20.370275+0000","last_peered":"2024-09-16T09:38:20.370275+0000","last_clean":"2024-09-16T09:38:20.370275+0000","last_became_active":"2024-09-16T09:25:01.708270+0000","last_became_peered":"2024-09-16T09:25:01.708270+0000","last_unstale":"2024-09-16T09:38:20.370275+0000","last_undegraded":"2024-09-16T09:38:20.370275+0000","last_fullsized":"2024-09-16T09:38:20.370275+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:18.392084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:37:18.392084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:33.733932+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":678,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.368287+0000","last_change":"2024-09-16T09:26:34.171190+0000","last_active":"2024-09-16T09:38:20.368287+0000","last_peered":"2024-09-16T09:38:20.368287+0000","last_clean":"2024-09-16T09:38:20.368287+0000","last_became_active":"2024-09-16T09:25:05.711964+0000","last_became_peered":"2024-09-16T09:25:05.711964+0000","last_unstale":"2024-09-16T09:38:20.368287+0000","last_undegraded":"2024-09-16T09:38:20.368287+0000","last_fullsized":"2024-09-16T09:38:20.368287+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:34.171139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:26:34.171139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:18.227141+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":570,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.360566+0000","last_change":"2024-09-16T09:34:50.866121+0000","last_active":"2024-09-16T09:38:20.360566+0000","last_peered":"2024-09-16T09:38:20.360566+0000","last_clean":"2024-09-16T09:38:20.360566+0000","last_became_active":"2024-09-16T09:31:09.278618+0000","last_became_peered":"2024-09-16T09:31:09.278618+0000","last_unstale":"2024-09-16T09:38:20.360566+0000","last_undegraded":"2024-09-16T09:38:20.360566+0000","last_fullsized":"2024-09-16T09:38:20.360566+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:50.866032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:34:50.866032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:09.987998+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3","version":"390'3611","reported_seq":3650,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:23.242228+0000","last_change":"2024-09-16T09:37:45.463216+0000","last_active":"2024-09-16T09:38:23.242228+0000","last_peered":"2024-09-16T09:38:23.242228+0000","last_clean":"2024-09-16T09:38:23.242228+0000","last_became_active":"2024-09-16T09:37:45.462708+0000","last_became_peered":"2024-09-16T09:37:45.462708+0000","last_unstale":"2024-09-16T09:38:23.242228+0000","last_undegraded":"2024-09-16T09:38:23.242228+0000","last_fullsized":"2024-09-16T09:38:23.242228+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_clean_scrub_stamp":"2024-09-16T09:37:44.378497+0000","objects_scrubbed":0,"log_size":3611,"log_dups_size":0,"ondisk_log_size":3611,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:05.295932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":236650496,"num_objects":3611,"num_object_clones":0,"num_object_copies":10833,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3611,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3611,"num_write_kb":231104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.8","version":"73'79","reported_seq":38,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.363244+0000","last_change":"2024-09-16T09:37:54.183161+0000","last_active":"2024-09-16T09:38:20.363244+0000","last_peered":"2024-09-16T09:38:20.363244+0000","last_clean":"2024-09-16T09:38:20.363244+0000","last_became_active":"2024-09-16T09:37:53.418691+0000","last_became_peered":"2024-09-16T09:37:53.418691+0000","last_unstale":"2024-09-16T09:38:20.363244+0000","last_undegraded":"2024-09-16T09:38:20.363244+0000","last_fullsized":"2024-09-16T09:38:20.363244+0000","mapping_epoch":382,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":383,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:37:54.183107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:37:54.183107+0000","objects_scrubbed":1,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:09.320645+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262688,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"285'31","reported_seq":756,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.369424+0000","last_change":"2024-09-16T09:35:28.861520+0000","last_active":"2024-09-16T09:38:20.369424+0000","last_peered":"2024-09-16T09:38:20.369424+0000","last_clean":"2024-09-16T09:38:20.369424+0000","last_became_active":"2024-09-16T09:25:03.713358+0000","last_became_peered":"2024-09-16T09:25:03.713358+0000","last_unstale":"2024-09-16T09:38:20.369424+0000","last_undegraded":"2024-09-16T09:38:20.369424+0000","last_fullsized":"2024-09-16T09:38:20.369424+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'31","last_scrub_stamp":"2024-09-16T09:35:28.861433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:35:28.861433+0000","objects_scrubbed":8,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:08.335954+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":50,"num_read_kb":40,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":704,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.360616+0000","last_change":"2024-09-16T09:37:52.902619+0000","last_active":"2024-09-16T09:38:20.360616+0000","last_peered":"2024-09-16T09:38:20.360616+0000","last_clean":"2024-09-16T09:38:20.360616+0000","last_became_active":"2024-09-16T09:25:01.703355+0000","last_became_peered":"2024-09-16T09:25:01.703355+0000","last_unstale":"2024-09-16T09:38:20.360616+0000","last_undegraded":"2024-09-16T09:38:20.360616+0000","last_fullsized":"2024-09-16T09:38:20.360616+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:52.902556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:37:52.902556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:04.411673+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":343,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.367758+0000","last_change":"2024-09-16T09:37:46.378016+0000","last_active":"2024-09-16T09:38:20.367758+0000","last_peered":"2024-09-16T09:38:20.367758+0000","last_clean":"2024-09-16T09:38:20.367758+0000","last_became_active":"2024-09-16T09:33:30.251186+0000","last_became_peered":"2024-09-16T09:33:30.251186+0000","last_unstale":"2024-09-16T09:38:20.367758+0000","last_undegraded":"2024-09-16T09:38:20.367758+0000","last_fullsized":"2024-09-16T09:38:20.367758+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:46.377951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:37:46.377951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:54.699212+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,0],"acting":[4,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":633,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.365099+0000","last_change":"2024-09-16T09:36:51.640537+0000","last_active":"2024-09-16T09:38:20.365099+0000","last_peered":"2024-09-16T09:38:20.365099+0000","last_clean":"2024-09-16T09:38:20.365099+0000","last_became_active":"2024-09-16T09:33:21.337291+0000","last_became_peered":"2024-09-16T09:33:21.337291+0000","last_unstale":"2024-09-16T09:38:20.365099+0000","last_undegraded":"2024-09-16T09:38:20.365099+0000","last_fullsized":"2024-09-16T09:38:20.365099+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:51.640472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:36:51.640472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4],"acting":[3,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"390'3526","reported_seq":3565,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:22.948939+0000","last_change":"2024-09-16T09:37:45.402477+0000","last_active":"2024-09-16T09:38:22.948939+0000","last_peered":"2024-09-16T09:38:22.948939+0000","last_clean":"2024-09-16T09:38:22.948939+0000","last_became_active":"2024-09-16T09:37:45.401847+0000","last_became_peered":"2024-09-16T09:37:45.401847+0000","last_unstale":"2024-09-16T09:38:22.948939+0000","last_undegraded":"2024-09-16T09:38:22.948939+0000","last_fullsized":"2024-09-16T09:38:22.948939+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_clean_scrub_stamp":"2024-09-16T09:37:44.378497+0000","objects_scrubbed":0,"log_size":3526,"log_dups_size":0,"ondisk_log_size":3526,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:05.829127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":231079936,"num_objects":3526,"num_object_clones":0,"num_object_copies":10578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3526,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3526,"num_write_kb":225664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,0],"acting":[4,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.9","version":"73'79","reported_seq":758,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.367590+0000","last_change":"2024-09-16T09:29:19.013948+0000","last_active":"2024-09-16T09:38:20.367590+0000","last_peered":"2024-09-16T09:38:20.367590+0000","last_clean":"2024-09-16T09:38:20.367590+0000","last_became_active":"2024-09-16T09:26:35.126066+0000","last_became_peered":"2024-09-16T09:26:35.126066+0000","last_unstale":"2024-09-16T09:38:20.367590+0000","last_undegraded":"2024-09-16T09:38:20.367590+0000","last_fullsized":"2024-09-16T09:38:20.367590+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:19.013865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:19.013865+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:10.452624+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"285'20","reported_seq":713,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.367984+0000","last_change":"2024-09-16T09:33:24.154567+0000","last_active":"2024-09-16T09:38:20.367984+0000","last_peered":"2024-09-16T09:38:20.367984+0000","last_clean":"2024-09-16T09:38:20.367984+0000","last_became_active":"2024-09-16T09:25:03.707723+0000","last_became_peered":"2024-09-16T09:25:03.707723+0000","last_unstale":"2024-09-16T09:38:20.367984+0000","last_undegraded":"2024-09-16T09:38:20.367984+0000","last_fullsized":"2024-09-16T09:38:20.367984+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-16T09:33:24.154507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:33:24.154507+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:14.304180+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":561,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.370191+0000","last_change":"2024-09-16T09:33:10.528181+0000","last_active":"2024-09-16T09:38:20.370191+0000","last_peered":"2024-09-16T09:38:20.370191+0000","last_clean":"2024-09-16T09:38:20.370191+0000","last_became_active":"2024-09-16T09:31:09.288825+0000","last_became_peered":"2024-09-16T09:31:09.288825+0000","last_unstale":"2024-09-16T09:38:20.370191+0000","last_undegraded":"2024-09-16T09:38:20.370191+0000","last_fullsized":"2024-09-16T09:38:20.370191+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:10.528115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:10.528115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:34.688091+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"228'9","reported_seq":436,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.366051+0000","last_change":"2024-09-16T09:37:16.515947+0000","last_active":"2024-09-16T09:38:20.366051+0000","last_peered":"2024-09-16T09:38:20.366051+0000","last_clean":"2024-09-16T09:38:20.366051+0000","last_became_active":"2024-09-16T09:33:21.337213+0000","last_became_peered":"2024-09-16T09:33:21.337213+0000","last_unstale":"2024-09-16T09:38:20.366051+0000","last_undegraded":"2024-09-16T09:38:20.366051+0000","last_fullsized":"2024-09-16T09:38:20.366051+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"228'9","last_scrub_stamp":"2024-09-16T09:37:16.514010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:37:16.514010+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":707,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.369301+0000","last_change":"2024-09-16T09:35:36.061013+0000","last_active":"2024-09-16T09:38:20.369301+0000","last_peered":"2024-09-16T09:38:20.369301+0000","last_clean":"2024-09-16T09:38:20.369301+0000","last_became_active":"2024-09-16T09:25:07.718958+0000","last_became_peered":"2024-09-16T09:25:07.718958+0000","last_unstale":"2024-09-16T09:38:20.369301+0000","last_undegraded":"2024-09-16T09:38:20.369301+0000","last_fullsized":"2024-09-16T09:38:20.369301+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:35:36.060942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:35:36.060942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":157,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.5","version":"390'3637","reported_seq":3676,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:24.178172+0000","last_change":"2024-09-16T09:37:45.463538+0000","last_active":"2024-09-16T09:38:24.178172+0000","last_peered":"2024-09-16T09:38:24.178172+0000","last_clean":"2024-09-16T09:38:24.178172+0000","last_became_active":"2024-09-16T09:37:45.462778+0000","last_became_peered":"2024-09-16T09:37:45.462778+0000","last_unstale":"2024-09-16T09:38:24.178172+0000","last_undegraded":"2024-09-16T09:38:24.178172+0000","last_fullsized":"2024-09-16T09:38:24.178172+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_clean_scrub_stamp":"2024-09-16T09:37:44.378497+0000","objects_scrubbed":0,"log_size":3637,"log_dups_size":0,"ondisk_log_size":3637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:09.642344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":238354432,"num_objects":3637,"num_object_clones":0,"num_object_copies":10911,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3637,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3637,"num_write_kb":232768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.e","version":"73'79","reported_seq":500,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.369268+0000","last_change":"2024-09-16T09:35:41.979877+0000","last_active":"2024-09-16T09:38:20.369268+0000","last_peered":"2024-09-16T09:38:20.369268+0000","last_clean":"2024-09-16T09:38:20.369268+0000","last_became_active":"2024-09-16T09:30:10.705072+0000","last_became_peered":"2024-09-16T09:30:10.705072+0000","last_unstale":"2024-09-16T09:38:20.369268+0000","last_undegraded":"2024-09-16T09:38:20.369268+0000","last_fullsized":"2024-09-16T09:38:20.369268+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:35:41.979814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:35:41.979814+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:20.177576+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,3],"acting":[4,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.b","version":"285'15","reported_seq":749,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.369972+0000","last_change":"2024-09-16T09:36:55.651561+0000","last_active":"2024-09-16T09:38:20.369972+0000","last_peered":"2024-09-16T09:38:20.369972+0000","last_clean":"2024-09-16T09:38:20.369972+0000","last_became_active":"2024-09-16T09:36:31.590202+0000","last_became_peered":"2024-09-16T09:36:31.590202+0000","last_unstale":"2024-09-16T09:38:20.369972+0000","last_undegraded":"2024-09-16T09:38:20.369972+0000","last_fullsized":"2024-09-16T09:38:20.369972+0000","mapping_epoch":316,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":317,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'15","last_scrub_stamp":"2024-09-16T09:36:55.651496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:36:55.651496+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":708,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.364397+0000","last_change":"2024-09-16T09:37:31.033631+0000","last_active":"2024-09-16T09:38:20.364397+0000","last_peered":"2024-09-16T09:38:20.364397+0000","last_clean":"2024-09-16T09:38:20.364397+0000","last_became_active":"2024-09-16T09:31:09.276516+0000","last_became_peered":"2024-09-16T09:31:09.276516+0000","last_unstale":"2024-09-16T09:38:20.364397+0000","last_undegraded":"2024-09-16T09:38:20.364397+0000","last_fullsized":"2024-09-16T09:38:20.364397+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:31.033577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:37:31.033577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:49.512309+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":629,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.367360+0000","last_change":"2024-09-16T09:33:40.260547+0000","last_active":"2024-09-16T09:38:20.367360+0000","last_peered":"2024-09-16T09:38:20.367360+0000","last_clean":"2024-09-16T09:38:20.367360+0000","last_became_active":"2024-09-16T09:33:21.337254+0000","last_became_peered":"2024-09-16T09:33:21.337254+0000","last_unstale":"2024-09-16T09:38:20.367360+0000","last_undegraded":"2024-09-16T09:38:20.367360+0000","last_fullsized":"2024-09-16T09:38:20.367360+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:40.260500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:40.260500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:39.339396+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3],"acting":[4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":700,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.373606+0000","last_change":"2024-09-16T09:36:53.647305+0000","last_active":"2024-09-16T09:38:20.373606+0000","last_peered":"2024-09-16T09:38:20.373606+0000","last_clean":"2024-09-16T09:38:20.373606+0000","last_became_active":"2024-09-16T09:25:07.718778+0000","last_became_peered":"2024-09-16T09:25:07.718778+0000","last_unstale":"2024-09-16T09:38:20.373606+0000","last_undegraded":"2024-09-16T09:38:20.373606+0000","last_fullsized":"2024-09-16T09:38:20.373606+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:53.647256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:36:53.647256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"390'3676","reported_seq":3715,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:24.229720+0000","last_change":"2024-09-16T09:37:45.400129+0000","last_active":"2024-09-16T09:38:24.229720+0000","last_peered":"2024-09-16T09:38:24.229720+0000","last_clean":"2024-09-16T09:38:24.229720+0000","last_became_active":"2024-09-16T09:37:45.399844+0000","last_became_peered":"2024-09-16T09:37:45.399844+0000","last_unstale":"2024-09-16T09:38:24.229720+0000","last_undegraded":"2024-09-16T09:38:24.229720+0000","last_fullsized":"2024-09-16T09:38:24.229720+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_clean_scrub_stamp":"2024-09-16T09:37:44.378497+0000","objects_scrubbed":0,"log_size":3676,"log_dups_size":0,"ondisk_log_size":3676,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:51.633388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":240910336,"num_objects":3676,"num_object_clones":0,"num_object_copies":11028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3676,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3676,"num_write_kb":235264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.f","version":"73'79","reported_seq":752,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.361589+0000","last_change":"2024-09-16T09:34:53.866426+0000","last_active":"2024-09-16T09:38:20.361589+0000","last_peered":"2024-09-16T09:38:20.361589+0000","last_clean":"2024-09-16T09:38:20.361589+0000","last_became_active":"2024-09-16T09:30:10.693477+0000","last_became_peered":"2024-09-16T09:30:10.693477+0000","last_unstale":"2024-09-16T09:38:20.361589+0000","last_undegraded":"2024-09-16T09:38:20.361589+0000","last_fullsized":"2024-09-16T09:38:20.361589+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:34:53.866337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:34:53.866337+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:59.878482+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"285'35","reported_seq":763,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.360402+0000","last_change":"2024-09-16T09:37:28.078195+0000","last_active":"2024-09-16T09:38:20.360402+0000","last_peered":"2024-09-16T09:38:20.360402+0000","last_clean":"2024-09-16T09:38:20.360402+0000","last_became_active":"2024-09-16T09:25:03.712783+0000","last_became_peered":"2024-09-16T09:25:03.712783+0000","last_unstale":"2024-09-16T09:38:20.360402+0000","last_undegraded":"2024-09-16T09:38:20.360402+0000","last_fullsized":"2024-09-16T09:38:20.360402+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'35","last_scrub_stamp":"2024-09-16T09:37:28.077841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:37:28.077841+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:52.547070+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":56,"num_read_kb":45,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"59'1","reported_seq":715,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.366205+0000","last_change":"2024-09-16T09:37:56.528872+0000","last_active":"2024-09-16T09:38:20.366205+0000","last_peered":"2024-09-16T09:38:20.366205+0000","last_clean":"2024-09-16T09:38:20.366205+0000","last_became_active":"2024-09-16T09:25:01.708069+0000","last_became_peered":"2024-09-16T09:25:01.708069+0000","last_unstale":"2024-09-16T09:38:20.366205+0000","last_undegraded":"2024-09-16T09:38:20.366205+0000","last_fullsized":"2024-09-16T09:38:20.366205+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:37:56.528796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:37:56.528796+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:57.114353+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":717,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.365962+0000","last_change":"2024-09-16T09:37:53.518635+0000","last_active":"2024-09-16T09:38:20.365962+0000","last_peered":"2024-09-16T09:38:20.365962+0000","last_clean":"2024-09-16T09:38:20.365962+0000","last_became_active":"2024-09-16T09:37:53.518284+0000","last_became_peered":"2024-09-16T09:37:53.518284+0000","last_unstale":"2024-09-16T09:38:20.365962+0000","last_undegraded":"2024-09-16T09:38:20.365962+0000","last_fullsized":"2024-09-16T09:38:20.365962+0000","mapping_epoch":382,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":383,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:59.543358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:36:59.543358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:24.211506+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":682,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.367424+0000","last_change":"2024-09-16T09:36:48.649739+0000","last_active":"2024-09-16T09:38:20.367424+0000","last_peered":"2024-09-16T09:38:20.367424+0000","last_clean":"2024-09-16T09:38:20.367424+0000","last_became_active":"2024-09-16T09:25:07.718750+0000","last_became_peered":"2024-09-16T09:25:07.718750+0000","last_unstale":"2024-09-16T09:38:20.367424+0000","last_undegraded":"2024-09-16T09:38:20.367424+0000","last_fullsized":"2024-09-16T09:38:20.367424+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:48.649696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:36:48.649696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.7","version":"390'3540","reported_seq":3579,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:22.720925+0000","last_change":"2024-09-16T09:37:45.404688+0000","last_active":"2024-09-16T09:38:22.720925+0000","last_peered":"2024-09-16T09:38:22.720925+0000","last_clean":"2024-09-16T09:38:22.720925+0000","last_became_active":"2024-09-16T09:37:45.404354+0000","last_became_peered":"2024-09-16T09:37:45.404354+0000","last_unstale":"2024-09-16T09:38:22.720925+0000","last_undegraded":"2024-09-16T09:38:22.720925+0000","last_fullsized":"2024-09-16T09:38:22.720925+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_clean_scrub_stamp":"2024-09-16T09:37:44.378497+0000","objects_scrubbed":0,"log_size":3540,"log_dups_size":0,"ondisk_log_size":3540,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:13.592270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":231997440,"num_objects":3540,"num_object_clones":0,"num_object_copies":10620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3540,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3540,"num_write_kb":226560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.c","version":"73'79","reported_seq":441,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.363434+0000","last_change":"2024-09-16T09:36:53.697797+0000","last_active":"2024-09-16T09:38:20.363434+0000","last_peered":"2024-09-16T09:38:20.363434+0000","last_clean":"2024-09-16T09:38:20.363434+0000","last_became_active":"2024-09-16T09:31:09.273770+0000","last_became_peered":"2024-09-16T09:31:09.273770+0000","last_unstale":"2024-09-16T09:38:20.363434+0000","last_undegraded":"2024-09-16T09:38:20.363434+0000","last_fullsized":"2024-09-16T09:38:20.363434+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:36:53.697742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:36:53.697742+0000","objects_scrubbed":1,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:00.926871+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":327680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"285'18","reported_seq":778,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.370139+0000","last_change":"2024-09-16T09:36:50.603724+0000","last_active":"2024-09-16T09:38:20.370139+0000","last_peered":"2024-09-16T09:38:20.370139+0000","last_clean":"2024-09-16T09:38:20.370139+0000","last_became_active":"2024-09-16T09:31:09.269047+0000","last_became_peered":"2024-09-16T09:31:09.269047+0000","last_unstale":"2024-09-16T09:38:20.370139+0000","last_undegraded":"2024-09-16T09:38:20.370139+0000","last_fullsized":"2024-09-16T09:38:20.370139+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'18","last_scrub_stamp":"2024-09-16T09:36:50.603660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:36:50.603660+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":709,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.366359+0000","last_change":"2024-09-16T09:37:57.529616+0000","last_active":"2024-09-16T09:38:20.366359+0000","last_peered":"2024-09-16T09:38:20.366359+0000","last_clean":"2024-09-16T09:38:20.366359+0000","last_became_active":"2024-09-16T09:25:01.707528+0000","last_became_peered":"2024-09-16T09:25:01.707528+0000","last_unstale":"2024-09-16T09:38:20.366359+0000","last_undegraded":"2024-09-16T09:38:20.366359+0000","last_fullsized":"2024-09-16T09:38:20.366359+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:57.529554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:37:57.529554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:05.467113+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":355,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.367857+0000","last_change":"2024-09-16T09:37:57.449473+0000","last_active":"2024-09-16T09:38:20.367857+0000","last_peered":"2024-09-16T09:38:20.367857+0000","last_clean":"2024-09-16T09:38:20.367857+0000","last_became_active":"2024-09-16T09:33:21.337309+0000","last_became_peered":"2024-09-16T09:33:21.337309+0000","last_unstale":"2024-09-16T09:38:20.367857+0000","last_undegraded":"2024-09-16T09:38:20.367857+0000","last_fullsized":"2024-09-16T09:38:20.367857+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:57.449105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:37:57.449105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:12.289276+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":699,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:21.097872+0000","last_change":"2024-09-16T09:38:21.097872+0000","last_active":"2024-09-16T09:38:21.097872+0000","last_peered":"2024-09-16T09:38:21.097872+0000","last_clean":"2024-09-16T09:38:21.097872+0000","last_became_active":"2024-09-16T09:25:07.718899+0000","last_became_peered":"2024-09-16T09:25:07.718899+0000","last_unstale":"2024-09-16T09:38:21.097872+0000","last_undegraded":"2024-09-16T09:38:21.097872+0000","last_fullsized":"2024-09-16T09:38:21.097872+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:38:21.097804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:38:21.097804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:46.836091+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.6","version":"390'3447","reported_seq":3486,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:23.237274+0000","last_change":"2024-09-16T09:37:45.404587+0000","last_active":"2024-09-16T09:38:23.237274+0000","last_peered":"2024-09-16T09:38:23.237274+0000","last_clean":"2024-09-16T09:38:23.237274+0000","last_became_active":"2024-09-16T09:37:45.404263+0000","last_became_peered":"2024-09-16T09:37:45.404263+0000","last_unstale":"2024-09-16T09:38:23.237274+0000","last_undegraded":"2024-09-16T09:38:23.237274+0000","last_fullsized":"2024-09-16T09:38:23.237274+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_clean_scrub_stamp":"2024-09-16T09:37:44.378497+0000","objects_scrubbed":0,"log_size":3447,"log_dups_size":0,"ondisk_log_size":3447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:58.611017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":225902592,"num_objects":3447,"num_object_clones":0,"num_object_copies":10341,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3447,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3447,"num_write_kb":220608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.d","version":"73'79","reported_seq":491,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.365700+0000","last_change":"2024-09-16T09:32:44.536868+0000","last_active":"2024-09-16T09:38:20.365700+0000","last_peered":"2024-09-16T09:38:20.365700+0000","last_clean":"2024-09-16T09:38:20.365700+0000","last_became_active":"2024-09-16T09:30:10.709958+0000","last_became_peered":"2024-09-16T09:30:10.709958+0000","last_unstale":"2024-09-16T09:38:20.365700+0000","last_undegraded":"2024-09-16T09:38:20.365700+0000","last_fullsized":"2024-09-16T09:38:20.365700+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:32:44.536799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:32:44.536799+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:18.434560+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"285'27","reported_seq":734,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.363376+0000","last_change":"2024-09-16T09:37:30.081518+0000","last_active":"2024-09-16T09:38:20.363376+0000","last_peered":"2024-09-16T09:38:20.363376+0000","last_clean":"2024-09-16T09:38:20.363376+0000","last_became_active":"2024-09-16T09:25:03.715733+0000","last_became_peered":"2024-09-16T09:25:03.715733+0000","last_unstale":"2024-09-16T09:38:20.363376+0000","last_undegraded":"2024-09-16T09:38:20.363376+0000","last_fullsized":"2024-09-16T09:38:20.363376+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'27","last_scrub_stamp":"2024-09-16T09:37:30.081447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:37:30.081447+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:45.797304+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":713,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.367339+0000","last_change":"2024-09-16T09:37:17.544132+0000","last_active":"2024-09-16T09:38:20.367339+0000","last_peered":"2024-09-16T09:38:20.367339+0000","last_clean":"2024-09-16T09:38:20.367339+0000","last_became_active":"2024-09-16T09:25:01.707372+0000","last_became_peered":"2024-09-16T09:25:01.707372+0000","last_unstale":"2024-09-16T09:38:20.367339+0000","last_undegraded":"2024-09-16T09:38:20.367339+0000","last_fullsized":"2024-09-16T09:38:20.367339+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-16T09:37:17.544069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:37:17.544069+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:23.672102+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"66'8","reported_seq":863,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.367535+0000","last_change":"2024-09-16T09:30:05.138362+0000","last_active":"2024-09-16T09:38:20.367535+0000","last_peered":"2024-09-16T09:38:20.367535+0000","last_clean":"2024-09-16T09:38:20.367535+0000","last_became_active":"2024-09-16T09:25:05.716189+0000","last_became_peered":"2024-09-16T09:25:05.716189+0000","last_unstale":"2024-09-16T09:38:20.367535+0000","last_undegraded":"2024-09-16T09:38:20.367535+0000","last_fullsized":"2024-09-16T09:38:20.367535+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:30:05.138288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:30:05.138288+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:14.462129+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":683,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.365185+0000","last_change":"2024-09-16T09:37:47.244099+0000","last_active":"2024-09-16T09:38:20.365185+0000","last_peered":"2024-09-16T09:38:20.365185+0000","last_clean":"2024-09-16T09:38:20.365185+0000","last_became_active":"2024-09-16T09:25:07.717213+0000","last_became_peered":"2024-09-16T09:25:07.717213+0000","last_unstale":"2024-09-16T09:38:20.365185+0000","last_undegraded":"2024-09-16T09:38:20.365185+0000","last_fullsized":"2024-09-16T09:38:20.365185+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:47.244032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:37:47.244032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:48.590206+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.12","version":"73'79","reported_seq":500,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.365402+0000","last_change":"2024-09-16T09:36:44.171217+0000","last_active":"2024-09-16T09:38:20.365402+0000","last_peered":"2024-09-16T09:38:20.365402+0000","last_clean":"2024-09-16T09:38:20.365402+0000","last_became_active":"2024-09-16T09:30:10.731413+0000","last_became_peered":"2024-09-16T09:30:10.731413+0000","last_unstale":"2024-09-16T09:38:20.365402+0000","last_undegraded":"2024-09-16T09:38:20.365402+0000","last_fullsized":"2024-09-16T09:38:20.365402+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:36:44.171151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:36:44.171151+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":717,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.359932+0000","last_change":"2024-09-16T09:37:55.863661+0000","last_active":"2024-09-16T09:38:20.359932+0000","last_peered":"2024-09-16T09:38:20.359932+0000","last_clean":"2024-09-16T09:38:20.359932+0000","last_became_active":"2024-09-16T09:37:53.518354+0000","last_became_peered":"2024-09-16T09:37:53.518354+0000","last_unstale":"2024-09-16T09:38:20.359932+0000","last_undegraded":"2024-09-16T09:38:20.359932+0000","last_fullsized":"2024-09-16T09:38:20.359932+0000","mapping_epoch":382,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":383,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:55.863613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:37:55.863613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:01.033155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"285'12","reported_seq":461,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.365345+0000","last_change":"2024-09-16T09:37:20.749151+0000","last_active":"2024-09-16T09:38:20.365345+0000","last_peered":"2024-09-16T09:38:20.365345+0000","last_clean":"2024-09-16T09:38:20.365345+0000","last_became_active":"2024-09-16T09:31:09.270685+0000","last_became_peered":"2024-09-16T09:31:09.270685+0000","last_unstale":"2024-09-16T09:38:20.365345+0000","last_undegraded":"2024-09-16T09:38:20.365345+0000","last_fullsized":"2024-09-16T09:38:20.365345+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'12","last_scrub_stamp":"2024-09-16T09:37:20.749101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:37:20.749101+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:32.592645+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":17,"num_read_kb":14,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":694,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.366021+0000","last_change":"2024-09-16T09:36:38.129449+0000","last_active":"2024-09-16T09:38:20.366021+0000","last_peered":"2024-09-16T09:38:20.366021+0000","last_clean":"2024-09-16T09:38:20.366021+0000","last_became_active":"2024-09-16T09:25:05.715616+0000","last_became_peered":"2024-09-16T09:25:05.715616+0000","last_unstale":"2024-09-16T09:38:20.366021+0000","last_undegraded":"2024-09-16T09:38:20.366021+0000","last_fullsized":"2024-09-16T09:38:20.366021+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:38.129314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:36:38.129314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":681,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.365843+0000","last_change":"2024-09-16T09:36:37.715166+0000","last_active":"2024-09-16T09:38:20.365843+0000","last_peered":"2024-09-16T09:38:20.365843+0000","last_clean":"2024-09-16T09:38:20.365843+0000","last_became_active":"2024-09-16T09:25:07.721059+0000","last_became_peered":"2024-09-16T09:25:07.721059+0000","last_unstale":"2024-09-16T09:38:20.365843+0000","last_undegraded":"2024-09-16T09:38:20.365843+0000","last_fullsized":"2024-09-16T09:38:20.365843+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:37.715114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:36:37.715114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.13","version":"73'79","reported_seq":586,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.361577+0000","last_change":"2024-09-16T09:34:48.260121+0000","last_active":"2024-09-16T09:38:20.361577+0000","last_peered":"2024-09-16T09:38:20.361577+0000","last_clean":"2024-09-16T09:38:20.361577+0000","last_became_active":"2024-09-16T09:33:30.248497+0000","last_became_peered":"2024-09-16T09:33:30.248497+0000","last_unstale":"2024-09-16T09:38:20.361577+0000","last_undegraded":"2024-09-16T09:38:20.361577+0000","last_fullsized":"2024-09-16T09:38:20.361577+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:34:48.260041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:34:48.260041+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:11.022422+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"285'15","reported_seq":774,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.368624+0000","last_change":"2024-09-16T09:33:48.681742+0000","last_active":"2024-09-16T09:38:20.368624+0000","last_peered":"2024-09-16T09:38:20.368624+0000","last_clean":"2024-09-16T09:38:20.368624+0000","last_became_active":"2024-09-16T09:31:09.273498+0000","last_became_peered":"2024-09-16T09:31:09.273498+0000","last_unstale":"2024-09-16T09:38:20.368624+0000","last_undegraded":"2024-09-16T09:38:20.368624+0000","last_fullsized":"2024-09-16T09:38:20.368624+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:33:48.681650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:33:48.681650+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:59.423578+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":701,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.367827+0000","last_change":"2024-09-16T09:33:13.312434+0000","last_active":"2024-09-16T09:38:20.367827+0000","last_peered":"2024-09-16T09:38:20.367827+0000","last_clean":"2024-09-16T09:38:20.367827+0000","last_became_active":"2024-09-16T09:25:01.706491+0000","last_became_peered":"2024-09-16T09:25:01.706491+0000","last_unstale":"2024-09-16T09:38:20.367827+0000","last_undegraded":"2024-09-16T09:38:20.367827+0000","last_fullsized":"2024-09-16T09:38:20.367827+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:13.312344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:13.312344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:37.450175+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":459,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.366365+0000","last_change":"2024-09-16T09:36:54.596000+0000","last_active":"2024-09-16T09:38:20.366365+0000","last_peered":"2024-09-16T09:38:20.366365+0000","last_clean":"2024-09-16T09:38:20.366365+0000","last_became_active":"2024-09-16T09:33:30.248992+0000","last_became_peered":"2024-09-16T09:33:30.248992+0000","last_unstale":"2024-09-16T09:38:20.366365+0000","last_undegraded":"2024-09-16T09:38:20.366365+0000","last_fullsized":"2024-09-16T09:38:20.366365+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:54.595924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:36:54.595924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":699,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.364990+0000","last_change":"2024-09-16T09:36:58.536167+0000","last_active":"2024-09-16T09:38:20.364990+0000","last_peered":"2024-09-16T09:38:20.364990+0000","last_clean":"2024-09-16T09:38:20.364990+0000","last_became_active":"2024-09-16T09:25:07.722050+0000","last_became_peered":"2024-09-16T09:25:07.722050+0000","last_unstale":"2024-09-16T09:38:20.364990+0000","last_undegraded":"2024-09-16T09:38:20.364990+0000","last_fullsized":"2024-09-16T09:38:20.364990+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:58.536121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:36:58.536121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:20.078148+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.10","version":"73'79","reported_seq":500,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.363153+0000","last_change":"2024-09-16T09:36:41.169117+0000","last_active":"2024-09-16T09:38:20.363153+0000","last_peered":"2024-09-16T09:38:20.363153+0000","last_clean":"2024-09-16T09:38:20.363153+0000","last_became_active":"2024-09-16T09:30:10.692770+0000","last_became_peered":"2024-09-16T09:30:10.692770+0000","last_unstale":"2024-09-16T09:38:20.363153+0000","last_undegraded":"2024-09-16T09:38:20.363153+0000","last_fullsized":"2024-09-16T09:38:20.363153+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:36:41.169069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:36:41.169069+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"285'15","reported_seq":723,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.364844+0000","last_change":"2024-09-16T09:37:55.203919+0000","last_active":"2024-09-16T09:38:20.364844+0000","last_peered":"2024-09-16T09:38:20.364844+0000","last_clean":"2024-09-16T09:38:20.364844+0000","last_became_active":"2024-09-16T09:25:03.717310+0000","last_became_peered":"2024-09-16T09:25:03.717310+0000","last_unstale":"2024-09-16T09:38:20.364844+0000","last_undegraded":"2024-09-16T09:38:20.364844+0000","last_fullsized":"2024-09-16T09:38:20.364844+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'15","last_scrub_stamp":"2024-09-16T09:37:55.203856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:37:55.203856+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:23.098469+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":733,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.368244+0000","last_change":"2024-09-16T09:37:50.471532+0000","last_active":"2024-09-16T09:38:20.368244+0000","last_peered":"2024-09-16T09:38:20.368244+0000","last_clean":"2024-09-16T09:38:20.368244+0000","last_became_active":"2024-09-16T09:31:09.272848+0000","last_became_peered":"2024-09-16T09:31:09.272848+0000","last_unstale":"2024-09-16T09:38:20.368244+0000","last_undegraded":"2024-09-16T09:38:20.368244+0000","last_fullsized":"2024-09-16T09:38:20.368244+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:50.471486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:37:50.471486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:09.546185+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"187'11","reported_seq":708,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.365867+0000","last_change":"2024-09-16T09:37:19.540982+0000","last_active":"2024-09-16T09:38:20.365867+0000","last_peered":"2024-09-16T09:38:20.365867+0000","last_clean":"2024-09-16T09:38:20.365867+0000","last_became_active":"2024-09-16T09:31:09.277688+0000","last_became_peered":"2024-09-16T09:31:09.277688+0000","last_unstale":"2024-09-16T09:38:20.365867+0000","last_undegraded":"2024-09-16T09:38:20.365867+0000","last_fullsized":"2024-09-16T09:38:20.365867+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"187'11","last_scrub_stamp":"2024-09-16T09:37:19.540886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:37:19.540886+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:47.772515+0000","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":707,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.368892+0000","last_change":"2024-09-16T09:36:49.628523+0000","last_active":"2024-09-16T09:38:20.368892+0000","last_peered":"2024-09-16T09:38:20.368892+0000","last_clean":"2024-09-16T09:38:20.368892+0000","last_became_active":"2024-09-16T09:25:07.718810+0000","last_became_peered":"2024-09-16T09:25:07.718810+0000","last_unstale":"2024-09-16T09:38:20.368892+0000","last_undegraded":"2024-09-16T09:38:20.368892+0000","last_fullsized":"2024-09-16T09:38:20.368892+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:49.628477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:36:49.628477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.11","version":"73'79","reported_seq":586,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.373804+0000","last_change":"2024-09-16T09:37:15.623104+0000","last_active":"2024-09-16T09:38:20.373804+0000","last_peered":"2024-09-16T09:38:20.373804+0000","last_clean":"2024-09-16T09:38:20.373804+0000","last_became_active":"2024-09-16T09:31:09.272208+0000","last_became_peered":"2024-09-16T09:31:09.272208+0000","last_unstale":"2024-09-16T09:38:20.373804+0000","last_undegraded":"2024-09-16T09:38:20.373804+0000","last_fullsized":"2024-09-16T09:38:20.373804+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:37:15.623041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:37:15.623041+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.14","version":"285'20","reported_seq":384,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.366004+0000","last_change":"2024-09-16T09:36:52.609515+0000","last_active":"2024-09-16T09:38:20.366004+0000","last_peered":"2024-09-16T09:38:20.366004+0000","last_clean":"2024-09-16T09:38:20.366004+0000","last_became_active":"2024-09-16T09:33:30.249769+0000","last_became_peered":"2024-09-16T09:33:30.249769+0000","last_unstale":"2024-09-16T09:38:20.366004+0000","last_undegraded":"2024-09-16T09:38:20.366004+0000","last_fullsized":"2024-09-16T09:38:20.366004+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'20","last_scrub_stamp":"2024-09-16T09:36:52.609440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:36:52.609440+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":693,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.365904+0000","last_change":"2024-09-16T09:29:56.021413+0000","last_active":"2024-09-16T09:38:20.365904+0000","last_peered":"2024-09-16T09:38:20.365904+0000","last_clean":"2024-09-16T09:38:20.365904+0000","last_became_active":"2024-09-16T09:25:01.707513+0000","last_became_peered":"2024-09-16T09:25:01.707513+0000","last_unstale":"2024-09-16T09:38:20.365904+0000","last_undegraded":"2024-09-16T09:38:20.365904+0000","last_fullsized":"2024-09-16T09:38:20.365904+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:56.021340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:56.021340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:13.464027+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"66'8","reported_seq":871,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.364665+0000","last_change":"2024-09-16T09:33:39.388721+0000","last_active":"2024-09-16T09:38:20.364665+0000","last_peered":"2024-09-16T09:38:20.364665+0000","last_clean":"2024-09-16T09:38:20.364665+0000","last_became_active":"2024-09-16T09:25:05.713833+0000","last_became_peered":"2024-09-16T09:25:05.713833+0000","last_unstale":"2024-09-16T09:38:20.364665+0000","last_undegraded":"2024-09-16T09:38:20.364665+0000","last_fullsized":"2024-09-16T09:38:20.364665+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:33:39.388463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:39.388463+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":713,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.368249+0000","last_change":"2024-09-16T09:33:42.666290+0000","last_active":"2024-09-16T09:38:20.368249+0000","last_peered":"2024-09-16T09:38:20.368249+0000","last_clean":"2024-09-16T09:38:20.368249+0000","last_became_active":"2024-09-16T09:31:09.275168+0000","last_became_peered":"2024-09-16T09:31:09.275168+0000","last_unstale":"2024-09-16T09:38:20.368249+0000","last_undegraded":"2024-09-16T09:38:20.368249+0000","last_fullsized":"2024-09-16T09:38:20.368249+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:42.666200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:33:42.666200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"285'19","reported_seq":741,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.367530+0000","last_change":"2024-09-16T09:37:31.362985+0000","last_active":"2024-09-16T09:38:20.367530+0000","last_peered":"2024-09-16T09:38:20.367530+0000","last_clean":"2024-09-16T09:38:20.367530+0000","last_became_active":"2024-09-16T09:26:33.291265+0000","last_became_peered":"2024-09-16T09:26:33.291265+0000","last_unstale":"2024-09-16T09:38:20.367530+0000","last_undegraded":"2024-09-16T09:38:20.367530+0000","last_fullsized":"2024-09-16T09:38:20.367530+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'19","last_scrub_stamp":"2024-09-16T09:37:31.362927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:37:31.362927+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:37.215450+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":719,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.368184+0000","last_change":"2024-09-16T09:37:53.239593+0000","last_active":"2024-09-16T09:38:20.368184+0000","last_peered":"2024-09-16T09:38:20.368184+0000","last_clean":"2024-09-16T09:38:20.368184+0000","last_became_active":"2024-09-16T09:25:01.705158+0000","last_became_peered":"2024-09-16T09:25:01.705158+0000","last_unstale":"2024-09-16T09:38:20.368184+0000","last_undegraded":"2024-09-16T09:38:20.368184+0000","last_fullsized":"2024-09-16T09:38:20.368184+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:53.239547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:37:53.239547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:02.480253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":639,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.365273+0000","last_change":"2024-09-16T09:37:58.252056+0000","last_active":"2024-09-16T09:38:20.365273+0000","last_peered":"2024-09-16T09:38:20.365273+0000","last_clean":"2024-09-16T09:38:20.365273+0000","last_became_active":"2024-09-16T09:37:41.515278+0000","last_became_peered":"2024-09-16T09:37:41.515278+0000","last_unstale":"2024-09-16T09:38:20.365273+0000","last_undegraded":"2024-09-16T09:38:20.365273+0000","last_fullsized":"2024-09-16T09:38:20.365273+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:58.251994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:37:58.251994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:15.408415+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":546,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.373486+0000","last_change":"2024-09-16T09:34:48.786847+0000","last_active":"2024-09-16T09:38:20.373486+0000","last_peered":"2024-09-16T09:38:20.373486+0000","last_clean":"2024-09-16T09:38:20.373486+0000","last_became_active":"2024-09-16T09:31:09.276042+0000","last_became_peered":"2024-09-16T09:31:09.276042+0000","last_unstale":"2024-09-16T09:38:20.373486+0000","last_undegraded":"2024-09-16T09:38:20.373486+0000","last_fullsized":"2024-09-16T09:38:20.373486+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:48.786785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:34:48.786785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"285'15","reported_seq":414,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.360535+0000","last_change":"2024-09-16T09:37:33.354025+0000","last_active":"2024-09-16T09:38:20.360535+0000","last_peered":"2024-09-16T09:38:20.360535+0000","last_clean":"2024-09-16T09:38:20.360535+0000","last_became_active":"2024-09-16T09:37:33.348223+0000","last_became_peered":"2024-09-16T09:37:33.348223+0000","last_unstale":"2024-09-16T09:38:20.360535+0000","last_undegraded":"2024-09-16T09:38:20.360535+0000","last_fullsized":"2024-09-16T09:38:20.360535+0000","mapping_epoch":367,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":368,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'15","last_scrub_stamp":"2024-09-16T09:36:55.935996+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:36:55.935996+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,2],"acting":[6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":693,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.367547+0000","last_change":"2024-09-16T09:32:47.502667+0000","last_active":"2024-09-16T09:38:20.367547+0000","last_peered":"2024-09-16T09:38:20.367547+0000","last_clean":"2024-09-16T09:38:20.367547+0000","last_became_active":"2024-09-16T09:25:01.706762+0000","last_became_peered":"2024-09-16T09:25:01.706762+0000","last_unstale":"2024-09-16T09:38:20.367547+0000","last_undegraded":"2024-09-16T09:38:20.367547+0000","last_fullsized":"2024-09-16T09:38:20.367547+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:47.502598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:32:47.502598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:17.409843+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":531,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.369684+0000","last_change":"2024-09-16T09:34:46.742640+0000","last_active":"2024-09-16T09:38:20.369684+0000","last_peered":"2024-09-16T09:38:20.369684+0000","last_clean":"2024-09-16T09:38:20.369684+0000","last_became_active":"2024-09-16T09:28:52.982865+0000","last_became_peered":"2024-09-16T09:28:52.982865+0000","last_unstale":"2024-09-16T09:38:20.369684+0000","last_undegraded":"2024-09-16T09:38:20.369684+0000","last_fullsized":"2024-09-16T09:38:20.369684+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:46.742562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:34:46.742562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":691,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.367808+0000","last_change":"2024-09-16T09:34:50.779241+0000","last_active":"2024-09-16T09:38:20.367808+0000","last_peered":"2024-09-16T09:38:20.367808+0000","last_clean":"2024-09-16T09:38:20.367808+0000","last_became_active":"2024-09-16T09:25:07.715301+0000","last_became_peered":"2024-09-16T09:25:07.715301+0000","last_unstale":"2024-09-16T09:38:20.367808+0000","last_undegraded":"2024-09-16T09:38:20.367808+0000","last_fullsized":"2024-09-16T09:38:20.367808+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:50.779164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:34:50.779164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.14","version":"73'79","reported_seq":500,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.364892+0000","last_change":"2024-09-16T09:36:48.597220+0000","last_active":"2024-09-16T09:38:20.364892+0000","last_peered":"2024-09-16T09:38:20.364892+0000","last_clean":"2024-09-16T09:38:20.364892+0000","last_became_active":"2024-09-16T09:30:10.700866+0000","last_became_peered":"2024-09-16T09:30:10.700866+0000","last_unstale":"2024-09-16T09:38:20.364892+0000","last_undegraded":"2024-09-16T09:38:20.364892+0000","last_fullsized":"2024-09-16T09:38:20.364892+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:36:48.597169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:36:48.597169+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:17.024182+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"285'19","reported_seq":581,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.367690+0000","last_change":"2024-09-16T09:28:55.643505+0000","last_active":"2024-09-16T09:38:20.367690+0000","last_peered":"2024-09-16T09:38:20.367690+0000","last_clean":"2024-09-16T09:38:20.367690+0000","last_became_active":"2024-09-16T09:28:52.971132+0000","last_became_peered":"2024-09-16T09:28:52.971132+0000","last_unstale":"2024-09-16T09:38:20.367690+0000","last_undegraded":"2024-09-16T09:38:20.367690+0000","last_fullsized":"2024-09-16T09:38:20.367690+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-16T09:28:55.643448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:55.643448+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:07.375359+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":710,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:21.213911+0000","last_change":"2024-09-16T09:38:21.213911+0000","last_active":"2024-09-16T09:38:21.213911+0000","last_peered":"2024-09-16T09:38:21.213911+0000","last_clean":"2024-09-16T09:38:21.213911+0000","last_became_active":"2024-09-16T09:25:01.700733+0000","last_became_peered":"2024-09-16T09:25:01.700733+0000","last_unstale":"2024-09-16T09:38:21.213911+0000","last_undegraded":"2024-09-16T09:38:21.213911+0000","last_fullsized":"2024-09-16T09:38:21.213911+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:38:21.213844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:38:21.213844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:38.561435+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":693,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.365229+0000","last_change":"2024-09-16T09:33:38.336281+0000","last_active":"2024-09-16T09:38:20.365229+0000","last_peered":"2024-09-16T09:38:20.365229+0000","last_clean":"2024-09-16T09:38:20.365229+0000","last_became_active":"2024-09-16T09:25:05.713479+0000","last_became_peered":"2024-09-16T09:25:05.713479+0000","last_unstale":"2024-09-16T09:38:20.365229+0000","last_undegraded":"2024-09-16T09:38:20.365229+0000","last_fullsized":"2024-09-16T09:38:20.365229+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:38.336221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:38.336221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:36.450450+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":554,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.370121+0000","last_change":"2024-09-16T09:34:51.781047+0000","last_active":"2024-09-16T09:38:20.370121+0000","last_peered":"2024-09-16T09:38:20.370121+0000","last_clean":"2024-09-16T09:38:20.370121+0000","last_became_active":"2024-09-16T09:31:09.274797+0000","last_became_peered":"2024-09-16T09:31:09.274797+0000","last_unstale":"2024-09-16T09:38:20.370121+0000","last_undegraded":"2024-09-16T09:38:20.370121+0000","last_fullsized":"2024-09-16T09:38:20.370121+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:51.780980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:34:51.780980+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"285'8","reported_seq":457,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.361342+0000","last_change":"2024-09-16T09:37:15.672673+0000","last_active":"2024-09-16T09:38:20.361342+0000","last_peered":"2024-09-16T09:38:20.361342+0000","last_clean":"2024-09-16T09:38:20.361342+0000","last_became_active":"2024-09-16T09:31:09.273682+0000","last_became_peered":"2024-09-16T09:31:09.273682+0000","last_unstale":"2024-09-16T09:38:20.361342+0000","last_undegraded":"2024-09-16T09:38:20.361342+0000","last_fullsized":"2024-09-16T09:38:20.361342+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'8","last_scrub_stamp":"2024-09-16T09:37:15.672596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:37:15.672596+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:20.990359+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":9,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":725,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.370997+0000","last_change":"2024-09-16T09:33:18.641115+0000","last_active":"2024-09-16T09:38:20.370997+0000","last_peered":"2024-09-16T09:38:20.370997+0000","last_clean":"2024-09-16T09:38:20.370997+0000","last_became_active":"2024-09-16T09:31:09.274719+0000","last_became_peered":"2024-09-16T09:31:09.274719+0000","last_unstale":"2024-09-16T09:38:20.370997+0000","last_undegraded":"2024-09-16T09:38:20.370997+0000","last_fullsized":"2024-09-16T09:38:20.370997+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:18.641069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:18.641069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:44.551604+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":711,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.360209+0000","last_change":"2024-09-16T09:33:37.941485+0000","last_active":"2024-09-16T09:38:20.360209+0000","last_peered":"2024-09-16T09:38:20.360209+0000","last_clean":"2024-09-16T09:38:20.360209+0000","last_became_active":"2024-09-16T09:25:05.712545+0000","last_became_peered":"2024-09-16T09:25:05.712545+0000","last_unstale":"2024-09-16T09:38:20.360209+0000","last_undegraded":"2024-09-16T09:38:20.360209+0000","last_fullsized":"2024-09-16T09:38:20.360209+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:37.941403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:37.941403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:33.011670+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":697,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.365297+0000","last_change":"2024-09-16T09:37:50.502460+0000","last_active":"2024-09-16T09:38:20.365297+0000","last_peered":"2024-09-16T09:38:20.365297+0000","last_clean":"2024-09-16T09:38:20.365297+0000","last_became_active":"2024-09-16T09:25:07.720117+0000","last_became_peered":"2024-09-16T09:25:07.720117+0000","last_unstale":"2024-09-16T09:38:20.365297+0000","last_undegraded":"2024-09-16T09:38:20.365297+0000","last_fullsized":"2024-09-16T09:38:20.365297+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:50.502398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:37:50.502398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:13.329570+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":334,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.364714+0000","last_change":"2024-09-16T09:36:52.091335+0000","last_active":"2024-09-16T09:38:20.364714+0000","last_peered":"2024-09-16T09:38:20.364714+0000","last_clean":"2024-09-16T09:38:20.364714+0000","last_became_active":"2024-09-16T09:33:30.248036+0000","last_became_peered":"2024-09-16T09:33:30.248036+0000","last_unstale":"2024-09-16T09:38:20.364714+0000","last_undegraded":"2024-09-16T09:38:20.364714+0000","last_fullsized":"2024-09-16T09:38:20.364714+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:52.088284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:36:52.088284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":560,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.368393+0000","last_change":"2024-09-16T09:37:18.825035+0000","last_active":"2024-09-16T09:38:20.368393+0000","last_peered":"2024-09-16T09:38:20.368393+0000","last_clean":"2024-09-16T09:38:20.368393+0000","last_became_active":"2024-09-16T09:31:09.271662+0000","last_became_peered":"2024-09-16T09:31:09.271662+0000","last_unstale":"2024-09-16T09:38:20.368393+0000","last_undegraded":"2024-09-16T09:38:20.368393+0000","last_fullsized":"2024-09-16T09:38:20.368393+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:18.824937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:37:18.824937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:29.541577+0000","scrub_duration":79,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"285'19","reported_seq":739,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.361444+0000","last_change":"2024-09-16T09:37:25.995663+0000","last_active":"2024-09-16T09:38:20.361444+0000","last_peered":"2024-09-16T09:38:20.361444+0000","last_clean":"2024-09-16T09:38:20.361444+0000","last_became_active":"2024-09-16T09:25:03.707769+0000","last_became_peered":"2024-09-16T09:25:03.707769+0000","last_unstale":"2024-09-16T09:38:20.361444+0000","last_undegraded":"2024-09-16T09:38:20.361444+0000","last_fullsized":"2024-09-16T09:38:20.361444+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'19","last_scrub_stamp":"2024-09-16T09:37:25.995607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:37:25.995607+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:34.249870+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":696,"reported_epoch":390,"state":"active+clean","last_fresh":"2024-09-16T09:38:20.368355+0000","last_change":"2024-09-16T09:37:26.646784+0000","last_active":"2024-09-16T09:38:20.368355+0000","last_peered":"2024-09-16T09:38:20.368355+0000","last_clean":"2024-09-16T09:38:20.368355+0000","last_became_active":"2024-09-16T09:25:05.711952+0000","last_became_peered":"2024-09-16T09:25:05.711952+0000","last_unstale":"2024-09-16T09:38:20.368355+0000","last_undegraded":"2024-09-16T09:38:20.368355+0000","last_fullsized":"2024-09-16T09:38:20.368355+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:26.646686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:37:26.646686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:54.298242+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":16,"stat_sum":{"num_bytes":3711303680,"num_objects":56630,"num_object_clones":0,"num_object_copies":169890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56630,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":56630,"num_write_kb":3624320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":11175067648,"data_stored":11175067648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":56630,"ondisk_log_size":56630,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":1117,"num_read_kb":874,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":134,"num_bytes_recovered":3576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":681,"ondisk_log_size":681,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1986,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":794,"num_read_kb":789,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":21,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":327680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1659,"ondisk_log_size":1659,"up":63,"acting":63,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266853,"num_pgs":66,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1461008,"kb_used_data":892544,"kb_used_omap":1,"kb_used_meta":568446,"kb_avail":92284144,"statfs":{"total":95995035648,"available":94498963456,"internally_reserved":0,"allocated":913965056,"data_stored":913334160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":582089162},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495786,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1995452,"kb_used_data":1599144,"kb_used_omap":1,"kb_used_meta":396286,"kb_avail":91749700,"statfs":{"total":95995035648,"available":93951692800,"internally_reserved":0,"allocated":1637523456,"data_stored":1636916038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":405797318},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724721,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1839944,"kb_used_data":1231224,"kb_used_omap":1,"kb_used_meta":608702,"kb_avail":91905208,"statfs":{"total":95995035648,"available":94110932992,"internally_reserved":0,"allocated":1260773376,"data_stored":1260150697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":623311302},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953654,"num_pgs":72,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2203668,"kb_used_data":1753216,"kb_used_omap":1,"kb_used_meta":450430,"kb_avail":91541484,"statfs":{"total":95995035648,"available":93738479616,"internally_reserved":0,"allocated":1795293184,"data_stored":1794662242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":461240774},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149884,"num_pgs":70,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1799008,"kb_used_data":1141712,"kb_used_omap":1,"kb_used_meta":657278,"kb_avail":91946144,"statfs":{"total":95995035648,"available":94152851456,"internally_reserved":0,"allocated":1169113088,"data_stored":1168466702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":673053127},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411521,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1775548,"kb_used_data":1329256,"kb_used_omap":1,"kb_used_meta":446270,"kb_avail":91969604,"statfs":{"total":95995035648,"available":94176874496,"internally_reserved":0,"allocated":1361158144,"data_stored":1360551092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":456980933},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575048,"num_pgs":65,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1347824,"kb_used_data":676640,"kb_used_omap":1,"kb_used_meta":671166,"kb_avail":92397328,"statfs":{"total":95995035648,"available":94614863872,"internally_reserved":0,"allocated":692879360,"data_stored":692215740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":687274437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738574,"num_pgs":62,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2858388,"kb_used_data":2277444,"kb_used_omap":1,"kb_used_meta":580926,"kb_avail":90886764,"statfs":{"total":95995035648,"available":93068046336,"internally_reserved":0,"allocated":2332102656,"data_stored":2331467930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":594868677},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2381971456,"data_stored":2381971456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":691601408,"data_stored":691601408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1396834304,"data_stored":1396834304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1176371200,"data_stored":1176371200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1835859968,"data_stored":1835859968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1165688832,"data_stored":1165688832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1603141632,"data_stored":1603141632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":923598848,"data_stored":923598848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T09:38:27.957 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2024-09-16T09:38:27.958 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd pool set unique_pool_3 pg_num 26 2024-09-16T09:38:28.007 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:38:28.037 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:38:28.065 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:28.066 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:28.066 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:38:28.067 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:38:28.167 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:38:28.213 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:28.214 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:28.214 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:38:28.315 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:38:28.372 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:28.372 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:28.372 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:38:28.473 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:38:28.512 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:28.512 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:28.512 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:38:28.541 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 32 16 64645 64629 126.193 91.75 0.00496654 0.0079198 2024-09-16T09:38:28.613 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:38:28.649 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:28.650 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:28.650 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:38:28.744 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:28 smithi070 ceph-mon[30388]: 6.15 scrub starts 2024-09-16T09:38:28.745 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:28 smithi070 ceph-mon[30388]: 6.15 scrub ok 2024-09-16T09:38:28.745 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:28 smithi070 ceph-mon[40691]: 6.15 scrub starts 2024-09-16T09:38:28.745 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:28 smithi070 ceph-mon[40691]: 6.15 scrub ok 2024-09-16T09:38:28.751 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:38:28.798 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:28.799 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:28.800 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:38:28.900 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:38:28.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:28 smithi084 ceph-mon[32629]: 6.15 scrub starts 2024-09-16T09:38:28.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:28 smithi084 ceph-mon[32629]: 6.15 scrub ok 2024-09-16T09:38:28.938 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:28.939 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:28.940 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:38:29.041 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:38:29.078 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:29.079 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:29.079 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:38:29.180 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:38:29.195 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T09:38:29.196 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [ 2024-09-16T09:38:29.196 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:29.196 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.15975.0:65183 10.3 e391/376 10:c897f365:::benchmark_data_smithi070_166744_object65182:head v 391'4160, mlcod=391'4158)", 2024-09-16T09:38:29.196 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:38:28.908728+0000", 2024-09-16T09:38:29.196 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.011802208, 2024-09-16T09:38:29.196 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.011828862000000001, 2024-09-16T09:38:29.196 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:38:29.196 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:38:29.196 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T09:38:29.196 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:38:29.196 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:29.196 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:38:29.197 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:38:28.908728+0000", 2024-09-16T09:38:29.197 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:38:29.197 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:29.197 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:29.197 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:38:29.197 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:38:28.908728+0000", 2024-09-16T09:38:29.197 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:38:29.197 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:29.197 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:29.197 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:38:29.197 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:38:28.908729+0000", 2024-09-16T09:38:29.197 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.1100000000000005e-07 2024-09-16T09:38:29.198 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:29.198 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:29.198 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:38:29.198 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:38:28.909334+0000", 2024-09-16T09:38:29.198 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00060521299999999995 2024-09-16T09:38:29.198 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:29.198 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:29.198 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:38:29.198 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:38:28.909349+0000", 2024-09-16T09:38:29.198 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.4761e-05 2024-09-16T09:38:29.198 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:29.198 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:29.198 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:38:29.199 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:38:28.909356+0000", 2024-09-16T09:38:29.199 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.9229999999999996e-06 2024-09-16T09:38:29.199 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:29.199 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:29.199 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:38:29.199 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:38:28.909415+0000", 2024-09-16T09:38:29.199 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8653000000000002e-05 2024-09-16T09:38:29.199 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:29.199 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:29.199 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:38:29.199 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:38:28.909432+0000", 2024-09-16T09:38:29.199 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7711999999999999e-05 2024-09-16T09:38:29.199 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:38:29.199 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:38:29.200 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:38:29.200 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:29.200 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:29.200 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.15975.0:65197 10.5 e391/376 10:ade07a92:::benchmark_data_smithi070_166744_object65196:head v 391'4096, mlcod=391'4094)", 2024-09-16T09:38:29.200 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:38:28.918035+0000", 2024-09-16T09:38:29.200 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.0024951159999999999, 2024-09-16T09:38:29.200 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0025804130000000002, 2024-09-16T09:38:29.200 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:38:29.200 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:38:29.200 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T09:38:29.200 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:38:29.200 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:29.200 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:38:29.200 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:38:28.918035+0000", 2024-09-16T09:38:29.201 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:38:29.201 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:29.201 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:29.201 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:38:29.201 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:38:28.918035+0000", 2024-09-16T09:38:29.201 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:38:29.201 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:29.201 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:29.201 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:38:29.201 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:38:28.918037+0000", 2024-09-16T09:38:29.201 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3009999999999999e-06 2024-09-16T09:38:29.201 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:29.201 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:29.201 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:38:29.201 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:38:28.918061+0000", 2024-09-16T09:38:29.202 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.4168e-05 2024-09-16T09:38:29.202 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:29.202 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:29.202 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:38:29.202 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:38:28.918075+0000", 2024-09-16T09:38:29.202 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3973e-05 2024-09-16T09:38:29.202 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:29.202 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:29.202 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:38:29.202 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:38:28.918082+0000", 2024-09-16T09:38:29.202 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.1509999999999998e-06 2024-09-16T09:38:29.202 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:29.202 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:29.203 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:38:29.203 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:38:28.918159+0000", 2024-09-16T09:38:29.203 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.7206999999999994e-05 2024-09-16T09:38:29.203 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:29.203 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:29.203 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:38:29.203 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:38:28.918178+0000", 2024-09-16T09:38:29.203 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.8556e-05 2024-09-16T09:38:29.203 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:38:29.203 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:38:29.203 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:38:29.203 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:29.203 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:29.203 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.15975.0:65194 10.3 e391/376 10:c80d1928:::benchmark_data_smithi070_166744_object65193:head v 391'4161, mlcod=391'4158)", 2024-09-16T09:38:29.203 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:38:28.919428+0000", 2024-09-16T09:38:29.204 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.001102694, 2024-09-16T09:38:29.204 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0012372100000000001, 2024-09-16T09:38:29.204 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:38:29.204 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:38:29.204 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T09:38:29.204 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:38:29.204 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:29.204 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:38:29.204 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:38:28.919428+0000", 2024-09-16T09:38:29.204 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:38:29.204 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:29.204 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:29.204 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:38:29.204 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:38:28.919428+0000", 2024-09-16T09:38:29.204 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:38:29.205 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:29.205 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:29.205 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:38:29.205 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:38:28.919429+0000", 2024-09-16T09:38:29.205 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3099999999999997e-07 2024-09-16T09:38:29.205 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:29.205 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:29.205 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:38:29.205 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:38:28.919467+0000", 2024-09-16T09:38:29.205 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8747e-05 2024-09-16T09:38:29.205 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:29.205 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:29.205 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:38:29.206 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:38:28.919480+0000", 2024-09-16T09:38:29.206 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.311e-05 2024-09-16T09:38:29.206 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:29.206 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:29.206 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:38:29.206 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:38:28.919488+0000", 2024-09-16T09:38:29.206 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.4109999999999999e-06 2024-09-16T09:38:29.206 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:29.206 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:29.206 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:38:29.206 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:38:28.919822+0000", 2024-09-16T09:38:29.206 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000333942 2024-09-16T09:38:29.206 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:29.206 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:29.207 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:38:29.207 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:38:28.919840+0000", 2024-09-16T09:38:29.207 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7869000000000001e-05 2024-09-16T09:38:29.207 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:38:29.207 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:38:29.207 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:38:29.207 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:38:29.207 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-16T09:38:29.207 INFO:teuthology.orchestra.run.smithi070.stdout: "num_ops": 3 2024-09-16T09:38:29.207 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T09:38:29.210 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:29.210 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:29.211 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:38:29.311 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:38:29.352 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:29.352 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:29.352 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:38:29.381 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T09:38:29.454 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:38:29.468 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:29 smithi070 ceph-mon[30388]: pgmap v709: 168 pgs: 168 active+clean; 3.5 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 511 B/s rd, 59 MiB/s wr, 937 op/s 2024-09-16T09:38:29.468 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:29 smithi070 ceph-mon[30388]: 6.2 scrub starts 2024-09-16T09:38:29.468 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:29 smithi070 ceph-mon[40691]: pgmap v709: 168 pgs: 168 active+clean; 3.5 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 511 B/s rd, 59 MiB/s wr, 937 op/s 2024-09-16T09:38:29.480 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:29.481 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:29.482 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:38:29.544 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 33 16 65939 65923 124.819 80.875 0.00633205 0.00800639 2024-09-16T09:38:29.583 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:38:29.618 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:29.618 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:29.619 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:38:29.720 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:38:29.753 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:29.754 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:29.754 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:38:29.855 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:38:29.891 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:29.891 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:29.892 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:38:29.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:29 smithi084 ceph-mon[32629]: pgmap v709: 168 pgs: 168 active+clean; 3.5 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 511 B/s rd, 59 MiB/s wr, 937 op/s 2024-09-16T09:38:29.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:29 smithi084 ceph-mon[32629]: 6.2 scrub starts 2024-09-16T09:38:29.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:29 smithi084 ceph-mon[32629]: 6.2 scrub ok 2024-09-16T09:38:29.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:29 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/768925626' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:38:29.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:29 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:38:29.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:29 smithi070 ceph-mon[30388]: 6.2 scrub ok 2024-09-16T09:38:29.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:29 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/768925626' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:38:29.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:29 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:38:29.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:29 smithi070 ceph-mon[40691]: 6.2 scrub starts 2024-09-16T09:38:29.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:29 smithi070 ceph-mon[40691]: 6.2 scrub ok 2024-09-16T09:38:29.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:29 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/768925626' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:38:29.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:29 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:38:29.993 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:38:30.030 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:30.031 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:30.031 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:38:30.132 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:38:30.169 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:30.169 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:30.170 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:38:30.270 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:38:30.306 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:30.306 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:30.307 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:38:30.363 INFO:teuthology.orchestra.run.smithi070.stderr:nodeep-scrub is unset 2024-09-16T09:38:30.407 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:38:30.441 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:30.441 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:30.442 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:38:30.526 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.3 dump_blocked_ops 2024-09-16T09:38:30.541 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 34 16 67451 67435 123.928 94.5 0.0130084 0.00806479 2024-09-16T09:38:30.543 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:38:30.576 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:30.576 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:30.577 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:38:30.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:30 smithi070 ceph-mon[30388]: 4.12 scrub starts 2024-09-16T09:38:30.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:30 smithi070 ceph-mon[30388]: 4.12 scrub ok 2024-09-16T09:38:30.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:30 smithi070 ceph-mon[30388]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-16T09:38:30.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:30 smithi070 ceph-mon[30388]: Cluster is now healthy 2024-09-16T09:38:30.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:30 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-16T09:38:30.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:30 smithi070 ceph-mon[30388]: osdmap e392: 8 total, 8 up, 8 in 2024-09-16T09:38:30.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:30 smithi070 ceph-mon[40691]: 4.12 scrub starts 2024-09-16T09:38:30.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:30 smithi070 ceph-mon[40691]: 4.12 scrub ok 2024-09-16T09:38:30.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:30 smithi070 ceph-mon[40691]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-16T09:38:30.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:30 smithi070 ceph-mon[40691]: Cluster is now healthy 2024-09-16T09:38:30.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:30 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-16T09:38:30.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:30 smithi070 ceph-mon[40691]: osdmap e392: 8 total, 8 up, 8 in 2024-09-16T09:38:30.678 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:38:30.709 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:30.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:30.710 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:38:30.810 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:38:30.844 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:30.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:30.845 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:38:30.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:30 smithi084 ceph-mon[32629]: 4.12 scrub starts 2024-09-16T09:38:30.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:30 smithi084 ceph-mon[32629]: 4.12 scrub ok 2024-09-16T09:38:30.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:30 smithi084 ceph-mon[32629]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-16T09:38:30.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:30 smithi084 ceph-mon[32629]: Cluster is now healthy 2024-09-16T09:38:30.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:30 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-16T09:38:30.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:30 smithi084 ceph-mon[32629]: osdmap e392: 8 total, 8 up, 8 in 2024-09-16T09:38:30.946 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:38:30.983 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:30.983 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:30.983 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:38:31.084 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:38:31.122 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:31.123 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:31.123 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:38:31.224 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:38:31.269 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:31.269 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:31.269 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:38:31.342 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:38:31.370 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:38:31.394 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:38:31.436 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:31.437 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:31.437 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:38:31.481 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:31 smithi070 ceph-mon[30388]: pgmap v710: 168 pgs: 168 active+clean; 3.9 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 980 B/s rd, 100 MiB/s wr, 1.60k op/s 2024-09-16T09:38:31.481 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:31 smithi070 ceph-mon[30388]: 3.3 scrub starts 2024-09-16T09:38:31.481 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:31 smithi070 ceph-mon[30388]: 3.3 scrub ok 2024-09-16T09:38:31.481 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:31 smithi070 ceph-mon[30388]: 6.3 scrub starts 2024-09-16T09:38:31.481 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:31 smithi070 ceph-mon[30388]: 6.3 scrub ok 2024-09-16T09:38:31.482 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:31 smithi070 ceph-mon[40691]: pgmap v710: 168 pgs: 168 active+clean; 3.9 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 980 B/s rd, 100 MiB/s wr, 1.60k op/s 2024-09-16T09:38:31.482 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:31 smithi070 ceph-mon[40691]: 3.3 scrub starts 2024-09-16T09:38:31.482 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:31 smithi070 ceph-mon[40691]: 3.3 scrub ok 2024-09-16T09:38:31.482 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:31 smithi070 ceph-mon[40691]: 6.3 scrub starts 2024-09-16T09:38:31.482 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:31 smithi070 ceph-mon[40691]: 6.3 scrub ok 2024-09-16T09:38:31.539 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:38:31.542 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 35 15 68892 68877 122.962 90.125 0.00952453 0.00812834 2024-09-16T09:38:31.575 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:31.576 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:31.576 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:38:31.658 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:38:31.677 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:38:31.703 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:31.704 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:31.705 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:38:31.805 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:38:31.840 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:31.841 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:31.841 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:38:31.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:31 smithi084 ceph-mon[32629]: pgmap v710: 168 pgs: 168 active+clean; 3.9 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 980 B/s rd, 100 MiB/s wr, 1.60k op/s 2024-09-16T09:38:31.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:31 smithi084 ceph-mon[32629]: 3.3 scrub starts 2024-09-16T09:38:31.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:31 smithi084 ceph-mon[32629]: 3.3 scrub ok 2024-09-16T09:38:31.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:31 smithi084 ceph-mon[32629]: 6.3 scrub starts 2024-09-16T09:38:31.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:31 smithi084 ceph-mon[32629]: 6.3 scrub ok 2024-09-16T09:38:31.943 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:38:31.987 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:31.988 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:31.988 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:38:32.089 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:38:32.122 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:38:31 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:38:31] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:38:32.129 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:32.130 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:32.130 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:38:32.231 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:38:32.271 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:32.271 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:32.272 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:38:32.373 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:38:32.417 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:32.417 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:32.418 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:38:32.502 INFO:teuthology.orchestra.run.smithi070.stderr:set pool 10 pg_num to 26 2024-09-16T09:38:32.519 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:38:32.532 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:38:32.532 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:38:32.542 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 36 16 70226 70210 121.86 83.3125 0.00588678 0.00820072 2024-09-16T09:38:32.548 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:32.549 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:32.549 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:38:32.571 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:38:32.571 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:38:32.600 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:38:32.600 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:38:32.633 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:38:32.633 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:38:32.650 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:38:32.670 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:38:32.670 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:38:32.682 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:32.682 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:32.682 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:38:32.699 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:38:32.699 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:38:32.700 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:32 smithi070 ceph-mon[30388]: 6.17 scrub starts 2024-09-16T09:38:32.700 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:32 smithi070 ceph-mon[30388]: 6.17 scrub ok 2024-09-16T09:38:32.700 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:32 smithi070 ceph-mon[30388]: 5.17 scrub starts 2024-09-16T09:38:32.700 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:32 smithi070 ceph-mon[30388]: 5.17 scrub ok 2024-09-16T09:38:32.700 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:32 smithi070 ceph-mon[30388]: 4.2 scrub starts 2024-09-16T09:38:32.700 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:32 smithi070 ceph-mon[30388]: 4.2 scrub ok 2024-09-16T09:38:32.700 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:32 smithi070 ceph-mon[40691]: 6.17 scrub starts 2024-09-16T09:38:32.700 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:32 smithi070 ceph-mon[40691]: 6.17 scrub ok 2024-09-16T09:38:32.701 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:32 smithi070 ceph-mon[40691]: 5.17 scrub starts 2024-09-16T09:38:32.701 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:32 smithi070 ceph-mon[40691]: 5.17 scrub ok 2024-09-16T09:38:32.701 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:32 smithi070 ceph-mon[40691]: 4.2 scrub starts 2024-09-16T09:38:32.701 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:32 smithi070 ceph-mon[40691]: 4.2 scrub ok 2024-09-16T09:38:32.738 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:38:32.739 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:38:32.775 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:38:32.776 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:38:32.783 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:38:32.828 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:32.829 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:32.829 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:38:32.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:32 smithi084 ceph-mon[32629]: 6.17 scrub starts 2024-09-16T09:38:32.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:32 smithi084 ceph-mon[32629]: 6.17 scrub ok 2024-09-16T09:38:32.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:32 smithi084 ceph-mon[32629]: 5.17 scrub starts 2024-09-16T09:38:32.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:32 smithi084 ceph-mon[32629]: 5.17 scrub ok 2024-09-16T09:38:32.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:32 smithi084 ceph-mon[32629]: 4.2 scrub starts 2024-09-16T09:38:32.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:32 smithi084 ceph-mon[32629]: 4.2 scrub ok 2024-09-16T09:38:32.930 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:38:32.967 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:32.968 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:32.968 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:38:33.069 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:38:33.105 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:33.106 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:33.106 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:38:33.207 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:38:33.241 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T09:38:33.242 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:38:33.251 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:33.251 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:33.251 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:38:33.276 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd set noscrub 2024-09-16T09:38:33.352 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:38:33.388 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:33.389 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:33.389 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:38:33.490 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:38:33.525 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:33.526 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:33.526 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:38:33.542 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 37 16 71576 71560 120.847 84.375 0.0160491 0.00826863 2024-09-16T09:38:33.627 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:38:33.659 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:33.660 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:38:33 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:38:33.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:33 smithi084 ceph-mon[32629]: pgmap v712: 168 pgs: 168 active+clean; 3.9 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 639 B/s rd, 70 MiB/s wr, 1.12k op/s 2024-09-16T09:38:33.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:33 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]} : dispatch 2024-09-16T09:38:33.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:33 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]} : dispatch 2024-09-16T09:38:33.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:33 smithi084 ceph-mon[32629]: 3.6 scrub starts 2024-09-16T09:38:33.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:33 smithi084 ceph-mon[32629]: 3.6 scrub ok 2024-09-16T09:38:33.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:33 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/653162483' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:38:33.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:33 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/893117179' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_3", "var": "pg_num", "val": "26"} : dispatch 2024-09-16T09:38:33.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:33 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]': finished 2024-09-16T09:38:33.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:33 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/893117179' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_3", "var": "pg_num", "val": "26"}]': finished 2024-09-16T09:38:33.662 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:33 smithi084 ceph-mon[32629]: osdmap e393: 8 total, 8 up, 8 in 2024-09-16T09:38:33.662 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:33 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:38:33.662 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:33 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:38:33.662 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:33.662 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:38:33.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:33 smithi070 ceph-mon[30388]: pgmap v712: 168 pgs: 168 active+clean; 3.9 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 639 B/s rd, 70 MiB/s wr, 1.12k op/s 2024-09-16T09:38:33.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:33 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]} : dispatch 2024-09-16T09:38:33.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:33 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]} : dispatch 2024-09-16T09:38:33.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:33 smithi070 ceph-mon[30388]: 3.6 scrub starts 2024-09-16T09:38:33.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:33 smithi070 ceph-mon[30388]: 3.6 scrub ok 2024-09-16T09:38:33.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:33 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/653162483' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:38:33.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:33 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/893117179' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_3", "var": "pg_num", "val": "26"} : dispatch 2024-09-16T09:38:33.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:33 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]': finished 2024-09-16T09:38:33.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:33 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/893117179' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_3", "var": "pg_num", "val": "26"}]': finished 2024-09-16T09:38:33.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:33 smithi070 ceph-mon[30388]: osdmap e393: 8 total, 8 up, 8 in 2024-09-16T09:38:33.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:33 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:38:33.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:33 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:38:33.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:33 smithi070 ceph-mon[40691]: pgmap v712: 168 pgs: 168 active+clean; 3.9 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 639 B/s rd, 70 MiB/s wr, 1.12k op/s 2024-09-16T09:38:33.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:33 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]} : dispatch 2024-09-16T09:38:33.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:33 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]} : dispatch 2024-09-16T09:38:33.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:33 smithi070 ceph-mon[40691]: 3.6 scrub starts 2024-09-16T09:38:33.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:33 smithi070 ceph-mon[40691]: 3.6 scrub ok 2024-09-16T09:38:33.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:33 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/653162483' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:38:33.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:33 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/893117179' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_3", "var": "pg_num", "val": "26"} : dispatch 2024-09-16T09:38:33.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:33 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]': finished 2024-09-16T09:38:33.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:33 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/893117179' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_3", "var": "pg_num", "val": "26"}]': finished 2024-09-16T09:38:33.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:33 smithi070 ceph-mon[40691]: osdmap e393: 8 total, 8 up, 8 in 2024-09-16T09:38:33.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:33 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:38:33.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:33 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:38:33.763 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:38:33.802 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:33.803 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:33.803 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:38:33.904 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:38:33.945 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:33.945 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:33.946 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:38:34.047 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:38:34.081 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:34.082 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:34.082 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:38:34.183 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:38:34.216 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:34.217 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:34.217 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:38:34.318 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:38:34.355 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:34.356 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:34.357 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:38:34.457 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:38:34.526 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:34.527 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:34.528 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:38:34.542 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 38 15 72745 72730 119.591 73.125 0.00504194 0.00833526 2024-09-16T09:38:34.628 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:38:34.649 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:34.650 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:34.650 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:38:34.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:34 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:38:34.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:34 smithi084 ceph-mon[32629]: 4.6 scrub starts 2024-09-16T09:38:34.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:34 smithi084 ceph-mon[32629]: 4.6 scrub ok 2024-09-16T09:38:34.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:34 smithi084 ceph-mon[32629]: osdmap e394: 8 total, 8 up, 8 in 2024-09-16T09:38:34.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:34 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_3", "var": "pg_num_actual", "val": "26"} : dispatch 2024-09-16T09:38:34.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:34 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_3", "var": "pg_num_actual", "val": "26"} : dispatch 2024-09-16T09:38:34.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:34 smithi084 ceph-mon[32629]: 4.10 scrub starts 2024-09-16T09:38:34.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:34 smithi084 ceph-mon[32629]: 4.10 scrub ok 2024-09-16T09:38:34.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:34 smithi084 ceph-mon[32629]: 6.19 scrub starts 2024-09-16T09:38:34.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:34 smithi084 ceph-mon[32629]: 6.19 scrub ok 2024-09-16T09:38:34.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:34 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_3", "var": "pg_num_actual", "val": "26"}]': finished 2024-09-16T09:38:34.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:34 smithi084 ceph-mon[32629]: osdmap e395: 8 total, 8 up, 8 in 2024-09-16T09:38:34.751 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:38:34.773 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:34.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:34.774 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:38:34.875 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:38:34.895 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:34.896 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:34.897 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:38:34.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:34 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:38:34.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:34 smithi070 ceph-mon[40691]: 4.6 scrub starts 2024-09-16T09:38:34.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:34 smithi070 ceph-mon[40691]: 4.6 scrub ok 2024-09-16T09:38:34.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:34 smithi070 ceph-mon[40691]: osdmap e394: 8 total, 8 up, 8 in 2024-09-16T09:38:34.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:34 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_3", "var": "pg_num_actual", "val": "26"} : dispatch 2024-09-16T09:38:34.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:34 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_3", "var": "pg_num_actual", "val": "26"} : dispatch 2024-09-16T09:38:34.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:34 smithi070 ceph-mon[40691]: 4.10 scrub starts 2024-09-16T09:38:34.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:34 smithi070 ceph-mon[40691]: 4.10 scrub ok 2024-09-16T09:38:34.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:34 smithi070 ceph-mon[40691]: 6.19 scrub starts 2024-09-16T09:38:34.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:34 smithi070 ceph-mon[40691]: 6.19 scrub ok 2024-09-16T09:38:34.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:34 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_3", "var": "pg_num_actual", "val": "26"}]': finished 2024-09-16T09:38:34.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:34 smithi070 ceph-mon[40691]: osdmap e395: 8 total, 8 up, 8 in 2024-09-16T09:38:34.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:34 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:38:34.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:34 smithi070 ceph-mon[30388]: 4.6 scrub starts 2024-09-16T09:38:34.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:34 smithi070 ceph-mon[30388]: 4.6 scrub ok 2024-09-16T09:38:34.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:34 smithi070 ceph-mon[30388]: osdmap e394: 8 total, 8 up, 8 in 2024-09-16T09:38:34.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:34 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_3", "var": "pg_num_actual", "val": "26"} : dispatch 2024-09-16T09:38:34.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:34 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_3", "var": "pg_num_actual", "val": "26"} : dispatch 2024-09-16T09:38:34.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:34 smithi070 ceph-mon[30388]: 4.10 scrub starts 2024-09-16T09:38:34.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:34 smithi070 ceph-mon[30388]: 4.10 scrub ok 2024-09-16T09:38:34.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:34 smithi070 ceph-mon[30388]: 6.19 scrub starts 2024-09-16T09:38:34.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:34 smithi070 ceph-mon[30388]: 6.19 scrub ok 2024-09-16T09:38:34.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:34 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_3", "var": "pg_num_actual", "val": "26"}]': finished 2024-09-16T09:38:34.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:34 smithi070 ceph-mon[30388]: osdmap e395: 8 total, 8 up, 8 in 2024-09-16T09:38:34.997 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:38:35.019 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:35.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:35.020 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:38:35.037 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.3 dump_historic_ops 2024-09-16T09:38:35.121 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:38:35.142 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:35.143 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:35.143 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:38:35.244 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:38:35.265 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:35.266 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:35.266 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:38:35.367 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:38:35.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:35.399 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:35.399 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:38:35.500 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:38:35.524 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:35.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:35.525 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:38:35.542 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 39 16 72868 72852 116.721 7.625 0.0030533 0.00855245 2024-09-16T09:38:35.627 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:38:35.658 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:35.659 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:35.659 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:38:35.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:35 smithi084 ceph-mon[32629]: pgmap v715: 168 pgs: 1 peering, 167 active+clean; 4.3 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.7 KiB/s rd, 136 MiB/s wr, 2.18k op/s 2024-09-16T09:38:35.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:35 smithi084 ceph-mon[32629]: 1.b scrub starts 2024-09-16T09:38:35.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:35 smithi084 ceph-mon[32629]: 1.b scrub ok 2024-09-16T09:38:35.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:35 smithi084 ceph-mon[32629]: 3.8 scrub starts 2024-09-16T09:38:35.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:35 smithi084 ceph-mon[32629]: 3.8 scrub ok 2024-09-16T09:38:35.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:35 smithi084 ceph-mon[32629]: 6.d scrub starts 2024-09-16T09:38:35.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:35 smithi084 ceph-mon[32629]: 6.d scrub ok 2024-09-16T09:38:35.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:35 smithi084 ceph-mon[32629]: 5.11 scrub starts 2024-09-16T09:38:35.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:35 smithi084 ceph-mon[32629]: 5.11 scrub ok 2024-09-16T09:38:35.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:35 smithi084 ceph-mon[32629]: osdmap e396: 8 total, 8 up, 8 in 2024-09-16T09:38:35.760 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:38:35.795 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:35.796 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:35.796 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:38:35.897 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:38:35.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:35 smithi070 ceph-mon[30388]: pgmap v715: 168 pgs: 1 peering, 167 active+clean; 4.3 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.7 KiB/s rd, 136 MiB/s wr, 2.18k op/s 2024-09-16T09:38:35.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:35 smithi070 ceph-mon[30388]: 1.b scrub starts 2024-09-16T09:38:35.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:35 smithi070 ceph-mon[30388]: 1.b scrub ok 2024-09-16T09:38:35.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:35 smithi070 ceph-mon[30388]: 3.8 scrub starts 2024-09-16T09:38:35.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:35 smithi070 ceph-mon[30388]: 3.8 scrub ok 2024-09-16T09:38:35.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:35 smithi070 ceph-mon[30388]: 6.d scrub starts 2024-09-16T09:38:35.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:35 smithi070 ceph-mon[30388]: 6.d scrub ok 2024-09-16T09:38:35.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:35 smithi070 ceph-mon[30388]: 5.11 scrub starts 2024-09-16T09:38:35.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:35 smithi070 ceph-mon[30388]: 5.11 scrub ok 2024-09-16T09:38:35.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:35 smithi070 ceph-mon[30388]: osdmap e396: 8 total, 8 up, 8 in 2024-09-16T09:38:35.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:35 smithi070 ceph-mon[40691]: pgmap v715: 168 pgs: 1 peering, 167 active+clean; 4.3 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.7 KiB/s rd, 136 MiB/s wr, 2.18k op/s 2024-09-16T09:38:35.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:35 smithi070 ceph-mon[40691]: 1.b scrub starts 2024-09-16T09:38:35.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:35 smithi070 ceph-mon[40691]: 1.b scrub ok 2024-09-16T09:38:35.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:35 smithi070 ceph-mon[40691]: 3.8 scrub starts 2024-09-16T09:38:35.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:35 smithi070 ceph-mon[40691]: 3.8 scrub ok 2024-09-16T09:38:35.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:35 smithi070 ceph-mon[40691]: 6.d scrub starts 2024-09-16T09:38:35.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:35 smithi070 ceph-mon[40691]: 6.d scrub ok 2024-09-16T09:38:35.923 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:35 smithi070 ceph-mon[40691]: 5.11 scrub starts 2024-09-16T09:38:35.923 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:35 smithi070 ceph-mon[40691]: 5.11 scrub ok 2024-09-16T09:38:35.923 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:35 smithi070 ceph-mon[40691]: osdmap e396: 8 total, 8 up, 8 in 2024-09-16T09:38:35.926 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:35.927 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:35.927 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:38:36.028 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:38:36.065 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:36.065 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:36.066 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:38:36.167 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:38:36.199 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:36.200 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:36.200 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:38:36.301 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:38:36.341 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:36.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:36.342 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:38:36.443 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:38:36.485 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:36.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:36.486 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:38:36.542 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:2024-09-16T09:38:36.541830+0000 min lat: 0.00121703 max lat: 1.12299 avg lat: 0.00861966 2024-09-16T09:38:36.542 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-16T09:38:36.542 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 40 16 74235 74219 115.939 85.4375 0.0310302 0.00861966 2024-09-16T09:38:36.588 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:38:36.619 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:36.619 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:36.620 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:38:36.720 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:38:36.741 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:36.741 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:36.742 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:38:36.842 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:38:36.863 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:36.864 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:36.864 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:38:36.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:36 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:38:36.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:36 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:38:36.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:36 smithi084 ceph-mon[32629]: 10.8 scrub starts 2024-09-16T09:38:36.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:36 smithi084 ceph-mon[32629]: 2.0 scrub starts 2024-09-16T09:38:36.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:36 smithi084 ceph-mon[32629]: 2.0 scrub ok 2024-09-16T09:38:36.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:36 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_3", "var": "pgp_num_actual", "val": "26"} : dispatch 2024-09-16T09:38:36.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:36 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_3", "var": "pgp_num_actual", "val": "26"} : dispatch 2024-09-16T09:38:36.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:36 smithi084 ceph-mon[32629]: 10.8 scrub ok 2024-09-16T09:38:36.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:36 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:38:36.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:36 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:38:36.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:36 smithi084 ceph-mon[32629]: 10.3 scrub starts 2024-09-16T09:38:36.912 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:36 smithi084 ceph-mon[32629]: 10.7 scrub starts 2024-09-16T09:38:36.924 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:36 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:38:36.924 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:36 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:38:36.924 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:36 smithi070 ceph-mon[30388]: 10.8 scrub starts 2024-09-16T09:38:36.924 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:36 smithi070 ceph-mon[30388]: 2.0 scrub starts 2024-09-16T09:38:36.924 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:36 smithi070 ceph-mon[30388]: 2.0 scrub ok 2024-09-16T09:38:36.924 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:36 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_3", "var": "pgp_num_actual", "val": "26"} : dispatch 2024-09-16T09:38:36.925 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:36 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_3", "var": "pgp_num_actual", "val": "26"} : dispatch 2024-09-16T09:38:36.925 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:36 smithi070 ceph-mon[30388]: 10.8 scrub ok 2024-09-16T09:38:36.925 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:36 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:38:36.925 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:36 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:38:36.925 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:36 smithi070 ceph-mon[30388]: 10.3 scrub starts 2024-09-16T09:38:36.925 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:36 smithi070 ceph-mon[30388]: 10.7 scrub starts 2024-09-16T09:38:36.925 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:36 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:38:36.925 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:36 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:38:36.925 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:36 smithi070 ceph-mon[40691]: 10.8 scrub starts 2024-09-16T09:38:36.925 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:36 smithi070 ceph-mon[40691]: 2.0 scrub starts 2024-09-16T09:38:36.926 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:36 smithi070 ceph-mon[40691]: 2.0 scrub ok 2024-09-16T09:38:36.926 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:36 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_3", "var": "pgp_num_actual", "val": "26"} : dispatch 2024-09-16T09:38:36.926 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:36 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "unique_pool_3", "var": "pgp_num_actual", "val": "26"} : dispatch 2024-09-16T09:38:36.926 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:36 smithi070 ceph-mon[40691]: 10.8 scrub ok 2024-09-16T09:38:36.926 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:36 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:38:36.926 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:36 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:38:36.926 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:36 smithi070 ceph-mon[40691]: 10.3 scrub starts 2024-09-16T09:38:36.926 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:36 smithi070 ceph-mon[40691]: 10.7 scrub starts 2024-09-16T09:38:36.966 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:38:36.987 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:36.987 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:36.988 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:38:37.089 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:38:37.111 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:37.112 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:37.112 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:38:37.213 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:38:37.242 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:37.243 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:37.243 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:38:37.344 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:38:37.366 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:37.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:37.367 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:38:37.468 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:38:37.490 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:37.490 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:37.490 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:38:37.542 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 41 16 74267 74251 113.16 2 0.00385418 0.00862654 2024-09-16T09:38:37.591 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:38:37.613 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:37.614 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:37.614 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:38:37.715 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:38:37.736 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:37.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:37.737 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:38:37.774 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:38:37.839 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:38:37.860 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:37.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:37.861 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:38:37.897 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:37 smithi070 ceph-mon[30388]: pgmap v718: 178 pgs: 10 unknown, 1 peering, 167 active+clean; 4.3 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 109 MiB/s wr, 1.74k op/s 2024-09-16T09:38:37.897 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:37 smithi070 ceph-mon[30388]: 10.2 scrub starts 2024-09-16T09:38:37.898 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:37 smithi070 ceph-mon[30388]: 10.18 scrub starts 2024-09-16T09:38:37.898 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:37 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_3", "var": "pgp_num_actual", "val": "26"}]': finished 2024-09-16T09:38:37.898 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:37 smithi070 ceph-mon[30388]: osdmap e397: 8 total, 8 up, 8 in 2024-09-16T09:38:37.898 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:37 smithi070 ceph-mon[30388]: 10.3 scrub ok 2024-09-16T09:38:37.898 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:37 smithi070 ceph-mon[30388]: 10.7 scrub ok 2024-09-16T09:38:37.898 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:37 smithi070 ceph-mon[30388]: 10.2 scrub ok 2024-09-16T09:38:37.898 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:37 smithi070 ceph-mon[30388]: 10.0 scrub starts 2024-09-16T09:38:37.898 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:37 smithi070 ceph-mon[30388]: 10.9 scrub starts 2024-09-16T09:38:37.898 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:37 smithi070 ceph-mon[30388]: osdmap e398: 8 total, 8 up, 8 in 2024-09-16T09:38:37.898 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:37 smithi070 ceph-mon[40691]: pgmap v718: 178 pgs: 10 unknown, 1 peering, 167 active+clean; 4.3 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 109 MiB/s wr, 1.74k op/s 2024-09-16T09:38:37.898 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:37 smithi070 ceph-mon[40691]: 10.2 scrub starts 2024-09-16T09:38:37.899 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:37 smithi070 ceph-mon[40691]: 10.18 scrub starts 2024-09-16T09:38:37.899 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:37 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_3", "var": "pgp_num_actual", "val": "26"}]': finished 2024-09-16T09:38:37.899 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:37 smithi070 ceph-mon[40691]: osdmap e397: 8 total, 8 up, 8 in 2024-09-16T09:38:37.899 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:37 smithi070 ceph-mon[40691]: 10.3 scrub ok 2024-09-16T09:38:37.899 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:37 smithi070 ceph-mon[40691]: 10.7 scrub ok 2024-09-16T09:38:37.899 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:37 smithi070 ceph-mon[40691]: 10.2 scrub ok 2024-09-16T09:38:37.899 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:37 smithi070 ceph-mon[40691]: 10.0 scrub starts 2024-09-16T09:38:37.899 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:37 smithi070 ceph-mon[40691]: 10.9 scrub starts 2024-09-16T09:38:37.899 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:37 smithi070 ceph-mon[40691]: osdmap e398: 8 total, 8 up, 8 in 2024-09-16T09:38:37.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:37 smithi084 ceph-mon[32629]: pgmap v718: 178 pgs: 10 unknown, 1 peering, 167 active+clean; 4.3 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 109 MiB/s wr, 1.74k op/s 2024-09-16T09:38:37.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:37 smithi084 ceph-mon[32629]: 10.2 scrub starts 2024-09-16T09:38:37.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:37 smithi084 ceph-mon[32629]: 10.18 scrub starts 2024-09-16T09:38:37.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:37 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_3", "var": "pgp_num_actual", "val": "26"}]': finished 2024-09-16T09:38:37.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:37 smithi084 ceph-mon[32629]: osdmap e397: 8 total, 8 up, 8 in 2024-09-16T09:38:37.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:37 smithi084 ceph-mon[32629]: 10.3 scrub ok 2024-09-16T09:38:37.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:37 smithi084 ceph-mon[32629]: 10.7 scrub ok 2024-09-16T09:38:37.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:37 smithi084 ceph-mon[32629]: 10.2 scrub ok 2024-09-16T09:38:37.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:37 smithi084 ceph-mon[32629]: 10.0 scrub starts 2024-09-16T09:38:37.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:37 smithi084 ceph-mon[32629]: 10.9 scrub starts 2024-09-16T09:38:37.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:37 smithi084 ceph-mon[32629]: osdmap e398: 8 total, 8 up, 8 in 2024-09-16T09:38:37.962 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:38:37.985 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:37.985 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:37.986 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:38:38.087 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:38:38.108 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:38.108 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:38.109 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:38:38.210 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:38:38.231 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:38.232 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:38.232 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:38:38.333 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:38:38.355 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:38.356 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:38.356 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:38:38.384 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:38:38.457 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:38:38.479 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:38.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:38.480 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:38:38.552 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 42 16 74267 74251 110.442 0 - 0.00862654 2024-09-16T09:38:38.581 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:38:38.631 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:38.631 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:38.631 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:38:38.732 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:38:38.760 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T09:38:38.761 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T09:38:38.761 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-16T09:38:38.761 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd dump -f json-pretty 2024-09-16T09:38:38.786 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:38.787 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:38.787 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:38:38.813 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:38 smithi070 ceph-mon[30388]: 10.1 scrub starts 2024-09-16T09:38:38.814 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:38 smithi070 ceph-mon[30388]: 10.0 scrub ok 2024-09-16T09:38:38.814 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:38 smithi070 ceph-mon[30388]: 10.9 scrub ok 2024-09-16T09:38:38.814 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:38 smithi070 ceph-mon[30388]: 10.1 scrub ok 2024-09-16T09:38:38.814 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:38 smithi070 ceph-mon[30388]: 10.4 scrub starts 2024-09-16T09:38:38.814 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:38 smithi070 ceph-mon[30388]: pgmap v721: 178 pgs: 10 unknown, 1 peering, 167 active+clean; 4.3 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 22 MiB/s wr, 345 op/s 2024-09-16T09:38:38.814 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:38 smithi070 ceph-mon[30388]: 10.4 scrub ok 2024-09-16T09:38:38.814 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:38 smithi070 ceph-mon[30388]: 10.6 scrub starts 2024-09-16T09:38:38.814 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:38 smithi070 ceph-mon[30388]: osdmap e399: 8 total, 8 up, 8 in 2024-09-16T09:38:38.815 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:38 smithi070 ceph-mon[40691]: 10.1 scrub starts 2024-09-16T09:38:38.815 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:38 smithi070 ceph-mon[40691]: 10.0 scrub ok 2024-09-16T09:38:38.815 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:38 smithi070 ceph-mon[40691]: 10.9 scrub ok 2024-09-16T09:38:38.815 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:38 smithi070 ceph-mon[40691]: 10.1 scrub ok 2024-09-16T09:38:38.815 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:38 smithi070 ceph-mon[40691]: 10.4 scrub starts 2024-09-16T09:38:38.815 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:38 smithi070 ceph-mon[40691]: pgmap v721: 178 pgs: 10 unknown, 1 peering, 167 active+clean; 4.3 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 22 MiB/s wr, 345 op/s 2024-09-16T09:38:38.815 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:38 smithi070 ceph-mon[40691]: 10.4 scrub ok 2024-09-16T09:38:38.815 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:38 smithi070 ceph-mon[40691]: 10.6 scrub starts 2024-09-16T09:38:38.815 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:38 smithi070 ceph-mon[40691]: osdmap e399: 8 total, 8 up, 8 in 2024-09-16T09:38:38.888 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:38:38.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:38 smithi084 ceph-mon[32629]: 10.1 scrub starts 2024-09-16T09:38:38.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:38 smithi084 ceph-mon[32629]: 10.0 scrub ok 2024-09-16T09:38:38.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:38 smithi084 ceph-mon[32629]: 10.9 scrub ok 2024-09-16T09:38:38.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:38 smithi084 ceph-mon[32629]: 10.1 scrub ok 2024-09-16T09:38:38.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:38 smithi084 ceph-mon[32629]: 10.4 scrub starts 2024-09-16T09:38:38.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:38 smithi084 ceph-mon[32629]: pgmap v721: 178 pgs: 10 unknown, 1 peering, 167 active+clean; 4.3 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 22 MiB/s wr, 345 op/s 2024-09-16T09:38:38.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:38 smithi084 ceph-mon[32629]: 10.4 scrub ok 2024-09-16T09:38:38.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:38 smithi084 ceph-mon[32629]: 10.6 scrub starts 2024-09-16T09:38:38.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:38 smithi084 ceph-mon[32629]: osdmap e399: 8 total, 8 up, 8 in 2024-09-16T09:38:38.915 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:38.916 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:38.916 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:38:39.016 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:38:39.067 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T09:38:39.067 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:38:39.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:39.068 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:39.068 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:38:39.169 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:38:39.191 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:39.193 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:39.193 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:38:39.294 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:38:39.318 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:39.319 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:39.319 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:38:39.420 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:38:39.446 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:39.446 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:39.447 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:38:39.548 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:38:39.552 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 43 16 74371 74355 108.025 3.25 0.0164965 0.00898684 2024-09-16T09:38:39.560 INFO:teuthology.orchestra.run.smithi070.stderr:noscrub is set 2024-09-16T09:38:39.574 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:39.575 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:39.576 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:38:39.676 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:38:39.699 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:39.700 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:39.700 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:38:39.720 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T09:38:39.749 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:39 smithi070 ceph-mon[30388]: 10.6 scrub ok 2024-09-16T09:38:39.749 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:39 smithi070 ceph-mon[30388]: 10.11 starting backfill to osd.5 from (0'0,0'0] MAX to 396'4611 2024-09-16T09:38:39.749 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:39 smithi070 ceph-mon[30388]: 10.12 starting backfill to osd.2 from (0'0,0'0] MAX to 396'4617 2024-09-16T09:38:39.749 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:39 smithi070 ceph-mon[30388]: 10.12 starting backfill to osd.5 from (0'0,0'0] MAX to 396'4617 2024-09-16T09:38:39.749 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:39 smithi070 ceph-mon[30388]: 10.5 scrub starts 2024-09-16T09:38:39.749 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:39 smithi070 ceph-mon[30388]: 10.12 starting backfill to osd.6 from (0'0,0'0] MAX to 396'4617 2024-09-16T09:38:39.749 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:39 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2271948407' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:38:39.749 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:39 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:38:39.749 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:39 smithi070 ceph-mon[30388]: 5.10 scrub starts 2024-09-16T09:38:39.749 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:39 smithi070 ceph-mon[30388]: 5.10 scrub ok 2024-09-16T09:38:39.750 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:39 smithi070 ceph-mon[40691]: 10.6 scrub ok 2024-09-16T09:38:39.750 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:39 smithi070 ceph-mon[40691]: 10.11 starting backfill to osd.5 from (0'0,0'0] MAX to 396'4611 2024-09-16T09:38:39.750 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:39 smithi070 ceph-mon[40691]: 10.12 starting backfill to osd.2 from (0'0,0'0] MAX to 396'4617 2024-09-16T09:38:39.750 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:39 smithi070 ceph-mon[40691]: 10.12 starting backfill to osd.5 from (0'0,0'0] MAX to 396'4617 2024-09-16T09:38:39.750 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:39 smithi070 ceph-mon[40691]: 10.5 scrub starts 2024-09-16T09:38:39.750 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:39 smithi070 ceph-mon[40691]: 10.12 starting backfill to osd.6 from (0'0,0'0] MAX to 396'4617 2024-09-16T09:38:39.750 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:39 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/2271948407' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:38:39.750 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:39 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:38:39.750 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:39 smithi070 ceph-mon[40691]: 5.10 scrub starts 2024-09-16T09:38:39.750 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:39 smithi070 ceph-mon[40691]: 5.10 scrub ok 2024-09-16T09:38:39.802 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:38:39.832 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:39.832 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:39.833 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:38:39.884 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.4 dump_ops_in_flight 2024-09-16T09:38:39.907 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:39 smithi084 ceph-mon[32629]: 10.6 scrub ok 2024-09-16T09:38:39.907 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:39 smithi084 ceph-mon[32629]: 10.11 starting backfill to osd.5 from (0'0,0'0] MAX to 396'4611 2024-09-16T09:38:39.907 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:39 smithi084 ceph-mon[32629]: 10.12 starting backfill to osd.2 from (0'0,0'0] MAX to 396'4617 2024-09-16T09:38:39.907 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:39 smithi084 ceph-mon[32629]: 10.12 starting backfill to osd.5 from (0'0,0'0] MAX to 396'4617 2024-09-16T09:38:39.907 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:39 smithi084 ceph-mon[32629]: 10.5 scrub starts 2024-09-16T09:38:39.907 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:39 smithi084 ceph-mon[32629]: 10.12 starting backfill to osd.6 from (0'0,0'0] MAX to 396'4617 2024-09-16T09:38:39.908 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:39 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/2271948407' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:38:39.908 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:39 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:38:39.908 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:39 smithi084 ceph-mon[32629]: 5.10 scrub starts 2024-09-16T09:38:39.908 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:39 smithi084 ceph-mon[32629]: 5.10 scrub ok 2024-09-16T09:38:39.933 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:38:39.958 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:39.958 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:39.959 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:38:40.059 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:38:40.085 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:40.087 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:40.087 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:38:40.188 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:38:40.190 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:38:40.215 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:40.216 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:40.216 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:38:40.317 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:38:40.341 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:40.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:40.342 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:38:40.443 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:38:40.469 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:40.470 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:40.470 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:38:40.552 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 44 16 74371 74355 105.571 0 - 0.00898684 2024-09-16T09:38:40.571 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:38:40.594 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:40.595 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:40.596 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:38:40.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:40 smithi084 ceph-mon[32629]: 10.5 scrub ok 2024-09-16T09:38:40.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:40 smithi084 ceph-mon[32629]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:38:40.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:40 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-16T09:38:40.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:40 smithi084 ceph-mon[32629]: osdmap e400: 8 total, 8 up, 8 in 2024-09-16T09:38:40.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:40 smithi084 ceph-mon[32629]: pgmap v724: 178 pgs: 1 active+recovering+undersized+remapped, 2 active+clean+scrubbing, 7 remapped+peering, 2 active+remapped+backfill_toofull, 166 active+clean; 4.5 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 13871/223446 objects misplaced (6.208%); 400 KiB/s, 6 objects/s recovering 2024-09-16T09:38:40.697 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:38:40.723 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:40.724 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:40.724 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:38:40.801 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:38:40.825 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:38:40.846 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:38:40.848 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:40.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:40.849 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:38:40.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:40 smithi070 ceph-mon[30388]: 10.5 scrub ok 2024-09-16T09:38:40.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:40 smithi070 ceph-mon[30388]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:38:40.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:40 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-16T09:38:40.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:40 smithi070 ceph-mon[30388]: osdmap e400: 8 total, 8 up, 8 in 2024-09-16T09:38:40.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:40 smithi070 ceph-mon[30388]: pgmap v724: 178 pgs: 1 active+recovering+undersized+remapped, 2 active+clean+scrubbing, 7 remapped+peering, 2 active+remapped+backfill_toofull, 166 active+clean; 4.5 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 13871/223446 objects misplaced (6.208%); 400 KiB/s, 6 objects/s recovering 2024-09-16T09:38:40.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:40 smithi070 ceph-mon[40691]: 10.5 scrub ok 2024-09-16T09:38:40.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:40 smithi070 ceph-mon[40691]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:38:40.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:40 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-16T09:38:40.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:40 smithi070 ceph-mon[40691]: osdmap e400: 8 total, 8 up, 8 in 2024-09-16T09:38:40.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:40 smithi070 ceph-mon[40691]: pgmap v724: 178 pgs: 1 active+recovering+undersized+remapped, 2 active+clean+scrubbing, 7 remapped+peering, 2 active+remapped+backfill_toofull, 166 active+clean; 4.5 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 13871/223446 objects misplaced (6.208%); 400 KiB/s, 6 objects/s recovering 2024-09-16T09:38:40.949 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:38:40.972 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:40.973 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:40.974 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:38:41.074 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:38:41.104 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:41.105 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:41.105 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:38:41.206 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:38:41.234 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:41.235 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:41.235 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:38:41.336 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:38:41.361 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:41.362 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:41.362 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:38:41.463 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:38:41.489 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:41.490 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:41.490 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:38:41.552 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 45 16 74371 74355 103.226 0 - 0.00898684 2024-09-16T09:38:41.591 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:38:41.615 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:41.616 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:41.616 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:38:41.653 INFO:teuthology.orchestra.run.smithi084.stdout:ERROR: (22) Invalid argument 2024-09-16T09:38:41.653 INFO:teuthology.orchestra.run.smithi084.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:38:41.673 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:41 smithi070 ceph-mon[30388]: Health check failed: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-16T09:38:41.674 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:41 smithi070 ceph-mon[40691]: Health check failed: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-16T09:38:41.717 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:38:41.744 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:41.745 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:41.745 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:38:41.846 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:38:41.869 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:41.870 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:41.870 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:38:41.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:41 smithi084 ceph-mon[32629]: Health check failed: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-16T09:38:41.971 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:38:42.008 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:42.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:42.009 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:38:42.110 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:38:42.144 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:42.144 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:42.145 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:38:42.246 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:38:42.292 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:42.293 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:42.293 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:38:42.394 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:38:42.421 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:38:41 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:38:41] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:38:42.433 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:42.433 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:42.434 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:38:42.468 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.4 dump_blocked_ops 2024-09-16T09:38:42.534 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:38:42.552 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 46 16 75048 75032 101.902 14.1042 0.0112073 0.00980843 2024-09-16T09:38:42.574 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:42.575 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:42.575 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:38:42.676 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:38:42.705 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:42.705 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:42.706 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:38:42.767 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-16T09:38:42.807 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:38:42.827 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:42 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:38:42.827 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:42 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:38:42.828 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:42 smithi070 ceph-mon[40691]: osdmap e401: 8 total, 8 up, 8 in 2024-09-16T09:38:42.828 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:42 smithi070 ceph-mon[40691]: pgmap v726: 178 pgs: 1 active+recovering+undersized+remapped, 2 active+clean+scrubbing, 7 remapped+peering, 2 active+remapped+backfill_toofull, 166 active+clean; 4.5 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 13871/223446 objects misplaced (6.208%); 377 KiB/s, 6 objects/s recovering 2024-09-16T09:38:42.828 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:42 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:38:42.829 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:42 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:38:42.829 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:42 smithi070 ceph-mon[30388]: osdmap e401: 8 total, 8 up, 8 in 2024-09-16T09:38:42.829 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:42 smithi070 ceph-mon[30388]: pgmap v726: 178 pgs: 1 active+recovering+undersized+remapped, 2 active+clean+scrubbing, 7 remapped+peering, 2 active+remapped+backfill_toofull, 166 active+clean; 4.5 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 13871/223446 objects misplaced (6.208%); 377 KiB/s, 6 objects/s recovering 2024-09-16T09:38:42.860 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:38:42.870 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:42.871 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:42.871 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:38:42.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:42 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:38:42.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:42 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:38:42.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:42 smithi084 ceph-mon[32629]: osdmap e401: 8 total, 8 up, 8 in 2024-09-16T09:38:42.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:42 smithi084 ceph-mon[32629]: pgmap v726: 178 pgs: 1 active+recovering+undersized+remapped, 2 active+clean+scrubbing, 7 remapped+peering, 2 active+remapped+backfill_toofull, 166 active+clean; 4.5 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 13871/223446 objects misplaced (6.208%); 377 KiB/s, 6 objects/s recovering 2024-09-16T09:38:42.972 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:38:42.993 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:42.994 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:42.994 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:38:43.095 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:38:43.142 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:43.143 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:43.144 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:38:43.245 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:38:43.277 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:43.278 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:43.278 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:38:43.379 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:38:43.417 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:43.418 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:43.418 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:38:43.458 INFO:teuthology.orchestra.run.smithi084.stdout:ERROR: (22) Invalid argument 2024-09-16T09:38:43.458 INFO:teuthology.orchestra.run.smithi084.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:38:43.519 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:38:43.552 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:43.552 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 47 16 75993 75977 100.991 59.0625 0.0166358 0.00989716 2024-09-16T09:38:43.553 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:43.553 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:38:43.574 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:38:43 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:38:43.654 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:38:43.682 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:43.683 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:43.683 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:38:43.784 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:38:43.827 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:43.828 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:43.828 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:38:43.836 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:43 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:38:43.836 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:43 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:38:43.836 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:43 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:38:43.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:43 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:38:43.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:43 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:38:43.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:43 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:38:43.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:43 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:38:43.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:43 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:38:43.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:43 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:38:43.930 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:38:43.970 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:43.971 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:43.971 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:38:44.072 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:38:44.095 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:44.096 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:44.096 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:38:44.197 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:38:44.202 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.4 dump_historic_ops 2024-09-16T09:38:44.236 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:44.237 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:44.237 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:38:44.338 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:38:44.371 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:44.372 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:44.372 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:38:44.473 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:38:44.516 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:44.516 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:44.517 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:38:44.552 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 48 16 76699 76683 99.8062 44.125 0.0169073 0.0100124 2024-09-16T09:38:44.576 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:38:44.618 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:38:44.652 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:44.653 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:44.653 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:38:44.754 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:38:44.775 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:44.776 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:44.776 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:38:44.877 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:38:44.899 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:44.899 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:44.900 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:38:44.912 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:44 smithi084 ceph-mon[32629]: osdmap e402: 8 total, 8 up, 8 in 2024-09-16T09:38:44.912 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:44 smithi084 ceph-mon[32629]: pgmap v728: 178 pgs: 5 active+recovering+undersized+remapped, 3 active+recovery_wait+undersized+degraded+remapped, 1 active+clean+scrubbing, 2 active+remapped+backfill_toofull, 167 active+clean; 4.6 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 975 B/s rd, 0 op/s; 13884/226224 objects degraded (6.137%); 27799/226224 objects misplaced (12.288%); 20 MiB/s, 318 objects/s recovering 2024-09-16T09:38:44.912 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:44 smithi084 ceph-mon[32629]: osdmap e403: 8 total, 8 up, 8 in 2024-09-16T09:38:44.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:44 smithi070 ceph-mon[40691]: osdmap e402: 8 total, 8 up, 8 in 2024-09-16T09:38:44.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:44 smithi070 ceph-mon[40691]: pgmap v728: 178 pgs: 5 active+recovering+undersized+remapped, 3 active+recovery_wait+undersized+degraded+remapped, 1 active+clean+scrubbing, 2 active+remapped+backfill_toofull, 167 active+clean; 4.6 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 975 B/s rd, 0 op/s; 13884/226224 objects degraded (6.137%); 27799/226224 objects misplaced (12.288%); 20 MiB/s, 318 objects/s recovering 2024-09-16T09:38:44.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:44 smithi070 ceph-mon[40691]: osdmap e403: 8 total, 8 up, 8 in 2024-09-16T09:38:44.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:44 smithi070 ceph-mon[30388]: osdmap e402: 8 total, 8 up, 8 in 2024-09-16T09:38:44.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:44 smithi070 ceph-mon[30388]: pgmap v728: 178 pgs: 5 active+recovering+undersized+remapped, 3 active+recovery_wait+undersized+degraded+remapped, 1 active+clean+scrubbing, 2 active+remapped+backfill_toofull, 167 active+clean; 4.6 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 975 B/s rd, 0 op/s; 13884/226224 objects degraded (6.137%); 27799/226224 objects misplaced (12.288%); 20 MiB/s, 318 objects/s recovering 2024-09-16T09:38:44.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:44 smithi070 ceph-mon[30388]: osdmap e403: 8 total, 8 up, 8 in 2024-09-16T09:38:45.001 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:38:45.034 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:45.035 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:45.035 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:38:45.136 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:38:45.171 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:45.171 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:45.171 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:38:45.273 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:38:45.314 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:45.314 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:45.315 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:38:45.416 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:38:45.462 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:45.463 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:45.463 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:38:45.552 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 49 16 77455 77439 98.7339 47.25 0.00295693 0.0101181 2024-09-16T09:38:45.564 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:38:45.608 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:45.608 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:45.608 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:38:45.709 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:38:45.752 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:45.753 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:45.753 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:38:45.854 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:38:45.876 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:45.876 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:45.877 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:38:45.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:45 smithi084 ceph-mon[32629]: Health check failed: Degraded data redundancy: 13884/226224 objects degraded (6.137%), 3 pgs degraded (PG_DEGRADED) 2024-09-16T09:38:45.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:45 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/507276849' entity='client.admin' cmd={"prefix": "osd dump", "format": "json-pretty"} : dispatch 2024-09-16T09:38:45.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:45 smithi070 ceph-mon[30388]: Health check failed: Degraded data redundancy: 13884/226224 objects degraded (6.137%), 3 pgs degraded (PG_DEGRADED) 2024-09-16T09:38:45.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:45 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/507276849' entity='client.admin' cmd={"prefix": "osd dump", "format": "json-pretty"} : dispatch 2024-09-16T09:38:45.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:45 smithi070 ceph-mon[40691]: Health check failed: Degraded data redundancy: 13884/226224 objects degraded (6.137%), 3 pgs degraded (PG_DEGRADED) 2024-09-16T09:38:45.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:45 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/507276849' entity='client.admin' cmd={"prefix": "osd dump", "format": "json-pretty"} : dispatch 2024-09-16T09:38:45.977 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:38:46.011 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:46.012 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:46.012 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:38:46.112 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:38:46.141 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:46.142 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:46.142 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:38:46.243 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:38:46.287 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:46.287 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:46.288 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:38:46.389 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:38:46.411 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:46.411 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:46.412 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:38:46.512 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:38:46.552 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 50 16 78374 78358 97.9081 57.4375 0.00920023 0.0102085 2024-09-16T09:38:46.569 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:46.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:46.570 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:38:46.671 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:38:46.704 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:46.704 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:46.705 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:38:46.805 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:38:46.849 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:46.850 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:46.850 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:38:46.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:46 smithi084 ceph-mon[32629]: pgmap v730: 178 pgs: 5 active+recovering+undersized+remapped, 3 active+recovery_wait+undersized+degraded+remapped, 2 active+remapped+backfill_toofull, 168 active+clean; 4.6 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 852 B/s rd, 0 op/s; 13884/227652 objects degraded (6.099%); 27521/227652 objects misplaced (12.089%); 20 MiB/s, 319 objects/s recovering 2024-09-16T09:38:46.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:46 smithi070 ceph-mon[30388]: pgmap v730: 178 pgs: 5 active+recovering+undersized+remapped, 3 active+recovery_wait+undersized+degraded+remapped, 2 active+remapped+backfill_toofull, 168 active+clean; 4.6 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 852 B/s rd, 0 op/s; 13884/227652 objects degraded (6.099%); 27521/227652 objects misplaced (12.089%); 20 MiB/s, 319 objects/s recovering 2024-09-16T09:38:46.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:46 smithi070 ceph-mon[40691]: pgmap v730: 178 pgs: 5 active+recovering+undersized+remapped, 3 active+recovery_wait+undersized+degraded+remapped, 2 active+remapped+backfill_toofull, 168 active+clean; 4.6 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 852 B/s rd, 0 op/s; 13884/227652 objects degraded (6.099%); 27521/227652 objects misplaced (12.089%); 20 MiB/s, 319 objects/s recovering 2024-09-16T09:38:46.951 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:38:46.982 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:46.982 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:46.983 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:38:47.084 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:38:47.111 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:47.112 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:47.112 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:38:47.213 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:38:47.243 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:47.244 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:47.244 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:38:47.345 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:38:47.367 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:47.368 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:47.369 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:38:47.469 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:38:47.510 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:47.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:47.511 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:38:47.552 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 51 16 79167 79151 96.9604 49.5625 0.0105364 0.0103083 2024-09-16T09:38:47.612 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:38:47.638 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:47.638 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:47.639 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:38:47.739 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:38:47.761 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:47.762 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:47.762 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:38:47.863 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:38:47.886 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:47.887 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:47.887 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:38:47.988 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:38:48.027 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:48.028 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:48.028 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:38:48.129 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:38:48.162 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:48.163 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:48.164 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:38:48.264 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:38:48.300 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:48.300 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:48.301 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:38:48.402 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:38:48.427 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:48.428 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:48.429 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:38:48.472 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:38:48.472 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T09:38:48.472 INFO:teuthology.orchestra.run.smithi070.stdout: "epoch": 403, 2024-09-16T09:38:48.472 INFO:teuthology.orchestra.run.smithi070.stdout: "fsid": "a1321834-740c-11ef-bceb-c7b262605968", 2024-09-16T09:38:48.472 INFO:teuthology.orchestra.run.smithi070.stdout: "created": "2024-09-16T09:19:14.121498+0000", 2024-09-16T09:38:48.473 INFO:teuthology.orchestra.run.smithi070.stdout: "modified": "2024-09-16T09:38:44.342870+0000", 2024-09-16T09:38:48.473 INFO:teuthology.orchestra.run.smithi070.stdout: "last_up_change": "2024-09-16T09:24:51.124496+0000", 2024-09-16T09:38:48.473 INFO:teuthology.orchestra.run.smithi070.stdout: "last_in_change": "2024-09-16T09:24:28.432920+0000", 2024-09-16T09:38:48.473 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": "noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-16T09:38:48.473 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_num": 5801984, 2024-09-16T09:38:48.473 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_set": [ 2024-09-16T09:38:48.474 INFO:teuthology.orchestra.run.smithi070.stdout: "noscrub", 2024-09-16T09:38:48.474 INFO:teuthology.orchestra.run.smithi070.stdout: "pglog_hardlimit", 2024-09-16T09:38:48.474 INFO:teuthology.orchestra.run.smithi070.stdout: "purged_snapdirs", 2024-09-16T09:38:48.474 INFO:teuthology.orchestra.run.smithi070.stdout: "recovery_deletes", 2024-09-16T09:38:48.474 INFO:teuthology.orchestra.run.smithi070.stdout: "sortbitwise" 2024-09-16T09:38:48.474 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-16T09:38:48.475 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_version": 18, 2024-09-16T09:38:48.475 INFO:teuthology.orchestra.run.smithi070.stdout: "full_ratio": 0.94999998807907104, 2024-09-16T09:38:48.475 INFO:teuthology.orchestra.run.smithi070.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-16T09:38:48.475 INFO:teuthology.orchestra.run.smithi070.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-16T09:38:48.475 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_snapshot": "", 2024-09-16T09:38:48.475 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_max": 10, 2024-09-16T09:38:48.475 INFO:teuthology.orchestra.run.smithi070.stdout: "max_osd": 8, 2024-09-16T09:38:48.476 INFO:teuthology.orchestra.run.smithi070.stdout: "require_min_compat_client": "luminous", 2024-09-16T09:38:48.476 INFO:teuthology.orchestra.run.smithi070.stdout: "min_compat_client": "luminous", 2024-09-16T09:38:48.476 INFO:teuthology.orchestra.run.smithi070.stdout: "require_osd_release": "squid", 2024-09-16T09:38:48.476 INFO:teuthology.orchestra.run.smithi070.stdout: "allow_crimson": false, 2024-09-16T09:38:48.476 INFO:teuthology.orchestra.run.smithi070.stdout: "pools": [ 2024-09-16T09:38:48.476 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.477 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 1, 2024-09-16T09:38:48.477 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": ".mgr", 2024-09-16T09:38:48.477 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2024-09-16T09:22:27.617694+0000", 2024-09-16T09:38:48.477 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2024-09-16T09:38:48.478 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2024-09-16T09:38:48.478 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2024-09-16T09:38:48.478 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 3, 2024-09-16T09:38:48.478 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 2, 2024-09-16T09:38:48.478 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2024-09-16T09:38:48.478 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2024-09-16T09:38:48.478 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2024-09-16T09:38:48.479 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T09:38:48.479 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T09:38:48.479 INFO:teuthology.orchestra.run.smithi070.stdout: "is_stretch_pool": false, 2024-09-16T09:38:48.479 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2024-09-16T09:38:48.479 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2024-09-16T09:38:48.479 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 21, 2024-09-16T09:38:48.479 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 21, 2024-09-16T09:38:48.479 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 21, 2024-09-16T09:38:48.480 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 21, 2024-09-16T09:38:48.480 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 21, 2024-09-16T09:38:48.480 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2024-09-16T09:38:48.480 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2024-09-16T09:38:48.480 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2024-09-16T09:38:48.481 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2024-09-16T09:38:48.481 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2024-09-16T09:38:48.481 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2024-09-16T09:38:48.481 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2024-09-16T09:38:48.481 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.481 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "162", 2024-09-16T09:38:48.482 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2024-09-16T09:38:48.482 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T09:38:48.482 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "160", 2024-09-16T09:38:48.482 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2024-09-16T09:38:48.482 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2024-09-16T09:38:48.482 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2024-09-16T09:38:48.483 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2024-09-16T09:38:48.483 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2024-09-16T09:38:48.483 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2024-09-16T09:38:48.483 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2024-09-16T09:38:48.483 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2024-09-16T09:38:48.484 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2024-09-16T09:38:48.484 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2024-09-16T09:38:48.484 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2024-09-16T09:38:48.484 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2024-09-16T09:38:48.484 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2024-09-16T09:38:48.485 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2024-09-16T09:38:48.485 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2024-09-16T09:38:48.485 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T09:38:48.485 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T09:38:48.485 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T09:38:48.485 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2024-09-16T09:38:48.486 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2024-09-16T09:38:48.486 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2024-09-16T09:38:48.486 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2024-09-16T09:38:48.486 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2024-09-16T09:38:48.486 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.486 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2024-09-16T09:38:48.486 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2024-09-16T09:38:48.486 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2024-09-16T09:38:48.487 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2024-09-16T09:38:48.487 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2024-09-16T09:38:48.487 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T09:38:48.487 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2024-09-16T09:38:48.487 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2024-09-16T09:38:48.487 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2024-09-16T09:38:48.487 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2024-09-16T09:38:48.487 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2024-09-16T09:38:48.487 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2024-09-16T09:38:48.488 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_max": 32, 2024-09-16T09:38:48.488 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_min": 1 2024-09-16T09:38:48.488 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.488 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2024-09-16T09:38:48.488 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": {} 2024-09-16T09:38:48.488 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.488 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2024-09-16T09:38:48.488 INFO:teuthology.orchestra.run.smithi070.stdout: "score_type": "Fair distribution", 2024-09-16T09:38:48.489 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 1.6699999570846558, 2024-09-16T09:38:48.489 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 1.6699999570846558, 2024-09-16T09:38:48.489 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1.1399999856948853, 2024-09-16T09:38:48.489 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 1.8999999761581421, 2024-09-16T09:38:48.489 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 1.8999999761581421, 2024-09-16T09:38:48.489 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 0.70719337463378906, 2024-09-16T09:38:48.490 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 0.81000000238418579, 2024-09-16T09:38:48.490 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 0.70999997854232788 2024-09-16T09:38:48.490 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:38:48.490 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.490 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.490 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 2, 2024-09-16T09:38:48.490 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "datapool", 2024-09-16T09:38:48.491 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2024-09-16T09:24:59.169244+0000", 2024-09-16T09:38:48.491 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 8193, 2024-09-16T09:38:48.491 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-16T09:38:48.491 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2024-09-16T09:38:48.491 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 3, 2024-09-16T09:38:48.491 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 2, 2024-09-16T09:38:48.491 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2024-09-16T09:38:48.491 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2024-09-16T09:38:48.492 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2024-09-16T09:38:48.492 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T09:38:48.492 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T09:38:48.492 INFO:teuthology.orchestra.run.smithi070.stdout: "is_stretch_pool": false, 2024-09-16T09:38:48.492 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2024-09-16T09:38:48.492 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2024-09-16T09:38:48.492 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 3, 2024-09-16T09:38:48.492 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 3, 2024-09-16T09:38:48.492 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 3, 2024-09-16T09:38:48.492 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 3, 2024-09-16T09:38:48.492 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 3, 2024-09-16T09:38:48.493 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2024-09-16T09:38:48.493 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2024-09-16T09:38:48.493 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2024-09-16T09:38:48.493 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2024-09-16T09:38:48.493 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2024-09-16T09:38:48.493 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2024-09-16T09:38:48.493 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2024-09-16T09:38:48.493 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.493 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "61", 2024-09-16T09:38:48.493 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2024-09-16T09:38:48.493 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T09:38:48.493 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T09:38:48.493 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2024-09-16T09:38:48.494 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2024-09-16T09:38:48.494 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 2, 2024-09-16T09:38:48.494 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 61, 2024-09-16T09:38:48.494 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2024-09-16T09:38:48.494 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2024-09-16T09:38:48.494 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2024-09-16T09:38:48.494 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2024-09-16T09:38:48.494 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2024-09-16T09:38:48.494 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2024-09-16T09:38:48.494 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2024-09-16T09:38:48.494 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2024-09-16T09:38:48.494 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2024-09-16T09:38:48.494 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2024-09-16T09:38:48.494 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2024-09-16T09:38:48.495 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T09:38:48.495 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T09:38:48.495 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T09:38:48.495 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2024-09-16T09:38:48.495 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2024-09-16T09:38:48.495 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2024-09-16T09:38:48.495 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2024-09-16T09:38:48.495 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2024-09-16T09:38:48.495 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.495 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2024-09-16T09:38:48.495 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2024-09-16T09:38:48.495 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2024-09-16T09:38:48.495 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2024-09-16T09:38:48.495 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2024-09-16T09:38:48.496 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T09:38:48.496 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2024-09-16T09:38:48.496 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2024-09-16T09:38:48.496 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2024-09-16T09:38:48.496 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2024-09-16T09:38:48.496 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2024-09-16T09:38:48.496 INFO:teuthology.orchestra.run.smithi070.stdout: "options": {}, 2024-09-16T09:38:48.496 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2024-09-16T09:38:48.496 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd": {} 2024-09-16T09:38:48.496 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.496 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2024-09-16T09:38:48.496 INFO:teuthology.orchestra.run.smithi070.stdout: "score_type": "Fair distribution", 2024-09-16T09:38:48.497 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 2.3299999237060547, 2024-09-16T09:38:48.497 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 2.3299999237060547, 2024-09-16T09:38:48.497 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 0.86000001430511475, 2024-09-16T09:38:48.497 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 2, 2024-09-16T09:38:48.497 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 2, 2024-09-16T09:38:48.497 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 0.70719337463378906, 2024-09-16T09:38:48.497 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 0.81000000238418579, 2024-09-16T09:38:48.497 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 0.70999997854232788 2024-09-16T09:38:48.497 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:38:48.497 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.497 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.497 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 3, 2024-09-16T09:38:48.497 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": ".rgw.root", 2024-09-16T09:38:48.497 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2024-09-16T09:25:00.056119+0000", 2024-09-16T09:38:48.498 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2024-09-16T09:38:48.498 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2024-09-16T09:38:48.498 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2024-09-16T09:38:48.498 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 3, 2024-09-16T09:38:48.498 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 2, 2024-09-16T09:38:48.498 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2024-09-16T09:38:48.498 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2024-09-16T09:38:48.498 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2024-09-16T09:38:48.498 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T09:38:48.498 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T09:38:48.498 INFO:teuthology.orchestra.run.smithi070.stdout: "is_stretch_pool": false, 2024-09-16T09:38:48.498 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2024-09-16T09:38:48.498 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2024-09-16T09:38:48.498 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 32, 2024-09-16T09:38:48.499 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 32, 2024-09-16T09:38:48.499 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 32, 2024-09-16T09:38:48.499 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 32, 2024-09-16T09:38:48.499 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 32, 2024-09-16T09:38:48.499 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2024-09-16T09:38:48.499 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2024-09-16T09:38:48.499 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2024-09-16T09:38:48.499 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2024-09-16T09:38:48.499 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2024-09-16T09:38:48.499 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2024-09-16T09:38:48.499 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2024-09-16T09:38:48.499 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.499 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "60", 2024-09-16T09:38:48.500 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2024-09-16T09:38:48.500 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T09:38:48.500 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T09:38:48.500 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2024-09-16T09:38:48.500 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2024-09-16T09:38:48.500 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2024-09-16T09:38:48.500 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2024-09-16T09:38:48.500 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2024-09-16T09:38:48.500 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2024-09-16T09:38:48.500 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2024-09-16T09:38:48.500 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2024-09-16T09:38:48.500 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2024-09-16T09:38:48.500 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2024-09-16T09:38:48.501 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2024-09-16T09:38:48.501 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2024-09-16T09:38:48.501 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2024-09-16T09:38:48.501 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2024-09-16T09:38:48.501 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2024-09-16T09:38:48.501 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T09:38:48.501 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T09:38:48.501 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T09:38:48.501 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2024-09-16T09:38:48.501 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2024-09-16T09:38:48.501 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2024-09-16T09:38:48.501 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2024-09-16T09:38:48.502 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2024-09-16T09:38:48.502 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.502 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2024-09-16T09:38:48.502 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2024-09-16T09:38:48.502 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2024-09-16T09:38:48.502 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2024-09-16T09:38:48.503 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2024-09-16T09:38:48.503 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T09:38:48.503 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2024-09-16T09:38:48.503 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2024-09-16T09:38:48.503 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2024-09-16T09:38:48.503 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2024-09-16T09:38:48.503 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2024-09-16T09:38:48.503 INFO:teuthology.orchestra.run.smithi070.stdout: "options": {}, 2024-09-16T09:38:48.503 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2024-09-16T09:38:48.503 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw": {} 2024-09-16T09:38:48.503 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.503 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2024-09-16T09:38:48.504 INFO:teuthology.orchestra.run.smithi070.stdout: "score_type": "Fair distribution", 2024-09-16T09:38:48.504 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 1.75, 2024-09-16T09:38:48.504 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 1.75, 2024-09-16T09:38:48.504 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1.1399999856948853, 2024-09-16T09:38:48.504 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 2, 2024-09-16T09:38:48.504 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 2, 2024-09-16T09:38:48.504 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 0.70719337463378906, 2024-09-16T09:38:48.504 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 0.81000000238418579, 2024-09-16T09:38:48.504 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 0.70999997854232788 2024-09-16T09:38:48.504 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:38:48.504 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.504 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.504 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 4, 2024-09-16T09:38:48.505 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "default.rgw.log", 2024-09-16T09:38:48.505 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2024-09-16T09:25:01.770016+0000", 2024-09-16T09:38:48.505 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2024-09-16T09:38:48.505 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2024-09-16T09:38:48.505 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2024-09-16T09:38:48.505 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 3, 2024-09-16T09:38:48.505 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 2, 2024-09-16T09:38:48.505 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2024-09-16T09:38:48.505 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2024-09-16T09:38:48.505 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2024-09-16T09:38:48.505 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T09:38:48.505 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T09:38:48.505 INFO:teuthology.orchestra.run.smithi070.stdout: "is_stretch_pool": false, 2024-09-16T09:38:48.506 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2024-09-16T09:38:48.506 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2024-09-16T09:38:48.506 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 32, 2024-09-16T09:38:48.506 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 32, 2024-09-16T09:38:48.506 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 32, 2024-09-16T09:38:48.506 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 32, 2024-09-16T09:38:48.506 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 32, 2024-09-16T09:38:48.506 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2024-09-16T09:38:48.506 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2024-09-16T09:38:48.506 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2024-09-16T09:38:48.506 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2024-09-16T09:38:48.506 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2024-09-16T09:38:48.506 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2024-09-16T09:38:48.506 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2024-09-16T09:38:48.507 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.507 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "62", 2024-09-16T09:38:48.507 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2024-09-16T09:38:48.507 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T09:38:48.507 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T09:38:48.507 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2024-09-16T09:38:48.507 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2024-09-16T09:38:48.507 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2024-09-16T09:38:48.507 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2024-09-16T09:38:48.507 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2024-09-16T09:38:48.507 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2024-09-16T09:38:48.507 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2024-09-16T09:38:48.507 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2024-09-16T09:38:48.507 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2024-09-16T09:38:48.508 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2024-09-16T09:38:48.508 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2024-09-16T09:38:48.508 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2024-09-16T09:38:48.508 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2024-09-16T09:38:48.508 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2024-09-16T09:38:48.508 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2024-09-16T09:38:48.508 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T09:38:48.508 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T09:38:48.508 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T09:38:48.508 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2024-09-16T09:38:48.508 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2024-09-16T09:38:48.508 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2024-09-16T09:38:48.508 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2024-09-16T09:38:48.509 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2024-09-16T09:38:48.509 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.509 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2024-09-16T09:38:48.509 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2024-09-16T09:38:48.509 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2024-09-16T09:38:48.509 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2024-09-16T09:38:48.509 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2024-09-16T09:38:48.509 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T09:38:48.509 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2024-09-16T09:38:48.509 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2024-09-16T09:38:48.509 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2024-09-16T09:38:48.509 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2024-09-16T09:38:48.509 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2024-09-16T09:38:48.509 INFO:teuthology.orchestra.run.smithi070.stdout: "options": {}, 2024-09-16T09:38:48.510 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2024-09-16T09:38:48.510 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw": {} 2024-09-16T09:38:48.510 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.510 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2024-09-16T09:38:48.510 INFO:teuthology.orchestra.run.smithi070.stdout: "score_type": "Fair distribution", 2024-09-16T09:38:48.510 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 6.8499999046325684, 2024-09-16T09:38:48.510 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 2.190000057220459, 2024-09-16T09:38:48.510 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1.1399999856948853, 2024-09-16T09:38:48.510 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 7.809999942779541, 2024-09-16T09:38:48.510 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 2.5, 2024-09-16T09:38:48.510 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 0.70719337463378906, 2024-09-16T09:38:48.510 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 0.81000000238418579, 2024-09-16T09:38:48.510 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 0.70999997854232788, 2024-09-16T09:38:48.510 INFO:teuthology.orchestra.run.smithi070.stdout: "error_message": "pool 4 has acting primaries on OSD(s) with primary affinity 0, read balance score is not accurate" 2024-09-16T09:38:48.511 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:38:48.511 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.511 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.511 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 5, 2024-09-16T09:38:48.511 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "default.rgw.control", 2024-09-16T09:38:48.511 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2024-09-16T09:25:03.725555+0000", 2024-09-16T09:38:48.511 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2024-09-16T09:38:48.511 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2024-09-16T09:38:48.511 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2024-09-16T09:38:48.511 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 3, 2024-09-16T09:38:48.511 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 2, 2024-09-16T09:38:48.512 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2024-09-16T09:38:48.512 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2024-09-16T09:38:48.512 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2024-09-16T09:38:48.512 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T09:38:48.512 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T09:38:48.512 INFO:teuthology.orchestra.run.smithi070.stdout: "is_stretch_pool": false, 2024-09-16T09:38:48.512 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2024-09-16T09:38:48.512 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2024-09-16T09:38:48.512 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 32, 2024-09-16T09:38:48.512 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 32, 2024-09-16T09:38:48.512 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 32, 2024-09-16T09:38:48.512 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 32, 2024-09-16T09:38:48.512 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 32, 2024-09-16T09:38:48.512 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2024-09-16T09:38:48.513 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2024-09-16T09:38:48.513 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2024-09-16T09:38:48.513 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2024-09-16T09:38:48.513 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2024-09-16T09:38:48.513 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2024-09-16T09:38:48.513 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2024-09-16T09:38:48.513 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.513 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "64", 2024-09-16T09:38:48.513 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2024-09-16T09:38:48.513 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T09:38:48.514 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T09:38:48.514 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2024-09-16T09:38:48.514 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2024-09-16T09:38:48.514 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2024-09-16T09:38:48.514 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2024-09-16T09:38:48.514 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2024-09-16T09:38:48.514 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2024-09-16T09:38:48.514 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2024-09-16T09:38:48.514 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2024-09-16T09:38:48.514 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2024-09-16T09:38:48.514 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2024-09-16T09:38:48.514 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2024-09-16T09:38:48.514 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2024-09-16T09:38:48.514 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2024-09-16T09:38:48.515 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2024-09-16T09:38:48.515 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2024-09-16T09:38:48.515 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T09:38:48.515 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T09:38:48.515 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T09:38:48.515 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2024-09-16T09:38:48.515 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2024-09-16T09:38:48.515 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2024-09-16T09:38:48.515 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2024-09-16T09:38:48.515 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2024-09-16T09:38:48.515 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.515 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2024-09-16T09:38:48.515 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2024-09-16T09:38:48.516 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2024-09-16T09:38:48.516 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2024-09-16T09:38:48.516 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2024-09-16T09:38:48.516 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T09:38:48.516 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2024-09-16T09:38:48.516 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2024-09-16T09:38:48.516 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2024-09-16T09:38:48.516 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2024-09-16T09:38:48.516 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2024-09-16T09:38:48.516 INFO:teuthology.orchestra.run.smithi070.stdout: "options": {}, 2024-09-16T09:38:48.516 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2024-09-16T09:38:48.516 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw": {} 2024-09-16T09:38:48.516 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.517 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2024-09-16T09:38:48.517 INFO:teuthology.orchestra.run.smithi070.stdout: "score_type": "Fair distribution", 2024-09-16T09:38:48.517 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 1.75, 2024-09-16T09:38:48.517 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 1.75, 2024-09-16T09:38:48.517 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1.1399999856948853, 2024-09-16T09:38:48.517 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 2, 2024-09-16T09:38:48.517 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 2, 2024-09-16T09:38:48.517 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 0.70719337463378906, 2024-09-16T09:38:48.517 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 0.81000000238418579, 2024-09-16T09:38:48.517 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 0.70999997854232788 2024-09-16T09:38:48.517 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:38:48.518 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.518 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.518 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 6, 2024-09-16T09:38:48.518 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "default.rgw.meta", 2024-09-16T09:38:48.518 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2024-09-16T09:25:05.773530+0000", 2024-09-16T09:38:48.518 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2024-09-16T09:38:48.518 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2024-09-16T09:38:48.519 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2024-09-16T09:38:48.519 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 3, 2024-09-16T09:38:48.519 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 2, 2024-09-16T09:38:48.519 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2024-09-16T09:38:48.519 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2024-09-16T09:38:48.519 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2024-09-16T09:38:48.520 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T09:38:48.520 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T09:38:48.520 INFO:teuthology.orchestra.run.smithi070.stdout: "is_stretch_pool": false, 2024-09-16T09:38:48.520 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2024-09-16T09:38:48.520 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2024-09-16T09:38:48.520 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 32, 2024-09-16T09:38:48.520 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 32, 2024-09-16T09:38:48.521 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 32, 2024-09-16T09:38:48.521 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 32, 2024-09-16T09:38:48.521 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 32, 2024-09-16T09:38:48.521 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2024-09-16T09:38:48.521 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2024-09-16T09:38:48.521 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2024-09-16T09:38:48.521 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2024-09-16T09:38:48.521 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2024-09-16T09:38:48.521 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2024-09-16T09:38:48.521 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2024-09-16T09:38:48.521 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.521 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "66", 2024-09-16T09:38:48.522 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2024-09-16T09:38:48.522 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T09:38:48.522 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T09:38:48.522 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2024-09-16T09:38:48.522 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2024-09-16T09:38:48.522 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2024-09-16T09:38:48.522 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2024-09-16T09:38:48.522 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2024-09-16T09:38:48.522 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2024-09-16T09:38:48.522 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2024-09-16T09:38:48.522 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2024-09-16T09:38:48.522 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2024-09-16T09:38:48.522 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2024-09-16T09:38:48.522 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2024-09-16T09:38:48.523 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2024-09-16T09:38:48.523 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2024-09-16T09:38:48.523 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2024-09-16T09:38:48.523 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2024-09-16T09:38:48.523 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T09:38:48.523 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T09:38:48.523 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T09:38:48.523 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2024-09-16T09:38:48.523 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2024-09-16T09:38:48.523 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2024-09-16T09:38:48.523 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2024-09-16T09:38:48.524 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2024-09-16T09:38:48.524 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.524 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2024-09-16T09:38:48.524 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2024-09-16T09:38:48.524 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2024-09-16T09:38:48.524 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2024-09-16T09:38:48.524 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2024-09-16T09:38:48.524 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T09:38:48.524 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2024-09-16T09:38:48.524 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2024-09-16T09:38:48.524 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2024-09-16T09:38:48.524 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2024-09-16T09:38:48.524 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2024-09-16T09:38:48.525 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2024-09-16T09:38:48.525 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_bias": 4 2024-09-16T09:38:48.525 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.525 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2024-09-16T09:38:48.525 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw": {} 2024-09-16T09:38:48.525 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.525 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2024-09-16T09:38:48.525 INFO:teuthology.orchestra.run.smithi070.stdout: "score_type": "Fair distribution", 2024-09-16T09:38:48.525 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 1.5399999618530273, 2024-09-16T09:38:48.525 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 1.5399999618530273, 2024-09-16T09:38:48.525 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1.1399999856948853, 2024-09-16T09:38:48.525 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 1.75, 2024-09-16T09:38:48.525 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 1.75, 2024-09-16T09:38:48.525 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 0.70719337463378906, 2024-09-16T09:38:48.526 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 0.81000000238418579, 2024-09-16T09:38:48.526 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 0.70999997854232788 2024-09-16T09:38:48.526 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:38:48.526 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.526 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.526 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 10, 2024-09-16T09:38:48.526 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "unique_pool_3", 2024-09-16T09:38:48.526 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2024-09-16T09:37:43.482267+0000", 2024-09-16T09:38:48.526 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2024-09-16T09:38:48.526 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2024-09-16T09:38:48.526 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2024-09-16T09:38:48.526 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 3, 2024-09-16T09:38:48.526 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 2, 2024-09-16T09:38:48.526 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2024-09-16T09:38:48.527 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2024-09-16T09:38:48.527 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2024-09-16T09:38:48.527 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T09:38:48.527 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T09:38:48.527 INFO:teuthology.orchestra.run.smithi070.stdout: "is_stretch_pool": false, 2024-09-16T09:38:48.527 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2024-09-16T09:38:48.527 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2024-09-16T09:38:48.527 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 26, 2024-09-16T09:38:48.527 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 26, 2024-09-16T09:38:48.527 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 26, 2024-09-16T09:38:48.527 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 26, 2024-09-16T09:38:48.527 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 26, 2024-09-16T09:38:48.527 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2024-09-16T09:38:48.527 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2024-09-16T09:38:48.528 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2024-09-16T09:38:48.528 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2024-09-16T09:38:48.528 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2024-09-16T09:38:48.528 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2024-09-16T09:38:48.528 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2024-09-16T09:38:48.528 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.528 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "397", 2024-09-16T09:38:48.528 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2024-09-16T09:38:48.528 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T09:38:48.528 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "395", 2024-09-16T09:38:48.528 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2024-09-16T09:38:48.529 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2024-09-16T09:38:48.529 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2024-09-16T09:38:48.529 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2024-09-16T09:38:48.529 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2024-09-16T09:38:48.529 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2024-09-16T09:38:48.529 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2024-09-16T09:38:48.529 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2024-09-16T09:38:48.529 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2024-09-16T09:38:48.529 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2024-09-16T09:38:48.529 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2024-09-16T09:38:48.529 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2024-09-16T09:38:48.529 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2024-09-16T09:38:48.529 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2024-09-16T09:38:48.529 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2024-09-16T09:38:48.530 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T09:38:48.530 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T09:38:48.530 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T09:38:48.530 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2024-09-16T09:38:48.530 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2024-09-16T09:38:48.530 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2024-09-16T09:38:48.530 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2024-09-16T09:38:48.530 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2024-09-16T09:38:48.530 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.530 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2024-09-16T09:38:48.530 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2024-09-16T09:38:48.531 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2024-09-16T09:38:48.531 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2024-09-16T09:38:48.531 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2024-09-16T09:38:48.531 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T09:38:48.532 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2024-09-16T09:38:48.532 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2024-09-16T09:38:48.532 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2024-09-16T09:38:48.532 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2024-09-16T09:38:48.532 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2024-09-16T09:38:48.532 INFO:teuthology.orchestra.run.smithi070.stdout: "options": {}, 2024-09-16T09:38:48.532 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2024-09-16T09:38:48.532 INFO:teuthology.orchestra.run.smithi070.stdout: "rados": {} 2024-09-16T09:38:48.532 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.532 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2024-09-16T09:38:48.532 INFO:teuthology.orchestra.run.smithi070.stdout: "score_type": "Fair distribution", 2024-09-16T09:38:48.532 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 1.6200000047683716, 2024-09-16T09:38:48.532 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 2.1600000858306885, 2024-09-16T09:38:48.533 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1.1399999856948853, 2024-09-16T09:38:48.533 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 1.8500000238418579, 2024-09-16T09:38:48.533 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 2.4600000381469727, 2024-09-16T09:38:48.533 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 0.70719337463378906, 2024-09-16T09:38:48.533 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 0.81000000238418579, 2024-09-16T09:38:48.533 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 0.70999997854232788 2024-09-16T09:38:48.533 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:38:48.533 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:38:48.533 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-16T09:38:48.533 INFO:teuthology.orchestra.run.smithi070.stdout: "osds": [ 2024-09-16T09:38:48.533 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.533 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 0, 2024-09-16T09:38:48.533 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "7ef288c8-3bf0-4814-8e86-f70a7de041ce", 2024-09-16T09:38:48.533 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2024-09-16T09:38:48.534 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2024-09-16T09:38:48.534 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2024-09-16T09:38:48.534 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2024-09-16T09:38:48.534 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2024-09-16T09:38:48.534 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2024-09-16T09:38:48.534 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 8, 2024-09-16T09:38:48.534 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 398, 2024-09-16T09:38:48.534 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2024-09-16T09:38:48.534 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2024-09-16T09:38:48.534 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2024-09-16T09:38:48.534 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:38:48.535 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.535 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:38:48.535 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6802", 2024-09-16T09:38:48.535 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2013075781 2024-09-16T09:38:48.535 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.535 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.535 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:38:48.535 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6803", 2024-09-16T09:38:48.535 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2013075781 2024-09-16T09:38:48.535 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:38:48.535 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:38:48.535 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.535 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2024-09-16T09:38:48.535 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:38:48.536 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.536 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:38:48.536 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6804", 2024-09-16T09:38:48.536 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2013075781 2024-09-16T09:38:48.536 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.536 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.536 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:38:48.536 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6805", 2024-09-16T09:38:48.536 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2013075781 2024-09-16T09:38:48.536 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:38:48.536 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:38:48.536 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.536 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2024-09-16T09:38:48.537 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:38:48.537 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.537 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:38:48.537 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6808", 2024-09-16T09:38:48.537 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2013075781 2024-09-16T09:38:48.537 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.537 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.537 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:38:48.537 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6809", 2024-09-16T09:38:48.537 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2013075781 2024-09-16T09:38:48.537 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:38:48.537 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:38:48.537 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.537 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2024-09-16T09:38:48.538 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:38:48.538 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.538 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:38:48.538 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6806", 2024-09-16T09:38:48.538 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2013075781 2024-09-16T09:38:48.538 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.538 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.538 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:38:48.538 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6807", 2024-09-16T09:38:48.538 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2013075781 2024-09-16T09:38:48.538 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:38:48.538 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:38:48.538 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.539 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6803/2013075781", 2024-09-16T09:38:48.539 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6805/2013075781", 2024-09-16T09:38:48.539 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6809/2013075781", 2024-09-16T09:38:48.539 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6807/2013075781", 2024-09-16T09:38:48.539 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2024-09-16T09:38:48.539 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2024-09-16T09:38:48.539 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2024-09-16T09:38:48.539 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:38:48.539 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.539 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.539 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 1, 2024-09-16T09:38:48.539 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "474d41d9-295a-450c-8f27-ce487a14678b", 2024-09-16T09:38:48.539 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2024-09-16T09:38:48.539 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2024-09-16T09:38:48.540 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2024-09-16T09:38:48.540 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 0, 2024-09-16T09:38:48.540 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2024-09-16T09:38:48.540 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2024-09-16T09:38:48.540 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2024-09-16T09:38:48.540 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 362, 2024-09-16T09:38:48.540 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2024-09-16T09:38:48.540 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2024-09-16T09:38:48.540 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2024-09-16T09:38:48.540 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:38:48.540 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.540 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:38:48.540 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6810", 2024-09-16T09:38:48.541 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1670012115 2024-09-16T09:38:48.541 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.541 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.541 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:38:48.541 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6811", 2024-09-16T09:38:48.541 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1670012115 2024-09-16T09:38:48.541 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:38:48.541 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:38:48.541 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.541 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2024-09-16T09:38:48.541 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:38:48.541 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.541 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:38:48.541 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6812", 2024-09-16T09:38:48.542 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1670012115 2024-09-16T09:38:48.542 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.542 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.542 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:38:48.542 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6813", 2024-09-16T09:38:48.542 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1670012115 2024-09-16T09:38:48.542 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:38:48.542 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:38:48.542 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.542 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2024-09-16T09:38:48.542 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:38:48.542 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.542 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:38:48.542 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6816", 2024-09-16T09:38:48.543 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1670012115 2024-09-16T09:38:48.543 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.543 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.543 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:38:48.543 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6817", 2024-09-16T09:38:48.543 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1670012115 2024-09-16T09:38:48.543 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:38:48.543 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:38:48.543 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.543 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2024-09-16T09:38:48.543 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:38:48.543 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.543 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:38:48.544 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6814", 2024-09-16T09:38:48.544 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1670012115 2024-09-16T09:38:48.544 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.544 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.544 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:38:48.544 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6815", 2024-09-16T09:38:48.544 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1670012115 2024-09-16T09:38:48.544 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:38:48.544 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:38:48.544 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.544 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6811/1670012115", 2024-09-16T09:38:48.545 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6813/1670012115", 2024-09-16T09:38:48.545 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6817/1670012115", 2024-09-16T09:38:48.545 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6815/1670012115", 2024-09-16T09:38:48.545 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2024-09-16T09:38:48.545 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2024-09-16T09:38:48.545 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2024-09-16T09:38:48.545 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:38:48.545 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.545 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.545 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 2, 2024-09-16T09:38:48.545 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "5b4ca4e8-cecc-4163-b291-c9209b02c4b7", 2024-09-16T09:38:48.545 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2024-09-16T09:38:48.545 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2024-09-16T09:38:48.546 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2024-09-16T09:38:48.546 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 0.1810150146484375, 2024-09-16T09:38:48.546 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2024-09-16T09:38:48.546 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2024-09-16T09:38:48.546 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 18, 2024-09-16T09:38:48.546 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 398, 2024-09-16T09:38:48.546 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2024-09-16T09:38:48.546 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2024-09-16T09:38:48.546 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2024-09-16T09:38:48.546 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:38:48.546 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.546 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:38:48.546 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6818", 2024-09-16T09:38:48.546 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3601682336 2024-09-16T09:38:48.547 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.547 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.547 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:38:48.547 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6819", 2024-09-16T09:38:48.547 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3601682336 2024-09-16T09:38:48.547 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:38:48.547 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:38:48.547 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.547 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2024-09-16T09:38:48.547 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:38:48.547 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.547 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:38:48.547 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6820", 2024-09-16T09:38:48.547 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3601682336 2024-09-16T09:38:48.548 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.548 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.548 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:38:48.548 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6821", 2024-09-16T09:38:48.548 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3601682336 2024-09-16T09:38:48.548 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:38:48.548 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:38:48.548 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.548 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2024-09-16T09:38:48.548 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:38:48.548 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.548 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:38:48.549 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6824", 2024-09-16T09:38:48.549 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3601682336 2024-09-16T09:38:48.549 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.549 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.549 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:38:48.549 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6825", 2024-09-16T09:38:48.549 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3601682336 2024-09-16T09:38:48.549 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:38:48.549 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:38:48.549 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.549 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2024-09-16T09:38:48.549 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:38:48.549 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.550 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:38:48.550 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6822", 2024-09-16T09:38:48.550 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3601682336 2024-09-16T09:38:48.550 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.550 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.550 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:38:48.550 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6823", 2024-09-16T09:38:48.550 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3601682336 2024-09-16T09:38:48.550 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:38:48.550 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:38:48.550 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.550 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6819/3601682336", 2024-09-16T09:38:48.550 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6821/3601682336", 2024-09-16T09:38:48.551 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6825/3601682336", 2024-09-16T09:38:48.551 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6823/3601682336", 2024-09-16T09:38:48.551 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2024-09-16T09:38:48.551 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2024-09-16T09:38:48.551 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2024-09-16T09:38:48.551 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:38:48.552 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.552 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.552 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 3, 2024-09-16T09:38:48.552 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "ffb8d527-1b58-4ecd-808c-47d4ba1bf387", 2024-09-16T09:38:48.552 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2024-09-16T09:38:48.552 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2024-09-16T09:38:48.552 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2024-09-16T09:38:48.552 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 0.476531982421875, 2024-09-16T09:38:48.552 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2024-09-16T09:38:48.552 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2024-09-16T09:38:48.552 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 26, 2024-09-16T09:38:48.553 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 398, 2024-09-16T09:38:48.553 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2024-09-16T09:38:48.553 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2024-09-16T09:38:48.553 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2024-09-16T09:38:48.553 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:38:48.553 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.553 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:38:48.553 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6826", 2024-09-16T09:38:48.553 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2038083007 2024-09-16T09:38:48.553 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.553 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.553 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:38:48.554 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6827", 2024-09-16T09:38:48.554 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2038083007 2024-09-16T09:38:48.554 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:38:48.554 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:38:48.554 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.554 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2024-09-16T09:38:48.554 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:38:48.554 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.554 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:38:48.554 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6828", 2024-09-16T09:38:48.555 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2038083007 2024-09-16T09:38:48.555 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.555 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.555 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:38:48.555 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6829", 2024-09-16T09:38:48.555 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2038083007 2024-09-16T09:38:48.555 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:38:48.555 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:38:48.555 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.555 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2024-09-16T09:38:48.556 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:38:48.556 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.556 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:38:48.556 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6832", 2024-09-16T09:38:48.556 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2038083007 2024-09-16T09:38:48.556 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.556 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.556 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:38:48.556 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6833", 2024-09-16T09:38:48.556 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2038083007 2024-09-16T09:38:48.556 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:38:48.556 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:38:48.557 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.557 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2024-09-16T09:38:48.557 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:38:48.557 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.557 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:38:48.557 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6830", 2024-09-16T09:38:48.557 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2038083007 2024-09-16T09:38:48.557 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.557 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.557 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:38:48.557 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6831", 2024-09-16T09:38:48.557 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2038083007 2024-09-16T09:38:48.557 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:38:48.558 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:38:48.558 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.558 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6827/2038083007", 2024-09-16T09:38:48.558 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6829/2038083007", 2024-09-16T09:38:48.558 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6833/2038083007", 2024-09-16T09:38:48.558 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6831/2038083007", 2024-09-16T09:38:48.558 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2024-09-16T09:38:48.558 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2024-09-16T09:38:48.558 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2024-09-16T09:38:48.558 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:38:48.558 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.558 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.558 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 4, 2024-09-16T09:38:48.558 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "a946760b-4d91-4a4b-8e9b-a07a50589211", 2024-09-16T09:38:48.559 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2024-09-16T09:38:48.559 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2024-09-16T09:38:48.559 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2024-09-16T09:38:48.559 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2024-09-16T09:38:48.559 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2024-09-16T09:38:48.559 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2024-09-16T09:38:48.559 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 32, 2024-09-16T09:38:48.559 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 398, 2024-09-16T09:38:48.559 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2024-09-16T09:38:48.559 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2024-09-16T09:38:48.559 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2024-09-16T09:38:48.560 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:38:48.560 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.560 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:38:48.560 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6800", 2024-09-16T09:38:48.560 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2250892710 2024-09-16T09:38:48.560 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.560 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.560 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:38:48.560 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6801", 2024-09-16T09:38:48.560 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2250892710 2024-09-16T09:38:48.560 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:38:48.561 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:38:48.561 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.561 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2024-09-16T09:38:48.561 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:38:48.561 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.561 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:38:48.561 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6802", 2024-09-16T09:38:48.561 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2250892710 2024-09-16T09:38:48.561 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.561 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.561 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:38:48.561 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6803", 2024-09-16T09:38:48.561 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2250892710 2024-09-16T09:38:48.561 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:38:48.562 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:38:48.562 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.562 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2024-09-16T09:38:48.562 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:38:48.562 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.562 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:38:48.562 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6806", 2024-09-16T09:38:48.562 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2250892710 2024-09-16T09:38:48.562 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.562 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.562 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:38:48.562 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6807", 2024-09-16T09:38:48.562 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2250892710 2024-09-16T09:38:48.562 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:38:48.563 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:38:48.563 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.563 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2024-09-16T09:38:48.563 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:38:48.563 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.563 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:38:48.563 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6804", 2024-09-16T09:38:48.563 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2250892710 2024-09-16T09:38:48.563 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.563 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.563 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:38:48.563 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6805", 2024-09-16T09:38:48.563 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2250892710 2024-09-16T09:38:48.563 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:38:48.564 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:38:48.564 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.564 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.84:6801/2250892710", 2024-09-16T09:38:48.564 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.84:6803/2250892710", 2024-09-16T09:38:48.564 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.84:6807/2250892710", 2024-09-16T09:38:48.564 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.84:6805/2250892710", 2024-09-16T09:38:48.564 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2024-09-16T09:38:48.564 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2024-09-16T09:38:48.564 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2024-09-16T09:38:48.564 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:38:48.564 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.564 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.564 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 5, 2024-09-16T09:38:48.564 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "67681760-75a9-4846-946e-a935d42c5e05", 2024-09-16T09:38:48.565 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2024-09-16T09:38:48.565 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2024-09-16T09:38:48.565 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2024-09-16T09:38:48.565 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2024-09-16T09:38:48.565 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2024-09-16T09:38:48.565 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2024-09-16T09:38:48.565 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 39, 2024-09-16T09:38:48.565 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 397, 2024-09-16T09:38:48.565 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2024-09-16T09:38:48.565 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2024-09-16T09:38:48.565 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2024-09-16T09:38:48.565 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:38:48.565 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.565 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:38:48.566 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6808", 2024-09-16T09:38:48.566 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1803945348 2024-09-16T09:38:48.566 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.566 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.566 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:38:48.566 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6809", 2024-09-16T09:38:48.566 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1803945348 2024-09-16T09:38:48.566 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:38:48.566 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:38:48.566 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.566 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2024-09-16T09:38:48.567 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:38:48.567 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.567 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:38:48.567 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6810", 2024-09-16T09:38:48.567 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1803945348 2024-09-16T09:38:48.567 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.567 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.567 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:38:48.567 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6811", 2024-09-16T09:38:48.567 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1803945348 2024-09-16T09:38:48.567 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:38:48.568 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:38:48.568 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.568 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2024-09-16T09:38:48.568 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:38:48.568 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.568 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:38:48.568 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6814", 2024-09-16T09:38:48.568 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1803945348 2024-09-16T09:38:48.568 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.568 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.568 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:38:48.568 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6815", 2024-09-16T09:38:48.568 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1803945348 2024-09-16T09:38:48.569 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:38:48.569 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:38:48.569 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.569 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2024-09-16T09:38:48.569 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:38:48.569 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.569 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:38:48.569 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6812", 2024-09-16T09:38:48.569 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1803945348 2024-09-16T09:38:48.569 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.570 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.570 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:38:48.570 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6813", 2024-09-16T09:38:48.570 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1803945348 2024-09-16T09:38:48.570 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:38:48.570 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:38:48.570 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.570 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.84:6809/1803945348", 2024-09-16T09:38:48.570 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.84:6811/1803945348", 2024-09-16T09:38:48.570 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.84:6815/1803945348", 2024-09-16T09:38:48.570 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.84:6813/1803945348", 2024-09-16T09:38:48.570 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2024-09-16T09:38:48.570 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2024-09-16T09:38:48.570 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2024-09-16T09:38:48.571 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:38:48.571 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.571 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.571 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 6, 2024-09-16T09:38:48.571 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "7c8f0152-f539-4b7e-8113-91dd7529d3a1", 2024-09-16T09:38:48.571 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2024-09-16T09:38:48.571 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2024-09-16T09:38:48.571 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2024-09-16T09:38:48.571 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2024-09-16T09:38:48.571 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2024-09-16T09:38:48.571 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2024-09-16T09:38:48.571 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 46, 2024-09-16T09:38:48.571 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 397, 2024-09-16T09:38:48.571 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2024-09-16T09:38:48.572 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2024-09-16T09:38:48.572 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2024-09-16T09:38:48.572 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:38:48.572 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.572 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:38:48.572 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6816", 2024-09-16T09:38:48.572 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 610727189 2024-09-16T09:38:48.572 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.572 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.572 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:38:48.572 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6817", 2024-09-16T09:38:48.572 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 610727189 2024-09-16T09:38:48.573 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:38:48.573 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:38:48.573 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.573 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2024-09-16T09:38:48.573 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:38:48.573 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.573 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:38:48.573 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6818", 2024-09-16T09:38:48.573 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 610727189 2024-09-16T09:38:48.573 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.573 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.573 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:38:48.574 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6819", 2024-09-16T09:38:48.574 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 610727189 2024-09-16T09:38:48.574 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:38:48.574 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:38:48.574 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.574 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2024-09-16T09:38:48.574 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:38:48.574 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.575 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:38:48.575 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6822", 2024-09-16T09:38:48.575 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 610727189 2024-09-16T09:38:48.575 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.575 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.575 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:38:48.575 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6823", 2024-09-16T09:38:48.575 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 610727189 2024-09-16T09:38:48.575 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:38:48.575 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:38:48.575 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.575 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2024-09-16T09:38:48.575 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:38:48.576 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.576 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:38:48.576 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6820", 2024-09-16T09:38:48.576 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 610727189 2024-09-16T09:38:48.576 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.576 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.576 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:38:48.576 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6821", 2024-09-16T09:38:48.576 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 610727189 2024-09-16T09:38:48.577 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:38:48.577 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:38:48.577 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.577 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.84:6817/610727189", 2024-09-16T09:38:48.577 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.84:6819/610727189", 2024-09-16T09:38:48.577 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.84:6823/610727189", 2024-09-16T09:38:48.577 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.84:6821/610727189", 2024-09-16T09:38:48.577 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2024-09-16T09:38:48.577 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2024-09-16T09:38:48.577 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2024-09-16T09:38:48.577 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:38:48.577 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.577 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.578 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 7, 2024-09-16T09:38:48.578 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "945efaba-225e-49af-aefd-b4d58ca8093b", 2024-09-16T09:38:48.578 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2024-09-16T09:38:48.578 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2024-09-16T09:38:48.578 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2024-09-16T09:38:48.578 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2024-09-16T09:38:48.578 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2024-09-16T09:38:48.578 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2024-09-16T09:38:48.578 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 53, 2024-09-16T09:38:48.578 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 397, 2024-09-16T09:38:48.578 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2024-09-16T09:38:48.578 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2024-09-16T09:38:48.578 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2024-09-16T09:38:48.579 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:38:48.579 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.579 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:38:48.579 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6824", 2024-09-16T09:38:48.579 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1002252531 2024-09-16T09:38:48.579 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.579 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.579 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:38:48.579 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6825", 2024-09-16T09:38:48.579 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1002252531 2024-09-16T09:38:48.580 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:38:48.580 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:38:48.580 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.580 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2024-09-16T09:38:48.580 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:38:48.580 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.581 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:38:48.581 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6826", 2024-09-16T09:38:48.581 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1002252531 2024-09-16T09:38:48.581 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.581 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.581 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:38:48.582 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6827", 2024-09-16T09:38:48.582 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1002252531 2024-09-16T09:38:48.582 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:38:48.582 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:38:48.582 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.582 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2024-09-16T09:38:48.582 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:38:48.583 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.583 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:38:48.583 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6830", 2024-09-16T09:38:48.583 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1002252531 2024-09-16T09:38:48.583 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.583 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.583 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:38:48.583 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6831", 2024-09-16T09:38:48.584 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1002252531 2024-09-16T09:38:48.584 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:38:48.584 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:38:48.584 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.584 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2024-09-16T09:38:48.584 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:38:48.584 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.585 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:38:48.585 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6828", 2024-09-16T09:38:48.585 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1002252531 2024-09-16T09:38:48.585 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.585 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.586 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:38:48.586 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6829", 2024-09-16T09:38:48.586 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1002252531 2024-09-16T09:38:48.586 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:38:48.586 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:38:48.586 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.586 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.84:6825/1002252531", 2024-09-16T09:38:48.587 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.84:6827/1002252531", 2024-09-16T09:38:48.587 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.84:6831/1002252531", 2024-09-16T09:38:48.587 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.84:6829/1002252531", 2024-09-16T09:38:48.587 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2024-09-16T09:38:48.587 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2024-09-16T09:38:48.587 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2024-09-16T09:38:48.588 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:38:48.588 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:38:48.588 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-16T09:38:48.588 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_xinfo": [ 2024-09-16T09:38:48.588 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.588 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 0, 2024-09-16T09:38:48.588 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2024-09-16T09:38:48.588 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2024-09-16T09:38:48.589 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2024-09-16T09:38:48.589 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540719139924082687, 2024-09-16T09:38:48.589 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2024-09-16T09:38:48.589 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2024-09-16T09:37:48.793810+0000", 2024-09-16T09:38:48.589 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2024-09-16T09:38:48.589 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.589 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.589 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 1, 2024-09-16T09:38:48.589 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2024-09-16T09:38:48.589 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2024-09-16T09:38:48.589 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2024-09-16T09:38:48.590 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540719139924082687, 2024-09-16T09:38:48.590 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2024-09-16T09:38:48.590 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2024-09-16T09:38:42.788265+0000", 2024-09-16T09:38:48.590 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2024-09-16T09:38:48.590 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.590 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.590 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 2, 2024-09-16T09:38:48.590 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2024-09-16T09:38:48.590 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2024-09-16T09:38:48.590 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2024-09-16T09:38:48.590 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540719139924082687, 2024-09-16T09:38:48.590 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2024-09-16T09:38:48.591 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2024-09-16T09:38:35.110455+0000", 2024-09-16T09:38:48.591 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2024-09-16T09:38:48.591 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.591 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.591 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 3, 2024-09-16T09:38:48.591 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2024-09-16T09:38:48.591 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2024-09-16T09:38:48.591 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2024-09-16T09:38:48.591 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540719139924082687, 2024-09-16T09:38:48.591 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2024-09-16T09:38:48.591 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2024-09-16T09:38:41.243696+0000", 2024-09-16T09:38:48.591 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2024-09-16T09:38:48.591 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.592 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.592 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 4, 2024-09-16T09:38:48.592 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2024-09-16T09:38:48.592 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2024-09-16T09:38:48.592 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2024-09-16T09:38:48.592 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540719139924082687, 2024-09-16T09:38:48.592 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2024-09-16T09:38:48.592 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2024-09-16T09:38:35.892571+0000", 2024-09-16T09:38:48.592 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2024-09-16T09:38:48.592 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.592 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.592 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 5, 2024-09-16T09:38:48.593 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2024-09-16T09:38:48.593 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2024-09-16T09:38:48.593 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2024-09-16T09:38:48.593 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540719139924082687, 2024-09-16T09:38:48.593 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2024-09-16T09:38:48.593 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2024-09-16T09:37:48.073602+0000", 2024-09-16T09:38:48.593 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2024-09-16T09:38:48.593 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.593 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.593 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 6, 2024-09-16T09:38:48.593 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2024-09-16T09:38:48.593 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2024-09-16T09:38:48.593 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2024-09-16T09:38:48.594 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540719139924082687, 2024-09-16T09:38:48.594 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2024-09-16T09:38:48.594 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2024-09-16T09:37:46.771979+0000", 2024-09-16T09:38:48.594 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2024-09-16T09:38:48.594 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.594 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.594 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 7, 2024-09-16T09:38:48.594 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2024-09-16T09:38:48.594 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2024-09-16T09:38:48.594 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2024-09-16T09:38:48.594 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540719139924082687, 2024-09-16T09:38:48.594 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2024-09-16T09:38:48.594 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2024-09-16T09:37:46.052484+0000", 2024-09-16T09:38:48.594 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2024-09-16T09:38:48.594 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:38:48.595 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-16T09:38:48.595 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap": [ 2024-09-16T09:38:48.595 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.595 INFO:teuthology.orchestra.run.smithi070.stdout: "pgid": "3.1d", 2024-09-16T09:38:48.595 INFO:teuthology.orchestra.run.smithi070.stdout: "osds": [ 2024-09-16T09:38:48.595 INFO:teuthology.orchestra.run.smithi070.stdout: 1, 2024-09-16T09:38:48.595 INFO:teuthology.orchestra.run.smithi070.stdout: 0, 2024-09-16T09:38:48.595 INFO:teuthology.orchestra.run.smithi070.stdout: 7 2024-09-16T09:38:48.595 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:38:48.595 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.595 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.595 INFO:teuthology.orchestra.run.smithi070.stdout: "pgid": "4.b", 2024-09-16T09:38:48.595 INFO:teuthology.orchestra.run.smithi070.stdout: "osds": [ 2024-09-16T09:38:48.596 INFO:teuthology.orchestra.run.smithi070.stdout: 0, 2024-09-16T09:38:48.596 INFO:teuthology.orchestra.run.smithi070.stdout: 5, 2024-09-16T09:38:48.596 INFO:teuthology.orchestra.run.smithi070.stdout: 1 2024-09-16T09:38:48.596 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:38:48.596 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:38:48.596 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-16T09:38:48.596 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap_items": [ 2024-09-16T09:38:48.596 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.596 INFO:teuthology.orchestra.run.smithi070.stdout: "pgid": "2.1", 2024-09-16T09:38:48.596 INFO:teuthology.orchestra.run.smithi070.stdout: "mappings": [ 2024-09-16T09:38:48.597 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.597 INFO:teuthology.orchestra.run.smithi070.stdout: "from": 3, 2024-09-16T09:38:48.597 INFO:teuthology.orchestra.run.smithi070.stdout: "to": 0 2024-09-16T09:38:48.597 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.597 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.597 INFO:teuthology.orchestra.run.smithi070.stdout: "from": 2, 2024-09-16T09:38:48.597 INFO:teuthology.orchestra.run.smithi070.stdout: "to": 1 2024-09-16T09:38:48.597 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:38:48.597 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:38:48.597 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.597 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.597 INFO:teuthology.orchestra.run.smithi070.stdout: "pgid": "3.1b", 2024-09-16T09:38:48.597 INFO:teuthology.orchestra.run.smithi070.stdout: "mappings": [ 2024-09-16T09:38:48.598 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.598 INFO:teuthology.orchestra.run.smithi070.stdout: "from": 0, 2024-09-16T09:38:48.598 INFO:teuthology.orchestra.run.smithi070.stdout: "to": 6 2024-09-16T09:38:48.598 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.598 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.598 INFO:teuthology.orchestra.run.smithi070.stdout: "from": 4, 2024-09-16T09:38:48.598 INFO:teuthology.orchestra.run.smithi070.stdout: "to": 7 2024-09-16T09:38:48.598 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:38:48.598 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:38:48.598 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.598 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.598 INFO:teuthology.orchestra.run.smithi070.stdout: "pgid": "4.f", 2024-09-16T09:38:48.598 INFO:teuthology.orchestra.run.smithi070.stdout: "mappings": [ 2024-09-16T09:38:48.598 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.599 INFO:teuthology.orchestra.run.smithi070.stdout: "from": 1, 2024-09-16T09:38:48.599 INFO:teuthology.orchestra.run.smithi070.stdout: "to": 2 2024-09-16T09:38:48.599 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:38:48.599 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:38:48.599 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.599 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.599 INFO:teuthology.orchestra.run.smithi070.stdout: "pgid": "4.13", 2024-09-16T09:38:48.599 INFO:teuthology.orchestra.run.smithi070.stdout: "mappings": [ 2024-09-16T09:38:48.599 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.599 INFO:teuthology.orchestra.run.smithi070.stdout: "from": 1, 2024-09-16T09:38:48.599 INFO:teuthology.orchestra.run.smithi070.stdout: "to": 5 2024-09-16T09:38:48.599 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:38:48.599 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:38:48.599 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.599 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.600 INFO:teuthology.orchestra.run.smithi070.stdout: "pgid": "4.17", 2024-09-16T09:38:48.600 INFO:teuthology.orchestra.run.smithi070.stdout: "mappings": [ 2024-09-16T09:38:48.600 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.600 INFO:teuthology.orchestra.run.smithi070.stdout: "from": 1, 2024-09-16T09:38:48.600 INFO:teuthology.orchestra.run.smithi070.stdout: "to": 2 2024-09-16T09:38:48.600 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:38:48.600 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:38:48.600 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:38:48.600 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-16T09:38:48.600 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap_primaries": [], 2024-09-16T09:38:48.600 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_temp": [ 2024-09-16T09:38:48.600 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.600 INFO:teuthology.orchestra.run.smithi070.stdout: "pgid": "10.10", 2024-09-16T09:38:48.600 INFO:teuthology.orchestra.run.smithi070.stdout: "osds": [ 2024-09-16T09:38:48.601 INFO:teuthology.orchestra.run.smithi070.stdout: 4, 2024-09-16T09:38:48.601 INFO:teuthology.orchestra.run.smithi070.stdout: 6 2024-09-16T09:38:48.601 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:38:48.601 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.601 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.601 INFO:teuthology.orchestra.run.smithi070.stdout: "pgid": "10.11", 2024-09-16T09:38:48.601 INFO:teuthology.orchestra.run.smithi070.stdout: "osds": [ 2024-09-16T09:38:48.601 INFO:teuthology.orchestra.run.smithi070.stdout: 0, 2024-09-16T09:38:48.601 INFO:teuthology.orchestra.run.smithi070.stdout: 7, 2024-09-16T09:38:48.601 INFO:teuthology.orchestra.run.smithi070.stdout: 4 2024-09-16T09:38:48.601 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:38:48.601 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.601 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.601 INFO:teuthology.orchestra.run.smithi070.stdout: "pgid": "10.12", 2024-09-16T09:38:48.602 INFO:teuthology.orchestra.run.smithi070.stdout: "osds": [ 2024-09-16T09:38:48.602 INFO:teuthology.orchestra.run.smithi070.stdout: 0, 2024-09-16T09:38:48.602 INFO:teuthology.orchestra.run.smithi070.stdout: 1, 2024-09-16T09:38:48.602 INFO:teuthology.orchestra.run.smithi070.stdout: 4 2024-09-16T09:38:48.602 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:38:48.602 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.602 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.602 INFO:teuthology.orchestra.run.smithi070.stdout: "pgid": "10.13", 2024-09-16T09:38:48.602 INFO:teuthology.orchestra.run.smithi070.stdout: "osds": [ 2024-09-16T09:38:48.602 INFO:teuthology.orchestra.run.smithi070.stdout: 2, 2024-09-16T09:38:48.602 INFO:teuthology.orchestra.run.smithi070.stdout: 1 2024-09-16T09:38:48.602 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:38:48.602 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.602 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.603 INFO:teuthology.orchestra.run.smithi070.stdout: "pgid": "10.14", 2024-09-16T09:38:48.603 INFO:teuthology.orchestra.run.smithi070.stdout: "osds": [ 2024-09-16T09:38:48.603 INFO:teuthology.orchestra.run.smithi070.stdout: 4, 2024-09-16T09:38:48.603 INFO:teuthology.orchestra.run.smithi070.stdout: 1 2024-09-16T09:38:48.603 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:38:48.603 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.603 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.603 INFO:teuthology.orchestra.run.smithi070.stdout: "pgid": "10.15", 2024-09-16T09:38:48.603 INFO:teuthology.orchestra.run.smithi070.stdout: "osds": [ 2024-09-16T09:38:48.603 INFO:teuthology.orchestra.run.smithi070.stdout: 3, 2024-09-16T09:38:48.603 INFO:teuthology.orchestra.run.smithi070.stdout: 1 2024-09-16T09:38:48.603 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:38:48.603 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.603 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.604 INFO:teuthology.orchestra.run.smithi070.stdout: "pgid": "10.16", 2024-09-16T09:38:48.604 INFO:teuthology.orchestra.run.smithi070.stdout: "osds": [ 2024-09-16T09:38:48.604 INFO:teuthology.orchestra.run.smithi070.stdout: 0, 2024-09-16T09:38:48.604 INFO:teuthology.orchestra.run.smithi070.stdout: 5 2024-09-16T09:38:48.604 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:38:48.604 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.604 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.604 INFO:teuthology.orchestra.run.smithi070.stdout: "pgid": "10.17", 2024-09-16T09:38:48.604 INFO:teuthology.orchestra.run.smithi070.stdout: "osds": [ 2024-09-16T09:38:48.604 INFO:teuthology.orchestra.run.smithi070.stdout: 3, 2024-09-16T09:38:48.604 INFO:teuthology.orchestra.run.smithi070.stdout: 6 2024-09-16T09:38:48.604 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:38:48.604 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.604 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.605 INFO:teuthology.orchestra.run.smithi070.stdout: "pgid": "10.18", 2024-09-16T09:38:48.605 INFO:teuthology.orchestra.run.smithi070.stdout: "osds": [ 2024-09-16T09:38:48.605 INFO:teuthology.orchestra.run.smithi070.stdout: 2, 2024-09-16T09:38:48.605 INFO:teuthology.orchestra.run.smithi070.stdout: 1 2024-09-16T09:38:48.605 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:38:48.605 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.605 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:38:48.605 INFO:teuthology.orchestra.run.smithi070.stdout: "pgid": "10.19", 2024-09-16T09:38:48.605 INFO:teuthology.orchestra.run.smithi070.stdout: "osds": [ 2024-09-16T09:38:48.605 INFO:teuthology.orchestra.run.smithi070.stdout: 2, 2024-09-16T09:38:48.605 INFO:teuthology.orchestra.run.smithi070.stdout: 1 2024-09-16T09:38:48.605 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:38:48.605 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:38:48.605 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-16T09:38:48.606 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_temp": [], 2024-09-16T09:38:48.606 INFO:teuthology.orchestra.run.smithi070.stdout: "blocklist": { 2024-09-16T09:38:48.606 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/125491368": "2024-09-17T09:25:31.465167+0000", 2024-09-16T09:38:48.606 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/3354284852": "2024-09-17T09:19:43.205989+0000", 2024-09-16T09:38:48.606 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/3284625547": "2024-09-17T09:25:31.465167+0000", 2024-09-16T09:38:48.606 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/186565817": "2024-09-17T09:19:43.205989+0000", 2024-09-16T09:38:48.606 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/2213852865": "2024-09-17T09:19:43.205989+0000", 2024-09-16T09:38:48.606 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/1812373760": "2024-09-17T09:20:09.306330+0000", 2024-09-16T09:38:48.606 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:6800/837338094": "2024-09-17T09:19:43.205989+0000", 2024-09-16T09:38:48.606 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:6800/1876661229": "2024-09-17T09:25:31.465167+0000", 2024-09-16T09:38:48.607 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/3351688485": "2024-09-17T09:20:09.306330+0000", 2024-09-16T09:38:48.607 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:6800/3871553358": "2024-09-17T09:20:09.306330+0000", 2024-09-16T09:38:48.607 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/1230482871": "2024-09-17T09:25:31.465167+0000", 2024-09-16T09:38:48.607 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:6801/837338094": "2024-09-17T09:19:43.205989+0000", 2024-09-16T09:38:48.607 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:6801/3871553358": "2024-09-17T09:20:09.306330+0000", 2024-09-16T09:38:48.607 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/1403001486": "2024-09-17T09:20:09.306330+0000", 2024-09-16T09:38:48.607 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/1723643739": "2024-09-17T09:25:31.465167+0000", 2024-09-16T09:38:48.607 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:6801/1876661229": "2024-09-17T09:25:31.465167+0000", 2024-09-16T09:38:48.607 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/3456417396": "2024-09-17T09:25:31.465167+0000" 2024-09-16T09:38:48.607 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.607 INFO:teuthology.orchestra.run.smithi070.stdout: "range_blocklist": {}, 2024-09-16T09:38:48.607 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profiles": { 2024-09-16T09:38:48.607 INFO:teuthology.orchestra.run.smithi070.stdout: "default": { 2024-09-16T09:38:48.607 INFO:teuthology.orchestra.run.smithi070.stdout: "crush-failure-domain": "osd", 2024-09-16T09:38:48.608 INFO:teuthology.orchestra.run.smithi070.stdout: "k": "2", 2024-09-16T09:38:48.608 INFO:teuthology.orchestra.run.smithi070.stdout: "m": "1", 2024-09-16T09:38:48.608 INFO:teuthology.orchestra.run.smithi070.stdout: "plugin": "jerasure", 2024-09-16T09:38:48.608 INFO:teuthology.orchestra.run.smithi070.stdout: "technique": "reed_sol_van" 2024-09-16T09:38:48.608 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:38:48.608 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:38:48.608 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps_queue": [], 2024-09-16T09:38:48.608 INFO:teuthology.orchestra.run.smithi070.stdout: "new_removed_snaps": [], 2024-09-16T09:38:48.608 INFO:teuthology.orchestra.run.smithi070.stdout: "new_purged_snaps": [], 2024-09-16T09:38:48.608 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_node_flags": {}, 2024-09-16T09:38:48.608 INFO:teuthology.orchestra.run.smithi070.stdout: "device_class_flags": {}, 2024-09-16T09:38:48.608 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode": { 2024-09-16T09:38:48.608 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode_enabled": false, 2024-09-16T09:38:48.609 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_bucket_count": 0, 2024-09-16T09:38:48.609 INFO:teuthology.orchestra.run.smithi070.stdout: "degraded_stretch_mode": 0, 2024-09-16T09:38:48.609 INFO:teuthology.orchestra.run.smithi070.stdout: "recovering_stretch_mode": 0, 2024-09-16T09:38:48.609 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode_bucket": 0 2024-09-16T09:38:48.609 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:38:48.609 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T09:38:48.609 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:38:48.610 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 52 16 79897 79881 95.9733 45.625 0.00369204 0.0104102 2024-09-16T09:38:48.649 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:48.650 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:48.650 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:38:48.751 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:38:48.789 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:48.789 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:48.790 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:38:48.891 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:38:48.913 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:48.913 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:48.914 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:38:49.015 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:38:49.056 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:49.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:49.057 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:38:49.158 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:38:49.200 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:49.200 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:49.201 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:38:49.302 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:38:49.324 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:49.325 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:49.326 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:38:49.426 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:38:49.466 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:49.467 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:49.467 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:38:49.553 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 53 16 80727 80711 95.1414 51.875 0.0163486 0.0105057 2024-09-16T09:38:49.568 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:38:49.604 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:49.605 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:49.605 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:38:49.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:49 smithi084 ceph-mon[32629]: pgmap v731: 178 pgs: 5 active+recovering+undersized+remapped, 3 active+recovery_wait+undersized+degraded+remapped, 2 active+remapped+backfill_toofull, 168 active+clean; 4.6 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 823 B/s rd, 0 op/s; 13884/227652 objects degraded (6.099%); 27521/227652 objects misplaced (12.089%); 19 MiB/s, 308 objects/s recovering 2024-09-16T09:38:49.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:49 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:38:49.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:49 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:38:49.666 INFO:teuthology.orchestra.run.smithi084.stdout:ERROR: (22) Invalid argument 2024-09-16T09:38:49.666 INFO:teuthology.orchestra.run.smithi084.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:38:49.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:49 smithi070 ceph-mon[30388]: pgmap v731: 178 pgs: 5 active+recovering+undersized+remapped, 3 active+recovery_wait+undersized+degraded+remapped, 2 active+remapped+backfill_toofull, 168 active+clean; 4.6 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 823 B/s rd, 0 op/s; 13884/227652 objects degraded (6.099%); 27521/227652 objects misplaced (12.089%); 19 MiB/s, 308 objects/s recovering 2024-09-16T09:38:49.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:49 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:38:49.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:49 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:38:49.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:49 smithi070 ceph-mon[40691]: pgmap v731: 178 pgs: 5 active+recovering+undersized+remapped, 3 active+recovery_wait+undersized+degraded+remapped, 2 active+remapped+backfill_toofull, 168 active+clean; 4.6 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 823 B/s rd, 0 op/s; 13884/227652 objects degraded (6.099%); 27521/227652 objects misplaced (12.089%); 19 MiB/s, 308 objects/s recovering 2024-09-16T09:38:49.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:49 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:38:49.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:49 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:38:49.706 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:38:49.731 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:49.731 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:49.732 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:38:49.833 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:38:49.864 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:49.864 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:49.865 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:38:49.966 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:38:50.006 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:50.007 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:50.007 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:38:50.108 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:38:50.132 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:50.133 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:50.133 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:38:50.234 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:38:50.256 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:50.257 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:50.257 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:38:50.358 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:38:50.413 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:50.414 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:50.414 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:38:50.515 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:38:50.553 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 54 16 81591 81575 94.3797 54 0.0188365 0.0105898 2024-09-16T09:38:50.554 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:50.555 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:50.555 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:38:50.656 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:38:50.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:50 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:38:50.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:50 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:38:50.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:50 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:38:50.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:50 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:38:50.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:50 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:38:50.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:50 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:38:50.690 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:50.690 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:50.691 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:38:50.792 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:38:50.833 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:50.834 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:50.834 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:38:50.934 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:38:50.971 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:50.973 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:50.973 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:38:51.074 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:38:51.096 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:51.097 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:51.097 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:38:51.198 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:38:51.229 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:51.229 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:51.230 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:38:51.331 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:38:51.357 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:51.358 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:51.358 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:38:51.459 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:38:51.496 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:51.497 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:51.497 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:38:51.553 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 55 16 82450 82434 93.64 53.6875 0.00655602 0.0106738 2024-09-16T09:38:51.599 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:38:51.620 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:51.621 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:51.621 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:38:51.659 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:51 smithi084 ceph-mon[32629]: pgmap v732: 178 pgs: 5 active+recovering+undersized+remapped, 3 active+recovery_wait+undersized+degraded+remapped, 2 active+remapped+backfill_toofull, 168 active+clean; 4.9 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.8 MiB/s wr, 94 op/s; 13884/239538 objects degraded (5.796%); 26165/239538 objects misplaced (10.923%); 26 MiB/s, 414 objects/s recovering 2024-09-16T09:38:51.659 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:51 smithi084 ceph-mon[32629]: Health check update: Degraded data redundancy: 13884/239538 objects degraded (5.796%), 3 pgs degraded (PG_DEGRADED) 2024-09-16T09:38:51.659 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:51 smithi084 ceph-mon[32629]: osdmap e404: 8 total, 8 up, 8 in 2024-09-16T09:38:51.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:51 smithi070 ceph-mon[30388]: pgmap v732: 178 pgs: 5 active+recovering+undersized+remapped, 3 active+recovery_wait+undersized+degraded+remapped, 2 active+remapped+backfill_toofull, 168 active+clean; 4.9 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.8 MiB/s wr, 94 op/s; 13884/239538 objects degraded (5.796%); 26165/239538 objects misplaced (10.923%); 26 MiB/s, 414 objects/s recovering 2024-09-16T09:38:51.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:51 smithi070 ceph-mon[30388]: Health check update: Degraded data redundancy: 13884/239538 objects degraded (5.796%), 3 pgs degraded (PG_DEGRADED) 2024-09-16T09:38:51.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:51 smithi070 ceph-mon[30388]: osdmap e404: 8 total, 8 up, 8 in 2024-09-16T09:38:51.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:51 smithi070 ceph-mon[40691]: pgmap v732: 178 pgs: 5 active+recovering+undersized+remapped, 3 active+recovery_wait+undersized+degraded+remapped, 2 active+remapped+backfill_toofull, 168 active+clean; 4.9 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.8 MiB/s wr, 94 op/s; 13884/239538 objects degraded (5.796%); 26165/239538 objects misplaced (10.923%); 26 MiB/s, 414 objects/s recovering 2024-09-16T09:38:51.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:51 smithi070 ceph-mon[40691]: Health check update: Degraded data redundancy: 13884/239538 objects degraded (5.796%), 3 pgs degraded (PG_DEGRADED) 2024-09-16T09:38:51.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:51 smithi070 ceph-mon[40691]: osdmap e404: 8 total, 8 up, 8 in 2024-09-16T09:38:51.722 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:38:51.761 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:51.761 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:51.762 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:38:51.862 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:38:51.909 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:51.910 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:51.910 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:38:52.011 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:38:52.035 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:52.036 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:52.036 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:38:52.137 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:38:52.175 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:52.176 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:52.176 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:38:52.277 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:38:52.305 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:52.305 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:52.306 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:38:52.406 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:38:52.420 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:38:51 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:38:51] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:38:52.443 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:52.443 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:52.444 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:38:52.544 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:38:52.553 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 56 16 83329 83313 92.9487 54.9375 0.0126927 0.0107514 2024-09-16T09:38:52.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:52.569 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:52.570 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:38:52.670 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:38:52.714 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:52.714 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:52.715 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:38:52.815 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:38:52.845 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:52.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:52.846 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:38:52.947 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:38:52.978 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:52.978 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:52.979 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:38:53.079 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:38:53.106 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:53.107 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:53.107 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:38:53.208 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:38:53.259 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:53.259 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:53.260 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:38:53.361 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:38:53.393 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:53.393 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:53.394 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:38:53.494 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:38:53.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:53.536 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:53.537 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:38:53.553 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 57 16 84234 84218 92.3104 56.5625 0.00587228 0.0108216 2024-09-16T09:38:53.638 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:38:53.660 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:38:53 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:38:53.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:53 smithi084 ceph-mon[32629]: pgmap v734: 178 pgs: 5 active+recovering+undersized+remapped, 3 active+recovery_wait+undersized+degraded+remapped, 2 active+remapped+backfill_toofull, 168 active+clean; 4.9 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 639 B/s rd, 16 MiB/s wr, 256 op/s; 13884/239538 objects degraded (5.796%); 26165/239538 objects misplaced (10.923%); 13 MiB/s, 208 objects/s recovering 2024-09-16T09:38:53.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:53 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/834884105' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:38:53.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:53 smithi070 ceph-mon[30388]: pgmap v734: 178 pgs: 5 active+recovering+undersized+remapped, 3 active+recovery_wait+undersized+degraded+remapped, 2 active+remapped+backfill_toofull, 168 active+clean; 4.9 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 639 B/s rd, 16 MiB/s wr, 256 op/s; 13884/239538 objects degraded (5.796%); 26165/239538 objects misplaced (10.923%); 13 MiB/s, 208 objects/s recovering 2024-09-16T09:38:53.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:53 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/834884105' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:38:53.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:53 smithi070 ceph-mon[40691]: pgmap v734: 178 pgs: 5 active+recovering+undersized+remapped, 3 active+recovery_wait+undersized+degraded+remapped, 2 active+remapped+backfill_toofull, 168 active+clean; 4.9 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 639 B/s rd, 16 MiB/s wr, 256 op/s; 13884/239538 objects degraded (5.796%); 26165/239538 objects misplaced (10.923%); 13 MiB/s, 208 objects/s recovering 2024-09-16T09:38:53.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:53 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/834884105' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:38:53.673 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:53.674 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:53.674 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:38:53.767 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:38:53.767 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:38:53.767 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:38:53.767 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:38:53.767 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:38:53.768 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:38:53.768 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:38:53.768 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:38:53.768 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:38:53.768 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:38:53.768 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:38:53.768 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:38:53.768 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:38:53.768 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:38:53.768 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:38:53.768 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:38:53.775 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:38:53.805 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:53.806 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:53.806 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:38:53.907 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:38:53.936 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:53.937 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:53.937 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:38:54.038 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:38:54.065 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:54.066 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:54.066 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:38:54.167 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:38:54.189 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:54.189 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:54.190 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:38:54.290 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:38:54.318 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:54.318 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:54.319 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:38:54.361 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.5 dump_ops_in_flight 2024-09-16T09:38:54.420 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:38:54.507 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:54.507 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:54.508 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:38:54.553 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 58 16 84564 84548 91.0748 20.625 0.0109571 0.0109753 2024-09-16T09:38:54.609 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:38:54.654 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:54.655 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:54.655 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:38:54.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:54 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:38:54.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:54 smithi084 ceph-mon[32629]: osdmap e405: 8 total, 8 up, 8 in 2024-09-16T09:38:54.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:54 smithi084 ceph-mon[32629]: osdmap e406: 8 total, 8 up, 8 in 2024-09-16T09:38:54.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:54 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:38:54.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:54 smithi070 ceph-mon[30388]: osdmap e405: 8 total, 8 up, 8 in 2024-09-16T09:38:54.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:54 smithi070 ceph-mon[30388]: osdmap e406: 8 total, 8 up, 8 in 2024-09-16T09:38:54.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:54 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:38:54.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:54 smithi070 ceph-mon[40691]: osdmap e405: 8 total, 8 up, 8 in 2024-09-16T09:38:54.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:54 smithi070 ceph-mon[40691]: osdmap e406: 8 total, 8 up, 8 in 2024-09-16T09:38:54.756 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:38:54.763 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:38:54.797 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:54.798 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:54.799 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:38:54.899 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:38:54.939 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:54.940 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:54.940 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:38:55.041 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:38:55.080 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:55.081 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:55.081 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:38:55.182 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:38:55.219 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:55.220 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:55.221 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:38:55.321 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:38:55.368 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:55.369 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:55.370 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:38:55.470 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:38:55.508 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:55.509 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:55.509 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:38:55.553 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 59 16 85647 85631 90.6783 67.6875 0.020676 0.0110185 2024-09-16T09:38:55.595 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-09-16T09:38:55.596 INFO:teuthology.orchestra.run.smithi084.stdout: "ops": [ 2024-09-16T09:38:55.596 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:55.596 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15975.0:85643 10.6 10:673d7c40:::benchmark_data_smithi070_166744_object85642:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e406)", 2024-09-16T09:38:55.596 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:38:55.523724+0000", 2024-09-16T09:38:55.596 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 0.070023906999999996, 2024-09-16T09:38:55.596 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.070049531999999998, 2024-09-16T09:38:55.596 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:38:55.596 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:38:55.596 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "waiting for sub ops", 2024-09-16T09:38:55.597 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:38:55.597 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15975", 2024-09-16T09:38:55.597 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3821727158", 2024-09-16T09:38:55.597 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 85643 2024-09-16T09:38:55.597 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:55.597 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:38:55.597 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:55.597 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:38:55.597 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:38:55.523724+0000", 2024-09-16T09:38:55.597 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:38:55.597 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:55.597 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:55.597 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:38:55.597 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:38:55.523724+0000", 2024-09-16T09:38:55.598 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:38:55.598 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:55.600 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:55.600 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:38:55.600 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:38:55.523726+0000", 2024-09-16T09:38:55.600 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.1619999999999998e-06 2024-09-16T09:38:55.600 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:55.600 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:55.600 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:38:55.601 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:38:55.523768+0000", 2024-09-16T09:38:55.601 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.1255999999999999e-05 2024-09-16T09:38:55.601 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:55.601 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:55.601 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:38:55.601 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:38:55.523790+0000", 2024-09-16T09:38:55.601 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.1829999999999999e-05 2024-09-16T09:38:55.601 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:55.601 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:55.601 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:38:55.601 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:38:55.523800+0000", 2024-09-16T09:38:55.601 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0220999999999999e-05 2024-09-16T09:38:55.601 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:55.601 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:55.602 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:38:55.602 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:38:55.523854+0000", 2024-09-16T09:38:55.602 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.4605999999999999e-05 2024-09-16T09:38:55.602 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:55.602 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:55.602 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:38:55.602 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:38:55.523982+0000", 2024-09-16T09:38:55.602 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00012809200000000001 2024-09-16T09:38:55.602 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:55.602 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:55.602 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,4", 2024-09-16T09:38:55.602 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:38:55.524241+0000", 2024-09-16T09:38:55.602 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00025881400000000001 2024-09-16T09:38:55.603 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:55.603 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:55.603 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:38:55.603 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:38:55.525684+0000", 2024-09-16T09:38:55.603 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.001442846 2024-09-16T09:38:55.603 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:55.603 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:55.603 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:38:55.603 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:38:55.528130+0000", 2024-09-16T09:38:55.603 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0024456629999999998 2024-09-16T09:38:55.603 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:38:55.603 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:38:55.604 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:38:55.604 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:38:55.604 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-09-16T09:38:55.604 INFO:teuthology.orchestra.run.smithi084.stdout: "num_ops": 1 2024-09-16T09:38:55.604 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-09-16T09:38:55.611 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:38:55.654 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:55.655 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:55.656 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:38:55.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:55 smithi070 ceph-mon[30388]: pgmap v736: 178 pgs: 5 active+recovering+undersized+remapped, 3 active+recovery_wait+undersized+degraded+remapped, 2 active+remapped+backfill_toofull, 168 active+clean; 5.1 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.2 KiB/s rd, 58 MiB/s wr, 930 op/s; 13884/249963 objects degraded (5.554%); 24087/249963 objects misplaced (9.636%); 24 MiB/s, 386 objects/s recovering 2024-09-16T09:38:55.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:55 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:38:55.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:55 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:38:55.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:55 smithi070 ceph-mon[30388]: 10.10 scrub starts 2024-09-16T09:38:55.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:55 smithi070 ceph-mon[40691]: pgmap v736: 178 pgs: 5 active+recovering+undersized+remapped, 3 active+recovery_wait+undersized+degraded+remapped, 2 active+remapped+backfill_toofull, 168 active+clean; 5.1 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.2 KiB/s rd, 58 MiB/s wr, 930 op/s; 13884/249963 objects degraded (5.554%); 24087/249963 objects misplaced (9.636%); 24 MiB/s, 386 objects/s recovering 2024-09-16T09:38:55.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:55 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:38:55.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:55 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:38:55.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:55 smithi070 ceph-mon[40691]: 10.10 scrub starts 2024-09-16T09:38:55.757 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:38:55.793 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:55.794 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:55.794 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:38:55.895 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:38:55.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:55 smithi084 ceph-mon[32629]: pgmap v736: 178 pgs: 5 active+recovering+undersized+remapped, 3 active+recovery_wait+undersized+degraded+remapped, 2 active+remapped+backfill_toofull, 168 active+clean; 5.1 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.2 KiB/s rd, 58 MiB/s wr, 930 op/s; 13884/249963 objects degraded (5.554%); 24087/249963 objects misplaced (9.636%); 24 MiB/s, 386 objects/s recovering 2024-09-16T09:38:55.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:55 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:38:55.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:55 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:38:55.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:55 smithi084 ceph-mon[32629]: 10.10 scrub starts 2024-09-16T09:38:55.939 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:55.940 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:55.940 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:38:56.041 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:38:56.071 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:56.072 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:56.072 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:38:56.173 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:38:56.212 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:56.213 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:56.213 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:38:56.314 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:38:56.343 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.5 dump_blocked_ops 2024-09-16T09:38:56.362 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:56.362 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:56.362 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:38:56.463 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:38:56.504 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:56.505 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:56.506 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:38:56.553 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:2024-09-16T09:38:56.553284+0000 min lat: 0.00121703 max lat: 5.46054 avg lat: 0.0110742 2024-09-16T09:38:56.554 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-16T09:38:56.554 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 60 16 86696 86680 90.2597 65.5625 0.0130803 0.0110742 2024-09-16T09:38:56.606 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:38:56.642 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:56.643 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:56.644 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:38:56.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:56 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:38:56.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:56 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:38:56.744 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:38:56.768 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:56 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:38:56.768 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:56 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:38:56.768 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:56 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:38:56.768 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:56 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:38:56.784 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:56.784 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:56.785 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:38:56.786 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:38:56.886 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:38:56.922 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:56.923 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:56.923 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:38:57.024 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:38:57.061 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:57.061 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:57.062 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:38:57.163 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:38:57.207 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:57.207 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:57.208 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:38:57.309 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:38:57.344 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:57.345 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:57.345 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:38:57.446 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:38:57.485 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:57.486 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:57.486 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:38:57.524 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:57 smithi084 ceph-mon[32629]: pgmap v738: 178 pgs: 5 active+recovering+undersized+remapped, 3 active+recovery_wait+undersized+degraded+remapped, 2 active+remapped+backfill_toofull, 168 active+clean; 5.1 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 853 B/s rd, 44 MiB/s wr, 703 op/s; 13884/252195 objects degraded (5.505%); 24190/252195 objects misplaced (9.592%); 20 MiB/s, 314 objects/s recovering 2024-09-16T09:38:57.525 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:57 smithi084 ceph-mon[32629]: Health check update: Degraded data redundancy: 13884/252195 objects degraded (5.505%), 3 pgs degraded (PG_DEGRADED) 2024-09-16T09:38:57.525 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:57 smithi084 ceph-mon[32629]: osdmap e407: 8 total, 8 up, 8 in 2024-09-16T09:38:57.526 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-09-16T09:38:57.526 INFO:teuthology.orchestra.run.smithi084.stdout: "ops": [], 2024-09-16T09:38:57.526 INFO:teuthology.orchestra.run.smithi084.stdout: "complaint_time": 30, 2024-09-16T09:38:57.526 INFO:teuthology.orchestra.run.smithi084.stdout: "num_blocked_ops": 0 2024-09-16T09:38:57.526 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-09-16T09:38:57.554 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 61 15 87726 87711 89.8364 64.4375 0.00581043 0.011123 2024-09-16T09:38:57.587 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:38:57.623 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:57.624 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:57.624 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:38:57.725 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:38:57.768 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:57.769 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:57.769 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:38:57.870 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:38:57.902 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:57.903 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:57.903 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:38:57.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:57 smithi070 ceph-mon[30388]: pgmap v738: 178 pgs: 5 active+recovering+undersized+remapped, 3 active+recovery_wait+undersized+degraded+remapped, 2 active+remapped+backfill_toofull, 168 active+clean; 5.1 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 853 B/s rd, 44 MiB/s wr, 703 op/s; 13884/252195 objects degraded (5.505%); 24190/252195 objects misplaced (9.592%); 20 MiB/s, 314 objects/s recovering 2024-09-16T09:38:57.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:57 smithi070 ceph-mon[30388]: Health check update: Degraded data redundancy: 13884/252195 objects degraded (5.505%), 3 pgs degraded (PG_DEGRADED) 2024-09-16T09:38:57.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:57 smithi070 ceph-mon[30388]: osdmap e407: 8 total, 8 up, 8 in 2024-09-16T09:38:57.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:57 smithi070 ceph-mon[40691]: pgmap v738: 178 pgs: 5 active+recovering+undersized+remapped, 3 active+recovery_wait+undersized+degraded+remapped, 2 active+remapped+backfill_toofull, 168 active+clean; 5.1 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 853 B/s rd, 44 MiB/s wr, 703 op/s; 13884/252195 objects degraded (5.505%); 24190/252195 objects misplaced (9.592%); 20 MiB/s, 314 objects/s recovering 2024-09-16T09:38:57.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:57 smithi070 ceph-mon[40691]: Health check update: Degraded data redundancy: 13884/252195 objects degraded (5.505%), 3 pgs degraded (PG_DEGRADED) 2024-09-16T09:38:57.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:57 smithi070 ceph-mon[40691]: osdmap e407: 8 total, 8 up, 8 in 2024-09-16T09:38:58.004 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:38:58.024 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:58.025 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:58.025 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:38:58.126 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:38:58.165 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:58.165 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:58.166 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:38:58.266 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:38:58.316 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:58.316 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:58.317 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:38:58.417 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:38:58.446 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:58.446 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:58.447 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:38:58.547 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:38:58.553 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 62 16 88778 88762 89.4469 65.6875 0.0140268 0.0111746 2024-09-16T09:38:58.567 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.5 dump_historic_ops 2024-09-16T09:38:58.586 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:58.587 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:58.588 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:38:58.688 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:38:58.734 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:58.735 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:58.735 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:38:58.836 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:38:58.877 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:58.878 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:58.878 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:38:58.944 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:38:58.979 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:38:59.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:59.020 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:59.020 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:38:59.121 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:38:59.157 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:59.157 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:59.158 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:38:59.258 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:38:59.298 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:59.298 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:38:59.298 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:38:59.399 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:38:59.438 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:59.439 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:59.439 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:38:59.540 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:38:59.554 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 63 16 89839 89823 89.0797 66.3125 0.0124848 0.0112204 2024-09-16T09:38:59.579 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:59.579 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:59.580 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:38:59.580 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-09-16T09:38:59.581 INFO:teuthology.orchestra.run.smithi084.stdout: "size": 20, 2024-09-16T09:38:59.581 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 600, 2024-09-16T09:38:59.581 INFO:teuthology.orchestra.run.smithi084.stdout: "ops": [ 2024-09-16T09:38:59.581 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.581 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.25064.0:106991 8.8 8:1dd7c7a4:::benchmark_data_smithi070_107919_object106990:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e186)", 2024-09-16T09:38:59.581 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:31:08.373667+0000", 2024-09-16T09:38:59.581 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 471.20103376499998, 2024-09-16T09:38:59.581 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1058546549999999, 2024-09-16T09:38:59.581 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:38:59.581 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:38:59.581 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:38:59.581 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:38:59.582 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.25064", 2024-09-16T09:38:59.582 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/2578256439", 2024-09-16T09:38:59.582 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 106991 2024-09-16T09:38:59.582 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.582 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:38:59.582 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.582 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:38:59.582 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:08.373667+0000", 2024-09-16T09:38:59.582 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:38:59.582 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.582 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.582 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:38:59.582 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:08.373667+0000", 2024-09-16T09:38:59.582 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:38:59.583 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.583 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.583 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:38:59.583 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:08.373667+0000", 2024-09-16T09:38:59.583 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.47e-07 2024-09-16T09:38:59.583 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.583 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.583 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:38:59.583 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:08.373710+0000", 2024-09-16T09:38:59.583 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.2564000000000001e-05 2024-09-16T09:38:59.583 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.583 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.583 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:38:59.583 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:08.373721+0000", 2024-09-16T09:38:59.584 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1045e-05 2024-09-16T09:38:59.584 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.584 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.584 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:38:59.584 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:08.373727+0000", 2024-09-16T09:38:59.584 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.5160000000000002e-06 2024-09-16T09:38:59.584 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.584 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.584 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:38:59.584 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:08.373779+0000", 2024-09-16T09:38:59.584 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.2237000000000001e-05 2024-09-16T09:38:59.584 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.584 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.585 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:38:59.585 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:08.373786+0000", 2024-09-16T09:38:59.585 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.1389999999999998e-06 2024-09-16T09:38:59.585 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.585 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.585 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:38:59.585 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:09.430374+0000", 2024-09-16T09:38:59.585 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0565881580000001 2024-09-16T09:38:59.585 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.585 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.585 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:38:59.585 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:09.430493+0000", 2024-09-16T09:38:59.585 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000118845 2024-09-16T09:38:59.586 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.586 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.586 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 3,7", 2024-09-16T09:38:59.586 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:09.430648+0000", 2024-09-16T09:38:59.586 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000155477 2024-09-16T09:38:59.586 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.586 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.586 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:38:59.586 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:09.436479+0000", 2024-09-16T09:38:59.586 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0058305120000000004 2024-09-16T09:38:59.586 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.586 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.586 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:38:59.587 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:09.436530+0000", 2024-09-16T09:38:59.587 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.1523999999999997e-05 2024-09-16T09:38:59.587 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.587 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.587 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:38:59.587 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:09.476247+0000", 2024-09-16T09:38:59.587 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.039716695000000003 2024-09-16T09:38:59.587 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.587 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.587 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:38:59.587 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:09.476349+0000", 2024-09-16T09:38:59.587 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000102117 2024-09-16T09:38:59.588 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.588 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.588 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:38:59.588 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:31:09.479521+0000", 2024-09-16T09:38:59.588 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0031719790000000001 2024-09-16T09:38:59.588 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:38:59.588 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:38:59.588 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:38:59.588 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.588 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.588 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:166950 9.16 9:6a3a0c76:::benchmark_data_smithi070_137000_object45239:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e298)", 2024-09-16T09:38:59.588 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:35:56.816107+0000", 2024-09-16T09:38:59.588 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 182.758593733, 2024-09-16T09:38:59.588 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0304473940000001, 2024-09-16T09:38:59.589 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:38:59.589 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:38:59.589 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "reached pg", 2024-09-16T09:38:59.589 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:38:59.589 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:38:59.589 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:38:59.589 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 166950 2024-09-16T09:38:59.589 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.589 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:38:59.589 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.589 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:38:59.589 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:35:56.816107+0000", 2024-09-16T09:38:59.589 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:38:59.589 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.590 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.590 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:38:59.590 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:35:56.816107+0000", 2024-09-16T09:38:59.590 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:38:59.590 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.590 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.590 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:38:59.590 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:35:56.816108+0000", 2024-09-16T09:38:59.590 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0330000000000001e-06 2024-09-16T09:38:59.590 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.590 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.590 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:38:59.590 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:35:56.816111+0000", 2024-09-16T09:38:59.591 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.4359999999999998e-06 2024-09-16T09:38:59.591 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.591 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.591 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:38:59.591 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:35:56.816119+0000", 2024-09-16T09:38:59.591 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.2199999999999992e-06 2024-09-16T09:38:59.591 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.591 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.591 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:38:59.591 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:35:56.816125+0000", 2024-09-16T09:38:59.591 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.8239999999999996e-06 2024-09-16T09:38:59.591 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.591 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.591 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:38:59.592 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:35:56.816163+0000", 2024-09-16T09:38:59.592 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.7849999999999998e-05 2024-09-16T09:38:59.592 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.592 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.592 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:38:59.592 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:35:56.816169+0000", 2024-09-16T09:38:59.592 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.7749999999999998e-06 2024-09-16T09:38:59.592 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.592 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.592 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:38:59.592 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:35:57.839609+0000", 2024-09-16T09:38:59.592 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0234398039999999 2024-09-16T09:38:59.592 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.593 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.593 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for missing object", 2024-09-16T09:38:59.593 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:35:57.839678+0000", 2024-09-16T09:38:59.593 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.9789000000000006e-05 2024-09-16T09:38:59.593 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.593 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.593 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:38:59.593 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:35:57.846509+0000", 2024-09-16T09:38:59.593 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0068310080000000004 2024-09-16T09:38:59.593 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.593 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.593 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:38:59.593 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:35:57.846554+0000", 2024-09-16T09:38:59.593 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.4654999999999999e-05 2024-09-16T09:38:59.594 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:38:59.594 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:38:59.594 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:38:59.594 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.594 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.594 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:166957 9.16 9:6be820a7:::benchmark_data_smithi070_137000_object45246:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e298)", 2024-09-16T09:38:59.594 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:35:56.816134+0000", 2024-09-16T09:38:59.594 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 182.75856594999999, 2024-09-16T09:38:59.594 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0415438909999999, 2024-09-16T09:38:59.594 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:38:59.594 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:38:59.594 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "reached pg", 2024-09-16T09:38:59.594 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:38:59.594 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:38:59.595 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:38:59.595 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 166957 2024-09-16T09:38:59.595 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.595 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:38:59.595 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.595 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:38:59.595 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:35:56.816134+0000", 2024-09-16T09:38:59.595 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:38:59.595 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.595 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.595 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:38:59.595 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:35:56.816134+0000", 2024-09-16T09:38:59.595 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:38:59.595 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.596 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.596 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:38:59.596 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:35:56.816135+0000", 2024-09-16T09:38:59.596 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.6899999999999999e-07 2024-09-16T09:38:59.596 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.596 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.596 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:38:59.596 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:35:56.816136+0000", 2024-09-16T09:38:59.596 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.2759999999999999e-06 2024-09-16T09:38:59.596 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.596 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.596 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:38:59.596 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:35:56.816141+0000", 2024-09-16T09:38:59.597 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.1100000000000002e-06 2024-09-16T09:38:59.597 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.597 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.597 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:38:59.597 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:35:56.816144+0000", 2024-09-16T09:38:59.597 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.8830000000000002e-06 2024-09-16T09:38:59.597 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.597 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.597 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:38:59.597 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:35:56.816189+0000", 2024-09-16T09:38:59.597 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.5108999999999998e-05 2024-09-16T09:38:59.597 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.597 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.598 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:38:59.598 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:35:56.816195+0000", 2024-09-16T09:38:59.598 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.9490000000000001e-06 2024-09-16T09:38:59.598 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.598 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.598 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:38:59.598 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:35:57.839854+0000", 2024-09-16T09:38:59.598 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0236591390000001 2024-09-16T09:38:59.598 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.598 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.598 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for missing object", 2024-09-16T09:38:59.598 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:35:57.839913+0000", 2024-09-16T09:38:59.598 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.8579999999999998e-05 2024-09-16T09:38:59.598 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.598 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.599 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:38:59.599 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:35:57.857600+0000", 2024-09-16T09:38:59.599 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.017687015 2024-09-16T09:38:59.599 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.599 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.599 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:38:59.599 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:35:57.857678+0000", 2024-09-16T09:38:59.599 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.8560999999999995e-05 2024-09-16T09:38:59.599 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:38:59.599 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:38:59.599 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:38:59.599 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.600 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.600 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:211546 9.6 9:6925415d:::benchmark_data_smithi070_137000_object89835:head [delete] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e330)", 2024-09-16T09:38:59.600 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:36:50.314875+0000", 2024-09-16T09:38:59.600 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 129.259825511, 2024-09-16T09:38:59.600 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0027659179999999, 2024-09-16T09:38:59.600 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:38:59.600 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:38:59.600 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:38:59.600 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:38:59.600 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:38:59.600 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:38:59.600 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 211546 2024-09-16T09:38:59.600 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.600 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:38:59.601 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.601 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:38:59.601 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.314875+0000", 2024-09-16T09:38:59.601 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:38:59.601 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.601 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.601 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:38:59.601 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.314875+0000", 2024-09-16T09:38:59.601 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:38:59.601 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.601 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.601 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:38:59.601 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.314876+0000", 2024-09-16T09:38:59.602 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.54e-07 2024-09-16T09:38:59.602 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.602 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.602 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:38:59.602 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.314879+0000", 2024-09-16T09:38:59.602 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.715e-06 2024-09-16T09:38:59.602 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.602 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.602 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:38:59.602 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.314888+0000", 2024-09-16T09:38:59.602 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.0229999999999998e-06 2024-09-16T09:38:59.602 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.603 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.603 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:38:59.603 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.314895+0000", 2024-09-16T09:38:59.603 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.6089999999999997e-06 2024-09-16T09:38:59.603 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.603 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.603 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:38:59.603 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.387332+0000", 2024-09-16T09:38:59.603 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.072437018000000006 2024-09-16T09:38:59.603 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.603 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.603 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:38:59.603 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.387341+0000", 2024-09-16T09:38:59.604 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.8589999999999994e-06 2024-09-16T09:38:59.604 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.604 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.604 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:38:59.604 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.316557+0000", 2024-09-16T09:38:59.604 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.92921574399999995 2024-09-16T09:38:59.604 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.604 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.604 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:38:59.604 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.316669+0000", 2024-09-16T09:38:59.604 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00011194 2024-09-16T09:38:59.604 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.604 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.605 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 1,2", 2024-09-16T09:38:59.605 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.316738+0000", 2024-09-16T09:38:59.605 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.9770000000000005e-05 2024-09-16T09:38:59.605 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.605 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.605 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:38:59.605 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.317506+0000", 2024-09-16T09:38:59.605 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00076750500000000003 2024-09-16T09:38:59.605 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.605 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.605 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:38:59.605 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.317537+0000", 2024-09-16T09:38:59.605 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.1285999999999998e-05 2024-09-16T09:38:59.606 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.606 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.606 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:38:59.606 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.317584+0000", 2024-09-16T09:38:59.606 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.7273e-05 2024-09-16T09:38:59.606 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.606 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.606 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:38:59.606 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.317617+0000", 2024-09-16T09:38:59.606 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.2521000000000003e-05 2024-09-16T09:38:59.606 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.606 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.606 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:38:59.607 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.317641+0000", 2024-09-16T09:38:59.607 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.3801000000000001e-05 2024-09-16T09:38:59.607 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:38:59.607 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:38:59.607 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:38:59.607 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.607 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.607 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:211689 9.6 9:6d597b0f:::benchmark_data_smithi070_137000_object89978:head [delete] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e330)", 2024-09-16T09:38:59.607 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:36:50.314951+0000", 2024-09-16T09:38:59.607 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 129.25974936099999, 2024-09-16T09:38:59.607 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0036079410000001, 2024-09-16T09:38:59.607 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:38:59.607 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:38:59.608 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:38:59.608 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:38:59.608 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:38:59.608 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:38:59.608 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 211689 2024-09-16T09:38:59.608 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.608 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:38:59.608 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.608 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:38:59.608 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.314951+0000", 2024-09-16T09:38:59.608 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:38:59.608 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.608 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.609 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:38:59.609 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.314951+0000", 2024-09-16T09:38:59.609 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:38:59.609 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.609 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.609 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:38:59.609 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.314952+0000", 2024-09-16T09:38:59.610 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.0299999999999998e-07 2024-09-16T09:38:59.610 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.610 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.610 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:38:59.610 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.314955+0000", 2024-09-16T09:38:59.610 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.2289999999999999e-06 2024-09-16T09:38:59.610 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.610 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.610 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:38:59.611 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.314963+0000", 2024-09-16T09:38:59.611 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.4749999999999993e-06 2024-09-16T09:38:59.611 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.611 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.611 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:38:59.611 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.314969+0000", 2024-09-16T09:38:59.611 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.8939999999999998e-06 2024-09-16T09:38:59.611 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.611 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.611 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:38:59.611 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.387519+0000", 2024-09-16T09:38:59.611 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.072549570999999993 2024-09-16T09:38:59.612 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.612 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.612 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:38:59.612 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.387525+0000", 2024-09-16T09:38:59.612 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.1349999999999997e-06 2024-09-16T09:38:59.612 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.612 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.612 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:38:59.612 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.317205+0000", 2024-09-16T09:38:59.612 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.92968029500000005 2024-09-16T09:38:59.612 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.612 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.612 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:38:59.613 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.317272+0000", 2024-09-16T09:38:59.613 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.7082999999999995e-05 2024-09-16T09:38:59.613 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.613 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.613 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 1,2", 2024-09-16T09:38:59.613 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.317343+0000", 2024-09-16T09:38:59.613 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.0680999999999996e-05 2024-09-16T09:38:59.613 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.613 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.613 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:38:59.613 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.318400+0000", 2024-09-16T09:38:59.613 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.001057288 2024-09-16T09:38:59.613 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.614 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.614 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:38:59.614 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.318438+0000", 2024-09-16T09:38:59.614 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.7401999999999999e-05 2024-09-16T09:38:59.614 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.614 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.614 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:38:59.614 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.318485+0000", 2024-09-16T09:38:59.614 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.6829000000000002e-05 2024-09-16T09:38:59.614 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.614 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.614 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:38:59.614 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.318528+0000", 2024-09-16T09:38:59.614 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.3179000000000001e-05 2024-09-16T09:38:59.615 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.615 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.615 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:38:59.615 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.318559+0000", 2024-09-16T09:38:59.615 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.1177000000000003e-05 2024-09-16T09:38:59.615 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:38:59.615 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:38:59.615 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:38:59.615 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.615 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.615 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:211690 9.6 9:69ad5405:::benchmark_data_smithi070_137000_object89979:head [delete] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e330)", 2024-09-16T09:38:59.615 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:36:50.314981+0000", 2024-09-16T09:38:59.615 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 129.259719789, 2024-09-16T09:38:59.616 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.004139173, 2024-09-16T09:38:59.616 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:38:59.616 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:38:59.616 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:38:59.616 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:38:59.616 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:38:59.616 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:38:59.616 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 211690 2024-09-16T09:38:59.616 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.616 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:38:59.616 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.616 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:38:59.617 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.314981+0000", 2024-09-16T09:38:59.617 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:38:59.617 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.617 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.617 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:38:59.617 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.314981+0000", 2024-09-16T09:38:59.617 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:38:59.617 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.617 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.617 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:38:59.617 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.314981+0000", 2024-09-16T09:38:59.617 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.3099999999999998e-07 2024-09-16T09:38:59.617 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.618 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.618 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:38:59.618 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.314983+0000", 2024-09-16T09:38:59.618 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.193e-06 2024-09-16T09:38:59.618 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.618 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.618 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:38:59.618 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.314990+0000", 2024-09-16T09:38:59.618 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.776e-06 2024-09-16T09:38:59.618 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.618 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.618 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:38:59.618 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.314995+0000", 2024-09-16T09:38:59.619 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.1050000000000003e-06 2024-09-16T09:38:59.619 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.619 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.619 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:38:59.619 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.387548+0000", 2024-09-16T09:38:59.619 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.072553138000000003 2024-09-16T09:38:59.619 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.619 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.619 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:38:59.619 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.387553+0000", 2024-09-16T09:38:59.619 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.9139999999999999e-06 2024-09-16T09:38:59.619 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.619 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.620 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:38:59.620 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.317735+0000", 2024-09-16T09:38:59.620 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.93018151000000004 2024-09-16T09:38:59.620 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.620 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.620 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:38:59.620 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.317830+0000", 2024-09-16T09:38:59.620 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.5123000000000004e-05 2024-09-16T09:38:59.620 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.620 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.620 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 1,2", 2024-09-16T09:38:59.620 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.317888+0000", 2024-09-16T09:38:59.620 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.8214999999999997e-05 2024-09-16T09:38:59.620 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.621 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.621 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:38:59.621 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.318568+0000", 2024-09-16T09:38:59.621 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00068012599999999997 2024-09-16T09:38:59.621 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.621 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.621 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:38:59.621 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.319010+0000", 2024-09-16T09:38:59.621 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00044167499999999999 2024-09-16T09:38:59.621 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.621 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.621 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:38:59.621 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.319058+0000", 2024-09-16T09:38:59.621 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.8050000000000002e-05 2024-09-16T09:38:59.622 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.622 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.622 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:38:59.622 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.319085+0000", 2024-09-16T09:38:59.622 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.7209e-05 2024-09-16T09:38:59.622 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.622 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.622 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:38:59.622 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.319120+0000", 2024-09-16T09:38:59.622 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.4708e-05 2024-09-16T09:38:59.622 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:38:59.622 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:38:59.622 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:38:59.622 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.623 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.623 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:211748 9.6 9:6abf73bd:::benchmark_data_smithi070_137000_object90037:head [delete] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e330)", 2024-09-16T09:38:59.623 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:36:50.315053+0000", 2024-09-16T09:38:59.623 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 129.25964686200001, 2024-09-16T09:38:59.623 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.004206473, 2024-09-16T09:38:59.623 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:38:59.623 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:38:59.623 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:38:59.623 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:38:59.623 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:38:59.623 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:38:59.623 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 211748 2024-09-16T09:38:59.623 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.623 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:38:59.624 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.624 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:38:59.624 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.315053+0000", 2024-09-16T09:38:59.624 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:38:59.624 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.624 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.624 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:38:59.624 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.315053+0000", 2024-09-16T09:38:59.624 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:38:59.624 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.624 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.624 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:38:59.625 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.315054+0000", 2024-09-16T09:38:59.625 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.3799999999999997e-07 2024-09-16T09:38:59.625 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.625 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.625 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:38:59.625 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.315057+0000", 2024-09-16T09:38:59.625 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.5919999999999999e-06 2024-09-16T09:38:59.625 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.625 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.625 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:38:59.625 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.315064+0000", 2024-09-16T09:38:59.625 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.3470000000000001e-06 2024-09-16T09:38:59.625 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.626 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.626 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:38:59.626 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.315070+0000", 2024-09-16T09:38:59.626 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.7300000000000002e-06 2024-09-16T09:38:59.626 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.626 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.626 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:38:59.626 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.387582+0000", 2024-09-16T09:38:59.626 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.072512338999999995 2024-09-16T09:38:59.626 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.626 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.626 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:38:59.626 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.387586+0000", 2024-09-16T09:38:59.626 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.0380000000000001e-06 2024-09-16T09:38:59.627 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.627 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.627 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:38:59.627 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.318053+0000", 2024-09-16T09:38:59.627 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.93046725799999996 2024-09-16T09:38:59.627 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.627 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.627 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:38:59.627 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.318134+0000", 2024-09-16T09:38:59.627 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.0827000000000004e-05 2024-09-16T09:38:59.627 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.627 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.627 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 1,2", 2024-09-16T09:38:59.627 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.318224+0000", 2024-09-16T09:38:59.628 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.9433999999999998e-05 2024-09-16T09:38:59.628 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.628 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.628 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:38:59.628 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.319163+0000", 2024-09-16T09:38:59.628 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00093904299999999997 2024-09-16T09:38:59.628 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.628 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.628 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:38:59.628 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.319170+0000", 2024-09-16T09:38:59.628 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.0169999999999999e-06 2024-09-16T09:38:59.628 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.628 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.629 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:38:59.629 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.319195+0000", 2024-09-16T09:38:59.629 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.4745000000000001e-05 2024-09-16T09:38:59.629 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.629 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.629 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:38:59.629 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.319229+0000", 2024-09-16T09:38:59.629 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.4106999999999997e-05 2024-09-16T09:38:59.629 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.629 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.629 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:38:59.629 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.319260+0000", 2024-09-16T09:38:59.629 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.1358e-05 2024-09-16T09:38:59.630 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:38:59.630 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:38:59.630 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:38:59.630 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.630 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.630 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:211783 9.6 9:6ab64b11:::benchmark_data_smithi070_137000_object90072:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e330)", 2024-09-16T09:38:59.630 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:36:50.315097+0000", 2024-09-16T09:38:59.630 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 129.259603312, 2024-09-16T09:38:59.630 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.005736382, 2024-09-16T09:38:59.630 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:38:59.630 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:38:59.630 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:38:59.630 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:38:59.631 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:38:59.631 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:38:59.631 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 211783 2024-09-16T09:38:59.631 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.631 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:38:59.631 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.631 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:38:59.631 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.315097+0000", 2024-09-16T09:38:59.631 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:38:59.631 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.632 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.632 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:38:59.632 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.315097+0000", 2024-09-16T09:38:59.632 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:38:59.632 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.632 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.632 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:38:59.632 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.315098+0000", 2024-09-16T09:38:59.632 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.6899999999999997e-07 2024-09-16T09:38:59.632 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.632 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.632 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:38:59.632 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.315100+0000", 2024-09-16T09:38:59.633 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.819e-06 2024-09-16T09:38:59.633 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.633 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.633 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:38:59.633 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.315108+0000", 2024-09-16T09:38:59.633 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.114e-06 2024-09-16T09:38:59.633 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.633 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.633 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:38:59.633 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.315113+0000", 2024-09-16T09:38:59.633 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.2499999999999997e-06 2024-09-16T09:38:59.633 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.633 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.634 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:38:59.634 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.387607+0000", 2024-09-16T09:38:59.634 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.072493713000000001 2024-09-16T09:38:59.634 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.634 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.634 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:38:59.634 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.387610+0000", 2024-09-16T09:38:59.634 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.631e-06 2024-09-16T09:38:59.634 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.634 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.634 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:38:59.634 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.318587+0000", 2024-09-16T09:38:59.634 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.93097701899999996 2024-09-16T09:38:59.634 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.634 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.635 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:38:59.635 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.318677+0000", 2024-09-16T09:38:59.635 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.0105999999999994e-05 2024-09-16T09:38:59.635 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.635 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.635 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 1,2", 2024-09-16T09:38:59.635 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.318780+0000", 2024-09-16T09:38:59.635 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000102898 2024-09-16T09:38:59.635 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.635 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.635 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:38:59.635 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.320624+0000", 2024-09-16T09:38:59.635 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0018438269999999999 2024-09-16T09:38:59.635 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.635 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.636 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:38:59.636 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.320660+0000", 2024-09-16T09:38:59.636 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.6395e-05 2024-09-16T09:38:59.636 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.636 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.636 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:38:59.636 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.320754+0000", 2024-09-16T09:38:59.636 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.3908999999999998e-05 2024-09-16T09:38:59.636 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.636 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.636 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:38:59.636 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.320801+0000", 2024-09-16T09:38:59.636 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.6739000000000001e-05 2024-09-16T09:38:59.636 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.637 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.637 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:38:59.637 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.320833+0000", 2024-09-16T09:38:59.637 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.2393e-05 2024-09-16T09:38:59.637 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:38:59.637 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:38:59.637 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:38:59.637 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.637 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.637 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:211596 9.6 9:631feafe:::benchmark_data_smithi070_137000_object89885:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e330)", 2024-09-16T09:38:59.637 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:36:50.352998+0000", 2024-09-16T09:38:59.637 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 129.221702698, 2024-09-16T09:38:59.637 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.97116658300000003, 2024-09-16T09:38:59.637 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:38:59.638 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:38:59.638 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:38:59.638 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:38:59.638 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:38:59.638 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:38:59.638 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 211596 2024-09-16T09:38:59.638 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.638 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:38:59.638 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.638 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:38:59.638 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.352998+0000", 2024-09-16T09:38:59.638 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:38:59.638 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.638 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.639 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:38:59.639 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.352998+0000", 2024-09-16T09:38:59.639 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:38:59.639 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.639 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.639 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:38:59.639 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.352998+0000", 2024-09-16T09:38:59.639 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.6300000000000001e-07 2024-09-16T09:38:59.639 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.639 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.639 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:38:59.639 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.352999+0000", 2024-09-16T09:38:59.639 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.263e-06 2024-09-16T09:38:59.640 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.640 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.640 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:38:59.640 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.353003+0000", 2024-09-16T09:38:59.640 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.625e-06 2024-09-16T09:38:59.640 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.640 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.640 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:38:59.640 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.353006+0000", 2024-09-16T09:38:59.640 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.875e-06 2024-09-16T09:38:59.640 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.640 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.640 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:38:59.641 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.387683+0000", 2024-09-16T09:38:59.641 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.034677463999999998 2024-09-16T09:38:59.641 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.641 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.641 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:38:59.641 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.387687+0000", 2024-09-16T09:38:59.641 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.3890000000000001e-06 2024-09-16T09:38:59.641 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.641 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.641 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:38:59.641 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.320217+0000", 2024-09-16T09:38:59.641 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.932530847 2024-09-16T09:38:59.641 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.641 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.642 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:38:59.642 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.320324+0000", 2024-09-16T09:38:59.642 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00010629900000000001 2024-09-16T09:38:59.642 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.642 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.642 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 1,2", 2024-09-16T09:38:59.642 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.320419+0000", 2024-09-16T09:38:59.642 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.4840000000000007e-05 2024-09-16T09:38:59.642 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.642 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.642 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:38:59.642 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.321681+0000", 2024-09-16T09:38:59.642 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0012626359999999999 2024-09-16T09:38:59.643 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.643 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.643 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:38:59.643 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.322444+0000", 2024-09-16T09:38:59.643 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00076252999999999998 2024-09-16T09:38:59.643 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.643 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.643 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:38:59.643 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.324085+0000", 2024-09-16T09:38:59.643 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00164153 2024-09-16T09:38:59.643 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.643 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.643 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:38:59.643 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.324129+0000", 2024-09-16T09:38:59.644 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.3405000000000002e-05 2024-09-16T09:38:59.644 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.644 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.644 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:38:59.644 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.324164+0000", 2024-09-16T09:38:59.644 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.5617000000000001e-05 2024-09-16T09:38:59.644 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:38:59.644 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:38:59.644 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:38:59.644 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.644 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.644 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:211647 9.6 9:61eba98e:::benchmark_data_smithi070_137000_object89936:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e330)", 2024-09-16T09:38:59.645 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:36:50.353009+0000", 2024-09-16T09:38:59.645 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 129.22169099300001, 2024-09-16T09:38:59.645 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.97128613799999997, 2024-09-16T09:38:59.645 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:38:59.645 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:38:59.645 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:38:59.645 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:38:59.645 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:38:59.645 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:38:59.645 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 211647 2024-09-16T09:38:59.645 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.645 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:38:59.645 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.646 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:38:59.646 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.353009+0000", 2024-09-16T09:38:59.646 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:38:59.646 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.646 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.646 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:38:59.646 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.353009+0000", 2024-09-16T09:38:59.646 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:38:59.646 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.646 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.646 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:38:59.646 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.353010+0000", 2024-09-16T09:38:59.646 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.0599999999999999e-07 2024-09-16T09:38:59.646 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.646 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.647 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:38:59.647 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.353011+0000", 2024-09-16T09:38:59.647 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0899999999999999e-06 2024-09-16T09:38:59.647 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.647 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.647 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:38:59.647 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.353015+0000", 2024-09-16T09:38:59.647 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.284e-06 2024-09-16T09:38:59.647 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.647 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.647 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:38:59.647 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.353018+0000", 2024-09-16T09:38:59.647 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.6579999999999998e-06 2024-09-16T09:38:59.648 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.648 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.648 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:38:59.648 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.387708+0000", 2024-09-16T09:38:59.648 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.034690029999999997 2024-09-16T09:38:59.648 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.648 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.648 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:38:59.648 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.387712+0000", 2024-09-16T09:38:59.648 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.1969999999999998e-06 2024-09-16T09:38:59.648 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.648 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.648 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:38:59.648 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.320932+0000", 2024-09-16T09:38:59.649 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.93322063700000002 2024-09-16T09:38:59.649 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.649 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.649 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:38:59.649 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.321028+0000", 2024-09-16T09:38:59.649 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.5895000000000003e-05 2024-09-16T09:38:59.649 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.649 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.649 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 1,2", 2024-09-16T09:38:59.649 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.321125+0000", 2024-09-16T09:38:59.649 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.6508999999999994e-05 2024-09-16T09:38:59.649 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.649 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.650 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:38:59.650 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.322070+0000", 2024-09-16T09:38:59.650 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00094508799999999996 2024-09-16T09:38:59.650 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.650 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.650 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:38:59.650 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.324053+0000", 2024-09-16T09:38:59.650 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.001983145 2024-09-16T09:38:59.650 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.650 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.650 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:38:59.650 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.324227+0000", 2024-09-16T09:38:59.650 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00017394099999999999 2024-09-16T09:38:59.650 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.650 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.651 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:38:59.651 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.324267+0000", 2024-09-16T09:38:59.651 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.9628000000000001e-05 2024-09-16T09:38:59.651 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.651 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.651 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:38:59.651 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.324295+0000", 2024-09-16T09:38:59.651 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.883e-05 2024-09-16T09:38:59.651 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:38:59.651 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:38:59.651 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:38:59.651 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.651 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.651 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:211676 9.6 9:652ffcc5:::benchmark_data_smithi070_137000_object89965:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e330)", 2024-09-16T09:38:59.651 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:36:50.353021+0000", 2024-09-16T09:38:59.652 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 129.221679706, 2024-09-16T09:38:59.652 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.97154827700000002, 2024-09-16T09:38:59.652 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:38:59.652 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:38:59.652 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:38:59.652 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:38:59.652 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:38:59.652 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:38:59.652 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 211676 2024-09-16T09:38:59.652 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.652 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:38:59.652 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.653 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:38:59.654 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.353021+0000", 2024-09-16T09:38:59.654 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:38:59.654 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.654 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.654 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:38:59.654 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.353021+0000", 2024-09-16T09:38:59.654 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:38:59.654 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.654 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.654 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:38:59.654 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.353021+0000", 2024-09-16T09:38:59.654 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.03e-07 2024-09-16T09:38:59.654 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.655 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.655 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:38:59.655 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.353022+0000", 2024-09-16T09:38:59.655 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.850000000000001e-07 2024-09-16T09:38:59.655 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.655 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.655 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:38:59.655 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.353026+0000", 2024-09-16T09:38:59.655 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.9110000000000003e-06 2024-09-16T09:38:59.655 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.655 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.655 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:38:59.655 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.353028+0000", 2024-09-16T09:38:59.656 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.6029999999999999e-06 2024-09-16T09:38:59.656 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.656 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.656 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:38:59.656 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.387733+0000", 2024-09-16T09:38:59.656 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.034704735 2024-09-16T09:38:59.656 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.656 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.656 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:38:59.656 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.387738+0000", 2024-09-16T09:38:59.656 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.763e-06 2024-09-16T09:38:59.656 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.656 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.657 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:38:59.657 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.321328+0000", 2024-09-16T09:38:59.657 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.93359050200000004 2024-09-16T09:38:59.657 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.657 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.657 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:38:59.657 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.321423+0000", 2024-09-16T09:38:59.657 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.4705999999999998e-05 2024-09-16T09:38:59.657 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.657 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.657 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 1,2", 2024-09-16T09:38:59.657 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.321519+0000", 2024-09-16T09:38:59.657 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.5620999999999998e-05 2024-09-16T09:38:59.658 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.658 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.658 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:38:59.658 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.322838+0000", 2024-09-16T09:38:59.658 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0013193619999999999 2024-09-16T09:38:59.658 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.658 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.658 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:38:59.658 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.324196+0000", 2024-09-16T09:38:59.658 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.001357893 2024-09-16T09:38:59.658 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.658 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.658 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:38:59.658 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.324461+0000", 2024-09-16T09:38:59.659 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00026498 2024-09-16T09:38:59.659 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.659 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.659 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:38:59.659 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.324537+0000", 2024-09-16T09:38:59.659 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.6240000000000002e-05 2024-09-16T09:38:59.659 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.659 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.659 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:38:59.659 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.324569+0000", 2024-09-16T09:38:59.659 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.1773000000000002e-05 2024-09-16T09:38:59.659 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:38:59.659 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:38:59.659 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:38:59.659 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.659 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.660 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:211720 9.6 9:65d5ca2f:::benchmark_data_smithi070_137000_object90009:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e330)", 2024-09-16T09:38:59.660 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:36:50.353031+0000", 2024-09-16T09:38:59.660 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 129.22166898399999, 2024-09-16T09:38:59.660 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.97262685400000004, 2024-09-16T09:38:59.660 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:38:59.660 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:38:59.660 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:38:59.660 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:38:59.660 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:38:59.660 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:38:59.660 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 211720 2024-09-16T09:38:59.660 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.660 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:38:59.660 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.661 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:38:59.661 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.353031+0000", 2024-09-16T09:38:59.661 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:38:59.661 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.661 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.661 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:38:59.661 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.353031+0000", 2024-09-16T09:38:59.661 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:38:59.661 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.661 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.661 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:38:59.661 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.353032+0000", 2024-09-16T09:38:59.661 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.2100000000000001e-07 2024-09-16T09:38:59.661 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.662 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.662 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:38:59.662 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.353033+0000", 2024-09-16T09:38:59.662 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0079999999999999e-06 2024-09-16T09:38:59.662 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.662 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.662 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:38:59.662 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.353037+0000", 2024-09-16T09:38:59.662 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.9190000000000001e-06 2024-09-16T09:38:59.662 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.662 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.662 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:38:59.662 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.353039+0000", 2024-09-16T09:38:59.662 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.5809999999999999e-06 2024-09-16T09:38:59.663 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.663 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.663 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:38:59.663 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.387758+0000", 2024-09-16T09:38:59.663 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.034718515999999998 2024-09-16T09:38:59.663 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.663 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.663 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:38:59.663 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.387761+0000", 2024-09-16T09:38:59.663 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.5810000000000001e-06 2024-09-16T09:38:59.663 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.663 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.663 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:38:59.663 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.321706+0000", 2024-09-16T09:38:59.663 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.933945052 2024-09-16T09:38:59.663 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.664 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.664 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:38:59.664 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.321797+0000", 2024-09-16T09:38:59.664 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.0968e-05 2024-09-16T09:38:59.664 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.664 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.664 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 1,2", 2024-09-16T09:38:59.664 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.321888+0000", 2024-09-16T09:38:59.664 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.1170999999999998e-05 2024-09-16T09:38:59.664 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.664 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.664 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:38:59.664 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.323252+0000", 2024-09-16T09:38:59.664 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0013631870000000001 2024-09-16T09:38:59.664 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.665 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.665 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:38:59.665 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.324326+0000", 2024-09-16T09:38:59.665 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.001073973 2024-09-16T09:38:59.665 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.665 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.665 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:38:59.665 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.324605+0000", 2024-09-16T09:38:59.665 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00027895600000000002 2024-09-16T09:38:59.665 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.665 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.665 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:38:59.666 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.324642+0000", 2024-09-16T09:38:59.666 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.701e-05 2024-09-16T09:38:59.666 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.666 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.666 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:38:59.666 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.325658+0000", 2024-09-16T09:38:59.666 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.001016711 2024-09-16T09:38:59.666 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:38:59.666 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:38:59.666 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:38:59.666 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.666 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.667 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:211728 9.6 9:65b3aeda:::benchmark_data_smithi070_137000_object90017:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e330)", 2024-09-16T09:38:59.667 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:36:50.353042+0000", 2024-09-16T09:38:59.667 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 129.22165824499999, 2024-09-16T09:38:59.667 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.97280060199999996, 2024-09-16T09:38:59.667 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:38:59.667 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:38:59.667 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:38:59.667 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:38:59.667 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:38:59.667 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:38:59.667 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 211728 2024-09-16T09:38:59.667 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.667 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:38:59.668 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.668 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:38:59.668 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.353042+0000", 2024-09-16T09:38:59.668 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:38:59.668 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.668 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.668 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:38:59.668 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.353042+0000", 2024-09-16T09:38:59.668 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:38:59.668 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.668 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.668 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:38:59.669 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.353042+0000", 2024-09-16T09:38:59.669 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.2499999999999999e-07 2024-09-16T09:38:59.669 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.669 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.669 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:38:59.669 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.353043+0000", 2024-09-16T09:38:59.669 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.012e-06 2024-09-16T09:38:59.669 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.669 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.669 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:38:59.669 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.353048+0000", 2024-09-16T09:38:59.669 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.172e-06 2024-09-16T09:38:59.669 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.669 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.670 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:38:59.670 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.353050+0000", 2024-09-16T09:38:59.670 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.8109999999999999e-06 2024-09-16T09:38:59.670 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.670 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.670 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:38:59.670 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.387783+0000", 2024-09-16T09:38:59.670 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.034732379000000001 2024-09-16T09:38:59.670 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.670 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.670 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:38:59.670 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.387787+0000", 2024-09-16T09:38:59.670 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.7890000000000002e-06 2024-09-16T09:38:59.670 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.671 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.671 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:38:59.671 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.322488+0000", 2024-09-16T09:38:59.671 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.934700431 2024-09-16T09:38:59.671 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.671 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.671 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:38:59.671 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.322579+0000", 2024-09-16T09:38:59.671 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.0975000000000002e-05 2024-09-16T09:38:59.671 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.671 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.671 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 1,2", 2024-09-16T09:38:59.671 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.322669+0000", 2024-09-16T09:38:59.672 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.0041000000000001e-05 2024-09-16T09:38:59.672 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.672 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.672 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:38:59.672 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.323644+0000", 2024-09-16T09:38:59.672 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00097474400000000004 2024-09-16T09:38:59.672 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.672 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.672 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:38:59.672 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.325710+0000", 2024-09-16T09:38:59.672 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.002066647 2024-09-16T09:38:59.672 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.672 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.672 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:38:59.673 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.325764+0000", 2024-09-16T09:38:59.673 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.4150000000000002e-05 2024-09-16T09:38:59.673 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.673 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.673 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:38:59.673 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.325810+0000", 2024-09-16T09:38:59.673 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.5287000000000002e-05 2024-09-16T09:38:59.673 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.673 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.673 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:38:59.673 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.325843+0000", 2024-09-16T09:38:59.673 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.2938999999999998e-05 2024-09-16T09:38:59.673 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:38:59.674 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:38:59.674 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:38:59.674 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.674 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.674 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:211729 9.6 9:62bd5b69:::benchmark_data_smithi070_137000_object90018:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e330)", 2024-09-16T09:38:59.674 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:36:50.353053+0000", 2024-09-16T09:38:59.674 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 129.22164710999999, 2024-09-16T09:38:59.674 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.97299811599999997, 2024-09-16T09:38:59.674 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:38:59.674 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:38:59.674 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:38:59.674 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:38:59.675 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:38:59.675 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:38:59.675 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 211729 2024-09-16T09:38:59.675 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.675 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:38:59.675 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.675 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:38:59.675 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.353053+0000", 2024-09-16T09:38:59.675 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:38:59.675 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.675 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.675 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:38:59.675 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.353053+0000", 2024-09-16T09:38:59.676 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:38:59.676 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.676 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.676 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:38:59.676 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.353053+0000", 2024-09-16T09:38:59.676 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.11e-07 2024-09-16T09:38:59.676 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.676 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.676 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:38:59.676 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.353054+0000", 2024-09-16T09:38:59.676 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.7199999999999997e-07 2024-09-16T09:38:59.676 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.676 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.676 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:38:59.677 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.353058+0000", 2024-09-16T09:38:59.677 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.9199999999999997e-06 2024-09-16T09:38:59.677 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.677 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.677 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:38:59.677 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.353061+0000", 2024-09-16T09:38:59.677 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.6220000000000001e-06 2024-09-16T09:38:59.677 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.677 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.677 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:38:59.677 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.387809+0000", 2024-09-16T09:38:59.677 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.034747792 2024-09-16T09:38:59.677 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.678 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.678 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:38:59.678 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.387813+0000", 2024-09-16T09:38:59.678 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.6739999999999996e-06 2024-09-16T09:38:59.678 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.678 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.678 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:38:59.678 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.322867+0000", 2024-09-16T09:38:59.678 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.93505334100000004 2024-09-16T09:38:59.678 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.678 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.678 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:38:59.678 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.322971+0000", 2024-09-16T09:38:59.678 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00010383299999999999 2024-09-16T09:38:59.679 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.679 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.679 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 1,2", 2024-09-16T09:38:59.679 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.323079+0000", 2024-09-16T09:38:59.679 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000108176 2024-09-16T09:38:59.679 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.679 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.679 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:38:59.679 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.325725+0000", 2024-09-16T09:38:59.679 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.002645938 2024-09-16T09:38:59.679 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.679 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.679 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:38:59.679 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.325875+0000", 2024-09-16T09:38:59.680 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00015008 2024-09-16T09:38:59.680 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.680 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.680 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:38:59.680 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.325980+0000", 2024-09-16T09:38:59.680 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000105146 2024-09-16T09:38:59.680 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.680 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.680 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:38:59.680 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.326021+0000", 2024-09-16T09:38:59.680 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.0788999999999999e-05 2024-09-16T09:38:59.680 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.680 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.680 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:38:59.681 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.326051+0000", 2024-09-16T09:38:59.681 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.0621999999999999e-05 2024-09-16T09:38:59.681 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:38:59.681 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:38:59.681 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:38:59.681 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.681 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.681 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:211741 9.6 9:640b6237:::benchmark_data_smithi070_137000_object90030:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e330)", 2024-09-16T09:38:59.681 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:36:50.353064+0000", 2024-09-16T09:38:59.681 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 129.22163622900001, 2024-09-16T09:38:59.681 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.97312102199999995, 2024-09-16T09:38:59.681 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:38:59.681 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:38:59.681 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:38:59.682 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:38:59.682 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:38:59.682 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:38:59.682 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 211741 2024-09-16T09:38:59.682 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.682 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:38:59.682 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.682 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:38:59.682 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.353064+0000", 2024-09-16T09:38:59.682 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:38:59.682 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.682 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.682 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:38:59.683 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.353064+0000", 2024-09-16T09:38:59.683 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:38:59.683 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.683 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.683 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:38:59.683 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.353064+0000", 2024-09-16T09:38:59.683 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.05e-07 2024-09-16T09:38:59.683 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.683 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.683 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:38:59.683 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.353065+0000", 2024-09-16T09:38:59.683 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.64e-07 2024-09-16T09:38:59.683 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.683 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.684 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:38:59.684 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.353069+0000", 2024-09-16T09:38:59.684 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.8809999999999998e-06 2024-09-16T09:38:59.684 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.684 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.684 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:38:59.684 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.353072+0000", 2024-09-16T09:38:59.684 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.8140000000000001e-06 2024-09-16T09:38:59.684 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.684 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.684 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:38:59.684 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.387833+0000", 2024-09-16T09:38:59.684 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.034761132 2024-09-16T09:38:59.684 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.685 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.685 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:38:59.685 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.387837+0000", 2024-09-16T09:38:59.685 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.411e-06 2024-09-16T09:38:59.685 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.685 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.685 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:38:59.685 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.323276+0000", 2024-09-16T09:38:59.685 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.93543996200000001 2024-09-16T09:38:59.685 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.685 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.685 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:38:59.685 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.323361+0000", 2024-09-16T09:38:59.685 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.4988999999999996e-05 2024-09-16T09:38:59.685 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.686 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.686 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 1,2", 2024-09-16T09:38:59.686 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.323450+0000", 2024-09-16T09:38:59.686 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.8856000000000004e-05 2024-09-16T09:38:59.686 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.686 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.686 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:38:59.686 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.325729+0000", 2024-09-16T09:38:59.686 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0022786870000000002 2024-09-16T09:38:59.686 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.686 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.686 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:38:59.686 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.325945+0000", 2024-09-16T09:38:59.686 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00021593400000000001 2024-09-16T09:38:59.686 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.687 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.687 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:38:59.687 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.326115+0000", 2024-09-16T09:38:59.687 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00017053899999999999 2024-09-16T09:38:59.687 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.687 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.687 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:38:59.687 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.326155+0000", 2024-09-16T09:38:59.687 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.9768999999999998e-05 2024-09-16T09:38:59.687 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.687 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.687 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:38:59.687 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.326185+0000", 2024-09-16T09:38:59.687 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.9879e-05 2024-09-16T09:38:59.688 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:38:59.688 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:38:59.688 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:38:59.688 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.688 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.688 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:211770 9.6 9:64e2bfd5:::benchmark_data_smithi070_137000_object90059:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e330)", 2024-09-16T09:38:59.688 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:36:50.353075+0000", 2024-09-16T09:38:59.688 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 129.221625383, 2024-09-16T09:38:59.688 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.97422515600000004, 2024-09-16T09:38:59.688 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:38:59.688 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:38:59.688 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:38:59.688 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:38:59.688 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:38:59.689 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:38:59.689 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 211770 2024-09-16T09:38:59.689 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.689 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:38:59.689 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.689 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:38:59.689 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.353075+0000", 2024-09-16T09:38:59.689 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:38:59.689 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.689 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.689 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:38:59.689 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.353075+0000", 2024-09-16T09:38:59.689 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:38:59.689 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.690 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.690 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:38:59.690 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.353075+0000", 2024-09-16T09:38:59.690 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.1400000000000001e-07 2024-09-16T09:38:59.690 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.690 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.690 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:38:59.690 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.353076+0000", 2024-09-16T09:38:59.690 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0559999999999999e-06 2024-09-16T09:38:59.690 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.690 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.690 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:38:59.690 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.353080+0000", 2024-09-16T09:38:59.690 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.0960000000000003e-06 2024-09-16T09:38:59.690 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.690 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.691 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:38:59.691 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.353083+0000", 2024-09-16T09:38:59.691 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.6230000000000001e-06 2024-09-16T09:38:59.691 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.691 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.691 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:38:59.691 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.387858+0000", 2024-09-16T09:38:59.691 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.034775421000000001 2024-09-16T09:38:59.691 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.691 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.691 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:38:59.691 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.387862+0000", 2024-09-16T09:38:59.691 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.6260000000000001e-06 2024-09-16T09:38:59.691 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.692 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.692 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:38:59.692 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.323671+0000", 2024-09-16T09:38:59.692 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.93580861800000004 2024-09-16T09:38:59.692 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.692 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.692 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:38:59.692 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.323757+0000", 2024-09-16T09:38:59.692 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.5913000000000002e-05 2024-09-16T09:38:59.692 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.692 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.692 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 1,2", 2024-09-16T09:38:59.692 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.323864+0000", 2024-09-16T09:38:59.692 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00010713900000000001 2024-09-16T09:38:59.693 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.693 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.693 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:38:59.693 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.325733+0000", 2024-09-16T09:38:59.693 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0018691339999999999 2024-09-16T09:38:59.693 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.693 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.693 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:38:59.693 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.326083+0000", 2024-09-16T09:38:59.693 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00035014999999999997 2024-09-16T09:38:59.693 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.693 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.693 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:38:59.693 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.327211+0000", 2024-09-16T09:38:59.694 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0011281430000000001 2024-09-16T09:38:59.694 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.694 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.694 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:38:59.694 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.327265+0000", 2024-09-16T09:38:59.694 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.3755999999999998e-05 2024-09-16T09:38:59.694 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.694 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.694 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:38:59.694 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.327300+0000", 2024-09-16T09:38:59.694 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.5267e-05 2024-09-16T09:38:59.694 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:38:59.694 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:38:59.694 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:38:59.694 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.695 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.695 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:211771 9.6 9:62d60a7f:::benchmark_data_smithi070_137000_object90060:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e330)", 2024-09-16T09:38:59.695 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:36:50.353086+0000", 2024-09-16T09:38:59.695 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 129.221614636, 2024-09-16T09:38:59.695 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.97706001899999995, 2024-09-16T09:38:59.695 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:38:59.695 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:38:59.695 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:38:59.695 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:38:59.695 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:38:59.695 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:38:59.695 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 211771 2024-09-16T09:38:59.695 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.696 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:38:59.696 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.696 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:38:59.696 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.353086+0000", 2024-09-16T09:38:59.696 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:38:59.696 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.696 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.696 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:38:59.696 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.353086+0000", 2024-09-16T09:38:59.696 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:38:59.696 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.696 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.696 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:38:59.696 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.353086+0000", 2024-09-16T09:38:59.697 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.1799999999999999e-07 2024-09-16T09:38:59.697 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.697 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.697 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:38:59.697 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.353087+0000", 2024-09-16T09:38:59.697 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.243e-06 2024-09-16T09:38:59.697 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.697 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.697 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:38:59.697 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.353090+0000", 2024-09-16T09:38:59.697 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.18e-06 2024-09-16T09:38:59.697 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.697 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.698 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:38:59.698 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.353093+0000", 2024-09-16T09:38:59.698 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.4279999999999999e-06 2024-09-16T09:38:59.698 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.698 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.698 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:38:59.698 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.387876+0000", 2024-09-16T09:38:59.698 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.034782865000000003 2024-09-16T09:38:59.698 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.698 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.698 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:38:59.698 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:50.387879+0000", 2024-09-16T09:38:59.698 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.4850000000000001e-06 2024-09-16T09:38:59.698 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.699 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.699 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:38:59.699 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.326210+0000", 2024-09-16T09:38:59.699 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.93833082400000001 2024-09-16T09:38:59.699 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.699 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.699 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:38:59.699 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.326301+0000", 2024-09-16T09:38:59.699 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.1168000000000005e-05 2024-09-16T09:38:59.699 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.699 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.699 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 1,2", 2024-09-16T09:38:59.699 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.326391+0000", 2024-09-16T09:38:59.699 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.0203000000000004e-05 2024-09-16T09:38:59.700 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.700 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.700 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:38:59.700 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.327677+0000", 2024-09-16T09:38:59.700 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.001285455 2024-09-16T09:38:59.700 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.700 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.700 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:38:59.700 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.330002+0000", 2024-09-16T09:38:59.700 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.002325569 2024-09-16T09:38:59.700 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.700 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.700 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:38:59.701 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.330043+0000", 2024-09-16T09:38:59.701 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.0763999999999998e-05 2024-09-16T09:38:59.701 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.701 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.701 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:38:59.701 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.330100+0000", 2024-09-16T09:38:59.701 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.6787999999999999e-05 2024-09-16T09:38:59.701 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.701 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.701 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:38:59.701 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:36:51.330146+0000", 2024-09-16T09:38:59.701 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.5828999999999998e-05 2024-09-16T09:38:59.701 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:38:59.701 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:38:59.702 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:38:59.702 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.702 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.702 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:231320 9.16 9:6c089b62:::benchmark_data_smithi070_137000_object109609:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e357)", 2024-09-16T09:38:59.702 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:37:18.511931+0000", 2024-09-16T09:38:59.702 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 101.06276892699999, 2024-09-16T09:38:59.702 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1082087490000001, 2024-09-16T09:38:59.702 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:38:59.702 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:38:59.702 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "reached pg", 2024-09-16T09:38:59.702 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:38:59.702 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:38:59.702 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:38:59.703 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 231320 2024-09-16T09:38:59.703 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.703 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:38:59.703 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.703 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:38:59.703 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.511931+0000", 2024-09-16T09:38:59.703 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:38:59.703 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.703 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.703 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:38:59.703 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.511931+0000", 2024-09-16T09:38:59.703 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:38:59.703 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.703 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.704 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:38:59.704 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.511934+0000", 2024-09-16T09:38:59.704 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.317e-06 2024-09-16T09:38:59.704 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.704 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.704 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:38:59.704 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.511942+0000", 2024-09-16T09:38:59.704 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.4910000000000005e-06 2024-09-16T09:38:59.704 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.704 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.704 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:38:59.704 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.511957+0000", 2024-09-16T09:38:59.704 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.4457999999999999e-05 2024-09-16T09:38:59.705 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.705 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.705 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:38:59.705 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.511965+0000", 2024-09-16T09:38:59.705 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.2800000000000003e-06 2024-09-16T09:38:59.705 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.705 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.705 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:38:59.705 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.703302+0000", 2024-09-16T09:38:59.705 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.19133676399999999 2024-09-16T09:38:59.705 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.705 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.706 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:38:59.706 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.703311+0000", 2024-09-16T09:38:59.706 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.4879999999999996e-06 2024-09-16T09:38:59.706 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.706 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.706 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:38:59.706 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.617075+0000", 2024-09-16T09:38:59.706 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.91376354699999995 2024-09-16T09:38:59.706 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.706 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.706 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for missing object", 2024-09-16T09:38:59.706 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.617230+0000", 2024-09-16T09:38:59.706 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00015520900000000001 2024-09-16T09:38:59.706 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.707 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.707 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:38:59.707 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.620075+0000", 2024-09-16T09:38:59.707 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0028451660000000001 2024-09-16T09:38:59.707 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.707 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.707 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:38:59.707 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.620140+0000", 2024-09-16T09:38:59.707 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.5029000000000004e-05 2024-09-16T09:38:59.707 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:38:59.707 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:38:59.707 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:38:59.707 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.707 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.708 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:232736 9.12 9.ebe00572 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e359)", 2024-09-16T09:38:59.708 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:37:20.535288+0000", 2024-09-16T09:38:59.708 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 99.039412288999998, 2024-09-16T09:38:59.708 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.99201526200000001, 2024-09-16T09:38:59.708 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:38:59.708 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:38:59.708 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "reached pg", 2024-09-16T09:38:59.708 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:38:59.708 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:38:59.708 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:38:59.708 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 232736 2024-09-16T09:38:59.708 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.708 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:38:59.708 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.709 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:38:59.709 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.535288+0000", 2024-09-16T09:38:59.709 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:38:59.709 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.709 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.709 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:38:59.709 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.535288+0000", 2024-09-16T09:38:59.709 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:38:59.709 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.709 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.709 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:38:59.709 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.535289+0000", 2024-09-16T09:38:59.709 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.279e-06 2024-09-16T09:38:59.709 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.710 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.710 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:38:59.710 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.535294+0000", 2024-09-16T09:38:59.710 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.8289999999999998e-06 2024-09-16T09:38:59.710 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.710 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.710 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:38:59.710 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.535303+0000", 2024-09-16T09:38:59.710 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.3090000000000001e-06 2024-09-16T09:38:59.710 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.710 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.710 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:38:59.710 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.535310+0000", 2024-09-16T09:38:59.710 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.9120000000000001e-06 2024-09-16T09:38:59.710 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.711 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.711 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:38:59.711 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.535394+0000", 2024-09-16T09:38:59.711 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.3869000000000006e-05 2024-09-16T09:38:59.711 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.711 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.711 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:38:59.711 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.535403+0000", 2024-09-16T09:38:59.711 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.0310000000000004e-06 2024-09-16T09:38:59.711 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.711 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.711 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:38:59.711 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:21.527289+0000", 2024-09-16T09:38:59.711 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.99188559499999995 2024-09-16T09:38:59.712 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.712 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.712 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:38:59.712 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:21.527303+0000", 2024-09-16T09:38:59.712 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.4438e-05 2024-09-16T09:38:59.712 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:38:59.712 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:38:59.712 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:38:59.712 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.712 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.712 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:232758 9.12 9.86d0f952 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e359)", 2024-09-16T09:38:59.712 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:37:20.553519+0000", 2024-09-16T09:38:59.712 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 99.021180982000004, 2024-09-16T09:38:59.712 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.97382663999999997, 2024-09-16T09:38:59.713 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:38:59.713 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:38:59.713 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "reached pg", 2024-09-16T09:38:59.713 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:38:59.713 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:38:59.713 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:38:59.713 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 232758 2024-09-16T09:38:59.713 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.713 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:38:59.713 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.713 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:38:59.713 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.553519+0000", 2024-09-16T09:38:59.713 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:38:59.713 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.714 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.714 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:38:59.714 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.553519+0000", 2024-09-16T09:38:59.714 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:38:59.714 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.714 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.714 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:38:59.714 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.553520+0000", 2024-09-16T09:38:59.714 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.2399999999999997e-07 2024-09-16T09:38:59.714 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.714 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.714 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:38:59.714 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.553524+0000", 2024-09-16T09:38:59.714 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.5190000000000001e-06 2024-09-16T09:38:59.714 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.715 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.715 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:38:59.715 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.553535+0000", 2024-09-16T09:38:59.715 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1603999999999999e-05 2024-09-16T09:38:59.715 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.715 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.715 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:38:59.715 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.553540+0000", 2024-09-16T09:38:59.715 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.8670000000000002e-06 2024-09-16T09:38:59.715 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.715 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.715 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:38:59.715 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.553591+0000", 2024-09-16T09:38:59.715 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.1091000000000003e-05 2024-09-16T09:38:59.715 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.716 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.716 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:38:59.716 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.553605+0000", 2024-09-16T09:38:59.716 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.3609e-05 2024-09-16T09:38:59.716 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.716 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.716 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:38:59.716 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:21.527336+0000", 2024-09-16T09:38:59.716 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.97373105900000001 2024-09-16T09:38:59.716 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:38:59.716 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:38:59.716 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:38:59.716 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:21.527346+0000", 2024-09-16T09:38:59.716 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0166999999999999e-05 2024-09-16T09:38:59.717 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:38:59.717 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:38:59.717 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:38:59.717 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:38:59.717 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:38:59.717 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-09-16T09:38:59.717 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:38:59.718 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:38:59 smithi084 ceph-mon[32629]: pgmap v740: 178 pgs: 5 active+recovering+undersized+remapped, 3 active+recovery_wait+undersized+degraded+remapped, 2 active+remapped+backfill_toofull, 168 active+clean; 5.1 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 853 B/s rd, 44 MiB/s wr, 703 op/s; 13884/252195 objects degraded (5.505%); 24190/252195 objects misplaced (9.592%); 20 MiB/s, 314 objects/s recovering 2024-09-16T09:38:59.740 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:59.741 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:59.741 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:38:59.754 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:38:59 smithi070 ceph-mon[30388]: pgmap v740: 178 pgs: 5 active+recovering+undersized+remapped, 3 active+recovery_wait+undersized+degraded+remapped, 2 active+remapped+backfill_toofull, 168 active+clean; 5.1 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 853 B/s rd, 44 MiB/s wr, 703 op/s; 13884/252195 objects degraded (5.505%); 24190/252195 objects misplaced (9.592%); 20 MiB/s, 314 objects/s recovering 2024-09-16T09:38:59.755 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:38:59 smithi070 ceph-mon[40691]: pgmap v740: 178 pgs: 5 active+recovering+undersized+remapped, 3 active+recovery_wait+undersized+degraded+remapped, 2 active+remapped+backfill_toofull, 168 active+clean; 5.1 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 853 B/s rd, 44 MiB/s wr, 703 op/s; 13884/252195 objects degraded (5.505%); 24190/252195 objects misplaced (9.592%); 20 MiB/s, 314 objects/s recovering 2024-09-16T09:38:59.842 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:38:59.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:38:59.895 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:38:59.895 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:38:59.996 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:39:00.018 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:00.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:00.019 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:39:00.120 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:39:00.143 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:00.144 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:00.144 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:39:00.245 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:39:00.268 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:00.269 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:00.269 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:39:00.370 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:39:00.400 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:00.401 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:00.401 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:39:00.502 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:39:00.523 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:00.524 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:00.524 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:39:00.554 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 64 16 90348 90332 88.1851 31.8125 0.12638 0.0113353 2024-09-16T09:39:00.568 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.6 dump_ops_in_flight 2024-09-16T09:39:00.625 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:39:00.670 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:00.670 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:00.671 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:39:00.771 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:39:00.812 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:00.813 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:00.813 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:39:00.914 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:39:00.952 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:00.952 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:00.953 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:39:00.957 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:39:01.054 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:39:01.092 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:01.092 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:01.093 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:39:01.194 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:39:01.225 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:01.226 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:01.227 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:39:01.327 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:39:01.361 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:01.361 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:01.362 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:39:01.462 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:39:01.510 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:01.511 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:01.511 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:39:01.556 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 65 16 91467 91451 87.901 69.9375 0.00938523 0.0113703 2024-09-16T09:39:01.562 INFO:tasks.thrashosds.thrasher:j is {'epoch': 403, 'fsid': 'a1321834-740c-11ef-bceb-c7b262605968', 'created': '2024-09-16T09:19:14.121498+0000', 'modified': '2024-09-16T09:38:44.342870+0000', 'last_up_change': '2024-09-16T09:24:51.124496+0000', 'last_in_change': '2024-09-16T09:24:28.432920+0000', 'flags': 'noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5801984, 'flags_set': ['noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 18, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 10, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T09:22:27.617694+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 21, 'pg_placement_num': 21, 'pg_placement_num_target': 21, 'pg_num_target': 21, 'pg_num_pending': 21, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '162', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '160', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6699999570846558, 'score_stable': 1.6699999570846558, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 1.899999976158142, 'raw_score_stable': 1.899999976158142, 'primary_affinity_weighted': 0.7071933746337891, 'average_primary_affinity': 0.8100000023841858, 'average_primary_affinity_weighted': 0.7099999785423279}}, {'pool': 2, 'pool_name': 'datapool', 'create_time': '2024-09-16T09:24:59.169244+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 3, 'pg_placement_num': 3, 'pg_placement_num_target': 3, 'pg_num_target': 3, 'pg_num_pending': 3, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 61, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.3299999237060547, 'score_stable': 2.3299999237060547, 'optimal_score': 0.8600000143051147, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.7071933746337891, 'average_primary_affinity': 0.8100000023841858, 'average_primary_affinity_weighted': 0.7099999785423279}}, {'pool': 3, 'pool_name': '.rgw.root', 'create_time': '2024-09-16T09:25:00.056119+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.7071933746337891, 'average_primary_affinity': 0.8100000023841858, 'average_primary_affinity_weighted': 0.7099999785423279}}, {'pool': 4, 'pool_name': 'default.rgw.log', 'create_time': '2024-09-16T09:25:01.770016+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '62', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.849999904632568, 'score_stable': 2.190000057220459, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 7.809999942779541, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 0.7071933746337891, 'average_primary_affinity': 0.8100000023841858, 'average_primary_affinity_weighted': 0.7099999785423279, 'error_message': 'pool 4 has acting primaries on OSD(s) with primary affinity 0, read balance score is not accurate'}}, {'pool': 5, 'pool_name': 'default.rgw.control', 'create_time': '2024-09-16T09:25:03.725555+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.7071933746337891, 'average_primary_affinity': 0.8100000023841858, 'average_primary_affinity_weighted': 0.7099999785423279}}, {'pool': 6, 'pool_name': 'default.rgw.meta', 'create_time': '2024-09-16T09:25:05.773530+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5399999618530273, 'score_stable': 1.5399999618530273, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 0.7071933746337891, 'average_primary_affinity': 0.8100000023841858, 'average_primary_affinity_weighted': 0.7099999785423279}}, {'pool': 10, 'pool_name': 'unique_pool_3', 'create_time': '2024-09-16T09:37:43.482267+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 26, 'pg_placement_num': 26, 'pg_placement_num_target': 26, 'pg_num_target': 26, 'pg_num_pending': 26, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '397', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '395', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rados': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6200000047683716, 'score_stable': 2.1600000858306885, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 1.850000023841858, 'raw_score_stable': 2.4600000381469727, 'primary_affinity_weighted': 0.7071933746337891, 'average_primary_affinity': 0.8100000023841858, 'average_primary_affinity_weighted': 0.7099999785423279}}], 'osds': [{'osd': 0, 'uuid': '7ef288c8-3bf0-4814-8e86-f70a7de041ce', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 8, 'up_thru': 398, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6802', 'nonce': 2013075781}, {'type': 'v1', 'addr': '172.21.15.70:6803', 'nonce': 2013075781}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6804', 'nonce': 2013075781}, {'type': 'v1', 'addr': '172.21.15.70:6805', 'nonce': 2013075781}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6808', 'nonce': 2013075781}, {'type': 'v1', 'addr': '172.21.15.70:6809', 'nonce': 2013075781}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6806', 'nonce': 2013075781}, {'type': 'v1', 'addr': '172.21.15.70:6807', 'nonce': 2013075781}]}, 'public_addr': '172.21.15.70:6803/2013075781', 'cluster_addr': '172.21.15.70:6805/2013075781', 'heartbeat_back_addr': '172.21.15.70:6809/2013075781', 'heartbeat_front_addr': '172.21.15.70:6807/2013075781', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '474d41d9-295a-450c-8f27-ce487a14678b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 362, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6810', 'nonce': 1670012115}, {'type': 'v1', 'addr': '172.21.15.70:6811', 'nonce': 1670012115}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6812', 'nonce': 1670012115}, {'type': 'v1', 'addr': '172.21.15.70:6813', 'nonce': 1670012115}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6816', 'nonce': 1670012115}, {'type': 'v1', 'addr': '172.21.15.70:6817', 'nonce': 1670012115}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6814', 'nonce': 1670012115}, {'type': 'v1', 'addr': '172.21.15.70:6815', 'nonce': 1670012115}]}, 'public_addr': '172.21.15.70:6811/1670012115', 'cluster_addr': '172.21.15.70:6813/1670012115', 'heartbeat_back_addr': '172.21.15.70:6817/1670012115', 'heartbeat_front_addr': '172.21.15.70:6815/1670012115', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '5b4ca4e8-cecc-4163-b291-c9209b02c4b7', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.1810150146484375, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 18, 'up_thru': 398, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6818', 'nonce': 3601682336}, {'type': 'v1', 'addr': '172.21.15.70:6819', 'nonce': 3601682336}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6820', 'nonce': 3601682336}, {'type': 'v1', 'addr': '172.21.15.70:6821', 'nonce': 3601682336}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6824', 'nonce': 3601682336}, {'type': 'v1', 'addr': '172.21.15.70:6825', 'nonce': 3601682336}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6822', 'nonce': 3601682336}, {'type': 'v1', 'addr': '172.21.15.70:6823', 'nonce': 3601682336}]}, 'public_addr': '172.21.15.70:6819/3601682336', 'cluster_addr': '172.21.15.70:6821/3601682336', 'heartbeat_back_addr': '172.21.15.70:6825/3601682336', 'heartbeat_front_addr': '172.21.15.70:6823/3601682336', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'ffb8d527-1b58-4ecd-808c-47d4ba1bf387', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.476531982421875, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 26, 'up_thru': 398, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6826', 'nonce': 2038083007}, {'type': 'v1', 'addr': '172.21.15.70:6827', 'nonce': 2038083007}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6828', 'nonce': 2038083007}, {'type': 'v1', 'addr': '172.21.15.70:6829', 'nonce': 2038083007}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6832', 'nonce': 2038083007}, {'type': 'v1', 'addr': '172.21.15.70:6833', 'nonce': 2038083007}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6830', 'nonce': 2038083007}, {'type': 'v1', 'addr': '172.21.15.70:6831', 'nonce': 2038083007}]}, 'public_addr': '172.21.15.70:6827/2038083007', 'cluster_addr': '172.21.15.70:6829/2038083007', 'heartbeat_back_addr': '172.21.15.70:6833/2038083007', 'heartbeat_front_addr': '172.21.15.70:6831/2038083007', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'a946760b-4d91-4a4b-8e9b-a07a50589211', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 32, 'up_thru': 398, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6800', 'nonce': 2250892710}, {'type': 'v1', 'addr': '172.21.15.84:6801', 'nonce': 2250892710}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6802', 'nonce': 2250892710}, {'type': 'v1', 'addr': '172.21.15.84:6803', 'nonce': 2250892710}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6806', 'nonce': 2250892710}, {'type': 'v1', 'addr': '172.21.15.84:6807', 'nonce': 2250892710}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6804', 'nonce': 2250892710}, {'type': 'v1', 'addr': '172.21.15.84:6805', 'nonce': 2250892710}]}, 'public_addr': '172.21.15.84:6801/2250892710', 'cluster_addr': '172.21.15.84:6803/2250892710', 'heartbeat_back_addr': '172.21.15.84:6807/2250892710', 'heartbeat_front_addr': '172.21.15.84:6805/2250892710', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '67681760-75a9-4846-946e-a935d42c5e05', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 39, 'up_thru': 397, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6808', 'nonce': 1803945348}, {'type': 'v1', 'addr': '172.21.15.84:6809', 'nonce': 1803945348}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6810', 'nonce': 1803945348}, {'type': 'v1', 'addr': '172.21.15.84:6811', 'nonce': 1803945348}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6814', 'nonce': 1803945348}, {'type': 'v1', 'addr': '172.21.15.84:6815', 'nonce': 1803945348}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6812', 'nonce': 1803945348}, {'type': 'v1', 'addr': '172.21.15.84:6813', 'nonce': 1803945348}]}, 'public_addr': '172.21.15.84:6809/1803945348', 'cluster_addr': '172.21.15.84:6811/1803945348', 'heartbeat_back_addr': '172.21.15.84:6815/1803945348', 'heartbeat_front_addr': '172.21.15.84:6813/1803945348', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '7c8f0152-f539-4b7e-8113-91dd7529d3a1', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 46, 'up_thru': 397, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6816', 'nonce': 610727189}, {'type': 'v1', 'addr': '172.21.15.84:6817', 'nonce': 610727189}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6818', 'nonce': 610727189}, {'type': 'v1', 'addr': '172.21.15.84:6819', 'nonce': 610727189}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6822', 'nonce': 610727189}, {'type': 'v1', 'addr': '172.21.15.84:6823', 'nonce': 610727189}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6820', 'nonce': 610727189}, {'type': 'v1', 'addr': '172.21.15.84:6821', 'nonce': 610727189}]}, 'public_addr': '172.21.15.84:6817/610727189', 'cluster_addr': '172.21.15.84:6819/610727189', 'heartbeat_back_addr': '172.21.15.84:6823/610727189', 'heartbeat_front_addr': '172.21.15.84:6821/610727189', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '945efaba-225e-49af-aefd-b4d58ca8093b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 53, 'up_thru': 397, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6824', 'nonce': 1002252531}, {'type': 'v1', 'addr': '172.21.15.84:6825', 'nonce': 1002252531}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6826', 'nonce': 1002252531}, {'type': 'v1', 'addr': '172.21.15.84:6827', 'nonce': 1002252531}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6830', 'nonce': 1002252531}, {'type': 'v1', 'addr': '172.21.15.84:6831', 'nonce': 1002252531}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6828', 'nonce': 1002252531}, {'type': 'v1', 'addr': '172.21.15.84:6829', 'nonce': 1002252531}]}, 'public_addr': '172.21.15.84:6825/1002252531', 'cluster_addr': '172.21.15.84:6827/1002252531', 'heartbeat_back_addr': '172.21.15.84:6831/1002252531', 'heartbeat_front_addr': '172.21.15.84:6829/1002252531', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T09:37:48.793810+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T09:38:42.788265+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T09:38:35.110455+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T09:38:41.243696+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T09:38:35.892571+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T09:37:48.073602+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T09:37:46.771979+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T09:37:46.052484+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.1d', 'osds': [1, 0, 7]}, {'pgid': '4.b', 'osds': [0, 5, 1]}], 'pg_upmap_items': [{'pgid': '2.1', 'mappings': [{'from': 3, 'to': 0}, {'from': 2, 'to': 1}]}, {'pgid': '3.1b', 'mappings': [{'from': 0, 'to': 6}, {'from': 4, 'to': 7}]}, {'pgid': '4.f', 'mappings': [{'from': 1, 'to': 2}]}, {'pgid': '4.13', 'mappings': [{'from': 1, 'to': 5}]}, {'pgid': '4.17', 'mappings': [{'from': 1, 'to': 2}]}], 'pg_upmap_primaries': [], 'pg_temp': [{'pgid': '10.10', 'osds': [4, 6]}, {'pgid': '10.11', 'osds': [0, 7, 4]}, {'pgid': '10.12', 'osds': [0, 1, 4]}, {'pgid': '10.13', 'osds': [2, 1]}, {'pgid': '10.14', 'osds': [4, 1]}, {'pgid': '10.15', 'osds': [3, 1]}, {'pgid': '10.16', 'osds': [0, 5]}, {'pgid': '10.17', 'osds': [3, 6]}, {'pgid': '10.18', 'osds': [2, 1]}, {'pgid': '10.19', 'osds': [2, 1]}], 'primary_temp': [], 'blocklist': {'172.21.15.70:0/125491368': '2024-09-17T09:25:31.465167+0000', '172.21.15.70:0/3354284852': '2024-09-17T09:19:43.205989+0000', '172.21.15.70:0/3284625547': '2024-09-17T09:25:31.465167+0000', '172.21.15.70:0/186565817': '2024-09-17T09:19:43.205989+0000', '172.21.15.70:0/2213852865': '2024-09-17T09:19:43.205989+0000', '172.21.15.70:0/1812373760': '2024-09-17T09:20:09.306330+0000', '172.21.15.70:6800/837338094': '2024-09-17T09:19:43.205989+0000', '172.21.15.70:6800/1876661229': '2024-09-17T09:25:31.465167+0000', '172.21.15.70:0/3351688485': '2024-09-17T09:20:09.306330+0000', '172.21.15.70:6800/3871553358': '2024-09-17T09:20:09.306330+0000', '172.21.15.70:0/1230482871': '2024-09-17T09:25:31.465167+0000', '172.21.15.70:6801/837338094': '2024-09-17T09:19:43.205989+0000', '172.21.15.70:6801/3871553358': '2024-09-17T09:20:09.306330+0000', '172.21.15.70:0/1403001486': '2024-09-17T09:20:09.306330+0000', '172.21.15.70:0/1723643739': '2024-09-17T09:25:31.465167+0000', '172.21.15.70:6801/1876661229': '2024-09-17T09:25:31.465167+0000', '172.21.15.70:0/3456417396': '2024-09-17T09:25:31.465167+0000'}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-16T09:39:01.563 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-16T09:39:01.584 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-09-16T09:39:01.585 INFO:teuthology.orchestra.run.smithi084.stdout: "ops": [ 2024-09-16T09:39:01.585 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:39:01.585 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15975.0:91485 10.b 10:d0fdcdfb:::benchmark_data_smithi070_166744_object91484:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e407)", 2024-09-16T09:39:01.585 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:39:01.573319+0000", 2024-09-16T09:39:01.585 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 0.0097709660000000007, 2024-09-16T09:39:01.585 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0098052069999999998, 2024-09-16T09:39:01.585 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:39:01.585 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:39:01.585 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "waiting for sub ops", 2024-09-16T09:39:01.586 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:39:01.586 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15975", 2024-09-16T09:39:01.586 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3821727158", 2024-09-16T09:39:01.586 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 91485 2024-09-16T09:39:01.586 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:39:01.586 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:39:01.586 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:39:01.586 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:39:01.586 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:39:01.573319+0000", 2024-09-16T09:39:01.586 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:39:01.587 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:39:01.587 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:39:01.587 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:39:01.587 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:39:01.573319+0000", 2024-09-16T09:39:01.587 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:39:01.587 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:39:01.587 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:39:01.587 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:39:01.587 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:39:01.573320+0000", 2024-09-16T09:39:01.587 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.457e-06 2024-09-16T09:39:01.587 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:39:01.587 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:39:01.588 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:39:01.588 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:39:01.573359+0000", 2024-09-16T09:39:01.588 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.8664000000000001e-05 2024-09-16T09:39:01.588 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:39:01.588 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:39:01.588 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:39:01.588 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:39:01.573375+0000", 2024-09-16T09:39:01.588 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.5566999999999999e-05 2024-09-16T09:39:01.588 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:39:01.588 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:39:01.588 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:39:01.588 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:39:01.573383+0000", 2024-09-16T09:39:01.588 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.2709999999999992e-06 2024-09-16T09:39:01.589 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:39:01.589 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:39:01.589 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:39:01.589 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:39:01.573453+0000", 2024-09-16T09:39:01.589 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.0420999999999995e-05 2024-09-16T09:39:01.589 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:39:01.589 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:39:01.589 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:39:01.589 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:39:01.573604+0000", 2024-09-16T09:39:01.589 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00015105399999999999 2024-09-16T09:39:01.589 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:39:01.589 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:39:01.590 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 0,1", 2024-09-16T09:39:01.590 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:39:01.573779+0000", 2024-09-16T09:39:01.590 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00017478700000000001 2024-09-16T09:39:01.590 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:39:01.590 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:39:01.590 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:39:01.590 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:39:01.576616+0000", 2024-09-16T09:39:01.590 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0028372330000000002 2024-09-16T09:39:01.590 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:39:01.590 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:39:01.590 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:39:01.591 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:39:01.591 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:39:01.591 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_repop(client.15975.0:91491 10.17 e407/398 10:eb4c00a3:::benchmark_data_smithi070_166744_object91490:head v 407'5183, mlcod=391'3905)", 2024-09-16T09:39:01.591 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:39:01.582578+0000", 2024-09-16T09:39:01.591 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 0.00051219099999999995, 2024-09-16T09:39:01.591 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00062001000000000005, 2024-09-16T09:39:01.591 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:39:01.591 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:39:01.591 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "started", 2024-09-16T09:39:01.591 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:39:01.591 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:39:01.592 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:39:01.592 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:39:01.582578+0000", 2024-09-16T09:39:01.592 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:39:01.592 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:39:01.592 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:39:01.592 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:39:01.592 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:39:01.582578+0000", 2024-09-16T09:39:01.592 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:39:01.592 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:39:01.592 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:39:01.593 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:39:01.593 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:39:01.582579+0000", 2024-09-16T09:39:01.593 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.2200000000000002e-07 2024-09-16T09:39:01.593 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:39:01.593 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:39:01.593 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:39:01.593 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:39:01.582612+0000", 2024-09-16T09:39:01.593 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.3285999999999999e-05 2024-09-16T09:39:01.593 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:39:01.593 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:39:01.593 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:39:01.593 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:39:01.582626+0000", 2024-09-16T09:39:01.594 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.4174e-05 2024-09-16T09:39:01.594 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:39:01.594 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:39:01.594 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:39:01.594 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:39:01.582633+0000", 2024-09-16T09:39:01.594 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.2740000000000002e-06 2024-09-16T09:39:01.594 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:39:01.594 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:39:01.594 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:39:01.594 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:39:01.582710+0000", 2024-09-16T09:39:01.594 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.6681000000000007e-05 2024-09-16T09:39:01.595 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:39:01.595 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:39:01.595 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:39:01.595 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:39:01.582728+0000", 2024-09-16T09:39:01.595 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.8236999999999998e-05 2024-09-16T09:39:01.595 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:39:01.595 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:39:01.595 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:39:01.595 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:39:01.595 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-09-16T09:39:01.595 INFO:teuthology.orchestra.run.smithi084.stdout: "num_ops": 2 2024-09-16T09:39:01.595 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-09-16T09:39:01.612 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:39:01.616 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:39:01.686 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:01.687 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:01.688 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:39:01.765 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:01 smithi070 ceph-mon[30388]: pgmap v741: 178 pgs: 4 active+recovering+undersized+remapped, 3 active+recovery_wait+undersized+degraded+remapped, 2 active+remapped+backfill_toofull, 169 active+clean; 5.4 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 801 B/s rd, 55 MiB/s wr, 883 op/s; 13884/266082 objects degraded (5.218%); 22924/266082 objects misplaced (8.615%); 17 MiB/s, 274 objects/s recovering 2024-09-16T09:39:01.765 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:01 smithi070 ceph-mon[40691]: pgmap v741: 178 pgs: 4 active+recovering+undersized+remapped, 3 active+recovery_wait+undersized+degraded+remapped, 2 active+remapped+backfill_toofull, 169 active+clean; 5.4 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 801 B/s rd, 55 MiB/s wr, 883 op/s; 13884/266082 objects degraded (5.218%); 22924/266082 objects misplaced (8.615%); 17 MiB/s, 274 objects/s recovering 2024-09-16T09:39:01.788 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:39:01.811 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T09:39:01.854 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:01.854 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:01.855 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:39:01.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:01 smithi084 ceph-mon[32629]: pgmap v741: 178 pgs: 4 active+recovering+undersized+remapped, 3 active+recovery_wait+undersized+degraded+remapped, 2 active+remapped+backfill_toofull, 169 active+clean; 5.4 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 801 B/s rd, 55 MiB/s wr, 883 op/s; 13884/266082 objects degraded (5.218%); 22924/266082 objects misplaced (8.615%); 17 MiB/s, 274 objects/s recovering 2024-09-16T09:39:01.955 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:39:02.029 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:02.030 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:02.030 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:39:02.053 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:39:01 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:39:01] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:39:02.131 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:39:02.192 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:02.193 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:02.193 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:39:02.293 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:39:02.321 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:02.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:02.322 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:39:02.337 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.6 dump_blocked_ops 2024-09-16T09:39:02.390 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:39:02.423 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:39:02.450 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:02.451 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:02.452 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:39:02.453 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:39:02.553 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:39:02.561 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 66 16 92353 92337 87.4071 55.375 0.0249165 0.0114362 2024-09-16T09:39:02.612 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:02.613 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:02.613 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:39:02.714 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:39:02.728 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:39:02.748 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:02.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:02.749 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:39:02.850 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:39:02.892 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:02.892 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:02.892 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:39:02.993 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:39:03.028 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:03.029 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:03.029 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:39:03.130 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:39:03.158 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:03.159 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:03.159 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:39:03.260 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:39:03.288 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:03.289 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:03.289 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:39:03.345 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-09-16T09:39:03.345 INFO:teuthology.orchestra.run.smithi084.stdout: "ops": [], 2024-09-16T09:39:03.345 INFO:teuthology.orchestra.run.smithi084.stdout: "complaint_time": 30, 2024-09-16T09:39:03.345 INFO:teuthology.orchestra.run.smithi084.stdout: "num_blocked_ops": 0 2024-09-16T09:39:03.345 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-09-16T09:39:03.390 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:39:03.432 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:03.433 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:03.433 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:39:03.448 INFO:teuthology.orchestra.run.smithi070.stderr:nodeep-scrub is set 2024-09-16T09:39:03.455 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:39:03 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:39:03.534 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:39:03.550 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:39:03.554 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2024-09-16T09:39:03.556 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:39:03.556 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:39:03.557 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 67 16 93311 93295 86.996 59.875 0.0101453 0.0114894 2024-09-16T09:39:03.576 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:03.577 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:03.577 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:39:03.599 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:39:03.599 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:39:03.648 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:39:03.648 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:39:03.679 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:39:03.692 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:39:03.692 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:39:03.693 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:03 smithi070 ceph-mon[30388]: pgmap v742: 178 pgs: 4 active+recovering+undersized+remapped, 3 active+recovery_wait+undersized+degraded+remapped, 2 active+remapped+backfill_toofull, 169 active+clean; 5.4 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 639 B/s rd, 42 MiB/s wr, 671 op/s; 13884/266082 objects degraded (5.218%); 22924/266082 objects misplaced (8.615%); 11 MiB/s, 175 objects/s recovering 2024-09-16T09:39:03.693 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:03 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1414097895' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:39:03.693 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:03 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:39:03.693 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:03 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:39:03.693 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:03 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:39:03.693 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:03 smithi070 ceph-mon[40691]: pgmap v742: 178 pgs: 4 active+recovering+undersized+remapped, 3 active+recovery_wait+undersized+degraded+remapped, 2 active+remapped+backfill_toofull, 169 active+clean; 5.4 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 639 B/s rd, 42 MiB/s wr, 671 op/s; 13884/266082 objects degraded (5.218%); 22924/266082 objects misplaced (8.615%); 11 MiB/s, 175 objects/s recovering 2024-09-16T09:39:03.693 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:03 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1414097895' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:39:03.694 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:03 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:39:03.694 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:03 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:39:03.694 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:03 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:39:03.710 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:03.711 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:03.712 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:39:03.726 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:39:03.727 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:39:03.765 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:39:03.765 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:39:03.780 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:03 smithi084 ceph-mon[32629]: pgmap v742: 178 pgs: 4 active+recovering+undersized+remapped, 3 active+recovery_wait+undersized+degraded+remapped, 2 active+remapped+backfill_toofull, 169 active+clean; 5.4 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 639 B/s rd, 42 MiB/s wr, 671 op/s; 13884/266082 objects degraded (5.218%); 22924/266082 objects misplaced (8.615%); 11 MiB/s, 175 objects/s recovering 2024-09-16T09:39:03.780 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:03 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1414097895' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:39:03.780 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:03 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:39:03.780 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:03 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:39:03.780 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:03 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:39:03.805 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:39:03.806 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:39:03.813 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:39:03.843 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:39:03.843 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:39:03.849 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:03.850 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:03.850 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:39:03.951 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:39:03.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:03.998 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:03.998 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:39:04.099 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:39:04.141 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:04.141 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:04.142 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:39:04.233 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.6 dump_historic_ops 2024-09-16T09:39:04.243 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:39:04.317 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:04.317 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:04.318 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:39:04.419 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:39:04.459 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:04.460 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:04.460 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:39:04.557 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 68 16 94412 94396 86.7286 68.8125 0.00665111 0.011526 2024-09-16T09:39:04.561 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:39:04.599 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:04.599 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:04.600 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:39:04.638 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:39:04.701 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:39:04.742 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:04.742 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:04.743 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:39:04.780 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:04 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:39:04.780 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:04 smithi084 ceph-mon[32629]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:39:04.780 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:04 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1414097895' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-16T09:39:04.780 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:04 smithi084 ceph-mon[32629]: osdmap e408: 8 total, 8 up, 8 in 2024-09-16T09:39:04.780 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:04 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/472319244' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:39:04.780 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:04 smithi084 ceph-mon[32629]: from='client.26018 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:39:04.780 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:04 smithi084 ceph-mon[32629]: Health check update: Degraded data redundancy: 13884/266082 objects degraded (5.218%), 3 pgs degraded (PG_DEGRADED) 2024-09-16T09:39:04.780 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:04 smithi084 ceph-mon[32629]: osdmap e409: 8 total, 8 up, 8 in 2024-09-16T09:39:04.844 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:39:04.872 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:04 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:39:04.873 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:04 smithi070 ceph-mon[30388]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:39:04.873 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:04 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1414097895' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-16T09:39:04.873 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:04 smithi070 ceph-mon[30388]: osdmap e408: 8 total, 8 up, 8 in 2024-09-16T09:39:04.873 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:04 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/472319244' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:39:04.873 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:04 smithi070 ceph-mon[30388]: from='client.26018 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:39:04.873 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:04 smithi070 ceph-mon[30388]: Health check update: Degraded data redundancy: 13884/266082 objects degraded (5.218%), 3 pgs degraded (PG_DEGRADED) 2024-09-16T09:39:04.873 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:04 smithi070 ceph-mon[30388]: osdmap e409: 8 total, 8 up, 8 in 2024-09-16T09:39:04.873 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:04 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:39:04.873 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:04 smithi070 ceph-mon[40691]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:39:04.873 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:04 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1414097895' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-16T09:39:04.873 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:04 smithi070 ceph-mon[40691]: osdmap e408: 8 total, 8 up, 8 in 2024-09-16T09:39:04.873 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:04 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/472319244' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:39:04.874 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:04 smithi070 ceph-mon[40691]: from='client.26018 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:39:04.874 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:04 smithi070 ceph-mon[40691]: Health check update: Degraded data redundancy: 13884/266082 objects degraded (5.218%), 3 pgs degraded (PG_DEGRADED) 2024-09-16T09:39:04.874 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:04 smithi070 ceph-mon[40691]: osdmap e409: 8 total, 8 up, 8 in 2024-09-16T09:39:04.887 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:04.888 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:04.888 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:39:04.989 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:39:05.029 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:05.030 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:05.030 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:39:05.131 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:39:05.167 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:05.168 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:05.168 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:39:05.269 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:39:05.311 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:05.312 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:05.312 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:39:05.413 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:39:05.446 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:05.446 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:05.447 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:39:05.449 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":743,"stamp":"2024-09-16T09:39:03.438796+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":6024796699,"num_objects":92123,"num_object_clones":0,"num_object_copies":276369,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13884,"num_objects_misplaced":22258,"num_objects_unfound":0,"num_objects_dirty":92123,"num_whiteouts":0,"num_read":1975,"num_read_kb":1727,"num_write":69484,"num_write_kb":4400685,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6919,"num_bytes_recovered":444405178,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":130982,"ondisk_log_size":130982,"up":534,"acting":527,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":553,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":22772128,"kb_used_data":17994012,"kb_used_omap":12,"kb_used_meta":4777971,"kb_avail":727189088,"statfs":{"total":767960285184,"available":744641626112,"internally_reserved":0,"allocated":18425868288,"data_stored":18419984617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":4892642866},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,4,2,13,10,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":528089088,"num_objects":8058,"num_object_clones":0,"num_object_copies":24174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-1932,"num_objects_unfound":0,"num_objects_dirty":8058,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":8058,"num_write_kb":515712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1543,"num_bytes_recovered":101122048,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.054169"},"pg_stats":[{"pgid":"10.16","version":"407'4956","reported_seq":7408,"reported_epoch":407,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-16T09:38:59.382933+0000","last_change":"2024-09-16T09:38:38.851838+0000","last_active":"2024-09-16T09:38:59.382933+0000","last_peered":"2024-09-16T09:38:59.382933+0000","last_clean":"2024-09-16T09:38:36.479625+0000","last_became_active":"2024-09-16T09:38:38.801008+0000","last_became_peered":"2024-09-16T09:38:38.801008+0000","last_unstale":"2024-09-16T09:38:59.382933+0000","last_undegraded":"2024-09-16T09:38:59.382933+0000","last_fullsized":"2024-09-16T09:38:38.529049+0000","mapping_epoch":398,"log_start":"0'0","ondisk_log_start":"0'0","created":395,"last_epoch_clean":396,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_clean_scrub_stamp":"2024-09-16T09:37:44.378497+0000","objects_scrubbed":0,"log_size":4956,"log_dups_size":0,"ondisk_log_size":4956,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":179961856,"num_objects":2746,"num_object_clones":0,"num_object_copies":8238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2998,"num_objects_unfound":0,"num_objects_dirty":2746,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2746,"num_write_kb":175744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":789,"num_bytes_recovered":51707904,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[0,5],"avail_no_missing":["0","4","5"],"object_location_counts":[{"shards":"0,4,5","objects":1499},{"shards":"0,5","objects":1247}],"blocked_by":[],"up_primary":6,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":746,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.426405+0000","last_change":"2024-09-16T09:32:39.384174+0000","last_active":"2024-09-16T09:38:56.426405+0000","last_peered":"2024-09-16T09:38:56.426405+0000","last_clean":"2024-09-16T09:38:56.426405+0000","last_became_active":"2024-09-16T09:25:01.709108+0000","last_became_peered":"2024-09-16T09:25:01.709108+0000","last_unstale":"2024-09-16T09:38:56.426405+0000","last_undegraded":"2024-09-16T09:38:56.426405+0000","last_fullsized":"2024-09-16T09:38:56.426405+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:39.384104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:32:39.384104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:20.331363+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"285'15","reported_seq":758,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.431264+0000","last_change":"2024-09-16T09:32:38.074301+0000","last_active":"2024-09-16T09:38:56.431264+0000","last_peered":"2024-09-16T09:38:56.431264+0000","last_clean":"2024-09-16T09:38:56.431264+0000","last_became_active":"2024-09-16T09:25:03.714635+0000","last_became_peered":"2024-09-16T09:25:03.714635+0000","last_unstale":"2024-09-16T09:38:56.431264+0000","last_undegraded":"2024-09-16T09:38:56.431264+0000","last_fullsized":"2024-09-16T09:38:56.431264+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:32:38.074231+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:32:38.074231+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:07.303929+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":369,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.446747+0000","last_change":"2024-09-16T09:36:53.098680+0000","last_active":"2024-09-16T09:38:56.446747+0000","last_peered":"2024-09-16T09:38:56.446747+0000","last_clean":"2024-09-16T09:38:56.446747+0000","last_became_active":"2024-09-16T09:33:30.242238+0000","last_became_peered":"2024-09-16T09:33:30.242238+0000","last_unstale":"2024-09-16T09:38:56.446747+0000","last_undegraded":"2024-09-16T09:38:56.446747+0000","last_fullsized":"2024-09-16T09:38:56.446747+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:53.098635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:36:53.098635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":725,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.433335+0000","last_change":"2024-09-16T09:37:56.314518+0000","last_active":"2024-09-16T09:38:56.433335+0000","last_peered":"2024-09-16T09:38:56.433335+0000","last_clean":"2024-09-16T09:38:56.433335+0000","last_became_active":"2024-09-16T09:25:07.715442+0000","last_became_peered":"2024-09-16T09:25:07.715442+0000","last_unstale":"2024-09-16T09:38:56.433335+0000","last_undegraded":"2024-09-16T09:38:56.433335+0000","last_fullsized":"2024-09-16T09:38:56.433335+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:56.314468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:37:56.314468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:04.475570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.17","version":"407'5093","reported_seq":8381,"reported_epoch":407,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-16T09:38:58.514615+0000","last_change":"2024-09-16T09:38:38.655048+0000","last_active":"2024-09-16T09:38:58.514615+0000","last_peered":"2024-09-16T09:38:58.514615+0000","last_clean":"2024-09-16T09:38:36.498579+0000","last_became_active":"2024-09-16T09:38:38.627104+0000","last_became_peered":"2024-09-16T09:38:38.627104+0000","last_unstale":"2024-09-16T09:38:58.514615+0000","last_undegraded":"2024-09-16T09:38:58.514615+0000","last_fullsized":"2024-09-16T09:38:38.529232+0000","mapping_epoch":398,"log_start":"0'0","ondisk_log_start":"0'0","created":395,"last_epoch_clean":396,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_clean_scrub_stamp":"2024-09-16T09:37:44.378497+0000","objects_scrubbed":0,"log_size":5093,"log_dups_size":0,"ondisk_log_size":5093,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":182681600,"num_objects":2788,"num_object_clones":0,"num_object_copies":8364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":764,"num_objects_unfound":0,"num_objects_dirty":2788,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2788,"num_write_kb":178400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1603,"num_bytes_recovered":105054208,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[3,6],"avail_no_missing":["3","5","6"],"object_location_counts":[{"shards":"3,5,6","objects":764},{"shards":"3,6","objects":2024}],"blocked_by":[],"up_primary":4,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":463,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.437676+0000","last_change":"2024-09-16T09:34:33.452784+0000","last_active":"2024-09-16T09:38:56.437676+0000","last_peered":"2024-09-16T09:38:56.437676+0000","last_clean":"2024-09-16T09:38:56.437676+0000","last_became_active":"2024-09-16T09:31:09.269342+0000","last_became_peered":"2024-09-16T09:31:09.269342+0000","last_unstale":"2024-09-16T09:38:56.437676+0000","last_undegraded":"2024-09-16T09:38:56.437676+0000","last_fullsized":"2024-09-16T09:38:56.437676+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:33.452706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:34:33.452706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:15.084990+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":604,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.439417+0000","last_change":"2024-09-16T09:33:08.816231+0000","last_active":"2024-09-16T09:38:56.439417+0000","last_peered":"2024-09-16T09:38:56.439417+0000","last_clean":"2024-09-16T09:38:56.439417+0000","last_became_active":"2024-09-16T09:31:09.275136+0000","last_became_peered":"2024-09-16T09:31:09.275136+0000","last_unstale":"2024-09-16T09:38:56.439417+0000","last_undegraded":"2024-09-16T09:38:56.439417+0000","last_fullsized":"2024-09-16T09:38:56.439417+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:08.816174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:08.816174+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:08.989282+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.19","version":"285'27","reported_seq":444,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.426918+0000","last_change":"2024-09-16T09:37:29.683657+0000","last_active":"2024-09-16T09:38:56.426918+0000","last_peered":"2024-09-16T09:38:56.426918+0000","last_clean":"2024-09-16T09:38:56.426918+0000","last_became_active":"2024-09-16T09:33:21.337284+0000","last_became_peered":"2024-09-16T09:33:21.337284+0000","last_unstale":"2024-09-16T09:38:56.426918+0000","last_undegraded":"2024-09-16T09:38:56.426918+0000","last_fullsized":"2024-09-16T09:38:56.426918+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'27","last_scrub_stamp":"2024-09-16T09:37:29.683568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:37:29.683568+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":737,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.431158+0000","last_change":"2024-09-16T09:32:44.114475+0000","last_active":"2024-09-16T09:38:56.431158+0000","last_peered":"2024-09-16T09:38:56.431158+0000","last_clean":"2024-09-16T09:38:56.431158+0000","last_became_active":"2024-09-16T09:25:05.717236+0000","last_became_peered":"2024-09-16T09:25:05.717236+0000","last_unstale":"2024-09-16T09:38:56.431158+0000","last_undegraded":"2024-09-16T09:38:56.431158+0000","last_fullsized":"2024-09-16T09:38:56.431158+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:44.114407+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:32:44.114407+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:03.327373+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.14","version":"407'5222","reported_seq":5320,"reported_epoch":407,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T09:39:02.901350+0000","last_change":"2024-09-16T09:38:41.875801+0000","last_active":"2024-09-16T09:39:02.901350+0000","last_peered":"2024-09-16T09:39:02.901350+0000","last_clean":"2024-09-16T09:38:36.507866+0000","last_became_active":"2024-09-16T09:38:41.846136+0000","last_became_peered":"2024-09-16T09:38:41.846136+0000","last_unstale":"2024-09-16T09:39:02.901350+0000","last_undegraded":"2024-09-16T09:38:41.100757+0000","last_fullsized":"2024-09-16T09:38:41.098050+0000","mapping_epoch":398,"log_start":"0'0","ondisk_log_start":"0'0","created":395,"last_epoch_clean":396,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_clean_scrub_stamp":"2024-09-16T09:37:44.378497+0000","objects_scrubbed":0,"log_size":5222,"log_dups_size":0,"ondisk_log_size":5222,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":191758336,"num_objects":2926,"num_object_clones":0,"num_object_copies":8778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4686,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2926,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2926,"num_write_kb":187264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":262144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1],"avail_no_missing":["4","0","3"],"object_location_counts":[{"shards":"0,3,4","objects":2343},{"shards":"1,4","objects":583}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":370,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.433633+0000","last_change":"2024-09-16T09:33:36.655594+0000","last_active":"2024-09-16T09:38:56.433633+0000","last_peered":"2024-09-16T09:38:56.433633+0000","last_clean":"2024-09-16T09:38:56.433633+0000","last_became_active":"2024-09-16T09:33:30.247733+0000","last_became_peered":"2024-09-16T09:33:30.247733+0000","last_unstale":"2024-09-16T09:38:56.433633+0000","last_undegraded":"2024-09-16T09:38:56.433633+0000","last_fullsized":"2024-09-16T09:38:56.433633+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:36.655236+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:36.655236+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:00.446453+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1a","version":"285'15","reported_seq":758,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.433658+0000","last_change":"2024-09-16T09:32:48.185484+0000","last_active":"2024-09-16T09:38:56.433658+0000","last_peered":"2024-09-16T09:38:56.433658+0000","last_clean":"2024-09-16T09:38:56.433658+0000","last_became_active":"2024-09-16T09:25:03.709147+0000","last_became_peered":"2024-09-16T09:25:03.709147+0000","last_unstale":"2024-09-16T09:38:56.433658+0000","last_undegraded":"2024-09-16T09:38:56.433658+0000","last_fullsized":"2024-09-16T09:38:56.433658+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:32:48.185380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:32:48.185380+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:12.337650+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":728,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.445649+0000","last_change":"2024-09-16T09:36:51.134364+0000","last_active":"2024-09-16T09:38:56.445649+0000","last_peered":"2024-09-16T09:38:56.445649+0000","last_clean":"2024-09-16T09:38:56.445649+0000","last_became_active":"2024-09-16T09:25:05.718134+0000","last_became_peered":"2024-09-16T09:25:05.718134+0000","last_unstale":"2024-09-16T09:38:56.445649+0000","last_undegraded":"2024-09-16T09:38:56.445649+0000","last_fullsized":"2024-09-16T09:38:56.445649+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:51.134314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:36:51.134314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"66'1","reported_seq":742,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.424338+0000","last_change":"2024-09-16T09:38:24.448521+0000","last_active":"2024-09-16T09:38:56.424338+0000","last_peered":"2024-09-16T09:38:56.424338+0000","last_clean":"2024-09-16T09:38:56.424338+0000","last_became_active":"2024-09-16T09:25:07.716558+0000","last_became_peered":"2024-09-16T09:25:07.716558+0000","last_unstale":"2024-09-16T09:38:56.424338+0000","last_undegraded":"2024-09-16T09:38:56.424338+0000","last_fullsized":"2024-09-16T09:38:56.424338+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-16T09:38:24.448344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:38:24.448344+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:52.609054+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.15","version":"407'5081","reported_seq":5164,"reported_epoch":407,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T09:38:58.478144+0000","last_change":"2024-09-16T09:38:38.821534+0000","last_active":"2024-09-16T09:38:58.478144+0000","last_peered":"2024-09-16T09:38:58.478144+0000","last_clean":"2024-09-16T09:38:36.402266+0000","last_became_active":"2024-09-16T09:38:38.811414+0000","last_became_peered":"2024-09-16T09:38:38.811414+0000","last_unstale":"2024-09-16T09:38:58.478144+0000","last_undegraded":"2024-09-16T09:38:38.667496+0000","last_fullsized":"2024-09-16T09:38:38.524975+0000","mapping_epoch":398,"log_start":"0'0","ondisk_log_start":"0'0","created":395,"last_epoch_clean":396,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_clean_scrub_stamp":"2024-09-16T09:37:44.378497+0000","objects_scrubbed":0,"log_size":5081,"log_dups_size":0,"ondisk_log_size":5081,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":181633024,"num_objects":2772,"num_object_clones":0,"num_object_copies":8316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4616,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2772,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2772,"num_write_kb":177376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,1],"avail_no_missing":["3","0","2"],"object_location_counts":[{"shards":"0,2,3","objects":2308},{"shards":"1,3","objects":464}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":728,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.444761+0000","last_change":"2024-09-16T09:33:19.326765+0000","last_active":"2024-09-16T09:38:56.444761+0000","last_peered":"2024-09-16T09:38:56.444761+0000","last_clean":"2024-09-16T09:38:56.444761+0000","last_became_active":"2024-09-16T09:25:01.696939+0000","last_became_peered":"2024-09-16T09:25:01.696939+0000","last_unstale":"2024-09-16T09:38:56.444761+0000","last_undegraded":"2024-09-16T09:38:56.444761+0000","last_fullsized":"2024-09-16T09:38:56.444761+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:19.326695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:19.326695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:11.230887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"285'7","reported_seq":746,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.435003+0000","last_change":"2024-09-16T09:35:39.975915+0000","last_active":"2024-09-16T09:38:56.435003+0000","last_peered":"2024-09-16T09:38:56.435003+0000","last_clean":"2024-09-16T09:38:56.435003+0000","last_became_active":"2024-09-16T09:25:03.716136+0000","last_became_peered":"2024-09-16T09:25:03.716136+0000","last_unstale":"2024-09-16T09:38:56.435003+0000","last_undegraded":"2024-09-16T09:38:56.435003+0000","last_fullsized":"2024-09-16T09:38:56.435003+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'7","last_scrub_stamp":"2024-09-16T09:35:39.975840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:35:39.975840+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:00.297013+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":10,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":719,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.430149+0000","last_change":"2024-09-16T09:28:52.746046+0000","last_active":"2024-09-16T09:38:56.430149+0000","last_peered":"2024-09-16T09:38:56.430149+0000","last_clean":"2024-09-16T09:38:56.430149+0000","last_became_active":"2024-09-16T09:25:05.718281+0000","last_became_peered":"2024-09-16T09:25:05.718281+0000","last_unstale":"2024-09-16T09:38:56.430149+0000","last_undegraded":"2024-09-16T09:38:56.430149+0000","last_fullsized":"2024-09-16T09:38:56.430149+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:28:52.745999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:28:52.745999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:04.420096+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":754,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.446822+0000","last_change":"2024-09-16T09:38:34.050374+0000","last_active":"2024-09-16T09:38:56.446822+0000","last_peered":"2024-09-16T09:38:56.446822+0000","last_clean":"2024-09-16T09:38:56.446822+0000","last_became_active":"2024-09-16T09:31:09.275656+0000","last_became_peered":"2024-09-16T09:31:09.275656+0000","last_unstale":"2024-09-16T09:38:56.446822+0000","last_undegraded":"2024-09-16T09:38:56.446822+0000","last_fullsized":"2024-09-16T09:38:56.446822+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:38:34.050185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:38:34.050185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:49.691046+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"407'5104","reported_seq":5193,"reported_epoch":407,"state":"active+remapped+backfill_toofull","last_fresh":"2024-09-16T09:38:59.245517+0000","last_change":"2024-09-16T09:38:38.663476+0000","last_active":"2024-09-16T09:38:59.245517+0000","last_peered":"2024-09-16T09:38:59.245517+0000","last_clean":"2024-09-16T09:38:36.435420+0000","last_became_active":"2024-09-16T09:38:38.654785+0000","last_became_peered":"2024-09-16T09:38:38.654785+0000","last_unstale":"2024-09-16T09:38:59.245517+0000","last_undegraded":"2024-09-16T09:38:59.245517+0000","last_fullsized":"2024-09-16T09:38:59.245517+0000","mapping_epoch":398,"log_start":"383'400","ondisk_log_start":"383'400","created":395,"last_epoch_clean":396,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_clean_scrub_stamp":"2024-09-16T09:37:44.378497+0000","objects_scrubbed":0,"log_size":4704,"log_dups_size":0,"ondisk_log_size":4704,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":184713216,"num_objects":2819,"num_object_clones":0,"num_object_copies":8457,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":8457,"num_objects_unfound":0,"num_objects_dirty":2819,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2819,"num_write_kb":180384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,5],"acting":[0,1,4],"avail_no_missing":["0","1","4"],"object_location_counts":[{"shards":"0,1,4","objects":2819}],"blocked_by":[],"up_primary":6,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":734,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.440635+0000","last_change":"2024-09-16T09:33:48.322010+0000","last_active":"2024-09-16T09:38:56.440635+0000","last_peered":"2024-09-16T09:38:56.440635+0000","last_clean":"2024-09-16T09:38:56.440635+0000","last_became_active":"2024-09-16T09:25:07.720793+0000","last_became_peered":"2024-09-16T09:25:07.720793+0000","last_unstale":"2024-09-16T09:38:56.440635+0000","last_undegraded":"2024-09-16T09:38:56.440635+0000","last_fullsized":"2024-09-16T09:38:56.440635+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:48.321895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:33:48.321895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":485,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.428432+0000","last_change":"2024-09-16T09:34:33.866299+0000","last_active":"2024-09-16T09:38:56.428432+0000","last_peered":"2024-09-16T09:38:56.428432+0000","last_clean":"2024-09-16T09:38:56.428432+0000","last_became_active":"2024-09-16T09:30:39.201824+0000","last_became_peered":"2024-09-16T09:30:39.201824+0000","last_unstale":"2024-09-16T09:38:56.428432+0000","last_undegraded":"2024-09-16T09:38:56.428432+0000","last_fullsized":"2024-09-16T09:38:56.428432+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:33.866142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:34:33.866142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"285'27","reported_seq":833,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.442320+0000","last_change":"2024-09-16T09:36:54.484021+0000","last_active":"2024-09-16T09:38:56.442320+0000","last_peered":"2024-09-16T09:38:56.442320+0000","last_clean":"2024-09-16T09:38:56.442320+0000","last_became_active":"2024-09-16T09:33:30.249798+0000","last_became_peered":"2024-09-16T09:33:30.249798+0000","last_unstale":"2024-09-16T09:38:56.442320+0000","last_undegraded":"2024-09-16T09:38:56.442320+0000","last_fullsized":"2024-09-16T09:38:56.442320+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'27","last_scrub_stamp":"2024-09-16T09:36:54.483966+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:36:54.483966+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:07.794557+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":369,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.435645+0000","last_change":"2024-09-16T09:33:43.326933+0000","last_active":"2024-09-16T09:38:56.435645+0000","last_peered":"2024-09-16T09:38:56.435645+0000","last_clean":"2024-09-16T09:38:56.435645+0000","last_became_active":"2024-09-16T09:33:30.251587+0000","last_became_peered":"2024-09-16T09:33:30.251587+0000","last_unstale":"2024-09-16T09:38:56.435645+0000","last_undegraded":"2024-09-16T09:38:56.435645+0000","last_fullsized":"2024-09-16T09:38:56.435645+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:43.326852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:43.326852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,0],"acting":[4,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.13","version":"407'5322","reported_seq":8627,"reported_epoch":407,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-16T09:39:03.126978+0000","last_change":"2024-09-16T09:38:38.840790+0000","last_active":"2024-09-16T09:39:03.126978+0000","last_peered":"2024-09-16T09:39:03.126978+0000","last_clean":"2024-09-16T09:38:36.484757+0000","last_became_active":"2024-09-16T09:38:38.794938+0000","last_became_peered":"2024-09-16T09:38:38.794938+0000","last_unstale":"2024-09-16T09:39:03.126978+0000","last_undegraded":"2024-09-16T09:39:03.126978+0000","last_fullsized":"2024-09-16T09:38:38.527225+0000","mapping_epoch":398,"log_start":"0'0","ondisk_log_start":"0'0","created":395,"last_epoch_clean":396,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_clean_scrub_stamp":"2024-09-16T09:37:44.378497+0000","objects_scrubbed":0,"log_size":5322,"log_dups_size":0,"ondisk_log_size":5322,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":197001216,"num_objects":3006,"num_object_clones":0,"num_object_copies":9018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2504,"num_objects_unfound":0,"num_objects_dirty":3006,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3006,"num_write_kb":192384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1074,"num_bytes_recovered":70385664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[2,1],"avail_no_missing":["2","3","5"],"object_location_counts":[{"shards":"1,2","objects":1754},{"shards":"2,3,5","objects":1252}],"blocked_by":[],"up_primary":7,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":582,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.439649+0000","last_change":"2024-09-16T09:34:37.804874+0000","last_active":"2024-09-16T09:38:56.439649+0000","last_peered":"2024-09-16T09:38:56.439649+0000","last_clean":"2024-09-16T09:38:56.439649+0000","last_became_active":"2024-09-16T09:28:52.974255+0000","last_became_peered":"2024-09-16T09:28:52.974255+0000","last_unstale":"2024-09-16T09:38:56.439649+0000","last_undegraded":"2024-09-16T09:38:56.439649+0000","last_fullsized":"2024-09-16T09:38:56.439649+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:37.804805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:34:37.804805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:00.917819+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":745,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.433797+0000","last_change":"2024-09-16T09:35:23.751992+0000","last_active":"2024-09-16T09:38:56.433797+0000","last_peered":"2024-09-16T09:38:56.433797+0000","last_clean":"2024-09-16T09:38:56.433797+0000","last_became_active":"2024-09-16T09:25:01.706491+0000","last_became_peered":"2024-09-16T09:25:01.706491+0000","last_unstale":"2024-09-16T09:38:56.433797+0000","last_undegraded":"2024-09-16T09:38:56.433797+0000","last_fullsized":"2024-09-16T09:38:56.433797+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:35:23.751946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:35:23.751946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:15.256399+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"285'18","reported_seq":421,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.435789+0000","last_change":"2024-09-16T09:33:39.284526+0000","last_active":"2024-09-16T09:38:56.435789+0000","last_peered":"2024-09-16T09:38:56.435789+0000","last_clean":"2024-09-16T09:38:56.435789+0000","last_became_active":"2024-09-16T09:33:30.251626+0000","last_became_peered":"2024-09-16T09:33:30.251626+0000","last_unstale":"2024-09-16T09:38:56.435789+0000","last_undegraded":"2024-09-16T09:38:56.435789+0000","last_fullsized":"2024-09-16T09:38:56.435789+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'12","last_scrub_stamp":"2024-09-16T09:33:39.284451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:33:39.284451+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":33,"num_read_kb":24,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":721,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.436366+0000","last_change":"2024-09-16T09:33:38.238661+0000","last_active":"2024-09-16T09:38:56.436366+0000","last_peered":"2024-09-16T09:38:56.436366+0000","last_clean":"2024-09-16T09:38:56.436366+0000","last_became_active":"2024-09-16T09:25:05.710093+0000","last_became_peered":"2024-09-16T09:25:05.710093+0000","last_unstale":"2024-09-16T09:38:56.436366+0000","last_undegraded":"2024-09-16T09:38:56.436366+0000","last_fullsized":"2024-09-16T09:38:56.436366+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:38.238593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:38.238593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.10","version":"407'5043","reported_seq":9631,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:39:02.946727+0000","last_change":"2024-09-16T09:38:55.151349+0000","last_active":"2024-09-16T09:39:02.946727+0000","last_peered":"2024-09-16T09:39:02.946727+0000","last_clean":"2024-09-16T09:39:02.946727+0000","last_became_active":"2024-09-16T09:38:54.391768+0000","last_became_peered":"2024-09-16T09:38:54.391768+0000","last_unstale":"2024-09-16T09:39:02.946727+0000","last_undegraded":"2024-09-16T09:39:02.946727+0000","last_fullsized":"2024-09-16T09:39:02.946727+0000","mapping_epoch":405,"log_start":"0'0","ondisk_log_start":"0'0","created":395,"last_epoch_clean":406,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_clean_scrub_stamp":"2024-09-16T09:37:44.378497+0000","objects_scrubbed":0,"log_size":5043,"log_dups_size":0,"ondisk_log_size":5043,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:39:25.087294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":186122240,"num_objects":2840,"num_object_clones":0,"num_object_copies":8520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2840,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2840,"num_write_kb":181760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2248,"num_bytes_recovered":147324928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":732,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.430573+0000","last_change":"2024-09-16T09:37:48.412421+0000","last_active":"2024-09-16T09:38:56.430573+0000","last_peered":"2024-09-16T09:38:56.430573+0000","last_clean":"2024-09-16T09:38:56.430573+0000","last_became_active":"2024-09-16T09:25:07.719380+0000","last_became_peered":"2024-09-16T09:25:07.719380+0000","last_unstale":"2024-09-16T09:38:56.430573+0000","last_undegraded":"2024-09-16T09:38:56.430573+0000","last_fullsized":"2024-09-16T09:38:56.430573+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-16T09:37:48.411054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:37:48.411054+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:01.043235+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":378,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.438173+0000","last_change":"2024-09-16T09:34:41.401691+0000","last_active":"2024-09-16T09:38:56.438173+0000","last_peered":"2024-09-16T09:38:56.438173+0000","last_clean":"2024-09-16T09:38:56.438173+0000","last_became_active":"2024-09-16T09:33:30.245048+0000","last_became_peered":"2024-09-16T09:33:30.245048+0000","last_unstale":"2024-09-16T09:38:56.438173+0000","last_undegraded":"2024-09-16T09:38:56.438173+0000","last_fullsized":"2024-09-16T09:38:56.438173+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:34:41.401641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:34:41.401641+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:03.993235+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"285'20","reported_seq":802,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.425832+0000","last_change":"2024-09-16T09:32:30.286077+0000","last_active":"2024-09-16T09:38:56.425832+0000","last_peered":"2024-09-16T09:38:56.425832+0000","last_clean":"2024-09-16T09:38:56.425832+0000","last_became_active":"2024-09-16T09:31:09.275050+0000","last_became_peered":"2024-09-16T09:31:09.275050+0000","last_unstale":"2024-09-16T09:38:56.425832+0000","last_undegraded":"2024-09-16T09:38:56.425832+0000","last_fullsized":"2024-09-16T09:38:56.425832+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-16T09:32:30.284919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:32:30.284919+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:02.456876+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"66'8","reported_seq":934,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.920075+0000","last_change":"2024-09-16T09:34:52.922123+0000","last_active":"2024-09-16T09:38:56.920075+0000","last_peered":"2024-09-16T09:38:56.920075+0000","last_clean":"2024-09-16T09:38:56.920075+0000","last_became_active":"2024-09-16T09:25:05.712516+0000","last_became_peered":"2024-09-16T09:25:05.712516+0000","last_unstale":"2024-09-16T09:38:56.920075+0000","last_undegraded":"2024-09-16T09:38:56.920075+0000","last_fullsized":"2024-09-16T09:38:56.920075+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:34:52.922047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:34:52.922047+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:17.998821+0000","scrub_duration":33,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1","version":"407'5217","reported_seq":5330,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:39:02.973836+0000","last_change":"2024-09-16T09:38:37.482511+0000","last_active":"2024-09-16T09:39:02.973836+0000","last_peered":"2024-09-16T09:39:02.973836+0000","last_clean":"2024-09-16T09:39:02.973836+0000","last_became_active":"2024-09-16T09:38:35.385153+0000","last_became_peered":"2024-09-16T09:38:35.385153+0000","last_unstale":"2024-09-16T09:39:02.973836+0000","last_undegraded":"2024-09-16T09:39:02.973836+0000","last_fullsized":"2024-09-16T09:39:02.973836+0000","mapping_epoch":376,"log_start":"383'300","ondisk_log_start":"383'300","created":376,"last_epoch_clean":396,"parent":"0.0","parent_split_bits":0,"last_scrub":"396'4603","last_scrub_stamp":"2024-09-16T09:38:37.482451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_clean_scrub_stamp":"2024-09-16T09:38:37.482451+0000","objects_scrubbed":2306,"log_size":4917,"log_dups_size":0,"ondisk_log_size":4917,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:46.576061+0000","scrub_duration":288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":191365120,"num_objects":2920,"num_object_clones":0,"num_object_copies":8760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2920,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":614,"num_write_kb":39296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.a","version":"73'79","reported_seq":668,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.447196+0000","last_change":"2024-09-16T09:37:52.203798+0000","last_active":"2024-09-16T09:38:56.447196+0000","last_peered":"2024-09-16T09:38:56.447196+0000","last_clean":"2024-09-16T09:38:56.447196+0000","last_became_active":"2024-09-16T09:26:35.127352+0000","last_became_peered":"2024-09-16T09:26:35.127352+0000","last_unstale":"2024-09-16T09:38:56.447196+0000","last_undegraded":"2024-09-16T09:38:56.447196+0000","last_fullsized":"2024-09-16T09:38:56.447196+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:37:52.203752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:37:52.203752+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:59.150165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"285'27","reported_seq":695,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.443181+0000","last_change":"2024-09-16T09:36:47.588508+0000","last_active":"2024-09-16T09:38:56.443181+0000","last_peered":"2024-09-16T09:38:56.443181+0000","last_clean":"2024-09-16T09:38:56.443181+0000","last_became_active":"2024-09-16T09:26:33.295672+0000","last_became_peered":"2024-09-16T09:26:33.295672+0000","last_unstale":"2024-09-16T09:38:56.443181+0000","last_undegraded":"2024-09-16T09:38:56.443181+0000","last_fullsized":"2024-09-16T09:38:56.443181+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'27","last_scrub_stamp":"2024-09-16T09:36:47.588449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:36:47.588449+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:14.953315+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":479,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.435862+0000","last_change":"2024-09-16T09:38:35.023594+0000","last_active":"2024-09-16T09:38:56.435862+0000","last_peered":"2024-09-16T09:38:56.435862+0000","last_clean":"2024-09-16T09:38:56.435862+0000","last_became_active":"2024-09-16T09:31:09.274534+0000","last_became_peered":"2024-09-16T09:31:09.274534+0000","last_unstale":"2024-09-16T09:38:56.435862+0000","last_undegraded":"2024-09-16T09:38:56.435862+0000","last_fullsized":"2024-09-16T09:38:56.435862+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:38:35.023518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:38:35.023518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:51.685265+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"66'8","reported_seq":899,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.921723+0000","last_change":"2024-09-16T09:37:49.336009+0000","last_active":"2024-09-16T09:38:56.921723+0000","last_peered":"2024-09-16T09:38:56.921723+0000","last_clean":"2024-09-16T09:38:56.921723+0000","last_became_active":"2024-09-16T09:25:05.713813+0000","last_became_peered":"2024-09-16T09:25:05.713813+0000","last_unstale":"2024-09-16T09:38:56.921723+0000","last_undegraded":"2024-09-16T09:38:56.921723+0000","last_fullsized":"2024-09-16T09:38:56.921723+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:37:49.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:37:49.335939+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:08.862747+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":733,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.445484+0000","last_change":"2024-09-16T09:38:35.081365+0000","last_active":"2024-09-16T09:38:56.445484+0000","last_peered":"2024-09-16T09:38:56.445484+0000","last_clean":"2024-09-16T09:38:56.445484+0000","last_became_active":"2024-09-16T09:25:07.717344+0000","last_became_peered":"2024-09-16T09:25:07.717344+0000","last_unstale":"2024-09-16T09:38:56.445484+0000","last_undegraded":"2024-09-16T09:38:56.445484+0000","last_fullsized":"2024-09-16T09:38:56.445484+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:38:35.081321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:38:35.081321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:41.294808+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.e","version":"407'5865","reported_seq":5941,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:39:02.970020+0000","last_change":"2024-09-16T09:37:45.464349+0000","last_active":"2024-09-16T09:39:02.970020+0000","last_peered":"2024-09-16T09:39:02.970020+0000","last_clean":"2024-09-16T09:39:02.970020+0000","last_became_active":"2024-09-16T09:37:45.463961+0000","last_became_peered":"2024-09-16T09:37:45.463961+0000","last_unstale":"2024-09-16T09:39:02.970020+0000","last_undegraded":"2024-09-16T09:39:02.970020+0000","last_fullsized":"2024-09-16T09:39:02.970020+0000","mapping_epoch":376,"log_start":"385'1985","ondisk_log_start":"385'1985","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_clean_scrub_stamp":"2024-09-16T09:37:44.378497+0000","objects_scrubbed":0,"log_size":3880,"log_dups_size":0,"ondisk_log_size":3880,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:04.043613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":384368640,"num_objects":5865,"num_object_clones":0,"num_object_copies":17595,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5865,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5865,"num_write_kb":375360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"73'79","reported_seq":700,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.436149+0000","last_change":"2024-09-16T09:32:42.123139+0000","last_active":"2024-09-16T09:38:56.436149+0000","last_peered":"2024-09-16T09:38:56.436149+0000","last_clean":"2024-09-16T09:38:56.436149+0000","last_became_active":"2024-09-16T09:30:08.763683+0000","last_became_peered":"2024-09-16T09:30:08.763683+0000","last_unstale":"2024-09-16T09:38:56.436149+0000","last_undegraded":"2024-09-16T09:38:56.436149+0000","last_fullsized":"2024-09-16T09:38:56.436149+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:32:42.123067+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:32:42.123067+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:09.339965+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"285'30","reported_seq":693,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.430278+0000","last_change":"2024-09-16T09:33:21.333996+0000","last_active":"2024-09-16T09:38:56.430278+0000","last_peered":"2024-09-16T09:38:56.430278+0000","last_clean":"2024-09-16T09:38:56.430278+0000","last_became_active":"2024-09-16T09:31:09.270434+0000","last_became_peered":"2024-09-16T09:31:09.270434+0000","last_unstale":"2024-09-16T09:38:56.430278+0000","last_undegraded":"2024-09-16T09:38:56.430278+0000","last_fullsized":"2024-09-16T09:38:56.430278+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'18","last_scrub_stamp":"2024-09-16T09:33:21.333937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:33:21.333937+0000","objects_scrubbed":8,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":51,"num_read_kb":39,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":572,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.431051+0000","last_change":"2024-09-16T09:36:50.622852+0000","last_active":"2024-09-16T09:38:56.431051+0000","last_peered":"2024-09-16T09:38:56.431051+0000","last_clean":"2024-09-16T09:38:56.431051+0000","last_became_active":"2024-09-16T09:28:52.983543+0000","last_became_peered":"2024-09-16T09:28:52.983543+0000","last_unstale":"2024-09-16T09:38:56.431051+0000","last_undegraded":"2024-09-16T09:38:56.431051+0000","last_fullsized":"2024-09-16T09:38:56.431051+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:50.622807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:36:50.622807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:59.525944+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":752,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.431803+0000","last_change":"2024-09-16T09:35:19.822713+0000","last_active":"2024-09-16T09:38:56.431803+0000","last_peered":"2024-09-16T09:38:56.431803+0000","last_clean":"2024-09-16T09:38:56.431803+0000","last_became_active":"2024-09-16T09:31:09.268864+0000","last_became_peered":"2024-09-16T09:31:09.268864+0000","last_unstale":"2024-09-16T09:38:56.431803+0000","last_undegraded":"2024-09-16T09:38:56.431803+0000","last_fullsized":"2024-09-16T09:38:56.431803+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:35:19.822635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:35:19.822635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:11.341259+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":748,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.433565+0000","last_change":"2024-09-16T09:38:28.151882+0000","last_active":"2024-09-16T09:38:56.433565+0000","last_peered":"2024-09-16T09:38:56.433565+0000","last_clean":"2024-09-16T09:38:56.433565+0000","last_became_active":"2024-09-16T09:31:09.269433+0000","last_became_peered":"2024-09-16T09:31:09.269433+0000","last_unstale":"2024-09-16T09:38:56.433565+0000","last_undegraded":"2024-09-16T09:38:56.433565+0000","last_fullsized":"2024-09-16T09:38:56.433565+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:38:28.151683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:38:28.151683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:41.184677+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"407'5745","reported_seq":5821,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:39:02.773497+0000","last_change":"2024-09-16T09:37:45.463119+0000","last_active":"2024-09-16T09:39:02.773497+0000","last_peered":"2024-09-16T09:39:02.773497+0000","last_clean":"2024-09-16T09:39:02.773497+0000","last_became_active":"2024-09-16T09:37:45.462743+0000","last_became_peered":"2024-09-16T09:37:45.462743+0000","last_unstale":"2024-09-16T09:39:02.773497+0000","last_undegraded":"2024-09-16T09:39:02.773497+0000","last_fullsized":"2024-09-16T09:39:02.773497+0000","mapping_epoch":376,"log_start":"385'900","ondisk_log_start":"385'900","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_clean_scrub_stamp":"2024-09-16T09:37:44.378497+0000","objects_scrubbed":0,"log_size":4845,"log_dups_size":0,"ondisk_log_size":4845,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:10.090944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":376504320,"num_objects":5745,"num_object_clones":0,"num_object_copies":17235,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5745,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5745,"num_write_kb":367680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.4","version":"73'79","reported_seq":471,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.436712+0000","last_change":"2024-09-16T09:37:17.664174+0000","last_active":"2024-09-16T09:38:56.436712+0000","last_peered":"2024-09-16T09:38:56.436712+0000","last_clean":"2024-09-16T09:38:56.436712+0000","last_became_active":"2024-09-16T09:31:09.275139+0000","last_became_peered":"2024-09-16T09:31:09.275139+0000","last_unstale":"2024-09-16T09:38:56.436712+0000","last_undegraded":"2024-09-16T09:38:56.436712+0000","last_fullsized":"2024-09-16T09:38:56.436712+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:37:17.664105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:37:17.664105+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"285'28","reported_seq":784,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.431818+0000","last_change":"2024-09-16T09:35:43.914378+0000","last_active":"2024-09-16T09:38:56.431818+0000","last_peered":"2024-09-16T09:38:56.431818+0000","last_clean":"2024-09-16T09:38:56.431818+0000","last_became_active":"2024-09-16T09:25:03.708830+0000","last_became_peered":"2024-09-16T09:25:03.708830+0000","last_unstale":"2024-09-16T09:38:56.431818+0000","last_undegraded":"2024-09-16T09:38:56.431818+0000","last_fullsized":"2024-09-16T09:38:56.431818+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'28","last_scrub_stamp":"2024-09-16T09:35:43.914312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:35:43.914312+0000","objects_scrubbed":7,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:19.189121+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":763,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.431366+0000","last_change":"2024-09-16T09:38:32.457898+0000","last_active":"2024-09-16T09:38:56.431366+0000","last_peered":"2024-09-16T09:38:56.431366+0000","last_clean":"2024-09-16T09:38:56.431366+0000","last_became_active":"2024-09-16T09:25:01.702992+0000","last_became_peered":"2024-09-16T09:25:01.702992+0000","last_unstale":"2024-09-16T09:38:56.431366+0000","last_undegraded":"2024-09-16T09:38:56.431366+0000","last_fullsized":"2024-09-16T09:38:56.431366+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:38:32.457818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:38:32.457818+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:34.953669+0000","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"235'12","reported_seq":462,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.920056+0000","last_change":"2024-09-16T09:37:19.380567+0000","last_active":"2024-09-16T09:38:56.920056+0000","last_peered":"2024-09-16T09:38:56.920056+0000","last_clean":"2024-09-16T09:38:56.920056+0000","last_became_active":"2024-09-16T09:33:30.251047+0000","last_became_peered":"2024-09-16T09:33:30.251047+0000","last_unstale":"2024-09-16T09:38:56.920056+0000","last_undegraded":"2024-09-16T09:38:56.920056+0000","last_fullsized":"2024-09-16T09:38:56.920056+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"235'12","last_scrub_stamp":"2024-09-16T09:37:19.380504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:37:19.380504+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":724,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.430390+0000","last_change":"2024-09-16T09:38:30.524850+0000","last_active":"2024-09-16T09:38:56.430390+0000","last_peered":"2024-09-16T09:38:56.430390+0000","last_clean":"2024-09-16T09:38:56.430390+0000","last_became_active":"2024-09-16T09:25:07.722017+0000","last_became_peered":"2024-09-16T09:25:07.722017+0000","last_unstale":"2024-09-16T09:38:56.430390+0000","last_undegraded":"2024-09-16T09:38:56.430390+0000","last_fullsized":"2024-09-16T09:38:56.430390+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:38:30.524772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:38:30.524772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:34.576374+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.c","version":"407'5751","reported_seq":5826,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:39:02.886134+0000","last_change":"2024-09-16T09:37:45.401117+0000","last_active":"2024-09-16T09:39:02.886134+0000","last_peered":"2024-09-16T09:39:02.886134+0000","last_clean":"2024-09-16T09:39:02.886134+0000","last_became_active":"2024-09-16T09:37:45.400778+0000","last_became_peered":"2024-09-16T09:37:45.400778+0000","last_unstale":"2024-09-16T09:39:02.886134+0000","last_undegraded":"2024-09-16T09:39:02.886134+0000","last_fullsized":"2024-09-16T09:39:02.886134+0000","mapping_epoch":376,"log_start":"385'1521","ondisk_log_start":"385'1521","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_clean_scrub_stamp":"2024-09-16T09:37:44.378497+0000","objects_scrubbed":0,"log_size":4230,"log_dups_size":0,"ondisk_log_size":4230,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:09.317674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":376897536,"num_objects":5751,"num_object_clones":0,"num_object_copies":17253,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5751,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5751,"num_write_kb":368064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"73'79","reported_seq":802,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.439560+0000","last_change":"2024-09-16T09:38:24.036642+0000","last_active":"2024-09-16T09:38:56.439560+0000","last_peered":"2024-09-16T09:38:56.439560+0000","last_clean":"2024-09-16T09:38:56.439560+0000","last_became_active":"2024-09-16T09:31:09.274707+0000","last_became_peered":"2024-09-16T09:31:09.274707+0000","last_unstale":"2024-09-16T09:38:56.439560+0000","last_undegraded":"2024-09-16T09:38:56.439560+0000","last_fullsized":"2024-09-16T09:38:56.439560+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:38:24.036564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:38:24.036564+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:44.383655+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"285'20","reported_seq":427,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.446626+0000","last_change":"2024-09-16T09:38:32.122419+0000","last_active":"2024-09-16T09:38:56.446626+0000","last_peered":"2024-09-16T09:38:56.446626+0000","last_clean":"2024-09-16T09:38:56.446626+0000","last_became_active":"2024-09-16T09:33:30.248453+0000","last_became_peered":"2024-09-16T09:33:30.248453+0000","last_unstale":"2024-09-16T09:38:56.446626+0000","last_undegraded":"2024-09-16T09:38:56.446626+0000","last_fullsized":"2024-09-16T09:38:56.446626+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'20","last_scrub_stamp":"2024-09-16T09:38:32.122303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:38:32.122303+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:49.089544+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":751,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.446156+0000","last_change":"2024-09-16T09:36:47.095703+0000","last_active":"2024-09-16T09:38:56.446156+0000","last_peered":"2024-09-16T09:38:56.446156+0000","last_clean":"2024-09-16T09:38:56.446156+0000","last_became_active":"2024-09-16T09:31:09.270619+0000","last_became_peered":"2024-09-16T09:31:09.270619+0000","last_unstale":"2024-09-16T09:38:56.446156+0000","last_undegraded":"2024-09-16T09:38:56.446156+0000","last_fullsized":"2024-09-16T09:38:56.446156+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:47.095658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:36:47.095658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"66'8","reported_seq":906,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.425442+0000","last_change":"2024-09-16T09:36:52.689167+0000","last_active":"2024-09-16T09:38:56.425442+0000","last_peered":"2024-09-16T09:38:56.425442+0000","last_clean":"2024-09-16T09:38:56.425442+0000","last_became_active":"2024-09-16T09:25:05.714507+0000","last_became_peered":"2024-09-16T09:25:05.714507+0000","last_unstale":"2024-09-16T09:38:56.425442+0000","last_undegraded":"2024-09-16T09:38:56.425442+0000","last_fullsized":"2024-09-16T09:38:56.425442+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:36:52.689091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:36:52.689091+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":740,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.424495+0000","last_change":"2024-09-16T09:32:33.362233+0000","last_active":"2024-09-16T09:38:56.424495+0000","last_peered":"2024-09-16T09:38:56.424495+0000","last_clean":"2024-09-16T09:38:56.424495+0000","last_became_active":"2024-09-16T09:31:09.283192+0000","last_became_peered":"2024-09-16T09:31:09.283192+0000","last_unstale":"2024-09-16T09:38:56.424495+0000","last_undegraded":"2024-09-16T09:38:56.424495+0000","last_fullsized":"2024-09-16T09:38:56.424495+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:33.362188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:32:33.362188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:18.341863+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.d","version":"407'5727","reported_seq":5803,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:39:02.773137+0000","last_change":"2024-09-16T09:37:45.405475+0000","last_active":"2024-09-16T09:39:02.773137+0000","last_peered":"2024-09-16T09:39:02.773137+0000","last_clean":"2024-09-16T09:39:02.773137+0000","last_became_active":"2024-09-16T09:37:45.405149+0000","last_became_peered":"2024-09-16T09:37:45.405149+0000","last_unstale":"2024-09-16T09:39:02.773137+0000","last_undegraded":"2024-09-16T09:39:02.773137+0000","last_fullsized":"2024-09-16T09:39:02.773137+0000","mapping_epoch":376,"log_start":"385'900","ondisk_log_start":"385'900","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_clean_scrub_stamp":"2024-09-16T09:37:44.378497+0000","objects_scrubbed":0,"log_size":4827,"log_dups_size":0,"ondisk_log_size":4827,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:09.552512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":375324672,"num_objects":5727,"num_object_clones":0,"num_object_copies":17181,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5727,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5727,"num_write_kb":366528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"73'79","reported_seq":767,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.439195+0000","last_change":"2024-09-16T09:34:42.728232+0000","last_active":"2024-09-16T09:38:56.439195+0000","last_peered":"2024-09-16T09:38:56.439195+0000","last_clean":"2024-09-16T09:38:56.439195+0000","last_became_active":"2024-09-16T09:31:09.274565+0000","last_became_peered":"2024-09-16T09:31:09.274565+0000","last_unstale":"2024-09-16T09:38:56.439195+0000","last_undegraded":"2024-09-16T09:38:56.439195+0000","last_fullsized":"2024-09-16T09:38:56.439195+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:34:42.728143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:34:42.728143+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"285'29","reported_seq":785,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.425688+0000","last_change":"2024-09-16T09:36:54.604573+0000","last_active":"2024-09-16T09:38:56.425688+0000","last_peered":"2024-09-16T09:38:56.425688+0000","last_clean":"2024-09-16T09:38:56.425688+0000","last_became_active":"2024-09-16T09:25:03.714797+0000","last_became_peered":"2024-09-16T09:25:03.714797+0000","last_unstale":"2024-09-16T09:38:56.425688+0000","last_undegraded":"2024-09-16T09:38:56.425688+0000","last_fullsized":"2024-09-16T09:38:56.425688+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'29","last_scrub_stamp":"2024-09-16T09:36:54.604484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:36:54.604484+0000","objects_scrubbed":8,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":54,"num_read_kb":40,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":361,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.444622+0000","last_change":"2024-09-16T09:33:30.247311+0000","last_active":"2024-09-16T09:38:56.444622+0000","last_peered":"2024-09-16T09:38:56.444622+0000","last_clean":"2024-09-16T09:38:56.444622+0000","last_became_active":"2024-09-16T09:33:30.246996+0000","last_became_peered":"2024-09-16T09:33:30.246996+0000","last_unstale":"2024-09-16T09:38:56.444622+0000","last_undegraded":"2024-09-16T09:38:56.444622+0000","last_fullsized":"2024-09-16T09:38:56.444622+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:11.407616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:11.407616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:19.244214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,2],"acting":[5,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":746,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.432548+0000","last_change":"2024-09-16T09:34:45.772748+0000","last_active":"2024-09-16T09:38:56.432548+0000","last_peered":"2024-09-16T09:38:56.432548+0000","last_clean":"2024-09-16T09:38:56.432548+0000","last_became_active":"2024-09-16T09:25:05.708663+0000","last_became_peered":"2024-09-16T09:25:05.708663+0000","last_unstale":"2024-09-16T09:38:56.432548+0000","last_undegraded":"2024-09-16T09:38:56.432548+0000","last_fullsized":"2024-09-16T09:38:56.432548+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:45.772674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:34:45.772674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:17.038064+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":377,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.427921+0000","last_change":"2024-09-16T09:37:56.882475+0000","last_active":"2024-09-16T09:38:56.427921+0000","last_peered":"2024-09-16T09:38:56.427921+0000","last_clean":"2024-09-16T09:38:56.427921+0000","last_became_active":"2024-09-16T09:33:30.250178+0000","last_became_peered":"2024-09-16T09:33:30.250178+0000","last_unstale":"2024-09-16T09:38:56.427921+0000","last_undegraded":"2024-09-16T09:38:56.427921+0000","last_fullsized":"2024-09-16T09:38:56.427921+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:56.882326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:37:56.882326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:02.807091+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,2],"acting":[6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.a","version":"407'5864","reported_seq":5940,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:39:02.707151+0000","last_change":"2024-09-16T09:37:45.404280+0000","last_active":"2024-09-16T09:39:02.707151+0000","last_peered":"2024-09-16T09:39:02.707151+0000","last_clean":"2024-09-16T09:39:02.707151+0000","last_became_active":"2024-09-16T09:37:45.404021+0000","last_became_peered":"2024-09-16T09:37:45.404021+0000","last_unstale":"2024-09-16T09:39:02.707151+0000","last_undegraded":"2024-09-16T09:39:02.707151+0000","last_fullsized":"2024-09-16T09:39:02.707151+0000","mapping_epoch":376,"log_start":"385'1100","ondisk_log_start":"385'1100","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_clean_scrub_stamp":"2024-09-16T09:37:44.378497+0000","objects_scrubbed":0,"log_size":4764,"log_dups_size":0,"ondisk_log_size":4764,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":384303104,"num_objects":5864,"num_object_clones":0,"num_object_copies":17592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5864,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5864,"num_write_kb":375296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"73'79","reported_seq":700,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.446614+0000","last_change":"2024-09-16T09:36:42.191578+0000","last_active":"2024-09-16T09:38:56.446614+0000","last_peered":"2024-09-16T09:38:56.446614+0000","last_clean":"2024-09-16T09:38:56.446614+0000","last_became_active":"2024-09-16T09:30:08.766350+0000","last_became_peered":"2024-09-16T09:30:08.766350+0000","last_unstale":"2024-09-16T09:38:56.446614+0000","last_undegraded":"2024-09-16T09:38:56.446614+0000","last_fullsized":"2024-09-16T09:38:56.446614+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:36:42.191521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:36:42.191521+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"285'42","reported_seq":1049,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.430047+0000","last_change":"2024-09-16T09:37:57.407386+0000","last_active":"2024-09-16T09:38:56.430047+0000","last_peered":"2024-09-16T09:38:56.430047+0000","last_clean":"2024-09-16T09:38:56.430047+0000","last_became_active":"2024-09-16T09:31:09.278839+0000","last_became_peered":"2024-09-16T09:31:09.278839+0000","last_unstale":"2024-09-16T09:38:56.430047+0000","last_undegraded":"2024-09-16T09:38:56.430047+0000","last_fullsized":"2024-09-16T09:38:56.430047+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'42","last_scrub_stamp":"2024-09-16T09:37:57.407297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:37:57.407297+0000","objects_scrubbed":10,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:23.815599+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":66,"num_read_kb":51,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":754,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.435375+0000","last_change":"2024-09-16T09:38:30.156162+0000","last_active":"2024-09-16T09:38:56.435375+0000","last_peered":"2024-09-16T09:38:56.435375+0000","last_clean":"2024-09-16T09:38:56.435375+0000","last_became_active":"2024-09-16T09:25:01.703076+0000","last_became_peered":"2024-09-16T09:25:01.703076+0000","last_unstale":"2024-09-16T09:38:56.435375+0000","last_undegraded":"2024-09-16T09:38:56.435375+0000","last_fullsized":"2024-09-16T09:38:56.435375+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:38:30.156100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:38:30.156100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:58.260470+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":755,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.446878+0000","last_change":"2024-09-16T09:38:27.174725+0000","last_active":"2024-09-16T09:38:56.446878+0000","last_peered":"2024-09-16T09:38:56.446878+0000","last_clean":"2024-09-16T09:38:56.446878+0000","last_became_active":"2024-09-16T09:25:00.696237+0000","last_became_peered":"2024-09-16T09:25:00.696237+0000","last_unstale":"2024-09-16T09:38:56.446878+0000","last_undegraded":"2024-09-16T09:38:56.446878+0000","last_fullsized":"2024-09-16T09:38:56.446878+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-16T09:38:27.174657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:38:27.174657+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:32.956741+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.0025137430000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":714,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.433899+0000","last_change":"2024-09-16T09:33:16.666704+0000","last_active":"2024-09-16T09:38:56.433899+0000","last_peered":"2024-09-16T09:38:56.433899+0000","last_clean":"2024-09-16T09:38:56.433899+0000","last_became_active":"2024-09-16T09:25:05.717591+0000","last_became_peered":"2024-09-16T09:25:05.717591+0000","last_unstale":"2024-09-16T09:38:56.433899+0000","last_undegraded":"2024-09-16T09:38:56.433899+0000","last_fullsized":"2024-09-16T09:38:56.433899+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:16.666648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:16.666648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":471,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.437272+0000","last_change":"2024-09-16T09:38:25.992634+0000","last_active":"2024-09-16T09:38:56.437272+0000","last_peered":"2024-09-16T09:38:56.437272+0000","last_clean":"2024-09-16T09:38:56.437272+0000","last_became_active":"2024-09-16T09:31:09.272609+0000","last_became_peered":"2024-09-16T09:31:09.272609+0000","last_unstale":"2024-09-16T09:38:56.437272+0000","last_undegraded":"2024-09-16T09:38:56.437272+0000","last_fullsized":"2024-09-16T09:38:56.437272+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:38:25.992569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:38:25.992569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:32.485045+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"407'5150","reported_seq":5267,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:39:03.237186+0000","last_change":"2024-09-16T09:38:37.442061+0000","last_active":"2024-09-16T09:39:03.237186+0000","last_peered":"2024-09-16T09:39:03.237186+0000","last_clean":"2024-09-16T09:39:03.237186+0000","last_became_active":"2024-09-16T09:38:35.372587+0000","last_became_peered":"2024-09-16T09:38:35.372587+0000","last_unstale":"2024-09-16T09:39:03.237186+0000","last_undegraded":"2024-09-16T09:39:03.237186+0000","last_fullsized":"2024-09-16T09:39:03.237186+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":396,"parent":"0.0","parent_split_bits":0,"last_scrub":"397'4558","last_scrub_stamp":"2024-09-16T09:38:37.441970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_clean_scrub_stamp":"2024-09-16T09:38:37.441970+0000","objects_scrubbed":2306,"log_size":5150,"log_dups_size":0,"ondisk_log_size":5150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:50.718717+0000","scrub_duration":280,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":189923328,"num_objects":2898,"num_object_clones":0,"num_object_copies":8694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2898,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":592,"num_write_kb":37888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"73'79","reported_seq":60,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.429948+0000","last_change":"2024-09-16T09:38:23.458801+0000","last_active":"2024-09-16T09:38:56.429948+0000","last_peered":"2024-09-16T09:38:56.429948+0000","last_clean":"2024-09-16T09:38:56.429948+0000","last_became_active":"2024-09-16T09:38:10.556320+0000","last_became_peered":"2024-09-16T09:38:10.556320+0000","last_unstale":"2024-09-16T09:38:56.429948+0000","last_undegraded":"2024-09-16T09:38:56.429948+0000","last_fullsized":"2024-09-16T09:38:56.429948+0000","mapping_epoch":386,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":387,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:38:23.458728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:38:23.458728+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:29.742533+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"285'17","reported_seq":460,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.446131+0000","last_change":"2024-09-16T09:37:49.246464+0000","last_active":"2024-09-16T09:38:56.446131+0000","last_peered":"2024-09-16T09:38:56.446131+0000","last_clean":"2024-09-16T09:38:56.446131+0000","last_became_active":"2024-09-16T09:37:33.349009+0000","last_became_peered":"2024-09-16T09:37:33.349009+0000","last_unstale":"2024-09-16T09:38:56.446131+0000","last_undegraded":"2024-09-16T09:38:56.446131+0000","last_fullsized":"2024-09-16T09:38:56.446131+0000","mapping_epoch":367,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":368,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'17","last_scrub_stamp":"2024-09-16T09:37:49.246379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:37:49.246379+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:56.883815+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":35,"num_read_kb":24,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":369,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.428012+0000","last_change":"2024-09-16T09:37:21.946685+0000","last_active":"2024-09-16T09:38:56.428012+0000","last_peered":"2024-09-16T09:38:56.428012+0000","last_clean":"2024-09-16T09:38:56.428012+0000","last_became_active":"2024-09-16T09:33:30.247596+0000","last_became_peered":"2024-09-16T09:33:30.247596+0000","last_unstale":"2024-09-16T09:38:56.428012+0000","last_undegraded":"2024-09-16T09:38:56.428012+0000","last_fullsized":"2024-09-16T09:38:56.428012+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:21.946641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:37:21.946641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,2],"acting":[6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":404,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.426992+0000","last_change":"2024-09-16T09:38:23.416822+0000","last_active":"2024-09-16T09:38:56.426992+0000","last_peered":"2024-09-16T09:38:56.426992+0000","last_clean":"2024-09-16T09:38:56.426992+0000","last_became_active":"2024-09-16T09:37:04.358985+0000","last_became_peered":"2024-09-16T09:37:04.358985+0000","last_unstale":"2024-09-16T09:38:56.426992+0000","last_undegraded":"2024-09-16T09:38:56.426992+0000","last_fullsized":"2024-09-16T09:38:56.426992+0000","mapping_epoch":343,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":344,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:38:23.416756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:38:23.416756+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:50.859015+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":754,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.442973+0000","last_change":"2024-09-16T09:36:55.530225+0000","last_active":"2024-09-16T09:38:56.442973+0000","last_peered":"2024-09-16T09:38:56.442973+0000","last_clean":"2024-09-16T09:38:56.442973+0000","last_became_active":"2024-09-16T09:25:05.711539+0000","last_became_peered":"2024-09-16T09:25:05.711539+0000","last_unstale":"2024-09-16T09:38:56.442973+0000","last_undegraded":"2024-09-16T09:38:56.442973+0000","last_fullsized":"2024-09-16T09:38:56.442973+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:55.530175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:36:55.530175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:19.090568+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":732,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.431307+0000","last_change":"2024-09-16T09:37:45.450074+0000","last_active":"2024-09-16T09:38:56.431307+0000","last_peered":"2024-09-16T09:38:56.431307+0000","last_clean":"2024-09-16T09:38:56.431307+0000","last_became_active":"2024-09-16T09:25:07.723282+0000","last_became_peered":"2024-09-16T09:25:07.723282+0000","last_unstale":"2024-09-16T09:38:56.431307+0000","last_undegraded":"2024-09-16T09:38:56.431307+0000","last_fullsized":"2024-09-16T09:38:56.431307+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:45.450022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:37:45.450022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:15.116366+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"407'5089","reported_seq":5203,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:59.371756+0000","last_change":"2024-09-16T09:38:35.885143+0000","last_active":"2024-09-16T09:38:59.371756+0000","last_peered":"2024-09-16T09:38:59.371756+0000","last_clean":"2024-09-16T09:38:59.371756+0000","last_became_active":"2024-09-16T09:38:35.364673+0000","last_became_peered":"2024-09-16T09:38:35.364673+0000","last_unstale":"2024-09-16T09:38:59.371756+0000","last_undegraded":"2024-09-16T09:38:59.371756+0000","last_fullsized":"2024-09-16T09:38:59.371756+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":396,"parent":"0.0","parent_split_bits":0,"last_scrub":"396'4570","last_scrub_stamp":"2024-09-16T09:38:35.884998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_clean_scrub_stamp":"2024-09-16T09:38:35.884998+0000","objects_scrubbed":2313,"log_size":5089,"log_dups_size":0,"ondisk_log_size":5089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:48.670412+0000","scrub_duration":388,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":186515456,"num_objects":2846,"num_object_clones":0,"num_object_copies":8538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2846,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":533,"num_write_kb":34112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,7],"acting":[0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"73'79","reported_seq":450,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.428525+0000","last_change":"2024-09-16T09:34:31.929462+0000","last_active":"2024-09-16T09:38:56.428525+0000","last_peered":"2024-09-16T09:38:56.428525+0000","last_clean":"2024-09-16T09:38:56.428525+0000","last_became_active":"2024-09-16T09:33:30.249309+0000","last_became_peered":"2024-09-16T09:33:30.249309+0000","last_unstale":"2024-09-16T09:38:56.428525+0000","last_undegraded":"2024-09-16T09:38:56.428525+0000","last_fullsized":"2024-09-16T09:38:56.428525+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:34:31.929388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:34:31.929388+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.6","version":"285'24","reported_seq":770,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.431174+0000","last_change":"2024-09-16T09:38:33.474900+0000","last_active":"2024-09-16T09:38:56.431174+0000","last_peered":"2024-09-16T09:38:56.431174+0000","last_clean":"2024-09-16T09:38:56.431174+0000","last_became_active":"2024-09-16T09:25:03.715315+0000","last_became_peered":"2024-09-16T09:25:03.715315+0000","last_unstale":"2024-09-16T09:38:56.431174+0000","last_undegraded":"2024-09-16T09:38:56.431174+0000","last_fullsized":"2024-09-16T09:38:56.431174+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'24","last_scrub_stamp":"2024-09-16T09:38:33.474821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:38:33.474821+0000","objects_scrubbed":6,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:34.438489+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":36,"num_read_kb":30,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"59'1","reported_seq":761,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.426769+0000","last_change":"2024-09-16T09:33:21.682893+0000","last_active":"2024-09-16T09:38:56.426769+0000","last_peered":"2024-09-16T09:38:56.426769+0000","last_clean":"2024-09-16T09:38:56.426769+0000","last_became_active":"2024-09-16T09:31:09.269627+0000","last_became_peered":"2024-09-16T09:31:09.269627+0000","last_unstale":"2024-09-16T09:38:56.426769+0000","last_undegraded":"2024-09-16T09:38:56.426769+0000","last_fullsized":"2024-09-16T09:38:56.426769+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:33:21.682840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:21.682840+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1986,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"67'5","reported_seq":1590,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:39:02.141195+0000","last_change":"2024-09-16T09:38:35.633711+0000","last_active":"2024-09-16T09:39:02.141195+0000","last_peered":"2024-09-16T09:39:02.141195+0000","last_clean":"2024-09-16T09:39:02.141195+0000","last_became_active":"2024-09-16T09:25:00.696546+0000","last_became_peered":"2024-09-16T09:25:00.696546+0000","last_unstale":"2024-09-16T09:39:02.141195+0000","last_undegraded":"2024-09-16T09:39:02.141195+0000","last_fullsized":"2024-09-16T09:39:02.141195+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'5","last_scrub_stamp":"2024-09-16T09:38:35.633534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:38:35.633534+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:47.436505+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0.0059770420000000001,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":834,"num_read_kb":829,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":713,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.446221+0000","last_change":"2024-09-16T09:27:07.337170+0000","last_active":"2024-09-16T09:38:56.446221+0000","last_peered":"2024-09-16T09:38:56.446221+0000","last_clean":"2024-09-16T09:38:56.446221+0000","last_became_active":"2024-09-16T09:25:05.714704+0000","last_became_peered":"2024-09-16T09:25:05.714704+0000","last_unstale":"2024-09-16T09:38:56.446221+0000","last_undegraded":"2024-09-16T09:38:56.446221+0000","last_fullsized":"2024-09-16T09:38:56.446221+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:07.337124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:07.337124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":639,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.444837+0000","last_change":"2024-09-16T09:36:43.150813+0000","last_active":"2024-09-16T09:38:56.444837+0000","last_peered":"2024-09-16T09:38:56.444837+0000","last_clean":"2024-09-16T09:38:56.444837+0000","last_became_active":"2024-09-16T09:33:30.245852+0000","last_became_peered":"2024-09-16T09:33:30.245852+0000","last_unstale":"2024-09-16T09:38:56.444837+0000","last_undegraded":"2024-09-16T09:38:56.444837+0000","last_fullsized":"2024-09-16T09:38:56.444837+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:43.150764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:36:43.150764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"407'5773","reported_seq":5849,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:39:02.767081+0000","last_change":"2024-09-16T09:37:45.406191+0000","last_active":"2024-09-16T09:39:02.767081+0000","last_peered":"2024-09-16T09:39:02.767081+0000","last_clean":"2024-09-16T09:39:02.767081+0000","last_became_active":"2024-09-16T09:37:45.405552+0000","last_became_peered":"2024-09-16T09:37:45.405552+0000","last_unstale":"2024-09-16T09:39:02.767081+0000","last_undegraded":"2024-09-16T09:39:02.767081+0000","last_fullsized":"2024-09-16T09:39:02.767081+0000","mapping_epoch":376,"log_start":"385'1000","ondisk_log_start":"385'1000","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_clean_scrub_stamp":"2024-09-16T09:37:44.378497+0000","objects_scrubbed":0,"log_size":4773,"log_dups_size":0,"ondisk_log_size":4773,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:12.748542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":378339328,"num_objects":5773,"num_object_clones":0,"num_object_copies":17319,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5773,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5773,"num_write_kb":369472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"285'28","reported_seq":951,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.919775+0000","last_change":"2024-09-16T09:33:16.891850+0000","last_active":"2024-09-16T09:38:56.919775+0000","last_peered":"2024-09-16T09:38:56.919775+0000","last_clean":"2024-09-16T09:38:56.919775+0000","last_became_active":"2024-09-16T09:25:03.714118+0000","last_became_peered":"2024-09-16T09:25:03.714118+0000","last_unstale":"2024-09-16T09:38:56.919775+0000","last_undegraded":"2024-09-16T09:38:56.919775+0000","last_fullsized":"2024-09-16T09:38:56.919775+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'16","last_scrub_stamp":"2024-09-16T09:33:16.891761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:33:16.891761+0000","objects_scrubbed":8,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:12.972702+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":43,"num_read_kb":33,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":587,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.446750+0000","last_change":"2024-09-16T09:36:50.114910+0000","last_active":"2024-09-16T09:38:56.446750+0000","last_peered":"2024-09-16T09:38:56.446750+0000","last_clean":"2024-09-16T09:38:56.446750+0000","last_became_active":"2024-09-16T09:31:09.270196+0000","last_became_peered":"2024-09-16T09:31:09.270196+0000","last_unstale":"2024-09-16T09:38:56.446750+0000","last_undegraded":"2024-09-16T09:38:56.446750+0000","last_fullsized":"2024-09-16T09:38:56.446750+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:50.114861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:36:50.114861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"73'79","reported_seq":817,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.430738+0000","last_change":"2024-09-16T09:32:43.535041+0000","last_active":"2024-09-16T09:38:56.430738+0000","last_peered":"2024-09-16T09:38:56.430738+0000","last_clean":"2024-09-16T09:38:56.430738+0000","last_became_active":"2024-09-16T09:30:08.870394+0000","last_became_peered":"2024-09-16T09:30:08.870394+0000","last_unstale":"2024-09-16T09:38:56.430738+0000","last_undegraded":"2024-09-16T09:38:56.430738+0000","last_fullsized":"2024-09-16T09:38:56.430738+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:32:43.534962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:32:43.534962+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:16.454386+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":728,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.430742+0000","last_change":"2024-09-16T09:37:52.517698+0000","last_active":"2024-09-16T09:38:56.430742+0000","last_peered":"2024-09-16T09:38:56.430742+0000","last_clean":"2024-09-16T09:38:56.430742+0000","last_became_active":"2024-09-16T09:25:05.714341+0000","last_became_peered":"2024-09-16T09:25:05.714341+0000","last_unstale":"2024-09-16T09:38:56.430742+0000","last_undegraded":"2024-09-16T09:38:56.430742+0000","last_fullsized":"2024-09-16T09:38:56.430742+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:52.517652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:37:52.517652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:20.774965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":725,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.445929+0000","last_change":"2024-09-16T09:37:50.218655+0000","last_active":"2024-09-16T09:38:56.445929+0000","last_peered":"2024-09-16T09:38:56.445929+0000","last_clean":"2024-09-16T09:38:56.445929+0000","last_became_active":"2024-09-16T09:25:07.714281+0000","last_became_peered":"2024-09-16T09:25:07.714281+0000","last_unstale":"2024-09-16T09:38:56.445929+0000","last_undegraded":"2024-09-16T09:38:56.445929+0000","last_fullsized":"2024-09-16T09:38:56.445929+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:50.218594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:37:50.218594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:20.077058+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.0","version":"407'4988","reported_seq":5101,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:39:02.776852+0000","last_change":"2024-09-16T09:38:37.410089+0000","last_active":"2024-09-16T09:39:02.776852+0000","last_peered":"2024-09-16T09:39:02.776852+0000","last_clean":"2024-09-16T09:39:02.776852+0000","last_became_active":"2024-09-16T09:38:35.375417+0000","last_became_peered":"2024-09-16T09:38:35.375417+0000","last_unstale":"2024-09-16T09:39:02.776852+0000","last_undegraded":"2024-09-16T09:39:02.776852+0000","last_fullsized":"2024-09-16T09:39:02.776852+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":396,"parent":"0.0","parent_split_bits":0,"last_scrub":"397'4446","last_scrub_stamp":"2024-09-16T09:38:37.409993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_clean_scrub_stamp":"2024-09-16T09:38:37.409993+0000","objects_scrubbed":2245,"log_size":4988,"log_dups_size":0,"ondisk_log_size":4988,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:40:06.540960+0000","scrub_duration":298,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":182648832,"num_objects":2787,"num_object_clones":0,"num_object_copies":8361,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2787,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":542,"num_write_kb":34688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.b","version":"73'79","reported_seq":561,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.425126+0000","last_change":"2024-09-16T09:38:34.541153+0000","last_active":"2024-09-16T09:38:56.425126+0000","last_peered":"2024-09-16T09:38:56.425126+0000","last_clean":"2024-09-16T09:38:56.425126+0000","last_became_active":"2024-09-16T09:33:30.251933+0000","last_became_peered":"2024-09-16T09:33:30.251933+0000","last_unstale":"2024-09-16T09:38:56.425126+0000","last_undegraded":"2024-09-16T09:38:56.425126+0000","last_fullsized":"2024-09-16T09:38:56.425126+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:38:34.541066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:38:34.541066+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:48.645347+0000","scrub_duration":53,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"285'19","reported_seq":814,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.440786+0000","last_change":"2024-09-16T09:37:33.355562+0000","last_active":"2024-09-16T09:38:56.440786+0000","last_peered":"2024-09-16T09:38:56.440786+0000","last_clean":"2024-09-16T09:38:56.440786+0000","last_became_active":"2024-09-16T09:37:33.348433+0000","last_became_peered":"2024-09-16T09:37:33.348433+0000","last_unstale":"2024-09-16T09:38:56.440786+0000","last_undegraded":"2024-09-16T09:38:56.440786+0000","last_fullsized":"2024-09-16T09:38:56.440786+0000","mapping_epoch":367,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":368,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'19","last_scrub_stamp":"2024-09-16T09:35:27.813206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:35:27.813206+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:13.318653+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":730,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.441740+0000","last_change":"2024-09-16T09:37:18.392256+0000","last_active":"2024-09-16T09:38:56.441740+0000","last_peered":"2024-09-16T09:38:56.441740+0000","last_clean":"2024-09-16T09:38:56.441740+0000","last_became_active":"2024-09-16T09:25:01.708270+0000","last_became_peered":"2024-09-16T09:25:01.708270+0000","last_unstale":"2024-09-16T09:38:56.441740+0000","last_undegraded":"2024-09-16T09:38:56.441740+0000","last_fullsized":"2024-09-16T09:38:56.441740+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:18.392084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:37:18.392084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":713,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.445790+0000","last_change":"2024-09-16T09:26:34.171190+0000","last_active":"2024-09-16T09:38:56.445790+0000","last_peered":"2024-09-16T09:38:56.445790+0000","last_clean":"2024-09-16T09:38:56.445790+0000","last_became_active":"2024-09-16T09:25:05.711964+0000","last_became_peered":"2024-09-16T09:25:05.711964+0000","last_unstale":"2024-09-16T09:38:56.445790+0000","last_undegraded":"2024-09-16T09:38:56.445790+0000","last_fullsized":"2024-09-16T09:38:56.445790+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:34.171139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:26:34.171139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:18.227141+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":604,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.428420+0000","last_change":"2024-09-16T09:34:50.866121+0000","last_active":"2024-09-16T09:38:56.428420+0000","last_peered":"2024-09-16T09:38:56.428420+0000","last_clean":"2024-09-16T09:38:56.428420+0000","last_became_active":"2024-09-16T09:31:09.278618+0000","last_became_peered":"2024-09-16T09:31:09.278618+0000","last_unstale":"2024-09-16T09:38:56.428420+0000","last_undegraded":"2024-09-16T09:38:56.428420+0000","last_fullsized":"2024-09-16T09:38:56.428420+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:50.866032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:34:50.866032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:09.987998+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3","version":"407'5263","reported_seq":5377,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:39:03.155657+0000","last_change":"2024-09-16T09:38:36.668960+0000","last_active":"2024-09-16T09:39:03.155657+0000","last_peered":"2024-09-16T09:39:03.155657+0000","last_clean":"2024-09-16T09:39:03.155657+0000","last_became_active":"2024-09-16T09:38:35.365258+0000","last_became_peered":"2024-09-16T09:38:35.365258+0000","last_unstale":"2024-09-16T09:39:03.155657+0000","last_undegraded":"2024-09-16T09:39:03.155657+0000","last_fullsized":"2024-09-16T09:39:03.155657+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":396,"parent":"0.0","parent_split_bits":0,"last_scrub":"397'4682","last_scrub_stamp":"2024-09-16T09:38:36.668842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_clean_scrub_stamp":"2024-09-16T09:38:36.668842+0000","objects_scrubbed":2403,"log_size":5263,"log_dups_size":0,"ondisk_log_size":5263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:40:03.820054+0000","scrub_duration":551,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":196149248,"num_objects":2993,"num_object_clones":0,"num_object_copies":8979,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2993,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":590,"num_write_kb":37760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.8","version":"73'79","reported_seq":72,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.444403+0000","last_change":"2024-09-16T09:37:54.183161+0000","last_active":"2024-09-16T09:38:56.444403+0000","last_peered":"2024-09-16T09:38:56.444403+0000","last_clean":"2024-09-16T09:38:56.444403+0000","last_became_active":"2024-09-16T09:37:53.418691+0000","last_became_peered":"2024-09-16T09:37:53.418691+0000","last_unstale":"2024-09-16T09:38:56.444403+0000","last_undegraded":"2024-09-16T09:38:56.444403+0000","last_fullsized":"2024-09-16T09:38:56.444403+0000","mapping_epoch":382,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":383,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:37:54.183107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:37:54.183107+0000","objects_scrubbed":1,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:09.320645+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262688,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"285'31","reported_seq":791,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.436008+0000","last_change":"2024-09-16T09:35:28.861520+0000","last_active":"2024-09-16T09:38:56.436008+0000","last_peered":"2024-09-16T09:38:56.436008+0000","last_clean":"2024-09-16T09:38:56.436008+0000","last_became_active":"2024-09-16T09:25:03.713358+0000","last_became_peered":"2024-09-16T09:25:03.713358+0000","last_unstale":"2024-09-16T09:38:56.436008+0000","last_undegraded":"2024-09-16T09:38:56.436008+0000","last_fullsized":"2024-09-16T09:38:56.436008+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'31","last_scrub_stamp":"2024-09-16T09:35:28.861433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:35:28.861433+0000","objects_scrubbed":8,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:08.335954+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":50,"num_read_kb":40,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":738,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.432953+0000","last_change":"2024-09-16T09:37:52.902619+0000","last_active":"2024-09-16T09:38:56.432953+0000","last_peered":"2024-09-16T09:38:56.432953+0000","last_clean":"2024-09-16T09:38:56.432953+0000","last_became_active":"2024-09-16T09:25:01.703355+0000","last_became_peered":"2024-09-16T09:25:01.703355+0000","last_unstale":"2024-09-16T09:38:56.432953+0000","last_undegraded":"2024-09-16T09:38:56.432953+0000","last_fullsized":"2024-09-16T09:38:56.432953+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:52.902556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:37:52.902556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:04.411673+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":377,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.436281+0000","last_change":"2024-09-16T09:37:46.378016+0000","last_active":"2024-09-16T09:38:56.436281+0000","last_peered":"2024-09-16T09:38:56.436281+0000","last_clean":"2024-09-16T09:38:56.436281+0000","last_became_active":"2024-09-16T09:33:30.251186+0000","last_became_peered":"2024-09-16T09:33:30.251186+0000","last_unstale":"2024-09-16T09:38:56.436281+0000","last_undegraded":"2024-09-16T09:38:56.436281+0000","last_fullsized":"2024-09-16T09:38:56.436281+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:46.377951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:37:46.377951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,0],"acting":[4,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":675,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.437740+0000","last_change":"2024-09-16T09:38:24.031758+0000","last_active":"2024-09-16T09:38:56.437740+0000","last_peered":"2024-09-16T09:38:56.437740+0000","last_clean":"2024-09-16T09:38:56.437740+0000","last_became_active":"2024-09-16T09:33:21.337291+0000","last_became_peered":"2024-09-16T09:33:21.337291+0000","last_unstale":"2024-09-16T09:38:56.437740+0000","last_undegraded":"2024-09-16T09:38:56.437740+0000","last_fullsized":"2024-09-16T09:38:56.437740+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:38:24.031592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:38:24.031592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:32.497795+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4],"acting":[3,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"407'5223","reported_seq":5336,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:39:03.006011+0000","last_change":"2024-09-16T09:38:36.700802+0000","last_active":"2024-09-16T09:39:03.006011+0000","last_peered":"2024-09-16T09:39:03.006011+0000","last_clean":"2024-09-16T09:39:03.006011+0000","last_became_active":"2024-09-16T09:38:35.380640+0000","last_became_peered":"2024-09-16T09:38:35.380640+0000","last_unstale":"2024-09-16T09:39:03.006011+0000","last_undegraded":"2024-09-16T09:39:03.006011+0000","last_fullsized":"2024-09-16T09:39:03.006011+0000","mapping_epoch":376,"log_start":"383'400","ondisk_log_start":"383'400","created":376,"last_epoch_clean":396,"parent":"0.0","parent_split_bits":0,"last_scrub":"396'4616","last_scrub_stamp":"2024-09-16T09:38:36.700747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_clean_scrub_stamp":"2024-09-16T09:38:36.700747+0000","objects_scrubbed":2301,"log_size":4823,"log_dups_size":0,"ondisk_log_size":4823,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:48.469578+0000","scrub_duration":525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":190644224,"num_objects":2909,"num_object_clones":0,"num_object_copies":8727,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2909,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":608,"num_write_kb":38912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,0],"acting":[4,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.9","version":"73'79","reported_seq":792,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.433528+0000","last_change":"2024-09-16T09:29:19.013948+0000","last_active":"2024-09-16T09:38:56.433528+0000","last_peered":"2024-09-16T09:38:56.433528+0000","last_clean":"2024-09-16T09:38:56.433528+0000","last_became_active":"2024-09-16T09:26:35.126066+0000","last_became_peered":"2024-09-16T09:26:35.126066+0000","last_unstale":"2024-09-16T09:38:56.433528+0000","last_undegraded":"2024-09-16T09:38:56.433528+0000","last_fullsized":"2024-09-16T09:38:56.433528+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:29:19.013865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:29:19.013865+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:10.452624+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"285'20","reported_seq":748,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.436519+0000","last_change":"2024-09-16T09:33:24.154567+0000","last_active":"2024-09-16T09:38:56.436519+0000","last_peered":"2024-09-16T09:38:56.436519+0000","last_clean":"2024-09-16T09:38:56.436519+0000","last_became_active":"2024-09-16T09:25:03.707723+0000","last_became_peered":"2024-09-16T09:25:03.707723+0000","last_unstale":"2024-09-16T09:38:56.436519+0000","last_undegraded":"2024-09-16T09:38:56.436519+0000","last_fullsized":"2024-09-16T09:38:56.436519+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'10","last_scrub_stamp":"2024-09-16T09:33:24.154507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:33:24.154507+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:14.304180+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":595,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.425852+0000","last_change":"2024-09-16T09:33:10.528181+0000","last_active":"2024-09-16T09:38:56.425852+0000","last_peered":"2024-09-16T09:38:56.425852+0000","last_clean":"2024-09-16T09:38:56.425852+0000","last_became_active":"2024-09-16T09:31:09.288825+0000","last_became_peered":"2024-09-16T09:31:09.288825+0000","last_unstale":"2024-09-16T09:38:56.425852+0000","last_undegraded":"2024-09-16T09:38:56.425852+0000","last_fullsized":"2024-09-16T09:38:56.425852+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:10.528115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:10.528115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"228'9","reported_seq":478,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.920826+0000","last_change":"2024-09-16T09:37:16.515947+0000","last_active":"2024-09-16T09:38:56.920826+0000","last_peered":"2024-09-16T09:38:56.920826+0000","last_clean":"2024-09-16T09:38:56.920826+0000","last_became_active":"2024-09-16T09:33:21.337213+0000","last_became_peered":"2024-09-16T09:33:21.337213+0000","last_unstale":"2024-09-16T09:38:56.920826+0000","last_undegraded":"2024-09-16T09:38:56.920826+0000","last_fullsized":"2024-09-16T09:38:56.920826+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"228'9","last_scrub_stamp":"2024-09-16T09:37:16.514010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:37:16.514010+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":750,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.440562+0000","last_change":"2024-09-16T09:38:26.182064+0000","last_active":"2024-09-16T09:38:56.440562+0000","last_peered":"2024-09-16T09:38:56.440562+0000","last_clean":"2024-09-16T09:38:56.440562+0000","last_became_active":"2024-09-16T09:25:07.718958+0000","last_became_peered":"2024-09-16T09:25:07.718958+0000","last_unstale":"2024-09-16T09:38:56.440562+0000","last_undegraded":"2024-09-16T09:38:56.440562+0000","last_fullsized":"2024-09-16T09:38:56.440562+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:38:26.181977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:38:26.181977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:49.688163+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.5","version":"407'5152","reported_seq":5271,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:59.319111+0000","last_change":"2024-09-16T09:38:39.277535+0000","last_active":"2024-09-16T09:38:59.319111+0000","last_peered":"2024-09-16T09:38:59.319111+0000","last_clean":"2024-09-16T09:38:59.319111+0000","last_became_active":"2024-09-16T09:38:35.378878+0000","last_became_peered":"2024-09-16T09:38:35.378878+0000","last_unstale":"2024-09-16T09:38:59.319111+0000","last_undegraded":"2024-09-16T09:38:59.319111+0000","last_fullsized":"2024-09-16T09:38:59.319111+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":396,"parent":"0.0","parent_split_bits":0,"last_scrub":"399'4666","last_scrub_stamp":"2024-09-16T09:38:39.277355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_clean_scrub_stamp":"2024-09-16T09:38:39.277355+0000","objects_scrubbed":2394,"log_size":5152,"log_dups_size":0,"ondisk_log_size":5152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:42.813511+0000","scrub_duration":720,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":188809216,"num_objects":2881,"num_object_clones":0,"num_object_copies":8643,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2881,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":487,"num_write_kb":31168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.e","version":"73'79","reported_seq":534,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.440474+0000","last_change":"2024-09-16T09:35:41.979877+0000","last_active":"2024-09-16T09:38:56.440474+0000","last_peered":"2024-09-16T09:38:56.440474+0000","last_clean":"2024-09-16T09:38:56.440474+0000","last_became_active":"2024-09-16T09:30:10.705072+0000","last_became_peered":"2024-09-16T09:30:10.705072+0000","last_unstale":"2024-09-16T09:38:56.440474+0000","last_undegraded":"2024-09-16T09:38:56.440474+0000","last_fullsized":"2024-09-16T09:38:56.440474+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:35:41.979814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:35:41.979814+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:20.177576+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,3],"acting":[4,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.b","version":"285'15","reported_seq":784,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.425601+0000","last_change":"2024-09-16T09:36:55.651561+0000","last_active":"2024-09-16T09:38:56.425601+0000","last_peered":"2024-09-16T09:38:56.425601+0000","last_clean":"2024-09-16T09:38:56.425601+0000","last_became_active":"2024-09-16T09:36:31.590202+0000","last_became_peered":"2024-09-16T09:36:31.590202+0000","last_unstale":"2024-09-16T09:38:56.425601+0000","last_undegraded":"2024-09-16T09:38:56.425601+0000","last_fullsized":"2024-09-16T09:38:56.425601+0000","mapping_epoch":316,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":317,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'15","last_scrub_stamp":"2024-09-16T09:36:55.651496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:36:55.651496+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":743,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.445026+0000","last_change":"2024-09-16T09:37:31.033631+0000","last_active":"2024-09-16T09:38:56.445026+0000","last_peered":"2024-09-16T09:38:56.445026+0000","last_clean":"2024-09-16T09:38:56.445026+0000","last_became_active":"2024-09-16T09:31:09.276516+0000","last_became_peered":"2024-09-16T09:31:09.276516+0000","last_unstale":"2024-09-16T09:38:56.445026+0000","last_undegraded":"2024-09-16T09:38:56.445026+0000","last_fullsized":"2024-09-16T09:38:56.445026+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:31.033577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:37:31.033577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":663,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.433423+0000","last_change":"2024-09-16T09:33:40.260547+0000","last_active":"2024-09-16T09:38:56.433423+0000","last_peered":"2024-09-16T09:38:56.433423+0000","last_clean":"2024-09-16T09:38:56.433423+0000","last_became_active":"2024-09-16T09:33:21.337254+0000","last_became_peered":"2024-09-16T09:33:21.337254+0000","last_unstale":"2024-09-16T09:38:56.433423+0000","last_undegraded":"2024-09-16T09:38:56.433423+0000","last_fullsized":"2024-09-16T09:38:56.433423+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:40.260500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:40.260500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3],"acting":[4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":734,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.426696+0000","last_change":"2024-09-16T09:36:53.647305+0000","last_active":"2024-09-16T09:38:56.426696+0000","last_peered":"2024-09-16T09:38:56.426696+0000","last_clean":"2024-09-16T09:38:56.426696+0000","last_became_active":"2024-09-16T09:25:07.718778+0000","last_became_peered":"2024-09-16T09:25:07.718778+0000","last_unstale":"2024-09-16T09:38:56.426696+0000","last_undegraded":"2024-09-16T09:38:56.426696+0000","last_fullsized":"2024-09-16T09:38:56.426696+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:53.647256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:36:53.647256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"407'5144","reported_seq":5261,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:59.383759+0000","last_change":"2024-09-16T09:38:37.794707+0000","last_active":"2024-09-16T09:38:59.383759+0000","last_peered":"2024-09-16T09:38:59.383759+0000","last_clean":"2024-09-16T09:38:59.383759+0000","last_became_active":"2024-09-16T09:38:35.386054+0000","last_became_peered":"2024-09-16T09:38:35.386054+0000","last_unstale":"2024-09-16T09:38:59.383759+0000","last_undegraded":"2024-09-16T09:38:59.383759+0000","last_fullsized":"2024-09-16T09:38:59.383759+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":396,"parent":"0.0","parent_split_bits":0,"last_scrub":"397'4640","last_scrub_stamp":"2024-09-16T09:38:37.794580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_clean_scrub_stamp":"2024-09-16T09:38:37.794580+0000","objects_scrubbed":2341,"log_size":5144,"log_dups_size":0,"ondisk_log_size":5144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:42.845324+0000","scrub_duration":265,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":186449920,"num_objects":2845,"num_object_clones":0,"num_object_copies":8535,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2845,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":504,"num_write_kb":32256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.f","version":"73'79","reported_seq":786,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.439972+0000","last_change":"2024-09-16T09:34:53.866426+0000","last_active":"2024-09-16T09:38:56.439972+0000","last_peered":"2024-09-16T09:38:56.439972+0000","last_clean":"2024-09-16T09:38:56.439972+0000","last_became_active":"2024-09-16T09:30:10.693477+0000","last_became_peered":"2024-09-16T09:30:10.693477+0000","last_unstale":"2024-09-16T09:38:56.439972+0000","last_undegraded":"2024-09-16T09:38:56.439972+0000","last_fullsized":"2024-09-16T09:38:56.439972+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:34:53.866337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:34:53.866337+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:59.878482+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"285'35","reported_seq":797,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.432637+0000","last_change":"2024-09-16T09:37:28.078195+0000","last_active":"2024-09-16T09:38:56.432637+0000","last_peered":"2024-09-16T09:38:56.432637+0000","last_clean":"2024-09-16T09:38:56.432637+0000","last_became_active":"2024-09-16T09:25:03.712783+0000","last_became_peered":"2024-09-16T09:25:03.712783+0000","last_unstale":"2024-09-16T09:38:56.432637+0000","last_undegraded":"2024-09-16T09:38:56.432637+0000","last_fullsized":"2024-09-16T09:38:56.432637+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'35","last_scrub_stamp":"2024-09-16T09:37:28.077841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:37:28.077841+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":56,"num_read_kb":45,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"59'1","reported_seq":750,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.431712+0000","last_change":"2024-09-16T09:37:56.528872+0000","last_active":"2024-09-16T09:38:56.431712+0000","last_peered":"2024-09-16T09:38:56.431712+0000","last_clean":"2024-09-16T09:38:56.431712+0000","last_became_active":"2024-09-16T09:25:01.708069+0000","last_became_peered":"2024-09-16T09:25:01.708069+0000","last_unstale":"2024-09-16T09:38:56.431712+0000","last_undegraded":"2024-09-16T09:38:56.431712+0000","last_fullsized":"2024-09-16T09:38:56.431712+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:37:56.528796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:37:56.528796+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:57.114353+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":760,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.444471+0000","last_change":"2024-09-16T09:38:25.083485+0000","last_active":"2024-09-16T09:38:56.444471+0000","last_peered":"2024-09-16T09:38:56.444471+0000","last_clean":"2024-09-16T09:38:56.444471+0000","last_became_active":"2024-09-16T09:37:53.518284+0000","last_became_peered":"2024-09-16T09:37:53.518284+0000","last_unstale":"2024-09-16T09:38:56.444471+0000","last_undegraded":"2024-09-16T09:38:56.444471+0000","last_fullsized":"2024-09-16T09:38:56.444471+0000","mapping_epoch":382,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":383,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:38:25.083277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:38:25.083277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:54.175038+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":716,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.430583+0000","last_change":"2024-09-16T09:36:48.649739+0000","last_active":"2024-09-16T09:38:56.430583+0000","last_peered":"2024-09-16T09:38:56.430583+0000","last_clean":"2024-09-16T09:38:56.430583+0000","last_became_active":"2024-09-16T09:25:07.718750+0000","last_became_peered":"2024-09-16T09:25:07.718750+0000","last_unstale":"2024-09-16T09:38:56.430583+0000","last_undegraded":"2024-09-16T09:38:56.430583+0000","last_fullsized":"2024-09-16T09:38:56.430583+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:48.649696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:36:48.649696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.7","version":"407'5229","reported_seq":5341,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:39:02.697150+0000","last_change":"2024-09-16T09:38:36.686541+0000","last_active":"2024-09-16T09:39:02.697150+0000","last_peered":"2024-09-16T09:39:02.697150+0000","last_clean":"2024-09-16T09:39:02.697150+0000","last_became_active":"2024-09-16T09:38:35.358645+0000","last_became_peered":"2024-09-16T09:38:35.358645+0000","last_unstale":"2024-09-16T09:39:02.697150+0000","last_undegraded":"2024-09-16T09:39:02.697150+0000","last_fullsized":"2024-09-16T09:39:02.697150+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":396,"parent":"0.0","parent_split_bits":0,"last_scrub":"396'4653","last_scrub_stamp":"2024-09-16T09:38:36.686443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_clean_scrub_stamp":"2024-09-16T09:38:36.686443+0000","objects_scrubbed":2327,"log_size":5229,"log_dups_size":0,"ondisk_log_size":5229,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:41.756958+0000","scrub_duration":549,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":190382080,"num_objects":2905,"num_object_clones":0,"num_object_copies":8715,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2905,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":578,"num_write_kb":36992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.c","version":"73'79","reported_seq":475,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.436616+0000","last_change":"2024-09-16T09:36:53.697797+0000","last_active":"2024-09-16T09:38:56.436616+0000","last_peered":"2024-09-16T09:38:56.436616+0000","last_clean":"2024-09-16T09:38:56.436616+0000","last_became_active":"2024-09-16T09:31:09.273770+0000","last_became_peered":"2024-09-16T09:31:09.273770+0000","last_unstale":"2024-09-16T09:38:56.436616+0000","last_undegraded":"2024-09-16T09:38:56.436616+0000","last_fullsized":"2024-09-16T09:38:56.436616+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:36:53.697742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:36:53.697742+0000","objects_scrubbed":1,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:00.926871+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":327680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"285'18","reported_seq":812,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.431958+0000","last_change":"2024-09-16T09:36:50.603724+0000","last_active":"2024-09-16T09:38:56.431958+0000","last_peered":"2024-09-16T09:38:56.431958+0000","last_clean":"2024-09-16T09:38:56.431958+0000","last_became_active":"2024-09-16T09:31:09.269047+0000","last_became_peered":"2024-09-16T09:31:09.269047+0000","last_unstale":"2024-09-16T09:38:56.431958+0000","last_undegraded":"2024-09-16T09:38:56.431958+0000","last_fullsized":"2024-09-16T09:38:56.431958+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'18","last_scrub_stamp":"2024-09-16T09:36:50.603660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:36:50.603660+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":744,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.430793+0000","last_change":"2024-09-16T09:37:57.529616+0000","last_active":"2024-09-16T09:38:56.430793+0000","last_peered":"2024-09-16T09:38:56.430793+0000","last_clean":"2024-09-16T09:38:56.430793+0000","last_became_active":"2024-09-16T09:25:01.707528+0000","last_became_peered":"2024-09-16T09:25:01.707528+0000","last_unstale":"2024-09-16T09:38:56.430793+0000","last_undegraded":"2024-09-16T09:38:56.430793+0000","last_fullsized":"2024-09-16T09:38:56.430793+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:57.529554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:37:57.529554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:05.467113+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":389,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.424255+0000","last_change":"2024-09-16T09:37:57.449473+0000","last_active":"2024-09-16T09:38:56.424255+0000","last_peered":"2024-09-16T09:38:56.424255+0000","last_clean":"2024-09-16T09:38:56.424255+0000","last_became_active":"2024-09-16T09:33:21.337309+0000","last_became_peered":"2024-09-16T09:33:21.337309+0000","last_unstale":"2024-09-16T09:38:56.424255+0000","last_undegraded":"2024-09-16T09:38:56.424255+0000","last_fullsized":"2024-09-16T09:38:56.424255+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:57.449105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:37:57.449105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:12.289276+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":734,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.437756+0000","last_change":"2024-09-16T09:38:21.097872+0000","last_active":"2024-09-16T09:38:56.437756+0000","last_peered":"2024-09-16T09:38:56.437756+0000","last_clean":"2024-09-16T09:38:56.437756+0000","last_became_active":"2024-09-16T09:25:07.718899+0000","last_became_peered":"2024-09-16T09:25:07.718899+0000","last_unstale":"2024-09-16T09:38:56.437756+0000","last_undegraded":"2024-09-16T09:38:56.437756+0000","last_fullsized":"2024-09-16T09:38:56.437756+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:38:21.097804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:38:21.097804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:46.836091+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.6","version":"407'5008","reported_seq":5122,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:39:03.215083+0000","last_change":"2024-09-16T09:38:38.320135+0000","last_active":"2024-09-16T09:39:03.215083+0000","last_peered":"2024-09-16T09:39:03.215083+0000","last_clean":"2024-09-16T09:39:03.215083+0000","last_became_active":"2024-09-16T09:38:35.382660+0000","last_became_peered":"2024-09-16T09:38:35.382660+0000","last_unstale":"2024-09-16T09:39:03.215083+0000","last_undegraded":"2024-09-16T09:39:03.215083+0000","last_fullsized":"2024-09-16T09:39:03.215083+0000","mapping_epoch":376,"log_start":"0'0","ondisk_log_start":"0'0","created":376,"last_epoch_clean":396,"parent":"0.0","parent_split_bits":0,"last_scrub":"397'4461","last_scrub_stamp":"2024-09-16T09:38:38.320065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_clean_scrub_stamp":"2024-09-16T09:38:38.320065+0000","objects_scrubbed":2226,"log_size":5008,"log_dups_size":0,"ondisk_log_size":5008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:58.259483+0000","scrub_duration":197,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":181731328,"num_objects":2773,"num_object_clones":0,"num_object_copies":8319,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2773,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":547,"num_write_kb":35008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.d","version":"73'79","reported_seq":525,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.430066+0000","last_change":"2024-09-16T09:32:44.536868+0000","last_active":"2024-09-16T09:38:56.430066+0000","last_peered":"2024-09-16T09:38:56.430066+0000","last_clean":"2024-09-16T09:38:56.430066+0000","last_became_active":"2024-09-16T09:30:10.709958+0000","last_became_peered":"2024-09-16T09:30:10.709958+0000","last_unstale":"2024-09-16T09:38:56.430066+0000","last_undegraded":"2024-09-16T09:38:56.430066+0000","last_fullsized":"2024-09-16T09:38:56.430066+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:32:44.536799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:32:44.536799+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:18.434560+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"285'27","reported_seq":768,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.444552+0000","last_change":"2024-09-16T09:37:30.081518+0000","last_active":"2024-09-16T09:38:56.444552+0000","last_peered":"2024-09-16T09:38:56.444552+0000","last_clean":"2024-09-16T09:38:56.444552+0000","last_became_active":"2024-09-16T09:25:03.715733+0000","last_became_peered":"2024-09-16T09:25:03.715733+0000","last_unstale":"2024-09-16T09:38:56.444552+0000","last_undegraded":"2024-09-16T09:38:56.444552+0000","last_fullsized":"2024-09-16T09:38:56.444552+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'27","last_scrub_stamp":"2024-09-16T09:37:30.081447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:37:30.081447+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":748,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.431094+0000","last_change":"2024-09-16T09:37:17.544132+0000","last_active":"2024-09-16T09:38:56.431094+0000","last_peered":"2024-09-16T09:38:56.431094+0000","last_clean":"2024-09-16T09:38:56.431094+0000","last_became_active":"2024-09-16T09:25:01.707372+0000","last_became_peered":"2024-09-16T09:25:01.707372+0000","last_unstale":"2024-09-16T09:38:56.431094+0000","last_undegraded":"2024-09-16T09:38:56.431094+0000","last_fullsized":"2024-09-16T09:38:56.431094+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-16T09:37:17.544069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:37:17.544069+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"66'8","reported_seq":905,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.921030+0000","last_change":"2024-09-16T09:30:05.138362+0000","last_active":"2024-09-16T09:38:56.921030+0000","last_peered":"2024-09-16T09:38:56.921030+0000","last_clean":"2024-09-16T09:38:56.921030+0000","last_became_active":"2024-09-16T09:25:05.716189+0000","last_became_peered":"2024-09-16T09:25:05.716189+0000","last_unstale":"2024-09-16T09:38:56.921030+0000","last_undegraded":"2024-09-16T09:38:56.921030+0000","last_fullsized":"2024-09-16T09:38:56.921030+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:30:05.138288+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:30:05.138288+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:14.462129+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":717,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.446352+0000","last_change":"2024-09-16T09:37:47.244099+0000","last_active":"2024-09-16T09:38:56.446352+0000","last_peered":"2024-09-16T09:38:56.446352+0000","last_clean":"2024-09-16T09:38:56.446352+0000","last_became_active":"2024-09-16T09:25:07.717213+0000","last_became_peered":"2024-09-16T09:25:07.717213+0000","last_unstale":"2024-09-16T09:38:56.446352+0000","last_undegraded":"2024-09-16T09:38:56.446352+0000","last_fullsized":"2024-09-16T09:38:56.446352+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:47.244032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:37:47.244032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.19","version":"407'5117","reported_seq":5200,"reported_epoch":407,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-16T09:39:03.004057+0000","last_change":"2024-09-16T09:38:38.833694+0000","last_active":"2024-09-16T09:39:03.004057+0000","last_peered":"2024-09-16T09:39:03.004057+0000","last_clean":"2024-09-16T09:38:36.504509+0000","last_became_active":"2024-09-16T09:38:38.804304+0000","last_became_peered":"2024-09-16T09:38:38.804304+0000","last_unstale":"2024-09-16T09:39:03.004057+0000","last_undegraded":"2024-09-16T09:38:38.795228+0000","last_fullsized":"2024-09-16T09:38:38.526678+0000","mapping_epoch":398,"log_start":"0'0","ondisk_log_start":"0'0","created":395,"last_epoch_clean":396,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_clean_scrub_stamp":"2024-09-16T09:37:44.378497+0000","objects_scrubbed":0,"log_size":5117,"log_dups_size":0,"ondisk_log_size":5117,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":187629568,"num_objects":2863,"num_object_clones":0,"num_object_copies":8589,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4582,"num_objects_misplaced":2291,"num_objects_unfound":0,"num_objects_dirty":2863,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2863,"num_write_kb":183232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[2,1],"avail_no_missing":["2","5","7"],"object_location_counts":[{"shards":"1,2","objects":572},{"shards":"2,5,7","objects":2291}],"blocked_by":[],"up_primary":4,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.12","version":"73'79","reported_seq":534,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.446542+0000","last_change":"2024-09-16T09:36:44.171217+0000","last_active":"2024-09-16T09:38:56.446542+0000","last_peered":"2024-09-16T09:38:56.446542+0000","last_clean":"2024-09-16T09:38:56.446542+0000","last_became_active":"2024-09-16T09:30:10.731413+0000","last_became_peered":"2024-09-16T09:30:10.731413+0000","last_unstale":"2024-09-16T09:38:56.446542+0000","last_undegraded":"2024-09-16T09:38:56.446542+0000","last_fullsized":"2024-09-16T09:38:56.446542+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:36:44.171151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:36:44.171151+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":752,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.427960+0000","last_change":"2024-09-16T09:37:55.863661+0000","last_active":"2024-09-16T09:38:56.427960+0000","last_peered":"2024-09-16T09:38:56.427960+0000","last_clean":"2024-09-16T09:38:56.427960+0000","last_became_active":"2024-09-16T09:37:53.518354+0000","last_became_peered":"2024-09-16T09:37:53.518354+0000","last_unstale":"2024-09-16T09:38:56.427960+0000","last_undegraded":"2024-09-16T09:38:56.427960+0000","last_fullsized":"2024-09-16T09:38:56.427960+0000","mapping_epoch":382,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":383,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:55.863613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:37:55.863613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:01.033155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"285'12","reported_seq":511,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.438020+0000","last_change":"2024-09-16T09:38:33.535691+0000","last_active":"2024-09-16T09:38:56.438020+0000","last_peered":"2024-09-16T09:38:56.438020+0000","last_clean":"2024-09-16T09:38:56.438020+0000","last_became_active":"2024-09-16T09:38:33.517889+0000","last_became_peered":"2024-09-16T09:38:33.517889+0000","last_unstale":"2024-09-16T09:38:56.438020+0000","last_undegraded":"2024-09-16T09:38:56.438020+0000","last_fullsized":"2024-09-16T09:38:56.438020+0000","mapping_epoch":393,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":394,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'12","last_scrub_stamp":"2024-09-16T09:37:20.749101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:37:20.749101+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":17,"num_read_kb":14,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":728,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.444921+0000","last_change":"2024-09-16T09:36:38.129449+0000","last_active":"2024-09-16T09:38:56.444921+0000","last_peered":"2024-09-16T09:38:56.444921+0000","last_clean":"2024-09-16T09:38:56.444921+0000","last_became_active":"2024-09-16T09:25:05.715616+0000","last_became_peered":"2024-09-16T09:25:05.715616+0000","last_unstale":"2024-09-16T09:38:56.444921+0000","last_undegraded":"2024-09-16T09:38:56.444921+0000","last_fullsized":"2024-09-16T09:38:56.444921+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:38.129314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:36:38.129314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":723,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.431224+0000","last_change":"2024-09-16T09:38:27.468922+0000","last_active":"2024-09-16T09:38:56.431224+0000","last_peered":"2024-09-16T09:38:56.431224+0000","last_clean":"2024-09-16T09:38:56.431224+0000","last_became_active":"2024-09-16T09:25:07.721059+0000","last_became_peered":"2024-09-16T09:25:07.721059+0000","last_unstale":"2024-09-16T09:38:56.431224+0000","last_undegraded":"2024-09-16T09:38:56.431224+0000","last_fullsized":"2024-09-16T09:38:56.431224+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:38:27.468854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:38:27.468854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:40.161925+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.18","version":"407'5157","reported_seq":8416,"reported_epoch":407,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-16T09:39:03.113061+0000","last_change":"2024-09-16T09:38:38.854987+0000","last_active":"2024-09-16T09:39:03.113061+0000","last_peered":"2024-09-16T09:39:03.113061+0000","last_clean":"2024-09-16T09:38:36.426945+0000","last_became_active":"2024-09-16T09:38:38.811640+0000","last_became_peered":"2024-09-16T09:38:38.811640+0000","last_unstale":"2024-09-16T09:39:03.113061+0000","last_undegraded":"2024-09-16T09:39:03.113061+0000","last_fullsized":"2024-09-16T09:38:38.524309+0000","mapping_epoch":398,"log_start":"0'0","ondisk_log_start":"0'0","created":395,"last_epoch_clean":396,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_clean_scrub_stamp":"2024-09-16T09:37:44.378497+0000","objects_scrubbed":0,"log_size":5157,"log_dups_size":0,"ondisk_log_size":5157,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":188874752,"num_objects":2882,"num_object_clones":0,"num_object_copies":8646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2444,"num_objects_unfound":0,"num_objects_dirty":2882,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2882,"num_write_kb":184448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1058,"num_bytes_recovered":69337088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1],"avail_no_missing":["2","0","7"],"object_location_counts":[{"shards":"0,2,7","objects":1222},{"shards":"1,2","objects":1660}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.13","version":"73'79","reported_seq":620,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.437935+0000","last_change":"2024-09-16T09:34:48.260121+0000","last_active":"2024-09-16T09:38:56.437935+0000","last_peered":"2024-09-16T09:38:56.437935+0000","last_clean":"2024-09-16T09:38:56.437935+0000","last_became_active":"2024-09-16T09:33:30.248497+0000","last_became_peered":"2024-09-16T09:33:30.248497+0000","last_unstale":"2024-09-16T09:38:56.437935+0000","last_undegraded":"2024-09-16T09:38:56.437935+0000","last_fullsized":"2024-09-16T09:38:56.437935+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:34:48.260041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:34:48.260041+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:11.022422+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"285'15","reported_seq":808,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.431275+0000","last_change":"2024-09-16T09:33:48.681742+0000","last_active":"2024-09-16T09:38:56.431275+0000","last_peered":"2024-09-16T09:38:56.431275+0000","last_clean":"2024-09-16T09:38:56.431275+0000","last_became_active":"2024-09-16T09:31:09.273498+0000","last_became_peered":"2024-09-16T09:31:09.273498+0000","last_unstale":"2024-09-16T09:38:56.431275+0000","last_undegraded":"2024-09-16T09:38:56.431275+0000","last_fullsized":"2024-09-16T09:38:56.431275+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'9","last_scrub_stamp":"2024-09-16T09:33:48.681650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:33:48.681650+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:38:59.423578+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":736,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.433795+0000","last_change":"2024-09-16T09:33:13.312434+0000","last_active":"2024-09-16T09:38:56.433795+0000","last_peered":"2024-09-16T09:38:56.433795+0000","last_clean":"2024-09-16T09:38:56.433795+0000","last_became_active":"2024-09-16T09:25:01.706491+0000","last_became_peered":"2024-09-16T09:25:01.706491+0000","last_unstale":"2024-09-16T09:38:56.433795+0000","last_undegraded":"2024-09-16T09:38:56.433795+0000","last_fullsized":"2024-09-16T09:38:56.433795+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:13.312344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:13.312344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":501,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.430892+0000","last_change":"2024-09-16T09:38:31.544996+0000","last_active":"2024-09-16T09:38:56.430892+0000","last_peered":"2024-09-16T09:38:56.430892+0000","last_clean":"2024-09-16T09:38:56.430892+0000","last_became_active":"2024-09-16T09:33:30.248992+0000","last_became_peered":"2024-09-16T09:33:30.248992+0000","last_unstale":"2024-09-16T09:38:56.430892+0000","last_undegraded":"2024-09-16T09:38:56.430892+0000","last_fullsized":"2024-09-16T09:38:56.430892+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:38:31.544921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:38:31.544921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:32.792114+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":733,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.441947+0000","last_change":"2024-09-16T09:36:58.536167+0000","last_active":"2024-09-16T09:38:56.441947+0000","last_peered":"2024-09-16T09:38:56.441947+0000","last_clean":"2024-09-16T09:38:56.441947+0000","last_became_active":"2024-09-16T09:25:07.722050+0000","last_became_peered":"2024-09-16T09:25:07.722050+0000","last_unstale":"2024-09-16T09:38:56.441947+0000","last_undegraded":"2024-09-16T09:38:56.441947+0000","last_fullsized":"2024-09-16T09:38:56.441947+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:58.536121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:36:58.536121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:20.078148+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.10","version":"73'79","reported_seq":534,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.444289+0000","last_change":"2024-09-16T09:36:41.169117+0000","last_active":"2024-09-16T09:38:56.444289+0000","last_peered":"2024-09-16T09:38:56.444289+0000","last_clean":"2024-09-16T09:38:56.444289+0000","last_became_active":"2024-09-16T09:30:10.692770+0000","last_became_peered":"2024-09-16T09:30:10.692770+0000","last_unstale":"2024-09-16T09:38:56.444289+0000","last_undegraded":"2024-09-16T09:38:56.444289+0000","last_fullsized":"2024-09-16T09:38:56.444289+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:36:41.169069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:36:41.169069+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"285'15","reported_seq":757,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.446076+0000","last_change":"2024-09-16T09:37:55.203919+0000","last_active":"2024-09-16T09:38:56.446076+0000","last_peered":"2024-09-16T09:38:56.446076+0000","last_clean":"2024-09-16T09:38:56.446076+0000","last_became_active":"2024-09-16T09:25:03.717310+0000","last_became_peered":"2024-09-16T09:25:03.717310+0000","last_unstale":"2024-09-16T09:38:56.446076+0000","last_undegraded":"2024-09-16T09:38:56.446076+0000","last_fullsized":"2024-09-16T09:38:56.446076+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'15","last_scrub_stamp":"2024-09-16T09:37:55.203856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:37:55.203856+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:23.098469+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":767,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.429635+0000","last_change":"2024-09-16T09:37:50.471532+0000","last_active":"2024-09-16T09:38:56.429635+0000","last_peered":"2024-09-16T09:38:56.429635+0000","last_clean":"2024-09-16T09:38:56.429635+0000","last_became_active":"2024-09-16T09:31:09.272848+0000","last_became_peered":"2024-09-16T09:31:09.272848+0000","last_unstale":"2024-09-16T09:38:56.429635+0000","last_undegraded":"2024-09-16T09:38:56.429635+0000","last_fullsized":"2024-09-16T09:38:56.429635+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:50.471486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:37:50.471486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:09.546185+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"187'11","reported_seq":750,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.920836+0000","last_change":"2024-09-16T09:37:19.540982+0000","last_active":"2024-09-16T09:38:56.920836+0000","last_peered":"2024-09-16T09:38:56.920836+0000","last_clean":"2024-09-16T09:38:56.920836+0000","last_became_active":"2024-09-16T09:31:09.277688+0000","last_became_peered":"2024-09-16T09:31:09.277688+0000","last_unstale":"2024-09-16T09:38:56.920836+0000","last_undegraded":"2024-09-16T09:38:56.920836+0000","last_fullsized":"2024-09-16T09:38:56.920836+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"187'11","last_scrub_stamp":"2024-09-16T09:37:19.540886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:37:19.540886+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":749,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.436468+0000","last_change":"2024-09-16T09:38:31.156207+0000","last_active":"2024-09-16T09:38:56.436468+0000","last_peered":"2024-09-16T09:38:56.436468+0000","last_clean":"2024-09-16T09:38:56.436468+0000","last_became_active":"2024-09-16T09:25:07.718810+0000","last_became_peered":"2024-09-16T09:25:07.718810+0000","last_unstale":"2024-09-16T09:38:56.436468+0000","last_undegraded":"2024-09-16T09:38:56.436468+0000","last_fullsized":"2024-09-16T09:38:56.436468+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:38:31.156135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:38:31.156135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:44.602349+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.11","version":"73'79","reported_seq":620,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.426844+0000","last_change":"2024-09-16T09:37:15.623104+0000","last_active":"2024-09-16T09:38:56.426844+0000","last_peered":"2024-09-16T09:38:56.426844+0000","last_clean":"2024-09-16T09:38:56.426844+0000","last_became_active":"2024-09-16T09:31:09.272208+0000","last_became_peered":"2024-09-16T09:31:09.272208+0000","last_unstale":"2024-09-16T09:38:56.426844+0000","last_undegraded":"2024-09-16T09:38:56.426844+0000","last_fullsized":"2024-09-16T09:38:56.426844+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:37:15.623041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:37:15.623041+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.14","version":"285'20","reported_seq":426,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.430844+0000","last_change":"2024-09-16T09:38:24.508342+0000","last_active":"2024-09-16T09:38:56.430844+0000","last_peered":"2024-09-16T09:38:56.430844+0000","last_clean":"2024-09-16T09:38:56.430844+0000","last_became_active":"2024-09-16T09:33:30.249769+0000","last_became_peered":"2024-09-16T09:33:30.249769+0000","last_unstale":"2024-09-16T09:38:56.430844+0000","last_undegraded":"2024-09-16T09:38:56.430844+0000","last_fullsized":"2024-09-16T09:38:56.430844+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'20","last_scrub_stamp":"2024-09-16T09:38:24.508261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:38:24.508261+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:36.617040+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":728,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.430880+0000","last_change":"2024-09-16T09:29:56.021413+0000","last_active":"2024-09-16T09:38:56.430880+0000","last_peered":"2024-09-16T09:38:56.430880+0000","last_clean":"2024-09-16T09:38:56.430880+0000","last_became_active":"2024-09-16T09:25:01.707513+0000","last_became_peered":"2024-09-16T09:25:01.707513+0000","last_unstale":"2024-09-16T09:38:56.430880+0000","last_undegraded":"2024-09-16T09:38:56.430880+0000","last_fullsized":"2024-09-16T09:38:56.430880+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:29:56.021340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:29:56.021340+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:13.464027+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"66'8","reported_seq":921,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.921047+0000","last_change":"2024-09-16T09:38:24.130552+0000","last_active":"2024-09-16T09:38:56.921047+0000","last_peered":"2024-09-16T09:38:56.921047+0000","last_clean":"2024-09-16T09:38:56.921047+0000","last_became_active":"2024-09-16T09:25:05.713833+0000","last_became_peered":"2024-09-16T09:25:05.713833+0000","last_unstale":"2024-09-16T09:38:56.921047+0000","last_undegraded":"2024-09-16T09:38:56.921047+0000","last_fullsized":"2024-09-16T09:38:56.921047+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:38:24.130499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:38:24.130499+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:26.455970+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":755,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.425618+0000","last_change":"2024-09-16T09:38:25.483802+0000","last_active":"2024-09-16T09:38:56.425618+0000","last_peered":"2024-09-16T09:38:56.425618+0000","last_clean":"2024-09-16T09:38:56.425618+0000","last_became_active":"2024-09-16T09:31:09.275168+0000","last_became_peered":"2024-09-16T09:31:09.275168+0000","last_unstale":"2024-09-16T09:38:56.425618+0000","last_undegraded":"2024-09-16T09:38:56.425618+0000","last_fullsized":"2024-09-16T09:38:56.425618+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:38:25.483736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:38:25.483736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:33.383406+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"285'19","reported_seq":776,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.435139+0000","last_change":"2024-09-16T09:37:31.362985+0000","last_active":"2024-09-16T09:38:56.435139+0000","last_peered":"2024-09-16T09:38:56.435139+0000","last_clean":"2024-09-16T09:38:56.435139+0000","last_became_active":"2024-09-16T09:26:33.291265+0000","last_became_peered":"2024-09-16T09:26:33.291265+0000","last_unstale":"2024-09-16T09:38:56.435139+0000","last_undegraded":"2024-09-16T09:38:56.435139+0000","last_fullsized":"2024-09-16T09:38:56.435139+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'19","last_scrub_stamp":"2024-09-16T09:37:31.362927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:37:31.362927+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":753,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.445393+0000","last_change":"2024-09-16T09:37:53.239593+0000","last_active":"2024-09-16T09:38:56.445393+0000","last_peered":"2024-09-16T09:38:56.445393+0000","last_clean":"2024-09-16T09:38:56.445393+0000","last_became_active":"2024-09-16T09:25:01.705158+0000","last_became_peered":"2024-09-16T09:25:01.705158+0000","last_unstale":"2024-09-16T09:38:56.445393+0000","last_undegraded":"2024-09-16T09:38:56.445393+0000","last_fullsized":"2024-09-16T09:38:56.445393+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:53.239547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:37:53.239547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:02.480253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":673,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.446425+0000","last_change":"2024-09-16T09:37:58.252056+0000","last_active":"2024-09-16T09:38:56.446425+0000","last_peered":"2024-09-16T09:38:56.446425+0000","last_clean":"2024-09-16T09:38:56.446425+0000","last_became_active":"2024-09-16T09:37:41.515278+0000","last_became_peered":"2024-09-16T09:37:41.515278+0000","last_unstale":"2024-09-16T09:38:56.446425+0000","last_undegraded":"2024-09-16T09:38:56.446425+0000","last_fullsized":"2024-09-16T09:38:56.446425+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:58.251994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:37:58.251994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:15.408415+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":580,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.426622+0000","last_change":"2024-09-16T09:34:48.786847+0000","last_active":"2024-09-16T09:38:56.426622+0000","last_peered":"2024-09-16T09:38:56.426622+0000","last_clean":"2024-09-16T09:38:56.426622+0000","last_became_active":"2024-09-16T09:31:09.276042+0000","last_became_peered":"2024-09-16T09:31:09.276042+0000","last_unstale":"2024-09-16T09:38:56.426622+0000","last_undegraded":"2024-09-16T09:38:56.426622+0000","last_fullsized":"2024-09-16T09:38:56.426622+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:48.786785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:34:48.786785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"285'15","reported_seq":456,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.432779+0000","last_change":"2024-09-16T09:38:30.104067+0000","last_active":"2024-09-16T09:38:56.432779+0000","last_peered":"2024-09-16T09:38:56.432779+0000","last_clean":"2024-09-16T09:38:56.432779+0000","last_became_active":"2024-09-16T09:37:33.348223+0000","last_became_peered":"2024-09-16T09:37:33.348223+0000","last_unstale":"2024-09-16T09:38:56.432779+0000","last_undegraded":"2024-09-16T09:38:56.432779+0000","last_fullsized":"2024-09-16T09:38:56.432779+0000","mapping_epoch":367,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":368,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'15","last_scrub_stamp":"2024-09-16T09:38:30.103857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:38:30.103857+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:41.048356+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,2],"acting":[6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":727,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.431581+0000","last_change":"2024-09-16T09:32:47.502667+0000","last_active":"2024-09-16T09:38:56.431581+0000","last_peered":"2024-09-16T09:38:56.431581+0000","last_clean":"2024-09-16T09:38:56.431581+0000","last_became_active":"2024-09-16T09:25:01.706762+0000","last_became_peered":"2024-09-16T09:25:01.706762+0000","last_unstale":"2024-09-16T09:38:56.431581+0000","last_undegraded":"2024-09-16T09:38:56.431581+0000","last_fullsized":"2024-09-16T09:38:56.431581+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:32:47.502598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:32:47.502598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:17.409843+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":565,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.425297+0000","last_change":"2024-09-16T09:34:46.742640+0000","last_active":"2024-09-16T09:38:56.425297+0000","last_peered":"2024-09-16T09:38:56.425297+0000","last_clean":"2024-09-16T09:38:56.425297+0000","last_became_active":"2024-09-16T09:28:52.982865+0000","last_became_peered":"2024-09-16T09:28:52.982865+0000","last_unstale":"2024-09-16T09:38:56.425297+0000","last_undegraded":"2024-09-16T09:38:56.425297+0000","last_fullsized":"2024-09-16T09:38:56.425297+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:46.742562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:34:46.742562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":725,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.424153+0000","last_change":"2024-09-16T09:34:50.779241+0000","last_active":"2024-09-16T09:38:56.424153+0000","last_peered":"2024-09-16T09:38:56.424153+0000","last_clean":"2024-09-16T09:38:56.424153+0000","last_became_active":"2024-09-16T09:25:07.715301+0000","last_became_peered":"2024-09-16T09:25:07.715301+0000","last_unstale":"2024-09-16T09:38:56.424153+0000","last_undegraded":"2024-09-16T09:38:56.424153+0000","last_fullsized":"2024-09-16T09:38:56.424153+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:50.779164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:34:50.779164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.14","version":"73'79","reported_seq":534,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.441843+0000","last_change":"2024-09-16T09:36:48.597220+0000","last_active":"2024-09-16T09:38:56.441843+0000","last_peered":"2024-09-16T09:38:56.441843+0000","last_clean":"2024-09-16T09:38:56.441843+0000","last_became_active":"2024-09-16T09:30:10.700866+0000","last_became_peered":"2024-09-16T09:30:10.700866+0000","last_unstale":"2024-09-16T09:38:56.441843+0000","last_undegraded":"2024-09-16T09:38:56.441843+0000","last_fullsized":"2024-09-16T09:38:56.441843+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:36:48.597169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:36:48.597169+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:17.024182+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"285'19","reported_seq":615,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.433627+0000","last_change":"2024-09-16T09:28:55.643505+0000","last_active":"2024-09-16T09:38:56.433627+0000","last_peered":"2024-09-16T09:38:56.433627+0000","last_clean":"2024-09-16T09:38:56.433627+0000","last_became_active":"2024-09-16T09:28:52.971132+0000","last_became_peered":"2024-09-16T09:28:52.971132+0000","last_unstale":"2024-09-16T09:38:56.433627+0000","last_undegraded":"2024-09-16T09:38:56.433627+0000","last_fullsized":"2024-09-16T09:38:56.433627+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'11","last_scrub_stamp":"2024-09-16T09:28:55.643448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:28:55.643448+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:07.375359+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":744,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.445047+0000","last_change":"2024-09-16T09:38:21.213911+0000","last_active":"2024-09-16T09:38:56.445047+0000","last_peered":"2024-09-16T09:38:56.445047+0000","last_clean":"2024-09-16T09:38:56.445047+0000","last_became_active":"2024-09-16T09:25:01.700733+0000","last_became_peered":"2024-09-16T09:25:01.700733+0000","last_unstale":"2024-09-16T09:38:56.445047+0000","last_undegraded":"2024-09-16T09:38:56.445047+0000","last_fullsized":"2024-09-16T09:38:56.445047+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:38:21.213844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:38:21.213844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:38.561435+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":736,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.431926+0000","last_change":"2024-09-16T09:38:38.663410+0000","last_active":"2024-09-16T09:38:56.431926+0000","last_peered":"2024-09-16T09:38:56.431926+0000","last_clean":"2024-09-16T09:38:56.431926+0000","last_became_active":"2024-09-16T09:25:05.713479+0000","last_became_peered":"2024-09-16T09:25:05.713479+0000","last_unstale":"2024-09-16T09:38:56.431926+0000","last_undegraded":"2024-09-16T09:38:56.431926+0000","last_fullsized":"2024-09-16T09:38:56.431926+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:38:38.663318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:38:38.663318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:43.900054+0000","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":588,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.425771+0000","last_change":"2024-09-16T09:34:51.781047+0000","last_active":"2024-09-16T09:38:56.425771+0000","last_peered":"2024-09-16T09:38:56.425771+0000","last_clean":"2024-09-16T09:38:56.425771+0000","last_became_active":"2024-09-16T09:31:09.274797+0000","last_became_peered":"2024-09-16T09:31:09.274797+0000","last_unstale":"2024-09-16T09:38:56.425771+0000","last_undegraded":"2024-09-16T09:38:56.425771+0000","last_fullsized":"2024-09-16T09:38:56.425771+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:51.780980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:34:51.780980+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"285'8","reported_seq":499,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.435720+0000","last_change":"2024-09-16T09:38:34.024059+0000","last_active":"2024-09-16T09:38:56.435720+0000","last_peered":"2024-09-16T09:38:56.435720+0000","last_clean":"2024-09-16T09:38:56.435720+0000","last_became_active":"2024-09-16T09:31:09.273682+0000","last_became_peered":"2024-09-16T09:31:09.273682+0000","last_unstale":"2024-09-16T09:38:56.435720+0000","last_undegraded":"2024-09-16T09:38:56.435720+0000","last_fullsized":"2024-09-16T09:38:56.435720+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'8","last_scrub_stamp":"2024-09-16T09:38:34.023979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:38:34.023979+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:58.880892+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":9,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":759,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.426478+0000","last_change":"2024-09-16T09:33:18.641115+0000","last_active":"2024-09-16T09:38:56.426478+0000","last_peered":"2024-09-16T09:38:56.426478+0000","last_clean":"2024-09-16T09:38:56.426478+0000","last_became_active":"2024-09-16T09:31:09.274719+0000","last_became_peered":"2024-09-16T09:31:09.274719+0000","last_unstale":"2024-09-16T09:38:56.426478+0000","last_undegraded":"2024-09-16T09:38:56.426478+0000","last_fullsized":"2024-09-16T09:38:56.426478+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:18.641069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:18.641069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":753,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.428017+0000","last_change":"2024-09-16T09:38:35.092025+0000","last_active":"2024-09-16T09:38:56.428017+0000","last_peered":"2024-09-16T09:38:56.428017+0000","last_clean":"2024-09-16T09:38:56.428017+0000","last_became_active":"2024-09-16T09:25:05.712545+0000","last_became_peered":"2024-09-16T09:25:05.712545+0000","last_unstale":"2024-09-16T09:38:56.428017+0000","last_undegraded":"2024-09-16T09:38:56.428017+0000","last_fullsized":"2024-09-16T09:38:56.428017+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:38:35.091983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:38:35.091983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:44.407146+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":731,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.430058+0000","last_change":"2024-09-16T09:37:50.502460+0000","last_active":"2024-09-16T09:38:56.430058+0000","last_peered":"2024-09-16T09:38:56.430058+0000","last_clean":"2024-09-16T09:38:56.430058+0000","last_became_active":"2024-09-16T09:25:07.720117+0000","last_became_peered":"2024-09-16T09:25:07.720117+0000","last_unstale":"2024-09-16T09:38:56.430058+0000","last_undegraded":"2024-09-16T09:38:56.430058+0000","last_fullsized":"2024-09-16T09:38:56.430058+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:50.502398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:37:50.502398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:39:13.329570+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"407'5077","reported_seq":5163,"reported_epoch":407,"state":"active+remapped+backfill_toofull","last_fresh":"2024-09-16T09:38:59.352725+0000","last_change":"2024-09-16T09:38:38.598113+0000","last_active":"2024-09-16T09:38:59.352725+0000","last_peered":"2024-09-16T09:38:59.352725+0000","last_clean":"2024-09-16T09:38:36.448336+0000","last_became_active":"2024-09-16T09:38:38.591252+0000","last_became_peered":"2024-09-16T09:38:38.591252+0000","last_unstale":"2024-09-16T09:38:59.352725+0000","last_undegraded":"2024-09-16T09:38:59.352725+0000","last_fullsized":"2024-09-16T09:38:59.352725+0000","mapping_epoch":398,"log_start":"383'300","ondisk_log_start":"383'300","created":395,"last_epoch_clean":396,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_clean_scrub_stamp":"2024-09-16T09:37:44.378497+0000","objects_scrubbed":0,"log_size":4777,"log_dups_size":0,"ondisk_log_size":4777,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":183468032,"num_objects":2800,"num_object_clones":0,"num_object_copies":8400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2800,"num_objects_unfound":0,"num_objects_dirty":2800,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2800,"num_write_kb":179168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[0,7,4],"avail_no_missing":["0","4","7"],"object_location_counts":[{"shards":"0,4,7","objects":2800}],"blocked_by":[],"up_primary":5,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":369,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.445937+0000","last_change":"2024-09-16T09:36:52.091335+0000","last_active":"2024-09-16T09:38:56.445937+0000","last_peered":"2024-09-16T09:38:56.445937+0000","last_clean":"2024-09-16T09:38:56.445937+0000","last_became_active":"2024-09-16T09:33:30.248036+0000","last_became_peered":"2024-09-16T09:33:30.248036+0000","last_unstale":"2024-09-16T09:38:56.445937+0000","last_undegraded":"2024-09-16T09:38:56.445937+0000","last_fullsized":"2024-09-16T09:38:56.445937+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:52.088284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:36:52.088284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":595,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.424786+0000","last_change":"2024-09-16T09:37:18.825035+0000","last_active":"2024-09-16T09:38:56.424786+0000","last_peered":"2024-09-16T09:38:56.424786+0000","last_clean":"2024-09-16T09:38:56.424786+0000","last_became_active":"2024-09-16T09:31:09.271662+0000","last_became_peered":"2024-09-16T09:31:09.271662+0000","last_unstale":"2024-09-16T09:38:56.424786+0000","last_undegraded":"2024-09-16T09:38:56.424786+0000","last_fullsized":"2024-09-16T09:38:56.424786+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:18.824937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:37:18.824937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":79,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"285'19","reported_seq":773,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.440056+0000","last_change":"2024-09-16T09:37:25.995663+0000","last_active":"2024-09-16T09:38:56.440056+0000","last_peered":"2024-09-16T09:38:56.440056+0000","last_clean":"2024-09-16T09:38:56.440056+0000","last_became_active":"2024-09-16T09:25:03.707769+0000","last_became_peered":"2024-09-16T09:25:03.707769+0000","last_unstale":"2024-09-16T09:38:56.440056+0000","last_undegraded":"2024-09-16T09:38:56.440056+0000","last_fullsized":"2024-09-16T09:38:56.440056+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'19","last_scrub_stamp":"2024-09-16T09:37:25.995607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:37:25.995607+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":730,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-16T09:38:56.425755+0000","last_change":"2024-09-16T09:37:26.646784+0000","last_active":"2024-09-16T09:38:56.425755+0000","last_peered":"2024-09-16T09:38:56.425755+0000","last_clean":"2024-09-16T09:38:56.425755+0000","last_became_active":"2024-09-16T09:25:05.711952+0000","last_became_peered":"2024-09-16T09:25:05.711952+0000","last_unstale":"2024-09-16T09:38:56.425755+0000","last_undegraded":"2024-09-16T09:38:56.425755+0000","last_fullsized":"2024-09-16T09:38:56.425755+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:26.646686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:37:26.646686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":26,"stat_sum":{"num_bytes":6024200192,"num_objects":91924,"num_object_clones":0,"num_object_copies":275772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13884,"num_objects_misplaced":22258,"num_objects_unfound":0,"num_objects_dirty":91924,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":68762,"num_write_kb":4400640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6776,"num_bytes_recovered":444071936,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":18533056512,"data_stored":18533056512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128554,"ondisk_log_size":128554,"up":78,"acting":71,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":1117,"num_read_kb":874,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":137,"num_bytes_recovered":3576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":681,"ondisk_log_size":681,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1986,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":834,"num_read_kb":829,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":21,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":327680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1659,"ondisk_log_size":1659,"up":63,"acting":63,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266861,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2299460,"kb_used_data":1694900,"kb_used_omap":1,"kb_used_meta":604542,"kb_avail":91445692,"statfs":{"total":95995035648,"available":93640388608,"internally_reserved":0,"allocated":1735577600,"data_stored":1734841710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":619051466},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495794,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3115436,"kb_used_data":2664988,"kb_used_omap":1,"kb_used_meta":450430,"kb_avail":90629716,"statfs":{"total":95995035648,"available":92804829184,"internally_reserved":0,"allocated":2728947712,"data_stored":2728235300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":461240774},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,3,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724729,"num_pgs":70,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2444348,"kb_used_data":1791212,"kb_used_omap":1,"kb_used_meta":653118,"kb_avail":91300804,"statfs":{"total":95995035648,"available":93492023296,"internally_reserved":0,"allocated":1834201088,"data_stored":1833473415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":668793286},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953662,"num_pgs":79,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3651524,"kb_used_data":3096948,"kb_used_omap":1,"kb_used_meta":554558,"kb_avail":90093628,"statfs":{"total":95995035648,"available":92255875072,"internally_reserved":0,"allocated":3171274752,"data_stored":3170538816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":567867846},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149891,"num_pgs":76,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2523156,"kb_used_data":1842244,"kb_used_omap":1,"kb_used_meta":680894,"kb_avail":91221996,"statfs":{"total":95995035648,"available":93411323904,"internally_reserved":0,"allocated":1886457856,"data_stored":1885706476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":697235911},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411529,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2748908,"kb_used_data":2226268,"kb_used_omap":1,"kb_used_meta":522622,"kb_avail":90996244,"statfs":{"total":95995035648,"available":93180153856,"internally_reserved":0,"allocated":2279698432,"data_stored":2278986386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":535165381},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,1,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575055,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1894696,"kb_used_data":1195732,"kb_used_omap":1,"kb_used_meta":698942,"kb_avail":91850456,"statfs":{"total":95995035648,"available":94054866944,"internally_reserved":0,"allocated":1224429568,"data_stored":1223660954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":715717061},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,2,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738581,"num_pgs":68,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":4094600,"kb_used_data":3481720,"kb_used_omap":1,"kb_used_meta":612862,"kb_avail":89650552,"statfs":{"total":95995035648,"available":91802165248,"internally_reserved":0,"allocated":3565281280,"data_stored":3564541560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":627571141},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,0,1,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":3581542400,"data_stored":3581542400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1250492416,"data_stored":1250492416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2295267328,"data_stored":2295267328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1888288768,"data_stored":1888288768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":3201236992,"data_stored":3201236992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1829896192,"data_stored":1829896192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":2748448768,"data_stored":2748448768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1737883648,"data_stored":1737883648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T09:39:05.457 INFO:tasks.thrashosds.thrasher:Setting 5.10 to [5, 3, 7, 4, 2, 0] 2024-09-16T09:39:05.457 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '5.10', '5', '3', '7', '4', '2', '0'] 2024-09-16T09:39:05.457 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd pg-upmap-items 5.10 5 3 7 4 2 0 2024-09-16T09:39:05.548 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:39:05.561 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 69 16 95500 95484 86.4525 68 0.0144139 0.011561 2024-09-16T09:39:05.587 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:05 smithi084 ceph-mon[32629]: pgmap v744: 178 pgs: 4 active+recovering+undersized+remapped, 3 active+recovery_wait+undersized+degraded+remapped, 2 active+remapped+backfill_toofull, 169 active+clean; 5.6 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.2 KiB/s rd, 65 MiB/s wr, 1.05k op/s; 13884/277299 objects degraded (5.007%); 21718/277299 objects misplaced (7.832%); 16 MiB/s, 260 objects/s recovering 2024-09-16T09:39:05.591 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:05.592 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:05.593 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:39:05.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:05 smithi070 ceph-mon[30388]: pgmap v744: 178 pgs: 4 active+recovering+undersized+remapped, 3 active+recovery_wait+undersized+degraded+remapped, 2 active+remapped+backfill_toofull, 169 active+clean; 5.6 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.2 KiB/s rd, 65 MiB/s wr, 1.05k op/s; 13884/277299 objects degraded (5.007%); 21718/277299 objects misplaced (7.832%); 16 MiB/s, 260 objects/s recovering 2024-09-16T09:39:05.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:05 smithi070 ceph-mon[40691]: pgmap v744: 178 pgs: 4 active+recovering+undersized+remapped, 3 active+recovery_wait+undersized+degraded+remapped, 2 active+remapped+backfill_toofull, 169 active+clean; 5.6 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.2 KiB/s rd, 65 MiB/s wr, 1.05k op/s; 13884/277299 objects degraded (5.007%); 21718/277299 objects misplaced (7.832%); 16 MiB/s, 260 objects/s recovering 2024-09-16T09:39:05.694 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:39:05.724 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:05.725 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:05.725 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:39:05.740 INFO:teuthology.orchestra.run.smithi084.stdout:ERROR: (22) Invalid argument 2024-09-16T09:39:05.740 INFO:teuthology.orchestra.run.smithi084.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:39:05.826 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:39:05.853 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:05.854 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:05.854 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:39:05.954 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:39:05.977 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:39:05.982 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:05.983 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:05.983 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:39:06.084 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:39:06.114 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:06.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:06.115 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:39:06.216 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:39:06.239 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:06.240 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:06.240 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:39:06.341 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:39:06.367 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:06.368 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:06.369 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:39:06.414 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.7 dump_ops_in_flight 2024-09-16T09:39:06.469 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:39:06.492 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:06.493 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:06.493 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:39:06.528 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd unset noscrub 2024-09-16T09:39:06.562 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 70 16 96020 96004 85.6818 32.5 0.0164615 0.0116664 2024-09-16T09:39:06.593 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:39:06.627 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:06.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:06.628 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:39:06.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:06 smithi084 ceph-mon[32629]: osdmap e410: 8 total, 8 up, 8 in 2024-09-16T09:39:06.729 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:39:06.744 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:39:06.751 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:06.752 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:06.752 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:39:06.853 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:39:06.912 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:06.913 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:06.913 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:39:06.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:06 smithi070 ceph-mon[30388]: osdmap e410: 8 total, 8 up, 8 in 2024-09-16T09:39:06.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:06 smithi070 ceph-mon[40691]: osdmap e410: 8 total, 8 up, 8 in 2024-09-16T09:39:07.015 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:39:07.036 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:07.037 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:07.037 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:39:07.138 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:39:07.177 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:07.177 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:07.177 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:39:07.279 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:39:07.300 INFO:teuthology.orchestra.run.smithi084.stdout:ERROR: (22) Invalid argument 2024-09-16T09:39:07.300 INFO:teuthology.orchestra.run.smithi084.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:39:07.302 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:07.303 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:07.303 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:39:07.404 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:39:07.437 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:07.438 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:07.438 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:39:07.539 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:39:07.560 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:07.561 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:07.561 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:39:07.562 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 71 16 96906 96890 85.255 55.375 0.00867582 0.0117171 2024-09-16T09:39:07.662 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:39:07.703 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:07.703 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:07.704 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:39:07.747 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:07 smithi084 ceph-mon[32629]: pgmap v747: 178 pgs: 4 active+recovering+undersized+remapped, 3 active+recovery_wait+undersized+degraded+remapped, 2 active+remapped+backfill_toofull, 169 active+clean; 5.7 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 852 B/s rd, 48 MiB/s wr, 765 op/s; 13884/279852 objects degraded (4.961%); 21857/279852 objects misplaced (7.810%); 11 MiB/s, 181 objects/s recovering 2024-09-16T09:39:07.747 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:07 smithi084 ceph-mon[32629]: osdmap e411: 8 total, 8 up, 8 in 2024-09-16T09:39:07.805 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:39:07.846 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:07.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:07.847 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:39:07.904 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T09:39:07.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:07 smithi070 ceph-mon[30388]: pgmap v747: 178 pgs: 4 active+recovering+undersized+remapped, 3 active+recovery_wait+undersized+degraded+remapped, 2 active+remapped+backfill_toofull, 169 active+clean; 5.7 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 852 B/s rd, 48 MiB/s wr, 765 op/s; 13884/279852 objects degraded (4.961%); 21857/279852 objects misplaced (7.810%); 11 MiB/s, 181 objects/s recovering 2024-09-16T09:39:07.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:07 smithi070 ceph-mon[30388]: osdmap e411: 8 total, 8 up, 8 in 2024-09-16T09:39:07.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:07 smithi070 ceph-mon[40691]: pgmap v747: 178 pgs: 4 active+recovering+undersized+remapped, 3 active+recovery_wait+undersized+degraded+remapped, 2 active+remapped+backfill_toofull, 169 active+clean; 5.7 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 852 B/s rd, 48 MiB/s wr, 765 op/s; 13884/279852 objects degraded (4.961%); 21857/279852 objects misplaced (7.810%); 11 MiB/s, 181 objects/s recovering 2024-09-16T09:39:07.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:07 smithi070 ceph-mon[40691]: osdmap e411: 8 total, 8 up, 8 in 2024-09-16T09:39:07.948 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:39:07.975 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:07.976 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:07.976 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:39:08.077 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:39:08.121 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.7 dump_blocked_ops 2024-09-16T09:39:08.127 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:08.128 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:08.128 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:39:08.229 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:39:08.251 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:08.252 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:08.253 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:39:08.353 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:39:08.399 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:08.400 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:08.400 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:39:08.481 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:39:08.501 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:39:08.557 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:08.558 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:08.558 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:39:08.563 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 72 16 97863 97847 84.9002 59.8125 0.0118662 0.0117735 2024-09-16T09:39:08.659 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:39:08.688 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:08.688 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:08.689 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:39:08.790 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:39:08.794 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:08 smithi084 ceph-mon[32629]: 10.17 scrub starts 2024-09-16T09:39:08.831 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:08.832 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:08.832 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:39:08.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:08 smithi070 ceph-mon[30388]: 10.17 scrub starts 2024-09-16T09:39:08.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:08 smithi070 ceph-mon[40691]: 10.17 scrub starts 2024-09-16T09:39:08.933 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:39:08.973 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:08.973 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:08.973 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:39:09.074 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:39:09.109 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:09.110 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:09.110 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:39:09.119 INFO:teuthology.orchestra.run.smithi084.stdout:ERROR: (22) Invalid argument 2024-09-16T09:39:09.119 INFO:teuthology.orchestra.run.smithi084.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:39:09.211 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:39:09.232 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:09.232 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:09.233 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:39:09.334 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:39:09.367 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:09.367 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:09.368 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:39:09.469 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:39:09.504 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:09.505 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:09.505 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:39:09.563 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 73 16 98764 98748 84.5087 56.3125 0.0267728 0.0118282 2024-09-16T09:39:09.606 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:39:09.646 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:09.647 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:09.647 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:39:09.748 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:39:09.786 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:09.787 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:09.787 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:39:09.888 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:39:09.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:09 smithi084 ceph-mon[32629]: pgmap v749: 178 pgs: 4 active+recovering+undersized+remapped, 3 active+recovery_wait+undersized+degraded+remapped, 2 active+remapped+backfill_toofull, 169 active+clean; 5.7 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 17 MiB/s wr, 267 op/s; 13884/279852 objects degraded (4.961%); 21857/279852 objects misplaced (7.810%); 11 MiB/s, 173 objects/s recovering 2024-09-16T09:39:09.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:09 smithi084 ceph-mon[32629]: Health check update: Degraded data redundancy: 13884/279852 objects degraded (4.961%), 3 pgs degraded (PG_DEGRADED) 2024-09-16T09:39:09.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:09 smithi070 ceph-mon[30388]: pgmap v749: 178 pgs: 4 active+recovering+undersized+remapped, 3 active+recovery_wait+undersized+degraded+remapped, 2 active+remapped+backfill_toofull, 169 active+clean; 5.7 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 17 MiB/s wr, 267 op/s; 13884/279852 objects degraded (4.961%); 21857/279852 objects misplaced (7.810%); 11 MiB/s, 173 objects/s recovering 2024-09-16T09:39:09.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:09 smithi070 ceph-mon[30388]: Health check update: Degraded data redundancy: 13884/279852 objects degraded (4.961%), 3 pgs degraded (PG_DEGRADED) 2024-09-16T09:39:09.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:09 smithi070 ceph-mon[40691]: pgmap v749: 178 pgs: 4 active+recovering+undersized+remapped, 3 active+recovery_wait+undersized+degraded+remapped, 2 active+remapped+backfill_toofull, 169 active+clean; 5.7 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 17 MiB/s wr, 267 op/s; 13884/279852 objects degraded (4.961%); 21857/279852 objects misplaced (7.810%); 11 MiB/s, 173 objects/s recovering 2024-09-16T09:39:09.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:09 smithi070 ceph-mon[40691]: Health check update: Degraded data redundancy: 13884/279852 objects degraded (4.961%), 3 pgs degraded (PG_DEGRADED) 2024-09-16T09:39:09.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:09.936 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:09.936 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:39:10.037 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:39:10.076 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:10.076 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:10.077 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:39:10.178 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:39:10.198 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:10.199 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:10.199 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:39:10.202 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.7 dump_historic_ops 2024-09-16T09:39:10.301 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:39:10.339 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:10.340 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:10.341 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:39:10.441 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:39:10.483 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:10.484 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:10.484 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:39:10.563 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 74 16 99751 99735 84.2003 61.6875 0.0122673 0.0118705 2024-09-16T09:39:10.573 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:39:10.585 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:39:10.629 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:10.629 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:10.629 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:39:10.730 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:39:10.762 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:10.762 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:10.763 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:39:10.807 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:10 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/4208272696' entity='client.admin' cmd={"prefix": "osd pg-upmap-items", "pgid": "5.10", "id": [5, 3, 7, 4, 2, 0]} : dispatch 2024-09-16T09:39:10.864 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:39:10.884 INFO:teuthology.orchestra.run.smithi070.stderr:set 5.10 pg_upmap_items mapping to [5->3,7->4,2->0] 2024-09-16T09:39:10.885 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:10 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/4208272696' entity='client.admin' cmd={"prefix": "osd pg-upmap-items", "pgid": "5.10", "id": [5, 3, 7, 4, 2, 0]} : dispatch 2024-09-16T09:39:10.885 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:10 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/4208272696' entity='client.admin' cmd={"prefix": "osd pg-upmap-items", "pgid": "5.10", "id": [5, 3, 7, 4, 2, 0]} : dispatch 2024-09-16T09:39:10.904 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:10.904 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:10.905 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:39:11.006 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:39:11.027 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:11.027 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:11.028 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:39:11.129 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:39:11.165 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:11.166 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:11.166 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:39:11.267 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:39:11.283 INFO:teuthology.orchestra.run.smithi084.stdout:ERROR: (22) Invalid argument 2024-09-16T09:39:11.284 INFO:teuthology.orchestra.run.smithi084.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:39:11.287 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:11.288 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:11.288 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:39:11.389 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:39:11.421 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:11.422 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:11.422 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:39:11.523 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:39:11.554 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:11.554 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:11.555 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:39:11.563 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 75 16 100650 100634 83.8269 56.1875 0.0157242 0.0119249 2024-09-16T09:39:11.656 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:39:11.688 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:11.689 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:11.690 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:39:11.790 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:39:11.843 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:11.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:11.845 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:39:11.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:11 smithi084 ceph-mon[32629]: pgmap v750: 178 pgs: 1 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 3 active+recovery_wait+undersized+degraded+remapped, 1 active+remapped+backfill_toofull, 170 active+clean; 6.0 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 852 B/s rd, 56 MiB/s wr, 890 op/s; 13884/293322 objects degraded (4.733%); 20542/293322 objects misplaced (7.003%); 13 MiB/s, 215 objects/s recovering 2024-09-16T09:39:11.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:11 smithi084 ceph-mon[32629]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull (PG_BACKFILL_FULL) 2024-09-16T09:39:11.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:11 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/4208272696' entity='client.admin' cmd='[{"prefix": "osd pg-upmap-items", "pgid": "5.10", "id": [5, 3, 7, 4, 2, 0]}]': finished 2024-09-16T09:39:11.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:11 smithi084 ceph-mon[32629]: osdmap e412: 8 total, 8 up, 8 in 2024-09-16T09:39:11.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:11 smithi070 ceph-mon[30388]: pgmap v750: 178 pgs: 1 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 3 active+recovery_wait+undersized+degraded+remapped, 1 active+remapped+backfill_toofull, 170 active+clean; 6.0 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 852 B/s rd, 56 MiB/s wr, 890 op/s; 13884/293322 objects degraded (4.733%); 20542/293322 objects misplaced (7.003%); 13 MiB/s, 215 objects/s recovering 2024-09-16T09:39:11.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:11 smithi070 ceph-mon[30388]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull (PG_BACKFILL_FULL) 2024-09-16T09:39:11.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:11 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/4208272696' entity='client.admin' cmd='[{"prefix": "osd pg-upmap-items", "pgid": "5.10", "id": [5, 3, 7, 4, 2, 0]}]': finished 2024-09-16T09:39:11.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:11 smithi070 ceph-mon[30388]: osdmap e412: 8 total, 8 up, 8 in 2024-09-16T09:39:11.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:11 smithi070 ceph-mon[40691]: pgmap v750: 178 pgs: 1 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 3 active+recovery_wait+undersized+degraded+remapped, 1 active+remapped+backfill_toofull, 170 active+clean; 6.0 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 852 B/s rd, 56 MiB/s wr, 890 op/s; 13884/293322 objects degraded (4.733%); 20542/293322 objects misplaced (7.003%); 13 MiB/s, 215 objects/s recovering 2024-09-16T09:39:11.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:11 smithi070 ceph-mon[40691]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull (PG_BACKFILL_FULL) 2024-09-16T09:39:11.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:11 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/4208272696' entity='client.admin' cmd='[{"prefix": "osd pg-upmap-items", "pgid": "5.10", "id": [5, 3, 7, 4, 2, 0]}]': finished 2024-09-16T09:39:11.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:11 smithi070 ceph-mon[40691]: osdmap e412: 8 total, 8 up, 8 in 2024-09-16T09:39:11.946 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:39:11.989 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:11.990 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:11.990 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:39:12.038 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.0 dump_ops_in_flight 2024-09-16T09:39:12.091 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:39:12.121 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:12.122 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:12.122 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:39:12.223 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:39:12.271 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:12.272 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:12.272 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:39:12.373 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:39:12.403 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:12.404 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:12.404 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:39:12.420 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:39:11 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:39:11] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:39:12.505 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:39:12.535 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:12.536 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:12.536 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:39:12.563 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 76 15 101687 101672 83.5775 64.875 0.00642584 0.0119602 2024-09-16T09:39:12.637 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:39:12.687 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:12.688 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:12.688 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:39:12.789 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:39:12.818 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:12.819 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:12.819 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:39:12.920 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:39:12.958 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:12.958 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:12.959 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:39:13.060 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:39:13.090 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:13.091 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:13.091 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:39:13.192 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:39:13.225 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:13.225 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:13.226 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:39:13.327 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:39:13.359 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:13.360 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:13.360 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:39:13.461 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:39:13.503 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:13.504 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:13.504 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:39:13.563 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 77 15 102714 102699 83.3257 64.1875 0.00859544 0.0119962 2024-09-16T09:39:13.605 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:39:13.634 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:13.635 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:13.635 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:39:13.644 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:39:13.644 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:39:13.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:13 smithi084 ceph-mon[32629]: pgmap v752: 178 pgs: 1 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 3 active+recovery_wait+undersized+degraded+remapped, 1 active+remapped+backfill_toofull, 170 active+clean; 6.0 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 814 B/s rd, 45 MiB/s wr, 714 op/s; 13884/293322 objects degraded (4.733%); 20542/293322 objects misplaced (7.003%); 11 MiB/s, 171 objects/s recovering 2024-09-16T09:39:13.660 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:39:13 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:39:13.737 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:39:13.764 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:13.765 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:13.765 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:39:13.785 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:13 smithi070 ceph-mon[30388]: pgmap v752: 178 pgs: 1 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 3 active+recovery_wait+undersized+degraded+remapped, 1 active+remapped+backfill_toofull, 170 active+clean; 6.0 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 814 B/s rd, 45 MiB/s wr, 714 op/s; 13884/293322 objects degraded (4.733%); 20542/293322 objects misplaced (7.003%); 11 MiB/s, 171 objects/s recovering 2024-09-16T09:39:13.785 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:13 smithi070 ceph-mon[40691]: pgmap v752: 178 pgs: 1 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 3 active+recovery_wait+undersized+degraded+remapped, 1 active+remapped+backfill_toofull, 170 active+clean; 6.0 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 814 B/s rd, 45 MiB/s wr, 714 op/s; 13884/293322 objects degraded (4.733%); 20542/293322 objects misplaced (7.003%); 11 MiB/s, 171 objects/s recovering 2024-09-16T09:39:13.866 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:39:13.917 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:13.918 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:13.918 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:39:14.019 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:39:14.069 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:14.070 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:14.070 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:39:14.171 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:39:14.213 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:14.213 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:14.214 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:39:14.314 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:39:14.342 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:14.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:14.343 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:39:14.444 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:39:14.486 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:14.487 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:14.487 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:39:14.564 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 78 16 103754 103738 83.09 64.9375 0.0232101 0.0120302 2024-09-16T09:39:14.588 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:39:14.633 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:14.633 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:14.634 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:39:14.735 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:39:14.763 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:14.764 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:14.764 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:39:14.865 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:39:14.900 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:14.901 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:14.901 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:39:14.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:14 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:39:14.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:14 smithi084 ceph-mon[32629]: Health check update: Degraded data redundancy: 13884/293322 objects degraded (4.733%), 3 pgs degraded (PG_DEGRADED) 2024-09-16T09:39:14.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:14 smithi084 ceph-mon[32629]: osdmap e413: 8 total, 8 up, 8 in 2024-09-16T09:39:14.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:14 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:39:14.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:14 smithi070 ceph-mon[30388]: Health check update: Degraded data redundancy: 13884/293322 objects degraded (4.733%), 3 pgs degraded (PG_DEGRADED) 2024-09-16T09:39:14.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:14 smithi070 ceph-mon[30388]: osdmap e413: 8 total, 8 up, 8 in 2024-09-16T09:39:14.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:14 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:39:14.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:14 smithi070 ceph-mon[40691]: Health check update: Degraded data redundancy: 13884/293322 objects degraded (4.733%), 3 pgs degraded (PG_DEGRADED) 2024-09-16T09:39:14.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:14 smithi070 ceph-mon[40691]: osdmap e413: 8 total, 8 up, 8 in 2024-09-16T09:39:15.003 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:39:15.024 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:15.024 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:15.025 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:39:15.125 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:39:15.171 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:15.171 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:15.172 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:39:15.272 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:39:15.295 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:15.296 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:15.296 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:39:15.397 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:39:15.429 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:15.430 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:15.430 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:39:15.531 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:39:15.568 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 79 16 104618 104602 82.7176 54 0.0122482 0.0120847 2024-09-16T09:39:15.579 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:15.580 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:15.580 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:39:15.681 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:39:15.729 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:15.729 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:15.730 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:39:15.831 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:39:15.851 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:15.852 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:15.852 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:39:15.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:15 smithi084 ceph-mon[32629]: pgmap v753: 178 pgs: 1 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 3 active+recovery_wait+undersized+degraded+remapped, 1 active+remapped+backfill_toofull, 170 active+clean; 6.2 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.2 KiB/s rd, 65 MiB/s wr, 1.05k op/s; 13884/305010 objects degraded (4.552%); 19550/305010 objects misplaced (6.410%); 12 MiB/s, 196 objects/s recovering 2024-09-16T09:39:15.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:15 smithi070 ceph-mon[30388]: pgmap v753: 178 pgs: 1 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 3 active+recovery_wait+undersized+degraded+remapped, 1 active+remapped+backfill_toofull, 170 active+clean; 6.2 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.2 KiB/s rd, 65 MiB/s wr, 1.05k op/s; 13884/305010 objects degraded (4.552%); 19550/305010 objects misplaced (6.410%); 12 MiB/s, 196 objects/s recovering 2024-09-16T09:39:15.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:15 smithi070 ceph-mon[40691]: pgmap v753: 178 pgs: 1 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 3 active+recovery_wait+undersized+degraded+remapped, 1 active+remapped+backfill_toofull, 170 active+clean; 6.2 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.2 KiB/s rd, 65 MiB/s wr, 1.05k op/s; 13884/305010 objects degraded (4.552%); 19550/305010 objects misplaced (6.410%); 12 MiB/s, 196 objects/s recovering 2024-09-16T09:39:15.953 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:39:15.995 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:15.995 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:15.996 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:39:16.097 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:39:16.121 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:16.121 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:16.122 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:39:16.223 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:39:16.246 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:16.247 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:16.247 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:39:16.348 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:39:16.386 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:16.386 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:16.387 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:39:16.487 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:39:16.510 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:16.511 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:16.511 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:39:16.568 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:2024-09-16T09:39:16.567586+0000 min lat: 0.00121703 max lat: 5.46054 avg lat: 0.0121657 2024-09-16T09:39:16.568 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-16T09:39:16.568 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 80 16 105235 105219 82.1658 38.5625 0.0215171 0.0121657 2024-09-16T09:39:16.612 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:39:16.644 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:16.645 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:16.645 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:39:16.738 INFO:teuthology.orchestra.run.smithi070.stderr:noscrub is unset 2024-09-16T09:39:16.745 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:39:16.789 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:16.790 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:16.790 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:39:16.891 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:39:16.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:16 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3543691772' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:39:16.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:16 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:39:16.912 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:16.913 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:16.913 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:39:16.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:16 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3543691772' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:39:16.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:16 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:39:16.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:16 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3543691772' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:39:16.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:16 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:39:17.014 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:39:17.047 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:17.048 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:17.048 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:39:17.149 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:39:17.191 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:17.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:17.192 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:39:17.292 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:39:17.331 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:17.332 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:17.332 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:39:17.433 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:39:17.476 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:17.477 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:17.477 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:39:17.568 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 81 16 106285 106269 81.9616 65.625 0.014777 0.0121967 2024-09-16T09:39:17.578 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:39:17.618 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:17.619 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:17.620 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:39:17.720 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:39:17.748 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:17.748 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:17.749 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:39:17.850 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:39:17.886 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:17.887 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:17.888 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:39:17.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:17 smithi084 ceph-mon[32629]: pgmap v755: 178 pgs: 1 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 3 active+recovery_wait+undersized+degraded+remapped, 1 active+remapped+backfill_toofull, 170 active+clean; 6.3 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.2 KiB/s rd, 73 MiB/s wr, 1.17k op/s; 13884/307863 objects degraded (4.510%); 19130/307863 objects misplaced (6.214%); 19 MiB/s, 296 objects/s recovering 2024-09-16T09:39:17.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:17 smithi084 ceph-mon[32629]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:39:17.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:17 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-16T09:39:17.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:17 smithi084 ceph-mon[32629]: osdmap e414: 8 total, 8 up, 8 in 2024-09-16T09:39:17.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:17 smithi070 ceph-mon[30388]: pgmap v755: 178 pgs: 1 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 3 active+recovery_wait+undersized+degraded+remapped, 1 active+remapped+backfill_toofull, 170 active+clean; 6.3 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.2 KiB/s rd, 73 MiB/s wr, 1.17k op/s; 13884/307863 objects degraded (4.510%); 19130/307863 objects misplaced (6.214%); 19 MiB/s, 296 objects/s recovering 2024-09-16T09:39:17.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:17 smithi070 ceph-mon[30388]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:39:17.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:17 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-16T09:39:17.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:17 smithi070 ceph-mon[30388]: osdmap e414: 8 total, 8 up, 8 in 2024-09-16T09:39:17.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:17 smithi070 ceph-mon[40691]: pgmap v755: 178 pgs: 1 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 3 active+recovery_wait+undersized+degraded+remapped, 1 active+remapped+backfill_toofull, 170 active+clean; 6.3 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.2 KiB/s rd, 73 MiB/s wr, 1.17k op/s; 13884/307863 objects degraded (4.510%); 19130/307863 objects misplaced (6.214%); 19 MiB/s, 296 objects/s recovering 2024-09-16T09:39:17.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:17 smithi070 ceph-mon[40691]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:39:17.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:17 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-16T09:39:17.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:17 smithi070 ceph-mon[40691]: osdmap e414: 8 total, 8 up, 8 in 2024-09-16T09:39:17.989 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:39:18.028 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:18.029 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:18.030 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:39:18.130 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:39:18.169 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:18.170 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:18.170 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:39:18.271 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:39:18.303 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:18.304 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:18.304 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:39:18.405 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:39:18.445 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:18.446 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:18.446 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:39:18.547 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:39:18.557 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T09:39:18.557 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T09:39:18.558 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-16T09:39:18.558 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph tell osd.0 'trim stale osdmaps' 2024-09-16T09:39:18.568 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 82 15 107323 107308 81.7534 64.9375 0.00986774 0.0122262 2024-09-16T09:39:18.588 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:18.589 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:18.589 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:39:18.690 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:39:18.718 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:18.719 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:18.719 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:39:18.820 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:39:18.892 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:18.893 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:18.893 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:39:18.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:18 smithi084 ceph-mon[32629]: pgmap v757: 178 pgs: 1 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 3 active+recovery_wait+undersized+degraded+remapped, 1 active+remapped+backfill_toofull, 170 active+clean; 6.3 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 707 B/s rd, 42 MiB/s wr, 670 op/s; 13884/307863 objects degraded (4.510%); 19130/307863 objects misplaced (6.214%); 11 MiB/s, 179 objects/s recovering 2024-09-16T09:39:18.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:18 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:39:18.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:18 smithi070 ceph-mon[40691]: pgmap v757: 178 pgs: 1 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 3 active+recovery_wait+undersized+degraded+remapped, 1 active+remapped+backfill_toofull, 170 active+clean; 6.3 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 707 B/s rd, 42 MiB/s wr, 670 op/s; 13884/307863 objects degraded (4.510%); 19130/307863 objects misplaced (6.214%); 11 MiB/s, 179 objects/s recovering 2024-09-16T09:39:18.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:18 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:39:18.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:18 smithi070 ceph-mon[30388]: pgmap v757: 178 pgs: 1 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 3 active+recovery_wait+undersized+degraded+remapped, 1 active+remapped+backfill_toofull, 170 active+clean; 6.3 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 707 B/s rd, 42 MiB/s wr, 670 op/s; 13884/307863 objects degraded (4.510%); 19130/307863 objects misplaced (6.214%); 11 MiB/s, 179 objects/s recovering 2024-09-16T09:39:18.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:18 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:39:18.994 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:39:19.017 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:19.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:19.019 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:39:19.120 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:39:19.161 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:19.162 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:19.162 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:39:19.263 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:39:19.305 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:19.306 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:19.306 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:39:19.407 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:39:19.428 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:19.429 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:19.429 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:39:19.530 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:39:19.569 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 83 16 108266 108250 81.4778 58.875 0.0169652 0.0122686 2024-09-16T09:39:19.574 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:19.575 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:19.576 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:39:19.676 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:39:19.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:19.709 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:19.709 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:39:19.810 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:39:19.842 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:19.843 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:19.844 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:39:19.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:19 smithi084 ceph-mon[32629]: Health check update: Degraded data redundancy: 13884/307863 objects degraded (4.510%), 3 pgs degraded (PG_DEGRADED) 2024-09-16T09:39:19.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:19 smithi084 ceph-mon[32629]: osdmap e415: 8 total, 8 up, 8 in 2024-09-16T09:39:19.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:19 smithi070 ceph-mon[30388]: Health check update: Degraded data redundancy: 13884/307863 objects degraded (4.510%), 3 pgs degraded (PG_DEGRADED) 2024-09-16T09:39:19.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:19 smithi070 ceph-mon[30388]: osdmap e415: 8 total, 8 up, 8 in 2024-09-16T09:39:19.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:19 smithi070 ceph-mon[40691]: Health check update: Degraded data redundancy: 13884/307863 objects degraded (4.510%), 3 pgs degraded (PG_DEGRADED) 2024-09-16T09:39:19.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:19 smithi070 ceph-mon[40691]: osdmap e415: 8 total, 8 up, 8 in 2024-09-16T09:39:19.944 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:39:19.978 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:19.978 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:19.979 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:39:20.079 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:39:20.131 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:20.131 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:20.132 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:39:20.233 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:39:20.278 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:20.279 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:20.279 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:39:20.380 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:39:20.401 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:20.402 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:20.402 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:39:20.503 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:39:20.538 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:20.539 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:20.539 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:39:20.569 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 84 16 109374 109358 81.3322 69.25 0.0104515 0.0122907 2024-09-16T09:39:20.641 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:39:20.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:20 smithi070 ceph-mon[30388]: pgmap v759: 178 pgs: 1 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 3 active+recovery_wait+undersized+degraded+remapped, 1 active+remapped+backfill_toofull, 170 active+clean; 6.5 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 853 B/s rd, 58 MiB/s wr, 924 op/s; 13884/321645 objects degraded (4.317%); 17346/321645 objects misplaced (5.393%); 24 MiB/s, 382 objects/s recovering 2024-09-16T09:39:20.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:20 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3242415458' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:39:20.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:20 smithi070 ceph-mon[40691]: pgmap v759: 178 pgs: 1 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 3 active+recovery_wait+undersized+degraded+remapped, 1 active+remapped+backfill_toofull, 170 active+clean; 6.5 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 853 B/s rd, 58 MiB/s wr, 924 op/s; 13884/321645 objects degraded (4.317%); 17346/321645 objects misplaced (5.393%); 24 MiB/s, 382 objects/s recovering 2024-09-16T09:39:20.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:20 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3242415458' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:39:20.673 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:20.674 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:20.674 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:39:20.775 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:39:20.806 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:20.807 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:20.807 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:39:20.908 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:39:20.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:20 smithi084 ceph-mon[32629]: pgmap v759: 178 pgs: 1 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 3 active+recovery_wait+undersized+degraded+remapped, 1 active+remapped+backfill_toofull, 170 active+clean; 6.5 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 853 B/s rd, 58 MiB/s wr, 924 op/s; 13884/321645 objects degraded (4.317%); 17346/321645 objects misplaced (5.393%); 24 MiB/s, 382 objects/s recovering 2024-09-16T09:39:20.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:20 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3242415458' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:39:20.940 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:39:20.940 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:39:20.941 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:39:20.941 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:39:20.941 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:39:20.941 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:39:20.941 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:39:20.941 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:39:20.941 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:39:20.941 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:39:20.942 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:39:20.942 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:39:20.942 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:39:20.942 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:39:20.942 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:39:20.942 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:39:20.945 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:20.945 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:20.946 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:39:21.046 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:39:21.079 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:21.079 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:21.080 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:39:21.180 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:39:21.217 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:21.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:21.219 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:39:21.319 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:39:21.355 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:21.356 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:21.356 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:39:21.457 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:39:21.489 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:21.489 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:21.490 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:39:21.569 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 85 16 110563 110547 81.2495 74.3125 0.0120255 0.0123031 2024-09-16T09:39:21.590 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:39:21.613 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:21.613 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:21.614 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:39:21.715 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:39:21.746 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:21.747 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:21.747 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:39:21.848 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:39:21.888 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:21.889 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:21.889 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:39:21.990 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:39:22.011 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:22.011 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:22.012 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:39:22.113 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:39:22.142 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:22.143 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:22.143 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:39:22.208 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:39:21 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:39:21] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:39:22.243 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:39:22.272 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:22.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:22.274 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:39:22.374 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:39:22.415 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:22.415 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:22.415 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:39:22.516 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:39:22.553 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:22.553 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:22.554 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:39:22.569 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 86 16 111376 111360 80.8957 50.8125 0.0117297 0.0123568 2024-09-16T09:39:22.655 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:39:22.702 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:22.703 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:22.703 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:39:22.805 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:39:22.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:22.849 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:22.849 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:39:22.950 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:39:22.991 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:22.992 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:22.992 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:39:23.093 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:39:23.150 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:23.151 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:23.151 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:39:23.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:22 smithi084 ceph-mon[32629]: pgmap v760: 178 pgs: 1 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 3 active+recovery_wait+undersized+degraded+remapped, 1 active+remapped+backfill_toofull, 170 active+clean; 6.5 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 689 B/s rd, 47 MiB/s wr, 747 op/s; 13884/321645 objects degraded (4.317%); 17346/321645 objects misplaced (5.393%); 19 MiB/s, 308 objects/s recovering 2024-09-16T09:39:23.170 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:22 smithi070 ceph-mon[30388]: pgmap v760: 178 pgs: 1 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 3 active+recovery_wait+undersized+degraded+remapped, 1 active+remapped+backfill_toofull, 170 active+clean; 6.5 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 689 B/s rd, 47 MiB/s wr, 747 op/s; 13884/321645 objects degraded (4.317%); 17346/321645 objects misplaced (5.393%); 19 MiB/s, 308 objects/s recovering 2024-09-16T09:39:23.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:22 smithi070 ceph-mon[40691]: pgmap v760: 178 pgs: 1 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 3 active+recovery_wait+undersized+degraded+remapped, 1 active+remapped+backfill_toofull, 170 active+clean; 6.5 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 689 B/s rd, 47 MiB/s wr, 747 op/s; 13884/321645 objects degraded (4.317%); 17346/321645 objects misplaced (5.393%); 19 MiB/s, 308 objects/s recovering 2024-09-16T09:39:23.193 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:39:23.252 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:39:23.258 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:39:23.330 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:23.330 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:23.330 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:39:23.431 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:39:23.479 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:23.479 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:23.480 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:39:23.569 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 87 16 112419 112403 80.7151 65.1875 0.0113872 0.0123845 2024-09-16T09:39:23.581 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:39:23.634 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:23.635 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:23.635 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:39:23.660 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:39:23 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:39:23.735 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:39:23.789 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:23.790 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:23.791 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:39:23.891 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:39:23.913 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:23.913 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:23.914 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:39:23.953 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:23 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:39:23.954 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:23 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:39:23.954 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:23 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:39:23.954 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:23 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:39:24.015 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:39:24.035 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T09:39:24.035 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [ 2024-09-16T09:39:24.035 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:24.035 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15975.0:112715 10.12 10:4c4ae58b:::benchmark_data_smithi070_166744_object112714:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e415)", 2024-09-16T09:39:24.035 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:39:23.857653+0000", 2024-09-16T09:39:24.035 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.094648540000000003, 2024-09-16T09:39:24.035 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.094679017000000004, 2024-09-16T09:39:24.036 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:39:24.036 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:39:24.036 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "waiting for sub ops", 2024-09-16T09:39:24.036 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:39:24.036 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15975", 2024-09-16T09:39:24.036 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3821727158", 2024-09-16T09:39:24.036 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 112715 2024-09-16T09:39:24.036 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:24.036 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:39:24.036 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:24.036 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:39:24.036 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:23.857653+0000", 2024-09-16T09:39:24.037 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:39:24.037 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:24.037 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:24.037 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:39:24.037 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:23.857653+0000", 2024-09-16T09:39:24.037 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:39:24.037 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:24.037 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:24.037 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:39:24.037 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:23.857656+0000", 2024-09-16T09:39:24.037 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.193e-06 2024-09-16T09:39:24.037 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:24.037 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:24.038 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:39:24.038 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:23.857681+0000", 2024-09-16T09:39:24.038 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.5530000000000001e-05 2024-09-16T09:39:24.038 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:24.038 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:24.038 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:39:24.038 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:23.857697+0000", 2024-09-16T09:39:24.038 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5988999999999999e-05 2024-09-16T09:39:24.038 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:24.038 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:24.038 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:39:24.038 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:23.857707+0000", 2024-09-16T09:39:24.038 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.7790000000000007e-06 2024-09-16T09:39:24.038 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:24.039 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:24.039 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:39:24.039 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:23.857772+0000", 2024-09-16T09:39:24.039 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.5179999999999996e-05 2024-09-16T09:39:24.039 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:24.039 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:24.039 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:39:24.039 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:23.857934+0000", 2024-09-16T09:39:24.039 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000162091 2024-09-16T09:39:24.039 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:24.039 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:24.039 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,2,4,5,6", 2024-09-16T09:39:24.039 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:23.858331+0000", 2024-09-16T09:39:24.039 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00039647100000000002 2024-09-16T09:39:24.040 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:24.040 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:24.040 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:39:24.040 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:23.863507+0000", 2024-09-16T09:39:24.040 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0051765259999999999 2024-09-16T09:39:24.040 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:24.040 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:24.040 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:39:24.040 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:23.863759+0000", 2024-09-16T09:39:24.040 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00025231299999999999 2024-09-16T09:39:24.040 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:24.040 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:24.040 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:39:24.040 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:23.865020+0000", 2024-09-16T09:39:24.040 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0012607479999999999 2024-09-16T09:39:24.041 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:39:24.041 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:39:24.041 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:39:24.041 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:39:24.041 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-16T09:39:24.041 INFO:teuthology.orchestra.run.smithi070.stdout: "num_ops": 1 2024-09-16T09:39:24.041 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T09:39:24.053 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:24.054 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:24.054 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:39:24.155 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:39:24.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:23 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:39:24.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:23 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:39:24.181 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:24.182 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:24.182 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:39:24.283 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:39:24.305 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:24.306 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:24.306 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:39:24.407 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:39:24.454 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:24.454 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:24.455 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:39:24.555 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:39:24.569 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 88 16 113090 113074 80.2746 41.9375 0.00749945 0.0124519 2024-09-16T09:39:24.588 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:24.589 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:24.589 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:39:24.690 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:39:24.729 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:24.730 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:24.730 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:39:24.784 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T09:39:24.831 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:39:24.852 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:24.852 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:24.853 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:39:24.953 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:39:24.987 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:24.988 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:24.988 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:39:25.089 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:39:25.109 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:25.110 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:25.110 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:39:25.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:24 smithi084 ceph-mon[32629]: pgmap v761: 178 pgs: 2 active+undersized+remapped, 1 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 1 active+recovery_wait+undersized+degraded+remapped, 1 active+remapped+backfill_toofull, 170 active+clean; 6.8 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1.2 KiB/s rd, 69 MiB/s wr, 1.10k op/s; 4686/334266 objects degraded (1.402%); 24926/334266 objects misplaced (7.457%); 18 MiB/s, 287 objects/s recovering 2024-09-16T09:39:25.187 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:24 smithi084 ceph-mon[32629]: osdmap e416: 8 total, 8 up, 8 in 2024-09-16T09:39:25.187 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:24 smithi084 ceph-mon[32629]: Health check update: Degraded data redundancy: 4686/334266 objects degraded (1.402%), 1 pg degraded (PG_DEGRADED) 2024-09-16T09:39:25.187 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:24 smithi084 ceph-mon[32629]: osdmap e417: 8 total, 8 up, 8 in 2024-09-16T09:39:25.188 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:24 smithi070 ceph-mon[30388]: pgmap v761: 178 pgs: 2 active+undersized+remapped, 1 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 1 active+recovery_wait+undersized+degraded+remapped, 1 active+remapped+backfill_toofull, 170 active+clean; 6.8 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1.2 KiB/s rd, 69 MiB/s wr, 1.10k op/s; 4686/334266 objects degraded (1.402%); 24926/334266 objects misplaced (7.457%); 18 MiB/s, 287 objects/s recovering 2024-09-16T09:39:25.188 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:24 smithi070 ceph-mon[30388]: osdmap e416: 8 total, 8 up, 8 in 2024-09-16T09:39:25.188 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:24 smithi070 ceph-mon[30388]: Health check update: Degraded data redundancy: 4686/334266 objects degraded (1.402%), 1 pg degraded (PG_DEGRADED) 2024-09-16T09:39:25.188 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:24 smithi070 ceph-mon[30388]: osdmap e417: 8 total, 8 up, 8 in 2024-09-16T09:39:25.188 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:24 smithi070 ceph-mon[40691]: pgmap v761: 178 pgs: 2 active+undersized+remapped, 1 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 1 active+recovery_wait+undersized+degraded+remapped, 1 active+remapped+backfill_toofull, 170 active+clean; 6.8 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1.2 KiB/s rd, 69 MiB/s wr, 1.10k op/s; 4686/334266 objects degraded (1.402%); 24926/334266 objects misplaced (7.457%); 18 MiB/s, 287 objects/s recovering 2024-09-16T09:39:25.188 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:24 smithi070 ceph-mon[40691]: osdmap e416: 8 total, 8 up, 8 in 2024-09-16T09:39:25.188 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:24 smithi070 ceph-mon[40691]: Health check update: Degraded data redundancy: 4686/334266 objects degraded (1.402%), 1 pg degraded (PG_DEGRADED) 2024-09-16T09:39:25.188 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:24 smithi070 ceph-mon[40691]: osdmap e417: 8 total, 8 up, 8 in 2024-09-16T09:39:25.212 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:39:25.245 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:25.245 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:25.246 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:39:25.247 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-16T09:39:25.347 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:39:25.371 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:25.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:25.372 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:39:25.473 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:39:25.495 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:25.496 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:25.497 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:39:25.570 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 89 16 113711 113695 79.8088 38.8125 0.0163056 0.0125247 2024-09-16T09:39:25.597 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:39:25.629 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:25.630 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:25.630 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:39:25.731 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:39:25.769 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:25.770 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:25.770 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:39:25.871 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:39:25.899 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:25.899 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:25.900 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:39:26.001 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:39:26.037 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:26.037 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:26.038 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:39:26.139 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:39:26.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:25 smithi084 ceph-mon[32629]: 10.13 scrub starts 2024-09-16T09:39:26.170 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:26.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:25 smithi070 ceph-mon[40691]: 10.13 scrub starts 2024-09-16T09:39:26.171 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:26.171 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:39:26.172 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:25 smithi070 ceph-mon[30388]: 10.13 scrub starts 2024-09-16T09:39:26.273 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:39:26.316 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:26.317 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:26.318 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:39:26.418 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:39:26.460 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:26.461 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:26.461 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:39:26.562 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:39:26.569 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Total time run: 90.01 2024-09-16T09:39:26.569 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Total writes made: 114353 2024-09-16T09:39:26.569 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Write size: 65536 2024-09-16T09:39:26.569 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Object size: 65536 2024-09-16T09:39:26.570 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Bandwidth (MB/sec): 79.403 2024-09-16T09:39:26.570 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Stddev Bandwidth: 46.263 2024-09-16T09:39:26.570 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Max bandwidth (MB/sec): 230.312 2024-09-16T09:39:26.570 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Min bandwidth (MB/sec): 0 2024-09-16T09:39:26.570 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Average IOPS: 1270 2024-09-16T09:39:26.570 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Stddev IOPS: 740.218 2024-09-16T09:39:26.570 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Max IOPS: 3685 2024-09-16T09:39:26.570 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Min IOPS: 0 2024-09-16T09:39:26.570 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Average Latency(s): 0.0125898 2024-09-16T09:39:26.570 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Stddev Latency(s): 0.0545798 2024-09-16T09:39:26.570 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Max latency(s): 5.46054 2024-09-16T09:39:26.570 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Min latency(s): 0.00121703 2024-09-16T09:39:26.576 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Cleaning up (deleting benchmark objects) 2024-09-16T09:39:26.591 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:26.591 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:26.591 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:39:26.692 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:39:26.723 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:26.724 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:26.724 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:39:26.825 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:39:26.851 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:26.852 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:26.852 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:39:26.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:26 smithi084 ceph-mon[32629]: 10.18 scrub starts 2024-09-16T09:39:26.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:26 smithi084 ceph-mon[32629]: 10.10 scrub starts 2024-09-16T09:39:26.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:26 smithi084 ceph-mon[32629]: pgmap v764: 178 pgs: 1 active+remapped+backfilling, 2 active+undersized+remapped, 3 active+recovering+undersized+remapped, 1 active+recovery_wait+undersized+degraded+remapped, 1 active+remapped+backfill_toofull, 170 active+clean; 6.8 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 796 B/s rd, 58 MiB/s wr, 922 op/s; 4686/336828 objects degraded (1.391%); 23988/336828 objects misplaced (7.122%); 27 MiB/s, 431 objects/s recovering 2024-09-16T09:39:26.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:26 smithi084 ceph-mon[32629]: 10.13 scrub ok 2024-09-16T09:39:26.953 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:39:26.989 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:26.990 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:26.990 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:39:27.091 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:39:27.112 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:27.112 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:27.113 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:39:27.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:26 smithi070 ceph-mon[30388]: 10.18 scrub starts 2024-09-16T09:39:27.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:26 smithi070 ceph-mon[30388]: 10.10 scrub starts 2024-09-16T09:39:27.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:26 smithi070 ceph-mon[30388]: pgmap v764: 178 pgs: 1 active+remapped+backfilling, 2 active+undersized+remapped, 3 active+recovering+undersized+remapped, 1 active+recovery_wait+undersized+degraded+remapped, 1 active+remapped+backfill_toofull, 170 active+clean; 6.8 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 796 B/s rd, 58 MiB/s wr, 922 op/s; 4686/336828 objects degraded (1.391%); 23988/336828 objects misplaced (7.122%); 27 MiB/s, 431 objects/s recovering 2024-09-16T09:39:27.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:26 smithi070 ceph-mon[30388]: 10.13 scrub ok 2024-09-16T09:39:27.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:26 smithi070 ceph-mon[40691]: 10.18 scrub starts 2024-09-16T09:39:27.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:26 smithi070 ceph-mon[40691]: 10.10 scrub starts 2024-09-16T09:39:27.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:26 smithi070 ceph-mon[40691]: pgmap v764: 178 pgs: 1 active+remapped+backfilling, 2 active+undersized+remapped, 3 active+recovering+undersized+remapped, 1 active+recovery_wait+undersized+degraded+remapped, 1 active+remapped+backfill_toofull, 170 active+clean; 6.8 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 796 B/s rd, 58 MiB/s wr, 922 op/s; 4686/336828 objects degraded (1.391%); 23988/336828 objects misplaced (7.122%); 27 MiB/s, 431 objects/s recovering 2024-09-16T09:39:27.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:26 smithi070 ceph-mon[40691]: 10.13 scrub ok 2024-09-16T09:39:27.214 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:39:27.254 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:27.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:27.255 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:39:27.355 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:39:27.403 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:27.404 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:27.404 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:39:27.505 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:39:27.527 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:27.528 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:27.528 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:39:27.629 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:39:27.674 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:27.675 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:27.675 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:39:27.776 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:39:27.797 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:27.798 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:27.798 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:39:27.899 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:39:27.935 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:27.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:27.936 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:39:28.037 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:39:28.074 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:28.074 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:28.074 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:39:28.142 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:27 smithi084 ceph-mon[32629]: 10.18 scrub ok 2024-09-16T09:39:28.143 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:27 smithi084 ceph-mon[32629]: 10.10 scrub ok 2024-09-16T09:39:28.143 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:27 smithi084 ceph-mon[32629]: 5.1a scrub starts 2024-09-16T09:39:28.143 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:27 smithi084 ceph-mon[32629]: 5.1a scrub ok 2024-09-16T09:39:28.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:27 smithi070 ceph-mon[30388]: 10.18 scrub ok 2024-09-16T09:39:28.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:27 smithi070 ceph-mon[30388]: 10.10 scrub ok 2024-09-16T09:39:28.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:27 smithi070 ceph-mon[30388]: 5.1a scrub starts 2024-09-16T09:39:28.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:27 smithi070 ceph-mon[30388]: 5.1a scrub ok 2024-09-16T09:39:28.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:27 smithi070 ceph-mon[40691]: 10.18 scrub ok 2024-09-16T09:39:28.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:27 smithi070 ceph-mon[40691]: 10.10 scrub ok 2024-09-16T09:39:28.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:27 smithi070 ceph-mon[40691]: 5.1a scrub starts 2024-09-16T09:39:28.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:27 smithi070 ceph-mon[40691]: 5.1a scrub ok 2024-09-16T09:39:28.175 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:39:28.219 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:28.219 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:28.220 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:39:28.321 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:39:28.362 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:28.363 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:28.363 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:39:28.464 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:39:28.497 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:28.498 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:28.499 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:39:28.599 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:39:28.638 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:28.639 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:28.639 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:39:28.740 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:39:28.776 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:28.777 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:28.777 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:39:28.878 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:39:28.898 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:28.899 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:28.900 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:39:29.001 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:39:29.029 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:29.030 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:29.030 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:39:29.131 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:39:29.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:28 smithi084 ceph-mon[32629]: pgmap v765: 178 pgs: 1 active+remapped+backfilling, 2 active+undersized+remapped, 3 active+recovering+undersized+remapped, 1 active+recovery_wait+undersized+degraded+remapped, 1 active+remapped+backfill_toofull, 170 active+clean; 6.8 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 639 B/s rd, 40 MiB/s wr, 633 op/s; 4686/336828 objects degraded (1.391%); 23988/336828 objects misplaced (7.122%); 15 MiB/s, 242 objects/s recovering 2024-09-16T09:39:29.167 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:29.167 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:29.167 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:39:29.170 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:28 smithi070 ceph-mon[30388]: pgmap v765: 178 pgs: 1 active+remapped+backfilling, 2 active+undersized+remapped, 3 active+recovering+undersized+remapped, 1 active+recovery_wait+undersized+degraded+remapped, 1 active+remapped+backfill_toofull, 170 active+clean; 6.8 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 639 B/s rd, 40 MiB/s wr, 633 op/s; 4686/336828 objects degraded (1.391%); 23988/336828 objects misplaced (7.122%); 15 MiB/s, 242 objects/s recovering 2024-09-16T09:39:29.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:28 smithi070 ceph-mon[40691]: pgmap v765: 178 pgs: 1 active+remapped+backfilling, 2 active+undersized+remapped, 3 active+recovering+undersized+remapped, 1 active+recovery_wait+undersized+degraded+remapped, 1 active+remapped+backfill_toofull, 170 active+clean; 6.8 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 639 B/s rd, 40 MiB/s wr, 633 op/s; 4686/336828 objects degraded (1.391%); 23988/336828 objects misplaced (7.122%); 15 MiB/s, 242 objects/s recovering 2024-09-16T09:39:29.269 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:39:29.295 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:29.296 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:29.296 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:39:29.397 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:39:29.432 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:29.433 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:29.434 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:39:29.534 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:39:29.573 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:29.574 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:29.574 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:39:29.675 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:39:29.696 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:29.697 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:29.697 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:39:29.797 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:39:29.831 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:29.832 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:29.833 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:39:29.934 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:39:29.972 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:29.973 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:29.973 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:39:30.074 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:39:30.108 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:30.109 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:30.110 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:39:30.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:29 smithi084 ceph-mon[32629]: 4.11 scrub starts 2024-09-16T09:39:30.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:29 smithi084 ceph-mon[32629]: 4.11 scrub ok 2024-09-16T09:39:30.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:29 smithi084 ceph-mon[32629]: osdmap e418: 8 total, 8 up, 8 in 2024-09-16T09:39:30.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:29 smithi084 ceph-mon[32629]: 3.1e scrub starts 2024-09-16T09:39:30.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:29 smithi084 ceph-mon[32629]: 3.1e scrub ok 2024-09-16T09:39:30.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:29 smithi084 ceph-mon[32629]: Health check update: Degraded data redundancy: 4686/336828 objects degraded (1.391%), 1 pg degraded (PG_DEGRADED) 2024-09-16T09:39:30.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:29 smithi084 ceph-mon[32629]: osdmap e419: 8 total, 8 up, 8 in 2024-09-16T09:39:30.170 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:29 smithi070 ceph-mon[30388]: 4.11 scrub starts 2024-09-16T09:39:30.170 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:29 smithi070 ceph-mon[30388]: 4.11 scrub ok 2024-09-16T09:39:30.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:29 smithi070 ceph-mon[30388]: osdmap e418: 8 total, 8 up, 8 in 2024-09-16T09:39:30.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:29 smithi070 ceph-mon[30388]: 3.1e scrub starts 2024-09-16T09:39:30.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:29 smithi070 ceph-mon[30388]: 3.1e scrub ok 2024-09-16T09:39:30.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:29 smithi070 ceph-mon[30388]: Health check update: Degraded data redundancy: 4686/336828 objects degraded (1.391%), 1 pg degraded (PG_DEGRADED) 2024-09-16T09:39:30.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:29 smithi070 ceph-mon[30388]: osdmap e419: 8 total, 8 up, 8 in 2024-09-16T09:39:30.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:29 smithi070 ceph-mon[40691]: 4.11 scrub starts 2024-09-16T09:39:30.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:29 smithi070 ceph-mon[40691]: 4.11 scrub ok 2024-09-16T09:39:30.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:29 smithi070 ceph-mon[40691]: osdmap e418: 8 total, 8 up, 8 in 2024-09-16T09:39:30.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:29 smithi070 ceph-mon[40691]: 3.1e scrub starts 2024-09-16T09:39:30.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:29 smithi070 ceph-mon[40691]: 3.1e scrub ok 2024-09-16T09:39:30.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:29 smithi070 ceph-mon[40691]: Health check update: Degraded data redundancy: 4686/336828 objects degraded (1.391%), 1 pg degraded (PG_DEGRADED) 2024-09-16T09:39:30.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:29 smithi070 ceph-mon[40691]: osdmap e419: 8 total, 8 up, 8 in 2024-09-16T09:39:30.211 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:39:30.233 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:30.234 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:30.234 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:39:30.335 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:39:30.366 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:30.367 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:30.367 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:39:30.468 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:39:30.498 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:30.499 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:30.499 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:39:30.600 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:39:30.641 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:30.641 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:30.641 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:39:30.742 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:39:30.767 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:30.767 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:30.768 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:39:30.869 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:39:30.906 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:30.906 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:30.907 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:39:31.007 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:39:31.041 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:31.042 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:31.042 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:39:31.143 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:39:31.180 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:31.182 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:31.182 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:39:31.283 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:39:31.324 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:31.324 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:31.324 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:39:31.425 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:39:31.446 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:31.446 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:31.447 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:39:31.547 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:39:31.582 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:31.583 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:31.584 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:39:31.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:31 smithi084 ceph-mon[32629]: pgmap v768: 178 pgs: 1 active+remapped+backfilling, 1 active+recovery_wait+undersized+degraded+remapped, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 172 active+clean; 6.9 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s; 4600/339306 objects degraded (1.356%); 22283/339306 objects misplaced (6.567%) 2024-09-16T09:39:31.663 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:31 smithi070 ceph-mon[30388]: pgmap v768: 178 pgs: 1 active+remapped+backfilling, 1 active+recovery_wait+undersized+degraded+remapped, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 172 active+clean; 6.9 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s; 4600/339306 objects degraded (1.356%); 22283/339306 objects misplaced (6.567%) 2024-09-16T09:39:31.663 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:31 smithi070 ceph-mon[40691]: pgmap v768: 178 pgs: 1 active+remapped+backfilling, 1 active+recovery_wait+undersized+degraded+remapped, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 172 active+clean; 6.9 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s; 4600/339306 objects degraded (1.356%); 22283/339306 objects misplaced (6.567%) 2024-09-16T09:39:31.684 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:39:31.734 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:31.735 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:31.735 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:39:31.836 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:39:31.877 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:31.877 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:31.878 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:39:31.979 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:39:32.000 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:32.001 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:32.001 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:39:32.102 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:39:32.142 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:32.143 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:32.143 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:39:32.244 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:39:32.269 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:32.271 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:32.271 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:39:32.372 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:39:32.414 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:32.415 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:32.415 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:39:32.420 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:39:31 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:39:31] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:39:32.516 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:39:32.537 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:32.538 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:32.538 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:39:32.639 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:39:32.670 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:32.671 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:32.671 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:39:32.772 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:39:32.804 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:32.805 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:32.805 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:39:32.881 INFO:teuthology.orchestra.run.smithi070.stderr:no valid command found; 10 closest matches: 2024-09-16T09:39:32.881 INFO:teuthology.orchestra.run.smithi070.stderr:0 2024-09-16T09:39:32.881 INFO:teuthology.orchestra.run.smithi070.stderr:1 2024-09-16T09:39:32.881 INFO:teuthology.orchestra.run.smithi070.stderr:2 2024-09-16T09:39:32.882 INFO:teuthology.orchestra.run.smithi070.stderr:abort 2024-09-16T09:39:32.882 INFO:teuthology.orchestra.run.smithi070.stderr:assert 2024-09-16T09:39:32.882 INFO:teuthology.orchestra.run.smithi070.stderr:bench [] [] [] [] 2024-09-16T09:39:32.882 INFO:teuthology.orchestra.run.smithi070.stderr:bluefs debug_inject_read_zeros 2024-09-16T09:39:32.882 INFO:teuthology.orchestra.run.smithi070.stderr:bluefs files list 2024-09-16T09:39:32.882 INFO:teuthology.orchestra.run.smithi070.stderr:bluefs stats 2024-09-16T09:39:32.882 INFO:teuthology.orchestra.run.smithi070.stderr:bluestore allocator dump block 2024-09-16T09:39:32.882 INFO:teuthology.orchestra.run.smithi070.stderr:Error EINVAL: invalid command 2024-09-16T09:39:32.906 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:39:32.939 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:32.939 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:32.940 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:39:33.040 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:39:33.061 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:33.062 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:33.062 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:39:33.163 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:39:33.187 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:33.188 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:33.188 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:39:33.289 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:39:33.314 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:33.315 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:33.315 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:39:33.416 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:39:33.438 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:33.439 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:33.439 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:39:33.540 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:39:33.575 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:33.576 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:33.576 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:39:33.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:33 smithi084 ceph-mon[32629]: pgmap v769: 178 pgs: 1 active+remapped+backfilling, 1 active+recovery_wait+undersized+degraded+remapped, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 172 active+clean; 6.9 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 827 B/s rd, 0 op/s; 4600/339306 objects degraded (1.356%); 22283/339306 objects misplaced (6.567%) 2024-09-16T09:39:33.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:33 smithi084 ceph-mon[32629]: osdmap e420: 8 total, 8 up, 8 in 2024-09-16T09:39:33.661 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:39:33 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:39:33.670 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:33 smithi070 ceph-mon[40691]: pgmap v769: 178 pgs: 1 active+remapped+backfilling, 1 active+recovery_wait+undersized+degraded+remapped, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 172 active+clean; 6.9 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 827 B/s rd, 0 op/s; 4600/339306 objects degraded (1.356%); 22283/339306 objects misplaced (6.567%) 2024-09-16T09:39:33.670 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:33 smithi070 ceph-mon[40691]: osdmap e420: 8 total, 8 up, 8 in 2024-09-16T09:39:33.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:33 smithi070 ceph-mon[30388]: pgmap v769: 178 pgs: 1 active+remapped+backfilling, 1 active+recovery_wait+undersized+degraded+remapped, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 172 active+clean; 6.9 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 827 B/s rd, 0 op/s; 4600/339306 objects degraded (1.356%); 22283/339306 objects misplaced (6.567%) 2024-09-16T09:39:33.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:33 smithi070 ceph-mon[30388]: osdmap e420: 8 total, 8 up, 8 in 2024-09-16T09:39:33.677 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:39:33.716 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:33.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:33.718 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:39:33.818 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:39:33.845 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:33.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:33.846 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:39:33.947 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:39:33.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:33.982 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:33.982 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:39:34.083 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:39:34.126 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:34.127 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:34.127 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:39:34.228 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:39:34.259 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:34.259 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:34.260 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:39:34.361 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:39:34.394 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:34.395 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:34.395 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:39:34.496 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:39:34.545 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:34.545 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:34.546 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:39:34.646 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:39:34.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:34 smithi084 ceph-mon[32629]: osdmap e421: 8 total, 8 up, 8 in 2024-09-16T09:39:34.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:34 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:39:34.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:34 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:39:34.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:34 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:39:34.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:34 smithi084 ceph-mon[32629]: 10.11 scrub starts 2024-09-16T09:39:34.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:34 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:39:34.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:34 smithi084 ceph-mon[32629]: Health check update: Degraded data redundancy: 4600/339306 objects degraded (1.356%), 1 pg degraded (PG_DEGRADED) 2024-09-16T09:39:34.668 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:34 smithi070 ceph-mon[30388]: osdmap e421: 8 total, 8 up, 8 in 2024-09-16T09:39:34.669 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:34 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:39:34.669 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:34 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:39:34.669 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:34 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:39:34.669 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:34 smithi070 ceph-mon[30388]: 10.11 scrub starts 2024-09-16T09:39:34.669 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:34 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:39:34.669 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:34 smithi070 ceph-mon[30388]: Health check update: Degraded data redundancy: 4600/339306 objects degraded (1.356%), 1 pg degraded (PG_DEGRADED) 2024-09-16T09:39:34.669 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:34 smithi070 ceph-mon[40691]: osdmap e421: 8 total, 8 up, 8 in 2024-09-16T09:39:34.669 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:34 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:39:34.669 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:34 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:39:34.669 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:34 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:39:34.670 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:34 smithi070 ceph-mon[40691]: 10.11 scrub starts 2024-09-16T09:39:34.670 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:34 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:39:34.670 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:34 smithi070 ceph-mon[40691]: Health check update: Degraded data redundancy: 4600/339306 objects degraded (1.356%), 1 pg degraded (PG_DEGRADED) 2024-09-16T09:39:34.712 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:34.713 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:34.713 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:39:34.737 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.0 dump_blocked_ops 2024-09-16T09:39:34.814 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:39:34.880 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:34.880 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:34.881 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:39:34.981 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:39:35.017 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:35.017 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:35.018 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:39:35.118 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:39:35.151 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:35.152 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:35.152 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:39:35.253 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:39:35.289 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:35.289 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:35.290 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:39:35.376 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-16T09:39:35.376 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-16T09:39:35.377 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph tell osd.1 'trim stale osdmaps' 2024-09-16T09:39:35.391 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:39:35.397 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:39:35.432 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:39:35.514 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:35.515 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:35.515 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:39:35.558 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:35 smithi070 ceph-mon[30388]: pgmap v772: 178 pgs: 1 peering, 1 active+recovery_wait, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 172 active+clean; 6.7 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1.0 KiB/s rd, 3.3 MiB/s wr, 910 op/s; 24643/327894 objects misplaced (7.516%); 27 MiB/s, 431 objects/s recovering 2024-09-16T09:39:35.558 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:35 smithi070 ceph-mon[30388]: 1.9 scrub starts 2024-09-16T09:39:35.558 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:35 smithi070 ceph-mon[30388]: 1.9 scrub ok 2024-09-16T09:39:35.558 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:35 smithi070 ceph-mon[30388]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 4600/339306 objects degraded (1.356%), 1 pg degraded) 2024-09-16T09:39:35.558 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:35 smithi070 ceph-mon[30388]: 10.11 scrub ok 2024-09-16T09:39:35.558 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:35 smithi070 ceph-mon[40691]: pgmap v772: 178 pgs: 1 peering, 1 active+recovery_wait, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 172 active+clean; 6.7 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1.0 KiB/s rd, 3.3 MiB/s wr, 910 op/s; 24643/327894 objects misplaced (7.516%); 27 MiB/s, 431 objects/s recovering 2024-09-16T09:39:35.559 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:35 smithi070 ceph-mon[40691]: 1.9 scrub starts 2024-09-16T09:39:35.559 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:35 smithi070 ceph-mon[40691]: 1.9 scrub ok 2024-09-16T09:39:35.559 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:35 smithi070 ceph-mon[40691]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 4600/339306 objects degraded (1.356%), 1 pg degraded) 2024-09-16T09:39:35.559 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:35 smithi070 ceph-mon[40691]: 10.11 scrub ok 2024-09-16T09:39:35.616 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:39:35.669 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:35.669 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:35.670 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:39:35.771 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:39:35.797 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:35.798 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:35.798 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:39:35.899 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:39:35.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:35 smithi084 ceph-mon[32629]: pgmap v772: 178 pgs: 1 peering, 1 active+recovery_wait, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 172 active+clean; 6.7 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1.0 KiB/s rd, 3.3 MiB/s wr, 910 op/s; 24643/327894 objects misplaced (7.516%); 27 MiB/s, 431 objects/s recovering 2024-09-16T09:39:35.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:35 smithi084 ceph-mon[32629]: 1.9 scrub starts 2024-09-16T09:39:35.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:35 smithi084 ceph-mon[32629]: 1.9 scrub ok 2024-09-16T09:39:35.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:35 smithi084 ceph-mon[32629]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 4600/339306 objects degraded (1.356%), 1 pg degraded) 2024-09-16T09:39:35.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:35 smithi084 ceph-mon[32629]: 10.11 scrub ok 2024-09-16T09:39:35.932 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:35.932 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:35.932 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:39:36.033 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:39:36.064 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:36.065 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:36.065 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:39:36.167 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:39:36.197 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:36.198 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:36.198 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:39:36.299 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:39:36.325 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:36.325 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:36.326 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:39:36.426 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:39:36.472 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:36.472 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:36.473 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:39:36.499 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:36 smithi070 ceph-mon[30388]: 4.1e scrub starts 2024-09-16T09:39:36.499 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:36 smithi070 ceph-mon[30388]: 4.1e scrub ok 2024-09-16T09:39:36.500 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:36 smithi070 ceph-mon[40691]: 4.1e scrub starts 2024-09-16T09:39:36.500 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:36 smithi070 ceph-mon[40691]: 4.1e scrub ok 2024-09-16T09:39:36.573 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:39:36.611 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:36.612 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:36.612 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:39:36.698 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:39:36.698 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:39:36.713 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:39:36.733 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:39:36.733 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:39:36.753 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:36.754 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:36.754 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:39:36.768 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:39:36.768 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:39:36.806 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:39:36.807 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:39:36.842 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:39:36.842 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:39:36.855 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:39:36.869 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:39:36.879 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:39:36.879 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:39:36.883 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:36.883 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:36.884 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:39:36.907 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:39:36.907 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:39:36.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:36 smithi084 ceph-mon[32629]: 4.1e scrub starts 2024-09-16T09:39:36.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:36 smithi084 ceph-mon[32629]: 4.1e scrub ok 2024-09-16T09:39:36.941 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:39:36.941 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:39:36.985 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:39:37.015 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:37.016 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:37.016 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:39:37.117 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:39:37.164 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:37.164 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:37.165 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:39:37.266 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:39:37.307 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:37.307 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:37.308 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:39:37.409 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:39:37.439 INFO:teuthology.orchestra.run.smithi070.stderr:nodeep-scrub is unset 2024-09-16T09:39:37.451 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:37.452 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:37.452 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:39:37.553 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:39:37.574 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:37.575 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:37.575 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:39:37.676 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:39:37.720 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:37.721 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:37.721 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:39:37.822 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:39:37.855 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:37.856 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:37.856 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:39:37.892 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:37 smithi070 ceph-mon[30388]: pgmap v773: 178 pgs: 1 peering, 1 active+recovery_wait, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 172 active+clean; 6.6 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 796 B/s rd, 2.5 MiB/s wr, 782 op/s; 24256/326250 objects misplaced (7.435%); 21 MiB/s, 330 objects/s recovering 2024-09-16T09:39:37.892 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:37 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3635067387' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:39:37.892 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:37 smithi070 ceph-mon[30388]: 6.0 scrub starts 2024-09-16T09:39:37.893 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:37 smithi070 ceph-mon[30388]: 6.0 scrub ok 2024-09-16T09:39:37.893 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:37 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3924004870' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:39:37.893 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:37 smithi070 ceph-mon[40691]: pgmap v773: 178 pgs: 1 peering, 1 active+recovery_wait, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 172 active+clean; 6.6 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 796 B/s rd, 2.5 MiB/s wr, 782 op/s; 24256/326250 objects misplaced (7.435%); 21 MiB/s, 330 objects/s recovering 2024-09-16T09:39:37.893 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:37 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3635067387' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:39:37.893 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:37 smithi070 ceph-mon[40691]: 6.0 scrub starts 2024-09-16T09:39:37.893 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:37 smithi070 ceph-mon[40691]: 6.0 scrub ok 2024-09-16T09:39:37.893 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:37 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3924004870' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:39:37.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:37 smithi084 ceph-mon[32629]: pgmap v773: 178 pgs: 1 peering, 1 active+recovery_wait, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 172 active+clean; 6.6 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 796 B/s rd, 2.5 MiB/s wr, 782 op/s; 24256/326250 objects misplaced (7.435%); 21 MiB/s, 330 objects/s recovering 2024-09-16T09:39:37.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:37 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3635067387' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:39:37.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:37 smithi084 ceph-mon[32629]: 6.0 scrub starts 2024-09-16T09:39:37.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:37 smithi084 ceph-mon[32629]: 6.0 scrub ok 2024-09-16T09:39:37.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:37 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3924004870' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:39:37.957 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:39:37.998 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:37.999 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:38.000 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:39:38.100 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:39:38.122 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:38.123 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:38.124 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:39:38.224 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:39:38.261 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:38.262 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:38.262 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:39:38.363 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:39:38.404 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:38.405 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:38.406 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:39:38.433 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:38 smithi070 ceph-mon[40691]: 3.13 scrub starts 2024-09-16T09:39:38.433 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:38 smithi070 ceph-mon[40691]: 3.13 scrub ok 2024-09-16T09:39:38.433 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:38 smithi070 ceph-mon[40691]: 10.17 scrub starts 2024-09-16T09:39:38.433 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:38 smithi070 ceph-mon[40691]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-16T09:39:38.433 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:38 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3635067387' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-16T09:39:38.433 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:38 smithi070 ceph-mon[40691]: osdmap e422: 8 total, 8 up, 8 in 2024-09-16T09:39:38.433 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:38 smithi070 ceph-mon[30388]: 3.13 scrub starts 2024-09-16T09:39:38.434 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:38 smithi070 ceph-mon[30388]: 3.13 scrub ok 2024-09-16T09:39:38.434 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:38 smithi070 ceph-mon[30388]: 10.17 scrub starts 2024-09-16T09:39:38.434 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:38 smithi070 ceph-mon[30388]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-16T09:39:38.434 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:38 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3635067387' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-16T09:39:38.434 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:38 smithi070 ceph-mon[30388]: osdmap e422: 8 total, 8 up, 8 in 2024-09-16T09:39:38.434 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:38 smithi070 ceph-mon[30388]: 3.1f scrub starts 2024-09-16T09:39:38.434 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:38 smithi070 ceph-mon[30388]: 3.1f scrub ok 2024-09-16T09:39:38.506 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:39:38.541 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:38.541 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:38.542 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:39:38.643 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:39:38.677 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:38.678 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:38.678 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:39:38.779 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:39:38.816 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:38.816 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:38.817 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:39:38.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:38 smithi084 ceph-mon[32629]: 3.13 scrub starts 2024-09-16T09:39:38.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:38 smithi084 ceph-mon[32629]: 3.13 scrub ok 2024-09-16T09:39:38.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:38 smithi084 ceph-mon[32629]: 10.17 scrub starts 2024-09-16T09:39:38.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:38 smithi084 ceph-mon[32629]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-16T09:39:38.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:38 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3635067387' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-16T09:39:38.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:38 smithi084 ceph-mon[32629]: osdmap e422: 8 total, 8 up, 8 in 2024-09-16T09:39:38.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:38 smithi084 ceph-mon[32629]: 3.1f scrub starts 2024-09-16T09:39:38.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:38 smithi084 ceph-mon[32629]: 3.1f scrub ok 2024-09-16T09:39:38.917 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:39:38.920 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:38 smithi070 ceph-mon[40691]: 3.1f scrub starts 2024-09-16T09:39:38.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:38 smithi070 ceph-mon[40691]: 3.1f scrub ok 2024-09-16T09:39:38.953 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:38.953 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:38.954 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:39:39.054 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:39:39.076 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:39.077 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:39.077 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:39:39.178 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:39:39.213 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:39.214 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:39.214 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:39:39.315 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:39:39.349 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:39.350 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:39.350 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:39:39.451 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:39:39.473 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:39.473 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:39.474 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:39:39.574 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:39:39.609 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:39.610 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:39.610 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:39:39.711 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:39:39.732 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:39.732 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:39.733 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:39:39.833 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:39:39.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:39.867 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:39.868 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:39:39.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:39 smithi084 ceph-mon[32629]: pgmap v775: 178 pgs: 1 peering, 1 active+recovery_wait, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 172 active+clean; 6.6 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 852 B/s rd, 512 KiB/s wr, 741 op/s; 24256/326250 objects misplaced (7.435%); 13 MiB/s, 212 objects/s recovering 2024-09-16T09:39:39.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:39 smithi084 ceph-mon[32629]: 5.9 scrub starts 2024-09-16T09:39:39.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:39 smithi084 ceph-mon[32629]: 5.9 scrub ok 2024-09-16T09:39:39.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:39 smithi084 ceph-mon[32629]: 10.17 scrub ok 2024-09-16T09:39:39.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:39 smithi084 ceph-mon[32629]: 3.b scrub starts 2024-09-16T09:39:39.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:39 smithi084 ceph-mon[32629]: 3.b scrub ok 2024-09-16T09:39:39.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:39 smithi084 ceph-mon[32629]: osdmap e423: 8 total, 8 up, 8 in 2024-09-16T09:39:39.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:39 smithi070 ceph-mon[30388]: pgmap v775: 178 pgs: 1 peering, 1 active+recovery_wait, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 172 active+clean; 6.6 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 852 B/s rd, 512 KiB/s wr, 741 op/s; 24256/326250 objects misplaced (7.435%); 13 MiB/s, 212 objects/s recovering 2024-09-16T09:39:39.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:39 smithi070 ceph-mon[30388]: 5.9 scrub starts 2024-09-16T09:39:39.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:39 smithi070 ceph-mon[30388]: 5.9 scrub ok 2024-09-16T09:39:39.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:39 smithi070 ceph-mon[30388]: 10.17 scrub ok 2024-09-16T09:39:39.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:39 smithi070 ceph-mon[30388]: 3.b scrub starts 2024-09-16T09:39:39.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:39 smithi070 ceph-mon[30388]: 3.b scrub ok 2024-09-16T09:39:39.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:39 smithi070 ceph-mon[30388]: osdmap e423: 8 total, 8 up, 8 in 2024-09-16T09:39:39.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:39 smithi070 ceph-mon[40691]: pgmap v775: 178 pgs: 1 peering, 1 active+recovery_wait, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 172 active+clean; 6.6 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 852 B/s rd, 512 KiB/s wr, 741 op/s; 24256/326250 objects misplaced (7.435%); 13 MiB/s, 212 objects/s recovering 2024-09-16T09:39:39.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:39 smithi070 ceph-mon[40691]: 5.9 scrub starts 2024-09-16T09:39:39.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:39 smithi070 ceph-mon[40691]: 5.9 scrub ok 2024-09-16T09:39:39.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:39 smithi070 ceph-mon[40691]: 10.17 scrub ok 2024-09-16T09:39:39.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:39 smithi070 ceph-mon[40691]: 3.b scrub starts 2024-09-16T09:39:39.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:39 smithi070 ceph-mon[40691]: 3.b scrub ok 2024-09-16T09:39:39.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:39 smithi070 ceph-mon[40691]: osdmap e423: 8 total, 8 up, 8 in 2024-09-16T09:39:39.968 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:39:40.009 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:40.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:40.010 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:39:40.111 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:39:40.152 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:40.153 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:40.153 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:39:40.254 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:39:40.282 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:40.282 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:40.283 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:39:40.383 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:39:40.418 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:40.419 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:40.420 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:39:40.520 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:39:40.552 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:40.553 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:40.553 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:39:40.653 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:39:40.690 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:40.691 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:40.692 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:39:40.792 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:39:40.825 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:40.825 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:40.826 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:39:40.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:40 smithi084 ceph-mon[32629]: 5.5 scrub starts 2024-09-16T09:39:40.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:40 smithi084 ceph-mon[32629]: 5.5 scrub ok 2024-09-16T09:39:40.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:40 smithi070 ceph-mon[30388]: 5.5 scrub starts 2024-09-16T09:39:40.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:40 smithi070 ceph-mon[30388]: 5.5 scrub ok 2024-09-16T09:39:40.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:40 smithi070 ceph-mon[40691]: 5.5 scrub starts 2024-09-16T09:39:40.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:40 smithi070 ceph-mon[40691]: 5.5 scrub ok 2024-09-16T09:39:40.926 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:39:40.947 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:40.948 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:40.948 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:39:41.049 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:39:41.073 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:41.073 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:41.074 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:39:41.174 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:39:41.209 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:41.210 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:41.210 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:39:41.311 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:39:41.343 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:41.344 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:41.344 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:39:41.445 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:39:41.481 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:41.481 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:41.484 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:39:41.585 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:39:41.612 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:41.613 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:41.613 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:39:41.714 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:39:41.715 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:41 smithi070 ceph-mon[30388]: pgmap v777: 178 pgs: 1 active+clean+scrubbing, 1 active+recovery_wait, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 172 active+clean; 6.3 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 799 B/s rd, 0 B/s wr, 332 op/s; 22544/310800 objects misplaced (7.254%) 2024-09-16T09:39:41.715 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:41 smithi070 ceph-mon[30388]: 1.0 scrub starts 2024-09-16T09:39:41.715 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:41 smithi070 ceph-mon[30388]: 1.0 scrub ok 2024-09-16T09:39:41.715 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:41 smithi070 ceph-mon[30388]: Health check failed: Degraded data redundancy: 2 pgs undersized (PG_DEGRADED) 2024-09-16T09:39:41.715 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:41 smithi070 ceph-mon[40691]: pgmap v777: 178 pgs: 1 active+clean+scrubbing, 1 active+recovery_wait, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 172 active+clean; 6.3 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 799 B/s rd, 0 B/s wr, 332 op/s; 22544/310800 objects misplaced (7.254%) 2024-09-16T09:39:41.715 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:41 smithi070 ceph-mon[40691]: 1.0 scrub starts 2024-09-16T09:39:41.716 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:41 smithi070 ceph-mon[40691]: 1.0 scrub ok 2024-09-16T09:39:41.716 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:41 smithi070 ceph-mon[40691]: Health check failed: Degraded data redundancy: 2 pgs undersized (PG_DEGRADED) 2024-09-16T09:39:41.752 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:41.753 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:41.753 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:39:41.854 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:39:41.894 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:41.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:41.896 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:39:41.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:41 smithi084 ceph-mon[32629]: pgmap v777: 178 pgs: 1 active+clean+scrubbing, 1 active+recovery_wait, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 172 active+clean; 6.3 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 799 B/s rd, 0 B/s wr, 332 op/s; 22544/310800 objects misplaced (7.254%) 2024-09-16T09:39:41.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:41 smithi084 ceph-mon[32629]: 1.0 scrub starts 2024-09-16T09:39:41.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:41 smithi084 ceph-mon[32629]: 1.0 scrub ok 2024-09-16T09:39:41.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:41 smithi084 ceph-mon[32629]: Health check failed: Degraded data redundancy: 2 pgs undersized (PG_DEGRADED) 2024-09-16T09:39:41.997 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:39:42.036 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:42.036 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:42.037 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:39:42.137 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:39:42.175 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:42.176 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:42.176 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:39:42.277 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:39:42.311 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:42.311 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:42.311 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:39:42.412 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:39:42.420 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:39:41 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:39:41] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:39:42.446 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:42.447 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:42.447 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:39:42.548 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:39:42.582 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:42.582 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:42.583 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:39:42.684 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:39:42.725 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:42.726 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:42.726 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:39:42.827 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:39:42.865 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:42.865 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:42.866 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:39:42.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:42 smithi084 ceph-mon[32629]: 3.1 scrub starts 2024-09-16T09:39:42.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:42 smithi084 ceph-mon[32629]: 3.1 scrub ok 2024-09-16T09:39:42.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:42 smithi070 ceph-mon[30388]: 3.1 scrub starts 2024-09-16T09:39:42.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:42 smithi070 ceph-mon[30388]: 3.1 scrub ok 2024-09-16T09:39:42.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:42 smithi070 ceph-mon[40691]: 3.1 scrub starts 2024-09-16T09:39:42.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:42 smithi070 ceph-mon[40691]: 3.1 scrub ok 2024-09-16T09:39:42.966 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:39:43.004 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:43.005 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:43.005 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:39:43.106 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:39:43.140 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:43.141 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:43.141 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:39:43.242 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:39:43.278 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:43.278 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:43.279 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:39:43.380 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:39:43.415 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:43.416 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:43.416 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:39:43.517 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:39:43.549 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:43.550 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:43.550 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:39:43.651 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:39:43.689 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:43.690 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:43.690 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:39:43.791 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:39:43.821 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:43.822 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:43.822 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:39:43.910 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:39:43 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:39:43.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:43 smithi084 ceph-mon[32629]: pgmap v778: 178 pgs: 1 active+clean+scrubbing, 1 active+recovery_wait, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 172 active+clean; 6.3 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 249 op/s; 22544/310800 objects misplaced (7.254%) 2024-09-16T09:39:43.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:43 smithi084 ceph-mon[32629]: 3.17 scrub starts 2024-09-16T09:39:43.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:43 smithi084 ceph-mon[32629]: 3.17 scrub ok 2024-09-16T09:39:43.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:43 smithi070 ceph-mon[30388]: pgmap v778: 178 pgs: 1 active+clean+scrubbing, 1 active+recovery_wait, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 172 active+clean; 6.3 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 249 op/s; 22544/310800 objects misplaced (7.254%) 2024-09-16T09:39:43.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:43 smithi070 ceph-mon[30388]: 3.17 scrub starts 2024-09-16T09:39:43.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:43 smithi070 ceph-mon[30388]: 3.17 scrub ok 2024-09-16T09:39:43.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:43 smithi070 ceph-mon[40691]: pgmap v778: 178 pgs: 1 active+clean+scrubbing, 1 active+recovery_wait, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 172 active+clean; 6.3 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 249 op/s; 22544/310800 objects misplaced (7.254%) 2024-09-16T09:39:43.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:43 smithi070 ceph-mon[40691]: 3.17 scrub starts 2024-09-16T09:39:43.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:43 smithi070 ceph-mon[40691]: 3.17 scrub ok 2024-09-16T09:39:43.923 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:39:43.958 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:43.959 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:43.959 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:39:44.060 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:39:44.089 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:44.089 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:44.090 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:39:44.191 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:39:44.224 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:44.225 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:44.225 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:39:44.326 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:39:44.366 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:44.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:44.367 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:39:44.468 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:39:44.506 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:44.506 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:44.507 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:39:44.608 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:39:44.643 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:44.644 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:44.645 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:39:44.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:44 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:39:44.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:44 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:39:44.745 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:39:44.787 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:44.787 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:44.788 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:39:44.889 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:39:44.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:44 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:39:44.944 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:44.944 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:44.945 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:39:45.046 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:39:45.075 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:45.076 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:45.076 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:39:45.141 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T09:39:45.141 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:39:45.177 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:39:45.226 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:45.227 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:45.227 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:39:45.328 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:39:45.364 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:45.365 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:45.365 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:39:45.371 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:39:45.466 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:39:45.512 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:45.512 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:45.513 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:39:45.614 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:39:45.650 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:45.650 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:45.651 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:39:45.751 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:39:45.789 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:45.789 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:45.789 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:39:45.890 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:39:45.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:45 smithi084 ceph-mon[32629]: pgmap v779: 178 pgs: 1 active+recovery_wait, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 173 active+clean; 6.0 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 259 op/s; 21373/297150 objects misplaced (7.193%) 2024-09-16T09:39:45.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:45 smithi084 ceph-mon[32629]: 3.15 scrub starts 2024-09-16T09:39:45.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:45 smithi084 ceph-mon[32629]: 3.15 scrub ok 2024-09-16T09:39:45.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:45 smithi070 ceph-mon[30388]: pgmap v779: 178 pgs: 1 active+recovery_wait, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 173 active+clean; 6.0 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 259 op/s; 21373/297150 objects misplaced (7.193%) 2024-09-16T09:39:45.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:45 smithi070 ceph-mon[30388]: 3.15 scrub starts 2024-09-16T09:39:45.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:45 smithi070 ceph-mon[30388]: 3.15 scrub ok 2024-09-16T09:39:45.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:45 smithi070 ceph-mon[40691]: pgmap v779: 178 pgs: 1 active+recovery_wait, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 173 active+clean; 6.0 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 259 op/s; 21373/297150 objects misplaced (7.193%) 2024-09-16T09:39:45.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:45 smithi070 ceph-mon[40691]: 3.15 scrub starts 2024-09-16T09:39:45.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:45 smithi070 ceph-mon[40691]: 3.15 scrub ok 2024-09-16T09:39:45.931 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:45.931 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:45.932 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:39:46.033 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:39:46.067 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:46.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:46.068 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:39:46.169 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:39:46.205 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:46.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:46.206 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:39:46.307 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:39:46.328 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:46.329 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:46.329 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:39:46.430 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:39:46.452 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:46.453 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:46.453 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:39:46.554 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:39:46.575 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:46.576 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:46.576 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:39:46.677 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:39:46.713 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:46.714 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:46.714 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:39:46.815 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:39:46.830 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T09:39:46.840 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:46.841 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:46.841 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:39:46.942 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:39:46.967 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:46.967 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:46.968 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:39:47.069 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:39:47.106 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:47.107 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:47.107 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:39:47.207 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:39:47.242 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:47.242 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:47.243 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:39:47.344 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:39:47.373 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:47.374 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:47.374 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:39:47.475 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:39:47.514 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:47.515 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:47.515 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:39:47.616 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:39:47.650 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:47.651 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:47.651 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:39:47.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:47 smithi070 ceph-mon[30388]: pgmap v780: 178 pgs: 1 active+recovery_wait, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 173 active+clean; 6.0 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 331 op/s; 20845/295074 objects misplaced (7.064%) 2024-09-16T09:39:47.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:47 smithi070 ceph-mon[30388]: 4.16 scrub starts 2024-09-16T09:39:47.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:47 smithi070 ceph-mon[30388]: 4.16 scrub ok 2024-09-16T09:39:47.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:47 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:39:47.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:47 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:39:47.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:47 smithi070 ceph-mon[40691]: pgmap v780: 178 pgs: 1 active+recovery_wait, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 173 active+clean; 6.0 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 331 op/s; 20845/295074 objects misplaced (7.064%) 2024-09-16T09:39:47.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:47 smithi070 ceph-mon[40691]: 4.16 scrub starts 2024-09-16T09:39:47.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:47 smithi070 ceph-mon[40691]: 4.16 scrub ok 2024-09-16T09:39:47.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:47 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:39:47.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:47 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:39:47.752 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:39:47.803 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:47.803 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:47.803 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:39:47.833 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.0 dump_historic_ops 2024-09-16T09:39:47.904 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:39:47.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:47 smithi084 ceph-mon[32629]: pgmap v780: 178 pgs: 1 active+recovery_wait, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 173 active+clean; 6.0 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 331 op/s; 20845/295074 objects misplaced (7.064%) 2024-09-16T09:39:47.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:47 smithi084 ceph-mon[32629]: 4.16 scrub starts 2024-09-16T09:39:47.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:47 smithi084 ceph-mon[32629]: 4.16 scrub ok 2024-09-16T09:39:47.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:47 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:39:47.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:47 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:39:47.960 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:47.961 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:47.961 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:39:48.062 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:39:48.128 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:48.129 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:48.130 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:39:48.231 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:39:48.281 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:48.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:48.282 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:39:48.383 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:39:48.437 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:48.438 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:48.438 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:39:48.445 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:39:48.531 INFO:teuthology.orchestra.run.smithi070.stderr:no valid command found; 10 closest matches: 2024-09-16T09:39:48.531 INFO:teuthology.orchestra.run.smithi070.stderr:0 2024-09-16T09:39:48.531 INFO:teuthology.orchestra.run.smithi070.stderr:1 2024-09-16T09:39:48.531 INFO:teuthology.orchestra.run.smithi070.stderr:2 2024-09-16T09:39:48.531 INFO:teuthology.orchestra.run.smithi070.stderr:abort 2024-09-16T09:39:48.531 INFO:teuthology.orchestra.run.smithi070.stderr:assert 2024-09-16T09:39:48.531 INFO:teuthology.orchestra.run.smithi070.stderr:bench [] [] [] [] 2024-09-16T09:39:48.531 INFO:teuthology.orchestra.run.smithi070.stderr:bluefs debug_inject_read_zeros 2024-09-16T09:39:48.531 INFO:teuthology.orchestra.run.smithi070.stderr:bluefs files list 2024-09-16T09:39:48.531 INFO:teuthology.orchestra.run.smithi070.stderr:bluefs stats 2024-09-16T09:39:48.531 INFO:teuthology.orchestra.run.smithi070.stderr:bluestore allocator dump block 2024-09-16T09:39:48.532 INFO:teuthology.orchestra.run.smithi070.stderr:Error EINVAL: invalid command 2024-09-16T09:39:48.539 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:39:48.606 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:48.607 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:48.607 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:39:48.708 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:39:48.748 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:48.748 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:48.748 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:39:48.788 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:48 smithi070 ceph-mon[40691]: osdmap e424: 8 total, 8 up, 8 in 2024-09-16T09:39:48.788 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:48 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:39:48.788 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:48 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:39:48.790 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:48 smithi070 ceph-mon[30388]: osdmap e424: 8 total, 8 up, 8 in 2024-09-16T09:39:48.790 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:48 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:39:48.790 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:48 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:39:48.852 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:39:48.856 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd set noscrub 2024-09-16T09:39:48.881 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:48.882 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:48.882 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:39:48.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:48 smithi084 ceph-mon[32629]: osdmap e424: 8 total, 8 up, 8 in 2024-09-16T09:39:48.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:48 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:39:48.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:48 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:39:48.983 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:39:49.039 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:49.039 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:49.040 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:39:49.141 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:39:49.192 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:39:49.192 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:39:49.211 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:49.212 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:49.212 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:39:49.234 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:39:49.234 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:39:49.268 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:39:49.268 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:39:49.303 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:39:49.303 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:39:49.313 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:39:49.339 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:39:49.339 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:39:49.341 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:49.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:49.342 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:39:49.372 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:39:49.373 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:39:49.412 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:39:49.412 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:39:49.443 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:39:49.452 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:39:49.453 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:39:49.491 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:49.492 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:49.492 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:39:49.593 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:39:49.629 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:49.630 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:49.630 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:39:49.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:49 smithi070 ceph-mon[30388]: pgmap v782: 178 pgs: 1 active+recovery_wait, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 173 active+clean; 6.0 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 607 B/s rd, 0 B/s wr, 238 op/s; 20845/295074 objects misplaced (7.064%) 2024-09-16T09:39:49.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:49 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/869580392' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:39:49.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:49 smithi070 ceph-mon[30388]: Health check update: Degraded data redundancy: 3 pgs undersized (PG_DEGRADED) 2024-09-16T09:39:49.673 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:49 smithi070 ceph-mon[30388]: osdmap e425: 8 total, 8 up, 8 in 2024-09-16T09:39:49.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:49 smithi070 ceph-mon[40691]: pgmap v782: 178 pgs: 1 active+recovery_wait, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 173 active+clean; 6.0 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 607 B/s rd, 0 B/s wr, 238 op/s; 20845/295074 objects misplaced (7.064%) 2024-09-16T09:39:49.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:49 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/869580392' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:39:49.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:49 smithi070 ceph-mon[40691]: Health check update: Degraded data redundancy: 3 pgs undersized (PG_DEGRADED) 2024-09-16T09:39:49.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:49 smithi070 ceph-mon[40691]: osdmap e425: 8 total, 8 up, 8 in 2024-09-16T09:39:49.731 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:39:49.765 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:49.765 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:49.766 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:39:49.866 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:39:49.906 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:49.907 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:49.907 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:39:49.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:49 smithi084 ceph-mon[32629]: pgmap v782: 178 pgs: 1 active+recovery_wait, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 173 active+clean; 6.0 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 607 B/s rd, 0 B/s wr, 238 op/s; 20845/295074 objects misplaced (7.064%) 2024-09-16T09:39:49.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:49 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/869580392' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:39:49.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:49 smithi084 ceph-mon[32629]: Health check update: Degraded data redundancy: 3 pgs undersized (PG_DEGRADED) 2024-09-16T09:39:49.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:49 smithi084 ceph-mon[32629]: osdmap e425: 8 total, 8 up, 8 in 2024-09-16T09:39:50.008 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:39:50.040 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:50.040 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:50.040 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:39:50.141 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:39:50.156 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-16T09:39:50.156 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-16T09:39:50.157 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph tell osd.2 'trim stale osdmaps' 2024-09-16T09:39:50.169 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:50.169 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:50.169 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:39:50.270 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:39:50.297 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:50.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:50.298 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:39:50.398 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:39:50.447 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:50.448 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:50.448 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:39:50.549 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:39:50.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:50.595 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:50.595 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:39:50.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:50 smithi070 ceph-mon[30388]: 1.d scrub starts 2024-09-16T09:39:50.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:50 smithi070 ceph-mon[30388]: 1.d scrub ok 2024-09-16T09:39:50.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:50 smithi070 ceph-mon[40691]: 1.d scrub starts 2024-09-16T09:39:50.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:50 smithi070 ceph-mon[40691]: 1.d scrub ok 2024-09-16T09:39:50.696 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:39:50.733 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:50.734 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:50.735 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:39:50.836 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:39:50.884 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:50.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:50.886 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:39:50.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:50 smithi084 ceph-mon[32629]: 1.d scrub starts 2024-09-16T09:39:50.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:50 smithi084 ceph-mon[32629]: 1.d scrub ok 2024-09-16T09:39:50.987 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:39:51.024 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:51.025 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:51.025 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:39:51.036 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:39:51.126 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:39:51.170 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:51.172 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:51.172 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:39:51.273 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:39:51.312 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:51.313 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:51.313 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:39:51.414 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:39:51.441 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:51.442 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:51.442 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:39:51.543 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:39:51.569 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:51.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:51.569 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:39:51.579 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:39:51.649 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:39:51.670 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:39:51.704 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:51.705 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:51.705 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:39:51.806 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:39:51.834 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:51 smithi070 ceph-mon[30388]: pgmap v784: 178 pgs: 1 active+recovery_wait, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 173 active+clean; 5.6 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 867 op/s; 19058/278220 objects misplaced (6.850%) 2024-09-16T09:39:51.834 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:51 smithi070 ceph-mon[30388]: 3.1d scrub starts 2024-09-16T09:39:51.834 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:51 smithi070 ceph-mon[30388]: 3.1d scrub ok 2024-09-16T09:39:51.835 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:51 smithi070 ceph-mon[40691]: pgmap v784: 178 pgs: 1 active+recovery_wait, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 173 active+clean; 5.6 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 867 op/s; 19058/278220 objects misplaced (6.850%) 2024-09-16T09:39:51.835 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:51 smithi070 ceph-mon[40691]: 3.1d scrub starts 2024-09-16T09:39:51.835 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:51 smithi070 ceph-mon[40691]: 3.1d scrub ok 2024-09-16T09:39:51.837 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:51.837 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:51.838 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:39:51.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:51 smithi084 ceph-mon[32629]: pgmap v784: 178 pgs: 1 active+recovery_wait, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 173 active+clean; 5.6 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 867 op/s; 19058/278220 objects misplaced (6.850%) 2024-09-16T09:39:51.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:51 smithi084 ceph-mon[32629]: 3.1d scrub starts 2024-09-16T09:39:51.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:51 smithi084 ceph-mon[32629]: 3.1d scrub ok 2024-09-16T09:39:51.939 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:39:51.980 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:51.980 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:51.981 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:39:52.082 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:39:52.115 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:52.116 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:52.116 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:39:52.171 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:39:51 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:39:51] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:39:52.217 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:39:52.260 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:52.260 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:52.261 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:39:52.362 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:39:52.394 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:52.394 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:52.395 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:39:52.496 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:39:52.538 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:52.539 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:52.539 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:39:52.640 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:39:52.680 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:52.681 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:52.681 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:39:52.782 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:39:52.816 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:52.818 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:52.818 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:39:52.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:52 smithi084 ceph-mon[32629]: 3.11 scrub starts 2024-09-16T09:39:52.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:52 smithi084 ceph-mon[32629]: 3.11 scrub ok 2024-09-16T09:39:52.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:52 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:39:52.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:52 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:39:52.916 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T09:39:52.919 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:39:52.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:52 smithi070 ceph-mon[30388]: 3.11 scrub starts 2024-09-16T09:39:52.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:52 smithi070 ceph-mon[30388]: 3.11 scrub ok 2024-09-16T09:39:52.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:52 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:39:52.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:52 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:39:52.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:52 smithi070 ceph-mon[40691]: 3.11 scrub starts 2024-09-16T09:39:52.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:52 smithi070 ceph-mon[40691]: 3.11 scrub ok 2024-09-16T09:39:52.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:52 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:39:52.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:52 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:39:52.952 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:52.953 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:52.953 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:39:53.054 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:39:53.083 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:53.084 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:53.084 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:39:53.185 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:39:53.256 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:53.256 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:53.257 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:39:53.357 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:39:53.382 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:53.382 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:53.383 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:39:53.483 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:39:53.505 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:53.506 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:53.506 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:39:53.607 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:39:53.645 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:53.646 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:53.647 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:39:53.747 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:39:53.780 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:53.781 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:53.781 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:39:53.811 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:39:53 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:39:53.812 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:53 smithi084 ceph-mon[32629]: pgmap v785: 178 pgs: 1 active+recovery_wait, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 173 active+clean; 5.6 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 788 op/s; 19058/278220 objects misplaced (6.850%); 4.5 MiB/s, 72 objects/s recovering 2024-09-16T09:39:53.812 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:53 smithi084 ceph-mon[32629]: 4.0 scrub starts 2024-09-16T09:39:53.812 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:53 smithi084 ceph-mon[32629]: 4.0 scrub ok 2024-09-16T09:39:53.812 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:53 smithi084 ceph-mon[32629]: osdmap e426: 8 total, 8 up, 8 in 2024-09-16T09:39:53.812 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:53 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:39:53.882 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:39:53.917 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:53.917 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:53.918 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:39:53.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:53 smithi070 ceph-mon[30388]: pgmap v785: 178 pgs: 1 active+recovery_wait, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 173 active+clean; 5.6 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 788 op/s; 19058/278220 objects misplaced (6.850%); 4.5 MiB/s, 72 objects/s recovering 2024-09-16T09:39:53.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:53 smithi070 ceph-mon[30388]: 4.0 scrub starts 2024-09-16T09:39:53.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:53 smithi070 ceph-mon[30388]: 4.0 scrub ok 2024-09-16T09:39:53.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:53 smithi070 ceph-mon[30388]: osdmap e426: 8 total, 8 up, 8 in 2024-09-16T09:39:53.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:53 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:39:53.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:53 smithi070 ceph-mon[40691]: pgmap v785: 178 pgs: 1 active+recovery_wait, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 173 active+clean; 5.6 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 788 op/s; 19058/278220 objects misplaced (6.850%); 4.5 MiB/s, 72 objects/s recovering 2024-09-16T09:39:53.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:53 smithi070 ceph-mon[40691]: 4.0 scrub starts 2024-09-16T09:39:53.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:53 smithi070 ceph-mon[40691]: 4.0 scrub ok 2024-09-16T09:39:53.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:53 smithi070 ceph-mon[40691]: osdmap e426: 8 total, 8 up, 8 in 2024-09-16T09:39:53.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:53 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:39:54.019 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:39:54.062 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:54.063 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:54.064 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:39:54.164 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:39:54.200 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:54.201 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:54.201 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:39:54.302 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:39:54.350 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:54.350 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:54.351 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:39:54.451 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:39:54.490 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:54.491 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:54.491 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:39:54.593 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:39:54.624 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:54.625 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:54.625 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:39:54.726 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:39:54.776 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:54.776 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:54.777 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:39:54.847 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:54 smithi070 ceph-mon[30388]: 3.7 scrub starts 2024-09-16T09:39:54.847 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:54 smithi070 ceph-mon[30388]: 3.7 scrub ok 2024-09-16T09:39:54.847 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:54 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:39:54.847 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:54 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:39:54.847 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:54 smithi070 ceph-mon[30388]: 5.13 scrub starts 2024-09-16T09:39:54.847 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:54 smithi070 ceph-mon[30388]: 5.13 scrub ok 2024-09-16T09:39:54.847 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:54 smithi070 ceph-mon[30388]: osdmap e427: 8 total, 8 up, 8 in 2024-09-16T09:39:54.848 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:54 smithi070 ceph-mon[30388]: pgmap v788: 178 pgs: 1 active+recovery_wait, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 173 active+clean; 5.4 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.75k op/s; 17628/263664 objects misplaced (6.686%); 13 MiB/s, 202 objects/s recovering 2024-09-16T09:39:54.848 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:54 smithi070 ceph-mon[40691]: 3.7 scrub starts 2024-09-16T09:39:54.848 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:54 smithi070 ceph-mon[40691]: 3.7 scrub ok 2024-09-16T09:39:54.848 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:54 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:39:54.848 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:54 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:39:54.848 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:54 smithi070 ceph-mon[40691]: 5.13 scrub starts 2024-09-16T09:39:54.848 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:54 smithi070 ceph-mon[40691]: 5.13 scrub ok 2024-09-16T09:39:54.848 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:54 smithi070 ceph-mon[40691]: osdmap e427: 8 total, 8 up, 8 in 2024-09-16T09:39:54.848 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:54 smithi070 ceph-mon[40691]: pgmap v788: 178 pgs: 1 active+recovery_wait, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 173 active+clean; 5.4 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.75k op/s; 17628/263664 objects misplaced (6.686%); 13 MiB/s, 202 objects/s recovering 2024-09-16T09:39:54.878 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:39:54.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:54 smithi084 ceph-mon[32629]: 3.7 scrub starts 2024-09-16T09:39:54.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:54 smithi084 ceph-mon[32629]: 3.7 scrub ok 2024-09-16T09:39:54.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:54 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:39:54.912 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:54 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:39:54.912 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:54 smithi084 ceph-mon[32629]: 5.13 scrub starts 2024-09-16T09:39:54.912 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:54 smithi084 ceph-mon[32629]: 5.13 scrub ok 2024-09-16T09:39:54.912 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:54 smithi084 ceph-mon[32629]: osdmap e427: 8 total, 8 up, 8 in 2024-09-16T09:39:54.912 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:54 smithi084 ceph-mon[32629]: pgmap v788: 178 pgs: 1 active+recovery_wait, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 173 active+clean; 5.4 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.75k op/s; 17628/263664 objects misplaced (6.686%); 13 MiB/s, 202 objects/s recovering 2024-09-16T09:39:54.912 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:54.913 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:54.913 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:39:55.014 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:39:55.058 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:55.059 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:55.059 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:39:55.160 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:39:55.194 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:55.195 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:55.196 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:39:55.296 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:39:55.317 INFO:teuthology.orchestra.run.smithi070.stderr:no valid command found; 10 closest matches: 2024-09-16T09:39:55.317 INFO:teuthology.orchestra.run.smithi070.stderr:0 2024-09-16T09:39:55.317 INFO:teuthology.orchestra.run.smithi070.stderr:1 2024-09-16T09:39:55.318 INFO:teuthology.orchestra.run.smithi070.stderr:2 2024-09-16T09:39:55.318 INFO:teuthology.orchestra.run.smithi070.stderr:abort 2024-09-16T09:39:55.318 INFO:teuthology.orchestra.run.smithi070.stderr:assert 2024-09-16T09:39:55.318 INFO:teuthology.orchestra.run.smithi070.stderr:bench [] [] [] [] 2024-09-16T09:39:55.318 INFO:teuthology.orchestra.run.smithi070.stderr:bluefs debug_inject_read_zeros 2024-09-16T09:39:55.318 INFO:teuthology.orchestra.run.smithi070.stderr:bluefs files list 2024-09-16T09:39:55.318 INFO:teuthology.orchestra.run.smithi070.stderr:bluefs stats 2024-09-16T09:39:55.318 INFO:teuthology.orchestra.run.smithi070.stderr:bluestore allocator dump block 2024-09-16T09:39:55.318 INFO:teuthology.orchestra.run.smithi070.stderr:Error EINVAL: invalid command 2024-09-16T09:39:55.341 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:55.343 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:55.343 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:39:55.444 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:39:55.479 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:55.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:55.480 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:39:55.581 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:39:55.625 INFO:teuthology.orchestra.run.smithi070.stderr:noscrub is set 2024-09-16T09:39:55.632 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:55.633 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:55.633 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:39:55.734 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:39:55.775 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:55.775 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:55.776 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:39:55.853 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:55 smithi070 ceph-mon[30388]: 5.d scrub starts 2024-09-16T09:39:55.870 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:55 smithi070 ceph-mon[30388]: 5.d scrub ok 2024-09-16T09:39:55.870 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:55 smithi070 ceph-mon[30388]: 6.13 scrub starts 2024-09-16T09:39:55.870 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:55 smithi070 ceph-mon[30388]: 6.13 scrub ok 2024-09-16T09:39:55.870 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:55 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/768315936' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:39:55.870 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:55 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:39:55.871 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:55 smithi070 ceph-mon[40691]: 5.d scrub starts 2024-09-16T09:39:55.871 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:55 smithi070 ceph-mon[40691]: 5.d scrub ok 2024-09-16T09:39:55.871 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:55 smithi070 ceph-mon[40691]: 6.13 scrub starts 2024-09-16T09:39:55.871 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:55 smithi070 ceph-mon[40691]: 6.13 scrub ok 2024-09-16T09:39:55.871 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:55 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/768315936' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:39:55.871 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:55 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:39:55.876 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:39:55.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:55 smithi084 ceph-mon[32629]: 5.d scrub starts 2024-09-16T09:39:55.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:55 smithi084 ceph-mon[32629]: 5.d scrub ok 2024-09-16T09:39:55.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:55 smithi084 ceph-mon[32629]: 6.13 scrub starts 2024-09-16T09:39:55.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:55 smithi084 ceph-mon[32629]: 6.13 scrub ok 2024-09-16T09:39:55.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:55 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/768315936' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:39:55.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:55 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:39:55.916 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:55.917 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:55.917 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:39:56.018 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:39:56.050 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:56.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:56.051 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:39:56.083 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T09:39:56.084 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 20, 2024-09-16T09:39:56.084 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 600, 2024-09-16T09:39:56.084 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [ 2024-09-16T09:39:56.084 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.084 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:56268 9.8 9:1675f6ad:::benchmark_data_smithi070_137000_object56267:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e247)", 2024-09-16T09:39:56.084 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:33:59.290075+0000", 2024-09-16T09:39:56.084 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 356.55767064100002, 2024-09-16T09:39:56.084 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.1582197380000001, 2024-09-16T09:39:56.084 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:39:56.084 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:39:56.084 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T09:39:56.085 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:39:56.085 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:39:56.085 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:39:56.085 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 56268 2024-09-16T09:39:56.085 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.085 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:39:56.085 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.085 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:39:56.085 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.290075+0000", 2024-09-16T09:39:56.085 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:39:56.085 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.085 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.085 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:39:56.085 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.290075+0000", 2024-09-16T09:39:56.086 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:39:56.086 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.086 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.086 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:39:56.086 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.290076+0000", 2024-09-16T09:39:56.086 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.655e-06 2024-09-16T09:39:56.086 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.086 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.086 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:39:56.086 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.290105+0000", 2024-09-16T09:39:56.086 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.9164000000000001e-05 2024-09-16T09:39:56.086 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.086 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.087 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:39:56.087 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.290119+0000", 2024-09-16T09:39:56.087 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.4029000000000001e-05 2024-09-16T09:39:56.087 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.087 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.087 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:39:56.087 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.290128+0000", 2024-09-16T09:39:56.087 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.7509999999999997e-06 2024-09-16T09:39:56.087 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.087 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.087 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:39:56.087 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.294353+0000", 2024-09-16T09:39:56.087 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.004224953 2024-09-16T09:39:56.087 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.088 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.088 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:39:56.088 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.294478+0000", 2024-09-16T09:39:56.088 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00012500599999999999 2024-09-16T09:39:56.088 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.088 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.088 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 3,5", 2024-09-16T09:39:56.088 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.294612+0000", 2024-09-16T09:39:56.088 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000133566 2024-09-16T09:39:56.088 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.088 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.088 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:39:56.088 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.304980+0000", 2024-09-16T09:39:56.089 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.010368376 2024-09-16T09:39:56.089 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.089 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.089 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:39:56.089 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.309978+0000", 2024-09-16T09:39:56.089 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.004998176 2024-09-16T09:39:56.089 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.089 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.089 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:39:56.089 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.324183+0000", 2024-09-16T09:39:56.089 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.014205044 2024-09-16T09:39:56.089 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.089 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.089 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:39:56.090 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.324200+0000", 2024-09-16T09:39:56.090 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6597999999999999e-05 2024-09-16T09:39:56.090 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.090 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.090 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:39:56.090 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:00.329393+0000", 2024-09-16T09:39:56.090 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0051930309999999 2024-09-16T09:39:56.090 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.090 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.090 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:39:56.090 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:00.329404+0000", 2024-09-16T09:39:56.090 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1428e-05 2024-09-16T09:39:56.090 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.091 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.091 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:39:56.091 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.443818+0000", 2024-09-16T09:39:56.091 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1144139829999999 2024-09-16T09:39:56.091 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.091 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.091 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for degraded object", 2024-09-16T09:39:56.091 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.444263+0000", 2024-09-16T09:39:56.091 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000444785 2024-09-16T09:39:56.091 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.091 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.091 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:39:56.091 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.448204+0000", 2024-09-16T09:39:56.091 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0039411899999999998 2024-09-16T09:39:56.091 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.092 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.092 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:39:56.092 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.448294+0000", 2024-09-16T09:39:56.092 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.0002999999999999e-05 2024-09-16T09:39:56.092 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:39:56.092 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:39:56.092 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:39:56.092 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.092 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.092 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:56272 9.8 9:16bf4ef5:::benchmark_data_smithi070_137000_object56271:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e247)", 2024-09-16T09:39:56.093 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:33:59.294216+0000", 2024-09-16T09:39:56.093 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 356.55352924900001, 2024-09-16T09:39:56.093 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.1564348999999998, 2024-09-16T09:39:56.093 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:39:56.093 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:39:56.093 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T09:39:56.093 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:39:56.093 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:39:56.093 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:39:56.093 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 56272 2024-09-16T09:39:56.093 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.093 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:39:56.093 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.093 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:39:56.094 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.294216+0000", 2024-09-16T09:39:56.094 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:39:56.094 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.094 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.094 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:39:56.094 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.294216+0000", 2024-09-16T09:39:56.094 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:39:56.094 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.094 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.094 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:39:56.094 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.294216+0000", 2024-09-16T09:39:56.094 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.7999999999999996e-07 2024-09-16T09:39:56.094 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.094 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.095 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:39:56.095 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.294235+0000", 2024-09-16T09:39:56.095 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.821e-05 2024-09-16T09:39:56.095 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.095 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.095 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:39:56.095 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.294241+0000", 2024-09-16T09:39:56.095 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.3929999999999996e-06 2024-09-16T09:39:56.095 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.095 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.095 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:39:56.095 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.294246+0000", 2024-09-16T09:39:56.095 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0640000000000001e-06 2024-09-16T09:39:56.095 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.096 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.096 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:39:56.096 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.305514+0000", 2024-09-16T09:39:56.096 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.01126776 2024-09-16T09:39:56.096 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.096 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.096 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:39:56.096 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.307083+0000", 2024-09-16T09:39:56.096 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0015691730000000001 2024-09-16T09:39:56.096 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.096 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.096 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 3,5", 2024-09-16T09:39:56.096 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.307248+0000", 2024-09-16T09:39:56.097 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000164897 2024-09-16T09:39:56.097 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.097 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.097 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:39:56.097 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.310953+0000", 2024-09-16T09:39:56.097 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.003705205 2024-09-16T09:39:56.097 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.097 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.097 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:39:56.097 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.316630+0000", 2024-09-16T09:39:56.097 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0056768859999999999 2024-09-16T09:39:56.097 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.097 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.097 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:39:56.098 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.324233+0000", 2024-09-16T09:39:56.098 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0076027580000000003 2024-09-16T09:39:56.098 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.098 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.098 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:39:56.098 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.324240+0000", 2024-09-16T09:39:56.098 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.4070000000000004e-06 2024-09-16T09:39:56.098 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.098 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.098 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:39:56.098 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:00.329438+0000", 2024-09-16T09:39:56.098 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0051972819999999 2024-09-16T09:39:56.098 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.098 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.099 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:39:56.099 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:00.329447+0000", 2024-09-16T09:39:56.099 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.3770000000000002e-06 2024-09-16T09:39:56.099 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.099 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.099 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:39:56.099 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.445367+0000", 2024-09-16T09:39:56.099 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.115919661 2024-09-16T09:39:56.099 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.099 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.099 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for degraded object", 2024-09-16T09:39:56.099 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.445830+0000", 2024-09-16T09:39:56.099 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00046355399999999997 2024-09-16T09:39:56.099 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.100 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.100 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:39:56.100 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.450558+0000", 2024-09-16T09:39:56.100 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0047278470000000003 2024-09-16T09:39:56.100 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.100 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.100 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:39:56.100 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.450651+0000", 2024-09-16T09:39:56.100 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.2946000000000001e-05 2024-09-16T09:39:56.100 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:39:56.100 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:39:56.100 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:39:56.100 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.101 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.101 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:56291 9.8 9:154da08f:::benchmark_data_smithi070_137000_object56290:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e248)", 2024-09-16T09:39:56.101 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:33:59.336366+0000", 2024-09-16T09:39:56.101 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 356.51137972499998, 2024-09-16T09:39:56.101 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.1217081750000002, 2024-09-16T09:39:56.101 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:39:56.101 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:39:56.101 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:39:56.101 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:39:56.101 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:39:56.101 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:39:56.101 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 56291 2024-09-16T09:39:56.101 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.101 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:39:56.102 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.102 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:39:56.102 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.336366+0000", 2024-09-16T09:39:56.102 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:39:56.102 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.102 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.102 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:39:56.102 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.336366+0000", 2024-09-16T09:39:56.102 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:39:56.102 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.102 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.102 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:39:56.102 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.336367+0000", 2024-09-16T09:39:56.102 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9e-06 2024-09-16T09:39:56.102 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.103 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.103 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:39:56.103 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.336400+0000", 2024-09-16T09:39:56.103 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2264000000000001e-05 2024-09-16T09:39:56.103 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.103 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.103 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:39:56.103 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.336413+0000", 2024-09-16T09:39:56.103 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3414999999999999e-05 2024-09-16T09:39:56.103 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.103 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.103 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:39:56.103 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.336421+0000", 2024-09-16T09:39:56.103 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.3280000000000006e-06 2024-09-16T09:39:56.103 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.104 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.104 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:39:56.104 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.336954+0000", 2024-09-16T09:39:56.104 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00053217100000000001 2024-09-16T09:39:56.104 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.104 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.104 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:39:56.104 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.336964+0000", 2024-09-16T09:39:56.104 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0788000000000001e-05 2024-09-16T09:39:56.104 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.104 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.104 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:39:56.104 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:00.329472+0000", 2024-09-16T09:39:56.105 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.992507892 2024-09-16T09:39:56.105 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.105 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.105 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:39:56.105 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:00.329479+0000", 2024-09-16T09:39:56.105 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.0949999999999998e-06 2024-09-16T09:39:56.105 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.105 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.105 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:39:56.105 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.450681+0000", 2024-09-16T09:39:56.105 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1212013789999999 2024-09-16T09:39:56.105 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.105 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.105 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:39:56.106 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.450820+0000", 2024-09-16T09:39:56.106 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00013962100000000001 2024-09-16T09:39:56.106 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.106 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.106 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,7", 2024-09-16T09:39:56.106 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.450994+0000", 2024-09-16T09:39:56.106 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00017363099999999999 2024-09-16T09:39:56.106 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.106 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.106 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:39:56.106 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.456555+0000", 2024-09-16T09:39:56.106 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0055610900000000003 2024-09-16T09:39:56.106 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.106 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.106 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:39:56.107 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.456690+0000", 2024-09-16T09:39:56.107 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000134957 2024-09-16T09:39:56.107 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.107 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.107 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:39:56.107 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.456706+0000", 2024-09-16T09:39:56.107 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5968999999999999e-05 2024-09-16T09:39:56.107 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.107 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.107 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:39:56.107 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.456755+0000", 2024-09-16T09:39:56.107 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.8538000000000002e-05 2024-09-16T09:39:56.107 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.107 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.108 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:39:56.108 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.458074+0000", 2024-09-16T09:39:56.108 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001319137 2024-09-16T09:39:56.108 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:39:56.108 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:39:56.108 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:39:56.108 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.108 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.108 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:56320 9.8 9:10470d54:::benchmark_data_smithi070_137000_object56319:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e248)", 2024-09-16T09:39:56.108 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:33:59.372799+0000", 2024-09-16T09:39:56.108 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 356.47494599599997, 2024-09-16T09:39:56.108 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0883279570000002, 2024-09-16T09:39:56.108 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:39:56.108 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:39:56.109 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:39:56.109 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:39:56.109 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:39:56.109 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:39:56.109 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 56320 2024-09-16T09:39:56.109 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.109 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:39:56.109 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.109 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:39:56.109 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.372799+0000", 2024-09-16T09:39:56.109 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:39:56.109 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.109 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.109 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:39:56.110 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.372799+0000", 2024-09-16T09:39:56.110 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:39:56.110 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.110 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.110 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:39:56.110 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.372801+0000", 2024-09-16T09:39:56.110 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.269e-06 2024-09-16T09:39:56.110 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.110 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.110 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:39:56.110 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.372824+0000", 2024-09-16T09:39:56.110 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3258999999999999e-05 2024-09-16T09:39:56.110 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.110 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.111 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:39:56.111 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.372840+0000", 2024-09-16T09:39:56.111 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5809999999999999e-05 2024-09-16T09:39:56.111 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.111 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.111 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:39:56.111 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.372847+0000", 2024-09-16T09:39:56.111 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.7139999999999993e-06 2024-09-16T09:39:56.111 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.111 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.111 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:39:56.111 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.373173+0000", 2024-09-16T09:39:56.111 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00032524399999999998 2024-09-16T09:39:56.112 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.112 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.112 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:39:56.112 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.373179+0000", 2024-09-16T09:39:56.112 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.5309999999999998e-06 2024-09-16T09:39:56.112 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.112 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.112 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:39:56.112 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:00.329503+0000", 2024-09-16T09:39:56.112 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.95632355099999999 2024-09-16T09:39:56.112 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.112 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.112 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:39:56.112 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:00.329510+0000", 2024-09-16T09:39:56.113 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.0759999999999997e-06 2024-09-16T09:39:56.113 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.113 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.113 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:39:56.113 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.452119+0000", 2024-09-16T09:39:56.113 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.122609137 2024-09-16T09:39:56.113 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.113 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.113 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:39:56.113 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.452252+0000", 2024-09-16T09:39:56.113 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00013358599999999999 2024-09-16T09:39:56.113 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.113 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.113 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,7", 2024-09-16T09:39:56.114 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.452410+0000", 2024-09-16T09:39:56.114 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000157508 2024-09-16T09:39:56.114 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.114 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.114 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:39:56.114 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.458086+0000", 2024-09-16T09:39:56.114 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.005675785 2024-09-16T09:39:56.114 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.114 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.114 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:39:56.114 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.459588+0000", 2024-09-16T09:39:56.114 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0015021780000000001 2024-09-16T09:39:56.114 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.114 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.114 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:39:56.115 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.459644+0000", 2024-09-16T09:39:56.115 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.588e-05 2024-09-16T09:39:56.115 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.115 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.115 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:39:56.115 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.459704+0000", 2024-09-16T09:39:56.115 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.0590000000000001e-05 2024-09-16T09:39:56.115 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.115 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.115 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:39:56.115 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.461127+0000", 2024-09-16T09:39:56.115 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001422839 2024-09-16T09:39:56.115 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:39:56.115 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:39:56.115 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:39:56.116 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.116 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.116 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:56331 9.8 9:15f0826d:::benchmark_data_smithi070_137000_object56330:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e248)", 2024-09-16T09:39:56.116 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:33:59.382251+0000", 2024-09-16T09:39:56.116 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 356.465494476, 2024-09-16T09:39:56.116 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0852117350000001, 2024-09-16T09:39:56.116 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:39:56.116 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:39:56.116 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:39:56.116 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:39:56.116 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:39:56.116 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:39:56.116 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 56331 2024-09-16T09:39:56.116 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.117 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:39:56.117 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.117 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:39:56.117 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.382251+0000", 2024-09-16T09:39:56.117 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:39:56.117 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.117 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.117 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:39:56.117 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.382251+0000", 2024-09-16T09:39:56.117 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:39:56.117 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.117 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.117 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:39:56.117 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.382253+0000", 2024-09-16T09:39:56.118 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.84e-06 2024-09-16T09:39:56.118 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.118 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.118 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:39:56.118 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.382275+0000", 2024-09-16T09:39:56.118 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.2112000000000001e-05 2024-09-16T09:39:56.118 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.118 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.118 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:39:56.118 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.382287+0000", 2024-09-16T09:39:56.118 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2593e-05 2024-09-16T09:39:56.118 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.118 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.118 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:39:56.119 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.382295+0000", 2024-09-16T09:39:56.119 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6040000000000004e-06 2024-09-16T09:39:56.119 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.119 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.119 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:39:56.119 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.382349+0000", 2024-09-16T09:39:56.119 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4032e-05 2024-09-16T09:39:56.119 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.119 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.119 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:39:56.119 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.382358+0000", 2024-09-16T09:39:56.119 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.789e-06 2024-09-16T09:39:56.119 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.119 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.120 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:39:56.120 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:00.329532+0000", 2024-09-16T09:39:56.120 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.94717437299999996 2024-09-16T09:39:56.120 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.120 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.120 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:39:56.120 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:00.329539+0000", 2024-09-16T09:39:56.120 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.9999999999999999e-06 2024-09-16T09:39:56.120 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.120 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.120 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:39:56.120 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.453657+0000", 2024-09-16T09:39:56.120 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1241178030000001 2024-09-16T09:39:56.120 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.120 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.121 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:39:56.121 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.453811+0000", 2024-09-16T09:39:56.121 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00015423800000000001 2024-09-16T09:39:56.121 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.121 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.121 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,7", 2024-09-16T09:39:56.121 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.453990+0000", 2024-09-16T09:39:56.121 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00017888499999999999 2024-09-16T09:39:56.121 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.121 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.121 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:39:56.121 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.465277+0000", 2024-09-16T09:39:56.121 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.011287346 2024-09-16T09:39:56.121 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.122 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.122 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:39:56.122 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.465324+0000", 2024-09-16T09:39:56.122 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.6168999999999998e-05 2024-09-16T09:39:56.122 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.122 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.122 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:39:56.122 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.466075+0000", 2024-09-16T09:39:56.122 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00075194300000000003 2024-09-16T09:39:56.122 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.122 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.122 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:39:56.122 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.466124+0000", 2024-09-16T09:39:56.122 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.8591000000000003e-05 2024-09-16T09:39:56.122 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.123 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.123 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:39:56.123 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.467463+0000", 2024-09-16T09:39:56.123 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0013384169999999999 2024-09-16T09:39:56.123 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:39:56.123 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:39:56.123 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:39:56.123 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.123 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.123 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:56335 9.8 9:17f26bc5:::benchmark_data_smithi070_137000_object56334:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e248)", 2024-09-16T09:39:56.123 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:33:59.387512+0000", 2024-09-16T09:39:56.123 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 356.46023330200001, 2024-09-16T09:39:56.123 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0818823069999999, 2024-09-16T09:39:56.124 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:39:56.124 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:39:56.124 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:39:56.124 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:39:56.124 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:39:56.124 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:39:56.124 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 56335 2024-09-16T09:39:56.124 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.124 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:39:56.124 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.124 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:39:56.124 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.387512+0000", 2024-09-16T09:39:56.124 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:39:56.124 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.125 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.125 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:39:56.125 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.387512+0000", 2024-09-16T09:39:56.125 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:39:56.125 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.125 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.125 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:39:56.125 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.387513+0000", 2024-09-16T09:39:56.125 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3009999999999999e-06 2024-09-16T09:39:56.125 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.125 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.125 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:39:56.125 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.387541+0000", 2024-09-16T09:39:56.125 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.7489000000000001e-05 2024-09-16T09:39:56.126 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.126 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.126 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:39:56.126 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.387552+0000", 2024-09-16T09:39:56.126 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1311000000000001e-05 2024-09-16T09:39:56.126 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.126 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.126 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:39:56.126 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.387560+0000", 2024-09-16T09:39:56.126 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6090000000000003e-06 2024-09-16T09:39:56.126 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.126 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.126 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:39:56.126 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.387621+0000", 2024-09-16T09:39:56.126 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1826000000000001e-05 2024-09-16T09:39:56.127 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.127 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.127 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:39:56.127 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.389396+0000", 2024-09-16T09:39:56.127 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0017745560000000001 2024-09-16T09:39:56.127 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.127 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.127 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:39:56.127 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:00.329562+0000", 2024-09-16T09:39:56.127 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.94016562599999998 2024-09-16T09:39:56.127 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.127 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.127 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:39:56.127 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:00.329569+0000", 2024-09-16T09:39:56.128 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3270000000000003e-06 2024-09-16T09:39:56.128 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.128 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.128 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:39:56.128 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.455069+0000", 2024-09-16T09:39:56.128 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1255000989999999 2024-09-16T09:39:56.128 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.128 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.128 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:39:56.128 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.455197+0000", 2024-09-16T09:39:56.128 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00012834399999999999 2024-09-16T09:39:56.128 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.128 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.128 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,7", 2024-09-16T09:39:56.129 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.455353+0000", 2024-09-16T09:39:56.129 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000155308 2024-09-16T09:39:56.129 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.129 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.129 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:39:56.129 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.465366+0000", 2024-09-16T09:39:56.129 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.01001286 2024-09-16T09:39:56.129 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.129 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.129 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:39:56.129 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.465415+0000", 2024-09-16T09:39:56.129 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.8909000000000002e-05 2024-09-16T09:39:56.129 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.129 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.130 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:39:56.130 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.467474+0000", 2024-09-16T09:39:56.130 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0020590360000000002 2024-09-16T09:39:56.130 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.130 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.130 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:39:56.130 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.467527+0000", 2024-09-16T09:39:56.130 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.3908e-05 2024-09-16T09:39:56.130 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.130 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.130 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:39:56.130 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.469394+0000", 2024-09-16T09:39:56.130 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001866798 2024-09-16T09:39:56.130 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:39:56.131 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:39:56.131 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:39:56.131 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.131 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.131 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:56337 9.5 9:a00652f1:::benchmark_data_smithi070_137000_object56336:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e248)", 2024-09-16T09:39:56.131 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:33:59.390215+0000", 2024-09-16T09:39:56.131 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 356.45753056699999, 2024-09-16T09:39:56.131 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0362062170000002, 2024-09-16T09:39:56.131 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:39:56.131 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:39:56.131 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:39:56.131 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:39:56.131 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:39:56.131 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:39:56.131 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 56337 2024-09-16T09:39:56.132 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.132 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:39:56.132 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.132 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:39:56.132 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.390215+0000", 2024-09-16T09:39:56.132 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:39:56.132 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.132 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.132 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:39:56.132 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.390215+0000", 2024-09-16T09:39:56.132 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:39:56.132 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.132 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.132 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:39:56.133 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.390216+0000", 2024-09-16T09:39:56.133 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.4789999999999999e-06 2024-09-16T09:39:56.133 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.133 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.133 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:39:56.133 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.390239+0000", 2024-09-16T09:39:56.133 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.2495000000000001e-05 2024-09-16T09:39:56.133 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.133 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.133 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:39:56.133 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.390250+0000", 2024-09-16T09:39:56.133 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.114e-05 2024-09-16T09:39:56.133 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.133 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.134 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:39:56.134 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.390256+0000", 2024-09-16T09:39:56.134 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.3040000000000001e-06 2024-09-16T09:39:56.134 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.134 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.134 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:39:56.134 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.390312+0000", 2024-09-16T09:39:56.134 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5562000000000001e-05 2024-09-16T09:39:56.134 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.134 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.134 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:39:56.134 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.390321+0000", 2024-09-16T09:39:56.134 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.9090000000000006e-06 2024-09-16T09:39:56.134 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.134 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.135 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:39:56.135 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:00.325539+0000", 2024-09-16T09:39:56.135 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.93521816300000005 2024-09-16T09:39:56.135 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.135 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.135 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:39:56.135 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:00.325551+0000", 2024-09-16T09:39:56.135 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2512e-05 2024-09-16T09:39:56.135 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.135 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.135 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:39:56.135 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.421298+0000", 2024-09-16T09:39:56.135 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.095746436 2024-09-16T09:39:56.135 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.136 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.136 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:39:56.136 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.421439+0000", 2024-09-16T09:39:56.136 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000141519 2024-09-16T09:39:56.136 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.136 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.136 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,5", 2024-09-16T09:39:56.136 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.421596+0000", 2024-09-16T09:39:56.136 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00015686199999999999 2024-09-16T09:39:56.136 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.136 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.136 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:39:56.136 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.424192+0000", 2024-09-16T09:39:56.136 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0025960250000000001 2024-09-16T09:39:56.137 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.137 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.137 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:39:56.137 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.424254+0000", 2024-09-16T09:39:56.137 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1910999999999997e-05 2024-09-16T09:39:56.137 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.137 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.137 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:39:56.137 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.425261+0000", 2024-09-16T09:39:56.137 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001006514 2024-09-16T09:39:56.137 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.137 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.137 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:39:56.137 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.425323+0000", 2024-09-16T09:39:56.138 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.2603999999999999e-05 2024-09-16T09:39:56.138 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.138 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.138 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:39:56.138 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.426421+0000", 2024-09-16T09:39:56.138 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0010977820000000001 2024-09-16T09:39:56.138 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:39:56.138 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:39:56.138 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:39:56.138 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.138 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.138 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:56346 9.8 9:12468c50:::benchmark_data_smithi070_137000_object56345:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e248)", 2024-09-16T09:39:56.138 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:33:59.398468+0000", 2024-09-16T09:39:56.139 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 356.44927679599999, 2024-09-16T09:39:56.139 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0731202789999998, 2024-09-16T09:39:56.139 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:39:56.139 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:39:56.139 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:39:56.139 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:39:56.139 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:39:56.139 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:39:56.139 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 56346 2024-09-16T09:39:56.139 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.139 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:39:56.139 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.139 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:39:56.139 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.398468+0000", 2024-09-16T09:39:56.140 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:39:56.140 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.140 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.140 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:39:56.140 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.398468+0000", 2024-09-16T09:39:56.140 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:39:56.140 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.140 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.140 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:39:56.140 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.398470+0000", 2024-09-16T09:39:56.140 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.4330000000000001e-06 2024-09-16T09:39:56.140 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.140 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.140 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:39:56.140 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.398492+0000", 2024-09-16T09:39:56.141 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.1801e-05 2024-09-16T09:39:56.141 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.141 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.141 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:39:56.141 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.398502+0000", 2024-09-16T09:39:56.141 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0226e-05 2024-09-16T09:39:56.141 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.141 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.141 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:39:56.141 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.398508+0000", 2024-09-16T09:39:56.141 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1369999999999999e-06 2024-09-16T09:39:56.141 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.141 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.142 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:39:56.142 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.400785+0000", 2024-09-16T09:39:56.142 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0022771190000000002 2024-09-16T09:39:56.142 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.142 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.142 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:39:56.142 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.400795+0000", 2024-09-16T09:39:56.142 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.4870000000000008e-06 2024-09-16T09:39:56.142 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.142 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.142 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:39:56.142 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:00.329592+0000", 2024-09-16T09:39:56.142 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.92879704500000004 2024-09-16T09:39:56.142 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.143 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.143 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:39:56.143 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:00.329599+0000", 2024-09-16T09:39:56.143 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.0700000000000001e-06 2024-09-16T09:39:56.143 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.143 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.143 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:39:56.143 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.458122+0000", 2024-09-16T09:39:56.143 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1285228519999999 2024-09-16T09:39:56.143 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.143 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.143 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:39:56.143 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.458231+0000", 2024-09-16T09:39:56.143 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000109332 2024-09-16T09:39:56.144 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.144 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.144 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,7", 2024-09-16T09:39:56.144 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.458441+0000", 2024-09-16T09:39:56.144 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00021033299999999999 2024-09-16T09:39:56.144 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.144 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.144 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:39:56.144 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.469462+0000", 2024-09-16T09:39:56.144 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.011020914999999999 2024-09-16T09:39:56.144 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.144 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.144 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:39:56.144 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.469499+0000", 2024-09-16T09:39:56.145 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6566999999999997e-05 2024-09-16T09:39:56.145 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.145 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.145 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:39:56.145 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.469801+0000", 2024-09-16T09:39:56.145 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00030174599999999998 2024-09-16T09:39:56.145 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.145 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.145 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:39:56.145 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.470011+0000", 2024-09-16T09:39:56.145 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000210595 2024-09-16T09:39:56.145 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.145 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.146 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:39:56.146 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.471589+0000", 2024-09-16T09:39:56.146 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001577621 2024-09-16T09:39:56.146 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:39:56.146 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:39:56.146 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:39:56.146 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.146 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.146 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:56348 9.8 9:151f704f:::benchmark_data_smithi070_137000_object56347:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e248)", 2024-09-16T09:39:56.146 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:33:59.401792+0000", 2024-09-16T09:39:56.146 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 356.445953401, 2024-09-16T09:39:56.146 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0809194390000001, 2024-09-16T09:39:56.146 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:39:56.146 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:39:56.146 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:39:56.147 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:39:56.147 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:39:56.147 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:39:56.147 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 56348 2024-09-16T09:39:56.147 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.147 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:39:56.147 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.147 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:39:56.147 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.401792+0000", 2024-09-16T09:39:56.147 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:39:56.147 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.147 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.148 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:39:56.148 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.401792+0000", 2024-09-16T09:39:56.148 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:39:56.148 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.148 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.148 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:39:56.148 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.401794+0000", 2024-09-16T09:39:56.148 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.657e-06 2024-09-16T09:39:56.148 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.148 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.148 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:39:56.148 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.401819+0000", 2024-09-16T09:39:56.148 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.5366000000000001e-05 2024-09-16T09:39:56.148 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.149 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.149 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:39:56.149 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.401831+0000", 2024-09-16T09:39:56.149 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2116000000000001e-05 2024-09-16T09:39:56.149 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.149 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.149 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:39:56.149 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.401837+0000", 2024-09-16T09:39:56.149 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.2049999999999999e-06 2024-09-16T09:39:56.149 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.149 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.149 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:39:56.149 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.401966+0000", 2024-09-16T09:39:56.149 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00012920400000000001 2024-09-16T09:39:56.149 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.150 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.150 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:39:56.150 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:33:59.401974+0000", 2024-09-16T09:39:56.150 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3320000000000003e-06 2024-09-16T09:39:56.150 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.150 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.150 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:39:56.150 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:00.329622+0000", 2024-09-16T09:39:56.150 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.92764788399999998 2024-09-16T09:39:56.150 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.150 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.150 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:39:56.150 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:00.329629+0000", 2024-09-16T09:39:56.150 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.9800000000000001e-06 2024-09-16T09:39:56.151 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.151 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.151 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:39:56.151 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.461167+0000", 2024-09-16T09:39:56.151 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1315382869999999 2024-09-16T09:39:56.151 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.151 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.151 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:39:56.151 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.461323+0000", 2024-09-16T09:39:56.151 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000156477 2024-09-16T09:39:56.151 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.151 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.151 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,7", 2024-09-16T09:39:56.151 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.461496+0000", 2024-09-16T09:39:56.151 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00017256800000000001 2024-09-16T09:39:56.151 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.152 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.152 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:39:56.152 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.469507+0000", 2024-09-16T09:39:56.152 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0080107310000000001 2024-09-16T09:39:56.152 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.152 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.152 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:39:56.152 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.469586+0000", 2024-09-16T09:39:56.152 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.9714000000000003e-05 2024-09-16T09:39:56.152 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.152 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.152 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:39:56.152 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.480994+0000", 2024-09-16T09:39:56.153 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.011407782 2024-09-16T09:39:56.153 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.153 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.153 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:39:56.153 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.481093+0000", 2024-09-16T09:39:56.153 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.8571000000000002e-05 2024-09-16T09:39:56.153 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.153 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.153 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:39:56.153 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:01.482711+0000", 2024-09-16T09:39:56.153 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001618565 2024-09-16T09:39:56.153 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:39:56.153 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:39:56.153 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:39:56.153 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.154 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.154 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:175691 9.9 9.b751ed49 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e304)", 2024-09-16T09:39:56.154 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:36:06.341011+0000", 2024-09-16T09:39:56.154 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 229.50673402199999, 2024-09-16T09:39:56.154 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0983883910000001, 2024-09-16T09:39:56.154 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:39:56.154 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:39:56.154 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T09:39:56.154 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:39:56.154 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:39:56.154 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:39:56.154 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 175691 2024-09-16T09:39:56.154 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.154 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:39:56.154 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.155 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:39:56.155 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:06.341011+0000", 2024-09-16T09:39:56.155 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:39:56.155 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.155 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.155 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:39:56.155 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:06.341011+0000", 2024-09-16T09:39:56.155 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:39:56.155 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.155 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.155 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:39:56.155 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:06.341013+0000", 2024-09-16T09:39:56.155 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.4139999999999999e-06 2024-09-16T09:39:56.155 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.155 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.156 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:39:56.156 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:06.341017+0000", 2024-09-16T09:39:56.156 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.8289999999999998e-06 2024-09-16T09:39:56.156 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.156 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.156 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:39:56.156 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:06.341032+0000", 2024-09-16T09:39:56.156 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.4956999999999999e-05 2024-09-16T09:39:56.156 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.156 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.156 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:39:56.156 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:06.341040+0000", 2024-09-16T09:39:56.156 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6290000000000001e-06 2024-09-16T09:39:56.156 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.156 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.157 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:39:56.157 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:06.341110+0000", 2024-09-16T09:39:56.157 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.9534000000000002e-05 2024-09-16T09:39:56.157 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.157 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.157 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op must wait for map", 2024-09-16T09:39:56.157 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:06.341116+0000", 2024-09-16T09:39:56.157 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.2090000000000002e-06 2024-09-16T09:39:56.157 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.157 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.157 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:39:56.157 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:06.350340+0000", 2024-09-16T09:39:56.157 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0092238479999999998 2024-09-16T09:39:56.157 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.157 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.158 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:39:56.158 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:06.350350+0000", 2024-09-16T09:39:56.158 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0281e-05 2024-09-16T09:39:56.158 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.158 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.158 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:39:56.158 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:07.439357+0000", 2024-09-16T09:39:56.158 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.08900679 2024-09-16T09:39:56.158 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.158 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.158 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:39:56.158 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:07.439400+0000", 2024-09-16T09:39:56.158 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2899999999999999e-05 2024-09-16T09:39:56.158 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:39:56.159 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:39:56.159 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:39:56.159 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.159 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.159 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:175720 9.19 9.74aa9ef9 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e304)", 2024-09-16T09:39:56.159 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:36:06.379575+0000", 2024-09-16T09:39:56.159 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 229.46816995699999, 2024-09-16T09:39:56.159 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0607223160000001, 2024-09-16T09:39:56.159 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:39:56.159 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:39:56.159 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T09:39:56.159 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:39:56.159 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:39:56.159 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:39:56.159 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 175720 2024-09-16T09:39:56.159 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.160 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:39:56.160 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.160 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:39:56.160 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:06.379575+0000", 2024-09-16T09:39:56.160 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:39:56.160 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.160 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.160 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:39:56.160 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:06.379575+0000", 2024-09-16T09:39:56.160 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:39:56.160 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.160 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.160 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:39:56.160 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:06.379576+0000", 2024-09-16T09:39:56.161 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.0800000000000003e-07 2024-09-16T09:39:56.161 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.161 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.161 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:39:56.161 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:06.379580+0000", 2024-09-16T09:39:56.161 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9210000000000002e-06 2024-09-16T09:39:56.161 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.161 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.161 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:39:56.161 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:06.379592+0000", 2024-09-16T09:39:56.161 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.184e-05 2024-09-16T09:39:56.161 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.161 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.161 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:39:56.162 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:06.379600+0000", 2024-09-16T09:39:56.162 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.8059999999999995e-06 2024-09-16T09:39:56.162 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.162 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.162 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:39:56.162 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:06.379666+0000", 2024-09-16T09:39:56.162 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.6243999999999998e-05 2024-09-16T09:39:56.162 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.162 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.162 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:39:56.162 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:06.379676+0000", 2024-09-16T09:39:56.162 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0413e-05 2024-09-16T09:39:56.162 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.162 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.163 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:39:56.163 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:07.440247+0000", 2024-09-16T09:39:56.163 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.060571003 2024-09-16T09:39:56.163 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.163 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.163 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:39:56.163 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:07.440298+0000", 2024-09-16T09:39:56.163 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0180999999999999e-05 2024-09-16T09:39:56.163 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:39:56.163 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:39:56.163 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:39:56.163 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.163 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.163 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:175736 9.9 9.2dd87e89 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e304)", 2024-09-16T09:39:56.164 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:36:06.395074+0000", 2024-09-16T09:39:56.164 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 229.45267142099999, 2024-09-16T09:39:56.164 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.044973742, 2024-09-16T09:39:56.164 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:39:56.164 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:39:56.164 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T09:39:56.164 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:39:56.164 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:39:56.164 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:39:56.164 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 175736 2024-09-16T09:39:56.164 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.164 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:39:56.164 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.164 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:39:56.165 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:06.395074+0000", 2024-09-16T09:39:56.165 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:39:56.165 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.165 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.165 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:39:56.165 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:06.395074+0000", 2024-09-16T09:39:56.165 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:39:56.165 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.165 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.165 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:39:56.165 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:06.395074+0000", 2024-09-16T09:39:56.165 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.2900000000000004e-07 2024-09-16T09:39:56.165 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.165 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.166 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:39:56.166 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:06.395077+0000", 2024-09-16T09:39:56.166 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.6129999999999998e-06 2024-09-16T09:39:56.166 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.166 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.166 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:39:56.166 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:06.395084+0000", 2024-09-16T09:39:56.166 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.8190000000000003e-06 2024-09-16T09:39:56.166 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.166 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.166 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:39:56.166 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:06.395089+0000", 2024-09-16T09:39:56.166 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.6189999999999999e-06 2024-09-16T09:39:56.166 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.167 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.167 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:39:56.167 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:06.395228+0000", 2024-09-16T09:39:56.167 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00013882099999999999 2024-09-16T09:39:56.167 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.167 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.167 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:39:56.167 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:06.395239+0000", 2024-09-16T09:39:56.167 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0645000000000001e-05 2024-09-16T09:39:56.167 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.167 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.167 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:39:56.167 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:07.440039+0000", 2024-09-16T09:39:56.167 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.044799668 2024-09-16T09:39:56.168 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.168 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.168 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:39:56.168 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:07.440048+0000", 2024-09-16T09:39:56.168 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.0280000000000006e-06 2024-09-16T09:39:56.168 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:39:56.168 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:39:56.168 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:39:56.168 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.168 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.168 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:231317 9.15 9:aaf7d9df:::benchmark_data_smithi070_137000_object109606:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e357)", 2024-09-16T09:39:56.168 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:37:18.512274+0000", 2024-09-16T09:39:56.168 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 157.335471289, 2024-09-16T09:39:56.169 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0402452229999999, 2024-09-16T09:39:56.169 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:39:56.169 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:39:56.169 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T09:39:56.169 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:39:56.169 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:39:56.169 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:39:56.169 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 231317 2024-09-16T09:39:56.169 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.169 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:39:56.169 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.169 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:39:56.169 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:18.512274+0000", 2024-09-16T09:39:56.169 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:39:56.170 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.170 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.170 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:39:56.170 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:18.512274+0000", 2024-09-16T09:39:56.170 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:39:56.170 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.170 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.170 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:39:56.170 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:18.512276+0000", 2024-09-16T09:39:56.170 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3140000000000002e-06 2024-09-16T09:39:56.170 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.170 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.170 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:39:56.170 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:18.512284+0000", 2024-09-16T09:39:56.170 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.106e-06 2024-09-16T09:39:56.171 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.171 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.171 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:39:56.171 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:18.512304+0000", 2024-09-16T09:39:56.171 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0038999999999999e-05 2024-09-16T09:39:56.171 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.171 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.171 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:39:56.171 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:18.512315+0000", 2024-09-16T09:39:56.171 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0587e-05 2024-09-16T09:39:56.171 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.171 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.171 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:39:56.171 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:18.625285+0000", 2024-09-16T09:39:56.171 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.112970271 2024-09-16T09:39:56.172 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.172 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.172 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:39:56.172 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:18.625293+0000", 2024-09-16T09:39:56.172 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6499999999999996e-06 2024-09-16T09:39:56.172 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.172 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.172 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:39:56.172 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:19.552302+0000", 2024-09-16T09:39:56.172 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.92700903000000001 2024-09-16T09:39:56.172 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.172 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.172 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:39:56.172 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:19.552519+0000", 2024-09-16T09:39:56.173 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000217226 2024-09-16T09:39:56.173 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:39:56.173 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:39:56.173 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:39:56.173 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.173 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.173 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:231330 9.19 9:9bc810af:::benchmark_data_smithi070_137000_object109619:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e357)", 2024-09-16T09:39:56.173 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:37:18.512335+0000", 2024-09-16T09:39:56.173 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 157.335410602, 2024-09-16T09:39:56.173 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.058821751, 2024-09-16T09:39:56.173 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:39:56.173 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:39:56.173 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:39:56.173 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:39:56.174 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:39:56.174 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:39:56.174 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 231330 2024-09-16T09:39:56.174 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.174 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:39:56.174 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.174 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:39:56.174 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:18.512335+0000", 2024-09-16T09:39:56.174 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:39:56.174 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.174 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.174 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:39:56.174 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:18.512335+0000", 2024-09-16T09:39:56.174 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:39:56.174 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.175 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.175 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:39:56.175 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:18.512335+0000", 2024-09-16T09:39:56.175 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0900000000000002e-07 2024-09-16T09:39:56.175 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.175 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.175 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:39:56.175 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:18.512338+0000", 2024-09-16T09:39:56.175 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.4530000000000001e-06 2024-09-16T09:39:56.175 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.175 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.175 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:39:56.175 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:18.512345+0000", 2024-09-16T09:39:56.175 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.9129999999999997e-06 2024-09-16T09:39:56.176 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.176 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.176 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:39:56.176 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:18.512350+0000", 2024-09-16T09:39:56.176 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.344e-06 2024-09-16T09:39:56.176 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.176 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.176 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:39:56.176 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:18.785156+0000", 2024-09-16T09:39:56.176 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.27280600599999999 2024-09-16T09:39:56.176 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.176 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.176 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:39:56.176 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:18.785164+0000", 2024-09-16T09:39:56.177 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.568e-06 2024-09-16T09:39:56.177 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.177 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.177 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:39:56.177 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:19.554018+0000", 2024-09-16T09:39:56.177 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.76885353300000003 2024-09-16T09:39:56.177 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.177 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.177 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:39:56.177 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:19.554177+0000", 2024-09-16T09:39:56.177 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000158548 2024-09-16T09:39:56.177 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.177 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.177 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 5,7", 2024-09-16T09:39:56.178 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:19.554311+0000", 2024-09-16T09:39:56.178 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00013476499999999999 2024-09-16T09:39:56.178 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.178 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.178 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:39:56.178 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:19.555602+0000", 2024-09-16T09:39:56.178 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001290975 2024-09-16T09:39:56.178 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.178 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.178 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:39:56.178 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:19.556689+0000", 2024-09-16T09:39:56.178 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0010867889999999999 2024-09-16T09:39:56.178 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.178 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.178 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:39:56.179 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:19.571062+0000", 2024-09-16T09:39:56.179 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.014372818000000001 2024-09-16T09:39:56.179 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.179 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.179 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:39:56.179 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:19.571116+0000", 2024-09-16T09:39:56.179 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.3925000000000003e-05 2024-09-16T09:39:56.179 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.179 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.179 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:39:56.179 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:19.571156+0000", 2024-09-16T09:39:56.179 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0605000000000002e-05 2024-09-16T09:39:56.179 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:39:56.179 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:39:56.179 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:39:56.180 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.180 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.180 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:232724 9.19 9:9df51a8e:::benchmark_data_smithi070_137000_object111013:head [delete] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e360)", 2024-09-16T09:39:56.180 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:37:21.522489+0000", 2024-09-16T09:39:56.180 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 154.32525664100001, 2024-09-16T09:39:56.180 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.123121356, 2024-09-16T09:39:56.180 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:39:56.180 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:39:56.180 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T09:39:56.180 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:39:56.180 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:39:56.180 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:39:56.180 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 232724 2024-09-16T09:39:56.180 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.181 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:39:56.181 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.181 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:39:56.181 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.522489+0000", 2024-09-16T09:39:56.181 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:39:56.181 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.181 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.181 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:39:56.181 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.522489+0000", 2024-09-16T09:39:56.181 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:39:56.181 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.181 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.181 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:39:56.181 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.522491+0000", 2024-09-16T09:39:56.182 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.2869999999999999e-06 2024-09-16T09:39:56.182 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.182 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.182 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:39:56.182 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.522499+0000", 2024-09-16T09:39:56.182 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.1009999999999992e-06 2024-09-16T09:39:56.182 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.182 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.182 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:39:56.182 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.522521+0000", 2024-09-16T09:39:56.182 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.2373000000000001e-05 2024-09-16T09:39:56.182 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.182 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.182 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:39:56.182 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.522533+0000", 2024-09-16T09:39:56.183 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1231e-05 2024-09-16T09:39:56.183 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.183 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.183 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:39:56.183 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.522754+0000", 2024-09-16T09:39:56.183 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00022174900000000001 2024-09-16T09:39:56.183 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.183 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.183 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op must wait for map", 2024-09-16T09:39:56.183 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.522760+0000", 2024-09-16T09:39:56.183 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.2379999999999997e-06 2024-09-16T09:39:56.183 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.183 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.184 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:39:56.184 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.528487+0000", 2024-09-16T09:39:56.184 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.005727195 2024-09-16T09:39:56.184 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.184 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.184 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:39:56.184 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.528499+0000", 2024-09-16T09:39:56.184 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2656999999999999e-05 2024-09-16T09:39:56.184 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.184 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.184 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:39:56.184 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.645538+0000", 2024-09-16T09:39:56.184 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1170389300000001 2024-09-16T09:39:56.184 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.185 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.185 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:39:56.185 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.645610+0000", 2024-09-16T09:39:56.185 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.1594999999999995e-05 2024-09-16T09:39:56.185 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:39:56.185 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:39:56.185 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:39:56.185 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.185 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.185 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:232730 9.19 9:99b28832:::benchmark_data_smithi070_137000_object111019:head [delete] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e360)", 2024-09-16T09:39:56.185 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:37:21.522561+0000", 2024-09-16T09:39:56.185 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 154.32518440199999, 2024-09-16T09:39:56.185 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.129695954, 2024-09-16T09:39:56.185 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:39:56.186 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:39:56.186 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:39:56.186 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:39:56.186 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:39:56.186 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:39:56.186 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 232730 2024-09-16T09:39:56.186 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.186 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:39:56.186 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.186 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:39:56.186 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.522561+0000", 2024-09-16T09:39:56.186 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:39:56.186 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.186 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.186 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:39:56.187 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.522561+0000", 2024-09-16T09:39:56.187 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:39:56.187 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.187 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.187 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:39:56.187 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.522561+0000", 2024-09-16T09:39:56.187 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8699999999999995e-07 2024-09-16T09:39:56.187 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.187 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.187 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:39:56.187 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.522565+0000", 2024-09-16T09:39:56.187 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3790000000000002e-06 2024-09-16T09:39:56.187 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.187 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.188 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:39:56.188 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.522573+0000", 2024-09-16T09:39:56.188 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.3799999999999994e-06 2024-09-16T09:39:56.188 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.188 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.188 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:39:56.188 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.522579+0000", 2024-09-16T09:39:56.188 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1099999999999999e-06 2024-09-16T09:39:56.188 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.188 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.188 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:39:56.188 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.522780+0000", 2024-09-16T09:39:56.188 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00020084399999999999 2024-09-16T09:39:56.188 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.189 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.189 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting_for_map not empty", 2024-09-16T09:39:56.189 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.522785+0000", 2024-09-16T09:39:56.189 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5750000000000002e-06 2024-09-16T09:39:56.189 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.189 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.189 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:39:56.189 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.528530+0000", 2024-09-16T09:39:56.189 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0057457580000000001 2024-09-16T09:39:56.189 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.189 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.189 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:39:56.189 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.528537+0000", 2024-09-16T09:39:56.189 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.0780000000000001e-06 2024-09-16T09:39:56.189 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.190 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.190 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:39:56.190 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.645817+0000", 2024-09-16T09:39:56.190 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.117280482 2024-09-16T09:39:56.190 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.190 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.190 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:39:56.190 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.645920+0000", 2024-09-16T09:39:56.190 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00010247999999999999 2024-09-16T09:39:56.190 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.190 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.190 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 3,5,6,7", 2024-09-16T09:39:56.190 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.646661+0000", 2024-09-16T09:39:56.190 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00074141599999999995 2024-09-16T09:39:56.191 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.191 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.191 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:39:56.191 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.649533+0000", 2024-09-16T09:39:56.191 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.002872281 2024-09-16T09:39:56.191 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.191 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.191 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:39:56.191 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.651288+0000", 2024-09-16T09:39:56.191 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0017543979999999999 2024-09-16T09:39:56.191 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.191 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.191 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:39:56.191 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.651495+0000", 2024-09-16T09:39:56.192 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00020767 2024-09-16T09:39:56.192 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.192 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.192 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:39:56.192 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.651661+0000", 2024-09-16T09:39:56.192 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00016588900000000001 2024-09-16T09:39:56.192 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.192 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.192 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:39:56.192 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.652067+0000", 2024-09-16T09:39:56.192 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000405735 2024-09-16T09:39:56.192 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.192 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.193 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:39:56.193 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.652124+0000", 2024-09-16T09:39:56.193 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7484000000000001e-05 2024-09-16T09:39:56.193 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.193 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.193 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:39:56.193 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.652257+0000", 2024-09-16T09:39:56.193 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00013240800000000001 2024-09-16T09:39:56.193 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:39:56.193 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:39:56.193 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:39:56.193 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.193 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.193 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:232734 9.18 9:1cfe7a7c:::benchmark_data_smithi070_137000_object111023:head [delete] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e360)", 2024-09-16T09:39:56.193 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:37:21.522588+0000", 2024-09-16T09:39:56.194 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 154.32515695199999, 2024-09-16T09:39:56.194 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.128298018, 2024-09-16T09:39:56.194 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:39:56.194 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:39:56.194 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:39:56.194 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:39:56.194 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:39:56.194 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:39:56.194 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 232734 2024-09-16T09:39:56.194 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.194 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:39:56.194 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.194 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:39:56.194 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.522588+0000", 2024-09-16T09:39:56.195 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:39:56.195 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.195 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.195 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:39:56.195 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.522588+0000", 2024-09-16T09:39:56.195 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:39:56.195 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.195 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.195 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:39:56.195 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.522589+0000", 2024-09-16T09:39:56.195 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9799999999999999e-07 2024-09-16T09:39:56.195 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.195 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.196 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:39:56.196 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.522591+0000", 2024-09-16T09:39:56.196 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.1799999999999999e-06 2024-09-16T09:39:56.196 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.196 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.196 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:39:56.196 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.522599+0000", 2024-09-16T09:39:56.196 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.388e-06 2024-09-16T09:39:56.196 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.196 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.196 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:39:56.196 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.522605+0000", 2024-09-16T09:39:56.196 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4330000000000003e-06 2024-09-16T09:39:56.196 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.196 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.197 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:39:56.197 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.522717+0000", 2024-09-16T09:39:56.197 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000112147 2024-09-16T09:39:56.197 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.197 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.197 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op must wait for map", 2024-09-16T09:39:56.197 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.522725+0000", 2024-09-16T09:39:56.197 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.259e-06 2024-09-16T09:39:56.197 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.197 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.197 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:39:56.197 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.529317+0000", 2024-09-16T09:39:56.197 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0065921959999999998 2024-09-16T09:39:56.197 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.198 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.198 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:39:56.198 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.529326+0000", 2024-09-16T09:39:56.198 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.738e-06 2024-09-16T09:39:56.198 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.198 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.198 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:39:56.198 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.642752+0000", 2024-09-16T09:39:56.198 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.113426391 2024-09-16T09:39:56.198 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.198 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.198 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:39:56.198 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.642982+0000", 2024-09-16T09:39:56.198 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00022971299999999999 2024-09-16T09:39:56.199 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.199 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.199 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,5,6,7", 2024-09-16T09:39:56.199 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.643102+0000", 2024-09-16T09:39:56.199 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00011989000000000001 2024-09-16T09:39:56.199 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.199 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.199 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:39:56.199 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.650395+0000", 2024-09-16T09:39:56.199 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0072925999999999998 2024-09-16T09:39:56.199 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.199 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.199 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:39:56.199 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.650604+0000", 2024-09-16T09:39:56.199 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000209803 2024-09-16T09:39:56.200 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.200 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.200 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:39:56.200 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.650647+0000", 2024-09-16T09:39:56.200 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2106e-05 2024-09-16T09:39:56.200 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.200 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.200 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:39:56.200 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.650683+0000", 2024-09-16T09:39:56.200 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6665999999999998e-05 2024-09-16T09:39:56.200 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.200 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.200 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:39:56.200 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.650722+0000", 2024-09-16T09:39:56.200 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.858e-05 2024-09-16T09:39:56.201 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.201 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.201 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:39:56.201 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.650778+0000", 2024-09-16T09:39:56.201 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.6115000000000001e-05 2024-09-16T09:39:56.201 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.201 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.201 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:39:56.201 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.650886+0000", 2024-09-16T09:39:56.201 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000108415 2024-09-16T09:39:56.201 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:39:56.201 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:39:56.201 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:39:56.201 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.202 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.202 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:232737 9.15 9:aef95ec3:::benchmark_data_smithi070_137000_object111026:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e360)", 2024-09-16T09:39:56.202 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:37:21.522620+0000", 2024-09-16T09:39:56.202 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 154.32512504300001, 2024-09-16T09:39:56.202 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0972297040000001, 2024-09-16T09:39:56.202 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:39:56.202 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:39:56.202 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:39:56.202 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:39:56.202 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:39:56.202 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:39:56.202 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 232737 2024-09-16T09:39:56.202 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.202 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:39:56.203 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.203 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:39:56.203 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.522620+0000", 2024-09-16T09:39:56.203 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:39:56.203 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.203 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.203 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:39:56.203 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.522620+0000", 2024-09-16T09:39:56.203 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:39:56.203 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.203 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.203 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:39:56.203 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.522621+0000", 2024-09-16T09:39:56.203 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.3799999999999997e-07 2024-09-16T09:39:56.204 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.204 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.204 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:39:56.204 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.522623+0000", 2024-09-16T09:39:56.204 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.2819999999999999e-06 2024-09-16T09:39:56.204 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.204 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.204 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:39:56.204 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.522630+0000", 2024-09-16T09:39:56.204 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.8990000000000004e-06 2024-09-16T09:39:56.204 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.204 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.204 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:39:56.204 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.522635+0000", 2024-09-16T09:39:56.205 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.9949999999999996e-06 2024-09-16T09:39:56.205 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.205 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.205 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:39:56.205 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.523000+0000", 2024-09-16T09:39:56.205 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00036526899999999999 2024-09-16T09:39:56.205 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.205 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.205 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op must wait for map", 2024-09-16T09:39:56.205 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.523007+0000", 2024-09-16T09:39:56.205 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.7950000000000002e-06 2024-09-16T09:39:56.205 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.205 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.205 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:39:56.206 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.532786+0000", 2024-09-16T09:39:56.206 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0097794809999999996 2024-09-16T09:39:56.206 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.206 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.206 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:39:56.206 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.532795+0000", 2024-09-16T09:39:56.206 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.7930000000000003e-06 2024-09-16T09:39:56.206 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.206 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.206 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:39:56.206 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.602050+0000", 2024-09-16T09:39:56.206 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.069254801 2024-09-16T09:39:56.206 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.206 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.206 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:39:56.207 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.602294+0000", 2024-09-16T09:39:56.207 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00024363299999999999 2024-09-16T09:39:56.207 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.207 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.207 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,3,5,7", 2024-09-16T09:39:56.207 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.602437+0000", 2024-09-16T09:39:56.207 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000143201 2024-09-16T09:39:56.207 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.207 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.207 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:39:56.207 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.603959+0000", 2024-09-16T09:39:56.207 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001521969 2024-09-16T09:39:56.207 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.207 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.208 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:39:56.208 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.604003+0000", 2024-09-16T09:39:56.208 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.4097000000000001e-05 2024-09-16T09:39:56.208 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.208 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.208 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:39:56.208 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.604403+0000", 2024-09-16T09:39:56.208 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00039965200000000002 2024-09-16T09:39:56.208 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.208 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.208 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:39:56.208 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.619638+0000", 2024-09-16T09:39:56.208 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.01523505 2024-09-16T09:39:56.209 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.209 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.209 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:39:56.209 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.619676+0000", 2024-09-16T09:39:56.209 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8154000000000001e-05 2024-09-16T09:39:56.209 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.209 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.209 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:39:56.209 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.619714+0000", 2024-09-16T09:39:56.209 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7957999999999998e-05 2024-09-16T09:39:56.209 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.209 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.209 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:39:56.209 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.619850+0000", 2024-09-16T09:39:56.210 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00013613699999999999 2024-09-16T09:39:56.210 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:39:56.210 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:39:56.210 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:39:56.210 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.210 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.210 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:232755 9.19 9:9c3a2274:::benchmark_data_smithi070_137000_object111044:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e360)", 2024-09-16T09:39:56.210 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:37:21.522649+0000", 2024-09-16T09:39:56.210 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 154.325096113, 2024-09-16T09:39:56.210 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.141048657, 2024-09-16T09:39:56.210 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:39:56.210 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:39:56.210 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:39:56.210 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:39:56.211 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:39:56.211 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:39:56.211 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 232755 2024-09-16T09:39:56.211 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.211 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:39:56.211 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.211 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:39:56.211 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.522649+0000", 2024-09-16T09:39:56.211 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:39:56.211 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.211 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.211 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:39:56.211 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.522649+0000", 2024-09-16T09:39:56.211 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:39:56.212 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.212 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.212 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:39:56.212 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.522650+0000", 2024-09-16T09:39:56.212 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9999999999999998e-07 2024-09-16T09:39:56.212 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.212 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.212 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:39:56.212 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.522652+0000", 2024-09-16T09:39:56.212 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0980000000000001e-06 2024-09-16T09:39:56.212 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.212 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.212 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:39:56.212 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.522659+0000", 2024-09-16T09:39:56.213 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.8990000000000004e-06 2024-09-16T09:39:56.213 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.213 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.213 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:39:56.213 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.522664+0000", 2024-09-16T09:39:56.213 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.169e-06 2024-09-16T09:39:56.213 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.213 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.213 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:39:56.213 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.522805+0000", 2024-09-16T09:39:56.213 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00014155999999999999 2024-09-16T09:39:56.213 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.213 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.213 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting_for_map not empty", 2024-09-16T09:39:56.214 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.522810+0000", 2024-09-16T09:39:56.214 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2429999999999999e-06 2024-09-16T09:39:56.214 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.214 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.214 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:39:56.214 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.528559+0000", 2024-09-16T09:39:56.214 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0057496800000000001 2024-09-16T09:39:56.214 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.214 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.214 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:39:56.214 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.528565+0000", 2024-09-16T09:39:56.214 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.054e-06 2024-09-16T09:39:56.214 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.214 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.215 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:39:56.215 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.648010+0000", 2024-09-16T09:39:56.215 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1194449900000001 2024-09-16T09:39:56.215 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.215 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.215 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:39:56.215 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.648126+0000", 2024-09-16T09:39:56.215 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00011561900000000001 2024-09-16T09:39:56.215 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.215 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.215 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 3,5,6,7", 2024-09-16T09:39:56.215 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.648246+0000", 2024-09-16T09:39:56.215 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00012061399999999999 2024-09-16T09:39:56.215 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.215 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.216 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:39:56.216 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.649539+0000", 2024-09-16T09:39:56.216 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001292587 2024-09-16T09:39:56.216 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.216 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.216 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:39:56.216 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.651534+0000", 2024-09-16T09:39:56.216 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001994699 2024-09-16T09:39:56.216 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.216 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.216 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:39:56.216 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.651708+0000", 2024-09-16T09:39:56.216 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000174298 2024-09-16T09:39:56.216 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.217 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.217 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:39:56.217 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.651819+0000", 2024-09-16T09:39:56.217 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00011064400000000001 2024-09-16T09:39:56.217 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.217 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.217 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:39:56.217 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.663512+0000", 2024-09-16T09:39:56.217 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.011693301 2024-09-16T09:39:56.217 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.217 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.217 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:39:56.217 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.663579+0000", 2024-09-16T09:39:56.217 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.6544999999999994e-05 2024-09-16T09:39:56.218 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.218 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.218 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:39:56.218 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.663698+0000", 2024-09-16T09:39:56.218 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000119257 2024-09-16T09:39:56.218 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:39:56.218 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:39:56.218 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:39:56.218 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.218 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.218 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:232771 9.15 9:aa0978e0:::benchmark_data_smithi070_137000_object111060:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e360)", 2024-09-16T09:39:56.218 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:37:21.557486+0000", 2024-09-16T09:39:56.218 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 154.29025914499999, 2024-09-16T09:39:56.218 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0625502760000001, 2024-09-16T09:39:56.219 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:39:56.219 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:39:56.219 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:39:56.219 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:39:56.219 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:39:56.219 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:39:56.219 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 232771 2024-09-16T09:39:56.219 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.219 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:39:56.219 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.219 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:39:56.219 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.557486+0000", 2024-09-16T09:39:56.219 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:39:56.219 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.220 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.220 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:39:56.220 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.557486+0000", 2024-09-16T09:39:56.220 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:39:56.220 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.220 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.220 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:39:56.220 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.557488+0000", 2024-09-16T09:39:56.220 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.733e-06 2024-09-16T09:39:56.220 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.220 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.220 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:39:56.220 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.557495+0000", 2024-09-16T09:39:56.220 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.5800000000000003e-06 2024-09-16T09:39:56.221 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.221 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.221 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:39:56.221 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.557514+0000", 2024-09-16T09:39:56.221 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.8436000000000001e-05 2024-09-16T09:39:56.221 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.221 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.221 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:39:56.221 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.557521+0000", 2024-09-16T09:39:56.221 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.2899999999999997e-06 2024-09-16T09:39:56.221 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.221 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.221 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:39:56.221 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.557584+0000", 2024-09-16T09:39:56.222 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.3122999999999999e-05 2024-09-16T09:39:56.222 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.222 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.222 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:39:56.222 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.557596+0000", 2024-09-16T09:39:56.222 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1474000000000001e-05 2024-09-16T09:39:56.222 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.222 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.222 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:39:56.222 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.603185+0000", 2024-09-16T09:39:56.222 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.045589487 2024-09-16T09:39:56.222 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.222 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.222 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:39:56.222 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.603270+0000", 2024-09-16T09:39:56.223 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.4728999999999995e-05 2024-09-16T09:39:56.223 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.223 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.223 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,3,5,7", 2024-09-16T09:39:56.223 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.603399+0000", 2024-09-16T09:39:56.223 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00012893599999999999 2024-09-16T09:39:56.223 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.223 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.223 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:39:56.223 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.604344+0000", 2024-09-16T09:39:56.223 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00094472899999999995 2024-09-16T09:39:56.223 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.223 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.223 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:39:56.224 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.604641+0000", 2024-09-16T09:39:56.224 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00029712100000000002 2024-09-16T09:39:56.224 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.224 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.224 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:39:56.224 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.605004+0000", 2024-09-16T09:39:56.224 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00036339199999999998 2024-09-16T09:39:56.224 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.224 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.224 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:39:56.224 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.619669+0000", 2024-09-16T09:39:56.224 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.014664877999999999 2024-09-16T09:39:56.224 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.224 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.225 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:39:56.225 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.619860+0000", 2024-09-16T09:39:56.225 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000191223 2024-09-16T09:39:56.225 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.225 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.225 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:39:56.225 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.619914+0000", 2024-09-16T09:39:56.225 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4163999999999998e-05 2024-09-16T09:39:56.225 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:39:56.228 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:39:56.228 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:39:56.228 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.620036+0000", 2024-09-16T09:39:56.228 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000121981 2024-09-16T09:39:56.228 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:39:56.229 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:39:56.229 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:39:56.229 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:39:56.229 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:39:56.229 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T09:39:56.229 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:39:56.263 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:56.263 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:56.264 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:39:56.365 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:39:56.400 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:56.401 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:56.401 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:39:56.502 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:39:56.536 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:56.536 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:56.537 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:39:56.638 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:39:56.668 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:56.668 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:56.669 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:39:56.770 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:39:56.813 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:56.813 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:56.814 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:39:56.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:56 smithi084 ceph-mon[32629]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:39:56.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:56 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-16T09:39:56.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:56 smithi084 ceph-mon[32629]: osdmap e428: 8 total, 8 up, 8 in 2024-09-16T09:39:56.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:56 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:39:56.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:56 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:39:56.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:56 smithi084 ceph-mon[32629]: pgmap v790: 178 pgs: 1 active+recovery_wait, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 173 active+clean; 5.3 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 933 op/s; 17049/261426 objects misplaced (6.522%); 6.6 MiB/s, 105 objects/s recovering 2024-09-16T09:39:56.915 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:39:56.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:56 smithi070 ceph-mon[30388]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:39:56.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:56 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-16T09:39:56.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:56 smithi070 ceph-mon[30388]: osdmap e428: 8 total, 8 up, 8 in 2024-09-16T09:39:56.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:56 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:39:56.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:56 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:39:56.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:56 smithi070 ceph-mon[30388]: pgmap v790: 178 pgs: 1 active+recovery_wait, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 173 active+clean; 5.3 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 933 op/s; 17049/261426 objects misplaced (6.522%); 6.6 MiB/s, 105 objects/s recovering 2024-09-16T09:39:56.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:56 smithi070 ceph-mon[40691]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:39:56.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:56 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-16T09:39:56.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:56 smithi070 ceph-mon[40691]: osdmap e428: 8 total, 8 up, 8 in 2024-09-16T09:39:56.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:56 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:39:56.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:56 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:39:56.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:56 smithi070 ceph-mon[40691]: pgmap v790: 178 pgs: 1 active+recovery_wait, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 173 active+clean; 5.3 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 933 op/s; 17049/261426 objects misplaced (6.522%); 6.6 MiB/s, 105 objects/s recovering 2024-09-16T09:39:56.944 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:56.945 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:56.945 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:39:57.046 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:39:57.069 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:57.070 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:57.070 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:39:57.171 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:39:57.210 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:57.210 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:57.211 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:39:57.312 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:39:57.351 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:57.351 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:57.352 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:39:57.452 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:39:57.492 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:57.493 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:57.493 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:39:57.531 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-16T09:39:57.532 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-16T09:39:57.532 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph tell osd.3 'trim stale osdmaps' 2024-09-16T09:39:57.595 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:39:57.624 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:57.624 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:57.625 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:39:57.717 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:57 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:39:57.717 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:57 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:39:57.717 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:57 smithi070 ceph-mon[30388]: osdmap e429: 8 total, 8 up, 8 in 2024-09-16T09:39:57.718 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:57 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:39:57.718 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:57 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:39:57.718 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:57 smithi070 ceph-mon[40691]: osdmap e429: 8 total, 8 up, 8 in 2024-09-16T09:39:57.725 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:39:57.793 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:57.794 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:57.794 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:39:57.895 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:39:57.923 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:57.924 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:57.924 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:39:58.025 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:39:58.068 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:58.068 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:58.068 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:39:58.111 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.1 dump_ops_in_flight 2024-09-16T09:39:58.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:57 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:39:58.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:57 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:39:58.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:57 smithi084 ceph-mon[32629]: osdmap e429: 8 total, 8 up, 8 in 2024-09-16T09:39:58.170 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:39:58.226 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:58.226 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:58.227 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:39:58.328 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:39:58.380 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:58.381 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:58.381 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:39:58.482 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:39:58.487 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:39:58.571 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:58.571 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:58.571 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:39:58.610 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:39:58.672 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:39:58.702 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:58.703 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:58.703 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:39:58.805 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:39:58.817 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:58 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:39:58.818 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:58 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:39:58.818 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:58 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:39:58.818 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:39:58 smithi070 ceph-mon[30388]: pgmap v792: 178 pgs: 1 active+recovery_wait, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 173 active+clean; 5.3 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 978 B/s rd, 0 B/s wr, 1.07k op/s; 17049/261426 objects misplaced (6.522%); 7.5 MiB/s, 120 objects/s recovering 2024-09-16T09:39:58.818 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:58 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:39:58.818 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:58 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:39:58.818 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:58 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:39:58.819 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:39:58 smithi070 ceph-mon[40691]: pgmap v792: 178 pgs: 1 active+recovery_wait, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 173 active+clean; 5.3 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 978 B/s rd, 0 B/s wr, 1.07k op/s; 17049/261426 objects misplaced (6.522%); 7.5 MiB/s, 120 objects/s recovering 2024-09-16T09:39:58.835 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:58.836 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:58.836 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:39:58.937 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:39:58.969 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:58.969 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:58.970 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:39:59.071 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:39:59.101 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:59.102 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:59.103 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:39:59.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:58 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:39:59.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:58 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:39:59.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:58 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:39:59.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:39:58 smithi084 ceph-mon[32629]: pgmap v792: 178 pgs: 1 active+recovery_wait, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 173 active+clean; 5.3 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 978 B/s rd, 0 B/s wr, 1.07k op/s; 17049/261426 objects misplaced (6.522%); 7.5 MiB/s, 120 objects/s recovering 2024-09-16T09:39:59.204 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:39:59.242 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:59.242 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:59.243 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:39:59.344 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:39:59.383 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:59.383 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:59.384 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:39:59.484 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:39:59.527 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:59.528 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:59.529 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:39:59.629 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:39:59.663 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:59.664 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:59.664 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:39:59.765 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:39:59.799 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:39:59.800 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:59.801 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:39:59.902 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:39:59.934 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:39:59.935 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:39:59.935 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:40:00.036 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:40:00.076 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:00.078 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:00.078 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:40:00.179 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:40:00.213 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:00.213 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:00.214 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:40:00.315 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:40:00.352 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:00.353 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:00.353 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:40:00.454 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:40:00.493 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:00.494 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:00.494 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:40:00.545 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-16T09:40:00.595 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:40:00.641 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:00.642 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:00.642 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:40:00.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:00 smithi084 ceph-mon[32629]: Health detail: HEALTH_WARN noscrub flag(s) set; Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull; Degraded data redundancy: 3 pgs undersized 2024-09-16T09:40:00.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:00 smithi084 ceph-mon[32629]: [WRN] OSDMAP_FLAGS: noscrub flag(s) set 2024-09-16T09:40:00.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:00 smithi084 ceph-mon[32629]: [WRN] PG_BACKFILL_FULL: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull 2024-09-16T09:40:00.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:00 smithi084 ceph-mon[32629]: pg 10.12 is active+remapped+backfill_toofull, acting [0,1,4] 2024-09-16T09:40:00.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:00 smithi084 ceph-mon[32629]: [WRN] PG_DEGRADED: Degraded data redundancy: 3 pgs undersized 2024-09-16T09:40:00.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:00 smithi084 ceph-mon[32629]: pg 10.14 is stuck undersized for 76s, current state active+recovering+undersized+remapped, last acting [4,1] 2024-09-16T09:40:00.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:00 smithi084 ceph-mon[32629]: pg 10.15 is stuck undersized for 79s, current state active+recovering+undersized+remapped, last acting [3,1] 2024-09-16T09:40:00.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:00 smithi084 ceph-mon[32629]: pg 10.19 is stuck undersized for 79s, current state active+recovering+undersized+remapped, last acting [2,1] 2024-09-16T09:40:00.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:00 smithi070 ceph-mon[40691]: Health detail: HEALTH_WARN noscrub flag(s) set; Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull; Degraded data redundancy: 3 pgs undersized 2024-09-16T09:40:00.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:00 smithi070 ceph-mon[40691]: [WRN] OSDMAP_FLAGS: noscrub flag(s) set 2024-09-16T09:40:00.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:00 smithi070 ceph-mon[40691]: [WRN] PG_BACKFILL_FULL: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull 2024-09-16T09:40:00.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:00 smithi070 ceph-mon[40691]: pg 10.12 is active+remapped+backfill_toofull, acting [0,1,4] 2024-09-16T09:40:00.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:00 smithi070 ceph-mon[40691]: [WRN] PG_DEGRADED: Degraded data redundancy: 3 pgs undersized 2024-09-16T09:40:00.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:00 smithi070 ceph-mon[40691]: pg 10.14 is stuck undersized for 76s, current state active+recovering+undersized+remapped, last acting [4,1] 2024-09-16T09:40:00.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:00 smithi070 ceph-mon[40691]: pg 10.15 is stuck undersized for 79s, current state active+recovering+undersized+remapped, last acting [3,1] 2024-09-16T09:40:00.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:00 smithi070 ceph-mon[40691]: pg 10.19 is stuck undersized for 79s, current state active+recovering+undersized+remapped, last acting [2,1] 2024-09-16T09:40:00.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:00 smithi070 ceph-mon[30388]: Health detail: HEALTH_WARN noscrub flag(s) set; Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull; Degraded data redundancy: 3 pgs undersized 2024-09-16T09:40:00.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:00 smithi070 ceph-mon[30388]: [WRN] OSDMAP_FLAGS: noscrub flag(s) set 2024-09-16T09:40:00.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:00 smithi070 ceph-mon[30388]: [WRN] PG_BACKFILL_FULL: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull 2024-09-16T09:40:00.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:00 smithi070 ceph-mon[30388]: pg 10.12 is active+remapped+backfill_toofull, acting [0,1,4] 2024-09-16T09:40:00.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:00 smithi070 ceph-mon[30388]: [WRN] PG_DEGRADED: Degraded data redundancy: 3 pgs undersized 2024-09-16T09:40:00.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:00 smithi070 ceph-mon[30388]: pg 10.14 is stuck undersized for 76s, current state active+recovering+undersized+remapped, last acting [4,1] 2024-09-16T09:40:00.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:00 smithi070 ceph-mon[30388]: pg 10.15 is stuck undersized for 79s, current state active+recovering+undersized+remapped, last acting [3,1] 2024-09-16T09:40:00.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:00 smithi070 ceph-mon[30388]: pg 10.19 is stuck undersized for 79s, current state active+recovering+undersized+remapped, last acting [2,1] 2024-09-16T09:40:00.744 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:40:00.795 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:00.796 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:00.796 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:40:00.897 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:40:00.942 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:00.942 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:00.943 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:40:01.043 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:40:01.079 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:01.080 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:01.080 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:40:01.181 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:40:01.210 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:01.211 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:01.212 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:40:01.312 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:40:01.341 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:01.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:01.342 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:40:01.443 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:40:01.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:01.465 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:01.465 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:40:01.566 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:40:01.590 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:01.591 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:01.591 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:40:01.627 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:01 smithi070 ceph-mon[30388]: pgmap v793: 178 pgs: 1 active+recovery_wait, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 173 active+clean; 5.0 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 822 B/s rd, 0 B/s wr, 966 op/s; 14802/246180 objects misplaced (6.013%); 10 MiB/s, 160 objects/s recovering 2024-09-16T09:40:01.628 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:01 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1028681739' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:40:01.628 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:01 smithi070 ceph-mon[40691]: pgmap v793: 178 pgs: 1 active+recovery_wait, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 173 active+clean; 5.0 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 822 B/s rd, 0 B/s wr, 966 op/s; 14802/246180 objects misplaced (6.013%); 10 MiB/s, 160 objects/s recovering 2024-09-16T09:40:01.628 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:01 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1028681739' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:40:01.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:01 smithi084 ceph-mon[32629]: pgmap v793: 178 pgs: 1 active+recovery_wait, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 173 active+clean; 5.0 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 822 B/s rd, 0 B/s wr, 966 op/s; 14802/246180 objects misplaced (6.013%); 10 MiB/s, 160 objects/s recovering 2024-09-16T09:40:01.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:01 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1028681739' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:40:01.692 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:40:01.734 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:01.735 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:01.735 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:40:01.836 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:40:01.857 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:01.858 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:01.858 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:40:01.959 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:40:01.993 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:01.994 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:01.994 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:40:02.095 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:40:02.128 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:02.128 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:02.128 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:40:02.229 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:40:02.262 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:02.263 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:02.263 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:40:02.363 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:40:02.397 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:02.398 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:02.398 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:40:02.420 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:40:01 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:40:01] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:40:02.499 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:40:02.514 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:40:02.514 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:40:02.514 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:40:02.514 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:40:02.514 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:40:02.514 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:40:02.515 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:40:02.515 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:40:02.515 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:40:02.515 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:40:02.515 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:40:02.515 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:40:02.515 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:40:02.515 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:40:02.515 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:40:02.515 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:40:02.545 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:02.546 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:02.547 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:40:02.647 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:40:02.687 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:02.688 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:02.688 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:40:02.789 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:40:02.825 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:02.826 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:02.826 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:40:02.927 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:40:02.951 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:02.952 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:02.952 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:40:03.053 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:40:03.085 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:03.085 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:03.086 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:40:03.187 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:40:03.222 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:03.223 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:03.223 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:40:03.324 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:40:03.360 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:03.361 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:03.361 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:40:03.462 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:40:03.495 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:03.496 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:03.496 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:40:03.597 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:40:03.633 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:03.634 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:03.634 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:40:03.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:03 smithi084 ceph-mon[32629]: pgmap v794: 178 pgs: 1 active+recovery_wait, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 173 active+clean; 5.0 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 728 op/s; 14802/246180 objects misplaced (6.013%); 6.1 MiB/s, 96 objects/s recovering 2024-09-16T09:40:03.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:03 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:40:03.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:03 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:40:03.661 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:40:03 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:40:03.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:03 smithi070 ceph-mon[30388]: pgmap v794: 178 pgs: 1 active+recovery_wait, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 173 active+clean; 5.0 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 728 op/s; 14802/246180 objects misplaced (6.013%); 6.1 MiB/s, 96 objects/s recovering 2024-09-16T09:40:03.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:03 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:40:03.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:03 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:40:03.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:03 smithi070 ceph-mon[40691]: pgmap v794: 178 pgs: 1 active+recovery_wait, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 173 active+clean; 5.0 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 728 op/s; 14802/246180 objects misplaced (6.013%); 6.1 MiB/s, 96 objects/s recovering 2024-09-16T09:40:03.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:03 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:40:03.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:03 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:40:03.735 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:40:03.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:03.774 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:03.774 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:40:03.875 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:40:03.910 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:03.911 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:03.911 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:40:04.012 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:40:04.041 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:04.042 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:04.042 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:40:04.143 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:40:04.175 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:04.176 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:04.177 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:40:04.277 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:40:04.298 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:04.299 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:04.299 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:40:04.400 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:40:04.434 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:40:04.458 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:04.460 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:04.460 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:40:04.561 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:40:04.601 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:04.602 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:04.602 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:40:04.703 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:40:04.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:04.728 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:04.728 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:40:04.829 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:40:04.864 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:04.865 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:04.865 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:40:04.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:04 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:40:04.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:04 smithi084 ceph-mon[32629]: osdmap e430: 8 total, 8 up, 8 in 2024-09-16T09:40:04.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:04 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:40:04.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:04 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:40:04.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:04 smithi084 ceph-mon[32629]: osdmap e431: 8 total, 8 up, 8 in 2024-09-16T09:40:04.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:04 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:40:04.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:04 smithi070 ceph-mon[30388]: osdmap e430: 8 total, 8 up, 8 in 2024-09-16T09:40:04.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:04 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:40:04.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:04 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:40:04.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:04 smithi070 ceph-mon[30388]: osdmap e431: 8 total, 8 up, 8 in 2024-09-16T09:40:04.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:04 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:40:04.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:04 smithi070 ceph-mon[40691]: osdmap e430: 8 total, 8 up, 8 in 2024-09-16T09:40:04.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:04 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:40:04.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:04 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:40:04.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:04 smithi070 ceph-mon[40691]: osdmap e431: 8 total, 8 up, 8 in 2024-09-16T09:40:04.966 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:40:04.999 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:05.000 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:05.000 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:40:05.101 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:40:05.135 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:05.136 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:05.136 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:40:05.237 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:40:05.274 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:05.275 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:05.275 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:40:05.376 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:40:05.410 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:05.411 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:05.411 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:40:05.512 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:40:05.547 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:05.548 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:05.548 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:40:05.649 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:40:05.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:05 smithi070 ceph-mon[40691]: pgmap v796: 178 pgs: 1 active+recovery_wait, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 173 active+clean; 4.7 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.23k op/s; 13112/231978 objects misplaced (5.652%); 12 MiB/s, 190 objects/s recovering 2024-09-16T09:40:05.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:05 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:40:05.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:05 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:40:05.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:05 smithi070 ceph-mon[30388]: pgmap v796: 178 pgs: 1 active+recovery_wait, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 173 active+clean; 4.7 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.23k op/s; 13112/231978 objects misplaced (5.652%); 12 MiB/s, 190 objects/s recovering 2024-09-16T09:40:05.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:05 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:40:05.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:05 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:40:05.687 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:05.688 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:05.689 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:40:05.789 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:40:05.821 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:05.822 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:05.822 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:40:05.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:05 smithi084 ceph-mon[32629]: pgmap v796: 178 pgs: 1 active+recovery_wait, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 173 active+clean; 4.7 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.23k op/s; 13112/231978 objects misplaced (5.652%); 12 MiB/s, 190 objects/s recovering 2024-09-16T09:40:05.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:05 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:40:05.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:05 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:40:05.923 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:40:05.956 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:05.957 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:05.957 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:40:06.058 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:40:06.092 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:06.093 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:06.093 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:40:06.194 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:40:06.224 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:06.225 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:06.225 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:40:06.326 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:40:06.368 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:06.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:06.371 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:40:06.472 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:40:06.499 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:06.500 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:06.500 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:40:06.601 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:40:06.648 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:06.648 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:06.649 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:40:06.703 INFO:teuthology.orchestra.run.smithi070.stderr:no valid command found; 10 closest matches: 2024-09-16T09:40:06.703 INFO:teuthology.orchestra.run.smithi070.stderr:0 2024-09-16T09:40:06.703 INFO:teuthology.orchestra.run.smithi070.stderr:1 2024-09-16T09:40:06.703 INFO:teuthology.orchestra.run.smithi070.stderr:2 2024-09-16T09:40:06.703 INFO:teuthology.orchestra.run.smithi070.stderr:abort 2024-09-16T09:40:06.703 INFO:teuthology.orchestra.run.smithi070.stderr:assert 2024-09-16T09:40:06.703 INFO:teuthology.orchestra.run.smithi070.stderr:bench [] [] [] [] 2024-09-16T09:40:06.703 INFO:teuthology.orchestra.run.smithi070.stderr:bluefs debug_inject_read_zeros 2024-09-16T09:40:06.704 INFO:teuthology.orchestra.run.smithi070.stderr:bluefs files list 2024-09-16T09:40:06.704 INFO:teuthology.orchestra.run.smithi070.stderr:bluefs stats 2024-09-16T09:40:06.704 INFO:teuthology.orchestra.run.smithi070.stderr:bluestore allocator dump block 2024-09-16T09:40:06.704 INFO:teuthology.orchestra.run.smithi070.stderr:Error EINVAL: invalid command 2024-09-16T09:40:06.750 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:40:06.786 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:06.787 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:06.787 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:40:06.888 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:40:06.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:06 smithi084 ceph-mon[32629]: osdmap e432: 8 total, 8 up, 8 in 2024-09-16T09:40:06.920 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:06.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:06 smithi070 ceph-mon[30388]: osdmap e432: 8 total, 8 up, 8 in 2024-09-16T09:40:06.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:06 smithi070 ceph-mon[40691]: osdmap e432: 8 total, 8 up, 8 in 2024-09-16T09:40:06.921 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:06.922 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:40:07.022 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:40:07.060 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:07.060 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:07.061 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:40:07.161 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:40:07.196 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:07.197 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:07.197 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:40:07.298 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:40:07.335 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:07.336 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:07.336 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:40:07.434 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-16T09:40:07.434 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-16T09:40:07.435 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph tell osd.4 'trim stale osdmaps' 2024-09-16T09:40:07.437 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:40:07.482 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:07.485 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:07.485 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:40:07.586 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:40:07.613 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:07.613 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:07.614 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:40:07.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:07 smithi070 ceph-mon[30388]: pgmap v799: 178 pgs: 1 active+recovery_wait, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 173 active+clean; 4.7 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 903 op/s; 12584/229926 objects misplaced (5.473%); 7.8 MiB/s, 125 objects/s recovering 2024-09-16T09:40:07.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:07 smithi070 ceph-mon[40691]: pgmap v799: 178 pgs: 1 active+recovery_wait, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 173 active+clean; 4.7 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 903 op/s; 12584/229926 objects misplaced (5.473%); 7.8 MiB/s, 125 objects/s recovering 2024-09-16T09:40:07.714 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:40:07.756 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T09:40:07.756 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:40:07.776 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:07.777 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:07.777 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:40:07.878 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:40:07.898 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T09:40:07.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:07 smithi084 ceph-mon[32629]: pgmap v799: 178 pgs: 1 active+recovery_wait, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 173 active+clean; 4.7 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 903 op/s; 12584/229926 objects misplaced (5.473%); 7.8 MiB/s, 125 objects/s recovering 2024-09-16T09:40:07.916 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:07.916 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:07.916 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:40:08.018 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:40:08.075 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:08.075 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:08.075 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:40:08.176 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:40:08.219 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:08.219 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:08.219 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:40:08.320 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:40:08.354 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:08.355 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:08.355 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:40:08.434 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:40:08.456 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:40:08.503 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:08.503 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:08.503 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:40:08.604 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:40:08.636 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:08.636 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:08.637 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:40:08.737 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:40:08.775 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:08.776 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:08.776 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:40:08.877 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:40:08.910 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:08.911 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:08.911 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:40:09.012 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:40:09.040 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:09.041 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:09.041 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:40:09.142 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:40:09.185 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:09.186 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:09.186 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:40:09.287 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:40:09.321 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:09.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:09.322 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:40:09.423 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:40:09.470 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:09.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:09.471 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:40:09.572 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:40:09.605 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:09.606 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:09.606 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:40:09.707 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:40:09.745 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:09.746 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:09.746 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:40:09.847 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:40:09.881 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:09.882 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:09.882 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:40:09.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:09 smithi084 ceph-mon[32629]: pgmap v800: 178 pgs: 1 active+recovery_wait, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 173 active+clean; 4.7 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 903 op/s; 12584/229926 objects misplaced (5.473%); 7.8 MiB/s, 125 objects/s recovering 2024-09-16T09:40:09.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:09 smithi084 ceph-mon[32629]: osdmap e433: 8 total, 8 up, 8 in 2024-09-16T09:40:09.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:09 smithi070 ceph-mon[30388]: pgmap v800: 178 pgs: 1 active+recovery_wait, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 173 active+clean; 4.7 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 903 op/s; 12584/229926 objects misplaced (5.473%); 7.8 MiB/s, 125 objects/s recovering 2024-09-16T09:40:09.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:09 smithi070 ceph-mon[30388]: osdmap e433: 8 total, 8 up, 8 in 2024-09-16T09:40:09.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:09 smithi070 ceph-mon[40691]: pgmap v800: 178 pgs: 1 active+recovery_wait, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 173 active+clean; 4.7 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 903 op/s; 12584/229926 objects misplaced (5.473%); 7.8 MiB/s, 125 objects/s recovering 2024-09-16T09:40:09.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:09 smithi070 ceph-mon[40691]: osdmap e433: 8 total, 8 up, 8 in 2024-09-16T09:40:09.983 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:40:10.020 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:10.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:10.021 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:40:10.122 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:40:10.153 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:10.154 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:10.154 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:40:10.255 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:40:10.290 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:10.291 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:10.291 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:40:10.392 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:40:10.416 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:10.417 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:10.417 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:40:10.518 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:40:10.558 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:10.558 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:10.559 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:40:10.661 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:40:10.690 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:10.691 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:10.691 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:40:10.792 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:40:10.818 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:10.819 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:10.819 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:40:10.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:10 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:40:10.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:10 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:40:10.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:10 smithi084 ceph-mon[32629]: 10.16 scrub starts 2024-09-16T09:40:10.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:10 smithi084 ceph-mon[32629]: osdmap e434: 8 total, 8 up, 8 in 2024-09-16T09:40:10.920 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:40:10.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:10 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:40:10.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:10 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:40:10.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:10 smithi070 ceph-mon[30388]: 10.16 scrub starts 2024-09-16T09:40:10.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:10 smithi070 ceph-mon[30388]: osdmap e434: 8 total, 8 up, 8 in 2024-09-16T09:40:10.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:10 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:40:10.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:10 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:40:10.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:10 smithi070 ceph-mon[40691]: 10.16 scrub starts 2024-09-16T09:40:10.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:10 smithi070 ceph-mon[40691]: osdmap e434: 8 total, 8 up, 8 in 2024-09-16T09:40:10.951 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:10.952 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:10.953 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:40:11.053 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:40:11.074 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:11.075 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:11.075 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:40:11.177 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:40:11.201 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:11.201 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:11.202 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:40:11.303 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:40:11.326 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:11.327 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:11.328 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:40:11.428 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:40:11.451 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:11.451 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:11.451 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:40:11.552 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:40:11.589 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:11.589 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:11.590 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:40:11.691 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:40:11.729 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:11.730 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:11.731 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:40:11.831 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:40:11.869 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:11.870 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:11.870 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:40:11.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:11 smithi084 ceph-mon[32629]: pgmap v802: 178 pgs: 1 active+recovery_wait, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 173 active+clean; 4.3 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.06k op/s; 10028/212931 objects misplaced (4.710%); 9.4 MiB/s, 149 objects/s recovering 2024-09-16T09:40:11.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:11 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:40:11.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:11 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:40:11.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:11 smithi084 ceph-mon[32629]: osdmap e435: 8 total, 8 up, 8 in 2024-09-16T09:40:11.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:11 smithi070 ceph-mon[30388]: pgmap v802: 178 pgs: 1 active+recovery_wait, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 173 active+clean; 4.3 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.06k op/s; 10028/212931 objects misplaced (4.710%); 9.4 MiB/s, 149 objects/s recovering 2024-09-16T09:40:11.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:11 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:40:11.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:11 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:40:11.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:11 smithi070 ceph-mon[30388]: osdmap e435: 8 total, 8 up, 8 in 2024-09-16T09:40:11.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:11 smithi070 ceph-mon[40691]: pgmap v802: 178 pgs: 1 active+recovery_wait, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 173 active+clean; 4.3 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.06k op/s; 10028/212931 objects misplaced (4.710%); 9.4 MiB/s, 149 objects/s recovering 2024-09-16T09:40:11.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:11 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:40:11.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:11 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:40:11.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:11 smithi070 ceph-mon[40691]: osdmap e435: 8 total, 8 up, 8 in 2024-09-16T09:40:11.972 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:40:12.008 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:12.009 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:12.010 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:40:12.110 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:40:12.151 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:12.151 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:12.152 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:40:12.252 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:40:12.288 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:12.288 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:12.289 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:40:12.390 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:40:12.421 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:40:11 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:40:11] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:40:12.423 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:12.424 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:12.424 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:40:12.525 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:40:12.562 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:12.563 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:12.563 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:40:12.664 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:40:12.695 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:12.696 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:12.696 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:40:12.797 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:40:12.827 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:12.828 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:12.828 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:40:12.929 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:40:12.955 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:12.956 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:12.956 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:40:13.057 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:40:13.078 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:13.078 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:13.079 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:40:13.136 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.1 dump_blocked_ops 2024-09-16T09:40:13.180 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:40:13.204 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:13.204 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:13.205 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:40:13.306 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:40:13.327 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:13.328 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:13.329 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:40:13.429 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:40:13.453 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:13.453 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:13.454 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:40:13.554 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:40:13.578 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:13.579 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:13.579 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:40:13.675 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:40:13.676 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:40:13.680 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:40:13.708 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:13.709 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:13.709 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:40:13.749 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:40:13.810 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:40:13.837 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:13 smithi070 ceph-mon[40691]: 10.15 scrub starts 2024-09-16T09:40:13.837 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:13 smithi070 ceph-mon[40691]: pgmap v805: 178 pgs: 1 active+recovery_wait, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 173 active+clean; 4.3 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 944 op/s; 10028/212931 objects misplaced (4.710%); 9.4 MiB/s, 149 objects/s recovering 2024-09-16T09:40:13.837 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:13 smithi070 ceph-mon[40691]: osdmap e436: 8 total, 8 up, 8 in 2024-09-16T09:40:13.837 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:13 smithi070 ceph-mon[30388]: 10.15 scrub starts 2024-09-16T09:40:13.837 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:13 smithi070 ceph-mon[30388]: pgmap v805: 178 pgs: 1 active+recovery_wait, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 173 active+clean; 4.3 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 944 op/s; 10028/212931 objects misplaced (4.710%); 9.4 MiB/s, 149 objects/s recovering 2024-09-16T09:40:13.837 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:13 smithi070 ceph-mon[30388]: osdmap e436: 8 total, 8 up, 8 in 2024-09-16T09:40:13.841 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:13.842 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:13.843 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:40:13.910 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:40:13 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:40:13.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:13 smithi084 ceph-mon[32629]: 10.15 scrub starts 2024-09-16T09:40:13.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:13 smithi084 ceph-mon[32629]: pgmap v805: 178 pgs: 1 active+recovery_wait, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 173 active+clean; 4.3 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 944 op/s; 10028/212931 objects misplaced (4.710%); 9.4 MiB/s, 149 objects/s recovering 2024-09-16T09:40:13.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:13 smithi084 ceph-mon[32629]: osdmap e436: 8 total, 8 up, 8 in 2024-09-16T09:40:13.944 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:40:13.990 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:13.991 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:13.992 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:40:14.092 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:40:14.127 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:14.128 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:14.128 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:40:14.229 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:40:14.265 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:14.266 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:14.266 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:40:14.367 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:40:14.404 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:14.404 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:14.405 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:40:14.505 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:40:14.539 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:14.540 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:14.540 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:40:14.577 INFO:teuthology.orchestra.run.smithi070.stderr:nodeep-scrub is set 2024-09-16T09:40:14.641 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:40:14.677 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:14.678 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:14.678 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:40:14.779 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:40:14.814 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:14.815 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:14.815 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:40:14.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:14 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:40:14.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:14 smithi084 ceph-mon[32629]: osdmap e437: 8 total, 8 up, 8 in 2024-09-16T09:40:14.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:14 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/651146524' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:40:14.916 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:40:14.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:14 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:40:14.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:14 smithi070 ceph-mon[30388]: osdmap e437: 8 total, 8 up, 8 in 2024-09-16T09:40:14.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:14 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/651146524' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:40:14.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:14 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:40:14.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:14 smithi070 ceph-mon[40691]: osdmap e437: 8 total, 8 up, 8 in 2024-09-16T09:40:14.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:14 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/651146524' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:40:14.943 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:14.943 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:14.944 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:40:15.045 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:40:15.075 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:15.076 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:15.077 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:40:15.178 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:40:15.221 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:15.222 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:15.222 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:40:15.323 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:40:15.359 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:15.360 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:15.360 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:40:15.461 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:40:15.501 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:15.502 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:15.503 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:40:15.603 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:40:15.640 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:15.640 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:15.641 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:40:15.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:15 smithi070 ceph-mon[30388]: pgmap v808: 178 pgs: 1 peering, 1 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 175 active+clean; 4.0 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.16k op/s; 8076/199047 objects misplaced (4.057%); 13 MiB/s, 202 objects/s recovering 2024-09-16T09:40:15.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:15 smithi070 ceph-mon[30388]: 10.14 scrub starts 2024-09-16T09:40:15.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:15 smithi070 ceph-mon[30388]: Health check update: Degraded data redundancy: 1 pg undersized (PG_DEGRADED) 2024-09-16T09:40:15.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:15 smithi070 ceph-mon[30388]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:40:15.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:15 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/651146524' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-16T09:40:15.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:15 smithi070 ceph-mon[30388]: osdmap e438: 8 total, 8 up, 8 in 2024-09-16T09:40:15.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:15 smithi070 ceph-mon[40691]: pgmap v808: 178 pgs: 1 peering, 1 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 175 active+clean; 4.0 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.16k op/s; 8076/199047 objects misplaced (4.057%); 13 MiB/s, 202 objects/s recovering 2024-09-16T09:40:15.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:15 smithi070 ceph-mon[40691]: 10.14 scrub starts 2024-09-16T09:40:15.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:15 smithi070 ceph-mon[40691]: Health check update: Degraded data redundancy: 1 pg undersized (PG_DEGRADED) 2024-09-16T09:40:15.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:15 smithi070 ceph-mon[40691]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:40:15.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:15 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/651146524' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-16T09:40:15.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:15 smithi070 ceph-mon[40691]: osdmap e438: 8 total, 8 up, 8 in 2024-09-16T09:40:15.741 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:40:15.783 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:15.784 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:15.784 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:40:15.785 INFO:teuthology.orchestra.run.smithi070.stderr:no valid command found; 10 closest matches: 2024-09-16T09:40:15.785 INFO:teuthology.orchestra.run.smithi070.stderr:0 2024-09-16T09:40:15.785 INFO:teuthology.orchestra.run.smithi070.stderr:1 2024-09-16T09:40:15.785 INFO:teuthology.orchestra.run.smithi070.stderr:2 2024-09-16T09:40:15.785 INFO:teuthology.orchestra.run.smithi070.stderr:abort 2024-09-16T09:40:15.785 INFO:teuthology.orchestra.run.smithi070.stderr:assert 2024-09-16T09:40:15.785 INFO:teuthology.orchestra.run.smithi070.stderr:bench [] [] [] [] 2024-09-16T09:40:15.785 INFO:teuthology.orchestra.run.smithi070.stderr:bluefs debug_inject_read_zeros 2024-09-16T09:40:15.786 INFO:teuthology.orchestra.run.smithi070.stderr:bluefs files list 2024-09-16T09:40:15.786 INFO:teuthology.orchestra.run.smithi070.stderr:bluefs stats 2024-09-16T09:40:15.786 INFO:teuthology.orchestra.run.smithi070.stderr:bluestore allocator dump block 2024-09-16T09:40:15.786 INFO:teuthology.orchestra.run.smithi070.stderr:Error EINVAL: invalid command 2024-09-16T09:40:15.885 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:40:15.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:15 smithi084 ceph-mon[32629]: pgmap v808: 178 pgs: 1 peering, 1 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 175 active+clean; 4.0 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.16k op/s; 8076/199047 objects misplaced (4.057%); 13 MiB/s, 202 objects/s recovering 2024-09-16T09:40:15.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:15 smithi084 ceph-mon[32629]: 10.14 scrub starts 2024-09-16T09:40:15.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:15 smithi084 ceph-mon[32629]: Health check update: Degraded data redundancy: 1 pg undersized (PG_DEGRADED) 2024-09-16T09:40:15.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:15 smithi084 ceph-mon[32629]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:40:15.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:15 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/651146524' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-16T09:40:15.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:15 smithi084 ceph-mon[32629]: osdmap e438: 8 total, 8 up, 8 in 2024-09-16T09:40:15.922 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:15.923 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:15.923 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:40:16.024 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:40:16.058 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:16.059 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:16.059 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:40:16.160 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:40:16.195 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:16.195 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:16.196 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:40:16.297 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:40:16.339 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:16.340 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:16.340 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:40:16.441 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:40:16.475 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:16.476 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:16.476 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:40:16.577 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:40:16.610 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:16.611 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:16.612 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:40:16.713 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:40:16.756 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:16.757 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:16.758 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:40:16.840 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:16 smithi070 ceph-mon[30388]: pgmap v810: 178 pgs: 1 peering, 1 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 175 active+clean; 4.0 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.17k op/s; 7626/197385 objects misplaced (3.864%); 11 MiB/s, 183 objects/s recovering 2024-09-16T09:40:16.840 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:16 smithi070 ceph-mon[40691]: pgmap v810: 178 pgs: 1 peering, 1 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 175 active+clean; 4.0 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.17k op/s; 7626/197385 objects misplaced (3.864%); 11 MiB/s, 183 objects/s recovering 2024-09-16T09:40:16.859 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:40:16.890 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:16.890 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:16.891 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:40:16.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:16 smithi084 ceph-mon[32629]: pgmap v810: 178 pgs: 1 peering, 1 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 175 active+clean; 4.0 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.17k op/s; 7626/197385 objects misplaced (3.864%); 11 MiB/s, 183 objects/s recovering 2024-09-16T09:40:16.992 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:40:17.028 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:17.029 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:17.029 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:40:17.130 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:40:17.171 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:17.172 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:17.172 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:40:17.197 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-16T09:40:17.198 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-16T09:40:17.198 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph tell osd.5 'trim stale osdmaps' 2024-09-16T09:40:17.273 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:40:17.357 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:17.358 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:17.358 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:40:17.445 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:40:17.446 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:40:17.459 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:40:17.489 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:40:17.489 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:40:17.510 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:17.511 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:17.511 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:40:17.529 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:40:17.529 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:40:17.572 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:40:17.572 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:40:17.605 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:40:17.605 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:40:17.613 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:40:17.637 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:40:17.637 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:40:17.639 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:17.640 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:17.640 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:40:17.669 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:40:17.670 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:40:17.703 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:17 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2161876452' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:40:17.703 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:17 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/2161876452' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:40:17.704 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:40:17.704 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:40:17.741 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:40:17.775 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:17.776 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:17.776 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:40:17.877 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:40:17.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:17 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/2161876452' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:40:17.912 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:17.912 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:17.912 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:40:18.013 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:40:18.050 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:18.050 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:18.051 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:40:18.151 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:40:18.187 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:18.188 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:18.188 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:40:18.289 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:40:18.323 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:18.323 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:18.324 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:40:18.356 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T09:40:18.356 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T09:40:18.356 INFO:teuthology.orchestra.run.smithi070.stdout: "complaint_time": 30, 2024-09-16T09:40:18.356 INFO:teuthology.orchestra.run.smithi070.stdout: "num_blocked_ops": 0 2024-09-16T09:40:18.356 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T09:40:18.425 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:40:18.458 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:18.459 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:18.459 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:40:18.540 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd unset noscrub 2024-09-16T09:40:18.560 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:40:18.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:18.626 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:18.627 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:40:18.727 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:40:18.775 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:18.777 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:18.777 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:40:18.878 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:40:18.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:18 smithi084 ceph-mon[32629]: pgmap v811: 178 pgs: 1 peering, 1 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 175 active+clean; 4.0 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 864 op/s; 7626/197385 objects misplaced (3.864%); 8.4 MiB/s, 134 objects/s recovering 2024-09-16T09:40:18.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:18 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:40:18.915 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:18.915 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:18.916 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:40:18.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:18 smithi070 ceph-mon[30388]: pgmap v811: 178 pgs: 1 peering, 1 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 175 active+clean; 4.0 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 864 op/s; 7626/197385 objects misplaced (3.864%); 8.4 MiB/s, 134 objects/s recovering 2024-09-16T09:40:18.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:18 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:40:18.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:18 smithi070 ceph-mon[40691]: pgmap v811: 178 pgs: 1 peering, 1 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 175 active+clean; 4.0 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 864 op/s; 7626/197385 objects misplaced (3.864%); 8.4 MiB/s, 134 objects/s recovering 2024-09-16T09:40:18.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:18 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:40:19.017 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:40:19.048 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:19.049 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:19.049 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:40:19.150 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:40:19.199 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:19.200 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:19.201 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:40:19.301 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:40:19.335 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:19.336 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:19.336 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:40:19.437 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:40:19.472 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:19.473 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:19.473 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:40:19.574 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:40:19.611 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:19.611 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:19.612 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:40:19.691 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.1 dump_historic_ops 2024-09-16T09:40:19.712 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:40:19.713 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:40:19.742 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:40:19.758 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:19.759 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:19.759 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:40:19.860 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:40:19.894 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:19.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:19.895 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:40:19.996 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:40:20.022 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:20.023 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:20.023 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:40:20.124 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:40:20.157 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:20.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:20.159 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:40:20.260 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:40:20.294 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:20.295 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:20.296 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:40:20.397 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:40:20.435 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:20.435 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:20.436 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:40:20.536 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:40:20.565 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:20.566 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:20.566 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:40:20.667 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:40:20.704 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:20.704 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:20.704 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:40:20.805 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:40:20.827 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:20.828 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:20.828 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:40:20.929 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:40:20.959 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:20.960 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:20.960 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:40:21.061 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:40:21.096 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:21.096 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:21.096 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:40:21.129 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T09:40:21.197 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:40:21.234 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:21.235 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:21.235 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:40:21.336 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:40:21.358 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:21.359 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:21.359 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:40:21.425 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:21 smithi070 ceph-mon[30388]: pgmap v812: 178 pgs: 1 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 176 active+clean; 3.6 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1.56k op/s; 6033/179034 objects misplaced (3.370%); 9.9 MiB/s, 158 objects/s recovering 2024-09-16T09:40:21.426 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:21 smithi070 ceph-mon[40691]: pgmap v812: 178 pgs: 1 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 176 active+clean; 3.6 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1.56k op/s; 6033/179034 objects misplaced (3.370%); 9.9 MiB/s, 158 objects/s recovering 2024-09-16T09:40:21.460 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:40:21.504 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:21.504 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:21.504 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:40:21.605 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:40:21.622 INFO:teuthology.orchestra.run.smithi070.stderr:no valid command found; 10 closest matches: 2024-09-16T09:40:21.622 INFO:teuthology.orchestra.run.smithi070.stderr:0 2024-09-16T09:40:21.623 INFO:teuthology.orchestra.run.smithi070.stderr:1 2024-09-16T09:40:21.623 INFO:teuthology.orchestra.run.smithi070.stderr:2 2024-09-16T09:40:21.623 INFO:teuthology.orchestra.run.smithi070.stderr:abort 2024-09-16T09:40:21.623 INFO:teuthology.orchestra.run.smithi070.stderr:assert 2024-09-16T09:40:21.623 INFO:teuthology.orchestra.run.smithi070.stderr:bench [] [] [] [] 2024-09-16T09:40:21.623 INFO:teuthology.orchestra.run.smithi070.stderr:bluefs debug_inject_read_zeros 2024-09-16T09:40:21.623 INFO:teuthology.orchestra.run.smithi070.stderr:bluefs files list 2024-09-16T09:40:21.624 INFO:teuthology.orchestra.run.smithi070.stderr:bluefs stats 2024-09-16T09:40:21.624 INFO:teuthology.orchestra.run.smithi070.stderr:bluestore allocator dump block 2024-09-16T09:40:21.624 INFO:teuthology.orchestra.run.smithi070.stderr:Error EINVAL: invalid command 2024-09-16T09:40:21.638 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:21.639 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:21.639 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:40:21.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:21 smithi084 ceph-mon[32629]: pgmap v812: 178 pgs: 1 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 176 active+clean; 3.6 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1.56k op/s; 6033/179034 objects misplaced (3.370%); 9.9 MiB/s, 158 objects/s recovering 2024-09-16T09:40:21.740 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:40:21.765 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:21.766 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:21.766 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:40:21.867 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:40:21.887 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:21.888 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:21.888 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:40:21.989 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:40:22.011 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:22.011 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:22.012 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:40:22.113 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:40:22.134 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:22.134 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:22.135 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:40:22.235 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:40:22.257 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:22.258 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:22.258 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:40:22.359 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:40:22.380 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:22.380 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:22.381 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:40:22.407 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:40:21 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:40:21] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:40:22.411 INFO:teuthology.orchestra.run.smithi070.stderr:noscrub is unset 2024-09-16T09:40:22.470 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-16T09:40:22.470 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-16T09:40:22.470 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph tell osd.6 'trim stale osdmaps' 2024-09-16T09:40:22.481 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:40:22.514 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:22.515 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:22.515 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:40:22.616 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:40:22.658 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:22.659 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:22.660 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:40:22.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:22 smithi084 ceph-mon[32629]: osdmap e439: 8 total, 8 up, 8 in 2024-09-16T09:40:22.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:22 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1763590982' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:40:22.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:22 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:40:22.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:22 smithi070 ceph-mon[30388]: osdmap e439: 8 total, 8 up, 8 in 2024-09-16T09:40:22.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:22 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1763590982' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:40:22.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:22 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:40:22.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:22 smithi070 ceph-mon[40691]: osdmap e439: 8 total, 8 up, 8 in 2024-09-16T09:40:22.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:22 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1763590982' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:40:22.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:22 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:40:22.761 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:40:22.795 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:22.796 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:22.796 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:40:22.897 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:40:22.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:22.933 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:22.933 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:40:23.034 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:40:23.070 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:23.071 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:23.071 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:40:23.172 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:40:23.208 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:23.208 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:23.209 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:40:23.219 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:40:23.258 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:40:23.300 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:40:23.309 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:40:23.344 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:23.345 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:23.345 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:40:23.446 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:40:23.482 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:23.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:23.483 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:40:23.584 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:40:23.620 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:23.621 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:23.621 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:40:23.635 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:23 smithi070 ceph-mon[30388]: pgmap v814: 178 pgs: 1 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 176 active+clean; 3.6 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 834 op/s; 6033/179034 objects misplaced (3.370%); 2.7 MiB/s, 42 objects/s recovering 2024-09-16T09:40:23.635 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:23 smithi070 ceph-mon[30388]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:40:23.636 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:23 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-16T09:40:23.636 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:23 smithi070 ceph-mon[30388]: osdmap e440: 8 total, 8 up, 8 in 2024-09-16T09:40:23.636 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:23 smithi070 ceph-mon[40691]: pgmap v814: 178 pgs: 1 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 176 active+clean; 3.6 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 834 op/s; 6033/179034 objects misplaced (3.370%); 2.7 MiB/s, 42 objects/s recovering 2024-09-16T09:40:23.636 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:23 smithi070 ceph-mon[40691]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:40:23.636 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:23 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-16T09:40:23.636 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:23 smithi070 ceph-mon[40691]: osdmap e440: 8 total, 8 up, 8 in 2024-09-16T09:40:23.722 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:40:23.757 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:23.757 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:23.758 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:40:23.859 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:40:23.890 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:23.891 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:23.891 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:40:23.910 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:40:23 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:40:23.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:23 smithi084 ceph-mon[32629]: pgmap v814: 178 pgs: 1 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 176 active+clean; 3.6 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 834 op/s; 6033/179034 objects misplaced (3.370%); 2.7 MiB/s, 42 objects/s recovering 2024-09-16T09:40:23.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:23 smithi084 ceph-mon[32629]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:40:23.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:23 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-16T09:40:23.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:23 smithi084 ceph-mon[32629]: osdmap e440: 8 total, 8 up, 8 in 2024-09-16T09:40:23.992 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:40:24.025 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:24.026 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:24.027 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:40:24.127 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:40:24.163 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:24.164 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:24.164 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:40:24.166 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T09:40:24.166 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 20, 2024-09-16T09:40:24.167 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 600, 2024-09-16T09:40:24.167 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [ 2024-09-16T09:40:24.167 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.167 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:64865 9.16 9:6ee6eb80:::benchmark_data_smithi070_137000_object64864:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e252)", 2024-09-16T09:40:24.167 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:34:09.127434+0000", 2024-09-16T09:40:24.167 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 375.03541126200003, 2024-09-16T09:40:24.167 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.190987296, 2024-09-16T09:40:24.167 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:40:24.167 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:40:24.167 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:40:24.167 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:40:24.167 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:40:24.167 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:40:24.168 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 64865 2024-09-16T09:40:24.168 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.168 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:40:24.168 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.168 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:40:24.168 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:09.127434+0000", 2024-09-16T09:40:24.168 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:24.168 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.168 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.168 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:40:24.168 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:09.127434+0000", 2024-09-16T09:40:24.168 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:24.168 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.168 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.169 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:40:24.169 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:09.127436+0000", 2024-09-16T09:40:24.169 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.8169999999999999e-06 2024-09-16T09:40:24.169 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.169 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.169 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:40:24.169 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:09.127460+0000", 2024-09-16T09:40:24.169 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.4292999999999999e-05 2024-09-16T09:40:24.169 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.169 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.169 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:40:24.169 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:09.127472+0000", 2024-09-16T09:40:24.170 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2284e-05 2024-09-16T09:40:24.170 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.170 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.170 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:40:24.170 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:09.127479+0000", 2024-09-16T09:40:24.170 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.9929999999999998e-06 2024-09-16T09:40:24.170 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.170 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.170 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:40:24.170 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:09.127893+0000", 2024-09-16T09:40:24.170 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00041387800000000001 2024-09-16T09:40:24.170 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.170 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.170 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:40:24.171 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:09.129470+0000", 2024-09-16T09:40:24.171 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001577097 2024-09-16T09:40:24.171 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.171 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.171 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 2,4", 2024-09-16T09:40:24.171 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:09.129633+0000", 2024-09-16T09:40:24.171 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000162953 2024-09-16T09:40:24.171 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.171 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.171 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:40:24.171 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:09.133349+0000", 2024-09-16T09:40:24.172 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0037157689999999998 2024-09-16T09:40:24.172 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.172 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.172 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:40:24.172 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:09.134292+0000", 2024-09-16T09:40:24.172 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00094281599999999997 2024-09-16T09:40:24.172 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.172 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.172 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:40:24.172 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:09.318056+0000", 2024-09-16T09:40:24.172 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.18376447400000001 2024-09-16T09:40:24.172 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.172 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.173 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:40:24.173 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:09.318131+0000", 2024-09-16T09:40:24.173 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.4157000000000002e-05 2024-09-16T09:40:24.173 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.173 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.173 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:40:24.173 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:09.318421+0000", 2024-09-16T09:40:24.173 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00029076500000000001 2024-09-16T09:40:24.173 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:24.174 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:24.174 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:24.174 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.174 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.174 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:71274 9.10 9:0dff0190:::benchmark_data_smithi070_137000_object71273:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e257)", 2024-09-16T09:40:24.174 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:34:16.586236+0000", 2024-09-16T09:40:24.174 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 367.57660946099998, 2024-09-16T09:40:24.174 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.19109680000000001, 2024-09-16T09:40:24.174 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:40:24.174 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:40:24.174 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:40:24.174 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:40:24.174 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:40:24.175 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:40:24.175 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 71274 2024-09-16T09:40:24.175 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.175 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:40:24.175 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.175 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:40:24.175 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:16.586236+0000", 2024-09-16T09:40:24.175 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:24.175 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.175 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.175 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:40:24.175 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:16.586236+0000", 2024-09-16T09:40:24.175 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:24.175 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.176 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.176 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:40:24.176 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:16.586237+0000", 2024-09-16T09:40:24.176 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.319e-06 2024-09-16T09:40:24.176 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.176 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.176 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:40:24.176 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:16.586258+0000", 2024-09-16T09:40:24.176 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0352000000000001e-05 2024-09-16T09:40:24.176 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.176 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.176 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:40:24.177 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:16.586267+0000", 2024-09-16T09:40:24.177 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.3880000000000006e-06 2024-09-16T09:40:24.177 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.177 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.177 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:40:24.177 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:16.586272+0000", 2024-09-16T09:40:24.177 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.2689999999999999e-06 2024-09-16T09:40:24.177 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.177 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.177 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:40:24.177 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:16.589116+0000", 2024-09-16T09:40:24.177 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0028433899999999999 2024-09-16T09:40:24.177 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.178 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.178 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:40:24.178 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:16.589244+0000", 2024-09-16T09:40:24.178 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00012846299999999999 2024-09-16T09:40:24.178 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.178 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.178 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 0,4,7", 2024-09-16T09:40:24.178 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:16.589403+0000", 2024-09-16T09:40:24.178 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00015850099999999999 2024-09-16T09:40:24.178 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.178 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.178 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:40:24.178 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:16.597202+0000", 2024-09-16T09:40:24.179 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0077995970000000001 2024-09-16T09:40:24.179 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.179 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.179 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:40:24.179 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:16.597369+0000", 2024-09-16T09:40:24.179 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00016683 2024-09-16T09:40:24.179 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.179 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.179 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:40:24.179 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:16.597450+0000", 2024-09-16T09:40:24.179 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.0926000000000004e-05 2024-09-16T09:40:24.179 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.179 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.180 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:40:24.180 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:16.775897+0000", 2024-09-16T09:40:24.180 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.17844701600000001 2024-09-16T09:40:24.180 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.180 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.180 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:40:24.180 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:16.776498+0000", 2024-09-16T09:40:24.180 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00060158300000000004 2024-09-16T09:40:24.180 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.180 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.180 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:40:24.180 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:16.777333+0000", 2024-09-16T09:40:24.180 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00083416600000000001 2024-09-16T09:40:24.181 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:24.181 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:24.181 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:24.181 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.181 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.181 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:232736 9.12 9:4ea007d7:::benchmark_data_smithi070_137000_object111025:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e360)", 2024-09-16T09:40:24.181 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:37:21.520060+0000", 2024-09-16T09:40:24.182 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 182.642785704, 2024-09-16T09:40:24.182 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0959157930000001, 2024-09-16T09:40:24.182 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:40:24.182 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:40:24.182 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:40:24.182 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:40:24.182 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:40:24.182 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:40:24.182 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 232736 2024-09-16T09:40:24.182 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.182 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:40:24.183 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.183 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:40:24.183 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.520060+0000", 2024-09-16T09:40:24.183 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:24.183 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.183 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.183 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:40:24.183 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.520060+0000", 2024-09-16T09:40:24.183 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:24.183 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.183 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.183 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:40:24.183 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.520061+0000", 2024-09-16T09:40:24.184 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.35e-06 2024-09-16T09:40:24.184 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.184 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.184 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:40:24.184 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.520065+0000", 2024-09-16T09:40:24.184 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2980000000000002e-06 2024-09-16T09:40:24.184 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.184 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.184 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:40:24.184 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.520077+0000", 2024-09-16T09:40:24.184 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2082999999999999e-05 2024-09-16T09:40:24.184 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.184 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.185 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:40:24.185 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.520087+0000", 2024-09-16T09:40:24.185 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.006e-05 2024-09-16T09:40:24.185 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.185 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.185 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:40:24.185 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.521031+0000", 2024-09-16T09:40:24.185 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00094380400000000002 2024-09-16T09:40:24.185 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.186 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.186 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op must wait for map", 2024-09-16T09:40:24.186 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.521038+0000", 2024-09-16T09:40:24.186 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.1359999999999999e-06 2024-09-16T09:40:24.186 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.186 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.186 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:40:24.186 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.527328+0000", 2024-09-16T09:40:24.186 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.006289195 2024-09-16T09:40:24.187 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.187 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.187 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:40:24.187 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.527338+0000", 2024-09-16T09:40:24.187 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0360000000000001e-05 2024-09-16T09:40:24.187 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.187 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.187 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:40:24.187 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.611637+0000", 2024-09-16T09:40:24.187 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0842987449999999 2024-09-16T09:40:24.187 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.187 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.187 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:40:24.187 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.611765+0000", 2024-09-16T09:40:24.187 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00012811699999999999 2024-09-16T09:40:24.188 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.188 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.188 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 0,2,3,5", 2024-09-16T09:40:24.188 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.611893+0000", 2024-09-16T09:40:24.188 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000128171 2024-09-16T09:40:24.188 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.188 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.188 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:40:24.188 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.615315+0000", 2024-09-16T09:40:24.188 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0034216429999999998 2024-09-16T09:40:24.188 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.188 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.188 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:40:24.188 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.615329+0000", 2024-09-16T09:40:24.188 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.4405e-05 2024-09-16T09:40:24.189 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.189 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.189 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:40:24.189 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.615364+0000", 2024-09-16T09:40:24.189 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5286000000000001e-05 2024-09-16T09:40:24.189 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.189 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.189 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:40:24.189 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.615412+0000", 2024-09-16T09:40:24.189 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.7367000000000002e-05 2024-09-16T09:40:24.189 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.189 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.189 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:40:24.189 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.615802+0000", 2024-09-16T09:40:24.190 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00039052599999999998 2024-09-16T09:40:24.190 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.190 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.190 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:40:24.190 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.615859+0000", 2024-09-16T09:40:24.190 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.6919000000000002e-05 2024-09-16T09:40:24.190 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.190 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.190 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:40:24.190 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.615975+0000", 2024-09-16T09:40:24.190 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000116328 2024-09-16T09:40:24.190 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:24.190 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:24.191 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:24.191 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.191 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.191 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:232744 9.16 9:6db6f31f:::benchmark_data_smithi070_137000_object111033:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e360)", 2024-09-16T09:40:24.191 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:37:21.520102+0000", 2024-09-16T09:40:24.191 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 182.642743137, 2024-09-16T09:40:24.191 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.092603174, 2024-09-16T09:40:24.191 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:40:24.191 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:40:24.191 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:40:24.191 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:40:24.191 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:40:24.191 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:40:24.192 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 232744 2024-09-16T09:40:24.192 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.192 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:40:24.192 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.192 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:40:24.192 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.520102+0000", 2024-09-16T09:40:24.192 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:24.192 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.192 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.192 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:40:24.192 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.520102+0000", 2024-09-16T09:40:24.193 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:24.193 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.193 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.193 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:40:24.193 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.520103+0000", 2024-09-16T09:40:24.193 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.1399999999999997e-07 2024-09-16T09:40:24.193 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.193 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.193 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:40:24.193 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.520105+0000", 2024-09-16T09:40:24.193 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3580000000000001e-06 2024-09-16T09:40:24.193 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.193 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.193 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:40:24.194 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.520111+0000", 2024-09-16T09:40:24.194 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.0990000000000004e-06 2024-09-16T09:40:24.194 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.194 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.194 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:40:24.194 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.520116+0000", 2024-09-16T09:40:24.194 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.2270000000000001e-06 2024-09-16T09:40:24.194 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.194 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.194 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:40:24.194 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.521263+0000", 2024-09-16T09:40:24.194 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0011466530000000001 2024-09-16T09:40:24.194 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.194 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.194 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op must wait for map", 2024-09-16T09:40:24.195 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.521269+0000", 2024-09-16T09:40:24.195 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1489999999999999e-06 2024-09-16T09:40:24.195 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.195 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.195 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:40:24.195 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.527938+0000", 2024-09-16T09:40:24.195 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0066689130000000003 2024-09-16T09:40:24.195 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.195 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.195 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:40:24.195 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.527947+0000", 2024-09-16T09:40:24.195 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.816e-06 2024-09-16T09:40:24.195 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.195 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.195 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:40:24.195 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.600757+0000", 2024-09-16T09:40:24.196 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0728101139999999 2024-09-16T09:40:24.196 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.196 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.196 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:40:24.196 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.600898+0000", 2024-09-16T09:40:24.196 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00014071799999999999 2024-09-16T09:40:24.196 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.196 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.196 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 2,4,5", 2024-09-16T09:40:24.196 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.601036+0000", 2024-09-16T09:40:24.196 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00013785100000000001 2024-09-16T09:40:24.196 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.197 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.197 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:40:24.197 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.602183+0000", 2024-09-16T09:40:24.197 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001147742 2024-09-16T09:40:24.197 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.197 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.197 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:40:24.197 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.602237+0000", 2024-09-16T09:40:24.197 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.3905e-05 2024-09-16T09:40:24.197 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.197 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.198 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:40:24.198 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.612452+0000", 2024-09-16T09:40:24.198 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.010215033 2024-09-16T09:40:24.198 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.198 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.198 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:40:24.198 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.612498+0000", 2024-09-16T09:40:24.198 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5612000000000003e-05 2024-09-16T09:40:24.198 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.198 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.199 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:40:24.199 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.612599+0000", 2024-09-16T09:40:24.199 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00010118700000000001 2024-09-16T09:40:24.199 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.199 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.199 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:40:24.199 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.612705+0000", 2024-09-16T09:40:24.199 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000106383 2024-09-16T09:40:24.199 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:24.199 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:24.199 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:24.199 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.199 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.199 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:232750 9.10 9:08f1a82d:::benchmark_data_smithi070_137000_object111039:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e360)", 2024-09-16T09:40:24.200 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:37:21.520133+0000", 2024-09-16T09:40:24.200 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 182.642711913, 2024-09-16T09:40:24.200 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1158816490000001, 2024-09-16T09:40:24.200 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:40:24.200 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:40:24.200 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:40:24.200 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:40:24.200 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:40:24.200 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:40:24.200 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 232750 2024-09-16T09:40:24.200 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.200 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:40:24.200 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.201 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:40:24.201 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.520133+0000", 2024-09-16T09:40:24.201 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:24.201 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.201 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.201 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:40:24.201 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.520133+0000", 2024-09-16T09:40:24.201 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:24.201 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.201 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.201 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:40:24.202 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.520134+0000", 2024-09-16T09:40:24.202 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.4499999999999997e-07 2024-09-16T09:40:24.202 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.202 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.202 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:40:24.202 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.520136+0000", 2024-09-16T09:40:24.202 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.1550000000000001e-06 2024-09-16T09:40:24.202 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.202 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.202 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:40:24.202 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.520143+0000", 2024-09-16T09:40:24.202 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.5579999999999997e-06 2024-09-16T09:40:24.202 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.202 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.202 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:40:24.203 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.520147+0000", 2024-09-16T09:40:24.203 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.9089999999999999e-06 2024-09-16T09:40:24.203 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.203 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.203 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:40:24.203 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.520793+0000", 2024-09-16T09:40:24.203 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00064582400000000003 2024-09-16T09:40:24.203 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.203 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.203 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op must wait for map", 2024-09-16T09:40:24.203 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.520802+0000", 2024-09-16T09:40:24.203 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.7579999999999998e-06 2024-09-16T09:40:24.203 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.203 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.204 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:40:24.204 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.527074+0000", 2024-09-16T09:40:24.204 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0062723149999999997 2024-09-16T09:40:24.204 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.204 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.204 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:40:24.204 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.527086+0000", 2024-09-16T09:40:24.204 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1206000000000001e-05 2024-09-16T09:40:24.204 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.204 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.204 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:40:24.204 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.618695+0000", 2024-09-16T09:40:24.204 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.091609826 2024-09-16T09:40:24.204 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.205 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.205 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:40:24.205 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.618780+0000", 2024-09-16T09:40:24.205 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.4135000000000007e-05 2024-09-16T09:40:24.205 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.205 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.205 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 0,2,4,6,7", 2024-09-16T09:40:24.205 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.618857+0000", 2024-09-16T09:40:24.205 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.7638000000000004e-05 2024-09-16T09:40:24.205 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.205 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.205 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:40:24.206 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.620139+0000", 2024-09-16T09:40:24.206 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0012817919999999999 2024-09-16T09:40:24.206 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.206 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.206 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:40:24.206 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.620277+0000", 2024-09-16T09:40:24.206 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000138467 2024-09-16T09:40:24.206 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.206 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.206 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:40:24.206 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.620434+0000", 2024-09-16T09:40:24.206 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00015646200000000001 2024-09-16T09:40:24.207 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.207 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.207 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:40:24.207 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.624194+0000", 2024-09-16T09:40:24.207 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0037599030000000002 2024-09-16T09:40:24.207 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.207 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.207 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:40:24.207 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.624290+0000", 2024-09-16T09:40:24.207 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.6267000000000005e-05 2024-09-16T09:40:24.207 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.207 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.207 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:40:24.208 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.635805+0000", 2024-09-16T09:40:24.208 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.011515107 2024-09-16T09:40:24.208 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.208 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.208 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:40:24.208 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.635870+0000", 2024-09-16T09:40:24.208 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.4678999999999995e-05 2024-09-16T09:40:24.208 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.208 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.208 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:40:24.208 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.636015+0000", 2024-09-16T09:40:24.208 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000145203 2024-09-16T09:40:24.208 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:24.209 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:24.209 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:24.209 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.209 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.209 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:232758 9.12 9:4a9f0b61:::benchmark_data_smithi070_137000_object111047:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e360)", 2024-09-16T09:40:24.209 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:37:21.520161+0000", 2024-09-16T09:40:24.209 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 182.642683987, 2024-09-16T09:40:24.209 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.096396232, 2024-09-16T09:40:24.209 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:40:24.209 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:40:24.209 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:40:24.210 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:40:24.210 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:40:24.210 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:40:24.210 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 232758 2024-09-16T09:40:24.210 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.210 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:40:24.210 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.210 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:40:24.210 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.520161+0000", 2024-09-16T09:40:24.210 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:24.210 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.210 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.210 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:40:24.210 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.520161+0000", 2024-09-16T09:40:24.211 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:24.211 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.211 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.211 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:40:24.211 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.520162+0000", 2024-09-16T09:40:24.211 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6800000000000001e-07 2024-09-16T09:40:24.211 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.211 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.211 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:40:24.211 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.520164+0000", 2024-09-16T09:40:24.212 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.951e-06 2024-09-16T09:40:24.212 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.212 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.212 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:40:24.212 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.520170+0000", 2024-09-16T09:40:24.212 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1e-06 2024-09-16T09:40:24.213 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.213 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.213 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:40:24.213 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.520175+0000", 2024-09-16T09:40:24.213 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.8729999999999998e-06 2024-09-16T09:40:24.214 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.214 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.214 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:40:24.214 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.521063+0000", 2024-09-16T09:40:24.214 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000888103 2024-09-16T09:40:24.214 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.214 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.214 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting_for_map not empty", 2024-09-16T09:40:24.214 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.521068+0000", 2024-09-16T09:40:24.214 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5809999999999996e-06 2024-09-16T09:40:24.214 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.215 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.215 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:40:24.215 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.536366+0000", 2024-09-16T09:40:24.215 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.015297471 2024-09-16T09:40:24.215 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.215 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.215 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:40:24.215 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.536395+0000", 2024-09-16T09:40:24.215 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.9260000000000001e-05 2024-09-16T09:40:24.215 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.215 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.215 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:40:24.216 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.614279+0000", 2024-09-16T09:40:24.216 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0778841029999999 2024-09-16T09:40:24.216 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.216 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.216 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:40:24.216 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.614593+0000", 2024-09-16T09:40:24.216 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00031368100000000001 2024-09-16T09:40:24.216 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.216 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.216 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 0,2,3,5", 2024-09-16T09:40:24.216 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.614716+0000", 2024-09-16T09:40:24.216 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00012332900000000001 2024-09-16T09:40:24.216 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.217 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.217 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:40:24.217 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.616163+0000", 2024-09-16T09:40:24.217 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0014472280000000001 2024-09-16T09:40:24.217 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.217 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.217 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:40:24.217 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.616255+0000", 2024-09-16T09:40:24.217 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.1528999999999997e-05 2024-09-16T09:40:24.217 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.217 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.217 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:40:24.217 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.616272+0000", 2024-09-16T09:40:24.218 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7098000000000001e-05 2024-09-16T09:40:24.218 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.218 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.218 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:40:24.218 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.616306+0000", 2024-09-16T09:40:24.218 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3847999999999998e-05 2024-09-16T09:40:24.218 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.218 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.218 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:40:24.218 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.616346+0000", 2024-09-16T09:40:24.218 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0528999999999998e-05 2024-09-16T09:40:24.218 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.218 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.219 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:40:24.219 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.616461+0000", 2024-09-16T09:40:24.219 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000114941 2024-09-16T09:40:24.219 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.219 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.219 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:40:24.219 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.616558+0000", 2024-09-16T09:40:24.219 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.6238999999999998e-05 2024-09-16T09:40:24.219 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:24.219 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:24.220 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:24.220 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.220 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.220 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:232762 9.10 9:0c9cd3ad:::benchmark_data_smithi070_137000_object111051:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e360)", 2024-09-16T09:40:24.220 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:37:21.520181+0000", 2024-09-16T09:40:24.220 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 182.64266463000001, 2024-09-16T09:40:24.220 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1160747520000001, 2024-09-16T09:40:24.220 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:40:24.220 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:40:24.220 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:40:24.220 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:40:24.220 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:40:24.221 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:40:24.221 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 232762 2024-09-16T09:40:24.221 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.221 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:40:24.221 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.221 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:40:24.221 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.520181+0000", 2024-09-16T09:40:24.221 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:24.221 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.221 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.221 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:40:24.222 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.520181+0000", 2024-09-16T09:40:24.222 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:24.222 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.222 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.222 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:40:24.222 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.520181+0000", 2024-09-16T09:40:24.222 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.0600000000000001e-07 2024-09-16T09:40:24.222 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.222 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.222 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:40:24.223 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.520183+0000", 2024-09-16T09:40:24.223 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.784e-06 2024-09-16T09:40:24.223 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.223 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.223 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:40:24.223 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.520188+0000", 2024-09-16T09:40:24.223 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5790000000000003e-06 2024-09-16T09:40:24.223 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.223 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.223 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:40:24.223 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.520193+0000", 2024-09-16T09:40:24.223 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.6460000000000001e-06 2024-09-16T09:40:24.223 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.224 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.224 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:40:24.224 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.520853+0000", 2024-09-16T09:40:24.224 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000660331 2024-09-16T09:40:24.224 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.224 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.224 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting_for_map not empty", 2024-09-16T09:40:24.224 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.520859+0000", 2024-09-16T09:40:24.224 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8289999999999996e-06 2024-09-16T09:40:24.224 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.224 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.224 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:40:24.224 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.527156+0000", 2024-09-16T09:40:24.224 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0062966719999999997 2024-09-16T09:40:24.224 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.225 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.225 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:40:24.225 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.527164+0000", 2024-09-16T09:40:24.225 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.2630000000000003e-06 2024-09-16T09:40:24.225 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.225 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.225 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:40:24.225 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.619488+0000", 2024-09-16T09:40:24.225 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.092323962 2024-09-16T09:40:24.225 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.225 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.225 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:40:24.225 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.619561+0000", 2024-09-16T09:40:24.225 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3312999999999994e-05 2024-09-16T09:40:24.226 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.226 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.226 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 0,2,4,6,7", 2024-09-16T09:40:24.226 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.619626+0000", 2024-09-16T09:40:24.226 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.4437999999999995e-05 2024-09-16T09:40:24.226 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.226 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.226 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:40:24.226 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.620761+0000", 2024-09-16T09:40:24.226 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0011348090000000001 2024-09-16T09:40:24.226 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.226 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.226 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:40:24.226 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.620774+0000", 2024-09-16T09:40:24.227 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2942000000000001e-05 2024-09-16T09:40:24.227 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.227 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.227 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:40:24.227 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.620804+0000", 2024-09-16T09:40:24.227 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.0054999999999999e-05 2024-09-16T09:40:24.227 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.227 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.227 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:40:24.227 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.624333+0000", 2024-09-16T09:40:24.227 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0035295159999999999 2024-09-16T09:40:24.227 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.227 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.227 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:40:24.228 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.635618+0000", 2024-09-16T09:40:24.228 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.011284591 2024-09-16T09:40:24.228 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.228 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.228 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:40:24.228 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.636059+0000", 2024-09-16T09:40:24.228 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00044175699999999998 2024-09-16T09:40:24.228 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.228 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.228 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:40:24.228 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.636118+0000", 2024-09-16T09:40:24.228 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8570000000000003e-05 2024-09-16T09:40:24.228 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.229 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.229 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:40:24.229 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.636255+0000", 2024-09-16T09:40:24.229 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000137389 2024-09-16T09:40:24.229 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:24.229 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:24.229 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:24.229 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.229 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.229 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.15975.0:113589 10.19 e417/398 10:9a90b735:::benchmark_data_smithi070_166744_object113588:head v 417'5779, mlcod=383'128)", 2024-09-16T09:40:24.229 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:39:25.243339+0000", 2024-09-16T09:40:24.229 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 58.919506603000002, 2024-09-16T09:40:24.230 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.25732612500000002, 2024-09-16T09:40:24.230 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:40:24.230 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:40:24.230 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:40:24.230 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:40:24.230 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.230 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:40:24.230 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.243339+0000", 2024-09-16T09:40:24.230 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:24.230 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.230 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.230 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:40:24.230 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.243339+0000", 2024-09-16T09:40:24.230 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:24.231 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.231 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.231 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:40:24.231 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.243339+0000", 2024-09-16T09:40:24.231 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.8400000000000004e-07 2024-09-16T09:40:24.231 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.231 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.231 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:40:24.231 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.243362+0000", 2024-09-16T09:40:24.231 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3e-05 2024-09-16T09:40:24.231 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.231 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.231 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:40:24.231 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.243379+0000", 2024-09-16T09:40:24.232 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6282000000000001e-05 2024-09-16T09:40:24.232 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.232 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.232 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:40:24.232 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.243385+0000", 2024-09-16T09:40:24.232 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1310000000000003e-06 2024-09-16T09:40:24.232 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.232 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.232 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:40:24.232 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.243579+0000", 2024-09-16T09:40:24.232 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00019384099999999999 2024-09-16T09:40:24.232 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.233 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.233 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:40:24.233 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.243601+0000", 2024-09-16T09:40:24.233 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.2269999999999999e-05 2024-09-16T09:40:24.233 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.233 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.233 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:40:24.233 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.249392+0000", 2024-09-16T09:40:24.233 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0057910590000000003 2024-09-16T09:40:24.233 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.233 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.233 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:40:24.234 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.500665+0000", 2024-09-16T09:40:24.234 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.25127285799999999 2024-09-16T09:40:24.234 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:24.234 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:24.234 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:24.234 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.234 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.234 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.15975.0:113599 10.13 e417/416 10:cf531bcf:::benchmark_data_smithi070_166744_object113598:head v 417'5970, mlcod=417'5966)", 2024-09-16T09:40:24.234 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:39:25.246192+0000", 2024-09-16T09:40:24.234 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 58.916653103000002, 2024-09-16T09:40:24.234 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.25445351799999999, 2024-09-16T09:40:24.234 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:40:24.234 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:40:24.235 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:40:24.235 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:40:24.235 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.235 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:40:24.235 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.246192+0000", 2024-09-16T09:40:24.235 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:24.235 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.235 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.235 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:40:24.235 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.246192+0000", 2024-09-16T09:40:24.235 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:24.235 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.236 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.236 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:40:24.236 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.246193+0000", 2024-09-16T09:40:24.236 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0750000000000001e-06 2024-09-16T09:40:24.236 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.236 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.236 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:40:24.236 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.246241+0000", 2024-09-16T09:40:24.236 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.7435000000000002e-05 2024-09-16T09:40:24.236 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.236 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.236 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:40:24.236 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.246253+0000", 2024-09-16T09:40:24.237 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1878999999999999e-05 2024-09-16T09:40:24.237 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.237 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.237 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:40:24.237 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.246261+0000", 2024-09-16T09:40:24.237 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.4239999999999993e-06 2024-09-16T09:40:24.237 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.237 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.237 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:40:24.237 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.246609+0000", 2024-09-16T09:40:24.237 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00034822999999999999 2024-09-16T09:40:24.237 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.237 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.237 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:40:24.238 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.246626+0000", 2024-09-16T09:40:24.238 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7229999999999999e-05 2024-09-16T09:40:24.238 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.238 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.238 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:40:24.238 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.500400+0000", 2024-09-16T09:40:24.238 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.253773521 2024-09-16T09:40:24.238 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.238 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.238 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:40:24.238 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.500646+0000", 2024-09-16T09:40:24.238 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00024572400000000001 2024-09-16T09:40:24.239 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:24.239 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:24.239 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:24.239 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.239 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.239 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.15975.0:113601 10.13 e417/416 10:ccd2f904:::benchmark_data_smithi070_166744_object113600:head v 417'5971, mlcod=417'5966)", 2024-09-16T09:40:24.239 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:39:25.248424+0000", 2024-09-16T09:40:24.239 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 58.914421019000002, 2024-09-16T09:40:24.239 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.25528167600000001, 2024-09-16T09:40:24.239 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:40:24.239 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:40:24.239 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:40:24.239 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:40:24.239 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.240 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:40:24.240 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.248424+0000", 2024-09-16T09:40:24.240 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:24.240 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.240 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.240 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:40:24.240 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.248424+0000", 2024-09-16T09:40:24.240 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:24.240 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.240 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.240 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:40:24.240 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.248425+0000", 2024-09-16T09:40:24.240 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.0100000000000003e-07 2024-09-16T09:40:24.240 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.241 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.241 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:40:24.241 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.248458+0000", 2024-09-16T09:40:24.241 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3151000000000002e-05 2024-09-16T09:40:24.241 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.241 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.241 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:40:24.241 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.248470+0000", 2024-09-16T09:40:24.241 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1456e-05 2024-09-16T09:40:24.241 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.241 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.241 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:40:24.241 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.248477+0000", 2024-09-16T09:40:24.242 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.5959999999999998e-06 2024-09-16T09:40:24.242 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.242 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.242 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:40:24.242 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.248676+0000", 2024-09-16T09:40:24.242 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000198984 2024-09-16T09:40:24.242 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.242 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.242 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:40:24.242 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.248716+0000", 2024-09-16T09:40:24.242 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9762999999999999e-05 2024-09-16T09:40:24.242 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.242 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.242 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:40:24.243 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.503649+0000", 2024-09-16T09:40:24.243 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.254932942 2024-09-16T09:40:24.243 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.243 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.243 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:40:24.243 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.503706+0000", 2024-09-16T09:40:24.243 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.6882999999999997e-05 2024-09-16T09:40:24.243 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:24.243 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:24.243 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:24.243 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.243 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.243 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.15975.0:113603 10.16 e417/398 10:6f5b7bce:::benchmark_data_smithi070_166744_object113602:head v 417'5683, mlcod=391'4079)", 2024-09-16T09:40:24.244 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:39:25.248991+0000", 2024-09-16T09:40:24.244 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 58.913854022999999, 2024-09-16T09:40:24.244 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.25780223800000002, 2024-09-16T09:40:24.244 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:40:24.244 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:40:24.244 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:40:24.244 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:40:24.244 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.244 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:40:24.244 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.248991+0000", 2024-09-16T09:40:24.245 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:24.245 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.245 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.245 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:40:24.245 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.248991+0000", 2024-09-16T09:40:24.245 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:24.245 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.245 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.245 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:40:24.245 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.248992+0000", 2024-09-16T09:40:24.245 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.9899999999999999e-07 2024-09-16T09:40:24.245 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.245 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.245 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:40:24.246 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.249017+0000", 2024-09-16T09:40:24.246 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.4332999999999999e-05 2024-09-16T09:40:24.246 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.246 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.246 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:40:24.246 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.249028+0000", 2024-09-16T09:40:24.246 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1734e-05 2024-09-16T09:40:24.246 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.246 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.246 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:40:24.246 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.249036+0000", 2024-09-16T09:40:24.246 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.9100000000000005e-06 2024-09-16T09:40:24.247 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.247 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.247 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:40:24.247 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.249112+0000", 2024-09-16T09:40:24.247 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.5683e-05 2024-09-16T09:40:24.247 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.247 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.247 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:40:24.247 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.249134+0000", 2024-09-16T09:40:24.248 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.2365999999999999e-05 2024-09-16T09:40:24.248 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.248 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.248 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:40:24.248 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.506749+0000", 2024-09-16T09:40:24.248 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.25761445300000002 2024-09-16T09:40:24.248 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.248 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.248 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:40:24.248 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.506794+0000", 2024-09-16T09:40:24.249 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.4860000000000001e-05 2024-09-16T09:40:24.249 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:24.249 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:24.249 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:24.249 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.249 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.249 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "MOSDPGPush(10.15 417/398 [PushOp(10:a9f4afe2:::benchmark_data_smithi070_166744_object2856:head, version: 383'171, data_included: [0~65536], data_size: 65536, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(10:a9f4afe2:::benchmark_data_smithi070_166744_object2856:head@383'171, size: 65536, copy_subset: [(0, 65536)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 65536, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-16T09:40:24.249 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:39:25.249003+0000", 2024-09-16T09:40:24.249 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 58.913842500000001, 2024-09-16T09:40:24.249 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.25139647799999998, 2024-09-16T09:40:24.249 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:40:24.249 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:40:24.250 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T09:40:24.250 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:40:24.250 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.250 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:40:24.250 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.249003+0000", 2024-09-16T09:40:24.250 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:24.250 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.250 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.250 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:40:24.250 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.249003+0000", 2024-09-16T09:40:24.250 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:24.251 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.251 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.251 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:40:24.251 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.249003+0000", 2024-09-16T09:40:24.251 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.68e-07 2024-09-16T09:40:24.251 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.251 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.251 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:40:24.251 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.249044+0000", 2024-09-16T09:40:24.251 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0290000000000002e-05 2024-09-16T09:40:24.251 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.251 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.251 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:40:24.251 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.249062+0000", 2024-09-16T09:40:24.251 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.8496e-05 2024-09-16T09:40:24.252 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.252 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.252 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:40:24.252 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.249079+0000", 2024-09-16T09:40:24.252 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6308e-05 2024-09-16T09:40:24.252 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.252 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.252 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:40:24.252 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.249194+0000", 2024-09-16T09:40:24.252 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000115813 2024-09-16T09:40:24.252 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.252 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.253 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:40:24.253 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.249207+0000", 2024-09-16T09:40:24.253 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2986000000000001e-05 2024-09-16T09:40:24.253 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.253 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.253 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:40:24.253 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.500399+0000", 2024-09-16T09:40:24.253 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.25119191699999999 2024-09-16T09:40:24.253 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:24.254 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:24.254 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:24.254 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.254 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.254 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "MOSDPGPush(10.16 417/398 [PushOp(10:6981da6b:::benchmark_data_smithi070_166744_object66882:head, version: 391'4085, data_included: [0~65536], data_size: 65536, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(10:6981da6b:::benchmark_data_smithi070_166744_object66882:head@391'4085, size: 65536, copy_subset: [(0, 65536)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 65536, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-16T09:40:24.254 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:39:25.253875+0000", 2024-09-16T09:40:24.254 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 58.908970191000002, 2024-09-16T09:40:24.254 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.25286863599999998, 2024-09-16T09:40:24.254 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:40:24.254 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:40:24.254 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T09:40:24.254 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:40:24.254 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.255 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:40:24.255 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.253875+0000", 2024-09-16T09:40:24.255 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:24.255 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.255 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.255 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:40:24.255 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.253875+0000", 2024-09-16T09:40:24.255 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:24.255 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.255 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.255 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:40:24.255 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.253877+0000", 2024-09-16T09:40:24.255 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.401e-06 2024-09-16T09:40:24.255 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.256 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.256 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:40:24.256 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.253914+0000", 2024-09-16T09:40:24.256 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7781999999999998e-05 2024-09-16T09:40:24.256 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.256 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.256 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:40:24.256 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.500529+0000", 2024-09-16T09:40:24.256 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.24661467500000001 2024-09-16T09:40:24.256 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.256 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.256 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:40:24.256 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.500551+0000", 2024-09-16T09:40:24.257 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.2459e-05 2024-09-16T09:40:24.257 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.257 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.257 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:40:24.257 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.504460+0000", 2024-09-16T09:40:24.258 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0039085600000000002 2024-09-16T09:40:24.258 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.258 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.258 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:40:24.258 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.504569+0000", 2024-09-16T09:40:24.258 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000108572 2024-09-16T09:40:24.258 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.258 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.258 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:40:24.258 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.506744+0000", 2024-09-16T09:40:24.258 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0021751869999999999 2024-09-16T09:40:24.258 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:24.258 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:24.258 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:24.259 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.259 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.259 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.15975.0:113616 10.0 e417/395 10:063ebb7c:::benchmark_data_smithi070_166744_object113615:head v 417'5658, mlcod=417'5656)", 2024-09-16T09:40:24.259 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:39:25.260314+0000", 2024-09-16T09:40:24.259 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 58.902531013000001, 2024-09-16T09:40:24.259 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.245055678, 2024-09-16T09:40:24.259 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:40:24.259 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:40:24.259 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:40:24.259 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:40:24.259 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.259 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:40:24.260 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.260314+0000", 2024-09-16T09:40:24.260 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:24.260 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.260 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.260 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:40:24.260 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.260314+0000", 2024-09-16T09:40:24.260 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:24.260 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.260 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.260 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:40:24.260 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.260316+0000", 2024-09-16T09:40:24.260 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.4529999999999999e-06 2024-09-16T09:40:24.260 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.260 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.261 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:40:24.261 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.260354+0000", 2024-09-16T09:40:24.261 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8655000000000002e-05 2024-09-16T09:40:24.261 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.261 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.261 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:40:24.261 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.500392+0000", 2024-09-16T09:40:24.261 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.24003755600000001 2024-09-16T09:40:24.261 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.261 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.261 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:40:24.262 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.500407+0000", 2024-09-16T09:40:24.262 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5182e-05 2024-09-16T09:40:24.262 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.262 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.262 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:40:24.262 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.500475+0000", 2024-09-16T09:40:24.262 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.8176999999999995e-05 2024-09-16T09:40:24.262 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.262 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.262 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:40:24.262 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.500498+0000", 2024-09-16T09:40:24.262 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.2191000000000001e-05 2024-09-16T09:40:24.262 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.262 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.263 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:40:24.263 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.505311+0000", 2024-09-16T09:40:24.263 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0048135870000000002 2024-09-16T09:40:24.263 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.263 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.263 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:40:24.263 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.505370+0000", 2024-09-16T09:40:24.263 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8876999999999999e-05 2024-09-16T09:40:24.263 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:24.263 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:24.263 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:24.263 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.263 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.263 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "MOSDPGPush(10.19 417/398 [PushOp(10:9aac0621:::benchmark_data_smithi070_166744_object2219:head, version: 383'132, data_included: [0~65536], data_size: 65536, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(10:9aac0621:::benchmark_data_smithi070_166744_object2219:head@383'132, size: 65536, copy_subset: [(0, 65536)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 65536, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-16T09:40:24.264 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:39:25.261549+0000", 2024-09-16T09:40:24.264 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 58.901295998000002, 2024-09-16T09:40:24.264 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.259429627, 2024-09-16T09:40:24.264 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:40:24.264 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:40:24.264 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T09:40:24.264 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:40:24.264 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.264 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:40:24.265 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.261549+0000", 2024-09-16T09:40:24.265 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:24.265 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.265 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.265 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:40:24.265 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.261549+0000", 2024-09-16T09:40:24.265 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:24.265 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.265 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.265 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:40:24.265 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.261550+0000", 2024-09-16T09:40:24.265 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.1500000000000003e-07 2024-09-16T09:40:24.265 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.265 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.266 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:40:24.266 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.261576+0000", 2024-09-16T09:40:24.266 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.5930000000000001e-05 2024-09-16T09:40:24.266 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.266 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.266 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:40:24.266 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.500469+0000", 2024-09-16T09:40:24.266 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.238892509 2024-09-16T09:40:24.266 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.266 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.266 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:40:24.266 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.500491+0000", 2024-09-16T09:40:24.266 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.2543000000000001e-05 2024-09-16T09:40:24.266 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.267 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.267 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:40:24.267 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.518931+0000", 2024-09-16T09:40:24.267 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.018439766 2024-09-16T09:40:24.267 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.267 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.267 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:40:24.267 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.518970+0000", 2024-09-16T09:40:24.267 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8656999999999999e-05 2024-09-16T09:40:24.267 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.267 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.267 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:40:24.267 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.520979+0000", 2024-09-16T09:40:24.267 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0020094069999999999 2024-09-16T09:40:24.268 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:24.268 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:24.268 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:24.268 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.268 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.268 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "MOSDPGPush(10.14 429/398 [PushOp(10:2b299b11:::benchmark_data_smithi070_166744_object49065:head, version: 389'3107, data_included: [0~65536], data_size: 65536, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(10:2b299b11:::benchmark_data_smithi070_166744_object49065:head@389'3107, size: 65536, copy_subset: [(0, 65536)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 65536, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-16T09:40:24.268 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:40:01.341126+0000", 2024-09-16T09:40:24.268 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 22.821719475999998, 2024-09-16T09:40:24.268 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.254867177, 2024-09-16T09:40:24.268 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:40:24.268 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:40:24.268 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T09:40:24.268 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:40:24.268 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.269 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:40:24.269 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.341126+0000", 2024-09-16T09:40:24.269 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:24.269 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.269 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.269 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:40:24.269 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.341126+0000", 2024-09-16T09:40:24.269 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:24.269 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.269 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.269 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:40:24.269 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.341127+0000", 2024-09-16T09:40:24.269 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.421e-06 2024-09-16T09:40:24.270 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.270 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.270 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:40:24.270 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.341178+0000", 2024-09-16T09:40:24.270 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0847000000000003e-05 2024-09-16T09:40:24.270 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.270 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.270 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:40:24.270 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.341201+0000", 2024-09-16T09:40:24.270 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3283999999999999e-05 2024-09-16T09:40:24.270 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.270 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.270 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:40:24.270 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.341220+0000", 2024-09-16T09:40:24.271 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.8289000000000001e-05 2024-09-16T09:40:24.271 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.271 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.271 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:40:24.271 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.594029+0000", 2024-09-16T09:40:24.271 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.25280975999999999 2024-09-16T09:40:24.271 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.271 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.271 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:40:24.271 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.594043+0000", 2024-09-16T09:40:24.271 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3455000000000001e-05 2024-09-16T09:40:24.271 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.271 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.272 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:40:24.272 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.595993+0000", 2024-09-16T09:40:24.272 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001950121 2024-09-16T09:40:24.272 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:24.272 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:24.272 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:24.272 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.272 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.272 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.15975.0:150386 10.14 e429/398 10:2e95170f:::benchmark_data_smithi070_166744_object36030:head v 429'6995, mlcod=389'3105)", 2024-09-16T09:40:24.272 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:40:01.341250+0000", 2024-09-16T09:40:24.272 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 22.821595157000001, 2024-09-16T09:40:24.272 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.26185008100000001, 2024-09-16T09:40:24.272 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:40:24.272 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:40:24.273 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:40:24.273 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:40:24.273 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.273 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:40:24.273 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.341250+0000", 2024-09-16T09:40:24.273 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:24.273 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.273 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.273 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:40:24.273 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.341250+0000", 2024-09-16T09:40:24.273 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:24.273 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.273 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.273 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:40:24.274 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.341251+0000", 2024-09-16T09:40:24.274 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.1699999999999999e-07 2024-09-16T09:40:24.274 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.274 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.274 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:40:24.274 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.341256+0000", 2024-09-16T09:40:24.274 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.1170000000000003e-06 2024-09-16T09:40:24.274 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.274 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.274 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:40:24.275 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.341263+0000", 2024-09-16T09:40:24.275 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.0149999999999998e-06 2024-09-16T09:40:24.275 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.275 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.275 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:40:24.275 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.341267+0000", 2024-09-16T09:40:24.275 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.7620000000000003e-06 2024-09-16T09:40:24.275 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.275 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.275 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:40:24.275 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.591255+0000", 2024-09-16T09:40:24.275 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.24998795700000001 2024-09-16T09:40:24.275 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.276 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.276 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:40:24.276 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.591287+0000", 2024-09-16T09:40:24.276 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1625999999999998e-05 2024-09-16T09:40:24.276 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.276 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.276 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:40:24.276 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.603044+0000", 2024-09-16T09:40:24.276 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.011756679000000001 2024-09-16T09:40:24.276 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.276 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.276 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:40:24.276 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.603100+0000", 2024-09-16T09:40:24.276 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.6508000000000002e-05 2024-09-16T09:40:24.277 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:24.277 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:24.277 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:24.277 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.277 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.277 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.15975.0:150389 10.2 e429/395 10:41a08be2:::benchmark_data_smithi070_166744_object36033:head v 429'7013, mlcod=429'7011)", 2024-09-16T09:40:24.277 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:40:01.341276+0000", 2024-09-16T09:40:24.277 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 22.821569137000001, 2024-09-16T09:40:24.277 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.25004327399999998, 2024-09-16T09:40:24.277 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:40:24.277 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:40:24.277 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:40:24.277 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:40:24.278 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.278 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:40:24.278 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.341276+0000", 2024-09-16T09:40:24.278 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:24.278 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.278 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.278 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:40:24.278 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.341276+0000", 2024-09-16T09:40:24.278 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:24.278 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.278 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.278 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:40:24.278 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.341276+0000", 2024-09-16T09:40:24.279 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.7500000000000001e-07 2024-09-16T09:40:24.279 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.279 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.279 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:40:24.279 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.341280+0000", 2024-09-16T09:40:24.279 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2799999999999999e-06 2024-09-16T09:40:24.279 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.279 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.279 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:40:24.279 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.341285+0000", 2024-09-16T09:40:24.279 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5690000000000004e-06 2024-09-16T09:40:24.279 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.279 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.279 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:40:24.279 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.341289+0000", 2024-09-16T09:40:24.280 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7919999999999999e-06 2024-09-16T09:40:24.280 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.280 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.280 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:40:24.280 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.341352+0000", 2024-09-16T09:40:24.280 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.2731999999999995e-05 2024-09-16T09:40:24.280 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.280 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.280 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:40:24.280 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.341377+0000", 2024-09-16T09:40:24.280 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.4686e-05 2024-09-16T09:40:24.280 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.281 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.281 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:40:24.281 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.591266+0000", 2024-09-16T09:40:24.281 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.24988992500000001 2024-09-16T09:40:24.281 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.281 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.281 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:40:24.281 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.591319+0000", 2024-09-16T09:40:24.281 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.3015e-05 2024-09-16T09:40:24.281 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:24.281 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:24.281 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:24.281 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.281 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.282 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.15975.0:150393 10.b e429/376 10:da2f4e9f:::benchmark_data_smithi070_166744_object36037:head v 429'9364, mlcod=429'9361)", 2024-09-16T09:40:24.282 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:40:01.345049+0000", 2024-09-16T09:40:24.282 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 22.817796286, 2024-09-16T09:40:24.282 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.25539264, 2024-09-16T09:40:24.282 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:40:24.282 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:40:24.282 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:40:24.282 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:40:24.282 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.282 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:40:24.282 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.345049+0000", 2024-09-16T09:40:24.282 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:24.282 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.282 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.282 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:40:24.282 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.345049+0000", 2024-09-16T09:40:24.283 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:24.283 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.283 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.283 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:40:24.283 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.345050+0000", 2024-09-16T09:40:24.283 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.7300000000000004e-07 2024-09-16T09:40:24.283 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.283 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.283 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:40:24.283 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.345056+0000", 2024-09-16T09:40:24.283 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1550000000000004e-06 2024-09-16T09:40:24.283 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.283 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.283 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:40:24.283 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.591338+0000", 2024-09-16T09:40:24.284 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.24628182400000001 2024-09-16T09:40:24.284 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.284 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.284 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:40:24.284 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.591348+0000", 2024-09-16T09:40:24.284 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.9699999999999994e-06 2024-09-16T09:40:24.284 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.284 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.284 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:40:24.284 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.591412+0000", 2024-09-16T09:40:24.284 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.4387000000000005e-05 2024-09-16T09:40:24.284 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.284 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.284 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:40:24.285 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.591426+0000", 2024-09-16T09:40:24.285 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.411e-05 2024-09-16T09:40:24.285 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.285 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.285 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:40:24.285 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.600407+0000", 2024-09-16T09:40:24.285 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0089803020000000008 2024-09-16T09:40:24.285 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.285 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.285 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:40:24.286 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.600442+0000", 2024-09-16T09:40:24.286 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4919000000000002e-05 2024-09-16T09:40:24.286 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:24.286 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:24.286 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:24.286 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.286 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.286 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.15975.0:150397 10.14 e429/398 10:2c54e8e2:::benchmark_data_smithi070_166744_object36041:head v 429'6996, mlcod=389'3105)", 2024-09-16T09:40:24.286 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:40:01.346819+0000", 2024-09-16T09:40:24.286 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 22.816026744999998, 2024-09-16T09:40:24.286 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.25636429500000002, 2024-09-16T09:40:24.286 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:40:24.286 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:40:24.286 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:40:24.287 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:40:24.287 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.287 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:40:24.287 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.346819+0000", 2024-09-16T09:40:24.287 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:24.287 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.287 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.287 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:40:24.287 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.346819+0000", 2024-09-16T09:40:24.287 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:24.287 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.287 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.287 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:40:24.288 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.346820+0000", 2024-09-16T09:40:24.288 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0750000000000001e-06 2024-09-16T09:40:24.288 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.288 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.288 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:40:24.288 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.346829+0000", 2024-09-16T09:40:24.288 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.1330000000000005e-06 2024-09-16T09:40:24.288 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.288 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.288 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:40:24.288 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.592302+0000", 2024-09-16T09:40:24.288 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.24547290199999999 2024-09-16T09:40:24.288 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.288 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.289 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:40:24.289 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.592309+0000", 2024-09-16T09:40:24.289 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.2629999999999997e-06 2024-09-16T09:40:24.289 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.289 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.289 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:40:24.289 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.596075+0000", 2024-09-16T09:40:24.289 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0037657459999999999 2024-09-16T09:40:24.289 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.289 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.289 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:40:24.289 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.596096+0000", 2024-09-16T09:40:24.289 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0801999999999998e-05 2024-09-16T09:40:24.289 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.290 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.290 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:40:24.290 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.603145+0000", 2024-09-16T09:40:24.290 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0070491750000000004 2024-09-16T09:40:24.290 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:24.290 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:24.290 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:40:24.290 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.603183+0000", 2024-09-16T09:40:24.290 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8198999999999998e-05 2024-09-16T09:40:24.290 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:24.290 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:24.290 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:24.290 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:24.290 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:24.291 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T09:40:24.291 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:40:24.326 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:24.327 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:24.328 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:40:24.429 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:40:24.463 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:24.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:24.465 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:40:24.518 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:24 smithi070 ceph-mon[30388]: 5.4 scrub starts 2024-09-16T09:40:24.518 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:24 smithi070 ceph-mon[30388]: 5.4 scrub ok 2024-09-16T09:40:24.519 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:24 smithi070 ceph-mon[30388]: 3.12 scrub starts 2024-09-16T09:40:24.519 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:24 smithi070 ceph-mon[30388]: 3.12 scrub ok 2024-09-16T09:40:24.519 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:24 smithi070 ceph-mon[30388]: 10.19 scrub starts 2024-09-16T09:40:24.519 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:24 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:40:24.519 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:24 smithi070 ceph-mon[30388]: 4.1c scrub starts 2024-09-16T09:40:24.519 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:24 smithi070 ceph-mon[30388]: 4.1c scrub ok 2024-09-16T09:40:24.519 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:24 smithi070 ceph-mon[30388]: 10.19 scrub ok 2024-09-16T09:40:24.519 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:24 smithi070 ceph-mon[30388]: 1.f scrub starts 2024-09-16T09:40:24.520 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:24 smithi070 ceph-mon[30388]: 1.f scrub ok 2024-09-16T09:40:24.520 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:24 smithi070 ceph-mon[30388]: 5.18 scrub starts 2024-09-16T09:40:24.520 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:24 smithi070 ceph-mon[30388]: 5.18 scrub ok 2024-09-16T09:40:24.520 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:24 smithi070 ceph-mon[40691]: 5.4 scrub starts 2024-09-16T09:40:24.520 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:24 smithi070 ceph-mon[40691]: 5.4 scrub ok 2024-09-16T09:40:24.520 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:24 smithi070 ceph-mon[40691]: 3.12 scrub starts 2024-09-16T09:40:24.520 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:24 smithi070 ceph-mon[40691]: 3.12 scrub ok 2024-09-16T09:40:24.520 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:24 smithi070 ceph-mon[40691]: 10.19 scrub starts 2024-09-16T09:40:24.520 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:24 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:40:24.521 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:24 smithi070 ceph-mon[40691]: 4.1c scrub starts 2024-09-16T09:40:24.521 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:24 smithi070 ceph-mon[40691]: 4.1c scrub ok 2024-09-16T09:40:24.521 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:24 smithi070 ceph-mon[40691]: 10.19 scrub ok 2024-09-16T09:40:24.521 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:24 smithi070 ceph-mon[40691]: 1.f scrub starts 2024-09-16T09:40:24.521 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:24 smithi070 ceph-mon[40691]: 1.f scrub ok 2024-09-16T09:40:24.521 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:24 smithi070 ceph-mon[40691]: 5.18 scrub starts 2024-09-16T09:40:24.521 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:24 smithi070 ceph-mon[40691]: 5.18 scrub ok 2024-09-16T09:40:24.566 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:40:24.601 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:24.601 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:24.602 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:40:24.703 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:40:24.725 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:24.725 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:24.726 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:40:24.826 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:40:24.860 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:24.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:24.861 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:40:24.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:24 smithi084 ceph-mon[32629]: 5.4 scrub starts 2024-09-16T09:40:24.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:24 smithi084 ceph-mon[32629]: 5.4 scrub ok 2024-09-16T09:40:24.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:24 smithi084 ceph-mon[32629]: 3.12 scrub starts 2024-09-16T09:40:24.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:24 smithi084 ceph-mon[32629]: 3.12 scrub ok 2024-09-16T09:40:24.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:24 smithi084 ceph-mon[32629]: 10.19 scrub starts 2024-09-16T09:40:24.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:24 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:40:24.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:24 smithi084 ceph-mon[32629]: 4.1c scrub starts 2024-09-16T09:40:24.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:24 smithi084 ceph-mon[32629]: 4.1c scrub ok 2024-09-16T09:40:24.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:24 smithi084 ceph-mon[32629]: 10.19 scrub ok 2024-09-16T09:40:24.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:24 smithi084 ceph-mon[32629]: 1.f scrub starts 2024-09-16T09:40:24.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:24 smithi084 ceph-mon[32629]: 1.f scrub ok 2024-09-16T09:40:24.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:24 smithi084 ceph-mon[32629]: 5.18 scrub starts 2024-09-16T09:40:24.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:24 smithi084 ceph-mon[32629]: 5.18 scrub ok 2024-09-16T09:40:24.952 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:40:24.952 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:40:24.952 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:40:24.953 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:40:24.953 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:40:24.953 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:40:24.953 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:40:24.953 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:40:24.953 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:40:24.953 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:40:24.953 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:40:24.953 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:40:24.962 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:40:24.985 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:40:24.985 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:40:24.992 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:24.993 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:24.993 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:40:25.023 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:40:25.023 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:40:25.094 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:40:25.133 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:25.134 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:25.134 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:40:25.165 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-16T09:40:25.235 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:40:25.272 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:25.273 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:25.273 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:40:25.374 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:40:25.405 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:25.405 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:25.406 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:40:25.506 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:40:25.544 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:25.545 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:25.545 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:40:25.646 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:40:25.679 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:25.680 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:25.680 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:40:25.690 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:25 smithi070 ceph-mon[30388]: 3.19 scrub starts 2024-09-16T09:40:25.690 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:25 smithi070 ceph-mon[30388]: 3.19 scrub ok 2024-09-16T09:40:25.690 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:25 smithi070 ceph-mon[30388]: pgmap v816: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 3.3 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.38k op/s; 5517/164175 objects misplaced (3.360%); 4.0 MiB/s, 64 objects/s recovering 2024-09-16T09:40:25.690 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:25 smithi070 ceph-mon[30388]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 1 pg undersized) 2024-09-16T09:40:25.690 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:25 smithi070 ceph-mon[30388]: 4.f scrub starts 2024-09-16T09:40:25.691 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:25 smithi070 ceph-mon[30388]: 4.f scrub ok 2024-09-16T09:40:25.691 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:25 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3084847247' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:40:25.691 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:25 smithi070 ceph-mon[40691]: 3.19 scrub starts 2024-09-16T09:40:25.691 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:25 smithi070 ceph-mon[40691]: 3.19 scrub ok 2024-09-16T09:40:25.691 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:25 smithi070 ceph-mon[40691]: pgmap v816: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 3.3 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.38k op/s; 5517/164175 objects misplaced (3.360%); 4.0 MiB/s, 64 objects/s recovering 2024-09-16T09:40:25.691 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:25 smithi070 ceph-mon[40691]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 1 pg undersized) 2024-09-16T09:40:25.691 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:25 smithi070 ceph-mon[40691]: 4.f scrub starts 2024-09-16T09:40:25.691 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:25 smithi070 ceph-mon[40691]: 4.f scrub ok 2024-09-16T09:40:25.691 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:25 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3084847247' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:40:25.781 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:40:25.795 INFO:teuthology.orchestra.run.smithi070.stderr:no valid command found; 10 closest matches: 2024-09-16T09:40:25.795 INFO:teuthology.orchestra.run.smithi070.stderr:0 2024-09-16T09:40:25.795 INFO:teuthology.orchestra.run.smithi070.stderr:1 2024-09-16T09:40:25.795 INFO:teuthology.orchestra.run.smithi070.stderr:2 2024-09-16T09:40:25.795 INFO:teuthology.orchestra.run.smithi070.stderr:abort 2024-09-16T09:40:25.796 INFO:teuthology.orchestra.run.smithi070.stderr:assert 2024-09-16T09:40:25.796 INFO:teuthology.orchestra.run.smithi070.stderr:bench [] [] [] [] 2024-09-16T09:40:25.796 INFO:teuthology.orchestra.run.smithi070.stderr:bluefs debug_inject_read_zeros 2024-09-16T09:40:25.796 INFO:teuthology.orchestra.run.smithi070.stderr:bluefs files list 2024-09-16T09:40:25.796 INFO:teuthology.orchestra.run.smithi070.stderr:bluefs stats 2024-09-16T09:40:25.796 INFO:teuthology.orchestra.run.smithi070.stderr:bluestore allocator dump block 2024-09-16T09:40:25.796 INFO:teuthology.orchestra.run.smithi070.stderr:Error EINVAL: invalid command 2024-09-16T09:40:25.819 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:25.819 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:25.820 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:40:25.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:25 smithi084 ceph-mon[32629]: 3.19 scrub starts 2024-09-16T09:40:25.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:25 smithi084 ceph-mon[32629]: 3.19 scrub ok 2024-09-16T09:40:25.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:25 smithi084 ceph-mon[32629]: pgmap v816: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 3.3 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.38k op/s; 5517/164175 objects misplaced (3.360%); 4.0 MiB/s, 64 objects/s recovering 2024-09-16T09:40:25.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:25 smithi084 ceph-mon[32629]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 1 pg undersized) 2024-09-16T09:40:25.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:25 smithi084 ceph-mon[32629]: 4.f scrub starts 2024-09-16T09:40:25.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:25 smithi084 ceph-mon[32629]: 4.f scrub ok 2024-09-16T09:40:25.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:25 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3084847247' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:40:25.921 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:40:25.954 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:25.955 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:25.955 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:40:26.056 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:40:26.087 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:26.088 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:26.088 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:40:26.189 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:40:26.218 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:26.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:26.219 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:40:26.300 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.2 dump_ops_in_flight 2024-09-16T09:40:26.320 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:40:26.352 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:26.352 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:26.353 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:40:26.454 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:40:26.496 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:26.496 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:26.496 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:40:26.597 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:40:26.646 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:26.647 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:26.647 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:40:26.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:26 smithi070 ceph-mon[30388]: 5.8 scrub starts 2024-09-16T09:40:26.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:26 smithi070 ceph-mon[30388]: 5.8 scrub ok 2024-09-16T09:40:26.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:26 smithi070 ceph-mon[30388]: 4.18 scrub starts 2024-09-16T09:40:26.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:26 smithi070 ceph-mon[30388]: 4.18 scrub ok 2024-09-16T09:40:26.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:26 smithi070 ceph-mon[30388]: 3.4 scrub starts 2024-09-16T09:40:26.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:26 smithi070 ceph-mon[30388]: 3.4 scrub ok 2024-09-16T09:40:26.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:26 smithi070 ceph-mon[40691]: 5.8 scrub starts 2024-09-16T09:40:26.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:26 smithi070 ceph-mon[40691]: 5.8 scrub ok 2024-09-16T09:40:26.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:26 smithi070 ceph-mon[40691]: 4.18 scrub starts 2024-09-16T09:40:26.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:26 smithi070 ceph-mon[40691]: 4.18 scrub ok 2024-09-16T09:40:26.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:26 smithi070 ceph-mon[40691]: 3.4 scrub starts 2024-09-16T09:40:26.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:26 smithi070 ceph-mon[40691]: 3.4 scrub ok 2024-09-16T09:40:26.748 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:40:26.789 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:26.789 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:26.790 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:40:26.891 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:40:26.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:26 smithi084 ceph-mon[32629]: 5.8 scrub starts 2024-09-16T09:40:26.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:26 smithi084 ceph-mon[32629]: 5.8 scrub ok 2024-09-16T09:40:26.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:26 smithi084 ceph-mon[32629]: 4.18 scrub starts 2024-09-16T09:40:26.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:26 smithi084 ceph-mon[32629]: 4.18 scrub ok 2024-09-16T09:40:26.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:26 smithi084 ceph-mon[32629]: 3.4 scrub starts 2024-09-16T09:40:26.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:26 smithi084 ceph-mon[32629]: 3.4 scrub ok 2024-09-16T09:40:26.923 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:26.924 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:26.924 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:40:27.025 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:40:27.064 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:27.065 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:27.066 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:40:27.168 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:40:27.206 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:27.207 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:27.207 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:40:27.308 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:40:27.337 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:27.338 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:27.339 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:40:27.439 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:40:27.475 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:27.476 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:27.476 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:40:27.577 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:40:27.607 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:27.608 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:27.609 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:40:27.710 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:40:27.755 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:27.756 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:27.756 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:40:27.857 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:40:27.893 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:27.894 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:27.894 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:40:27.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:27 smithi084 ceph-mon[32629]: pgmap v817: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 3.3 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.48k op/s; 4881/161931 objects misplaced (3.014%); 4.0 MiB/s, 64 objects/s recovering 2024-09-16T09:40:27.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:27 smithi084 ceph-mon[32629]: 6.16 scrub starts 2024-09-16T09:40:27.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:27 smithi084 ceph-mon[32629]: 6.16 scrub ok 2024-09-16T09:40:27.915 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:27 smithi070 ceph-mon[30388]: pgmap v817: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 3.3 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.48k op/s; 4881/161931 objects misplaced (3.014%); 4.0 MiB/s, 64 objects/s recovering 2024-09-16T09:40:27.915 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:27 smithi070 ceph-mon[30388]: 6.16 scrub starts 2024-09-16T09:40:27.915 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:27 smithi070 ceph-mon[30388]: 6.16 scrub ok 2024-09-16T09:40:27.915 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:27 smithi070 ceph-mon[40691]: pgmap v817: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 3.3 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.48k op/s; 4881/161931 objects misplaced (3.014%); 4.0 MiB/s, 64 objects/s recovering 2024-09-16T09:40:27.916 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:27 smithi070 ceph-mon[40691]: 6.16 scrub starts 2024-09-16T09:40:27.916 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:27 smithi070 ceph-mon[40691]: 6.16 scrub ok 2024-09-16T09:40:27.995 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:40:28.030 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:28.031 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:28.031 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:40:28.132 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:40:28.170 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:28.171 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:28.171 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:40:28.241 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:40:28.272 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:40:28.275 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-16T09:40:28.276 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-16T09:40:28.276 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph tell osd.7 'trim stale osdmaps' 2024-09-16T09:40:28.302 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:28.303 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:28.303 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:40:28.404 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:40:28.448 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:28.448 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:28.448 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:40:28.515 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:40:28.535 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:28 smithi070 ceph-mon[40691]: 1.2 scrub starts 2024-09-16T09:40:28.535 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:28 smithi070 ceph-mon[40691]: 1.2 scrub ok 2024-09-16T09:40:28.535 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:28 smithi070 ceph-mon[40691]: 6.1f scrub starts 2024-09-16T09:40:28.535 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:28 smithi070 ceph-mon[40691]: 6.1f scrub ok 2024-09-16T09:40:28.535 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:28 smithi070 ceph-mon[40691]: 5.16 scrub starts 2024-09-16T09:40:28.535 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:28 smithi070 ceph-mon[40691]: 5.16 scrub ok 2024-09-16T09:40:28.535 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:28 smithi070 ceph-mon[30388]: 1.2 scrub starts 2024-09-16T09:40:28.536 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:28 smithi070 ceph-mon[30388]: 1.2 scrub ok 2024-09-16T09:40:28.536 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:28 smithi070 ceph-mon[30388]: 6.1f scrub starts 2024-09-16T09:40:28.536 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:28 smithi070 ceph-mon[30388]: 6.1f scrub ok 2024-09-16T09:40:28.536 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:28 smithi070 ceph-mon[30388]: 5.16 scrub starts 2024-09-16T09:40:28.536 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:28 smithi070 ceph-mon[30388]: 5.16 scrub ok 2024-09-16T09:40:28.549 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:40:28.596 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:28.597 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:28.600 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:40:28.700 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:40:28.739 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:28.740 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:28.740 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:40:28.841 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:40:28.875 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:28.875 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:28.876 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:40:28.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:28 smithi084 ceph-mon[32629]: 1.2 scrub starts 2024-09-16T09:40:28.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:28 smithi084 ceph-mon[32629]: 1.2 scrub ok 2024-09-16T09:40:28.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:28 smithi084 ceph-mon[32629]: 6.1f scrub starts 2024-09-16T09:40:28.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:28 smithi084 ceph-mon[32629]: 6.1f scrub ok 2024-09-16T09:40:28.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:28 smithi084 ceph-mon[32629]: 5.16 scrub starts 2024-09-16T09:40:28.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:28 smithi084 ceph-mon[32629]: 5.16 scrub ok 2024-09-16T09:40:28.977 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:40:29.013 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:29.014 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:29.015 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:40:29.116 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:40:29.148 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:29.149 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:29.149 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:40:29.250 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:40:29.292 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:29.293 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:29.293 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:40:29.394 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:40:29.425 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:29.426 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:29.426 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:40:29.471 INFO:teuthology.orchestra.run.smithi070.stderr:nodeep-scrub is unset 2024-09-16T09:40:29.496 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T09:40:29.497 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:40:29.527 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:40:29.542 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T09:40:29.567 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:29.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:29.569 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:40:29.669 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:40:29.713 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:29.713 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:29.714 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:40:29.815 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:40:29.848 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:29.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:29.849 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:40:29.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:29 smithi084 ceph-mon[32629]: pgmap v818: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 3.3 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 712 op/s; 4881/161931 objects misplaced (3.014%); 1.3 MiB/s, 21 objects/s recovering 2024-09-16T09:40:29.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:29 smithi084 ceph-mon[32629]: 1.14 scrub starts 2024-09-16T09:40:29.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:29 smithi084 ceph-mon[32629]: 1.14 scrub ok 2024-09-16T09:40:29.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:29 smithi084 ceph-mon[32629]: 4.6 scrub starts 2024-09-16T09:40:29.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:29 smithi084 ceph-mon[32629]: 4.6 scrub ok 2024-09-16T09:40:29.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:29 smithi084 ceph-mon[32629]: 1.5 scrub starts 2024-09-16T09:40:29.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:29 smithi084 ceph-mon[32629]: 1.5 scrub ok 2024-09-16T09:40:29.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:29 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3336602534' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:40:29.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:29 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:40:29.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:29 smithi070 ceph-mon[30388]: pgmap v818: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 3.3 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 712 op/s; 4881/161931 objects misplaced (3.014%); 1.3 MiB/s, 21 objects/s recovering 2024-09-16T09:40:29.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:29 smithi070 ceph-mon[30388]: 1.14 scrub starts 2024-09-16T09:40:29.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:29 smithi070 ceph-mon[30388]: 1.14 scrub ok 2024-09-16T09:40:29.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:29 smithi070 ceph-mon[30388]: 4.6 scrub starts 2024-09-16T09:40:29.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:29 smithi070 ceph-mon[30388]: 4.6 scrub ok 2024-09-16T09:40:29.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:29 smithi070 ceph-mon[30388]: 1.5 scrub starts 2024-09-16T09:40:29.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:29 smithi070 ceph-mon[30388]: 1.5 scrub ok 2024-09-16T09:40:29.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:29 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3336602534' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:40:29.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:29 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:40:29.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:29 smithi070 ceph-mon[40691]: pgmap v818: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 3.3 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 712 op/s; 4881/161931 objects misplaced (3.014%); 1.3 MiB/s, 21 objects/s recovering 2024-09-16T09:40:29.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:29 smithi070 ceph-mon[40691]: 1.14 scrub starts 2024-09-16T09:40:29.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:29 smithi070 ceph-mon[40691]: 1.14 scrub ok 2024-09-16T09:40:29.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:29 smithi070 ceph-mon[40691]: 4.6 scrub starts 2024-09-16T09:40:29.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:29 smithi070 ceph-mon[40691]: 4.6 scrub ok 2024-09-16T09:40:29.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:29 smithi070 ceph-mon[40691]: 1.5 scrub starts 2024-09-16T09:40:29.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:29 smithi070 ceph-mon[40691]: 1.5 scrub ok 2024-09-16T09:40:29.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:29 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3336602534' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:40:29.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:29 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:40:29.950 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:40:29.968 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:40:30.008 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:30.009 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:30.009 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:40:30.110 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:40:30.146 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:30.146 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:30.146 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:40:30.248 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:40:30.284 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:30.285 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:30.285 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:40:30.386 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:40:30.422 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:30.423 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:30.423 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:40:30.524 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:40:30.562 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:30.563 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:30.563 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:40:30.664 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:40:30.697 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:30.698 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:30.698 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:40:30.740 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:30 smithi070 ceph-mon[30388]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-16T09:40:30.740 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:30 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-16T09:40:30.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:30 smithi070 ceph-mon[30388]: osdmap e441: 8 total, 8 up, 8 in 2024-09-16T09:40:30.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:30 smithi070 ceph-mon[40691]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-16T09:40:30.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:30 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-16T09:40:30.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:30 smithi070 ceph-mon[40691]: osdmap e441: 8 total, 8 up, 8 in 2024-09-16T09:40:30.791 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.2 dump_blocked_ops 2024-09-16T09:40:30.799 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:40:30.830 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:30.831 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:30.831 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:40:30.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:30 smithi084 ceph-mon[32629]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-16T09:40:30.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:30 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-16T09:40:30.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:30 smithi084 ceph-mon[32629]: osdmap e441: 8 total, 8 up, 8 in 2024-09-16T09:40:30.932 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:40:30.958 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:30.959 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:30.959 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:40:31.060 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:40:31.093 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:31.094 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:31.094 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:40:31.195 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:40:31.247 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:31.248 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:31.248 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:40:31.349 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:40:31.382 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:31.382 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:31.383 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:40:31.484 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:40:31.506 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:40:31.516 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:31.517 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:31.517 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:40:31.563 INFO:teuthology.orchestra.run.smithi070.stderr:no valid command found; 10 closest matches: 2024-09-16T09:40:31.563 INFO:teuthology.orchestra.run.smithi070.stderr:0 2024-09-16T09:40:31.563 INFO:teuthology.orchestra.run.smithi070.stderr:1 2024-09-16T09:40:31.563 INFO:teuthology.orchestra.run.smithi070.stderr:2 2024-09-16T09:40:31.563 INFO:teuthology.orchestra.run.smithi070.stderr:abort 2024-09-16T09:40:31.563 INFO:teuthology.orchestra.run.smithi070.stderr:assert 2024-09-16T09:40:31.563 INFO:teuthology.orchestra.run.smithi070.stderr:bench [] [] [] [] 2024-09-16T09:40:31.563 INFO:teuthology.orchestra.run.smithi070.stderr:bluefs debug_inject_read_zeros 2024-09-16T09:40:31.563 INFO:teuthology.orchestra.run.smithi070.stderr:bluefs files list 2024-09-16T09:40:31.563 INFO:teuthology.orchestra.run.smithi070.stderr:bluefs stats 2024-09-16T09:40:31.563 INFO:teuthology.orchestra.run.smithi070.stderr:bluestore allocator dump block 2024-09-16T09:40:31.563 INFO:teuthology.orchestra.run.smithi070.stderr:Error EINVAL: invalid command 2024-09-16T09:40:31.571 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:40:31.618 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:40:31.642 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:31.643 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:31.643 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:40:31.743 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:40:31.775 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:31.776 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:31.776 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:40:31.877 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:40:31.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:31 smithi084 ceph-mon[32629]: pgmap v820: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 2.9 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 907 op/s; 4167/140970 objects misplaced (2.956%) 2024-09-16T09:40:31.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:31 smithi084 ceph-mon[32629]: 5.17 scrub starts 2024-09-16T09:40:31.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:31 smithi084 ceph-mon[32629]: 5.17 scrub ok 2024-09-16T09:40:31.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:31 smithi084 ceph-mon[32629]: 4.1a scrub starts 2024-09-16T09:40:31.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:31 smithi084 ceph-mon[32629]: 4.1a scrub ok 2024-09-16T09:40:31.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:31 smithi084 ceph-mon[32629]: 3.6 scrub starts 2024-09-16T09:40:31.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:31 smithi084 ceph-mon[32629]: 3.6 scrub ok 2024-09-16T09:40:31.914 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:31.915 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:31.915 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:40:31.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:31 smithi070 ceph-mon[30388]: pgmap v820: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 2.9 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 907 op/s; 4167/140970 objects misplaced (2.956%) 2024-09-16T09:40:31.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:31 smithi070 ceph-mon[30388]: 5.17 scrub starts 2024-09-16T09:40:31.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:31 smithi070 ceph-mon[30388]: 5.17 scrub ok 2024-09-16T09:40:31.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:31 smithi070 ceph-mon[30388]: 4.1a scrub starts 2024-09-16T09:40:31.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:31 smithi070 ceph-mon[30388]: 4.1a scrub ok 2024-09-16T09:40:31.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:31 smithi070 ceph-mon[30388]: 3.6 scrub starts 2024-09-16T09:40:31.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:31 smithi070 ceph-mon[30388]: 3.6 scrub ok 2024-09-16T09:40:31.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:31 smithi070 ceph-mon[40691]: pgmap v820: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 2.9 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 907 op/s; 4167/140970 objects misplaced (2.956%) 2024-09-16T09:40:31.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:31 smithi070 ceph-mon[40691]: 5.17 scrub starts 2024-09-16T09:40:31.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:31 smithi070 ceph-mon[40691]: 5.17 scrub ok 2024-09-16T09:40:31.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:31 smithi070 ceph-mon[40691]: 4.1a scrub starts 2024-09-16T09:40:31.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:31 smithi070 ceph-mon[40691]: 4.1a scrub ok 2024-09-16T09:40:31.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:31 smithi070 ceph-mon[40691]: 3.6 scrub starts 2024-09-16T09:40:31.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:31 smithi070 ceph-mon[40691]: 3.6 scrub ok 2024-09-16T09:40:32.016 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:40:32.048 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:32.049 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:32.049 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:40:32.150 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:40:32.186 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:32.187 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:32.187 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:40:32.288 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:40:32.323 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:32.323 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:32.324 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:40:32.420 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:40:31 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:40:31] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:40:32.425 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:40:32.455 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:32.455 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:32.456 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:40:32.556 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:40:32.591 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:32.591 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:32.592 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:40:32.693 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:40:32.719 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:32.719 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:32.720 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:40:32.821 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:40:32.853 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:32.854 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:32.854 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:40:32.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:32 smithi084 ceph-mon[32629]: 1.13 scrub starts 2024-09-16T09:40:32.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:32 smithi084 ceph-mon[32629]: 1.13 scrub ok 2024-09-16T09:40:32.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:32 smithi084 ceph-mon[32629]: 6.3 scrub starts 2024-09-16T09:40:32.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:32 smithi084 ceph-mon[32629]: 6.3 scrub ok 2024-09-16T09:40:32.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:32 smithi084 ceph-mon[32629]: 5.6 scrub starts 2024-09-16T09:40:32.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:32 smithi084 ceph-mon[32629]: 5.6 scrub ok 2024-09-16T09:40:32.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:32 smithi084 ceph-mon[32629]: 6.c scrub starts 2024-09-16T09:40:32.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:32 smithi084 ceph-mon[32629]: 6.c scrub ok 2024-09-16T09:40:32.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:32 smithi070 ceph-mon[40691]: 1.13 scrub starts 2024-09-16T09:40:32.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:32 smithi070 ceph-mon[40691]: 1.13 scrub ok 2024-09-16T09:40:32.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:32 smithi070 ceph-mon[40691]: 6.3 scrub starts 2024-09-16T09:40:32.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:32 smithi070 ceph-mon[40691]: 6.3 scrub ok 2024-09-16T09:40:32.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:32 smithi070 ceph-mon[40691]: 5.6 scrub starts 2024-09-16T09:40:32.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:32 smithi070 ceph-mon[40691]: 5.6 scrub ok 2024-09-16T09:40:32.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:32 smithi070 ceph-mon[40691]: 6.c scrub starts 2024-09-16T09:40:32.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:32 smithi070 ceph-mon[40691]: 6.c scrub ok 2024-09-16T09:40:32.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:32 smithi070 ceph-mon[30388]: 1.13 scrub starts 2024-09-16T09:40:32.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:32 smithi070 ceph-mon[30388]: 1.13 scrub ok 2024-09-16T09:40:32.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:32 smithi070 ceph-mon[30388]: 6.3 scrub starts 2024-09-16T09:40:32.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:32 smithi070 ceph-mon[30388]: 6.3 scrub ok 2024-09-16T09:40:32.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:32 smithi070 ceph-mon[30388]: 5.6 scrub starts 2024-09-16T09:40:32.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:32 smithi070 ceph-mon[30388]: 5.6 scrub ok 2024-09-16T09:40:32.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:32 smithi070 ceph-mon[30388]: 6.c scrub starts 2024-09-16T09:40:32.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:32 smithi070 ceph-mon[30388]: 6.c scrub ok 2024-09-16T09:40:32.955 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:40:32.986 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:32.987 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:32.987 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:40:33.088 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:40:33.110 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:33.111 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:33.111 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:40:33.212 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:40:33.242 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:33.243 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:33.243 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:40:33.256 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd set noscrub 2024-09-16T09:40:33.345 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:40:33.366 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:33.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:33.367 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:40:33.467 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:40:33.489 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:33.489 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:33.490 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:40:33.591 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:40:33.626 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:33.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:33.627 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:40:33.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:33 smithi070 ceph-mon[30388]: 6.1b scrub starts 2024-09-16T09:40:33.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:33 smithi070 ceph-mon[30388]: 6.1b scrub ok 2024-09-16T09:40:33.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:33 smithi070 ceph-mon[30388]: pgmap v821: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 2.9 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 771 op/s; 4167/140970 objects misplaced (2.956%) 2024-09-16T09:40:33.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:33 smithi070 ceph-mon[30388]: 6.4 scrub starts 2024-09-16T09:40:33.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:33 smithi070 ceph-mon[30388]: 6.4 scrub ok 2024-09-16T09:40:33.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:33 smithi070 ceph-mon[40691]: 6.1b scrub starts 2024-09-16T09:40:33.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:33 smithi070 ceph-mon[40691]: 6.1b scrub ok 2024-09-16T09:40:33.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:33 smithi070 ceph-mon[40691]: pgmap v821: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 2.9 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 771 op/s; 4167/140970 objects misplaced (2.956%) 2024-09-16T09:40:33.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:33 smithi070 ceph-mon[40691]: 6.4 scrub starts 2024-09-16T09:40:33.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:33 smithi070 ceph-mon[40691]: 6.4 scrub ok 2024-09-16T09:40:33.728 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:40:33.758 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:33.759 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:33.759 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:40:33.860 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:40:33.895 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:33.896 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:33.896 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:40:33.910 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:40:33 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:40:33.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:33 smithi084 ceph-mon[32629]: 6.1b scrub starts 2024-09-16T09:40:33.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:33 smithi084 ceph-mon[32629]: 6.1b scrub ok 2024-09-16T09:40:33.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:33 smithi084 ceph-mon[32629]: pgmap v821: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 2.9 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 771 op/s; 4167/140970 objects misplaced (2.956%) 2024-09-16T09:40:33.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:33 smithi084 ceph-mon[32629]: 6.4 scrub starts 2024-09-16T09:40:33.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:33 smithi084 ceph-mon[32629]: 6.4 scrub ok 2024-09-16T09:40:33.997 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:40:34.035 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:34.036 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:34.036 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:40:34.137 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:40:34.174 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:34.175 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:34.175 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:40:34.276 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:40:34.307 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:34.308 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:34.309 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:40:34.409 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:40:34.446 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:34.447 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:34.447 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:40:34.548 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:40:34.583 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:34.584 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:34.584 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:40:34.685 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:40:34.722 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:34.723 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:34.723 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:40:34.824 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:40:34.858 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:34.859 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:34.859 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:40:34.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:34 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:40:34.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:34 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:40:34.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:34 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:40:34.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:34 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:40:34.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:34 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:40:34.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:34 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:40:34.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:34 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:40:34.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:34 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:40:34.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:34 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:40:34.960 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:40:34.993 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:34.994 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:34.994 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:40:35.095 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:40:35.131 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:35.132 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:35.132 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:40:35.233 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:40:35.269 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:35.270 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:35.270 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:40:35.372 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:40:35.414 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:35.414 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:35.414 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:40:35.515 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:40:35.553 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:35.554 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:35.554 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:40:35.655 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:40:35.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:35 smithi070 ceph-mon[30388]: pgmap v822: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 2.5 GiB data, 14 GiB used, 702 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 830 op/s; 4167/122967 objects misplaced (3.389%) 2024-09-16T09:40:35.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:35 smithi070 ceph-mon[30388]: 4.c scrub starts 2024-09-16T09:40:35.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:35 smithi070 ceph-mon[30388]: 4.c scrub ok 2024-09-16T09:40:35.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:35 smithi070 ceph-mon[30388]: 5.2 scrub starts 2024-09-16T09:40:35.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:35 smithi070 ceph-mon[30388]: 5.2 scrub ok 2024-09-16T09:40:35.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:35 smithi070 ceph-mon[40691]: pgmap v822: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 2.5 GiB data, 14 GiB used, 702 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 830 op/s; 4167/122967 objects misplaced (3.389%) 2024-09-16T09:40:35.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:35 smithi070 ceph-mon[40691]: 4.c scrub starts 2024-09-16T09:40:35.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:35 smithi070 ceph-mon[40691]: 4.c scrub ok 2024-09-16T09:40:35.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:35 smithi070 ceph-mon[40691]: 5.2 scrub starts 2024-09-16T09:40:35.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:35 smithi070 ceph-mon[40691]: 5.2 scrub ok 2024-09-16T09:40:35.689 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:35.690 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:35.690 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:40:35.791 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:40:35.823 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:35.823 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:35.824 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:40:35.844 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-16T09:40:35.844 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-16T09:40:35.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:35 smithi084 ceph-mon[32629]: pgmap v822: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 2.5 GiB data, 14 GiB used, 702 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 830 op/s; 4167/122967 objects misplaced (3.389%) 2024-09-16T09:40:35.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:35 smithi084 ceph-mon[32629]: 4.c scrub starts 2024-09-16T09:40:35.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:35 smithi084 ceph-mon[32629]: 4.c scrub ok 2024-09-16T09:40:35.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:35 smithi084 ceph-mon[32629]: 5.2 scrub starts 2024-09-16T09:40:35.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:35 smithi084 ceph-mon[32629]: 5.2 scrub ok 2024-09-16T09:40:35.925 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:40:35.961 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:35.962 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:35.962 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:40:36.063 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:40:36.110 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:36.111 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:36.111 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:40:36.212 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:40:36.245 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:36.246 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:36.246 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:40:36.346 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:40:36.383 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:36.383 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:36.384 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:40:36.481 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:40:36.481 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:40:36.484 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:40:36.523 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:40:36.523 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:40:36.524 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:36.524 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:36.525 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:40:36.566 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:40:36.567 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:40:36.600 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:40:36.600 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:40:36.625 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:40:36.634 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:40:36.634 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:40:36.663 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:36.664 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:36.664 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:40:36.665 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:40:36.665 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:40:36.698 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:40:36.698 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:40:36.732 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:40:36.732 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:40:36.765 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:40:36.794 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:36.795 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:36.795 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:40:36.822 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:36 smithi070 ceph-mon[30388]: 4.14 scrub starts 2024-09-16T09:40:36.822 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:36 smithi070 ceph-mon[30388]: 4.14 scrub ok 2024-09-16T09:40:36.822 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:36 smithi070 ceph-mon[30388]: 6.14 scrub starts 2024-09-16T09:40:36.822 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:36 smithi070 ceph-mon[30388]: 6.14 scrub ok 2024-09-16T09:40:36.822 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:36 smithi070 ceph-mon[30388]: 10.5 scrub starts 2024-09-16T09:40:36.822 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:36 smithi070 ceph-mon[30388]: 10.5 scrub ok 2024-09-16T09:40:36.822 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:36 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/447266050' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:40:36.823 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:36 smithi070 ceph-mon[40691]: 4.14 scrub starts 2024-09-16T09:40:36.823 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:36 smithi070 ceph-mon[40691]: 4.14 scrub ok 2024-09-16T09:40:36.823 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:36 smithi070 ceph-mon[40691]: 6.14 scrub starts 2024-09-16T09:40:36.823 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:36 smithi070 ceph-mon[40691]: 6.14 scrub ok 2024-09-16T09:40:36.823 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:36 smithi070 ceph-mon[40691]: 10.5 scrub starts 2024-09-16T09:40:36.823 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:36 smithi070 ceph-mon[40691]: 10.5 scrub ok 2024-09-16T09:40:36.823 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:36 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/447266050' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:40:36.896 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:40:36.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:36 smithi084 ceph-mon[32629]: 4.14 scrub starts 2024-09-16T09:40:36.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:36 smithi084 ceph-mon[32629]: 4.14 scrub ok 2024-09-16T09:40:36.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:36 smithi084 ceph-mon[32629]: 6.14 scrub starts 2024-09-16T09:40:36.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:36 smithi084 ceph-mon[32629]: 6.14 scrub ok 2024-09-16T09:40:36.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:36 smithi084 ceph-mon[32629]: 10.5 scrub starts 2024-09-16T09:40:36.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:36 smithi084 ceph-mon[32629]: 10.5 scrub ok 2024-09-16T09:40:36.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:36 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/447266050' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:40:36.934 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:36.935 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:36.935 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:40:37.036 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:40:37.085 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:37.086 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:37.086 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:40:37.187 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:40:37.217 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:37.219 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:37.220 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:40:37.320 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:40:37.336 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T09:40:37.336 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T09:40:37.336 INFO:teuthology.orchestra.run.smithi070.stdout: "complaint_time": 30, 2024-09-16T09:40:37.336 INFO:teuthology.orchestra.run.smithi070.stdout: "num_blocked_ops": 0 2024-09-16T09:40:37.337 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T09:40:37.364 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:37.364 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:37.365 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:40:37.465 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:40:37.499 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:37.500 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:37.501 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:40:37.601 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:40:37.636 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:37.637 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:37.637 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:40:37.738 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:40:37.774 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:37.774 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:37.774 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:40:37.875 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:40:37.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:37 smithi084 ceph-mon[32629]: 1.c scrub starts 2024-09-16T09:40:37.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:37 smithi084 ceph-mon[32629]: 1.c scrub ok 2024-09-16T09:40:37.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:37 smithi084 ceph-mon[32629]: pgmap v823: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 2.4 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 843 op/s; 3570/120339 objects misplaced (2.967%) 2024-09-16T09:40:37.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:37 smithi084 ceph-mon[32629]: 10.4 scrub starts 2024-09-16T09:40:37.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:37 smithi084 ceph-mon[32629]: 10.4 scrub ok 2024-09-16T09:40:37.911 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:37.912 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:37.912 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:40:37.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:37 smithi070 ceph-mon[30388]: 1.c scrub starts 2024-09-16T09:40:37.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:37 smithi070 ceph-mon[30388]: 1.c scrub ok 2024-09-16T09:40:37.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:37 smithi070 ceph-mon[30388]: pgmap v823: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 2.4 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 843 op/s; 3570/120339 objects misplaced (2.967%) 2024-09-16T09:40:37.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:37 smithi070 ceph-mon[30388]: 10.4 scrub starts 2024-09-16T09:40:37.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:37 smithi070 ceph-mon[30388]: 10.4 scrub ok 2024-09-16T09:40:37.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:37 smithi070 ceph-mon[40691]: 1.c scrub starts 2024-09-16T09:40:37.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:37 smithi070 ceph-mon[40691]: 1.c scrub ok 2024-09-16T09:40:37.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:37 smithi070 ceph-mon[40691]: pgmap v823: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 2.4 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 843 op/s; 3570/120339 objects misplaced (2.967%) 2024-09-16T09:40:37.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:37 smithi070 ceph-mon[40691]: 10.4 scrub starts 2024-09-16T09:40:37.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:37 smithi070 ceph-mon[40691]: 10.4 scrub ok 2024-09-16T09:40:38.012 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:40:38.056 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:38.058 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:38.058 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:40:38.071 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.2 dump_historic_ops 2024-09-16T09:40:38.159 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:40:38.190 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:38.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:38.191 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:40:38.292 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:40:38.326 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:38.326 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:38.327 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:40:38.427 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:40:38.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:38.465 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:38.465 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:40:38.538 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:40:38.566 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:40:38.593 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:38.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:38.594 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:40:38.626 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:40:38.695 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:40:38.727 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:38.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:38.728 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:40:38.829 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:40:38.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:38 smithi070 ceph-mon[30388]: 6.15 scrub starts 2024-09-16T09:40:38.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:38 smithi070 ceph-mon[30388]: 6.15 scrub ok 2024-09-16T09:40:38.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:38 smithi070 ceph-mon[30388]: 4.1d scrub starts 2024-09-16T09:40:38.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:38 smithi070 ceph-mon[30388]: 4.1d scrub ok 2024-09-16T09:40:38.855 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:38 smithi070 ceph-mon[30388]: pgmap v824: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 2.4 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 911 op/s; 3570/118323 objects misplaced (3.017%) 2024-09-16T09:40:38.856 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:38 smithi070 ceph-mon[30388]: 1.b scrub starts 2024-09-16T09:40:38.856 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:38 smithi070 ceph-mon[30388]: 1.b scrub ok 2024-09-16T09:40:38.856 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:38 smithi070 ceph-mon[30388]: 5.1f scrub starts 2024-09-16T09:40:38.856 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:38 smithi070 ceph-mon[30388]: 5.1f scrub ok 2024-09-16T09:40:38.856 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:38 smithi070 ceph-mon[30388]: 1.12 scrub starts 2024-09-16T09:40:38.856 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:38 smithi070 ceph-mon[30388]: 1.12 scrub ok 2024-09-16T09:40:38.856 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:38 smithi070 ceph-mon[40691]: 6.15 scrub starts 2024-09-16T09:40:38.856 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:38 smithi070 ceph-mon[40691]: 6.15 scrub ok 2024-09-16T09:40:38.856 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:38 smithi070 ceph-mon[40691]: 4.1d scrub starts 2024-09-16T09:40:38.857 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:38 smithi070 ceph-mon[40691]: 4.1d scrub ok 2024-09-16T09:40:38.857 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:38 smithi070 ceph-mon[40691]: pgmap v824: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 2.4 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 911 op/s; 3570/118323 objects misplaced (3.017%) 2024-09-16T09:40:38.857 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:38 smithi070 ceph-mon[40691]: 1.b scrub starts 2024-09-16T09:40:38.857 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:38 smithi070 ceph-mon[40691]: 1.b scrub ok 2024-09-16T09:40:38.857 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:38 smithi070 ceph-mon[40691]: 5.1f scrub starts 2024-09-16T09:40:38.857 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:38 smithi070 ceph-mon[40691]: 5.1f scrub ok 2024-09-16T09:40:38.857 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:38 smithi070 ceph-mon[40691]: 1.12 scrub starts 2024-09-16T09:40:38.857 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:38 smithi070 ceph-mon[40691]: 1.12 scrub ok 2024-09-16T09:40:38.862 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:38.863 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:38.863 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:40:38.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:38 smithi084 ceph-mon[32629]: 6.15 scrub starts 2024-09-16T09:40:38.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:38 smithi084 ceph-mon[32629]: 6.15 scrub ok 2024-09-16T09:40:38.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:38 smithi084 ceph-mon[32629]: 4.1d scrub starts 2024-09-16T09:40:38.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:38 smithi084 ceph-mon[32629]: 4.1d scrub ok 2024-09-16T09:40:38.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:38 smithi084 ceph-mon[32629]: pgmap v824: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 2.4 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 911 op/s; 3570/118323 objects misplaced (3.017%) 2024-09-16T09:40:38.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:38 smithi084 ceph-mon[32629]: 1.b scrub starts 2024-09-16T09:40:38.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:38 smithi084 ceph-mon[32629]: 1.b scrub ok 2024-09-16T09:40:38.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:38 smithi084 ceph-mon[32629]: 5.1f scrub starts 2024-09-16T09:40:38.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:38 smithi084 ceph-mon[32629]: 5.1f scrub ok 2024-09-16T09:40:38.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:38 smithi084 ceph-mon[32629]: 1.12 scrub starts 2024-09-16T09:40:38.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:38 smithi084 ceph-mon[32629]: 1.12 scrub ok 2024-09-16T09:40:38.964 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:40:38.994 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:38.995 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:38.995 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:40:39.096 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:40:39.135 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:39.136 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:39.136 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:40:39.237 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:40:39.270 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:39.271 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:39.271 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:40:39.372 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:40:39.408 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:39.409 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:39.409 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:40:39.510 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:40:39.544 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:39.545 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:39.545 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:40:39.646 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:40:39.676 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:39.676 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:39.677 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:40:39.778 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:40:39.811 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:39.811 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:39.812 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:40:39.812 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T09:40:39.812 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 20, 2024-09-16T09:40:39.812 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 600, 2024-09-16T09:40:39.812 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [ 2024-09-16T09:40:39.812 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.813 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.15975.0:33579 10.9 e386/376 10:99683988:::benchmark_data_smithi070_166744_object33578:head v 386'2079, mlcod=386'2076)", 2024-09-16T09:40:39.813 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:38:09.640462+0000", 2024-09-16T09:40:39.813 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 150.16738065199999, 2024-09-16T09:40:39.813 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.28017080700000002, 2024-09-16T09:40:39.813 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:40:39.813 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:40:39.813 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:40:39.813 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:40:39.813 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.813 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:40:39.813 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:38:09.640462+0000", 2024-09-16T09:40:39.813 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:39.813 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.814 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.814 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:40:39.814 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:38:09.640462+0000", 2024-09-16T09:40:39.814 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:39.814 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.814 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.814 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:40:39.814 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:38:09.640463+0000", 2024-09-16T09:40:39.814 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.5499999999999997e-07 2024-09-16T09:40:39.814 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.814 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.814 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:40:39.814 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:38:09.640526+0000", 2024-09-16T09:40:39.815 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.3503999999999994e-05 2024-09-16T09:40:39.815 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.815 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.815 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:40:39.815 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:38:09.640535+0000", 2024-09-16T09:40:39.815 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.2029999999999998e-06 2024-09-16T09:40:39.815 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.815 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.815 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:40:39.815 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:38:09.640542+0000", 2024-09-16T09:40:39.815 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.5010000000000001e-06 2024-09-16T09:40:39.815 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.815 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.815 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:40:39.816 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:38:09.917405+0000", 2024-09-16T09:40:39.816 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.27686350799999998 2024-09-16T09:40:39.816 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.816 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.816 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:40:39.816 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:38:09.917429+0000", 2024-09-16T09:40:39.816 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.4017000000000001e-05 2024-09-16T09:40:39.816 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.816 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.816 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:40:39.816 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:38:09.920588+0000", 2024-09-16T09:40:39.816 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0031587820000000002 2024-09-16T09:40:39.816 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.817 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.817 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:40:39.817 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:38:09.920633+0000", 2024-09-16T09:40:39.817 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.4536999999999997e-05 2024-09-16T09:40:39.817 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:39.817 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:39.817 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:39.817 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.817 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.817 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.15975.0:33595 10.f e386/376 10:fdf68c83:::benchmark_data_smithi070_166744_object33594:head v 386'2134, mlcod=386'2132)", 2024-09-16T09:40:39.817 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:38:09.647007+0000", 2024-09-16T09:40:39.817 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 150.16083546600001, 2024-09-16T09:40:39.817 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.27538453699999998, 2024-09-16T09:40:39.818 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:40:39.818 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:40:39.818 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:40:39.818 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:40:39.818 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.818 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:40:39.818 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:38:09.647007+0000", 2024-09-16T09:40:39.818 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:39.818 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.818 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.818 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:40:39.818 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:38:09.647007+0000", 2024-09-16T09:40:39.819 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:39.819 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.819 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.819 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:40:39.819 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:38:09.647008+0000", 2024-09-16T09:40:39.819 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.043e-06 2024-09-16T09:40:39.819 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.819 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.819 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:40:39.819 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:38:09.647046+0000", 2024-09-16T09:40:39.819 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7463999999999999e-05 2024-09-16T09:40:39.819 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.819 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.819 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:40:39.820 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:38:09.917354+0000", 2024-09-16T09:40:39.820 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.27030826499999999 2024-09-16T09:40:39.820 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.820 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.820 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:40:39.820 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:38:09.917365+0000", 2024-09-16T09:40:39.820 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1069e-05 2024-09-16T09:40:39.820 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.820 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.820 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:40:39.820 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:38:09.917423+0000", 2024-09-16T09:40:39.820 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8158000000000001e-05 2024-09-16T09:40:39.820 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.820 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.821 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:40:39.821 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:38:09.917437+0000", 2024-09-16T09:40:39.821 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.4046000000000001e-05 2024-09-16T09:40:39.821 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.821 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.821 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:40:39.821 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:38:09.922337+0000", 2024-09-16T09:40:39.821 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0048999910000000002 2024-09-16T09:40:39.821 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.821 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.821 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:40:39.821 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:38:09.922392+0000", 2024-09-16T09:40:39.821 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4500999999999999e-05 2024-09-16T09:40:39.822 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:39.822 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:39.822 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:39.822 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.822 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.822 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15975.0:90178 10.19 10:98316a94:::benchmark_data_smithi070_166744_object90177:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e407)", 2024-09-16T09:40:39.822 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:39:00.047476+0000", 2024-09-16T09:40:39.822 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 99.760366241, 2024-09-16T09:40:39.822 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.29388039399999999, 2024-09-16T09:40:39.822 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:40:39.822 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:40:39.822 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:40:39.822 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:40:39.822 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15975", 2024-09-16T09:40:39.823 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3821727158", 2024-09-16T09:40:39.823 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 90178 2024-09-16T09:40:39.823 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.823 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:40:39.823 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.823 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:40:39.823 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.047476+0000", 2024-09-16T09:40:39.823 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:39.823 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.823 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.823 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:40:39.823 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.047476+0000", 2024-09-16T09:40:39.823 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:39.823 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.824 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.824 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:40:39.824 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.047478+0000", 2024-09-16T09:40:39.824 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.807e-06 2024-09-16T09:40:39.824 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.824 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.824 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:40:39.824 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.047502+0000", 2024-09-16T09:40:39.824 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3686e-05 2024-09-16T09:40:39.824 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.824 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.824 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:40:39.824 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.047515+0000", 2024-09-16T09:40:39.824 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2624e-05 2024-09-16T09:40:39.825 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.825 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.825 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:40:39.825 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.047522+0000", 2024-09-16T09:40:39.825 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.7419999999999997e-06 2024-09-16T09:40:39.825 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.825 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.825 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:40:39.825 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.047699+0000", 2024-09-16T09:40:39.825 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00017703200000000001 2024-09-16T09:40:39.825 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.825 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.825 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:40:39.826 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.047861+0000", 2024-09-16T09:40:39.826 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00016131300000000001 2024-09-16T09:40:39.826 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.826 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.826 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,3,4", 2024-09-16T09:40:39.826 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.048028+0000", 2024-09-16T09:40:39.826 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00016713399999999999 2024-09-16T09:40:39.826 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.826 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.826 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:40:39.826 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.050803+0000", 2024-09-16T09:40:39.826 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0027749530000000001 2024-09-16T09:40:39.826 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.827 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.827 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:40:39.827 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.052688+0000", 2024-09-16T09:40:39.827 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0018849439999999999 2024-09-16T09:40:39.827 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.827 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.827 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:40:39.827 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.053087+0000", 2024-09-16T09:40:39.827 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00039976999999999998 2024-09-16T09:40:39.827 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.827 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.828 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:40:39.828 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.053137+0000", 2024-09-16T09:40:39.828 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.9425000000000002e-05 2024-09-16T09:40:39.828 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.828 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.828 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:40:39.828 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.053200+0000", 2024-09-16T09:40:39.828 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.3540999999999994e-05 2024-09-16T09:40:39.828 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.828 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.828 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:40:39.828 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.341357+0000", 2024-09-16T09:40:39.828 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.28815642299999999 2024-09-16T09:40:39.829 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:39.829 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:39.829 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:39.829 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.829 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.829 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.15975.0:90184 10.e e407/376 10:797b90f7:::benchmark_data_smithi070_166744_object90183:head v 407'5707, mlcod=407'5705)", 2024-09-16T09:40:39.829 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:39:00.052118+0000", 2024-09-16T09:40:39.829 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 99.755724946000001, 2024-09-16T09:40:39.829 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.28798647100000002, 2024-09-16T09:40:39.830 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:40:39.830 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:40:39.830 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:40:39.830 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:40:39.830 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.830 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:40:39.830 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.052118+0000", 2024-09-16T09:40:39.830 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:39.830 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.830 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.830 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:40:39.830 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.052118+0000", 2024-09-16T09:40:39.831 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:39.831 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.831 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.831 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:40:39.831 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.052118+0000", 2024-09-16T09:40:39.831 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.9500000000000002e-07 2024-09-16T09:40:39.831 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.831 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.831 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:40:39.831 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.052166+0000", 2024-09-16T09:40:39.831 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.744e-05 2024-09-16T09:40:39.831 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.831 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.831 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:40:39.832 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.052174+0000", 2024-09-16T09:40:39.832 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.8099999999999998e-06 2024-09-16T09:40:39.832 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.832 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.832 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:40:39.832 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.052179+0000", 2024-09-16T09:40:39.832 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.1549999999999998e-06 2024-09-16T09:40:39.832 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.832 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.832 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:40:39.832 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.052223+0000", 2024-09-16T09:40:39.832 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.4329000000000002e-05 2024-09-16T09:40:39.832 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.833 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.833 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:40:39.833 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.052238+0000", 2024-09-16T09:40:39.833 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.4891e-05 2024-09-16T09:40:39.833 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.833 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.833 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:40:39.833 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.340062+0000", 2024-09-16T09:40:39.833 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.28782371400000001 2024-09-16T09:40:39.833 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.833 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.833 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:40:39.833 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.340104+0000", 2024-09-16T09:40:39.834 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2437e-05 2024-09-16T09:40:39.834 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:39.834 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:39.834 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:39.834 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.834 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.834 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop_reply(client.15975.0:90178 10.19 e407/398 ondisk, result = 0)", 2024-09-16T09:40:39.834 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:39:00.053008+0000", 2024-09-16T09:40:39.834 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 99.754834724000006, 2024-09-16T09:40:39.834 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.28836234700000002, 2024-09-16T09:40:39.834 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:40:39.834 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:40:39.835 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T09:40:39.835 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:40:39.835 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.835 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:40:39.835 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.053008+0000", 2024-09-16T09:40:39.835 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:39.835 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.835 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.835 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:40:39.835 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.053008+0000", 2024-09-16T09:40:39.835 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:39.835 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.835 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.835 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:40:39.836 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.053009+0000", 2024-09-16T09:40:39.836 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.3600000000000002e-07 2024-09-16T09:40:39.836 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.836 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.836 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:40:39.836 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.053017+0000", 2024-09-16T09:40:39.836 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.7010000000000002e-06 2024-09-16T09:40:39.836 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.836 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.836 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:40:39.836 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.053032+0000", 2024-09-16T09:40:39.836 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.4387e-05 2024-09-16T09:40:39.836 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.836 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.837 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:40:39.837 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.053038+0000", 2024-09-16T09:40:39.837 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.2129999999999996e-06 2024-09-16T09:40:39.837 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.837 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.837 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:40:39.837 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.053122+0000", 2024-09-16T09:40:39.837 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.3595000000000002e-05 2024-09-16T09:40:39.837 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.837 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.837 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:40:39.838 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.053131+0000", 2024-09-16T09:40:39.838 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.8419999999999994e-06 2024-09-16T09:40:39.838 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.838 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.838 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:40:39.838 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.341370+0000", 2024-09-16T09:40:39.838 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.28823977299999998 2024-09-16T09:40:39.838 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:39.838 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:39.838 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:39.838 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.838 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.838 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15975.0:90189 10.13 10:c9862219:::benchmark_data_smithi070_166744_object90188:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e407)", 2024-09-16T09:40:39.838 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:39:00.061217+0000", 2024-09-16T09:40:39.839 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 99.746625589000004, 2024-09-16T09:40:39.839 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.30579520900000001, 2024-09-16T09:40:39.839 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:40:39.839 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:40:39.839 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:40:39.839 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:40:39.839 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15975", 2024-09-16T09:40:39.839 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3821727158", 2024-09-16T09:40:39.839 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 90189 2024-09-16T09:40:39.839 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.839 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:40:39.839 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.839 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:40:39.840 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.061217+0000", 2024-09-16T09:40:39.840 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:39.840 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.840 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.840 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:40:39.840 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.061217+0000", 2024-09-16T09:40:39.840 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:39.840 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.840 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.840 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:40:39.840 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.061220+0000", 2024-09-16T09:40:39.840 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.7769999999999998e-06 2024-09-16T09:40:39.840 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.841 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.841 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:40:39.841 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.061248+0000", 2024-09-16T09:40:39.841 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.8285000000000001e-05 2024-09-16T09:40:39.841 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.841 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.841 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:40:39.841 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.340052+0000", 2024-09-16T09:40:39.841 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.27880404600000003 2024-09-16T09:40:39.841 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.841 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.841 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:40:39.841 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.340068+0000", 2024-09-16T09:40:39.841 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5645999999999999e-05 2024-09-16T09:40:39.842 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.842 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.842 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:40:39.842 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.347875+0000", 2024-09-16T09:40:39.843 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0078067780000000003 2024-09-16T09:40:39.843 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.843 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.843 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:40:39.843 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.348050+0000", 2024-09-16T09:40:39.843 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000175391 2024-09-16T09:40:39.843 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.843 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.843 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,7", 2024-09-16T09:40:39.843 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.348242+0000", 2024-09-16T09:40:39.843 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00019203800000000001 2024-09-16T09:40:39.843 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.844 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.844 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:40:39.844 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.354481+0000", 2024-09-16T09:40:39.844 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.006239423 2024-09-16T09:40:39.844 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.844 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.844 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:40:39.844 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.358919+0000", 2024-09-16T09:40:39.844 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0044373679999999997 2024-09-16T09:40:39.844 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.844 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.844 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:40:39.844 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.362108+0000", 2024-09-16T09:40:39.845 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0031896509999999999 2024-09-16T09:40:39.845 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.845 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.845 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:40:39.845 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.362182+0000", 2024-09-16T09:40:39.845 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3209999999999999e-05 2024-09-16T09:40:39.845 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.845 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.845 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:40:39.845 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.367012+0000", 2024-09-16T09:40:39.845 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0048305960000000004 2024-09-16T09:40:39.845 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:39.846 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:39.846 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:39.846 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.846 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.846 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.15975.0:90180 10.8 e407/395 10:139d7062:::benchmark_data_smithi070_166744_object90179:head v 407'5105, mlcod=407'5103)", 2024-09-16T09:40:39.846 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:39:00.064659+0000", 2024-09-16T09:40:39.846 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 99.743183212999995, 2024-09-16T09:40:39.846 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.28228003800000001, 2024-09-16T09:40:39.846 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:40:39.846 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:40:39.846 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:40:39.846 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:40:39.846 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.846 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:40:39.847 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.064659+0000", 2024-09-16T09:40:39.847 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:39.847 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.847 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.847 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:40:39.847 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.064659+0000", 2024-09-16T09:40:39.847 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:39.847 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.847 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.847 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:40:39.847 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.064661+0000", 2024-09-16T09:40:39.847 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3400000000000001e-06 2024-09-16T09:40:39.847 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.848 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.848 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:40:39.848 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.064693+0000", 2024-09-16T09:40:39.848 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2611999999999999e-05 2024-09-16T09:40:39.848 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.848 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.848 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:40:39.848 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.340058+0000", 2024-09-16T09:40:39.848 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.27536475999999999 2024-09-16T09:40:39.848 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.848 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.848 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:40:39.848 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.340067+0000", 2024-09-16T09:40:39.849 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.1700000000000003e-06 2024-09-16T09:40:39.849 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.849 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.849 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:40:39.849 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.340332+0000", 2024-09-16T09:40:39.849 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00026422400000000002 2024-09-16T09:40:39.849 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.849 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.849 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:40:39.849 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.340350+0000", 2024-09-16T09:40:39.849 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.8117999999999998e-05 2024-09-16T09:40:39.849 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.849 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.850 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:40:39.850 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.346912+0000", 2024-09-16T09:40:39.850 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0065623679999999998 2024-09-16T09:40:39.850 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.850 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.850 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:40:39.850 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.346939+0000", 2024-09-16T09:40:39.850 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.7446000000000001e-05 2024-09-16T09:40:39.850 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:39.850 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:39.850 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:39.850 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.850 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.851 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15975.0:112717 10.18 10:1897c748:::benchmark_data_smithi070_166744_object112716:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e416)", 2024-09-16T09:40:39.851 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:39:23.958012+0000", 2024-09-16T09:40:39.851 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 75.849830515999997, 2024-09-16T09:40:39.851 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.44687069099999999, 2024-09-16T09:40:39.851 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:40:39.851 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:40:39.851 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:40:39.851 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:40:39.851 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15975", 2024-09-16T09:40:39.851 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3821727158", 2024-09-16T09:40:39.851 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 112717 2024-09-16T09:40:39.851 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.852 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:40:39.852 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.852 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:40:39.852 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:23.958012+0000", 2024-09-16T09:40:39.852 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:39.852 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.852 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.852 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:40:39.852 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:23.958012+0000", 2024-09-16T09:40:39.852 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:39.852 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.852 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.852 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:40:39.853 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:23.958014+0000", 2024-09-16T09:40:39.853 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9259999999999999e-06 2024-09-16T09:40:39.853 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.853 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.853 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:40:39.853 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:23.958041+0000", 2024-09-16T09:40:39.853 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.6678000000000001e-05 2024-09-16T09:40:39.853 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.853 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.853 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:40:39.853 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:23.958058+0000", 2024-09-16T09:40:39.853 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7292999999999999e-05 2024-09-16T09:40:39.853 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.854 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.854 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:40:39.854 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:23.958065+0000", 2024-09-16T09:40:39.854 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.2899999999999997e-06 2024-09-16T09:40:39.854 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.854 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.854 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:40:39.854 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:23.958147+0000", 2024-09-16T09:40:39.854 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.1785000000000004e-05 2024-09-16T09:40:39.854 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.854 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.854 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:40:39.854 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:23.958156+0000", 2024-09-16T09:40:39.855 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.5620000000000004e-06 2024-09-16T09:40:39.855 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.855 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.855 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:40:39.855 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.377617+0000", 2024-09-16T09:40:39.855 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.41946122899999999 2024-09-16T09:40:39.855 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.855 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.855 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:40:39.855 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.377766+0000", 2024-09-16T09:40:39.855 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00014864699999999999 2024-09-16T09:40:39.855 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.855 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.856 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,3", 2024-09-16T09:40:39.856 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.378151+0000", 2024-09-16T09:40:39.856 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000385022 2024-09-16T09:40:39.856 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.856 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.856 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:40:39.856 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.393768+0000", 2024-09-16T09:40:39.856 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.015617396 2024-09-16T09:40:39.856 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.856 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.856 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:40:39.856 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.397799+0000", 2024-09-16T09:40:39.856 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.004031364 2024-09-16T09:40:39.857 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.857 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.857 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:40:39.857 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.404733+0000", 2024-09-16T09:40:39.857 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0069333579999999997 2024-09-16T09:40:39.857 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.857 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.857 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:40:39.857 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.404835+0000", 2024-09-16T09:40:39.857 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00010249799999999999 2024-09-16T09:40:39.857 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.857 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.857 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:40:39.858 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.404883+0000", 2024-09-16T09:40:39.858 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.7642999999999998e-05 2024-09-16T09:40:39.858 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:39.858 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:39.858 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:39.858 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.858 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.858 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15975.0:112754 10.18 10:1d8a67e2:::benchmark_data_smithi070_166744_object112753:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e416)", 2024-09-16T09:40:39.858 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:39:24.000857+0000", 2024-09-16T09:40:39.858 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 75.806985714999996, 2024-09-16T09:40:39.858 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.40415521199999999, 2024-09-16T09:40:39.858 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:40:39.858 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:40:39.859 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:40:39.859 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:40:39.859 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15975", 2024-09-16T09:40:39.859 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3821727158", 2024-09-16T09:40:39.859 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 112754 2024-09-16T09:40:39.859 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.859 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:40:39.859 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.859 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:40:39.859 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.000857+0000", 2024-09-16T09:40:39.859 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:39.859 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.859 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.859 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:40:39.860 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.000857+0000", 2024-09-16T09:40:39.860 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:39.860 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.860 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.860 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:40:39.860 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.000859+0000", 2024-09-16T09:40:39.860 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.357e-06 2024-09-16T09:40:39.860 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.860 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.860 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:40:39.860 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.000883+0000", 2024-09-16T09:40:39.860 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.4119999999999999e-05 2024-09-16T09:40:39.861 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.861 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.861 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:40:39.861 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.000898+0000", 2024-09-16T09:40:39.861 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.4183e-05 2024-09-16T09:40:39.861 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.861 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.861 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:40:39.861 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.000906+0000", 2024-09-16T09:40:39.861 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.1689999999999992e-06 2024-09-16T09:40:39.861 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.861 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.861 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:40:39.862 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.000983+0000", 2024-09-16T09:40:39.862 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.7476000000000002e-05 2024-09-16T09:40:39.862 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.862 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.862 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:40:39.862 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.000993+0000", 2024-09-16T09:40:39.862 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0028e-05 2024-09-16T09:40:39.862 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.862 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.862 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:40:39.862 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.380231+0000", 2024-09-16T09:40:39.862 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.37923777800000003 2024-09-16T09:40:39.862 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.862 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.863 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:40:39.863 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.380352+0000", 2024-09-16T09:40:39.863 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000120901 2024-09-16T09:40:39.863 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.863 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.863 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,3", 2024-09-16T09:40:39.863 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.380859+0000", 2024-09-16T09:40:39.863 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00050672499999999997 2024-09-16T09:40:39.863 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.863 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.863 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:40:39.863 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.393947+0000", 2024-09-16T09:40:39.863 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.013088123 2024-09-16T09:40:39.864 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.864 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.864 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:40:39.864 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.398580+0000", 2024-09-16T09:40:39.864 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0046332500000000002 2024-09-16T09:40:39.864 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.864 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.864 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:40:39.864 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.404927+0000", 2024-09-16T09:40:39.864 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0063469679999999997 2024-09-16T09:40:39.864 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.864 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.864 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:40:39.865 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.404978+0000", 2024-09-16T09:40:39.865 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.1091000000000003e-05 2024-09-16T09:40:39.865 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.865 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.865 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:40:39.865 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.405012+0000", 2024-09-16T09:40:39.865 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4042999999999999e-05 2024-09-16T09:40:39.865 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:39.865 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:39.865 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:39.865 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.865 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.865 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15975.0:112856 10.18 10:18c0c17d:::benchmark_data_smithi070_166744_object112855:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e416)", 2024-09-16T09:40:39.865 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:39:24.094929+0000", 2024-09-16T09:40:39.866 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 75.712913568999994, 2024-09-16T09:40:39.866 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.31034393399999999, 2024-09-16T09:40:39.866 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:40:39.866 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:40:39.866 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:40:39.866 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:40:39.866 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15975", 2024-09-16T09:40:39.866 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3821727158", 2024-09-16T09:40:39.866 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 112856 2024-09-16T09:40:39.866 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.866 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:40:39.866 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.866 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:40:39.866 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.094929+0000", 2024-09-16T09:40:39.867 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:39.867 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.867 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.867 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:40:39.867 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.094929+0000", 2024-09-16T09:40:39.867 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:39.868 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.868 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.868 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:40:39.868 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.094931+0000", 2024-09-16T09:40:39.868 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0059999999999999e-06 2024-09-16T09:40:39.868 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.868 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.868 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:40:39.868 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.094955+0000", 2024-09-16T09:40:39.868 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3757e-05 2024-09-16T09:40:39.868 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.868 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.869 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:40:39.869 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.094968+0000", 2024-09-16T09:40:39.869 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.34e-05 2024-09-16T09:40:39.869 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.869 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.869 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:40:39.869 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.094976+0000", 2024-09-16T09:40:39.869 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.7200000000000006e-06 2024-09-16T09:40:39.869 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.869 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.869 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:40:39.869 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.096929+0000", 2024-09-16T09:40:39.869 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0019525949999999999 2024-09-16T09:40:39.870 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.870 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.870 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:40:39.870 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.096939+0000", 2024-09-16T09:40:39.870 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0912e-05 2024-09-16T09:40:39.870 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.870 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.870 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:40:39.870 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.382244+0000", 2024-09-16T09:40:39.870 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.28530434100000002 2024-09-16T09:40:39.870 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.870 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.870 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:40:39.870 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.382366+0000", 2024-09-16T09:40:39.870 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00012229299999999999 2024-09-16T09:40:39.871 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.871 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.871 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,3", 2024-09-16T09:40:39.871 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.382638+0000", 2024-09-16T09:40:39.871 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00027176499999999998 2024-09-16T09:40:39.871 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.871 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.871 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:40:39.871 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.393952+0000", 2024-09-16T09:40:39.871 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.011314204 2024-09-16T09:40:39.871 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.871 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.871 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:40:39.872 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.398630+0000", 2024-09-16T09:40:39.872 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0046776329999999996 2024-09-16T09:40:39.872 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.872 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.872 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:40:39.872 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.405051+0000", 2024-09-16T09:40:39.872 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0064212089999999998 2024-09-16T09:40:39.872 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.872 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.872 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:40:39.872 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.405232+0000", 2024-09-16T09:40:39.872 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00018135100000000001 2024-09-16T09:40:39.872 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.873 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.873 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:40:39.873 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.405273+0000", 2024-09-16T09:40:39.873 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0747999999999997e-05 2024-09-16T09:40:39.873 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:39.873 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:39.873 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:39.873 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.873 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.873 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15975.0:112858 10.18 10:1b69b42e:::benchmark_data_smithi070_166744_object112857:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e416)", 2024-09-16T09:40:39.873 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:39:24.099083+0000", 2024-09-16T09:40:39.873 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 75.708759553999997, 2024-09-16T09:40:39.873 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.30636600800000002, 2024-09-16T09:40:39.874 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:40:39.874 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:40:39.874 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:40:39.874 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:40:39.874 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15975", 2024-09-16T09:40:39.874 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3821727158", 2024-09-16T09:40:39.874 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 112858 2024-09-16T09:40:39.874 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.874 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:40:39.874 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.874 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:40:39.874 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.099083+0000", 2024-09-16T09:40:39.874 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:39.874 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.875 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.875 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:40:39.875 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.099083+0000", 2024-09-16T09:40:39.875 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:39.875 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.875 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.875 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:40:39.875 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.099085+0000", 2024-09-16T09:40:39.875 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.702e-06 2024-09-16T09:40:39.875 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.875 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.875 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:40:39.875 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.099109+0000", 2024-09-16T09:40:39.876 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3972e-05 2024-09-16T09:40:39.876 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.876 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.876 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:40:39.876 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.099137+0000", 2024-09-16T09:40:39.876 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.8126000000000001e-05 2024-09-16T09:40:39.876 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.876 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.876 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:40:39.876 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.099145+0000", 2024-09-16T09:40:39.876 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.2640000000000008e-06 2024-09-16T09:40:39.876 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.876 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.877 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:40:39.877 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.099217+0000", 2024-09-16T09:40:39.877 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.1962e-05 2024-09-16T09:40:39.877 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.877 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.877 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:40:39.877 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.099227+0000", 2024-09-16T09:40:39.877 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.8500000000000006e-06 2024-09-16T09:40:39.877 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.877 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.877 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:40:39.877 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.384669+0000", 2024-09-16T09:40:39.877 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.285442003 2024-09-16T09:40:39.877 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.878 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.878 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:40:39.878 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.384840+0000", 2024-09-16T09:40:39.878 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000171151 2024-09-16T09:40:39.878 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.878 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.878 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,3", 2024-09-16T09:40:39.878 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.385178+0000", 2024-09-16T09:40:39.878 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00033784199999999998 2024-09-16T09:40:39.878 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.878 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.878 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:40:39.878 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.393957+0000", 2024-09-16T09:40:39.878 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.008779238 2024-09-16T09:40:39.879 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.879 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.879 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:40:39.879 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.404622+0000", 2024-09-16T09:40:39.879 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.010664585000000001 2024-09-16T09:40:39.879 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.879 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.879 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:40:39.879 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.405316+0000", 2024-09-16T09:40:39.879 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00069413099999999996 2024-09-16T09:40:39.879 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.879 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.879 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:40:39.880 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.405412+0000", 2024-09-16T09:40:39.880 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.6385000000000006e-05 2024-09-16T09:40:39.880 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.880 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.880 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:40:39.880 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.405449+0000", 2024-09-16T09:40:39.880 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6797e-05 2024-09-16T09:40:39.880 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:39.880 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:39.880 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:39.880 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.880 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.880 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15975.0:112872 10.18 10:1de29386:::benchmark_data_smithi070_166744_object112871:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e416)", 2024-09-16T09:40:39.880 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:39:24.111890+0000", 2024-09-16T09:40:39.881 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 75.695952487, 2024-09-16T09:40:39.881 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.29379652000000001, 2024-09-16T09:40:39.881 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:40:39.881 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:40:39.881 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:40:39.881 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:40:39.881 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15975", 2024-09-16T09:40:39.881 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3821727158", 2024-09-16T09:40:39.881 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 112872 2024-09-16T09:40:39.881 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.881 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:40:39.881 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.881 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:40:39.882 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.111890+0000", 2024-09-16T09:40:39.882 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:39.882 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.882 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.882 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:40:39.882 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.111890+0000", 2024-09-16T09:40:39.882 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:39.882 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.882 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.882 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:40:39.882 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.111892+0000", 2024-09-16T09:40:39.882 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.722e-06 2024-09-16T09:40:39.882 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.882 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.883 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:40:39.883 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.111918+0000", 2024-09-16T09:40:39.883 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.6417000000000001e-05 2024-09-16T09:40:39.883 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.883 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.883 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:40:39.883 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.111936+0000", 2024-09-16T09:40:39.883 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7329e-05 2024-09-16T09:40:39.883 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.883 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.883 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:40:39.883 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.111944+0000", 2024-09-16T09:40:39.883 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.1370000000000002e-06 2024-09-16T09:40:39.883 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.884 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.884 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:40:39.884 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.112014+0000", 2024-09-16T09:40:39.884 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.0241999999999996e-05 2024-09-16T09:40:39.884 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.884 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.884 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:40:39.884 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.112024+0000", 2024-09-16T09:40:39.884 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.842e-06 2024-09-16T09:40:39.884 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.884 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.884 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:40:39.884 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.386913+0000", 2024-09-16T09:40:39.885 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.27488880700000001 2024-09-16T09:40:39.885 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.885 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.885 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:40:39.885 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.387044+0000", 2024-09-16T09:40:39.885 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000131211 2024-09-16T09:40:39.885 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.885 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.885 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,3", 2024-09-16T09:40:39.885 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.387392+0000", 2024-09-16T09:40:39.885 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00034859600000000001 2024-09-16T09:40:39.885 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.885 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.885 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:40:39.886 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.393964+0000", 2024-09-16T09:40:39.886 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0065714900000000001 2024-09-16T09:40:39.886 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.886 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.887 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:40:39.887 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.404659+0000", 2024-09-16T09:40:39.887 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.010694560000000001 2024-09-16T09:40:39.887 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.887 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.887 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:40:39.887 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.405491+0000", 2024-09-16T09:40:39.887 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00083268700000000003 2024-09-16T09:40:39.887 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.887 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.887 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:40:39.887 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.405649+0000", 2024-09-16T09:40:39.888 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00015801000000000001 2024-09-16T09:40:39.888 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.888 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.888 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:40:39.888 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.405687+0000", 2024-09-16T09:40:39.888 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7469999999999999e-05 2024-09-16T09:40:39.888 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:39.888 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:39.888 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:39.888 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.888 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.888 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15975.0:112907 10.18 10:1deac3cc:::benchmark_data_smithi070_166744_object112906:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e416)", 2024-09-16T09:40:39.888 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:39:24.163480+0000", 2024-09-16T09:40:39.888 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 75.644362217999998, 2024-09-16T09:40:39.889 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.24246647800000001, 2024-09-16T09:40:39.889 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:40:39.889 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:40:39.889 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:40:39.889 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:40:39.889 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15975", 2024-09-16T09:40:39.889 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3821727158", 2024-09-16T09:40:39.889 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 112907 2024-09-16T09:40:39.889 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.889 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:40:39.889 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.889 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:40:39.889 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.163480+0000", 2024-09-16T09:40:39.890 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:39.890 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.890 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.890 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:40:39.890 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.163480+0000", 2024-09-16T09:40:39.890 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:39.890 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.890 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.890 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:40:39.890 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.163482+0000", 2024-09-16T09:40:39.890 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.796e-06 2024-09-16T09:40:39.890 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.890 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.890 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:40:39.891 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.163506+0000", 2024-09-16T09:40:39.891 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3915000000000001e-05 2024-09-16T09:40:39.891 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.891 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.891 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:40:39.891 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.163524+0000", 2024-09-16T09:40:39.891 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.8213999999999999e-05 2024-09-16T09:40:39.891 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.891 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.891 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:40:39.891 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.163533+0000", 2024-09-16T09:40:39.891 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.7590000000000003e-06 2024-09-16T09:40:39.891 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.891 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.892 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:40:39.892 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.163619+0000", 2024-09-16T09:40:39.892 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.5499000000000003e-05 2024-09-16T09:40:39.892 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.892 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.892 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:40:39.892 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.163626+0000", 2024-09-16T09:40:39.892 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.8220000000000007e-06 2024-09-16T09:40:39.892 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.892 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.892 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:40:39.892 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.390915+0000", 2024-09-16T09:40:39.892 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.22728837399999999 2024-09-16T09:40:39.893 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.893 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.893 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:40:39.893 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.391052+0000", 2024-09-16T09:40:39.893 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000137565 2024-09-16T09:40:39.893 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.893 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.893 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,3", 2024-09-16T09:40:39.893 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.391424+0000", 2024-09-16T09:40:39.893 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00037168900000000003 2024-09-16T09:40:39.893 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.893 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.893 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:40:39.893 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.393968+0000", 2024-09-16T09:40:39.894 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0025443269999999999 2024-09-16T09:40:39.894 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.894 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.894 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:40:39.894 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.404698+0000", 2024-09-16T09:40:39.894 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.010729114999999999 2024-09-16T09:40:39.894 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.894 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.894 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:40:39.894 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.405731+0000", 2024-09-16T09:40:39.894 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001033946 2024-09-16T09:40:39.894 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.894 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.895 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:40:39.895 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.405906+0000", 2024-09-16T09:40:39.895 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00017405900000000001 2024-09-16T09:40:39.895 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.895 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.895 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:40:39.895 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.405947+0000", 2024-09-16T09:40:39.895 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.1397999999999999e-05 2024-09-16T09:40:39.895 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:39.895 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:39.895 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:39.895 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.895 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.895 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15975.0:113589 10.19 10:9a90b735:::benchmark_data_smithi070_166744_object113588:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e417)", 2024-09-16T09:40:39.896 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:39:25.237150+0000", 2024-09-16T09:40:39.896 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 74.570692719999997, 2024-09-16T09:40:39.896 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.267162607, 2024-09-16T09:40:39.896 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:40:39.896 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:40:39.896 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:40:39.896 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:40:39.896 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15975", 2024-09-16T09:40:39.896 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3821727158", 2024-09-16T09:40:39.896 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 113589 2024-09-16T09:40:39.896 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.896 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:40:39.896 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.897 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:40:39.897 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.237150+0000", 2024-09-16T09:40:39.897 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:39.897 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.897 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.897 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:40:39.897 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.237150+0000", 2024-09-16T09:40:39.897 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:39.897 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.897 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.897 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:40:39.897 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.237152+0000", 2024-09-16T09:40:39.897 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.798e-06 2024-09-16T09:40:39.897 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.898 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.898 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:40:39.898 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.237178+0000", 2024-09-16T09:40:39.898 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.6007e-05 2024-09-16T09:40:39.898 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.898 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.898 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:40:39.898 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.237188+0000", 2024-09-16T09:40:39.898 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0501e-05 2024-09-16T09:40:39.898 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.898 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.898 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:40:39.898 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.237195+0000", 2024-09-16T09:40:39.898 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.5749999999999997e-06 2024-09-16T09:40:39.898 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.899 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.899 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:40:39.899 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.237250+0000", 2024-09-16T09:40:39.899 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5435e-05 2024-09-16T09:40:39.899 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.899 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.899 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:40:39.899 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.237432+0000", 2024-09-16T09:40:39.899 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000182203 2024-09-16T09:40:39.899 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.899 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.899 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,3,4", 2024-09-16T09:40:39.899 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.237847+0000", 2024-09-16T09:40:39.899 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00041432200000000001 2024-09-16T09:40:39.899 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.900 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.900 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:40:39.900 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.247292+0000", 2024-09-16T09:40:39.900 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0094457159999999998 2024-09-16T09:40:39.900 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.900 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.900 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:40:39.900 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.247340+0000", 2024-09-16T09:40:39.900 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.7809000000000003e-05 2024-09-16T09:40:39.900 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.900 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.900 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:40:39.901 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.261859+0000", 2024-09-16T09:40:39.901 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.014518358 2024-09-16T09:40:39.901 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.901 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.901 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:40:39.901 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.502876+0000", 2024-09-16T09:40:39.901 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.241017381 2024-09-16T09:40:39.901 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.901 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.901 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:40:39.901 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.502951+0000", 2024-09-16T09:40:39.901 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.4686999999999998e-05 2024-09-16T09:40:39.901 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.902 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.902 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:40:39.902 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.504313+0000", 2024-09-16T09:40:39.902 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0013618149999999999 2024-09-16T09:40:39.902 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:39.902 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:39.902 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:39.902 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.902 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.902 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15975.0:113610 10.19 10:9d9dd046:::benchmark_data_smithi070_166744_object113609:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e417)", 2024-09-16T09:40:39.902 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:39:25.252441+0000", 2024-09-16T09:40:39.902 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 74.555401488000001, 2024-09-16T09:40:39.902 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.26758622599999998, 2024-09-16T09:40:39.903 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:40:39.903 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:40:39.903 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:40:39.903 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:40:39.903 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15975", 2024-09-16T09:40:39.903 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3821727158", 2024-09-16T09:40:39.903 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 113610 2024-09-16T09:40:39.903 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.903 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:40:39.903 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.903 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:40:39.903 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.252441+0000", 2024-09-16T09:40:39.903 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:39.904 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.904 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.904 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:40:39.904 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.252441+0000", 2024-09-16T09:40:39.904 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:39.904 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.904 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.904 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:40:39.904 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.252443+0000", 2024-09-16T09:40:39.904 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5850000000000001e-06 2024-09-16T09:40:39.904 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.904 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.904 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:40:39.904 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.252466+0000", 2024-09-16T09:40:39.905 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3336000000000002e-05 2024-09-16T09:40:39.905 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.905 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.905 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:40:39.905 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.252484+0000", 2024-09-16T09:40:39.905 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7442999999999999e-05 2024-09-16T09:40:39.905 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.905 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.905 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:40:39.905 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.252492+0000", 2024-09-16T09:40:39.905 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.259e-06 2024-09-16T09:40:39.905 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.905 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.906 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:40:39.906 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.261907+0000", 2024-09-16T09:40:39.906 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.009415606 2024-09-16T09:40:39.906 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.906 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.906 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:40:39.906 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.262075+0000", 2024-09-16T09:40:39.906 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00016741799999999999 2024-09-16T09:40:39.906 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.906 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.906 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,3,4", 2024-09-16T09:40:39.906 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.262466+0000", 2024-09-16T09:40:39.906 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000390837 2024-09-16T09:40:39.907 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.907 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.907 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:40:39.907 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.266227+0000", 2024-09-16T09:40:39.907 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0037611150000000002 2024-09-16T09:40:39.907 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.907 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.907 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:40:39.907 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.271986+0000", 2024-09-16T09:40:39.907 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0057590690000000003 2024-09-16T09:40:39.907 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.907 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.907 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:40:39.907 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.274510+0000", 2024-09-16T09:40:39.907 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0025244550000000001 2024-09-16T09:40:39.908 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.908 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.908 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:40:39.908 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.518671+0000", 2024-09-16T09:40:39.908 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.24416111099999999 2024-09-16T09:40:39.908 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.908 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.908 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:40:39.908 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.518736+0000", 2024-09-16T09:40:39.908 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.4535999999999993e-05 2024-09-16T09:40:39.908 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.908 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.908 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:40:39.908 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.520027+0000", 2024-09-16T09:40:39.909 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0012914560000000001 2024-09-16T09:40:39.909 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:39.909 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:39.909 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:39.909 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.909 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.909 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15975.0:113611 10.19 10:9fded355:::benchmark_data_smithi070_166744_object113610:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e417)", 2024-09-16T09:40:39.909 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:39:25.255167+0000", 2024-09-16T09:40:39.909 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 74.552675210999993, 2024-09-16T09:40:39.909 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.26572839500000001, 2024-09-16T09:40:39.909 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:40:39.909 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:40:39.909 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:40:39.910 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:40:39.910 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15975", 2024-09-16T09:40:39.910 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3821727158", 2024-09-16T09:40:39.910 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 113611 2024-09-16T09:40:39.910 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.910 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:40:39.910 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.910 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:40:39.910 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.255167+0000", 2024-09-16T09:40:39.910 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:39.910 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.910 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.910 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:40:39.910 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.255167+0000", 2024-09-16T09:40:39.911 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:39.911 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.911 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.911 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:40:39.911 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.255169+0000", 2024-09-16T09:40:39.911 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.8419999999999999e-06 2024-09-16T09:40:39.911 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.911 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.911 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:40:39.911 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.255192+0000", 2024-09-16T09:40:39.911 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3156e-05 2024-09-16T09:40:39.911 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.911 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.911 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:40:39.912 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.255205+0000", 2024-09-16T09:40:39.912 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2738000000000001e-05 2024-09-16T09:40:39.912 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.912 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.912 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:40:39.912 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.255214+0000", 2024-09-16T09:40:39.912 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.7730000000000005e-06 2024-09-16T09:40:39.912 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.912 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.912 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:40:39.912 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.264233+0000", 2024-09-16T09:40:39.912 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0090189160000000001 2024-09-16T09:40:39.912 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.912 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.913 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:40:39.913 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.264394+0000", 2024-09-16T09:40:39.913 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00016140600000000001 2024-09-16T09:40:39.913 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.913 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.913 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,3,4", 2024-09-16T09:40:39.913 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.264634+0000", 2024-09-16T09:40:39.913 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00024003499999999999 2024-09-16T09:40:39.913 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.913 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.913 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:40:39.913 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.266235+0000", 2024-09-16T09:40:39.913 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0016005279999999999 2024-09-16T09:40:39.914 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.914 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.914 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:40:39.914 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.272031+0000", 2024-09-16T09:40:39.914 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.005796107 2024-09-16T09:40:39.914 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.914 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.914 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:40:39.914 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.274583+0000", 2024-09-16T09:40:39.914 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0025517199999999999 2024-09-16T09:40:39.914 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.914 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.914 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:40:39.915 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.520093+0000", 2024-09-16T09:40:39.915 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.245510587 2024-09-16T09:40:39.915 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.915 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.915 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:40:39.915 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.520155+0000", 2024-09-16T09:40:39.915 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1271000000000003e-05 2024-09-16T09:40:39.915 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.915 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.915 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:40:39.915 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.520896+0000", 2024-09-16T09:40:39.915 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000741316 2024-09-16T09:40:39.915 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:39.915 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:39.916 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:39.916 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.916 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.916 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15975.0:113645 10.19 10:9c234406:::benchmark_data_smithi070_166744_object113644:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e417)", 2024-09-16T09:40:39.916 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:39:25.285212+0000", 2024-09-16T09:40:39.916 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 74.522630245000002, 2024-09-16T09:40:39.916 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.23697822099999999, 2024-09-16T09:40:39.916 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:40:39.916 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:40:39.916 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:40:39.916 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:40:39.916 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15975", 2024-09-16T09:40:39.916 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3821727158", 2024-09-16T09:40:39.917 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 113645 2024-09-16T09:40:39.917 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.917 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:40:39.917 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.917 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:40:39.917 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.285212+0000", 2024-09-16T09:40:39.917 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:39.917 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.917 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.917 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:40:39.917 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.285212+0000", 2024-09-16T09:40:39.917 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:39.917 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.918 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.918 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:40:39.918 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.285214+0000", 2024-09-16T09:40:39.918 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9589999999999998e-06 2024-09-16T09:40:39.918 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.918 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.918 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:40:39.918 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.285238+0000", 2024-09-16T09:40:39.918 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3713000000000001e-05 2024-09-16T09:40:39.918 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.918 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.918 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:40:39.918 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.285251+0000", 2024-09-16T09:40:39.918 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3213999999999999e-05 2024-09-16T09:40:39.919 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.919 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.919 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:40:39.919 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.285258+0000", 2024-09-16T09:40:39.919 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.2049999999999996e-06 2024-09-16T09:40:39.919 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.919 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.919 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:40:39.919 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.285317+0000", 2024-09-16T09:40:39.919 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8332999999999999e-05 2024-09-16T09:40:39.919 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.919 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.919 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:40:39.919 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.285472+0000", 2024-09-16T09:40:39.920 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00015523699999999999 2024-09-16T09:40:39.920 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.920 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.920 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,3,4", 2024-09-16T09:40:39.920 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.285876+0000", 2024-09-16T09:40:39.920 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00040423800000000001 2024-09-16T09:40:39.920 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.920 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.920 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:40:39.920 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.287206+0000", 2024-09-16T09:40:39.920 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001329998 2024-09-16T09:40:39.920 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.920 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.921 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:40:39.921 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.292433+0000", 2024-09-16T09:40:39.921 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0052262319999999999 2024-09-16T09:40:39.921 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.921 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.921 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:40:39.921 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.292887+0000", 2024-09-16T09:40:39.921 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00045446800000000003 2024-09-16T09:40:39.921 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.921 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.921 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:40:39.921 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.521236+0000", 2024-09-16T09:40:39.921 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.228348632 2024-09-16T09:40:39.921 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.922 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.922 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:40:39.922 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.521295+0000", 2024-09-16T09:40:39.922 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.9697000000000001e-05 2024-09-16T09:40:39.922 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.922 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.922 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:40:39.922 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.522191+0000", 2024-09-16T09:40:39.922 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00089529500000000005 2024-09-16T09:40:39.922 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:39.922 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:39.922 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:39.922 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.922 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.923 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15975.0:113647 10.19 10:9f9de402:::benchmark_data_smithi070_166744_object113646:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e417)", 2024-09-16T09:40:39.923 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:39:25.287206+0000", 2024-09-16T09:40:39.923 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 74.520636461999999, 2024-09-16T09:40:39.923 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.23593757500000001, 2024-09-16T09:40:39.923 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:40:39.923 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:40:39.923 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:40:39.923 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:40:39.923 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15975", 2024-09-16T09:40:39.923 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3821727158", 2024-09-16T09:40:39.923 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 113647 2024-09-16T09:40:39.923 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.923 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:40:39.923 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.924 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:40:39.924 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.287206+0000", 2024-09-16T09:40:39.924 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:39.924 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.924 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.924 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:40:39.924 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.287206+0000", 2024-09-16T09:40:39.924 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:39.924 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.924 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.924 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:40:39.924 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.287208+0000", 2024-09-16T09:40:39.924 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.46e-06 2024-09-16T09:40:39.925 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.925 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.925 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:40:39.925 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.287232+0000", 2024-09-16T09:40:39.925 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.4287999999999999e-05 2024-09-16T09:40:39.925 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.925 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.925 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:40:39.925 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.287249+0000", 2024-09-16T09:40:39.925 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7513999999999999e-05 2024-09-16T09:40:39.925 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.925 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.925 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:40:39.925 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.287258+0000", 2024-09-16T09:40:39.926 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.4279999999999996e-06 2024-09-16T09:40:39.926 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.926 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.926 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:40:39.926 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.287321+0000", 2024-09-16T09:40:39.926 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.2707999999999997e-05 2024-09-16T09:40:39.926 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.926 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.926 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:40:39.926 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.287502+0000", 2024-09-16T09:40:39.926 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00018115500000000001 2024-09-16T09:40:39.926 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.926 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.926 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,3,4", 2024-09-16T09:40:39.927 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.287817+0000", 2024-09-16T09:40:39.927 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00031515100000000001 2024-09-16T09:40:39.927 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.927 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.927 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:40:39.927 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.289540+0000", 2024-09-16T09:40:39.927 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001722903 2024-09-16T09:40:39.927 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.927 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.927 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:40:39.927 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.292490+0000", 2024-09-16T09:40:39.927 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0029498250000000001 2024-09-16T09:40:39.927 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.927 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.928 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:40:39.928 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.292939+0000", 2024-09-16T09:40:39.928 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00044983300000000001 2024-09-16T09:40:39.928 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.928 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.928 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:40:39.929 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:40:39.929 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.522250+0000", 2024-09-16T09:40:39.930 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.229310663 2024-09-16T09:40:39.930 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.930 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.930 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:40:39.930 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.522288+0000", 2024-09-16T09:40:39.930 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8350999999999999e-05 2024-09-16T09:40:39.930 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.930 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.930 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:40:39.930 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.523144+0000", 2024-09-16T09:40:39.930 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00085529599999999996 2024-09-16T09:40:39.930 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:39.931 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:39.931 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:39.931 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.931 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.931 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15975.0:150410 10.19 10:99b85175:::benchmark_data_smithi070_166744_object36054:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e429)", 2024-09-16T09:40:39.931 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:40:01.355771+0000", 2024-09-16T09:40:39.931 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 38.452072067000003, 2024-09-16T09:40:39.931 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.247207607, 2024-09-16T09:40:39.931 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:40:39.931 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:40:39.931 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:40:39.931 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:40:39.931 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15975", 2024-09-16T09:40:39.932 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3821727158", 2024-09-16T09:40:39.932 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 150410 2024-09-16T09:40:39.932 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.932 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:40:39.932 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.932 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:40:39.932 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.355771+0000", 2024-09-16T09:40:39.932 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:39.932 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.932 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.932 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:40:39.932 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.355771+0000", 2024-09-16T09:40:39.932 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:39.932 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.933 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.933 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:40:39.933 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.355772+0000", 2024-09-16T09:40:39.933 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.55e-06 2024-09-16T09:40:39.933 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.933 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.933 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:40:39.933 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.355778+0000", 2024-09-16T09:40:39.933 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4190000000000001e-06 2024-09-16T09:40:39.933 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.933 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.933 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:40:39.933 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.355789+0000", 2024-09-16T09:40:39.934 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1119e-05 2024-09-16T09:40:39.934 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.934 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.934 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:40:39.934 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.355796+0000", 2024-09-16T09:40:39.934 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.1330000000000001e-06 2024-09-16T09:40:39.934 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.934 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.934 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:40:39.934 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.356366+0000", 2024-09-16T09:40:39.934 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00057015899999999999 2024-09-16T09:40:39.934 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.934 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.935 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:40:39.935 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.356522+0000", 2024-09-16T09:40:39.935 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000156236 2024-09-16T09:40:39.935 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.935 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.935 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,3,4", 2024-09-16T09:40:39.935 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.356971+0000", 2024-09-16T09:40:39.935 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00044910799999999998 2024-09-16T09:40:39.935 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.935 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.935 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:40:39.935 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.360629+0000", 2024-09-16T09:40:39.935 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.003657361 2024-09-16T09:40:39.935 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.936 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.936 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:40:39.936 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.368284+0000", 2024-09-16T09:40:39.936 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0076550480000000002 2024-09-16T09:40:39.936 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.936 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.936 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:40:39.936 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.368366+0000", 2024-09-16T09:40:39.936 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.1969000000000001e-05 2024-09-16T09:40:39.936 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.936 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.936 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:40:39.936 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.601617+0000", 2024-09-16T09:40:39.937 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.23325089199999999 2024-09-16T09:40:39.937 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.937 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.937 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:40:39.937 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.601729+0000", 2024-09-16T09:40:39.937 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000112235 2024-09-16T09:40:39.937 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.937 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.937 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:40:39.937 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.602978+0000", 2024-09-16T09:40:39.937 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001249378 2024-09-16T09:40:39.937 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:39.937 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:39.937 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:39.938 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.938 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.938 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15975.0:150417 10.19 10:9c31ffcf:::benchmark_data_smithi070_166744_object36061:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e429)", 2024-09-16T09:40:39.938 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:40:01.362746+0000", 2024-09-16T09:40:39.938 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 38.445096620000001, 2024-09-16T09:40:39.938 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.24787088500000001, 2024-09-16T09:40:39.938 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:40:39.938 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:40:39.938 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:40:39.938 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:40:39.938 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15975", 2024-09-16T09:40:39.938 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3821727158", 2024-09-16T09:40:39.938 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 150417 2024-09-16T09:40:39.938 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.939 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:40:39.939 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.939 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:40:39.939 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.362746+0000", 2024-09-16T09:40:39.939 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:39.939 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.939 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.939 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:40:39.939 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.362746+0000", 2024-09-16T09:40:39.939 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:39.939 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.939 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.939 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:40:39.939 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.362748+0000", 2024-09-16T09:40:39.940 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6470000000000001e-06 2024-09-16T09:40:39.940 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.940 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.940 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:40:39.940 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.362759+0000", 2024-09-16T09:40:39.940 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0995000000000001e-05 2024-09-16T09:40:39.940 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.940 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.940 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:40:39.940 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.362776+0000", 2024-09-16T09:40:39.940 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7187999999999999e-05 2024-09-16T09:40:39.940 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.940 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.941 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:40:39.941 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.362787+0000", 2024-09-16T09:40:39.941 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0686000000000001e-05 2024-09-16T09:40:39.941 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.941 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.941 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:40:39.941 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.362853+0000", 2024-09-16T09:40:39.941 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.6030000000000003e-05 2024-09-16T09:40:39.941 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.941 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.941 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:40:39.941 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.363067+0000", 2024-09-16T09:40:39.941 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00021482599999999999 2024-09-16T09:40:39.942 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.942 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.942 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,3,4", 2024-09-16T09:40:39.942 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.363553+0000", 2024-09-16T09:40:39.942 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000485315 2024-09-16T09:40:39.942 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.942 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.942 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:40:39.942 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.366668+0000", 2024-09-16T09:40:39.942 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0031155269999999999 2024-09-16T09:40:39.942 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.942 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.942 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:40:39.942 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.368328+0000", 2024-09-16T09:40:39.943 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0016592849999999999 2024-09-16T09:40:39.943 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.943 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.943 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:40:39.943 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.368554+0000", 2024-09-16T09:40:39.943 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00022619900000000001 2024-09-16T09:40:39.943 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.943 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.943 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:40:39.943 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.608909+0000", 2024-09-16T09:40:39.943 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.240355649 2024-09-16T09:40:39.943 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.943 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.943 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:40:39.944 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.609026+0000", 2024-09-16T09:40:39.944 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000116582 2024-09-16T09:40:39.944 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:39.944 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:39.944 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:40:39.944 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.610617+0000", 2024-09-16T09:40:39.944 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0015909559999999999 2024-09-16T09:40:39.944 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:39.944 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:39.944 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:39.944 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:39.944 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:39.944 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T09:40:39.961 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:39.961 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:39.961 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:40:40.062 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:40:40.095 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:40.096 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:40.096 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:40:40.197 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:40:40.236 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:40.237 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:40.237 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:40:40.338 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:40:40.370 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:40.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:40.371 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:40:40.441 INFO:teuthology.orchestra.run.smithi070.stderr:noscrub is set 2024-09-16T09:40:40.472 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:40:40.508 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:40.509 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:40.509 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:40:40.581 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T09:40:40.610 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:40:40.676 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:40.677 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:40.677 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:40:40.778 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:40:40.812 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:40.813 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:40.813 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:40:40.845 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T09:40:40.845 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T09:40:40.845 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-16T09:40:40.845 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd dump -f json-pretty 2024-09-16T09:40:40.875 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:40 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/2078600252' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:40:40.875 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:40 smithi070 ceph-mon[40691]: 5.b scrub starts 2024-09-16T09:40:40.875 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:40 smithi070 ceph-mon[40691]: 5.b scrub ok 2024-09-16T09:40:40.875 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:40 smithi070 ceph-mon[40691]: 10.16 scrub starts 2024-09-16T09:40:40.876 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:40 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2078600252' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:40:40.876 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:40 smithi070 ceph-mon[30388]: 5.b scrub starts 2024-09-16T09:40:40.876 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:40 smithi070 ceph-mon[30388]: 5.b scrub ok 2024-09-16T09:40:40.876 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:40 smithi070 ceph-mon[30388]: 10.16 scrub starts 2024-09-16T09:40:40.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:40 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/2078600252' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:40:40.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:40 smithi084 ceph-mon[32629]: 5.b scrub starts 2024-09-16T09:40:40.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:40 smithi084 ceph-mon[32629]: 5.b scrub ok 2024-09-16T09:40:40.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:40 smithi084 ceph-mon[32629]: 10.16 scrub starts 2024-09-16T09:40:40.915 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:40:40.939 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.3 dump_ops_in_flight 2024-09-16T09:40:40.977 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:40.978 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:40.979 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:40:41.079 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:40:41.114 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:41.114 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:41.114 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:40:41.215 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:40:41.245 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:41.246 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:41.246 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:40:41.347 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:40:41.376 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:41.377 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:41.377 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:40:41.478 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:40:41.511 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:41.512 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:41.513 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:40:41.613 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:40:41.646 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:41.647 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:41.647 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:40:41.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:41 smithi070 ceph-mon[30388]: pgmap v825: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 2.0 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 988 B/s rd, 0 B/s wr, 1.43k op/s; 2814/99348 objects misplaced (2.832%) 2024-09-16T09:40:41.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:41 smithi070 ceph-mon[30388]: 5.10 scrub starts 2024-09-16T09:40:41.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:41 smithi070 ceph-mon[30388]: 5.10 scrub ok 2024-09-16T09:40:41.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:41 smithi070 ceph-mon[30388]: 10.16 scrub ok 2024-09-16T09:40:41.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:41 smithi070 ceph-mon[30388]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:40:41.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:41 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2078600252' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-16T09:40:41.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:41 smithi070 ceph-mon[30388]: osdmap e442: 8 total, 8 up, 8 in 2024-09-16T09:40:41.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:41 smithi070 ceph-mon[40691]: pgmap v825: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 2.0 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 988 B/s rd, 0 B/s wr, 1.43k op/s; 2814/99348 objects misplaced (2.832%) 2024-09-16T09:40:41.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:41 smithi070 ceph-mon[40691]: 5.10 scrub starts 2024-09-16T09:40:41.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:41 smithi070 ceph-mon[40691]: 5.10 scrub ok 2024-09-16T09:40:41.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:41 smithi070 ceph-mon[40691]: 10.16 scrub ok 2024-09-16T09:40:41.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:41 smithi070 ceph-mon[40691]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:40:41.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:41 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/2078600252' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-16T09:40:41.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:41 smithi070 ceph-mon[40691]: osdmap e442: 8 total, 8 up, 8 in 2024-09-16T09:40:41.748 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:40:41.781 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:41.781 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:41.782 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:40:41.882 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:40:41.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:41 smithi084 ceph-mon[32629]: pgmap v825: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 2.0 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 988 B/s rd, 0 B/s wr, 1.43k op/s; 2814/99348 objects misplaced (2.832%) 2024-09-16T09:40:41.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:41 smithi084 ceph-mon[32629]: 5.10 scrub starts 2024-09-16T09:40:41.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:41 smithi084 ceph-mon[32629]: 5.10 scrub ok 2024-09-16T09:40:41.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:41 smithi084 ceph-mon[32629]: 10.16 scrub ok 2024-09-16T09:40:41.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:41 smithi084 ceph-mon[32629]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:40:41.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:41 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/2078600252' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-16T09:40:41.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:41 smithi084 ceph-mon[32629]: osdmap e442: 8 total, 8 up, 8 in 2024-09-16T09:40:41.918 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:41.919 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:41.919 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:40:42.020 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:40:42.055 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:42.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:42.056 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:40:42.157 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:40:42.195 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:42.196 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:42.196 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:40:42.297 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:40:42.327 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:42.327 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:42.328 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:40:42.421 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:40:41 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:40:41] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:40:42.428 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:40:42.449 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:42.450 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:42.450 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:40:42.551 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:40:42.587 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:42.588 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:42.588 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:40:42.689 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:40:42.722 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:42.722 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:42.723 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:40:42.823 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:40:42.845 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:42.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:42.846 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:40:42.947 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:40:42.982 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:42.982 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:42.983 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:40:43.083 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:40:43.116 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:43.117 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:43.118 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:40:43.218 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:40:43.251 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:43.252 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:43.252 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:40:43.353 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:40:43.388 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:43.389 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:43.389 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:40:43.490 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:40:43.525 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:43.526 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:43.527 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:40:43.627 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:40:43.670 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:43.671 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:43.671 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:40:43.703 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:43 smithi070 ceph-mon[30388]: 10.15 scrub starts 2024-09-16T09:40:43.703 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:43 smithi070 ceph-mon[30388]: pgmap v827: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 2.0 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.39k op/s; 2814/99348 objects misplaced (2.832%) 2024-09-16T09:40:43.707 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:43 smithi070 ceph-mon[40691]: 10.15 scrub starts 2024-09-16T09:40:43.708 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:43 smithi070 ceph-mon[40691]: pgmap v827: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 2.0 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.39k op/s; 2814/99348 objects misplaced (2.832%) 2024-09-16T09:40:43.708 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:40:43.773 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:40:43.807 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:43.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:43.809 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:40:43.909 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:40:43.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:43 smithi084 ceph-mon[32629]: 10.15 scrub starts 2024-09-16T09:40:43.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:43 smithi084 ceph-mon[32629]: pgmap v827: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 2.0 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.39k op/s; 2814/99348 objects misplaced (2.832%) 2024-09-16T09:40:43.911 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:40:43 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:40:43.946 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:43.946 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:43.946 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:40:44.019 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:40:44.047 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:40:44.101 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:44.101 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:44.102 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:40:44.203 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:40:44.237 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:44.238 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:44.238 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:40:44.316 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:40:44.339 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:40:44.371 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:44.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:44.372 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:40:44.379 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:40:44.379 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:40:44.409 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:40:44.409 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:40:44.442 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:40:44.442 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:40:44.473 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:40:44.478 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:40:44.478 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:40:44.507 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:44.508 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:44.509 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:40:44.512 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:40:44.512 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:40:44.546 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:40:44.547 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:40:44.575 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:44 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:40:44.575 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:44 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3958028130' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:40:44.576 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:44 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:40:44.576 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:44 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3958028130' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:40:44.578 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:40:44.579 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:40:44.609 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:40:44.620 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:40:44.620 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:40:44.652 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:44.653 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:44.653 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:40:44.754 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:40:44.789 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:44.790 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:44.790 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:40:44.891 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:40:44.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:44 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:40:44.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:44 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3958028130' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:40:44.926 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:44.927 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:44.927 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:40:45.028 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:40:45.064 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:45.064 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:45.064 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:40:45.165 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:40:45.199 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:45.199 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:45.200 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:40:45.301 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:40:45.335 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:45.335 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:45.335 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:40:45.436 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:40:45.470 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:45.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:45.471 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:40:45.572 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:40:45.609 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:45.610 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:45.610 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:40:45.705 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-16T09:40:45.711 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:40:45.737 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:45 smithi070 ceph-mon[30388]: pgmap v828: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 1.7 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 762 op/s; 2814/81912 objects misplaced (3.435%) 2024-09-16T09:40:45.738 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:45 smithi070 ceph-mon[30388]: 10.14 scrub starts 2024-09-16T09:40:45.738 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:45 smithi070 ceph-mon[40691]: pgmap v828: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 1.7 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 762 op/s; 2814/81912 objects misplaced (3.435%) 2024-09-16T09:40:45.738 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:45 smithi070 ceph-mon[40691]: 10.14 scrub starts 2024-09-16T09:40:45.783 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:45.783 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:45.784 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:40:45.884 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:40:45.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:45 smithi084 ceph-mon[32629]: pgmap v828: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 1.7 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 762 op/s; 2814/81912 objects misplaced (3.435%) 2024-09-16T09:40:45.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:45 smithi084 ceph-mon[32629]: 10.14 scrub starts 2024-09-16T09:40:45.928 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:45.929 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:45.930 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:40:46.030 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:40:46.066 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:46.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:46.068 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:40:46.168 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:40:46.205 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:46.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:46.206 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:40:46.307 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:40:46.336 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:46.336 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:46.337 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:40:46.438 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:40:46.473 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:46.473 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:46.474 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:40:46.575 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:40:46.603 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:40:46.603 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T09:40:46.603 INFO:teuthology.orchestra.run.smithi070.stdout: "epoch": 442, 2024-09-16T09:40:46.603 INFO:teuthology.orchestra.run.smithi070.stdout: "fsid": "a1321834-740c-11ef-bceb-c7b262605968", 2024-09-16T09:40:46.603 INFO:teuthology.orchestra.run.smithi070.stdout: "created": "2024-09-16T09:19:14.121498+0000", 2024-09-16T09:40:46.603 INFO:teuthology.orchestra.run.smithi070.stdout: "modified": "2024-09-16T09:40:40.423641+0000", 2024-09-16T09:40:46.603 INFO:teuthology.orchestra.run.smithi070.stdout: "last_up_change": "2024-09-16T09:24:51.124496+0000", 2024-09-16T09:40:46.603 INFO:teuthology.orchestra.run.smithi070.stdout: "last_in_change": "2024-09-16T09:24:28.432920+0000", 2024-09-16T09:40:46.603 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": "noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-16T09:40:46.603 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_num": 5801984, 2024-09-16T09:40:46.604 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_set": [ 2024-09-16T09:40:46.604 INFO:teuthology.orchestra.run.smithi070.stdout: "noscrub", 2024-09-16T09:40:46.604 INFO:teuthology.orchestra.run.smithi070.stdout: "pglog_hardlimit", 2024-09-16T09:40:46.604 INFO:teuthology.orchestra.run.smithi070.stdout: "purged_snapdirs", 2024-09-16T09:40:46.604 INFO:teuthology.orchestra.run.smithi070.stdout: "recovery_deletes", 2024-09-16T09:40:46.604 INFO:teuthology.orchestra.run.smithi070.stdout: "sortbitwise" 2024-09-16T09:40:46.604 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-16T09:40:46.604 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_version": 18, 2024-09-16T09:40:46.604 INFO:teuthology.orchestra.run.smithi070.stdout: "full_ratio": 0.94999998807907104, 2024-09-16T09:40:46.604 INFO:teuthology.orchestra.run.smithi070.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-16T09:40:46.604 INFO:teuthology.orchestra.run.smithi070.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-16T09:40:46.604 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_snapshot": "", 2024-09-16T09:40:46.604 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_max": 10, 2024-09-16T09:40:46.605 INFO:teuthology.orchestra.run.smithi070.stdout: "max_osd": 8, 2024-09-16T09:40:46.605 INFO:teuthology.orchestra.run.smithi070.stdout: "require_min_compat_client": "luminous", 2024-09-16T09:40:46.605 INFO:teuthology.orchestra.run.smithi070.stdout: "min_compat_client": "luminous", 2024-09-16T09:40:46.605 INFO:teuthology.orchestra.run.smithi070.stdout: "require_osd_release": "squid", 2024-09-16T09:40:46.605 INFO:teuthology.orchestra.run.smithi070.stdout: "allow_crimson": false, 2024-09-16T09:40:46.605 INFO:teuthology.orchestra.run.smithi070.stdout: "pools": [ 2024-09-16T09:40:46.605 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.605 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 1, 2024-09-16T09:40:46.605 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": ".mgr", 2024-09-16T09:40:46.605 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2024-09-16T09:22:27.617694+0000", 2024-09-16T09:40:46.605 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2024-09-16T09:40:46.605 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2024-09-16T09:40:46.605 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2024-09-16T09:40:46.605 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 3, 2024-09-16T09:40:46.606 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 2, 2024-09-16T09:40:46.606 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2024-09-16T09:40:46.606 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2024-09-16T09:40:46.606 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2024-09-16T09:40:46.606 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T09:40:46.606 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T09:40:46.606 INFO:teuthology.orchestra.run.smithi070.stdout: "is_stretch_pool": false, 2024-09-16T09:40:46.606 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2024-09-16T09:40:46.606 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2024-09-16T09:40:46.606 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 21, 2024-09-16T09:40:46.606 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 21, 2024-09-16T09:40:46.606 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 21, 2024-09-16T09:40:46.606 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 21, 2024-09-16T09:40:46.606 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 21, 2024-09-16T09:40:46.606 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2024-09-16T09:40:46.607 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2024-09-16T09:40:46.607 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2024-09-16T09:40:46.607 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2024-09-16T09:40:46.607 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2024-09-16T09:40:46.607 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2024-09-16T09:40:46.607 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2024-09-16T09:40:46.607 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.607 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "162", 2024-09-16T09:40:46.607 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2024-09-16T09:40:46.607 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T09:40:46.607 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "160", 2024-09-16T09:40:46.607 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2024-09-16T09:40:46.607 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2024-09-16T09:40:46.607 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2024-09-16T09:40:46.608 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2024-09-16T09:40:46.608 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2024-09-16T09:40:46.608 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2024-09-16T09:40:46.608 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2024-09-16T09:40:46.608 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2024-09-16T09:40:46.608 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2024-09-16T09:40:46.608 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2024-09-16T09:40:46.608 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2024-09-16T09:40:46.608 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2024-09-16T09:40:46.608 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2024-09-16T09:40:46.608 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2024-09-16T09:40:46.608 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2024-09-16T09:40:46.608 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T09:40:46.609 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T09:40:46.609 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T09:40:46.609 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2024-09-16T09:40:46.609 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2024-09-16T09:40:46.609 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2024-09-16T09:40:46.609 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2024-09-16T09:40:46.609 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2024-09-16T09:40:46.609 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.609 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2024-09-16T09:40:46.609 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2024-09-16T09:40:46.609 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2024-09-16T09:40:46.609 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2024-09-16T09:40:46.609 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2024-09-16T09:40:46.610 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T09:40:46.610 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2024-09-16T09:40:46.610 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2024-09-16T09:40:46.610 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2024-09-16T09:40:46.610 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2024-09-16T09:40:46.610 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2024-09-16T09:40:46.610 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2024-09-16T09:40:46.610 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_max": 32, 2024-09-16T09:40:46.610 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_min": 1 2024-09-16T09:40:46.610 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.610 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2024-09-16T09:40:46.610 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": {} 2024-09-16T09:40:46.610 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.610 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2024-09-16T09:40:46.611 INFO:teuthology.orchestra.run.smithi070.stdout: "score_type": "Fair distribution", 2024-09-16T09:40:46.611 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 1.6699999570846558, 2024-09-16T09:40:46.611 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 1.6699999570846558, 2024-09-16T09:40:46.611 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1.1399999856948853, 2024-09-16T09:40:46.611 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 1.8999999761581421, 2024-09-16T09:40:46.611 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 1.8999999761581421, 2024-09-16T09:40:46.611 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 0.70719337463378906, 2024-09-16T09:40:46.611 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 0.81000000238418579, 2024-09-16T09:40:46.611 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 0.70999997854232788 2024-09-16T09:40:46.611 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:46.611 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.611 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.611 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 2, 2024-09-16T09:40:46.611 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "datapool", 2024-09-16T09:40:46.612 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2024-09-16T09:24:59.169244+0000", 2024-09-16T09:40:46.612 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 8193, 2024-09-16T09:40:46.612 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-16T09:40:46.612 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2024-09-16T09:40:46.612 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 3, 2024-09-16T09:40:46.612 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 2, 2024-09-16T09:40:46.612 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2024-09-16T09:40:46.612 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2024-09-16T09:40:46.612 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2024-09-16T09:40:46.612 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T09:40:46.612 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T09:40:46.612 INFO:teuthology.orchestra.run.smithi070.stdout: "is_stretch_pool": false, 2024-09-16T09:40:46.612 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2024-09-16T09:40:46.612 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2024-09-16T09:40:46.612 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 3, 2024-09-16T09:40:46.613 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 3, 2024-09-16T09:40:46.613 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 3, 2024-09-16T09:40:46.613 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 3, 2024-09-16T09:40:46.613 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 3, 2024-09-16T09:40:46.613 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2024-09-16T09:40:46.613 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2024-09-16T09:40:46.613 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2024-09-16T09:40:46.613 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2024-09-16T09:40:46.613 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2024-09-16T09:40:46.613 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2024-09-16T09:40:46.613 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2024-09-16T09:40:46.613 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.613 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "61", 2024-09-16T09:40:46.613 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2024-09-16T09:40:46.614 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T09:40:46.614 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T09:40:46.614 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2024-09-16T09:40:46.614 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2024-09-16T09:40:46.614 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 2, 2024-09-16T09:40:46.614 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 61, 2024-09-16T09:40:46.614 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2024-09-16T09:40:46.614 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2024-09-16T09:40:46.614 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2024-09-16T09:40:46.614 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2024-09-16T09:40:46.614 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2024-09-16T09:40:46.614 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2024-09-16T09:40:46.614 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2024-09-16T09:40:46.614 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2024-09-16T09:40:46.615 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2024-09-16T09:40:46.615 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2024-09-16T09:40:46.615 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2024-09-16T09:40:46.615 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T09:40:46.615 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T09:40:46.615 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T09:40:46.615 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2024-09-16T09:40:46.615 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2024-09-16T09:40:46.615 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2024-09-16T09:40:46.615 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2024-09-16T09:40:46.615 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2024-09-16T09:40:46.615 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.615 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2024-09-16T09:40:46.615 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2024-09-16T09:40:46.615 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2024-09-16T09:40:46.615 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2024-09-16T09:40:46.616 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2024-09-16T09:40:46.616 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T09:40:46.616 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2024-09-16T09:40:46.616 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2024-09-16T09:40:46.616 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2024-09-16T09:40:46.616 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2024-09-16T09:40:46.616 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2024-09-16T09:40:46.616 INFO:teuthology.orchestra.run.smithi070.stdout: "options": {}, 2024-09-16T09:40:46.616 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2024-09-16T09:40:46.616 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd": {} 2024-09-16T09:40:46.617 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.617 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2024-09-16T09:40:46.617 INFO:teuthology.orchestra.run.smithi070.stdout: "score_type": "Fair distribution", 2024-09-16T09:40:46.617 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 2.3299999237060547, 2024-09-16T09:40:46.617 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 2.3299999237060547, 2024-09-16T09:40:46.617 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 0.86000001430511475, 2024-09-16T09:40:46.617 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 2, 2024-09-16T09:40:46.617 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 2, 2024-09-16T09:40:46.617 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 0.70719337463378906, 2024-09-16T09:40:46.617 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 0.81000000238418579, 2024-09-16T09:40:46.617 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 0.70999997854232788 2024-09-16T09:40:46.617 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:46.617 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.617 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.618 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 3, 2024-09-16T09:40:46.618 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": ".rgw.root", 2024-09-16T09:40:46.618 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2024-09-16T09:25:00.056119+0000", 2024-09-16T09:40:46.618 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2024-09-16T09:40:46.618 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2024-09-16T09:40:46.618 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2024-09-16T09:40:46.618 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 3, 2024-09-16T09:40:46.618 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 2, 2024-09-16T09:40:46.618 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2024-09-16T09:40:46.618 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2024-09-16T09:40:46.618 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2024-09-16T09:40:46.618 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T09:40:46.619 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T09:40:46.619 INFO:teuthology.orchestra.run.smithi070.stdout: "is_stretch_pool": false, 2024-09-16T09:40:46.619 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2024-09-16T09:40:46.619 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2024-09-16T09:40:46.619 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 32, 2024-09-16T09:40:46.619 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 32, 2024-09-16T09:40:46.619 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 32, 2024-09-16T09:40:46.619 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 32, 2024-09-16T09:40:46.619 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 32, 2024-09-16T09:40:46.619 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2024-09-16T09:40:46.620 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2024-09-16T09:40:46.620 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2024-09-16T09:40:46.620 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2024-09-16T09:40:46.620 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2024-09-16T09:40:46.620 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2024-09-16T09:40:46.620 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2024-09-16T09:40:46.621 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.621 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "60", 2024-09-16T09:40:46.621 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2024-09-16T09:40:46.624 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T09:40:46.624 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T09:40:46.624 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2024-09-16T09:40:46.624 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2024-09-16T09:40:46.624 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2024-09-16T09:40:46.625 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2024-09-16T09:40:46.625 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2024-09-16T09:40:46.625 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2024-09-16T09:40:46.625 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2024-09-16T09:40:46.625 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2024-09-16T09:40:46.625 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2024-09-16T09:40:46.625 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2024-09-16T09:40:46.625 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2024-09-16T09:40:46.625 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2024-09-16T09:40:46.625 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2024-09-16T09:40:46.626 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2024-09-16T09:40:46.626 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2024-09-16T09:40:46.626 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T09:40:46.626 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T09:40:46.626 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T09:40:46.626 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2024-09-16T09:40:46.626 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2024-09-16T09:40:46.626 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2024-09-16T09:40:46.626 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2024-09-16T09:40:46.626 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2024-09-16T09:40:46.627 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.627 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2024-09-16T09:40:46.627 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2024-09-16T09:40:46.627 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2024-09-16T09:40:46.627 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2024-09-16T09:40:46.627 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2024-09-16T09:40:46.627 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T09:40:46.627 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2024-09-16T09:40:46.627 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2024-09-16T09:40:46.627 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2024-09-16T09:40:46.627 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2024-09-16T09:40:46.628 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2024-09-16T09:40:46.628 INFO:teuthology.orchestra.run.smithi070.stdout: "options": {}, 2024-09-16T09:40:46.628 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2024-09-16T09:40:46.628 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw": {} 2024-09-16T09:40:46.628 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.628 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2024-09-16T09:40:46.628 INFO:teuthology.orchestra.run.smithi070.stdout: "score_type": "Fair distribution", 2024-09-16T09:40:46.628 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 1.75, 2024-09-16T09:40:46.628 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 1.75, 2024-09-16T09:40:46.628 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1.1399999856948853, 2024-09-16T09:40:46.628 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 2, 2024-09-16T09:40:46.628 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 2, 2024-09-16T09:40:46.629 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 0.70719337463378906, 2024-09-16T09:40:46.629 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 0.81000000238418579, 2024-09-16T09:40:46.629 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 0.70999997854232788 2024-09-16T09:40:46.629 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:46.629 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.629 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.629 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 4, 2024-09-16T09:40:46.629 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "default.rgw.log", 2024-09-16T09:40:46.629 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2024-09-16T09:25:01.770016+0000", 2024-09-16T09:40:46.629 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2024-09-16T09:40:46.629 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2024-09-16T09:40:46.629 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2024-09-16T09:40:46.630 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 3, 2024-09-16T09:40:46.630 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 2, 2024-09-16T09:40:46.630 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2024-09-16T09:40:46.630 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2024-09-16T09:40:46.630 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2024-09-16T09:40:46.630 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T09:40:46.630 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T09:40:46.630 INFO:teuthology.orchestra.run.smithi070.stdout: "is_stretch_pool": false, 2024-09-16T09:40:46.630 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2024-09-16T09:40:46.630 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2024-09-16T09:40:46.630 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 32, 2024-09-16T09:40:46.631 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 32, 2024-09-16T09:40:46.631 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 32, 2024-09-16T09:40:46.631 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 32, 2024-09-16T09:40:46.631 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 32, 2024-09-16T09:40:46.631 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2024-09-16T09:40:46.631 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2024-09-16T09:40:46.631 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2024-09-16T09:40:46.631 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2024-09-16T09:40:46.631 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2024-09-16T09:40:46.631 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2024-09-16T09:40:46.631 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2024-09-16T09:40:46.632 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.632 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "62", 2024-09-16T09:40:46.632 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2024-09-16T09:40:46.632 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T09:40:46.632 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T09:40:46.632 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2024-09-16T09:40:46.632 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2024-09-16T09:40:46.632 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2024-09-16T09:40:46.632 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2024-09-16T09:40:46.632 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2024-09-16T09:40:46.633 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2024-09-16T09:40:46.633 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2024-09-16T09:40:46.633 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2024-09-16T09:40:46.633 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2024-09-16T09:40:46.633 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2024-09-16T09:40:46.633 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2024-09-16T09:40:46.633 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2024-09-16T09:40:46.633 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2024-09-16T09:40:46.633 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2024-09-16T09:40:46.633 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2024-09-16T09:40:46.633 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T09:40:46.633 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T09:40:46.634 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T09:40:46.634 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2024-09-16T09:40:46.634 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2024-09-16T09:40:46.634 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2024-09-16T09:40:46.634 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2024-09-16T09:40:46.634 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2024-09-16T09:40:46.634 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.634 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2024-09-16T09:40:46.634 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2024-09-16T09:40:46.634 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2024-09-16T09:40:46.634 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2024-09-16T09:40:46.634 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2024-09-16T09:40:46.635 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T09:40:46.635 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2024-09-16T09:40:46.635 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2024-09-16T09:40:46.635 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2024-09-16T09:40:46.635 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2024-09-16T09:40:46.635 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2024-09-16T09:40:46.635 INFO:teuthology.orchestra.run.smithi070.stdout: "options": {}, 2024-09-16T09:40:46.635 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2024-09-16T09:40:46.635 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw": {} 2024-09-16T09:40:46.635 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.635 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2024-09-16T09:40:46.635 INFO:teuthology.orchestra.run.smithi070.stdout: "score_type": "Fair distribution", 2024-09-16T09:40:46.635 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 6.8499999046325684, 2024-09-16T09:40:46.636 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 2.190000057220459, 2024-09-16T09:40:46.636 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1.1399999856948853, 2024-09-16T09:40:46.636 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 7.809999942779541, 2024-09-16T09:40:46.636 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 2.5, 2024-09-16T09:40:46.636 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 0.70719337463378906, 2024-09-16T09:40:46.636 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 0.81000000238418579, 2024-09-16T09:40:46.636 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 0.70999997854232788, 2024-09-16T09:40:46.636 INFO:teuthology.orchestra.run.smithi070.stdout: "error_message": "pool 4 has acting primaries on OSD(s) with primary affinity 0, read balance score is not accurate" 2024-09-16T09:40:46.636 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:46.636 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.636 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.637 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 5, 2024-09-16T09:40:46.637 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "default.rgw.control", 2024-09-16T09:40:46.637 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2024-09-16T09:25:03.725555+0000", 2024-09-16T09:40:46.637 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2024-09-16T09:40:46.637 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2024-09-16T09:40:46.637 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2024-09-16T09:40:46.637 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 3, 2024-09-16T09:40:46.637 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 2, 2024-09-16T09:40:46.637 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2024-09-16T09:40:46.637 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2024-09-16T09:40:46.638 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2024-09-16T09:40:46.638 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T09:40:46.638 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T09:40:46.638 INFO:teuthology.orchestra.run.smithi070.stdout: "is_stretch_pool": false, 2024-09-16T09:40:46.638 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2024-09-16T09:40:46.638 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2024-09-16T09:40:46.638 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 32, 2024-09-16T09:40:46.638 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 32, 2024-09-16T09:40:46.638 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 32, 2024-09-16T09:40:46.638 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 32, 2024-09-16T09:40:46.638 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 32, 2024-09-16T09:40:46.639 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2024-09-16T09:40:46.639 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2024-09-16T09:40:46.639 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2024-09-16T09:40:46.639 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2024-09-16T09:40:46.639 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2024-09-16T09:40:46.639 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2024-09-16T09:40:46.639 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2024-09-16T09:40:46.640 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.640 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "64", 2024-09-16T09:40:46.640 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2024-09-16T09:40:46.640 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T09:40:46.640 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T09:40:46.640 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2024-09-16T09:40:46.640 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2024-09-16T09:40:46.640 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2024-09-16T09:40:46.640 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2024-09-16T09:40:46.640 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2024-09-16T09:40:46.640 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2024-09-16T09:40:46.641 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2024-09-16T09:40:46.641 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2024-09-16T09:40:46.641 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2024-09-16T09:40:46.641 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2024-09-16T09:40:46.641 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2024-09-16T09:40:46.641 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2024-09-16T09:40:46.641 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2024-09-16T09:40:46.641 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2024-09-16T09:40:46.641 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2024-09-16T09:40:46.641 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T09:40:46.641 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T09:40:46.642 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T09:40:46.642 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2024-09-16T09:40:46.642 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2024-09-16T09:40:46.642 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2024-09-16T09:40:46.642 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2024-09-16T09:40:46.642 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2024-09-16T09:40:46.642 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.642 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2024-09-16T09:40:46.642 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2024-09-16T09:40:46.642 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2024-09-16T09:40:46.642 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2024-09-16T09:40:46.643 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2024-09-16T09:40:46.643 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T09:40:46.643 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2024-09-16T09:40:46.643 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2024-09-16T09:40:46.643 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2024-09-16T09:40:46.643 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2024-09-16T09:40:46.643 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2024-09-16T09:40:46.643 INFO:teuthology.orchestra.run.smithi070.stdout: "options": {}, 2024-09-16T09:40:46.643 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2024-09-16T09:40:46.643 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw": {} 2024-09-16T09:40:46.643 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.643 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2024-09-16T09:40:46.644 INFO:teuthology.orchestra.run.smithi070.stdout: "score_type": "Fair distribution", 2024-09-16T09:40:46.644 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 1.75, 2024-09-16T09:40:46.644 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 1.75, 2024-09-16T09:40:46.644 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1.1399999856948853, 2024-09-16T09:40:46.644 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 2, 2024-09-16T09:40:46.644 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 2, 2024-09-16T09:40:46.644 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 0.70719337463378906, 2024-09-16T09:40:46.644 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 0.81000000238418579, 2024-09-16T09:40:46.644 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 0.70999997854232788 2024-09-16T09:40:46.644 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:46.645 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.645 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.645 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 6, 2024-09-16T09:40:46.645 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "default.rgw.meta", 2024-09-16T09:40:46.645 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2024-09-16T09:25:05.773530+0000", 2024-09-16T09:40:46.645 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2024-09-16T09:40:46.645 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2024-09-16T09:40:46.645 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2024-09-16T09:40:46.645 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 3, 2024-09-16T09:40:46.645 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 2, 2024-09-16T09:40:46.645 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2024-09-16T09:40:46.646 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2024-09-16T09:40:46.646 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2024-09-16T09:40:46.646 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T09:40:46.646 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T09:40:46.646 INFO:teuthology.orchestra.run.smithi070.stdout: "is_stretch_pool": false, 2024-09-16T09:40:46.646 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2024-09-16T09:40:46.646 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2024-09-16T09:40:46.646 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 32, 2024-09-16T09:40:46.646 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 32, 2024-09-16T09:40:46.646 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 32, 2024-09-16T09:40:46.646 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 32, 2024-09-16T09:40:46.646 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 32, 2024-09-16T09:40:46.647 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2024-09-16T09:40:46.647 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2024-09-16T09:40:46.647 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2024-09-16T09:40:46.647 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2024-09-16T09:40:46.647 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2024-09-16T09:40:46.647 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2024-09-16T09:40:46.647 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2024-09-16T09:40:46.647 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.647 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "66", 2024-09-16T09:40:46.647 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2024-09-16T09:40:46.647 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T09:40:46.648 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T09:40:46.648 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2024-09-16T09:40:46.648 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2024-09-16T09:40:46.648 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2024-09-16T09:40:46.648 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2024-09-16T09:40:46.648 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2024-09-16T09:40:46.648 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2024-09-16T09:40:46.648 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2024-09-16T09:40:46.648 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2024-09-16T09:40:46.648 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2024-09-16T09:40:46.648 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2024-09-16T09:40:46.648 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2024-09-16T09:40:46.649 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2024-09-16T09:40:46.649 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2024-09-16T09:40:46.649 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2024-09-16T09:40:46.649 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2024-09-16T09:40:46.649 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T09:40:46.649 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T09:40:46.649 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T09:40:46.649 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2024-09-16T09:40:46.649 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2024-09-16T09:40:46.650 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2024-09-16T09:40:46.650 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2024-09-16T09:40:46.650 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2024-09-16T09:40:46.650 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.650 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2024-09-16T09:40:46.650 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2024-09-16T09:40:46.650 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2024-09-16T09:40:46.650 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2024-09-16T09:40:46.650 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2024-09-16T09:40:46.650 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T09:40:46.650 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2024-09-16T09:40:46.650 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2024-09-16T09:40:46.651 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2024-09-16T09:40:46.651 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2024-09-16T09:40:46.651 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2024-09-16T09:40:46.651 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2024-09-16T09:40:46.651 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_bias": 4 2024-09-16T09:40:46.651 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.651 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2024-09-16T09:40:46.651 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw": {} 2024-09-16T09:40:46.651 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.651 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2024-09-16T09:40:46.651 INFO:teuthology.orchestra.run.smithi070.stdout: "score_type": "Fair distribution", 2024-09-16T09:40:46.652 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 1.5399999618530273, 2024-09-16T09:40:46.652 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 1.5399999618530273, 2024-09-16T09:40:46.652 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1.1399999856948853, 2024-09-16T09:40:46.652 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 1.75, 2024-09-16T09:40:46.652 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 1.75, 2024-09-16T09:40:46.652 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 0.70719337463378906, 2024-09-16T09:40:46.652 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 0.81000000238418579, 2024-09-16T09:40:46.652 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 0.70999997854232788 2024-09-16T09:40:46.652 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:46.652 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.653 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.653 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 10, 2024-09-16T09:40:46.653 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "unique_pool_3", 2024-09-16T09:40:46.653 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2024-09-16T09:37:43.482267+0000", 2024-09-16T09:40:46.653 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2024-09-16T09:40:46.653 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2024-09-16T09:40:46.653 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2024-09-16T09:40:46.653 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 3, 2024-09-16T09:40:46.653 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 2, 2024-09-16T09:40:46.654 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2024-09-16T09:40:46.654 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2024-09-16T09:40:46.654 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2024-09-16T09:40:46.654 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T09:40:46.654 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T09:40:46.654 INFO:teuthology.orchestra.run.smithi070.stdout: "is_stretch_pool": false, 2024-09-16T09:40:46.654 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2024-09-16T09:40:46.654 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2024-09-16T09:40:46.654 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 26, 2024-09-16T09:40:46.654 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 26, 2024-09-16T09:40:46.654 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 26, 2024-09-16T09:40:46.654 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 26, 2024-09-16T09:40:46.655 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 26, 2024-09-16T09:40:46.655 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2024-09-16T09:40:46.655 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2024-09-16T09:40:46.655 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2024-09-16T09:40:46.655 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2024-09-16T09:40:46.655 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2024-09-16T09:40:46.655 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2024-09-16T09:40:46.655 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2024-09-16T09:40:46.655 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.655 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "397", 2024-09-16T09:40:46.655 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2024-09-16T09:40:46.655 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T09:40:46.656 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "395", 2024-09-16T09:40:46.656 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2024-09-16T09:40:46.656 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2024-09-16T09:40:46.656 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2024-09-16T09:40:46.656 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2024-09-16T09:40:46.656 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2024-09-16T09:40:46.656 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2024-09-16T09:40:46.656 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2024-09-16T09:40:46.656 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2024-09-16T09:40:46.656 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2024-09-16T09:40:46.656 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2024-09-16T09:40:46.657 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2024-09-16T09:40:46.657 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2024-09-16T09:40:46.657 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2024-09-16T09:40:46.657 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2024-09-16T09:40:46.657 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2024-09-16T09:40:46.657 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T09:40:46.657 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T09:40:46.657 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T09:40:46.657 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2024-09-16T09:40:46.657 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2024-09-16T09:40:46.658 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2024-09-16T09:40:46.658 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2024-09-16T09:40:46.658 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2024-09-16T09:40:46.658 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.658 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2024-09-16T09:40:46.658 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2024-09-16T09:40:46.658 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2024-09-16T09:40:46.658 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2024-09-16T09:40:46.658 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2024-09-16T09:40:46.658 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T09:40:46.658 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2024-09-16T09:40:46.658 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2024-09-16T09:40:46.659 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2024-09-16T09:40:46.659 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2024-09-16T09:40:46.659 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2024-09-16T09:40:46.659 INFO:teuthology.orchestra.run.smithi070.stdout: "options": {}, 2024-09-16T09:40:46.659 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2024-09-16T09:40:46.659 INFO:teuthology.orchestra.run.smithi070.stdout: "rados": {} 2024-09-16T09:40:46.659 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.659 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2024-09-16T09:40:46.659 INFO:teuthology.orchestra.run.smithi070.stdout: "score_type": "Fair distribution", 2024-09-16T09:40:46.659 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 1.8899999856948853, 2024-09-16T09:40:46.659 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 2.1600000858306885, 2024-09-16T09:40:46.660 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1.1399999856948853, 2024-09-16T09:40:46.660 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 2.1500000953674316, 2024-09-16T09:40:46.660 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 2.4600000381469727, 2024-09-16T09:40:46.660 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 0.70719337463378906, 2024-09-16T09:40:46.660 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 0.81000000238418579, 2024-09-16T09:40:46.660 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 0.70999997854232788 2024-09-16T09:40:46.660 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:46.660 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:46.660 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-16T09:40:46.660 INFO:teuthology.orchestra.run.smithi070.stdout: "osds": [ 2024-09-16T09:40:46.660 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.661 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 0, 2024-09-16T09:40:46.661 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "7ef288c8-3bf0-4814-8e86-f70a7de041ce", 2024-09-16T09:40:46.661 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2024-09-16T09:40:46.661 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2024-09-16T09:40:46.661 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2024-09-16T09:40:46.661 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2024-09-16T09:40:46.661 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2024-09-16T09:40:46.661 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2024-09-16T09:40:46.661 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 8, 2024-09-16T09:40:46.661 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 419, 2024-09-16T09:40:46.661 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2024-09-16T09:40:46.661 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2024-09-16T09:40:46.662 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2024-09-16T09:40:46.662 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:40:46.662 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.662 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:40:46.662 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6802", 2024-09-16T09:40:46.662 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2013075781 2024-09-16T09:40:46.662 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.662 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.662 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:40:46.662 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6803", 2024-09-16T09:40:46.662 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2013075781 2024-09-16T09:40:46.662 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:46.663 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:46.663 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.663 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2024-09-16T09:40:46.663 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:40:46.663 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.663 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:40:46.663 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6804", 2024-09-16T09:40:46.663 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2013075781 2024-09-16T09:40:46.663 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.663 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.663 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:40:46.663 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6805", 2024-09-16T09:40:46.664 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2013075781 2024-09-16T09:40:46.664 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:46.664 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:46.664 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.664 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:46.665 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:46.665 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:40:46.666 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2024-09-16T09:40:46.666 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:40:46.666 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.666 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:40:46.666 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6808", 2024-09-16T09:40:46.666 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2013075781 2024-09-16T09:40:46.666 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.667 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.667 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:40:46.667 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6809", 2024-09-16T09:40:46.667 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2013075781 2024-09-16T09:40:46.667 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:46.667 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:46.667 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.667 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2024-09-16T09:40:46.667 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:40:46.667 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.667 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:40:46.668 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6806", 2024-09-16T09:40:46.668 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2013075781 2024-09-16T09:40:46.668 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.668 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.668 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:40:46.668 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6807", 2024-09-16T09:40:46.668 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2013075781 2024-09-16T09:40:46.668 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:46.668 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:46.668 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.668 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6803/2013075781", 2024-09-16T09:40:46.669 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6805/2013075781", 2024-09-16T09:40:46.669 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6809/2013075781", 2024-09-16T09:40:46.669 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6807/2013075781", 2024-09-16T09:40:46.669 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2024-09-16T09:40:46.669 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2024-09-16T09:40:46.669 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2024-09-16T09:40:46.669 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:46.669 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.669 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.669 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 1, 2024-09-16T09:40:46.669 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "474d41d9-295a-450c-8f27-ce487a14678b", 2024-09-16T09:40:46.669 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2024-09-16T09:40:46.670 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2024-09-16T09:40:46.670 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2024-09-16T09:40:46.670 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 0, 2024-09-16T09:40:46.670 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2024-09-16T09:40:46.670 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2024-09-16T09:40:46.670 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2024-09-16T09:40:46.670 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 362, 2024-09-16T09:40:46.670 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2024-09-16T09:40:46.670 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2024-09-16T09:40:46.670 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2024-09-16T09:40:46.670 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:40:46.670 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.670 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:40:46.671 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6810", 2024-09-16T09:40:46.671 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1670012115 2024-09-16T09:40:46.671 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.671 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.671 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:40:46.671 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6811", 2024-09-16T09:40:46.671 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1670012115 2024-09-16T09:40:46.671 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:46.671 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:46.671 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.671 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2024-09-16T09:40:46.672 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:40:46.672 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.672 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:40:46.672 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6812", 2024-09-16T09:40:46.672 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1670012115 2024-09-16T09:40:46.672 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.672 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.672 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:40:46.672 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6813", 2024-09-16T09:40:46.672 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1670012115 2024-09-16T09:40:46.673 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:46.673 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:46.673 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.673 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2024-09-16T09:40:46.673 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:40:46.673 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.673 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:40:46.673 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6816", 2024-09-16T09:40:46.673 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1670012115 2024-09-16T09:40:46.673 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.673 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.673 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:40:46.674 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6817", 2024-09-16T09:40:46.674 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1670012115 2024-09-16T09:40:46.674 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:46.674 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:46.674 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.674 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2024-09-16T09:40:46.674 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:40:46.674 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.674 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:40:46.674 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6814", 2024-09-16T09:40:46.674 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1670012115 2024-09-16T09:40:46.674 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.675 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.675 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:40:46.675 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6815", 2024-09-16T09:40:46.675 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1670012115 2024-09-16T09:40:46.675 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:46.675 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:46.675 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.675 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6811/1670012115", 2024-09-16T09:40:46.675 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6813/1670012115", 2024-09-16T09:40:46.675 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6817/1670012115", 2024-09-16T09:40:46.675 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6815/1670012115", 2024-09-16T09:40:46.676 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2024-09-16T09:40:46.676 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2024-09-16T09:40:46.676 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2024-09-16T09:40:46.676 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:46.676 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.676 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.676 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 2, 2024-09-16T09:40:46.676 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "5b4ca4e8-cecc-4163-b291-c9209b02c4b7", 2024-09-16T09:40:46.676 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2024-09-16T09:40:46.676 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2024-09-16T09:40:46.676 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2024-09-16T09:40:46.677 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 0.1810150146484375, 2024-09-16T09:40:46.677 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2024-09-16T09:40:46.677 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2024-09-16T09:40:46.677 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 18, 2024-09-16T09:40:46.677 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 438, 2024-09-16T09:40:46.677 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2024-09-16T09:40:46.677 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2024-09-16T09:40:46.677 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2024-09-16T09:40:46.677 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:40:46.677 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.677 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:40:46.678 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6818", 2024-09-16T09:40:46.678 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3601682336 2024-09-16T09:40:46.678 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.678 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.678 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:40:46.678 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6819", 2024-09-16T09:40:46.678 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3601682336 2024-09-16T09:40:46.678 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:46.678 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:46.678 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.678 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2024-09-16T09:40:46.678 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:40:46.679 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.679 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:40:46.679 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6820", 2024-09-16T09:40:46.679 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3601682336 2024-09-16T09:40:46.679 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.679 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.679 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:40:46.680 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6821", 2024-09-16T09:40:46.680 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3601682336 2024-09-16T09:40:46.680 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:46.680 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:46.680 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.680 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2024-09-16T09:40:46.680 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:40:46.680 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.680 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:40:46.680 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6824", 2024-09-16T09:40:46.680 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3601682336 2024-09-16T09:40:46.680 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.681 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.681 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:40:46.681 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6825", 2024-09-16T09:40:46.681 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3601682336 2024-09-16T09:40:46.681 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:46.681 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:46.681 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.681 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2024-09-16T09:40:46.681 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:40:46.681 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.681 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:40:46.682 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6822", 2024-09-16T09:40:46.682 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3601682336 2024-09-16T09:40:46.682 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.682 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.682 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:40:46.682 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6823", 2024-09-16T09:40:46.682 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3601682336 2024-09-16T09:40:46.682 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:46.682 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:46.682 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.682 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6819/3601682336", 2024-09-16T09:40:46.682 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6821/3601682336", 2024-09-16T09:40:46.683 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6825/3601682336", 2024-09-16T09:40:46.683 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6823/3601682336", 2024-09-16T09:40:46.683 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2024-09-16T09:40:46.683 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2024-09-16T09:40:46.683 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2024-09-16T09:40:46.683 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:46.683 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.683 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.683 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 3, 2024-09-16T09:40:46.683 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "ffb8d527-1b58-4ecd-808c-47d4ba1bf387", 2024-09-16T09:40:46.683 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2024-09-16T09:40:46.683 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2024-09-16T09:40:46.684 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2024-09-16T09:40:46.684 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 0.476531982421875, 2024-09-16T09:40:46.684 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2024-09-16T09:40:46.684 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2024-09-16T09:40:46.684 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 26, 2024-09-16T09:40:46.684 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 434, 2024-09-16T09:40:46.684 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2024-09-16T09:40:46.684 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2024-09-16T09:40:46.684 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2024-09-16T09:40:46.684 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:40:46.684 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.684 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:40:46.685 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6826", 2024-09-16T09:40:46.685 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2038083007 2024-09-16T09:40:46.685 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.685 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.685 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:40:46.685 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6827", 2024-09-16T09:40:46.685 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2038083007 2024-09-16T09:40:46.685 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:46.685 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:46.685 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.685 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2024-09-16T09:40:46.685 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:40:46.686 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.686 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:40:46.686 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6828", 2024-09-16T09:40:46.686 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2038083007 2024-09-16T09:40:46.686 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.686 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.686 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:40:46.686 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6829", 2024-09-16T09:40:46.686 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2038083007 2024-09-16T09:40:46.686 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:46.686 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:46.687 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.687 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2024-09-16T09:40:46.687 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:40:46.687 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.687 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:40:46.687 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6832", 2024-09-16T09:40:46.687 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2038083007 2024-09-16T09:40:46.687 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.687 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.687 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:40:46.687 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6833", 2024-09-16T09:40:46.688 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2038083007 2024-09-16T09:40:46.688 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:46.688 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:46.688 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.688 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2024-09-16T09:40:46.688 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:40:46.688 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.688 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:40:46.688 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6830", 2024-09-16T09:40:46.689 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2038083007 2024-09-16T09:40:46.689 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.689 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.689 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:40:46.689 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6831", 2024-09-16T09:40:46.689 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2038083007 2024-09-16T09:40:46.689 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:46.689 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:46.689 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.689 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6827/2038083007", 2024-09-16T09:40:46.689 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6829/2038083007", 2024-09-16T09:40:46.689 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6833/2038083007", 2024-09-16T09:40:46.690 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6831/2038083007", 2024-09-16T09:40:46.690 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2024-09-16T09:40:46.690 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2024-09-16T09:40:46.690 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2024-09-16T09:40:46.690 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:46.690 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.690 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.690 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 4, 2024-09-16T09:40:46.690 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "a946760b-4d91-4a4b-8e9b-a07a50589211", 2024-09-16T09:40:46.690 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2024-09-16T09:40:46.690 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2024-09-16T09:40:46.690 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2024-09-16T09:40:46.691 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2024-09-16T09:40:46.691 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2024-09-16T09:40:46.691 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2024-09-16T09:40:46.691 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 32, 2024-09-16T09:40:46.691 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 439, 2024-09-16T09:40:46.691 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2024-09-16T09:40:46.691 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2024-09-16T09:40:46.691 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2024-09-16T09:40:46.691 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:40:46.691 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.691 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:40:46.691 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6800", 2024-09-16T09:40:46.692 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2250892710 2024-09-16T09:40:46.692 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.692 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.692 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:40:46.692 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6801", 2024-09-16T09:40:46.692 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2250892710 2024-09-16T09:40:46.692 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:46.692 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:46.692 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.692 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2024-09-16T09:40:46.693 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:40:46.693 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.693 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:40:46.693 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6802", 2024-09-16T09:40:46.693 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2250892710 2024-09-16T09:40:46.693 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.693 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.693 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:40:46.693 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6803", 2024-09-16T09:40:46.693 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2250892710 2024-09-16T09:40:46.693 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:46.694 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:46.694 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.694 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2024-09-16T09:40:46.694 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:40:46.694 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.694 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:40:46.694 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6806", 2024-09-16T09:40:46.694 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2250892710 2024-09-16T09:40:46.694 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.694 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.694 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:40:46.694 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6807", 2024-09-16T09:40:46.695 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2250892710 2024-09-16T09:40:46.695 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:46.695 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:46.695 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.695 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2024-09-16T09:40:46.695 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:40:46.695 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.695 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:40:46.695 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6804", 2024-09-16T09:40:46.695 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2250892710 2024-09-16T09:40:46.695 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.695 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.695 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:40:46.696 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6805", 2024-09-16T09:40:46.696 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2250892710 2024-09-16T09:40:46.696 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:46.696 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:46.696 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.696 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.84:6801/2250892710", 2024-09-16T09:40:46.696 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.84:6803/2250892710", 2024-09-16T09:40:46.696 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.84:6807/2250892710", 2024-09-16T09:40:46.696 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.84:6805/2250892710", 2024-09-16T09:40:46.696 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2024-09-16T09:40:46.696 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2024-09-16T09:40:46.697 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2024-09-16T09:40:46.697 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:46.697 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.697 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.697 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 5, 2024-09-16T09:40:46.697 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "67681760-75a9-4846-946e-a935d42c5e05", 2024-09-16T09:40:46.697 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2024-09-16T09:40:46.697 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2024-09-16T09:40:46.697 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2024-09-16T09:40:46.697 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2024-09-16T09:40:46.697 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2024-09-16T09:40:46.698 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2024-09-16T09:40:46.698 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 39, 2024-09-16T09:40:46.698 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 420, 2024-09-16T09:40:46.698 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2024-09-16T09:40:46.698 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2024-09-16T09:40:46.698 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2024-09-16T09:40:46.698 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:40:46.698 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.698 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:40:46.698 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6808", 2024-09-16T09:40:46.698 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1803945348 2024-09-16T09:40:46.698 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.699 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.699 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:40:46.699 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6809", 2024-09-16T09:40:46.699 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1803945348 2024-09-16T09:40:46.699 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:46.699 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:46.699 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.699 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2024-09-16T09:40:46.699 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:40:46.699 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.700 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:40:46.700 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6810", 2024-09-16T09:40:46.700 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1803945348 2024-09-16T09:40:46.700 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.700 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.700 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:40:46.700 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6811", 2024-09-16T09:40:46.700 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1803945348 2024-09-16T09:40:46.700 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:46.700 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:46.700 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.701 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2024-09-16T09:40:46.701 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:40:46.701 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.701 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:40:46.701 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6814", 2024-09-16T09:40:46.701 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1803945348 2024-09-16T09:40:46.701 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.701 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.701 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:40:46.701 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6815", 2024-09-16T09:40:46.701 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1803945348 2024-09-16T09:40:46.701 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:46.702 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:46.702 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.702 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2024-09-16T09:40:46.702 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:40:46.702 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.702 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:40:46.702 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6812", 2024-09-16T09:40:46.702 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1803945348 2024-09-16T09:40:46.702 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.702 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.702 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:40:46.702 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6813", 2024-09-16T09:40:46.703 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1803945348 2024-09-16T09:40:46.703 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:46.703 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:46.703 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.703 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.84:6809/1803945348", 2024-09-16T09:40:46.703 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.84:6811/1803945348", 2024-09-16T09:40:46.703 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.84:6815/1803945348", 2024-09-16T09:40:46.703 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.84:6813/1803945348", 2024-09-16T09:40:46.703 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2024-09-16T09:40:46.703 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2024-09-16T09:40:46.703 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2024-09-16T09:40:46.704 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:46.704 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.704 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.704 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 6, 2024-09-16T09:40:46.704 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "7c8f0152-f539-4b7e-8113-91dd7529d3a1", 2024-09-16T09:40:46.704 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2024-09-16T09:40:46.704 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2024-09-16T09:40:46.704 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2024-09-16T09:40:46.704 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2024-09-16T09:40:46.704 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2024-09-16T09:40:46.704 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2024-09-16T09:40:46.705 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 46, 2024-09-16T09:40:46.705 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 418, 2024-09-16T09:40:46.705 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2024-09-16T09:40:46.705 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2024-09-16T09:40:46.705 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2024-09-16T09:40:46.705 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:40:46.705 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.705 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:40:46.705 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6816", 2024-09-16T09:40:46.705 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 610727189 2024-09-16T09:40:46.705 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.706 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.706 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:40:46.706 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6817", 2024-09-16T09:40:46.706 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 610727189 2024-09-16T09:40:46.706 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:46.706 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:46.706 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.706 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2024-09-16T09:40:46.706 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:40:46.706 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.706 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:40:46.706 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6818", 2024-09-16T09:40:46.706 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 610727189 2024-09-16T09:40:46.707 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.707 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.707 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:40:46.707 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6819", 2024-09-16T09:40:46.707 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 610727189 2024-09-16T09:40:46.707 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:46.707 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:46.707 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.707 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2024-09-16T09:40:46.707 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:40:46.707 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.707 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:40:46.707 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6822", 2024-09-16T09:40:46.708 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 610727189 2024-09-16T09:40:46.708 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.708 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.708 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:40:46.708 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6823", 2024-09-16T09:40:46.708 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 610727189 2024-09-16T09:40:46.708 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:46.708 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:46.708 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.708 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2024-09-16T09:40:46.708 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:40:46.709 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.709 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:40:46.709 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6820", 2024-09-16T09:40:46.709 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 610727189 2024-09-16T09:40:46.709 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.709 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.709 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:40:46.709 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6821", 2024-09-16T09:40:46.709 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 610727189 2024-09-16T09:40:46.709 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:46.709 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:46.709 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.710 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.84:6817/610727189", 2024-09-16T09:40:46.710 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.84:6819/610727189", 2024-09-16T09:40:46.710 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.84:6823/610727189", 2024-09-16T09:40:46.710 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T09:40:46.710 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:40:46.711 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.84:6821/610727189", 2024-09-16T09:40:46.711 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2024-09-16T09:40:46.711 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2024-09-16T09:40:46.711 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2024-09-16T09:40:46.711 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:46.711 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.711 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.711 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 7, 2024-09-16T09:40:46.711 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "945efaba-225e-49af-aefd-b4d58ca8093b", 2024-09-16T09:40:46.711 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2024-09-16T09:40:46.711 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2024-09-16T09:40:46.711 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2024-09-16T09:40:46.712 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2024-09-16T09:40:46.712 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2024-09-16T09:40:46.712 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2024-09-16T09:40:46.712 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 53, 2024-09-16T09:40:46.712 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 416, 2024-09-16T09:40:46.712 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2024-09-16T09:40:46.712 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2024-09-16T09:40:46.712 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2024-09-16T09:40:46.712 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:40:46.712 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.713 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:40:46.713 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6824", 2024-09-16T09:40:46.713 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1002252531 2024-09-16T09:40:46.713 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.713 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.713 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:40:46.713 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6825", 2024-09-16T09:40:46.713 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1002252531 2024-09-16T09:40:46.713 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:46.713 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:46.713 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.713 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2024-09-16T09:40:46.714 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:40:46.714 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.714 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:40:46.714 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6826", 2024-09-16T09:40:46.714 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1002252531 2024-09-16T09:40:46.714 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.714 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.714 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:40:46.714 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6827", 2024-09-16T09:40:46.714 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1002252531 2024-09-16T09:40:46.714 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:46.714 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:46.715 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.715 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2024-09-16T09:40:46.715 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:40:46.715 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.715 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:40:46.715 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6830", 2024-09-16T09:40:46.715 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1002252531 2024-09-16T09:40:46.715 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.715 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.715 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:40:46.715 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6831", 2024-09-16T09:40:46.715 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1002252531 2024-09-16T09:40:46.716 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:46.716 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:46.716 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.716 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2024-09-16T09:40:46.716 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:40:46.716 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.716 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:40:46.716 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6828", 2024-09-16T09:40:46.716 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1002252531 2024-09-16T09:40:46.716 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.716 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.716 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:40:46.716 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6829", 2024-09-16T09:40:46.717 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1002252531 2024-09-16T09:40:46.717 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:46.717 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:46.717 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.717 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.84:6825/1002252531", 2024-09-16T09:40:46.717 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.84:6827/1002252531", 2024-09-16T09:40:46.717 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.84:6831/1002252531", 2024-09-16T09:40:46.717 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.84:6829/1002252531", 2024-09-16T09:40:46.717 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2024-09-16T09:40:46.717 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2024-09-16T09:40:46.718 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2024-09-16T09:40:46.718 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:46.718 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:46.718 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-16T09:40:46.718 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_xinfo": [ 2024-09-16T09:40:46.718 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.718 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 0, 2024-09-16T09:40:46.718 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2024-09-16T09:40:46.718 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2024-09-16T09:40:46.718 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2024-09-16T09:40:46.718 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540719139924082687, 2024-09-16T09:40:46.718 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2024-09-16T09:40:46.719 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2024-09-16T09:40:05.066170+0000", 2024-09-16T09:40:46.719 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2024-09-16T09:40:46.719 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.719 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.719 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 1, 2024-09-16T09:40:46.719 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2024-09-16T09:40:46.719 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2024-09-16T09:40:46.719 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2024-09-16T09:40:46.719 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540719139924082687, 2024-09-16T09:40:46.719 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2024-09-16T09:40:46.719 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2024-09-16T09:39:46.783782+0000", 2024-09-16T09:40:46.720 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2024-09-16T09:40:46.720 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.720 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.720 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 2, 2024-09-16T09:40:46.720 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2024-09-16T09:40:46.720 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2024-09-16T09:40:46.720 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2024-09-16T09:40:46.720 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540719139924082687, 2024-09-16T09:40:46.720 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2024-09-16T09:40:46.720 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2024-09-16T09:40:02.687893+0000", 2024-09-16T09:40:46.720 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2024-09-16T09:40:46.720 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.720 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.721 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 3, 2024-09-16T09:40:46.721 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2024-09-16T09:40:46.721 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2024-09-16T09:40:46.721 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2024-09-16T09:40:46.721 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540719139924082687, 2024-09-16T09:40:46.721 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2024-09-16T09:40:46.721 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2024-09-16T09:39:56.587649+0000", 2024-09-16T09:40:46.721 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2024-09-16T09:40:46.721 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.721 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.721 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 4, 2024-09-16T09:40:46.722 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2024-09-16T09:40:46.722 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2024-09-16T09:40:46.722 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2024-09-16T09:40:46.722 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540719139924082687, 2024-09-16T09:40:46.722 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2024-09-16T09:40:46.722 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2024-09-16T09:39:51.841830+0000", 2024-09-16T09:40:46.722 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2024-09-16T09:40:46.722 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.722 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.722 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 5, 2024-09-16T09:40:46.722 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2024-09-16T09:40:46.722 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2024-09-16T09:40:46.723 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2024-09-16T09:40:46.723 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540719139924082687, 2024-09-16T09:40:46.723 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2024-09-16T09:40:46.723 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2024-09-16T09:40:11.033015+0000", 2024-09-16T09:40:46.723 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2024-09-16T09:40:46.723 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.723 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.723 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 6, 2024-09-16T09:40:46.723 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2024-09-16T09:40:46.723 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2024-09-16T09:40:46.723 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2024-09-16T09:40:46.723 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540719139924082687, 2024-09-16T09:40:46.724 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2024-09-16T09:40:46.724 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2024-09-16T09:39:53.193344+0000", 2024-09-16T09:40:46.724 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2024-09-16T09:40:46.724 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.724 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.724 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 7, 2024-09-16T09:40:46.724 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2024-09-16T09:40:46.724 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2024-09-16T09:40:46.724 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2024-09-16T09:40:46.724 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540719139924082687, 2024-09-16T09:40:46.724 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2024-09-16T09:40:46.725 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2024-09-16T09:40:09.520391+0000", 2024-09-16T09:40:46.725 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2024-09-16T09:40:46.725 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:46.725 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-16T09:40:46.725 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap": [ 2024-09-16T09:40:46.725 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.725 INFO:teuthology.orchestra.run.smithi070.stdout: "pgid": "3.1d", 2024-09-16T09:40:46.725 INFO:teuthology.orchestra.run.smithi070.stdout: "osds": [ 2024-09-16T09:40:46.725 INFO:teuthology.orchestra.run.smithi070.stdout: 1, 2024-09-16T09:40:46.725 INFO:teuthology.orchestra.run.smithi070.stdout: 0, 2024-09-16T09:40:46.725 INFO:teuthology.orchestra.run.smithi070.stdout: 7 2024-09-16T09:40:46.726 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:46.726 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.726 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.726 INFO:teuthology.orchestra.run.smithi070.stdout: "pgid": "4.b", 2024-09-16T09:40:46.726 INFO:teuthology.orchestra.run.smithi070.stdout: "osds": [ 2024-09-16T09:40:46.726 INFO:teuthology.orchestra.run.smithi070.stdout: 0, 2024-09-16T09:40:46.726 INFO:teuthology.orchestra.run.smithi070.stdout: 5, 2024-09-16T09:40:46.726 INFO:teuthology.orchestra.run.smithi070.stdout: 1 2024-09-16T09:40:46.726 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:46.726 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:46.726 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-16T09:40:46.726 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap_items": [ 2024-09-16T09:40:46.727 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.727 INFO:teuthology.orchestra.run.smithi070.stdout: "pgid": "2.1", 2024-09-16T09:40:46.727 INFO:teuthology.orchestra.run.smithi070.stdout: "mappings": [ 2024-09-16T09:40:46.727 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.727 INFO:teuthology.orchestra.run.smithi070.stdout: "from": 3, 2024-09-16T09:40:46.727 INFO:teuthology.orchestra.run.smithi070.stdout: "to": 0 2024-09-16T09:40:46.727 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.727 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.727 INFO:teuthology.orchestra.run.smithi070.stdout: "from": 2, 2024-09-16T09:40:46.727 INFO:teuthology.orchestra.run.smithi070.stdout: "to": 1 2024-09-16T09:40:46.728 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:46.728 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:46.728 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.728 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.728 INFO:teuthology.orchestra.run.smithi070.stdout: "pgid": "3.1b", 2024-09-16T09:40:46.728 INFO:teuthology.orchestra.run.smithi070.stdout: "mappings": [ 2024-09-16T09:40:46.728 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.728 INFO:teuthology.orchestra.run.smithi070.stdout: "from": 0, 2024-09-16T09:40:46.728 INFO:teuthology.orchestra.run.smithi070.stdout: "to": 6 2024-09-16T09:40:46.728 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.728 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.729 INFO:teuthology.orchestra.run.smithi070.stdout: "from": 4, 2024-09-16T09:40:46.729 INFO:teuthology.orchestra.run.smithi070.stdout: "to": 7 2024-09-16T09:40:46.729 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:46.729 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:46.729 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.729 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.729 INFO:teuthology.orchestra.run.smithi070.stdout: "pgid": "4.f", 2024-09-16T09:40:46.729 INFO:teuthology.orchestra.run.smithi070.stdout: "mappings": [ 2024-09-16T09:40:46.729 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.730 INFO:teuthology.orchestra.run.smithi070.stdout: "from": 1, 2024-09-16T09:40:46.730 INFO:teuthology.orchestra.run.smithi070.stdout: "to": 2 2024-09-16T09:40:46.730 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:46.730 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:46.730 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.730 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.730 INFO:teuthology.orchestra.run.smithi070.stdout: "pgid": "4.13", 2024-09-16T09:40:46.730 INFO:teuthology.orchestra.run.smithi070.stdout: "mappings": [ 2024-09-16T09:40:46.730 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.730 INFO:teuthology.orchestra.run.smithi070.stdout: "from": 1, 2024-09-16T09:40:46.731 INFO:teuthology.orchestra.run.smithi070.stdout: "to": 5 2024-09-16T09:40:46.731 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:46.731 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:46.731 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.731 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.731 INFO:teuthology.orchestra.run.smithi070.stdout: "pgid": "4.17", 2024-09-16T09:40:46.731 INFO:teuthology.orchestra.run.smithi070.stdout: "mappings": [ 2024-09-16T09:40:46.731 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.731 INFO:teuthology.orchestra.run.smithi070.stdout: "from": 1, 2024-09-16T09:40:46.731 INFO:teuthology.orchestra.run.smithi070.stdout: "to": 2 2024-09-16T09:40:46.731 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:46.731 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:46.732 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.732 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.732 INFO:teuthology.orchestra.run.smithi070.stdout: "pgid": "5.10", 2024-09-16T09:40:46.732 INFO:teuthology.orchestra.run.smithi070.stdout: "mappings": [ 2024-09-16T09:40:46.732 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.732 INFO:teuthology.orchestra.run.smithi070.stdout: "from": 7, 2024-09-16T09:40:46.732 INFO:teuthology.orchestra.run.smithi070.stdout: "to": 4 2024-09-16T09:40:46.732 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:46.732 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:46.732 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:46.732 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-16T09:40:46.733 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap_primaries": [], 2024-09-16T09:40:46.733 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_temp": [ 2024-09-16T09:40:46.733 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:46.733 INFO:teuthology.orchestra.run.smithi070.stdout: "pgid": "10.12", 2024-09-16T09:40:46.733 INFO:teuthology.orchestra.run.smithi070.stdout: "osds": [ 2024-09-16T09:40:46.733 INFO:teuthology.orchestra.run.smithi070.stdout: 0, 2024-09-16T09:40:46.733 INFO:teuthology.orchestra.run.smithi070.stdout: 1, 2024-09-16T09:40:46.733 INFO:teuthology.orchestra.run.smithi070.stdout: 4 2024-09-16T09:40:46.733 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:46.733 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:46.733 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-16T09:40:46.734 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_temp": [], 2024-09-16T09:40:46.734 INFO:teuthology.orchestra.run.smithi070.stdout: "blocklist": { 2024-09-16T09:40:46.734 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/125491368": "2024-09-17T09:25:31.465167+0000", 2024-09-16T09:40:46.734 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/3354284852": "2024-09-17T09:19:43.205989+0000", 2024-09-16T09:40:46.734 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/3284625547": "2024-09-17T09:25:31.465167+0000", 2024-09-16T09:40:46.734 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/186565817": "2024-09-17T09:19:43.205989+0000", 2024-09-16T09:40:46.734 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/2213852865": "2024-09-17T09:19:43.205989+0000", 2024-09-16T09:40:46.734 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/1812373760": "2024-09-17T09:20:09.306330+0000", 2024-09-16T09:40:46.734 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:6800/837338094": "2024-09-17T09:19:43.205989+0000", 2024-09-16T09:40:46.734 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:6800/1876661229": "2024-09-17T09:25:31.465167+0000", 2024-09-16T09:40:46.734 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/3351688485": "2024-09-17T09:20:09.306330+0000", 2024-09-16T09:40:46.735 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:6800/3871553358": "2024-09-17T09:20:09.306330+0000", 2024-09-16T09:40:46.735 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/1230482871": "2024-09-17T09:25:31.465167+0000", 2024-09-16T09:40:46.735 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:6801/837338094": "2024-09-17T09:19:43.205989+0000", 2024-09-16T09:40:46.735 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:6801/3871553358": "2024-09-17T09:20:09.306330+0000", 2024-09-16T09:40:46.735 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/1403001486": "2024-09-17T09:20:09.306330+0000", 2024-09-16T09:40:46.735 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/1723643739": "2024-09-17T09:25:31.465167+0000", 2024-09-16T09:40:46.735 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:6801/1876661229": "2024-09-17T09:25:31.465167+0000", 2024-09-16T09:40:46.735 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/3456417396": "2024-09-17T09:25:31.465167+0000" 2024-09-16T09:40:46.735 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.735 INFO:teuthology.orchestra.run.smithi070.stdout: "range_blocklist": {}, 2024-09-16T09:40:46.735 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profiles": { 2024-09-16T09:40:46.736 INFO:teuthology.orchestra.run.smithi070.stdout: "default": { 2024-09-16T09:40:46.736 INFO:teuthology.orchestra.run.smithi070.stdout: "crush-failure-domain": "osd", 2024-09-16T09:40:46.736 INFO:teuthology.orchestra.run.smithi070.stdout: "k": "2", 2024-09-16T09:40:46.736 INFO:teuthology.orchestra.run.smithi070.stdout: "m": "1", 2024-09-16T09:40:46.736 INFO:teuthology.orchestra.run.smithi070.stdout: "plugin": "jerasure", 2024-09-16T09:40:46.736 INFO:teuthology.orchestra.run.smithi070.stdout: "technique": "reed_sol_van" 2024-09-16T09:40:46.736 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:46.736 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:46.736 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps_queue": [], 2024-09-16T09:40:46.736 INFO:teuthology.orchestra.run.smithi070.stdout: "new_removed_snaps": [], 2024-09-16T09:40:46.737 INFO:teuthology.orchestra.run.smithi070.stdout: "new_purged_snaps": [], 2024-09-16T09:40:46.737 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_node_flags": {}, 2024-09-16T09:40:46.737 INFO:teuthology.orchestra.run.smithi070.stdout: "device_class_flags": {}, 2024-09-16T09:40:46.737 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode": { 2024-09-16T09:40:46.737 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode_enabled": false, 2024-09-16T09:40:46.737 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_bucket_count": 0, 2024-09-16T09:40:46.737 INFO:teuthology.orchestra.run.smithi070.stdout: "degraded_stretch_mode": 0, 2024-09-16T09:40:46.737 INFO:teuthology.orchestra.run.smithi070.stdout: "recovering_stretch_mode": 0, 2024-09-16T09:40:46.737 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode_bucket": 0 2024-09-16T09:40:46.737 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:46.737 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T09:40:46.766 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:40:46.802 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:46.803 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:46.804 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:40:46.904 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:40:46.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:46.936 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:46.937 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:40:47.037 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:40:47.079 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:47.080 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:47.080 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:40:47.181 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:40:47.212 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:47.213 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:47.214 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:40:47.314 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:40:47.350 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:47.350 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:47.351 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:40:47.452 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:40:47.487 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:47.488 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:47.488 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:40:47.589 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:40:47.630 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:47.631 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:47.631 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:40:47.732 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:40:47.760 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:47.760 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:47.761 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:40:47.778 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:47 smithi070 ceph-mon[30388]: pgmap v829: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 1.6 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 757 op/s; 2226/79422 objects misplaced (2.803%) 2024-09-16T09:40:47.778 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:47 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1673776175' entity='client.admin' cmd={"prefix": "osd dump", "format": "json-pretty"} : dispatch 2024-09-16T09:40:47.778 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:47 smithi070 ceph-mon[40691]: pgmap v829: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 1.6 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 757 op/s; 2226/79422 objects misplaced (2.803%) 2024-09-16T09:40:47.778 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:47 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1673776175' entity='client.admin' cmd={"prefix": "osd dump", "format": "json-pretty"} : dispatch 2024-09-16T09:40:47.824 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.3 dump_blocked_ops 2024-09-16T09:40:47.861 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:40:47.897 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:47.898 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:47.898 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:40:47.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:47 smithi084 ceph-mon[32629]: pgmap v829: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 1.6 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 757 op/s; 2226/79422 objects misplaced (2.803%) 2024-09-16T09:40:47.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:47 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1673776175' entity='client.admin' cmd={"prefix": "osd dump", "format": "json-pretty"} : dispatch 2024-09-16T09:40:48.000 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:40:48.033 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:48.034 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:48.034 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:40:48.135 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:40:48.165 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:48.165 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:48.166 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:40:48.236 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:40:48.258 INFO:tasks.thrashosds.thrasher:j is {'epoch': 442, 'fsid': 'a1321834-740c-11ef-bceb-c7b262605968', 'created': '2024-09-16T09:19:14.121498+0000', 'modified': '2024-09-16T09:40:40.423641+0000', 'last_up_change': '2024-09-16T09:24:51.124496+0000', 'last_in_change': '2024-09-16T09:24:28.432920+0000', 'flags': 'noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5801984, 'flags_set': ['noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 18, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 10, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T09:22:27.617694+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 21, 'pg_placement_num': 21, 'pg_placement_num_target': 21, 'pg_num_target': 21, 'pg_num_pending': 21, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '162', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '160', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6699999570846558, 'score_stable': 1.6699999570846558, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 1.899999976158142, 'raw_score_stable': 1.899999976158142, 'primary_affinity_weighted': 0.7071933746337891, 'average_primary_affinity': 0.8100000023841858, 'average_primary_affinity_weighted': 0.7099999785423279}}, {'pool': 2, 'pool_name': 'datapool', 'create_time': '2024-09-16T09:24:59.169244+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 3, 'pg_placement_num': 3, 'pg_placement_num_target': 3, 'pg_num_target': 3, 'pg_num_pending': 3, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 61, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.3299999237060547, 'score_stable': 2.3299999237060547, 'optimal_score': 0.8600000143051147, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.7071933746337891, 'average_primary_affinity': 0.8100000023841858, 'average_primary_affinity_weighted': 0.7099999785423279}}, {'pool': 3, 'pool_name': '.rgw.root', 'create_time': '2024-09-16T09:25:00.056119+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.7071933746337891, 'average_primary_affinity': 0.8100000023841858, 'average_primary_affinity_weighted': 0.7099999785423279}}, {'pool': 4, 'pool_name': 'default.rgw.log', 'create_time': '2024-09-16T09:25:01.770016+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '62', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.849999904632568, 'score_stable': 2.190000057220459, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 7.809999942779541, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 0.7071933746337891, 'average_primary_affinity': 0.8100000023841858, 'average_primary_affinity_weighted': 0.7099999785423279, 'error_message': 'pool 4 has acting primaries on OSD(s) with primary affinity 0, read balance score is not accurate'}}, {'pool': 5, 'pool_name': 'default.rgw.control', 'create_time': '2024-09-16T09:25:03.725555+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.7071933746337891, 'average_primary_affinity': 0.8100000023841858, 'average_primary_affinity_weighted': 0.7099999785423279}}, {'pool': 6, 'pool_name': 'default.rgw.meta', 'create_time': '2024-09-16T09:25:05.773530+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5399999618530273, 'score_stable': 1.5399999618530273, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 0.7071933746337891, 'average_primary_affinity': 0.8100000023841858, 'average_primary_affinity_weighted': 0.7099999785423279}}, {'pool': 10, 'pool_name': 'unique_pool_3', 'create_time': '2024-09-16T09:37:43.482267+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 26, 'pg_placement_num': 26, 'pg_placement_num_target': 26, 'pg_num_target': 26, 'pg_num_pending': 26, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '397', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '395', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rados': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8899999856948853, 'score_stable': 2.1600000858306885, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 2.1500000953674316, 'raw_score_stable': 2.4600000381469727, 'primary_affinity_weighted': 0.7071933746337891, 'average_primary_affinity': 0.8100000023841858, 'average_primary_affinity_weighted': 0.7099999785423279}}], 'osds': [{'osd': 0, 'uuid': '7ef288c8-3bf0-4814-8e86-f70a7de041ce', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 8, 'up_thru': 419, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6802', 'nonce': 2013075781}, {'type': 'v1', 'addr': '172.21.15.70:6803', 'nonce': 2013075781}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6804', 'nonce': 2013075781}, {'type': 'v1', 'addr': '172.21.15.70:6805', 'nonce': 2013075781}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6808', 'nonce': 2013075781}, {'type': 'v1', 'addr': '172.21.15.70:6809', 'nonce': 2013075781}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6806', 'nonce': 2013075781}, {'type': 'v1', 'addr': '172.21.15.70:6807', 'nonce': 2013075781}]}, 'public_addr': '172.21.15.70:6803/2013075781', 'cluster_addr': '172.21.15.70:6805/2013075781', 'heartbeat_back_addr': '172.21.15.70:6809/2013075781', 'heartbeat_front_addr': '172.21.15.70:6807/2013075781', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '474d41d9-295a-450c-8f27-ce487a14678b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 362, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6810', 'nonce': 1670012115}, {'type': 'v1', 'addr': '172.21.15.70:6811', 'nonce': 1670012115}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6812', 'nonce': 1670012115}, {'type': 'v1', 'addr': '172.21.15.70:6813', 'nonce': 1670012115}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6816', 'nonce': 1670012115}, {'type': 'v1', 'addr': '172.21.15.70:6817', 'nonce': 1670012115}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6814', 'nonce': 1670012115}, {'type': 'v1', 'addr': '172.21.15.70:6815', 'nonce': 1670012115}]}, 'public_addr': '172.21.15.70:6811/1670012115', 'cluster_addr': '172.21.15.70:6813/1670012115', 'heartbeat_back_addr': '172.21.15.70:6817/1670012115', 'heartbeat_front_addr': '172.21.15.70:6815/1670012115', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '5b4ca4e8-cecc-4163-b291-c9209b02c4b7', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.1810150146484375, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 18, 'up_thru': 438, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6818', 'nonce': 3601682336}, {'type': 'v1', 'addr': '172.21.15.70:6819', 'nonce': 3601682336}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6820', 'nonce': 3601682336}, {'type': 'v1', 'addr': '172.21.15.70:6821', 'nonce': 3601682336}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6824', 'nonce': 3601682336}, {'type': 'v1', 'addr': '172.21.15.70:6825', 'nonce': 3601682336}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6822', 'nonce': 3601682336}, {'type': 'v1', 'addr': '172.21.15.70:6823', 'nonce': 3601682336}]}, 'public_addr': '172.21.15.70:6819/3601682336', 'cluster_addr': '172.21.15.70:6821/3601682336', 'heartbeat_back_addr': '172.21.15.70:6825/3601682336', 'heartbeat_front_addr': '172.21.15.70:6823/3601682336', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'ffb8d527-1b58-4ecd-808c-47d4ba1bf387', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.476531982421875, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 26, 'up_thru': 434, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6826', 'nonce': 2038083007}, {'type': 'v1', 'addr': '172.21.15.70:6827', 'nonce': 2038083007}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6828', 'nonce': 2038083007}, {'type': 'v1', 'addr': '172.21.15.70:6829', 'nonce': 2038083007}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6832', 'nonce': 2038083007}, {'type': 'v1', 'addr': '172.21.15.70:6833', 'nonce': 2038083007}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6830', 'nonce': 2038083007}, {'type': 'v1', 'addr': '172.21.15.70:6831', 'nonce': 2038083007}]}, 'public_addr': '172.21.15.70:6827/2038083007', 'cluster_addr': '172.21.15.70:6829/2038083007', 'heartbeat_back_addr': '172.21.15.70:6833/2038083007', 'heartbeat_front_addr': '172.21.15.70:6831/2038083007', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'a946760b-4d91-4a4b-8e9b-a07a50589211', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 32, 'up_thru': 439, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6800', 'nonce': 2250892710}, {'type': 'v1', 'addr': '172.21.15.84:6801', 'nonce': 2250892710}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6802', 'nonce': 2250892710}, {'type': 'v1', 'addr': '172.21.15.84:6803', 'nonce': 2250892710}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6806', 'nonce': 2250892710}, {'type': 'v1', 'addr': '172.21.15.84:6807', 'nonce': 2250892710}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6804', 'nonce': 2250892710}, {'type': 'v1', 'addr': '172.21.15.84:6805', 'nonce': 2250892710}]}, 'public_addr': '172.21.15.84:6801/2250892710', 'cluster_addr': '172.21.15.84:6803/2250892710', 'heartbeat_back_addr': '172.21.15.84:6807/2250892710', 'heartbeat_front_addr': '172.21.15.84:6805/2250892710', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '67681760-75a9-4846-946e-a935d42c5e05', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 39, 'up_thru': 420, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6808', 'nonce': 1803945348}, {'type': 'v1', 'addr': '172.21.15.84:6809', 'nonce': 1803945348}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6810', 'nonce': 1803945348}, {'type': 'v1', 'addr': '172.21.15.84:6811', 'nonce': 1803945348}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6814', 'nonce': 1803945348}, {'type': 'v1', 'addr': '172.21.15.84:6815', 'nonce': 1803945348}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6812', 'nonce': 1803945348}, {'type': 'v1', 'addr': '172.21.15.84:6813', 'nonce': 1803945348}]}, 'public_addr': '172.21.15.84:6809/1803945348', 'cluster_addr': '172.21.15.84:6811/1803945348', 'heartbeat_back_addr': '172.21.15.84:6815/1803945348', 'heartbeat_front_addr': '172.21.15.84:6813/1803945348', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '7c8f0152-f539-4b7e-8113-91dd7529d3a1', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 46, 'up_thru': 418, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6816', 'nonce': 610727189}, {'type': 'v1', 'addr': '172.21.15.84:6817', 'nonce': 610727189}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6818', 'nonce': 610727189}, {'type': 'v1', 'addr': '172.21.15.84:6819', 'nonce': 610727189}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6822', 'nonce': 610727189}, {'type': 'v1', 'addr': '172.21.15.84:6823', 'nonce': 610727189}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6820', 'nonce': 610727189}, {'type': 'v1', 'addr': '172.21.15.84:6821', 'nonce': 610727189}]}, 'public_addr': '172.21.15.84:6817/610727189', 'cluster_addr': '172.21.15.84:6819/610727189', 'heartbeat_back_addr': '172.21.15.84:6823/610727189', 'heartbeat_front_addr': '172.21.15.84:6821/610727189', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '945efaba-225e-49af-aefd-b4d58ca8093b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 53, 'up_thru': 416, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6824', 'nonce': 1002252531}, {'type': 'v1', 'addr': '172.21.15.84:6825', 'nonce': 1002252531}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6826', 'nonce': 1002252531}, {'type': 'v1', 'addr': '172.21.15.84:6827', 'nonce': 1002252531}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6830', 'nonce': 1002252531}, {'type': 'v1', 'addr': '172.21.15.84:6831', 'nonce': 1002252531}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6828', 'nonce': 1002252531}, {'type': 'v1', 'addr': '172.21.15.84:6829', 'nonce': 1002252531}]}, 'public_addr': '172.21.15.84:6825/1002252531', 'cluster_addr': '172.21.15.84:6827/1002252531', 'heartbeat_back_addr': '172.21.15.84:6831/1002252531', 'heartbeat_front_addr': '172.21.15.84:6829/1002252531', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T09:40:05.066170+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T09:39:46.783782+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T09:40:02.687893+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T09:39:56.587649+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T09:39:51.841830+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T09:40:11.033015+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T09:39:53.193344+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T09:40:09.520391+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.1d', 'osds': [1, 0, 7]}, {'pgid': '4.b', 'osds': [0, 5, 1]}], 'pg_upmap_items': [{'pgid': '2.1', 'mappings': [{'from': 3, 'to': 0}, {'from': 2, 'to': 1}]}, {'pgid': '3.1b', 'mappings': [{'from': 0, 'to': 6}, {'from': 4, 'to': 7}]}, {'pgid': '4.f', 'mappings': [{'from': 1, 'to': 2}]}, {'pgid': '4.13', 'mappings': [{'from': 1, 'to': 5}]}, {'pgid': '4.17', 'mappings': [{'from': 1, 'to': 2}]}, {'pgid': '5.10', 'mappings': [{'from': 7, 'to': 4}]}], 'pg_upmap_primaries': [], 'pg_temp': [{'pgid': '10.12', 'osds': [0, 1, 4]}], 'primary_temp': [], 'blocklist': {'172.21.15.70:0/125491368': '2024-09-17T09:25:31.465167+0000', '172.21.15.70:0/3354284852': '2024-09-17T09:19:43.205989+0000', '172.21.15.70:0/3284625547': '2024-09-17T09:25:31.465167+0000', '172.21.15.70:0/186565817': '2024-09-17T09:19:43.205989+0000', '172.21.15.70:0/2213852865': '2024-09-17T09:19:43.205989+0000', '172.21.15.70:0/1812373760': '2024-09-17T09:20:09.306330+0000', '172.21.15.70:6800/837338094': '2024-09-17T09:19:43.205989+0000', '172.21.15.70:6800/1876661229': '2024-09-17T09:25:31.465167+0000', '172.21.15.70:0/3351688485': '2024-09-17T09:20:09.306330+0000', '172.21.15.70:6800/3871553358': '2024-09-17T09:20:09.306330+0000', '172.21.15.70:0/1230482871': '2024-09-17T09:25:31.465167+0000', '172.21.15.70:6801/837338094': '2024-09-17T09:19:43.205989+0000', '172.21.15.70:6801/3871553358': '2024-09-17T09:20:09.306330+0000', '172.21.15.70:0/1403001486': '2024-09-17T09:20:09.306330+0000', '172.21.15.70:0/1723643739': '2024-09-17T09:25:31.465167+0000', '172.21.15.70:6801/1876661229': '2024-09-17T09:25:31.465167+0000', '172.21.15.70:0/3456417396': '2024-09-17T09:25:31.465167+0000'}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-16T09:40:48.258 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-16T09:40:48.267 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:40:48.295 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:48.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:48.296 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:40:48.359 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T09:40:48.396 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:40:48.450 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:48.450 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:48.450 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:40:48.551 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:40:48.627 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:48.628 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:48.628 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:40:48.729 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:40:48.765 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:48.766 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:48.766 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:40:48.868 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:40:48.913 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:40:48.918 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:48.919 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:48.920 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:40:48.991 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:40:49.021 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:40:49.054 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:49.055 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:49.055 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:40:49.156 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:40:49.208 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:49.209 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:49.209 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:40:49.310 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:40:49.349 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:49.350 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:49.350 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:40:49.451 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:40:49.481 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:49.481 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:49.482 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:40:49.574 INFO:teuthology.orchestra.run.smithi070.stderr:nodeep-scrub is set 2024-09-16T09:40:49.582 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:40:49.605 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:49.606 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:49.606 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:40:49.707 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:40:49.731 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:49.732 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:49.732 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:40:49.833 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:40:49.854 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:49.854 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:49.855 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:40:49.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:49 smithi084 ceph-mon[32629]: pgmap v830: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 1.6 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 759 op/s; 2226/77346 objects misplaced (2.878%) 2024-09-16T09:40:49.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:49 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:40:49.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:49 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:40:49.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:49 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/946826596' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:40:49.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:49 smithi070 ceph-mon[30388]: pgmap v830: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 1.6 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 759 op/s; 2226/77346 objects misplaced (2.878%) 2024-09-16T09:40:49.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:49 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:40:49.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:49 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:40:49.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:49 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/946826596' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:40:49.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:49 smithi070 ceph-mon[40691]: pgmap v830: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 1.6 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 759 op/s; 2226/77346 objects misplaced (2.878%) 2024-09-16T09:40:49.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:49 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:40:49.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:49 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:40:49.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:49 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/946826596' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:40:49.956 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:40:49.997 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:49.998 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:49.998 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:40:50.099 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:40:50.138 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:50.139 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:50.139 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:40:50.240 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:40:50.278 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:50.279 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:50.279 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:40:50.380 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:40:50.419 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:50.420 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:50.420 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:40:50.521 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:40:50.555 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:50.555 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:50.556 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:40:50.657 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:40:50.692 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:50.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:50.693 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:40:50.794 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:40:50.831 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:50.832 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:50.832 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:40:50.862 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:50 smithi070 ceph-mon[30388]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:40:50.862 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:50 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/946826596' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-16T09:40:50.862 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:50 smithi070 ceph-mon[30388]: osdmap e443: 8 total, 8 up, 8 in 2024-09-16T09:40:50.862 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:50 smithi070 ceph-mon[40691]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:40:50.863 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:50 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/946826596' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-16T09:40:50.863 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:50 smithi070 ceph-mon[40691]: osdmap e443: 8 total, 8 up, 8 in 2024-09-16T09:40:50.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:50 smithi084 ceph-mon[32629]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:40:50.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:50 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/946826596' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-16T09:40:50.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:50 smithi084 ceph-mon[32629]: osdmap e443: 8 total, 8 up, 8 in 2024-09-16T09:40:50.934 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:40:50.973 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:50.973 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:50.974 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:40:51.074 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:40:51.110 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:51.110 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:51.111 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:40:51.212 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:40:51.250 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:51.250 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:51.250 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:40:51.351 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:40:51.379 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:51.380 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:51.380 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:40:51.481 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:40:51.520 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:51.521 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:51.521 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:40:51.521 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T09:40:51.521 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:40:51.575 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:40:51.580 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2024-09-16T09:40:51.622 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:40:51.645 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:51.645 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:51.646 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:40:51.746 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:40:51.779 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:51.780 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:51.780 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:40:51.881 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:40:51.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:51 smithi084 ceph-mon[32629]: pgmap v832: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 1.2 GiB data, 9.4 GiB used, 706 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 823 op/s; 1620/57990 objects misplaced (2.794%) 2024-09-16T09:40:51.913 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:51.914 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:51.914 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:40:51.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:51 smithi070 ceph-mon[30388]: pgmap v832: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 1.2 GiB data, 9.4 GiB used, 706 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 823 op/s; 1620/57990 objects misplaced (2.794%) 2024-09-16T09:40:51.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:51 smithi070 ceph-mon[40691]: pgmap v832: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 1.2 GiB data, 9.4 GiB used, 706 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 823 op/s; 1620/57990 objects misplaced (2.794%) 2024-09-16T09:40:52.015 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:40:52.052 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:52.053 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:52.053 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:40:52.154 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:40:52.187 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:52.188 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:52.189 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:40:52.289 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:40:52.326 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:40:51 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:40:51] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:40:52.326 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:52.327 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:52.327 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:40:52.428 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:40:52.469 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:52.470 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:52.470 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:40:52.556 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:40:52.571 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:40:52.606 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:52.606 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:52.607 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:40:52.707 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:40:52.742 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:52.743 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:52.743 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:40:52.844 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:40:52.879 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:52.879 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:52.880 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:40:52.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:52 smithi084 ceph-mon[32629]: from='client.16215 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:40:52.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:52 smithi070 ceph-mon[30388]: from='client.16215 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:40:52.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:52 smithi070 ceph-mon[40691]: from='client.16215 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:40:52.980 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:40:53.018 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:53.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:53.019 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:40:53.120 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:40:53.122 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.3 dump_historic_ops 2024-09-16T09:40:53.154 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:53.154 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:53.154 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:40:53.255 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:40:53.308 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:53.309 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:53.309 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:40:53.410 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:40:53.447 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:53.448 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:53.448 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:40:53.549 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:40:53.583 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:53.584 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:53.585 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:40:53.685 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:40:53.688 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":832,"stamp":"2024-09-16T09:40:49.829799+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1254234675,"num_objects":19330,"num_object_clones":0,"num_object_copies":57990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1620,"num_objects_unfound":0,"num_objects_dirty":19330,"num_whiteouts":0,"num_read":2081,"num_read_kb":1833,"num_write":157278,"num_write_kb":4244782,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4798,"num_bytes_recovered":305403322,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":121500,"ondisk_log_size":121500,"up":534,"acting":534,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":543,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":9834536,"kb_used_data":4353704,"kb_used_omap":12,"kb_used_meta":5480435,"kb_avail":740126680,"statfs":{"total":767960285184,"available":757889720320,"internally_reserved":0,"allocated":4458192896,"data_stored":4453257089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":5611966002},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-903479296,"num_objects":-13786,"num_object_clones":0,"num_object_copies":-41358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-1194,"num_objects_unfound":0,"num_objects_dirty":-13786,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":7718,"num_write_kb":-224128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-2289,"num_bytes_recovered":-149946368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.389913"},"pg_stats":[{"pgid":"10.16","version":"442'8651","reported_seq":15706,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:47.916856+0000","last_change":"2024-09-16T09:40:40.312408+0000","last_active":"2024-09-16T09:40:47.916856+0000","last_peered":"2024-09-16T09:40:47.916856+0000","last_clean":"2024-09-16T09:40:47.916856+0000","last_became_active":"2024-09-16T09:39:29.389154+0000","last_became_peered":"2024-09-16T09:39:29.389154+0000","last_unstale":"2024-09-16T09:40:47.916856+0000","last_undegraded":"2024-09-16T09:40:47.916856+0000","last_fullsized":"2024-09-16T09:40:47.916856+0000","mapping_epoch":418,"log_start":"390'3368","ondisk_log_start":"390'3368","created":395,"last_epoch_clean":433,"parent":"0.0","parent_split_bits":5,"last_scrub":"441'8316","last_scrub_stamp":"2024-09-16T09:40:40.312213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_clean_scrub_stamp":"2024-09-16T09:40:40.312213+0000","objects_scrubbed":936,"log_size":5283,"log_dups_size":0,"ondisk_log_size":5283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:03.427588+0000","scrub_duration":294,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38535168,"num_objects":588,"num_object_clones":0,"num_object_copies":1764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":588,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":348,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":825,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.452696+0000","last_change":"2024-09-16T09:39:37.489941+0000","last_active":"2024-09-16T09:40:40.452696+0000","last_peered":"2024-09-16T09:40:40.452696+0000","last_clean":"2024-09-16T09:40:40.452696+0000","last_became_active":"2024-09-16T09:25:01.709108+0000","last_became_peered":"2024-09-16T09:25:01.709108+0000","last_unstale":"2024-09-16T09:40:40.452696+0000","last_undegraded":"2024-09-16T09:40:40.452696+0000","last_fullsized":"2024-09-16T09:40:40.452696+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:39:37.489866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:39:37.489866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:06.535991+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"285'15","reported_seq":837,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.448678+0000","last_change":"2024-09-16T09:40:25.147185+0000","last_active":"2024-09-16T09:40:40.448678+0000","last_peered":"2024-09-16T09:40:40.448678+0000","last_clean":"2024-09-16T09:40:40.448678+0000","last_became_active":"2024-09-16T09:25:03.714635+0000","last_became_peered":"2024-09-16T09:25:03.714635+0000","last_unstale":"2024-09-16T09:40:40.448678+0000","last_undegraded":"2024-09-16T09:40:40.448678+0000","last_fullsized":"2024-09-16T09:40:40.448678+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'15","last_scrub_stamp":"2024-09-16T09:40:25.147102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:40:25.147102+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:35.881849+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":440,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.450407+0000","last_change":"2024-09-16T09:36:53.098680+0000","last_active":"2024-09-16T09:40:40.450407+0000","last_peered":"2024-09-16T09:40:40.450407+0000","last_clean":"2024-09-16T09:40:40.450407+0000","last_became_active":"2024-09-16T09:33:30.242238+0000","last_became_peered":"2024-09-16T09:33:30.242238+0000","last_unstale":"2024-09-16T09:40:40.450407+0000","last_undegraded":"2024-09-16T09:40:40.450407+0000","last_fullsized":"2024-09-16T09:40:40.450407+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:53.098635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:36:53.098635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":796,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.443106+0000","last_change":"2024-09-16T09:37:56.314518+0000","last_active":"2024-09-16T09:40:40.443106+0000","last_peered":"2024-09-16T09:40:40.443106+0000","last_clean":"2024-09-16T09:40:40.443106+0000","last_became_active":"2024-09-16T09:25:07.715442+0000","last_became_peered":"2024-09-16T09:25:07.715442+0000","last_unstale":"2024-09-16T09:40:40.443106+0000","last_undegraded":"2024-09-16T09:40:40.443106+0000","last_fullsized":"2024-09-16T09:40:40.443106+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:56.314468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:37:56.314468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.17","version":"442'8936","reported_seq":13877,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:48.079655+0000","last_change":"2024-09-16T09:39:38.233207+0000","last_active":"2024-09-16T09:40:48.079655+0000","last_peered":"2024-09-16T09:40:48.079655+0000","last_clean":"2024-09-16T09:40:48.079655+0000","last_became_active":"2024-09-16T09:39:06.504236+0000","last_became_peered":"2024-09-16T09:39:06.504236+0000","last_unstale":"2024-09-16T09:40:48.079655+0000","last_undegraded":"2024-09-16T09:40:48.079655+0000","last_fullsized":"2024-09-16T09:40:48.079655+0000","mapping_epoch":410,"log_start":"407'5111","ondisk_log_start":"407'5111","created":395,"last_epoch_clean":411,"parent":"0.0","parent_split_bits":5,"last_scrub":"422'6258","last_scrub_stamp":"2024-09-16T09:39:38.232956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_clean_scrub_stamp":"2024-09-16T09:39:38.232956+0000","objects_scrubbed":3308,"log_size":3825,"log_dups_size":0,"ondisk_log_size":3825,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:00.944285+0000","scrub_duration":833,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40828928,"num_objects":623,"num_object_clones":0,"num_object_copies":1869,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":623,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2685,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":541,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.448321+0000","last_change":"2024-09-16T09:40:31.723526+0000","last_active":"2024-09-16T09:40:40.448321+0000","last_peered":"2024-09-16T09:40:40.448321+0000","last_clean":"2024-09-16T09:40:40.448321+0000","last_became_active":"2024-09-16T09:31:09.269342+0000","last_became_peered":"2024-09-16T09:31:09.269342+0000","last_unstale":"2024-09-16T09:40:40.448321+0000","last_undegraded":"2024-09-16T09:40:40.448321+0000","last_fullsized":"2024-09-16T09:40:40.448321+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:40:31.723468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:40:31.723468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:58.588310+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":682,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.446846+0000","last_change":"2024-09-16T09:39:28.963009+0000","last_active":"2024-09-16T09:40:40.446846+0000","last_peered":"2024-09-16T09:40:40.446846+0000","last_clean":"2024-09-16T09:40:40.446846+0000","last_became_active":"2024-09-16T09:31:09.275136+0000","last_became_peered":"2024-09-16T09:31:09.275136+0000","last_unstale":"2024-09-16T09:40:40.446846+0000","last_undegraded":"2024-09-16T09:40:40.446846+0000","last_fullsized":"2024-09-16T09:40:40.446846+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:39:28.962926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:39:28.962926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:40:50.065960+0000","scrub_duration":59,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.19","version":"285'27","reported_seq":515,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.455944+0000","last_change":"2024-09-16T09:37:29.683657+0000","last_active":"2024-09-16T09:40:40.455944+0000","last_peered":"2024-09-16T09:40:40.455944+0000","last_clean":"2024-09-16T09:40:40.455944+0000","last_became_active":"2024-09-16T09:33:21.337284+0000","last_became_peered":"2024-09-16T09:33:21.337284+0000","last_unstale":"2024-09-16T09:40:40.455944+0000","last_undegraded":"2024-09-16T09:40:40.455944+0000","last_fullsized":"2024-09-16T09:40:40.455944+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'27","last_scrub_stamp":"2024-09-16T09:37:29.683568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:37:29.683568+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:14.089757+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":816,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.448594+0000","last_change":"2024-09-16T09:40:24.111297+0000","last_active":"2024-09-16T09:40:40.448594+0000","last_peered":"2024-09-16T09:40:40.448594+0000","last_clean":"2024-09-16T09:40:40.448594+0000","last_became_active":"2024-09-16T09:25:05.717236+0000","last_became_peered":"2024-09-16T09:25:05.717236+0000","last_unstale":"2024-09-16T09:40:40.448594+0000","last_undegraded":"2024-09-16T09:40:40.448594+0000","last_fullsized":"2024-09-16T09:40:40.448594+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:40:24.111240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:40:24.111240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:51.101297+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.14","version":"442'8847","reported_seq":16058,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:48.030039+0000","last_change":"2024-09-16T09:40:45.153823+0000","last_active":"2024-09-16T09:40:48.030039+0000","last_peered":"2024-09-16T09:40:48.030039+0000","last_clean":"2024-09-16T09:40:48.030039+0000","last_became_active":"2024-09-16T09:40:13.569230+0000","last_became_peered":"2024-09-16T09:40:13.569230+0000","last_unstale":"2024-09-16T09:40:48.030039+0000","last_undegraded":"2024-09-16T09:40:48.030039+0000","last_fullsized":"2024-09-16T09:40:48.030039+0000","mapping_epoch":436,"log_start":"407'5033","ondisk_log_start":"407'5033","created":395,"last_epoch_clean":437,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_clean_scrub_stamp":"2024-09-16T09:37:44.378497+0000","objects_scrubbed":0,"log_size":3814,"log_dups_size":0,"ondisk_log_size":3814,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:41:15.149874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37683200,"num_objects":575,"num_object_clones":0,"num_object_copies":1725,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":575,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6551,"num_write_kb":228032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2347,"num_bytes_recovered":153812992,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":448,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.453776+0000","last_change":"2024-09-16T09:39:50.457683+0000","last_active":"2024-09-16T09:40:40.453776+0000","last_peered":"2024-09-16T09:40:40.453776+0000","last_clean":"2024-09-16T09:40:40.453776+0000","last_became_active":"2024-09-16T09:33:30.247733+0000","last_became_peered":"2024-09-16T09:33:30.247733+0000","last_unstale":"2024-09-16T09:40:40.453776+0000","last_undegraded":"2024-09-16T09:40:40.453776+0000","last_fullsized":"2024-09-16T09:40:40.453776+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:39:50.457498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:39:50.457498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:40:50.648670+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1a","version":"285'15","reported_seq":837,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.443386+0000","last_change":"2024-09-16T09:40:30.176398+0000","last_active":"2024-09-16T09:40:40.443386+0000","last_peered":"2024-09-16T09:40:40.443386+0000","last_clean":"2024-09-16T09:40:40.443386+0000","last_became_active":"2024-09-16T09:25:03.709147+0000","last_became_peered":"2024-09-16T09:25:03.709147+0000","last_unstale":"2024-09-16T09:40:40.443386+0000","last_undegraded":"2024-09-16T09:40:40.443386+0000","last_fullsized":"2024-09-16T09:40:40.443386+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'15","last_scrub_stamp":"2024-09-16T09:40:30.176320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:40:30.176320+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:41.476752+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":799,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.450272+0000","last_change":"2024-09-16T09:36:51.134364+0000","last_active":"2024-09-16T09:40:40.450272+0000","last_peered":"2024-09-16T09:40:40.450272+0000","last_clean":"2024-09-16T09:40:40.450272+0000","last_became_active":"2024-09-16T09:25:05.718134+0000","last_became_peered":"2024-09-16T09:25:05.718134+0000","last_unstale":"2024-09-16T09:40:40.450272+0000","last_undegraded":"2024-09-16T09:40:40.450272+0000","last_fullsized":"2024-09-16T09:40:40.450272+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:51.134314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:36:51.134314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"66'1","reported_seq":813,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.452295+0000","last_change":"2024-09-16T09:38:24.448521+0000","last_active":"2024-09-16T09:40:40.452295+0000","last_peered":"2024-09-16T09:40:40.452295+0000","last_clean":"2024-09-16T09:40:40.452295+0000","last_became_active":"2024-09-16T09:25:07.716558+0000","last_became_peered":"2024-09-16T09:25:07.716558+0000","last_unstale":"2024-09-16T09:40:40.452295+0000","last_undegraded":"2024-09-16T09:40:40.452295+0000","last_fullsized":"2024-09-16T09:40:40.452295+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-16T09:38:24.448344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:38:24.448344+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.15","version":"442'8847","reported_seq":15939,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:48.523022+0000","last_change":"2024-09-16T09:40:41.783317+0000","last_active":"2024-09-16T09:40:48.523022+0000","last_peered":"2024-09-16T09:40:48.523022+0000","last_clean":"2024-09-16T09:40:48.523022+0000","last_became_active":"2024-09-16T09:40:11.454311+0000","last_became_peered":"2024-09-16T09:40:11.454311+0000","last_unstale":"2024-09-16T09:40:48.523022+0000","last_undegraded":"2024-09-16T09:40:48.523022+0000","last_fullsized":"2024-09-16T09:40:48.523022+0000","mapping_epoch":434,"log_start":"403'4772","ondisk_log_start":"403'4772","created":395,"last_epoch_clean":435,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_clean_scrub_stamp":"2024-09-16T09:37:44.378497+0000","objects_scrubbed":0,"log_size":4075,"log_dups_size":0,"ondisk_log_size":4075,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"user requested scrub scheduled @ 2024-09-16T09:41:11.778571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39944192,"num_objects":610,"num_object_clones":0,"num_object_copies":1830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":610,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6538,"num_write_kb":228704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2308,"num_bytes_recovered":151257088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":799,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.451646+0000","last_change":"2024-09-16T09:33:19.326765+0000","last_active":"2024-09-16T09:40:40.451646+0000","last_peered":"2024-09-16T09:40:40.451646+0000","last_clean":"2024-09-16T09:40:40.451646+0000","last_became_active":"2024-09-16T09:25:01.696939+0000","last_became_peered":"2024-09-16T09:25:01.696939+0000","last_unstale":"2024-09-16T09:40:40.451646+0000","last_undegraded":"2024-09-16T09:40:40.451646+0000","last_fullsized":"2024-09-16T09:40:40.451646+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:19.326695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:19.326695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:21.201594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"285'7","reported_seq":817,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.442574+0000","last_change":"2024-09-16T09:35:39.975915+0000","last_active":"2024-09-16T09:40:40.442574+0000","last_peered":"2024-09-16T09:40:40.442574+0000","last_clean":"2024-09-16T09:40:40.442574+0000","last_became_active":"2024-09-16T09:25:03.716136+0000","last_became_peered":"2024-09-16T09:25:03.716136+0000","last_unstale":"2024-09-16T09:40:40.442574+0000","last_undegraded":"2024-09-16T09:40:40.442574+0000","last_fullsized":"2024-09-16T09:40:40.442574+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'7","last_scrub_stamp":"2024-09-16T09:35:39.975840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:35:39.975840+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":10,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":798,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.447604+0000","last_change":"2024-09-16T09:39:26.850780+0000","last_active":"2024-09-16T09:40:40.447604+0000","last_peered":"2024-09-16T09:40:40.447604+0000","last_clean":"2024-09-16T09:40:40.447604+0000","last_became_active":"2024-09-16T09:25:05.718281+0000","last_became_peered":"2024-09-16T09:25:05.718281+0000","last_unstale":"2024-09-16T09:40:40.447604+0000","last_undegraded":"2024-09-16T09:40:40.447604+0000","last_fullsized":"2024-09-16T09:40:40.447604+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:39:26.850685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:39:26.850685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:40:50.837198+0000","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":825,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.450420+0000","last_change":"2024-09-16T09:38:34.050374+0000","last_active":"2024-09-16T09:40:40.450420+0000","last_peered":"2024-09-16T09:40:40.450420+0000","last_clean":"2024-09-16T09:40:40.450420+0000","last_became_active":"2024-09-16T09:31:09.275656+0000","last_became_peered":"2024-09-16T09:31:09.275656+0000","last_unstale":"2024-09-16T09:40:40.450420+0000","last_undegraded":"2024-09-16T09:40:40.450420+0000","last_fullsized":"2024-09-16T09:40:40.450420+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:38:34.050185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:38:34.050185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"442'8957","reported_seq":9132,"reported_epoch":442,"state":"active+remapped+backfill_toofull","last_fresh":"2024-09-16T09:40:49.427947+0000","last_change":"2024-09-16T09:40:38.693634+0000","last_active":"2024-09-16T09:40:49.427947+0000","last_peered":"2024-09-16T09:40:49.427947+0000","last_clean":"2024-09-16T09:38:36.435420+0000","last_became_active":"2024-09-16T09:38:38.654785+0000","last_became_peered":"2024-09-16T09:38:38.654785+0000","last_unstale":"2024-09-16T09:40:49.427947+0000","last_undegraded":"2024-09-16T09:40:49.427947+0000","last_fullsized":"2024-09-16T09:40:49.427947+0000","mapping_epoch":398,"log_start":"392'4385","ondisk_log_start":"392'4385","created":395,"last_epoch_clean":396,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_clean_scrub_stamp":"2024-09-16T09:37:44.378497+0000","objects_scrubbed":0,"log_size":4572,"log_dups_size":0,"ondisk_log_size":4572,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35356672,"num_objects":540,"num_object_clones":0,"num_object_copies":1620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1620,"num_objects_unfound":0,"num_objects_dirty":540,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6672,"num_write_kb":230752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,5],"acting":[0,1,4],"avail_no_missing":["0","1","4"],"object_location_counts":[{"shards":"0,1,4","objects":540}],"blocked_by":[],"up_primary":6,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":805,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.443426+0000","last_change":"2024-09-16T09:33:48.322010+0000","last_active":"2024-09-16T09:40:40.443426+0000","last_peered":"2024-09-16T09:40:40.443426+0000","last_clean":"2024-09-16T09:40:40.443426+0000","last_became_active":"2024-09-16T09:25:07.720793+0000","last_became_peered":"2024-09-16T09:25:07.720793+0000","last_unstale":"2024-09-16T09:40:40.443426+0000","last_undegraded":"2024-09-16T09:40:40.443426+0000","last_fullsized":"2024-09-16T09:40:40.443426+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:48.321895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:33:48.321895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":556,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.448290+0000","last_change":"2024-09-16T09:34:33.866299+0000","last_active":"2024-09-16T09:40:40.448290+0000","last_peered":"2024-09-16T09:40:40.448290+0000","last_clean":"2024-09-16T09:40:40.448290+0000","last_became_active":"2024-09-16T09:30:39.201824+0000","last_became_peered":"2024-09-16T09:30:39.201824+0000","last_unstale":"2024-09-16T09:40:40.448290+0000","last_undegraded":"2024-09-16T09:40:40.448290+0000","last_fullsized":"2024-09-16T09:40:40.448290+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:33.866142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:34:33.866142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:17.810540+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"285'27","reported_seq":912,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.451523+0000","last_change":"2024-09-16T09:40:23.465274+0000","last_active":"2024-09-16T09:40:40.451523+0000","last_peered":"2024-09-16T09:40:40.451523+0000","last_clean":"2024-09-16T09:40:40.451523+0000","last_became_active":"2024-09-16T09:33:30.249798+0000","last_became_peered":"2024-09-16T09:33:30.249798+0000","last_unstale":"2024-09-16T09:40:40.451523+0000","last_undegraded":"2024-09-16T09:40:40.451523+0000","last_fullsized":"2024-09-16T09:40:40.451523+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'27","last_scrub_stamp":"2024-09-16T09:40:23.465200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:40:23.465200+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:48.319658+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":439,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.442382+0000","last_change":"2024-09-16T09:33:43.326933+0000","last_active":"2024-09-16T09:40:40.442382+0000","last_peered":"2024-09-16T09:40:40.442382+0000","last_clean":"2024-09-16T09:40:40.442382+0000","last_became_active":"2024-09-16T09:33:30.251587+0000","last_became_peered":"2024-09-16T09:33:30.251587+0000","last_unstale":"2024-09-16T09:40:40.442382+0000","last_undegraded":"2024-09-16T09:40:40.442382+0000","last_fullsized":"2024-09-16T09:40:40.442382+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:43.326852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:43.326852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,0],"acting":[4,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.13","version":"442'9010","reported_seq":16194,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:47.470559+0000","last_change":"2024-09-16T09:39:26.490013+0000","last_active":"2024-09-16T09:40:47.470559+0000","last_peered":"2024-09-16T09:40:47.470559+0000","last_clean":"2024-09-16T09:40:47.470559+0000","last_became_active":"2024-09-16T09:39:24.371502+0000","last_became_peered":"2024-09-16T09:39:24.371502+0000","last_unstale":"2024-09-16T09:40:47.470559+0000","last_undegraded":"2024-09-16T09:40:47.470559+0000","last_fullsized":"2024-09-16T09:40:47.470559+0000","mapping_epoch":416,"log_start":"394'4599","ondisk_log_start":"394'4599","created":395,"last_epoch_clean":417,"parent":"0.0","parent_split_bits":5,"last_scrub":"417'5996","last_scrub_stamp":"2024-09-16T09:39:26.489942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_clean_scrub_stamp":"2024-09-16T09:39:26.489942+0000","objects_scrubbed":3617,"log_size":4411,"log_dups_size":0,"ondisk_log_size":4411,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:46.130633+0000","scrub_duration":1608,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40763392,"num_objects":622,"num_object_clones":0,"num_object_copies":1866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":622,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3031,"num_write_kb":1152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":661,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.452830+0000","last_change":"2024-09-16T09:40:27.916994+0000","last_active":"2024-09-16T09:40:40.452830+0000","last_peered":"2024-09-16T09:40:40.452830+0000","last_clean":"2024-09-16T09:40:40.452830+0000","last_became_active":"2024-09-16T09:28:52.974255+0000","last_became_peered":"2024-09-16T09:28:52.974255+0000","last_unstale":"2024-09-16T09:40:40.452830+0000","last_undegraded":"2024-09-16T09:40:40.452830+0000","last_fullsized":"2024-09-16T09:40:40.452830+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:40:27.916860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:40:27.916860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:56.058958+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":816,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.443515+0000","last_change":"2024-09-16T09:35:23.751992+0000","last_active":"2024-09-16T09:40:40.443515+0000","last_peered":"2024-09-16T09:40:40.443515+0000","last_clean":"2024-09-16T09:40:40.443515+0000","last_became_active":"2024-09-16T09:25:01.706491+0000","last_became_peered":"2024-09-16T09:25:01.706491+0000","last_unstale":"2024-09-16T09:40:40.443515+0000","last_undegraded":"2024-09-16T09:40:40.443515+0000","last_fullsized":"2024-09-16T09:40:40.443515+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:35:23.751946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:35:23.751946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"285'18","reported_seq":499,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.442510+0000","last_change":"2024-09-16T09:40:37.206512+0000","last_active":"2024-09-16T09:40:40.442510+0000","last_peered":"2024-09-16T09:40:40.442510+0000","last_clean":"2024-09-16T09:40:40.442510+0000","last_became_active":"2024-09-16T09:33:30.251626+0000","last_became_peered":"2024-09-16T09:33:30.251626+0000","last_unstale":"2024-09-16T09:40:40.442510+0000","last_undegraded":"2024-09-16T09:40:40.442510+0000","last_fullsized":"2024-09-16T09:40:40.442510+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'18","last_scrub_stamp":"2024-09-16T09:40:37.206432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:40:37.206432+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:59.565226+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":33,"num_read_kb":24,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":792,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.443305+0000","last_change":"2024-09-16T09:33:38.238661+0000","last_active":"2024-09-16T09:40:40.443305+0000","last_peered":"2024-09-16T09:40:40.443305+0000","last_clean":"2024-09-16T09:40:40.443305+0000","last_became_active":"2024-09-16T09:25:05.710093+0000","last_became_peered":"2024-09-16T09:25:05.710093+0000","last_unstale":"2024-09-16T09:40:40.443305+0000","last_undegraded":"2024-09-16T09:40:40.443305+0000","last_fullsized":"2024-09-16T09:40:40.443305+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:38.238593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:38.238593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.10","version":"442'8664","reported_seq":13365,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:48.005574+0000","last_change":"2024-09-16T09:39:26.355493+0000","last_active":"2024-09-16T09:40:48.005574+0000","last_peered":"2024-09-16T09:40:48.005574+0000","last_clean":"2024-09-16T09:40:48.005574+0000","last_became_active":"2024-09-16T09:38:54.391768+0000","last_became_peered":"2024-09-16T09:38:54.391768+0000","last_unstale":"2024-09-16T09:40:48.005574+0000","last_undegraded":"2024-09-16T09:40:48.005574+0000","last_fullsized":"2024-09-16T09:40:48.005574+0000","mapping_epoch":405,"log_start":"406'4829","ondisk_log_start":"406'4829","created":395,"last_epoch_clean":406,"parent":"0.0","parent_split_bits":5,"last_scrub":"417'5730","last_scrub_stamp":"2024-09-16T09:39:26.355414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_clean_scrub_stamp":"2024-09-16T09:39:26.355414+0000","objects_scrubbed":3518,"log_size":3835,"log_dups_size":0,"ondisk_log_size":3835,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":936,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39518208,"num_objects":603,"num_object_clones":0,"num_object_copies":1809,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":603,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2941,"num_write_kb":832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":803,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.448629+0000","last_change":"2024-09-16T09:37:48.412421+0000","last_active":"2024-09-16T09:40:40.448629+0000","last_peered":"2024-09-16T09:40:40.448629+0000","last_clean":"2024-09-16T09:40:40.448629+0000","last_became_active":"2024-09-16T09:25:07.719380+0000","last_became_peered":"2024-09-16T09:25:07.719380+0000","last_unstale":"2024-09-16T09:40:40.448629+0000","last_undegraded":"2024-09-16T09:40:40.448629+0000","last_fullsized":"2024-09-16T09:40:40.448629+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-16T09:37:48.411054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:37:48.411054+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:13.081887+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":457,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.448428+0000","last_change":"2024-09-16T09:40:23.587656+0000","last_active":"2024-09-16T09:40:40.448428+0000","last_peered":"2024-09-16T09:40:40.448428+0000","last_clean":"2024-09-16T09:40:40.448428+0000","last_became_active":"2024-09-16T09:33:30.245048+0000","last_became_peered":"2024-09-16T09:33:30.245048+0000","last_unstale":"2024-09-16T09:40:40.448428+0000","last_undegraded":"2024-09-16T09:40:40.448428+0000","last_fullsized":"2024-09-16T09:40:40.448428+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:40:23.587589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:40:23.587589+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:35.058183+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"285'20","reported_seq":881,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.453215+0000","last_change":"2024-09-16T09:39:35.550760+0000","last_active":"2024-09-16T09:40:40.453215+0000","last_peered":"2024-09-16T09:40:40.453215+0000","last_clean":"2024-09-16T09:40:40.453215+0000","last_became_active":"2024-09-16T09:31:09.275050+0000","last_became_peered":"2024-09-16T09:31:09.275050+0000","last_unstale":"2024-09-16T09:40:40.453215+0000","last_undegraded":"2024-09-16T09:40:40.453215+0000","last_fullsized":"2024-09-16T09:40:40.453215+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'20","last_scrub_stamp":"2024-09-16T09:39:35.548333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:39:35.548333+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:40:44.434932+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"66'8","reported_seq":1034,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:41.941262+0000","last_change":"2024-09-16T09:40:38.007095+0000","last_active":"2024-09-16T09:40:41.941262+0000","last_peered":"2024-09-16T09:40:41.941262+0000","last_clean":"2024-09-16T09:40:41.941262+0000","last_became_active":"2024-09-16T09:25:05.712516+0000","last_became_peered":"2024-09-16T09:25:05.712516+0000","last_unstale":"2024-09-16T09:40:41.941262+0000","last_undegraded":"2024-09-16T09:40:41.941262+0000","last_fullsized":"2024-09-16T09:40:41.941262+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:40:38.007034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:40:38.007034+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:01.522582+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1","version":"442'8860","reported_seq":9044,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:48.082017+0000","last_change":"2024-09-16T09:38:37.482511+0000","last_active":"2024-09-16T09:40:48.082017+0000","last_peered":"2024-09-16T09:40:48.082017+0000","last_clean":"2024-09-16T09:40:48.082017+0000","last_became_active":"2024-09-16T09:38:35.385153+0000","last_became_peered":"2024-09-16T09:38:35.385153+0000","last_unstale":"2024-09-16T09:40:48.082017+0000","last_undegraded":"2024-09-16T09:40:48.082017+0000","last_fullsized":"2024-09-16T09:40:48.082017+0000","mapping_epoch":376,"log_start":"406'5004","ondisk_log_start":"406'5004","created":376,"last_epoch_clean":396,"parent":"0.0","parent_split_bits":0,"last_scrub":"396'4603","last_scrub_stamp":"2024-09-16T09:38:37.482451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_clean_scrub_stamp":"2024-09-16T09:38:37.482451+0000","objects_scrubbed":2306,"log_size":3856,"log_dups_size":0,"ondisk_log_size":3856,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":288,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37683200,"num_objects":575,"num_object_clones":0,"num_object_copies":1725,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":575,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4257,"num_write_kb":80832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.a","version":"73'79","reported_seq":739,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.450618+0000","last_change":"2024-09-16T09:37:52.203798+0000","last_active":"2024-09-16T09:40:40.450618+0000","last_peered":"2024-09-16T09:40:40.450618+0000","last_clean":"2024-09-16T09:40:40.450618+0000","last_became_active":"2024-09-16T09:26:35.127352+0000","last_became_peered":"2024-09-16T09:26:35.127352+0000","last_unstale":"2024-09-16T09:40:40.450618+0000","last_undegraded":"2024-09-16T09:40:40.450618+0000","last_fullsized":"2024-09-16T09:40:40.450618+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:37:52.203752+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:37:52.203752+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"285'27","reported_seq":774,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.449932+0000","last_change":"2024-09-16T09:40:24.444708+0000","last_active":"2024-09-16T09:40:40.449932+0000","last_peered":"2024-09-16T09:40:40.449932+0000","last_clean":"2024-09-16T09:40:40.449932+0000","last_became_active":"2024-09-16T09:26:33.295672+0000","last_became_peered":"2024-09-16T09:26:33.295672+0000","last_unstale":"2024-09-16T09:40:40.449932+0000","last_undegraded":"2024-09-16T09:40:40.449932+0000","last_fullsized":"2024-09-16T09:40:40.449932+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'27","last_scrub_stamp":"2024-09-16T09:40:24.444622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:40:24.444622+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:28.482120+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":550,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.448049+0000","last_change":"2024-09-16T09:38:35.023594+0000","last_active":"2024-09-16T09:40:40.448049+0000","last_peered":"2024-09-16T09:40:40.448049+0000","last_clean":"2024-09-16T09:40:40.448049+0000","last_became_active":"2024-09-16T09:31:09.274534+0000","last_became_peered":"2024-09-16T09:31:09.274534+0000","last_unstale":"2024-09-16T09:40:40.448049+0000","last_undegraded":"2024-09-16T09:40:40.448049+0000","last_fullsized":"2024-09-16T09:40:40.448049+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:38:35.023518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:38:35.023518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"66'8","reported_seq":990,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:41.939104+0000","last_change":"2024-09-16T09:37:49.336009+0000","last_active":"2024-09-16T09:40:41.939104+0000","last_peered":"2024-09-16T09:40:41.939104+0000","last_clean":"2024-09-16T09:40:41.939104+0000","last_became_active":"2024-09-16T09:25:05.713813+0000","last_became_peered":"2024-09-16T09:25:05.713813+0000","last_unstale":"2024-09-16T09:40:41.939104+0000","last_undegraded":"2024-09-16T09:40:41.939104+0000","last_fullsized":"2024-09-16T09:40:41.939104+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:37:49.335939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:37:49.335939+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":803,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.450265+0000","last_change":"2024-09-16T09:38:35.081365+0000","last_active":"2024-09-16T09:40:40.450265+0000","last_peered":"2024-09-16T09:40:40.450265+0000","last_clean":"2024-09-16T09:40:40.450265+0000","last_became_active":"2024-09-16T09:25:07.717344+0000","last_became_peered":"2024-09-16T09:25:07.717344+0000","last_unstale":"2024-09-16T09:40:40.450265+0000","last_undegraded":"2024-09-16T09:40:40.450265+0000","last_fullsized":"2024-09-16T09:40:40.450265+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:38:35.081321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:38:35.081321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.e","version":"442'13300","reported_seq":13447,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:48.071135+0000","last_change":"2024-09-16T09:37:45.464349+0000","last_active":"2024-09-16T09:40:48.071135+0000","last_peered":"2024-09-16T09:40:48.071135+0000","last_clean":"2024-09-16T09:40:48.071135+0000","last_became_active":"2024-09-16T09:37:45.463961+0000","last_became_peered":"2024-09-16T09:37:45.463961+0000","last_unstale":"2024-09-16T09:40:48.071135+0000","last_undegraded":"2024-09-16T09:40:48.071135+0000","last_fullsized":"2024-09-16T09:40:48.071135+0000","mapping_epoch":376,"log_start":"429'9485","ondisk_log_start":"429'9485","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_clean_scrub_stamp":"2024-09-16T09:37:44.378497+0000","objects_scrubbed":0,"log_size":3815,"log_dups_size":0,"ondisk_log_size":3815,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79429632,"num_objects":1212,"num_object_clones":0,"num_object_copies":3636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1212,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13300,"num_write_kb":464384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"73'79","reported_seq":779,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.443129+0000","last_change":"2024-09-16T09:40:29.124394+0000","last_active":"2024-09-16T09:40:40.443129+0000","last_peered":"2024-09-16T09:40:40.443129+0000","last_clean":"2024-09-16T09:40:40.443129+0000","last_became_active":"2024-09-16T09:30:08.763683+0000","last_became_peered":"2024-09-16T09:30:08.763683+0000","last_unstale":"2024-09-16T09:40:40.443129+0000","last_undegraded":"2024-09-16T09:40:40.443129+0000","last_fullsized":"2024-09-16T09:40:40.443129+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:40:29.124314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:40:29.124314+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:55.225417+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"285'30","reported_seq":771,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.447907+0000","last_change":"2024-09-16T09:39:52.033613+0000","last_active":"2024-09-16T09:40:40.447907+0000","last_peered":"2024-09-16T09:40:40.447907+0000","last_clean":"2024-09-16T09:40:40.447907+0000","last_became_active":"2024-09-16T09:31:09.270434+0000","last_became_peered":"2024-09-16T09:31:09.270434+0000","last_unstale":"2024-09-16T09:40:40.447907+0000","last_undegraded":"2024-09-16T09:40:40.447907+0000","last_fullsized":"2024-09-16T09:40:40.447907+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'30","last_scrub_stamp":"2024-09-16T09:39:52.033528+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:39:52.033528+0000","objects_scrubbed":8,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:05.444525+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":51,"num_read_kb":39,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":651,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.448742+0000","last_change":"2024-09-16T09:39:52.998500+0000","last_active":"2024-09-16T09:40:40.448742+0000","last_peered":"2024-09-16T09:40:40.448742+0000","last_clean":"2024-09-16T09:40:40.448742+0000","last_became_active":"2024-09-16T09:28:52.983543+0000","last_became_peered":"2024-09-16T09:28:52.983543+0000","last_unstale":"2024-09-16T09:40:40.448742+0000","last_undegraded":"2024-09-16T09:40:40.448742+0000","last_fullsized":"2024-09-16T09:40:40.448742+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:39:52.998427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:39:52.998427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:12.246836+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":823,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.442464+0000","last_change":"2024-09-16T09:35:19.822713+0000","last_active":"2024-09-16T09:40:40.442464+0000","last_peered":"2024-09-16T09:40:40.442464+0000","last_clean":"2024-09-16T09:40:40.442464+0000","last_became_active":"2024-09-16T09:31:09.268864+0000","last_became_peered":"2024-09-16T09:31:09.268864+0000","last_unstale":"2024-09-16T09:40:40.442464+0000","last_undegraded":"2024-09-16T09:40:40.442464+0000","last_fullsized":"2024-09-16T09:40:40.442464+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:35:19.822635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:35:19.822635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":819,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.443296+0000","last_change":"2024-09-16T09:38:28.151882+0000","last_active":"2024-09-16T09:40:40.443296+0000","last_peered":"2024-09-16T09:40:40.443296+0000","last_clean":"2024-09-16T09:40:40.443296+0000","last_became_active":"2024-09-16T09:31:09.269433+0000","last_became_peered":"2024-09-16T09:31:09.269433+0000","last_unstale":"2024-09-16T09:40:40.443296+0000","last_undegraded":"2024-09-16T09:40:40.443296+0000","last_fullsized":"2024-09-16T09:40:40.443296+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:38:28.151683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:38:28.151683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"442'13006","reported_seq":13153,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:47.903004+0000","last_change":"2024-09-16T09:37:45.463119+0000","last_active":"2024-09-16T09:40:47.903004+0000","last_peered":"2024-09-16T09:40:47.903004+0000","last_clean":"2024-09-16T09:40:47.903004+0000","last_became_active":"2024-09-16T09:37:45.462743+0000","last_became_peered":"2024-09-16T09:37:45.462743+0000","last_unstale":"2024-09-16T09:40:47.903004+0000","last_undegraded":"2024-09-16T09:40:47.903004+0000","last_fullsized":"2024-09-16T09:40:47.903004+0000","mapping_epoch":376,"log_start":"423'8200","ondisk_log_start":"423'8200","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_clean_scrub_stamp":"2024-09-16T09:37:44.378497+0000","objects_scrubbed":0,"log_size":4806,"log_dups_size":0,"ondisk_log_size":4806,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":82051072,"num_objects":1252,"num_object_clones":0,"num_object_copies":3756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1252,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13006,"num_write_kb":456256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.4","version":"73'79","reported_seq":542,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.448915+0000","last_change":"2024-09-16T09:37:17.664174+0000","last_active":"2024-09-16T09:40:40.448915+0000","last_peered":"2024-09-16T09:40:40.448915+0000","last_clean":"2024-09-16T09:40:40.448915+0000","last_became_active":"2024-09-16T09:31:09.275139+0000","last_became_peered":"2024-09-16T09:31:09.275139+0000","last_unstale":"2024-09-16T09:40:40.448915+0000","last_undegraded":"2024-09-16T09:40:40.448915+0000","last_fullsized":"2024-09-16T09:40:40.448915+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:37:17.664105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:37:17.664105+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"285'28","reported_seq":855,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.442546+0000","last_change":"2024-09-16T09:35:43.914378+0000","last_active":"2024-09-16T09:40:40.442546+0000","last_peered":"2024-09-16T09:40:40.442546+0000","last_clean":"2024-09-16T09:40:40.442546+0000","last_became_active":"2024-09-16T09:25:03.708830+0000","last_became_peered":"2024-09-16T09:25:03.708830+0000","last_unstale":"2024-09-16T09:40:40.442546+0000","last_undegraded":"2024-09-16T09:40:40.442546+0000","last_fullsized":"2024-09-16T09:40:40.442546+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'28","last_scrub_stamp":"2024-09-16T09:35:43.914312+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:35:43.914312+0000","objects_scrubbed":7,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":842,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.453511+0000","last_change":"2024-09-16T09:40:31.037325+0000","last_active":"2024-09-16T09:40:40.453511+0000","last_peered":"2024-09-16T09:40:40.453511+0000","last_clean":"2024-09-16T09:40:40.453511+0000","last_became_active":"2024-09-16T09:25:01.702992+0000","last_became_peered":"2024-09-16T09:25:01.702992+0000","last_unstale":"2024-09-16T09:40:40.453511+0000","last_undegraded":"2024-09-16T09:40:40.453511+0000","last_fullsized":"2024-09-16T09:40:40.453511+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:40:31.037258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:40:31.037258+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:51.866286+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"235'12","reported_seq":554,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:41.939009+0000","last_change":"2024-09-16T09:37:19.380567+0000","last_active":"2024-09-16T09:40:41.939009+0000","last_peered":"2024-09-16T09:40:41.939009+0000","last_clean":"2024-09-16T09:40:41.939009+0000","last_became_active":"2024-09-16T09:33:30.251047+0000","last_became_peered":"2024-09-16T09:33:30.251047+0000","last_unstale":"2024-09-16T09:40:41.939009+0000","last_undegraded":"2024-09-16T09:40:41.939009+0000","last_fullsized":"2024-09-16T09:40:41.939009+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"235'12","last_scrub_stamp":"2024-09-16T09:37:19.380504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:37:19.380504+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":803,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.447113+0000","last_change":"2024-09-16T09:40:30.956991+0000","last_active":"2024-09-16T09:40:40.447113+0000","last_peered":"2024-09-16T09:40:40.447113+0000","last_clean":"2024-09-16T09:40:40.447113+0000","last_became_active":"2024-09-16T09:25:07.722017+0000","last_became_peered":"2024-09-16T09:25:07.722017+0000","last_unstale":"2024-09-16T09:40:40.447113+0000","last_undegraded":"2024-09-16T09:40:40.447113+0000","last_fullsized":"2024-09-16T09:40:40.447113+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:40:30.956926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:40:30.956926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:31.490124+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.c","version":"442'12997","reported_seq":13143,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:47.439561+0000","last_change":"2024-09-16T09:37:45.401117+0000","last_active":"2024-09-16T09:40:47.439561+0000","last_peered":"2024-09-16T09:40:47.439561+0000","last_clean":"2024-09-16T09:40:47.439561+0000","last_became_active":"2024-09-16T09:37:45.400778+0000","last_became_peered":"2024-09-16T09:37:45.400778+0000","last_unstale":"2024-09-16T09:40:47.439561+0000","last_undegraded":"2024-09-16T09:40:47.439561+0000","last_fullsized":"2024-09-16T09:40:47.439561+0000","mapping_epoch":376,"log_start":"425'8621","ondisk_log_start":"425'8621","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_clean_scrub_stamp":"2024-09-16T09:37:44.378497+0000","objects_scrubbed":0,"log_size":4376,"log_dups_size":0,"ondisk_log_size":4376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:18.979175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":86835200,"num_objects":1325,"num_object_clones":0,"num_object_copies":3975,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1325,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12997,"num_write_kb":458304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"73'79","reported_seq":873,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.452745+0000","last_change":"2024-09-16T09:38:24.036642+0000","last_active":"2024-09-16T09:40:40.452745+0000","last_peered":"2024-09-16T09:40:40.452745+0000","last_clean":"2024-09-16T09:40:40.452745+0000","last_became_active":"2024-09-16T09:31:09.274707+0000","last_became_peered":"2024-09-16T09:31:09.274707+0000","last_unstale":"2024-09-16T09:40:40.452745+0000","last_undegraded":"2024-09-16T09:40:40.452745+0000","last_fullsized":"2024-09-16T09:40:40.452745+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:38:24.036564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:38:24.036564+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"285'20","reported_seq":498,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.451019+0000","last_change":"2024-09-16T09:38:32.122419+0000","last_active":"2024-09-16T09:40:40.451019+0000","last_peered":"2024-09-16T09:40:40.451019+0000","last_clean":"2024-09-16T09:40:40.451019+0000","last_became_active":"2024-09-16T09:33:30.248453+0000","last_became_peered":"2024-09-16T09:33:30.248453+0000","last_unstale":"2024-09-16T09:40:40.451019+0000","last_undegraded":"2024-09-16T09:40:40.451019+0000","last_fullsized":"2024-09-16T09:40:40.451019+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'20","last_scrub_stamp":"2024-09-16T09:38:32.122303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:38:32.122303+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":822,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.450716+0000","last_change":"2024-09-16T09:36:47.095703+0000","last_active":"2024-09-16T09:40:40.450716+0000","last_peered":"2024-09-16T09:40:40.450716+0000","last_clean":"2024-09-16T09:40:40.450716+0000","last_became_active":"2024-09-16T09:31:09.270619+0000","last_became_peered":"2024-09-16T09:31:09.270619+0000","last_unstale":"2024-09-16T09:40:40.450716+0000","last_undegraded":"2024-09-16T09:40:40.450716+0000","last_fullsized":"2024-09-16T09:40:40.450716+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:47.095658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:36:47.095658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"66'8","reported_seq":999,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:41.939553+0000","last_change":"2024-09-16T09:36:52.689167+0000","last_active":"2024-09-16T09:40:41.939553+0000","last_peered":"2024-09-16T09:40:41.939553+0000","last_clean":"2024-09-16T09:40:41.939553+0000","last_became_active":"2024-09-16T09:25:05.714507+0000","last_became_peered":"2024-09-16T09:25:05.714507+0000","last_unstale":"2024-09-16T09:40:41.939553+0000","last_undegraded":"2024-09-16T09:40:41.939553+0000","last_fullsized":"2024-09-16T09:40:41.939553+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:36:52.689091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:36:52.689091+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:02.336892+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":819,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.455400+0000","last_change":"2024-09-16T09:39:36.520294+0000","last_active":"2024-09-16T09:40:40.455400+0000","last_peered":"2024-09-16T09:40:40.455400+0000","last_clean":"2024-09-16T09:40:40.455400+0000","last_became_active":"2024-09-16T09:31:09.283192+0000","last_became_peered":"2024-09-16T09:31:09.283192+0000","last_unstale":"2024-09-16T09:40:40.455400+0000","last_undegraded":"2024-09-16T09:40:40.455400+0000","last_fullsized":"2024-09-16T09:40:40.455400+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:39:36.520205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:39:36.520205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:40:50.155402+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.d","version":"442'12965","reported_seq":13113,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:47.916248+0000","last_change":"2024-09-16T09:37:45.405475+0000","last_active":"2024-09-16T09:40:47.916248+0000","last_peered":"2024-09-16T09:40:47.916248+0000","last_clean":"2024-09-16T09:40:47.916248+0000","last_became_active":"2024-09-16T09:37:45.405149+0000","last_became_peered":"2024-09-16T09:37:45.405149+0000","last_unstale":"2024-09-16T09:40:47.916248+0000","last_undegraded":"2024-09-16T09:40:47.916248+0000","last_fullsized":"2024-09-16T09:40:47.916248+0000","mapping_epoch":376,"log_start":"423'8200","ondisk_log_start":"423'8200","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_clean_scrub_stamp":"2024-09-16T09:37:44.378497+0000","objects_scrubbed":0,"log_size":4765,"log_dups_size":0,"ondisk_log_size":4765,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":79036440,"num_objects":1207,"num_object_clones":0,"num_object_copies":3621,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1207,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12965,"num_write_kb":453441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"73'79","reported_seq":838,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.446748+0000","last_change":"2024-09-16T09:34:42.728232+0000","last_active":"2024-09-16T09:40:40.446748+0000","last_peered":"2024-09-16T09:40:40.446748+0000","last_clean":"2024-09-16T09:40:40.446748+0000","last_became_active":"2024-09-16T09:31:09.274565+0000","last_became_peered":"2024-09-16T09:31:09.274565+0000","last_unstale":"2024-09-16T09:40:40.446748+0000","last_undegraded":"2024-09-16T09:40:40.446748+0000","last_fullsized":"2024-09-16T09:40:40.446748+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:34:42.728143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:34:42.728143+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:16.765504+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"285'29","reported_seq":856,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.453016+0000","last_change":"2024-09-16T09:36:54.604573+0000","last_active":"2024-09-16T09:40:40.453016+0000","last_peered":"2024-09-16T09:40:40.453016+0000","last_clean":"2024-09-16T09:40:40.453016+0000","last_became_active":"2024-09-16T09:25:03.714797+0000","last_became_peered":"2024-09-16T09:25:03.714797+0000","last_unstale":"2024-09-16T09:40:40.453016+0000","last_undegraded":"2024-09-16T09:40:40.453016+0000","last_fullsized":"2024-09-16T09:40:40.453016+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'29","last_scrub_stamp":"2024-09-16T09:36:54.604484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:36:54.604484+0000","objects_scrubbed":8,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:10.233603+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":54,"num_read_kb":40,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":440,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.451512+0000","last_change":"2024-09-16T09:40:26.273165+0000","last_active":"2024-09-16T09:40:40.451512+0000","last_peered":"2024-09-16T09:40:40.451512+0000","last_clean":"2024-09-16T09:40:40.451512+0000","last_became_active":"2024-09-16T09:33:30.246996+0000","last_became_peered":"2024-09-16T09:33:30.246996+0000","last_unstale":"2024-09-16T09:40:40.451512+0000","last_undegraded":"2024-09-16T09:40:40.451512+0000","last_fullsized":"2024-09-16T09:40:40.451512+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:40:26.273106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:40:26.273106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:26.666617+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,2],"acting":[5,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":825,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.445916+0000","last_change":"2024-09-16T09:40:34.943633+0000","last_active":"2024-09-16T09:40:40.445916+0000","last_peered":"2024-09-16T09:40:40.445916+0000","last_clean":"2024-09-16T09:40:40.445916+0000","last_became_active":"2024-09-16T09:25:05.708663+0000","last_became_peered":"2024-09-16T09:25:05.708663+0000","last_unstale":"2024-09-16T09:40:40.445916+0000","last_undegraded":"2024-09-16T09:40:40.445916+0000","last_fullsized":"2024-09-16T09:40:40.445916+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:40:34.943575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:40:34.943575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:43.181020+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":448,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.445859+0000","last_change":"2024-09-16T09:37:56.882475+0000","last_active":"2024-09-16T09:40:40.445859+0000","last_peered":"2024-09-16T09:40:40.445859+0000","last_clean":"2024-09-16T09:40:40.445859+0000","last_became_active":"2024-09-16T09:33:30.250178+0000","last_became_peered":"2024-09-16T09:33:30.250178+0000","last_unstale":"2024-09-16T09:40:40.445859+0000","last_undegraded":"2024-09-16T09:40:40.445859+0000","last_fullsized":"2024-09-16T09:40:40.445859+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:56.882326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:37:56.882326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,2],"acting":[6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.a","version":"442'13210","reported_seq":13357,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:47.918351+0000","last_change":"2024-09-16T09:37:45.404280+0000","last_active":"2024-09-16T09:40:47.918351+0000","last_peered":"2024-09-16T09:40:47.918351+0000","last_clean":"2024-09-16T09:40:47.918351+0000","last_became_active":"2024-09-16T09:37:45.404021+0000","last_became_peered":"2024-09-16T09:37:45.404021+0000","last_unstale":"2024-09-16T09:40:47.918351+0000","last_undegraded":"2024-09-16T09:40:47.918351+0000","last_fullsized":"2024-09-16T09:40:47.918351+0000","mapping_epoch":376,"log_start":"423'8400","ondisk_log_start":"423'8400","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_clean_scrub_stamp":"2024-09-16T09:37:44.378497+0000","objects_scrubbed":0,"log_size":4810,"log_dups_size":0,"ondisk_log_size":4810,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":78512128,"num_objects":1198,"num_object_clones":0,"num_object_copies":3594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1198,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13210,"num_write_kb":461056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"73'79","reported_seq":771,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.450234+0000","last_change":"2024-09-16T09:36:42.191578+0000","last_active":"2024-09-16T09:40:40.450234+0000","last_peered":"2024-09-16T09:40:40.450234+0000","last_clean":"2024-09-16T09:40:40.450234+0000","last_became_active":"2024-09-16T09:30:08.766350+0000","last_became_peered":"2024-09-16T09:30:08.766350+0000","last_unstale":"2024-09-16T09:40:40.450234+0000","last_undegraded":"2024-09-16T09:40:40.450234+0000","last_fullsized":"2024-09-16T09:40:40.450234+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:36:42.191521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:36:42.191521+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"285'42","reported_seq":1142,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:41.938973+0000","last_change":"2024-09-16T09:37:57.407386+0000","last_active":"2024-09-16T09:40:41.938973+0000","last_peered":"2024-09-16T09:40:41.938973+0000","last_clean":"2024-09-16T09:40:41.938973+0000","last_became_active":"2024-09-16T09:31:09.278839+0000","last_became_peered":"2024-09-16T09:31:09.278839+0000","last_unstale":"2024-09-16T09:40:41.938973+0000","last_undegraded":"2024-09-16T09:40:41.938973+0000","last_fullsized":"2024-09-16T09:40:41.938973+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'42","last_scrub_stamp":"2024-09-16T09:37:57.407297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:37:57.407297+0000","objects_scrubbed":10,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:20.634644+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":66,"num_read_kb":51,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":825,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.443505+0000","last_change":"2024-09-16T09:38:30.156162+0000","last_active":"2024-09-16T09:40:40.443505+0000","last_peered":"2024-09-16T09:40:40.443505+0000","last_clean":"2024-09-16T09:40:40.443505+0000","last_became_active":"2024-09-16T09:25:01.703076+0000","last_became_peered":"2024-09-16T09:25:01.703076+0000","last_unstale":"2024-09-16T09:40:40.443505+0000","last_undegraded":"2024-09-16T09:40:40.443505+0000","last_fullsized":"2024-09-16T09:40:40.443505+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:38:30.156100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:38:30.156100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":826,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.451233+0000","last_change":"2024-09-16T09:38:27.174725+0000","last_active":"2024-09-16T09:40:40.451233+0000","last_peered":"2024-09-16T09:40:40.451233+0000","last_clean":"2024-09-16T09:40:40.451233+0000","last_became_active":"2024-09-16T09:25:00.696237+0000","last_became_peered":"2024-09-16T09:25:00.696237+0000","last_unstale":"2024-09-16T09:40:40.451233+0000","last_undegraded":"2024-09-16T09:40:40.451233+0000","last_fullsized":"2024-09-16T09:40:40.451233+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-16T09:38:27.174657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:38:27.174657+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.0025137430000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":793,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.454308+0000","last_change":"2024-09-16T09:39:39.498275+0000","last_active":"2024-09-16T09:40:40.454308+0000","last_peered":"2024-09-16T09:40:40.454308+0000","last_clean":"2024-09-16T09:40:40.454308+0000","last_became_active":"2024-09-16T09:25:05.717591+0000","last_became_peered":"2024-09-16T09:25:05.717591+0000","last_unstale":"2024-09-16T09:40:40.454308+0000","last_undegraded":"2024-09-16T09:40:40.454308+0000","last_fullsized":"2024-09-16T09:40:40.454308+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:39:39.498205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:39:39.498205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:40:48.192278+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":542,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.449137+0000","last_change":"2024-09-16T09:38:25.992634+0000","last_active":"2024-09-16T09:40:40.449137+0000","last_peered":"2024-09-16T09:40:40.449137+0000","last_clean":"2024-09-16T09:40:40.449137+0000","last_became_active":"2024-09-16T09:31:09.272609+0000","last_became_peered":"2024-09-16T09:31:09.272609+0000","last_unstale":"2024-09-16T09:40:40.449137+0000","last_undegraded":"2024-09-16T09:40:40.449137+0000","last_fullsized":"2024-09-16T09:40:40.449137+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:38:25.992569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:38:25.992569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"442'8844","reported_seq":9032,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:48.230259+0000","last_change":"2024-09-16T09:38:37.442061+0000","last_active":"2024-09-16T09:40:48.230259+0000","last_peered":"2024-09-16T09:40:48.230259+0000","last_clean":"2024-09-16T09:40:48.230259+0000","last_became_active":"2024-09-16T09:38:35.372587+0000","last_became_peered":"2024-09-16T09:38:35.372587+0000","last_unstale":"2024-09-16T09:40:48.230259+0000","last_undegraded":"2024-09-16T09:40:48.230259+0000","last_fullsized":"2024-09-16T09:40:48.230259+0000","mapping_epoch":376,"log_start":"391'4197","ondisk_log_start":"391'4197","created":376,"last_epoch_clean":396,"parent":"0.0","parent_split_bits":0,"last_scrub":"397'4558","last_scrub_stamp":"2024-09-16T09:38:37.441970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_clean_scrub_stamp":"2024-09-16T09:38:37.441970+0000","objects_scrubbed":2306,"log_size":4647,"log_dups_size":0,"ondisk_log_size":4647,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":280,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40108032,"num_objects":612,"num_object_clones":0,"num_object_copies":1836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":612,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4286,"num_write_kb":82944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"73'79","reported_seq":139,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.446957+0000","last_change":"2024-09-16T09:40:26.918606+0000","last_active":"2024-09-16T09:40:40.446957+0000","last_peered":"2024-09-16T09:40:40.446957+0000","last_clean":"2024-09-16T09:40:40.446957+0000","last_became_active":"2024-09-16T09:38:10.556320+0000","last_became_peered":"2024-09-16T09:38:10.556320+0000","last_unstale":"2024-09-16T09:40:40.446957+0000","last_undegraded":"2024-09-16T09:40:40.446957+0000","last_fullsized":"2024-09-16T09:40:40.446957+0000","mapping_epoch":386,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":387,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:40:26.918532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:40:26.918532+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:44.861316+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"285'17","reported_seq":531,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.451108+0000","last_change":"2024-09-16T09:37:49.246464+0000","last_active":"2024-09-16T09:40:40.451108+0000","last_peered":"2024-09-16T09:40:40.451108+0000","last_clean":"2024-09-16T09:40:40.451108+0000","last_became_active":"2024-09-16T09:37:33.349009+0000","last_became_peered":"2024-09-16T09:37:33.349009+0000","last_unstale":"2024-09-16T09:40:40.451108+0000","last_undegraded":"2024-09-16T09:40:40.451108+0000","last_fullsized":"2024-09-16T09:40:40.451108+0000","mapping_epoch":367,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":368,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'17","last_scrub_stamp":"2024-09-16T09:37:49.246379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:37:49.246379+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":35,"num_read_kb":24,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":440,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.445850+0000","last_change":"2024-09-16T09:37:21.946685+0000","last_active":"2024-09-16T09:40:40.445850+0000","last_peered":"2024-09-16T09:40:40.445850+0000","last_clean":"2024-09-16T09:40:40.445850+0000","last_became_active":"2024-09-16T09:33:30.247596+0000","last_became_peered":"2024-09-16T09:33:30.247596+0000","last_unstale":"2024-09-16T09:40:40.445850+0000","last_undegraded":"2024-09-16T09:40:40.445850+0000","last_fullsized":"2024-09-16T09:40:40.445850+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:21.946641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:37:21.946641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,2],"acting":[6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":475,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.456081+0000","last_change":"2024-09-16T09:38:23.416822+0000","last_active":"2024-09-16T09:40:40.456081+0000","last_peered":"2024-09-16T09:40:40.456081+0000","last_clean":"2024-09-16T09:40:40.456081+0000","last_became_active":"2024-09-16T09:37:04.358985+0000","last_became_peered":"2024-09-16T09:37:04.358985+0000","last_unstale":"2024-09-16T09:40:40.456081+0000","last_undegraded":"2024-09-16T09:40:40.456081+0000","last_fullsized":"2024-09-16T09:40:40.456081+0000","mapping_epoch":343,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":344,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:38:23.416756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:38:23.416756+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":833,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.450826+0000","last_change":"2024-09-16T09:40:31.546340+0000","last_active":"2024-09-16T09:40:40.450826+0000","last_peered":"2024-09-16T09:40:40.450826+0000","last_clean":"2024-09-16T09:40:40.450826+0000","last_became_active":"2024-09-16T09:25:05.711539+0000","last_became_peered":"2024-09-16T09:25:05.711539+0000","last_unstale":"2024-09-16T09:40:40.450826+0000","last_undegraded":"2024-09-16T09:40:40.450826+0000","last_fullsized":"2024-09-16T09:40:40.450826+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:40:31.546195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:40:31.546195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:45.073777+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":803,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.448363+0000","last_change":"2024-09-16T09:37:45.450074+0000","last_active":"2024-09-16T09:40:40.448363+0000","last_peered":"2024-09-16T09:40:40.448363+0000","last_clean":"2024-09-16T09:40:40.448363+0000","last_became_active":"2024-09-16T09:25:07.723282+0000","last_became_peered":"2024-09-16T09:25:07.723282+0000","last_unstale":"2024-09-16T09:40:40.448363+0000","last_undegraded":"2024-09-16T09:40:40.448363+0000","last_fullsized":"2024-09-16T09:40:40.448363+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:45.450022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:37:45.450022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:20.916779+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"442'8948","reported_seq":9133,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:49.431229+0000","last_change":"2024-09-16T09:38:35.885143+0000","last_active":"2024-09-16T09:40:49.431229+0000","last_peered":"2024-09-16T09:40:49.431229+0000","last_clean":"2024-09-16T09:40:49.431229+0000","last_became_active":"2024-09-16T09:38:35.364673+0000","last_became_peered":"2024-09-16T09:38:35.364673+0000","last_unstale":"2024-09-16T09:40:49.431229+0000","last_undegraded":"2024-09-16T09:40:49.431229+0000","last_fullsized":"2024-09-16T09:40:49.431229+0000","mapping_epoch":376,"log_start":"392'4372","ondisk_log_start":"392'4372","created":376,"last_epoch_clean":396,"parent":"0.0","parent_split_bits":0,"last_scrub":"396'4570","last_scrub_stamp":"2024-09-16T09:38:35.884998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_clean_scrub_stamp":"2024-09-16T09:38:35.884998+0000","objects_scrubbed":2313,"log_size":4576,"log_dups_size":0,"ondisk_log_size":4576,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:40.869899+0000","scrub_duration":388,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37027840,"num_objects":565,"num_object_clones":0,"num_object_copies":1695,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":565,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4392,"num_write_kb":84608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,7],"acting":[0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"73'79","reported_seq":521,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.448339+0000","last_change":"2024-09-16T09:34:31.929462+0000","last_active":"2024-09-16T09:40:40.448339+0000","last_peered":"2024-09-16T09:40:40.448339+0000","last_clean":"2024-09-16T09:40:40.448339+0000","last_became_active":"2024-09-16T09:33:30.249309+0000","last_became_peered":"2024-09-16T09:33:30.249309+0000","last_unstale":"2024-09-16T09:40:40.448339+0000","last_undegraded":"2024-09-16T09:40:40.448339+0000","last_fullsized":"2024-09-16T09:40:40.448339+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:34:31.929388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:34:31.929388+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:15.809663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.6","version":"285'24","reported_seq":849,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.453350+0000","last_change":"2024-09-16T09:40:29.074141+0000","last_active":"2024-09-16T09:40:40.453350+0000","last_peered":"2024-09-16T09:40:40.453350+0000","last_clean":"2024-09-16T09:40:40.453350+0000","last_became_active":"2024-09-16T09:25:03.715315+0000","last_became_peered":"2024-09-16T09:25:03.715315+0000","last_unstale":"2024-09-16T09:40:40.453350+0000","last_undegraded":"2024-09-16T09:40:40.453350+0000","last_fullsized":"2024-09-16T09:40:40.453350+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'24","last_scrub_stamp":"2024-09-16T09:40:29.074075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:40:29.074075+0000","objects_scrubbed":6,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:30.097248+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":36,"num_read_kb":30,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"59'1","reported_seq":840,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.455738+0000","last_change":"2024-09-16T09:39:41.560703+0000","last_active":"2024-09-16T09:40:40.455738+0000","last_peered":"2024-09-16T09:40:40.455738+0000","last_clean":"2024-09-16T09:40:40.455738+0000","last_became_active":"2024-09-16T09:31:09.269627+0000","last_became_peered":"2024-09-16T09:31:09.269627+0000","last_unstale":"2024-09-16T09:40:40.455738+0000","last_undegraded":"2024-09-16T09:40:40.455738+0000","last_fullsized":"2024-09-16T09:40:40.455738+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:39:41.560654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:39:41.560654+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:04.368665+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1986,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"67'5","reported_seq":1765,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:46.290252+0000","last_change":"2024-09-16T09:38:35.633711+0000","last_active":"2024-09-16T09:40:46.290252+0000","last_peered":"2024-09-16T09:40:46.290252+0000","last_clean":"2024-09-16T09:40:46.290252+0000","last_became_active":"2024-09-16T09:25:00.696546+0000","last_became_peered":"2024-09-16T09:25:00.696546+0000","last_unstale":"2024-09-16T09:40:46.290252+0000","last_undegraded":"2024-09-16T09:40:46.290252+0000","last_fullsized":"2024-09-16T09:40:46.290252+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'5","last_scrub_stamp":"2024-09-16T09:38:35.633534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:38:35.633534+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:43.105236+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0.0059770420000000001,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":939,"num_read_kb":934,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":784,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.451201+0000","last_change":"2024-09-16T09:27:07.337170+0000","last_active":"2024-09-16T09:40:40.451201+0000","last_peered":"2024-09-16T09:40:40.451201+0000","last_clean":"2024-09-16T09:40:40.451201+0000","last_became_active":"2024-09-16T09:25:05.714704+0000","last_became_peered":"2024-09-16T09:25:05.714704+0000","last_unstale":"2024-09-16T09:40:40.451201+0000","last_undegraded":"2024-09-16T09:40:40.451201+0000","last_fullsized":"2024-09-16T09:40:40.451201+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:07.337124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:07.337124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:17.260306+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":718,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.451724+0000","last_change":"2024-09-16T09:40:33.229294+0000","last_active":"2024-09-16T09:40:40.451724+0000","last_peered":"2024-09-16T09:40:40.451724+0000","last_clean":"2024-09-16T09:40:40.451724+0000","last_became_active":"2024-09-16T09:33:30.245852+0000","last_became_peered":"2024-09-16T09:33:30.245852+0000","last_unstale":"2024-09-16T09:40:40.451724+0000","last_undegraded":"2024-09-16T09:40:40.451724+0000","last_fullsized":"2024-09-16T09:40:40.451724+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:40:33.229250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:40:33.229250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:02.848996+0000","scrub_duration":97,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"442'13016","reported_seq":13163,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:47.914993+0000","last_change":"2024-09-16T09:37:45.406191+0000","last_active":"2024-09-16T09:40:47.914993+0000","last_peered":"2024-09-16T09:40:47.914993+0000","last_clean":"2024-09-16T09:40:47.914993+0000","last_became_active":"2024-09-16T09:37:45.405552+0000","last_became_peered":"2024-09-16T09:37:45.405552+0000","last_unstale":"2024-09-16T09:40:47.914993+0000","last_undegraded":"2024-09-16T09:40:47.914993+0000","last_fullsized":"2024-09-16T09:40:47.914993+0000","mapping_epoch":376,"log_start":"423'8200","ondisk_log_start":"423'8200","created":376,"last_epoch_clean":377,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_clean_scrub_stamp":"2024-09-16T09:37:44.378497+0000","objects_scrubbed":0,"log_size":4816,"log_dups_size":0,"ondisk_log_size":4816,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":78512128,"num_objects":1198,"num_object_clones":0,"num_object_copies":3594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1198,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13016,"num_write_kb":454848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"285'28","reported_seq":1043,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:41.942093+0000","last_change":"2024-09-16T09:33:16.891850+0000","last_active":"2024-09-16T09:40:41.942093+0000","last_peered":"2024-09-16T09:40:41.942093+0000","last_clean":"2024-09-16T09:40:41.942093+0000","last_became_active":"2024-09-16T09:25:03.714118+0000","last_became_peered":"2024-09-16T09:25:03.714118+0000","last_unstale":"2024-09-16T09:40:41.942093+0000","last_undegraded":"2024-09-16T09:40:41.942093+0000","last_fullsized":"2024-09-16T09:40:41.942093+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'16","last_scrub_stamp":"2024-09-16T09:33:16.891761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:33:16.891761+0000","objects_scrubbed":8,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:13.813623+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":43,"num_read_kb":33,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":658,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.451100+0000","last_change":"2024-09-16T09:36:50.114910+0000","last_active":"2024-09-16T09:40:40.451100+0000","last_peered":"2024-09-16T09:40:40.451100+0000","last_clean":"2024-09-16T09:40:40.451100+0000","last_became_active":"2024-09-16T09:31:09.270196+0000","last_became_peered":"2024-09-16T09:31:09.270196+0000","last_unstale":"2024-09-16T09:40:40.451100+0000","last_undegraded":"2024-09-16T09:40:40.451100+0000","last_fullsized":"2024-09-16T09:40:40.451100+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:50.114861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:36:50.114861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"73'79","reported_seq":895,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.448087+0000","last_change":"2024-09-16T09:39:39.878540+0000","last_active":"2024-09-16T09:40:40.448087+0000","last_peered":"2024-09-16T09:40:40.448087+0000","last_clean":"2024-09-16T09:40:40.448087+0000","last_became_active":"2024-09-16T09:30:08.870394+0000","last_became_peered":"2024-09-16T09:30:08.870394+0000","last_unstale":"2024-09-16T09:40:40.448087+0000","last_undegraded":"2024-09-16T09:40:40.448087+0000","last_fullsized":"2024-09-16T09:40:40.448087+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:39:39.878410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:39:39.878410+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:40:43.588453+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":807,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.448800+0000","last_change":"2024-09-16T09:40:22.941843+0000","last_active":"2024-09-16T09:40:40.448800+0000","last_peered":"2024-09-16T09:40:40.448800+0000","last_clean":"2024-09-16T09:40:40.448800+0000","last_became_active":"2024-09-16T09:25:05.714341+0000","last_became_peered":"2024-09-16T09:25:05.714341+0000","last_unstale":"2024-09-16T09:40:40.448800+0000","last_undegraded":"2024-09-16T09:40:40.448800+0000","last_fullsized":"2024-09-16T09:40:40.448800+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:40:22.941427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:40:22.941427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:49.076571+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":796,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.450963+0000","last_change":"2024-09-16T09:37:50.218655+0000","last_active":"2024-09-16T09:40:40.450963+0000","last_peered":"2024-09-16T09:40:40.450963+0000","last_clean":"2024-09-16T09:40:40.450963+0000","last_became_active":"2024-09-16T09:25:07.714281+0000","last_became_peered":"2024-09-16T09:25:07.714281+0000","last_unstale":"2024-09-16T09:40:40.450963+0000","last_undegraded":"2024-09-16T09:40:40.450963+0000","last_fullsized":"2024-09-16T09:40:40.450963+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:50.218594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:37:50.218594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.0","version":"442'8541","reported_seq":8725,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:47.846320+0000","last_change":"2024-09-16T09:38:37.410089+0000","last_active":"2024-09-16T09:40:47.846320+0000","last_peered":"2024-09-16T09:40:47.846320+0000","last_clean":"2024-09-16T09:40:47.846320+0000","last_became_active":"2024-09-16T09:38:35.375417+0000","last_became_peered":"2024-09-16T09:38:35.375417+0000","last_unstale":"2024-09-16T09:40:47.846320+0000","last_undegraded":"2024-09-16T09:40:47.846320+0000","last_fullsized":"2024-09-16T09:40:47.846320+0000","mapping_epoch":376,"log_start":"390'3174","ondisk_log_start":"390'3174","created":376,"last_epoch_clean":396,"parent":"0.0","parent_split_bits":0,"last_scrub":"397'4446","last_scrub_stamp":"2024-09-16T09:38:37.409993+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_clean_scrub_stamp":"2024-09-16T09:38:37.409993+0000","objects_scrubbed":2245,"log_size":5367,"log_dups_size":0,"ondisk_log_size":5367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":298,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40894464,"num_objects":624,"num_object_clones":0,"num_object_copies":1872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":624,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4095,"num_write_kb":79168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.b","version":"73'79","reported_seq":640,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.452402+0000","last_change":"2024-09-16T09:40:37.917868+0000","last_active":"2024-09-16T09:40:40.452402+0000","last_peered":"2024-09-16T09:40:40.452402+0000","last_clean":"2024-09-16T09:40:40.452402+0000","last_became_active":"2024-09-16T09:33:30.251933+0000","last_became_peered":"2024-09-16T09:33:30.251933+0000","last_unstale":"2024-09-16T09:40:40.452402+0000","last_undegraded":"2024-09-16T09:40:40.452402+0000","last_fullsized":"2024-09-16T09:40:40.452402+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:40:37.917691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:40:37.917691+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:38.348265+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"285'19","reported_seq":885,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.443616+0000","last_change":"2024-09-16T09:37:33.355562+0000","last_active":"2024-09-16T09:40:40.443616+0000","last_peered":"2024-09-16T09:40:40.443616+0000","last_clean":"2024-09-16T09:40:40.443616+0000","last_became_active":"2024-09-16T09:37:33.348433+0000","last_became_peered":"2024-09-16T09:37:33.348433+0000","last_unstale":"2024-09-16T09:40:40.443616+0000","last_undegraded":"2024-09-16T09:40:40.443616+0000","last_fullsized":"2024-09-16T09:40:40.443616+0000","mapping_epoch":367,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":368,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'19","last_scrub_stamp":"2024-09-16T09:35:27.813206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:35:27.813206+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":801,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.442949+0000","last_change":"2024-09-16T09:37:18.392256+0000","last_active":"2024-09-16T09:40:40.442949+0000","last_peered":"2024-09-16T09:40:40.442949+0000","last_clean":"2024-09-16T09:40:40.442949+0000","last_became_active":"2024-09-16T09:25:01.708270+0000","last_became_peered":"2024-09-16T09:25:01.708270+0000","last_unstale":"2024-09-16T09:40:40.442949+0000","last_undegraded":"2024-09-16T09:40:40.442949+0000","last_fullsized":"2024-09-16T09:40:40.442949+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:18.392084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:37:18.392084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":783,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.450809+0000","last_change":"2024-09-16T09:26:34.171190+0000","last_active":"2024-09-16T09:40:40.450809+0000","last_peered":"2024-09-16T09:40:40.450809+0000","last_clean":"2024-09-16T09:40:40.450809+0000","last_became_active":"2024-09-16T09:25:05.711964+0000","last_became_peered":"2024-09-16T09:25:05.711964+0000","last_unstale":"2024-09-16T09:40:40.450809+0000","last_undegraded":"2024-09-16T09:40:40.450809+0000","last_fullsized":"2024-09-16T09:40:40.450809+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:34.171139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:26:34.171139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:14.345408+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":683,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.446242+0000","last_change":"2024-09-16T09:40:31.948601+0000","last_active":"2024-09-16T09:40:40.446242+0000","last_peered":"2024-09-16T09:40:40.446242+0000","last_clean":"2024-09-16T09:40:40.446242+0000","last_became_active":"2024-09-16T09:31:09.278618+0000","last_became_peered":"2024-09-16T09:31:09.278618+0000","last_unstale":"2024-09-16T09:40:40.446242+0000","last_undegraded":"2024-09-16T09:40:40.446242+0000","last_fullsized":"2024-09-16T09:40:40.446242+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:40:31.948536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:40:31.948536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:01.350977+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3","version":"442'8974","reported_seq":9159,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:48.271802+0000","last_change":"2024-09-16T09:38:36.668960+0000","last_active":"2024-09-16T09:40:48.271802+0000","last_peered":"2024-09-16T09:40:48.271802+0000","last_clean":"2024-09-16T09:40:48.271802+0000","last_became_active":"2024-09-16T09:38:35.365258+0000","last_became_peered":"2024-09-16T09:38:35.365258+0000","last_unstale":"2024-09-16T09:40:48.271802+0000","last_undegraded":"2024-09-16T09:40:48.271802+0000","last_fullsized":"2024-09-16T09:40:48.271802+0000","mapping_epoch":376,"log_start":"392'4374","ondisk_log_start":"392'4374","created":376,"last_epoch_clean":396,"parent":"0.0","parent_split_bits":0,"last_scrub":"397'4682","last_scrub_stamp":"2024-09-16T09:38:36.668842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_clean_scrub_stamp":"2024-09-16T09:38:36.668842+0000","objects_scrubbed":2403,"log_size":4600,"log_dups_size":0,"ondisk_log_size":4600,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":551,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38797312,"num_objects":592,"num_object_clones":0,"num_object_copies":1776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":592,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4301,"num_write_kb":79680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.8","version":"73'79","reported_seq":143,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.449806+0000","last_change":"2024-09-16T09:37:54.183161+0000","last_active":"2024-09-16T09:40:40.449806+0000","last_peered":"2024-09-16T09:40:40.449806+0000","last_clean":"2024-09-16T09:40:40.449806+0000","last_became_active":"2024-09-16T09:37:53.418691+0000","last_became_peered":"2024-09-16T09:37:53.418691+0000","last_unstale":"2024-09-16T09:40:40.449806+0000","last_undegraded":"2024-09-16T09:40:40.449806+0000","last_fullsized":"2024-09-16T09:40:40.449806+0000","mapping_epoch":382,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":383,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:37:54.183107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:37:54.183107+0000","objects_scrubbed":1,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262688,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"285'31","reported_seq":862,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.443032+0000","last_change":"2024-09-16T09:35:28.861520+0000","last_active":"2024-09-16T09:40:40.443032+0000","last_peered":"2024-09-16T09:40:40.443032+0000","last_clean":"2024-09-16T09:40:40.443032+0000","last_became_active":"2024-09-16T09:25:03.713358+0000","last_became_peered":"2024-09-16T09:25:03.713358+0000","last_unstale":"2024-09-16T09:40:40.443032+0000","last_undegraded":"2024-09-16T09:40:40.443032+0000","last_fullsized":"2024-09-16T09:40:40.443032+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'31","last_scrub_stamp":"2024-09-16T09:35:28.861433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:35:28.861433+0000","objects_scrubbed":8,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":50,"num_read_kb":40,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":809,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.446094+0000","last_change":"2024-09-16T09:37:52.902619+0000","last_active":"2024-09-16T09:40:40.446094+0000","last_peered":"2024-09-16T09:40:40.446094+0000","last_clean":"2024-09-16T09:40:40.446094+0000","last_became_active":"2024-09-16T09:25:01.703355+0000","last_became_peered":"2024-09-16T09:25:01.703355+0000","last_unstale":"2024-09-16T09:40:40.446094+0000","last_undegraded":"2024-09-16T09:40:40.446094+0000","last_fullsized":"2024-09-16T09:40:40.446094+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:52.902556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:37:52.902556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":448,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.443214+0000","last_change":"2024-09-16T09:37:46.378016+0000","last_active":"2024-09-16T09:40:40.443214+0000","last_peered":"2024-09-16T09:40:40.443214+0000","last_clean":"2024-09-16T09:40:40.443214+0000","last_became_active":"2024-09-16T09:33:30.251186+0000","last_became_peered":"2024-09-16T09:33:30.251186+0000","last_unstale":"2024-09-16T09:40:40.443214+0000","last_undegraded":"2024-09-16T09:40:40.443214+0000","last_fullsized":"2024-09-16T09:40:40.443214+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:46.377951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:37:46.377951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,0],"acting":[4,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":746,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.457721+0000","last_change":"2024-09-16T09:38:24.031758+0000","last_active":"2024-09-16T09:40:40.457721+0000","last_peered":"2024-09-16T09:40:40.457721+0000","last_clean":"2024-09-16T09:40:40.457721+0000","last_became_active":"2024-09-16T09:33:21.337291+0000","last_became_peered":"2024-09-16T09:33:21.337291+0000","last_unstale":"2024-09-16T09:40:40.457721+0000","last_undegraded":"2024-09-16T09:40:40.457721+0000","last_fullsized":"2024-09-16T09:40:40.457721+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:38:24.031592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:38:24.031592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4],"acting":[3,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"442'8838","reported_seq":9022,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:48.070866+0000","last_change":"2024-09-16T09:38:36.700802+0000","last_active":"2024-09-16T09:40:48.070866+0000","last_peered":"2024-09-16T09:40:48.070866+0000","last_clean":"2024-09-16T09:40:48.070866+0000","last_became_active":"2024-09-16T09:38:35.380640+0000","last_became_peered":"2024-09-16T09:38:35.380640+0000","last_unstale":"2024-09-16T09:40:48.070866+0000","last_undegraded":"2024-09-16T09:40:48.070866+0000","last_fullsized":"2024-09-16T09:40:48.070866+0000","mapping_epoch":376,"log_start":"406'5010","ondisk_log_start":"406'5010","created":376,"last_epoch_clean":396,"parent":"0.0","parent_split_bits":0,"last_scrub":"396'4616","last_scrub_stamp":"2024-09-16T09:38:36.700747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_clean_scrub_stamp":"2024-09-16T09:38:36.700747+0000","objects_scrubbed":2301,"log_size":3828,"log_dups_size":0,"ondisk_log_size":3828,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39190528,"num_objects":598,"num_object_clones":0,"num_object_copies":1794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":598,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4223,"num_write_kb":80640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,0],"acting":[4,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.9","version":"73'79","reported_seq":871,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.447431+0000","last_change":"2024-09-16T09:39:33.905770+0000","last_active":"2024-09-16T09:40:40.447431+0000","last_peered":"2024-09-16T09:40:40.447431+0000","last_clean":"2024-09-16T09:40:40.447431+0000","last_became_active":"2024-09-16T09:26:35.126066+0000","last_became_peered":"2024-09-16T09:26:35.126066+0000","last_unstale":"2024-09-16T09:40:40.447431+0000","last_undegraded":"2024-09-16T09:40:40.447431+0000","last_fullsized":"2024-09-16T09:40:40.447431+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:39:33.905703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:39:33.905703+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:03.761872+0000","scrub_duration":47,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"285'20","reported_seq":827,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.443681+0000","last_change":"2024-09-16T09:40:34.155468+0000","last_active":"2024-09-16T09:40:40.443681+0000","last_peered":"2024-09-16T09:40:40.443681+0000","last_clean":"2024-09-16T09:40:40.443681+0000","last_became_active":"2024-09-16T09:25:03.707723+0000","last_became_peered":"2024-09-16T09:25:03.707723+0000","last_unstale":"2024-09-16T09:40:40.443681+0000","last_undegraded":"2024-09-16T09:40:40.443681+0000","last_fullsized":"2024-09-16T09:40:40.443681+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'20","last_scrub_stamp":"2024-09-16T09:40:34.154914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:40:34.154914+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:34.275480+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":674,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.453168+0000","last_change":"2024-09-16T09:39:38.508106+0000","last_active":"2024-09-16T09:40:40.453168+0000","last_peered":"2024-09-16T09:40:40.453168+0000","last_clean":"2024-09-16T09:40:40.453168+0000","last_became_active":"2024-09-16T09:31:09.288825+0000","last_became_peered":"2024-09-16T09:31:09.288825+0000","last_unstale":"2024-09-16T09:40:40.453168+0000","last_undegraded":"2024-09-16T09:40:40.453168+0000","last_fullsized":"2024-09-16T09:40:40.453168+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:39:38.507898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:39:38.507898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"228'9","reported_seq":578,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:41.938851+0000","last_change":"2024-09-16T09:39:53.948713+0000","last_active":"2024-09-16T09:40:41.938851+0000","last_peered":"2024-09-16T09:40:41.938851+0000","last_clean":"2024-09-16T09:40:41.938851+0000","last_became_active":"2024-09-16T09:33:21.337213+0000","last_became_peered":"2024-09-16T09:33:21.337213+0000","last_unstale":"2024-09-16T09:40:41.938851+0000","last_undegraded":"2024-09-16T09:40:41.938851+0000","last_fullsized":"2024-09-16T09:40:41.938851+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"228'9","last_scrub_stamp":"2024-09-16T09:39:53.948635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:39:53.948635+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:01.771359+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":820,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.443349+0000","last_change":"2024-09-16T09:38:26.182064+0000","last_active":"2024-09-16T09:40:40.443349+0000","last_peered":"2024-09-16T09:40:40.443349+0000","last_clean":"2024-09-16T09:40:40.443349+0000","last_became_active":"2024-09-16T09:25:07.718958+0000","last_became_peered":"2024-09-16T09:25:07.718958+0000","last_unstale":"2024-09-16T09:40:40.443349+0000","last_undegraded":"2024-09-16T09:40:40.443349+0000","last_fullsized":"2024-09-16T09:40:40.443349+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:38:26.181977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:38:26.181977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.5","version":"442'9025","reported_seq":9234,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:49.434183+0000","last_change":"2024-09-16T09:40:36.308556+0000","last_active":"2024-09-16T09:40:49.434183+0000","last_peered":"2024-09-16T09:40:49.434183+0000","last_clean":"2024-09-16T09:40:49.434183+0000","last_became_active":"2024-09-16T09:38:35.378878+0000","last_became_peered":"2024-09-16T09:38:35.378878+0000","last_unstale":"2024-09-16T09:40:49.434183+0000","last_undegraded":"2024-09-16T09:40:49.434183+0000","last_fullsized":"2024-09-16T09:40:49.434183+0000","mapping_epoch":376,"log_start":"393'4465","ondisk_log_start":"393'4465","created":376,"last_epoch_clean":396,"parent":"0.0","parent_split_bits":0,"last_scrub":"441'8454","last_scrub_stamp":"2024-09-16T09:40:36.308265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_clean_scrub_stamp":"2024-09-16T09:40:36.308265+0000","objects_scrubbed":1130,"log_size":4560,"log_dups_size":0,"ondisk_log_size":4560,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:56.781453+0000","scrub_duration":334,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36175872,"num_objects":552,"num_object_clones":0,"num_object_copies":1656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":552,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4360,"num_write_kb":80576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.e","version":"73'79","reported_seq":605,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.443264+0000","last_change":"2024-09-16T09:35:41.979877+0000","last_active":"2024-09-16T09:40:40.443264+0000","last_peered":"2024-09-16T09:40:40.443264+0000","last_clean":"2024-09-16T09:40:40.443264+0000","last_became_active":"2024-09-16T09:30:10.705072+0000","last_became_peered":"2024-09-16T09:30:10.705072+0000","last_unstale":"2024-09-16T09:40:40.443264+0000","last_undegraded":"2024-09-16T09:40:40.443264+0000","last_fullsized":"2024-09-16T09:40:40.443264+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:35:41.979814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:35:41.979814+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,3],"acting":[4,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.b","version":"285'15","reported_seq":855,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.452973+0000","last_change":"2024-09-16T09:36:55.651561+0000","last_active":"2024-09-16T09:40:40.452973+0000","last_peered":"2024-09-16T09:40:40.452973+0000","last_clean":"2024-09-16T09:40:40.452973+0000","last_became_active":"2024-09-16T09:36:31.590202+0000","last_became_peered":"2024-09-16T09:36:31.590202+0000","last_unstale":"2024-09-16T09:40:40.452973+0000","last_undegraded":"2024-09-16T09:40:40.452973+0000","last_fullsized":"2024-09-16T09:40:40.452973+0000","mapping_epoch":316,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":317,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'15","last_scrub_stamp":"2024-09-16T09:36:55.651496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:36:55.651496+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:00.405646+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":814,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.451845+0000","last_change":"2024-09-16T09:37:31.033631+0000","last_active":"2024-09-16T09:40:40.451845+0000","last_peered":"2024-09-16T09:40:40.451845+0000","last_clean":"2024-09-16T09:40:40.451845+0000","last_became_active":"2024-09-16T09:31:09.276516+0000","last_became_peered":"2024-09-16T09:31:09.276516+0000","last_unstale":"2024-09-16T09:40:40.451845+0000","last_undegraded":"2024-09-16T09:40:40.451845+0000","last_fullsized":"2024-09-16T09:40:40.451845+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:31.033577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:37:31.033577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":734,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.443122+0000","last_change":"2024-09-16T09:33:40.260547+0000","last_active":"2024-09-16T09:40:40.443122+0000","last_peered":"2024-09-16T09:40:40.443122+0000","last_clean":"2024-09-16T09:40:40.443122+0000","last_became_active":"2024-09-16T09:33:21.337254+0000","last_became_peered":"2024-09-16T09:33:21.337254+0000","last_unstale":"2024-09-16T09:40:40.443122+0000","last_undegraded":"2024-09-16T09:40:40.443122+0000","last_fullsized":"2024-09-16T09:40:40.443122+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:40.260500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:40.260500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3],"acting":[4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":805,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.452509+0000","last_change":"2024-09-16T09:36:53.647305+0000","last_active":"2024-09-16T09:40:40.452509+0000","last_peered":"2024-09-16T09:40:40.452509+0000","last_clean":"2024-09-16T09:40:40.452509+0000","last_became_active":"2024-09-16T09:25:07.718778+0000","last_became_peered":"2024-09-16T09:25:07.718778+0000","last_unstale":"2024-09-16T09:40:40.452509+0000","last_undegraded":"2024-09-16T09:40:40.452509+0000","last_fullsized":"2024-09-16T09:40:40.452509+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:53.647256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:36:53.647256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:07.329292+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"442'8969","reported_seq":9177,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:49.453089+0000","last_change":"2024-09-16T09:40:37.352913+0000","last_active":"2024-09-16T09:40:49.453089+0000","last_peered":"2024-09-16T09:40:49.453089+0000","last_clean":"2024-09-16T09:40:49.453089+0000","last_became_active":"2024-09-16T09:38:35.386054+0000","last_became_peered":"2024-09-16T09:38:35.386054+0000","last_unstale":"2024-09-16T09:40:49.453089+0000","last_undegraded":"2024-09-16T09:40:49.453089+0000","last_fullsized":"2024-09-16T09:40:49.453089+0000","mapping_epoch":376,"log_start":"392'4390","ondisk_log_start":"392'4390","created":376,"last_epoch_clean":396,"parent":"0.0","parent_split_bits":0,"last_scrub":"441'8438","last_scrub_stamp":"2024-09-16T09:40:37.352765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_clean_scrub_stamp":"2024-09-16T09:40:37.352765+0000","objects_scrubbed":1086,"log_size":4579,"log_dups_size":0,"ondisk_log_size":4579,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:52.837832+0000","scrub_duration":397,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36175872,"num_objects":552,"num_object_clones":0,"num_object_copies":1656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":552,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4329,"num_write_kb":81280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.f","version":"73'79","reported_seq":865,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.453143+0000","last_change":"2024-09-16T09:40:23.896463+0000","last_active":"2024-09-16T09:40:40.453143+0000","last_peered":"2024-09-16T09:40:40.453143+0000","last_clean":"2024-09-16T09:40:40.453143+0000","last_became_active":"2024-09-16T09:30:10.693477+0000","last_became_peered":"2024-09-16T09:30:10.693477+0000","last_unstale":"2024-09-16T09:40:40.453143+0000","last_undegraded":"2024-09-16T09:40:40.453143+0000","last_fullsized":"2024-09-16T09:40:40.453143+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:40:23.896346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:40:23.896346+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:28.637690+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"285'35","reported_seq":868,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.446006+0000","last_change":"2024-09-16T09:37:28.078195+0000","last_active":"2024-09-16T09:40:40.446006+0000","last_peered":"2024-09-16T09:40:40.446006+0000","last_clean":"2024-09-16T09:40:40.446006+0000","last_became_active":"2024-09-16T09:25:03.712783+0000","last_became_peered":"2024-09-16T09:25:03.712783+0000","last_unstale":"2024-09-16T09:40:40.446006+0000","last_undegraded":"2024-09-16T09:40:40.446006+0000","last_fullsized":"2024-09-16T09:40:40.446006+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'35","last_scrub_stamp":"2024-09-16T09:37:28.077841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:37:28.077841+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":56,"num_read_kb":45,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"59'1","reported_seq":821,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.448653+0000","last_change":"2024-09-16T09:37:56.528872+0000","last_active":"2024-09-16T09:40:40.448653+0000","last_peered":"2024-09-16T09:40:40.448653+0000","last_clean":"2024-09-16T09:40:40.448653+0000","last_became_active":"2024-09-16T09:25:01.708069+0000","last_became_peered":"2024-09-16T09:25:01.708069+0000","last_unstale":"2024-09-16T09:40:40.448653+0000","last_undegraded":"2024-09-16T09:40:40.448653+0000","last_fullsized":"2024-09-16T09:40:40.448653+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:37:56.528796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:37:56.528796+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:11.118413+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":838,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.451461+0000","last_change":"2024-09-16T09:40:39.696450+0000","last_active":"2024-09-16T09:40:40.451461+0000","last_peered":"2024-09-16T09:40:40.451461+0000","last_clean":"2024-09-16T09:40:40.451461+0000","last_became_active":"2024-09-16T09:37:53.518284+0000","last_became_peered":"2024-09-16T09:37:53.518284+0000","last_unstale":"2024-09-16T09:40:40.451461+0000","last_undegraded":"2024-09-16T09:40:40.451461+0000","last_fullsized":"2024-09-16T09:40:40.451461+0000","mapping_epoch":382,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":383,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:40:39.696383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:40:39.696383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:00.771862+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":787,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.448035+0000","last_change":"2024-09-16T09:36:48.649739+0000","last_active":"2024-09-16T09:40:40.448035+0000","last_peered":"2024-09-16T09:40:40.448035+0000","last_clean":"2024-09-16T09:40:40.448035+0000","last_became_active":"2024-09-16T09:25:07.718750+0000","last_became_peered":"2024-09-16T09:25:07.718750+0000","last_unstale":"2024-09-16T09:40:40.448035+0000","last_undegraded":"2024-09-16T09:40:40.448035+0000","last_fullsized":"2024-09-16T09:40:40.448035+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:48.649696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:36:48.649696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:08.137530+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.7","version":"442'8854","reported_seq":9037,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:47.879378+0000","last_change":"2024-09-16T09:38:36.686541+0000","last_active":"2024-09-16T09:40:47.879378+0000","last_peered":"2024-09-16T09:40:47.879378+0000","last_clean":"2024-09-16T09:40:47.879378+0000","last_became_active":"2024-09-16T09:38:35.358645+0000","last_became_peered":"2024-09-16T09:38:35.358645+0000","last_unstale":"2024-09-16T09:40:47.879378+0000","last_undegraded":"2024-09-16T09:40:47.879378+0000","last_fullsized":"2024-09-16T09:40:47.879378+0000","mapping_epoch":376,"log_start":"390'3565","ondisk_log_start":"390'3565","created":376,"last_epoch_clean":396,"parent":"0.0","parent_split_bits":0,"last_scrub":"396'4653","last_scrub_stamp":"2024-09-16T09:38:36.686443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_clean_scrub_stamp":"2024-09-16T09:38:36.686443+0000","objects_scrubbed":2327,"log_size":5289,"log_dups_size":0,"ondisk_log_size":5289,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":549,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38010880,"num_objects":580,"num_object_clones":0,"num_object_copies":1740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":580,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4203,"num_write_kb":78592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.c","version":"73'79","reported_seq":554,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.448820+0000","last_change":"2024-09-16T09:40:35.772064+0000","last_active":"2024-09-16T09:40:40.448820+0000","last_peered":"2024-09-16T09:40:40.448820+0000","last_clean":"2024-09-16T09:40:40.448820+0000","last_became_active":"2024-09-16T09:31:09.273770+0000","last_became_peered":"2024-09-16T09:31:09.273770+0000","last_unstale":"2024-09-16T09:40:40.448820+0000","last_undegraded":"2024-09-16T09:40:40.448820+0000","last_fullsized":"2024-09-16T09:40:40.448820+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:40:35.772017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:40:35.772017+0000","objects_scrubbed":1,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:00.885825+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":327680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"285'18","reported_seq":883,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.442780+0000","last_change":"2024-09-16T09:36:50.603724+0000","last_active":"2024-09-16T09:40:40.442780+0000","last_peered":"2024-09-16T09:40:40.442780+0000","last_clean":"2024-09-16T09:40:40.442780+0000","last_became_active":"2024-09-16T09:31:09.269047+0000","last_became_peered":"2024-09-16T09:31:09.269047+0000","last_unstale":"2024-09-16T09:40:40.442780+0000","last_undegraded":"2024-09-16T09:40:40.442780+0000","last_fullsized":"2024-09-16T09:40:40.442780+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'18","last_scrub_stamp":"2024-09-16T09:36:50.603660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:36:50.603660+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":815,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.449139+0000","last_change":"2024-09-16T09:37:57.529616+0000","last_active":"2024-09-16T09:40:40.449139+0000","last_peered":"2024-09-16T09:40:40.449139+0000","last_clean":"2024-09-16T09:40:40.449139+0000","last_became_active":"2024-09-16T09:25:01.707528+0000","last_became_peered":"2024-09-16T09:25:01.707528+0000","last_unstale":"2024-09-16T09:40:40.449139+0000","last_undegraded":"2024-09-16T09:40:40.449139+0000","last_fullsized":"2024-09-16T09:40:40.449139+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:57.529554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:37:57.529554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:14.993053+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":468,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.452206+0000","last_change":"2024-09-16T09:40:25.011759+0000","last_active":"2024-09-16T09:40:40.452206+0000","last_peered":"2024-09-16T09:40:40.452206+0000","last_clean":"2024-09-16T09:40:40.452206+0000","last_became_active":"2024-09-16T09:33:21.337309+0000","last_became_peered":"2024-09-16T09:33:21.337309+0000","last_unstale":"2024-09-16T09:40:40.452206+0000","last_undegraded":"2024-09-16T09:40:40.452206+0000","last_fullsized":"2024-09-16T09:40:40.452206+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:40:25.011687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:40:25.011687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:31.455077+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":804,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.448415+0000","last_change":"2024-09-16T09:38:21.097872+0000","last_active":"2024-09-16T09:40:40.448415+0000","last_peered":"2024-09-16T09:40:40.448415+0000","last_clean":"2024-09-16T09:40:40.448415+0000","last_became_active":"2024-09-16T09:25:07.718899+0000","last_became_peered":"2024-09-16T09:25:07.718899+0000","last_unstale":"2024-09-16T09:40:40.448415+0000","last_undegraded":"2024-09-16T09:40:40.448415+0000","last_fullsized":"2024-09-16T09:40:40.448415+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:38:21.097804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:38:21.097804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.6","version":"442'8525","reported_seq":8709,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:48.269147+0000","last_change":"2024-09-16T09:38:38.320135+0000","last_active":"2024-09-16T09:40:48.269147+0000","last_peered":"2024-09-16T09:40:48.269147+0000","last_clean":"2024-09-16T09:40:48.269147+0000","last_became_active":"2024-09-16T09:38:35.382660+0000","last_became_peered":"2024-09-16T09:38:35.382660+0000","last_unstale":"2024-09-16T09:40:48.269147+0000","last_undegraded":"2024-09-16T09:40:48.269147+0000","last_fullsized":"2024-09-16T09:40:48.269147+0000","mapping_epoch":376,"log_start":"391'3655","ondisk_log_start":"391'3655","created":376,"last_epoch_clean":396,"parent":"0.0","parent_split_bits":0,"last_scrub":"397'4461","last_scrub_stamp":"2024-09-16T09:38:38.320065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_clean_scrub_stamp":"2024-09-16T09:38:38.320065+0000","objects_scrubbed":2226,"log_size":4870,"log_dups_size":0,"ondisk_log_size":4870,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":197,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39059456,"num_objects":596,"num_object_clones":0,"num_object_copies":1788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":596,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4064,"num_write_kb":77888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.d","version":"73'79","reported_seq":604,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.448366+0000","last_change":"2024-09-16T09:39:49.055357+0000","last_active":"2024-09-16T09:40:40.448366+0000","last_peered":"2024-09-16T09:40:40.448366+0000","last_clean":"2024-09-16T09:40:40.448366+0000","last_became_active":"2024-09-16T09:30:10.709958+0000","last_became_peered":"2024-09-16T09:30:10.709958+0000","last_unstale":"2024-09-16T09:40:40.448366+0000","last_undegraded":"2024-09-16T09:40:40.448366+0000","last_fullsized":"2024-09-16T09:40:40.448366+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:39:49.055292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:39:49.055292+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:40:56.611486+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"285'27","reported_seq":839,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.449852+0000","last_change":"2024-09-16T09:37:30.081518+0000","last_active":"2024-09-16T09:40:40.449852+0000","last_peered":"2024-09-16T09:40:40.449852+0000","last_clean":"2024-09-16T09:40:40.449852+0000","last_became_active":"2024-09-16T09:25:03.715733+0000","last_became_peered":"2024-09-16T09:25:03.715733+0000","last_unstale":"2024-09-16T09:40:40.449852+0000","last_undegraded":"2024-09-16T09:40:40.449852+0000","last_fullsized":"2024-09-16T09:40:40.449852+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'27","last_scrub_stamp":"2024-09-16T09:37:30.081447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:37:30.081447+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":818,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.448816+0000","last_change":"2024-09-16T09:37:17.544132+0000","last_active":"2024-09-16T09:40:40.448816+0000","last_peered":"2024-09-16T09:40:40.448816+0000","last_clean":"2024-09-16T09:40:40.448816+0000","last_became_active":"2024-09-16T09:25:01.707372+0000","last_became_peered":"2024-09-16T09:25:01.707372+0000","last_unstale":"2024-09-16T09:40:40.448816+0000","last_undegraded":"2024-09-16T09:40:40.448816+0000","last_fullsized":"2024-09-16T09:40:40.448816+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-16T09:37:17.544069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:37:17.544069+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:09.144451+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"66'8","reported_seq":1005,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:41.939136+0000","last_change":"2024-09-16T09:39:37.890572+0000","last_active":"2024-09-16T09:40:41.939136+0000","last_peered":"2024-09-16T09:40:41.939136+0000","last_clean":"2024-09-16T09:40:41.939136+0000","last_became_active":"2024-09-16T09:25:05.716189+0000","last_became_peered":"2024-09-16T09:25:05.716189+0000","last_unstale":"2024-09-16T09:40:41.939136+0000","last_undegraded":"2024-09-16T09:40:41.939136+0000","last_fullsized":"2024-09-16T09:40:41.939136+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:39:37.890495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:39:37.890495+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:40:59.828626+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":788,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.450724+0000","last_change":"2024-09-16T09:37:47.244099+0000","last_active":"2024-09-16T09:40:40.450724+0000","last_peered":"2024-09-16T09:40:40.450724+0000","last_clean":"2024-09-16T09:40:40.450724+0000","last_became_active":"2024-09-16T09:25:07.717213+0000","last_became_peered":"2024-09-16T09:25:07.717213+0000","last_unstale":"2024-09-16T09:40:40.450724+0000","last_undegraded":"2024-09-16T09:40:40.450724+0000","last_fullsized":"2024-09-16T09:40:40.450724+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:47.244032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:37:47.244032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.19","version":"442'8732","reported_seq":18085,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:48.052014+0000","last_change":"2024-09-16T09:40:23.497766+0000","last_active":"2024-09-16T09:40:48.052014+0000","last_peered":"2024-09-16T09:40:48.052014+0000","last_clean":"2024-09-16T09:40:48.052014+0000","last_became_active":"2024-09-16T09:40:22.412813+0000","last_became_peered":"2024-09-16T09:40:22.412813+0000","last_unstale":"2024-09-16T09:40:48.052014+0000","last_undegraded":"2024-09-16T09:40:48.052014+0000","last_fullsized":"2024-09-16T09:40:48.052014+0000","mapping_epoch":439,"log_start":"406'4892","ondisk_log_start":"406'4892","created":395,"last_epoch_clean":440,"parent":"0.0","parent_split_bits":5,"last_scrub":"440'7691","last_scrub_stamp":"2024-09-16T09:40:23.497534+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_clean_scrub_stamp":"2024-09-16T09:40:23.497534+0000","objects_scrubbed":1664,"log_size":3840,"log_dups_size":0,"ondisk_log_size":3840,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:42.347931+0000","scrub_duration":359,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40632320,"num_objects":620,"num_object_clones":0,"num_object_copies":1860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":620,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1044,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.12","version":"73'79","reported_seq":613,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.450928+0000","last_change":"2024-09-16T09:40:38.120017+0000","last_active":"2024-09-16T09:40:40.450928+0000","last_peered":"2024-09-16T09:40:40.450928+0000","last_clean":"2024-09-16T09:40:40.450928+0000","last_became_active":"2024-09-16T09:30:10.731413+0000","last_became_peered":"2024-09-16T09:30:10.731413+0000","last_unstale":"2024-09-16T09:40:40.450928+0000","last_undegraded":"2024-09-16T09:40:40.450928+0000","last_fullsized":"2024-09-16T09:40:40.450928+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:40:38.119809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:40:38.119809+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:05.666162+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":822,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.445686+0000","last_change":"2024-09-16T09:37:55.863661+0000","last_active":"2024-09-16T09:40:40.445686+0000","last_peered":"2024-09-16T09:40:40.445686+0000","last_clean":"2024-09-16T09:40:40.445686+0000","last_became_active":"2024-09-16T09:37:53.518354+0000","last_became_peered":"2024-09-16T09:37:53.518354+0000","last_unstale":"2024-09-16T09:40:40.445686+0000","last_undegraded":"2024-09-16T09:40:40.445686+0000","last_fullsized":"2024-09-16T09:40:40.445686+0000","mapping_epoch":382,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":383,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:55.863613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:37:55.863613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"285'12","reported_seq":581,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.458035+0000","last_change":"2024-09-16T09:38:33.535691+0000","last_active":"2024-09-16T09:40:40.458035+0000","last_peered":"2024-09-16T09:40:40.458035+0000","last_clean":"2024-09-16T09:40:40.458035+0000","last_became_active":"2024-09-16T09:38:33.517889+0000","last_became_peered":"2024-09-16T09:38:33.517889+0000","last_unstale":"2024-09-16T09:40:40.458035+0000","last_undegraded":"2024-09-16T09:40:40.458035+0000","last_fullsized":"2024-09-16T09:40:40.458035+0000","mapping_epoch":393,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":394,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'12","last_scrub_stamp":"2024-09-16T09:37:20.749101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:37:20.749101+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":17,"num_read_kb":14,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":807,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.450758+0000","last_change":"2024-09-16T09:40:28.258952+0000","last_active":"2024-09-16T09:40:40.450758+0000","last_peered":"2024-09-16T09:40:40.450758+0000","last_clean":"2024-09-16T09:40:40.450758+0000","last_became_active":"2024-09-16T09:25:05.715616+0000","last_became_peered":"2024-09-16T09:25:05.715616+0000","last_unstale":"2024-09-16T09:40:40.450758+0000","last_undegraded":"2024-09-16T09:40:40.450758+0000","last_fullsized":"2024-09-16T09:40:40.450758+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:40:28.258907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:40:28.258907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:56.348797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":802,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.448279+0000","last_change":"2024-09-16T09:40:37.040258+0000","last_active":"2024-09-16T09:40:40.448279+0000","last_peered":"2024-09-16T09:40:40.448279+0000","last_clean":"2024-09-16T09:40:40.448279+0000","last_became_active":"2024-09-16T09:25:07.721059+0000","last_became_peered":"2024-09-16T09:25:07.721059+0000","last_unstale":"2024-09-16T09:40:40.448279+0000","last_undegraded":"2024-09-16T09:40:40.448279+0000","last_fullsized":"2024-09-16T09:40:40.448279+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:40:37.040205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:40:37.040205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:05.338645+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.18","version":"442'8727","reported_seq":15772,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:48.305919+0000","last_change":"2024-09-16T09:39:26.324950+0000","last_active":"2024-09-16T09:40:48.305919+0000","last_peered":"2024-09-16T09:40:48.305919+0000","last_clean":"2024-09-16T09:40:48.305919+0000","last_became_active":"2024-09-16T09:39:24.372407+0000","last_became_peered":"2024-09-16T09:39:24.372407+0000","last_unstale":"2024-09-16T09:40:48.305919+0000","last_undegraded":"2024-09-16T09:40:48.305919+0000","last_fullsized":"2024-09-16T09:40:48.305919+0000","mapping_epoch":416,"log_start":"385'1559","ondisk_log_start":"385'1559","created":395,"last_epoch_clean":417,"parent":"0.0","parent_split_bits":5,"last_scrub":"417'5794","last_scrub_stamp":"2024-09-16T09:39:26.324691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_clean_scrub_stamp":"2024-09-16T09:39:26.324691+0000","objects_scrubbed":3483,"log_size":7168,"log_dups_size":0,"ondisk_log_size":7168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:40:49.206180+0000","scrub_duration":951,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37355520,"num_objects":570,"num_object_clones":0,"num_object_copies":1710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":570,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2937,"num_write_kb":768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.13","version":"73'79","reported_seq":699,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.448573+0000","last_change":"2024-09-16T09:40:30.696059+0000","last_active":"2024-09-16T09:40:40.448573+0000","last_peered":"2024-09-16T09:40:40.448573+0000","last_clean":"2024-09-16T09:40:40.448573+0000","last_became_active":"2024-09-16T09:33:30.248497+0000","last_became_peered":"2024-09-16T09:33:30.248497+0000","last_unstale":"2024-09-16T09:40:40.448573+0000","last_undegraded":"2024-09-16T09:40:40.448573+0000","last_fullsized":"2024-09-16T09:40:40.448573+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:40:30.695866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:40:30.695866+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:53.960674+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"285'15","reported_seq":887,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.453434+0000","last_change":"2024-09-16T09:39:46.504126+0000","last_active":"2024-09-16T09:40:40.453434+0000","last_peered":"2024-09-16T09:40:40.453434+0000","last_clean":"2024-09-16T09:40:40.453434+0000","last_became_active":"2024-09-16T09:31:09.273498+0000","last_became_peered":"2024-09-16T09:31:09.273498+0000","last_unstale":"2024-09-16T09:40:40.453434+0000","last_undegraded":"2024-09-16T09:40:40.453434+0000","last_fullsized":"2024-09-16T09:40:40.453434+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'15","last_scrub_stamp":"2024-09-16T09:39:46.504045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:39:46.504045+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:09.366862+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":814,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.447654+0000","last_change":"2024-09-16T09:39:50.980277+0000","last_active":"2024-09-16T09:40:40.447654+0000","last_peered":"2024-09-16T09:40:40.447654+0000","last_clean":"2024-09-16T09:40:40.447654+0000","last_became_active":"2024-09-16T09:25:01.706491+0000","last_became_peered":"2024-09-16T09:25:01.706491+0000","last_unstale":"2024-09-16T09:40:40.447654+0000","last_undegraded":"2024-09-16T09:40:40.447654+0000","last_fullsized":"2024-09-16T09:40:40.447654+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:39:50.980221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:39:50.980221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:02.037457+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":580,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.448539+0000","last_change":"2024-09-16T09:40:29.980587+0000","last_active":"2024-09-16T09:40:40.448539+0000","last_peered":"2024-09-16T09:40:40.448539+0000","last_clean":"2024-09-16T09:40:40.448539+0000","last_became_active":"2024-09-16T09:33:30.248992+0000","last_became_peered":"2024-09-16T09:33:30.248992+0000","last_unstale":"2024-09-16T09:40:40.448539+0000","last_undegraded":"2024-09-16T09:40:40.448539+0000","last_fullsized":"2024-09-16T09:40:40.448539+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:40:29.980515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:40:29.980515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:43.227203+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":812,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.451093+0000","last_change":"2024-09-16T09:40:35.646704+0000","last_active":"2024-09-16T09:40:40.451093+0000","last_peered":"2024-09-16T09:40:40.451093+0000","last_clean":"2024-09-16T09:40:40.451093+0000","last_became_active":"2024-09-16T09:25:07.722050+0000","last_became_peered":"2024-09-16T09:25:07.722050+0000","last_unstale":"2024-09-16T09:40:40.451093+0000","last_undegraded":"2024-09-16T09:40:40.451093+0000","last_fullsized":"2024-09-16T09:40:40.451093+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:40:35.646535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:40:35.646535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:49.661431+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.10","version":"73'79","reported_seq":605,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.449746+0000","last_change":"2024-09-16T09:36:41.169117+0000","last_active":"2024-09-16T09:40:40.449746+0000","last_peered":"2024-09-16T09:40:40.449746+0000","last_clean":"2024-09-16T09:40:40.449746+0000","last_became_active":"2024-09-16T09:30:10.692770+0000","last_became_peered":"2024-09-16T09:30:10.692770+0000","last_unstale":"2024-09-16T09:40:40.449746+0000","last_undegraded":"2024-09-16T09:40:40.449746+0000","last_fullsized":"2024-09-16T09:40:40.449746+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:36:41.169069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:36:41.169069+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"285'15","reported_seq":828,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.450630+0000","last_change":"2024-09-16T09:37:55.203919+0000","last_active":"2024-09-16T09:40:40.450630+0000","last_peered":"2024-09-16T09:40:40.450630+0000","last_clean":"2024-09-16T09:40:40.450630+0000","last_became_active":"2024-09-16T09:25:03.717310+0000","last_became_peered":"2024-09-16T09:25:03.717310+0000","last_unstale":"2024-09-16T09:40:40.450630+0000","last_undegraded":"2024-09-16T09:40:40.450630+0000","last_fullsized":"2024-09-16T09:40:40.450630+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'15","last_scrub_stamp":"2024-09-16T09:37:55.203856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:37:55.203856+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":846,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.458439+0000","last_change":"2024-09-16T09:40:23.065431+0000","last_active":"2024-09-16T09:40:40.458439+0000","last_peered":"2024-09-16T09:40:40.458439+0000","last_clean":"2024-09-16T09:40:40.458439+0000","last_became_active":"2024-09-16T09:31:09.272848+0000","last_became_peered":"2024-09-16T09:31:09.272848+0000","last_unstale":"2024-09-16T09:40:40.458439+0000","last_undegraded":"2024-09-16T09:40:40.458439+0000","last_fullsized":"2024-09-16T09:40:40.458439+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:40:23.065365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:40:23.065365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:43.725941+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"187'11","reported_seq":842,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:41.938920+0000","last_change":"2024-09-16T09:37:19.540982+0000","last_active":"2024-09-16T09:40:41.938920+0000","last_peered":"2024-09-16T09:40:41.938920+0000","last_clean":"2024-09-16T09:40:41.938920+0000","last_became_active":"2024-09-16T09:31:09.277688+0000","last_became_peered":"2024-09-16T09:31:09.277688+0000","last_unstale":"2024-09-16T09:40:41.938920+0000","last_undegraded":"2024-09-16T09:40:41.938920+0000","last_fullsized":"2024-09-16T09:40:41.938920+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"187'11","last_scrub_stamp":"2024-09-16T09:37:19.540886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:37:19.540886+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:10.143855+0000","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":820,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.442530+0000","last_change":"2024-09-16T09:38:31.156207+0000","last_active":"2024-09-16T09:40:40.442530+0000","last_peered":"2024-09-16T09:40:40.442530+0000","last_clean":"2024-09-16T09:40:40.442530+0000","last_became_active":"2024-09-16T09:25:07.718810+0000","last_became_peered":"2024-09-16T09:25:07.718810+0000","last_unstale":"2024-09-16T09:40:40.442530+0000","last_undegraded":"2024-09-16T09:40:40.442530+0000","last_fullsized":"2024-09-16T09:40:40.442530+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:38:31.156135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:38:31.156135+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.11","version":"73'79","reported_seq":691,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.455825+0000","last_change":"2024-09-16T09:37:15.623104+0000","last_active":"2024-09-16T09:40:40.455825+0000","last_peered":"2024-09-16T09:40:40.455825+0000","last_clean":"2024-09-16T09:40:40.455825+0000","last_became_active":"2024-09-16T09:31:09.272208+0000","last_became_peered":"2024-09-16T09:31:09.272208+0000","last_unstale":"2024-09-16T09:40:40.455825+0000","last_undegraded":"2024-09-16T09:40:40.455825+0000","last_fullsized":"2024-09-16T09:40:40.455825+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:37:15.623041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:37:15.623041+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:06.375385+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.14","version":"285'20","reported_seq":505,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.448880+0000","last_change":"2024-09-16T09:40:35.027221+0000","last_active":"2024-09-16T09:40:40.448880+0000","last_peered":"2024-09-16T09:40:40.448880+0000","last_clean":"2024-09-16T09:40:40.448880+0000","last_became_active":"2024-09-16T09:33:30.249769+0000","last_became_peered":"2024-09-16T09:33:30.249769+0000","last_unstale":"2024-09-16T09:40:40.448880+0000","last_undegraded":"2024-09-16T09:40:40.448880+0000","last_fullsized":"2024-09-16T09:40:40.448880+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'20","last_scrub_stamp":"2024-09-16T09:40:35.027096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:40:35.027096+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:41.457202+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":806,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.447376+0000","last_change":"2024-09-16T09:39:36.839344+0000","last_active":"2024-09-16T09:40:40.447376+0000","last_peered":"2024-09-16T09:40:40.447376+0000","last_clean":"2024-09-16T09:40:40.447376+0000","last_became_active":"2024-09-16T09:25:01.707513+0000","last_became_peered":"2024-09-16T09:25:01.707513+0000","last_unstale":"2024-09-16T09:40:40.447376+0000","last_undegraded":"2024-09-16T09:40:40.447376+0000","last_fullsized":"2024-09-16T09:40:40.447376+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:39:36.839301+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:39:36.839301+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:01.125763+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"66'8","reported_seq":1013,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:41.939472+0000","last_change":"2024-09-16T09:38:24.130552+0000","last_active":"2024-09-16T09:40:41.939472+0000","last_peered":"2024-09-16T09:40:41.939472+0000","last_clean":"2024-09-16T09:40:41.939472+0000","last_became_active":"2024-09-16T09:25:05.713833+0000","last_became_peered":"2024-09-16T09:25:05.713833+0000","last_unstale":"2024-09-16T09:40:41.939472+0000","last_undegraded":"2024-09-16T09:40:41.939472+0000","last_fullsized":"2024-09-16T09:40:41.939472+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:38:24.130499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:38:24.130499+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":834,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.452999+0000","last_change":"2024-09-16T09:40:26.019049+0000","last_active":"2024-09-16T09:40:40.452999+0000","last_peered":"2024-09-16T09:40:40.452999+0000","last_clean":"2024-09-16T09:40:40.452999+0000","last_became_active":"2024-09-16T09:31:09.275168+0000","last_became_peered":"2024-09-16T09:31:09.275168+0000","last_unstale":"2024-09-16T09:40:40.452999+0000","last_undegraded":"2024-09-16T09:40:40.452999+0000","last_fullsized":"2024-09-16T09:40:40.452999+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:40:26.018988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:40:26.018988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:39.631325+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"285'19","reported_seq":846,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.442669+0000","last_change":"2024-09-16T09:37:31.362985+0000","last_active":"2024-09-16T09:40:40.442669+0000","last_peered":"2024-09-16T09:40:40.442669+0000","last_clean":"2024-09-16T09:40:40.442669+0000","last_became_active":"2024-09-16T09:26:33.291265+0000","last_became_peered":"2024-09-16T09:26:33.291265+0000","last_unstale":"2024-09-16T09:40:40.442669+0000","last_undegraded":"2024-09-16T09:40:40.442669+0000","last_fullsized":"2024-09-16T09:40:40.442669+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'19","last_scrub_stamp":"2024-09-16T09:37:31.362927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:37:31.362927+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":824,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.443777+0000","last_change":"2024-09-16T09:37:53.239593+0000","last_active":"2024-09-16T09:40:40.443777+0000","last_peered":"2024-09-16T09:40:40.443777+0000","last_clean":"2024-09-16T09:40:40.443777+0000","last_became_active":"2024-09-16T09:25:01.705158+0000","last_became_peered":"2024-09-16T09:25:01.705158+0000","last_unstale":"2024-09-16T09:40:40.443777+0000","last_undegraded":"2024-09-16T09:40:40.443777+0000","last_fullsized":"2024-09-16T09:40:40.443777+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:53.239547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:37:53.239547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":744,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.450800+0000","last_change":"2024-09-16T09:37:58.252056+0000","last_active":"2024-09-16T09:40:40.450800+0000","last_peered":"2024-09-16T09:40:40.450800+0000","last_clean":"2024-09-16T09:40:40.450800+0000","last_became_active":"2024-09-16T09:37:41.515278+0000","last_became_peered":"2024-09-16T09:37:41.515278+0000","last_unstale":"2024-09-16T09:40:40.450800+0000","last_undegraded":"2024-09-16T09:40:40.450800+0000","last_fullsized":"2024-09-16T09:40:40.450800+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:58.251994+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:37:58.251994+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":651,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.452405+0000","last_change":"2024-09-16T09:34:48.786847+0000","last_active":"2024-09-16T09:40:40.452405+0000","last_peered":"2024-09-16T09:40:40.452405+0000","last_clean":"2024-09-16T09:40:40.452405+0000","last_became_active":"2024-09-16T09:31:09.276042+0000","last_became_peered":"2024-09-16T09:31:09.276042+0000","last_unstale":"2024-09-16T09:40:40.452405+0000","last_undegraded":"2024-09-16T09:40:40.452405+0000","last_fullsized":"2024-09-16T09:40:40.452405+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:48.786785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:34:48.786785+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:40:59.365275+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"285'15","reported_seq":527,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.446057+0000","last_change":"2024-09-16T09:38:30.104067+0000","last_active":"2024-09-16T09:40:40.446057+0000","last_peered":"2024-09-16T09:40:40.446057+0000","last_clean":"2024-09-16T09:40:40.446057+0000","last_became_active":"2024-09-16T09:37:33.348223+0000","last_became_peered":"2024-09-16T09:37:33.348223+0000","last_unstale":"2024-09-16T09:40:40.446057+0000","last_undegraded":"2024-09-16T09:40:40.446057+0000","last_fullsized":"2024-09-16T09:40:40.446057+0000","mapping_epoch":367,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":368,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'15","last_scrub_stamp":"2024-09-16T09:38:30.103857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:38:30.103857+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,2],"acting":[6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":806,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.448544+0000","last_change":"2024-09-16T09:39:43.974565+0000","last_active":"2024-09-16T09:40:40.448544+0000","last_peered":"2024-09-16T09:40:40.448544+0000","last_clean":"2024-09-16T09:40:40.448544+0000","last_became_active":"2024-09-16T09:25:01.706762+0000","last_became_peered":"2024-09-16T09:25:01.706762+0000","last_unstale":"2024-09-16T09:40:40.448544+0000","last_undegraded":"2024-09-16T09:40:40.448544+0000","last_fullsized":"2024-09-16T09:40:40.448544+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:39:43.974495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:39:43.974495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:09.278400+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":644,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.452565+0000","last_change":"2024-09-16T09:39:53.547824+0000","last_active":"2024-09-16T09:40:40.452565+0000","last_peered":"2024-09-16T09:40:40.452565+0000","last_clean":"2024-09-16T09:40:40.452565+0000","last_became_active":"2024-09-16T09:28:52.982865+0000","last_became_peered":"2024-09-16T09:28:52.982865+0000","last_unstale":"2024-09-16T09:40:40.452565+0000","last_undegraded":"2024-09-16T09:40:40.452565+0000","last_fullsized":"2024-09-16T09:40:40.452565+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:39:53.547749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:39:53.547749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:40:56.313049+0000","scrub_duration":136,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":796,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.452158+0000","last_change":"2024-09-16T09:34:50.779241+0000","last_active":"2024-09-16T09:40:40.452158+0000","last_peered":"2024-09-16T09:40:40.452158+0000","last_clean":"2024-09-16T09:40:40.452158+0000","last_became_active":"2024-09-16T09:25:07.715301+0000","last_became_peered":"2024-09-16T09:25:07.715301+0000","last_unstale":"2024-09-16T09:40:40.452158+0000","last_undegraded":"2024-09-16T09:40:40.452158+0000","last_fullsized":"2024-09-16T09:40:40.452158+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:50.779164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:34:50.779164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:40:57.381159+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.14","version":"73'79","reported_seq":613,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.450991+0000","last_change":"2024-09-16T09:40:28.499828+0000","last_active":"2024-09-16T09:40:40.450991+0000","last_peered":"2024-09-16T09:40:40.450991+0000","last_clean":"2024-09-16T09:40:40.450991+0000","last_became_active":"2024-09-16T09:30:10.700866+0000","last_became_peered":"2024-09-16T09:30:10.700866+0000","last_unstale":"2024-09-16T09:40:40.450991+0000","last_undegraded":"2024-09-16T09:40:40.450991+0000","last_fullsized":"2024-09-16T09:40:40.450991+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:40:28.499763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:40:28.499763+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:36.575921+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"285'19","reported_seq":694,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.447528+0000","last_change":"2024-09-16T09:39:28.862654+0000","last_active":"2024-09-16T09:40:40.447528+0000","last_peered":"2024-09-16T09:40:40.447528+0000","last_clean":"2024-09-16T09:40:40.447528+0000","last_became_active":"2024-09-16T09:28:52.971132+0000","last_became_peered":"2024-09-16T09:28:52.971132+0000","last_unstale":"2024-09-16T09:40:40.447528+0000","last_undegraded":"2024-09-16T09:40:40.447528+0000","last_fullsized":"2024-09-16T09:40:40.447528+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'19","last_scrub_stamp":"2024-09-16T09:39:28.862503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:39:28.862503+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:40:44.605059+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":815,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.450886+0000","last_change":"2024-09-16T09:38:21.213911+0000","last_active":"2024-09-16T09:40:40.450886+0000","last_peered":"2024-09-16T09:40:40.450886+0000","last_clean":"2024-09-16T09:40:40.450886+0000","last_became_active":"2024-09-16T09:25:01.700733+0000","last_became_peered":"2024-09-16T09:25:01.700733+0000","last_unstale":"2024-09-16T09:40:40.450886+0000","last_undegraded":"2024-09-16T09:40:40.450886+0000","last_fullsized":"2024-09-16T09:40:40.450886+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:38:21.213844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:38:21.213844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":814,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.447825+0000","last_change":"2024-09-16T09:40:39.969447+0000","last_active":"2024-09-16T09:40:40.447825+0000","last_peered":"2024-09-16T09:40:40.447825+0000","last_clean":"2024-09-16T09:40:40.447825+0000","last_became_active":"2024-09-16T09:25:05.713479+0000","last_became_peered":"2024-09-16T09:25:05.713479+0000","last_unstale":"2024-09-16T09:40:40.447825+0000","last_undegraded":"2024-09-16T09:40:40.447825+0000","last_fullsized":"2024-09-16T09:40:40.447825+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:40:39.969383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:40:39.969383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:57.516232+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":667,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.453094+0000","last_change":"2024-09-16T09:39:54.372966+0000","last_active":"2024-09-16T09:40:40.453094+0000","last_peered":"2024-09-16T09:40:40.453094+0000","last_clean":"2024-09-16T09:40:40.453094+0000","last_became_active":"2024-09-16T09:31:09.274797+0000","last_became_peered":"2024-09-16T09:31:09.274797+0000","last_unstale":"2024-09-16T09:40:40.453094+0000","last_undegraded":"2024-09-16T09:40:40.453094+0000","last_fullsized":"2024-09-16T09:40:40.453094+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:39:54.372758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:39:54.372758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:40:55.327792+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"285'8","reported_seq":570,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.447945+0000","last_change":"2024-09-16T09:38:34.024059+0000","last_active":"2024-09-16T09:40:40.447945+0000","last_peered":"2024-09-16T09:40:40.447945+0000","last_clean":"2024-09-16T09:40:40.447945+0000","last_became_active":"2024-09-16T09:31:09.273682+0000","last_became_peered":"2024-09-16T09:31:09.273682+0000","last_unstale":"2024-09-16T09:40:40.447945+0000","last_undegraded":"2024-09-16T09:40:40.447945+0000","last_fullsized":"2024-09-16T09:40:40.447945+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'8","last_scrub_stamp":"2024-09-16T09:38:34.023979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:38:34.023979+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":9,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":838,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.452733+0000","last_change":"2024-09-16T09:39:42.545066+0000","last_active":"2024-09-16T09:40:40.452733+0000","last_peered":"2024-09-16T09:40:40.452733+0000","last_clean":"2024-09-16T09:40:40.452733+0000","last_became_active":"2024-09-16T09:31:09.274719+0000","last_became_peered":"2024-09-16T09:31:09.274719+0000","last_unstale":"2024-09-16T09:40:40.452733+0000","last_undegraded":"2024-09-16T09:40:40.452733+0000","last_fullsized":"2024-09-16T09:40:40.452733+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:39:42.544998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:39:42.544998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:07.906091+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":824,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.445955+0000","last_change":"2024-09-16T09:38:35.092025+0000","last_active":"2024-09-16T09:40:40.445955+0000","last_peered":"2024-09-16T09:40:40.445955+0000","last_clean":"2024-09-16T09:40:40.445955+0000","last_became_active":"2024-09-16T09:25:05.712545+0000","last_became_peered":"2024-09-16T09:25:05.712545+0000","last_unstale":"2024-09-16T09:40:40.445955+0000","last_undegraded":"2024-09-16T09:40:40.445955+0000","last_fullsized":"2024-09-16T09:40:40.445955+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:38:35.091983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:38:35.091983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":802,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.447527+0000","last_change":"2024-09-16T09:37:50.502460+0000","last_active":"2024-09-16T09:40:40.447527+0000","last_peered":"2024-09-16T09:40:40.447527+0000","last_clean":"2024-09-16T09:40:40.447527+0000","last_became_active":"2024-09-16T09:25:07.720117+0000","last_became_peered":"2024-09-16T09:25:07.720117+0000","last_unstale":"2024-09-16T09:40:40.447527+0000","last_undegraded":"2024-09-16T09:40:40.447527+0000","last_fullsized":"2024-09-16T09:40:40.447527+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:50.502398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:37:50.502398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:19.948445+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"442'8806","reported_seq":15778,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:48.238776+0000","last_change":"2024-09-16T09:39:35.024562+0000","last_active":"2024-09-16T09:40:48.238776+0000","last_peered":"2024-09-16T09:40:48.238776+0000","last_clean":"2024-09-16T09:40:48.238776+0000","last_became_active":"2024-09-16T09:39:33.470653+0000","last_became_peered":"2024-09-16T09:39:33.470653+0000","last_unstale":"2024-09-16T09:40:48.238776+0000","last_undegraded":"2024-09-16T09:40:48.238776+0000","last_fullsized":"2024-09-16T09:40:48.238776+0000","mapping_epoch":420,"log_start":"391'4117","ondisk_log_start":"391'4117","created":395,"last_epoch_clean":421,"parent":"0.0","parent_split_bits":5,"last_scrub":"421'6014","last_scrub_stamp":"2024-09-16T09:39:35.024382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:37:44.378497+0000","last_clean_scrub_stamp":"2024-09-16T09:39:35.024382+0000","objects_scrubbed":3347,"log_size":4689,"log_dups_size":0,"ondisk_log_size":4689,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:40:57.641185+0000","scrub_duration":835,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35520512,"num_objects":542,"num_object_clones":0,"num_object_copies":1626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":542,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2805,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":439,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.450496+0000","last_change":"2024-09-16T09:36:52.091335+0000","last_active":"2024-09-16T09:40:40.450496+0000","last_peered":"2024-09-16T09:40:40.450496+0000","last_clean":"2024-09-16T09:40:40.450496+0000","last_became_active":"2024-09-16T09:33:30.248036+0000","last_became_peered":"2024-09-16T09:33:30.248036+0000","last_unstale":"2024-09-16T09:40:40.450496+0000","last_undegraded":"2024-09-16T09:40:40.450496+0000","last_fullsized":"2024-09-16T09:40:40.450496+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:52.088284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:36:52.088284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":665,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.455647+0000","last_change":"2024-09-16T09:37:18.825035+0000","last_active":"2024-09-16T09:40:40.455647+0000","last_peered":"2024-09-16T09:40:40.455647+0000","last_clean":"2024-09-16T09:40:40.455647+0000","last_became_active":"2024-09-16T09:31:09.271662+0000","last_became_peered":"2024-09-16T09:31:09.271662+0000","last_unstale":"2024-09-16T09:40:40.455647+0000","last_undegraded":"2024-09-16T09:40:40.455647+0000","last_fullsized":"2024-09-16T09:40:40.455647+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:18.824937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:37:18.824937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:11.218858+0000","scrub_duration":79,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"285'19","reported_seq":844,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.453201+0000","last_change":"2024-09-16T09:37:25.995663+0000","last_active":"2024-09-16T09:40:40.453201+0000","last_peered":"2024-09-16T09:40:40.453201+0000","last_clean":"2024-09-16T09:40:40.453201+0000","last_became_active":"2024-09-16T09:25:03.707769+0000","last_became_peered":"2024-09-16T09:25:03.707769+0000","last_unstale":"2024-09-16T09:40:40.453201+0000","last_undegraded":"2024-09-16T09:40:40.453201+0000","last_fullsized":"2024-09-16T09:40:40.453201+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'19","last_scrub_stamp":"2024-09-16T09:37:25.995607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:37:25.995607+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":801,"reported_epoch":442,"state":"active+clean","last_fresh":"2024-09-16T09:40:40.453141+0000","last_change":"2024-09-16T09:37:26.646784+0000","last_active":"2024-09-16T09:40:40.453141+0000","last_peered":"2024-09-16T09:40:40.453141+0000","last_clean":"2024-09-16T09:40:40.453141+0000","last_became_active":"2024-09-16T09:25:05.711952+0000","last_became_peered":"2024-09-16T09:25:05.711952+0000","last_unstale":"2024-09-16T09:40:40.453141+0000","last_undegraded":"2024-09-16T09:40:40.453141+0000","last_fullsized":"2024-09-16T09:40:40.453141+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:26.646686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:37:26.646686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:21.088482+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":26,"stat_sum":{"num_bytes":1253638168,"num_objects":19131,"num_object_clones":0,"num_object_copies":57393,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1620,"num_objects_unfound":0,"num_objects_dirty":19131,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":156556,"num_write_kb":4244737,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4655,"num_bytes_recovered":305070080,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4265160704,"data_stored":4265148488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119072,"ondisk_log_size":119072,"up":78,"acting":78,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":1117,"num_read_kb":874,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":137,"num_bytes_recovered":3576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":681,"ondisk_log_size":681,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1986,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":939,"num_read_kb":934,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":21,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":327680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1659,"ondisk_log_size":1659,"up":63,"acting":63,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266882,"num_pgs":69,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":998156,"kb_used_data":335292,"kb_used_omap":1,"kb_used_meta":662846,"kb_avail":92746996,"statfs":{"total":95995035648,"available":94972923904,"internally_reserved":0,"allocated":343339008,"data_stored":342723128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":678754762},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495815,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1191352,"kb_used_data":628392,"kb_used_omap":1,"kb_used_meta":562878,"kb_avail":92553800,"statfs":{"total":95995035648,"available":94775091200,"internally_reserved":0,"allocated":643473408,"data_stored":642876934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":576387526},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724750,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1173316,"kb_used_data":438260,"kb_used_omap":1,"kb_used_meta":735038,"kb_avail":92571836,"statfs":{"total":95995035648,"available":94793560064,"internally_reserved":0,"allocated":448778240,"data_stored":448170577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":752679366},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953683,"num_pgs":79,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1455056,"kb_used_data":819904,"kb_used_omap":1,"kb_used_meta":635070,"kb_avail":92290096,"statfs":{"total":95995035648,"available":94505058304,"internally_reserved":0,"allocated":839581696,"data_stored":838961698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":650312134},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149913,"num_pgs":74,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1133724,"kb_used_data":394508,"kb_used_omap":1,"kb_used_meta":739198,"kb_avail":92611428,"statfs":{"total":95995035648,"available":94834102272,"internally_reserved":0,"allocated":403976192,"data_stored":403344822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":756939207},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411550,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1056180,"kb_used_data":439140,"kb_used_omap":1,"kb_used_meta":617022,"kb_avail":92688972,"statfs":{"total":95995035648,"available":94913507328,"internally_reserved":0,"allocated":449679360,"data_stored":449087324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":631830981},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575077,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1242476,"kb_used_data":412892,"kb_used_omap":1,"kb_used_meta":829438,"kb_avail":92502676,"statfs":{"total":95995035648,"available":94722740224,"internally_reserved":0,"allocated":422801408,"data_stored":422152804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":849344965},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738603,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1584276,"kb_used_data":885316,"kb_used_omap":1,"kb_used_meta":698942,"kb_avail":92160876,"statfs":{"total":95995035648,"available":94372737024,"internally_reserved":0,"allocated":906563584,"data_stored":905939802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":715717061},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":879431680,"data_stored":879427608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":417398784,"data_stored":417398784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":429195264,"data_stored":429195264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":382205952,"data_stored":382205952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":785649664,"data_stored":785645592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":441778176,"data_stored":441778176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":596971520,"data_stored":596967448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":332529664,"data_stored":332529664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T09:40:53.695 INFO:tasks.thrashosds.thrasher:Setting 4.19 to [4, 7, 5] 2024-09-16T09:40:53.695 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '4.19', '4', '7', '5'] 2024-09-16T09:40:53.695 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd pg-upmap 4.19 4 7 5 2024-09-16T09:40:53.719 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:53.719 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:53.720 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:40:53.811 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:53 smithi084 ceph-mon[32629]: pgmap v833: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 1.2 GiB data, 9.4 GiB used, 706 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 772 op/s; 1620/57990 objects misplaced (2.794%) 2024-09-16T09:40:53.811 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:53 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:40:53.811 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:40:53 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:40:53.821 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:40:53.866 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:53.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:53.867 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:40:53.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:53 smithi070 ceph-mon[30388]: pgmap v833: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 1.2 GiB data, 9.4 GiB used, 706 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 772 op/s; 1620/57990 objects misplaced (2.794%) 2024-09-16T09:40:53.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:53 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:40:53.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:53 smithi070 ceph-mon[40691]: pgmap v833: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 1.2 GiB data, 9.4 GiB used, 706 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 772 op/s; 1620/57990 objects misplaced (2.794%) 2024-09-16T09:40:53.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:53 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:40:53.968 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:40:53.998 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:53.998 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:53.999 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:40:54.099 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:40:54.142 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:54.143 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:54.143 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:40:54.244 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:40:54.279 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:54.279 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:54.279 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:40:54.380 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:40:54.435 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:54.435 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:54.436 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:40:54.537 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:40:54.575 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:54.575 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:54.576 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:40:54.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:54 smithi070 ceph-mon[30388]: pgmap v834: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 844 MiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.36k op/s; 1620/41106 objects misplaced (3.941%) 2024-09-16T09:40:54.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:54 smithi070 ceph-mon[40691]: pgmap v834: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 844 MiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.36k op/s; 1620/41106 objects misplaced (3.941%) 2024-09-16T09:40:54.677 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:40:54.709 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:40:54.709 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:40:54.730 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:40:54.735 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:54.736 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:54.737 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:40:54.744 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:40:54.744 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:40:54.764 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:40:54.779 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:40:54.780 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:40:54.814 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:40:54.814 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:40:54.838 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:40:54.847 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:40:54.848 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:40:54.873 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:54.874 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:54.874 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:40:54.882 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:40:54.882 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:40:54.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:54 smithi084 ceph-mon[32629]: pgmap v834: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 844 MiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.36k op/s; 1620/41106 objects misplaced (3.941%) 2024-09-16T09:40:54.918 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:40:54.918 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:40:54.951 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:40:54.951 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:40:54.975 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:40:55.009 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:55.009 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:55.010 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:40:55.110 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:40:55.146 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:55.147 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:55.147 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:40:55.248 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:40:55.285 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:55.286 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:55.286 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:40:55.387 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:40:55.425 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:55.425 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:55.426 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:40:55.526 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:40:55.563 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:55.564 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:55.564 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:40:55.665 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:40:55.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:55 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2604348894' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:40:55.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:55 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:40:55.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:55 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:40:55.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:55 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/2604348894' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:40:55.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:55 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:40:55.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:55 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:40:55.702 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:55.702 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:55.703 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:40:55.804 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:40:55.841 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:55.842 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:55.842 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:40:55.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:55 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/2604348894' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:40:55.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:55 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:40:55.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:55 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:40:55.943 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:40:55.974 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:55.975 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:55.975 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:40:56.076 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:40:56.109 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:56.110 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:56.110 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:40:56.159 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd unset noscrub 2024-09-16T09:40:56.211 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:40:56.271 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:56.272 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:56.272 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:40:56.287 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T09:40:56.287 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 20, 2024-09-16T09:40:56.287 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 600, 2024-09-16T09:40:56.287 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [ 2024-09-16T09:40:56.287 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.287 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:85224 9.19 9:9bf6c0c6:::benchmark_data_smithi070_137000_object85223:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e266)", 2024-09-16T09:40:56.288 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:34:30.373671+0000", 2024-09-16T09:40:56.288 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 385.907928013, 2024-09-16T09:40:56.288 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0883978459999999, 2024-09-16T09:40:56.288 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:40:56.288 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:40:56.288 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T09:40:56.288 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:40:56.288 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:40:56.288 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:40:56.288 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 85224 2024-09-16T09:40:56.288 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.288 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:40:56.288 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.288 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:40:56.289 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.373671+0000", 2024-09-16T09:40:56.289 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:56.289 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.289 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.289 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:40:56.289 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.373671+0000", 2024-09-16T09:40:56.289 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:56.289 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.289 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.289 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:40:56.289 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.373673+0000", 2024-09-16T09:40:56.289 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.4600000000000002e-06 2024-09-16T09:40:56.289 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.290 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.290 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:40:56.290 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.373702+0000", 2024-09-16T09:40:56.290 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.8934000000000001e-05 2024-09-16T09:40:56.290 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.290 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.290 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:40:56.290 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.373725+0000", 2024-09-16T09:40:56.290 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3468999999999998e-05 2024-09-16T09:40:56.290 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.291 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.291 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:40:56.291 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.373737+0000", 2024-09-16T09:40:56.291 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1252e-05 2024-09-16T09:40:56.291 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.291 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.291 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:40:56.291 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.386012+0000", 2024-09-16T09:40:56.291 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.012275569 2024-09-16T09:40:56.291 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.291 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.291 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:40:56.292 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.386022+0000", 2024-09-16T09:40:56.292 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.2609999999999999e-06 2024-09-16T09:40:56.292 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.292 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.292 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:40:56.292 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.428618+0000", 2024-09-16T09:40:56.292 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0425969100000001 2024-09-16T09:40:56.292 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.292 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.292 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for missing object", 2024-09-16T09:40:56.292 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.428759+0000", 2024-09-16T09:40:56.292 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00014022899999999999 2024-09-16T09:40:56.292 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.293 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.293 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:40:56.293 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.432901+0000", 2024-09-16T09:40:56.293 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0041425020000000002 2024-09-16T09:40:56.293 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.293 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.293 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for degraded object", 2024-09-16T09:40:56.293 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.432957+0000", 2024-09-16T09:40:56.293 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5377999999999997e-05 2024-09-16T09:40:56.293 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.293 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.293 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:40:56.293 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.462010+0000", 2024-09-16T09:40:56.294 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.029052923000000001 2024-09-16T09:40:56.294 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.294 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.294 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:40:56.294 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.462068+0000", 2024-09-16T09:40:56.294 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8959000000000003e-05 2024-09-16T09:40:56.294 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:56.294 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:56.294 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:56.294 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.294 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.294 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:85228 9.19 9:98aa15be:::benchmark_data_smithi070_137000_object85227:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e266)", 2024-09-16T09:40:56.294 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:34:30.376655+0000", 2024-09-16T09:40:56.294 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 385.90494325700001, 2024-09-16T09:40:56.295 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0565689490000001, 2024-09-16T09:40:56.295 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:40:56.295 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:40:56.295 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:40:56.295 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:40:56.295 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:40:56.295 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:40:56.295 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 85228 2024-09-16T09:40:56.295 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.295 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:40:56.295 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.295 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:40:56.295 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.376655+0000", 2024-09-16T09:40:56.295 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:56.295 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.296 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.296 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:40:56.296 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.376655+0000", 2024-09-16T09:40:56.296 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:56.296 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.296 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.296 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:40:56.296 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.376657+0000", 2024-09-16T09:40:56.296 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.4419999999999999e-06 2024-09-16T09:40:56.296 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.296 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.296 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:40:56.296 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.376680+0000", 2024-09-16T09:40:56.296 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.2923000000000001e-05 2024-09-16T09:40:56.297 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.297 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.297 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:40:56.297 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.376692+0000", 2024-09-16T09:40:56.297 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1861999999999999e-05 2024-09-16T09:40:56.297 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.297 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.297 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:40:56.297 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.376701+0000", 2024-09-16T09:40:56.297 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.0080000000000008e-06 2024-09-16T09:40:56.297 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.297 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.297 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:40:56.297 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.386042+0000", 2024-09-16T09:40:56.298 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0093412960000000007 2024-09-16T09:40:56.298 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.298 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.298 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:40:56.298 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.386048+0000", 2024-09-16T09:40:56.298 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.9170000000000003e-06 2024-09-16T09:40:56.298 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.298 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.298 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:40:56.298 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.429089+0000", 2024-09-16T09:40:56.298 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0430408870000001 2024-09-16T09:40:56.298 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.298 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.298 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:40:56.299 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.429187+0000", 2024-09-16T09:40:56.299 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.8227999999999995e-05 2024-09-16T09:40:56.299 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.299 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.299 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 5,6", 2024-09-16T09:40:56.299 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.429456+0000", 2024-09-16T09:40:56.299 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00026942699999999998 2024-09-16T09:40:56.299 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.299 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.299 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:40:56.299 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.433028+0000", 2024-09-16T09:40:56.299 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0035718199999999999 2024-09-16T09:40:56.299 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.299 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.300 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:40:56.300 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.433093+0000", 2024-09-16T09:40:56.300 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.4393000000000005e-05 2024-09-16T09:40:56.300 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.300 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.300 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:40:56.301 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.433126+0000", 2024-09-16T09:40:56.301 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3083000000000002e-05 2024-09-16T09:40:56.301 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.301 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.301 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:40:56.301 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.433160+0000", 2024-09-16T09:40:56.301 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4746999999999998e-05 2024-09-16T09:40:56.302 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.302 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.302 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:40:56.302 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.433224+0000", 2024-09-16T09:40:56.302 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.3916000000000003e-05 2024-09-16T09:40:56.302 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:56.302 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:56.302 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:56.302 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.303 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.303 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:85232 9.19 9:9ebfcfc9:::benchmark_data_smithi070_137000_object85231:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e266)", 2024-09-16T09:40:56.303 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:34:30.381300+0000", 2024-09-16T09:40:56.303 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 385.90029894600002, 2024-09-16T09:40:56.303 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0521562879999999, 2024-09-16T09:40:56.303 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:40:56.303 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:40:56.303 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:40:56.303 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:40:56.303 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:40:56.303 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:40:56.304 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 85232 2024-09-16T09:40:56.304 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.304 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:40:56.304 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.304 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:40:56.304 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.381300+0000", 2024-09-16T09:40:56.304 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:56.304 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.305 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.305 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:40:56.305 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.381300+0000", 2024-09-16T09:40:56.305 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:56.305 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.305 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.305 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:40:56.305 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.381301+0000", 2024-09-16T09:40:56.305 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6959999999999999e-06 2024-09-16T09:40:56.305 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.305 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.306 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:40:56.306 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.381325+0000", 2024-09-16T09:40:56.306 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3213999999999998e-05 2024-09-16T09:40:56.306 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.306 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.306 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:40:56.306 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.381338+0000", 2024-09-16T09:40:56.306 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3237999999999999e-05 2024-09-16T09:40:56.306 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.306 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.306 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:40:56.306 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.381346+0000", 2024-09-16T09:40:56.306 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.8390000000000007e-06 2024-09-16T09:40:56.306 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.307 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.307 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:40:56.307 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.386067+0000", 2024-09-16T09:40:56.307 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0047214400000000004 2024-09-16T09:40:56.307 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.307 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.307 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:40:56.307 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.386073+0000", 2024-09-16T09:40:56.307 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5790000000000003e-06 2024-09-16T09:40:56.307 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.307 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.307 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:40:56.307 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.430684+0000", 2024-09-16T09:40:56.307 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.044610824 2024-09-16T09:40:56.308 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.308 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.308 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:40:56.308 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.430767+0000", 2024-09-16T09:40:56.308 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.3483e-05 2024-09-16T09:40:56.308 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.308 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.308 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 5,6", 2024-09-16T09:40:56.308 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.430988+0000", 2024-09-16T09:40:56.308 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00022084900000000001 2024-09-16T09:40:56.308 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.308 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.308 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:40:56.308 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.433032+0000", 2024-09-16T09:40:56.309 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0020444159999999999 2024-09-16T09:40:56.309 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.309 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.309 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:40:56.309 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.433282+0000", 2024-09-16T09:40:56.309 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000250024 2024-09-16T09:40:56.309 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.309 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.309 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:40:56.309 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.433335+0000", 2024-09-16T09:40:56.309 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.3096999999999997e-05 2024-09-16T09:40:56.309 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.309 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.309 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:40:56.310 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.433370+0000", 2024-09-16T09:40:56.310 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4465000000000003e-05 2024-09-16T09:40:56.310 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.310 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.310 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:40:56.310 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.433456+0000", 2024-09-16T09:40:56.310 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.6124000000000004e-05 2024-09-16T09:40:56.310 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:56.310 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:56.310 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:56.310 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.310 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.310 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:85266 9.19 9:9983710d:::benchmark_data_smithi070_137000_object85265:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e266)", 2024-09-16T09:40:56.310 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:34:30.409167+0000", 2024-09-16T09:40:56.311 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 385.87243114699999, 2024-09-16T09:40:56.311 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.029484174, 2024-09-16T09:40:56.311 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:40:56.311 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:40:56.311 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:40:56.311 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:40:56.311 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:40:56.311 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:40:56.311 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 85266 2024-09-16T09:40:56.311 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.311 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:40:56.311 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.311 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:40:56.311 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.409167+0000", 2024-09-16T09:40:56.312 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:56.312 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.312 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.312 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:40:56.312 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.409167+0000", 2024-09-16T09:40:56.312 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:56.312 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.312 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.312 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:40:56.312 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.409169+0000", 2024-09-16T09:40:56.312 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6339999999999999e-06 2024-09-16T09:40:56.312 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.312 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.312 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:40:56.313 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.409201+0000", 2024-09-16T09:40:56.313 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2193000000000002e-05 2024-09-16T09:40:56.313 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.313 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.313 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:40:56.313 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.409218+0000", 2024-09-16T09:40:56.313 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6481e-05 2024-09-16T09:40:56.313 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.313 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.313 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:40:56.313 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.409225+0000", 2024-09-16T09:40:56.313 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6469999999999998e-06 2024-09-16T09:40:56.313 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.314 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.314 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:40:56.314 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.410186+0000", 2024-09-16T09:40:56.314 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000960673 2024-09-16T09:40:56.314 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.314 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.314 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:40:56.314 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.410198+0000", 2024-09-16T09:40:56.314 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1993e-05 2024-09-16T09:40:56.314 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.314 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.314 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:40:56.314 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.433567+0000", 2024-09-16T09:40:56.314 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.023368582 2024-09-16T09:40:56.314 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.315 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.315 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:40:56.315 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.433672+0000", 2024-09-16T09:40:56.315 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000105046 2024-09-16T09:40:56.315 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.315 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.315 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 5,6", 2024-09-16T09:40:56.315 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.433833+0000", 2024-09-16T09:40:56.315 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00016096599999999999 2024-09-16T09:40:56.316 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.316 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.316 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:40:56.316 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.438355+0000", 2024-09-16T09:40:56.316 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.004522206 2024-09-16T09:40:56.316 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.316 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.316 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:40:56.316 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.438417+0000", 2024-09-16T09:40:56.316 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1888000000000001e-05 2024-09-16T09:40:56.316 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.316 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.317 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:40:56.317 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.438515+0000", 2024-09-16T09:40:56.317 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.8084999999999993e-05 2024-09-16T09:40:56.317 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.317 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.317 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:40:56.317 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.438550+0000", 2024-09-16T09:40:56.317 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5172000000000001e-05 2024-09-16T09:40:56.317 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.317 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.318 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:40:56.318 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.438652+0000", 2024-09-16T09:40:56.318 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00010160799999999999 2024-09-16T09:40:56.318 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:56.318 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:56.318 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:56.318 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.319 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.319 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:211248 9.12 9:499944f5:::benchmark_data_smithi070_137000_object89537:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e327)", 2024-09-16T09:40:56.319 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:36:47.708810+0000", 2024-09-16T09:40:56.319 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 248.57278844000001, 2024-09-16T09:40:56.319 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0511326670000001, 2024-09-16T09:40:56.319 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:40:56.319 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:40:56.319 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:40:56.319 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:40:56.319 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:40:56.319 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:40:56.319 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 211248 2024-09-16T09:40:56.320 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.320 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:40:56.320 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.320 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:40:56.320 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.708810+0000", 2024-09-16T09:40:56.320 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:56.320 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.320 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.321 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:40:56.321 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.708810+0000", 2024-09-16T09:40:56.321 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:56.321 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.321 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.322 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:40:56.322 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.708812+0000", 2024-09-16T09:40:56.322 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.4780000000000001e-06 2024-09-16T09:40:56.322 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.322 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.322 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:40:56.322 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.708816+0000", 2024-09-16T09:40:56.323 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.7090000000000001e-06 2024-09-16T09:40:56.323 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.323 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.323 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:40:56.323 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.708827+0000", 2024-09-16T09:40:56.323 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0274e-05 2024-09-16T09:40:56.323 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.323 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.323 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:40:56.323 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.708833+0000", 2024-09-16T09:40:56.323 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.7660000000000001e-06 2024-09-16T09:40:56.323 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.323 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.323 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:40:56.323 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.711366+0000", 2024-09-16T09:40:56.323 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0025325529999999999 2024-09-16T09:40:56.324 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.324 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.324 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:40:56.324 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.711385+0000", 2024-09-16T09:40:56.324 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9368000000000002e-05 2024-09-16T09:40:56.324 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.324 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.324 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:40:56.324 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.748172+0000", 2024-09-16T09:40:56.324 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0367871070000001 2024-09-16T09:40:56.324 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.325 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.325 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:40:56.325 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.748254+0000", 2024-09-16T09:40:56.325 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.1648000000000002e-05 2024-09-16T09:40:56.325 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.325 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.325 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,2", 2024-09-16T09:40:56.325 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.748561+0000", 2024-09-16T09:40:56.325 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00030718299999999999 2024-09-16T09:40:56.325 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.325 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.325 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:40:56.325 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.755820+0000", 2024-09-16T09:40:56.326 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0072592109999999998 2024-09-16T09:40:56.326 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.326 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.326 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:40:56.326 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.759723+0000", 2024-09-16T09:40:56.326 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0039020249999999999 2024-09-16T09:40:56.326 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.326 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.326 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:40:56.326 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.759846+0000", 2024-09-16T09:40:56.326 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00012310000000000001 2024-09-16T09:40:56.326 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.326 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.326 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:40:56.327 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.759901+0000", 2024-09-16T09:40:56.327 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5315000000000001e-05 2024-09-16T09:40:56.327 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.327 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.327 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:40:56.327 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.759943+0000", 2024-09-16T09:40:56.327 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.193e-05 2024-09-16T09:40:56.327 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:56.327 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:56.327 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:56.327 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.327 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.327 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:211251 9.12 9:49db1cb8:::benchmark_data_smithi070_137000_object89540:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e327)", 2024-09-16T09:40:56.328 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:36:47.713400+0000", 2024-09-16T09:40:56.328 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 248.56819846499999, 2024-09-16T09:40:56.328 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0484086290000001, 2024-09-16T09:40:56.328 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:40:56.328 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:40:56.328 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:40:56.328 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:40:56.328 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:40:56.328 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:40:56.328 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 211251 2024-09-16T09:40:56.329 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.329 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:40:56.329 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.329 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:40:56.329 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.713400+0000", 2024-09-16T09:40:56.329 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:56.329 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.329 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.329 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:40:56.329 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.713400+0000", 2024-09-16T09:40:56.329 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:56.329 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.330 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.330 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:40:56.330 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.713402+0000", 2024-09-16T09:40:56.330 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7439999999999999e-06 2024-09-16T09:40:56.330 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.330 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.331 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:40:56.331 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.713407+0000", 2024-09-16T09:40:56.331 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4260000000000002e-06 2024-09-16T09:40:56.331 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.331 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.331 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:40:56.331 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.713423+0000", 2024-09-16T09:40:56.331 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5974999999999999e-05 2024-09-16T09:40:56.331 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.331 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.331 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:40:56.332 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.713431+0000", 2024-09-16T09:40:56.332 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.5920000000000003e-06 2024-09-16T09:40:56.332 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.332 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.332 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:40:56.332 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.713501+0000", 2024-09-16T09:40:56.332 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.0025999999999997e-05 2024-09-16T09:40:56.332 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.332 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.332 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:40:56.332 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.713510+0000", 2024-09-16T09:40:56.332 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.5089999999999999e-06 2024-09-16T09:40:56.332 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.333 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.333 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:40:56.333 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.755869+0000", 2024-09-16T09:40:56.333 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0423587969999999 2024-09-16T09:40:56.333 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.333 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.333 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:40:56.333 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.755973+0000", 2024-09-16T09:40:56.333 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000103688 2024-09-16T09:40:56.333 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.333 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.333 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,2", 2024-09-16T09:40:56.334 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.756051+0000", 2024-09-16T09:40:56.334 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.8528000000000004e-05 2024-09-16T09:40:56.334 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.334 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.334 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:40:56.334 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.759988+0000", 2024-09-16T09:40:56.334 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0039360829999999999 2024-09-16T09:40:56.334 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.334 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.334 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:40:56.334 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.760034+0000", 2024-09-16T09:40:56.334 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.6269e-05 2024-09-16T09:40:56.334 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.334 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.334 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:40:56.334 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.761690+0000", 2024-09-16T09:40:56.335 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0016558860000000001 2024-09-16T09:40:56.335 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.335 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.335 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:40:56.335 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.761768+0000", 2024-09-16T09:40:56.335 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.8317000000000002e-05 2024-09-16T09:40:56.335 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.335 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.335 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:40:56.335 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.761809+0000", 2024-09-16T09:40:56.335 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0788999999999999e-05 2024-09-16T09:40:56.335 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:56.335 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:56.335 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:56.336 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.336 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.336 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:211260 9.12 9:4be11eeb:::benchmark_data_smithi070_137000_object89549:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e327)", 2024-09-16T09:40:56.336 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:36:47.713450+0000", 2024-09-16T09:40:56.336 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 248.568148127, 2024-09-16T09:40:56.336 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0500658940000001, 2024-09-16T09:40:56.336 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:40:56.336 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:40:56.336 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:40:56.336 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:40:56.336 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:40:56.336 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:40:56.336 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 211260 2024-09-16T09:40:56.336 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.337 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:40:56.337 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.337 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:40:56.337 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.713450+0000", 2024-09-16T09:40:56.337 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:56.337 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.337 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.337 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:40:56.337 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.713450+0000", 2024-09-16T09:40:56.337 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:56.337 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.337 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.337 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:40:56.338 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.713451+0000", 2024-09-16T09:40:56.338 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.3000000000000001e-07 2024-09-16T09:40:56.338 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.338 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.338 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:40:56.338 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.713453+0000", 2024-09-16T09:40:56.338 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.4650000000000001e-06 2024-09-16T09:40:56.338 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.338 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.338 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:40:56.338 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.713464+0000", 2024-09-16T09:40:56.338 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0682999999999999e-05 2024-09-16T09:40:56.338 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.338 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.338 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:40:56.339 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.713469+0000", 2024-09-16T09:40:56.339 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.8749999999999999e-06 2024-09-16T09:40:56.339 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.339 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.339 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:40:56.339 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.714719+0000", 2024-09-16T09:40:56.339 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001249586 2024-09-16T09:40:56.339 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.339 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.339 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:40:56.339 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.714728+0000", 2024-09-16T09:40:56.339 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.2599999999999994e-06 2024-09-16T09:40:56.339 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.339 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.340 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:40:56.340 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.757394+0000", 2024-09-16T09:40:56.340 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0426663780000001 2024-09-16T09:40:56.340 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.340 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.340 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:40:56.340 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.757502+0000", 2024-09-16T09:40:56.340 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000107833 2024-09-16T09:40:56.340 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.340 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.340 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,2", 2024-09-16T09:40:56.340 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.757619+0000", 2024-09-16T09:40:56.340 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000117453 2024-09-16T09:40:56.341 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.341 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.341 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:40:56.341 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.761817+0000", 2024-09-16T09:40:56.341 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0041977969999999996 2024-09-16T09:40:56.341 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.341 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.341 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:40:56.341 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.763289+0000", 2024-09-16T09:40:56.341 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0014714890000000001 2024-09-16T09:40:56.342 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.342 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.342 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:40:56.342 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.763422+0000", 2024-09-16T09:40:56.342 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00013334600000000001 2024-09-16T09:40:56.342 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.342 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.342 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:40:56.342 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.763477+0000", 2024-09-16T09:40:56.342 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5334000000000002e-05 2024-09-16T09:40:56.342 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.342 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.342 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:40:56.343 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.763516+0000", 2024-09-16T09:40:56.343 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8964999999999997e-05 2024-09-16T09:40:56.343 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:56.343 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:56.343 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:56.343 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.343 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.343 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:211279 9.12 9:4baade64:::benchmark_data_smithi070_137000_object89568:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e327)", 2024-09-16T09:40:56.343 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:36:47.728918+0000", 2024-09-16T09:40:56.343 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 248.552680925, 2024-09-16T09:40:56.343 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0347446069999999, 2024-09-16T09:40:56.343 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:40:56.343 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:40:56.343 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:40:56.344 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:40:56.344 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:40:56.344 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:40:56.344 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 211279 2024-09-16T09:40:56.344 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.344 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:40:56.344 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.344 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:40:56.344 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.728918+0000", 2024-09-16T09:40:56.344 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:56.344 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.344 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.344 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:40:56.344 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.728918+0000", 2024-09-16T09:40:56.345 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:56.345 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.345 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.345 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:40:56.345 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.728919+0000", 2024-09-16T09:40:56.345 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1620000000000001e-06 2024-09-16T09:40:56.345 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.345 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.346 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:40:56.346 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.728923+0000", 2024-09-16T09:40:56.346 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0609999999999997e-06 2024-09-16T09:40:56.346 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.346 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.346 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:40:56.346 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.728936+0000", 2024-09-16T09:40:56.346 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2891000000000001e-05 2024-09-16T09:40:56.346 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.346 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.346 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:40:56.346 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.728943+0000", 2024-09-16T09:40:56.346 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.9650000000000002e-06 2024-09-16T09:40:56.347 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.347 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.347 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:40:56.347 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.729002+0000", 2024-09-16T09:40:56.347 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.9392999999999999e-05 2024-09-16T09:40:56.347 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.347 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.347 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:40:56.347 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.729022+0000", 2024-09-16T09:40:56.347 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9752999999999999e-05 2024-09-16T09:40:56.347 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.347 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.347 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:40:56.347 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.758680+0000", 2024-09-16T09:40:56.348 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.029658484 2024-09-16T09:40:56.348 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.348 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.348 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:40:56.348 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.758761+0000", 2024-09-16T09:40:56.348 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.0319000000000001e-05 2024-09-16T09:40:56.348 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.348 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.348 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,2", 2024-09-16T09:40:56.348 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.758823+0000", 2024-09-16T09:40:56.348 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.2467999999999999e-05 2024-09-16T09:40:56.348 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.348 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.349 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:40:56.349 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.763303+0000", 2024-09-16T09:40:56.349 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0044794739999999998 2024-09-16T09:40:56.349 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.349 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.349 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:40:56.349 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.763362+0000", 2024-09-16T09:40:56.349 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.9268999999999998e-05 2024-09-16T09:40:56.349 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.349 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.349 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:40:56.349 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.763574+0000", 2024-09-16T09:40:56.349 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00021250800000000001 2024-09-16T09:40:56.350 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.350 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.350 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:40:56.350 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.763623+0000", 2024-09-16T09:40:56.350 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.8494e-05 2024-09-16T09:40:56.350 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.350 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.350 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:40:56.350 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.763662+0000", 2024-09-16T09:40:56.350 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9366000000000002e-05 2024-09-16T09:40:56.350 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:56.350 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:56.350 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:56.351 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.351 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.351 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:211280 9.12 9:4ff46297:::benchmark_data_smithi070_137000_object89569:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e327)", 2024-09-16T09:40:56.351 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:36:47.730052+0000", 2024-09-16T09:40:56.351 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 248.55154646599999, 2024-09-16T09:40:56.351 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.03381397, 2024-09-16T09:40:56.351 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:40:56.351 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:40:56.351 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:40:56.351 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:40:56.351 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:40:56.351 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:40:56.351 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 211280 2024-09-16T09:40:56.351 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.352 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:40:56.352 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.352 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:40:56.352 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.730052+0000", 2024-09-16T09:40:56.352 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:56.352 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.352 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.352 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:40:56.352 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.730052+0000", 2024-09-16T09:40:56.352 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:56.352 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.352 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.352 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:40:56.353 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.730053+0000", 2024-09-16T09:40:56.353 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.108e-06 2024-09-16T09:40:56.353 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.353 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.353 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:40:56.353 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.730058+0000", 2024-09-16T09:40:56.353 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.4039999999999997e-06 2024-09-16T09:40:56.353 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.353 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.353 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:40:56.353 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.730109+0000", 2024-09-16T09:40:56.353 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.1700000000000003e-05 2024-09-16T09:40:56.353 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.353 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.354 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:40:56.354 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.730141+0000", 2024-09-16T09:40:56.354 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1483000000000003e-05 2024-09-16T09:40:56.354 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.354 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.354 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:40:56.354 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.730225+0000", 2024-09-16T09:40:56.354 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.3858999999999998e-05 2024-09-16T09:40:56.354 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.354 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.354 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:40:56.354 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.730233+0000", 2024-09-16T09:40:56.354 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.4030000000000007e-06 2024-09-16T09:40:56.355 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.355 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.355 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:40:56.355 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.760090+0000", 2024-09-16T09:40:56.355 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0298566229999999 2024-09-16T09:40:56.355 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.355 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.355 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:40:56.355 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.760195+0000", 2024-09-16T09:40:56.355 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000105503 2024-09-16T09:40:56.355 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.355 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.355 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,2", 2024-09-16T09:40:56.355 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.760353+0000", 2024-09-16T09:40:56.356 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00015804 2024-09-16T09:40:56.356 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.356 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.356 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:40:56.356 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.763526+0000", 2024-09-16T09:40:56.356 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0031731810000000002 2024-09-16T09:40:56.356 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.356 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.356 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:40:56.356 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.763723+0000", 2024-09-16T09:40:56.356 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00019647100000000001 2024-09-16T09:40:56.356 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.356 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.356 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:40:56.357 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.763770+0000", 2024-09-16T09:40:56.357 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.7450999999999997e-05 2024-09-16T09:40:56.357 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.357 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.357 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:40:56.357 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.763820+0000", 2024-09-16T09:40:56.357 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.9749000000000001e-05 2024-09-16T09:40:56.357 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.357 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.357 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:40:56.357 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.763866+0000", 2024-09-16T09:40:56.357 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5995000000000003e-05 2024-09-16T09:40:56.357 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:56.357 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:56.357 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:56.358 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.358 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.358 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:211328 9.12 9:487331f4:::benchmark_data_smithi070_137000_object89617:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e327)", 2024-09-16T09:40:56.358 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:36:47.762277+0000", 2024-09-16T09:40:56.358 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 248.519321187, 2024-09-16T09:40:56.358 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.007375701, 2024-09-16T09:40:56.358 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:40:56.358 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:40:56.358 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:40:56.358 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:40:56.358 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:40:56.358 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:40:56.358 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 211328 2024-09-16T09:40:56.358 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.359 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:40:56.359 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.359 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:40:56.359 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.762277+0000", 2024-09-16T09:40:56.359 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:56.359 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.359 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.359 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:40:56.359 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.762277+0000", 2024-09-16T09:40:56.359 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:56.359 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.359 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.359 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:40:56.359 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.762278+0000", 2024-09-16T09:40:56.360 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.9199999999999999e-07 2024-09-16T09:40:56.360 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.360 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.360 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:40:56.360 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.762283+0000", 2024-09-16T09:40:56.360 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.7190000000000001e-06 2024-09-16T09:40:56.360 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.360 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.360 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:40:56.360 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.762295+0000", 2024-09-16T09:40:56.360 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2009e-05 2024-09-16T09:40:56.360 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.360 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.361 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:40:56.361 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.762306+0000", 2024-09-16T09:40:56.361 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0730000000000001e-05 2024-09-16T09:40:56.361 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.361 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.361 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:40:56.361 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.762359+0000", 2024-09-16T09:40:56.361 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.3554999999999998e-05 2024-09-16T09:40:56.361 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.361 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.361 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:40:56.361 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.762370+0000", 2024-09-16T09:40:56.361 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0219e-05 2024-09-16T09:40:56.362 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.362 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.362 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:40:56.362 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.764182+0000", 2024-09-16T09:40:56.362 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.001812323 2024-09-16T09:40:56.362 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.362 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.362 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:40:56.362 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.764285+0000", 2024-09-16T09:40:56.362 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000102872 2024-09-16T09:40:56.362 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.362 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.362 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,2", 2024-09-16T09:40:56.362 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.764397+0000", 2024-09-16T09:40:56.362 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000111922 2024-09-16T09:40:56.363 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.363 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.363 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:40:56.363 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.769453+0000", 2024-09-16T09:40:56.363 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0050566739999999997 2024-09-16T09:40:56.363 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.363 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.363 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:40:56.363 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.769506+0000", 2024-09-16T09:40:56.364 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.2123999999999998e-05 2024-09-16T09:40:56.365 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.365 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.365 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:40:56.365 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.769526+0000", 2024-09-16T09:40:56.365 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0806e-05 2024-09-16T09:40:56.365 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.365 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.365 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:40:56.365 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.769612+0000", 2024-09-16T09:40:56.365 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.5620999999999999e-05 2024-09-16T09:40:56.366 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.366 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.366 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:40:56.366 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.769653+0000", 2024-09-16T09:40:56.366 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.1134999999999998e-05 2024-09-16T09:40:56.366 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:56.366 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:56.366 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:56.366 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.366 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.366 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:211329 9.12 9:4b744b27:::benchmark_data_smithi070_137000_object89618:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e327)", 2024-09-16T09:40:56.366 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:36:47.764514+0000", 2024-09-16T09:40:56.366 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 248.51708447499999, 2024-09-16T09:40:56.367 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0096066159999999, 2024-09-16T09:40:56.367 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:40:56.367 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:40:56.367 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:40:56.367 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:40:56.367 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:40:56.367 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:40:56.367 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 211329 2024-09-16T09:40:56.367 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.367 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:40:56.367 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.367 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:40:56.367 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.764514+0000", 2024-09-16T09:40:56.367 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:56.367 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.368 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.368 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:40:56.368 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.764514+0000", 2024-09-16T09:40:56.368 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:56.368 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.368 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.368 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:40:56.368 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.764516+0000", 2024-09-16T09:40:56.368 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.4100000000000001e-06 2024-09-16T09:40:56.368 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.368 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.368 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:40:56.368 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.764521+0000", 2024-09-16T09:40:56.368 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.2599999999999996e-06 2024-09-16T09:40:56.369 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.369 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.369 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:40:56.369 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.764531+0000", 2024-09-16T09:40:56.369 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0322e-05 2024-09-16T09:40:56.369 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.369 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.369 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:40:56.369 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.764538+0000", 2024-09-16T09:40:56.369 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.0600000000000002e-06 2024-09-16T09:40:56.369 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.369 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.369 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:40:56.369 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.764618+0000", 2024-09-16T09:40:56.370 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.0177000000000002e-05 2024-09-16T09:40:56.370 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.370 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.370 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:40:56.370 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.764628+0000", 2024-09-16T09:40:56.370 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.5400000000000001e-06 2024-09-16T09:40:56.370 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.370 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.370 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:40:56.370 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.767496+0000", 2024-09-16T09:40:56.370 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0028676009999999 2024-09-16T09:40:56.370 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.371 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.371 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:40:56.371 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.767619+0000", 2024-09-16T09:40:56.371 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00012315699999999999 2024-09-16T09:40:56.371 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.371 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.371 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,2", 2024-09-16T09:40:56.371 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.767750+0000", 2024-09-16T09:40:56.371 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00013136499999999999 2024-09-16T09:40:56.371 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.371 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.371 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:40:56.371 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.773939+0000", 2024-09-16T09:40:56.371 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0061884619999999996 2024-09-16T09:40:56.372 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.372 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.372 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:40:56.372 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.773955+0000", 2024-09-16T09:40:56.372 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6376999999999999e-05 2024-09-16T09:40:56.372 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.372 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.372 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:40:56.372 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.774033+0000", 2024-09-16T09:40:56.372 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.8088000000000002e-05 2024-09-16T09:40:56.372 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.372 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.372 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:40:56.372 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.774082+0000", 2024-09-16T09:40:56.372 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.9354999999999998e-05 2024-09-16T09:40:56.373 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.373 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.373 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:40:56.373 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.774121+0000", 2024-09-16T09:40:56.373 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8442000000000002e-05 2024-09-16T09:40:56.373 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:56.373 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:56.373 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:56.373 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.373 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.374 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:220422 9.2 9.55f30b02 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e344)", 2024-09-16T09:40:56.374 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:37:04.372312+0000", 2024-09-16T09:40:56.374 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 231.90928700699999, 2024-09-16T09:40:56.374 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0049641709999999, 2024-09-16T09:40:56.374 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:40:56.374 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:40:56.374 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T09:40:56.374 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:40:56.374 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:40:56.374 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:40:56.374 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 220422 2024-09-16T09:40:56.374 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.374 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:40:56.374 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.375 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:40:56.375 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:04.372312+0000", 2024-09-16T09:40:56.375 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:56.375 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.375 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.375 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:40:56.375 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:04.372312+0000", 2024-09-16T09:40:56.375 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:56.375 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.375 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.375 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:40:56.375 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:04.372313+0000", 2024-09-16T09:40:56.375 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6610000000000001e-06 2024-09-16T09:40:56.375 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.375 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.376 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:40:56.376 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:04.372318+0000", 2024-09-16T09:40:56.376 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.9830000000000004e-06 2024-09-16T09:40:56.376 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.376 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.376 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:40:56.376 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:04.372333+0000", 2024-09-16T09:40:56.376 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.4544e-05 2024-09-16T09:40:56.376 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.376 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.376 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:40:56.376 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:04.372341+0000", 2024-09-16T09:40:56.376 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.8650000000000001e-06 2024-09-16T09:40:56.376 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.377 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.377 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:40:56.377 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:04.372416+0000", 2024-09-16T09:40:56.377 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.4883999999999997e-05 2024-09-16T09:40:56.377 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.377 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.377 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:40:56.377 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:04.372426+0000", 2024-09-16T09:40:56.377 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.078e-05 2024-09-16T09:40:56.377 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.377 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.377 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:40:56.377 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:05.377221+0000", 2024-09-16T09:40:56.377 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.004794207 2024-09-16T09:40:56.378 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.378 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.378 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:40:56.378 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:05.377276+0000", 2024-09-16T09:40:56.378 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5247000000000001e-05 2024-09-16T09:40:56.378 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:56.378 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:56.378 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:56.378 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.378 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.378 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:220449 9.2 9:4d2f3ff5:::benchmark_data_smithi070_137000_object98738:head [delete] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e346)", 2024-09-16T09:40:56.378 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:37:06.366595+0000", 2024-09-16T09:40:56.378 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 229.91500365100001, 2024-09-16T09:40:56.378 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.005813316, 2024-09-16T09:40:56.378 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:40:56.378 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:40:56.379 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:40:56.379 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:40:56.379 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:40:56.379 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:40:56.379 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 220449 2024-09-16T09:40:56.379 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.379 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:40:56.379 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.379 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:40:56.379 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366595+0000", 2024-09-16T09:40:56.379 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:56.379 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.380 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.380 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:40:56.380 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366595+0000", 2024-09-16T09:40:56.380 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:56.380 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.380 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.380 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:40:56.380 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366596+0000", 2024-09-16T09:40:56.381 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.054e-06 2024-09-16T09:40:56.381 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.381 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.381 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:40:56.381 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366601+0000", 2024-09-16T09:40:56.381 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.7400000000000004e-06 2024-09-16T09:40:56.381 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.381 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.381 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:40:56.381 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366660+0000", 2024-09-16T09:40:56.381 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.9509999999999998e-05 2024-09-16T09:40:56.381 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.381 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.381 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:40:56.382 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366669+0000", 2024-09-16T09:40:56.382 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.0019999999999995e-06 2024-09-16T09:40:56.382 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.382 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.382 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:40:56.382 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.460193+0000", 2024-09-16T09:40:56.382 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.093523464000000001 2024-09-16T09:40:56.382 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.382 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.382 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:40:56.382 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.460206+0000", 2024-09-16T09:40:56.382 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3113e-05 2024-09-16T09:40:56.382 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.382 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.382 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:40:56.383 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.371051+0000", 2024-09-16T09:40:56.383 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.91084521600000001 2024-09-16T09:40:56.383 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.383 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.383 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:40:56.383 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.371151+0000", 2024-09-16T09:40:56.383 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.9918000000000001e-05 2024-09-16T09:40:56.383 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.383 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.383 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,2", 2024-09-16T09:40:56.383 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.371223+0000", 2024-09-16T09:40:56.383 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.2237000000000007e-05 2024-09-16T09:40:56.383 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.383 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.384 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:40:56.384 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.372257+0000", 2024-09-16T09:40:56.384 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001033422 2024-09-16T09:40:56.384 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.384 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.384 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:40:56.384 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.372276+0000", 2024-09-16T09:40:56.384 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9151999999999999e-05 2024-09-16T09:40:56.384 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.384 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.384 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:40:56.384 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.372326+0000", 2024-09-16T09:40:56.384 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0175e-05 2024-09-16T09:40:56.384 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.384 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.385 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:40:56.385 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.372384+0000", 2024-09-16T09:40:56.385 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7680999999999999e-05 2024-09-16T09:40:56.385 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.385 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.385 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:40:56.385 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.372408+0000", 2024-09-16T09:40:56.385 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.4632000000000001e-05 2024-09-16T09:40:56.385 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:56.385 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:56.385 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:56.385 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.386 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.386 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:220479 9.2 9:4adb387c:::benchmark_data_smithi070_137000_object98768:head [delete] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e346)", 2024-09-16T09:40:56.386 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:37:06.366688+0000", 2024-09-16T09:40:56.386 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 229.91491082499999, 2024-09-16T09:40:56.386 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0069119719999999, 2024-09-16T09:40:56.386 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:40:56.386 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:40:56.386 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:40:56.386 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:40:56.386 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:40:56.386 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:40:56.386 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 220479 2024-09-16T09:40:56.386 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.386 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:40:56.386 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.387 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:40:56.387 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366688+0000", 2024-09-16T09:40:56.387 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:56.387 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.387 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.387 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:40:56.387 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366688+0000", 2024-09-16T09:40:56.387 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:56.387 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.387 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.388 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:40:56.388 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366688+0000", 2024-09-16T09:40:56.388 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.6100000000000001e-07 2024-09-16T09:40:56.388 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.388 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.388 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:40:56.388 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366692+0000", 2024-09-16T09:40:56.388 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2969999999999999e-06 2024-09-16T09:40:56.388 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.388 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.388 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:40:56.388 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366699+0000", 2024-09-16T09:40:56.388 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.5299999999999999e-06 2024-09-16T09:40:56.389 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.389 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.389 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:40:56.389 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366705+0000", 2024-09-16T09:40:56.389 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4489999999999998e-06 2024-09-16T09:40:56.389 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.389 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.389 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:40:56.389 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.460555+0000", 2024-09-16T09:40:56.389 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.093850180000000005 2024-09-16T09:40:56.389 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.389 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.389 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:40:56.390 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.460562+0000", 2024-09-16T09:40:56.390 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.6819999999999997e-06 2024-09-16T09:40:56.390 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.390 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.390 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:40:56.390 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.371896+0000", 2024-09-16T09:40:56.390 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.91133457399999995 2024-09-16T09:40:56.390 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.390 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.390 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:40:56.391 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.371993+0000", 2024-09-16T09:40:56.391 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.6768999999999995e-05 2024-09-16T09:40:56.391 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.391 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.391 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,2", 2024-09-16T09:40:56.391 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.372058+0000", 2024-09-16T09:40:56.391 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.4678999999999995e-05 2024-09-16T09:40:56.391 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.391 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.391 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:40:56.391 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.373407+0000", 2024-09-16T09:40:56.391 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001349056 2024-09-16T09:40:56.392 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.392 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.392 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:40:56.392 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.373457+0000", 2024-09-16T09:40:56.392 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0291999999999999e-05 2024-09-16T09:40:56.392 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.392 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.392 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:40:56.392 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.373501+0000", 2024-09-16T09:40:56.392 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.4558999999999998e-05 2024-09-16T09:40:56.392 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.392 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.392 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:40:56.393 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.373558+0000", 2024-09-16T09:40:56.393 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.6956000000000002e-05 2024-09-16T09:40:56.393 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.393 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.393 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:40:56.393 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.373600+0000", 2024-09-16T09:40:56.393 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.1387999999999997e-05 2024-09-16T09:40:56.393 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:56.393 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:56.393 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:56.393 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.393 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.394 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:220549 9.2 9:4ee8c96f:::benchmark_data_smithi070_137000_object98838:head [delete] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e346)", 2024-09-16T09:40:56.394 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:37:06.366728+0000", 2024-09-16T09:40:56.394 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 229.91487034799999, 2024-09-16T09:40:56.394 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0076284369999999, 2024-09-16T09:40:56.394 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:40:56.394 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:40:56.394 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:40:56.394 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:40:56.394 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:40:56.394 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:40:56.394 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 220549 2024-09-16T09:40:56.394 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.394 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:40:56.394 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.394 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:40:56.395 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366728+0000", 2024-09-16T09:40:56.395 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:56.395 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.395 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.395 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:40:56.395 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366728+0000", 2024-09-16T09:40:56.395 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:56.395 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.395 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.395 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:40:56.395 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366729+0000", 2024-09-16T09:40:56.395 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8899999999999999e-07 2024-09-16T09:40:56.395 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.395 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.395 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:40:56.396 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366732+0000", 2024-09-16T09:40:56.396 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.717e-06 2024-09-16T09:40:56.396 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.396 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.396 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:40:56.396 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366739+0000", 2024-09-16T09:40:56.396 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.0400000000000004e-06 2024-09-16T09:40:56.396 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.396 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.396 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:40:56.396 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366744+0000", 2024-09-16T09:40:56.396 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4700000000000001e-06 2024-09-16T09:40:56.397 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.397 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.397 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:40:56.397 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.460597+0000", 2024-09-16T09:40:56.397 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.093853013999999998 2024-09-16T09:40:56.397 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.397 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.397 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:40:56.397 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.460606+0000", 2024-09-16T09:40:56.397 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.5490000000000006e-06 2024-09-16T09:40:56.397 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.397 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.397 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:40:56.397 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.372452+0000", 2024-09-16T09:40:56.398 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.91184652300000002 2024-09-16T09:40:56.398 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.398 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.398 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:40:56.398 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.372557+0000", 2024-09-16T09:40:56.398 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000104411 2024-09-16T09:40:56.398 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.398 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.398 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,2", 2024-09-16T09:40:56.398 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.372649+0000", 2024-09-16T09:40:56.398 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.1911999999999997e-05 2024-09-16T09:40:56.398 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.398 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.398 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:40:56.398 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.373670+0000", 2024-09-16T09:40:56.398 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001021903 2024-09-16T09:40:56.399 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.399 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.399 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:40:56.399 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.373683+0000", 2024-09-16T09:40:56.399 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2655e-05 2024-09-16T09:40:56.399 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.399 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.399 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:40:56.399 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.374270+0000", 2024-09-16T09:40:56.399 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00058656799999999999 2024-09-16T09:40:56.399 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.399 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.399 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:40:56.399 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.374320+0000", 2024-09-16T09:40:56.399 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0627999999999997e-05 2024-09-16T09:40:56.399 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.400 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.400 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:40:56.400 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.374357+0000", 2024-09-16T09:40:56.400 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6458000000000002e-05 2024-09-16T09:40:56.400 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:56.400 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:56.400 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:56.400 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.400 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.400 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:220550 9.2 9:4bb12f00:::benchmark_data_smithi070_137000_object98839:head [delete] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e346)", 2024-09-16T09:40:56.400 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:37:06.366752+0000", 2024-09-16T09:40:56.400 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 229.91484674700001, 2024-09-16T09:40:56.400 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.00861194, 2024-09-16T09:40:56.400 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:40:56.400 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:40:56.400 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:40:56.401 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:40:56.401 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:40:56.401 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:40:56.401 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 220550 2024-09-16T09:40:56.401 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.401 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:40:56.401 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.401 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:40:56.401 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366752+0000", 2024-09-16T09:40:56.402 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:56.402 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.402 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.402 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:40:56.402 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366752+0000", 2024-09-16T09:40:56.402 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:56.402 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.402 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.402 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:40:56.402 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366752+0000", 2024-09-16T09:40:56.402 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.9999999999999999e-07 2024-09-16T09:40:56.402 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.402 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.402 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:40:56.402 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366754+0000", 2024-09-16T09:40:56.403 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9439999999999999e-06 2024-09-16T09:40:56.403 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.403 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.403 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:40:56.403 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366760+0000", 2024-09-16T09:40:56.403 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8529999999999997e-06 2024-09-16T09:40:56.403 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.403 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.403 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:40:56.403 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366765+0000", 2024-09-16T09:40:56.403 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5399999999999997e-06 2024-09-16T09:40:56.403 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.403 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.403 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:40:56.403 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.460637+0000", 2024-09-16T09:40:56.403 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.093872609999999995 2024-09-16T09:40:56.404 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.404 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.404 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:40:56.404 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.460643+0000", 2024-09-16T09:40:56.404 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5160000000000002e-06 2024-09-16T09:40:56.404 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.404 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.404 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:40:56.404 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.372992+0000", 2024-09-16T09:40:56.404 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.91234923700000004 2024-09-16T09:40:56.404 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.404 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.404 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:40:56.404 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.373087+0000", 2024-09-16T09:40:56.404 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.4653000000000003e-05 2024-09-16T09:40:56.405 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.405 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.405 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,2", 2024-09-16T09:40:56.405 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.373154+0000", 2024-09-16T09:40:56.405 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.7946000000000003e-05 2024-09-16T09:40:56.405 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.405 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.405 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:40:56.405 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.374404+0000", 2024-09-16T09:40:56.405 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0012499570000000001 2024-09-16T09:40:56.406 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.406 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.406 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:40:56.406 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.374439+0000", 2024-09-16T09:40:56.406 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4578e-05 2024-09-16T09:40:56.406 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.406 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.406 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:40:56.406 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.375280+0000", 2024-09-16T09:40:56.406 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00084091799999999996 2024-09-16T09:40:56.406 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.406 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.406 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:40:56.407 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.375329+0000", 2024-09-16T09:40:56.407 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.9123999999999999e-05 2024-09-16T09:40:56.407 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.407 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.407 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:40:56.407 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.375364+0000", 2024-09-16T09:40:56.407 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4764000000000001e-05 2024-09-16T09:40:56.407 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:56.407 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:56.407 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:56.407 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.407 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.407 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:220634 9.2 9:497a55c3:::benchmark_data_smithi070_137000_object98923:head [delete] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e346)", 2024-09-16T09:40:56.407 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:37:06.366770+0000", 2024-09-16T09:40:56.408 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 229.91482822, 2024-09-16T09:40:56.408 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.010067807, 2024-09-16T09:40:56.408 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:40:56.408 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:40:56.408 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:40:56.408 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:40:56.408 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:40:56.408 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:40:56.408 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 220634 2024-09-16T09:40:56.408 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.408 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:40:56.408 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.408 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:40:56.408 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366770+0000", 2024-09-16T09:40:56.409 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:56.409 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.409 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.409 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:40:56.409 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366770+0000", 2024-09-16T09:40:56.409 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:56.409 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.409 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.409 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:40:56.409 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366771+0000", 2024-09-16T09:40:56.409 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.8099999999999999e-07 2024-09-16T09:40:56.409 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.409 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.409 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:40:56.410 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366777+0000", 2024-09-16T09:40:56.410 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.6120000000000004e-06 2024-09-16T09:40:56.410 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.410 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.410 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:40:56.410 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366783+0000", 2024-09-16T09:40:56.410 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8089999999999998e-06 2024-09-16T09:40:56.410 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.410 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.410 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:40:56.410 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366788+0000", 2024-09-16T09:40:56.410 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5210000000000004e-06 2024-09-16T09:40:56.410 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.410 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.410 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:40:56.411 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.460677+0000", 2024-09-16T09:40:56.411 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.093889792999999999 2024-09-16T09:40:56.411 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.411 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.411 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:40:56.411 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.460684+0000", 2024-09-16T09:40:56.411 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.8959999999999997e-06 2024-09-16T09:40:56.411 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.411 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.411 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:40:56.411 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.373725+0000", 2024-09-16T09:40:56.411 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.91304026400000005 2024-09-16T09:40:56.412 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.412 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.412 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:40:56.412 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.373825+0000", 2024-09-16T09:40:56.412 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000100238 2024-09-16T09:40:56.412 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.412 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.412 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,2", 2024-09-16T09:40:56.412 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.373937+0000", 2024-09-16T09:40:56.412 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000112669 2024-09-16T09:40:56.412 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.412 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.412 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:40:56.412 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.375849+0000", 2024-09-16T09:40:56.413 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001911966 2024-09-16T09:40:56.413 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.413 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.413 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:40:56.413 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.375893+0000", 2024-09-16T09:40:56.413 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.3689000000000001e-05 2024-09-16T09:40:56.413 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.413 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.413 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:40:56.413 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.376754+0000", 2024-09-16T09:40:56.413 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00086106599999999996 2024-09-16T09:40:56.413 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.413 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.414 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:40:56.414 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.376800+0000", 2024-09-16T09:40:56.414 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5964999999999998e-05 2024-09-16T09:40:56.414 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.414 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.414 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:40:56.414 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.376838+0000", 2024-09-16T09:40:56.414 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8037999999999997e-05 2024-09-16T09:40:56.414 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:56.414 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:56.414 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:56.414 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.414 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.414 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:220635 9.2 9:48fa68c0:::benchmark_data_smithi070_137000_object98924:head [delete] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e346)", 2024-09-16T09:40:56.415 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:37:06.366794+0000", 2024-09-16T09:40:56.415 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 229.91480491799999, 2024-09-16T09:40:56.415 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0102374599999999, 2024-09-16T09:40:56.415 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:40:56.415 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:40:56.415 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:40:56.415 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:40:56.415 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:40:56.415 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:40:56.415 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 220635 2024-09-16T09:40:56.415 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.415 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:40:56.415 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.415 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:40:56.416 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366794+0000", 2024-09-16T09:40:56.416 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:56.416 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.416 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.416 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:40:56.416 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366794+0000", 2024-09-16T09:40:56.416 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:56.416 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.416 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.416 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:40:56.416 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366794+0000", 2024-09-16T09:40:56.416 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1699999999999999e-07 2024-09-16T09:40:56.416 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.416 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.417 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:40:56.417 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366796+0000", 2024-09-16T09:40:56.417 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.792e-06 2024-09-16T09:40:56.417 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.417 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.417 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:40:56.417 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366801+0000", 2024-09-16T09:40:56.417 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5999999999999997e-06 2024-09-16T09:40:56.417 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.417 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.417 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:40:56.417 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366806+0000", 2024-09-16T09:40:56.417 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.7469999999999996e-06 2024-09-16T09:40:56.418 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.418 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.418 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:40:56.418 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.460714+0000", 2024-09-16T09:40:56.418 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.093907861999999995 2024-09-16T09:40:56.418 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.418 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.418 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:40:56.418 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.460719+0000", 2024-09-16T09:40:56.418 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4070000000000001e-06 2024-09-16T09:40:56.418 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.418 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.418 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:40:56.418 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.374493+0000", 2024-09-16T09:40:56.419 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.91377397100000002 2024-09-16T09:40:56.419 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.419 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.419 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:40:56.419 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.374586+0000", 2024-09-16T09:40:56.419 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.2905000000000006e-05 2024-09-16T09:40:56.419 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.419 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.419 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,2", 2024-09-16T09:40:56.419 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.374680+0000", 2024-09-16T09:40:56.419 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.3769000000000003e-05 2024-09-16T09:40:56.419 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.419 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.420 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:40:56.420 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.375855+0000", 2024-09-16T09:40:56.420 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001175073 2024-09-16T09:40:56.420 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.420 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.420 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:40:56.420 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.376878+0000", 2024-09-16T09:40:56.420 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0010223820000000001 2024-09-16T09:40:56.420 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.420 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.420 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:40:56.420 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.376950+0000", 2024-09-16T09:40:56.420 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.1966999999999997e-05 2024-09-16T09:40:56.421 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.421 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.421 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:40:56.421 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.376997+0000", 2024-09-16T09:40:56.421 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.7987e-05 2024-09-16T09:40:56.421 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.421 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.421 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:40:56.421 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.377031+0000", 2024-09-16T09:40:56.421 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:40:56.423 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3680999999999998e-05 2024-09-16T09:40:56.423 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:56.423 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:56.423 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:56.423 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.423 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.423 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:220638 9.2 9:4df3d7d7:::benchmark_data_smithi070_137000_object98927:head [delete] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e346)", 2024-09-16T09:40:56.423 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:37:06.366812+0000", 2024-09-16T09:40:56.423 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 229.914786808, 2024-09-16T09:40:56.423 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0103380449999999, 2024-09-16T09:40:56.423 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:40:56.423 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:40:56.424 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:40:56.424 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:40:56.424 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:40:56.424 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:40:56.424 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 220638 2024-09-16T09:40:56.424 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.424 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:40:56.424 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.424 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:40:56.424 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366812+0000", 2024-09-16T09:40:56.424 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:56.424 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.424 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.424 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:40:56.425 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366812+0000", 2024-09-16T09:40:56.425 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:56.425 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.425 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.425 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:40:56.425 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366812+0000", 2024-09-16T09:40:56.425 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.58e-07 2024-09-16T09:40:56.425 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.425 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.425 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:40:56.425 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366814+0000", 2024-09-16T09:40:56.425 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6670000000000001e-06 2024-09-16T09:40:56.425 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.426 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.426 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:40:56.426 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366820+0000", 2024-09-16T09:40:56.426 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.6799999999999998e-06 2024-09-16T09:40:56.426 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.426 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.426 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:40:56.426 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366824+0000", 2024-09-16T09:40:56.426 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.7400000000000004e-06 2024-09-16T09:40:56.426 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.426 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.426 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:40:56.427 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.460752+0000", 2024-09-16T09:40:56.427 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.093927963000000003 2024-09-16T09:40:56.427 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.427 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.427 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:40:56.427 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.460759+0000", 2024-09-16T09:40:56.427 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.7989999999999996e-06 2024-09-16T09:40:56.427 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.427 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.427 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:40:56.427 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.374896+0000", 2024-09-16T09:40:56.427 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.91413692400000002 2024-09-16T09:40:56.427 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.427 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.428 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:40:56.428 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.374996+0000", 2024-09-16T09:40:56.428 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.9656999999999998e-05 2024-09-16T09:40:56.428 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.428 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.428 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,2", 2024-09-16T09:40:56.428 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.375097+0000", 2024-09-16T09:40:56.428 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000101109 2024-09-16T09:40:56.428 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.428 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.428 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:40:56.428 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.376713+0000", 2024-09-16T09:40:56.429 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0016159530000000001 2024-09-16T09:40:56.429 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.429 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.429 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:40:56.429 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.376914+0000", 2024-09-16T09:40:56.429 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00020110700000000001 2024-09-16T09:40:56.429 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.429 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.429 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:40:56.429 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.377068+0000", 2024-09-16T09:40:56.429 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00015383500000000001 2024-09-16T09:40:56.429 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.429 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.430 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:40:56.430 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.377115+0000", 2024-09-16T09:40:56.430 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.7604e-05 2024-09-16T09:40:56.430 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.430 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.430 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:40:56.430 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.377150+0000", 2024-09-16T09:40:56.430 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4649e-05 2024-09-16T09:40:56.430 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:56.430 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:56.430 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:56.430 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.430 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.430 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:220657 9.2 9:4cd451a8:::benchmark_data_smithi070_137000_object98946:head [delete] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e346)", 2024-09-16T09:40:56.430 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:37:06.366830+0000", 2024-09-16T09:40:56.431 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 229.91476866799999, 2024-09-16T09:40:56.431 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.010672051, 2024-09-16T09:40:56.431 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:40:56.431 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:40:56.431 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:40:56.431 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:40:56.431 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:40:56.431 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:40:56.431 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 220657 2024-09-16T09:40:56.431 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.431 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:40:56.432 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.432 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:40:56.432 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366830+0000", 2024-09-16T09:40:56.432 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:56.432 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.432 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.432 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:40:56.432 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366830+0000", 2024-09-16T09:40:56.432 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:40:56.432 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.433 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.433 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:40:56.433 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366830+0000", 2024-09-16T09:40:56.433 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.65e-07 2024-09-16T09:40:56.433 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.433 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.433 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:40:56.433 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366832+0000", 2024-09-16T09:40:56.433 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6479999999999999e-06 2024-09-16T09:40:56.433 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.433 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.433 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:40:56.433 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366838+0000", 2024-09-16T09:40:56.434 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.6640000000000003e-06 2024-09-16T09:40:56.434 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.434 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.434 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:40:56.434 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366847+0000", 2024-09-16T09:40:56.434 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.8670000000000006e-06 2024-09-16T09:40:56.434 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.434 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.434 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:40:56.434 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.460788+0000", 2024-09-16T09:40:56.434 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.093940896999999995 2024-09-16T09:40:56.434 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.434 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.434 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:40:56.434 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.460795+0000", 2024-09-16T09:40:56.435 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.5699999999999998e-06 2024-09-16T09:40:56.435 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.435 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.435 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:40:56.435 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.375406+0000", 2024-09-16T09:40:56.435 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.91461130199999996 2024-09-16T09:40:56.435 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.435 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.435 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:40:56.435 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.375587+0000", 2024-09-16T09:40:56.435 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00018084700000000001 2024-09-16T09:40:56.435 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.435 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.435 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,2", 2024-09-16T09:40:56.435 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.375684+0000", 2024-09-16T09:40:56.436 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.7095999999999993e-05 2024-09-16T09:40:56.436 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.436 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.436 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:40:56.436 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.376717+0000", 2024-09-16T09:40:56.436 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0010328729999999999 2024-09-16T09:40:56.436 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.436 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.436 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:40:56.436 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.377357+0000", 2024-09-16T09:40:56.436 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00063957 2024-09-16T09:40:56.436 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.436 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.436 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:40:56.436 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.377417+0000", 2024-09-16T09:40:56.437 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.0062999999999997e-05 2024-09-16T09:40:56.437 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.437 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.437 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:40:56.437 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.377466+0000", 2024-09-16T09:40:56.437 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.9407999999999999e-05 2024-09-16T09:40:56.437 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:40:56.437 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:40:56.437 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:40:56.437 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.377502+0000", 2024-09-16T09:40:56.437 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5981e-05 2024-09-16T09:40:56.437 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:56.437 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:56.438 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:56.438 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:40:56.438 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:40:56.438 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T09:40:56.480 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:56.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:56.481 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:40:56.582 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:40:56.618 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:56.619 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:56.619 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:40:56.720 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:40:56.763 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:56.764 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:56.764 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:40:56.865 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:40:56.901 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:56.902 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:56.902 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:40:56.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:56 smithi084 ceph-mon[32629]: osdmap e444: 8 total, 8 up, 8 in 2024-09-16T09:40:56.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:56 smithi084 ceph-mon[32629]: pgmap v836: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 788 MiB data, 8.2 GiB used, 707 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.62k op/s; 981/38421 objects misplaced (2.553%) 2024-09-16T09:40:56.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:56 smithi070 ceph-mon[30388]: osdmap e444: 8 total, 8 up, 8 in 2024-09-16T09:40:56.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:56 smithi070 ceph-mon[30388]: pgmap v836: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 788 MiB data, 8.2 GiB used, 707 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.62k op/s; 981/38421 objects misplaced (2.553%) 2024-09-16T09:40:56.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:56 smithi070 ceph-mon[40691]: osdmap e444: 8 total, 8 up, 8 in 2024-09-16T09:40:56.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:56 smithi070 ceph-mon[40691]: pgmap v836: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 788 MiB data, 8.2 GiB used, 707 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.62k op/s; 981/38421 objects misplaced (2.553%) 2024-09-16T09:40:57.003 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:40:57.041 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:57.041 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:57.042 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:40:57.142 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:40:57.174 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:57.175 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:57.175 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:40:57.276 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:40:57.307 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:57.319 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:57.320 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:40:57.420 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:40:57.453 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:57.454 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:57.454 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:40:57.555 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:40:57.589 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:57.589 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:57.590 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:40:57.647 INFO:teuthology.orchestra.run.smithi070.stderr:set 4.19 pg_upmap mapping to [4,7,5] 2024-09-16T09:40:57.691 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:40:57.727 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:57.728 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:57.728 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:40:57.786 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T09:40:57.829 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:40:57.867 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:57.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:57.868 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:40:57.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:57 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/139876998' entity='client.admin' cmd={"prefix": "osd pg-upmap", "pgid": "4.19", "id": [4, 7, 5]} : dispatch 2024-09-16T09:40:57.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:57 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pg-upmap", "pgid": "4.19", "id": [4, 7, 5]} : dispatch 2024-09-16T09:40:57.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:57 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/139876998' entity='client.admin' cmd={"prefix": "osd pg-upmap", "pgid": "4.19", "id": [4, 7, 5]} : dispatch 2024-09-16T09:40:57.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:57 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pg-upmap", "pgid": "4.19", "id": [4, 7, 5]} : dispatch 2024-09-16T09:40:57.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:57 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/139876998' entity='client.admin' cmd={"prefix": "osd pg-upmap", "pgid": "4.19", "id": [4, 7, 5]} : dispatch 2024-09-16T09:40:57.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:57 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pg-upmap", "pgid": "4.19", "id": [4, 7, 5]} : dispatch 2024-09-16T09:40:57.968 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:40:57.998 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:57.998 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:57.999 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:40:58.099 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:40:58.130 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:58.131 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:58.131 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:40:58.232 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:40:58.267 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:58.268 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:58.268 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:40:58.369 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:40:58.406 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:58.407 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:58.407 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:40:58.508 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:40:58.541 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:58.541 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:58.542 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:40:58.643 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:40:58.672 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:58.673 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:58.673 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:40:58.721 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.4 dump_ops_in_flight 2024-09-16T09:40:58.774 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:40:58.841 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:58.842 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:58.842 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:40:58.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:58 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pg-upmap", "pgid": "4.19", "id": [4, 7, 5]}]': finished 2024-09-16T09:40:58.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:58 smithi070 ceph-mon[30388]: osdmap e445: 8 total, 8 up, 8 in 2024-09-16T09:40:58.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:58 smithi070 ceph-mon[30388]: pgmap v838: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 748 MiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 897 op/s; 981/36462 objects misplaced (2.690%) 2024-09-16T09:40:58.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:58 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:40:58.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:58 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pg-upmap", "pgid": "4.19", "id": [4, 7, 5]}]': finished 2024-09-16T09:40:58.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:58 smithi070 ceph-mon[40691]: osdmap e445: 8 total, 8 up, 8 in 2024-09-16T09:40:58.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:58 smithi070 ceph-mon[40691]: pgmap v838: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 748 MiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 897 op/s; 981/36462 objects misplaced (2.690%) 2024-09-16T09:40:58.923 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:58 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:40:58.943 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:40:58.976 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:58.977 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:58.977 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:40:59.078 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:40:59.110 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:59.111 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:59.112 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:40:59.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:58 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pg-upmap", "pgid": "4.19", "id": [4, 7, 5]}]': finished 2024-09-16T09:40:59.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:58 smithi084 ceph-mon[32629]: osdmap e445: 8 total, 8 up, 8 in 2024-09-16T09:40:59.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:58 smithi084 ceph-mon[32629]: pgmap v838: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 748 MiB data, 8.0 GiB used, 707 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 897 op/s; 981/36462 objects misplaced (2.690%) 2024-09-16T09:40:59.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:58 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:40:59.163 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:40:59.212 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:40:59.249 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:59.249 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:59.250 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:40:59.350 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:40:59.386 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:59.387 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:59.387 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:40:59.449 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:40:59.489 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:40:59.515 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:40:59.539 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:59.539 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:59.539 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:40:59.640 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:40:59.678 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:59.679 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:59.680 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:40:59.740 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:59 smithi070 ceph-mon[30388]: osdmap e446: 8 total, 8 up, 8 in 2024-09-16T09:40:59.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:59 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:40:59.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:59 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:40:59.741 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:40:59 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:40:59.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:59 smithi070 ceph-mon[40691]: osdmap e446: 8 total, 8 up, 8 in 2024-09-16T09:40:59.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:59 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:40:59.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:59 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:40:59.741 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:40:59 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:40:59.781 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:40:59.813 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:59.814 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:40:59.814 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:40:59.914 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:40:59.948 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:40:59.949 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:40:59.949 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:41:00.050 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:41:00.081 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:00.082 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:00.082 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:41:00.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:59 smithi084 ceph-mon[32629]: osdmap e446: 8 total, 8 up, 8 in 2024-09-16T09:41:00.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:59 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:41:00.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:59 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:41:00.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:40:59 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:41:00.184 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:41:00.217 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:00.218 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:00.218 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:41:00.319 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:41:00.358 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:00.358 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:00.359 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:41:00.460 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:41:00.500 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:00.500 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:00.501 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:41:00.547 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:41:00.548 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:41:00.589 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:41:00.589 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:41:00.602 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:41:00.622 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:41:00.623 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:41:00.635 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:00.635 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:00.636 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:41:00.665 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:41:00.665 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:41:00.706 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:41:00.706 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:41:00.737 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:41:00.741 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:41:00.741 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:41:00.777 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:00.778 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:00.778 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:41:00.779 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:41:00.779 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:41:00.810 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:41:00.810 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:41:00.879 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:41:00.916 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:00.917 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:00.917 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:41:01.018 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:41:01.046 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:01.047 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:01.047 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:41:01.101 INFO:teuthology.orchestra.run.smithi084.stdout:ERROR: (22) Invalid argument 2024-09-16T09:41:01.101 INFO:teuthology.orchestra.run.smithi084.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:41:01.148 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:41:01.191 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:01.192 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:01.192 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:41:01.217 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:00 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:41:01.217 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:00 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:41:01.217 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:00 smithi070 ceph-mon[30388]: pgmap v840: 178 pgs: 1 peering, 1 active+remapped+backfill_toofull, 176 active+clean; 354 MiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 1.31k op/s; 270/17574 objects misplaced (1.536%) 2024-09-16T09:41:01.217 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:00 smithi070 ceph-mon[30388]: osdmap e447: 8 total, 8 up, 8 in 2024-09-16T09:41:01.217 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:00 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2285942914' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:41:01.218 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:00 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3666174485' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:41:01.218 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:00 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:41:01.218 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:00 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:41:01.218 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:00 smithi070 ceph-mon[40691]: pgmap v840: 178 pgs: 1 peering, 1 active+remapped+backfill_toofull, 176 active+clean; 354 MiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 1.31k op/s; 270/17574 objects misplaced (1.536%) 2024-09-16T09:41:01.218 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:00 smithi070 ceph-mon[40691]: osdmap e447: 8 total, 8 up, 8 in 2024-09-16T09:41:01.218 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:00 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/2285942914' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:41:01.218 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:00 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3666174485' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:41:01.293 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:41:01.325 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:01.326 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:01.326 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:41:01.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:00 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:41:01.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:00 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:41:01.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:00 smithi084 ceph-mon[32629]: pgmap v840: 178 pgs: 1 peering, 1 active+remapped+backfill_toofull, 176 active+clean; 354 MiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 1.31k op/s; 270/17574 objects misplaced (1.536%) 2024-09-16T09:41:01.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:00 smithi084 ceph-mon[32629]: osdmap e447: 8 total, 8 up, 8 in 2024-09-16T09:41:01.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:00 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/2285942914' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:41:01.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:00 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3666174485' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:41:01.427 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:41:01.461 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:01.462 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:01.462 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:41:01.564 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:41:01.595 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:01.595 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:01.596 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:41:01.678 INFO:teuthology.orchestra.run.smithi070.stderr:noscrub is unset 2024-09-16T09:41:01.696 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:41:01.730 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:01.730 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:01.731 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:41:01.832 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:41:01.872 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:01.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:01.873 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:41:01.974 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:41:02.006 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:02.007 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:02.007 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:41:02.044 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.4 dump_blocked_ops 2024-09-16T09:41:02.108 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:41:02.138 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:02.139 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:02.139 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:41:02.240 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:41:02.272 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:02.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:02.273 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:41:02.374 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:41:02.384 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Removed 114353 objects 2024-09-16T09:41:02.384 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Clean up completed and total clean up time :95.8082 2024-09-16T09:41:02.390 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:01 smithi070 ceph-mon[30388]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:41:02.390 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:01 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3666174485' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-16T09:41:02.390 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:01 smithi070 ceph-mon[30388]: osdmap e448: 8 total, 8 up, 8 in 2024-09-16T09:41:02.391 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:01 smithi070 ceph-mon[40691]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:41:02.391 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:01 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3666174485' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-16T09:41:02.391 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:01 smithi070 ceph-mon[40691]: osdmap e448: 8 total, 8 up, 8 in 2024-09-16T09:41:02.391 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:41:01 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:41:01] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:41:02.399 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:02.400 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:02.400 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:41:02.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:01 smithi084 ceph-mon[32629]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:41:02.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:01 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3666174485' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-16T09:41:02.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:01 smithi084 ceph-mon[32629]: osdmap e448: 8 total, 8 up, 8 in 2024-09-16T09:41:02.430 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:41:02.501 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:41:02.523 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:02.523 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:02.524 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:41:02.624 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:41:02.646 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:02.646 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:02.647 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:41:02.747 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:41:02.769 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:02.769 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:02.770 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:41:02.870 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:41:02.892 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:02.893 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:02.893 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:41:02.994 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:41:03.015 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:03.016 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:03.016 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:41:03.117 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:41:03.141 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:03.142 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:03.142 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:41:03.243 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:41:03.265 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:03.266 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:03.266 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:41:03.367 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:41:03.390 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:03.390 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:03.391 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:41:03.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:02 smithi084 ceph-mon[32629]: pgmap v843: 178 pgs: 1 peering, 1 active+remapped+backfill_toofull, 176 active+clean; 354 MiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 0 B/s wr, 1.50k op/s; 270/17574 objects misplaced (1.536%) 2024-09-16T09:41:03.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:02 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:41:03.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:02 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:41:03.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:02 smithi084 ceph-mon[32629]: osdmap e449: 8 total, 8 up, 8 in 2024-09-16T09:41:03.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:02 smithi070 ceph-mon[30388]: pgmap v843: 178 pgs: 1 peering, 1 active+remapped+backfill_toofull, 176 active+clean; 354 MiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 0 B/s wr, 1.50k op/s; 270/17574 objects misplaced (1.536%) 2024-09-16T09:41:03.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:02 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:41:03.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:02 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:41:03.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:02 smithi070 ceph-mon[30388]: osdmap e449: 8 total, 8 up, 8 in 2024-09-16T09:41:03.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:02 smithi070 ceph-mon[40691]: pgmap v843: 178 pgs: 1 peering, 1 active+remapped+backfill_toofull, 176 active+clean; 354 MiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 0 B/s wr, 1.50k op/s; 270/17574 objects misplaced (1.536%) 2024-09-16T09:41:03.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:02 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:41:03.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:02 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:41:03.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:02 smithi070 ceph-mon[40691]: osdmap e449: 8 total, 8 up, 8 in 2024-09-16T09:41:03.492 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:41:03.515 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:03.515 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:03.516 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:41:03.568 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T09:41:03.617 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:41:03.638 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:03.639 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:03.639 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:41:03.740 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:41:03.766 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:03.767 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:03.767 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:41:03.799 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:41:03 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:41:03.829 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:41:03.868 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:41:03.892 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:03.892 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:03.893 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:41:03.914 INFO:teuthology.orchestra.run.smithi084.stdout:ERROR: (22) Invalid argument 2024-09-16T09:41:03.914 INFO:teuthology.orchestra.run.smithi084.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:41:03.993 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:41:04.015 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:04.015 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:04.016 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:41:04.116 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:41:04.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:04.139 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:04.139 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:41:04.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:03 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:41:04.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:03 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:41:04.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:03 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:41:04.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:03 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:41:04.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:03 smithi084 ceph-mon[32629]: osdmap e450: 8 total, 8 up, 8 in 2024-09-16T09:41:04.239 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:41:04.262 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:04.262 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:04.262 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:41:04.269 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:03 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:41:04.269 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:03 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:41:04.269 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:03 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:41:04.269 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:03 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:41:04.269 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:03 smithi070 ceph-mon[30388]: osdmap e450: 8 total, 8 up, 8 in 2024-09-16T09:41:04.269 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:03 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:41:04.269 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:03 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:41:04.269 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:03 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:41:04.269 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:03 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:41:04.269 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:03 smithi070 ceph-mon[40691]: osdmap e450: 8 total, 8 up, 8 in 2024-09-16T09:41:04.363 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:41:04.376 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:41:04.377 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:41:04.385 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:04.385 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:04.386 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:41:04.386 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:41:04.386 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:41:04.402 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:41:04.402 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:41:04.416 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-16T09:41:04.421 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:41:04.421 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:41:04.435 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:41:04.435 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:41:04.452 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:41:04.452 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:41:04.483 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:41:04.484 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:41:04.487 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:41:04.500 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T09:41:04.500 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T09:41:04.500 INFO:tasks.thrashosds.thrasher:inject_pause on osd.1 2024-09-16T09:41:04.500 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-16T09:41:04.500 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-16T09:41:04.501 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.1 config set filestore_inject_stall 3 2024-09-16T09:41:04.501 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:41:04.501 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:41:04.508 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:04.509 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:04.510 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:41:04.513 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.4 dump_historic_ops 2024-09-16T09:41:04.610 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:41:04.632 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:04.632 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:04.633 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:41:04.733 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:41:04.756 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:04.757 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:04.757 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:41:04.764 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:41:04.858 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:41:04.880 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:04.880 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:04.881 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:41:04.982 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:41:05.003 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:05.004 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:05.004 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:41:05.105 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:41:05.127 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:05.127 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:05.128 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:41:05.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:04 smithi084 ceph-mon[32629]: 4.e scrub starts 2024-09-16T09:41:05.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:04 smithi084 ceph-mon[32629]: 4.e scrub ok 2024-09-16T09:41:05.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:04 smithi084 ceph-mon[32629]: pgmap v846: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 31 MiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.29k op/s; 270/2055 objects misplaced (13.139%); 27 B/s, 1 objects/s recovering 2024-09-16T09:41:05.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:04 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/176081635' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:41:05.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:04 smithi070 ceph-mon[30388]: 4.e scrub starts 2024-09-16T09:41:05.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:04 smithi070 ceph-mon[30388]: 4.e scrub ok 2024-09-16T09:41:05.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:04 smithi070 ceph-mon[30388]: pgmap v846: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 31 MiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.29k op/s; 270/2055 objects misplaced (13.139%); 27 B/s, 1 objects/s recovering 2024-09-16T09:41:05.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:04 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/176081635' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:41:05.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:04 smithi070 ceph-mon[40691]: 4.e scrub starts 2024-09-16T09:41:05.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:04 smithi070 ceph-mon[40691]: 4.e scrub ok 2024-09-16T09:41:05.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:04 smithi070 ceph-mon[40691]: pgmap v846: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 31 MiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.29k op/s; 270/2055 objects misplaced (13.139%); 27 B/s, 1 objects/s recovering 2024-09-16T09:41:05.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:04 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/176081635' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:41:05.229 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:41:05.235 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:41:05.238 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:41:05.251 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:05.252 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:05.252 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:41:05.349 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-09-16T09:41:05.349 INFO:teuthology.orchestra.run.smithi084.stdout: "size": 20, 2024-09-16T09:41:05.349 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 600, 2024-09-16T09:41:05.349 INFO:teuthology.orchestra.run.smithi084.stdout: "ops": [ 2024-09-16T09:41:05.349 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.349 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.25064.0:170549 8.b 8:d1a36449:::benchmark_data_smithi070_107919_object45954:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e204)", 2024-09-16T09:41:05.349 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:32:07.438095+0000", 2024-09-16T09:41:05.350 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 537.88383024100006, 2024-09-16T09:41:05.350 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.004960772, 2024-09-16T09:41:05.350 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:41:05.350 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:41:05.350 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:41:05.350 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:41:05.350 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.25064", 2024-09-16T09:41:05.350 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/2578256439", 2024-09-16T09:41:05.350 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 170549 2024-09-16T09:41:05.350 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.350 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:41:05.350 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.350 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:41:05.351 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.438095+0000", 2024-09-16T09:41:05.351 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:41:05.351 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.351 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.351 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:41:05.351 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.438095+0000", 2024-09-16T09:41:05.351 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:41:05.351 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.351 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.351 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:41:05.351 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.438096+0000", 2024-09-16T09:41:05.351 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.7370000000000001e-06 2024-09-16T09:41:05.351 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.352 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.352 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:41:05.352 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.438103+0000", 2024-09-16T09:41:05.352 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.3670000000000002e-06 2024-09-16T09:41:05.352 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.352 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.352 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:41:05.352 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.438119+0000", 2024-09-16T09:41:05.352 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.6084999999999999e-05 2024-09-16T09:41:05.352 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.352 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.352 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:41:05.352 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.438126+0000", 2024-09-16T09:41:05.353 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.7349999999999999e-06 2024-09-16T09:41:05.353 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.353 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.353 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:41:05.353 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.438179+0000", 2024-09-16T09:41:05.353 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.2961000000000003e-05 2024-09-16T09:41:05.353 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.353 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.353 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:41:05.353 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.438187+0000", 2024-09-16T09:41:05.353 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.1890000000000007e-06 2024-09-16T09:41:05.353 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.353 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.354 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:41:05.354 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.436726+0000", 2024-09-16T09:41:05.354 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.99853915599999998 2024-09-16T09:41:05.354 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.354 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.354 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:41:05.354 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.436844+0000", 2024-09-16T09:41:05.354 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000118089 2024-09-16T09:41:05.354 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.354 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.354 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 1,3", 2024-09-16T09:41:05.354 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.436976+0000", 2024-09-16T09:41:05.354 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00013196700000000001 2024-09-16T09:41:05.354 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.355 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.355 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:41:05.355 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.442912+0000", 2024-09-16T09:41:05.355 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0059363970000000004 2024-09-16T09:41:05.355 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.355 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.355 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:41:05.355 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.442954+0000", 2024-09-16T09:41:05.355 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.1761000000000002e-05 2024-09-16T09:41:05.355 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.355 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.355 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:41:05.355 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.442990+0000", 2024-09-16T09:41:05.356 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.6091999999999999e-05 2024-09-16T09:41:05.356 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.356 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.356 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:41:05.356 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.443029+0000", 2024-09-16T09:41:05.356 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.9186000000000001e-05 2024-09-16T09:41:05.356 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.356 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.356 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:41:05.356 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.443055+0000", 2024-09-16T09:41:05.356 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.605e-05 2024-09-16T09:41:05.356 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:41:05.356 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:41:05.356 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:41:05.357 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.357 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.357 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.25064.0:170577 8.b 8:d5966a91:::benchmark_data_smithi070_107919_object45982:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e204)", 2024-09-16T09:41:05.357 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:32:07.478909+0000", 2024-09-16T09:41:05.357 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 537.84301536299995, 2024-09-16T09:41:05.357 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.96429471700000002, 2024-09-16T09:41:05.357 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:41:05.357 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:41:05.357 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:41:05.357 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:41:05.357 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.25064", 2024-09-16T09:41:05.357 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/2578256439", 2024-09-16T09:41:05.357 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 170577 2024-09-16T09:41:05.357 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.358 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:41:05.358 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.358 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:41:05.358 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.478909+0000", 2024-09-16T09:41:05.358 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:41:05.358 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.358 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.358 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:41:05.358 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.478909+0000", 2024-09-16T09:41:05.358 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:41:05.358 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.358 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.358 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:41:05.359 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.478911+0000", 2024-09-16T09:41:05.359 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.8759999999999999e-06 2024-09-16T09:41:05.359 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.359 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.359 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:41:05.359 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.478918+0000", 2024-09-16T09:41:05.359 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.6590000000000001e-06 2024-09-16T09:41:05.359 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.359 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.359 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:41:05.359 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.478936+0000", 2024-09-16T09:41:05.359 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.827e-05 2024-09-16T09:41:05.359 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.359 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.360 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:41:05.360 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.478945+0000", 2024-09-16T09:41:05.360 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.2139999999999996e-06 2024-09-16T09:41:05.360 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.360 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.360 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:41:05.360 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.479008+0000", 2024-09-16T09:41:05.360 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.3459000000000003e-05 2024-09-16T09:41:05.360 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.360 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.360 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:41:05.360 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.479032+0000", 2024-09-16T09:41:05.360 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.3918e-05 2024-09-16T09:41:05.360 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.361 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.361 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:41:05.361 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.439436+0000", 2024-09-16T09:41:05.361 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.96040375899999997 2024-09-16T09:41:05.361 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.361 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.361 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:41:05.361 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.439545+0000", 2024-09-16T09:41:05.361 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000109576 2024-09-16T09:41:05.361 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.361 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.361 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 1,3", 2024-09-16T09:41:05.361 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.439667+0000", 2024-09-16T09:41:05.362 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00012150999999999999 2024-09-16T09:41:05.362 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.362 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.362 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:41:05.362 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.443079+0000", 2024-09-16T09:41:05.362 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0034118569999999999 2024-09-16T09:41:05.362 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.362 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.362 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:41:05.362 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.443131+0000", 2024-09-16T09:41:05.362 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.2565000000000002e-05 2024-09-16T09:41:05.362 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.362 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.363 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:41:05.363 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.443149+0000", 2024-09-16T09:41:05.363 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.7343999999999999e-05 2024-09-16T09:41:05.363 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.363 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.363 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:41:05.363 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.443177+0000", 2024-09-16T09:41:05.363 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.8654e-05 2024-09-16T09:41:05.363 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.363 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.363 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:41:05.363 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.443204+0000", 2024-09-16T09:41:05.363 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.7056e-05 2024-09-16T09:41:05.364 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:41:05.364 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:41:05.364 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:41:05.364 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.364 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.364 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.25064.0:170589 8.b 8:d7b2d79b:::benchmark_data_smithi070_107919_object45994:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e204)", 2024-09-16T09:41:05.364 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:32:07.486872+0000", 2024-09-16T09:41:05.364 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 537.83505241900002, 2024-09-16T09:41:05.364 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.97349502600000004, 2024-09-16T09:41:05.364 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:41:05.364 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:41:05.364 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:41:05.364 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:41:05.364 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.25064", 2024-09-16T09:41:05.365 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/2578256439", 2024-09-16T09:41:05.365 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 170589 2024-09-16T09:41:05.365 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.365 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:41:05.365 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.365 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:41:05.365 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.486872+0000", 2024-09-16T09:41:05.365 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:41:05.365 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.365 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.365 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:41:05.365 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.486872+0000", 2024-09-16T09:41:05.365 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:41:05.365 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.366 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.366 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:41:05.366 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.486874+0000", 2024-09-16T09:41:05.366 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.6300000000000001e-06 2024-09-16T09:41:05.366 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.366 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.366 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:41:05.366 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.486879+0000", 2024-09-16T09:41:05.366 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.1320000000000002e-06 2024-09-16T09:41:05.366 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.366 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.366 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:41:05.366 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.486895+0000", 2024-09-16T09:41:05.367 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.5719999999999999e-05 2024-09-16T09:41:05.367 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.367 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.367 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:41:05.367 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.486924+0000", 2024-09-16T09:41:05.367 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.8742e-05 2024-09-16T09:41:05.367 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.367 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.367 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:41:05.367 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.486985+0000", 2024-09-16T09:41:05.367 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.1099999999999994e-05 2024-09-16T09:41:05.367 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.367 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.367 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:41:05.368 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.486995+0000", 2024-09-16T09:41:05.368 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0538e-05 2024-09-16T09:41:05.368 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.368 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.368 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:41:05.368 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.441464+0000", 2024-09-16T09:41:05.368 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.95446911700000003 2024-09-16T09:41:05.368 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.368 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.368 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:41:05.368 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.441585+0000", 2024-09-16T09:41:05.368 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000120128 2024-09-16T09:41:05.368 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.368 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.368 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 1,3", 2024-09-16T09:41:05.369 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.441713+0000", 2024-09-16T09:41:05.369 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00012807699999999999 2024-09-16T09:41:05.369 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.369 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.369 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:41:05.369 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.444780+0000", 2024-09-16T09:41:05.369 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0030674370000000001 2024-09-16T09:41:05.369 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.369 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.369 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:41:05.369 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.448814+0000", 2024-09-16T09:41:05.369 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0040338570000000001 2024-09-16T09:41:05.369 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.370 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.370 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:41:05.370 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.460268+0000", 2024-09-16T09:41:05.370 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.011454097 2024-09-16T09:41:05.370 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.370 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.370 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:41:05.370 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.460321+0000", 2024-09-16T09:41:05.370 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.2630000000000003e-05 2024-09-16T09:41:05.370 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.370 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.370 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:41:05.370 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.460367+0000", 2024-09-16T09:41:05.370 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.6820999999999998e-05 2024-09-16T09:41:05.370 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:41:05.371 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:41:05.371 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:41:05.371 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.371 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.371 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.25064.0:170602 8.b 8:d9b50ae1:::benchmark_data_smithi070_107919_object46007:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e204)", 2024-09-16T09:41:05.371 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:32:07.497614+0000", 2024-09-16T09:41:05.371 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 537.82431048000001, 2024-09-16T09:41:05.371 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.96754656299999997, 2024-09-16T09:41:05.371 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:41:05.371 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:41:05.371 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:41:05.371 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:41:05.371 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.25064", 2024-09-16T09:41:05.371 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/2578256439", 2024-09-16T09:41:05.372 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 170602 2024-09-16T09:41:05.372 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.372 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:41:05.372 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.372 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:41:05.372 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.497614+0000", 2024-09-16T09:41:05.372 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:41:05.372 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.372 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.372 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:41:05.372 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.497614+0000", 2024-09-16T09:41:05.372 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:41:05.372 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.372 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.373 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:41:05.373 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.497616+0000", 2024-09-16T09:41:05.373 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.1050000000000002e-06 2024-09-16T09:41:05.373 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.373 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.373 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:41:05.373 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.497624+0000", 2024-09-16T09:41:05.373 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.0840000000000003e-06 2024-09-16T09:41:05.373 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.373 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.373 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:41:05.373 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.497645+0000", 2024-09-16T09:41:05.373 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.1508000000000001e-05 2024-09-16T09:41:05.373 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.374 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.374 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:41:05.374 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.497655+0000", 2024-09-16T09:41:05.374 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.9879999999999999e-06 2024-09-16T09:41:05.374 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.374 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.374 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:41:05.374 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.497728+0000", 2024-09-16T09:41:05.374 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.2469e-05 2024-09-16T09:41:05.374 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.374 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.374 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:41:05.374 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.497739+0000", 2024-09-16T09:41:05.375 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1053000000000001e-05 2024-09-16T09:41:05.375 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.375 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.375 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:41:05.375 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.443231+0000", 2024-09-16T09:41:05.375 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.94549259399999996 2024-09-16T09:41:05.375 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.375 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.375 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:41:05.375 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.443292+0000", 2024-09-16T09:41:05.375 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.0851999999999999e-05 2024-09-16T09:41:05.375 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.375 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.376 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 1,3", 2024-09-16T09:41:05.376 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.443352+0000", 2024-09-16T09:41:05.376 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.9738999999999998e-05 2024-09-16T09:41:05.376 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.376 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.376 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:41:05.376 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.448824+0000", 2024-09-16T09:41:05.376 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0054719779999999997 2024-09-16T09:41:05.376 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.376 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.376 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:41:05.376 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.451707+0000", 2024-09-16T09:41:05.376 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0028830180000000002 2024-09-16T09:41:05.376 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.377 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.377 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:41:05.377 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.465097+0000", 2024-09-16T09:41:05.377 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.013389961000000001 2024-09-16T09:41:05.377 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.377 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.377 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:41:05.377 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.465131+0000", 2024-09-16T09:41:05.377 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.4198e-05 2024-09-16T09:41:05.377 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.377 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.377 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:41:05.377 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.465161+0000", 2024-09-16T09:41:05.378 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.0015999999999999e-05 2024-09-16T09:41:05.378 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:41:05.378 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:41:05.378 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:41:05.378 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.378 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.378 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.25064.0:170610 8.b 8:daf1a32b:::benchmark_data_smithi070_107919_object46015:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e204)", 2024-09-16T09:41:05.378 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:32:07.504240+0000", 2024-09-16T09:41:05.378 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 537.81768525200005, 2024-09-16T09:41:05.378 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.96107343000000001, 2024-09-16T09:41:05.378 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:41:05.378 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:41:05.378 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:41:05.379 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:41:05.379 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.25064", 2024-09-16T09:41:05.379 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/2578256439", 2024-09-16T09:41:05.379 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 170610 2024-09-16T09:41:05.379 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.379 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:41:05.379 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.379 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:41:05.379 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.504240+0000", 2024-09-16T09:41:05.379 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:41:05.379 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.379 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.379 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:41:05.380 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.504240+0000", 2024-09-16T09:41:05.380 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:41:05.380 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.380 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.380 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:41:05.380 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.504240+0000", 2024-09-16T09:41:05.380 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.5099999999999999e-07 2024-09-16T09:41:05.380 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.380 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.380 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:41:05.380 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.504244+0000", 2024-09-16T09:41:05.380 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.4750000000000002e-06 2024-09-16T09:41:05.380 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.381 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.381 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:41:05.381 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.504253+0000", 2024-09-16T09:41:05.381 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.2110000000000004e-06 2024-09-16T09:41:05.381 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.381 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.381 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:41:05.381 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.504259+0000", 2024-09-16T09:41:05.381 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.003e-06 2024-09-16T09:41:05.381 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.381 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.381 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:41:05.382 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.505205+0000", 2024-09-16T09:41:05.382 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000945793 2024-09-16T09:41:05.382 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.382 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.382 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:41:05.382 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:07.505221+0000", 2024-09-16T09:41:05.382 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.6079999999999999e-05 2024-09-16T09:41:05.382 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.382 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.382 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:41:05.382 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.444807+0000", 2024-09-16T09:41:05.382 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.93958567500000001 2024-09-16T09:41:05.382 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.383 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.383 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:41:05.383 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.444917+0000", 2024-09-16T09:41:05.383 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000110856 2024-09-16T09:41:05.383 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.383 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.383 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 1,3", 2024-09-16T09:41:05.383 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.445032+0000", 2024-09-16T09:41:05.383 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000114125 2024-09-16T09:41:05.383 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.383 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.383 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:41:05.383 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.451674+0000", 2024-09-16T09:41:05.384 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0066422909999999998 2024-09-16T09:41:05.384 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.384 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.384 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:41:05.384 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.460207+0000", 2024-09-16T09:41:05.384 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0085334610000000009 2024-09-16T09:41:05.384 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.384 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.384 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:41:05.384 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.465215+0000", 2024-09-16T09:41:05.384 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0050073909999999999 2024-09-16T09:41:05.384 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.384 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.385 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:41:05.385 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.465267+0000", 2024-09-16T09:41:05.385 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.2315000000000003e-05 2024-09-16T09:41:05.385 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.385 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.385 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:41:05.385 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:32:08.465313+0000", 2024-09-16T09:41:05.385 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.6003e-05 2024-09-16T09:41:05.385 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:41:05.385 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:41:05.385 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:41:05.385 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.386 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.386 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:67145 9.16 9:6eb0471e:::benchmark_data_smithi070_137000_object67144:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e254)", 2024-09-16T09:41:05.386 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:34:11.477245+0000", 2024-09-16T09:41:05.386 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 413.84467946299998, 2024-09-16T09:41:05.386 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.00133326, 2024-09-16T09:41:05.386 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:41:05.386 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:41:05.386 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:41:05.386 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:41:05.386 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:41:05.386 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:41:05.386 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 67145 2024-09-16T09:41:05.386 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.387 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:41:05.387 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.387 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:41:05.387 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.477245+0000", 2024-09-16T09:41:05.387 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:41:05.387 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.387 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.387 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:41:05.387 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.477245+0000", 2024-09-16T09:41:05.387 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:41:05.387 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.387 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.387 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:41:05.388 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.477247+0000", 2024-09-16T09:41:05.388 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.2529999999999999e-06 2024-09-16T09:41:05.388 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.388 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.388 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:41:05.388 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.477323+0000", 2024-09-16T09:41:05.388 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.6688999999999997e-05 2024-09-16T09:41:05.388 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.388 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.388 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:41:05.388 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.477339+0000", 2024-09-16T09:41:05.388 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.5832999999999999e-05 2024-09-16T09:41:05.388 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.389 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.389 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:41:05.389 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.477346+0000", 2024-09-16T09:41:05.389 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.153e-06 2024-09-16T09:41:05.389 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.389 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.389 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:41:05.389 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.477404+0000", 2024-09-16T09:41:05.389 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.8041999999999998e-05 2024-09-16T09:41:05.389 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.389 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.389 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:41:05.389 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.477413+0000", 2024-09-16T09:41:05.389 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.5110000000000003e-06 2024-09-16T09:41:05.390 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.390 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.390 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:41:05.390 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.452134+0000", 2024-09-16T09:41:05.390 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.97472151100000004 2024-09-16T09:41:05.390 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.390 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.390 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:41:05.390 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.452270+0000", 2024-09-16T09:41:05.390 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00013596699999999999 2024-09-16T09:41:05.390 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.390 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.390 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 1,2", 2024-09-16T09:41:05.390 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.452413+0000", 2024-09-16T09:41:05.390 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000142884 2024-09-16T09:41:05.390 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.391 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.391 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:41:05.391 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.458265+0000", 2024-09-16T09:41:05.391 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0058516549999999999 2024-09-16T09:41:05.391 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.391 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.391 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:41:05.391 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.460714+0000", 2024-09-16T09:41:05.391 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0024490620000000001 2024-09-16T09:41:05.391 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.391 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.391 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:41:05.391 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.478484+0000", 2024-09-16T09:41:05.391 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.017769589999999998 2024-09-16T09:41:05.392 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.392 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.392 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:41:05.392 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.478535+0000", 2024-09-16T09:41:05.392 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.1928999999999997e-05 2024-09-16T09:41:05.392 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.392 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.392 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:41:05.392 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.478579+0000", 2024-09-16T09:41:05.392 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.3180999999999999e-05 2024-09-16T09:41:05.392 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:41:05.392 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:41:05.392 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:41:05.393 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.393 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.393 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:67148 9.16 9:6b4987c7:::benchmark_data_smithi070_137000_object67147:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e254)", 2024-09-16T09:41:05.393 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:34:11.477768+0000", 2024-09-16T09:41:05.393 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 413.84415708199998, 2024-09-16T09:41:05.393 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.001389544, 2024-09-16T09:41:05.393 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:41:05.393 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:41:05.393 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:41:05.393 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:41:05.393 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:41:05.393 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:41:05.393 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 67148 2024-09-16T09:41:05.393 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.394 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:41:05.394 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.394 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:41:05.394 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.477768+0000", 2024-09-16T09:41:05.394 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:41:05.394 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.394 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.394 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:41:05.394 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.477768+0000", 2024-09-16T09:41:05.394 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:41:05.394 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.394 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.394 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:41:05.395 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.477769+0000", 2024-09-16T09:41:05.395 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.7000000000000003e-07 2024-09-16T09:41:05.395 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.395 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.395 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:41:05.395 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.477849+0000", 2024-09-16T09:41:05.395 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.0500999999999994e-05 2024-09-16T09:41:05.395 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.395 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.395 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:41:05.396 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.477867+0000", 2024-09-16T09:41:05.396 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.8182e-05 2024-09-16T09:41:05.396 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.396 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.396 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:41:05.396 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.477876+0000", 2024-09-16T09:41:05.396 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.0900000000000005e-06 2024-09-16T09:41:05.396 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.396 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.396 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:41:05.396 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.478146+0000", 2024-09-16T09:41:05.396 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00027060400000000003 2024-09-16T09:41:05.396 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.397 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.397 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:41:05.397 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.478156+0000", 2024-09-16T09:41:05.397 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.7990000000000005e-06 2024-09-16T09:41:05.397 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.397 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.397 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:41:05.397 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.453564+0000", 2024-09-16T09:41:05.397 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.97540761799999998 2024-09-16T09:41:05.397 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.398 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.398 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:41:05.398 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.453683+0000", 2024-09-16T09:41:05.398 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000119882 2024-09-16T09:41:05.398 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.398 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.398 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 1,2", 2024-09-16T09:41:05.398 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.453825+0000", 2024-09-16T09:41:05.398 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000142006 2024-09-16T09:41:05.399 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.399 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.399 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:41:05.399 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.458271+0000", 2024-09-16T09:41:05.399 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0044451899999999999 2024-09-16T09:41:05.399 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.399 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.399 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:41:05.399 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.460761+0000", 2024-09-16T09:41:05.399 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0024901599999999999 2024-09-16T09:41:05.399 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.399 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.399 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:41:05.399 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.479057+0000", 2024-09-16T09:41:05.400 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.018296501 2024-09-16T09:41:05.400 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.400 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.400 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:41:05.400 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.479120+0000", 2024-09-16T09:41:05.400 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.3081000000000001e-05 2024-09-16T09:41:05.400 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.400 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.400 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:41:05.400 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.479157+0000", 2024-09-16T09:41:05.400 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.6959999999999998e-05 2024-09-16T09:41:05.400 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:41:05.400 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:41:05.401 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:41:05.401 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.401 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.401 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:67153 9.16 9:6b621114:::benchmark_data_smithi070_137000_object67152:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e254)", 2024-09-16T09:41:05.401 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:34:11.481611+0000", 2024-09-16T09:41:05.401 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 413.84031374800003, 2024-09-16T09:41:05.401 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.99775295500000005, 2024-09-16T09:41:05.401 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:41:05.401 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:41:05.401 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:41:05.401 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:41:05.401 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:41:05.401 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:41:05.401 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 67153 2024-09-16T09:41:05.402 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.402 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:41:05.402 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.402 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:41:05.402 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.481611+0000", 2024-09-16T09:41:05.402 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:41:05.402 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.402 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.402 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:41:05.402 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.481611+0000", 2024-09-16T09:41:05.402 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:41:05.402 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.402 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.402 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:41:05.403 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.481612+0000", 2024-09-16T09:41:05.403 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.1900000000000001e-07 2024-09-16T09:41:05.403 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.403 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.403 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:41:05.403 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.481650+0000", 2024-09-16T09:41:05.403 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.7489e-05 2024-09-16T09:41:05.404 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.404 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.404 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:41:05.404 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.481659+0000", 2024-09-16T09:41:05.404 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.4909999999999994e-06 2024-09-16T09:41:05.404 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.404 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.404 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:41:05.404 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.481666+0000", 2024-09-16T09:41:05.404 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.9809999999999997e-06 2024-09-16T09:41:05.404 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.404 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.405 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:41:05.405 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.481724+0000", 2024-09-16T09:41:05.405 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.8035000000000003e-05 2024-09-16T09:41:05.405 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.405 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.405 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:41:05.405 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.481733+0000", 2024-09-16T09:41:05.405 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.3489999999999998e-06 2024-09-16T09:41:05.405 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.405 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.405 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:41:05.405 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.455089+0000", 2024-09-16T09:41:05.405 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.97335562499999995 2024-09-16T09:41:05.406 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.406 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.406 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:41:05.406 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.455208+0000", 2024-09-16T09:41:05.406 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000119305 2024-09-16T09:41:05.406 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.406 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.406 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 1,2", 2024-09-16T09:41:05.406 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.455357+0000", 2024-09-16T09:41:05.406 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00014880000000000001 2024-09-16T09:41:05.406 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.406 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.406 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:41:05.407 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.459587+0000", 2024-09-16T09:41:05.407 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0042294289999999998 2024-09-16T09:41:05.407 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.407 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.407 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:41:05.407 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.460813+0000", 2024-09-16T09:41:05.407 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0012265399999999999 2024-09-16T09:41:05.407 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.407 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.407 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:41:05.408 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.479275+0000", 2024-09-16T09:41:05.408 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.018462135000000001 2024-09-16T09:41:05.408 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.408 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.408 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:41:05.408 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.479321+0000", 2024-09-16T09:41:05.408 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.6267000000000003e-05 2024-09-16T09:41:05.408 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.408 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.408 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:41:05.408 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.479364+0000", 2024-09-16T09:41:05.408 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.2589999999999997e-05 2024-09-16T09:41:05.408 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:41:05.409 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:41:05.409 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:41:05.409 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.409 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.409 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:67177 9.16 9:6c8c3aa4:::benchmark_data_smithi070_137000_object67176:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e254)", 2024-09-16T09:41:05.409 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:34:11.503406+0000", 2024-09-16T09:41:05.409 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 413.81851886499999, 2024-09-16T09:41:05.409 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.97608242499999998, 2024-09-16T09:41:05.409 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:41:05.409 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:41:05.409 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:41:05.409 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:41:05.409 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:41:05.409 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:41:05.410 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 67177 2024-09-16T09:41:05.410 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.410 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:41:05.410 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.410 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:41:05.410 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.503406+0000", 2024-09-16T09:41:05.410 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:41:05.410 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.410 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.410 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:41:05.410 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.503406+0000", 2024-09-16T09:41:05.411 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:41:05.411 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.411 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.411 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:41:05.411 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.503406+0000", 2024-09-16T09:41:05.411 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.01e-07 2024-09-16T09:41:05.411 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.411 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.411 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:41:05.411 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.503432+0000", 2024-09-16T09:41:05.411 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.5673e-05 2024-09-16T09:41:05.411 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.411 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.412 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:41:05.412 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.503442+0000", 2024-09-16T09:41:05.412 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.7040000000000006e-06 2024-09-16T09:41:05.412 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.412 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.412 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:41:05.412 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.503445+0000", 2024-09-16T09:41:05.412 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.6339999999999999e-06 2024-09-16T09:41:05.412 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.412 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.412 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:41:05.412 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.503516+0000", 2024-09-16T09:41:05.412 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.0438999999999994e-05 2024-09-16T09:41:05.412 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.413 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.413 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:41:05.413 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.503528+0000", 2024-09-16T09:41:05.413 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.2349e-05 2024-09-16T09:41:05.413 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.413 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.413 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:41:05.413 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.456887+0000", 2024-09-16T09:41:05.413 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.95335896399999998 2024-09-16T09:41:05.413 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.413 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.413 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:41:05.413 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.457028+0000", 2024-09-16T09:41:05.413 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00014072600000000001 2024-09-16T09:41:05.414 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.414 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.414 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 1,2", 2024-09-16T09:41:05.414 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.457172+0000", 2024-09-16T09:41:05.414 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000144389 2024-09-16T09:41:05.414 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.414 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.414 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:41:05.414 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.460657+0000", 2024-09-16T09:41:05.414 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0034844250000000002 2024-09-16T09:41:05.414 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.414 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.414 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:41:05.415 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.460862+0000", 2024-09-16T09:41:05.415 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00020545900000000001 2024-09-16T09:41:05.415 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.415 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.415 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:41:05.415 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.479405+0000", 2024-09-16T09:41:05.415 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.018543343 2024-09-16T09:41:05.415 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.415 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.415 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:41:05.415 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.479458+0000", 2024-09-16T09:41:05.415 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.2652999999999999e-05 2024-09-16T09:41:05.415 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.415 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.415 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:41:05.416 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.479488+0000", 2024-09-16T09:41:05.416 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.0266000000000001e-05 2024-09-16T09:41:05.416 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:41:05.416 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:41:05.416 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:41:05.416 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.416 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.416 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:67186 9.16 9:6ddb896c:::benchmark_data_smithi070_137000_object67185:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e254)", 2024-09-16T09:41:05.416 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:34:11.514674+0000", 2024-09-16T09:41:05.416 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 413.80725085300003, 2024-09-16T09:41:05.416 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.96492478400000004, 2024-09-16T09:41:05.416 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:41:05.416 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:41:05.416 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:41:05.417 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:41:05.417 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:41:05.417 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:41:05.417 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 67186 2024-09-16T09:41:05.417 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.417 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:41:05.417 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.417 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:41:05.417 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.514674+0000", 2024-09-16T09:41:05.417 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:41:05.417 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.417 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.417 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:41:05.418 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.514674+0000", 2024-09-16T09:41:05.418 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:41:05.418 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.418 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.418 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:41:05.418 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.514676+0000", 2024-09-16T09:41:05.418 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.9690000000000001e-06 2024-09-16T09:41:05.418 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.418 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.418 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:41:05.418 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.514846+0000", 2024-09-16T09:41:05.418 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00016989199999999999 2024-09-16T09:41:05.418 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.418 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.419 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:41:05.419 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.514862+0000", 2024-09-16T09:41:05.419 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.6317999999999999e-05 2024-09-16T09:41:05.419 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.419 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.419 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:41:05.419 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.514870+0000", 2024-09-16T09:41:05.419 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.1699999999999997e-06 2024-09-16T09:41:05.419 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.419 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.419 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:41:05.419 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.514942+0000", 2024-09-16T09:41:05.419 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.1619999999999995e-05 2024-09-16T09:41:05.419 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.420 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.420 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:41:05.420 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:11.514952+0000", 2024-09-16T09:41:05.420 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.577e-06 2024-09-16T09:41:05.420 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.420 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.420 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:41:05.420 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.458305+0000", 2024-09-16T09:41:05.420 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.94335340499999998 2024-09-16T09:41:05.420 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.420 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.420 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:41:05.420 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.458454+0000", 2024-09-16T09:41:05.420 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00014879499999999999 2024-09-16T09:41:05.421 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.421 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.421 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 1,2", 2024-09-16T09:41:05.421 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.458547+0000", 2024-09-16T09:41:05.421 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.3602000000000003e-05 2024-09-16T09:41:05.421 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.421 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.421 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:41:05.421 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.460826+0000", 2024-09-16T09:41:05.421 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0022790449999999999 2024-09-16T09:41:05.421 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.421 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.421 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:41:05.421 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.466858+0000", 2024-09-16T09:41:05.422 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0060317620000000004 2024-09-16T09:41:05.422 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.422 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.422 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:41:05.422 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.479526+0000", 2024-09-16T09:41:05.422 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.012668008 2024-09-16T09:41:05.422 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.422 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.422 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:41:05.422 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.479570+0000", 2024-09-16T09:41:05.422 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.3455000000000003e-05 2024-09-16T09:41:05.422 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.422 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.422 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:41:05.422 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:34:12.479599+0000", 2024-09-16T09:41:05.423 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.9166000000000002e-05 2024-09-16T09:41:05.423 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:41:05.423 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:41:05.423 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:41:05.423 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.423 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.423 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:231311 9.14 9:2a6a4bc0:::benchmark_data_smithi070_137000_object109600:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e357)", 2024-09-16T09:41:05.423 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:37:18.512297+0000", 2024-09-16T09:41:05.423 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 226.80962746099999, 2024-09-16T09:41:05.423 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0604506849999999, 2024-09-16T09:41:05.423 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:41:05.423 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:41:05.423 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "reached pg", 2024-09-16T09:41:05.424 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:41:05.424 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:41:05.424 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:41:05.424 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 231311 2024-09-16T09:41:05.424 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.424 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:41:05.424 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.424 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:41:05.424 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.512297+0000", 2024-09-16T09:41:05.424 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:41:05.424 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.424 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.424 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:41:05.424 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.512297+0000", 2024-09-16T09:41:05.424 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:41:05.425 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.425 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.425 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:41:05.425 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.512300+0000", 2024-09-16T09:41:05.425 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.3499999999999999e-06 2024-09-16T09:41:05.425 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.425 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.425 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:41:05.425 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.512306+0000", 2024-09-16T09:41:05.425 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.246e-06 2024-09-16T09:41:05.425 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.425 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.425 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:41:05.425 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.512327+0000", 2024-09-16T09:41:05.426 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.0866e-05 2024-09-16T09:41:05.426 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.426 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.426 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:41:05.426 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.512337+0000", 2024-09-16T09:41:05.426 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.7780000000000002e-06 2024-09-16T09:41:05.426 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.426 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.426 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:41:05.426 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.633134+0000", 2024-09-16T09:41:05.426 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.120797824 2024-09-16T09:41:05.426 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.426 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.427 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:41:05.427 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.633143+0000", 2024-09-16T09:41:05.427 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.6379999999999993e-06 2024-09-16T09:41:05.427 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.427 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.427 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:41:05.427 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.572704+0000", 2024-09-16T09:41:05.427 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.93956075500000003 2024-09-16T09:41:05.427 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.427 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.427 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:41:05.427 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.572748+0000", 2024-09-16T09:41:05.427 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.4227999999999997e-05 2024-09-16T09:41:05.427 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:41:05.427 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:41:05.428 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:41:05.428 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.428 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.428 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:231323 9.4 9:248ba587:::benchmark_data_smithi070_137000_object109612:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e357)", 2024-09-16T09:41:05.428 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:37:18.512353+0000", 2024-09-16T09:41:05.428 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 226.80957194600001, 2024-09-16T09:41:05.428 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.057531448, 2024-09-16T09:41:05.428 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:41:05.428 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:41:05.428 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:41:05.428 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:41:05.428 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:41:05.428 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:41:05.428 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 231323 2024-09-16T09:41:05.429 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.429 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:41:05.429 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.429 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:41:05.429 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.512353+0000", 2024-09-16T09:41:05.429 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:41:05.429 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.429 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.429 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:41:05.429 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.512353+0000", 2024-09-16T09:41:05.429 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:41:05.429 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.429 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.429 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:41:05.430 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.512353+0000", 2024-09-16T09:41:05.430 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.2500000000000001e-07 2024-09-16T09:41:05.430 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.430 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.430 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:41:05.430 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.512356+0000", 2024-09-16T09:41:05.430 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.3539999999999998e-06 2024-09-16T09:41:05.430 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.430 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.430 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:41:05.430 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.512366+0000", 2024-09-16T09:41:05.430 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.9310000000000003e-06 2024-09-16T09:41:05.430 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.430 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.431 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:41:05.431 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.512371+0000", 2024-09-16T09:41:05.431 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.2850000000000002e-06 2024-09-16T09:41:05.431 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.431 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.431 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:41:05.431 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.633167+0000", 2024-09-16T09:41:05.431 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.12079590699999999 2024-09-16T09:41:05.431 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.431 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.431 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:41:05.431 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.633174+0000", 2024-09-16T09:41:05.431 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.2359999999999998e-06 2024-09-16T09:41:05.431 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.432 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.432 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:41:05.432 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.552690+0000", 2024-09-16T09:41:05.432 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.91951564699999999 2024-09-16T09:41:05.432 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.432 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.432 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:41:05.432 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.552776+0000", 2024-09-16T09:41:05.432 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.6244999999999998e-05 2024-09-16T09:41:05.432 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.432 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.432 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 1,3", 2024-09-16T09:41:05.432 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.552844+0000", 2024-09-16T09:41:05.433 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.7615999999999998e-05 2024-09-16T09:41:05.433 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.433 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.433 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:41:05.433 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.555525+0000", 2024-09-16T09:41:05.433 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0026812189999999999 2024-09-16T09:41:05.433 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.433 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.433 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:41:05.433 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.568683+0000", 2024-09-16T09:41:05.433 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.013158433000000001 2024-09-16T09:41:05.433 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.434 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.434 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:41:05.434 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.569788+0000", 2024-09-16T09:41:05.434 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00110486 2024-09-16T09:41:05.434 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.434 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.434 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:41:05.434 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.569844+0000", 2024-09-16T09:41:05.434 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.5894999999999999e-05 2024-09-16T09:41:05.434 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.434 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.434 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:41:05.434 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.569884+0000", 2024-09-16T09:41:05.434 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.0395000000000002e-05 2024-09-16T09:41:05.435 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:41:05.435 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:41:05.435 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:41:05.435 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.435 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.435 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:232744 9.16 9.f8cf6db6 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e359)", 2024-09-16T09:41:05.435 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:37:20.541411+0000", 2024-09-16T09:41:05.435 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 224.780513789, 2024-09-16T09:41:05.435 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.97886637600000004, 2024-09-16T09:41:05.435 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:41:05.435 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:41:05.435 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "reached pg", 2024-09-16T09:41:05.435 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:41:05.435 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:41:05.436 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:41:05.436 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 232744 2024-09-16T09:41:05.436 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.436 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:41:05.436 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.436 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:41:05.436 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.541411+0000", 2024-09-16T09:41:05.436 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:41:05.436 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.436 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.436 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:41:05.436 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.541411+0000", 2024-09-16T09:41:05.436 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:41:05.437 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.437 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.437 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:41:05.437 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.541413+0000", 2024-09-16T09:41:05.437 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.0899999999999999e-06 2024-09-16T09:41:05.437 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.437 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.437 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:41:05.437 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.541421+0000", 2024-09-16T09:41:05.437 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.5059999999999998e-06 2024-09-16T09:41:05.437 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.437 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.437 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:41:05.437 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.541444+0000", 2024-09-16T09:41:05.438 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.3776000000000001e-05 2024-09-16T09:41:05.438 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.438 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.438 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:41:05.438 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.541455+0000", 2024-09-16T09:41:05.439 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0285999999999999e-05 2024-09-16T09:41:05.439 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.439 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.439 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:41:05.439 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.542060+0000", 2024-09-16T09:41:05.440 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00060544400000000001 2024-09-16T09:41:05.440 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.440 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.440 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:41:05.440 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.542073+0000", 2024-09-16T09:41:05.440 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.2604e-05 2024-09-16T09:41:05.440 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.440 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.440 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:41:05.440 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:21.520267+0000", 2024-09-16T09:41:05.440 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.97819373099999996 2024-09-16T09:41:05.440 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.441 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.441 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:41:05.441 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:21.520277+0000", 2024-09-16T09:41:05.441 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0939e-05 2024-09-16T09:41:05.441 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:41:05.441 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:41:05.441 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:41:05.441 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.441 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.441 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:232750 9.10 9.b4158f10 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e359)", 2024-09-16T09:41:05.441 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:37:20.544912+0000", 2024-09-16T09:41:05.441 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 224.77701237700001, 2024-09-16T09:41:05.441 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.97432714600000003, 2024-09-16T09:41:05.441 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:41:05.442 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:41:05.442 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "reached pg", 2024-09-16T09:41:05.442 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:41:05.442 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:41:05.442 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:41:05.442 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 232750 2024-09-16T09:41:05.442 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.442 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:41:05.442 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.442 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:41:05.442 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.544912+0000", 2024-09-16T09:41:05.442 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:41:05.442 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.443 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.443 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:41:05.443 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.544912+0000", 2024-09-16T09:41:05.443 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:41:05.443 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.443 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.443 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:41:05.443 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.544913+0000", 2024-09-16T09:41:05.443 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.6899999999999996e-07 2024-09-16T09:41:05.443 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.444 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.444 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:41:05.444 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.544918+0000", 2024-09-16T09:41:05.444 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.8609999999999997e-06 2024-09-16T09:41:05.444 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.444 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.444 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:41:05.444 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.544936+0000", 2024-09-16T09:41:05.444 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.7781000000000001e-05 2024-09-16T09:41:05.444 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.444 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.444 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:41:05.445 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.544945+0000", 2024-09-16T09:41:05.445 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.5480000000000002e-06 2024-09-16T09:41:05.445 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.445 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.445 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:41:05.445 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.545003+0000", 2024-09-16T09:41:05.445 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.8072000000000002e-05 2024-09-16T09:41:05.445 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.445 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.445 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:41:05.445 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:20.545013+0000", 2024-09-16T09:41:05.446 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.049e-05 2024-09-16T09:41:05.446 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.446 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.446 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:41:05.446 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:21.519228+0000", 2024-09-16T09:41:05.446 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.97421449199999999 2024-09-16T09:41:05.446 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.446 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.446 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:41:05.446 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:21.519240+0000", 2024-09-16T09:41:05.446 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1933000000000001e-05 2024-09-16T09:41:05.447 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:41:05.447 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:41:05.447 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:41:05.447 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.447 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.447 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15975.0:84074 10.10 10:0e64316a:::benchmark_data_smithi070_166744_object84073:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e404)", 2024-09-16T09:41:05.447 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:38:53.381069+0000", 2024-09-16T09:41:05.447 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 131.94085584999999, 2024-09-16T09:41:05.447 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.012507338, 2024-09-16T09:41:05.447 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:41:05.447 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:41:05.447 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "reached pg", 2024-09-16T09:41:05.448 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:41:05.448 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15975", 2024-09-16T09:41:05.448 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3821727158", 2024-09-16T09:41:05.448 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 84074 2024-09-16T09:41:05.448 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.448 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:41:05.448 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.448 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:41:05.448 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:38:53.381069+0000", 2024-09-16T09:41:05.448 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:41:05.448 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.448 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.449 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:41:05.449 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:38:53.381069+0000", 2024-09-16T09:41:05.449 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:41:05.449 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.449 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.449 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:41:05.449 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:38:53.381070+0000", 2024-09-16T09:41:05.449 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.4309999999999999e-06 2024-09-16T09:41:05.449 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.449 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.449 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:41:05.449 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:38:53.381098+0000", 2024-09-16T09:41:05.449 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.7946e-05 2024-09-16T09:41:05.450 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.450 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.450 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:41:05.450 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:38:53.381109+0000", 2024-09-16T09:41:05.450 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1077000000000001e-05 2024-09-16T09:41:05.450 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.450 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.450 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:41:05.450 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:38:53.381116+0000", 2024-09-16T09:41:05.450 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.9500000000000004e-06 2024-09-16T09:41:05.450 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.451 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.451 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:41:05.451 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:38:53.381226+0000", 2024-09-16T09:41:05.451 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00010962400000000001 2024-09-16T09:41:05.451 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.451 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.451 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:41:05.451 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:38:53.381545+0000", 2024-09-16T09:41:05.451 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00031932 2024-09-16T09:41:05.451 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.452 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.452 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 6,7", 2024-09-16T09:41:05.452 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:38:53.381924+0000", 2024-09-16T09:41:05.452 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:41:05.454 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00037895499999999999 2024-09-16T09:41:05.454 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.454 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.454 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:41:05.454 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:38:53.386409+0000", 2024-09-16T09:41:05.454 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0044849410000000001 2024-09-16T09:41:05.454 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.454 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.454 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:41:05.454 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:38:53.396250+0000", 2024-09-16T09:41:05.455 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0098403080000000007 2024-09-16T09:41:05.455 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.455 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.455 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:41:05.455 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:38:53.396259+0000", 2024-09-16T09:41:05.455 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.2089999999999994e-06 2024-09-16T09:41:05.455 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.455 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.455 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:41:05.455 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:38:54.393527+0000", 2024-09-16T09:41:05.455 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.997268457 2024-09-16T09:41:05.455 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.456 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.456 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:41:05.456 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:38:54.393576+0000", 2024-09-16T09:41:05.456 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.9119999999999997e-05 2024-09-16T09:41:05.456 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:41:05.456 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:41:05.456 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:41:05.456 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.456 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.456 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15975.0:84112 10.10 10:0bc72f94:::benchmark_data_smithi070_166744_object84111:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e405)", 2024-09-16T09:41:05.456 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:38:53.428548+0000", 2024-09-16T09:41:05.456 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 131.89337712400001, 2024-09-16T09:41:05.456 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.97027836199999995, 2024-09-16T09:41:05.457 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:41:05.457 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:41:05.457 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:41:05.457 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:41:05.457 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15975", 2024-09-16T09:41:05.457 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3821727158", 2024-09-16T09:41:05.457 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 84112 2024-09-16T09:41:05.457 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.457 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:41:05.457 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.457 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:41:05.457 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:38:53.428548+0000", 2024-09-16T09:41:05.458 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:41:05.458 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.458 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.458 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:41:05.458 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:38:53.428548+0000", 2024-09-16T09:41:05.458 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:41:05.458 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.458 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.458 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:41:05.458 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:38:53.428549+0000", 2024-09-16T09:41:05.458 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.06e-06 2024-09-16T09:41:05.459 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.459 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.459 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:41:05.459 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:38:53.428605+0000", 2024-09-16T09:41:05.459 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.5831000000000001e-05 2024-09-16T09:41:05.459 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.459 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.459 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:41:05.459 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:38:53.428615+0000", 2024-09-16T09:41:05.459 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0212e-05 2024-09-16T09:41:05.459 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.459 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.459 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:41:05.459 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:38:53.428620+0000", 2024-09-16T09:41:05.460 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.198e-06 2024-09-16T09:41:05.460 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.460 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.460 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:41:05.460 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:38:53.428675+0000", 2024-09-16T09:41:05.460 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.5402999999999998e-05 2024-09-16T09:41:05.460 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.460 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.460 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:41:05.460 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:38:53.428684+0000", 2024-09-16T09:41:05.460 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.5979999999999997e-06 2024-09-16T09:41:05.460 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.460 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.460 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:41:05.460 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:38:54.393977+0000", 2024-09-16T09:41:05.461 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.96529284599999998 2024-09-16T09:41:05.461 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.461 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.461 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:41:05.461 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:38:54.394100+0000", 2024-09-16T09:41:05.461 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00012338299999999999 2024-09-16T09:41:05.461 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.461 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.461 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 6,7", 2024-09-16T09:41:05.461 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:38:54.394293+0000", 2024-09-16T09:41:05.461 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00019298300000000001 2024-09-16T09:41:05.461 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.461 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.461 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:41:05.462 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:38:54.398602+0000", 2024-09-16T09:41:05.462 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0043089010000000004 2024-09-16T09:41:05.462 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.462 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.462 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:41:05.462 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:38:54.398703+0000", 2024-09-16T09:41:05.462 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00010066800000000001 2024-09-16T09:41:05.462 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.462 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.462 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:41:05.462 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:38:54.398735+0000", 2024-09-16T09:41:05.462 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.2060999999999997e-05 2024-09-16T09:41:05.462 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.463 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.463 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:41:05.463 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:38:54.398787+0000", 2024-09-16T09:41:05.463 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.2024000000000002e-05 2024-09-16T09:41:05.463 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.463 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.463 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:41:05.463 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:38:54.398826+0000", 2024-09-16T09:41:05.463 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.9193999999999998e-05 2024-09-16T09:41:05.463 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:41:05.463 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:41:05.463 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:41:05.463 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.464 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.464 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15975.0:173406 10.19 10:9a3334ef:::benchmark_data_smithi070_166744_object59050:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e439)", 2024-09-16T09:41:05.464 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:40:21.440051+0000", 2024-09-16T09:41:05.464 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 43.881874252999999, 2024-09-16T09:41:05.464 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.00408823, 2024-09-16T09:41:05.464 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:41:05.464 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:41:05.464 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:41:05.464 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:41:05.464 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15975", 2024-09-16T09:41:05.464 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3821727158", 2024-09-16T09:41:05.464 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 173406 2024-09-16T09:41:05.464 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.464 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:41:05.465 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.465 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:41:05.465 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:40:21.440051+0000", 2024-09-16T09:41:05.465 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:41:05.465 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.465 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.465 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:41:05.465 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:40:21.440051+0000", 2024-09-16T09:41:05.465 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:41:05.465 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.465 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.465 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:41:05.465 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:40:21.440052+0000", 2024-09-16T09:41:05.465 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.2109999999999999e-06 2024-09-16T09:41:05.465 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.466 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.466 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:41:05.466 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:40:21.440056+0000", 2024-09-16T09:41:05.466 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.8500000000000004e-06 2024-09-16T09:41:05.466 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.466 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.466 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:41:05.466 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:40:21.440065+0000", 2024-09-16T09:41:05.466 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.3309999999999993e-06 2024-09-16T09:41:05.466 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.466 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.466 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:41:05.466 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:40:21.440071+0000", 2024-09-16T09:41:05.466 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.8030000000000002e-06 2024-09-16T09:41:05.466 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.467 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.467 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:41:05.467 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:40:21.440158+0000", 2024-09-16T09:41:05.467 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.7175000000000005e-05 2024-09-16T09:41:05.467 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.467 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.467 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:41:05.467 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:40:21.440169+0000", 2024-09-16T09:41:05.467 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0729e-05 2024-09-16T09:41:05.467 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.467 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.467 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:41:05.467 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:40:22.422412+0000", 2024-09-16T09:41:05.467 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.98224296700000002 2024-09-16T09:41:05.467 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.467 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.468 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:41:05.468 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:40:22.422493+0000", 2024-09-16T09:41:05.468 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.1205000000000006e-05 2024-09-16T09:41:05.468 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.468 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.468 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 1,3", 2024-09-16T09:41:05.468 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:40:22.422692+0000", 2024-09-16T09:41:05.468 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00019887 2024-09-16T09:41:05.468 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.468 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.468 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:41:05.468 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:40:22.441532+0000", 2024-09-16T09:41:05.468 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.018839999 2024-09-16T09:41:05.469 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.469 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.469 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:41:05.469 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:40:22.441587+0000", 2024-09-16T09:41:05.469 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.5269999999999998e-05 2024-09-16T09:41:05.469 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.469 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.469 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:41:05.469 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:40:22.444068+0000", 2024-09-16T09:41:05.469 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.002480786 2024-09-16T09:41:05.469 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.469 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.469 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:41:05.469 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:40:22.444098+0000", 2024-09-16T09:41:05.469 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.0409999999999999e-05 2024-09-16T09:41:05.470 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.470 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.470 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:41:05.470 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:40:22.444139+0000", 2024-09-16T09:41:05.470 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.0624000000000003e-05 2024-09-16T09:41:05.470 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:41:05.470 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:41:05.470 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:41:05.470 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.470 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.470 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15975.0:173424 10.19 10:9bfc77f5:::benchmark_data_smithi070_166744_object59068:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e439)", 2024-09-16T09:41:05.470 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:40:21.453477+0000", 2024-09-16T09:41:05.470 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 43.868448139000002, 2024-09-16T09:41:05.470 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.99297290900000001, 2024-09-16T09:41:05.471 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:41:05.471 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:41:05.471 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:41:05.471 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:41:05.471 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15975", 2024-09-16T09:41:05.471 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3821727158", 2024-09-16T09:41:05.471 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 173424 2024-09-16T09:41:05.471 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.471 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:41:05.471 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.471 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:41:05.471 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:40:21.453477+0000", 2024-09-16T09:41:05.471 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:41:05.472 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.472 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.472 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:41:05.472 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:40:21.453477+0000", 2024-09-16T09:41:05.472 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:41:05.472 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.472 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.472 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:41:05.472 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:40:21.453477+0000", 2024-09-16T09:41:05.472 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.0200000000000003e-07 2024-09-16T09:41:05.472 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.472 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.473 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:41:05.473 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:40:21.453479+0000", 2024-09-16T09:41:05.473 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.99e-06 2024-09-16T09:41:05.473 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.473 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.473 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:41:05.473 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:40:21.453485+0000", 2024-09-16T09:41:05.473 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.2500000000000003e-06 2024-09-16T09:41:05.473 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.473 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.473 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:41:05.474 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:40:21.453491+0000", 2024-09-16T09:41:05.474 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.4090000000000002e-06 2024-09-16T09:41:05.474 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.474 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.474 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:41:05.474 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:40:21.453533+0000", 2024-09-16T09:41:05.474 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.2375e-05 2024-09-16T09:41:05.474 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.474 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.474 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:41:05.474 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:40:21.453539+0000", 2024-09-16T09:41:05.474 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.3249999999999996e-06 2024-09-16T09:41:05.474 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.474 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.475 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:41:05.475 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:40:22.430424+0000", 2024-09-16T09:41:05.475 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.97688414499999998 2024-09-16T09:41:05.475 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.475 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.475 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:41:05.475 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:40:22.430496+0000", 2024-09-16T09:41:05.475 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.2340000000000002e-05 2024-09-16T09:41:05.475 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.475 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.475 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 1,3", 2024-09-16T09:41:05.475 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:40:22.430694+0000", 2024-09-16T09:41:05.475 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00019780700000000001 2024-09-16T09:41:05.476 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.476 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.476 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:41:05.476 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:40:22.441577+0000", 2024-09-16T09:41:05.476 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.01088335 2024-09-16T09:41:05.476 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.476 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.476 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:41:05.476 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:40:22.446337+0000", 2024-09-16T09:41:05.476 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0047598229999999998 2024-09-16T09:41:05.476 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.476 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.476 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:41:05.477 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:40:22.446381+0000", 2024-09-16T09:41:05.477 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.4125999999999997e-05 2024-09-16T09:41:05.477 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.477 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.477 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:41:05.477 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:40:22.446410+0000", 2024-09-16T09:41:05.477 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.8697999999999999e-05 2024-09-16T09:41:05.477 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.477 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.477 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:41:05.477 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:40:22.446450+0000", 2024-09-16T09:41:05.477 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.9869000000000001e-05 2024-09-16T09:41:05.477 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:41:05.477 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:41:05.478 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:41:05.478 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.478 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.478 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15975.0:173462 10.19 10:9f462f3d:::benchmark_data_smithi070_166744_object59106:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e439)", 2024-09-16T09:41:05.478 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:40:21.484450+0000", 2024-09-16T09:41:05.478 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 43.837474493999999, 2024-09-16T09:41:05.478 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.96564167400000001, 2024-09-16T09:41:05.478 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:41:05.478 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:41:05.478 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:41:05.478 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:41:05.478 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15975", 2024-09-16T09:41:05.478 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3821727158", 2024-09-16T09:41:05.478 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 173462 2024-09-16T09:41:05.479 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.479 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:41:05.479 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.479 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:41:05.479 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:40:21.484450+0000", 2024-09-16T09:41:05.479 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:41:05.479 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.479 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.479 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:41:05.479 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:40:21.484450+0000", 2024-09-16T09:41:05.479 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:41:05.479 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.479 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.479 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:41:05.479 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:40:21.484451+0000", 2024-09-16T09:41:05.480 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.4399999999999999e-07 2024-09-16T09:41:05.480 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.480 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.480 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:41:05.480 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:40:21.484456+0000", 2024-09-16T09:41:05.480 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.3869999999999997e-06 2024-09-16T09:41:05.480 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.480 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.480 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:41:05.480 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:40:21.484489+0000", 2024-09-16T09:41:05.480 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.3890000000000002e-05 2024-09-16T09:41:05.480 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.480 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.480 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:41:05.481 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:40:21.484498+0000", 2024-09-16T09:41:05.481 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.0989999999999999e-06 2024-09-16T09:41:05.481 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.481 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.481 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:41:05.481 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:40:21.484586+0000", 2024-09-16T09:41:05.481 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.8753999999999998e-05 2024-09-16T09:41:05.481 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.481 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.481 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:41:05.481 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:40:21.484596+0000", 2024-09-16T09:41:05.481 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.2690000000000005e-06 2024-09-16T09:41:05.481 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.481 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.482 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:41:05.482 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:40:22.431566+0000", 2024-09-16T09:41:05.482 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.94697031499999995 2024-09-16T09:41:05.482 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.482 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.482 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:41:05.482 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:40:22.431632+0000", 2024-09-16T09:41:05.482 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.6319e-05 2024-09-16T09:41:05.482 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.482 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.482 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 1,3", 2024-09-16T09:41:05.482 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:40:22.431813+0000", 2024-09-16T09:41:05.482 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000181084 2024-09-16T09:41:05.483 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.483 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.483 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:41:05.483 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:40:22.446374+0000", 2024-09-16T09:41:05.483 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.014560506000000001 2024-09-16T09:41:05.483 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.483 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.483 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:41:05.483 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:40:22.446454+0000", 2024-09-16T09:41:05.483 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.0496999999999999e-05 2024-09-16T09:41:05.483 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.483 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.483 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:41:05.483 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:40:22.450031+0000", 2024-09-16T09:41:05.484 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0035768470000000002 2024-09-16T09:41:05.484 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.484 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.484 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:41:05.484 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:40:22.450065+0000", 2024-09-16T09:41:05.484 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.43e-05 2024-09-16T09:41:05.484 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.484 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.484 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:41:05.484 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:40:22.450092+0000", 2024-09-16T09:41:05.484 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.6563e-05 2024-09-16T09:41:05.484 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:41:05.484 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:41:05.484 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:41:05.484 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.485 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.485 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15975.0:173480 10.19 10:98ae6f1b:::benchmark_data_smithi070_166744_object59124:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e439)", 2024-09-16T09:41:05.485 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:40:21.500171+0000", 2024-09-16T09:41:05.485 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 43.821754083000002, 2024-09-16T09:41:05.485 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.96247617299999999, 2024-09-16T09:41:05.485 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:41:05.485 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:41:05.485 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:41:05.485 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:41:05.485 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15975", 2024-09-16T09:41:05.485 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3821727158", 2024-09-16T09:41:05.485 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 173480 2024-09-16T09:41:05.485 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.485 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:41:05.486 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.486 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:41:05.486 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:40:21.500171+0000", 2024-09-16T09:41:05.486 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:41:05.486 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.486 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.486 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:41:05.486 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:40:21.500171+0000", 2024-09-16T09:41:05.486 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:41:05.486 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.486 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.486 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:41:05.486 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:40:21.500172+0000", 2024-09-16T09:41:05.486 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.043e-06 2024-09-16T09:41:05.486 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.486 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.487 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:41:05.487 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:40:21.500176+0000", 2024-09-16T09:41:05.487 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.4490000000000001e-06 2024-09-16T09:41:05.487 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.487 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.487 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:41:05.487 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:40:21.500189+0000", 2024-09-16T09:41:05.487 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.2310000000000001e-05 2024-09-16T09:41:05.487 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.487 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.487 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:41:05.487 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:40:21.500195+0000", 2024-09-16T09:41:05.487 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.55e-06 2024-09-16T09:41:05.487 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.487 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.488 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:41:05.488 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:40:21.502000+0000", 2024-09-16T09:41:05.488 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00180471 2024-09-16T09:41:05.488 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.488 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.488 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:41:05.488 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:40:21.502010+0000", 2024-09-16T09:41:05.488 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.8020000000000003e-06 2024-09-16T09:41:05.488 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.488 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.488 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:41:05.488 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:40:22.432623+0000", 2024-09-16T09:41:05.488 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.93061301299999999 2024-09-16T09:41:05.488 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.489 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.489 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:41:05.489 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:40:22.432686+0000", 2024-09-16T09:41:05.489 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.3200000000000005e-05 2024-09-16T09:41:05.489 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.489 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.489 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 1,3", 2024-09-16T09:41:05.489 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:40:22.432834+0000", 2024-09-16T09:41:05.489 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00014786000000000001 2024-09-16T09:41:05.489 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.489 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.489 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:41:05.489 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:40:22.446457+0000", 2024-09-16T09:41:05.489 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.013623217 2024-09-16T09:41:05.490 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.490 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.490 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:41:05.490 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:40:22.450137+0000", 2024-09-16T09:41:05.490 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0036805029999999999 2024-09-16T09:41:05.490 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.490 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.490 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:41:05.490 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:40:22.462546+0000", 2024-09-16T09:41:05.490 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.012408957 2024-09-16T09:41:05.490 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.490 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.490 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:41:05.490 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:40:22.462608+0000", 2024-09-16T09:41:05.490 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.1382000000000003e-05 2024-09-16T09:41:05.491 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:05.491 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:05.491 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:41:05.491 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:40:22.462647+0000", 2024-09-16T09:41:05.491 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.9177000000000001e-05 2024-09-16T09:41:05.491 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:41:05.491 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:41:05.491 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:41:05.491 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:41:05.491 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:41:05.491 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-09-16T09:41:05.513 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:05.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:05.514 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:41:05.571 INFO:tasks.cephadm.ceph_manager.ceph:removing pool_name unique_pool_3 2024-09-16T09:41:05.571 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd pool rm unique_pool_3 unique_pool_3 --yes-i-really-really-mean-it 2024-09-16T09:41:05.615 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:41:05.636 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:05.636 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:05.637 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:41:05.737 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:41:05.759 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:05.759 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:05.760 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:41:05.861 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:41:05.882 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:05.883 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:05.883 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:41:05.984 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:41:05.997 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T09:41:05.997 INFO:teuthology.orchestra.run.smithi070.stdout: "success": "" 2024-09-16T09:41:05.998 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T09:41:06.005 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:06.006 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:06.006 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:41:06.088 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:05 smithi070 ceph-mon[30388]: 4.10 scrub starts 2024-09-16T09:41:06.089 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:05 smithi070 ceph-mon[30388]: 4.10 scrub ok 2024-09-16T09:41:06.089 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:05 smithi070 ceph-mon[30388]: 5.9 scrub starts 2024-09-16T09:41:06.089 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:05 smithi070 ceph-mon[30388]: 5.9 scrub ok 2024-09-16T09:41:06.089 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:05 smithi070 ceph-mon[40691]: 4.10 scrub starts 2024-09-16T09:41:06.089 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:05 smithi070 ceph-mon[40691]: 4.10 scrub ok 2024-09-16T09:41:06.089 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:05 smithi070 ceph-mon[40691]: 5.9 scrub starts 2024-09-16T09:41:06.089 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:05 smithi070 ceph-mon[40691]: 5.9 scrub ok 2024-09-16T09:41:06.108 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:41:06.129 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:06.129 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:06.130 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:41:06.202 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:05 smithi084 ceph-mon[32629]: 4.10 scrub starts 2024-09-16T09:41:06.202 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:05 smithi084 ceph-mon[32629]: 4.10 scrub ok 2024-09-16T09:41:06.202 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:05 smithi084 ceph-mon[32629]: 5.9 scrub starts 2024-09-16T09:41:06.202 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:05 smithi084 ceph-mon[32629]: 5.9 scrub ok 2024-09-16T09:41:06.231 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.5 dump_ops_in_flight 2024-09-16T09:41:06.232 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:41:06.255 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:06.255 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:06.255 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:41:06.356 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:41:06.379 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:06.379 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:06.380 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:41:06.480 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:41:06.482 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:41:06.503 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:06.503 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:06.503 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:41:06.604 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:41:06.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:06.626 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:06.627 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:41:06.727 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:41:06.750 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:06.751 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:06.751 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:41:06.852 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:41:06.873 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:06.874 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:06.874 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:41:06.950 INFO:teuthology.orchestra.run.smithi070.stderr:nodeep-scrub is unset 2024-09-16T09:41:06.974 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-09-16T09:41:06.974 INFO:teuthology.orchestra.run.smithi084.stdout: "ops": [], 2024-09-16T09:41:06.974 INFO:teuthology.orchestra.run.smithi084.stdout: "num_ops": 0 2024-09-16T09:41:06.974 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-09-16T09:41:06.975 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:41:06.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:06.996 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:06.996 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:41:07.097 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:41:07.120 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:07.121 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:07.121 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:41:07.218 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T09:41:07.222 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:41:07.240 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:06 smithi070 ceph-mon[30388]: pgmap v847: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 3.0 MiB data, 5.7 GiB used, 710 GiB / 715 GiB avail; 867 B/s rd, 0 B/s wr, 958 op/s; 18 B/s, 1 objects/s recovering 2024-09-16T09:41:07.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:06 smithi070 ceph-mon[30388]: 5.19 scrub starts 2024-09-16T09:41:07.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:06 smithi070 ceph-mon[30388]: 5.19 scrub ok 2024-09-16T09:41:07.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:06 smithi070 ceph-mon[30388]: 6.11 scrub starts 2024-09-16T09:41:07.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:06 smithi070 ceph-mon[30388]: 6.11 scrub ok 2024-09-16T09:41:07.241 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:06 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/4248960586' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:41:07.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:06 smithi070 ceph-mon[40691]: pgmap v847: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 3.0 MiB data, 5.7 GiB used, 710 GiB / 715 GiB avail; 867 B/s rd, 0 B/s wr, 958 op/s; 18 B/s, 1 objects/s recovering 2024-09-16T09:41:07.241 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:06 smithi070 ceph-mon[40691]: 5.19 scrub starts 2024-09-16T09:41:07.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:06 smithi070 ceph-mon[40691]: 5.19 scrub ok 2024-09-16T09:41:07.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:06 smithi070 ceph-mon[40691]: 6.11 scrub starts 2024-09-16T09:41:07.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:06 smithi070 ceph-mon[40691]: 6.11 scrub ok 2024-09-16T09:41:07.242 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:06 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/4248960586' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:41:07.263 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:07.263 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:07.264 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:41:07.278 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:06 smithi084 ceph-mon[32629]: pgmap v847: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 3.0 MiB data, 5.7 GiB used, 710 GiB / 715 GiB avail; 867 B/s rd, 0 B/s wr, 958 op/s; 18 B/s, 1 objects/s recovering 2024-09-16T09:41:07.278 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:06 smithi084 ceph-mon[32629]: 5.19 scrub starts 2024-09-16T09:41:07.278 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:06 smithi084 ceph-mon[32629]: 5.19 scrub ok 2024-09-16T09:41:07.278 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:06 smithi084 ceph-mon[32629]: 6.11 scrub starts 2024-09-16T09:41:07.278 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:06 smithi084 ceph-mon[32629]: 6.11 scrub ok 2024-09-16T09:41:07.279 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:06 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/4248960586' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:41:07.364 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:41:07.387 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:07.387 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:07.388 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:41:07.489 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:41:07.510 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:07.511 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:07.511 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:41:07.612 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:41:07.634 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.5 dump_blocked_ops 2024-09-16T09:41:07.635 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:07.636 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:07.636 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:41:07.737 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:41:07.759 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:07.760 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:07.760 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:41:07.861 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:41:07.880 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:41:07.883 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:07.884 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:07.884 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:41:07.985 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:41:08.007 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:08.008 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:08.008 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:41:08.017 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:41:08.108 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:41:08.130 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:08.131 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:08.131 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:41:08.232 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:41:08.253 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:08.253 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:08.254 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:41:08.285 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:07 smithi070 ceph-mon[30388]: 1.1 scrub starts 2024-09-16T09:41:08.285 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:07 smithi070 ceph-mon[30388]: 1.1 scrub ok 2024-09-16T09:41:08.285 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:07 smithi070 ceph-mon[30388]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-16T09:41:08.285 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:07 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/4248960586' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-16T09:41:08.285 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:07 smithi070 ceph-mon[30388]: osdmap e451: 8 total, 8 up, 8 in 2024-09-16T09:41:08.286 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:07 smithi070 ceph-mon[40691]: 1.1 scrub starts 2024-09-16T09:41:08.286 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:07 smithi070 ceph-mon[40691]: 1.1 scrub ok 2024-09-16T09:41:08.286 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:07 smithi070 ceph-mon[40691]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-16T09:41:08.286 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:07 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/4248960586' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-16T09:41:08.286 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:07 smithi070 ceph-mon[40691]: osdmap e451: 8 total, 8 up, 8 in 2024-09-16T09:41:08.298 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:07 smithi084 ceph-mon[32629]: 1.1 scrub starts 2024-09-16T09:41:08.298 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:07 smithi084 ceph-mon[32629]: 1.1 scrub ok 2024-09-16T09:41:08.298 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:07 smithi084 ceph-mon[32629]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-16T09:41:08.299 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:07 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/4248960586' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-16T09:41:08.299 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:07 smithi084 ceph-mon[32629]: osdmap e451: 8 total, 8 up, 8 in 2024-09-16T09:41:08.355 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:41:08.377 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:08.378 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:08.378 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:41:08.398 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-09-16T09:41:08.398 INFO:teuthology.orchestra.run.smithi084.stdout: "ops": [], 2024-09-16T09:41:08.398 INFO:teuthology.orchestra.run.smithi084.stdout: "complaint_time": 30, 2024-09-16T09:41:08.398 INFO:teuthology.orchestra.run.smithi084.stdout: "num_blocked_ops": 0 2024-09-16T09:41:08.399 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-09-16T09:41:08.479 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:41:08.500 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:08.501 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:08.502 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:41:08.602 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:41:08.620 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:41:08.625 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:08.625 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:08.626 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:41:08.727 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:41:08.751 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:08.752 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:08.752 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:41:08.853 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:41:08.876 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:08.876 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:08.877 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:41:08.963 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'unique_pool_3' removed 2024-09-16T09:41:08.977 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:41:09.007 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:09.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:09.008 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:41:09.109 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:41:09.132 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:09.133 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:09.133 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:41:09.146 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:08 smithi070 ceph-mon[30388]: pgmap v849: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 3.0 MiB data, 5.7 GiB used, 710 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 943 op/s; 18 B/s, 1 objects/s recovering 2024-09-16T09:41:09.146 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:08 smithi070 ceph-mon[30388]: 3.a scrub starts 2024-09-16T09:41:09.146 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:08 smithi070 ceph-mon[30388]: 3.a scrub ok 2024-09-16T09:41:09.146 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:08 smithi070 ceph-mon[30388]: 3.13 scrub starts 2024-09-16T09:41:09.146 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:08 smithi070 ceph-mon[30388]: 3.13 scrub ok 2024-09-16T09:41:09.146 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:08 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1028206558' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "unique_pool_3", "pool2": "unique_pool_3", "yes_i_really_really_mean_it": true} : dispatch 2024-09-16T09:41:09.146 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:08 smithi070 ceph-mon[40691]: pgmap v849: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 3.0 MiB data, 5.7 GiB used, 710 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 943 op/s; 18 B/s, 1 objects/s recovering 2024-09-16T09:41:09.146 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:08 smithi070 ceph-mon[40691]: 3.a scrub starts 2024-09-16T09:41:09.147 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:08 smithi070 ceph-mon[40691]: 3.a scrub ok 2024-09-16T09:41:09.147 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:08 smithi070 ceph-mon[40691]: 3.13 scrub starts 2024-09-16T09:41:09.147 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:08 smithi070 ceph-mon[40691]: 3.13 scrub ok 2024-09-16T09:41:09.147 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:08 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1028206558' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "unique_pool_3", "pool2": "unique_pool_3", "yes_i_really_really_mean_it": true} : dispatch 2024-09-16T09:41:09.234 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:41:09.256 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:09.256 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:09.257 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:41:09.357 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:41:09.357 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:41:09.357 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:41:09.357 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:41:09.357 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:41:09.357 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:41:09.357 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:41:09.357 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:41:09.358 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:41:09.358 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:41:09.358 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:41:09.358 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:41:09.358 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:41:09.358 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:41:09.358 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:41:09.358 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:41:09.359 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:41:09.379 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:09.380 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:09.380 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:41:09.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:08 smithi084 ceph-mon[32629]: pgmap v849: 178 pgs: 1 active+remapped+backfill_toofull, 177 active+clean; 3.0 MiB data, 5.7 GiB used, 710 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 943 op/s; 18 B/s, 1 objects/s recovering 2024-09-16T09:41:09.412 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:08 smithi084 ceph-mon[32629]: 3.a scrub starts 2024-09-16T09:41:09.412 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:08 smithi084 ceph-mon[32629]: 3.a scrub ok 2024-09-16T09:41:09.412 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:08 smithi084 ceph-mon[32629]: 3.13 scrub starts 2024-09-16T09:41:09.412 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:08 smithi084 ceph-mon[32629]: 3.13 scrub ok 2024-09-16T09:41:09.412 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:08 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1028206558' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "unique_pool_3", "pool2": "unique_pool_3", "yes_i_really_really_mean_it": true} : dispatch 2024-09-16T09:41:09.481 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:41:09.503 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:09.503 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:09.504 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:41:09.605 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:41:09.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:09.627 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:09.627 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:41:09.728 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:41:09.750 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:09.750 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:09.750 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:41:09.853 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:41:09.875 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:09.875 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:09.875 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:41:09.976 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:41:09.987 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd set noscrub 2024-09-16T09:41:09.997 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:09.998 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:09.998 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:41:10.099 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:41:10.121 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:10.122 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:10.122 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:41:10.160 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:41:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=sqlstore.transactions t=2024-09-16T09:41:09.805594332Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-16T09:41:10.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:09 smithi084 ceph-mon[32629]: 3.1a scrub starts 2024-09-16T09:41:10.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:09 smithi084 ceph-mon[32629]: 3.1a scrub ok 2024-09-16T09:41:10.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:09 smithi084 ceph-mon[32629]: 1.10 scrub starts 2024-09-16T09:41:10.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:09 smithi084 ceph-mon[32629]: 1.10 scrub ok 2024-09-16T09:41:10.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:09 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1028206558' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_3", "pool2": "unique_pool_3", "yes_i_really_really_mean_it": true}]': finished 2024-09-16T09:41:10.162 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:09 smithi084 ceph-mon[32629]: osdmap e452: 8 total, 8 up, 8 in 2024-09-16T09:41:10.162 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:09 smithi084 ceph-mon[32629]: 4.b scrub starts 2024-09-16T09:41:10.162 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:09 smithi084 ceph-mon[32629]: 4.b scrub ok 2024-09-16T09:41:10.162 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:09 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/550079999' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:41:10.162 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:09 smithi084 ceph-mon[32629]: osdmap e453: 8 total, 8 up, 8 in 2024-09-16T09:41:10.223 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:41:10.246 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:10.246 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:10.247 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:41:10.348 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:41:10.372 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:10.372 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:10.373 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:41:10.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:09 smithi070 ceph-mon[30388]: 3.1a scrub starts 2024-09-16T09:41:10.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:09 smithi070 ceph-mon[30388]: 3.1a scrub ok 2024-09-16T09:41:10.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:09 smithi070 ceph-mon[30388]: 1.10 scrub starts 2024-09-16T09:41:10.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:09 smithi070 ceph-mon[30388]: 1.10 scrub ok 2024-09-16T09:41:10.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:09 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1028206558' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_3", "pool2": "unique_pool_3", "yes_i_really_really_mean_it": true}]': finished 2024-09-16T09:41:10.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:09 smithi070 ceph-mon[30388]: osdmap e452: 8 total, 8 up, 8 in 2024-09-16T09:41:10.422 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:09 smithi070 ceph-mon[30388]: 4.b scrub starts 2024-09-16T09:41:10.422 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:09 smithi070 ceph-mon[30388]: 4.b scrub ok 2024-09-16T09:41:10.422 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:09 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/550079999' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:41:10.422 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:09 smithi070 ceph-mon[30388]: osdmap e453: 8 total, 8 up, 8 in 2024-09-16T09:41:10.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:09 smithi070 ceph-mon[40691]: 3.1a scrub starts 2024-09-16T09:41:10.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:09 smithi070 ceph-mon[40691]: 3.1a scrub ok 2024-09-16T09:41:10.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:09 smithi070 ceph-mon[40691]: 1.10 scrub starts 2024-09-16T09:41:10.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:09 smithi070 ceph-mon[40691]: 1.10 scrub ok 2024-09-16T09:41:10.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:09 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1028206558' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_3", "pool2": "unique_pool_3", "yes_i_really_really_mean_it": true}]': finished 2024-09-16T09:41:10.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:09 smithi070 ceph-mon[40691]: osdmap e452: 8 total, 8 up, 8 in 2024-09-16T09:41:10.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:09 smithi070 ceph-mon[40691]: 4.b scrub starts 2024-09-16T09:41:10.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:09 smithi070 ceph-mon[40691]: 4.b scrub ok 2024-09-16T09:41:10.423 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:09 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/550079999' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:41:10.423 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:09 smithi070 ceph-mon[40691]: osdmap e453: 8 total, 8 up, 8 in 2024-09-16T09:41:10.473 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:41:10.495 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:10.496 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:10.496 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:41:10.557 INFO:teuthology.task.full_sequential:In full_sequential, running task radosbench... 2024-09-16T09:41:10.558 INFO:tasks.radosbench:Beginning radosbench... 2024-09-16T09:41:10.559 INFO:tasks.cephadm.ceph_manager.ceph:creating pool_name unique_pool_4 2024-09-16T09:41:10.559 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd pool create unique_pool_4 16 2024-09-16T09:41:10.597 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:41:10.623 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:10.624 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:10.624 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:41:10.681 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:41:10.725 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:41:10.747 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:10.747 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:10.748 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:41:10.848 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:41:10.875 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:10.876 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:10.876 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:41:10.977 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:41:11.003 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:11.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:11.004 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:41:11.024 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.5 dump_historic_ops 2024-09-16T09:41:11.085 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:10 smithi070 ceph-mon[30388]: 3.2 scrub starts 2024-09-16T09:41:11.085 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:10 smithi070 ceph-mon[30388]: 3.2 scrub ok 2024-09-16T09:41:11.085 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:10 smithi070 ceph-mon[30388]: pgmap v852: 152 pgs: 152 active+clean; 583 KiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-09-16T09:41:11.085 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:10 smithi070 ceph-mon[30388]: 4.12 scrub starts 2024-09-16T09:41:11.085 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:10 smithi070 ceph-mon[30388]: 4.12 scrub ok 2024-09-16T09:41:11.085 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:10 smithi070 ceph-mon[30388]: Health check cleared: PG_BACKFILL_FULL (was: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull) 2024-09-16T09:41:11.085 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:10 smithi070 ceph-mon[30388]: Cluster is now healthy 2024-09-16T09:41:11.085 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:10 smithi070 ceph-mon[40691]: 3.2 scrub starts 2024-09-16T09:41:11.085 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:10 smithi070 ceph-mon[40691]: 3.2 scrub ok 2024-09-16T09:41:11.085 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:10 smithi070 ceph-mon[40691]: pgmap v852: 152 pgs: 152 active+clean; 583 KiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-09-16T09:41:11.086 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:10 smithi070 ceph-mon[40691]: 4.12 scrub starts 2024-09-16T09:41:11.086 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:10 smithi070 ceph-mon[40691]: 4.12 scrub ok 2024-09-16T09:41:11.086 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:10 smithi070 ceph-mon[40691]: Health check cleared: PG_BACKFILL_FULL (was: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull) 2024-09-16T09:41:11.086 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:10 smithi070 ceph-mon[40691]: Cluster is now healthy 2024-09-16T09:41:11.105 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:41:11.126 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:11.127 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:11.127 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:41:11.228 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:41:11.231 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:41:11.249 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:11.250 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:11.250 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:41:11.315 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:41:11.351 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:41:11.373 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:11.373 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:11.374 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:41:11.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:10 smithi084 ceph-mon[32629]: 3.2 scrub starts 2024-09-16T09:41:11.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:10 smithi084 ceph-mon[32629]: 3.2 scrub ok 2024-09-16T09:41:11.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:10 smithi084 ceph-mon[32629]: pgmap v852: 152 pgs: 152 active+clean; 583 KiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-09-16T09:41:11.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:10 smithi084 ceph-mon[32629]: 4.12 scrub starts 2024-09-16T09:41:11.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:10 smithi084 ceph-mon[32629]: 4.12 scrub ok 2024-09-16T09:41:11.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:10 smithi084 ceph-mon[32629]: Health check cleared: PG_BACKFILL_FULL (was: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull) 2024-09-16T09:41:11.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:10 smithi084 ceph-mon[32629]: Cluster is now healthy 2024-09-16T09:41:11.474 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:41:11.496 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:11.497 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:11.497 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:41:11.598 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:41:11.621 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:11.621 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:11.622 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:41:11.722 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:41:11.744 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:11.745 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:11.745 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:41:11.846 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:41:11.866 INFO:teuthology.orchestra.run.smithi084.stdout:ERROR: (22) Invalid argument 2024-09-16T09:41:11.866 INFO:teuthology.orchestra.run.smithi084.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:41:11.868 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:11.868 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:11.868 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:41:11.969 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:41:11.983 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'unique_pool_4' created 2024-09-16T09:41:11.991 INFO:teuthology.orchestra.run.smithi070.stderr:noscrub is set 2024-09-16T09:41:11.996 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:11.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:11.997 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:41:12.057 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T09:41:12.098 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:41:12.120 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:12.121 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:12.121 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:41:12.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:11 smithi084 ceph-mon[32629]: 1.7 scrub starts 2024-09-16T09:41:12.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:11 smithi084 ceph-mon[32629]: 1.7 scrub ok 2024-09-16T09:41:12.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:11 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3657467601' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:41:12.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:11 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:41:12.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:11 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1600986234' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "unique_pool_4", "pg_num": 16} : dispatch 2024-09-16T09:41:12.222 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:41:12.244 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:12.244 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:12.245 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:41:12.346 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:41:12.366 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:12.367 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:12.367 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:41:12.403 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T09:41:12.403 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T09:41:12.403 INFO:tasks.thrashosds.thrasher:inject_pause on osd.0 2024-09-16T09:41:12.403 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-16T09:41:12.403 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-16T09:41:12.404 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.0 config set filestore_inject_stall 3 2024-09-16T09:41:12.420 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:41:11 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:41:11] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:41:12.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:11 smithi070 ceph-mon[30388]: 1.7 scrub starts 2024-09-16T09:41:12.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:11 smithi070 ceph-mon[30388]: 1.7 scrub ok 2024-09-16T09:41:12.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:11 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3657467601' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:41:12.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:11 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:41:12.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:11 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1600986234' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "unique_pool_4", "pg_num": 16} : dispatch 2024-09-16T09:41:12.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:11 smithi070 ceph-mon[40691]: 1.7 scrub starts 2024-09-16T09:41:12.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:11 smithi070 ceph-mon[40691]: 1.7 scrub ok 2024-09-16T09:41:12.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:11 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3657467601' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:41:12.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:11 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:41:12.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:11 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1600986234' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "unique_pool_4", "pg_num": 16} : dispatch 2024-09-16T09:41:12.469 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:41:12.491 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:12.492 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:12.492 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:41:12.565 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.6 dump_ops_in_flight 2024-09-16T09:41:12.593 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:41:12.613 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:12.614 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:12.615 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:41:12.715 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:41:12.737 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:12.738 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:12.738 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:41:12.810 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:41:12.839 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:41:12.861 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:12.862 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:12.862 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:41:12.963 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:41:12.986 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:12.987 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:12.987 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:41:13.088 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:41:13.110 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:13.111 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:13.111 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:41:13.212 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:41:13.224 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:12 smithi084 ceph-mon[32629]: 4.1 scrub starts 2024-09-16T09:41:13.224 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:12 smithi084 ceph-mon[32629]: 4.1 scrub ok 2024-09-16T09:41:13.224 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:12 smithi084 ceph-mon[32629]: 4.8 scrub starts 2024-09-16T09:41:13.224 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:12 smithi084 ceph-mon[32629]: pgmap v853: 152 pgs: 152 active+clean; 583 KiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-09-16T09:41:13.224 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:12 smithi084 ceph-mon[32629]: 4.8 scrub ok 2024-09-16T09:41:13.225 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:12 smithi084 ceph-mon[32629]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:41:13.225 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:12 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-16T09:41:13.225 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:12 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1600986234' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_4", "pg_num": 16}]': finished 2024-09-16T09:41:13.225 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:12 smithi084 ceph-mon[32629]: osdmap e454: 8 total, 8 up, 8 in 2024-09-16T09:41:13.236 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:13.236 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:13.236 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:41:13.337 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:41:13.340 INFO:teuthology.orchestra.run.smithi084.stdout:ERROR: (22) Invalid argument 2024-09-16T09:41:13.340 INFO:teuthology.orchestra.run.smithi084.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:41:13.359 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:13.360 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:13.360 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:41:13.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:12 smithi070 ceph-mon[30388]: 4.1 scrub starts 2024-09-16T09:41:13.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:12 smithi070 ceph-mon[30388]: 4.1 scrub ok 2024-09-16T09:41:13.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:12 smithi070 ceph-mon[30388]: 4.8 scrub starts 2024-09-16T09:41:13.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:12 smithi070 ceph-mon[30388]: pgmap v853: 152 pgs: 152 active+clean; 583 KiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-09-16T09:41:13.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:12 smithi070 ceph-mon[30388]: 4.8 scrub ok 2024-09-16T09:41:13.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:12 smithi070 ceph-mon[30388]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:41:13.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:12 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-16T09:41:13.422 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:12 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1600986234' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_4", "pg_num": 16}]': finished 2024-09-16T09:41:13.422 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:12 smithi070 ceph-mon[30388]: osdmap e454: 8 total, 8 up, 8 in 2024-09-16T09:41:13.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:12 smithi070 ceph-mon[40691]: 4.1 scrub starts 2024-09-16T09:41:13.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:12 smithi070 ceph-mon[40691]: 4.1 scrub ok 2024-09-16T09:41:13.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:12 smithi070 ceph-mon[40691]: 4.8 scrub starts 2024-09-16T09:41:13.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:12 smithi070 ceph-mon[40691]: pgmap v853: 152 pgs: 152 active+clean; 583 KiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-09-16T09:41:13.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:12 smithi070 ceph-mon[40691]: 4.8 scrub ok 2024-09-16T09:41:13.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:12 smithi070 ceph-mon[40691]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:41:13.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:12 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-16T09:41:13.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:12 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1600986234' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_4", "pg_num": 16}]': finished 2024-09-16T09:41:13.423 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:12 smithi070 ceph-mon[40691]: osdmap e454: 8 total, 8 up, 8 in 2024-09-16T09:41:13.461 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:41:13.482 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:13.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:13.483 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:41:13.584 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:41:13.607 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:13.608 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:13.608 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:41:13.709 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:41:13.725 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:41:13.734 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:13.735 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:13.735 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:41:13.745 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:41:13.836 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:41:13.858 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:13.859 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:13.859 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:41:13.910 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:41:13 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:41:13.960 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:41:13.983 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:13.984 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:13.984 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:41:14.023 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.6 dump_blocked_ops 2024-09-16T09:41:14.086 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:41:14.107 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:14.108 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:14.108 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:41:14.123 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:13 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:41:14.123 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:13 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:41:14.123 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:13 smithi070 ceph-mon[30388]: osdmap e455: 8 total, 8 up, 8 in 2024-09-16T09:41:14.123 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:13 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:41:14.123 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:13 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:41:14.123 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:13 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:41:14.123 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:13 smithi070 ceph-mon[40691]: osdmap e455: 8 total, 8 up, 8 in 2024-09-16T09:41:14.124 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:13 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:41:14.209 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:41:14.231 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:14.232 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:14.232 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:41:14.266 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:41:14.333 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:41:14.355 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:14.356 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:14.356 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:41:14.386 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:13 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:41:14.387 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:13 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:41:14.387 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:13 smithi084 ceph-mon[32629]: osdmap e455: 8 total, 8 up, 8 in 2024-09-16T09:41:14.387 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:13 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:41:14.460 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:41:14.481 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:14.482 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:14.483 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:41:14.583 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:41:14.605 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:14.606 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:14.606 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:41:14.691 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd pool application enable unique_pool_4 rados --yes-i-really-mean-it || true 2024-09-16T09:41:14.706 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:41:14.756 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:14.757 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:14.757 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:41:14.765 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T09:41:14.765 INFO:teuthology.orchestra.run.smithi070.stdout: "success": "" 2024-09-16T09:41:14.766 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T09:41:14.781 INFO:teuthology.orchestra.run.smithi084.stdout:ERROR: (22) Invalid argument 2024-09-16T09:41:14.781 INFO:teuthology.orchestra.run.smithi084.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:41:14.858 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:41:14.880 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:14.881 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:14.881 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:41:14.982 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:41:15.006 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:15.006 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:15.007 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:41:15.108 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:41:15.129 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:15.130 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:15.130 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:41:15.231 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:41:15.252 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:14 smithi070 ceph-mon[30388]: pgmap v856: 168 pgs: 1 creating+activating, 8 creating+peering, 1 unknown, 158 active+clean; 583 KiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-16T09:41:15.252 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:14 smithi070 ceph-mon[30388]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T09:41:15.252 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:14 smithi070 ceph-mon[30388]: osdmap e456: 8 total, 8 up, 8 in 2024-09-16T09:41:15.252 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:14 smithi070 ceph-mon[30388]: osdmap e457: 8 total, 8 up, 8 in 2024-09-16T09:41:15.253 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:14 smithi070 ceph-mon[40691]: pgmap v856: 168 pgs: 1 creating+activating, 8 creating+peering, 1 unknown, 158 active+clean; 583 KiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-16T09:41:15.253 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:14 smithi070 ceph-mon[40691]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T09:41:15.253 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:14 smithi070 ceph-mon[40691]: osdmap e456: 8 total, 8 up, 8 in 2024-09-16T09:41:15.253 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:14 smithi070 ceph-mon[40691]: osdmap e457: 8 total, 8 up, 8 in 2024-09-16T09:41:15.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:15.254 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:15.254 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:41:15.355 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:41:15.378 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:15.378 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:15.379 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:41:15.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:14 smithi084 ceph-mon[32629]: pgmap v856: 168 pgs: 1 creating+activating, 8 creating+peering, 1 unknown, 158 active+clean; 583 KiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-16T09:41:15.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:14 smithi084 ceph-mon[32629]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T09:41:15.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:14 smithi084 ceph-mon[32629]: osdmap e456: 8 total, 8 up, 8 in 2024-09-16T09:41:15.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:14 smithi084 ceph-mon[32629]: osdmap e457: 8 total, 8 up, 8 in 2024-09-16T09:41:15.455 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.6 dump_historic_ops 2024-09-16T09:41:15.479 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:41:15.519 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:15.519 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:15.519 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:41:15.620 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:41:15.643 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:15.643 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:15.644 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:41:15.702 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:41:15.745 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:41:15.766 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:15.767 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:15.767 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:41:15.868 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:41:15.889 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:15.890 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:15.892 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:41:15.901 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:41:15.901 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:41:15.912 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:41:15.912 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:41:15.928 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:41:15.928 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:41:15.944 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:41:15.944 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:41:15.961 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:41:15.961 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:41:15.978 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:41:15.978 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:41:15.993 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:41:15.994 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:41:15.995 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:41:16.015 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:16.015 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:16.016 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:41:16.016 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:41:16.016 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:41:16.117 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:41:16.139 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:16.140 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:16.140 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:41:16.155 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:16 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/224185582' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:41:16.241 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:41:16.256 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-09-16T09:41:16.256 INFO:teuthology.orchestra.run.smithi084.stdout: "size": 20, 2024-09-16T09:41:16.256 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 600, 2024-09-16T09:41:16.256 INFO:teuthology.orchestra.run.smithi084.stdout: "ops": [ 2024-09-16T09:41:16.256 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.256 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:222663 9.1 9:817a7893:::benchmark_data_smithi070_137000_object100952:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e349)", 2024-09-16T09:41:16.256 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:37:10.363067+0000", 2024-09-16T09:41:16.256 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 245.89029090700001, 2024-09-16T09:41:16.257 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.030122435, 2024-09-16T09:41:16.257 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:41:16.257 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:41:16.257 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "reached pg", 2024-09-16T09:41:16.257 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:41:16.257 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:41:16.257 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:41:16.257 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 222663 2024-09-16T09:41:16.257 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.257 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:41:16.257 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.257 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:41:16.257 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363067+0000", 2024-09-16T09:41:16.257 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:41:16.258 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.258 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.258 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:41:16.258 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363067+0000", 2024-09-16T09:41:16.258 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:41:16.258 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.258 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.258 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:41:16.258 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363074+0000", 2024-09-16T09:41:16.258 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.905e-06 2024-09-16T09:41:16.258 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.258 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.258 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:41:16.258 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363083+0000", 2024-09-16T09:41:16.259 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.5019999999999992e-06 2024-09-16T09:41:16.259 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.259 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.259 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:41:16.259 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363099+0000", 2024-09-16T09:41:16.259 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.6427999999999999e-05 2024-09-16T09:41:16.259 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.259 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.259 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:41:16.259 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363108+0000", 2024-09-16T09:41:16.259 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.1900000000000001e-06 2024-09-16T09:41:16.259 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.259 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.259 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:41:16.260 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.397601+0000", 2024-09-16T09:41:16.260 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.034492918999999997 2024-09-16T09:41:16.260 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.260 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.260 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:41:16.260 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.397613+0000", 2024-09-16T09:41:16.260 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.2275e-05 2024-09-16T09:41:16.260 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.260 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.260 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:41:16.260 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.393137+0000", 2024-09-16T09:41:16.260 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.99552360100000004 2024-09-16T09:41:16.260 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.260 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.261 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:41:16.261 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.393190+0000", 2024-09-16T09:41:16.261 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.2614999999999997e-05 2024-09-16T09:41:16.261 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:41:16.261 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:41:16.261 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:41:16.261 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.261 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.261 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:222725 9.1 9:811a4bf7:::benchmark_data_smithi070_137000_object101014:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e349)", 2024-09-16T09:41:16.261 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:37:10.363127+0000", 2024-09-16T09:41:16.261 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 245.89023128900001, 2024-09-16T09:41:16.261 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0316495720000001, 2024-09-16T09:41:16.261 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:41:16.261 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:41:16.262 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:41:16.262 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:41:16.262 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:41:16.262 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:41:16.262 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 222725 2024-09-16T09:41:16.262 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.262 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:41:16.262 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.262 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:41:16.262 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363127+0000", 2024-09-16T09:41:16.262 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:41:16.262 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.262 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.262 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:41:16.263 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363127+0000", 2024-09-16T09:41:16.263 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:41:16.263 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.263 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.263 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:41:16.263 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363127+0000", 2024-09-16T09:41:16.263 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.9499999999999998e-07 2024-09-16T09:41:16.263 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.263 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.263 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:41:16.263 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363129+0000", 2024-09-16T09:41:16.263 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.261e-06 2024-09-16T09:41:16.263 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.263 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.264 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:41:16.264 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363135+0000", 2024-09-16T09:41:16.264 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.1580000000000004e-06 2024-09-16T09:41:16.264 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.264 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.264 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:41:16.264 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363138+0000", 2024-09-16T09:41:16.264 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.7620000000000001e-06 2024-09-16T09:41:16.264 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.264 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.264 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:41:16.264 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.397654+0000", 2024-09-16T09:41:16.264 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.034515420999999998 2024-09-16T09:41:16.264 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.265 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.265 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:41:16.265 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.397664+0000", 2024-09-16T09:41:16.265 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0549e-05 2024-09-16T09:41:16.265 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.265 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.265 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:41:16.265 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.393573+0000", 2024-09-16T09:41:16.265 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.99590824499999997 2024-09-16T09:41:16.265 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.265 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.265 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:41:16.265 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.393659+0000", 2024-09-16T09:41:16.265 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.6203000000000001e-05 2024-09-16T09:41:16.266 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.266 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.266 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 3,7", 2024-09-16T09:41:16.266 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.393727+0000", 2024-09-16T09:41:16.266 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.7935000000000006e-05 2024-09-16T09:41:16.266 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.266 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.266 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:41:16.266 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.394640+0000", 2024-09-16T09:41:16.266 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00091379700000000005 2024-09-16T09:41:16.266 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.266 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.266 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:41:16.266 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.394674+0000", 2024-09-16T09:41:16.267 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.3800000000000002e-05 2024-09-16T09:41:16.267 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.267 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.267 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:41:16.267 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.394719+0000", 2024-09-16T09:41:16.267 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.4880999999999999e-05 2024-09-16T09:41:16.267 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.267 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.267 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:41:16.267 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.394750+0000", 2024-09-16T09:41:16.267 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.0514e-05 2024-09-16T09:41:16.267 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.267 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.267 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:41:16.267 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.394776+0000", 2024-09-16T09:41:16.268 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.6650999999999999e-05 2024-09-16T09:41:16.268 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:41:16.268 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:41:16.268 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:41:16.268 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.268 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.268 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:222759 9.1 9:84f52a19:::benchmark_data_smithi070_137000_object101048:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e349)", 2024-09-16T09:41:16.268 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:37:10.363143+0000", 2024-09-16T09:41:16.268 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 245.89021521500001, 2024-09-16T09:41:16.268 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.03180293, 2024-09-16T09:41:16.268 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:41:16.268 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:41:16.268 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:41:16.269 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:41:16.269 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:41:16.269 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:41:16.269 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 222759 2024-09-16T09:41:16.269 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.269 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:41:16.269 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.269 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:41:16.269 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363143+0000", 2024-09-16T09:41:16.269 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:41:16.269 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.269 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.269 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:41:16.269 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363143+0000", 2024-09-16T09:41:16.269 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:41:16.270 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.270 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.270 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:41:16.270 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363143+0000", 2024-09-16T09:41:16.270 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.2499999999999999e-07 2024-09-16T09:41:16.270 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.270 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.270 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:41:16.270 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363144+0000", 2024-09-16T09:41:16.270 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1510000000000001e-06 2024-09-16T09:41:16.270 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.270 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.270 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:41:16.271 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363149+0000", 2024-09-16T09:41:16.271 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.3189999999999996e-06 2024-09-16T09:41:16.271 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.271 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.271 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:41:16.271 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363152+0000", 2024-09-16T09:41:16.271 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.9969999999999999e-06 2024-09-16T09:41:16.271 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.271 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.271 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:41:16.271 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.397717+0000", 2024-09-16T09:41:16.271 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.034565851000000002 2024-09-16T09:41:16.271 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.272 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.272 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:41:16.272 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.397728+0000", 2024-09-16T09:41:16.272 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0832999999999999e-05 2024-09-16T09:41:16.272 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.272 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.272 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:41:16.272 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.393932+0000", 2024-09-16T09:41:16.272 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.99620329399999996 2024-09-16T09:41:16.272 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.272 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.272 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:41:16.272 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.394030+0000", 2024-09-16T09:41:16.272 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.8080999999999998e-05 2024-09-16T09:41:16.273 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.273 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.273 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 3,7", 2024-09-16T09:41:16.273 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.394094+0000", 2024-09-16T09:41:16.273 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.4604999999999996e-05 2024-09-16T09:41:16.273 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.273 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.273 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:41:16.273 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.394843+0000", 2024-09-16T09:41:16.273 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00074905100000000004 2024-09-16T09:41:16.273 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.274 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.274 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:41:16.274 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.394854+0000", 2024-09-16T09:41:16.274 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0249e-05 2024-09-16T09:41:16.274 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.274 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.274 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:41:16.274 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.394878+0000", 2024-09-16T09:41:16.274 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.4173e-05 2024-09-16T09:41:16.274 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.274 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.274 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:41:16.274 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.394913+0000", 2024-09-16T09:41:16.274 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.5515000000000001e-05 2024-09-16T09:41:16.275 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.275 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.275 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:41:16.275 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.394946+0000", 2024-09-16T09:41:16.275 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.2586000000000003e-05 2024-09-16T09:41:16.275 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:41:16.275 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:41:16.275 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:41:16.275 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.275 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.275 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:222834 9.1 9:85e273e5:::benchmark_data_smithi070_137000_object101123:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e349)", 2024-09-16T09:41:16.275 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:37:10.363155+0000", 2024-09-16T09:41:16.275 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 245.89020340900001, 2024-09-16T09:41:16.275 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.032180122, 2024-09-16T09:41:16.276 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:41:16.276 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:41:16.276 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:41:16.276 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:41:16.276 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:41:16.276 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:41:16.276 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 222834 2024-09-16T09:41:16.276 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.276 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:41:16.276 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.276 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:41:16.276 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363155+0000", 2024-09-16T09:41:16.276 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:41:16.276 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.277 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.277 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:41:16.277 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363155+0000", 2024-09-16T09:41:16.277 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:41:16.277 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.277 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.277 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:41:16.277 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363155+0000", 2024-09-16T09:41:16.277 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.0800000000000001e-07 2024-09-16T09:41:16.277 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.277 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.277 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:41:16.277 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363156+0000", 2024-09-16T09:41:16.277 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.02e-06 2024-09-16T09:41:16.278 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.278 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.278 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:41:16.278 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363160+0000", 2024-09-16T09:41:16.278 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.9770000000000002e-06 2024-09-16T09:41:16.278 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.278 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.278 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:41:16.278 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363163+0000", 2024-09-16T09:41:16.278 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.174e-06 2024-09-16T09:41:16.278 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.278 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.278 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:41:16.278 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.397768+0000", 2024-09-16T09:41:16.279 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.034605455 2024-09-16T09:41:16.279 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.279 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.279 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:41:16.279 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.397776+0000", 2024-09-16T09:41:16.279 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.5429999999999996e-06 2024-09-16T09:41:16.279 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.279 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.279 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:41:16.279 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.394293+0000", 2024-09-16T09:41:16.279 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.99651656700000002 2024-09-16T09:41:16.279 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.279 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.279 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:41:16.280 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.394365+0000", 2024-09-16T09:41:16.280 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.2575000000000002e-05 2024-09-16T09:41:16.280 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.280 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.280 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 3,7", 2024-09-16T09:41:16.280 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.394423+0000", 2024-09-16T09:41:16.280 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.8220000000000002e-05 2024-09-16T09:41:16.280 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.280 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.280 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:41:16.280 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.394986+0000", 2024-09-16T09:41:16.280 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00056240100000000003 2024-09-16T09:41:16.280 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.281 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.281 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:41:16.281 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.394994+0000", 2024-09-16T09:41:16.281 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.4169999999999992e-06 2024-09-16T09:41:16.281 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.281 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.281 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:41:16.281 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.395287+0000", 2024-09-16T09:41:16.281 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00029250199999999999 2024-09-16T09:41:16.281 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.281 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.281 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:41:16.281 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.395313+0000", 2024-09-16T09:41:16.281 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.6721999999999999e-05 2024-09-16T09:41:16.282 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.282 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.282 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:41:16.282 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.395335+0000", 2024-09-16T09:41:16.282 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.1341000000000001e-05 2024-09-16T09:41:16.282 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:41:16.282 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:41:16.282 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:41:16.282 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.282 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.282 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:222844 9.1 9:80f5d6a4:::benchmark_data_smithi070_137000_object101133:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e349)", 2024-09-16T09:41:16.282 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:37:10.363166+0000", 2024-09-16T09:41:16.282 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 245.89019214000001, 2024-09-16T09:41:16.282 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0331230769999999, 2024-09-16T09:41:16.282 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:41:16.283 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:41:16.283 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:41:16.283 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:41:16.283 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:41:16.283 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:41:16.283 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 222844 2024-09-16T09:41:16.283 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.283 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:41:16.283 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.283 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:41:16.283 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363166+0000", 2024-09-16T09:41:16.283 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:41:16.285 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:16.286 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:16.286 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.286 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.286 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:41:16.286 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363166+0000", 2024-09-16T09:41:16.286 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:41:16.286 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.286 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.286 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:41:16.286 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363166+0000", 2024-09-16T09:41:16.286 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.0900000000000001e-07 2024-09-16T09:41:16.287 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.287 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.287 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:41:16.287 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363167+0000", 2024-09-16T09:41:16.287 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0189999999999999e-06 2024-09-16T09:41:16.287 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.287 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.287 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:41:16.287 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363170+0000", 2024-09-16T09:41:16.287 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.1259999999999998e-06 2024-09-16T09:41:16.287 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.287 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.287 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:41:16.287 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363173+0000", 2024-09-16T09:41:16.288 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.39e-06 2024-09-16T09:41:16.288 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.288 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.288 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:41:16.288 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.397812+0000", 2024-09-16T09:41:16.288 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.034639239000000002 2024-09-16T09:41:16.288 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.288 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.288 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:41:16.288 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.397821+0000", 2024-09-16T09:41:16.288 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.2629999999999992e-06 2024-09-16T09:41:16.288 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.288 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.289 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:41:16.289 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.395016+0000", 2024-09-16T09:41:16.289 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.99719489299999997 2024-09-16T09:41:16.289 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.289 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.289 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:41:16.289 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.395079+0000", 2024-09-16T09:41:16.289 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.2699000000000004e-05 2024-09-16T09:41:16.289 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.289 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.289 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 3,7", 2024-09-16T09:41:16.289 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.395137+0000", 2024-09-16T09:41:16.289 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.8158000000000001e-05 2024-09-16T09:41:16.290 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.290 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.290 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:41:16.290 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.396134+0000", 2024-09-16T09:41:16.290 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00099750399999999997 2024-09-16T09:41:16.290 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.290 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.290 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:41:16.290 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.396186+0000", 2024-09-16T09:41:16.290 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.1962000000000002e-05 2024-09-16T09:41:16.290 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.290 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.290 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:41:16.290 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.396200+0000", 2024-09-16T09:41:16.291 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.4056000000000001e-05 2024-09-16T09:41:16.291 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.291 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.291 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:41:16.291 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.396246+0000", 2024-09-16T09:41:16.291 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.5121999999999999e-05 2024-09-16T09:41:16.291 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.291 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.291 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:41:16.291 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.396289+0000", 2024-09-16T09:41:16.291 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.3436999999999998e-05 2024-09-16T09:41:16.291 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:41:16.291 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:41:16.291 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:41:16.292 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.292 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.292 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:222861 9.1 9:876cd0fc:::benchmark_data_smithi070_137000_object101150:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e349)", 2024-09-16T09:41:16.292 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:37:10.363175+0000", 2024-09-16T09:41:16.292 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 245.89018259100001, 2024-09-16T09:41:16.292 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.033248876, 2024-09-16T09:41:16.292 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:41:16.292 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:41:16.292 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:41:16.292 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:41:16.292 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:41:16.292 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:41:16.292 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 222861 2024-09-16T09:41:16.293 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.293 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:41:16.293 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.293 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:41:16.293 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363175+0000", 2024-09-16T09:41:16.293 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:41:16.293 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.293 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.293 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:41:16.293 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363175+0000", 2024-09-16T09:41:16.293 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:41:16.293 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.293 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.293 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:41:16.294 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363176+0000", 2024-09-16T09:41:16.294 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.0599999999999999e-07 2024-09-16T09:41:16.294 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.294 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.294 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:41:16.294 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363177+0000", 2024-09-16T09:41:16.294 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.062e-06 2024-09-16T09:41:16.294 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.294 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.294 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:41:16.294 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363181+0000", 2024-09-16T09:41:16.294 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.7840000000000001e-06 2024-09-16T09:41:16.294 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.294 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.294 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:41:16.295 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363183+0000", 2024-09-16T09:41:16.295 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.8140000000000001e-06 2024-09-16T09:41:16.295 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.295 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.295 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:41:16.295 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.397869+0000", 2024-09-16T09:41:16.295 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.034685939999999998 2024-09-16T09:41:16.295 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.295 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.295 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:41:16.295 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.397877+0000", 2024-09-16T09:41:16.295 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.157e-06 2024-09-16T09:41:16.295 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.295 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.296 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:41:16.296 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.395362+0000", 2024-09-16T09:41:16.296 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.99748467399999996 2024-09-16T09:41:16.296 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.296 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.296 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:41:16.296 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.395428+0000", 2024-09-16T09:41:16.296 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.5549999999999994e-05 2024-09-16T09:41:16.296 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.296 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.296 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 3,7", 2024-09-16T09:41:16.296 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.395484+0000", 2024-09-16T09:41:16.296 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.6709999999999997e-05 2024-09-16T09:41:16.296 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.297 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.297 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:41:16.297 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.396296+0000", 2024-09-16T09:41:16.297 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00081205299999999997 2024-09-16T09:41:16.297 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.297 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.297 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:41:16.297 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.396337+0000", 2024-09-16T09:41:16.297 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.0167999999999999e-05 2024-09-16T09:41:16.297 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.297 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.297 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:41:16.297 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.396370+0000", 2024-09-16T09:41:16.297 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.3441000000000001e-05 2024-09-16T09:41:16.298 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.298 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.298 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:41:16.298 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.396400+0000", 2024-09-16T09:41:16.298 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.9737999999999999e-05 2024-09-16T09:41:16.298 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.298 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.298 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:41:16.298 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.396424+0000", 2024-09-16T09:41:16.298 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.4579e-05 2024-09-16T09:41:16.298 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:41:16.298 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:41:16.298 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:41:16.298 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.299 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.299 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:222881 9.1 9:86c78cd8:::benchmark_data_smithi070_137000_object101170:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e349)", 2024-09-16T09:41:16.299 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:37:10.363186+0000", 2024-09-16T09:41:16.299 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 245.89017195400001, 2024-09-16T09:41:16.299 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0333547729999999, 2024-09-16T09:41:16.299 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:41:16.299 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:41:16.299 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:41:16.299 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:41:16.299 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:41:16.299 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:41:16.299 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 222881 2024-09-16T09:41:16.299 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.299 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:41:16.300 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.300 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:41:16.300 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363186+0000", 2024-09-16T09:41:16.300 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:41:16.300 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.300 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.300 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:41:16.300 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363186+0000", 2024-09-16T09:41:16.300 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:41:16.300 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.300 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.300 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:41:16.300 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363186+0000", 2024-09-16T09:41:16.301 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.9999999999999999e-07 2024-09-16T09:41:16.301 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.301 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.301 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:41:16.301 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363187+0000", 2024-09-16T09:41:16.301 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.125e-06 2024-09-16T09:41:16.301 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.301 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.301 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:41:16.301 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363191+0000", 2024-09-16T09:41:16.301 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.7969999999999998e-06 2024-09-16T09:41:16.301 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.301 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.301 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:41:16.302 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363194+0000", 2024-09-16T09:41:16.302 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.7489999999999999e-06 2024-09-16T09:41:16.302 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.302 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.302 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:41:16.302 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.397946+0000", 2024-09-16T09:41:16.302 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.034751690000000002 2024-09-16T09:41:16.302 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.302 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.302 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:41:16.302 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.397955+0000", 2024-09-16T09:41:16.302 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.5990000000000008e-06 2024-09-16T09:41:16.302 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.302 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.303 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:41:16.303 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.395673+0000", 2024-09-16T09:41:16.303 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.99771777100000003 2024-09-16T09:41:16.303 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.303 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.303 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:41:16.303 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.395791+0000", 2024-09-16T09:41:16.303 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00011786300000000001 2024-09-16T09:41:16.303 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.303 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.303 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 3,7", 2024-09-16T09:41:16.303 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.395897+0000", 2024-09-16T09:41:16.303 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00010618 2024-09-16T09:41:16.303 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.304 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.304 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:41:16.304 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.396433+0000", 2024-09-16T09:41:16.304 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00053625899999999998 2024-09-16T09:41:16.304 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.304 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.304 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:41:16.304 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.396463+0000", 2024-09-16T09:41:16.304 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.9346999999999999e-05 2024-09-16T09:41:16.304 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.304 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.304 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:41:16.304 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.396495+0000", 2024-09-16T09:41:16.304 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.2490000000000002e-05 2024-09-16T09:41:16.305 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.305 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.305 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:41:16.305 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.396520+0000", 2024-09-16T09:41:16.305 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.5052e-05 2024-09-16T09:41:16.305 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.305 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.305 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:41:16.305 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.396541+0000", 2024-09-16T09:41:16.305 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.0650999999999999e-05 2024-09-16T09:41:16.305 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:41:16.305 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:41:16.305 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:41:16.305 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.306 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.306 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:222925 9.1 9:81973e20:::benchmark_data_smithi070_137000_object101214:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e349)", 2024-09-16T09:41:16.306 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:37:10.363197+0000", 2024-09-16T09:41:16.306 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 245.89016072300001, 2024-09-16T09:41:16.306 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0353614579999999, 2024-09-16T09:41:16.306 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:41:16.306 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:41:16.306 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:41:16.306 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:41:16.306 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:41:16.306 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:41:16.306 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 222925 2024-09-16T09:41:16.306 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.306 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:41:16.307 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.307 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:41:16.307 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363197+0000", 2024-09-16T09:41:16.307 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:41:16.307 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.307 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.307 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:41:16.307 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363197+0000", 2024-09-16T09:41:16.307 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:41:16.307 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.307 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.307 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:41:16.307 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363198+0000", 2024-09-16T09:41:16.307 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.0200000000000001e-07 2024-09-16T09:41:16.308 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.308 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.308 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:41:16.308 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363199+0000", 2024-09-16T09:41:16.308 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.079e-06 2024-09-16T09:41:16.308 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.308 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.308 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:41:16.308 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363203+0000", 2024-09-16T09:41:16.308 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.8920000000000002e-06 2024-09-16T09:41:16.308 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.308 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.308 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:41:16.308 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363205+0000", 2024-09-16T09:41:16.309 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.7539999999999998e-06 2024-09-16T09:41:16.309 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.309 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.309 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:41:16.309 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.398001+0000", 2024-09-16T09:41:16.309 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.034795595999999998 2024-09-16T09:41:16.309 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.309 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.309 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:41:16.309 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.398008+0000", 2024-09-16T09:41:16.309 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.5850000000000002e-06 2024-09-16T09:41:16.309 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.309 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.309 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:41:16.310 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.396565+0000", 2024-09-16T09:41:16.310 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.99855653799999999 2024-09-16T09:41:16.310 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.310 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.310 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:41:16.310 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.396627+0000", 2024-09-16T09:41:16.310 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.2395000000000001e-05 2024-09-16T09:41:16.310 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.310 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.310 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 3,7", 2024-09-16T09:41:16.310 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.396684+0000", 2024-09-16T09:41:16.310 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.7045e-05 2024-09-16T09:41:16.310 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.310 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.311 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:41:16.311 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.397837+0000", 2024-09-16T09:41:16.311 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0011528070000000001 2024-09-16T09:41:16.311 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.311 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.311 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:41:16.311 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.397880+0000", 2024-09-16T09:41:16.311 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.2701999999999999e-05 2024-09-16T09:41:16.311 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.311 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.311 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:41:16.311 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.398501+0000", 2024-09-16T09:41:16.311 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00062094699999999995 2024-09-16T09:41:16.311 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.311 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.312 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:41:16.312 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.398524+0000", 2024-09-16T09:41:16.312 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.3388e-05 2024-09-16T09:41:16.312 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.312 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.312 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:41:16.312 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.398559+0000", 2024-09-16T09:41:16.312 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.4527999999999999e-05 2024-09-16T09:41:16.312 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:41:16.312 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:41:16.312 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:41:16.312 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.312 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.313 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:225149 9.0 9:0124f23a:::benchmark_data_smithi070_137000_object103438:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e351)", 2024-09-16T09:41:16.313 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:37:12.384254+0000", 2024-09-16T09:41:16.313 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 243.869104099, 2024-09-16T09:41:16.313 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.128723189, 2024-09-16T09:41:16.313 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:41:16.313 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:41:16.313 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "reached pg", 2024-09-16T09:41:16.313 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:41:16.313 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:41:16.313 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:41:16.313 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 225149 2024-09-16T09:41:16.313 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.313 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:41:16.314 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.314 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:41:16.314 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.384254+0000", 2024-09-16T09:41:16.314 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:41:16.314 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.314 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.314 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:41:16.314 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.384254+0000", 2024-09-16T09:41:16.314 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:41:16.314 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.314 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.314 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:41:16.314 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.384255+0000", 2024-09-16T09:41:16.314 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.35e-06 2024-09-16T09:41:16.314 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.315 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.315 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:41:16.315 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.384260+0000", 2024-09-16T09:41:16.315 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.729e-06 2024-09-16T09:41:16.315 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.315 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.315 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:41:16.315 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.384273+0000", 2024-09-16T09:41:16.315 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.3456e-05 2024-09-16T09:41:16.315 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.315 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.315 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:41:16.315 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.384281+0000", 2024-09-16T09:41:16.315 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.515e-06 2024-09-16T09:41:16.316 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.316 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.316 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:41:16.316 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.384332+0000", 2024-09-16T09:41:16.316 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.0850000000000003e-05 2024-09-16T09:41:16.316 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.316 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.316 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:41:16.316 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.384455+0000", 2024-09-16T09:41:16.316 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00012285599999999999 2024-09-16T09:41:16.316 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.316 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.316 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 1,2", 2024-09-16T09:41:16.317 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.384564+0000", 2024-09-16T09:41:16.317 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00010888 2024-09-16T09:41:16.317 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.317 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.317 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:41:16.317 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.387738+0000", 2024-09-16T09:41:16.317 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.003174507 2024-09-16T09:41:16.317 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.317 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.317 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:41:16.317 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.394990+0000", 2024-09-16T09:41:16.317 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0072515399999999999 2024-09-16T09:41:16.317 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.317 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.318 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:41:16.318 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.400340+0000", 2024-09-16T09:41:16.318 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0053498829999999997 2024-09-16T09:41:16.318 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.318 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.318 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:41:16.318 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.400349+0000", 2024-09-16T09:41:16.318 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.9580000000000002e-06 2024-09-16T09:41:16.318 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.318 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.318 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:41:16.318 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:13.512961+0000", 2024-09-16T09:41:16.318 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1126110739999999 2024-09-16T09:41:16.318 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.319 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.319 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:41:16.319 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:13.512977+0000", 2024-09-16T09:41:16.319 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.6591000000000001e-05 2024-09-16T09:41:16.319 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:41:16.319 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:41:16.319 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:41:16.319 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.319 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.319 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:225159 9.0 9:033db585:::benchmark_data_smithi070_137000_object103448:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e351)", 2024-09-16T09:41:16.319 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:37:12.389563+0000", 2024-09-16T09:41:16.319 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 243.863794912, 2024-09-16T09:41:16.319 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1234441260000001, 2024-09-16T09:41:16.319 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:41:16.320 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:41:16.320 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "reached pg", 2024-09-16T09:41:16.320 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:41:16.320 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:41:16.320 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:41:16.320 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 225159 2024-09-16T09:41:16.320 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.320 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:41:16.320 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.320 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:41:16.320 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.389563+0000", 2024-09-16T09:41:16.320 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:41:16.320 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.321 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.321 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:41:16.321 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.389563+0000", 2024-09-16T09:41:16.321 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:41:16.321 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.321 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.321 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:41:16.321 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.389565+0000", 2024-09-16T09:41:16.321 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.86e-06 2024-09-16T09:41:16.321 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.321 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.321 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:41:16.321 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.389571+0000", 2024-09-16T09:41:16.321 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.2740000000000004e-06 2024-09-16T09:41:16.322 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.322 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.322 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:41:16.322 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.389590+0000", 2024-09-16T09:41:16.322 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.8753000000000002e-05 2024-09-16T09:41:16.322 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.322 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.322 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:41:16.322 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.389599+0000", 2024-09-16T09:41:16.322 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.5380000000000002e-06 2024-09-16T09:41:16.322 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.322 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.322 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:41:16.322 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.389654+0000", 2024-09-16T09:41:16.323 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.5142000000000002e-05 2024-09-16T09:41:16.323 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.323 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.323 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:41:16.323 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.389768+0000", 2024-09-16T09:41:16.323 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00011462400000000001 2024-09-16T09:41:16.323 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.323 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.323 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 1,2", 2024-09-16T09:41:16.323 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.389871+0000", 2024-09-16T09:41:16.323 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000103113 2024-09-16T09:41:16.323 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.323 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.324 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:41:16.324 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.395092+0000", 2024-09-16T09:41:16.324 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.005220298 2024-09-16T09:41:16.324 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.324 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.324 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:41:16.324 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.400656+0000", 2024-09-16T09:41:16.324 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0055647259999999999 2024-09-16T09:41:16.324 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.324 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.324 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:41:16.324 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.400664+0000", 2024-09-16T09:41:16.324 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.6729999999999992e-06 2024-09-16T09:41:16.324 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.325 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.325 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:41:16.325 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:13.513002+0000", 2024-09-16T09:41:16.325 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1123373750000001 2024-09-16T09:41:16.325 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.325 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.325 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:41:16.325 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:13.513007+0000", 2024-09-16T09:41:16.325 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.75e-06 2024-09-16T09:41:16.325 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:41:16.325 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:41:16.325 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:41:16.325 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.325 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.326 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:225235 9.0 9.50884480 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e352)", 2024-09-16T09:41:16.326 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:37:12.434208+0000", 2024-09-16T09:41:16.326 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 243.81914980900001, 2024-09-16T09:41:16.326 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0788204320000001, 2024-09-16T09:41:16.326 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:41:16.326 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:41:16.326 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "reached pg", 2024-09-16T09:41:16.326 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:41:16.326 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:41:16.326 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:41:16.326 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 225235 2024-09-16T09:41:16.326 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.326 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:41:16.326 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.327 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:41:16.327 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.434208+0000", 2024-09-16T09:41:16.327 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:41:16.327 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.327 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.327 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:41:16.327 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.434208+0000", 2024-09-16T09:41:16.327 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:41:16.327 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.327 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.327 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:41:16.327 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.434209+0000", 2024-09-16T09:41:16.327 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.158e-06 2024-09-16T09:41:16.328 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.328 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.328 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:41:16.328 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.434215+0000", 2024-09-16T09:41:16.328 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.1699999999999996e-06 2024-09-16T09:41:16.328 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.328 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.328 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:41:16.328 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.434228+0000", 2024-09-16T09:41:16.328 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.332e-05 2024-09-16T09:41:16.328 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.328 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.328 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:41:16.328 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.434235+0000", 2024-09-16T09:41:16.329 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.8859999999999998e-06 2024-09-16T09:41:16.329 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.329 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.329 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:41:16.329 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.434292+0000", 2024-09-16T09:41:16.329 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.7704999999999997e-05 2024-09-16T09:41:16.329 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.329 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.329 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:41:16.329 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.434301+0000", 2024-09-16T09:41:16.329 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.2260000000000005e-06 2024-09-16T09:41:16.329 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.329 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.329 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:41:16.330 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:13.513024+0000", 2024-09-16T09:41:16.330 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0787235479999999 2024-09-16T09:41:16.330 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.330 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.330 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:41:16.330 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:13.513029+0000", 2024-09-16T09:41:16.330 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.4190000000000004e-06 2024-09-16T09:41:16.330 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:41:16.330 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:41:16.330 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:41:16.330 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.330 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.330 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:225256 9.10 9.c1fc3710 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e352)", 2024-09-16T09:41:16.330 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:37:12.441649+0000", 2024-09-16T09:41:16.331 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 243.81170924200001, 2024-09-16T09:41:16.331 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0708169569999999, 2024-09-16T09:41:16.331 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:41:16.331 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:41:16.331 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "reached pg", 2024-09-16T09:41:16.331 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:41:16.331 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:41:16.331 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:41:16.331 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 225256 2024-09-16T09:41:16.331 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.331 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:41:16.331 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.331 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:41:16.332 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.441649+0000", 2024-09-16T09:41:16.332 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:41:16.332 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.332 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.332 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:41:16.332 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.441649+0000", 2024-09-16T09:41:16.332 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:41:16.332 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.332 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.332 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:41:16.332 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.441650+0000", 2024-09-16T09:41:16.332 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.47e-07 2024-09-16T09:41:16.332 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.332 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.333 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:41:16.333 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.441654+0000", 2024-09-16T09:41:16.333 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.6589999999999998e-06 2024-09-16T09:41:16.333 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.333 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.333 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:41:16.333 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.441671+0000", 2024-09-16T09:41:16.333 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.6897000000000001e-05 2024-09-16T09:41:16.333 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.333 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.333 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:41:16.333 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.441681+0000", 2024-09-16T09:41:16.333 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.9129999999999998e-06 2024-09-16T09:41:16.333 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.334 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.334 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:41:16.334 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.441783+0000", 2024-09-16T09:41:16.334 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00010139700000000001 2024-09-16T09:41:16.334 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.334 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.334 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:41:16.334 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.441792+0000", 2024-09-16T09:41:16.334 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.0710000000000001e-06 2024-09-16T09:41:16.334 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.334 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.334 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:41:16.334 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:13.512445+0000", 2024-09-16T09:41:16.334 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.070653525 2024-09-16T09:41:16.335 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.335 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.335 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:41:16.335 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:13.512466+0000", 2024-09-16T09:41:16.335 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.0548e-05 2024-09-16T09:41:16.335 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:41:16.335 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:41:16.335 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:41:16.335 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.335 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.335 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:225291 9.0 9.9752b0a0 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e352)", 2024-09-16T09:41:16.335 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:37:12.458884+0000", 2024-09-16T09:41:16.335 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 243.79447430499999, 2024-09-16T09:41:16.335 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0541780789999999, 2024-09-16T09:41:16.336 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:41:16.336 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:41:16.336 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "reached pg", 2024-09-16T09:41:16.336 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:41:16.336 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:41:16.336 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:41:16.336 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 225291 2024-09-16T09:41:16.336 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.336 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:41:16.336 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.336 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:41:16.336 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.458884+0000", 2024-09-16T09:41:16.336 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:41:16.336 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.337 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.337 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:41:16.337 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.458884+0000", 2024-09-16T09:41:16.337 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:41:16.337 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.337 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.337 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:41:16.337 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.458885+0000", 2024-09-16T09:41:16.337 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.4399999999999998e-07 2024-09-16T09:41:16.337 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.337 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.337 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:41:16.337 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.458889+0000", 2024-09-16T09:41:16.337 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.1400000000000002e-06 2024-09-16T09:41:16.338 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.338 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.338 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:41:16.338 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.458907+0000", 2024-09-16T09:41:16.338 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.8561000000000001e-05 2024-09-16T09:41:16.338 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.338 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.338 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:41:16.338 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.458915+0000", 2024-09-16T09:41:16.338 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.2350000000000002e-06 2024-09-16T09:41:16.338 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.338 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.338 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:41:16.339 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.458986+0000", 2024-09-16T09:41:16.339 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.1273999999999996e-05 2024-09-16T09:41:16.339 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.339 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.339 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:41:16.339 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.458998+0000", 2024-09-16T09:41:16.339 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.2004999999999999e-05 2024-09-16T09:41:16.339 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.339 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.339 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:41:16.339 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:13.513039+0000", 2024-09-16T09:41:16.339 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.054041204 2024-09-16T09:41:16.339 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.339 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.339 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:41:16.340 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:13.513062+0000", 2024-09-16T09:41:16.340 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.2716000000000001e-05 2024-09-16T09:41:16.340 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:41:16.340 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:41:16.340 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:41:16.340 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.340 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.340 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:225292 9.0 9.fcff160 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e352)", 2024-09-16T09:41:16.340 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:37:12.458943+0000", 2024-09-16T09:41:16.340 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 243.79441491, 2024-09-16T09:41:16.340 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0541260480000001, 2024-09-16T09:41:16.340 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:41:16.340 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:41:16.341 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "reached pg", 2024-09-16T09:41:16.341 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:41:16.341 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:41:16.341 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:41:16.341 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 225292 2024-09-16T09:41:16.341 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.341 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:41:16.341 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.341 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:41:16.341 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.458943+0000", 2024-09-16T09:41:16.341 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:41:16.341 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.341 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.341 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:41:16.342 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.458943+0000", 2024-09-16T09:41:16.342 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:41:16.342 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.342 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.342 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:41:16.342 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.458944+0000", 2024-09-16T09:41:16.342 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.7899999999999999e-07 2024-09-16T09:41:16.342 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.342 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.342 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:41:16.342 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.458946+0000", 2024-09-16T09:41:16.342 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.4619999999999999e-06 2024-09-16T09:41:16.342 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.343 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.343 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:41:16.343 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.458953+0000", 2024-09-16T09:41:16.343 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.8390000000000001e-06 2024-09-16T09:41:16.343 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.343 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.343 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:41:16.343 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.458958+0000", 2024-09-16T09:41:16.343 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.0010000000000001e-06 2024-09-16T09:41:16.343 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.343 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.343 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:41:16.343 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.459049+0000", 2024-09-16T09:41:16.343 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.1378000000000005e-05 2024-09-16T09:41:16.344 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.344 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.344 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:41:16.344 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.459059+0000", 2024-09-16T09:41:16.344 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.5519999999999993e-06 2024-09-16T09:41:16.344 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.344 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.344 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:41:16.344 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:13.513061+0000", 2024-09-16T09:41:16.344 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.054001653 2024-09-16T09:41:16.344 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.344 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.344 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:41:16.344 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:13.513069+0000", 2024-09-16T09:41:16.345 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.6840000000000002e-06 2024-09-16T09:41:16.345 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:41:16.345 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:41:16.345 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:41:16.345 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.345 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.345 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:225293 9.10 9.ab745fd0 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e352)", 2024-09-16T09:41:16.345 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:37:12.459755+0000", 2024-09-16T09:41:16.345 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 243.79360322900001, 2024-09-16T09:41:16.345 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0531002519999999, 2024-09-16T09:41:16.345 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:41:16.345 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:41:16.345 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "reached pg", 2024-09-16T09:41:16.345 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:41:16.346 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:41:16.346 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:41:16.346 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 225293 2024-09-16T09:41:16.346 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.346 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:41:16.346 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.346 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:41:16.346 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.459755+0000", 2024-09-16T09:41:16.346 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:41:16.346 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.346 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.346 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:41:16.346 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.459755+0000", 2024-09-16T09:41:16.347 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:41:16.347 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.347 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.347 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:41:16.347 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.459756+0000", 2024-09-16T09:41:16.347 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.9699999999999995e-07 2024-09-16T09:41:16.347 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.347 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.347 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:41:16.347 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.459759+0000", 2024-09-16T09:41:16.347 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.782e-06 2024-09-16T09:41:16.347 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.347 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.347 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:41:16.348 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.459772+0000", 2024-09-16T09:41:16.348 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.253e-05 2024-09-16T09:41:16.348 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.348 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.348 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:41:16.348 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.459779+0000", 2024-09-16T09:41:16.348 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.6980000000000001e-06 2024-09-16T09:41:16.348 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.348 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.348 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:41:16.348 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.459823+0000", 2024-09-16T09:41:16.348 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.3968999999999999e-05 2024-09-16T09:41:16.348 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.348 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.349 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:41:16.349 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.459829+0000", 2024-09-16T09:41:16.349 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.9290000000000003e-06 2024-09-16T09:41:16.349 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.349 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.349 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:41:16.349 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:13.512847+0000", 2024-09-16T09:41:16.349 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.053018314 2024-09-16T09:41:16.349 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.349 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.349 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:41:16.349 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:13.512855+0000", 2024-09-16T09:41:16.349 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.2330000000000006e-06 2024-09-16T09:41:16.349 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:41:16.350 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:41:16.350 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:41:16.350 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.350 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.350 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:231327 9.10 9:0da6d7f3:::benchmark_data_smithi070_137000_object109616:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e357)", 2024-09-16T09:41:16.350 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:37:18.512211+0000", 2024-09-16T09:41:16.350 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 237.74114689999999, 2024-09-16T09:41:16.350 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.104184364, 2024-09-16T09:41:16.350 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:41:16.350 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:41:16.350 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "reached pg", 2024-09-16T09:41:16.350 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:41:16.350 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:41:16.350 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:41:16.351 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 231327 2024-09-16T09:41:16.351 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.351 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:41:16.351 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.351 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:41:16.351 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.512211+0000", 2024-09-16T09:41:16.351 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:41:16.351 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.351 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.351 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:41:16.351 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.512211+0000", 2024-09-16T09:41:16.351 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:41:16.351 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.351 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.352 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:41:16.352 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.512213+0000", 2024-09-16T09:41:16.352 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.8589999999999999e-06 2024-09-16T09:41:16.352 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.352 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.352 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:41:16.352 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.512216+0000", 2024-09-16T09:41:16.352 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.174e-06 2024-09-16T09:41:16.352 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.352 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.352 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:41:16.352 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.512234+0000", 2024-09-16T09:41:16.352 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.7380999999999998e-05 2024-09-16T09:41:16.352 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.353 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.353 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:41:16.353 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.512242+0000", 2024-09-16T09:41:16.353 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.3680000000000002e-06 2024-09-16T09:41:16.353 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.353 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.353 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:41:16.353 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.684712+0000", 2024-09-16T09:41:16.353 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.17247021300000001 2024-09-16T09:41:16.353 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.353 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.353 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:41:16.353 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.684720+0000", 2024-09-16T09:41:16.353 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.2989999999999997e-06 2024-09-16T09:41:16.354 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.354 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.354 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:41:16.354 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.612846+0000", 2024-09-16T09:41:16.354 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.92812537399999995 2024-09-16T09:41:16.354 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.354 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.354 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for missing object", 2024-09-16T09:41:16.354 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.612970+0000", 2024-09-16T09:41:16.354 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00012453200000000001 2024-09-16T09:41:16.354 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.354 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.354 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:41:16.354 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.616209+0000", 2024-09-16T09:41:16.355 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.003238922 2024-09-16T09:41:16.355 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.355 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.355 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:41:16.355 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.616396+0000", 2024-09-16T09:41:16.355 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00018624200000000001 2024-09-16T09:41:16.355 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:41:16.355 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:41:16.355 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:41:16.355 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.355 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.355 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:231328 9.1 9:8132d7b0:::benchmark_data_smithi070_137000_object109617:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e357)", 2024-09-16T09:41:16.355 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:37:18.512255+0000", 2024-09-16T09:41:16.355 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 237.74110318699999, 2024-09-16T09:41:16.356 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0795497709999999, 2024-09-16T09:41:16.356 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:41:16.356 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:41:16.356 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "reached pg", 2024-09-16T09:41:16.356 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:41:16.356 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:41:16.356 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:41:16.356 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 231328 2024-09-16T09:41:16.356 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.356 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:41:16.356 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.356 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:41:16.356 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.512255+0000", 2024-09-16T09:41:16.356 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:41:16.357 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.357 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.357 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:41:16.357 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.512255+0000", 2024-09-16T09:41:16.357 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:41:16.357 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.357 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.357 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:41:16.357 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.512256+0000", 2024-09-16T09:41:16.357 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.034e-06 2024-09-16T09:41:16.357 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.357 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.357 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:41:16.357 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.512260+0000", 2024-09-16T09:41:16.358 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.9129999999999996e-06 2024-09-16T09:41:16.358 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.358 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.358 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:41:16.358 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.512268+0000", 2024-09-16T09:41:16.358 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.7530000000000001e-06 2024-09-16T09:41:16.358 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.358 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.358 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:41:16.358 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.512273+0000", 2024-09-16T09:41:16.358 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.5149999999999997e-06 2024-09-16T09:41:16.358 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.358 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.358 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:41:16.359 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.684946+0000", 2024-09-16T09:41:16.359 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.17267326299999999 2024-09-16T09:41:16.359 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.359 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.359 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:41:16.359 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.684954+0000", 2024-09-16T09:41:16.359 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.4769999999999997e-06 2024-09-16T09:41:16.359 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.359 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.359 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:41:16.359 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.538886+0000", 2024-09-16T09:41:16.359 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.85393251999999997 2024-09-16T09:41:16.359 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.359 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.360 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for missing object", 2024-09-16T09:41:16.360 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.539061+0000", 2024-09-16T09:41:16.360 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000174694 2024-09-16T09:41:16.360 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.360 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.360 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:41:16.360 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.591746+0000", 2024-09-16T09:41:16.360 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.052684624999999999 2024-09-16T09:41:16.360 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.360 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.360 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:41:16.360 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.591805+0000", 2024-09-16T09:41:16.360 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.8977000000000001e-05 2024-09-16T09:41:16.360 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:41:16.361 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:41:16.361 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:41:16.361 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.361 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.361 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:231329 9.11 9:8a8e13aa:::benchmark_data_smithi070_137000_object109618:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e357)", 2024-09-16T09:41:16.361 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:37:18.512281+0000", 2024-09-16T09:41:16.361 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 237.74107733700001, 2024-09-16T09:41:16.361 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1196006590000001, 2024-09-16T09:41:16.361 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:41:16.361 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:41:16.361 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "reached pg", 2024-09-16T09:41:16.361 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:41:16.361 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:41:16.362 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:41:16.362 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 231329 2024-09-16T09:41:16.362 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.362 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:41:16.362 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.362 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:41:16.362 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.512281+0000", 2024-09-16T09:41:16.362 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:41:16.362 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.362 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.362 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:41:16.362 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.512281+0000", 2024-09-16T09:41:16.362 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:41:16.362 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.363 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.363 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:41:16.363 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.512281+0000", 2024-09-16T09:41:16.363 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.3700000000000001e-07 2024-09-16T09:41:16.363 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.363 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.363 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:41:16.363 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.512283+0000", 2024-09-16T09:41:16.363 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.39e-06 2024-09-16T09:41:16.363 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.363 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.363 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:41:16.363 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.512291+0000", 2024-09-16T09:41:16.363 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.17e-06 2024-09-16T09:41:16.363 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.364 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.364 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:41:16.364 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.512295+0000", 2024-09-16T09:41:16.364 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.2370000000000003e-06 2024-09-16T09:41:16.364 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.364 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.364 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:41:16.364 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.684979+0000", 2024-09-16T09:41:16.364 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.17268373300000001 2024-09-16T09:41:16.364 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.364 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.364 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:41:16.364 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.684985+0000", 2024-09-16T09:41:16.364 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.4239999999999998e-06 2024-09-16T09:41:16.365 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.365 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.365 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:41:16.365 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.627360+0000", 2024-09-16T09:41:16.365 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.94237468999999996 2024-09-16T09:41:16.365 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.365 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.365 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for missing object", 2024-09-16T09:41:16.365 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.627557+0000", 2024-09-16T09:41:16.365 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00019702200000000001 2024-09-16T09:41:16.365 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.365 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.365 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:41:16.365 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.631826+0000", 2024-09-16T09:41:16.366 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0042689950000000003 2024-09-16T09:41:16.366 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.366 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.366 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:41:16.366 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.631881+0000", 2024-09-16T09:41:16.366 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.5661000000000001e-05 2024-09-16T09:41:16.366 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:41:16.366 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:41:16.366 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:41:16.366 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.366 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.366 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:231331 9.10 9:0df6dab9:::benchmark_data_smithi070_137000_object109620:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e357)", 2024-09-16T09:41:16.366 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:37:18.539893+0000", 2024-09-16T09:41:16.366 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 237.71346457000001, 2024-09-16T09:41:16.366 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0771409569999999, 2024-09-16T09:41:16.367 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:41:16.367 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:41:16.367 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:41:16.367 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:41:16.367 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:41:16.367 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:41:16.367 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 231331 2024-09-16T09:41:16.367 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.367 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:41:16.367 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.367 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:41:16.367 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.539893+0000", 2024-09-16T09:41:16.367 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:41:16.367 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.368 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.368 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:41:16.368 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.539893+0000", 2024-09-16T09:41:16.368 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:41:16.368 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.368 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.368 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:41:16.368 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.539896+0000", 2024-09-16T09:41:16.368 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.4310000000000001e-06 2024-09-16T09:41:16.368 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.368 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.368 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:41:16.368 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.542675+0000", 2024-09-16T09:41:16.368 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0027787900000000002 2024-09-16T09:41:16.369 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.369 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.369 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:41:16.369 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.542701+0000", 2024-09-16T09:41:16.369 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.6122000000000002e-05 2024-09-16T09:41:16.369 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.369 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.369 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:41:16.369 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:41:16.370 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:16 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/224185582' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:41:16.370 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.542712+0000", 2024-09-16T09:41:16.370 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1443e-05 2024-09-16T09:41:16.370 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.370 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.371 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:41:16.371 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.684747+0000", 2024-09-16T09:41:16.371 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.14203510999999999 2024-09-16T09:41:16.371 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.371 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.371 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:41:16.371 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.684754+0000", 2024-09-16T09:41:16.371 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.7320000000000001e-06 2024-09-16T09:41:16.371 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.371 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.371 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:41:16.371 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.613242+0000", 2024-09-16T09:41:16.371 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.92848750300000005 2024-09-16T09:41:16.372 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.372 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.372 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:41:16.372 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.613347+0000", 2024-09-16T09:41:16.372 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000105361 2024-09-16T09:41:16.372 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.372 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.372 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 1,2", 2024-09-16T09:41:16.372 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.613462+0000", 2024-09-16T09:41:16.372 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00011495 2024-09-16T09:41:16.372 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.372 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.372 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:41:16.372 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.615859+0000", 2024-09-16T09:41:16.373 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0023971729999999998 2024-09-16T09:41:16.373 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.373 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.373 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:41:16.373 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.616173+0000", 2024-09-16T09:41:16.373 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00031437799999999998 2024-09-16T09:41:16.373 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.373 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.373 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:41:16.373 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.616845+0000", 2024-09-16T09:41:16.373 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00067190000000000001 2024-09-16T09:41:16.373 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.373 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.373 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:41:16.374 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.616931+0000", 2024-09-16T09:41:16.374 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.5439000000000006e-05 2024-09-16T09:41:16.374 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.374 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.374 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:41:16.374 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.617034+0000", 2024-09-16T09:41:16.374 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000103625 2024-09-16T09:41:16.374 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:41:16.374 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:41:16.374 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:41:16.374 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.374 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.374 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:231332 9.1 9:83c10802:::benchmark_data_smithi070_137000_object109621:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e357)", 2024-09-16T09:41:16.374 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:37:18.542736+0000", 2024-09-16T09:41:16.375 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 237.710621866, 2024-09-16T09:41:16.375 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0489751380000001, 2024-09-16T09:41:16.375 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:41:16.375 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:41:16.375 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:41:16.375 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:41:16.375 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:41:16.375 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:41:16.375 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 231332 2024-09-16T09:41:16.375 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.375 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:41:16.375 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.375 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:41:16.375 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.542736+0000", 2024-09-16T09:41:16.376 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:41:16.376 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.376 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.376 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:41:16.376 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.542736+0000", 2024-09-16T09:41:16.376 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:41:16.376 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.376 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.376 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:41:16.376 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.542738+0000", 2024-09-16T09:41:16.376 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.3480000000000001e-06 2024-09-16T09:41:16.376 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.376 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.376 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:41:16.377 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.542740+0000", 2024-09-16T09:41:16.377 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.261e-06 2024-09-16T09:41:16.377 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.377 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.377 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:41:16.377 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.542746+0000", 2024-09-16T09:41:16.377 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.2670000000000003e-06 2024-09-16T09:41:16.377 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.377 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.377 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:41:16.377 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.542751+0000", 2024-09-16T09:41:16.377 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.8010000000000003e-06 2024-09-16T09:41:16.377 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.377 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.378 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:41:16.378 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.685007+0000", 2024-09-16T09:41:16.378 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.14225658599999999 2024-09-16T09:41:16.378 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.378 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.378 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:41:16.378 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.685016+0000", 2024-09-16T09:41:16.378 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.5199999999999997e-06 2024-09-16T09:41:16.378 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.378 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.378 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:41:16.378 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.542235+0000", 2024-09-16T09:41:16.378 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.85721875300000006 2024-09-16T09:41:16.378 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.379 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.379 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:41:16.379 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.542315+0000", 2024-09-16T09:41:16.379 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.0000999999999995e-05 2024-09-16T09:41:16.379 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.379 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.379 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 3,7", 2024-09-16T09:41:16.379 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.542396+0000", 2024-09-16T09:41:16.379 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.1600000000000005e-05 2024-09-16T09:41:16.379 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.379 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.379 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:41:16.379 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.543116+0000", 2024-09-16T09:41:16.379 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00071917400000000005 2024-09-16T09:41:16.379 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.380 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.380 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:41:16.380 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.546040+0000", 2024-09-16T09:41:16.380 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0029248270000000001 2024-09-16T09:41:16.380 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.380 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.380 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:41:16.380 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.591579+0000", 2024-09-16T09:41:16.380 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.045538617000000003 2024-09-16T09:41:16.380 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.380 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.380 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:41:16.380 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.591626+0000", 2024-09-16T09:41:16.380 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.7274000000000002e-05 2024-09-16T09:41:16.381 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:41:16.381 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:41:16.381 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:41:16.381 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.591711+0000", 2024-09-16T09:41:16.381 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.5109000000000001e-05 2024-09-16T09:41:16.381 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:41:16.381 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:41:16.381 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:41:16.381 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:41:16.381 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:41:16.381 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-09-16T09:41:16.381 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:16 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/224185582' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:41:16.470 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:41:16.492 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:16.492 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:16.493 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:41:16.593 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:41:16.615 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:16.616 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:16.616 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:41:16.717 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:41:16.740 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:16.740 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:16.741 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:41:16.841 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:41:16.863 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:16.863 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:16.863 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:41:16.964 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:41:16.986 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:16.986 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:16.986 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:41:17.063 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:41:17.087 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:41:17.108 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:17.109 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:17.109 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:41:17.210 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:41:17.231 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:17.231 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:17.232 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:41:17.305 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:17 smithi084 ceph-mon[32629]: pgmap v859: 168 pgs: 1 creating+activating, 8 creating+peering, 159 active+clean; 583 KiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:41:17.319 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:17 smithi070 ceph-mon[30388]: pgmap v859: 168 pgs: 1 creating+activating, 8 creating+peering, 159 active+clean; 583 KiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:41:17.319 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:17 smithi070 ceph-mon[40691]: pgmap v859: 168 pgs: 1 creating+activating, 8 creating+peering, 159 active+clean; 583 KiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:41:17.332 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.7 dump_ops_in_flight 2024-09-16T09:41:17.333 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:41:17.355 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:17.356 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:17.356 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:41:17.457 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:41:17.478 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:17.479 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:17.479 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:41:17.580 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:41:17.580 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:41:17.602 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:17.603 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:17.603 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:41:17.704 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:41:17.704 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-16T09:41:17.727 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:17.727 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:17.728 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:41:17.828 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:41:17.851 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:17.852 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:17.852 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:41:17.951 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:41:17.953 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:41:17.976 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:17.977 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:17.977 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:41:18.009 INFO:teuthology.orchestra.run.smithi070.stderr:enabled application 'rados' on pool 'unique_pool_4' 2024-09-16T09:41:18.060 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-09-16T09:41:18.060 INFO:teuthology.orchestra.run.smithi084.stdout: "ops": [], 2024-09-16T09:41:18.061 INFO:teuthology.orchestra.run.smithi084.stdout: "num_ops": 0 2024-09-16T09:41:18.061 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-09-16T09:41:18.078 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:41:18.101 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:18.101 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:18.102 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:41:18.202 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:41:18.224 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:18.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:18.225 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:41:18.271 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:18 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1639919627' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "unique_pool_4", "app": "rados", "yes_i_really_mean_it": true} : dispatch 2024-09-16T09:41:18.271 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:18 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1639919627' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "unique_pool_4", "app": "rados", "yes_i_really_mean_it": true} : dispatch 2024-09-16T09:41:18.326 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:41:18.348 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:18.348 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:18.349 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:41:18.361 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:18 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1639919627' entity='client.admin' cmd={"prefix": "osd pool application enable", "pool": "unique_pool_4", "app": "rados", "yes_i_really_mean_it": true} : dispatch 2024-09-16T09:41:18.450 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:41:18.471 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:18.472 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:18.472 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:41:18.573 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:41:18.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:18.594 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:18.595 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:41:18.682 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.7 dump_blocked_ops 2024-09-16T09:41:18.695 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:41:18.745 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:18.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:18.746 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:41:18.847 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:41:18.868 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:18.869 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:18.869 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:41:18.928 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:41:18.970 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:41:18.993 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:18.994 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:18.994 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:41:19.020 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T09:41:19.095 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:41:19.116 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:19.117 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:19.117 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:41:19.218 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:41:19.240 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:19.241 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:19.241 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:41:19.246 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:19 smithi070 ceph-mon[30388]: pgmap v860: 168 pgs: 1 creating+activating, 4 creating+peering, 163 active+clean; 583 KiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-16T09:41:19.246 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:19 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1639919627' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_4", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-16T09:41:19.246 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:19 smithi070 ceph-mon[30388]: osdmap e458: 8 total, 8 up, 8 in 2024-09-16T09:41:19.246 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:19 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:41:19.246 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:19 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:41:19.247 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:19 smithi070 ceph-mon[40691]: pgmap v860: 168 pgs: 1 creating+activating, 4 creating+peering, 163 active+clean; 583 KiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-16T09:41:19.247 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:19 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1639919627' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_4", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-16T09:41:19.247 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:19 smithi070 ceph-mon[40691]: osdmap e458: 8 total, 8 up, 8 in 2024-09-16T09:41:19.247 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:19 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:41:19.247 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:19 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:41:19.307 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:19 smithi084 ceph-mon[32629]: pgmap v860: 168 pgs: 1 creating+activating, 4 creating+peering, 163 active+clean; 583 KiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-16T09:41:19.307 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:19 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1639919627' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_4", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-16T09:41:19.307 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:19 smithi084 ceph-mon[32629]: osdmap e458: 8 total, 8 up, 8 in 2024-09-16T09:41:19.307 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:19 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:41:19.307 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:19 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:41:19.321 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:41:19.342 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:41:19.363 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:19.364 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:19.364 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:41:19.423 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-09-16T09:41:19.423 INFO:teuthology.orchestra.run.smithi084.stdout: "ops": [], 2024-09-16T09:41:19.424 INFO:teuthology.orchestra.run.smithi084.stdout: "complaint_time": 30, 2024-09-16T09:41:19.424 INFO:teuthology.orchestra.run.smithi084.stdout: "num_blocked_ops": 0 2024-09-16T09:41:19.424 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-09-16T09:41:19.465 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:41:19.487 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:19.488 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:19.488 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:41:19.589 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:41:19.611 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:19.612 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:19.612 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:41:19.713 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:41:19.734 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:19.735 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:19.735 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:41:19.816 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:41:19.817 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:41:19.833 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:41:19.833 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:41:19.836 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:41:19.850 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:41:19.850 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:41:19.856 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:19.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:19.857 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:41:19.866 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:41:19.866 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:41:19.883 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:41:19.884 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:41:19.900 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:41:19.901 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:41:19.917 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:41:19.918 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:41:19.934 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:41:19.934 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:41:19.957 DEBUG:teuthology.orchestra.run.smithi070:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.0 -b 65536 --object-size 65536 -p unique_pool_4 bench 90 write' 2024-09-16T09:41:19.957 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:41:19.960 INFO:tasks.radosbench:joining radosbench (timing out after 3000s) 2024-09-16T09:41:19.960 INFO:teuthology.orchestra.run:waiting for 3000 2024-09-16T09:41:19.979 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:19.979 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:19.980 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:41:20.081 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:41:20.088 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:hints = 1 2024-09-16T09:41:20.088 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Maintaining 16 concurrent writes of 65536 bytes to objects of size 65536 for up to 90 seconds or 0 objects 2024-09-16T09:41:20.088 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Object prefix: benchmark_data_smithi070_191495 2024-09-16T09:41:20.088 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-16T09:41:20.089 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 0 0 0 0 0 0 - 0 2024-09-16T09:41:20.115 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:20.116 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:20.116 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:41:20.118 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.7 dump_historic_ops 2024-09-16T09:41:20.217 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:41:20.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:20.255 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:20.255 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:41:20.356 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:41:20.389 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:20.390 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:20.391 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:41:20.420 INFO:teuthology.orchestra.run.smithi070.stderr:nodeep-scrub is set 2024-09-16T09:41:20.492 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:41:20.532 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:20.532 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:20.533 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:41:20.556 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:41:20.634 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:41:20.669 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:20.669 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:20.670 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:41:20.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:20 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1365332954' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:41:20.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:20 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/207759640' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:41:20.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:20 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1365332954' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:41:20.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:20 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/207759640' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:41:20.690 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:20 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1365332954' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:41:20.690 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:20 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/207759640' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:41:20.771 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:41:20.807 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:20.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:20.809 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:41:20.909 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:41:20.947 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:20.947 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:20.948 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:41:21.049 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:41:21.086 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:21.087 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:21.087 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:41:21.088 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 1 16 3698 3682 230.109 230.125 0.00246179 0.00432784 2024-09-16T09:41:21.188 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:41:21.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:21.224 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:21.224 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:41:21.231 INFO:teuthology.orchestra.run.smithi084.stdout:ERROR: (22) Invalid argument 2024-09-16T09:41:21.231 INFO:teuthology.orchestra.run.smithi084.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:41:21.325 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:41:21.361 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:21.362 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:21.362 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:41:21.383 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T09:41:21.383 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T09:41:21.383 INFO:tasks.thrashosds.thrasher:inject_pause on osd.6 2024-09-16T09:41:21.384 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-16T09:41:21.384 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-16T09:41:21.384 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.6 config set filestore_inject_stall 3 2024-09-16T09:41:21.463 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:41:21.498 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:21.499 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:21.499 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:41:21.588 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:21 smithi070 ceph-mon[30388]: pgmap v862: 168 pgs: 168 active+clean; 583 KiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:41:21.589 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:21 smithi070 ceph-mon[30388]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-16T09:41:21.589 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:21 smithi070 ceph-mon[30388]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:41:21.589 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:21 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1365332954' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-16T09:41:21.589 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:21 smithi070 ceph-mon[30388]: osdmap e459: 8 total, 8 up, 8 in 2024-09-16T09:41:21.589 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:21 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:41:21.589 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:21 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:41:21.589 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:21 smithi070 ceph-mon[40691]: pgmap v862: 168 pgs: 168 active+clean; 583 KiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:41:21.589 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:21 smithi070 ceph-mon[40691]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-16T09:41:21.589 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:21 smithi070 ceph-mon[40691]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:41:21.590 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:21 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1365332954' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-16T09:41:21.590 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:21 smithi070 ceph-mon[40691]: osdmap e459: 8 total, 8 up, 8 in 2024-09-16T09:41:21.590 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:21 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:41:21.590 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:21 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:41:21.600 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:41:21.635 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:21.636 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:21.636 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:41:21.737 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:41:21.772 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:21.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:21.773 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:41:21.874 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:41:21.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:21 smithi084 ceph-mon[32629]: pgmap v862: 168 pgs: 168 active+clean; 583 KiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:41:21.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:21 smithi084 ceph-mon[32629]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-16T09:41:21.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:21 smithi084 ceph-mon[32629]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:41:21.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:21 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1365332954' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-16T09:41:21.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:21 smithi084 ceph-mon[32629]: osdmap e459: 8 total, 8 up, 8 in 2024-09-16T09:41:21.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:21 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:41:21.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:21 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:41:21.912 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:21.912 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:21.912 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:41:22.013 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:41:22.035 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:22.036 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:22.036 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:41:22.089 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 2 16 6802 6786 212.046 194 0.00705336 0.00470678 2024-09-16T09:41:22.137 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:41:22.170 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:22.171 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:22.171 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:41:22.272 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:41:22.309 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:22.310 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:22.310 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:41:22.411 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:41:22.420 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:41:21 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:41:21] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:41:22.450 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:22.450 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:22.451 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:41:22.551 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:41:22.587 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:22.588 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:22.589 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:41:22.651 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.0 dump_ops_in_flight 2024-09-16T09:41:22.679 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:41:22.684 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:22 smithi070 ceph-mon[30388]: osdmap e460: 8 total, 8 up, 8 in 2024-09-16T09:41:22.684 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:22 smithi070 ceph-mon[40691]: osdmap e460: 8 total, 8 up, 8 in 2024-09-16T09:41:22.690 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:41:22.727 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:22.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:22.729 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:41:22.813 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:22 smithi084 ceph-mon[32629]: osdmap e460: 8 total, 8 up, 8 in 2024-09-16T09:41:22.830 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:41:22.872 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:22.873 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:22.873 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:41:22.937 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T09:41:22.974 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:41:23.034 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:23.035 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:23.035 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:41:23.089 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:41:23.089 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 3 16 10034 10018 208.688 202 0.0094507 0.00477919 2024-09-16T09:41:23.136 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:41:23.157 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:23.157 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:23.158 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:41:23.259 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:41:23.296 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:23.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:23.297 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:41:23.398 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:41:23.437 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:23.438 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:23.439 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:41:23.540 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:41:23.560 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:23 smithi070 ceph-mon[30388]: pgmap v865: 168 pgs: 168 active+clean; 583 KiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:41:23.560 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:23 smithi070 ceph-mon[40691]: pgmap v865: 168 pgs: 168 active+clean; 583 KiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:41:23.574 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:23.574 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:23.575 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:41:23.642 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd unset noscrub 2024-09-16T09:41:23.676 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:41:23.685 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T09:41:23.685 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:41:23.726 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:23.726 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:23.727 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:41:23.827 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:41:23.857 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:23.858 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:23.858 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:41:23.910 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:41:23 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:41:23.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:23 smithi084 ceph-mon[32629]: pgmap v865: 168 pgs: 168 active+clean; 583 KiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:41:23.959 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:41:23.996 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:23.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:23.997 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:41:24.089 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 4 16 12427 12411 193.903 149.562 0.00544535 0.00514946 2024-09-16T09:41:24.097 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:41:24.133 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:24.134 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:24.135 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:41:24.184 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-09-16T09:41:24.184 INFO:teuthology.orchestra.run.smithi084.stdout: "success": "" 2024-09-16T09:41:24.184 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-09-16T09:41:24.235 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:41:24.271 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:24.272 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:24.273 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:41:24.374 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:41:24.378 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.0 dump_blocked_ops 2024-09-16T09:41:24.399 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:41:24.415 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:24.416 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:24.416 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:41:24.517 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:41:24.552 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:24.552 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:24.553 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:41:24.584 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:24 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:41:24.585 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:24 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:41:24.653 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:41:24.687 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:24.688 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:24.688 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:41:24.752 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:24 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:41:24.789 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:41:24.825 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:24.825 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:24.826 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:41:24.927 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:41:24.972 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:24.973 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:24.973 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:41:25.074 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:41:25.079 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:41:25.080 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:41:25.090 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 5 16 15043 15027 187.82 163.5 0.00716622 0.00531565 2024-09-16T09:41:25.099 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:41:25.100 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:41:25.103 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:25.103 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:25.104 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:41:25.116 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:41:25.116 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:41:25.155 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:41:25.156 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:41:25.190 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:41:25.190 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:41:25.205 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:41:25.221 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:41:25.221 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:41:25.240 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:25.241 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:25.241 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:41:25.255 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:41:25.255 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:41:25.289 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:41:25.289 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:41:25.320 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:41:25.320 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:41:25.342 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:41:25.378 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:25.379 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:25.379 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:41:25.480 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:41:25.513 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:25.514 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:25.514 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:41:25.615 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:41:25.649 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:25.650 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:25.650 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:41:25.751 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:41:25.783 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:25.784 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:25.784 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:41:25.787 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:25 smithi070 ceph-mon[30388]: pgmap v866: 168 pgs: 168 active+clean; 574 MiB data, 7.0 GiB used, 708 GiB / 715 GiB avail; 853 B/s rd, 96 MiB/s wr, 1.53k op/s 2024-09-16T09:41:25.787 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:25 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2334318039' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:41:25.787 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:25 smithi070 ceph-mon[40691]: pgmap v866: 168 pgs: 168 active+clean; 574 MiB data, 7.0 GiB used, 708 GiB / 715 GiB avail; 853 B/s rd, 96 MiB/s wr, 1.53k op/s 2024-09-16T09:41:25.788 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:25 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/2334318039' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:41:25.794 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:25 smithi084 ceph-mon[32629]: pgmap v866: 168 pgs: 168 active+clean; 574 MiB data, 7.0 GiB used, 708 GiB / 715 GiB avail; 853 B/s rd, 96 MiB/s wr, 1.53k op/s 2024-09-16T09:41:25.794 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:25 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/2334318039' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:41:25.885 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:41:25.922 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:25.923 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:25.923 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:41:26.024 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:41:26.057 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:26.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:26.058 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:41:26.089 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 6 15 17659 17644 183.774 163.562 0.00433699 0.00543494 2024-09-16T09:41:26.159 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:41:26.182 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:26.183 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:26.183 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:41:26.284 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:41:26.326 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:26.327 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:26.327 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:41:26.428 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:41:26.452 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T09:41:26.453 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T09:41:26.453 INFO:teuthology.orchestra.run.smithi070.stdout: "complaint_time": 30, 2024-09-16T09:41:26.453 INFO:teuthology.orchestra.run.smithi070.stdout: "num_blocked_ops": 0 2024-09-16T09:41:26.453 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T09:41:26.458 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:26.458 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:26.459 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:41:26.559 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:41:26.598 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:26.599 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:26.600 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:41:26.701 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:41:26.740 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:26.741 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:26.741 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:41:26.842 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:41:26.875 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:26.876 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:26.876 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:41:26.977 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:41:27.012 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:27.013 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:27.014 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:41:27.089 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 7 16 20043 20027 178.796 148.938 0.00737805 0.0055875 2024-09-16T09:41:27.115 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:41:27.152 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:27.153 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:27.153 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:41:27.254 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:41:27.277 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:27.278 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:27.278 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:41:27.379 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:41:27.412 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:27.413 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:27.413 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:41:27.471 INFO:teuthology.orchestra.run.smithi070.stderr:noscrub is unset 2024-09-16T09:41:27.514 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:41:27.550 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:27.551 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:27.551 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:41:27.619 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:27 smithi070 ceph-mon[30388]: pgmap v867: 168 pgs: 168 active+clean; 629 MiB data, 7.2 GiB used, 708 GiB / 715 GiB avail; 653 B/s rd, 80 MiB/s wr, 1.28k op/s 2024-09-16T09:41:27.620 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:27 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2374070416' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:41:27.620 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:27 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:41:27.620 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:27 smithi070 ceph-mon[40691]: pgmap v867: 168 pgs: 168 active+clean; 629 MiB data, 7.2 GiB used, 708 GiB / 715 GiB avail; 653 B/s rd, 80 MiB/s wr, 1.28k op/s 2024-09-16T09:41:27.620 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:27 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/2374070416' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:41:27.620 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:27 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:41:27.652 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:41:27.677 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.0 dump_historic_ops 2024-09-16T09:41:27.690 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:27.691 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:27.691 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:41:27.792 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:41:27.825 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:27.826 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:27.826 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:41:27.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:27 smithi084 ceph-mon[32629]: pgmap v867: 168 pgs: 168 active+clean; 629 MiB data, 7.2 GiB used, 708 GiB / 715 GiB avail; 653 B/s rd, 80 MiB/s wr, 1.28k op/s 2024-09-16T09:41:27.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:27 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/2374070416' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:41:27.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:27 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:41:27.927 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:41:27.949 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:27.950 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:27.950 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:41:28.051 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:41:28.089 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 8 15 21979 21964 171.578 121.062 0.00674989 0.00582246 2024-09-16T09:41:28.099 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:28.099 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:28.100 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:41:28.200 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:41:28.226 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:28.226 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:28.227 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:41:28.328 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:41:28.364 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:28.365 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:28.365 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:41:28.466 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:41:28.500 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:28.501 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:28.502 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:41:28.602 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:41:28.636 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:28.637 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:28.638 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:41:28.738 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:41:28.773 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:28.774 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:28.774 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:41:28.857 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:28 smithi070 ceph-mon[30388]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:41:28.857 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:28 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-16T09:41:28.857 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:28 smithi070 ceph-mon[30388]: osdmap e461: 8 total, 8 up, 8 in 2024-09-16T09:41:28.857 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:28 smithi070 ceph-mon[30388]: 4.7 scrub starts 2024-09-16T09:41:28.857 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:28 smithi070 ceph-mon[30388]: 4.7 scrub ok 2024-09-16T09:41:28.858 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:28 smithi070 ceph-mon[40691]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:41:28.858 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:28 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-16T09:41:28.858 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:28 smithi070 ceph-mon[40691]: osdmap e461: 8 total, 8 up, 8 in 2024-09-16T09:41:28.858 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:28 smithi070 ceph-mon[40691]: 4.7 scrub starts 2024-09-16T09:41:28.858 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:28 smithi070 ceph-mon[40691]: 4.7 scrub ok 2024-09-16T09:41:28.875 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:41:28.903 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:28.903 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:28.904 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:41:28.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:28 smithi084 ceph-mon[32629]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:41:28.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:28 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-16T09:41:28.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:28 smithi084 ceph-mon[32629]: osdmap e461: 8 total, 8 up, 8 in 2024-09-16T09:41:28.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:28 smithi084 ceph-mon[32629]: 4.7 scrub starts 2024-09-16T09:41:28.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:28 smithi084 ceph-mon[32629]: 4.7 scrub ok 2024-09-16T09:41:28.927 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T09:41:29.005 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:41:29.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:29.053 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:29.053 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:41:29.089 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 9 16 24228 24212 168.12 140.5 0.00536586 0.0059425 2024-09-16T09:41:29.154 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:41:29.191 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:29.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:29.192 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:41:29.293 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:41:29.333 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:29.334 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:29.334 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:41:29.435 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:41:29.462 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:29.462 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:29.463 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:41:29.563 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:41:29.596 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:29.596 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:29.597 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:41:29.698 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:41:29.733 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:29.734 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:29.734 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:41:29.835 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:41:29.869 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:29.869 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:29.870 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:41:29.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:29 smithi084 ceph-mon[32629]: pgmap v869: 168 pgs: 168 active+clean; 828 MiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 1.3 KiB/s rd, 112 MiB/s wr, 1.78k op/s 2024-09-16T09:41:29.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:29 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:41:29.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:29 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:41:29.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:29 smithi084 ceph-mon[32629]: osdmap e462: 8 total, 8 up, 8 in 2024-09-16T09:41:29.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:29 smithi084 ceph-mon[32629]: 4.f scrub starts 2024-09-16T09:41:29.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:29 smithi084 ceph-mon[32629]: 4.f scrub ok 2024-09-16T09:41:29.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:29 smithi084 ceph-mon[32629]: 1.a scrub starts 2024-09-16T09:41:29.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:29 smithi084 ceph-mon[32629]: 1.a scrub ok 2024-09-16T09:41:29.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:29 smithi084 ceph-mon[32629]: 1.f scrub starts 2024-09-16T09:41:29.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:29 smithi084 ceph-mon[32629]: 1.f scrub ok 2024-09-16T09:41:29.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:29 smithi084 ceph-mon[32629]: 3.14 scrub starts 2024-09-16T09:41:29.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:29 smithi084 ceph-mon[32629]: 3.14 scrub ok 2024-09-16T09:41:29.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:29 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:41:29.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:29 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:41:29.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:29 smithi070 ceph-mon[30388]: pgmap v869: 168 pgs: 168 active+clean; 828 MiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 1.3 KiB/s rd, 112 MiB/s wr, 1.78k op/s 2024-09-16T09:41:29.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:29 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:41:29.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:29 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:41:29.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:29 smithi070 ceph-mon[30388]: osdmap e462: 8 total, 8 up, 8 in 2024-09-16T09:41:29.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:29 smithi070 ceph-mon[30388]: 4.f scrub starts 2024-09-16T09:41:29.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:29 smithi070 ceph-mon[30388]: 4.f scrub ok 2024-09-16T09:41:29.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:29 smithi070 ceph-mon[30388]: 1.a scrub starts 2024-09-16T09:41:29.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:29 smithi070 ceph-mon[30388]: 1.a scrub ok 2024-09-16T09:41:29.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:29 smithi070 ceph-mon[30388]: 1.f scrub starts 2024-09-16T09:41:29.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:29 smithi070 ceph-mon[30388]: 1.f scrub ok 2024-09-16T09:41:29.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:29 smithi070 ceph-mon[30388]: 3.14 scrub starts 2024-09-16T09:41:29.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:29 smithi070 ceph-mon[30388]: 3.14 scrub ok 2024-09-16T09:41:29.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:29 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:41:29.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:29 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:41:29.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:29 smithi070 ceph-mon[40691]: pgmap v869: 168 pgs: 168 active+clean; 828 MiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 1.3 KiB/s rd, 112 MiB/s wr, 1.78k op/s 2024-09-16T09:41:29.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:29 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:41:29.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:29 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:41:29.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:29 smithi070 ceph-mon[40691]: osdmap e462: 8 total, 8 up, 8 in 2024-09-16T09:41:29.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:29 smithi070 ceph-mon[40691]: 4.f scrub starts 2024-09-16T09:41:29.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:29 smithi070 ceph-mon[40691]: 4.f scrub ok 2024-09-16T09:41:29.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:29 smithi070 ceph-mon[40691]: 1.a scrub starts 2024-09-16T09:41:29.923 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:29 smithi070 ceph-mon[40691]: 1.a scrub ok 2024-09-16T09:41:29.923 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:29 smithi070 ceph-mon[40691]: 1.f scrub starts 2024-09-16T09:41:29.923 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:29 smithi070 ceph-mon[40691]: 1.f scrub ok 2024-09-16T09:41:29.923 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:29 smithi070 ceph-mon[40691]: 3.14 scrub starts 2024-09-16T09:41:29.923 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:29 smithi070 ceph-mon[40691]: 3.14 scrub ok 2024-09-16T09:41:29.923 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:29 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:41:29.923 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:29 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:41:29.970 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:41:30.005 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:30.005 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:30.006 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:41:30.090 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 10 16 26458 26442 165.245 139.375 0.00298386 0.00604585 2024-09-16T09:41:30.106 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:41:30.142 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:30.142 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:30.143 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:41:30.187 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T09:41:30.187 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T09:41:30.188 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-16T09:41:30.188 INFO:tasks.thrashosds.thrasher:Setting osd 6 primary_affinity to 1.000000 2024-09-16T09:41:30.188 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd primary-affinity 6 1 2024-09-16T09:41:30.244 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:41:30.276 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:30.276 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:30.277 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:41:30.378 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:41:30.407 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:30.407 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:30.408 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:41:30.480 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:30 smithi070 ceph-mon[30388]: 5.1e scrub starts 2024-09-16T09:41:30.509 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:41:30.554 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:30.554 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:30.555 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:41:30.655 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:41:30.688 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:30.689 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:30.689 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:41:30.790 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:41:30.824 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:30.825 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:30.825 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:41:30.862 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:30 smithi070 ceph-mon[30388]: 5.1e scrub ok 2024-09-16T09:41:30.862 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:30 smithi070 ceph-mon[30388]: 6.12 scrub starts 2024-09-16T09:41:30.863 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:30 smithi070 ceph-mon[30388]: 6.12 scrub ok 2024-09-16T09:41:30.863 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:30 smithi070 ceph-mon[30388]: osdmap e463: 8 total, 8 up, 8 in 2024-09-16T09:41:30.863 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:30 smithi070 ceph-mon[40691]: 5.1e scrub starts 2024-09-16T09:41:30.863 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:30 smithi070 ceph-mon[40691]: 5.1e scrub ok 2024-09-16T09:41:30.863 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:30 smithi070 ceph-mon[40691]: 6.12 scrub starts 2024-09-16T09:41:30.863 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:30 smithi070 ceph-mon[40691]: 6.12 scrub ok 2024-09-16T09:41:30.863 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:30 smithi070 ceph-mon[40691]: osdmap e463: 8 total, 8 up, 8 in 2024-09-16T09:41:30.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:30 smithi084 ceph-mon[32629]: 5.1e scrub starts 2024-09-16T09:41:30.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:30 smithi084 ceph-mon[32629]: 5.1e scrub ok 2024-09-16T09:41:30.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:30 smithi084 ceph-mon[32629]: 6.12 scrub starts 2024-09-16T09:41:30.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:30 smithi084 ceph-mon[32629]: 6.12 scrub ok 2024-09-16T09:41:30.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:30 smithi084 ceph-mon[32629]: osdmap e463: 8 total, 8 up, 8 in 2024-09-16T09:41:30.927 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:41:30.954 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:30.955 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:30.955 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:41:30.957 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:41:30.998 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:41:31.056 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:41:31.090 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 11 16 28491 28475 161.77 127.062 0.00373202 0.00617606 2024-09-16T09:41:31.091 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:31.092 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:31.092 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:41:31.193 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:41:31.229 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:31.230 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:31.230 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:41:31.331 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:41:31.364 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:31.365 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:31.365 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:41:31.466 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:41:31.501 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:31.502 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:31.502 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:41:31.603 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:41:31.636 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:31.637 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:31.637 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:41:31.738 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:41:31.744 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:31 smithi070 ceph-mon[30388]: pgmap v872: 168 pgs: 168 active+clean; 1.4 GiB data, 9.5 GiB used, 706 GiB / 715 GiB avail; 853 B/s rd, 136 MiB/s wr, 2.17k op/s 2024-09-16T09:41:31.744 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:31 smithi070 ceph-mon[30388]: 6.1a scrub starts 2024-09-16T09:41:31.744 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:31 smithi070 ceph-mon[30388]: 6.1a scrub ok 2024-09-16T09:41:31.744 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:31 smithi070 ceph-mon[30388]: 6.0 scrub starts 2024-09-16T09:41:31.745 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:31 smithi070 ceph-mon[30388]: 6.0 scrub ok 2024-09-16T09:41:31.745 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:31 smithi070 ceph-mon[40691]: pgmap v872: 168 pgs: 168 active+clean; 1.4 GiB data, 9.5 GiB used, 706 GiB / 715 GiB avail; 853 B/s rd, 136 MiB/s wr, 2.17k op/s 2024-09-16T09:41:31.745 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:31 smithi070 ceph-mon[40691]: 6.1a scrub starts 2024-09-16T09:41:31.745 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:31 smithi070 ceph-mon[40691]: 6.1a scrub ok 2024-09-16T09:41:31.745 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:31 smithi070 ceph-mon[40691]: 6.0 scrub starts 2024-09-16T09:41:31.745 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:31 smithi070 ceph-mon[40691]: 6.0 scrub ok 2024-09-16T09:41:31.771 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:31.772 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:31.772 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:41:31.872 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:41:31.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:31 smithi084 ceph-mon[32629]: pgmap v872: 168 pgs: 168 active+clean; 1.4 GiB data, 9.5 GiB used, 706 GiB / 715 GiB avail; 853 B/s rd, 136 MiB/s wr, 2.17k op/s 2024-09-16T09:41:31.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:31 smithi084 ceph-mon[32629]: 6.1a scrub starts 2024-09-16T09:41:31.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:31 smithi084 ceph-mon[32629]: 6.1a scrub ok 2024-09-16T09:41:31.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:31 smithi084 ceph-mon[32629]: 6.0 scrub starts 2024-09-16T09:41:31.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:31 smithi084 ceph-mon[32629]: 6.0 scrub ok 2024-09-16T09:41:31.913 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:31.914 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:31.914 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:41:32.004 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:41:32.004 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:41:32.004 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:41:32.004 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:41:32.005 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:41:31 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:41:31] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:41:32.014 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:41:32.027 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:41:32.027 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:41:32.047 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:32.048 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:32.048 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:41:32.060 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:41:32.061 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:41:32.090 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 12 15 30553 30538 159.033 128.938 0.00618041 0.00628331 2024-09-16T09:41:32.094 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:41:32.094 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:41:32.113 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T09:41:32.113 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:41:32.132 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:41:32.132 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:41:32.149 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:41:32.172 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:41:32.173 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:41:32.189 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:32.189 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:32.190 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:41:32.205 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:41:32.205 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:41:32.291 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:41:32.326 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:32.327 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:32.327 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:41:32.329 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:41:32.428 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:41:32.460 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:32.461 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:32.461 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:41:32.562 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:41:32.594 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:32.595 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:32.595 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:41:32.645 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:32 smithi070 ceph-mon[30388]: 6.5 scrub starts 2024-09-16T09:41:32.645 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:32 smithi070 ceph-mon[30388]: 6.5 scrub ok 2024-09-16T09:41:32.645 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:32 smithi070 ceph-mon[30388]: 1.8 scrub starts 2024-09-16T09:41:32.645 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:32 smithi070 ceph-mon[30388]: 1.8 scrub ok 2024-09-16T09:41:32.645 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:32 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2800228598' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:41:32.645 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:32 smithi070 ceph-mon[30388]: 5.e scrub starts 2024-09-16T09:41:32.645 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:32 smithi070 ceph-mon[30388]: 5.e scrub ok 2024-09-16T09:41:32.646 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:32 smithi070 ceph-mon[40691]: 6.5 scrub starts 2024-09-16T09:41:32.646 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:32 smithi070 ceph-mon[40691]: 6.5 scrub ok 2024-09-16T09:41:32.646 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:32 smithi070 ceph-mon[40691]: 1.8 scrub starts 2024-09-16T09:41:32.646 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:32 smithi070 ceph-mon[40691]: 1.8 scrub ok 2024-09-16T09:41:32.646 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:32 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/2800228598' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:41:32.646 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:32 smithi070 ceph-mon[40691]: 5.e scrub starts 2024-09-16T09:41:32.646 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:32 smithi070 ceph-mon[40691]: 5.e scrub ok 2024-09-16T09:41:32.696 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:41:32.734 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:32.734 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:32.734 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:41:32.835 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:41:32.866 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:32.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:32.867 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:41:32.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:32 smithi084 ceph-mon[32629]: 6.5 scrub starts 2024-09-16T09:41:32.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:32 smithi084 ceph-mon[32629]: 6.5 scrub ok 2024-09-16T09:41:32.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:32 smithi084 ceph-mon[32629]: 1.8 scrub starts 2024-09-16T09:41:32.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:32 smithi084 ceph-mon[32629]: 1.8 scrub ok 2024-09-16T09:41:32.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:32 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/2800228598' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:41:32.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:32 smithi084 ceph-mon[32629]: 5.e scrub starts 2024-09-16T09:41:32.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:32 smithi084 ceph-mon[32629]: 5.e scrub ok 2024-09-16T09:41:32.941 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-16T09:41:32.969 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:41:33.003 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:33.003 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:33.004 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:41:33.091 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 13 16 32750 32734 157.353 137.25 0.0141992 0.0063498 2024-09-16T09:41:33.104 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:41:33.137 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:33.137 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:33.138 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:41:33.239 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:41:33.288 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:33.289 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:33.289 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:41:33.390 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:41:33.430 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:33.431 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:33.431 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:41:33.532 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:41:33.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:33.569 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:33.569 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:41:33.570 INFO:teuthology.orchestra.run.smithi070.stderr:set osd.6 primary-affinity to 1 (8655362) 2024-09-16T09:41:33.670 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:41:33.706 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:33.707 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:33.707 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:41:33.808 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:41:33.840 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:33.841 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:33.841 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:41:33.910 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:41:33 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:41:33.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:33 smithi084 ceph-mon[32629]: pgmap v873: 168 pgs: 168 active+clean; 1.4 GiB data, 9.5 GiB used, 706 GiB / 715 GiB avail; 853 B/s rd, 127 MiB/s wr, 2.03k op/s 2024-09-16T09:41:33.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:33 smithi084 ceph-mon[32629]: 6.2 scrub starts 2024-09-16T09:41:33.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:33 smithi084 ceph-mon[32629]: 6.2 scrub ok 2024-09-16T09:41:33.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:33 smithi084 ceph-mon[32629]: 3.1d scrub starts 2024-09-16T09:41:33.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:33 smithi084 ceph-mon[32629]: 3.1d scrub ok 2024-09-16T09:41:33.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:33 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/2065128885' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 6, "weight": 1.0} : dispatch 2024-09-16T09:41:33.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:33 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 6, "weight": 1.0} : dispatch 2024-09-16T09:41:33.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:33 smithi070 ceph-mon[30388]: pgmap v873: 168 pgs: 168 active+clean; 1.4 GiB data, 9.5 GiB used, 706 GiB / 715 GiB avail; 853 B/s rd, 127 MiB/s wr, 2.03k op/s 2024-09-16T09:41:33.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:33 smithi070 ceph-mon[30388]: 6.2 scrub starts 2024-09-16T09:41:33.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:33 smithi070 ceph-mon[30388]: 6.2 scrub ok 2024-09-16T09:41:33.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:33 smithi070 ceph-mon[30388]: 3.1d scrub starts 2024-09-16T09:41:33.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:33 smithi070 ceph-mon[30388]: 3.1d scrub ok 2024-09-16T09:41:33.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:33 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2065128885' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 6, "weight": 1.0} : dispatch 2024-09-16T09:41:33.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:33 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 6, "weight": 1.0} : dispatch 2024-09-16T09:41:33.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:33 smithi070 ceph-mon[40691]: pgmap v873: 168 pgs: 168 active+clean; 1.4 GiB data, 9.5 GiB used, 706 GiB / 715 GiB avail; 853 B/s rd, 127 MiB/s wr, 2.03k op/s 2024-09-16T09:41:33.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:33 smithi070 ceph-mon[40691]: 6.2 scrub starts 2024-09-16T09:41:33.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:33 smithi070 ceph-mon[40691]: 6.2 scrub ok 2024-09-16T09:41:33.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:33 smithi070 ceph-mon[40691]: 3.1d scrub starts 2024-09-16T09:41:33.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:33 smithi070 ceph-mon[40691]: 3.1d scrub ok 2024-09-16T09:41:33.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:33 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/2065128885' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 6, "weight": 1.0} : dispatch 2024-09-16T09:41:33.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:33 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 6, "weight": 1.0} : dispatch 2024-09-16T09:41:33.942 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:41:33.977 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:33.977 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:33.977 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:41:34.078 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:41:34.090 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 14 16 34615 34599 154.439 116.562 0.00479203 0.00646962 2024-09-16T09:41:34.109 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:34.109 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:34.110 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:41:34.211 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:41:34.250 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:34.251 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:34.251 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:41:34.352 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:41:34.374 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:34.375 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:34.375 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:41:34.476 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:41:34.497 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:34.498 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:34.498 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:41:34.599 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:41:34.634 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:34.635 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:34.635 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:41:34.736 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:41:34.773 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:34.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:34.774 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:41:34.874 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:41:34.909 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:34.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:34 smithi084 ceph-mon[32629]: 5.d scrub starts 2024-09-16T09:41:34.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:34 smithi084 ceph-mon[32629]: 5.d scrub ok 2024-09-16T09:41:34.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:34 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:41:34.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:34 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 6, "weight": 1.0}]': finished 2024-09-16T09:41:34.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:34 smithi084 ceph-mon[32629]: osdmap e464: 8 total, 8 up, 8 in 2024-09-16T09:41:34.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:34 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:41:34.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:34 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:41:34.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:34 smithi084 ceph-mon[32629]: 6.13 scrub starts 2024-09-16T09:41:34.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:34 smithi084 ceph-mon[32629]: 6.13 scrub ok 2024-09-16T09:41:34.911 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:34.911 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:41:34.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:34 smithi070 ceph-mon[30388]: 5.d scrub starts 2024-09-16T09:41:34.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:34 smithi070 ceph-mon[30388]: 5.d scrub ok 2024-09-16T09:41:34.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:34 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:41:34.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:34 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 6, "weight": 1.0}]': finished 2024-09-16T09:41:34.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:34 smithi070 ceph-mon[30388]: osdmap e464: 8 total, 8 up, 8 in 2024-09-16T09:41:34.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:34 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:41:34.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:34 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:41:34.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:34 smithi070 ceph-mon[30388]: 6.13 scrub starts 2024-09-16T09:41:34.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:34 smithi070 ceph-mon[30388]: 6.13 scrub ok 2024-09-16T09:41:34.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:34 smithi070 ceph-mon[40691]: 5.d scrub starts 2024-09-16T09:41:34.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:34 smithi070 ceph-mon[40691]: 5.d scrub ok 2024-09-16T09:41:34.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:34 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:41:34.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:34 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 6, "weight": 1.0}]': finished 2024-09-16T09:41:34.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:34 smithi070 ceph-mon[40691]: osdmap e464: 8 total, 8 up, 8 in 2024-09-16T09:41:34.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:34 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:41:34.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:34 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:41:34.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:34 smithi070 ceph-mon[40691]: 6.13 scrub starts 2024-09-16T09:41:34.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:34 smithi070 ceph-mon[40691]: 6.13 scrub ok 2024-09-16T09:41:35.013 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:41:35.047 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:35.048 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:35.048 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:41:35.090 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 15 16 36345 36329 151.35 108.125 0.00360888 0.00660191 2024-09-16T09:41:35.148 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:41:35.182 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:35.183 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:35.183 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:41:35.284 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:41:35.317 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:35.318 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:35.318 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:41:35.419 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:41:35.454 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:35.455 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:35.455 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:41:35.556 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:41:35.587 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:35.588 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:35.588 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:41:35.688 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:41:35.725 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:35.725 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:35.726 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:41:35.826 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:41:35.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:35.848 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:35.848 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:41:35.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:35 smithi084 ceph-mon[32629]: pgmap v875: 168 pgs: 168 active+clean; 2.0 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 853 B/s rd, 198 MiB/s wr, 3.16k op/s 2024-09-16T09:41:35.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:35 smithi070 ceph-mon[30388]: pgmap v875: 168 pgs: 168 active+clean; 2.0 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 853 B/s rd, 198 MiB/s wr, 3.16k op/s 2024-09-16T09:41:35.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:35 smithi070 ceph-mon[40691]: pgmap v875: 168 pgs: 168 active+clean; 2.0 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 853 B/s rd, 198 MiB/s wr, 3.16k op/s 2024-09-16T09:41:35.949 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:41:35.986 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:35.987 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:35.987 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:41:36.088 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:41:36.091 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 16 15 38044 38029 148.531 106.25 0.00556953 0.00672671 2024-09-16T09:41:36.123 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:36.124 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:36.124 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:41:36.225 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:41:36.259 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:36.259 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:36.260 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:41:36.360 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:41:36.395 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:36.396 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:36.396 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:41:36.497 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:41:36.526 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:36.526 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:36.527 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:41:36.627 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:41:36.668 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:36.669 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:36.669 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:41:36.770 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:41:36.795 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:36.796 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:36.796 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:41:36.897 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:41:36.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:36 smithi084 ceph-mon[32629]: 6.17 scrub starts 2024-09-16T09:41:36.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:36 smithi084 ceph-mon[32629]: 6.17 scrub ok 2024-09-16T09:41:36.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:36 smithi084 ceph-mon[32629]: 5.13 scrub starts 2024-09-16T09:41:36.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:36 smithi084 ceph-mon[32629]: 5.13 scrub ok 2024-09-16T09:41:36.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:36 smithi070 ceph-mon[30388]: 6.17 scrub starts 2024-09-16T09:41:36.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:36 smithi070 ceph-mon[30388]: 6.17 scrub ok 2024-09-16T09:41:36.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:36 smithi070 ceph-mon[30388]: 5.13 scrub starts 2024-09-16T09:41:36.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:36 smithi070 ceph-mon[30388]: 5.13 scrub ok 2024-09-16T09:41:36.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:36 smithi070 ceph-mon[40691]: 6.17 scrub starts 2024-09-16T09:41:36.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:36 smithi070 ceph-mon[40691]: 6.17 scrub ok 2024-09-16T09:41:36.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:36 smithi070 ceph-mon[40691]: 5.13 scrub starts 2024-09-16T09:41:36.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:36 smithi070 ceph-mon[40691]: 5.13 scrub ok 2024-09-16T09:41:36.930 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:36.930 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:36.931 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:41:37.032 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:41:37.066 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:37.067 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:37.067 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:41:37.091 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 17 16 39971 39955 146.874 120.375 0.00409944 0.00680333 2024-09-16T09:41:37.168 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:41:37.190 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:37.190 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:37.191 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:41:37.292 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:41:37.328 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:37.328 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:37.329 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:41:37.430 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:41:37.463 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:37.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:37.464 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:41:37.565 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:41:37.603 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:37.604 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:37.604 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:41:37.705 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:41:37.736 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:37.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:37.738 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:41:37.839 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:41:37.869 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:37.869 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:37.870 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:41:37.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:37 smithi084 ceph-mon[32629]: pgmap v876: 168 pgs: 168 active+clean; 2.0 GiB data, 12 GiB used, 704 GiB / 715 GiB avail; 695 B/s rd, 115 MiB/s wr, 1.85k op/s 2024-09-16T09:41:37.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:37 smithi084 ceph-mon[32629]: 3.11 scrub starts 2024-09-16T09:41:37.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:37 smithi084 ceph-mon[32629]: 3.11 scrub ok 2024-09-16T09:41:37.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:37 smithi084 ceph-mon[32629]: 3.1 scrub starts 2024-09-16T09:41:37.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:37 smithi084 ceph-mon[32629]: 3.1 scrub ok 2024-09-16T09:41:37.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:37 smithi070 ceph-mon[30388]: pgmap v876: 168 pgs: 168 active+clean; 2.0 GiB data, 12 GiB used, 704 GiB / 715 GiB avail; 695 B/s rd, 115 MiB/s wr, 1.85k op/s 2024-09-16T09:41:37.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:37 smithi070 ceph-mon[30388]: 3.11 scrub starts 2024-09-16T09:41:37.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:37 smithi070 ceph-mon[30388]: 3.11 scrub ok 2024-09-16T09:41:37.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:37 smithi070 ceph-mon[30388]: 3.1 scrub starts 2024-09-16T09:41:37.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:37 smithi070 ceph-mon[30388]: 3.1 scrub ok 2024-09-16T09:41:37.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:37 smithi070 ceph-mon[40691]: pgmap v876: 168 pgs: 168 active+clean; 2.0 GiB data, 12 GiB used, 704 GiB / 715 GiB avail; 695 B/s rd, 115 MiB/s wr, 1.85k op/s 2024-09-16T09:41:37.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:37 smithi070 ceph-mon[40691]: 3.11 scrub starts 2024-09-16T09:41:37.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:37 smithi070 ceph-mon[40691]: 3.11 scrub ok 2024-09-16T09:41:37.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:37 smithi070 ceph-mon[40691]: 3.1 scrub starts 2024-09-16T09:41:37.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:37 smithi070 ceph-mon[40691]: 3.1 scrub ok 2024-09-16T09:41:37.970 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:41:38.004 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:38.005 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:38.005 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:41:38.091 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 18 16 41752 41736 144.898 111.312 0.00535138 0.00689682 2024-09-16T09:41:38.106 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:41:38.145 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:38.146 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:38.146 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:41:38.247 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:41:38.282 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:38.283 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:38.283 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:41:38.385 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:41:38.419 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:38.420 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:38.420 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:41:38.521 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:41:38.553 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:38.554 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:38.555 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:41:38.612 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.1 dump_ops_in_flight 2024-09-16T09:41:38.655 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:41:38.686 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:38.686 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:38.687 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:41:38.787 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:41:38.835 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:38.836 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:38.836 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:41:38.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:38 smithi084 ceph-mon[32629]: 3.19 scrub starts 2024-09-16T09:41:38.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:38 smithi084 ceph-mon[32629]: 3.19 scrub ok 2024-09-16T09:41:38.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:38 smithi084 ceph-mon[32629]: 5.12 scrub starts 2024-09-16T09:41:38.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:38 smithi084 ceph-mon[32629]: 5.12 scrub ok 2024-09-16T09:41:38.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:38 smithi084 ceph-mon[32629]: pgmap v877: 168 pgs: 168 active+clean; 2.1 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1.2 KiB/s rd, 103 MiB/s wr, 1.65k op/s 2024-09-16T09:41:38.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:38 smithi070 ceph-mon[40691]: 3.19 scrub starts 2024-09-16T09:41:38.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:38 smithi070 ceph-mon[40691]: 3.19 scrub ok 2024-09-16T09:41:38.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:38 smithi070 ceph-mon[40691]: 5.12 scrub starts 2024-09-16T09:41:38.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:38 smithi070 ceph-mon[40691]: 5.12 scrub ok 2024-09-16T09:41:38.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:38 smithi070 ceph-mon[40691]: pgmap v877: 168 pgs: 168 active+clean; 2.1 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1.2 KiB/s rd, 103 MiB/s wr, 1.65k op/s 2024-09-16T09:41:38.923 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:38 smithi070 ceph-mon[30388]: 3.19 scrub starts 2024-09-16T09:41:38.923 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:38 smithi070 ceph-mon[30388]: 3.19 scrub ok 2024-09-16T09:41:38.923 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:38 smithi070 ceph-mon[30388]: 5.12 scrub starts 2024-09-16T09:41:38.924 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:38 smithi070 ceph-mon[30388]: 5.12 scrub ok 2024-09-16T09:41:38.924 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:38 smithi070 ceph-mon[30388]: pgmap v877: 168 pgs: 168 active+clean; 2.1 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1.2 KiB/s rd, 103 MiB/s wr, 1.65k op/s 2024-09-16T09:41:38.937 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:41:38.972 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:38.972 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:38.973 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:41:39.074 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:41:39.091 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 19 15 43548 43533 143.183 112.312 0.00571626 0.00697858 2024-09-16T09:41:39.101 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:39.102 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:39.102 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:41:39.203 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:41:39.241 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:39.242 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:39.242 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:41:39.343 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:41:39.374 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:39.375 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:39.375 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:41:39.476 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:41:39.509 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:39.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:39.510 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:41:39.611 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:41:39.646 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:39.647 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:39.647 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:41:39.748 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:41:39.780 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:39.781 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:39.781 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:41:39.882 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:41:39.883 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:41:39.885 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:41:39.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:39 smithi084 ceph-mon[32629]: 1.14 scrub starts 2024-09-16T09:41:39.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:39 smithi084 ceph-mon[32629]: 1.14 scrub ok 2024-09-16T09:41:39.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:39 smithi084 ceph-mon[32629]: 6.7 scrub starts 2024-09-16T09:41:39.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:39 smithi084 ceph-mon[32629]: 6.7 scrub ok 2024-09-16T09:41:39.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:39 smithi084 ceph-mon[32629]: 6.e scrub starts 2024-09-16T09:41:39.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:39 smithi084 ceph-mon[32629]: 6.e scrub ok 2024-09-16T09:41:39.915 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:39.916 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:39.916 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:41:39.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:39 smithi070 ceph-mon[30388]: 1.14 scrub starts 2024-09-16T09:41:39.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:39 smithi070 ceph-mon[30388]: 1.14 scrub ok 2024-09-16T09:41:39.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:39 smithi070 ceph-mon[30388]: 6.7 scrub starts 2024-09-16T09:41:39.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:39 smithi070 ceph-mon[30388]: 6.7 scrub ok 2024-09-16T09:41:39.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:39 smithi070 ceph-mon[30388]: 6.e scrub starts 2024-09-16T09:41:39.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:39 smithi070 ceph-mon[30388]: 6.e scrub ok 2024-09-16T09:41:39.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:39 smithi070 ceph-mon[40691]: 1.14 scrub starts 2024-09-16T09:41:39.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:39 smithi070 ceph-mon[40691]: 1.14 scrub ok 2024-09-16T09:41:39.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:39 smithi070 ceph-mon[40691]: 6.7 scrub starts 2024-09-16T09:41:39.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:39 smithi070 ceph-mon[40691]: 6.7 scrub ok 2024-09-16T09:41:39.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:39 smithi070 ceph-mon[40691]: 6.e scrub starts 2024-09-16T09:41:39.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:39 smithi070 ceph-mon[40691]: 6.e scrub ok 2024-09-16T09:41:40.016 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:41:40.050 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:40.050 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:40.050 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:41:40.091 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:2024-09-16T09:41:40.090778+0000 min lat: 0.00125032 max lat: 0.183392 avg lat: 0.00705203 2024-09-16T09:41:40.091 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-16T09:41:40.091 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 20 15 45368 45353 141.711 113.75 0.0048146 0.00705203 2024-09-16T09:41:40.151 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:41:40.182 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:40.183 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:40.183 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:41:40.284 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:41:40.315 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:40.316 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:40.316 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:41:40.417 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:41:40.448 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:40.449 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:40.450 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:41:40.551 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:41:40.597 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:40.598 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:40.598 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:41:40.699 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:41:40.730 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:40.731 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:40.731 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:41:40.832 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:41:40.866 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:40 smithi070 ceph-mon[30388]: pgmap v878: 168 pgs: 168 active+clean; 2.6 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1023 B/s rd, 123 MiB/s wr, 1.96k op/s 2024-09-16T09:41:40.867 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:40 smithi070 ceph-mon[30388]: 3.1f scrub starts 2024-09-16T09:41:40.867 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:40 smithi070 ceph-mon[30388]: 3.1f scrub ok 2024-09-16T09:41:40.867 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:40 smithi070 ceph-mon[30388]: 3.3 scrub starts 2024-09-16T09:41:40.867 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:40 smithi070 ceph-mon[30388]: 3.3 scrub ok 2024-09-16T09:41:40.867 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:40 smithi070 ceph-mon[40691]: pgmap v878: 168 pgs: 168 active+clean; 2.6 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1023 B/s rd, 123 MiB/s wr, 1.96k op/s 2024-09-16T09:41:40.867 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:40 smithi070 ceph-mon[40691]: 3.1f scrub starts 2024-09-16T09:41:40.867 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:40 smithi070 ceph-mon[40691]: 3.1f scrub ok 2024-09-16T09:41:40.867 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:40 smithi070 ceph-mon[40691]: 3.3 scrub starts 2024-09-16T09:41:40.867 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:40 smithi070 ceph-mon[40691]: 3.3 scrub ok 2024-09-16T09:41:40.871 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:40.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:40.872 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:41:40.882 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T09:41:40.883 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:41:40.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:40 smithi084 ceph-mon[32629]: pgmap v878: 168 pgs: 168 active+clean; 2.6 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1023 B/s rd, 123 MiB/s wr, 1.96k op/s 2024-09-16T09:41:40.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:40 smithi084 ceph-mon[32629]: 3.1f scrub starts 2024-09-16T09:41:40.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:40 smithi084 ceph-mon[32629]: 3.1f scrub ok 2024-09-16T09:41:40.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:40 smithi084 ceph-mon[32629]: 3.3 scrub starts 2024-09-16T09:41:40.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:40 smithi084 ceph-mon[32629]: 3.3 scrub ok 2024-09-16T09:41:40.973 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:41:41.002 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:41.002 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:41.003 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:41:41.091 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 21 16 47009 46993 139.843 102.5 0.00763961 0.00714617 2024-09-16T09:41:41.103 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:41:41.140 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:41.140 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:41.141 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:41:41.242 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:41:41.274 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:41.274 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:41.275 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:41:41.376 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:41:41.408 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:41.409 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:41.409 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:41:41.510 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:41:41.543 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:41.544 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:41.544 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:41:41.645 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:41:41.665 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:41.666 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:41.666 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:41:41.767 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:41:41.802 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:41.803 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:41.803 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:41:41.856 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T09:41:41.893 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:41 smithi070 ceph-mon[30388]: 1.9 scrub starts 2024-09-16T09:41:41.893 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:41 smithi070 ceph-mon[30388]: 1.9 scrub ok 2024-09-16T09:41:41.893 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:41 smithi070 ceph-mon[30388]: 4.15 scrub starts 2024-09-16T09:41:41.893 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:41 smithi070 ceph-mon[30388]: 4.15 scrub ok 2024-09-16T09:41:41.893 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:41 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3673886056' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:41:41.894 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:41 smithi070 ceph-mon[40691]: 1.9 scrub starts 2024-09-16T09:41:41.894 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:41 smithi070 ceph-mon[40691]: 1.9 scrub ok 2024-09-16T09:41:41.894 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:41 smithi070 ceph-mon[40691]: 4.15 scrub starts 2024-09-16T09:41:41.894 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:41 smithi070 ceph-mon[40691]: 4.15 scrub ok 2024-09-16T09:41:41.894 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:41 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3673886056' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:41:41.905 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:41:41.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:41 smithi084 ceph-mon[32629]: 1.9 scrub starts 2024-09-16T09:41:41.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:41 smithi084 ceph-mon[32629]: 1.9 scrub ok 2024-09-16T09:41:41.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:41 smithi084 ceph-mon[32629]: 4.15 scrub starts 2024-09-16T09:41:41.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:41 smithi084 ceph-mon[32629]: 4.15 scrub ok 2024-09-16T09:41:41.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:41 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3673886056' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:41:41.934 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:41.935 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:41.935 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:41:42.036 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:41:42.072 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:42.074 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:42.074 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:41:42.091 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 22 16 48546 48530 137.853 96.0625 0.00620232 0.00724854 2024-09-16T09:41:42.173 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:41:41 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:41:41] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:41:42.176 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:41:42.204 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:42.205 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:42.205 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:41:42.306 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:41:42.330 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:42.331 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:42.331 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:41:42.432 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:41:42.464 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:42.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:42.465 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:41:42.566 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:41:42.601 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:42.601 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:42.602 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:41:42.703 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:41:42.741 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:42.742 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:42.742 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:41:42.843 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:41:42.875 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:42.875 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:42.876 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:41:42.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:42 smithi084 ceph-mon[32629]: 4.0 scrub starts 2024-09-16T09:41:42.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:42 smithi084 ceph-mon[32629]: 4.0 scrub ok 2024-09-16T09:41:42.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:42 smithi084 ceph-mon[32629]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-16T09:41:42.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:42 smithi084 ceph-mon[32629]: Cluster is now healthy 2024-09-16T09:41:42.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:42 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3673886056' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-16T09:41:42.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:42 smithi084 ceph-mon[32629]: osdmap e465: 8 total, 8 up, 8 in 2024-09-16T09:41:42.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:42 smithi084 ceph-mon[32629]: pgmap v880: 168 pgs: 168 active+clean; 2.6 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 618 B/s rd, 88 MiB/s wr, 1.40k op/s 2024-09-16T09:41:42.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:42 smithi084 ceph-mon[32629]: 2.1 scrub starts 2024-09-16T09:41:42.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:42 smithi084 ceph-mon[32629]: 2.1 scrub ok 2024-09-16T09:41:42.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:42 smithi084 ceph-mon[32629]: 4.c scrub starts 2024-09-16T09:41:42.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:42 smithi084 ceph-mon[32629]: 4.c scrub ok 2024-09-16T09:41:42.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:42 smithi070 ceph-mon[30388]: 4.0 scrub starts 2024-09-16T09:41:42.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:42 smithi070 ceph-mon[30388]: 4.0 scrub ok 2024-09-16T09:41:42.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:42 smithi070 ceph-mon[30388]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-16T09:41:42.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:42 smithi070 ceph-mon[30388]: Cluster is now healthy 2024-09-16T09:41:42.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:42 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3673886056' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-16T09:41:42.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:42 smithi070 ceph-mon[30388]: osdmap e465: 8 total, 8 up, 8 in 2024-09-16T09:41:42.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:42 smithi070 ceph-mon[30388]: pgmap v880: 168 pgs: 168 active+clean; 2.6 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 618 B/s rd, 88 MiB/s wr, 1.40k op/s 2024-09-16T09:41:42.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:42 smithi070 ceph-mon[30388]: 2.1 scrub starts 2024-09-16T09:41:42.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:42 smithi070 ceph-mon[30388]: 2.1 scrub ok 2024-09-16T09:41:42.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:42 smithi070 ceph-mon[30388]: 4.c scrub starts 2024-09-16T09:41:42.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:42 smithi070 ceph-mon[30388]: 4.c scrub ok 2024-09-16T09:41:42.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:42 smithi070 ceph-mon[40691]: 4.0 scrub starts 2024-09-16T09:41:42.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:42 smithi070 ceph-mon[40691]: 4.0 scrub ok 2024-09-16T09:41:42.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:42 smithi070 ceph-mon[40691]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-16T09:41:42.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:42 smithi070 ceph-mon[40691]: Cluster is now healthy 2024-09-16T09:41:42.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:42 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3673886056' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-16T09:41:42.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:42 smithi070 ceph-mon[40691]: osdmap e465: 8 total, 8 up, 8 in 2024-09-16T09:41:42.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:42 smithi070 ceph-mon[40691]: pgmap v880: 168 pgs: 168 active+clean; 2.6 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 618 B/s rd, 88 MiB/s wr, 1.40k op/s 2024-09-16T09:41:42.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:42 smithi070 ceph-mon[40691]: 2.1 scrub starts 2024-09-16T09:41:42.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:42 smithi070 ceph-mon[40691]: 2.1 scrub ok 2024-09-16T09:41:42.923 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:42 smithi070 ceph-mon[40691]: 4.c scrub starts 2024-09-16T09:41:42.923 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:42 smithi070 ceph-mon[40691]: 4.c scrub ok 2024-09-16T09:41:42.976 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:41:43.021 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:43.022 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:43.022 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:41:43.041 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.1 dump_blocked_ops 2024-09-16T09:41:43.091 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 23 16 50226 50210 136.424 105 0.0146246 0.00732497 2024-09-16T09:41:43.094 INFO:teuthology.orchestra.run.smithi070.stderr:nodeep-scrub is unset 2024-09-16T09:41:43.123 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:41:43.153 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:43.154 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:43.154 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:41:43.255 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:41:43.286 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:43.287 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:43.287 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:41:43.388 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:41:43.418 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:43.418 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:43.419 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:41:43.520 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:41:43.561 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:43.562 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:43.562 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:41:43.663 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:41:43.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:43 smithi070 ceph-mon[30388]: 5.15 scrub starts 2024-09-16T09:41:43.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:43 smithi070 ceph-mon[30388]: 5.15 scrub ok 2024-09-16T09:41:43.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:43 smithi070 ceph-mon[30388]: 4.18 scrub starts 2024-09-16T09:41:43.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:43 smithi070 ceph-mon[30388]: 4.18 scrub ok 2024-09-16T09:41:43.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:43 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:41:43.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:43 smithi070 ceph-mon[40691]: 5.15 scrub starts 2024-09-16T09:41:43.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:43 smithi070 ceph-mon[40691]: 5.15 scrub ok 2024-09-16T09:41:43.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:43 smithi070 ceph-mon[40691]: 4.18 scrub starts 2024-09-16T09:41:43.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:43 smithi070 ceph-mon[40691]: 4.18 scrub ok 2024-09-16T09:41:43.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:43 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:41:43.702 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:43.702 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:43.703 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:41:43.803 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:41:43.847 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:43.847 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:43.848 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:41:43.879 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:41:43.891 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:41:43.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:43 smithi084 ceph-mon[32629]: 5.15 scrub starts 2024-09-16T09:41:43.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:43 smithi084 ceph-mon[32629]: 5.15 scrub ok 2024-09-16T09:41:43.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:43 smithi084 ceph-mon[32629]: 4.18 scrub starts 2024-09-16T09:41:43.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:43 smithi084 ceph-mon[32629]: 4.18 scrub ok 2024-09-16T09:41:43.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:43 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:41:43.911 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:41:43 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:41:43.949 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:41:43.981 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:43.982 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:43.982 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:41:44.083 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:41:44.091 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 24 16 51865 51849 135.007 102.438 0.00779558 0.00740264 2024-09-16T09:41:44.118 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:44.118 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:44.119 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:41:44.186 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T09:41:44.186 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T09:41:44.186 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-16T09:41:44.187 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd dump -f json-pretty 2024-09-16T09:41:44.219 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:41:44.285 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:44.286 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:44.286 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:41:44.388 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:41:44.419 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:44.419 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:44.420 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:41:44.521 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:41:44.543 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:44.544 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:44.545 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:41:44.645 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:41:44.679 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:44.680 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:44.680 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:41:44.781 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:41:44.817 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:44.817 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:44.817 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:41:44.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:44 smithi084 ceph-mon[32629]: 2.0 scrub starts 2024-09-16T09:41:44.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:44 smithi084 ceph-mon[32629]: 2.0 scrub ok 2024-09-16T09:41:44.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:44 smithi084 ceph-mon[32629]: 2.2 scrub starts 2024-09-16T09:41:44.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:44 smithi084 ceph-mon[32629]: 2.2 scrub ok 2024-09-16T09:41:44.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:44 smithi084 ceph-mon[32629]: pgmap v881: 168 pgs: 168 active+clean; 3.0 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 1023 B/s rd, 110 MiB/s wr, 1.76k op/s 2024-09-16T09:41:44.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:44 smithi084 ceph-mon[32629]: osdmap e466: 8 total, 8 up, 8 in 2024-09-16T09:41:44.918 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:41:44.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:44 smithi070 ceph-mon[30388]: 2.0 scrub starts 2024-09-16T09:41:44.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:44 smithi070 ceph-mon[30388]: 2.0 scrub ok 2024-09-16T09:41:44.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:44 smithi070 ceph-mon[30388]: 2.2 scrub starts 2024-09-16T09:41:44.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:44 smithi070 ceph-mon[30388]: 2.2 scrub ok 2024-09-16T09:41:44.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:44 smithi070 ceph-mon[30388]: pgmap v881: 168 pgs: 168 active+clean; 3.0 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 1023 B/s rd, 110 MiB/s wr, 1.76k op/s 2024-09-16T09:41:44.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:44 smithi070 ceph-mon[30388]: osdmap e466: 8 total, 8 up, 8 in 2024-09-16T09:41:44.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:44 smithi070 ceph-mon[40691]: 2.0 scrub starts 2024-09-16T09:41:44.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:44 smithi070 ceph-mon[40691]: 2.0 scrub ok 2024-09-16T09:41:44.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:44 smithi070 ceph-mon[40691]: 2.2 scrub starts 2024-09-16T09:41:44.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:44 smithi070 ceph-mon[40691]: 2.2 scrub ok 2024-09-16T09:41:44.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:44 smithi070 ceph-mon[40691]: pgmap v881: 168 pgs: 168 active+clean; 3.0 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 1023 B/s rd, 110 MiB/s wr, 1.76k op/s 2024-09-16T09:41:44.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:44 smithi070 ceph-mon[40691]: osdmap e466: 8 total, 8 up, 8 in 2024-09-16T09:41:44.954 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:44.955 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:44.955 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:41:45.056 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:41:45.092 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 25 16 53355 53339 133.328 93.125 0.00469793 0.00749315 2024-09-16T09:41:45.093 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:45.094 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:45.094 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:41:45.195 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:41:45.225 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:45.225 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:45.226 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:41:45.326 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:41:45.353 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:45.354 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:45.354 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:41:45.455 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:41:45.468 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:41:45.468 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:41:45.468 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:41:45.468 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:41:45.468 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:41:45.468 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:41:45.468 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:41:45.468 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:41:45.470 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:41:45.470 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:41:45.493 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:45.493 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:45.494 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:41:45.501 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:41:45.501 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:41:45.540 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:41:45.540 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:41:45.576 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:41:45.576 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:41:45.594 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:41:45.625 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:45.625 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:45.626 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:41:45.726 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:41:45.757 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:45.757 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:45.757 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:41:45.815 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd set noscrub 2024-09-16T09:41:45.816 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:45 smithi070 ceph-mon[30388]: 3.15 scrub starts 2024-09-16T09:41:45.816 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:45 smithi070 ceph-mon[30388]: 3.15 scrub ok 2024-09-16T09:41:45.816 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:45 smithi070 ceph-mon[30388]: 1.4 scrub starts 2024-09-16T09:41:45.816 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:45 smithi070 ceph-mon[30388]: 1.4 scrub ok 2024-09-16T09:41:45.816 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:45 smithi070 ceph-mon[30388]: 3.17 scrub starts 2024-09-16T09:41:45.817 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:45 smithi070 ceph-mon[30388]: 3.17 scrub ok 2024-09-16T09:41:45.817 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:45 smithi070 ceph-mon[30388]: 5.1c scrub starts 2024-09-16T09:41:45.817 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:45 smithi070 ceph-mon[30388]: 5.1c scrub ok 2024-09-16T09:41:45.817 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:45 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/468932899' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:41:45.817 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:45 smithi070 ceph-mon[40691]: 3.15 scrub starts 2024-09-16T09:41:45.817 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:45 smithi070 ceph-mon[40691]: 3.15 scrub ok 2024-09-16T09:41:45.818 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:45 smithi070 ceph-mon[40691]: 1.4 scrub starts 2024-09-16T09:41:45.818 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:45 smithi070 ceph-mon[40691]: 1.4 scrub ok 2024-09-16T09:41:45.818 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:45 smithi070 ceph-mon[40691]: 3.17 scrub starts 2024-09-16T09:41:45.818 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:45 smithi070 ceph-mon[40691]: 3.17 scrub ok 2024-09-16T09:41:45.818 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:45 smithi070 ceph-mon[40691]: 5.1c scrub starts 2024-09-16T09:41:45.818 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:45 smithi070 ceph-mon[40691]: 5.1c scrub ok 2024-09-16T09:41:45.818 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:45 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/468932899' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:41:45.859 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:41:45.906 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:45.907 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:45.907 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:41:45.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:45 smithi084 ceph-mon[32629]: 3.15 scrub starts 2024-09-16T09:41:45.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:45 smithi084 ceph-mon[32629]: 3.15 scrub ok 2024-09-16T09:41:45.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:45 smithi084 ceph-mon[32629]: 1.4 scrub starts 2024-09-16T09:41:45.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:45 smithi084 ceph-mon[32629]: 1.4 scrub ok 2024-09-16T09:41:45.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:45 smithi084 ceph-mon[32629]: 3.17 scrub starts 2024-09-16T09:41:45.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:45 smithi084 ceph-mon[32629]: 3.17 scrub ok 2024-09-16T09:41:45.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:45 smithi084 ceph-mon[32629]: 5.1c scrub starts 2024-09-16T09:41:45.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:45 smithi084 ceph-mon[32629]: 5.1c scrub ok 2024-09-16T09:41:45.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:45 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/468932899' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:41:45.988 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T09:41:45.988 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T09:41:45.989 INFO:teuthology.orchestra.run.smithi070.stdout: "complaint_time": 30, 2024-09-16T09:41:45.989 INFO:teuthology.orchestra.run.smithi070.stdout: "num_blocked_ops": 0 2024-09-16T09:41:45.989 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T09:41:46.008 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:41:46.041 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:46.042 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:46.042 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:41:46.092 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 26 16 54836 54820 131.76 92.5625 0.00959935 0.00758468 2024-09-16T09:41:46.143 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:41:46.182 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:46.183 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:46.183 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:41:46.284 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:41:46.317 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:46.317 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:46.318 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:41:46.418 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:41:46.453 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:46.454 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:46.454 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:41:46.555 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:41:46.586 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:46.586 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:46.587 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:41:46.687 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:41:46.720 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:46.721 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:46.721 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:41:46.822 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:41:46.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:46.860 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:46.861 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:41:46.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:46 smithi084 ceph-mon[32629]: 3.7 scrub starts 2024-09-16T09:41:46.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:46 smithi084 ceph-mon[32629]: 3.7 scrub ok 2024-09-16T09:41:46.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:46 smithi084 ceph-mon[32629]: 5.6 scrub starts 2024-09-16T09:41:46.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:46 smithi084 ceph-mon[32629]: 5.6 scrub ok 2024-09-16T09:41:46.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:46 smithi084 ceph-mon[32629]: pgmap v883: 168 pgs: 168 active+clean; 3.1 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 639 B/s rd, 118 MiB/s wr, 1.89k op/s 2024-09-16T09:41:46.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:46 smithi084 ceph-mon[32629]: 3.0 scrub starts 2024-09-16T09:41:46.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:46 smithi084 ceph-mon[32629]: 3.0 scrub ok 2024-09-16T09:41:46.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:46 smithi070 ceph-mon[30388]: 3.7 scrub starts 2024-09-16T09:41:46.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:46 smithi070 ceph-mon[30388]: 3.7 scrub ok 2024-09-16T09:41:46.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:46 smithi070 ceph-mon[30388]: 5.6 scrub starts 2024-09-16T09:41:46.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:46 smithi070 ceph-mon[30388]: 5.6 scrub ok 2024-09-16T09:41:46.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:46 smithi070 ceph-mon[30388]: pgmap v883: 168 pgs: 168 active+clean; 3.1 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 639 B/s rd, 118 MiB/s wr, 1.89k op/s 2024-09-16T09:41:46.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:46 smithi070 ceph-mon[30388]: 3.0 scrub starts 2024-09-16T09:41:46.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:46 smithi070 ceph-mon[30388]: 3.0 scrub ok 2024-09-16T09:41:46.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:46 smithi070 ceph-mon[40691]: 3.7 scrub starts 2024-09-16T09:41:46.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:46 smithi070 ceph-mon[40691]: 3.7 scrub ok 2024-09-16T09:41:46.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:46 smithi070 ceph-mon[40691]: 5.6 scrub starts 2024-09-16T09:41:46.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:46 smithi070 ceph-mon[40691]: 5.6 scrub ok 2024-09-16T09:41:46.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:46 smithi070 ceph-mon[40691]: pgmap v883: 168 pgs: 168 active+clean; 3.1 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 639 B/s rd, 118 MiB/s wr, 1.89k op/s 2024-09-16T09:41:46.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:46 smithi070 ceph-mon[40691]: 3.0 scrub starts 2024-09-16T09:41:46.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:46 smithi070 ceph-mon[40691]: 3.0 scrub ok 2024-09-16T09:41:46.961 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:41:46.994 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:46.995 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:46.995 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:41:47.092 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 27 16 56108 56092 129.824 79.5 0.00501459 0.00769805 2024-09-16T09:41:47.097 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:41:47.134 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:47.135 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:47.135 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:41:47.236 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:41:47.269 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:47.270 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:47.270 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:41:47.371 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:41:47.404 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:47.405 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:47.405 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:41:47.506 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:41:47.536 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:47.537 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:47.537 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:41:47.624 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.1 dump_historic_ops 2024-09-16T09:41:47.638 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:41:47.665 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:47.666 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:47.666 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:41:47.767 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:41:47.799 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:47.800 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:47.800 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:41:47.809 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:47 smithi070 ceph-mon[30388]: 5.2 scrub starts 2024-09-16T09:41:47.809 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:47 smithi070 ceph-mon[30388]: 5.2 scrub ok 2024-09-16T09:41:47.809 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:47 smithi070 ceph-mon[30388]: 4.1a scrub starts 2024-09-16T09:41:47.809 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:47 smithi070 ceph-mon[30388]: 4.1a scrub ok 2024-09-16T09:41:47.810 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:47 smithi070 ceph-mon[40691]: 5.2 scrub starts 2024-09-16T09:41:47.810 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:47 smithi070 ceph-mon[40691]: 5.2 scrub ok 2024-09-16T09:41:47.810 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:47 smithi070 ceph-mon[40691]: 4.1a scrub starts 2024-09-16T09:41:47.810 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:47 smithi070 ceph-mon[40691]: 4.1a scrub ok 2024-09-16T09:41:47.900 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:41:47.929 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:47.930 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:47.930 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:41:48.031 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:41:48.061 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:48.061 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:48.062 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:41:48.092 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 28 16 57623 57607 128.569 94.6875 0.00951954 0.00777282 2024-09-16T09:41:48.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:47 smithi084 ceph-mon[32629]: 5.2 scrub starts 2024-09-16T09:41:48.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:47 smithi084 ceph-mon[32629]: 5.2 scrub ok 2024-09-16T09:41:48.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:47 smithi084 ceph-mon[32629]: 4.1a scrub starts 2024-09-16T09:41:48.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:47 smithi084 ceph-mon[32629]: 4.1a scrub ok 2024-09-16T09:41:48.163 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:41:48.172 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:41:48.178 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:41:48.221 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:48.221 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:48.222 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:41:48.322 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:41:48.355 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:48.356 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:48.356 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:41:48.457 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:41:48.487 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:48.488 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:48.488 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:41:48.589 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:41:48.631 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:48.632 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:48.632 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:41:48.733 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:41:48.739 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:48 smithi070 ceph-mon[30388]: 3.16 scrub starts 2024-09-16T09:41:48.739 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:48 smithi070 ceph-mon[30388]: 3.16 scrub ok 2024-09-16T09:41:48.739 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:48 smithi070 ceph-mon[30388]: pgmap v884: 168 pgs: 168 active+clean; 3.2 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1.2 KiB/s rd, 81 MiB/s wr, 1.30k op/s 2024-09-16T09:41:48.739 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:48 smithi070 ceph-mon[30388]: 6.18 scrub starts 2024-09-16T09:41:48.740 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:48 smithi070 ceph-mon[30388]: 6.18 scrub ok 2024-09-16T09:41:48.740 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:48 smithi070 ceph-mon[40691]: 3.16 scrub starts 2024-09-16T09:41:48.740 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:48 smithi070 ceph-mon[40691]: 3.16 scrub ok 2024-09-16T09:41:48.740 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:48 smithi070 ceph-mon[40691]: pgmap v884: 168 pgs: 168 active+clean; 3.2 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1.2 KiB/s rd, 81 MiB/s wr, 1.30k op/s 2024-09-16T09:41:48.740 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:48 smithi070 ceph-mon[40691]: 6.18 scrub starts 2024-09-16T09:41:48.740 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:48 smithi070 ceph-mon[40691]: 6.18 scrub ok 2024-09-16T09:41:48.765 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:48.766 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:48.766 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:41:48.867 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:41:48.920 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:48.920 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:48.920 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:41:48.954 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:41:49.022 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:41:49.063 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:49.064 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:49.064 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:41:49.093 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 29 16 59057 59041 127.223 89.625 0.013931 0.00785514 2024-09-16T09:41:49.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:48 smithi084 ceph-mon[32629]: 3.16 scrub starts 2024-09-16T09:41:49.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:48 smithi084 ceph-mon[32629]: 3.16 scrub ok 2024-09-16T09:41:49.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:48 smithi084 ceph-mon[32629]: pgmap v884: 168 pgs: 168 active+clean; 3.2 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1.2 KiB/s rd, 81 MiB/s wr, 1.30k op/s 2024-09-16T09:41:49.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:48 smithi084 ceph-mon[32629]: 6.18 scrub starts 2024-09-16T09:41:49.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:48 smithi084 ceph-mon[32629]: 6.18 scrub ok 2024-09-16T09:41:49.165 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:41:49.188 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:49.189 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:49.189 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:41:49.290 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:41:49.322 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:49.322 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:49.323 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:41:49.423 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:41:49.449 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:49.450 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:49.450 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:41:49.551 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:41:49.586 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:49.587 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:49.587 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:41:49.597 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:41:49.598 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T09:41:49.598 INFO:teuthology.orchestra.run.smithi070.stdout: "epoch": 466, 2024-09-16T09:41:49.598 INFO:teuthology.orchestra.run.smithi070.stdout: "fsid": "a1321834-740c-11ef-bceb-c7b262605968", 2024-09-16T09:41:49.598 INFO:teuthology.orchestra.run.smithi070.stdout: "created": "2024-09-16T09:19:14.121498+0000", 2024-09-16T09:41:49.598 INFO:teuthology.orchestra.run.smithi070.stdout: "modified": "2024-09-16T09:41:44.410061+0000", 2024-09-16T09:41:49.598 INFO:teuthology.orchestra.run.smithi070.stdout: "last_up_change": "2024-09-16T09:24:51.124496+0000", 2024-09-16T09:41:49.598 INFO:teuthology.orchestra.run.smithi070.stdout: "last_in_change": "2024-09-16T09:24:28.432920+0000", 2024-09-16T09:41:49.598 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-16T09:41:49.598 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_num": 5799936, 2024-09-16T09:41:49.599 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_set": [ 2024-09-16T09:41:49.599 INFO:teuthology.orchestra.run.smithi070.stdout: "pglog_hardlimit", 2024-09-16T09:41:49.599 INFO:teuthology.orchestra.run.smithi070.stdout: "purged_snapdirs", 2024-09-16T09:41:49.599 INFO:teuthology.orchestra.run.smithi070.stdout: "recovery_deletes", 2024-09-16T09:41:49.599 INFO:teuthology.orchestra.run.smithi070.stdout: "sortbitwise" 2024-09-16T09:41:49.599 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-16T09:41:49.599 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_version": 18, 2024-09-16T09:41:49.599 INFO:teuthology.orchestra.run.smithi070.stdout: "full_ratio": 0.94999998807907104, 2024-09-16T09:41:49.599 INFO:teuthology.orchestra.run.smithi070.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-16T09:41:49.599 INFO:teuthology.orchestra.run.smithi070.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-16T09:41:49.599 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_snapshot": "", 2024-09-16T09:41:49.599 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_max": 11, 2024-09-16T09:41:49.599 INFO:teuthology.orchestra.run.smithi070.stdout: "max_osd": 8, 2024-09-16T09:41:49.600 INFO:teuthology.orchestra.run.smithi070.stdout: "require_min_compat_client": "luminous", 2024-09-16T09:41:49.600 INFO:teuthology.orchestra.run.smithi070.stdout: "min_compat_client": "luminous", 2024-09-16T09:41:49.600 INFO:teuthology.orchestra.run.smithi070.stdout: "require_osd_release": "squid", 2024-09-16T09:41:49.600 INFO:teuthology.orchestra.run.smithi070.stdout: "allow_crimson": false, 2024-09-16T09:41:49.600 INFO:teuthology.orchestra.run.smithi070.stdout: "pools": [ 2024-09-16T09:41:49.600 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.600 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 1, 2024-09-16T09:41:49.600 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": ".mgr", 2024-09-16T09:41:49.600 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2024-09-16T09:22:27.617694+0000", 2024-09-16T09:41:49.600 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2024-09-16T09:41:49.600 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2024-09-16T09:41:49.600 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2024-09-16T09:41:49.600 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 3, 2024-09-16T09:41:49.601 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 2, 2024-09-16T09:41:49.601 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2024-09-16T09:41:49.601 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2024-09-16T09:41:49.601 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2024-09-16T09:41:49.601 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T09:41:49.601 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T09:41:49.601 INFO:teuthology.orchestra.run.smithi070.stdout: "is_stretch_pool": false, 2024-09-16T09:41:49.601 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2024-09-16T09:41:49.601 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2024-09-16T09:41:49.601 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 21, 2024-09-16T09:41:49.602 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 21, 2024-09-16T09:41:49.602 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 21, 2024-09-16T09:41:49.602 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 21, 2024-09-16T09:41:49.602 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 21, 2024-09-16T09:41:49.602 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2024-09-16T09:41:49.602 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2024-09-16T09:41:49.602 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2024-09-16T09:41:49.602 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2024-09-16T09:41:49.602 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2024-09-16T09:41:49.602 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2024-09-16T09:41:49.602 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2024-09-16T09:41:49.602 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.603 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "162", 2024-09-16T09:41:49.603 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2024-09-16T09:41:49.603 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T09:41:49.603 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "160", 2024-09-16T09:41:49.603 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2024-09-16T09:41:49.603 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2024-09-16T09:41:49.603 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2024-09-16T09:41:49.603 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2024-09-16T09:41:49.603 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2024-09-16T09:41:49.603 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2024-09-16T09:41:49.603 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2024-09-16T09:41:49.603 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2024-09-16T09:41:49.603 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2024-09-16T09:41:49.603 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2024-09-16T09:41:49.604 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2024-09-16T09:41:49.604 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2024-09-16T09:41:49.604 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2024-09-16T09:41:49.604 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2024-09-16T09:41:49.604 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2024-09-16T09:41:49.604 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T09:41:49.604 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T09:41:49.604 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T09:41:49.604 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2024-09-16T09:41:49.604 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2024-09-16T09:41:49.604 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2024-09-16T09:41:49.604 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2024-09-16T09:41:49.604 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2024-09-16T09:41:49.604 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.605 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2024-09-16T09:41:49.605 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2024-09-16T09:41:49.605 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2024-09-16T09:41:49.605 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2024-09-16T09:41:49.605 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2024-09-16T09:41:49.605 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T09:41:49.605 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2024-09-16T09:41:49.605 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2024-09-16T09:41:49.605 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2024-09-16T09:41:49.605 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2024-09-16T09:41:49.605 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2024-09-16T09:41:49.605 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2024-09-16T09:41:49.605 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_max": 32, 2024-09-16T09:41:49.605 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_min": 1 2024-09-16T09:41:49.606 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.606 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2024-09-16T09:41:49.606 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": {} 2024-09-16T09:41:49.606 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.606 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2024-09-16T09:41:49.606 INFO:teuthology.orchestra.run.smithi070.stdout: "score_type": "Fair distribution", 2024-09-16T09:41:49.606 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 1.6699999570846558, 2024-09-16T09:41:49.606 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 1.6699999570846558, 2024-09-16T09:41:49.606 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1.1399999856948853, 2024-09-16T09:41:49.606 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 1.8999999761581421, 2024-09-16T09:41:49.606 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 1.8999999761581421, 2024-09-16T09:41:49.606 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 0.70719337463378906, 2024-09-16T09:41:49.607 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 0.81000000238418579, 2024-09-16T09:41:49.607 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 0.70999997854232788 2024-09-16T09:41:49.607 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:41:49.607 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.607 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.607 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 2, 2024-09-16T09:41:49.607 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "datapool", 2024-09-16T09:41:49.607 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2024-09-16T09:24:59.169244+0000", 2024-09-16T09:41:49.607 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 8193, 2024-09-16T09:41:49.607 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-16T09:41:49.607 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2024-09-16T09:41:49.607 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 3, 2024-09-16T09:41:49.607 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 2, 2024-09-16T09:41:49.608 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2024-09-16T09:41:49.608 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2024-09-16T09:41:49.608 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2024-09-16T09:41:49.608 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T09:41:49.608 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T09:41:49.608 INFO:teuthology.orchestra.run.smithi070.stdout: "is_stretch_pool": false, 2024-09-16T09:41:49.608 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2024-09-16T09:41:49.608 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2024-09-16T09:41:49.608 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 3, 2024-09-16T09:41:49.608 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 3, 2024-09-16T09:41:49.608 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 3, 2024-09-16T09:41:49.608 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 3, 2024-09-16T09:41:49.608 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 3, 2024-09-16T09:41:49.609 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2024-09-16T09:41:49.609 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2024-09-16T09:41:49.609 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2024-09-16T09:41:49.609 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2024-09-16T09:41:49.609 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2024-09-16T09:41:49.609 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2024-09-16T09:41:49.609 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2024-09-16T09:41:49.609 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.609 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "61", 2024-09-16T09:41:49.609 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2024-09-16T09:41:49.609 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T09:41:49.609 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T09:41:49.609 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2024-09-16T09:41:49.609 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2024-09-16T09:41:49.610 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 2, 2024-09-16T09:41:49.610 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 61, 2024-09-16T09:41:49.610 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2024-09-16T09:41:49.610 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2024-09-16T09:41:49.610 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2024-09-16T09:41:49.610 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2024-09-16T09:41:49.610 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2024-09-16T09:41:49.610 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2024-09-16T09:41:49.610 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2024-09-16T09:41:49.610 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2024-09-16T09:41:49.610 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2024-09-16T09:41:49.610 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2024-09-16T09:41:49.610 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2024-09-16T09:41:49.610 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T09:41:49.611 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T09:41:49.611 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T09:41:49.611 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2024-09-16T09:41:49.611 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2024-09-16T09:41:49.611 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2024-09-16T09:41:49.611 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2024-09-16T09:41:49.611 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2024-09-16T09:41:49.611 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.611 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2024-09-16T09:41:49.611 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2024-09-16T09:41:49.611 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2024-09-16T09:41:49.611 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2024-09-16T09:41:49.611 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2024-09-16T09:41:49.611 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T09:41:49.612 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2024-09-16T09:41:49.612 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2024-09-16T09:41:49.612 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2024-09-16T09:41:49.612 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2024-09-16T09:41:49.612 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2024-09-16T09:41:49.612 INFO:teuthology.orchestra.run.smithi070.stdout: "options": {}, 2024-09-16T09:41:49.612 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2024-09-16T09:41:49.612 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd": {} 2024-09-16T09:41:49.612 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.612 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2024-09-16T09:41:49.612 INFO:teuthology.orchestra.run.smithi070.stdout: "score_type": "Fair distribution", 2024-09-16T09:41:49.612 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 2.3299999237060547, 2024-09-16T09:41:49.612 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 2.3299999237060547, 2024-09-16T09:41:49.612 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 0.86000001430511475, 2024-09-16T09:41:49.613 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 2, 2024-09-16T09:41:49.613 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 2, 2024-09-16T09:41:49.613 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 0.70719337463378906, 2024-09-16T09:41:49.613 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 0.81000000238418579, 2024-09-16T09:41:49.613 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 0.70999997854232788 2024-09-16T09:41:49.613 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:41:49.613 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.613 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.613 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 3, 2024-09-16T09:41:49.613 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": ".rgw.root", 2024-09-16T09:41:49.613 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2024-09-16T09:25:00.056119+0000", 2024-09-16T09:41:49.613 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2024-09-16T09:41:49.614 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2024-09-16T09:41:49.614 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2024-09-16T09:41:49.614 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 3, 2024-09-16T09:41:49.614 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 2, 2024-09-16T09:41:49.614 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2024-09-16T09:41:49.614 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2024-09-16T09:41:49.614 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2024-09-16T09:41:49.614 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T09:41:49.614 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T09:41:49.614 INFO:teuthology.orchestra.run.smithi070.stdout: "is_stretch_pool": false, 2024-09-16T09:41:49.614 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2024-09-16T09:41:49.614 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2024-09-16T09:41:49.614 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 32, 2024-09-16T09:41:49.614 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 32, 2024-09-16T09:41:49.615 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 32, 2024-09-16T09:41:49.615 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 32, 2024-09-16T09:41:49.615 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 32, 2024-09-16T09:41:49.615 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2024-09-16T09:41:49.615 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2024-09-16T09:41:49.615 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2024-09-16T09:41:49.615 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2024-09-16T09:41:49.615 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2024-09-16T09:41:49.616 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2024-09-16T09:41:49.616 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2024-09-16T09:41:49.616 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.616 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "60", 2024-09-16T09:41:49.616 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2024-09-16T09:41:49.616 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T09:41:49.616 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T09:41:49.616 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2024-09-16T09:41:49.616 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2024-09-16T09:41:49.616 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2024-09-16T09:41:49.616 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2024-09-16T09:41:49.616 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2024-09-16T09:41:49.616 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2024-09-16T09:41:49.617 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2024-09-16T09:41:49.617 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2024-09-16T09:41:49.617 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2024-09-16T09:41:49.617 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2024-09-16T09:41:49.617 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2024-09-16T09:41:49.617 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2024-09-16T09:41:49.617 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2024-09-16T09:41:49.617 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2024-09-16T09:41:49.617 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2024-09-16T09:41:49.617 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T09:41:49.617 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T09:41:49.617 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T09:41:49.617 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2024-09-16T09:41:49.617 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2024-09-16T09:41:49.618 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2024-09-16T09:41:49.618 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2024-09-16T09:41:49.618 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2024-09-16T09:41:49.618 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.618 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2024-09-16T09:41:49.618 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2024-09-16T09:41:49.618 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2024-09-16T09:41:49.618 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2024-09-16T09:41:49.618 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2024-09-16T09:41:49.618 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T09:41:49.618 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2024-09-16T09:41:49.618 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2024-09-16T09:41:49.618 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2024-09-16T09:41:49.618 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2024-09-16T09:41:49.619 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2024-09-16T09:41:49.619 INFO:teuthology.orchestra.run.smithi070.stdout: "options": {}, 2024-09-16T09:41:49.619 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2024-09-16T09:41:49.619 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw": {} 2024-09-16T09:41:49.619 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.619 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2024-09-16T09:41:49.619 INFO:teuthology.orchestra.run.smithi070.stdout: "score_type": "Fair distribution", 2024-09-16T09:41:49.619 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 1.75, 2024-09-16T09:41:49.619 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 1.75, 2024-09-16T09:41:49.619 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1.1399999856948853, 2024-09-16T09:41:49.619 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 2, 2024-09-16T09:41:49.619 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 2, 2024-09-16T09:41:49.619 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 0.70719337463378906, 2024-09-16T09:41:49.619 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 0.81000000238418579, 2024-09-16T09:41:49.620 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 0.70999997854232788 2024-09-16T09:41:49.620 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:41:49.620 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.620 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.620 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 4, 2024-09-16T09:41:49.620 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "default.rgw.log", 2024-09-16T09:41:49.620 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2024-09-16T09:25:01.770016+0000", 2024-09-16T09:41:49.620 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2024-09-16T09:41:49.620 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2024-09-16T09:41:49.620 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2024-09-16T09:41:49.620 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 3, 2024-09-16T09:41:49.620 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 2, 2024-09-16T09:41:49.620 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2024-09-16T09:41:49.621 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2024-09-16T09:41:49.621 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2024-09-16T09:41:49.621 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T09:41:49.621 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T09:41:49.621 INFO:teuthology.orchestra.run.smithi070.stdout: "is_stretch_pool": false, 2024-09-16T09:41:49.621 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2024-09-16T09:41:49.621 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2024-09-16T09:41:49.621 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 32, 2024-09-16T09:41:49.621 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 32, 2024-09-16T09:41:49.621 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 32, 2024-09-16T09:41:49.621 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 32, 2024-09-16T09:41:49.621 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 32, 2024-09-16T09:41:49.621 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2024-09-16T09:41:49.622 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2024-09-16T09:41:49.622 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2024-09-16T09:41:49.622 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2024-09-16T09:41:49.622 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2024-09-16T09:41:49.622 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2024-09-16T09:41:49.622 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2024-09-16T09:41:49.622 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.622 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "62", 2024-09-16T09:41:49.622 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2024-09-16T09:41:49.622 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T09:41:49.622 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T09:41:49.622 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2024-09-16T09:41:49.622 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2024-09-16T09:41:49.622 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2024-09-16T09:41:49.622 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2024-09-16T09:41:49.623 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2024-09-16T09:41:49.623 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2024-09-16T09:41:49.623 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2024-09-16T09:41:49.623 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2024-09-16T09:41:49.623 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2024-09-16T09:41:49.623 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2024-09-16T09:41:49.623 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2024-09-16T09:41:49.623 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2024-09-16T09:41:49.623 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2024-09-16T09:41:49.623 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2024-09-16T09:41:49.623 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2024-09-16T09:41:49.623 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T09:41:49.623 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T09:41:49.623 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T09:41:49.624 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2024-09-16T09:41:49.624 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2024-09-16T09:41:49.624 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2024-09-16T09:41:49.624 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2024-09-16T09:41:49.624 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2024-09-16T09:41:49.624 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.624 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2024-09-16T09:41:49.624 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2024-09-16T09:41:49.624 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2024-09-16T09:41:49.624 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2024-09-16T09:41:49.624 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2024-09-16T09:41:49.624 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T09:41:49.624 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2024-09-16T09:41:49.624 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2024-09-16T09:41:49.625 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2024-09-16T09:41:49.625 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2024-09-16T09:41:49.625 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2024-09-16T09:41:49.625 INFO:teuthology.orchestra.run.smithi070.stdout: "options": {}, 2024-09-16T09:41:49.625 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2024-09-16T09:41:49.625 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw": {} 2024-09-16T09:41:49.625 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.625 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2024-09-16T09:41:49.625 INFO:teuthology.orchestra.run.smithi070.stdout: "score_type": "Fair distribution", 2024-09-16T09:41:49.625 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 6.8499999046325684, 2024-09-16T09:41:49.625 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 2.4100000858306885, 2024-09-16T09:41:49.625 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1.1399999856948853, 2024-09-16T09:41:49.625 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 7.809999942779541, 2024-09-16T09:41:49.625 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 2.75, 2024-09-16T09:41:49.625 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 0.70719337463378906, 2024-09-16T09:41:49.626 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 0.81000000238418579, 2024-09-16T09:41:49.626 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 0.70999997854232788, 2024-09-16T09:41:49.626 INFO:teuthology.orchestra.run.smithi070.stdout: "error_message": "pool 4 has acting primaries on OSD(s) with primary affinity 0, read balance score is not accurate" 2024-09-16T09:41:49.626 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:41:49.626 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.626 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.626 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 5, 2024-09-16T09:41:49.626 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "default.rgw.control", 2024-09-16T09:41:49.626 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2024-09-16T09:25:03.725555+0000", 2024-09-16T09:41:49.626 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2024-09-16T09:41:49.626 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2024-09-16T09:41:49.626 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2024-09-16T09:41:49.626 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 3, 2024-09-16T09:41:49.626 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 2, 2024-09-16T09:41:49.626 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2024-09-16T09:41:49.627 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2024-09-16T09:41:49.627 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2024-09-16T09:41:49.627 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T09:41:49.627 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T09:41:49.627 INFO:teuthology.orchestra.run.smithi070.stdout: "is_stretch_pool": false, 2024-09-16T09:41:49.627 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2024-09-16T09:41:49.627 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2024-09-16T09:41:49.627 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 32, 2024-09-16T09:41:49.627 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 32, 2024-09-16T09:41:49.627 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 32, 2024-09-16T09:41:49.627 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 32, 2024-09-16T09:41:49.627 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 32, 2024-09-16T09:41:49.627 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2024-09-16T09:41:49.627 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2024-09-16T09:41:49.627 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2024-09-16T09:41:49.628 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2024-09-16T09:41:49.628 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2024-09-16T09:41:49.628 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2024-09-16T09:41:49.628 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2024-09-16T09:41:49.628 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.628 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "64", 2024-09-16T09:41:49.628 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2024-09-16T09:41:49.628 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T09:41:49.628 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T09:41:49.628 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2024-09-16T09:41:49.628 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2024-09-16T09:41:49.628 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2024-09-16T09:41:49.628 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2024-09-16T09:41:49.628 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2024-09-16T09:41:49.628 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2024-09-16T09:41:49.629 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2024-09-16T09:41:49.629 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2024-09-16T09:41:49.629 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2024-09-16T09:41:49.629 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2024-09-16T09:41:49.629 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2024-09-16T09:41:49.629 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2024-09-16T09:41:49.629 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2024-09-16T09:41:49.629 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2024-09-16T09:41:49.629 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2024-09-16T09:41:49.629 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T09:41:49.629 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T09:41:49.629 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T09:41:49.629 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2024-09-16T09:41:49.630 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2024-09-16T09:41:49.630 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2024-09-16T09:41:49.630 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2024-09-16T09:41:49.630 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2024-09-16T09:41:49.630 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.630 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2024-09-16T09:41:49.630 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2024-09-16T09:41:49.630 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2024-09-16T09:41:49.630 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2024-09-16T09:41:49.630 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2024-09-16T09:41:49.630 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T09:41:49.630 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2024-09-16T09:41:49.630 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2024-09-16T09:41:49.631 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2024-09-16T09:41:49.631 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2024-09-16T09:41:49.631 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2024-09-16T09:41:49.631 INFO:teuthology.orchestra.run.smithi070.stdout: "options": {}, 2024-09-16T09:41:49.631 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2024-09-16T09:41:49.631 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw": {} 2024-09-16T09:41:49.631 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.631 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2024-09-16T09:41:49.631 INFO:teuthology.orchestra.run.smithi070.stdout: "score_type": "Fair distribution", 2024-09-16T09:41:49.631 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 1.75, 2024-09-16T09:41:49.631 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 1.75, 2024-09-16T09:41:49.631 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1.1399999856948853, 2024-09-16T09:41:49.631 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 2, 2024-09-16T09:41:49.631 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 2, 2024-09-16T09:41:49.631 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 0.70719337463378906, 2024-09-16T09:41:49.632 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 0.81000000238418579, 2024-09-16T09:41:49.632 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 0.70999997854232788 2024-09-16T09:41:49.632 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:41:49.632 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.632 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.632 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 6, 2024-09-16T09:41:49.632 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "default.rgw.meta", 2024-09-16T09:41:49.632 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2024-09-16T09:25:05.773530+0000", 2024-09-16T09:41:49.632 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2024-09-16T09:41:49.632 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2024-09-16T09:41:49.632 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2024-09-16T09:41:49.632 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 3, 2024-09-16T09:41:49.632 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 2, 2024-09-16T09:41:49.632 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2024-09-16T09:41:49.633 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2024-09-16T09:41:49.633 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2024-09-16T09:41:49.633 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T09:41:49.633 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T09:41:49.633 INFO:teuthology.orchestra.run.smithi070.stdout: "is_stretch_pool": false, 2024-09-16T09:41:49.633 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2024-09-16T09:41:49.633 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2024-09-16T09:41:49.633 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 32, 2024-09-16T09:41:49.633 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 32, 2024-09-16T09:41:49.633 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 32, 2024-09-16T09:41:49.633 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 32, 2024-09-16T09:41:49.633 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 32, 2024-09-16T09:41:49.633 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2024-09-16T09:41:49.633 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2024-09-16T09:41:49.634 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2024-09-16T09:41:49.634 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2024-09-16T09:41:49.634 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2024-09-16T09:41:49.634 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2024-09-16T09:41:49.634 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2024-09-16T09:41:49.634 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.634 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "66", 2024-09-16T09:41:49.634 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2024-09-16T09:41:49.634 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T09:41:49.634 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T09:41:49.634 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2024-09-16T09:41:49.634 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2024-09-16T09:41:49.634 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2024-09-16T09:41:49.634 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2024-09-16T09:41:49.635 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2024-09-16T09:41:49.635 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2024-09-16T09:41:49.635 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2024-09-16T09:41:49.635 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2024-09-16T09:41:49.635 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2024-09-16T09:41:49.635 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2024-09-16T09:41:49.635 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2024-09-16T09:41:49.635 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2024-09-16T09:41:49.635 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2024-09-16T09:41:49.635 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2024-09-16T09:41:49.635 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2024-09-16T09:41:49.635 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T09:41:49.635 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T09:41:49.635 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T09:41:49.635 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2024-09-16T09:41:49.635 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2024-09-16T09:41:49.636 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2024-09-16T09:41:49.636 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2024-09-16T09:41:49.636 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2024-09-16T09:41:49.636 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.636 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2024-09-16T09:41:49.636 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2024-09-16T09:41:49.636 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2024-09-16T09:41:49.636 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2024-09-16T09:41:49.636 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2024-09-16T09:41:49.636 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T09:41:49.636 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2024-09-16T09:41:49.636 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2024-09-16T09:41:49.636 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2024-09-16T09:41:49.636 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2024-09-16T09:41:49.637 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2024-09-16T09:41:49.637 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2024-09-16T09:41:49.637 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_bias": 4 2024-09-16T09:41:49.637 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.637 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2024-09-16T09:41:49.637 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw": {} 2024-09-16T09:41:49.637 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.637 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2024-09-16T09:41:49.637 INFO:teuthology.orchestra.run.smithi070.stdout: "score_type": "Fair distribution", 2024-09-16T09:41:49.637 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 1.5399999618530273, 2024-09-16T09:41:49.637 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 1.5399999618530273, 2024-09-16T09:41:49.637 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1.1399999856948853, 2024-09-16T09:41:49.637 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 1.75, 2024-09-16T09:41:49.637 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 1.75, 2024-09-16T09:41:49.637 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 0.70719337463378906, 2024-09-16T09:41:49.638 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 0.81000000238418579, 2024-09-16T09:41:49.638 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 0.70999997854232788 2024-09-16T09:41:49.638 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:41:49.638 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.638 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.638 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 11, 2024-09-16T09:41:49.638 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "unique_pool_4", 2024-09-16T09:41:49.638 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2024-09-16T09:41:11.917610+0000", 2024-09-16T09:41:49.638 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2024-09-16T09:41:49.638 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2024-09-16T09:41:49.638 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2024-09-16T09:41:49.638 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 3, 2024-09-16T09:41:49.638 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 2, 2024-09-16T09:41:49.638 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2024-09-16T09:41:49.638 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2024-09-16T09:41:49.639 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2024-09-16T09:41:49.639 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T09:41:49.639 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T09:41:49.639 INFO:teuthology.orchestra.run.smithi070.stdout: "is_stretch_pool": false, 2024-09-16T09:41:49.639 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2024-09-16T09:41:49.639 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2024-09-16T09:41:49.639 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 16, 2024-09-16T09:41:49.639 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 16, 2024-09-16T09:41:49.639 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 16, 2024-09-16T09:41:49.639 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 16, 2024-09-16T09:41:49.639 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 16, 2024-09-16T09:41:49.639 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2024-09-16T09:41:49.639 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2024-09-16T09:41:49.639 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2024-09-16T09:41:49.639 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2024-09-16T09:41:49.640 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2024-09-16T09:41:49.640 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2024-09-16T09:41:49.640 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2024-09-16T09:41:49.640 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.640 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "458", 2024-09-16T09:41:49.640 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2024-09-16T09:41:49.640 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T09:41:49.640 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T09:41:49.640 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2024-09-16T09:41:49.640 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2024-09-16T09:41:49.640 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2024-09-16T09:41:49.640 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2024-09-16T09:41:49.640 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2024-09-16T09:41:49.640 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2024-09-16T09:41:49.640 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2024-09-16T09:41:49.641 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2024-09-16T09:41:49.641 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2024-09-16T09:41:49.641 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2024-09-16T09:41:49.641 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2024-09-16T09:41:49.641 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2024-09-16T09:41:49.641 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2024-09-16T09:41:49.641 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2024-09-16T09:41:49.641 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2024-09-16T09:41:49.641 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T09:41:49.641 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T09:41:49.641 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T09:41:49.641 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2024-09-16T09:41:49.641 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2024-09-16T09:41:49.641 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2024-09-16T09:41:49.642 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2024-09-16T09:41:49.642 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2024-09-16T09:41:49.642 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.642 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2024-09-16T09:41:49.642 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2024-09-16T09:41:49.642 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2024-09-16T09:41:49.642 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2024-09-16T09:41:49.642 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2024-09-16T09:41:49.642 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T09:41:49.642 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2024-09-16T09:41:49.642 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2024-09-16T09:41:49.642 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2024-09-16T09:41:49.642 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2024-09-16T09:41:49.642 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2024-09-16T09:41:49.643 INFO:teuthology.orchestra.run.smithi070.stdout: "options": {}, 2024-09-16T09:41:49.643 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2024-09-16T09:41:49.643 INFO:teuthology.orchestra.run.smithi070.stdout: "rados": {} 2024-09-16T09:41:49.643 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.643 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2024-09-16T09:41:49.643 INFO:teuthology.orchestra.run.smithi070.stdout: "score_type": "Fair distribution", 2024-09-16T09:41:49.643 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 1.75, 2024-09-16T09:41:49.643 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 1.75, 2024-09-16T09:41:49.643 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1.1399999856948853, 2024-09-16T09:41:49.643 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 2, 2024-09-16T09:41:49.643 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 2, 2024-09-16T09:41:49.643 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 0.70719337463378906, 2024-09-16T09:41:49.643 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 0.81000000238418579, 2024-09-16T09:41:49.643 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 0.70999997854232788 2024-09-16T09:41:49.643 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:41:49.644 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:41:49.644 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-16T09:41:49.644 INFO:teuthology.orchestra.run.smithi070.stdout: "osds": [ 2024-09-16T09:41:49.644 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.644 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 0, 2024-09-16T09:41:49.644 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "7ef288c8-3bf0-4814-8e86-f70a7de041ce", 2024-09-16T09:41:49.644 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2024-09-16T09:41:49.644 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2024-09-16T09:41:49.644 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2024-09-16T09:41:49.644 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2024-09-16T09:41:49.644 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2024-09-16T09:41:49.644 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2024-09-16T09:41:49.644 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 8, 2024-09-16T09:41:49.644 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 454, 2024-09-16T09:41:49.644 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2024-09-16T09:41:49.645 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2024-09-16T09:41:49.645 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2024-09-16T09:41:49.645 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:41:49.645 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.645 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:41:49.645 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6802", 2024-09-16T09:41:49.645 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2013075781 2024-09-16T09:41:49.645 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.645 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.645 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:41:49.645 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6803", 2024-09-16T09:41:49.645 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2013075781 2024-09-16T09:41:49.645 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:41:49.645 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:41:49.646 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.646 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2024-09-16T09:41:49.646 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:41:49.646 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.646 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:41:49.646 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6804", 2024-09-16T09:41:49.646 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2013075781 2024-09-16T09:41:49.646 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.646 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.646 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:41:49.646 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6805", 2024-09-16T09:41:49.646 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2013075781 2024-09-16T09:41:49.646 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:41:49.647 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:41:49.647 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.647 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2024-09-16T09:41:49.647 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:41:49.647 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.647 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:41:49.647 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6808", 2024-09-16T09:41:49.647 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2013075781 2024-09-16T09:41:49.648 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.648 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.648 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:41:49.648 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6809", 2024-09-16T09:41:49.648 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2013075781 2024-09-16T09:41:49.648 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:41:49.648 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:41:49.648 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.648 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2024-09-16T09:41:49.648 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:41:49.648 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.648 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:41:49.648 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6806", 2024-09-16T09:41:49.648 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2013075781 2024-09-16T09:41:49.649 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.649 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.649 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:41:49.649 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6807", 2024-09-16T09:41:49.649 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2013075781 2024-09-16T09:41:49.649 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:41:49.649 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:41:49.649 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.649 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6803/2013075781", 2024-09-16T09:41:49.649 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6805/2013075781", 2024-09-16T09:41:49.649 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6809/2013075781", 2024-09-16T09:41:49.649 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6807/2013075781", 2024-09-16T09:41:49.650 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2024-09-16T09:41:49.650 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2024-09-16T09:41:49.650 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2024-09-16T09:41:49.650 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:41:49.650 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.650 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.650 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 1, 2024-09-16T09:41:49.650 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "474d41d9-295a-450c-8f27-ce487a14678b", 2024-09-16T09:41:49.650 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2024-09-16T09:41:49.650 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2024-09-16T09:41:49.650 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2024-09-16T09:41:49.650 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 0, 2024-09-16T09:41:49.651 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2024-09-16T09:41:49.651 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2024-09-16T09:41:49.651 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2024-09-16T09:41:49.651 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 362, 2024-09-16T09:41:49.651 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2024-09-16T09:41:49.651 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2024-09-16T09:41:49.651 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2024-09-16T09:41:49.651 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:41:49.651 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.651 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:41:49.651 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6810", 2024-09-16T09:41:49.651 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1670012115 2024-09-16T09:41:49.651 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.652 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.652 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:41:49.652 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6811", 2024-09-16T09:41:49.652 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1670012115 2024-09-16T09:41:49.652 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:41:49.652 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:41:49.652 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.652 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2024-09-16T09:41:49.652 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:41:49.652 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.652 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:41:49.652 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6812", 2024-09-16T09:41:49.652 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1670012115 2024-09-16T09:41:49.653 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.653 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.653 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:41:49.653 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6813", 2024-09-16T09:41:49.653 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1670012115 2024-09-16T09:41:49.653 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:41:49.653 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:41:49.653 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.653 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2024-09-16T09:41:49.653 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:41:49.653 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.653 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:41:49.653 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6816", 2024-09-16T09:41:49.654 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1670012115 2024-09-16T09:41:49.654 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.654 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.654 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:41:49.654 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6817", 2024-09-16T09:41:49.654 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1670012115 2024-09-16T09:41:49.654 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:41:49.654 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:41:49.654 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.654 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2024-09-16T09:41:49.654 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:41:49.654 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.654 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:41:49.655 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6814", 2024-09-16T09:41:49.655 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1670012115 2024-09-16T09:41:49.655 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.655 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.655 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:41:49.655 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6815", 2024-09-16T09:41:49.655 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1670012115 2024-09-16T09:41:49.655 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:41:49.655 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:41:49.655 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.655 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6811/1670012115", 2024-09-16T09:41:49.655 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6813/1670012115", 2024-09-16T09:41:49.655 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6817/1670012115", 2024-09-16T09:41:49.655 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6815/1670012115", 2024-09-16T09:41:49.656 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2024-09-16T09:41:49.656 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2024-09-16T09:41:49.656 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2024-09-16T09:41:49.656 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:41:49.656 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.656 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.656 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 2, 2024-09-16T09:41:49.656 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "5b4ca4e8-cecc-4163-b291-c9209b02c4b7", 2024-09-16T09:41:49.656 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2024-09-16T09:41:49.656 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2024-09-16T09:41:49.656 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2024-09-16T09:41:49.657 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 0.1810150146484375, 2024-09-16T09:41:49.657 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2024-09-16T09:41:49.657 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2024-09-16T09:41:49.657 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 18, 2024-09-16T09:41:49.657 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 454, 2024-09-16T09:41:49.657 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2024-09-16T09:41:49.657 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2024-09-16T09:41:49.657 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2024-09-16T09:41:49.657 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:41:49.657 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.657 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:41:49.657 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6818", 2024-09-16T09:41:49.657 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3601682336 2024-09-16T09:41:49.658 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.658 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.658 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:41:49.658 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6819", 2024-09-16T09:41:49.658 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3601682336 2024-09-16T09:41:49.658 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:41:49.658 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:41:49.658 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.658 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2024-09-16T09:41:49.658 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:41:49.658 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.658 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:41:49.658 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6820", 2024-09-16T09:41:49.659 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3601682336 2024-09-16T09:41:49.659 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.659 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.659 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:41:49.659 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6821", 2024-09-16T09:41:49.659 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3601682336 2024-09-16T09:41:49.659 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:41:49.659 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:41:49.659 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.659 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2024-09-16T09:41:49.659 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:41:49.659 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.659 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:41:49.659 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6824", 2024-09-16T09:41:49.660 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3601682336 2024-09-16T09:41:49.660 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.660 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.660 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:41:49.660 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6825", 2024-09-16T09:41:49.660 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3601682336 2024-09-16T09:41:49.660 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:41:49.660 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:41:49.660 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.660 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2024-09-16T09:41:49.660 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:41:49.660 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.660 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:41:49.661 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6822", 2024-09-16T09:41:49.661 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3601682336 2024-09-16T09:41:49.661 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.661 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.661 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:41:49.661 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6823", 2024-09-16T09:41:49.661 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3601682336 2024-09-16T09:41:49.661 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:41:49.661 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:41:49.661 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.661 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6819/3601682336", 2024-09-16T09:41:49.661 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6821/3601682336", 2024-09-16T09:41:49.662 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6825/3601682336", 2024-09-16T09:41:49.662 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6823/3601682336", 2024-09-16T09:41:49.662 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2024-09-16T09:41:49.662 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2024-09-16T09:41:49.662 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2024-09-16T09:41:49.662 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:41:49.662 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.662 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.663 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 3, 2024-09-16T09:41:49.663 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "ffb8d527-1b58-4ecd-808c-47d4ba1bf387", 2024-09-16T09:41:49.663 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2024-09-16T09:41:49.663 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2024-09-16T09:41:49.663 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2024-09-16T09:41:49.663 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 0.476531982421875, 2024-09-16T09:41:49.663 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2024-09-16T09:41:49.663 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2024-09-16T09:41:49.663 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 26, 2024-09-16T09:41:49.663 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 454, 2024-09-16T09:41:49.663 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2024-09-16T09:41:49.663 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2024-09-16T09:41:49.664 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2024-09-16T09:41:49.664 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:41:49.664 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.664 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:41:49.664 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6826", 2024-09-16T09:41:49.664 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2038083007 2024-09-16T09:41:49.664 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.664 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.664 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:41:49.664 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6827", 2024-09-16T09:41:49.664 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2038083007 2024-09-16T09:41:49.664 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:41:49.664 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:41:49.665 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.665 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2024-09-16T09:41:49.665 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:41:49.665 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.665 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:41:49.665 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6828", 2024-09-16T09:41:49.665 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2038083007 2024-09-16T09:41:49.665 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.665 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.665 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:41:49.665 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6829", 2024-09-16T09:41:49.665 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2038083007 2024-09-16T09:41:49.665 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:41:49.665 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:41:49.666 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.666 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2024-09-16T09:41:49.666 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:41:49.666 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.666 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:41:49.666 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6832", 2024-09-16T09:41:49.666 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2038083007 2024-09-16T09:41:49.666 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.666 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.666 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:41:49.666 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6833", 2024-09-16T09:41:49.666 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2038083007 2024-09-16T09:41:49.666 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:41:49.667 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:41:49.667 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.667 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2024-09-16T09:41:49.667 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:41:49.667 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.667 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:41:49.667 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6830", 2024-09-16T09:41:49.667 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2038083007 2024-09-16T09:41:49.667 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.667 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.667 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:41:49.667 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6831", 2024-09-16T09:41:49.667 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2038083007 2024-09-16T09:41:49.667 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:41:49.668 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:41:49.668 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.668 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6827/2038083007", 2024-09-16T09:41:49.668 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6829/2038083007", 2024-09-16T09:41:49.668 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6833/2038083007", 2024-09-16T09:41:49.668 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6831/2038083007", 2024-09-16T09:41:49.668 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2024-09-16T09:41:49.668 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2024-09-16T09:41:49.668 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2024-09-16T09:41:49.668 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:41:49.668 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.668 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.668 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 4, 2024-09-16T09:41:49.668 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "a946760b-4d91-4a4b-8e9b-a07a50589211", 2024-09-16T09:41:49.669 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2024-09-16T09:41:49.669 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2024-09-16T09:41:49.669 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2024-09-16T09:41:49.669 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2024-09-16T09:41:49.669 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2024-09-16T09:41:49.669 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2024-09-16T09:41:49.669 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 32, 2024-09-16T09:41:49.669 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 454, 2024-09-16T09:41:49.669 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2024-09-16T09:41:49.669 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2024-09-16T09:41:49.669 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2024-09-16T09:41:49.669 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:41:49.669 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.670 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:41:49.670 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6800", 2024-09-16T09:41:49.670 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2250892710 2024-09-16T09:41:49.670 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.670 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.670 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:41:49.670 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6801", 2024-09-16T09:41:49.670 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2250892710 2024-09-16T09:41:49.670 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:41:49.670 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:41:49.670 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.670 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2024-09-16T09:41:49.670 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:41:49.671 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.671 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:41:49.671 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6802", 2024-09-16T09:41:49.671 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2250892710 2024-09-16T09:41:49.671 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.671 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.671 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:41:49.671 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6803", 2024-09-16T09:41:49.671 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2250892710 2024-09-16T09:41:49.671 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:41:49.671 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:41:49.671 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.672 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2024-09-16T09:41:49.672 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:41:49.672 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.672 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:41:49.672 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6806", 2024-09-16T09:41:49.672 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2250892710 2024-09-16T09:41:49.672 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.672 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.672 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:41:49.672 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6807", 2024-09-16T09:41:49.672 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2250892710 2024-09-16T09:41:49.672 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:41:49.673 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:41:49.673 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.673 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2024-09-16T09:41:49.673 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:41:49.673 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.673 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:41:49.673 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6804", 2024-09-16T09:41:49.673 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2250892710 2024-09-16T09:41:49.673 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.673 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.673 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:41:49.673 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6805", 2024-09-16T09:41:49.674 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2250892710 2024-09-16T09:41:49.674 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:41:49.674 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:41:49.674 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.674 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.84:6801/2250892710", 2024-09-16T09:41:49.674 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.84:6803/2250892710", 2024-09-16T09:41:49.674 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.84:6807/2250892710", 2024-09-16T09:41:49.674 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.84:6805/2250892710", 2024-09-16T09:41:49.674 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2024-09-16T09:41:49.674 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2024-09-16T09:41:49.674 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2024-09-16T09:41:49.674 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:41:49.675 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.675 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.675 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 5, 2024-09-16T09:41:49.675 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "67681760-75a9-4846-946e-a935d42c5e05", 2024-09-16T09:41:49.675 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2024-09-16T09:41:49.675 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2024-09-16T09:41:49.675 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2024-09-16T09:41:49.675 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2024-09-16T09:41:49.675 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2024-09-16T09:41:49.675 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2024-09-16T09:41:49.675 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 39, 2024-09-16T09:41:49.675 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 454, 2024-09-16T09:41:49.675 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2024-09-16T09:41:49.676 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2024-09-16T09:41:49.676 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2024-09-16T09:41:49.676 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:41:49.676 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.676 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:41:49.676 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6808", 2024-09-16T09:41:49.676 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1803945348 2024-09-16T09:41:49.676 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.676 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.676 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:41:49.676 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6809", 2024-09-16T09:41:49.676 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1803945348 2024-09-16T09:41:49.676 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:41:49.677 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:41:49.677 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.677 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2024-09-16T09:41:49.677 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:41:49.677 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.677 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:41:49.677 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6810", 2024-09-16T09:41:49.677 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1803945348 2024-09-16T09:41:49.677 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.677 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.677 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:41:49.677 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6811", 2024-09-16T09:41:49.677 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1803945348 2024-09-16T09:41:49.678 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:41:49.678 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:41:49.678 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.678 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2024-09-16T09:41:49.678 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:41:49.678 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.678 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:41:49.678 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6814", 2024-09-16T09:41:49.678 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1803945348 2024-09-16T09:41:49.678 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.678 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.678 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:41:49.679 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6815", 2024-09-16T09:41:49.679 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1803945348 2024-09-16T09:41:49.679 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:41:49.679 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:41:49.679 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.679 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2024-09-16T09:41:49.679 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:41:49.679 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.679 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:41:49.679 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6812", 2024-09-16T09:41:49.679 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1803945348 2024-09-16T09:41:49.679 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.679 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.680 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:41:49.680 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6813", 2024-09-16T09:41:49.680 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1803945348 2024-09-16T09:41:49.680 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:41:49.680 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:41:49.680 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.680 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.84:6809/1803945348", 2024-09-16T09:41:49.680 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.84:6811/1803945348", 2024-09-16T09:41:49.680 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.84:6815/1803945348", 2024-09-16T09:41:49.680 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.84:6813/1803945348", 2024-09-16T09:41:49.680 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2024-09-16T09:41:49.680 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2024-09-16T09:41:49.680 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2024-09-16T09:41:49.680 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:41:49.681 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.681 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.681 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 6, 2024-09-16T09:41:49.681 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "7c8f0152-f539-4b7e-8113-91dd7529d3a1", 2024-09-16T09:41:49.681 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2024-09-16T09:41:49.681 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2024-09-16T09:41:49.681 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2024-09-16T09:41:49.681 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2024-09-16T09:41:49.681 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2024-09-16T09:41:49.681 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2024-09-16T09:41:49.681 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 46, 2024-09-16T09:41:49.681 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 454, 2024-09-16T09:41:49.682 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2024-09-16T09:41:49.682 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2024-09-16T09:41:49.682 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2024-09-16T09:41:49.682 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:41:49.682 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.682 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:41:49.682 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6816", 2024-09-16T09:41:49.682 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 610727189 2024-09-16T09:41:49.682 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.682 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.682 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:41:49.682 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6817", 2024-09-16T09:41:49.682 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 610727189 2024-09-16T09:41:49.682 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:41:49.682 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:41:49.683 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.683 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2024-09-16T09:41:49.683 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:41:49.683 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.683 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:41:49.683 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6818", 2024-09-16T09:41:49.683 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 610727189 2024-09-16T09:41:49.683 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.683 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.683 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:41:49.683 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6819", 2024-09-16T09:41:49.683 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 610727189 2024-09-16T09:41:49.683 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:41:49.684 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:41:49.684 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.684 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2024-09-16T09:41:49.684 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:41:49.684 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.684 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:41:49.684 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6822", 2024-09-16T09:41:49.684 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 610727189 2024-09-16T09:41:49.684 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.684 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.684 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:41:49.684 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6823", 2024-09-16T09:41:49.684 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 610727189 2024-09-16T09:41:49.684 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:41:49.684 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:41:49.685 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.685 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2024-09-16T09:41:49.685 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:41:49.685 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.685 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:41:49.685 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6820", 2024-09-16T09:41:49.685 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 610727189 2024-09-16T09:41:49.685 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.685 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.685 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:41:49.685 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6821", 2024-09-16T09:41:49.685 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 610727189 2024-09-16T09:41:49.685 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:41:49.685 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:41:49.685 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.686 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.84:6817/610727189", 2024-09-16T09:41:49.686 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.84:6819/610727189", 2024-09-16T09:41:49.686 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.84:6823/610727189", 2024-09-16T09:41:49.686 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.84:6821/610727189", 2024-09-16T09:41:49.686 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2024-09-16T09:41:49.686 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2024-09-16T09:41:49.687 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2024-09-16T09:41:49.687 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:41:49.687 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.687 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.687 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 7, 2024-09-16T09:41:49.687 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "945efaba-225e-49af-aefd-b4d58ca8093b", 2024-09-16T09:41:49.687 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2024-09-16T09:41:49.687 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2024-09-16T09:41:49.687 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2024-09-16T09:41:49.687 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2024-09-16T09:41:49.687 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2024-09-16T09:41:49.687 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2024-09-16T09:41:49.688 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 53, 2024-09-16T09:41:49.688 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 454, 2024-09-16T09:41:49.688 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2024-09-16T09:41:49.688 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2024-09-16T09:41:49.688 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2024-09-16T09:41:49.688 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:41:49.688 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.688 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:41:49.688 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6824", 2024-09-16T09:41:49.688 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1002252531 2024-09-16T09:41:49.688 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.688 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.688 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:41:49.688 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6825", 2024-09-16T09:41:49.688 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1002252531 2024-09-16T09:41:49.689 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:41:49.689 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:41:49.689 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.689 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2024-09-16T09:41:49.689 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:41:49.689 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.689 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:41:49.689 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6826", 2024-09-16T09:41:49.689 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1002252531 2024-09-16T09:41:49.689 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.689 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.689 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:41:49.689 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6827", 2024-09-16T09:41:49.689 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1002252531 2024-09-16T09:41:49.690 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:41:49.690 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:41:49.690 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.690 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2024-09-16T09:41:49.690 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:41:49.690 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.690 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:41:49.690 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6830", 2024-09-16T09:41:49.690 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1002252531 2024-09-16T09:41:49.690 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.690 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.690 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:41:49.690 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6831", 2024-09-16T09:41:49.690 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1002252531 2024-09-16T09:41:49.690 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:41:49.691 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:41:49.691 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.691 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2024-09-16T09:41:49.691 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:41:49.691 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.691 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:41:49.691 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6828", 2024-09-16T09:41:49.691 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1002252531 2024-09-16T09:41:49.691 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.691 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.691 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:41:49.691 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6829", 2024-09-16T09:41:49.691 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1002252531 2024-09-16T09:41:49.691 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:41:49.691 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:41:49.692 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.692 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.84:6825/1002252531", 2024-09-16T09:41:49.692 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.84:6827/1002252531", 2024-09-16T09:41:49.692 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.84:6831/1002252531", 2024-09-16T09:41:49.692 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.84:6829/1002252531", 2024-09-16T09:41:49.692 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2024-09-16T09:41:49.692 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2024-09-16T09:41:49.692 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2024-09-16T09:41:49.692 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:41:49.692 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:41:49.692 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-16T09:41:49.692 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_xinfo": [ 2024-09-16T09:41:49.692 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.692 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 0, 2024-09-16T09:41:49.693 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2024-09-16T09:41:49.693 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2024-09-16T09:41:49.693 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2024-09-16T09:41:49.693 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540719139924082687, 2024-09-16T09:41:49.693 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2024-09-16T09:41:49.693 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2024-09-16T09:41:12.518411+0000", 2024-09-16T09:41:49.693 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2024-09-16T09:41:49.693 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.693 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.693 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 1, 2024-09-16T09:41:49.693 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2024-09-16T09:41:49.693 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2024-09-16T09:41:49.693 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2024-09-16T09:41:49.694 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540719139924082687, 2024-09-16T09:41:49.694 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2024-09-16T09:41:49.694 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2024-09-16T09:41:01.871112+0000", 2024-09-16T09:41:49.694 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2024-09-16T09:41:49.694 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.694 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.694 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 2, 2024-09-16T09:41:49.694 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2024-09-16T09:41:49.694 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2024-09-16T09:41:49.694 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2024-09-16T09:41:49.694 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540719139924082687, 2024-09-16T09:41:49.694 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2024-09-16T09:41:49.694 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2024-09-16T09:41:21.197490+0000", 2024-09-16T09:41:49.694 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2024-09-16T09:41:49.694 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.695 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.695 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 3, 2024-09-16T09:41:49.695 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2024-09-16T09:41:49.695 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2024-09-16T09:41:49.695 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2024-09-16T09:41:49.695 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540719139924082687, 2024-09-16T09:41:49.695 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2024-09-16T09:41:49.695 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2024-09-16T09:41:02.801287+0000", 2024-09-16T09:41:49.695 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2024-09-16T09:41:49.695 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.695 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.695 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 4, 2024-09-16T09:41:49.695 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2024-09-16T09:41:49.695 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2024-09-16T09:41:49.696 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2024-09-16T09:41:49.696 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540719139924082687, 2024-09-16T09:41:49.696 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2024-09-16T09:41:49.696 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2024-09-16T09:40:54.659737+0000", 2024-09-16T09:41:49.696 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2024-09-16T09:41:49.696 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.696 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.696 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 5, 2024-09-16T09:41:49.696 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2024-09-16T09:41:49.696 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2024-09-16T09:41:49.696 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2024-09-16T09:41:49.696 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540719139924082687, 2024-09-16T09:41:49.696 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2024-09-16T09:41:49.696 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2024-09-16T09:41:29.134973+0000", 2024-09-16T09:41:49.697 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2024-09-16T09:41:49.697 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.697 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.697 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 6, 2024-09-16T09:41:49.697 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2024-09-16T09:41:49.697 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2024-09-16T09:41:49.697 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2024-09-16T09:41:49.697 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540719139924082687, 2024-09-16T09:41:49.697 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2024-09-16T09:41:49.697 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2024-09-16T09:40:59.352436+0000", 2024-09-16T09:41:49.697 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2024-09-16T09:41:49.697 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.697 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.697 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 7, 2024-09-16T09:41:49.698 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2024-09-16T09:41:49.698 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2024-09-16T09:41:49.698 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2024-09-16T09:41:49.698 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540719139924082687, 2024-09-16T09:41:49.698 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2024-09-16T09:41:49.698 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2024-09-16T09:41:27.957285+0000", 2024-09-16T09:41:49.698 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2024-09-16T09:41:49.698 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:41:49.698 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-16T09:41:49.698 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap": [ 2024-09-16T09:41:49.698 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.698 INFO:teuthology.orchestra.run.smithi070.stdout: "pgid": "3.1d", 2024-09-16T09:41:49.699 INFO:teuthology.orchestra.run.smithi070.stdout: "osds": [ 2024-09-16T09:41:49.699 INFO:teuthology.orchestra.run.smithi070.stdout: 1, 2024-09-16T09:41:49.699 INFO:teuthology.orchestra.run.smithi070.stdout: 0, 2024-09-16T09:41:49.699 INFO:teuthology.orchestra.run.smithi070.stdout: 7 2024-09-16T09:41:49.699 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:41:49.699 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.699 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.699 INFO:teuthology.orchestra.run.smithi070.stdout: "pgid": "4.b", 2024-09-16T09:41:49.699 INFO:teuthology.orchestra.run.smithi070.stdout: "osds": [ 2024-09-16T09:41:49.699 INFO:teuthology.orchestra.run.smithi070.stdout: 0, 2024-09-16T09:41:49.699 INFO:teuthology.orchestra.run.smithi070.stdout: 5, 2024-09-16T09:41:49.699 INFO:teuthology.orchestra.run.smithi070.stdout: 1 2024-09-16T09:41:49.700 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:41:49.700 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.700 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.700 INFO:teuthology.orchestra.run.smithi070.stdout: "pgid": "4.19", 2024-09-16T09:41:49.700 INFO:teuthology.orchestra.run.smithi070.stdout: "osds": [ 2024-09-16T09:41:49.700 INFO:teuthology.orchestra.run.smithi070.stdout: 4, 2024-09-16T09:41:49.700 INFO:teuthology.orchestra.run.smithi070.stdout: 7, 2024-09-16T09:41:49.700 INFO:teuthology.orchestra.run.smithi070.stdout: 5 2024-09-16T09:41:49.700 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:41:49.700 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:41:49.700 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-16T09:41:49.700 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap_items": [ 2024-09-16T09:41:49.700 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.701 INFO:teuthology.orchestra.run.smithi070.stdout: "pgid": "2.1", 2024-09-16T09:41:49.701 INFO:teuthology.orchestra.run.smithi070.stdout: "mappings": [ 2024-09-16T09:41:49.701 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.701 INFO:teuthology.orchestra.run.smithi070.stdout: "from": 3, 2024-09-16T09:41:49.701 INFO:teuthology.orchestra.run.smithi070.stdout: "to": 0 2024-09-16T09:41:49.701 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.701 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.701 INFO:teuthology.orchestra.run.smithi070.stdout: "from": 2, 2024-09-16T09:41:49.701 INFO:teuthology.orchestra.run.smithi070.stdout: "to": 1 2024-09-16T09:41:49.701 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:41:49.701 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:41:49.701 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.701 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.702 INFO:teuthology.orchestra.run.smithi070.stdout: "pgid": "3.1b", 2024-09-16T09:41:49.702 INFO:teuthology.orchestra.run.smithi070.stdout: "mappings": [ 2024-09-16T09:41:49.702 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.702 INFO:teuthology.orchestra.run.smithi070.stdout: "from": 0, 2024-09-16T09:41:49.702 INFO:teuthology.orchestra.run.smithi070.stdout: "to": 6 2024-09-16T09:41:49.702 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.702 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.702 INFO:teuthology.orchestra.run.smithi070.stdout: "from": 4, 2024-09-16T09:41:49.702 INFO:teuthology.orchestra.run.smithi070.stdout: "to": 7 2024-09-16T09:41:49.702 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:41:49.702 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:41:49.702 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.702 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.702 INFO:teuthology.orchestra.run.smithi070.stdout: "pgid": "4.f", 2024-09-16T09:41:49.703 INFO:teuthology.orchestra.run.smithi070.stdout: "mappings": [ 2024-09-16T09:41:49.703 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.703 INFO:teuthology.orchestra.run.smithi070.stdout: "from": 1, 2024-09-16T09:41:49.703 INFO:teuthology.orchestra.run.smithi070.stdout: "to": 2 2024-09-16T09:41:49.703 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:41:49.703 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:41:49.703 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.703 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.703 INFO:teuthology.orchestra.run.smithi070.stdout: "pgid": "4.13", 2024-09-16T09:41:49.703 INFO:teuthology.orchestra.run.smithi070.stdout: "mappings": [ 2024-09-16T09:41:49.703 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.703 INFO:teuthology.orchestra.run.smithi070.stdout: "from": 1, 2024-09-16T09:41:49.703 INFO:teuthology.orchestra.run.smithi070.stdout: "to": 5 2024-09-16T09:41:49.703 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:41:49.703 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:41:49.704 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.704 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.704 INFO:teuthology.orchestra.run.smithi070.stdout: "pgid": "4.17", 2024-09-16T09:41:49.704 INFO:teuthology.orchestra.run.smithi070.stdout: "mappings": [ 2024-09-16T09:41:49.704 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.704 INFO:teuthology.orchestra.run.smithi070.stdout: "from": 1, 2024-09-16T09:41:49.704 INFO:teuthology.orchestra.run.smithi070.stdout: "to": 2 2024-09-16T09:41:49.704 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:41:49.704 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:41:49.704 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.704 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.704 INFO:teuthology.orchestra.run.smithi070.stdout: "pgid": "5.10", 2024-09-16T09:41:49.704 INFO:teuthology.orchestra.run.smithi070.stdout: "mappings": [ 2024-09-16T09:41:49.704 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:49.705 INFO:teuthology.orchestra.run.smithi070.stdout: "from": 7, 2024-09-16T09:41:49.705 INFO:teuthology.orchestra.run.smithi070.stdout: "to": 4 2024-09-16T09:41:49.705 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:41:49.705 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:41:49.705 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:41:49.705 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-16T09:41:49.705 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap_primaries": [], 2024-09-16T09:41:49.705 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_temp": [], 2024-09-16T09:41:49.705 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_temp": [], 2024-09-16T09:41:49.705 INFO:teuthology.orchestra.run.smithi070.stdout: "blocklist": { 2024-09-16T09:41:49.705 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/125491368": "2024-09-17T09:25:31.465167+0000", 2024-09-16T09:41:49.705 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/3354284852": "2024-09-17T09:19:43.205989+0000", 2024-09-16T09:41:49.705 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/3284625547": "2024-09-17T09:25:31.465167+0000", 2024-09-16T09:41:49.705 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/186565817": "2024-09-17T09:19:43.205989+0000", 2024-09-16T09:41:49.705 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/2213852865": "2024-09-17T09:19:43.205989+0000", 2024-09-16T09:41:49.706 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/1812373760": "2024-09-17T09:20:09.306330+0000", 2024-09-16T09:41:49.706 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:6800/837338094": "2024-09-17T09:19:43.205989+0000", 2024-09-16T09:41:49.706 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:6800/1876661229": "2024-09-17T09:25:31.465167+0000", 2024-09-16T09:41:49.706 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/3351688485": "2024-09-17T09:20:09.306330+0000", 2024-09-16T09:41:49.706 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:6800/3871553358": "2024-09-17T09:20:09.306330+0000", 2024-09-16T09:41:49.706 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/1230482871": "2024-09-17T09:25:31.465167+0000", 2024-09-16T09:41:49.706 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:6801/837338094": "2024-09-17T09:19:43.205989+0000", 2024-09-16T09:41:49.706 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:6801/3871553358": "2024-09-17T09:20:09.306330+0000", 2024-09-16T09:41:49.706 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/1403001486": "2024-09-17T09:20:09.306330+0000", 2024-09-16T09:41:49.706 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/1723643739": "2024-09-17T09:25:31.465167+0000", 2024-09-16T09:41:49.706 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:6801/1876661229": "2024-09-17T09:25:31.465167+0000", 2024-09-16T09:41:49.706 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/3456417396": "2024-09-17T09:25:31.465167+0000" 2024-09-16T09:41:49.706 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.706 INFO:teuthology.orchestra.run.smithi070.stdout: "range_blocklist": {}, 2024-09-16T09:41:49.706 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profiles": { 2024-09-16T09:41:49.707 INFO:teuthology.orchestra.run.smithi070.stdout: "default": { 2024-09-16T09:41:49.707 INFO:teuthology.orchestra.run.smithi070.stdout: "crush-failure-domain": "osd", 2024-09-16T09:41:49.707 INFO:teuthology.orchestra.run.smithi070.stdout: "k": "2", 2024-09-16T09:41:49.707 INFO:teuthology.orchestra.run.smithi070.stdout: "m": "1", 2024-09-16T09:41:49.707 INFO:teuthology.orchestra.run.smithi070.stdout: "plugin": "jerasure", 2024-09-16T09:41:49.707 INFO:teuthology.orchestra.run.smithi070.stdout: "technique": "reed_sol_van" 2024-09-16T09:41:49.707 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:41:49.707 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:49.707 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps_queue": [], 2024-09-16T09:41:49.707 INFO:teuthology.orchestra.run.smithi070.stdout: "new_removed_snaps": [], 2024-09-16T09:41:49.707 INFO:teuthology.orchestra.run.smithi070.stdout: "new_purged_snaps": [], 2024-09-16T09:41:49.707 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_node_flags": {}, 2024-09-16T09:41:49.707 INFO:teuthology.orchestra.run.smithi070.stdout: "device_class_flags": {}, 2024-09-16T09:41:49.707 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode": { 2024-09-16T09:41:49.707 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode_enabled": false, 2024-09-16T09:41:49.708 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_bucket_count": 0, 2024-09-16T09:41:49.708 INFO:teuthology.orchestra.run.smithi070.stdout: "degraded_stretch_mode": 0, 2024-09-16T09:41:49.708 INFO:teuthology.orchestra.run.smithi070.stdout: "recovering_stretch_mode": 0, 2024-09-16T09:41:49.708 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode_bucket": 0 2024-09-16T09:41:49.708 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:41:49.708 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T09:41:49.708 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:41:49.710 INFO:teuthology.orchestra.run.smithi070.stderr:noscrub is set 2024-09-16T09:41:49.738 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:49.738 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:49.738 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:41:49.839 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:41:49.869 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:49.870 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:49.870 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:41:49.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:49 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:41:49.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:49 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:41:49.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:49 smithi070 ceph-mon[30388]: 6.d scrub starts 2024-09-16T09:41:49.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:49 smithi070 ceph-mon[30388]: 6.d scrub ok 2024-09-16T09:41:49.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:49 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2328571211' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:41:49.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:49 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:41:49.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:49 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1495700925' entity='client.admin' cmd={"prefix": "osd dump", "format": "json-pretty"} : dispatch 2024-09-16T09:41:49.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:49 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:41:49.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:49 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:41:49.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:49 smithi070 ceph-mon[40691]: 6.d scrub starts 2024-09-16T09:41:49.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:49 smithi070 ceph-mon[40691]: 6.d scrub ok 2024-09-16T09:41:49.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:49 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/2328571211' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:41:49.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:49 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:41:49.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:49 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1495700925' entity='client.admin' cmd={"prefix": "osd dump", "format": "json-pretty"} : dispatch 2024-09-16T09:41:49.971 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:41:50.006 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:50.006 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:50.007 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:41:50.093 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 30 16 60413 60397 125.807 84.75 0.00993124 0.00794319 2024-09-16T09:41:50.108 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:41:50.143 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:50.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:50.145 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:41:50.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:49 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:41:50.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:49 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:41:50.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:49 smithi084 ceph-mon[32629]: 6.d scrub starts 2024-09-16T09:41:50.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:49 smithi084 ceph-mon[32629]: 6.d scrub ok 2024-09-16T09:41:50.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:49 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/2328571211' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:41:50.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:49 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:41:50.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:49 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1495700925' entity='client.admin' cmd={"prefix": "osd dump", "format": "json-pretty"} : dispatch 2024-09-16T09:41:50.167 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T09:41:50.246 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:41:50.288 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:50.289 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:50.289 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:41:50.324 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T09:41:50.324 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 20, 2024-09-16T09:41:50.324 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 600, 2024-09-16T09:41:50.324 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [ 2024-09-16T09:41:50.324 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.324 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:232736 9.12 9:4ea007d7:::benchmark_data_smithi070_137000_object111025:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e360)", 2024-09-16T09:41:50.324 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:37:21.520060+0000", 2024-09-16T09:41:50.324 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 268.79437143799998, 2024-09-16T09:41:50.324 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0959157930000001, 2024-09-16T09:41:50.324 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:41:50.324 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:41:50.325 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:41:50.325 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:41:50.325 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:41:50.325 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:41:50.325 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 232736 2024-09-16T09:41:50.325 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.325 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:41:50.325 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.325 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:41:50.325 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.520060+0000", 2024-09-16T09:41:50.325 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:41:50.325 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.325 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.326 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:41:50.326 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.520060+0000", 2024-09-16T09:41:50.326 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:41:50.326 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.326 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.326 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:41:50.326 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.520061+0000", 2024-09-16T09:41:50.326 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.35e-06 2024-09-16T09:41:50.326 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.326 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.326 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:41:50.326 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.520065+0000", 2024-09-16T09:41:50.326 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2980000000000002e-06 2024-09-16T09:41:50.326 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.327 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.327 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:41:50.327 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.520077+0000", 2024-09-16T09:41:50.327 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2082999999999999e-05 2024-09-16T09:41:50.327 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.327 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.327 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:41:50.327 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.520087+0000", 2024-09-16T09:41:50.327 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.006e-05 2024-09-16T09:41:50.327 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.327 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.327 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:41:50.327 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.521031+0000", 2024-09-16T09:41:50.328 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00094380400000000002 2024-09-16T09:41:50.328 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.328 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.328 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op must wait for map", 2024-09-16T09:41:50.328 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.521038+0000", 2024-09-16T09:41:50.328 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.1359999999999999e-06 2024-09-16T09:41:50.328 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.328 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.328 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:41:50.328 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.527328+0000", 2024-09-16T09:41:50.328 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.006289195 2024-09-16T09:41:50.328 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.328 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.328 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:41:50.329 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.527338+0000", 2024-09-16T09:41:50.329 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0360000000000001e-05 2024-09-16T09:41:50.329 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.329 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.329 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:41:50.329 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.611637+0000", 2024-09-16T09:41:50.329 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0842987449999999 2024-09-16T09:41:50.329 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.329 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.329 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:41:50.330 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.611765+0000", 2024-09-16T09:41:50.330 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00012811699999999999 2024-09-16T09:41:50.330 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.330 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.330 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 0,2,3,5", 2024-09-16T09:41:50.330 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.611893+0000", 2024-09-16T09:41:50.330 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000128171 2024-09-16T09:41:50.330 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.330 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.330 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:41:50.330 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.615315+0000", 2024-09-16T09:41:50.330 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0034216429999999998 2024-09-16T09:41:50.330 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.331 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.331 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:41:50.331 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.615329+0000", 2024-09-16T09:41:50.331 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.4405e-05 2024-09-16T09:41:50.331 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.331 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.331 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:41:50.331 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.615364+0000", 2024-09-16T09:41:50.331 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5286000000000001e-05 2024-09-16T09:41:50.331 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.331 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.331 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:41:50.331 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.615412+0000", 2024-09-16T09:41:50.331 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.7367000000000002e-05 2024-09-16T09:41:50.331 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.332 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.332 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:41:50.332 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.615802+0000", 2024-09-16T09:41:50.332 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00039052599999999998 2024-09-16T09:41:50.332 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.332 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.332 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:41:50.332 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.615859+0000", 2024-09-16T09:41:50.332 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.6919000000000002e-05 2024-09-16T09:41:50.332 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.332 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.332 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:41:50.332 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.615975+0000", 2024-09-16T09:41:50.332 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000116328 2024-09-16T09:41:50.333 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:41:50.333 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:41:50.333 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:41:50.333 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.333 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.333 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:232744 9.16 9:6db6f31f:::benchmark_data_smithi070_137000_object111033:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e360)", 2024-09-16T09:41:50.333 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:37:21.520102+0000", 2024-09-16T09:41:50.333 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 268.794328871, 2024-09-16T09:41:50.333 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.092603174, 2024-09-16T09:41:50.333 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:41:50.333 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:41:50.333 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:41:50.333 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:41:50.333 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:41:50.334 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:41:50.334 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 232744 2024-09-16T09:41:50.334 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.334 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:41:50.334 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.334 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:41:50.334 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.520102+0000", 2024-09-16T09:41:50.334 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:41:50.334 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.334 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.334 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:41:50.334 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.520102+0000", 2024-09-16T09:41:50.334 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:41:50.334 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.334 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.335 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:41:50.335 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.520103+0000", 2024-09-16T09:41:50.335 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.1399999999999997e-07 2024-09-16T09:41:50.335 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.335 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.335 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:41:50.335 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.520105+0000", 2024-09-16T09:41:50.335 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3580000000000001e-06 2024-09-16T09:41:50.335 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.335 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.335 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:41:50.335 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.520111+0000", 2024-09-16T09:41:50.335 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.0990000000000004e-06 2024-09-16T09:41:50.335 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.335 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.336 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:41:50.336 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.520116+0000", 2024-09-16T09:41:50.336 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.2270000000000001e-06 2024-09-16T09:41:50.336 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.336 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.336 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:41:50.336 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.521263+0000", 2024-09-16T09:41:50.336 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0011466530000000001 2024-09-16T09:41:50.336 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.336 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.336 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op must wait for map", 2024-09-16T09:41:50.336 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.521269+0000", 2024-09-16T09:41:50.336 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1489999999999999e-06 2024-09-16T09:41:50.336 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.336 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.337 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:41:50.337 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.527938+0000", 2024-09-16T09:41:50.337 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0066689130000000003 2024-09-16T09:41:50.337 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.337 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.337 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:41:50.337 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.527947+0000", 2024-09-16T09:41:50.337 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.816e-06 2024-09-16T09:41:50.337 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.337 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.337 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:41:50.337 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.600757+0000", 2024-09-16T09:41:50.337 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0728101139999999 2024-09-16T09:41:50.337 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.338 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.338 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:41:50.338 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.600898+0000", 2024-09-16T09:41:50.338 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00014071799999999999 2024-09-16T09:41:50.338 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.338 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.338 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 2,4,5", 2024-09-16T09:41:50.338 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.601036+0000", 2024-09-16T09:41:50.338 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00013785100000000001 2024-09-16T09:41:50.338 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.338 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.339 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:41:50.340 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.602183+0000", 2024-09-16T09:41:50.340 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001147742 2024-09-16T09:41:50.340 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.340 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.340 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:41:50.340 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.602237+0000", 2024-09-16T09:41:50.340 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.3905e-05 2024-09-16T09:41:50.340 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.340 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.340 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:41:50.340 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.612452+0000", 2024-09-16T09:41:50.340 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.010215033 2024-09-16T09:41:50.340 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.341 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.341 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:41:50.341 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.612498+0000", 2024-09-16T09:41:50.341 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5612000000000003e-05 2024-09-16T09:41:50.341 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.341 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.341 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:41:50.341 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.612599+0000", 2024-09-16T09:41:50.341 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00010118700000000001 2024-09-16T09:41:50.341 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.341 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.341 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:41:50.342 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.612705+0000", 2024-09-16T09:41:50.342 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000106383 2024-09-16T09:41:50.342 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:41:50.342 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:41:50.342 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:41:50.342 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.342 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.342 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:232750 9.10 9:08f1a82d:::benchmark_data_smithi070_137000_object111039:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e360)", 2024-09-16T09:41:50.342 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:37:21.520133+0000", 2024-09-16T09:41:50.342 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 268.79429764700001, 2024-09-16T09:41:50.342 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1158816490000001, 2024-09-16T09:41:50.342 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:41:50.342 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:41:50.342 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:41:50.342 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:41:50.343 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:41:50.343 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:41:50.343 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 232750 2024-09-16T09:41:50.343 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.343 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:41:50.343 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.343 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:41:50.343 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.520133+0000", 2024-09-16T09:41:50.343 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:41:50.343 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.343 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.343 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:41:50.343 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.520133+0000", 2024-09-16T09:41:50.344 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:41:50.344 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.344 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.344 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:41:50.344 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.520134+0000", 2024-09-16T09:41:50.344 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.4499999999999997e-07 2024-09-16T09:41:50.344 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.344 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.344 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:41:50.344 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.520136+0000", 2024-09-16T09:41:50.344 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.1550000000000001e-06 2024-09-16T09:41:50.344 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.344 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.345 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:41:50.345 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.520143+0000", 2024-09-16T09:41:50.345 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.5579999999999997e-06 2024-09-16T09:41:50.345 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.345 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.345 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:41:50.345 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.520147+0000", 2024-09-16T09:41:50.345 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.9089999999999999e-06 2024-09-16T09:41:50.345 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.345 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.345 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:41:50.345 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.520793+0000", 2024-09-16T09:41:50.345 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00064582400000000003 2024-09-16T09:41:50.346 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.346 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.346 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op must wait for map", 2024-09-16T09:41:50.346 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.520802+0000", 2024-09-16T09:41:50.346 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.7579999999999998e-06 2024-09-16T09:41:50.346 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.346 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.346 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:41:50.346 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.527074+0000", 2024-09-16T09:41:50.346 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0062723149999999997 2024-09-16T09:41:50.346 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.346 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.346 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:41:50.347 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.527086+0000", 2024-09-16T09:41:50.347 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1206000000000001e-05 2024-09-16T09:41:50.347 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.347 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.347 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:41:50.347 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.618695+0000", 2024-09-16T09:41:50.347 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.091609826 2024-09-16T09:41:50.347 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.347 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.347 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:41:50.347 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.618780+0000", 2024-09-16T09:41:50.347 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.4135000000000007e-05 2024-09-16T09:41:50.347 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.348 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.348 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 0,2,4,6,7", 2024-09-16T09:41:50.348 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.618857+0000", 2024-09-16T09:41:50.348 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.7638000000000004e-05 2024-09-16T09:41:50.348 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.348 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.348 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:41:50.348 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.620139+0000", 2024-09-16T09:41:50.348 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0012817919999999999 2024-09-16T09:41:50.348 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.348 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.348 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:41:50.348 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.620277+0000", 2024-09-16T09:41:50.348 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000138467 2024-09-16T09:41:50.349 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.349 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.349 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:41:50.349 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.620434+0000", 2024-09-16T09:41:50.349 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00015646200000000001 2024-09-16T09:41:50.349 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.349 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.349 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:41:50.349 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.624194+0000", 2024-09-16T09:41:50.349 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0037599030000000002 2024-09-16T09:41:50.349 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.349 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.349 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:41:50.349 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.624290+0000", 2024-09-16T09:41:50.350 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.6267000000000005e-05 2024-09-16T09:41:50.350 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.350 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.350 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:41:50.350 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.635805+0000", 2024-09-16T09:41:50.350 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.011515107 2024-09-16T09:41:50.350 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.350 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.350 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:41:50.350 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.635870+0000", 2024-09-16T09:41:50.350 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.4678999999999995e-05 2024-09-16T09:41:50.350 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.350 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.350 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:41:50.351 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.636015+0000", 2024-09-16T09:41:50.351 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000145203 2024-09-16T09:41:50.351 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:41:50.351 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:41:50.351 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:41:50.351 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.351 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.351 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:232758 9.12 9:4a9f0b61:::benchmark_data_smithi070_137000_object111047:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e360)", 2024-09-16T09:41:50.351 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:37:21.520161+0000", 2024-09-16T09:41:50.351 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 268.79426972099998, 2024-09-16T09:41:50.351 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.096396232, 2024-09-16T09:41:50.351 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:41:50.351 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:41:50.352 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:41:50.352 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:41:50.352 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:41:50.352 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:41:50.352 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 232758 2024-09-16T09:41:50.352 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.352 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:41:50.352 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.352 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:41:50.352 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.520161+0000", 2024-09-16T09:41:50.352 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:41:50.352 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.352 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.352 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:41:50.353 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.520161+0000", 2024-09-16T09:41:50.353 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:41:50.353 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.353 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.353 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:41:50.353 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.520162+0000", 2024-09-16T09:41:50.353 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6800000000000001e-07 2024-09-16T09:41:50.353 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.353 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.353 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:41:50.353 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.520164+0000", 2024-09-16T09:41:50.353 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.951e-06 2024-09-16T09:41:50.353 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.354 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.354 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:41:50.354 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.520170+0000", 2024-09-16T09:41:50.354 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1e-06 2024-09-16T09:41:50.354 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.354 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.354 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:41:50.354 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.520175+0000", 2024-09-16T09:41:50.354 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.8729999999999998e-06 2024-09-16T09:41:50.354 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.354 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.354 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:41:50.354 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.521063+0000", 2024-09-16T09:41:50.355 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000888103 2024-09-16T09:41:50.355 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.355 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.355 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting_for_map not empty", 2024-09-16T09:41:50.355 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.521068+0000", 2024-09-16T09:41:50.355 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5809999999999996e-06 2024-09-16T09:41:50.355 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.355 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.355 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:41:50.355 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.536366+0000", 2024-09-16T09:41:50.355 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.015297471 2024-09-16T09:41:50.355 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.355 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.356 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:41:50.356 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.536395+0000", 2024-09-16T09:41:50.356 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.9260000000000001e-05 2024-09-16T09:41:50.356 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.356 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.356 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:41:50.356 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.614279+0000", 2024-09-16T09:41:50.356 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0778841029999999 2024-09-16T09:41:50.356 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.356 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.356 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:41:50.356 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.614593+0000", 2024-09-16T09:41:50.356 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00031368100000000001 2024-09-16T09:41:50.356 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.357 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.357 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 0,2,3,5", 2024-09-16T09:41:50.357 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.614716+0000", 2024-09-16T09:41:50.357 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00012332900000000001 2024-09-16T09:41:50.357 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.357 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.357 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:41:50.357 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.616163+0000", 2024-09-16T09:41:50.357 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0014472280000000001 2024-09-16T09:41:50.357 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.357 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.357 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:41:50.358 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.616255+0000", 2024-09-16T09:41:50.358 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.1528999999999997e-05 2024-09-16T09:41:50.358 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.358 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.358 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:41:50.358 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.616272+0000", 2024-09-16T09:41:50.358 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7098000000000001e-05 2024-09-16T09:41:50.358 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.358 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.358 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:41:50.358 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.616306+0000", 2024-09-16T09:41:50.358 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3847999999999998e-05 2024-09-16T09:41:50.358 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.358 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.359 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:41:50.359 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.616346+0000", 2024-09-16T09:41:50.359 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0528999999999998e-05 2024-09-16T09:41:50.359 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.359 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.359 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:41:50.359 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.616461+0000", 2024-09-16T09:41:50.359 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000114941 2024-09-16T09:41:50.359 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.359 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.359 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:41:50.359 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.616558+0000", 2024-09-16T09:41:50.359 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.6238999999999998e-05 2024-09-16T09:41:50.359 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:41:50.360 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:41:50.360 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:41:50.360 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.360 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.360 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:232762 9.10 9:0c9cd3ad:::benchmark_data_smithi070_137000_object111051:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e360)", 2024-09-16T09:41:50.360 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:37:21.520181+0000", 2024-09-16T09:41:50.360 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 268.79425036399999, 2024-09-16T09:41:50.360 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1160747520000001, 2024-09-16T09:41:50.360 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:41:50.360 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:41:50.360 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:41:50.360 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:41:50.360 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:41:50.360 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:41:50.361 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 232762 2024-09-16T09:41:50.361 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.361 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:41:50.361 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.361 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:41:50.361 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.520181+0000", 2024-09-16T09:41:50.361 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:41:50.361 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.361 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.361 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:41:50.361 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.520181+0000", 2024-09-16T09:41:50.361 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:41:50.361 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.362 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.362 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:41:50.362 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.520181+0000", 2024-09-16T09:41:50.362 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.0600000000000001e-07 2024-09-16T09:41:50.362 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.362 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.362 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:41:50.362 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.520183+0000", 2024-09-16T09:41:50.362 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.784e-06 2024-09-16T09:41:50.362 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.362 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.362 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:41:50.362 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.520188+0000", 2024-09-16T09:41:50.362 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5790000000000003e-06 2024-09-16T09:41:50.363 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.363 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.363 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:41:50.363 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.520193+0000", 2024-09-16T09:41:50.363 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.6460000000000001e-06 2024-09-16T09:41:50.363 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.363 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.363 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:41:50.363 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.520853+0000", 2024-09-16T09:41:50.363 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000660331 2024-09-16T09:41:50.363 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.363 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.363 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting_for_map not empty", 2024-09-16T09:41:50.363 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.520859+0000", 2024-09-16T09:41:50.364 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8289999999999996e-06 2024-09-16T09:41:50.364 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.364 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.364 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:41:50.364 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.527156+0000", 2024-09-16T09:41:50.364 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0062966719999999997 2024-09-16T09:41:50.364 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.364 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.364 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:41:50.364 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:21.527164+0000", 2024-09-16T09:41:50.364 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.2630000000000003e-06 2024-09-16T09:41:50.364 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.364 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.364 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:41:50.365 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.619488+0000", 2024-09-16T09:41:50.365 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.092323962 2024-09-16T09:41:50.365 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.365 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.365 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:41:50.365 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.619561+0000", 2024-09-16T09:41:50.365 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3312999999999994e-05 2024-09-16T09:41:50.365 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.365 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.366 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 0,2,4,6,7", 2024-09-16T09:41:50.366 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.619626+0000", 2024-09-16T09:41:50.366 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.4437999999999995e-05 2024-09-16T09:41:50.366 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.366 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.366 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:41:50.366 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.620761+0000", 2024-09-16T09:41:50.366 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0011348090000000001 2024-09-16T09:41:50.366 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.366 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.366 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:41:50.366 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.620774+0000", 2024-09-16T09:41:50.366 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2942000000000001e-05 2024-09-16T09:41:50.367 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.367 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.367 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:41:50.367 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.620804+0000", 2024-09-16T09:41:50.367 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.0054999999999999e-05 2024-09-16T09:41:50.367 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.367 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.368 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:41:50.368 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.624333+0000", 2024-09-16T09:41:50.368 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0035295159999999999 2024-09-16T09:41:50.368 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.368 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.368 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:41:50.368 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.635618+0000", 2024-09-16T09:41:50.368 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.011284591 2024-09-16T09:41:50.368 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.368 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.368 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:41:50.369 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.636059+0000", 2024-09-16T09:41:50.369 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00044175699999999998 2024-09-16T09:41:50.369 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.369 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.369 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:41:50.369 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.636118+0000", 2024-09-16T09:41:50.369 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8570000000000003e-05 2024-09-16T09:41:50.369 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.369 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.369 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:41:50.369 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:22.636255+0000", 2024-09-16T09:41:50.369 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000137389 2024-09-16T09:41:50.369 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:41:50.369 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:41:50.369 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:41:50.370 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.370 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.370 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.15975.0:113589 10.19 e417/398 10:9a90b735:::benchmark_data_smithi070_166744_object113588:head v 417'5779, mlcod=383'128)", 2024-09-16T09:41:50.370 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:39:25.243339+0000", 2024-09-16T09:41:50.370 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 145.07109233700001, 2024-09-16T09:41:50.370 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.25732612500000002, 2024-09-16T09:41:50.370 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:41:50.370 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:41:50.370 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:41:50.370 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:41:50.370 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.370 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:41:50.370 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.243339+0000", 2024-09-16T09:41:50.370 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:41:50.371 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.371 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.371 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:41:50.371 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.243339+0000", 2024-09-16T09:41:50.371 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:41:50.371 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.371 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.371 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:41:50.371 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.243339+0000", 2024-09-16T09:41:50.371 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.8400000000000004e-07 2024-09-16T09:41:50.371 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.371 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.371 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:41:50.371 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.243362+0000", 2024-09-16T09:41:50.372 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3e-05 2024-09-16T09:41:50.372 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.372 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.372 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:41:50.372 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.243379+0000", 2024-09-16T09:41:50.372 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6282000000000001e-05 2024-09-16T09:41:50.372 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.372 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.372 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:41:50.372 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.243385+0000", 2024-09-16T09:41:50.372 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1310000000000003e-06 2024-09-16T09:41:50.372 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.372 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.372 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:41:50.373 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.243579+0000", 2024-09-16T09:41:50.373 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00019384099999999999 2024-09-16T09:41:50.373 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.373 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.373 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:41:50.373 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.243601+0000", 2024-09-16T09:41:50.373 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.2269999999999999e-05 2024-09-16T09:41:50.373 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.373 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.373 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:41:50.373 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.249392+0000", 2024-09-16T09:41:50.373 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0057910590000000003 2024-09-16T09:41:50.373 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.373 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.373 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:41:50.373 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.500665+0000", 2024-09-16T09:41:50.374 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.25127285799999999 2024-09-16T09:41:50.374 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:41:50.374 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:41:50.374 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:41:50.374 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.374 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.374 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.15975.0:113599 10.13 e417/416 10:cf531bcf:::benchmark_data_smithi070_166744_object113598:head v 417'5970, mlcod=417'5966)", 2024-09-16T09:41:50.374 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:39:25.246192+0000", 2024-09-16T09:41:50.374 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 145.068238837, 2024-09-16T09:41:50.374 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.25445351799999999, 2024-09-16T09:41:50.374 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:41:50.374 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:41:50.374 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:41:50.374 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:41:50.375 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.375 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:41:50.375 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.246192+0000", 2024-09-16T09:41:50.375 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:41:50.375 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.375 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.375 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:41:50.375 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.246192+0000", 2024-09-16T09:41:50.375 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:41:50.375 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.375 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.375 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:41:50.375 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.246193+0000", 2024-09-16T09:41:50.375 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0750000000000001e-06 2024-09-16T09:41:50.376 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.376 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.376 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:41:50.376 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.246241+0000", 2024-09-16T09:41:50.376 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.7435000000000002e-05 2024-09-16T09:41:50.376 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.376 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.376 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:41:50.376 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.246253+0000", 2024-09-16T09:41:50.376 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1878999999999999e-05 2024-09-16T09:41:50.376 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.376 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.376 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:41:50.376 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.246261+0000", 2024-09-16T09:41:50.377 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.4239999999999993e-06 2024-09-16T09:41:50.377 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.377 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.377 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:41:50.377 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.246609+0000", 2024-09-16T09:41:50.377 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00034822999999999999 2024-09-16T09:41:50.377 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.377 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.377 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:41:50.377 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.246626+0000", 2024-09-16T09:41:50.377 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7229999999999999e-05 2024-09-16T09:41:50.377 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.377 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.377 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:41:50.377 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.500400+0000", 2024-09-16T09:41:50.378 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.253773521 2024-09-16T09:41:50.378 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.378 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.378 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:41:50.378 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.500646+0000", 2024-09-16T09:41:50.378 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00024572400000000001 2024-09-16T09:41:50.378 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:41:50.378 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:41:50.378 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:41:50.378 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.378 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.378 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.15975.0:113601 10.13 e417/416 10:ccd2f904:::benchmark_data_smithi070_166744_object113600:head v 417'5971, mlcod=417'5966)", 2024-09-16T09:41:50.378 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:39:25.248424+0000", 2024-09-16T09:41:50.378 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 145.06600675300001, 2024-09-16T09:41:50.378 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.25528167600000001, 2024-09-16T09:41:50.379 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:41:50.379 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:41:50.379 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:41:50.379 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:41:50.379 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.379 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:41:50.379 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.248424+0000", 2024-09-16T09:41:50.379 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:41:50.379 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.379 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.379 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:41:50.379 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.248424+0000", 2024-09-16T09:41:50.379 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:41:50.379 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.380 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.380 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:41:50.380 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.248425+0000", 2024-09-16T09:41:50.380 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.0100000000000003e-07 2024-09-16T09:41:50.380 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.380 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.380 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:41:50.380 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.248458+0000", 2024-09-16T09:41:50.380 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3151000000000002e-05 2024-09-16T09:41:50.380 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.380 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.380 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:41:50.380 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.248470+0000", 2024-09-16T09:41:50.380 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1456e-05 2024-09-16T09:41:50.380 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.380 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.381 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:41:50.381 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.248477+0000", 2024-09-16T09:41:50.381 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.5959999999999998e-06 2024-09-16T09:41:50.381 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.381 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.381 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:41:50.381 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.248676+0000", 2024-09-16T09:41:50.381 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000198984 2024-09-16T09:41:50.381 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.381 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.381 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:41:50.381 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.248716+0000", 2024-09-16T09:41:50.381 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9762999999999999e-05 2024-09-16T09:41:50.382 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.382 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.382 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:41:50.382 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.503649+0000", 2024-09-16T09:41:50.382 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.254932942 2024-09-16T09:41:50.382 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.382 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.382 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:41:50.382 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.503706+0000", 2024-09-16T09:41:50.382 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.6882999999999997e-05 2024-09-16T09:41:50.382 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:41:50.382 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:41:50.382 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:41:50.383 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.383 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.383 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.15975.0:113603 10.16 e417/398 10:6f5b7bce:::benchmark_data_smithi070_166744_object113602:head v 417'5683, mlcod=391'4079)", 2024-09-16T09:41:50.383 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:39:25.248991+0000", 2024-09-16T09:41:50.383 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 145.06543975700001, 2024-09-16T09:41:50.383 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.25780223800000002, 2024-09-16T09:41:50.383 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:41:50.383 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:41:50.383 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:41:50.383 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:41:50.383 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.383 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:41:50.383 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.248991+0000", 2024-09-16T09:41:50.383 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:41:50.383 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.384 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.384 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:41:50.384 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.248991+0000", 2024-09-16T09:41:50.384 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:41:50.384 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.384 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.384 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:41:50.384 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.248992+0000", 2024-09-16T09:41:50.384 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.9899999999999999e-07 2024-09-16T09:41:50.384 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.384 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.384 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:41:50.384 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.249017+0000", 2024-09-16T09:41:50.384 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.4332999999999999e-05 2024-09-16T09:41:50.384 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.385 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.385 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:41:50.385 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.249028+0000", 2024-09-16T09:41:50.385 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1734e-05 2024-09-16T09:41:50.385 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.385 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.385 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:41:50.385 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.249036+0000", 2024-09-16T09:41:50.385 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.9100000000000005e-06 2024-09-16T09:41:50.385 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.385 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.385 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:41:50.385 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.249112+0000", 2024-09-16T09:41:50.385 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.5683e-05 2024-09-16T09:41:50.386 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.386 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.386 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:41:50.386 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.249134+0000", 2024-09-16T09:41:50.386 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.2365999999999999e-05 2024-09-16T09:41:50.386 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.386 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.386 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:41:50.386 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.506749+0000", 2024-09-16T09:41:50.386 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.25761445300000002 2024-09-16T09:41:50.386 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.386 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.386 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:41:50.387 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.506794+0000", 2024-09-16T09:41:50.387 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.4860000000000001e-05 2024-09-16T09:41:50.387 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:41:50.387 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:41:50.387 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:41:50.387 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.387 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.387 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "MOSDPGPush(10.15 417/398 [PushOp(10:a9f4afe2:::benchmark_data_smithi070_166744_object2856:head, version: 383'171, data_included: [0~65536], data_size: 65536, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(10:a9f4afe2:::benchmark_data_smithi070_166744_object2856:head@383'171, size: 65536, copy_subset: [(0, 65536)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 65536, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-16T09:41:50.387 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:39:25.249003+0000", 2024-09-16T09:41:50.387 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 145.065428234, 2024-09-16T09:41:50.387 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.25139647799999998, 2024-09-16T09:41:50.387 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:41:50.387 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:41:50.387 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T09:41:50.388 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:41:50.388 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.388 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:41:50.388 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.249003+0000", 2024-09-16T09:41:50.388 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:41:50.388 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.388 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.388 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:41:50.388 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.249003+0000", 2024-09-16T09:41:50.388 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:41:50.388 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.388 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.388 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:41:50.388 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.249003+0000", 2024-09-16T09:41:50.389 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.68e-07 2024-09-16T09:41:50.389 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.389 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.389 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:41:50.389 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.249044+0000", 2024-09-16T09:41:50.389 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0290000000000002e-05 2024-09-16T09:41:50.389 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.389 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.389 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:41:50.389 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.249062+0000", 2024-09-16T09:41:50.389 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.8496e-05 2024-09-16T09:41:50.389 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.389 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.389 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:41:50.389 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.249079+0000", 2024-09-16T09:41:50.390 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6308e-05 2024-09-16T09:41:50.390 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.390 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.390 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:41:50.390 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.249194+0000", 2024-09-16T09:41:50.390 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000115813 2024-09-16T09:41:50.390 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.390 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.390 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:41:50.390 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.249207+0000", 2024-09-16T09:41:50.390 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2986000000000001e-05 2024-09-16T09:41:50.390 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.390 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.390 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:41:50.390 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.500399+0000", 2024-09-16T09:41:50.391 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.25119191699999999 2024-09-16T09:41:50.391 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:41:50.391 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:41:50.391 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:41:50.391 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.391 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.391 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "MOSDPGPush(10.16 417/398 [PushOp(10:6981da6b:::benchmark_data_smithi070_166744_object66882:head, version: 391'4085, data_included: [0~65536], data_size: 65536, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(10:6981da6b:::benchmark_data_smithi070_166744_object66882:head@391'4085, size: 65536, copy_subset: [(0, 65536)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 65536, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-16T09:41:50.391 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:39:25.253875+0000", 2024-09-16T09:41:50.391 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 145.06055592499999, 2024-09-16T09:41:50.391 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.25286863599999998, 2024-09-16T09:41:50.391 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:41:50.391 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:41:50.391 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T09:41:50.391 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:41:50.391 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.392 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:41:50.392 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.253875+0000", 2024-09-16T09:41:50.392 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:41:50.392 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.392 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.392 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:41:50.392 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.253875+0000", 2024-09-16T09:41:50.392 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:41:50.392 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.392 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.392 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:41:50.392 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.253877+0000", 2024-09-16T09:41:50.392 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.401e-06 2024-09-16T09:41:50.392 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.393 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.393 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:41:50.393 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.253914+0000", 2024-09-16T09:41:50.393 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7781999999999998e-05 2024-09-16T09:41:50.393 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.393 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.393 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:41:50.393 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.500529+0000", 2024-09-16T09:41:50.393 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.24661467500000001 2024-09-16T09:41:50.393 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.393 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.393 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:41:50.393 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.500551+0000", 2024-09-16T09:41:50.393 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.2459e-05 2024-09-16T09:41:50.393 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.394 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.394 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:41:50.394 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.504460+0000", 2024-09-16T09:41:50.394 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0039085600000000002 2024-09-16T09:41:50.394 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.394 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.394 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:41:50.394 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.504569+0000", 2024-09-16T09:41:50.394 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000108572 2024-09-16T09:41:50.394 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.394 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.394 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:41:50.394 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.506744+0000", 2024-09-16T09:41:50.395 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0021751869999999999 2024-09-16T09:41:50.395 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:41:50.395 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:41:50.395 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:41:50.395 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.395 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.395 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.15975.0:113616 10.0 e417/395 10:063ebb7c:::benchmark_data_smithi070_166744_object113615:head v 417'5658, mlcod=417'5656)", 2024-09-16T09:41:50.395 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:39:25.260314+0000", 2024-09-16T09:41:50.395 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 145.05411674699999, 2024-09-16T09:41:50.395 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.245055678, 2024-09-16T09:41:50.395 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:41:50.395 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:41:50.395 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:41:50.395 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:41:50.395 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.396 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:41:50.396 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.260314+0000", 2024-09-16T09:41:50.396 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:41:50.396 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.396 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.396 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:41:50.396 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.260314+0000", 2024-09-16T09:41:50.396 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:41:50.396 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.396 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.396 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:41:50.396 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.260316+0000", 2024-09-16T09:41:50.396 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.4529999999999999e-06 2024-09-16T09:41:50.396 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.397 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.397 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:41:50.397 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.260354+0000", 2024-09-16T09:41:50.397 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8655000000000002e-05 2024-09-16T09:41:50.397 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.397 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.397 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:41:50.397 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.500392+0000", 2024-09-16T09:41:50.397 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.24003755600000001 2024-09-16T09:41:50.397 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.397 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.397 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:41:50.397 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.500407+0000", 2024-09-16T09:41:50.397 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5182e-05 2024-09-16T09:41:50.398 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.398 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.398 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:41:50.398 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.500475+0000", 2024-09-16T09:41:50.398 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.8176999999999995e-05 2024-09-16T09:41:50.398 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.398 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.398 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:41:50.398 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.500498+0000", 2024-09-16T09:41:50.398 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.2191000000000001e-05 2024-09-16T09:41:50.398 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.398 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.398 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:41:50.398 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.505311+0000", 2024-09-16T09:41:50.399 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0048135870000000002 2024-09-16T09:41:50.399 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.399 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.399 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:41:50.399 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.505370+0000", 2024-09-16T09:41:50.399 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8876999999999999e-05 2024-09-16T09:41:50.399 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:41:50.399 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:41:50.399 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:41:50.399 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.399 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.399 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "MOSDPGPush(10.19 417/398 [PushOp(10:9aac0621:::benchmark_data_smithi070_166744_object2219:head, version: 383'132, data_included: [0~65536], data_size: 65536, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(10:9aac0621:::benchmark_data_smithi070_166744_object2219:head@383'132, size: 65536, copy_subset: [(0, 65536)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 65536, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-16T09:41:50.399 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:39:25.261549+0000", 2024-09-16T09:41:50.400 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 145.052881732, 2024-09-16T09:41:50.400 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.259429627, 2024-09-16T09:41:50.400 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:41:50.400 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:41:50.400 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T09:41:50.400 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:41:50.400 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.400 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:41:50.400 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.261549+0000", 2024-09-16T09:41:50.400 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:41:50.400 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.400 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.400 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:41:50.400 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.261549+0000", 2024-09-16T09:41:50.400 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:41:50.401 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.401 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.401 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:41:50.401 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.261550+0000", 2024-09-16T09:41:50.401 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.1500000000000003e-07 2024-09-16T09:41:50.401 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.401 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.401 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:41:50.401 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.261576+0000", 2024-09-16T09:41:50.401 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.5930000000000001e-05 2024-09-16T09:41:50.401 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.401 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.401 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:41:50.402 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.500469+0000", 2024-09-16T09:41:50.402 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.238892509 2024-09-16T09:41:50.402 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.402 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.402 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:41:50.402 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.500491+0000", 2024-09-16T09:41:50.402 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.2543000000000001e-05 2024-09-16T09:41:50.402 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.402 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.402 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:41:50.402 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.518931+0000", 2024-09-16T09:41:50.402 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.018439766 2024-09-16T09:41:50.402 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.402 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.403 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:41:50.403 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.518970+0000", 2024-09-16T09:41:50.403 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8656999999999999e-05 2024-09-16T09:41:50.403 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.403 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.403 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:41:50.403 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.520979+0000", 2024-09-16T09:41:50.403 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0020094069999999999 2024-09-16T09:41:50.403 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:41:50.403 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:41:50.403 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:41:50.403 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.403 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.403 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "MOSDPGPush(10.14 429/398 [PushOp(10:2b299b11:::benchmark_data_smithi070_166744_object49065:head, version: 389'3107, data_included: [0~65536], data_size: 65536, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(10:2b299b11:::benchmark_data_smithi070_166744_object49065:head@389'3107, size: 65536, copy_subset: [(0, 65536)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 65536, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-16T09:41:50.403 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:40:01.341126+0000", 2024-09-16T09:41:50.404 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 108.97330521000001, 2024-09-16T09:41:50.404 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.254867177, 2024-09-16T09:41:50.404 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:41:50.404 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:41:50.404 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T09:41:50.404 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:41:50.404 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.404 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:41:50.404 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.341126+0000", 2024-09-16T09:41:50.404 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:41:50.404 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.404 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.404 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:41:50.404 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.341126+0000", 2024-09-16T09:41:50.404 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:41:50.405 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.405 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.405 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:41:50.405 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:41:50.406 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.341127+0000", 2024-09-16T09:41:50.406 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.421e-06 2024-09-16T09:41:50.406 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.406 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.406 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:41:50.406 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.341178+0000", 2024-09-16T09:41:50.406 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0847000000000003e-05 2024-09-16T09:41:50.406 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.406 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.406 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:41:50.407 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.341201+0000", 2024-09-16T09:41:50.407 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3283999999999999e-05 2024-09-16T09:41:50.407 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.407 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.407 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:41:50.407 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.341220+0000", 2024-09-16T09:41:50.407 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.8289000000000001e-05 2024-09-16T09:41:50.407 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.407 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.407 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:41:50.407 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.594029+0000", 2024-09-16T09:41:50.407 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.25280975999999999 2024-09-16T09:41:50.407 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.407 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.408 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:41:50.408 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.594043+0000", 2024-09-16T09:41:50.408 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3455000000000001e-05 2024-09-16T09:41:50.408 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.408 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.408 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:41:50.408 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.595993+0000", 2024-09-16T09:41:50.408 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001950121 2024-09-16T09:41:50.408 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:41:50.408 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:41:50.408 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:41:50.408 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.408 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.409 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.15975.0:150386 10.14 e429/398 10:2e95170f:::benchmark_data_smithi070_166744_object36030:head v 429'6995, mlcod=389'3105)", 2024-09-16T09:41:50.409 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:40:01.341250+0000", 2024-09-16T09:41:50.409 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 108.973180891, 2024-09-16T09:41:50.409 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.26185008100000001, 2024-09-16T09:41:50.409 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:41:50.409 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:41:50.409 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:41:50.409 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:41:50.409 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.409 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:41:50.409 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.341250+0000", 2024-09-16T09:41:50.409 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:41:50.409 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.410 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.410 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:41:50.410 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.341250+0000", 2024-09-16T09:41:50.410 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:41:50.410 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.410 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.410 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:41:50.410 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.341251+0000", 2024-09-16T09:41:50.410 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.1699999999999999e-07 2024-09-16T09:41:50.410 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.410 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.410 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:41:50.410 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.341256+0000", 2024-09-16T09:41:50.411 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.1170000000000003e-06 2024-09-16T09:41:50.411 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.411 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.411 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:41:50.411 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.341263+0000", 2024-09-16T09:41:50.411 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.0149999999999998e-06 2024-09-16T09:41:50.411 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.411 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.411 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:41:50.411 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.341267+0000", 2024-09-16T09:41:50.411 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.7620000000000003e-06 2024-09-16T09:41:50.411 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.411 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.411 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:41:50.412 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.591255+0000", 2024-09-16T09:41:50.412 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.24998795700000001 2024-09-16T09:41:50.412 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.412 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.412 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:41:50.412 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.591287+0000", 2024-09-16T09:41:50.412 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1625999999999998e-05 2024-09-16T09:41:50.412 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.412 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.412 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:41:50.412 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.603044+0000", 2024-09-16T09:41:50.412 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.011756679000000001 2024-09-16T09:41:50.412 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.413 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.413 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:41:50.413 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.603100+0000", 2024-09-16T09:41:50.413 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.6508000000000002e-05 2024-09-16T09:41:50.413 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:41:50.413 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:41:50.413 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:41:50.413 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.413 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.413 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.15975.0:150389 10.2 e429/395 10:41a08be2:::benchmark_data_smithi070_166744_object36033:head v 429'7013, mlcod=429'7011)", 2024-09-16T09:41:50.413 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:40:01.341276+0000", 2024-09-16T09:41:50.413 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 108.97315487100001, 2024-09-16T09:41:50.413 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.25004327399999998, 2024-09-16T09:41:50.413 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:41:50.414 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:41:50.414 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:41:50.414 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:41:50.414 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.414 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:41:50.414 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.341276+0000", 2024-09-16T09:41:50.414 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:41:50.414 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.414 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.414 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:41:50.414 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.341276+0000", 2024-09-16T09:41:50.414 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:41:50.414 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.414 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.415 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:41:50.415 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.341276+0000", 2024-09-16T09:41:50.415 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.7500000000000001e-07 2024-09-16T09:41:50.415 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.415 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.415 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:41:50.415 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.341280+0000", 2024-09-16T09:41:50.415 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2799999999999999e-06 2024-09-16T09:41:50.415 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.415 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.415 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:41:50.415 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.341285+0000", 2024-09-16T09:41:50.415 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5690000000000004e-06 2024-09-16T09:41:50.416 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.416 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.416 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:41:50.416 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.341289+0000", 2024-09-16T09:41:50.416 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7919999999999999e-06 2024-09-16T09:41:50.416 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.416 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.416 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:41:50.416 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.341352+0000", 2024-09-16T09:41:50.416 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.2731999999999995e-05 2024-09-16T09:41:50.416 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.416 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.416 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:41:50.416 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.341377+0000", 2024-09-16T09:41:50.417 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.4686e-05 2024-09-16T09:41:50.417 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.417 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.417 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:41:50.417 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.591266+0000", 2024-09-16T09:41:50.417 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.24988992500000001 2024-09-16T09:41:50.417 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.417 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.417 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:41:50.417 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.591319+0000", 2024-09-16T09:41:50.417 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.3015e-05 2024-09-16T09:41:50.417 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:41:50.417 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:41:50.417 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:41:50.418 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.418 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.418 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.15975.0:150393 10.b e429/376 10:da2f4e9f:::benchmark_data_smithi070_166744_object36037:head v 429'9364, mlcod=429'9361)", 2024-09-16T09:41:50.418 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:40:01.345049+0000", 2024-09-16T09:41:50.418 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 108.96938202, 2024-09-16T09:41:50.418 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.25539264, 2024-09-16T09:41:50.418 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:41:50.418 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:41:50.418 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:41:50.418 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:41:50.418 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.418 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:41:50.418 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.345049+0000", 2024-09-16T09:41:50.418 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:41:50.419 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.419 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.419 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:41:50.419 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.345049+0000", 2024-09-16T09:41:50.419 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:41:50.419 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.419 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.419 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:41:50.419 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.345050+0000", 2024-09-16T09:41:50.419 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.7300000000000004e-07 2024-09-16T09:41:50.419 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.419 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.419 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:41:50.420 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.345056+0000", 2024-09-16T09:41:50.420 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1550000000000004e-06 2024-09-16T09:41:50.420 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.420 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.420 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:41:50.420 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.591338+0000", 2024-09-16T09:41:50.420 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.24628182400000001 2024-09-16T09:41:50.420 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.420 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.420 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:41:50.420 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.591348+0000", 2024-09-16T09:41:50.420 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.9699999999999994e-06 2024-09-16T09:41:50.421 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.421 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.421 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:41:50.421 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.591412+0000", 2024-09-16T09:41:50.421 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.4387000000000005e-05 2024-09-16T09:41:50.421 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.421 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.421 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:41:50.421 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.591426+0000", 2024-09-16T09:41:50.421 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.411e-05 2024-09-16T09:41:50.421 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.421 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.421 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:41:50.422 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.600407+0000", 2024-09-16T09:41:50.422 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0089803020000000008 2024-09-16T09:41:50.422 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.422 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.422 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:41:50.422 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.600442+0000", 2024-09-16T09:41:50.422 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4919000000000002e-05 2024-09-16T09:41:50.422 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:41:50.422 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:41:50.422 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:41:50.422 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.422 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.422 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.15975.0:150397 10.14 e429/398 10:2c54e8e2:::benchmark_data_smithi070_166744_object36041:head v 429'6996, mlcod=389'3105)", 2024-09-16T09:41:50.422 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:40:01.346819+0000", 2024-09-16T09:41:50.423 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 108.967612479, 2024-09-16T09:41:50.423 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.25636429500000002, 2024-09-16T09:41:50.423 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:41:50.423 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:41:50.423 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:41:50.423 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:41:50.423 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.423 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:41:50.423 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.346819+0000", 2024-09-16T09:41:50.423 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:41:50.423 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.423 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.423 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:41:50.423 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.346819+0000", 2024-09-16T09:41:50.424 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:41:50.424 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.424 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.424 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:41:50.424 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.346820+0000", 2024-09-16T09:41:50.424 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0750000000000001e-06 2024-09-16T09:41:50.424 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.424 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.424 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:41:50.424 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.346829+0000", 2024-09-16T09:41:50.424 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.1330000000000005e-06 2024-09-16T09:41:50.424 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.424 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.424 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:41:50.425 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.592302+0000", 2024-09-16T09:41:50.425 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.24547290199999999 2024-09-16T09:41:50.425 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.425 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.425 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:41:50.425 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.592309+0000", 2024-09-16T09:41:50.425 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.2629999999999997e-06 2024-09-16T09:41:50.425 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.425 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.425 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:41:50.425 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.596075+0000", 2024-09-16T09:41:50.425 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0037657459999999999 2024-09-16T09:41:50.425 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.426 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.426 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:41:50.426 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.596096+0000", 2024-09-16T09:41:50.426 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0801999999999998e-05 2024-09-16T09:41:50.426 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.426 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.426 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:41:50.426 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.603145+0000", 2024-09-16T09:41:50.426 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0070491750000000004 2024-09-16T09:41:50.426 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.426 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.426 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:41:50.426 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.603183+0000", 2024-09-16T09:41:50.426 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8198999999999998e-05 2024-09-16T09:41:50.426 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:41:50.427 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:41:50.427 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:41:50.427 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.427 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.427 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.15975.0:201399 10.16 e442/418 10:6f797fc7:::benchmark_data_smithi070_166744_object87043:head v 442'8394, mlcod=442'8392)", 2024-09-16T09:41:50.427 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:40:42.287088+0000", 2024-09-16T09:41:50.427 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 68.027342916999999, 2024-09-16T09:41:50.427 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.195435212, 2024-09-16T09:41:50.427 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:41:50.427 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:41:50.427 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:41:50.427 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:41:50.427 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.427 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:41:50.428 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:42.287088+0000", 2024-09-16T09:41:50.428 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:41:50.428 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.428 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.428 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:41:50.428 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:42.287088+0000", 2024-09-16T09:41:50.428 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:41:50.428 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.428 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.428 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:41:50.428 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:42.287089+0000", 2024-09-16T09:41:50.428 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.23e-07 2024-09-16T09:41:50.428 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.428 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.429 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:41:50.429 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:42.287095+0000", 2024-09-16T09:41:50.429 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8950000000000003e-06 2024-09-16T09:41:50.429 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.429 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.429 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:41:50.429 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:42.287104+0000", 2024-09-16T09:41:50.429 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.0359999999999995e-06 2024-09-16T09:41:50.429 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.429 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.429 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:41:50.429 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:42.287109+0000", 2024-09-16T09:41:50.429 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.3159999999999996e-06 2024-09-16T09:41:50.430 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.430 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.430 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:41:50.430 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:42.287154+0000", 2024-09-16T09:41:50.430 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.4892000000000003e-05 2024-09-16T09:41:50.430 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.430 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.430 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:41:50.430 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:42.287167+0000", 2024-09-16T09:41:50.430 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3064000000000001e-05 2024-09-16T09:41:50.430 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.430 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.430 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:41:50.430 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:42.482402+0000", 2024-09-16T09:41:50.431 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.19523507700000001 2024-09-16T09:41:50.431 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.431 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.431 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:41:50.431 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:42.482523+0000", 2024-09-16T09:41:50.431 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000121209 2024-09-16T09:41:50.431 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:41:50.431 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:41:50.431 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:41:50.431 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.431 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.431 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.15975.0:201403 10.15 e442/434 10:ac92296d:::benchmark_data_smithi070_166744_object87047:head v 442'8572, mlcod=442'8570)", 2024-09-16T09:41:50.431 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:40:42.287806+0000", 2024-09-16T09:41:50.432 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 68.026625104000004, 2024-09-16T09:41:50.432 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.19616398400000001, 2024-09-16T09:41:50.432 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:41:50.432 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:41:50.432 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:41:50.432 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:41:50.432 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.432 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:41:50.432 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:42.287806+0000", 2024-09-16T09:41:50.432 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:41:50.432 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.432 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.432 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:41:50.433 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:42.287806+0000", 2024-09-16T09:41:50.433 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:41:50.433 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.433 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.433 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:41:50.433 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:42.287807+0000", 2024-09-16T09:41:50.433 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.0599999999999999e-07 2024-09-16T09:41:50.433 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.433 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.433 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:41:50.433 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:42.287812+0000", 2024-09-16T09:41:50.433 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5809999999999996e-06 2024-09-16T09:41:50.433 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.434 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.434 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:41:50.434 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:42.477572+0000", 2024-09-16T09:41:50.434 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.18975939999999999 2024-09-16T09:41:50.434 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.434 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.434 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:41:50.434 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:42.477602+0000", 2024-09-16T09:41:50.434 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.0151999999999999e-05 2024-09-16T09:41:50.434 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.434 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.434 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:41:50.434 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:42.477736+0000", 2024-09-16T09:41:50.434 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00013442099999999999 2024-09-16T09:41:50.435 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.435 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.435 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:41:50.435 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:42.477760+0000", 2024-09-16T09:41:50.435 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3679000000000001e-05 2024-09-16T09:41:50.435 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.435 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.435 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:41:50.435 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:42.483940+0000", 2024-09-16T09:41:50.435 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0061803209999999999 2024-09-16T09:41:50.435 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:41:50.435 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:41:50.435 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:41:50.435 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:42.483970+0000", 2024-09-16T09:41:50.435 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.9624e-05 2024-09-16T09:41:50.436 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:41:50.436 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:41:50.436 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:41:50.436 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:41:50.436 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:41:50.436 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T09:41:50.472 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:50.472 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:50.473 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:41:50.574 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:41:50.617 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:50.618 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:50.618 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:41:50.719 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:41:50.776 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:50.777 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:50.778 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:41:50.878 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:41:50.921 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:50.921 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:50.921 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:41:51.022 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:41:51.062 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:51.063 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:51.063 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:41:51.093 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 31 16 61834 61818 124.614 88.8125 0.013368 0.00802021 2024-09-16T09:41:51.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:50 smithi084 ceph-mon[32629]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:41:51.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:50 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-16T09:41:51.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:50 smithi084 ceph-mon[32629]: osdmap e467: 8 total, 8 up, 8 in 2024-09-16T09:41:51.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:50 smithi084 ceph-mon[32629]: pgmap v886: 168 pgs: 168 active+clean; 3.5 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 124 MiB/s wr, 1.98k op/s 2024-09-16T09:41:51.165 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:41:51.170 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:50 smithi070 ceph-mon[30388]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:41:51.170 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:50 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-16T09:41:51.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:50 smithi070 ceph-mon[30388]: osdmap e467: 8 total, 8 up, 8 in 2024-09-16T09:41:51.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:50 smithi070 ceph-mon[30388]: pgmap v886: 168 pgs: 168 active+clean; 3.5 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 124 MiB/s wr, 1.98k op/s 2024-09-16T09:41:51.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:50 smithi070 ceph-mon[40691]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:41:51.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:50 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-16T09:41:51.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:50 smithi070 ceph-mon[40691]: osdmap e467: 8 total, 8 up, 8 in 2024-09-16T09:41:51.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:50 smithi070 ceph-mon[40691]: pgmap v886: 168 pgs: 168 active+clean; 3.5 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 124 MiB/s wr, 1.98k op/s 2024-09-16T09:41:51.193 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:51.193 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:51.194 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:41:51.295 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:41:51.330 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:51.331 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:51.332 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:41:51.432 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:41:51.460 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:51.460 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:51.461 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:41:51.562 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:41:51.591 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:51.592 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:51.592 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:41:51.693 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:41:51.729 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:51.730 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:51.730 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:41:51.831 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:41:51.874 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:51.874 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:51.875 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:41:51.976 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:41:52.010 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:52.011 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:52.011 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:41:52.094 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 32 16 63362 63346 123.703 95.5 0.0104195 0.00807909 2024-09-16T09:41:52.112 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:41:52.150 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:52.150 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:52.151 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:41:52.251 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:41:52.284 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:52.285 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:52.285 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:41:52.386 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:41:52.421 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:52.421 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:41:51 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:41:51] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:41:52.422 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:52.422 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:41:52.523 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:41:52.556 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:52.557 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:52.557 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:41:52.658 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:41:52.695 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:52.695 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:52.696 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:41:52.796 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:41:52.824 INFO:tasks.thrashosds.thrasher:j is {'epoch': 466, 'fsid': 'a1321834-740c-11ef-bceb-c7b262605968', 'created': '2024-09-16T09:19:14.121498+0000', 'modified': '2024-09-16T09:41:44.410061+0000', 'last_up_change': '2024-09-16T09:24:51.124496+0000', 'last_in_change': '2024-09-16T09:24:28.432920+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 18, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 11, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T09:22:27.617694+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 21, 'pg_placement_num': 21, 'pg_placement_num_target': 21, 'pg_num_target': 21, 'pg_num_pending': 21, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '162', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '160', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6699999570846558, 'score_stable': 1.6699999570846558, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 1.899999976158142, 'raw_score_stable': 1.899999976158142, 'primary_affinity_weighted': 0.7071933746337891, 'average_primary_affinity': 0.8100000023841858, 'average_primary_affinity_weighted': 0.7099999785423279}}, {'pool': 2, 'pool_name': 'datapool', 'create_time': '2024-09-16T09:24:59.169244+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 3, 'pg_placement_num': 3, 'pg_placement_num_target': 3, 'pg_num_target': 3, 'pg_num_pending': 3, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 61, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.3299999237060547, 'score_stable': 2.3299999237060547, 'optimal_score': 0.8600000143051147, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.7071933746337891, 'average_primary_affinity': 0.8100000023841858, 'average_primary_affinity_weighted': 0.7099999785423279}}, {'pool': 3, 'pool_name': '.rgw.root', 'create_time': '2024-09-16T09:25:00.056119+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.7071933746337891, 'average_primary_affinity': 0.8100000023841858, 'average_primary_affinity_weighted': 0.7099999785423279}}, {'pool': 4, 'pool_name': 'default.rgw.log', 'create_time': '2024-09-16T09:25:01.770016+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '62', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.849999904632568, 'score_stable': 2.4100000858306885, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 7.809999942779541, 'raw_score_stable': 2.75, 'primary_affinity_weighted': 0.7071933746337891, 'average_primary_affinity': 0.8100000023841858, 'average_primary_affinity_weighted': 0.7099999785423279, 'error_message': 'pool 4 has acting primaries on OSD(s) with primary affinity 0, read balance score is not accurate'}}, {'pool': 5, 'pool_name': 'default.rgw.control', 'create_time': '2024-09-16T09:25:03.725555+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.7071933746337891, 'average_primary_affinity': 0.8100000023841858, 'average_primary_affinity_weighted': 0.7099999785423279}}, {'pool': 6, 'pool_name': 'default.rgw.meta', 'create_time': '2024-09-16T09:25:05.773530+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5399999618530273, 'score_stable': 1.5399999618530273, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 0.7071933746337891, 'average_primary_affinity': 0.8100000023841858, 'average_primary_affinity_weighted': 0.7099999785423279}}, {'pool': 11, 'pool_name': 'unique_pool_4', 'create_time': '2024-09-16T09:41:11.917610+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 16, 'pg_placement_num': 16, 'pg_placement_num_target': 16, 'pg_num_target': 16, 'pg_num_pending': 16, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '458', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rados': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.7071933746337891, 'average_primary_affinity': 0.8100000023841858, 'average_primary_affinity_weighted': 0.7099999785423279}}], 'osds': [{'osd': 0, 'uuid': '7ef288c8-3bf0-4814-8e86-f70a7de041ce', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 8, 'up_thru': 454, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6802', 'nonce': 2013075781}, {'type': 'v1', 'addr': '172.21.15.70:6803', 'nonce': 2013075781}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6804', 'nonce': 2013075781}, {'type': 'v1', 'addr': '172.21.15.70:6805', 'nonce': 2013075781}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6808', 'nonce': 2013075781}, {'type': 'v1', 'addr': '172.21.15.70:6809', 'nonce': 2013075781}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6806', 'nonce': 2013075781}, {'type': 'v1', 'addr': '172.21.15.70:6807', 'nonce': 2013075781}]}, 'public_addr': '172.21.15.70:6803/2013075781', 'cluster_addr': '172.21.15.70:6805/2013075781', 'heartbeat_back_addr': '172.21.15.70:6809/2013075781', 'heartbeat_front_addr': '172.21.15.70:6807/2013075781', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '474d41d9-295a-450c-8f27-ce487a14678b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 362, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6810', 'nonce': 1670012115}, {'type': 'v1', 'addr': '172.21.15.70:6811', 'nonce': 1670012115}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6812', 'nonce': 1670012115}, {'type': 'v1', 'addr': '172.21.15.70:6813', 'nonce': 1670012115}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6816', 'nonce': 1670012115}, {'type': 'v1', 'addr': '172.21.15.70:6817', 'nonce': 1670012115}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6814', 'nonce': 1670012115}, {'type': 'v1', 'addr': '172.21.15.70:6815', 'nonce': 1670012115}]}, 'public_addr': '172.21.15.70:6811/1670012115', 'cluster_addr': '172.21.15.70:6813/1670012115', 'heartbeat_back_addr': '172.21.15.70:6817/1670012115', 'heartbeat_front_addr': '172.21.15.70:6815/1670012115', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '5b4ca4e8-cecc-4163-b291-c9209b02c4b7', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.1810150146484375, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 18, 'up_thru': 454, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6818', 'nonce': 3601682336}, {'type': 'v1', 'addr': '172.21.15.70:6819', 'nonce': 3601682336}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6820', 'nonce': 3601682336}, {'type': 'v1', 'addr': '172.21.15.70:6821', 'nonce': 3601682336}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6824', 'nonce': 3601682336}, {'type': 'v1', 'addr': '172.21.15.70:6825', 'nonce': 3601682336}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6822', 'nonce': 3601682336}, {'type': 'v1', 'addr': '172.21.15.70:6823', 'nonce': 3601682336}]}, 'public_addr': '172.21.15.70:6819/3601682336', 'cluster_addr': '172.21.15.70:6821/3601682336', 'heartbeat_back_addr': '172.21.15.70:6825/3601682336', 'heartbeat_front_addr': '172.21.15.70:6823/3601682336', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'ffb8d527-1b58-4ecd-808c-47d4ba1bf387', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.476531982421875, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 26, 'up_thru': 454, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6826', 'nonce': 2038083007}, {'type': 'v1', 'addr': '172.21.15.70:6827', 'nonce': 2038083007}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6828', 'nonce': 2038083007}, {'type': 'v1', 'addr': '172.21.15.70:6829', 'nonce': 2038083007}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6832', 'nonce': 2038083007}, {'type': 'v1', 'addr': '172.21.15.70:6833', 'nonce': 2038083007}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6830', 'nonce': 2038083007}, {'type': 'v1', 'addr': '172.21.15.70:6831', 'nonce': 2038083007}]}, 'public_addr': '172.21.15.70:6827/2038083007', 'cluster_addr': '172.21.15.70:6829/2038083007', 'heartbeat_back_addr': '172.21.15.70:6833/2038083007', 'heartbeat_front_addr': '172.21.15.70:6831/2038083007', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'a946760b-4d91-4a4b-8e9b-a07a50589211', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 32, 'up_thru': 454, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6800', 'nonce': 2250892710}, {'type': 'v1', 'addr': '172.21.15.84:6801', 'nonce': 2250892710}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6802', 'nonce': 2250892710}, {'type': 'v1', 'addr': '172.21.15.84:6803', 'nonce': 2250892710}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6806', 'nonce': 2250892710}, {'type': 'v1', 'addr': '172.21.15.84:6807', 'nonce': 2250892710}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6804', 'nonce': 2250892710}, {'type': 'v1', 'addr': '172.21.15.84:6805', 'nonce': 2250892710}]}, 'public_addr': '172.21.15.84:6801/2250892710', 'cluster_addr': '172.21.15.84:6803/2250892710', 'heartbeat_back_addr': '172.21.15.84:6807/2250892710', 'heartbeat_front_addr': '172.21.15.84:6805/2250892710', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '67681760-75a9-4846-946e-a935d42c5e05', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 39, 'up_thru': 454, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6808', 'nonce': 1803945348}, {'type': 'v1', 'addr': '172.21.15.84:6809', 'nonce': 1803945348}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6810', 'nonce': 1803945348}, {'type': 'v1', 'addr': '172.21.15.84:6811', 'nonce': 1803945348}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6814', 'nonce': 1803945348}, {'type': 'v1', 'addr': '172.21.15.84:6815', 'nonce': 1803945348}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6812', 'nonce': 1803945348}, {'type': 'v1', 'addr': '172.21.15.84:6813', 'nonce': 1803945348}]}, 'public_addr': '172.21.15.84:6809/1803945348', 'cluster_addr': '172.21.15.84:6811/1803945348', 'heartbeat_back_addr': '172.21.15.84:6815/1803945348', 'heartbeat_front_addr': '172.21.15.84:6813/1803945348', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '7c8f0152-f539-4b7e-8113-91dd7529d3a1', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 46, 'up_thru': 454, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6816', 'nonce': 610727189}, {'type': 'v1', 'addr': '172.21.15.84:6817', 'nonce': 610727189}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6818', 'nonce': 610727189}, {'type': 'v1', 'addr': '172.21.15.84:6819', 'nonce': 610727189}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6822', 'nonce': 610727189}, {'type': 'v1', 'addr': '172.21.15.84:6823', 'nonce': 610727189}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6820', 'nonce': 610727189}, {'type': 'v1', 'addr': '172.21.15.84:6821', 'nonce': 610727189}]}, 'public_addr': '172.21.15.84:6817/610727189', 'cluster_addr': '172.21.15.84:6819/610727189', 'heartbeat_back_addr': '172.21.15.84:6823/610727189', 'heartbeat_front_addr': '172.21.15.84:6821/610727189', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '945efaba-225e-49af-aefd-b4d58ca8093b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 53, 'up_thru': 454, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6824', 'nonce': 1002252531}, {'type': 'v1', 'addr': '172.21.15.84:6825', 'nonce': 1002252531}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6826', 'nonce': 1002252531}, {'type': 'v1', 'addr': '172.21.15.84:6827', 'nonce': 1002252531}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6830', 'nonce': 1002252531}, {'type': 'v1', 'addr': '172.21.15.84:6831', 'nonce': 1002252531}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6828', 'nonce': 1002252531}, {'type': 'v1', 'addr': '172.21.15.84:6829', 'nonce': 1002252531}]}, 'public_addr': '172.21.15.84:6825/1002252531', 'cluster_addr': '172.21.15.84:6827/1002252531', 'heartbeat_back_addr': '172.21.15.84:6831/1002252531', 'heartbeat_front_addr': '172.21.15.84:6829/1002252531', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T09:41:12.518411+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T09:41:01.871112+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T09:41:21.197490+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T09:41:02.801287+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T09:40:54.659737+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T09:41:29.134973+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T09:40:59.352436+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T09:41:27.957285+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.1d', 'osds': [1, 0, 7]}, {'pgid': '4.b', 'osds': [0, 5, 1]}, {'pgid': '4.19', 'osds': [4, 7, 5]}], 'pg_upmap_items': [{'pgid': '2.1', 'mappings': [{'from': 3, 'to': 0}, {'from': 2, 'to': 1}]}, {'pgid': '3.1b', 'mappings': [{'from': 0, 'to': 6}, {'from': 4, 'to': 7}]}, {'pgid': '4.f', 'mappings': [{'from': 1, 'to': 2}]}, {'pgid': '4.13', 'mappings': [{'from': 1, 'to': 5}]}, {'pgid': '4.17', 'mappings': [{'from': 1, 'to': 2}]}, {'pgid': '5.10', 'mappings': [{'from': 7, 'to': 4}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {'172.21.15.70:0/125491368': '2024-09-17T09:25:31.465167+0000', '172.21.15.70:0/3354284852': '2024-09-17T09:19:43.205989+0000', '172.21.15.70:0/3284625547': '2024-09-17T09:25:31.465167+0000', '172.21.15.70:0/186565817': '2024-09-17T09:19:43.205989+0000', '172.21.15.70:0/2213852865': '2024-09-17T09:19:43.205989+0000', '172.21.15.70:0/1812373760': '2024-09-17T09:20:09.306330+0000', '172.21.15.70:6800/837338094': '2024-09-17T09:19:43.205989+0000', '172.21.15.70:6800/1876661229': '2024-09-17T09:25:31.465167+0000', '172.21.15.70:0/3351688485': '2024-09-17T09:20:09.306330+0000', '172.21.15.70:6800/3871553358': '2024-09-17T09:20:09.306330+0000', '172.21.15.70:0/1230482871': '2024-09-17T09:25:31.465167+0000', '172.21.15.70:6801/837338094': '2024-09-17T09:19:43.205989+0000', '172.21.15.70:6801/3871553358': '2024-09-17T09:20:09.306330+0000', '172.21.15.70:0/1403001486': '2024-09-17T09:20:09.306330+0000', '172.21.15.70:0/1723643739': '2024-09-17T09:25:31.465167+0000', '172.21.15.70:6801/1876661229': '2024-09-17T09:25:31.465167+0000', '172.21.15.70:0/3456417396': '2024-09-17T09:25:31.465167+0000'}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-16T09:41:52.824 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-16T09:41:52.844 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:52.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:52.845 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:41:52.946 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:41:52.986 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:52.987 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:52.987 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:41:53.088 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:41:53.093 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 33 16 64835 64819 122.744 92.0625 0.0119584 0.00814169 2024-09-16T09:41:53.122 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:53.122 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:53.123 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:41:53.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:52 smithi084 ceph-mon[32629]: pgmap v887: 168 pgs: 168 active+clean; 3.5 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 639 B/s rd, 62 MiB/s wr, 989 op/s 2024-09-16T09:41:53.170 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:52 smithi070 ceph-mon[30388]: pgmap v887: 168 pgs: 168 active+clean; 3.5 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 639 B/s rd, 62 MiB/s wr, 989 op/s 2024-09-16T09:41:53.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:52 smithi070 ceph-mon[40691]: pgmap v887: 168 pgs: 168 active+clean; 3.5 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 639 B/s rd, 62 MiB/s wr, 989 op/s 2024-09-16T09:41:53.224 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:41:53.268 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:53.269 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:53.269 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:41:53.326 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.2 dump_ops_in_flight 2024-09-16T09:41:53.342 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:41:53.370 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:41:53.418 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:53.418 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:53.419 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:41:53.519 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:41:53.561 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:53.562 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:53.562 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:41:53.662 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:41:53.663 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:41:53.691 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:53.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:53.692 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:41:53.793 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:41:53.826 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:53.827 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:53.827 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:41:53.910 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:41:53 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:41:53.928 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:41:53.967 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:53.968 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:53.968 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:41:54.069 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:41:54.094 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 34 16 66155 66139 121.558 82.5 0.0138344 0.00822211 2024-09-16T09:41:54.104 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:54.105 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:54.105 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:41:54.110 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-16T09:41:54.150 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:53 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:41:54.150 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:53 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:41:54.206 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:41:54.236 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:54.237 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:54.237 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:41:54.338 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:41:54.383 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:54.384 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:54.384 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:41:54.390 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:41:54.390 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:41:54.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:53 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:41:54.420 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:41:54.420 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:41:54.469 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:41:54.469 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:41:54.485 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:41:54.510 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:41:54.510 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:41:54.512 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:54.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:54.513 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:41:54.545 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:41:54.545 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:41:54.546 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:41:54.588 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:41:54.588 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:41:54.614 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:41:54.636 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:41:54.637 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:41:54.676 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:41:54.676 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:41:54.677 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:54.678 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:54.678 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:41:54.779 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:41:54.817 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:54.818 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:54.818 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:41:54.846 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:41:54.850 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2024-09-16T09:41:54.919 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:41:54.950 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:54.951 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:54.951 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:41:55.052 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:41:55.086 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:55.087 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:55.087 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:41:55.095 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 35 16 67286 67270 120.104 70.6875 0.0072549 0.00832076 2024-09-16T09:41:55.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:54 smithi070 ceph-mon[30388]: pgmap v888: 168 pgs: 168 active+clean; 3.9 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.1 KiB/s rd, 97 MiB/s wr, 1.55k op/s 2024-09-16T09:41:55.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:54 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/4135754395' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:41:55.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:54 smithi070 ceph-mon[30388]: osdmap e468: 8 total, 8 up, 8 in 2024-09-16T09:41:55.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:54 smithi070 ceph-mon[40691]: pgmap v888: 168 pgs: 168 active+clean; 3.9 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.1 KiB/s rd, 97 MiB/s wr, 1.55k op/s 2024-09-16T09:41:55.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:54 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/4135754395' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:41:55.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:54 smithi070 ceph-mon[40691]: osdmap e468: 8 total, 8 up, 8 in 2024-09-16T09:41:55.189 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:41:55.230 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:55.231 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:55.231 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:41:55.332 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:41:55.333 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:54 smithi084 ceph-mon[32629]: pgmap v888: 168 pgs: 168 active+clean; 3.9 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.1 KiB/s rd, 97 MiB/s wr, 1.55k op/s 2024-09-16T09:41:55.333 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:54 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/4135754395' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:41:55.333 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:54 smithi084 ceph-mon[32629]: osdmap e468: 8 total, 8 up, 8 in 2024-09-16T09:41:55.365 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:55.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:55.366 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:41:55.467 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:41:55.502 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:55.503 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:55.503 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:41:55.604 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:41:55.642 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:55.643 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:55.643 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:41:55.744 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:41:55.778 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:55.778 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:55.779 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:41:55.879 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:41:55.916 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:55.916 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:55.917 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:41:56.017 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:41:56.039 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:56.040 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:56.040 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:41:56.095 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 36 15 68483 68468 118.848 74.875 0.0158896 0.00840921 2024-09-16T09:41:56.140 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:41:56.161 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:56.162 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:56.162 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:41:56.263 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:41:56.298 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:56.298 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:56.299 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:41:56.400 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:41:56.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:55 smithi084 ceph-mon[32629]: from='client.26264 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:41:56.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:55 smithi070 ceph-mon[30388]: from='client.26264 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:41:56.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:55 smithi070 ceph-mon[40691]: from='client.26264 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:41:56.435 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:56.436 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:56.436 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:41:56.537 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:41:56.579 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:56.580 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:56.580 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:41:56.681 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:41:56.716 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:56.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:56.717 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:41:56.818 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:41:56.849 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:56.850 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:56.850 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:41:56.934 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T09:41:56.934 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:41:56.952 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:41:56.986 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:56.987 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:56.987 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:41:57.088 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:41:57.094 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 37 16 69580 69564 117.487 68.5 0.00391995 0.00850667 2024-09-16T09:41:57.127 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:57.128 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:57.128 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:41:57.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:56 smithi070 ceph-mon[30388]: pgmap v890: 168 pgs: 168 active+clean; 4.0 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 639 B/s rd, 99 MiB/s wr, 1.58k op/s 2024-09-16T09:41:57.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:56 smithi070 ceph-mon[40691]: pgmap v890: 168 pgs: 168 active+clean; 4.0 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 639 B/s rd, 99 MiB/s wr, 1.58k op/s 2024-09-16T09:41:57.229 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:41:57.262 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:57.263 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:57.263 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:41:57.364 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:41:57.396 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:57.397 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:57.397 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:41:57.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:56 smithi084 ceph-mon[32629]: pgmap v890: 168 pgs: 168 active+clean; 4.0 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 639 B/s rd, 99 MiB/s wr, 1.58k op/s 2024-09-16T09:41:57.497 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:41:57.537 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:57.537 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:57.538 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:41:57.639 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:41:57.677 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:57.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:57.678 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:41:57.779 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:41:57.817 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:57.818 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:57.818 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:41:57.919 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:41:57.959 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:57.959 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:57.960 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:41:58.061 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:41:58.095 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 38 16 71000 70984 116.731 88.75 0.0107075 0.00856133 2024-09-16T09:41:58.096 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:58.096 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:58.097 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:41:58.197 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:41:58.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:58.219 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:58.219 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:41:58.320 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:41:58.357 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:58.358 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:58.358 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:41:58.459 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:41:58.474 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":889,"stamp":"2024-09-16T09:41:54.447945+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4221245979,"num_objects":64601,"num_object_clones":0,"num_object_copies":193803,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64601,"num_whiteouts":0,"num_read":2145,"num_read_kb":1897,"num_write":65124,"num_write_kb":4121773,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":150,"num_bytes_recovered":333352,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":66830,"ondisk_log_size":66830,"up":504,"acting":504,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":505,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":18284080,"kb_used_data":12317616,"kb_used_omap":12,"kb_used_meta":5966323,"kb_avail":731677136,"statfs":{"total":767960285184,"available":749237387264,"internally_reserved":0,"allocated":12613238784,"data_stored":12609364766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":6109515314},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,7,8,10,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":923402240,"num_objects":14090,"num_object_clones":0,"num_object_copies":42270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14090,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":14090,"num_write_kb":901760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.598419"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":881,"reported_epoch":466,"state":"active+clean","last_fresh":"2024-09-16T09:41:44.424150+0000","last_change":"2024-09-16T09:41:40.073159+0000","last_active":"2024-09-16T09:41:44.424150+0000","last_peered":"2024-09-16T09:41:44.424150+0000","last_clean":"2024-09-16T09:41:44.424150+0000","last_became_active":"2024-09-16T09:25:01.709108+0000","last_became_peered":"2024-09-16T09:25:01.709108+0000","last_unstale":"2024-09-16T09:41:44.424150+0000","last_undegraded":"2024-09-16T09:41:44.424150+0000","last_fullsized":"2024-09-16T09:41:44.424150+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:40.073089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:41:40.073089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:58.125354+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"285'15","reported_seq":894,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.710722+0000","last_change":"2024-09-16T09:41:43.064790+0000","last_active":"2024-09-16T09:41:49.710722+0000","last_peered":"2024-09-16T09:41:49.710722+0000","last_clean":"2024-09-16T09:41:49.710722+0000","last_became_active":"2024-09-16T09:25:03.714635+0000","last_became_peered":"2024-09-16T09:25:03.714635+0000","last_unstale":"2024-09-16T09:41:49.710722+0000","last_undegraded":"2024-09-16T09:41:49.710722+0000","last_fullsized":"2024-09-16T09:41:49.710722+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'15","last_scrub_stamp":"2024-09-16T09:41:43.064729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:41:43.064729+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:53.613534+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":497,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.713711+0000","last_change":"2024-09-16T09:41:05.861631+0000","last_active":"2024-09-16T09:41:49.713711+0000","last_peered":"2024-09-16T09:41:49.713711+0000","last_clean":"2024-09-16T09:41:49.713711+0000","last_became_active":"2024-09-16T09:33:30.242238+0000","last_became_peered":"2024-09-16T09:33:30.242238+0000","last_unstale":"2024-09-16T09:41:49.713711+0000","last_undegraded":"2024-09-16T09:41:49.713711+0000","last_fullsized":"2024-09-16T09:41:49.713711+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:05.861578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:41:05.861578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:08.589335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":853,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.710365+0000","last_change":"2024-09-16T09:41:31.078139+0000","last_active":"2024-09-16T09:41:49.710365+0000","last_peered":"2024-09-16T09:41:49.710365+0000","last_clean":"2024-09-16T09:41:49.710365+0000","last_became_active":"2024-09-16T09:25:07.715442+0000","last_became_peered":"2024-09-16T09:25:07.715442+0000","last_unstale":"2024-09-16T09:41:49.710365+0000","last_undegraded":"2024-09-16T09:41:49.710365+0000","last_fullsized":"2024-09-16T09:41:49.710365+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:31.078058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:41:31.078058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:32.264866+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":591,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.708630+0000","last_change":"2024-09-16T09:40:31.723526+0000","last_active":"2024-09-16T09:41:49.708630+0000","last_peered":"2024-09-16T09:41:49.708630+0000","last_clean":"2024-09-16T09:41:49.708630+0000","last_became_active":"2024-09-16T09:31:09.269342+0000","last_became_peered":"2024-09-16T09:31:09.269342+0000","last_unstale":"2024-09-16T09:41:49.708630+0000","last_undegraded":"2024-09-16T09:41:49.708630+0000","last_fullsized":"2024-09-16T09:41:49.708630+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:40:31.723468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:40:31.723468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:58.588310+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":732,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.711432+0000","last_change":"2024-09-16T09:39:28.963009+0000","last_active":"2024-09-16T09:41:49.711432+0000","last_peered":"2024-09-16T09:41:49.711432+0000","last_clean":"2024-09-16T09:41:49.711432+0000","last_became_active":"2024-09-16T09:31:09.275136+0000","last_became_peered":"2024-09-16T09:31:09.275136+0000","last_unstale":"2024-09-16T09:41:49.711432+0000","last_undegraded":"2024-09-16T09:41:49.711432+0000","last_fullsized":"2024-09-16T09:41:49.711432+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:39:28.962926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:39:28.962926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:24.910431+0000","scrub_duration":59,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.19","version":"285'27","reported_seq":84,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.709982+0000","last_change":"2024-09-16T09:40:58.802023+0000","last_active":"2024-09-16T09:41:49.709982+0000","last_peered":"2024-09-16T09:41:49.709982+0000","last_clean":"2024-09-16T09:41:49.709982+0000","last_became_active":"2024-09-16T09:40:58.699357+0000","last_became_peered":"2024-09-16T09:40:58.699357+0000","last_unstale":"2024-09-16T09:41:49.709982+0000","last_undegraded":"2024-09-16T09:41:49.709982+0000","last_fullsized":"2024-09-16T09:41:49.709982+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'27","last_scrub_stamp":"2024-09-16T09:37:29.683568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:37:29.683568+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:21.293250+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,5],"acting":[4,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":865,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.710633+0000","last_change":"2024-09-16T09:40:24.111297+0000","last_active":"2024-09-16T09:41:49.710633+0000","last_peered":"2024-09-16T09:41:49.710633+0000","last_clean":"2024-09-16T09:41:49.710633+0000","last_became_active":"2024-09-16T09:25:05.717236+0000","last_became_peered":"2024-09-16T09:25:05.717236+0000","last_unstale":"2024-09-16T09:41:49.710633+0000","last_undegraded":"2024-09-16T09:41:49.710633+0000","last_fullsized":"2024-09-16T09:41:49.710633+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:40:24.111240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:40:24.111240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:51.101297+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":504,"reported_epoch":466,"state":"active+clean","last_fresh":"2024-09-16T09:41:44.423570+0000","last_change":"2024-09-16T09:41:33.135452+0000","last_active":"2024-09-16T09:41:44.423570+0000","last_peered":"2024-09-16T09:41:44.423570+0000","last_clean":"2024-09-16T09:41:44.423570+0000","last_became_active":"2024-09-16T09:33:30.247733+0000","last_became_peered":"2024-09-16T09:33:30.247733+0000","last_unstale":"2024-09-16T09:41:44.423570+0000","last_undegraded":"2024-09-16T09:41:44.423570+0000","last_fullsized":"2024-09-16T09:41:44.423570+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:33.135342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:41:33.135342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:57.435366+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1a","version":"285'15","reported_seq":894,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.710655+0000","last_change":"2024-09-16T09:41:47.100374+0000","last_active":"2024-09-16T09:41:49.710655+0000","last_peered":"2024-09-16T09:41:49.710655+0000","last_clean":"2024-09-16T09:41:49.710655+0000","last_became_active":"2024-09-16T09:25:03.709147+0000","last_became_peered":"2024-09-16T09:25:03.709147+0000","last_unstale":"2024-09-16T09:41:49.710655+0000","last_undegraded":"2024-09-16T09:41:49.710655+0000","last_fullsized":"2024-09-16T09:41:49.710655+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'15","last_scrub_stamp":"2024-09-16T09:41:47.100292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:41:47.100292+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:52.405007+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":848,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.714107+0000","last_change":"2024-09-16T09:36:51.134364+0000","last_active":"2024-09-16T09:41:49.714107+0000","last_peered":"2024-09-16T09:41:49.714107+0000","last_clean":"2024-09-16T09:41:49.714107+0000","last_became_active":"2024-09-16T09:25:05.718134+0000","last_became_peered":"2024-09-16T09:25:05.718134+0000","last_unstale":"2024-09-16T09:41:49.714107+0000","last_undegraded":"2024-09-16T09:41:49.714107+0000","last_fullsized":"2024-09-16T09:41:49.714107+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:51.134314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:36:51.134314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:50.085630+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"66'1","reported_seq":868,"reported_epoch":466,"state":"active+clean","last_fresh":"2024-09-16T09:41:48.089052+0000","last_change":"2024-09-16T09:41:48.089052+0000","last_active":"2024-09-16T09:41:48.089052+0000","last_peered":"2024-09-16T09:41:48.089052+0000","last_clean":"2024-09-16T09:41:48.089052+0000","last_became_active":"2024-09-16T09:25:07.716558+0000","last_became_peered":"2024-09-16T09:25:07.716558+0000","last_unstale":"2024-09-16T09:41:48.089052+0000","last_undegraded":"2024-09-16T09:41:48.089052+0000","last_fullsized":"2024-09-16T09:41:48.089052+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-16T09:41:48.088985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:41:48.088985+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:03.846875+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":848,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.714500+0000","last_change":"2024-09-16T09:33:19.326765+0000","last_active":"2024-09-16T09:41:49.714500+0000","last_peered":"2024-09-16T09:41:49.714500+0000","last_clean":"2024-09-16T09:41:49.714500+0000","last_became_active":"2024-09-16T09:25:01.696939+0000","last_became_peered":"2024-09-16T09:25:01.696939+0000","last_unstale":"2024-09-16T09:41:49.714500+0000","last_undegraded":"2024-09-16T09:41:49.714500+0000","last_fullsized":"2024-09-16T09:41:49.714500+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:19.326695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:19.326695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:21.783748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"285'7","reported_seq":866,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.710833+0000","last_change":"2024-09-16T09:35:39.975915+0000","last_active":"2024-09-16T09:41:49.710833+0000","last_peered":"2024-09-16T09:41:49.710833+0000","last_clean":"2024-09-16T09:41:49.710833+0000","last_became_active":"2024-09-16T09:25:03.716136+0000","last_became_peered":"2024-09-16T09:25:03.716136+0000","last_unstale":"2024-09-16T09:41:49.710833+0000","last_undegraded":"2024-09-16T09:41:49.710833+0000","last_fullsized":"2024-09-16T09:41:49.710833+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'7","last_scrub_stamp":"2024-09-16T09:35:39.975840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:35:39.975840+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:56.253730+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":10,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":846,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.717489+0000","last_change":"2024-09-16T09:39:26.850780+0000","last_active":"2024-09-16T09:41:49.717489+0000","last_peered":"2024-09-16T09:41:49.717489+0000","last_clean":"2024-09-16T09:41:49.717489+0000","last_became_active":"2024-09-16T09:25:05.718281+0000","last_became_peered":"2024-09-16T09:25:05.718281+0000","last_unstale":"2024-09-16T09:41:49.717489+0000","last_undegraded":"2024-09-16T09:41:49.717489+0000","last_fullsized":"2024-09-16T09:41:49.717489+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:39:26.850685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:39:26.850685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:54.255414+0000","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":874,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.713782+0000","last_change":"2024-09-16T09:38:34.050374+0000","last_active":"2024-09-16T09:41:49.713782+0000","last_peered":"2024-09-16T09:41:49.713782+0000","last_clean":"2024-09-16T09:41:49.713782+0000","last_became_active":"2024-09-16T09:31:09.275656+0000","last_became_peered":"2024-09-16T09:31:09.275656+0000","last_unstale":"2024-09-16T09:41:49.713782+0000","last_undegraded":"2024-09-16T09:41:49.713782+0000","last_fullsized":"2024-09-16T09:41:49.713782+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:38:34.050185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:38:34.050185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":855,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.711688+0000","last_change":"2024-09-16T09:33:48.322010+0000","last_active":"2024-09-16T09:41:49.711688+0000","last_peered":"2024-09-16T09:41:49.711688+0000","last_clean":"2024-09-16T09:41:49.711688+0000","last_became_active":"2024-09-16T09:25:07.720793+0000","last_became_peered":"2024-09-16T09:25:07.720793+0000","last_unstale":"2024-09-16T09:41:49.711688+0000","last_undegraded":"2024-09-16T09:41:49.711688+0000","last_fullsized":"2024-09-16T09:41:49.711688+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:48.321895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:33:48.321895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:52.239128+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":605,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.709317+0000","last_change":"2024-09-16T09:34:33.866299+0000","last_active":"2024-09-16T09:41:49.709317+0000","last_peered":"2024-09-16T09:41:49.709317+0000","last_clean":"2024-09-16T09:41:49.709317+0000","last_became_active":"2024-09-16T09:30:39.201824+0000","last_became_peered":"2024-09-16T09:30:39.201824+0000","last_unstale":"2024-09-16T09:41:49.709317+0000","last_undegraded":"2024-09-16T09:41:49.709317+0000","last_fullsized":"2024-09-16T09:41:49.709317+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:33.866142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:34:33.866142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:22.864352+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"285'27","reported_seq":962,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.710969+0000","last_change":"2024-09-16T09:40:23.465274+0000","last_active":"2024-09-16T09:41:49.710969+0000","last_peered":"2024-09-16T09:41:49.710969+0000","last_clean":"2024-09-16T09:41:49.710969+0000","last_became_active":"2024-09-16T09:33:30.249798+0000","last_became_peered":"2024-09-16T09:33:30.249798+0000","last_unstale":"2024-09-16T09:41:49.710969+0000","last_undegraded":"2024-09-16T09:41:49.710969+0000","last_fullsized":"2024-09-16T09:41:49.710969+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'27","last_scrub_stamp":"2024-09-16T09:40:23.465200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:40:23.465200+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":489,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.711917+0000","last_change":"2024-09-16T09:33:43.326933+0000","last_active":"2024-09-16T09:41:49.711917+0000","last_peered":"2024-09-16T09:41:49.711917+0000","last_clean":"2024-09-16T09:41:49.711917+0000","last_became_active":"2024-09-16T09:33:30.251587+0000","last_became_peered":"2024-09-16T09:33:30.251587+0000","last_unstale":"2024-09-16T09:41:49.711917+0000","last_undegraded":"2024-09-16T09:41:49.711917+0000","last_fullsized":"2024-09-16T09:41:49.711917+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:43.326852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:43.326852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:53.260256+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,0],"acting":[4,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":711,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.711514+0000","last_change":"2024-09-16T09:40:27.916994+0000","last_active":"2024-09-16T09:41:49.711514+0000","last_peered":"2024-09-16T09:41:49.711514+0000","last_clean":"2024-09-16T09:41:49.711514+0000","last_became_active":"2024-09-16T09:28:52.974255+0000","last_became_peered":"2024-09-16T09:28:52.974255+0000","last_unstale":"2024-09-16T09:41:49.711514+0000","last_undegraded":"2024-09-16T09:41:49.711514+0000","last_fullsized":"2024-09-16T09:41:49.711514+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:40:27.916860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:40:27.916860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:56.058958+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":873,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.710723+0000","last_change":"2024-09-16T09:41:08.223235+0000","last_active":"2024-09-16T09:41:49.710723+0000","last_peered":"2024-09-16T09:41:49.710723+0000","last_clean":"2024-09-16T09:41:49.710723+0000","last_became_active":"2024-09-16T09:25:01.706491+0000","last_became_peered":"2024-09-16T09:25:01.706491+0000","last_unstale":"2024-09-16T09:41:49.710723+0000","last_undegraded":"2024-09-16T09:41:49.710723+0000","last_fullsized":"2024-09-16T09:41:49.710723+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:08.223190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:41:08.223190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:23.681454+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"285'18","reported_seq":549,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.712035+0000","last_change":"2024-09-16T09:40:37.206512+0000","last_active":"2024-09-16T09:41:49.712035+0000","last_peered":"2024-09-16T09:41:49.712035+0000","last_clean":"2024-09-16T09:41:49.712035+0000","last_became_active":"2024-09-16T09:33:30.251626+0000","last_became_peered":"2024-09-16T09:33:30.251626+0000","last_unstale":"2024-09-16T09:41:49.712035+0000","last_undegraded":"2024-09-16T09:41:49.712035+0000","last_fullsized":"2024-09-16T09:41:49.712035+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'18","last_scrub_stamp":"2024-09-16T09:40:37.206432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:40:37.206432+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:59.565226+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":33,"num_read_kb":24,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":849,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.711079+0000","last_change":"2024-09-16T09:41:45.118370+0000","last_active":"2024-09-16T09:41:49.711079+0000","last_peered":"2024-09-16T09:41:49.711079+0000","last_clean":"2024-09-16T09:41:49.711079+0000","last_became_active":"2024-09-16T09:25:05.710093+0000","last_became_peered":"2024-09-16T09:25:05.710093+0000","last_unstale":"2024-09-16T09:41:49.711079+0000","last_undegraded":"2024-09-16T09:41:49.711079+0000","last_fullsized":"2024-09-16T09:41:49.711079+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:45.118305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:41:45.118305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:14.199468+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":851,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.717975+0000","last_change":"2024-09-16T09:37:48.412421+0000","last_active":"2024-09-16T09:41:49.717975+0000","last_peered":"2024-09-16T09:41:49.717975+0000","last_clean":"2024-09-16T09:41:49.717975+0000","last_became_active":"2024-09-16T09:25:07.719380+0000","last_became_peered":"2024-09-16T09:25:07.719380+0000","last_unstale":"2024-09-16T09:41:49.717975+0000","last_undegraded":"2024-09-16T09:41:49.717975+0000","last_fullsized":"2024-09-16T09:41:49.717975+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-16T09:37:48.411054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:37:48.411054+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:24.108878+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":514,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.707350+0000","last_change":"2024-09-16T09:41:36.612463+0000","last_active":"2024-09-16T09:41:49.707350+0000","last_peered":"2024-09-16T09:41:49.707350+0000","last_clean":"2024-09-16T09:41:49.707350+0000","last_became_active":"2024-09-16T09:33:30.245048+0000","last_became_peered":"2024-09-16T09:33:30.245048+0000","last_unstale":"2024-09-16T09:41:49.707350+0000","last_undegraded":"2024-09-16T09:41:49.707350+0000","last_fullsized":"2024-09-16T09:41:49.707350+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:41:36.612280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:41:36.612280+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:02.315301+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"285'20","reported_seq":929,"reported_epoch":466,"state":"active+clean","last_fresh":"2024-09-16T09:41:44.427960+0000","last_change":"2024-09-16T09:39:35.550760+0000","last_active":"2024-09-16T09:41:44.427960+0000","last_peered":"2024-09-16T09:41:44.427960+0000","last_clean":"2024-09-16T09:41:44.427960+0000","last_became_active":"2024-09-16T09:31:09.275050+0000","last_became_peered":"2024-09-16T09:31:09.275050+0000","last_unstale":"2024-09-16T09:41:44.427960+0000","last_undegraded":"2024-09-16T09:41:44.427960+0000","last_fullsized":"2024-09-16T09:41:44.427960+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'20","last_scrub_stamp":"2024-09-16T09:39:35.548333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:39:35.548333+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:52.946522+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"66'8","reported_seq":1097,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.711730+0000","last_change":"2024-09-16T09:40:38.007095+0000","last_active":"2024-09-16T09:41:49.711730+0000","last_peered":"2024-09-16T09:41:49.711730+0000","last_clean":"2024-09-16T09:41:49.711730+0000","last_became_active":"2024-09-16T09:25:05.712516+0000","last_became_peered":"2024-09-16T09:25:05.712516+0000","last_unstale":"2024-09-16T09:41:49.711730+0000","last_undegraded":"2024-09-16T09:41:49.711730+0000","last_fullsized":"2024-09-16T09:41:49.711730+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:40:38.007034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:40:38.007034+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:01.522582+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.0","version":"467'4084","reported_seq":4120,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:52.513134+0000","last_change":"2024-09-16T09:41:13.139275+0000","last_active":"2024-09-16T09:41:52.513134+0000","last_peered":"2024-09-16T09:41:52.513134+0000","last_clean":"2024-09-16T09:41:52.513134+0000","last_became_active":"2024-09-16T09:41:13.138983+0000","last_became_peered":"2024-09-16T09:41:13.138983+0000","last_unstale":"2024-09-16T09:41:52.513134+0000","last_undegraded":"2024-09-16T09:41:52.513134+0000","last_fullsized":"2024-09-16T09:41:52.513134+0000","mapping_epoch":454,"log_start":"0'0","ondisk_log_start":"0'0","created":454,"last_epoch_clean":455,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:11.976785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:41:11.976785+0000","last_clean_scrub_stamp":"2024-09-16T09:41:11.976785+0000","objects_scrubbed":0,"log_size":4084,"log_dups_size":0,"ondisk_log_size":4084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:27.450124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":267649024,"num_objects":4084,"num_object_clones":0,"num_object_copies":12252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4084,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4084,"num_write_kb":261376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3],"acting":[7,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.a","version":"73'79","reported_seq":797,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.713586+0000","last_change":"2024-09-16T09:41:28.733947+0000","last_active":"2024-09-16T09:41:49.713586+0000","last_peered":"2024-09-16T09:41:49.713586+0000","last_clean":"2024-09-16T09:41:49.713586+0000","last_became_active":"2024-09-16T09:26:35.127352+0000","last_became_peered":"2024-09-16T09:26:35.127352+0000","last_unstale":"2024-09-16T09:41:49.713586+0000","last_undegraded":"2024-09-16T09:41:49.713586+0000","last_fullsized":"2024-09-16T09:41:49.713586+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:41:28.733854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:41:28.733854+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:35.418965+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"285'27","reported_seq":832,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.709406+0000","last_change":"2024-09-16T09:41:28.675906+0000","last_active":"2024-09-16T09:41:49.709406+0000","last_peered":"2024-09-16T09:41:49.709406+0000","last_clean":"2024-09-16T09:41:49.709406+0000","last_became_active":"2024-09-16T09:26:33.295672+0000","last_became_peered":"2024-09-16T09:26:33.295672+0000","last_unstale":"2024-09-16T09:41:49.709406+0000","last_undegraded":"2024-09-16T09:41:49.709406+0000","last_fullsized":"2024-09-16T09:41:49.709406+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'27","last_scrub_stamp":"2024-09-16T09:41:28.675741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:41:28.675741+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:47.696328+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":599,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.712185+0000","last_change":"2024-09-16T09:38:35.023594+0000","last_active":"2024-09-16T09:41:49.712185+0000","last_peered":"2024-09-16T09:41:49.712185+0000","last_clean":"2024-09-16T09:41:49.712185+0000","last_became_active":"2024-09-16T09:31:09.274534+0000","last_became_peered":"2024-09-16T09:31:09.274534+0000","last_unstale":"2024-09-16T09:41:49.712185+0000","last_undegraded":"2024-09-16T09:41:49.712185+0000","last_fullsized":"2024-09-16T09:41:49.712185+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:38:35.023518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:38:35.023518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:59.827793+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"66'8","reported_seq":1061,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.711815+0000","last_change":"2024-09-16T09:41:32.036261+0000","last_active":"2024-09-16T09:41:49.711815+0000","last_peered":"2024-09-16T09:41:49.711815+0000","last_clean":"2024-09-16T09:41:49.711815+0000","last_became_active":"2024-09-16T09:25:05.713813+0000","last_became_peered":"2024-09-16T09:25:05.713813+0000","last_unstale":"2024-09-16T09:41:49.711815+0000","last_undegraded":"2024-09-16T09:41:49.711815+0000","last_fullsized":"2024-09-16T09:41:49.711815+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:41:32.036192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:41:32.036192+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:57.987875+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":861,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.713955+0000","last_change":"2024-09-16T09:41:48.730584+0000","last_active":"2024-09-16T09:41:49.713955+0000","last_peered":"2024-09-16T09:41:49.713955+0000","last_clean":"2024-09-16T09:41:49.713955+0000","last_became_active":"2024-09-16T09:25:07.717344+0000","last_became_peered":"2024-09-16T09:25:07.717344+0000","last_unstale":"2024-09-16T09:41:49.713955+0000","last_undegraded":"2024-09-16T09:41:49.713955+0000","last_fullsized":"2024-09-16T09:41:49.713955+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:48.730521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:41:48.730521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:08.160356+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.f","version":"467'3978","reported_seq":4014,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:52.505112+0000","last_change":"2024-09-16T09:41:13.139299+0000","last_active":"2024-09-16T09:41:52.505112+0000","last_peered":"2024-09-16T09:41:52.505112+0000","last_clean":"2024-09-16T09:41:52.505112+0000","last_became_active":"2024-09-16T09:41:13.138830+0000","last_became_peered":"2024-09-16T09:41:13.138830+0000","last_unstale":"2024-09-16T09:41:52.505112+0000","last_undegraded":"2024-09-16T09:41:52.505112+0000","last_fullsized":"2024-09-16T09:41:52.505112+0000","mapping_epoch":454,"log_start":"0'0","ondisk_log_start":"0'0","created":454,"last_epoch_clean":455,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:11.976785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:41:11.976785+0000","last_clean_scrub_stamp":"2024-09-16T09:41:11.976785+0000","objects_scrubbed":0,"log_size":3978,"log_dups_size":0,"ondisk_log_size":3978,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:38.677682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":260702208,"num_objects":3978,"num_object_clones":0,"num_object_copies":11934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3978,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3978,"num_write_kb":254592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.5","version":"73'79","reported_seq":829,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.712196+0000","last_change":"2024-09-16T09:40:29.124394+0000","last_active":"2024-09-16T09:41:49.712196+0000","last_peered":"2024-09-16T09:41:49.712196+0000","last_clean":"2024-09-16T09:41:49.712196+0000","last_became_active":"2024-09-16T09:30:08.763683+0000","last_became_peered":"2024-09-16T09:30:08.763683+0000","last_unstale":"2024-09-16T09:41:49.712196+0000","last_undegraded":"2024-09-16T09:41:49.712196+0000","last_fullsized":"2024-09-16T09:41:49.712196+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:40:29.124314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:40:29.124314+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:55.225417+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"285'30","reported_seq":828,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.716435+0000","last_change":"2024-09-16T09:41:41.151427+0000","last_active":"2024-09-16T09:41:49.716435+0000","last_peered":"2024-09-16T09:41:49.716435+0000","last_clean":"2024-09-16T09:41:49.716435+0000","last_became_active":"2024-09-16T09:31:09.270434+0000","last_became_peered":"2024-09-16T09:31:09.270434+0000","last_unstale":"2024-09-16T09:41:49.716435+0000","last_undegraded":"2024-09-16T09:41:49.716435+0000","last_fullsized":"2024-09-16T09:41:49.716435+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'30","last_scrub_stamp":"2024-09-16T09:41:41.151345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:41:41.151345+0000","objects_scrubbed":8,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:55.242044+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":51,"num_read_kb":39,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":708,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.718039+0000","last_change":"2024-09-16T09:41:45.162761+0000","last_active":"2024-09-16T09:41:49.718039+0000","last_peered":"2024-09-16T09:41:49.718039+0000","last_clean":"2024-09-16T09:41:49.718039+0000","last_became_active":"2024-09-16T09:28:52.983543+0000","last_became_peered":"2024-09-16T09:28:52.983543+0000","last_unstale":"2024-09-16T09:41:49.718039+0000","last_undegraded":"2024-09-16T09:41:49.718039+0000","last_fullsized":"2024-09-16T09:41:49.718039+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:45.162688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:41:45.162688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:01.911633+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":873,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.710970+0000","last_change":"2024-09-16T09:35:19.822713+0000","last_active":"2024-09-16T09:41:49.710970+0000","last_peered":"2024-09-16T09:41:49.710970+0000","last_clean":"2024-09-16T09:41:49.710970+0000","last_became_active":"2024-09-16T09:31:09.268864+0000","last_became_peered":"2024-09-16T09:31:09.268864+0000","last_unstale":"2024-09-16T09:41:49.710970+0000","last_undegraded":"2024-09-16T09:41:49.710970+0000","last_fullsized":"2024-09-16T09:41:49.710970+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:35:19.822635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:35:19.822635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:01.276296+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":877,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.710572+0000","last_change":"2024-09-16T09:41:33.051473+0000","last_active":"2024-09-16T09:41:49.710572+0000","last_peered":"2024-09-16T09:41:49.710572+0000","last_clean":"2024-09-16T09:41:49.710572+0000","last_became_active":"2024-09-16T09:31:09.269433+0000","last_became_peered":"2024-09-16T09:31:09.269433+0000","last_unstale":"2024-09-16T09:41:49.710572+0000","last_undegraded":"2024-09-16T09:41:49.710572+0000","last_fullsized":"2024-09-16T09:41:49.710572+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:33.051405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:41:33.051405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:02.030034+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"467'4096","reported_seq":4133,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:53.116015+0000","last_change":"2024-09-16T09:41:13.140372+0000","last_active":"2024-09-16T09:41:53.116015+0000","last_peered":"2024-09-16T09:41:53.116015+0000","last_clean":"2024-09-16T09:41:53.116015+0000","last_became_active":"2024-09-16T09:41:13.140053+0000","last_became_peered":"2024-09-16T09:41:13.140053+0000","last_unstale":"2024-09-16T09:41:53.116015+0000","last_undegraded":"2024-09-16T09:41:53.116015+0000","last_fullsized":"2024-09-16T09:41:53.116015+0000","mapping_epoch":454,"log_start":"0'0","ondisk_log_start":"0'0","created":454,"last_epoch_clean":455,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:11.976785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:41:11.976785+0000","last_clean_scrub_stamp":"2024-09-16T09:41:11.976785+0000","objects_scrubbed":0,"log_size":4096,"log_dups_size":0,"ondisk_log_size":4096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:23.943934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":268435456,"num_objects":4096,"num_object_clones":0,"num_object_copies":12288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4096,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4096,"num_write_kb":262144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,7],"acting":[4,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"73'79","reported_seq":600,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.708999+0000","last_change":"2024-09-16T09:41:44.602634+0000","last_active":"2024-09-16T09:41:49.708999+0000","last_peered":"2024-09-16T09:41:49.708999+0000","last_clean":"2024-09-16T09:41:49.708999+0000","last_became_active":"2024-09-16T09:31:09.275139+0000","last_became_peered":"2024-09-16T09:31:09.275139+0000","last_unstale":"2024-09-16T09:41:49.708999+0000","last_undegraded":"2024-09-16T09:41:49.708999+0000","last_fullsized":"2024-09-16T09:41:49.708999+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:41:44.602557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:41:44.602557+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:02.511638+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"285'28","reported_seq":913,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.710220+0000","last_change":"2024-09-16T09:41:11.201590+0000","last_active":"2024-09-16T09:41:49.710220+0000","last_peered":"2024-09-16T09:41:49.710220+0000","last_clean":"2024-09-16T09:41:49.710220+0000","last_became_active":"2024-09-16T09:25:03.708830+0000","last_became_peered":"2024-09-16T09:25:03.708830+0000","last_unstale":"2024-09-16T09:41:49.710220+0000","last_undegraded":"2024-09-16T09:41:49.710220+0000","last_fullsized":"2024-09-16T09:41:49.710220+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'28","last_scrub_stamp":"2024-09-16T09:41:11.201523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:41:11.201523+0000","objects_scrubbed":7,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:16.854189+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":889,"reported_epoch":466,"state":"active+clean","last_fresh":"2024-09-16T09:41:44.428286+0000","last_change":"2024-09-16T09:40:31.037325+0000","last_active":"2024-09-16T09:41:44.428286+0000","last_peered":"2024-09-16T09:41:44.428286+0000","last_clean":"2024-09-16T09:41:44.428286+0000","last_became_active":"2024-09-16T09:25:01.702992+0000","last_became_peered":"2024-09-16T09:25:01.702992+0000","last_unstale":"2024-09-16T09:41:44.428286+0000","last_undegraded":"2024-09-16T09:41:44.428286+0000","last_fullsized":"2024-09-16T09:41:44.428286+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:40:31.037258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:40:31.037258+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:51.866286+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"235'12","reported_seq":617,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.710661+0000","last_change":"2024-09-16T09:37:19.380567+0000","last_active":"2024-09-16T09:41:49.710661+0000","last_peered":"2024-09-16T09:41:49.710661+0000","last_clean":"2024-09-16T09:41:49.710661+0000","last_became_active":"2024-09-16T09:33:30.251047+0000","last_became_peered":"2024-09-16T09:33:30.251047+0000","last_unstale":"2024-09-16T09:41:49.710661+0000","last_undegraded":"2024-09-16T09:41:49.710661+0000","last_fullsized":"2024-09-16T09:41:49.710661+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"235'12","last_scrub_stamp":"2024-09-16T09:37:19.380504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:37:19.380504+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:20.256135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":852,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.717893+0000","last_change":"2024-09-16T09:40:30.956991+0000","last_active":"2024-09-16T09:41:49.717893+0000","last_peered":"2024-09-16T09:41:49.717893+0000","last_clean":"2024-09-16T09:41:49.717893+0000","last_became_active":"2024-09-16T09:25:07.722017+0000","last_became_peered":"2024-09-16T09:25:07.722017+0000","last_unstale":"2024-09-16T09:41:49.717893+0000","last_undegraded":"2024-09-16T09:41:49.717893+0000","last_fullsized":"2024-09-16T09:41:49.717893+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:40:30.956926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:40:30.956926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.d","version":"467'4069","reported_seq":4106,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:53.071866+0000","last_change":"2024-09-16T09:41:13.005502+0000","last_active":"2024-09-16T09:41:53.071866+0000","last_peered":"2024-09-16T09:41:53.071866+0000","last_clean":"2024-09-16T09:41:53.071866+0000","last_became_active":"2024-09-16T09:41:13.005224+0000","last_became_peered":"2024-09-16T09:41:13.005224+0000","last_unstale":"2024-09-16T09:41:53.071866+0000","last_undegraded":"2024-09-16T09:41:53.071866+0000","last_fullsized":"2024-09-16T09:41:53.071866+0000","mapping_epoch":454,"log_start":"0'0","ondisk_log_start":"0'0","created":454,"last_epoch_clean":455,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:11.976785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:41:11.976785+0000","last_clean_scrub_stamp":"2024-09-16T09:41:11.976785+0000","objects_scrubbed":0,"log_size":4069,"log_dups_size":0,"ondisk_log_size":4069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:17.246036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":266665984,"num_objects":4069,"num_object_clones":0,"num_object_copies":12207,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4069,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4069,"num_write_kb":260416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"73'79","reported_seq":931,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.711437+0000","last_change":"2024-09-16T09:41:10.908936+0000","last_active":"2024-09-16T09:41:49.711437+0000","last_peered":"2024-09-16T09:41:49.711437+0000","last_clean":"2024-09-16T09:41:49.711437+0000","last_became_active":"2024-09-16T09:31:09.274707+0000","last_became_peered":"2024-09-16T09:31:09.274707+0000","last_unstale":"2024-09-16T09:41:49.711437+0000","last_undegraded":"2024-09-16T09:41:49.711437+0000","last_fullsized":"2024-09-16T09:41:49.711437+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:41:10.908852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:41:10.908852+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:22.406722+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"285'20","reported_seq":548,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.714112+0000","last_change":"2024-09-16T09:38:32.122419+0000","last_active":"2024-09-16T09:41:49.714112+0000","last_peered":"2024-09-16T09:41:49.714112+0000","last_clean":"2024-09-16T09:41:49.714112+0000","last_became_active":"2024-09-16T09:33:30.248453+0000","last_became_peered":"2024-09-16T09:33:30.248453+0000","last_unstale":"2024-09-16T09:41:49.714112+0000","last_undegraded":"2024-09-16T09:41:49.714112+0000","last_fullsized":"2024-09-16T09:41:49.714112+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'20","last_scrub_stamp":"2024-09-16T09:38:32.122303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:38:32.122303+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:49.721122+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":872,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.714447+0000","last_change":"2024-09-16T09:36:47.095703+0000","last_active":"2024-09-16T09:41:49.714447+0000","last_peered":"2024-09-16T09:41:49.714447+0000","last_clean":"2024-09-16T09:41:49.714447+0000","last_became_active":"2024-09-16T09:31:09.270619+0000","last_became_peered":"2024-09-16T09:31:09.270619+0000","last_unstale":"2024-09-16T09:41:49.714447+0000","last_undegraded":"2024-09-16T09:41:49.714447+0000","last_fullsized":"2024-09-16T09:41:49.714447+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:47.095658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:36:47.095658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:55.009889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"66'8","reported_seq":1059,"reported_epoch":466,"state":"active+clean","last_fresh":"2024-09-16T09:41:46.950431+0000","last_change":"2024-09-16T09:36:52.689167+0000","last_active":"2024-09-16T09:41:46.950431+0000","last_peered":"2024-09-16T09:41:46.950431+0000","last_clean":"2024-09-16T09:41:46.950431+0000","last_became_active":"2024-09-16T09:25:05.714507+0000","last_became_peered":"2024-09-16T09:25:05.714507+0000","last_unstale":"2024-09-16T09:41:46.950431+0000","last_undegraded":"2024-09-16T09:41:46.950431+0000","last_fullsized":"2024-09-16T09:41:46.950431+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:36:52.689091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:36:52.689091+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:13.970400+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":875,"reported_epoch":466,"state":"active+clean","last_fresh":"2024-09-16T09:41:44.423056+0000","last_change":"2024-09-16T09:41:31.179732+0000","last_active":"2024-09-16T09:41:44.423056+0000","last_peered":"2024-09-16T09:41:44.423056+0000","last_clean":"2024-09-16T09:41:44.423056+0000","last_became_active":"2024-09-16T09:31:09.283192+0000","last_became_peered":"2024-09-16T09:31:09.283192+0000","last_unstale":"2024-09-16T09:41:44.423056+0000","last_undegraded":"2024-09-16T09:41:44.423056+0000","last_fullsized":"2024-09-16T09:41:44.423056+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:31.179662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:41:31.179662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:39.168681+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.c","version":"467'4073","reported_seq":4110,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:53.281947+0000","last_change":"2024-09-16T09:41:13.139322+0000","last_active":"2024-09-16T09:41:53.281947+0000","last_peered":"2024-09-16T09:41:53.281947+0000","last_clean":"2024-09-16T09:41:53.281947+0000","last_became_active":"2024-09-16T09:41:13.138917+0000","last_became_peered":"2024-09-16T09:41:13.138917+0000","last_unstale":"2024-09-16T09:41:53.281947+0000","last_undegraded":"2024-09-16T09:41:53.281947+0000","last_fullsized":"2024-09-16T09:41:53.281947+0000","mapping_epoch":454,"log_start":"0'0","ondisk_log_start":"0'0","created":454,"last_epoch_clean":455,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:11.976785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:41:11.976785+0000","last_clean_scrub_stamp":"2024-09-16T09:41:11.976785+0000","objects_scrubbed":0,"log_size":4073,"log_dups_size":0,"ondisk_log_size":4073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:18.800527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":266928128,"num_objects":4073,"num_object_clones":0,"num_object_copies":12219,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4073,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4073,"num_write_kb":260672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"73'79","reported_seq":888,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.710213+0000","last_change":"2024-09-16T09:34:42.728232+0000","last_active":"2024-09-16T09:41:49.710213+0000","last_peered":"2024-09-16T09:41:49.710213+0000","last_clean":"2024-09-16T09:41:49.710213+0000","last_became_active":"2024-09-16T09:31:09.274565+0000","last_became_peered":"2024-09-16T09:31:09.274565+0000","last_unstale":"2024-09-16T09:41:49.710213+0000","last_undegraded":"2024-09-16T09:41:49.710213+0000","last_fullsized":"2024-09-16T09:41:49.710213+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:34:42.728143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:34:42.728143+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:20.880320+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"285'29","reported_seq":904,"reported_epoch":466,"state":"active+clean","last_fresh":"2024-09-16T09:41:44.423352+0000","last_change":"2024-09-16T09:36:54.604573+0000","last_active":"2024-09-16T09:41:44.423352+0000","last_peered":"2024-09-16T09:41:44.423352+0000","last_clean":"2024-09-16T09:41:44.423352+0000","last_became_active":"2024-09-16T09:25:03.714797+0000","last_became_peered":"2024-09-16T09:25:03.714797+0000","last_unstale":"2024-09-16T09:41:44.423352+0000","last_undegraded":"2024-09-16T09:41:44.423352+0000","last_fullsized":"2024-09-16T09:41:44.423352+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'29","last_scrub_stamp":"2024-09-16T09:36:54.604484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:36:54.604484+0000","objects_scrubbed":8,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:19.009098+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":54,"num_read_kb":40,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":490,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.714414+0000","last_change":"2024-09-16T09:40:26.273165+0000","last_active":"2024-09-16T09:41:49.714414+0000","last_peered":"2024-09-16T09:41:49.714414+0000","last_clean":"2024-09-16T09:41:49.714414+0000","last_became_active":"2024-09-16T09:33:30.246996+0000","last_became_peered":"2024-09-16T09:33:30.246996+0000","last_unstale":"2024-09-16T09:41:49.714414+0000","last_undegraded":"2024-09-16T09:41:49.714414+0000","last_fullsized":"2024-09-16T09:41:49.714414+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:40:26.273106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:40:26.273106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,2],"acting":[5,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":883,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.709674+0000","last_change":"2024-09-16T09:41:46.974010+0000","last_active":"2024-09-16T09:41:49.709674+0000","last_peered":"2024-09-16T09:41:49.709674+0000","last_clean":"2024-09-16T09:41:49.709674+0000","last_became_active":"2024-09-16T09:25:05.708663+0000","last_became_peered":"2024-09-16T09:25:05.708663+0000","last_unstale":"2024-09-16T09:41:49.709674+0000","last_undegraded":"2024-09-16T09:41:49.709674+0000","last_fullsized":"2024-09-16T09:41:49.709674+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:46.973947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:41:46.973947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:14.396908+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":498,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.709254+0000","last_change":"2024-09-16T09:37:56.882475+0000","last_active":"2024-09-16T09:41:49.709254+0000","last_peered":"2024-09-16T09:41:49.709254+0000","last_clean":"2024-09-16T09:41:49.709254+0000","last_became_active":"2024-09-16T09:33:30.250178+0000","last_became_peered":"2024-09-16T09:33:30.250178+0000","last_unstale":"2024-09-16T09:41:49.709254+0000","last_undegraded":"2024-09-16T09:41:49.709254+0000","last_fullsized":"2024-09-16T09:41:49.709254+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:56.882326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:37:56.882326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:59.024535+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,2],"acting":[6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.b","version":"467'4107","reported_seq":4144,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:53.650856+0000","last_change":"2024-09-16T09:41:13.140706+0000","last_active":"2024-09-16T09:41:53.650856+0000","last_peered":"2024-09-16T09:41:53.650856+0000","last_clean":"2024-09-16T09:41:53.650856+0000","last_became_active":"2024-09-16T09:41:13.140363+0000","last_became_peered":"2024-09-16T09:41:13.140363+0000","last_unstale":"2024-09-16T09:41:53.650856+0000","last_undegraded":"2024-09-16T09:41:53.650856+0000","last_fullsized":"2024-09-16T09:41:53.650856+0000","mapping_epoch":454,"log_start":"0'0","ondisk_log_start":"0'0","created":454,"last_epoch_clean":455,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:11.976785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:41:11.976785+0000","last_clean_scrub_stamp":"2024-09-16T09:41:11.976785+0000","objects_scrubbed":0,"log_size":4107,"log_dups_size":0,"ondisk_log_size":4107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:17.999413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":269156352,"num_objects":4107,"num_object_clones":0,"num_object_copies":12321,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4107,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4107,"num_write_kb":262848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"73'79","reported_seq":829,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.713555+0000","last_change":"2024-09-16T09:41:06.841984+0000","last_active":"2024-09-16T09:41:49.713555+0000","last_peered":"2024-09-16T09:41:49.713555+0000","last_clean":"2024-09-16T09:41:49.713555+0000","last_became_active":"2024-09-16T09:30:08.766350+0000","last_became_peered":"2024-09-16T09:30:08.766350+0000","last_unstale":"2024-09-16T09:41:49.713555+0000","last_undegraded":"2024-09-16T09:41:49.713555+0000","last_fullsized":"2024-09-16T09:41:49.713555+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:41:06.841934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:41:06.841934+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:19.225514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"285'42","reported_seq":1206,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.716075+0000","last_change":"2024-09-16T09:37:57.407386+0000","last_active":"2024-09-16T09:41:49.716075+0000","last_peered":"2024-09-16T09:41:49.716075+0000","last_clean":"2024-09-16T09:41:49.716075+0000","last_became_active":"2024-09-16T09:31:09.278839+0000","last_became_peered":"2024-09-16T09:31:09.278839+0000","last_unstale":"2024-09-16T09:41:49.716075+0000","last_undegraded":"2024-09-16T09:41:49.716075+0000","last_fullsized":"2024-09-16T09:41:49.716075+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'42","last_scrub_stamp":"2024-09-16T09:37:57.407297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:37:57.407297+0000","objects_scrubbed":10,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:22.315877+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":66,"num_read_kb":51,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":883,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.711027+0000","last_change":"2024-09-16T09:41:40.124915+0000","last_active":"2024-09-16T09:41:49.711027+0000","last_peered":"2024-09-16T09:41:49.711027+0000","last_clean":"2024-09-16T09:41:49.711027+0000","last_became_active":"2024-09-16T09:25:01.703076+0000","last_became_peered":"2024-09-16T09:25:01.703076+0000","last_unstale":"2024-09-16T09:41:49.711027+0000","last_undegraded":"2024-09-16T09:41:49.711027+0000","last_fullsized":"2024-09-16T09:41:49.711027+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:40.124830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:41:40.124830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:54.216409+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":884,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.716210+0000","last_change":"2024-09-16T09:41:43.807570+0000","last_active":"2024-09-16T09:41:49.716210+0000","last_peered":"2024-09-16T09:41:49.716210+0000","last_clean":"2024-09-16T09:41:49.716210+0000","last_became_active":"2024-09-16T09:25:00.696237+0000","last_became_peered":"2024-09-16T09:25:00.696237+0000","last_unstale":"2024-09-16T09:41:49.716210+0000","last_undegraded":"2024-09-16T09:41:49.716210+0000","last_fullsized":"2024-09-16T09:41:49.716210+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-16T09:41:43.807521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:41:43.807521+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:02.549457+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.0025137430000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":840,"reported_epoch":466,"state":"active+clean","last_fresh":"2024-09-16T09:41:44.424527+0000","last_change":"2024-09-16T09:39:39.498275+0000","last_active":"2024-09-16T09:41:44.424527+0000","last_peered":"2024-09-16T09:41:44.424527+0000","last_clean":"2024-09-16T09:41:44.424527+0000","last_became_active":"2024-09-16T09:25:05.717591+0000","last_became_peered":"2024-09-16T09:25:05.717591+0000","last_unstale":"2024-09-16T09:41:44.424527+0000","last_undegraded":"2024-09-16T09:41:44.424527+0000","last_fullsized":"2024-09-16T09:41:44.424527+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:39:39.498205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:39:39.498205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:55.945040+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":591,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.708597+0000","last_change":"2024-09-16T09:38:25.992634+0000","last_active":"2024-09-16T09:41:49.708597+0000","last_peered":"2024-09-16T09:41:49.708597+0000","last_clean":"2024-09-16T09:41:49.708597+0000","last_became_active":"2024-09-16T09:31:09.272609+0000","last_became_peered":"2024-09-16T09:31:09.272609+0000","last_unstale":"2024-09-16T09:41:49.708597+0000","last_undegraded":"2024-09-16T09:41:49.708597+0000","last_fullsized":"2024-09-16T09:41:49.708597+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:38:25.992569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:38:25.992569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:54.875302+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.8","version":"467'3941","reported_seq":3977,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:52.955092+0000","last_change":"2024-09-16T09:41:13.001905+0000","last_active":"2024-09-16T09:41:52.955092+0000","last_peered":"2024-09-16T09:41:52.955092+0000","last_clean":"2024-09-16T09:41:52.955092+0000","last_became_active":"2024-09-16T09:41:13.001618+0000","last_became_peered":"2024-09-16T09:41:13.001618+0000","last_unstale":"2024-09-16T09:41:52.955092+0000","last_undegraded":"2024-09-16T09:41:52.955092+0000","last_fullsized":"2024-09-16T09:41:52.955092+0000","mapping_epoch":454,"log_start":"0'0","ondisk_log_start":"0'0","created":454,"last_epoch_clean":455,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:11.976785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:41:11.976785+0000","last_clean_scrub_stamp":"2024-09-16T09:41:11.976785+0000","objects_scrubbed":0,"log_size":3941,"log_dups_size":0,"ondisk_log_size":3941,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:38.628474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":258277376,"num_objects":3941,"num_object_clones":0,"num_object_copies":11823,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3941,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3941,"num_write_kb":252224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.2","version":"73'79","reported_seq":187,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.717295+0000","last_change":"2024-09-16T09:40:26.918606+0000","last_active":"2024-09-16T09:41:49.717295+0000","last_peered":"2024-09-16T09:41:49.717295+0000","last_clean":"2024-09-16T09:41:49.717295+0000","last_became_active":"2024-09-16T09:38:10.556320+0000","last_became_peered":"2024-09-16T09:38:10.556320+0000","last_unstale":"2024-09-16T09:41:49.717295+0000","last_undegraded":"2024-09-16T09:41:49.717295+0000","last_fullsized":"2024-09-16T09:41:49.717295+0000","mapping_epoch":386,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":387,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:40:26.918532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:40:26.918532+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"285'17","reported_seq":588,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.715938+0000","last_change":"2024-09-16T09:41:27.770120+0000","last_active":"2024-09-16T09:41:49.715938+0000","last_peered":"2024-09-16T09:41:49.715938+0000","last_clean":"2024-09-16T09:41:49.715938+0000","last_became_active":"2024-09-16T09:37:33.349009+0000","last_became_peered":"2024-09-16T09:37:33.349009+0000","last_unstale":"2024-09-16T09:41:49.715938+0000","last_undegraded":"2024-09-16T09:41:49.715938+0000","last_fullsized":"2024-09-16T09:41:49.715938+0000","mapping_epoch":367,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":368,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'17","last_scrub_stamp":"2024-09-16T09:41:27.770036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:41:27.770036+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:57.206439+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":35,"num_read_kb":24,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":498,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.708503+0000","last_change":"2024-09-16T09:41:46.003680+0000","last_active":"2024-09-16T09:41:49.708503+0000","last_peered":"2024-09-16T09:41:49.708503+0000","last_clean":"2024-09-16T09:41:49.708503+0000","last_became_active":"2024-09-16T09:33:30.247596+0000","last_became_peered":"2024-09-16T09:33:30.247596+0000","last_unstale":"2024-09-16T09:41:49.708503+0000","last_undegraded":"2024-09-16T09:41:49.708503+0000","last_fullsized":"2024-09-16T09:41:49.708503+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:46.003500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:41:46.003500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:01.852542+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,2],"acting":[6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":531,"reported_epoch":466,"state":"active+clean","last_fresh":"2024-09-16T09:41:44.426826+0000","last_change":"2024-09-16T09:41:42.039205+0000","last_active":"2024-09-16T09:41:44.426826+0000","last_peered":"2024-09-16T09:41:44.426826+0000","last_clean":"2024-09-16T09:41:44.426826+0000","last_became_active":"2024-09-16T09:37:04.358985+0000","last_became_peered":"2024-09-16T09:37:04.358985+0000","last_unstale":"2024-09-16T09:41:44.426826+0000","last_undegraded":"2024-09-16T09:41:44.426826+0000","last_fullsized":"2024-09-16T09:41:44.426826+0000","mapping_epoch":343,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":344,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:41:42.038861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:41:42.038861+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:43.040519+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":891,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.710076+0000","last_change":"2024-09-16T09:41:45.549361+0000","last_active":"2024-09-16T09:41:49.710076+0000","last_peered":"2024-09-16T09:41:49.710076+0000","last_clean":"2024-09-16T09:41:49.710076+0000","last_became_active":"2024-09-16T09:25:05.711539+0000","last_became_peered":"2024-09-16T09:25:05.711539+0000","last_unstale":"2024-09-16T09:41:49.710076+0000","last_undegraded":"2024-09-16T09:41:49.710076+0000","last_fullsized":"2024-09-16T09:41:49.710076+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:45.549186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:41:45.549186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:58.155707+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":859,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.717608+0000","last_change":"2024-09-16T09:41:31.217733+0000","last_active":"2024-09-16T09:41:49.717608+0000","last_peered":"2024-09-16T09:41:49.717608+0000","last_clean":"2024-09-16T09:41:49.717608+0000","last_became_active":"2024-09-16T09:25:07.723282+0000","last_became_peered":"2024-09-16T09:25:07.723282+0000","last_unstale":"2024-09-16T09:41:49.717608+0000","last_undegraded":"2024-09-16T09:41:49.717608+0000","last_fullsized":"2024-09-16T09:41:49.717608+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:31.217664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:41:31.217664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:45.454634+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.9","version":"467'4156","reported_seq":4193,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:53.675665+0000","last_change":"2024-09-16T09:41:13.001775+0000","last_active":"2024-09-16T09:41:53.675665+0000","last_peered":"2024-09-16T09:41:53.675665+0000","last_clean":"2024-09-16T09:41:53.675665+0000","last_became_active":"2024-09-16T09:41:13.001488+0000","last_became_peered":"2024-09-16T09:41:13.001488+0000","last_unstale":"2024-09-16T09:41:53.675665+0000","last_undegraded":"2024-09-16T09:41:53.675665+0000","last_fullsized":"2024-09-16T09:41:53.675665+0000","mapping_epoch":454,"log_start":"0'0","ondisk_log_start":"0'0","created":454,"last_epoch_clean":455,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:11.976785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:41:11.976785+0000","last_clean_scrub_stamp":"2024-09-16T09:41:11.976785+0000","objects_scrubbed":0,"log_size":4156,"log_dups_size":0,"ondisk_log_size":4156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:15.559115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":272367616,"num_objects":4156,"num_object_clones":0,"num_object_copies":12468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4156,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4156,"num_write_kb":265984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"73'79","reported_seq":571,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.709447+0000","last_change":"2024-09-16T09:34:31.929462+0000","last_active":"2024-09-16T09:41:49.709447+0000","last_peered":"2024-09-16T09:41:49.709447+0000","last_clean":"2024-09-16T09:41:49.709447+0000","last_became_active":"2024-09-16T09:33:30.249309+0000","last_became_peered":"2024-09-16T09:33:30.249309+0000","last_unstale":"2024-09-16T09:41:49.709447+0000","last_undegraded":"2024-09-16T09:41:49.709447+0000","last_fullsized":"2024-09-16T09:41:49.709447+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:34:31.929388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:34:31.929388+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:18.945206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.6","version":"285'24","reported_seq":896,"reported_epoch":466,"state":"active+clean","last_fresh":"2024-09-16T09:41:44.428098+0000","last_change":"2024-09-16T09:40:29.074141+0000","last_active":"2024-09-16T09:41:44.428098+0000","last_peered":"2024-09-16T09:41:44.428098+0000","last_clean":"2024-09-16T09:41:44.428098+0000","last_became_active":"2024-09-16T09:25:03.715315+0000","last_became_peered":"2024-09-16T09:25:03.715315+0000","last_unstale":"2024-09-16T09:41:44.428098+0000","last_undegraded":"2024-09-16T09:41:44.428098+0000","last_fullsized":"2024-09-16T09:41:44.428098+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'24","last_scrub_stamp":"2024-09-16T09:40:29.074075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:40:29.074075+0000","objects_scrubbed":6,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":36,"num_read_kb":30,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"59'1","reported_seq":896,"reported_epoch":466,"state":"active+clean","last_fresh":"2024-09-16T09:41:44.423148+0000","last_change":"2024-09-16T09:41:37.223719+0000","last_active":"2024-09-16T09:41:44.423148+0000","last_peered":"2024-09-16T09:41:44.423148+0000","last_clean":"2024-09-16T09:41:44.423148+0000","last_became_active":"2024-09-16T09:31:09.269627+0000","last_became_peered":"2024-09-16T09:31:09.269627+0000","last_unstale":"2024-09-16T09:41:44.423148+0000","last_undegraded":"2024-09-16T09:41:44.423148+0000","last_fullsized":"2024-09-16T09:41:44.423148+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:41:37.223543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:41:37.223543+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:50.479230+0000","scrub_duration":61,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1986,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"67'5","reported_seq":1887,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:51.390732+0000","last_change":"2024-09-16T09:41:43.153303+0000","last_active":"2024-09-16T09:41:51.390732+0000","last_peered":"2024-09-16T09:41:51.390732+0000","last_clean":"2024-09-16T09:41:51.390732+0000","last_became_active":"2024-09-16T09:25:00.696546+0000","last_became_peered":"2024-09-16T09:25:00.696546+0000","last_unstale":"2024-09-16T09:41:51.390732+0000","last_undegraded":"2024-09-16T09:41:51.390732+0000","last_fullsized":"2024-09-16T09:41:51.390732+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'5","last_scrub_stamp":"2024-09-16T09:41:43.153174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:41:43.153174+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:45.073584+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0.0059770420000000001,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1004,"num_read_kb":999,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":833,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.716020+0000","last_change":"2024-09-16T09:27:07.337170+0000","last_active":"2024-09-16T09:41:49.716020+0000","last_peered":"2024-09-16T09:41:49.716020+0000","last_clean":"2024-09-16T09:41:49.716020+0000","last_became_active":"2024-09-16T09:25:05.714704+0000","last_became_peered":"2024-09-16T09:25:05.714704+0000","last_unstale":"2024-09-16T09:41:49.716020+0000","last_undegraded":"2024-09-16T09:41:49.716020+0000","last_fullsized":"2024-09-16T09:41:49.716020+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:07.337124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:07.337124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:18.736447+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":768,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.714599+0000","last_change":"2024-09-16T09:40:33.229294+0000","last_active":"2024-09-16T09:41:49.714599+0000","last_peered":"2024-09-16T09:41:49.714599+0000","last_clean":"2024-09-16T09:41:49.714599+0000","last_became_active":"2024-09-16T09:33:30.245852+0000","last_became_peered":"2024-09-16T09:33:30.245852+0000","last_unstale":"2024-09-16T09:41:49.714599+0000","last_undegraded":"2024-09-16T09:41:49.714599+0000","last_fullsized":"2024-09-16T09:41:49.714599+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:40:33.229250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:40:33.229250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:02.848996+0000","scrub_duration":97,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.a","version":"467'3943","reported_seq":3979,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:52.930165+0000","last_change":"2024-09-16T09:41:13.002241+0000","last_active":"2024-09-16T09:41:52.930165+0000","last_peered":"2024-09-16T09:41:52.930165+0000","last_clean":"2024-09-16T09:41:52.930165+0000","last_became_active":"2024-09-16T09:41:13.001938+0000","last_became_peered":"2024-09-16T09:41:13.001938+0000","last_unstale":"2024-09-16T09:41:52.930165+0000","last_undegraded":"2024-09-16T09:41:52.930165+0000","last_fullsized":"2024-09-16T09:41:52.930165+0000","mapping_epoch":454,"log_start":"0'0","ondisk_log_start":"0'0","created":454,"last_epoch_clean":455,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:11.976785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:41:11.976785+0000","last_clean_scrub_stamp":"2024-09-16T09:41:11.976785+0000","objects_scrubbed":0,"log_size":3943,"log_dups_size":0,"ondisk_log_size":3943,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:19.474805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":258408448,"num_objects":3943,"num_object_clones":0,"num_object_copies":11829,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3943,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3943,"num_write_kb":252352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"285'28","reported_seq":1106,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.711656+0000","last_change":"2024-09-16T09:33:16.891850+0000","last_active":"2024-09-16T09:41:49.711656+0000","last_peered":"2024-09-16T09:41:49.711656+0000","last_clean":"2024-09-16T09:41:49.711656+0000","last_became_active":"2024-09-16T09:25:03.714118+0000","last_became_peered":"2024-09-16T09:25:03.714118+0000","last_unstale":"2024-09-16T09:41:49.711656+0000","last_undegraded":"2024-09-16T09:41:49.711656+0000","last_fullsized":"2024-09-16T09:41:49.711656+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'16","last_scrub_stamp":"2024-09-16T09:33:16.891761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:33:16.891761+0000","objects_scrubbed":8,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:13.927146+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":43,"num_read_kb":33,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":716,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.716115+0000","last_change":"2024-09-16T09:41:09.792929+0000","last_active":"2024-09-16T09:41:49.716115+0000","last_peered":"2024-09-16T09:41:49.716115+0000","last_clean":"2024-09-16T09:41:49.716115+0000","last_became_active":"2024-09-16T09:31:09.270196+0000","last_became_peered":"2024-09-16T09:31:09.270196+0000","last_unstale":"2024-09-16T09:41:49.716115+0000","last_undegraded":"2024-09-16T09:41:49.716115+0000","last_fullsized":"2024-09-16T09:41:49.716115+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:09.792874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:41:09.792874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:38.425208+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"73'79","reported_seq":944,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.716736+0000","last_change":"2024-09-16T09:39:39.878540+0000","last_active":"2024-09-16T09:41:49.716736+0000","last_peered":"2024-09-16T09:41:49.716736+0000","last_clean":"2024-09-16T09:41:49.716736+0000","last_became_active":"2024-09-16T09:30:08.870394+0000","last_became_peered":"2024-09-16T09:30:08.870394+0000","last_unstale":"2024-09-16T09:41:49.716736+0000","last_undegraded":"2024-09-16T09:41:49.716736+0000","last_fullsized":"2024-09-16T09:41:49.716736+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:39:39.878410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:39:39.878410+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":856,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.718153+0000","last_change":"2024-09-16T09:40:22.941843+0000","last_active":"2024-09-16T09:41:49.718153+0000","last_peered":"2024-09-16T09:41:49.718153+0000","last_clean":"2024-09-16T09:41:49.718153+0000","last_became_active":"2024-09-16T09:25:05.714341+0000","last_became_peered":"2024-09-16T09:25:05.714341+0000","last_unstale":"2024-09-16T09:41:49.718153+0000","last_undegraded":"2024-09-16T09:41:49.718153+0000","last_fullsized":"2024-09-16T09:41:49.718153+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:40:22.941427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:40:22.941427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":853,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.715798+0000","last_change":"2024-09-16T09:41:38.660523+0000","last_active":"2024-09-16T09:41:49.715798+0000","last_peered":"2024-09-16T09:41:49.715798+0000","last_clean":"2024-09-16T09:41:49.715798+0000","last_became_active":"2024-09-16T09:25:07.714281+0000","last_became_peered":"2024-09-16T09:25:07.714281+0000","last_unstale":"2024-09-16T09:41:49.715798+0000","last_undegraded":"2024-09-16T09:41:49.715798+0000","last_fullsized":"2024-09-16T09:41:49.715798+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:38.660336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:41:38.660336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:44.303685+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1","version":"467'3940","reported_seq":3973,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:52.964698+0000","last_change":"2024-09-16T09:41:13.003885+0000","last_active":"2024-09-16T09:41:52.964698+0000","last_peered":"2024-09-16T09:41:52.964698+0000","last_clean":"2024-09-16T09:41:52.964698+0000","last_became_active":"2024-09-16T09:41:13.003635+0000","last_became_peered":"2024-09-16T09:41:13.003635+0000","last_unstale":"2024-09-16T09:41:52.964698+0000","last_undegraded":"2024-09-16T09:41:52.964698+0000","last_fullsized":"2024-09-16T09:41:52.964698+0000","mapping_epoch":454,"log_start":"0'0","ondisk_log_start":"0'0","created":454,"last_epoch_clean":455,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:11.976785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:41:11.976785+0000","last_clean_scrub_stamp":"2024-09-16T09:41:11.976785+0000","objects_scrubbed":0,"log_size":3940,"log_dups_size":0,"ondisk_log_size":3940,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:11.978746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":258211840,"num_objects":3940,"num_object_clones":0,"num_object_copies":11820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3940,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3940,"num_write_kb":252160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.b","version":"73'79","reported_seq":687,"reported_epoch":466,"state":"active+clean","last_fresh":"2024-09-16T09:41:44.422839+0000","last_change":"2024-09-16T09:40:37.917868+0000","last_active":"2024-09-16T09:41:44.422839+0000","last_peered":"2024-09-16T09:41:44.422839+0000","last_clean":"2024-09-16T09:41:44.422839+0000","last_became_active":"2024-09-16T09:33:30.251933+0000","last_became_peered":"2024-09-16T09:33:30.251933+0000","last_unstale":"2024-09-16T09:41:44.422839+0000","last_undegraded":"2024-09-16T09:41:44.422839+0000","last_fullsized":"2024-09-16T09:41:44.422839+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:40:37.917691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:40:37.917691+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"285'19","reported_seq":942,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.711870+0000","last_change":"2024-09-16T09:41:03.243490+0000","last_active":"2024-09-16T09:41:49.711870+0000","last_peered":"2024-09-16T09:41:49.711870+0000","last_clean":"2024-09-16T09:41:49.711870+0000","last_became_active":"2024-09-16T09:37:33.348433+0000","last_became_peered":"2024-09-16T09:37:33.348433+0000","last_unstale":"2024-09-16T09:41:49.711870+0000","last_undegraded":"2024-09-16T09:41:49.711870+0000","last_fullsized":"2024-09-16T09:41:49.711870+0000","mapping_epoch":367,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":368,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'19","last_scrub_stamp":"2024-09-16T09:41:03.243406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:41:03.243406+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:30.272372+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":850,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.710549+0000","last_change":"2024-09-16T09:37:18.392256+0000","last_active":"2024-09-16T09:41:49.710549+0000","last_peered":"2024-09-16T09:41:49.710549+0000","last_clean":"2024-09-16T09:41:49.710549+0000","last_became_active":"2024-09-16T09:25:01.708270+0000","last_became_peered":"2024-09-16T09:25:01.708270+0000","last_unstale":"2024-09-16T09:41:49.710549+0000","last_undegraded":"2024-09-16T09:41:49.710549+0000","last_fullsized":"2024-09-16T09:41:49.710549+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:18.392084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:37:18.392084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:15.281480+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":833,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.715574+0000","last_change":"2024-09-16T09:26:34.171190+0000","last_active":"2024-09-16T09:41:49.715574+0000","last_peered":"2024-09-16T09:41:49.715574+0000","last_clean":"2024-09-16T09:41:49.715574+0000","last_became_active":"2024-09-16T09:25:05.711964+0000","last_became_peered":"2024-09-16T09:25:05.711964+0000","last_unstale":"2024-09-16T09:41:49.715574+0000","last_undegraded":"2024-09-16T09:41:49.715574+0000","last_fullsized":"2024-09-16T09:41:49.715574+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:34.171139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:26:34.171139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:16.797486+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":732,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.710316+0000","last_change":"2024-09-16T09:40:31.948601+0000","last_active":"2024-09-16T09:41:49.710316+0000","last_peered":"2024-09-16T09:41:49.710316+0000","last_clean":"2024-09-16T09:41:49.710316+0000","last_became_active":"2024-09-16T09:31:09.278618+0000","last_became_peered":"2024-09-16T09:31:09.278618+0000","last_unstale":"2024-09-16T09:41:49.710316+0000","last_undegraded":"2024-09-16T09:41:49.710316+0000","last_fullsized":"2024-09-16T09:41:49.710316+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:40:31.948536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:40:31.948536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:01.350977+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"467'4068","reported_seq":4105,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:53.675999+0000","last_change":"2024-09-16T09:41:12.998092+0000","last_active":"2024-09-16T09:41:53.675999+0000","last_peered":"2024-09-16T09:41:53.675999+0000","last_clean":"2024-09-16T09:41:53.675999+0000","last_became_active":"2024-09-16T09:41:12.997802+0000","last_became_peered":"2024-09-16T09:41:12.997802+0000","last_unstale":"2024-09-16T09:41:53.675999+0000","last_undegraded":"2024-09-16T09:41:53.675999+0000","last_fullsized":"2024-09-16T09:41:53.675999+0000","mapping_epoch":454,"log_start":"0'0","ondisk_log_start":"0'0","created":454,"last_epoch_clean":455,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:11.976785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:41:11.976785+0000","last_clean_scrub_stamp":"2024-09-16T09:41:11.976785+0000","objects_scrubbed":0,"log_size":4068,"log_dups_size":0,"ondisk_log_size":4068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:20.435005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":266600448,"num_objects":4068,"num_object_clones":0,"num_object_copies":12204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4068,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4068,"num_write_kb":260352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.8","version":"73'79","reported_seq":200,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.713303+0000","last_change":"2024-09-16T09:41:31.767349+0000","last_active":"2024-09-16T09:41:49.713303+0000","last_peered":"2024-09-16T09:41:49.713303+0000","last_clean":"2024-09-16T09:41:49.713303+0000","last_became_active":"2024-09-16T09:37:53.418691+0000","last_became_peered":"2024-09-16T09:37:53.418691+0000","last_unstale":"2024-09-16T09:41:49.713303+0000","last_undegraded":"2024-09-16T09:41:49.713303+0000","last_fullsized":"2024-09-16T09:41:49.713303+0000","mapping_epoch":382,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":383,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:41:31.767299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:41:31.767299+0000","objects_scrubbed":1,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:51.743338+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262688,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"285'31","reported_seq":911,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.712128+0000","last_change":"2024-09-16T09:35:28.861520+0000","last_active":"2024-09-16T09:41:49.712128+0000","last_peered":"2024-09-16T09:41:49.712128+0000","last_clean":"2024-09-16T09:41:49.712128+0000","last_became_active":"2024-09-16T09:25:03.713358+0000","last_became_peered":"2024-09-16T09:25:03.713358+0000","last_unstale":"2024-09-16T09:41:49.712128+0000","last_undegraded":"2024-09-16T09:41:49.712128+0000","last_fullsized":"2024-09-16T09:41:49.712128+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'31","last_scrub_stamp":"2024-09-16T09:35:28.861433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:35:28.861433+0000","objects_scrubbed":8,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:59.274362+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":50,"num_read_kb":40,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":866,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.710105+0000","last_change":"2024-09-16T09:41:07.972297+0000","last_active":"2024-09-16T09:41:49.710105+0000","last_peered":"2024-09-16T09:41:49.710105+0000","last_clean":"2024-09-16T09:41:49.710105+0000","last_became_active":"2024-09-16T09:25:01.703355+0000","last_became_peered":"2024-09-16T09:25:01.703355+0000","last_unstale":"2024-09-16T09:41:49.710105+0000","last_undegraded":"2024-09-16T09:41:49.710105+0000","last_fullsized":"2024-09-16T09:41:49.710105+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:07.972246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:41:07.972246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:31.789728+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":497,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.710021+0000","last_change":"2024-09-16T09:37:46.378016+0000","last_active":"2024-09-16T09:41:49.710021+0000","last_peered":"2024-09-16T09:41:49.710021+0000","last_clean":"2024-09-16T09:41:49.710021+0000","last_became_active":"2024-09-16T09:33:30.251186+0000","last_became_peered":"2024-09-16T09:33:30.251186+0000","last_unstale":"2024-09-16T09:41:49.710021+0000","last_undegraded":"2024-09-16T09:41:49.710021+0000","last_fullsized":"2024-09-16T09:41:49.710021+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:46.377951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:37:46.377951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:22.269606+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,0],"acting":[4,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":795,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.707617+0000","last_change":"2024-09-16T09:38:24.031758+0000","last_active":"2024-09-16T09:41:49.707617+0000","last_peered":"2024-09-16T09:41:49.707617+0000","last_clean":"2024-09-16T09:41:49.707617+0000","last_became_active":"2024-09-16T09:33:21.337291+0000","last_became_peered":"2024-09-16T09:33:21.337291+0000","last_unstale":"2024-09-16T09:41:49.707617+0000","last_undegraded":"2024-09-16T09:41:49.707617+0000","last_fullsized":"2024-09-16T09:41:49.707617+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:38:24.031592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:38:24.031592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:56.866653+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4],"acting":[3,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3","version":"467'3991","reported_seq":4027,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:52.939153+0000","last_change":"2024-09-16T09:41:13.139231+0000","last_active":"2024-09-16T09:41:52.939153+0000","last_peered":"2024-09-16T09:41:52.939153+0000","last_clean":"2024-09-16T09:41:52.939153+0000","last_became_active":"2024-09-16T09:41:13.138925+0000","last_became_peered":"2024-09-16T09:41:13.138925+0000","last_unstale":"2024-09-16T09:41:52.939153+0000","last_undegraded":"2024-09-16T09:41:52.939153+0000","last_fullsized":"2024-09-16T09:41:52.939153+0000","mapping_epoch":454,"log_start":"0'0","ondisk_log_start":"0'0","created":454,"last_epoch_clean":455,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:11.976785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:41:11.976785+0000","last_clean_scrub_stamp":"2024-09-16T09:41:11.976785+0000","objects_scrubbed":0,"log_size":3991,"log_dups_size":0,"ondisk_log_size":3991,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:33.187528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":261554176,"num_objects":3991,"num_object_clones":0,"num_object_copies":11973,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3991,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3991,"num_write_kb":255424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1],"acting":[6,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.9","version":"73'79","reported_seq":928,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.717289+0000","last_change":"2024-09-16T09:41:40.124011+0000","last_active":"2024-09-16T09:41:49.717289+0000","last_peered":"2024-09-16T09:41:49.717289+0000","last_clean":"2024-09-16T09:41:49.717289+0000","last_became_active":"2024-09-16T09:26:35.126066+0000","last_became_peered":"2024-09-16T09:26:35.126066+0000","last_unstale":"2024-09-16T09:41:49.717289+0000","last_undegraded":"2024-09-16T09:41:49.717289+0000","last_fullsized":"2024-09-16T09:41:49.717289+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:41:40.123937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:41:40.123937+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:04.683364+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"285'20","reported_seq":884,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.711232+0000","last_change":"2024-09-16T09:41:42.111004+0000","last_active":"2024-09-16T09:41:49.711232+0000","last_peered":"2024-09-16T09:41:49.711232+0000","last_clean":"2024-09-16T09:41:49.711232+0000","last_became_active":"2024-09-16T09:25:03.707723+0000","last_became_peered":"2024-09-16T09:25:03.707723+0000","last_unstale":"2024-09-16T09:41:49.711232+0000","last_undegraded":"2024-09-16T09:41:49.711232+0000","last_fullsized":"2024-09-16T09:41:49.711232+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'20","last_scrub_stamp":"2024-09-16T09:41:42.110923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:41:42.110923+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:46.525293+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":721,"reported_epoch":466,"state":"active+clean","last_fresh":"2024-09-16T09:41:44.423523+0000","last_change":"2024-09-16T09:39:38.508106+0000","last_active":"2024-09-16T09:41:44.423523+0000","last_peered":"2024-09-16T09:41:44.423523+0000","last_clean":"2024-09-16T09:41:44.423523+0000","last_became_active":"2024-09-16T09:31:09.288825+0000","last_became_peered":"2024-09-16T09:31:09.288825+0000","last_unstale":"2024-09-16T09:41:44.423523+0000","last_undegraded":"2024-09-16T09:41:44.423523+0000","last_fullsized":"2024-09-16T09:41:44.423523+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:39:38.507898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:39:38.507898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:49.888651+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"228'9","reported_seq":647,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.717739+0000","last_change":"2024-09-16T09:41:33.174507+0000","last_active":"2024-09-16T09:41:49.717739+0000","last_peered":"2024-09-16T09:41:49.717739+0000","last_clean":"2024-09-16T09:41:49.717739+0000","last_became_active":"2024-09-16T09:33:21.337213+0000","last_became_peered":"2024-09-16T09:33:21.337213+0000","last_unstale":"2024-09-16T09:41:49.717739+0000","last_undegraded":"2024-09-16T09:41:49.717739+0000","last_fullsized":"2024-09-16T09:41:49.717739+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"228'9","last_scrub_stamp":"2024-09-16T09:41:33.174444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:41:33.174444+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:44.705989+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":878,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.711602+0000","last_change":"2024-09-16T09:41:39.100268+0000","last_active":"2024-09-16T09:41:49.711602+0000","last_peered":"2024-09-16T09:41:49.711602+0000","last_clean":"2024-09-16T09:41:49.711602+0000","last_became_active":"2024-09-16T09:25:07.718958+0000","last_became_peered":"2024-09-16T09:25:07.718958+0000","last_unstale":"2024-09-16T09:41:49.711602+0000","last_undegraded":"2024-09-16T09:41:49.711602+0000","last_fullsized":"2024-09-16T09:41:49.711602+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:39.100089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:41:39.100089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:42.983457+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.4","version":"467'4026","reported_seq":4062,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:52.498762+0000","last_change":"2024-09-16T09:41:13.004443+0000","last_active":"2024-09-16T09:41:52.498762+0000","last_peered":"2024-09-16T09:41:52.498762+0000","last_clean":"2024-09-16T09:41:52.498762+0000","last_became_active":"2024-09-16T09:41:13.004032+0000","last_became_peered":"2024-09-16T09:41:13.004032+0000","last_unstale":"2024-09-16T09:41:52.498762+0000","last_undegraded":"2024-09-16T09:41:52.498762+0000","last_fullsized":"2024-09-16T09:41:52.498762+0000","mapping_epoch":454,"log_start":"0'0","ondisk_log_start":"0'0","created":454,"last_epoch_clean":455,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:11.976785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:41:11.976785+0000","last_clean_scrub_stamp":"2024-09-16T09:41:11.976785+0000","objects_scrubbed":0,"log_size":4026,"log_dups_size":0,"ondisk_log_size":4026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:17.025591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":263847936,"num_objects":4026,"num_object_clones":0,"num_object_copies":12078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4026,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4026,"num_write_kb":257664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.e","version":"73'79","reported_seq":655,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.711510+0000","last_change":"2024-09-16T09:35:41.979877+0000","last_active":"2024-09-16T09:41:49.711510+0000","last_peered":"2024-09-16T09:41:49.711510+0000","last_clean":"2024-09-16T09:41:49.711510+0000","last_became_active":"2024-09-16T09:30:10.705072+0000","last_became_peered":"2024-09-16T09:30:10.705072+0000","last_unstale":"2024-09-16T09:41:49.711510+0000","last_undegraded":"2024-09-16T09:41:49.711510+0000","last_fullsized":"2024-09-16T09:41:49.711510+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:35:41.979814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:35:41.979814+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:12.209303+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,3],"acting":[4,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.b","version":"285'15","reported_seq":910,"reported_epoch":466,"state":"active+clean","last_fresh":"2024-09-16T09:41:44.423216+0000","last_change":"2024-09-16T09:41:09.042482+0000","last_active":"2024-09-16T09:41:44.423216+0000","last_peered":"2024-09-16T09:41:44.423216+0000","last_clean":"2024-09-16T09:41:44.423216+0000","last_became_active":"2024-09-16T09:36:31.590202+0000","last_became_peered":"2024-09-16T09:36:31.590202+0000","last_unstale":"2024-09-16T09:41:44.423216+0000","last_undegraded":"2024-09-16T09:41:44.423216+0000","last_fullsized":"2024-09-16T09:41:44.423216+0000","mapping_epoch":316,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":317,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'15","last_scrub_stamp":"2024-09-16T09:41:09.039458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:41:09.039458+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:17.178456+0000","scrub_duration":41,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":863,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.714636+0000","last_change":"2024-09-16T09:37:31.033631+0000","last_active":"2024-09-16T09:41:49.714636+0000","last_peered":"2024-09-16T09:41:49.714636+0000","last_clean":"2024-09-16T09:41:49.714636+0000","last_became_active":"2024-09-16T09:31:09.276516+0000","last_became_peered":"2024-09-16T09:31:09.276516+0000","last_unstale":"2024-09-16T09:41:49.714636+0000","last_undegraded":"2024-09-16T09:41:49.714636+0000","last_fullsized":"2024-09-16T09:41:49.714636+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:31.033577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:37:31.033577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:20.735658+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":783,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.710445+0000","last_change":"2024-09-16T09:33:40.260547+0000","last_active":"2024-09-16T09:41:49.710445+0000","last_peered":"2024-09-16T09:41:49.710445+0000","last_clean":"2024-09-16T09:41:49.710445+0000","last_became_active":"2024-09-16T09:33:21.337254+0000","last_became_peered":"2024-09-16T09:33:21.337254+0000","last_unstale":"2024-09-16T09:41:49.710445+0000","last_undegraded":"2024-09-16T09:41:49.710445+0000","last_fullsized":"2024-09-16T09:41:49.710445+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:40.260500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:40.260500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:51.225205+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3],"acting":[4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":852,"reported_epoch":466,"state":"active+clean","last_fresh":"2024-09-16T09:41:44.423006+0000","last_change":"2024-09-16T09:36:53.647305+0000","last_active":"2024-09-16T09:41:44.423006+0000","last_peered":"2024-09-16T09:41:44.423006+0000","last_clean":"2024-09-16T09:41:44.423006+0000","last_became_active":"2024-09-16T09:25:07.718778+0000","last_became_peered":"2024-09-16T09:25:07.718778+0000","last_unstale":"2024-09-16T09:41:44.423006+0000","last_undegraded":"2024-09-16T09:41:44.423006+0000","last_fullsized":"2024-09-16T09:41:44.423006+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:53.647256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:36:53.647256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:17.963060+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.5","version":"467'4046","reported_seq":4082,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:52.472162+0000","last_change":"2024-09-16T09:41:13.004151+0000","last_active":"2024-09-16T09:41:52.472162+0000","last_peered":"2024-09-16T09:41:52.472162+0000","last_clean":"2024-09-16T09:41:52.472162+0000","last_became_active":"2024-09-16T09:41:13.003693+0000","last_became_peered":"2024-09-16T09:41:13.003693+0000","last_unstale":"2024-09-16T09:41:52.472162+0000","last_undegraded":"2024-09-16T09:41:52.472162+0000","last_fullsized":"2024-09-16T09:41:52.472162+0000","mapping_epoch":454,"log_start":"0'0","ondisk_log_start":"0'0","created":454,"last_epoch_clean":455,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:11.976785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:41:11.976785+0000","last_clean_scrub_stamp":"2024-09-16T09:41:11.976785+0000","objects_scrubbed":0,"log_size":4046,"log_dups_size":0,"ondisk_log_size":4046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:35.079186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":265158656,"num_objects":4046,"num_object_clones":0,"num_object_copies":12138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4046,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4046,"num_write_kb":258944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.f","version":"73'79","reported_seq":923,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.711684+0000","last_change":"2024-09-16T09:41:28.953879+0000","last_active":"2024-09-16T09:41:49.711684+0000","last_peered":"2024-09-16T09:41:49.711684+0000","last_clean":"2024-09-16T09:41:49.711684+0000","last_became_active":"2024-09-16T09:30:10.693477+0000","last_became_peered":"2024-09-16T09:30:10.693477+0000","last_unstale":"2024-09-16T09:41:49.711684+0000","last_undegraded":"2024-09-16T09:41:49.711684+0000","last_fullsized":"2024-09-16T09:41:49.711684+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:41:28.953810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:41:28.953810+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:52.357805+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"285'35","reported_seq":917,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.709864+0000","last_change":"2024-09-16T09:37:28.078195+0000","last_active":"2024-09-16T09:41:49.709864+0000","last_peered":"2024-09-16T09:41:49.709864+0000","last_clean":"2024-09-16T09:41:49.709864+0000","last_became_active":"2024-09-16T09:25:03.712783+0000","last_became_peered":"2024-09-16T09:25:03.712783+0000","last_unstale":"2024-09-16T09:41:49.709864+0000","last_undegraded":"2024-09-16T09:41:49.709864+0000","last_fullsized":"2024-09-16T09:41:49.709864+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'35","last_scrub_stamp":"2024-09-16T09:37:28.077841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:37:28.077841+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:51.991401+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":56,"num_read_kb":45,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"59'1","reported_seq":869,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.718120+0000","last_change":"2024-09-16T09:37:56.528872+0000","last_active":"2024-09-16T09:41:49.718120+0000","last_peered":"2024-09-16T09:41:49.718120+0000","last_clean":"2024-09-16T09:41:49.718120+0000","last_became_active":"2024-09-16T09:25:01.708069+0000","last_became_peered":"2024-09-16T09:25:01.708069+0000","last_unstale":"2024-09-16T09:41:49.718120+0000","last_undegraded":"2024-09-16T09:41:49.718120+0000","last_fullsized":"2024-09-16T09:41:49.718120+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:37:56.528796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:37:56.528796+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:19.088387+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":889,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.707901+0000","last_change":"2024-09-16T09:40:39.696450+0000","last_active":"2024-09-16T09:41:49.707901+0000","last_peered":"2024-09-16T09:41:49.707901+0000","last_clean":"2024-09-16T09:41:49.707901+0000","last_became_active":"2024-09-16T09:37:53.518284+0000","last_became_peered":"2024-09-16T09:37:53.518284+0000","last_unstale":"2024-09-16T09:41:49.707901+0000","last_undegraded":"2024-09-16T09:41:49.707901+0000","last_fullsized":"2024-09-16T09:41:49.707901+0000","mapping_epoch":382,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":383,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:40:39.696383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:40:39.696383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:00.771862+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":835,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.716649+0000","last_change":"2024-09-16T09:36:48.649739+0000","last_active":"2024-09-16T09:41:49.716649+0000","last_peered":"2024-09-16T09:41:49.716649+0000","last_clean":"2024-09-16T09:41:49.716649+0000","last_became_active":"2024-09-16T09:25:07.718750+0000","last_became_peered":"2024-09-16T09:25:07.718750+0000","last_unstale":"2024-09-16T09:41:49.716649+0000","last_undegraded":"2024-09-16T09:41:49.716649+0000","last_fullsized":"2024-09-16T09:41:49.716649+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:48.649696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:36:48.649696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:13.079073+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.6","version":"466'3748","reported_seq":3782,"reported_epoch":466,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.619910+0000","last_change":"2024-09-16T09:41:13.139206+0000","last_active":"2024-09-16T09:41:49.619910+0000","last_peered":"2024-09-16T09:41:49.619910+0000","last_clean":"2024-09-16T09:41:49.619910+0000","last_became_active":"2024-09-16T09:41:13.138910+0000","last_became_peered":"2024-09-16T09:41:13.138910+0000","last_unstale":"2024-09-16T09:41:49.619910+0000","last_undegraded":"2024-09-16T09:41:49.619910+0000","last_fullsized":"2024-09-16T09:41:49.619910+0000","mapping_epoch":454,"log_start":"0'0","ondisk_log_start":"0'0","created":454,"last_epoch_clean":455,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:11.976785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:41:11.976785+0000","last_clean_scrub_stamp":"2024-09-16T09:41:11.976785+0000","objects_scrubbed":0,"log_size":3748,"log_dups_size":0,"ondisk_log_size":3748,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:24.077359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":245628928,"num_objects":3748,"num_object_clones":0,"num_object_copies":11244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3748,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3748,"num_write_kb":239872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.c","version":"73'79","reported_seq":603,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.708909+0000","last_change":"2024-09-16T09:40:35.772064+0000","last_active":"2024-09-16T09:41:49.708909+0000","last_peered":"2024-09-16T09:41:49.708909+0000","last_clean":"2024-09-16T09:41:49.708909+0000","last_became_active":"2024-09-16T09:31:09.273770+0000","last_became_peered":"2024-09-16T09:31:09.273770+0000","last_unstale":"2024-09-16T09:41:49.708909+0000","last_undegraded":"2024-09-16T09:41:49.708909+0000","last_fullsized":"2024-09-16T09:41:49.708909+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:40:35.772017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:40:35.772017+0000","objects_scrubbed":1,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:00.885825+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":327680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"285'18","reported_seq":932,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.710411+0000","last_change":"2024-09-16T09:36:50.603724+0000","last_active":"2024-09-16T09:41:49.710411+0000","last_peered":"2024-09-16T09:41:49.710411+0000","last_clean":"2024-09-16T09:41:49.710411+0000","last_became_active":"2024-09-16T09:31:09.269047+0000","last_became_peered":"2024-09-16T09:31:09.269047+0000","last_unstale":"2024-09-16T09:41:49.710411+0000","last_undegraded":"2024-09-16T09:41:49.710411+0000","last_fullsized":"2024-09-16T09:41:49.710411+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'18","last_scrub_stamp":"2024-09-16T09:36:50.603660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:36:50.603660+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:14.276752+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":863,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.719523+0000","last_change":"2024-09-16T09:37:57.529616+0000","last_active":"2024-09-16T09:41:49.719523+0000","last_peered":"2024-09-16T09:41:49.719523+0000","last_clean":"2024-09-16T09:41:49.719523+0000","last_became_active":"2024-09-16T09:25:01.707528+0000","last_became_peered":"2024-09-16T09:25:01.707528+0000","last_unstale":"2024-09-16T09:41:49.719523+0000","last_undegraded":"2024-09-16T09:41:49.719523+0000","last_fullsized":"2024-09-16T09:41:49.719523+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:57.529554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:37:57.529554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:25.132084+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":515,"reported_epoch":466,"state":"active+clean","last_fresh":"2024-09-16T09:41:44.422788+0000","last_change":"2024-09-16T09:40:25.011759+0000","last_active":"2024-09-16T09:41:44.422788+0000","last_peered":"2024-09-16T09:41:44.422788+0000","last_clean":"2024-09-16T09:41:44.422788+0000","last_became_active":"2024-09-16T09:33:21.337309+0000","last_became_peered":"2024-09-16T09:33:21.337309+0000","last_unstale":"2024-09-16T09:41:44.422788+0000","last_undegraded":"2024-09-16T09:41:44.422788+0000","last_fullsized":"2024-09-16T09:41:44.422788+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:40:25.011687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:40:25.011687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":854,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.712535+0000","last_change":"2024-09-16T09:38:21.097872+0000","last_active":"2024-09-16T09:41:49.712535+0000","last_peered":"2024-09-16T09:41:49.712535+0000","last_clean":"2024-09-16T09:41:49.712535+0000","last_became_active":"2024-09-16T09:25:07.718899+0000","last_became_peered":"2024-09-16T09:25:07.718899+0000","last_unstale":"2024-09-16T09:41:49.712535+0000","last_undegraded":"2024-09-16T09:41:49.712535+0000","last_fullsized":"2024-09-16T09:41:49.712535+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:38:21.097804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:38:21.097804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:58.820270+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.7","version":"467'4136","reported_seq":4172,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:53.282641+0000","last_change":"2024-09-16T09:41:13.139096+0000","last_active":"2024-09-16T09:41:53.282641+0000","last_peered":"2024-09-16T09:41:53.282641+0000","last_clean":"2024-09-16T09:41:53.282641+0000","last_became_active":"2024-09-16T09:41:13.138764+0000","last_became_peered":"2024-09-16T09:41:13.138764+0000","last_unstale":"2024-09-16T09:41:53.282641+0000","last_undegraded":"2024-09-16T09:41:53.282641+0000","last_fullsized":"2024-09-16T09:41:53.282641+0000","mapping_epoch":454,"log_start":"0'0","ondisk_log_start":"0'0","created":454,"last_epoch_clean":455,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:11.976785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:41:11.976785+0000","last_clean_scrub_stamp":"2024-09-16T09:41:11.976785+0000","objects_scrubbed":0,"log_size":4136,"log_dups_size":0,"ondisk_log_size":4136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:40.396787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":271056896,"num_objects":4136,"num_object_clones":0,"num_object_copies":12408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4136,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4136,"num_write_kb":264704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.d","version":"73'79","reported_seq":653,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.717430+0000","last_change":"2024-09-16T09:39:49.055357+0000","last_active":"2024-09-16T09:41:49.717430+0000","last_peered":"2024-09-16T09:41:49.717430+0000","last_clean":"2024-09-16T09:41:49.717430+0000","last_became_active":"2024-09-16T09:30:10.709958+0000","last_became_peered":"2024-09-16T09:30:10.709958+0000","last_unstale":"2024-09-16T09:41:49.717430+0000","last_undegraded":"2024-09-16T09:41:49.717430+0000","last_fullsized":"2024-09-16T09:41:49.717430+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:39:49.055292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:39:49.055292+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:57.208604+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"285'27","reported_seq":896,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.713473+0000","last_change":"2024-09-16T09:41:11.837814+0000","last_active":"2024-09-16T09:41:49.713473+0000","last_peered":"2024-09-16T09:41:49.713473+0000","last_clean":"2024-09-16T09:41:49.713473+0000","last_became_active":"2024-09-16T09:25:03.715733+0000","last_became_peered":"2024-09-16T09:25:03.715733+0000","last_unstale":"2024-09-16T09:41:49.713473+0000","last_undegraded":"2024-09-16T09:41:49.713473+0000","last_fullsized":"2024-09-16T09:41:49.713473+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'27","last_scrub_stamp":"2024-09-16T09:41:11.837756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:41:11.837756+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:27.113297+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":867,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.718084+0000","last_change":"2024-09-16T09:37:17.544132+0000","last_active":"2024-09-16T09:41:49.718084+0000","last_peered":"2024-09-16T09:41:49.718084+0000","last_clean":"2024-09-16T09:41:49.718084+0000","last_became_active":"2024-09-16T09:25:01.707372+0000","last_became_peered":"2024-09-16T09:25:01.707372+0000","last_unstale":"2024-09-16T09:41:49.718084+0000","last_undegraded":"2024-09-16T09:41:49.718084+0000","last_fullsized":"2024-09-16T09:41:49.718084+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-16T09:37:17.544069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:37:17.544069+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:14.037434+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"66'8","reported_seq":1074,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.715794+0000","last_change":"2024-09-16T09:41:05.060541+0000","last_active":"2024-09-16T09:41:49.715794+0000","last_peered":"2024-09-16T09:41:49.715794+0000","last_clean":"2024-09-16T09:41:49.715794+0000","last_became_active":"2024-09-16T09:25:05.716189+0000","last_became_peered":"2024-09-16T09:25:05.716189+0000","last_unstale":"2024-09-16T09:41:49.715794+0000","last_undegraded":"2024-09-16T09:41:49.715794+0000","last_fullsized":"2024-09-16T09:41:49.715794+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:41:05.060488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:41:05.060488+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:10.111729+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":837,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.713732+0000","last_change":"2024-09-16T09:37:47.244099+0000","last_active":"2024-09-16T09:41:49.713732+0000","last_peered":"2024-09-16T09:41:49.713732+0000","last_clean":"2024-09-16T09:41:49.713732+0000","last_became_active":"2024-09-16T09:25:07.717213+0000","last_became_peered":"2024-09-16T09:25:07.717213+0000","last_unstale":"2024-09-16T09:41:49.713732+0000","last_undegraded":"2024-09-16T09:41:49.713732+0000","last_fullsized":"2024-09-16T09:41:49.713732+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:47.244032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:37:47.244032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:19.690229+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.12","version":"73'79","reported_seq":663,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.714012+0000","last_change":"2024-09-16T09:40:38.120017+0000","last_active":"2024-09-16T09:41:49.714012+0000","last_peered":"2024-09-16T09:41:49.714012+0000","last_clean":"2024-09-16T09:41:49.714012+0000","last_became_active":"2024-09-16T09:30:10.731413+0000","last_became_peered":"2024-09-16T09:30:10.731413+0000","last_unstale":"2024-09-16T09:41:49.714012+0000","last_undegraded":"2024-09-16T09:41:49.714012+0000","last_fullsized":"2024-09-16T09:41:49.714012+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:40:38.119809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:40:38.119809+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:05.666162+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":872,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.708328+0000","last_change":"2024-09-16T09:37:55.863661+0000","last_active":"2024-09-16T09:41:49.708328+0000","last_peered":"2024-09-16T09:41:49.708328+0000","last_clean":"2024-09-16T09:41:49.708328+0000","last_became_active":"2024-09-16T09:37:53.518354+0000","last_became_peered":"2024-09-16T09:37:53.518354+0000","last_unstale":"2024-09-16T09:41:49.708328+0000","last_undegraded":"2024-09-16T09:41:49.708328+0000","last_fullsized":"2024-09-16T09:41:49.708328+0000","mapping_epoch":382,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":383,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:55.863613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:37:55.863613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:56.995160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"285'12","reported_seq":631,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.707821+0000","last_change":"2024-09-16T09:38:33.535691+0000","last_active":"2024-09-16T09:41:49.707821+0000","last_peered":"2024-09-16T09:41:49.707821+0000","last_clean":"2024-09-16T09:41:49.707821+0000","last_became_active":"2024-09-16T09:38:33.517889+0000","last_became_peered":"2024-09-16T09:38:33.517889+0000","last_unstale":"2024-09-16T09:41:49.707821+0000","last_undegraded":"2024-09-16T09:41:49.707821+0000","last_fullsized":"2024-09-16T09:41:49.707821+0000","mapping_epoch":393,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":394,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'12","last_scrub_stamp":"2024-09-16T09:37:20.749101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:37:20.749101+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:50.891712+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":17,"num_read_kb":14,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":856,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.714811+0000","last_change":"2024-09-16T09:40:28.258952+0000","last_active":"2024-09-16T09:41:49.714811+0000","last_peered":"2024-09-16T09:41:49.714811+0000","last_clean":"2024-09-16T09:41:49.714811+0000","last_became_active":"2024-09-16T09:25:05.715616+0000","last_became_peered":"2024-09-16T09:25:05.715616+0000","last_unstale":"2024-09-16T09:41:49.714811+0000","last_undegraded":"2024-09-16T09:41:49.714811+0000","last_fullsized":"2024-09-16T09:41:49.714811+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:40:28.258907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:40:28.258907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:56.348797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":850,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.717527+0000","last_change":"2024-09-16T09:40:37.040258+0000","last_active":"2024-09-16T09:41:49.717527+0000","last_peered":"2024-09-16T09:41:49.717527+0000","last_clean":"2024-09-16T09:41:49.717527+0000","last_became_active":"2024-09-16T09:25:07.721059+0000","last_became_peered":"2024-09-16T09:25:07.721059+0000","last_unstale":"2024-09-16T09:41:49.717527+0000","last_undegraded":"2024-09-16T09:41:49.717527+0000","last_fullsized":"2024-09-16T09:41:49.717527+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:40:37.040205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:40:37.040205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:05.338645+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.13","version":"73'79","reported_seq":748,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.709984+0000","last_change":"2024-09-16T09:40:30.696059+0000","last_active":"2024-09-16T09:41:49.709984+0000","last_peered":"2024-09-16T09:41:49.709984+0000","last_clean":"2024-09-16T09:41:49.709984+0000","last_became_active":"2024-09-16T09:33:30.248497+0000","last_became_peered":"2024-09-16T09:33:30.248497+0000","last_unstale":"2024-09-16T09:41:49.709984+0000","last_undegraded":"2024-09-16T09:41:49.709984+0000","last_fullsized":"2024-09-16T09:41:49.709984+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:40:30.695866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:40:30.695866+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:53.960674+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"285'15","reported_seq":934,"reported_epoch":466,"state":"active+clean","last_fresh":"2024-09-16T09:41:44.428189+0000","last_change":"2024-09-16T09:39:46.504126+0000","last_active":"2024-09-16T09:41:44.428189+0000","last_peered":"2024-09-16T09:41:44.428189+0000","last_clean":"2024-09-16T09:41:44.428189+0000","last_became_active":"2024-09-16T09:31:09.273498+0000","last_became_peered":"2024-09-16T09:31:09.273498+0000","last_unstale":"2024-09-16T09:41:44.428189+0000","last_undegraded":"2024-09-16T09:41:44.428189+0000","last_fullsized":"2024-09-16T09:41:44.428189+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'15","last_scrub_stamp":"2024-09-16T09:39:46.504045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:39:46.504045+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":871,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.717548+0000","last_change":"2024-09-16T09:41:36.151500+0000","last_active":"2024-09-16T09:41:49.717548+0000","last_peered":"2024-09-16T09:41:49.717548+0000","last_clean":"2024-09-16T09:41:49.717548+0000","last_became_active":"2024-09-16T09:25:01.706491+0000","last_became_peered":"2024-09-16T09:25:01.706491+0000","last_unstale":"2024-09-16T09:41:49.717548+0000","last_undegraded":"2024-09-16T09:41:49.717548+0000","last_fullsized":"2024-09-16T09:41:49.717548+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:36.151432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:41:36.151432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:53.930320+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":628,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.717671+0000","last_change":"2024-09-16T09:40:29.980587+0000","last_active":"2024-09-16T09:41:49.717671+0000","last_peered":"2024-09-16T09:41:49.717671+0000","last_clean":"2024-09-16T09:41:49.717671+0000","last_became_active":"2024-09-16T09:33:30.248992+0000","last_became_peered":"2024-09-16T09:33:30.248992+0000","last_unstale":"2024-09-16T09:41:49.717671+0000","last_undegraded":"2024-09-16T09:41:49.717671+0000","last_fullsized":"2024-09-16T09:41:49.717671+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:40:29.980515+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:40:29.980515+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":863,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.710689+0000","last_change":"2024-09-16T09:40:35.646704+0000","last_active":"2024-09-16T09:41:49.710689+0000","last_peered":"2024-09-16T09:41:49.710689+0000","last_clean":"2024-09-16T09:41:49.710689+0000","last_became_active":"2024-09-16T09:25:07.722050+0000","last_became_peered":"2024-09-16T09:25:07.722050+0000","last_unstale":"2024-09-16T09:41:49.710689+0000","last_undegraded":"2024-09-16T09:41:49.710689+0000","last_fullsized":"2024-09-16T09:41:49.710689+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:40:35.646535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:40:35.646535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.10","version":"73'79","reported_seq":663,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.713564+0000","last_change":"2024-09-16T09:41:08.828451+0000","last_active":"2024-09-16T09:41:49.713564+0000","last_peered":"2024-09-16T09:41:49.713564+0000","last_clean":"2024-09-16T09:41:49.713564+0000","last_became_active":"2024-09-16T09:30:10.692770+0000","last_became_peered":"2024-09-16T09:30:10.692770+0000","last_unstale":"2024-09-16T09:41:49.713564+0000","last_undegraded":"2024-09-16T09:41:49.713564+0000","last_fullsized":"2024-09-16T09:41:49.713564+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:41:08.828394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:41:08.828394+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:19.530053+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"285'15","reported_seq":885,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.714366+0000","last_change":"2024-09-16T09:41:40.700560+0000","last_active":"2024-09-16T09:41:49.714366+0000","last_peered":"2024-09-16T09:41:49.714366+0000","last_clean":"2024-09-16T09:41:49.714366+0000","last_became_active":"2024-09-16T09:25:03.717310+0000","last_became_peered":"2024-09-16T09:25:03.717310+0000","last_unstale":"2024-09-16T09:41:49.714366+0000","last_undegraded":"2024-09-16T09:41:49.714366+0000","last_fullsized":"2024-09-16T09:41:49.714366+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'15","last_scrub_stamp":"2024-09-16T09:41:40.700318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:41:40.700318+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:44.976002+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":894,"reported_epoch":466,"state":"active+clean","last_fresh":"2024-09-16T09:41:44.422851+0000","last_change":"2024-09-16T09:40:23.065431+0000","last_active":"2024-09-16T09:41:44.422851+0000","last_peered":"2024-09-16T09:41:44.422851+0000","last_clean":"2024-09-16T09:41:44.422851+0000","last_became_active":"2024-09-16T09:31:09.272848+0000","last_became_peered":"2024-09-16T09:31:09.272848+0000","last_unstale":"2024-09-16T09:41:44.422851+0000","last_undegraded":"2024-09-16T09:41:44.422851+0000","last_fullsized":"2024-09-16T09:41:44.422851+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:40:23.065365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:40:23.065365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"187'11","reported_seq":903,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.718108+0000","last_change":"2024-09-16T09:37:19.540982+0000","last_active":"2024-09-16T09:41:49.718108+0000","last_peered":"2024-09-16T09:41:49.718108+0000","last_clean":"2024-09-16T09:41:49.718108+0000","last_became_active":"2024-09-16T09:31:09.277688+0000","last_became_peered":"2024-09-16T09:31:09.277688+0000","last_unstale":"2024-09-16T09:41:49.718108+0000","last_undegraded":"2024-09-16T09:41:49.718108+0000","last_fullsized":"2024-09-16T09:41:49.718108+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"187'11","last_scrub_stamp":"2024-09-16T09:37:19.540886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:37:19.540886+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:17.028744+0000","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":877,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.711424+0000","last_change":"2024-09-16T09:41:36.098715+0000","last_active":"2024-09-16T09:41:49.711424+0000","last_peered":"2024-09-16T09:41:49.711424+0000","last_clean":"2024-09-16T09:41:49.711424+0000","last_became_active":"2024-09-16T09:25:07.718810+0000","last_became_peered":"2024-09-16T09:25:07.718810+0000","last_unstale":"2024-09-16T09:41:49.711424+0000","last_undegraded":"2024-09-16T09:41:49.711424+0000","last_fullsized":"2024-09-16T09:41:49.711424+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:36.098648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:41:36.098648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:43.824638+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.11","version":"73'79","reported_seq":738,"reported_epoch":466,"state":"active+clean","last_fresh":"2024-09-16T09:41:44.423202+0000","last_change":"2024-09-16T09:37:15.623104+0000","last_active":"2024-09-16T09:41:44.423202+0000","last_peered":"2024-09-16T09:41:44.423202+0000","last_clean":"2024-09-16T09:41:44.423202+0000","last_became_active":"2024-09-16T09:31:09.272208+0000","last_became_peered":"2024-09-16T09:31:09.272208+0000","last_unstale":"2024-09-16T09:41:44.423202+0000","last_undegraded":"2024-09-16T09:41:44.423202+0000","last_fullsized":"2024-09-16T09:41:44.423202+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:37:15.623041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:37:15.623041+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:14.936019+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.14","version":"285'20","reported_seq":553,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.718233+0000","last_change":"2024-09-16T09:40:35.027221+0000","last_active":"2024-09-16T09:41:49.718233+0000","last_peered":"2024-09-16T09:41:49.718233+0000","last_clean":"2024-09-16T09:41:49.718233+0000","last_became_active":"2024-09-16T09:33:30.249769+0000","last_became_peered":"2024-09-16T09:33:30.249769+0000","last_unstale":"2024-09-16T09:41:49.718233+0000","last_undegraded":"2024-09-16T09:41:49.718233+0000","last_fullsized":"2024-09-16T09:41:49.718233+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'20","last_scrub_stamp":"2024-09-16T09:40:35.027096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:40:35.027096+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":863,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.718159+0000","last_change":"2024-09-16T09:41:08.005775+0000","last_active":"2024-09-16T09:41:49.718159+0000","last_peered":"2024-09-16T09:41:49.718159+0000","last_clean":"2024-09-16T09:41:49.718159+0000","last_became_active":"2024-09-16T09:25:01.707513+0000","last_became_peered":"2024-09-16T09:25:01.707513+0000","last_unstale":"2024-09-16T09:41:49.718159+0000","last_undegraded":"2024-09-16T09:41:49.718159+0000","last_fullsized":"2024-09-16T09:41:49.718159+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:08.005728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:41:08.005728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:35.931885+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"66'8","reported_seq":1083,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.714137+0000","last_change":"2024-09-16T09:41:42.773433+0000","last_active":"2024-09-16T09:41:49.714137+0000","last_peered":"2024-09-16T09:41:49.714137+0000","last_clean":"2024-09-16T09:41:49.714137+0000","last_became_active":"2024-09-16T09:25:05.713833+0000","last_became_peered":"2024-09-16T09:25:05.713833+0000","last_unstale":"2024-09-16T09:41:49.714137+0000","last_undegraded":"2024-09-16T09:41:49.714137+0000","last_fullsized":"2024-09-16T09:41:49.714137+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:41:42.773359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:41:42.773359+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:42.814428+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":881,"reported_epoch":466,"state":"active+clean","last_fresh":"2024-09-16T09:41:44.423412+0000","last_change":"2024-09-16T09:40:26.019049+0000","last_active":"2024-09-16T09:41:44.423412+0000","last_peered":"2024-09-16T09:41:44.423412+0000","last_clean":"2024-09-16T09:41:44.423412+0000","last_became_active":"2024-09-16T09:31:09.275168+0000","last_became_peered":"2024-09-16T09:31:09.275168+0000","last_unstale":"2024-09-16T09:41:44.423412+0000","last_undegraded":"2024-09-16T09:41:44.423412+0000","last_fullsized":"2024-09-16T09:41:44.423412+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:40:26.018988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:40:26.018988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"285'19","reported_seq":896,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.710871+0000","last_change":"2024-09-16T09:37:31.362985+0000","last_active":"2024-09-16T09:41:49.710871+0000","last_peered":"2024-09-16T09:41:49.710871+0000","last_clean":"2024-09-16T09:41:49.710871+0000","last_became_active":"2024-09-16T09:26:33.291265+0000","last_became_peered":"2024-09-16T09:26:33.291265+0000","last_unstale":"2024-09-16T09:41:49.710871+0000","last_undegraded":"2024-09-16T09:41:49.710871+0000","last_fullsized":"2024-09-16T09:41:49.710871+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'19","last_scrub_stamp":"2024-09-16T09:37:31.362927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:37:31.362927+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:16.270309+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":882,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.711429+0000","last_change":"2024-09-16T09:41:29.092238+0000","last_active":"2024-09-16T09:41:49.711429+0000","last_peered":"2024-09-16T09:41:49.711429+0000","last_clean":"2024-09-16T09:41:49.711429+0000","last_became_active":"2024-09-16T09:25:01.705158+0000","last_became_peered":"2024-09-16T09:25:01.705158+0000","last_unstale":"2024-09-16T09:41:49.711429+0000","last_undegraded":"2024-09-16T09:41:49.711429+0000","last_fullsized":"2024-09-16T09:41:49.711429+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:29.091987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:41:29.091987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:56.880823+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":801,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.713815+0000","last_change":"2024-09-16T09:41:37.694845+0000","last_active":"2024-09-16T09:41:49.713815+0000","last_peered":"2024-09-16T09:41:49.713815+0000","last_clean":"2024-09-16T09:41:49.713815+0000","last_became_active":"2024-09-16T09:37:41.515278+0000","last_became_peered":"2024-09-16T09:37:41.515278+0000","last_unstale":"2024-09-16T09:41:49.713815+0000","last_undegraded":"2024-09-16T09:41:49.713815+0000","last_fullsized":"2024-09-16T09:41:49.713815+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:37.694724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:41:37.694724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:43.549832+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":706,"reported_epoch":466,"state":"active+clean","last_fresh":"2024-09-16T09:41:44.422862+0000","last_change":"2024-09-16T09:41:05.972422+0000","last_active":"2024-09-16T09:41:44.422862+0000","last_peered":"2024-09-16T09:41:44.422862+0000","last_clean":"2024-09-16T09:41:44.422862+0000","last_became_active":"2024-09-16T09:31:09.276042+0000","last_became_peered":"2024-09-16T09:31:09.276042+0000","last_unstale":"2024-09-16T09:41:44.422862+0000","last_undegraded":"2024-09-16T09:41:44.422862+0000","last_fullsized":"2024-09-16T09:41:44.422862+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:05.972363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:41:05.972363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:09.478445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"285'15","reported_seq":584,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.710011+0000","last_change":"2024-09-16T09:41:09.949578+0000","last_active":"2024-09-16T09:41:49.710011+0000","last_peered":"2024-09-16T09:41:49.710011+0000","last_clean":"2024-09-16T09:41:49.710011+0000","last_became_active":"2024-09-16T09:37:33.348223+0000","last_became_peered":"2024-09-16T09:37:33.348223+0000","last_unstale":"2024-09-16T09:41:49.710011+0000","last_undegraded":"2024-09-16T09:41:49.710011+0000","last_fullsized":"2024-09-16T09:41:49.710011+0000","mapping_epoch":367,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":368,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'15","last_scrub_stamp":"2024-09-16T09:41:09.949519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:41:09.949519+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:35.859726+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,2],"acting":[6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":863,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.717922+0000","last_change":"2024-09-16T09:41:44.170251+0000","last_active":"2024-09-16T09:41:49.717922+0000","last_peered":"2024-09-16T09:41:49.717922+0000","last_clean":"2024-09-16T09:41:49.717922+0000","last_became_active":"2024-09-16T09:25:01.706762+0000","last_became_peered":"2024-09-16T09:25:01.706762+0000","last_unstale":"2024-09-16T09:41:49.717922+0000","last_undegraded":"2024-09-16T09:41:49.717922+0000","last_fullsized":"2024-09-16T09:41:49.717922+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:44.170186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:41:44.170186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:51.467157+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":699,"reported_epoch":466,"state":"active+clean","last_fresh":"2024-09-16T09:41:44.422994+0000","last_change":"2024-09-16T09:41:36.130305+0000","last_active":"2024-09-16T09:41:44.422994+0000","last_peered":"2024-09-16T09:41:44.422994+0000","last_clean":"2024-09-16T09:41:44.422994+0000","last_became_active":"2024-09-16T09:28:52.982865+0000","last_became_peered":"2024-09-16T09:28:52.982865+0000","last_unstale":"2024-09-16T09:41:44.422994+0000","last_undegraded":"2024-09-16T09:41:44.422994+0000","last_fullsized":"2024-09-16T09:41:44.422994+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:36.130240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:41:36.130240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:05.802731+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":844,"reported_epoch":466,"state":"active+clean","last_fresh":"2024-09-16T09:41:44.422727+0000","last_change":"2024-09-16T09:34:50.779241+0000","last_active":"2024-09-16T09:41:44.422727+0000","last_peered":"2024-09-16T09:41:44.422727+0000","last_clean":"2024-09-16T09:41:44.422727+0000","last_became_active":"2024-09-16T09:25:07.715301+0000","last_became_peered":"2024-09-16T09:25:07.715301+0000","last_unstale":"2024-09-16T09:41:44.422727+0000","last_undegraded":"2024-09-16T09:41:44.422727+0000","last_fullsized":"2024-09-16T09:41:44.422727+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:50.779164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:34:50.779164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:58.017822+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.14","version":"73'79","reported_seq":672,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.710659+0000","last_change":"2024-09-16T09:41:38.531852+0000","last_active":"2024-09-16T09:41:49.710659+0000","last_peered":"2024-09-16T09:41:49.710659+0000","last_clean":"2024-09-16T09:41:49.710659+0000","last_became_active":"2024-09-16T09:30:10.700866+0000","last_became_peered":"2024-09-16T09:30:10.700866+0000","last_unstale":"2024-09-16T09:41:49.710659+0000","last_undegraded":"2024-09-16T09:41:49.710659+0000","last_fullsized":"2024-09-16T09:41:49.710659+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:41:38.531780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:41:38.531780+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:55.556470+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"285'19","reported_seq":742,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.717411+0000","last_change":"2024-09-16T09:39:28.862654+0000","last_active":"2024-09-16T09:41:49.717411+0000","last_peered":"2024-09-16T09:41:49.717411+0000","last_clean":"2024-09-16T09:41:49.717411+0000","last_became_active":"2024-09-16T09:28:52.971132+0000","last_became_peered":"2024-09-16T09:28:52.971132+0000","last_unstale":"2024-09-16T09:41:49.717411+0000","last_undegraded":"2024-09-16T09:41:49.717411+0000","last_fullsized":"2024-09-16T09:41:49.717411+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'19","last_scrub_stamp":"2024-09-16T09:39:28.862503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:39:28.862503+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:50.180249+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":872,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.714926+0000","last_change":"2024-09-16T09:41:47.768655+0000","last_active":"2024-09-16T09:41:49.714926+0000","last_peered":"2024-09-16T09:41:49.714926+0000","last_clean":"2024-09-16T09:41:49.714926+0000","last_became_active":"2024-09-16T09:25:01.700733+0000","last_became_peered":"2024-09-16T09:25:01.700733+0000","last_unstale":"2024-09-16T09:41:49.714926+0000","last_undegraded":"2024-09-16T09:41:49.714926+0000","last_fullsized":"2024-09-16T09:41:49.714926+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:47.768574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:41:47.768574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:13.312512+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":863,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.716232+0000","last_change":"2024-09-16T09:40:39.969447+0000","last_active":"2024-09-16T09:41:49.716232+0000","last_peered":"2024-09-16T09:41:49.716232+0000","last_clean":"2024-09-16T09:41:49.716232+0000","last_became_active":"2024-09-16T09:25:05.713479+0000","last_became_peered":"2024-09-16T09:25:05.713479+0000","last_unstale":"2024-09-16T09:41:49.716232+0000","last_undegraded":"2024-09-16T09:41:49.716232+0000","last_fullsized":"2024-09-16T09:41:49.716232+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:40:39.969383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:40:39.969383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:57.516232+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":722,"reported_epoch":466,"state":"active+clean","last_fresh":"2024-09-16T09:41:44.423432+0000","last_change":"2024-09-16T09:41:34.178100+0000","last_active":"2024-09-16T09:41:44.423432+0000","last_peered":"2024-09-16T09:41:44.423432+0000","last_clean":"2024-09-16T09:41:44.423432+0000","last_became_active":"2024-09-16T09:31:09.274797+0000","last_became_peered":"2024-09-16T09:31:09.274797+0000","last_unstale":"2024-09-16T09:41:44.423432+0000","last_undegraded":"2024-09-16T09:41:44.423432+0000","last_fullsized":"2024-09-16T09:41:44.423432+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:34.178020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:41:34.178020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:55.711514+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"285'8","reported_seq":627,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.709790+0000","last_change":"2024-09-16T09:41:04.769967+0000","last_active":"2024-09-16T09:41:49.709790+0000","last_peered":"2024-09-16T09:41:49.709790+0000","last_clean":"2024-09-16T09:41:49.709790+0000","last_became_active":"2024-09-16T09:31:09.273682+0000","last_became_peered":"2024-09-16T09:31:09.273682+0000","last_unstale":"2024-09-16T09:41:49.709790+0000","last_undegraded":"2024-09-16T09:41:49.709790+0000","last_fullsized":"2024-09-16T09:41:49.709790+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'8","last_scrub_stamp":"2024-09-16T09:41:04.769863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:41:04.769863+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:18.575428+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":9,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":893,"reported_epoch":466,"state":"active+clean","last_fresh":"2024-09-16T09:41:45.045179+0000","last_change":"2024-09-16T09:41:45.045179+0000","last_active":"2024-09-16T09:41:45.045179+0000","last_peered":"2024-09-16T09:41:45.045179+0000","last_clean":"2024-09-16T09:41:45.045179+0000","last_became_active":"2024-09-16T09:31:09.274719+0000","last_became_peered":"2024-09-16T09:31:09.274719+0000","last_unstale":"2024-09-16T09:41:45.045179+0000","last_undegraded":"2024-09-16T09:41:45.045179+0000","last_fullsized":"2024-09-16T09:41:45.045179+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:45.045117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:41:45.045117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:56.298444+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":874,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.709220+0000","last_change":"2024-09-16T09:38:35.092025+0000","last_active":"2024-09-16T09:41:49.709220+0000","last_peered":"2024-09-16T09:41:49.709220+0000","last_clean":"2024-09-16T09:41:49.709220+0000","last_became_active":"2024-09-16T09:25:05.712545+0000","last_became_peered":"2024-09-16T09:25:05.712545+0000","last_unstale":"2024-09-16T09:41:49.709220+0000","last_undegraded":"2024-09-16T09:41:49.709220+0000","last_fullsized":"2024-09-16T09:41:49.709220+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:38:35.091983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:38:35.091983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:14.921045+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":859,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.717397+0000","last_change":"2024-09-16T09:41:29.261932+0000","last_active":"2024-09-16T09:41:49.717397+0000","last_peered":"2024-09-16T09:41:49.717397+0000","last_clean":"2024-09-16T09:41:49.717397+0000","last_became_active":"2024-09-16T09:25:07.720117+0000","last_became_peered":"2024-09-16T09:25:07.720117+0000","last_unstale":"2024-09-16T09:41:49.717397+0000","last_undegraded":"2024-09-16T09:41:49.717397+0000","last_fullsized":"2024-09-16T09:41:49.717397+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:29.261841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:41:29.261841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:57.269436+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":489,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.714230+0000","last_change":"2024-09-16T09:36:52.091335+0000","last_active":"2024-09-16T09:41:49.714230+0000","last_peered":"2024-09-16T09:41:49.714230+0000","last_clean":"2024-09-16T09:41:49.714230+0000","last_became_active":"2024-09-16T09:33:30.248036+0000","last_became_peered":"2024-09-16T09:33:30.248036+0000","last_unstale":"2024-09-16T09:41:49.714230+0000","last_undegraded":"2024-09-16T09:41:49.714230+0000","last_fullsized":"2024-09-16T09:41:49.714230+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:52.088284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:36:52.088284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:52.994001+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":713,"reported_epoch":466,"state":"active+clean","last_fresh":"2024-09-16T09:41:44.423269+0000","last_change":"2024-09-16T09:37:18.825035+0000","last_active":"2024-09-16T09:41:44.423269+0000","last_peered":"2024-09-16T09:41:44.423269+0000","last_clean":"2024-09-16T09:41:44.423269+0000","last_became_active":"2024-09-16T09:31:09.271662+0000","last_became_peered":"2024-09-16T09:31:09.271662+0000","last_unstale":"2024-09-16T09:41:44.423269+0000","last_undegraded":"2024-09-16T09:41:44.423269+0000","last_fullsized":"2024-09-16T09:41:44.423269+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:18.824937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:37:18.824937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:27.104501+0000","scrub_duration":79,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"285'19","reported_seq":894,"reported_epoch":467,"state":"active+clean","last_fresh":"2024-09-16T09:41:49.711781+0000","last_change":"2024-09-16T09:37:25.995663+0000","last_active":"2024-09-16T09:41:49.711781+0000","last_peered":"2024-09-16T09:41:49.711781+0000","last_clean":"2024-09-16T09:41:49.711781+0000","last_became_active":"2024-09-16T09:25:03.707769+0000","last_became_peered":"2024-09-16T09:25:03.707769+0000","last_unstale":"2024-09-16T09:41:49.711781+0000","last_undegraded":"2024-09-16T09:41:49.711781+0000","last_fullsized":"2024-09-16T09:41:49.711781+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'19","last_scrub_stamp":"2024-09-16T09:37:25.995607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:37:25.995607+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:41:49.965137+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":857,"reported_epoch":466,"state":"active+clean","last_fresh":"2024-09-16T09:41:44.423914+0000","last_change":"2024-09-16T09:41:29.136632+0000","last_active":"2024-09-16T09:41:44.423914+0000","last_peered":"2024-09-16T09:41:44.423914+0000","last_clean":"2024-09-16T09:41:44.423914+0000","last_became_active":"2024-09-16T09:25:05.711952+0000","last_became_peered":"2024-09-16T09:25:05.711952+0000","last_unstale":"2024-09-16T09:41:44.423914+0000","last_undegraded":"2024-09-16T09:41:44.423914+0000","last_fullsized":"2024-09-16T09:41:44.423914+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:29.136452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:41:29.136452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:51.188109+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":16,"stat_sum":{"num_bytes":4220649472,"num_objects":64402,"num_object_clones":0,"num_object_copies":193206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64402,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":64402,"num_write_kb":4121728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12675448832,"data_stored":12675448832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64402,"ondisk_log_size":64402,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":1117,"num_read_kb":874,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":3686,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":681,"ondisk_log_size":681,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1986,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1004,"num_read_kb":999,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":21,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":327680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1659,"ondisk_log_size":1659,"up":63,"acting":63,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266895,"num_pgs":69,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2240524,"kb_used_data":1533244,"kb_used_omap":1,"kb_used_meta":707262,"kb_avail":91504628,"statfs":{"total":95995035648,"available":93700739072,"internally_reserved":0,"allocated":1570041856,"data_stored":1569552689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":724236746},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495828,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2691696,"kb_used_data":2056608,"kb_used_omap":1,"kb_used_meta":635070,"kb_avail":91053456,"statfs":{"total":95995035648,"available":93238738944,"internally_reserved":0,"allocated":2105966592,"data_stored":2105504889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":650312134},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,2,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724763,"num_pgs":62,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1783300,"kb_used_data":1026036,"kb_used_omap":1,"kb_used_meta":757246,"kb_avail":91961852,"statfs":{"total":95995035648,"available":94168936448,"internally_reserved":0,"allocated":1050660864,"data_stored":1050179914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":775420358},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953696,"num_pgs":69,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1696780,"kb_used_data":1025596,"kb_used_omap":1,"kb_used_meta":671166,"kb_avail":92048372,"statfs":{"total":95995035648,"available":94257532928,"internally_reserved":0,"allocated":1050210304,"data_stored":1049721091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":687274438},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149926,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2660116,"kb_used_data":1826500,"kb_used_omap":1,"kb_used_meta":833598,"kb_avail":91085036,"statfs":{"total":95995035648,"available":93271076864,"internally_reserved":0,"allocated":1870336000,"data_stored":1869839315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":853604807},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,3,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411563,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2190956,"kb_used_data":1541980,"kb_used_omap":1,"kb_used_meta":648958,"kb_avail":91554196,"statfs":{"total":95995035648,"available":93751496704,"internally_reserved":0,"allocated":1578987520,"data_stored":1578530169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":664533445},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575090,"num_pgs":69,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3018920,"kb_used_data":2095064,"kb_used_omap":1,"kb_used_meta":923838,"kb_avail":90726232,"statfs":{"total":95995035648,"available":92903661568,"internally_reserved":0,"allocated":2145345536,"data_stored":2144827631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":946010565},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738615,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2001788,"kb_used_data":1212588,"kb_used_omap":1,"kb_used_meta":789182,"kb_avail":91743364,"statfs":{"total":95995035648,"available":93945204736,"internally_reserved":0,"allocated":1241690112,"data_stored":1241209068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":808122821},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2],"upper_bound":4},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":1788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1223819264,"data_stored":1223819264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2157510656,"data_stored":2157510656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1603796992,"data_stored":1603796992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1869086720,"data_stored":1869086720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1071316992,"data_stored":1071316992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1062076416,"data_stored":1062076416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":2105737216,"data_stored":2105737216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1582104576,"data_stored":1582104576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T09:41:58.482 INFO:tasks.thrashosds.thrasher:Setting 5.1a to [6, 7, 4] 2024-09-16T09:41:58.482 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '5.1a', '6', '7', '4'] 2024-09-16T09:41:58.482 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd pg-upmap 5.1a 6 7 4 2024-09-16T09:41:58.483 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:41:58.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:58.484 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:58.484 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:41:58.584 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:41:58.606 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:58.606 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:58.606 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:41:58.707 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:41:58.732 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:58.748 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:58.749 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:41:58.850 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:41:58.870 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:58.871 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:58.871 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:41:58.972 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:41:59.002 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:59.003 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:59.003 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:41:59.095 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 39 16 71536 71520 114.597 33.5 0.00765843 0.0087216 2024-09-16T09:41:59.104 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:41:59.133 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:59.134 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:59.134 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:41:59.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:41:58 smithi084 ceph-mon[32629]: pgmap v891: 168 pgs: 168 active+clean; 4.1 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 66 MiB/s wr, 1.06k op/s 2024-09-16T09:41:59.162 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:41:58 smithi070 ceph-mon[30388]: pgmap v891: 168 pgs: 168 active+clean; 4.1 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 66 MiB/s wr, 1.06k op/s 2024-09-16T09:41:59.163 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:41:58 smithi070 ceph-mon[40691]: pgmap v891: 168 pgs: 168 active+clean; 4.1 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 66 MiB/s wr, 1.06k op/s 2024-09-16T09:41:59.216 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.2 dump_blocked_ops 2024-09-16T09:41:59.235 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:41:59.269 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:59.269 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:59.270 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:41:59.370 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:41:59.403 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:59.404 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:59.404 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:41:59.498 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:41:59.505 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:41:59.535 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:59.536 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:59.536 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:41:59.637 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:41:59.675 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:59.676 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:59.676 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:41:59.777 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:41:59.809 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:59.810 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:41:59.810 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:41:59.811 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T09:41:59.910 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:41:59.950 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:41:59.950 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:41:59.951 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:42:00.052 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:42:00.096 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:2024-09-16T09:42:00.095399+0000 min lat: 0.00125032 max lat: 0.55794 avg lat: 0.00881478 2024-09-16T09:42:00.096 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-16T09:42:00.096 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 40 16 72585 72569 113.369 65.5625 0.00982955 0.00881478 2024-09-16T09:42:00.130 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:00.131 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:00.132 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:42:00.233 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:42:00.257 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:00.258 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:00.258 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:42:00.260 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:42:00.359 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:42:00.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:00.427 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:00.428 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:42:00.468 INFO:teuthology.orchestra.run.smithi070.stderr:nodeep-scrub is set 2024-09-16T09:42:00.529 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:42:00.560 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:00.561 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:00.561 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:42:00.662 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:42:00.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:00 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:42:00.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:00 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2598893973' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:42:00.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:00 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:42:00.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:00 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:42:00.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:00 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:42:00.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:00 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:42:00.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:00 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:42:00.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:00 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/2598893973' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:42:00.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:00 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:42:00.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:00 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:42:00.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:00 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:42:00.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:00 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:42:00.710 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:00.711 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:00.711 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:42:00.812 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:42:00.822 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:42:00.859 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:00.860 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:00.860 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:42:00.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:00 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:42:00.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:00 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/2598893973' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:42:00.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:00 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:42:00.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:00 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:42:00.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:00 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:42:00.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:00 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:42:00.961 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:42:00.991 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:00.991 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:00.992 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:42:01.093 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:42:01.096 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 41 16 73723 73707 112.339 71.125 0.0290373 0.00889647 2024-09-16T09:42:01.137 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T09:42:01.137 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:42:01.154 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:01.155 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:01.155 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:42:01.256 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:42:01.283 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:01.284 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:01.284 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:42:01.385 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:42:01.417 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:01.417 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:01.418 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:42:01.482 INFO:teuthology.orchestra.run.smithi070.stderr:set 5.1a pg_upmap mapping to [6,7,4] 2024-09-16T09:42:01.518 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:42:01.562 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:01.563 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:01.563 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:42:01.664 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:42:01.704 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:01.705 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:01.705 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:42:01.721 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:01 smithi070 ceph-mon[30388]: pgmap v892: 168 pgs: 168 active+clean; 4.3 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 1023 B/s rd, 82 MiB/s wr, 1.32k op/s 2024-09-16T09:42:01.721 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:01 smithi070 ceph-mon[30388]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:42:01.721 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:01 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2598893973' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-16T09:42:01.721 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:01 smithi070 ceph-mon[30388]: osdmap e469: 8 total, 8 up, 8 in 2024-09-16T09:42:01.721 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:01 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3267305540' entity='client.admin' cmd={"prefix": "osd pg-upmap", "pgid": "5.1a", "id": [6, 7, 4]} : dispatch 2024-09-16T09:42:01.721 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:01 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:42:01.722 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:01 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:42:01.722 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:01 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:42:01.722 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:01 smithi070 ceph-mon[40691]: pgmap v892: 168 pgs: 168 active+clean; 4.3 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 1023 B/s rd, 82 MiB/s wr, 1.32k op/s 2024-09-16T09:42:01.722 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:01 smithi070 ceph-mon[40691]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:42:01.722 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:01 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/2598893973' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-16T09:42:01.722 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:01 smithi070 ceph-mon[40691]: osdmap e469: 8 total, 8 up, 8 in 2024-09-16T09:42:01.722 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:01 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3267305540' entity='client.admin' cmd={"prefix": "osd pg-upmap", "pgid": "5.1a", "id": [6, 7, 4]} : dispatch 2024-09-16T09:42:01.722 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:01 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:42:01.722 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:01 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:42:01.723 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:01 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:42:01.806 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:42:01.842 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:01.843 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:01.843 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:42:01.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:01 smithi084 ceph-mon[32629]: pgmap v892: 168 pgs: 168 active+clean; 4.3 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 1023 B/s rd, 82 MiB/s wr, 1.32k op/s 2024-09-16T09:42:01.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:01 smithi084 ceph-mon[32629]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:42:01.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:01 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/2598893973' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-16T09:42:01.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:01 smithi084 ceph-mon[32629]: osdmap e469: 8 total, 8 up, 8 in 2024-09-16T09:42:01.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:01 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3267305540' entity='client.admin' cmd={"prefix": "osd pg-upmap", "pgid": "5.1a", "id": [6, 7, 4]} : dispatch 2024-09-16T09:42:01.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:01 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:42:01.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:01 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:42:01.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:01 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:42:01.944 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:42:01.982 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:01.983 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:01.983 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:42:02.084 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:42:02.096 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 42 16 74912 74896 111.433 74.3125 0.0104319 0.00896941 2024-09-16T09:42:02.127 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:02.128 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:02.128 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:42:02.171 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:42:01 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:42:01] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:42:02.230 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:42:02.267 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:02.267 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:02.268 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:42:02.369 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:42:02.403 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:02.403 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:02.404 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:42:02.504 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:42:02.534 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:02.535 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:02.535 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:42:02.587 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:02 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3267305540' entity='client.admin' cmd='[{"prefix": "osd pg-upmap", "pgid": "5.1a", "id": [6, 7, 4]}]': finished 2024-09-16T09:42:02.587 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:02 smithi070 ceph-mon[40691]: osdmap e470: 8 total, 8 up, 8 in 2024-09-16T09:42:02.587 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:02 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3267305540' entity='client.admin' cmd='[{"prefix": "osd pg-upmap", "pgid": "5.1a", "id": [6, 7, 4]}]': finished 2024-09-16T09:42:02.587 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:02 smithi070 ceph-mon[30388]: osdmap e470: 8 total, 8 up, 8 in 2024-09-16T09:42:02.635 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:42:02.671 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:02.671 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:02.672 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:42:02.773 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:42:02.806 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:02.806 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:02.807 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:42:02.907 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:42:02.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:02 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3267305540' entity='client.admin' cmd='[{"prefix": "osd pg-upmap", "pgid": "5.1a", "id": [6, 7, 4]}]': finished 2024-09-16T09:42:02.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:02 smithi084 ceph-mon[32629]: osdmap e470: 8 total, 8 up, 8 in 2024-09-16T09:42:02.948 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:02.948 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:02.949 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:42:03.049 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:42:03.088 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:03.089 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:03.089 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:42:03.096 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 43 15 76269 76254 110.815 84.875 0.0233976 0.00901939 2024-09-16T09:42:03.190 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:42:03.220 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:03.221 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:03.221 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:42:03.322 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:42:03.358 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:03.358 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:03.359 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:42:03.460 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:42:03.500 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:03.501 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:03.501 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:42:03.558 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:03 smithi070 ceph-mon[30388]: pgmap v895: 168 pgs: 168 active+clean; 4.3 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 691 B/s rd, 55 MiB/s wr, 876 op/s 2024-09-16T09:42:03.558 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:03 smithi070 ceph-mon[30388]: osdmap e471: 8 total, 8 up, 8 in 2024-09-16T09:42:03.559 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:03 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1114340695' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:42:03.559 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:03 smithi070 ceph-mon[40691]: pgmap v895: 168 pgs: 168 active+clean; 4.3 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 691 B/s rd, 55 MiB/s wr, 876 op/s 2024-09-16T09:42:03.559 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:03 smithi070 ceph-mon[40691]: osdmap e471: 8 total, 8 up, 8 in 2024-09-16T09:42:03.559 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:03 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1114340695' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:42:03.561 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:42:03.561 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:42:03.561 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:42:03.561 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:42:03.561 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:42:03.561 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:42:03.561 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:42:03.561 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:42:03.561 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:42:03.561 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:42:03.562 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:42:03.562 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:42:03.562 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:42:03.562 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:42:03.562 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:42:03.562 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:42:03.602 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:42:03.633 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:03.633 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:03.634 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:42:03.721 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd unset noscrub 2024-09-16T09:42:03.735 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:42:03.769 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:03.770 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:03.771 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:42:03.871 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:42:03.903 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:03.904 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:03.904 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:42:03.910 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:42:03 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:42:03.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:03 smithi084 ceph-mon[32629]: pgmap v895: 168 pgs: 168 active+clean; 4.3 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 691 B/s rd, 55 MiB/s wr, 876 op/s 2024-09-16T09:42:03.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:03 smithi084 ceph-mon[32629]: osdmap e471: 8 total, 8 up, 8 in 2024-09-16T09:42:03.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:03 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1114340695' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:42:04.006 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:42:04.050 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:04.051 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:04.051 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:42:04.096 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 44 16 77531 77515 110.087 78.8125 0.00907042 0.009079 2024-09-16T09:42:04.152 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:42:04.188 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:04.189 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:04.189 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:42:04.290 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:42:04.322 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:04.323 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:04.323 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:42:04.424 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:42:04.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:04.481 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:04.481 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:42:04.582 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:42:04.592 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:04 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:42:04.592 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:04 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:42:04.592 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:04 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:42:04.592 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:04 smithi070 ceph-mon[30388]: osdmap e472: 8 total, 8 up, 8 in 2024-09-16T09:42:04.593 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:04 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:42:04.593 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:04 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:42:04.593 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:04 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:42:04.593 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:04 smithi070 ceph-mon[40691]: osdmap e472: 8 total, 8 up, 8 in 2024-09-16T09:42:04.612 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:04.613 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:04.613 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:42:04.652 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.2 dump_historic_ops 2024-09-16T09:42:04.714 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:42:04.745 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:42:04.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:04.773 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:04.774 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:42:04.874 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:42:04.905 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:04.905 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:04.906 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:42:04.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:04 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:42:04.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:04 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:42:04.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:04 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:42:04.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:04 smithi084 ceph-mon[32629]: osdmap e472: 8 total, 8 up, 8 in 2024-09-16T09:42:05.007 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:42:05.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:05.051 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:05.052 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:42:05.096 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 45 16 78778 78762 109.373 77.9375 0.0100173 0.0091373 2024-09-16T09:42:05.152 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:42:05.195 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:05.195 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:05.196 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:42:05.297 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:42:05.334 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:05.335 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:05.335 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:42:05.436 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:42:05.478 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:05.479 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:05.480 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:42:05.580 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:42:05.616 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:05.616 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:05.617 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:42:05.717 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:42:05.743 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:42:05.752 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:05.752 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:05.753 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:42:05.854 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:42:05.880 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:05 smithi070 ceph-mon[30388]: pgmap v897: 168 pgs: 168 active+clean; 4.6 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 852 B/s rd, 100 MiB/s wr, 1.60k op/s 2024-09-16T09:42:05.880 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:05 smithi070 ceph-mon[40691]: pgmap v897: 168 pgs: 168 active+clean; 4.6 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 852 B/s rd, 100 MiB/s wr, 1.60k op/s 2024-09-16T09:42:05.898 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:05.899 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:05.899 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:42:05.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:05 smithi084 ceph-mon[32629]: pgmap v897: 168 pgs: 168 active+clean; 4.6 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 852 B/s rd, 100 MiB/s wr, 1.60k op/s 2024-09-16T09:42:06.000 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:42:06.040 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:06.041 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:06.041 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:42:06.096 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 46 16 80036 80020 108.704 78.625 0.00855115 0.00919505 2024-09-16T09:42:06.142 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:42:06.183 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:06.183 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:06.184 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:42:06.284 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:42:06.318 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:06.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:06.322 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:42:06.360 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T09:42:06.389 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T09:42:06.389 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 20, 2024-09-16T09:42:06.389 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 600, 2024-09-16T09:42:06.389 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [ 2024-09-16T09:42:06.389 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.390 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.15975.0:33579 10.9 e386/376 10:99683988:::benchmark_data_smithi070_166744_object33578:head v 386'2079, mlcod=386'2076)", 2024-09-16T09:42:06.390 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:38:09.640462+0000", 2024-09-16T09:42:06.390 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 236.74431923200001, 2024-09-16T09:42:06.390 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.28017080700000002, 2024-09-16T09:42:06.390 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:42:06.390 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:42:06.390 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:42:06.390 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:42:06.390 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.390 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:42:06.390 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:38:09.640462+0000", 2024-09-16T09:42:06.390 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:42:06.390 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.390 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.391 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:42:06.391 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:38:09.640462+0000", 2024-09-16T09:42:06.391 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:42:06.391 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.391 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.391 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:42:06.391 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:38:09.640463+0000", 2024-09-16T09:42:06.391 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.5499999999999997e-07 2024-09-16T09:42:06.391 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.391 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.391 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:42:06.391 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:38:09.640526+0000", 2024-09-16T09:42:06.391 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.3503999999999994e-05 2024-09-16T09:42:06.391 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.392 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.392 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:42:06.392 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:38:09.640535+0000", 2024-09-16T09:42:06.392 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.2029999999999998e-06 2024-09-16T09:42:06.392 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.392 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.392 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:42:06.392 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:38:09.640542+0000", 2024-09-16T09:42:06.392 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.5010000000000001e-06 2024-09-16T09:42:06.392 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.392 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.392 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:42:06.392 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:38:09.917405+0000", 2024-09-16T09:42:06.393 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.27686350799999998 2024-09-16T09:42:06.393 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.393 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.393 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:42:06.393 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:38:09.917429+0000", 2024-09-16T09:42:06.393 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.4017000000000001e-05 2024-09-16T09:42:06.393 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.393 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.393 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:42:06.393 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:38:09.920588+0000", 2024-09-16T09:42:06.393 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0031587820000000002 2024-09-16T09:42:06.393 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.393 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.394 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:42:06.394 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:38:09.920633+0000", 2024-09-16T09:42:06.394 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.4536999999999997e-05 2024-09-16T09:42:06.394 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:42:06.394 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:42:06.394 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:42:06.394 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.394 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.394 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.15975.0:33595 10.f e386/376 10:fdf68c83:::benchmark_data_smithi070_166744_object33594:head v 386'2134, mlcod=386'2132)", 2024-09-16T09:42:06.394 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:38:09.647007+0000", 2024-09-16T09:42:06.394 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 236.737774046, 2024-09-16T09:42:06.394 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.27538453699999998, 2024-09-16T09:42:06.394 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:42:06.394 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:42:06.395 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:42:06.395 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:42:06.395 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.395 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:42:06.395 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:38:09.647007+0000", 2024-09-16T09:42:06.395 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:42:06.395 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.395 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.395 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:42:06.395 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:38:09.647007+0000", 2024-09-16T09:42:06.395 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:42:06.395 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.395 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.395 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:42:06.396 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:38:09.647008+0000", 2024-09-16T09:42:06.396 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.043e-06 2024-09-16T09:42:06.396 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.396 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.396 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:42:06.396 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:38:09.647046+0000", 2024-09-16T09:42:06.396 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7463999999999999e-05 2024-09-16T09:42:06.396 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.396 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.396 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:42:06.396 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:38:09.917354+0000", 2024-09-16T09:42:06.396 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.27030826499999999 2024-09-16T09:42:06.396 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.396 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.397 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:42:06.397 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:38:09.917365+0000", 2024-09-16T09:42:06.397 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1069e-05 2024-09-16T09:42:06.397 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.397 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.397 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:42:06.397 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:38:09.917423+0000", 2024-09-16T09:42:06.397 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8158000000000001e-05 2024-09-16T09:42:06.397 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.397 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.397 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:42:06.398 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:38:09.917437+0000", 2024-09-16T09:42:06.398 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.4046000000000001e-05 2024-09-16T09:42:06.398 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.398 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.398 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:42:06.398 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:38:09.922337+0000", 2024-09-16T09:42:06.398 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0048999910000000002 2024-09-16T09:42:06.398 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.398 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.398 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:42:06.398 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:38:09.922392+0000", 2024-09-16T09:42:06.398 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4500999999999999e-05 2024-09-16T09:42:06.398 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:42:06.398 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:42:06.398 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:42:06.399 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.399 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.399 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15975.0:90178 10.19 10:98316a94:::benchmark_data_smithi070_166744_object90177:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e407)", 2024-09-16T09:42:06.399 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:39:00.047476+0000", 2024-09-16T09:42:06.399 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 186.337304821, 2024-09-16T09:42:06.399 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.29388039399999999, 2024-09-16T09:42:06.399 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:42:06.399 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:42:06.399 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:42:06.399 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:42:06.399 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15975", 2024-09-16T09:42:06.399 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3821727158", 2024-09-16T09:42:06.399 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 90178 2024-09-16T09:42:06.399 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.400 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:42:06.400 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.400 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:42:06.400 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.047476+0000", 2024-09-16T09:42:06.400 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:42:06.400 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.400 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.400 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:42:06.400 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.047476+0000", 2024-09-16T09:42:06.400 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:42:06.400 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.400 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.400 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:42:06.400 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.047478+0000", 2024-09-16T09:42:06.401 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.807e-06 2024-09-16T09:42:06.401 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.401 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.401 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:42:06.401 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.047502+0000", 2024-09-16T09:42:06.401 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3686e-05 2024-09-16T09:42:06.401 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.401 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.401 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:42:06.401 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.047515+0000", 2024-09-16T09:42:06.401 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2624e-05 2024-09-16T09:42:06.401 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.401 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.401 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:42:06.402 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.047522+0000", 2024-09-16T09:42:06.402 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.7419999999999997e-06 2024-09-16T09:42:06.402 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.402 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.402 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:42:06.402 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.047699+0000", 2024-09-16T09:42:06.402 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00017703200000000001 2024-09-16T09:42:06.402 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.402 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.402 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:42:06.402 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.047861+0000", 2024-09-16T09:42:06.402 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00016131300000000001 2024-09-16T09:42:06.402 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.402 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.402 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,3,4", 2024-09-16T09:42:06.403 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.048028+0000", 2024-09-16T09:42:06.403 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00016713399999999999 2024-09-16T09:42:06.403 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.403 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.403 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:42:06.403 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.050803+0000", 2024-09-16T09:42:06.403 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0027749530000000001 2024-09-16T09:42:06.403 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.403 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.403 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:42:06.403 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.052688+0000", 2024-09-16T09:42:06.403 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0018849439999999999 2024-09-16T09:42:06.403 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.403 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.404 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:42:06.404 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.053087+0000", 2024-09-16T09:42:06.404 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00039976999999999998 2024-09-16T09:42:06.404 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.404 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.404 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:42:06.404 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.053137+0000", 2024-09-16T09:42:06.404 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.9425000000000002e-05 2024-09-16T09:42:06.404 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.404 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.404 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:42:06.404 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.053200+0000", 2024-09-16T09:42:06.404 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.3540999999999994e-05 2024-09-16T09:42:06.405 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.405 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.405 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:42:06.405 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.341357+0000", 2024-09-16T09:42:06.405 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.28815642299999999 2024-09-16T09:42:06.405 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:42:06.405 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:42:06.405 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:42:06.405 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.405 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.405 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.15975.0:90184 10.e e407/376 10:797b90f7:::benchmark_data_smithi070_166744_object90183:head v 407'5707, mlcod=407'5705)", 2024-09-16T09:42:06.405 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:39:00.052118+0000", 2024-09-16T09:42:06.405 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 186.332663526, 2024-09-16T09:42:06.405 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.28798647100000002, 2024-09-16T09:42:06.405 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:42:06.406 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:42:06.406 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:42:06.406 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:42:06.406 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.406 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:42:06.406 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.052118+0000", 2024-09-16T09:42:06.406 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:42:06.406 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.406 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.406 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:42:06.406 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.052118+0000", 2024-09-16T09:42:06.406 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:42:06.406 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.406 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.406 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:42:06.407 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.052118+0000", 2024-09-16T09:42:06.407 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.9500000000000002e-07 2024-09-16T09:42:06.407 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.407 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.407 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:42:06.407 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.052166+0000", 2024-09-16T09:42:06.407 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.744e-05 2024-09-16T09:42:06.407 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.407 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.407 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:42:06.407 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.052174+0000", 2024-09-16T09:42:06.407 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.8099999999999998e-06 2024-09-16T09:42:06.407 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.407 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.408 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:42:06.408 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.052179+0000", 2024-09-16T09:42:06.408 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.1549999999999998e-06 2024-09-16T09:42:06.408 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.408 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.408 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:42:06.408 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.052223+0000", 2024-09-16T09:42:06.408 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.4329000000000002e-05 2024-09-16T09:42:06.408 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.408 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.409 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:42:06.409 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.052238+0000", 2024-09-16T09:42:06.409 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.4891e-05 2024-09-16T09:42:06.409 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.409 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.409 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:42:06.409 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.340062+0000", 2024-09-16T09:42:06.409 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.28782371400000001 2024-09-16T09:42:06.409 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.409 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.409 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:42:06.409 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.340104+0000", 2024-09-16T09:42:06.409 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2437e-05 2024-09-16T09:42:06.410 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:42:06.410 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:42:06.410 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:42:06.410 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.410 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.410 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop_reply(client.15975.0:90178 10.19 e407/398 ondisk, result = 0)", 2024-09-16T09:42:06.410 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:39:00.053008+0000", 2024-09-16T09:42:06.410 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 186.331773304, 2024-09-16T09:42:06.410 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.28836234700000002, 2024-09-16T09:42:06.410 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:42:06.411 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:42:06.411 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T09:42:06.411 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:42:06.411 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.411 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:42:06.411 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.053008+0000", 2024-09-16T09:42:06.411 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:42:06.411 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.411 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.411 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:42:06.411 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.053008+0000", 2024-09-16T09:42:06.411 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:42:06.411 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.412 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.412 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:42:06.412 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.053009+0000", 2024-09-16T09:42:06.412 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.3600000000000002e-07 2024-09-16T09:42:06.412 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.412 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.412 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:42:06.412 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.053017+0000", 2024-09-16T09:42:06.412 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.7010000000000002e-06 2024-09-16T09:42:06.412 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.412 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.412 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:42:06.412 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.053032+0000", 2024-09-16T09:42:06.412 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.4387e-05 2024-09-16T09:42:06.413 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.413 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.413 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:42:06.413 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.053038+0000", 2024-09-16T09:42:06.413 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.2129999999999996e-06 2024-09-16T09:42:06.413 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.413 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.413 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:42:06.413 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.053122+0000", 2024-09-16T09:42:06.413 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.3595000000000002e-05 2024-09-16T09:42:06.413 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.414 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.414 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:42:06.414 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.053131+0000", 2024-09-16T09:42:06.414 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.8419999999999994e-06 2024-09-16T09:42:06.414 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.414 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.414 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:42:06.414 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.341370+0000", 2024-09-16T09:42:06.414 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.28823977299999998 2024-09-16T09:42:06.414 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:42:06.415 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:42:06.415 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:42:06.415 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.415 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.415 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15975.0:90189 10.13 10:c9862219:::benchmark_data_smithi070_166744_object90188:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e407)", 2024-09-16T09:42:06.415 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:39:00.061217+0000", 2024-09-16T09:42:06.415 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 186.32356416900001, 2024-09-16T09:42:06.415 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.30579520900000001, 2024-09-16T09:42:06.415 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:42:06.415 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:42:06.415 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:42:06.415 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:42:06.415 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15975", 2024-09-16T09:42:06.415 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3821727158", 2024-09-16T09:42:06.416 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 90189 2024-09-16T09:42:06.416 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.416 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:42:06.416 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.416 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:42:06.416 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.061217+0000", 2024-09-16T09:42:06.416 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:42:06.416 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.416 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.416 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:42:06.416 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.061217+0000", 2024-09-16T09:42:06.416 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:42:06.416 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.417 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.417 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:42:06.417 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.061220+0000", 2024-09-16T09:42:06.417 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.7769999999999998e-06 2024-09-16T09:42:06.417 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.417 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.417 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:42:06.417 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.061248+0000", 2024-09-16T09:42:06.417 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.8285000000000001e-05 2024-09-16T09:42:06.417 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.417 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.417 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:42:06.418 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.340052+0000", 2024-09-16T09:42:06.418 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.27880404600000003 2024-09-16T09:42:06.418 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.418 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.418 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:42:06.418 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.340068+0000", 2024-09-16T09:42:06.418 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5645999999999999e-05 2024-09-16T09:42:06.418 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.418 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.418 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:42:06.419 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.347875+0000", 2024-09-16T09:42:06.419 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0078067780000000003 2024-09-16T09:42:06.419 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.420 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.420 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:42:06.420 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.348050+0000", 2024-09-16T09:42:06.420 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000175391 2024-09-16T09:42:06.420 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.420 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.420 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,7", 2024-09-16T09:42:06.420 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.348242+0000", 2024-09-16T09:42:06.420 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00019203800000000001 2024-09-16T09:42:06.420 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.420 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.420 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:42:06.420 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.354481+0000", 2024-09-16T09:42:06.421 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.006239423 2024-09-16T09:42:06.421 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.421 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.421 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:42:06.421 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.358919+0000", 2024-09-16T09:42:06.421 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0044373679999999997 2024-09-16T09:42:06.421 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.421 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.421 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:42:06.421 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.362108+0000", 2024-09-16T09:42:06.421 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0031896509999999999 2024-09-16T09:42:06.421 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.421 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.422 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:42:06.422 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.362182+0000", 2024-09-16T09:42:06.422 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3209999999999999e-05 2024-09-16T09:42:06.422 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.422 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.422 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:42:06.422 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.367012+0000", 2024-09-16T09:42:06.422 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0048305960000000004 2024-09-16T09:42:06.422 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:42:06.422 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:42:06.422 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:42:06.422 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.422 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.422 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.15975.0:90180 10.8 e407/395 10:139d7062:::benchmark_data_smithi070_166744_object90179:head v 407'5105, mlcod=407'5103)", 2024-09-16T09:42:06.423 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:39:00.064659+0000", 2024-09-16T09:42:06.423 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 186.320121793, 2024-09-16T09:42:06.423 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.28228003800000001, 2024-09-16T09:42:06.423 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:42:06.423 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:42:06.423 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:42:06.423 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:42:06.423 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.423 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:42:06.423 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.064659+0000", 2024-09-16T09:42:06.423 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:42:06.423 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.423 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.423 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:42:06.424 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.064659+0000", 2024-09-16T09:42:06.424 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:42:06.424 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.424 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.424 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:42:06.424 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.064661+0000", 2024-09-16T09:42:06.424 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3400000000000001e-06 2024-09-16T09:42:06.424 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.424 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.424 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:42:06.424 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.064693+0000", 2024-09-16T09:42:06.424 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2611999999999999e-05 2024-09-16T09:42:06.424 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.424 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.424 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:42:06.425 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.340058+0000", 2024-09-16T09:42:06.425 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.27536475999999999 2024-09-16T09:42:06.425 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.425 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.425 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:42:06.425 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.340067+0000", 2024-09-16T09:42:06.425 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.1700000000000003e-06 2024-09-16T09:42:06.425 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.425 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.425 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:42:06.425 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.340332+0000", 2024-09-16T09:42:06.425 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00026422400000000002 2024-09-16T09:42:06.425 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.425 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.426 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:42:06.426 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.340350+0000", 2024-09-16T09:42:06.426 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.8117999999999998e-05 2024-09-16T09:42:06.426 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.426 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.426 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:42:06.426 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.346912+0000", 2024-09-16T09:42:06.426 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0065623679999999998 2024-09-16T09:42:06.426 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.426 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.426 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:42:06.426 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:00.346939+0000", 2024-09-16T09:42:06.426 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.7446000000000001e-05 2024-09-16T09:42:06.426 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:42:06.427 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:42:06.427 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:42:06.427 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.427 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.427 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15975.0:112717 10.18 10:1897c748:::benchmark_data_smithi070_166744_object112716:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e416)", 2024-09-16T09:42:06.427 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:39:23.958012+0000", 2024-09-16T09:42:06.427 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 162.42676909599999, 2024-09-16T09:42:06.427 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.44687069099999999, 2024-09-16T09:42:06.427 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:42:06.427 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:42:06.427 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:42:06.427 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:42:06.427 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15975", 2024-09-16T09:42:06.428 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3821727158", 2024-09-16T09:42:06.428 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 112717 2024-09-16T09:42:06.428 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.428 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:42:06.428 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.428 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:42:06.428 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:23.958012+0000", 2024-09-16T09:42:06.428 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:42:06.428 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.428 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.428 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:42:06.428 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:23.958012+0000", 2024-09-16T09:42:06.428 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:42:06.428 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.429 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.429 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:42:06.429 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:23.958014+0000", 2024-09-16T09:42:06.429 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9259999999999999e-06 2024-09-16T09:42:06.429 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.429 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.429 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:42:06.429 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:23.958041+0000", 2024-09-16T09:42:06.429 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.6678000000000001e-05 2024-09-16T09:42:06.429 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.429 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.429 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:42:06.429 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:23.958058+0000", 2024-09-16T09:42:06.429 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7292999999999999e-05 2024-09-16T09:42:06.430 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.430 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.430 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:42:06.430 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:23.958065+0000", 2024-09-16T09:42:06.430 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.2899999999999997e-06 2024-09-16T09:42:06.430 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.430 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.430 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:42:06.430 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:23.958147+0000", 2024-09-16T09:42:06.430 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.1785000000000004e-05 2024-09-16T09:42:06.430 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.430 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.430 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:42:06.431 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:23.958156+0000", 2024-09-16T09:42:06.431 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.5620000000000004e-06 2024-09-16T09:42:06.431 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.431 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.431 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:42:06.431 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.377617+0000", 2024-09-16T09:42:06.431 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.41946122899999999 2024-09-16T09:42:06.431 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.431 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.431 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:42:06.431 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.377766+0000", 2024-09-16T09:42:06.431 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00014864699999999999 2024-09-16T09:42:06.431 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.431 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.432 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,3", 2024-09-16T09:42:06.432 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.378151+0000", 2024-09-16T09:42:06.432 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000385022 2024-09-16T09:42:06.432 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.432 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.432 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:42:06.432 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.393768+0000", 2024-09-16T09:42:06.432 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.015617396 2024-09-16T09:42:06.432 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.432 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.432 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:42:06.432 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.397799+0000", 2024-09-16T09:42:06.432 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.004031364 2024-09-16T09:42:06.432 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.433 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.433 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:42:06.433 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.404733+0000", 2024-09-16T09:42:06.433 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0069333579999999997 2024-09-16T09:42:06.433 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.433 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.433 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:42:06.433 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.404835+0000", 2024-09-16T09:42:06.433 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00010249799999999999 2024-09-16T09:42:06.433 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.433 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.433 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:42:06.433 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.404883+0000", 2024-09-16T09:42:06.434 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.7642999999999998e-05 2024-09-16T09:42:06.434 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:42:06.434 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:42:06.434 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:42:06.434 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.434 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.434 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15975.0:112754 10.18 10:1d8a67e2:::benchmark_data_smithi070_166744_object112753:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e416)", 2024-09-16T09:42:06.434 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:39:24.000857+0000", 2024-09-16T09:42:06.434 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 162.38392429500001, 2024-09-16T09:42:06.434 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.40415521199999999, 2024-09-16T09:42:06.434 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:42:06.434 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:42:06.434 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:42:06.434 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:42:06.435 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15975", 2024-09-16T09:42:06.435 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3821727158", 2024-09-16T09:42:06.435 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 112754 2024-09-16T09:42:06.435 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.435 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:42:06.435 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.435 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:42:06.435 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.000857+0000", 2024-09-16T09:42:06.435 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:42:06.435 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.435 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.435 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:42:06.435 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.000857+0000", 2024-09-16T09:42:06.435 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:42:06.436 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.436 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.436 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:42:06.436 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.000859+0000", 2024-09-16T09:42:06.436 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.357e-06 2024-09-16T09:42:06.436 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.436 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.436 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:42:06.436 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.000883+0000", 2024-09-16T09:42:06.436 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.4119999999999999e-05 2024-09-16T09:42:06.436 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.436 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.436 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:42:06.436 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.000898+0000", 2024-09-16T09:42:06.437 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.4183e-05 2024-09-16T09:42:06.437 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.437 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.437 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:42:06.437 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.000906+0000", 2024-09-16T09:42:06.437 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.1689999999999992e-06 2024-09-16T09:42:06.437 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.437 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.437 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:42:06.437 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.000983+0000", 2024-09-16T09:42:06.437 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.7476000000000002e-05 2024-09-16T09:42:06.437 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.437 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.438 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:42:06.438 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.000993+0000", 2024-09-16T09:42:06.438 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0028e-05 2024-09-16T09:42:06.438 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.438 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.438 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:42:06.438 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.380231+0000", 2024-09-16T09:42:06.438 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.37923777800000003 2024-09-16T09:42:06.438 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.438 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.438 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:42:06.438 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.380352+0000", 2024-09-16T09:42:06.438 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000120901 2024-09-16T09:42:06.438 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.439 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.439 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,3", 2024-09-16T09:42:06.439 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.380859+0000", 2024-09-16T09:42:06.439 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00050672499999999997 2024-09-16T09:42:06.439 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.439 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.439 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:42:06.439 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.393947+0000", 2024-09-16T09:42:06.439 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.013088123 2024-09-16T09:42:06.439 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.439 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.439 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:42:06.439 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.398580+0000", 2024-09-16T09:42:06.439 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0046332500000000002 2024-09-16T09:42:06.440 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.440 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.440 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:42:06.440 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.404927+0000", 2024-09-16T09:42:06.440 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0063469679999999997 2024-09-16T09:42:06.440 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.440 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.440 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:42:06.440 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.404978+0000", 2024-09-16T09:42:06.440 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.1091000000000003e-05 2024-09-16T09:42:06.440 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.440 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.440 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:42:06.440 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.405012+0000", 2024-09-16T09:42:06.441 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4042999999999999e-05 2024-09-16T09:42:06.441 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:42:06.441 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:42:06.441 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:42:06.441 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.441 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.441 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15975.0:112856 10.18 10:18c0c17d:::benchmark_data_smithi070_166744_object112855:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e416)", 2024-09-16T09:42:06.441 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:39:24.094929+0000", 2024-09-16T09:42:06.441 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 162.28985214900001, 2024-09-16T09:42:06.441 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.31034393399999999, 2024-09-16T09:42:06.441 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:42:06.441 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:42:06.441 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:42:06.441 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:42:06.442 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15975", 2024-09-16T09:42:06.442 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3821727158", 2024-09-16T09:42:06.442 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 112856 2024-09-16T09:42:06.442 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.442 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:42:06.442 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.442 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:42:06.442 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.094929+0000", 2024-09-16T09:42:06.442 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:42:06.442 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.442 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.442 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:42:06.442 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.094929+0000", 2024-09-16T09:42:06.442 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:42:06.442 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.443 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.443 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:42:06.443 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.094931+0000", 2024-09-16T09:42:06.443 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0059999999999999e-06 2024-09-16T09:42:06.443 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.443 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.443 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:42:06.443 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.094955+0000", 2024-09-16T09:42:06.443 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3757e-05 2024-09-16T09:42:06.443 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.443 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.443 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:42:06.443 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.094968+0000", 2024-09-16T09:42:06.443 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.34e-05 2024-09-16T09:42:06.444 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.444 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.444 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:42:06.444 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.094976+0000", 2024-09-16T09:42:06.444 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.7200000000000006e-06 2024-09-16T09:42:06.444 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.444 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.444 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:42:06.444 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.096929+0000", 2024-09-16T09:42:06.444 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0019525949999999999 2024-09-16T09:42:06.444 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.444 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.444 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:42:06.444 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.096939+0000", 2024-09-16T09:42:06.445 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0912e-05 2024-09-16T09:42:06.445 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.445 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.445 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:42:06.445 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.382244+0000", 2024-09-16T09:42:06.445 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.28530434100000002 2024-09-16T09:42:06.445 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.445 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.445 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:42:06.445 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.382366+0000", 2024-09-16T09:42:06.445 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00012229299999999999 2024-09-16T09:42:06.445 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.445 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.445 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,3", 2024-09-16T09:42:06.446 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.382638+0000", 2024-09-16T09:42:06.446 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00027176499999999998 2024-09-16T09:42:06.446 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.446 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.446 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:42:06.446 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.393952+0000", 2024-09-16T09:42:06.446 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.011314204 2024-09-16T09:42:06.446 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.446 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.446 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:42:06.446 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.398630+0000", 2024-09-16T09:42:06.446 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0046776329999999996 2024-09-16T09:42:06.446 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.446 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.447 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:42:06.447 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.405051+0000", 2024-09-16T09:42:06.447 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0064212089999999998 2024-09-16T09:42:06.447 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.447 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.447 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:42:06.447 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.405232+0000", 2024-09-16T09:42:06.447 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00018135100000000001 2024-09-16T09:42:06.447 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.447 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.447 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:42:06.447 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.405273+0000", 2024-09-16T09:42:06.447 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0747999999999997e-05 2024-09-16T09:42:06.447 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:42:06.448 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:42:06.448 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:42:06.448 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.448 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.448 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15975.0:112858 10.18 10:1b69b42e:::benchmark_data_smithi070_166744_object112857:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e416)", 2024-09-16T09:42:06.448 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:39:24.099083+0000", 2024-09-16T09:42:06.448 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 162.285698134, 2024-09-16T09:42:06.448 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.30636600800000002, 2024-09-16T09:42:06.448 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:42:06.448 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:42:06.448 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:42:06.448 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:42:06.448 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15975", 2024-09-16T09:42:06.449 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3821727158", 2024-09-16T09:42:06.449 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 112858 2024-09-16T09:42:06.449 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.449 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:42:06.449 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.449 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:42:06.449 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.099083+0000", 2024-09-16T09:42:06.449 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:42:06.449 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.449 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.449 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:42:06.449 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.099083+0000", 2024-09-16T09:42:06.449 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:42:06.449 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.450 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.450 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:42:06.450 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.099085+0000", 2024-09-16T09:42:06.450 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.702e-06 2024-09-16T09:42:06.450 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.450 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.450 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:42:06.450 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.099109+0000", 2024-09-16T09:42:06.450 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3972e-05 2024-09-16T09:42:06.450 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.450 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.450 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:42:06.450 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.099137+0000", 2024-09-16T09:42:06.450 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.8126000000000001e-05 2024-09-16T09:42:06.450 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.451 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.451 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:42:06.451 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.099145+0000", 2024-09-16T09:42:06.451 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.2640000000000008e-06 2024-09-16T09:42:06.451 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.451 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.451 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:42:06.451 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.099217+0000", 2024-09-16T09:42:06.451 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.1962e-05 2024-09-16T09:42:06.451 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.451 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.451 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:42:06.451 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.099227+0000", 2024-09-16T09:42:06.451 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.8500000000000006e-06 2024-09-16T09:42:06.452 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.452 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.452 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:42:06.452 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.384669+0000", 2024-09-16T09:42:06.452 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.285442003 2024-09-16T09:42:06.452 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.452 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.452 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:42:06.452 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.384840+0000", 2024-09-16T09:42:06.452 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000171151 2024-09-16T09:42:06.452 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.452 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.452 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,3", 2024-09-16T09:42:06.452 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.385178+0000", 2024-09-16T09:42:06.453 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00033784199999999998 2024-09-16T09:42:06.453 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.453 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.453 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:42:06.453 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.393957+0000", 2024-09-16T09:42:06.453 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.008779238 2024-09-16T09:42:06.453 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.453 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.453 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:42:06.453 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.404622+0000", 2024-09-16T09:42:06.453 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.010664585000000001 2024-09-16T09:42:06.453 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.453 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.454 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:42:06.454 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.405316+0000", 2024-09-16T09:42:06.454 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00069413099999999996 2024-09-16T09:42:06.454 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.454 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.454 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:42:06.454 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.405412+0000", 2024-09-16T09:42:06.454 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.6385000000000006e-05 2024-09-16T09:42:06.454 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.454 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.454 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:42:06.454 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.405449+0000", 2024-09-16T09:42:06.454 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6797e-05 2024-09-16T09:42:06.454 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:42:06.454 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:42:06.455 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:42:06.455 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.455 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.455 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15975.0:112872 10.18 10:1de29386:::benchmark_data_smithi070_166744_object112871:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e416)", 2024-09-16T09:42:06.455 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:39:24.111890+0000", 2024-09-16T09:42:06.455 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 162.27289106699999, 2024-09-16T09:42:06.455 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.29379652000000001, 2024-09-16T09:42:06.455 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:42:06.455 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:42:06.455 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:42:06.455 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:42:06.455 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15975", 2024-09-16T09:42:06.455 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3821727158", 2024-09-16T09:42:06.455 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 112872 2024-09-16T09:42:06.456 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.456 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:42:06.456 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.456 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:42:06.456 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.111890+0000", 2024-09-16T09:42:06.456 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:42:06.456 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.456 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.456 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:42:06.456 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.111890+0000", 2024-09-16T09:42:06.456 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:42:06.456 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.456 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.457 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:42:06.457 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.111892+0000", 2024-09-16T09:42:06.457 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.722e-06 2024-09-16T09:42:06.457 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.457 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.457 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:42:06.457 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.111918+0000", 2024-09-16T09:42:06.457 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.6417000000000001e-05 2024-09-16T09:42:06.457 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.457 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.457 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:42:06.457 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.111936+0000", 2024-09-16T09:42:06.457 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7329e-05 2024-09-16T09:42:06.457 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.458 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.458 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:42:06.458 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.111944+0000", 2024-09-16T09:42:06.458 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.1370000000000002e-06 2024-09-16T09:42:06.458 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.458 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.458 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:42:06.458 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.112014+0000", 2024-09-16T09:42:06.458 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.0241999999999996e-05 2024-09-16T09:42:06.458 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.458 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.458 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:42:06.458 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.112024+0000", 2024-09-16T09:42:06.458 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.842e-06 2024-09-16T09:42:06.459 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.459 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.459 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:42:06.459 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.386913+0000", 2024-09-16T09:42:06.459 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.27488880700000001 2024-09-16T09:42:06.459 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.459 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.459 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:42:06.459 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.387044+0000", 2024-09-16T09:42:06.459 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000131211 2024-09-16T09:42:06.459 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.459 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.459 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,3", 2024-09-16T09:42:06.459 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.387392+0000", 2024-09-16T09:42:06.460 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00034859600000000001 2024-09-16T09:42:06.460 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.460 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.460 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:42:06.460 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.393964+0000", 2024-09-16T09:42:06.460 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0065714900000000001 2024-09-16T09:42:06.460 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.460 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.460 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:42:06.461 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:42:06.461 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.404659+0000", 2024-09-16T09:42:06.462 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.010694560000000001 2024-09-16T09:42:06.462 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.462 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.462 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:42:06.462 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.405491+0000", 2024-09-16T09:42:06.462 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00083268700000000003 2024-09-16T09:42:06.462 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.462 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.462 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:42:06.462 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.405649+0000", 2024-09-16T09:42:06.462 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00015801000000000001 2024-09-16T09:42:06.462 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.462 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.463 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:42:06.463 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.405687+0000", 2024-09-16T09:42:06.463 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7469999999999999e-05 2024-09-16T09:42:06.463 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:42:06.463 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:42:06.463 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:42:06.463 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.463 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.463 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15975.0:112907 10.18 10:1deac3cc:::benchmark_data_smithi070_166744_object112906:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e416)", 2024-09-16T09:42:06.463 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:39:24.163480+0000", 2024-09-16T09:42:06.463 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 162.22130079799999, 2024-09-16T09:42:06.463 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.24246647800000001, 2024-09-16T09:42:06.463 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:42:06.464 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:42:06.464 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:42:06.464 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:42:06.464 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15975", 2024-09-16T09:42:06.464 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3821727158", 2024-09-16T09:42:06.464 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 112907 2024-09-16T09:42:06.464 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.464 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:42:06.464 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.464 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:42:06.464 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.163480+0000", 2024-09-16T09:42:06.464 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:42:06.464 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.464 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.465 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:42:06.465 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.163480+0000", 2024-09-16T09:42:06.465 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:42:06.465 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.465 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.465 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:42:06.465 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.163482+0000", 2024-09-16T09:42:06.465 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.796e-06 2024-09-16T09:42:06.465 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.465 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.465 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:42:06.465 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.163506+0000", 2024-09-16T09:42:06.465 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3915000000000001e-05 2024-09-16T09:42:06.466 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.466 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.466 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:42:06.466 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.163524+0000", 2024-09-16T09:42:06.466 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.8213999999999999e-05 2024-09-16T09:42:06.466 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.466 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.466 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:42:06.466 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.163533+0000", 2024-09-16T09:42:06.466 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.7590000000000003e-06 2024-09-16T09:42:06.466 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.466 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.466 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:42:06.467 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.163619+0000", 2024-09-16T09:42:06.467 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.5499000000000003e-05 2024-09-16T09:42:06.467 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.467 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.467 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:42:06.467 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.163626+0000", 2024-09-16T09:42:06.467 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.8220000000000007e-06 2024-09-16T09:42:06.467 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.467 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.467 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:42:06.467 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.390915+0000", 2024-09-16T09:42:06.467 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.22728837399999999 2024-09-16T09:42:06.467 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.467 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.468 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:42:06.468 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.391052+0000", 2024-09-16T09:42:06.468 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000137565 2024-09-16T09:42:06.468 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.468 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.468 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,3", 2024-09-16T09:42:06.468 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.391424+0000", 2024-09-16T09:42:06.468 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00037168900000000003 2024-09-16T09:42:06.468 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.468 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.468 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:42:06.468 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.393968+0000", 2024-09-16T09:42:06.468 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0025443269999999999 2024-09-16T09:42:06.469 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.469 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.469 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:42:06.469 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.404698+0000", 2024-09-16T09:42:06.469 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.010729114999999999 2024-09-16T09:42:06.469 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.469 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.469 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:42:06.469 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.405731+0000", 2024-09-16T09:42:06.469 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001033946 2024-09-16T09:42:06.469 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.469 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.469 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:42:06.469 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.405906+0000", 2024-09-16T09:42:06.470 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00017405900000000001 2024-09-16T09:42:06.470 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.470 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.470 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:42:06.470 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:24.405947+0000", 2024-09-16T09:42:06.470 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.1397999999999999e-05 2024-09-16T09:42:06.470 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:42:06.470 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:42:06.470 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:42:06.470 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.470 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.470 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15975.0:113589 10.19 10:9a90b735:::benchmark_data_smithi070_166744_object113588:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e417)", 2024-09-16T09:42:06.471 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:39:25.237150+0000", 2024-09-16T09:42:06.471 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 161.1476313, 2024-09-16T09:42:06.471 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.267162607, 2024-09-16T09:42:06.471 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:42:06.471 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:42:06.471 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:42:06.471 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:42:06.471 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15975", 2024-09-16T09:42:06.471 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3821727158", 2024-09-16T09:42:06.471 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 113589 2024-09-16T09:42:06.471 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.471 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:42:06.471 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.471 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:42:06.472 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.237150+0000", 2024-09-16T09:42:06.472 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:42:06.472 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.472 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.472 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:42:06.472 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.237150+0000", 2024-09-16T09:42:06.472 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:42:06.472 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.472 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.472 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:42:06.472 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.237152+0000", 2024-09-16T09:42:06.472 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.798e-06 2024-09-16T09:42:06.472 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.472 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.473 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:42:06.473 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.237178+0000", 2024-09-16T09:42:06.473 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.6007e-05 2024-09-16T09:42:06.473 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.473 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.473 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:42:06.473 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.237188+0000", 2024-09-16T09:42:06.473 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0501e-05 2024-09-16T09:42:06.473 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.473 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.473 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:42:06.473 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.237195+0000", 2024-09-16T09:42:06.473 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.5749999999999997e-06 2024-09-16T09:42:06.473 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.474 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.474 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:42:06.474 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.237250+0000", 2024-09-16T09:42:06.474 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5435e-05 2024-09-16T09:42:06.474 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.474 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.474 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:42:06.474 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.237432+0000", 2024-09-16T09:42:06.474 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000182203 2024-09-16T09:42:06.474 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.474 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.474 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,3,4", 2024-09-16T09:42:06.474 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.237847+0000", 2024-09-16T09:42:06.475 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00041432200000000001 2024-09-16T09:42:06.475 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.475 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.475 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:42:06.475 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.247292+0000", 2024-09-16T09:42:06.475 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0094457159999999998 2024-09-16T09:42:06.475 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.475 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.475 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:42:06.475 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.247340+0000", 2024-09-16T09:42:06.475 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.7809000000000003e-05 2024-09-16T09:42:06.475 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.475 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.476 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:42:06.476 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.261859+0000", 2024-09-16T09:42:06.476 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.014518358 2024-09-16T09:42:06.476 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.476 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.476 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:42:06.476 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.502876+0000", 2024-09-16T09:42:06.476 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.241017381 2024-09-16T09:42:06.476 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.476 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.476 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:42:06.476 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.502951+0000", 2024-09-16T09:42:06.476 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.4686999999999998e-05 2024-09-16T09:42:06.476 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.477 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.477 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:42:06.477 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.504313+0000", 2024-09-16T09:42:06.477 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0013618149999999999 2024-09-16T09:42:06.477 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:42:06.477 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:42:06.477 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:42:06.477 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.477 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.477 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15975.0:113610 10.19 10:9d9dd046:::benchmark_data_smithi070_166744_object113609:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e417)", 2024-09-16T09:42:06.477 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:39:25.252441+0000", 2024-09-16T09:42:06.477 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 161.13234006799999, 2024-09-16T09:42:06.477 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.26758622599999998, 2024-09-16T09:42:06.477 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:42:06.478 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:42:06.478 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:42:06.478 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:42:06.478 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15975", 2024-09-16T09:42:06.478 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3821727158", 2024-09-16T09:42:06.478 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 113610 2024-09-16T09:42:06.478 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.478 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:42:06.478 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.478 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:42:06.478 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.252441+0000", 2024-09-16T09:42:06.478 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:42:06.478 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.479 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.479 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:42:06.479 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.252441+0000", 2024-09-16T09:42:06.479 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:42:06.479 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.479 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.479 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:42:06.479 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.252443+0000", 2024-09-16T09:42:06.479 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5850000000000001e-06 2024-09-16T09:42:06.479 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.479 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.479 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:42:06.479 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.252466+0000", 2024-09-16T09:42:06.480 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3336000000000002e-05 2024-09-16T09:42:06.480 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.480 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.480 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:42:06.480 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.252484+0000", 2024-09-16T09:42:06.480 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7442999999999999e-05 2024-09-16T09:42:06.480 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.480 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.480 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:42:06.480 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.252492+0000", 2024-09-16T09:42:06.480 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.259e-06 2024-09-16T09:42:06.480 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.480 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.480 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:42:06.481 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.261907+0000", 2024-09-16T09:42:06.481 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.009415606 2024-09-16T09:42:06.481 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.481 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.481 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:42:06.481 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.262075+0000", 2024-09-16T09:42:06.481 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00016741799999999999 2024-09-16T09:42:06.481 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.482 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.482 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,3,4", 2024-09-16T09:42:06.482 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.262466+0000", 2024-09-16T09:42:06.482 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000390837 2024-09-16T09:42:06.482 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.482 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.482 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:42:06.482 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.266227+0000", 2024-09-16T09:42:06.482 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0037611150000000002 2024-09-16T09:42:06.482 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.482 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.482 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:42:06.482 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.271986+0000", 2024-09-16T09:42:06.483 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0057590690000000003 2024-09-16T09:42:06.483 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.483 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.483 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:42:06.483 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.274510+0000", 2024-09-16T09:42:06.483 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0025244550000000001 2024-09-16T09:42:06.483 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.483 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.483 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:42:06.483 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.518671+0000", 2024-09-16T09:42:06.483 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.24416111099999999 2024-09-16T09:42:06.483 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.483 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.484 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:42:06.484 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.518736+0000", 2024-09-16T09:42:06.484 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.4535999999999993e-05 2024-09-16T09:42:06.484 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.484 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.484 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:42:06.484 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.520027+0000", 2024-09-16T09:42:06.484 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0012914560000000001 2024-09-16T09:42:06.484 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:42:06.484 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:42:06.484 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:42:06.484 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.484 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.484 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15975.0:113611 10.19 10:9fded355:::benchmark_data_smithi070_166744_object113610:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e417)", 2024-09-16T09:42:06.485 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:39:25.255167+0000", 2024-09-16T09:42:06.485 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 161.129613791, 2024-09-16T09:42:06.485 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.26572839500000001, 2024-09-16T09:42:06.485 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:42:06.485 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:42:06.485 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:42:06.485 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:42:06.485 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15975", 2024-09-16T09:42:06.485 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3821727158", 2024-09-16T09:42:06.485 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 113611 2024-09-16T09:42:06.485 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.485 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:42:06.485 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.486 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:42:06.486 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.255167+0000", 2024-09-16T09:42:06.486 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:42:06.486 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.486 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.486 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:42:06.486 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.255167+0000", 2024-09-16T09:42:06.486 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:42:06.486 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.486 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.486 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:42:06.486 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.255169+0000", 2024-09-16T09:42:06.486 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.8419999999999999e-06 2024-09-16T09:42:06.487 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.487 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.487 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:42:06.487 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.255192+0000", 2024-09-16T09:42:06.487 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3156e-05 2024-09-16T09:42:06.487 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.487 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.487 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:42:06.487 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.255205+0000", 2024-09-16T09:42:06.487 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2738000000000001e-05 2024-09-16T09:42:06.487 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.487 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.487 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:42:06.487 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.255214+0000", 2024-09-16T09:42:06.488 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.7730000000000005e-06 2024-09-16T09:42:06.488 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.488 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.488 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:42:06.488 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.264233+0000", 2024-09-16T09:42:06.488 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0090189160000000001 2024-09-16T09:42:06.488 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.488 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.488 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:42:06.488 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.264394+0000", 2024-09-16T09:42:06.488 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00016140600000000001 2024-09-16T09:42:06.488 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.488 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.489 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,3,4", 2024-09-16T09:42:06.489 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.264634+0000", 2024-09-16T09:42:06.489 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00024003499999999999 2024-09-16T09:42:06.489 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.489 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.489 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:42:06.489 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.266235+0000", 2024-09-16T09:42:06.489 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0016005279999999999 2024-09-16T09:42:06.489 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.489 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.489 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:42:06.489 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.272031+0000", 2024-09-16T09:42:06.489 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.005796107 2024-09-16T09:42:06.489 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.490 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.490 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:42:06.490 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.274583+0000", 2024-09-16T09:42:06.490 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0025517199999999999 2024-09-16T09:42:06.490 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.490 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.490 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:42:06.490 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.520093+0000", 2024-09-16T09:42:06.490 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.245510587 2024-09-16T09:42:06.490 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.490 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.490 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:42:06.490 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.520155+0000", 2024-09-16T09:42:06.491 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1271000000000003e-05 2024-09-16T09:42:06.491 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.491 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.491 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:42:06.491 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.520896+0000", 2024-09-16T09:42:06.491 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000741316 2024-09-16T09:42:06.491 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:42:06.491 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:42:06.491 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:42:06.491 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.491 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.491 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15975.0:113645 10.19 10:9c234406:::benchmark_data_smithi070_166744_object113644:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e417)", 2024-09-16T09:42:06.491 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:39:25.285212+0000", 2024-09-16T09:42:06.491 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 161.09956882500001, 2024-09-16T09:42:06.492 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.23697822099999999, 2024-09-16T09:42:06.492 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:42:06.492 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:42:06.492 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:42:06.492 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:42:06.492 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15975", 2024-09-16T09:42:06.492 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3821727158", 2024-09-16T09:42:06.492 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 113645 2024-09-16T09:42:06.492 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.492 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:42:06.492 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.492 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:42:06.492 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.285212+0000", 2024-09-16T09:42:06.493 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:42:06.493 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.493 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.493 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:42:06.493 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.285212+0000", 2024-09-16T09:42:06.493 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:42:06.493 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.493 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.493 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:42:06.493 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.285214+0000", 2024-09-16T09:42:06.493 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9589999999999998e-06 2024-09-16T09:42:06.493 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.494 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.494 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:42:06.494 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.285238+0000", 2024-09-16T09:42:06.494 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3713000000000001e-05 2024-09-16T09:42:06.494 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.494 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.494 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:42:06.494 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.285251+0000", 2024-09-16T09:42:06.494 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3213999999999999e-05 2024-09-16T09:42:06.494 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.494 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.494 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:42:06.494 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.285258+0000", 2024-09-16T09:42:06.495 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.2049999999999996e-06 2024-09-16T09:42:06.495 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.495 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.495 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:42:06.495 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.285317+0000", 2024-09-16T09:42:06.495 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8332999999999999e-05 2024-09-16T09:42:06.495 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.495 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.495 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:42:06.495 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.285472+0000", 2024-09-16T09:42:06.495 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00015523699999999999 2024-09-16T09:42:06.495 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.495 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.495 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,3,4", 2024-09-16T09:42:06.496 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.285876+0000", 2024-09-16T09:42:06.496 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00040423800000000001 2024-09-16T09:42:06.496 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.496 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.496 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:42:06.496 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.287206+0000", 2024-09-16T09:42:06.496 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001329998 2024-09-16T09:42:06.496 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.496 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.496 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:42:06.496 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.292433+0000", 2024-09-16T09:42:06.496 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0052262319999999999 2024-09-16T09:42:06.496 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.497 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.497 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:42:06.497 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.292887+0000", 2024-09-16T09:42:06.497 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00045446800000000003 2024-09-16T09:42:06.497 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.497 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.497 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:42:06.497 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.521236+0000", 2024-09-16T09:42:06.497 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.228348632 2024-09-16T09:42:06.497 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.497 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.497 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:42:06.497 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.521295+0000", 2024-09-16T09:42:06.498 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.9697000000000001e-05 2024-09-16T09:42:06.498 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.498 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.498 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:42:06.498 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.522191+0000", 2024-09-16T09:42:06.498 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00089529500000000005 2024-09-16T09:42:06.498 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:42:06.498 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:42:06.498 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:42:06.498 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.498 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.498 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15975.0:113647 10.19 10:9f9de402:::benchmark_data_smithi070_166744_object113646:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e417)", 2024-09-16T09:42:06.498 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:39:25.287206+0000", 2024-09-16T09:42:06.498 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 161.09757504199999, 2024-09-16T09:42:06.498 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.23593757500000001, 2024-09-16T09:42:06.499 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:42:06.499 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:42:06.499 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:42:06.499 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:42:06.499 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15975", 2024-09-16T09:42:06.499 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3821727158", 2024-09-16T09:42:06.499 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 113647 2024-09-16T09:42:06.499 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.499 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:42:06.499 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.499 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:42:06.499 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.287206+0000", 2024-09-16T09:42:06.499 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:42:06.499 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.500 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.500 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:42:06.500 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.287206+0000", 2024-09-16T09:42:06.500 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:42:06.500 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.500 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.500 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:42:06.500 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.287208+0000", 2024-09-16T09:42:06.500 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.46e-06 2024-09-16T09:42:06.500 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.500 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.500 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:42:06.500 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.287232+0000", 2024-09-16T09:42:06.501 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.4287999999999999e-05 2024-09-16T09:42:06.501 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.501 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.501 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:42:06.501 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.287249+0000", 2024-09-16T09:42:06.501 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7513999999999999e-05 2024-09-16T09:42:06.501 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.501 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.501 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:42:06.501 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.287258+0000", 2024-09-16T09:42:06.501 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.4279999999999996e-06 2024-09-16T09:42:06.501 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.501 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.501 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:42:06.502 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.287321+0000", 2024-09-16T09:42:06.502 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.2707999999999997e-05 2024-09-16T09:42:06.502 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.502 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.502 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:42:06.502 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.287502+0000", 2024-09-16T09:42:06.502 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00018115500000000001 2024-09-16T09:42:06.502 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.502 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.502 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,3,4", 2024-09-16T09:42:06.502 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.287817+0000", 2024-09-16T09:42:06.502 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00031515100000000001 2024-09-16T09:42:06.502 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.503 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.503 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:42:06.503 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.289540+0000", 2024-09-16T09:42:06.503 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001722903 2024-09-16T09:42:06.503 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.503 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.503 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:42:06.503 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.292490+0000", 2024-09-16T09:42:06.503 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0029498250000000001 2024-09-16T09:42:06.503 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.503 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.503 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:42:06.503 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.292939+0000", 2024-09-16T09:42:06.503 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00044983300000000001 2024-09-16T09:42:06.504 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.504 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.504 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:42:06.505 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.522250+0000", 2024-09-16T09:42:06.505 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.229310663 2024-09-16T09:42:06.505 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.505 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.505 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:42:06.505 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.522288+0000", 2024-09-16T09:42:06.505 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8350999999999999e-05 2024-09-16T09:42:06.505 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.505 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.505 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:42:06.506 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:39:25.523144+0000", 2024-09-16T09:42:06.506 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00085529599999999996 2024-09-16T09:42:06.506 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:42:06.506 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:42:06.506 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:42:06.506 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.506 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.506 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15975.0:150410 10.19 10:99b85175:::benchmark_data_smithi070_166744_object36054:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e429)", 2024-09-16T09:42:06.506 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:40:01.355771+0000", 2024-09-16T09:42:06.506 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 125.02901064700001, 2024-09-16T09:42:06.506 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.247207607, 2024-09-16T09:42:06.506 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:42:06.506 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:42:06.507 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:42:06.507 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:42:06.507 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15975", 2024-09-16T09:42:06.507 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3821727158", 2024-09-16T09:42:06.507 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 150410 2024-09-16T09:42:06.507 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.507 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:42:06.507 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.507 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:42:06.507 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.355771+0000", 2024-09-16T09:42:06.507 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:42:06.507 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.507 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.507 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:42:06.508 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.355771+0000", 2024-09-16T09:42:06.508 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:42:06.508 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.508 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.508 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:42:06.508 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.355772+0000", 2024-09-16T09:42:06.508 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.55e-06 2024-09-16T09:42:06.508 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.508 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.508 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:42:06.508 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.355778+0000", 2024-09-16T09:42:06.508 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4190000000000001e-06 2024-09-16T09:42:06.508 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.509 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.509 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:42:06.509 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.355789+0000", 2024-09-16T09:42:06.509 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1119e-05 2024-09-16T09:42:06.509 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.509 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.509 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:42:06.509 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.355796+0000", 2024-09-16T09:42:06.509 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.1330000000000001e-06 2024-09-16T09:42:06.509 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.509 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.509 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:42:06.509 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.356366+0000", 2024-09-16T09:42:06.509 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00057015899999999999 2024-09-16T09:42:06.510 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.510 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.510 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:42:06.510 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.356522+0000", 2024-09-16T09:42:06.510 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000156236 2024-09-16T09:42:06.510 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.510 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.510 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,3,4", 2024-09-16T09:42:06.510 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.356971+0000", 2024-09-16T09:42:06.510 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00044910799999999998 2024-09-16T09:42:06.510 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.510 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.510 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:42:06.511 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.360629+0000", 2024-09-16T09:42:06.511 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.003657361 2024-09-16T09:42:06.511 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.511 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.511 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:42:06.511 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.368284+0000", 2024-09-16T09:42:06.511 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0076550480000000002 2024-09-16T09:42:06.511 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.511 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.511 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:42:06.511 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.368366+0000", 2024-09-16T09:42:06.511 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.1969000000000001e-05 2024-09-16T09:42:06.511 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.511 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.512 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:42:06.512 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.601617+0000", 2024-09-16T09:42:06.512 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.23325089199999999 2024-09-16T09:42:06.512 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.512 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.512 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:42:06.512 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.601729+0000", 2024-09-16T09:42:06.512 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000112235 2024-09-16T09:42:06.512 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.512 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.512 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:42:06.512 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.602978+0000", 2024-09-16T09:42:06.512 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001249378 2024-09-16T09:42:06.513 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:42:06.513 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:42:06.513 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:42:06.513 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.513 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.513 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15975.0:150417 10.19 10:9c31ffcf:::benchmark_data_smithi070_166744_object36061:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e429)", 2024-09-16T09:42:06.513 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:40:01.362746+0000", 2024-09-16T09:42:06.513 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 125.0220352, 2024-09-16T09:42:06.513 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.24787088500000001, 2024-09-16T09:42:06.513 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:42:06.513 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:42:06.513 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:42:06.513 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:42:06.513 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15975", 2024-09-16T09:42:06.514 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3821727158", 2024-09-16T09:42:06.514 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 150417 2024-09-16T09:42:06.514 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.514 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:42:06.514 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.514 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:42:06.514 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.362746+0000", 2024-09-16T09:42:06.514 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:42:06.514 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.514 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.514 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:42:06.514 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.362746+0000", 2024-09-16T09:42:06.514 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:42:06.515 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.515 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.515 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:42:06.515 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.362748+0000", 2024-09-16T09:42:06.515 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6470000000000001e-06 2024-09-16T09:42:06.515 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.515 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.515 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:42:06.515 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.362759+0000", 2024-09-16T09:42:06.515 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0995000000000001e-05 2024-09-16T09:42:06.515 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.515 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.515 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:42:06.515 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.362776+0000", 2024-09-16T09:42:06.516 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7187999999999999e-05 2024-09-16T09:42:06.516 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.516 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.516 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:42:06.516 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.362787+0000", 2024-09-16T09:42:06.516 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0686000000000001e-05 2024-09-16T09:42:06.516 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.516 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.516 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:42:06.516 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.362853+0000", 2024-09-16T09:42:06.516 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.6030000000000003e-05 2024-09-16T09:42:06.516 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.516 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.517 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:42:06.517 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.363067+0000", 2024-09-16T09:42:06.517 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00021482599999999999 2024-09-16T09:42:06.517 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.517 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.517 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,3,4", 2024-09-16T09:42:06.517 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.363553+0000", 2024-09-16T09:42:06.517 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000485315 2024-09-16T09:42:06.517 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.517 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.517 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:42:06.517 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.366668+0000", 2024-09-16T09:42:06.517 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0031155269999999999 2024-09-16T09:42:06.517 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.518 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.518 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:42:06.518 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.368328+0000", 2024-09-16T09:42:06.518 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0016592849999999999 2024-09-16T09:42:06.518 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.518 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.518 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:42:06.518 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.368554+0000", 2024-09-16T09:42:06.518 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00022619900000000001 2024-09-16T09:42:06.518 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.518 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.518 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:42:06.518 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.608909+0000", 2024-09-16T09:42:06.519 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.240355649 2024-09-16T09:42:06.519 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.519 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.519 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:42:06.519 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.609026+0000", 2024-09-16T09:42:06.519 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000116582 2024-09-16T09:42:06.519 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:06.519 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:06.519 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:42:06.519 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:40:01.610617+0000", 2024-09-16T09:42:06.519 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0015909559999999999 2024-09-16T09:42:06.519 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:42:06.519 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:42:06.520 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:42:06.520 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:42:06.520 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:42:06.520 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T09:42:06.521 INFO:teuthology.orchestra.run.smithi070.stderr:noscrub is unset 2024-09-16T09:42:06.535 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:06.536 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:06.536 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:42:06.637 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:42:06.686 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:06.686 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:06.687 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:42:06.788 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:42:06.815 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:06.816 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:06.816 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:42:06.874 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:06 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2938517243' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:42:06.874 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:06 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/2938517243' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:42:06.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:06 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/2938517243' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:42:06.917 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:42:06.965 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:06.966 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:06.966 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:42:07.067 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:42:07.094 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:07.094 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:07.095 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:42:07.096 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 47 16 81271 81255 108.033 77.1875 0.0130878 0.00925175 2024-09-16T09:42:07.196 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:42:07.226 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:07.227 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:07.227 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:42:07.328 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:42:07.363 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:07.364 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:07.364 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:42:07.465 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:42:07.499 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:07.499 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:07.500 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:42:07.600 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:42:07.633 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:07.634 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:07.634 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:42:07.642 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T09:42:07.642 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T09:42:07.642 INFO:tasks.thrashosds.thrasher:inject_pause on osd.7 2024-09-16T09:42:07.642 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-16T09:42:07.642 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-16T09:42:07.642 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.7 config set filestore_inject_stall 3 2024-09-16T09:42:07.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:07 smithi070 ceph-mon[30388]: pgmap v899: 168 pgs: 168 active+clean; 4.7 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 950 B/s rd, 63 MiB/s wr, 1.01k op/s 2024-09-16T09:42:07.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:07 smithi070 ceph-mon[30388]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:42:07.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:07 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2938517243' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-16T09:42:07.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:07 smithi070 ceph-mon[30388]: osdmap e473: 8 total, 8 up, 8 in 2024-09-16T09:42:07.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:07 smithi070 ceph-mon[30388]: 4.d scrub starts 2024-09-16T09:42:07.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:07 smithi070 ceph-mon[30388]: 4.d scrub ok 2024-09-16T09:42:07.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:07 smithi070 ceph-mon[40691]: pgmap v899: 168 pgs: 168 active+clean; 4.7 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 950 B/s rd, 63 MiB/s wr, 1.01k op/s 2024-09-16T09:42:07.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:07 smithi070 ceph-mon[40691]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:42:07.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:07 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/2938517243' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-16T09:42:07.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:07 smithi070 ceph-mon[40691]: osdmap e473: 8 total, 8 up, 8 in 2024-09-16T09:42:07.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:07 smithi070 ceph-mon[40691]: 4.d scrub starts 2024-09-16T09:42:07.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:07 smithi070 ceph-mon[40691]: 4.d scrub ok 2024-09-16T09:42:07.735 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:42:07.785 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:07.785 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:07.785 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:42:07.886 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:42:07.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:07 smithi084 ceph-mon[32629]: pgmap v899: 168 pgs: 168 active+clean; 4.7 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 950 B/s rd, 63 MiB/s wr, 1.01k op/s 2024-09-16T09:42:07.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:07 smithi084 ceph-mon[32629]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:42:07.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:07 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/2938517243' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-16T09:42:07.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:07 smithi084 ceph-mon[32629]: osdmap e473: 8 total, 8 up, 8 in 2024-09-16T09:42:07.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:07 smithi084 ceph-mon[32629]: 4.d scrub starts 2024-09-16T09:42:07.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:07 smithi084 ceph-mon[32629]: 4.d scrub ok 2024-09-16T09:42:07.925 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:07.925 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:07.926 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:42:08.027 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:42:08.069 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:08.069 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:08.070 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:42:08.086 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:42:08.096 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 48 16 82606 82590 107.521 83.4375 0.0199967 0.00929558 2024-09-16T09:42:08.171 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:42:08.212 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:08.212 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:08.213 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:42:08.313 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:42:08.347 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:08.348 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:08.349 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:42:08.449 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:42:08.485 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:08.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:08.486 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:42:08.554 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.3 dump_ops_in_flight 2024-09-16T09:42:08.588 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:42:08.592 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:42:08.621 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:08.621 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:08.622 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:42:08.701 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-09-16T09:42:08.701 INFO:teuthology.orchestra.run.smithi084.stdout: "success": "" 2024-09-16T09:42:08.701 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-09-16T09:42:08.723 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:42:08.765 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:08.766 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:08.766 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:42:08.777 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:08 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:42:08.778 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:08 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:42:08.778 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:08 smithi070 ceph-mon[30388]: osdmap e474: 8 total, 8 up, 8 in 2024-09-16T09:42:08.778 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:08 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:42:08.778 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:08 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:42:08.778 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:08 smithi070 ceph-mon[40691]: osdmap e474: 8 total, 8 up, 8 in 2024-09-16T09:42:08.867 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:42:08.899 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:08.900 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:08.901 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:42:08.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:08 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:42:08.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:08 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:42:08.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:08 smithi084 ceph-mon[32629]: osdmap e474: 8 total, 8 up, 8 in 2024-09-16T09:42:09.001 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:42:09.047 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:09.047 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:09.047 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:42:09.097 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 49 16 83877 83861 106.948 79.4375 0.0132081 0.00934572 2024-09-16T09:42:09.148 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:42:09.180 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:09.181 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:09.181 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:42:09.282 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:42:09.298 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:42:09.309 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:42:09.309 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:42:09.330 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:09.331 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:09.332 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:42:09.344 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:42:09.344 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:42:09.376 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:42:09.377 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:42:09.411 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:42:09.411 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:42:09.433 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:42:09.442 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:42:09.442 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:42:09.472 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:42:09.473 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:42:09.474 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:09.474 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:09.474 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:42:09.506 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:42:09.507 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:42:09.544 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:42:09.545 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:42:09.575 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:42:09.604 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:09.605 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:09.606 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:42:09.628 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:09 smithi070 ceph-mon[30388]: 6.1b scrub starts 2024-09-16T09:42:09.628 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:09 smithi070 ceph-mon[30388]: 6.1b scrub ok 2024-09-16T09:42:09.628 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:09 smithi070 ceph-mon[30388]: pgmap v902: 168 pgs: 168 active+clean; 4.8 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.9 KiB/s rd, 82 MiB/s wr, 1.32k op/s 2024-09-16T09:42:09.629 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:09 smithi070 ceph-mon[30388]: 5.17 scrub starts 2024-09-16T09:42:09.629 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:09 smithi070 ceph-mon[30388]: 5.17 scrub ok 2024-09-16T09:42:09.629 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:09 smithi070 ceph-mon[30388]: 5.1a scrub starts 2024-09-16T09:42:09.629 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:09 smithi070 ceph-mon[30388]: 5.1a scrub ok 2024-09-16T09:42:09.629 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:09 smithi070 ceph-mon[30388]: 5.1 scrub starts 2024-09-16T09:42:09.629 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:09 smithi070 ceph-mon[30388]: 5.1 scrub ok 2024-09-16T09:42:09.629 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:09 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/4195598801' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:42:09.629 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:09 smithi070 ceph-mon[40691]: 6.1b scrub starts 2024-09-16T09:42:09.629 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:09 smithi070 ceph-mon[40691]: 6.1b scrub ok 2024-09-16T09:42:09.629 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:09 smithi070 ceph-mon[40691]: pgmap v902: 168 pgs: 168 active+clean; 4.8 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.9 KiB/s rd, 82 MiB/s wr, 1.32k op/s 2024-09-16T09:42:09.629 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:09 smithi070 ceph-mon[40691]: 5.17 scrub starts 2024-09-16T09:42:09.629 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:09 smithi070 ceph-mon[40691]: 5.17 scrub ok 2024-09-16T09:42:09.630 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:09 smithi070 ceph-mon[40691]: 5.1a scrub starts 2024-09-16T09:42:09.630 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:09 smithi070 ceph-mon[40691]: 5.1a scrub ok 2024-09-16T09:42:09.630 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:09 smithi070 ceph-mon[40691]: 5.1 scrub starts 2024-09-16T09:42:09.630 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:09 smithi070 ceph-mon[40691]: 5.1 scrub ok 2024-09-16T09:42:09.630 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:09 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/4195598801' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:42:09.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:09 smithi084 ceph-mon[32629]: 6.1b scrub starts 2024-09-16T09:42:09.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:09 smithi084 ceph-mon[32629]: 6.1b scrub ok 2024-09-16T09:42:09.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:09 smithi084 ceph-mon[32629]: pgmap v902: 168 pgs: 168 active+clean; 4.8 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.9 KiB/s rd, 82 MiB/s wr, 1.32k op/s 2024-09-16T09:42:09.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:09 smithi084 ceph-mon[32629]: 5.17 scrub starts 2024-09-16T09:42:09.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:09 smithi084 ceph-mon[32629]: 5.17 scrub ok 2024-09-16T09:42:09.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:09 smithi084 ceph-mon[32629]: 5.1a scrub starts 2024-09-16T09:42:09.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:09 smithi084 ceph-mon[32629]: 5.1a scrub ok 2024-09-16T09:42:09.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:09 smithi084 ceph-mon[32629]: 5.1 scrub starts 2024-09-16T09:42:09.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:09 smithi084 ceph-mon[32629]: 5.1 scrub ok 2024-09-16T09:42:09.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:09 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/4195598801' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:42:09.706 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:42:09.741 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:09.742 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:09.742 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:42:09.839 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-16T09:42:09.844 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:42:09.925 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:09.925 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:09.926 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:42:10.027 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:42:10.071 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:10.071 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:10.072 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:42:10.097 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 50 16 85056 85040 106.282 73.6875 0.00921983 0.00940433 2024-09-16T09:42:10.173 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:42:10.204 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:10.204 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:10.204 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:42:10.305 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:42:10.341 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:10.341 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:10.342 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:42:10.443 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:42:10.472 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:10.472 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:10.473 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:42:10.573 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:42:10.606 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:10.607 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:10.607 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:42:10.709 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:42:10.749 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:10.750 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:10.750 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:42:10.851 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:42:10.885 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:10.886 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:10.886 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:42:10.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:10 smithi084 ceph-mon[32629]: 6.c scrub starts 2024-09-16T09:42:10.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:10 smithi084 ceph-mon[32629]: 6.c scrub ok 2024-09-16T09:42:10.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:10 smithi084 ceph-mon[32629]: 5.18 scrub starts 2024-09-16T09:42:10.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:10 smithi084 ceph-mon[32629]: 5.18 scrub ok 2024-09-16T09:42:10.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:10 smithi070 ceph-mon[30388]: 6.c scrub starts 2024-09-16T09:42:10.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:10 smithi070 ceph-mon[30388]: 6.c scrub ok 2024-09-16T09:42:10.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:10 smithi070 ceph-mon[30388]: 5.18 scrub starts 2024-09-16T09:42:10.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:10 smithi070 ceph-mon[30388]: 5.18 scrub ok 2024-09-16T09:42:10.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:10 smithi070 ceph-mon[40691]: 6.c scrub starts 2024-09-16T09:42:10.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:10 smithi070 ceph-mon[40691]: 6.c scrub ok 2024-09-16T09:42:10.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:10 smithi070 ceph-mon[40691]: 5.18 scrub starts 2024-09-16T09:42:10.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:10 smithi070 ceph-mon[40691]: 5.18 scrub ok 2024-09-16T09:42:10.988 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:42:11.024 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:11.025 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:11.025 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:42:11.097 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 51 16 86406 86390 105.853 84.375 0.00888301 0.00944209 2024-09-16T09:42:11.127 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:42:11.164 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:11.165 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:11.165 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:42:11.266 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:42:11.299 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:11.300 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:11.300 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:42:11.401 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:42:11.436 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:11.436 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:11.437 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:42:11.538 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:42:11.574 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:11.574 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:11.575 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:42:11.675 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:42:11.715 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:11.716 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:11.716 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:42:11.747 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T09:42:11.747 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:42:11.817 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:42:11.844 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:11.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:11.845 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:42:11.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:11 smithi084 ceph-mon[32629]: 1.c scrub starts 2024-09-16T09:42:11.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:11 smithi084 ceph-mon[32629]: 1.c scrub ok 2024-09-16T09:42:11.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:11 smithi084 ceph-mon[32629]: pgmap v903: 168 pgs: 168 active+clean; 5.0 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 852 B/s rd, 70 MiB/s wr, 1.12k op/s 2024-09-16T09:42:11.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:11 smithi084 ceph-mon[32629]: 1.2 scrub starts 2024-09-16T09:42:11.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:11 smithi084 ceph-mon[32629]: 1.2 scrub ok 2024-09-16T09:42:11.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:11 smithi084 ceph-mon[32629]: 5.5 scrub starts 2024-09-16T09:42:11.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:11 smithi084 ceph-mon[32629]: 5.5 scrub ok 2024-09-16T09:42:11.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:11 smithi084 ceph-mon[32629]: 5.1f scrub starts 2024-09-16T09:42:11.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:11 smithi084 ceph-mon[32629]: 5.1f scrub ok 2024-09-16T09:42:11.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:11 smithi070 ceph-mon[30388]: 1.c scrub starts 2024-09-16T09:42:11.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:11 smithi070 ceph-mon[30388]: 1.c scrub ok 2024-09-16T09:42:11.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:11 smithi070 ceph-mon[30388]: pgmap v903: 168 pgs: 168 active+clean; 5.0 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 852 B/s rd, 70 MiB/s wr, 1.12k op/s 2024-09-16T09:42:11.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:11 smithi070 ceph-mon[30388]: 1.2 scrub starts 2024-09-16T09:42:11.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:11 smithi070 ceph-mon[30388]: 1.2 scrub ok 2024-09-16T09:42:11.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:11 smithi070 ceph-mon[30388]: 5.5 scrub starts 2024-09-16T09:42:11.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:11 smithi070 ceph-mon[30388]: 5.5 scrub ok 2024-09-16T09:42:11.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:11 smithi070 ceph-mon[30388]: 5.1f scrub starts 2024-09-16T09:42:11.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:11 smithi070 ceph-mon[30388]: 5.1f scrub ok 2024-09-16T09:42:11.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:11 smithi070 ceph-mon[40691]: 1.c scrub starts 2024-09-16T09:42:11.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:11 smithi070 ceph-mon[40691]: 1.c scrub ok 2024-09-16T09:42:11.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:11 smithi070 ceph-mon[40691]: pgmap v903: 168 pgs: 168 active+clean; 5.0 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 852 B/s rd, 70 MiB/s wr, 1.12k op/s 2024-09-16T09:42:11.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:11 smithi070 ceph-mon[40691]: 1.2 scrub starts 2024-09-16T09:42:11.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:11 smithi070 ceph-mon[40691]: 1.2 scrub ok 2024-09-16T09:42:11.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:11 smithi070 ceph-mon[40691]: 5.5 scrub starts 2024-09-16T09:42:11.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:11 smithi070 ceph-mon[40691]: 5.5 scrub ok 2024-09-16T09:42:11.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:11 smithi070 ceph-mon[40691]: 5.1f scrub starts 2024-09-16T09:42:11.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:11 smithi070 ceph-mon[40691]: 5.1f scrub ok 2024-09-16T09:42:11.946 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:42:11.979 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:11.979 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:11.980 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:42:12.080 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:42:12.097 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 52 16 87612 87596 105.266 75.375 0.0209626 0.00949516 2024-09-16T09:42:12.116 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:12.117 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:12.118 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:42:12.218 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:42:12.254 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:12.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:12.255 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:42:12.355 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:42:12.394 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:12.395 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:12.395 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:42:12.420 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:42:11 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:42:11] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:42:12.496 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:42:12.534 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:12.534 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:12.535 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:42:12.636 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:42:12.657 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T09:42:12.673 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:12.674 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:12.674 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:42:12.775 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:42:12.810 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:12.811 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:12.811 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:42:12.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:12 smithi084 ceph-mon[32629]: 5.4 scrub starts 2024-09-16T09:42:12.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:12 smithi084 ceph-mon[32629]: 5.4 scrub ok 2024-09-16T09:42:12.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:12 smithi084 ceph-mon[32629]: 6.4 scrub starts 2024-09-16T09:42:12.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:12 smithi084 ceph-mon[32629]: 6.4 scrub ok 2024-09-16T09:42:12.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:12 smithi084 ceph-mon[32629]: 4.6 scrub starts 2024-09-16T09:42:12.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:12 smithi084 ceph-mon[32629]: 4.6 scrub ok 2024-09-16T09:42:12.912 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:42:12.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:12 smithi070 ceph-mon[30388]: 5.4 scrub starts 2024-09-16T09:42:12.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:12 smithi070 ceph-mon[30388]: 5.4 scrub ok 2024-09-16T09:42:12.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:12 smithi070 ceph-mon[30388]: 6.4 scrub starts 2024-09-16T09:42:12.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:12 smithi070 ceph-mon[30388]: 6.4 scrub ok 2024-09-16T09:42:12.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:12 smithi070 ceph-mon[30388]: 4.6 scrub starts 2024-09-16T09:42:12.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:12 smithi070 ceph-mon[30388]: 4.6 scrub ok 2024-09-16T09:42:12.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:12 smithi070 ceph-mon[40691]: 5.4 scrub starts 2024-09-16T09:42:12.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:12 smithi070 ceph-mon[40691]: 5.4 scrub ok 2024-09-16T09:42:12.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:12 smithi070 ceph-mon[40691]: 6.4 scrub starts 2024-09-16T09:42:12.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:12 smithi070 ceph-mon[40691]: 6.4 scrub ok 2024-09-16T09:42:12.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:12 smithi070 ceph-mon[40691]: 4.6 scrub starts 2024-09-16T09:42:12.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:12 smithi070 ceph-mon[40691]: 4.6 scrub ok 2024-09-16T09:42:12.956 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:12.956 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:12.956 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:42:13.057 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:42:13.096 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:13.096 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:13.097 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:42:13.097 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 53 16 88888 88872 104.785 79.75 0.0090833 0.00953908 2024-09-16T09:42:13.198 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:42:13.234 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:13.234 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:13.235 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:42:13.335 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:42:13.368 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:13.368 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:13.369 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:42:13.469 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:42:13.501 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:13.501 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:13.502 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:42:13.603 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:42:13.641 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:13.642 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:13.642 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:42:13.743 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:42:13.764 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:13.765 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:13.765 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:42:13.866 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:42:13.887 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:13.888 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:13.888 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:42:13.910 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:42:13 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:42:13.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:13 smithi084 ceph-mon[32629]: pgmap v904: 168 pgs: 168 active+clean; 5.0 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 689 B/s rd, 57 MiB/s wr, 905 op/s 2024-09-16T09:42:13.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:13 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:42:13.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:13 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:42:13.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:13 smithi084 ceph-mon[32629]: 11.1 scrub starts 2024-09-16T09:42:13.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:13 smithi084 ceph-mon[32629]: 5.8 scrub starts 2024-09-16T09:42:13.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:13 smithi084 ceph-mon[32629]: 5.8 scrub ok 2024-09-16T09:42:13.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:13 smithi070 ceph-mon[30388]: pgmap v904: 168 pgs: 168 active+clean; 5.0 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 689 B/s rd, 57 MiB/s wr, 905 op/s 2024-09-16T09:42:13.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:13 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:42:13.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:13 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:42:13.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:13 smithi070 ceph-mon[30388]: 11.1 scrub starts 2024-09-16T09:42:13.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:13 smithi070 ceph-mon[30388]: 5.8 scrub starts 2024-09-16T09:42:13.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:13 smithi070 ceph-mon[30388]: 5.8 scrub ok 2024-09-16T09:42:13.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:13 smithi070 ceph-mon[40691]: pgmap v904: 168 pgs: 168 active+clean; 5.0 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 689 B/s rd, 57 MiB/s wr, 905 op/s 2024-09-16T09:42:13.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:13 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:42:13.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:13 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:42:13.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:13 smithi070 ceph-mon[40691]: 11.1 scrub starts 2024-09-16T09:42:13.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:13 smithi070 ceph-mon[40691]: 5.8 scrub starts 2024-09-16T09:42:13.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:13 smithi070 ceph-mon[40691]: 5.8 scrub ok 2024-09-16T09:42:13.989 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:42:14.017 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:14.018 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:14.018 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:42:14.097 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 54 16 90040 90024 104.178 72 0.0148665 0.00959454 2024-09-16T09:42:14.119 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:42:14.150 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:14.151 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:14.151 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:42:14.252 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:42:14.291 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:14.291 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:14.291 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:42:14.392 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:42:14.426 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:14.426 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:14.427 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:42:14.458 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T09:42:14.458 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T09:42:14.458 INFO:tasks.thrashosds.thrasher:fixing pg num pool unique_pool_4 2024-09-16T09:42:14.459 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-16T09:42:14.528 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:42:14.568 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:14.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:14.569 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:42:14.670 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:42:14.702 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:14.703 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:14.703 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:42:14.804 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:42:14.843 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:14.844 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:14.844 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:42:14.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:14 smithi084 ceph-mon[32629]: osdmap e475: 8 total, 8 up, 8 in 2024-09-16T09:42:14.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:14 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:42:14.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:14 smithi084 ceph-mon[32629]: osdmap e476: 8 total, 8 up, 8 in 2024-09-16T09:42:14.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:14 smithi070 ceph-mon[30388]: osdmap e475: 8 total, 8 up, 8 in 2024-09-16T09:42:14.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:14 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:42:14.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:14 smithi070 ceph-mon[30388]: osdmap e476: 8 total, 8 up, 8 in 2024-09-16T09:42:14.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:14 smithi070 ceph-mon[40691]: osdmap e475: 8 total, 8 up, 8 in 2024-09-16T09:42:14.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:14 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:42:14.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:14 smithi070 ceph-mon[40691]: osdmap e476: 8 total, 8 up, 8 in 2024-09-16T09:42:14.945 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:42:14.981 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:14.982 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:14.983 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:42:15.083 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:42:15.097 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 55 15 91222 91207 103.628 73.9375 0.00443961 0.0096447 2024-09-16T09:42:15.115 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:15.116 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:15.116 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:42:15.168 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.3 dump_blocked_ops 2024-09-16T09:42:15.217 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:42:15.246 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:42:15.266 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:42:15.297 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:15.298 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:15.298 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:42:15.399 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:42:15.434 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:15.435 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:15.435 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:42:15.536 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:42:15.587 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:15.588 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:15.588 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:42:15.641 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:15 smithi070 ceph-mon[30388]: pgmap v906: 168 pgs: 1 active+clean+scrubbing, 167 active+clean; 5.4 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.4 KiB/s rd, 104 MiB/s wr, 1.67k op/s 2024-09-16T09:42:15.641 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:15 smithi070 ceph-mon[30388]: 11.1 scrub ok 2024-09-16T09:42:15.641 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:15 smithi070 ceph-mon[30388]: 5.3 scrub starts 2024-09-16T09:42:15.641 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:15 smithi070 ceph-mon[30388]: 5.3 scrub ok 2024-09-16T09:42:15.642 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:15 smithi070 ceph-mon[40691]: pgmap v906: 168 pgs: 1 active+clean+scrubbing, 167 active+clean; 5.4 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.4 KiB/s rd, 104 MiB/s wr, 1.67k op/s 2024-09-16T09:42:15.642 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:15 smithi070 ceph-mon[40691]: 11.1 scrub ok 2024-09-16T09:42:15.642 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:15 smithi070 ceph-mon[40691]: 5.3 scrub starts 2024-09-16T09:42:15.642 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:15 smithi070 ceph-mon[40691]: 5.3 scrub ok 2024-09-16T09:42:15.689 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:42:15.723 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:15.724 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:15.724 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:42:15.825 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:42:15.854 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:15.855 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:15.855 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:42:15.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:15 smithi084 ceph-mon[32629]: pgmap v906: 168 pgs: 1 active+clean+scrubbing, 167 active+clean; 5.4 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.4 KiB/s rd, 104 MiB/s wr, 1.67k op/s 2024-09-16T09:42:15.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:15 smithi084 ceph-mon[32629]: 11.1 scrub ok 2024-09-16T09:42:15.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:15 smithi084 ceph-mon[32629]: 5.3 scrub starts 2024-09-16T09:42:15.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:15 smithi084 ceph-mon[32629]: 5.3 scrub ok 2024-09-16T09:42:15.956 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:42:15.987 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:15.987 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:15.987 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:42:16.088 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:42:16.097 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 56 16 92361 92345 103.047 71.125 0.0142094 0.00969854 2024-09-16T09:42:16.117 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:16.117 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:16.118 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:42:16.219 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:42:16.262 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:16.262 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:16.263 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:42:16.364 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:42:16.373 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:42:16.373 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:42:16.373 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:42:16.373 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:42:16.373 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:42:16.374 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:42:16.374 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:42:16.374 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:42:16.374 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:42:16.374 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:42:16.374 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:42:16.374 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:42:16.374 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:42:16.374 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:42:16.404 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:16.405 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:16.405 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:42:16.415 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:42:16.415 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:42:16.422 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:42:16.506 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:42:16.554 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:16.555 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:16.555 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:42:16.633 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:42:16.636 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2024-09-16T09:42:16.656 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:42:16.686 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:16.687 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:16.687 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:42:16.788 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:42:16.822 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:16.823 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:16.824 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:42:16.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:16 smithi084 ceph-mon[32629]: 6.8 scrub starts 2024-09-16T09:42:16.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:16 smithi084 ceph-mon[32629]: 6.8 scrub ok 2024-09-16T09:42:16.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:16 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/4059755600' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:42:16.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:16 smithi084 ceph-mon[32629]: 1.e scrub starts 2024-09-16T09:42:16.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:16 smithi084 ceph-mon[32629]: 1.e scrub ok 2024-09-16T09:42:16.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:16 smithi070 ceph-mon[30388]: 6.8 scrub starts 2024-09-16T09:42:16.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:16 smithi070 ceph-mon[30388]: 6.8 scrub ok 2024-09-16T09:42:16.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:16 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/4059755600' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:42:16.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:16 smithi070 ceph-mon[30388]: 1.e scrub starts 2024-09-16T09:42:16.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:16 smithi070 ceph-mon[30388]: 1.e scrub ok 2024-09-16T09:42:16.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:16 smithi070 ceph-mon[40691]: 6.8 scrub starts 2024-09-16T09:42:16.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:16 smithi070 ceph-mon[40691]: 6.8 scrub ok 2024-09-16T09:42:16.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:16 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/4059755600' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:42:16.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:16 smithi070 ceph-mon[40691]: 1.e scrub starts 2024-09-16T09:42:16.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:16 smithi070 ceph-mon[40691]: 1.e scrub ok 2024-09-16T09:42:16.925 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:42:16.972 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:16.973 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:16.973 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:42:17.074 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:42:17.097 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 57 16 93481 93465 102.467 70 0.0108038 0.00975445 2024-09-16T09:42:17.110 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:17.110 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:17.111 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:42:17.211 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:42:17.252 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:17.253 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:17.253 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:42:17.354 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:42:17.388 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:17.389 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:17.389 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:42:17.490 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:42:17.529 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:17.530 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:17.530 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:42:17.631 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:42:17.664 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:17.664 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:17.665 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:42:17.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:17 smithi070 ceph-mon[30388]: 11.9 scrub starts 2024-09-16T09:42:17.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:17 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:42:17.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:17 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:42:17.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:17 smithi070 ceph-mon[30388]: pgmap v908: 168 pgs: 1 active+clean+scrubbing, 167 active+clean; 5.4 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 639 B/s rd, 86 MiB/s wr, 1.38k op/s 2024-09-16T09:42:17.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:17 smithi070 ceph-mon[30388]: osdmap e477: 8 total, 8 up, 8 in 2024-09-16T09:42:17.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:17 smithi070 ceph-mon[30388]: 1.12 scrub starts 2024-09-16T09:42:17.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:17 smithi070 ceph-mon[30388]: 1.12 scrub ok 2024-09-16T09:42:17.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:17 smithi070 ceph-mon[30388]: 1.11 scrub starts 2024-09-16T09:42:17.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:17 smithi070 ceph-mon[30388]: 1.11 scrub ok 2024-09-16T09:42:17.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:17 smithi070 ceph-mon[40691]: 11.9 scrub starts 2024-09-16T09:42:17.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:17 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:42:17.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:17 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:42:17.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:17 smithi070 ceph-mon[40691]: pgmap v908: 168 pgs: 1 active+clean+scrubbing, 167 active+clean; 5.4 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 639 B/s rd, 86 MiB/s wr, 1.38k op/s 2024-09-16T09:42:17.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:17 smithi070 ceph-mon[40691]: osdmap e477: 8 total, 8 up, 8 in 2024-09-16T09:42:17.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:17 smithi070 ceph-mon[40691]: 1.12 scrub starts 2024-09-16T09:42:17.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:17 smithi070 ceph-mon[40691]: 1.12 scrub ok 2024-09-16T09:42:17.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:17 smithi070 ceph-mon[40691]: 1.11 scrub starts 2024-09-16T09:42:17.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:17 smithi070 ceph-mon[40691]: 1.11 scrub ok 2024-09-16T09:42:17.766 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:42:17.800 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:17.801 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:17.801 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:42:17.902 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:42:17.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:17 smithi084 ceph-mon[32629]: 11.9 scrub starts 2024-09-16T09:42:17.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:17 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:42:17.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:17 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:42:17.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:17 smithi084 ceph-mon[32629]: pgmap v908: 168 pgs: 1 active+clean+scrubbing, 167 active+clean; 5.4 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 639 B/s rd, 86 MiB/s wr, 1.38k op/s 2024-09-16T09:42:17.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:17 smithi084 ceph-mon[32629]: osdmap e477: 8 total, 8 up, 8 in 2024-09-16T09:42:17.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:17 smithi084 ceph-mon[32629]: 1.12 scrub starts 2024-09-16T09:42:17.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:17 smithi084 ceph-mon[32629]: 1.12 scrub ok 2024-09-16T09:42:17.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:17 smithi084 ceph-mon[32629]: 1.11 scrub starts 2024-09-16T09:42:17.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:17 smithi084 ceph-mon[32629]: 1.11 scrub ok 2024-09-16T09:42:17.924 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:17.924 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:17.925 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:42:18.025 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:42:18.057 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:18.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:18.058 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:42:18.098 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 58 16 94628 94612 101.937 71.6875 0.0131493 0.00980552 2024-09-16T09:42:18.158 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:42:18.188 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:18.189 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:18.189 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:42:18.290 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:42:18.327 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:18.327 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:18.328 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:42:18.398 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:42:18.429 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:42:18.457 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:18.458 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:18.458 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:42:18.546 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":909,"stamp":"2024-09-16T09:42:16.594542+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":5832317467,"num_objects":89184,"num_object_clones":0,"num_object_copies":267552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89184,"num_whiteouts":0,"num_read":2165,"num_read_kb":1917,"num_write":89707,"num_write_kb":5695085,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":150,"num_bytes_recovered":333352,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":77313,"ondisk_log_size":77313,"up":504,"acting":504,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":504,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":23151868,"kb_used_data":17075700,"kb_used_omap":12,"kb_used_meta":6076019,"kb_avail":726809348,"statfs":{"total":767960285184,"available":744252772352,"internally_reserved":0,"allocated":17485516800,"data_stored":17481410117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":6221844018},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":412549120,"num_objects":6295,"num_object_clones":0,"num_object_copies":18885,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6295,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":6295,"num_write_kb":402880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.735817"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":901,"reported_epoch":476,"state":"active+clean","last_fresh":"2024-09-16T09:42:14.443620+0000","last_change":"2024-09-16T09:41:40.073159+0000","last_active":"2024-09-16T09:42:14.443620+0000","last_peered":"2024-09-16T09:42:14.443620+0000","last_clean":"2024-09-16T09:42:14.443620+0000","last_became_active":"2024-09-16T09:25:01.709108+0000","last_became_peered":"2024-09-16T09:25:01.709108+0000","last_unstale":"2024-09-16T09:42:14.443620+0000","last_undegraded":"2024-09-16T09:42:14.443620+0000","last_fullsized":"2024-09-16T09:42:14.443620+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:40.073089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:41:40.073089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:58.125354+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"285'15","reported_seq":909,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:07.526366+0000","last_change":"2024-09-16T09:41:43.064790+0000","last_active":"2024-09-16T09:42:07.526366+0000","last_peered":"2024-09-16T09:42:07.526366+0000","last_clean":"2024-09-16T09:42:07.526366+0000","last_became_active":"2024-09-16T09:25:03.714635+0000","last_became_peered":"2024-09-16T09:25:03.714635+0000","last_unstale":"2024-09-16T09:42:07.526366+0000","last_undegraded":"2024-09-16T09:42:07.526366+0000","last_fullsized":"2024-09-16T09:42:07.526366+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'15","last_scrub_stamp":"2024-09-16T09:41:43.064729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:41:43.064729+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:53.613534+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":511,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:07.527205+0000","last_change":"2024-09-16T09:41:05.861631+0000","last_active":"2024-09-16T09:42:07.527205+0000","last_peered":"2024-09-16T09:42:07.527205+0000","last_clean":"2024-09-16T09:42:07.527205+0000","last_became_active":"2024-09-16T09:33:30.242238+0000","last_became_peered":"2024-09-16T09:33:30.242238+0000","last_unstale":"2024-09-16T09:42:07.527205+0000","last_undegraded":"2024-09-16T09:42:07.527205+0000","last_fullsized":"2024-09-16T09:42:07.527205+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:05.861578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:41:05.861578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:08.589335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":867,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:07.526943+0000","last_change":"2024-09-16T09:41:31.078139+0000","last_active":"2024-09-16T09:42:07.526943+0000","last_peered":"2024-09-16T09:42:07.526943+0000","last_clean":"2024-09-16T09:42:07.526943+0000","last_became_active":"2024-09-16T09:25:07.715442+0000","last_became_peered":"2024-09-16T09:25:07.715442+0000","last_unstale":"2024-09-16T09:42:07.526943+0000","last_undegraded":"2024-09-16T09:42:07.526943+0000","last_fullsized":"2024-09-16T09:42:07.526943+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:31.078058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:41:31.078058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:32.264866+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":615,"reported_epoch":475,"state":"active+clean","last_fresh":"2024-09-16T09:42:13.566275+0000","last_change":"2024-09-16T09:42:07.714808+0000","last_active":"2024-09-16T09:42:13.566275+0000","last_peered":"2024-09-16T09:42:13.566275+0000","last_clean":"2024-09-16T09:42:13.566275+0000","last_became_active":"2024-09-16T09:31:09.269342+0000","last_became_peered":"2024-09-16T09:31:09.269342+0000","last_unstale":"2024-09-16T09:42:13.566275+0000","last_undegraded":"2024-09-16T09:42:13.566275+0000","last_fullsized":"2024-09-16T09:42:13.566275+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:42:07.714650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:42:07.714650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:27.261818+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":746,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:07.528289+0000","last_change":"2024-09-16T09:39:28.963009+0000","last_active":"2024-09-16T09:42:07.528289+0000","last_peered":"2024-09-16T09:42:07.528289+0000","last_clean":"2024-09-16T09:42:07.528289+0000","last_became_active":"2024-09-16T09:31:09.275136+0000","last_became_peered":"2024-09-16T09:31:09.275136+0000","last_unstale":"2024-09-16T09:42:07.528289+0000","last_undegraded":"2024-09-16T09:42:07.528289+0000","last_fullsized":"2024-09-16T09:42:07.528289+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:39:28.962926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:39:28.962926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:24.910431+0000","scrub_duration":59,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.19","version":"285'27","reported_seq":98,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:07.526661+0000","last_change":"2024-09-16T09:40:58.802023+0000","last_active":"2024-09-16T09:42:07.526661+0000","last_peered":"2024-09-16T09:42:07.526661+0000","last_clean":"2024-09-16T09:42:07.526661+0000","last_became_active":"2024-09-16T09:40:58.699357+0000","last_became_peered":"2024-09-16T09:40:58.699357+0000","last_unstale":"2024-09-16T09:42:07.526661+0000","last_undegraded":"2024-09-16T09:42:07.526661+0000","last_fullsized":"2024-09-16T09:42:07.526661+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'27","last_scrub_stamp":"2024-09-16T09:37:29.683568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:37:29.683568+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:21.293250+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,5],"acting":[4,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":888,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:10.051996+0000","last_change":"2024-09-16T09:42:10.051996+0000","last_active":"2024-09-16T09:42:10.051996+0000","last_peered":"2024-09-16T09:42:10.051996+0000","last_clean":"2024-09-16T09:42:10.051996+0000","last_became_active":"2024-09-16T09:25:05.717236+0000","last_became_peered":"2024-09-16T09:25:05.717236+0000","last_unstale":"2024-09-16T09:42:10.051996+0000","last_undegraded":"2024-09-16T09:42:10.051996+0000","last_fullsized":"2024-09-16T09:42:10.051996+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:42:10.051939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:42:10.051939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:22.638958+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":524,"reported_epoch":476,"state":"active+clean","last_fresh":"2024-09-16T09:42:14.443127+0000","last_change":"2024-09-16T09:41:33.135452+0000","last_active":"2024-09-16T09:42:14.443127+0000","last_peered":"2024-09-16T09:42:14.443127+0000","last_clean":"2024-09-16T09:42:14.443127+0000","last_became_active":"2024-09-16T09:33:30.247733+0000","last_became_peered":"2024-09-16T09:33:30.247733+0000","last_unstale":"2024-09-16T09:42:14.443127+0000","last_undegraded":"2024-09-16T09:42:14.443127+0000","last_fullsized":"2024-09-16T09:42:14.443127+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:33.135342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:41:33.135342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:57.435366+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1a","version":"285'15","reported_seq":908,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:07.527327+0000","last_change":"2024-09-16T09:41:47.100374+0000","last_active":"2024-09-16T09:42:07.527327+0000","last_peered":"2024-09-16T09:42:07.527327+0000","last_clean":"2024-09-16T09:42:07.527327+0000","last_became_active":"2024-09-16T09:25:03.709147+0000","last_became_peered":"2024-09-16T09:25:03.709147+0000","last_unstale":"2024-09-16T09:42:07.527327+0000","last_undegraded":"2024-09-16T09:42:07.527327+0000","last_fullsized":"2024-09-16T09:42:07.527327+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'15","last_scrub_stamp":"2024-09-16T09:41:47.100292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:41:47.100292+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:52.405007+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":862,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:07.526533+0000","last_change":"2024-09-16T09:36:51.134364+0000","last_active":"2024-09-16T09:42:07.526533+0000","last_peered":"2024-09-16T09:42:07.526533+0000","last_clean":"2024-09-16T09:42:07.526533+0000","last_became_active":"2024-09-16T09:25:05.718134+0000","last_became_peered":"2024-09-16T09:25:05.718134+0000","last_unstale":"2024-09-16T09:42:07.526533+0000","last_undegraded":"2024-09-16T09:42:07.526533+0000","last_fullsized":"2024-09-16T09:42:07.526533+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:51.134314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:36:51.134314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:51.766506+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"66'1","reported_seq":889,"reported_epoch":476,"state":"active+clean","last_fresh":"2024-09-16T09:42:14.442293+0000","last_change":"2024-09-16T09:41:48.089052+0000","last_active":"2024-09-16T09:42:14.442293+0000","last_peered":"2024-09-16T09:42:14.442293+0000","last_clean":"2024-09-16T09:42:14.442293+0000","last_became_active":"2024-09-16T09:25:07.716558+0000","last_became_peered":"2024-09-16T09:25:07.716558+0000","last_unstale":"2024-09-16T09:42:14.442293+0000","last_undegraded":"2024-09-16T09:42:14.442293+0000","last_fullsized":"2024-09-16T09:42:14.442293+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-16T09:41:48.088985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:41:48.088985+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:03.846875+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":862,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:07.526941+0000","last_change":"2024-09-16T09:33:19.326765+0000","last_active":"2024-09-16T09:42:07.526941+0000","last_peered":"2024-09-16T09:42:07.526941+0000","last_clean":"2024-09-16T09:42:07.526941+0000","last_became_active":"2024-09-16T09:25:01.696939+0000","last_became_peered":"2024-09-16T09:25:01.696939+0000","last_unstale":"2024-09-16T09:42:07.526941+0000","last_undegraded":"2024-09-16T09:42:07.526941+0000","last_fullsized":"2024-09-16T09:42:07.526941+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:19.326695+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:33:19.326695+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:21.783748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"285'7","reported_seq":880,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:07.527671+0000","last_change":"2024-09-16T09:35:39.975915+0000","last_active":"2024-09-16T09:42:07.527671+0000","last_peered":"2024-09-16T09:42:07.527671+0000","last_clean":"2024-09-16T09:42:07.527671+0000","last_became_active":"2024-09-16T09:25:03.716136+0000","last_became_peered":"2024-09-16T09:25:03.716136+0000","last_unstale":"2024-09-16T09:42:07.527671+0000","last_undegraded":"2024-09-16T09:42:07.527671+0000","last_fullsized":"2024-09-16T09:42:07.527671+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'7","last_scrub_stamp":"2024-09-16T09:35:39.975840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:35:39.975840+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:04.157782+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":10,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":26,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:08.915352+0000","last_change":"2024-09-16T09:42:08.915352+0000","last_active":"2024-09-16T09:42:08.915352+0000","last_peered":"2024-09-16T09:42:08.915352+0000","last_clean":"2024-09-16T09:42:08.915352+0000","last_became_active":"2024-09-16T09:42:02.494290+0000","last_became_peered":"2024-09-16T09:42:02.494290+0000","last_unstale":"2024-09-16T09:42:08.915352+0000","last_undegraded":"2024-09-16T09:42:08.915352+0000","last_fullsized":"2024-09-16T09:42:08.915352+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:42:08.915303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:42:08.915303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:19.974744+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4],"acting":[6,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":889,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:07.527331+0000","last_change":"2024-09-16T09:38:34.050374+0000","last_active":"2024-09-16T09:42:07.527331+0000","last_peered":"2024-09-16T09:42:07.527331+0000","last_clean":"2024-09-16T09:42:07.527331+0000","last_became_active":"2024-09-16T09:31:09.275656+0000","last_became_peered":"2024-09-16T09:31:09.275656+0000","last_unstale":"2024-09-16T09:42:07.527331+0000","last_undegraded":"2024-09-16T09:42:07.527331+0000","last_fullsized":"2024-09-16T09:42:07.527331+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:38:34.050185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:38:34.050185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:52.760980+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":869,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:07.529961+0000","last_change":"2024-09-16T09:33:48.322010+0000","last_active":"2024-09-16T09:42:07.529961+0000","last_peered":"2024-09-16T09:42:07.529961+0000","last_clean":"2024-09-16T09:42:07.529961+0000","last_became_active":"2024-09-16T09:25:07.720793+0000","last_became_peered":"2024-09-16T09:25:07.720793+0000","last_unstale":"2024-09-16T09:42:07.529961+0000","last_undegraded":"2024-09-16T09:42:07.529961+0000","last_fullsized":"2024-09-16T09:42:07.529961+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:48.321895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:33:48.321895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:54.184395+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":619,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:07.528361+0000","last_change":"2024-09-16T09:34:33.866299+0000","last_active":"2024-09-16T09:42:07.528361+0000","last_peered":"2024-09-16T09:42:07.528361+0000","last_clean":"2024-09-16T09:42:07.528361+0000","last_became_active":"2024-09-16T09:30:39.201824+0000","last_became_peered":"2024-09-16T09:30:39.201824+0000","last_unstale":"2024-09-16T09:42:07.528361+0000","last_undegraded":"2024-09-16T09:42:07.528361+0000","last_fullsized":"2024-09-16T09:42:07.528361+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:33.866142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:34:33.866142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:22.864352+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"285'27","reported_seq":976,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:07.527972+0000","last_change":"2024-09-16T09:40:23.465274+0000","last_active":"2024-09-16T09:42:07.527972+0000","last_peered":"2024-09-16T09:42:07.527972+0000","last_clean":"2024-09-16T09:42:07.527972+0000","last_became_active":"2024-09-16T09:33:30.249798+0000","last_became_peered":"2024-09-16T09:33:30.249798+0000","last_unstale":"2024-09-16T09:42:07.527972+0000","last_undegraded":"2024-09-16T09:42:07.527972+0000","last_fullsized":"2024-09-16T09:42:07.527972+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'27","last_scrub_stamp":"2024-09-16T09:40:23.465200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:40:23.465200+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:52.565425+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":503,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:07.529187+0000","last_change":"2024-09-16T09:33:43.326933+0000","last_active":"2024-09-16T09:42:07.529187+0000","last_peered":"2024-09-16T09:42:07.529187+0000","last_clean":"2024-09-16T09:42:07.529187+0000","last_became_active":"2024-09-16T09:33:30.251587+0000","last_became_peered":"2024-09-16T09:33:30.251587+0000","last_unstale":"2024-09-16T09:42:07.529187+0000","last_undegraded":"2024-09-16T09:42:07.529187+0000","last_fullsized":"2024-09-16T09:42:07.529187+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:43.326852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:43.326852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:55.169781+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,0],"acting":[4,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":725,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:07.528474+0000","last_change":"2024-09-16T09:40:27.916994+0000","last_active":"2024-09-16T09:42:07.528474+0000","last_peered":"2024-09-16T09:42:07.528474+0000","last_clean":"2024-09-16T09:42:07.528474+0000","last_became_active":"2024-09-16T09:28:52.974255+0000","last_became_peered":"2024-09-16T09:28:52.974255+0000","last_unstale":"2024-09-16T09:42:07.528474+0000","last_undegraded":"2024-09-16T09:42:07.528474+0000","last_fullsized":"2024-09-16T09:42:07.528474+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:40:27.916860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:40:27.916860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:00.899019+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":887,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:07.527464+0000","last_change":"2024-09-16T09:41:08.223235+0000","last_active":"2024-09-16T09:42:07.527464+0000","last_peered":"2024-09-16T09:42:07.527464+0000","last_clean":"2024-09-16T09:42:07.527464+0000","last_became_active":"2024-09-16T09:25:01.706491+0000","last_became_peered":"2024-09-16T09:25:01.706491+0000","last_unstale":"2024-09-16T09:42:07.527464+0000","last_undegraded":"2024-09-16T09:42:07.527464+0000","last_fullsized":"2024-09-16T09:42:07.527464+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:08.223190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:41:08.223190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:23.681454+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"285'18","reported_seq":563,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:07.529329+0000","last_change":"2024-09-16T09:40:37.206512+0000","last_active":"2024-09-16T09:42:07.529329+0000","last_peered":"2024-09-16T09:42:07.529329+0000","last_clean":"2024-09-16T09:42:07.529329+0000","last_became_active":"2024-09-16T09:33:30.251626+0000","last_became_peered":"2024-09-16T09:33:30.251626+0000","last_unstale":"2024-09-16T09:42:07.529329+0000","last_undegraded":"2024-09-16T09:42:07.529329+0000","last_fullsized":"2024-09-16T09:42:07.529329+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'18","last_scrub_stamp":"2024-09-16T09:40:37.206432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:40:37.206432+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":33,"num_read_kb":24,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":863,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:07.527514+0000","last_change":"2024-09-16T09:41:45.118370+0000","last_active":"2024-09-16T09:42:07.527514+0000","last_peered":"2024-09-16T09:42:07.527514+0000","last_clean":"2024-09-16T09:42:07.527514+0000","last_became_active":"2024-09-16T09:25:05.710093+0000","last_became_peered":"2024-09-16T09:25:05.710093+0000","last_unstale":"2024-09-16T09:42:07.527514+0000","last_undegraded":"2024-09-16T09:42:07.527514+0000","last_fullsized":"2024-09-16T09:42:07.527514+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:45.118305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:41:45.118305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:14.199468+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":865,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:07.532057+0000","last_change":"2024-09-16T09:37:48.412421+0000","last_active":"2024-09-16T09:42:07.532057+0000","last_peered":"2024-09-16T09:42:07.532057+0000","last_clean":"2024-09-16T09:42:07.532057+0000","last_became_active":"2024-09-16T09:25:07.719380+0000","last_became_peered":"2024-09-16T09:25:07.719380+0000","last_unstale":"2024-09-16T09:42:07.532057+0000","last_undegraded":"2024-09-16T09:42:07.532057+0000","last_fullsized":"2024-09-16T09:42:07.532057+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-16T09:37:48.411054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:37:48.411054+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:24.108878+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":530,"reported_epoch":475,"state":"active+clean","last_fresh":"2024-09-16T09:42:13.567950+0000","last_change":"2024-09-16T09:41:36.612463+0000","last_active":"2024-09-16T09:42:13.567950+0000","last_peered":"2024-09-16T09:42:13.567950+0000","last_clean":"2024-09-16T09:42:13.567950+0000","last_became_active":"2024-09-16T09:33:30.245048+0000","last_became_peered":"2024-09-16T09:33:30.245048+0000","last_unstale":"2024-09-16T09:42:13.567950+0000","last_undegraded":"2024-09-16T09:42:13.567950+0000","last_fullsized":"2024-09-16T09:42:13.567950+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:41:36.612280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:41:36.612280+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:02.315301+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"285'20","reported_seq":949,"reported_epoch":476,"state":"active+clean","last_fresh":"2024-09-16T09:42:14.446275+0000","last_change":"2024-09-16T09:39:35.550760+0000","last_active":"2024-09-16T09:42:14.446275+0000","last_peered":"2024-09-16T09:42:14.446275+0000","last_clean":"2024-09-16T09:42:14.446275+0000","last_became_active":"2024-09-16T09:31:09.275050+0000","last_became_peered":"2024-09-16T09:31:09.275050+0000","last_unstale":"2024-09-16T09:42:14.446275+0000","last_undegraded":"2024-09-16T09:42:14.446275+0000","last_fullsized":"2024-09-16T09:42:14.446275+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'20","last_scrub_stamp":"2024-09-16T09:39:35.548333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:39:35.548333+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:05.787546+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"66'8","reported_seq":1123,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:10.905060+0000","last_change":"2024-09-16T09:42:10.904990+0000","last_active":"2024-09-16T09:42:10.905060+0000","last_peered":"2024-09-16T09:42:10.905060+0000","last_clean":"2024-09-16T09:42:10.905060+0000","last_became_active":"2024-09-16T09:25:05.712516+0000","last_became_peered":"2024-09-16T09:25:05.712516+0000","last_unstale":"2024-09-16T09:42:10.905060+0000","last_undegraded":"2024-09-16T09:42:10.905060+0000","last_fullsized":"2024-09-16T09:42:10.905060+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:42:10.904920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:42:10.904920+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:19.925242+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.0","version":"474'5624","reported_seq":5673,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:12.489128+0000","last_change":"2024-09-16T09:41:13.139275+0000","last_active":"2024-09-16T09:42:12.489128+0000","last_peered":"2024-09-16T09:42:12.489128+0000","last_clean":"2024-09-16T09:42:12.489128+0000","last_became_active":"2024-09-16T09:41:13.138983+0000","last_became_peered":"2024-09-16T09:41:13.138983+0000","last_unstale":"2024-09-16T09:42:12.489128+0000","last_undegraded":"2024-09-16T09:42:12.489128+0000","last_fullsized":"2024-09-16T09:42:12.489128+0000","mapping_epoch":454,"log_start":"460'1200","ondisk_log_start":"460'1200","created":454,"last_epoch_clean":455,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:11.976785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:41:11.976785+0000","last_clean_scrub_stamp":"2024-09-16T09:41:11.976785+0000","objects_scrubbed":0,"log_size":4424,"log_dups_size":0,"ondisk_log_size":4424,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:27.450124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":368574464,"num_objects":5624,"num_object_clones":0,"num_object_copies":16872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5624,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5624,"num_write_kb":359936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3],"acting":[7,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.a","version":"73'79","reported_seq":811,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:07.526135+0000","last_change":"2024-09-16T09:41:28.733947+0000","last_active":"2024-09-16T09:42:07.526135+0000","last_peered":"2024-09-16T09:42:07.526135+0000","last_clean":"2024-09-16T09:42:07.526135+0000","last_became_active":"2024-09-16T09:26:35.127352+0000","last_became_peered":"2024-09-16T09:26:35.127352+0000","last_unstale":"2024-09-16T09:42:07.526135+0000","last_undegraded":"2024-09-16T09:42:07.526135+0000","last_fullsized":"2024-09-16T09:42:07.526135+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:41:28.733854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:41:28.733854+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:35.418965+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"285'27","reported_seq":846,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:07.528118+0000","last_change":"2024-09-16T09:41:28.675906+0000","last_active":"2024-09-16T09:42:07.528118+0000","last_peered":"2024-09-16T09:42:07.528118+0000","last_clean":"2024-09-16T09:42:07.528118+0000","last_became_active":"2024-09-16T09:26:33.295672+0000","last_became_peered":"2024-09-16T09:26:33.295672+0000","last_unstale":"2024-09-16T09:42:07.528118+0000","last_undegraded":"2024-09-16T09:42:07.528118+0000","last_fullsized":"2024-09-16T09:42:07.528118+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'27","last_scrub_stamp":"2024-09-16T09:41:28.675741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:41:28.675741+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:47.696328+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":616,"reported_epoch":475,"state":"active+clean","last_fresh":"2024-09-16T09:42:13.567527+0000","last_change":"2024-09-16T09:38:35.023594+0000","last_active":"2024-09-16T09:42:13.567527+0000","last_peered":"2024-09-16T09:42:13.567527+0000","last_clean":"2024-09-16T09:42:13.567527+0000","last_became_active":"2024-09-16T09:31:09.274534+0000","last_became_peered":"2024-09-16T09:31:09.274534+0000","last_unstale":"2024-09-16T09:42:13.567527+0000","last_undegraded":"2024-09-16T09:42:13.567527+0000","last_fullsized":"2024-09-16T09:42:13.567527+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:38:35.023518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:38:35.023518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:00.691678+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"66'8","reported_seq":1079,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:07.530066+0000","last_change":"2024-09-16T09:41:32.036261+0000","last_active":"2024-09-16T09:42:07.530066+0000","last_peered":"2024-09-16T09:42:07.530066+0000","last_clean":"2024-09-16T09:42:07.530066+0000","last_became_active":"2024-09-16T09:25:05.713813+0000","last_became_peered":"2024-09-16T09:25:05.713813+0000","last_unstale":"2024-09-16T09:42:07.530066+0000","last_undegraded":"2024-09-16T09:42:07.530066+0000","last_fullsized":"2024-09-16T09:42:07.530066+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:41:32.036192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:41:32.036192+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:57.987875+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":875,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:07.526807+0000","last_change":"2024-09-16T09:41:48.730584+0000","last_active":"2024-09-16T09:42:07.526807+0000","last_peered":"2024-09-16T09:42:07.526807+0000","last_clean":"2024-09-16T09:42:07.526807+0000","last_became_active":"2024-09-16T09:25:07.717344+0000","last_became_peered":"2024-09-16T09:25:07.717344+0000","last_unstale":"2024-09-16T09:42:07.526807+0000","last_undegraded":"2024-09-16T09:42:07.526807+0000","last_fullsized":"2024-09-16T09:42:07.526807+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:48.730521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:41:48.730521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:08.160356+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.f","version":"474'5487","reported_seq":5537,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:12.528306+0000","last_change":"2024-09-16T09:41:13.139299+0000","last_active":"2024-09-16T09:42:12.528306+0000","last_peered":"2024-09-16T09:42:12.528306+0000","last_clean":"2024-09-16T09:42:12.528306+0000","last_became_active":"2024-09-16T09:41:13.138830+0000","last_became_peered":"2024-09-16T09:41:13.138830+0000","last_unstale":"2024-09-16T09:42:12.528306+0000","last_undegraded":"2024-09-16T09:42:12.528306+0000","last_fullsized":"2024-09-16T09:42:12.528306+0000","mapping_epoch":454,"log_start":"460'1100","ondisk_log_start":"460'1100","created":454,"last_epoch_clean":455,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:11.976785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:41:11.976785+0000","last_clean_scrub_stamp":"2024-09-16T09:41:11.976785+0000","objects_scrubbed":0,"log_size":4387,"log_dups_size":0,"ondisk_log_size":4387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:38.677682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":359596032,"num_objects":5487,"num_object_clones":0,"num_object_copies":16461,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5487,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5487,"num_write_kb":351168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.5","version":"73'79","reported_seq":843,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:07.529626+0000","last_change":"2024-09-16T09:40:29.124394+0000","last_active":"2024-09-16T09:42:07.529626+0000","last_peered":"2024-09-16T09:42:07.529626+0000","last_clean":"2024-09-16T09:42:07.529626+0000","last_became_active":"2024-09-16T09:30:08.763683+0000","last_became_peered":"2024-09-16T09:30:08.763683+0000","last_unstale":"2024-09-16T09:42:07.529626+0000","last_undegraded":"2024-09-16T09:42:07.529626+0000","last_fullsized":"2024-09-16T09:42:07.529626+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:40:29.124314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:40:29.124314+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"285'30","reported_seq":842,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:07.532131+0000","last_change":"2024-09-16T09:41:41.151427+0000","last_active":"2024-09-16T09:42:07.532131+0000","last_peered":"2024-09-16T09:42:07.532131+0000","last_clean":"2024-09-16T09:42:07.532131+0000","last_became_active":"2024-09-16T09:31:09.270434+0000","last_became_peered":"2024-09-16T09:31:09.270434+0000","last_unstale":"2024-09-16T09:42:07.532131+0000","last_undegraded":"2024-09-16T09:42:07.532131+0000","last_fullsized":"2024-09-16T09:42:07.532131+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'30","last_scrub_stamp":"2024-09-16T09:41:41.151345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:41:41.151345+0000","objects_scrubbed":8,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:55.242044+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":51,"num_read_kb":39,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":722,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:07.536570+0000","last_change":"2024-09-16T09:41:45.162761+0000","last_active":"2024-09-16T09:42:07.536570+0000","last_peered":"2024-09-16T09:42:07.536570+0000","last_clean":"2024-09-16T09:42:07.536570+0000","last_became_active":"2024-09-16T09:28:52.983543+0000","last_became_peered":"2024-09-16T09:28:52.983543+0000","last_unstale":"2024-09-16T09:42:07.536570+0000","last_undegraded":"2024-09-16T09:42:07.536570+0000","last_fullsized":"2024-09-16T09:42:07.536570+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:45.162688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:41:45.162688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:01.911633+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":895,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:09.103716+0000","last_change":"2024-09-16T09:42:09.103716+0000","last_active":"2024-09-16T09:42:09.103716+0000","last_peered":"2024-09-16T09:42:09.103716+0000","last_clean":"2024-09-16T09:42:09.103716+0000","last_became_active":"2024-09-16T09:31:09.268864+0000","last_became_peered":"2024-09-16T09:31:09.268864+0000","last_unstale":"2024-09-16T09:42:09.103716+0000","last_undegraded":"2024-09-16T09:42:09.103716+0000","last_fullsized":"2024-09-16T09:42:09.103716+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:42:09.103657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:42:09.103657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:13.695404+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":891,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:07.527225+0000","last_change":"2024-09-16T09:41:33.051473+0000","last_active":"2024-09-16T09:42:07.527225+0000","last_peered":"2024-09-16T09:42:07.527225+0000","last_clean":"2024-09-16T09:42:07.527225+0000","last_became_active":"2024-09-16T09:31:09.269433+0000","last_became_peered":"2024-09-16T09:31:09.269433+0000","last_unstale":"2024-09-16T09:42:07.527225+0000","last_undegraded":"2024-09-16T09:42:07.527225+0000","last_fullsized":"2024-09-16T09:42:07.527225+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:33.051405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:41:33.051405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:02.030034+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"474'5628","reported_seq":5679,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:13.132284+0000","last_change":"2024-09-16T09:41:13.140372+0000","last_active":"2024-09-16T09:42:13.132284+0000","last_peered":"2024-09-16T09:42:13.132284+0000","last_clean":"2024-09-16T09:42:13.132284+0000","last_became_active":"2024-09-16T09:41:13.140053+0000","last_became_peered":"2024-09-16T09:41:13.140053+0000","last_unstale":"2024-09-16T09:42:13.132284+0000","last_undegraded":"2024-09-16T09:42:13.132284+0000","last_fullsized":"2024-09-16T09:42:13.132284+0000","mapping_epoch":454,"log_start":"460'1200","ondisk_log_start":"460'1200","created":454,"last_epoch_clean":455,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:11.976785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:41:11.976785+0000","last_clean_scrub_stamp":"2024-09-16T09:41:11.976785+0000","objects_scrubbed":0,"log_size":4428,"log_dups_size":0,"ondisk_log_size":4428,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:23.943934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":368836608,"num_objects":5628,"num_object_clones":0,"num_object_copies":16884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5628,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5628,"num_write_kb":360192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,7],"acting":[4,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"73'79","reported_seq":616,"reported_epoch":475,"state":"active+clean","last_fresh":"2024-09-16T09:42:13.567462+0000","last_change":"2024-09-16T09:41:44.602634+0000","last_active":"2024-09-16T09:42:13.567462+0000","last_peered":"2024-09-16T09:42:13.567462+0000","last_clean":"2024-09-16T09:42:13.567462+0000","last_became_active":"2024-09-16T09:31:09.275139+0000","last_became_peered":"2024-09-16T09:31:09.275139+0000","last_unstale":"2024-09-16T09:42:13.567462+0000","last_undegraded":"2024-09-16T09:42:13.567462+0000","last_fullsized":"2024-09-16T09:42:13.567462+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:41:44.602557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:41:44.602557+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:02.511638+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"285'28","reported_seq":927,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:07.526897+0000","last_change":"2024-09-16T09:41:11.201590+0000","last_active":"2024-09-16T09:42:07.526897+0000","last_peered":"2024-09-16T09:42:07.526897+0000","last_clean":"2024-09-16T09:42:07.526897+0000","last_became_active":"2024-09-16T09:25:03.708830+0000","last_became_peered":"2024-09-16T09:25:03.708830+0000","last_unstale":"2024-09-16T09:42:07.526897+0000","last_undegraded":"2024-09-16T09:42:07.526897+0000","last_fullsized":"2024-09-16T09:42:07.526897+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'28","last_scrub_stamp":"2024-09-16T09:41:11.201523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:41:11.201523+0000","objects_scrubbed":7,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:16.854189+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":909,"reported_epoch":476,"state":"active+clean","last_fresh":"2024-09-16T09:42:14.446531+0000","last_change":"2024-09-16T09:40:31.037325+0000","last_active":"2024-09-16T09:42:14.446531+0000","last_peered":"2024-09-16T09:42:14.446531+0000","last_clean":"2024-09-16T09:42:14.446531+0000","last_became_active":"2024-09-16T09:25:01.702992+0000","last_became_peered":"2024-09-16T09:25:01.702992+0000","last_unstale":"2024-09-16T09:42:14.446531+0000","last_undegraded":"2024-09-16T09:42:14.446531+0000","last_fullsized":"2024-09-16T09:42:14.446531+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:40:31.037258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:40:31.037258+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"235'12","reported_seq":635,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:07.526203+0000","last_change":"2024-09-16T09:37:19.380567+0000","last_active":"2024-09-16T09:42:07.526203+0000","last_peered":"2024-09-16T09:42:07.526203+0000","last_clean":"2024-09-16T09:42:07.526203+0000","last_became_active":"2024-09-16T09:33:30.251047+0000","last_became_peered":"2024-09-16T09:33:30.251047+0000","last_unstale":"2024-09-16T09:42:07.526203+0000","last_undegraded":"2024-09-16T09:42:07.526203+0000","last_fullsized":"2024-09-16T09:42:07.526203+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"235'12","last_scrub_stamp":"2024-09-16T09:37:19.380504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:37:19.380504+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:20.256135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":866,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:07.531708+0000","last_change":"2024-09-16T09:40:30.956991+0000","last_active":"2024-09-16T09:42:07.531708+0000","last_peered":"2024-09-16T09:42:07.531708+0000","last_clean":"2024-09-16T09:42:07.531708+0000","last_became_active":"2024-09-16T09:25:07.722017+0000","last_became_peered":"2024-09-16T09:25:07.722017+0000","last_unstale":"2024-09-16T09:42:07.531708+0000","last_undegraded":"2024-09-16T09:42:07.531708+0000","last_fullsized":"2024-09-16T09:42:07.531708+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:40:30.956926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:40:30.956926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:01.216573+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.d","version":"474'5631","reported_seq":5682,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:13.106577+0000","last_change":"2024-09-16T09:41:13.005502+0000","last_active":"2024-09-16T09:42:13.106577+0000","last_peered":"2024-09-16T09:42:13.106577+0000","last_clean":"2024-09-16T09:42:13.106577+0000","last_became_active":"2024-09-16T09:41:13.005224+0000","last_became_peered":"2024-09-16T09:41:13.005224+0000","last_unstale":"2024-09-16T09:42:13.106577+0000","last_undegraded":"2024-09-16T09:42:13.106577+0000","last_fullsized":"2024-09-16T09:42:13.106577+0000","mapping_epoch":454,"log_start":"460'1200","ondisk_log_start":"460'1200","created":454,"last_epoch_clean":455,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:11.976785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:41:11.976785+0000","last_clean_scrub_stamp":"2024-09-16T09:41:11.976785+0000","objects_scrubbed":0,"log_size":4431,"log_dups_size":0,"ondisk_log_size":4431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:17.246036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369033216,"num_objects":5631,"num_object_clones":0,"num_object_copies":16893,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5631,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5631,"num_write_kb":360384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"73'79","reported_seq":945,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:07.528385+0000","last_change":"2024-09-16T09:41:10.908936+0000","last_active":"2024-09-16T09:42:07.528385+0000","last_peered":"2024-09-16T09:42:07.528385+0000","last_clean":"2024-09-16T09:42:07.528385+0000","last_became_active":"2024-09-16T09:31:09.274707+0000","last_became_peered":"2024-09-16T09:31:09.274707+0000","last_unstale":"2024-09-16T09:42:07.528385+0000","last_undegraded":"2024-09-16T09:42:07.528385+0000","last_fullsized":"2024-09-16T09:42:07.528385+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:41:10.908852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:41:10.908852+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:22.406722+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"285'20","reported_seq":562,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:07.527951+0000","last_change":"2024-09-16T09:38:32.122419+0000","last_active":"2024-09-16T09:42:07.527951+0000","last_peered":"2024-09-16T09:42:07.527951+0000","last_clean":"2024-09-16T09:42:07.527951+0000","last_became_active":"2024-09-16T09:33:30.248453+0000","last_became_peered":"2024-09-16T09:33:30.248453+0000","last_unstale":"2024-09-16T09:42:07.527951+0000","last_undegraded":"2024-09-16T09:42:07.527951+0000","last_fullsized":"2024-09-16T09:42:07.527951+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'20","last_scrub_stamp":"2024-09-16T09:38:32.122303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:38:32.122303+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:49.721122+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":886,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:07.527421+0000","last_change":"2024-09-16T09:36:47.095703+0000","last_active":"2024-09-16T09:42:07.527421+0000","last_peered":"2024-09-16T09:42:07.527421+0000","last_clean":"2024-09-16T09:42:07.527421+0000","last_became_active":"2024-09-16T09:31:09.270619+0000","last_became_peered":"2024-09-16T09:31:09.270619+0000","last_unstale":"2024-09-16T09:42:07.527421+0000","last_undegraded":"2024-09-16T09:42:07.527421+0000","last_fullsized":"2024-09-16T09:42:07.527421+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:47.095658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:36:47.095658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:00.700577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"66'8","reported_seq":1084,"reported_epoch":476,"state":"active+clean","last_fresh":"2024-09-16T09:42:14.444360+0000","last_change":"2024-09-16T09:36:52.689167+0000","last_active":"2024-09-16T09:42:14.444360+0000","last_peered":"2024-09-16T09:42:14.444360+0000","last_clean":"2024-09-16T09:42:14.444360+0000","last_became_active":"2024-09-16T09:25:05.714507+0000","last_became_peered":"2024-09-16T09:25:05.714507+0000","last_unstale":"2024-09-16T09:42:14.444360+0000","last_undegraded":"2024-09-16T09:42:14.444360+0000","last_fullsized":"2024-09-16T09:42:14.444360+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:36:52.689091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:36:52.689091+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":895,"reported_epoch":476,"state":"active+clean","last_fresh":"2024-09-16T09:42:14.442424+0000","last_change":"2024-09-16T09:41:31.179732+0000","last_active":"2024-09-16T09:42:14.442424+0000","last_peered":"2024-09-16T09:42:14.442424+0000","last_clean":"2024-09-16T09:42:14.442424+0000","last_became_active":"2024-09-16T09:31:09.283192+0000","last_became_peered":"2024-09-16T09:31:09.283192+0000","last_unstale":"2024-09-16T09:42:14.442424+0000","last_undegraded":"2024-09-16T09:42:14.442424+0000","last_fullsized":"2024-09-16T09:42:14.442424+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:31.179662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:41:31.179662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:39.168681+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.c","version":"474'5538","reported_seq":5587,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:13.391096+0000","last_change":"2024-09-16T09:41:13.139322+0000","last_active":"2024-09-16T09:42:13.391096+0000","last_peered":"2024-09-16T09:42:13.391096+0000","last_clean":"2024-09-16T09:42:13.391096+0000","last_became_active":"2024-09-16T09:41:13.138917+0000","last_became_peered":"2024-09-16T09:41:13.138917+0000","last_unstale":"2024-09-16T09:42:13.391096+0000","last_undegraded":"2024-09-16T09:42:13.391096+0000","last_fullsized":"2024-09-16T09:42:13.391096+0000","mapping_epoch":454,"log_start":"0'0","ondisk_log_start":"0'0","created":454,"last_epoch_clean":455,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:11.976785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:41:11.976785+0000","last_clean_scrub_stamp":"2024-09-16T09:41:11.976785+0000","objects_scrubbed":0,"log_size":5538,"log_dups_size":0,"ondisk_log_size":5538,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:18.800527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362938368,"num_objects":5538,"num_object_clones":0,"num_object_copies":16614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5538,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5538,"num_write_kb":354432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"73'79","reported_seq":902,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:07.528087+0000","last_change":"2024-09-16T09:34:42.728232+0000","last_active":"2024-09-16T09:42:07.528087+0000","last_peered":"2024-09-16T09:42:07.528087+0000","last_clean":"2024-09-16T09:42:07.528087+0000","last_became_active":"2024-09-16T09:31:09.274565+0000","last_became_peered":"2024-09-16T09:31:09.274565+0000","last_unstale":"2024-09-16T09:42:07.528087+0000","last_undegraded":"2024-09-16T09:42:07.528087+0000","last_fullsized":"2024-09-16T09:42:07.528087+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:34:42.728143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:34:42.728143+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:20.880320+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"285'29","reported_seq":924,"reported_epoch":476,"state":"active+clean","last_fresh":"2024-09-16T09:42:14.444584+0000","last_change":"2024-09-16T09:36:54.604573+0000","last_active":"2024-09-16T09:42:14.444584+0000","last_peered":"2024-09-16T09:42:14.444584+0000","last_clean":"2024-09-16T09:42:14.444584+0000","last_became_active":"2024-09-16T09:25:03.714797+0000","last_became_peered":"2024-09-16T09:25:03.714797+0000","last_unstale":"2024-09-16T09:42:14.444584+0000","last_undegraded":"2024-09-16T09:42:14.444584+0000","last_fullsized":"2024-09-16T09:42:14.444584+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'29","last_scrub_stamp":"2024-09-16T09:36:54.604484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:36:54.604484+0000","objects_scrubbed":8,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:19.009098+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":54,"num_read_kb":40,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":504,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:07.526797+0000","last_change":"2024-09-16T09:40:26.273165+0000","last_active":"2024-09-16T09:42:07.526797+0000","last_peered":"2024-09-16T09:42:07.526797+0000","last_clean":"2024-09-16T09:42:07.526797+0000","last_became_active":"2024-09-16T09:33:30.246996+0000","last_became_peered":"2024-09-16T09:33:30.246996+0000","last_unstale":"2024-09-16T09:42:07.526797+0000","last_undegraded":"2024-09-16T09:42:07.526797+0000","last_fullsized":"2024-09-16T09:42:07.526797+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:40:26.273106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:40:26.273106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:02.707559+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,2],"acting":[5,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":897,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:07.528866+0000","last_change":"2024-09-16T09:41:46.974010+0000","last_active":"2024-09-16T09:42:07.528866+0000","last_peered":"2024-09-16T09:42:07.528866+0000","last_clean":"2024-09-16T09:42:07.528866+0000","last_became_active":"2024-09-16T09:25:05.708663+0000","last_became_peered":"2024-09-16T09:25:05.708663+0000","last_unstale":"2024-09-16T09:42:07.528866+0000","last_undegraded":"2024-09-16T09:42:07.528866+0000","last_fullsized":"2024-09-16T09:42:07.528866+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:46.973947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:41:46.973947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:14.396908+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":512,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:07.528119+0000","last_change":"2024-09-16T09:37:56.882475+0000","last_active":"2024-09-16T09:42:07.528119+0000","last_peered":"2024-09-16T09:42:07.528119+0000","last_clean":"2024-09-16T09:42:07.528119+0000","last_became_active":"2024-09-16T09:33:30.250178+0000","last_became_peered":"2024-09-16T09:33:30.250178+0000","last_unstale":"2024-09-16T09:42:07.528119+0000","last_undegraded":"2024-09-16T09:42:07.528119+0000","last_fullsized":"2024-09-16T09:42:07.528119+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:56.882326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:37:56.882326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:59.888050+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,2],"acting":[6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.b","version":"475'5566","reported_seq":5619,"reported_epoch":475,"state":"active+clean","last_fresh":"2024-09-16T09:42:13.654676+0000","last_change":"2024-09-16T09:41:13.140706+0000","last_active":"2024-09-16T09:42:13.654676+0000","last_peered":"2024-09-16T09:42:13.654676+0000","last_clean":"2024-09-16T09:42:13.654676+0000","last_became_active":"2024-09-16T09:41:13.140363+0000","last_became_peered":"2024-09-16T09:41:13.140363+0000","last_unstale":"2024-09-16T09:42:13.654676+0000","last_undegraded":"2024-09-16T09:42:13.654676+0000","last_fullsized":"2024-09-16T09:42:13.654676+0000","mapping_epoch":454,"log_start":"460'1300","ondisk_log_start":"460'1300","created":454,"last_epoch_clean":455,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:11.976785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:41:11.976785+0000","last_clean_scrub_stamp":"2024-09-16T09:41:11.976785+0000","objects_scrubbed":0,"log_size":4266,"log_dups_size":0,"ondisk_log_size":4266,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:17.999413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":364773376,"num_objects":5566,"num_object_clones":0,"num_object_copies":16698,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5566,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5566,"num_write_kb":356224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"73'79","reported_seq":843,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:07.527036+0000","last_change":"2024-09-16T09:41:06.841984+0000","last_active":"2024-09-16T09:42:07.527036+0000","last_peered":"2024-09-16T09:42:07.527036+0000","last_clean":"2024-09-16T09:42:07.527036+0000","last_became_active":"2024-09-16T09:30:08.766350+0000","last_became_peered":"2024-09-16T09:30:08.766350+0000","last_unstale":"2024-09-16T09:42:07.527036+0000","last_undegraded":"2024-09-16T09:42:07.527036+0000","last_fullsized":"2024-09-16T09:42:07.527036+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:41:06.841934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:41:06.841934+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:19.225514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"285'42","reported_seq":1226,"reported_epoch":475,"state":"active+clean","last_fresh":"2024-09-16T09:42:13.574452+0000","last_change":"2024-09-16T09:37:57.407386+0000","last_active":"2024-09-16T09:42:13.574452+0000","last_peered":"2024-09-16T09:42:13.574452+0000","last_clean":"2024-09-16T09:42:13.574452+0000","last_became_active":"2024-09-16T09:31:09.278839+0000","last_became_peered":"2024-09-16T09:31:09.278839+0000","last_unstale":"2024-09-16T09:42:13.574452+0000","last_undegraded":"2024-09-16T09:42:13.574452+0000","last_fullsized":"2024-09-16T09:42:13.574452+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'42","last_scrub_stamp":"2024-09-16T09:37:57.407297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:37:57.407297+0000","objects_scrubbed":10,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:22.315877+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":66,"num_read_kb":51,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":897,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:07.527990+0000","last_change":"2024-09-16T09:41:40.124915+0000","last_active":"2024-09-16T09:42:07.527990+0000","last_peered":"2024-09-16T09:42:07.527990+0000","last_clean":"2024-09-16T09:42:07.527990+0000","last_became_active":"2024-09-16T09:25:01.703076+0000","last_became_peered":"2024-09-16T09:25:01.703076+0000","last_unstale":"2024-09-16T09:42:07.527990+0000","last_undegraded":"2024-09-16T09:42:07.527990+0000","last_fullsized":"2024-09-16T09:42:07.527990+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:40.124830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:41:40.124830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:54.216409+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":898,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:07.528239+0000","last_change":"2024-09-16T09:41:43.807570+0000","last_active":"2024-09-16T09:42:07.528239+0000","last_peered":"2024-09-16T09:42:07.528239+0000","last_clean":"2024-09-16T09:42:07.528239+0000","last_became_active":"2024-09-16T09:25:00.696237+0000","last_became_peered":"2024-09-16T09:25:00.696237+0000","last_unstale":"2024-09-16T09:42:07.528239+0000","last_undegraded":"2024-09-16T09:42:07.528239+0000","last_fullsized":"2024-09-16T09:42:07.528239+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-16T09:41:43.807521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:41:43.807521+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:02.549457+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.0025137430000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":869,"reported_epoch":476,"state":"active+clean","last_fresh":"2024-09-16T09:42:14.443351+0000","last_change":"2024-09-16T09:42:10.887831+0000","last_active":"2024-09-16T09:42:14.443351+0000","last_peered":"2024-09-16T09:42:14.443351+0000","last_clean":"2024-09-16T09:42:14.443351+0000","last_became_active":"2024-09-16T09:25:05.717591+0000","last_became_peered":"2024-09-16T09:25:05.717591+0000","last_unstale":"2024-09-16T09:42:14.443351+0000","last_undegraded":"2024-09-16T09:42:14.443351+0000","last_fullsized":"2024-09-16T09:42:14.443351+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:42:10.887766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:42:10.887766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:20.634787+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":608,"reported_epoch":475,"state":"active+clean","last_fresh":"2024-09-16T09:42:13.568347+0000","last_change":"2024-09-16T09:38:25.992634+0000","last_active":"2024-09-16T09:42:13.568347+0000","last_peered":"2024-09-16T09:42:13.568347+0000","last_clean":"2024-09-16T09:42:13.568347+0000","last_became_active":"2024-09-16T09:31:09.272609+0000","last_became_peered":"2024-09-16T09:31:09.272609+0000","last_unstale":"2024-09-16T09:42:13.568347+0000","last_undegraded":"2024-09-16T09:42:13.568347+0000","last_fullsized":"2024-09-16T09:42:13.568347+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:38:25.992569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:38:25.992569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:55.709117+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.8","version":"474'5446","reported_seq":5496,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:12.930911+0000","last_change":"2024-09-16T09:41:13.001905+0000","last_active":"2024-09-16T09:42:12.930911+0000","last_peered":"2024-09-16T09:42:12.930911+0000","last_clean":"2024-09-16T09:42:12.930911+0000","last_became_active":"2024-09-16T09:41:13.001618+0000","last_became_peered":"2024-09-16T09:41:13.001618+0000","last_unstale":"2024-09-16T09:42:12.930911+0000","last_undegraded":"2024-09-16T09:42:12.930911+0000","last_fullsized":"2024-09-16T09:42:12.930911+0000","mapping_epoch":454,"log_start":"460'500","ondisk_log_start":"460'500","created":454,"last_epoch_clean":455,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:11.976785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:41:11.976785+0000","last_clean_scrub_stamp":"2024-09-16T09:41:11.976785+0000","objects_scrubbed":0,"log_size":4946,"log_dups_size":0,"ondisk_log_size":4946,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:38.628474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":356909056,"num_objects":5446,"num_object_clones":0,"num_object_copies":16338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5446,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5446,"num_write_kb":348544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.2","version":"73'79","reported_seq":209,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:10.223632+0000","last_change":"2024-09-16T09:42:10.223632+0000","last_active":"2024-09-16T09:42:10.223632+0000","last_peered":"2024-09-16T09:42:10.223632+0000","last_clean":"2024-09-16T09:42:10.223632+0000","last_became_active":"2024-09-16T09:38:10.556320+0000","last_became_peered":"2024-09-16T09:38:10.556320+0000","last_unstale":"2024-09-16T09:42:10.223632+0000","last_undegraded":"2024-09-16T09:42:10.223632+0000","last_fullsized":"2024-09-16T09:42:10.223632+0000","mapping_epoch":386,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":387,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:42:10.223553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:42:10.223553+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:25.442293+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"285'17","reported_seq":603,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:07.530142+0000","last_change":"2024-09-16T09:41:27.770120+0000","last_active":"2024-09-16T09:42:07.530142+0000","last_peered":"2024-09-16T09:42:07.530142+0000","last_clean":"2024-09-16T09:42:07.530142+0000","last_became_active":"2024-09-16T09:37:33.349009+0000","last_became_peered":"2024-09-16T09:37:33.349009+0000","last_unstale":"2024-09-16T09:42:07.530142+0000","last_undegraded":"2024-09-16T09:42:07.530142+0000","last_fullsized":"2024-09-16T09:42:07.530142+0000","mapping_epoch":367,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":368,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'17","last_scrub_stamp":"2024-09-16T09:41:27.770036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:41:27.770036+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:57.206439+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":35,"num_read_kb":24,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":512,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:07.529146+0000","last_change":"2024-09-16T09:41:46.003680+0000","last_active":"2024-09-16T09:42:07.529146+0000","last_peered":"2024-09-16T09:42:07.529146+0000","last_clean":"2024-09-16T09:42:07.529146+0000","last_became_active":"2024-09-16T09:33:30.247596+0000","last_became_peered":"2024-09-16T09:33:30.247596+0000","last_unstale":"2024-09-16T09:42:07.529146+0000","last_undegraded":"2024-09-16T09:42:07.529146+0000","last_fullsized":"2024-09-16T09:42:07.529146+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:46.003500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:41:46.003500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:01.852542+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,2],"acting":[6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":551,"reported_epoch":476,"state":"active+clean","last_fresh":"2024-09-16T09:42:14.445177+0000","last_change":"2024-09-16T09:41:42.039205+0000","last_active":"2024-09-16T09:42:14.445177+0000","last_peered":"2024-09-16T09:42:14.445177+0000","last_clean":"2024-09-16T09:42:14.445177+0000","last_became_active":"2024-09-16T09:37:04.358985+0000","last_became_peered":"2024-09-16T09:37:04.358985+0000","last_unstale":"2024-09-16T09:42:14.445177+0000","last_undegraded":"2024-09-16T09:42:14.445177+0000","last_fullsized":"2024-09-16T09:42:14.445177+0000","mapping_epoch":343,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":344,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:41:42.038861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:41:42.038861+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:43.040519+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3],"acting":[0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":905,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:07.528329+0000","last_change":"2024-09-16T09:41:45.549361+0000","last_active":"2024-09-16T09:42:07.528329+0000","last_peered":"2024-09-16T09:42:07.528329+0000","last_clean":"2024-09-16T09:42:07.528329+0000","last_became_active":"2024-09-16T09:25:05.711539+0000","last_became_peered":"2024-09-16T09:25:05.711539+0000","last_unstale":"2024-09-16T09:42:07.528329+0000","last_undegraded":"2024-09-16T09:42:07.528329+0000","last_fullsized":"2024-09-16T09:42:07.528329+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:45.549186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:41:45.549186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:58.155707+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":874,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:07.535007+0000","last_change":"2024-09-16T09:41:31.217733+0000","last_active":"2024-09-16T09:42:07.535007+0000","last_peered":"2024-09-16T09:42:07.535007+0000","last_clean":"2024-09-16T09:42:07.535007+0000","last_became_active":"2024-09-16T09:25:07.723282+0000","last_became_peered":"2024-09-16T09:25:07.723282+0000","last_unstale":"2024-09-16T09:42:07.535007+0000","last_undegraded":"2024-09-16T09:42:07.535007+0000","last_fullsized":"2024-09-16T09:42:07.535007+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:31.217664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:41:31.217664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:45.454634+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.9","version":"475'5648","reported_seq":5701,"reported_epoch":475,"state":"active+clean","last_fresh":"2024-09-16T09:42:13.678873+0000","last_change":"2024-09-16T09:41:13.001775+0000","last_active":"2024-09-16T09:42:13.678873+0000","last_peered":"2024-09-16T09:42:13.678873+0000","last_clean":"2024-09-16T09:42:13.678873+0000","last_became_active":"2024-09-16T09:41:13.001488+0000","last_became_peered":"2024-09-16T09:41:13.001488+0000","last_unstale":"2024-09-16T09:42:13.678873+0000","last_undegraded":"2024-09-16T09:42:13.678873+0000","last_fullsized":"2024-09-16T09:42:13.678873+0000","mapping_epoch":454,"log_start":"461'1400","ondisk_log_start":"461'1400","created":454,"last_epoch_clean":455,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:11.976785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:41:11.976785+0000","last_clean_scrub_stamp":"2024-09-16T09:41:11.976785+0000","objects_scrubbed":0,"log_size":4248,"log_dups_size":0,"ondisk_log_size":4248,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:15.559115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":370147328,"num_objects":5648,"num_object_clones":0,"num_object_copies":16944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5648,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5648,"num_write_kb":361472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"73'79","reported_seq":585,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:07.528451+0000","last_change":"2024-09-16T09:34:31.929462+0000","last_active":"2024-09-16T09:42:07.528451+0000","last_peered":"2024-09-16T09:42:07.528451+0000","last_clean":"2024-09-16T09:42:07.528451+0000","last_became_active":"2024-09-16T09:33:30.249309+0000","last_became_peered":"2024-09-16T09:33:30.249309+0000","last_unstale":"2024-09-16T09:42:07.528451+0000","last_undegraded":"2024-09-16T09:42:07.528451+0000","last_fullsized":"2024-09-16T09:42:07.528451+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:34:31.929388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:34:31.929388+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:18.945206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.6","version":"285'24","reported_seq":924,"reported_epoch":476,"state":"active+clean","last_fresh":"2024-09-16T09:42:14.446358+0000","last_change":"2024-09-16T09:42:11.911983+0000","last_active":"2024-09-16T09:42:14.446358+0000","last_peered":"2024-09-16T09:42:14.446358+0000","last_clean":"2024-09-16T09:42:14.446358+0000","last_became_active":"2024-09-16T09:25:03.715315+0000","last_became_peered":"2024-09-16T09:25:03.715315+0000","last_unstale":"2024-09-16T09:42:14.446358+0000","last_undegraded":"2024-09-16T09:42:14.446358+0000","last_fullsized":"2024-09-16T09:42:14.446358+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'24","last_scrub_stamp":"2024-09-16T09:42:11.911889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:42:11.911889+0000","objects_scrubbed":6,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:32.028734+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":36,"num_read_kb":30,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"59'1","reported_seq":916,"reported_epoch":476,"state":"active+clean","last_fresh":"2024-09-16T09:42:14.444833+0000","last_change":"2024-09-16T09:41:37.223719+0000","last_active":"2024-09-16T09:42:14.444833+0000","last_peered":"2024-09-16T09:42:14.444833+0000","last_clean":"2024-09-16T09:42:14.444833+0000","last_became_active":"2024-09-16T09:31:09.269627+0000","last_became_peered":"2024-09-16T09:31:09.269627+0000","last_unstale":"2024-09-16T09:42:14.444833+0000","last_undegraded":"2024-09-16T09:42:14.444833+0000","last_fullsized":"2024-09-16T09:42:14.444833+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:41:37.223543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:41:37.223543+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:50.479230+0000","scrub_duration":61,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1986,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"67'5","reported_seq":1921,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:11.417223+0000","last_change":"2024-09-16T09:41:43.153303+0000","last_active":"2024-09-16T09:42:11.417223+0000","last_peered":"2024-09-16T09:42:11.417223+0000","last_clean":"2024-09-16T09:42:11.417223+0000","last_became_active":"2024-09-16T09:25:00.696546+0000","last_became_peered":"2024-09-16T09:25:00.696546+0000","last_unstale":"2024-09-16T09:42:11.417223+0000","last_undegraded":"2024-09-16T09:42:11.417223+0000","last_fullsized":"2024-09-16T09:42:11.417223+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'5","last_scrub_stamp":"2024-09-16T09:41:43.153174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:41:43.153174+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:45.073584+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0.0059770420000000001,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1024,"num_read_kb":1019,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":847,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:07.530118+0000","last_change":"2024-09-16T09:27:07.337170+0000","last_active":"2024-09-16T09:42:07.530118+0000","last_peered":"2024-09-16T09:42:07.530118+0000","last_clean":"2024-09-16T09:42:07.530118+0000","last_became_active":"2024-09-16T09:25:05.714704+0000","last_became_peered":"2024-09-16T09:25:05.714704+0000","last_unstale":"2024-09-16T09:42:07.530118+0000","last_undegraded":"2024-09-16T09:42:07.530118+0000","last_fullsized":"2024-09-16T09:42:07.530118+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:27:07.337124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:27:07.337124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:18.736447+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":790,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:11.738344+0000","last_change":"2024-09-16T09:42:11.738344+0000","last_active":"2024-09-16T09:42:11.738344+0000","last_peered":"2024-09-16T09:42:11.738344+0000","last_clean":"2024-09-16T09:42:11.738344+0000","last_became_active":"2024-09-16T09:33:30.245852+0000","last_became_peered":"2024-09-16T09:33:30.245852+0000","last_unstale":"2024-09-16T09:42:11.738344+0000","last_undegraded":"2024-09-16T09:42:11.738344+0000","last_fullsized":"2024-09-16T09:42:11.738344+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:42:11.738297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:42:11.738297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:23.216675+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.a","version":"474'5454","reported_seq":5504,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:12.899434+0000","last_change":"2024-09-16T09:41:13.002241+0000","last_active":"2024-09-16T09:42:12.899434+0000","last_peered":"2024-09-16T09:42:12.899434+0000","last_clean":"2024-09-16T09:42:12.899434+0000","last_became_active":"2024-09-16T09:41:13.001938+0000","last_became_peered":"2024-09-16T09:41:13.001938+0000","last_unstale":"2024-09-16T09:42:12.899434+0000","last_undegraded":"2024-09-16T09:42:12.899434+0000","last_fullsized":"2024-09-16T09:42:12.899434+0000","mapping_epoch":454,"log_start":"460'500","ondisk_log_start":"460'500","created":454,"last_epoch_clean":455,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:11.976785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:41:11.976785+0000","last_clean_scrub_stamp":"2024-09-16T09:41:11.976785+0000","objects_scrubbed":0,"log_size":4954,"log_dups_size":0,"ondisk_log_size":4954,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:19.474805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":357433344,"num_objects":5454,"num_object_clones":0,"num_object_copies":16362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5454,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5454,"num_write_kb":349056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"285'28","reported_seq":1124,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:07.529390+0000","last_change":"2024-09-16T09:33:16.891850+0000","last_active":"2024-09-16T09:42:07.529390+0000","last_peered":"2024-09-16T09:42:07.529390+0000","last_clean":"2024-09-16T09:42:07.529390+0000","last_became_active":"2024-09-16T09:25:03.714118+0000","last_became_peered":"2024-09-16T09:25:03.714118+0000","last_unstale":"2024-09-16T09:42:07.529390+0000","last_undegraded":"2024-09-16T09:42:07.529390+0000","last_fullsized":"2024-09-16T09:42:07.529390+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'16","last_scrub_stamp":"2024-09-16T09:33:16.891761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:33:16.891761+0000","objects_scrubbed":8,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:13.927146+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":43,"num_read_kb":33,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":730,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:07.528131+0000","last_change":"2024-09-16T09:41:09.792929+0000","last_active":"2024-09-16T09:42:07.528131+0000","last_peered":"2024-09-16T09:42:07.528131+0000","last_clean":"2024-09-16T09:42:07.528131+0000","last_became_active":"2024-09-16T09:31:09.270196+0000","last_became_peered":"2024-09-16T09:31:09.270196+0000","last_unstale":"2024-09-16T09:42:07.528131+0000","last_undegraded":"2024-09-16T09:42:07.528131+0000","last_fullsized":"2024-09-16T09:42:07.528131+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:09.792874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:41:09.792874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:38.425208+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"73'79","reported_seq":958,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:07.532516+0000","last_change":"2024-09-16T09:39:39.878540+0000","last_active":"2024-09-16T09:42:07.532516+0000","last_peered":"2024-09-16T09:42:07.532516+0000","last_clean":"2024-09-16T09:42:07.532516+0000","last_became_active":"2024-09-16T09:30:08.870394+0000","last_became_peered":"2024-09-16T09:30:08.870394+0000","last_unstale":"2024-09-16T09:42:07.532516+0000","last_undegraded":"2024-09-16T09:42:07.532516+0000","last_fullsized":"2024-09-16T09:42:07.532516+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:39:39.878410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:39:39.878410+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:56.170803+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":878,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:11.246777+0000","last_change":"2024-09-16T09:42:11.246777+0000","last_active":"2024-09-16T09:42:11.246777+0000","last_peered":"2024-09-16T09:42:11.246777+0000","last_clean":"2024-09-16T09:42:11.246777+0000","last_became_active":"2024-09-16T09:25:05.714341+0000","last_became_peered":"2024-09-16T09:25:05.714341+0000","last_unstale":"2024-09-16T09:42:11.246777+0000","last_undegraded":"2024-09-16T09:42:11.246777+0000","last_fullsized":"2024-09-16T09:42:11.246777+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:42:11.246697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:42:11.246697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:29.196378+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":867,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:07.530029+0000","last_change":"2024-09-16T09:41:38.660523+0000","last_active":"2024-09-16T09:42:07.530029+0000","last_peered":"2024-09-16T09:42:07.530029+0000","last_clean":"2024-09-16T09:42:07.530029+0000","last_became_active":"2024-09-16T09:25:07.714281+0000","last_became_peered":"2024-09-16T09:25:07.714281+0000","last_unstale":"2024-09-16T09:42:07.530029+0000","last_undegraded":"2024-09-16T09:42:07.530029+0000","last_fullsized":"2024-09-16T09:42:07.530029+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:38.660336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:41:38.660336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:44.303685+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1","version":"474'5472","reported_seq":5528,"reported_epoch":474,"state":"active+clean+scrubbing","last_fresh":"2024-09-16T09:42:12.974029+0000","last_change":"2024-09-16T09:42:12.824817+0000","last_active":"2024-09-16T09:42:12.974029+0000","last_peered":"2024-09-16T09:42:12.974029+0000","last_clean":"2024-09-16T09:42:12.974029+0000","last_became_active":"2024-09-16T09:41:13.003635+0000","last_became_peered":"2024-09-16T09:41:13.003635+0000","last_unstale":"2024-09-16T09:42:12.974029+0000","last_undegraded":"2024-09-16T09:42:12.974029+0000","last_fullsized":"2024-09-16T09:42:12.974029+0000","mapping_epoch":454,"log_start":"460'500","ondisk_log_start":"460'500","created":454,"last_epoch_clean":455,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:11.976785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:41:11.976785+0000","last_clean_scrub_stamp":"2024-09-16T09:41:11.976785+0000","objects_scrubbed":599,"log_size":4972,"log_dups_size":0,"ondisk_log_size":4972,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"scrubbing for 0s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358612992,"num_objects":5472,"num_object_clones":0,"num_object_copies":16416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5472,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5472,"num_write_kb":350208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.b","version":"73'79","reported_seq":707,"reported_epoch":476,"state":"active+clean","last_fresh":"2024-09-16T09:42:14.444131+0000","last_change":"2024-09-16T09:40:37.917868+0000","last_active":"2024-09-16T09:42:14.444131+0000","last_peered":"2024-09-16T09:42:14.444131+0000","last_clean":"2024-09-16T09:42:14.444131+0000","last_became_active":"2024-09-16T09:33:30.251933+0000","last_became_peered":"2024-09-16T09:33:30.251933+0000","last_unstale":"2024-09-16T09:42:14.444131+0000","last_undegraded":"2024-09-16T09:42:14.444131+0000","last_fullsized":"2024-09-16T09:42:14.444131+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:40:37.917691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:40:37.917691+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"285'19","reported_seq":956,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:07.530107+0000","last_change":"2024-09-16T09:41:03.243490+0000","last_active":"2024-09-16T09:42:07.530107+0000","last_peered":"2024-09-16T09:42:07.530107+0000","last_clean":"2024-09-16T09:42:07.530107+0000","last_became_active":"2024-09-16T09:37:33.348433+0000","last_became_peered":"2024-09-16T09:37:33.348433+0000","last_unstale":"2024-09-16T09:42:07.530107+0000","last_undegraded":"2024-09-16T09:42:07.530107+0000","last_fullsized":"2024-09-16T09:42:07.530107+0000","mapping_epoch":367,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":368,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'19","last_scrub_stamp":"2024-09-16T09:41:03.243406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:41:03.243406+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:30.272372+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":864,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:07.527255+0000","last_change":"2024-09-16T09:37:18.392256+0000","last_active":"2024-09-16T09:42:07.527255+0000","last_peered":"2024-09-16T09:42:07.527255+0000","last_clean":"2024-09-16T09:42:07.527255+0000","last_became_active":"2024-09-16T09:25:01.708270+0000","last_became_peered":"2024-09-16T09:25:01.708270+0000","last_unstale":"2024-09-16T09:42:07.527255+0000","last_undegraded":"2024-09-16T09:42:07.527255+0000","last_fullsized":"2024-09-16T09:42:07.527255+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:18.392084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:37:18.392084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:15.281480+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":847,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:07.529948+0000","last_change":"2024-09-16T09:26:34.171190+0000","last_active":"2024-09-16T09:42:07.529948+0000","last_peered":"2024-09-16T09:42:07.529948+0000","last_clean":"2024-09-16T09:42:07.529948+0000","last_became_active":"2024-09-16T09:25:05.711964+0000","last_became_peered":"2024-09-16T09:25:05.711964+0000","last_unstale":"2024-09-16T09:42:07.529948+0000","last_undegraded":"2024-09-16T09:42:07.529948+0000","last_fullsized":"2024-09-16T09:42:07.529948+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:26:34.171139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:26:34.171139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:16.797486+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":754,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:09.944417+0000","last_change":"2024-09-16T09:42:09.944417+0000","last_active":"2024-09-16T09:42:09.944417+0000","last_peered":"2024-09-16T09:42:09.944417+0000","last_clean":"2024-09-16T09:42:09.944417+0000","last_became_active":"2024-09-16T09:31:09.278618+0000","last_became_peered":"2024-09-16T09:31:09.278618+0000","last_unstale":"2024-09-16T09:42:09.944417+0000","last_undegraded":"2024-09-16T09:42:09.944417+0000","last_fullsized":"2024-09-16T09:42:09.944417+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:42:09.944311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:42:09.944311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:12.734923+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"475'5527","reported_seq":5579,"reported_epoch":475,"state":"active+clean","last_fresh":"2024-09-16T09:42:13.712024+0000","last_change":"2024-09-16T09:41:12.998092+0000","last_active":"2024-09-16T09:42:13.712024+0000","last_peered":"2024-09-16T09:42:13.712024+0000","last_clean":"2024-09-16T09:42:13.712024+0000","last_became_active":"2024-09-16T09:41:12.997802+0000","last_became_peered":"2024-09-16T09:41:12.997802+0000","last_unstale":"2024-09-16T09:42:13.712024+0000","last_undegraded":"2024-09-16T09:42:13.712024+0000","last_fullsized":"2024-09-16T09:42:13.712024+0000","mapping_epoch":454,"log_start":"461'1300","ondisk_log_start":"461'1300","created":454,"last_epoch_clean":455,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:11.976785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:41:11.976785+0000","last_clean_scrub_stamp":"2024-09-16T09:41:11.976785+0000","objects_scrubbed":0,"log_size":4227,"log_dups_size":0,"ondisk_log_size":4227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:20.435005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362217472,"num_objects":5527,"num_object_clones":0,"num_object_copies":16581,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5527,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5527,"num_write_kb":353728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.8","version":"73'79","reported_seq":215,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:07.525922+0000","last_change":"2024-09-16T09:41:31.767349+0000","last_active":"2024-09-16T09:42:07.525922+0000","last_peered":"2024-09-16T09:42:07.525922+0000","last_clean":"2024-09-16T09:42:07.525922+0000","last_became_active":"2024-09-16T09:37:53.418691+0000","last_became_peered":"2024-09-16T09:37:53.418691+0000","last_unstale":"2024-09-16T09:42:07.525922+0000","last_undegraded":"2024-09-16T09:42:07.525922+0000","last_fullsized":"2024-09-16T09:42:07.525922+0000","mapping_epoch":382,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":383,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:41:31.767299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:41:31.767299+0000","objects_scrubbed":1,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:51.743338+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262688,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"285'31","reported_seq":933,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:07.529527+0000","last_change":"2024-09-16T09:42:07.153346+0000","last_active":"2024-09-16T09:42:07.529527+0000","last_peered":"2024-09-16T09:42:07.529527+0000","last_clean":"2024-09-16T09:42:07.529527+0000","last_became_active":"2024-09-16T09:25:03.713358+0000","last_became_peered":"2024-09-16T09:25:03.713358+0000","last_unstale":"2024-09-16T09:42:07.529527+0000","last_undegraded":"2024-09-16T09:42:07.529527+0000","last_fullsized":"2024-09-16T09:42:07.529527+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'31","last_scrub_stamp":"2024-09-16T09:42:07.153276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:42:07.153276+0000","objects_scrubbed":8,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:21.444481+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":50,"num_read_kb":40,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":880,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:07.529280+0000","last_change":"2024-09-16T09:41:07.972297+0000","last_active":"2024-09-16T09:42:07.529280+0000","last_peered":"2024-09-16T09:42:07.529280+0000","last_clean":"2024-09-16T09:42:07.529280+0000","last_became_active":"2024-09-16T09:25:01.703355+0000","last_became_peered":"2024-09-16T09:25:01.703355+0000","last_unstale":"2024-09-16T09:42:07.529280+0000","last_undegraded":"2024-09-16T09:42:07.529280+0000","last_fullsized":"2024-09-16T09:42:07.529280+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:07.972246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:41:07.972246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:31.789728+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":511,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:07.526604+0000","last_change":"2024-09-16T09:37:46.378016+0000","last_active":"2024-09-16T09:42:07.526604+0000","last_peered":"2024-09-16T09:42:07.526604+0000","last_clean":"2024-09-16T09:42:07.526604+0000","last_became_active":"2024-09-16T09:33:30.251186+0000","last_became_peered":"2024-09-16T09:33:30.251186+0000","last_unstale":"2024-09-16T09:42:07.526604+0000","last_undegraded":"2024-09-16T09:42:07.526604+0000","last_fullsized":"2024-09-16T09:42:07.526604+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:46.377951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:37:46.377951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:22.269606+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,0],"acting":[4,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":811,"reported_epoch":475,"state":"active+clean","last_fresh":"2024-09-16T09:42:13.567773+0000","last_change":"2024-09-16T09:38:24.031758+0000","last_active":"2024-09-16T09:42:13.567773+0000","last_peered":"2024-09-16T09:42:13.567773+0000","last_clean":"2024-09-16T09:42:13.567773+0000","last_became_active":"2024-09-16T09:33:21.337291+0000","last_became_peered":"2024-09-16T09:33:21.337291+0000","last_unstale":"2024-09-16T09:42:13.567773+0000","last_undegraded":"2024-09-16T09:42:13.567773+0000","last_fullsized":"2024-09-16T09:42:13.567773+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:38:24.031592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:38:24.031592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:58.675217+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4],"acting":[3,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3","version":"474'5519","reported_seq":5569,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:12.974266+0000","last_change":"2024-09-16T09:41:13.139231+0000","last_active":"2024-09-16T09:42:12.974266+0000","last_peered":"2024-09-16T09:42:12.974266+0000","last_clean":"2024-09-16T09:42:12.974266+0000","last_became_active":"2024-09-16T09:41:13.138925+0000","last_became_peered":"2024-09-16T09:41:13.138925+0000","last_unstale":"2024-09-16T09:42:12.974266+0000","last_undegraded":"2024-09-16T09:42:12.974266+0000","last_fullsized":"2024-09-16T09:42:12.974266+0000","mapping_epoch":454,"log_start":"460'600","ondisk_log_start":"460'600","created":454,"last_epoch_clean":455,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:11.976785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:41:11.976785+0000","last_clean_scrub_stamp":"2024-09-16T09:41:11.976785+0000","objects_scrubbed":0,"log_size":4919,"log_dups_size":0,"ondisk_log_size":4919,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:33.187528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":361693184,"num_objects":5519,"num_object_clones":0,"num_object_copies":16557,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5519,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5519,"num_write_kb":353216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1],"acting":[6,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.9","version":"73'79","reported_seq":942,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:07.537053+0000","last_change":"2024-09-16T09:41:40.124011+0000","last_active":"2024-09-16T09:42:07.537053+0000","last_peered":"2024-09-16T09:42:07.537053+0000","last_clean":"2024-09-16T09:42:07.537053+0000","last_became_active":"2024-09-16T09:26:35.126066+0000","last_became_peered":"2024-09-16T09:26:35.126066+0000","last_unstale":"2024-09-16T09:42:07.537053+0000","last_undegraded":"2024-09-16T09:42:07.537053+0000","last_fullsized":"2024-09-16T09:42:07.537053+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:41:40.123937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:41:40.123937+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:04.683364+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"285'20","reported_seq":898,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:07.527680+0000","last_change":"2024-09-16T09:41:42.111004+0000","last_active":"2024-09-16T09:42:07.527680+0000","last_peered":"2024-09-16T09:42:07.527680+0000","last_clean":"2024-09-16T09:42:07.527680+0000","last_became_active":"2024-09-16T09:25:03.707723+0000","last_became_peered":"2024-09-16T09:25:03.707723+0000","last_unstale":"2024-09-16T09:42:07.527680+0000","last_undegraded":"2024-09-16T09:42:07.527680+0000","last_fullsized":"2024-09-16T09:42:07.527680+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'20","last_scrub_stamp":"2024-09-16T09:41:42.110923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:41:42.110923+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:46.525293+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":741,"reported_epoch":476,"state":"active+clean","last_fresh":"2024-09-16T09:42:14.444755+0000","last_change":"2024-09-16T09:39:38.508106+0000","last_active":"2024-09-16T09:42:14.444755+0000","last_peered":"2024-09-16T09:42:14.444755+0000","last_clean":"2024-09-16T09:42:14.444755+0000","last_became_active":"2024-09-16T09:31:09.288825+0000","last_became_peered":"2024-09-16T09:31:09.288825+0000","last_unstale":"2024-09-16T09:42:14.444755+0000","last_undegraded":"2024-09-16T09:42:14.444755+0000","last_fullsized":"2024-09-16T09:42:14.444755+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:39:38.507898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:39:38.507898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:50.039752+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"228'9","reported_seq":665,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:07.535115+0000","last_change":"2024-09-16T09:41:33.174507+0000","last_active":"2024-09-16T09:42:07.535115+0000","last_peered":"2024-09-16T09:42:07.535115+0000","last_clean":"2024-09-16T09:42:07.535115+0000","last_became_active":"2024-09-16T09:33:21.337213+0000","last_became_peered":"2024-09-16T09:33:21.337213+0000","last_unstale":"2024-09-16T09:42:07.535115+0000","last_undegraded":"2024-09-16T09:42:07.535115+0000","last_fullsized":"2024-09-16T09:42:07.535115+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"228'9","last_scrub_stamp":"2024-09-16T09:41:33.174444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:41:33.174444+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:44.705989+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":892,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:07.529880+0000","last_change":"2024-09-16T09:41:39.100268+0000","last_active":"2024-09-16T09:42:07.529880+0000","last_peered":"2024-09-16T09:42:07.529880+0000","last_clean":"2024-09-16T09:42:07.529880+0000","last_became_active":"2024-09-16T09:25:07.718958+0000","last_became_peered":"2024-09-16T09:25:07.718958+0000","last_unstale":"2024-09-16T09:42:07.529880+0000","last_undegraded":"2024-09-16T09:42:07.529880+0000","last_fullsized":"2024-09-16T09:42:07.529880+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:39.100089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:41:39.100089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:42.983457+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.4","version":"474'5530","reported_seq":5580,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:12.520802+0000","last_change":"2024-09-16T09:41:13.004443+0000","last_active":"2024-09-16T09:42:12.520802+0000","last_peered":"2024-09-16T09:42:12.520802+0000","last_clean":"2024-09-16T09:42:12.520802+0000","last_became_active":"2024-09-16T09:41:13.004032+0000","last_became_peered":"2024-09-16T09:41:13.004032+0000","last_unstale":"2024-09-16T09:42:12.520802+0000","last_undegraded":"2024-09-16T09:42:12.520802+0000","last_fullsized":"2024-09-16T09:42:12.520802+0000","mapping_epoch":454,"log_start":"460'1100","ondisk_log_start":"460'1100","created":454,"last_epoch_clean":455,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:11.976785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:41:11.976785+0000","last_clean_scrub_stamp":"2024-09-16T09:41:11.976785+0000","objects_scrubbed":0,"log_size":4430,"log_dups_size":0,"ondisk_log_size":4430,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:17.025591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362414080,"num_objects":5530,"num_object_clones":0,"num_object_copies":16590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5530,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5530,"num_write_kb":353920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.e","version":"73'79","reported_seq":669,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:07.529779+0000","last_change":"2024-09-16T09:35:41.979877+0000","last_active":"2024-09-16T09:42:07.529779+0000","last_peered":"2024-09-16T09:42:07.529779+0000","last_clean":"2024-09-16T09:42:07.529779+0000","last_became_active":"2024-09-16T09:30:10.705072+0000","last_became_peered":"2024-09-16T09:30:10.705072+0000","last_unstale":"2024-09-16T09:42:07.529779+0000","last_undegraded":"2024-09-16T09:42:07.529779+0000","last_fullsized":"2024-09-16T09:42:07.529779+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:35:41.979814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:35:41.979814+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:12.209303+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,3],"acting":[4,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.b","version":"285'15","reported_seq":930,"reported_epoch":476,"state":"active+clean","last_fresh":"2024-09-16T09:42:14.444495+0000","last_change":"2024-09-16T09:41:09.042482+0000","last_active":"2024-09-16T09:42:14.444495+0000","last_peered":"2024-09-16T09:42:14.444495+0000","last_clean":"2024-09-16T09:42:14.444495+0000","last_became_active":"2024-09-16T09:36:31.590202+0000","last_became_peered":"2024-09-16T09:36:31.590202+0000","last_unstale":"2024-09-16T09:42:14.444495+0000","last_undegraded":"2024-09-16T09:42:14.444495+0000","last_fullsized":"2024-09-16T09:42:14.444495+0000","mapping_epoch":316,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":317,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'15","last_scrub_stamp":"2024-09-16T09:41:09.039458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:41:09.039458+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:17.178456+0000","scrub_duration":41,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":877,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:07.527144+0000","last_change":"2024-09-16T09:37:31.033631+0000","last_active":"2024-09-16T09:42:07.527144+0000","last_peered":"2024-09-16T09:42:07.527144+0000","last_clean":"2024-09-16T09:42:07.527144+0000","last_became_active":"2024-09-16T09:31:09.276516+0000","last_became_peered":"2024-09-16T09:31:09.276516+0000","last_unstale":"2024-09-16T09:42:07.527144+0000","last_undegraded":"2024-09-16T09:42:07.527144+0000","last_fullsized":"2024-09-16T09:42:07.527144+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:31.033577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:37:31.033577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:20.735658+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":798,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:07.527031+0000","last_change":"2024-09-16T09:33:40.260547+0000","last_active":"2024-09-16T09:42:07.527031+0000","last_peered":"2024-09-16T09:42:07.527031+0000","last_clean":"2024-09-16T09:42:07.527031+0000","last_became_active":"2024-09-16T09:33:21.337254+0000","last_became_peered":"2024-09-16T09:33:21.337254+0000","last_unstale":"2024-09-16T09:42:07.527031+0000","last_undegraded":"2024-09-16T09:42:07.527031+0000","last_fullsized":"2024-09-16T09:42:07.527031+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:40.260500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:40.260500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:53.192937+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3],"acting":[4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":872,"reported_epoch":476,"state":"active+clean","last_fresh":"2024-09-16T09:42:14.444730+0000","last_change":"2024-09-16T09:36:53.647305+0000","last_active":"2024-09-16T09:42:14.444730+0000","last_peered":"2024-09-16T09:42:14.444730+0000","last_clean":"2024-09-16T09:42:14.444730+0000","last_became_active":"2024-09-16T09:25:07.718778+0000","last_became_peered":"2024-09-16T09:25:07.718778+0000","last_unstale":"2024-09-16T09:42:14.444730+0000","last_undegraded":"2024-09-16T09:42:14.444730+0000","last_fullsized":"2024-09-16T09:42:14.444730+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:53.647256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:36:53.647256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:17.963060+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.5","version":"474'5551","reported_seq":5601,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:12.533376+0000","last_change":"2024-09-16T09:41:13.004151+0000","last_active":"2024-09-16T09:42:12.533376+0000","last_peered":"2024-09-16T09:42:12.533376+0000","last_clean":"2024-09-16T09:42:12.533376+0000","last_became_active":"2024-09-16T09:41:13.003693+0000","last_became_peered":"2024-09-16T09:41:13.003693+0000","last_unstale":"2024-09-16T09:42:12.533376+0000","last_undegraded":"2024-09-16T09:42:12.533376+0000","last_fullsized":"2024-09-16T09:42:12.533376+0000","mapping_epoch":454,"log_start":"460'1200","ondisk_log_start":"460'1200","created":454,"last_epoch_clean":455,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:11.976785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:41:11.976785+0000","last_clean_scrub_stamp":"2024-09-16T09:41:11.976785+0000","objects_scrubbed":0,"log_size":4351,"log_dups_size":0,"ondisk_log_size":4351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:35.079186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":363790336,"num_objects":5551,"num_object_clones":0,"num_object_copies":16653,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5551,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5551,"num_write_kb":355264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.f","version":"73'79","reported_seq":937,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:07.528695+0000","last_change":"2024-09-16T09:41:28.953879+0000","last_active":"2024-09-16T09:42:07.528695+0000","last_peered":"2024-09-16T09:42:07.528695+0000","last_clean":"2024-09-16T09:42:07.528695+0000","last_became_active":"2024-09-16T09:30:10.693477+0000","last_became_peered":"2024-09-16T09:30:10.693477+0000","last_unstale":"2024-09-16T09:42:07.528695+0000","last_undegraded":"2024-09-16T09:42:07.528695+0000","last_fullsized":"2024-09-16T09:42:07.528695+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:41:28.953810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:41:28.953810+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:52.357805+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"285'35","reported_seq":931,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:07.529049+0000","last_change":"2024-09-16T09:37:28.078195+0000","last_active":"2024-09-16T09:42:07.529049+0000","last_peered":"2024-09-16T09:42:07.529049+0000","last_clean":"2024-09-16T09:42:07.529049+0000","last_became_active":"2024-09-16T09:25:03.712783+0000","last_became_peered":"2024-09-16T09:25:03.712783+0000","last_unstale":"2024-09-16T09:42:07.529049+0000","last_undegraded":"2024-09-16T09:42:07.529049+0000","last_fullsized":"2024-09-16T09:42:07.529049+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'35","last_scrub_stamp":"2024-09-16T09:37:28.077841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:37:28.077841+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:56.886841+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":56,"num_read_kb":45,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"59'1","reported_seq":883,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:07.535699+0000","last_change":"2024-09-16T09:37:56.528872+0000","last_active":"2024-09-16T09:42:07.535699+0000","last_peered":"2024-09-16T09:42:07.535699+0000","last_clean":"2024-09-16T09:42:07.535699+0000","last_became_active":"2024-09-16T09:25:01.708069+0000","last_became_peered":"2024-09-16T09:25:01.708069+0000","last_unstale":"2024-09-16T09:42:07.535699+0000","last_undegraded":"2024-09-16T09:42:07.535699+0000","last_fullsized":"2024-09-16T09:42:07.535699+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:37:56.528796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:37:56.528796+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:19.088387+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":903,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:07.527515+0000","last_change":"2024-09-16T09:40:39.696450+0000","last_active":"2024-09-16T09:42:07.527515+0000","last_peered":"2024-09-16T09:42:07.527515+0000","last_clean":"2024-09-16T09:42:07.527515+0000","last_became_active":"2024-09-16T09:37:53.518284+0000","last_became_peered":"2024-09-16T09:37:53.518284+0000","last_unstale":"2024-09-16T09:42:07.527515+0000","last_undegraded":"2024-09-16T09:42:07.527515+0000","last_fullsized":"2024-09-16T09:42:07.527515+0000","mapping_epoch":382,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":383,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:40:39.696383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:40:39.696383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:02.607251+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":849,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:07.532433+0000","last_change":"2024-09-16T09:36:48.649739+0000","last_active":"2024-09-16T09:42:07.532433+0000","last_peered":"2024-09-16T09:42:07.532433+0000","last_clean":"2024-09-16T09:42:07.532433+0000","last_became_active":"2024-09-16T09:25:07.718750+0000","last_became_peered":"2024-09-16T09:25:07.718750+0000","last_unstale":"2024-09-16T09:42:07.532433+0000","last_undegraded":"2024-09-16T09:42:07.532433+0000","last_fullsized":"2024-09-16T09:42:07.532433+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:48.649696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:36:48.649696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:13.079073+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.6","version":"476'5732","reported_seq":5786,"reported_epoch":476,"state":"active+clean","last_fresh":"2024-09-16T09:42:14.620164+0000","last_change":"2024-09-16T09:41:13.139206+0000","last_active":"2024-09-16T09:42:14.620164+0000","last_peered":"2024-09-16T09:42:14.620164+0000","last_clean":"2024-09-16T09:42:14.620164+0000","last_became_active":"2024-09-16T09:41:13.138910+0000","last_became_peered":"2024-09-16T09:41:13.138910+0000","last_unstale":"2024-09-16T09:42:14.620164+0000","last_undegraded":"2024-09-16T09:42:14.620164+0000","last_fullsized":"2024-09-16T09:42:14.620164+0000","mapping_epoch":454,"log_start":"459'300","ondisk_log_start":"459'300","created":454,"last_epoch_clean":455,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:11.976785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:41:11.976785+0000","last_clean_scrub_stamp":"2024-09-16T09:41:11.976785+0000","objects_scrubbed":0,"log_size":5432,"log_dups_size":0,"ondisk_log_size":5432,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:24.077359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":375652352,"num_objects":5732,"num_object_clones":0,"num_object_copies":17196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5732,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5732,"num_write_kb":366848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.c","version":"73'79","reported_seq":627,"reported_epoch":475,"state":"active+clean","last_fresh":"2024-09-16T09:42:13.567366+0000","last_change":"2024-09-16T09:42:09.710822+0000","last_active":"2024-09-16T09:42:13.567366+0000","last_peered":"2024-09-16T09:42:13.567366+0000","last_clean":"2024-09-16T09:42:13.567366+0000","last_became_active":"2024-09-16T09:31:09.273770+0000","last_became_peered":"2024-09-16T09:31:09.273770+0000","last_unstale":"2024-09-16T09:42:13.567366+0000","last_undegraded":"2024-09-16T09:42:13.567366+0000","last_fullsized":"2024-09-16T09:42:13.567366+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:42:09.710756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:42:09.710756+0000","objects_scrubbed":1,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:23.803158+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":327680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"285'18","reported_seq":946,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:07.527103+0000","last_change":"2024-09-16T09:36:50.603724+0000","last_active":"2024-09-16T09:42:07.527103+0000","last_peered":"2024-09-16T09:42:07.527103+0000","last_clean":"2024-09-16T09:42:07.527103+0000","last_became_active":"2024-09-16T09:31:09.269047+0000","last_became_peered":"2024-09-16T09:31:09.269047+0000","last_unstale":"2024-09-16T09:42:07.527103+0000","last_undegraded":"2024-09-16T09:42:07.527103+0000","last_fullsized":"2024-09-16T09:42:07.527103+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'18","last_scrub_stamp":"2024-09-16T09:36:50.603660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:36:50.603660+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:14.276752+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":877,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:07.532579+0000","last_change":"2024-09-16T09:37:57.529616+0000","last_active":"2024-09-16T09:42:07.532579+0000","last_peered":"2024-09-16T09:42:07.532579+0000","last_clean":"2024-09-16T09:42:07.532579+0000","last_became_active":"2024-09-16T09:25:01.707528+0000","last_became_peered":"2024-09-16T09:25:01.707528+0000","last_unstale":"2024-09-16T09:42:07.532579+0000","last_undegraded":"2024-09-16T09:42:07.532579+0000","last_fullsized":"2024-09-16T09:42:07.532579+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:57.529554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:37:57.529554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:25.132084+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":544,"reported_epoch":476,"state":"active+clean","last_fresh":"2024-09-16T09:42:14.442061+0000","last_change":"2024-09-16T09:42:12.922219+0000","last_active":"2024-09-16T09:42:14.442061+0000","last_peered":"2024-09-16T09:42:14.442061+0000","last_clean":"2024-09-16T09:42:14.442061+0000","last_became_active":"2024-09-16T09:33:21.337309+0000","last_became_peered":"2024-09-16T09:33:21.337309+0000","last_unstale":"2024-09-16T09:42:14.442061+0000","last_undegraded":"2024-09-16T09:42:14.442061+0000","last_fullsized":"2024-09-16T09:42:14.442061+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:42:12.922136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:42:12.922136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:19.988335+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":870,"reported_epoch":475,"state":"active+clean","last_fresh":"2024-09-16T09:42:13.566463+0000","last_change":"2024-09-16T09:38:21.097872+0000","last_active":"2024-09-16T09:42:13.566463+0000","last_peered":"2024-09-16T09:42:13.566463+0000","last_clean":"2024-09-16T09:42:13.566463+0000","last_became_active":"2024-09-16T09:25:07.718899+0000","last_became_peered":"2024-09-16T09:25:07.718899+0000","last_unstale":"2024-09-16T09:42:13.566463+0000","last_undegraded":"2024-09-16T09:42:13.566463+0000","last_fullsized":"2024-09-16T09:42:13.566463+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:38:21.097804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:38:21.097804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:59.651567+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.7","version":"474'5632","reported_seq":5681,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:13.297214+0000","last_change":"2024-09-16T09:41:13.139096+0000","last_active":"2024-09-16T09:42:13.297214+0000","last_peered":"2024-09-16T09:42:13.297214+0000","last_clean":"2024-09-16T09:42:13.297214+0000","last_became_active":"2024-09-16T09:41:13.138764+0000","last_became_peered":"2024-09-16T09:41:13.138764+0000","last_unstale":"2024-09-16T09:42:13.297214+0000","last_undegraded":"2024-09-16T09:42:13.297214+0000","last_fullsized":"2024-09-16T09:42:13.297214+0000","mapping_epoch":454,"log_start":"460'700","ondisk_log_start":"460'700","created":454,"last_epoch_clean":455,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:11.976785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:41:11.976785+0000","last_clean_scrub_stamp":"2024-09-16T09:41:11.976785+0000","objects_scrubbed":0,"log_size":4932,"log_dups_size":0,"ondisk_log_size":4932,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:40.396787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369098752,"num_objects":5632,"num_object_clones":0,"num_object_copies":16896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5632,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5632,"num_write_kb":360448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.d","version":"73'79","reported_seq":667,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:07.534827+0000","last_change":"2024-09-16T09:39:49.055357+0000","last_active":"2024-09-16T09:42:07.534827+0000","last_peered":"2024-09-16T09:42:07.534827+0000","last_clean":"2024-09-16T09:42:07.534827+0000","last_became_active":"2024-09-16T09:30:10.709958+0000","last_became_peered":"2024-09-16T09:30:10.709958+0000","last_unstale":"2024-09-16T09:42:07.534827+0000","last_undegraded":"2024-09-16T09:42:07.534827+0000","last_fullsized":"2024-09-16T09:42:07.534827+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:39:49.055292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:39:49.055292+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:00.227133+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"285'27","reported_seq":910,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:07.526024+0000","last_change":"2024-09-16T09:41:11.837814+0000","last_active":"2024-09-16T09:42:07.526024+0000","last_peered":"2024-09-16T09:42:07.526024+0000","last_clean":"2024-09-16T09:42:07.526024+0000","last_became_active":"2024-09-16T09:25:03.715733+0000","last_became_peered":"2024-09-16T09:25:03.715733+0000","last_unstale":"2024-09-16T09:42:07.526024+0000","last_undegraded":"2024-09-16T09:42:07.526024+0000","last_fullsized":"2024-09-16T09:42:07.526024+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'27","last_scrub_stamp":"2024-09-16T09:41:11.837756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:41:11.837756+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:27.113297+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":881,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:07.536647+0000","last_change":"2024-09-16T09:37:17.544132+0000","last_active":"2024-09-16T09:42:07.536647+0000","last_peered":"2024-09-16T09:42:07.536647+0000","last_clean":"2024-09-16T09:42:07.536647+0000","last_became_active":"2024-09-16T09:25:01.707372+0000","last_became_peered":"2024-09-16T09:25:01.707372+0000","last_unstale":"2024-09-16T09:42:07.536647+0000","last_undegraded":"2024-09-16T09:42:07.536647+0000","last_fullsized":"2024-09-16T09:42:07.536647+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-16T09:37:17.544069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:37:17.544069+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:14.037434+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"66'8","reported_seq":1092,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:07.536649+0000","last_change":"2024-09-16T09:41:05.060541+0000","last_active":"2024-09-16T09:42:07.536649+0000","last_peered":"2024-09-16T09:42:07.536649+0000","last_clean":"2024-09-16T09:42:07.536649+0000","last_became_active":"2024-09-16T09:25:05.716189+0000","last_became_peered":"2024-09-16T09:25:05.716189+0000","last_unstale":"2024-09-16T09:42:07.536649+0000","last_undegraded":"2024-09-16T09:42:07.536649+0000","last_fullsized":"2024-09-16T09:42:07.536649+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:41:05.060488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:41:05.060488+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":851,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:07.527693+0000","last_change":"2024-09-16T09:37:47.244099+0000","last_active":"2024-09-16T09:42:07.527693+0000","last_peered":"2024-09-16T09:42:07.527693+0000","last_clean":"2024-09-16T09:42:07.527693+0000","last_became_active":"2024-09-16T09:25:07.717213+0000","last_became_peered":"2024-09-16T09:25:07.717213+0000","last_unstale":"2024-09-16T09:42:07.527693+0000","last_undegraded":"2024-09-16T09:42:07.527693+0000","last_fullsized":"2024-09-16T09:42:07.527693+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:47.244032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:37:47.244032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:19.690229+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.12","version":"73'79","reported_seq":677,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:07.527848+0000","last_change":"2024-09-16T09:40:38.120017+0000","last_active":"2024-09-16T09:42:07.527848+0000","last_peered":"2024-09-16T09:42:07.527848+0000","last_clean":"2024-09-16T09:42:07.527848+0000","last_became_active":"2024-09-16T09:30:10.731413+0000","last_became_peered":"2024-09-16T09:30:10.731413+0000","last_unstale":"2024-09-16T09:42:07.527848+0000","last_undegraded":"2024-09-16T09:42:07.527848+0000","last_fullsized":"2024-09-16T09:42:07.527848+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:40:38.119809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:40:38.119809+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":886,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:07.528967+0000","last_change":"2024-09-16T09:37:55.863661+0000","last_active":"2024-09-16T09:42:07.528967+0000","last_peered":"2024-09-16T09:42:07.528967+0000","last_clean":"2024-09-16T09:42:07.528967+0000","last_became_active":"2024-09-16T09:37:53.518354+0000","last_became_peered":"2024-09-16T09:37:53.518354+0000","last_unstale":"2024-09-16T09:42:07.528967+0000","last_undegraded":"2024-09-16T09:42:07.528967+0000","last_fullsized":"2024-09-16T09:42:07.528967+0000","mapping_epoch":382,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":383,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:55.863613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:37:55.863613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:58.876996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"285'12","reported_seq":647,"reported_epoch":475,"state":"active+clean","last_fresh":"2024-09-16T09:42:13.567975+0000","last_change":"2024-09-16T09:38:33.535691+0000","last_active":"2024-09-16T09:42:13.567975+0000","last_peered":"2024-09-16T09:42:13.567975+0000","last_clean":"2024-09-16T09:42:13.567975+0000","last_became_active":"2024-09-16T09:38:33.517889+0000","last_became_peered":"2024-09-16T09:38:33.517889+0000","last_unstale":"2024-09-16T09:42:13.567975+0000","last_undegraded":"2024-09-16T09:42:13.567975+0000","last_fullsized":"2024-09-16T09:42:13.567975+0000","mapping_epoch":393,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":394,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'12","last_scrub_stamp":"2024-09-16T09:37:20.749101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:37:20.749101+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:51.658883+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":17,"num_read_kb":14,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":871,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:07.527363+0000","last_change":"2024-09-16T09:40:28.258952+0000","last_active":"2024-09-16T09:42:07.527363+0000","last_peered":"2024-09-16T09:42:07.527363+0000","last_clean":"2024-09-16T09:42:07.527363+0000","last_became_active":"2024-09-16T09:25:05.715616+0000","last_became_peered":"2024-09-16T09:25:05.715616+0000","last_unstale":"2024-09-16T09:42:07.527363+0000","last_undegraded":"2024-09-16T09:42:07.527363+0000","last_fullsized":"2024-09-16T09:42:07.527363+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:40:28.258907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:40:28.258907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:03.671149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":865,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:07.534930+0000","last_change":"2024-09-16T09:40:37.040258+0000","last_active":"2024-09-16T09:42:07.534930+0000","last_peered":"2024-09-16T09:42:07.534930+0000","last_clean":"2024-09-16T09:42:07.534930+0000","last_became_active":"2024-09-16T09:25:07.721059+0000","last_became_peered":"2024-09-16T09:25:07.721059+0000","last_unstale":"2024-09-16T09:42:07.534930+0000","last_undegraded":"2024-09-16T09:42:07.534930+0000","last_fullsized":"2024-09-16T09:42:07.534930+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:40:37.040205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:40:37.040205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.13","version":"73'79","reported_seq":764,"reported_epoch":475,"state":"active+clean","last_fresh":"2024-09-16T09:42:13.566584+0000","last_change":"2024-09-16T09:40:30.696059+0000","last_active":"2024-09-16T09:42:13.566584+0000","last_peered":"2024-09-16T09:42:13.566584+0000","last_clean":"2024-09-16T09:42:13.566584+0000","last_became_active":"2024-09-16T09:33:30.248497+0000","last_became_peered":"2024-09-16T09:33:30.248497+0000","last_unstale":"2024-09-16T09:42:13.566584+0000","last_undegraded":"2024-09-16T09:42:13.566584+0000","last_fullsized":"2024-09-16T09:42:13.566584+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:40:30.695866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:40:30.695866+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:54.647315+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"285'15","reported_seq":955,"reported_epoch":476,"state":"active+clean","last_fresh":"2024-09-16T09:42:14.446448+0000","last_change":"2024-09-16T09:39:46.504126+0000","last_active":"2024-09-16T09:42:14.446448+0000","last_peered":"2024-09-16T09:42:14.446448+0000","last_clean":"2024-09-16T09:42:14.446448+0000","last_became_active":"2024-09-16T09:31:09.273498+0000","last_became_peered":"2024-09-16T09:31:09.273498+0000","last_unstale":"2024-09-16T09:42:14.446448+0000","last_undegraded":"2024-09-16T09:42:14.446448+0000","last_fullsized":"2024-09-16T09:42:14.446448+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'15","last_scrub_stamp":"2024-09-16T09:39:46.504045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:39:46.504045+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:52.043358+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":885,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:07.540111+0000","last_change":"2024-09-16T09:41:36.151500+0000","last_active":"2024-09-16T09:42:07.540111+0000","last_peered":"2024-09-16T09:42:07.540111+0000","last_clean":"2024-09-16T09:42:07.540111+0000","last_became_active":"2024-09-16T09:25:01.706491+0000","last_became_peered":"2024-09-16T09:25:01.706491+0000","last_unstale":"2024-09-16T09:42:07.540111+0000","last_undegraded":"2024-09-16T09:42:07.540111+0000","last_fullsized":"2024-09-16T09:42:07.540111+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:36.151432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:41:36.151432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:53.930320+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":650,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:08.180799+0000","last_change":"2024-09-16T09:42:08.180799+0000","last_active":"2024-09-16T09:42:08.180799+0000","last_peered":"2024-09-16T09:42:08.180799+0000","last_clean":"2024-09-16T09:42:08.180799+0000","last_became_active":"2024-09-16T09:33:30.248992+0000","last_became_peered":"2024-09-16T09:33:30.248992+0000","last_unstale":"2024-09-16T09:42:08.180799+0000","last_undegraded":"2024-09-16T09:42:08.180799+0000","last_fullsized":"2024-09-16T09:42:08.180799+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:42:08.180735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:42:08.180735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:09.054380+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":877,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:07.527757+0000","last_change":"2024-09-16T09:40:35.646704+0000","last_active":"2024-09-16T09:42:07.527757+0000","last_peered":"2024-09-16T09:42:07.527757+0000","last_clean":"2024-09-16T09:42:07.527757+0000","last_became_active":"2024-09-16T09:25:07.722050+0000","last_became_peered":"2024-09-16T09:25:07.722050+0000","last_unstale":"2024-09-16T09:42:07.527757+0000","last_undegraded":"2024-09-16T09:42:07.527757+0000","last_fullsized":"2024-09-16T09:42:07.527757+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:40:35.646535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:40:35.646535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:53.543871+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.10","version":"73'79","reported_seq":677,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:07.525789+0000","last_change":"2024-09-16T09:41:08.828451+0000","last_active":"2024-09-16T09:42:07.525789+0000","last_peered":"2024-09-16T09:42:07.525789+0000","last_clean":"2024-09-16T09:42:07.525789+0000","last_became_active":"2024-09-16T09:30:10.692770+0000","last_became_peered":"2024-09-16T09:30:10.692770+0000","last_unstale":"2024-09-16T09:42:07.525789+0000","last_undegraded":"2024-09-16T09:42:07.525789+0000","last_fullsized":"2024-09-16T09:42:07.525789+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:41:08.828394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:41:08.828394+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:19.530053+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"285'15","reported_seq":900,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:07.527340+0000","last_change":"2024-09-16T09:41:40.700560+0000","last_active":"2024-09-16T09:42:07.527340+0000","last_peered":"2024-09-16T09:42:07.527340+0000","last_clean":"2024-09-16T09:42:07.527340+0000","last_became_active":"2024-09-16T09:25:03.717310+0000","last_became_peered":"2024-09-16T09:25:03.717310+0000","last_unstale":"2024-09-16T09:42:07.527340+0000","last_undegraded":"2024-09-16T09:42:07.527340+0000","last_fullsized":"2024-09-16T09:42:07.527340+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'15","last_scrub_stamp":"2024-09-16T09:41:40.700318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:41:40.700318+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:44.976002+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":914,"reported_epoch":476,"state":"active+clean","last_fresh":"2024-09-16T09:42:14.444042+0000","last_change":"2024-09-16T09:40:23.065431+0000","last_active":"2024-09-16T09:42:14.444042+0000","last_peered":"2024-09-16T09:42:14.444042+0000","last_clean":"2024-09-16T09:42:14.444042+0000","last_became_active":"2024-09-16T09:31:09.272848+0000","last_became_peered":"2024-09-16T09:31:09.272848+0000","last_unstale":"2024-09-16T09:42:14.444042+0000","last_undegraded":"2024-09-16T09:42:14.444042+0000","last_fullsized":"2024-09-16T09:42:14.444042+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:40:23.065365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:40:23.065365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"187'11","reported_seq":921,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:07.533947+0000","last_change":"2024-09-16T09:37:19.540982+0000","last_active":"2024-09-16T09:42:07.533947+0000","last_peered":"2024-09-16T09:42:07.533947+0000","last_clean":"2024-09-16T09:42:07.533947+0000","last_became_active":"2024-09-16T09:31:09.277688+0000","last_became_peered":"2024-09-16T09:31:09.277688+0000","last_unstale":"2024-09-16T09:42:07.533947+0000","last_undegraded":"2024-09-16T09:42:07.533947+0000","last_fullsized":"2024-09-16T09:42:07.533947+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"187'11","last_scrub_stamp":"2024-09-16T09:37:19.540886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:37:19.540886+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:17.028744+0000","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":892,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:07.527370+0000","last_change":"2024-09-16T09:41:36.098715+0000","last_active":"2024-09-16T09:42:07.527370+0000","last_peered":"2024-09-16T09:42:07.527370+0000","last_clean":"2024-09-16T09:42:07.527370+0000","last_became_active":"2024-09-16T09:25:07.718810+0000","last_became_peered":"2024-09-16T09:25:07.718810+0000","last_unstale":"2024-09-16T09:42:07.527370+0000","last_undegraded":"2024-09-16T09:42:07.527370+0000","last_fullsized":"2024-09-16T09:42:07.527370+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:36.098648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:41:36.098648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:43.824638+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.11","version":"73'79","reported_seq":759,"reported_epoch":476,"state":"active+clean","last_fresh":"2024-09-16T09:42:14.444935+0000","last_change":"2024-09-16T09:37:15.623104+0000","last_active":"2024-09-16T09:42:14.444935+0000","last_peered":"2024-09-16T09:42:14.444935+0000","last_clean":"2024-09-16T09:42:14.444935+0000","last_became_active":"2024-09-16T09:31:09.272208+0000","last_became_peered":"2024-09-16T09:31:09.272208+0000","last_unstale":"2024-09-16T09:42:14.444935+0000","last_undegraded":"2024-09-16T09:42:14.444935+0000","last_fullsized":"2024-09-16T09:42:14.444935+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:37:15.623041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:37:15.623041+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:14.936019+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.14","version":"285'20","reported_seq":567,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:07.532665+0000","last_change":"2024-09-16T09:40:35.027221+0000","last_active":"2024-09-16T09:42:07.532665+0000","last_peered":"2024-09-16T09:42:07.532665+0000","last_clean":"2024-09-16T09:42:07.532665+0000","last_became_active":"2024-09-16T09:33:30.249769+0000","last_became_peered":"2024-09-16T09:33:30.249769+0000","last_unstale":"2024-09-16T09:42:07.532665+0000","last_undegraded":"2024-09-16T09:42:07.532665+0000","last_fullsized":"2024-09-16T09:42:07.532665+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'20","last_scrub_stamp":"2024-09-16T09:40:35.027096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:40:35.027096+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:04.153892+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":877,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:07.531973+0000","last_change":"2024-09-16T09:41:08.005775+0000","last_active":"2024-09-16T09:42:07.531973+0000","last_peered":"2024-09-16T09:42:07.531973+0000","last_clean":"2024-09-16T09:42:07.531973+0000","last_became_active":"2024-09-16T09:25:01.707513+0000","last_became_peered":"2024-09-16T09:25:01.707513+0000","last_unstale":"2024-09-16T09:42:07.531973+0000","last_undegraded":"2024-09-16T09:42:07.531973+0000","last_fullsized":"2024-09-16T09:42:07.531973+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:08.005728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:41:08.005728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:35.931885+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"66'8","reported_seq":1102,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:07.527106+0000","last_change":"2024-09-16T09:41:42.773433+0000","last_active":"2024-09-16T09:42:07.527106+0000","last_peered":"2024-09-16T09:42:07.527106+0000","last_clean":"2024-09-16T09:42:07.527106+0000","last_became_active":"2024-09-16T09:25:05.713833+0000","last_became_peered":"2024-09-16T09:25:05.713833+0000","last_unstale":"2024-09-16T09:42:07.527106+0000","last_undegraded":"2024-09-16T09:42:07.527106+0000","last_fullsized":"2024-09-16T09:42:07.527106+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:41:42.773359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:41:42.773359+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:42.814428+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":902,"reported_epoch":476,"state":"active+clean","last_fresh":"2024-09-16T09:42:14.443233+0000","last_change":"2024-09-16T09:40:26.019049+0000","last_active":"2024-09-16T09:42:14.443233+0000","last_peered":"2024-09-16T09:42:14.443233+0000","last_clean":"2024-09-16T09:42:14.443233+0000","last_became_active":"2024-09-16T09:31:09.275168+0000","last_became_peered":"2024-09-16T09:31:09.275168+0000","last_unstale":"2024-09-16T09:42:14.443233+0000","last_undegraded":"2024-09-16T09:42:14.443233+0000","last_fullsized":"2024-09-16T09:42:14.443233+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:40:26.018988+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:40:26.018988+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"285'19","reported_seq":910,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:07.527756+0000","last_change":"2024-09-16T09:37:31.362985+0000","last_active":"2024-09-16T09:42:07.527756+0000","last_peered":"2024-09-16T09:42:07.527756+0000","last_clean":"2024-09-16T09:42:07.527756+0000","last_became_active":"2024-09-16T09:26:33.291265+0000","last_became_peered":"2024-09-16T09:26:33.291265+0000","last_unstale":"2024-09-16T09:42:07.527756+0000","last_undegraded":"2024-09-16T09:42:07.527756+0000","last_fullsized":"2024-09-16T09:42:07.527756+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'19","last_scrub_stamp":"2024-09-16T09:37:31.362927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:37:31.362927+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:16.270309+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":896,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:07.527877+0000","last_change":"2024-09-16T09:41:29.092238+0000","last_active":"2024-09-16T09:42:07.527877+0000","last_peered":"2024-09-16T09:42:07.527877+0000","last_clean":"2024-09-16T09:42:07.527877+0000","last_became_active":"2024-09-16T09:25:01.705158+0000","last_became_peered":"2024-09-16T09:25:01.705158+0000","last_unstale":"2024-09-16T09:42:07.527877+0000","last_undegraded":"2024-09-16T09:42:07.527877+0000","last_fullsized":"2024-09-16T09:42:07.527877+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:29.091987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:41:29.091987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:56.880823+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":815,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:07.527772+0000","last_change":"2024-09-16T09:41:37.694845+0000","last_active":"2024-09-16T09:42:07.527772+0000","last_peered":"2024-09-16T09:42:07.527772+0000","last_clean":"2024-09-16T09:42:07.527772+0000","last_became_active":"2024-09-16T09:37:41.515278+0000","last_became_peered":"2024-09-16T09:37:41.515278+0000","last_unstale":"2024-09-16T09:42:07.527772+0000","last_undegraded":"2024-09-16T09:42:07.527772+0000","last_fullsized":"2024-09-16T09:42:07.527772+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:37.694724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:41:37.694724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:43.549832+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":726,"reported_epoch":476,"state":"active+clean","last_fresh":"2024-09-16T09:42:14.444479+0000","last_change":"2024-09-16T09:41:05.972422+0000","last_active":"2024-09-16T09:42:14.444479+0000","last_peered":"2024-09-16T09:42:14.444479+0000","last_clean":"2024-09-16T09:42:14.444479+0000","last_became_active":"2024-09-16T09:31:09.276042+0000","last_became_peered":"2024-09-16T09:31:09.276042+0000","last_unstale":"2024-09-16T09:42:14.444479+0000","last_undegraded":"2024-09-16T09:42:14.444479+0000","last_fullsized":"2024-09-16T09:42:14.444479+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:05.972363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:41:05.972363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"285'15","reported_seq":598,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:07.529187+0000","last_change":"2024-09-16T09:41:09.949578+0000","last_active":"2024-09-16T09:42:07.529187+0000","last_peered":"2024-09-16T09:42:07.529187+0000","last_clean":"2024-09-16T09:42:07.529187+0000","last_became_active":"2024-09-16T09:37:33.348223+0000","last_became_peered":"2024-09-16T09:37:33.348223+0000","last_unstale":"2024-09-16T09:42:07.529187+0000","last_undegraded":"2024-09-16T09:42:07.529187+0000","last_fullsized":"2024-09-16T09:42:07.529187+0000","mapping_epoch":367,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":368,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'15","last_scrub_stamp":"2024-09-16T09:41:09.949519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:41:09.949519+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:35.859726+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,2],"acting":[6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":877,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:07.535247+0000","last_change":"2024-09-16T09:41:44.170251+0000","last_active":"2024-09-16T09:42:07.535247+0000","last_peered":"2024-09-16T09:42:07.535247+0000","last_clean":"2024-09-16T09:42:07.535247+0000","last_became_active":"2024-09-16T09:25:01.706762+0000","last_became_peered":"2024-09-16T09:25:01.706762+0000","last_unstale":"2024-09-16T09:42:07.535247+0000","last_undegraded":"2024-09-16T09:42:07.535247+0000","last_fullsized":"2024-09-16T09:42:07.535247+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:44.170186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:41:44.170186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:51.467157+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":719,"reported_epoch":476,"state":"active+clean","last_fresh":"2024-09-16T09:42:14.444260+0000","last_change":"2024-09-16T09:41:36.130305+0000","last_active":"2024-09-16T09:42:14.444260+0000","last_peered":"2024-09-16T09:42:14.444260+0000","last_clean":"2024-09-16T09:42:14.444260+0000","last_became_active":"2024-09-16T09:28:52.982865+0000","last_became_peered":"2024-09-16T09:28:52.982865+0000","last_unstale":"2024-09-16T09:42:14.444260+0000","last_undegraded":"2024-09-16T09:42:14.444260+0000","last_fullsized":"2024-09-16T09:42:14.444260+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:36.130240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:41:36.130240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:05.802731+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":864,"reported_epoch":476,"state":"active+clean","last_fresh":"2024-09-16T09:42:14.442022+0000","last_change":"2024-09-16T09:34:50.779241+0000","last_active":"2024-09-16T09:42:14.442022+0000","last_peered":"2024-09-16T09:42:14.442022+0000","last_clean":"2024-09-16T09:42:14.442022+0000","last_became_active":"2024-09-16T09:25:07.715301+0000","last_became_peered":"2024-09-16T09:25:07.715301+0000","last_unstale":"2024-09-16T09:42:14.442022+0000","last_undegraded":"2024-09-16T09:42:14.442022+0000","last_fullsized":"2024-09-16T09:42:14.442022+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:50.779164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:34:50.779164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:04.743563+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.14","version":"73'79","reported_seq":686,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:07.527717+0000","last_change":"2024-09-16T09:41:38.531852+0000","last_active":"2024-09-16T09:42:07.527717+0000","last_peered":"2024-09-16T09:42:07.527717+0000","last_clean":"2024-09-16T09:42:07.527717+0000","last_became_active":"2024-09-16T09:30:10.700866+0000","last_became_peered":"2024-09-16T09:30:10.700866+0000","last_unstale":"2024-09-16T09:42:07.527717+0000","last_undegraded":"2024-09-16T09:42:07.527717+0000","last_fullsized":"2024-09-16T09:42:07.527717+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:41:38.531780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:41:38.531780+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:55.556470+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"285'19","reported_seq":756,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:07.539971+0000","last_change":"2024-09-16T09:39:28.862654+0000","last_active":"2024-09-16T09:42:07.539971+0000","last_peered":"2024-09-16T09:42:07.539971+0000","last_clean":"2024-09-16T09:42:07.539971+0000","last_became_active":"2024-09-16T09:28:52.971132+0000","last_became_peered":"2024-09-16T09:28:52.971132+0000","last_unstale":"2024-09-16T09:42:07.539971+0000","last_undegraded":"2024-09-16T09:42:07.539971+0000","last_fullsized":"2024-09-16T09:42:07.539971+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'19","last_scrub_stamp":"2024-09-16T09:39:28.862503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:39:28.862503+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:57.153062+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":887,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:07.527491+0000","last_change":"2024-09-16T09:41:47.768655+0000","last_active":"2024-09-16T09:42:07.527491+0000","last_peered":"2024-09-16T09:42:07.527491+0000","last_clean":"2024-09-16T09:42:07.527491+0000","last_became_active":"2024-09-16T09:25:01.700733+0000","last_became_peered":"2024-09-16T09:25:01.700733+0000","last_unstale":"2024-09-16T09:42:07.527491+0000","last_undegraded":"2024-09-16T09:42:07.527491+0000","last_fullsized":"2024-09-16T09:42:07.527491+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:47.768574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:41:47.768574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:13.312512+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":877,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:07.531843+0000","last_change":"2024-09-16T09:40:39.969447+0000","last_active":"2024-09-16T09:42:07.531843+0000","last_peered":"2024-09-16T09:42:07.531843+0000","last_clean":"2024-09-16T09:42:07.531843+0000","last_became_active":"2024-09-16T09:25:05.713479+0000","last_became_peered":"2024-09-16T09:25:05.713479+0000","last_unstale":"2024-09-16T09:42:07.531843+0000","last_undegraded":"2024-09-16T09:42:07.531843+0000","last_fullsized":"2024-09-16T09:42:07.531843+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:40:39.969383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:40:39.969383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":742,"reported_epoch":476,"state":"active+clean","last_fresh":"2024-09-16T09:42:14.444674+0000","last_change":"2024-09-16T09:41:34.178100+0000","last_active":"2024-09-16T09:42:14.444674+0000","last_peered":"2024-09-16T09:42:14.444674+0000","last_clean":"2024-09-16T09:42:14.444674+0000","last_became_active":"2024-09-16T09:31:09.274797+0000","last_became_peered":"2024-09-16T09:31:09.274797+0000","last_unstale":"2024-09-16T09:42:14.444674+0000","last_undegraded":"2024-09-16T09:42:14.444674+0000","last_fullsized":"2024-09-16T09:42:14.444674+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:34.178020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:41:34.178020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:55.711514+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"285'8","reported_seq":643,"reported_epoch":475,"state":"active+clean","last_fresh":"2024-09-16T09:42:13.566973+0000","last_change":"2024-09-16T09:41:04.769967+0000","last_active":"2024-09-16T09:42:13.566973+0000","last_peered":"2024-09-16T09:42:13.566973+0000","last_clean":"2024-09-16T09:42:13.566973+0000","last_became_active":"2024-09-16T09:31:09.273682+0000","last_became_peered":"2024-09-16T09:31:09.273682+0000","last_unstale":"2024-09-16T09:42:13.566973+0000","last_undegraded":"2024-09-16T09:42:13.566973+0000","last_fullsized":"2024-09-16T09:42:13.566973+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'8","last_scrub_stamp":"2024-09-16T09:41:04.769863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:41:04.769863+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:18.575428+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":9,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":914,"reported_epoch":476,"state":"active+clean","last_fresh":"2024-09-16T09:42:14.443688+0000","last_change":"2024-09-16T09:41:45.045179+0000","last_active":"2024-09-16T09:42:14.443688+0000","last_peered":"2024-09-16T09:42:14.443688+0000","last_clean":"2024-09-16T09:42:14.443688+0000","last_became_active":"2024-09-16T09:31:09.274719+0000","last_became_peered":"2024-09-16T09:31:09.274719+0000","last_unstale":"2024-09-16T09:42:14.443688+0000","last_undegraded":"2024-09-16T09:42:14.443688+0000","last_fullsized":"2024-09-16T09:42:14.443688+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:45.045117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:41:45.045117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:56.298444+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":888,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:07.528226+0000","last_change":"2024-09-16T09:38:35.092025+0000","last_active":"2024-09-16T09:42:07.528226+0000","last_peered":"2024-09-16T09:42:07.528226+0000","last_clean":"2024-09-16T09:42:07.528226+0000","last_became_active":"2024-09-16T09:25:05.712545+0000","last_became_peered":"2024-09-16T09:25:05.712545+0000","last_unstale":"2024-09-16T09:42:07.528226+0000","last_undegraded":"2024-09-16T09:42:07.528226+0000","last_fullsized":"2024-09-16T09:42:07.528226+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:38:35.091983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:38:35.091983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:14.921045+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":873,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:07.531530+0000","last_change":"2024-09-16T09:41:29.261932+0000","last_active":"2024-09-16T09:42:07.531530+0000","last_peered":"2024-09-16T09:42:07.531530+0000","last_clean":"2024-09-16T09:42:07.531530+0000","last_became_active":"2024-09-16T09:25:07.720117+0000","last_became_peered":"2024-09-16T09:25:07.720117+0000","last_unstale":"2024-09-16T09:42:07.531530+0000","last_undegraded":"2024-09-16T09:42:07.531530+0000","last_fullsized":"2024-09-16T09:42:07.531530+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:29.261841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:41:29.261841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:57.269436+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":503,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:07.527202+0000","last_change":"2024-09-16T09:36:52.091335+0000","last_active":"2024-09-16T09:42:07.527202+0000","last_peered":"2024-09-16T09:42:07.527202+0000","last_clean":"2024-09-16T09:42:07.527202+0000","last_became_active":"2024-09-16T09:33:30.248036+0000","last_became_peered":"2024-09-16T09:33:30.248036+0000","last_unstale":"2024-09-16T09:42:07.527202+0000","last_undegraded":"2024-09-16T09:42:07.527202+0000","last_fullsized":"2024-09-16T09:42:07.527202+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:52.088284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:36:52.088284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:53.777831+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":733,"reported_epoch":476,"state":"active+clean","last_fresh":"2024-09-16T09:42:14.442607+0000","last_change":"2024-09-16T09:37:18.825035+0000","last_active":"2024-09-16T09:42:14.442607+0000","last_peered":"2024-09-16T09:42:14.442607+0000","last_clean":"2024-09-16T09:42:14.442607+0000","last_became_active":"2024-09-16T09:31:09.271662+0000","last_became_peered":"2024-09-16T09:31:09.271662+0000","last_unstale":"2024-09-16T09:42:14.442607+0000","last_undegraded":"2024-09-16T09:42:14.442607+0000","last_fullsized":"2024-09-16T09:42:14.442607+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:18.824937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:37:18.824937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:27.104501+0000","scrub_duration":79,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"285'19","reported_seq":908,"reported_epoch":474,"state":"active+clean","last_fresh":"2024-09-16T09:42:07.528778+0000","last_change":"2024-09-16T09:37:25.995663+0000","last_active":"2024-09-16T09:42:07.528778+0000","last_peered":"2024-09-16T09:42:07.528778+0000","last_clean":"2024-09-16T09:42:07.528778+0000","last_became_active":"2024-09-16T09:25:03.707769+0000","last_became_peered":"2024-09-16T09:25:03.707769+0000","last_unstale":"2024-09-16T09:42:07.528778+0000","last_undegraded":"2024-09-16T09:42:07.528778+0000","last_fullsized":"2024-09-16T09:42:07.528778+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'19","last_scrub_stamp":"2024-09-16T09:37:25.995607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:37:25.995607+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:55.934178+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":877,"reported_epoch":476,"state":"active+clean","last_fresh":"2024-09-16T09:42:14.446107+0000","last_change":"2024-09-16T09:41:29.136632+0000","last_active":"2024-09-16T09:42:14.446107+0000","last_peered":"2024-09-16T09:42:14.446107+0000","last_clean":"2024-09-16T09:42:14.446107+0000","last_became_active":"2024-09-16T09:25:05.711952+0000","last_became_peered":"2024-09-16T09:25:05.711952+0000","last_unstale":"2024-09-16T09:42:14.446107+0000","last_undegraded":"2024-09-16T09:42:14.446107+0000","last_fullsized":"2024-09-16T09:42:14.446107+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:29.136452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:41:29.136452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:51.188109+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":16,"stat_sum":{"num_bytes":5831720960,"num_objects":88985,"num_object_clones":0,"num_object_copies":266955,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88985,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":88985,"num_write_kb":5695040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":17551130624,"data_stored":17551130624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74885,"ondisk_log_size":74885,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":1117,"num_read_kb":874,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":3686,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":681,"ondisk_log_size":681,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1986,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1024,"num_read_kb":1019,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":21,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":327680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1659,"ondisk_log_size":1659,"up":63,"acting":63,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266899,"num_pgs":69,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2847008,"kb_used_data":2121680,"kb_used_omap":1,"kb_used_meta":725310,"kb_avail":90898144,"statfs":{"total":95995035648,"available":93079699456,"internally_reserved":0,"allocated":2172600320,"data_stored":2172083542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":742717898},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495832,"num_pgs":61,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3476612,"kb_used_data":2827636,"kb_used_omap":1,"kb_used_meta":648958,"kb_avail":90268540,"statfs":{"total":95995035648,"available":92434984960,"internally_reserved":0,"allocated":2895499264,"data_stored":2895009950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":664533446},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724767,"num_pgs":62,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2191704,"kb_used_data":1412232,"kb_used_omap":1,"kb_used_meta":779454,"kb_avail":91553448,"statfs":{"total":95995035648,"available":93750730752,"internally_reserved":0,"allocated":1446125568,"data_stored":1445617007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":798161350},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953700,"num_pgs":69,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2113248,"kb_used_data":1432336,"kb_used_omap":1,"kb_used_meta":680894,"kb_avail":91631904,"statfs":{"total":95995035648,"available":93831069696,"internally_reserved":0,"allocated":1466712064,"data_stored":1466195240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":697235910},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149930,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3329976,"kb_used_data":2496356,"kb_used_omap":1,"kb_used_meta":833598,"kb_avail":90415176,"statfs":{"total":95995035648,"available":92585140224,"internally_reserved":0,"allocated":2556268544,"data_stored":2555740350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":853604807},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411567,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2788928,"kb_used_data":2121904,"kb_used_omap":1,"kb_used_meta":667006,"kb_avail":90956224,"statfs":{"total":95995035648,"available":93139173376,"internally_reserved":0,"allocated":2172829696,"data_stored":2172344734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":683014597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575094,"num_pgs":68,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3803964,"kb_used_data":2870380,"kb_used_omap":1,"kb_used_meta":933566,"kb_avail":89941188,"statfs":{"total":95995035648,"available":92099776512,"internally_reserved":0,"allocated":2939269120,"data_stored":2938723604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":955972037},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738620,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2600428,"kb_used_data":1793176,"kb_used_omap":1,"kb_used_meta":807230,"kb_avail":91144724,"statfs":{"total":95995035648,"available":93332197376,"internally_reserved":0,"allocated":1836212224,"data_stored":1835695690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":826603973},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":1788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1852506112,"data_stored":1852506112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2946760704,"data_stored":2946760704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2194931712,"data_stored":2194931712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2554789888,"data_stored":2554789888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1467285504,"data_stored":1467285504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1454243840,"data_stored":1454243840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":2900230144,"data_stored":2900230144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":2180382720,"data_stored":2180382720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T09:42:18.553 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd pool set unique_pool_4 pgp_num 16 2024-09-16T09:42:18.559 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:42:18.649 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:18.650 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:18.650 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:42:18.751 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:42:18.772 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:18 smithi070 ceph-mon[40691]: from='client.25921 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:42:18.772 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:18 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:42:18.772 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:18 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:42:18.772 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:18 smithi070 ceph-mon[40691]: 4.9 scrub starts 2024-09-16T09:42:18.772 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:18 smithi070 ceph-mon[40691]: 4.9 scrub ok 2024-09-16T09:42:18.772 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:18 smithi070 ceph-mon[40691]: osdmap e478: 8 total, 8 up, 8 in 2024-09-16T09:42:18.772 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:18 smithi070 ceph-mon[40691]: 4.5 scrub starts 2024-09-16T09:42:18.772 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:18 smithi070 ceph-mon[40691]: 4.5 scrub ok 2024-09-16T09:42:18.773 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:18 smithi070 ceph-mon[40691]: 3.9 scrub starts 2024-09-16T09:42:18.773 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:18 smithi070 ceph-mon[40691]: 3.9 scrub ok 2024-09-16T09:42:18.774 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:18 smithi070 ceph-mon[30388]: from='client.25921 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:42:18.774 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:18 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:42:18.775 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:18 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:42:18.775 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:18 smithi070 ceph-mon[30388]: 4.9 scrub starts 2024-09-16T09:42:18.775 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:18 smithi070 ceph-mon[30388]: 4.9 scrub ok 2024-09-16T09:42:18.775 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:18 smithi070 ceph-mon[30388]: osdmap e478: 8 total, 8 up, 8 in 2024-09-16T09:42:18.775 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:18 smithi070 ceph-mon[30388]: 4.5 scrub starts 2024-09-16T09:42:18.775 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:18 smithi070 ceph-mon[30388]: 4.5 scrub ok 2024-09-16T09:42:18.775 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:18 smithi070 ceph-mon[30388]: 3.9 scrub starts 2024-09-16T09:42:18.775 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:18 smithi070 ceph-mon[30388]: 3.9 scrub ok 2024-09-16T09:42:18.804 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:18.805 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:18.805 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:42:18.906 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:42:18.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:18 smithi084 ceph-mon[32629]: from='client.25921 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:42:18.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:18 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:42:18.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:18 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:42:18.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:18 smithi084 ceph-mon[32629]: 4.9 scrub starts 2024-09-16T09:42:18.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:18 smithi084 ceph-mon[32629]: 4.9 scrub ok 2024-09-16T09:42:18.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:18 smithi084 ceph-mon[32629]: osdmap e478: 8 total, 8 up, 8 in 2024-09-16T09:42:18.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:18 smithi084 ceph-mon[32629]: 4.5 scrub starts 2024-09-16T09:42:18.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:18 smithi084 ceph-mon[32629]: 4.5 scrub ok 2024-09-16T09:42:18.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:18 smithi084 ceph-mon[32629]: 3.9 scrub starts 2024-09-16T09:42:18.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:18 smithi084 ceph-mon[32629]: 3.9 scrub ok 2024-09-16T09:42:18.955 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:18.956 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:18.956 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:42:19.057 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:42:19.083 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:19.083 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:19.084 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:42:19.097 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 59 16 95807 95791 101.458 73.6875 0.0135146 0.00985132 2024-09-16T09:42:19.185 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:42:19.229 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:19.230 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:19.230 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:42:19.331 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:42:19.366 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:19.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:19.367 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:42:19.467 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:42:19.502 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:19.503 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:19.503 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:42:19.604 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:42:19.621 INFO:teuthology.orchestra.run.smithi070.stderr:nodeep-scrub is unset 2024-09-16T09:42:19.644 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:19.644 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:19.645 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:42:19.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:19 smithi070 ceph-mon[30388]: pgmap v911: 168 pgs: 1 active+clean+scrubbing, 167 active+clean; 5.5 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.2 KiB/s rd, 44 MiB/s wr, 702 op/s 2024-09-16T09:42:19.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:19 smithi070 ceph-mon[30388]: 11.9 scrub ok 2024-09-16T09:42:19.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:19 smithi070 ceph-mon[30388]: 3.f scrub starts 2024-09-16T09:42:19.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:19 smithi070 ceph-mon[30388]: 3.f scrub ok 2024-09-16T09:42:19.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:19 smithi070 ceph-mon[30388]: 5.f scrub starts 2024-09-16T09:42:19.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:19 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:42:19.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:19 smithi070 ceph-mon[30388]: 5.f scrub ok 2024-09-16T09:42:19.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:19 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/792096397' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:42:19.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:19 smithi070 ceph-mon[40691]: pgmap v911: 168 pgs: 1 active+clean+scrubbing, 167 active+clean; 5.5 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.2 KiB/s rd, 44 MiB/s wr, 702 op/s 2024-09-16T09:42:19.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:19 smithi070 ceph-mon[40691]: 11.9 scrub ok 2024-09-16T09:42:19.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:19 smithi070 ceph-mon[40691]: 3.f scrub starts 2024-09-16T09:42:19.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:19 smithi070 ceph-mon[40691]: 3.f scrub ok 2024-09-16T09:42:19.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:19 smithi070 ceph-mon[40691]: 5.f scrub starts 2024-09-16T09:42:19.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:19 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:42:19.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:19 smithi070 ceph-mon[40691]: 5.f scrub ok 2024-09-16T09:42:19.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:19 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/792096397' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:42:19.746 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:42:19.775 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:19.776 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:19.776 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:42:19.877 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:42:19.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:19 smithi084 ceph-mon[32629]: pgmap v911: 168 pgs: 1 active+clean+scrubbing, 167 active+clean; 5.5 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.2 KiB/s rd, 44 MiB/s wr, 702 op/s 2024-09-16T09:42:19.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:19 smithi084 ceph-mon[32629]: 11.9 scrub ok 2024-09-16T09:42:19.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:19 smithi084 ceph-mon[32629]: 3.f scrub starts 2024-09-16T09:42:19.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:19 smithi084 ceph-mon[32629]: 3.f scrub ok 2024-09-16T09:42:19.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:19 smithi084 ceph-mon[32629]: 5.f scrub starts 2024-09-16T09:42:19.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:19 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:42:19.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:19 smithi084 ceph-mon[32629]: 5.f scrub ok 2024-09-16T09:42:19.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:19 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/792096397' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:42:19.923 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:19.923 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:19.924 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:42:20.025 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:42:20.066 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:20.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:20.068 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:42:20.098 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:2024-09-16T09:42:20.097567+0000 min lat: 0.00125032 max lat: 0.55794 avg lat: 0.00989874 2024-09-16T09:42:20.098 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-16T09:42:20.098 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 60 16 96958 96942 100.966 71.9375 0.0139466 0.00989874 2024-09-16T09:42:20.169 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:42:20.205 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:20.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:20.206 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:42:20.307 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:42:20.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:20.343 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:20.343 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:42:20.444 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:42:20.476 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:20.477 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:20.477 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:42:20.505 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T09:42:20.505 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T09:42:20.505 INFO:teuthology.orchestra.run.smithi070.stdout: "complaint_time": 30, 2024-09-16T09:42:20.505 INFO:teuthology.orchestra.run.smithi070.stdout: "num_blocked_ops": 0 2024-09-16T09:42:20.505 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T09:42:20.578 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:42:20.620 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:20.621 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:20.621 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:42:20.722 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:42:20.746 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:20.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:20.747 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:42:20.848 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:42:20.879 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:20.880 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:20.880 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:42:20.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:20 smithi084 ceph-mon[32629]: 5.14 scrub starts 2024-09-16T09:42:20.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:20 smithi084 ceph-mon[32629]: 5.14 scrub ok 2024-09-16T09:42:20.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:20 smithi084 ceph-mon[32629]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-16T09:42:20.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:20 smithi084 ceph-mon[32629]: Cluster is now healthy 2024-09-16T09:42:20.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:20 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/792096397' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-16T09:42:20.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:20 smithi084 ceph-mon[32629]: osdmap e479: 8 total, 8 up, 8 in 2024-09-16T09:42:20.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:20 smithi084 ceph-mon[32629]: 11.c scrub starts 2024-09-16T09:42:20.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:20 smithi084 ceph-mon[32629]: 6.9 scrub starts 2024-09-16T09:42:20.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:20 smithi084 ceph-mon[32629]: 6.9 scrub ok 2024-09-16T09:42:20.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:20 smithi084 ceph-mon[32629]: 4.13 scrub starts 2024-09-16T09:42:20.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:20 smithi084 ceph-mon[32629]: 4.13 scrub ok 2024-09-16T09:42:20.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:20 smithi070 ceph-mon[30388]: 5.14 scrub starts 2024-09-16T09:42:20.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:20 smithi070 ceph-mon[30388]: 5.14 scrub ok 2024-09-16T09:42:20.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:20 smithi070 ceph-mon[30388]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-16T09:42:20.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:20 smithi070 ceph-mon[30388]: Cluster is now healthy 2024-09-16T09:42:20.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:20 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/792096397' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-16T09:42:20.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:20 smithi070 ceph-mon[30388]: osdmap e479: 8 total, 8 up, 8 in 2024-09-16T09:42:20.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:20 smithi070 ceph-mon[30388]: 11.c scrub starts 2024-09-16T09:42:20.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:20 smithi070 ceph-mon[30388]: 6.9 scrub starts 2024-09-16T09:42:20.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:20 smithi070 ceph-mon[30388]: 6.9 scrub ok 2024-09-16T09:42:20.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:20 smithi070 ceph-mon[30388]: 4.13 scrub starts 2024-09-16T09:42:20.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:20 smithi070 ceph-mon[30388]: 4.13 scrub ok 2024-09-16T09:42:20.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:20 smithi070 ceph-mon[40691]: 5.14 scrub starts 2024-09-16T09:42:20.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:20 smithi070 ceph-mon[40691]: 5.14 scrub ok 2024-09-16T09:42:20.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:20 smithi070 ceph-mon[40691]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-16T09:42:20.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:20 smithi070 ceph-mon[40691]: Cluster is now healthy 2024-09-16T09:42:20.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:20 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/792096397' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-16T09:42:20.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:20 smithi070 ceph-mon[40691]: osdmap e479: 8 total, 8 up, 8 in 2024-09-16T09:42:20.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:20 smithi070 ceph-mon[40691]: 11.c scrub starts 2024-09-16T09:42:20.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:20 smithi070 ceph-mon[40691]: 6.9 scrub starts 2024-09-16T09:42:20.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:20 smithi070 ceph-mon[40691]: 6.9 scrub ok 2024-09-16T09:42:20.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:20 smithi070 ceph-mon[40691]: 4.13 scrub starts 2024-09-16T09:42:20.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:20 smithi070 ceph-mon[40691]: 4.13 scrub ok 2024-09-16T09:42:20.981 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:42:21.016 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:21.017 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:21.017 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:42:21.098 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 61 16 98191 98175 100.574 77.0625 0.0110675 0.00993812 2024-09-16T09:42:21.119 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:42:21.161 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:21.162 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:21.162 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:42:21.263 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:42:21.295 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:21.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:21.296 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:42:21.397 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:42:21.431 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:21.432 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:21.432 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:42:21.533 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:42:21.560 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:21.561 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:21.561 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:42:21.662 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:42:21.694 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:21.694 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:21.695 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:42:21.796 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:42:21.819 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:21.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:21.820 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:42:21.874 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:21 smithi070 ceph-mon[30388]: pgmap v913: 168 pgs: 168 active+clean; 5.8 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 942 B/s rd, 71 MiB/s wr, 1.14k op/s 2024-09-16T09:42:21.874 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:21 smithi070 ceph-mon[40691]: pgmap v913: 168 pgs: 168 active+clean; 5.8 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 942 B/s rd, 71 MiB/s wr, 1.14k op/s 2024-09-16T09:42:21.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:21 smithi084 ceph-mon[32629]: pgmap v913: 168 pgs: 168 active+clean; 5.8 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 942 B/s rd, 71 MiB/s wr, 1.14k op/s 2024-09-16T09:42:21.921 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:42:21.966 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T09:42:21.967 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:21.968 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:21.968 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:42:22.069 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:42:22.098 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 62 16 99336 99320 100.106 71.5625 0.0157056 0.00998472 2024-09-16T09:42:22.098 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:22.099 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:22.099 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:42:22.171 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:42:21 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:42:21] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:42:22.200 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:42:22.259 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:22.260 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:22.260 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:42:22.361 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:42:22.409 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:22.409 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:22.410 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:42:22.510 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:42:22.545 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:22.545 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:22.546 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:42:22.646 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:42:22.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:22 smithi070 ceph-mon[30388]: 5.7 scrub starts 2024-09-16T09:42:22.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:22 smithi070 ceph-mon[30388]: 5.7 scrub ok 2024-09-16T09:42:22.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:22 smithi070 ceph-mon[30388]: 1.3 scrub starts 2024-09-16T09:42:22.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:22 smithi070 ceph-mon[30388]: 1.3 scrub ok 2024-09-16T09:42:22.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:22 smithi070 ceph-mon[30388]: pgmap v914: 168 pgs: 168 active+clean; 5.8 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 853 B/s rd, 60 MiB/s wr, 961 op/s 2024-09-16T09:42:22.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:22 smithi070 ceph-mon[30388]: 11.c scrub ok 2024-09-16T09:42:22.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:22 smithi070 ceph-mon[40691]: 5.7 scrub starts 2024-09-16T09:42:22.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:22 smithi070 ceph-mon[40691]: 5.7 scrub ok 2024-09-16T09:42:22.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:22 smithi070 ceph-mon[40691]: 1.3 scrub starts 2024-09-16T09:42:22.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:22 smithi070 ceph-mon[40691]: 1.3 scrub ok 2024-09-16T09:42:22.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:22 smithi070 ceph-mon[40691]: pgmap v914: 168 pgs: 168 active+clean; 5.8 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 853 B/s rd, 60 MiB/s wr, 961 op/s 2024-09-16T09:42:22.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:22 smithi070 ceph-mon[40691]: 11.c scrub ok 2024-09-16T09:42:22.679 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:22.680 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:22.680 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:42:22.781 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:42:22.802 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:22.803 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:22.803 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:42:22.904 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:42:22.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:22 smithi084 ceph-mon[32629]: 5.7 scrub starts 2024-09-16T09:42:22.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:22 smithi084 ceph-mon[32629]: 5.7 scrub ok 2024-09-16T09:42:22.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:22 smithi084 ceph-mon[32629]: 1.3 scrub starts 2024-09-16T09:42:22.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:22 smithi084 ceph-mon[32629]: 1.3 scrub ok 2024-09-16T09:42:22.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:22 smithi084 ceph-mon[32629]: pgmap v914: 168 pgs: 168 active+clean; 5.8 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 853 B/s rd, 60 MiB/s wr, 961 op/s 2024-09-16T09:42:22.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:22 smithi084 ceph-mon[32629]: 11.c scrub ok 2024-09-16T09:42:22.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:22.936 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:22.936 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:42:23.037 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:42:23.064 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:23.065 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:23.066 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:42:23.098 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 63 16 100470 100454 99.6417 70.875 0.0131992 0.0100315 2024-09-16T09:42:23.167 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:42:23.205 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:23.205 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:23.206 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:42:23.306 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:42:23.328 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:23.329 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:23.329 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:42:23.430 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:42:23.452 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:23.452 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:23.453 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:42:23.554 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:42:23.576 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:23.576 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:23.576 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:42:23.677 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:42:23.711 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:23.711 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:23.712 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:42:23.812 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:42:23.847 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:23.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:23.849 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:42:23.910 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:42:23 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:42:23.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:23 smithi084 ceph-mon[32629]: 11.b scrub starts 2024-09-16T09:42:23.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:23 smithi084 ceph-mon[32629]: 3.1c scrub starts 2024-09-16T09:42:23.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:23 smithi084 ceph-mon[32629]: 3.1c scrub ok 2024-09-16T09:42:23.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:23 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:42:23.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:23 smithi070 ceph-mon[30388]: 11.b scrub starts 2024-09-16T09:42:23.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:23 smithi070 ceph-mon[30388]: 3.1c scrub starts 2024-09-16T09:42:23.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:23 smithi070 ceph-mon[30388]: 3.1c scrub ok 2024-09-16T09:42:23.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:23 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:42:23.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:23 smithi070 ceph-mon[40691]: 11.b scrub starts 2024-09-16T09:42:23.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:23 smithi070 ceph-mon[40691]: 3.1c scrub starts 2024-09-16T09:42:23.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:23 smithi070 ceph-mon[40691]: 3.1c scrub ok 2024-09-16T09:42:23.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:23 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:42:23.950 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:42:23.981 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:23.982 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:23.982 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:42:24.083 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:42:24.098 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 64 16 101245 101229 98.8416 48.4375 0.00708539 0.010112 2024-09-16T09:42:24.112 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:24.112 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:24.113 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:42:24.214 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:42:24.244 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:24.245 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:24.246 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:42:24.347 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:42:24.377 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:24.378 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:24.378 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:42:24.479 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:42:24.532 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:24.532 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:24.532 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:42:24.633 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:42:24.676 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:24.677 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:24.677 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:42:24.780 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:42:24.821 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:24.822 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:24.822 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:42:24.923 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:42:24.958 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:24.959 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:24.959 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:42:25.060 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:42:25.096 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:25.097 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:25.097 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:42:25.098 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 65 16 102393 102377 98.4248 71.75 0.0108208 0.0101552 2024-09-16T09:42:25.198 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:42:25.229 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:25.230 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:25.230 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:42:25.331 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:42:25.361 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.3 dump_historic_ops 2024-09-16T09:42:25.380 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:42:25.391 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:25.391 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:25.392 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:42:25.403 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:42:25.493 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:42:25.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:25.535 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:25.535 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:42:25.636 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:42:25.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:25.666 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:25.666 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:42:25.718 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:25 smithi070 ceph-mon[30388]: pgmap v915: 168 pgs: 1 active+clean+scrubbing, 167 active+clean; 6.1 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.4 KiB/s rd, 96 MiB/s wr, 1.53k op/s 2024-09-16T09:42:25.718 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:25 smithi070 ceph-mon[30388]: 3.d scrub starts 2024-09-16T09:42:25.718 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:25 smithi070 ceph-mon[30388]: 3.d scrub ok 2024-09-16T09:42:25.718 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:25 smithi070 ceph-mon[30388]: osdmap e480: 8 total, 8 up, 8 in 2024-09-16T09:42:25.718 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:25 smithi070 ceph-mon[30388]: 11.b scrub ok 2024-09-16T09:42:25.718 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:25 smithi070 ceph-mon[30388]: 1.6 scrub starts 2024-09-16T09:42:25.718 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:25 smithi070 ceph-mon[30388]: 1.6 scrub ok 2024-09-16T09:42:25.718 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:25 smithi070 ceph-mon[40691]: pgmap v915: 168 pgs: 1 active+clean+scrubbing, 167 active+clean; 6.1 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.4 KiB/s rd, 96 MiB/s wr, 1.53k op/s 2024-09-16T09:42:25.718 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:25 smithi070 ceph-mon[40691]: 3.d scrub starts 2024-09-16T09:42:25.718 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:25 smithi070 ceph-mon[40691]: 3.d scrub ok 2024-09-16T09:42:25.719 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:25 smithi070 ceph-mon[40691]: osdmap e480: 8 total, 8 up, 8 in 2024-09-16T09:42:25.719 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:25 smithi070 ceph-mon[40691]: 11.b scrub ok 2024-09-16T09:42:25.719 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:25 smithi070 ceph-mon[40691]: 1.6 scrub starts 2024-09-16T09:42:25.719 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:25 smithi070 ceph-mon[40691]: 1.6 scrub ok 2024-09-16T09:42:25.766 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:42:25.810 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:25.810 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:25.811 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:42:25.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:25 smithi084 ceph-mon[32629]: pgmap v915: 168 pgs: 1 active+clean+scrubbing, 167 active+clean; 6.1 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.4 KiB/s rd, 96 MiB/s wr, 1.53k op/s 2024-09-16T09:42:25.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:25 smithi084 ceph-mon[32629]: 3.d scrub starts 2024-09-16T09:42:25.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:25 smithi084 ceph-mon[32629]: 3.d scrub ok 2024-09-16T09:42:25.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:25 smithi084 ceph-mon[32629]: osdmap e480: 8 total, 8 up, 8 in 2024-09-16T09:42:25.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:25 smithi084 ceph-mon[32629]: 11.b scrub ok 2024-09-16T09:42:25.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:25 smithi084 ceph-mon[32629]: 1.6 scrub starts 2024-09-16T09:42:25.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:25 smithi084 ceph-mon[32629]: 1.6 scrub ok 2024-09-16T09:42:25.912 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:42:25.946 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:25.946 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:25.947 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:42:26.048 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:42:26.077 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:26.077 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:26.078 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:42:26.098 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 66 16 103475 103459 97.9576 67.625 0.031793 0.0102036 2024-09-16T09:42:26.178 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:42:26.198 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:42:26.198 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:42:26.199 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:42:26.199 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:42:26.207 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:26.207 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:26.208 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:42:26.225 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:42:26.226 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:42:26.263 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:42:26.264 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:42:26.298 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:42:26.299 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:42:26.308 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:42:26.332 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:42:26.332 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:42:26.340 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:26.340 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:26.341 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:42:26.374 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:42:26.374 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:42:26.409 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:42:26.409 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:42:26.442 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:42:26.476 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:26.476 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:26.477 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:42:26.578 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:42:26.607 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:26.608 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:26.608 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:42:26.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:26 smithi070 ceph-mon[30388]: 4.3 scrub starts 2024-09-16T09:42:26.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:26 smithi070 ceph-mon[30388]: 4.3 scrub ok 2024-09-16T09:42:26.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:26 smithi070 ceph-mon[30388]: 6.1c scrub starts 2024-09-16T09:42:26.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:26 smithi070 ceph-mon[30388]: 6.1c scrub ok 2024-09-16T09:42:26.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:26 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/231733128' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:42:26.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:26 smithi070 ceph-mon[40691]: 4.3 scrub starts 2024-09-16T09:42:26.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:26 smithi070 ceph-mon[40691]: 4.3 scrub ok 2024-09-16T09:42:26.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:26 smithi070 ceph-mon[40691]: 6.1c scrub starts 2024-09-16T09:42:26.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:26 smithi070 ceph-mon[40691]: 6.1c scrub ok 2024-09-16T09:42:26.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:26 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/231733128' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:42:26.709 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:42:26.764 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:26.765 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:26.765 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:42:26.786 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:26 smithi084 ceph-mon[32629]: 4.3 scrub starts 2024-09-16T09:42:26.786 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:26 smithi084 ceph-mon[32629]: 4.3 scrub ok 2024-09-16T09:42:26.786 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:26 smithi084 ceph-mon[32629]: 6.1c scrub starts 2024-09-16T09:42:26.786 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:26 smithi084 ceph-mon[32629]: 6.1c scrub ok 2024-09-16T09:42:26.786 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:26 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/231733128' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:42:26.866 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:42:26.901 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:26.901 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:26.902 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:42:27.002 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:42:27.033 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:27.034 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:27.034 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:42:27.099 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 67 16 104590 104574 97.5356 69.6875 0.00830818 0.0102478 2024-09-16T09:42:27.134 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:42:27.158 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd set noscrub 2024-09-16T09:42:27.163 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:27.164 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:27.164 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:42:27.197 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:42:27.266 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:42:27.331 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:27.332 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:27.332 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:42:27.433 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:42:27.466 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:27.466 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:27.467 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:42:27.477 INFO:teuthology.orchestra.run.smithi070.stderr:set pool 11 pgp_num to 16 2024-09-16T09:42:27.567 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:42:27.600 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:27.601 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:27.601 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:42:27.702 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:42:27.743 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:27.743 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:27.744 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:42:27.844 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:42:27.879 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:27.880 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:27.880 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:42:27.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:27 smithi084 ceph-mon[32629]: pgmap v917: 168 pgs: 1 active+clean+scrubbing, 167 active+clean; 6.1 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 639 B/s rd, 78 MiB/s wr, 1.25k op/s 2024-09-16T09:42:27.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:27 smithi084 ceph-mon[32629]: 4.4 scrub starts 2024-09-16T09:42:27.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:27 smithi084 ceph-mon[32629]: 4.4 scrub ok 2024-09-16T09:42:27.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:27 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/899979933' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_4", "var": "pgp_num", "val": "16"} : dispatch 2024-09-16T09:42:27.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:27 smithi084 ceph-mon[32629]: 5.0 scrub starts 2024-09-16T09:42:27.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:27 smithi084 ceph-mon[32629]: 5.0 scrub ok 2024-09-16T09:42:27.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:27 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:42:27.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:27 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:42:27.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:27 smithi070 ceph-mon[30388]: pgmap v917: 168 pgs: 1 active+clean+scrubbing, 167 active+clean; 6.1 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 639 B/s rd, 78 MiB/s wr, 1.25k op/s 2024-09-16T09:42:27.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:27 smithi070 ceph-mon[30388]: 4.4 scrub starts 2024-09-16T09:42:27.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:27 smithi070 ceph-mon[30388]: 4.4 scrub ok 2024-09-16T09:42:27.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:27 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/899979933' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_4", "var": "pgp_num", "val": "16"} : dispatch 2024-09-16T09:42:27.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:27 smithi070 ceph-mon[30388]: 5.0 scrub starts 2024-09-16T09:42:27.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:27 smithi070 ceph-mon[30388]: 5.0 scrub ok 2024-09-16T09:42:27.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:27 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:42:27.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:27 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:42:27.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:27 smithi070 ceph-mon[40691]: pgmap v917: 168 pgs: 1 active+clean+scrubbing, 167 active+clean; 6.1 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 639 B/s rd, 78 MiB/s wr, 1.25k op/s 2024-09-16T09:42:27.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:27 smithi070 ceph-mon[40691]: 4.4 scrub starts 2024-09-16T09:42:27.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:27 smithi070 ceph-mon[40691]: 4.4 scrub ok 2024-09-16T09:42:27.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:27 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/899979933' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "unique_pool_4", "var": "pgp_num", "val": "16"} : dispatch 2024-09-16T09:42:27.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:27 smithi070 ceph-mon[40691]: 5.0 scrub starts 2024-09-16T09:42:27.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:27 smithi070 ceph-mon[40691]: 5.0 scrub ok 2024-09-16T09:42:27.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:27 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:42:27.923 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:27 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:42:27.981 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:42:28.020 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:28.021 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:28.021 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:42:28.099 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 68 16 105759 105743 97.1756 73.0625 0.00494378 0.0102856 2024-09-16T09:42:28.121 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:42:28.160 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:28.161 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:28.161 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:42:28.262 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:42:28.290 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:28.291 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:28.291 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:42:28.392 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:42:28.416 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:28.417 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:28.417 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:42:28.518 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:42:28.556 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:28.557 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:28.557 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:42:28.658 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:42:28.666 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T09:42:28.666 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:42:28.694 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:28.694 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:28.695 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:42:28.796 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:42:28.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:28.834 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:28.834 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:42:28.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:28 smithi084 ceph-mon[32629]: 1.b scrub starts 2024-09-16T09:42:28.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:28 smithi084 ceph-mon[32629]: 1.b scrub ok 2024-09-16T09:42:28.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:28 smithi084 ceph-mon[32629]: 4.10 scrub starts 2024-09-16T09:42:28.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:28 smithi084 ceph-mon[32629]: 4.10 scrub ok 2024-09-16T09:42:28.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:28 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/899979933' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_4", "var": "pgp_num", "val": "16"}]': finished 2024-09-16T09:42:28.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:28 smithi084 ceph-mon[32629]: osdmap e481: 8 total, 8 up, 8 in 2024-09-16T09:42:28.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:28 smithi070 ceph-mon[30388]: 1.b scrub starts 2024-09-16T09:42:28.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:28 smithi070 ceph-mon[30388]: 1.b scrub ok 2024-09-16T09:42:28.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:28 smithi070 ceph-mon[30388]: 4.10 scrub starts 2024-09-16T09:42:28.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:28 smithi070 ceph-mon[30388]: 4.10 scrub ok 2024-09-16T09:42:28.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:28 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/899979933' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_4", "var": "pgp_num", "val": "16"}]': finished 2024-09-16T09:42:28.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:28 smithi070 ceph-mon[30388]: osdmap e481: 8 total, 8 up, 8 in 2024-09-16T09:42:28.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:28 smithi070 ceph-mon[40691]: 1.b scrub starts 2024-09-16T09:42:28.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:28 smithi070 ceph-mon[40691]: 1.b scrub ok 2024-09-16T09:42:28.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:28 smithi070 ceph-mon[40691]: 4.10 scrub starts 2024-09-16T09:42:28.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:28 smithi070 ceph-mon[40691]: 4.10 scrub ok 2024-09-16T09:42:28.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:28 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/899979933' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_4", "var": "pgp_num", "val": "16"}]': finished 2024-09-16T09:42:28.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:28 smithi070 ceph-mon[40691]: osdmap e481: 8 total, 8 up, 8 in 2024-09-16T09:42:28.935 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:42:28.961 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:28.961 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:28.962 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:42:29.063 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:42:29.093 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:29.094 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:29.094 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:42:29.099 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 69 16 107016 107000 96.9058 78.5625 0.0228426 0.0103145 2024-09-16T09:42:29.195 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:42:29.219 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:29.220 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:29.220 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:42:29.249 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T09:42:29.322 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:42:29.351 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:29.352 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:29.352 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:42:29.453 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:42:29.579 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:29.580 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:29.580 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:42:29.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:29 smithi070 ceph-mon[30388]: pgmap v919: 168 pgs: 1 active+clean+scrubbing, 167 active+clean; 6.2 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.2 KiB/s rd, 54 MiB/s wr, 873 op/s 2024-09-16T09:42:29.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:29 smithi070 ceph-mon[30388]: 6.a scrub starts 2024-09-16T09:42:29.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:29 smithi070 ceph-mon[30388]: 6.a scrub ok 2024-09-16T09:42:29.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:29 smithi070 ceph-mon[30388]: osdmap e482: 8 total, 8 up, 8 in 2024-09-16T09:42:29.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:29 smithi070 ceph-mon[40691]: pgmap v919: 168 pgs: 1 active+clean+scrubbing, 167 active+clean; 6.2 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.2 KiB/s rd, 54 MiB/s wr, 873 op/s 2024-09-16T09:42:29.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:29 smithi070 ceph-mon[40691]: 6.a scrub starts 2024-09-16T09:42:29.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:29 smithi070 ceph-mon[40691]: 6.a scrub ok 2024-09-16T09:42:29.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:29 smithi070 ceph-mon[40691]: osdmap e482: 8 total, 8 up, 8 in 2024-09-16T09:42:29.681 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:42:29.703 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:29.703 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:29.704 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:42:29.804 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:42:29.830 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:29.831 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:29.831 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:42:29.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:29 smithi084 ceph-mon[32629]: pgmap v919: 168 pgs: 1 active+clean+scrubbing, 167 active+clean; 6.2 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.2 KiB/s rd, 54 MiB/s wr, 873 op/s 2024-09-16T09:42:29.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:29 smithi084 ceph-mon[32629]: 6.a scrub starts 2024-09-16T09:42:29.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:29 smithi084 ceph-mon[32629]: 6.a scrub ok 2024-09-16T09:42:29.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:29 smithi084 ceph-mon[32629]: osdmap e482: 8 total, 8 up, 8 in 2024-09-16T09:42:29.932 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:42:29.967 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:29.968 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:29.968 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:42:30.069 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:42:30.099 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 70 16 107962 107946 96.366 59.125 0.00367191 0.0103716 2024-09-16T09:42:30.118 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:30.118 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:30.119 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:42:30.220 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:42:30.253 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:30.253 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:30.254 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:42:30.354 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:42:30.381 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:30.381 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:30.382 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:42:30.483 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:42:30.509 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:30.509 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:30.510 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:42:30.610 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:42:30.646 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:30.647 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:30.647 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:42:30.748 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:42:30.787 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:30.787 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:30.788 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:42:30.888 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:42:30.919 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:30.920 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:30.920 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:42:31.021 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:42:31.061 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:31.061 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:31.062 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:42:31.099 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 71 16 109013 108997 95.9339 65.6875 0.00812044 0.0104194 2024-09-16T09:42:31.162 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:42:31.193 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:31.193 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:31.194 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:42:31.203 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.4 dump_ops_in_flight 2024-09-16T09:42:31.294 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:42:31.332 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:31.332 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:31.332 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:42:31.395 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:42:31.422 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:42:31.433 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:42:31.489 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:31.490 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:31.490 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:42:31.591 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:42:31.631 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:31.631 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:31.632 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:42:31.635 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:42:31.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:31 smithi084 ceph-mon[32629]: pgmap v921: 168 pgs: 168 active+clean; 6.5 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 853 B/s rd, 60 MiB/s wr, 955 op/s 2024-09-16T09:42:31.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:31 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:42:31.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:31 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:42:31.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:31 smithi084 ceph-mon[32629]: osdmap e483: 8 total, 8 up, 8 in 2024-09-16T09:42:31.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:31 smithi084 ceph-mon[32629]: 3.c scrub starts 2024-09-16T09:42:31.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:31 smithi084 ceph-mon[32629]: 3.c scrub ok 2024-09-16T09:42:31.733 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:42:31.758 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:31 smithi070 ceph-mon[30388]: pgmap v921: 168 pgs: 168 active+clean; 6.5 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 853 B/s rd, 60 MiB/s wr, 955 op/s 2024-09-16T09:42:31.758 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:31 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:42:31.758 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:31 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:42:31.759 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:31 smithi070 ceph-mon[30388]: osdmap e483: 8 total, 8 up, 8 in 2024-09-16T09:42:31.759 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:31 smithi070 ceph-mon[30388]: 3.c scrub starts 2024-09-16T09:42:31.759 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:31 smithi070 ceph-mon[30388]: 3.c scrub ok 2024-09-16T09:42:31.759 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:31 smithi070 ceph-mon[40691]: pgmap v921: 168 pgs: 168 active+clean; 6.5 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 853 B/s rd, 60 MiB/s wr, 955 op/s 2024-09-16T09:42:31.759 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:31 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:42:31.760 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:31 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:42:31.760 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:31 smithi070 ceph-mon[40691]: osdmap e483: 8 total, 8 up, 8 in 2024-09-16T09:42:31.760 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:31 smithi070 ceph-mon[40691]: 3.c scrub starts 2024-09-16T09:42:31.760 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:31 smithi070 ceph-mon[40691]: 3.c scrub ok 2024-09-16T09:42:31.766 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:31.767 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:31.767 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:42:31.868 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:42:31.910 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:31.911 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:31.911 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:42:32.012 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:42:32.045 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:32.045 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:32.046 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:42:32.099 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 72 16 110257 110241 95.6812 77.75 0.00932985 0.0104465 2024-09-16T09:42:32.109 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:42:31 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:42:31] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:42:32.146 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:42:32.182 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:32.183 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:32.183 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:42:32.284 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:42:32.311 INFO:teuthology.orchestra.run.smithi084.stdout:ERROR: (22) Invalid argument 2024-09-16T09:42:32.311 INFO:teuthology.orchestra.run.smithi084.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:42:32.319 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:32.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:32.321 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:42:32.421 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:42:32.446 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:32.447 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:32.447 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:42:32.459 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:42:32.459 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:42:32.459 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:42:32.459 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:42:32.459 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:42:32.459 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:42:32.459 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:42:32.459 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:42:32.471 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:42:32.472 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:42:32.503 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:42:32.503 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:42:32.548 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:42:32.554 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:42:32.554 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:42:32.577 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:32.578 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:32.578 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:42:32.594 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:42:32.595 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:42:32.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:32 smithi084 ceph-mon[32629]: 3.18 scrub starts 2024-09-16T09:42:32.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:32 smithi084 ceph-mon[32629]: 3.18 scrub ok 2024-09-16T09:42:32.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:32 smithi084 ceph-mon[32629]: 5.19 scrub starts 2024-09-16T09:42:32.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:32 smithi084 ceph-mon[32629]: 5.19 scrub ok 2024-09-16T09:42:32.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:32 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:42:32.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:32 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:42:32.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:32 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/4050665033' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:42:32.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:32 smithi070 ceph-mon[30388]: 3.18 scrub starts 2024-09-16T09:42:32.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:32 smithi070 ceph-mon[30388]: 3.18 scrub ok 2024-09-16T09:42:32.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:32 smithi070 ceph-mon[30388]: 5.19 scrub starts 2024-09-16T09:42:32.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:32 smithi070 ceph-mon[30388]: 5.19 scrub ok 2024-09-16T09:42:32.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:32 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:42:32.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:32 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:42:32.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:32 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/4050665033' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:42:32.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:32 smithi070 ceph-mon[40691]: 3.18 scrub starts 2024-09-16T09:42:32.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:32 smithi070 ceph-mon[40691]: 3.18 scrub ok 2024-09-16T09:42:32.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:32 smithi070 ceph-mon[40691]: 5.19 scrub starts 2024-09-16T09:42:32.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:32 smithi070 ceph-mon[40691]: 5.19 scrub ok 2024-09-16T09:42:32.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:32 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:42:32.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:32 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:42:32.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:32 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/4050665033' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:42:32.679 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:42:32.716 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:32.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:32.718 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:42:32.818 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:42:32.861 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:32.861 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:32.862 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:42:32.962 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:42:32.998 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:32.999 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:32.999 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:42:33.099 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 73 16 111432 111416 95.3764 73.4375 0.00542001 0.0104806 2024-09-16T09:42:33.099 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:42:33.135 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:33.135 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:33.136 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:42:33.154 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.4 dump_blocked_ops 2024-09-16T09:42:33.237 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:42:33.268 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:33.269 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:33.269 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:42:33.370 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:42:33.407 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:33.407 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:33.408 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:42:33.508 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:42:33.520 INFO:teuthology.orchestra.run.smithi070.stderr:noscrub is set 2024-09-16T09:42:33.552 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:33.553 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:33.553 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:42:33.596 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:42:33.654 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:42:33.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:33 smithi070 ceph-mon[30388]: pgmap v923: 168 pgs: 168 active+clean; 6.5 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 853 B/s rd, 56 MiB/s wr, 898 op/s 2024-09-16T09:42:33.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:33 smithi070 ceph-mon[30388]: osdmap e484: 8 total, 8 up, 8 in 2024-09-16T09:42:33.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:33 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2413200602' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:42:33.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:33 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:42:33.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:33 smithi070 ceph-mon[40691]: pgmap v923: 168 pgs: 168 active+clean; 6.5 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 853 B/s rd, 56 MiB/s wr, 898 op/s 2024-09-16T09:42:33.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:33 smithi070 ceph-mon[40691]: osdmap e484: 8 total, 8 up, 8 in 2024-09-16T09:42:33.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:33 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/2413200602' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:42:33.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:33 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:42:33.693 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:33.694 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:33.694 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:42:33.771 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:33 smithi084 ceph-mon[32629]: pgmap v923: 168 pgs: 168 active+clean; 6.5 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 853 B/s rd, 56 MiB/s wr, 898 op/s 2024-09-16T09:42:33.772 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:33 smithi084 ceph-mon[32629]: osdmap e484: 8 total, 8 up, 8 in 2024-09-16T09:42:33.772 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:33 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/2413200602' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:42:33.772 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:33 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:42:33.772 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:42:33 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:42:33.795 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:42:33.832 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:33.832 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:33.833 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:42:33.933 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:42:33.977 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:33.977 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:33.978 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:42:34.078 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:42:34.099 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 74 16 112737 112721 95.1897 81.5625 0.0106909 0.0105008 2024-09-16T09:42:34.110 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:34.111 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:34.111 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:42:34.212 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:42:34.227 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-09-16T09:42:34.227 INFO:teuthology.orchestra.run.smithi084.stdout: "ops": [], 2024-09-16T09:42:34.227 INFO:teuthology.orchestra.run.smithi084.stdout: "complaint_time": 30, 2024-09-16T09:42:34.227 INFO:teuthology.orchestra.run.smithi084.stdout: "num_blocked_ops": 0 2024-09-16T09:42:34.227 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-09-16T09:42:34.249 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:34.249 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:34.250 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:42:34.351 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:42:34.381 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:34.381 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:34.382 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:42:34.482 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:42:34.524 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:34.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:34.525 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:42:34.626 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:42:34.666 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:34.667 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:34.667 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:42:34.768 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:42:34.798 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:34.799 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:34.799 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:42:34.900 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:42:34.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:34 smithi084 ceph-mon[32629]: 6.16 scrub starts 2024-09-16T09:42:34.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:34 smithi084 ceph-mon[32629]: 6.16 scrub ok 2024-09-16T09:42:34.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:34 smithi084 ceph-mon[32629]: 3.e scrub starts 2024-09-16T09:42:34.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:34 smithi084 ceph-mon[32629]: 3.e scrub ok 2024-09-16T09:42:34.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:34 smithi084 ceph-mon[32629]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:42:34.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:34 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-16T09:42:34.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:34 smithi084 ceph-mon[32629]: osdmap e485: 8 total, 8 up, 8 in 2024-09-16T09:42:34.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:34 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:42:34.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:34 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:42:34.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:34 smithi084 ceph-mon[32629]: osdmap e486: 8 total, 8 up, 8 in 2024-09-16T09:42:34.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:34 smithi070 ceph-mon[30388]: 6.16 scrub starts 2024-09-16T09:42:34.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:34 smithi070 ceph-mon[30388]: 6.16 scrub ok 2024-09-16T09:42:34.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:34 smithi070 ceph-mon[30388]: 3.e scrub starts 2024-09-16T09:42:34.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:34 smithi070 ceph-mon[30388]: 3.e scrub ok 2024-09-16T09:42:34.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:34 smithi070 ceph-mon[30388]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:42:34.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:34 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-16T09:42:34.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:34 smithi070 ceph-mon[30388]: osdmap e485: 8 total, 8 up, 8 in 2024-09-16T09:42:34.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:34 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:42:34.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:34 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:42:34.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:34 smithi070 ceph-mon[30388]: osdmap e486: 8 total, 8 up, 8 in 2024-09-16T09:42:34.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:34 smithi070 ceph-mon[40691]: 6.16 scrub starts 2024-09-16T09:42:34.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:34 smithi070 ceph-mon[40691]: 6.16 scrub ok 2024-09-16T09:42:34.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:34 smithi070 ceph-mon[40691]: 3.e scrub starts 2024-09-16T09:42:34.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:34 smithi070 ceph-mon[40691]: 3.e scrub ok 2024-09-16T09:42:34.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:34 smithi070 ceph-mon[40691]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:42:34.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:34 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-16T09:42:34.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:34 smithi070 ceph-mon[40691]: osdmap e485: 8 total, 8 up, 8 in 2024-09-16T09:42:34.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:34 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:42:34.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:34 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:42:34.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:34 smithi070 ceph-mon[40691]: osdmap e486: 8 total, 8 up, 8 in 2024-09-16T09:42:34.942 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:34.943 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:34.943 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:42:35.044 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:42:35.078 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:35.078 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:35.079 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:42:35.099 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 75 16 114062 114046 95.0246 82.8125 0.0165708 0.0105192 2024-09-16T09:42:35.168 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.4 dump_historic_ops 2024-09-16T09:42:35.180 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:42:35.253 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:35.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:35.254 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:42:35.355 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:42:35.387 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:35.388 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:35.388 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:42:35.468 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T09:42:35.489 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:42:35.554 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:35.555 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:35.555 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:42:35.655 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:42:35.681 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:42:35.700 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:35.701 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:35.701 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:42:35.802 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:42:35.822 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:35 smithi084 ceph-mon[32629]: pgmap v926: 168 pgs: 168 active+clean; 6.8 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1.1 KiB/s rd, 80 MiB/s wr, 1.29k op/s 2024-09-16T09:42:35.822 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:35 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:42:35.822 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:35 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:42:35.854 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:35.854 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:35.854 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:42:35.910 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:42:35.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:35 smithi070 ceph-mon[40691]: pgmap v926: 168 pgs: 168 active+clean; 6.8 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1.1 KiB/s rd, 80 MiB/s wr, 1.29k op/s 2024-09-16T09:42:35.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:35 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:42:35.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:35 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:42:35.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:35 smithi070 ceph-mon[30388]: pgmap v926: 168 pgs: 168 active+clean; 6.8 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1.1 KiB/s rd, 80 MiB/s wr, 1.29k op/s 2024-09-16T09:42:35.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:35 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:42:35.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:35 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:42:35.956 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:42:35.983 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:35.983 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:35.984 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:42:36.085 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:42:36.100 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 76 16 115297 115281 94.7893 77.1875 0.0124502 0.0105446 2024-09-16T09:42:36.114 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:36.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:36.115 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:42:36.216 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:42:36.259 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:36.260 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:36.260 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:42:36.312 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-16T09:42:36.361 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:42:36.367 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T09:42:36.367 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T09:42:36.367 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-16T09:42:36.367 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd dump -f json-pretty 2024-09-16T09:42:36.387 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:36.387 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:36.387 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:42:36.489 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:42:36.566 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:36.566 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:36.566 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:42:36.593 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:42:36.594 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:42:36.635 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:42:36.635 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:42:36.668 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:42:36.672 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:42:36.673 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:42:36.718 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:42:36.719 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:42:36.742 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:36.742 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:36.742 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:42:36.751 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:42:36.751 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:42:36.751 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:42:36.789 INFO:teuthology.orchestra.run.smithi084.stdout:ERROR: (22) Invalid argument 2024-09-16T09:42:36.789 INFO:teuthology.orchestra.run.smithi084.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:42:36.790 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:42:36.790 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:42:36.825 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:42:36.825 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:42:36.844 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:42:36.855 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:42:36.855 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:42:36.887 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:36.887 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:36.888 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:42:36.988 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:42:37.027 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:37.028 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:37.029 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:42:37.100 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 77 16 116286 116270 94.361 61.8125 0.0105864 0.0105934 2024-09-16T09:42:37.129 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:42:37.166 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:37.166 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:37.167 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:42:37.268 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:42:37.313 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:37.314 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:37.314 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:42:37.415 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:42:37.452 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:37.452 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:37.453 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:42:37.553 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:42:37.581 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:37.581 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:37.582 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:42:37.638 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:42:37.658 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.5 dump_ops_in_flight 2024-09-16T09:42:37.682 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:42:37.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:37.737 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:37.737 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:42:37.775 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:37 smithi070 ceph-mon[30388]: pgmap v928: 168 pgs: 168 active+clean; 6.8 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 951 B/s rd, 66 MiB/s wr, 1.06k op/s 2024-09-16T09:42:37.776 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:37 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2294008732' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:42:37.776 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:37 smithi070 ceph-mon[40691]: pgmap v928: 168 pgs: 168 active+clean; 6.8 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 951 B/s rd, 66 MiB/s wr, 1.06k op/s 2024-09-16T09:42:37.776 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:37 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/2294008732' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:42:37.838 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:42:37.878 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:37.879 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:37.879 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:42:37.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:37 smithi084 ceph-mon[32629]: pgmap v928: 168 pgs: 168 active+clean; 6.8 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 951 B/s rd, 66 MiB/s wr, 1.06k op/s 2024-09-16T09:42:37.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:37 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/2294008732' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:42:37.980 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:42:38.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:38.019 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:38.019 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:42:38.053 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:42:38.100 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 78 16 117498 117482 94.1223 75.75 0.0069424 0.0106198 2024-09-16T09:42:38.120 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:42:38.163 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:38.164 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:38.165 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:42:38.266 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:42:38.308 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:38.309 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:38.309 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:42:38.410 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:42:38.438 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:38.438 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:38.439 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:42:38.540 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:42:38.550 INFO:teuthology.orchestra.run.smithi070.stderr:nodeep-scrub is set 2024-09-16T09:42:38.585 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:38.586 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:38.587 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:42:38.687 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:42:38.700 INFO:teuthology.orchestra.run.smithi084.stdout:ERROR: (22) Invalid argument 2024-09-16T09:42:38.701 INFO:teuthology.orchestra.run.smithi084.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:42:38.728 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:38.729 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:38.729 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:42:38.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:38 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2292599483' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:42:38.743 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:38 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:42:38.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:38 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/2292599483' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:42:38.743 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:38 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:42:38.830 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:42:38.869 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:38.870 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:38.870 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:42:38.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:38 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/2292599483' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:42:38.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:38 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:42:38.972 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:42:39.009 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:39.009 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:39.010 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:42:39.100 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 79 16 118718 118702 93.896 76.25 0.00540997 0.0106455 2024-09-16T09:42:39.111 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:42:39.146 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:42:39.147 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T09:42:39.147 INFO:teuthology.orchestra.run.smithi070.stdout: "epoch": 487, 2024-09-16T09:42:39.147 INFO:teuthology.orchestra.run.smithi070.stdout: "fsid": "a1321834-740c-11ef-bceb-c7b262605968", 2024-09-16T09:42:39.147 INFO:teuthology.orchestra.run.smithi070.stdout: "created": "2024-09-16T09:19:14.121498+0000", 2024-09-16T09:42:39.147 INFO:teuthology.orchestra.run.smithi070.stdout: "modified": "2024-09-16T09:42:38.527450+0000", 2024-09-16T09:42:39.147 INFO:teuthology.orchestra.run.smithi070.stdout: "last_up_change": "2024-09-16T09:24:51.124496+0000", 2024-09-16T09:42:39.147 INFO:teuthology.orchestra.run.smithi070.stdout: "last_in_change": "2024-09-16T09:24:28.432920+0000", 2024-09-16T09:42:39.148 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-16T09:42:39.148 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_num": 5806080, 2024-09-16T09:42:39.148 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_set": [ 2024-09-16T09:42:39.148 INFO:teuthology.orchestra.run.smithi070.stdout: "nodeep-scrub", 2024-09-16T09:42:39.148 INFO:teuthology.orchestra.run.smithi070.stdout: "noscrub", 2024-09-16T09:42:39.148 INFO:teuthology.orchestra.run.smithi070.stdout: "pglog_hardlimit", 2024-09-16T09:42:39.148 INFO:teuthology.orchestra.run.smithi070.stdout: "purged_snapdirs", 2024-09-16T09:42:39.148 INFO:teuthology.orchestra.run.smithi070.stdout: "recovery_deletes", 2024-09-16T09:42:39.148 INFO:teuthology.orchestra.run.smithi070.stdout: "sortbitwise" 2024-09-16T09:42:39.148 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-16T09:42:39.148 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_version": 18, 2024-09-16T09:42:39.148 INFO:teuthology.orchestra.run.smithi070.stdout: "full_ratio": 0.94999998807907104, 2024-09-16T09:42:39.149 INFO:teuthology.orchestra.run.smithi070.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-16T09:42:39.149 INFO:teuthology.orchestra.run.smithi070.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-16T09:42:39.149 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_snapshot": "", 2024-09-16T09:42:39.149 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_max": 11, 2024-09-16T09:42:39.149 INFO:teuthology.orchestra.run.smithi070.stdout: "max_osd": 8, 2024-09-16T09:42:39.149 INFO:teuthology.orchestra.run.smithi070.stdout: "require_min_compat_client": "luminous", 2024-09-16T09:42:39.149 INFO:teuthology.orchestra.run.smithi070.stdout: "min_compat_client": "luminous", 2024-09-16T09:42:39.149 INFO:teuthology.orchestra.run.smithi070.stdout: "require_osd_release": "squid", 2024-09-16T09:42:39.149 INFO:teuthology.orchestra.run.smithi070.stdout: "allow_crimson": false, 2024-09-16T09:42:39.149 INFO:teuthology.orchestra.run.smithi070.stdout: "pools": [ 2024-09-16T09:42:39.149 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.149 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 1, 2024-09-16T09:42:39.149 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": ".mgr", 2024-09-16T09:42:39.149 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2024-09-16T09:22:27.617694+0000", 2024-09-16T09:42:39.150 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2024-09-16T09:42:39.150 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2024-09-16T09:42:39.150 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2024-09-16T09:42:39.150 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 3, 2024-09-16T09:42:39.150 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 2, 2024-09-16T09:42:39.150 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2024-09-16T09:42:39.150 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2024-09-16T09:42:39.150 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2024-09-16T09:42:39.150 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T09:42:39.150 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T09:42:39.150 INFO:teuthology.orchestra.run.smithi070.stdout: "is_stretch_pool": false, 2024-09-16T09:42:39.150 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2024-09-16T09:42:39.150 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2024-09-16T09:42:39.151 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 21, 2024-09-16T09:42:39.151 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 21, 2024-09-16T09:42:39.151 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 21, 2024-09-16T09:42:39.151 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 21, 2024-09-16T09:42:39.151 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 21, 2024-09-16T09:42:39.151 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2024-09-16T09:42:39.151 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2024-09-16T09:42:39.151 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2024-09-16T09:42:39.151 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2024-09-16T09:42:39.151 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2024-09-16T09:42:39.151 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2024-09-16T09:42:39.151 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2024-09-16T09:42:39.152 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.152 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "162", 2024-09-16T09:42:39.152 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2024-09-16T09:42:39.152 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T09:42:39.152 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "160", 2024-09-16T09:42:39.152 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2024-09-16T09:42:39.152 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2024-09-16T09:42:39.152 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2024-09-16T09:42:39.152 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2024-09-16T09:42:39.152 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2024-09-16T09:42:39.153 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2024-09-16T09:42:39.153 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2024-09-16T09:42:39.153 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2024-09-16T09:42:39.153 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2024-09-16T09:42:39.153 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2024-09-16T09:42:39.153 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2024-09-16T09:42:39.153 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2024-09-16T09:42:39.153 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2024-09-16T09:42:39.153 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2024-09-16T09:42:39.153 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2024-09-16T09:42:39.153 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T09:42:39.153 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T09:42:39.153 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T09:42:39.154 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2024-09-16T09:42:39.154 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2024-09-16T09:42:39.154 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2024-09-16T09:42:39.154 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2024-09-16T09:42:39.154 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2024-09-16T09:42:39.154 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.154 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2024-09-16T09:42:39.154 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2024-09-16T09:42:39.154 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2024-09-16T09:42:39.154 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2024-09-16T09:42:39.154 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2024-09-16T09:42:39.154 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T09:42:39.154 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2024-09-16T09:42:39.155 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2024-09-16T09:42:39.155 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2024-09-16T09:42:39.155 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2024-09-16T09:42:39.155 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2024-09-16T09:42:39.155 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2024-09-16T09:42:39.155 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_max": 32, 2024-09-16T09:42:39.155 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_min": 1 2024-09-16T09:42:39.155 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.155 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2024-09-16T09:42:39.155 INFO:teuthology.orchestra.run.smithi070.stdout: "mgr": {} 2024-09-16T09:42:39.155 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.155 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2024-09-16T09:42:39.156 INFO:teuthology.orchestra.run.smithi070.stdout: "score_type": "Fair distribution", 2024-09-16T09:42:39.156 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 1.6699999570846558, 2024-09-16T09:42:39.156 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 1.6699999570846558, 2024-09-16T09:42:39.156 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1.1399999856948853, 2024-09-16T09:42:39.156 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 1.8999999761581421, 2024-09-16T09:42:39.156 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 1.8999999761581421, 2024-09-16T09:42:39.156 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 0.70719337463378906, 2024-09-16T09:42:39.156 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 0.81000000238418579, 2024-09-16T09:42:39.156 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 0.70999997854232788 2024-09-16T09:42:39.156 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:42:39.156 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.156 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.156 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 2, 2024-09-16T09:42:39.156 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "datapool", 2024-09-16T09:42:39.157 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2024-09-16T09:24:59.169244+0000", 2024-09-16T09:42:39.157 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 8193, 2024-09-16T09:42:39.157 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-16T09:42:39.157 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2024-09-16T09:42:39.157 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 3, 2024-09-16T09:42:39.157 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 2, 2024-09-16T09:42:39.157 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2024-09-16T09:42:39.157 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2024-09-16T09:42:39.157 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2024-09-16T09:42:39.157 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T09:42:39.157 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T09:42:39.157 INFO:teuthology.orchestra.run.smithi070.stdout: "is_stretch_pool": false, 2024-09-16T09:42:39.157 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2024-09-16T09:42:39.157 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2024-09-16T09:42:39.158 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 3, 2024-09-16T09:42:39.158 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 3, 2024-09-16T09:42:39.158 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 3, 2024-09-16T09:42:39.158 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 3, 2024-09-16T09:42:39.158 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 3, 2024-09-16T09:42:39.158 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2024-09-16T09:42:39.158 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2024-09-16T09:42:39.158 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2024-09-16T09:42:39.158 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2024-09-16T09:42:39.158 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2024-09-16T09:42:39.158 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2024-09-16T09:42:39.158 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2024-09-16T09:42:39.158 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.158 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "61", 2024-09-16T09:42:39.159 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2024-09-16T09:42:39.159 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T09:42:39.159 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T09:42:39.159 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2024-09-16T09:42:39.159 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2024-09-16T09:42:39.159 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 2, 2024-09-16T09:42:39.159 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 61, 2024-09-16T09:42:39.159 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2024-09-16T09:42:39.159 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2024-09-16T09:42:39.159 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2024-09-16T09:42:39.159 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2024-09-16T09:42:39.159 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2024-09-16T09:42:39.159 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2024-09-16T09:42:39.160 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2024-09-16T09:42:39.160 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2024-09-16T09:42:39.160 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2024-09-16T09:42:39.160 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2024-09-16T09:42:39.160 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2024-09-16T09:42:39.160 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T09:42:39.160 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T09:42:39.160 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T09:42:39.160 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2024-09-16T09:42:39.160 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2024-09-16T09:42:39.160 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2024-09-16T09:42:39.160 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2024-09-16T09:42:39.160 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2024-09-16T09:42:39.160 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.160 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2024-09-16T09:42:39.161 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2024-09-16T09:42:39.161 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2024-09-16T09:42:39.161 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2024-09-16T09:42:39.161 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2024-09-16T09:42:39.161 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T09:42:39.161 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2024-09-16T09:42:39.161 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2024-09-16T09:42:39.161 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2024-09-16T09:42:39.161 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2024-09-16T09:42:39.161 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2024-09-16T09:42:39.161 INFO:teuthology.orchestra.run.smithi070.stdout: "options": {}, 2024-09-16T09:42:39.161 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2024-09-16T09:42:39.161 INFO:teuthology.orchestra.run.smithi070.stdout: "rbd": {} 2024-09-16T09:42:39.161 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.162 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2024-09-16T09:42:39.162 INFO:teuthology.orchestra.run.smithi070.stdout: "score_type": "Fair distribution", 2024-09-16T09:42:39.162 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 2.3299999237060547, 2024-09-16T09:42:39.162 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 2.3299999237060547, 2024-09-16T09:42:39.162 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 0.86000001430511475, 2024-09-16T09:42:39.162 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 2, 2024-09-16T09:42:39.162 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 2, 2024-09-16T09:42:39.162 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 0.70719337463378906, 2024-09-16T09:42:39.162 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 0.81000000238418579, 2024-09-16T09:42:39.162 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 0.70999997854232788 2024-09-16T09:42:39.162 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:42:39.162 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.162 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.162 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 3, 2024-09-16T09:42:39.163 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": ".rgw.root", 2024-09-16T09:42:39.163 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2024-09-16T09:25:00.056119+0000", 2024-09-16T09:42:39.163 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2024-09-16T09:42:39.163 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2024-09-16T09:42:39.163 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2024-09-16T09:42:39.163 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 3, 2024-09-16T09:42:39.163 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 2, 2024-09-16T09:42:39.163 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2024-09-16T09:42:39.163 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2024-09-16T09:42:39.163 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2024-09-16T09:42:39.163 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T09:42:39.163 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T09:42:39.163 INFO:teuthology.orchestra.run.smithi070.stdout: "is_stretch_pool": false, 2024-09-16T09:42:39.163 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2024-09-16T09:42:39.163 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2024-09-16T09:42:39.164 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 32, 2024-09-16T09:42:39.164 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 32, 2024-09-16T09:42:39.164 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 32, 2024-09-16T09:42:39.164 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 32, 2024-09-16T09:42:39.164 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 32, 2024-09-16T09:42:39.164 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2024-09-16T09:42:39.164 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2024-09-16T09:42:39.164 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2024-09-16T09:42:39.165 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2024-09-16T09:42:39.165 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2024-09-16T09:42:39.165 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2024-09-16T09:42:39.165 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2024-09-16T09:42:39.165 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.165 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "60", 2024-09-16T09:42:39.165 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2024-09-16T09:42:39.165 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T09:42:39.165 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T09:42:39.165 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2024-09-16T09:42:39.165 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2024-09-16T09:42:39.165 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2024-09-16T09:42:39.165 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2024-09-16T09:42:39.166 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2024-09-16T09:42:39.166 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2024-09-16T09:42:39.166 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2024-09-16T09:42:39.166 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2024-09-16T09:42:39.166 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2024-09-16T09:42:39.166 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2024-09-16T09:42:39.166 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2024-09-16T09:42:39.166 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2024-09-16T09:42:39.166 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2024-09-16T09:42:39.166 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2024-09-16T09:42:39.166 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2024-09-16T09:42:39.166 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T09:42:39.166 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T09:42:39.166 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T09:42:39.167 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2024-09-16T09:42:39.167 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2024-09-16T09:42:39.167 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2024-09-16T09:42:39.167 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2024-09-16T09:42:39.167 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2024-09-16T09:42:39.167 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.167 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2024-09-16T09:42:39.167 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2024-09-16T09:42:39.167 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2024-09-16T09:42:39.167 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2024-09-16T09:42:39.167 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2024-09-16T09:42:39.167 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T09:42:39.167 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2024-09-16T09:42:39.167 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2024-09-16T09:42:39.168 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2024-09-16T09:42:39.168 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2024-09-16T09:42:39.168 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2024-09-16T09:42:39.168 INFO:teuthology.orchestra.run.smithi070.stdout: "options": {}, 2024-09-16T09:42:39.168 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2024-09-16T09:42:39.168 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw": {} 2024-09-16T09:42:39.168 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.168 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2024-09-16T09:42:39.168 INFO:teuthology.orchestra.run.smithi070.stdout: "score_type": "Fair distribution", 2024-09-16T09:42:39.168 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 1.75, 2024-09-16T09:42:39.168 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 1.75, 2024-09-16T09:42:39.168 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1.1399999856948853, 2024-09-16T09:42:39.168 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 2, 2024-09-16T09:42:39.168 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 2, 2024-09-16T09:42:39.169 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 0.70719337463378906, 2024-09-16T09:42:39.169 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 0.81000000238418579, 2024-09-16T09:42:39.169 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 0.70999997854232788 2024-09-16T09:42:39.169 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:42:39.169 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.169 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.169 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 4, 2024-09-16T09:42:39.169 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "default.rgw.log", 2024-09-16T09:42:39.169 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2024-09-16T09:25:01.770016+0000", 2024-09-16T09:42:39.169 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2024-09-16T09:42:39.169 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2024-09-16T09:42:39.170 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2024-09-16T09:42:39.170 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 3, 2024-09-16T09:42:39.170 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 2, 2024-09-16T09:42:39.170 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2024-09-16T09:42:39.170 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2024-09-16T09:42:39.170 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2024-09-16T09:42:39.170 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T09:42:39.170 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T09:42:39.170 INFO:teuthology.orchestra.run.smithi070.stdout: "is_stretch_pool": false, 2024-09-16T09:42:39.170 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2024-09-16T09:42:39.170 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2024-09-16T09:42:39.170 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 32, 2024-09-16T09:42:39.170 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 32, 2024-09-16T09:42:39.170 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 32, 2024-09-16T09:42:39.171 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 32, 2024-09-16T09:42:39.171 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 32, 2024-09-16T09:42:39.171 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2024-09-16T09:42:39.171 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2024-09-16T09:42:39.171 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2024-09-16T09:42:39.171 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2024-09-16T09:42:39.171 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2024-09-16T09:42:39.171 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2024-09-16T09:42:39.171 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2024-09-16T09:42:39.171 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.171 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "62", 2024-09-16T09:42:39.171 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2024-09-16T09:42:39.171 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T09:42:39.171 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T09:42:39.172 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2024-09-16T09:42:39.172 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2024-09-16T09:42:39.172 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2024-09-16T09:42:39.172 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2024-09-16T09:42:39.172 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2024-09-16T09:42:39.172 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2024-09-16T09:42:39.172 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2024-09-16T09:42:39.172 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2024-09-16T09:42:39.172 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2024-09-16T09:42:39.172 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2024-09-16T09:42:39.172 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2024-09-16T09:42:39.172 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2024-09-16T09:42:39.172 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2024-09-16T09:42:39.173 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2024-09-16T09:42:39.173 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2024-09-16T09:42:39.173 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T09:42:39.173 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T09:42:39.173 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T09:42:39.173 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2024-09-16T09:42:39.173 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2024-09-16T09:42:39.173 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2024-09-16T09:42:39.173 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2024-09-16T09:42:39.173 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2024-09-16T09:42:39.173 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.173 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2024-09-16T09:42:39.174 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2024-09-16T09:42:39.174 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2024-09-16T09:42:39.174 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2024-09-16T09:42:39.174 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2024-09-16T09:42:39.174 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T09:42:39.174 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2024-09-16T09:42:39.174 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2024-09-16T09:42:39.174 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2024-09-16T09:42:39.174 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2024-09-16T09:42:39.174 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2024-09-16T09:42:39.174 INFO:teuthology.orchestra.run.smithi070.stdout: "options": {}, 2024-09-16T09:42:39.174 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2024-09-16T09:42:39.174 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw": {} 2024-09-16T09:42:39.174 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.174 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2024-09-16T09:42:39.175 INFO:teuthology.orchestra.run.smithi070.stdout: "score_type": "Fair distribution", 2024-09-16T09:42:39.175 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 6.8499999046325684, 2024-09-16T09:42:39.175 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 2.4100000858306885, 2024-09-16T09:42:39.175 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1.1399999856948853, 2024-09-16T09:42:39.175 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 7.809999942779541, 2024-09-16T09:42:39.175 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 2.75, 2024-09-16T09:42:39.175 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 0.70719337463378906, 2024-09-16T09:42:39.175 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 0.81000000238418579, 2024-09-16T09:42:39.175 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 0.70999997854232788, 2024-09-16T09:42:39.175 INFO:teuthology.orchestra.run.smithi070.stdout: "error_message": "pool 4 has acting primaries on OSD(s) with primary affinity 0, read balance score is not accurate" 2024-09-16T09:42:39.175 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:42:39.175 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.175 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.175 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 5, 2024-09-16T09:42:39.176 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "default.rgw.control", 2024-09-16T09:42:39.176 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2024-09-16T09:25:03.725555+0000", 2024-09-16T09:42:39.176 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2024-09-16T09:42:39.176 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2024-09-16T09:42:39.176 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2024-09-16T09:42:39.176 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 3, 2024-09-16T09:42:39.176 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 2, 2024-09-16T09:42:39.176 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2024-09-16T09:42:39.176 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2024-09-16T09:42:39.176 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2024-09-16T09:42:39.176 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T09:42:39.176 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T09:42:39.176 INFO:teuthology.orchestra.run.smithi070.stdout: "is_stretch_pool": false, 2024-09-16T09:42:39.176 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2024-09-16T09:42:39.177 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2024-09-16T09:42:39.177 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 32, 2024-09-16T09:42:39.177 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 32, 2024-09-16T09:42:39.177 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 32, 2024-09-16T09:42:39.177 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 32, 2024-09-16T09:42:39.177 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 32, 2024-09-16T09:42:39.177 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2024-09-16T09:42:39.177 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2024-09-16T09:42:39.177 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2024-09-16T09:42:39.177 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2024-09-16T09:42:39.177 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2024-09-16T09:42:39.177 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2024-09-16T09:42:39.177 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2024-09-16T09:42:39.178 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.178 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "64", 2024-09-16T09:42:39.178 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2024-09-16T09:42:39.178 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T09:42:39.178 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T09:42:39.178 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2024-09-16T09:42:39.178 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2024-09-16T09:42:39.178 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2024-09-16T09:42:39.178 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2024-09-16T09:42:39.178 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2024-09-16T09:42:39.178 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2024-09-16T09:42:39.178 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2024-09-16T09:42:39.178 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2024-09-16T09:42:39.178 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2024-09-16T09:42:39.178 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2024-09-16T09:42:39.178 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2024-09-16T09:42:39.179 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2024-09-16T09:42:39.179 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2024-09-16T09:42:39.179 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2024-09-16T09:42:39.179 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2024-09-16T09:42:39.179 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T09:42:39.179 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T09:42:39.179 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T09:42:39.179 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2024-09-16T09:42:39.179 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2024-09-16T09:42:39.179 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2024-09-16T09:42:39.179 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2024-09-16T09:42:39.179 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2024-09-16T09:42:39.179 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.179 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2024-09-16T09:42:39.179 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2024-09-16T09:42:39.180 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2024-09-16T09:42:39.180 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2024-09-16T09:42:39.180 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2024-09-16T09:42:39.180 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T09:42:39.180 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2024-09-16T09:42:39.180 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2024-09-16T09:42:39.180 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2024-09-16T09:42:39.180 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2024-09-16T09:42:39.180 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2024-09-16T09:42:39.180 INFO:teuthology.orchestra.run.smithi070.stdout: "options": {}, 2024-09-16T09:42:39.180 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2024-09-16T09:42:39.180 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw": {} 2024-09-16T09:42:39.181 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:39.181 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:39.182 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:42:39.182 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.182 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2024-09-16T09:42:39.182 INFO:teuthology.orchestra.run.smithi070.stdout: "score_type": "Fair distribution", 2024-09-16T09:42:39.182 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 1.75, 2024-09-16T09:42:39.182 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 1.75, 2024-09-16T09:42:39.182 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1.1399999856948853, 2024-09-16T09:42:39.182 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 2, 2024-09-16T09:42:39.182 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 2, 2024-09-16T09:42:39.183 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 0.70719337463378906, 2024-09-16T09:42:39.183 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 0.81000000238418579, 2024-09-16T09:42:39.183 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 0.70999997854232788 2024-09-16T09:42:39.183 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:42:39.183 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.183 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.183 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 6, 2024-09-16T09:42:39.183 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "default.rgw.meta", 2024-09-16T09:42:39.183 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2024-09-16T09:25:05.773530+0000", 2024-09-16T09:42:39.183 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2024-09-16T09:42:39.183 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2024-09-16T09:42:39.183 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2024-09-16T09:42:39.183 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 3, 2024-09-16T09:42:39.183 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 2, 2024-09-16T09:42:39.184 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2024-09-16T09:42:39.184 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2024-09-16T09:42:39.184 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2024-09-16T09:42:39.184 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T09:42:39.184 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T09:42:39.184 INFO:teuthology.orchestra.run.smithi070.stdout: "is_stretch_pool": false, 2024-09-16T09:42:39.184 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2024-09-16T09:42:39.184 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2024-09-16T09:42:39.184 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 32, 2024-09-16T09:42:39.184 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 32, 2024-09-16T09:42:39.184 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 32, 2024-09-16T09:42:39.184 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 32, 2024-09-16T09:42:39.184 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 32, 2024-09-16T09:42:39.185 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2024-09-16T09:42:39.185 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2024-09-16T09:42:39.185 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2024-09-16T09:42:39.185 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2024-09-16T09:42:39.185 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2024-09-16T09:42:39.185 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2024-09-16T09:42:39.185 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2024-09-16T09:42:39.185 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.185 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "66", 2024-09-16T09:42:39.185 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2024-09-16T09:42:39.185 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T09:42:39.185 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T09:42:39.185 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2024-09-16T09:42:39.186 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2024-09-16T09:42:39.186 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2024-09-16T09:42:39.186 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2024-09-16T09:42:39.186 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2024-09-16T09:42:39.186 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2024-09-16T09:42:39.186 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2024-09-16T09:42:39.186 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2024-09-16T09:42:39.186 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2024-09-16T09:42:39.186 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2024-09-16T09:42:39.186 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2024-09-16T09:42:39.186 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2024-09-16T09:42:39.186 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2024-09-16T09:42:39.186 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2024-09-16T09:42:39.187 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2024-09-16T09:42:39.187 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T09:42:39.187 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T09:42:39.187 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T09:42:39.187 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2024-09-16T09:42:39.187 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2024-09-16T09:42:39.187 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2024-09-16T09:42:39.187 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2024-09-16T09:42:39.187 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2024-09-16T09:42:39.187 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.187 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2024-09-16T09:42:39.187 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2024-09-16T09:42:39.188 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2024-09-16T09:42:39.188 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2024-09-16T09:42:39.188 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2024-09-16T09:42:39.188 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T09:42:39.188 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2024-09-16T09:42:39.188 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2024-09-16T09:42:39.188 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2024-09-16T09:42:39.188 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2024-09-16T09:42:39.188 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2024-09-16T09:42:39.188 INFO:teuthology.orchestra.run.smithi070.stdout: "options": { 2024-09-16T09:42:39.188 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_bias": 4 2024-09-16T09:42:39.188 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.188 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2024-09-16T09:42:39.189 INFO:teuthology.orchestra.run.smithi070.stdout: "rgw": {} 2024-09-16T09:42:39.189 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.189 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2024-09-16T09:42:39.189 INFO:teuthology.orchestra.run.smithi070.stdout: "score_type": "Fair distribution", 2024-09-16T09:42:39.189 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 1.5399999618530273, 2024-09-16T09:42:39.189 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 1.5399999618530273, 2024-09-16T09:42:39.189 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1.1399999856948853, 2024-09-16T09:42:39.189 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 1.75, 2024-09-16T09:42:39.189 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 1.75, 2024-09-16T09:42:39.189 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 0.70719337463378906, 2024-09-16T09:42:39.189 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 0.81000000238418579, 2024-09-16T09:42:39.189 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 0.70999997854232788 2024-09-16T09:42:39.189 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:42:39.190 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.190 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.190 INFO:teuthology.orchestra.run.smithi070.stdout: "pool": 11, 2024-09-16T09:42:39.190 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_name": "unique_pool_4", 2024-09-16T09:42:39.190 INFO:teuthology.orchestra.run.smithi070.stdout: "create_time": "2024-09-16T09:41:11.917610+0000", 2024-09-16T09:42:39.190 INFO:teuthology.orchestra.run.smithi070.stdout: "flags": 1, 2024-09-16T09:42:39.190 INFO:teuthology.orchestra.run.smithi070.stdout: "flags_names": "hashpspool", 2024-09-16T09:42:39.190 INFO:teuthology.orchestra.run.smithi070.stdout: "type": 1, 2024-09-16T09:42:39.190 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 3, 2024-09-16T09:42:39.190 INFO:teuthology.orchestra.run.smithi070.stdout: "min_size": 2, 2024-09-16T09:42:39.190 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_rule": 0, 2024-09-16T09:42:39.190 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_count": 0, 2024-09-16T09:42:39.190 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_target": 0, 2024-09-16T09:42:39.191 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_barrier": 0, 2024-09-16T09:42:39.191 INFO:teuthology.orchestra.run.smithi070.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-16T09:42:39.191 INFO:teuthology.orchestra.run.smithi070.stdout: "is_stretch_pool": false, 2024-09-16T09:42:39.192 INFO:teuthology.orchestra.run.smithi070.stdout: "object_hash": 2, 2024-09-16T09:42:39.192 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_autoscale_mode": "off", 2024-09-16T09:42:39.192 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num": 16, 2024-09-16T09:42:39.192 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num": 16, 2024-09-16T09:42:39.192 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_placement_num_target": 16, 2024-09-16T09:42:39.192 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_target": 16, 2024-09-16T09:42:39.192 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_num_pending": 16, 2024-09-16T09:42:39.192 INFO:teuthology.orchestra.run.smithi070.stdout: "last_pg_merge_meta": { 2024-09-16T09:42:39.192 INFO:teuthology.orchestra.run.smithi070.stdout: "source_pgid": "0.0", 2024-09-16T09:42:39.192 INFO:teuthology.orchestra.run.smithi070.stdout: "ready_epoch": 0, 2024-09-16T09:42:39.192 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_started": 0, 2024-09-16T09:42:39.192 INFO:teuthology.orchestra.run.smithi070.stdout: "last_epoch_clean": 0, 2024-09-16T09:42:39.193 INFO:teuthology.orchestra.run.smithi070.stdout: "source_version": "0'0", 2024-09-16T09:42:39.193 INFO:teuthology.orchestra.run.smithi070.stdout: "target_version": "0'0" 2024-09-16T09:42:39.193 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.193 INFO:teuthology.orchestra.run.smithi070.stdout: "last_change": "481", 2024-09-16T09:42:39.193 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend": "0", 2024-09-16T09:42:39.193 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-16T09:42:39.193 INFO:teuthology.orchestra.run.smithi070.stdout: "last_force_op_resend_preluminous": "0", 2024-09-16T09:42:39.193 INFO:teuthology.orchestra.run.smithi070.stdout: "auid": 0, 2024-09-16T09:42:39.193 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_mode": "selfmanaged", 2024-09-16T09:42:39.193 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_seq": 0, 2024-09-16T09:42:39.193 INFO:teuthology.orchestra.run.smithi070.stdout: "snap_epoch": 0, 2024-09-16T09:42:39.193 INFO:teuthology.orchestra.run.smithi070.stdout: "pool_snaps": [], 2024-09-16T09:42:39.194 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps": "[]", 2024-09-16T09:42:39.194 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_bytes": 0, 2024-09-16T09:42:39.194 INFO:teuthology.orchestra.run.smithi070.stdout: "quota_max_objects": 0, 2024-09-16T09:42:39.194 INFO:teuthology.orchestra.run.smithi070.stdout: "tiers": [], 2024-09-16T09:42:39.194 INFO:teuthology.orchestra.run.smithi070.stdout: "tier_of": -1, 2024-09-16T09:42:39.194 INFO:teuthology.orchestra.run.smithi070.stdout: "read_tier": -1, 2024-09-16T09:42:39.194 INFO:teuthology.orchestra.run.smithi070.stdout: "write_tier": -1, 2024-09-16T09:42:39.194 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_mode": "none", 2024-09-16T09:42:39.194 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_bytes": 0, 2024-09-16T09:42:39.194 INFO:teuthology.orchestra.run.smithi070.stdout: "target_max_objects": 0, 2024-09-16T09:42:39.194 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-16T09:42:39.194 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-16T09:42:39.194 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-16T09:42:39.194 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_flush_age": 0, 2024-09-16T09:42:39.195 INFO:teuthology.orchestra.run.smithi070.stdout: "cache_min_evict_age": 0, 2024-09-16T09:42:39.195 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profile": "", 2024-09-16T09:42:39.195 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_params": { 2024-09-16T09:42:39.195 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "none" 2024-09-16T09:42:39.195 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.195 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_period": 0, 2024-09-16T09:42:39.195 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_count": 0, 2024-09-16T09:42:39.195 INFO:teuthology.orchestra.run.smithi070.stdout: "use_gmt_hitset": true, 2024-09-16T09:42:39.195 INFO:teuthology.orchestra.run.smithi070.stdout: "min_read_recency_for_promote": 0, 2024-09-16T09:42:39.195 INFO:teuthology.orchestra.run.smithi070.stdout: "min_write_recency_for_promote": 0, 2024-09-16T09:42:39.195 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_grade_decay_rate": 0, 2024-09-16T09:42:39.195 INFO:teuthology.orchestra.run.smithi070.stdout: "hit_set_search_last_n": 0, 2024-09-16T09:42:39.195 INFO:teuthology.orchestra.run.smithi070.stdout: "grade_table": [], 2024-09-16T09:42:39.195 INFO:teuthology.orchestra.run.smithi070.stdout: "stripe_width": 0, 2024-09-16T09:42:39.195 INFO:teuthology.orchestra.run.smithi070.stdout: "expected_num_objects": 0, 2024-09-16T09:42:39.196 INFO:teuthology.orchestra.run.smithi070.stdout: "fast_read": false, 2024-09-16T09:42:39.196 INFO:teuthology.orchestra.run.smithi070.stdout: "options": {}, 2024-09-16T09:42:39.196 INFO:teuthology.orchestra.run.smithi070.stdout: "application_metadata": { 2024-09-16T09:42:39.196 INFO:teuthology.orchestra.run.smithi070.stdout: "rados": {} 2024-09-16T09:42:39.196 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.196 INFO:teuthology.orchestra.run.smithi070.stdout: "read_balance": { 2024-09-16T09:42:39.196 INFO:teuthology.orchestra.run.smithi070.stdout: "score_type": "Fair distribution", 2024-09-16T09:42:39.196 INFO:teuthology.orchestra.run.smithi070.stdout: "score_acting": 1.75, 2024-09-16T09:42:39.196 INFO:teuthology.orchestra.run.smithi070.stdout: "score_stable": 1.75, 2024-09-16T09:42:39.196 INFO:teuthology.orchestra.run.smithi070.stdout: "optimal_score": 1.1399999856948853, 2024-09-16T09:42:39.196 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_acting": 2, 2024-09-16T09:42:39.196 INFO:teuthology.orchestra.run.smithi070.stdout: "raw_score_stable": 2, 2024-09-16T09:42:39.196 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity_weighted": 0.70719337463378906, 2024-09-16T09:42:39.197 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity": 0.81000000238418579, 2024-09-16T09:42:39.197 INFO:teuthology.orchestra.run.smithi070.stdout: "average_primary_affinity_weighted": 0.70999997854232788 2024-09-16T09:42:39.197 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:42:39.197 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:42:39.197 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-16T09:42:39.197 INFO:teuthology.orchestra.run.smithi070.stdout: "osds": [ 2024-09-16T09:42:39.197 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.197 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 0, 2024-09-16T09:42:39.197 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "7ef288c8-3bf0-4814-8e86-f70a7de041ce", 2024-09-16T09:42:39.197 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2024-09-16T09:42:39.197 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2024-09-16T09:42:39.197 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2024-09-16T09:42:39.197 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2024-09-16T09:42:39.198 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2024-09-16T09:42:39.198 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2024-09-16T09:42:39.198 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 8, 2024-09-16T09:42:39.198 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 454, 2024-09-16T09:42:39.198 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2024-09-16T09:42:39.198 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2024-09-16T09:42:39.198 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2024-09-16T09:42:39.198 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:42:39.198 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.198 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:42:39.198 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6802", 2024-09-16T09:42:39.198 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2013075781 2024-09-16T09:42:39.198 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.198 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.198 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:42:39.199 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6803", 2024-09-16T09:42:39.199 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2013075781 2024-09-16T09:42:39.199 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:42:39.199 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:42:39.199 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.199 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2024-09-16T09:42:39.199 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:42:39.199 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.199 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:42:39.199 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6804", 2024-09-16T09:42:39.199 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2013075781 2024-09-16T09:42:39.199 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.199 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.199 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:42:39.200 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6805", 2024-09-16T09:42:39.200 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2013075781 2024-09-16T09:42:39.200 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:42:39.200 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:42:39.200 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.200 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2024-09-16T09:42:39.200 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:42:39.200 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.200 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:42:39.200 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6808", 2024-09-16T09:42:39.200 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2013075781 2024-09-16T09:42:39.200 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.201 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.201 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:42:39.201 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6809", 2024-09-16T09:42:39.201 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2013075781 2024-09-16T09:42:39.201 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:42:39.201 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:42:39.201 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.201 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2024-09-16T09:42:39.201 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:42:39.201 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.201 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:42:39.201 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6806", 2024-09-16T09:42:39.201 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2013075781 2024-09-16T09:42:39.202 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.202 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.202 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:42:39.202 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6807", 2024-09-16T09:42:39.202 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2013075781 2024-09-16T09:42:39.202 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:42:39.202 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:42:39.202 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.202 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6803/2013075781", 2024-09-16T09:42:39.202 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6805/2013075781", 2024-09-16T09:42:39.202 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6809/2013075781", 2024-09-16T09:42:39.202 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6807/2013075781", 2024-09-16T09:42:39.202 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2024-09-16T09:42:39.202 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2024-09-16T09:42:39.203 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2024-09-16T09:42:39.203 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:42:39.203 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.203 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.203 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 1, 2024-09-16T09:42:39.203 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "474d41d9-295a-450c-8f27-ce487a14678b", 2024-09-16T09:42:39.203 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2024-09-16T09:42:39.203 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2024-09-16T09:42:39.203 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2024-09-16T09:42:39.203 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 0, 2024-09-16T09:42:39.203 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2024-09-16T09:42:39.203 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2024-09-16T09:42:39.203 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 13, 2024-09-16T09:42:39.204 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 362, 2024-09-16T09:42:39.204 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2024-09-16T09:42:39.204 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2024-09-16T09:42:39.204 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2024-09-16T09:42:39.204 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:42:39.204 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.204 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:42:39.204 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6810", 2024-09-16T09:42:39.204 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1670012115 2024-09-16T09:42:39.204 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.204 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.204 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:42:39.205 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6811", 2024-09-16T09:42:39.205 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1670012115 2024-09-16T09:42:39.205 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:42:39.205 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:42:39.205 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.205 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2024-09-16T09:42:39.205 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:42:39.205 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.205 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:42:39.205 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6812", 2024-09-16T09:42:39.205 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1670012115 2024-09-16T09:42:39.205 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.205 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.206 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:42:39.206 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6813", 2024-09-16T09:42:39.206 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1670012115 2024-09-16T09:42:39.206 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:42:39.206 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:42:39.206 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.206 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2024-09-16T09:42:39.206 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:42:39.206 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.206 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:42:39.206 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6816", 2024-09-16T09:42:39.206 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1670012115 2024-09-16T09:42:39.206 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.206 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.207 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:42:39.207 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6817", 2024-09-16T09:42:39.207 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1670012115 2024-09-16T09:42:39.207 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:42:39.207 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:42:39.207 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.207 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2024-09-16T09:42:39.207 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:42:39.207 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.207 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:42:39.207 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6814", 2024-09-16T09:42:39.207 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1670012115 2024-09-16T09:42:39.207 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.207 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.207 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:42:39.208 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6815", 2024-09-16T09:42:39.208 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1670012115 2024-09-16T09:42:39.208 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:42:39.208 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:42:39.208 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.208 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6811/1670012115", 2024-09-16T09:42:39.208 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6813/1670012115", 2024-09-16T09:42:39.208 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6817/1670012115", 2024-09-16T09:42:39.208 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6815/1670012115", 2024-09-16T09:42:39.208 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2024-09-16T09:42:39.208 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2024-09-16T09:42:39.208 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2024-09-16T09:42:39.209 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:42:39.209 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.209 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.209 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 2, 2024-09-16T09:42:39.209 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "5b4ca4e8-cecc-4163-b291-c9209b02c4b7", 2024-09-16T09:42:39.209 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2024-09-16T09:42:39.209 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2024-09-16T09:42:39.209 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2024-09-16T09:42:39.209 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 0.1810150146484375, 2024-09-16T09:42:39.209 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2024-09-16T09:42:39.209 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2024-09-16T09:42:39.209 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 18, 2024-09-16T09:42:39.209 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 454, 2024-09-16T09:42:39.210 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2024-09-16T09:42:39.210 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2024-09-16T09:42:39.210 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2024-09-16T09:42:39.210 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:42:39.210 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.210 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:42:39.210 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6818", 2024-09-16T09:42:39.210 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3601682336 2024-09-16T09:42:39.210 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.210 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.210 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:42:39.210 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6819", 2024-09-16T09:42:39.210 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3601682336 2024-09-16T09:42:39.210 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:42:39.211 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:42:39.211 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.211 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2024-09-16T09:42:39.211 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:42:39.211 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.211 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:42:39.211 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6820", 2024-09-16T09:42:39.211 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3601682336 2024-09-16T09:42:39.211 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.211 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.211 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:42:39.211 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6821", 2024-09-16T09:42:39.211 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3601682336 2024-09-16T09:42:39.211 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:42:39.212 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:42:39.212 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.212 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2024-09-16T09:42:39.212 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:42:39.212 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.212 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:42:39.212 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6824", 2024-09-16T09:42:39.212 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3601682336 2024-09-16T09:42:39.212 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.212 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.212 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:42:39.212 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6825", 2024-09-16T09:42:39.212 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3601682336 2024-09-16T09:42:39.212 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:42:39.212 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:42:39.213 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.213 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2024-09-16T09:42:39.213 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:42:39.213 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.213 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:42:39.213 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6822", 2024-09-16T09:42:39.213 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3601682336 2024-09-16T09:42:39.213 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.213 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.213 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:42:39.213 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6823", 2024-09-16T09:42:39.213 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 3601682336 2024-09-16T09:42:39.213 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:42:39.214 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:42:39.214 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.214 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6819/3601682336", 2024-09-16T09:42:39.214 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6821/3601682336", 2024-09-16T09:42:39.214 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6825/3601682336", 2024-09-16T09:42:39.214 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6823/3601682336", 2024-09-16T09:42:39.214 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2024-09-16T09:42:39.214 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2024-09-16T09:42:39.214 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2024-09-16T09:42:39.214 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:42:39.214 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.214 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.214 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 3, 2024-09-16T09:42:39.214 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "ffb8d527-1b58-4ecd-808c-47d4ba1bf387", 2024-09-16T09:42:39.215 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2024-09-16T09:42:39.215 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2024-09-16T09:42:39.215 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2024-09-16T09:42:39.215 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 0.476531982421875, 2024-09-16T09:42:39.215 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2024-09-16T09:42:39.215 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2024-09-16T09:42:39.215 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 26, 2024-09-16T09:42:39.215 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 454, 2024-09-16T09:42:39.215 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2024-09-16T09:42:39.215 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2024-09-16T09:42:39.215 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2024-09-16T09:42:39.215 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:42:39.215 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.215 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:42:39.215 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6826", 2024-09-16T09:42:39.216 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2038083007 2024-09-16T09:42:39.216 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.216 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.216 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:42:39.216 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6827", 2024-09-16T09:42:39.216 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2038083007 2024-09-16T09:42:39.216 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:42:39.216 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:42:39.216 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.216 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2024-09-16T09:42:39.216 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:42:39.216 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.216 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:42:39.216 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6828", 2024-09-16T09:42:39.216 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2038083007 2024-09-16T09:42:39.217 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.217 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.217 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:42:39.217 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6829", 2024-09-16T09:42:39.217 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2038083007 2024-09-16T09:42:39.217 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:42:39.217 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:42:39.217 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.217 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2024-09-16T09:42:39.217 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:42:39.217 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.217 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:42:39.217 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6832", 2024-09-16T09:42:39.217 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2038083007 2024-09-16T09:42:39.218 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.218 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.218 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:42:39.218 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6833", 2024-09-16T09:42:39.218 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2038083007 2024-09-16T09:42:39.218 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:42:39.218 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:42:39.218 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.218 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2024-09-16T09:42:39.218 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:42:39.218 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.219 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:42:39.219 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6830", 2024-09-16T09:42:39.219 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2038083007 2024-09-16T09:42:39.219 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.219 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.220 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:42:39.220 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.70:6831", 2024-09-16T09:42:39.220 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2038083007 2024-09-16T09:42:39.220 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:42:39.220 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:42:39.220 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.220 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.70:6827/2038083007", 2024-09-16T09:42:39.220 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.70:6829/2038083007", 2024-09-16T09:42:39.220 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.70:6833/2038083007", 2024-09-16T09:42:39.220 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.70:6831/2038083007", 2024-09-16T09:42:39.220 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2024-09-16T09:42:39.220 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2024-09-16T09:42:39.220 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2024-09-16T09:42:39.221 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:42:39.221 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.221 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.221 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 4, 2024-09-16T09:42:39.221 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "a946760b-4d91-4a4b-8e9b-a07a50589211", 2024-09-16T09:42:39.221 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2024-09-16T09:42:39.221 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2024-09-16T09:42:39.221 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2024-09-16T09:42:39.221 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2024-09-16T09:42:39.221 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2024-09-16T09:42:39.221 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2024-09-16T09:42:39.221 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 32, 2024-09-16T09:42:39.221 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 454, 2024-09-16T09:42:39.222 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2024-09-16T09:42:39.222 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2024-09-16T09:42:39.222 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2024-09-16T09:42:39.222 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:42:39.222 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.222 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:42:39.222 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6800", 2024-09-16T09:42:39.222 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2250892710 2024-09-16T09:42:39.222 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.222 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.222 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:42:39.222 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6801", 2024-09-16T09:42:39.222 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2250892710 2024-09-16T09:42:39.222 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:42:39.223 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:42:39.223 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.223 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2024-09-16T09:42:39.223 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:42:39.223 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.223 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:42:39.223 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6802", 2024-09-16T09:42:39.223 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2250892710 2024-09-16T09:42:39.223 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.223 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.223 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:42:39.223 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6803", 2024-09-16T09:42:39.223 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2250892710 2024-09-16T09:42:39.224 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:42:39.224 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:42:39.224 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.224 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2024-09-16T09:42:39.224 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:42:39.224 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.224 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:42:39.224 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6806", 2024-09-16T09:42:39.224 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2250892710 2024-09-16T09:42:39.224 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.224 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.224 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:42:39.224 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6807", 2024-09-16T09:42:39.224 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2250892710 2024-09-16T09:42:39.225 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:42:39.225 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:42:39.225 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.225 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2024-09-16T09:42:39.225 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:42:39.225 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.225 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:42:39.225 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6804", 2024-09-16T09:42:39.225 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2250892710 2024-09-16T09:42:39.225 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.225 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.225 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:42:39.225 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6805", 2024-09-16T09:42:39.225 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 2250892710 2024-09-16T09:42:39.225 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:42:39.226 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:42:39.226 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.226 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.84:6801/2250892710", 2024-09-16T09:42:39.226 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.84:6803/2250892710", 2024-09-16T09:42:39.226 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.84:6807/2250892710", 2024-09-16T09:42:39.226 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.84:6805/2250892710", 2024-09-16T09:42:39.226 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2024-09-16T09:42:39.226 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2024-09-16T09:42:39.226 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2024-09-16T09:42:39.226 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:42:39.226 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.226 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.226 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 5, 2024-09-16T09:42:39.226 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "67681760-75a9-4846-946e-a935d42c5e05", 2024-09-16T09:42:39.227 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2024-09-16T09:42:39.227 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2024-09-16T09:42:39.227 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2024-09-16T09:42:39.227 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2024-09-16T09:42:39.227 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2024-09-16T09:42:39.227 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2024-09-16T09:42:39.227 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 39, 2024-09-16T09:42:39.227 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 454, 2024-09-16T09:42:39.227 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2024-09-16T09:42:39.227 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2024-09-16T09:42:39.227 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2024-09-16T09:42:39.227 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:42:39.227 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.227 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:42:39.227 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6808", 2024-09-16T09:42:39.228 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1803945348 2024-09-16T09:42:39.228 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.228 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.228 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:42:39.228 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6809", 2024-09-16T09:42:39.228 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1803945348 2024-09-16T09:42:39.228 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:42:39.228 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:42:39.228 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.228 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2024-09-16T09:42:39.228 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:42:39.228 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.228 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:42:39.228 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6810", 2024-09-16T09:42:39.229 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1803945348 2024-09-16T09:42:39.229 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.229 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.229 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:42:39.229 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6811", 2024-09-16T09:42:39.229 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1803945348 2024-09-16T09:42:39.229 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:42:39.229 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:42:39.229 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.229 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2024-09-16T09:42:39.229 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:42:39.229 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.229 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:42:39.229 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6814", 2024-09-16T09:42:39.230 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1803945348 2024-09-16T09:42:39.230 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.230 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.230 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:42:39.230 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6815", 2024-09-16T09:42:39.230 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1803945348 2024-09-16T09:42:39.230 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:42:39.230 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:42:39.230 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.230 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2024-09-16T09:42:39.230 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:42:39.230 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.230 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:42:39.230 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6812", 2024-09-16T09:42:39.230 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1803945348 2024-09-16T09:42:39.231 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.231 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.231 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:42:39.231 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6813", 2024-09-16T09:42:39.231 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1803945348 2024-09-16T09:42:39.231 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:42:39.231 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:42:39.231 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.231 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.84:6809/1803945348", 2024-09-16T09:42:39.231 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.84:6811/1803945348", 2024-09-16T09:42:39.231 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.84:6815/1803945348", 2024-09-16T09:42:39.231 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.84:6813/1803945348", 2024-09-16T09:42:39.231 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2024-09-16T09:42:39.231 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2024-09-16T09:42:39.231 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2024-09-16T09:42:39.232 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:42:39.232 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.232 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.232 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 6, 2024-09-16T09:42:39.232 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "7c8f0152-f539-4b7e-8113-91dd7529d3a1", 2024-09-16T09:42:39.232 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2024-09-16T09:42:39.232 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2024-09-16T09:42:39.232 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2024-09-16T09:42:39.232 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2024-09-16T09:42:39.232 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2024-09-16T09:42:39.232 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2024-09-16T09:42:39.232 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 46, 2024-09-16T09:42:39.233 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 470, 2024-09-16T09:42:39.233 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2024-09-16T09:42:39.233 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2024-09-16T09:42:39.233 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2024-09-16T09:42:39.233 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:42:39.233 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.233 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:42:39.233 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6816", 2024-09-16T09:42:39.233 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 610727189 2024-09-16T09:42:39.233 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.233 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.233 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:42:39.233 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6817", 2024-09-16T09:42:39.234 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 610727189 2024-09-16T09:42:39.234 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:42:39.234 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:42:39.234 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.234 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2024-09-16T09:42:39.234 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:42:39.234 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.234 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:42:39.234 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6818", 2024-09-16T09:42:39.234 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 610727189 2024-09-16T09:42:39.234 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.234 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.234 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:42:39.234 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6819", 2024-09-16T09:42:39.235 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 610727189 2024-09-16T09:42:39.235 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:42:39.235 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:42:39.235 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.235 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2024-09-16T09:42:39.235 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:42:39.235 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.235 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:42:39.235 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6822", 2024-09-16T09:42:39.235 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 610727189 2024-09-16T09:42:39.235 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.235 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.235 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:42:39.235 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6823", 2024-09-16T09:42:39.236 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 610727189 2024-09-16T09:42:39.236 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:42:39.236 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:42:39.236 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.236 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2024-09-16T09:42:39.236 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:42:39.236 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.236 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:42:39.236 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6820", 2024-09-16T09:42:39.236 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 610727189 2024-09-16T09:42:39.236 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.236 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.236 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:42:39.236 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6821", 2024-09-16T09:42:39.237 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 610727189 2024-09-16T09:42:39.237 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:42:39.237 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:42:39.237 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.237 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.84:6817/610727189", 2024-09-16T09:42:39.237 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.84:6819/610727189", 2024-09-16T09:42:39.237 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.84:6823/610727189", 2024-09-16T09:42:39.237 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.84:6821/610727189", 2024-09-16T09:42:39.237 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2024-09-16T09:42:39.237 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2024-09-16T09:42:39.237 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2024-09-16T09:42:39.237 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:42:39.237 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.238 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.238 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 7, 2024-09-16T09:42:39.238 INFO:teuthology.orchestra.run.smithi070.stdout: "uuid": "945efaba-225e-49af-aefd-b4d58ca8093b", 2024-09-16T09:42:39.238 INFO:teuthology.orchestra.run.smithi070.stdout: "up": 1, 2024-09-16T09:42:39.238 INFO:teuthology.orchestra.run.smithi070.stdout: "in": 1, 2024-09-16T09:42:39.238 INFO:teuthology.orchestra.run.smithi070.stdout: "weight": 1, 2024-09-16T09:42:39.238 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_affinity": 1, 2024-09-16T09:42:39.238 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_begin": 0, 2024-09-16T09:42:39.238 INFO:teuthology.orchestra.run.smithi070.stdout: "last_clean_end": 0, 2024-09-16T09:42:39.238 INFO:teuthology.orchestra.run.smithi070.stdout: "up_from": 53, 2024-09-16T09:42:39.238 INFO:teuthology.orchestra.run.smithi070.stdout: "up_thru": 454, 2024-09-16T09:42:39.238 INFO:teuthology.orchestra.run.smithi070.stdout: "down_at": 0, 2024-09-16T09:42:39.238 INFO:teuthology.orchestra.run.smithi070.stdout: "lost_at": 0, 2024-09-16T09:42:39.238 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addrs": { 2024-09-16T09:42:39.239 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:42:39.239 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.239 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:42:39.239 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6824", 2024-09-16T09:42:39.239 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1002252531 2024-09-16T09:42:39.239 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.239 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.239 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:42:39.239 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6825", 2024-09-16T09:42:39.239 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1002252531 2024-09-16T09:42:39.239 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:42:39.239 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:42:39.239 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.239 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addrs": { 2024-09-16T09:42:39.239 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:42:39.240 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.240 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:42:39.240 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6826", 2024-09-16T09:42:39.240 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1002252531 2024-09-16T09:42:39.240 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.240 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.240 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:42:39.240 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6827", 2024-09-16T09:42:39.240 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1002252531 2024-09-16T09:42:39.240 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:42:39.240 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:42:39.240 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.240 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addrs": { 2024-09-16T09:42:39.241 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:42:39.241 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.241 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:42:39.241 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6830", 2024-09-16T09:42:39.241 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1002252531 2024-09-16T09:42:39.241 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.241 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.241 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:42:39.241 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6831", 2024-09-16T09:42:39.241 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1002252531 2024-09-16T09:42:39.241 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:42:39.241 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:42:39.241 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.242 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addrs": { 2024-09-16T09:42:39.242 INFO:teuthology.orchestra.run.smithi070.stdout: "addrvec": [ 2024-09-16T09:42:39.242 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.242 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v2", 2024-09-16T09:42:39.242 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6828", 2024-09-16T09:42:39.242 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1002252531 2024-09-16T09:42:39.242 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.242 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.242 INFO:teuthology.orchestra.run.smithi070.stdout: "type": "v1", 2024-09-16T09:42:39.242 INFO:teuthology.orchestra.run.smithi070.stdout: "addr": "172.21.15.84:6829", 2024-09-16T09:42:39.242 INFO:teuthology.orchestra.run.smithi070.stdout: "nonce": 1002252531 2024-09-16T09:42:39.242 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:42:39.242 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:42:39.242 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.243 INFO:teuthology.orchestra.run.smithi070.stdout: "public_addr": "172.21.15.84:6825/1002252531", 2024-09-16T09:42:39.243 INFO:teuthology.orchestra.run.smithi070.stdout: "cluster_addr": "172.21.15.84:6827/1002252531", 2024-09-16T09:42:39.243 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_back_addr": "172.21.15.84:6831/1002252531", 2024-09-16T09:42:39.243 INFO:teuthology.orchestra.run.smithi070.stdout: "heartbeat_front_addr": "172.21.15.84:6829/1002252531", 2024-09-16T09:42:39.243 INFO:teuthology.orchestra.run.smithi070.stdout: "state": [ 2024-09-16T09:42:39.243 INFO:teuthology.orchestra.run.smithi070.stdout: "exists", 2024-09-16T09:42:39.243 INFO:teuthology.orchestra.run.smithi070.stdout: "up" 2024-09-16T09:42:39.243 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:42:39.243 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:42:39.243 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-16T09:42:39.243 INFO:teuthology.orchestra.run.smithi070.stdout: "osd_xinfo": [ 2024-09-16T09:42:39.243 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.243 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 0, 2024-09-16T09:42:39.243 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2024-09-16T09:42:39.244 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2024-09-16T09:42:39.244 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2024-09-16T09:42:39.244 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540719139924082687, 2024-09-16T09:42:39.244 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2024-09-16T09:42:39.244 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2024-09-16T09:42:34.217354+0000", 2024-09-16T09:42:39.244 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2024-09-16T09:42:39.244 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.244 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.244 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 1, 2024-09-16T09:42:39.244 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2024-09-16T09:42:39.244 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2024-09-16T09:42:39.244 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2024-09-16T09:42:39.244 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540719139924082687, 2024-09-16T09:42:39.245 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2024-09-16T09:42:39.245 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2024-09-16T09:42:17.171480+0000", 2024-09-16T09:42:39.245 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2024-09-16T09:42:39.245 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.245 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.245 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 2, 2024-09-16T09:42:39.245 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2024-09-16T09:42:39.245 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2024-09-16T09:42:39.245 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2024-09-16T09:42:39.246 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540719139924082687, 2024-09-16T09:42:39.246 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2024-09-16T09:42:39.246 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2024-09-16T09:42:27.282369+0000", 2024-09-16T09:42:39.246 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2024-09-16T09:42:39.246 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.246 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.246 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 3, 2024-09-16T09:42:39.246 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2024-09-16T09:42:39.246 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2024-09-16T09:42:39.246 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2024-09-16T09:42:39.246 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540719139924082687, 2024-09-16T09:42:39.246 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2024-09-16T09:42:39.246 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2024-09-16T09:42:15.794466+0000", 2024-09-16T09:42:39.247 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2024-09-16T09:42:39.247 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.247 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.247 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 4, 2024-09-16T09:42:39.247 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2024-09-16T09:42:39.247 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2024-09-16T09:42:39.247 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2024-09-16T09:42:39.247 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540719139924082687, 2024-09-16T09:42:39.247 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2024-09-16T09:42:39.247 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2024-09-16T09:42:07.271810+0000", 2024-09-16T09:42:39.247 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2024-09-16T09:42:39.247 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.247 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.247 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 5, 2024-09-16T09:42:39.248 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2024-09-16T09:42:39.248 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2024-09-16T09:42:39.248 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2024-09-16T09:42:39.248 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540719139924082687, 2024-09-16T09:42:39.248 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2024-09-16T09:42:39.248 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2024-09-16T09:42:31.818184+0000", 2024-09-16T09:42:39.248 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2024-09-16T09:42:39.248 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.248 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.248 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 6, 2024-09-16T09:42:39.248 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2024-09-16T09:42:39.248 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2024-09-16T09:42:39.248 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2024-09-16T09:42:39.249 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540719139924082687, 2024-09-16T09:42:39.249 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2024-09-16T09:42:39.249 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2024-09-16T09:42:12.564894+0000", 2024-09-16T09:42:39.249 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2024-09-16T09:42:39.249 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.249 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.249 INFO:teuthology.orchestra.run.smithi070.stdout: "osd": 7, 2024-09-16T09:42:39.249 INFO:teuthology.orchestra.run.smithi070.stdout: "down_stamp": "0.000000", 2024-09-16T09:42:39.249 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_probability": 0, 2024-09-16T09:42:39.249 INFO:teuthology.orchestra.run.smithi070.stdout: "laggy_interval": 0, 2024-09-16T09:42:39.249 INFO:teuthology.orchestra.run.smithi070.stdout: "features": 4540719139924082687, 2024-09-16T09:42:39.249 INFO:teuthology.orchestra.run.smithi070.stdout: "old_weight": 0, 2024-09-16T09:42:39.249 INFO:teuthology.orchestra.run.smithi070.stdout: "last_purged_snaps_scrub": "2024-09-16T09:42:29.901448+0000", 2024-09-16T09:42:39.250 INFO:teuthology.orchestra.run.smithi070.stdout: "dead_epoch": 0 2024-09-16T09:42:39.250 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:42:39.250 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-16T09:42:39.250 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap": [ 2024-09-16T09:42:39.250 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.250 INFO:teuthology.orchestra.run.smithi070.stdout: "pgid": "3.1d", 2024-09-16T09:42:39.250 INFO:teuthology.orchestra.run.smithi070.stdout: "osds": [ 2024-09-16T09:42:39.250 INFO:teuthology.orchestra.run.smithi070.stdout: 1, 2024-09-16T09:42:39.250 INFO:teuthology.orchestra.run.smithi070.stdout: 0, 2024-09-16T09:42:39.250 INFO:teuthology.orchestra.run.smithi070.stdout: 7 2024-09-16T09:42:39.250 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:42:39.250 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.250 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.250 INFO:teuthology.orchestra.run.smithi070.stdout: "pgid": "4.b", 2024-09-16T09:42:39.251 INFO:teuthology.orchestra.run.smithi070.stdout: "osds": [ 2024-09-16T09:42:39.251 INFO:teuthology.orchestra.run.smithi070.stdout: 0, 2024-09-16T09:42:39.251 INFO:teuthology.orchestra.run.smithi070.stdout: 5, 2024-09-16T09:42:39.251 INFO:teuthology.orchestra.run.smithi070.stdout: 1 2024-09-16T09:42:39.251 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:42:39.251 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.251 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.251 INFO:teuthology.orchestra.run.smithi070.stdout: "pgid": "4.19", 2024-09-16T09:42:39.251 INFO:teuthology.orchestra.run.smithi070.stdout: "osds": [ 2024-09-16T09:42:39.251 INFO:teuthology.orchestra.run.smithi070.stdout: 4, 2024-09-16T09:42:39.251 INFO:teuthology.orchestra.run.smithi070.stdout: 7, 2024-09-16T09:42:39.251 INFO:teuthology.orchestra.run.smithi070.stdout: 5 2024-09-16T09:42:39.251 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:42:39.251 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.252 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.252 INFO:teuthology.orchestra.run.smithi070.stdout: "pgid": "5.1a", 2024-09-16T09:42:39.252 INFO:teuthology.orchestra.run.smithi070.stdout: "osds": [ 2024-09-16T09:42:39.252 INFO:teuthology.orchestra.run.smithi070.stdout: 6, 2024-09-16T09:42:39.252 INFO:teuthology.orchestra.run.smithi070.stdout: 7, 2024-09-16T09:42:39.252 INFO:teuthology.orchestra.run.smithi070.stdout: 4 2024-09-16T09:42:39.252 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:42:39.252 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:42:39.252 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-16T09:42:39.252 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap_items": [ 2024-09-16T09:42:39.252 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.252 INFO:teuthology.orchestra.run.smithi070.stdout: "pgid": "2.1", 2024-09-16T09:42:39.252 INFO:teuthology.orchestra.run.smithi070.stdout: "mappings": [ 2024-09-16T09:42:39.253 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.253 INFO:teuthology.orchestra.run.smithi070.stdout: "from": 3, 2024-09-16T09:42:39.253 INFO:teuthology.orchestra.run.smithi070.stdout: "to": 0 2024-09-16T09:42:39.253 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.253 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.253 INFO:teuthology.orchestra.run.smithi070.stdout: "from": 2, 2024-09-16T09:42:39.253 INFO:teuthology.orchestra.run.smithi070.stdout: "to": 1 2024-09-16T09:42:39.253 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:42:39.253 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:42:39.253 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.253 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.253 INFO:teuthology.orchestra.run.smithi070.stdout: "pgid": "3.1b", 2024-09-16T09:42:39.253 INFO:teuthology.orchestra.run.smithi070.stdout: "mappings": [ 2024-09-16T09:42:39.254 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.254 INFO:teuthology.orchestra.run.smithi070.stdout: "from": 0, 2024-09-16T09:42:39.254 INFO:teuthology.orchestra.run.smithi070.stdout: "to": 6 2024-09-16T09:42:39.254 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.254 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.254 INFO:teuthology.orchestra.run.smithi070.stdout: "from": 4, 2024-09-16T09:42:39.254 INFO:teuthology.orchestra.run.smithi070.stdout: "to": 7 2024-09-16T09:42:39.254 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:42:39.254 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:42:39.254 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.254 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.254 INFO:teuthology.orchestra.run.smithi070.stdout: "pgid": "4.f", 2024-09-16T09:42:39.254 INFO:teuthology.orchestra.run.smithi070.stdout: "mappings": [ 2024-09-16T09:42:39.254 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.255 INFO:teuthology.orchestra.run.smithi070.stdout: "from": 1, 2024-09-16T09:42:39.255 INFO:teuthology.orchestra.run.smithi070.stdout: "to": 2 2024-09-16T09:42:39.255 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:42:39.255 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:42:39.255 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.255 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.255 INFO:teuthology.orchestra.run.smithi070.stdout: "pgid": "4.13", 2024-09-16T09:42:39.255 INFO:teuthology.orchestra.run.smithi070.stdout: "mappings": [ 2024-09-16T09:42:39.255 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.255 INFO:teuthology.orchestra.run.smithi070.stdout: "from": 1, 2024-09-16T09:42:39.255 INFO:teuthology.orchestra.run.smithi070.stdout: "to": 5 2024-09-16T09:42:39.255 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:42:39.255 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:42:39.256 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.256 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.256 INFO:teuthology.orchestra.run.smithi070.stdout: "pgid": "4.17", 2024-09-16T09:42:39.256 INFO:teuthology.orchestra.run.smithi070.stdout: "mappings": [ 2024-09-16T09:42:39.256 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.256 INFO:teuthology.orchestra.run.smithi070.stdout: "from": 1, 2024-09-16T09:42:39.256 INFO:teuthology.orchestra.run.smithi070.stdout: "to": 2 2024-09-16T09:42:39.256 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:42:39.256 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:42:39.256 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.256 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.256 INFO:teuthology.orchestra.run.smithi070.stdout: "pgid": "5.10", 2024-09-16T09:42:39.256 INFO:teuthology.orchestra.run.smithi070.stdout: "mappings": [ 2024-09-16T09:42:39.257 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:42:39.257 INFO:teuthology.orchestra.run.smithi070.stdout: "from": 7, 2024-09-16T09:42:39.257 INFO:teuthology.orchestra.run.smithi070.stdout: "to": 4 2024-09-16T09:42:39.257 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:42:39.257 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:42:39.257 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:42:39.257 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-16T09:42:39.257 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_upmap_primaries": [], 2024-09-16T09:42:39.257 INFO:teuthology.orchestra.run.smithi070.stdout: "pg_temp": [], 2024-09-16T09:42:39.257 INFO:teuthology.orchestra.run.smithi070.stdout: "primary_temp": [], 2024-09-16T09:42:39.257 INFO:teuthology.orchestra.run.smithi070.stdout: "blocklist": { 2024-09-16T09:42:39.257 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/125491368": "2024-09-17T09:25:31.465167+0000", 2024-09-16T09:42:39.257 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/3354284852": "2024-09-17T09:19:43.205989+0000", 2024-09-16T09:42:39.257 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/3284625547": "2024-09-17T09:25:31.465167+0000", 2024-09-16T09:42:39.258 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/186565817": "2024-09-17T09:19:43.205989+0000", 2024-09-16T09:42:39.258 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/2213852865": "2024-09-17T09:19:43.205989+0000", 2024-09-16T09:42:39.258 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/1812373760": "2024-09-17T09:20:09.306330+0000", 2024-09-16T09:42:39.258 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:6800/837338094": "2024-09-17T09:19:43.205989+0000", 2024-09-16T09:42:39.258 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:6800/1876661229": "2024-09-17T09:25:31.465167+0000", 2024-09-16T09:42:39.258 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/3351688485": "2024-09-17T09:20:09.306330+0000", 2024-09-16T09:42:39.258 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:6800/3871553358": "2024-09-17T09:20:09.306330+0000", 2024-09-16T09:42:39.258 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/1230482871": "2024-09-17T09:25:31.465167+0000", 2024-09-16T09:42:39.258 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:6801/837338094": "2024-09-17T09:19:43.205989+0000", 2024-09-16T09:42:39.258 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:6801/3871553358": "2024-09-17T09:20:09.306330+0000", 2024-09-16T09:42:39.258 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/1403001486": "2024-09-17T09:20:09.306330+0000", 2024-09-16T09:42:39.258 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/1723643739": "2024-09-17T09:25:31.465167+0000", 2024-09-16T09:42:39.258 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:6801/1876661229": "2024-09-17T09:25:31.465167+0000", 2024-09-16T09:42:39.259 INFO:teuthology.orchestra.run.smithi070.stdout: "172.21.15.70:0/3456417396": "2024-09-17T09:25:31.465167+0000" 2024-09-16T09:42:39.259 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.259 INFO:teuthology.orchestra.run.smithi070.stdout: "range_blocklist": {}, 2024-09-16T09:42:39.259 INFO:teuthology.orchestra.run.smithi070.stdout: "erasure_code_profiles": { 2024-09-16T09:42:39.259 INFO:teuthology.orchestra.run.smithi070.stdout: "default": { 2024-09-16T09:42:39.259 INFO:teuthology.orchestra.run.smithi070.stdout: "crush-failure-domain": "osd", 2024-09-16T09:42:39.259 INFO:teuthology.orchestra.run.smithi070.stdout: "k": "2", 2024-09-16T09:42:39.259 INFO:teuthology.orchestra.run.smithi070.stdout: "m": "1", 2024-09-16T09:42:39.259 INFO:teuthology.orchestra.run.smithi070.stdout: "plugin": "jerasure", 2024-09-16T09:42:39.259 INFO:teuthology.orchestra.run.smithi070.stdout: "technique": "reed_sol_van" 2024-09-16T09:42:39.259 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:42:39.259 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:42:39.259 INFO:teuthology.orchestra.run.smithi070.stdout: "removed_snaps_queue": [], 2024-09-16T09:42:39.259 INFO:teuthology.orchestra.run.smithi070.stdout: "new_removed_snaps": [], 2024-09-16T09:42:39.260 INFO:teuthology.orchestra.run.smithi070.stdout: "new_purged_snaps": [], 2024-09-16T09:42:39.260 INFO:teuthology.orchestra.run.smithi070.stdout: "crush_node_flags": {}, 2024-09-16T09:42:39.260 INFO:teuthology.orchestra.run.smithi070.stdout: "device_class_flags": {}, 2024-09-16T09:42:39.260 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode": { 2024-09-16T09:42:39.260 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode_enabled": false, 2024-09-16T09:42:39.260 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_bucket_count": 0, 2024-09-16T09:42:39.260 INFO:teuthology.orchestra.run.smithi070.stdout: "degraded_stretch_mode": 0, 2024-09-16T09:42:39.260 INFO:teuthology.orchestra.run.smithi070.stdout: "recovering_stretch_mode": 0, 2024-09-16T09:42:39.260 INFO:teuthology.orchestra.run.smithi070.stdout: "stretch_mode_bucket": 0 2024-09-16T09:42:39.260 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:42:39.260 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T09:42:39.283 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:42:39.321 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:39.322 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:39.322 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:42:39.423 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:42:39.427 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.5 dump_blocked_ops 2024-09-16T09:42:39.460 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:39.460 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:39.460 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:42:39.562 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:42:39.600 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:39.601 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:39.601 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:42:39.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:39 smithi084 ceph-mon[32629]: pgmap v929: 168 pgs: 168 active+clean; 6.9 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1.7 KiB/s rd, 76 MiB/s wr, 1.21k op/s 2024-09-16T09:42:39.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:39 smithi084 ceph-mon[32629]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:42:39.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:39 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-16T09:42:39.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:39 smithi084 ceph-mon[32629]: osdmap e487: 8 total, 8 up, 8 in 2024-09-16T09:42:39.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:39 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/2179235553' entity='client.admin' cmd={"prefix": "osd dump", "format": "json-pretty"} : dispatch 2024-09-16T09:42:39.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:39 smithi084 ceph-mon[32629]: osdmap e488: 8 total, 8 up, 8 in 2024-09-16T09:42:39.702 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:42:39.733 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:39.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:39.734 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:42:39.835 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:42:39.870 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:39.870 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:42:39.871 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:39.871 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:42:39.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:39 smithi070 ceph-mon[30388]: pgmap v929: 168 pgs: 168 active+clean; 6.9 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1.7 KiB/s rd, 76 MiB/s wr, 1.21k op/s 2024-09-16T09:42:39.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:39 smithi070 ceph-mon[30388]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:42:39.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:39 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-16T09:42:39.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:39 smithi070 ceph-mon[30388]: osdmap e487: 8 total, 8 up, 8 in 2024-09-16T09:42:39.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:39 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2179235553' entity='client.admin' cmd={"prefix": "osd dump", "format": "json-pretty"} : dispatch 2024-09-16T09:42:39.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:39 smithi070 ceph-mon[30388]: osdmap e488: 8 total, 8 up, 8 in 2024-09-16T09:42:39.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:39 smithi070 ceph-mon[40691]: pgmap v929: 168 pgs: 168 active+clean; 6.9 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1.7 KiB/s rd, 76 MiB/s wr, 1.21k op/s 2024-09-16T09:42:39.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:39 smithi070 ceph-mon[40691]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:42:39.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:39 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-16T09:42:39.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:39 smithi070 ceph-mon[40691]: osdmap e487: 8 total, 8 up, 8 in 2024-09-16T09:42:39.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:39 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/2179235553' entity='client.admin' cmd={"prefix": "osd dump", "format": "json-pretty"} : dispatch 2024-09-16T09:42:39.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:39 smithi070 ceph-mon[40691]: osdmap e488: 8 total, 8 up, 8 in 2024-09-16T09:42:39.973 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:42:40.005 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:40.006 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:40.006 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:42:40.100 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:2024-09-16T09:42:40.100020+0000 min lat: 0.00125032 max lat: 0.55794 avg lat: 0.0106682 2024-09-16T09:42:40.100 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-16T09:42:40.100 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 80 16 119959 119943 93.6918 77.5625 0.0145127 0.0106682 2024-09-16T09:42:40.107 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:42:40.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:40.145 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:40.145 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:42:40.246 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:42:40.256 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T09:42:40.282 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:40.282 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:40.283 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:42:40.383 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:42:40.427 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:40.428 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:40.428 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:42:40.529 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:42:40.544 INFO:teuthology.orchestra.run.smithi084.stdout:ERROR: (22) Invalid argument 2024-09-16T09:42:40.544 INFO:teuthology.orchestra.run.smithi084.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:42:40.563 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:40.564 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:40.564 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:42:40.665 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:42:40.701 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:40.702 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:40.703 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:42:40.730 INFO:tasks.thrashosds.thrasher:j is {'epoch': 487, 'fsid': 'a1321834-740c-11ef-bceb-c7b262605968', 'created': '2024-09-16T09:19:14.121498+0000', 'modified': '2024-09-16T09:42:38.527450+0000', 'last_up_change': '2024-09-16T09:24:51.124496+0000', 'last_in_change': '2024-09-16T09:24:28.432920+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 18, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 11, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T09:22:27.617694+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 21, 'pg_placement_num': 21, 'pg_placement_num_target': 21, 'pg_num_target': 21, 'pg_num_pending': 21, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '162', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '160', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6699999570846558, 'score_stable': 1.6699999570846558, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 1.899999976158142, 'raw_score_stable': 1.899999976158142, 'primary_affinity_weighted': 0.7071933746337891, 'average_primary_affinity': 0.8100000023841858, 'average_primary_affinity_weighted': 0.7099999785423279}}, {'pool': 2, 'pool_name': 'datapool', 'create_time': '2024-09-16T09:24:59.169244+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 3, 'pg_placement_num': 3, 'pg_placement_num_target': 3, 'pg_num_target': 3, 'pg_num_pending': 3, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 61, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.3299999237060547, 'score_stable': 2.3299999237060547, 'optimal_score': 0.8600000143051147, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.7071933746337891, 'average_primary_affinity': 0.8100000023841858, 'average_primary_affinity_weighted': 0.7099999785423279}}, {'pool': 3, 'pool_name': '.rgw.root', 'create_time': '2024-09-16T09:25:00.056119+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.7071933746337891, 'average_primary_affinity': 0.8100000023841858, 'average_primary_affinity_weighted': 0.7099999785423279}}, {'pool': 4, 'pool_name': 'default.rgw.log', 'create_time': '2024-09-16T09:25:01.770016+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '62', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.849999904632568, 'score_stable': 2.4100000858306885, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 7.809999942779541, 'raw_score_stable': 2.75, 'primary_affinity_weighted': 0.7071933746337891, 'average_primary_affinity': 0.8100000023841858, 'average_primary_affinity_weighted': 0.7099999785423279, 'error_message': 'pool 4 has acting primaries on OSD(s) with primary affinity 0, read balance score is not accurate'}}, {'pool': 5, 'pool_name': 'default.rgw.control', 'create_time': '2024-09-16T09:25:03.725555+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.7071933746337891, 'average_primary_affinity': 0.8100000023841858, 'average_primary_affinity_weighted': 0.7099999785423279}}, {'pool': 6, 'pool_name': 'default.rgw.meta', 'create_time': '2024-09-16T09:25:05.773530+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5399999618530273, 'score_stable': 1.5399999618530273, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 0.7071933746337891, 'average_primary_affinity': 0.8100000023841858, 'average_primary_affinity_weighted': 0.7099999785423279}}, {'pool': 11, 'pool_name': 'unique_pool_4', 'create_time': '2024-09-16T09:41:11.917610+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 16, 'pg_placement_num': 16, 'pg_placement_num_target': 16, 'pg_num_target': 16, 'pg_num_pending': 16, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '481', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rados': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.7071933746337891, 'average_primary_affinity': 0.8100000023841858, 'average_primary_affinity_weighted': 0.7099999785423279}}], 'osds': [{'osd': 0, 'uuid': '7ef288c8-3bf0-4814-8e86-f70a7de041ce', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 8, 'up_thru': 454, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6802', 'nonce': 2013075781}, {'type': 'v1', 'addr': '172.21.15.70:6803', 'nonce': 2013075781}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6804', 'nonce': 2013075781}, {'type': 'v1', 'addr': '172.21.15.70:6805', 'nonce': 2013075781}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6808', 'nonce': 2013075781}, {'type': 'v1', 'addr': '172.21.15.70:6809', 'nonce': 2013075781}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6806', 'nonce': 2013075781}, {'type': 'v1', 'addr': '172.21.15.70:6807', 'nonce': 2013075781}]}, 'public_addr': '172.21.15.70:6803/2013075781', 'cluster_addr': '172.21.15.70:6805/2013075781', 'heartbeat_back_addr': '172.21.15.70:6809/2013075781', 'heartbeat_front_addr': '172.21.15.70:6807/2013075781', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '474d41d9-295a-450c-8f27-ce487a14678b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 362, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6810', 'nonce': 1670012115}, {'type': 'v1', 'addr': '172.21.15.70:6811', 'nonce': 1670012115}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6812', 'nonce': 1670012115}, {'type': 'v1', 'addr': '172.21.15.70:6813', 'nonce': 1670012115}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6816', 'nonce': 1670012115}, {'type': 'v1', 'addr': '172.21.15.70:6817', 'nonce': 1670012115}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6814', 'nonce': 1670012115}, {'type': 'v1', 'addr': '172.21.15.70:6815', 'nonce': 1670012115}]}, 'public_addr': '172.21.15.70:6811/1670012115', 'cluster_addr': '172.21.15.70:6813/1670012115', 'heartbeat_back_addr': '172.21.15.70:6817/1670012115', 'heartbeat_front_addr': '172.21.15.70:6815/1670012115', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '5b4ca4e8-cecc-4163-b291-c9209b02c4b7', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.1810150146484375, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 18, 'up_thru': 454, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6818', 'nonce': 3601682336}, {'type': 'v1', 'addr': '172.21.15.70:6819', 'nonce': 3601682336}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6820', 'nonce': 3601682336}, {'type': 'v1', 'addr': '172.21.15.70:6821', 'nonce': 3601682336}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6824', 'nonce': 3601682336}, {'type': 'v1', 'addr': '172.21.15.70:6825', 'nonce': 3601682336}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6822', 'nonce': 3601682336}, {'type': 'v1', 'addr': '172.21.15.70:6823', 'nonce': 3601682336}]}, 'public_addr': '172.21.15.70:6819/3601682336', 'cluster_addr': '172.21.15.70:6821/3601682336', 'heartbeat_back_addr': '172.21.15.70:6825/3601682336', 'heartbeat_front_addr': '172.21.15.70:6823/3601682336', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'ffb8d527-1b58-4ecd-808c-47d4ba1bf387', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.476531982421875, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 26, 'up_thru': 454, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6826', 'nonce': 2038083007}, {'type': 'v1', 'addr': '172.21.15.70:6827', 'nonce': 2038083007}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6828', 'nonce': 2038083007}, {'type': 'v1', 'addr': '172.21.15.70:6829', 'nonce': 2038083007}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6832', 'nonce': 2038083007}, {'type': 'v1', 'addr': '172.21.15.70:6833', 'nonce': 2038083007}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.70:6830', 'nonce': 2038083007}, {'type': 'v1', 'addr': '172.21.15.70:6831', 'nonce': 2038083007}]}, 'public_addr': '172.21.15.70:6827/2038083007', 'cluster_addr': '172.21.15.70:6829/2038083007', 'heartbeat_back_addr': '172.21.15.70:6833/2038083007', 'heartbeat_front_addr': '172.21.15.70:6831/2038083007', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'a946760b-4d91-4a4b-8e9b-a07a50589211', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 32, 'up_thru': 454, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6800', 'nonce': 2250892710}, {'type': 'v1', 'addr': '172.21.15.84:6801', 'nonce': 2250892710}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6802', 'nonce': 2250892710}, {'type': 'v1', 'addr': '172.21.15.84:6803', 'nonce': 2250892710}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6806', 'nonce': 2250892710}, {'type': 'v1', 'addr': '172.21.15.84:6807', 'nonce': 2250892710}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6804', 'nonce': 2250892710}, {'type': 'v1', 'addr': '172.21.15.84:6805', 'nonce': 2250892710}]}, 'public_addr': '172.21.15.84:6801/2250892710', 'cluster_addr': '172.21.15.84:6803/2250892710', 'heartbeat_back_addr': '172.21.15.84:6807/2250892710', 'heartbeat_front_addr': '172.21.15.84:6805/2250892710', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '67681760-75a9-4846-946e-a935d42c5e05', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 39, 'up_thru': 454, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6808', 'nonce': 1803945348}, {'type': 'v1', 'addr': '172.21.15.84:6809', 'nonce': 1803945348}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6810', 'nonce': 1803945348}, {'type': 'v1', 'addr': '172.21.15.84:6811', 'nonce': 1803945348}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6814', 'nonce': 1803945348}, {'type': 'v1', 'addr': '172.21.15.84:6815', 'nonce': 1803945348}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6812', 'nonce': 1803945348}, {'type': 'v1', 'addr': '172.21.15.84:6813', 'nonce': 1803945348}]}, 'public_addr': '172.21.15.84:6809/1803945348', 'cluster_addr': '172.21.15.84:6811/1803945348', 'heartbeat_back_addr': '172.21.15.84:6815/1803945348', 'heartbeat_front_addr': '172.21.15.84:6813/1803945348', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '7c8f0152-f539-4b7e-8113-91dd7529d3a1', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 46, 'up_thru': 470, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6816', 'nonce': 610727189}, {'type': 'v1', 'addr': '172.21.15.84:6817', 'nonce': 610727189}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6818', 'nonce': 610727189}, {'type': 'v1', 'addr': '172.21.15.84:6819', 'nonce': 610727189}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6822', 'nonce': 610727189}, {'type': 'v1', 'addr': '172.21.15.84:6823', 'nonce': 610727189}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6820', 'nonce': 610727189}, {'type': 'v1', 'addr': '172.21.15.84:6821', 'nonce': 610727189}]}, 'public_addr': '172.21.15.84:6817/610727189', 'cluster_addr': '172.21.15.84:6819/610727189', 'heartbeat_back_addr': '172.21.15.84:6823/610727189', 'heartbeat_front_addr': '172.21.15.84:6821/610727189', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '945efaba-225e-49af-aefd-b4d58ca8093b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 53, 'up_thru': 454, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6824', 'nonce': 1002252531}, {'type': 'v1', 'addr': '172.21.15.84:6825', 'nonce': 1002252531}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6826', 'nonce': 1002252531}, {'type': 'v1', 'addr': '172.21.15.84:6827', 'nonce': 1002252531}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6830', 'nonce': 1002252531}, {'type': 'v1', 'addr': '172.21.15.84:6831', 'nonce': 1002252531}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6828', 'nonce': 1002252531}, {'type': 'v1', 'addr': '172.21.15.84:6829', 'nonce': 1002252531}]}, 'public_addr': '172.21.15.84:6825/1002252531', 'cluster_addr': '172.21.15.84:6827/1002252531', 'heartbeat_back_addr': '172.21.15.84:6831/1002252531', 'heartbeat_front_addr': '172.21.15.84:6829/1002252531', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T09:42:34.217354+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T09:42:17.171480+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T09:42:27.282369+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T09:42:15.794466+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T09:42:07.271810+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T09:42:31.818184+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T09:42:12.564894+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-16T09:42:29.901448+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.1d', 'osds': [1, 0, 7]}, {'pgid': '4.b', 'osds': [0, 5, 1]}, {'pgid': '4.19', 'osds': [4, 7, 5]}, {'pgid': '5.1a', 'osds': [6, 7, 4]}], 'pg_upmap_items': [{'pgid': '2.1', 'mappings': [{'from': 3, 'to': 0}, {'from': 2, 'to': 1}]}, {'pgid': '3.1b', 'mappings': [{'from': 0, 'to': 6}, {'from': 4, 'to': 7}]}, {'pgid': '4.f', 'mappings': [{'from': 1, 'to': 2}]}, {'pgid': '4.13', 'mappings': [{'from': 1, 'to': 5}]}, {'pgid': '4.17', 'mappings': [{'from': 1, 'to': 2}]}, {'pgid': '5.10', 'mappings': [{'from': 7, 'to': 4}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {'172.21.15.70:0/125491368': '2024-09-17T09:25:31.465167+0000', '172.21.15.70:0/3354284852': '2024-09-17T09:19:43.205989+0000', '172.21.15.70:0/3284625547': '2024-09-17T09:25:31.465167+0000', '172.21.15.70:0/186565817': '2024-09-17T09:19:43.205989+0000', '172.21.15.70:0/2213852865': '2024-09-17T09:19:43.205989+0000', '172.21.15.70:0/1812373760': '2024-09-17T09:20:09.306330+0000', '172.21.15.70:6800/837338094': '2024-09-17T09:19:43.205989+0000', '172.21.15.70:6800/1876661229': '2024-09-17T09:25:31.465167+0000', '172.21.15.70:0/3351688485': '2024-09-17T09:20:09.306330+0000', '172.21.15.70:6800/3871553358': '2024-09-17T09:20:09.306330+0000', '172.21.15.70:0/1230482871': '2024-09-17T09:25:31.465167+0000', '172.21.15.70:6801/837338094': '2024-09-17T09:19:43.205989+0000', '172.21.15.70:6801/3871553358': '2024-09-17T09:20:09.306330+0000', '172.21.15.70:0/1403001486': '2024-09-17T09:20:09.306330+0000', '172.21.15.70:0/1723643739': '2024-09-17T09:25:31.465167+0000', '172.21.15.70:6801/1876661229': '2024-09-17T09:25:31.465167+0000', '172.21.15.70:0/3456417396': '2024-09-17T09:25:31.465167+0000'}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-16T09:42:40.731 INFO:tasks.thrashosds.thrasher:Clearing pg_upmap on 2.1 2024-09-16T09:42:40.731 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd rm-pg-upmap-items 2.1 2024-09-16T09:42:40.803 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:42:40.822 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:42:40.827 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:40.828 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:40.828 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:42:40.929 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:42:40.980 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:40.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:40.981 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:42:41.082 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:42:41.100 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 81 16 121097 121081 93.413 71.125 0.033128 0.010698 2024-09-16T09:42:41.113 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:41.113 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:41.114 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:42:41.215 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:42:41.256 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:41.257 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:41.257 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:42:41.358 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:42:41.395 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:41.396 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:41.396 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:42:41.497 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:42:41.498 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:42:41.498 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:42:41.524 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:41.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:41.525 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:42:41.537 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:42:41.538 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:42:41.572 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:41 smithi070 ceph-mon[30388]: pgmap v932: 168 pgs: 168 active+clean; 7.2 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 852 B/s rd, 67 MiB/s wr, 1.08k op/s 2024-09-16T09:42:41.572 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:41 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/827566098' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:42:41.572 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:41 smithi070 ceph-mon[40691]: pgmap v932: 168 pgs: 168 active+clean; 7.2 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 852 B/s rd, 67 MiB/s wr, 1.08k op/s 2024-09-16T09:42:41.573 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:41 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/827566098' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:42:41.574 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:42:41.575 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:42:41.575 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:42:41.612 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:42:41.612 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:42:41.626 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:42:41.648 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:42:41.648 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:42:41.655 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:41.656 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:41.657 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:42:41.681 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:42:41.681 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:42:41.718 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:42:41.718 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:42:41.733 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd unset noscrub 2024-09-16T09:42:41.751 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:42:41.751 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:42:41.757 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:42:41.785 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:41.785 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:41.785 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:42:41.886 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:42:41.906 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:41.907 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:41.907 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:42:41.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:41 smithi084 ceph-mon[32629]: pgmap v932: 168 pgs: 168 active+clean; 7.2 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 852 B/s rd, 67 MiB/s wr, 1.08k op/s 2024-09-16T09:42:41.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:41 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/827566098' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:42:42.008 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:42:42.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:42.052 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:42.053 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:42:42.101 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 82 16 122148 122132 93.0737 65.6875 0.00872503 0.0107397 2024-09-16T09:42:42.153 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:42:42.187 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:42.187 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:42.188 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:42:42.288 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:42:42.310 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:42.311 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:42.312 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:42:42.346 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:42:41 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:42:41] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:42:42.413 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:42:42.447 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:42.448 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:42.448 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:42:42.550 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:42:42.571 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:42.571 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:42.572 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:42:42.673 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:42:42.707 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:42.707 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:42.707 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:42:42.808 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:42:42.844 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:42.844 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:42.844 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:42:42.945 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:42:42.982 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:42.983 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:42.983 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:42:43.084 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:42:43.101 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 83 16 122945 122929 92.5524 49.8125 0.0117994 0.0108001 2024-09-16T09:42:43.111 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:43.111 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:43.112 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:42:43.213 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:42:43.248 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:43.249 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:43.249 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:42:43.350 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:42:43.382 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:43.382 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:43.383 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:42:43.484 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:42:43.516 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:43.517 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:43.518 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:42:43.565 INFO:teuthology.orchestra.run.smithi070.stderr:clear 2.1 pg_upmap_items mapping 2024-09-16T09:42:43.619 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:42:43.650 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:43.650 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:43.651 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:42:43.752 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:42:43.787 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:43.788 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:43.788 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:42:43.812 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:42:43.889 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:42:43.910 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:42:43 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:42:43.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:43 smithi084 ceph-mon[32629]: pgmap v933: 168 pgs: 168 active+clean; 7.2 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 688 B/s rd, 54 MiB/s wr, 869 op/s 2024-09-16T09:42:43.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:43 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1898929407' entity='client.admin' cmd={"prefix": "osd rm-pg-upmap-items", "pgid": "2.1"} : dispatch 2024-09-16T09:42:43.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:43 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd={"prefix": "osd rm-pg-upmap-items", "pgid": "2.1"} : dispatch 2024-09-16T09:42:43.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:43 smithi070 ceph-mon[30388]: pgmap v933: 168 pgs: 168 active+clean; 7.2 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 688 B/s rd, 54 MiB/s wr, 869 op/s 2024-09-16T09:42:43.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:43 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1898929407' entity='client.admin' cmd={"prefix": "osd rm-pg-upmap-items", "pgid": "2.1"} : dispatch 2024-09-16T09:42:43.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:43 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd={"prefix": "osd rm-pg-upmap-items", "pgid": "2.1"} : dispatch 2024-09-16T09:42:43.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:43 smithi070 ceph-mon[40691]: pgmap v933: 168 pgs: 168 active+clean; 7.2 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 688 B/s rd, 54 MiB/s wr, 869 op/s 2024-09-16T09:42:43.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:43 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1898929407' entity='client.admin' cmd={"prefix": "osd rm-pg-upmap-items", "pgid": "2.1"} : dispatch 2024-09-16T09:42:43.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:43 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd={"prefix": "osd rm-pg-upmap-items", "pgid": "2.1"} : dispatch 2024-09-16T09:42:43.945 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:43.945 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:43.946 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:42:44.046 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:42:44.082 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:44.082 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:44.083 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:42:44.103 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 84 16 124177 124161 92.3659 77 0.0101308 0.0108221 2024-09-16T09:42:44.184 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:42:44.224 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:44.225 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:44.225 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:42:44.326 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:42:44.352 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:44.352 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:44.353 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:42:44.454 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:42:44.484 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:44.484 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:44.485 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:42:44.585 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:42:44.606 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:44.607 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:44.607 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:42:44.708 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:42:44.743 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:44.744 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:44.744 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:42:44.845 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:42:44.879 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:44.879 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:44.880 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:42:44.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:44 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd rm-pg-upmap-items", "pgid": "2.1"}]': finished 2024-09-16T09:42:44.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:44 smithi084 ceph-mon[32629]: osdmap e489: 8 total, 8 up, 8 in 2024-09-16T09:42:44.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:44 smithi084 ceph-mon[32629]: osdmap e490: 8 total, 8 up, 8 in 2024-09-16T09:42:44.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:44 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd rm-pg-upmap-items", "pgid": "2.1"}]': finished 2024-09-16T09:42:44.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:44 smithi070 ceph-mon[30388]: osdmap e489: 8 total, 8 up, 8 in 2024-09-16T09:42:44.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:44 smithi070 ceph-mon[30388]: osdmap e490: 8 total, 8 up, 8 in 2024-09-16T09:42:44.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:44 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd rm-pg-upmap-items", "pgid": "2.1"}]': finished 2024-09-16T09:42:44.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:44 smithi070 ceph-mon[40691]: osdmap e489: 8 total, 8 up, 8 in 2024-09-16T09:42:44.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:44 smithi070 ceph-mon[40691]: osdmap e490: 8 total, 8 up, 8 in 2024-09-16T09:42:44.980 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:42:45.009 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:45.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:45.010 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:42:45.103 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 85 16 125327 125311 92.1248 71.875 0.00445182 0.0108503 2024-09-16T09:42:45.110 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:42:45.139 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:45.140 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:45.140 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:42:45.241 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:42:45.271 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:45.271 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:45.272 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:42:45.373 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:42:45.408 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:45.409 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:45.409 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:42:45.510 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:42:45.549 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:45.550 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:45.550 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:42:45.565 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T09:42:45.612 INFO:teuthology.orchestra.run.smithi070.stderr:noscrub is unset 2024-09-16T09:42:45.651 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:42:45.684 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:45.684 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:45.685 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:42:45.785 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:42:45.816 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:45.817 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:45.817 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:42:45.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:45 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:42:45.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:45 smithi084 ceph-mon[32629]: pgmap v935: 168 pgs: 168 active+clean; 7.5 GiB data, 28 GiB used, 687 GiB / 715 GiB avail; 853 B/s rd, 101 MiB/s wr, 1.62k op/s 2024-09-16T09:42:45.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:45 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/53995779' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:42:45.918 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:42:45.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:45 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:42:45.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:45 smithi070 ceph-mon[30388]: pgmap v935: 168 pgs: 168 active+clean; 7.5 GiB data, 28 GiB used, 687 GiB / 715 GiB avail; 853 B/s rd, 101 MiB/s wr, 1.62k op/s 2024-09-16T09:42:45.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:45 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/53995779' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:42:45.923 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:45 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:42:45.923 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:45 smithi070 ceph-mon[40691]: pgmap v935: 168 pgs: 168 active+clean; 7.5 GiB data, 28 GiB used, 687 GiB / 715 GiB avail; 853 B/s rd, 101 MiB/s wr, 1.62k op/s 2024-09-16T09:42:45.923 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:45 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/53995779' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:42:45.959 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:45.959 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:45.960 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:42:46.060 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:42:46.092 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:46.093 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:46.093 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:42:46.103 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 86 16 126406 126390 91.8376 67.4375 0.00597024 0.0108846 2024-09-16T09:42:46.194 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:42:46.234 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:46.234 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:46.235 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:42:46.335 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:42:46.371 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:46.372 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:46.372 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:42:46.473 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:42:46.479 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.5 dump_historic_ops 2024-09-16T09:42:46.501 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:46.502 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:46.502 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:42:46.604 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:42:46.617 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:42:46.643 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:46.644 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:46.644 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:42:46.745 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:42:46.752 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:46 smithi070 ceph-mon[30388]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:42:46.752 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:46 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/53995779' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-16T09:42:46.752 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:46 smithi070 ceph-mon[30388]: osdmap e491: 8 total, 8 up, 8 in 2024-09-16T09:42:46.752 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:46 smithi070 ceph-mon[30388]: 4.8 scrub starts 2024-09-16T09:42:46.752 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:46 smithi070 ceph-mon[30388]: 4.8 scrub ok 2024-09-16T09:42:46.752 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:46 smithi070 ceph-mon[40691]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:42:46.753 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:46 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/53995779' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-16T09:42:46.753 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:46 smithi070 ceph-mon[40691]: osdmap e491: 8 total, 8 up, 8 in 2024-09-16T09:42:46.753 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:46 smithi070 ceph-mon[40691]: 4.8 scrub starts 2024-09-16T09:42:46.753 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:46 smithi070 ceph-mon[40691]: 4.8 scrub ok 2024-09-16T09:42:46.782 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:46.783 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:46.783 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:42:46.884 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:42:46.910 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:42:46.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:46 smithi084 ceph-mon[32629]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:42:46.912 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:46 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/53995779' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-16T09:42:46.912 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:46 smithi084 ceph-mon[32629]: osdmap e491: 8 total, 8 up, 8 in 2024-09-16T09:42:46.912 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:46 smithi084 ceph-mon[32629]: 4.8 scrub starts 2024-09-16T09:42:46.912 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:46 smithi084 ceph-mon[32629]: 4.8 scrub ok 2024-09-16T09:42:46.925 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:46.925 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:46.926 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:42:47.026 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:42:47.069 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:47.069 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:47.070 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:42:47.103 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 87 16 127683 127667 91.6994 79.8125 0.00683203 0.0109006 2024-09-16T09:42:47.170 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:42:47.213 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:47.214 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:47.214 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:42:47.289 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:42:47.289 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:42:47.301 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:42:47.302 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:42:47.315 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:42:47.337 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:42:47.337 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:42:47.349 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:47.349 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:47.349 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:42:47.378 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:42:47.378 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:42:47.412 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:42:47.413 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:42:47.447 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:42:47.447 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:42:47.450 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:42:47.482 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:42:47.482 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:42:47.486 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:47.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:47.487 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:42:47.585 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:42:47.585 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:42:47.588 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:42:47.623 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:47.624 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:47.624 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:42:47.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:47 smithi070 ceph-mon[30388]: pgmap v938: 168 pgs: 168 active+clean; 7.5 GiB data, 28 GiB used, 687 GiB / 715 GiB avail; 853 B/s rd, 57 MiB/s wr, 915 op/s; 0 B/s, 0 objects/s recovering 2024-09-16T09:42:47.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:47 smithi070 ceph-mon[30388]: 11.8 scrub starts 2024-09-16T09:42:47.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:47 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/4148370006' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:42:47.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:47 smithi070 ceph-mon[40691]: pgmap v938: 168 pgs: 168 active+clean; 7.5 GiB data, 28 GiB used, 687 GiB / 715 GiB avail; 853 B/s rd, 57 MiB/s wr, 915 op/s; 0 B/s, 0 objects/s recovering 2024-09-16T09:42:47.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:47 smithi070 ceph-mon[40691]: 11.8 scrub starts 2024-09-16T09:42:47.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:47 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/4148370006' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:42:47.726 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:42:47.764 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:47.764 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:47.765 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:42:47.865 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:42:47.899 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:47.900 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:47.900 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:42:47.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:47 smithi084 ceph-mon[32629]: pgmap v938: 168 pgs: 168 active+clean; 7.5 GiB data, 28 GiB used, 687 GiB / 715 GiB avail; 853 B/s rd, 57 MiB/s wr, 915 op/s; 0 B/s, 0 objects/s recovering 2024-09-16T09:42:47.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:47 smithi084 ceph-mon[32629]: 11.8 scrub starts 2024-09-16T09:42:47.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:47 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/4148370006' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:42:48.002 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:42:48.030 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:48.031 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:48.031 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:42:48.103 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 88 16 128720 128704 91.3938 64.8125 0.00849324 0.0109371 2024-09-16T09:42:48.132 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:42:48.168 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:48.169 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:48.169 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:42:48.270 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:42:48.296 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:48.297 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:48.297 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:42:48.398 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:42:48.433 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:48.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:48.434 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:42:48.507 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-16T09:42:48.535 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:42:48.560 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:48.560 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:48.561 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:42:48.662 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:42:48.700 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:48.701 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:48.701 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:42:48.780 INFO:teuthology.orchestra.run.smithi084.stdout:ERROR: (22) Invalid argument 2024-09-16T09:42:48.781 INFO:teuthology.orchestra.run.smithi084.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:42:48.802 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:42:48.841 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:48.841 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:48.842 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:42:48.943 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:42:48.963 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:48.964 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:48.964 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:42:49.065 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:42:49.099 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:49.099 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:49.100 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:42:49.103 INFO:tasks.radosbench.radosbench.0.smithi070.stdout: 89 16 129736 129720 91.0803 63.5 0.0153581 0.010975 2024-09-16T09:42:49.201 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:42:49.235 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:49.236 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:49.236 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:42:49.337 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:42:49.357 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:49.358 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:49.358 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:42:49.459 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:42:49.494 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:49.495 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:49.495 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:42:49.596 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:42:49.616 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:49.617 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:49.617 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:42:49.718 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:42:49.740 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:49.741 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:49.741 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:42:49.842 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:42:49.878 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:49.878 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:49.878 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:42:49.900 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:49 smithi070 ceph-mon[30388]: pgmap v939: 168 pgs: 168 active+clean; 7.6 GiB data, 29 GiB used, 687 GiB / 715 GiB avail; 1.7 KiB/s rd, 72 MiB/s wr, 1.15k op/s; 0 B/s, 0 objects/s recovering 2024-09-16T09:42:49.900 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:49 smithi070 ceph-mon[30388]: 1.7 scrub starts 2024-09-16T09:42:49.900 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:49 smithi070 ceph-mon[30388]: 1.7 scrub ok 2024-09-16T09:42:49.900 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:49 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:42:49.901 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:49 smithi070 ceph-mon[30388]: osdmap e492: 8 total, 8 up, 8 in 2024-09-16T09:42:49.901 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:49 smithi070 ceph-mon[40691]: pgmap v939: 168 pgs: 168 active+clean; 7.6 GiB data, 29 GiB used, 687 GiB / 715 GiB avail; 1.7 KiB/s rd, 72 MiB/s wr, 1.15k op/s; 0 B/s, 0 objects/s recovering 2024-09-16T09:42:49.901 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:49 smithi070 ceph-mon[40691]: 1.7 scrub starts 2024-09-16T09:42:49.901 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:49 smithi070 ceph-mon[40691]: 1.7 scrub ok 2024-09-16T09:42:49.901 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:49 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:42:49.901 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:49 smithi070 ceph-mon[40691]: osdmap e492: 8 total, 8 up, 8 in 2024-09-16T09:42:49.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:49 smithi084 ceph-mon[32629]: pgmap v939: 168 pgs: 168 active+clean; 7.6 GiB data, 29 GiB used, 687 GiB / 715 GiB avail; 1.7 KiB/s rd, 72 MiB/s wr, 1.15k op/s; 0 B/s, 0 objects/s recovering 2024-09-16T09:42:49.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:49 smithi084 ceph-mon[32629]: 1.7 scrub starts 2024-09-16T09:42:49.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:49 smithi084 ceph-mon[32629]: 1.7 scrub ok 2024-09-16T09:42:49.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:49 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:42:49.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:49 smithi084 ceph-mon[32629]: osdmap e492: 8 total, 8 up, 8 in 2024-09-16T09:42:49.979 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:42:49.998 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T09:42:49.998 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T09:42:49.998 INFO:tasks.thrashosds.thrasher:Shrinking pool unique_pool_4 2024-09-16T09:42:49.998 INFO:tasks.cephadm.ceph_manager.ceph:contract_pool unique_pool_4 by 10 min 8 2024-09-16T09:42:49.998 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-16T09:42:50.007 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:50.007 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:50.008 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:42:50.103 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Total time run: 90.0122 2024-09-16T09:42:50.104 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Total writes made: 130467 2024-09-16T09:42:50.104 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Write size: 65536 2024-09-16T09:42:50.104 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Object size: 65536 2024-09-16T09:42:50.104 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Bandwidth (MB/sec): 90.5898 2024-09-16T09:42:50.104 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Stddev Bandwidth: 33.4278 2024-09-16T09:42:50.104 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Max bandwidth (MB/sec): 230.125 2024-09-16T09:42:50.104 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Min bandwidth (MB/sec): 33.5 2024-09-16T09:42:50.104 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Average IOPS: 1449 2024-09-16T09:42:50.104 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Stddev IOPS: 534.845 2024-09-16T09:42:50.104 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Max IOPS: 3682 2024-09-16T09:42:50.104 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Min IOPS: 536 2024-09-16T09:42:50.104 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Average Latency(s): 0.0110345 2024-09-16T09:42:50.104 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Stddev Latency(s): 0.0126346 2024-09-16T09:42:50.104 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Max latency(s): 0.55794 2024-09-16T09:42:50.104 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Min latency(s): 0.00125032 2024-09-16T09:42:50.108 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:42:50.110 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Cleaning up (deleting benchmark objects) 2024-09-16T09:42:50.137 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:50.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:50.138 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:42:50.168 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.6 dump_ops_in_flight 2024-09-16T09:42:50.239 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:42:50.284 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:50.284 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:50.285 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:42:50.385 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:42:50.424 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:50.424 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:50.425 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:42:50.526 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:42:50.555 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:50.555 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:50.556 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:42:50.607 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:42:50.656 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:42:50.690 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:50.690 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:50.691 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:42:50.791 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:42:50.832 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:50.833 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:50.833 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:42:50.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:50 smithi084 ceph-mon[32629]: 11.8 scrub ok 2024-09-16T09:42:50.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:50 smithi070 ceph-mon[30388]: 11.8 scrub ok 2024-09-16T09:42:50.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:50 smithi070 ceph-mon[40691]: 11.8 scrub ok 2024-09-16T09:42:50.934 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:42:50.955 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:50.955 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:50.956 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:42:51.056 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:42:51.080 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:51.081 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:51.081 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:42:51.182 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:42:51.211 INFO:teuthology.orchestra.run.smithi084.stdout:ERROR: (22) Invalid argument 2024-09-16T09:42:51.211 INFO:teuthology.orchestra.run.smithi084.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:42:51.217 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:51.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:51.218 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:42:51.319 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:42:51.353 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:51.354 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:51.354 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:42:51.455 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:42:51.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:51.487 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:51.487 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:42:51.588 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:42:51.628 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:51.629 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:51.629 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:42:51.730 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:42:51.759 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:51.759 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:51.760 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:42:51.861 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:42:51.893 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:51.893 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:51.894 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:42:51.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:51 smithi084 ceph-mon[32629]: pgmap v941: 168 pgs: 1 active+clean+scrubbing, 167 active+clean; 7.9 GiB data, 29 GiB used, 686 GiB / 715 GiB avail; 853 B/s rd, 63 MiB/s wr, 1k op/s; 0 B/s, 0 objects/s recovering 2024-09-16T09:42:51.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:51 smithi084 ceph-mon[32629]: 11.a scrub starts 2024-09-16T09:42:51.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:51 smithi070 ceph-mon[30388]: pgmap v941: 168 pgs: 1 active+clean+scrubbing, 167 active+clean; 7.9 GiB data, 29 GiB used, 686 GiB / 715 GiB avail; 853 B/s rd, 63 MiB/s wr, 1k op/s; 0 B/s, 0 objects/s recovering 2024-09-16T09:42:51.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:51 smithi070 ceph-mon[30388]: 11.a scrub starts 2024-09-16T09:42:51.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:51 smithi070 ceph-mon[40691]: pgmap v941: 168 pgs: 1 active+clean+scrubbing, 167 active+clean; 7.9 GiB data, 29 GiB used, 686 GiB / 715 GiB avail; 853 B/s rd, 63 MiB/s wr, 1k op/s; 0 B/s, 0 objects/s recovering 2024-09-16T09:42:51.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:51 smithi070 ceph-mon[40691]: 11.a scrub starts 2024-09-16T09:42:51.961 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.6 dump_blocked_ops 2024-09-16T09:42:51.995 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:42:52.045 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:52.045 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:52.045 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:42:52.146 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:42:52.182 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:52.182 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:52.183 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:42:52.284 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:42:52.341 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:52.341 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:52.341 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:42:52.376 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:42:51 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:42:51] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:42:52.442 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:42:52.447 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:42:52.475 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:52.476 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:52.476 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:42:52.577 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:42:52.598 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:52.598 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:52.599 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:42:52.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:52 smithi070 ceph-mon[30388]: 5.9 scrub starts 2024-09-16T09:42:52.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:52 smithi070 ceph-mon[30388]: 5.9 scrub ok 2024-09-16T09:42:52.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:52 smithi070 ceph-mon[30388]: 4.f scrub starts 2024-09-16T09:42:52.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:52 smithi070 ceph-mon[30388]: 4.f scrub ok 2024-09-16T09:42:52.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:52 smithi070 ceph-mon[40691]: 5.9 scrub starts 2024-09-16T09:42:52.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:52 smithi070 ceph-mon[40691]: 5.9 scrub ok 2024-09-16T09:42:52.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:52 smithi070 ceph-mon[40691]: 4.f scrub starts 2024-09-16T09:42:52.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:52 smithi070 ceph-mon[40691]: 4.f scrub ok 2024-09-16T09:42:52.700 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:42:52.721 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:52.722 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:52.722 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:42:52.823 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:42:52.843 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:52.844 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:52.844 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:42:52.861 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:42:52.904 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:42:52.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:52 smithi084 ceph-mon[32629]: 5.9 scrub starts 2024-09-16T09:42:52.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:52 smithi084 ceph-mon[32629]: 5.9 scrub ok 2024-09-16T09:42:52.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:52 smithi084 ceph-mon[32629]: 4.f scrub starts 2024-09-16T09:42:52.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:52 smithi084 ceph-mon[32629]: 4.f scrub ok 2024-09-16T09:42:52.945 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:42:52.986 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:52.986 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:52.986 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:42:53.087 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:42:53.125 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:53.126 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:53.126 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:42:53.227 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:42:53.256 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:53.257 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:53.257 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:42:53.358 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:42:53.385 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:53.386 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:53.386 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:42:53.487 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:42:53.523 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:53.524 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:53.524 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:42:53.625 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:42:53.669 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:53.669 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:53.669 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:42:53.678 INFO:teuthology.orchestra.run.smithi084.stdout:ERROR: (22) Invalid argument 2024-09-16T09:42:53.678 INFO:teuthology.orchestra.run.smithi084.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:42:53.770 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:42:53.809 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:53.809 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:53.810 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:42:53.856 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:53 smithi070 ceph-mon[30388]: pgmap v942: 168 pgs: 1 active+clean+scrubbing, 167 active+clean; 7.9 GiB data, 29 GiB used, 686 GiB / 715 GiB avail; 689 B/s rd, 50 MiB/s wr, 808 op/s; 0 B/s, 0 objects/s recovering 2024-09-16T09:42:53.856 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:53 smithi070 ceph-mon[30388]: 11.a scrub ok 2024-09-16T09:42:53.856 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:53 smithi070 ceph-mon[30388]: 4.1c scrub starts 2024-09-16T09:42:53.856 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:53 smithi070 ceph-mon[30388]: 4.1c scrub ok 2024-09-16T09:42:53.856 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:53 smithi070 ceph-mon[40691]: pgmap v942: 168 pgs: 1 active+clean+scrubbing, 167 active+clean; 7.9 GiB data, 29 GiB used, 686 GiB / 715 GiB avail; 689 B/s rd, 50 MiB/s wr, 808 op/s; 0 B/s, 0 objects/s recovering 2024-09-16T09:42:53.856 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:53 smithi070 ceph-mon[40691]: 11.a scrub ok 2024-09-16T09:42:53.856 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:53 smithi070 ceph-mon[40691]: 4.1c scrub starts 2024-09-16T09:42:53.856 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:53 smithi070 ceph-mon[40691]: 4.1c scrub ok 2024-09-16T09:42:53.910 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:42:53 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:42:53.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:53 smithi084 ceph-mon[32629]: pgmap v942: 168 pgs: 1 active+clean+scrubbing, 167 active+clean; 7.9 GiB data, 29 GiB used, 686 GiB / 715 GiB avail; 689 B/s rd, 50 MiB/s wr, 808 op/s; 0 B/s, 0 objects/s recovering 2024-09-16T09:42:53.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:53 smithi084 ceph-mon[32629]: 11.a scrub ok 2024-09-16T09:42:53.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:53 smithi084 ceph-mon[32629]: 4.1c scrub starts 2024-09-16T09:42:53.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:53 smithi084 ceph-mon[32629]: 4.1c scrub ok 2024-09-16T09:42:53.911 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:42:53.945 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:53.945 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:53.946 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:42:54.046 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:42:54.082 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:42:54.085 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2024-09-16T09:42:54.087 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:54.087 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:54.087 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:42:54.188 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:42:54.227 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:54.227 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:54.228 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:42:54.328 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:42:54.365 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:54.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:54.366 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:42:54.467 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:42:54.524 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.6 dump_historic_ops 2024-09-16T09:42:54.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:54.526 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:54.526 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:42:54.627 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:42:54.675 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:54.676 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:54.676 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:42:54.749 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:54 smithi070 ceph-mon[30388]: 3.b scrub starts 2024-09-16T09:42:54.749 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:54 smithi070 ceph-mon[30388]: 3.b scrub ok 2024-09-16T09:42:54.749 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:54 smithi070 ceph-mon[30388]: 4.17 scrub starts 2024-09-16T09:42:54.749 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:54 smithi070 ceph-mon[30388]: 4.17 scrub ok 2024-09-16T09:42:54.749 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:54 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:42:54.749 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:54 smithi070 ceph-mon[30388]: 3.a scrub starts 2024-09-16T09:42:54.749 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:54 smithi070 ceph-mon[30388]: 3.a scrub ok 2024-09-16T09:42:54.749 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:54 smithi070 ceph-mon[30388]: pgmap v943: 168 pgs: 168 active+clean; 7.8 GiB data, 29 GiB used, 686 GiB / 715 GiB avail; 1.3 KiB/s rd, 55 MiB/s wr, 1.24k op/s 2024-09-16T09:42:54.749 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:54 smithi070 ceph-mon[30388]: from='client.26357 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:42:54.750 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:54 smithi070 ceph-mon[40691]: 3.b scrub starts 2024-09-16T09:42:54.750 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:54 smithi070 ceph-mon[40691]: 3.b scrub ok 2024-09-16T09:42:54.750 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:54 smithi070 ceph-mon[40691]: 4.17 scrub starts 2024-09-16T09:42:54.750 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:54 smithi070 ceph-mon[40691]: 4.17 scrub ok 2024-09-16T09:42:54.750 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:54 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:42:54.750 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:54 smithi070 ceph-mon[40691]: 3.a scrub starts 2024-09-16T09:42:54.750 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:54 smithi070 ceph-mon[40691]: 3.a scrub ok 2024-09-16T09:42:54.750 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:54 smithi070 ceph-mon[40691]: pgmap v943: 168 pgs: 168 active+clean; 7.8 GiB data, 29 GiB used, 686 GiB / 715 GiB avail; 1.3 KiB/s rd, 55 MiB/s wr, 1.24k op/s 2024-09-16T09:42:54.750 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:54 smithi070 ceph-mon[40691]: from='client.26357 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:42:54.777 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:42:54.814 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:54.815 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:54.815 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:42:54.877 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T09:42:54.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:54 smithi084 ceph-mon[32629]: 3.b scrub starts 2024-09-16T09:42:54.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:54 smithi084 ceph-mon[32629]: 3.b scrub ok 2024-09-16T09:42:54.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:54 smithi084 ceph-mon[32629]: 4.17 scrub starts 2024-09-16T09:42:54.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:54 smithi084 ceph-mon[32629]: 4.17 scrub ok 2024-09-16T09:42:54.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:54 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:42:54.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:54 smithi084 ceph-mon[32629]: 3.a scrub starts 2024-09-16T09:42:54.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:54 smithi084 ceph-mon[32629]: 3.a scrub ok 2024-09-16T09:42:54.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:54 smithi084 ceph-mon[32629]: pgmap v943: 168 pgs: 168 active+clean; 7.8 GiB data, 29 GiB used, 686 GiB / 715 GiB avail; 1.3 KiB/s rd, 55 MiB/s wr, 1.24k op/s 2024-09-16T09:42:54.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:54 smithi084 ceph-mon[32629]: from='client.26357 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:42:54.916 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:42:54.955 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:54.955 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:54.956 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:42:54.965 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:42:55.057 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:42:55.082 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:55.083 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:55.083 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:42:55.184 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:42:55.205 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:55.205 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:55.206 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:42:55.230 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":943,"stamp":"2024-09-16T09:42:53.877027+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":8357550643,"num_objects":127717,"num_object_clones":0,"num_object_copies":383151,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":127717,"num_whiteouts":0,"num_read":2206,"num_read_kb":1958,"num_write":134096,"num_write_kb":8348526,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":151,"num_bytes_recovered":333352,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":77888,"ondisk_log_size":77888,"up":504,"acting":504,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":504,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":30808428,"kb_used_data":24583528,"kb_used_omap":12,"kb_used_meta":6224563,"kb_avail":719152788,"statfs":{"total":767960285184,"available":736412454912,"internally_reserved":0,"allocated":25173532672,"data_stored":25168857593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":6373953074},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":288227352,"num_objects":4399,"num_object_clones":0,"num_object_copies":13197,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4399,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":10255,"num_write_kb":468865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.262927"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":933,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.468894+0000","last_change":"2024-09-16T09:41:40.073159+0000","last_active":"2024-09-16T09:42:49.468894+0000","last_peered":"2024-09-16T09:42:49.468894+0000","last_clean":"2024-09-16T09:42:49.468894+0000","last_became_active":"2024-09-16T09:25:01.709108+0000","last_became_peered":"2024-09-16T09:25:01.709108+0000","last_unstale":"2024-09-16T09:42:49.468894+0000","last_undegraded":"2024-09-16T09:42:49.468894+0000","last_fullsized":"2024-09-16T09:42:49.468894+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:40.073089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:41:40.073089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:58.125354+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"285'15","reported_seq":945,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.449860+0000","last_change":"2024-09-16T09:41:43.064790+0000","last_active":"2024-09-16T09:42:49.449860+0000","last_peered":"2024-09-16T09:42:49.449860+0000","last_clean":"2024-09-16T09:42:49.449860+0000","last_became_active":"2024-09-16T09:25:03.714635+0000","last_became_peered":"2024-09-16T09:25:03.714635+0000","last_unstale":"2024-09-16T09:42:49.449860+0000","last_undegraded":"2024-09-16T09:42:49.449860+0000","last_fullsized":"2024-09-16T09:42:49.449860+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'15","last_scrub_stamp":"2024-09-16T09:41:43.064729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:41:43.064729+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:53.613534+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":555,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.448310+0000","last_change":"2024-09-16T09:42:31.714835+0000","last_active":"2024-09-16T09:42:49.448310+0000","last_peered":"2024-09-16T09:42:49.448310+0000","last_clean":"2024-09-16T09:42:49.448310+0000","last_became_active":"2024-09-16T09:33:30.242238+0000","last_became_peered":"2024-09-16T09:33:30.242238+0000","last_unstale":"2024-09-16T09:42:49.448310+0000","last_undegraded":"2024-09-16T09:42:49.448310+0000","last_fullsized":"2024-09-16T09:42:49.448310+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:42:31.714749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:42:31.714749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:32.597828+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":904,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.449776+0000","last_change":"2024-09-16T09:41:31.078139+0000","last_active":"2024-09-16T09:42:49.449776+0000","last_peered":"2024-09-16T09:42:49.449776+0000","last_clean":"2024-09-16T09:42:49.449776+0000","last_became_active":"2024-09-16T09:25:07.715442+0000","last_became_peered":"2024-09-16T09:25:07.715442+0000","last_unstale":"2024-09-16T09:42:49.449776+0000","last_undegraded":"2024-09-16T09:42:49.449776+0000","last_fullsized":"2024-09-16T09:42:49.449776+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:31.078058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:41:31.078058+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":649,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.456918+0000","last_change":"2024-09-16T09:42:07.714808+0000","last_active":"2024-09-16T09:42:49.456918+0000","last_peered":"2024-09-16T09:42:49.456918+0000","last_clean":"2024-09-16T09:42:49.456918+0000","last_became_active":"2024-09-16T09:31:09.269342+0000","last_became_peered":"2024-09-16T09:31:09.269342+0000","last_unstale":"2024-09-16T09:42:49.456918+0000","last_undegraded":"2024-09-16T09:42:49.456918+0000","last_fullsized":"2024-09-16T09:42:49.456918+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:42:07.714650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:42:07.714650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:27.261818+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":783,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.458206+0000","last_change":"2024-09-16T09:39:28.963009+0000","last_active":"2024-09-16T09:42:49.458206+0000","last_peered":"2024-09-16T09:42:49.458206+0000","last_clean":"2024-09-16T09:42:49.458206+0000","last_became_active":"2024-09-16T09:31:09.275136+0000","last_became_peered":"2024-09-16T09:31:09.275136+0000","last_unstale":"2024-09-16T09:42:49.458206+0000","last_undegraded":"2024-09-16T09:42:49.458206+0000","last_fullsized":"2024-09-16T09:42:49.458206+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:39:28.962926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:39:28.962926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:42.662305+0000","scrub_duration":59,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.19","version":"285'27","reported_seq":134,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.449692+0000","last_change":"2024-09-16T09:40:58.802023+0000","last_active":"2024-09-16T09:42:49.449692+0000","last_peered":"2024-09-16T09:42:49.449692+0000","last_clean":"2024-09-16T09:42:49.449692+0000","last_became_active":"2024-09-16T09:40:58.699357+0000","last_became_peered":"2024-09-16T09:40:58.699357+0000","last_unstale":"2024-09-16T09:42:49.449692+0000","last_undegraded":"2024-09-16T09:42:49.449692+0000","last_fullsized":"2024-09-16T09:42:49.449692+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'27","last_scrub_stamp":"2024-09-16T09:37:29.683568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:37:29.683568+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:37.005048+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,5],"acting":[4,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":924,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.449758+0000","last_change":"2024-09-16T09:42:10.051996+0000","last_active":"2024-09-16T09:42:49.449758+0000","last_peered":"2024-09-16T09:42:49.449758+0000","last_clean":"2024-09-16T09:42:49.449758+0000","last_became_active":"2024-09-16T09:25:05.717236+0000","last_became_peered":"2024-09-16T09:25:05.717236+0000","last_unstale":"2024-09-16T09:42:49.449758+0000","last_undegraded":"2024-09-16T09:42:49.449758+0000","last_fullsized":"2024-09-16T09:42:49.449758+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:42:10.051939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:42:10.051939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:22.638958+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":556,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.462422+0000","last_change":"2024-09-16T09:41:33.135452+0000","last_active":"2024-09-16T09:42:49.462422+0000","last_peered":"2024-09-16T09:42:49.462422+0000","last_clean":"2024-09-16T09:42:49.462422+0000","last_became_active":"2024-09-16T09:33:30.247733+0000","last_became_peered":"2024-09-16T09:33:30.247733+0000","last_unstale":"2024-09-16T09:42:49.462422+0000","last_undegraded":"2024-09-16T09:42:49.462422+0000","last_fullsized":"2024-09-16T09:42:49.462422+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:33.135342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:41:33.135342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:57.435366+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1a","version":"285'15","reported_seq":945,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.450810+0000","last_change":"2024-09-16T09:41:47.100374+0000","last_active":"2024-09-16T09:42:49.450810+0000","last_peered":"2024-09-16T09:42:49.450810+0000","last_clean":"2024-09-16T09:42:49.450810+0000","last_became_active":"2024-09-16T09:25:03.709147+0000","last_became_peered":"2024-09-16T09:25:03.709147+0000","last_unstale":"2024-09-16T09:42:49.450810+0000","last_undegraded":"2024-09-16T09:42:49.450810+0000","last_fullsized":"2024-09-16T09:42:49.450810+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'15","last_scrub_stamp":"2024-09-16T09:41:47.100292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:41:47.100292+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:52.405007+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":899,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.448747+0000","last_change":"2024-09-16T09:36:51.134364+0000","last_active":"2024-09-16T09:42:49.448747+0000","last_peered":"2024-09-16T09:42:49.448747+0000","last_clean":"2024-09-16T09:42:49.448747+0000","last_became_active":"2024-09-16T09:25:05.718134+0000","last_became_peered":"2024-09-16T09:25:05.718134+0000","last_unstale":"2024-09-16T09:42:49.448747+0000","last_undegraded":"2024-09-16T09:42:49.448747+0000","last_fullsized":"2024-09-16T09:42:49.448747+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:51.134314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:36:51.134314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:51.766506+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"66'1","reported_seq":921,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.460577+0000","last_change":"2024-09-16T09:41:48.089052+0000","last_active":"2024-09-16T09:42:49.460577+0000","last_peered":"2024-09-16T09:42:49.460577+0000","last_clean":"2024-09-16T09:42:49.460577+0000","last_became_active":"2024-09-16T09:25:07.716558+0000","last_became_peered":"2024-09-16T09:25:07.716558+0000","last_unstale":"2024-09-16T09:42:49.460577+0000","last_undegraded":"2024-09-16T09:42:49.460577+0000","last_fullsized":"2024-09-16T09:42:49.460577+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-16T09:41:48.088985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:41:48.088985+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:03.846875+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":907,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.449421+0000","last_change":"2024-09-16T09:42:22.732120+0000","last_active":"2024-09-16T09:42:49.449421+0000","last_peered":"2024-09-16T09:42:49.449421+0000","last_clean":"2024-09-16T09:42:49.449421+0000","last_became_active":"2024-09-16T09:25:01.696939+0000","last_became_peered":"2024-09-16T09:25:01.696939+0000","last_unstale":"2024-09-16T09:42:49.449421+0000","last_undegraded":"2024-09-16T09:42:49.449421+0000","last_fullsized":"2024-09-16T09:42:49.449421+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:42:22.732048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:42:22.732048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:32.729048+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"285'7","reported_seq":917,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.450182+0000","last_change":"2024-09-16T09:35:39.975915+0000","last_active":"2024-09-16T09:42:49.450182+0000","last_peered":"2024-09-16T09:42:49.450182+0000","last_clean":"2024-09-16T09:42:49.450182+0000","last_became_active":"2024-09-16T09:25:03.716136+0000","last_became_peered":"2024-09-16T09:25:03.716136+0000","last_unstale":"2024-09-16T09:42:49.450182+0000","last_undegraded":"2024-09-16T09:42:49.450182+0000","last_fullsized":"2024-09-16T09:42:49.450182+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'7","last_scrub_stamp":"2024-09-16T09:35:39.975840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:35:39.975840+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:04.157782+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":10,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":62,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.456804+0000","last_change":"2024-09-16T09:42:08.915352+0000","last_active":"2024-09-16T09:42:49.456804+0000","last_peered":"2024-09-16T09:42:49.456804+0000","last_clean":"2024-09-16T09:42:49.456804+0000","last_became_active":"2024-09-16T09:42:02.494290+0000","last_became_peered":"2024-09-16T09:42:02.494290+0000","last_unstale":"2024-09-16T09:42:49.456804+0000","last_undegraded":"2024-09-16T09:42:49.456804+0000","last_fullsized":"2024-09-16T09:42:49.456804+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:42:08.915303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:42:08.915303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:19.974744+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4],"acting":[6,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":925,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.448327+0000","last_change":"2024-09-16T09:38:34.050374+0000","last_active":"2024-09-16T09:42:49.448327+0000","last_peered":"2024-09-16T09:42:49.448327+0000","last_clean":"2024-09-16T09:42:49.448327+0000","last_became_active":"2024-09-16T09:31:09.275656+0000","last_became_peered":"2024-09-16T09:31:09.275656+0000","last_unstale":"2024-09-16T09:42:49.448327+0000","last_undegraded":"2024-09-16T09:42:49.448327+0000","last_fullsized":"2024-09-16T09:42:49.448327+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:38:34.050185+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:38:34.050185+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:52.760980+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":905,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.451449+0000","last_change":"2024-09-16T09:33:48.322010+0000","last_active":"2024-09-16T09:42:49.451449+0000","last_peered":"2024-09-16T09:42:49.451449+0000","last_clean":"2024-09-16T09:42:49.451449+0000","last_became_active":"2024-09-16T09:25:07.720793+0000","last_became_peered":"2024-09-16T09:25:07.720793+0000","last_unstale":"2024-09-16T09:42:49.451449+0000","last_undegraded":"2024-09-16T09:42:49.451449+0000","last_fullsized":"2024-09-16T09:42:49.451449+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:48.321895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:33:48.321895+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:54.184395+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":655,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.461391+0000","last_change":"2024-09-16T09:34:33.866299+0000","last_active":"2024-09-16T09:42:49.461391+0000","last_peered":"2024-09-16T09:42:49.461391+0000","last_clean":"2024-09-16T09:42:49.461391+0000","last_became_active":"2024-09-16T09:30:39.201824+0000","last_became_peered":"2024-09-16T09:30:39.201824+0000","last_unstale":"2024-09-16T09:42:49.461391+0000","last_undegraded":"2024-09-16T09:42:49.461391+0000","last_fullsized":"2024-09-16T09:42:49.461391+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:33.866142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:34:33.866142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:37.667042+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"285'27","reported_seq":1021,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:52.892017+0000","last_change":"2024-09-16T09:42:52.892017+0000","last_active":"2024-09-16T09:42:52.892017+0000","last_peered":"2024-09-16T09:42:52.892017+0000","last_clean":"2024-09-16T09:42:52.892017+0000","last_became_active":"2024-09-16T09:33:30.249798+0000","last_became_peered":"2024-09-16T09:33:30.249798+0000","last_unstale":"2024-09-16T09:42:52.892017+0000","last_undegraded":"2024-09-16T09:42:52.892017+0000","last_fullsized":"2024-09-16T09:42:52.892017+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'27","last_scrub_stamp":"2024-09-16T09:42:52.891931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:42:52.891931+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:44:11.404491+0000","scrub_duration":168,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":539,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.452639+0000","last_change":"2024-09-16T09:33:43.326933+0000","last_active":"2024-09-16T09:42:49.452639+0000","last_peered":"2024-09-16T09:42:49.452639+0000","last_clean":"2024-09-16T09:42:49.452639+0000","last_became_active":"2024-09-16T09:33:30.251587+0000","last_became_peered":"2024-09-16T09:33:30.251587+0000","last_unstale":"2024-09-16T09:42:49.452639+0000","last_undegraded":"2024-09-16T09:42:49.452639+0000","last_fullsized":"2024-09-16T09:42:49.452639+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:43.326852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:43.326852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:55.169781+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,0],"acting":[4,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":761,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.458033+0000","last_change":"2024-09-16T09:40:27.916994+0000","last_active":"2024-09-16T09:42:49.458033+0000","last_peered":"2024-09-16T09:42:49.458033+0000","last_clean":"2024-09-16T09:42:49.458033+0000","last_became_active":"2024-09-16T09:28:52.974255+0000","last_became_peered":"2024-09-16T09:28:52.974255+0000","last_unstale":"2024-09-16T09:42:49.458033+0000","last_undegraded":"2024-09-16T09:42:49.458033+0000","last_fullsized":"2024-09-16T09:42:49.458033+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:40:27.916860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:40:27.916860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:00.899019+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":924,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.450997+0000","last_change":"2024-09-16T09:41:08.223235+0000","last_active":"2024-09-16T09:42:49.450997+0000","last_peered":"2024-09-16T09:42:49.450997+0000","last_clean":"2024-09-16T09:42:49.450997+0000","last_became_active":"2024-09-16T09:25:01.706491+0000","last_became_peered":"2024-09-16T09:25:01.706491+0000","last_unstale":"2024-09-16T09:42:49.450997+0000","last_undegraded":"2024-09-16T09:42:49.450997+0000","last_fullsized":"2024-09-16T09:42:49.450997+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:08.223190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:41:08.223190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"285'18","reported_seq":599,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.452807+0000","last_change":"2024-09-16T09:40:37.206512+0000","last_active":"2024-09-16T09:42:49.452807+0000","last_peered":"2024-09-16T09:42:49.452807+0000","last_clean":"2024-09-16T09:42:49.452807+0000","last_became_active":"2024-09-16T09:33:30.251626+0000","last_became_peered":"2024-09-16T09:33:30.251626+0000","last_unstale":"2024-09-16T09:42:49.452807+0000","last_undegraded":"2024-09-16T09:42:49.452807+0000","last_fullsized":"2024-09-16T09:42:49.452807+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'18","last_scrub_stamp":"2024-09-16T09:40:37.206432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:40:37.206432+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":33,"num_read_kb":24,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":900,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.450812+0000","last_change":"2024-09-16T09:41:45.118370+0000","last_active":"2024-09-16T09:42:49.450812+0000","last_peered":"2024-09-16T09:42:49.450812+0000","last_clean":"2024-09-16T09:42:49.450812+0000","last_became_active":"2024-09-16T09:25:05.710093+0000","last_became_peered":"2024-09-16T09:25:05.710093+0000","last_unstale":"2024-09-16T09:42:49.450812+0000","last_undegraded":"2024-09-16T09:42:49.450812+0000","last_fullsized":"2024-09-16T09:42:49.450812+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:45.118305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:41:45.118305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:14.199468+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":910,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.459660+0000","last_change":"2024-09-16T09:42:25.250180+0000","last_active":"2024-09-16T09:42:49.459660+0000","last_peered":"2024-09-16T09:42:49.459660+0000","last_clean":"2024-09-16T09:42:49.459660+0000","last_became_active":"2024-09-16T09:25:07.719380+0000","last_became_peered":"2024-09-16T09:25:07.719380+0000","last_unstale":"2024-09-16T09:42:49.459660+0000","last_undegraded":"2024-09-16T09:42:49.459660+0000","last_fullsized":"2024-09-16T09:42:49.459660+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-16T09:42:25.250120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:42:25.250120+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:38.809452+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":564,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.463287+0000","last_change":"2024-09-16T09:41:36.612463+0000","last_active":"2024-09-16T09:42:49.463287+0000","last_peered":"2024-09-16T09:42:49.463287+0000","last_clean":"2024-09-16T09:42:49.463287+0000","last_became_active":"2024-09-16T09:33:30.245048+0000","last_became_peered":"2024-09-16T09:33:30.245048+0000","last_unstale":"2024-09-16T09:42:49.463287+0000","last_undegraded":"2024-09-16T09:42:49.463287+0000","last_fullsized":"2024-09-16T09:42:49.463287+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:41:36.612280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:41:36.612280+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:02.315301+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"285'20","reported_seq":981,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.463454+0000","last_change":"2024-09-16T09:39:35.550760+0000","last_active":"2024-09-16T09:42:49.463454+0000","last_peered":"2024-09-16T09:42:49.463454+0000","last_clean":"2024-09-16T09:42:49.463454+0000","last_became_active":"2024-09-16T09:31:09.275050+0000","last_became_peered":"2024-09-16T09:31:09.275050+0000","last_unstale":"2024-09-16T09:42:49.463454+0000","last_undegraded":"2024-09-16T09:42:49.463454+0000","last_fullsized":"2024-09-16T09:42:49.463454+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'20","last_scrub_stamp":"2024-09-16T09:39:35.548333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:39:35.548333+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:05.787546+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"66'8","reported_seq":1167,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.458546+0000","last_change":"2024-09-16T09:42:10.904990+0000","last_active":"2024-09-16T09:42:49.458546+0000","last_peered":"2024-09-16T09:42:49.458546+0000","last_clean":"2024-09-16T09:42:49.458546+0000","last_became_active":"2024-09-16T09:25:05.712516+0000","last_became_peered":"2024-09-16T09:25:05.712516+0000","last_unstale":"2024-09-16T09:42:49.458546+0000","last_undegraded":"2024-09-16T09:42:49.458546+0000","last_fullsized":"2024-09-16T09:42:49.458546+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:42:10.904920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:42:10.904920+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:19.925242+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.0","version":"492'8361","reported_seq":8448,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:52.479392+0000","last_change":"2024-09-16T09:41:13.139275+0000","last_active":"2024-09-16T09:42:52.479392+0000","last_peered":"2024-09-16T09:42:52.479392+0000","last_clean":"2024-09-16T09:42:52.479392+0000","last_became_active":"2024-09-16T09:41:13.138983+0000","last_became_peered":"2024-09-16T09:41:13.138983+0000","last_unstale":"2024-09-16T09:42:52.479392+0000","last_undegraded":"2024-09-16T09:42:52.479392+0000","last_fullsized":"2024-09-16T09:42:52.479392+0000","mapping_epoch":454,"log_start":"467'4000","ondisk_log_start":"467'4000","created":454,"last_epoch_clean":455,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:11.976785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:41:11.976785+0000","last_clean_scrub_stamp":"2024-09-16T09:41:11.976785+0000","objects_scrubbed":0,"log_size":4361,"log_dups_size":0,"ondisk_log_size":4361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":525008896,"num_objects":8011,"num_object_clones":0,"num_object_copies":24033,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8011,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8361,"num_write_kb":523904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3],"acting":[7,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.a","version":"73'79","reported_seq":847,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.448191+0000","last_change":"2024-09-16T09:41:28.733947+0000","last_active":"2024-09-16T09:42:49.448191+0000","last_peered":"2024-09-16T09:42:49.448191+0000","last_clean":"2024-09-16T09:42:49.448191+0000","last_became_active":"2024-09-16T09:26:35.127352+0000","last_became_peered":"2024-09-16T09:26:35.127352+0000","last_unstale":"2024-09-16T09:42:49.448191+0000","last_undegraded":"2024-09-16T09:42:49.448191+0000","last_fullsized":"2024-09-16T09:42:49.448191+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:41:28.733854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:41:28.733854+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"285'27","reported_seq":891,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:51.725308+0000","last_change":"2024-09-16T09:42:51.725308+0000","last_active":"2024-09-16T09:42:51.725308+0000","last_peered":"2024-09-16T09:42:51.725308+0000","last_clean":"2024-09-16T09:42:51.725308+0000","last_became_active":"2024-09-16T09:26:33.295672+0000","last_became_peered":"2024-09-16T09:26:33.295672+0000","last_unstale":"2024-09-16T09:42:51.725308+0000","last_undegraded":"2024-09-16T09:42:51.725308+0000","last_fullsized":"2024-09-16T09:42:51.725308+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'27","last_scrub_stamp":"2024-09-16T09:42:51.725226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:42:51.725226+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:44:05.407088+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":650,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.462466+0000","last_change":"2024-09-16T09:38:35.023594+0000","last_active":"2024-09-16T09:42:49.462466+0000","last_peered":"2024-09-16T09:42:49.462466+0000","last_clean":"2024-09-16T09:42:49.462466+0000","last_became_active":"2024-09-16T09:31:09.274534+0000","last_became_peered":"2024-09-16T09:31:09.274534+0000","last_unstale":"2024-09-16T09:42:49.462466+0000","last_undegraded":"2024-09-16T09:42:49.462466+0000","last_fullsized":"2024-09-16T09:42:49.462466+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:38:35.023518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:38:35.023518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:00.691678+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"66'8","reported_seq":1123,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.451564+0000","last_change":"2024-09-16T09:41:32.036261+0000","last_active":"2024-09-16T09:42:49.451564+0000","last_peered":"2024-09-16T09:42:49.451564+0000","last_clean":"2024-09-16T09:42:49.451564+0000","last_became_active":"2024-09-16T09:25:05.713813+0000","last_became_peered":"2024-09-16T09:25:05.713813+0000","last_unstale":"2024-09-16T09:42:49.451564+0000","last_undegraded":"2024-09-16T09:42:49.451564+0000","last_fullsized":"2024-09-16T09:42:49.451564+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:41:32.036192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:41:32.036192+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:57.987875+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":911,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.449511+0000","last_change":"2024-09-16T09:41:48.730584+0000","last_active":"2024-09-16T09:42:49.449511+0000","last_peered":"2024-09-16T09:42:49.449511+0000","last_clean":"2024-09-16T09:42:49.449511+0000","last_became_active":"2024-09-16T09:25:07.717344+0000","last_became_peered":"2024-09-16T09:25:07.717344+0000","last_unstale":"2024-09-16T09:42:49.449511+0000","last_undegraded":"2024-09-16T09:42:49.449511+0000","last_fullsized":"2024-09-16T09:42:49.449511+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:48.730521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:41:48.730521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:08.160356+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.f","version":"492'8326","reported_seq":8413,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:52.470652+0000","last_change":"2024-09-16T09:41:13.139299+0000","last_active":"2024-09-16T09:42:52.470652+0000","last_peered":"2024-09-16T09:42:52.470652+0000","last_clean":"2024-09-16T09:42:52.470652+0000","last_became_active":"2024-09-16T09:41:13.138830+0000","last_became_peered":"2024-09-16T09:41:13.138830+0000","last_unstale":"2024-09-16T09:42:52.470652+0000","last_undegraded":"2024-09-16T09:42:52.470652+0000","last_fullsized":"2024-09-16T09:42:52.470652+0000","mapping_epoch":454,"log_start":"467'3900","ondisk_log_start":"467'3900","created":454,"last_epoch_clean":455,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:11.976785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:41:11.976785+0000","last_clean_scrub_stamp":"2024-09-16T09:41:11.976785+0000","objects_scrubbed":0,"log_size":4426,"log_dups_size":0,"ondisk_log_size":4426,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":521404416,"num_objects":7956,"num_object_clones":0,"num_object_copies":23868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7956,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8326,"num_write_kb":521024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.5","version":"73'79","reported_seq":879,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.453121+0000","last_change":"2024-09-16T09:40:29.124394+0000","last_active":"2024-09-16T09:42:49.453121+0000","last_peered":"2024-09-16T09:42:49.453121+0000","last_clean":"2024-09-16T09:42:49.453121+0000","last_became_active":"2024-09-16T09:30:08.763683+0000","last_became_peered":"2024-09-16T09:30:08.763683+0000","last_unstale":"2024-09-16T09:42:49.453121+0000","last_undegraded":"2024-09-16T09:42:49.453121+0000","last_fullsized":"2024-09-16T09:42:49.453121+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:40:29.124314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:40:29.124314+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:41.966021+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"285'30","reported_seq":878,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.458657+0000","last_change":"2024-09-16T09:41:41.151427+0000","last_active":"2024-09-16T09:42:49.458657+0000","last_peered":"2024-09-16T09:42:49.458657+0000","last_clean":"2024-09-16T09:42:49.458657+0000","last_became_active":"2024-09-16T09:31:09.270434+0000","last_became_peered":"2024-09-16T09:31:09.270434+0000","last_unstale":"2024-09-16T09:42:49.458657+0000","last_undegraded":"2024-09-16T09:42:49.458657+0000","last_fullsized":"2024-09-16T09:42:49.458657+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'30","last_scrub_stamp":"2024-09-16T09:41:41.151345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:41:41.151345+0000","objects_scrubbed":8,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:55.242044+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":51,"num_read_kb":39,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":758,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.460679+0000","last_change":"2024-09-16T09:41:45.162761+0000","last_active":"2024-09-16T09:42:49.460679+0000","last_peered":"2024-09-16T09:42:49.460679+0000","last_clean":"2024-09-16T09:42:49.460679+0000","last_became_active":"2024-09-16T09:28:52.983543+0000","last_became_peered":"2024-09-16T09:28:52.983543+0000","last_unstale":"2024-09-16T09:42:49.460679+0000","last_undegraded":"2024-09-16T09:42:49.460679+0000","last_fullsized":"2024-09-16T09:42:49.460679+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:45.162688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:41:45.162688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:01.911633+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":931,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.449916+0000","last_change":"2024-09-16T09:42:09.103716+0000","last_active":"2024-09-16T09:42:49.449916+0000","last_peered":"2024-09-16T09:42:49.449916+0000","last_clean":"2024-09-16T09:42:49.449916+0000","last_became_active":"2024-09-16T09:31:09.268864+0000","last_became_peered":"2024-09-16T09:31:09.268864+0000","last_unstale":"2024-09-16T09:42:49.449916+0000","last_undegraded":"2024-09-16T09:42:49.449916+0000","last_fullsized":"2024-09-16T09:42:49.449916+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:42:09.103657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:42:09.103657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:13.695404+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":927,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.450718+0000","last_change":"2024-09-16T09:41:33.051473+0000","last_active":"2024-09-16T09:42:49.450718+0000","last_peered":"2024-09-16T09:42:49.450718+0000","last_clean":"2024-09-16T09:42:49.450718+0000","last_became_active":"2024-09-16T09:31:09.269433+0000","last_became_peered":"2024-09-16T09:31:09.269433+0000","last_unstale":"2024-09-16T09:42:49.450718+0000","last_undegraded":"2024-09-16T09:42:49.450718+0000","last_fullsized":"2024-09-16T09:42:49.450718+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:33.051405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:41:33.051405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:02.030034+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"492'8421","reported_seq":8508,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:53.164319+0000","last_change":"2024-09-16T09:41:13.140372+0000","last_active":"2024-09-16T09:42:53.164319+0000","last_peered":"2024-09-16T09:42:53.164319+0000","last_clean":"2024-09-16T09:42:53.164319+0000","last_became_active":"2024-09-16T09:41:13.140053+0000","last_became_peered":"2024-09-16T09:41:13.140053+0000","last_unstale":"2024-09-16T09:42:53.164319+0000","last_undegraded":"2024-09-16T09:42:53.164319+0000","last_fullsized":"2024-09-16T09:42:53.164319+0000","mapping_epoch":454,"log_start":"467'4000","ondisk_log_start":"467'4000","created":454,"last_epoch_clean":455,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:11.976785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:41:11.976785+0000","last_clean_scrub_stamp":"2024-09-16T09:41:11.976785+0000","objects_scrubbed":0,"log_size":4421,"log_dups_size":0,"ondisk_log_size":4421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":525664256,"num_objects":8021,"num_object_clones":0,"num_object_copies":24063,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8021,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8421,"num_write_kb":526144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,7],"acting":[4,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"73'79","reported_seq":650,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.457439+0000","last_change":"2024-09-16T09:41:44.602634+0000","last_active":"2024-09-16T09:42:49.457439+0000","last_peered":"2024-09-16T09:42:49.457439+0000","last_clean":"2024-09-16T09:42:49.457439+0000","last_became_active":"2024-09-16T09:31:09.275139+0000","last_became_peered":"2024-09-16T09:31:09.275139+0000","last_unstale":"2024-09-16T09:42:49.457439+0000","last_undegraded":"2024-09-16T09:42:49.457439+0000","last_fullsized":"2024-09-16T09:42:49.457439+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:41:44.602557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:41:44.602557+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:02.511638+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"285'28","reported_seq":963,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.449980+0000","last_change":"2024-09-16T09:41:11.201590+0000","last_active":"2024-09-16T09:42:49.449980+0000","last_peered":"2024-09-16T09:42:49.449980+0000","last_clean":"2024-09-16T09:42:49.449980+0000","last_became_active":"2024-09-16T09:25:03.708830+0000","last_became_peered":"2024-09-16T09:25:03.708830+0000","last_unstale":"2024-09-16T09:42:49.449980+0000","last_undegraded":"2024-09-16T09:42:49.449980+0000","last_fullsized":"2024-09-16T09:42:49.449980+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'28","last_scrub_stamp":"2024-09-16T09:41:11.201523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:41:11.201523+0000","objects_scrubbed":7,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":942,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.469332+0000","last_change":"2024-09-16T09:40:31.037325+0000","last_active":"2024-09-16T09:42:49.469332+0000","last_peered":"2024-09-16T09:42:49.469332+0000","last_clean":"2024-09-16T09:42:49.469332+0000","last_became_active":"2024-09-16T09:25:01.702992+0000","last_became_peered":"2024-09-16T09:25:01.702992+0000","last_unstale":"2024-09-16T09:42:49.469332+0000","last_undegraded":"2024-09-16T09:42:49.469332+0000","last_fullsized":"2024-09-16T09:42:49.469332+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:40:31.037258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:40:31.037258+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:35.887354+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"235'12","reported_seq":687,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.449685+0000","last_change":"2024-09-16T09:42:27.113239+0000","last_active":"2024-09-16T09:42:49.449685+0000","last_peered":"2024-09-16T09:42:49.449685+0000","last_clean":"2024-09-16T09:42:49.449685+0000","last_became_active":"2024-09-16T09:33:30.251047+0000","last_became_peered":"2024-09-16T09:33:30.251047+0000","last_unstale":"2024-09-16T09:42:49.449685+0000","last_undegraded":"2024-09-16T09:42:49.449685+0000","last_fullsized":"2024-09-16T09:42:49.449685+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"235'12","last_scrub_stamp":"2024-09-16T09:42:27.113174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:42:27.113174+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:34.721191+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":902,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.460261+0000","last_change":"2024-09-16T09:40:30.956991+0000","last_active":"2024-09-16T09:42:49.460261+0000","last_peered":"2024-09-16T09:42:49.460261+0000","last_clean":"2024-09-16T09:42:49.460261+0000","last_became_active":"2024-09-16T09:25:07.722017+0000","last_became_peered":"2024-09-16T09:25:07.722017+0000","last_unstale":"2024-09-16T09:42:49.460261+0000","last_undegraded":"2024-09-16T09:42:49.460261+0000","last_fullsized":"2024-09-16T09:42:49.460261+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:40:30.956926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:40:30.956926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:01.216573+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.d","version":"492'8464","reported_seq":8553,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:53.144101+0000","last_change":"2024-09-16T09:41:13.005502+0000","last_active":"2024-09-16T09:42:53.144101+0000","last_peered":"2024-09-16T09:42:53.144101+0000","last_clean":"2024-09-16T09:42:53.144101+0000","last_became_active":"2024-09-16T09:41:13.005224+0000","last_became_peered":"2024-09-16T09:41:13.005224+0000","last_unstale":"2024-09-16T09:42:53.144101+0000","last_undegraded":"2024-09-16T09:42:53.144101+0000","last_fullsized":"2024-09-16T09:42:53.144101+0000","mapping_epoch":454,"log_start":"467'4100","ondisk_log_start":"467'4100","created":454,"last_epoch_clean":455,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:11.976785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:41:11.976785+0000","last_clean_scrub_stamp":"2024-09-16T09:41:11.976785+0000","objects_scrubbed":0,"log_size":4364,"log_dups_size":0,"ondisk_log_size":4364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":528547864,"num_objects":8066,"num_object_clones":0,"num_object_copies":24198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8066,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8464,"num_write_kb":528897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"73'79","reported_seq":989,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.457095+0000","last_change":"2024-09-16T09:42:48.661290+0000","last_active":"2024-09-16T09:42:49.457095+0000","last_peered":"2024-09-16T09:42:49.457095+0000","last_clean":"2024-09-16T09:42:49.457095+0000","last_became_active":"2024-09-16T09:31:09.274707+0000","last_became_peered":"2024-09-16T09:31:09.274707+0000","last_unstale":"2024-09-16T09:42:49.457095+0000","last_undegraded":"2024-09-16T09:42:49.457095+0000","last_fullsized":"2024-09-16T09:42:49.457095+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:42:48.661215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:42:48.661215+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:55.659607+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"285'20","reported_seq":598,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.448603+0000","last_change":"2024-09-16T09:38:32.122419+0000","last_active":"2024-09-16T09:42:49.448603+0000","last_peered":"2024-09-16T09:42:49.448603+0000","last_clean":"2024-09-16T09:42:49.448603+0000","last_became_active":"2024-09-16T09:33:30.248453+0000","last_became_peered":"2024-09-16T09:33:30.248453+0000","last_unstale":"2024-09-16T09:42:49.448603+0000","last_undegraded":"2024-09-16T09:42:49.448603+0000","last_fullsized":"2024-09-16T09:42:49.448603+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'20","last_scrub_stamp":"2024-09-16T09:38:32.122303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:38:32.122303+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:49.721122+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":922,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.450561+0000","last_change":"2024-09-16T09:36:47.095703+0000","last_active":"2024-09-16T09:42:49.450561+0000","last_peered":"2024-09-16T09:42:49.450561+0000","last_clean":"2024-09-16T09:42:49.450561+0000","last_became_active":"2024-09-16T09:31:09.270619+0000","last_became_peered":"2024-09-16T09:31:09.270619+0000","last_unstale":"2024-09-16T09:42:49.450561+0000","last_undegraded":"2024-09-16T09:42:49.450561+0000","last_fullsized":"2024-09-16T09:42:49.450561+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:47.095658+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:36:47.095658+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:00.700577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"66'8","reported_seq":1131,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.461421+0000","last_change":"2024-09-16T09:42:14.883939+0000","last_active":"2024-09-16T09:42:49.461421+0000","last_peered":"2024-09-16T09:42:49.461421+0000","last_clean":"2024-09-16T09:42:49.461421+0000","last_became_active":"2024-09-16T09:25:05.714507+0000","last_became_peered":"2024-09-16T09:25:05.714507+0000","last_unstale":"2024-09-16T09:42:49.461421+0000","last_undegraded":"2024-09-16T09:42:49.461421+0000","last_fullsized":"2024-09-16T09:42:49.461421+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:42:14.883874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:42:14.883874+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:17.330904+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":927,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.460709+0000","last_change":"2024-09-16T09:41:31.179732+0000","last_active":"2024-09-16T09:42:49.460709+0000","last_peered":"2024-09-16T09:42:49.460709+0000","last_clean":"2024-09-16T09:42:49.460709+0000","last_became_active":"2024-09-16T09:31:09.283192+0000","last_became_peered":"2024-09-16T09:31:09.283192+0000","last_unstale":"2024-09-16T09:42:49.460709+0000","last_undegraded":"2024-09-16T09:42:49.460709+0000","last_fullsized":"2024-09-16T09:42:49.460709+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:31.179662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:41:31.179662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:42.797631+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.c","version":"492'8317","reported_seq":8474,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:53.376355+0000","last_change":"2024-09-16T09:42:21.965705+0000","last_active":"2024-09-16T09:42:53.376355+0000","last_peered":"2024-09-16T09:42:53.376355+0000","last_clean":"2024-09-16T09:42:53.376355+0000","last_became_active":"2024-09-16T09:41:13.138917+0000","last_became_peered":"2024-09-16T09:41:13.138917+0000","last_unstale":"2024-09-16T09:42:53.376355+0000","last_undegraded":"2024-09-16T09:42:53.376355+0000","last_fullsized":"2024-09-16T09:42:53.376355+0000","mapping_epoch":454,"log_start":"464'2114","ondisk_log_start":"464'2114","created":454,"last_epoch_clean":455,"parent":"0.0","parent_split_bits":0,"last_scrub":"479'6141","last_scrub_stamp":"2024-09-16T09:42:21.965465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:41:11.976785+0000","last_clean_scrub_stamp":"2024-09-16T09:42:21.965465+0000","objects_scrubbed":6075,"log_size":6203,"log_dups_size":0,"ondisk_log_size":6203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:48.200076+0000","scrub_duration":2192,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":519372800,"num_objects":7925,"num_object_clones":0,"num_object_copies":23775,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7925,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8317,"num_write_kb":519744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4],"acting":[2,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"73'79","reported_seq":946,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.457308+0000","last_change":"2024-09-16T09:42:24.697048+0000","last_active":"2024-09-16T09:42:49.457308+0000","last_peered":"2024-09-16T09:42:49.457308+0000","last_clean":"2024-09-16T09:42:49.457308+0000","last_became_active":"2024-09-16T09:31:09.274565+0000","last_became_peered":"2024-09-16T09:31:09.274565+0000","last_unstale":"2024-09-16T09:42:49.457308+0000","last_undegraded":"2024-09-16T09:42:49.457308+0000","last_fullsized":"2024-09-16T09:42:49.457308+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:42:24.696973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:42:24.696973+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:43.993452+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"285'29","reported_seq":964,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.461640+0000","last_change":"2024-09-16T09:42:24.900527+0000","last_active":"2024-09-16T09:42:49.461640+0000","last_peered":"2024-09-16T09:42:49.461640+0000","last_clean":"2024-09-16T09:42:49.461640+0000","last_became_active":"2024-09-16T09:25:03.714797+0000","last_became_peered":"2024-09-16T09:25:03.714797+0000","last_unstale":"2024-09-16T09:42:49.461640+0000","last_undegraded":"2024-09-16T09:42:49.461640+0000","last_fullsized":"2024-09-16T09:42:49.461640+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'29","last_scrub_stamp":"2024-09-16T09:42:24.900426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:42:24.900426+0000","objects_scrubbed":8,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:43.245759+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":54,"num_read_kb":40,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":540,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.449032+0000","last_change":"2024-09-16T09:40:26.273165+0000","last_active":"2024-09-16T09:42:49.449032+0000","last_peered":"2024-09-16T09:42:49.449032+0000","last_clean":"2024-09-16T09:42:49.449032+0000","last_became_active":"2024-09-16T09:33:30.246996+0000","last_became_peered":"2024-09-16T09:33:30.246996+0000","last_unstale":"2024-09-16T09:42:49.449032+0000","last_undegraded":"2024-09-16T09:42:49.449032+0000","last_fullsized":"2024-09-16T09:42:49.449032+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:40:26.273106+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:40:26.273106+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:02.707559+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,2],"acting":[5,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":933,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.457073+0000","last_change":"2024-09-16T09:41:46.974010+0000","last_active":"2024-09-16T09:42:49.457073+0000","last_peered":"2024-09-16T09:42:49.457073+0000","last_clean":"2024-09-16T09:42:49.457073+0000","last_became_active":"2024-09-16T09:25:05.708663+0000","last_became_peered":"2024-09-16T09:25:05.708663+0000","last_unstale":"2024-09-16T09:42:49.457073+0000","last_undegraded":"2024-09-16T09:42:49.457073+0000","last_fullsized":"2024-09-16T09:42:49.457073+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:46.973947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:41:46.973947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:14.396908+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":548,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.458360+0000","last_change":"2024-09-16T09:37:56.882475+0000","last_active":"2024-09-16T09:42:49.458360+0000","last_peered":"2024-09-16T09:42:49.458360+0000","last_clean":"2024-09-16T09:42:49.458360+0000","last_became_active":"2024-09-16T09:33:30.250178+0000","last_became_peered":"2024-09-16T09:33:30.250178+0000","last_unstale":"2024-09-16T09:42:49.458360+0000","last_undegraded":"2024-09-16T09:42:49.458360+0000","last_fullsized":"2024-09-16T09:42:49.458360+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:56.882326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:37:56.882326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:59.888050+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,2],"acting":[6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.b","version":"492'8278","reported_seq":8438,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:53.719590+0000","last_change":"2024-09-16T09:42:24.526072+0000","last_active":"2024-09-16T09:42:53.719590+0000","last_peered":"2024-09-16T09:42:53.719590+0000","last_clean":"2024-09-16T09:42:53.719590+0000","last_became_active":"2024-09-16T09:41:13.140363+0000","last_became_peered":"2024-09-16T09:41:13.140363+0000","last_unstale":"2024-09-16T09:42:53.719590+0000","last_undegraded":"2024-09-16T09:42:53.719590+0000","last_fullsized":"2024-09-16T09:42:53.719590+0000","mapping_epoch":454,"log_start":"467'4000","ondisk_log_start":"467'4000","created":454,"last_epoch_clean":455,"parent":"0.0","parent_split_bits":0,"last_scrub":"480'6299","last_scrub_stamp":"2024-09-16T09:42:24.525963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:41:11.976785+0000","last_clean_scrub_stamp":"2024-09-16T09:42:24.525963+0000","objects_scrubbed":6215,"log_size":4278,"log_dups_size":0,"ondisk_log_size":4278,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:51.285617+0000","scrub_duration":2772,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":512884736,"num_objects":7826,"num_object_clones":0,"num_object_copies":23478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7826,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8278,"num_write_kb":515328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"73'79","reported_seq":879,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.448183+0000","last_change":"2024-09-16T09:41:06.841984+0000","last_active":"2024-09-16T09:42:49.448183+0000","last_peered":"2024-09-16T09:42:49.448183+0000","last_clean":"2024-09-16T09:42:49.448183+0000","last_became_active":"2024-09-16T09:30:08.766350+0000","last_became_peered":"2024-09-16T09:30:08.766350+0000","last_unstale":"2024-09-16T09:42:49.448183+0000","last_undegraded":"2024-09-16T09:42:49.448183+0000","last_fullsized":"2024-09-16T09:42:49.448183+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:41:06.841934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:41:06.841934+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:34.769069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"285'42","reported_seq":1276,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.461725+0000","last_change":"2024-09-16T09:42:26.433819+0000","last_active":"2024-09-16T09:42:49.461725+0000","last_peered":"2024-09-16T09:42:49.461725+0000","last_clean":"2024-09-16T09:42:49.461725+0000","last_became_active":"2024-09-16T09:31:09.278839+0000","last_became_peered":"2024-09-16T09:31:09.278839+0000","last_unstale":"2024-09-16T09:42:49.461725+0000","last_undegraded":"2024-09-16T09:42:49.461725+0000","last_fullsized":"2024-09-16T09:42:49.461725+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'42","last_scrub_stamp":"2024-09-16T09:42:26.433720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:42:26.433720+0000","objects_scrubbed":10,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:50.838471+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":66,"num_read_kb":51,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":933,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.450545+0000","last_change":"2024-09-16T09:41:40.124915+0000","last_active":"2024-09-16T09:42:49.450545+0000","last_peered":"2024-09-16T09:42:49.450545+0000","last_clean":"2024-09-16T09:42:49.450545+0000","last_became_active":"2024-09-16T09:25:01.703076+0000","last_became_peered":"2024-09-16T09:25:01.703076+0000","last_unstale":"2024-09-16T09:42:49.450545+0000","last_undegraded":"2024-09-16T09:42:49.450545+0000","last_fullsized":"2024-09-16T09:42:49.450545+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:40.124830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:41:40.124830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:54.216409+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":934,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.448708+0000","last_change":"2024-09-16T09:41:43.807570+0000","last_active":"2024-09-16T09:42:49.448708+0000","last_peered":"2024-09-16T09:42:49.448708+0000","last_clean":"2024-09-16T09:42:49.448708+0000","last_became_active":"2024-09-16T09:25:00.696237+0000","last_became_peered":"2024-09-16T09:25:00.696237+0000","last_unstale":"2024-09-16T09:42:49.448708+0000","last_undegraded":"2024-09-16T09:42:49.448708+0000","last_fullsized":"2024-09-16T09:42:49.448708+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-16T09:41:43.807521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:41:43.807521+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:02.549457+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.0025137430000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":901,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.462927+0000","last_change":"2024-09-16T09:42:10.887831+0000","last_active":"2024-09-16T09:42:49.462927+0000","last_peered":"2024-09-16T09:42:49.462927+0000","last_clean":"2024-09-16T09:42:49.462927+0000","last_became_active":"2024-09-16T09:25:05.717591+0000","last_became_peered":"2024-09-16T09:25:05.717591+0000","last_unstale":"2024-09-16T09:42:49.462927+0000","last_undegraded":"2024-09-16T09:42:49.462927+0000","last_fullsized":"2024-09-16T09:42:49.462927+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:42:10.887766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:42:10.887766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:20.634787+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":642,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.457432+0000","last_change":"2024-09-16T09:38:25.992634+0000","last_active":"2024-09-16T09:42:49.457432+0000","last_peered":"2024-09-16T09:42:49.457432+0000","last_clean":"2024-09-16T09:42:49.457432+0000","last_became_active":"2024-09-16T09:31:09.272609+0000","last_became_peered":"2024-09-16T09:31:09.272609+0000","last_unstale":"2024-09-16T09:42:49.457432+0000","last_undegraded":"2024-09-16T09:42:49.457432+0000","last_fullsized":"2024-09-16T09:42:49.457432+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:38:25.992569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:38:25.992569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:55.709117+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.8","version":"492'8272","reported_seq":8450,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:53.058036+0000","last_change":"2024-09-16T09:42:49.548244+0000","last_active":"2024-09-16T09:42:53.058036+0000","last_peered":"2024-09-16T09:42:53.058036+0000","last_clean":"2024-09-16T09:42:53.058036+0000","last_became_active":"2024-09-16T09:41:13.001618+0000","last_became_peered":"2024-09-16T09:41:13.001618+0000","last_unstale":"2024-09-16T09:42:53.058036+0000","last_undegraded":"2024-09-16T09:42:53.058036+0000","last_fullsized":"2024-09-16T09:42:53.058036+0000","mapping_epoch":454,"log_start":"466'3300","ondisk_log_start":"466'3300","created":454,"last_epoch_clean":455,"parent":"0.0","parent_split_bits":0,"last_scrub":"492'8081","last_scrub_stamp":"2024-09-16T09:42:49.548189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:41:11.976785+0000","last_clean_scrub_stamp":"2024-09-16T09:42:49.548189+0000","objects_scrubbed":8004,"log_size":4972,"log_dups_size":0,"ondisk_log_size":4972,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:44:09.543407+0000","scrub_duration":2845,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":519962624,"num_objects":7934,"num_object_clones":0,"num_object_copies":23802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7934,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8272,"num_write_kb":518592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.2","version":"73'79","reported_seq":246,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.458613+0000","last_change":"2024-09-16T09:42:10.223632+0000","last_active":"2024-09-16T09:42:49.458613+0000","last_peered":"2024-09-16T09:42:49.458613+0000","last_clean":"2024-09-16T09:42:49.458613+0000","last_became_active":"2024-09-16T09:38:10.556320+0000","last_became_peered":"2024-09-16T09:38:10.556320+0000","last_unstale":"2024-09-16T09:42:49.458613+0000","last_undegraded":"2024-09-16T09:42:49.458613+0000","last_fullsized":"2024-09-16T09:42:49.458613+0000","mapping_epoch":386,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":387,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:42:10.223553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:42:10.223553+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:25.442293+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"285'17","reported_seq":639,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.451994+0000","last_change":"2024-09-16T09:41:27.770120+0000","last_active":"2024-09-16T09:42:49.451994+0000","last_peered":"2024-09-16T09:42:49.451994+0000","last_clean":"2024-09-16T09:42:49.451994+0000","last_became_active":"2024-09-16T09:37:33.349009+0000","last_became_peered":"2024-09-16T09:37:33.349009+0000","last_unstale":"2024-09-16T09:42:49.451994+0000","last_undegraded":"2024-09-16T09:42:49.451994+0000","last_fullsized":"2024-09-16T09:42:49.451994+0000","mapping_epoch":367,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":368,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'17","last_scrub_stamp":"2024-09-16T09:41:27.770036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:41:27.770036+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:57.206439+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":35,"num_read_kb":24,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":548,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.467206+0000","last_change":"2024-09-16T09:41:46.003680+0000","last_active":"2024-09-16T09:42:49.467206+0000","last_peered":"2024-09-16T09:42:49.467206+0000","last_clean":"2024-09-16T09:42:49.467206+0000","last_became_active":"2024-09-16T09:33:30.247596+0000","last_became_peered":"2024-09-16T09:33:30.247596+0000","last_unstale":"2024-09-16T09:42:49.467206+0000","last_undegraded":"2024-09-16T09:42:49.467206+0000","last_fullsized":"2024-09-16T09:42:49.467206+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:46.003500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:41:46.003500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:01.852542+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,2],"acting":[6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":594,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.463283+0000","last_change":"2024-09-16T09:42:44.473767+0000","last_active":"2024-09-16T09:42:49.463283+0000","last_peered":"2024-09-16T09:42:49.463283+0000","last_clean":"2024-09-16T09:42:49.463283+0000","last_became_active":"2024-09-16T09:42:44.464360+0000","last_became_peered":"2024-09-16T09:42:44.464360+0000","last_unstale":"2024-09-16T09:42:49.463283+0000","last_undegraded":"2024-09-16T09:42:49.463283+0000","last_fullsized":"2024-09-16T09:42:49.463283+0000","mapping_epoch":489,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":490,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:41:42.038861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:41:42.038861+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:57.855955+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":942,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.454917+0000","last_change":"2024-09-16T09:41:45.549361+0000","last_active":"2024-09-16T09:42:49.454917+0000","last_peered":"2024-09-16T09:42:49.454917+0000","last_clean":"2024-09-16T09:42:49.454917+0000","last_became_active":"2024-09-16T09:25:05.711539+0000","last_became_peered":"2024-09-16T09:25:05.711539+0000","last_unstale":"2024-09-16T09:42:49.454917+0000","last_undegraded":"2024-09-16T09:42:49.454917+0000","last_fullsized":"2024-09-16T09:42:49.454917+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:45.549186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:41:45.549186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:58.155707+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":910,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.459731+0000","last_change":"2024-09-16T09:41:31.217733+0000","last_active":"2024-09-16T09:42:49.459731+0000","last_peered":"2024-09-16T09:42:49.459731+0000","last_clean":"2024-09-16T09:42:49.459731+0000","last_became_active":"2024-09-16T09:25:07.723282+0000","last_became_peered":"2024-09-16T09:25:07.723282+0000","last_unstale":"2024-09-16T09:42:49.459731+0000","last_undegraded":"2024-09-16T09:42:49.459731+0000","last_fullsized":"2024-09-16T09:42:49.459731+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:31.217664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:41:31.217664+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.9","version":"492'8331","reported_seq":8488,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:53.765765+0000","last_change":"2024-09-16T09:42:18.030926+0000","last_active":"2024-09-16T09:42:53.765765+0000","last_peered":"2024-09-16T09:42:53.765765+0000","last_clean":"2024-09-16T09:42:53.765765+0000","last_became_active":"2024-09-16T09:41:13.001488+0000","last_became_peered":"2024-09-16T09:41:13.001488+0000","last_unstale":"2024-09-16T09:42:53.765765+0000","last_undegraded":"2024-09-16T09:42:53.765765+0000","last_fullsized":"2024-09-16T09:42:53.765765+0000","mapping_epoch":454,"log_start":"467'4100","ondisk_log_start":"467'4100","created":454,"last_epoch_clean":455,"parent":"0.0","parent_split_bits":0,"last_scrub":"478'5946","last_scrub_stamp":"2024-09-16T09:42:18.030634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:41:11.976785+0000","last_clean_scrub_stamp":"2024-09-16T09:42:18.030634+0000","objects_scrubbed":5872,"log_size":4231,"log_dups_size":0,"ondisk_log_size":4231,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:45.360602+0000","scrub_duration":2321,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":514392064,"num_objects":7849,"num_object_clones":0,"num_object_copies":23547,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7849,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8331,"num_write_kb":517760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"73'79","reported_seq":629,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.461445+0000","last_change":"2024-09-16T09:42:21.764931+0000","last_active":"2024-09-16T09:42:49.461445+0000","last_peered":"2024-09-16T09:42:49.461445+0000","last_clean":"2024-09-16T09:42:49.461445+0000","last_became_active":"2024-09-16T09:33:30.249309+0000","last_became_peered":"2024-09-16T09:33:30.249309+0000","last_unstale":"2024-09-16T09:42:49.461445+0000","last_undegraded":"2024-09-16T09:42:49.461445+0000","last_fullsized":"2024-09-16T09:42:49.461445+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:42:21.764853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:42:21.764853+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:51.421747+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.6","version":"285'24","reported_seq":957,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.463543+0000","last_change":"2024-09-16T09:42:11.911983+0000","last_active":"2024-09-16T09:42:49.463543+0000","last_peered":"2024-09-16T09:42:49.463543+0000","last_clean":"2024-09-16T09:42:49.463543+0000","last_became_active":"2024-09-16T09:25:03.715315+0000","last_became_peered":"2024-09-16T09:25:03.715315+0000","last_unstale":"2024-09-16T09:42:49.463543+0000","last_undegraded":"2024-09-16T09:42:49.463543+0000","last_fullsized":"2024-09-16T09:42:49.463543+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'24","last_scrub_stamp":"2024-09-16T09:42:11.911889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:42:11.911889+0000","objects_scrubbed":6,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:32.028734+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":36,"num_read_kb":30,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"59'1","reported_seq":948,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.463194+0000","last_change":"2024-09-16T09:41:37.223719+0000","last_active":"2024-09-16T09:42:49.463194+0000","last_peered":"2024-09-16T09:42:49.463194+0000","last_clean":"2024-09-16T09:42:49.463194+0000","last_became_active":"2024-09-16T09:31:09.269627+0000","last_became_peered":"2024-09-16T09:31:09.269627+0000","last_unstale":"2024-09-16T09:42:49.463194+0000","last_undegraded":"2024-09-16T09:42:49.463194+0000","last_fullsized":"2024-09-16T09:42:49.463194+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:41:37.223543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:41:37.223543+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:50.479230+0000","scrub_duration":61,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1986,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"67'5","reported_seq":1997,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:51.493498+0000","last_change":"2024-09-16T09:41:43.153303+0000","last_active":"2024-09-16T09:42:51.493498+0000","last_peered":"2024-09-16T09:42:51.493498+0000","last_clean":"2024-09-16T09:42:51.493498+0000","last_became_active":"2024-09-16T09:25:00.696546+0000","last_became_peered":"2024-09-16T09:25:00.696546+0000","last_unstale":"2024-09-16T09:42:51.493498+0000","last_undegraded":"2024-09-16T09:42:51.493498+0000","last_fullsized":"2024-09-16T09:42:51.493498+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'5","last_scrub_stamp":"2024-09-16T09:41:43.153174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:41:43.153174+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0.0059770420000000001,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1064,"num_read_kb":1059,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":892,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.452072+0000","last_change":"2024-09-16T09:42:21.752388+0000","last_active":"2024-09-16T09:42:49.452072+0000","last_peered":"2024-09-16T09:42:49.452072+0000","last_clean":"2024-09-16T09:42:49.452072+0000","last_became_active":"2024-09-16T09:25:05.714704+0000","last_became_peered":"2024-09-16T09:25:05.714704+0000","last_unstale":"2024-09-16T09:42:49.452072+0000","last_undegraded":"2024-09-16T09:42:49.452072+0000","last_fullsized":"2024-09-16T09:42:49.452072+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:42:21.752330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:42:21.752330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:45.192562+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":826,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.449251+0000","last_change":"2024-09-16T09:42:11.738344+0000","last_active":"2024-09-16T09:42:49.449251+0000","last_peered":"2024-09-16T09:42:49.449251+0000","last_clean":"2024-09-16T09:42:49.449251+0000","last_became_active":"2024-09-16T09:33:30.245852+0000","last_became_peered":"2024-09-16T09:33:30.245852+0000","last_unstale":"2024-09-16T09:42:49.449251+0000","last_undegraded":"2024-09-16T09:42:49.449251+0000","last_fullsized":"2024-09-16T09:42:49.449251+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:42:11.738297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:42:11.738297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:23.216675+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.a","version":"492'8300","reported_seq":8475,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:53.055154+0000","last_change":"2024-09-16T09:42:52.574225+0000","last_active":"2024-09-16T09:42:53.055154+0000","last_peered":"2024-09-16T09:42:53.055154+0000","last_clean":"2024-09-16T09:42:53.055154+0000","last_became_active":"2024-09-16T09:41:13.001938+0000","last_became_peered":"2024-09-16T09:41:13.001938+0000","last_unstale":"2024-09-16T09:42:53.055154+0000","last_undegraded":"2024-09-16T09:42:53.055154+0000","last_fullsized":"2024-09-16T09:42:53.055154+0000","mapping_epoch":454,"log_start":"466'3300","ondisk_log_start":"466'3300","created":454,"last_epoch_clean":455,"parent":"0.0","parent_split_bits":0,"last_scrub":"492'8290","last_scrub_stamp":"2024-09-16T09:42:52.574151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:41:11.976785+0000","last_clean_scrub_stamp":"2024-09-16T09:42:52.574151+0000","objects_scrubbed":8001,"log_size":5000,"log_dups_size":0,"ondisk_log_size":5000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:57.743817+0000","scrub_duration":1955,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":517341184,"num_objects":7894,"num_object_clones":0,"num_object_copies":23682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7894,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8300,"num_write_kb":518208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"285'28","reported_seq":1176,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.460691+0000","last_change":"2024-09-16T09:42:17.838803+0000","last_active":"2024-09-16T09:42:49.460691+0000","last_peered":"2024-09-16T09:42:49.460691+0000","last_clean":"2024-09-16T09:42:49.460691+0000","last_became_active":"2024-09-16T09:25:03.714118+0000","last_became_peered":"2024-09-16T09:25:03.714118+0000","last_unstale":"2024-09-16T09:42:49.460691+0000","last_undegraded":"2024-09-16T09:42:49.460691+0000","last_fullsized":"2024-09-16T09:42:49.460691+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'28","last_scrub_stamp":"2024-09-16T09:42:17.838730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:42:17.838730+0000","objects_scrubbed":8,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:20.558460+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":43,"num_read_kb":33,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":766,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.448695+0000","last_change":"2024-09-16T09:41:09.792929+0000","last_active":"2024-09-16T09:42:49.448695+0000","last_peered":"2024-09-16T09:42:49.448695+0000","last_clean":"2024-09-16T09:42:49.448695+0000","last_became_active":"2024-09-16T09:31:09.270196+0000","last_became_peered":"2024-09-16T09:31:09.270196+0000","last_unstale":"2024-09-16T09:42:49.448695+0000","last_undegraded":"2024-09-16T09:42:49.448695+0000","last_fullsized":"2024-09-16T09:42:49.448695+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:09.792874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:41:09.792874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"73'79","reported_seq":994,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.458979+0000","last_change":"2024-09-16T09:39:39.878540+0000","last_active":"2024-09-16T09:42:49.458979+0000","last_peered":"2024-09-16T09:42:49.458979+0000","last_clean":"2024-09-16T09:42:49.458979+0000","last_became_active":"2024-09-16T09:30:08.870394+0000","last_became_peered":"2024-09-16T09:30:08.870394+0000","last_unstale":"2024-09-16T09:42:49.458979+0000","last_undegraded":"2024-09-16T09:42:49.458979+0000","last_fullsized":"2024-09-16T09:42:49.458979+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:39:39.878410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:39:39.878410+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:56.170803+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":914,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.459749+0000","last_change":"2024-09-16T09:42:11.246777+0000","last_active":"2024-09-16T09:42:49.459749+0000","last_peered":"2024-09-16T09:42:49.459749+0000","last_clean":"2024-09-16T09:42:49.459749+0000","last_became_active":"2024-09-16T09:25:05.714341+0000","last_became_peered":"2024-09-16T09:25:05.714341+0000","last_unstale":"2024-09-16T09:42:49.459749+0000","last_undegraded":"2024-09-16T09:42:49.459749+0000","last_fullsized":"2024-09-16T09:42:49.459749+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:42:11.246697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:42:11.246697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:29.196378+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":904,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.451897+0000","last_change":"2024-09-16T09:41:38.660523+0000","last_active":"2024-09-16T09:42:49.451897+0000","last_peered":"2024-09-16T09:42:49.451897+0000","last_clean":"2024-09-16T09:42:49.451897+0000","last_became_active":"2024-09-16T09:25:07.714281+0000","last_became_peered":"2024-09-16T09:25:07.714281+0000","last_unstale":"2024-09-16T09:42:49.451897+0000","last_undegraded":"2024-09-16T09:42:49.451897+0000","last_fullsized":"2024-09-16T09:42:49.451897+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:38.660336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:41:38.660336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1","version":"492'8240","reported_seq":8391,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:53.007343+0000","last_change":"2024-09-16T09:42:14.662949+0000","last_active":"2024-09-16T09:42:53.007343+0000","last_peered":"2024-09-16T09:42:53.007343+0000","last_clean":"2024-09-16T09:42:53.007343+0000","last_became_active":"2024-09-16T09:41:13.003635+0000","last_became_peered":"2024-09-16T09:41:13.003635+0000","last_unstale":"2024-09-16T09:42:53.007343+0000","last_undegraded":"2024-09-16T09:42:53.007343+0000","last_fullsized":"2024-09-16T09:42:53.007343+0000","mapping_epoch":454,"log_start":"466'3300","ondisk_log_start":"466'3300","created":454,"last_epoch_clean":455,"parent":"0.0","parent_split_bits":0,"last_scrub":"476'5603","last_scrub_stamp":"2024-09-16T09:42:14.662822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:41:11.976785+0000","last_clean_scrub_stamp":"2024-09-16T09:42:14.662822+0000","objects_scrubbed":5530,"log_size":4940,"log_dups_size":0,"ondisk_log_size":4940,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:31.298949+0000","scrub_duration":1836,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":519831552,"num_objects":7932,"num_object_clones":0,"num_object_copies":23796,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7932,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8240,"num_write_kb":517504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.b","version":"73'79","reported_seq":748,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.461208+0000","last_change":"2024-09-16T09:42:26.898435+0000","last_active":"2024-09-16T09:42:49.461208+0000","last_peered":"2024-09-16T09:42:49.461208+0000","last_clean":"2024-09-16T09:42:49.461208+0000","last_became_active":"2024-09-16T09:33:30.251933+0000","last_became_peered":"2024-09-16T09:33:30.251933+0000","last_unstale":"2024-09-16T09:42:49.461208+0000","last_undegraded":"2024-09-16T09:42:49.461208+0000","last_fullsized":"2024-09-16T09:42:49.461208+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:42:26.898235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:42:26.898235+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:48.140906+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"285'19","reported_seq":992,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.451609+0000","last_change":"2024-09-16T09:41:03.243490+0000","last_active":"2024-09-16T09:42:49.451609+0000","last_peered":"2024-09-16T09:42:49.451609+0000","last_clean":"2024-09-16T09:42:49.451609+0000","last_became_active":"2024-09-16T09:37:33.348433+0000","last_became_peered":"2024-09-16T09:37:33.348433+0000","last_unstale":"2024-09-16T09:42:49.451609+0000","last_undegraded":"2024-09-16T09:42:49.451609+0000","last_fullsized":"2024-09-16T09:42:49.451609+0000","mapping_epoch":367,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":368,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'19","last_scrub_stamp":"2024-09-16T09:41:03.243406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:41:03.243406+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":909,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.450170+0000","last_change":"2024-09-16T09:42:18.132162+0000","last_active":"2024-09-16T09:42:49.450170+0000","last_peered":"2024-09-16T09:42:49.450170+0000","last_clean":"2024-09-16T09:42:49.450170+0000","last_became_active":"2024-09-16T09:25:01.708270+0000","last_became_peered":"2024-09-16T09:25:01.708270+0000","last_unstale":"2024-09-16T09:42:49.450170+0000","last_undegraded":"2024-09-16T09:42:49.450170+0000","last_fullsized":"2024-09-16T09:42:49.450170+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:42:18.132091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:42:18.132091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:32.241263+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":891,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.451841+0000","last_change":"2024-09-16T09:42:18.730769+0000","last_active":"2024-09-16T09:42:49.451841+0000","last_peered":"2024-09-16T09:42:49.451841+0000","last_clean":"2024-09-16T09:42:49.451841+0000","last_became_active":"2024-09-16T09:25:05.711964+0000","last_became_peered":"2024-09-16T09:25:05.711964+0000","last_unstale":"2024-09-16T09:42:49.451841+0000","last_undegraded":"2024-09-16T09:42:49.451841+0000","last_fullsized":"2024-09-16T09:42:49.451841+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:42:18.730723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:42:18.730723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:36.315848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":791,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.457222+0000","last_change":"2024-09-16T09:42:09.944417+0000","last_active":"2024-09-16T09:42:49.457222+0000","last_peered":"2024-09-16T09:42:49.457222+0000","last_clean":"2024-09-16T09:42:49.457222+0000","last_became_active":"2024-09-16T09:31:09.278618+0000","last_became_peered":"2024-09-16T09:31:09.278618+0000","last_unstale":"2024-09-16T09:42:49.457222+0000","last_undegraded":"2024-09-16T09:42:49.457222+0000","last_fullsized":"2024-09-16T09:42:49.457222+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:42:09.944311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:42:09.944311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:12.734923+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"492'8350","reported_seq":8438,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:53.763657+0000","last_change":"2024-09-16T09:41:12.998092+0000","last_active":"2024-09-16T09:42:53.763657+0000","last_peered":"2024-09-16T09:42:53.763657+0000","last_clean":"2024-09-16T09:42:53.763657+0000","last_became_active":"2024-09-16T09:41:12.997802+0000","last_became_peered":"2024-09-16T09:41:12.997802+0000","last_unstale":"2024-09-16T09:42:53.763657+0000","last_undegraded":"2024-09-16T09:42:53.763657+0000","last_fullsized":"2024-09-16T09:42:53.763657+0000","mapping_epoch":454,"log_start":"467'4100","ondisk_log_start":"467'4100","created":454,"last_epoch_clean":455,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:11.976785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:41:11.976785+0000","last_clean_scrub_stamp":"2024-09-16T09:41:11.976785+0000","objects_scrubbed":0,"log_size":4250,"log_dups_size":0,"ondisk_log_size":4250,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:44.008139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":518651904,"num_objects":7914,"num_object_clones":0,"num_object_copies":23742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7914,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8350,"num_write_kb":520448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.8","version":"73'79","reported_seq":251,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.448188+0000","last_change":"2024-09-16T09:41:31.767349+0000","last_active":"2024-09-16T09:42:49.448188+0000","last_peered":"2024-09-16T09:42:49.448188+0000","last_clean":"2024-09-16T09:42:49.448188+0000","last_became_active":"2024-09-16T09:37:53.418691+0000","last_became_peered":"2024-09-16T09:37:53.418691+0000","last_unstale":"2024-09-16T09:42:49.448188+0000","last_undegraded":"2024-09-16T09:42:49.448188+0000","last_fullsized":"2024-09-16T09:42:49.448188+0000","mapping_epoch":382,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":383,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:41:31.767299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:41:31.767299+0000","objects_scrubbed":1,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:51.743338+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262688,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"285'31","reported_seq":969,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.452997+0000","last_change":"2024-09-16T09:42:07.153346+0000","last_active":"2024-09-16T09:42:49.452997+0000","last_peered":"2024-09-16T09:42:49.452997+0000","last_clean":"2024-09-16T09:42:49.452997+0000","last_became_active":"2024-09-16T09:25:03.713358+0000","last_became_peered":"2024-09-16T09:25:03.713358+0000","last_unstale":"2024-09-16T09:42:49.452997+0000","last_undegraded":"2024-09-16T09:42:49.452997+0000","last_fullsized":"2024-09-16T09:42:49.452997+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'31","last_scrub_stamp":"2024-09-16T09:42:07.153276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:42:07.153276+0000","objects_scrubbed":8,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:21.444481+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":50,"num_read_kb":40,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":917,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.458129+0000","last_change":"2024-09-16T09:41:07.972297+0000","last_active":"2024-09-16T09:42:49.458129+0000","last_peered":"2024-09-16T09:42:49.458129+0000","last_clean":"2024-09-16T09:42:49.458129+0000","last_became_active":"2024-09-16T09:25:01.703355+0000","last_became_peered":"2024-09-16T09:25:01.703355+0000","last_unstale":"2024-09-16T09:42:49.458129+0000","last_undegraded":"2024-09-16T09:42:49.458129+0000","last_fullsized":"2024-09-16T09:42:49.458129+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:07.972246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:41:07.972246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":548,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.450733+0000","last_change":"2024-09-16T09:37:46.378016+0000","last_active":"2024-09-16T09:42:49.450733+0000","last_peered":"2024-09-16T09:42:49.450733+0000","last_clean":"2024-09-16T09:42:49.450733+0000","last_became_active":"2024-09-16T09:33:30.251186+0000","last_became_peered":"2024-09-16T09:33:30.251186+0000","last_unstale":"2024-09-16T09:42:49.450733+0000","last_undegraded":"2024-09-16T09:42:49.450733+0000","last_fullsized":"2024-09-16T09:42:49.450733+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:46.377951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:37:46.377951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:39.888937+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,0],"acting":[4,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":846,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.467028+0000","last_change":"2024-09-16T09:38:24.031758+0000","last_active":"2024-09-16T09:42:49.467028+0000","last_peered":"2024-09-16T09:42:49.467028+0000","last_clean":"2024-09-16T09:42:49.467028+0000","last_became_active":"2024-09-16T09:33:21.337291+0000","last_became_peered":"2024-09-16T09:33:21.337291+0000","last_unstale":"2024-09-16T09:42:49.467028+0000","last_undegraded":"2024-09-16T09:42:49.467028+0000","last_fullsized":"2024-09-16T09:42:49.467028+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:38:24.031592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:38:24.031592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:58.675217+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4],"acting":[3,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3","version":"492'8316","reported_seq":8403,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:53.071765+0000","last_change":"2024-09-16T09:41:13.139231+0000","last_active":"2024-09-16T09:42:53.071765+0000","last_peered":"2024-09-16T09:42:53.071765+0000","last_clean":"2024-09-16T09:42:53.071765+0000","last_became_active":"2024-09-16T09:41:13.138925+0000","last_became_peered":"2024-09-16T09:41:13.138925+0000","last_unstale":"2024-09-16T09:42:53.071765+0000","last_undegraded":"2024-09-16T09:42:53.071765+0000","last_fullsized":"2024-09-16T09:42:53.071765+0000","mapping_epoch":454,"log_start":"466'3300","ondisk_log_start":"466'3300","created":454,"last_epoch_clean":455,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:11.976785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:41:11.976785+0000","last_clean_scrub_stamp":"2024-09-16T09:41:11.976785+0000","objects_scrubbed":0,"log_size":5016,"log_dups_size":0,"ondisk_log_size":5016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":521273344,"num_objects":7954,"num_object_clones":0,"num_object_copies":23862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7954,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8316,"num_write_kb":520640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1],"acting":[6,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.9","version":"73'79","reported_seq":978,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.459013+0000","last_change":"2024-09-16T09:41:40.124011+0000","last_active":"2024-09-16T09:42:49.459013+0000","last_peered":"2024-09-16T09:42:49.459013+0000","last_clean":"2024-09-16T09:42:49.459013+0000","last_became_active":"2024-09-16T09:26:35.126066+0000","last_became_peered":"2024-09-16T09:26:35.126066+0000","last_unstale":"2024-09-16T09:42:49.459013+0000","last_undegraded":"2024-09-16T09:42:49.459013+0000","last_fullsized":"2024-09-16T09:42:49.459013+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:41:40.123937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:41:40.123937+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:04.683364+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"285'20","reported_seq":934,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.450942+0000","last_change":"2024-09-16T09:41:42.111004+0000","last_active":"2024-09-16T09:42:49.450942+0000","last_peered":"2024-09-16T09:42:49.450942+0000","last_clean":"2024-09-16T09:42:49.450942+0000","last_became_active":"2024-09-16T09:25:03.707723+0000","last_became_peered":"2024-09-16T09:25:03.707723+0000","last_unstale":"2024-09-16T09:42:49.450942+0000","last_undegraded":"2024-09-16T09:42:49.450942+0000","last_fullsized":"2024-09-16T09:42:49.450942+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'20","last_scrub_stamp":"2024-09-16T09:41:42.110923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:41:42.110923+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":774,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.461807+0000","last_change":"2024-09-16T09:39:38.508106+0000","last_active":"2024-09-16T09:42:49.461807+0000","last_peered":"2024-09-16T09:42:49.461807+0000","last_clean":"2024-09-16T09:42:49.461807+0000","last_became_active":"2024-09-16T09:31:09.288825+0000","last_became_peered":"2024-09-16T09:31:09.288825+0000","last_unstale":"2024-09-16T09:42:49.461807+0000","last_undegraded":"2024-09-16T09:42:49.461807+0000","last_fullsized":"2024-09-16T09:42:49.461807+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:39:38.507898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:39:38.507898+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:50.039752+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"228'9","reported_seq":710,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.459848+0000","last_change":"2024-09-16T09:41:33.174507+0000","last_active":"2024-09-16T09:42:49.459848+0000","last_peered":"2024-09-16T09:42:49.459848+0000","last_clean":"2024-09-16T09:42:49.459848+0000","last_became_active":"2024-09-16T09:33:21.337213+0000","last_became_peered":"2024-09-16T09:33:21.337213+0000","last_unstale":"2024-09-16T09:42:49.459848+0000","last_undegraded":"2024-09-16T09:42:49.459848+0000","last_fullsized":"2024-09-16T09:42:49.459848+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"228'9","last_scrub_stamp":"2024-09-16T09:41:33.174444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:41:33.174444+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":928,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.451370+0000","last_change":"2024-09-16T09:41:39.100268+0000","last_active":"2024-09-16T09:42:49.451370+0000","last_peered":"2024-09-16T09:42:49.451370+0000","last_clean":"2024-09-16T09:42:49.451370+0000","last_became_active":"2024-09-16T09:25:07.718958+0000","last_became_peered":"2024-09-16T09:25:07.718958+0000","last_unstale":"2024-09-16T09:42:49.451370+0000","last_undegraded":"2024-09-16T09:42:49.451370+0000","last_fullsized":"2024-09-16T09:42:49.451370+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:39.100089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:41:39.100089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.4","version":"492'8350","reported_seq":8437,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:52.529102+0000","last_change":"2024-09-16T09:41:13.004443+0000","last_active":"2024-09-16T09:42:52.529102+0000","last_peered":"2024-09-16T09:42:52.529102+0000","last_clean":"2024-09-16T09:42:52.529102+0000","last_became_active":"2024-09-16T09:41:13.004032+0000","last_became_peered":"2024-09-16T09:41:13.004032+0000","last_unstale":"2024-09-16T09:42:52.529102+0000","last_undegraded":"2024-09-16T09:42:52.529102+0000","last_fullsized":"2024-09-16T09:42:52.529102+0000","mapping_epoch":454,"log_start":"467'4000","ondisk_log_start":"467'4000","created":454,"last_epoch_clean":455,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:11.976785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:41:11.976785+0000","last_clean_scrub_stamp":"2024-09-16T09:41:11.976785+0000","objects_scrubbed":0,"log_size":4350,"log_dups_size":0,"ondisk_log_size":4350,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":521928704,"num_objects":7964,"num_object_clones":0,"num_object_copies":23892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7964,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8350,"num_write_kb":522048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.e","version":"73'79","reported_seq":713,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.451254+0000","last_change":"2024-09-16T09:42:16.139285+0000","last_active":"2024-09-16T09:42:49.451254+0000","last_peered":"2024-09-16T09:42:49.451254+0000","last_clean":"2024-09-16T09:42:49.451254+0000","last_became_active":"2024-09-16T09:30:10.705072+0000","last_became_peered":"2024-09-16T09:30:10.705072+0000","last_unstale":"2024-09-16T09:42:49.451254+0000","last_undegraded":"2024-09-16T09:42:49.451254+0000","last_fullsized":"2024-09-16T09:42:49.451254+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:42:16.139208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:42:16.139208+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:23.579502+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,3],"acting":[4,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.b","version":"285'15","reported_seq":962,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.461554+0000","last_change":"2024-09-16T09:41:09.042482+0000","last_active":"2024-09-16T09:42:49.461554+0000","last_peered":"2024-09-16T09:42:49.461554+0000","last_clean":"2024-09-16T09:42:49.461554+0000","last_became_active":"2024-09-16T09:36:31.590202+0000","last_became_peered":"2024-09-16T09:36:31.590202+0000","last_unstale":"2024-09-16T09:42:49.461554+0000","last_undegraded":"2024-09-16T09:42:49.461554+0000","last_fullsized":"2024-09-16T09:42:49.461554+0000","mapping_epoch":316,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":317,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'15","last_scrub_stamp":"2024-09-16T09:41:09.039458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:41:09.039458+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:39.830621+0000","scrub_duration":41,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":921,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.449341+0000","last_change":"2024-09-16T09:42:30.709733+0000","last_active":"2024-09-16T09:42:49.449341+0000","last_peered":"2024-09-16T09:42:49.449341+0000","last_clean":"2024-09-16T09:42:49.449341+0000","last_became_active":"2024-09-16T09:31:09.276516+0000","last_became_peered":"2024-09-16T09:31:09.276516+0000","last_unstale":"2024-09-16T09:42:49.449341+0000","last_undegraded":"2024-09-16T09:42:49.449341+0000","last_fullsized":"2024-09-16T09:42:49.449341+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:42:30.709673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:42:30.709673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:37.628810+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":834,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.449827+0000","last_change":"2024-09-16T09:33:40.260547+0000","last_active":"2024-09-16T09:42:49.449827+0000","last_peered":"2024-09-16T09:42:49.449827+0000","last_clean":"2024-09-16T09:42:49.449827+0000","last_became_active":"2024-09-16T09:33:21.337254+0000","last_became_peered":"2024-09-16T09:33:21.337254+0000","last_unstale":"2024-09-16T09:42:49.449827+0000","last_undegraded":"2024-09-16T09:42:49.449827+0000","last_fullsized":"2024-09-16T09:42:49.449827+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:33:40.260500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:33:40.260500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:53.192937+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3],"acting":[4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":913,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.463135+0000","last_change":"2024-09-16T09:42:19.851067+0000","last_active":"2024-09-16T09:42:49.463135+0000","last_peered":"2024-09-16T09:42:49.463135+0000","last_clean":"2024-09-16T09:42:49.463135+0000","last_became_active":"2024-09-16T09:25:07.718778+0000","last_became_peered":"2024-09-16T09:25:07.718778+0000","last_unstale":"2024-09-16T09:42:49.463135+0000","last_undegraded":"2024-09-16T09:42:49.463135+0000","last_fullsized":"2024-09-16T09:42:49.463135+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:42:19.850990+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:42:19.850990+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:45.616393+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.5","version":"492'8491","reported_seq":8578,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:52.551896+0000","last_change":"2024-09-16T09:41:13.004151+0000","last_active":"2024-09-16T09:42:52.551896+0000","last_peered":"2024-09-16T09:42:52.551896+0000","last_clean":"2024-09-16T09:42:52.551896+0000","last_became_active":"2024-09-16T09:41:13.003693+0000","last_became_peered":"2024-09-16T09:41:13.003693+0000","last_unstale":"2024-09-16T09:42:52.551896+0000","last_undegraded":"2024-09-16T09:42:52.551896+0000","last_fullsized":"2024-09-16T09:42:52.551896+0000","mapping_epoch":454,"log_start":"467'4100","ondisk_log_start":"467'4100","created":454,"last_epoch_clean":455,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:11.976785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:41:11.976785+0000","last_clean_scrub_stamp":"2024-09-16T09:41:11.976785+0000","objects_scrubbed":0,"log_size":4391,"log_dups_size":0,"ondisk_log_size":4391,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":530907136,"num_objects":8101,"num_object_clones":0,"num_object_copies":24303,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8491,"num_write_kb":530944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.f","version":"73'79","reported_seq":973,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.459081+0000","last_change":"2024-09-16T09:41:28.953879+0000","last_active":"2024-09-16T09:42:49.459081+0000","last_peered":"2024-09-16T09:42:49.459081+0000","last_clean":"2024-09-16T09:42:49.459081+0000","last_became_active":"2024-09-16T09:30:10.693477+0000","last_became_peered":"2024-09-16T09:30:10.693477+0000","last_unstale":"2024-09-16T09:42:49.459081+0000","last_undegraded":"2024-09-16T09:42:49.459081+0000","last_fullsized":"2024-09-16T09:42:49.459081+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:41:28.953810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:41:28.953810+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:52.357805+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"285'35","reported_seq":968,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.457261+0000","last_change":"2024-09-16T09:37:28.078195+0000","last_active":"2024-09-16T09:42:49.457261+0000","last_peered":"2024-09-16T09:42:49.457261+0000","last_clean":"2024-09-16T09:42:49.457261+0000","last_became_active":"2024-09-16T09:25:03.712783+0000","last_became_peered":"2024-09-16T09:25:03.712783+0000","last_unstale":"2024-09-16T09:42:49.457261+0000","last_undegraded":"2024-09-16T09:42:49.457261+0000","last_fullsized":"2024-09-16T09:42:49.457261+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'35","last_scrub_stamp":"2024-09-16T09:37:28.077841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:37:28.077841+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:56.886841+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":56,"num_read_kb":45,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"59'1","reported_seq":927,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.460051+0000","last_change":"2024-09-16T09:42:24.291119+0000","last_active":"2024-09-16T09:42:49.460051+0000","last_peered":"2024-09-16T09:42:49.460051+0000","last_clean":"2024-09-16T09:42:49.460051+0000","last_became_active":"2024-09-16T09:25:01.708069+0000","last_became_peered":"2024-09-16T09:25:01.708069+0000","last_unstale":"2024-09-16T09:42:49.460051+0000","last_undegraded":"2024-09-16T09:42:49.460051+0000","last_fullsized":"2024-09-16T09:42:49.460051+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:42:24.291047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:42:24.291047+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:49.390756+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":939,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.454810+0000","last_change":"2024-09-16T09:40:39.696450+0000","last_active":"2024-09-16T09:42:49.454810+0000","last_peered":"2024-09-16T09:42:49.454810+0000","last_clean":"2024-09-16T09:42:49.454810+0000","last_became_active":"2024-09-16T09:37:53.518284+0000","last_became_peered":"2024-09-16T09:37:53.518284+0000","last_unstale":"2024-09-16T09:42:49.454810+0000","last_undegraded":"2024-09-16T09:42:49.454810+0000","last_fullsized":"2024-09-16T09:42:49.454810+0000","mapping_epoch":382,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":383,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:40:39.696383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:40:39.696383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:02.607251+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":894,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.458934+0000","last_change":"2024-09-16T09:42:15.241979+0000","last_active":"2024-09-16T09:42:49.458934+0000","last_peered":"2024-09-16T09:42:49.458934+0000","last_clean":"2024-09-16T09:42:49.458934+0000","last_became_active":"2024-09-16T09:25:07.718750+0000","last_became_peered":"2024-09-16T09:25:07.718750+0000","last_unstale":"2024-09-16T09:42:49.458934+0000","last_undegraded":"2024-09-16T09:42:49.458934+0000","last_fullsized":"2024-09-16T09:42:49.458934+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:42:15.241925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:42:15.241925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:43.792299+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.6","version":"492'8194","reported_seq":8279,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.469970+0000","last_change":"2024-09-16T09:41:13.139206+0000","last_active":"2024-09-16T09:42:49.469970+0000","last_peered":"2024-09-16T09:42:49.469970+0000","last_clean":"2024-09-16T09:42:49.469970+0000","last_became_active":"2024-09-16T09:41:13.138910+0000","last_became_peered":"2024-09-16T09:41:13.138910+0000","last_unstale":"2024-09-16T09:42:49.469970+0000","last_undegraded":"2024-09-16T09:42:49.469970+0000","last_fullsized":"2024-09-16T09:42:49.469970+0000","mapping_epoch":454,"log_start":"464'2800","ondisk_log_start":"464'2800","created":454,"last_epoch_clean":455,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:11.976785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:41:11.976785+0000","last_clean_scrub_stamp":"2024-09-16T09:41:11.976785+0000","objects_scrubbed":0,"log_size":5394,"log_dups_size":0,"ondisk_log_size":5394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:38.820039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":537001984,"num_objects":8194,"num_object_clones":0,"num_object_copies":24582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8194,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8194,"num_write_kb":524416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.c","version":"73'79","reported_seq":662,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.457331+0000","last_change":"2024-09-16T09:42:09.710822+0000","last_active":"2024-09-16T09:42:49.457331+0000","last_peered":"2024-09-16T09:42:49.457331+0000","last_clean":"2024-09-16T09:42:49.457331+0000","last_became_active":"2024-09-16T09:31:09.273770+0000","last_became_peered":"2024-09-16T09:31:09.273770+0000","last_unstale":"2024-09-16T09:42:49.457331+0000","last_undegraded":"2024-09-16T09:42:49.457331+0000","last_fullsized":"2024-09-16T09:42:49.457331+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:42:09.710756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:42:09.710756+0000","objects_scrubbed":1,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:23.803158+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":327680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"285'18","reported_seq":991,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.450071+0000","last_change":"2024-09-16T09:42:17.189319+0000","last_active":"2024-09-16T09:42:49.450071+0000","last_peered":"2024-09-16T09:42:49.450071+0000","last_clean":"2024-09-16T09:42:49.450071+0000","last_became_active":"2024-09-16T09:31:09.269047+0000","last_became_peered":"2024-09-16T09:31:09.269047+0000","last_unstale":"2024-09-16T09:42:49.450071+0000","last_undegraded":"2024-09-16T09:42:49.450071+0000","last_fullsized":"2024-09-16T09:42:49.450071+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'18","last_scrub_stamp":"2024-09-16T09:42:17.189230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:42:17.189230+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:31.195811+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":921,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.460405+0000","last_change":"2024-09-16T09:42:33.321184+0000","last_active":"2024-09-16T09:42:49.460405+0000","last_peered":"2024-09-16T09:42:49.460405+0000","last_clean":"2024-09-16T09:42:49.460405+0000","last_became_active":"2024-09-16T09:25:01.707528+0000","last_became_peered":"2024-09-16T09:25:01.707528+0000","last_unstale":"2024-09-16T09:42:49.460405+0000","last_undegraded":"2024-09-16T09:42:49.460405+0000","last_fullsized":"2024-09-16T09:42:49.460405+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:42:33.321125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:42:33.321125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:44.591182+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":576,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.460504+0000","last_change":"2024-09-16T09:42:12.922219+0000","last_active":"2024-09-16T09:42:49.460504+0000","last_peered":"2024-09-16T09:42:49.460504+0000","last_clean":"2024-09-16T09:42:49.460504+0000","last_became_active":"2024-09-16T09:33:21.337309+0000","last_became_peered":"2024-09-16T09:33:21.337309+0000","last_unstale":"2024-09-16T09:42:49.460504+0000","last_undegraded":"2024-09-16T09:42:49.460504+0000","last_fullsized":"2024-09-16T09:42:49.460504+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:42:12.922136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:42:12.922136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:19.988335+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":904,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.457680+0000","last_change":"2024-09-16T09:38:21.097872+0000","last_active":"2024-09-16T09:42:49.457680+0000","last_peered":"2024-09-16T09:42:49.457680+0000","last_clean":"2024-09-16T09:42:49.457680+0000","last_became_active":"2024-09-16T09:25:07.718899+0000","last_became_peered":"2024-09-16T09:25:07.718899+0000","last_unstale":"2024-09-16T09:42:49.457680+0000","last_undegraded":"2024-09-16T09:42:49.457680+0000","last_fullsized":"2024-09-16T09:42:49.457680+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:38:21.097804+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:38:21.097804+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:59.651567+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.7","version":"492'8363","reported_seq":8450,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:53.328782+0000","last_change":"2024-09-16T09:41:13.139096+0000","last_active":"2024-09-16T09:42:53.328782+0000","last_peered":"2024-09-16T09:42:53.328782+0000","last_clean":"2024-09-16T09:42:53.328782+0000","last_became_active":"2024-09-16T09:41:13.138764+0000","last_became_peered":"2024-09-16T09:41:13.138764+0000","last_unstale":"2024-09-16T09:42:53.328782+0000","last_undegraded":"2024-09-16T09:42:53.328782+0000","last_fullsized":"2024-09-16T09:42:53.328782+0000","mapping_epoch":454,"log_start":"466'3500","ondisk_log_start":"466'3500","created":454,"last_epoch_clean":455,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:11.976785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:41:11.976785+0000","last_clean_scrub_stamp":"2024-09-16T09:41:11.976785+0000","objects_scrubbed":0,"log_size":4863,"log_dups_size":0,"ondisk_log_size":4863,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:43.894479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":522780672,"num_objects":7977,"num_object_clones":0,"num_object_copies":23931,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7977,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8363,"num_write_kb":522880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.d","version":"73'79","reported_seq":703,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.459549+0000","last_change":"2024-09-16T09:39:49.055357+0000","last_active":"2024-09-16T09:42:49.459549+0000","last_peered":"2024-09-16T09:42:49.459549+0000","last_clean":"2024-09-16T09:42:49.459549+0000","last_became_active":"2024-09-16T09:30:10.709958+0000","last_became_peered":"2024-09-16T09:30:10.709958+0000","last_unstale":"2024-09-16T09:42:49.459549+0000","last_undegraded":"2024-09-16T09:42:49.459549+0000","last_fullsized":"2024-09-16T09:42:49.459549+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:39:49.055292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:39:49.055292+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:00.227133+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"285'27","reported_seq":955,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.448407+0000","last_change":"2024-09-16T09:42:45.839190+0000","last_active":"2024-09-16T09:42:49.448407+0000","last_peered":"2024-09-16T09:42:49.448407+0000","last_clean":"2024-09-16T09:42:49.448407+0000","last_became_active":"2024-09-16T09:25:03.715733+0000","last_became_peered":"2024-09-16T09:25:03.715733+0000","last_unstale":"2024-09-16T09:42:49.448407+0000","last_undegraded":"2024-09-16T09:42:49.448407+0000","last_fullsized":"2024-09-16T09:42:49.448407+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'27","last_scrub_stamp":"2024-09-16T09:42:45.839109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:42:45.839109+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:44:08.412046+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":925,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.460758+0000","last_change":"2024-09-16T09:42:18.192222+0000","last_active":"2024-09-16T09:42:49.460758+0000","last_peered":"2024-09-16T09:42:49.460758+0000","last_clean":"2024-09-16T09:42:49.460758+0000","last_became_active":"2024-09-16T09:25:01.707372+0000","last_became_peered":"2024-09-16T09:25:01.707372+0000","last_unstale":"2024-09-16T09:42:49.460758+0000","last_undegraded":"2024-09-16T09:42:49.460758+0000","last_fullsized":"2024-09-16T09:42:49.460758+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-16T09:42:18.192142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:42:18.192142+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:37.294343+0000","scrub_duration":32,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"66'8","reported_seq":1145,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:51.690486+0000","last_change":"2024-09-16T09:42:51.690430+0000","last_active":"2024-09-16T09:42:51.690486+0000","last_peered":"2024-09-16T09:42:51.690486+0000","last_clean":"2024-09-16T09:42:51.690486+0000","last_became_active":"2024-09-16T09:25:05.716189+0000","last_became_peered":"2024-09-16T09:25:05.716189+0000","last_unstale":"2024-09-16T09:42:51.690486+0000","last_undegraded":"2024-09-16T09:42:51.690486+0000","last_fullsized":"2024-09-16T09:42:51.690486+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:42:51.690361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:42:51.690361+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:44:14.524408+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":896,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.448317+0000","last_change":"2024-09-16T09:42:28.703087+0000","last_active":"2024-09-16T09:42:49.448317+0000","last_peered":"2024-09-16T09:42:49.448317+0000","last_clean":"2024-09-16T09:42:49.448317+0000","last_became_active":"2024-09-16T09:25:07.717213+0000","last_became_peered":"2024-09-16T09:25:07.717213+0000","last_unstale":"2024-09-16T09:42:49.448317+0000","last_undegraded":"2024-09-16T09:42:49.448317+0000","last_fullsized":"2024-09-16T09:42:49.448317+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:42:28.702918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:42:28.702918+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:40.078117+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.12","version":"73'79","reported_seq":721,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.448543+0000","last_change":"2024-09-16T09:42:16.729102+0000","last_active":"2024-09-16T09:42:49.448543+0000","last_peered":"2024-09-16T09:42:49.448543+0000","last_clean":"2024-09-16T09:42:49.448543+0000","last_became_active":"2024-09-16T09:30:10.731413+0000","last_became_peered":"2024-09-16T09:30:10.731413+0000","last_unstale":"2024-09-16T09:42:49.448543+0000","last_undegraded":"2024-09-16T09:42:49.448543+0000","last_fullsized":"2024-09-16T09:42:49.448543+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:42:16.728929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:42:16.728929+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:36.863523+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":922,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.467006+0000","last_change":"2024-09-16T09:37:55.863661+0000","last_active":"2024-09-16T09:42:49.467006+0000","last_peered":"2024-09-16T09:42:49.467006+0000","last_clean":"2024-09-16T09:42:49.467006+0000","last_became_active":"2024-09-16T09:37:53.518354+0000","last_became_peered":"2024-09-16T09:37:53.518354+0000","last_unstale":"2024-09-16T09:42:49.467006+0000","last_undegraded":"2024-09-16T09:42:49.467006+0000","last_fullsized":"2024-09-16T09:42:49.467006+0000","mapping_epoch":382,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":383,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:37:55.863613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:37:55.863613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:58.876996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"285'12","reported_seq":689,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:52.958906+0000","last_change":"2024-09-16T09:42:52.958906+0000","last_active":"2024-09-16T09:42:52.958906+0000","last_peered":"2024-09-16T09:42:52.958906+0000","last_clean":"2024-09-16T09:42:52.958906+0000","last_became_active":"2024-09-16T09:38:33.517889+0000","last_became_peered":"2024-09-16T09:38:33.517889+0000","last_unstale":"2024-09-16T09:42:52.958906+0000","last_undegraded":"2024-09-16T09:42:52.958906+0000","last_fullsized":"2024-09-16T09:42:52.958906+0000","mapping_epoch":393,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":394,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'12","last_scrub_stamp":"2024-09-16T09:42:52.958841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:42:52.958841+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:44:00.216862+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":17,"num_read_kb":14,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":907,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.449401+0000","last_change":"2024-09-16T09:40:28.258952+0000","last_active":"2024-09-16T09:42:49.449401+0000","last_peered":"2024-09-16T09:42:49.449401+0000","last_clean":"2024-09-16T09:42:49.449401+0000","last_became_active":"2024-09-16T09:25:05.715616+0000","last_became_peered":"2024-09-16T09:25:05.715616+0000","last_unstale":"2024-09-16T09:42:49.449401+0000","last_undegraded":"2024-09-16T09:42:49.449401+0000","last_fullsized":"2024-09-16T09:42:49.449401+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:40:28.258907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:40:28.258907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:03.671149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":901,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.459645+0000","last_change":"2024-09-16T09:40:37.040258+0000","last_active":"2024-09-16T09:42:49.459645+0000","last_peered":"2024-09-16T09:42:49.459645+0000","last_clean":"2024-09-16T09:42:49.459645+0000","last_became_active":"2024-09-16T09:25:07.721059+0000","last_became_peered":"2024-09-16T09:25:07.721059+0000","last_unstale":"2024-09-16T09:42:49.459645+0000","last_undegraded":"2024-09-16T09:42:49.459645+0000","last_fullsized":"2024-09-16T09:42:49.459645+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:40:37.040205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:40:37.040205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:43.460971+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.13","version":"73'79","reported_seq":799,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.463997+0000","last_change":"2024-09-16T09:40:30.696059+0000","last_active":"2024-09-16T09:42:49.463997+0000","last_peered":"2024-09-16T09:42:49.463997+0000","last_clean":"2024-09-16T09:42:49.463997+0000","last_became_active":"2024-09-16T09:33:30.248497+0000","last_became_peered":"2024-09-16T09:33:30.248497+0000","last_unstale":"2024-09-16T09:42:49.463997+0000","last_undegraded":"2024-09-16T09:42:49.463997+0000","last_fullsized":"2024-09-16T09:42:49.463997+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:40:30.695866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:40:30.695866+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:54.647315+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"285'15","reported_seq":987,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.469228+0000","last_change":"2024-09-16T09:39:46.504126+0000","last_active":"2024-09-16T09:42:49.469228+0000","last_peered":"2024-09-16T09:42:49.469228+0000","last_clean":"2024-09-16T09:42:49.469228+0000","last_became_active":"2024-09-16T09:31:09.273498+0000","last_became_peered":"2024-09-16T09:31:09.273498+0000","last_unstale":"2024-09-16T09:42:49.469228+0000","last_undegraded":"2024-09-16T09:42:49.469228+0000","last_fullsized":"2024-09-16T09:42:49.469228+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'15","last_scrub_stamp":"2024-09-16T09:39:46.504045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:39:46.504045+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:52.043358+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":921,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.459213+0000","last_change":"2024-09-16T09:41:36.151500+0000","last_active":"2024-09-16T09:42:49.459213+0000","last_peered":"2024-09-16T09:42:49.459213+0000","last_clean":"2024-09-16T09:42:49.459213+0000","last_became_active":"2024-09-16T09:25:01.706491+0000","last_became_peered":"2024-09-16T09:25:01.706491+0000","last_unstale":"2024-09-16T09:42:49.459213+0000","last_undegraded":"2024-09-16T09:42:49.459213+0000","last_fullsized":"2024-09-16T09:42:49.459213+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:36.151432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:41:36.151432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:53.930320+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":686,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.460355+0000","last_change":"2024-09-16T09:42:08.180799+0000","last_active":"2024-09-16T09:42:49.460355+0000","last_peered":"2024-09-16T09:42:49.460355+0000","last_clean":"2024-09-16T09:42:49.460355+0000","last_became_active":"2024-09-16T09:33:30.248992+0000","last_became_peered":"2024-09-16T09:33:30.248992+0000","last_unstale":"2024-09-16T09:42:49.460355+0000","last_undegraded":"2024-09-16T09:42:49.460355+0000","last_fullsized":"2024-09-16T09:42:49.460355+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:42:08.180735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:42:08.180735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:09.054380+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":913,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.455810+0000","last_change":"2024-09-16T09:40:35.646704+0000","last_active":"2024-09-16T09:42:49.455810+0000","last_peered":"2024-09-16T09:42:49.455810+0000","last_clean":"2024-09-16T09:42:49.455810+0000","last_became_active":"2024-09-16T09:25:07.722050+0000","last_became_peered":"2024-09-16T09:25:07.722050+0000","last_unstale":"2024-09-16T09:42:49.455810+0000","last_undegraded":"2024-09-16T09:42:49.455810+0000","last_fullsized":"2024-09-16T09:42:49.455810+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:40:35.646535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:40:35.646535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:53.543871+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.10","version":"73'79","reported_seq":713,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.448245+0000","last_change":"2024-09-16T09:41:08.828451+0000","last_active":"2024-09-16T09:42:49.448245+0000","last_peered":"2024-09-16T09:42:49.448245+0000","last_clean":"2024-09-16T09:42:49.448245+0000","last_became_active":"2024-09-16T09:30:10.692770+0000","last_became_peered":"2024-09-16T09:30:10.692770+0000","last_unstale":"2024-09-16T09:42:49.448245+0000","last_undegraded":"2024-09-16T09:42:49.448245+0000","last_fullsized":"2024-09-16T09:42:49.448245+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:41:08.828394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:41:08.828394+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:38.833397+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"285'15","reported_seq":936,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.450459+0000","last_change":"2024-09-16T09:41:40.700560+0000","last_active":"2024-09-16T09:42:49.450459+0000","last_peered":"2024-09-16T09:42:49.450459+0000","last_clean":"2024-09-16T09:42:49.450459+0000","last_became_active":"2024-09-16T09:25:03.717310+0000","last_became_peered":"2024-09-16T09:25:03.717310+0000","last_unstale":"2024-09-16T09:42:49.450459+0000","last_undegraded":"2024-09-16T09:42:49.450459+0000","last_fullsized":"2024-09-16T09:42:49.450459+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'15","last_scrub_stamp":"2024-09-16T09:41:40.700318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:41:40.700318+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":946,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.462897+0000","last_change":"2024-09-16T09:40:23.065431+0000","last_active":"2024-09-16T09:42:49.462897+0000","last_peered":"2024-09-16T09:42:49.462897+0000","last_clean":"2024-09-16T09:42:49.462897+0000","last_became_active":"2024-09-16T09:31:09.272848+0000","last_became_peered":"2024-09-16T09:31:09.272848+0000","last_unstale":"2024-09-16T09:42:49.462897+0000","last_undegraded":"2024-09-16T09:42:49.462897+0000","last_fullsized":"2024-09-16T09:42:49.462897+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:40:23.065365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:40:23.065365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:33.883685+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"187'11","reported_seq":973,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.459725+0000","last_change":"2024-09-16T09:42:19.191400+0000","last_active":"2024-09-16T09:42:49.459725+0000","last_peered":"2024-09-16T09:42:49.459725+0000","last_clean":"2024-09-16T09:42:49.459725+0000","last_became_active":"2024-09-16T09:31:09.277688+0000","last_became_peered":"2024-09-16T09:31:09.277688+0000","last_unstale":"2024-09-16T09:42:49.459725+0000","last_undegraded":"2024-09-16T09:42:49.459725+0000","last_fullsized":"2024-09-16T09:42:49.459725+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"187'11","last_scrub_stamp":"2024-09-16T09:42:19.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:42:19.191311+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:24.981070+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":928,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.451456+0000","last_change":"2024-09-16T09:41:36.098715+0000","last_active":"2024-09-16T09:42:49.451456+0000","last_peered":"2024-09-16T09:42:49.451456+0000","last_clean":"2024-09-16T09:42:49.451456+0000","last_became_active":"2024-09-16T09:25:07.718810+0000","last_became_peered":"2024-09-16T09:25:07.718810+0000","last_unstale":"2024-09-16T09:42:49.451456+0000","last_undegraded":"2024-09-16T09:42:49.451456+0000","last_fullsized":"2024-09-16T09:42:49.451456+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:36.098648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:41:36.098648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.11","version":"73'79","reported_seq":799,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.463224+0000","last_change":"2024-09-16T09:42:16.839630+0000","last_active":"2024-09-16T09:42:49.463224+0000","last_peered":"2024-09-16T09:42:49.463224+0000","last_clean":"2024-09-16T09:42:49.463224+0000","last_became_active":"2024-09-16T09:31:09.272208+0000","last_became_peered":"2024-09-16T09:31:09.272208+0000","last_unstale":"2024-09-16T09:42:49.463224+0000","last_undegraded":"2024-09-16T09:42:49.463224+0000","last_fullsized":"2024-09-16T09:42:49.463224+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:42:16.839458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:42:16.839458+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:17.207711+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.14","version":"285'20","reported_seq":603,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.459769+0000","last_change":"2024-09-16T09:40:35.027221+0000","last_active":"2024-09-16T09:42:49.459769+0000","last_peered":"2024-09-16T09:42:49.459769+0000","last_clean":"2024-09-16T09:42:49.459769+0000","last_became_active":"2024-09-16T09:33:30.249769+0000","last_became_peered":"2024-09-16T09:33:30.249769+0000","last_unstale":"2024-09-16T09:42:49.459769+0000","last_undegraded":"2024-09-16T09:42:49.459769+0000","last_fullsized":"2024-09-16T09:42:49.459769+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'20","last_scrub_stamp":"2024-09-16T09:40:35.027096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:40:35.027096+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:04.153892+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":913,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.460492+0000","last_change":"2024-09-16T09:41:08.005775+0000","last_active":"2024-09-16T09:42:49.460492+0000","last_peered":"2024-09-16T09:42:49.460492+0000","last_clean":"2024-09-16T09:42:49.460492+0000","last_became_active":"2024-09-16T09:25:01.707513+0000","last_became_peered":"2024-09-16T09:25:01.707513+0000","last_unstale":"2024-09-16T09:42:49.460492+0000","last_undegraded":"2024-09-16T09:42:49.460492+0000","last_fullsized":"2024-09-16T09:42:49.460492+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:08.005728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:41:08.005728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"66'8","reported_seq":1146,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.452342+0000","last_change":"2024-09-16T09:41:42.773433+0000","last_active":"2024-09-16T09:42:49.452342+0000","last_peered":"2024-09-16T09:42:49.452342+0000","last_clean":"2024-09-16T09:42:49.452342+0000","last_became_active":"2024-09-16T09:25:05.713833+0000","last_became_peered":"2024-09-16T09:25:05.713833+0000","last_unstale":"2024-09-16T09:42:49.452342+0000","last_undegraded":"2024-09-16T09:42:49.452342+0000","last_fullsized":"2024-09-16T09:42:49.452342+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:41:42.773359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:41:42.773359+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":942,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.463127+0000","last_change":"2024-09-16T09:42:32.913873+0000","last_active":"2024-09-16T09:42:49.463127+0000","last_peered":"2024-09-16T09:42:49.463127+0000","last_clean":"2024-09-16T09:42:49.463127+0000","last_became_active":"2024-09-16T09:31:09.275168+0000","last_became_peered":"2024-09-16T09:31:09.275168+0000","last_unstale":"2024-09-16T09:42:49.463127+0000","last_undegraded":"2024-09-16T09:42:49.463127+0000","last_fullsized":"2024-09-16T09:42:49.463127+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:42:32.913795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:42:32.913795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:41.837733+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"285'19","reported_seq":954,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.450271+0000","last_change":"2024-09-16T09:42:20.151407+0000","last_active":"2024-09-16T09:42:49.450271+0000","last_peered":"2024-09-16T09:42:49.450271+0000","last_clean":"2024-09-16T09:42:49.450271+0000","last_became_active":"2024-09-16T09:26:33.291265+0000","last_became_peered":"2024-09-16T09:26:33.291265+0000","last_unstale":"2024-09-16T09:42:49.450271+0000","last_undegraded":"2024-09-16T09:42:49.450271+0000","last_fullsized":"2024-09-16T09:42:49.450271+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'19","last_scrub_stamp":"2024-09-16T09:42:20.151313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:42:20.151313+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:46.045990+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":932,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.451376+0000","last_change":"2024-09-16T09:41:29.092238+0000","last_active":"2024-09-16T09:42:49.451376+0000","last_peered":"2024-09-16T09:42:49.451376+0000","last_clean":"2024-09-16T09:42:49.451376+0000","last_became_active":"2024-09-16T09:25:01.705158+0000","last_became_peered":"2024-09-16T09:25:01.705158+0000","last_unstale":"2024-09-16T09:42:49.451376+0000","last_undegraded":"2024-09-16T09:42:49.451376+0000","last_fullsized":"2024-09-16T09:42:49.451376+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:29.091987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:41:29.091987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:56.880823+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":852,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.448550+0000","last_change":"2024-09-16T09:41:37.694845+0000","last_active":"2024-09-16T09:42:49.448550+0000","last_peered":"2024-09-16T09:42:49.448550+0000","last_clean":"2024-09-16T09:42:49.448550+0000","last_became_active":"2024-09-16T09:37:41.515278+0000","last_became_peered":"2024-09-16T09:37:41.515278+0000","last_unstale":"2024-09-16T09:42:49.448550+0000","last_undegraded":"2024-09-16T09:42:49.448550+0000","last_fullsized":"2024-09-16T09:42:49.448550+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:37.694724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:41:37.694724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":758,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.463101+0000","last_change":"2024-09-16T09:41:05.972422+0000","last_active":"2024-09-16T09:42:49.463101+0000","last_peered":"2024-09-16T09:42:49.463101+0000","last_clean":"2024-09-16T09:42:49.463101+0000","last_became_active":"2024-09-16T09:31:09.276042+0000","last_became_peered":"2024-09-16T09:31:09.276042+0000","last_unstale":"2024-09-16T09:42:49.463101+0000","last_undegraded":"2024-09-16T09:42:49.463101+0000","last_fullsized":"2024-09-16T09:42:49.463101+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:05.972363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:41:05.972363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:37.856067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"285'15","reported_seq":635,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.458080+0000","last_change":"2024-09-16T09:41:09.949578+0000","last_active":"2024-09-16T09:42:49.458080+0000","last_peered":"2024-09-16T09:42:49.458080+0000","last_clean":"2024-09-16T09:42:49.458080+0000","last_became_active":"2024-09-16T09:37:33.348223+0000","last_became_peered":"2024-09-16T09:37:33.348223+0000","last_unstale":"2024-09-16T09:42:49.458080+0000","last_undegraded":"2024-09-16T09:42:49.458080+0000","last_fullsized":"2024-09-16T09:42:49.458080+0000","mapping_epoch":367,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":368,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'15","last_scrub_stamp":"2024-09-16T09:41:09.949519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:41:09.949519+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,2],"acting":[6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":913,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.459951+0000","last_change":"2024-09-16T09:41:44.170251+0000","last_active":"2024-09-16T09:42:49.459951+0000","last_peered":"2024-09-16T09:42:49.459951+0000","last_clean":"2024-09-16T09:42:49.459951+0000","last_became_active":"2024-09-16T09:25:01.706762+0000","last_became_peered":"2024-09-16T09:25:01.706762+0000","last_unstale":"2024-09-16T09:42:49.459951+0000","last_undegraded":"2024-09-16T09:42:49.459951+0000","last_fullsized":"2024-09-16T09:42:49.459951+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:44.170186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:41:44.170186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:51.467157+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":751,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.461339+0000","last_change":"2024-09-16T09:41:36.130305+0000","last_active":"2024-09-16T09:42:49.461339+0000","last_peered":"2024-09-16T09:42:49.461339+0000","last_clean":"2024-09-16T09:42:49.461339+0000","last_became_active":"2024-09-16T09:28:52.982865+0000","last_became_peered":"2024-09-16T09:28:52.982865+0000","last_unstale":"2024-09-16T09:42:49.461339+0000","last_undegraded":"2024-09-16T09:42:49.461339+0000","last_fullsized":"2024-09-16T09:42:49.461339+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:36.130240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:41:36.130240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:05.802731+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":896,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.460407+0000","last_change":"2024-09-16T09:34:50.779241+0000","last_active":"2024-09-16T09:42:49.460407+0000","last_peered":"2024-09-16T09:42:49.460407+0000","last_clean":"2024-09-16T09:42:49.460407+0000","last_became_active":"2024-09-16T09:25:07.715301+0000","last_became_peered":"2024-09-16T09:25:07.715301+0000","last_unstale":"2024-09-16T09:42:49.460407+0000","last_undegraded":"2024-09-16T09:42:49.460407+0000","last_fullsized":"2024-09-16T09:42:49.460407+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:50.779164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:34:50.779164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:04.743563+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.14","version":"73'79","reported_seq":722,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.455715+0000","last_change":"2024-09-16T09:41:38.531852+0000","last_active":"2024-09-16T09:42:49.455715+0000","last_peered":"2024-09-16T09:42:49.455715+0000","last_clean":"2024-09-16T09:42:49.455715+0000","last_became_active":"2024-09-16T09:30:10.700866+0000","last_became_peered":"2024-09-16T09:30:10.700866+0000","last_unstale":"2024-09-16T09:42:49.455715+0000","last_undegraded":"2024-09-16T09:42:49.455715+0000","last_fullsized":"2024-09-16T09:42:49.455715+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:41:38.531780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:41:38.531780+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:55.556470+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"285'19","reported_seq":793,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.459131+0000","last_change":"2024-09-16T09:39:28.862654+0000","last_active":"2024-09-16T09:42:49.459131+0000","last_peered":"2024-09-16T09:42:49.459131+0000","last_clean":"2024-09-16T09:42:49.459131+0000","last_became_active":"2024-09-16T09:28:52.971132+0000","last_became_peered":"2024-09-16T09:28:52.971132+0000","last_unstale":"2024-09-16T09:42:49.459131+0000","last_undegraded":"2024-09-16T09:42:49.459131+0000","last_fullsized":"2024-09-16T09:42:49.459131+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'19","last_scrub_stamp":"2024-09-16T09:39:28.862503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:39:28.862503+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:57.153062+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":923,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.449525+0000","last_change":"2024-09-16T09:41:47.768655+0000","last_active":"2024-09-16T09:42:49.449525+0000","last_peered":"2024-09-16T09:42:49.449525+0000","last_clean":"2024-09-16T09:42:49.449525+0000","last_became_active":"2024-09-16T09:25:01.700733+0000","last_became_peered":"2024-09-16T09:25:01.700733+0000","last_unstale":"2024-09-16T09:42:49.449525+0000","last_undegraded":"2024-09-16T09:42:49.449525+0000","last_fullsized":"2024-09-16T09:42:49.449525+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:47.768574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:41:47.768574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:13.312512+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":913,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.458491+0000","last_change":"2024-09-16T09:40:39.969447+0000","last_active":"2024-09-16T09:42:49.458491+0000","last_peered":"2024-09-16T09:42:49.458491+0000","last_clean":"2024-09-16T09:42:49.458491+0000","last_became_active":"2024-09-16T09:25:05.713479+0000","last_became_peered":"2024-09-16T09:25:05.713479+0000","last_unstale":"2024-09-16T09:42:49.458491+0000","last_undegraded":"2024-09-16T09:42:49.458491+0000","last_fullsized":"2024-09-16T09:42:49.458491+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:40:39.969383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:40:39.969383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:38.327940+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":774,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.461724+0000","last_change":"2024-09-16T09:41:34.178100+0000","last_active":"2024-09-16T09:42:49.461724+0000","last_peered":"2024-09-16T09:42:49.461724+0000","last_clean":"2024-09-16T09:42:49.461724+0000","last_became_active":"2024-09-16T09:31:09.274797+0000","last_became_peered":"2024-09-16T09:31:09.274797+0000","last_unstale":"2024-09-16T09:42:49.461724+0000","last_undegraded":"2024-09-16T09:42:49.461724+0000","last_fullsized":"2024-09-16T09:42:49.461724+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:34.178020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:41:34.178020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:55.711514+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"285'8","reported_seq":686,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.462353+0000","last_change":"2024-09-16T09:42:26.954507+0000","last_active":"2024-09-16T09:42:49.462353+0000","last_peered":"2024-09-16T09:42:49.462353+0000","last_clean":"2024-09-16T09:42:49.462353+0000","last_became_active":"2024-09-16T09:31:09.273682+0000","last_became_peered":"2024-09-16T09:31:09.273682+0000","last_unstale":"2024-09-16T09:42:49.462353+0000","last_undegraded":"2024-09-16T09:42:49.462353+0000","last_fullsized":"2024-09-16T09:42:49.462353+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'8","last_scrub_stamp":"2024-09-16T09:42:26.954218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:42:26.954218+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:43.105517+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":9,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":946,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.468984+0000","last_change":"2024-09-16T09:41:45.045179+0000","last_active":"2024-09-16T09:42:49.468984+0000","last_peered":"2024-09-16T09:42:49.468984+0000","last_clean":"2024-09-16T09:42:49.468984+0000","last_became_active":"2024-09-16T09:31:09.274719+0000","last_became_peered":"2024-09-16T09:31:09.274719+0000","last_unstale":"2024-09-16T09:42:49.468984+0000","last_undegraded":"2024-09-16T09:42:49.468984+0000","last_fullsized":"2024-09-16T09:42:49.468984+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:45.045117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:41:45.045117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:56.298444+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":924,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.459079+0000","last_change":"2024-09-16T09:38:35.092025+0000","last_active":"2024-09-16T09:42:49.459079+0000","last_peered":"2024-09-16T09:42:49.459079+0000","last_clean":"2024-09-16T09:42:49.459079+0000","last_became_active":"2024-09-16T09:25:05.712545+0000","last_became_peered":"2024-09-16T09:25:05.712545+0000","last_unstale":"2024-09-16T09:42:49.459079+0000","last_undegraded":"2024-09-16T09:42:49.459079+0000","last_fullsized":"2024-09-16T09:42:49.459079+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:38:35.091983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:38:35.091983+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:41.632288+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":909,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.458718+0000","last_change":"2024-09-16T09:41:29.261932+0000","last_active":"2024-09-16T09:42:49.458718+0000","last_peered":"2024-09-16T09:42:49.458718+0000","last_clean":"2024-09-16T09:42:49.458718+0000","last_became_active":"2024-09-16T09:25:07.720117+0000","last_became_peered":"2024-09-16T09:25:07.720117+0000","last_unstale":"2024-09-16T09:42:49.458718+0000","last_undegraded":"2024-09-16T09:42:49.458718+0000","last_fullsized":"2024-09-16T09:42:49.458718+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:29.261841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:41:29.261841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:57.269436+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":539,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.450311+0000","last_change":"2024-09-16T09:36:52.091335+0000","last_active":"2024-09-16T09:42:49.450311+0000","last_peered":"2024-09-16T09:42:49.450311+0000","last_clean":"2024-09-16T09:42:49.450311+0000","last_became_active":"2024-09-16T09:33:30.248036+0000","last_became_peered":"2024-09-16T09:33:30.248036+0000","last_unstale":"2024-09-16T09:42:49.450311+0000","last_undegraded":"2024-09-16T09:42:49.450311+0000","last_fullsized":"2024-09-16T09:42:49.450311+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:36:52.088284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:36:52.088284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:53.777831+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":773,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.460891+0000","last_change":"2024-09-16T09:42:30.898163+0000","last_active":"2024-09-16T09:42:49.460891+0000","last_peered":"2024-09-16T09:42:49.460891+0000","last_clean":"2024-09-16T09:42:49.460891+0000","last_became_active":"2024-09-16T09:31:09.271662+0000","last_became_peered":"2024-09-16T09:31:09.271662+0000","last_unstale":"2024-09-16T09:42:49.460891+0000","last_undegraded":"2024-09-16T09:42:49.460891+0000","last_fullsized":"2024-09-16T09:42:49.460891+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:42:30.898094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:42:30.898094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:43:46.282325+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"285'19","reported_seq":944,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.459169+0000","last_change":"2024-09-16T09:37:25.995663+0000","last_active":"2024-09-16T09:42:49.459169+0000","last_peered":"2024-09-16T09:42:49.459169+0000","last_clean":"2024-09-16T09:42:49.459169+0000","last_became_active":"2024-09-16T09:25:03.707769+0000","last_became_peered":"2024-09-16T09:25:03.707769+0000","last_unstale":"2024-09-16T09:42:49.459169+0000","last_undegraded":"2024-09-16T09:42:49.459169+0000","last_fullsized":"2024-09-16T09:42:49.459169+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'19","last_scrub_stamp":"2024-09-16T09:37:25.995607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:37:25.995607+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:55.934178+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":909,"reported_epoch":492,"state":"active+clean","last_fresh":"2024-09-16T09:42:49.463281+0000","last_change":"2024-09-16T09:41:29.136632+0000","last_active":"2024-09-16T09:42:49.463281+0000","last_peered":"2024-09-16T09:42:49.463281+0000","last_clean":"2024-09-16T09:42:49.463281+0000","last_became_active":"2024-09-16T09:25:05.711952+0000","last_became_peered":"2024-09-16T09:25:05.711952+0000","last_unstale":"2024-09-16T09:42:49.463281+0000","last_undegraded":"2024-09-16T09:42:49.463281+0000","last_fullsized":"2024-09-16T09:42:49.463281+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:29.136452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:41:29.136452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:42:51.188109+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":16,"stat_sum":{"num_bytes":8356954136,"num_objects":127518,"num_object_clones":0,"num_object_copies":382554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":127518,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":133374,"num_write_kb":8348481,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":25053966336,"data_stored":25053954120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75460,"ondisk_log_size":75460,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":1117,"num_read_kb":874,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":3686,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":681,"ondisk_log_size":681,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1986,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1064,"num_read_kb":1059,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":21,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":327680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1659,"ondisk_log_size":1659,"up":63,"acting":63,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266907,"num_pgs":69,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3841528,"kb_used_data":3102312,"kb_used_omap":1,"kb_used_meta":739198,"kb_avail":89903624,"statfs":{"total":95995035648,"available":92061310976,"internally_reserved":0,"allocated":3176767488,"data_stored":3176180213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":756939210},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495840,"num_pgs":61,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":4754144,"kb_used_data":4087120,"kb_used_omap":1,"kb_used_meta":667006,"kb_avail":88991008,"statfs":{"total":95995035648,"available":91126792192,"internally_reserved":0,"allocated":4185210880,"data_stored":4184646997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":683014598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724775,"num_pgs":62,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2841200,"kb_used_data":2047840,"kb_used_omap":1,"kb_used_meta":793342,"kb_avail":90903952,"statfs":{"total":95995035648,"available":93085646848,"internally_reserved":0,"allocated":2096988160,"data_stored":2096409102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":812382662},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953708,"num_pgs":69,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2742588,"kb_used_data":2053356,"kb_used_omap":1,"kb_used_meta":689214,"kb_avail":91002564,"statfs":{"total":95995035648,"available":93186625536,"internally_reserved":0,"allocated":2102636544,"data_stored":2102045151,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":705755590},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149938,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":4427456,"kb_used_data":3557744,"kb_used_omap":1,"kb_used_meta":869694,"kb_avail":89317696,"statfs":{"total":95995035648,"available":91461320704,"internally_reserved":0,"allocated":3643129856,"data_stored":3642535063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":890567111},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411575,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3753948,"kb_used_data":3073036,"kb_used_omap":1,"kb_used_meta":680894,"kb_avail":89991204,"statfs":{"total":95995035648,"available":92150992896,"internally_reserved":0,"allocated":3146788864,"data_stored":3146229333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":697235909},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575102,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":5035476,"kb_used_data":4071172,"kb_used_omap":1,"kb_used_meta":964094,"kb_avail":88709676,"statfs":{"total":95995035648,"available":90838708224,"internally_reserved":0,"allocated":4168880128,"data_stored":4168264115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":987232709},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738627,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3412088,"kb_used_data":2590948,"kb_used_omap":1,"kb_used_meta":821118,"kb_avail":90333064,"statfs":{"total":95995035648,"available":92501057536,"internally_reserved":0,"allocated":2653130752,"data_stored":2652547619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":840825285},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":1788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2652176384,"data_stored":2652176384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4153671680,"data_stored":4153671680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3127709696,"data_stored":3127705624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":3612409856,"data_stored":3612409856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":2098991104,"data_stored":2098987032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":2084896768,"data_stored":2084896768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4175892480,"data_stored":4175888408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":3148218368,"data_stored":3148218368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T09:42:55.237 INFO:tasks.cephadm.ceph_manager.ceph:would drop below min_pgs, proj 6, currently 16 2024-09-16T09:42:55.289 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:42:55.307 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:42:55.328 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:55.328 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:55.329 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:42:55.430 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:42:55.458 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:55.458 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:55.459 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:42:55.559 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:42:55.589 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:55.590 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:55.590 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:42:55.639 INFO:teuthology.orchestra.run.smithi070.stderr:nodeep-scrub is unset 2024-09-16T09:42:55.690 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:42:55.724 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:55.725 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:55.725 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:42:55.826 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:42:55.830 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:55 smithi070 ceph-mon[30388]: 4.16 scrub starts 2024-09-16T09:42:55.831 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:55 smithi070 ceph-mon[30388]: 4.16 scrub ok 2024-09-16T09:42:55.831 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:55 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2416623040' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:42:55.831 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:55 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:42:55.831 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:55 smithi070 ceph-mon[40691]: 4.16 scrub starts 2024-09-16T09:42:55.831 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:55 smithi070 ceph-mon[40691]: 4.16 scrub ok 2024-09-16T09:42:55.831 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:55 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/2416623040' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:42:55.831 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:55 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:42:55.861 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:55.861 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:55.862 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:42:55.897 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:42:55.897 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:42:55.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:55 smithi084 ceph-mon[32629]: 4.16 scrub starts 2024-09-16T09:42:55.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:55 smithi084 ceph-mon[32629]: 4.16 scrub ok 2024-09-16T09:42:55.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:55 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/2416623040' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:42:55.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:55 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:42:55.932 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:42:55.933 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:42:55.962 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:42:55.969 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:42:55.969 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:42:55.999 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:55.999 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:56.000 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:42:56.010 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:42:56.010 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:42:56.043 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:42:56.043 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:42:56.076 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:42:56.076 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:42:56.100 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:42:56.111 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:42:56.111 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:42:56.137 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:56.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:56.138 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:42:56.145 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:42:56.145 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:42:56.239 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:42:56.267 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:56.267 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:56.268 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:42:56.368 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:42:56.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:56.399 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:56.399 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:42:56.500 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:42:56.533 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:56.533 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:56.534 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:42:56.634 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:42:56.664 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:56.664 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:56.665 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:42:56.766 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:42:56.805 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:56.806 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:56.806 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:42:56.907 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:42:56.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:56 smithi070 ceph-mon[30388]: 1.13 scrub starts 2024-09-16T09:42:56.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:56 smithi070 ceph-mon[30388]: 1.13 scrub ok 2024-09-16T09:42:56.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:56 smithi070 ceph-mon[30388]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-16T09:42:56.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:56 smithi070 ceph-mon[30388]: Cluster is now healthy 2024-09-16T09:42:56.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:56 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-16T09:42:56.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:56 smithi070 ceph-mon[30388]: osdmap e493: 8 total, 8 up, 8 in 2024-09-16T09:42:56.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:56 smithi070 ceph-mon[30388]: 6.14 scrub starts 2024-09-16T09:42:56.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:56 smithi070 ceph-mon[30388]: 6.14 scrub ok 2024-09-16T09:42:56.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:56 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/740981024' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:42:56.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:56 smithi070 ceph-mon[30388]: pgmap v945: 168 pgs: 168 active+clean; 7.8 GiB data, 29 GiB used, 686 GiB / 715 GiB avail; 767 B/s rd, 46 MiB/s wr, 1.15k op/s 2024-09-16T09:42:56.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:56 smithi070 ceph-mon[30388]: 6.6 scrub starts 2024-09-16T09:42:56.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:56 smithi070 ceph-mon[30388]: 6.6 scrub ok 2024-09-16T09:42:56.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:56 smithi070 ceph-mon[30388]: 5.a scrub starts 2024-09-16T09:42:56.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:56 smithi070 ceph-mon[30388]: 5.a scrub ok 2024-09-16T09:42:56.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:56 smithi070 ceph-mon[40691]: 1.13 scrub starts 2024-09-16T09:42:56.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:56 smithi070 ceph-mon[40691]: 1.13 scrub ok 2024-09-16T09:42:56.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:56 smithi070 ceph-mon[40691]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-16T09:42:56.923 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:56 smithi070 ceph-mon[40691]: Cluster is now healthy 2024-09-16T09:42:56.923 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:56 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-16T09:42:56.923 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:56 smithi070 ceph-mon[40691]: osdmap e493: 8 total, 8 up, 8 in 2024-09-16T09:42:56.923 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:56 smithi070 ceph-mon[40691]: 6.14 scrub starts 2024-09-16T09:42:56.923 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:56 smithi070 ceph-mon[40691]: 6.14 scrub ok 2024-09-16T09:42:56.923 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:56 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/740981024' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:42:56.923 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:56 smithi070 ceph-mon[40691]: pgmap v945: 168 pgs: 168 active+clean; 7.8 GiB data, 29 GiB used, 686 GiB / 715 GiB avail; 767 B/s rd, 46 MiB/s wr, 1.15k op/s 2024-09-16T09:42:56.923 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:56 smithi070 ceph-mon[40691]: 6.6 scrub starts 2024-09-16T09:42:56.923 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:56 smithi070 ceph-mon[40691]: 6.6 scrub ok 2024-09-16T09:42:56.923 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:56 smithi070 ceph-mon[40691]: 5.a scrub starts 2024-09-16T09:42:56.923 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:56 smithi070 ceph-mon[40691]: 5.a scrub ok 2024-09-16T09:42:56.941 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:56.941 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:56.942 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:42:56.987 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:56 smithi084 ceph-mon[32629]: 1.13 scrub starts 2024-09-16T09:42:56.988 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:56 smithi084 ceph-mon[32629]: 1.13 scrub ok 2024-09-16T09:42:56.988 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:56 smithi084 ceph-mon[32629]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-16T09:42:56.988 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:56 smithi084 ceph-mon[32629]: Cluster is now healthy 2024-09-16T09:42:56.988 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:56 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-16T09:42:56.988 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:56 smithi084 ceph-mon[32629]: osdmap e493: 8 total, 8 up, 8 in 2024-09-16T09:42:56.988 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:56 smithi084 ceph-mon[32629]: 6.14 scrub starts 2024-09-16T09:42:56.988 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:56 smithi084 ceph-mon[32629]: 6.14 scrub ok 2024-09-16T09:42:56.988 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:56 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/740981024' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:42:56.988 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:56 smithi084 ceph-mon[32629]: pgmap v945: 168 pgs: 168 active+clean; 7.8 GiB data, 29 GiB used, 686 GiB / 715 GiB avail; 767 B/s rd, 46 MiB/s wr, 1.15k op/s 2024-09-16T09:42:56.988 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:56 smithi084 ceph-mon[32629]: 6.6 scrub starts 2024-09-16T09:42:56.988 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:56 smithi084 ceph-mon[32629]: 6.6 scrub ok 2024-09-16T09:42:56.989 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:56 smithi084 ceph-mon[32629]: 5.a scrub starts 2024-09-16T09:42:56.989 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:56 smithi084 ceph-mon[32629]: 5.a scrub ok 2024-09-16T09:42:57.042 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:42:57.088 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:57.088 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:57.089 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:42:57.160 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-09-16T09:42:57.160 INFO:teuthology.orchestra.run.smithi084.stdout: "size": 20, 2024-09-16T09:42:57.160 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 600, 2024-09-16T09:42:57.160 INFO:teuthology.orchestra.run.smithi084.stdout: "ops": [ 2024-09-16T09:42:57.160 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.161 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:222663 9.1 9:817a7893:::benchmark_data_smithi070_137000_object100952:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e349)", 2024-09-16T09:42:57.161 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:37:10.363067+0000", 2024-09-16T09:42:57.161 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 346.79170548000002, 2024-09-16T09:42:57.161 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.030122435, 2024-09-16T09:42:57.161 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:42:57.161 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:42:57.161 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "reached pg", 2024-09-16T09:42:57.161 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:42:57.161 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:42:57.161 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:42:57.161 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 222663 2024-09-16T09:42:57.161 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.161 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:42:57.162 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.162 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:42:57.162 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363067+0000", 2024-09-16T09:42:57.162 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:42:57.162 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.162 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.162 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:42:57.162 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363067+0000", 2024-09-16T09:42:57.162 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:42:57.162 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.162 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.162 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:42:57.162 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363074+0000", 2024-09-16T09:42:57.162 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.905e-06 2024-09-16T09:42:57.163 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.163 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.163 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:42:57.163 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363083+0000", 2024-09-16T09:42:57.163 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.5019999999999992e-06 2024-09-16T09:42:57.163 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.163 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.163 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:42:57.163 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363099+0000", 2024-09-16T09:42:57.163 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.6427999999999999e-05 2024-09-16T09:42:57.163 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.163 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.163 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:42:57.164 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363108+0000", 2024-09-16T09:42:57.164 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.1900000000000001e-06 2024-09-16T09:42:57.164 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.164 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.164 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:42:57.164 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.397601+0000", 2024-09-16T09:42:57.164 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.034492918999999997 2024-09-16T09:42:57.164 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.164 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.164 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:42:57.164 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.397613+0000", 2024-09-16T09:42:57.164 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.2275e-05 2024-09-16T09:42:57.164 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.164 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.165 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:42:57.165 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.393137+0000", 2024-09-16T09:42:57.165 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.99552360100000004 2024-09-16T09:42:57.165 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.165 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.165 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:42:57.165 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.393190+0000", 2024-09-16T09:42:57.165 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.2614999999999997e-05 2024-09-16T09:42:57.165 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:42:57.165 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:42:57.165 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:42:57.165 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.166 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.166 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:222725 9.1 9:811a4bf7:::benchmark_data_smithi070_137000_object101014:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e349)", 2024-09-16T09:42:57.166 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:37:10.363127+0000", 2024-09-16T09:42:57.166 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 346.791645862, 2024-09-16T09:42:57.166 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0316495720000001, 2024-09-16T09:42:57.166 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:42:57.166 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:42:57.166 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:42:57.166 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:42:57.166 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:42:57.166 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:42:57.166 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 222725 2024-09-16T09:42:57.166 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.167 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:42:57.167 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.167 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:42:57.167 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363127+0000", 2024-09-16T09:42:57.167 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:42:57.167 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.167 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.167 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:42:57.167 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363127+0000", 2024-09-16T09:42:57.167 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:42:57.167 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.167 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.167 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:42:57.167 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363127+0000", 2024-09-16T09:42:57.168 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.9499999999999998e-07 2024-09-16T09:42:57.168 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.168 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.168 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:42:57.168 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363129+0000", 2024-09-16T09:42:57.168 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.261e-06 2024-09-16T09:42:57.168 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.168 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.168 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:42:57.168 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363135+0000", 2024-09-16T09:42:57.168 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.1580000000000004e-06 2024-09-16T09:42:57.168 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.168 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.168 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:42:57.169 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363138+0000", 2024-09-16T09:42:57.169 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.7620000000000001e-06 2024-09-16T09:42:57.169 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.169 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.169 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:42:57.169 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.397654+0000", 2024-09-16T09:42:57.169 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.034515420999999998 2024-09-16T09:42:57.169 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.169 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.170 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:42:57.170 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.397664+0000", 2024-09-16T09:42:57.170 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0549e-05 2024-09-16T09:42:57.170 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.170 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.170 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:42:57.170 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.393573+0000", 2024-09-16T09:42:57.170 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.99590824499999997 2024-09-16T09:42:57.170 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.170 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.170 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:42:57.170 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.393659+0000", 2024-09-16T09:42:57.170 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.6203000000000001e-05 2024-09-16T09:42:57.170 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.171 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.171 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 3,7", 2024-09-16T09:42:57.171 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.393727+0000", 2024-09-16T09:42:57.171 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.7935000000000006e-05 2024-09-16T09:42:57.171 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.171 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.171 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:42:57.171 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.394640+0000", 2024-09-16T09:42:57.171 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00091379700000000005 2024-09-16T09:42:57.171 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.171 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.171 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:42:57.171 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.394674+0000", 2024-09-16T09:42:57.171 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.3800000000000002e-05 2024-09-16T09:42:57.172 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.172 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.172 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:42:57.172 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.394719+0000", 2024-09-16T09:42:57.172 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.4880999999999999e-05 2024-09-16T09:42:57.172 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.172 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.172 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:42:57.172 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.394750+0000", 2024-09-16T09:42:57.172 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.0514e-05 2024-09-16T09:42:57.172 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.172 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.172 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:42:57.173 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.394776+0000", 2024-09-16T09:42:57.173 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.6650999999999999e-05 2024-09-16T09:42:57.173 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:42:57.173 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:42:57.173 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:42:57.173 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.173 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.173 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:222759 9.1 9:84f52a19:::benchmark_data_smithi070_137000_object101048:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e349)", 2024-09-16T09:42:57.173 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:37:10.363143+0000", 2024-09-16T09:42:57.173 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 346.79162978800002, 2024-09-16T09:42:57.173 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.03180293, 2024-09-16T09:42:57.173 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:42:57.173 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:42:57.173 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:42:57.174 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:42:57.174 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:42:57.174 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:42:57.174 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 222759 2024-09-16T09:42:57.174 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.174 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:42:57.174 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.174 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:42:57.174 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363143+0000", 2024-09-16T09:42:57.174 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:42:57.174 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.174 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.174 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:42:57.174 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363143+0000", 2024-09-16T09:42:57.175 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:42:57.175 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.175 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.175 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:42:57.175 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363143+0000", 2024-09-16T09:42:57.175 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.2499999999999999e-07 2024-09-16T09:42:57.175 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.175 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.175 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:42:57.175 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363144+0000", 2024-09-16T09:42:57.175 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1510000000000001e-06 2024-09-16T09:42:57.175 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.176 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.176 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:42:57.176 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363149+0000", 2024-09-16T09:42:57.176 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.3189999999999996e-06 2024-09-16T09:42:57.176 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.176 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.176 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:42:57.176 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363152+0000", 2024-09-16T09:42:57.176 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.9969999999999999e-06 2024-09-16T09:42:57.176 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.176 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.176 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:42:57.176 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.397717+0000", 2024-09-16T09:42:57.176 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.034565851000000002 2024-09-16T09:42:57.177 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.177 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.177 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:42:57.177 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.397728+0000", 2024-09-16T09:42:57.177 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0832999999999999e-05 2024-09-16T09:42:57.177 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.177 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.177 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:42:57.177 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.393932+0000", 2024-09-16T09:42:57.177 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.99620329399999996 2024-09-16T09:42:57.177 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.177 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.177 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:42:57.178 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.394030+0000", 2024-09-16T09:42:57.178 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.8080999999999998e-05 2024-09-16T09:42:57.178 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.178 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.178 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 3,7", 2024-09-16T09:42:57.178 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.394094+0000", 2024-09-16T09:42:57.178 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.4604999999999996e-05 2024-09-16T09:42:57.178 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.178 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.178 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:42:57.178 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.394843+0000", 2024-09-16T09:42:57.178 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00074905100000000004 2024-09-16T09:42:57.178 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.178 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.179 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:42:57.179 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.394854+0000", 2024-09-16T09:42:57.179 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0249e-05 2024-09-16T09:42:57.179 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.179 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.179 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:42:57.179 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.394878+0000", 2024-09-16T09:42:57.179 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.4173e-05 2024-09-16T09:42:57.179 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.179 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.179 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:42:57.179 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.394913+0000", 2024-09-16T09:42:57.179 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.5515000000000001e-05 2024-09-16T09:42:57.179 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.180 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.180 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:42:57.180 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.394946+0000", 2024-09-16T09:42:57.180 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.2586000000000003e-05 2024-09-16T09:42:57.180 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:42:57.180 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:42:57.180 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:42:57.180 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.180 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.180 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:222834 9.1 9:85e273e5:::benchmark_data_smithi070_137000_object101123:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e349)", 2024-09-16T09:42:57.180 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:37:10.363155+0000", 2024-09-16T09:42:57.180 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 346.79161798199999, 2024-09-16T09:42:57.180 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.032180122, 2024-09-16T09:42:57.180 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:42:57.181 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:42:57.181 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:42:57.181 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:42:57.181 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:42:57.181 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:42:57.181 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 222834 2024-09-16T09:42:57.181 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.181 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:42:57.181 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.181 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:42:57.181 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363155+0000", 2024-09-16T09:42:57.181 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:42:57.181 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.181 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.182 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:42:57.182 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363155+0000", 2024-09-16T09:42:57.182 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:42:57.182 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.182 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.182 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:42:57.182 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363155+0000", 2024-09-16T09:42:57.182 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.0800000000000001e-07 2024-09-16T09:42:57.182 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.182 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.182 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:42:57.182 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363156+0000", 2024-09-16T09:42:57.182 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.02e-06 2024-09-16T09:42:57.183 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.183 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.183 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:42:57.183 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363160+0000", 2024-09-16T09:42:57.183 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.9770000000000002e-06 2024-09-16T09:42:57.183 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.183 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.183 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:42:57.183 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363163+0000", 2024-09-16T09:42:57.183 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.174e-06 2024-09-16T09:42:57.183 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.183 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.183 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:42:57.183 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.397768+0000", 2024-09-16T09:42:57.183 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.034605455 2024-09-16T09:42:57.184 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.184 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.184 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:42:57.184 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.397776+0000", 2024-09-16T09:42:57.184 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.5429999999999996e-06 2024-09-16T09:42:57.184 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.184 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.184 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:42:57.184 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.394293+0000", 2024-09-16T09:42:57.184 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.99651656700000002 2024-09-16T09:42:57.184 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.184 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.184 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:42:57.184 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.394365+0000", 2024-09-16T09:42:57.185 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.2575000000000002e-05 2024-09-16T09:42:57.185 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.185 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.185 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 3,7", 2024-09-16T09:42:57.185 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.394423+0000", 2024-09-16T09:42:57.185 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.8220000000000002e-05 2024-09-16T09:42:57.185 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.185 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.185 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:42:57.185 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.394986+0000", 2024-09-16T09:42:57.185 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00056240100000000003 2024-09-16T09:42:57.185 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.185 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.185 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:42:57.186 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.394994+0000", 2024-09-16T09:42:57.186 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.4169999999999992e-06 2024-09-16T09:42:57.186 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.186 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.186 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:42:57.186 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.395287+0000", 2024-09-16T09:42:57.186 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00029250199999999999 2024-09-16T09:42:57.186 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.186 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.186 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:42:57.186 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.395313+0000", 2024-09-16T09:42:57.186 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.6721999999999999e-05 2024-09-16T09:42:57.186 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.186 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.187 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:42:57.187 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.395335+0000", 2024-09-16T09:42:57.187 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.1341000000000001e-05 2024-09-16T09:42:57.187 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:42:57.187 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:42:57.187 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:42:57.187 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.187 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.187 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:222844 9.1 9:80f5d6a4:::benchmark_data_smithi070_137000_object101133:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e349)", 2024-09-16T09:42:57.187 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:37:10.363166+0000", 2024-09-16T09:42:57.187 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 346.79160671300002, 2024-09-16T09:42:57.187 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0331230769999999, 2024-09-16T09:42:57.187 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:42:57.187 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:42:57.188 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:42:57.188 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:42:57.188 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:42:57.188 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:42:57.188 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 222844 2024-09-16T09:42:57.188 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.188 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:42:57.188 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.188 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:42:57.188 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363166+0000", 2024-09-16T09:42:57.188 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:42:57.189 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.189 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.189 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:42:57.189 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363166+0000", 2024-09-16T09:42:57.189 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:42:57.189 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.189 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.190 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:42:57.190 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363166+0000", 2024-09-16T09:42:57.190 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.0900000000000001e-07 2024-09-16T09:42:57.190 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.190 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.190 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:42:57.190 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363167+0000", 2024-09-16T09:42:57.190 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0189999999999999e-06 2024-09-16T09:42:57.190 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.190 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.190 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:42:57.190 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363170+0000", 2024-09-16T09:42:57.190 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.1259999999999998e-06 2024-09-16T09:42:57.191 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.191 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.191 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:42:57.191 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363173+0000", 2024-09-16T09:42:57.191 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.39e-06 2024-09-16T09:42:57.191 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.191 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.191 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:42:57.191 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.397812+0000", 2024-09-16T09:42:57.191 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.034639239000000002 2024-09-16T09:42:57.191 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.191 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.191 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:42:57.191 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.397821+0000", 2024-09-16T09:42:57.191 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.2629999999999992e-06 2024-09-16T09:42:57.192 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.192 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.192 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:42:57.192 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.395016+0000", 2024-09-16T09:42:57.192 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.99719489299999997 2024-09-16T09:42:57.192 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.192 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.192 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:42:57.192 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.395079+0000", 2024-09-16T09:42:57.192 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.2699000000000004e-05 2024-09-16T09:42:57.192 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.192 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.192 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 3,7", 2024-09-16T09:42:57.192 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.395137+0000", 2024-09-16T09:42:57.193 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.8158000000000001e-05 2024-09-16T09:42:57.193 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.193 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.193 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:42:57.193 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.396134+0000", 2024-09-16T09:42:57.193 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00099750399999999997 2024-09-16T09:42:57.193 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.193 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.193 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:42:57.193 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.396186+0000", 2024-09-16T09:42:57.193 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.1962000000000002e-05 2024-09-16T09:42:57.193 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.193 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.193 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:42:57.194 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.396200+0000", 2024-09-16T09:42:57.194 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.4056000000000001e-05 2024-09-16T09:42:57.194 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.194 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.194 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:42:57.194 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.396246+0000", 2024-09-16T09:42:57.194 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.5121999999999999e-05 2024-09-16T09:42:57.194 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.194 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.194 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:42:57.194 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.396289+0000", 2024-09-16T09:42:57.194 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.3436999999999998e-05 2024-09-16T09:42:57.194 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:42:57.194 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:42:57.195 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:42:57.195 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.195 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.195 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:222861 9.1 9:876cd0fc:::benchmark_data_smithi070_137000_object101150:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e349)", 2024-09-16T09:42:57.195 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:37:10.363175+0000", 2024-09-16T09:42:57.195 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 346.791597164, 2024-09-16T09:42:57.195 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.033248876, 2024-09-16T09:42:57.195 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:42:57.195 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:42:57.195 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:42:57.195 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:42:57.195 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:42:57.195 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:42:57.196 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 222861 2024-09-16T09:42:57.196 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.196 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:42:57.196 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.196 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:42:57.196 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363175+0000", 2024-09-16T09:42:57.196 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:42:57.196 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.196 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.196 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:42:57.196 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363175+0000", 2024-09-16T09:42:57.196 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:42:57.196 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.196 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.197 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:42:57.197 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363176+0000", 2024-09-16T09:42:57.197 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.0599999999999999e-07 2024-09-16T09:42:57.197 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.197 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.197 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:42:57.197 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363177+0000", 2024-09-16T09:42:57.197 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.062e-06 2024-09-16T09:42:57.197 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.197 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.197 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:42:57.197 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363181+0000", 2024-09-16T09:42:57.197 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.7840000000000001e-06 2024-09-16T09:42:57.197 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.198 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.198 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:42:57.198 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363183+0000", 2024-09-16T09:42:57.198 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.8140000000000001e-06 2024-09-16T09:42:57.198 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.198 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.198 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:42:57.198 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.397869+0000", 2024-09-16T09:42:57.198 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.034685939999999998 2024-09-16T09:42:57.198 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.198 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.198 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:42:57.198 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.397877+0000", 2024-09-16T09:42:57.198 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.157e-06 2024-09-16T09:42:57.198 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.199 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.199 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:42:57.199 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.395362+0000", 2024-09-16T09:42:57.199 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.99748467399999996 2024-09-16T09:42:57.199 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.199 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.199 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:42:57.199 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.395428+0000", 2024-09-16T09:42:57.199 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.5549999999999994e-05 2024-09-16T09:42:57.199 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.199 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.199 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 3,7", 2024-09-16T09:42:57.199 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.395484+0000", 2024-09-16T09:42:57.200 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.6709999999999997e-05 2024-09-16T09:42:57.200 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.200 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.200 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:42:57.200 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.396296+0000", 2024-09-16T09:42:57.200 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00081205299999999997 2024-09-16T09:42:57.200 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.200 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.200 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:42:57.200 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.396337+0000", 2024-09-16T09:42:57.200 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.0167999999999999e-05 2024-09-16T09:42:57.200 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.200 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.200 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:42:57.201 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.396370+0000", 2024-09-16T09:42:57.201 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.3441000000000001e-05 2024-09-16T09:42:57.201 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.201 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.201 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:42:57.201 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.396400+0000", 2024-09-16T09:42:57.201 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.9737999999999999e-05 2024-09-16T09:42:57.201 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.201 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.201 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:42:57.201 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.396424+0000", 2024-09-16T09:42:57.201 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.4579e-05 2024-09-16T09:42:57.201 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:42:57.201 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:42:57.202 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:42:57.202 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.202 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.202 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:222881 9.1 9:86c78cd8:::benchmark_data_smithi070_137000_object101170:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e349)", 2024-09-16T09:42:57.202 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:37:10.363186+0000", 2024-09-16T09:42:57.202 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 346.79158652699999, 2024-09-16T09:42:57.202 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0333547729999999, 2024-09-16T09:42:57.202 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:42:57.202 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:42:57.202 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:42:57.202 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:42:57.202 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:42:57.202 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:42:57.202 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 222881 2024-09-16T09:42:57.202 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.202 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:42:57.203 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.203 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:42:57.203 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363186+0000", 2024-09-16T09:42:57.203 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:42:57.203 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.203 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.203 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:42:57.203 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363186+0000", 2024-09-16T09:42:57.203 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:42:57.203 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.203 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.203 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:42:57.203 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363186+0000", 2024-09-16T09:42:57.204 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.9999999999999999e-07 2024-09-16T09:42:57.204 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.204 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.204 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:42:57.204 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363187+0000", 2024-09-16T09:42:57.204 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.125e-06 2024-09-16T09:42:57.204 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.204 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.204 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:42:57.204 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363191+0000", 2024-09-16T09:42:57.204 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.7969999999999998e-06 2024-09-16T09:42:57.204 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.204 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.204 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:42:57.205 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363194+0000", 2024-09-16T09:42:57.205 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.7489999999999999e-06 2024-09-16T09:42:57.205 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.205 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.205 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:42:57.205 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.397946+0000", 2024-09-16T09:42:57.205 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.034751690000000002 2024-09-16T09:42:57.205 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.205 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.205 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:42:57.205 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.397955+0000", 2024-09-16T09:42:57.205 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.5990000000000008e-06 2024-09-16T09:42:57.205 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.205 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.205 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:42:57.206 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.395673+0000", 2024-09-16T09:42:57.206 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.99771777100000003 2024-09-16T09:42:57.206 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.206 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.206 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:42:57.206 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.395791+0000", 2024-09-16T09:42:57.206 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00011786300000000001 2024-09-16T09:42:57.206 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.206 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.206 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 3,7", 2024-09-16T09:42:57.206 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.395897+0000", 2024-09-16T09:42:57.206 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00010618 2024-09-16T09:42:57.206 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.206 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.207 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:42:57.207 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.396433+0000", 2024-09-16T09:42:57.207 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00053625899999999998 2024-09-16T09:42:57.207 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.207 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.207 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:42:57.207 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.396463+0000", 2024-09-16T09:42:57.207 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.9346999999999999e-05 2024-09-16T09:42:57.207 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.207 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.207 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:42:57.207 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.396495+0000", 2024-09-16T09:42:57.207 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.2490000000000002e-05 2024-09-16T09:42:57.208 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.208 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.208 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:42:57.208 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.396520+0000", 2024-09-16T09:42:57.208 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.5052e-05 2024-09-16T09:42:57.208 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.208 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.208 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:42:57.208 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.396541+0000", 2024-09-16T09:42:57.208 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.0650999999999999e-05 2024-09-16T09:42:57.208 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:42:57.208 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:42:57.208 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:42:57.208 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.209 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.209 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:222925 9.1 9:81973e20:::benchmark_data_smithi070_137000_object101214:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e349)", 2024-09-16T09:42:57.209 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:37:10.363197+0000", 2024-09-16T09:42:57.209 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 346.79157529600002, 2024-09-16T09:42:57.209 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0353614579999999, 2024-09-16T09:42:57.209 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:42:57.209 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:42:57.209 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:42:57.209 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:42:57.209 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:42:57.209 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:42:57.209 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 222925 2024-09-16T09:42:57.209 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.210 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:42:57.210 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.210 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:42:57.210 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363197+0000", 2024-09-16T09:42:57.210 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:42:57.210 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.210 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.210 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:42:57.210 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363197+0000", 2024-09-16T09:42:57.210 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:42:57.210 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.210 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.210 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:42:57.210 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363198+0000", 2024-09-16T09:42:57.211 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.0200000000000001e-07 2024-09-16T09:42:57.211 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.211 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.211 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:42:57.211 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363199+0000", 2024-09-16T09:42:57.211 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.079e-06 2024-09-16T09:42:57.211 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.211 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.211 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:42:57.211 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363203+0000", 2024-09-16T09:42:57.211 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.8920000000000002e-06 2024-09-16T09:42:57.211 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.211 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.211 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:42:57.212 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363205+0000", 2024-09-16T09:42:57.212 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.7539999999999998e-06 2024-09-16T09:42:57.212 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.212 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.212 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:42:57.212 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.398001+0000", 2024-09-16T09:42:57.212 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.034795595999999998 2024-09-16T09:42:57.212 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.212 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.212 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:42:57.212 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.398008+0000", 2024-09-16T09:42:57.212 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.5850000000000002e-06 2024-09-16T09:42:57.212 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.212 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.212 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:42:57.213 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.396565+0000", 2024-09-16T09:42:57.213 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.99855653799999999 2024-09-16T09:42:57.213 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.213 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.213 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:42:57.213 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.396627+0000", 2024-09-16T09:42:57.213 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.2395000000000001e-05 2024-09-16T09:42:57.213 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.213 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.213 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 3,7", 2024-09-16T09:42:57.213 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.396684+0000", 2024-09-16T09:42:57.213 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.7045e-05 2024-09-16T09:42:57.213 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.213 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.213 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:42:57.214 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.397837+0000", 2024-09-16T09:42:57.214 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0011528070000000001 2024-09-16T09:42:57.214 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.214 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.214 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:42:57.214 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.397880+0000", 2024-09-16T09:42:57.214 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.2701999999999999e-05 2024-09-16T09:42:57.214 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.214 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.214 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:42:57.214 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.398501+0000", 2024-09-16T09:42:57.214 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00062094699999999995 2024-09-16T09:42:57.214 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.214 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.215 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:42:57.215 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.398524+0000", 2024-09-16T09:42:57.215 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.3388e-05 2024-09-16T09:42:57.215 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.215 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.215 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:42:57.215 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.398559+0000", 2024-09-16T09:42:57.215 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.4527999999999999e-05 2024-09-16T09:42:57.215 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:42:57.215 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:42:57.215 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:42:57.215 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.215 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.215 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:225149 9.0 9:0124f23a:::benchmark_data_smithi070_137000_object103438:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e351)", 2024-09-16T09:42:57.215 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:37:12.384254+0000", 2024-09-16T09:42:57.216 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 344.77051867199998, 2024-09-16T09:42:57.216 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.128723189, 2024-09-16T09:42:57.216 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:42:57.216 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:42:57.216 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "reached pg", 2024-09-16T09:42:57.216 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:42:57.216 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:42:57.216 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:42:57.216 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 225149 2024-09-16T09:42:57.216 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.216 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:42:57.216 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.216 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:42:57.217 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.384254+0000", 2024-09-16T09:42:57.217 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:42:57.217 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.217 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.217 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:42:57.217 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.384254+0000", 2024-09-16T09:42:57.217 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:42:57.217 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.217 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.217 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:42:57.217 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.384255+0000", 2024-09-16T09:42:57.217 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.35e-06 2024-09-16T09:42:57.217 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.217 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.218 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:42:57.218 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.384260+0000", 2024-09-16T09:42:57.218 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.729e-06 2024-09-16T09:42:57.218 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.218 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.218 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:42:57.218 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.384273+0000", 2024-09-16T09:42:57.218 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.3456e-05 2024-09-16T09:42:57.218 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.218 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.218 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:42:57.218 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.384281+0000", 2024-09-16T09:42:57.219 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.515e-06 2024-09-16T09:42:57.219 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.219 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.219 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:42:57.219 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.384332+0000", 2024-09-16T09:42:57.219 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.0850000000000003e-05 2024-09-16T09:42:57.219 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.219 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.219 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:42:57.219 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.384455+0000", 2024-09-16T09:42:57.219 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00012285599999999999 2024-09-16T09:42:57.219 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.219 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.219 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 1,2", 2024-09-16T09:42:57.219 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.384564+0000", 2024-09-16T09:42:57.220 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00010888 2024-09-16T09:42:57.220 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.220 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.220 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:42:57.220 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.387738+0000", 2024-09-16T09:42:57.220 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.003174507 2024-09-16T09:42:57.220 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.220 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.220 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:42:57.221 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.394990+0000", 2024-09-16T09:42:57.221 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0072515399999999999 2024-09-16T09:42:57.221 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.221 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.221 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:42:57.221 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.400340+0000", 2024-09-16T09:42:57.221 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0053498829999999997 2024-09-16T09:42:57.221 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.221 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.221 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:42:57.221 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.400349+0000", 2024-09-16T09:42:57.221 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.9580000000000002e-06 2024-09-16T09:42:57.221 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.221 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.222 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:42:57.222 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:13.512961+0000", 2024-09-16T09:42:57.222 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1126110739999999 2024-09-16T09:42:57.222 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.222 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.222 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:42:57.222 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:13.512977+0000", 2024-09-16T09:42:57.222 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.6591000000000001e-05 2024-09-16T09:42:57.222 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:42:57.222 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:42:57.222 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:42:57.222 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.222 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.222 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:225159 9.0 9:033db585:::benchmark_data_smithi070_137000_object103448:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e351)", 2024-09-16T09:42:57.223 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:37:12.389563+0000", 2024-09-16T09:42:57.223 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 344.76520948500001, 2024-09-16T09:42:57.223 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1234441260000001, 2024-09-16T09:42:57.223 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:42:57.223 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:42:57.223 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "reached pg", 2024-09-16T09:42:57.223 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:42:57.223 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:42:57.223 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:42:57.223 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 225159 2024-09-16T09:42:57.223 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.223 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:42:57.223 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.223 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:42:57.224 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.389563+0000", 2024-09-16T09:42:57.224 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:42:57.224 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.224 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.224 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:42:57.224 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.389563+0000", 2024-09-16T09:42:57.224 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:42:57.224 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.224 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.224 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:42:57.224 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.389565+0000", 2024-09-16T09:42:57.224 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.86e-06 2024-09-16T09:42:57.224 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.224 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.224 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:42:57.225 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.389571+0000", 2024-09-16T09:42:57.225 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.2740000000000004e-06 2024-09-16T09:42:57.225 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.225 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.225 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:42:57.225 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.389590+0000", 2024-09-16T09:42:57.225 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.8753000000000002e-05 2024-09-16T09:42:57.225 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.225 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.225 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:42:57.225 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.389599+0000", 2024-09-16T09:42:57.225 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.5380000000000002e-06 2024-09-16T09:42:57.225 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.226 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.226 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:42:57.226 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.389654+0000", 2024-09-16T09:42:57.226 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.5142000000000002e-05 2024-09-16T09:42:57.226 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.226 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.226 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:42:57.226 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.389768+0000", 2024-09-16T09:42:57.226 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00011462400000000001 2024-09-16T09:42:57.226 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.226 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.226 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 1,2", 2024-09-16T09:42:57.226 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.389871+0000", 2024-09-16T09:42:57.226 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000103113 2024-09-16T09:42:57.226 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.227 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.227 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:42:57.227 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.395092+0000", 2024-09-16T09:42:57.227 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.005220298 2024-09-16T09:42:57.227 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.227 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.227 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:42:57.227 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.400656+0000", 2024-09-16T09:42:57.227 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0055647259999999999 2024-09-16T09:42:57.227 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.227 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.227 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:42:57.227 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.400664+0000", 2024-09-16T09:42:57.227 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.6729999999999992e-06 2024-09-16T09:42:57.228 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.228 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.228 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:42:57.228 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:13.513002+0000", 2024-09-16T09:42:57.228 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1123373750000001 2024-09-16T09:42:57.228 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.228 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.228 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:42:57.228 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:13.513007+0000", 2024-09-16T09:42:57.228 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.75e-06 2024-09-16T09:42:57.228 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:42:57.228 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:42:57.228 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:42:57.228 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.229 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.229 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:225235 9.0 9.50884480 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e352)", 2024-09-16T09:42:57.229 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:37:12.434208+0000", 2024-09-16T09:42:57.229 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 344.72056438200002, 2024-09-16T09:42:57.229 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0788204320000001, 2024-09-16T09:42:57.229 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:42:57.229 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:42:57.229 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "reached pg", 2024-09-16T09:42:57.229 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:42:57.229 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:42:57.229 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:42:57.229 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 225235 2024-09-16T09:42:57.229 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.229 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:42:57.229 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.230 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:42:57.230 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.434208+0000", 2024-09-16T09:42:57.230 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:42:57.230 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.230 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.230 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:42:57.230 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.434208+0000", 2024-09-16T09:42:57.230 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:42:57.230 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.230 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.230 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:42:57.230 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:42:57.232 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.434209+0000", 2024-09-16T09:42:57.232 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.158e-06 2024-09-16T09:42:57.232 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.232 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.232 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:42:57.232 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.434215+0000", 2024-09-16T09:42:57.232 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.1699999999999996e-06 2024-09-16T09:42:57.232 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.232 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.232 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:42:57.232 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.434228+0000", 2024-09-16T09:42:57.232 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.332e-05 2024-09-16T09:42:57.233 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.233 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.233 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:42:57.233 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.434235+0000", 2024-09-16T09:42:57.233 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.8859999999999998e-06 2024-09-16T09:42:57.233 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.233 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.233 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:42:57.233 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.434292+0000", 2024-09-16T09:42:57.233 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.7704999999999997e-05 2024-09-16T09:42:57.233 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.233 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.233 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:42:57.233 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.434301+0000", 2024-09-16T09:42:57.234 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.2260000000000005e-06 2024-09-16T09:42:57.234 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.234 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.234 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:42:57.234 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:13.513024+0000", 2024-09-16T09:42:57.234 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0787235479999999 2024-09-16T09:42:57.234 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.234 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.234 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:42:57.234 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:13.513029+0000", 2024-09-16T09:42:57.234 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.4190000000000004e-06 2024-09-16T09:42:57.234 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:42:57.235 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:42:57.235 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:42:57.235 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.235 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.235 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:225256 9.10 9.c1fc3710 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e352)", 2024-09-16T09:42:57.235 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:37:12.441649+0000", 2024-09-16T09:42:57.235 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 344.71312381500002, 2024-09-16T09:42:57.235 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0708169569999999, 2024-09-16T09:42:57.235 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:42:57.235 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:42:57.235 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "reached pg", 2024-09-16T09:42:57.235 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:42:57.235 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:42:57.235 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:42:57.235 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 225256 2024-09-16T09:42:57.236 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.236 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:42:57.236 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.236 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:42:57.236 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.441649+0000", 2024-09-16T09:42:57.236 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:42:57.236 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.236 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.236 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:42:57.236 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.441649+0000", 2024-09-16T09:42:57.236 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:42:57.236 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.236 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.237 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:42:57.237 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.441650+0000", 2024-09-16T09:42:57.237 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.47e-07 2024-09-16T09:42:57.237 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.237 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.237 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:42:57.237 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.441654+0000", 2024-09-16T09:42:57.237 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.6589999999999998e-06 2024-09-16T09:42:57.237 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.237 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.237 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:42:57.237 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.441671+0000", 2024-09-16T09:42:57.237 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.6897000000000001e-05 2024-09-16T09:42:57.238 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.238 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.238 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:42:57.238 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.441681+0000", 2024-09-16T09:42:57.238 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.9129999999999998e-06 2024-09-16T09:42:57.238 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.238 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.238 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:42:57.238 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.441783+0000", 2024-09-16T09:42:57.238 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00010139700000000001 2024-09-16T09:42:57.238 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.238 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.238 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:42:57.239 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.441792+0000", 2024-09-16T09:42:57.239 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.0710000000000001e-06 2024-09-16T09:42:57.239 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.239 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.239 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:42:57.239 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:13.512445+0000", 2024-09-16T09:42:57.239 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.070653525 2024-09-16T09:42:57.239 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.239 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.239 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:42:57.239 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:13.512466+0000", 2024-09-16T09:42:57.239 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.0548e-05 2024-09-16T09:42:57.239 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:42:57.239 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:42:57.239 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:42:57.240 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.240 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.240 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:225291 9.0 9.9752b0a0 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e352)", 2024-09-16T09:42:57.240 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:37:12.458884+0000", 2024-09-16T09:42:57.240 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 344.69588887800001, 2024-09-16T09:42:57.240 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0541780789999999, 2024-09-16T09:42:57.240 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:42:57.240 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:42:57.240 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "reached pg", 2024-09-16T09:42:57.240 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:42:57.240 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:42:57.240 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:42:57.240 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 225291 2024-09-16T09:42:57.240 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.241 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:42:57.241 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.241 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:42:57.241 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.458884+0000", 2024-09-16T09:42:57.241 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:42:57.241 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.241 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.241 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:42:57.241 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.458884+0000", 2024-09-16T09:42:57.241 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:42:57.241 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.241 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.241 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:42:57.241 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.458885+0000", 2024-09-16T09:42:57.242 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.4399999999999998e-07 2024-09-16T09:42:57.242 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.242 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.242 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:42:57.242 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.458889+0000", 2024-09-16T09:42:57.242 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.1400000000000002e-06 2024-09-16T09:42:57.242 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.242 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.242 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:42:57.242 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.458907+0000", 2024-09-16T09:42:57.242 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.8561000000000001e-05 2024-09-16T09:42:57.242 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.242 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.242 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:42:57.243 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.458915+0000", 2024-09-16T09:42:57.243 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.2350000000000002e-06 2024-09-16T09:42:57.243 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.243 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.243 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:42:57.243 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.458986+0000", 2024-09-16T09:42:57.243 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.1273999999999996e-05 2024-09-16T09:42:57.243 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.243 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.243 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:42:57.243 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.458998+0000", 2024-09-16T09:42:57.243 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.2004999999999999e-05 2024-09-16T09:42:57.243 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.243 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.243 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:42:57.244 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:13.513039+0000", 2024-09-16T09:42:57.244 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.054041204 2024-09-16T09:42:57.244 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.244 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.244 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:42:57.244 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:13.513062+0000", 2024-09-16T09:42:57.244 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.2716000000000001e-05 2024-09-16T09:42:57.244 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:42:57.244 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:42:57.244 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:42:57.244 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.244 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.244 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:225292 9.0 9.fcff160 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e352)", 2024-09-16T09:42:57.244 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:37:12.458943+0000", 2024-09-16T09:42:57.245 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 344.69582948300001, 2024-09-16T09:42:57.245 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0541260480000001, 2024-09-16T09:42:57.245 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:42:57.245 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:42:57.245 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "reached pg", 2024-09-16T09:42:57.245 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:42:57.245 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:42:57.245 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:42:57.245 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 225292 2024-09-16T09:42:57.245 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.245 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:42:57.245 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.245 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:42:57.245 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.458943+0000", 2024-09-16T09:42:57.246 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:42:57.246 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.246 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.246 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:42:57.246 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.458943+0000", 2024-09-16T09:42:57.246 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:42:57.246 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.246 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.246 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:42:57.246 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.458944+0000", 2024-09-16T09:42:57.246 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.7899999999999999e-07 2024-09-16T09:42:57.246 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.246 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.246 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:42:57.247 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.458946+0000", 2024-09-16T09:42:57.247 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.4619999999999999e-06 2024-09-16T09:42:57.247 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.247 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.247 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:42:57.247 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.458953+0000", 2024-09-16T09:42:57.247 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.8390000000000001e-06 2024-09-16T09:42:57.247 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.247 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.248 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:42:57.248 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.458958+0000", 2024-09-16T09:42:57.248 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.0010000000000001e-06 2024-09-16T09:42:57.248 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.248 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.248 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:42:57.248 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.459049+0000", 2024-09-16T09:42:57.248 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.1378000000000005e-05 2024-09-16T09:42:57.248 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.248 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.248 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:42:57.248 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.459059+0000", 2024-09-16T09:42:57.248 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.5519999999999993e-06 2024-09-16T09:42:57.249 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.249 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.249 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:42:57.249 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:13.513061+0000", 2024-09-16T09:42:57.249 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.054001653 2024-09-16T09:42:57.249 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.249 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.249 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:42:57.249 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:13.513069+0000", 2024-09-16T09:42:57.249 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.6840000000000002e-06 2024-09-16T09:42:57.249 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:42:57.249 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:42:57.249 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:42:57.249 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.250 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.250 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:225293 9.10 9.ab745fd0 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e352)", 2024-09-16T09:42:57.250 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:37:12.459755+0000", 2024-09-16T09:42:57.250 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 344.695017802, 2024-09-16T09:42:57.250 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0531002519999999, 2024-09-16T09:42:57.250 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:42:57.250 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:42:57.250 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "reached pg", 2024-09-16T09:42:57.250 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:42:57.250 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:42:57.250 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:42:57.250 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 225293 2024-09-16T09:42:57.250 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.250 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:42:57.251 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.251 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:42:57.251 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.459755+0000", 2024-09-16T09:42:57.251 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:42:57.251 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.251 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.251 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:42:57.251 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.459755+0000", 2024-09-16T09:42:57.251 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:42:57.251 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.251 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.251 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:42:57.251 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.459756+0000", 2024-09-16T09:42:57.251 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.9699999999999995e-07 2024-09-16T09:42:57.252 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.252 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.252 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:42:57.252 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.459759+0000", 2024-09-16T09:42:57.252 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.782e-06 2024-09-16T09:42:57.252 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.252 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.252 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:42:57.252 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.459772+0000", 2024-09-16T09:42:57.252 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.253e-05 2024-09-16T09:42:57.252 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.252 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.252 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:42:57.252 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.459779+0000", 2024-09-16T09:42:57.253 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.6980000000000001e-06 2024-09-16T09:42:57.253 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.253 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.253 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:42:57.253 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.459823+0000", 2024-09-16T09:42:57.253 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.3968999999999999e-05 2024-09-16T09:42:57.253 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.253 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.253 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:42:57.253 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.459829+0000", 2024-09-16T09:42:57.253 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.9290000000000003e-06 2024-09-16T09:42:57.253 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.253 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.253 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:42:57.254 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:13.512847+0000", 2024-09-16T09:42:57.254 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.053018314 2024-09-16T09:42:57.254 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.254 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.254 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:42:57.254 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:13.512855+0000", 2024-09-16T09:42:57.254 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.2330000000000006e-06 2024-09-16T09:42:57.254 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:42:57.254 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:42:57.254 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:42:57.254 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.254 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.254 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:231327 9.10 9:0da6d7f3:::benchmark_data_smithi070_137000_object109616:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e357)", 2024-09-16T09:42:57.254 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:37:18.512211+0000", 2024-09-16T09:42:57.255 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 338.642561473, 2024-09-16T09:42:57.255 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.104184364, 2024-09-16T09:42:57.255 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:42:57.255 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:42:57.255 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "reached pg", 2024-09-16T09:42:57.255 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:42:57.255 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:42:57.255 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:42:57.255 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 231327 2024-09-16T09:42:57.255 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.255 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:42:57.255 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.255 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:42:57.255 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.512211+0000", 2024-09-16T09:42:57.255 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:42:57.256 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.256 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.256 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:42:57.256 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.512211+0000", 2024-09-16T09:42:57.256 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:42:57.256 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.256 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.256 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:42:57.256 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.512213+0000", 2024-09-16T09:42:57.256 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.8589999999999999e-06 2024-09-16T09:42:57.256 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.256 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.256 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:42:57.256 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.512216+0000", 2024-09-16T09:42:57.257 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.174e-06 2024-09-16T09:42:57.257 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.257 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.257 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:42:57.257 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.512234+0000", 2024-09-16T09:42:57.257 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.7380999999999998e-05 2024-09-16T09:42:57.257 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.257 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.257 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:42:57.257 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.512242+0000", 2024-09-16T09:42:57.257 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.3680000000000002e-06 2024-09-16T09:42:57.257 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.257 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.257 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:42:57.258 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.684712+0000", 2024-09-16T09:42:57.258 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.17247021300000001 2024-09-16T09:42:57.258 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.258 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.258 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:42:57.258 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.684720+0000", 2024-09-16T09:42:57.258 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.2989999999999997e-06 2024-09-16T09:42:57.258 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.258 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.258 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:42:57.258 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.612846+0000", 2024-09-16T09:42:57.258 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.92812537399999995 2024-09-16T09:42:57.258 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.258 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.259 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for missing object", 2024-09-16T09:42:57.259 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.612970+0000", 2024-09-16T09:42:57.259 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00012453200000000001 2024-09-16T09:42:57.259 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.259 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.259 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:42:57.259 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.616209+0000", 2024-09-16T09:42:57.259 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.003238922 2024-09-16T09:42:57.259 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.259 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.259 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:42:57.259 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.616396+0000", 2024-09-16T09:42:57.259 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00018624200000000001 2024-09-16T09:42:57.259 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:42:57.259 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:42:57.260 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:42:57.260 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.260 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.260 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:231328 9.1 9:8132d7b0:::benchmark_data_smithi070_137000_object109617:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e357)", 2024-09-16T09:42:57.260 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:37:18.512255+0000", 2024-09-16T09:42:57.260 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 338.64251775999998, 2024-09-16T09:42:57.260 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0795497709999999, 2024-09-16T09:42:57.260 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:42:57.260 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:42:57.260 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "reached pg", 2024-09-16T09:42:57.260 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:42:57.260 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:42:57.260 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:42:57.261 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 231328 2024-09-16T09:42:57.261 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.261 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:42:57.261 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.261 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:42:57.261 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.512255+0000", 2024-09-16T09:42:57.261 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:42:57.261 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.261 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.261 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:42:57.261 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.512255+0000", 2024-09-16T09:42:57.261 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:42:57.261 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.261 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.262 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:42:57.262 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.512256+0000", 2024-09-16T09:42:57.262 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.034e-06 2024-09-16T09:42:57.262 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.262 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.262 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:42:57.262 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.512260+0000", 2024-09-16T09:42:57.262 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.9129999999999996e-06 2024-09-16T09:42:57.262 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.262 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.262 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:42:57.262 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.512268+0000", 2024-09-16T09:42:57.262 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.7530000000000001e-06 2024-09-16T09:42:57.262 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.263 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.263 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:42:57.263 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.512273+0000", 2024-09-16T09:42:57.263 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.5149999999999997e-06 2024-09-16T09:42:57.263 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.263 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.263 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:42:57.263 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.684946+0000", 2024-09-16T09:42:57.263 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.17267326299999999 2024-09-16T09:42:57.263 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.263 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.263 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:42:57.263 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.684954+0000", 2024-09-16T09:42:57.264 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.4769999999999997e-06 2024-09-16T09:42:57.264 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.264 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.264 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:42:57.264 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.538886+0000", 2024-09-16T09:42:57.264 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.85393251999999997 2024-09-16T09:42:57.264 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.264 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.264 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for missing object", 2024-09-16T09:42:57.264 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.539061+0000", 2024-09-16T09:42:57.264 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000174694 2024-09-16T09:42:57.264 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.264 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.264 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:42:57.265 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.591746+0000", 2024-09-16T09:42:57.265 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.052684624999999999 2024-09-16T09:42:57.265 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.265 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.265 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:42:57.265 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.591805+0000", 2024-09-16T09:42:57.265 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.8977000000000001e-05 2024-09-16T09:42:57.265 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:42:57.265 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:42:57.265 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:42:57.265 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.265 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.265 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:231329 9.11 9:8a8e13aa:::benchmark_data_smithi070_137000_object109618:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e357)", 2024-09-16T09:42:57.266 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:37:18.512281+0000", 2024-09-16T09:42:57.266 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 338.64249190999999, 2024-09-16T09:42:57.266 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1196006590000001, 2024-09-16T09:42:57.266 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:42:57.266 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:42:57.266 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "reached pg", 2024-09-16T09:42:57.266 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:42:57.266 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:42:57.266 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:42:57.266 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 231329 2024-09-16T09:42:57.266 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.266 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:42:57.266 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.266 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:42:57.266 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.512281+0000", 2024-09-16T09:42:57.267 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:42:57.267 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.267 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.267 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:42:57.267 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.512281+0000", 2024-09-16T09:42:57.267 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:42:57.267 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.267 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.267 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:42:57.267 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.512281+0000", 2024-09-16T09:42:57.267 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.3700000000000001e-07 2024-09-16T09:42:57.267 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.267 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.267 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:42:57.268 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.512283+0000", 2024-09-16T09:42:57.268 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.39e-06 2024-09-16T09:42:57.268 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.268 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.268 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:42:57.268 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.512291+0000", 2024-09-16T09:42:57.268 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.17e-06 2024-09-16T09:42:57.268 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.268 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.268 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:42:57.268 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.512295+0000", 2024-09-16T09:42:57.268 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.2370000000000003e-06 2024-09-16T09:42:57.268 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.268 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.268 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:42:57.268 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.684979+0000", 2024-09-16T09:42:57.269 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.17268373300000001 2024-09-16T09:42:57.269 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.269 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.269 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:42:57.269 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.684985+0000", 2024-09-16T09:42:57.269 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.4239999999999998e-06 2024-09-16T09:42:57.269 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.269 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.269 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:42:57.269 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.627360+0000", 2024-09-16T09:42:57.269 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.94237468999999996 2024-09-16T09:42:57.269 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.269 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.270 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for missing object", 2024-09-16T09:42:57.270 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.627557+0000", 2024-09-16T09:42:57.270 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00019702200000000001 2024-09-16T09:42:57.270 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.270 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.270 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:42:57.270 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.631826+0000", 2024-09-16T09:42:57.270 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0042689950000000003 2024-09-16T09:42:57.270 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.270 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.270 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:42:57.270 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.631881+0000", 2024-09-16T09:42:57.270 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.5661000000000001e-05 2024-09-16T09:42:57.270 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:42:57.271 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:42:57.271 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:42:57.271 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.271 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.271 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:231331 9.10 9:0df6dab9:::benchmark_data_smithi070_137000_object109620:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e357)", 2024-09-16T09:42:57.271 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:37:18.539893+0000", 2024-09-16T09:42:57.271 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 338.614879143, 2024-09-16T09:42:57.271 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0771409569999999, 2024-09-16T09:42:57.271 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:42:57.271 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:42:57.271 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:42:57.271 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:42:57.271 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:42:57.271 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:42:57.271 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 231331 2024-09-16T09:42:57.272 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.272 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:42:57.272 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.272 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:42:57.272 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.539893+0000", 2024-09-16T09:42:57.272 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:42:57.272 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.272 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.272 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:42:57.272 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.539893+0000", 2024-09-16T09:42:57.272 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:42:57.272 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.272 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.272 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:42:57.272 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.539896+0000", 2024-09-16T09:42:57.272 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.4310000000000001e-06 2024-09-16T09:42:57.273 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.273 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.273 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:42:57.273 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.542675+0000", 2024-09-16T09:42:57.273 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0027787900000000002 2024-09-16T09:42:57.273 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.273 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.273 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:42:57.273 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.542701+0000", 2024-09-16T09:42:57.273 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.6122000000000002e-05 2024-09-16T09:42:57.273 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.273 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.273 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:42:57.274 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.542712+0000", 2024-09-16T09:42:57.274 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1443e-05 2024-09-16T09:42:57.274 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.274 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.274 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:42:57.274 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.684747+0000", 2024-09-16T09:42:57.274 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.14203510999999999 2024-09-16T09:42:57.274 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.274 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.275 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:42:57.275 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.684754+0000", 2024-09-16T09:42:57.275 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.7320000000000001e-06 2024-09-16T09:42:57.275 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.275 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.275 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:42:57.275 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.613242+0000", 2024-09-16T09:42:57.275 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.92848750300000005 2024-09-16T09:42:57.275 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.275 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.275 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:42:57.275 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.613347+0000", 2024-09-16T09:42:57.275 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000105361 2024-09-16T09:42:57.276 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.276 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.276 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 1,2", 2024-09-16T09:42:57.276 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.613462+0000", 2024-09-16T09:42:57.276 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00011495 2024-09-16T09:42:57.276 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.276 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.276 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:42:57.276 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.615859+0000", 2024-09-16T09:42:57.276 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0023971729999999998 2024-09-16T09:42:57.276 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.276 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.277 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:42:57.277 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.616173+0000", 2024-09-16T09:42:57.277 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00031437799999999998 2024-09-16T09:42:57.277 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.277 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.277 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:42:57.277 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.616845+0000", 2024-09-16T09:42:57.277 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00067190000000000001 2024-09-16T09:42:57.277 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.277 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.277 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:42:57.277 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.616931+0000", 2024-09-16T09:42:57.277 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.5439000000000006e-05 2024-09-16T09:42:57.278 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.278 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.278 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:42:57.278 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.617034+0000", 2024-09-16T09:42:57.278 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000103625 2024-09-16T09:42:57.278 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:42:57.278 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:42:57.278 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:42:57.278 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.278 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.278 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:231332 9.1 9:83c10802:::benchmark_data_smithi070_137000_object109621:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e357)", 2024-09-16T09:42:57.278 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:37:18.542736+0000", 2024-09-16T09:42:57.278 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 338.61203643900001, 2024-09-16T09:42:57.279 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0489751380000001, 2024-09-16T09:42:57.279 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:42:57.279 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:42:57.279 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:42:57.279 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:42:57.279 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:42:57.279 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:42:57.279 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 231332 2024-09-16T09:42:57.279 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.279 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:42:57.279 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.279 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:42:57.279 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.542736+0000", 2024-09-16T09:42:57.279 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:42:57.280 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.280 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.280 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:42:57.280 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.542736+0000", 2024-09-16T09:42:57.280 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:42:57.280 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.280 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.280 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:42:57.280 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.542738+0000", 2024-09-16T09:42:57.280 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.3480000000000001e-06 2024-09-16T09:42:57.280 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.280 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.280 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:42:57.281 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.542740+0000", 2024-09-16T09:42:57.281 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.261e-06 2024-09-16T09:42:57.281 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.281 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.281 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:42:57.281 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.542746+0000", 2024-09-16T09:42:57.281 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.2670000000000003e-06 2024-09-16T09:42:57.281 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.281 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.281 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:42:57.281 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.542751+0000", 2024-09-16T09:42:57.281 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.8010000000000003e-06 2024-09-16T09:42:57.281 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.281 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.282 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:42:57.282 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.685007+0000", 2024-09-16T09:42:57.282 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.14225658599999999 2024-09-16T09:42:57.282 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.282 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.282 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:42:57.282 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.685016+0000", 2024-09-16T09:42:57.282 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.5199999999999997e-06 2024-09-16T09:42:57.282 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.282 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.282 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:42:57.282 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.542235+0000", 2024-09-16T09:42:57.282 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.85721875300000006 2024-09-16T09:42:57.282 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.282 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.282 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:42:57.283 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.542315+0000", 2024-09-16T09:42:57.283 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.0000999999999995e-05 2024-09-16T09:42:57.283 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.283 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.283 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 3,7", 2024-09-16T09:42:57.283 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.542396+0000", 2024-09-16T09:42:57.283 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.1600000000000005e-05 2024-09-16T09:42:57.283 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.283 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.283 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:42:57.283 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.543116+0000", 2024-09-16T09:42:57.283 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00071917400000000005 2024-09-16T09:42:57.283 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.283 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.283 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:42:57.283 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.546040+0000", 2024-09-16T09:42:57.284 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0029248270000000001 2024-09-16T09:42:57.284 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.284 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.284 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:42:57.284 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.591579+0000", 2024-09-16T09:42:57.284 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.045538617000000003 2024-09-16T09:42:57.284 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.284 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.284 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:42:57.284 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.591626+0000", 2024-09-16T09:42:57.284 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.7274000000000002e-05 2024-09-16T09:42:57.284 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:42:57.284 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:42:57.284 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:42:57.284 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.591711+0000", 2024-09-16T09:42:57.285 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.5109000000000001e-05 2024-09-16T09:42:57.285 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:42:57.285 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:42:57.285 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:42:57.285 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:42:57.285 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:42:57.285 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-09-16T09:42:57.316 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:57.317 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:57.317 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:42:57.418 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:42:57.457 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:57.458 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:57.458 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:42:57.559 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:42:57.593 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:57.593 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:57.594 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:42:57.695 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:42:57.725 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:57.726 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:57.726 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:42:57.827 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:42:57.864 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:57.865 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:57.865 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:42:57.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:57 smithi070 ceph-mon[30388]: 1.0 scrub starts 2024-09-16T09:42:57.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:57 smithi070 ceph-mon[30388]: 1.0 scrub ok 2024-09-16T09:42:57.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:57 smithi070 ceph-mon[30388]: 6.1e scrub starts 2024-09-16T09:42:57.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:57 smithi070 ceph-mon[30388]: 6.1e scrub ok 2024-09-16T09:42:57.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:57 smithi070 ceph-mon[40691]: 1.0 scrub starts 2024-09-16T09:42:57.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:57 smithi070 ceph-mon[40691]: 1.0 scrub ok 2024-09-16T09:42:57.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:57 smithi070 ceph-mon[40691]: 6.1e scrub starts 2024-09-16T09:42:57.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:57 smithi070 ceph-mon[40691]: 6.1e scrub ok 2024-09-16T09:42:57.966 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:42:58.002 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:58.003 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:58.003 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:42:58.104 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:42:58.132 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:57 smithi084 ceph-mon[32629]: 1.0 scrub starts 2024-09-16T09:42:58.132 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:57 smithi084 ceph-mon[32629]: 1.0 scrub ok 2024-09-16T09:42:58.132 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:57 smithi084 ceph-mon[32629]: 6.1e scrub starts 2024-09-16T09:42:58.132 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:57 smithi084 ceph-mon[32629]: 6.1e scrub ok 2024-09-16T09:42:58.135 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:58.135 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:58.136 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:42:58.236 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:42:58.268 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:58.269 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:58.270 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:42:58.370 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:42:58.409 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:58.409 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:58.410 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:42:58.511 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:42:58.547 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:58.547 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:58.548 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:42:58.585 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.7 dump_ops_in_flight 2024-09-16T09:42:58.648 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:42:58.683 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:58.683 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd set noscrub 2024-09-16T09:42:58.684 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:58.684 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:42:58.786 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:42:58.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:58.820 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:58.820 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:42:58.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:58 smithi084 ceph-mon[32629]: 3.1 scrub starts 2024-09-16T09:42:58.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:58 smithi084 ceph-mon[32629]: 3.1 scrub ok 2024-09-16T09:42:58.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:58 smithi084 ceph-mon[32629]: 4.11 scrub starts 2024-09-16T09:42:58.912 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:58 smithi084 ceph-mon[32629]: 4.11 scrub ok 2024-09-16T09:42:58.912 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:58 smithi084 ceph-mon[32629]: 5.1b scrub starts 2024-09-16T09:42:58.912 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:58 smithi084 ceph-mon[32629]: 5.1b scrub ok 2024-09-16T09:42:58.912 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:58 smithi084 ceph-mon[32629]: pgmap v946: 168 pgs: 168 active+clean; 7.7 GiB data, 29 GiB used, 686 GiB / 715 GiB avail; 1.3 KiB/s rd, 15 MiB/s wr, 783 op/s 2024-09-16T09:42:58.921 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:42:58.955 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:58.956 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:58.956 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:42:59.033 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:42:59.057 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:42:59.104 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:59.105 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:59.105 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:42:59.145 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:42:59.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:58 smithi070 ceph-mon[30388]: 3.1 scrub starts 2024-09-16T09:42:59.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:58 smithi070 ceph-mon[30388]: 3.1 scrub ok 2024-09-16T09:42:59.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:58 smithi070 ceph-mon[30388]: 4.11 scrub starts 2024-09-16T09:42:59.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:58 smithi070 ceph-mon[30388]: 4.11 scrub ok 2024-09-16T09:42:59.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:58 smithi070 ceph-mon[30388]: 5.1b scrub starts 2024-09-16T09:42:59.172 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:58 smithi070 ceph-mon[30388]: 5.1b scrub ok 2024-09-16T09:42:59.172 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:58 smithi070 ceph-mon[30388]: pgmap v946: 168 pgs: 168 active+clean; 7.7 GiB data, 29 GiB used, 686 GiB / 715 GiB avail; 1.3 KiB/s rd, 15 MiB/s wr, 783 op/s 2024-09-16T09:42:59.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:58 smithi070 ceph-mon[40691]: 3.1 scrub starts 2024-09-16T09:42:59.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:58 smithi070 ceph-mon[40691]: 3.1 scrub ok 2024-09-16T09:42:59.173 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:58 smithi070 ceph-mon[40691]: 4.11 scrub starts 2024-09-16T09:42:59.173 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:58 smithi070 ceph-mon[40691]: 4.11 scrub ok 2024-09-16T09:42:59.173 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:58 smithi070 ceph-mon[40691]: 5.1b scrub starts 2024-09-16T09:42:59.173 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:58 smithi070 ceph-mon[40691]: 5.1b scrub ok 2024-09-16T09:42:59.173 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:58 smithi070 ceph-mon[40691]: pgmap v946: 168 pgs: 168 active+clean; 7.7 GiB data, 29 GiB used, 686 GiB / 715 GiB avail; 1.3 KiB/s rd, 15 MiB/s wr, 783 op/s 2024-09-16T09:42:59.206 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:42:59.243 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:59.244 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:59.244 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:42:59.345 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:42:59.379 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:59.380 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:59.381 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:42:59.481 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:42:59.520 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:59.521 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:59.521 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:42:59.574 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T09:42:59.622 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:42:59.661 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:42:59.661 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:59.662 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:42:59.720 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:59 smithi084 ceph-mon[32629]: 4.1f scrub starts 2024-09-16T09:42:59.763 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:42:59.784 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:59.785 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:59.786 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:42:59.887 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:42:59.928 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:59 smithi070 ceph-mon[30388]: 4.1f scrub starts 2024-09-16T09:42:59.929 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:59 smithi070 ceph-mon[30388]: 4.1f scrub ok 2024-09-16T09:42:59.929 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:59 smithi070 ceph-mon[30388]: 6.1d scrub starts 2024-09-16T09:42:59.929 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:42:59 smithi070 ceph-mon[30388]: 6.1d scrub ok 2024-09-16T09:42:59.929 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:59 smithi070 ceph-mon[40691]: 4.1f scrub starts 2024-09-16T09:42:59.929 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:59 smithi070 ceph-mon[40691]: 4.1f scrub ok 2024-09-16T09:42:59.929 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:59 smithi070 ceph-mon[40691]: 6.1d scrub starts 2024-09-16T09:42:59.929 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:42:59 smithi070 ceph-mon[40691]: 6.1d scrub ok 2024-09-16T09:42:59.937 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:42:59.938 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:42:59.938 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:43:00.009 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:43:00.040 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:43:00.068 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-09-16T09:43:00.068 INFO:teuthology.orchestra.run.smithi084.stdout: "ops": [ 2024-09-16T09:43:00.068 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:43:00.068 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.16305.0:141345 11.f 11:fb8891e6:::benchmark_data_smithi070_191495_object10875:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e493)", 2024-09-16T09:43:00.068 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:42:59.704806+0000", 2024-09-16T09:43:00.068 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 0.014295271999999999, 2024-09-16T09:43:00.068 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.014322905, 2024-09-16T09:43:00.068 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:43:00.068 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:43:00.068 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "waiting for sub ops", 2024-09-16T09:43:00.069 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:43:00.069 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.16305", 2024-09-16T09:43:00.069 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3818031514", 2024-09-16T09:43:00.069 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 141345 2024-09-16T09:43:00.069 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:43:00.069 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:43:00.069 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:43:00.069 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:43:00.069 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:42:59.704806+0000", 2024-09-16T09:43:00.069 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:43:00.069 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:43:00.069 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:43:00.069 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:43:00.070 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:42:59.704806+0000", 2024-09-16T09:43:00.070 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:43:00.070 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:43:00.070 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:43:00.070 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:43:00.070 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:42:59.704808+0000", 2024-09-16T09:43:00.070 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.5209999999999999e-06 2024-09-16T09:43:00.070 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:43:00.070 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:43:00.070 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:43:00.070 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:42:59.704813+0000", 2024-09-16T09:43:00.070 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.5300000000000004e-06 2024-09-16T09:43:00.070 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:43:00.071 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:43:00.071 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:43:00.071 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:42:59.704829+0000", 2024-09-16T09:43:00.071 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.5218999999999999e-05 2024-09-16T09:43:00.071 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:43:00.071 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:43:00.071 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:43:00.071 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:42:59.704837+0000", 2024-09-16T09:43:00.071 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.3939999999999996e-06 2024-09-16T09:43:00.071 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:43:00.071 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:43:00.071 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:43:00.071 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:42:59.704907+0000", 2024-09-16T09:43:00.072 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.9465e-05 2024-09-16T09:43:00.072 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:43:00.072 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:43:00.072 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:43:00.072 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:42:59.704991+0000", 2024-09-16T09:43:00.072 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.4061999999999996e-05 2024-09-16T09:43:00.072 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:43:00.072 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:43:00.072 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 1,2", 2024-09-16T09:43:00.072 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:42:59.705055+0000", 2024-09-16T09:43:00.072 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.4295999999999996e-05 2024-09-16T09:43:00.072 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:43:00.073 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:43:00.073 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:43:00.073 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:42:59.707686+0000", 2024-09-16T09:43:00.073 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0026310259999999999 2024-09-16T09:43:00.073 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:43:00.073 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:43:00.073 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:43:00.073 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:43:00.073 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:43:00.073 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.16305.0:141350 11.0 11:058ae953:::benchmark_data_smithi070_191495_object10880:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e493)", 2024-09-16T09:43:00.073 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:42:59.711542+0000", 2024-09-16T09:43:00.073 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 0.0075599769999999998, 2024-09-16T09:43:00.073 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0076591990000000002, 2024-09-16T09:43:00.074 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:43:00.074 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:43:00.074 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "waiting for sub ops", 2024-09-16T09:43:00.074 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:43:00.074 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.16305", 2024-09-16T09:43:00.074 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3818031514", 2024-09-16T09:43:00.074 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 141350 2024-09-16T09:43:00.074 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:43:00.074 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:43:00.074 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:43:00.074 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:43:00.074 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:42:59.711542+0000", 2024-09-16T09:43:00.074 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:43:00.075 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:43:00.075 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:43:00.075 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:43:00.075 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:42:59.711542+0000", 2024-09-16T09:43:00.075 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:43:00.075 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:43:00.075 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:43:00.075 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:43:00.075 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:42:59.711543+0000", 2024-09-16T09:43:00.075 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.0299999999999997e-07 2024-09-16T09:43:00.075 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:43:00.075 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:43:00.075 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:43:00.075 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:42:59.711546+0000", 2024-09-16T09:43:00.076 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.5700000000000001e-06 2024-09-16T09:43:00.076 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:43:00.076 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:43:00.076 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:43:00.076 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:42:59.711554+0000", 2024-09-16T09:43:00.076 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.2390000000000002e-06 2024-09-16T09:43:00.076 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:43:00.076 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:43:00.076 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:43:00.076 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:42:59.711561+0000", 2024-09-16T09:43:00.076 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.3600000000000001e-06 2024-09-16T09:43:00.076 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:43:00.076 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:43:00.076 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:43:00.077 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:42:59.711619+0000", 2024-09-16T09:43:00.077 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.8619000000000002e-05 2024-09-16T09:43:00.077 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:43:00.077 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:43:00.077 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:43:00.077 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:42:59.711731+0000", 2024-09-16T09:43:00.077 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000111421 2024-09-16T09:43:00.077 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:43:00.077 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:43:00.077 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 1,3", 2024-09-16T09:43:00.077 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:42:59.711838+0000", 2024-09-16T09:43:00.077 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000107272 2024-09-16T09:43:00.077 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:43:00.078 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:43:00.078 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:43:00.078 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:42:59.719016+0000", 2024-09-16T09:43:00.078 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0071781880000000003 2024-09-16T09:43:00.078 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:43:00.078 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:43:00.078 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:43:00.078 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:42:59.719061+0000", 2024-09-16T09:43:00.078 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.4688999999999999e-05 2024-09-16T09:43:00.078 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:43:00.078 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:43:00.078 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:43:00.078 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:43:00.078 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:43:00.079 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.16305.0:141354 11.0 11:0abd8f9f:::benchmark_data_smithi070_191495_object10884:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e493)", 2024-09-16T09:43:00.079 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:42:59.712527+0000", 2024-09-16T09:43:00.079 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 0.0065745430000000004, 2024-09-16T09:43:00.079 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0067398370000000003, 2024-09-16T09:43:00.079 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:43:00.079 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:43:00.079 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "waiting for sub ops", 2024-09-16T09:43:00.079 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:43:00.079 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.16305", 2024-09-16T09:43:00.079 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3818031514", 2024-09-16T09:43:00.079 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 141354 2024-09-16T09:43:00.079 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:43:00.079 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:43:00.079 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:43:00.080 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:43:00.080 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:42:59.712527+0000", 2024-09-16T09:43:00.080 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:43:00.080 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:43:00.080 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:43:00.080 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:43:00.080 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:42:59.712527+0000", 2024-09-16T09:43:00.080 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:43:00.080 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:43:00.080 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:43:00.080 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:43:00.080 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:42:59.712528+0000", 2024-09-16T09:43:00.080 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.71e-07 2024-09-16T09:43:00.081 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:43:00.081 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:43:00.081 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:43:00.081 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:42:59.712532+0000", 2024-09-16T09:43:00.081 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.0620000000000002e-06 2024-09-16T09:43:00.081 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:43:00.081 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:43:00.081 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:43:00.081 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:42:59.712543+0000", 2024-09-16T09:43:00.081 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0956e-05 2024-09-16T09:43:00.081 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:43:00.081 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:43:00.081 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:43:00.081 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:42:59.712550+0000", 2024-09-16T09:43:00.081 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.063e-06 2024-09-16T09:43:00.082 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:43:00.082 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:43:00.082 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:43:00.082 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:42:59.717023+0000", 2024-09-16T09:43:00.082 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0044730689999999997 2024-09-16T09:43:00.082 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:43:00.082 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:43:00.082 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:43:00.082 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:42:59.717149+0000", 2024-09-16T09:43:00.082 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000125962 2024-09-16T09:43:00.082 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:43:00.082 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:43:00.082 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 1,3", 2024-09-16T09:43:00.083 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:42:59.717273+0000", 2024-09-16T09:43:00.083 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000123652 2024-09-16T09:43:00.083 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:43:00.083 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:43:00.083 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:43:00.083 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:42:59.719154+0000", 2024-09-16T09:43:00.083 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.001880751 2024-09-16T09:43:00.083 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:43:00.083 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:43:00.083 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:43:00.083 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:43:00.083 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:43:00.083 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.16305.0:141356 11.4 11:29b6eb49:::benchmark_data_smithi070_191495_object10886:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e493)", 2024-09-16T09:43:00.084 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:42:59.713326+0000", 2024-09-16T09:43:00.084 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 0.005775896, 2024-09-16T09:43:00.084 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.006003057, 2024-09-16T09:43:00.084 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:43:00.084 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:43:00.084 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "waiting for sub ops", 2024-09-16T09:43:00.084 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:43:00.084 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.16305", 2024-09-16T09:43:00.084 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3818031514", 2024-09-16T09:43:00.084 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 141356 2024-09-16T09:43:00.084 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:43:00.084 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:43:00.084 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:43:00.084 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:43:00.085 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:42:59.713326+0000", 2024-09-16T09:43:00.085 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:43:00.085 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:43:00.085 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:43:00.085 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:43:00.085 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:42:59.713326+0000", 2024-09-16T09:43:00.085 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:43:00.085 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:43:00.085 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:43:00.085 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:43:00.085 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:42:59.713327+0000", 2024-09-16T09:43:00.085 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.096e-06 2024-09-16T09:43:00.085 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:43:00.085 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:43:00.086 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:43:00.086 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:42:59.713331+0000", 2024-09-16T09:43:00.086 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.515e-06 2024-09-16T09:43:00.086 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:43:00.086 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:43:00.086 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:43:00.086 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:42:59.713343+0000", 2024-09-16T09:43:00.086 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1839e-05 2024-09-16T09:43:00.086 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:43:00.086 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:43:00.086 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:43:00.086 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:42:59.713351+0000", 2024-09-16T09:43:00.086 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.4630000000000004e-06 2024-09-16T09:43:00.086 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:43:00.087 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:43:00.087 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:43:00.087 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:42:59.713406+0000", 2024-09-16T09:43:00.087 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.4889999999999998e-05 2024-09-16T09:43:00.087 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:43:00.087 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:43:00.087 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:43:00.087 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:42:59.713504+0000", 2024-09-16T09:43:00.087 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.8054999999999995e-05 2024-09-16T09:43:00.087 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:43:00.087 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:43:00.087 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 2,6", 2024-09-16T09:43:00.087 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:42:59.713598+0000", 2024-09-16T09:43:00.087 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.4069999999999999e-05 2024-09-16T09:43:00.088 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:43:00.088 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:43:00.088 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:43:00.088 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:42:59.718535+0000", 2024-09-16T09:43:00.088 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0049368210000000001 2024-09-16T09:43:00.088 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:43:00.088 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:43:00.088 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:43:00.088 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:43:00.088 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:43:00.088 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.16305.0:141358 11.4 11:215f258e:::benchmark_data_smithi070_191495_object10888:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e493)", 2024-09-16T09:43:00.088 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:42:59.718125+0000", 2024-09-16T09:43:00.088 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 0.00097706500000000005, 2024-09-16T09:43:00.089 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.001264194, 2024-09-16T09:43:00.089 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:43:00.089 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:43:00.089 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "waiting for sub ops", 2024-09-16T09:43:00.089 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:43:00.089 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.16305", 2024-09-16T09:43:00.089 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3818031514", 2024-09-16T09:43:00.089 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 141358 2024-09-16T09:43:00.089 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:43:00.089 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:43:00.089 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:43:00.089 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:43:00.089 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:42:59.718125+0000", 2024-09-16T09:43:00.089 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:43:00.089 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:43:00.090 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:43:00.090 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:43:00.090 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:42:59.718125+0000", 2024-09-16T09:43:00.090 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:43:00.090 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:43:00.090 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:43:00.090 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:43:00.090 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:42:59.718126+0000", 2024-09-16T09:43:00.090 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.5540000000000001e-06 2024-09-16T09:43:00.090 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:43:00.090 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:43:00.090 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:43:00.090 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:42:59.718131+0000", 2024-09-16T09:43:00.090 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.7859999999999996e-06 2024-09-16T09:43:00.091 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:43:00.091 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:43:00.091 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:43:00.091 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:42:59.718143+0000", 2024-09-16T09:43:00.091 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.2445e-05 2024-09-16T09:43:00.091 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:43:00.091 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:43:00.091 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:43:00.091 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:42:59.718150+0000", 2024-09-16T09:43:00.091 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.7830000000000001e-06 2024-09-16T09:43:00.091 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:43:00.091 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:43:00.091 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:43:00.091 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:42:59.718560+0000", 2024-09-16T09:43:00.092 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00040956599999999999 2024-09-16T09:43:00.092 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:43:00.092 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:43:00.092 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:43:00.092 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:42:59.718647+0000", 2024-09-16T09:43:00.092 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.7542999999999999e-05 2024-09-16T09:43:00.092 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:43:00.092 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:43:00.092 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 2,6", 2024-09-16T09:43:00.092 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:42:59.718711+0000", 2024-09-16T09:43:00.092 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.3633000000000006e-05 2024-09-16T09:43:00.092 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:43:00.092 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:43:00.092 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:43:00.093 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:43:00.093 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-09-16T09:43:00.093 INFO:teuthology.orchestra.run.smithi084.stdout: "num_ops": 5 2024-09-16T09:43:00.093 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-09-16T09:43:00.093 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:59 smithi084 ceph-mon[32629]: 4.1f scrub ok 2024-09-16T09:43:00.093 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:59 smithi084 ceph-mon[32629]: 6.1d scrub starts 2024-09-16T09:43:00.093 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:42:59 smithi084 ceph-mon[32629]: 6.1d scrub ok 2024-09-16T09:43:00.095 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:00.096 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:00.096 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:43:00.197 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:43:00.238 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T09:43:00.238 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T09:43:00.238 INFO:tasks.thrashosds.thrasher:Killing osd 2, live_osds are [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-16T09:43:00.239 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-16T09:43:00.239 DEBUG:teuthology.orchestra.run.smithi070:> sudo systemctl stop ceph-a1321834-740c-11ef-bceb-c7b262605968@osd.2 2024-09-16T09:43:00.240 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:00.241 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:00.241 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:43:00.342 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:43:00.364 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:00.365 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:00.365 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:43:00.421 INFO:journalctl@ceph.osd.2.smithi070.stdout:Sep 16 09:43:00 smithi070 systemd[1]: Stopping Ceph osd.2 for a1321834-740c-11ef-bceb-c7b262605968... 2024-09-16T09:43:00.465 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:43:00.495 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:00.496 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:00.496 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:43:00.597 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:43:00.632 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:00.635 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:00.636 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:43:00.704 INFO:teuthology.orchestra.run.smithi070.stderr:noscrub is set 2024-09-16T09:43:00.720 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:00 smithi070 ceph-mon[30388]: 4.a scrub starts 2024-09-16T09:43:00.721 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:00 smithi070 ceph-mon[30388]: 4.a scrub ok 2024-09-16T09:43:00.721 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:00 smithi070 ceph-mon[30388]: pgmap v947: 168 pgs: 168 active+clean; 7.4 GiB data, 28 GiB used, 687 GiB / 715 GiB avail; 1.1 KiB/s rd, 10 MiB/s wr, 1.06k op/s 2024-09-16T09:43:00.721 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:00 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1950439019' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:43:00.721 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:00 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:43:00.721 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:00 smithi070 ceph-mon[40691]: 4.a scrub starts 2024-09-16T09:43:00.721 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:00 smithi070 ceph-mon[40691]: 4.a scrub ok 2024-09-16T09:43:00.722 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:00 smithi070 ceph-mon[40691]: pgmap v947: 168 pgs: 168 active+clean; 7.4 GiB data, 28 GiB used, 687 GiB / 715 GiB avail; 1.1 KiB/s rd, 10 MiB/s wr, 1.06k op/s 2024-09-16T09:43:00.722 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:00 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1950439019' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:43:00.722 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:00 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:43:00.737 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:43:00.791 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:00.792 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:00.792 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:43:00.893 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:43:00.928 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:00.929 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:00.929 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:43:00.975 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:00 smithi084 ceph-mon[32629]: 4.a scrub starts 2024-09-16T09:43:00.975 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:00 smithi084 ceph-mon[32629]: 4.a scrub ok 2024-09-16T09:43:00.975 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:00 smithi084 ceph-mon[32629]: pgmap v947: 168 pgs: 168 active+clean; 7.4 GiB data, 28 GiB used, 687 GiB / 715 GiB avail; 1.1 KiB/s rd, 10 MiB/s wr, 1.06k op/s 2024-09-16T09:43:00.975 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:00 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1950439019' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:43:00.975 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:00 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:43:01.032 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:43:01.067 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:01.068 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:01.068 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:43:01.118 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:43:01.118 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:43:01.118 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:43:01.118 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:43:01.118 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:43:01.118 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:43:01.119 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:43:01.119 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:43:01.119 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:43:01.119 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:43:01.119 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:43:01.119 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:43:01.152 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:43:01.152 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:43:01.169 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:43:01.188 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:43:01.189 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:43:01.199 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:01.200 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:01.200 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:43:01.301 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:43:01.337 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.7 dump_blocked_ops 2024-09-16T09:43:01.338 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:01.338 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:01.338 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:43:01.439 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:43:01.481 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:01.482 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:01.482 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:43:01.583 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:43:01.625 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:01.625 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:01.626 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:43:01.727 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:43:01.761 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:01.762 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:01.762 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:43:01.781 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:43:01.863 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:43:01.906 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:01.907 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:01.907 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:43:01.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:01 smithi070 ceph-mon[30388]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:43:01.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:01 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-16T09:43:01.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:01 smithi070 ceph-mon[30388]: osdmap e494: 8 total, 8 up, 8 in 2024-09-16T09:43:01.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:01 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2555503730' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:43:01.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:01 smithi070 ceph-mon[40691]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:43:01.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:01 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-16T09:43:01.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:01 smithi070 ceph-mon[40691]: osdmap e494: 8 total, 8 up, 8 in 2024-09-16T09:43:01.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:01 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/2555503730' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:43:01.969 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:01 smithi084 ceph-mon[32629]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:43:01.969 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:01 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-16T09:43:01.969 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:01 smithi084 ceph-mon[32629]: osdmap e494: 8 total, 8 up, 8 in 2024-09-16T09:43:01.969 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:01 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/2555503730' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:43:02.008 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:43:02.040 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:02.041 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:02.042 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:43:02.145 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:43:02.183 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:02.184 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:02.184 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:43:02.285 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:43:02.323 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:02.323 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:02.323 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:43:02.421 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:43:01 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:43:01] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:43:02.424 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:43:02.456 INFO:teuthology.orchestra.run.smithi084.stdout:ERROR: (22) Invalid argument 2024-09-16T09:43:02.456 INFO:teuthology.orchestra.run.smithi084.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:43:02.457 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:02.458 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:02.458 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:43:02.559 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:43:02.589 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:02.589 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:02.589 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:43:02.690 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:43:02.719 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:02.720 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:02.721 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:43:02.822 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:43:02.856 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:02.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:02.857 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:43:02.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:02 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:43:02.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:02 smithi070 ceph-mon[30388]: pgmap v949: 168 pgs: 168 active+clean; 7.4 GiB data, 28 GiB used, 687 GiB / 715 GiB avail; 639 B/s rd, 176 KiB/s wr, 755 op/s 2024-09-16T09:43:02.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:02 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:43:02.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:02 smithi070 ceph-mon[40691]: pgmap v949: 168 pgs: 168 active+clean; 7.4 GiB data, 28 GiB used, 687 GiB / 715 GiB avail; 639 B/s rd, 176 KiB/s wr, 755 op/s 2024-09-16T09:43:02.922 INFO:journalctl@ceph.osd.2.smithi070.stdout:Sep 16 09:43:02 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-2[56773]: 2024-09-16T09:43:02.758+0000 7f6d69f4f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T09:43:02.922 INFO:journalctl@ceph.osd.2.smithi070.stdout:Sep 16 09:43:02 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-2[56773]: 2024-09-16T09:43:02.758+0000 7f6d69f4f640 -1 osd.2 494 *** Got signal Terminated *** 2024-09-16T09:43:02.922 INFO:journalctl@ceph.osd.2.smithi070.stdout:Sep 16 09:43:02 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-2[56773]: 2024-09-16T09:43:02.758+0000 7f6d69f4f640 -1 osd.2 494 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T09:43:02.958 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:43:02.989 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:02.990 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:02.990 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:43:03.091 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:43:03.127 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:03.128 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:03.128 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:43:03.153 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:02 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:43:03.153 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:02 smithi084 ceph-mon[32629]: pgmap v949: 168 pgs: 168 active+clean; 7.4 GiB data, 28 GiB used, 687 GiB / 715 GiB avail; 639 B/s rd, 176 KiB/s wr, 755 op/s 2024-09-16T09:43:03.212 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.7 dump_historic_ops 2024-09-16T09:43:03.229 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:43:03.285 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:03.286 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:03.286 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:43:03.387 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:43:03.436 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:03.437 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:03.438 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:43:03.538 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:43:03.573 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:03.574 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:03.574 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:43:03.649 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:43:03.675 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:43:03.701 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-16T09:43:03.725 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:03.725 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:03.726 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:43:03.827 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:43:03.848 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:03.849 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:03.849 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:43:03.910 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:43:03 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:43:03.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:03 smithi084 ceph-mon[32629]: osd.2 marked itself down and dead 2024-09-16T09:43:03.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:03 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:43:03.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:03 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:43:03.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:03 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:43:03.951 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:43:03.973 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:03.974 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:03 smithi070 ceph-mon[30388]: osd.2 marked itself down and dead 2024-09-16T09:43:03.974 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:03 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:43:03.974 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:03 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:43:03.974 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:03 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:43:03.974 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:03 smithi070 ceph-mon[40691]: osd.2 marked itself down and dead 2024-09-16T09:43:03.974 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:03 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:43:03.974 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:03 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:43:03.975 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:03 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:43:03.975 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:03.975 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:43:03.979 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:43:04.076 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:43:04.098 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:04.099 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:04.100 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:43:04.140 INFO:teuthology.orchestra.run.smithi084.stdout:ERROR: (22) Invalid argument 2024-09-16T09:43:04.140 INFO:teuthology.orchestra.run.smithi084.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:43:04.200 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:43:04.222 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:04.222 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:04.223 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:43:04.323 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:43:04.345 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:04.346 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:04.346 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:43:04.447 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:43:04.474 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:04.475 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:04.475 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:43:04.576 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:43:04.597 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:04.598 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:04.599 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:43:04.602 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T09:43:04.606 INFO:journalctl@ceph.osd.2.smithi070.stdout:Sep 16 09:43:04 smithi070 podman[203635]: 2024-09-16 09:43:04.285162097 +0000 UTC m=+1.684288187 container died c1e56f45d139e42b93b87ed71631db379e0843146738bce917cf31eb218d219b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-16T09:43:04.700 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:43:04.734 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:04.735 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:04.735 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:43:04.836 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:43:04.877 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:04 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:43:04.878 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:04 smithi084 ceph-mon[32629]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-16T09:43:04.878 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:04 smithi084 ceph-mon[32629]: osdmap e495: 8 total, 7 up, 8 in 2024-09-16T09:43:04.878 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:04 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:43:04.878 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:04 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:43:04.878 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:04 smithi084 ceph-mon[32629]: pgmap v951: 168 pgs: 5 peering, 6 stale+active+clean, 157 active+clean; 7.1 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.46k op/s 2024-09-16T09:43:04.887 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:04.888 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:04.888 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:43:04.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:04 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:43:04.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:04 smithi070 ceph-mon[30388]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-16T09:43:04.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:04 smithi070 ceph-mon[30388]: osdmap e495: 8 total, 7 up, 8 in 2024-09-16T09:43:04.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:04 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:43:04.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:04 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:43:04.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:04 smithi070 ceph-mon[30388]: pgmap v951: 168 pgs: 5 peering, 6 stale+active+clean, 157 active+clean; 7.1 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.46k op/s 2024-09-16T09:43:04.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:04 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:43:04.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:04 smithi070 ceph-mon[40691]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-16T09:43:04.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:04 smithi070 ceph-mon[40691]: osdmap e495: 8 total, 7 up, 8 in 2024-09-16T09:43:04.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:04 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:43:04.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:04 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:43:04.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:04 smithi070 ceph-mon[40691]: pgmap v951: 168 pgs: 5 peering, 6 stale+active+clean, 157 active+clean; 7.1 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.46k op/s 2024-09-16T09:43:04.922 INFO:journalctl@ceph.osd.2.smithi070.stdout:Sep 16 09:43:04 smithi070 podman[203635]: 2024-09-16 09:43:04.820047242 +0000 UTC m=+2.219173320 container cleanup c1e56f45d139e42b93b87ed71631db379e0843146738bce917cf31eb218d219b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-2, maintainer=Guillaume Abrioux , RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2) 2024-09-16T09:43:04.922 INFO:journalctl@ceph.osd.2.smithi070.stdout:Sep 16 09:43:04 smithi070 bash[203635]: ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-2 2024-09-16T09:43:04.927 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.0 dump_ops_in_flight 2024-09-16T09:43:04.989 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:43:05.024 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:05.025 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:05.025 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:43:05.126 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:43:05.156 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:05.156 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:05.157 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:43:05.257 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:43:05.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:05.281 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:05.281 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:43:05.382 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:43:05.417 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:05.418 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:05.418 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:43:05.420 INFO:journalctl@ceph.osd.2.smithi070.stdout:Sep 16 09:43:05 smithi070 podman[203862]: 2024-09-16 09:43:05.174536123 +0000 UTC m=+0.854448637 container remove c1e56f45d139e42b93b87ed71631db379e0843146738bce917cf31eb218d219b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-16T09:43:05.519 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:43:05.554 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:05.554 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:05.555 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:43:05.656 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:43:05.687 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:05.688 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:05.688 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:43:05.790 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:43:05.822 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:05.823 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:05.823 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:43:05.924 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:43:05.961 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:05.962 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:05.962 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:43:06.063 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:43:06.099 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:06.100 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:06.100 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:43:06.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:05 smithi084 ceph-mon[32629]: osdmap e496: 8 total, 7 up, 8 in 2024-09-16T09:43:06.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:05 smithi084 ceph-mon[32629]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-16T09:43:06.170 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:05 smithi070 ceph-mon[30388]: osdmap e496: 8 total, 7 up, 8 in 2024-09-16T09:43:06.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:05 smithi070 ceph-mon[30388]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-16T09:43:06.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:05 smithi070 ceph-mon[40691]: osdmap e496: 8 total, 7 up, 8 in 2024-09-16T09:43:06.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:05 smithi070 ceph-mon[40691]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-16T09:43:06.201 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:43:06.236 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:06.237 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:06.237 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:43:06.338 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:43:06.370 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:06.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:06.371 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:43:06.472 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:43:06.509 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:06.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:06.510 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:43:06.611 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:43:06.642 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:06.643 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:06.643 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:43:06.744 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:43:06.780 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:06.780 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:06.781 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:43:06.881 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:43:06.916 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:06.917 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:06.917 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:43:06.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:06 smithi070 ceph-mon[30388]: pgmap v953: 168 pgs: 6 active+undersized, 5 peering, 5 stale+active+clean, 3 active+undersized+degraded, 149 active+clean; 7.0 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 682 B/s rd, 0 B/s wr, 1.06k op/s; 7191/346143 objects degraded (2.077%) 2024-09-16T09:43:06.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:06 smithi070 ceph-mon[40691]: pgmap v953: 168 pgs: 6 active+undersized, 5 peering, 5 stale+active+clean, 3 active+undersized+degraded, 149 active+clean; 7.0 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 682 B/s rd, 0 B/s wr, 1.06k op/s; 7191/346143 objects degraded (2.077%) 2024-09-16T09:43:07.018 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:43:07.049 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:07.050 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:07.050 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:43:07.151 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:43:07.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:06 smithi084 ceph-mon[32629]: pgmap v953: 168 pgs: 6 active+undersized, 5 peering, 5 stale+active+clean, 3 active+undersized+degraded, 149 active+clean; 7.0 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 682 B/s rd, 0 B/s wr, 1.06k op/s; 7191/346143 objects degraded (2.077%) 2024-09-16T09:43:07.186 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:07.186 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:07.186 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:43:07.287 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:43:07.317 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:07.318 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:07.318 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:43:07.419 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:43:07.446 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:07.447 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:07.447 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:43:07.506 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:43:07.548 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:43:07.576 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:07.577 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:07.577 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:43:07.678 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:43:07.710 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:07.711 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:07.711 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:43:07.725 INFO:teuthology.orchestra.run.smithi070.stderr:nodeep-scrub is set 2024-09-16T09:43:07.812 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:43:07.846 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:07.847 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:07.847 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:43:07.948 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:43:07.981 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:07.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:07.982 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:43:08.083 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:43:08.108 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:07 smithi070 ceph-mon[30388]: Health check failed: Degraded data redundancy: 7191/346143 objects degraded (2.077%), 3 pgs degraded (PG_DEGRADED) 2024-09-16T09:43:08.109 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:07 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/679017735' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:43:08.109 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:07 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:43:08.109 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:07 smithi070 ceph-mon[40691]: Health check failed: Degraded data redundancy: 7191/346143 objects degraded (2.077%), 3 pgs degraded (PG_DEGRADED) 2024-09-16T09:43:08.109 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:07 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/679017735' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:43:08.109 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:07 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:43:08.119 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:08.120 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:08.121 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:43:08.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:07 smithi084 ceph-mon[32629]: Health check failed: Degraded data redundancy: 7191/346143 objects degraded (2.077%), 3 pgs degraded (PG_DEGRADED) 2024-09-16T09:43:08.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:07 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/679017735' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:43:08.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:07 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:43:08.221 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:43:08.254 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:08.255 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:08.256 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:43:08.356 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:43:08.404 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:08.404 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:08.405 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:43:08.506 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:43:08.538 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:08.539 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:08.539 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:43:08.640 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:43:08.674 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:08.675 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:08.675 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:43:08.751 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:43:08.772 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:43:08.772 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:43:08.775 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:43:08.804 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:08.805 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:08.805 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:43:08.806 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:43:08.806 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:43:08.807 INFO:teuthology.orchestra.run.smithi070.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-16T09:43:08.838 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:43:08.838 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:43:08.874 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:43:08.874 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:43:08.905 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:43:08.905 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:43:08.906 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:43:08.936 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:43:08.936 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:43:08.937 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:08.937 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:08.937 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:43:08.971 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:43:08.971 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:43:08.972 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:08 smithi070 ceph-mon[30388]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:43:08.972 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:08 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-16T09:43:08.972 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:08 smithi070 ceph-mon[30388]: osdmap e497: 8 total, 7 up, 8 in 2024-09-16T09:43:08.972 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:08 smithi070 ceph-mon[30388]: pgmap v955: 168 pgs: 13 active+undersized, 5 peering, 4 stale+active+clean, 8 active+undersized+degraded, 138 active+clean; 6.9 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 2.7 KiB/s rd, 8.2 KiB/s wr, 1.31k op/s; 20891/341652 objects degraded (6.115%); 2.9 KiB/s, 0 objects/s recovering 2024-09-16T09:43:08.972 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:08 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:43:08.972 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:08 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:43:08.972 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:08 smithi070 ceph-mon[40691]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:43:08.972 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:08 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-16T09:43:08.972 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:08 smithi070 ceph-mon[40691]: osdmap e497: 8 total, 7 up, 8 in 2024-09-16T09:43:08.973 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:08 smithi070 ceph-mon[40691]: pgmap v955: 168 pgs: 13 active+undersized, 5 peering, 4 stale+active+clean, 8 active+undersized+degraded, 138 active+clean; 6.9 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 2.7 KiB/s rd, 8.2 KiB/s wr, 1.31k op/s; 20891/341652 objects degraded (6.115%); 2.9 KiB/s, 0 objects/s recovering 2024-09-16T09:43:08.973 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:08 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:43:08.973 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:08 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:43:09.038 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:43:09.067 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:09.068 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:09.068 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:43:09.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:08 smithi084 ceph-mon[32629]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:43:09.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:08 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-16T09:43:09.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:08 smithi084 ceph-mon[32629]: osdmap e497: 8 total, 7 up, 8 in 2024-09-16T09:43:09.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:08 smithi084 ceph-mon[32629]: pgmap v955: 168 pgs: 13 active+undersized, 5 peering, 4 stale+active+clean, 8 active+undersized+degraded, 138 active+clean; 6.9 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 2.7 KiB/s rd, 8.2 KiB/s wr, 1.31k op/s; 20891/341652 objects degraded (6.115%); 2.9 KiB/s, 0 objects/s recovering 2024-09-16T09:43:09.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:08 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:43:09.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:08 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:43:09.169 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:43:09.199 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:09.200 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:09.200 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:43:09.301 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:43:09.339 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:09.339 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:09.340 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:43:09.441 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:43:09.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:09.480 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:09.480 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:43:09.581 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:43:09.618 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:09.619 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:09.619 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:43:09.720 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:43:09.749 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T09:43:09.750 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T09:43:09.766 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:09.767 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:09.767 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:43:09.868 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:43:09.904 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:09.905 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:09.905 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:43:09.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:09 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3810764406' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:43:09.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:09 smithi084 ceph-mon[32629]: osdmap e498: 8 total, 7 up, 8 in 2024-09-16T09:43:09.966 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:09 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3810764406' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:43:09.966 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:09 smithi070 ceph-mon[30388]: osdmap e498: 8 total, 7 up, 8 in 2024-09-16T09:43:09.966 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:09 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3810764406' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:43:09.966 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:09 smithi070 ceph-mon[40691]: osdmap e498: 8 total, 7 up, 8 in 2024-09-16T09:43:10.006 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:43:10.023 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T09:43:10.023 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [ 2024-09-16T09:43:10.023 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:10.023 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.16305.0:154695 11.6 11:6230ec16:::benchmark_data_smithi070_191495_object24225:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e498)", 2024-09-16T09:43:10.023 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:43:10.004948+0000", 2024-09-16T09:43:10.024 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.011088392000000001, 2024-09-16T09:43:10.024 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.011118569999999999, 2024-09-16T09:43:10.024 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:43:10.024 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:43:10.024 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "waiting for sub ops", 2024-09-16T09:43:10.024 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:43:10.024 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.16305", 2024-09-16T09:43:10.024 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3818031514", 2024-09-16T09:43:10.024 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 154695 2024-09-16T09:43:10.024 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:10.024 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:43:10.024 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:10.024 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:43:10.025 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:10.004948+0000", 2024-09-16T09:43:10.025 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:43:10.025 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:10.025 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:10.025 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:43:10.025 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:10.004948+0000", 2024-09-16T09:43:10.025 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:43:10.025 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:10.025 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:10.025 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:43:10.025 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:10.004950+0000", 2024-09-16T09:43:10.025 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0650000000000001e-06 2024-09-16T09:43:10.025 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:10.026 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:10.026 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:43:10.026 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:10.004956+0000", 2024-09-16T09:43:10.026 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.2029999999999997e-06 2024-09-16T09:43:10.026 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:10.026 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:10.026 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:43:10.026 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:10.004975+0000", 2024-09-16T09:43:10.026 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.8243000000000001e-05 2024-09-16T09:43:10.026 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:10.026 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:10.026 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:43:10.026 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:10.004982+0000", 2024-09-16T09:43:10.026 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.8420000000000005e-06 2024-09-16T09:43:10.027 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:10.027 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:10.027 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:43:10.027 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:10.005963+0000", 2024-09-16T09:43:10.027 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0009805320000000001 2024-09-16T09:43:10.027 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:10.027 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:10.027 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:43:10.027 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:10.006101+0000", 2024-09-16T09:43:10.027 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00013831299999999999 2024-09-16T09:43:10.027 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:10.027 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:10.027 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1", 2024-09-16T09:43:10.028 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:10.006245+0000", 2024-09-16T09:43:10.028 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00014417999999999999 2024-09-16T09:43:10.028 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:10.028 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:10.028 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:43:10.028 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:10.009061+0000", 2024-09-16T09:43:10.028 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0028159589999999998 2024-09-16T09:43:10.028 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:43:10.028 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:43:10.028 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:43:10.028 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:10.028 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:10.028 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.16305.0:154704 11.6 11:634c4eb6:::benchmark_data_smithi070_191495_object24234:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e498)", 2024-09-16T09:43:10.029 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:43:10.010132+0000", 2024-09-16T09:43:10.029 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.0059041739999999999, 2024-09-16T09:43:10.029 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0060057360000000002, 2024-09-16T09:43:10.029 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:43:10.029 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:43:10.029 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "waiting for sub ops", 2024-09-16T09:43:10.029 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:43:10.029 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.16305", 2024-09-16T09:43:10.029 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3818031514", 2024-09-16T09:43:10.029 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 154704 2024-09-16T09:43:10.029 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:10.029 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:43:10.029 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:10.029 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:43:10.030 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:10.010132+0000", 2024-09-16T09:43:10.030 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:43:10.030 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:10.030 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:10.030 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:43:10.030 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:10.010132+0000", 2024-09-16T09:43:10.030 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:43:10.030 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:10.030 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:10.030 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:43:10.030 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:10.010143+0000", 2024-09-16T09:43:10.030 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.081e-05 2024-09-16T09:43:10.030 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:10.031 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:10.031 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:43:10.031 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:10.010183+0000", 2024-09-16T09:43:10.031 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0187e-05 2024-09-16T09:43:10.031 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:10.031 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:10.031 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:43:10.031 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:10.010196+0000", 2024-09-16T09:43:10.031 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2561999999999999e-05 2024-09-16T09:43:10.031 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:10.031 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:10.031 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:43:10.031 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:10.010203+0000", 2024-09-16T09:43:10.032 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.5619999999999998e-06 2024-09-16T09:43:10.032 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:10.032 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:10.032 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:43:10.032 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:10.012228+0000", 2024-09-16T09:43:10.032 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0020246320000000002 2024-09-16T09:43:10.032 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:10.032 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:10.032 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:43:10.032 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:10.012393+0000", 2024-09-16T09:43:10.032 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00016516 2024-09-16T09:43:10.032 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:10.032 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:10.032 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1", 2024-09-16T09:43:10.033 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:10.012503+0000", 2024-09-16T09:43:10.033 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000110068 2024-09-16T09:43:10.033 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:43:10.033 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:43:10.033 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:43:10.033 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:10.033 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:10.033 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop_reply(client.16305.0:154695 11.6 e498/495)", 2024-09-16T09:43:10.033 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:43:10.013481+0000", 2024-09-16T09:43:10.033 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.002555599, 2024-09-16T09:43:10.033 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0027121229999999999, 2024-09-16T09:43:10.033 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:43:10.033 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:43:10.034 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T09:43:10.034 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:43:10.034 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:10.034 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:43:10.034 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:10.013481+0000", 2024-09-16T09:43:10.034 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:43:10.034 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:10.034 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:10.034 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:43:10.034 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:10.013481+0000", 2024-09-16T09:43:10.034 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:43:10.034 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:10.034 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:10.034 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:43:10.035 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:10.013482+0000", 2024-09-16T09:43:10.035 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6000000000000003e-07 2024-09-16T09:43:10.035 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:10.035 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:10.035 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:43:10.035 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:10.013487+0000", 2024-09-16T09:43:10.035 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.1200000000000001e-06 2024-09-16T09:43:10.035 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:10.035 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:10.035 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:43:10.035 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:10.013497+0000", 2024-09-16T09:43:10.035 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0463e-05 2024-09-16T09:43:10.035 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:10.035 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:10.036 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:43:10.036 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:10.013504+0000", 2024-09-16T09:43:10.036 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.9489999999999999e-06 2024-09-16T09:43:10.036 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:43:10.036 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:43:10.036 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:43:10.036 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:10.036 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:10.037 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.16305.0:154711 11.b e498/454 11:def0ce25:::benchmark_data_smithi070_191495_object24241:head v 498'9570, mlcod=498'9568)", 2024-09-16T09:43:10.037 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:43:10.015489+0000", 2024-09-16T09:43:10.037 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.000547884, 2024-09-16T09:43:10.037 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000742679, 2024-09-16T09:43:10.037 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:43:10.037 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:43:10.037 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T09:43:10.037 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:43:10.037 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:10.037 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:43:10.037 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:10.015489+0000", 2024-09-16T09:43:10.038 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:43:10.038 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:10.038 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:10.038 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:43:10.038 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:10.015489+0000", 2024-09-16T09:43:10.038 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:43:10.038 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:10.038 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:10.038 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:43:10.038 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:10.015489+0000", 2024-09-16T09:43:10.038 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.5300000000000003e-07 2024-09-16T09:43:10.038 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:10.038 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:10.038 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:43:10.039 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:10.015496+0000", 2024-09-16T09:43:10.039 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.9990000000000002e-06 2024-09-16T09:43:10.039 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:10.039 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:10.039 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:43:10.039 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:10.015511+0000", 2024-09-16T09:43:10.039 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.4508000000000001e-05 2024-09-16T09:43:10.039 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:10.039 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:10.039 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:43:10.039 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:10.015518+0000", 2024-09-16T09:43:10.039 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.5379999999999997e-06 2024-09-16T09:43:10.039 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:10.040 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:10.040 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:43:10.040 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:10.015595+0000", 2024-09-16T09:43:10.040 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6767000000000005e-05 2024-09-16T09:43:10.040 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:10.040 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:10.040 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:43:10.040 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:10.015614+0000", 2024-09-16T09:43:10.040 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.8581999999999999e-05 2024-09-16T09:43:10.040 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:43:10.040 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:43:10.040 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:43:10.040 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:43:10.040 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-16T09:43:10.041 INFO:teuthology.orchestra.run.smithi070.stdout: "num_ops": 4 2024-09-16T09:43:10.041 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T09:43:10.045 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:10.045 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:10.046 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:43:10.146 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:43:10.180 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:10.181 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:10.182 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:43:10.217 INFO:journalctl@ceph.osd.2.smithi070.stdout:Sep 16 09:43:10 smithi070 podman[204468]: 2024-09-16 09:43:09.978598054 +0000 UTC m=+0.105362024 image pull 425e316ba8ebf74f04aa3640f7deb4fa3f293a78d0c8d097cb679eaf86ddca6c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce 2024-09-16T09:43:10.217 INFO:journalctl@ceph.osd.2.smithi070.stdout:Sep 16 09:43:10 smithi070 podman[204468]: 2024-09-16 09:43:10.099947619 +0000 UTC m=+0.226711561 container create 64c4829a8d2fee878a8d5ae6aedb7a690277309860cd6c3fb99e5fee3b1027ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-2-deactivate, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T09:43:10.282 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:43:10.320 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:10.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:10.321 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:43:10.422 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:43:10.461 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:10.462 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:10.462 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:43:10.500 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd unset noscrub 2024-09-16T09:43:10.563 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:43:10.603 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:10.603 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:10.603 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:43:10.704 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:43:10.736 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:10.736 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:10.737 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:43:10.837 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:43:10.877 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:10.877 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:10.878 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:43:10.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:10 smithi070 ceph-mon[30388]: pgmap v957: 168 pgs: 31 active+undersized, 18 active+undersized+degraded, 119 active+clean; 6.7 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1.8 KiB/s rd, 8.2 KiB/s wr, 1.07k op/s; 40941/327861 objects degraded (12.487%); 2.9 KiB/s, 0 objects/s recovering 2024-09-16T09:43:10.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:10 smithi070 ceph-mon[40691]: pgmap v957: 168 pgs: 31 active+undersized, 18 active+undersized+degraded, 119 active+clean; 6.7 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1.8 KiB/s rd, 8.2 KiB/s wr, 1.07k op/s; 40941/327861 objects degraded (12.487%); 2.9 KiB/s, 0 objects/s recovering 2024-09-16T09:43:10.978 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:43:11.013 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:11.014 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:11.014 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:43:11.102 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.0 dump_blocked_ops 2024-09-16T09:43:11.114 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:43:11.150 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:11.151 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:11.151 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:43:11.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:10 smithi084 ceph-mon[32629]: pgmap v957: 168 pgs: 31 active+undersized, 18 active+undersized+degraded, 119 active+clean; 6.7 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1.8 KiB/s rd, 8.2 KiB/s wr, 1.07k op/s; 40941/327861 objects degraded (12.487%); 2.9 KiB/s, 0 objects/s recovering 2024-09-16T09:43:11.252 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:43:11.299 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:11.300 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:11.300 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:43:11.311 INFO:journalctl@ceph.osd.2.smithi070.stdout:Sep 16 09:43:11 smithi070 podman[204468]: 2024-09-16 09:43:11.201621289 +0000 UTC m=+1.328385242 container init 64c4829a8d2fee878a8d5ae6aedb7a690277309860cd6c3fb99e5fee3b1027ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-16T09:43:11.311 INFO:journalctl@ceph.osd.2.smithi070.stdout:Sep 16 09:43:11 smithi070 podman[204468]: 2024-09-16 09:43:11.212890141 +0000 UTC m=+1.339654088 container start 64c4829a8d2fee878a8d5ae6aedb7a690277309860cd6c3fb99e5fee3b1027ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-16T09:43:11.311 INFO:journalctl@ceph.osd.2.smithi070.stdout:Sep 16 09:43:11 smithi070 podman[204468]: 2024-09-16 09:43:11.309858816 +0000 UTC m=+1.436622790 container attach 64c4829a8d2fee878a8d5ae6aedb7a690277309860cd6c3fb99e5fee3b1027ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-2-deactivate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, GIT_BRANCH=HEAD) 2024-09-16T09:43:11.401 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:43:11.428 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:11.429 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:11.429 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:43:11.478 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:43:11.530 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:43:11.534 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:43:11.556 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:11.557 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:11.558 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:43:11.658 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:43:11.696 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:11.696 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:11.696 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:43:11.751 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T09:43:11.797 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:43:11.824 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:11.825 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:11.825 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:43:11.829 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:11 smithi070 ceph-mon[30388]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-16T09:43:11.830 INFO:journalctl@ceph.osd.2.smithi070.stdout:Sep 16 09:43:11 smithi070 podman[204468]: 2024-09-16 09:43:11.750589768 +0000 UTC m=+1.877353720 container died 64c4829a8d2fee878a8d5ae6aedb7a690277309860cd6c3fb99e5fee3b1027ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-2-deactivate, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS) 2024-09-16T09:43:11.831 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:11 smithi070 ceph-mon[40691]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-16T09:43:11.926 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:43:11.956 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:11.957 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:11.957 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:43:12.058 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:43:12.089 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:12.089 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:12.090 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:43:12.117 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:43:11 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:43:11] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:43:12.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:11 smithi084 ceph-mon[32629]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-16T09:43:12.190 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:43:12.227 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:12.227 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:12.228 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:43:12.328 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:43:12.364 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:12.365 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:12.365 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:43:12.466 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:43:12.503 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:12.503 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:12.504 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:43:12.604 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:43:12.648 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:12.649 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:12.649 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:43:12.750 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:43:12.764 INFO:teuthology.orchestra.run.smithi070.stderr:noscrub is unset 2024-09-16T09:43:12.788 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:12.789 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:12.789 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:43:12.890 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:43:12.924 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:12.925 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:12.925 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:43:13.026 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:43:13.058 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:13.059 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:13.059 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:43:13.160 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:43:13.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:12 smithi084 ceph-mon[32629]: pgmap v958: 168 pgs: 31 active+undersized, 18 active+undersized+degraded, 119 active+clean; 6.7 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1.5 KiB/s rd, 6.8 KiB/s wr, 899 op/s; 40941/327861 objects degraded (12.487%); 2.4 KiB/s, 0 objects/s recovering 2024-09-16T09:43:13.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:12 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/2345266845' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:43:13.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:12 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:43:13.170 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:12 smithi070 ceph-mon[30388]: pgmap v958: 168 pgs: 31 active+undersized, 18 active+undersized+degraded, 119 active+clean; 6.7 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1.5 KiB/s rd, 6.8 KiB/s wr, 899 op/s; 40941/327861 objects degraded (12.487%); 2.4 KiB/s, 0 objects/s recovering 2024-09-16T09:43:13.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:12 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2345266845' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:43:13.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:12 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:43:13.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:12 smithi070 ceph-mon[40691]: pgmap v958: 168 pgs: 31 active+undersized, 18 active+undersized+degraded, 119 active+clean; 6.7 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1.5 KiB/s rd, 6.8 KiB/s wr, 899 op/s; 40941/327861 objects degraded (12.487%); 2.4 KiB/s, 0 objects/s recovering 2024-09-16T09:43:13.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:12 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/2345266845' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:43:13.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:12 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:43:13.198 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:13.198 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:13.199 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:43:13.299 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:43:13.333 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:13.333 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:13.334 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:43:13.435 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:43:13.467 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:13.467 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:13.468 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:43:13.569 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:43:13.605 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:13.605 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:13.606 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:43:13.706 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:43:13.740 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:13.740 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:13.741 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:43:13.841 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:43:13.878 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:13.879 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:13.879 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:43:13.910 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:43:13 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:43:13.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:13 smithi084 ceph-mon[32629]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:43:13.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:13 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-16T09:43:13.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:13 smithi084 ceph-mon[32629]: osdmap e499: 8 total, 7 up, 8 in 2024-09-16T09:43:13.980 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:43:14.015 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:14.015 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:14.016 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:43:14.052 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:13 smithi070 ceph-mon[30388]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:43:14.053 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:13 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-16T09:43:14.053 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:13 smithi070 ceph-mon[30388]: osdmap e499: 8 total, 7 up, 8 in 2024-09-16T09:43:14.053 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:13 smithi070 ceph-mon[40691]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:43:14.053 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:13 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-16T09:43:14.053 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:13 smithi070 ceph-mon[40691]: osdmap e499: 8 total, 7 up, 8 in 2024-09-16T09:43:14.074 DEBUG:teuthology.orchestra.run.smithi070:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a1321834-740c-11ef-bceb-c7b262605968@osd.2.service' 2024-09-16T09:43:14.116 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:43:14.155 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:14.156 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:14.156 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:43:14.158 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T09:43:14.158 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-16T09:43:14.158 INFO:tasks.thrashosds.thrasher:Testing ceph-objectstore-tool on down osd.2 2024-09-16T09:43:14.159 INFO:tasks.thrashosds.thrasher:remote for osd 2 is ubuntu@smithi070.front.sepia.ceph.com 2024-09-16T09:43:14.159 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell -n osd.2 --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-2 --journal-path /var/lib/ceph/osd/ceph-2/journal --op list-pgs 2024-09-16T09:43:14.257 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:43:14.293 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:14.294 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:14.294 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:43:14.395 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:43:14.432 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:14.432 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:14.433 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:43:14.533 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:43:14.575 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:14.576 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:14.577 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:43:14.678 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:43:14.709 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:14.710 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:14.711 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:43:14.794 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/osd.2/config 2024-09-16T09:43:14.812 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:43:14.820 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:14 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:43:14.820 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:14 smithi070 ceph-mon[40691]: 6.19 scrub starts 2024-09-16T09:43:14.820 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:14 smithi070 ceph-mon[40691]: 6.19 scrub ok 2024-09-16T09:43:14.820 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:14 smithi070 ceph-mon[40691]: pgmap v960: 168 pgs: 31 active+undersized, 18 active+undersized+degraded, 119 active+clean; 6.2 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 664 B/s rd, 0 B/s wr, 1.94k op/s; 38518/305817 objects degraded (12.595%); 0 B/s, 0 objects/s recovering 2024-09-16T09:43:14.821 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:14 smithi070 ceph-mon[40691]: Health check update: Degraded data redundancy: 40941/327861 objects degraded (12.487%), 18 pgs degraded (PG_DEGRADED) 2024-09-16T09:43:14.822 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:14 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:43:14.822 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:14 smithi070 ceph-mon[30388]: 6.19 scrub starts 2024-09-16T09:43:14.822 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:14 smithi070 ceph-mon[30388]: 6.19 scrub ok 2024-09-16T09:43:14.823 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:14 smithi070 ceph-mon[30388]: pgmap v960: 168 pgs: 31 active+undersized, 18 active+undersized+degraded, 119 active+clean; 6.2 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 664 B/s rd, 0 B/s wr, 1.94k op/s; 38518/305817 objects degraded (12.595%); 0 B/s, 0 objects/s recovering 2024-09-16T09:43:14.823 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:14 smithi070 ceph-mon[30388]: Health check update: Degraded data redundancy: 40941/327861 objects degraded (12.487%), 18 pgs degraded (PG_DEGRADED) 2024-09-16T09:43:14.823 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T09:43:14.823 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T09:43:14.823 INFO:teuthology.orchestra.run.smithi070.stdout: "complaint_time": 30, 2024-09-16T09:43:14.823 INFO:teuthology.orchestra.run.smithi070.stdout: "num_blocked_ops": 0 2024-09-16T09:43:14.823 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T09:43:14.854 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:14.855 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:14.855 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:43:14.956 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:43:14.992 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:14.993 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:14.993 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:43:15.094 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:43:15.126 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:15.127 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:15.127 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:43:15.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:14 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:43:15.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:14 smithi084 ceph-mon[32629]: 6.19 scrub starts 2024-09-16T09:43:15.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:14 smithi084 ceph-mon[32629]: 6.19 scrub ok 2024-09-16T09:43:15.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:14 smithi084 ceph-mon[32629]: pgmap v960: 168 pgs: 31 active+undersized, 18 active+undersized+degraded, 119 active+clean; 6.2 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 664 B/s rd, 0 B/s wr, 1.94k op/s; 38518/305817 objects degraded (12.595%); 0 B/s, 0 objects/s recovering 2024-09-16T09:43:15.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:14 smithi084 ceph-mon[32629]: Health check update: Degraded data redundancy: 40941/327861 objects degraded (12.487%), 18 pgs degraded (PG_DEGRADED) 2024-09-16T09:43:15.228 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:43:15.263 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:15.264 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:15.264 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:43:15.365 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:43:15.402 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:15.402 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:15.402 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:43:15.503 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:43:15.534 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:15.534 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:15.535 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:43:15.635 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:43:15.661 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:15.662 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:15.663 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:43:15.695 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:43:15.717 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.0 dump_historic_ops 2024-09-16T09:43:15.764 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:43:15.800 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:15.801 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:15.801 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:43:15.902 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:43:15.920 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:15 smithi070 ceph-mon[40691]: 11.0 scrub starts 2024-09-16T09:43:15.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:15 smithi070 ceph-mon[30388]: 11.0 scrub starts 2024-09-16T09:43:15.942 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:15.942 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:15.943 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:43:16.044 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:43:16.079 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:16.080 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:16.080 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:43:16.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:15 smithi084 ceph-mon[32629]: 11.0 scrub starts 2024-09-16T09:43:16.180 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:43:16.215 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:16.216 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:16.216 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:43:16.317 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:43:16.348 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:16.348 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:16.349 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:43:16.449 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:43:16.482 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:16.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:16.483 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:43:16.584 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:43:16.592 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-16T09:43:16.612 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:16.613 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:16.613 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:43:16.714 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:43:16.750 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:16.751 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:16.751 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:43:16.852 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:43:16.892 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:16.893 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:16.893 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:43:16.993 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:43:17.023 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:17.024 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:17.024 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:43:17.077 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:16 smithi070 ceph-mon[30388]: pgmap v961: 168 pgs: 31 active+undersized, 18 active+undersized+degraded, 119 active+clean; 6.2 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 511 B/s rd, 0 B/s wr, 1.55k op/s; 38055/304428 objects degraded (12.500%); 0 B/s, 0 objects/s recovering 2024-09-16T09:43:17.077 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:16 smithi070 ceph-mon[30388]: 4.1d scrub starts 2024-09-16T09:43:17.077 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:16 smithi070 ceph-mon[30388]: 4.1d scrub ok 2024-09-16T09:43:17.078 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:16 smithi070 ceph-mon[40691]: pgmap v961: 168 pgs: 31 active+undersized, 18 active+undersized+degraded, 119 active+clean; 6.2 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 511 B/s rd, 0 B/s wr, 1.55k op/s; 38055/304428 objects degraded (12.500%); 0 B/s, 0 objects/s recovering 2024-09-16T09:43:17.078 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:16 smithi070 ceph-mon[40691]: 4.1d scrub starts 2024-09-16T09:43:17.078 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:16 smithi070 ceph-mon[40691]: 4.1d scrub ok 2024-09-16T09:43:17.125 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:43:17.158 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:17.159 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:17.159 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:43:17.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:16 smithi084 ceph-mon[32629]: pgmap v961: 168 pgs: 31 active+undersized, 18 active+undersized+degraded, 119 active+clean; 6.2 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 511 B/s rd, 0 B/s wr, 1.55k op/s; 38055/304428 objects degraded (12.500%); 0 B/s, 0 objects/s recovering 2024-09-16T09:43:17.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:16 smithi084 ceph-mon[32629]: 4.1d scrub starts 2024-09-16T09:43:17.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:16 smithi084 ceph-mon[32629]: 4.1d scrub ok 2024-09-16T09:43:17.260 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:43:17.294 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:17.294 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:17.295 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:43:17.396 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:43:17.429 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:17.430 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:17.430 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:43:17.531 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:43:17.561 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:17.562 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:17.562 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:43:17.573 INFO:teuthology.orchestra.run.smithi070.stderr:Creating an OSD daemon form without an OSD FSID value 2024-09-16T09:43:17.664 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:43:17.693 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:17.694 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:17.694 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:43:17.795 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:43:17.831 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:17.832 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:17.832 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:43:17.909 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:43:17.910 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:43:17.933 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:43:17.944 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:43:17.945 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:43:17.945 INFO:teuthology.orchestra.run.smithi070.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-16T09:43:17.971 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:17.972 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:17.972 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:43:17.980 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:43:17.980 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:43:18.011 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:43:18.012 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:43:18.041 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:17 smithi070 ceph-mon[30388]: 11.0 scrub ok 2024-09-16T09:43:18.041 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:17 smithi070 ceph-mon[30388]: 6.1f scrub starts 2024-09-16T09:43:18.041 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:17 smithi070 ceph-mon[30388]: 6.1f scrub ok 2024-09-16T09:43:18.042 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:17 smithi070 ceph-mon[40691]: 11.0 scrub ok 2024-09-16T09:43:18.042 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:17 smithi070 ceph-mon[40691]: 6.1f scrub starts 2024-09-16T09:43:18.042 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:17 smithi070 ceph-mon[40691]: 6.1f scrub ok 2024-09-16T09:43:18.042 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:43:18.042 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:43:18.073 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:43:18.073 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:43:18.074 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:43:18.107 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:43:18.107 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:43:18.116 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:18.117 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:18.118 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:43:18.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:17 smithi084 ceph-mon[32629]: 11.0 scrub ok 2024-09-16T09:43:18.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:17 smithi084 ceph-mon[32629]: 6.1f scrub starts 2024-09-16T09:43:18.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:17 smithi084 ceph-mon[32629]: 6.1f scrub ok 2024-09-16T09:43:18.218 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:43:18.250 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:18.250 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:18.250 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:43:18.351 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:43:18.384 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:18.384 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:18.385 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:43:18.486 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:43:18.518 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:18.518 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:18.519 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:43:18.620 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:43:18.651 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:18.652 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:18.652 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:43:18.753 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:43:18.782 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:18.783 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:18.783 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:43:18.884 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:43:18.922 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:18.923 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:18.923 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:43:18.941 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T09:43:18.941 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T09:43:19.024 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:43:19.060 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:19.061 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:19.062 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:43:19.124 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:18 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2116736746' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:43:19.124 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:18 smithi070 ceph-mon[30388]: pgmap v962: 168 pgs: 31 active+undersized, 18 active+undersized+degraded, 119 active+clean; 6.1 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.25k op/s; 37105/298671 objects degraded (12.423%); 0 B/s, 0 objects/s recovering 2024-09-16T09:43:19.124 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:18 smithi070 ceph-mon[30388]: 4.1 scrub starts 2024-09-16T09:43:19.124 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:18 smithi070 ceph-mon[30388]: 4.1 scrub ok 2024-09-16T09:43:19.125 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:18 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/2116736746' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:43:19.125 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:18 smithi070 ceph-mon[40691]: pgmap v962: 168 pgs: 31 active+undersized, 18 active+undersized+degraded, 119 active+clean; 6.1 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.25k op/s; 37105/298671 objects degraded (12.423%); 0 B/s, 0 objects/s recovering 2024-09-16T09:43:19.125 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:18 smithi070 ceph-mon[40691]: 4.1 scrub starts 2024-09-16T09:43:19.125 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:18 smithi070 ceph-mon[40691]: 4.1 scrub ok 2024-09-16T09:43:19.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:18 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/2116736746' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:43:19.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:18 smithi084 ceph-mon[32629]: pgmap v962: 168 pgs: 31 active+undersized, 18 active+undersized+degraded, 119 active+clean; 6.1 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.25k op/s; 37105/298671 objects degraded (12.423%); 0 B/s, 0 objects/s recovering 2024-09-16T09:43:19.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:18 smithi084 ceph-mon[32629]: 4.1 scrub starts 2024-09-16T09:43:19.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:18 smithi084 ceph-mon[32629]: 4.1 scrub ok 2024-09-16T09:43:19.163 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:43:19.193 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:19.194 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:19.194 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:43:19.203 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:43:19.207 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:43:19.295 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:43:19.337 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:19.337 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:19.338 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:43:19.439 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:43:19.470 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:19.470 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:19.470 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:43:19.571 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:43:19.608 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:19.609 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:19.609 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:43:19.710 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:43:19.745 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:19.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:19.746 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:43:19.847 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:43:19.884 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:19.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:19.885 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:43:19.986 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:43:20.009 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T09:43:20.009 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:43:20.009 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:19 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:43:20.009 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:19 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:43:20.009 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:19 smithi070 ceph-mon[30388]: Health check update: Degraded data redundancy: 37105/298671 objects degraded (12.423%), 18 pgs degraded (PG_DEGRADED) 2024-09-16T09:43:20.010 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:19 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:43:20.010 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:19 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:43:20.010 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:19 smithi070 ceph-mon[40691]: Health check update: Degraded data redundancy: 37105/298671 objects degraded (12.423%), 18 pgs degraded (PG_DEGRADED) 2024-09-16T09:43:20.018 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:20.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:20.019 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:43:20.120 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:43:20.157 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:20.158 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:20.158 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:43:20.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:19 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:43:20.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:19 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:43:20.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:19 smithi084 ceph-mon[32629]: Health check update: Degraded data redundancy: 37105/298671 objects degraded (12.423%), 18 pgs degraded (PG_DEGRADED) 2024-09-16T09:43:20.258 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:43:20.300 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:20.301 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:20.301 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:43:20.402 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:43:20.448 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:20.449 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:20.449 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:43:20.550 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:43:20.592 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:20.593 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:20.593 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:43:20.694 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:43:20.735 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:20.735 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:20.736 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:43:20.830 INFO:teuthology.orchestra.run.smithi070.stderr:nodeep-scrub is unset 2024-09-16T09:43:20.836 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:43:20.865 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:20.866 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:20.866 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:43:20.942 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T09:43:20.967 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:43:21.022 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:21.023 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:21.023 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:43:21.124 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:43:21.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:20 smithi084 ceph-mon[32629]: pgmap v963: 168 pgs: 31 active+undersized, 18 active+undersized+degraded, 119 active+clean; 5.7 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.54k op/s; 35158/281526 objects degraded (12.488%) 2024-09-16T09:43:21.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:20 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/823036153' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:43:21.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:20 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:43:21.170 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:20 smithi070 ceph-mon[30388]: pgmap v963: 168 pgs: 31 active+undersized, 18 active+undersized+degraded, 119 active+clean; 5.7 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.54k op/s; 35158/281526 objects degraded (12.488%) 2024-09-16T09:43:21.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:20 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/823036153' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:43:21.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:20 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:43:21.173 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:20 smithi070 ceph-mon[40691]: pgmap v963: 168 pgs: 31 active+undersized, 18 active+undersized+degraded, 119 active+clean; 5.7 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.54k op/s; 35158/281526 objects degraded (12.488%) 2024-09-16T09:43:21.173 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:20 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/823036153' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:43:21.173 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:20 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:43:21.177 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:21.177 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:21.177 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:43:21.278 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:43:21.303 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:21.304 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:21.304 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:43:21.383 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.1 dump_ops_in_flight 2024-09-16T09:43:21.405 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:43:21.437 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:21.438 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:21.438 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:43:21.539 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:43:21.567 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:21.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:21.568 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:43:21.669 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:43:21.697 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:21.698 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:21.698 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:43:21.799 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:43:21.825 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:21.826 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:21.826 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:43:21.923 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:43:21.927 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:43:21.957 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:21.958 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:21.958 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:43:21.983 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:43:22.059 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:43:22.074 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:21 smithi070 ceph-mon[30388]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-16T09:43:22.074 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:21 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-16T09:43:22.074 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:21 smithi070 ceph-mon[30388]: osdmap e500: 8 total, 7 up, 8 in 2024-09-16T09:43:22.074 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:21 smithi070 ceph-mon[30388]: 3.17 scrub starts 2024-09-16T09:43:22.074 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:21 smithi070 ceph-mon[30388]: 3.17 scrub ok 2024-09-16T09:43:22.075 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:21 smithi070 ceph-mon[40691]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-16T09:43:22.075 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:21 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-16T09:43:22.075 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:21 smithi070 ceph-mon[40691]: osdmap e500: 8 total, 7 up, 8 in 2024-09-16T09:43:22.075 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:21 smithi070 ceph-mon[40691]: 3.17 scrub starts 2024-09-16T09:43:22.075 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:21 smithi070 ceph-mon[40691]: 3.17 scrub ok 2024-09-16T09:43:22.075 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:43:21 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:43:21] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:43:22.112 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:22.113 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:22.113 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:43:22.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:21 smithi084 ceph-mon[32629]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-16T09:43:22.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:21 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-16T09:43:22.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:21 smithi084 ceph-mon[32629]: osdmap e500: 8 total, 7 up, 8 in 2024-09-16T09:43:22.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:21 smithi084 ceph-mon[32629]: 3.17 scrub starts 2024-09-16T09:43:22.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:21 smithi084 ceph-mon[32629]: 3.17 scrub ok 2024-09-16T09:43:22.213 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:43:22.253 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:22.253 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:22.254 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:43:22.354 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:43:22.381 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:22.382 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:22.382 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:43:22.483 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:43:22.523 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:22.523 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:22.524 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:43:22.624 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:43:22.664 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:22.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:22.665 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:43:22.766 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:43:22.800 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:22.801 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:22.802 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:43:22.902 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:43:22.934 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:22.934 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:22.935 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:43:23.036 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:43:23.079 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:23.080 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:23.080 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:43:23.092 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:22 smithi084 ceph-mon[32629]: 11.3 scrub starts 2024-09-16T09:43:23.093 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:22 smithi084 ceph-mon[32629]: pgmap v965: 168 pgs: 31 active+undersized, 18 active+undersized+degraded, 119 active+clean; 5.7 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 560 B/s rd, 0 B/s wr, 1.48k op/s; 35158/281526 objects degraded (12.488%) 2024-09-16T09:43:23.093 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:22 smithi084 ceph-mon[32629]: 4.e scrub starts 2024-09-16T09:43:23.093 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:22 smithi084 ceph-mon[32629]: 4.e scrub ok 2024-09-16T09:43:23.093 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:22 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:43:23.093 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:22 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:43:23.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:22 smithi070 ceph-mon[30388]: 11.3 scrub starts 2024-09-16T09:43:23.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:22 smithi070 ceph-mon[30388]: pgmap v965: 168 pgs: 31 active+undersized, 18 active+undersized+degraded, 119 active+clean; 5.7 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 560 B/s rd, 0 B/s wr, 1.48k op/s; 35158/281526 objects degraded (12.488%) 2024-09-16T09:43:23.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:22 smithi070 ceph-mon[30388]: 4.e scrub starts 2024-09-16T09:43:23.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:22 smithi070 ceph-mon[30388]: 4.e scrub ok 2024-09-16T09:43:23.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:22 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:43:23.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:22 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:43:23.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:22 smithi070 ceph-mon[40691]: 11.3 scrub starts 2024-09-16T09:43:23.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:22 smithi070 ceph-mon[40691]: pgmap v965: 168 pgs: 31 active+undersized, 18 active+undersized+degraded, 119 active+clean; 5.7 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 560 B/s rd, 0 B/s wr, 1.48k op/s; 35158/281526 objects degraded (12.488%) 2024-09-16T09:43:23.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:22 smithi070 ceph-mon[40691]: 4.e scrub starts 2024-09-16T09:43:23.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:22 smithi070 ceph-mon[40691]: 4.e scrub ok 2024-09-16T09:43:23.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:22 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:43:23.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:22 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:43:23.181 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:43:23.219 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:23.219 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:23.220 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:43:23.320 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:43:23.353 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:23.354 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:23.354 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:43:23.455 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:43:23.472 INFO:teuthology.orchestra.run.smithi070.stdout:6.f 2024-09-16T09:43:23.472 INFO:teuthology.orchestra.run.smithi070.stdout:6.e 2024-09-16T09:43:23.472 INFO:teuthology.orchestra.run.smithi070.stdout:6.9 2024-09-16T09:43:23.472 INFO:teuthology.orchestra.run.smithi070.stdout:6.3 2024-09-16T09:43:23.472 INFO:teuthology.orchestra.run.smithi070.stdout:6.1c 2024-09-16T09:43:23.472 INFO:teuthology.orchestra.run.smithi070.stdout:6.1 2024-09-16T09:43:23.473 INFO:teuthology.orchestra.run.smithi070.stdout:6.17 2024-09-16T09:43:23.473 INFO:teuthology.orchestra.run.smithi070.stdout:6.12 2024-09-16T09:43:23.473 INFO:teuthology.orchestra.run.smithi070.stdout:6.14 2024-09-16T09:43:23.473 INFO:teuthology.orchestra.run.smithi070.stdout:6.0 2024-09-16T09:43:23.473 INFO:teuthology.orchestra.run.smithi070.stdout:5.d 2024-09-16T09:43:23.473 INFO:teuthology.orchestra.run.smithi070.stdout:5.b 2024-09-16T09:43:23.473 INFO:teuthology.orchestra.run.smithi070.stdout:5.8 2024-09-16T09:43:23.473 INFO:teuthology.orchestra.run.smithi070.stdout:5.6 2024-09-16T09:43:23.473 INFO:teuthology.orchestra.run.smithi070.stdout:5.1e 2024-09-16T09:43:23.473 INFO:teuthology.orchestra.run.smithi070.stdout:5.1c 2024-09-16T09:43:23.473 INFO:teuthology.orchestra.run.smithi070.stdout:3.13 2024-09-16T09:43:23.473 INFO:teuthology.orchestra.run.smithi070.stdout:11.2 2024-09-16T09:43:23.473 INFO:teuthology.orchestra.run.smithi070.stdout:11.4 2024-09-16T09:43:23.473 INFO:teuthology.orchestra.run.smithi070.stdout:6.2 2024-09-16T09:43:23.474 INFO:teuthology.orchestra.run.smithi070.stdout:11.d 2024-09-16T09:43:23.474 INFO:teuthology.orchestra.run.smithi070.stdout:4.4 2024-09-16T09:43:23.474 INFO:teuthology.orchestra.run.smithi070.stdout:2.1 2024-09-16T09:43:23.474 INFO:teuthology.orchestra.run.smithi070.stdout:3.0 2024-09-16T09:43:23.474 INFO:teuthology.orchestra.run.smithi070.stdout:1.9 2024-09-16T09:43:23.474 INFO:teuthology.orchestra.run.smithi070.stdout:1.a 2024-09-16T09:43:23.474 INFO:teuthology.orchestra.run.smithi070.stdout:11.c 2024-09-16T09:43:23.474 INFO:teuthology.orchestra.run.smithi070.stdout:1.5 2024-09-16T09:43:23.474 INFO:teuthology.orchestra.run.smithi070.stdout:1.1 2024-09-16T09:43:23.474 INFO:teuthology.orchestra.run.smithi070.stdout:11.f 2024-09-16T09:43:23.474 INFO:teuthology.orchestra.run.smithi070.stdout:3.1f 2024-09-16T09:43:23.474 INFO:teuthology.orchestra.run.smithi070.stdout:11.6 2024-09-16T09:43:23.475 INFO:teuthology.orchestra.run.smithi070.stdout:5.a 2024-09-16T09:43:23.475 INFO:teuthology.orchestra.run.smithi070.stdout:1.14 2024-09-16T09:43:23.475 INFO:teuthology.orchestra.run.smithi070.stdout:3.1a 2024-09-16T09:43:23.475 INFO:teuthology.orchestra.run.smithi070.stdout:6.8 2024-09-16T09:43:23.475 INFO:teuthology.orchestra.run.smithi070.stdout:4.d 2024-09-16T09:43:23.475 INFO:teuthology.orchestra.run.smithi070.stdout:3.1e 2024-09-16T09:43:23.475 INFO:teuthology.orchestra.run.smithi070.stdout:3.5 2024-09-16T09:43:23.475 INFO:teuthology.orchestra.run.smithi070.stdout:4.1c 2024-09-16T09:43:23.475 INFO:teuthology.orchestra.run.smithi070.stdout:3.4 2024-09-16T09:43:23.475 INFO:teuthology.orchestra.run.smithi070.stdout:4.f 2024-09-16T09:43:23.475 INFO:teuthology.orchestra.run.smithi070.stdout:4.12 2024-09-16T09:43:23.475 INFO:teuthology.orchestra.run.smithi070.stdout:3.9 2024-09-16T09:43:23.475 INFO:teuthology.orchestra.run.smithi070.stdout:4.17 2024-09-16T09:43:23.476 INFO:teuthology.orchestra.run.smithi070.stdout:5.4 2024-09-16T09:43:23.476 INFO:teuthology.orchestra.run.smithi070.stdout:4.6 2024-09-16T09:43:23.476 INFO:teuthology.orchestra.run.smithi070.stdout:1.8 2024-09-16T09:43:23.476 INFO:teuthology.orchestra.run.smithi070.stdout:5.14 2024-09-16T09:43:23.501 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:23.501 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:23.502 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:43:23.602 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:43:23.636 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:23.637 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:23.637 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:43:23.738 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:43:23.772 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:23.773 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:23.773 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:43:23.853 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd set noscrub 2024-09-16T09:43:23.874 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:43:23.920 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:23.921 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:23.921 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:43:24.022 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:43:24.063 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:24.064 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:24.064 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:43:24.149 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:43:23 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:43:24.149 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:23 smithi084 ceph-mon[32629]: osdmap e501: 8 total, 7 up, 8 in 2024-09-16T09:43:24.149 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:23 smithi084 ceph-mon[32629]: 11.3 scrub ok 2024-09-16T09:43:24.149 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:23 smithi084 ceph-mon[32629]: 3.1d scrub starts 2024-09-16T09:43:24.150 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:23 smithi084 ceph-mon[32629]: 3.1d scrub ok 2024-09-16T09:43:24.165 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:43:24.170 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:23 smithi070 ceph-mon[40691]: osdmap e501: 8 total, 7 up, 8 in 2024-09-16T09:43:24.170 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:23 smithi070 ceph-mon[40691]: 11.3 scrub ok 2024-09-16T09:43:24.170 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:23 smithi070 ceph-mon[40691]: 3.1d scrub starts 2024-09-16T09:43:24.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:23 smithi070 ceph-mon[40691]: 3.1d scrub ok 2024-09-16T09:43:24.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:23 smithi070 ceph-mon[30388]: osdmap e501: 8 total, 7 up, 8 in 2024-09-16T09:43:24.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:23 smithi070 ceph-mon[30388]: 11.3 scrub ok 2024-09-16T09:43:24.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:23 smithi070 ceph-mon[30388]: 3.1d scrub starts 2024-09-16T09:43:24.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:23 smithi070 ceph-mon[30388]: 3.1d scrub ok 2024-09-16T09:43:24.214 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:24.214 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:24.215 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:43:24.315 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:43:24.346 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:24.347 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:24.347 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:43:24.448 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:43:24.495 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:24.496 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:24.497 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:43:24.598 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:43:24.628 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:24.629 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:24.629 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:43:24.730 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:43:24.764 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:24.765 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:24.765 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:43:24.866 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:43:24.903 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:24.904 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:24.904 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:43:24.988 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell -n osd.2 --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-2 --journal-path /var/lib/ceph/osd/ceph-2/journal --op export --pgid 6.f --file /var/log/ceph/exp.6.f.2 2024-09-16T09:43:25.005 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:43:25.040 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:25.041 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:25.041 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:43:25.132 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:24 smithi070 ceph-mon[30388]: 11.e scrub starts 2024-09-16T09:43:25.132 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:24 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:43:25.132 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:24 smithi070 ceph-mon[30388]: pgmap v967: 168 pgs: 2 active+clean+scrubbing, 31 active+undersized, 18 active+undersized+degraded, 117 active+clean; 5.3 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.74k op/s; 33048/262650 objects degraded (12.583%) 2024-09-16T09:43:25.133 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:24 smithi070 ceph-mon[30388]: Health check update: Degraded data redundancy: 35158/281526 objects degraded (12.488%), 18 pgs degraded (PG_DEGRADED) 2024-09-16T09:43:25.133 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:24 smithi070 ceph-mon[40691]: 11.e scrub starts 2024-09-16T09:43:25.133 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:24 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:43:25.134 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:24 smithi070 ceph-mon[40691]: pgmap v967: 168 pgs: 2 active+clean+scrubbing, 31 active+undersized, 18 active+undersized+degraded, 117 active+clean; 5.3 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.74k op/s; 33048/262650 objects degraded (12.583%) 2024-09-16T09:43:25.134 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:24 smithi070 ceph-mon[40691]: Health check update: Degraded data redundancy: 35158/281526 objects degraded (12.488%), 18 pgs degraded (PG_DEGRADED) 2024-09-16T09:43:25.142 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:43:25.163 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:24 smithi084 ceph-mon[32629]: 11.e scrub starts 2024-09-16T09:43:25.163 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:24 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:43:25.163 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:24 smithi084 ceph-mon[32629]: pgmap v967: 168 pgs: 2 active+clean+scrubbing, 31 active+undersized, 18 active+undersized+degraded, 117 active+clean; 5.3 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.74k op/s; 33048/262650 objects degraded (12.583%) 2024-09-16T09:43:25.163 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:24 smithi084 ceph-mon[32629]: Health check update: Degraded data redundancy: 35158/281526 objects degraded (12.488%), 18 pgs degraded (PG_DEGRADED) 2024-09-16T09:43:25.181 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:25.181 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:25.182 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:43:25.282 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:43:25.291 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/osd.2/config 2024-09-16T09:43:25.307 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:43:25.307 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:43:25.338 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:25.339 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:25.339 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:43:25.341 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:43:25.342 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:43:25.343 INFO:teuthology.orchestra.run.smithi070.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-16T09:43:25.377 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:43:25.377 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:43:25.408 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:43:25.408 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:43:25.436 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:43:25.436 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:43:25.441 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:43:25.473 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:43:25.473 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:43:25.479 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:25.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:25.480 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:43:25.513 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:43:25.513 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:43:25.582 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:43:25.614 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:25.614 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:25.614 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:43:25.715 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:43:25.747 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:25.747 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:25.748 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:43:25.848 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:43:25.895 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:25.896 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:25.896 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:43:25.997 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:43:26.028 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:26.028 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:26.029 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:43:26.039 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:25 smithi070 ceph-mon[30388]: 11.e scrub ok 2024-09-16T09:43:26.040 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:25 smithi070 ceph-mon[30388]: 1.f scrub starts 2024-09-16T09:43:26.040 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:25 smithi070 ceph-mon[30388]: 1.f scrub ok 2024-09-16T09:43:26.040 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:25 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:43:26.040 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:25 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:43:26.040 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:25 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/4066186604' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:43:26.041 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:25 smithi070 ceph-mon[40691]: 11.e scrub ok 2024-09-16T09:43:26.041 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:25 smithi070 ceph-mon[40691]: 1.f scrub starts 2024-09-16T09:43:26.041 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:25 smithi070 ceph-mon[40691]: 1.f scrub ok 2024-09-16T09:43:26.041 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:25 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:43:26.041 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:25 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:43:26.041 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:25 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/4066186604' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:43:26.130 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:43:26.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:25 smithi084 ceph-mon[32629]: 11.e scrub ok 2024-09-16T09:43:26.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:25 smithi084 ceph-mon[32629]: 1.f scrub starts 2024-09-16T09:43:26.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:25 smithi084 ceph-mon[32629]: 1.f scrub ok 2024-09-16T09:43:26.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:25 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:43:26.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:25 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:43:26.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:25 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/4066186604' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:43:26.163 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:26.164 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:26.164 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:43:26.265 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:43:26.297 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:26.298 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:26.298 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:43:26.399 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:43:26.432 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:26.433 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:26.433 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:43:26.534 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:43:26.568 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:26.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:26.569 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:43:26.670 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:43:26.704 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:26.705 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:26.705 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:43:26.806 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:43:26.842 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:26.843 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:26.843 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:43:26.943 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:43:26.977 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:26.978 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:26.978 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:43:27.028 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T09:43:27.028 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T09:43:27.079 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:43:27.112 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:27.113 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:27.113 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:43:27.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:26 smithi084 ceph-mon[32629]: 5.16 scrub starts 2024-09-16T09:43:27.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:26 smithi084 ceph-mon[32629]: 5.16 scrub ok 2024-09-16T09:43:27.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:26 smithi084 ceph-mon[32629]: 1.c scrub starts 2024-09-16T09:43:27.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:26 smithi084 ceph-mon[32629]: 1.c scrub ok 2024-09-16T09:43:27.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:26 smithi084 ceph-mon[32629]: osdmap e502: 8 total, 7 up, 8 in 2024-09-16T09:43:27.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:26 smithi084 ceph-mon[32629]: pgmap v969: 168 pgs: 2 active+clean+scrubbing, 31 active+undersized, 18 active+undersized+degraded, 117 active+clean; 5.3 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 1.12k op/s; 32637/261417 objects degraded (12.485%) 2024-09-16T09:43:27.170 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:26 smithi070 ceph-mon[30388]: 5.16 scrub starts 2024-09-16T09:43:27.170 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:26 smithi070 ceph-mon[30388]: 5.16 scrub ok 2024-09-16T09:43:27.170 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:26 smithi070 ceph-mon[30388]: 1.c scrub starts 2024-09-16T09:43:27.170 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:26 smithi070 ceph-mon[30388]: 1.c scrub ok 2024-09-16T09:43:27.170 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:26 smithi070 ceph-mon[30388]: osdmap e502: 8 total, 7 up, 8 in 2024-09-16T09:43:27.170 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:26 smithi070 ceph-mon[30388]: pgmap v969: 168 pgs: 2 active+clean+scrubbing, 31 active+undersized, 18 active+undersized+degraded, 117 active+clean; 5.3 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 1.12k op/s; 32637/261417 objects degraded (12.485%) 2024-09-16T09:43:27.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:26 smithi070 ceph-mon[40691]: 5.16 scrub starts 2024-09-16T09:43:27.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:26 smithi070 ceph-mon[40691]: 5.16 scrub ok 2024-09-16T09:43:27.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:26 smithi070 ceph-mon[40691]: 1.c scrub starts 2024-09-16T09:43:27.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:26 smithi070 ceph-mon[40691]: 1.c scrub ok 2024-09-16T09:43:27.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:26 smithi070 ceph-mon[40691]: osdmap e502: 8 total, 7 up, 8 in 2024-09-16T09:43:27.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:26 smithi070 ceph-mon[40691]: pgmap v969: 168 pgs: 2 active+clean+scrubbing, 31 active+undersized, 18 active+undersized+degraded, 117 active+clean; 5.3 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 1.12k op/s; 32637/261417 objects degraded (12.485%) 2024-09-16T09:43:27.214 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:43:27.245 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:27.246 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:27.247 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:43:27.347 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:43:27.383 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:27.383 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:27.384 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:43:27.484 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:43:27.496 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T09:43:27.497 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [ 2024-09-16T09:43:27.497 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:27.497 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.16305.0:180317 11.7 e502/454 11:eedcf357:::benchmark_data_smithi070_191495_object49847:head v 502'11357, mlcod=502'11355)", 2024-09-16T09:43:27.497 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:43:27.482819+0000", 2024-09-16T09:43:27.497 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.011009250999999999, 2024-09-16T09:43:27.497 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.011033685, 2024-09-16T09:43:27.497 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:43:27.497 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:43:27.497 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T09:43:27.497 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:43:27.497 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:27.497 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:43:27.497 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:27.482819+0000", 2024-09-16T09:43:27.497 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:43:27.498 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:27.498 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:27.498 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:43:27.498 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:27.482819+0000", 2024-09-16T09:43:27.498 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:43:27.498 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:27.498 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:27.498 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:43:27.498 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:27.482820+0000", 2024-09-16T09:43:27.498 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.8599999999999997e-07 2024-09-16T09:43:27.498 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:27.498 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:27.498 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:43:27.498 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:27.482825+0000", 2024-09-16T09:43:27.498 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4519999999999996e-06 2024-09-16T09:43:27.499 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:27.499 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:27.499 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:43:27.499 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:27.482834+0000", 2024-09-16T09:43:27.499 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.5629999999999992e-06 2024-09-16T09:43:27.499 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:27.499 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:27.499 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:43:27.499 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:27.482840+0000", 2024-09-16T09:43:27.499 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.0569999999999998e-06 2024-09-16T09:43:27.499 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:27.500 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:27.500 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:43:27.500 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:27.482989+0000", 2024-09-16T09:43:27.500 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00014871200000000001 2024-09-16T09:43:27.500 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:27.500 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:27.500 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:43:27.500 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:27.483031+0000", 2024-09-16T09:43:27.500 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2259000000000003e-05 2024-09-16T09:43:27.500 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:43:27.500 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:43:27.500 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:43:27.500 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:27.500 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:27.501 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.16305.0:180316 11.0 e502/454 11:04dc71b9:::benchmark_data_smithi070_191495_object49846:head v 502'11398, mlcod=502'11394)", 2024-09-16T09:43:27.501 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:43:27.484303+0000", 2024-09-16T09:43:27.501 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.0095251060000000002, 2024-09-16T09:43:27.501 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0096022539999999993, 2024-09-16T09:43:27.501 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:43:27.501 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:43:27.501 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T09:43:27.501 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:43:27.501 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:27.501 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:43:27.501 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:27.484303+0000", 2024-09-16T09:43:27.501 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:43:27.501 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:27.502 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:27.502 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:43:27.502 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:27.484303+0000", 2024-09-16T09:43:27.502 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:43:27.502 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:27.502 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:27.502 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:43:27.502 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:27.484304+0000", 2024-09-16T09:43:27.502 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0500000000000004e-07 2024-09-16T09:43:27.502 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:27.502 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:27.502 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:43:27.502 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:27.484308+0000", 2024-09-16T09:43:27.502 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.7049999999999998e-06 2024-09-16T09:43:27.502 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:27.503 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:27.503 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:43:27.503 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:27.484316+0000", 2024-09-16T09:43:27.503 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.9929999999999995e-06 2024-09-16T09:43:27.503 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:27.503 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:27.503 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:43:27.503 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:27.484321+0000", 2024-09-16T09:43:27.503 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.04e-06 2024-09-16T09:43:27.503 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:27.503 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:27.503 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:43:27.503 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:27.484632+0000", 2024-09-16T09:43:27.504 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00031042100000000002 2024-09-16T09:43:27.504 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:27.504 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:27.504 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:43:27.504 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:27.484749+0000", 2024-09-16T09:43:27.504 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000117255 2024-09-16T09:43:27.504 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:43:27.504 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:43:27.504 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:43:27.504 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:27.504 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:27.504 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.16305.0:180321 11.f e502/495 11:f252584c:::benchmark_data_smithi070_191495_object49851:head v 502'11252, mlcod=502'11250)", 2024-09-16T09:43:27.504 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:43:27.485660+0000", 2024-09-16T09:43:27.505 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.0081677409999999992, 2024-09-16T09:43:27.505 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0082918309999999995, 2024-09-16T09:43:27.505 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:43:27.505 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:43:27.505 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T09:43:27.505 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:43:27.505 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:27.505 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:43:27.505 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:27.485660+0000", 2024-09-16T09:43:27.505 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:43:27.506 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:27.506 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:27.506 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:43:27.506 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:27.485660+0000", 2024-09-16T09:43:27.506 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:43:27.506 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:27.506 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:27.506 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:43:27.506 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:27.485661+0000", 2024-09-16T09:43:27.506 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.8899999999999998e-07 2024-09-16T09:43:27.507 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:27.507 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:27.507 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:43:27.507 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:27.485667+0000", 2024-09-16T09:43:27.507 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.3489999999999999e-06 2024-09-16T09:43:27.507 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:27.507 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:27.507 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:43:27.507 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:27.485680+0000", 2024-09-16T09:43:27.507 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3254e-05 2024-09-16T09:43:27.507 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:27.507 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:27.507 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:43:27.507 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:27.485686+0000", 2024-09-16T09:43:27.508 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.4150000000000004e-06 2024-09-16T09:43:27.508 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:27.508 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:27.508 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:43:27.508 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:27.487316+0000", 2024-09-16T09:43:27.508 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0016301359999999999 2024-09-16T09:43:27.508 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:27.508 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:27.508 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:43:27.508 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:27.487337+0000", 2024-09-16T09:43:27.508 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0210000000000001e-05 2024-09-16T09:43:27.508 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:43:27.508 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:43:27.508 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:43:27.509 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:27.509 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:27.509 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.16305.0:180322 11.7 e502/454 11:eecbfb73:::benchmark_data_smithi070_191495_object49852:head v 502'11358, mlcod=502'11355)", 2024-09-16T09:43:27.509 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:43:27.486940+0000", 2024-09-16T09:43:27.509 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.0068882149999999996, 2024-09-16T09:43:27.509 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0070565799999999998, 2024-09-16T09:43:27.509 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:43:27.509 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:43:27.509 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T09:43:27.509 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:43:27.509 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:27.509 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:43:27.510 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:27.486940+0000", 2024-09-16T09:43:27.510 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:43:27.510 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:27.510 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:27.510 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:43:27.510 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:27.486940+0000", 2024-09-16T09:43:27.510 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:43:27.510 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:27.510 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:27.510 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:43:27.510 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:27.486941+0000", 2024-09-16T09:43:27.510 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.0100000000000004e-07 2024-09-16T09:43:27.510 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:27.511 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:27.511 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:43:27.511 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:27.486946+0000", 2024-09-16T09:43:27.511 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.6330000000000001e-06 2024-09-16T09:43:27.511 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:27.511 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:27.511 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:43:27.511 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:27.486961+0000", 2024-09-16T09:43:27.511 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.4192000000000001e-05 2024-09-16T09:43:27.511 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:27.511 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:27.511 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:43:27.511 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:27.486968+0000", 2024-09-16T09:43:27.511 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.9430000000000003e-06 2024-09-16T09:43:27.512 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:27.512 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:27.512 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:43:27.512 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:27.489052+0000", 2024-09-16T09:43:27.512 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0020844689999999998 2024-09-16T09:43:27.512 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:27.512 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:27.512 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:43:27.512 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:27.489075+0000", 2024-09-16T09:43:27.512 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.2827e-05 2024-09-16T09:43:27.513 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:43:27.513 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:43:27.513 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:43:27.513 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:27.513 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:27.513 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.16305.0:180327 11.e e502/454 11:7cbe8fd9:::benchmark_data_smithi070_191495_object49857:head v 502'11358, mlcod=502'11356)", 2024-09-16T09:43:27.513 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:43:27.489745+0000", 2024-09-16T09:43:27.513 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.0040836450000000003, 2024-09-16T09:43:27.513 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0042974210000000001, 2024-09-16T09:43:27.513 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:43:27.513 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:43:27.513 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T09:43:27.513 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:43:27.513 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:27.514 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:43:27.514 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:27.489745+0000", 2024-09-16T09:43:27.514 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:43:27.514 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:27.514 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:27.514 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:43:27.514 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:27.489745+0000", 2024-09-16T09:43:27.514 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:43:27.514 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:27.514 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:27.514 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:43:27.514 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:27.489745+0000", 2024-09-16T09:43:27.515 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.8199999999999998e-07 2024-09-16T09:43:27.515 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:27.515 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:27.515 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:43:27.515 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:27.489752+0000", 2024-09-16T09:43:27.515 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.674e-06 2024-09-16T09:43:27.515 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:27.515 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:27.515 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:43:27.515 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:27.489767+0000", 2024-09-16T09:43:27.515 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.4836e-05 2024-09-16T09:43:27.515 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:27.515 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:27.515 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:43:27.516 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:27.489774+0000", 2024-09-16T09:43:27.516 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.9460000000000001e-06 2024-09-16T09:43:27.516 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:27.516 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:27.516 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:43:27.516 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:27.489874+0000", 2024-09-16T09:43:27.516 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000100197 2024-09-16T09:43:27.516 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:27.516 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:27.516 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:43:27.516 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:27.489894+0000", 2024-09-16T09:43:27.516 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9890000000000001e-05 2024-09-16T09:43:27.516 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:43:27.516 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:43:27.517 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:43:27.517 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:27.517 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:27.517 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.16305.0:180326 11.6 e502/495 11:66fa9858:::benchmark_data_smithi070_191495_object49856:head v 502'11367, mlcod=502'11365)", 2024-09-16T09:43:27.517 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:43:27.489808+0000", 2024-09-16T09:43:27.517 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.0040200069999999999, 2024-09-16T09:43:27.517 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0042782749999999998, 2024-09-16T09:43:27.517 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:43:27.517 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:43:27.517 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T09:43:27.517 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:43:27.517 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:27.517 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:43:27.517 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:27.489808+0000", 2024-09-16T09:43:27.518 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:43:27.518 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:27.518 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:27.518 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:43:27.518 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:27.489808+0000", 2024-09-16T09:43:27.518 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:43:27.518 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:27.518 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:27.518 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:43:27.518 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:27.489809+0000", 2024-09-16T09:43:27.518 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0100000000000005e-07 2024-09-16T09:43:27.518 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:27.518 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:27.518 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:43:27.518 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:27.489813+0000", 2024-09-16T09:43:27.519 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2259999999999999e-06 2024-09-16T09:43:27.519 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:27.519 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:27.519 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:43:27.519 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:27.489821+0000", 2024-09-16T09:43:27.519 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6389999999999992e-06 2024-09-16T09:43:27.519 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:27.519 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:27.519 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:43:27.519 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:27.489825+0000", 2024-09-16T09:43:27.519 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.8250000000000004e-06 2024-09-16T09:43:27.519 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:27.519 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:27.519 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:43:27.520 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:27.492075+0000", 2024-09-16T09:43:27.520 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0022495549999999999 2024-09-16T09:43:27.520 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:27.520 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:27.520 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:43:27.520 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:27.492095+0000", 2024-09-16T09:43:27.520 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9941000000000001e-05 2024-09-16T09:43:27.520 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:43:27.520 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:43:27.520 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:43:27.520 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:27.521 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:27.521 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.16305.0:180318 11.0 e502/454 11:07102bb8:::benchmark_data_smithi070_191495_object49848:head v 502'11399, mlcod=502'11396)", 2024-09-16T09:43:27.521 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:43:27.490203+0000", 2024-09-16T09:43:27.521 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.003624997, 2024-09-16T09:43:27.521 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.003930177, 2024-09-16T09:43:27.521 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:43:27.521 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:43:27.521 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T09:43:27.521 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:43:27.521 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:27.521 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:43:27.521 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:27.490203+0000", 2024-09-16T09:43:27.521 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:43:27.521 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:27.522 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:27.522 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:43:27.522 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:27.490203+0000", 2024-09-16T09:43:27.522 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:43:27.522 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:27.522 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:27.522 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:43:27.522 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:27.490204+0000", 2024-09-16T09:43:27.522 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.1999999999999998e-07 2024-09-16T09:43:27.523 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:27.523 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:27.523 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:43:27.523 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:27.490210+0000", 2024-09-16T09:43:27.523 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8069999999999996e-06 2024-09-16T09:43:27.523 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:27.523 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:27.523 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:43:27.523 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:27.490225+0000", 2024-09-16T09:43:27.523 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.485e-05 2024-09-16T09:43:27.523 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:27.523 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:27.524 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:43:27.524 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:27.490231+0000", 2024-09-16T09:43:27.524 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.5699999999999998e-06 2024-09-16T09:43:27.524 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:27.524 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:27.524 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:43:27.524 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:27.492489+0000", 2024-09-16T09:43:27.524 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0022572920000000002 2024-09-16T09:43:27.524 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:27.524 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:27.524 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:43:27.524 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:27.492510+0000", 2024-09-16T09:43:27.524 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.1154000000000001e-05 2024-09-16T09:43:27.525 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:43:27.525 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:43:27.525 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:43:27.525 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:27.525 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:27.525 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.16305.0:180330 11.6 e502/495)", 2024-09-16T09:43:27.525 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:43:27.493314+0000", 2024-09-16T09:43:27.525 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.00051404300000000005, 2024-09-16T09:43:27.525 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00086018800000000001, 2024-09-16T09:43:27.525 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:43:27.525 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:43:27.525 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T09:43:27.525 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:43:27.525 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:27.526 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:43:27.526 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:27.493314+0000", 2024-09-16T09:43:27.526 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:43:27.526 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:27.526 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:27.526 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:43:27.526 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:27.493314+0000", 2024-09-16T09:43:27.526 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:43:27.526 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:27.526 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:27.526 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:43:27.526 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:27.493315+0000", 2024-09-16T09:43:27.526 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.7199999999999998e-07 2024-09-16T09:43:27.526 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:27.527 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:27.527 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:43:27.527 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:27.493321+0000", 2024-09-16T09:43:27.527 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.0100000000000001e-06 2024-09-16T09:43:27.527 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:27.527 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:27.527 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:43:27.527 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:27.493332+0000", 2024-09-16T09:43:27.527 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0886e-05 2024-09-16T09:43:27.527 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:27.527 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:27.527 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:43:27.527 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:27.493338+0000", 2024-09-16T09:43:27.527 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.2779999999999999e-06 2024-09-16T09:43:27.528 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:43:27.528 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:43:27.528 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:43:27.528 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:27.528 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:27.528 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.16305.0:180329 11.7 e502/454 11:ea90b846:::benchmark_data_smithi070_191495_object49859:head v 502'11359, mlcod=502'11355)", 2024-09-16T09:43:27.528 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:43:27.493422+0000", 2024-09-16T09:43:27.528 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.00040634500000000001, 2024-09-16T09:43:27.528 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00079731400000000003, 2024-09-16T09:43:27.528 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:43:27.528 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:43:27.528 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T09:43:27.528 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:43:27.529 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:27.529 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:43:27.529 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:27.493422+0000", 2024-09-16T09:43:27.529 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:43:27.529 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:27.529 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:27.529 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:43:27.529 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:27.493422+0000", 2024-09-16T09:43:27.529 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:43:27.529 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:27.529 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:27.529 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:43:27.529 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:27.493422+0000", 2024-09-16T09:43:27.529 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.75e-07 2024-09-16T09:43:27.530 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:27.530 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:27.530 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:43:27.530 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:27.493427+0000", 2024-09-16T09:43:27.530 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2540000000000003e-06 2024-09-16T09:43:27.530 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:27.530 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:27.530 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:43:27.530 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:27.493437+0000", 2024-09-16T09:43:27.530 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0081000000000001e-05 2024-09-16T09:43:27.530 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:27.530 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:27.531 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:43:27.531 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:27.493442+0000", 2024-09-16T09:43:27.531 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7189999999999998e-06 2024-09-16T09:43:27.531 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:27.531 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:27.531 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:43:27.531 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:27.493511+0000", 2024-09-16T09:43:27.531 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.8065000000000006e-05 2024-09-16T09:43:27.531 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:27.531 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:27.531 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:43:27.531 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:27.493527+0000", 2024-09-16T09:43:27.531 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6263e-05 2024-09-16T09:43:27.532 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:43:27.532 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:43:27.532 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:43:27.532 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:43:27.532 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-16T09:43:27.532 INFO:teuthology.orchestra.run.smithi070.stdout: "num_ops": 9 2024-09-16T09:43:27.532 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T09:43:27.533 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:27.533 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:27.533 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:43:27.634 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:43:27.670 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:27.671 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:27.671 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:43:27.772 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:43:27.807 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:27.808 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:27.808 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:43:27.909 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:43:27.941 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:27.942 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:27.942 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:43:27.949 INFO:teuthology.orchestra.run.smithi070.stderr:Creating an OSD daemon form without an OSD FSID value 2024-09-16T09:43:28.043 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:43:28.064 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:28.065 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:28.065 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:43:28.166 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:43:28.198 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:28.199 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:28.199 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:43:28.221 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:43:28.300 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:43:28.339 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:28.339 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:28.340 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:43:28.441 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:43:28.477 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:28.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:28.481 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:43:28.581 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:43:28.615 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:28.615 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:28.616 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:43:28.717 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:43:28.749 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:28.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:28.750 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:43:28.851 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:43:28.887 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:28.888 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:28.889 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:43:28.989 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:43:29.028 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:29.028 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T09:43:29.030 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.1 dump_blocked_ops 2024-09-16T09:43:29.030 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:29.031 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:43:29.132 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:43:29.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:28 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:43:29.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:28 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:43:29.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:28 smithi084 ceph-mon[32629]: osdmap e503: 8 total, 7 up, 8 in 2024-09-16T09:43:29.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:28 smithi084 ceph-mon[32629]: pgmap v971: 168 pgs: 2 active+clean+scrubbing, 31 active+undersized, 18 active+undersized+degraded, 117 active+clean; 5.2 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.42k op/s; 31773/256014 objects degraded (12.411%) 2024-09-16T09:43:29.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:28 smithi084 ceph-mon[32629]: 6.18 scrub starts 2024-09-16T09:43:29.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:28 smithi084 ceph-mon[32629]: 6.18 scrub ok 2024-09-16T09:43:29.164 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:29.164 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:29.164 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:43:29.170 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:28 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:43:29.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:28 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:43:29.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:28 smithi070 ceph-mon[30388]: osdmap e503: 8 total, 7 up, 8 in 2024-09-16T09:43:29.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:28 smithi070 ceph-mon[30388]: pgmap v971: 168 pgs: 2 active+clean+scrubbing, 31 active+undersized, 18 active+undersized+degraded, 117 active+clean; 5.2 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.42k op/s; 31773/256014 objects degraded (12.411%) 2024-09-16T09:43:29.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:28 smithi070 ceph-mon[30388]: 6.18 scrub starts 2024-09-16T09:43:29.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:28 smithi070 ceph-mon[30388]: 6.18 scrub ok 2024-09-16T09:43:29.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:28 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:43:29.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:28 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:43:29.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:28 smithi070 ceph-mon[40691]: osdmap e503: 8 total, 7 up, 8 in 2024-09-16T09:43:29.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:28 smithi070 ceph-mon[40691]: pgmap v971: 168 pgs: 2 active+clean+scrubbing, 31 active+undersized, 18 active+undersized+degraded, 117 active+clean; 5.2 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.42k op/s; 31773/256014 objects degraded (12.411%) 2024-09-16T09:43:29.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:28 smithi070 ceph-mon[40691]: 6.18 scrub starts 2024-09-16T09:43:29.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:28 smithi070 ceph-mon[40691]: 6.18 scrub ok 2024-09-16T09:43:29.265 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:43:29.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:29.296 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:29.296 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:43:29.397 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:43:29.435 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:29.436 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:29.436 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:43:29.537 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:43:29.573 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:29.574 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:29.574 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:43:29.675 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:43:29.718 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:29.719 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:29.719 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:43:29.820 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:43:29.856 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:29.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:29.857 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:43:29.958 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:43:29.997 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:29.998 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:29.998 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:43:30.099 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:43:30.137 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:30.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:30.138 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:43:30.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:29 smithi084 ceph-mon[32629]: Health check update: Degraded data redundancy: 31773/256014 objects degraded (12.411%), 18 pgs degraded (PG_DEGRADED) 2024-09-16T09:43:30.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:29 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/876960277' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:43:30.160 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:29 smithi070 ceph-mon[30388]: Health check update: Degraded data redundancy: 31773/256014 objects degraded (12.411%), 18 pgs degraded (PG_DEGRADED) 2024-09-16T09:43:30.160 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:29 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/876960277' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:43:30.161 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:29 smithi070 ceph-mon[40691]: Health check update: Degraded data redundancy: 31773/256014 objects degraded (12.411%), 18 pgs degraded (PG_DEGRADED) 2024-09-16T09:43:30.161 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:29 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/876960277' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:43:30.239 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:43:30.269 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:30.270 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:30.270 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:43:30.371 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:43:30.391 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:30.392 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:30.392 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:43:30.463 INFO:teuthology.orchestra.run.smithi070.stderr:noscrub is set 2024-09-16T09:43:30.493 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:43:30.514 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:30.515 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:30.515 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:43:30.616 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:43:30.648 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:30.649 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:30.649 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:43:30.750 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:43:30.787 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:30.788 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:30.788 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:43:30.889 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:43:30.972 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:30.973 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:30.973 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:43:31.074 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:43:31.106 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:31.106 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:31.107 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:43:31.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:30 smithi084 ceph-mon[32629]: pgmap v972: 168 pgs: 31 active+undersized, 18 active+undersized+degraded, 119 active+clean; 4.9 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 725 B/s rd, 0 B/s wr, 1.75k op/s; 29897/239205 objects degraded (12.498%) 2024-09-16T09:43:31.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:30 smithi084 ceph-mon[32629]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:43:31.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:30 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/876960277' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-16T09:43:31.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:30 smithi084 ceph-mon[32629]: osdmap e504: 8 total, 7 up, 8 in 2024-09-16T09:43:31.170 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:30 smithi070 ceph-mon[30388]: pgmap v972: 168 pgs: 31 active+undersized, 18 active+undersized+degraded, 119 active+clean; 4.9 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 725 B/s rd, 0 B/s wr, 1.75k op/s; 29897/239205 objects degraded (12.498%) 2024-09-16T09:43:31.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:30 smithi070 ceph-mon[30388]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:43:31.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:30 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/876960277' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-16T09:43:31.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:30 smithi070 ceph-mon[30388]: osdmap e504: 8 total, 7 up, 8 in 2024-09-16T09:43:31.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:30 smithi070 ceph-mon[40691]: pgmap v972: 168 pgs: 31 active+undersized, 18 active+undersized+degraded, 119 active+clean; 4.9 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 725 B/s rd, 0 B/s wr, 1.75k op/s; 29897/239205 objects degraded (12.498%) 2024-09-16T09:43:31.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:30 smithi070 ceph-mon[40691]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:43:31.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:30 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/876960277' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-16T09:43:31.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:30 smithi070 ceph-mon[40691]: osdmap e504: 8 total, 7 up, 8 in 2024-09-16T09:43:31.208 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:43:31.240 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:31.241 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:31.242 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:43:31.344 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:43:31.378 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:31.379 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:31.379 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:43:31.480 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:43:31.508 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:31.508 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:31.509 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:43:31.610 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:43:31.646 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:31.647 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:31.648 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:43:31.748 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:43:31.787 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:31.787 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:31.787 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:43:31.888 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:43:31.924 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:31.925 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:31.925 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:43:32.026 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:43:32.061 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:32.061 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:32.062 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:43:32.164 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:43:32.172 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:43:31 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:43:31] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:43:32.191 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:32.192 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:32.193 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:43:32.293 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:43:32.325 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:32.326 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:32.326 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:43:32.427 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:43:32.461 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:32.462 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:32.462 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:43:32.563 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:43:32.599 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:32.599 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:32.600 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:43:32.700 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:43:32.732 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:32.732 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:32.733 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:43:32.804 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:43:32.833 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:43:32.876 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:32.877 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:32.877 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:43:32.962 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:32 smithi070 ceph-mon[30388]: pgmap v974: 168 pgs: 31 active+undersized, 18 active+undersized+degraded, 119 active+clean; 4.9 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 847 B/s rd, 0 B/s wr, 1.23k op/s; 29897/239205 objects degraded (12.498%) 2024-09-16T09:43:32.962 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:32 smithi070 ceph-mon[40691]: pgmap v974: 168 pgs: 31 active+undersized, 18 active+undersized+degraded, 119 active+clean; 4.9 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 847 B/s rd, 0 B/s wr, 1.23k op/s; 29897/239205 objects degraded (12.498%) 2024-09-16T09:43:32.977 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:43:33.007 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:33.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:33.008 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:43:33.109 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:43:33.141 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:33.142 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:33.142 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:43:33.243 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:43:33.274 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:33.275 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:33.275 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:43:33.376 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:43:33.405 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:43:33.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:32 smithi084 ceph-mon[32629]: pgmap v974: 168 pgs: 31 active+undersized, 18 active+undersized+degraded, 119 active+clean; 4.9 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 847 B/s rd, 0 B/s wr, 1.23k op/s; 29897/239205 objects degraded (12.498%) 2024-09-16T09:43:33.419 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:33.420 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:33.420 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:43:33.449 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T09:43:33.449 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T09:43:33.449 INFO:teuthology.orchestra.run.smithi070.stdout: "complaint_time": 30, 2024-09-16T09:43:33.449 INFO:teuthology.orchestra.run.smithi070.stdout: "num_blocked_ops": 0 2024-09-16T09:43:33.449 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T09:43:33.521 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:43:33.572 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:33.572 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:33.572 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:43:33.673 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:43:33.709 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:33.712 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:33.712 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:43:33.813 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:43:33.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:33.846 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:33.847 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:43:33.947 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:43:33.953 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:43:33 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:43:33.953 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:33 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:43:33.983 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:33.984 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:33.984 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:43:34.085 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:43:34.115 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:34.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:34.116 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:43:34.217 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:43:34.255 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:34.256 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:34.256 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:43:34.301 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:33 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:43:34.301 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:33 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:43:34.357 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:43:34.394 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:34.395 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:34.395 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:43:34.496 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:43:34.497 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:43:34.497 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:43:34.497 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:43:34.497 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:43:34.498 INFO:teuthology.orchestra.run.smithi070.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-16T09:43:34.498 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:43:34.498 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:43:34.498 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:43:34.498 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:43:34.498 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:43:34.498 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:43:34.498 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:43:34.498 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:43:34.519 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:43:34.519 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:43:34.536 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:34.537 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:34.538 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:43:34.638 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:43:34.673 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:34.674 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:34.674 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:43:34.766 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-16T09:43:34.775 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:43:34.854 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:34.855 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:34.855 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:43:34.956 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:43:34.991 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:34.992 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:34.992 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:43:35.093 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:43:35.131 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:35.131 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:35.131 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:43:35.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:34 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:43:35.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:35 smithi070 ceph-mon[40691]: pgmap v975: 168 pgs: 31 active+undersized, 18 active+undersized+degraded, 119 active+clean; 4.4 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.81k op/s; 27503/217989 objects degraded (12.617%) 2024-09-16T09:43:35.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:35 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/2774111035' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:43:35.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:35 smithi070 ceph-mon[40691]: Health check update: Degraded data redundancy: 27503/217989 objects degraded (12.617%), 18 pgs degraded (PG_DEGRADED) 2024-09-16T09:43:35.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:34 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:43:35.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:35 smithi070 ceph-mon[30388]: pgmap v975: 168 pgs: 31 active+undersized, 18 active+undersized+degraded, 119 active+clean; 4.4 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.81k op/s; 27503/217989 objects degraded (12.617%) 2024-09-16T09:43:35.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:35 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2774111035' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:43:35.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:35 smithi070 ceph-mon[30388]: Health check update: Degraded data redundancy: 27503/217989 objects degraded (12.617%), 18 pgs degraded (PG_DEGRADED) 2024-09-16T09:43:35.232 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:43:35.266 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:35.267 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:35.267 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:43:35.359 INFO:teuthology.orchestra.run.smithi070.stderr:Exporting 6.f info 6.f( empty local-lis/les=227/228 n=0 ec=64/64 lis/c=227/227 les/c/f=228/228/0 sis=227) 2024-09-16T09:43:35.369 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:43:35.380 INFO:teuthology.orchestra.run.smithi070.stderr:Export successful 2024-09-16T09:43:35.401 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:35.402 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:35.402 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:43:35.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:34 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:43:35.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:34 smithi084 ceph-mon[32629]: pgmap v975: 168 pgs: 31 active+undersized, 18 active+undersized+degraded, 119 active+clean; 4.4 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.81k op/s; 27503/217989 objects degraded (12.617%) 2024-09-16T09:43:35.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:34 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/2774111035' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:43:35.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:34 smithi084 ceph-mon[32629]: Health check update: Degraded data redundancy: 27503/217989 objects degraded (12.617%), 18 pgs degraded (PG_DEGRADED) 2024-09-16T09:43:35.503 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:43:35.536 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:35.536 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:35.537 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:43:35.638 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:43:35.678 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:35.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:35.679 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:43:35.780 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:43:35.801 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:35.801 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:35.802 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:43:35.902 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:43:35.934 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:35.935 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:35.935 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:43:36.036 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:43:36.068 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:36.069 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:36.069 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:43:36.170 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:43:36.201 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:36.202 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:36.202 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:43:36.280 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:35 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:43:36.280 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:35 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:43:36.280 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:35 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:43:36.280 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:35 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:43:36.303 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:43:36.329 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:36.330 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:36.330 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:43:36.335 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.1 dump_historic_ops 2024-09-16T09:43:36.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:35 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:43:36.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:35 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:43:36.431 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:43:36.466 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:36.467 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:36.467 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:43:36.568 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:43:36.602 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:36.603 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:36.603 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:43:36.704 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:43:36.743 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:36.743 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:36.744 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:43:36.839 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T09:43:36.840 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T09:43:36.844 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:43:36.884 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:36.885 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:36.885 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:43:36.986 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:43:37.025 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:37.025 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:37.026 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:43:37.127 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:43:37.150 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:36 smithi070 ceph-mon[30388]: pgmap v976: 168 pgs: 31 active+undersized, 18 active+undersized+degraded, 119 active+clean; 4.4 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 638 B/s rd, 0 B/s wr, 1.64k op/s; 27052/216636 objects degraded (12.487%) 2024-09-16T09:43:37.150 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:36 smithi070 ceph-mon[30388]: osdmap e505: 8 total, 7 up, 8 in 2024-09-16T09:43:37.150 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:36 smithi070 ceph-mon[40691]: pgmap v976: 168 pgs: 31 active+undersized, 18 active+undersized+degraded, 119 active+clean; 4.4 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 638 B/s rd, 0 B/s wr, 1.64k op/s; 27052/216636 objects degraded (12.487%) 2024-09-16T09:43:37.150 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:36 smithi070 ceph-mon[40691]: osdmap e505: 8 total, 7 up, 8 in 2024-09-16T09:43:37.157 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:37.157 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:37.158 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:43:37.259 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:43:37.292 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:37.293 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:37.293 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:43:37.394 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:43:37.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:36 smithi084 ceph-mon[32629]: pgmap v976: 168 pgs: 31 active+undersized, 18 active+undersized+degraded, 119 active+clean; 4.4 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 638 B/s rd, 0 B/s wr, 1.64k op/s; 27052/216636 objects degraded (12.487%) 2024-09-16T09:43:37.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:36 smithi084 ceph-mon[32629]: osdmap e505: 8 total, 7 up, 8 in 2024-09-16T09:43:37.421 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:37.422 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:37.422 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:43:37.523 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:43:37.533 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell -n osd.2 --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-2 --journal-path /var/lib/ceph/osd/ceph-2/journal --force --op remove --pgid 6.f 2024-09-16T09:43:37.536 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:43:37.555 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:37.556 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:37.556 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:43:37.580 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:43:37.657 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:43:37.693 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:37.693 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:37.694 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:43:37.795 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:43:37.833 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:37.834 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:37.834 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:43:37.850 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/osd.2/config 2024-09-16T09:43:37.935 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:43:37.973 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:37.974 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:37.974 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:43:38.075 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:43:38.112 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:38.113 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:38.113 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:43:38.148 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:37 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:43:38.148 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:37 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:43:38.148 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:37 smithi070 ceph-mon[30388]: osdmap e506: 8 total, 7 up, 8 in 2024-09-16T09:43:38.148 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:37 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:43:38.149 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:37 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:43:38.149 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:37 smithi070 ceph-mon[40691]: osdmap e506: 8 total, 7 up, 8 in 2024-09-16T09:43:38.214 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:43:38.244 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:38.245 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:38.245 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:43:38.346 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:43:38.382 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:38.383 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:38.383 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:43:38.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:37 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:43:38.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:37 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:43:38.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:37 smithi084 ceph-mon[32629]: osdmap e506: 8 total, 7 up, 8 in 2024-09-16T09:43:38.485 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:43:38.526 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:38.527 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:38.528 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:43:38.629 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:43:38.668 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T09:43:38.668 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:43:38.675 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:38.676 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:38.676 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:43:38.777 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:43:38.811 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:38.812 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:38.812 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:43:38.841 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T09:43:38.913 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:43:38.952 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:38.952 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:38.953 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:43:39.003 INFO:teuthology.orchestra.run.smithi070.stderr:nodeep-scrub is set 2024-09-16T09:43:39.054 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:43:39.090 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:39.090 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:39.091 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:43:39.158 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:39 smithi070 ceph-mon[40691]: pgmap v979: 168 pgs: 31 active+undersized, 18 active+undersized+degraded, 119 active+clean; 4.3 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.26k op/s; 26120/211104 objects degraded (12.373%) 2024-09-16T09:43:39.158 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:39 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/2345238940' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:43:39.158 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:39 smithi070 ceph-mon[30388]: pgmap v979: 168 pgs: 31 active+undersized, 18 active+undersized+degraded, 119 active+clean; 4.3 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.26k op/s; 26120/211104 objects degraded (12.373%) 2024-09-16T09:43:39.159 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:39 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2345238940' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:43:39.191 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:43:39.234 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:39.234 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:39.235 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:43:39.336 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:43:39.371 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:39.372 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:39.372 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:43:39.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:39 smithi084 ceph-mon[32629]: pgmap v979: 168 pgs: 31 active+undersized, 18 active+undersized+degraded, 119 active+clean; 4.3 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.26k op/s; 26120/211104 objects degraded (12.373%) 2024-09-16T09:43:39.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:39 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/2345238940' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:43:39.473 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:43:39.514 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:39.515 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:39.515 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:43:39.616 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:43:39.650 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:39.651 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:39.652 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:43:39.752 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:43:39.785 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:39.786 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:39.787 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:43:39.887 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:43:39.931 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:39.932 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:39.932 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:43:40.033 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:43:40.066 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:40.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:40.067 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:43:40.168 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:43:40.203 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:40.203 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:40.204 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:43:40.305 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:43:40.339 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:40.339 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:40.340 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:43:40.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:40 smithi084 ceph-mon[32629]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:43:40.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:40 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/2345238940' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-16T09:43:40.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:40 smithi084 ceph-mon[32629]: osdmap e507: 8 total, 7 up, 8 in 2024-09-16T09:43:40.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:40 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:43:40.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:40 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:43:40.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:40 smithi084 ceph-mon[32629]: Health check update: Degraded data redundancy: 26120/211104 objects degraded (12.373%), 18 pgs degraded (PG_DEGRADED) 2024-09-16T09:43:40.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:40 smithi070 ceph-mon[30388]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:43:40.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:40 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2345238940' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-16T09:43:40.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:40 smithi070 ceph-mon[30388]: osdmap e507: 8 total, 7 up, 8 in 2024-09-16T09:43:40.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:40 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:43:40.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:40 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:43:40.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:40 smithi070 ceph-mon[30388]: Health check update: Degraded data redundancy: 26120/211104 objects degraded (12.373%), 18 pgs degraded (PG_DEGRADED) 2024-09-16T09:43:40.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:40 smithi070 ceph-mon[40691]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:43:40.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:40 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/2345238940' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-16T09:43:40.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:40 smithi070 ceph-mon[40691]: osdmap e507: 8 total, 7 up, 8 in 2024-09-16T09:43:40.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:40 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:43:40.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:40 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:43:40.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:40 smithi070 ceph-mon[40691]: Health check update: Degraded data redundancy: 26120/211104 objects degraded (12.373%), 18 pgs degraded (PG_DEGRADED) 2024-09-16T09:43:40.441 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:43:40.462 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:40.463 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:40.463 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:43:40.564 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:43:40.596 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:40.597 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:40.597 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:43:40.698 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:43:40.733 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:40.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:40.734 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:43:40.834 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:43:40.864 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:40.865 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:40.865 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:43:40.939 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.3 dump_ops_in_flight 2024-09-16T09:43:40.966 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:43:41.019 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:41.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:41.020 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:43:41.120 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:43:41.154 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:41.155 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:41.155 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:43:41.170 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:41 smithi070 ceph-mon[30388]: pgmap v981: 168 pgs: 31 active+undersized, 18 active+undersized+degraded, 119 active+clean; 3.9 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 1.31k op/s; 24293/194316 objects degraded (12.502%) 2024-09-16T09:43:41.170 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:41 smithi070 ceph-mon[30388]: osdmap e508: 8 total, 7 up, 8 in 2024-09-16T09:43:41.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:41 smithi070 ceph-mon[40691]: pgmap v981: 168 pgs: 31 active+undersized, 18 active+undersized+degraded, 119 active+clean; 3.9 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 1.31k op/s; 24293/194316 objects degraded (12.502%) 2024-09-16T09:43:41.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:41 smithi070 ceph-mon[40691]: osdmap e508: 8 total, 7 up, 8 in 2024-09-16T09:43:41.256 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:43:41.293 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:41.294 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:41.294 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:43:41.395 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:43:41.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:41 smithi084 ceph-mon[32629]: pgmap v981: 168 pgs: 31 active+undersized, 18 active+undersized+degraded, 119 active+clean; 3.9 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 1.31k op/s; 24293/194316 objects degraded (12.502%) 2024-09-16T09:43:41.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:41 smithi084 ceph-mon[32629]: osdmap e508: 8 total, 7 up, 8 in 2024-09-16T09:43:41.421 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:41.422 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:41.422 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:43:41.523 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:43:41.558 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:41.558 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:41.558 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:43:41.659 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:43:41.688 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:41.689 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:41.689 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:43:41.792 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:43:41.834 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:41.835 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:41.835 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:43:41.936 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:43:41.973 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:41.974 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:41.974 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:43:41.975 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd unset noscrub 2024-09-16T09:43:42.048 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:43:41 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:43:41] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:43:42.074 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:43:42.110 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:42.110 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:42.111 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:43:42.212 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:43:42.252 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:42.253 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:42.253 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:43:42.354 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:43:42.386 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:42.386 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:42.387 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:43:42.488 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:43:42.527 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:42.527 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:42.527 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:43:42.538 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:43:42.566 INFO:teuthology.orchestra.run.smithi070.stderr:Creating an OSD daemon form without an OSD FSID value 2024-09-16T09:43:42.628 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:43:42.677 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:42.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:42.679 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:43:42.780 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:43:42.816 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:42.817 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:42.817 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:43:42.918 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:43:42.955 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:42.956 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:42.956 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:43:43.057 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:43:43.093 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:43.094 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:43.094 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:43:43.138 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:43 smithi070 ceph-mon[30388]: pgmap v983: 168 pgs: 31 active+undersized, 18 active+undersized+degraded, 119 active+clean; 3.9 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 864 B/s rd, 0 B/s wr, 1.26k op/s; 24293/194316 objects degraded (12.502%) 2024-09-16T09:43:43.139 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:43 smithi070 ceph-mon[40691]: pgmap v983: 168 pgs: 31 active+undersized, 18 active+undersized+degraded, 119 active+clean; 3.9 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 864 B/s rd, 0 B/s wr, 1.26k op/s; 24293/194316 objects degraded (12.502%) 2024-09-16T09:43:43.194 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:43:43.231 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:43.231 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:43.232 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:43:43.332 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:43:43.365 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:43.365 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:43.366 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:43:43.406 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:43:43.406 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:43:43.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:43 smithi084 ceph-mon[32629]: pgmap v983: 168 pgs: 31 active+undersized, 18 active+undersized+degraded, 119 active+clean; 3.9 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 864 B/s rd, 0 B/s wr, 1.26k op/s; 24293/194316 objects degraded (12.502%) 2024-09-16T09:43:43.434 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:43:43.434 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:43:43.436 INFO:teuthology.orchestra.run.smithi070.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-16T09:43:43.467 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:43:43.473 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:43:43.473 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:43:43.498 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:43.499 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:43.499 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:43:43.503 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:43:43.503 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:43:43.544 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:43:43.545 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:43:43.579 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:43:43.579 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:43:43.600 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:43:43.612 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:43:43.613 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:43:43.645 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:43.645 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:43.646 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:43:43.746 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:43:43.783 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:43.784 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:43.784 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:43:43.873 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:43:43.885 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:43:43.937 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:43.937 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:43.937 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:43:44.038 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:43:44.046 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:43:43 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:43:44.046 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:44 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3636933560' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:43:44.075 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:44.075 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:44.076 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:43:44.177 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:43:44.209 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:44.210 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:44.210 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:43:44.311 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:43:44.350 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:44.350 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:44.351 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:43:44.353 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:44 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3636933560' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:43:44.355 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:44 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3636933560' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:43:44.405 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T09:43:44.405 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T09:43:44.452 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:43:44.485 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:44.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:44.486 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:43:44.587 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:43:44.620 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:44.621 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:44.621 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:43:44.722 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:43:44.750 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:44.750 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:44.751 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:43:44.851 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:43:44.894 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:44.894 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:44.895 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:43:44.915 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:43:44.996 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:43:45.032 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:45.032 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:45.033 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:43:45.133 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:43:45.161 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:45.162 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:45.162 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:43:45.170 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:45 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:43:45.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:45 smithi070 ceph-mon[30388]: pgmap v984: 168 pgs: 31 active+undersized, 18 active+undersized+degraded, 119 active+clean; 3.5 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 2.12k op/s; 21928/172722 objects degraded (12.696%) 2024-09-16T09:43:45.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:45 smithi070 ceph-mon[30388]: Health check update: Degraded data redundancy: 21928/172722 objects degraded (12.696%), 18 pgs degraded (PG_DEGRADED) 2024-09-16T09:43:45.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:45 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:43:45.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:45 smithi070 ceph-mon[40691]: pgmap v984: 168 pgs: 31 active+undersized, 18 active+undersized+degraded, 119 active+clean; 3.5 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 2.12k op/s; 21928/172722 objects degraded (12.696%) 2024-09-16T09:43:45.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:45 smithi070 ceph-mon[40691]: Health check update: Degraded data redundancy: 21928/172722 objects degraded (12.696%), 18 pgs degraded (PG_DEGRADED) 2024-09-16T09:43:45.264 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:43:45.302 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:45.305 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:45.305 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:43:45.406 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:43:45.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:45 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:43:45.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:45 smithi084 ceph-mon[32629]: pgmap v984: 168 pgs: 31 active+undersized, 18 active+undersized+degraded, 119 active+clean; 3.5 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 2.12k op/s; 21928/172722 objects degraded (12.696%) 2024-09-16T09:43:45.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:45 smithi084 ceph-mon[32629]: Health check update: Degraded data redundancy: 21928/172722 objects degraded (12.696%), 18 pgs degraded (PG_DEGRADED) 2024-09-16T09:43:45.444 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:45.445 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:45.445 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:43:45.545 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:43:45.563 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T09:43:45.563 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [ 2024-09-16T09:43:45.563 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:45.564 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.16305.0:207422 11.b 11:dcb858d2:::benchmark_data_smithi070_191495_object76952:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e508)", 2024-09-16T09:43:45.564 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:43:45.547664+0000", 2024-09-16T09:43:45.564 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.010401904999999999, 2024-09-16T09:43:45.564 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.010430485999999999, 2024-09-16T09:43:45.564 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:43:45.564 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:43:45.565 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "waiting for sub ops", 2024-09-16T09:43:45.565 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:43:45.565 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.16305", 2024-09-16T09:43:45.565 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3818031514", 2024-09-16T09:43:45.565 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 207422 2024-09-16T09:43:45.566 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:45.566 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:43:45.566 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:45.566 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:43:45.566 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:45.547664+0000", 2024-09-16T09:43:45.566 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:43:45.567 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:45.567 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:45.567 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:43:45.567 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:45.547664+0000", 2024-09-16T09:43:45.567 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:43:45.567 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:45.568 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:45.568 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:43:45.568 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:45.547666+0000", 2024-09-16T09:43:45.568 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.283e-06 2024-09-16T09:43:45.568 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:45.568 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:45.569 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:43:45.569 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:45.547671+0000", 2024-09-16T09:43:45.569 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.3689999999999998e-06 2024-09-16T09:43:45.569 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:45.569 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:45.569 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:43:45.570 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:45.547685+0000", 2024-09-16T09:43:45.570 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3984e-05 2024-09-16T09:43:45.570 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:45.570 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:45.570 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:43:45.570 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:45.547699+0000", 2024-09-16T09:43:45.571 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.4309e-05 2024-09-16T09:43:45.571 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:45.571 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:45.571 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:43:45.571 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:45.547987+0000", 2024-09-16T09:43:45.572 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000287869 2024-09-16T09:43:45.572 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:45.572 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:45.572 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:43:45.572 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:45.548103+0000", 2024-09-16T09:43:45.573 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00011548900000000001 2024-09-16T09:43:45.573 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:45.573 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:45.573 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 0,1", 2024-09-16T09:43:45.573 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:45.548218+0000", 2024-09-16T09:43:45.574 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000115666 2024-09-16T09:43:45.574 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:45.574 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:45.574 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:43:45.574 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:45.555973+0000", 2024-09-16T09:43:45.574 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0077543559999999996 2024-09-16T09:43:45.575 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:43:45.575 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:43:45.575 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:43:45.575 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:45.575 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:45.576 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.16305.0:207423 11.b 11:da0c9e92:::benchmark_data_smithi070_191495_object76953:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e508)", 2024-09-16T09:43:45.576 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:43:45.547723+0000", 2024-09-16T09:43:45.576 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.010343231, 2024-09-16T09:43:45.576 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.010447018000000001, 2024-09-16T09:43:45.576 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:43:45.576 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:43:45.577 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "waiting for sub ops", 2024-09-16T09:43:45.577 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:43:45.577 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.16305", 2024-09-16T09:43:45.577 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3818031514", 2024-09-16T09:43:45.577 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 207423 2024-09-16T09:43:45.578 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:45.578 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:43:45.578 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:45.578 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:43:45.578 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:45.547723+0000", 2024-09-16T09:43:45.578 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:43:45.579 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:45.579 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:45.579 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:43:45.579 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:45.547723+0000", 2024-09-16T09:43:45.579 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:43:45.580 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:45.580 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:45.580 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:43:45.580 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:45.547724+0000", 2024-09-16T09:43:45.580 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1600000000000001e-07 2024-09-16T09:43:45.580 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:45.581 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:45.581 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:43:45.581 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:45.547727+0000", 2024-09-16T09:43:45.581 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.044e-06 2024-09-16T09:43:45.581 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:45.581 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:45.582 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:43:45.582 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:45.547739+0000", 2024-09-16T09:43:45.582 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.258e-05 2024-09-16T09:43:45.582 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:45.582 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:45.582 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:43:45.583 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:45.547746+0000", 2024-09-16T09:43:45.583 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.3629999999999999e-06 2024-09-16T09:43:45.583 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:45.583 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:45.583 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:43:45.583 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:45.551585+0000", 2024-09-16T09:43:45.584 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0038396699999999999 2024-09-16T09:43:45.584 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:45.584 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:45.584 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:43:45.584 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:45.551693+0000", 2024-09-16T09:43:45.585 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00010793000000000001 2024-09-16T09:43:45.585 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:45.585 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:45.585 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 0,1", 2024-09-16T09:43:45.585 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:45.551807+0000", 2024-09-16T09:43:45.585 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000114218 2024-09-16T09:43:45.586 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:43:45.586 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:43:45.586 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:43:45.586 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:45.586 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:45.586 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.16305.0:207428 11.b 11:d004dba0:::benchmark_data_smithi070_191495_object76958:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e508)", 2024-09-16T09:43:45.587 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:43:45.553122+0000", 2024-09-16T09:43:45.587 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.0049437719999999999, 2024-09-16T09:43:45.587 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0051056269999999997, 2024-09-16T09:43:45.587 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:43:45.587 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:43:45.587 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "waiting for sub ops", 2024-09-16T09:43:45.588 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:43:45.588 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.16305", 2024-09-16T09:43:45.588 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3818031514", 2024-09-16T09:43:45.588 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 207428 2024-09-16T09:43:45.588 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:45.588 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:43:45.589 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:45.589 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:43:45.589 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:45.553122+0000", 2024-09-16T09:43:45.589 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:43:45.589 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:45.589 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:45.590 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:43:45.590 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:45.553122+0000", 2024-09-16T09:43:45.590 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:43:45.590 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:45.590 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:45.591 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:43:45.591 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:45.553124+0000", 2024-09-16T09:43:45.591 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2669999999999999e-06 2024-09-16T09:43:45.591 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:45.591 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:45.591 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:43:45.592 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:45.553129+0000", 2024-09-16T09:43:45.592 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.8489999999999997e-06 2024-09-16T09:43:45.592 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:45.592 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:45.592 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:43:45.592 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:45.553139+0000", 2024-09-16T09:43:45.593 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.045e-05 2024-09-16T09:43:45.593 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:45.593 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:45.593 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:43:45.593 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:45.553146+0000", 2024-09-16T09:43:45.593 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.9700000000000002e-06 2024-09-16T09:43:45.594 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:45.594 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:45.594 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:43:45.594 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:45.556047+0000", 2024-09-16T09:43:45.594 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0029006240000000001 2024-09-16T09:43:45.594 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:45.595 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:45.595 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:43:45.595 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:45.556169+0000", 2024-09-16T09:43:45.595 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000122576 2024-09-16T09:43:45.595 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:45.595 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:45.596 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 0,1", 2024-09-16T09:43:45.596 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:45.556290+0000", 2024-09-16T09:43:45.596 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000121172 2024-09-16T09:43:45.596 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:43:45.596 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:43:45.596 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:43:45.597 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:45.597 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:45.597 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.16305.0:207433 11.b 11.92a38f9b (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e508)", 2024-09-16T09:43:45.597 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:43:45.556717+0000", 2024-09-16T09:43:45.597 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.00134921, 2024-09-16T09:43:45.598 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0015657360000000001, 2024-09-16T09:43:45.598 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:43:45.598 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:43:45.599 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T09:43:45.599 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:43:45.599 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.16305", 2024-09-16T09:43:45.599 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3818031514", 2024-09-16T09:43:45.599 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 207433 2024-09-16T09:43:45.600 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:45.600 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:43:45.600 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:45.600 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:43:45.600 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:45.556717+0000", 2024-09-16T09:43:45.600 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:43:45.600 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:45.600 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:45.600 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:43:45.600 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:45.556717+0000", 2024-09-16T09:43:45.601 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:43:45.601 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:45.601 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:45.601 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:43:45.601 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:45.556718+0000", 2024-09-16T09:43:45.601 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.243e-06 2024-09-16T09:43:45.601 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:45.601 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:45.601 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:43:45.601 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:45.556723+0000", 2024-09-16T09:43:45.601 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.1719999999999998e-06 2024-09-16T09:43:45.601 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:45.601 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:45.602 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:43:45.602 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:45.556733+0000", 2024-09-16T09:43:45.602 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.8079999999999999e-06 2024-09-16T09:43:45.602 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:45.602 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:45.602 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:43:45.602 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:45.556741+0000", 2024-09-16T09:43:45.602 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.2819999999999999e-06 2024-09-16T09:43:45.602 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:43:45.602 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:43:45.602 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:43:45.602 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:45.603 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:45.603 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.16305.0:207432 11.a e508/454 11:53409cd1:::benchmark_data_smithi070_191495_object76962:head v 508'12811, mlcod=508'12809)", 2024-09-16T09:43:45.603 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:43:45.556901+0000", 2024-09-16T09:43:45.603 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.0011648349999999999, 2024-09-16T09:43:45.603 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001424025, 2024-09-16T09:43:45.603 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:43:45.603 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:43:45.603 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T09:43:45.603 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:43:45.603 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:45.603 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:43:45.603 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:45.556901+0000", 2024-09-16T09:43:45.603 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:43:45.604 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:45.604 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:45.604 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:43:45.604 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:45.556901+0000", 2024-09-16T09:43:45.604 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:43:45.604 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:45.604 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:45.604 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:43:45.604 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:45.556902+0000", 2024-09-16T09:43:45.604 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.9500000000000002e-07 2024-09-16T09:43:45.604 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:45.604 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:45.604 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:43:45.605 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:45.556907+0000", 2024-09-16T09:43:45.605 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.7740000000000004e-06 2024-09-16T09:43:45.605 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:45.605 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:45.605 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:43:45.605 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:45.556918+0000", 2024-09-16T09:43:45.605 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1097000000000001e-05 2024-09-16T09:43:45.605 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:45.605 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:45.605 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:43:45.605 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:45.556925+0000", 2024-09-16T09:43:45.605 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.8510000000000001e-06 2024-09-16T09:43:45.605 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:45.606 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:45.606 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:43:45.606 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:45.557742+0000", 2024-09-16T09:43:45.606 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000816923 2024-09-16T09:43:45.606 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:45.606 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:45.606 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:43:45.606 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:45.557760+0000", 2024-09-16T09:43:45.606 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7866000000000001e-05 2024-09-16T09:43:45.606 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:43:45.606 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:43:45.606 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:43:45.606 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:45.606 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:45.607 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop_reply(client.16305.0:207422 11.b e508/454)", 2024-09-16T09:43:45.607 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:43:45.557092+0000", 2024-09-16T09:43:45.607 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.00097422100000000005, 2024-09-16T09:43:45.607 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0012752219999999999, 2024-09-16T09:43:45.607 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:43:45.607 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:43:45.607 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T09:43:45.607 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:43:45.607 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:45.607 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:43:45.607 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:45.557092+0000", 2024-09-16T09:43:45.607 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:43:45.607 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:45.608 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:45.608 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:43:45.608 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:45.557092+0000", 2024-09-16T09:43:45.608 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:43:45.608 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:45.608 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:45.608 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:43:45.608 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:45.557098+0000", 2024-09-16T09:43:45.608 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.835e-06 2024-09-16T09:43:45.608 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:45.608 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:45.608 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:43:45.608 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:45.557101+0000", 2024-09-16T09:43:45.609 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3440000000000001e-06 2024-09-16T09:43:45.609 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:45.609 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:45.609 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:43:45.609 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:45.557112+0000", 2024-09-16T09:43:45.609 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1158000000000001e-05 2024-09-16T09:43:45.609 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:45.609 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:45.609 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:43:45.609 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:45.557119+0000", 2024-09-16T09:43:45.609 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.6980000000000001e-06 2024-09-16T09:43:45.609 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:43:45.609 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:43:45.609 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:43:45.610 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:45.610 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:45.610 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop_reply(client.16305.0:207423 11.b e508/454)", 2024-09-16T09:43:45.610 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:43:45.557182+0000", 2024-09-16T09:43:45.610 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.00088408599999999996, 2024-09-16T09:43:45.610 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0012252560000000001, 2024-09-16T09:43:45.610 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:43:45.610 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:43:45.610 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T09:43:45.610 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:43:45.610 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:45.610 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:43:45.610 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:45.557182+0000", 2024-09-16T09:43:45.611 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:43:45.611 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:45.611 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:45.611 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:43:45.611 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:45.557182+0000", 2024-09-16T09:43:45.611 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:43:45.611 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:45.611 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:45.611 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:43:45.611 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:45.557183+0000", 2024-09-16T09:43:45.611 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8200000000000001e-07 2024-09-16T09:43:45.611 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:45.611 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:45.611 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:43:45.612 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:45.557185+0000", 2024-09-16T09:43:45.612 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.8480000000000001e-06 2024-09-16T09:43:45.612 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:45.612 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:45.612 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:43:45.612 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:45.557193+0000", 2024-09-16T09:43:45.612 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.1100000000000003e-06 2024-09-16T09:43:45.612 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:45.612 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:45.612 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:43:45.612 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:45.557199+0000", 2024-09-16T09:43:45.612 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7180000000000001e-06 2024-09-16T09:43:45.612 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:43:45.613 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:43:45.613 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:43:45.613 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:45.613 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:45.613 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_repop(client.16305.0:207435 11.0 e508/454 11:04952a65:::benchmark_data_smithi070_191495_object76965:head v 508'13102, mlcod=508'13100)", 2024-09-16T09:43:45.613 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:43:45.557587+0000", 2024-09-16T09:43:45.613 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 0.00047961999999999999, 2024-09-16T09:43:45.613 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00085781099999999999, 2024-09-16T09:43:45.613 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:43:45.613 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:43:45.613 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T09:43:45.613 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:43:45.613 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:45.614 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:43:45.614 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:45.557587+0000", 2024-09-16T09:43:45.614 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:43:45.614 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:45.614 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:45.614 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:43:45.614 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:45.557587+0000", 2024-09-16T09:43:45.614 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:43:45.614 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:45.614 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:45.614 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:43:45.614 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:45.557587+0000", 2024-09-16T09:43:45.614 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.4199999999999995e-07 2024-09-16T09:43:45.615 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:45.615 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:45.615 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:43:45.615 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:45.557593+0000", 2024-09-16T09:43:45.615 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.361e-06 2024-09-16T09:43:45.615 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:45.615 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:45.615 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:43:45.615 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:45.557603+0000", 2024-09-16T09:43:45.615 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0892e-05 2024-09-16T09:43:45.615 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:45.615 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:45.615 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:43:45.616 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:45.557610+0000", 2024-09-16T09:43:45.616 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.5939999999999999e-06 2024-09-16T09:43:45.616 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:45.616 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:45.616 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:43:45.616 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:45.557676+0000", 2024-09-16T09:43:45.616 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.5462000000000004e-05 2024-09-16T09:43:45.616 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:45.616 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:45.616 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:43:45.616 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:45.557695+0000", 2024-09-16T09:43:45.616 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9780999999999999e-05 2024-09-16T09:43:45.616 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:43:45.617 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:43:45.617 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:43:45.617 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:45.617 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:45.617 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.16305.0:207442 11.b 11.6702d78b (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e508)", 2024-09-16T09:43:45.617 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:43:45.558394+0000", 2024-09-16T09:43:45.617 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 4294967295.9996719, 2024-09-16T09:43:45.617 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.7079999999999999e-05, 2024-09-16T09:43:45.617 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:43:45.617 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:43:45.617 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "queued for pg", 2024-09-16T09:43:45.617 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:43:45.617 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.16305", 2024-09-16T09:43:45.618 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/3818031514", 2024-09-16T09:43:45.618 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 207442 2024-09-16T09:43:45.618 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:45.618 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:43:45.618 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:45.618 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:43:45.618 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:45.558394+0000", 2024-09-16T09:43:45.618 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:43:45.618 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:45.618 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:45.618 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:43:45.618 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:45.558394+0000", 2024-09-16T09:43:45.618 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:43:45.618 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:45.618 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:45.619 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:43:45.619 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:45.558395+0000", 2024-09-16T09:43:45.619 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.8899999999999998e-07 2024-09-16T09:43:45.619 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:45.619 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:45.619 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:43:45.619 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:45.558399+0000", 2024-09-16T09:43:45.619 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2189999999999998e-06 2024-09-16T09:43:45.619 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:45.619 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:45.619 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:43:45.619 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:45.558415+0000", 2024-09-16T09:43:45.619 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5241e-05 2024-09-16T09:43:45.620 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:45.620 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:45.620 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:43:45.620 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:43:45.558423+0000", 2024-09-16T09:43:45.620 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.8730000000000007e-06 2024-09-16T09:43:45.620 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:43:45.620 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:43:45.620 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:43:45.620 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:43:45.620 INFO:teuthology.orchestra.run.smithi070.stdout: ], 2024-09-16T09:43:45.620 INFO:teuthology.orchestra.run.smithi070.stdout: "num_ops": 9 2024-09-16T09:43:45.620 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T09:43:45.621 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:45.622 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:45.622 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:43:45.723 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:43:45.759 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:45.760 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:45.760 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:43:45.861 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:43:45.898 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:45.899 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:45.899 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:43:46.000 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:43:46.032 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:46.033 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:46.033 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:43:46.062 INFO:teuthology.orchestra.run.smithi070.stderr:noscrub is unset 2024-09-16T09:43:46.134 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:43:46.168 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:46.169 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:46.169 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:43:46.270 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:43:46.306 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:46.307 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:46.307 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:43:46.407 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T09:43:46.408 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:43:46.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:46 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/684931862' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:43:46.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:46 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/684931862' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:43:46.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:46 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/684931862' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:43:46.450 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:46.450 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:46.451 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:43:46.552 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:43:46.559 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.3 dump_blocked_ops 2024-09-16T09:43:46.581 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:46.582 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:46.582 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:43:46.683 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:43:46.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:46.746 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:46.746 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:43:46.847 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:43:46.902 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:46.903 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:46.904 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:43:47.004 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:43:47.034 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:47.035 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:47.036 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:43:47.137 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:43:47.175 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:47.177 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:47.177 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:43:47.278 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:43:47.318 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:47.318 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:47.319 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:43:47.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:47 smithi084 ceph-mon[32629]: pgmap v985: 168 pgs: 31 active+undersized, 18 active+undersized+degraded, 119 active+clean; 3.5 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.66k op/s; 21418/171192 objects degraded (12.511%) 2024-09-16T09:43:47.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:47 smithi084 ceph-mon[32629]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:43:47.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:47 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/684931862' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-16T09:43:47.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:47 smithi084 ceph-mon[32629]: osdmap e509: 8 total, 7 up, 8 in 2024-09-16T09:43:47.420 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:43:47.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:47 smithi070 ceph-mon[30388]: pgmap v985: 168 pgs: 31 active+undersized, 18 active+undersized+degraded, 119 active+clean; 3.5 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.66k op/s; 21418/171192 objects degraded (12.511%) 2024-09-16T09:43:47.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:47 smithi070 ceph-mon[30388]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:43:47.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:47 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/684931862' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-16T09:43:47.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:47 smithi070 ceph-mon[30388]: osdmap e509: 8 total, 7 up, 8 in 2024-09-16T09:43:47.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:47 smithi070 ceph-mon[40691]: pgmap v985: 168 pgs: 31 active+undersized, 18 active+undersized+degraded, 119 active+clean; 3.5 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.66k op/s; 21418/171192 objects degraded (12.511%) 2024-09-16T09:43:47.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:47 smithi070 ceph-mon[40691]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:43:47.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:47 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/684931862' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-16T09:43:47.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:47 smithi070 ceph-mon[40691]: osdmap e509: 8 total, 7 up, 8 in 2024-09-16T09:43:47.456 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:47.456 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:47.456 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:43:47.557 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:43:47.593 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:47.594 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:47.594 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:43:47.695 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:43:47.729 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:47.732 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:47.733 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:43:47.834 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:43:47.875 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:47.875 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:47.876 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:43:47.976 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:43:48.008 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:48.009 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:48.009 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:43:48.110 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:43:48.148 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:48.148 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:48.149 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:43:48.250 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:43:48.286 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:48.287 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:48.287 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:43:48.388 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:43:48.391 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:48 smithi070 ceph-mon[30388]: 3.15 scrub starts 2024-09-16T09:43:48.391 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:48 smithi070 ceph-mon[30388]: 3.15 scrub ok 2024-09-16T09:43:48.391 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:48 smithi070 ceph-mon[40691]: 3.15 scrub starts 2024-09-16T09:43:48.392 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:48 smithi070 ceph-mon[40691]: 3.15 scrub ok 2024-09-16T09:43:48.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:48 smithi084 ceph-mon[32629]: 3.15 scrub starts 2024-09-16T09:43:48.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:48 smithi084 ceph-mon[32629]: 3.15 scrub ok 2024-09-16T09:43:48.417 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:48.418 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:48.418 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:43:48.519 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:43:48.524 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:43:48.547 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:48.547 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:48.548 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:43:48.593 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:43:48.648 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:43:48.679 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:48.679 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:48.680 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:43:48.781 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:43:48.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:48.821 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:48.821 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:43:48.922 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:43:48.954 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:48.954 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:48.955 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:43:49.055 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:43:49.092 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:49.092 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:49.093 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:43:49.194 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:43:49.231 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:49 smithi070 ceph-mon[30388]: 4.15 scrub starts 2024-09-16T09:43:49.231 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:49 smithi070 ceph-mon[30388]: 4.15 scrub ok 2024-09-16T09:43:49.231 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:49 smithi070 ceph-mon[30388]: pgmap v987: 168 pgs: 31 active+undersized, 18 active+undersized+degraded, 119 active+clean; 3.4 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.19k op/s; 20523/165789 objects degraded (12.379%) 2024-09-16T09:43:49.231 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:49 smithi070 ceph-mon[30388]: 4.5 scrub starts 2024-09-16T09:43:49.231 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:49 smithi070 ceph-mon[30388]: 4.5 scrub ok 2024-09-16T09:43:49.231 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:49 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:43:49.231 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:49 smithi070 ceph-mon[40691]: 4.15 scrub starts 2024-09-16T09:43:49.232 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:49 smithi070 ceph-mon[40691]: 4.15 scrub ok 2024-09-16T09:43:49.232 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:49 smithi070 ceph-mon[40691]: pgmap v987: 168 pgs: 31 active+undersized, 18 active+undersized+degraded, 119 active+clean; 3.4 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.19k op/s; 20523/165789 objects degraded (12.379%) 2024-09-16T09:43:49.232 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:49 smithi070 ceph-mon[40691]: 4.5 scrub starts 2024-09-16T09:43:49.232 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:49 smithi070 ceph-mon[40691]: 4.5 scrub ok 2024-09-16T09:43:49.232 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:49 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:43:49.233 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:49.233 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:49.233 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:43:49.280 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:43:49.281 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:43:49.319 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:43:49.319 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:43:49.324 INFO:teuthology.orchestra.run.smithi070.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-16T09:43:49.334 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:43:49.356 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:43:49.356 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:43:49.369 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:49.370 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:49.370 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:43:49.394 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:43:49.394 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:43:49.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:49 smithi084 ceph-mon[32629]: 4.15 scrub starts 2024-09-16T09:43:49.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:49 smithi084 ceph-mon[32629]: 4.15 scrub ok 2024-09-16T09:43:49.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:49 smithi084 ceph-mon[32629]: pgmap v987: 168 pgs: 31 active+undersized, 18 active+undersized+degraded, 119 active+clean; 3.4 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.19k op/s; 20523/165789 objects degraded (12.379%) 2024-09-16T09:43:49.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:49 smithi084 ceph-mon[32629]: 4.5 scrub starts 2024-09-16T09:43:49.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:49 smithi084 ceph-mon[32629]: 4.5 scrub ok 2024-09-16T09:43:49.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:49 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:43:49.432 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:43:49.432 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:43:49.471 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:43:49.472 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:43:49.472 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:43:49.496 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:49.496 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:49.497 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:43:49.509 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:43:49.510 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:43:49.597 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:43:49.629 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:49.630 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:49.630 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:43:49.731 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:43:49.768 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:49.768 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:49.769 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:43:49.869 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:43:49.906 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:49.907 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:49.907 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:43:50.008 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:43:50.035 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:50.036 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:50.036 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:43:50.086 INFO:teuthology.orchestra.run.smithi070.stdout:ERROR: (22) Invalid argument 2024-09-16T09:43:50.086 INFO:teuthology.orchestra.run.smithi070.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:43:50.137 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:43:50.172 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:50 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/380862941' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:43:50.172 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:50 smithi070 ceph-mon[30388]: 3.11 scrub starts 2024-09-16T09:43:50.172 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:50 smithi070 ceph-mon[30388]: 3.11 scrub ok 2024-09-16T09:43:50.172 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:50 smithi070 ceph-mon[30388]: Health check update: Degraded data redundancy: 20523/165789 objects degraded (12.379%), 18 pgs degraded (PG_DEGRADED) 2024-09-16T09:43:50.173 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:50 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/380862941' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:43:50.173 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:50 smithi070 ceph-mon[40691]: 3.11 scrub starts 2024-09-16T09:43:50.173 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:50 smithi070 ceph-mon[40691]: 3.11 scrub ok 2024-09-16T09:43:50.173 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:50 smithi070 ceph-mon[40691]: Health check update: Degraded data redundancy: 20523/165789 objects degraded (12.379%), 18 pgs degraded (PG_DEGRADED) 2024-09-16T09:43:50.173 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:50.174 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:50.174 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:43:50.275 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:43:50.312 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:50.312 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:50.313 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:43:50.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:50 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/380862941' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:43:50.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:50 smithi084 ceph-mon[32629]: 3.11 scrub starts 2024-09-16T09:43:50.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:50 smithi084 ceph-mon[32629]: 3.11 scrub ok 2024-09-16T09:43:50.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:50 smithi084 ceph-mon[32629]: Health check update: Degraded data redundancy: 20523/165789 objects degraded (12.379%), 18 pgs degraded (PG_DEGRADED) 2024-09-16T09:43:50.414 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:43:50.445 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-16T09:43:50.446 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:50.447 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:50.447 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:43:50.549 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:43:50.586 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:50.586 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:50.587 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:43:50.687 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:43:50.730 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:50.731 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:50.731 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:43:50.832 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:43:50.872 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:50.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:50.873 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:43:50.974 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:43:51.006 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:51.007 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:51.007 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:43:51.108 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:43:51.143 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:51.144 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:51.144 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:43:51.245 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:43:51.279 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:51.280 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:51.280 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:43:51.343 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:51 smithi070 ceph-mon[30388]: 4.c scrub starts 2024-09-16T09:43:51.343 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:51 smithi070 ceph-mon[30388]: 4.c scrub ok 2024-09-16T09:43:51.343 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:51 smithi070 ceph-mon[30388]: 4.7 scrub starts 2024-09-16T09:43:51.343 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:51 smithi070 ceph-mon[30388]: 4.7 scrub ok 2024-09-16T09:43:51.343 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:51 smithi070 ceph-mon[30388]: pgmap v988: 168 pgs: 31 active+undersized, 18 active+undersized+degraded, 119 active+clean; 3.0 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.52k op/s; 18662/149217 objects degraded (12.507%) 2024-09-16T09:43:51.344 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:51 smithi070 ceph-mon[30388]: 11.1 scrub starts 2024-09-16T09:43:51.344 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:51 smithi070 ceph-mon[30388]: 4.0 scrub starts 2024-09-16T09:43:51.344 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:51 smithi070 ceph-mon[30388]: 4.0 scrub ok 2024-09-16T09:43:51.344 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:51 smithi070 ceph-mon[40691]: 4.c scrub starts 2024-09-16T09:43:51.344 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:51 smithi070 ceph-mon[40691]: 4.c scrub ok 2024-09-16T09:43:51.344 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:51 smithi070 ceph-mon[40691]: 4.7 scrub starts 2024-09-16T09:43:51.344 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:51 smithi070 ceph-mon[40691]: 4.7 scrub ok 2024-09-16T09:43:51.344 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:51 smithi070 ceph-mon[40691]: pgmap v988: 168 pgs: 31 active+undersized, 18 active+undersized+degraded, 119 active+clean; 3.0 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.52k op/s; 18662/149217 objects degraded (12.507%) 2024-09-16T09:43:51.344 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:51 smithi070 ceph-mon[40691]: 11.1 scrub starts 2024-09-16T09:43:51.344 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:51 smithi070 ceph-mon[40691]: 4.0 scrub starts 2024-09-16T09:43:51.344 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:51 smithi070 ceph-mon[40691]: 4.0 scrub ok 2024-09-16T09:43:51.381 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:43:51.392 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T09:43:51.393 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T09:43:51.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:51 smithi084 ceph-mon[32629]: 4.c scrub starts 2024-09-16T09:43:51.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:51 smithi084 ceph-mon[32629]: 4.c scrub ok 2024-09-16T09:43:51.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:51 smithi084 ceph-mon[32629]: 4.7 scrub starts 2024-09-16T09:43:51.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:51 smithi084 ceph-mon[32629]: 4.7 scrub ok 2024-09-16T09:43:51.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:51 smithi084 ceph-mon[32629]: pgmap v988: 168 pgs: 31 active+undersized, 18 active+undersized+degraded, 119 active+clean; 3.0 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.52k op/s; 18662/149217 objects degraded (12.507%) 2024-09-16T09:43:51.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:51 smithi084 ceph-mon[32629]: 11.1 scrub starts 2024-09-16T09:43:51.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:51 smithi084 ceph-mon[32629]: 4.0 scrub starts 2024-09-16T09:43:51.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:51 smithi084 ceph-mon[32629]: 4.0 scrub ok 2024-09-16T09:43:51.418 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:51.418 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:51.419 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:43:51.519 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:43:51.551 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:51.552 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:51.553 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:43:51.653 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:43:51.687 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:51.687 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:51.688 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:43:51.789 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:43:51.823 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:51.823 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:51.824 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:43:51.925 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:43:51.954 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:51.954 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:51.955 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:43:51.988 INFO:teuthology.orchestra.run.smithi070.stdout: marking collection for removal 2024-09-16T09:43:51.988 INFO:teuthology.orchestra.run.smithi070.stdout:setting '_remove' omap key 2024-09-16T09:43:51.988 INFO:teuthology.orchestra.run.smithi070.stdout:finish_remove_pgs 6.f_head removing 6.f 2024-09-16T09:43:51.991 INFO:teuthology.orchestra.run.smithi070.stdout:Remove successful 2024-09-16T09:43:52.056 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:43:52.095 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:52.095 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:52.096 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:43:52.196 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:43:52.226 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:52.226 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:52.227 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:43:52.328 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:43:52.362 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:52.363 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:52.363 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:43:52.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:52 smithi084 ceph-mon[32629]: 1.6 scrub starts 2024-09-16T09:43:52.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:52 smithi084 ceph-mon[32629]: 1.6 scrub ok 2024-09-16T09:43:52.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:52 smithi084 ceph-mon[32629]: 11.1 scrub ok 2024-09-16T09:43:52.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:52 smithi070 ceph-mon[30388]: 1.6 scrub starts 2024-09-16T09:43:52.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:52 smithi070 ceph-mon[30388]: 1.6 scrub ok 2024-09-16T09:43:52.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:52 smithi070 ceph-mon[30388]: 11.1 scrub ok 2024-09-16T09:43:52.421 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:43:51 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:43:51] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:43:52.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:52 smithi070 ceph-mon[40691]: 1.6 scrub starts 2024-09-16T09:43:52.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:52 smithi070 ceph-mon[40691]: 1.6 scrub ok 2024-09-16T09:43:52.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:52 smithi070 ceph-mon[40691]: 11.1 scrub ok 2024-09-16T09:43:52.465 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:43:52.504 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:52.504 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:52.504 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:43:52.605 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:43:52.643 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:52.643 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:52.644 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:43:52.744 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:43:52.776 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:52.777 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:52.777 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:43:52.878 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:43:52.911 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:52.912 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:52.913 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:43:53.013 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:43:53.042 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:53.043 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:53.043 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:43:53.144 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:43:53.177 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:53.178 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:53.178 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:43:53.279 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:43:53.309 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:53.310 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:53.310 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:43:53.394 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T09:43:53.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:53 smithi084 ceph-mon[32629]: pgmap v989: 168 pgs: 31 active+undersized, 18 active+undersized+degraded, 119 active+clean; 3.0 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.50k op/s; 18662/149217 objects degraded (12.507%) 2024-09-16T09:43:53.411 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:43:53.420 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:53 smithi070 ceph-mon[40691]: pgmap v989: 168 pgs: 31 active+undersized, 18 active+undersized+degraded, 119 active+clean; 3.0 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.50k op/s; 18662/149217 objects degraded (12.507%) 2024-09-16T09:43:53.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:53 smithi070 ceph-mon[30388]: pgmap v989: 168 pgs: 31 active+undersized, 18 active+undersized+degraded, 119 active+clean; 3.0 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.50k op/s; 18662/149217 objects degraded (12.507%) 2024-09-16T09:43:53.463 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:53.463 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:53.464 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:43:53.505 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.3 dump_historic_ops 2024-09-16T09:43:53.565 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:43:53.617 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:53.617 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:53.618 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:43:53.718 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:43:53.756 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:53.756 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:53.757 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:43:53.858 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:43:53.889 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:53.890 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:53.890 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:43:53.991 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:43:54.024 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:54.025 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:54.025 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:43:54.111 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:43:53 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:43:54.119 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell -n osd.2 --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph-objectstore-tool --err-to-stderr --data-path /var/lib/ceph/osd/ceph-2 --journal-path /var/lib/ceph/osd/ceph-2/journal '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --op import --file /var/log/ceph/exp.6.f.2 2024-09-16T09:43:54.126 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:43:54.158 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:54.159 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:54.159 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:43:54.260 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:43:54.262 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:43:54.285 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:54.285 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:54.285 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:43:54.291 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:43:54.298 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:43:54.344 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:54 smithi070 ceph-mon[30388]: 2.2 scrub starts 2024-09-16T09:43:54.344 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:54 smithi070 ceph-mon[30388]: 2.2 scrub ok 2024-09-16T09:43:54.344 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:54 smithi070 ceph-mon[40691]: 2.2 scrub starts 2024-09-16T09:43:54.344 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:54 smithi070 ceph-mon[40691]: 2.2 scrub ok 2024-09-16T09:43:54.386 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:43:54.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:54 smithi084 ceph-mon[32629]: 2.2 scrub starts 2024-09-16T09:43:54.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:54 smithi084 ceph-mon[32629]: 2.2 scrub ok 2024-09-16T09:43:54.427 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:54.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:54.428 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:43:54.502 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/osd.2/config 2024-09-16T09:43:54.528 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:43:54.570 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:54.571 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:54.571 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:43:54.672 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:43:54.709 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:54.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:54.710 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:43:54.810 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:43:54.841 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:54.842 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:54.842 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:43:54.943 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:43:54.982 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:54.983 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:54.984 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:43:55.084 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:43:55.119 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:55.120 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:55.120 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:43:55.221 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:43:55.258 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:55.259 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:55.259 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:43:55.291 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:55 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:43:55.291 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:55 smithi070 ceph-mon[40691]: pgmap v990: 168 pgs: 31 active+undersized, 18 active+undersized+degraded, 119 active+clean; 2.6 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.45k op/s; 16458/129264 objects degraded (12.732%) 2024-09-16T09:43:55.291 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:55 smithi070 ceph-mon[40691]: 1.3 scrub starts 2024-09-16T09:43:55.291 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:55 smithi070 ceph-mon[40691]: 1.3 scrub ok 2024-09-16T09:43:55.291 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:55 smithi070 ceph-mon[40691]: Health check update: Degraded data redundancy: 16458/129264 objects degraded (12.732%), 18 pgs degraded (PG_DEGRADED) 2024-09-16T09:43:55.291 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:55 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:43:55.291 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:55 smithi070 ceph-mon[30388]: pgmap v990: 168 pgs: 31 active+undersized, 18 active+undersized+degraded, 119 active+clean; 2.6 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.45k op/s; 16458/129264 objects degraded (12.732%) 2024-09-16T09:43:55.291 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:55 smithi070 ceph-mon[30388]: 1.3 scrub starts 2024-09-16T09:43:55.291 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:55 smithi070 ceph-mon[30388]: 1.3 scrub ok 2024-09-16T09:43:55.291 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:55 smithi070 ceph-mon[30388]: Health check update: Degraded data redundancy: 16458/129264 objects degraded (12.732%), 18 pgs degraded (PG_DEGRADED) 2024-09-16T09:43:55.360 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:43:55.394 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:55.395 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:55.395 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:43:55.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:55 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:43:55.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:55 smithi084 ceph-mon[32629]: pgmap v990: 168 pgs: 31 active+undersized, 18 active+undersized+degraded, 119 active+clean; 2.6 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.45k op/s; 16458/129264 objects degraded (12.732%) 2024-09-16T09:43:55.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:55 smithi084 ceph-mon[32629]: 1.3 scrub starts 2024-09-16T09:43:55.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:55 smithi084 ceph-mon[32629]: 1.3 scrub ok 2024-09-16T09:43:55.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:55 smithi084 ceph-mon[32629]: Health check update: Degraded data redundancy: 16458/129264 objects degraded (12.732%), 18 pgs degraded (PG_DEGRADED) 2024-09-16T09:43:55.496 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:43:55.527 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:55.528 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:55.528 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:43:55.534 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:43:55.534 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:43:55.534 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:43:55.534 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:43:55.534 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:43:55.534 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:43:55.534 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:43:55.535 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:43:55.535 INFO:teuthology.orchestra.run.smithi070.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-16T09:43:55.535 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:43:55.535 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:43:55.535 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:43:55.535 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:43:55.566 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:43:55.566 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:43:55.629 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:43:55.670 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:55.671 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:55.671 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:43:55.772 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:43:55.806 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:55.806 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:55.807 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:43:55.907 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:43:55.945 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:55.945 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:55.945 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:43:56.046 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:43:56.082 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:56.083 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:56.083 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:43:56.135 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T09:43:56.135 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 20, 2024-09-16T09:43:56.135 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 600, 2024-09-16T09:43:56.135 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [ 2024-09-16T09:43:56.135 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.135 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:85224 9.19 9:9bf6c0c6:::benchmark_data_smithi070_137000_object85223:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e266)", 2024-09-16T09:43:56.135 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:34:30.373671+0000", 2024-09-16T09:43:56.136 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 565.75679374599997, 2024-09-16T09:43:56.136 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0883978459999999, 2024-09-16T09:43:56.136 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:43:56.136 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:43:56.136 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T09:43:56.136 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:43:56.136 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:43:56.136 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:43:56.136 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 85224 2024-09-16T09:43:56.136 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.136 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:43:56.136 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.137 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:43:56.137 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.373671+0000", 2024-09-16T09:43:56.137 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:43:56.137 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.137 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.137 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:43:56.137 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.373671+0000", 2024-09-16T09:43:56.137 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:43:56.137 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.137 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.137 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:43:56.137 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.373673+0000", 2024-09-16T09:43:56.137 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.4600000000000002e-06 2024-09-16T09:43:56.137 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.138 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.138 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:43:56.138 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.373702+0000", 2024-09-16T09:43:56.138 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.8934000000000001e-05 2024-09-16T09:43:56.138 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.138 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.138 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:43:56.138 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.373725+0000", 2024-09-16T09:43:56.138 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3468999999999998e-05 2024-09-16T09:43:56.138 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.138 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.138 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:43:56.138 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.373737+0000", 2024-09-16T09:43:56.139 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1252e-05 2024-09-16T09:43:56.139 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.139 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.139 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:43:56.139 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.386012+0000", 2024-09-16T09:43:56.139 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.012275569 2024-09-16T09:43:56.139 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.139 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.139 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:43:56.139 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.386022+0000", 2024-09-16T09:43:56.139 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.2609999999999999e-06 2024-09-16T09:43:56.139 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.139 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.139 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:43:56.140 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.428618+0000", 2024-09-16T09:43:56.140 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0425969100000001 2024-09-16T09:43:56.140 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.140 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.140 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for missing object", 2024-09-16T09:43:56.140 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.428759+0000", 2024-09-16T09:43:56.140 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00014022899999999999 2024-09-16T09:43:56.140 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.140 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.140 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:43:56.140 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.432901+0000", 2024-09-16T09:43:56.140 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0041425020000000002 2024-09-16T09:43:56.140 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.141 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.141 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for degraded object", 2024-09-16T09:43:56.141 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.432957+0000", 2024-09-16T09:43:56.141 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5377999999999997e-05 2024-09-16T09:43:56.141 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.141 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.141 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:43:56.141 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.462010+0000", 2024-09-16T09:43:56.141 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.029052923000000001 2024-09-16T09:43:56.141 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.141 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.141 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:43:56.141 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.462068+0000", 2024-09-16T09:43:56.142 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8959000000000003e-05 2024-09-16T09:43:56.142 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:43:56.142 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:43:56.142 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:43:56.142 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.142 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.142 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:85228 9.19 9:98aa15be:::benchmark_data_smithi070_137000_object85227:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e266)", 2024-09-16T09:43:56.142 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:34:30.376655+0000", 2024-09-16T09:43:56.142 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 565.75380899000004, 2024-09-16T09:43:56.142 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0565689490000001, 2024-09-16T09:43:56.142 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:43:56.142 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:43:56.142 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:43:56.143 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:43:56.143 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:43:56.143 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:43:56.143 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 85228 2024-09-16T09:43:56.143 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.143 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:43:56.143 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.143 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:43:56.143 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.376655+0000", 2024-09-16T09:43:56.143 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:43:56.143 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.143 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.143 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:43:56.143 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.376655+0000", 2024-09-16T09:43:56.143 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:43:56.144 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.144 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.144 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:43:56.144 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.376657+0000", 2024-09-16T09:43:56.144 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.4419999999999999e-06 2024-09-16T09:43:56.144 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.144 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.144 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:43:56.144 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.376680+0000", 2024-09-16T09:43:56.144 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.2923000000000001e-05 2024-09-16T09:43:56.144 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.144 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.144 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:43:56.145 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.376692+0000", 2024-09-16T09:43:56.145 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1861999999999999e-05 2024-09-16T09:43:56.145 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.145 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.145 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:43:56.145 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.376701+0000", 2024-09-16T09:43:56.145 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.0080000000000008e-06 2024-09-16T09:43:56.145 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.145 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.149 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:43:56.149 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.386042+0000", 2024-09-16T09:43:56.149 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0093412960000000007 2024-09-16T09:43:56.149 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.149 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.149 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:43:56.150 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.386048+0000", 2024-09-16T09:43:56.150 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.9170000000000003e-06 2024-09-16T09:43:56.150 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.150 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.150 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:43:56.150 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.429089+0000", 2024-09-16T09:43:56.150 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0430408870000001 2024-09-16T09:43:56.150 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.150 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.150 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:43:56.150 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.429187+0000", 2024-09-16T09:43:56.150 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.8227999999999995e-05 2024-09-16T09:43:56.150 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.150 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.150 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 5,6", 2024-09-16T09:43:56.150 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.429456+0000", 2024-09-16T09:43:56.151 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00026942699999999998 2024-09-16T09:43:56.151 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.151 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.151 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:43:56.151 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.433028+0000", 2024-09-16T09:43:56.151 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0035718199999999999 2024-09-16T09:43:56.151 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.151 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.151 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:43:56.151 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.433093+0000", 2024-09-16T09:43:56.151 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.4393000000000005e-05 2024-09-16T09:43:56.151 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.151 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.151 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:43:56.151 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.433126+0000", 2024-09-16T09:43:56.152 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3083000000000002e-05 2024-09-16T09:43:56.152 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.152 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.152 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:43:56.152 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.433160+0000", 2024-09-16T09:43:56.152 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4746999999999998e-05 2024-09-16T09:43:56.152 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.152 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.152 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:43:56.152 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.433224+0000", 2024-09-16T09:43:56.152 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.3916000000000003e-05 2024-09-16T09:43:56.152 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:43:56.152 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:43:56.152 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:43:56.153 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.153 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.153 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:85232 9.19 9:9ebfcfc9:::benchmark_data_smithi070_137000_object85231:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e266)", 2024-09-16T09:43:56.154 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:34:30.381300+0000", 2024-09-16T09:43:56.154 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 565.74916467900005, 2024-09-16T09:43:56.154 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0521562879999999, 2024-09-16T09:43:56.154 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:43:56.154 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:43:56.154 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:43:56.154 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:43:56.154 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:43:56.154 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:43:56.154 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 85232 2024-09-16T09:43:56.154 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.154 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:43:56.155 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.155 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:43:56.155 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.381300+0000", 2024-09-16T09:43:56.155 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:43:56.155 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.155 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.155 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:43:56.155 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.381300+0000", 2024-09-16T09:43:56.155 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:43:56.155 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.155 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.155 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:43:56.155 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.381301+0000", 2024-09-16T09:43:56.155 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6959999999999999e-06 2024-09-16T09:43:56.156 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.156 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.156 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:43:56.156 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.381325+0000", 2024-09-16T09:43:56.156 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.3213999999999998e-05 2024-09-16T09:43:56.156 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.156 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.156 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:43:56.156 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.381338+0000", 2024-09-16T09:43:56.156 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3237999999999999e-05 2024-09-16T09:43:56.156 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.156 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.156 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:43:56.157 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.381346+0000", 2024-09-16T09:43:56.157 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.8390000000000007e-06 2024-09-16T09:43:56.157 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.157 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.157 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:43:56.157 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.386067+0000", 2024-09-16T09:43:56.157 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0047214400000000004 2024-09-16T09:43:56.157 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.157 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.157 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:43:56.157 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.386073+0000", 2024-09-16T09:43:56.157 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5790000000000003e-06 2024-09-16T09:43:56.157 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.158 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.158 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:43:56.158 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.430684+0000", 2024-09-16T09:43:56.158 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.044610824 2024-09-16T09:43:56.158 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.158 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.158 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:43:56.158 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.430767+0000", 2024-09-16T09:43:56.158 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.3483e-05 2024-09-16T09:43:56.158 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.158 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.158 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 5,6", 2024-09-16T09:43:56.158 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.430988+0000", 2024-09-16T09:43:56.158 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00022084900000000001 2024-09-16T09:43:56.159 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.159 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.159 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:43:56.159 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.433032+0000", 2024-09-16T09:43:56.159 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0020444159999999999 2024-09-16T09:43:56.159 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.159 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.159 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:43:56.159 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.433282+0000", 2024-09-16T09:43:56.159 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000250024 2024-09-16T09:43:56.159 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.159 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.159 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:43:56.159 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.433335+0000", 2024-09-16T09:43:56.160 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.3096999999999997e-05 2024-09-16T09:43:56.160 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.160 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.160 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:43:56.160 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.433370+0000", 2024-09-16T09:43:56.160 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4465000000000003e-05 2024-09-16T09:43:56.160 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.160 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.160 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:43:56.160 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.433456+0000", 2024-09-16T09:43:56.160 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.6124000000000004e-05 2024-09-16T09:43:56.160 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:43:56.160 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:43:56.160 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:43:56.161 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.161 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.161 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:85266 9.19 9:9983710d:::benchmark_data_smithi070_137000_object85265:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e266)", 2024-09-16T09:43:56.161 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:34:30.409167+0000", 2024-09-16T09:43:56.161 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 565.72129687999995, 2024-09-16T09:43:56.161 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.029484174, 2024-09-16T09:43:56.161 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:43:56.161 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:43:56.161 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:43:56.161 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:43:56.161 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:43:56.161 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:43:56.161 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 85266 2024-09-16T09:43:56.162 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.162 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:43:56.162 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.162 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:43:56.162 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.409167+0000", 2024-09-16T09:43:56.162 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:43:56.162 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.162 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.162 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:43:56.162 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.409167+0000", 2024-09-16T09:43:56.162 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:43:56.162 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.162 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.162 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:43:56.163 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.409169+0000", 2024-09-16T09:43:56.163 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6339999999999999e-06 2024-09-16T09:43:56.163 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.163 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.163 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:43:56.163 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.409201+0000", 2024-09-16T09:43:56.163 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2193000000000002e-05 2024-09-16T09:43:56.163 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.163 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.163 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:43:56.163 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.409218+0000", 2024-09-16T09:43:56.163 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6481e-05 2024-09-16T09:43:56.163 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.163 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.163 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:43:56.164 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.409225+0000", 2024-09-16T09:43:56.164 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.6469999999999998e-06 2024-09-16T09:43:56.164 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.164 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.164 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:43:56.164 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.410186+0000", 2024-09-16T09:43:56.164 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000960673 2024-09-16T09:43:56.164 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.164 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.164 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:43:56.164 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:30.410198+0000", 2024-09-16T09:43:56.164 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1993e-05 2024-09-16T09:43:56.164 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.165 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.165 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:43:56.165 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.433567+0000", 2024-09-16T09:43:56.165 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.023368582 2024-09-16T09:43:56.165 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.165 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.165 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:43:56.165 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.433672+0000", 2024-09-16T09:43:56.165 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000105046 2024-09-16T09:43:56.165 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.165 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.165 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 5,6", 2024-09-16T09:43:56.165 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.433833+0000", 2024-09-16T09:43:56.165 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00016096599999999999 2024-09-16T09:43:56.166 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.166 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.166 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:43:56.166 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.438355+0000", 2024-09-16T09:43:56.166 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.004522206 2024-09-16T09:43:56.166 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.166 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.166 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:43:56.166 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.438417+0000", 2024-09-16T09:43:56.166 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1888000000000001e-05 2024-09-16T09:43:56.166 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.166 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.166 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:43:56.166 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.438515+0000", 2024-09-16T09:43:56.167 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.8084999999999993e-05 2024-09-16T09:43:56.167 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.167 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.167 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:43:56.167 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.438550+0000", 2024-09-16T09:43:56.167 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5172000000000001e-05 2024-09-16T09:43:56.167 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.167 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.167 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:43:56.167 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:34:31.438652+0000", 2024-09-16T09:43:56.167 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00010160799999999999 2024-09-16T09:43:56.167 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:43:56.168 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:43:56.168 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:43:56.168 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.168 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.168 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:211248 9.12 9:499944f5:::benchmark_data_smithi070_137000_object89537:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e327)", 2024-09-16T09:43:56.168 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:36:47.708810+0000", 2024-09-16T09:43:56.168 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 428.42165417299998, 2024-09-16T09:43:56.168 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0511326670000001, 2024-09-16T09:43:56.168 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:43:56.168 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:43:56.168 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:43:56.168 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:43:56.168 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:43:56.168 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:43:56.169 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 211248 2024-09-16T09:43:56.169 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.169 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:43:56.169 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.169 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:43:56.169 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.708810+0000", 2024-09-16T09:43:56.169 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:43:56.169 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.169 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.169 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:43:56.169 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.708810+0000", 2024-09-16T09:43:56.169 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:43:56.169 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.169 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.169 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:43:56.170 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.708812+0000", 2024-09-16T09:43:56.170 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.4780000000000001e-06 2024-09-16T09:43:56.170 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.170 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.170 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:43:56.170 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.708816+0000", 2024-09-16T09:43:56.170 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.7090000000000001e-06 2024-09-16T09:43:56.170 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.170 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.170 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:43:56.170 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.708827+0000", 2024-09-16T09:43:56.170 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0274e-05 2024-09-16T09:43:56.170 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.171 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.171 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:43:56.171 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.708833+0000", 2024-09-16T09:43:56.171 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.7660000000000001e-06 2024-09-16T09:43:56.171 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.171 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.171 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:43:56.171 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.711366+0000", 2024-09-16T09:43:56.171 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0025325529999999999 2024-09-16T09:43:56.171 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.171 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.171 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:43:56.171 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.711385+0000", 2024-09-16T09:43:56.172 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9368000000000002e-05 2024-09-16T09:43:56.172 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.172 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.172 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:43:56.172 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.748172+0000", 2024-09-16T09:43:56.172 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0367871070000001 2024-09-16T09:43:56.172 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.172 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.172 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:43:56.172 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.748254+0000", 2024-09-16T09:43:56.172 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.1648000000000002e-05 2024-09-16T09:43:56.172 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.172 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.173 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,2", 2024-09-16T09:43:56.173 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.748561+0000", 2024-09-16T09:43:56.173 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00030718299999999999 2024-09-16T09:43:56.173 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.173 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.173 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:43:56.173 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.755820+0000", 2024-09-16T09:43:56.173 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0072592109999999998 2024-09-16T09:43:56.173 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.173 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.173 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:43:56.173 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.759723+0000", 2024-09-16T09:43:56.173 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0039020249999999999 2024-09-16T09:43:56.174 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.174 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.174 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:43:56.174 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.759846+0000", 2024-09-16T09:43:56.174 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00012310000000000001 2024-09-16T09:43:56.174 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.174 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.174 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:43:56.174 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.759901+0000", 2024-09-16T09:43:56.174 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5315000000000001e-05 2024-09-16T09:43:56.174 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.174 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.174 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:43:56.174 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.759943+0000", 2024-09-16T09:43:56.175 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.193e-05 2024-09-16T09:43:56.175 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:43:56.175 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:43:56.175 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:43:56.175 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.175 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.175 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:211251 9.12 9:49db1cb8:::benchmark_data_smithi070_137000_object89540:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e327)", 2024-09-16T09:43:56.175 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:36:47.713400+0000", 2024-09-16T09:43:56.175 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 428.41706419799999, 2024-09-16T09:43:56.175 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0484086290000001, 2024-09-16T09:43:56.175 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:43:56.175 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:43:56.175 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:43:56.175 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:43:56.175 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:43:56.176 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:43:56.176 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 211251 2024-09-16T09:43:56.176 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.176 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:43:56.176 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.176 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:43:56.176 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.713400+0000", 2024-09-16T09:43:56.176 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:43:56.176 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.176 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.176 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:43:56.176 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.713400+0000", 2024-09-16T09:43:56.176 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:43:56.176 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.176 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.176 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:43:56.177 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.713402+0000", 2024-09-16T09:43:56.177 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7439999999999999e-06 2024-09-16T09:43:56.177 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.177 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.177 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:43:56.177 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.713407+0000", 2024-09-16T09:43:56.177 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4260000000000002e-06 2024-09-16T09:43:56.177 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.177 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.177 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:43:56.177 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.713423+0000", 2024-09-16T09:43:56.177 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5974999999999999e-05 2024-09-16T09:43:56.178 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.178 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.178 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:43:56.178 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.713431+0000", 2024-09-16T09:43:56.178 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.5920000000000003e-06 2024-09-16T09:43:56.178 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.178 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.178 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:43:56.178 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.713501+0000", 2024-09-16T09:43:56.178 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.0025999999999997e-05 2024-09-16T09:43:56.178 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.178 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.178 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:43:56.179 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.713510+0000", 2024-09-16T09:43:56.179 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.5089999999999999e-06 2024-09-16T09:43:56.179 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.179 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.179 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:43:56.179 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.755869+0000", 2024-09-16T09:43:56.179 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0423587969999999 2024-09-16T09:43:56.179 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.179 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.179 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:43:56.179 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.755973+0000", 2024-09-16T09:43:56.179 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000103688 2024-09-16T09:43:56.179 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.179 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.180 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,2", 2024-09-16T09:43:56.180 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.756051+0000", 2024-09-16T09:43:56.180 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.8528000000000004e-05 2024-09-16T09:43:56.180 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.180 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.180 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:43:56.180 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.759988+0000", 2024-09-16T09:43:56.180 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0039360829999999999 2024-09-16T09:43:56.180 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.180 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.180 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:43:56.180 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.760034+0000", 2024-09-16T09:43:56.180 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.6269e-05 2024-09-16T09:43:56.181 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.181 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.181 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:43:56.181 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.761690+0000", 2024-09-16T09:43:56.181 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0016558860000000001 2024-09-16T09:43:56.181 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.181 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.181 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:43:56.181 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.761768+0000", 2024-09-16T09:43:56.181 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.8317000000000002e-05 2024-09-16T09:43:56.181 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.181 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.181 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:43:56.182 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.761809+0000", 2024-09-16T09:43:56.182 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0788999999999999e-05 2024-09-16T09:43:56.182 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:43:56.182 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:43:56.182 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:43:56.182 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.182 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.182 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:211260 9.12 9:4be11eeb:::benchmark_data_smithi070_137000_object89549:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e327)", 2024-09-16T09:43:56.182 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:36:47.713450+0000", 2024-09-16T09:43:56.182 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 428.41701386, 2024-09-16T09:43:56.182 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0500658940000001, 2024-09-16T09:43:56.182 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:43:56.182 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:43:56.182 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:43:56.183 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:43:56.183 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:43:56.183 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:43:56.183 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 211260 2024-09-16T09:43:56.183 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.183 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:43:56.183 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.183 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:43:56.183 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.713450+0000", 2024-09-16T09:43:56.183 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:43:56.183 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.183 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.183 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:43:56.183 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.713450+0000", 2024-09-16T09:43:56.184 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:43:56.184 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.184 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.184 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:43:56.184 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.713451+0000", 2024-09-16T09:43:56.184 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.3000000000000001e-07 2024-09-16T09:43:56.184 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.184 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.184 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:43:56.184 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.713453+0000", 2024-09-16T09:43:56.184 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.4650000000000001e-06 2024-09-16T09:43:56.184 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.184 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.185 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:43:56.185 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.713464+0000", 2024-09-16T09:43:56.185 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0682999999999999e-05 2024-09-16T09:43:56.185 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.185 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.185 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:43:56.185 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.713469+0000", 2024-09-16T09:43:56.185 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.8749999999999999e-06 2024-09-16T09:43:56.185 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.185 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.185 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:43:56.186 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.714719+0000", 2024-09-16T09:43:56.186 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001249586 2024-09-16T09:43:56.186 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.186 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.186 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:43:56.186 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.714728+0000", 2024-09-16T09:43:56.186 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.2599999999999994e-06 2024-09-16T09:43:56.186 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.186 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.186 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:43:56.186 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.757394+0000", 2024-09-16T09:43:56.186 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0426663780000001 2024-09-16T09:43:56.186 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.187 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.187 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:43:56.187 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.757502+0000", 2024-09-16T09:43:56.187 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000107833 2024-09-16T09:43:56.187 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.187 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.187 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,2", 2024-09-16T09:43:56.187 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.757619+0000", 2024-09-16T09:43:56.187 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000117453 2024-09-16T09:43:56.187 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.187 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.188 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:43:56.188 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.761817+0000", 2024-09-16T09:43:56.188 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0041977969999999996 2024-09-16T09:43:56.188 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.188 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.188 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:43:56.188 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.763289+0000", 2024-09-16T09:43:56.188 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0014714890000000001 2024-09-16T09:43:56.188 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.188 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.188 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:43:56.188 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.763422+0000", 2024-09-16T09:43:56.188 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00013334600000000001 2024-09-16T09:43:56.189 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.189 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.189 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:43:56.189 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.763477+0000", 2024-09-16T09:43:56.189 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5334000000000002e-05 2024-09-16T09:43:56.189 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.189 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.189 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:43:56.189 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.763516+0000", 2024-09-16T09:43:56.189 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8964999999999997e-05 2024-09-16T09:43:56.189 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:43:56.189 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:43:56.189 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:43:56.189 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.190 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.190 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:211279 9.12 9:4baade64:::benchmark_data_smithi070_137000_object89568:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e327)", 2024-09-16T09:43:56.190 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:36:47.728918+0000", 2024-09-16T09:43:56.190 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 428.40154665799997, 2024-09-16T09:43:56.190 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0347446069999999, 2024-09-16T09:43:56.190 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:43:56.190 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:43:56.190 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:43:56.190 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:43:56.190 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:43:56.190 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:43:56.190 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 211279 2024-09-16T09:43:56.190 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.190 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:43:56.190 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.191 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:43:56.191 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.728918+0000", 2024-09-16T09:43:56.191 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:43:56.191 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.191 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.191 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:43:56.191 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.728918+0000", 2024-09-16T09:43:56.191 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:43:56.191 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.191 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.191 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:43:56.191 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.728919+0000", 2024-09-16T09:43:56.191 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.1620000000000001e-06 2024-09-16T09:43:56.191 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.192 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.192 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:43:56.192 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.728923+0000", 2024-09-16T09:43:56.192 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0609999999999997e-06 2024-09-16T09:43:56.192 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.192 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.192 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:43:56.192 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.728936+0000", 2024-09-16T09:43:56.192 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2891000000000001e-05 2024-09-16T09:43:56.192 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.192 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.192 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:43:56.192 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.728943+0000", 2024-09-16T09:43:56.192 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.9650000000000002e-06 2024-09-16T09:43:56.193 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.193 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.193 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:43:56.193 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.729002+0000", 2024-09-16T09:43:56.193 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.9392999999999999e-05 2024-09-16T09:43:56.193 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.193 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.193 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:43:56.193 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.729022+0000", 2024-09-16T09:43:56.193 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9752999999999999e-05 2024-09-16T09:43:56.193 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.193 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.193 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:43:56.194 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.758680+0000", 2024-09-16T09:43:56.194 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.029658484 2024-09-16T09:43:56.194 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.194 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.194 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:43:56.194 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.758761+0000", 2024-09-16T09:43:56.194 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.0319000000000001e-05 2024-09-16T09:43:56.194 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.194 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.194 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,2", 2024-09-16T09:43:56.194 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.758823+0000", 2024-09-16T09:43:56.194 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.2467999999999999e-05 2024-09-16T09:43:56.194 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.194 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.195 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:43:56.195 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.763303+0000", 2024-09-16T09:43:56.195 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0044794739999999998 2024-09-16T09:43:56.195 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.195 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.195 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:43:56.195 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.763362+0000", 2024-09-16T09:43:56.195 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.9268999999999998e-05 2024-09-16T09:43:56.195 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.195 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.195 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:43:56.195 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:43:56.197 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.763574+0000", 2024-09-16T09:43:56.197 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00021250800000000001 2024-09-16T09:43:56.197 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.197 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.197 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:43:56.197 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.763623+0000", 2024-09-16T09:43:56.197 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.8494e-05 2024-09-16T09:43:56.197 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.198 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.198 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:43:56.198 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.763662+0000", 2024-09-16T09:43:56.198 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.9366000000000002e-05 2024-09-16T09:43:56.198 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:43:56.198 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:43:56.198 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:43:56.198 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.198 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.198 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:211280 9.12 9:4ff46297:::benchmark_data_smithi070_137000_object89569:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e327)", 2024-09-16T09:43:56.198 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:36:47.730052+0000", 2024-09-16T09:43:56.198 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 428.40041219900002, 2024-09-16T09:43:56.198 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.03381397, 2024-09-16T09:43:56.198 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:43:56.199 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:43:56.199 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:43:56.199 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:43:56.199 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:43:56.199 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:43:56.199 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 211280 2024-09-16T09:43:56.199 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.199 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:43:56.199 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.199 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:43:56.199 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.730052+0000", 2024-09-16T09:43:56.199 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:43:56.199 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.199 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.200 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:43:56.200 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.730052+0000", 2024-09-16T09:43:56.200 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:43:56.200 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.200 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.200 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:43:56.200 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.730053+0000", 2024-09-16T09:43:56.200 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.108e-06 2024-09-16T09:43:56.200 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.200 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.201 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:43:56.201 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.730058+0000", 2024-09-16T09:43:56.201 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.4039999999999997e-06 2024-09-16T09:43:56.201 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.201 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.201 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:43:56.201 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.730109+0000", 2024-09-16T09:43:56.201 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.1700000000000003e-05 2024-09-16T09:43:56.201 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.201 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.201 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:43:56.202 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.730141+0000", 2024-09-16T09:43:56.202 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1483000000000003e-05 2024-09-16T09:43:56.202 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.202 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.202 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:43:56.202 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.730225+0000", 2024-09-16T09:43:56.202 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.3858999999999998e-05 2024-09-16T09:43:56.202 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.202 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.202 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:43:56.202 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.730233+0000", 2024-09-16T09:43:56.202 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.4030000000000007e-06 2024-09-16T09:43:56.202 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.202 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.202 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:43:56.203 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.760090+0000", 2024-09-16T09:43:56.203 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0298566229999999 2024-09-16T09:43:56.203 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.203 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.203 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:43:56.203 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.760195+0000", 2024-09-16T09:43:56.203 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000105503 2024-09-16T09:43:56.203 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.203 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.203 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,2", 2024-09-16T09:43:56.203 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.760353+0000", 2024-09-16T09:43:56.203 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00015804 2024-09-16T09:43:56.203 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.204 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.204 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:43:56.204 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.763526+0000", 2024-09-16T09:43:56.204 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0031731810000000002 2024-09-16T09:43:56.204 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.204 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.204 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:43:56.204 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.763723+0000", 2024-09-16T09:43:56.204 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00019647100000000001 2024-09-16T09:43:56.204 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.204 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.204 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:43:56.205 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.763770+0000", 2024-09-16T09:43:56.205 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.7450999999999997e-05 2024-09-16T09:43:56.205 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.205 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.205 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:43:56.205 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.763820+0000", 2024-09-16T09:43:56.205 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.9749000000000001e-05 2024-09-16T09:43:56.205 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.205 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.205 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:43:56.205 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.763866+0000", 2024-09-16T09:43:56.205 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5995000000000003e-05 2024-09-16T09:43:56.205 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:43:56.206 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:43:56.206 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:43:56.206 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.206 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.206 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:211328 9.12 9:487331f4:::benchmark_data_smithi070_137000_object89617:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e327)", 2024-09-16T09:43:56.206 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:36:47.762277+0000", 2024-09-16T09:43:56.206 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 428.36818692000003, 2024-09-16T09:43:56.206 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.007375701, 2024-09-16T09:43:56.206 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:43:56.206 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:43:56.206 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:43:56.206 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:43:56.206 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:43:56.206 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:43:56.207 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 211328 2024-09-16T09:43:56.207 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.207 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:43:56.207 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.207 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:43:56.207 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.762277+0000", 2024-09-16T09:43:56.207 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:43:56.207 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.207 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.207 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:43:56.207 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.762277+0000", 2024-09-16T09:43:56.207 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:43:56.207 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.207 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.208 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:43:56.208 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.762278+0000", 2024-09-16T09:43:56.208 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.9199999999999999e-07 2024-09-16T09:43:56.208 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.208 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.208 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:43:56.208 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.762283+0000", 2024-09-16T09:43:56.208 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.7190000000000001e-06 2024-09-16T09:43:56.208 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.208 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.208 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:43:56.208 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.762295+0000", 2024-09-16T09:43:56.208 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2009e-05 2024-09-16T09:43:56.209 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.209 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.209 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:43:56.209 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.762306+0000", 2024-09-16T09:43:56.209 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0730000000000001e-05 2024-09-16T09:43:56.209 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.209 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.209 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:43:56.209 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.762359+0000", 2024-09-16T09:43:56.209 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.3554999999999998e-05 2024-09-16T09:43:56.209 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.209 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.209 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:43:56.209 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.762370+0000", 2024-09-16T09:43:56.210 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0219e-05 2024-09-16T09:43:56.210 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.210 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.210 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:43:56.210 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.764182+0000", 2024-09-16T09:43:56.210 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.001812323 2024-09-16T09:43:56.210 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.210 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.210 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:43:56.210 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.764285+0000", 2024-09-16T09:43:56.210 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000102872 2024-09-16T09:43:56.210 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.210 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.211 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,2", 2024-09-16T09:43:56.211 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.764397+0000", 2024-09-16T09:43:56.211 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000111922 2024-09-16T09:43:56.211 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.211 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.211 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:43:56.211 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.769453+0000", 2024-09-16T09:43:56.211 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0050566739999999997 2024-09-16T09:43:56.211 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.211 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.211 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:43:56.211 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.769506+0000", 2024-09-16T09:43:56.211 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.2123999999999998e-05 2024-09-16T09:43:56.211 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.212 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.212 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:43:56.212 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.769526+0000", 2024-09-16T09:43:56.212 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.0806e-05 2024-09-16T09:43:56.212 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.212 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.212 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:43:56.212 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.769612+0000", 2024-09-16T09:43:56.212 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.5620999999999999e-05 2024-09-16T09:43:56.212 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.212 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.212 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:43:56.213 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.769653+0000", 2024-09-16T09:43:56.213 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.1134999999999998e-05 2024-09-16T09:43:56.213 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:43:56.213 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:43:56.213 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:43:56.213 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.213 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.213 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:211329 9.12 9:4b744b27:::benchmark_data_smithi070_137000_object89618:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e327)", 2024-09-16T09:43:56.213 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:36:47.764514+0000", 2024-09-16T09:43:56.213 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 428.36595020800002, 2024-09-16T09:43:56.213 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0096066159999999, 2024-09-16T09:43:56.213 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:43:56.213 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:43:56.214 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:43:56.214 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:43:56.214 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:43:56.214 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:43:56.214 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 211329 2024-09-16T09:43:56.214 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.214 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:43:56.214 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.214 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:43:56.214 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.764514+0000", 2024-09-16T09:43:56.214 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:43:56.214 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.214 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.214 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:43:56.214 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.764514+0000", 2024-09-16T09:43:56.215 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:43:56.215 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.215 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.215 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:43:56.215 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.764516+0000", 2024-09-16T09:43:56.215 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.4100000000000001e-06 2024-09-16T09:43:56.215 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.215 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.215 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:43:56.215 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.764521+0000", 2024-09-16T09:43:56.215 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.2599999999999996e-06 2024-09-16T09:43:56.215 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.215 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.215 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:43:56.215 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.764531+0000", 2024-09-16T09:43:56.216 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0322e-05 2024-09-16T09:43:56.216 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.216 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.216 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:43:56.216 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.764538+0000", 2024-09-16T09:43:56.216 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.0600000000000002e-06 2024-09-16T09:43:56.216 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.216 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.216 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:43:56.216 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.764618+0000", 2024-09-16T09:43:56.216 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.0177000000000002e-05 2024-09-16T09:43:56.216 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.216 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.216 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:43:56.217 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:47.764628+0000", 2024-09-16T09:43:56.217 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.5400000000000001e-06 2024-09-16T09:43:56.217 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.217 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.217 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:43:56.217 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.767496+0000", 2024-09-16T09:43:56.217 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0028676009999999 2024-09-16T09:43:56.217 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.217 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.217 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:43:56.217 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.767619+0000", 2024-09-16T09:43:56.217 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00012315699999999999 2024-09-16T09:43:56.217 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.218 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.218 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,2", 2024-09-16T09:43:56.218 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.767750+0000", 2024-09-16T09:43:56.218 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00013136499999999999 2024-09-16T09:43:56.218 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.218 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.218 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:43:56.218 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.773939+0000", 2024-09-16T09:43:56.218 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0061884619999999996 2024-09-16T09:43:56.218 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.218 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.218 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:43:56.218 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.773955+0000", 2024-09-16T09:43:56.218 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6376999999999999e-05 2024-09-16T09:43:56.219 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.219 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.219 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:43:56.219 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.774033+0000", 2024-09-16T09:43:56.219 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.8088000000000002e-05 2024-09-16T09:43:56.219 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.219 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.219 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:43:56.219 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.774082+0000", 2024-09-16T09:43:56.219 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.9354999999999998e-05 2024-09-16T09:43:56.219 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.219 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.219 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:43:56.219 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:36:48.774121+0000", 2024-09-16T09:43:56.220 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8442000000000002e-05 2024-09-16T09:43:56.220 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:43:56.220 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:43:56.220 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:43:56.220 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.220 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.220 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:220422 9.2 9.55f30b02 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e344)", 2024-09-16T09:43:56.220 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:37:04.372312+0000", 2024-09-16T09:43:56.220 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 411.75815274000001, 2024-09-16T09:43:56.220 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0049641709999999, 2024-09-16T09:43:56.220 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:43:56.220 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:43:56.220 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "reached pg", 2024-09-16T09:43:56.221 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:43:56.221 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:43:56.221 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:43:56.221 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 220422 2024-09-16T09:43:56.221 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.221 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:43:56.221 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.221 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:43:56.221 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:04.372312+0000", 2024-09-16T09:43:56.221 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:43:56.221 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.221 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.222 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:43:56.222 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:04.372312+0000", 2024-09-16T09:43:56.222 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:43:56.222 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.222 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.222 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:43:56.222 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:04.372313+0000", 2024-09-16T09:43:56.222 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6610000000000001e-06 2024-09-16T09:43:56.222 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.222 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.222 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:43:56.222 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:04.372318+0000", 2024-09-16T09:43:56.223 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.9830000000000004e-06 2024-09-16T09:43:56.223 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.223 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.223 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:43:56.223 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:04.372333+0000", 2024-09-16T09:43:56.223 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.4544e-05 2024-09-16T09:43:56.223 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.223 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.223 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:43:56.223 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:04.372341+0000", 2024-09-16T09:43:56.223 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.8650000000000001e-06 2024-09-16T09:43:56.223 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.223 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.223 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:43:56.224 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:04.372416+0000", 2024-09-16T09:43:56.224 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.4883999999999997e-05 2024-09-16T09:43:56.224 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.224 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.224 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:43:56.224 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:04.372426+0000", 2024-09-16T09:43:56.224 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.078e-05 2024-09-16T09:43:56.224 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.224 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.224 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:43:56.224 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:05.377221+0000", 2024-09-16T09:43:56.224 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.004794207 2024-09-16T09:43:56.224 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.225 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.225 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:43:56.225 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:05.377276+0000", 2024-09-16T09:43:56.225 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5247000000000001e-05 2024-09-16T09:43:56.225 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:43:56.225 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:43:56.225 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:43:56.225 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.225 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.225 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:220449 9.2 9:4d2f3ff5:::benchmark_data_smithi070_137000_object98738:head [delete] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e346)", 2024-09-16T09:43:56.225 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:37:06.366595+0000", 2024-09-16T09:43:56.225 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 409.76386938399997, 2024-09-16T09:43:56.225 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.005813316, 2024-09-16T09:43:56.226 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:43:56.226 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:43:56.226 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:43:56.226 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:43:56.226 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:43:56.226 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:43:56.226 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 220449 2024-09-16T09:43:56.226 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.226 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:43:56.226 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.226 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:43:56.226 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366595+0000", 2024-09-16T09:43:56.226 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:43:56.227 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.227 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.227 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:43:56.227 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366595+0000", 2024-09-16T09:43:56.227 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:43:56.227 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.227 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.227 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:43:56.227 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366596+0000", 2024-09-16T09:43:56.227 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.054e-06 2024-09-16T09:43:56.227 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.227 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.227 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:43:56.227 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366601+0000", 2024-09-16T09:43:56.228 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.7400000000000004e-06 2024-09-16T09:43:56.228 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.228 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.228 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:43:56.228 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366660+0000", 2024-09-16T09:43:56.228 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.9509999999999998e-05 2024-09-16T09:43:56.228 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.228 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.228 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:43:56.228 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366669+0000", 2024-09-16T09:43:56.228 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.0019999999999995e-06 2024-09-16T09:43:56.228 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.228 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.229 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:43:56.229 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.460193+0000", 2024-09-16T09:43:56.229 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.093523464000000001 2024-09-16T09:43:56.229 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.229 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.229 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:43:56.229 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.460206+0000", 2024-09-16T09:43:56.229 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3113e-05 2024-09-16T09:43:56.229 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.229 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.229 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:43:56.229 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.371051+0000", 2024-09-16T09:43:56.229 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.91084521600000001 2024-09-16T09:43:56.230 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.230 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.230 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:43:56.230 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.371151+0000", 2024-09-16T09:43:56.230 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.9918000000000001e-05 2024-09-16T09:43:56.230 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.230 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.230 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,2", 2024-09-16T09:43:56.230 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.371223+0000", 2024-09-16T09:43:56.230 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.2237000000000007e-05 2024-09-16T09:43:56.230 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.230 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.230 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:43:56.231 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.372257+0000", 2024-09-16T09:43:56.231 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001033422 2024-09-16T09:43:56.231 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.231 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.231 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:43:56.231 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.372276+0000", 2024-09-16T09:43:56.231 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9151999999999999e-05 2024-09-16T09:43:56.231 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.231 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.231 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:43:56.231 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.372326+0000", 2024-09-16T09:43:56.231 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0175e-05 2024-09-16T09:43:56.231 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.231 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.232 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:43:56.232 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.372384+0000", 2024-09-16T09:43:56.232 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7680999999999999e-05 2024-09-16T09:43:56.232 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.232 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.232 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:43:56.232 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.372408+0000", 2024-09-16T09:43:56.232 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.4632000000000001e-05 2024-09-16T09:43:56.232 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:43:56.232 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:43:56.232 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:43:56.232 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.232 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.233 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:220479 9.2 9:4adb387c:::benchmark_data_smithi070_137000_object98768:head [delete] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e346)", 2024-09-16T09:43:56.233 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:37:06.366688+0000", 2024-09-16T09:43:56.233 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 409.76377655800002, 2024-09-16T09:43:56.233 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0069119719999999, 2024-09-16T09:43:56.233 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:43:56.233 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:43:56.233 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:43:56.233 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:43:56.233 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:43:56.233 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:43:56.233 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 220479 2024-09-16T09:43:56.233 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.233 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:43:56.234 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.234 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:43:56.234 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366688+0000", 2024-09-16T09:43:56.234 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:43:56.234 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.234 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.234 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:43:56.234 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366688+0000", 2024-09-16T09:43:56.234 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:43:56.234 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.234 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.234 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:43:56.234 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366688+0000", 2024-09-16T09:43:56.234 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.6100000000000001e-07 2024-09-16T09:43:56.235 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.235 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.235 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:43:56.235 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366692+0000", 2024-09-16T09:43:56.235 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2969999999999999e-06 2024-09-16T09:43:56.235 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.235 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.235 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:43:56.235 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366699+0000", 2024-09-16T09:43:56.235 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.5299999999999999e-06 2024-09-16T09:43:56.235 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.235 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.235 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:43:56.235 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366705+0000", 2024-09-16T09:43:56.235 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4489999999999998e-06 2024-09-16T09:43:56.236 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.236 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.236 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:43:56.236 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.460555+0000", 2024-09-16T09:43:56.236 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.093850180000000005 2024-09-16T09:43:56.236 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.236 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.236 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:43:56.236 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.460562+0000", 2024-09-16T09:43:56.236 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.6819999999999997e-06 2024-09-16T09:43:56.236 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.236 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.236 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:43:56.236 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.371896+0000", 2024-09-16T09:43:56.237 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.91133457399999995 2024-09-16T09:43:56.237 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.237 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.237 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:43:56.237 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.371993+0000", 2024-09-16T09:43:56.237 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.6768999999999995e-05 2024-09-16T09:43:56.237 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.237 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.237 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,2", 2024-09-16T09:43:56.237 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.372058+0000", 2024-09-16T09:43:56.237 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.4678999999999995e-05 2024-09-16T09:43:56.237 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.237 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.237 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:43:56.238 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.373407+0000", 2024-09-16T09:43:56.238 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001349056 2024-09-16T09:43:56.238 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.238 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.238 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:43:56.238 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.373457+0000", 2024-09-16T09:43:56.238 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0291999999999999e-05 2024-09-16T09:43:56.238 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.238 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.238 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:43:56.238 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.373501+0000", 2024-09-16T09:43:56.238 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.4558999999999998e-05 2024-09-16T09:43:56.238 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.239 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.239 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:43:56.239 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.373558+0000", 2024-09-16T09:43:56.239 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.6956000000000002e-05 2024-09-16T09:43:56.239 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.239 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.239 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:43:56.239 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.373600+0000", 2024-09-16T09:43:56.239 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.1387999999999997e-05 2024-09-16T09:43:56.239 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:43:56.239 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:43:56.239 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:43:56.239 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.239 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.239 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:220549 9.2 9:4ee8c96f:::benchmark_data_smithi070_137000_object98838:head [delete] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e346)", 2024-09-16T09:43:56.240 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:37:06.366728+0000", 2024-09-16T09:43:56.240 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 409.76373608099999, 2024-09-16T09:43:56.240 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0076284369999999, 2024-09-16T09:43:56.240 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:43:56.240 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:43:56.240 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:43:56.240 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:43:56.240 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:43:56.240 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:43:56.240 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 220549 2024-09-16T09:43:56.240 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.240 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:43:56.240 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.241 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:43:56.241 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366728+0000", 2024-09-16T09:43:56.241 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:43:56.241 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.241 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.241 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:43:56.241 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366728+0000", 2024-09-16T09:43:56.241 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:43:56.241 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.241 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.241 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:43:56.241 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366729+0000", 2024-09-16T09:43:56.241 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8899999999999999e-07 2024-09-16T09:43:56.242 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.242 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.242 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:43:56.242 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366732+0000", 2024-09-16T09:43:56.242 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.717e-06 2024-09-16T09:43:56.242 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.242 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.242 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:43:56.242 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366739+0000", 2024-09-16T09:43:56.242 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.0400000000000004e-06 2024-09-16T09:43:56.242 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.242 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.242 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:43:56.242 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366744+0000", 2024-09-16T09:43:56.243 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4700000000000001e-06 2024-09-16T09:43:56.243 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.243 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.243 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:43:56.243 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.460597+0000", 2024-09-16T09:43:56.243 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.093853013999999998 2024-09-16T09:43:56.243 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.243 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.243 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:43:56.243 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.460606+0000", 2024-09-16T09:43:56.243 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.5490000000000006e-06 2024-09-16T09:43:56.243 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.243 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.243 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:43:56.244 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.372452+0000", 2024-09-16T09:43:56.244 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.91184652300000002 2024-09-16T09:43:56.244 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.244 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.244 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:43:56.244 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.372557+0000", 2024-09-16T09:43:56.244 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000104411 2024-09-16T09:43:56.244 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.244 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.244 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,2", 2024-09-16T09:43:56.244 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.372649+0000", 2024-09-16T09:43:56.244 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.1911999999999997e-05 2024-09-16T09:43:56.244 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.244 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.244 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:43:56.245 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.373670+0000", 2024-09-16T09:43:56.245 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001021903 2024-09-16T09:43:56.245 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.245 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.245 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:43:56.245 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.373683+0000", 2024-09-16T09:43:56.245 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.2655e-05 2024-09-16T09:43:56.245 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.245 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.245 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:43:56.245 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.374270+0000", 2024-09-16T09:43:56.245 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00058656799999999999 2024-09-16T09:43:56.245 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.246 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.246 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:43:56.246 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.374320+0000", 2024-09-16T09:43:56.246 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0627999999999997e-05 2024-09-16T09:43:56.246 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.246 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.246 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:43:56.246 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.374357+0000", 2024-09-16T09:43:56.246 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6458000000000002e-05 2024-09-16T09:43:56.246 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:43:56.246 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:43:56.246 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:43:56.246 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.247 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.247 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:220550 9.2 9:4bb12f00:::benchmark_data_smithi070_137000_object98839:head [delete] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e346)", 2024-09-16T09:43:56.247 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:37:06.366752+0000", 2024-09-16T09:43:56.247 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 409.76371247999998, 2024-09-16T09:43:56.247 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.00861194, 2024-09-16T09:43:56.247 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:43:56.247 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:43:56.247 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:43:56.247 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:43:56.247 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:43:56.247 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:43:56.247 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 220550 2024-09-16T09:43:56.247 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.247 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:43:56.248 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.248 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:43:56.248 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366752+0000", 2024-09-16T09:43:56.248 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:43:56.248 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.248 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.248 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:43:56.248 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366752+0000", 2024-09-16T09:43:56.248 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:43:56.248 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.248 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.248 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:43:56.248 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366752+0000", 2024-09-16T09:43:56.249 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.9999999999999999e-07 2024-09-16T09:43:56.249 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.249 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.249 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:43:56.249 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366754+0000", 2024-09-16T09:43:56.249 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.9439999999999999e-06 2024-09-16T09:43:56.249 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.249 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.249 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:43:56.249 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366760+0000", 2024-09-16T09:43:56.249 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8529999999999997e-06 2024-09-16T09:43:56.249 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.249 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.250 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:43:56.250 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366765+0000", 2024-09-16T09:43:56.250 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5399999999999997e-06 2024-09-16T09:43:56.250 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.250 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.250 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:43:56.250 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.460637+0000", 2024-09-16T09:43:56.250 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.093872609999999995 2024-09-16T09:43:56.250 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.250 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.250 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:43:56.250 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.460643+0000", 2024-09-16T09:43:56.251 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5160000000000002e-06 2024-09-16T09:43:56.251 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.251 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.251 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:43:56.251 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.372992+0000", 2024-09-16T09:43:56.251 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.91234923700000004 2024-09-16T09:43:56.251 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.251 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.251 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:43:56.251 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.373087+0000", 2024-09-16T09:43:56.251 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.4653000000000003e-05 2024-09-16T09:43:56.251 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.251 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.252 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,2", 2024-09-16T09:43:56.252 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.373154+0000", 2024-09-16T09:43:56.252 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.7946000000000003e-05 2024-09-16T09:43:56.252 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.252 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.252 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:43:56.252 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.374404+0000", 2024-09-16T09:43:56.252 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0012499570000000001 2024-09-16T09:43:56.252 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.252 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.252 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:43:56.252 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.374439+0000", 2024-09-16T09:43:56.252 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4578e-05 2024-09-16T09:43:56.253 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.253 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.253 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:43:56.253 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.375280+0000", 2024-09-16T09:43:56.253 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00084091799999999996 2024-09-16T09:43:56.253 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.253 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.253 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:43:56.253 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.375329+0000", 2024-09-16T09:43:56.253 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.9123999999999999e-05 2024-09-16T09:43:56.253 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.253 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.253 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:43:56.253 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.375364+0000", 2024-09-16T09:43:56.254 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4764000000000001e-05 2024-09-16T09:43:56.254 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:43:56.254 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:43:56.254 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:43:56.254 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.254 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.255 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:220634 9.2 9:497a55c3:::benchmark_data_smithi070_137000_object98923:head [delete] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e346)", 2024-09-16T09:43:56.255 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:37:06.366770+0000", 2024-09-16T09:43:56.255 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 409.76369395299997, 2024-09-16T09:43:56.255 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.010067807, 2024-09-16T09:43:56.255 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:43:56.255 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:43:56.255 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:43:56.256 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:43:56.256 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:43:56.256 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:43:56.256 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 220634 2024-09-16T09:43:56.256 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.256 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:43:56.256 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.256 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:43:56.256 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366770+0000", 2024-09-16T09:43:56.256 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:43:56.256 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.256 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.256 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:43:56.257 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366770+0000", 2024-09-16T09:43:56.257 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:43:56.257 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.257 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.257 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:43:56.257 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366771+0000", 2024-09-16T09:43:56.257 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.8099999999999999e-07 2024-09-16T09:43:56.257 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.257 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.257 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:43:56.257 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366777+0000", 2024-09-16T09:43:56.257 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.6120000000000004e-06 2024-09-16T09:43:56.258 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.258 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.258 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:43:56.258 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366783+0000", 2024-09-16T09:43:56.258 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8089999999999998e-06 2024-09-16T09:43:56.258 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.258 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.258 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:43:56.258 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366788+0000", 2024-09-16T09:43:56.258 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5210000000000004e-06 2024-09-16T09:43:56.258 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.258 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.258 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:43:56.259 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.460677+0000", 2024-09-16T09:43:56.259 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.093889792999999999 2024-09-16T09:43:56.259 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.259 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.259 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:43:56.259 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.460684+0000", 2024-09-16T09:43:56.259 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.8959999999999997e-06 2024-09-16T09:43:56.259 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.259 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.259 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:43:56.259 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.373725+0000", 2024-09-16T09:43:56.259 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.91304026400000005 2024-09-16T09:43:56.259 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.260 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.260 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:43:56.260 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.373825+0000", 2024-09-16T09:43:56.260 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000100238 2024-09-16T09:43:56.260 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.260 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.260 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,2", 2024-09-16T09:43:56.260 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.373937+0000", 2024-09-16T09:43:56.260 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000112669 2024-09-16T09:43:56.260 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.260 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.260 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:43:56.260 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.375849+0000", 2024-09-16T09:43:56.261 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001911966 2024-09-16T09:43:56.261 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.261 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.261 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:43:56.261 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.375893+0000", 2024-09-16T09:43:56.261 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.3689000000000001e-05 2024-09-16T09:43:56.261 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.261 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.261 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:43:56.261 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.376754+0000", 2024-09-16T09:43:56.261 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00086106599999999996 2024-09-16T09:43:56.261 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.261 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.262 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:43:56.262 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.376800+0000", 2024-09-16T09:43:56.262 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.5964999999999998e-05 2024-09-16T09:43:56.262 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.262 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.262 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:43:56.262 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.376838+0000", 2024-09-16T09:43:56.262 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.8037999999999997e-05 2024-09-16T09:43:56.262 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:43:56.262 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:43:56.262 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:43:56.262 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.262 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.263 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:220635 9.2 9:48fa68c0:::benchmark_data_smithi070_137000_object98924:head [delete] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e346)", 2024-09-16T09:43:56.263 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:37:06.366794+0000", 2024-09-16T09:43:56.263 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 409.76367065099998, 2024-09-16T09:43:56.263 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0102374599999999, 2024-09-16T09:43:56.263 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:43:56.263 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:43:56.263 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:43:56.263 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:43:56.263 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:43:56.263 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:43:56.263 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 220635 2024-09-16T09:43:56.263 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.263 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:43:56.264 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.264 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:43:56.264 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366794+0000", 2024-09-16T09:43:56.264 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:43:56.264 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.264 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.264 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:43:56.264 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366794+0000", 2024-09-16T09:43:56.264 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:43:56.264 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.264 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.264 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:43:56.264 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366794+0000", 2024-09-16T09:43:56.265 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.1699999999999999e-07 2024-09-16T09:43:56.265 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.265 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.265 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:43:56.265 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366796+0000", 2024-09-16T09:43:56.265 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.792e-06 2024-09-16T09:43:56.265 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.265 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.265 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:43:56.265 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366801+0000", 2024-09-16T09:43:56.265 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5999999999999997e-06 2024-09-16T09:43:56.265 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.265 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.265 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:43:56.266 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366806+0000", 2024-09-16T09:43:56.266 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.7469999999999996e-06 2024-09-16T09:43:56.266 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.266 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.266 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:43:56.266 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.460714+0000", 2024-09-16T09:43:56.266 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.093907861999999995 2024-09-16T09:43:56.266 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.266 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.266 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:43:56.266 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.460719+0000", 2024-09-16T09:43:56.266 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4070000000000001e-06 2024-09-16T09:43:56.266 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.267 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.267 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:43:56.267 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.374493+0000", 2024-09-16T09:43:56.267 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.91377397100000002 2024-09-16T09:43:56.267 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.267 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.267 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:43:56.267 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.374586+0000", 2024-09-16T09:43:56.267 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.2905000000000006e-05 2024-09-16T09:43:56.267 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.267 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.267 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,2", 2024-09-16T09:43:56.267 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.374680+0000", 2024-09-16T09:43:56.268 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.3769000000000003e-05 2024-09-16T09:43:56.268 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.268 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.268 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:43:56.268 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.375855+0000", 2024-09-16T09:43:56.268 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001175073 2024-09-16T09:43:56.268 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.268 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.268 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:43:56.268 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.376878+0000", 2024-09-16T09:43:56.268 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0010223820000000001 2024-09-16T09:43:56.268 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.268 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.268 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:43:56.269 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.376950+0000", 2024-09-16T09:43:56.269 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.1966999999999997e-05 2024-09-16T09:43:56.269 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.269 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.269 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:43:56.269 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.376997+0000", 2024-09-16T09:43:56.269 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.7987e-05 2024-09-16T09:43:56.269 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.269 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.269 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:43:56.269 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.377031+0000", 2024-09-16T09:43:56.269 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.3680999999999998e-05 2024-09-16T09:43:56.269 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:43:56.270 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:43:56.270 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:43:56.270 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.270 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.270 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:220638 9.2 9:4df3d7d7:::benchmark_data_smithi070_137000_object98927:head [delete] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e346)", 2024-09-16T09:43:56.270 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:37:06.366812+0000", 2024-09-16T09:43:56.270 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 409.763652541, 2024-09-16T09:43:56.270 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0103380449999999, 2024-09-16T09:43:56.270 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:43:56.270 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:43:56.270 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:43:56.270 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:43:56.270 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:43:56.271 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:43:56.271 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 220638 2024-09-16T09:43:56.271 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.271 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:43:56.271 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.271 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:43:56.271 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366812+0000", 2024-09-16T09:43:56.271 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:43:56.271 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.271 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.271 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:43:56.271 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366812+0000", 2024-09-16T09:43:56.271 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:43:56.271 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.272 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.272 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:43:56.272 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366812+0000", 2024-09-16T09:43:56.272 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.58e-07 2024-09-16T09:43:56.272 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.272 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.272 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:43:56.272 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366814+0000", 2024-09-16T09:43:56.272 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6670000000000001e-06 2024-09-16T09:43:56.272 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.272 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.272 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:43:56.272 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366820+0000", 2024-09-16T09:43:56.272 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.6799999999999998e-06 2024-09-16T09:43:56.273 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.273 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.273 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:43:56.273 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366824+0000", 2024-09-16T09:43:56.273 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.7400000000000004e-06 2024-09-16T09:43:56.273 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.273 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.273 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:43:56.273 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.460752+0000", 2024-09-16T09:43:56.273 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.093927963000000003 2024-09-16T09:43:56.273 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.273 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.273 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:43:56.274 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.460759+0000", 2024-09-16T09:43:56.274 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.7989999999999996e-06 2024-09-16T09:43:56.274 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.274 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.274 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:43:56.274 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.374896+0000", 2024-09-16T09:43:56.274 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.91413692400000002 2024-09-16T09:43:56.274 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.274 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.274 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:43:56.274 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.374996+0000", 2024-09-16T09:43:56.274 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.9656999999999998e-05 2024-09-16T09:43:56.275 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.275 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.275 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,2", 2024-09-16T09:43:56.275 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.375097+0000", 2024-09-16T09:43:56.275 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000101109 2024-09-16T09:43:56.275 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.275 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.275 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:43:56.275 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.376713+0000", 2024-09-16T09:43:56.275 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0016159530000000001 2024-09-16T09:43:56.275 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.275 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.275 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:43:56.275 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.376914+0000", 2024-09-16T09:43:56.276 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00020110700000000001 2024-09-16T09:43:56.276 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.276 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.276 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:43:56.276 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.377068+0000", 2024-09-16T09:43:56.276 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00015383500000000001 2024-09-16T09:43:56.276 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.276 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.276 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:43:56.276 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.377115+0000", 2024-09-16T09:43:56.276 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.7604e-05 2024-09-16T09:43:56.276 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.276 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.277 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:43:56.277 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.377150+0000", 2024-09-16T09:43:56.277 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.4649e-05 2024-09-16T09:43:56.277 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:43:56.277 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:43:56.277 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:43:56.277 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.277 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.277 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "osd_op(client.15588.0:220657 9.2 9:4cd451a8:::benchmark_data_smithi070_137000_object98946:head [delete] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e346)", 2024-09-16T09:43:56.277 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:37:06.366830+0000", 2024-09-16T09:43:56.277 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 409.76363440099999, 2024-09-16T09:43:56.277 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.010672051, 2024-09-16T09:43:56.277 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:43:56.277 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:43:56.278 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:43:56.278 INFO:teuthology.orchestra.run.smithi070.stdout: "client_info": { 2024-09-16T09:43:56.278 INFO:teuthology.orchestra.run.smithi070.stdout: "client": "client.15588", 2024-09-16T09:43:56.278 INFO:teuthology.orchestra.run.smithi070.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:43:56.278 INFO:teuthology.orchestra.run.smithi070.stdout: "tid": 220657 2024-09-16T09:43:56.278 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.278 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:43:56.278 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.278 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:43:56.278 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366830+0000", 2024-09-16T09:43:56.278 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:43:56.278 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.278 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.278 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:43:56.279 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366830+0000", 2024-09-16T09:43:56.279 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:43:56.279 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.279 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.279 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:43:56.279 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366830+0000", 2024-09-16T09:43:56.279 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.65e-07 2024-09-16T09:43:56.279 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.279 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.279 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:43:56.279 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366832+0000", 2024-09-16T09:43:56.279 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6479999999999999e-06 2024-09-16T09:43:56.279 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.279 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.280 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:43:56.280 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366838+0000", 2024-09-16T09:43:56.280 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.6640000000000003e-06 2024-09-16T09:43:56.280 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.280 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.280 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:43:56.280 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.366847+0000", 2024-09-16T09:43:56.280 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.8670000000000006e-06 2024-09-16T09:43:56.280 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.280 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.280 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:43:56.280 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.460788+0000", 2024-09-16T09:43:56.280 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.093940896999999995 2024-09-16T09:43:56.281 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.281 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.281 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for peered", 2024-09-16T09:43:56.281 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:06.460795+0000", 2024-09-16T09:43:56.281 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.5699999999999998e-06 2024-09-16T09:43:56.281 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.281 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.281 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:43:56.281 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.375406+0000", 2024-09-16T09:43:56.281 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.91461130199999996 2024-09-16T09:43:56.281 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.281 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.281 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:43:56.282 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.375587+0000", 2024-09-16T09:43:56.282 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00018084700000000001 2024-09-16T09:43:56.282 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.282 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.282 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "waiting for subops from 1,2", 2024-09-16T09:43:56.282 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.375684+0000", 2024-09-16T09:43:56.282 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.7095999999999993e-05 2024-09-16T09:43:56.282 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.282 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.282 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "op_commit", 2024-09-16T09:43:56.282 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.376717+0000", 2024-09-16T09:43:56.282 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0010328729999999999 2024-09-16T09:43:56.282 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.282 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.283 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:43:56.283 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.377357+0000", 2024-09-16T09:43:56.283 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00063957 2024-09-16T09:43:56.283 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.283 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.283 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:43:56.283 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.377417+0000", 2024-09-16T09:43:56.283 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.0062999999999997e-05 2024-09-16T09:43:56.283 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.283 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.283 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "commit_sent", 2024-09-16T09:43:56.283 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.377466+0000", 2024-09-16T09:43:56.283 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.9407999999999999e-05 2024-09-16T09:43:56.283 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:43:56.284 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:43:56.284 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:43:56.284 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:37:07.377502+0000", 2024-09-16T09:43:56.284 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.5981e-05 2024-09-16T09:43:56.284 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:43:56.284 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:43:56.284 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:43:56.284 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:43:56.284 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:43:56.284 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T09:43:56.284 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:56 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2858412168' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:43:56.285 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:56 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/2858412168' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:43:56.287 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:56.287 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:56.288 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:43:56.388 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:43:56.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:56 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/2858412168' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:43:56.424 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:56.425 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:56.425 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:43:56.526 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:43:56.563 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:56.563 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:56.563 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:43:56.664 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:43:56.695 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:56.696 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:56.697 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:43:56.797 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:43:56.836 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:56.837 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:56.837 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:43:56.938 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:43:56.974 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:56.975 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:56.975 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:43:57.076 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:43:57.108 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:57.109 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:57.109 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:43:57.148 INFO:teuthology.orchestra.run.smithi070.stderr:nodeep-scrub is unset 2024-09-16T09:43:57.210 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:43:57.245 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:57.246 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:57.246 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:43:57.347 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:43:57.381 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:57.381 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:57.382 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:43:57.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:57 smithi084 ceph-mon[32629]: pgmap v991: 168 pgs: 31 active+undersized, 18 active+undersized+degraded, 119 active+clean; 2.6 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.44k op/s; 15990/127860 objects degraded (12.506%) 2024-09-16T09:43:57.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:57 smithi084 ceph-mon[32629]: 1.7 scrub starts 2024-09-16T09:43:57.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:57 smithi084 ceph-mon[32629]: 1.7 scrub ok 2024-09-16T09:43:57.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:57 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1766837978' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:43:57.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:57 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:43:57.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:57 smithi070 ceph-mon[30388]: pgmap v991: 168 pgs: 31 active+undersized, 18 active+undersized+degraded, 119 active+clean; 2.6 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.44k op/s; 15990/127860 objects degraded (12.506%) 2024-09-16T09:43:57.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:57 smithi070 ceph-mon[30388]: 1.7 scrub starts 2024-09-16T09:43:57.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:57 smithi070 ceph-mon[30388]: 1.7 scrub ok 2024-09-16T09:43:57.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:57 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1766837978' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:43:57.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:57 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:43:57.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:57 smithi070 ceph-mon[40691]: pgmap v991: 168 pgs: 31 active+undersized, 18 active+undersized+degraded, 119 active+clean; 2.6 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.44k op/s; 15990/127860 objects degraded (12.506%) 2024-09-16T09:43:57.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:57 smithi070 ceph-mon[40691]: 1.7 scrub starts 2024-09-16T09:43:57.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:57 smithi070 ceph-mon[40691]: 1.7 scrub ok 2024-09-16T09:43:57.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:57 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1766837978' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:43:57.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:57 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:43:57.482 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:43:57.519 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:57.519 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:57.520 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:43:57.620 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:43:57.652 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:57.653 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:57.653 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:43:57.754 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:43:57.786 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:57.786 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:57.787 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:43:57.888 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:43:57.917 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:57.918 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:57.919 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:43:58.019 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:43:58.051 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:58.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:58.052 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:43:58.153 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:43:58.189 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:58.190 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:58.190 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:43:58.291 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:43:58.323 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:58.324 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:58.324 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:43:58.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:58 smithi084 ceph-mon[32629]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-16T09:43:58.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:58 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-16T09:43:58.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:58 smithi084 ceph-mon[32629]: osdmap e510: 8 total, 7 up, 8 in 2024-09-16T09:43:58.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:58 smithi070 ceph-mon[30388]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-16T09:43:58.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:58 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-16T09:43:58.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:58 smithi070 ceph-mon[30388]: osdmap e510: 8 total, 7 up, 8 in 2024-09-16T09:43:58.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:58 smithi070 ceph-mon[40691]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-16T09:43:58.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:58 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-16T09:43:58.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:58 smithi070 ceph-mon[40691]: osdmap e510: 8 total, 7 up, 8 in 2024-09-16T09:43:58.426 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:43:58.458 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:58.458 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:58.459 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:43:58.559 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:43:58.593 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:58.593 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:58.594 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:43:58.694 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:43:58.729 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:58.730 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:58.730 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:43:58.831 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:43:58.854 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.4 dump_ops_in_flight 2024-09-16T09:43:58.862 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:58.863 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:58.863 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:43:58.964 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:43:59.009 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:59.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:59.010 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:43:59.111 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:43:59.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:59 smithi084 ceph-mon[32629]: pgmap v993: 168 pgs: 31 active+undersized, 18 active+undersized+degraded, 119 active+clean; 2.5 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.46k op/s; 15016/122046 objects degraded (12.304%) 2024-09-16T09:43:59.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:43:59 smithi084 ceph-mon[32629]: 11.9 scrub starts 2024-09-16T09:43:59.161 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:59.161 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:59.162 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:43:59.262 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:43:59.292 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:59.293 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:59.293 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:43:59.317 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:43:59.329 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T09:43:59.329 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T09:43:59.394 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:43:59.403 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:59 smithi070 ceph-mon[30388]: pgmap v993: 168 pgs: 31 active+undersized, 18 active+undersized+degraded, 119 active+clean; 2.5 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.46k op/s; 15016/122046 objects degraded (12.304%) 2024-09-16T09:43:59.403 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:43:59 smithi070 ceph-mon[30388]: 11.9 scrub starts 2024-09-16T09:43:59.404 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:59 smithi070 ceph-mon[40691]: pgmap v993: 168 pgs: 31 active+undersized, 18 active+undersized+degraded, 119 active+clean; 2.5 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.46k op/s; 15016/122046 objects degraded (12.304%) 2024-09-16T09:43:59.404 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:43:59 smithi070 ceph-mon[40691]: 11.9 scrub starts 2024-09-16T09:43:59.442 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:59.443 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:59.443 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:43:59.544 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:43:59.571 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:59.572 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:59.572 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:43:59.673 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:43:59.703 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:59.704 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:59.704 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:43:59.806 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:43:59.851 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:43:59.852 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:59.852 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:43:59.953 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:43:59.988 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:43:59.989 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:43:59.989 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:43:59.992 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-09-16T09:43:59.992 INFO:teuthology.orchestra.run.smithi084.stdout: "ops": [ 2024-09-16T09:43:59.992 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:43:59.992 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.16305.0:228785 11.e 11:73095f71:::benchmark_data_smithi070_191495_object98315:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e510)", 2024-09-16T09:43:59.992 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:43:59.982454+0000", 2024-09-16T09:43:59.993 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 0.0075374559999999997, 2024-09-16T09:43:59.993 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0075687189999999998, 2024-09-16T09:43:59.993 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:43:59.993 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:43:59.993 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "waiting for sub ops", 2024-09-16T09:43:59.993 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:43:59.993 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.16305", 2024-09-16T09:43:59.993 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3818031514", 2024-09-16T09:43:59.993 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 228785 2024-09-16T09:43:59.993 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:43:59.993 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:43:59.993 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:43:59.993 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:43:59.993 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:43:59.982454+0000", 2024-09-16T09:43:59.994 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:43:59.994 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:43:59.994 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:43:59.994 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:43:59.994 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:43:59.982454+0000", 2024-09-16T09:43:59.994 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:43:59.994 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:43:59.994 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:43:59.994 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:43:59.994 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:43:59.982455+0000", 2024-09-16T09:43:59.994 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.4720000000000001e-06 2024-09-16T09:43:59.994 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:43:59.994 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:43:59.995 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:43:59.995 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:43:59.982461+0000", 2024-09-16T09:43:59.995 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.3680000000000001e-06 2024-09-16T09:43:59.995 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:43:59.995 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:43:59.995 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:43:59.995 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:43:59.982477+0000", 2024-09-16T09:43:59.995 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.6150999999999998e-05 2024-09-16T09:43:59.995 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:43:59.995 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:43:59.995 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:43:59.995 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:43:59.982485+0000", 2024-09-16T09:43:59.995 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.0919999999999998e-06 2024-09-16T09:43:59.995 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:43:59.996 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:43:59.996 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:43:59.996 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:43:59.982541+0000", 2024-09-16T09:43:59.996 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.5838999999999998e-05 2024-09-16T09:43:59.996 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:43:59.996 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:43:59.996 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:43:59.996 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:43:59.982661+0000", 2024-09-16T09:43:59.996 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00012015699999999999 2024-09-16T09:43:59.996 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:43:59.996 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:43:59.996 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 1,7", 2024-09-16T09:43:59.996 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:43:59.982783+0000", 2024-09-16T09:43:59.996 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000121521 2024-09-16T09:43:59.997 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:43:59.997 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:43:59.997 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:43:59.997 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:43:59.985726+0000", 2024-09-16T09:43:59.997 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0029436060000000001 2024-09-16T09:43:59.997 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:43:59.997 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:43:59.997 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:43:59.997 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:43:59.986577+0000", 2024-09-16T09:43:59.997 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00085039100000000004 2024-09-16T09:43:59.997 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:43:59.997 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:43:59.997 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:43:59.997 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:43:59.998 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:43:59.998 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.16305.0:228793 11.c 11:3c01e4e3:::benchmark_data_smithi070_191495_object98323:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e510)", 2024-09-16T09:43:59.998 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:43:59.987794+0000", 2024-09-16T09:43:59.998 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 0.0021977870000000001, 2024-09-16T09:43:59.998 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0023105740000000001, 2024-09-16T09:43:59.998 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:43:59.998 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:43:59.998 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "waiting for sub ops", 2024-09-16T09:43:59.998 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:43:59.998 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.16305", 2024-09-16T09:43:59.998 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/3818031514", 2024-09-16T09:43:59.998 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 228793 2024-09-16T09:43:59.998 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:43:59.998 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:43:59.999 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:43:59.999 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:43:59.999 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:43:59.987794+0000", 2024-09-16T09:43:59.999 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:43:59.999 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:43:59.999 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:43:59.999 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:43:59.999 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:43:59.987794+0000", 2024-09-16T09:43:59.999 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:43:59.999 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:43:59.999 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:43:59.999 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:43:59.999 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:43:59.987795+0000", 2024-09-16T09:43:59.999 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.73e-06 2024-09-16T09:44:00.000 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:00.000 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:00.000 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:44:00.000 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:43:59.987801+0000", 2024-09-16T09:44:00.000 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.7810000000000002e-06 2024-09-16T09:44:00.000 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:00.000 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:00.000 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:44:00.000 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:43:59.987819+0000", 2024-09-16T09:44:00.000 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.7424000000000001e-05 2024-09-16T09:44:00.000 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:00.000 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:00.000 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:44:00.000 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:43:59.987826+0000", 2024-09-16T09:44:00.001 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.8569999999999995e-06 2024-09-16T09:44:00.001 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:00.001 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:00.001 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:44:00.001 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:43:59.987886+0000", 2024-09-16T09:44:00.001 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.9419999999999997e-05 2024-09-16T09:44:00.001 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:00.001 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:00.001 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:44:00.001 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:43:59.988016+0000", 2024-09-16T09:44:00.001 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000130195 2024-09-16T09:44:00.001 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:00.001 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:00.001 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 1", 2024-09-16T09:44:00.001 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:43:59.988128+0000", 2024-09-16T09:44:00.002 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000111912 2024-09-16T09:44:00.002 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:44:00.002 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:44:00.002 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:44:00.002 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:00.002 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:00.002 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_repop_reply(client.16305.0:228785 11.e e510/454 ondisk, result = 0)", 2024-09-16T09:44:00.002 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:43:59.989953+0000", 2024-09-16T09:44:00.002 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 3.8334999999999998e-05, 2024-09-16T09:44:00.002 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00021246200000000001, 2024-09-16T09:44:00.002 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:44:00.002 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:44:00.002 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "started", 2024-09-16T09:44:00.002 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:44:00.003 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:00.003 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:44:00.003 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:43:59.989953+0000", 2024-09-16T09:44:00.003 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:44:00.003 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:00.003 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:00.003 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:44:00.003 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:43:59.989953+0000", 2024-09-16T09:44:00.003 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:44:00.003 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:00.003 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:00.003 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:44:00.003 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:43:59.989954+0000", 2024-09-16T09:44:00.003 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.9399999999999993e-07 2024-09-16T09:44:00.003 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:00.004 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:00.004 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:44:00.004 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:43:59.989959+0000", 2024-09-16T09:44:00.004 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.5709999999999999e-06 2024-09-16T09:44:00.004 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:00.004 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:00.004 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:44:00.004 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:43:59.989974+0000", 2024-09-16T09:44:00.004 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.5770999999999999e-05 2024-09-16T09:44:00.004 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:00.004 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:00.004 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:44:00.004 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:43:59.989982+0000", 2024-09-16T09:44:00.004 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.9219999999999997e-06 2024-09-16T09:44:00.005 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:00.005 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:00.005 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:44:00.005 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:43:59.990061+0000", 2024-09-16T09:44:00.005 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.9122999999999994e-05 2024-09-16T09:44:00.005 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:00.005 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:00.005 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:44:00.005 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:43:59.990069+0000", 2024-09-16T09:44:00.005 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.8329999999999994e-06 2024-09-16T09:44:00.005 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:44:00.005 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:44:00.005 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:44:00.005 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:44:00.005 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-09-16T09:44:00.006 INFO:teuthology.orchestra.run.smithi084.stdout: "num_ops": 3 2024-09-16T09:44:00.006 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-09-16T09:44:00.089 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:44:00.124 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:00.125 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:00.125 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:44:00.226 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:44:00.261 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:00.262 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:00.263 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:44:00.364 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:44:00.398 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:00.398 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:00.399 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:44:00.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:00 smithi084 ceph-mon[32629]: 11.9 scrub ok 2024-09-16T09:44:00.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:00 smithi084 ceph-mon[32629]: Health check update: Degraded data redundancy: 15016/122046 objects degraded (12.304%), 18 pgs degraded (PG_DEGRADED) 2024-09-16T09:44:00.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:00 smithi070 ceph-mon[30388]: 11.9 scrub ok 2024-09-16T09:44:00.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:00 smithi070 ceph-mon[30388]: Health check update: Degraded data redundancy: 15016/122046 objects degraded (12.304%), 18 pgs degraded (PG_DEGRADED) 2024-09-16T09:44:00.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:00 smithi070 ceph-mon[40691]: 11.9 scrub ok 2024-09-16T09:44:00.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:00 smithi070 ceph-mon[40691]: Health check update: Degraded data redundancy: 15016/122046 objects degraded (12.304%), 18 pgs degraded (PG_DEGRADED) 2024-09-16T09:44:00.499 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:44:00.536 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:00.537 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:00.537 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:44:00.611 INFO:teuthology.orchestra.run.smithi070.stderr:Creating an OSD daemon form without an OSD FSID value 2024-09-16T09:44:00.637 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:44:00.676 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:00.677 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:00.677 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:44:00.778 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:44:00.816 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:00.816 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:00.816 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:44:00.857 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.4 dump_blocked_ops 2024-09-16T09:44:00.918 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:44:01.007 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:01.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:01.008 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:44:01.109 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:44:01.154 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:01.155 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:01.156 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:44:01.181 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:01 smithi070 ceph-mon[30388]: 4.1a scrub starts 2024-09-16T09:44:01.181 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:01 smithi070 ceph-mon[30388]: 4.1a scrub ok 2024-09-16T09:44:01.181 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:01 smithi070 ceph-mon[30388]: pgmap v994: 168 pgs: 31 active+undersized, 18 active+undersized+degraded, 119 active+clean; 2.1 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.48k op/s; 13070/104736 objects degraded (12.479%) 2024-09-16T09:44:01.181 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:01 smithi070 ceph-mon[30388]: 3.7 scrub starts 2024-09-16T09:44:01.181 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:01 smithi070 ceph-mon[30388]: 3.7 scrub ok 2024-09-16T09:44:01.181 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:01 smithi070 ceph-mon[30388]: 5.3 scrub starts 2024-09-16T09:44:01.181 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:01 smithi070 ceph-mon[30388]: 5.3 scrub ok 2024-09-16T09:44:01.181 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:01 smithi070 ceph-mon[40691]: 4.1a scrub starts 2024-09-16T09:44:01.181 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:01 smithi070 ceph-mon[40691]: 4.1a scrub ok 2024-09-16T09:44:01.182 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:01 smithi070 ceph-mon[40691]: pgmap v994: 168 pgs: 31 active+undersized, 18 active+undersized+degraded, 119 active+clean; 2.1 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.48k op/s; 13070/104736 objects degraded (12.479%) 2024-09-16T09:44:01.182 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:01 smithi070 ceph-mon[40691]: 3.7 scrub starts 2024-09-16T09:44:01.182 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:01 smithi070 ceph-mon[40691]: 3.7 scrub ok 2024-09-16T09:44:01.182 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:01 smithi070 ceph-mon[40691]: 5.3 scrub starts 2024-09-16T09:44:01.182 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:01 smithi070 ceph-mon[40691]: 5.3 scrub ok 2024-09-16T09:44:01.256 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:44:01.309 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:01.310 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:01.311 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:44:01.330 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T09:44:01.337 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:44:01.411 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:44:01.412 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:01 smithi084 ceph-mon[32629]: 4.1a scrub starts 2024-09-16T09:44:01.412 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:01 smithi084 ceph-mon[32629]: 4.1a scrub ok 2024-09-16T09:44:01.412 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:01 smithi084 ceph-mon[32629]: pgmap v994: 168 pgs: 31 active+undersized, 18 active+undersized+degraded, 119 active+clean; 2.1 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.48k op/s; 13070/104736 objects degraded (12.479%) 2024-09-16T09:44:01.412 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:01 smithi084 ceph-mon[32629]: 3.7 scrub starts 2024-09-16T09:44:01.412 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:01 smithi084 ceph-mon[32629]: 3.7 scrub ok 2024-09-16T09:44:01.412 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:01 smithi084 ceph-mon[32629]: 5.3 scrub starts 2024-09-16T09:44:01.412 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:01 smithi084 ceph-mon[32629]: 5.3 scrub ok 2024-09-16T09:44:01.455 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:01.455 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:01.456 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:44:01.556 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:44:01.596 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:01.597 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:01.630 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:44:01.731 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:44:01.737 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:44:01.772 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:01.773 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:01.773 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:44:01.778 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd set noscrub 2024-09-16T09:44:01.874 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:44:01.908 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:01.909 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:01.909 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:44:01.961 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-09-16T09:44:01.961 INFO:teuthology.orchestra.run.smithi084.stdout: "ops": [], 2024-09-16T09:44:01.961 INFO:teuthology.orchestra.run.smithi084.stdout: "complaint_time": 30, 2024-09-16T09:44:01.961 INFO:teuthology.orchestra.run.smithi084.stdout: "num_blocked_ops": 0 2024-09-16T09:44:01.961 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-09-16T09:44:01.990 INFO:teuthology.orchestra.run.smithi070.stdout:get_pg_num_history pg_num_history pg_num_history(e494 pg_nums {1={160=21},2={57=3},3={58=32},4={60=32},5={62=32},6={64=32},10={395=26},11={454=16}} deleted_pools 452,10) 2024-09-16T09:44:01.991 INFO:teuthology.orchestra.run.smithi070.stdout:Importing pgid 6.f 2024-09-16T09:44:01.991 INFO:teuthology.orchestra.run.smithi070.stdout:write_pg epoch 472 info 6.f( empty local-lis/les=227/228 n=0 ec=64/64 lis/c=227/227 les/c/f=228/228/0 sis=227) 2024-09-16T09:44:01.991 INFO:teuthology.orchestra.run.smithi070.stdout:Import successful 2024-09-16T09:44:02.010 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:44:02.044 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:02.044 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:02.044 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:44:02.100 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:44:01 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:44:01] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:44:02.145 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:44:02.177 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:02.178 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:02.178 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:44:02.279 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:44:02.315 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:02.316 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:02.316 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:44:02.394 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:02 smithi070 ceph-mon[30388]: 11.a scrub starts 2024-09-16T09:44:02.394 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:02 smithi070 ceph-mon[40691]: 11.a scrub starts 2024-09-16T09:44:02.417 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:44:02.451 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:02.452 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:44:02.452 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:02.452 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:44:02.453 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:44:02.489 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:44:02.490 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:44:02.491 INFO:teuthology.orchestra.run.smithi070.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-16T09:44:02.527 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:44:02.527 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:44:02.554 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:44:02.567 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:44:02.567 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:44:02.590 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:02.591 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:02.592 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:44:02.603 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:44:02.603 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:44:02.638 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:44:02.639 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:44:02.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:02 smithi084 ceph-mon[32629]: 11.a scrub starts 2024-09-16T09:44:02.674 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:44:02.674 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:44:02.693 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:44:02.721 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:02.722 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:02.722 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:44:02.823 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:44:02.833 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.4 dump_historic_ops 2024-09-16T09:44:02.858 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:02.859 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:02.859 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:44:02.960 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:44:03.005 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:03.006 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:03.006 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:44:03.107 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:44:03.150 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:03.151 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:03.151 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:44:03.252 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:44:03.283 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:03.284 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:03.284 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:44:03.285 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:44:03.385 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:44:03.417 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:03.418 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:03.418 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:44:03.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:03 smithi070 ceph-mon[30388]: 11.a scrub ok 2024-09-16T09:44:03.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:03 smithi070 ceph-mon[30388]: pgmap v995: 168 pgs: 31 active+undersized, 18 active+undersized+degraded, 119 active+clean; 2.1 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.48k op/s; 13070/104736 objects degraded (12.479%) 2024-09-16T09:44:03.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:03 smithi070 ceph-mon[30388]: 11.b scrub starts 2024-09-16T09:44:03.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:03 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3291000915' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:44:03.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:03 smithi070 ceph-mon[30388]: 1.d scrub starts 2024-09-16T09:44:03.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:03 smithi070 ceph-mon[30388]: 1.d scrub ok 2024-09-16T09:44:03.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:03 smithi070 ceph-mon[40691]: 11.a scrub ok 2024-09-16T09:44:03.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:03 smithi070 ceph-mon[40691]: pgmap v995: 168 pgs: 31 active+undersized, 18 active+undersized+degraded, 119 active+clean; 2.1 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.48k op/s; 13070/104736 objects degraded (12.479%) 2024-09-16T09:44:03.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:03 smithi070 ceph-mon[40691]: 11.b scrub starts 2024-09-16T09:44:03.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:03 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3291000915' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:44:03.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:03 smithi070 ceph-mon[40691]: 1.d scrub starts 2024-09-16T09:44:03.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:03 smithi070 ceph-mon[40691]: 1.d scrub ok 2024-09-16T09:44:03.423 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:03 smithi084 ceph-mon[32629]: 11.a scrub ok 2024-09-16T09:44:03.423 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:03 smithi084 ceph-mon[32629]: pgmap v995: 168 pgs: 31 active+undersized, 18 active+undersized+degraded, 119 active+clean; 2.1 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.48k op/s; 13070/104736 objects degraded (12.479%) 2024-09-16T09:44:03.423 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:03 smithi084 ceph-mon[32629]: 11.b scrub starts 2024-09-16T09:44:03.423 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:03 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3291000915' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:44:03.423 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:03 smithi084 ceph-mon[32629]: 1.d scrub starts 2024-09-16T09:44:03.423 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:03 smithi084 ceph-mon[32629]: 1.d scrub ok 2024-09-16T09:44:03.512 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T09:44:03.513 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T09:44:03.519 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:44:03.551 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:03.552 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:03.553 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:44:03.585 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:44:03.653 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:44:03.681 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:03.681 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:03.682 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:44:03.782 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:44:03.830 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:03.831 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:03.831 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:44:03.932 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:44:03.953 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:03.954 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:03.954 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:44:04.055 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:44:04.104 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:04.106 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:04.106 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:44:04.110 DEBUG:teuthology.orchestra.run.smithi070:> sudo rm -f /var/log/ceph/a1321834-740c-11ef-bceb-c7b262605968/exp.6.f.2 2024-09-16T09:44:04.160 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:44:03 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:44:04.207 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:44:04.243 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:04.244 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:04.244 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:44:04.345 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:44:04.384 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:04.384 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:04.385 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:44:04.390 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:04 smithi070 ceph-mon[30388]: 5.1d scrub starts 2024-09-16T09:44:04.390 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:04 smithi070 ceph-mon[30388]: 5.1d scrub ok 2024-09-16T09:44:04.390 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:04 smithi070 ceph-mon[30388]: 6.d scrub starts 2024-09-16T09:44:04.390 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:04 smithi070 ceph-mon[30388]: 6.d scrub ok 2024-09-16T09:44:04.391 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:04 smithi070 ceph-mon[30388]: 11.b scrub ok 2024-09-16T09:44:04.391 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:04 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:44:04.391 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:04 smithi070 ceph-mon[30388]: 5.5 scrub starts 2024-09-16T09:44:04.391 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:04 smithi070 ceph-mon[30388]: 5.5 scrub ok 2024-09-16T09:44:04.391 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:04 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:44:04.391 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:04 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:44:04.391 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:04 smithi070 ceph-mon[40691]: 5.1d scrub starts 2024-09-16T09:44:04.392 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:04 smithi070 ceph-mon[40691]: 5.1d scrub ok 2024-09-16T09:44:04.392 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:04 smithi070 ceph-mon[40691]: 6.d scrub starts 2024-09-16T09:44:04.392 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:04 smithi070 ceph-mon[40691]: 6.d scrub ok 2024-09-16T09:44:04.392 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:04 smithi070 ceph-mon[40691]: 11.b scrub ok 2024-09-16T09:44:04.392 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:04 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:44:04.392 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:04 smithi070 ceph-mon[40691]: 5.5 scrub starts 2024-09-16T09:44:04.392 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:04 smithi070 ceph-mon[40691]: 5.5 scrub ok 2024-09-16T09:44:04.392 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:04 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:44:04.392 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:04 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:44:04.486 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:44:04.520 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:04.521 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:04.521 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:44:04.622 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:44:04.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:04 smithi084 ceph-mon[32629]: 5.1d scrub starts 2024-09-16T09:44:04.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:04 smithi084 ceph-mon[32629]: 5.1d scrub ok 2024-09-16T09:44:04.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:04 smithi084 ceph-mon[32629]: 6.d scrub starts 2024-09-16T09:44:04.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:04 smithi084 ceph-mon[32629]: 6.d scrub ok 2024-09-16T09:44:04.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:04 smithi084 ceph-mon[32629]: 11.b scrub ok 2024-09-16T09:44:04.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:04 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:44:04.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:04 smithi084 ceph-mon[32629]: 5.5 scrub starts 2024-09-16T09:44:04.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:04 smithi084 ceph-mon[32629]: 5.5 scrub ok 2024-09-16T09:44:04.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:04 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:44:04.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:04 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:44:04.662 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:04.663 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:04.663 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:44:04.765 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:44:04.795 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:04.796 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:04.796 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:44:04.897 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:44:04.932 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:04.933 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:04.933 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:44:05.034 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:44:05.060 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:05.061 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:05.061 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:44:05.162 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:44:05.184 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:05.185 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:05.185 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:44:05.286 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:44:05.318 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:05.318 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:05.319 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:44:05.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:05 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:44:05.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:05 smithi084 ceph-mon[32629]: 3.16 scrub starts 2024-09-16T09:44:05.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:05 smithi084 ceph-mon[32629]: 3.16 scrub ok 2024-09-16T09:44:05.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:05 smithi084 ceph-mon[32629]: pgmap v996: 168 pgs: 31 active+undersized, 18 active+undersized+degraded, 119 active+clean; 1.7 GiB data, 14 GiB used, 702 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.51k op/s; 10755/83850 objects degraded (12.826%) 2024-09-16T09:44:05.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:05 smithi084 ceph-mon[32629]: 6.b scrub starts 2024-09-16T09:44:05.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:05 smithi084 ceph-mon[32629]: 6.b scrub ok 2024-09-16T09:44:05.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:05 smithi084 ceph-mon[32629]: Health check update: Degraded data redundancy: 13070/104736 objects degraded (12.479%), 18 pgs degraded (PG_DEGRADED) 2024-09-16T09:44:05.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:05 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1735206765' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:44:05.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:05 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:44:05.419 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:44:05.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:05 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:44:05.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:05 smithi070 ceph-mon[40691]: 3.16 scrub starts 2024-09-16T09:44:05.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:05 smithi070 ceph-mon[40691]: 3.16 scrub ok 2024-09-16T09:44:05.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:05 smithi070 ceph-mon[40691]: pgmap v996: 168 pgs: 31 active+undersized, 18 active+undersized+degraded, 119 active+clean; 1.7 GiB data, 14 GiB used, 702 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.51k op/s; 10755/83850 objects degraded (12.826%) 2024-09-16T09:44:05.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:05 smithi070 ceph-mon[40691]: 6.b scrub starts 2024-09-16T09:44:05.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:05 smithi070 ceph-mon[40691]: 6.b scrub ok 2024-09-16T09:44:05.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:05 smithi070 ceph-mon[40691]: Health check update: Degraded data redundancy: 13070/104736 objects degraded (12.479%), 18 pgs degraded (PG_DEGRADED) 2024-09-16T09:44:05.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:05 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1735206765' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:44:05.422 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:05 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:44:05.422 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:05 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:44:05.422 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:05 smithi070 ceph-mon[30388]: 3.16 scrub starts 2024-09-16T09:44:05.422 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:05 smithi070 ceph-mon[30388]: 3.16 scrub ok 2024-09-16T09:44:05.422 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:05 smithi070 ceph-mon[30388]: pgmap v996: 168 pgs: 31 active+undersized, 18 active+undersized+degraded, 119 active+clean; 1.7 GiB data, 14 GiB used, 702 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.51k op/s; 10755/83850 objects degraded (12.826%) 2024-09-16T09:44:05.422 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:05 smithi070 ceph-mon[30388]: 6.b scrub starts 2024-09-16T09:44:05.422 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:05 smithi070 ceph-mon[30388]: 6.b scrub ok 2024-09-16T09:44:05.422 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:05 smithi070 ceph-mon[30388]: Health check update: Degraded data redundancy: 13070/104736 objects degraded (12.479%), 18 pgs degraded (PG_DEGRADED) 2024-09-16T09:44:05.422 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:05 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1735206765' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:44:05.422 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:05 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:44:05.447 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:05.448 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:05.448 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:44:05.514 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T09:44:05.549 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:44:05.580 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:05.581 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:05.581 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:44:05.682 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:44:05.715 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:05.716 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:05.716 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:44:05.817 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:44:05.848 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:05.849 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:05.849 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:44:05.886 INFO:teuthology.orchestra.run.smithi070.stderr:noscrub is set 2024-09-16T09:44:05.906 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:44:05.950 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:44:05.981 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:05.982 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:05.982 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:44:06.083 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:44:06.107 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:06.108 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:06.108 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:44:06.209 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:44:06.245 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:06.246 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:06.246 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:44:06.347 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:44:06.360 INFO:teuthology.orchestra.run.smithi084.stdout:ERROR: (22) Invalid argument 2024-09-16T09:44:06.361 INFO:teuthology.orchestra.run.smithi084.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:44:06.379 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:06.380 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:06.380 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:44:06.481 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:44:06.516 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:06.516 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:06.517 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:44:06.618 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:44:06.653 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:06.654 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:06.654 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:44:06.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:06 smithi084 ceph-mon[32629]: 6.4 scrub starts 2024-09-16T09:44:06.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:06 smithi084 ceph-mon[32629]: 6.4 scrub ok 2024-09-16T09:44:06.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:06 smithi084 ceph-mon[32629]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:44:06.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:06 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-16T09:44:06.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:06 smithi084 ceph-mon[32629]: osdmap e511: 8 total, 7 up, 8 in 2024-09-16T09:44:06.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:06 smithi070 ceph-mon[30388]: 6.4 scrub starts 2024-09-16T09:44:06.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:06 smithi070 ceph-mon[30388]: 6.4 scrub ok 2024-09-16T09:44:06.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:06 smithi070 ceph-mon[30388]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:44:06.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:06 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-16T09:44:06.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:06 smithi070 ceph-mon[30388]: osdmap e511: 8 total, 7 up, 8 in 2024-09-16T09:44:06.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:06 smithi070 ceph-mon[40691]: 6.4 scrub starts 2024-09-16T09:44:06.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:06 smithi070 ceph-mon[40691]: 6.4 scrub ok 2024-09-16T09:44:06.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:06 smithi070 ceph-mon[40691]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:44:06.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:06 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-16T09:44:06.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:06 smithi070 ceph-mon[40691]: osdmap e511: 8 total, 7 up, 8 in 2024-09-16T09:44:06.755 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:44:06.778 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:06.779 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:06.779 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:44:06.880 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:44:06.912 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:06.913 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:06.913 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:44:07.014 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:44:07.042 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:07.043 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:07.043 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:44:07.144 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:44:07.181 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:07.182 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:07.183 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:44:07.283 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:44:07.305 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:07.305 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:07.306 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:44:07.406 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:44:07.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:07 smithi070 ceph-mon[30388]: 4.18 scrub starts 2024-09-16T09:44:07.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:07 smithi070 ceph-mon[30388]: 4.18 scrub ok 2024-09-16T09:44:07.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:07 smithi070 ceph-mon[30388]: 3.10 scrub starts 2024-09-16T09:44:07.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:07 smithi070 ceph-mon[30388]: 3.10 scrub ok 2024-09-16T09:44:07.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:07 smithi070 ceph-mon[30388]: pgmap v998: 168 pgs: 31 active+undersized, 18 active+undersized+degraded, 119 active+clean; 1.7 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.73k op/s; 10270/82395 objects degraded (12.464%) 2024-09-16T09:44:07.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:07 smithi070 ceph-mon[40691]: 4.18 scrub starts 2024-09-16T09:44:07.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:07 smithi070 ceph-mon[40691]: 4.18 scrub ok 2024-09-16T09:44:07.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:07 smithi070 ceph-mon[40691]: 3.10 scrub starts 2024-09-16T09:44:07.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:07 smithi070 ceph-mon[40691]: 3.10 scrub ok 2024-09-16T09:44:07.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:07 smithi070 ceph-mon[40691]: pgmap v998: 168 pgs: 31 active+undersized, 18 active+undersized+degraded, 119 active+clean; 1.7 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.73k op/s; 10270/82395 objects degraded (12.464%) 2024-09-16T09:44:07.439 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:07.440 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:07.440 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:44:07.517 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:44:07.517 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:44:07.541 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:44:07.551 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:44:07.551 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:44:07.551 INFO:teuthology.orchestra.run.smithi070.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-16T09:44:07.573 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:07.574 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:07.574 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:44:07.578 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:44:07.578 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:44:07.597 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:44:07.597 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:44:07.632 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:44:07.632 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:44:07.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:07 smithi084 ceph-mon[32629]: 4.18 scrub starts 2024-09-16T09:44:07.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:07 smithi084 ceph-mon[32629]: 4.18 scrub ok 2024-09-16T09:44:07.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:07 smithi084 ceph-mon[32629]: 3.10 scrub starts 2024-09-16T09:44:07.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:07 smithi084 ceph-mon[32629]: 3.10 scrub ok 2024-09-16T09:44:07.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:07 smithi084 ceph-mon[32629]: pgmap v998: 168 pgs: 31 active+undersized, 18 active+undersized+degraded, 119 active+clean; 1.7 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.73k op/s; 10270/82395 objects degraded (12.464%) 2024-09-16T09:44:07.665 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:44:07.666 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:44:07.675 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:44:07.698 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:44:07.698 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:44:07.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:07.709 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:07.709 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:44:07.810 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:44:07.840 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:07.841 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:07.841 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:44:07.942 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:44:07.968 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:07.969 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:07.970 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:44:08.070 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:44:08.103 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:08.104 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:08.104 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:44:08.205 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:44:08.227 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:08.228 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:08.228 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:44:08.329 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:44:08.361 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:08.362 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:08.362 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:44:08.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:08 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3234522033' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:44:08.420 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:08 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3234522033' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:44:08.463 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:44:08.495 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:08.496 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:08.496 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:44:08.597 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:44:08.618 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:08.619 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:08.619 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:44:08.622 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T09:44:08.622 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T09:44:08.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:08 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3234522033' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:44:08.720 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:44:08.757 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:08.758 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:08.758 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:44:08.784 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd set nodeep-scrub 2024-09-16T09:44:08.859 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:44:08.882 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:08.882 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:08.883 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:44:08.983 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:44:09.013 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:09.013 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:09.014 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:44:09.115 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:44:09.157 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:09.157 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:09.158 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:44:09.165 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:44:09.210 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [2] live_osds: [0, 1, 3, 4, 5, 6, 7] 2024-09-16T09:44:09.210 INFO:tasks.thrashosds.thrasher:Reviving osd 2 2024-09-16T09:44:09.210 INFO:tasks.cephadm.osd.2:Restarting osd.2 (starting--it wasn't running)... 2024-09-16T09:44:09.211 DEBUG:teuthology.orchestra.run.smithi070:osd.2> sudo journalctl -f -n 0 -u ceph-a1321834-740c-11ef-bceb-c7b262605968@osd.2.service 2024-09-16T09:44:09.214 DEBUG:teuthology.orchestra.run.smithi070:> sudo systemctl start ceph-a1321834-740c-11ef-bceb-c7b262605968@osd.2 2024-09-16T09:44:09.259 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:44:09.293 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:09.293 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:09.293 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:44:09.394 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:44:09.429 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:09.430 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:09.430 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:44:09.464 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:09 smithi070 ceph-mon[30388]: pgmap v999: 168 pgs: 31 active+undersized, 18 active+undersized+degraded, 119 active+clean; 1.6 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.50k op/s; 9392/77202 objects degraded (12.165%) 2024-09-16T09:44:09.465 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:09 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:44:09.465 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:09 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:44:09.465 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:09 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:44:09.465 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:09 smithi070 ceph-mon[40691]: pgmap v999: 168 pgs: 31 active+undersized, 18 active+undersized+degraded, 119 active+clean; 1.6 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.50k op/s; 9392/77202 objects degraded (12.165%) 2024-09-16T09:44:09.465 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:09 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:44:09.465 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:09 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:44:09.465 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:09 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:44:09.466 INFO:journalctl@ceph.osd.2.smithi070.stdout:Sep 16 09:44:09 smithi070 systemd[1]: Starting Ceph osd.2 for a1321834-740c-11ef-bceb-c7b262605968... 2024-09-16T09:44:09.531 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:44:09.563 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:09.564 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:09.564 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:44:09.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:09 smithi084 ceph-mon[32629]: pgmap v999: 168 pgs: 31 active+undersized, 18 active+undersized+degraded, 119 active+clean; 1.6 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.50k op/s; 9392/77202 objects degraded (12.165%) 2024-09-16T09:44:09.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:09 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:44:09.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:09 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:44:09.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:09 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:44:09.665 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:44:09.703 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:09.704 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:09.704 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:44:09.805 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:44:09.838 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:09.839 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:09.839 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:44:09.940 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:44:09.973 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:09.973 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:09.974 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:44:10.019 INFO:journalctl@ceph.osd.2.smithi070.stdout:Sep 16 09:44:09 smithi070 podman[212704]: 2024-09-16 09:44:09.875580039 +0000 UTC m=+0.037148112 image pull 425e316ba8ebf74f04aa3640f7deb4fa3f293a78d0c8d097cb679eaf86ddca6c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce 2024-09-16T09:44:10.019 INFO:journalctl@ceph.osd.2.smithi070.stdout:Sep 16 09:44:10 smithi070 podman[212704]: 2024-09-16 09:44:10.017588235 +0000 UTC m=+0.179156297 container create 1776a8cafdba9941f80db0b6e37cd73a3937a3abded305b3c250a2ca3dafcc41 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-2-activate, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T09:44:10.074 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:44:10.105 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:10.106 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:10.106 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:44:10.198 INFO:teuthology.orchestra.run.smithi070.stderr:nodeep-scrub is set 2024-09-16T09:44:10.207 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:44:10.240 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:10.241 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:10.241 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:44:10.323 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:10 smithi070 ceph-mon[30388]: Health check update: Degraded data redundancy: 9392/77202 objects degraded (12.165%), 18 pgs degraded, 49 pgs undersized (PG_DEGRADED) 2024-09-16T09:44:10.323 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:10 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/57426702' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:44:10.323 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:10 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:44:10.323 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:10 smithi070 ceph-mon[40691]: Health check update: Degraded data redundancy: 9392/77202 objects degraded (12.165%), 18 pgs degraded, 49 pgs undersized (PG_DEGRADED) 2024-09-16T09:44:10.323 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:10 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/57426702' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:44:10.324 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:10 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:44:10.342 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:44:10.365 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:10.365 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:10.366 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:44:10.466 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:44:10.503 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:10.504 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:10.504 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:44:10.605 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:44:10.623 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T09:44:10.641 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:10.641 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:10.642 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:44:10.655 INFO:journalctl@ceph.osd.2.smithi070.stdout:Sep 16 09:44:10 smithi070 podman[212704]: 2024-09-16 09:44:10.322605063 +0000 UTC m=+0.484173139 container init 1776a8cafdba9941f80db0b6e37cd73a3937a3abded305b3c250a2ca3dafcc41 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-2-activate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.license=GPLv2) 2024-09-16T09:44:10.656 INFO:journalctl@ceph.osd.2.smithi070.stdout:Sep 16 09:44:10 smithi070 podman[212704]: 2024-09-16 09:44:10.330759006 +0000 UTC m=+0.492327069 container start 1776a8cafdba9941f80db0b6e37cd73a3937a3abded305b3c250a2ca3dafcc41 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-2-activate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-guits-main-2024-09-13-1248-8293d73) 2024-09-16T09:44:10.656 INFO:journalctl@ceph.osd.2.smithi070.stdout:Sep 16 09:44:10 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-2-activate[212719]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-16T09:44:10.656 INFO:journalctl@ceph.osd.2.smithi070.stdout:Sep 16 09:44:10 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-2-activate[212719]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-16T09:44:10.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:10 smithi084 ceph-mon[32629]: Health check update: Degraded data redundancy: 9392/77202 objects degraded (12.165%), 18 pgs degraded, 49 pgs undersized (PG_DEGRADED) 2024-09-16T09:44:10.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:10 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/57426702' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:44:10.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:10 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:44:10.742 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:44:10.773 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:10.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:10.774 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:44:10.875 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:44:10.918 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:10.918 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:10.919 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:44:10.922 INFO:journalctl@ceph.osd.2.smithi070.stdout:Sep 16 09:44:10 smithi070 podman[212704]: 2024-09-16 09:44:10.664594628 +0000 UTC m=+0.826162690 container attach 1776a8cafdba9941f80db0b6e37cd73a3937a3abded305b3c250a2ca3dafcc41 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-2-activate, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-16T09:44:10.922 INFO:journalctl@ceph.osd.2.smithi070.stdout:Sep 16 09:44:10 smithi070 bash[212704]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-16T09:44:10.922 INFO:journalctl@ceph.osd.2.smithi070.stdout:Sep 16 09:44:10 smithi070 bash[212704]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-16T09:44:11.020 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:44:11.053 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:11.054 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:11.054 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:44:11.155 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:44:11.183 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:11.184 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:11.184 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:44:11.285 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:44:11.314 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:11.314 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:11.314 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:44:11.415 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:44:11.448 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:11.449 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:11.449 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:44:11.544 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:44:11.550 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:44:11.583 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:11.584 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:11.585 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:44:11.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:11 smithi084 ceph-mon[32629]: pgmap v1000: 168 pgs: 31 active+undersized, 18 active+undersized+degraded, 119 active+clean; 1.2 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.45k op/s; 7609/61299 objects degraded (12.413%) 2024-09-16T09:44:11.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:11 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-16T09:44:11.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:11 smithi084 ceph-mon[32629]: osdmap e512: 8 total, 7 up, 8 in 2024-09-16T09:44:11.670 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:11 smithi070 ceph-mon[40691]: pgmap v1000: 168 pgs: 31 active+undersized, 18 active+undersized+degraded, 119 active+clean; 1.2 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.45k op/s; 7609/61299 objects degraded (12.413%) 2024-09-16T09:44:11.670 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:11 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-16T09:44:11.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:11 smithi070 ceph-mon[40691]: osdmap e512: 8 total, 7 up, 8 in 2024-09-16T09:44:11.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:11 smithi070 ceph-mon[30388]: pgmap v1000: 168 pgs: 31 active+undersized, 18 active+undersized+degraded, 119 active+clean; 1.2 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.45k op/s; 7609/61299 objects degraded (12.413%) 2024-09-16T09:44:11.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:11 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-16T09:44:11.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:11 smithi070 ceph-mon[30388]: osdmap e512: 8 total, 7 up, 8 in 2024-09-16T09:44:11.685 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:44:11.720 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:11.721 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:11.721 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:44:11.822 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:44:11.858 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:11.859 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:11.859 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:44:11.960 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:44:12.001 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:12.002 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:12.002 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:44:12.033 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:44:11 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:44:11] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:44:12.102 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:44:12.136 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:12.137 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:12.137 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:44:12.192 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:44:12.192 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:44:12.232 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:44:12.232 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:44:12.233 INFO:teuthology.orchestra.run.smithi070.stderr:Error ENXIO: problem getting command descriptions from osd.2 2024-09-16T09:44:12.238 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:44:12.269 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:44:12.269 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:44:12.274 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:12.275 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:12.275 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:44:12.308 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:12 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2892878661' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:44:12.308 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:12 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/2892878661' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:44:12.308 INFO:journalctl@ceph.osd.2.smithi070.stdout:Sep 16 09:44:12 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-2-activate[212719]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-16T09:44:12.308 INFO:journalctl@ceph.osd.2.smithi070.stdout:Sep 16 09:44:12 smithi070 bash[212704]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-16T09:44:12.308 INFO:journalctl@ceph.osd.2.smithi070.stdout:Sep 16 09:44:12 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-2-activate[212719]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-09-16T09:44:12.309 INFO:journalctl@ceph.osd.2.smithi070.stdout:Sep 16 09:44:12 smithi070 bash[212704]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-09-16T09:44:12.309 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:44:12.309 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:44:12.339 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:44:12.339 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:44:12.377 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:44:12.390 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:44:12.390 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:44:12.398 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:12.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:12.399 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:44:12.433 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:44:12.433 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:44:12.500 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:44:12.532 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:12.533 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:12.534 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:44:12.574 INFO:journalctl@ceph.osd.2.smithi070.stdout:Sep 16 09:44:12 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-2-activate[212719]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-09-16T09:44:12.574 INFO:journalctl@ceph.osd.2.smithi070.stdout:Sep 16 09:44:12 smithi070 bash[212704]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-09-16T09:44:12.634 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:44:12.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:12 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/2892878661' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:44:12.669 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:12.670 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:12.670 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:44:12.771 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:44:12.805 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:12.806 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:12.806 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:44:12.907 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:44:12.921 INFO:journalctl@ceph.osd.2.smithi070.stdout:Sep 16 09:44:12 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-2-activate[212719]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-09-16T09:44:12.921 INFO:journalctl@ceph.osd.2.smithi070.stdout:Sep 16 09:44:12 smithi070 bash[212704]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-09-16T09:44:12.921 INFO:journalctl@ceph.osd.2.smithi070.stdout:Sep 16 09:44:12 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-2-activate[212719]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-09-16T09:44:12.921 INFO:journalctl@ceph.osd.2.smithi070.stdout:Sep 16 09:44:12 smithi070 bash[212704]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-09-16T09:44:12.921 INFO:journalctl@ceph.osd.2.smithi070.stdout:Sep 16 09:44:12 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-2-activate[212719]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-16T09:44:12.921 INFO:journalctl@ceph.osd.2.smithi070.stdout:Sep 16 09:44:12 smithi070 bash[212704]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-16T09:44:12.921 INFO:journalctl@ceph.osd.2.smithi070.stdout:Sep 16 09:44:12 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-2-activate[212719]: --> ceph-volume raw activate successful for osd ID: 2 2024-09-16T09:44:12.922 INFO:journalctl@ceph.osd.2.smithi070.stdout:Sep 16 09:44:12 smithi070 bash[212704]: --> ceph-volume raw activate successful for osd ID: 2 2024-09-16T09:44:12.922 INFO:journalctl@ceph.osd.2.smithi070.stdout:Sep 16 09:44:12 smithi070 podman[212704]: 2024-09-16 09:44:12.674895519 +0000 UTC m=+2.836463576 container died 1776a8cafdba9941f80db0b6e37cd73a3937a3abded305b3c250a2ca3dafcc41 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-2-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-16T09:44:12.941 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:12.942 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:12.944 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:44:13.045 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:44:13.073 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:13.073 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:13.074 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:44:13.174 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:44:13.203 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:13.204 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:13.204 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:44:13.305 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:44:13.339 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:13.340 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:13.340 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:44:13.388 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd unset noscrub 2024-09-16T09:44:13.441 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:44:13.459 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:13 smithi070 ceph-mon[30388]: pgmap v1002: 168 pgs: 31 active+undersized, 18 active+undersized+degraded, 119 active+clean; 1.2 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 939 op/s; 7609/61299 objects degraded (12.413%) 2024-09-16T09:44:13.459 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:13 smithi070 ceph-mon[40691]: pgmap v1002: 168 pgs: 31 active+undersized, 18 active+undersized+degraded, 119 active+clean; 1.2 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 939 op/s; 7609/61299 objects degraded (12.413%) 2024-09-16T09:44:13.471 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:13.472 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:13.472 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:44:13.573 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:44:13.612 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:13.613 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:13.613 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:44:13.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:13 smithi084 ceph-mon[32629]: pgmap v1002: 168 pgs: 31 active+undersized, 18 active+undersized+degraded, 119 active+clean; 1.2 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 939 op/s; 7609/61299 objects degraded (12.413%) 2024-09-16T09:44:13.713 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:44:13.749 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:13.750 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:13.750 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:44:13.851 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:44:13.881 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:13.882 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:13.882 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:44:13.883 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.5 dump_ops_in_flight 2024-09-16T09:44:13.983 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:44:14.024 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:14.025 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:14.025 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:44:14.126 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:44:14.160 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:44:13 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:44:14.165 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:14.166 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:14.167 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:44:14.267 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:44:14.312 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:14.312 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:14.313 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:44:14.326 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:44:14.414 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:44:14.420 INFO:journalctl@ceph.osd.2.smithi070.stdout:Sep 16 09:44:14 smithi070 podman[213147]: 2024-09-16 09:44:14.148680428 +0000 UTC m=+1.514316897 container remove 1776a8cafdba9941f80db0b6e37cd73a3937a3abded305b3c250a2ca3dafcc41 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-2-activate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True) 2024-09-16T09:44:14.449 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:14.449 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:14.450 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:44:14.550 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:44:14.588 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:14.589 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:14.589 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:44:14.690 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:44:14.729 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:14.729 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:14.730 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:44:14.831 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:44:14.870 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:14.871 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:14.872 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:44:14.973 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:44:14.995 INFO:teuthology.orchestra.run.smithi084.stdout:ERROR: (22) Invalid argument 2024-09-16T09:44:14.995 INFO:teuthology.orchestra.run.smithi084.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:44:15.007 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:15.008 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:15.008 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:44:15.109 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:44:15.146 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:15.146 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:15.147 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:44:15.247 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:44:15.285 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:15.286 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:15.286 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:44:15.387 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:44:15.409 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:15.409 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:15.410 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:44:15.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:15 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:44:15.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:15 smithi070 ceph-mon[30388]: pgmap v1003: 168 pgs: 31 active+undersized, 18 active+undersized+degraded, 119 active+clean; 853 MiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.70k op/s; 5431/41511 objects degraded (13.083%) 2024-09-16T09:44:15.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:15 smithi070 ceph-mon[30388]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:44:15.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:15 smithi070 ceph-mon[30388]: Health check update: Degraded data redundancy: 5431/41511 objects degraded (13.083%), 18 pgs degraded, 49 pgs undersized (PG_DEGRADED) 2024-09-16T09:44:15.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:15 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:44:15.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:15 smithi070 ceph-mon[40691]: pgmap v1003: 168 pgs: 31 active+undersized, 18 active+undersized+degraded, 119 active+clean; 853 MiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.70k op/s; 5431/41511 objects degraded (13.083%) 2024-09-16T09:44:15.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:15 smithi070 ceph-mon[40691]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:44:15.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:15 smithi070 ceph-mon[40691]: Health check update: Degraded data redundancy: 5431/41511 objects degraded (13.083%), 18 pgs degraded, 49 pgs undersized (PG_DEGRADED) 2024-09-16T09:44:15.511 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:44:15.539 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:15.539 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:15.540 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:44:15.640 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:44:15.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:15 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:44:15.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:15 smithi084 ceph-mon[32629]: pgmap v1003: 168 pgs: 31 active+undersized, 18 active+undersized+degraded, 119 active+clean; 853 MiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.70k op/s; 5431/41511 objects degraded (13.083%) 2024-09-16T09:44:15.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:15 smithi084 ceph-mon[32629]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:44:15.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:15 smithi084 ceph-mon[32629]: Health check update: Degraded data redundancy: 5431/41511 objects degraded (13.083%), 18 pgs degraded, 49 pgs undersized (PG_DEGRADED) 2024-09-16T09:44:15.673 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:15.674 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:15.674 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:44:15.775 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:44:15.810 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:15.810 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:15.811 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:44:15.834 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.5 dump_blocked_ops 2024-09-16T09:44:15.912 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:44:15.950 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:15.950 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:15.951 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:44:16.051 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:44:16.091 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:16.091 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:16.092 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:44:16.193 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:44:16.238 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:16.239 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:16.240 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:44:16.265 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-16T09:44:16.265 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-16T09:44:16.270 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:44:16.276 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:44:16.340 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:44:16.383 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:16.384 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:16.384 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:44:16.485 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:44:16.520 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:16.521 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:16.522 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:44:16.622 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:44:16.655 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:16.656 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:16.656 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:44:16.757 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:44:16.788 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:16.789 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:16.790 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:44:16.828 INFO:journalctl@ceph.osd.2.smithi070.stdout:Sep 16 09:44:16 smithi070 podman[213466]: 2024-09-16 09:44:16.583971795 +0000 UTC m=+0.265075315 image pull 425e316ba8ebf74f04aa3640f7deb4fa3f293a78d0c8d097cb679eaf86ddca6c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce 2024-09-16T09:44:16.828 INFO:journalctl@ceph.osd.2.smithi070.stdout:Sep 16 09:44:16 smithi070 podman[213466]: 2024-09-16 09:44:16.716233174 +0000 UTC m=+0.397336697 container create ed83dc3d2c50b8160eba470147aa8f3f4e2f10e6d90733c5af157c674117edc6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-16T09:44:16.890 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:44:16.933 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:16.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:16.934 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:44:16.958 INFO:teuthology.orchestra.run.smithi084.stdout:ERROR: (22) Invalid argument 2024-09-16T09:44:16.958 INFO:teuthology.orchestra.run.smithi084.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:44:17.035 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:44:17.066 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:17.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:17.067 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:44:17.168 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:44:17.199 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:17.200 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:17.200 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:44:17.274 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.2 dump_ops_in_flight 2024-09-16T09:44:17.301 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:44:17.333 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:17.333 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:17.334 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:44:17.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:17 smithi070 ceph-mon[30388]: pgmap v1004: 168 pgs: 31 active+undersized, 18 active+undersized+degraded, 119 active+clean; 826 MiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.41k op/s; 5007/40239 objects degraded (12.443%) 2024-09-16T09:44:17.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:17 smithi070 ceph-mon[40691]: pgmap v1004: 168 pgs: 31 active+undersized, 18 active+undersized+degraded, 119 active+clean; 826 MiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.41k op/s; 5007/40239 objects degraded (12.443%) 2024-09-16T09:44:17.421 INFO:journalctl@ceph.osd.2.smithi070.stdout:Sep 16 09:44:17 smithi070 podman[213466]: 2024-09-16 09:44:17.154832058 +0000 UTC m=+0.835935582 container init ed83dc3d2c50b8160eba470147aa8f3f4e2f10e6d90733c5af157c674117edc6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-16T09:44:17.421 INFO:journalctl@ceph.osd.2.smithi070.stdout:Sep 16 09:44:17 smithi070 podman[213466]: 2024-09-16 09:44:17.165433312 +0000 UTC m=+0.846536834 container start ed83dc3d2c50b8160eba470147aa8f3f4e2f10e6d90733c5af157c674117edc6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True) 2024-09-16T09:44:17.421 INFO:journalctl@ceph.osd.2.smithi070.stdout:Sep 16 09:44:17 smithi070 bash[213466]: ed83dc3d2c50b8160eba470147aa8f3f4e2f10e6d90733c5af157c674117edc6 2024-09-16T09:44:17.421 INFO:journalctl@ceph.osd.2.smithi070.stdout:Sep 16 09:44:17 smithi070 systemd[1]: Started Ceph osd.2 for a1321834-740c-11ef-bceb-c7b262605968. 2024-09-16T09:44:17.435 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:44:17.467 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:17.468 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:17.468 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:44:17.569 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:44:17.610 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:17.611 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:17.611 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:44:17.627 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:17 smithi084 ceph-mon[32629]: pgmap v1004: 168 pgs: 31 active+undersized, 18 active+undersized+degraded, 119 active+clean; 826 MiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.41k op/s; 5007/40239 objects degraded (12.443%) 2024-09-16T09:44:17.676 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.5 dump_historic_ops 2024-09-16T09:44:17.689 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:44:17.712 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:44:17.755 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:17.755 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:17.756 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:44:17.856 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:44:17.892 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:17.892 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:17.893 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:44:17.994 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:44:18.027 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:18.027 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:18.028 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:44:18.113 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:44:18.128 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:44:18.177 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:18.177 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:18.178 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:44:18.243 INFO:teuthology.orchestra.run.smithi070.stderr:noscrub is unset 2024-09-16T09:44:18.267 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T09:44:18.278 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:44:18.309 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:18.310 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:18.310 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:44:18.321 INFO:teuthology.orchestra.run.smithi070.stderr:no valid command found; 10 closest matches: 2024-09-16T09:44:18.321 INFO:teuthology.orchestra.run.smithi070.stderr:0 2024-09-16T09:44:18.321 INFO:teuthology.orchestra.run.smithi070.stderr:1 2024-09-16T09:44:18.321 INFO:teuthology.orchestra.run.smithi070.stderr:2 2024-09-16T09:44:18.321 INFO:teuthology.orchestra.run.smithi070.stderr:abort 2024-09-16T09:44:18.321 INFO:teuthology.orchestra.run.smithi070.stderr:assert 2024-09-16T09:44:18.321 INFO:teuthology.orchestra.run.smithi070.stderr:config diff 2024-09-16T09:44:18.321 INFO:teuthology.orchestra.run.smithi070.stderr:config diff get 2024-09-16T09:44:18.321 INFO:teuthology.orchestra.run.smithi070.stderr:config get 2024-09-16T09:44:18.321 INFO:teuthology.orchestra.run.smithi070.stderr:config help [] 2024-09-16T09:44:18.321 INFO:teuthology.orchestra.run.smithi070.stderr:config set ... 2024-09-16T09:44:18.321 INFO:teuthology.orchestra.run.smithi070.stderr:admin_socket: invalid command 2024-09-16T09:44:18.411 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:44:18.420 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:18 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:44:18.420 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:18 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:44:18.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:18 smithi070 ceph-mon[40691]: osdmap e513: 8 total, 7 up, 8 in 2024-09-16T09:44:18.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:18 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/2430194045' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:44:18.421 INFO:journalctl@ceph.osd.2.smithi070.stdout:Sep 16 09:44:18 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-2[213542]: 2024-09-16T09:44:18.072+0000 7f139745e640 -1 Falling back to public interface 2024-09-16T09:44:18.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:18 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:44:18.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:18 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:44:18.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:18 smithi070 ceph-mon[30388]: osdmap e513: 8 total, 7 up, 8 in 2024-09-16T09:44:18.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:18 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2430194045' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:44:18.454 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:18.455 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:18.455 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:44:18.501 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:18 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:44:18.501 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:18 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:44:18.502 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:18 smithi084 ceph-mon[32629]: osdmap e513: 8 total, 7 up, 8 in 2024-09-16T09:44:18.502 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:18 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/2430194045' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:44:18.556 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:44:18.578 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:18.579 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:18.579 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:44:18.680 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:44:18.720 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:18.720 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:18.720 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:44:18.821 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:44:18.842 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:18.843 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:18.843 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:44:18.944 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:44:18.979 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:18.979 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:18.980 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:44:19.081 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:44:19.103 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:19.104 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:19.104 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:44:19.205 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:44:19.240 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:19.242 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:19.242 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:44:19.343 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:44:19.382 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:19.383 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:19.383 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:44:19.484 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:44:19.511 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:19 smithi070 ceph-mon[30388]: pgmap v1006: 168 pgs: 31 active+undersized, 18 active+undersized+degraded, 119 active+clean; 710 MiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.11k op/s; 4057/34632 objects degraded (11.715%) 2024-09-16T09:44:19.511 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:19 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2430194045' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-16T09:44:19.512 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:19 smithi070 ceph-mon[30388]: osdmap e514: 8 total, 7 up, 8 in 2024-09-16T09:44:19.512 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:19 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:44:19.512 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:19 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:44:19.512 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:19 smithi070 ceph-mon[40691]: pgmap v1006: 168 pgs: 31 active+undersized, 18 active+undersized+degraded, 119 active+clean; 710 MiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.11k op/s; 4057/34632 objects degraded (11.715%) 2024-09-16T09:44:19.512 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:19 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/2430194045' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-16T09:44:19.512 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:19 smithi070 ceph-mon[40691]: osdmap e514: 8 total, 7 up, 8 in 2024-09-16T09:44:19.512 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:19 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:44:19.512 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:19 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:44:19.518 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:19.519 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:19.520 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:44:19.522 INFO:teuthology.orchestra.run.smithi084.stdout:ERROR: (22) Invalid argument 2024-09-16T09:44:19.522 INFO:teuthology.orchestra.run.smithi084.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:44:19.523 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:19 smithi084 ceph-mon[32629]: pgmap v1006: 168 pgs: 31 active+undersized, 18 active+undersized+degraded, 119 active+clean; 710 MiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.11k op/s; 4057/34632 objects degraded (11.715%) 2024-09-16T09:44:19.523 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:19 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/2430194045' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-16T09:44:19.523 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:19 smithi084 ceph-mon[32629]: osdmap e514: 8 total, 7 up, 8 in 2024-09-16T09:44:19.523 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:19 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:44:19.523 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:19 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:44:19.621 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:44:19.657 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:19.657 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:19.658 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:44:19.758 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:44:19.791 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:19.791 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:19.792 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:44:19.893 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:44:19.926 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:19.927 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:19.927 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:44:20.028 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:44:20.065 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:20.066 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:20.066 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:44:20.167 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:44:20.201 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:20.201 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:20.202 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:44:20.303 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:44:20.340 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:20.341 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:20.341 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:44:20.442 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:44:20.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:20.464 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:20.465 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:44:20.565 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:44:20.596 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:20.596 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:20.597 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:44:20.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:20 smithi084 ceph-mon[32629]: 6.a scrub starts 2024-09-16T09:44:20.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:20 smithi084 ceph-mon[32629]: 6.a scrub ok 2024-09-16T09:44:20.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:20 smithi084 ceph-mon[32629]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:44:20.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:20 smithi084 ceph-mon[32629]: Health check update: Degraded data redundancy: 4057/34632 objects degraded (11.715%), 18 pgs degraded, 49 pgs undersized (PG_DEGRADED) 2024-09-16T09:44:20.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:20 smithi070 ceph-mon[30388]: 6.a scrub starts 2024-09-16T09:44:20.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:20 smithi070 ceph-mon[30388]: 6.a scrub ok 2024-09-16T09:44:20.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:20 smithi070 ceph-mon[30388]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:44:20.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:20 smithi070 ceph-mon[30388]: Health check update: Degraded data redundancy: 4057/34632 objects degraded (11.715%), 18 pgs degraded, 49 pgs undersized (PG_DEGRADED) 2024-09-16T09:44:20.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:20 smithi070 ceph-mon[40691]: 6.a scrub starts 2024-09-16T09:44:20.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:20 smithi070 ceph-mon[40691]: 6.a scrub ok 2024-09-16T09:44:20.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:20 smithi070 ceph-mon[40691]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:44:20.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:20 smithi070 ceph-mon[40691]: Health check update: Degraded data redundancy: 4057/34632 objects degraded (11.715%), 18 pgs degraded, 49 pgs undersized (PG_DEGRADED) 2024-09-16T09:44:20.698 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:44:20.726 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:20.727 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:20.727 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:44:20.828 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:44:20.864 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:20.865 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:20.865 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:44:20.966 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:44:21.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:21.004 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:21.005 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:44:21.105 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:44:21.147 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:21.148 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:21.149 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:44:21.249 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:44:21.280 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:21.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:21.281 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:44:21.382 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:44:21.413 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:21.413 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:21.414 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:44:21.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:21 smithi070 ceph-mon[30388]: pgmap v1008: 168 pgs: 31 active+undersized, 18 active+undersized+degraded, 119 active+clean; 367 MiB data, 10 GiB used, 705 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.79k op/s; 2232/18210 objects degraded (12.257%) 2024-09-16T09:44:21.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:21 smithi070 ceph-mon[30388]: 5.7 scrub starts 2024-09-16T09:44:21.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:21 smithi070 ceph-mon[30388]: 5.7 scrub ok 2024-09-16T09:44:21.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:21 smithi070 ceph-mon[40691]: pgmap v1008: 168 pgs: 31 active+undersized, 18 active+undersized+degraded, 119 active+clean; 367 MiB data, 10 GiB used, 705 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.79k op/s; 2232/18210 objects degraded (12.257%) 2024-09-16T09:44:21.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:21 smithi070 ceph-mon[40691]: 5.7 scrub starts 2024-09-16T09:44:21.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:21 smithi070 ceph-mon[40691]: 5.7 scrub ok 2024-09-16T09:44:21.515 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:44:21.548 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:21.549 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:21.549 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:44:21.650 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:44:21.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:21 smithi084 ceph-mon[32629]: pgmap v1008: 168 pgs: 31 active+undersized, 18 active+undersized+degraded, 119 active+clean; 367 MiB data, 10 GiB used, 705 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.79k op/s; 2232/18210 objects degraded (12.257%) 2024-09-16T09:44:21.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:21 smithi084 ceph-mon[32629]: 5.7 scrub starts 2024-09-16T09:44:21.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:21 smithi084 ceph-mon[32629]: 5.7 scrub ok 2024-09-16T09:44:21.677 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:21.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:21.678 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:44:21.698 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-16T09:44:21.699 INFO:tasks.cephadm.ceph_manager.ceph:waiting on admin_socket for osd-2, ['dump_ops_in_flight'] 2024-09-16T09:44:21.780 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:44:21.814 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:21.814 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:21.815 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:44:21.915 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:44:21.936 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:21.937 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:21.937 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:44:22.038 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:44:22.059 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:22.060 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:22.060 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:44:22.161 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:44:22.193 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:22.193 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:22.194 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:44:22.256 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:44:21 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:44:21] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:44:22.256 INFO:journalctl@ceph.osd.2.smithi070.stdout:Sep 16 09:44:21 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-2[213542]: 2024-09-16T09:44:21.926+0000 7f139745e640 -1 osd.2 494 log_to_monitors true 2024-09-16T09:44:22.295 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:44:22.323 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:22.324 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:22.324 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:44:22.425 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:44:22.448 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:22.449 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:22.450 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:44:22.550 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:44:22.589 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:22.590 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:22.590 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:44:22.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:22 smithi084 ceph-mon[32629]: 1.4 scrub starts 2024-09-16T09:44:22.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:22 smithi084 ceph-mon[32629]: 1.4 scrub ok 2024-09-16T09:44:22.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:22 smithi084 ceph-mon[32629]: from='osd.2 [v2:172.21.15.70:6818/3826536127,v1:172.21.15.70:6819/3826536127]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-16T09:44:22.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:22 smithi084 ceph-mon[32629]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-16T09:44:22.669 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:22 smithi070 ceph-mon[30388]: 1.4 scrub starts 2024-09-16T09:44:22.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:22 smithi070 ceph-mon[30388]: 1.4 scrub ok 2024-09-16T09:44:22.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:22 smithi070 ceph-mon[30388]: from='osd.2 [v2:172.21.15.70:6818/3826536127,v1:172.21.15.70:6819/3826536127]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-16T09:44:22.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:22 smithi070 ceph-mon[30388]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-16T09:44:22.670 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:22 smithi070 ceph-mon[40691]: 1.4 scrub starts 2024-09-16T09:44:22.670 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:22 smithi070 ceph-mon[40691]: 1.4 scrub ok 2024-09-16T09:44:22.670 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:22 smithi070 ceph-mon[40691]: from='osd.2 [v2:172.21.15.70:6818/3826536127,v1:172.21.15.70:6819/3826536127]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-16T09:44:22.670 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:22 smithi070 ceph-mon[40691]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-16T09:44:22.671 INFO:journalctl@ceph.osd.2.smithi070.stdout:Sep 16 09:44:22 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-2[213542]: 2024-09-16T09:44:22.457+0000 7f138f208640 -1 osd.2 494 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T09:44:22.691 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:44:22.723 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:22.724 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:22.724 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:44:22.825 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:44:22.839 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:44:22.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:22.868 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:22.868 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:44:22.969 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:44:23.003 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:23.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:23.005 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:44:23.050 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.6 dump_ops_in_flight 2024-09-16T09:44:23.106 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:44:23.149 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:23.150 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Removed 130467 objects 2024-09-16T09:44:23.150 INFO:tasks.radosbench.radosbench.0.smithi070.stdout:Clean up completed and total clean up time :93.0399 2024-09-16T09:44:23.150 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:23.151 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:44:23.252 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:44:23.272 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:23 smithi084 ceph-mon[32629]: pgmap v1009: 168 pgs: 31 active+undersized, 18 active+undersized+degraded, 119 active+clean; 367 MiB data, 10 GiB used, 705 GiB / 715 GiB avail; 511 B/s rd, 0 B/s wr, 970 op/s; 2232/18210 objects degraded (12.257%) 2024-09-16T09:44:23.272 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:23 smithi084 ceph-mon[32629]: 4.8 scrub starts 2024-09-16T09:44:23.272 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:23 smithi084 ceph-mon[32629]: 4.8 scrub ok 2024-09-16T09:44:23.293 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:23.294 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:23.294 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:44:23.352 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:44:23.395 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:44:23.419 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:23.419 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:23.420 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:44:23.435 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:44:23.435 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:44:23.452 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:44:23.452 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:44:23.468 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:44:23.468 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:44:23.484 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:44:23.484 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:44:23.501 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:44:23.501 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:44:23.517 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:44:23.518 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:44:23.521 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:44:23.534 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:44:23.534 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:44:23.535 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:23 smithi070 ceph-mon[30388]: pgmap v1009: 168 pgs: 31 active+undersized, 18 active+undersized+degraded, 119 active+clean; 367 MiB data, 10 GiB used, 705 GiB / 715 GiB avail; 511 B/s rd, 0 B/s wr, 970 op/s; 2232/18210 objects degraded (12.257%) 2024-09-16T09:44:23.535 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:23 smithi070 ceph-mon[30388]: 4.8 scrub starts 2024-09-16T09:44:23.535 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:23 smithi070 ceph-mon[30388]: 4.8 scrub ok 2024-09-16T09:44:23.535 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:23 smithi070 ceph-mon[30388]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-16T09:44:23.535 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:23 smithi070 ceph-mon[30388]: from='osd.2 [v2:172.21.15.70:6818/3826536127,v1:172.21.15.70:6819/3826536127]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi070", "root=default"]} : dispatch 2024-09-16T09:44:23.535 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:23 smithi070 ceph-mon[30388]: osdmap e515: 8 total, 7 up, 8 in 2024-09-16T09:44:23.535 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:23 smithi070 ceph-mon[30388]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi070", "root=default"]} : dispatch 2024-09-16T09:44:23.535 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:23 smithi070 ceph-mon[30388]: 1.2 scrub starts 2024-09-16T09:44:23.535 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:23 smithi070 ceph-mon[30388]: 1.2 scrub ok 2024-09-16T09:44:23.535 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:23 smithi070 ceph-mon[40691]: pgmap v1009: 168 pgs: 31 active+undersized, 18 active+undersized+degraded, 119 active+clean; 367 MiB data, 10 GiB used, 705 GiB / 715 GiB avail; 511 B/s rd, 0 B/s wr, 970 op/s; 2232/18210 objects degraded (12.257%) 2024-09-16T09:44:23.535 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:23 smithi070 ceph-mon[40691]: 4.8 scrub starts 2024-09-16T09:44:23.536 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:23 smithi070 ceph-mon[40691]: 4.8 scrub ok 2024-09-16T09:44:23.536 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:23 smithi070 ceph-mon[40691]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-16T09:44:23.536 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:23 smithi070 ceph-mon[40691]: from='osd.2 [v2:172.21.15.70:6818/3826536127,v1:172.21.15.70:6819/3826536127]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi070", "root=default"]} : dispatch 2024-09-16T09:44:23.536 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:23 smithi070 ceph-mon[40691]: osdmap e515: 8 total, 7 up, 8 in 2024-09-16T09:44:23.536 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:23 smithi070 ceph-mon[40691]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi070", "root=default"]} : dispatch 2024-09-16T09:44:23.536 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:23 smithi070 ceph-mon[40691]: 1.2 scrub starts 2024-09-16T09:44:23.536 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:23 smithi070 ceph-mon[40691]: 1.2 scrub ok 2024-09-16T09:44:23.540 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:23 smithi084 ceph-mon[32629]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-16T09:44:23.540 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:23 smithi084 ceph-mon[32629]: from='osd.2 [v2:172.21.15.70:6818/3826536127,v1:172.21.15.70:6819/3826536127]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi070", "root=default"]} : dispatch 2024-09-16T09:44:23.540 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:23 smithi084 ceph-mon[32629]: osdmap e515: 8 total, 7 up, 8 in 2024-09-16T09:44:23.541 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:23 smithi084 ceph-mon[32629]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi070", "root=default"]} : dispatch 2024-09-16T09:44:23.541 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:23 smithi084 ceph-mon[32629]: 1.2 scrub starts 2024-09-16T09:44:23.541 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:23 smithi084 ceph-mon[32629]: 1.2 scrub ok 2024-09-16T09:44:23.542 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:23.543 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:23.543 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:44:23.551 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:44:23.552 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:44:23.644 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:44:23.668 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:23.668 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:23.669 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:44:23.769 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:44:23.792 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:23.793 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:23.793 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:44:23.818 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:44:23 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:44:23.894 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:44:23.917 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:23.918 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:23.918 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:44:23.968 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-09-16T09:44:23.969 INFO:teuthology.orchestra.run.smithi084.stdout: "ops": [], 2024-09-16T09:44:23.969 INFO:teuthology.orchestra.run.smithi084.stdout: "num_ops": 0 2024-09-16T09:44:23.969 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-09-16T09:44:24.018 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:44:24.041 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:24.041 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:24.042 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:44:24.142 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:44:24.165 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:24.166 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:24.167 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:44:24.268 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:44:24.296 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:24.296 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:24.297 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:44:24.398 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:44:24.419 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:24.420 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:24.420 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:44:24.521 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:44:24.543 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:24.543 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:24.544 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:44:24.644 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:44:24.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:24 smithi084 ceph-mon[32629]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-16T09:44:24.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:24 smithi084 ceph-mon[32629]: osd.2 [v2:172.21.15.70:6818/3826536127,v1:172.21.15.70:6819/3826536127] boot 2024-09-16T09:44:24.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:24 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-16T09:44:24.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:24 smithi084 ceph-mon[32629]: osdmap e516: 8 total, 8 up, 8 in 2024-09-16T09:44:24.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:24 smithi084 ceph-mon[32629]: 3.14 scrub starts 2024-09-16T09:44:24.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:24 smithi084 ceph-mon[32629]: 3.14 scrub ok 2024-09-16T09:44:24.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:24 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1114088500' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:44:24.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:24 smithi084 ceph-mon[32629]: osdmap e517: 8 total, 8 up, 8 in 2024-09-16T09:44:24.667 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:24.667 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:24.668 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:44:24.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:24 smithi070 ceph-mon[30388]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-16T09:44:24.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:24 smithi070 ceph-mon[30388]: osd.2 [v2:172.21.15.70:6818/3826536127,v1:172.21.15.70:6819/3826536127] boot 2024-09-16T09:44:24.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:24 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-16T09:44:24.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:24 smithi070 ceph-mon[30388]: osdmap e516: 8 total, 8 up, 8 in 2024-09-16T09:44:24.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:24 smithi070 ceph-mon[30388]: 3.14 scrub starts 2024-09-16T09:44:24.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:24 smithi070 ceph-mon[30388]: 3.14 scrub ok 2024-09-16T09:44:24.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:24 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1114088500' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:44:24.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:24 smithi070 ceph-mon[30388]: osdmap e517: 8 total, 8 up, 8 in 2024-09-16T09:44:24.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:24 smithi070 ceph-mon[40691]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-16T09:44:24.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:24 smithi070 ceph-mon[40691]: osd.2 [v2:172.21.15.70:6818/3826536127,v1:172.21.15.70:6819/3826536127] boot 2024-09-16T09:44:24.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:24 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-16T09:44:24.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:24 smithi070 ceph-mon[40691]: osdmap e516: 8 total, 8 up, 8 in 2024-09-16T09:44:24.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:24 smithi070 ceph-mon[40691]: 3.14 scrub starts 2024-09-16T09:44:24.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:24 smithi070 ceph-mon[40691]: 3.14 scrub ok 2024-09-16T09:44:24.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:24 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1114088500' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:44:24.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:24 smithi070 ceph-mon[40691]: osdmap e517: 8 total, 8 up, 8 in 2024-09-16T09:44:24.719 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-16T09:44:24.768 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:44:24.789 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:24.790 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:24.790 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:44:24.849 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.6 dump_blocked_ops 2024-09-16T09:44:24.892 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:44:24.913 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:24.914 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:24.915 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:44:24.970 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:44:25.016 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:44:25.038 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:25.039 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:25.039 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:44:25.117 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:44:25.140 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:44:25.162 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:25.162 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:25.163 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:44:25.263 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:44:25.293 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:25.294 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:25.294 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:44:25.395 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:44:25.400 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:25 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:44:25.400 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:25 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:44:25.400 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:25 smithi070 ceph-mon[30388]: 4.1f scrub starts 2024-09-16T09:44:25.400 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:25 smithi070 ceph-mon[30388]: 4.1f scrub ok 2024-09-16T09:44:25.400 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:25 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:44:25.400 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:25 smithi070 ceph-mon[30388]: pgmap v1012: 168 pgs: 23 peering, 15 active+undersized, 11 active+undersized+degraded, 119 active+clean; 50 MiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 1.76k op/s; 735/2970 objects degraded (24.747%) 2024-09-16T09:44:25.400 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:25 smithi070 ceph-mon[30388]: 1.13 scrub starts 2024-09-16T09:44:25.400 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:25 smithi070 ceph-mon[30388]: 1.13 scrub ok 2024-09-16T09:44:25.400 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:25 smithi070 ceph-mon[30388]: Health check update: Degraded data redundancy: 735/2970 objects degraded (24.747%), 11 pgs degraded, 26 pgs undersized (PG_DEGRADED) 2024-09-16T09:44:25.400 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:25 smithi070 ceph-mon[30388]: osdmap e518: 8 total, 8 up, 8 in 2024-09-16T09:44:25.401 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:25 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:44:25.401 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:25 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:44:25.401 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:25 smithi070 ceph-mon[40691]: 4.1f scrub starts 2024-09-16T09:44:25.401 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:25 smithi070 ceph-mon[40691]: 4.1f scrub ok 2024-09-16T09:44:25.401 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:25 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:44:25.401 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:25 smithi070 ceph-mon[40691]: pgmap v1012: 168 pgs: 23 peering, 15 active+undersized, 11 active+undersized+degraded, 119 active+clean; 50 MiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 1.76k op/s; 735/2970 objects degraded (24.747%) 2024-09-16T09:44:25.401 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:25 smithi070 ceph-mon[40691]: 1.13 scrub starts 2024-09-16T09:44:25.401 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:25 smithi070 ceph-mon[40691]: 1.13 scrub ok 2024-09-16T09:44:25.401 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:25 smithi070 ceph-mon[40691]: Health check update: Degraded data redundancy: 735/2970 objects degraded (24.747%), 11 pgs degraded, 26 pgs undersized (PG_DEGRADED) 2024-09-16T09:44:25.401 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:25 smithi070 ceph-mon[40691]: osdmap e518: 8 total, 8 up, 8 in 2024-09-16T09:44:25.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:25 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:44:25.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:25 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:44:25.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:25 smithi084 ceph-mon[32629]: 4.1f scrub starts 2024-09-16T09:44:25.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:25 smithi084 ceph-mon[32629]: 4.1f scrub ok 2024-09-16T09:44:25.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:25 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:44:25.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:25 smithi084 ceph-mon[32629]: pgmap v1012: 168 pgs: 23 peering, 15 active+undersized, 11 active+undersized+degraded, 119 active+clean; 50 MiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 1.76k op/s; 735/2970 objects degraded (24.747%) 2024-09-16T09:44:25.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:25 smithi084 ceph-mon[32629]: 1.13 scrub starts 2024-09-16T09:44:25.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:25 smithi084 ceph-mon[32629]: 1.13 scrub ok 2024-09-16T09:44:25.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:25 smithi084 ceph-mon[32629]: Health check update: Degraded data redundancy: 735/2970 objects degraded (24.747%), 11 pgs degraded, 26 pgs undersized (PG_DEGRADED) 2024-09-16T09:44:25.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:25 smithi084 ceph-mon[32629]: osdmap e518: 8 total, 8 up, 8 in 2024-09-16T09:44:25.417 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:25.417 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:25.418 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:44:25.518 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:44:25.540 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:25.540 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:25.541 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:44:25.641 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:44:25.664 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:25.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:25.665 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:44:25.766 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:44:25.789 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:25.790 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:25.790 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:44:25.891 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:44:25.918 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:25.919 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:25.919 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:44:26.020 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:44:26.043 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:26.043 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:26.044 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:44:26.144 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:44:26.168 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:26.168 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:26.169 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:44:26.177 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-09-16T09:44:26.177 INFO:teuthology.orchestra.run.smithi084.stdout: "ops": [], 2024-09-16T09:44:26.177 INFO:teuthology.orchestra.run.smithi084.stdout: "complaint_time": 30, 2024-09-16T09:44:26.177 INFO:teuthology.orchestra.run.smithi084.stdout: "num_blocked_ops": 0 2024-09-16T09:44:26.178 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-09-16T09:44:26.269 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:44:26.289 INFO:teuthology.orchestra.run.smithi070.stderr:nodeep-scrub is unset 2024-09-16T09:44:26.292 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:26.292 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:26.293 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:44:26.373 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-16T09:44:26.393 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:44:26.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:26 smithi084 ceph-mon[32629]: 1.14 scrub starts 2024-09-16T09:44:26.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:26 smithi084 ceph-mon[32629]: 1.14 scrub ok 2024-09-16T09:44:26.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:26 smithi084 ceph-mon[32629]: 6.0 scrub starts 2024-09-16T09:44:26.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:26 smithi084 ceph-mon[32629]: 6.0 scrub ok 2024-09-16T09:44:26.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:26 smithi084 ceph-mon[32629]: 6.f scrub starts 2024-09-16T09:44:26.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:26 smithi084 ceph-mon[32629]: 6.f scrub ok 2024-09-16T09:44:26.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:26 smithi084 ceph-mon[32629]: 11.d starting backfill to osd.2 from (471'4900,494'9272] MAX to 515'16530 2024-09-16T09:44:26.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:26 smithi084 ceph-mon[32629]: 11.2 starting backfill to osd.2 from (471'4800,494'9092] MAX to 515'16264 2024-09-16T09:44:26.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:26 smithi084 ceph-mon[32629]: 11.c starting backfill to osd.2 from (464'2914,494'9098] MAX to 515'16242 2024-09-16T09:44:26.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:26 smithi084 ceph-mon[32629]: 11.f starting backfill to osd.2 from (471'4700,494'9140] MAX to 515'16282 2024-09-16T09:44:26.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:26 smithi084 ceph-mon[32629]: 11.4 starting backfill to osd.2 from (471'4800,494'9190] MAX to 515'16314 2024-09-16T09:44:26.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:26 smithi084 ceph-mon[32629]: 11.6 starting backfill to osd.2 from (467'3800,494'9240] MAX to 515'16432 2024-09-16T09:44:26.411 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:26 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3079512644' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:44:26.412 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:26 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:44:26.416 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:26.416 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:26.417 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:44:26.518 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:44:26.540 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:26.541 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:26.541 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:44:26.642 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:44:26.665 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:26.666 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:26.666 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:44:26.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:26 smithi070 ceph-mon[40691]: 1.14 scrub starts 2024-09-16T09:44:26.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:26 smithi070 ceph-mon[40691]: 1.14 scrub ok 2024-09-16T09:44:26.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:26 smithi070 ceph-mon[40691]: 6.0 scrub starts 2024-09-16T09:44:26.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:26 smithi070 ceph-mon[40691]: 6.0 scrub ok 2024-09-16T09:44:26.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:26 smithi070 ceph-mon[40691]: 6.f scrub starts 2024-09-16T09:44:26.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:26 smithi070 ceph-mon[40691]: 6.f scrub ok 2024-09-16T09:44:26.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:26 smithi070 ceph-mon[40691]: 11.d starting backfill to osd.2 from (471'4900,494'9272] MAX to 515'16530 2024-09-16T09:44:26.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:26 smithi070 ceph-mon[40691]: 11.2 starting backfill to osd.2 from (471'4800,494'9092] MAX to 515'16264 2024-09-16T09:44:26.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:26 smithi070 ceph-mon[40691]: 11.c starting backfill to osd.2 from (464'2914,494'9098] MAX to 515'16242 2024-09-16T09:44:26.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:26 smithi070 ceph-mon[40691]: 11.f starting backfill to osd.2 from (471'4700,494'9140] MAX to 515'16282 2024-09-16T09:44:26.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:26 smithi070 ceph-mon[40691]: 11.4 starting backfill to osd.2 from (471'4800,494'9190] MAX to 515'16314 2024-09-16T09:44:26.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:26 smithi070 ceph-mon[40691]: 11.6 starting backfill to osd.2 from (467'3800,494'9240] MAX to 515'16432 2024-09-16T09:44:26.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:26 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3079512644' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:44:26.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:26 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:44:26.674 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:26 smithi070 ceph-mon[30388]: 1.14 scrub starts 2024-09-16T09:44:26.674 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:26 smithi070 ceph-mon[30388]: 1.14 scrub ok 2024-09-16T09:44:26.674 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:26 smithi070 ceph-mon[30388]: 6.0 scrub starts 2024-09-16T09:44:26.674 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:26 smithi070 ceph-mon[30388]: 6.0 scrub ok 2024-09-16T09:44:26.674 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:26 smithi070 ceph-mon[30388]: 6.f scrub starts 2024-09-16T09:44:26.674 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:26 smithi070 ceph-mon[30388]: 6.f scrub ok 2024-09-16T09:44:26.674 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:26 smithi070 ceph-mon[30388]: 11.d starting backfill to osd.2 from (471'4900,494'9272] MAX to 515'16530 2024-09-16T09:44:26.674 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:26 smithi070 ceph-mon[30388]: 11.2 starting backfill to osd.2 from (471'4800,494'9092] MAX to 515'16264 2024-09-16T09:44:26.674 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:26 smithi070 ceph-mon[30388]: 11.c starting backfill to osd.2 from (464'2914,494'9098] MAX to 515'16242 2024-09-16T09:44:26.674 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:26 smithi070 ceph-mon[30388]: 11.f starting backfill to osd.2 from (471'4700,494'9140] MAX to 515'16282 2024-09-16T09:44:26.674 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:26 smithi070 ceph-mon[30388]: 11.4 starting backfill to osd.2 from (471'4800,494'9190] MAX to 515'16314 2024-09-16T09:44:26.674 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:26 smithi070 ceph-mon[30388]: 11.6 starting backfill to osd.2 from (467'3800,494'9240] MAX to 515'16432 2024-09-16T09:44:26.674 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:26 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3079512644' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:44:26.674 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:26 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:44:26.700 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.2 dump_ops_in_flight 2024-09-16T09:44:26.767 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:44:26.789 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:26.789 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:26.790 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:44:26.891 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:44:26.912 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:26.912 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:26.913 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:44:27.014 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:44:27.036 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:27.036 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:27.037 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:44:27.137 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:44:27.159 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:27.160 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:27.160 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:44:27.237 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.6 dump_historic_ops 2024-09-16T09:44:27.261 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:44:27.305 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:27.306 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:27.306 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:44:27.407 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:44:27.428 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:27.429 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:27.429 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:44:27.507 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:44:27.529 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:44:27.553 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:27.553 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:27.553 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:44:27.628 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:27 smithi084 ceph-mon[32629]: 6.1 scrub starts 2024-09-16T09:44:27.629 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:27 smithi084 ceph-mon[32629]: 6.1 scrub ok 2024-09-16T09:44:27.629 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:27 smithi084 ceph-mon[32629]: pgmap v1015: 168 pgs: 1 remapped+peering, 23 peering, 10 active+undersized, 8 active+undersized+degraded, 126 active+clean; 35 MiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.33k op/s; 484/2238 objects degraded (21.626%) 2024-09-16T09:44:27.629 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:27 smithi084 ceph-mon[32629]: 3.5 scrub starts 2024-09-16T09:44:27.629 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:27 smithi084 ceph-mon[32629]: 3.5 scrub ok 2024-09-16T09:44:27.629 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:27 smithi084 ceph-mon[32629]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-16T09:44:27.629 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:27 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-16T09:44:27.629 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:27 smithi084 ceph-mon[32629]: osdmap e519: 8 total, 8 up, 8 in 2024-09-16T09:44:27.654 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:44:27.670 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:27 smithi070 ceph-mon[40691]: 6.1 scrub starts 2024-09-16T09:44:27.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:27 smithi070 ceph-mon[40691]: 6.1 scrub ok 2024-09-16T09:44:27.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:27 smithi070 ceph-mon[40691]: pgmap v1015: 168 pgs: 1 remapped+peering, 23 peering, 10 active+undersized, 8 active+undersized+degraded, 126 active+clean; 35 MiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.33k op/s; 484/2238 objects degraded (21.626%) 2024-09-16T09:44:27.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:27 smithi070 ceph-mon[40691]: 3.5 scrub starts 2024-09-16T09:44:27.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:27 smithi070 ceph-mon[40691]: 3.5 scrub ok 2024-09-16T09:44:27.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:27 smithi070 ceph-mon[40691]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-16T09:44:27.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:27 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-16T09:44:27.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:27 smithi070 ceph-mon[40691]: osdmap e519: 8 total, 8 up, 8 in 2024-09-16T09:44:27.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:27 smithi070 ceph-mon[30388]: 6.1 scrub starts 2024-09-16T09:44:27.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:27 smithi070 ceph-mon[30388]: 6.1 scrub ok 2024-09-16T09:44:27.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:27 smithi070 ceph-mon[30388]: pgmap v1015: 168 pgs: 1 remapped+peering, 23 peering, 10 active+undersized, 8 active+undersized+degraded, 126 active+clean; 35 MiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.33k op/s; 484/2238 objects degraded (21.626%) 2024-09-16T09:44:27.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:27 smithi070 ceph-mon[30388]: 3.5 scrub starts 2024-09-16T09:44:27.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:27 smithi070 ceph-mon[30388]: 3.5 scrub ok 2024-09-16T09:44:27.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:27 smithi070 ceph-mon[30388]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-16T09:44:27.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:27 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-16T09:44:27.672 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:27 smithi070 ceph-mon[30388]: osdmap e519: 8 total, 8 up, 8 in 2024-09-16T09:44:27.674 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:27.675 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:27.675 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:44:27.776 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:44:27.798 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:27.799 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:27.799 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:44:27.900 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:44:27.921 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:27.922 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:27.922 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:44:28.023 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:44:28.045 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:28.045 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:28.046 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:44:28.122 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-09-16T09:44:28.122 INFO:teuthology.orchestra.run.smithi084.stdout: "size": 20, 2024-09-16T09:44:28.122 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 600, 2024-09-16T09:44:28.122 INFO:teuthology.orchestra.run.smithi084.stdout: "ops": [ 2024-09-16T09:44:28.122 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.122 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:222663 9.1 9:817a7893:::benchmark_data_smithi070_137000_object100952:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e349)", 2024-09-16T09:44:28.122 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:37:10.363067+0000", 2024-09-16T09:44:28.122 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 437.756103467, 2024-09-16T09:44:28.122 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.030122435, 2024-09-16T09:44:28.123 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:44:28.123 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:44:28.123 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "reached pg", 2024-09-16T09:44:28.123 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:44:28.123 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:44:28.123 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:44:28.123 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 222663 2024-09-16T09:44:28.123 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.123 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:44:28.123 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.123 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:44:28.123 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363067+0000", 2024-09-16T09:44:28.123 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:44:28.124 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.124 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.124 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:44:28.124 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363067+0000", 2024-09-16T09:44:28.124 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:44:28.124 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.124 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.124 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:44:28.124 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363074+0000", 2024-09-16T09:44:28.124 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.905e-06 2024-09-16T09:44:28.124 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.125 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.125 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:44:28.125 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363083+0000", 2024-09-16T09:44:28.125 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.5019999999999992e-06 2024-09-16T09:44:28.125 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.125 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.125 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:44:28.125 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363099+0000", 2024-09-16T09:44:28.125 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.6427999999999999e-05 2024-09-16T09:44:28.125 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.125 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.126 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:44:28.126 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363108+0000", 2024-09-16T09:44:28.126 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.1900000000000001e-06 2024-09-16T09:44:28.126 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.126 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.126 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:44:28.126 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.397601+0000", 2024-09-16T09:44:28.126 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.034492918999999997 2024-09-16T09:44:28.126 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.126 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.126 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:44:28.126 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.397613+0000", 2024-09-16T09:44:28.127 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.2275e-05 2024-09-16T09:44:28.127 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.127 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.127 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:44:28.127 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.393137+0000", 2024-09-16T09:44:28.127 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.99552360100000004 2024-09-16T09:44:28.127 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.127 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.127 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:44:28.127 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.393190+0000", 2024-09-16T09:44:28.127 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.2614999999999997e-05 2024-09-16T09:44:28.127 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:44:28.128 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:44:28.128 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:44:28.128 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.128 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.128 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:222725 9.1 9:811a4bf7:::benchmark_data_smithi070_137000_object101014:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e349)", 2024-09-16T09:44:28.128 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:37:10.363127+0000", 2024-09-16T09:44:28.128 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 437.75604384899998, 2024-09-16T09:44:28.128 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0316495720000001, 2024-09-16T09:44:28.128 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:44:28.128 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:44:28.128 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:44:28.128 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:44:28.129 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:44:28.129 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:44:28.129 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 222725 2024-09-16T09:44:28.129 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.129 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:44:28.129 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.129 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:44:28.129 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363127+0000", 2024-09-16T09:44:28.129 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:44:28.130 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.130 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.130 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:44:28.130 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363127+0000", 2024-09-16T09:44:28.130 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:44:28.130 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.130 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.130 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:44:28.130 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363127+0000", 2024-09-16T09:44:28.130 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.9499999999999998e-07 2024-09-16T09:44:28.130 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.130 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.130 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:44:28.131 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363129+0000", 2024-09-16T09:44:28.131 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.261e-06 2024-09-16T09:44:28.131 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.131 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.131 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:44:28.131 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363135+0000", 2024-09-16T09:44:28.131 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.1580000000000004e-06 2024-09-16T09:44:28.131 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.131 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.131 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:44:28.131 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363138+0000", 2024-09-16T09:44:28.131 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.7620000000000001e-06 2024-09-16T09:44:28.131 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.132 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.132 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:44:28.132 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.397654+0000", 2024-09-16T09:44:28.132 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.034515420999999998 2024-09-16T09:44:28.132 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.132 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.132 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:44:28.132 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.397664+0000", 2024-09-16T09:44:28.132 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0549e-05 2024-09-16T09:44:28.132 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.132 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.132 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:44:28.132 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.393573+0000", 2024-09-16T09:44:28.133 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.99590824499999997 2024-09-16T09:44:28.133 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.133 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.133 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:44:28.133 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.393659+0000", 2024-09-16T09:44:28.133 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.6203000000000001e-05 2024-09-16T09:44:28.133 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.133 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.133 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 3,7", 2024-09-16T09:44:28.133 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.393727+0000", 2024-09-16T09:44:28.133 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.7935000000000006e-05 2024-09-16T09:44:28.133 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.133 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.134 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:44:28.134 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.394640+0000", 2024-09-16T09:44:28.134 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00091379700000000005 2024-09-16T09:44:28.134 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.134 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.134 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:44:28.134 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.394674+0000", 2024-09-16T09:44:28.134 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.3800000000000002e-05 2024-09-16T09:44:28.134 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.134 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.134 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:44:28.134 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.394719+0000", 2024-09-16T09:44:28.135 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.4880999999999999e-05 2024-09-16T09:44:28.135 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.135 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.135 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:44:28.135 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.394750+0000", 2024-09-16T09:44:28.135 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.0514e-05 2024-09-16T09:44:28.135 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.135 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.135 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:44:28.135 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.394776+0000", 2024-09-16T09:44:28.135 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.6650999999999999e-05 2024-09-16T09:44:28.135 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:44:28.135 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:44:28.136 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:44:28.136 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.136 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.136 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:222759 9.1 9:84f52a19:::benchmark_data_smithi070_137000_object101048:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e349)", 2024-09-16T09:44:28.136 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:37:10.363143+0000", 2024-09-16T09:44:28.136 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 437.75602777500001, 2024-09-16T09:44:28.136 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.03180293, 2024-09-16T09:44:28.136 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:44:28.136 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:44:28.136 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:44:28.136 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:44:28.136 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:44:28.136 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:44:28.136 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 222759 2024-09-16T09:44:28.137 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.137 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:44:28.137 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.137 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:44:28.137 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363143+0000", 2024-09-16T09:44:28.137 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:44:28.137 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.137 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.137 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:44:28.137 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363143+0000", 2024-09-16T09:44:28.137 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:44:28.137 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.137 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.138 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:44:28.138 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363143+0000", 2024-09-16T09:44:28.138 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.2499999999999999e-07 2024-09-16T09:44:28.138 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.138 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.138 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:44:28.138 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363144+0000", 2024-09-16T09:44:28.138 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1510000000000001e-06 2024-09-16T09:44:28.138 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.138 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.138 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:44:28.139 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363149+0000", 2024-09-16T09:44:28.139 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.3189999999999996e-06 2024-09-16T09:44:28.139 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.139 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.139 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:44:28.139 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363152+0000", 2024-09-16T09:44:28.139 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.9969999999999999e-06 2024-09-16T09:44:28.139 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.139 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.139 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:44:28.139 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.397717+0000", 2024-09-16T09:44:28.139 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.034565851000000002 2024-09-16T09:44:28.139 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.140 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.140 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:44:28.140 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.397728+0000", 2024-09-16T09:44:28.140 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0832999999999999e-05 2024-09-16T09:44:28.140 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.140 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.140 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:44:28.140 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.393932+0000", 2024-09-16T09:44:28.140 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.99620329399999996 2024-09-16T09:44:28.141 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.141 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.141 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:44:28.141 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.394030+0000", 2024-09-16T09:44:28.141 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.8080999999999998e-05 2024-09-16T09:44:28.141 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.141 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.141 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 3,7", 2024-09-16T09:44:28.141 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.394094+0000", 2024-09-16T09:44:28.141 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.4604999999999996e-05 2024-09-16T09:44:28.141 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.141 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.141 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:44:28.142 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.394843+0000", 2024-09-16T09:44:28.142 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00074905100000000004 2024-09-16T09:44:28.142 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.142 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.142 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:44:28.142 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.394854+0000", 2024-09-16T09:44:28.142 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0249e-05 2024-09-16T09:44:28.142 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.142 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.142 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:44:28.142 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.394878+0000", 2024-09-16T09:44:28.142 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.4173e-05 2024-09-16T09:44:28.142 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.142 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.143 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:44:28.143 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.394913+0000", 2024-09-16T09:44:28.143 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.5515000000000001e-05 2024-09-16T09:44:28.143 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.143 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.143 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:44:28.143 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.394946+0000", 2024-09-16T09:44:28.143 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.2586000000000003e-05 2024-09-16T09:44:28.143 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:44:28.143 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:44:28.143 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:44:28.143 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.143 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.144 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:222834 9.1 9:85e273e5:::benchmark_data_smithi070_137000_object101123:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e349)", 2024-09-16T09:44:28.144 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:37:10.363155+0000", 2024-09-16T09:44:28.144 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 437.75601596899998, 2024-09-16T09:44:28.144 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.032180122, 2024-09-16T09:44:28.144 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:44:28.144 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:44:28.144 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:44:28.144 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:44:28.144 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:44:28.144 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:44:28.144 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 222834 2024-09-16T09:44:28.144 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.144 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:44:28.145 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.145 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:44:28.145 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363155+0000", 2024-09-16T09:44:28.145 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:44:28.145 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.145 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.145 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:44:28.145 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363155+0000", 2024-09-16T09:44:28.145 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:44:28.145 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.145 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.145 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:44:28.145 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363155+0000", 2024-09-16T09:44:28.146 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.0800000000000001e-07 2024-09-16T09:44:28.146 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.146 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.146 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:44:28.146 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363156+0000", 2024-09-16T09:44:28.146 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.02e-06 2024-09-16T09:44:28.146 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.146 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.146 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:44:28.146 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363160+0000", 2024-09-16T09:44:28.146 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.9770000000000002e-06 2024-09-16T09:44:28.146 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.146 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.146 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:44:28.147 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363163+0000", 2024-09-16T09:44:28.147 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.174e-06 2024-09-16T09:44:28.147 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.147 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.147 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:44:28.147 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.397768+0000", 2024-09-16T09:44:28.147 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.034605455 2024-09-16T09:44:28.147 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.147 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.147 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:44:28.147 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.397776+0000", 2024-09-16T09:44:28.147 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.5429999999999996e-06 2024-09-16T09:44:28.147 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.147 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.148 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:44:28.148 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.394293+0000", 2024-09-16T09:44:28.148 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.99651656700000002 2024-09-16T09:44:28.148 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.148 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.148 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:44:28.148 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.394365+0000", 2024-09-16T09:44:28.148 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.2575000000000002e-05 2024-09-16T09:44:28.148 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.148 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.148 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 3,7", 2024-09-16T09:44:28.148 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.394423+0000", 2024-09-16T09:44:28.148 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.8220000000000002e-05 2024-09-16T09:44:28.149 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.149 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.149 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:44:28.149 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.394986+0000", 2024-09-16T09:44:28.149 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00056240100000000003 2024-09-16T09:44:28.149 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.149 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.149 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:44:28.149 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.394994+0000", 2024-09-16T09:44:28.149 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.4169999999999992e-06 2024-09-16T09:44:28.149 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.149 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.149 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:44:28.149 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.395287+0000", 2024-09-16T09:44:28.149 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00029250199999999999 2024-09-16T09:44:28.150 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.150 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.150 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:44:28.150 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.395313+0000", 2024-09-16T09:44:28.150 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.6721999999999999e-05 2024-09-16T09:44:28.150 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.150 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.150 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:44:28.150 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.395335+0000", 2024-09-16T09:44:28.150 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.1341000000000001e-05 2024-09-16T09:44:28.150 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:44:28.150 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:44:28.150 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:44:28.151 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.151 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.151 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:222844 9.1 9:80f5d6a4:::benchmark_data_smithi070_137000_object101133:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e349)", 2024-09-16T09:44:28.151 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:37:10.363166+0000", 2024-09-16T09:44:28.151 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 437.75600470000001, 2024-09-16T09:44:28.151 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0331230769999999, 2024-09-16T09:44:28.151 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:44:28.151 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:44:28.151 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:44:28.151 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:44:28.151 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:44:28.151 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:44:28.151 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 222844 2024-09-16T09:44:28.151 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.152 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:44:28.152 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.152 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:44:28.152 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363166+0000", 2024-09-16T09:44:28.152 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:44:28.152 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:44:28.154 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.154 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.154 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:44:28.154 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363166+0000", 2024-09-16T09:44:28.154 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:44:28.154 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.154 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.154 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:44:28.154 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363166+0000", 2024-09-16T09:44:28.154 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.0900000000000001e-07 2024-09-16T09:44:28.154 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.155 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.155 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:44:28.155 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363167+0000", 2024-09-16T09:44:28.155 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0189999999999999e-06 2024-09-16T09:44:28.155 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.155 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.155 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:44:28.155 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363170+0000", 2024-09-16T09:44:28.155 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.1259999999999998e-06 2024-09-16T09:44:28.155 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.155 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.155 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:44:28.156 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363173+0000", 2024-09-16T09:44:28.156 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.39e-06 2024-09-16T09:44:28.156 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.156 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.156 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:44:28.156 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.397812+0000", 2024-09-16T09:44:28.156 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.034639239000000002 2024-09-16T09:44:28.156 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.156 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.156 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:44:28.156 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.397821+0000", 2024-09-16T09:44:28.156 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.2629999999999992e-06 2024-09-16T09:44:28.156 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.156 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.157 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:44:28.157 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.395016+0000", 2024-09-16T09:44:28.157 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.99719489299999997 2024-09-16T09:44:28.157 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.157 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.157 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:44:28.157 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.395079+0000", 2024-09-16T09:44:28.157 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.2699000000000004e-05 2024-09-16T09:44:28.157 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.157 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.157 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 3,7", 2024-09-16T09:44:28.157 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.395137+0000", 2024-09-16T09:44:28.157 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.8158000000000001e-05 2024-09-16T09:44:28.158 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.158 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.158 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:44:28.158 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.396134+0000", 2024-09-16T09:44:28.158 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00099750399999999997 2024-09-16T09:44:28.158 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.158 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.158 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:44:28.158 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.396186+0000", 2024-09-16T09:44:28.158 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.1962000000000002e-05 2024-09-16T09:44:28.158 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.158 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.158 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:44:28.159 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.396200+0000", 2024-09-16T09:44:28.159 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.4056000000000001e-05 2024-09-16T09:44:28.159 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.159 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.159 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:44:28.159 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.396246+0000", 2024-09-16T09:44:28.159 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.5121999999999999e-05 2024-09-16T09:44:28.159 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.159 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.159 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:44:28.159 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.396289+0000", 2024-09-16T09:44:28.159 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.3436999999999998e-05 2024-09-16T09:44:28.159 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:44:28.160 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:44:28.160 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:44:28.160 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.160 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.160 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:222861 9.1 9:876cd0fc:::benchmark_data_smithi070_137000_object101150:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e349)", 2024-09-16T09:44:28.160 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:37:10.363175+0000", 2024-09-16T09:44:28.160 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 437.75599515099998, 2024-09-16T09:44:28.160 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.033248876, 2024-09-16T09:44:28.160 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:44:28.160 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:44:28.160 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:44:28.160 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:44:28.160 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:44:28.161 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:44:28.161 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 222861 2024-09-16T09:44:28.161 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.161 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:44:28.161 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.161 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:44:28.161 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363175+0000", 2024-09-16T09:44:28.161 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:44:28.161 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.161 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.161 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:44:28.161 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363175+0000", 2024-09-16T09:44:28.161 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:44:28.161 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.162 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.162 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:44:28.162 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363176+0000", 2024-09-16T09:44:28.162 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.0599999999999999e-07 2024-09-16T09:44:28.162 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.162 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.162 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:44:28.162 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363177+0000", 2024-09-16T09:44:28.162 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.062e-06 2024-09-16T09:44:28.162 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.162 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.162 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:44:28.162 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363181+0000", 2024-09-16T09:44:28.162 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.7840000000000001e-06 2024-09-16T09:44:28.163 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.163 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.163 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:44:28.163 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363183+0000", 2024-09-16T09:44:28.163 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.8140000000000001e-06 2024-09-16T09:44:28.163 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.163 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.163 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:44:28.163 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.397869+0000", 2024-09-16T09:44:28.163 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.034685939999999998 2024-09-16T09:44:28.163 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.163 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.163 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:44:28.164 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.397877+0000", 2024-09-16T09:44:28.164 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.157e-06 2024-09-16T09:44:28.164 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.164 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.164 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:44:28.164 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.395362+0000", 2024-09-16T09:44:28.164 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.99748467399999996 2024-09-16T09:44:28.164 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.164 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.164 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:44:28.164 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.395428+0000", 2024-09-16T09:44:28.164 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.5549999999999994e-05 2024-09-16T09:44:28.164 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.165 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.165 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 3,7", 2024-09-16T09:44:28.165 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.395484+0000", 2024-09-16T09:44:28.165 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.6709999999999997e-05 2024-09-16T09:44:28.165 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.165 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.165 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:44:28.165 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.396296+0000", 2024-09-16T09:44:28.165 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00081205299999999997 2024-09-16T09:44:28.165 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.165 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.165 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:44:28.165 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.396337+0000", 2024-09-16T09:44:28.165 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.0167999999999999e-05 2024-09-16T09:44:28.166 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.166 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.166 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:44:28.166 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.396370+0000", 2024-09-16T09:44:28.166 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.3441000000000001e-05 2024-09-16T09:44:28.166 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.166 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.166 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:44:28.166 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.396400+0000", 2024-09-16T09:44:28.166 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.9737999999999999e-05 2024-09-16T09:44:28.166 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.166 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.166 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:44:28.167 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.396424+0000", 2024-09-16T09:44:28.167 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.4579e-05 2024-09-16T09:44:28.167 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:44:28.167 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:44:28.167 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:44:28.167 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.167 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.167 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:222881 9.1 9:86c78cd8:::benchmark_data_smithi070_137000_object101170:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e349)", 2024-09-16T09:44:28.167 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:37:10.363186+0000", 2024-09-16T09:44:28.167 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 437.75598451399998, 2024-09-16T09:44:28.167 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0333547729999999, 2024-09-16T09:44:28.167 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:44:28.167 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:44:28.167 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:44:28.168 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:44:28.168 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:44:28.168 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:44:28.168 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 222881 2024-09-16T09:44:28.168 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.168 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:44:28.168 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.168 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:44:28.168 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363186+0000", 2024-09-16T09:44:28.168 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:44:28.168 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.168 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.168 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:44:28.169 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363186+0000", 2024-09-16T09:44:28.169 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:44:28.169 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.169 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.169 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:44:28.169 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363186+0000", 2024-09-16T09:44:28.169 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.9999999999999999e-07 2024-09-16T09:44:28.169 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.169 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.169 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:44:28.169 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363187+0000", 2024-09-16T09:44:28.169 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.125e-06 2024-09-16T09:44:28.169 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.170 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.170 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:44:28.170 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363191+0000", 2024-09-16T09:44:28.170 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.7969999999999998e-06 2024-09-16T09:44:28.170 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.170 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.170 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:44:28.170 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363194+0000", 2024-09-16T09:44:28.170 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.7489999999999999e-06 2024-09-16T09:44:28.170 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.170 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.170 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:44:28.170 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.397946+0000", 2024-09-16T09:44:28.170 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.034751690000000002 2024-09-16T09:44:28.171 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.171 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.171 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:44:28.171 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.397955+0000", 2024-09-16T09:44:28.171 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.5990000000000008e-06 2024-09-16T09:44:28.171 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.171 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.171 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:44:28.171 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.395673+0000", 2024-09-16T09:44:28.171 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.99771777100000003 2024-09-16T09:44:28.171 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.171 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.171 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:44:28.172 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.395791+0000", 2024-09-16T09:44:28.172 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00011786300000000001 2024-09-16T09:44:28.172 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.172 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.172 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 3,7", 2024-09-16T09:44:28.172 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.395897+0000", 2024-09-16T09:44:28.172 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00010618 2024-09-16T09:44:28.172 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.172 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.172 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:44:28.172 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.396433+0000", 2024-09-16T09:44:28.172 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00053625899999999998 2024-09-16T09:44:28.172 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.173 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.173 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:44:28.173 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.396463+0000", 2024-09-16T09:44:28.173 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.9346999999999999e-05 2024-09-16T09:44:28.173 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.173 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.173 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:44:28.173 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.396495+0000", 2024-09-16T09:44:28.173 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.2490000000000002e-05 2024-09-16T09:44:28.173 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.173 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.173 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:44:28.173 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.396520+0000", 2024-09-16T09:44:28.173 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.5052e-05 2024-09-16T09:44:28.174 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.174 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.174 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:44:28.174 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.396541+0000", 2024-09-16T09:44:28.174 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.0650999999999999e-05 2024-09-16T09:44:28.174 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:44:28.174 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:44:28.174 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:44:28.174 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.174 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.174 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:222925 9.1 9:81973e20:::benchmark_data_smithi070_137000_object101214:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e349)", 2024-09-16T09:44:28.174 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:37:10.363197+0000", 2024-09-16T09:44:28.175 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 437.755973283, 2024-09-16T09:44:28.175 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0353614579999999, 2024-09-16T09:44:28.175 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:44:28.175 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:44:28.175 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:44:28.175 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:44:28.175 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:44:28.175 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:44:28.175 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 222925 2024-09-16T09:44:28.175 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.175 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:44:28.175 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.175 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:44:28.176 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363197+0000", 2024-09-16T09:44:28.176 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:44:28.176 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.176 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.176 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:44:28.176 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363197+0000", 2024-09-16T09:44:28.176 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:44:28.176 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.176 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.176 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:44:28.176 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363198+0000", 2024-09-16T09:44:28.176 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.0200000000000001e-07 2024-09-16T09:44:28.176 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.176 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.177 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:44:28.177 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363199+0000", 2024-09-16T09:44:28.177 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.079e-06 2024-09-16T09:44:28.177 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.177 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.177 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:44:28.177 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363203+0000", 2024-09-16T09:44:28.177 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.8920000000000002e-06 2024-09-16T09:44:28.177 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.177 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.177 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:44:28.177 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.363205+0000", 2024-09-16T09:44:28.177 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.7539999999999998e-06 2024-09-16T09:44:28.178 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.178 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.178 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:44:28.178 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.398001+0000", 2024-09-16T09:44:28.178 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.034795595999999998 2024-09-16T09:44:28.178 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.178 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.178 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:44:28.178 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:10.398008+0000", 2024-09-16T09:44:28.178 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.5850000000000002e-06 2024-09-16T09:44:28.178 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.178 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.178 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:44:28.178 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.396565+0000", 2024-09-16T09:44:28.179 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.99855653799999999 2024-09-16T09:44:28.179 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.179 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.179 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:44:28.179 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.396627+0000", 2024-09-16T09:44:28.179 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.2395000000000001e-05 2024-09-16T09:44:28.179 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.179 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.179 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 3,7", 2024-09-16T09:44:28.179 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.396684+0000", 2024-09-16T09:44:28.179 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.7045e-05 2024-09-16T09:44:28.179 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.179 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.180 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:44:28.180 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.397837+0000", 2024-09-16T09:44:28.180 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0011528070000000001 2024-09-16T09:44:28.180 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.180 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.180 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:44:28.180 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.397880+0000", 2024-09-16T09:44:28.180 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.2701999999999999e-05 2024-09-16T09:44:28.180 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.180 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.180 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:44:28.180 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.398501+0000", 2024-09-16T09:44:28.180 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00062094699999999995 2024-09-16T09:44:28.180 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.181 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.181 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:44:28.181 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.398524+0000", 2024-09-16T09:44:28.181 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.3388e-05 2024-09-16T09:44:28.181 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.181 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.181 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:44:28.181 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:11.398559+0000", 2024-09-16T09:44:28.181 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.4527999999999999e-05 2024-09-16T09:44:28.181 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:44:28.181 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:44:28.181 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:44:28.181 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.182 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.182 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:225149 9.0 9:0124f23a:::benchmark_data_smithi070_137000_object103438:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e351)", 2024-09-16T09:44:28.182 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:37:12.384254+0000", 2024-09-16T09:44:28.182 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 435.73491665900002, 2024-09-16T09:44:28.182 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.128723189, 2024-09-16T09:44:28.182 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:44:28.182 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:44:28.182 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "reached pg", 2024-09-16T09:44:28.182 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:44:28.182 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:44:28.182 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:44:28.182 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 225149 2024-09-16T09:44:28.183 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.183 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:44:28.183 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.183 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:44:28.183 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.384254+0000", 2024-09-16T09:44:28.183 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:44:28.183 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.183 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.183 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:44:28.183 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.384254+0000", 2024-09-16T09:44:28.183 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:44:28.183 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.183 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.183 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:44:28.183 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.384255+0000", 2024-09-16T09:44:28.184 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.35e-06 2024-09-16T09:44:28.184 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.184 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.184 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:44:28.184 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.384260+0000", 2024-09-16T09:44:28.184 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.729e-06 2024-09-16T09:44:28.184 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.184 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.184 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:44:28.184 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.384273+0000", 2024-09-16T09:44:28.184 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.3456e-05 2024-09-16T09:44:28.184 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.184 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.184 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:44:28.185 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.384281+0000", 2024-09-16T09:44:28.185 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.515e-06 2024-09-16T09:44:28.185 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.185 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.185 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:44:28.185 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.384332+0000", 2024-09-16T09:44:28.185 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.0850000000000003e-05 2024-09-16T09:44:28.185 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.185 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.185 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:44:28.185 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.384455+0000", 2024-09-16T09:44:28.185 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00012285599999999999 2024-09-16T09:44:28.185 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.185 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.186 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 1,2", 2024-09-16T09:44:28.186 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.384564+0000", 2024-09-16T09:44:28.186 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00010888 2024-09-16T09:44:28.186 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.186 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.186 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:44:28.186 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.387738+0000", 2024-09-16T09:44:28.186 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.003174507 2024-09-16T09:44:28.186 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.186 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.186 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:44:28.186 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.394990+0000", 2024-09-16T09:44:28.186 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0072515399999999999 2024-09-16T09:44:28.186 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.186 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.187 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:44:28.187 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.400340+0000", 2024-09-16T09:44:28.187 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0053498829999999997 2024-09-16T09:44:28.187 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.187 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.187 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:44:28.187 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.400349+0000", 2024-09-16T09:44:28.187 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.9580000000000002e-06 2024-09-16T09:44:28.187 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.187 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.187 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:44:28.187 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:13.512961+0000", 2024-09-16T09:44:28.187 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1126110739999999 2024-09-16T09:44:28.187 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.187 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.188 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:44:28.188 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:13.512977+0000", 2024-09-16T09:44:28.188 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.6591000000000001e-05 2024-09-16T09:44:28.188 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:44:28.188 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:44:28.188 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:44:28.188 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.188 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.188 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:225159 9.0 9:033db585:::benchmark_data_smithi070_137000_object103448:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e351)", 2024-09-16T09:44:28.188 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:37:12.389563+0000", 2024-09-16T09:44:28.188 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 435.729607472, 2024-09-16T09:44:28.188 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1234441260000001, 2024-09-16T09:44:28.188 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:44:28.188 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:44:28.189 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "reached pg", 2024-09-16T09:44:28.189 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:44:28.189 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:44:28.189 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:44:28.189 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 225159 2024-09-16T09:44:28.189 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.189 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:44:28.189 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.189 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:44:28.189 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.389563+0000", 2024-09-16T09:44:28.189 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:44:28.189 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.189 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.190 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:44:28.190 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.389563+0000", 2024-09-16T09:44:28.190 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:44:28.190 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.190 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.190 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:44:28.190 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.389565+0000", 2024-09-16T09:44:28.190 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.86e-06 2024-09-16T09:44:28.190 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.190 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.190 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:44:28.190 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.389571+0000", 2024-09-16T09:44:28.190 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.2740000000000004e-06 2024-09-16T09:44:28.191 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.191 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.191 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:44:28.191 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.389590+0000", 2024-09-16T09:44:28.191 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.8753000000000002e-05 2024-09-16T09:44:28.191 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.191 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.191 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:44:28.191 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.389599+0000", 2024-09-16T09:44:28.191 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.5380000000000002e-06 2024-09-16T09:44:28.191 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.191 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.192 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:44:28.192 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.389654+0000", 2024-09-16T09:44:28.192 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.5142000000000002e-05 2024-09-16T09:44:28.192 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.192 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.192 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:44:28.192 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.389768+0000", 2024-09-16T09:44:28.192 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00011462400000000001 2024-09-16T09:44:28.192 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.192 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.192 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 1,2", 2024-09-16T09:44:28.192 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.389871+0000", 2024-09-16T09:44:28.192 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000103113 2024-09-16T09:44:28.193 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.193 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.193 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:44:28.193 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.395092+0000", 2024-09-16T09:44:28.193 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.005220298 2024-09-16T09:44:28.193 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.193 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.193 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:44:28.193 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.400656+0000", 2024-09-16T09:44:28.193 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0055647259999999999 2024-09-16T09:44:28.193 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.193 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.193 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:44:28.194 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.400664+0000", 2024-09-16T09:44:28.194 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.6729999999999992e-06 2024-09-16T09:44:28.194 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.194 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.194 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:44:28.194 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:13.513002+0000", 2024-09-16T09:44:28.194 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1123373750000001 2024-09-16T09:44:28.194 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.194 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.194 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:44:28.194 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:13.513007+0000", 2024-09-16T09:44:28.194 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.75e-06 2024-09-16T09:44:28.194 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:44:28.194 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:44:28.195 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:44:28.195 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.195 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.195 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:225235 9.0 9.50884480 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e352)", 2024-09-16T09:44:28.195 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:37:12.434208+0000", 2024-09-16T09:44:28.195 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 435.684962369, 2024-09-16T09:44:28.195 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0788204320000001, 2024-09-16T09:44:28.195 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:44:28.195 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:44:28.195 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "reached pg", 2024-09-16T09:44:28.195 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:44:28.195 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:44:28.195 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:44:28.196 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 225235 2024-09-16T09:44:28.196 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.196 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:44:28.196 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.196 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:44:28.196 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.434208+0000", 2024-09-16T09:44:28.196 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:44:28.196 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.196 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.196 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:44:28.196 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.434208+0000", 2024-09-16T09:44:28.196 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:44:28.196 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.196 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.197 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:44:28.197 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.434209+0000", 2024-09-16T09:44:28.197 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.158e-06 2024-09-16T09:44:28.197 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.197 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.197 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:44:28.197 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.434215+0000", 2024-09-16T09:44:28.197 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.1699999999999996e-06 2024-09-16T09:44:28.197 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.197 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.197 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:44:28.197 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.434228+0000", 2024-09-16T09:44:28.197 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.332e-05 2024-09-16T09:44:28.198 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.198 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.198 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:44:28.198 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.434235+0000", 2024-09-16T09:44:28.198 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.8859999999999998e-06 2024-09-16T09:44:28.198 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.198 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.198 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:44:28.198 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.434292+0000", 2024-09-16T09:44:28.198 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.7704999999999997e-05 2024-09-16T09:44:28.198 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.198 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.198 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:44:28.198 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.434301+0000", 2024-09-16T09:44:28.199 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.2260000000000005e-06 2024-09-16T09:44:28.199 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.199 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.199 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:44:28.199 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:13.513024+0000", 2024-09-16T09:44:28.199 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0787235479999999 2024-09-16T09:44:28.199 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.199 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.199 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:44:28.199 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:13.513029+0000", 2024-09-16T09:44:28.199 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.4190000000000004e-06 2024-09-16T09:44:28.199 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:44:28.199 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:44:28.199 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:44:28.200 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.200 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.200 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:225256 9.10 9.c1fc3710 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e352)", 2024-09-16T09:44:28.200 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:37:12.441649+0000", 2024-09-16T09:44:28.200 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 435.677521802, 2024-09-16T09:44:28.200 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0708169569999999, 2024-09-16T09:44:28.200 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:44:28.200 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:44:28.200 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "reached pg", 2024-09-16T09:44:28.200 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:44:28.200 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:44:28.200 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:44:28.200 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 225256 2024-09-16T09:44:28.201 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.201 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:44:28.201 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.201 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:44:28.201 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.441649+0000", 2024-09-16T09:44:28.201 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:44:28.201 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.201 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.201 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:44:28.201 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.441649+0000", 2024-09-16T09:44:28.201 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:44:28.201 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.202 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.202 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:44:28.202 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.441650+0000", 2024-09-16T09:44:28.202 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.47e-07 2024-09-16T09:44:28.202 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.202 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.202 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:44:28.202 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.441654+0000", 2024-09-16T09:44:28.202 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.6589999999999998e-06 2024-09-16T09:44:28.202 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.202 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.202 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:44:28.202 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.441671+0000", 2024-09-16T09:44:28.202 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.6897000000000001e-05 2024-09-16T09:44:28.203 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.203 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.203 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:44:28.203 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.441681+0000", 2024-09-16T09:44:28.203 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.9129999999999998e-06 2024-09-16T09:44:28.203 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.203 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.203 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:44:28.203 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.441783+0000", 2024-09-16T09:44:28.203 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00010139700000000001 2024-09-16T09:44:28.203 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.203 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.203 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:44:28.204 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.441792+0000", 2024-09-16T09:44:28.204 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.0710000000000001e-06 2024-09-16T09:44:28.204 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.204 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.204 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:44:28.204 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:13.512445+0000", 2024-09-16T09:44:28.204 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.070653525 2024-09-16T09:44:28.204 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.204 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.204 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:44:28.204 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:13.512466+0000", 2024-09-16T09:44:28.204 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.0548e-05 2024-09-16T09:44:28.204 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:44:28.205 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:44:28.205 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:44:28.205 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.205 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.205 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:225291 9.0 9.9752b0a0 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e352)", 2024-09-16T09:44:28.205 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:37:12.458884+0000", 2024-09-16T09:44:28.205 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 435.66028686499999, 2024-09-16T09:44:28.205 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0541780789999999, 2024-09-16T09:44:28.205 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:44:28.205 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:44:28.205 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "reached pg", 2024-09-16T09:44:28.205 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:44:28.205 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:44:28.205 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:44:28.206 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 225291 2024-09-16T09:44:28.206 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.206 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:44:28.206 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.206 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:44:28.206 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.458884+0000", 2024-09-16T09:44:28.206 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:44:28.206 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.206 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.206 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:44:28.206 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.458884+0000", 2024-09-16T09:44:28.206 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:44:28.206 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.207 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.207 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:44:28.207 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.458885+0000", 2024-09-16T09:44:28.207 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.4399999999999998e-07 2024-09-16T09:44:28.207 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.207 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.207 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:44:28.207 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.458889+0000", 2024-09-16T09:44:28.207 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.1400000000000002e-06 2024-09-16T09:44:28.207 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.207 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.207 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:44:28.207 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.458907+0000", 2024-09-16T09:44:28.207 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.8561000000000001e-05 2024-09-16T09:44:28.208 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.208 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.208 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:44:28.208 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.458915+0000", 2024-09-16T09:44:28.208 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.2350000000000002e-06 2024-09-16T09:44:28.208 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.208 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.208 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:44:28.208 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.458986+0000", 2024-09-16T09:44:28.208 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.1273999999999996e-05 2024-09-16T09:44:28.208 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.208 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.208 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:44:28.209 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.458998+0000", 2024-09-16T09:44:28.209 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.2004999999999999e-05 2024-09-16T09:44:28.209 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.209 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.209 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:44:28.209 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:13.513039+0000", 2024-09-16T09:44:28.209 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.054041204 2024-09-16T09:44:28.209 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.209 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.209 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:44:28.209 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:13.513062+0000", 2024-09-16T09:44:28.209 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.2716000000000001e-05 2024-09-16T09:44:28.209 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:44:28.209 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:44:28.210 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:44:28.210 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.210 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.210 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:225292 9.0 9.fcff160 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e352)", 2024-09-16T09:44:28.210 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:37:12.458943+0000", 2024-09-16T09:44:28.210 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 435.66022747, 2024-09-16T09:44:28.210 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0541260480000001, 2024-09-16T09:44:28.210 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:44:28.210 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:44:28.210 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "reached pg", 2024-09-16T09:44:28.210 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:44:28.210 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:44:28.210 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:44:28.211 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 225292 2024-09-16T09:44:28.211 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.211 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:44:28.211 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.211 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:44:28.211 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.458943+0000", 2024-09-16T09:44:28.211 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:44:28.211 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.211 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.211 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:44:28.211 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.458943+0000", 2024-09-16T09:44:28.212 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:44:28.212 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.213 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.213 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:44:28.213 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.458944+0000", 2024-09-16T09:44:28.213 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.7899999999999999e-07 2024-09-16T09:44:28.213 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.213 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.213 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:44:28.213 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.458946+0000", 2024-09-16T09:44:28.213 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.4619999999999999e-06 2024-09-16T09:44:28.213 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.213 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.213 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:44:28.213 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.458953+0000", 2024-09-16T09:44:28.214 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.8390000000000001e-06 2024-09-16T09:44:28.214 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.214 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.214 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:44:28.214 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.458958+0000", 2024-09-16T09:44:28.214 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.0010000000000001e-06 2024-09-16T09:44:28.214 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.214 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.214 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:44:28.214 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.459049+0000", 2024-09-16T09:44:28.214 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.1378000000000005e-05 2024-09-16T09:44:28.214 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.214 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.215 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:44:28.215 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.459059+0000", 2024-09-16T09:44:28.215 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 9.5519999999999993e-06 2024-09-16T09:44:28.215 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.215 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.215 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:44:28.215 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:13.513061+0000", 2024-09-16T09:44:28.215 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.054001653 2024-09-16T09:44:28.215 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.215 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.215 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:44:28.215 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:13.513069+0000", 2024-09-16T09:44:28.215 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.6840000000000002e-06 2024-09-16T09:44:28.216 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:44:28.216 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:44:28.216 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:44:28.216 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.216 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.216 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:225293 9.10 9.ab745fd0 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e352)", 2024-09-16T09:44:28.216 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:37:12.459755+0000", 2024-09-16T09:44:28.216 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 435.65941578899998, 2024-09-16T09:44:28.216 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0531002519999999, 2024-09-16T09:44:28.216 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:44:28.216 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:44:28.216 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "reached pg", 2024-09-16T09:44:28.216 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:44:28.217 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:44:28.217 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:44:28.217 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 225293 2024-09-16T09:44:28.217 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.217 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:44:28.217 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.217 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:44:28.217 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.459755+0000", 2024-09-16T09:44:28.217 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:44:28.217 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.217 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.217 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:44:28.217 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.459755+0000", 2024-09-16T09:44:28.218 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:44:28.218 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.218 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.218 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:44:28.218 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.459756+0000", 2024-09-16T09:44:28.218 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.9699999999999995e-07 2024-09-16T09:44:28.218 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.218 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.218 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:44:28.218 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.459759+0000", 2024-09-16T09:44:28.218 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.782e-06 2024-09-16T09:44:28.218 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.218 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.219 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:44:28.219 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.459772+0000", 2024-09-16T09:44:28.219 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.253e-05 2024-09-16T09:44:28.219 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.219 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.219 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:44:28.219 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.459779+0000", 2024-09-16T09:44:28.219 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.6980000000000001e-06 2024-09-16T09:44:28.219 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.219 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.219 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:44:28.219 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.459823+0000", 2024-09-16T09:44:28.219 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.3968999999999999e-05 2024-09-16T09:44:28.219 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.220 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.220 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:44:28.220 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:12.459829+0000", 2024-09-16T09:44:28.220 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.9290000000000003e-06 2024-09-16T09:44:28.220 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.220 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.220 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:44:28.220 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:13.512847+0000", 2024-09-16T09:44:28.220 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.053018314 2024-09-16T09:44:28.220 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.220 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.220 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:44:28.221 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:13.512855+0000", 2024-09-16T09:44:28.221 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.2330000000000006e-06 2024-09-16T09:44:28.221 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:44:28.221 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:44:28.221 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:44:28.221 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.221 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.221 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:231327 9.10 9:0da6d7f3:::benchmark_data_smithi070_137000_object109616:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e357)", 2024-09-16T09:44:28.221 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:37:18.512211+0000", 2024-09-16T09:44:28.221 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 429.60695945999998, 2024-09-16T09:44:28.221 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.104184364, 2024-09-16T09:44:28.221 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:44:28.221 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:44:28.222 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "reached pg", 2024-09-16T09:44:28.222 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:44:28.222 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:44:28.222 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:44:28.222 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 231327 2024-09-16T09:44:28.222 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.222 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:44:28.222 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.222 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:44:28.222 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.512211+0000", 2024-09-16T09:44:28.222 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:44:28.222 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.222 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.222 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:44:28.223 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.512211+0000", 2024-09-16T09:44:28.223 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:44:28.223 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.223 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.223 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:44:28.223 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.512213+0000", 2024-09-16T09:44:28.223 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.8589999999999999e-06 2024-09-16T09:44:28.223 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.223 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.223 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:44:28.223 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.512216+0000", 2024-09-16T09:44:28.223 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.174e-06 2024-09-16T09:44:28.223 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.224 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.224 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:44:28.224 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.512234+0000", 2024-09-16T09:44:28.224 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.7380999999999998e-05 2024-09-16T09:44:28.224 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.224 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.224 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:44:28.224 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.512242+0000", 2024-09-16T09:44:28.224 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.3680000000000002e-06 2024-09-16T09:44:28.224 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.224 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.224 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:44:28.224 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.684712+0000", 2024-09-16T09:44:28.224 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.17247021300000001 2024-09-16T09:44:28.225 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.225 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.225 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:44:28.225 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.684720+0000", 2024-09-16T09:44:28.225 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.2989999999999997e-06 2024-09-16T09:44:28.225 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.225 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.225 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:44:28.225 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.612846+0000", 2024-09-16T09:44:28.225 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.92812537399999995 2024-09-16T09:44:28.225 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.225 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.225 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for missing object", 2024-09-16T09:44:28.225 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.612970+0000", 2024-09-16T09:44:28.226 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00012453200000000001 2024-09-16T09:44:28.226 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.226 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.226 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:44:28.226 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.616209+0000", 2024-09-16T09:44:28.226 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.003238922 2024-09-16T09:44:28.226 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.226 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.226 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:44:28.226 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.616396+0000", 2024-09-16T09:44:28.226 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00018624200000000001 2024-09-16T09:44:28.226 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:44:28.226 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:44:28.227 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:44:28.227 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.227 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.227 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:231328 9.1 9:8132d7b0:::benchmark_data_smithi070_137000_object109617:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e357)", 2024-09-16T09:44:28.227 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:37:18.512255+0000", 2024-09-16T09:44:28.227 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 429.60691574700002, 2024-09-16T09:44:28.227 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0795497709999999, 2024-09-16T09:44:28.227 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:44:28.227 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:44:28.227 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "reached pg", 2024-09-16T09:44:28.227 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:44:28.227 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:44:28.227 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:44:28.227 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 231328 2024-09-16T09:44:28.228 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.228 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:44:28.228 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.228 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:44:28.228 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.512255+0000", 2024-09-16T09:44:28.228 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:44:28.228 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.228 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.228 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:44:28.228 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.512255+0000", 2024-09-16T09:44:28.228 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:44:28.228 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.228 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.229 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:44:28.229 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.512256+0000", 2024-09-16T09:44:28.229 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.034e-06 2024-09-16T09:44:28.229 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.229 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.229 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:44:28.229 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.512260+0000", 2024-09-16T09:44:28.229 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.9129999999999996e-06 2024-09-16T09:44:28.229 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.229 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.229 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:44:28.229 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.512268+0000", 2024-09-16T09:44:28.229 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.7530000000000001e-06 2024-09-16T09:44:28.230 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.230 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.230 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:44:28.230 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.512273+0000", 2024-09-16T09:44:28.230 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.5149999999999997e-06 2024-09-16T09:44:28.230 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.230 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.230 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:44:28.230 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.684946+0000", 2024-09-16T09:44:28.230 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.17267326299999999 2024-09-16T09:44:28.230 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.230 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.230 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:44:28.230 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.684954+0000", 2024-09-16T09:44:28.231 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.4769999999999997e-06 2024-09-16T09:44:28.231 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.231 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.231 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:44:28.231 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.538886+0000", 2024-09-16T09:44:28.231 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.85393251999999997 2024-09-16T09:44:28.231 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.231 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.231 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for missing object", 2024-09-16T09:44:28.231 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.539061+0000", 2024-09-16T09:44:28.231 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000174694 2024-09-16T09:44:28.231 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.232 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.232 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:44:28.232 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.591746+0000", 2024-09-16T09:44:28.232 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.052684624999999999 2024-09-16T09:44:28.232 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.232 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.232 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:44:28.232 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.591805+0000", 2024-09-16T09:44:28.232 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.8977000000000001e-05 2024-09-16T09:44:28.232 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:44:28.232 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:44:28.232 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:44:28.232 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.233 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.233 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:231329 9.11 9:8a8e13aa:::benchmark_data_smithi070_137000_object109618:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e357)", 2024-09-16T09:44:28.233 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:37:18.512281+0000", 2024-09-16T09:44:28.233 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 429.60688989699997, 2024-09-16T09:44:28.233 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1196006590000001, 2024-09-16T09:44:28.233 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:44:28.233 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:44:28.233 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "reached pg", 2024-09-16T09:44:28.233 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:44:28.233 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:44:28.233 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:44:28.233 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 231329 2024-09-16T09:44:28.233 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.233 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:44:28.234 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.234 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:44:28.234 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.512281+0000", 2024-09-16T09:44:28.234 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:44:28.234 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.234 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.234 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:44:28.234 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.512281+0000", 2024-09-16T09:44:28.234 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:44:28.234 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.234 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.234 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:44:28.234 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.512281+0000", 2024-09-16T09:44:28.234 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 3.3700000000000001e-07 2024-09-16T09:44:28.235 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.235 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.235 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:44:28.235 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.512283+0000", 2024-09-16T09:44:28.235 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.39e-06 2024-09-16T09:44:28.235 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.235 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.235 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:44:28.235 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.512291+0000", 2024-09-16T09:44:28.235 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 7.17e-06 2024-09-16T09:44:28.235 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.235 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.235 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:44:28.235 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.512295+0000", 2024-09-16T09:44:28.236 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.2370000000000003e-06 2024-09-16T09:44:28.236 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.236 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.236 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:44:28.236 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.684979+0000", 2024-09-16T09:44:28.236 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.17268373300000001 2024-09-16T09:44:28.236 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.236 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.236 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:44:28.236 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.684985+0000", 2024-09-16T09:44:28.236 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.4239999999999998e-06 2024-09-16T09:44:28.236 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.236 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.237 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:44:28.237 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.627360+0000", 2024-09-16T09:44:28.237 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.94237468999999996 2024-09-16T09:44:28.237 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.237 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.237 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for missing object", 2024-09-16T09:44:28.237 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.627557+0000", 2024-09-16T09:44:28.237 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00019702200000000001 2024-09-16T09:44:28.237 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.237 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.237 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:44:28.237 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.631826+0000", 2024-09-16T09:44:28.237 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0042689950000000003 2024-09-16T09:44:28.237 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.238 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.238 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:44:28.238 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.631881+0000", 2024-09-16T09:44:28.238 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 5.5661000000000001e-05 2024-09-16T09:44:28.238 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:44:28.238 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:44:28.238 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:44:28.238 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.238 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.238 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:231331 9.10 9:0df6dab9:::benchmark_data_smithi070_137000_object109620:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e357)", 2024-09-16T09:44:28.238 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:37:18.539893+0000", 2024-09-16T09:44:28.238 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 429.57927712999998, 2024-09-16T09:44:28.238 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0771409569999999, 2024-09-16T09:44:28.238 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:44:28.239 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:44:28.239 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:44:28.239 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:44:28.239 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:44:28.239 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:44:28.239 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 231331 2024-09-16T09:44:28.239 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.239 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:44:28.239 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.239 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:44:28.239 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.539893+0000", 2024-09-16T09:44:28.239 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:44:28.239 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.239 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.240 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:44:28.240 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.539893+0000", 2024-09-16T09:44:28.240 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:44:28.240 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.240 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.240 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:44:28.240 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.539896+0000", 2024-09-16T09:44:28.240 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.4310000000000001e-06 2024-09-16T09:44:28.240 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.240 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.240 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:44:28.240 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.542675+0000", 2024-09-16T09:44:28.240 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0027787900000000002 2024-09-16T09:44:28.241 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.241 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.241 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:44:28.241 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.542701+0000", 2024-09-16T09:44:28.241 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.6122000000000002e-05 2024-09-16T09:44:28.241 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.241 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.241 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:44:28.241 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.542712+0000", 2024-09-16T09:44:28.241 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.1443e-05 2024-09-16T09:44:28.241 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.241 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.241 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:44:28.241 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.684747+0000", 2024-09-16T09:44:28.242 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.14203510999999999 2024-09-16T09:44:28.242 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.242 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.242 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:44:28.242 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.684754+0000", 2024-09-16T09:44:28.242 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.7320000000000001e-06 2024-09-16T09:44:28.242 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.242 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.242 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:44:28.242 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.613242+0000", 2024-09-16T09:44:28.242 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.92848750300000005 2024-09-16T09:44:28.242 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.242 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.243 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:44:28.243 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.613347+0000", 2024-09-16T09:44:28.243 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000105361 2024-09-16T09:44:28.243 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.243 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.243 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 1,2", 2024-09-16T09:44:28.243 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.613462+0000", 2024-09-16T09:44:28.243 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00011495 2024-09-16T09:44:28.243 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.243 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.243 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:44:28.243 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.615859+0000", 2024-09-16T09:44:28.243 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0023971729999999998 2024-09-16T09:44:28.244 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.244 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.244 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:44:28.244 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.616173+0000", 2024-09-16T09:44:28.244 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00031437799999999998 2024-09-16T09:44:28.244 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.244 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.244 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:44:28.244 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.616845+0000", 2024-09-16T09:44:28.244 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00067190000000000001 2024-09-16T09:44:28.244 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.244 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.244 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:44:28.244 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.616931+0000", 2024-09-16T09:44:28.245 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.5439000000000006e-05 2024-09-16T09:44:28.245 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.245 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.245 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:44:28.245 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.617034+0000", 2024-09-16T09:44:28.245 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.000103625 2024-09-16T09:44:28.245 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:44:28.245 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:44:28.245 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:44:28.245 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.245 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.245 INFO:teuthology.orchestra.run.smithi084.stdout: "description": "osd_op(client.15588.0:231332 9.1 9:83c10802:::benchmark_data_smithi070_137000_object109621:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e357)", 2024-09-16T09:44:28.246 INFO:teuthology.orchestra.run.smithi084.stdout: "initiated_at": "2024-09-16T09:37:18.542736+0000", 2024-09-16T09:44:28.246 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 429.57643442599999, 2024-09-16T09:44:28.246 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.0489751380000001, 2024-09-16T09:44:28.246 INFO:teuthology.orchestra.run.smithi084.stdout: "continuous": false, 2024-09-16T09:44:28.246 INFO:teuthology.orchestra.run.smithi084.stdout: "type_data": { 2024-09-16T09:44:28.246 INFO:teuthology.orchestra.run.smithi084.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-16T09:44:28.246 INFO:teuthology.orchestra.run.smithi084.stdout: "client_info": { 2024-09-16T09:44:28.246 INFO:teuthology.orchestra.run.smithi084.stdout: "client": "client.15588", 2024-09-16T09:44:28.246 INFO:teuthology.orchestra.run.smithi084.stdout: "client_addr": "172.21.15.70:0/1751296835", 2024-09-16T09:44:28.246 INFO:teuthology.orchestra.run.smithi084.stdout: "tid": 231332 2024-09-16T09:44:28.246 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.246 INFO:teuthology.orchestra.run.smithi084.stdout: "events": [ 2024-09-16T09:44:28.246 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.246 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "initiated", 2024-09-16T09:44:28.247 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.542736+0000", 2024-09-16T09:44:28.247 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:44:28.247 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.247 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.247 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "header_read", 2024-09-16T09:44:28.247 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.542736+0000", 2024-09-16T09:44:28.247 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0 2024-09-16T09:44:28.247 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.247 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.247 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "throttled", 2024-09-16T09:44:28.247 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.542738+0000", 2024-09-16T09:44:28.247 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 1.3480000000000001e-06 2024-09-16T09:44:28.247 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.247 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.248 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "all_read", 2024-09-16T09:44:28.248 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.542740+0000", 2024-09-16T09:44:28.248 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 2.261e-06 2024-09-16T09:44:28.248 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.248 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.248 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "dispatched", 2024-09-16T09:44:28.248 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.542746+0000", 2024-09-16T09:44:28.248 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 6.2670000000000003e-06 2024-09-16T09:44:28.248 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.248 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.248 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "queued_for_pg", 2024-09-16T09:44:28.248 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.542751+0000", 2024-09-16T09:44:28.249 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.8010000000000003e-06 2024-09-16T09:44:28.249 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.249 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.249 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:44:28.249 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.685007+0000", 2024-09-16T09:44:28.249 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.14225658599999999 2024-09-16T09:44:28.249 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.249 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.249 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for peered", 2024-09-16T09:44:28.249 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:18.685016+0000", 2024-09-16T09:44:28.249 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.5199999999999997e-06 2024-09-16T09:44:28.249 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.249 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.249 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "reached_pg", 2024-09-16T09:44:28.250 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.542235+0000", 2024-09-16T09:44:28.250 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.85721875300000006 2024-09-16T09:44:28.250 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.250 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.250 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "started", 2024-09-16T09:44:28.250 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.542315+0000", 2024-09-16T09:44:28.250 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.0000999999999995e-05 2024-09-16T09:44:28.250 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.250 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.250 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "waiting for subops from 3,7", 2024-09-16T09:44:28.250 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.542396+0000", 2024-09-16T09:44:28.250 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.1600000000000005e-05 2024-09-16T09:44:28.251 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.251 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.251 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:44:28.251 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.543116+0000", 2024-09-16T09:44:28.251 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.00071917400000000005 2024-09-16T09:44:28.251 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.251 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.251 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "sub_op_commit_rec", 2024-09-16T09:44:28.251 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.546040+0000", 2024-09-16T09:44:28.251 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.0029248270000000001 2024-09-16T09:44:28.251 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.251 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.251 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "op_commit", 2024-09-16T09:44:28.251 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.591579+0000", 2024-09-16T09:44:28.252 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 0.045538617000000003 2024-09-16T09:44:28.252 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.252 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.252 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "commit_sent", 2024-09-16T09:44:28.252 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.591626+0000", 2024-09-16T09:44:28.252 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 4.7274000000000002e-05 2024-09-16T09:44:28.252 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-09-16T09:44:28.252 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-09-16T09:44:28.252 INFO:teuthology.orchestra.run.smithi084.stdout: "event": "done", 2024-09-16T09:44:28.252 INFO:teuthology.orchestra.run.smithi084.stdout: "time": "2024-09-16T09:37:19.591711+0000", 2024-09-16T09:44:28.252 INFO:teuthology.orchestra.run.smithi084.stdout: "duration": 8.5109000000000001e-05 2024-09-16T09:44:28.252 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:44:28.252 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:44:28.253 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:44:28.253 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-09-16T09:44:28.253 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-09-16T09:44:28.253 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-09-16T09:44:28.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:28.254 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:28.255 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:44:28.355 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:44:28.376 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:28.377 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:28.377 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:44:28.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:28 smithi084 ceph-mon[32629]: 3.1e scrub starts 2024-09-16T09:44:28.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:28 smithi084 ceph-mon[32629]: 3.1e scrub ok 2024-09-16T09:44:28.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:28 smithi084 ceph-mon[32629]: osdmap e520: 8 total, 8 up, 8 in 2024-09-16T09:44:28.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:28 smithi084 ceph-mon[32629]: 1.5 scrub starts 2024-09-16T09:44:28.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:28 smithi084 ceph-mon[32629]: 1.5 scrub ok 2024-09-16T09:44:28.478 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:44:28.499 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:28.499 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:28.500 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:44:28.601 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:44:28.623 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:28.624 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:28.624 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:44:28.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:28 smithi070 ceph-mon[30388]: 3.1e scrub starts 2024-09-16T09:44:28.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:28 smithi070 ceph-mon[30388]: 3.1e scrub ok 2024-09-16T09:44:28.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:28 smithi070 ceph-mon[30388]: osdmap e520: 8 total, 8 up, 8 in 2024-09-16T09:44:28.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:28 smithi070 ceph-mon[30388]: 1.5 scrub starts 2024-09-16T09:44:28.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:28 smithi070 ceph-mon[30388]: 1.5 scrub ok 2024-09-16T09:44:28.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:28 smithi070 ceph-mon[40691]: 3.1e scrub starts 2024-09-16T09:44:28.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:28 smithi070 ceph-mon[40691]: 3.1e scrub ok 2024-09-16T09:44:28.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:28 smithi070 ceph-mon[40691]: osdmap e520: 8 total, 8 up, 8 in 2024-09-16T09:44:28.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:28 smithi070 ceph-mon[40691]: 1.5 scrub starts 2024-09-16T09:44:28.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:28 smithi070 ceph-mon[40691]: 1.5 scrub ok 2024-09-16T09:44:28.725 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:44:28.747 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:28.747 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:28.747 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:44:28.848 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:44:28.870 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:28.871 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:28.872 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:44:28.972 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:44:28.993 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:28.994 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:28.995 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:44:29.088 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.7 dump_ops_in_flight 2024-09-16T09:44:29.095 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:44:29.101 INFO:tasks.cephadm.ceph_manager.ceph:removing pool_name unique_pool_4 2024-09-16T09:44:29.101 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd pool rm unique_pool_4 unique_pool_4 --yes-i-really-really-mean-it 2024-09-16T09:44:29.152 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:29.152 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:29.153 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:44:29.175 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:44:29.213 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:44:29.253 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:44:29.275 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:29.276 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:29.276 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:44:29.328 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:29 smithi070 ceph-mon[40691]: 3.4 scrub starts 2024-09-16T09:44:29.328 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:29 smithi070 ceph-mon[40691]: 3.4 scrub ok 2024-09-16T09:44:29.329 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:29 smithi070 ceph-mon[40691]: pgmap v1018: 168 pgs: 1 active+undersized+remapped+backfill_toofull, 1 remapped+peering, 25 peering, 141 active+clean; 583 KiB data, 7.1 GiB used, 708 GiB / 715 GiB avail; 0 B/s rd, 0 B/s wr, 197 op/s 2024-09-16T09:44:29.329 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:29 smithi070 ceph-mon[40691]: Health check failed: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull (PG_BACKFILL_FULL) 2024-09-16T09:44:29.329 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:29 smithi070 ceph-mon[40691]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 484/2238 objects degraded (21.626%), 8 pgs degraded, 18 pgs undersized) 2024-09-16T09:44:29.329 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:29 smithi070 ceph-mon[40691]: osdmap e521: 8 total, 8 up, 8 in 2024-09-16T09:44:29.329 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:29 smithi070 ceph-mon[40691]: 3.1a scrub starts 2024-09-16T09:44:29.329 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:29 smithi070 ceph-mon[40691]: 3.1a scrub ok 2024-09-16T09:44:29.329 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:29 smithi070 ceph-mon[30388]: 3.4 scrub starts 2024-09-16T09:44:29.329 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:29 smithi070 ceph-mon[30388]: 3.4 scrub ok 2024-09-16T09:44:29.329 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:29 smithi070 ceph-mon[30388]: pgmap v1018: 168 pgs: 1 active+undersized+remapped+backfill_toofull, 1 remapped+peering, 25 peering, 141 active+clean; 583 KiB data, 7.1 GiB used, 708 GiB / 715 GiB avail; 0 B/s rd, 0 B/s wr, 197 op/s 2024-09-16T09:44:29.329 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:29 smithi070 ceph-mon[30388]: Health check failed: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull (PG_BACKFILL_FULL) 2024-09-16T09:44:29.329 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:29 smithi070 ceph-mon[30388]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 484/2238 objects degraded (21.626%), 8 pgs degraded, 18 pgs undersized) 2024-09-16T09:44:29.330 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:29 smithi070 ceph-mon[30388]: osdmap e521: 8 total, 8 up, 8 in 2024-09-16T09:44:29.330 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:29 smithi070 ceph-mon[30388]: 3.1a scrub starts 2024-09-16T09:44:29.330 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:29 smithi070 ceph-mon[30388]: 3.1a scrub ok 2024-09-16T09:44:29.337 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:44:29.377 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:44:29.398 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:29.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:29.399 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:44:29.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:29 smithi084 ceph-mon[32629]: 3.4 scrub starts 2024-09-16T09:44:29.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:29 smithi084 ceph-mon[32629]: 3.4 scrub ok 2024-09-16T09:44:29.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:29 smithi084 ceph-mon[32629]: pgmap v1018: 168 pgs: 1 active+undersized+remapped+backfill_toofull, 1 remapped+peering, 25 peering, 141 active+clean; 583 KiB data, 7.1 GiB used, 708 GiB / 715 GiB avail; 0 B/s rd, 0 B/s wr, 197 op/s 2024-09-16T09:44:29.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:29 smithi084 ceph-mon[32629]: Health check failed: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull (PG_BACKFILL_FULL) 2024-09-16T09:44:29.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:29 smithi084 ceph-mon[32629]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 484/2238 objects degraded (21.626%), 8 pgs degraded, 18 pgs undersized) 2024-09-16T09:44:29.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:29 smithi084 ceph-mon[32629]: osdmap e521: 8 total, 8 up, 8 in 2024-09-16T09:44:29.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:29 smithi084 ceph-mon[32629]: 3.1a scrub starts 2024-09-16T09:44:29.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:29 smithi084 ceph-mon[32629]: 3.1a scrub ok 2024-09-16T09:44:29.500 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:44:29.522 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:29.523 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:29.523 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:44:29.624 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:44:29.646 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:29.646 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:29.647 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:44:29.747 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:44:29.770 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:29.771 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:29.771 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:44:29.872 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:44:29.889 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-09-16T09:44:29.889 INFO:teuthology.orchestra.run.smithi084.stdout: "ops": [], 2024-09-16T09:44:29.889 INFO:teuthology.orchestra.run.smithi084.stdout: "num_ops": 0 2024-09-16T09:44:29.889 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-09-16T09:44:29.893 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:29.893 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:29.894 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:44:29.981 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T09:44:29.981 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T09:44:29.981 INFO:teuthology.orchestra.run.smithi070.stdout: "num_ops": 0 2024-09-16T09:44:29.981 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T09:44:29.995 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:44:30.017 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:30.017 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:30.018 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:44:30.118 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:44:30.141 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:30.142 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:30.142 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:44:30.243 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:44:30.265 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:30.266 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:30.266 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:44:30.367 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:44:30.389 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:30.390 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:30.390 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:44:30.491 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:44:30.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:30.513 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:30.513 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:44:30.526 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:30 smithi084 ceph-mon[32629]: 4.12 scrub starts 2024-09-16T09:44:30.527 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:30 smithi084 ceph-mon[32629]: 4.12 scrub ok 2024-09-16T09:44:30.527 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:30 smithi084 ceph-mon[32629]: osdmap e522: 8 total, 8 up, 8 in 2024-09-16T09:44:30.527 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:30 smithi084 ceph-mon[32629]: 6.2 scrub starts 2024-09-16T09:44:30.527 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:30 smithi084 ceph-mon[32629]: 6.2 scrub ok 2024-09-16T09:44:30.527 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:30 smithi084 ceph-mon[32629]: 6.3 scrub starts 2024-09-16T09:44:30.527 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:30 smithi084 ceph-mon[32629]: 6.3 scrub ok 2024-09-16T09:44:30.537 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:44:30.537 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:44:30.554 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.7 dump_blocked_ops 2024-09-16T09:44:30.556 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:44:30.556 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:44:30.556 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:30 smithi070 ceph-mon[30388]: 4.12 scrub starts 2024-09-16T09:44:30.556 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:30 smithi070 ceph-mon[30388]: 4.12 scrub ok 2024-09-16T09:44:30.556 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:30 smithi070 ceph-mon[30388]: osdmap e522: 8 total, 8 up, 8 in 2024-09-16T09:44:30.557 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:30 smithi070 ceph-mon[30388]: 6.2 scrub starts 2024-09-16T09:44:30.557 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:30 smithi070 ceph-mon[30388]: 6.2 scrub ok 2024-09-16T09:44:30.557 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:30 smithi070 ceph-mon[30388]: 6.3 scrub starts 2024-09-16T09:44:30.557 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:30 smithi070 ceph-mon[30388]: 6.3 scrub ok 2024-09-16T09:44:30.557 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:30 smithi070 ceph-mon[40691]: 4.12 scrub starts 2024-09-16T09:44:30.557 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:30 smithi070 ceph-mon[40691]: 4.12 scrub ok 2024-09-16T09:44:30.557 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:30 smithi070 ceph-mon[40691]: osdmap e522: 8 total, 8 up, 8 in 2024-09-16T09:44:30.557 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:30 smithi070 ceph-mon[40691]: 6.2 scrub starts 2024-09-16T09:44:30.557 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:30 smithi070 ceph-mon[40691]: 6.2 scrub ok 2024-09-16T09:44:30.558 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:30 smithi070 ceph-mon[40691]: 6.3 scrub starts 2024-09-16T09:44:30.558 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:30 smithi070 ceph-mon[40691]: 6.3 scrub ok 2024-09-16T09:44:30.572 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:44:30.573 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:44:30.589 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:44:30.589 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:44:30.604 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:44:30.605 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:44:30.614 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:44:30.621 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:44:30.621 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:44:30.638 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:44:30.638 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:44:30.654 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:44:30.655 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:44:30.659 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:30.659 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:30.659 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:44:30.760 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:44:30.782 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:30.783 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:30.783 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:44:30.854 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:44:30.884 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:44:30.906 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:30.906 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:30.907 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:44:30.971 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd reweight 0 1 2024-09-16T09:44:31.007 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:44:31.036 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:31.037 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:31.037 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:44:31.096 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd set noscrub 2024-09-16T09:44:31.138 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:44:31.159 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:31.160 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:31.160 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:44:31.210 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:44:31.261 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:44:31.282 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:31.283 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:31.284 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:44:31.357 INFO:teuthology.orchestra.run.smithi084.stdout:ERROR: (22) Invalid argument 2024-09-16T09:44:31.357 INFO:teuthology.orchestra.run.smithi084.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:44:31.384 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:44:31.405 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:31.406 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:31.406 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:44:31.507 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5 2024-09-16T09:44:31.528 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:31.529 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:31.529 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.5' 2024-09-16T09:44:31.630 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:44:31.633 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:31 smithi070 ceph-mon[30388]: 1.1 scrub starts 2024-09-16T09:44:31.633 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:31 smithi070 ceph-mon[30388]: 1.1 scrub ok 2024-09-16T09:44:31.633 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:31 smithi070 ceph-mon[30388]: pgmap v1021: 168 pgs: 2 active+undersized+remapped+backfill_toofull, 4 peering, 162 active+clean; 583 KiB data, 7.1 GiB used, 708 GiB / 715 GiB avail; 0 B/s rd, 0 B/s wr, 136 op/s 2024-09-16T09:44:31.633 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:31 smithi070 ceph-mon[30388]: osdmap e523: 8 total, 8 up, 8 in 2024-09-16T09:44:31.633 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:31 smithi070 ceph-mon[30388]: 4.4 scrub starts 2024-09-16T09:44:31.633 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:31 smithi070 ceph-mon[30388]: 4.4 scrub ok 2024-09-16T09:44:31.634 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:31 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2259572411' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:44:31.634 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:31 smithi070 ceph-mon[40691]: 1.1 scrub starts 2024-09-16T09:44:31.634 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:31 smithi070 ceph-mon[40691]: 1.1 scrub ok 2024-09-16T09:44:31.634 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:31 smithi070 ceph-mon[40691]: pgmap v1021: 168 pgs: 2 active+undersized+remapped+backfill_toofull, 4 peering, 162 active+clean; 583 KiB data, 7.1 GiB used, 708 GiB / 715 GiB avail; 0 B/s rd, 0 B/s wr, 136 op/s 2024-09-16T09:44:31.634 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:31 smithi070 ceph-mon[40691]: osdmap e523: 8 total, 8 up, 8 in 2024-09-16T09:44:31.634 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:31 smithi070 ceph-mon[40691]: 4.4 scrub starts 2024-09-16T09:44:31.634 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:31 smithi070 ceph-mon[40691]: 4.4 scrub ok 2024-09-16T09:44:31.634 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:31 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/2259572411' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:44:31.653 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:31.654 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:31.654 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:44:31.657 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:31 smithi084 ceph-mon[32629]: 1.1 scrub starts 2024-09-16T09:44:31.657 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:31 smithi084 ceph-mon[32629]: 1.1 scrub ok 2024-09-16T09:44:31.657 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:31 smithi084 ceph-mon[32629]: pgmap v1021: 168 pgs: 2 active+undersized+remapped+backfill_toofull, 4 peering, 162 active+clean; 583 KiB data, 7.1 GiB used, 708 GiB / 715 GiB avail; 0 B/s rd, 0 B/s wr, 136 op/s 2024-09-16T09:44:31.657 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:31 smithi084 ceph-mon[32629]: osdmap e523: 8 total, 8 up, 8 in 2024-09-16T09:44:31.657 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:31 smithi084 ceph-mon[32629]: 4.4 scrub starts 2024-09-16T09:44:31.657 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:31 smithi084 ceph-mon[32629]: 4.4 scrub ok 2024-09-16T09:44:31.658 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:31 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/2259572411' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:44:31.755 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:44:31.777 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:31.778 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:31.778 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:44:31.879 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:44:31.903 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:31.903 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:31.903 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:44:32.005 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:44:32.027 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:32.027 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:32.027 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:44:32.128 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:44:32.145 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:44:31 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:44:31] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:44:32.150 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:32.150 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:32.151 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:44:32.202 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.7 dump_historic_ops 2024-09-16T09:44:32.246 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:44:32.252 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:44:32.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:32.274 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:32.274 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:44:32.295 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:44:32.375 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:44:32.397 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:32.397 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:32.397 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:44:32.412 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:32 smithi070 ceph-mon[30388]: 3.0 scrub starts 2024-09-16T09:44:32.412 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:32 smithi070 ceph-mon[30388]: 3.0 scrub ok 2024-09-16T09:44:32.413 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:32 smithi070 ceph-mon[30388]: 1.a scrub starts 2024-09-16T09:44:32.413 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:32 smithi070 ceph-mon[30388]: 1.a scrub ok 2024-09-16T09:44:32.413 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:32 smithi070 ceph-mon[40691]: 3.0 scrub starts 2024-09-16T09:44:32.413 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:32 smithi070 ceph-mon[40691]: 3.0 scrub ok 2024-09-16T09:44:32.413 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:32 smithi070 ceph-mon[40691]: 1.a scrub starts 2024-09-16T09:44:32.413 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:32 smithi070 ceph-mon[40691]: 1.a scrub ok 2024-09-16T09:44:32.453 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T09:44:32.499 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:44:32.521 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:32.521 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:32.522 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:44:32.574 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:32 smithi084 ceph-mon[32629]: 3.0 scrub starts 2024-09-16T09:44:32.574 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:32 smithi084 ceph-mon[32629]: 3.0 scrub ok 2024-09-16T09:44:32.574 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:32 smithi084 ceph-mon[32629]: 1.a scrub starts 2024-09-16T09:44:32.574 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:32 smithi084 ceph-mon[32629]: 1.a scrub ok 2024-09-16T09:44:32.623 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:44:32.645 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:32.646 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:32.646 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:44:32.747 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:44:32.768 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:32.769 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:32.769 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:44:32.870 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:44:32.892 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:32.893 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:32.893 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:44:32.994 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:44:33.016 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:33.017 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:33.017 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:44:33.118 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:44:33.141 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:33.142 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:33.142 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:44:33.243 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:44:33.267 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:33.267 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:33.268 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:44:33.317 INFO:teuthology.orchestra.run.smithi070.stderr:pool 'unique_pool_4' removed 2024-09-16T09:44:33.369 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:44:33.391 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:33.392 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:33.392 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:44:33.493 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:44:33.499 INFO:teuthology.orchestra.run.smithi070.stderr:reweighted osd.0 to 1 (10000) 2024-09-16T09:44:33.530 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:33.530 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:33.531 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:44:33.631 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:44:33.655 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:33.655 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:33.656 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:44:33.659 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:33 smithi070 ceph-mon[30388]: 1.8 scrub starts 2024-09-16T09:44:33.659 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:33 smithi070 ceph-mon[30388]: 1.8 scrub ok 2024-09-16T09:44:33.659 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:33 smithi070 ceph-mon[30388]: pgmap v1023: 168 pgs: 2 active+undersized+remapped+backfill_toofull, 4 peering, 162 active+clean; 583 KiB data, 7.1 GiB used, 708 GiB / 715 GiB avail; 0 B/s rd, 0 B/s wr, 117 op/s 2024-09-16T09:44:33.660 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:33 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3239028710' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "unique_pool_4", "pool2": "unique_pool_4", "yes_i_really_really_mean_it": true} : dispatch 2024-09-16T09:44:33.660 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:33 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3796101465' entity='client.admin' cmd={"prefix": "osd reweight", "id": 0, "weight": 1.0} : dispatch 2024-09-16T09:44:33.660 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:33 smithi070 ceph-mon[40691]: 1.8 scrub starts 2024-09-16T09:44:33.660 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:33 smithi070 ceph-mon[40691]: 1.8 scrub ok 2024-09-16T09:44:33.660 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:33 smithi070 ceph-mon[40691]: pgmap v1023: 168 pgs: 2 active+undersized+remapped+backfill_toofull, 4 peering, 162 active+clean; 583 KiB data, 7.1 GiB used, 708 GiB / 715 GiB avail; 0 B/s rd, 0 B/s wr, 117 op/s 2024-09-16T09:44:33.660 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:33 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3239028710' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "unique_pool_4", "pool2": "unique_pool_4", "yes_i_really_really_mean_it": true} : dispatch 2024-09-16T09:44:33.660 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:33 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3796101465' entity='client.admin' cmd={"prefix": "osd reweight", "id": 0, "weight": 1.0} : dispatch 2024-09-16T09:44:33.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:33 smithi084 ceph-mon[32629]: 1.8 scrub starts 2024-09-16T09:44:33.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:33 smithi084 ceph-mon[32629]: 1.8 scrub ok 2024-09-16T09:44:33.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:33 smithi084 ceph-mon[32629]: pgmap v1023: 168 pgs: 2 active+undersized+remapped+backfill_toofull, 4 peering, 162 active+clean; 583 KiB data, 7.1 GiB used, 708 GiB / 715 GiB avail; 0 B/s rd, 0 B/s wr, 117 op/s 2024-09-16T09:44:33.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:33 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3239028710' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "unique_pool_4", "pool2": "unique_pool_4", "yes_i_really_really_mean_it": true} : dispatch 2024-09-16T09:44:33.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:33 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3796101465' entity='client.admin' cmd={"prefix": "osd reweight", "id": 0, "weight": 1.0} : dispatch 2024-09-16T09:44:33.663 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-16T09:44:33.756 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:44:33.778 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:33.778 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:33.779 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:44:33.880 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:44:33.902 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:33.902 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:33.903 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:44:34.004 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:44:34.025 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:34.026 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:34.026 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:44:34.127 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:44:34.149 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:34.149 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:34.150 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:44:34.160 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:44:33 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:44:34.252 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:44:34.275 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:34.275 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:34.276 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:44:34.377 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:44:34.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:34.399 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:34.400 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:44:34.436 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:44:34.501 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:44:34.526 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:34.526 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:34.527 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:44:34.628 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1 2024-09-16T09:44:34.651 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:34.651 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:34.652 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.1' 2024-09-16T09:44:34.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:34 smithi084 ceph-mon[32629]: 5.b scrub starts 2024-09-16T09:44:34.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:34 smithi084 ceph-mon[32629]: 5.b scrub ok 2024-09-16T09:44:34.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:34 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3239028710' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_4", "pool2": "unique_pool_4", "yes_i_really_really_mean_it": true}]': finished 2024-09-16T09:44:34.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:34 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3796101465' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-16T09:44:34.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:34 smithi084 ceph-mon[32629]: osdmap e524: 8 total, 8 up, 8 in 2024-09-16T09:44:34.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:34 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:44:34.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:34 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:44:34.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:34 smithi070 ceph-mon[30388]: 5.b scrub starts 2024-09-16T09:44:34.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:34 smithi070 ceph-mon[30388]: 5.b scrub ok 2024-09-16T09:44:34.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:34 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3239028710' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_4", "pool2": "unique_pool_4", "yes_i_really_really_mean_it": true}]': finished 2024-09-16T09:44:34.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:34 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3796101465' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-16T09:44:34.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:34 smithi070 ceph-mon[30388]: osdmap e524: 8 total, 8 up, 8 in 2024-09-16T09:44:34.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:34 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:44:34.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:34 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:44:34.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:34 smithi070 ceph-mon[40691]: 5.b scrub starts 2024-09-16T09:44:34.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:34 smithi070 ceph-mon[40691]: 5.b scrub ok 2024-09-16T09:44:34.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:34 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3239028710' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_4", "pool2": "unique_pool_4", "yes_i_really_really_mean_it": true}]': finished 2024-09-16T09:44:34.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:34 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3796101465' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-16T09:44:34.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:34 smithi070 ceph-mon[40691]: osdmap e524: 8 total, 8 up, 8 in 2024-09-16T09:44:34.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:34 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:44:34.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:34 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:44:34.752 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2 2024-09-16T09:44:34.774 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:34.775 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:34.775 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.2' 2024-09-16T09:44:34.876 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:44:34.898 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:34.899 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:34.899 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:44:34.923 INFO:teuthology.orchestra.run.smithi084.stdout:ERROR: (22) Invalid argument 2024-09-16T09:44:34.923 INFO:teuthology.orchestra.run.smithi084.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-16T09:44:35.000 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:44:35.020 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd reweight 1 1 2024-09-16T09:44:35.022 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:35.022 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:35.023 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:44:35.124 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:44:35.145 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:35.146 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:35.146 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:44:35.247 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6 2024-09-16T09:44:35.272 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:35.273 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:35.273 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.6' 2024-09-16T09:44:35.374 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4 2024-09-16T09:44:35.396 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:35.397 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:35.397 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.4' 2024-09-16T09:44:35.497 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0 2024-09-16T09:44:35.518 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:35.519 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:35.519 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.0' 2024-09-16T09:44:35.603 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:35 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:44:35.603 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:35 smithi084 ceph-mon[32629]: pgmap v1025: 152 pgs: 152 active+clean; 583 KiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 727 B/s rd, 0 op/s 2024-09-16T09:44:35.603 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:35 smithi084 ceph-mon[32629]: 6.1b scrub starts 2024-09-16T09:44:35.603 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:35 smithi084 ceph-mon[32629]: 6.1b scrub ok 2024-09-16T09:44:35.603 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:35 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:44:35.603 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:35 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:44:35.603 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:35 smithi084 ceph-mon[32629]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-16T09:44:35.603 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:35 smithi084 ceph-mon[32629]: Health check cleared: PG_BACKFILL_FULL (was: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull) 2024-09-16T09:44:35.603 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:35 smithi084 ceph-mon[32629]: Cluster is now healthy 2024-09-16T09:44:35.620 DEBUG:teuthology.orchestra.run.smithi084:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7 2024-09-16T09:44:35.630 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.2 dump_ops_in_flight 2024-09-16T09:44:35.641 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: docker: command not found 2024-09-16T09:44:35.641 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:35.642 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi084 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.7' 2024-09-16T09:44:35.652 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:35 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:44:35.653 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:35 smithi070 ceph-mon[40691]: pgmap v1025: 152 pgs: 152 active+clean; 583 KiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 727 B/s rd, 0 op/s 2024-09-16T09:44:35.653 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:35 smithi070 ceph-mon[40691]: 6.1b scrub starts 2024-09-16T09:44:35.653 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:35 smithi070 ceph-mon[40691]: 6.1b scrub ok 2024-09-16T09:44:35.653 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:35 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:44:35.653 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:35 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:44:35.653 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:35 smithi070 ceph-mon[40691]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-16T09:44:35.653 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:35 smithi070 ceph-mon[40691]: Health check cleared: PG_BACKFILL_FULL (was: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull) 2024-09-16T09:44:35.653 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:35 smithi070 ceph-mon[40691]: Cluster is now healthy 2024-09-16T09:44:35.653 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:35 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:44:35.653 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:35 smithi070 ceph-mon[30388]: pgmap v1025: 152 pgs: 152 active+clean; 583 KiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 727 B/s rd, 0 op/s 2024-09-16T09:44:35.653 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:35 smithi070 ceph-mon[30388]: 6.1b scrub starts 2024-09-16T09:44:35.653 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:35 smithi070 ceph-mon[30388]: 6.1b scrub ok 2024-09-16T09:44:35.654 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:35 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:44:35.654 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:35 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:44:35.654 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:35 smithi070 ceph-mon[30388]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-16T09:44:35.654 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:35 smithi070 ceph-mon[30388]: Health check cleared: PG_BACKFILL_FULL (was: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull) 2024-09-16T09:44:35.654 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:35 smithi070 ceph-mon[30388]: Cluster is now healthy 2024-09-16T09:44:35.742 DEBUG:teuthology.orchestra.run.smithi070:> sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3 2024-09-16T09:44:35.763 INFO:teuthology.orchestra.run.smithi070.stderr:sudo: docker: command not found 2024-09-16T09:44:35.764 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T09:44:35.764 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi070 with status 1: 'sudo docker kill -s 1 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd.3' 2024-09-16T09:44:35.824 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2024-09-16T09:44:35.834 INFO:tasks.thrashosds:joining thrashosds 2024-09-16T09:44:36.290 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:44:36.300 INFO:teuthology.orchestra.run.smithi070.stdout:osd.0: {} 2024-09-16T09:44:36.300 INFO:teuthology.orchestra.run.smithi070.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-16T09:44:36.319 INFO:teuthology.orchestra.run.smithi070.stdout:osd.1: {} 2024-09-16T09:44:36.319 INFO:teuthology.orchestra.run.smithi070.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-16T09:44:36.334 INFO:teuthology.orchestra.run.smithi070.stderr:noscrub is set 2024-09-16T09:44:36.340 INFO:teuthology.orchestra.run.smithi070.stdout:osd.2: {} 2024-09-16T09:44:36.341 INFO:teuthology.orchestra.run.smithi070.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-16T09:44:36.358 INFO:teuthology.orchestra.run.smithi070.stdout:osd.3: {} 2024-09-16T09:44:36.359 INFO:teuthology.orchestra.run.smithi070.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-16T09:44:36.376 INFO:teuthology.orchestra.run.smithi070.stdout:osd.4: {} 2024-09-16T09:44:36.377 INFO:teuthology.orchestra.run.smithi070.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-16T09:44:36.393 INFO:teuthology.orchestra.run.smithi070.stdout:osd.5: {} 2024-09-16T09:44:36.393 INFO:teuthology.orchestra.run.smithi070.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-16T09:44:36.410 INFO:teuthology.orchestra.run.smithi070.stdout:osd.6: {} 2024-09-16T09:44:36.411 INFO:teuthology.orchestra.run.smithi070.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-16T09:44:36.427 INFO:teuthology.orchestra.run.smithi070.stdout:osd.7: {} 2024-09-16T09:44:36.427 INFO:teuthology.orchestra.run.smithi070.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-16T09:44:36.566 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:36 smithi070 ceph-mon[40691]: 3.2 scrub starts 2024-09-16T09:44:36.566 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:36 smithi070 ceph-mon[40691]: 3.2 scrub ok 2024-09-16T09:44:36.566 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:36 smithi070 ceph-mon[40691]: 3.12 scrub starts 2024-09-16T09:44:36.566 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:36 smithi070 ceph-mon[40691]: 3.12 scrub ok 2024-09-16T09:44:36.566 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:36 smithi070 ceph-mon[40691]: 6.6 scrub starts 2024-09-16T09:44:36.567 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:36 smithi070 ceph-mon[40691]: 6.6 scrub ok 2024-09-16T09:44:36.567 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:36 smithi070 ceph-mon[40691]: osdmap e525: 8 total, 8 up, 8 in 2024-09-16T09:44:36.567 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:36 smithi070 ceph-mon[40691]: 6.1a scrub starts 2024-09-16T09:44:36.567 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:36 smithi070 ceph-mon[40691]: 6.1a scrub ok 2024-09-16T09:44:36.567 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:36 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/2217472583' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:44:36.567 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:36 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:44:36.567 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:36 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3734081667' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:44:36.567 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:36 smithi070 ceph-mon[30388]: 3.2 scrub starts 2024-09-16T09:44:36.567 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:36 smithi070 ceph-mon[30388]: 3.2 scrub ok 2024-09-16T09:44:36.567 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:36 smithi070 ceph-mon[30388]: 3.12 scrub starts 2024-09-16T09:44:36.567 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:36 smithi070 ceph-mon[30388]: 3.12 scrub ok 2024-09-16T09:44:36.567 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:36 smithi070 ceph-mon[30388]: 6.6 scrub starts 2024-09-16T09:44:36.568 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:36 smithi070 ceph-mon[30388]: 6.6 scrub ok 2024-09-16T09:44:36.568 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:36 smithi070 ceph-mon[30388]: osdmap e525: 8 total, 8 up, 8 in 2024-09-16T09:44:36.568 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:36 smithi070 ceph-mon[30388]: 6.1a scrub starts 2024-09-16T09:44:36.568 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:36 smithi070 ceph-mon[30388]: 6.1a scrub ok 2024-09-16T09:44:36.568 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:36 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2217472583' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:44:36.568 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:36 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:44:36.568 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:36 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3734081667' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:44:36.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:36 smithi084 ceph-mon[32629]: 3.2 scrub starts 2024-09-16T09:44:36.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:36 smithi084 ceph-mon[32629]: 3.2 scrub ok 2024-09-16T09:44:36.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:36 smithi084 ceph-mon[32629]: 3.12 scrub starts 2024-09-16T09:44:36.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:36 smithi084 ceph-mon[32629]: 3.12 scrub ok 2024-09-16T09:44:36.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:36 smithi084 ceph-mon[32629]: 6.6 scrub starts 2024-09-16T09:44:36.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:36 smithi084 ceph-mon[32629]: 6.6 scrub ok 2024-09-16T09:44:36.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:36 smithi084 ceph-mon[32629]: osdmap e525: 8 total, 8 up, 8 in 2024-09-16T09:44:36.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:36 smithi084 ceph-mon[32629]: 6.1a scrub starts 2024-09-16T09:44:36.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:36 smithi084 ceph-mon[32629]: 6.1a scrub ok 2024-09-16T09:44:36.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:36 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/2217472583' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:44:36.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:36 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd={"prefix": "osd set", "key": "noscrub"} : dispatch 2024-09-16T09:44:36.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:36 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3734081667' entity='client.admin' cmd={"prefix": "osd ls"} : dispatch 2024-09-16T09:44:37.478 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:37 smithi070 ceph-mon[40691]: 5.15 scrub starts 2024-09-16T09:44:37.478 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:37 smithi070 ceph-mon[40691]: 5.15 scrub ok 2024-09-16T09:44:37.479 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:37 smithi070 ceph-mon[40691]: pgmap v1027: 152 pgs: 152 active+clean; 583 KiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-09-16T09:44:37.479 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:37 smithi070 ceph-mon[40691]: 5.1e scrub starts 2024-09-16T09:44:37.479 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:37 smithi070 ceph-mon[40691]: 5.1e scrub ok 2024-09-16T09:44:37.479 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:37 smithi070 ceph-mon[40691]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:44:37.479 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:37 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-16T09:44:37.479 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:37 smithi070 ceph-mon[40691]: osdmap e526: 8 total, 8 up, 8 in 2024-09-16T09:44:37.479 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:37 smithi070 ceph-mon[30388]: 5.15 scrub starts 2024-09-16T09:44:37.479 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:37 smithi070 ceph-mon[30388]: 5.15 scrub ok 2024-09-16T09:44:37.479 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:37 smithi070 ceph-mon[30388]: pgmap v1027: 152 pgs: 152 active+clean; 583 KiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-09-16T09:44:37.480 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:37 smithi070 ceph-mon[30388]: 5.1e scrub starts 2024-09-16T09:44:37.480 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:37 smithi070 ceph-mon[30388]: 5.1e scrub ok 2024-09-16T09:44:37.480 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:37 smithi070 ceph-mon[30388]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:44:37.480 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:37 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-16T09:44:37.480 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:37 smithi070 ceph-mon[30388]: osdmap e526: 8 total, 8 up, 8 in 2024-09-16T09:44:37.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:37 smithi084 ceph-mon[32629]: 5.15 scrub starts 2024-09-16T09:44:37.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:37 smithi084 ceph-mon[32629]: 5.15 scrub ok 2024-09-16T09:44:37.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:37 smithi084 ceph-mon[32629]: pgmap v1027: 152 pgs: 152 active+clean; 583 KiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-09-16T09:44:37.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:37 smithi084 ceph-mon[32629]: 5.1e scrub starts 2024-09-16T09:44:37.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:37 smithi084 ceph-mon[32629]: 5.1e scrub ok 2024-09-16T09:44:37.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:37 smithi084 ceph-mon[32629]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-16T09:44:37.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:37 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-16T09:44:37.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:37 smithi084 ceph-mon[32629]: osdmap e526: 8 total, 8 up, 8 in 2024-09-16T09:44:37.690 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:44:38.340 INFO:teuthology.orchestra.run.smithi070.stderr:reweighted osd.1 to 1 (10000) 2024-09-16T09:44:38.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:38 smithi084 ceph-mon[32629]: 6.c scrub starts 2024-09-16T09:44:38.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:38 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:44:38.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:38 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:44:38.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:38 smithi084 ceph-mon[32629]: osdmap e527: 8 total, 8 up, 8 in 2024-09-16T09:44:38.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:38 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/739209795' entity='client.admin' cmd={"prefix": "osd reweight", "id": 1, "weight": 1.0} : dispatch 2024-09-16T09:44:38.661 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:38 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 1, "weight": 1.0} : dispatch 2024-09-16T09:44:38.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:38 smithi070 ceph-mon[30388]: 6.c scrub starts 2024-09-16T09:44:38.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:38 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:44:38.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:38 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:44:38.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:38 smithi070 ceph-mon[30388]: osdmap e527: 8 total, 8 up, 8 in 2024-09-16T09:44:38.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:38 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/739209795' entity='client.admin' cmd={"prefix": "osd reweight", "id": 1, "weight": 1.0} : dispatch 2024-09-16T09:44:38.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:38 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 1, "weight": 1.0} : dispatch 2024-09-16T09:44:38.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:38 smithi070 ceph-mon[40691]: 6.c scrub starts 2024-09-16T09:44:38.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:38 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:44:38.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:38 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:44:38.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:38 smithi070 ceph-mon[40691]: osdmap e527: 8 total, 8 up, 8 in 2024-09-16T09:44:38.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:38 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/739209795' entity='client.admin' cmd={"prefix": "osd reweight", "id": 1, "weight": 1.0} : dispatch 2024-09-16T09:44:38.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:38 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 1, "weight": 1.0} : dispatch 2024-09-16T09:44:39.262 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd reweight 2 1 2024-09-16T09:44:39.487 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:39 smithi070 ceph-mon[30388]: pgmap v1030: 152 pgs: 152 active+clean; 583 KiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-16T09:44:39.487 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:39 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-16T09:44:39.487 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:39 smithi070 ceph-mon[30388]: osdmap e528: 8 total, 8 up, 8 in 2024-09-16T09:44:39.488 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:39 smithi070 ceph-mon[40691]: pgmap v1030: 152 pgs: 152 active+clean; 583 KiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-16T09:44:39.488 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:39 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-16T09:44:39.488 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:39 smithi070 ceph-mon[40691]: osdmap e528: 8 total, 8 up, 8 in 2024-09-16T09:44:39.494 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T09:44:39.494 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T09:44:39.495 INFO:teuthology.orchestra.run.smithi070.stdout: "num_ops": 0 2024-09-16T09:44:39.495 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T09:44:39.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:39 smithi084 ceph-mon[32629]: pgmap v1030: 152 pgs: 152 active+clean; 583 KiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-16T09:44:39.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:39 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-16T09:44:39.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:39 smithi084 ceph-mon[32629]: osdmap e528: 8 total, 8 up, 8 in 2024-09-16T09:44:40.136 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.2 dump_blocked_ops 2024-09-16T09:44:40.170 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:44:40.292 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd unset noscrub 2024-09-16T09:44:40.420 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:40 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:44:40.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:40 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:44:40.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:40 smithi070 ceph-mon[40691]: osdmap e529: 8 total, 8 up, 8 in 2024-09-16T09:44:40.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:40 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:44:40.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:40 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:44:40.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:40 smithi070 ceph-mon[30388]: osdmap e529: 8 total, 8 up, 8 in 2024-09-16T09:44:40.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:40 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:44:40.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:40 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:44:40.660 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:40 smithi084 ceph-mon[32629]: osdmap e529: 8 total, 8 up, 8 in 2024-09-16T09:44:40.771 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:44:40.793 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:44:41.265 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T09:44:41.265 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [], 2024-09-16T09:44:41.265 INFO:teuthology.orchestra.run.smithi070.stdout: "complaint_time": 30, 2024-09-16T09:44:41.265 INFO:teuthology.orchestra.run.smithi070.stdout: "num_blocked_ops": 0 2024-09-16T09:44:41.265 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T09:44:41.491 INFO:teuthology.orchestra.run.smithi070.stderr:reweighted osd.2 to 1 (10000) 2024-09-16T09:44:41.894 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:41 smithi070 ceph-mon[30388]: pgmap v1033: 152 pgs: 152 active+clean; 583 KiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:44:41.895 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:41 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3054122034' entity='client.admin' cmd={"prefix": "osd reweight", "id": 2, "weight": 1.0} : dispatch 2024-09-16T09:44:41.895 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:41 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 2, "weight": 1.0} : dispatch 2024-09-16T09:44:41.895 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:41 smithi070 ceph-mon[40691]: pgmap v1033: 152 pgs: 152 active+clean; 583 KiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:44:41.895 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:41 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3054122034' entity='client.admin' cmd={"prefix": "osd reweight", "id": 2, "weight": 1.0} : dispatch 2024-09-16T09:44:41.895 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:41 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 2, "weight": 1.0} : dispatch 2024-09-16T09:44:41.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:41 smithi084 ceph-mon[32629]: pgmap v1033: 152 pgs: 152 active+clean; 583 KiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:44:41.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:41 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3054122034' entity='client.admin' cmd={"prefix": "osd reweight", "id": 2, "weight": 1.0} : dispatch 2024-09-16T09:44:41.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:41 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 2, "weight": 1.0} : dispatch 2024-09-16T09:44:41.923 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph daemon osd.2 dump_historic_ops 2024-09-16T09:44:42.171 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:44:41 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:44:41] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:44:42.566 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:42 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-16T09:44:42.566 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:42 smithi070 ceph-mon[30388]: osdmap e530: 8 total, 8 up, 8 in 2024-09-16T09:44:42.566 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:42 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-16T09:44:42.566 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:42 smithi070 ceph-mon[40691]: osdmap e530: 8 total, 8 up, 8 in 2024-09-16T09:44:42.594 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd reweight 3 1 2024-09-16T09:44:42.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:42 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-16T09:44:42.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:42 smithi084 ceph-mon[32629]: osdmap e530: 8 total, 8 up, 8 in 2024-09-16T09:44:43.118 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:44:43.118 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:44:43.501 INFO:teuthology.orchestra.run.smithi070.stderr:noscrub is unset 2024-09-16T09:44:43.633 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:43 smithi070 ceph-mon[30388]: pgmap v1035: 152 pgs: 152 active+clean; 583 KiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-16T09:44:43.633 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:43 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/814743436' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:44:43.633 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:43 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:44:43.633 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:43 smithi070 ceph-mon[40691]: pgmap v1035: 152 pgs: 152 active+clean; 583 KiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-16T09:44:43.633 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:43 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/814743436' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:44:43.634 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:43 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:44:43.910 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:44:43 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:44:43.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:43 smithi084 ceph-mon[32629]: pgmap v1035: 152 pgs: 152 active+clean; 583 KiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-16T09:44:43.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:43 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/814743436' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:44:43.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:43 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "noscrub"} : dispatch 2024-09-16T09:44:44.368 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd unset nodeep-scrub 2024-09-16T09:44:44.630 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:44 smithi070 ceph-mon[40691]: Health check cleared: OSDMAP_FLAGS (was: noscrub flag(s) set) 2024-09-16T09:44:44.630 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:44 smithi070 ceph-mon[40691]: Cluster is now healthy 2024-09-16T09:44:44.630 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:44 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-16T09:44:44.630 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:44 smithi070 ceph-mon[40691]: osdmap e531: 8 total, 8 up, 8 in 2024-09-16T09:44:44.630 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:44 smithi070 ceph-mon[40691]: 1.10 scrub starts 2024-09-16T09:44:44.630 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:44 smithi070 ceph-mon[40691]: 1.10 scrub ok 2024-09-16T09:44:44.630 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:44 smithi070 ceph-mon[30388]: Health check cleared: OSDMAP_FLAGS (was: noscrub flag(s) set) 2024-09-16T09:44:44.630 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:44 smithi070 ceph-mon[30388]: Cluster is now healthy 2024-09-16T09:44:44.630 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:44 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-16T09:44:44.630 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:44 smithi070 ceph-mon[30388]: osdmap e531: 8 total, 8 up, 8 in 2024-09-16T09:44:44.631 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:44 smithi070 ceph-mon[30388]: 1.10 scrub starts 2024-09-16T09:44:44.631 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:44 smithi070 ceph-mon[30388]: 1.10 scrub ok 2024-09-16T09:44:44.650 INFO:teuthology.orchestra.run.smithi070.stdout:{ 2024-09-16T09:44:44.651 INFO:teuthology.orchestra.run.smithi070.stdout: "size": 20, 2024-09-16T09:44:44.651 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 600, 2024-09-16T09:44:44.651 INFO:teuthology.orchestra.run.smithi070.stdout: "ops": [ 2024-09-16T09:44:44.651 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.651 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "backfill_remove(11.d e518 11:b0004d4e:::benchmark_data_smithi070_191495_object63957:head,467'4002,11:b0005196:::benchmark_data_smithi070_191495_object59597:head,466'3731,11:b000f9c7:::benchmark_data_smithi070_191495_object96398:head,478'6110,11:b00147f9:::benchmark_data_smithi070_191495_object118772:head,487'7534,11:b0018f8f:::benchmark_data_smithi070_191495_object70429:head,468'4396,11:b001b536:::benchmark_data_smithi070_191495_object46030:head,464'2875,11:b001d284:::benchmark_data_smithi070_191495_object17620:head,460'1130,11:b002b863:::benchmark_data_smithi070_191495_object37711:head,464'2356,11:b002c579:::benchmark_data_smithi070_191495_object72578:head,468'4541,11:b002d982:::benchmark_data_smithi070_191495_object63548:head,467'3977,11:b00413cf:::benchmark_data_smithi070_191495_object69527:head,468'4339,11:b004192e:::benchmark_data_smithi070_191495_object117323:head,486'7429,11:b004e22a:::benchmark_data_smithi070_191495_object53272:head,466'3348,11:b00524a9:::benchmark_data_smithi070_191495_object116307:head,486'7373,11:b00554f2:::benchmark_data_smithi070_191495_object29394:head,463'1828,11:b0063b0f:::benchmark_data_smithi070_191495_object32683:head,463'2031,11:b007891e:::benchmark_data_smithi070_191495_object48961:head,465'3074,11:b007b393:::benchmark_data_smithi070_191495_object64086:head,467'4010,11:b008562a:::benchmark_data_smithi070_191495_object81751:head,473'5175,11:b0089085:::benchmark_data_smithi070_191495_object27845:head,463'1743,11:b009c2b5:::benchmark_data_smithi070_191495_object123764:head,489'7851,11:b00b96b6:::benchmark_data_smithi070_191495_object114781:head,486'7280,11:b00ce03d:::benchmark_data_smithi070_191495_object28902:head,463'1802,11:b00d86c0:::benchmark_data_smithi070_191495_object39215:head,464'2450,11:b00d8e1f:::benchmark_data_smithi070_191495_object98178:head,479'6232,11:b00f0254:::benchmark_data_smithi070_191495_object78539:head,472'4961,11:b00f3eef:::benchmark_data_smithi070_191495_object107189:head,481'6790,11:b00f8c4f:::benchmark_data_smithi070_191495_object108892:head,483'6903,11:b00fa910:::benchmark_data_smithi070_191495_object126114:head,491'7999,11:b01118c0:::benchmark_data_smithi070_191495_object56187:head,466'3534,11:b0111a89:::benchmark_data_smithi070_191495_object71305:head,468'4452,11:b0119163:::benchmark_data_smithi070_191495_object86883:head,474'5509,11:b013a82d:::benchmark_data_smithi070_191495_object98793:head,479'6272,11:b0140925:::benchmark_data_smithi070_191495_object82734:head,474'5241,11:b01451f5:::benchmark_data_smithi070_191495_object82484:head,474'5225,11:b014c978:::benchmark_data_smithi070_191495_object81919:head,474'5190,11:b014cbc4:::benchmark_data_smithi070_191495_object126428:head,491'8015,11:b0150cef:::benchmark_data_smithi070_191495_object107452:head,482'6806,11:b0159d3a:::benchmark_data_smithi070_191495_object123829:head,489'7856,11:b0160abb:::benchmark_data_smithi070_191495_object37843:head,464'2362,11:b016c8de:::benchmark_data_smithi070_191495_object123393:head,488'7827,11:b0170020:::benchmark_data_smithi070_191495_object38477:head,464'2402,11:b0171538:::benchmark_data_smithi070_191495_object88402:head,474'5602,11:b017ee7a:::benchmark_data_smithi070_191495_object113117:head,485'7178,11:b01852eb:::benchmark_data_smithi070_191495_object111356:head,484'7043,11:b0187f02:::benchmark_data_smithi070_191495_object24426:head,462'1543,11:b0188924:::benchmark_data_smithi070_191495_object32908:head,463'2053,11:b018a1e1:::benchmark_data_smithi070_191495_object49179:head,465'3091,11:b019314b:::benchmark_data_smithi070_191495_object123146:head,488'7812,11:b019aaf0:::benchmark_data_smithi070_191495_object73885:head,469'4616,11:b01b5847:::benchmark_data_smithi070_191495_object96702:head,479'6140,11:b01c4b30:::benchmark_data_smithi070_191495_object50533:head,465'3174,11:b01c4b45:::benchmark_data_smithi070_191495_object33996:head,464'2116,11:b01ce624:::benchmark_data_smithi070_191495_object84873:head,474'5353,11:b01cfde5:::benchmark_data_smithi070_191495_object58366:head,466'3672,11:b01d2f2b:::benchmark_data_smithi070_191495_object52338:head,465'3280,11:b01e012e:::benchmark_data_smithi070_191495_object76959:head,471'4853,11:b0202bf0:::benchmark_data_smithi070_191495_object119038:head,487'7555,11:b0202e6d:::benchmark_data_smithi070_191495_object20399:head,460'1301,11:b021913b:::benchmark_data_smithi070_191495_object34151:head,464'2124,11:b021af15:::benchmark_data_smithi070_191495_object117162:head,486'7422,11:b021f270:::benchmark_data_smithi070_191495_object42390:head,464'2654,11:b022f04e:::benchmark_data_smithi070_191495_object119869:head,488'7603,11:b0238781:::benchmark_data_smithi070_191495_object55833:head,466'3514,11:b0239c46:::benchmark_data_smithi070_191495_object67869:head,468'4243,11:b024a348:::benchmark_data_smithi070_191495_object61118:head,467'3832,11:b025a3d6:::benchmark_data_smithi070_191495_object25963:head,463'1627,11:b025fd11:::benchmark_data_smithi070_191495_object104267:head,480'6619,11:b0264c42:::benchmark_data_smithi070_191495_object75380:head,470'4731,11:b0269e57:::benchmark_data_smithi070_191495_object80838:head,473'5110,11:b026c734:::benchmark_data_smithi070_191495_object53600:head,466'3362,11:b0278918:::benchmark_data_smithi070_191495_object123778:head,489'7852,11:b027a5db:::benchmark_data_smithi070_191495_object75538:head,471'4739,11:b027a7a3:::benchmark_data_smithi070_191495_object59656:head,466'3737,11:b0288e73:::benchmark_data_smithi070_191495_object108497:head,483'6875,11:b029547c:::benchmark_data_smithi070_191495_object96924:head,479'6154,11:b029c075:::benchmark_data_smithi070_191495_object106786:head,481'6769,11:b029fc27:::benchmark_data_smithi070_191495_object60826:head,467'3810,11:b02a0645:::benchmark_data_smithi070_191495_object26080:head,463'1632,11:b02bcf47:::benchmark_data_smithi070_191495_object77485:head,471'4901,11:b02bdae8:::benchmark_data_smithi070_191495_object44778:head,464'2797,11:b02c3b86:::benchmark_data_smithi070_191495_object93729:head,477'5939,11:b02cc34c:::benchmark_data_smithi070_191495_object79996:head,472'5059,11:b02cfca0:::benchmark_data_smithi070_191495_object65820:head,467'4125,11:b02daef6:::benchmark_data_smithi070_191495_object39487:head,464'2460,11:b02e5d1c:::benchmark_data_smithi070_191495_object57582:head,466'3625,11:b02f5de9:::benchmark_data_smithi070_191495_object80985:head,473'5123,11:b02fe775:::benchmark_data_smithi070_191495_object39075:head,464'2441,11:b03082c1:::benchmark_data_smithi070_191495_object77283:head,471'4886,11:b0309f56:::benchmark_data_smithi070_191495_object104853:head,480'6660,11:b0315781:::benchmark_data_smithi070_191495_object81691:head,473'5169,11:b03261ed:::benchmark_data_smithi070_191495_object73072:head,469'4575,11:b033e382:::benchmark_data_smithi070_191495_object60582:head,467'3791,11:b0343d85:::benchmark_data_smithi070_191495_object25683:head,463'1609,11:b034c843:::benchmark_data_smithi070_191495_object64094:head,467'4011,11:b034cff4:::benchmark_data_smithi070_191495_object44103:head,464'2747,11:b0351962:::benchmark_data_smithi070_191495_object123411:head,488'7828,11:b03590e7:::benchmark_data_smithi070_191495_object39881:head,464'2488,11:b0366750:::benchmark_data_smithi070_191495_object43361:head,464'2713,11:b0371b54:::benchmark_data_smithi070_191495_object51973:head,465'3262,11:b03731a3:::benchmark_data_smithi070_191495_object127932:head,491'8109,11:b037d300:::benchmark_data_smithi070_191495_object91430:head,476'5780,11:b0380302:::benchmark_data_smithi070_191495_object54712:head,466'3440,11:b0382dec:::benchmark_data_smithi070_191495_object94362:head,478'5985,11:b0388b3c:::benchmark_data_smithi070_191495_object59525:head,466'3729,11:b038f5db:::benchmark_data_smithi070_191495_object101620:head,479'6453,11:b03a2da0:::benchmark_data_smithi070_191495_object97653:head,479'6206,11:b03a4ca8:::benchmark_data_smithi070_191495_object102046:head,480'6478,11:b03a6487:::benchmark_data_smithi070_191495_object96638:head,479'6134,11:b03a83e1:::benchmark_data_smithi070_191495_object19452:head,460'1249,11:b03b15f8:::benchmark_data_smithi070_191495_object27907:head,463'1748,11:b03be7f6:::benchmark_data_smithi070_191495_object73226:head,469'4584,11:b03da33e:::benchmark_data_smithi070_191495_object128792:head,491'8170,11:b03dd54f:::benchmark_data_smithi070_191495_object50531:head,465'3173,11:b03df36b:::benchmark_data_smithi070_191495_object112788:head,485'7159,11:b03e1abd:::benchmark_data_smithi070_191495_object59955:head,467'3752,11:b03e703b:::benchmark_data_smithi070_191495_object129241:head,491'8193,11:b03e8304:::benchmark_data_smithi070_191495_object109943:head,483'6968,11:b03f8cd8:::benchmark_data_smithi070_191495_object29211:head,463'1815,11:b03fc3b0:::benchmark_data_smithi070_191495_object65724:head,467'4117,11:b03ffce9:::benchmark_data_smithi070_191495_object59964:head,467'3753,11:b0402979:::benchmark_data_smithi070_191495_object124773:head,490'7916,11:b0414fe1:::benchmark_data_smithi070_191495_object50213:head,465'3151,11:b041a305:::benchmark_data_smithi070_191495_object40036:head,464'2496,11:b041dc03:::benchmark_data_smithi070_191495_object74273:head,470'4654,11:b041dd33:::benchmark_data_smithi070_191495_object80134:head,472'5072,11:b041dd8d:::benchmark_data_smithi070_191495_object126846:head,491'8049,11:b0428b52:::benchmark_data_smithi070_191495_object63805:head,467'3992,11:b042ad33:::benchmark_data_smithi070_191495_object89599:head,475'5675,11:b04394c9:::benchmark_data_smithi070_191495_object64721:head,467'4061,11:b04522e1:::benchmark_data_smithi070_191495_object117602:head,486'7450,11:b0457324:::benchmark_data_smithi070_191495_object89142:head,474'5644,11:b045939b:::benchmark_data_smithi070_191495_object119116:head,487'7560,11:b046cece:::benchmark_data_smithi070_191495_object86743:head,474'5496,11:b046d7ac:::benchmark_data_smithi070_191495_object30276:head,463'1887,11:b046dc06:::benchmark_data_smithi070_191495_object104670:head,480'6642,11:b0470fdc:::benchmark_data_smithi070_191495_object48831:head,465'3062,11:b0478cc1:::benchmark_data_smithi070_191495_object22203:head,461'1402,11:b048d7e3:::benchmark_data_smithi070_191495_object28757:head,463'1792,11:b049baa1:::benchmark_data_smithi070_191495_object116505:head,486'7383,11:b049dd1f:::benchmark_data_smithi070_191495_object96159:head,478'6089,11:b04a134f:::benchmark_data_smithi070_191495_object44680:head,464'2792,11:b04a2be6:::benchmark_data_smithi070_191495_object40624:head,464'2535,11:b04aa22c:::benchmark_data_smithi070_191495_object30678:head,463'1906,11:b04af464:::benchmark_data_smithi070_191495_object130441:head,492'8263,11:b04b42b2:::benchmark_data_smithi070_191495_object106595:head,481'6754,11:b04b8d6f:::benchmark_data_smithi070_191495_object98182:head,479'6234,11:b04c4d9d:::benchmark_data_smithi070_191495_object35136:head,464'2187,11:b04ca605:::benchmark_data_smithi070_191495_object56367:head,466'3548,11:b04cb033:::benchmark_data_smithi070_191495_object72183:head,468'4512,11:b04cb5c4:::benchmark_data_smithi070_191495_object93317:head,477'5925,11:b04f0aa2:::benchmark_data_smithi070_191495_object16215:head,460'1027,11:b0503995:::benchmark_data_smithi070_191495_object115055:head,486'7294,11:b0505a95:::benchmark_data_smithi070_191495_object77337:head,471'4891,11:b050f37a:::benchmark_data_smithi070_191495_object37520:head,464'2345,11:b0514a0c:::benchmark_data_smithi070_191495_object121623:head,488'7707,11:b05159fd:::benchmark_data_smithi070_191495_object44525:head,464'2785,11:b052b853:::benchmark_data_smithi070_191495_object72999:head,469'4572,11:b053238d:::benchmark_data_smithi070_191495_object118373:head,487'7508,11:b053adbe:::benchmark_data_smithi070_191495_object40500:head,464'2527,11:b05466b5:::benchmark_data_smithi070_191495_object109319:head,483'6929,11:b054ac9a:::benchmark_data_smithi070_191495_object88805:head,474'5623,11:b054ccdb:::benchmark_data_smithi070_191495_object48032:head,465'3005,11:b056103e:::benchmark_data_smithi070_191495_object105178:head,481'6678,11:b056a307:::benchmark_data_smithi070_191495_object102704:head,480'6514,11:b0574b82:::benchmark_data_smithi070_191495_object34531:head,464'2156,11:b0578667:::benchmark_data_smithi070_191495_object53623:head,466'3363,11:b057a1c7:::benchmark_data_smithi070_191495_object111373:head,484'7045,11:b058781c:::benchmark_data_smithi070_191495_object22432:head,461'1415,11:b059a818:::benchmark_data_smithi070_191495_object35180:head,464'2192,11:b05ab6e2:::benchmark_data_smithi070_191495_object85576:head,474'5412,11:b05ac569:::benchmark_data_smithi070_191495_object87380:head,474'5536,11:b05b65a3:::benchmark_data_smithi070_191495_object71630:head,468'4477,11:b05b7f88:::benchmark_data_smithi070_191495_object118290:head,487'7503,11:b05bcc7f:::benchmark_data_smithi070_191495_object89145:head,474'5645,11:b05bd755:::benchmark_data_smithi070_191495_object112418:head,485'7138,11:b05c5af5:::benchmark_data_smithi070_191495_object81725:head,473'5172,11:b05c9db0:::benchmark_data_smithi070_191495_object47679:head,464'2980,11:b05dc0ca:::benchmark_data_smithi070_191495_object50236:head,465'3154,11:b05de38e:::benchmark_data_smithi070_191495_object48880:head,465'3068,11:b05f2614:::benchmark_data_smithi070_191495_object41657:head,464'2593,11:b05f3fa5:::benchmark_data_smithi070_191495_object21229:head,461'1345,11:b05f722b:::benchmark_data_smithi070_191495_object80649:head,473'5102,11:b05f9803:::benchmark_data_smithi070_191495_object92233:head,476'5845,11:b06020c3:::benchmark_data_smithi070_191495_object56377:head,466'3549,11:b0602aab:::benchmark_data_smithi070_191495_object48414:head,465'3033,11:b060aec4:::benchmark_data_smithi070_191495_object60354:head,467'3775,11:b06197df:::benchmark_data_smithi070_191495_object37970:head,464'2376,11:b061f677:::benchmark_data_smithi070_191495_object77630:head,471'4910,11:b062427b:::benchmark_data_smithi070_191495_object92212:head,476'5843,11:b063faac:::benchmark_data_smithi070_191495_object113337:head,486'7191,11:b0646262:::benchmark_data_smithi070_191495_object41992:head,464'2619,11:b0652943:::benchmark_data_smithi070_191495_object60692:head,467'3799,11:b0657529:::benchmark_data_smithi070_191495_object129092:head,491'8187,11:b0662422:::benchmark_data_smithi070_191495_object86365:head,474'5464,11:b06706e3:::benchmark_data_smithi070_191495_object78176:head,472'4941,11:b0673232:::benchmark_data_smithi070_191495_object95763:head,478'6074,11:b0678119:::benchmark_data_smithi070_191495_object110853:head,484'7025,11:b06886c3:::benchmark_data_smithi070_191495_object63586:head,467'3980,11:b068e88b:::benchmark_data_smithi070_191495_object18215:head,460'1173,11:b0692e28:::benchmark_data_smithi070_191495_object80781:head,473'5109,11:b0694469:::benchmark_data_smithi070_191495_object45267:head,464'2825,11:b06945b0:::benchmark_data_smithi070_191495_object78128:head,472'4936,11:b06991ac:::benchmark_data_smithi070_191495_object39146:head,464'2445,11:b069b2b2:::benchmark_data_smithi070_191495_object42114:head,464'2626,11:b06a04d1:::benchmark_data_smithi070_191495_object125578:head,490'7965,11:b06a3b49:::benchmark_data_smithi070_191495_object88812:head,474'5624,11:b06b443e:::benchmark_data_smithi070_191495_object101948:head,480'6475,11:b06b721a:::benchmark_data_smithi070_191495_object94117:head,478'5972,11:b06c9761:::benchmark_data_smithi070_191495_object31511:head,463'1954,11:b06cde6c:::benchmark_data_smithi070_191495_object126709:head,491'8041,11:b06d001a:::benchmark_data_smithi070_191495_object33731:head,464'2103,11:b06dc6c4:::benchmark_data_smithi070_191495_object17276:head,460'1106,11:b06de3de:::benchmark_data_smithi070_191495_object114035:head,486'7232,11:b06e6ee0:::benchmark_data_smithi070_191495_object51518:head,465'3234,11:b06e8960:::benchmark_data_smithi070_191495_object27364:head,463'1710,11:b06eaa1e:::benchmark_data_smithi070_191495_object126800:head,491'8045,11:b06eba29:::benchmark_data_smithi070_191495_object36459:head,464'2276,11:b06f61cb:::benchmark_data_smithi070_191495_object39578:head,464'2468,11:b06fa639:::benchmark_data_smithi070_191495_object82529:head,474'5229,11:b0700b8d:::benchmark_data_smithi070_191495_object52123:head,465'3270,11:b070625d:::benchmark_data_smithi070_191495_object29837:head,463'1863,11:b0729306:::benchmark_data_smithi070_191495_object97523:head,479'6195,11:b072b1b5:::benchmark_data_smithi070_191495_object64103:head,467'4013,11:b072e23a:::benchmark_data_smithi070_191495_object94414:head,478'5988,11:b0736d70:::benchmark_data_smithi070_191495_object17289:head,460'1107,11:b0740046:::benchmark_data_smithi070_191495_object42702:head,464'2672,11:b0757774:::benchmark_data_smithi070_191495_object106524:head,481'6752,11:b0759d59:::benchmark_data_smithi070_191495_object99573:head,479'6321,11:b075d1cc:::benchmark_data_smithi070_191495_object126483:head,491'8019,11:b07631ff:::benchmark_data_smithi070_191495_object111631:head,484'7074,11:b076e80b:::benchmark_data_smithi070_191495_object44842:head,464'2801,11:b07769b6:::benchmark_data_smithi070_191495_object92257:head,476'5849,11:b0784cf8:::benchmark_data_smithi070_191495_object19308:head,460'1240,11:b0788559:::benchmark_data_smithi070_191495_object25844:head,463'1617,11:b07a2e64:::benchmark_data_smithi070_191495_object33145:head,463'2069,11:b07bdf46:::benchmark_data_smithi070_191495_object32371:head,463'2009,11:b07d1057:::benchmark_data_smithi070_191495_object24359:head,462'1537,11:b07d35f3:::benchmark_data_smithi070_191495_object55295:head,466'3475,11:b07e0e45:::benchmark_data_smithi070_191495_object56035:head,466'3527,11:b07eae2e:::benchmark_data_smithi070_191495_object83559:head,474'5277,11:b08043d2:::benchmark_data_smithi070_191495_object77338:head,471'4892,11:b080c0f5:::benchmark_data_smithi070_191495_object52672:head,466'3310,11:b081458d:::benchmark_data_smithi070_191495_object96172:head,478'6090,11:b081dd79:::benchmark_data_smithi070_191495_object125596:head,490'7966,11:b08284c7:::benchmark_data_smithi070_191495_object50658:head,465'3184,11:b082b4b5:::benchmark_data_smithi070_191495_object75477:head,471'4736,11:b084340c:::benchmark_data_smithi070_191495_object123997:head,489'7871,11:b08464bd:::benchmark_data_smithi070_191495_object119660:head,488'7595,11:b084d6ab:::benchmark_data_smithi070_191495_object85652:head,474'5416,11:b0852d57:::benchmark_data_smithi070_191495_object118491:head,487'7515,11:b0856569:::benchmark_data_smithi070_191495_object103084:head,480'6547,11:b086085b:::benchmark_data_smithi070_191495_object92401:head,476'5859,11:b0861ef8:::benchmark_data_smithi070_191495_object19953:head,460'1282,11:b0865480:::benchmark_data_smithi070_191495_object102133:head,480'6479,11:b086d42a:::benchmark_data_smithi070_191495_object30647:head,463'1905,11:b086d722:::benchmark_data_smithi070_191495_object108353:head,482'6863,11:b0877dc3:::benchmark_data_smithi070_191495_object122269:head,488'7759,11:b0886c93:::benchmark_data_smithi070_191495_object62237:head,467'3902,11:b088898c:::benchmark_data_smithi070_191495_object28377:head,463'1771,11:b0895737:::benchmark_data_smithi070_191495_object18489:head,460'1185,11:b0897303:::benchmark_data_smithi070_191495_object27678:head,463'1730,11:b089bc38:::benchmark_data_smithi070_191495_object93970:head,477'5959,11:b089db54:::benchmark_data_smithi070_191495_object68964:head,468'4299,11:b08bd1fd:::benchmark_data_smithi070_191495_object100164:head,479'6361,11:b08cbf26:::benchmark_data_smithi070_191495_object70330:head,468'4390,11:b08d9e86:::benchmark_data_smithi070_191495_object79560:head,472'5034,11:b08e19dd:::benchmark_data_smithi070_191495_object96658:head,479'6137,11:b08e468b:::benchmark_data_smithi070_191495_object86268:head,474'5456,11:b08ecec6:::benchmark_data_smithi070_191495_object59760:head,466'3743,11:b08f17c3:::benchmark_data_smithi070_191495_object29676:head,463'1846,11:b09010b6:::benchmark_data_smithi070_191495_object80060:head,472'5067,11:b09030e2:::benchmark_data_smithi070_191495_object32099:head,463'1994,11:b091ad99:::benchmark_data_smithi070_191495_object53206:head,466'3343,11:b0924735:::benchmark_data_smithi070_191495_object18313:head,460'1176,11:b0939055:::benchmark_data_smithi070_191495_object124393:head,489'7901,11:b093c143:::benchmark_data_smithi070_191495_object18537:head,460'1187,11:b09596a2:::benchmark_data_smithi070_191495_object23045:head,462'1457,11:b095f73e:::benchmark_data_smithi070_191495_object71264:head,468'4447,11:b0963909:::benchmark_data_smithi070_191495_object50425:head,465'3162,11:b09654b6:::benchmark_data_smithi070_191495_object55758:head,466'3505,11:b096a819:::benchmark_data_smithi070_191495_object77804:head,471'4921,11:b096c76d:::benchmark_data_smithi070_191495_object40202:head,464'2506,11:b0970523:::benchmark_data_smithi070_191495_object33574:head,464'2094,11:b0976df4:::benchmark_data_smithi070_191495_object25826:head,463'1615,11:b097971d:::benchmark_data_smithi070_191495_object31702:head,463'1963,11:b097d463:::benchmark_data_smithi070_191495_object85890:head,474'5436,11:b098ba69:::benchmark_data_smithi070_191495_object101834:head,480'6463,11:b0997807:::benchmark_data_smithi070_191495_object128770:head,491'8164,11:b09a6529:::benchmark_data_smithi070_191495_object55945:head,466'3523,11:b09ac055:::benchmark_data_smithi070_191495_object105664:head,481'6699,11:b09b2a22:::benchmark_data_smithi070_191495_object77651:head,471'4911,11:b09be5eb:::benchmark_data_smithi070_191495_object64375:head,467'4032,11:b09cd235:::benchmark_data_smithi070_191495_object91831:head,476'5813,11:b09d600b:::benchmark_data_smithi070_191495_object30335:head,463'1889,11:b09d96a1:::benchmark_data_smithi070_191495_object70776:head,468'4418,11:b09e5948:::benchmark_data_smithi070_191495_object86463:head,474'5473,11:b09efd54:::benchmark_data_smithi070_191495_object16485:head,460'1045,11:b09f6cec:::benchmark_data_smithi070_191495_object96230:head,478'6098,11:b0a0a83f:::benchmark_data_smithi070_191495_object88212:head,474'5593,11:b0a180e7:::benchmark_data_smithi070_191495_object96963:head,479'6159,11:b0a25d93:::benchmark_data_smithi070_191495_object58196:head,466'3660,11:b0a27019:::benchmark_data_smithi070_191495_object36806:head,464'2297,11:b0a28f39:::benchmark_data_smithi070_191495_object101635:head,479'6455,11:b0a3368c:::benchmark_data_smithi070_191495_object105182:head,481'6679,11:b0a3e683:::benchmark_data_smithi070_191495_object47149:head,464'2949,11:b0a4c42e:::benchmark_data_smithi070_191495_object69086:head,468'4313,11:b0a50dc8:::benchmark_data_smithi070_191495_object56034:head,466'3526,11:b0a55b58:::benchmark_data_smithi070_191495_object28293:head,463'1765,11:b0a5eb9d:::benchmark_data_smithi070_191495_object107175:head,481'6789,11:b0a5f1af:::benchmark_data_smithi070_191495_object77079:head,471'4868,11:b0a5f46a:::benchmark_data_smithi070_191495_object40867:head,464'2547,11:b0a698a5:::benchmark_data_smithi070_191495_object125090:head,490'7939,11:b0a6ca0e:::benchmark_data_smithi070_191495_object56895:head,466'3580,11:b0a7172b:::benchmark_data_smithi070_191495_object114322:head,486'7250,11:b0a734ba:::benchmark_data_smithi070_191495_object77204:head,471'4879,11:b0a74481:::benchmark_data_smithi070_191495_object99864:head,479'6341,11:b0a7a074:::benchmark_data_smithi070_191495_object108893:head,483'6904,11:b0a8d80f:::benchmark_data_smithi070_191495_object126581:head,491'8029,11:b0a967f8:::benchmark_data_smithi070_191495_object51666:head,465'3244,11:b0a993ca:::benchmark_data_smithi070_191495_object89492:head,475'5666,11:b0a9ddff:::benchmark_data_smithi070_191495_object84639:head,474'5339,11:b0aa5664:::benchmark_data_smithi070_191495_object68443:head,468'4267,11:b0aa8acb:::benchmark_data_smithi070_191495_object46724:head,464'2908,11:b0ab2b1d:::benchmark_data_smithi070_191495_object61724:head,467'3871,11:b0aba63b:::benchmark_data_smithi070_191495_object119248:head,488'7569,11:b0aba955:::benchmark_data_smithi070_191495_object96547:head,479'6124,11:b0abb746:::benchmark_data_smithi070_191495_object67759:head,468'4236,11:b0abc895:::benchmark_data_smithi070_191495_object80694:head,473'5105,11:b0abef31:::benchmark_data_smithi070_191495_object97561:head,479'6199,11:b0abfeb1:::benchmark_data_smithi070_191495_object90830:head,476'5747,11:b0ac712a:::benchmark_data_smithi070_191495_object60289:head,467'3770,11:b0ad89b7:::benchmark_data_smithi070_191495_object121473:head,488'7697,11:b0ad9ad1:::benchmark_data_smithi070_191495_object17103:head,460'1096,11:b0ae292c:::benchmark_data_smithi070_191495_object36848:head,464'2305,11:b0ae4504:::benchmark_data_smithi070_191495_object69046:head,468'4310,11:b0ae4d90:::benchmark_data_smithi070_191495_object93209:head,477'5918,11:b0aea670:::benchmark_data_smithi070_191495_object25567:head,463'1601,11:b0aecd93:::benchmark_data_smithi070_191495_object47291:head,464'2957,11:b0af38f6:::benchmark_data_smithi070_191495_object92598:head,476'5875,11:b0b164d4:::benchmark_data_smithi070_191495_object86287:head,474'5458,11:b0b1e1f2:::benchmark_data_smithi070_191495_object75339:head,470'4727,11:b0b32377:::benchmark_data_smithi070_191495_object47836:head,464'2995,11:b0b325e6:::benchmark_data_smithi070_191495_object17815:head,460'1143,11:b0b35a6c:::benchmark_data_smithi070_191495_object56827:head,466'3574,11:b0b35b12:::benchmark_data_smithi070_191495_object76515:head,471'4818,11:b0b36347:::benchmark_data_smithi070_191495_object115598:head,486'7322,11:b0b3a38b:::benchmark_data_smithi070_191495_object18787:head,460'1206,11:b0b70067:::benchmark_data_smithi070_191495_object63654:head,467'3984,11:b0b71b2f:::benchmark_data_smithi070_191495_object113829:head,486'7222,11:b0b8ba3a:::benchmark_data_smithi070_191495_object97054:head,479'6167,11:b0b8f957:::benchmark_data_smithi070_191495_object52550:head,466'3298,11:b0b988ba:::benchmark_data_smithi070_191495_object89624:head,475'5676,11:b0b989e4:::benchmark_data_smithi070_191495_object24680:head,462'1559,11:b0b9b3c9:::benchmark_data_smithi070_191495_object125859:head,490'7983,11:b0bab9db:::benchmark_data_smithi070_191495_object66745:head,468'4171,11:b0bacba2:::benchmark_data_smithi070_191495_object85546:head,474'5408,11:b0bb22ec:::benchmark_data_smithi070_191495_object63149:head,467'3951,11:b0bc30fd:::benchmark_data_smithi070_191495_object26816:head,463'1678,11:b0be03d0:::benchmark_data_smithi070_191495_object130233:head,492'8254,11:b0beea6d:::benchmark_data_smithi070_191495_object20637:head,460'1315,11:b0bf079e:::benchmark_data_smithi070_191495_object70986:head,468'4432,11:b0bfa060:::benchmark_data_smithi070_191495_object57496:head,466'3619,11:b0c0bba4:::benchmark_data_smithi070_191495_object93858:head,477'5946,11:b0c0f0dd:::benchmark_data_smithi070_191495_object51412:head,465'3230,11:b0c1b875:::benchmark_data_smithi070_191495_object80395:head,472'5085,11:b0c1fc29:::benchmark_data_smithi070_191495_object48544:head,465'3044,11:b0c28bf6:::benchmark_data_smithi070_191495_object65982:head,467'4135,11:b0c2d7f3:::benchmark_data_smithi070_191495_object34178:head,464'2128,11:b0c30cb7:::benchmark_data_smithi070_191495_object105763:head,481'6703,11:b0c3ee95:::benchmark_data_smithi070_191495_object109684:head,483'6956,11:b0c54d8e:::benchmark_data_smithi070_191495_object60634:head,467'3795,11:b0c61efe:::benchmark_data_smithi070_191495_object41778:head,464'2601,11:b0c79527:::benchmark_data_smithi070_191495_object38552:head,464'2405,11:b0c84bd4:::benchmark_data_smithi070_191495_object18833:head,460'1209,11:b0c8a291:::benchmark_data_smithi070_191495_object74126:head,469'4636,11:b0c8b98a:::benchmark_data_smithi070_191495_object22974:head,462'1453,11:b0c8c39f:::benchmark_data_smithi070_191495_object20583:head,460'1309,11:b0c97f7d:::benchmark_data_smithi070_191495_object45555:head,464'2841,11:b0c9ff94:::benchmark_data_smithi070_191495_object123696:head,489'7846,11:b0ca807a:::benchmark_data_smithi070_191495_object36599:head,464'2283,11:b0ca9ff1:::benchmark_data_smithi070_191495_object107954:head,482'6837,11:b0ccb7fd:::benchmark_data_smithi070_191495_object50038:head,465'3138,11:b0ccee61:::benchmark_data_smithi070_191495_object96409:head,478'6111,11:b0cd6450:::benchmark_data_smithi070_191495_object16245:head,460'1029,11:b0cdb54f:::benchmark_data_smithi070_191495_object90000:head,475'5702,11:b0cf91dc:::benchmark_data_smithi070_191495_object111534:head,484'7063,11:b0d066f7:::benchmark_data_smithi070_191495_object32048:head,463'1988,11:b0d0c75c:::benchmark_data_smithi070_191495_object23539:head,462'1481,11:b0d13165:::benchmark_data_smithi070_191495_object54356:head,466'3413,11:b0d21965:::benchmark_data_smithi070_191495_object71835:head,468'4486,11:b0d25944:::benchmark_data_smithi070_191495_object31730:head,463'1967,11:b0d3a7d2:::benchmark_data_smithi070_191495_object125394:head,490'7955,11:b0d41dd7:::benchmark_data_smithi070_191495_object74731:head,470'4683,11:b0d4898b:::benchmark_data_smithi070_191495_object94224:head,478'5977,11:b0d4c0e0:::benchmark_data_smithi070_191495_object22320:head,461'1410,11:b0d4cb81:::benchmark_data_smithi070_191495_object107262:head,481'6793,11:b0d51639:::benchmark_data_smithi070_191495_object49557:head,465'3115,11:b0d517c1:::benchmark_data_smithi070_191495_object99422:head,479'6310,11:b0d676f4:::benchmark_data_smithi070_191495_object45579:head,464'2843,11:b0d6d6f2:::benchmark_data_smithi070_191495_object36182:head,464'2249,11:b0d784a4:::benchmark_data_smithi070_191495_object72486:head,468'4534,11:b0d7aad2:::benchmark_data_smithi070_191495_object52145:head,465'3272,11:b0d7f521:::benchmark_data_smithi070_191495_object16760:head,460'1064,11:b0d9c6e1:::benchmark_data_smithi070_191495_object35441:head,464'2206,11:b0d9c852:::benchmark_data_smithi070_191495_object51208:head,465'3213,11:b0da2e6a:::benchmark_data_smithi070_191495_object44336:head,464'2771,11:b0da869a:::benchmark_data_smithi070_191495_object69140:head,468'4317,11:b0dcb6d9:::benchmark_data_smithi070_191495_object92774:head,476'5892,11:b0dd67c0:::benchmark_data_smithi070_191495_object44080:head,464'2746,11:b0ddcf30:::benchmark_data_smithi070_191495_object36260:head,464'2255,11:b0def89b:::benchmark_data_smithi070_191495_object93430:head,477'5929,11:b0e03076:::benchmark_data_smithi070_191495_object30941:head,463'1918,11:b0e0b883:::benchmark_data_smithi070_191495_object95257:head,478'6032,11:b0e17f4c:::benchmark_data_smithi070_191495_object119255:head,488'7570,11:b0e1e319:::benchmark_data_smithi070_191495_object108154:head,482'6852,11:b0e20f2c:::benchmark_data_smithi070_191495_object90388:head,475'5724,11:b0e23d84:::benchmark_data_smithi070_191495_object31159:head,463'1937,11:b0e2ef12:::benchmark_data_smithi070_191495_object117969:head,486'7476,11:b0e31f50:::benchmark_data_smithi070_191495_object93920:head,477'5951,11:b0e36c98:::benchmark_data_smithi070_191495_object47824:head,464'2993,11:b0e3ef1c:::benchmark_data_smithi070_191495_object64173:head,467'4019,11:b0e46fba:::benchmark_data_smithi070_191495_object45407:head,464'2831,11:b0e493e9:::benchmark_data_smithi070_191495_object120626:head,488'7643,11:b0e4c1a0:::benchmark_data_smithi070_191495_object36847:head,464'2304,11:b0e4e149:::benchmark_data_smithi070_191495_object67779:head,468'4238,11:b0e4e9b1:::benchmark_data_smithi070_191495_object75856:head,471'4766,11:b0e766c9:::benchmark_data_smithi070_191495_object96668:head,479'6138,11:b0e8384f:::benchmark_data_smithi070_191495_object61886:head,467'3879,11:b0e8ed26:::benchmark_data_smithi070_191495_object110329:head,483'6990,11:b0ea1d66:::benchmark_data_smithi070_191495_object72123:head,468'4506,11:b0ebd9b0:::benchmark_data_smithi070_191495_object108981:head,483'6907,11:b0ec124f:::benchmark_data_smithi070_191495_object90469:head,476'5727,11:b0ed3831:::benchmark_data_smithi070_191495_object52353:head,465'3282,11:b0ee622b:::benchmark_data_smithi070_191495_object73614:head,469'4607,11:b0efdc45:::benchmark_data_smithi070_191495_object103788:head,480'6593,11:b0f06164:::benchmark_data_smithi070_191495_object22814:head,462'1441,11:b0f07209:::benchmark_data_smithi070_191495_object69105:head,468'4315,11:b0f11b80:::benchmark_data_smithi070_191495_object42027:head,464'2621,11:b0f1201b:::benchmark_data_smithi070_191495_object70176:head,468'4379,11:b0f12216:::benchmark_data_smithi070_191495_object109770:head,483'6962,11:b0f127f8:::benchmark_data_smithi070_191495_object39021:head,464'2437,11:b0f1486f:::benchmark_data_smithi070_191495_object89560:head,475'5671,11:b0f18d24:::benchmark_data_smithi070_191495_object104769:head,480'6651,11:b0f19610:::benchmark_data_smithi070_191495_object82261:head,474'5207,11:b0f19a11:::benchmark_data_smithi070_191495_object84061:head,474'5309,11:b0f2861d:::benchmark_data_smithi070_191495_object74247:head,470'4649,11:b0f3be61:::benchmark_data_smithi070_191495_object82803:head,474'5242,11:b0f438ae:::benchmark_data_smithi070_191495_object86585:head,474'5482,11:b0f4af82:::benchmark_data_smithi070_191495_object44199:head,464'2755,11:b0f54859:::benchmark_data_smithi070_191495_object57575:head,466'3624,11:b0f628d1:::benchmark_data_smithi070_191495_object86462:head,474'5472,11:b0f62ca0:::benchmark_data_smithi070_191495_object91560:head,476'5789,11:b0f747cd:::benchmark_data_smithi070_191495_object17028:head,460'1088,11:b0f74e0e:::benchmark_data_smithi070_191495_object82997:head,474'5255,11:b0f78322:::benchmark_data_smithi070_191495_object66585:head,468'4160,11:b0f7f7cd:::benchmark_data_smithi070_191495_object116464:head,486'7381,11:b0f84fbc:::benchmark_data_smithi070_191495_object54254:head,466'3406,11:b0f898fd:::benchmark_data_smithi070_191495_object63271:head,467'3958,11:b0fa160e:::benchmark_data_smithi070_191495_object29836:head,463'1862,11:b0fa357f:::benchmark_data_smithi070_191495_object80282:head,472'5081,11:b0fae940:::benchmark_data_smithi070_191495_object72702:head,468'4549,11:b0faee7a:::benchmark_data_smithi070_191495_object25008:head,462'1578,11:b0fb1f3b:::benchmark_data_smithi070_191495_object120118:head,488'7614,11:b0fbc8c4:::benchmark_data_smithi070_191495_object128191:head,491'8127,11:b0fcb4f7:::benchmark_data_smithi070_191495_object103525:head,480'6571,11:b0fdbec1:::benchmark_data_smithi070_191495_object52686:head,466'3312,11:b0fe6ff7:::benchmark_data_smithi070_191495_object85356:head,474'5386,11:b0fe883c:::benchmark_data_smithi070_191495_object37129:head,464'2323,11:b0fe98c1:::benchmark_data_smithi070_191495_object63965:head,467'4003,11:b0ff817d:::benchmark_data_smithi070_191495_object104864:head,480'6661,11:b10030e2:::benchmark_data_smithi070_191495_object73661:head,469'4613,11:b1011c8d:::benchmark_data_smithi070_191495_object54030:head,466'3391,11:b101bacd:::benchmark_data_smithi070_191495_object53482:head,466'3354,11:b103b6bf:::benchmark_data_smithi070_191495_object45242:head,464'2822,11:b103f774:::benchmark_data_smithi070_191495_object53228:head,466'3347,11:b1062224:::benchmark_data_smithi070_191495_object91549:head,476'5787,11:b10658f8:::benchmark_data_smithi070_191495_object20627:head,460'1314,11:b1066362:::benchmark_data_smithi070_191495_object80292:head,472'5082,11:b1067157:::benchmark_data_smithi070_191495_object45898:head,464'2867,11:b1067ed9:::benchmark_data_smithi070_191495_object104847:head,480'6659,11:b106c2d1:::benchmark_data_smithi070_191495_object56786:head,466'3568,11:b106c8f9:::benchmark_data_smithi070_191495_object17149:head,460'1098,11:b10790fb:::benchmark_data_smithi070_191495_object37216:head,464'2330,11:b107e5ee:::benchmark_data_smithi070_191495_object127488:head,491'8086,11:b1087ef0:::benchmark_data_smithi070_191495_object17561:head,460'1128,11:b108aec3:::benchmark_data_smithi070_191495_object91435:head,476'5782,11:b108c657:::benchmark_data_smithi070_191495_object116685:head,486'7394,11:b10a627f:::benchmark_data_smithi070_191495_object92228:head,476'5844,11:b10b2440:::benchmark_data_smithi070_191495_object64976:head,467'4077,11:b10baf9e:::benchmark_data_smithi070_191495_object91969:head,476'5824,11:b10bb7a4:::benchmark_data_smithi070_191495_object77413:head,471'4896,11:b10c190f:::benchmark_data_smithi070_191495_object36580:head,464'2282,11:b10db882:::benchmark_data_smithi070_191495_object42448:head,464'2658,11:b10df322:::benchmark_data_smithi070_191495_object66524:head,468'4155,11:b10e48f3:::benchmark_data_smithi070_191495_object43282:head,464'2708,11:b10e7119:::benchmark_data_smithi070_191495_object49409:head,465'3101,11:b10f5a8b:::benchmark_data_smithi070_191495_object65376:head,467'4104,11:b10f63dd:::benchmark_data_smithi070_191495_object18847:head,460'1210,11:b10f8468:::benchmark_data_smithi070_191495_object123936:head,489'7865,11:b110077f:::benchmark_data_smithi070_191495_object68921:head,468'4296,11:b1101199:::benchmark_data_smithi070_191495_object112755:head,485'7155,11:b11032cb:::benchmark_data_smithi070_191495_object121296:head,488'7685,11:b1104e39:::benchmark_data_smithi070_191495_object68285:head,468'4262,11:b110e463:::benchmark_data_smithi070_191495_object88310:head,474'5597,11:b110e5f9:::benchmark_data_smithi070_191495_object44890:head,464'2805,11:b1111e69:::benchmark_data_smithi070_191495_object100255:head,479'6365,11:b1112813:::benchmark_data_smithi070_191495_object98929:head,479'6280,11:b1119ca1:::benchmark_data_smithi070_191495_object22307:head,461'1407,11:b11290aa:::benchmark_data_smithi070_191495_object93925:head,477'5952,11:b1134ba4:::benchmark_data_smithi070_191495_object80247:head,472'5079)", 2024-09-16T09:44:44.651 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:44:25.827838+0000", 2024-09-16T09:44:44.651 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 18.804194772999999, 2024-09-16T09:44:44.652 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.011927536000000001, 2024-09-16T09:44:44.652 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:44:44.652 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:44:44.652 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T09:44:44.652 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:44:44.652 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.652 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:44:44.652 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:25.827838+0000", 2024-09-16T09:44:44.652 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:44:44.652 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.652 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.652 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:44:44.652 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:25.827838+0000", 2024-09-16T09:44:44.653 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:44:44.653 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.653 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.653 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:44:44.653 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:25.827839+0000", 2024-09-16T09:44:44.653 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5199999999999997e-07 2024-09-16T09:44:44.653 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.653 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.653 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:44:44.653 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:25.828121+0000", 2024-09-16T09:44:44.653 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00028167300000000003 2024-09-16T09:44:44.654 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.654 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.654 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:44:44.654 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:25.828585+0000", 2024-09-16T09:44:44.654 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00046415999999999997 2024-09-16T09:44:44.654 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.654 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.654 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:44:44.654 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:25.829513+0000", 2024-09-16T09:44:44.654 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00092781600000000004 2024-09-16T09:44:44.655 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.655 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.655 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:44:44.655 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:25.836800+0000", 2024-09-16T09:44:44.655 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0072872859999999996 2024-09-16T09:44:44.655 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.655 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.655 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:44:44.655 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:25.837758+0000", 2024-09-16T09:44:44.655 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00095769900000000001 2024-09-16T09:44:44.655 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.655 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.655 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:44:44.656 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:25.839766+0000", 2024-09-16T09:44:44.656 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0020083499999999999 2024-09-16T09:44:44.656 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:44:44.656 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:44:44.656 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:44:44.656 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.656 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.656 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "pg_scan(get_digest 11.d 11:b48f3090:::benchmark_data_smithi070_191495_object28461:head-MIN e 518/517)", 2024-09-16T09:44:44.656 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:44:26.153173+0000", 2024-09-16T09:44:44.656 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 18.478859618000001, 2024-09-16T09:44:44.656 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.039027994000000003, 2024-09-16T09:44:44.656 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:44:44.656 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:44:44.657 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T09:44:44.657 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:44:44.657 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.657 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:44:44.657 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.153173+0000", 2024-09-16T09:44:44.657 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:44:44.657 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.657 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.657 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:44:44.657 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.153173+0000", 2024-09-16T09:44:44.657 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:44:44.657 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.657 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.658 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:44:44.658 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.153175+0000", 2024-09-16T09:44:44.658 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.311e-06 2024-09-16T09:44:44.658 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.658 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.658 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:44:44.658 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.153184+0000", 2024-09-16T09:44:44.658 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.4550000000000001e-06 2024-09-16T09:44:44.658 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.658 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.658 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:44:44.658 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.153198+0000", 2024-09-16T09:44:44.658 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.3613e-05 2024-09-16T09:44:44.659 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.659 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.659 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:44:44.659 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.153206+0000", 2024-09-16T09:44:44.659 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.1890000000000007e-06 2024-09-16T09:44:44.659 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.659 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.659 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:44:44.659 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.153264+0000", 2024-09-16T09:44:44.659 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.8180999999999997e-05 2024-09-16T09:44:44.659 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.659 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.659 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:44:44.659 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.153274+0000", 2024-09-16T09:44:44.660 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.7550000000000006e-06 2024-09-16T09:44:44.660 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.660 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.660 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:44:44.660 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.192201+0000", 2024-09-16T09:44:44.660 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.038927490000000002 2024-09-16T09:44:44.660 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:44:44.660 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:44:44.660 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:44:44.660 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.660 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.661 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "backfill_remove(11.d e518 11:b36a50d2:::benchmark_data_smithi070_191495_object118636:head,487'7522,11:b36b3469:::benchmark_data_smithi070_191495_object109175:head,483'6918,11:b36b403f:::benchmark_data_smithi070_191495_object52502:head,466'3292,11:b36b48d0:::benchmark_data_smithi070_191495_object67119:head,468'4191,11:b36bc3c8:::benchmark_data_smithi070_191495_object19283:head,460'1238,11:b36cb92c:::benchmark_data_smithi070_191495_object42420:head,464'2655,11:b36cfc52:::benchmark_data_smithi070_191495_object16718:head,460'1059,11:b36d9c17:::benchmark_data_smithi070_191495_object69281:head,468'4326,11:b36e7cb6:::benchmark_data_smithi070_191495_object91079:head,476'5758,11:b36e834a:::benchmark_data_smithi070_191495_object46819:head,464'2920,11:b36ee1a3:::benchmark_data_smithi070_191495_object29052:head,463'1809,11:b36efc65:::benchmark_data_smithi070_191495_object105453:head,481'6691,11:b36ff946:::benchmark_data_smithi070_191495_object109084:head,483'6911,11:b37038d4:::benchmark_data_smithi070_191495_object46757:head,464'2912,11:b3703b3f:::benchmark_data_smithi070_191495_object58815:head,466'3694,11:b3706f4f:::benchmark_data_smithi070_191495_object54301:head,466'3412,11:b372ee6a:::benchmark_data_smithi070_191495_object50722:head,465'3187,11:b3735f33:::benchmark_data_smithi070_191495_object85548:head,474'5409,11:b373a36c:::benchmark_data_smithi070_191495_object21098:head,461'1337,11:b37482ba:::benchmark_data_smithi070_191495_object54215:head,466'3404,11:b375142b:::benchmark_data_smithi070_191495_object77113:head,471'4871,11:b3755429:::benchmark_data_smithi070_191495_object53208:head,466'3344,11:b375f935:::benchmark_data_smithi070_191495_object82936:head,474'5251,11:b3777b1c:::benchmark_data_smithi070_191495_object118882:head,487'7543,11:b3788a62:::benchmark_data_smithi070_191495_object66204:head,467'4143,11:b378ae49:::benchmark_data_smithi070_191495_object117905:head,486'7468,11:b37b49b8:::benchmark_data_smithi070_191495_object118211:head,487'7495,11:b37d8e5e:::benchmark_data_smithi070_191495_object128788:head,491'8168,11:b37ddf70:::benchmark_data_smithi070_191495_object76755:head,471'4835,11:b37e9e22:::benchmark_data_smithi070_191495_object27674:head,463'1729,11:b37fd910:::benchmark_data_smithi070_191495_object47004:head,464'2934,11:b3801a1c:::benchmark_data_smithi070_191495_object110116:head,483'6978,11:b380b285:::benchmark_data_smithi070_191495_object27573:head,463'1722,11:b380d94b:::benchmark_data_smithi070_191495_object42802:head,464'2678,11:b381624d:::benchmark_data_smithi070_191495_object110581:head,483'7009,11:b38193c5:::benchmark_data_smithi070_191495_object100816:head,479'6400,11:b3819e14:::benchmark_data_smithi070_191495_object28390:head,463'1772,11:b381e994:::benchmark_data_smithi070_191495_object126491:head,491'8021,11:b382900e:::benchmark_data_smithi070_191495_object50097:head,465'3145,11:b382ae84:::benchmark_data_smithi070_191495_object109576:head,483'6949,11:b382be6f:::benchmark_data_smithi070_191495_object74877:head,470'4695,11:b38338df:::benchmark_data_smithi070_191495_object16691:head,460'1057,11:b383698c:::benchmark_data_smithi070_191495_object68407:head,468'4265,11:b38467ac:::benchmark_data_smithi070_191495_object71968:head,468'4496,11:b384c4fd:::benchmark_data_smithi070_191495_object77374:head,471'4893,11:b3850966:::benchmark_data_smithi070_191495_object54107:head,466'3398,11:b385ce2e:::benchmark_data_smithi070_191495_object84477:head,474'5333,11:b385e17e:::benchmark_data_smithi070_191495_object28572:head,463'1781,11:b386b712:::benchmark_data_smithi070_191495_object24686:head,462'1560,11:b386efa0:::benchmark_data_smithi070_191495_object68812:head,468'4290,11:b3870df5:::benchmark_data_smithi070_191495_object61652:head,467'3866,11:b3892135:::benchmark_data_smithi070_191495_object87548:head,474'5549,11:b38aac1f:::benchmark_data_smithi070_191495_object37870:head,464'2365,11:b38bbb21:::benchmark_data_smithi070_191495_object124319:head,489'7898,11:b38bd622:::benchmark_data_smithi070_191495_object46951:head,464'2931,11:b38e766d:::benchmark_data_smithi070_191495_object117044:head,486'7417,11:b38e8d18:::benchmark_data_smithi070_191495_object85517:head,474'5407,11:b38ea8a9:::benchmark_data_smithi070_191495_object59652:head,466'3736,11:b38f684e:::benchmark_data_smithi070_191495_object84806:head,474'5350,11:b38ff581:::benchmark_data_smithi070_191495_object49858:head,465'3130,11:b39219d8:::benchmark_data_smithi070_191495_object91954:head,476'5821,11:b3927bf7:::benchmark_data_smithi070_191495_object54911:head,466'3453,11:b392cfab:::benchmark_data_smithi070_191495_object67528:head,468'4220,11:b39303e5:::benchmark_data_smithi070_191495_object31603:head,463'1958,11:b393e092:::benchmark_data_smithi070_191495_object29010:head,463'1806,11:b393e51f:::benchmark_data_smithi070_191495_object88651:head,474'5614,11:b3946018:::benchmark_data_smithi070_191495_object80671:head,473'5104,11:b3951bc2:::benchmark_data_smithi070_191495_object121134:head,488'7679,11:b395f608:::benchmark_data_smithi070_191495_object74314:head,470'4656,11:b3962f91:::benchmark_data_smithi070_191495_object16227:head,460'1028,11:b3969fa5:::benchmark_data_smithi070_191495_object40492:head,464'2525,11:b397b7d8:::benchmark_data_smithi070_191495_object113438:head,486'7199,11:b397cef0:::benchmark_data_smithi070_191495_object41748:head,464'2599,11:b39811ea:::benchmark_data_smithi070_191495_object37804:head,464'2359,11:b3982967:::benchmark_data_smithi070_191495_object121894:head,488'7729,11:b398ff0d:::benchmark_data_smithi070_191495_object43222:head,464'2701,11:b3991877:::benchmark_data_smithi070_191495_object64594:head,467'4047,11:b399b357:::benchmark_data_smithi070_191495_object81895:head,474'5189,11:b39a6701:::benchmark_data_smithi070_191495_object88877:head,474'5629,11:b39ae086:::benchmark_data_smithi070_191495_object128516:head,491'8147,11:b39aea9f:::benchmark_data_smithi070_191495_object91110:head,476'5760,11:b39b2fd0:::benchmark_data_smithi070_191495_object73630:head,469'4610,11:b39b9e28:::benchmark_data_smithi070_191495_object38076:head,464'2381,11:b39c95a5:::benchmark_data_smithi070_191495_object16925:head,460'1076,11:b39d2103:::benchmark_data_smithi070_191495_object71075:head,468'4438,11:b39dd3d6:::benchmark_data_smithi070_191495_object22836:head,462'1443,11:b39de3e0:::benchmark_data_smithi070_191495_object90338:head,475'5720,11:b3a01973:::benchmark_data_smithi070_191495_object87039:head,474'5515,11:b3a0a0dc:::benchmark_data_smithi070_191495_object123387:head,488'7826,11:b3a119bd:::benchmark_data_smithi070_191495_object101259:head,479'6430,11:b3a17172:::benchmark_data_smithi070_191495_object93251:head,477'5921,11:b3a17816:::benchmark_data_smithi070_191495_object107460:head,482'6809,11:b3a28667:::benchmark_data_smithi070_191495_object23282:head,462'1469,11:b3a33d73:::benchmark_data_smithi070_191495_object49101:head,465'3086,11:b3a366ec:::benchmark_data_smithi070_191495_object42381:head,464'2652,11:b3a45e19:::benchmark_data_smithi070_191495_object85949:head,474'5438,11:b3a462a4:::benchmark_data_smithi070_191495_object64765:head,467'4066,11:b3a4ebb0:::benchmark_data_smithi070_191495_object34269:head,464'2137,11:b3a528ca:::benchmark_data_smithi070_191495_object31074:head,463'1929,11:b3a54458:::benchmark_data_smithi070_191495_object37959:head,464'2373,11:b3a57864:::benchmark_data_smithi070_191495_object53067:head,466'3334,11:b3a5a623:::benchmark_data_smithi070_191495_object112867:head,485'7162,11:b3a63105:::benchmark_data_smithi070_191495_object125198:head,490'7944,11:b3a64fcf:::benchmark_data_smithi070_191495_object117784:head,486'7461,11:b3a80522:::benchmark_data_smithi070_191495_object68997:head,468'4304,11:b3a8885c:::benchmark_data_smithi070_191495_object43438:head,464'2717,11:b3a8a3a9:::benchmark_data_smithi070_191495_object117720:head,486'7458,11:b3a9a9fe:::benchmark_data_smithi070_191495_object39134:head,464'2443,11:b3aa776e:::benchmark_data_smithi070_191495_object40448:head,464'2520,11:b3aad650:::benchmark_data_smithi070_191495_object98151:head,479'6229,11:b3aaea14:::benchmark_data_smithi070_191495_object98917:head,479'6279,11:b3ab67be:::benchmark_data_smithi070_191495_object116826:head,486'7404,11:b3ac43dc:::benchmark_data_smithi070_191495_object30009:head,463'1877,11:b3ac52fa:::benchmark_data_smithi070_191495_object96611:head,479'6130,11:b3adf494:::benchmark_data_smithi070_191495_object41244:head,464'2565,11:b3b03f1d:::benchmark_data_smithi070_191495_object50345:head,465'3159,11:b3b1566d:::benchmark_data_smithi070_191495_object49718:head,465'3123,11:b3b61ae9:::benchmark_data_smithi070_191495_object96345:head,478'6105,11:b3b6a615:::benchmark_data_smithi070_191495_object51695:head,465'3247,11:b3b6af15:::benchmark_data_smithi070_191495_object30244:head,463'1884,11:b3b70c5a:::benchmark_data_smithi070_191495_object60258:head,467'3767,11:b3b797bf:::benchmark_data_smithi070_191495_object76216:head,471'4796,11:b3b7e25c:::benchmark_data_smithi070_191495_object102945:head,480'6532,11:b3b83d52:::benchmark_data_smithi070_191495_object86628:head,474'5484,11:b3b8610f:::benchmark_data_smithi070_191495_object84810:head,474'5351,11:b3b8f95a:::benchmark_data_smithi070_191495_object48146:head,465'3017,11:b3b974ea:::benchmark_data_smithi070_191495_object44437:head,464'2780,11:b3b99b8f:::benchmark_data_smithi070_191495_object46813:head,464'2918,11:b3ba2fdb:::benchmark_data_smithi070_191495_object111411:head,484'7048,11:b3ba7722:::benchmark_data_smithi070_191495_object54178:head,466'3403,11:b3ba8103:::benchmark_data_smithi070_191495_object99385:head,479'6307,11:b3bb1bc5:::benchmark_data_smithi070_191495_object127187:head,491'8064,11:b3bc6b0f:::benchmark_data_smithi070_191495_object60355:head,467'3776,11:b3bcf397:::benchmark_data_smithi070_191495_object113657:head,486'7211,11:b3bcf4e3:::benchmark_data_smithi070_191495_object96206:head,478'6093,11:b3bcff99:::benchmark_data_smithi070_191495_object25512:head,463'1598,11:b3bd2e02:::benchmark_data_smithi070_191495_object50221:head,465'3152,11:b3bdabd0:::benchmark_data_smithi070_191495_object97357:head,479'6183,11:b3be0a77:::benchmark_data_smithi070_191495_object93887:head,477'5948,11:b3be8a0f:::benchmark_data_smithi070_191495_object18535:head,460'1186,11:b3beac77:::benchmark_data_smithi070_191495_object34252:head,464'2135,11:b3c07958:::benchmark_data_smithi070_191495_object38384:head,464'2396,11:b3c0be86:::benchmark_data_smithi070_191495_object104131:head,480'6612,11:b3c0d2c9:::benchmark_data_smithi070_191495_object108863:head,483'6902,11:b3c0deaf:::benchmark_data_smithi070_191495_object18123:head,460'1162,11:b3c13780:::benchmark_data_smithi070_191495_object76825:head,471'4843,11:b3c30887:::benchmark_data_smithi070_191495_object82446:head,474'5218,11:b3c326a2:::benchmark_data_smithi070_191495_object85453:head,474'5395,11:b3c3b48f:::benchmark_data_smithi070_191495_object66633:head,468'4163,11:b3c40eba:::benchmark_data_smithi070_191495_object116983:head,486'7409,11:b3c42508:::benchmark_data_smithi070_191495_object109207:head,483'6923,11:b3c581bd:::benchmark_data_smithi070_191495_object74173:head,469'4641,11:b3c79496:::benchmark_data_smithi070_191495_object17764:head,460'1140,11:b3c7ca57:::benchmark_data_smithi070_191495_object55754:head,466'3504,11:b3c8418b:::benchmark_data_smithi070_191495_object40094:head,464'2498,11:b3c85d16:::benchmark_data_smithi070_191495_object59699:head,466'3740,11:b3c88ead:::benchmark_data_smithi070_191495_object101414:head,479'6443,11:b3c890aa:::benchmark_data_smithi070_191495_object74107:head,469'4634,11:b3c924f1:::benchmark_data_smithi070_191495_object109744:head,483'6961,11:b3ca388e:::benchmark_data_smithi070_191495_object87424:head,474'5542,11:b3ca654f:::benchmark_data_smithi070_191495_object34160:head,464'2126,11:b3cc4d6b:::benchmark_data_smithi070_191495_object81527:head,473'5158,11:b3cc874c:::benchmark_data_smithi070_191495_object97887:head,479'6214,11:b3ccb3d8:::benchmark_data_smithi070_191495_object51880:head,465'3257,11:b3cf7ec6:::benchmark_data_smithi070_191495_object90618:head,476'5734,11:b3cf82d0:::benchmark_data_smithi070_191495_object111194:head,484'7032,11:b3d07d9e:::benchmark_data_smithi070_191495_object51090:head,465'3209,11:b3d0d1ca:::benchmark_data_smithi070_191495_object45637:head,464'2850,11:b3d1abb9:::benchmark_data_smithi070_191495_object69880:head,468'4358,11:b3d500a3:::benchmark_data_smithi070_191495_object57461:head,466'3617,11:b3d57bb0:::benchmark_data_smithi070_191495_object49692:head,465'3121,11:b3d70fff:::benchmark_data_smithi070_191495_object76537:head,471'4820,11:b3d74e90:::benchmark_data_smithi070_191495_object45863:head,464'2862,11:b3d905d4:::benchmark_data_smithi070_191495_object80597:head,473'5098,11:b3d9729e:::benchmark_data_smithi070_191495_object54061:head,466'3394,11:b3d9d986:::benchmark_data_smithi070_191495_object99313:head,479'6303,11:b3d9dd70:::benchmark_data_smithi070_191495_object67057:head,468'4189,11:b3d9ea01:::benchmark_data_smithi070_191495_object74255:head,470'4651,11:b3daacf3:::benchmark_data_smithi070_191495_object82890:head,474'5250,11:b3dbcbcb:::benchmark_data_smithi070_191495_object51312:head,465'3223,11:b3dbe0aa:::benchmark_data_smithi070_191495_object106008:head,481'6719,11:b3dc297a:::benchmark_data_smithi070_191495_object91104:head,476'5759,11:b3dc32ee:::benchmark_data_smithi070_191495_object26163:head,463'1636,11:b3dcfada:::benchmark_data_smithi070_191495_object66162:head,467'4142,11:b3dd3ffd:::benchmark_data_smithi070_191495_object21604:head,461'1366,11:b3dd46f7:::benchmark_data_smithi070_191495_object103930:head,480'6606,11:b3de195f:::benchmark_data_smithi070_191495_object116419:head,486'7375,11:b3dea335:::benchmark_data_smithi070_191495_object38425:head,464'2398,11:b3defd3d:::benchmark_data_smithi070_191495_object123353:head,488'7824,11:b3dfd447:::benchmark_data_smithi070_191495_object118334:head,487'7506,11:b3e00255:::benchmark_data_smithi070_191495_object47710:head,464'2985,11:b3e00694:::benchmark_data_smithi070_191495_object103287:head,480'6553,11:b3e0b651:::benchmark_data_smithi070_191495_object81020:head,473'5125,11:b3e0e755:::benchmark_data_smithi070_191495_object22547:head,461'1421,11:b3e10b14:::benchmark_data_smithi070_191495_object103072:head,480'6545,11:b3e120bf:::benchmark_data_smithi070_191495_object21629:head,461'1369,11:b3e14596:::benchmark_data_smithi070_191495_object49469:head,465'3106,11:b3e17559:::benchmark_data_smithi070_191495_object19596:head,460'1265,11:b3e178dd:::benchmark_data_smithi070_191495_object31798:head,463'1971,11:b3e2842e:::benchmark_data_smithi070_191495_object92154:head,476'5840,11:b3e2f63f:::benchmark_data_smithi070_191495_object112265:head,485'7122,11:b3e3a57b:::benchmark_data_smithi070_191495_object101238:head,479'6428,11:b3e40274:::benchmark_data_smithi070_191495_object39274:head,464'2452,11:b3e46945:::benchmark_data_smithi070_191495_object81568:head,473'5164,11:b3e52145:::benchmark_data_smithi070_191495_object129962:head,491'8243,11:b3e566f4:::benchmark_data_smithi070_191495_object88171:head,474'5588,11:b3e765f6:::benchmark_data_smithi070_191495_object24323:head,462'1534,11:b3e7a498:::benchmark_data_smithi070_191495_object38561:head,464'2406,11:b3e80064:::benchmark_data_smithi070_191495_object88508:head,474'5607,11:b3e91f48:::benchmark_data_smithi070_191495_object36430:head,464'2275,11:b3e9722e:::benchmark_data_smithi070_191495_object82343:head,474'5212,11:b3eb6c76:::benchmark_data_smithi070_191495_object64608:head,467'4048,11:b3ebf3b8:::benchmark_data_smithi070_191495_object96573:head,479'6127,11:b3ec06e3:::benchmark_data_smithi070_191495_object81289:head,473'5144,11:b3ec6c44:::benchmark_data_smithi070_191495_object88669:head,474'5615,11:b3ec7f65:::benchmark_data_smithi070_191495_object92182:head,476'5842,11:b3eda824:::benchmark_data_smithi070_191495_object79140:head,472'4997,11:b3ee2a8f:::benchmark_data_smithi070_191495_object88851:head,474'5626,11:b3eecfbd:::benchmark_data_smithi070_191495_object69021:head,468'4308,11:b3ef6331:::benchmark_data_smithi070_191495_object74075:head,469'4630,11:b3f06cbb:::benchmark_data_smithi070_191495_object91798:head,476'5810,11:b3f105e1:::benchmark_data_smithi070_191495_object89751:head,475'5682,11:b3f3318c:::benchmark_data_smithi070_191495_object16896:head,460'1071,11:b3f3b3eb:::benchmark_data_smithi070_191495_object35582:head,464'2211,11:b3f3f9ac:::benchmark_data_smithi070_191495_object86562:head,474'5479,11:b3f40bae:::benchmark_data_smithi070_191495_object58453:head,466'3676,11:b3f417cd:::benchmark_data_smithi070_191495_object74940:head,470'4702,11:b3f45ba1:::benchmark_data_smithi070_191495_object67277:head,468'4203,11:b3f51bfa:::benchmark_data_smithi070_191495_object48540:head,465'3043,11:b3f53481:::benchmark_data_smithi070_191495_object23069:head,462'1458,11:b3f53e84:::benchmark_data_smithi070_191495_object115289:head,486'7303,11:b3f56ac4:::benchmark_data_smithi070_191495_object94751:head,478'6009,11:b3f5ddb6:::benchmark_data_smithi070_191495_object58925:head,466'3699,11:b3f5e7bf:::benchmark_data_smithi070_191495_object24013:head,462'1510,11:b3f616ca:::benchmark_data_smithi070_191495_object45580:head,464'2844,11:b3f6e8de:::benchmark_data_smithi070_191495_object116035:head,486'7353,11:b3f78fbb:::benchmark_data_smithi070_191495_object38790:head,464'2419,11:b3f83d9c:::benchmark_data_smithi070_191495_object54652:head,466'3435,11:b3f8813d:::benchmark_data_smithi070_191495_object92012:head,476'5827,11:b3f8f4fe:::benchmark_data_smithi070_191495_object46883:head,464'2923,11:b3f91214:::benchmark_data_smithi070_191495_object92755:head,476'5889,11:b3f94597:::benchmark_data_smithi070_191495_object35790:head,464'2226,11:b3faf755:::benchmark_data_smithi070_191495_object28853:head,463'1797,11:b3fb4baa:::benchmark_data_smithi070_191495_object73858:head,469'4615,11:b3fc001d:::benchmark_data_smithi070_191495_object121707:head,488'7713,11:b3fc18c3:::benchmark_data_smithi070_191495_object106241:head,481'6736,11:b3fcdd10:::benchmark_data_smithi070_191495_object129117:head,491'8188,11:b3fd7c42:::benchmark_data_smithi070_191495_object56385:head,466'3550,11:b3fd88dd:::benchmark_data_smithi070_191495_object39775:head,464'2482,11:b3fde07d:::benchmark_data_smithi070_191495_object56958:head,466'3588,11:b3fe23ba:::benchmark_data_smithi070_191495_object58236:head,466'3664,11:b3fe8972:::benchmark_data_smithi070_191495_object52920:head,466'3326,11:b3fe8f19:::benchmark_data_smithi070_191495_object96804:head,479'6148,11:b40000ca:::benchmark_data_smithi070_191495_object78090:head,472'4933,11:b400e05b:::benchmark_data_smithi070_191495_object69365:head,468'4331,11:b4010ad6:::benchmark_data_smithi070_191495_object98725:head,479'6264,11:b401c230:::benchmark_data_smithi070_191495_object43427:head,464'2715,11:b401d342:::benchmark_data_smithi070_191495_object47412:head,464'2969,11:b40219b4:::benchmark_data_smithi070_191495_object112672:head,485'7151,11:b402265d:::benchmark_data_smithi070_191495_object105844:head,481'6705,11:b402292a:::benchmark_data_smithi070_191495_object112787:head,485'7158,11:b402ad52:::benchmark_data_smithi070_191495_object45095:head,464'2811,11:b4033576:::benchmark_data_smithi070_191495_object63184:head,467'3953,11:b403751e:::benchmark_data_smithi070_191495_object74137:head,469'4637,11:b4039b34:::benchmark_data_smithi070_191495_object64781:head,467'4067,11:b403a8b1:::benchmark_data_smithi070_191495_object71604:head,468'4474,11:b4049149:::benchmark_data_smithi070_191495_object19068:head,460'1221,11:b405d8bf:::benchmark_data_smithi070_191495_object108422:head,483'6868,11:b406a2b1:::benchmark_data_smithi070_191495_object57960:head,466'3648,11:b4077abf:::benchmark_data_smithi070_191495_object65064:head,467'4086,11:b407f152:::benchmark_data_smithi070_191495_object18770:head,460'1204,11:b4080eda:::benchmark_data_smithi070_191495_object85250:head,474'5376,11:b408b30b:::benchmark_data_smithi070_191495_object43834:head,464'2739,11:b40937f3:::benchmark_data_smithi070_191495_object87078:head,474'5519,11:b4096089:::benchmark_data_smithi070_191495_object76371:head,471'4811,11:b40af703:::benchmark_data_smithi070_191495_object17995:head,460'1152,11:b40ba908:::benchmark_data_smithi070_191495_object129435:head,491'8208,11:b40c78ab:::benchmark_data_smithi070_191495_object110022:head,483'6976,11:b40c82f2:::benchmark_data_smithi070_191495_object118762:head,487'7532,11:b40d6a9d:::benchmark_data_smithi070_191495_object24799:head,462'1568,11:b40d7d28:::benchmark_data_smithi070_191495_object33043:head,463'2060,11:b40d7db7:::benchmark_data_smithi070_191495_object90781:head,476'5744,11:b40d89ab:::benchmark_data_smithi070_191495_object38268:head,464'2390,11:b40e3cfa:::benchmark_data_smithi070_191495_object32072:head,463'1990,11:b40e848e:::benchmark_data_smithi070_191495_object44692:head,464'2793,11:b40edd8a:::benchmark_data_smithi070_191495_object47008:head,464'2935,11:b410028d:::benchmark_data_smithi070_191495_object128877:head,491'8175,11:b4103ded:::benchmark_data_smithi070_191495_object104510:head,480'6633,11:b4120221:::benchmark_data_smithi070_191495_object83920:head,474'5303,11:b412799a:::benchmark_data_smithi070_191495_object121415:head,488'7694,11:b412ffc6:::benchmark_data_smithi070_191495_object44129:head,464'2751,11:b4135e73:::benchmark_data_smithi070_191495_object48066:head,465'3011,11:b4138a9d:::benchmark_data_smithi070_191495_object128326:head,491'8135,11:b4143e1b:::benchmark_data_smithi070_191495_object77988:head,471'4930,11:b415c61e:::benchmark_data_smithi070_191495_object72789:head,468'4554,11:b416280b:::benchmark_data_smithi070_191495_object29539:head,463'1839,11:b4178c7f:::benchmark_data_smithi070_191495_object70827:head,468'4421,11:b417b353:::benchmark_data_smithi070_191495_object19672:head,460'1268,11:b4184cee:::benchmark_data_smithi070_191495_object50561:head,465'3175,11:b4196cf0:::benchmark_data_smithi070_191495_object119231:head,488'7567,11:b4197a3a:::benchmark_data_smithi070_191495_object29713:head,463'1851,11:b41999ac:::benchmark_data_smithi070_191495_object35997:head,464'2238,11:b419cc07:::benchmark_data_smithi070_191495_object99485:head,479'6314,11:b41a28d3:::benchmark_data_smithi070_191495_object56619:head,466'3558,11:b41a70d2:::benchmark_data_smithi070_191495_object66968:head,468'4180,11:b41b1d64:::benchmark_data_smithi070_191495_object68968:head,468'4300,11:b41b26f0:::benchmark_data_smithi070_191495_object27264:head,463'1705,11:b41bf8e3:::benchmark_data_smithi070_191495_object73233:head,469'4585,11:b41ccd44:::benchmark_data_smithi070_191495_object21744:head,461'1378,11:b41e0119:::benchmark_data_smithi070_191495_object110764:head,484'7022,11:b41e0773:::benchmark_data_smithi070_191495_object126154:head,491'8001,11:b41e48e6:::benchmark_data_smithi070_191495_object54812:head,466'3451,11:b41ecda8:::benchmark_data_smithi070_191495_object103375:head,480'6561,11:b41f49db:::benchmark_data_smithi070_191495_object109132:head,483'6913,11:b42077d0:::benchmark_data_smithi070_191495_object60174:head,467'3763,11:b4210774:::benchmark_data_smithi070_191495_object71274:head,468'4448,11:b421443d:::benchmark_data_smithi070_191495_object110324:head,483'6988,11:b4216236:::benchmark_data_smithi070_191495_object73146:head,469'4580,11:b42236f0:::benchmark_data_smithi070_191495_object114845:head,486'7283,11:b4223fc1:::benchmark_data_smithi070_191495_object24510:head,462'1547,11:b422cdce:::benchmark_data_smithi070_191495_object123656:head,489'7843,11:b422d524:::benchmark_data_smithi070_191495_object34398:head,464'2149,11:b422dd61:::benchmark_data_smithi070_191495_object71961:head,468'4494,11:b423ba15:::benchmark_data_smithi070_191495_object18428:head,460'1182,11:b424aa2a:::benchmark_data_smithi070_191495_object119287:head,488'7573,11:b425ad48:::benchmark_data_smithi070_191495_object121551:head,488'7701,11:b426043f:::benchmark_data_smithi070_191495_object105235:head,481'6682,11:b4293e93:::benchmark_data_smithi070_191495_object16526:head,460'1049,11:b4297042:::benchmark_data_smithi070_191495_object108514:head,483'6878,11:b4297408:::benchmark_data_smithi070_191495_object126294:head,491'8008,11:b42afb54:::benchmark_data_smithi070_191495_object34672:head,464'2161,11:b42b0055:::benchmark_data_smithi070_191495_object41166:head,464'2561,11:b42b17cc:::benchmark_data_smithi070_191495_object118886:head,487'7545,11:b42bb23d:::benchmark_data_smithi070_191495_object27545:head,463'1720,11:b42bc30d:::benchmark_data_smithi070_191495_object98547:head,479'6255,11:b42bdc5d:::benchmark_data_smithi070_191495_object63339:head,467'3966,11:b42c0263:::benchmark_data_smithi070_191495_object33049:head,463'2061,11:b42c3b95:::benchmark_data_smithi070_191495_object33828:head,464'2112,11:b42c556e:::benchmark_data_smithi070_191495_object29781:head,463'1857,11:b42f2256:::benchmark_data_smithi070_191495_object63774:head,467'3991,11:b42f5846:::benchmark_data_smithi070_191495_object84058:head,474'5308,11:b42fb762:::benchmark_data_smithi070_191495_object47447:head,464'2973,11:b430e4eb:::benchmark_data_smithi070_191495_object117476:head,486'7441,11:b430f1ca:::benchmark_data_smithi070_191495_object68521:head,468'4271,11:b4317a08:::benchmark_data_smithi070_191495_object85698:head,474'5418,11:b4318c25:::benchmark_data_smithi070_191495_object105078:head,481'6671,11:b432e08e:::benchmark_data_smithi070_191495_object56817:head,466'3571,11:b4333d64:::benchmark_data_smithi070_191495_object127826:head,491'8105,11:b4342049:::benchmark_data_smithi070_191495_object64516:head,467'4039,11:b434690e:::benchmark_data_smithi070_191495_object39684:head,464'2475,11:b434cbc0:::benchmark_data_smithi070_191495_object25881:head,463'1619,11:b435b9b0:::benchmark_data_smithi070_191495_object60767:head,467'3806,11:b435f1ef:::benchmark_data_smithi070_191495_object65136:head,467'4088,11:b4368185:::benchmark_data_smithi070_191495_object57915:head,466'3647,11:b4377a51:::benchmark_data_smithi070_191495_object124575:head,490'7911,11:b43a6da4:::benchmark_data_smithi070_191495_object90106:head,475'5709,11:b43a8557:::benchmark_data_smithi070_191495_object72270:head,468'4518,11:b43a9a2f:::benchmark_data_smithi070_191495_object47805:head,464'2989,11:b43ab132:::benchmark_data_smithi070_191495_object20382:head,460'1300,11:b43ac2bc:::benchmark_data_smithi070_191495_object27059:head,463'1696,11:b43ad31f:::benchmark_data_smithi070_191495_object129876:head,491'8239,11:b43d044b:::benchmark_data_smithi070_191495_object45635:head,464'2849,11:b43e2691:::benchmark_data_smithi070_191495_object96648:head,479'6136,11:b43ebd4c:::benchmark_data_smithi070_191495_object67044:head,468'4187,11:b43f54a9:::benchmark_data_smithi070_191495_object80084:head,472'5069,11:b43f69d8:::benchmark_data_smithi070_191495_object61402:head,467'3853,11:b43f9f70:::benchmark_data_smithi070_191495_object72877:head,468'4561,11:b43ff125:::benchmark_data_smithi070_191495_object97776:head,479'6209,11:b43ff2dd:::benchmark_data_smithi070_191495_object20059:head,460'1289,11:b44042db:::benchmark_data_smithi070_191495_object57656:head,466'3630,11:b440bfd7:::benchmark_data_smithi070_191495_object30351:head,463'1890,11:b4411717:::benchmark_data_smithi070_191495_object65598:head,467'4110,11:b441e8ed:::benchmark_data_smithi070_191495_object24636:head,462'1557,11:b44247e1:::benchmark_data_smithi070_191495_object23713:head,462'1495,11:b442b09d:::benchmark_data_smithi070_191495_object79777:head,472'5046,11:b442c7af:::benchmark_data_smithi070_191495_object103582:head,480'6574,11:b4436025:::benchmark_data_smithi070_191495_object120811:head,488'7658,11:b443a8af:::benchmark_data_smithi070_191495_object112402:head,485'7135,11:b4446ab3:::benchmark_data_smithi070_191495_object86259:head,474'5455,11:b446bb40:::benchmark_data_smithi070_191495_object128747:head,491'8163,11:b4477e02:::benchmark_data_smithi070_191495_object103815:head,480'6596,11:b448c156:::benchmark_data_smithi070_191495_object44207:head,464'2756,11:b44ae94a:::benchmark_data_smithi070_191495_object83720:head,474'5287,11:b44aef21:::benchmark_data_smithi070_191495_object93715:head,477'5938,11:b44b1136:::benchmark_data_smithi070_191495_object23651:head,462'1488,11:b44beffa:::benchmark_data_smithi070_191495_object32520:head,463'2017,11:b44c20fe:::benchmark_data_smithi070_191495_object52585:head,466'3302,11:b44c6421:::benchmark_data_smithi070_191495_object62266:head,467'3904,11:b44c938d:::benchmark_data_smithi070_191495_object121816:head,488'7724,11:b44d579f:::benchmark_data_smithi070_191495_object99761:head,479'6336,11:b44d8b54:::benchmark_data_smithi070_191495_object90755:head,476'5743,11:b44dacc9:::benchmark_data_smithi070_191495_object72140:head,468'4508,11:b44e39d8:::benchmark_data_smithi070_191495_object44175:head,464'2753,11:b44f4be3:::benchmark_data_smithi070_191495_object16456:head,460'1042,11:b450bab4:::benchmark_data_smithi070_191495_object58296:head,466'3668,11:b4513ea4:::benchmark_data_smithi070_191495_object27434:head,463'1714,11:b452cff6:::benchmark_data_smithi070_191495_object16406:head,460'1041,11:b45335f3:::benchmark_data_smithi070_191495_object108670:head,483'6888,11:b453c51e:::benchmark_data_smithi070_191495_object24420:head,462'1542,11:b454432d:::benchmark_data_smithi070_191495_object118788:head,487'7537,11:b454b18f:::benchmark_data_smithi070_191495_object123886:head,489'7862,11:b4557ac6:::benchmark_data_smithi070_191495_object123175:head,488'7814,11:b455830f:::benchmark_data_smithi070_191495_object46998:head,464'2933,11:b456f10e:::benchmark_data_smithi070_191495_object98437:head,479'6248,11:b457ce55:::benchmark_data_smithi070_191495_object32019:head,463'1986,11:b45804d5:::benchmark_data_smithi070_191495_object75940:head,471'4773,11:b458a959:::benchmark_data_smithi070_191495_object61184:head,467'3836,11:b458c568:::benchmark_data_smithi070_191495_object117954:head,486'7475,11:b458f848:::benchmark_data_smithi070_191495_object78681:head,472'4972,11:b4591683:::benchmark_data_smithi070_191495_object85073:head,474'5363,11:b4593a17:::benchmark_data_smithi070_191495_object36423:head,464'2272,11:b45a0145:::benchmark_data_smithi070_191495_object37800:head,464'2358,11:b45a5011:::benchmark_data_smithi070_191495_object21318:head,461'1350,11:b45b3449:::benchmark_data_smithi070_191495_object41804:head,464'2603,11:b45d4d68:::benchmark_data_smithi070_191495_object44209:head,464'2757,11:b45e4ab2:::benchmark_data_smithi070_191495_object38345:head,464'2394,11:b45e59bc:::benchmark_data_smithi070_191495_object100825:head,479'6401,11:b45e5d5f:::benchmark_data_smithi070_191495_object82445:head,474'5217,11:b45ee876:::benchmark_data_smithi070_191495_object109552:head,483'6947,11:b45f3c19:::benchmark_data_smithi070_191495_object60485:head,467'3785,11:b45f77e9:::benchmark_data_smithi070_191495_object29616:head,463'1844,11:b45f8375:::benchmark_data_smithi070_191495_object55855:head,466'3515,11:b4604f36:::benchmark_data_smithi070_191495_object24470:head,462'1546,11:b461333b:::benchmark_data_smithi070_191495_object87048:head,474'5516,11:b46207bf:::benchmark_data_smithi070_191495_object71540:head,468'4471,11:b462122f:::benchmark_data_smithi070_191495_object17687:head,460'1134,11:b4622011:::benchmark_data_smithi070_191495_object18090:head,460'1159,11:b4624755:::benchmark_data_smithi070_191495_object32201:head,463'1999,11:b4625693:::benchmark_data_smithi070_191495_object69146:head,468'4318,11:b4631438:::benchmark_data_smithi070_191495_object41517:head,464'2584,11:b4633cff:::benchmark_data_smithi070_191495_object39057:head,464'2440,11:b4636102:::benchmark_data_smithi070_191495_object19791:head,460'1275,11:b463d053:::benchmark_data_smithi070_191495_object53824:head,466'3373,11:b4642ad3:::benchmark_data_smithi070_191495_object97194:head,479'6177,11:b4644b4f:::benchmark_data_smithi070_191495_object110708:head,484'7017,11:b4649a89:::benchmark_data_smithi070_191495_object126494:head,491'8022,11:b464c40b:::benchmark_data_smithi070_191495_object97920:head,479'6216,11:b464d10e:::benchmark_data_smithi070_191495_object21173:head,461'1341,11:b465e147:::benchmark_data_smithi070_191495_object55785:head,466'3509,11:b465e581:::benchmark_data_smithi070_191495_object115945:head,486'7346,11:b46637d7:::benchmark_data_smithi070_191495_object92452:head,476'5867,11:b4665eb2:::benchmark_data_smithi070_191495_object67217:head,468'4200,11:b466e3c5:::benchmark_data_smithi070_191495_object86830:head,474'5501,11:b4671c49:::benchmark_data_smithi070_191495_object76829:head,471'4844,11:b4677e71:::benchmark_data_smithi070_191495_object85768:head,474'5425,11:b468556a:::benchmark_data_smithi070_191495_object93797:head,477'5943,11:b4687c2d:::benchmark_data_smithi070_191495_object33627:head,464'2097,11:b4687d23:::benchmark_data_smithi070_191495_object74755:head,470'4685,11:b468f9aa:::benchmark_data_smithi070_191495_object53749:head,466'3366,11:b46942d8:::benchmark_data_smithi070_191495_object69531:head,468'4340,11:b469b248:::benchmark_data_smithi070_191495_object27537:head,463'1719,11:b469b7c9:::benchmark_data_smithi070_191495_object42644:head,464'2670,11:b469cf1d:::benchmark_data_smithi070_191495_object97847:head,479'6212,11:b46c34c5:::benchmark_data_smithi070_191495_object75371:head,470'4729,11:b46c55cb:::benchmark_data_smithi070_191495_object69568:head,468'4342,11:b46caa65:::benchmark_data_smithi070_191495_object18188:head,460'1170,11:b46d4b01:::benchmark_data_smithi070_191495_object90542:head,476'5730,11:b46e9d1e:::benchmark_data_smithi070_191495_object38840:head,464'2422,11:b46f321c:::benchmark_data_smithi070_191495_object69837:head,468'4354,11:b470671a:::benchmark_data_smithi070_191495_object30823:head,463'1910,11:b47083f0:::benchmark_data_smithi070_191495_object40416:head,464'2516,11:b4709f08:::benchmark_data_smithi070_191495_object129524:head,491'8215,11:b470d8e4:::benchmark_data_smithi070_191495_object67114:head,468'4190,11:b471da57:::benchmark_data_smithi070_191495_object103466:head,480'6565,11:b4720a71:::benchmark_data_smithi070_191495_object90728:head,476'5742,11:b4720d76:::benchmark_data_smithi070_191495_object48937:head,465'3072,11:b4721fc1:::benchmark_data_smithi070_191495_object35606:head,464'2213,11:b4726398:::benchmark_data_smithi070_191495_object81250:head,473'5141,11:b47305a5:::benchmark_data_smithi070_191495_object96819:head,479'6149,11:b4737f9c:::benchmark_data_smithi070_191495_object111413:head,484'7049,11:b474acc0:::benchmark_data_smithi070_191495_object22783:head,461'1438,11:b47511e8:::benchmark_data_smithi070_191495_object107457:head,482'6808,11:b4759839:::benchmark_data_smithi070_191495_object88210:head,474'5591,11:b4765c6e:::benchmark_data_smithi070_191495_object81535:head,473'5160,11:b476e161:::benchmark_data_smithi070_191495_object65139:head,467'4089,11:b4778448:::benchmark_data_smithi070_191495_object111688:head,484'7082,11:b478367d:::benchmark_data_smithi070_191495_object100270:head,479'6368,11:b478f878:::benchmark_data_smithi070_191495_object113349:head,486'7192,11:b479a68d:::benchmark_data_smithi070_191495_object89562:head,475'5673,11:b47a5d04:::benchmark_data_smithi070_191495_object117946:head,486'7473,11:b47ab116:::benchmark_data_smithi070_191495_object27268:head,463'1706,11:b47abff6:::benchmark_data_smithi070_191495_object61928:head,467'3884,11:b47c85db:::benchmark_data_smithi070_191495_object17980:head,460'1151,11:b47cf24a:::benchmark_data_smithi070_191495_object61403:head,467'3854,11:b47daefe:::benchmark_data_smithi070_191495_object37851:head,464'2363,11:b4807a81:::benchmark_data_smithi070_191495_object116765:head,486'7401,11:b480dee8:::benchmark_data_smithi070_191495_object74008:head,469'4625,11:b4817327:::benchmark_data_smithi070_191495_object128151:head,491'8125,11:b482b118:::benchmark_data_smithi070_191495_object118126:head,487'7486,11:b48407a0:::benchmark_data_smithi070_191495_object31882:head,463'1977,11:b4846dea:::benchmark_data_smithi070_191495_object26343:head,463'1648,11:b484731d:::benchmark_data_smithi070_191495_object29790:head,463'1860,11:b484cbd6:::benchmark_data_smithi070_191495_object95871:head,478'6077,11:b48526bc:::benchmark_data_smithi070_191495_object61153:head,467'3834,11:b485fc93:::benchmark_data_smithi070_191495_object58726:head,466'3690,11:b48645b7:::benchmark_data_smithi070_191495_object16092:head,460'1019,11:b4865fe8:::benchmark_data_smithi070_191495_object110279:head,483'6986,11:b4866dc3:::benchmark_data_smithi070_191495_object49966:head,465'3135,11:b4869088:::benchmark_data_smithi070_191495_object24304:head,462'1532,11:b486ff5e:::benchmark_data_smithi070_191495_object100888:head,479'6410,11:b48878e8:::benchmark_data_smithi070_191495_object62766:head,467'3934,11:b489668f:::benchmark_data_smithi070_191495_object29563:head,463'1842,11:b4896d46:::benchmark_data_smithi070_191495_object101140:head,479'6421,11:b4899c0f:::benchmark_data_smithi070_191495_object49434:head,465'3103,11:b489d84f:::benchmark_data_smithi070_191495_object86671:head,474'5489,11:b48b0109:::benchmark_data_smithi070_191495_object125902:head,490'7985,11:b48c70e1:::benchmark_data_smithi070_191495_object89971:head,475'5700,11:b48c93da:::benchmark_data_smithi070_191495_object60632:head,467'3794,11:b48da2a9:::benchmark_data_smithi070_191495_object70885:head,468'4426,11:b48e9cbf:::benchmark_data_smithi070_191495_object120729:head,488'7651,11:b48eddb4:::benchmark_data_smithi070_191495_object61122:head,467'3833,11:b48efd5f:::benchmark_data_smithi070_191495_object94434:head,478'5989)", 2024-09-16T09:44:44.661 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:44:26.154256+0000", 2024-09-16T09:44:44.661 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 18.477777450000001, 2024-09-16T09:44:44.661 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.043199553000000002, 2024-09-16T09:44:44.661 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:44:44.661 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:44:44.661 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T09:44:44.661 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:44:44.661 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.661 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:44:44.661 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.154256+0000", 2024-09-16T09:44:44.661 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:44:44.661 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.662 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.662 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:44:44.662 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.154256+0000", 2024-09-16T09:44:44.662 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:44:44.662 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.662 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.662 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:44:44.662 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.154256+0000", 2024-09-16T09:44:44.662 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.9500000000000002e-07 2024-09-16T09:44:44.662 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.662 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.662 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:44:44.663 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.154426+0000", 2024-09-16T09:44:44.663 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00016990899999999999 2024-09-16T09:44:44.663 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.663 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.663 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:44:44.663 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.154647+0000", 2024-09-16T09:44:44.663 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00022126000000000001 2024-09-16T09:44:44.663 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.663 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.663 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:44:44.663 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.155106+0000", 2024-09-16T09:44:44.663 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00045817399999999999 2024-09-16T09:44:44.663 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.664 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.664 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:44:44.664 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.194631+0000", 2024-09-16T09:44:44.664 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.039525680000000001 2024-09-16T09:44:44.664 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.664 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.664 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:44:44.664 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.195308+0000", 2024-09-16T09:44:44.664 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00067696 2024-09-16T09:44:44.664 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.664 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.665 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:44:44.665 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.197455+0000", 2024-09-16T09:44:44.665 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0021469750000000002 2024-09-16T09:44:44.665 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:44:44.665 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:44:44.665 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:44:44.665 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.665 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.665 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "pg_backfill(progress 11.d e 518/518 lb 11:b48efd5f:::benchmark_data_smithi070_191495_object94434:head)", 2024-09-16T09:44:44.665 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:44:26.157360+0000", 2024-09-16T09:44:44.665 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 18.474672795, 2024-09-16T09:44:44.665 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.040177345000000003, 2024-09-16T09:44:44.665 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:44:44.666 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:44:44.666 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T09:44:44.666 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:44:44.666 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.666 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:44:44.666 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.157360+0000", 2024-09-16T09:44:44.666 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:44:44.666 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.666 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.666 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:44:44.667 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.157360+0000", 2024-09-16T09:44:44.667 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:44:44.667 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.667 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.667 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:44:44.667 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.157361+0000", 2024-09-16T09:44:44.667 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0999999999999999e-07 2024-09-16T09:44:44.667 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.667 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.667 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:44:44.667 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.157368+0000", 2024-09-16T09:44:44.667 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.5639999999999999e-06 2024-09-16T09:44:44.667 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.668 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.668 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:44:44.668 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.157378+0000", 2024-09-16T09:44:44.668 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.6390000000000004e-06 2024-09-16T09:44:44.668 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.668 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.668 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:44:44.669 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.157382+0000", 2024-09-16T09:44:44.669 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.3939999999999998e-06 2024-09-16T09:44:44.669 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.669 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.669 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:44:44.669 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.197491+0000", 2024-09-16T09:44:44.669 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.040108637000000003 2024-09-16T09:44:44.669 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.669 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.669 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:44:44.669 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.197501+0000", 2024-09-16T09:44:44.669 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.0188e-05 2024-09-16T09:44:44.669 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.670 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.670 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:44:44.670 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.197538+0000", 2024-09-16T09:44:44.670 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6513000000000001e-05 2024-09-16T09:44:44.670 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:44:44.670 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:44:44.670 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:44:44.670 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.670 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.670 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "pg_scan(get_digest 11.4 11:247dd8fa:::benchmark_data_smithi070_191495_object96798:head-MIN e 518/517)", 2024-09-16T09:44:44.670 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:44:26.161057+0000", 2024-09-16T09:44:44.670 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 18.470975875000001, 2024-09-16T09:44:44.670 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.036310427999999999, 2024-09-16T09:44:44.671 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:44:44.671 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:44:44.671 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T09:44:44.671 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:44:44.671 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.671 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:44:44.671 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.161057+0000", 2024-09-16T09:44:44.671 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:44:44.671 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.671 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.671 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:44:44.671 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.161057+0000", 2024-09-16T09:44:44.671 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:44:44.671 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.672 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.672 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:44:44.672 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.161058+0000", 2024-09-16T09:44:44.672 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.6700000000000003e-07 2024-09-16T09:44:44.672 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.672 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.672 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:44:44.672 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.161062+0000", 2024-09-16T09:44:44.672 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0509999999999998e-06 2024-09-16T09:44:44.672 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.672 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.672 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:44:44.673 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.190415+0000", 2024-09-16T09:44:44.673 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.029353331999999999 2024-09-16T09:44:44.673 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.673 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.673 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:44:44.673 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.190438+0000", 2024-09-16T09:44:44.673 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 2.313e-05 2024-09-16T09:44:44.673 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.673 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.673 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:44:44.673 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.190519+0000", 2024-09-16T09:44:44.673 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.0207e-05 2024-09-16T09:44:44.673 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.674 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.674 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:44:44.674 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.190535+0000", 2024-09-16T09:44:44.674 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6483000000000001e-05 2024-09-16T09:44:44.674 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.674 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.674 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:44:44.674 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.197368+0000", 2024-09-16T09:44:44.674 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0068325579999999999 2024-09-16T09:44:44.674 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:44:44.674 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:44:44.674 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:44:44.674 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.675 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.675 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "pg_scan(get_digest 11.4 11:25a9c880:::benchmark_data_smithi070_191495_object37293:head-MIN e 518/517)", 2024-09-16T09:44:44.675 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:44:26.268438+0000", 2024-09-16T09:44:44.675 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 18.363594982999999, 2024-09-16T09:44:44.675 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.017659491999999999, 2024-09-16T09:44:44.675 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:44:44.675 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:44:44.675 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T09:44:44.675 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:44:44.675 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.675 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:44:44.675 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.268438+0000", 2024-09-16T09:44:44.675 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:44:44.676 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.676 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.676 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:44:44.676 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.268438+0000", 2024-09-16T09:44:44.676 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:44:44.676 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.676 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.676 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:44:44.676 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.268439+0000", 2024-09-16T09:44:44.676 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7599999999999997e-07 2024-09-16T09:44:44.676 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.676 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.677 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:44:44.677 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.268443+0000", 2024-09-16T09:44:44.677 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.1740000000000002e-06 2024-09-16T09:44:44.677 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.677 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.677 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:44:44.677 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.268449+0000", 2024-09-16T09:44:44.677 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.6039999999999998e-06 2024-09-16T09:44:44.677 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.677 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.677 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:44:44.677 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.268456+0000", 2024-09-16T09:44:44.677 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.5390000000000004e-06 2024-09-16T09:44:44.678 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.678 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.678 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:44:44.678 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.276074+0000", 2024-09-16T09:44:44.678 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0076176760000000003 2024-09-16T09:44:44.678 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.678 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.678 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:44:44.678 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.276082+0000", 2024-09-16T09:44:44.678 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.072e-06 2024-09-16T09:44:44.678 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.678 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.678 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:44:44.679 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.286098+0000", 2024-09-16T09:44:44.679 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.010015850999999999 2024-09-16T09:44:44.679 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:44:44.679 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:44:44.679 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:44:44.679 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.679 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.679 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "backfill_remove(11.4 e518 11:247dd8fa:::benchmark_data_smithi070_191495_object96798:head,479'6103,11:247e3376:::benchmark_data_smithi070_191495_object113490:head,486'7107,11:247f4d3e:::benchmark_data_smithi070_191495_object50180:head,465'3117,11:2480ff6e:::benchmark_data_smithi070_191495_object95730:head,478'6036,11:24816211:::benchmark_data_smithi070_191495_object96893:head,479'6107,11:2481a719:::benchmark_data_smithi070_191495_object99698:head,479'6265,11:2482056b:::benchmark_data_smithi070_191495_object63020:head,467'3969,11:2482d8cc:::benchmark_data_smithi070_191495_object82532:head,474'5203,11:248368ca:::benchmark_data_smithi070_191495_object93635:head,477'5895,11:2483ab5a:::benchmark_data_smithi070_191495_object67589:head,468'4256,11:2483f283:::benchmark_data_smithi070_191495_object80263:head,472'5031,11:24848102:::benchmark_data_smithi070_191495_object39449:head,464'2513,11:248487ab:::benchmark_data_smithi070_191495_object89658:head,475'5631,11:24878059:::benchmark_data_smithi070_191495_object130264:head,492'8147,11:2487d511:::benchmark_data_smithi070_191495_object55961:head,466'3492,11:2489753c:::benchmark_data_smithi070_191495_object77266:head,471'4846,11:248a6aca:::benchmark_data_smithi070_191495_object33974:head,464'2149,11:248b4fb5:::benchmark_data_smithi070_191495_object23945:head,462'1554,11:248d9d7b:::benchmark_data_smithi070_191495_object128751:head,491'8063,11:248db345:::benchmark_data_smithi070_191495_object123890:head,489'7740,11:248e7788:::benchmark_data_smithi070_191495_object116318:head,486'7271,11:248fd8da:::benchmark_data_smithi070_191495_object104232:head,480'6538,11:2491155c:::benchmark_data_smithi070_191495_object50645:head,465'3142,11:2491b429:::benchmark_data_smithi070_191495_object107495:head,482'6734,11:2491c678:::benchmark_data_smithi070_191495_object47772:head,464'2984,11:24921eb2:::benchmark_data_smithi070_191495_object71000:head,468'4476,11:249388ac:::benchmark_data_smithi070_191495_object94395:head,478'5955,11:249435b7:::benchmark_data_smithi070_191495_object31127:head,463'1991,11:249452ae:::benchmark_data_smithi070_191495_object120873:head,488'7570,11:249463b0:::benchmark_data_smithi070_191495_object33868:head,464'2141,11:2494c5d3:::benchmark_data_smithi070_191495_object44666:head,464'2811,11:2494f4bd:::benchmark_data_smithi070_191495_object46322:head,464'2912,11:24951cb3:::benchmark_data_smithi070_191495_object66549:head,468'4192,11:2495f637:::benchmark_data_smithi070_191495_object129437:head,491'8095,11:24978466:::benchmark_data_smithi070_191495_object46879:head,464'2945,11:2497d13d:::benchmark_data_smithi070_191495_object100099:head,479'6295,11:2497feb1:::benchmark_data_smithi070_191495_object82248:head,474'5176,11:249a65af:::benchmark_data_smithi070_191495_object116849:head,486'7308,11:249b8902:::benchmark_data_smithi070_191495_object110071:head,483'6899,11:249c24f2:::benchmark_data_smithi070_191495_object56164:head,466'3500,11:249c3159:::benchmark_data_smithi070_191495_object60431:head,467'3788,11:249d1fbb:::benchmark_data_smithi070_191495_object57038:head,466'3561,11:249d2f57:::benchmark_data_smithi070_191495_object62353:head,467'3912,11:249d472f:::benchmark_data_smithi070_191495_object130044:head,491'8134,11:249e61f7:::benchmark_data_smithi070_191495_object105096:head,481'6590,11:249ecfbf:::benchmark_data_smithi070_191495_object128664:head,491'8053,11:24a0167d:::benchmark_data_smithi070_191495_object22705:head,461'1461,11:24a04498:::benchmark_data_smithi070_191495_object73237:head,469'4619,11:24a05119:::benchmark_data_smithi070_191495_object87879:head,474'5513,11:24a1e1b8:::benchmark_data_smithi070_191495_object36987:head,464'2345,11:24a21257:::benchmark_data_smithi070_191495_object16054:head,460'1041,11:24a21925:::benchmark_data_smithi070_191495_object75871:head,471'4768,11:24a3336e:::benchmark_data_smithi070_191495_object74266:head,470'4676,11:24a3556d:::benchmark_data_smithi070_191495_object124112:head,489'7754,11:24a3c04a:::benchmark_data_smithi070_191495_object98790:head,479'6210,11:24a4d0c1:::benchmark_data_smithi070_191495_object125367:head,490'7845,11:24a52e3b:::benchmark_data_smithi070_191495_object40283:head,464'2560,11:24a58f21:::benchmark_data_smithi070_191495_object103033:head,480'6466,11:24a61e6c:::benchmark_data_smithi070_191495_object24187:head,462'1572,11:24a63fc1:::benchmark_data_smithi070_191495_object80324:head,472'5037,11:24a6d4d6:::benchmark_data_smithi070_191495_object110775:head,484'6940,11:24a75ffc:::benchmark_data_smithi070_191495_object92718:head,476'5817,11:24a7bb5b:::benchmark_data_smithi070_191495_object92784:head,476'5822,11:24a7c431:::benchmark_data_smithi070_191495_object123588:head,489'7723,11:24a7f17a:::benchmark_data_smithi070_191495_object65115:head,467'4103,11:24a8157f:::benchmark_data_smithi070_191495_object25581:head,463'1671,11:24a84b79:::benchmark_data_smithi070_191495_object89851:head,475'5645,11:24a8d54c:::benchmark_data_smithi070_191495_object94807:head,478'5980,11:24a96a82:::benchmark_data_smithi070_191495_object36041:head,464'2294,11:24a9ccf5:::benchmark_data_smithi070_191495_object28213:head,463'1804,11:24aa0e98:::benchmark_data_smithi070_191495_object32567:head,463'2064,11:24aa2cb4:::benchmark_data_smithi070_191495_object69649:head,468'4394,11:24ab1b56:::benchmark_data_smithi070_191495_object30314:head,463'1930,11:24abbc0b:::benchmark_data_smithi070_191495_object93633:head,477'5894,11:24abe3a7:::benchmark_data_smithi070_191495_object44547:head,464'2803,11:24abe48c:::benchmark_data_smithi070_191495_object123102:head,488'7694,11:24acca47:::benchmark_data_smithi070_191495_object126728:head,491'7941,11:24ad94ed:::benchmark_data_smithi070_191495_object51029:head,465'3159,11:24ae9136:::benchmark_data_smithi070_191495_object19993:head,460'1285,11:24b055d9:::benchmark_data_smithi070_191495_object20479:head,460'1316,11:24b0aa2a:::benchmark_data_smithi070_191495_object118571:head,487'7411,11:24b108db:::benchmark_data_smithi070_191495_object91640:head,476'5749,11:24b21491:::benchmark_data_smithi070_191495_object107930:head,482'6770,11:24b2d5fb:::benchmark_data_smithi070_191495_object19129:head,460'1234,11:24b2f9fa:::benchmark_data_smithi070_191495_object94332:head,478'5946,11:24b46810:::benchmark_data_smithi070_191495_object87290:head,474'5485,11:24b4c281:::benchmark_data_smithi070_191495_object102206:head,480'6425,11:24b4f78b:::benchmark_data_smithi070_191495_object71955:head,468'4542,11:24b5083a:::benchmark_data_smithi070_191495_object87740:head,474'5504,11:24b59231:::benchmark_data_smithi070_191495_object76138:head,471'4788,11:24b675ac:::benchmark_data_smithi070_191495_object61259:head,467'3843,11:24b6854d:::benchmark_data_smithi070_191495_object69775:head,468'4401,11:24b6ffe0:::benchmark_data_smithi070_191495_object51491:head,465'3199,11:24b8b611:::benchmark_data_smithi070_191495_object20438:head,460'1314,11:24b8d207:::benchmark_data_smithi070_191495_object45524:head,464'2868,11:24badbcb:::benchmark_data_smithi070_191495_object70054:head,468'4423,11:24bb0a79:::benchmark_data_smithi070_191495_object91504:head,476'5737,11:24bb5208:::benchmark_data_smithi070_191495_object24578:head,462'1599,11:24bba963:::benchmark_data_smithi070_191495_object59306:head,466'3715,11:24bc657a:::benchmark_data_smithi070_191495_object16852:head,460'1087,11:24bc74ed:::benchmark_data_smithi070_191495_object56135:head,466'3499,11:24bddd27:::benchmark_data_smithi070_191495_object98975:head,479'6228,11:24be7650:::benchmark_data_smithi070_191495_object48934:head,465'3046,11:24bf8fd4:::benchmark_data_smithi070_191495_object16476:head,460'1066,11:24bf9c84:::benchmark_data_smithi070_191495_object120522:head,488'7547,11:24bfa80c:::benchmark_data_smithi070_191495_object50095:head,465'3111,11:24bfce01:::benchmark_data_smithi070_191495_object130420:head,492'8157,11:24bff44a:::benchmark_data_smithi070_191495_object123325:head,488'7707,11:24c02151:::benchmark_data_smithi070_191495_object60934:head,467'3819,11:24c0228b:::benchmark_data_smithi070_191495_object34318:head,464'2169,11:24c05af3:::benchmark_data_smithi070_191495_object36483:head,464'2319,11:24c0c5ee:::benchmark_data_smithi070_191495_object35912:head,464'2288,11:24c22e60:::benchmark_data_smithi070_191495_object21231:head,461'1367,11:24c25f3d:::benchmark_data_smithi070_191495_object110969:head,484'6955,11:24c27e75:::benchmark_data_smithi070_191495_object23837:head,462'1544,11:24c35410:::benchmark_data_smithi070_191495_object116486:head,486'7282,11:24c3c038:::benchmark_data_smithi070_191495_object91384:head,476'5728,11:24c4a15e:::benchmark_data_smithi070_191495_object99464:head,479'6253,11:24c5be8a:::benchmark_data_smithi070_191495_object106700:head,481'6678,11:24c637ad:::benchmark_data_smithi070_191495_object99307:head,479'6243,11:24c699ca:::benchmark_data_smithi070_191495_object129807:head,491'8119,11:24c87519:::benchmark_data_smithi070_191495_object80872:head,473'5086,11:24c99a1b:::benchmark_data_smithi070_191495_object109779:head,483'6886,11:24c9b98c:::benchmark_data_smithi070_191495_object78981:head,472'4946,11:24cbb446:::benchmark_data_smithi070_191495_object31033:head,463'1982,11:24cc272b:::benchmark_data_smithi070_191495_object65556:head,467'4133,11:24cc3a65:::benchmark_data_smithi070_191495_object75001:head,470'4720,11:24cc68da:::benchmark_data_smithi070_191495_object66171:head,467'4169,11:24cce7ce:::benchmark_data_smithi070_191495_object16291:head,460'1053,11:24cec367:::benchmark_data_smithi070_191495_object105421:head,481'6608,11:24cee18f:::benchmark_data_smithi070_191495_object44007:head,464'2770,11:24cf2943:::benchmark_data_smithi070_191495_object85618:head,474'5390,11:24d12b86:::benchmark_data_smithi070_191495_object115695:head,486'7238,11:24d1dc52:::benchmark_data_smithi070_191495_object67684:head,468'4266,11:24d2486f:::benchmark_data_smithi070_191495_object55645:head,466'3472,11:24d2800f:::benchmark_data_smithi070_191495_object73742:head,469'4645,11:24d30d3d:::benchmark_data_smithi070_191495_object42557:head,464'2686,11:24d43853:::benchmark_data_smithi070_191495_object24334:head,462'1581,11:24d461d9:::benchmark_data_smithi070_191495_object125452:head,490'7851,11:24d46f14:::benchmark_data_smithi070_191495_object74188:head,470'4671,11:24d4ad9b:::benchmark_data_smithi070_191495_object67710:head,468'4268,11:24d51b80:::benchmark_data_smithi070_191495_object102519:head,480'6441,11:24d577d1:::benchmark_data_smithi070_191495_object123639:head,489'7726,11:24d5f502:::benchmark_data_smithi070_191495_object110845:head,484'6947,11:24d68af0:::benchmark_data_smithi070_191495_object106490:head,481'6664,11:24d6c74e:::benchmark_data_smithi070_191495_object27445:head,463'1761,11:24d6d9a5:::benchmark_data_smithi070_191495_object123024:head,488'7690,11:24d71e99:::benchmark_data_smithi070_191495_object108650:head,483'6820,11:24d75deb:::benchmark_data_smithi070_191495_object101722:head,480'6387,11:24d8af51:::benchmark_data_smithi070_191495_object64334:head,467'4049,11:24d8bac4:::benchmark_data_smithi070_191495_object105686:head,481'6625,11:24d94142:::benchmark_data_smithi070_191495_object111455:head,484'6991,11:24d962ac:::benchmark_data_smithi070_191495_object109698:head,483'6880,11:24d96c93:::benchmark_data_smithi070_191495_object66588:head,468'4194,11:24daea8e:::benchmark_data_smithi070_191495_object54118:head,466'3366,11:24db346e:::benchmark_data_smithi070_191495_object112377:head,485'7054,11:24dc0d1c:::benchmark_data_smithi070_191495_object53258:head,466'3307,11:24dcd009:::benchmark_data_smithi070_191495_object55047:head,466'3421,11:24dcddaa:::benchmark_data_smithi070_191495_object53535:head,466'3325,11:24dce961:::benchmark_data_smithi070_191495_object128221:head,491'8025,11:24dcfa96:::benchmark_data_smithi070_191495_object31926:head,463'2032,11:24dd1029:::benchmark_data_smithi070_191495_object123727:head,489'7730,11:24dda22c:::benchmark_data_smithi070_191495_object91577:head,476'5744,11:24ddada0:::benchmark_data_smithi070_191495_object88690:head,474'5561,11:24ddff47:::benchmark_data_smithi070_191495_object62570:head,467'3936,11:24de1443:::benchmark_data_smithi070_191495_object107201:head,481'6713,11:24de68f2:::benchmark_data_smithi070_191495_object18534:head,460'1197,11:24dfcffd:::benchmark_data_smithi070_191495_object64513:head,467'4064,11:24dff7e9:::benchmark_data_smithi070_191495_object85435:head,474'5380,11:24e00500:::benchmark_data_smithi070_191495_object46502:head,464'2924,11:24e04ede:::benchmark_data_smithi070_191495_object25316:head,463'1649,11:24e05598:::benchmark_data_smithi070_191495_object98317:head,479'6190,11:24e05d97:::benchmark_data_smithi070_191495_object65539:head,467'4131,11:24e09a2e:::benchmark_data_smithi070_191495_object114664:head,486'7163,11:24e11b1d:::benchmark_data_smithi070_191495_object83803:head,474'5271,11:24e3b071:::benchmark_data_smithi070_191495_object19018:head,460'1226,11:24e4d016:::benchmark_data_smithi070_191495_object72458:head,468'4578,11:24e562ec:::benchmark_data_smithi070_191495_object46533:head,464'2926,11:24e56cda:::benchmark_data_smithi070_191495_object35050:head,464'2224,11:24e6c662:::benchmark_data_smithi070_191495_object73351:head,469'4625,11:24e6fad4:::benchmark_data_smithi070_191495_object65422:head,467'4121,11:24e74dd0:::benchmark_data_smithi070_191495_object98395:head,479'6193,11:24e7df4a:::benchmark_data_smithi070_191495_object99433:head,479'6252,11:24e7e05e:::benchmark_data_smithi070_191495_object114605:head,486'7159,11:24e92999:::benchmark_data_smithi070_191495_object67388:head,468'4243,11:24ea14a4:::benchmark_data_smithi070_191495_object130211:head,492'8144,11:24ea2ce0:::benchmark_data_smithi070_191495_object38326:head,464'2441,11:24ea677c:::benchmark_data_smithi070_191495_object130020:head,491'8132,11:24eb283d:::benchmark_data_smithi070_191495_object30225:head,463'1923,11:24eb91f7:::benchmark_data_smithi070_191495_object121171:head,488'7590,11:24ec45b3:::benchmark_data_smithi070_191495_object115985:head,486'7252,11:24ed2d34:::benchmark_data_smithi070_191495_object31761:head,463'2024,11:24ed94de:::benchmark_data_smithi070_191495_object86185:head,474'5421,11:24ef23d0:::benchmark_data_smithi070_191495_object89899:head,475'5647,11:24ef445c:::benchmark_data_smithi070_191495_object83948:head,474'5280,11:24ef83de:::benchmark_data_smithi070_191495_object65690:head,467'4146,11:24f025f9:::benchmark_data_smithi070_191495_object60828:head,467'3812,11:24f0df01:::benchmark_data_smithi070_191495_object70895:head,468'4467,11:24f115d4:::benchmark_data_smithi070_191495_object81449:head,473'5123,11:24f11da4:::benchmark_data_smithi070_191495_object44272:head,464'2783,11:24f1493e:::benchmark_data_smithi070_191495_object113561:head,486'7111,11:24f19842:::benchmark_data_smithi070_191495_object63596:head,467'4004,11:24f1f1b9:::benchmark_data_smithi070_191495_object50698:head,465'3145,11:24f259bb:::benchmark_data_smithi070_191495_object95319:head,478'6011,11:24f25fc4:::benchmark_data_smithi070_191495_object118624:head,487'7414,11:24f2fa34:::benchmark_data_smithi070_191495_object108028:head,482'6780,11:24f32dee:::benchmark_data_smithi070_191495_object64578:head,467'4071,11:24f33044:::benchmark_data_smithi070_191495_object71226:head,468'4494,11:24f44520:::benchmark_data_smithi070_191495_object63296:head,467'3988,11:24f49c1f:::benchmark_data_smithi070_191495_object128053:head,491'8020,11:24f5a95a:::benchmark_data_smithi070_191495_object121691:head,488'7618,11:24f616ea:::benchmark_data_smithi070_191495_object38415:head,464'2445,11:24f7cccc:::benchmark_data_smithi070_191495_object76480:head,471'4809,11:24f83096:::benchmark_data_smithi070_191495_object21801:head,461'1399,11:24fa13bd:::benchmark_data_smithi070_191495_object91159:head,476'5716,11:24fab7c1:::benchmark_data_smithi070_191495_object26032:head,463'1692,11:24fc2797:::benchmark_data_smithi070_191495_object84523:head,474'5313,11:24fc76c5:::benchmark_data_smithi070_191495_object29753:head,463'1899,11:24fceff4:::benchmark_data_smithi070_191495_object85312:head,474'5369,11:24fd2c5f:::benchmark_data_smithi070_191495_object85388:head,474'5373,11:24fd3530:::benchmark_data_smithi070_191495_object68790:head,468'4341,11:24fd9d52:::benchmark_data_smithi070_191495_object39326:head,464'2503,11:24ffc85b:::benchmark_data_smithi070_191495_object121332:head,488'7601,11:250004cb:::benchmark_data_smithi070_191495_object118054:head,486'7381,11:25003543:::benchmark_data_smithi070_191495_object112163:head,485'7040,11:250512c5:::benchmark_data_smithi070_191495_object26467:head,463'1710,11:2505984f:::benchmark_data_smithi070_191495_object72219:head,468'4559,11:25064af1:::benchmark_data_smithi070_191495_object61366:head,467'3847,11:25069810:::benchmark_data_smithi070_191495_object29803:head,463'1902,11:2507c148:::benchmark_data_smithi070_191495_object125704:head,490'7871,11:25088ef6:::benchmark_data_smithi070_191495_object107308:head,481'6719,11:2509212f:::benchmark_data_smithi070_191495_object63129:head,467'3978,11:250922c3:::benchmark_data_smithi070_191495_object63058:head,467'3973,11:25093e8f:::benchmark_data_smithi070_191495_object49397:head,465'3075,11:250954e9:::benchmark_data_smithi070_191495_object41659:head,464'2644,11:2509e7e5:::benchmark_data_smithi070_191495_object30172:head,463'1920,11:250a7c42:::benchmark_data_smithi070_191495_object95908:head,478'6046,11:250baad7:::benchmark_data_smithi070_191495_object63786:head,467'4015,11:250c615c:::benchmark_data_smithi070_191495_object51420:head,465'3191,11:250c854a:::benchmark_data_smithi070_191495_object93111:head,477'5848,11:250dbd2b:::benchmark_data_smithi070_191495_object69968:head,468'4413,11:250efbd6:::benchmark_data_smithi070_191495_object120231:head,488'7525,11:250f884a:::benchmark_data_smithi070_191495_object86783:head,474'5459,11:25100922:::benchmark_data_smithi070_191495_object115439:head,486'7215,11:2510cf22:::benchmark_data_smithi070_191495_object116666:head,486'7294,11:2510da54:::benchmark_data_smithi070_191495_object105144:head,481'6594,11:251122a5:::benchmark_data_smithi070_191495_object60671:head,467'3801,11:25117095:::benchmark_data_smithi070_191495_object20159:head,460'1295,11:25117f51:::benchmark_data_smithi070_191495_object70932:head,468'4468,11:25118247:::benchmark_data_smithi070_191495_object61960:head,467'3885,11:2511e3c6:::benchmark_data_smithi070_191495_object33000:head,463'2092,11:25124f7a:::benchmark_data_smithi070_191495_object119818:head,488'7488,11:2512cf76:::benchmark_data_smithi070_191495_object94884:head,478'5984,11:2514566b:::benchmark_data_smithi070_191495_object41059:head,464'2615,11:25149439:::benchmark_data_smithi070_191495_object59739:head,466'3741,11:2514ae5c:::benchmark_data_smithi070_191495_object81633:head,473'5135,11:251506d7:::benchmark_data_smithi070_191495_object32802:head,463'2077,11:25162735:::benchmark_data_smithi070_191495_object102869:head,480'6459,11:2516cb72:::benchmark_data_smithi070_191495_object103958:head,480'6519,11:2517cafd:::benchmark_data_smithi070_191495_object89114:head,474'5591,11:25186cd3:::benchmark_data_smithi070_191495_object38229:head,464'2433,11:2518e8dd:::benchmark_data_smithi070_191495_object108505:head,483'6809,11:25191e55:::benchmark_data_smithi070_191495_object22260:head,461'1431,11:251a2155:::benchmark_data_smithi070_191495_object85110:head,474'5357,11:251bd21d:::benchmark_data_smithi070_191495_object119432:head,488'7467,11:251c06f4:::benchmark_data_smithi070_191495_object24953:head,462'1627,11:251c404d:::benchmark_data_smithi070_191495_object94377:head,478'5951,11:251c48f8:::benchmark_data_smithi070_191495_object34707:head,464'2195,11:251cd649:::benchmark_data_smithi070_191495_object54774:head,466'3401,11:251daded:::benchmark_data_smithi070_191495_object55290:head,466'3446,11:251dff0c:::benchmark_data_smithi070_191495_object129223:head,491'8083,11:251e4719:::benchmark_data_smithi070_191495_object86686:head,474'5449,11:251f0dd1:::benchmark_data_smithi070_191495_object103256:head,480'6481,11:25220a69:::benchmark_data_smithi070_191495_object45092:head,464'2844,11:2522a3a1:::benchmark_data_smithi070_191495_object35608:head,464'2266,11:25240f19:::benchmark_data_smithi070_191495_object56571:head,466'3524,11:25242f7f:::benchmark_data_smithi070_191495_object59395:head,466'3721,11:2524634e:::benchmark_data_smithi070_191495_object24804:head,462'1618,11:2524c203:::benchmark_data_smithi070_191495_object92482:head,476'5802,11:2524d904:::benchmark_data_smithi070_191495_object42730:head,464'2697,11:25256f93:::benchmark_data_smithi070_191495_object77590:head,471'4865,11:25273e21:::benchmark_data_smithi070_191495_object28105:head,463'1795,11:2527e4bb:::benchmark_data_smithi070_191495_object90975:head,476'5705,11:25281af6:::benchmark_data_smithi070_191495_object49225:head,465'3063,11:2528ade4:::benchmark_data_smithi070_191495_object30157:head,463'1919,11:2528ebc7:::benchmark_data_smithi070_191495_object29547:head,463'1885,11:2529481d:::benchmark_data_smithi070_191495_object28387:head,463'1816,11:25297967:::benchmark_data_smithi070_191495_object95673:head,478'6033,11:252a1b3b:::benchmark_data_smithi070_191495_object101094:head,479'6350,11:252a344a:::benchmark_data_smithi070_191495_object43457:head,464'2738,11:252ac1e1:::benchmark_data_smithi070_191495_object101250:head,479'6358,11:252b37d5:::benchmark_data_smithi070_191495_object64098:head,467'4036,11:252bf63c:::benchmark_data_smithi070_191495_object45074:head,464'2841,11:252c0307:::benchmark_data_smithi070_191495_object36175:head,464'2305,11:252c190c:::benchmark_data_smithi070_191495_object18365:head,460'1183,11:252c3276:::benchmark_data_smithi070_191495_object82646:head,474'5209,11:252c48ea:::benchmark_data_smithi070_191495_object73558:head,469'4637,11:252c8c03:::benchmark_data_smithi070_191495_object117990:head,486'7378,11:252d4c89:::benchmark_data_smithi070_191495_object45912:head,464'2894,11:252e560e:::benchmark_data_smithi070_191495_object74892:head,470'4715,11:252faeea:::benchmark_data_smithi070_191495_object33484:head,463'2124,11:25300798:::benchmark_data_smithi070_191495_object38228:head,464'2432,11:25303318:::benchmark_data_smithi070_191495_object55096:head,466'3424,11:253048d8:::benchmark_data_smithi070_191495_object20978:head,461'1348,11:2530ca06:::benchmark_data_smithi070_191495_object81080:head,473'5102,11:2530cf95:::benchmark_data_smithi070_191495_object83205:head,474'5236,11:2530fe55:::benchmark_data_smithi070_191495_object65856:head,467'4154,11:25310178:::benchmark_data_smithi070_191495_object86639:head,474'5447,11:2532aa6c:::benchmark_data_smithi070_191495_object129407:head,491'8092,11:25330a87:::benchmark_data_smithi070_191495_object80173:head,472'5021,11:25339cf5:::benchmark_data_smithi070_191495_object52352:head,465'3253,11:25343597:::benchmark_data_smithi070_191495_object42952:head,464'2713,11:2534b2e4:::benchmark_data_smithi070_191495_object49917:head,465'3103,11:2534c8ca:::benchmark_data_smithi070_191495_object103700:head,480'6503,11:253503bf:::benchmark_data_smithi070_191495_object54804:head,466'3403,11:2535506a:::benchmark_data_smithi070_191495_object75612:head,471'4751,11:2535caff:::benchmark_data_smithi070_191495_object52073:head,465'3237,11:253621fa:::benchmark_data_smithi070_191495_object86219:head,474'5424,11:25365e47:::benchmark_data_smithi070_191495_object24732:head,462'1611,11:2536e435:::benchmark_data_smithi070_191495_object128631:head,491'8049,11:2537647a:::benchmark_data_smithi070_191495_object129169:head,491'8080,11:2537807f:::benchmark_data_smithi070_191495_object25467:head,463'1663,11:25383c0d:::benchmark_data_smithi070_191495_object81731:head,473'5144,11:25383f65:::benchmark_data_smithi070_191495_object113803:head,486'7122,11:253855a1:::benchmark_data_smithi070_191495_object98846:head,479'6217,11:25395737:::benchmark_data_smithi070_191495_object107120:head,481'6705,11:2539c647:::benchmark_data_smithi070_191495_object58400:head,466'3637,11:2539ce34:::benchmark_data_smithi070_191495_object100154:head,479'6299,11:253aa876:::benchmark_data_smithi070_191495_object32267:head,463'2048,11:253ce1b2:::benchmark_data_smithi070_191495_object36401:head,464'2316,11:253ce968:::benchmark_data_smithi070_191495_object24185:head,462'1571,11:253d6441:::benchmark_data_smithi070_191495_object73786:head,469'4648,11:253d90a7:::benchmark_data_smithi070_191495_object80421:head,472'5044,11:253e23f1:::benchmark_data_smithi070_191495_object127957:head,491'8015,11:253e26c5:::benchmark_data_smithi070_191495_object107190:head,481'6710,11:253e38b9:::benchmark_data_smithi070_191495_object19472:head,460'1257,11:253ea6ae:::benchmark_data_smithi070_191495_object106537:head,481'6669,11:253f1c49:::benchmark_data_smithi070_191495_object111705:head,484'7013,11:253f2046:::benchmark_data_smithi070_191495_object58723:head,466'3659,11:25407562:::benchmark_data_smithi070_191495_object40254:head,464'2556,11:2540989e:::benchmark_data_smithi070_191495_object23217:head,462'1495,11:25414c3a:::benchmark_data_smithi070_191495_object101097:head,479'6351,11:25414eca:::benchmark_data_smithi070_191495_object121867:head,488'7628,11:25425f4c:::benchmark_data_smithi070_191495_object77073:head,471'4834,11:2542c97a:::benchmark_data_smithi070_191495_object61442:head,467'3854,11:2542fa07:::benchmark_data_smithi070_191495_object17152:head,460'1107,11:2544b306:::benchmark_data_smithi070_191495_object67009:head,468'4223,11:2544ec5f:::benchmark_data_smithi070_191495_object92727:head,476'5818,11:254577b9:::benchmark_data_smithi070_191495_object106253:head,481'6649,11:254584eb:::benchmark_data_smithi070_191495_object99981:head,479'6286,11:2546a630:::benchmark_data_smithi070_191495_object117381:head,486'7342,11:25471a7f:::benchmark_data_smithi070_191495_object99283:head,479'6242,11:25486098:::benchmark_data_smithi070_191495_object64587:head,467'4074,11:2548ef57:::benchmark_data_smithi070_191495_object68348:head,468'4311,11:2549280c:::benchmark_data_smithi070_191495_object93542:head,477'5888,11:254957bc:::benchmark_data_smithi070_191495_object119388:head,488'7462,11:25495e7b:::benchmark_data_smithi070_191495_object60201:head,467'3768,11:254beaab:::benchmark_data_smithi070_191495_object127390:head,491'7976,11:254c2140:::benchmark_data_smithi070_191495_object88468:head,474'5544,11:254c403c:::benchmark_data_smithi070_191495_object126486:head,491'7919,11:254e37c2:::benchmark_data_smithi070_191495_object87928:head,474'5519,11:254e585f:::benchmark_data_smithi070_191495_object91310:head,476'5722,11:254e9586:::benchmark_data_smithi070_191495_object48312:head,465'3014,11:254f35d8:::benchmark_data_smithi070_191495_object46598:head,464'2931,11:25508a56:::benchmark_data_smithi070_191495_object116683:head,486'7296,11:2550aeb0:::benchmark_data_smithi070_191495_object93988:head,477'5922,11:25522851:::benchmark_data_smithi070_191495_object127778:head,491'8002,11:25523c45:::benchmark_data_smithi070_191495_object58077:head,466'3618,11:25525439:::benchmark_data_smithi070_191495_object84700:head,474'5329,11:2552b1d9:::benchmark_data_smithi070_191495_object98727:head,479'6206,11:2553c07c:::benchmark_data_smithi070_191495_object34559:head,464'2185,11:25545517:::benchmark_data_smithi070_191495_object122936:head,488'7685,11:255497fb:::benchmark_data_smithi070_191495_object70560:head,468'4448,11:2554cb6a:::benchmark_data_smithi070_191495_object33859:head,464'2140,11:255544db:::benchmark_data_smithi070_191495_object18222:head,460'1173,11:2555f7c6:::benchmark_data_smithi070_191495_object65460:head,467'4124,11:2556a8a4:::benchmark_data_smithi070_191495_object124351:head,489'7768,11:2556d4f4:::benchmark_data_smithi070_191495_object58754:head,466'3666,11:255a0be5:::benchmark_data_smithi070_191495_object43131:head,464'2722,11:255a9bd7:::benchmark_data_smithi070_191495_object105185:head,481'6598,11:255a9eba:::benchmark_data_smithi070_191495_object99481:head,479'6255,11:255af7b1:::benchmark_data_smithi070_191495_object19328:head,460'1248,11:255b1844:::benchmark_data_smithi070_191495_object96885:head,479'6106,11:255ccae9:::benchmark_data_smithi070_191495_object107810:head,482'6759,11:255df287:::benchmark_data_smithi070_191495_object33328:head,463'2114,11:255e1653:::benchmark_data_smithi070_191495_object94349:head,478'5949,11:255ea3e4:::benchmark_data_smithi070_191495_object103859:head,480'6515,11:255ed191:::benchmark_data_smithi070_191495_object120079:head,488'7510,11:255f4a52:::benchmark_data_smithi070_191495_object71392:head,468'4503,11:255febc5:::benchmark_data_smithi070_191495_object78186:head,472'4897,11:25615c28:::benchmark_data_smithi070_191495_object73991:head,469'4656,11:2561ad35:::benchmark_data_smithi070_191495_object96127:head,478'6063,11:2561e09b:::benchmark_data_smithi070_191495_object115035:head,486'7185,11:256255fe:::benchmark_data_smithi070_191495_object38085:head,464'2420,11:25645f92:::benchmark_data_smithi070_191495_object68027:head,468'4285,11:256474eb:::benchmark_data_smithi070_191495_object38441:head,464'2447,11:2564f5b9:::benchmark_data_smithi070_191495_object125527:head,490'7859,11:2565def5:::benchmark_data_smithi070_191495_object94600:head,478'5966,11:25663407:::benchmark_data_smithi070_191495_object84779:head,474'5337,11:2566f6fb:::benchmark_data_smithi070_191495_object26497:head,463'1712,11:256721a6:::benchmark_data_smithi070_191495_object99736:head,479'6270,11:256854b3:::benchmark_data_smithi070_191495_object92150:head,476'5781,11:25689969:::benchmark_data_smithi070_191495_object124356:head,489'7769,11:2569bcf5:::benchmark_data_smithi070_191495_object87886:head,474'5515,11:256a1c71:::benchmark_data_smithi070_191495_object93541:head,477'5887,11:256af153:::benchmark_data_smithi070_191495_object61901:head,467'3881,11:256c2622:::benchmark_data_smithi070_191495_object63753:head,467'4014,11:256c5361:::benchmark_data_smithi070_191495_object89922:head,475'5649,11:256d0c62:::benchmark_data_smithi070_191495_object67174:head,468'4229,11:256d4609:::benchmark_data_smithi070_191495_object54168:head,466'3369,11:256d581d:::benchmark_data_smithi070_191495_object114880:head,486'7178,11:256ec35c:::benchmark_data_smithi070_191495_object35291:head,464'2244,11:256ee553:::benchmark_data_smithi070_191495_object80863:head,473'5084,11:256ef23d:::benchmark_data_smithi070_191495_object118791:head,487'7424,11:256f7f98:::benchmark_data_smithi070_191495_object68716:head,468'4333,11:256fc463:::benchmark_data_smithi070_191495_object114065:head,486'7133,11:256fd266:::benchmark_data_smithi070_191495_object40801:head,464'2601,11:25703c1b:::benchmark_data_smithi070_191495_object93330:head,477'5868,11:2570794c:::benchmark_data_smithi070_191495_object41347:head,464'2632,11:2570d31a:::benchmark_data_smithi070_191495_object107257:head,481'6715,11:257151c4:::benchmark_data_smithi070_191495_object52255:head,465'3245,11:257163ed:::benchmark_data_smithi070_191495_object43133:head,464'2723,11:25717808:::benchmark_data_smithi070_191495_object125890:head,490'7888,11:257186e2:::benchmark_data_smithi070_191495_object56371:head,466'3508,11:2571e837:::benchmark_data_smithi070_191495_object102174:head,480'6420,11:2572f602:::benchmark_data_smithi070_191495_object51278:head,465'3177,11:2573205c:::benchmark_data_smithi070_191495_object110287:head,483'6909,11:257332f7:::benchmark_data_smithi070_191495_object16061:head,460'1043,11:25739df0:::benchmark_data_smithi070_191495_object74413:head,470'4683,11:2574a74c:::benchmark_data_smithi070_191495_object55112:head,466'3426,11:2574afbe:::benchmark_data_smithi070_191495_object44906:head,464'2827,11:25756372:::benchmark_data_smithi070_191495_object21376:head,461'1381,11:25767083:::benchmark_data_smithi070_191495_object30908:head,463'1970,11:2577c346:::benchmark_data_smithi070_191495_object114382:head,486'7149,11:25780294:::benchmark_data_smithi070_191495_object21343:head,461'1377,11:2578e6a6:::benchmark_data_smithi070_191495_object28895:head,463'1846,11:25790bd1:::benchmark_data_smithi070_191495_object80752:head,473'5072,11:25792f7e:::benchmark_data_smithi070_191495_object51298:head,465'3178,11:2579ec4e:::benchmark_data_smithi070_191495_object125214:head,490'7832,11:257a1b58:::benchmark_data_smithi070_191495_object124897:head,490'7810,11:257a3b7c:::benchmark_data_smithi070_191495_object69534:head,468'4388,11:257a5a26:::benchmark_data_smithi070_191495_object100202:head,479'6305,11:257a67e3:::benchmark_data_smithi070_191495_object22835:head,462'1466,11:257b3a9e:::benchmark_data_smithi070_191495_object31487:head,463'2011,11:257c751a:::benchmark_data_smithi070_191495_object63598:head,467'4005,11:257c9ea6:::benchmark_data_smithi070_191495_object71829:head,468'4531,11:257cdd35:::benchmark_data_smithi070_191495_object19823:head,460'1271,11:257d4d9e:::benchmark_data_smithi070_191495_object123574:head,489'7722,11:257daaaf:::benchmark_data_smithi070_191495_object33930:head,464'2146,11:257e9e01:::benchmark_data_smithi070_191495_object102547:head,480'6443,11:257f4431:::benchmark_data_smithi070_191495_object33663:head,464'2135,11:257ff44e:::benchmark_data_smithi070_191495_object86383:head,474'5432,11:25804a33:::benchmark_data_smithi070_191495_object106717:head,481'6680,11:25807c97:::benchmark_data_smithi070_191495_object126239:head,491'7906,11:25807e38:::benchmark_data_smithi070_191495_object80593:head,473'5060,11:25814d76:::benchmark_data_smithi070_191495_object54939:head,466'3411,11:258168e8:::benchmark_data_smithi070_191495_object61858:head,467'3879,11:25831d33:::benchmark_data_smithi070_191495_object98297:head,479'6188,11:2584710b:::benchmark_data_smithi070_191495_object122968:head,488'7686,11:25848c64:::benchmark_data_smithi070_191495_object99369:head,479'6247,11:25870979:::benchmark_data_smithi070_191495_object34747:head,464'2200,11:25878eb9:::benchmark_data_smithi070_191495_object23012:head,462'1477,11:2587f54d:::benchmark_data_smithi070_191495_object29468:head,463'1880,11:25885101:::benchmark_data_smithi070_191495_object114089:head,486'7136,11:25889066:::benchmark_data_smithi070_191495_object24620:head,462'1601,11:2589a5c5:::benchmark_data_smithi070_191495_object39601:head,464'2524,11:2589b23b:::benchmark_data_smithi070_191495_object74096:head,469'4663,11:258a88c2:::benchmark_data_smithi070_191495_object38628:head,464'2465,11:258b84fa:::benchmark_data_smithi070_191495_object114321:head,486'7145,11:258df25d:::benchmark_data_smithi070_191495_object35395:head,464'2251,11:258e1450:::benchmark_data_smithi070_191495_object48343:head,465'3020,11:258e7e7a:::benchmark_data_smithi070_191495_object92859:head,476'5830,11:258f2d27:::benchmark_data_smithi070_191495_object29284:head,463'1869,11:258f6f77:::benchmark_data_smithi070_191495_object92548:head,476'5807,11:25901966:::benchmark_data_smithi070_191495_object27838:head,463'1782,11:25905fe8:::benchmark_data_smithi070_191495_object85783:head,474'5395,11:2590cf6d:::benchmark_data_smithi070_191495_object51578:head,465'3206,11:25910b8b:::benchmark_data_smithi070_191495_object126997:head,491'7955,11:2591d543:::benchmark_data_smithi070_191495_object60768:head,467'3807,11:259243f7:::benchmark_data_smithi070_191495_object47010:head,464'2950,11:2592d664:::benchmark_data_smithi070_191495_object124466:head,489'7780,11:259340ae:::benchmark_data_smithi070_191495_object34653:head,464'2190,11:2593d520:::benchmark_data_smithi070_191495_object110044:head,483'6895,11:2595d154:::benchmark_data_smithi070_191495_object42355:head,464'2678,11:2596340c:::benchmark_data_smithi070_191495_object36125:head,464'2297,11:25963a65:::benchmark_data_smithi070_191495_object66242:head,467'4176,11:2596517a:::benchmark_data_smithi070_191495_object16711:head,460'1079,11:259689bd:::benchmark_data_smithi070_191495_object78370:head,472'4908,11:2598d5c0:::benchmark_data_smithi070_191495_object111433:head,484'6988,11:2598dfcc:::benchmark_data_smithi070_191495_object28220:head,463'1806,11:259911fa:::benchmark_data_smithi070_191495_object32835:head,463'2081,11:259a1e6a:::benchmark_data_smithi070_191495_object68587:head,468'4327,11:259bfc0e:::benchmark_data_smithi070_191495_object74705:head,470'4701,11:259ce751:::benchmark_data_smithi070_191495_object115305:head,486'7206,11:259d357f:::benchmark_data_smithi070_191495_object40673:head,464'2589,11:259e0025:::benchmark_data_smithi070_191495_object103587:head,480'6497,11:259e3c3d:::benchmark_data_smithi070_191495_object51451:head,465'3196,11:259eecd6:::benchmark_data_smithi070_191495_object70574:head,468'4449,11:259f1c3e:::benchmark_data_smithi070_191495_object18214:head,460'1172,11:25a0020d:::benchmark_data_smithi070_191495_object108089:head,482'6784,11:25a03ea8:::benchmark_data_smithi070_191495_object76394:head,471'4804,11:25a09fab:::benchmark_data_smithi070_191495_object110473:head,483'6919,11:25a2e328:::benchmark_data_smithi070_191495_object70997:head,468'4475,11:25a306b0:::benchmark_data_smithi070_191495_object18412:head,460'1188,11:25a3df5e:::benchmark_data_smithi070_191495_object99748:head,479'6272,11:25a54c75:::benchmark_data_smithi070_191495_object40758:head,464'2596,11:25a6ea8e:::benchmark_data_smithi070_191495_object31144:head,463'1992,11:25a82080:::benchmark_data_smithi070_191495_object67138:head,468'4227,11:25a84b1f:::benchmark_data_smithi070_191495_object118585:head,487'7412,11:25a8deb5:::benchmark_data_smithi070_191495_object97502:head,479'6143,11:25a90207:::benchmark_data_smithi070_191495_object78713:head,472'4933,11:25a989c2:::benchmark_data_smithi070_191495_object111248:head,484'6971)", 2024-09-16T09:44:44.679 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:44:26.269010+0000", 2024-09-16T09:44:44.679 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 18.363023210000001, 2024-09-16T09:44:44.680 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.023216816000000001, 2024-09-16T09:44:44.680 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:44:44.680 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:44:44.680 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T09:44:44.680 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:44:44.680 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.680 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:44:44.680 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.269010+0000", 2024-09-16T09:44:44.681 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:44:44.681 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.681 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.681 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:44:44.681 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.269010+0000", 2024-09-16T09:44:44.681 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:44:44.681 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.681 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.681 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:44:44.681 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.269010+0000", 2024-09-16T09:44:44.681 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6300000000000001e-07 2024-09-16T09:44:44.681 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.682 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.682 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:44:44.682 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.269174+0000", 2024-09-16T09:44:44.682 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00016398399999999999 2024-09-16T09:44:44.682 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.682 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.682 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:44:44.682 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.269398+0000", 2024-09-16T09:44:44.682 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00022383200000000001 2024-09-16T09:44:44.682 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.682 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.682 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:44:44.682 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.269855+0000", 2024-09-16T09:44:44.683 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00045668299999999999 2024-09-16T09:44:44.683 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.683 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.683 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:44:44.683 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.288414+0000", 2024-09-16T09:44:44.683 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.018559671999999999 2024-09-16T09:44:44.683 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.683 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.683 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:44:44.683 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.289307+0000", 2024-09-16T09:44:44.683 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00089215300000000002 2024-09-16T09:44:44.683 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.683 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.683 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:44:44.684 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.292227+0000", 2024-09-16T09:44:44.684 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.002920129 2024-09-16T09:44:44.684 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:44:44.684 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:44:44.684 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:44:44.684 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.684 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.684 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "pg_backfill(progress 11.4 e 518/518 lb 11:25a989c2:::benchmark_data_smithi070_191495_object111248:head)", 2024-09-16T09:44:44.684 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:44:26.271555+0000", 2024-09-16T09:44:44.684 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 18.360478315999998, 2024-09-16T09:44:44.684 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.020785954999999998, 2024-09-16T09:44:44.684 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:44:44.685 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:44:44.685 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T09:44:44.685 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:44:44.685 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.685 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:44:44.685 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.271555+0000", 2024-09-16T09:44:44.685 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:44:44.685 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.685 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.685 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:44:44.685 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.271555+0000", 2024-09-16T09:44:44.686 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:44:44.686 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.686 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.686 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:44:44.686 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.271555+0000", 2024-09-16T09:44:44.686 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.9200000000000001e-07 2024-09-16T09:44:44.686 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.686 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.686 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:44:44.686 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.271562+0000", 2024-09-16T09:44:44.686 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.0249999999999997e-06 2024-09-16T09:44:44.687 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.687 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.687 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:44:44.687 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.271571+0000", 2024-09-16T09:44:44.687 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.6799999999999999e-06 2024-09-16T09:44:44.687 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.687 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.687 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:44:44.687 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.271576+0000", 2024-09-16T09:44:44.687 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0830000000000003e-06 2024-09-16T09:44:44.687 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.687 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.687 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:44:44.688 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.292284+0000", 2024-09-16T09:44:44.688 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.020707980000000001 2024-09-16T09:44:44.688 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.688 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.688 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:44:44.688 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.292300+0000", 2024-09-16T09:44:44.688 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5625999999999999e-05 2024-09-16T09:44:44.688 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.688 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.688 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:44:44.688 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.292341+0000", 2024-09-16T09:44:44.688 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.0969e-05 2024-09-16T09:44:44.689 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:44:44.689 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:44:44.689 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:44:44.689 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.689 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.689 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "backfill_remove(11.d e518 11:b5bcef61:::benchmark_data_smithi070_191495_object103740:head,480'6589,11:b5bd84e3:::benchmark_data_smithi070_191495_object108708:head,483'6893,11:b5bdc096:::benchmark_data_smithi070_191495_object19724:head,460'1271,11:b5bef720:::benchmark_data_smithi070_191495_object72520:head,468'4539,11:b5bf39f1:::benchmark_data_smithi070_191495_object27471:head,463'1715,11:b5bf5c1c:::benchmark_data_smithi070_191495_object67677:head,468'4231,11:b5bf8c1b:::benchmark_data_smithi070_191495_object127257:head,491'8069,11:b5c19c7a:::benchmark_data_smithi070_191495_object48965:head,465'3075,11:b5c1d2cd:::benchmark_data_smithi070_191495_object64113:head,467'4015,11:b5c2d484:::benchmark_data_smithi070_191495_object39044:head,464'2439,11:b5c3c584:::benchmark_data_smithi070_191495_object110673:head,483'7015,11:b5c5733c:::benchmark_data_smithi070_191495_object119862:head,488'7602,11:b5c6dcea:::benchmark_data_smithi070_191495_object23746:head,462'1496,11:b5c8352c:::benchmark_data_smithi070_191495_object101068:head,479'6417,11:b5c84049:::benchmark_data_smithi070_191495_object86375:head,474'5466,11:b5c87d4d:::benchmark_data_smithi070_191495_object106951:head,481'6777,11:b5c938cc:::benchmark_data_smithi070_191495_object52477:head,466'3291,11:b5c9392d:::benchmark_data_smithi070_191495_object43242:head,464'2702,11:b5c98353:::benchmark_data_smithi070_191495_object128722:head,491'8162,11:b5cafdbb:::benchmark_data_smithi070_191495_object26294:head,463'1644,11:b5cb468b:::benchmark_data_smithi070_191495_object112182:head,485'7115,11:b5cc61cc:::benchmark_data_smithi070_191495_object44706:head,464'2794,11:b5ccd779:::benchmark_data_smithi070_191495_object74092:head,469'4633,11:b5ccff98:::benchmark_data_smithi070_191495_object118102:head,487'7485,11:b5cd536c:::benchmark_data_smithi070_191495_object99696:head,479'6333,11:b5cd8ad9:::benchmark_data_smithi070_191495_object83786:head,474'5296,11:b5ce7648:::benchmark_data_smithi070_191495_object62968:head,467'3941,11:b5ce8644:::benchmark_data_smithi070_191495_object41723:head,464'2598,11:b5cffe0d:::benchmark_data_smithi070_191495_object103363:head,480'6559,11:b5d02b18:::benchmark_data_smithi070_191495_object52877:head,466'3325,11:b5d0cc87:::benchmark_data_smithi070_191495_object93180:head,477'5914,11:b5d0df94:::benchmark_data_smithi070_191495_object35880:head,464'2232,11:b5d0f903:::benchmark_data_smithi070_191495_object41707:head,464'2596,11:b5d129ec:::benchmark_data_smithi070_191495_object57753:head,466'3640,11:b5d20736:::benchmark_data_smithi070_191495_object105469:head,481'6693,11:b5d2debd:::benchmark_data_smithi070_191495_object79077:head,472'4993,11:b5d33675:::benchmark_data_smithi070_191495_object62446:head,467'3920,11:b5d3824f:::benchmark_data_smithi070_191495_object106923:head,481'6775,11:b5d3a018:::benchmark_data_smithi070_191495_object75106:head,470'4707,11:b5d47ae7:::benchmark_data_smithi070_191495_object123865:head,489'7860,11:b5d4ba7c:::benchmark_data_smithi070_191495_object76177:head,471'4793,11:b5d4f1c8:::benchmark_data_smithi070_191495_object19157:head,460'1230,11:b5d5f0ed:::benchmark_data_smithi070_191495_object106346:head,481'6739,11:b5d5fae1:::benchmark_data_smithi070_191495_object62333:head,467'3911,11:b5d6a4f6:::benchmark_data_smithi070_191495_object32871:head,463'2051,11:b5d7b645:::benchmark_data_smithi070_191495_object60418:head,467'3782,11:b5d7bd6b:::benchmark_data_smithi070_191495_object95476:head,478'6054,11:b5d8400d:::benchmark_data_smithi070_191495_object69899:head,468'4362,11:b5d94505:::benchmark_data_smithi070_191495_object70579:head,468'4407,11:b5d9c0ad:::benchmark_data_smithi070_191495_object99552:head,479'6317,11:b5d9ca80:::benchmark_data_smithi070_191495_object110204:head,483'6981,11:b5daceb5:::benchmark_data_smithi070_191495_object103619:head,480'6580,11:b5dae6e9:::benchmark_data_smithi070_191495_object93168:head,477'5912,11:b5db3d79:::benchmark_data_smithi070_191495_object124963:head,490'7927,11:b5dcc73e:::benchmark_data_smithi070_191495_object34243:head,464'2133,11:b5dd0169:::benchmark_data_smithi070_191495_object51747:head,465'3250,11:b5dd121b:::benchmark_data_smithi070_191495_object126264:head,491'8006,11:b5dd25ab:::benchmark_data_smithi070_191495_object126781:head,491'8044,11:b5dd7ce4:::benchmark_data_smithi070_191495_object50192:head,465'3149,11:b5dddc24:::benchmark_data_smithi070_191495_object92423:head,476'5863,11:b5de299e:::benchmark_data_smithi070_191495_object33358:head,463'2078,11:b5ded643:::benchmark_data_smithi070_191495_object111379:head,484'7046,11:b5df0629:::benchmark_data_smithi070_191495_object115730:head,486'7332,11:b5df4e86:::benchmark_data_smithi070_191495_object35116:head,464'2186,11:b5dff125:::benchmark_data_smithi070_191495_object75711:head,471'4756,11:b5dff2c9:::benchmark_data_smithi070_191495_object124714:head,490'7915,11:b5e0694a:::benchmark_data_smithi070_191495_object31061:head,463'1928,11:b5e0a77f:::benchmark_data_smithi070_191495_object39457:head,464'2459,11:b5e0d343:::benchmark_data_smithi070_191495_object79347:head,472'5016,11:b5e204f1:::benchmark_data_smithi070_191495_object21350:head,461'1351,11:b5e26f16:::benchmark_data_smithi070_191495_object48611:head,465'3048,11:b5e2703f:::benchmark_data_smithi070_191495_object53175:head,466'3339,11:b5e2affd:::benchmark_data_smithi070_191495_object71473:head,468'4468,11:b5e4226d:::benchmark_data_smithi070_191495_object120492:head,488'7634,11:b5e42a56:::benchmark_data_smithi070_191495_object101814:head,480'6461,11:b5e47902:::benchmark_data_smithi070_191495_object45291:head,464'2826,11:b5e48a69:::benchmark_data_smithi070_191495_object112210:head,485'7116,11:b5e4a6b6:::benchmark_data_smithi070_191495_object23561:head,462'1483,11:b5e4a6f1:::benchmark_data_smithi070_191495_object39576:head,464'2466,11:b5e5b2bc:::benchmark_data_smithi070_191495_object68246:head,468'4260,11:b5e5d748:::benchmark_data_smithi070_191495_object114430:head,486'7255,11:b5e7a3a2:::benchmark_data_smithi070_191495_object100224:head,479'6364,11:b5e86335:::benchmark_data_smithi070_191495_object108327:head,482'6861,11:b5e9346d:::benchmark_data_smithi070_191495_object27700:head,463'1732,11:b5e9e52a:::benchmark_data_smithi070_191495_object26135:head,463'1635,11:b5e9e8f9:::benchmark_data_smithi070_191495_object121714:head,488'7714,11:b5ea9f79:::benchmark_data_smithi070_191495_object40471:head,464'2523,11:b5ebe0b9:::benchmark_data_smithi070_191495_object94525:head,478'5997,11:b5ecbefe:::benchmark_data_smithi070_191495_object92704:head,476'5884,11:b5ed7985:::benchmark_data_smithi070_191495_object75889:head,471'4769,11:b5ee8992:::benchmark_data_smithi070_191495_object84802:head,474'5349,11:b5ef8604:::benchmark_data_smithi070_191495_object69206:head,468'4321,11:b5efb75e:::benchmark_data_smithi070_191495_object49002:head,465'3078,11:b5f01fd7:::benchmark_data_smithi070_191495_object72115:head,468'4504,11:b5f166c4:::benchmark_data_smithi070_191495_object119932:head,488'7609,11:b5f171e7:::benchmark_data_smithi070_191495_object50427:head,465'3163,11:b5f20285:::benchmark_data_smithi070_191495_object29042:head,463'1808,11:b5f79a17:::benchmark_data_smithi070_191495_object100872:head,479'6408,11:b5f7d3bb:::benchmark_data_smithi070_191495_object84878:head,474'5354,11:b5f7d5b7:::benchmark_data_smithi070_191495_object36757:head,464'2290,11:b5f87fa8:::benchmark_data_smithi070_191495_object19221:head,460'1235,11:b5f90f14:::benchmark_data_smithi070_191495_object86017:head,474'5446,11:b5f9ea5e:::benchmark_data_smithi070_191495_object98202:head,479'6235,11:b5f9eab6:::benchmark_data_smithi070_191495_object65725:head,467'4118,11:b5fa6a56:::benchmark_data_smithi070_191495_object88940:head,474'5633,11:b5fbbdba:::benchmark_data_smithi070_191495_object122539:head,488'7780,11:b5fbc97e:::benchmark_data_smithi070_191495_object116652:head,486'7391,11:b5fc75ae:::benchmark_data_smithi070_191495_object46990:head,464'2932,11:b5fd6464:::benchmark_data_smithi070_191495_object124285:head,489'7894,11:b5fda358:::benchmark_data_smithi070_191495_object46361:head,464'2888,11:b5fe2dd5:::benchmark_data_smithi070_191495_object45144:head,464'2816,11:b5ff4a8f:::benchmark_data_smithi070_191495_object61099:head,467'3829,11:b5ff5f82:::benchmark_data_smithi070_191495_object45192:head,464'2820,11:b5ffda82:::benchmark_data_smithi070_191495_object33791:head,464'2108,11:b5ffdc7a:::benchmark_data_smithi070_191495_object21878:head,461'1389,11:b5ffedce:::benchmark_data_smithi070_191495_object61946:head,467'3885,11:b60034bb:::benchmark_data_smithi070_191495_object89937:head,475'5698,11:b6020ece:::benchmark_data_smithi070_191495_object98833:head,479'6275,11:b6040c92:::benchmark_data_smithi070_191495_object65114:head,467'4087,11:b604e663:::benchmark_data_smithi070_191495_object106622:head,481'6755,11:b604f543:::benchmark_data_smithi070_191495_object87709:head,474'5562,11:b606a780:::benchmark_data_smithi070_191495_object101651:head,480'6456,11:b6074339:::benchmark_data_smithi070_191495_object118743:head,487'7531,11:b60824fb:::benchmark_data_smithi070_191495_object125304:head,490'7951,11:b608bbb6:::benchmark_data_smithi070_191495_object122310:head,488'7760,11:b608f665:::benchmark_data_smithi070_191495_object113995:head,486'7231,11:b60b0993:::benchmark_data_smithi070_191495_object29058:head,463'1810,11:b60b6168:::benchmark_data_smithi070_191495_object29121:head,463'1812,11:b60bb78e:::benchmark_data_smithi070_191495_object122229:head,488'7754,11:b60caac2:::benchmark_data_smithi070_191495_object54281:head,466'3411,11:b60d2ea4:::benchmark_data_smithi070_191495_object29787:head,463'1858,11:b60e8b83:::benchmark_data_smithi070_191495_object97144:head,479'6174,11:b60fcd26:::benchmark_data_smithi070_191495_object125512:head,490'7960,11:b60ff56c:::benchmark_data_smithi070_191495_object124258:head,489'7888,11:b610341e:::benchmark_data_smithi070_191495_object98757:head,479'6268,11:b61235df:::benchmark_data_smithi070_191495_object105941:head,481'6713,11:b6129c39:::benchmark_data_smithi070_191495_object38949:head,464'2431,11:b613b257:::benchmark_data_smithi070_191495_object86344:head,474'5463,11:b614352f:::benchmark_data_smithi070_191495_object103775:head,480'6592,11:b61489bf:::benchmark_data_smithi070_191495_object57127:head,466'3601,11:b6150e2f:::benchmark_data_smithi070_191495_object76881:head,471'4847,11:b61719ec:::benchmark_data_smithi070_191495_object65804:head,467'4124,11:b617e103:::benchmark_data_smithi070_191495_object79994:head,472'5058,11:b618271e:::benchmark_data_smithi070_191495_object25062:head,462'1581,11:b619734e:::benchmark_data_smithi070_191495_object99523:head,479'6316,11:b61987b2:::benchmark_data_smithi070_191495_object115653:head,486'7327,11:b61a5bc3:::benchmark_data_smithi070_191495_object79130:head,472'4996,11:b61b92fd:::benchmark_data_smithi070_191495_object86133:head,474'5451,11:b61c3768:::benchmark_data_smithi070_191495_object88144:head,474'5585,11:b61ca4e0:::benchmark_data_smithi070_191495_object41412:head,464'2578,11:b61cddf7:::benchmark_data_smithi070_191495_object44014:head,464'2744,11:b61ce756:::benchmark_data_smithi070_191495_object126679:head,491'8038,11:b61d2216:::benchmark_data_smithi070_191495_object46733:head,464'2910,11:b61d34d2:::benchmark_data_smithi070_191495_object103690:head,480'6585,11:b61e4b5e:::benchmark_data_smithi070_191495_object20929:head,461'1330,11:b61eab08:::benchmark_data_smithi070_191495_object74440:head,470'4664,11:b61ed6c0:::benchmark_data_smithi070_191495_object55599:head,466'3499,11:b61f07dd:::benchmark_data_smithi070_191495_object61363:head,467'3852,11:b6204c1f:::benchmark_data_smithi070_191495_object18678:head,460'1195,11:b62084ec:::benchmark_data_smithi070_191495_object29355:head,463'1827,11:b620d2c6:::benchmark_data_smithi070_191495_object37312:head,464'2335,11:b621a725:::benchmark_data_smithi070_191495_object126001:head,491'7991,11:b621e8b4:::benchmark_data_smithi070_191495_object103178:head,480'6551,11:b622272a:::benchmark_data_smithi070_191495_object100562:head,479'6389,11:b622d1fa:::benchmark_data_smithi070_191495_object16077:head,460'1015,11:b623622a:::benchmark_data_smithi070_191495_object98179:head,479'6233,11:b623bea5:::benchmark_data_smithi070_191495_object77471:head,471'4900,11:b623c70e:::benchmark_data_smithi070_191495_object65605:head,467'4111,11:b623cd2e:::benchmark_data_smithi070_191495_object69824:head,468'4353,11:b623f588:::benchmark_data_smithi070_191495_object57660:head,466'3632,11:b6243331:::benchmark_data_smithi070_191495_object123929:head,489'7864,11:b6243feb:::benchmark_data_smithi070_191495_object127175:head,491'8063,11:b624af07:::benchmark_data_smithi070_191495_object21486:head,461'1360,11:b624eb05:::benchmark_data_smithi070_191495_object20018:head,460'1288,11:b62660c8:::benchmark_data_smithi070_191495_object89370:head,474'5655,11:b62679c3:::benchmark_data_smithi070_191495_object74153:head,469'4640,11:b626d417:::benchmark_data_smithi070_191495_object36782:head,464'2295,11:b627396a:::benchmark_data_smithi070_191495_object36317:head,464'2261,11:b627e363:::benchmark_data_smithi070_191495_object68260:head,468'4261,11:b62841ce:::benchmark_data_smithi070_191495_object35984:head,464'2236,11:b628e7f9:::benchmark_data_smithi070_191495_object121754:head,488'7720,11:b629b63a:::benchmark_data_smithi070_191495_object60035:head,467'3759,11:b629bb75:::benchmark_data_smithi070_191495_object129157:head,491'8190,11:b629daf6:::benchmark_data_smithi070_191495_object82385:head,474'5215,11:b62a1b7d:::benchmark_data_smithi070_191495_object76223:head,471'4798,11:b62afe77:::benchmark_data_smithi070_191495_object76990:head,471'4855,11:b62d8419:::benchmark_data_smithi070_191495_object78393:head,472'4955,11:b62dc4d2:::benchmark_data_smithi070_191495_object105562:head,481'6697,11:b62e2d07:::benchmark_data_smithi070_191495_object112878:head,485'7163,11:b62ec5db:::benchmark_data_smithi070_191495_object49271:head,465'3097,11:b62f6f12:::benchmark_data_smithi070_191495_object120997:head,488'7668,11:b62fb3bc:::benchmark_data_smithi070_191495_object80743:head,473'5107,11:b6302c56:::benchmark_data_smithi070_191495_object57110:head,466'3595,11:b6305d53:::benchmark_data_smithi070_191495_object121853:head,488'7726,11:b630c888:::benchmark_data_smithi070_191495_object41282:head,464'2566,11:b63260a6:::benchmark_data_smithi070_191495_object57477:head,466'3618,11:b6331f86:::benchmark_data_smithi070_191495_object116256:head,486'7366,11:b633531d:::benchmark_data_smithi070_191495_object55405:head,466'3481,11:b63381b3:::benchmark_data_smithi070_191495_object29735:head,463'1854,11:b633d4a7:::benchmark_data_smithi070_191495_object116714:head,486'7397,11:b6342880:::benchmark_data_smithi070_191495_object43054:head,464'2690,11:b634a93a:::benchmark_data_smithi070_191495_object51684:head,465'3245,11:b634bda5:::benchmark_data_smithi070_191495_object60488:head,467'3786,11:b6379cc8:::benchmark_data_smithi070_191495_object24315:head,462'1533,11:b639688b:::benchmark_data_smithi070_191495_object128144:head,491'8124,11:b6398dcf:::benchmark_data_smithi070_191495_object51210:head,465'3214,11:b6398f57:::benchmark_data_smithi070_191495_object40652:head,464'2538,11:b63a0474:::benchmark_data_smithi070_191495_object49295:head,465'3099,11:b63a625e:::benchmark_data_smithi070_191495_object66568:head,468'4159,11:b63b627d:::benchmark_data_smithi070_191495_object48245:head,465'3024,11:b63cf31d:::benchmark_data_smithi070_191495_object109651:head,483'6954,11:b63f28e5:::benchmark_data_smithi070_191495_object20749:head,460'1322,11:b63fcff4:::benchmark_data_smithi070_191495_object72843:head,468'4556,11:b63ffca9:::benchmark_data_smithi070_191495_object129736:head,491'8232,11:b64017eb:::benchmark_data_smithi070_191495_object75997:head,471'4778,11:b6404d3b:::benchmark_data_smithi070_191495_object47344:head,464'2962,11:b640af81:::benchmark_data_smithi070_191495_object23309:head,462'1471,11:b640cbe9:::benchmark_data_smithi070_191495_object111000:head,484'7029,11:b641c01c:::benchmark_data_smithi070_191495_object87493:head,474'5547,11:b6423983:::benchmark_data_smithi070_191495_object16144:head,460'1022,11:b6428b2a:::benchmark_data_smithi070_191495_object37430:head,464'2339,11:b643418b:::benchmark_data_smithi070_191495_object60383:head,467'3779,11:b64417b8:::benchmark_data_smithi070_191495_object110739:head,484'7021,11:b6449d7a:::benchmark_data_smithi070_191495_object48606:head,465'3047,11:b6450a30:::benchmark_data_smithi070_191495_object21599:head,461'1365,11:b645a619:::benchmark_data_smithi070_191495_object88444:head,474'5603,11:b6463a51:::benchmark_data_smithi070_191495_object24528:head,462'1549,11:b646bdb2:::benchmark_data_smithi070_191495_object16030:head,460'1011,11:b6477052:::benchmark_data_smithi070_191495_object112626:head,485'7148,11:b647dc34:::benchmark_data_smithi070_191495_object60580:head,467'3790,11:b647e9f0:::benchmark_data_smithi070_191495_object79206:head,472'5006,11:b648912f:::benchmark_data_smithi070_191495_object74082:head,469'4631,11:b64acda2:::benchmark_data_smithi070_191495_object115395:head,486'7309,11:b64b6685:::benchmark_data_smithi070_191495_object64200:head,467'4021,11:b64b942d:::benchmark_data_smithi070_191495_object25654:head,463'1606,11:b64bfbe4:::benchmark_data_smithi070_191495_object108330:head,482'6862,11:b64c3f8c:::benchmark_data_smithi070_191495_object51693:head,465'3246,11:b64c88f2:::benchmark_data_smithi070_191495_object25482:head,463'1593,11:b64cf3a4:::benchmark_data_smithi070_191495_object56914:head,466'3581,11:b64d173f:::benchmark_data_smithi070_191495_object99473:head,479'6313,11:b64e0355:::benchmark_data_smithi070_191495_object77321:head,471'4889,11:b64f6716:::benchmark_data_smithi070_191495_object29935:head,463'1869,11:b64f6b46:::benchmark_data_smithi070_191495_object82167:head,474'5202,11:b64f7ae0:::benchmark_data_smithi070_191495_object19869:head,460'1279,11:b64f9642:::benchmark_data_smithi070_191495_object102161:head,480'6480,11:b64fc94e:::benchmark_data_smithi070_191495_object72956:head,469'4569,11:b6508a18:::benchmark_data_smithi070_191495_object70244:head,468'4387,11:b6508c97:::benchmark_data_smithi070_191495_object86768:head,474'5497,11:b650dfd2:::benchmark_data_smithi070_191495_object99697:head,479'6334,11:b6510435:::benchmark_data_smithi070_191495_object43646:head,464'2726,11:b651383a:::benchmark_data_smithi070_191495_object64870:head,467'4071,11:b651406a:::benchmark_data_smithi070_191495_object40160:head,464'2503,11:b6514981:::benchmark_data_smithi070_191495_object90202:head,475'5711,11:b6530216:::benchmark_data_smithi070_191495_object92906:head,477'5901,11:b6537c71:::benchmark_data_smithi070_191495_object45472:head,464'2834,11:b653c670:::benchmark_data_smithi070_191495_object76357:head,471'4810,11:b654393b:::benchmark_data_smithi070_191495_object36091:head,464'2245,11:b656b09f:::benchmark_data_smithi070_191495_object119066:head,487'7558,11:b656f283:::benchmark_data_smithi070_191495_object16787:head,460'1066,11:b6589587:::benchmark_data_smithi070_191495_object59408:head,466'3723,11:b6594365:::benchmark_data_smithi070_191495_object74564:head,470'4672,11:b65ad04e:::benchmark_data_smithi070_191495_object91571:head,476'5791,11:b65b399d:::benchmark_data_smithi070_191495_object46694:head,464'2906,11:b65b9a52:::benchmark_data_smithi070_191495_object62283:head,467'3906,11:b65ba760:::benchmark_data_smithi070_191495_object44662:head,464'2790,11:b65bd9ec:::benchmark_data_smithi070_191495_object21517:head,461'1362,11:b65c0042:::benchmark_data_smithi070_191495_object115506:head,486'7317,11:b65d0e93:::benchmark_data_smithi070_191495_object39454:head,464'2458,11:b65dc076:::benchmark_data_smithi070_191495_object46400:head,464'2890,11:b65e2708:::benchmark_data_smithi070_191495_object107967:head,482'6839,11:b65e3c51:::benchmark_data_smithi070_191495_object102703:head,480'6513,11:b65f64de:::benchmark_data_smithi070_191495_object68669:head,468'4277,11:b66026cb:::benchmark_data_smithi070_191495_object107656:head,482'6818,11:b6603803:::benchmark_data_smithi070_191495_object52265:head,465'3277,11:b660446b:::benchmark_data_smithi070_191495_object108113:head,482'6849,11:b6607271:::benchmark_data_smithi070_191495_object65288:head,467'4096,11:b660845f:::benchmark_data_smithi070_191495_object53968:head,466'3386,11:b660de8d:::benchmark_data_smithi070_191495_object78773:head,472'4977,11:b661bfce:::benchmark_data_smithi070_191495_object38717:head,464'2414,11:b661cff6:::benchmark_data_smithi070_191495_object87591:head,474'5554,11:b66479d6:::benchmark_data_smithi070_191495_object97234:head,479'6179,11:b66491ff:::benchmark_data_smithi070_191495_object107214:head,481'6791,11:b66910c1:::benchmark_data_smithi070_191495_object42251:head,464'2639,11:b6696ab0:::benchmark_data_smithi070_191495_object72083:head,468'4501,11:b66a1d54:::benchmark_data_smithi070_191495_object102967:head,480'6539,11:b66b429b:::benchmark_data_smithi070_191495_object54381:head,466'3416,11:b66bf11c:::benchmark_data_smithi070_191495_object112286:head,485'7124,11:b66bff37:::benchmark_data_smithi070_191495_object59603:head,466'3732,11:b66d1d0c:::benchmark_data_smithi070_191495_object23127:head,462'1462,11:b66e2a6f:::benchmark_data_smithi070_191495_object29908:head,463'1868,11:b66fa203:::benchmark_data_smithi070_191495_object126564:head,491'8028,11:b6702347:::benchmark_data_smithi070_191495_object40648:head,464'2537,11:b6711153:::benchmark_data_smithi070_191495_object79455:head,472'5026,11:b6718e77:::benchmark_data_smithi070_191495_object19141:head,460'1229,11:b671bafa:::benchmark_data_smithi070_191495_object99011:head,479'6286,11:b6721061:::benchmark_data_smithi070_191495_object115831:head,486'7336,11:b6722a55:::benchmark_data_smithi070_191495_object74635:head,470'4677,11:b6726f87:::benchmark_data_smithi070_191495_object125107:head,490'7940,11:b6737022:::benchmark_data_smithi070_191495_object72092:head,468'4502,11:b6742225:::benchmark_data_smithi070_191495_object19013:head,460'1219,11:b6746268:::benchmark_data_smithi070_191495_object106154:head,481'6729,11:b674a0a7:::benchmark_data_smithi070_191495_object122137:head,488'7746,11:b6750f76:::benchmark_data_smithi070_191495_object120656:head,488'7646,11:b6753218:::benchmark_data_smithi070_191495_object106425:head,481'6743,11:b676b753:::benchmark_data_smithi070_191495_object50742:head,465'3188,11:b677a6b8:::benchmark_data_smithi070_191495_object35253:head,464'2197,11:b6782f30:::benchmark_data_smithi070_191495_object52608:head,466'3305,11:b678bdf8:::benchmark_data_smithi070_191495_object75327:head,470'4725,11:b678d1bd:::benchmark_data_smithi070_191495_object121717:head,488'7715,11:b678d25d:::benchmark_data_smithi070_191495_object88097:head,474'5581,11:b6793737:::benchmark_data_smithi070_191495_object110713:head,484'7018,11:b6796988:::benchmark_data_smithi070_191495_object94435:head,478'5990,11:b679985e:::benchmark_data_smithi070_191495_object114712:head,486'7277,11:b67aec4b:::benchmark_data_smithi070_191495_object126667:head,491'8036,11:b67b3955:::benchmark_data_smithi070_191495_object97767:head,479'6208,11:b67badbb:::benchmark_data_smithi070_191495_object110950:head,484'7028,11:b67bc3f6:::benchmark_data_smithi070_191495_object66021:head,467'4137,11:b67ce4f7:::benchmark_data_smithi070_191495_object63771:head,467'3990,11:b67d1097:::benchmark_data_smithi070_191495_object33809:head,464'2110,11:b67ec239:::benchmark_data_smithi070_191495_object77030:head,471'4859,11:b67edf7d:::benchmark_data_smithi070_191495_object126369:head,491'8013,11:b67f1fbc:::benchmark_data_smithi070_191495_object76162:head,471'4792,11:b67f6313:::benchmark_data_smithi070_191495_object85968:head,474'5439,11:b6800b65:::benchmark_data_smithi070_191495_object30135:head,463'1881,11:b680950b:::benchmark_data_smithi070_191495_object17509:head,460'1122,11:b680d174:::benchmark_data_smithi070_191495_object64255:head,467'4022,11:b6812fa8:::benchmark_data_smithi070_191495_object30545:head,463'1900,11:b681bd61:::benchmark_data_smithi070_191495_object54434:head,466'3418,11:b6820c97:::benchmark_data_smithi070_191495_object129718:head,491'8229,11:b682516f:::benchmark_data_smithi070_191495_object121190:head,488'7681,11:b6857d64:::benchmark_data_smithi070_191495_object41972:head,464'2617,11:b685c573:::benchmark_data_smithi070_191495_object54072:head,466'3396,11:b687b20d:::benchmark_data_smithi070_191495_object129599:head,491'8222,11:b687b51c:::benchmark_data_smithi070_191495_object36473:head,464'2278,11:b687e800:::benchmark_data_smithi070_191495_object98261:head,479'6239,11:b6886c80:::benchmark_data_smithi070_191495_object58747:head,466'3691,11:b68872e9:::benchmark_data_smithi070_191495_object118007:head,486'7478,11:b6894e5e:::benchmark_data_smithi070_191495_object85153:head,474'5368,11:b689af7d:::benchmark_data_smithi070_191495_object120776:head,488'7655,11:b68a0294:::benchmark_data_smithi070_191495_object48230:head,465'3023,11:b68a8fea:::benchmark_data_smithi070_191495_object60262:head,467'3768,11:b68aff5f:::benchmark_data_smithi070_191495_object46450:head,464'2894,11:b68b0614:::benchmark_data_smithi070_191495_object90413:head,476'5726,11:b68b3da2:::benchmark_data_smithi070_191495_object45347:head,464'2829,11:b68c4c25:::benchmark_data_smithi070_191495_object98360:head,479'6245,11:b68cb34e:::benchmark_data_smithi070_191495_object23780:head,462'1497,11:b68ce8a1:::benchmark_data_smithi070_191495_object120119:head,488'7615,11:b68ced3e:::benchmark_data_smithi070_191495_object128489:head,491'8144,11:b68d869b:::benchmark_data_smithi070_191495_object28138:head,463'1756,11:b68f1109:::benchmark_data_smithi070_191495_object27680:head,463'1731,11:b68f3bd5:::benchmark_data_smithi070_191495_object85469:head,474'5399,11:b6901a84:::benchmark_data_smithi070_191495_object75272:head,470'4721,11:b69070d3:::benchmark_data_smithi070_191495_object91978:head,476'5825,11:b6918e64:::benchmark_data_smithi070_191495_object61297:head,467'3846,11:b691a83c:::benchmark_data_smithi070_191495_object63926:head,467'4000,11:b691b9ea:::benchmark_data_smithi070_191495_object29987:head,463'1874,11:b692d095:::benchmark_data_smithi070_191495_object111838:head,484'7095,11:b692efa3:::benchmark_data_smithi070_191495_object19498:head,460'1253,11:b693bf64:::benchmark_data_smithi070_191495_object36268:head,464'2256,11:b693d2b9:::benchmark_data_smithi070_191495_object80980:head,473'5122,11:b69427b0:::benchmark_data_smithi070_191495_object64805:head,467'4069,11:b6946e31:::benchmark_data_smithi070_191495_object78356:head,472'4953,11:b69491ef:::benchmark_data_smithi070_191495_object119058:head,487'7557,11:b69548f3:::benchmark_data_smithi070_191495_object17854:head,460'1145,11:b695c868:::benchmark_data_smithi070_191495_object92304:head,476'5853,11:b696b86c:::benchmark_data_smithi070_191495_object22168:head,461'1399,11:b6970bdc:::benchmark_data_smithi070_191495_object28705:head,463'1788,11:b69764e3:::benchmark_data_smithi070_191495_object66726:head,468'4170,11:b6993d9c:::benchmark_data_smithi070_191495_object82023:head,474'5195,11:b699e29f:::benchmark_data_smithi070_191495_object97696:head,479'6207,11:b69b24f4:::benchmark_data_smithi070_191495_object17904:head,460'1147,11:b69bbbca:::benchmark_data_smithi070_191495_object122216:head,488'7751,11:b69bf974:::benchmark_data_smithi070_191495_object36377:head,464'2267,11:b69c65d0:::benchmark_data_smithi070_191495_object127435:head,491'8082,11:b69c75d9:::benchmark_data_smithi070_191495_object121841:head,488'7725,11:b69dcab3:::benchmark_data_smithi070_191495_object75874:head,471'4767,11:b69e2409:::benchmark_data_smithi070_191495_object125709:head,490'7974,11:b69e505e:::benchmark_data_smithi070_191495_object87642:head,474'5558,11:b69f28c0:::benchmark_data_smithi070_191495_object51072:head,465'3205,11:b69f48ee:::benchmark_data_smithi070_191495_object93936:head,477'5955,11:b69f4cda:::benchmark_data_smithi070_191495_object38695:head,464'2413,11:b69f742e:::benchmark_data_smithi070_191495_object62142:head,467'3893,11:b6a10ca6:::benchmark_data_smithi070_191495_object92609:head,476'5879,11:b6a17821:::benchmark_data_smithi070_191495_object113316:head,486'7189,11:b6a192d2:::benchmark_data_smithi070_191495_object28183:head,463'1758,11:b6a27d2c:::benchmark_data_smithi070_191495_object33632:head,464'2098,11:b6a28ea7:::benchmark_data_smithi070_191495_object121744:head,488'7718,11:b6a2c297:::benchmark_data_smithi070_191495_object85445:head,474'5394,11:b6a3743c:::benchmark_data_smithi070_191495_object55946:head,466'3524,11:b6a3e8dd:::benchmark_data_smithi070_191495_object87936:head,474'5577,11:b6a46339:::benchmark_data_smithi070_191495_object48579:head,465'3045,11:b6a5a8ba:::benchmark_data_smithi070_191495_object59641:head,466'3735,11:b6a6261e:::benchmark_data_smithi070_191495_object46949:head,464'2930,11:b6a6ed66:::benchmark_data_smithi070_191495_object101712:head,480'6457,11:b6a70b06:::benchmark_data_smithi070_191495_object47906:head,464'2997,11:b6a76a32:::benchmark_data_smithi070_191495_object88958:head,474'5636,11:b6a774e4:::benchmark_data_smithi070_191495_object22574:head,461'1423,11:b6a8a3ff:::benchmark_data_smithi070_191495_object109311:head,483'6928,11:b6a8c20e:::benchmark_data_smithi070_191495_object47815:head,464'2990,11:b6a912a0:::benchmark_data_smithi070_191495_object36425:head,464'2273,11:b6a9a092:::benchmark_data_smithi070_191495_object86579:head,474'5481,11:b6aa1375:::benchmark_data_smithi070_191495_object29680:head,463'1847,11:b6aa3974:::benchmark_data_smithi070_191495_object26118:head,463'1634,11:b6ac53ed:::benchmark_data_smithi070_191495_object106206:head,481'6733,11:b6ac7a66:::benchmark_data_smithi070_191495_object97058:head,479'6169,11:b6ad9e40:::benchmark_data_smithi070_191495_object92902:head,477'5900,11:b6aef6a0:::benchmark_data_smithi070_191495_object72512:head,468'4537,11:b6af25b3:::benchmark_data_smithi070_191495_object44300:head,464'2767,11:b6af4cfb:::benchmark_data_smithi070_191495_object109468:head,483'6941,11:b6af5d40:::benchmark_data_smithi070_191495_object29421:head,463'1832,11:b6af9e28:::benchmark_data_smithi070_191495_object122410:head,488'7772,11:b6aff15c:::benchmark_data_smithi070_191495_object73200:head,469'4582,11:b6b3283a:::benchmark_data_smithi070_191495_object106952:head,481'6778,11:b6b373f7:::benchmark_data_smithi070_191495_object72517:head,468'4538,11:b6b403bd:::benchmark_data_smithi070_191495_object66255:head,467'4144,11:b6b40909:::benchmark_data_smithi070_191495_object35055:head,464'2183,11:b6b44b8f:::benchmark_data_smithi070_191495_object74397:head,470'4663,11:b6b45bcf:::benchmark_data_smithi070_191495_object122123:head,488'7743,11:b6b529e4:::benchmark_data_smithi070_191495_object81827:head,473'5186,11:b6b5903c:::benchmark_data_smithi070_191495_object115930:head,486'7345,11:b6b5922b:::benchmark_data_smithi070_191495_object121185:head,488'7680,11:b6b5bf9d:::benchmark_data_smithi070_191495_object127664:head,491'8095,11:b6b5d415:::benchmark_data_smithi070_191495_object51226:head,465'3215,11:b6b684ae:::benchmark_data_smithi070_191495_object111577:head,484'7069,11:b6b6cd2b:::benchmark_data_smithi070_191495_object48064:head,465'3010,11:b6b82a44:::benchmark_data_smithi070_191495_object48749:head,465'3058,11:b6b8f64f:::benchmark_data_smithi070_191495_object121799:head,488'7723,11:b6b8fbce:::benchmark_data_smithi070_191495_object31904:head,463'1980,11:b6b97575:::benchmark_data_smithi070_191495_object49934:head,465'3133,11:b6b98784:::benchmark_data_smithi070_191495_object111575:head,484'7068,11:b6b99b15:::benchmark_data_smithi070_191495_object122544:head,488'7781,11:b6b9b051:::benchmark_data_smithi070_191495_object16474:head,460'1044,11:b6ba40c4:::benchmark_data_smithi070_191495_object51018:head,465'3202,11:b6ba9fed:::benchmark_data_smithi070_191495_object91795:head,476'5809,11:b6bba351:::benchmark_data_smithi070_191495_object124147:head,489'7881,11:b6bbfdf4:::benchmark_data_smithi070_191495_object103370:head,480'6560,11:b6bc14ad:::benchmark_data_smithi070_191495_object127810:head,491'8102,11:b6bd4bed:::benchmark_data_smithi070_191495_object44982:head,464'2807,11:b6bd6fa4:::benchmark_data_smithi070_191495_object61027:head,467'3821,11:b6bd78b1:::benchmark_data_smithi070_191495_object118152:head,487'7488,11:b6be17f9:::benchmark_data_smithi070_191495_object109891:head,483'6966,11:b6bef6ff:::benchmark_data_smithi070_191495_object115422:head,486'7312,11:b6bf19e1:::benchmark_data_smithi070_191495_object103483:head,480'6567,11:b6bf250f:::benchmark_data_smithi070_191495_object74124:head,469'4635,11:b6bf91a8:::benchmark_data_smithi070_191495_object122557:head,488'7782,11:b6c0bfe6:::benchmark_data_smithi070_191495_object75594:head,471'4746,11:b6c0ca5f:::benchmark_data_smithi070_191495_object114765:head,486'7279,11:b6c1fdb5:::benchmark_data_smithi070_191495_object76631:head,471'4829,11:b6c2949c:::benchmark_data_smithi070_191495_object64630:head,467'4049,11:b6c2ae0b:::benchmark_data_smithi070_191495_object36751:head,464'2288,11:b6c31842:::benchmark_data_smithi070_191495_object47484:head,464'2976,11:b6c42009:::benchmark_data_smithi070_191495_object67607:head,468'4225,11:b6c4400b:::benchmark_data_smithi070_191495_object113116:head,485'7177,11:b6c5ff65:::benchmark_data_smithi070_191495_object97983:head,479'6219,11:b6c67e88:::benchmark_data_smithi070_191495_object28145:head,463'1757,11:b6c7011b:::benchmark_data_smithi070_191495_object91205:head,476'5764,11:b6c7525a:::benchmark_data_smithi070_191495_object123576:head,489'7839,11:b6c83357:::benchmark_data_smithi070_191495_object44106:head,464'2748,11:b6c8ab73:::benchmark_data_smithi070_191495_object67981:head,468'4253,11:b6c92cd8:::benchmark_data_smithi070_191495_object123523:head,489'7837,11:b6ca7729:::benchmark_data_smithi070_191495_object126056:head,491'7994,11:b6caa812:::benchmark_data_smithi070_191495_object102204:head,480'6484,11:b6cb80ce:::benchmark_data_smithi070_191495_object101330:head,479'6434,11:b6cc3b59:::benchmark_data_smithi070_191495_object73205:head,469'4583,11:b6cc727e:::benchmark_data_smithi070_191495_object45294:head,464'2827,11:b6cd71cf:::benchmark_data_smithi070_191495_object34897:head,464'2177,11:b6cea2d0:::benchmark_data_smithi070_191495_object52114:head,465'3269,11:b6cf0d38:::benchmark_data_smithi070_191495_object34085:head,464'2120,11:b6cf6bd2:::benchmark_data_smithi070_191495_object75883:head,471'4768,11:b6cf87f4:::benchmark_data_smithi070_191495_object83057:head,474'5257,11:b6cfb0e3:::benchmark_data_smithi070_191495_object38769:head,464'2417,11:b6d02a4f:::benchmark_data_smithi070_191495_object84240:head,474'5322,11:b6d0d816:::benchmark_data_smithi070_191495_object42168:head,464'2631,11:b6d0eb9a:::benchmark_data_smithi070_191495_object91551:head,476'5788,11:b6d1602f:::benchmark_data_smithi070_191495_object18796:head,460'1208,11:b6d1af6e:::benchmark_data_smithi070_191495_object99850:head,479'6340,11:b6d26c8c:::benchmark_data_smithi070_191495_object103825:head,480'6598,11:b6d30ffb:::benchmark_data_smithi070_191495_object48902:head,465'3070,11:b6d31191:::benchmark_data_smithi070_191495_object122330:head,488'7764,11:b6d439a0:::benchmark_data_smithi070_191495_object39322:head,464'2453,11:b6d4464a:::benchmark_data_smithi070_191495_object76557:head,471'4823,11:b6d50336:::benchmark_data_smithi070_191495_object30475:head,463'1898,11:b6d550d1:::benchmark_data_smithi070_191495_object125964:head,490'7987,11:b6d5ee0a:::benchmark_data_smithi070_191495_object26280:head,463'1643,11:b6d6273e:::benchmark_data_smithi070_191495_object82819:head,474'5244,11:b6d62bc5:::benchmark_data_smithi070_191495_object122864:head,488'7794,11:b6d6f57a:::benchmark_data_smithi070_191495_object16205:head,460'1025,11:b6d700fd:::benchmark_data_smithi070_191495_object99685:head,479'6332,11:b6d8290b:::benchmark_data_smithi070_191495_object119046:head,487'7556,11:b6d853f6:::benchmark_data_smithi070_191495_object119766:head,488'7599,11:b6d8746b:::benchmark_data_smithi070_191495_object48515:head,465'3040,11:b6d8826b:::benchmark_data_smithi070_191495_object88839:head,474'5625,11:b6d8d9ba:::benchmark_data_smithi070_191495_object117594:head,486'7449,11:b6d90abe:::benchmark_data_smithi070_191495_object22550:head,461'1422,11:b6d91c31:::benchmark_data_smithi070_191495_object74185:head,470'4643,11:b6d94eb6:::benchmark_data_smithi070_191495_object96085:head,478'6086,11:b6dad5e4:::benchmark_data_smithi070_191495_object114611:head,486'7272,11:b6dbce05:::benchmark_data_smithi070_191495_object41376:head,464'2573,11:b6dd1308:::benchmark_data_smithi070_191495_object26409:head,463'1653,11:b6dec5be:::benchmark_data_smithi070_191495_object47652:head,464'2979,11:b6df432f:::benchmark_data_smithi070_191495_object127453:head,491'8083,11:b6e04f25:::benchmark_data_smithi070_191495_object101345:head,479'6437,11:b6e0954c:::benchmark_data_smithi070_191495_object120054:head,488'7612,11:b6e0a292:::benchmark_data_smithi070_191495_object106651:head,481'6757,11:b6e17236:::benchmark_data_smithi070_191495_object99106:head,479'6293,11:b6e40a90:::benchmark_data_smithi070_191495_object125489:head,490'7959,11:b6e48871:::benchmark_data_smithi070_191495_object100122:head,479'6357,11:b6e56855:::benchmark_data_smithi070_191495_object67977:head,468'4251,11:b6e59381:::benchmark_data_smithi070_191495_object55491:head,466'3491,11:b6e5b58b:::benchmark_data_smithi070_191495_object72432:head,468'4530,11:b6e5cc34:::benchmark_data_smithi070_191495_object100275:head,479'6369,11:b6e5df01:::benchmark_data_smithi070_191495_object118095:head,487'7484)", 2024-09-16T09:44:44.689 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:44:26.334530+0000", 2024-09-16T09:44:44.689 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 18.297503496000001, 2024-09-16T09:44:44.690 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.012012063999999999, 2024-09-16T09:44:44.690 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:44:44.690 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:44:44.690 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T09:44:44.690 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:44:44.690 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.690 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:44:44.690 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.334530+0000", 2024-09-16T09:44:44.690 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:44:44.690 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.690 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.690 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:44:44.690 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.334530+0000", 2024-09-16T09:44:44.690 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:44:44.691 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.691 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.691 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:44:44.691 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.334530+0000", 2024-09-16T09:44:44.691 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.08e-07 2024-09-16T09:44:44.691 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.691 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.691 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:44:44.691 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.334761+0000", 2024-09-16T09:44:44.691 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000231277 2024-09-16T09:44:44.691 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.691 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.691 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:44:44.692 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.335005+0000", 2024-09-16T09:44:44.692 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00024325700000000001 2024-09-16T09:44:44.692 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.692 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.692 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:44:44.692 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.335517+0000", 2024-09-16T09:44:44.692 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00051272600000000002 2024-09-16T09:44:44.692 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.692 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.692 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:44:44.692 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.344107+0000", 2024-09-16T09:44:44.692 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0085897539999999998 2024-09-16T09:44:44.692 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.693 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.693 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:44:44.693 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.344687+0000", 2024-09-16T09:44:44.693 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00057991499999999997 2024-09-16T09:44:44.693 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.693 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.693 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:44:44.693 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.346542+0000", 2024-09-16T09:44:44.693 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001854727 2024-09-16T09:44:44.693 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:44:44.693 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:44:44.693 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:44:44.694 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.694 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.694 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "backfill_remove(11.d e519 11:b6e60d8c:::benchmark_data_smithi070_191495_object51608:head,465'3238,11:b6e674a9:::benchmark_data_smithi070_191495_object43023:head,464'2687,11:b6e67ef9:::benchmark_data_smithi070_191495_object70184:head,468'4380,11:b6e6fe2d:::benchmark_data_smithi070_191495_object47473:head,464'2975,11:b6e84f50:::benchmark_data_smithi070_191495_object90646:head,476'5739,11:b6e8c030:::benchmark_data_smithi070_191495_object116613:head,486'7388,11:b6e943b3:::benchmark_data_smithi070_191495_object108854:head,483'6901,11:b6e9c401:::benchmark_data_smithi070_191495_object119789:head,488'7600,11:b6ea8883:::benchmark_data_smithi070_191495_object116288:head,486'7370,11:b6ea8f64:::benchmark_data_smithi070_191495_object31889:head,463'1978,11:b6eaae08:::benchmark_data_smithi070_191495_object65009:head,467'4080,11:b6eacb19:::benchmark_data_smithi070_191495_object55349:head,466'3476,11:b6eb40e0:::benchmark_data_smithi070_191495_object20734:head,460'1321,11:b6eb77d6:::benchmark_data_smithi070_191495_object92439:head,476'5866,11:b6ed8f38:::benchmark_data_smithi070_191495_object44068:head,464'2745,11:b6edaade:::benchmark_data_smithi070_191495_object123174:head,488'7813,11:b6eee53e:::benchmark_data_smithi070_191495_object44875:head,464'2804,11:b6eef7a5:::benchmark_data_smithi070_191495_object85201:head,474'5371,11:b6ef0c78:::benchmark_data_smithi070_191495_object56173:head,466'3531,11:b6ef5ed0:::benchmark_data_smithi070_191495_object17331:head,460'1110,11:b6ef7427:::benchmark_data_smithi070_191495_object82854:head,474'5245,11:b6f01c44:::benchmark_data_smithi070_191495_object95668:head,478'6067,11:b6f2ac9a:::benchmark_data_smithi070_191495_object96521:head,479'6121,11:b6f2d8ec:::benchmark_data_smithi070_191495_object99986:head,479'6348,11:b6f33a74:::benchmark_data_smithi070_191495_object76719:head,471'4833,11:b6f39292:::benchmark_data_smithi070_191495_object63971:head,467'4004,11:b6f41150:::benchmark_data_smithi070_191495_object47134:head,464'2944,11:b6f43e0a:::benchmark_data_smithi070_191495_object53854:head,466'3374,11:b6f46871:::benchmark_data_smithi070_191495_object91802:head,476'5811,11:b6f4771e:::benchmark_data_smithi070_191495_object111782:head,484'7090,11:b6f4a768:::benchmark_data_smithi070_191495_object93969:head,477'5958,11:b6f4e078:::benchmark_data_smithi070_191495_object120338:head,488'7626,11:b6f51fc8:::benchmark_data_smithi070_191495_object114703:head,486'7275,11:b6f6077e:::benchmark_data_smithi070_191495_object18278:head,460'1175,11:b6f68aba:::benchmark_data_smithi070_191495_object73508:head,469'4601,11:b6f6c7b8:::benchmark_data_smithi070_191495_object85984:head,474'5443,11:b6f7cef6:::benchmark_data_smithi070_191495_object117532:head,486'7447,11:b6f7d6e9:::benchmark_data_smithi070_191495_object101913:head,480'6471,11:b6f84ae2:::benchmark_data_smithi070_191495_object67204:head,468'4199,11:b6f9e3e4:::benchmark_data_smithi070_191495_object71316:head,468'4454,11:b6fa751e:::benchmark_data_smithi070_191495_object106204:head,481'6732,11:b6fb0704:::benchmark_data_smithi070_191495_object126286:head,491'8007,11:b6fb59d7:::benchmark_data_smithi070_191495_object29269:head,463'1822,11:b6fb7104:::benchmark_data_smithi070_191495_object102910:head,480'6529,11:b6fbd681:::benchmark_data_smithi070_191495_object29723:head,463'1853,11:b6fc11df:::benchmark_data_smithi070_191495_object21902:head,461'1392,11:b6fcbe90:::benchmark_data_smithi070_191495_object18724:head,460'1201,11:b6fcd1c8:::benchmark_data_smithi070_191495_object100295:head,479'6371,11:b6ff7c7f:::benchmark_data_smithi070_191495_object29996:head,463'1876,11:b6ffa183:::benchmark_data_smithi070_191495_object101386:head,479'6441,11:b70032a5:::benchmark_data_smithi070_191495_object93738:head,477'5940,11:b7004702:::benchmark_data_smithi070_191495_object36781:head,464'2294,11:b7008e4f:::benchmark_data_smithi070_191495_object76060:head,471'4784,11:b700ef84:::benchmark_data_smithi070_191495_object124268:head,489'7891,11:b7018dd6:::benchmark_data_smithi070_191495_object22926:head,462'1450,11:b701e538:::benchmark_data_smithi070_191495_object100341:head,479'6374,11:b702b9e2:::benchmark_data_smithi070_191495_object33396:head,463'2081,11:b702ea23:::benchmark_data_smithi070_191495_object49138:head,465'3088,11:b70315d0:::benchmark_data_smithi070_191495_object85465:head,474'5397,11:b7036a35:::benchmark_data_smithi070_191495_object92770:head,476'5891,11:b7045af6:::benchmark_data_smithi070_191495_object21880:head,461'1390,11:b705598c:::benchmark_data_smithi070_191495_object56174:head,466'3532,11:b7066e53:::benchmark_data_smithi070_191495_object46352:head,464'2886,11:b706db41:::benchmark_data_smithi070_191495_object86833:head,474'5503,11:b706f21c:::benchmark_data_smithi070_191495_object67605:head,468'4224,11:b7097b96:::benchmark_data_smithi070_191495_object40996:head,464'2553,11:b70b8d06:::benchmark_data_smithi070_191495_object107310:head,481'6795,11:b70bdc34:::benchmark_data_smithi070_191495_object77037:head,471'4860,11:b70c78d7:::benchmark_data_smithi070_191495_object80910:head,473'5115,11:b70e1a0a:::benchmark_data_smithi070_191495_object86972:head,474'5513,11:b70e2ce9:::benchmark_data_smithi070_191495_object67459:head,468'4216,11:b70e735f:::benchmark_data_smithi070_191495_object60718:head,467'3800,11:b70f5e82:::benchmark_data_smithi070_191495_object123958:head,489'7868,11:b710799b:::benchmark_data_smithi070_191495_object122594:head,488'7783,11:b710f7ff:::benchmark_data_smithi070_191495_object55885:head,466'3519,11:b71108e6:::benchmark_data_smithi070_191495_object30965:head,463'1920,11:b7114992:::benchmark_data_smithi070_191495_object104364:head,480'6624,11:b7119811:::benchmark_data_smithi070_191495_object98992:head,479'6285,11:b711d09f:::benchmark_data_smithi070_191495_object55719:head,466'3502,11:b7123014:::benchmark_data_smithi070_191495_object109824:head,483'6964,11:b712db11:::benchmark_data_smithi070_191495_object111424:head,484'7052,11:b7144761:::benchmark_data_smithi070_191495_object105127:head,481'6674,11:b71460b8:::benchmark_data_smithi070_191495_object56821:head,466'3573,11:b717569b:::benchmark_data_smithi070_191495_object57618:head,466'3628,11:b71793c8:::benchmark_data_smithi070_191495_object119639:head,488'7591,11:b717c7e7:::benchmark_data_smithi070_191495_object16313:head,460'1034,11:b71831f0:::benchmark_data_smithi070_191495_object97418:head,479'6185,11:b7184bd9:::benchmark_data_smithi070_191495_object115919:head,486'7344,11:b718ef53:::benchmark_data_smithi070_191495_object109951:head,483'6969,11:b719aa6a:::benchmark_data_smithi070_191495_object118526:head,487'7519,11:b719b980:::benchmark_data_smithi070_191495_object117404:head,486'7432,11:b719dddf:::benchmark_data_smithi070_191495_object119155:head,488'7564,11:b719e75e:::benchmark_data_smithi070_191495_object76310:head,471'4807,11:b719f9c4:::benchmark_data_smithi070_191495_object129777:head,491'8237,11:b71a6414:::benchmark_data_smithi070_191495_object41900:head,464'2609,11:b71a67dc:::benchmark_data_smithi070_191495_object108056:head,482'6842,11:b71a6e0a:::benchmark_data_smithi070_191495_object82864:head,474'5246,11:b71ac19c:::benchmark_data_smithi070_191495_object75182:head,470'4713,11:b71ac8e5:::benchmark_data_smithi070_191495_object87492:head,474'5546,11:b71b0784:::benchmark_data_smithi070_191495_object99242:head,479'6299,11:b71bcbf9:::benchmark_data_smithi070_191495_object49550:head,465'3113,11:b71c5fc9:::benchmark_data_smithi070_191495_object49072:head,465'3083,11:b71cf449:::benchmark_data_smithi070_191495_object90521:head,476'5729,11:b71dedac:::benchmark_data_smithi070_191495_object123340:head,488'7822,11:b71ea965:::benchmark_data_smithi070_191495_object32641:head,463'2029,11:b71ebefa:::benchmark_data_smithi070_191495_object27087:head,463'1698,11:b71fd0df:::benchmark_data_smithi070_191495_object79178:head,472'5000,11:b721f857:::benchmark_data_smithi070_191495_object51895:head,465'3260,11:b7226d2f:::benchmark_data_smithi070_191495_object97057:head,479'6168,11:b722aef9:::benchmark_data_smithi070_191495_object122084:head,488'7739,11:b722e993:::benchmark_data_smithi070_191495_object24717:head,462'1564,11:b723803f:::benchmark_data_smithi070_191495_object115431:head,486'7313,11:b724adf1:::benchmark_data_smithi070_191495_object58639:head,466'3685,11:b724ce87:::benchmark_data_smithi070_191495_object73130:head,469'4578,11:b724e9d5:::benchmark_data_smithi070_191495_object96984:head,479'6161,11:b7257b3d:::benchmark_data_smithi070_191495_object72870:head,468'4560,11:b7262d3a:::benchmark_data_smithi070_191495_object125546:head,490'7961,11:b726d3ee:::benchmark_data_smithi070_191495_object129235:head,491'8192,11:b727c779:::benchmark_data_smithi070_191495_object29125:head,463'1813,11:b727fcdb:::benchmark_data_smithi070_191495_object40703:head,464'2541,11:b7281ac2:::benchmark_data_smithi070_191495_object41164:head,464'2560,11:b7293379:::benchmark_data_smithi070_191495_object95356:head,478'6042,11:b72a16ea:::benchmark_data_smithi070_191495_object41910:head,464'2612,11:b72b6b87:::benchmark_data_smithi070_191495_object101515:head,479'6447,11:b72b7609:::benchmark_data_smithi070_191495_object94720:head,478'6007,11:b72c2674:::benchmark_data_smithi070_191495_object84150:head,474'5316,11:b72cc133:::benchmark_data_smithi070_191495_object94243:head,478'5979,11:b72cc562:::benchmark_data_smithi070_191495_object73933:head,469'4618,11:b72d0c46:::benchmark_data_smithi070_191495_object59291:head,466'3716,11:b72ea0d1:::benchmark_data_smithi070_191495_object118356:head,487'7507,11:b72f5c7a:::benchmark_data_smithi070_191495_object37664:head,464'2350,11:b72f85f9:::benchmark_data_smithi070_191495_object58322:head,466'3670,11:b730160f:::benchmark_data_smithi070_191495_object40334:head,464'2512,11:b7302ec9:::benchmark_data_smithi070_191495_object104347:head,480'6622,11:b730bd65:::benchmark_data_smithi070_191495_object36973:head,464'2314,11:b730fb4d:::benchmark_data_smithi070_191495_object81095:head,473'5135,11:b73129c2:::benchmark_data_smithi070_191495_object108443:head,483'6870,11:b7312e9d:::benchmark_data_smithi070_191495_object41521:head,464'2585,11:b73410d8:::benchmark_data_smithi070_191495_object36817:head,464'2298,11:b734837d:::benchmark_data_smithi070_191495_object85247:head,474'5375,11:b734dac7:::benchmark_data_smithi070_191495_object88603:head,474'5612,11:b734f758:::benchmark_data_smithi070_191495_object88311:head,474'5598,11:b7362581:::benchmark_data_smithi070_191495_object89561:head,475'5672,11:b737606e:::benchmark_data_smithi070_191495_object48904:head,465'3071,11:b737930d:::benchmark_data_smithi070_191495_object56874:head,466'3579,11:b738a998:::benchmark_data_smithi070_191495_object51323:head,465'3225,11:b738bcf8:::benchmark_data_smithi070_191495_object103035:head,480'6543,11:b7392dcd:::benchmark_data_smithi070_191495_object121753:head,488'7719,11:b73a3c5b:::benchmark_data_smithi070_191495_object86700:head,474'5493,11:b73a4810:::benchmark_data_smithi070_191495_object122509:head,488'7777,11:b73bb140:::benchmark_data_smithi070_191495_object29444:head,463'1833,11:b73bc4a7:::benchmark_data_smithi070_191495_object78549:head,472'4964,11:b73d2efe:::benchmark_data_smithi070_191495_object81189:head,473'5138,11:b73e82aa:::benchmark_data_smithi070_191495_object17488:head,460'1121,11:b7412e3f:::benchmark_data_smithi070_191495_object66293:head,467'4145,11:b7414bf1:::benchmark_data_smithi070_191495_object79928:head,472'5056,11:b7416f4b:::benchmark_data_smithi070_191495_object64760:head,467'4064,11:b7424945:::benchmark_data_smithi070_191495_object36860:head,464'2307,11:b742ad7f:::benchmark_data_smithi070_191495_object48161:head,465'3019,11:b7432a90:::benchmark_data_smithi070_191495_object29218:head,463'1816,11:b7432e90:::benchmark_data_smithi070_191495_object83753:head,474'5290,11:b743f36f:::benchmark_data_smithi070_191495_object99332:head,479'6306,11:b744c773:::benchmark_data_smithi070_191495_object104334:head,480'6621,11:b7463d9c:::benchmark_data_smithi070_191495_object27179:head,463'1702,11:b7464285:::benchmark_data_smithi070_191495_object51419:head,465'3232,11:b7468a98:::benchmark_data_smithi070_191495_object127698:head,491'8098,11:b746d33c:::benchmark_data_smithi070_191495_object39687:head,464'2476,11:b747479a:::benchmark_data_smithi070_191495_object76129:head,471'4790,11:b7475ba1:::benchmark_data_smithi070_191495_object103348:head,480'6557,11:b7477c9e:::benchmark_data_smithi070_191495_object56157:head,466'3530,11:b747b7b8:::benchmark_data_smithi070_191495_object59261:head,466'3715,11:b7499c63:::benchmark_data_smithi070_191495_object85280:head,474'5381,11:b749cd42:::benchmark_data_smithi070_191495_object27855:head,463'1745,11:b74aa207:::benchmark_data_smithi070_191495_object67725:head,468'4233,11:b74af27c:::benchmark_data_smithi070_191495_object51314:head,465'3224,11:b74c8a02:::benchmark_data_smithi070_191495_object74229:head,470'4646,11:b74d64c4:::benchmark_data_smithi070_191495_object81517:head,473'5157,11:b74d73e1:::benchmark_data_smithi070_191495_object73646:head,469'4611,11:b74d853a:::benchmark_data_smithi070_191495_object105544:head,481'6695,11:b74df3f7:::benchmark_data_smithi070_191495_object75566:head,471'4742,11:b74fdcbf:::benchmark_data_smithi070_191495_object78617:head,472'4969,11:b750ac26:::benchmark_data_smithi070_191495_object54437:head,466'3420,11:b750ceb8:::benchmark_data_smithi070_191495_object104457:head,480'6628,11:b7515e31:::benchmark_data_smithi070_191495_object119757:head,488'7598,11:b751887f:::benchmark_data_smithi070_191495_object59315:head,466'3720,11:b751eac9:::benchmark_data_smithi070_191495_object33323:head,463'2075,11:b752c042:::benchmark_data_smithi070_191495_object94498:head,478'5995,11:b7534a77:::benchmark_data_smithi070_191495_object85784:head,474'5428,11:b753869a:::benchmark_data_smithi070_191495_object118014:head,486'7479,11:b754063b:::benchmark_data_smithi070_191495_object47264:head,464'2954,11:b75417b9:::benchmark_data_smithi070_191495_object16890:head,460'1070,11:b75432bf:::benchmark_data_smithi070_191495_object122132:head,488'7745,11:b7543820:::benchmark_data_smithi070_191495_object93254:head,477'5922,11:b7546483:::benchmark_data_smithi070_191495_object54646:head,466'3433,11:b754b2a7:::benchmark_data_smithi070_191495_object91519:head,476'5786,11:b754c5a7:::benchmark_data_smithi070_191495_object121876:head,488'7728,11:b754f314:::benchmark_data_smithi070_191495_object92320:head,476'5854,11:b755596b:::benchmark_data_smithi070_191495_object30439:head,463'1897,11:b756098d:::benchmark_data_smithi070_191495_object61408:head,467'3856,11:b756d6f7:::benchmark_data_smithi070_191495_object66850:head,468'4175,11:b758d355:::benchmark_data_smithi070_191495_object52503:head,466'3293,11:b759c298:::benchmark_data_smithi070_191495_object69455:head,468'4337,11:b759ede0:::benchmark_data_smithi070_191495_object94570:head,478'5999,11:b75ad9bc:::benchmark_data_smithi070_191495_object18766:head,460'1203,11:b75b4dcd:::benchmark_data_smithi070_191495_object87863:head,474'5572,11:b75bb80c:::benchmark_data_smithi070_191495_object43599:head,464'2723,11:b75bcf79:::benchmark_data_smithi070_191495_object107798:head,482'6825,11:b75bd562:::benchmark_data_smithi070_191495_object72802:head,468'4555,11:b75bdf60:::benchmark_data_smithi070_191495_object40221:head,464'2507,11:b75c5336:::benchmark_data_smithi070_191495_object83905:head,474'5302,11:b75ccbec:::benchmark_data_smithi070_191495_object16296:head,460'1031,11:b75deb4f:::benchmark_data_smithi070_191495_object49087:head,465'3084,11:b75e4ebd:::benchmark_data_smithi070_191495_object25878:head,463'1618,11:b75fb625:::benchmark_data_smithi070_191495_object77812:head,471'4922,11:b7601205:::benchmark_data_smithi070_191495_object96820:head,479'6150,11:b760f918:::benchmark_data_smithi070_191495_object63585:head,467'3979,11:b761163a:::benchmark_data_smithi070_191495_object96352:head,478'6106,11:b7612965:::benchmark_data_smithi070_191495_object62887:head,467'3938,11:b7616854:::benchmark_data_smithi070_191495_object105866:head,481'6707,11:b761c10b:::benchmark_data_smithi070_191495_object21439:head,461'1354,11:b762582b:::benchmark_data_smithi070_191495_object54726:head,466'3442,11:b7626108:::benchmark_data_smithi070_191495_object105306:head,481'6684,11:b7638239:::benchmark_data_smithi070_191495_object56204:head,466'3537,11:b76399e1:::benchmark_data_smithi070_191495_object43047:head,464'2689,11:b7639b8b:::benchmark_data_smithi070_191495_object42179:head,464'2633,11:b7647a80:::benchmark_data_smithi070_191495_object19174:head,460'1231,11:b7649ca6:::benchmark_data_smithi070_191495_object40167:head,464'2504,11:b766e336:::benchmark_data_smithi070_191495_object111295:head,484'7040,11:b766fc71:::benchmark_data_smithi070_191495_object60359:head,467'3777,11:b7670826:::benchmark_data_smithi070_191495_object29569:head,463'1843,11:b768df0a:::benchmark_data_smithi070_191495_object126432:head,491'8017,11:b768e9a9:::benchmark_data_smithi070_191495_object18139:head,460'1164,11:b7691020:::benchmark_data_smithi070_191495_object81935:head,474'5191,11:b76b7276:::benchmark_data_smithi070_191495_object22600:head,461'1426,11:b76c31aa:::benchmark_data_smithi070_191495_object129667:head,491'8225,11:b76d1b58:::benchmark_data_smithi070_191495_object36076:head,464'2242,11:b76d5763:::benchmark_data_smithi070_191495_object92034:head,476'5829,11:b76d8a4d:::benchmark_data_smithi070_191495_object117226:head,486'7426,11:b76e1946:::benchmark_data_smithi070_191495_object37512:head,464'2344,11:b76f3a51:::benchmark_data_smithi070_191495_object100982:head,479'6414,11:b76f999e:::benchmark_data_smithi070_191495_object27415:head,463'1713,11:b76fcef2:::benchmark_data_smithi070_191495_object47382:head,464'2967,11:b7706b9b:::benchmark_data_smithi070_191495_object107958:head,482'6838,11:b770cf73:::benchmark_data_smithi070_191495_object128029:head,491'8115,11:b7720554:::benchmark_data_smithi070_191495_object91911:head,476'5819,11:b7725357:::benchmark_data_smithi070_191495_object21892:head,461'1391,11:b773297d:::benchmark_data_smithi070_191495_object106895:head,481'6773,11:b773fd9a:::benchmark_data_smithi070_191495_object64709:head,467'4059,11:b7746b0e:::benchmark_data_smithi070_191495_object78780:head,472'4979,11:b7760f2b:::benchmark_data_smithi070_191495_object105143:head,481'6676,11:b7774a94:::benchmark_data_smithi070_191495_object23887:head,462'1504,11:b7774d37:::benchmark_data_smithi070_191495_object71362:head,468'4458,11:b7784eb6:::benchmark_data_smithi070_191495_object113049:head,485'7171,11:b7786703:::benchmark_data_smithi070_191495_object66866:head,468'4176,11:b7786d9f:::benchmark_data_smithi070_191495_object17435:head,460'1116,11:b778fddb:::benchmark_data_smithi070_191495_object31778:head,463'1969,11:b7792ea8:::benchmark_data_smithi070_191495_object22871:head,462'1446,11:b779e085:::benchmark_data_smithi070_191495_object52986:head,466'3331,11:b77a16c6:::benchmark_data_smithi070_191495_object114252:head,486'7246,11:b77a58d4:::benchmark_data_smithi070_191495_object52775:head,466'3318,11:b77ac3ed:::benchmark_data_smithi070_191495_object129276:head,491'8195,11:b77b8310:::benchmark_data_smithi070_191495_object62192:head,467'3897,11:b77bb4b1:::benchmark_data_smithi070_191495_object19986:head,460'1286,11:b77c1a90:::benchmark_data_smithi070_191495_object84145:head,474'5315,11:b77d4aaa:::benchmark_data_smithi070_191495_object116205:head,486'7363,11:b77d9e9c:::benchmark_data_smithi070_191495_object122036:head,488'7738,11:b77e5833:::benchmark_data_smithi070_191495_object60456:head,467'3784,11:b77e9771:::benchmark_data_smithi070_191495_object90403:head,475'5725,11:b77f1696:::benchmark_data_smithi070_191495_object85833:head,474'5432,11:b77f30a6:::benchmark_data_smithi070_191495_object79032:head,472'4990,11:b77fbd30:::benchmark_data_smithi070_191495_object61737:head,467'3873,11:b78066d7:::benchmark_data_smithi070_191495_object30843:head,463'1912,11:b782210a:::benchmark_data_smithi070_191495_object57739:head,466'3637,11:b7827b06:::benchmark_data_smithi070_191495_object33780:head,464'2105,11:b7831a92:::benchmark_data_smithi070_191495_object39506:head,464'2461,11:b7839235:::benchmark_data_smithi070_191495_object119638:head,488'7590,11:b7841c40:::benchmark_data_smithi070_191495_object24559:head,462'1551,11:b7842cb4:::benchmark_data_smithi070_191495_object61435:head,467'3858,11:b784357e:::benchmark_data_smithi070_191495_object53961:head,466'3385,11:b784f0e6:::benchmark_data_smithi070_191495_object73425:head,469'4596,11:b78589fb:::benchmark_data_smithi070_191495_object71574:head,468'4472,11:b785ea6a:::benchmark_data_smithi070_191495_object129306:head,491'8197,11:b7866470:::benchmark_data_smithi070_191495_object36747:head,464'2287,11:b78734b1:::benchmark_data_smithi070_191495_object69134:head,468'4316,11:b787807e:::benchmark_data_smithi070_191495_object31035:head,463'1925,11:b789d332:::benchmark_data_smithi070_191495_object51087:head,465'3207,11:b789d502:::benchmark_data_smithi070_191495_object127762:head,491'8100,11:b78a2cbb:::benchmark_data_smithi070_191495_object87834:head,474'5571,11:b78a7198:::benchmark_data_smithi070_191495_object83492:head,474'5274,11:b78b2214:::benchmark_data_smithi070_191495_object35200:head,464'2193,11:b78b23ad:::benchmark_data_smithi070_191495_object49223:head,465'3094,11:b78bb16b:::benchmark_data_smithi070_191495_object90634:head,476'5737,11:b78bb6ff:::benchmark_data_smithi070_191495_object58635:head,466'3684,11:b78c0aa7:::benchmark_data_smithi070_191495_object68609:head,468'4275,11:b78e4b43:::benchmark_data_smithi070_191495_object55169:head,466'3472,11:b78ed056:::benchmark_data_smithi070_191495_object20701:head,460'1320,11:b78f1083:::benchmark_data_smithi070_191495_object36763:head,464'2291,11:b78f817a:::benchmark_data_smithi070_191495_object68493:head,468'4269,11:b78fe94f:::benchmark_data_smithi070_191495_object94879:head,478'6014,11:b79039df:::benchmark_data_smithi070_191495_object46893:head,464'2924,11:b790ba35:::benchmark_data_smithi070_191495_object37492:head,464'2343,11:b7910238:::benchmark_data_smithi070_191495_object121907:head,488'7731,11:b7910904:::benchmark_data_smithi070_191495_object108018:head,482'6841,11:b7931c09:::benchmark_data_smithi070_191495_object21083:head,461'1335,11:b793b180:::benchmark_data_smithi070_191495_object76325:head,471'4808,11:b793e660:::benchmark_data_smithi070_191495_object44992:head,464'2809,11:b794ba68:::benchmark_data_smithi070_191495_object84689:head,474'5342,11:b794fd69:::benchmark_data_smithi070_191495_object89768:head,475'5683,11:b795c94e:::benchmark_data_smithi070_191495_object74377:head,470'4662,11:b795e752:::benchmark_data_smithi070_191495_object32451:head,463'2012,11:b79632fc:::benchmark_data_smithi070_191495_object71044:head,468'4435,11:b797e604:::benchmark_data_smithi070_191495_object70192:head,468'4381,11:b7993be5:::benchmark_data_smithi070_191495_object50452:head,465'3166,11:b7995a9e:::benchmark_data_smithi070_191495_object55003:head,466'3462,11:b799895e:::benchmark_data_smithi070_191495_object35815:head,464'2227,11:b799baa9:::benchmark_data_smithi070_191495_object80506:head,472'5093,11:b79a6ec5:::benchmark_data_smithi070_191495_object50647:head,465'3183,11:b79bd500:::benchmark_data_smithi070_191495_object113984:head,486'7229,11:b79c372d:::benchmark_data_smithi070_191495_object85880:head,474'5435,11:b79c5029:::benchmark_data_smithi070_191495_object107995:head,482'6840,11:b79d304c:::benchmark_data_smithi070_191495_object27531:head,463'1718,11:b79d6dd2:::benchmark_data_smithi070_191495_object49997:head,465'3136,11:b79ecf67:::benchmark_data_smithi070_191495_object43639:head,464'2725,11:b79f0294:::benchmark_data_smithi070_191495_object104143:head,480'6613,11:b79fa8f6:::benchmark_data_smithi070_191495_object108501:head,483'6876,11:b79fcbae:::benchmark_data_smithi070_191495_object119508:head,488'7581,11:b7a0d119:::benchmark_data_smithi070_191495_object127541:head,491'8089,11:b7a0dd3f:::benchmark_data_smithi070_191495_object116817:head,486'7403,11:b7a15e40:::benchmark_data_smithi070_191495_object41909:head,464'2611,11:b7a1e55a:::benchmark_data_smithi070_191495_object55450:head,466'3489,11:b7a20ed8:::benchmark_data_smithi070_191495_object21547:head,461'1363,11:b7a2dcf6:::benchmark_data_smithi070_191495_object81539:head,473'5161,11:b7a2e06c:::benchmark_data_smithi070_191495_object77721:head,471'4916,11:b7a2e4e3:::benchmark_data_smithi070_191495_object22656:head,461'1427,11:b7a2f516:::benchmark_data_smithi070_191495_object105386:head,481'6687,11:b7a3008c:::benchmark_data_smithi070_191495_object82038:head,474'5196,11:b7a389e3:::benchmark_data_smithi070_191495_object98453:head,479'6249,11:b7a3c157:::benchmark_data_smithi070_191495_object112221:head,485'7119,11:b7a3f354:::benchmark_data_smithi070_191495_object68538:head,468'4274,11:b7a40a63:::benchmark_data_smithi070_191495_object65028:head,467'4084,11:b7a46514:::benchmark_data_smithi070_191495_object112690:head,485'7152,11:b7a54912:::benchmark_data_smithi070_191495_object61787:head,467'3874,11:b7a731d6:::benchmark_data_smithi070_191495_object97480:head,479'6190,11:b7a848b8:::benchmark_data_smithi070_191495_object35586:head,464'2212,11:b7a865a1:::benchmark_data_smithi070_191495_object63089:head,467'3944,11:b7a9376d:::benchmark_data_smithi070_191495_object111593:head,484'7071,11:b7a939b7:::benchmark_data_smithi070_191495_object107737:head,482'6822,11:b7aaa272:::benchmark_data_smithi070_191495_object78574:head,472'4967,11:b7aac0d9:::benchmark_data_smithi070_191495_object85918:head,474'5437,11:b7ab3e15:::benchmark_data_smithi070_191495_object39851:head,464'2487,11:b7ab3ffe:::benchmark_data_smithi070_191495_object32052:head,463'1989,11:b7ab8657:::benchmark_data_smithi070_191495_object56297:head,466'3544,11:b7abe2f5:::benchmark_data_smithi070_191495_object100552:head,479'6387,11:b7abf3e0:::benchmark_data_smithi070_191495_object93219:head,477'5919,11:b7ac196a:::benchmark_data_smithi070_191495_object94940:head,478'6016,11:b7acc915:::benchmark_data_smithi070_191495_object96437:head,478'6117,11:b7acf17b:::benchmark_data_smithi070_191495_object81785:head,473'5180,11:b7ad5b1c:::benchmark_data_smithi070_191495_object25132:head,462'1584,11:b7ada06f:::benchmark_data_smithi070_191495_object74602:head,470'4675,11:b7aeaa4a:::benchmark_data_smithi070_191495_object21576:head,461'1364,11:b7aeaf3b:::benchmark_data_smithi070_191495_object71107:head,468'4440,11:b7aecdb4:::benchmark_data_smithi070_191495_object40427:head,464'2518,11:b7aed3ee:::benchmark_data_smithi070_191495_object46242:head,464'2881,11:b7aedb46:::benchmark_data_smithi070_191495_object119701:head,488'7596,11:b7af42d5:::benchmark_data_smithi070_191495_object54454:head,466'3421,11:b7af8ee6:::benchmark_data_smithi070_191495_object91823:head,476'5812,11:b7afcf1d:::benchmark_data_smithi070_191495_object119620:head,488'7588,11:b7afd431:::benchmark_data_smithi070_191495_object66684:head,468'4167,11:b7b00cb5:::benchmark_data_smithi070_191495_object17786:head,460'1141,11:b7b0d565:::benchmark_data_smithi070_191495_object91513:head,476'5785,11:b7b0d57b:::benchmark_data_smithi070_191495_object87819:head,474'5569,11:b7b18d2e:::benchmark_data_smithi070_191495_object42587:head,464'2665,11:b7b2139d:::benchmark_data_smithi070_191495_object99099:head,479'6292,11:b7b227fe:::benchmark_data_smithi070_191495_object23231:head,462'1467,11:b7b259c5:::benchmark_data_smithi070_191495_object36891:head,464'2309,11:b7b25f2e:::benchmark_data_smithi070_191495_object115505:head,486'7316,11:b7b29dd2:::benchmark_data_smithi070_191495_object100558:head,479'6388,11:b7b34bc8:::benchmark_data_smithi070_191495_object61347:head,467'3851,11:b7b3a2e8:::benchmark_data_smithi070_191495_object98991:head,479'6284,11:b7b4430d:::benchmark_data_smithi070_191495_object39115:head,464'2442,11:b7b54395:::benchmark_data_smithi070_191495_object56970:head,466'3589,11:b7b5b59d:::benchmark_data_smithi070_191495_object42203:head,464'2634,11:b7b6237d:::benchmark_data_smithi070_191495_object109590:head,483'6950,11:b7b64380:::benchmark_data_smithi070_191495_object118202:head,487'7494,11:b7b65f45:::benchmark_data_smithi070_191495_object111212:head,484'7035,11:b7b7c3cd:::benchmark_data_smithi070_191495_object33109:head,463'2065,11:b7b7d2ee:::benchmark_data_smithi070_191495_object49554:head,465'3114,11:b7b7e92f:::benchmark_data_smithi070_191495_object48656:head,465'3052,11:b7b81d8e:::benchmark_data_smithi070_191495_object30320:head,463'1888,11:b7b90abf:::benchmark_data_smithi070_191495_object104742:head,480'6647,11:b7b96b03:::benchmark_data_smithi070_191495_object76423:head,471'4813,11:b7ba4568:::benchmark_data_smithi070_191495_object44215:head,464'2758,11:b7ba7d7a:::benchmark_data_smithi070_191495_object56670:head,466'3559,11:b7bad480:::benchmark_data_smithi070_191495_object57458:head,466'3615,11:b7bc0d8c:::benchmark_data_smithi070_191495_object39235:head,464'2451,11:b7bc1493:::benchmark_data_smithi070_191495_object16280:head,460'1030,11:b7be16b2:::benchmark_data_smithi070_191495_object101486:head,479'6444,11:b7be20fc:::benchmark_data_smithi070_191495_object110544:head,483'7006,11:b7bf6199:::benchmark_data_smithi070_191495_object82290:head,474'5209,11:b7c055d2:::benchmark_data_smithi070_191495_object113398:head,486'7196,11:b7c0698f:::benchmark_data_smithi070_191495_object37130:head,464'2324,11:b7c12b9e:::benchmark_data_smithi070_191495_object26042:head,463'1631,11:b7c1e22f:::benchmark_data_smithi070_191495_object74897:head,470'4699,11:b7c3a805:::benchmark_data_smithi070_191495_object42213:head,464'2636,11:b7c41ad1:::benchmark_data_smithi070_191495_object43749:head,464'2735,11:b7c4d12e:::benchmark_data_smithi070_191495_object67933:head,468'4247,11:b7c6114f:::benchmark_data_smithi070_191495_object114996:head,486'7292,11:b7c6db2c:::benchmark_data_smithi070_191495_object86061:head,474'5448,11:b7c7baa8:::benchmark_data_smithi070_191495_object117508:head,486'7444,11:b7c894f0:::benchmark_data_smithi070_191495_object78193:head,472'4943,11:b7c907be:::benchmark_data_smithi070_191495_object78137:head,472'4938,11:b7c9606c:::benchmark_data_smithi070_191495_object82656:head,474'5236,11:b7c97cba:::benchmark_data_smithi070_191495_object114890:head,486'7284,11:b7c9c2da:::benchmark_data_smithi070_191495_object90513:head,476'5728,11:b7ca3ae1:::benchmark_data_smithi070_191495_object70214:head,468'4385,11:b7ca4e1e:::benchmark_data_smithi070_191495_object115092:head,486'7296,11:b7cc23b9:::benchmark_data_smithi070_191495_object99315:head,479'6304,11:b7ccee53:::benchmark_data_smithi070_191495_object60619:head,467'3793,11:b7ccf69a:::benchmark_data_smithi070_191495_object63874:head,467'3998,11:b7cf9298:::benchmark_data_smithi070_191495_object65372:head,467'4103,11:b7d025e5:::benchmark_data_smithi070_191495_object107811:head,482'6827,11:b7d0a2d1:::benchmark_data_smithi070_191495_object128057:head,491'8116,11:b7d0b7b0:::benchmark_data_smithi070_191495_object34893:head,464'2176,11:b7d0d6b3:::benchmark_data_smithi070_191495_object100455:head,479'6381,11:b7d182c5:::benchmark_data_smithi070_191495_object29770:head,463'1856,11:b7d1c496:::benchmark_data_smithi070_191495_object91566:head,476'5790,11:b7d1d4c7:::benchmark_data_smithi070_191495_object77303:head,471'4888,11:b7d28684:::benchmark_data_smithi070_191495_object75611:head,471'4747,11:b7d28e4b:::benchmark_data_smithi070_191495_object93299:head,477'5924,11:b7d3325b:::benchmark_data_smithi070_191495_object34263:head,464'2136,11:b7d43eb3:::benchmark_data_smithi070_191495_object81190:head,473'5139,11:b7d46bb7:::benchmark_data_smithi070_191495_object115918:head,486'7343,11:b7d4bf35:::benchmark_data_smithi070_191495_object24621:head,462'1556,11:b7d57b06:::benchmark_data_smithi070_191495_object69859:head,468'4357,11:b7d5ba9e:::benchmark_data_smithi070_191495_object92594:head,476'5873,11:b7d5e4da:::benchmark_data_smithi070_191495_object72229:head,468'4514,11:b7d676bb:::benchmark_data_smithi070_191495_object67488:head,468'4217,11:b7d6ffd7:::benchmark_data_smithi070_191495_object119569:head,488'7585,11:b7d87fd5:::benchmark_data_smithi070_191495_object53550:head,466'3358,11:b7d90247:::benchmark_data_smithi070_191495_object120026:head,488'7611,11:b7d92a7e:::benchmark_data_smithi070_191495_object30354:head,463'1891,11:b7d9416d:::benchmark_data_smithi070_191495_object87392:head,474'5539,11:b7d9b327:::benchmark_data_smithi070_191495_object36351:head,464'2264,11:b7da18f0:::benchmark_data_smithi070_191495_object35646:head,464'2216,11:b7da8576:::benchmark_data_smithi070_191495_object58879:head,466'3696,11:b7da8618:::benchmark_data_smithi070_191495_object111956:head,484'7105,11:b7db3ecb:::benchmark_data_smithi070_191495_object119136:head,487'7562,11:b7db9e34:::benchmark_data_smithi070_191495_object97428:head,479'6187,11:b7dc696e:::benchmark_data_smithi070_191495_object36417:head,464'2271,11:b7dd2e71:::benchmark_data_smithi070_191495_object24079:head,462'1516,11:b7dd3f6a:::benchmark_data_smithi070_191495_object79448:head,472'5025,11:b7ddfe0d:::benchmark_data_smithi070_191495_object93598:head,477'5935,11:b7df4d29:::benchmark_data_smithi070_191495_object42144:head,464'2628,11:b7e0700f:::benchmark_data_smithi070_191495_object92123:head,476'5836,11:b7e0d597:::benchmark_data_smithi070_191495_object69177:head,468'4320,11:b7e134e7:::benchmark_data_smithi070_191495_object63982:head,467'4005,11:b7e18f28:::benchmark_data_smithi070_191495_object80654:head,473'5103,11:b7e21d72:::benchmark_data_smithi070_191495_object63471:head,467'3974,11:b7e285bd:::benchmark_data_smithi070_191495_object129971:head,491'8244,11:b7e2f732:::benchmark_data_smithi070_191495_object32817:head,463'2044,11:b7e34b67:::benchmark_data_smithi070_191495_object20960:head,461'1332,11:b7e37b8a:::benchmark_data_smithi070_191495_object40437:head,464'2519,11:b7e3d0ec:::benchmark_data_smithi070_191495_object108191:head,482'6854,11:b7e44f46:::benchmark_data_smithi070_191495_object107519:head,482'6811,11:b7e46e1b:::benchmark_data_smithi070_191495_object99070:head,479'6289,11:b7e577b3:::benchmark_data_smithi070_191495_object44981:head,464'2806,11:b7e72cf0:::benchmark_data_smithi070_191495_object75710:head,471'4755,11:b7e7443a:::benchmark_data_smithi070_191495_object84880:head,474'5355,11:b7e77a7c:::benchmark_data_smithi070_191495_object45998:head,464'2873,11:b7e7c3be:::benchmark_data_smithi070_191495_object47068:head,464'2939,11:b7e81188:::benchmark_data_smithi070_191495_object36667:head,464'2285,11:b7e9433f:::benchmark_data_smithi070_191495_object48040:head,465'3007,11:b7e965a3:::benchmark_data_smithi070_191495_object50925:head,465'3196,11:b7e9f2f8:::benchmark_data_smithi070_191495_object56549:head,466'3556,11:b7ea1895:::benchmark_data_smithi070_191495_object96220:head,478'6096,11:b7eab19b:::benchmark_data_smithi070_191495_object72400:head,468'4529,11:b7eac174:::benchmark_data_smithi070_191495_object39615:head,464'2470,11:b7eb86a0:::benchmark_data_smithi070_191495_object106516:head,481'6749,11:b7ebf1e4:::benchmark_data_smithi070_191495_object80398:head,472'5087,11:b7eca6fe:::benchmark_data_smithi070_191495_object128804:head,491'8172,11:b7ecfb07:::benchmark_data_smithi070_191495_object102943:head,480'6531,11:b7ed239d:::benchmark_data_smithi070_191495_object31637:head,463'1959,11:b7ed7256:::benchmark_data_smithi070_191495_object88682:head,474'5617,11:b7ed7a14:::benchmark_data_smithi070_191495_object73353:head,469'4593,11:b7eec239:::benchmark_data_smithi070_191495_object35373:head,464'2204,11:b7eefd68:::benchmark_data_smithi070_191495_object25895:head,463'1621,11:b7ef91ad:::benchmark_data_smithi070_191495_object41927:head,464'2614,11:b7efd0c8:::benchmark_data_smithi070_191495_object117477:head,486'7442,11:b7f022c3:::benchmark_data_smithi070_191495_object38899:head,464'2426,11:b7f0bc4a:::benchmark_data_smithi070_191495_object49456:head,465'3105,11:b7f2b087:::benchmark_data_smithi070_191495_object59973:head,467'3756,11:b7f526f5:::benchmark_data_smithi070_191495_object97564:head,479'6201,11:b7f71709:::benchmark_data_smithi070_191495_object24028:head,462'1512,11:b7f75b4d:::benchmark_data_smithi070_191495_object54480:head,466'3422,11:b7f75ccf:::benchmark_data_smithi070_191495_object88054:head,474'5578,11:b7f78b34:::benchmark_data_smithi070_191495_object32207:head,463'2000,11:b7f7c7a3:::benchmark_data_smithi070_191495_object126504:head,491'8024,11:b7f7d969:::benchmark_data_smithi070_191495_object51288:head,465'3219,11:b7f7fc8b:::benchmark_data_smithi070_191495_object56916:head,466'3582,11:b7f84979:::benchmark_data_smithi070_191495_object54700:head,466'3438,11:b7f86f84:::benchmark_data_smithi070_191495_object113743:head,486'7215,11:b7f8a727:::benchmark_data_smithi070_191495_object22318:head,461'1409,11:b7f9dafd:::benchmark_data_smithi070_191495_object120791:head,488'7657,11:b7fa9c98:::benchmark_data_smithi070_191495_object120330:head,488'7625,11:b7fae812:::benchmark_data_smithi070_191495_object24407:head,462'1541,11:b7fb343d:::benchmark_data_smithi070_191495_object122340:head,488'7765,11:b7fbfbf6:::benchmark_data_smithi070_191495_object101588:head,479'6450,11:b7fc17f7:::benchmark_data_smithi070_191495_object60842:head,467'3812,11:b7fcce19:::benchmark_data_smithi070_191495_object25564:head,463'1600,11:b7fcd1ff:::benchmark_data_smithi070_191495_object19789:head,460'1274,11:b7fe2081:::benchmark_data_smithi070_191495_object21457:head,461'1357,11:b800833a:::benchmark_data_smithi070_191495_object84200:head,474'5319)", 2024-09-16T09:44:44.694 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:44:26.415533+0000", 2024-09-16T09:44:44.694 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 18.216499935000002, 2024-09-16T09:44:44.694 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.012118782999999999, 2024-09-16T09:44:44.694 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:44:44.695 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:44:44.695 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T09:44:44.695 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:44:44.695 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.695 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:44:44.695 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.415533+0000", 2024-09-16T09:44:44.695 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:44:44.695 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.695 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.695 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:44:44.695 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.415533+0000", 2024-09-16T09:44:44.695 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:44:44.695 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.696 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.696 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:44:44.696 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.415534+0000", 2024-09-16T09:44:44.696 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.7800000000000001e-07 2024-09-16T09:44:44.696 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.696 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.696 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:44:44.696 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.415778+0000", 2024-09-16T09:44:44.696 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00024390300000000001 2024-09-16T09:44:44.696 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.696 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.696 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:44:44.696 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.416251+0000", 2024-09-16T09:44:44.697 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00047290500000000003 2024-09-16T09:44:44.697 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.697 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.697 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:44:44.697 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.417243+0000", 2024-09-16T09:44:44.697 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00099245699999999997 2024-09-16T09:44:44.697 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.697 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.697 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:44:44.697 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.424900+0000", 2024-09-16T09:44:44.697 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0076566560000000004 2024-09-16T09:44:44.697 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.697 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.698 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:44:44.698 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.425446+0000", 2024-09-16T09:44:44.698 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00054615499999999999 2024-09-16T09:44:44.698 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.698 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.698 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:44:44.698 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.427652+0000", 2024-09-16T09:44:44.698 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0022061289999999998 2024-09-16T09:44:44.698 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:44:44.698 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:44:44.698 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:44:44.698 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.698 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.699 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "backfill_remove(11.4 e519 11:26d11479:::benchmark_data_smithi070_191495_object111505:head,484'6994,11:26d12638:::benchmark_data_smithi070_191495_object62645:head,467'3942,11:26d1bbf4:::benchmark_data_smithi070_191495_object65859:head,467'4155,11:26d39f4d:::benchmark_data_smithi070_191495_object126444:head,491'7916,11:26d47779:::benchmark_data_smithi070_191495_object75120:head,470'4728,11:26d5391f:::benchmark_data_smithi070_191495_object52450:head,466'3262,11:26d5e9e1:::benchmark_data_smithi070_191495_object23439:head,462'1518,11:26d685c8:::benchmark_data_smithi070_191495_object90422:head,476'5671,11:26d7098a:::benchmark_data_smithi070_191495_object56508:head,466'3518,11:26d84f26:::benchmark_data_smithi070_191495_object74998:head,470'4719,11:26d8a591:::benchmark_data_smithi070_191495_object58168:head,466'3625,11:26d8eecc:::benchmark_data_smithi070_191495_object111653:head,484'7010,11:26d9536f:::benchmark_data_smithi070_191495_object61639:head,467'3866,11:26d9ad24:::benchmark_data_smithi070_191495_object34525:head,464'2182,11:26dab6ca:::benchmark_data_smithi070_191495_object42979:head,464'2715,11:26dc63ea:::benchmark_data_smithi070_191495_object32156:head,463'2043,11:26dcc778:::benchmark_data_smithi070_191495_object99522:head,479'6257,11:26dcf07a:::benchmark_data_smithi070_191495_object61891:head,467'3880,11:26dd3c30:::benchmark_data_smithi070_191495_object64141:head,467'4038,11:26ddaabb:::benchmark_data_smithi070_191495_object124197:head,489'7760,11:26dead3f:::benchmark_data_smithi070_191495_object30828:head,463'1959,11:26debff5:::benchmark_data_smithi070_191495_object39816:head,464'2536,11:26df0113:::benchmark_data_smithi070_191495_object112004:head,485'7034,11:26df3563:::benchmark_data_smithi070_191495_object19623:head,460'1261,11:26df36a2:::benchmark_data_smithi070_191495_object33024:head,463'2096,11:26e05647:::benchmark_data_smithi070_191495_object24725:head,462'1609,11:26e31eee:::benchmark_data_smithi070_191495_object115195:head,486'7200,11:26e3db3d:::benchmark_data_smithi070_191495_object96037:head,478'6053,11:26e3fffa:::benchmark_data_smithi070_191495_object84197:head,474'5296,11:26e55f5c:::benchmark_data_smithi070_191495_object42683:head,464'2694,11:26e5cbc9:::benchmark_data_smithi070_191495_object97441:head,479'6141,11:26e5de1b:::benchmark_data_smithi070_191495_object119102:head,487'7442,11:26e5eabb:::benchmark_data_smithi070_191495_object39181:head,464'2490,11:26e6a9f9:::benchmark_data_smithi070_191495_object80226:head,472'5027,11:26e7780f:::benchmark_data_smithi070_191495_object84492:head,474'5310,11:26e7a358:::benchmark_data_smithi070_191495_object84245:head,474'5300,11:26e7b97d:::benchmark_data_smithi070_191495_object71553:head,468'4512,11:26e84343:::benchmark_data_smithi070_191495_object50350:head,465'3128,11:26e84734:::benchmark_data_smithi070_191495_object69801:head,468'4403,11:26e87216:::benchmark_data_smithi070_191495_object25409:head,463'1657,11:26e88771:::benchmark_data_smithi070_191495_object31552:head,463'2017,11:26e895d6:::benchmark_data_smithi070_191495_object56879:head,466'3547,11:26e8b8bd:::benchmark_data_smithi070_191495_object65523:head,467'4130,11:26e9ccc4:::benchmark_data_smithi070_191495_object81328:head,473'5114,11:26ea6d1a:::benchmark_data_smithi070_191495_object59189:head,466'3707,11:26eaf614:::benchmark_data_smithi070_191495_object96516:head,479'6087,11:26eafb75:::benchmark_data_smithi070_191495_object93285:head,477'5863,11:26eb58e7:::benchmark_data_smithi070_191495_object113121:head,485'7086,11:26eb76da:::benchmark_data_smithi070_191495_object126769:head,491'7947,11:26eb7cbc:::benchmark_data_smithi070_191495_object47538:head,464'2975,11:26ed42f1:::benchmark_data_smithi070_191495_object19269:head,460'1245,11:26edcf05:::benchmark_data_smithi070_191495_object51319:head,465'3183,11:26ee6094:::benchmark_data_smithi070_191495_object69914:head,468'4409,11:26ef72e4:::benchmark_data_smithi070_191495_object95566:head,478'6030,11:26eff993:::benchmark_data_smithi070_191495_object39690:head,464'2532,11:26f00dae:::benchmark_data_smithi070_191495_object97026:head,479'6115,11:26f020ec:::benchmark_data_smithi070_191495_object100631:head,479'6326,11:26f065b2:::benchmark_data_smithi070_191495_object36838:head,464'2335,11:26f0bf94:::benchmark_data_smithi070_191495_object33434:head,463'2122,11:26f22d82:::benchmark_data_smithi070_191495_object66341:head,467'4184,11:26f23f9f:::benchmark_data_smithi070_191495_object24213:head,462'1576,11:26f25ad7:::benchmark_data_smithi070_191495_object102865:head,480'6458,11:26f37d4d:::benchmark_data_smithi070_191495_object34274:head,464'2166,11:26f3a404:::benchmark_data_smithi070_191495_object90706:head,476'5689,11:26f3d346:::benchmark_data_smithi070_191495_object69598:head,468'4392,11:26f5282e:::benchmark_data_smithi070_191495_object32491:head,463'2062,11:26f52b22:::benchmark_data_smithi070_191495_object29697:head,463'1896,11:26f67333:::benchmark_data_smithi070_191495_object79253:head,472'4967,11:26f6bd68:::benchmark_data_smithi070_191495_object59071:head,466'3698,11:26f70005:::benchmark_data_smithi070_191495_object49146:head,465'3057,11:26f7039e:::benchmark_data_smithi070_191495_object72322:head,468'4568,11:26f72d47:::benchmark_data_smithi070_191495_object113373:head,486'7099,11:26f7807c:::benchmark_data_smithi070_191495_object84198:head,474'5297,11:26f907af:::benchmark_data_smithi070_191495_object29905:head,463'1908,11:26f93e3c:::benchmark_data_smithi070_191495_object100902:head,479'6338,11:26fa2c5e:::benchmark_data_smithi070_191495_object102486:head,480'6439,11:26fac17c:::benchmark_data_smithi070_191495_object52260:head,465'3247,11:26fadfbf:::benchmark_data_smithi070_191495_object129112:head,491'8077,11:26fbe251:::benchmark_data_smithi070_191495_object113304:head,486'7095,11:26fcc6b7:::benchmark_data_smithi070_191495_object98822:head,479'6216,11:26fd0ed9:::benchmark_data_smithi070_191495_object62621:head,467'3939,11:26fd573e:::benchmark_data_smithi070_191495_object64505:head,467'4063,11:26fd7c7d:::benchmark_data_smithi070_191495_object92494:head,476'5803,11:26feae9b:::benchmark_data_smithi070_191495_object116980:head,486'7315,11:26ffe936:::benchmark_data_smithi070_191495_object110781:head,484'6941,11:27000f9a:::benchmark_data_smithi070_191495_object110658:head,483'6932,11:27001dce:::benchmark_data_smithi070_191495_object60335:head,467'3778,11:2700cff1:::benchmark_data_smithi070_191495_object58600:head,466'3652,11:27016ef8:::benchmark_data_smithi070_191495_object60550:head,467'3795,11:2702484d:::benchmark_data_smithi070_191495_object33191:head,463'2105,11:27038a21:::benchmark_data_smithi070_191495_object114310:head,486'7144,11:27047dab:::benchmark_data_smithi070_191495_object98693:head,479'6205,11:27055bfe:::benchmark_data_smithi070_191495_object89503:head,475'5620,11:270696d9:::benchmark_data_smithi070_191495_object54618:head,466'3394,11:2706b916:::benchmark_data_smithi070_191495_object77442:head,471'4855,11:270720fe:::benchmark_data_smithi070_191495_object78154:head,472'4893,11:27074598:::benchmark_data_smithi070_191495_object39623:head,464'2525,11:2708126c:::benchmark_data_smithi070_191495_object115626:head,486'7233,11:27082ae9:::benchmark_data_smithi070_191495_object101786:head,480'6393,11:2708bd22:::benchmark_data_smithi070_191495_object106472:head,481'6663,11:2708ebc1:::benchmark_data_smithi070_191495_object63014:head,467'3968,11:2709e11a:::benchmark_data_smithi070_191495_object38746:head,464'2471,11:270aa2af:::benchmark_data_smithi070_191495_object35535:head,464'2263,11:270c1037:::benchmark_data_smithi070_191495_object75102:head,470'4726,11:270c5af9:::benchmark_data_smithi070_191495_object91860:head,476'5757,11:270c7323:::benchmark_data_smithi070_191495_object97814:head,479'6155,11:270cb241:::benchmark_data_smithi070_191495_object129702:head,491'8111,11:270de8f4:::benchmark_data_smithi070_191495_object37588:head,464'2383,11:270e822e:::benchmark_data_smithi070_191495_object54263:head,466'3375,11:270eab2b:::benchmark_data_smithi070_191495_object41793:head,464'2651,11:271088b7:::benchmark_data_smithi070_191495_object66622:head,468'4197,11:2710aad9:::benchmark_data_smithi070_191495_object129834:head,491'8121,11:2711865b:::benchmark_data_smithi070_191495_object75522:head,471'4746,11:2711e21b:::benchmark_data_smithi070_191495_object89004:head,474'5581,11:2712511c:::benchmark_data_smithi070_191495_object59088:head,466'3701,11:2712e2fd:::benchmark_data_smithi070_191495_object111281:head,484'6977,11:2713bad4:::benchmark_data_smithi070_191495_object19937:head,460'1280,11:2713cae6:::benchmark_data_smithi070_191495_object16656:head,460'1078,11:2713d3ad:::benchmark_data_smithi070_191495_object117136:head,486'7328,11:27152a90:::benchmark_data_smithi070_191495_object82893:head,474'5227,11:27156f13:::benchmark_data_smithi070_191495_object95009:head,478'5994,11:27158298:::benchmark_data_smithi070_191495_object29670:head,463'1892,11:27159c97:::benchmark_data_smithi070_191495_object64522:head,467'4065,11:2715d766:::benchmark_data_smithi070_191495_object129974:head,491'8127,11:271679a8:::benchmark_data_smithi070_191495_object44290:head,464'2786,11:27167eed:::benchmark_data_smithi070_191495_object20782:head,460'1334,11:2717604f:::benchmark_data_smithi070_191495_object36925:head,464'2339,11:271840c6:::benchmark_data_smithi070_191495_object96473:head,479'6083,11:2718c6a1:::benchmark_data_smithi070_191495_object42300:head,464'2677,11:27193c28:::benchmark_data_smithi070_191495_object40688:head,464'2592,11:271a7dee:::benchmark_data_smithi070_191495_object69486:head,468'4385,11:271ab73f:::benchmark_data_smithi070_191495_object40819:head,464'2603,11:271bc4e1:::benchmark_data_smithi070_191495_object28443:head,463'1819,11:271c15e4:::benchmark_data_smithi070_191495_object112312:head,485'7049,11:271caf02:::benchmark_data_smithi070_191495_object102705:head,480'6449,11:271ce15c:::benchmark_data_smithi070_191495_object55640:head,466'3470,11:271d7689:::benchmark_data_smithi070_191495_object88180:head,474'5529,11:271d95eb:::benchmark_data_smithi070_191495_object48735:head,465'3039,11:271ee5cf:::benchmark_data_smithi070_191495_object107011:head,481'6697,11:271fc416:::benchmark_data_smithi070_191495_object69094:head,468'4357,11:2720d80c:::benchmark_data_smithi070_191495_object27046:head,463'1737,11:2721b3d8:::benchmark_data_smithi070_191495_object57615:head,466'3589,11:27228abf:::benchmark_data_smithi070_191495_object37806:head,464'2392,11:2722dc1b:::benchmark_data_smithi070_191495_object84585:head,474'5319,11:2724cee4:::benchmark_data_smithi070_191495_object78134:head,472'4891,11:272515e4:::benchmark_data_smithi070_191495_object37410:head,464'2372,11:2725318b:::benchmark_data_smithi070_191495_object33130:head,463'2102,11:2725bd05:::benchmark_data_smithi070_191495_object120120:head,488'7517,11:2725fbc3:::benchmark_data_smithi070_191495_object82727:head,474'5218,11:2726247a:::benchmark_data_smithi070_191495_object90298:head,475'5665,11:2726b583:::benchmark_data_smithi070_191495_object80311:head,472'5035,11:27270182:::benchmark_data_smithi070_191495_object61851:head,467'3878,11:272757e2:::benchmark_data_smithi070_191495_object69358:head,468'4375,11:27278500:::benchmark_data_smithi070_191495_object22993:head,462'1474,11:2729fcb9:::benchmark_data_smithi070_191495_object40805:head,464'2602,11:272ad9ee:::benchmark_data_smithi070_191495_object114330:head,486'7146,11:272aea45:::benchmark_data_smithi070_191495_object92816:head,476'5827,11:272b1642:::benchmark_data_smithi070_191495_object62271:head,467'3904,11:272b1893:::benchmark_data_smithi070_191495_object94392:head,478'5953,11:272d5b66:::benchmark_data_smithi070_191495_object124400:head,489'7771,11:272d8e40:::benchmark_data_smithi070_191495_object71904:head,468'4536,11:272dbca3:::benchmark_data_smithi070_191495_object82661:head,474'5210,11:272e7b9b:::benchmark_data_smithi070_191495_object32807:head,463'2078,11:272ffdf2:::benchmark_data_smithi070_191495_object27148:head,463'1751,11:2730b4c3:::benchmark_data_smithi070_191495_object26083:head,463'1698,11:2730dd19:::benchmark_data_smithi070_191495_object73222:head,469'4618,11:2730eb69:::benchmark_data_smithi070_191495_object40311:head,464'2562,11:27313813:::benchmark_data_smithi070_191495_object22701:head,461'1460,11:27319767:::benchmark_data_smithi070_191495_object47841:head,464'2991,11:27321cfe:::benchmark_data_smithi070_191495_object56789:head,466'3543,11:27331d0c:::benchmark_data_smithi070_191495_object121115:head,488'7584,11:27334269:::benchmark_data_smithi070_191495_object35748:head,464'2274,11:273470ad:::benchmark_data_smithi070_191495_object94690:head,478'5970,11:27348cbc:::benchmark_data_smithi070_191495_object34917:head,464'2215,11:2735ec8c:::benchmark_data_smithi070_191495_object61952:head,467'3884,11:2736120a:::benchmark_data_smithi070_191495_object45428:head,464'2864,11:2736a276:::benchmark_data_smithi070_191495_object122199:head,488'7639,11:27370250:::benchmark_data_smithi070_191495_object57528:head,466'3587,11:27377ebb:::benchmark_data_smithi070_191495_object126528:head,491'7925,11:273817e4:::benchmark_data_smithi070_191495_object88414:head,474'5541,11:2739fc57:::benchmark_data_smithi070_191495_object69277:head,468'4370,11:273ac78e:::benchmark_data_smithi070_191495_object20778:head,460'1333,11:273ae84f:::benchmark_data_smithi070_191495_object45022:head,464'2833,11:273b518c:::benchmark_data_smithi070_191495_object105654:head,481'6624,11:273b6f4d:::benchmark_data_smithi070_191495_object89035:head,474'5586,11:273c128f:::benchmark_data_smithi070_191495_object99578:head,479'6259,11:273ca7a4:::benchmark_data_smithi070_191495_object99821:head,479'6277,11:273cb83e:::benchmark_data_smithi070_191495_object84307:head,474'5303,11:273d652d:::benchmark_data_smithi070_191495_object66793:head,468'4213,11:273df4e0:::benchmark_data_smithi070_191495_object123154:head,488'7697,11:273e1316:::benchmark_data_smithi070_191495_object44707:head,464'2814,11:273e8954:::benchmark_data_smithi070_191495_object82141:head,474'5169,11:273ebcf6:::benchmark_data_smithi070_191495_object24544:head,462'1597,11:273f52d1:::benchmark_data_smithi070_191495_object56834:head,466'3544,11:27412c95:::benchmark_data_smithi070_191495_object19427:head,460'1254,11:2743a154:::benchmark_data_smithi070_191495_object87376:head,474'5489,11:2743f2aa:::benchmark_data_smithi070_191495_object101135:head,479'6353,11:27443774:::benchmark_data_smithi070_191495_object24347:head,462'1582,11:27447a79:::benchmark_data_smithi070_191495_object121624:head,488'7614,11:2744cfa3:::benchmark_data_smithi070_191495_object22785:head,461'1463,11:27461da8:::benchmark_data_smithi070_191495_object61615:head,467'3865,11:27465cbc:::benchmark_data_smithi070_191495_object117693:head,486'7359,11:2746f9ab:::benchmark_data_smithi070_191495_object128721:head,491'8058,11:27473f17:::benchmark_data_smithi070_191495_object96363:head,478'6075,11:2747e2d2:::benchmark_data_smithi070_191495_object75294:head,470'4738,11:27482253:::benchmark_data_smithi070_191495_object84491:head,474'5309,11:274867ff:::benchmark_data_smithi070_191495_object17329:head,460'1114,11:274935a8:::benchmark_data_smithi070_191495_object56957:head,466'3556,11:27495048:::benchmark_data_smithi070_191495_object85509:head,474'5383,11:274acb36:::benchmark_data_smithi070_191495_object28707:head,463'1835,11:274b0fa4:::benchmark_data_smithi070_191495_object104790:head,480'6570,11:274b6096:::benchmark_data_smithi070_191495_object38505:head,464'2455,11:274c1873:::benchmark_data_smithi070_191495_object66797:head,468'4214,11:274c1d6c:::benchmark_data_smithi070_191495_object76865:head,471'4828,11:274ccdd5:::benchmark_data_smithi070_191495_object73327:head,469'4623,11:274cddd5:::benchmark_data_smithi070_191495_object95090:head,478'5996,11:274d27af:::benchmark_data_smithi070_191495_object30646:head,463'1951,11:274e9380:::benchmark_data_smithi070_191495_object78755:head,472'4936,11:274ef2de:::benchmark_data_smithi070_191495_object87000:head,474'5470,11:274f287e:::benchmark_data_smithi070_191495_object80071:head,472'5011,11:274f654e:::benchmark_data_smithi070_191495_object33597:head,464'2130,11:274fa38f:::benchmark_data_smithi070_191495_object77795:head,471'4879,11:274fb9df:::benchmark_data_smithi070_191495_object116667:head,486'7295,11:27502165:::benchmark_data_smithi070_191495_object121305:head,488'7597,11:275096f7:::benchmark_data_smithi070_191495_object85198:head,474'5361,11:27520a53:::benchmark_data_smithi070_191495_object123074:head,488'7693,11:2752f451:::benchmark_data_smithi070_191495_object104109:head,480'6529,11:27550eb5:::benchmark_data_smithi070_191495_object117473:head,486'7348,11:275613fe:::benchmark_data_smithi070_191495_object96775:head,479'6102,11:2756d6bf:::benchmark_data_smithi070_191495_object78458:head,472'4915,11:27577ce2:::benchmark_data_smithi070_191495_object60407:head,467'3786,11:27579de4:::benchmark_data_smithi070_191495_object58634:head,466'3655,11:2758cc0b:::benchmark_data_smithi070_191495_object48340:head,465'3019,11:275a476e:::benchmark_data_smithi070_191495_object79582:head,472'4987,11:275a52f5:::benchmark_data_smithi070_191495_object53054:head,466'3297,11:275ac270:::benchmark_data_smithi070_191495_object97613:head,479'6148,11:275b0a95:::benchmark_data_smithi070_191495_object125833:head,490'7884,11:275b1cad:::benchmark_data_smithi070_191495_object94005:head,477'5923,11:275b4361:::benchmark_data_smithi070_191495_object54741:head,466'3400,11:275c5f33:::benchmark_data_smithi070_191495_object65643:head,467'4143,11:275ccf0a:::benchmark_data_smithi070_191495_object23962:head,462'1555,11:275d4479:::benchmark_data_smithi070_191495_object45887:head,464'2892,11:275d736e:::benchmark_data_smithi070_191495_object58663:head,466'3656,11:275d7622:::benchmark_data_smithi070_191495_object111726:head,484'7015,11:275dd311:::benchmark_data_smithi070_191495_object82480:head,474'5195,11:275f3ff4:::benchmark_data_smithi070_191495_object44058:head,464'2773,11:275f4060:::benchmark_data_smithi070_191495_object112613:head,485'7063,11:275f860f:::benchmark_data_smithi070_191495_object26901:head,463'1734,11:275fc561:::benchmark_data_smithi070_191495_object63810:head,467'4016,11:27602e82:::benchmark_data_smithi070_191495_object114616:head,486'7160,11:27603399:::benchmark_data_smithi070_191495_object68307:head,468'4309,11:27614169:::benchmark_data_smithi070_191495_object53468:head,466'3317,11:27614310:::benchmark_data_smithi070_191495_object71024:head,468'4479,11:27615843:::benchmark_data_smithi070_191495_object17408:head,460'1118,11:276281aa:::benchmark_data_smithi070_191495_object81989:head,474'5161,11:2762e370:::benchmark_data_smithi070_191495_object86118:head,474'5415,11:2763482c:::benchmark_data_smithi070_191495_object82453:head,474'5193,11:2764aed2:::benchmark_data_smithi070_191495_object24797:head,462'1617,11:27656d96:::benchmark_data_smithi070_191495_object80515:head,472'5057,11:27659113:::benchmark_data_smithi070_191495_object52428:head,466'3260,11:27678e30:::benchmark_data_smithi070_191495_object35198:head,464'2238,11:27682ca1:::benchmark_data_smithi070_191495_object87818:head,474'5509,11:27686845:::benchmark_data_smithi070_191495_object106785:head,481'6683,11:27690530:::benchmark_data_smithi070_191495_object74084:head,469'4662,11:27699581:::benchmark_data_smithi070_191495_object73650:head,469'4641,11:276ab176:::benchmark_data_smithi070_191495_object31289:head,463'2004,11:276be262:::benchmark_data_smithi070_191495_object27175:head,463'1755,11:276cbf52:::benchmark_data_smithi070_191495_object17057:head,460'1101,11:276d3741:::benchmark_data_smithi070_191495_object84572:head,474'5318,11:276d7ac5:::benchmark_data_smithi070_191495_object35356:head,464'2247,11:276d88e6:::benchmark_data_smithi070_191495_object64928:head,467'4093,11:276d92af:::benchmark_data_smithi070_191495_object78541:head,472'4920,11:276d985a:::benchmark_data_smithi070_191495_object59829:head,467'3745,11:276f1f0f:::benchmark_data_smithi070_191495_object126975:head,491'7952,11:276fe513:::benchmark_data_smithi070_191495_object82490:head,474'5197,11:27705a83:::benchmark_data_smithi070_191495_object23756:head,462'1537,11:2771bdbc:::benchmark_data_smithi070_191495_object49466:head,465'3082,11:27746570:::benchmark_data_smithi070_191495_object93063:head,477'5844,11:27746c61:::benchmark_data_smithi070_191495_object126284:head,491'7910,11:2774b7bd:::benchmark_data_smithi070_191495_object61572:head,467'3862,11:27761777:::benchmark_data_smithi070_191495_object23315:head,462'1510,11:27765495:::benchmark_data_smithi070_191495_object18310:head,460'1179,11:27767599:::benchmark_data_smithi070_191495_object18989:head,460'1222,11:2776d692:::benchmark_data_smithi070_191495_object52250:head,465'3244,11:27774673:::benchmark_data_smithi070_191495_object107397:head,482'6728,11:2778611e:::benchmark_data_smithi070_191495_object73128:head,469'4612,11:27792f28:::benchmark_data_smithi070_191495_object53815:head,466'3347,11:2779a84f:::benchmark_data_smithi070_191495_object56269:head,466'3504,11:277a2050:::benchmark_data_smithi070_191495_object74533:head,470'4690,11:277afa5e:::benchmark_data_smithi070_191495_object21024:head,461'1355,11:277bbc5a:::benchmark_data_smithi070_191495_object115914:head,486'7250,11:277be1ce:::benchmark_data_smithi070_191495_object35459:head,464'2258,11:277c20ed:::benchmark_data_smithi070_191495_object88679:head,474'5559,11:277de3c3:::benchmark_data_smithi070_191495_object62456:head,467'3924,11:277e6593:::benchmark_data_smithi070_191495_object123252:head,488'7703,11:277ec9f3:::benchmark_data_smithi070_191495_object52211:head,465'3243,11:277f4846:::benchmark_data_smithi070_191495_object37150:head,464'2357,11:277fd9e1:::benchmark_data_smithi070_191495_object124468:head,489'7781,11:27805445:::benchmark_data_smithi070_191495_object112944:head,485'7080,11:2780877e:::benchmark_data_smithi070_191495_object37956:head,464'2410,11:27809471:::benchmark_data_smithi070_191495_object89216:head,474'5600,11:27812fcd:::benchmark_data_smithi070_191495_object46418:head,464'2920,11:27814431:::benchmark_data_smithi070_191495_object26175:head,463'1701,11:2782a1ef:::benchmark_data_smithi070_191495_object82774:head,474'5219,11:2782cde3:::benchmark_data_smithi070_191495_object50616:head,465'3137,11:278691b3:::benchmark_data_smithi070_191495_object76743:head,471'4825,11:27869cff:::benchmark_data_smithi070_191495_object63552:head,467'4002,11:27870805:::benchmark_data_smithi070_191495_object29148:head,463'1863,11:2787bd08:::benchmark_data_smithi070_191495_object89301:head,474'5606,11:2787ecf2:::benchmark_data_smithi070_191495_object53754:head,466'3341,11:27887e5b:::benchmark_data_smithi070_191495_object67499:head,468'4251,11:27888740:::benchmark_data_smithi070_191495_object122419:head,488'7658,11:2788b53e:::benchmark_data_smithi070_191495_object56938:head,466'3552,11:278923d6:::benchmark_data_smithi070_191495_object68930:head,468'4348,11:27893695:::benchmark_data_smithi070_191495_object110605:head,483'6930,11:27894bbc:::benchmark_data_smithi070_191495_object111600:head,484'7002,11:278a18eb:::benchmark_data_smithi070_191495_object71617:head,468'4518,11:278a1fe1:::benchmark_data_smithi070_191495_object125663:head,490'7868,11:278b5a83:::benchmark_data_smithi070_191495_object56100:head,466'3498,11:278b8410:::benchmark_data_smithi070_191495_object41550:head,464'2641,11:278c9e26:::benchmark_data_smithi070_191495_object24204:head,462'1574,11:278cd88d:::benchmark_data_smithi070_191495_object108927:head,483'6837,11:278d009d:::benchmark_data_smithi070_191495_object120506:head,488'7546,11:278d6d22:::benchmark_data_smithi070_191495_object112048:head,485'7036,11:278de283:::benchmark_data_smithi070_191495_object35904:head,464'2286,11:278eefdb:::benchmark_data_smithi070_191495_object72066:head,468'4549,11:27905856:::benchmark_data_smithi070_191495_object111154:head,484'6964,11:27907da2:::benchmark_data_smithi070_191495_object69788:head,468'4402,11:2790e72d:::benchmark_data_smithi070_191495_object93095:head,477'5846,11:2790f311:::benchmark_data_smithi070_191495_object86445:head,474'5434,11:27915ef1:::benchmark_data_smithi070_191495_object24697:head,462'1606,11:27933c35:::benchmark_data_smithi070_191495_object66723:head,468'4209,11:2793b14a:::benchmark_data_smithi070_191495_object63616:head,467'4006,11:2793cca2:::benchmark_data_smithi070_191495_object75627:head,471'4753,11:27944c3a:::benchmark_data_smithi070_191495_object91464:head,476'5734,11:2794c7a4:::benchmark_data_smithi070_191495_object127505:head,491'7981,11:27957feb:::benchmark_data_smithi070_191495_object112739:head,485'7070,11:2796b83c:::benchmark_data_smithi070_191495_object36823:head,464'2333,11:27976f03:::benchmark_data_smithi070_191495_object102778:head,480'6453,11:27988fec:::benchmark_data_smithi070_191495_object17040:head,460'1099,11:279938f9:::benchmark_data_smithi070_191495_object45825:head,464'2881,11:279a0e33:::benchmark_data_smithi070_191495_object17585:head,460'1130,11:279a5970:::benchmark_data_smithi070_191495_object24362:head,462'1584,11:279a70d6:::benchmark_data_smithi070_191495_object118893:head,487'7431,11:279ba55a:::benchmark_data_smithi070_191495_object94954:head,478'5987,11:279c205e:::benchmark_data_smithi070_191495_object101792:head,480'6394,11:279c74ff:::benchmark_data_smithi070_191495_object17822:head,460'1149,11:279ce67d:::benchmark_data_smithi070_191495_object62990:head,467'3966,11:279d73f5:::benchmark_data_smithi070_191495_object50298:head,465'3125,11:279df650:::benchmark_data_smithi070_191495_object126615:head,491'7932,11:279fd474:::benchmark_data_smithi070_191495_object23238:head,462'1499,11:27a0127a:::benchmark_data_smithi070_191495_object55400:head,466'3454,11:27a0418c:::benchmark_data_smithi070_191495_object81366:head,473'5117,11:27a15cea:::benchmark_data_smithi070_191495_object91900:head,476'5761,11:27a1c8bb:::benchmark_data_smithi070_191495_object74837:head,470'4710,11:27a20531:::benchmark_data_smithi070_191495_object62480:head,467'3928,11:27a217dc:::benchmark_data_smithi070_191495_object39526:head,464'2518,11:27a47992:::benchmark_data_smithi070_191495_object125774:head,490'7876,11:27a49f03:::benchmark_data_smithi070_191495_object124132:head,489'7756,11:27a4d785:::benchmark_data_smithi070_191495_object92003:head,476'5766,11:27a4e6a4:::benchmark_data_smithi070_191495_object16865:head,460'1089,11:27a51f06:::benchmark_data_smithi070_191495_object125019:head,490'7819,11:27a636f4:::benchmark_data_smithi070_191495_object124667:head,490'7797,11:27a645b4:::benchmark_data_smithi070_191495_object35782:head,464'2276,11:27a64c59:::benchmark_data_smithi070_191495_object27892:head,463'1784,11:27a72ae3:::benchmark_data_smithi070_191495_object105538:head,481'6616,11:27a74f72:::benchmark_data_smithi070_191495_object55384:head,466'3452,11:27a7760f:::benchmark_data_smithi070_191495_object35131:head,464'2231,11:27a80c70:::benchmark_data_smithi070_191495_object16170:head,460'1049,11:27a9600c:::benchmark_data_smithi070_191495_object68996:head,468'4352,11:27a9bf01:::benchmark_data_smithi070_191495_object68505:head,468'4322,11:27aabef1:::benchmark_data_smithi070_191495_object59958:head,467'3752,11:27abd0ee:::benchmark_data_smithi070_191495_object84994:head,474'5349,11:27acb09b:::benchmark_data_smithi070_191495_object119562:head,488'7472,11:27accf50:::benchmark_data_smithi070_191495_object123169:head,488'7698,11:27ad818e:::benchmark_data_smithi070_191495_object107998:head,482'6776,11:27ad9f04:::benchmark_data_smithi070_191495_object79227:head,472'4962,11:27ae1571:::benchmark_data_smithi070_191495_object61937:head,467'3883,11:27ae749c:::benchmark_data_smithi070_191495_object91677:head,476'5751,11:27af24e1:::benchmark_data_smithi070_191495_object49651:head,465'3091,11:27b0e0af:::benchmark_data_smithi070_191495_object113944:head,486'7130,11:27b1f3a8:::benchmark_data_smithi070_191495_object62278:head,467'3906,11:27b228e3:::benchmark_data_smithi070_191495_object67586:head,468'4255,11:27b2c71a:::benchmark_data_smithi070_191495_object25153:head,462'1641,11:27b3092c:::benchmark_data_smithi070_191495_object61423:head,467'3853,11:27b4e6dc:::benchmark_data_smithi070_191495_object109480:head,483'6868,11:27b628b7:::benchmark_data_smithi070_191495_object43127:head,464'2721,11:27b69083:::benchmark_data_smithi070_191495_object21036:head,461'1356,11:27b7025b:::benchmark_data_smithi070_191495_object72022:head,468'4544,11:27b713a2:::benchmark_data_smithi070_191495_object30644:head,463'1950,11:27b83265:::benchmark_data_smithi070_191495_object80097:head,472'5015,11:27b83ad8:::benchmark_data_smithi070_191495_object44420:head,464'2791,11:27b8b30e:::benchmark_data_smithi070_191495_object86838:head,474'5461,11:27b94132:::benchmark_data_smithi070_191495_object122529:head,488'7666,11:27b99bd8:::benchmark_data_smithi070_191495_object88507:head,474'5546,11:27ba4b9d:::benchmark_data_smithi070_191495_object82399:head,474'5188,11:27bb7147:::benchmark_data_smithi070_191495_object50592:head,465'3136,11:27bd5a01:::benchmark_data_smithi070_191495_object69066:head,468'4353,11:27bdf263:::benchmark_data_smithi070_191495_object29045:head,463'1859,11:27beaef4:::benchmark_data_smithi070_191495_object127796:head,491'8005,11:27becf12:::benchmark_data_smithi070_191495_object34413:head,464'2173,11:27bf8b4f:::benchmark_data_smithi070_191495_object63985:head,467'4026,11:27bfa0b9:::benchmark_data_smithi070_191495_object107268:head,481'6716,11:27bfa1e7:::benchmark_data_smithi070_191495_object120449:head,488'7539,11:27bfaa1c:::benchmark_data_smithi070_191495_object34236:head,464'2162,11:27bfaaf5:::benchmark_data_smithi070_191495_object123923:head,489'7742,11:27c04088:::benchmark_data_smithi070_191495_object50958:head,465'3156,11:27c0562b:::benchmark_data_smithi070_191495_object108682:head,483'6823,11:27c07cab:::benchmark_data_smithi070_191495_object78852:head,472'4940,11:27c113a9:::benchmark_data_smithi070_191495_object25184:head,463'1643,11:27c1478e:::benchmark_data_smithi070_191495_object66359:head,467'4185,11:27c175c1:::benchmark_data_smithi070_191495_object64783:head,467'4083,11:27c19982:::benchmark_data_smithi070_191495_object88890:head,474'5573,11:27c1a41d:::benchmark_data_smithi070_191495_object33067:head,463'2099,11:27c1d7ac:::benchmark_data_smithi070_191495_object70049:head,468'4422,11:27c1ec79:::benchmark_data_smithi070_191495_object91414:head,476'5730,11:27c34adf:::benchmark_data_smithi070_191495_object79571:head,472'4986,11:27c3d59a:::benchmark_data_smithi070_191495_object106848:head,481'6687,11:27c50941:::benchmark_data_smithi070_191495_object90820:head,476'5695,11:27c580c1:::benchmark_data_smithi070_191495_object42855:head,464'2708,11:27c5fdb0:::benchmark_data_smithi070_191495_object93853:head,477'5915,11:27c70145:::benchmark_data_smithi070_191495_object54921:head,466'3410,11:27c81ca8:::benchmark_data_smithi070_191495_object124847:head,490'7804,11:27c8462f:::benchmark_data_smithi070_191495_object118978:head,487'7436,11:27c8aace:::benchmark_data_smithi070_191495_object121357:head,488'7602,11:27c8cdd1:::benchmark_data_smithi070_191495_object72241:head,468'4561,11:27ca424f:::benchmark_data_smithi070_191495_object63994:head,467'4027,11:27caa2b7:::benchmark_data_smithi070_191495_object130160:head,492'8141,11:27cacea3:::benchmark_data_smithi070_191495_object80556:head,473'5059,11:27cb7c24:::benchmark_data_smithi070_191495_object69201:head,468'4365,11:27cbbe63:::benchmark_data_smithi070_191495_object34700:head,464'2192,11:27ccdf5e:::benchmark_data_smithi070_191495_object25974:head,463'1688,11:27cd0290:::benchmark_data_smithi070_191495_object88450:head,474'5543,11:27cd606f:::benchmark_data_smithi070_191495_object71771:head,468'4527,11:27ce45bc:::benchmark_data_smithi070_191495_object130075:head,491'8135,11:27cea3af:::benchmark_data_smithi070_191495_object50152:head,465'3115,11:27cf1a6b:::benchmark_data_smithi070_191495_object98658:head,479'6200,11:27d18938:::benchmark_data_smithi070_191495_object56782:head,466'3541,11:27d21c06:::benchmark_data_smithi070_191495_object93529:head,477'5884,11:27d23680:::benchmark_data_smithi070_191495_object47362:head,464'2969,11:27d3df7c:::benchmark_data_smithi070_191495_object22077:head,461'1417,11:27d44607:::benchmark_data_smithi070_191495_object88396:head,474'5538,11:27d49ecc:::benchmark_data_smithi070_191495_object21879:head,461'1403,11:27d4ca6e:::benchmark_data_smithi070_191495_object116272:head,486'7269,11:27d64f9f:::benchmark_data_smithi070_191495_object59297:head,466'3714,11:27d6757b:::benchmark_data_smithi070_191495_object46379:head,464'2916,11:27d74b56:::benchmark_data_smithi070_191495_object30546:head,463'1943,11:27d7bb76:::benchmark_data_smithi070_191495_object72286:head,468'4563,11:27d86333:::benchmark_data_smithi070_191495_object81525:head,473'5128,11:27d8cd50:::benchmark_data_smithi070_191495_object52699:head,466'3275,11:27d96715:::benchmark_data_smithi070_191495_object74879:head,470'4714,11:27d9d367:::benchmark_data_smithi070_191495_object23647:head,462'1530,11:27da11f2:::benchmark_data_smithi070_191495_object60689:head,467'3802,11:27da6202:::benchmark_data_smithi070_191495_object27127:head,463'1749,11:27db9271:::benchmark_data_smithi070_191495_object55903:head,466'3489,11:27dc1447:::benchmark_data_smithi070_191495_object44825:head,464'2820,11:27dda1a6:::benchmark_data_smithi070_191495_object41342:head,464'2631,11:27df926d:::benchmark_data_smithi070_191495_object56742:head,466'3538,11:27dfc831:::benchmark_data_smithi070_191495_object119281:head,488'7458,11:27dfff05:::benchmark_data_smithi070_191495_object115639:head,486'7234,11:27e02854:::benchmark_data_smithi070_191495_object45041:head,464'2835,11:27e04d65:::benchmark_data_smithi070_191495_object57205:head,466'3572,11:27e18afc:::benchmark_data_smithi070_191495_object40745:head,464'2595,11:27e22de5:::benchmark_data_smithi070_191495_object42817:head,464'2704,11:27e29c69:::benchmark_data_smithi070_191495_object123218:head,488'7701,11:27e2d249:::benchmark_data_smithi070_191495_object120100:head,488'7514,11:27e2df28:::benchmark_data_smithi070_191495_object18705:head,460'1207,11:27e2edc1:::benchmark_data_smithi070_191495_object106301:head,481'6651,11:27e382c3:::benchmark_data_smithi070_191495_object51215:head,465'3172,11:27e3d2f1:::benchmark_data_smithi070_191495_object110276:head,483'6908,11:27e477e9:::benchmark_data_smithi070_191495_object17537:head,460'1126,11:27e5547f:::benchmark_data_smithi070_191495_object37457:head,464'2376,11:27e55ebf:::benchmark_data_smithi070_191495_object50466:head,465'3132,11:27e58028:::benchmark_data_smithi070_191495_object72990:head,469'4603,11:27e5a813:::benchmark_data_smithi070_191495_object54944:head,466'3412,11:27e5c74e:::benchmark_data_smithi070_191495_object47227:head,464'2960,11:27e5cf29:::benchmark_data_smithi070_191495_object121672:head,488'7617,11:27e5e456:::benchmark_data_smithi070_191495_object104165:head,480'6534,11:27e6d4f0:::benchmark_data_smithi070_191495_object76302:head,471'4799,11:27e85364:::benchmark_data_smithi070_191495_object43984:head,464'2768,11:27e99bed:::benchmark_data_smithi070_191495_object47078:head,464'2954,11:27e9a155:::benchmark_data_smithi070_191495_object34742:head,464'2199,11:27e9bf0e:::benchmark_data_smithi070_191495_object34440:head,464'2176,11:27ea7652:::benchmark_data_smithi070_191495_object91000:head,476'5709,11:27ea79ee:::benchmark_data_smithi070_191495_object87034:head,474'5473,11:27ead95a:::benchmark_data_smithi070_191495_object100670:head,479'6328,11:27ebeee9:::benchmark_data_smithi070_191495_object47315:head,464'2966,11:27ec4e11:::benchmark_data_smithi070_191495_object115447:head,486'7216,11:27ec86fc:::benchmark_data_smithi070_191495_object51826:head,465'3218,11:27ec9ad3:::benchmark_data_smithi070_191495_object32699:head,463'2070,11:27ed7551:::benchmark_data_smithi070_191495_object22934:head,462'1469,11:27ed9156:::benchmark_data_smithi070_191495_object93756:head,477'5907,11:27ed97bf:::benchmark_data_smithi070_191495_object43806:head,464'2754,11:27ef6c07:::benchmark_data_smithi070_191495_object71805:head,468'4528,11:27f0dae7:::benchmark_data_smithi070_191495_object106599:head,481'6675,11:27f10a53:::benchmark_data_smithi070_191495_object71921:head,468'4538,11:27f1c602:::benchmark_data_smithi070_191495_object16489:head,460'1067,11:27f2e0b4:::benchmark_data_smithi070_191495_object59639:head,466'3734,11:27f38c17:::benchmark_data_smithi070_191495_object58764:head,466'3667,11:27f3c591:::benchmark_data_smithi070_191495_object93147:head,477'5851,11:27f451a5:::benchmark_data_smithi070_191495_object115492:head,486'7223,11:27f4756e:::benchmark_data_smithi070_191495_object60360:head,467'3782,11:27f483e2:::benchmark_data_smithi070_191495_object23314:head,462'1509,11:27f494fd:::benchmark_data_smithi070_191495_object110493:head,483'6920,11:27f4e6c5:::benchmark_data_smithi070_191495_object28834:head,463'1840,11:27f64c3c:::benchmark_data_smithi070_191495_object99715:head,479'6269,11:27f6b98d:::benchmark_data_smithi070_191495_object22284:head,461'1433,11:27f784f2:::benchmark_data_smithi070_191495_object93781:head,477'5908,11:27f9d4dd:::benchmark_data_smithi070_191495_object54090:head,466'3361,11:27fa5fbe:::benchmark_data_smithi070_191495_object93036:head,477'5842,11:27fad27c:::benchmark_data_smithi070_191495_object18505:head,460'1193,11:27fb2e40:::benchmark_data_smithi070_191495_object57059:head,466'3562)", 2024-09-16T09:44:44.699 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:44:26.441674+0000", 2024-09-16T09:44:44.699 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 18.190359005000001, 2024-09-16T09:44:44.699 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.015198457, 2024-09-16T09:44:44.699 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:44:44.699 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:44:44.699 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T09:44:44.699 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:44:44.699 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.699 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:44:44.700 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.441674+0000", 2024-09-16T09:44:44.700 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:44:44.700 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.700 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.700 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:44:44.700 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.441674+0000", 2024-09-16T09:44:44.704 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:44:44.704 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.704 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.704 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:44:44.704 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.441675+0000", 2024-09-16T09:44:44.704 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.2900000000000003e-07 2024-09-16T09:44:44.704 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.704 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.704 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:44:44.704 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.441820+0000", 2024-09-16T09:44:44.704 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00014506 2024-09-16T09:44:44.705 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.705 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.705 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:44:44.705 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.442102+0000", 2024-09-16T09:44:44.705 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00028266400000000002 2024-09-16T09:44:44.705 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.705 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.705 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:44:44.705 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.442587+0000", 2024-09-16T09:44:44.705 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00048425600000000002 2024-09-16T09:44:44.705 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.705 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.705 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:44:44.706 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.454466+0000", 2024-09-16T09:44:44.706 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.011879014 2024-09-16T09:44:44.706 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.706 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.706 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:44:44.706 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.454924+0000", 2024-09-16T09:44:44.706 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00045822799999999998 2024-09-16T09:44:44.706 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.706 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.706 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:44:44.706 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.456873+0000", 2024-09-16T09:44:44.706 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0019486060000000001 2024-09-16T09:44:44.706 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:44:44.707 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:44:44.707 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:44:44.707 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.707 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.707 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "pg_backfill(progress 11.4 e 519/519 lb 11:27fb2e40:::benchmark_data_smithi070_191495_object57059:head)", 2024-09-16T09:44:44.707 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:44:26.443058+0000", 2024-09-16T09:44:44.707 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 18.188974650999999, 2024-09-16T09:44:44.707 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.013916085, 2024-09-16T09:44:44.707 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:44:44.707 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:44:44.708 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T09:44:44.708 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:44:44.708 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.708 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:44:44.708 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.443058+0000", 2024-09-16T09:44:44.708 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:44:44.708 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.708 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.708 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:44:44.708 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.443058+0000", 2024-09-16T09:44:44.708 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:44:44.708 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.708 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.709 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:44:44.709 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.443059+0000", 2024-09-16T09:44:44.709 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.5499999999999998e-07 2024-09-16T09:44:44.709 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.709 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.709 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:44:44.709 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.443069+0000", 2024-09-16T09:44:44.709 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.7869999999999996e-06 2024-09-16T09:44:44.709 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.709 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.709 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:44:44.709 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.443079+0000", 2024-09-16T09:44:44.709 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.024e-05 2024-09-16T09:44:44.709 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.710 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.710 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:44:44.710 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.443085+0000", 2024-09-16T09:44:44.710 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.4239999999999998e-06 2024-09-16T09:44:44.710 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.710 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.710 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:44:44.710 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.456913+0000", 2024-09-16T09:44:44.710 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.013827136 2024-09-16T09:44:44.710 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.710 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.710 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:44:44.710 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.456928+0000", 2024-09-16T09:44:44.711 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5631999999999999e-05 2024-09-16T09:44:44.711 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.711 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.711 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:44:44.711 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.456975+0000", 2024-09-16T09:44:44.711 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.6310999999999998e-05 2024-09-16T09:44:44.711 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:44:44.711 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:44:44.711 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:44:44.711 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.711 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.711 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "backfill_remove(11.d e519 11:b800b8e2:::benchmark_data_smithi070_191495_object36927:head,464'2312,11:b802192a:::benchmark_data_smithi070_191495_object124279:head,489'7892,11:b8024b08:::benchmark_data_smithi070_191495_object119367:head,488'7575,11:b8025553:::benchmark_data_smithi070_191495_object120748:head,488'7653,11:b8031661:::benchmark_data_smithi070_191495_object55647:head,466'3501,11:b803bdd4:::benchmark_data_smithi070_191495_object32135:head,463'1995,11:b8046407:::benchmark_data_smithi070_191495_object53786:head,466'3370,11:b804ac59:::benchmark_data_smithi070_191495_object90559:head,476'5731,11:b80517c3:::benchmark_data_smithi070_191495_object47470:head,464'2974,11:b8056ea3:::benchmark_data_smithi070_191495_object67840:head,468'4242,11:b8058c28:::benchmark_data_smithi070_191495_object19212:head,460'1234,11:b8080458:::benchmark_data_smithi070_191495_object40960:head,464'2551,11:b8088e6a:::benchmark_data_smithi070_191495_object111425:head,484'7053,11:b8089ec7:::benchmark_data_smithi070_191495_object91753:head,476'5804,11:b80928ad:::benchmark_data_smithi070_191495_object25761:head,463'1612,11:b80a8bed:::benchmark_data_smithi070_191495_object58500:head,466'3679,11:b80b42f5:::benchmark_data_smithi070_191495_object78545:head,472'4963,11:b80b4cfa:::benchmark_data_smithi070_191495_object81894:head,474'5188,11:b80b9e42:::benchmark_data_smithi070_191495_object108493:head,483'6874,11:b80d69aa:::benchmark_data_smithi070_191495_object87679:head,474'5559,11:b80e244f:::benchmark_data_smithi070_191495_object126227:head,491'8005,11:b80ec662:::benchmark_data_smithi070_191495_object106376:head,481'6740,11:b80eece8:::benchmark_data_smithi070_191495_object32620:head,463'2026,11:b80f819b:::benchmark_data_smithi070_191495_object112212:head,485'7117,11:b8101aa8:::benchmark_data_smithi070_191495_object69741:head,468'4347,11:b8106f62:::benchmark_data_smithi070_191495_object123800:head,489'7853,11:b810ecef:::benchmark_data_smithi070_191495_object123017:head,488'7807,11:b8111ca3:::benchmark_data_smithi070_191495_object125838:head,490'7980,11:b81158a9:::benchmark_data_smithi070_191495_object33523:head,463'2091,11:b81167bb:::benchmark_data_smithi070_191495_object103074:head,480'6546,11:b81182ef:::benchmark_data_smithi070_191495_object102946:head,480'6533,11:b813a12f:::benchmark_data_smithi070_191495_object70534:head,468'4402,11:b8152418:::benchmark_data_smithi070_191495_object71479:head,468'4470,11:b815f133:::benchmark_data_smithi070_191495_object108705:head,483'6892,11:b81640ff:::benchmark_data_smithi070_191495_object31167:head,463'1938,11:b8168f79:::benchmark_data_smithi070_191495_object60522:head,467'3787,11:b816d29d:::benchmark_data_smithi070_191495_object102786:head,480'6520,11:b816f873:::benchmark_data_smithi070_191495_object21679:head,461'1373,11:b8177816:::benchmark_data_smithi070_191495_object98541:head,479'6253,11:b8177d53:::benchmark_data_smithi070_191495_object60986:head,467'3819,11:b817d36e:::benchmark_data_smithi070_191495_object103147:head,480'6550,11:b81893c3:::benchmark_data_smithi070_191495_object45135:head,464'2815,11:b818bfa3:::benchmark_data_smithi070_191495_object23683:head,462'1493,11:b8190fe4:::benchmark_data_smithi070_191495_object70839:head,468'4423,11:b81a4acf:::benchmark_data_smithi070_191495_object16736:head,460'1062,11:b81b6d1c:::benchmark_data_smithi070_191495_object17676:head,460'1132,11:b81b8e7b:::benchmark_data_smithi070_191495_object36324:head,464'2262,11:b81c2487:::benchmark_data_smithi070_191495_object105924:head,481'6712,11:b81c9231:::benchmark_data_smithi070_191495_object113575:head,486'7207,11:b81cb4e6:::benchmark_data_smithi070_191495_object124300:head,489'7896,11:b81d1a27:::benchmark_data_smithi070_191495_object31689:head,463'1961,11:b81d831f:::benchmark_data_smithi070_191495_object71349:head,468'4457,11:b81de996:::benchmark_data_smithi070_191495_object51089:head,465'3208,11:b81f39d9:::benchmark_data_smithi070_191495_object66660:head,468'4165,11:b81f4972:::benchmark_data_smithi070_191495_object104867:head,480'6662,11:b8202290:::benchmark_data_smithi070_191495_object102948:head,480'6535,11:b8202e3c:::benchmark_data_smithi070_191495_object94571:head,478'6000,11:b82069f0:::benchmark_data_smithi070_191495_object89147:head,474'5646,11:b8214139:::benchmark_data_smithi070_191495_object28618:head,463'1782,11:b82144cb:::benchmark_data_smithi070_191495_object39001:head,464'2436,11:b8215d29:::benchmark_data_smithi070_191495_object28286:head,463'1764,11:b82272a2:::benchmark_data_smithi070_191495_object64446:head,467'4034,11:b824ce03:::benchmark_data_smithi070_191495_object72626:head,468'4543,11:b8254a31:::benchmark_data_smithi070_191495_object66433:head,467'4152,11:b8256b18:::benchmark_data_smithi070_191495_object124005:head,489'7873,11:b8262d1a:::benchmark_data_smithi070_191495_object86097:head,474'5449,11:b826b46c:::benchmark_data_smithi070_191495_object90950:head,476'5752,11:b826de47:::benchmark_data_smithi070_191495_object101889:head,480'6468,11:b8272f9b:::benchmark_data_smithi070_191495_object45603:head,464'2846,11:b8274a3e:::benchmark_data_smithi070_191495_object99416:head,479'6308,11:b827d23e:::benchmark_data_smithi070_191495_object107816:head,482'6828,11:b8280973:::benchmark_data_smithi070_191495_object19083:head,460'1222,11:b828e72d:::benchmark_data_smithi070_191495_object100563:head,479'6390,11:b829c6bc:::benchmark_data_smithi070_191495_object117023:head,486'7414,11:b82a4ec3:::benchmark_data_smithi070_191495_object107338:head,481'6800,11:b82a770f:::benchmark_data_smithi070_191495_object33495:head,463'2087,11:b82b27d1:::benchmark_data_smithi070_191495_object61547:head,467'3862,11:b82b27da:::benchmark_data_smithi070_191495_object77957:head,471'4928,11:b82bb056:::benchmark_data_smithi070_191495_object48485:head,465'3038,11:b82c55a9:::benchmark_data_smithi070_191495_object101223:head,479'6426,11:b82d185a:::benchmark_data_smithi070_191495_object108213:head,482'6855,11:b82d322c:::benchmark_data_smithi070_191495_object22380:head,461'1414,11:b82d58d2:::benchmark_data_smithi070_191495_object100843:head,479'6404,11:b82deebf:::benchmark_data_smithi070_191495_object56211:head,466'3538,11:b82e0d7c:::benchmark_data_smithi070_191495_object112072:head,485'7110,11:b82e73d1:::benchmark_data_smithi070_191495_object52005:head,465'3265,11:b82f6226:::benchmark_data_smithi070_191495_object36082:head,464'2243,11:b82fa7cd:::benchmark_data_smithi070_191495_object120747:head,488'7652,11:b831245b:::benchmark_data_smithi070_191495_object43119:head,464'2695,11:b831d125:::benchmark_data_smithi070_191495_object96474:head,479'6120,11:b831d1ae:::benchmark_data_smithi070_191495_object67651:head,468'4230,11:b83256c8:::benchmark_data_smithi070_191495_object54008:head,466'3389,11:b8328649:::benchmark_data_smithi070_191495_object89512:head,475'5667,11:b832c5a9:::benchmark_data_smithi070_191495_object31533:head,463'1955,11:b832da75:::benchmark_data_smithi070_191495_object57336:head,466'3610,11:b833415a:::benchmark_data_smithi070_191495_object107855:head,482'6831,11:b833b4e5:::benchmark_data_smithi070_191495_object57270:head,466'3607,11:b834a7f2:::benchmark_data_smithi070_191495_object129448:head,491'8210,11:b835b9a6:::benchmark_data_smithi070_191495_object105665:head,481'6700,11:b836742f:::benchmark_data_smithi070_191495_object51614:head,465'3239,11:b8373828:::benchmark_data_smithi070_191495_object88512:head,474'5608,11:b8376679:::benchmark_data_smithi070_191495_object24520:head,462'1548,11:b837e984:::benchmark_data_smithi070_191495_object87281:head,474'5529,11:b8392109:::benchmark_data_smithi070_191495_object81781:head,473'5179,11:b83996fe:::benchmark_data_smithi070_191495_object88732:head,474'5620,11:b839eea8:::benchmark_data_smithi070_191495_object47834:head,464'2994,11:b83a54d2:::benchmark_data_smithi070_191495_object111432:head,484'7055,11:b83a8e70:::benchmark_data_smithi070_191495_object107380:head,481'6804,11:b83b68f7:::benchmark_data_smithi070_191495_object55026:head,466'3464,11:b83c183d:::benchmark_data_smithi070_191495_object38900:head,464'2427,11:b83d4693:::benchmark_data_smithi070_191495_object79514:head,472'5030,11:b83e1ab0:::benchmark_data_smithi070_191495_object30969:head,463'1921,11:b83e1f2e:::benchmark_data_smithi070_191495_object82415:head,474'5216,11:b83f1a29:::benchmark_data_smithi070_191495_object26265:head,463'1642,11:b83f37d7:::benchmark_data_smithi070_191495_object79184:head,472'5002,11:b83f7ed9:::benchmark_data_smithi070_191495_object99673:head,479'6328,11:b8409912:::benchmark_data_smithi070_191495_object98024:head,479'6223,11:b840a9cc:::benchmark_data_smithi070_191495_object127968:head,491'8110,11:b840c536:::benchmark_data_smithi070_191495_object80630:head,473'5100,11:b8411c24:::benchmark_data_smithi070_191495_object95803:head,478'6075,11:b8418494:::benchmark_data_smithi070_191495_object96315:head,478'6103,11:b8425c12:::benchmark_data_smithi070_191495_object16666:head,460'1055,11:b842b130:::benchmark_data_smithi070_191495_object103085:head,480'6548,11:b842b80a:::benchmark_data_smithi070_191495_object57885:head,466'3645,11:b842df4a:::benchmark_data_smithi070_191495_object16344:head,460'1038,11:b843ee33:::benchmark_data_smithi070_191495_object41647:head,464'2591,11:b844251a:::benchmark_data_smithi070_191495_object124207:head,489'7886,11:b844ee8a:::benchmark_data_smithi070_191495_object43557:head,464'2721,11:b8459bb4:::benchmark_data_smithi070_191495_object41487:head,464'2581,11:b845ba1d:::benchmark_data_smithi070_191495_object38430:head,464'2399,11:b845ec3b:::benchmark_data_smithi070_191495_object127520:head,491'8088,11:b846bac9:::benchmark_data_smithi070_191495_object117744:head,486'7459,11:b846dfd6:::benchmark_data_smithi070_191495_object101606:head,479'6451,11:b8474504:::benchmark_data_smithi070_191495_object106643:head,481'6756,11:b8476ccb:::benchmark_data_smithi070_191495_object74203:head,470'4644,11:b8477f76:::benchmark_data_smithi070_191495_object84541:head,474'5337,11:b847c14e:::benchmark_data_smithi070_191495_object72695:head,468'4548,11:b8486b8c:::benchmark_data_smithi070_191495_object75072:head,470'4705,11:b84886eb:::benchmark_data_smithi070_191495_object44366:head,464'2774,11:b848aa73:::benchmark_data_smithi070_191495_object117139:head,486'7421,11:b8494406:::benchmark_data_smithi070_191495_object126175:head,491'8003,11:b849dffa:::benchmark_data_smithi070_191495_object49761:head,465'3125,11:b84a265c:::benchmark_data_smithi070_191495_object77188:head,471'4876,11:b84ad420:::benchmark_data_smithi070_191495_object55529:head,466'3494,11:b84c1060:::benchmark_data_smithi070_191495_object100649:head,479'6392,11:b84c1075:::benchmark_data_smithi070_191495_object109366:head,483'6932,11:b84c4b44:::benchmark_data_smithi070_191495_object48658:head,465'3053,11:b84c6f07:::benchmark_data_smithi070_191495_object31104:head,463'1931,11:b84e0e27:::benchmark_data_smithi070_191495_object18885:head,460'1212,11:b84e339a:::benchmark_data_smithi070_191495_object26429:head,463'1654,11:b84f24ef:::benchmark_data_smithi070_191495_object88236:head,474'5594,11:b84f5fd1:::benchmark_data_smithi070_191495_object76071:head,471'4786,11:b8507817:::benchmark_data_smithi070_191495_object100450:head,479'6380,11:b851e555:::benchmark_data_smithi070_191495_object84108:head,474'5311,11:b852667e:::benchmark_data_smithi070_191495_object47334:head,464'2960,11:b8531ebe:::benchmark_data_smithi070_191495_object126549:head,491'8025,11:b853423b:::benchmark_data_smithi070_191495_object86510:head,474'5476,11:b8538157:::benchmark_data_smithi070_191495_object76288:head,471'4803,11:b854484b:::benchmark_data_smithi070_191495_object120494:head,488'7635,11:b854639a:::benchmark_data_smithi070_191495_object116154:head,486'7360,11:b854ccb7:::benchmark_data_smithi070_191495_object59442:head,466'3725,11:b855c106:::benchmark_data_smithi070_191495_object62906:head,467'3939,11:b8563356:::benchmark_data_smithi070_191495_object23808:head,462'1500,11:b856f557:::benchmark_data_smithi070_191495_object108559:head,483'6880,11:b8571b4e:::benchmark_data_smithi070_191495_object18771:head,460'1205,11:b8574477:::benchmark_data_smithi070_191495_object88957:head,474'5635,11:b857edec:::benchmark_data_smithi070_191495_object37284:head,464'2334,11:b85806a4:::benchmark_data_smithi070_191495_object48144:head,465'3016,11:b858180b:::benchmark_data_smithi070_191495_object21454:head,461'1356,11:b858db5c:::benchmark_data_smithi070_191495_object68969:head,468'4301,11:b8597cf0:::benchmark_data_smithi070_191495_object87257:head,474'5527,11:b8598fc6:::benchmark_data_smithi070_191495_object39793:head,464'2484,11:b859bc18:::benchmark_data_smithi070_191495_object101819:head,480'6462,11:b859d9b0:::benchmark_data_smithi070_191495_object32990:head,463'2057,11:b85badc6:::benchmark_data_smithi070_191495_object66384:head,467'4149,11:b85d6f08:::benchmark_data_smithi070_191495_object97317:head,479'6182,11:b85e062f:::benchmark_data_smithi070_191495_object100002:head,479'6352,11:b85eed86:::benchmark_data_smithi070_191495_object71958:head,468'4493,11:b8606dd8:::benchmark_data_smithi070_191495_object40159:head,464'2502,11:b860805d:::benchmark_data_smithi070_191495_object63603:head,467'3981,11:b860d385:::benchmark_data_smithi070_191495_object130260:head,492'8257,11:b86103aa:::benchmark_data_smithi070_191495_object27966:head,463'1750,11:b8615de2:::benchmark_data_smithi070_191495_object54784:head,466'3448,11:b8615e71:::benchmark_last_metadata:head,492'8265,11:b861a317:::benchmark_data_smithi070_191495_object55164:head,466'3470,11:b861c136:::benchmark_data_smithi070_191495_object32868:head,463'2050,11:b861ea44:::benchmark_data_smithi070_191495_object46809:head,464'2917,11:b863c449:::benchmark_data_smithi070_191495_object99818:head,479'6338,11:b864e146:::benchmark_data_smithi070_191495_object48635:head,465'3051,11:b8651c54:::benchmark_data_smithi070_191495_object32685:head,463'2032,11:b865c0eb:::benchmark_data_smithi070_191495_object103915:head,480'6603,11:b8661b3c:::benchmark_data_smithi070_191495_object48612:head,465'3049,11:b86640ef:::benchmark_data_smithi070_191495_object51750:head,465'3251,11:b8685d3e:::benchmark_data_smithi070_191495_object68455:head,468'4268,11:b8687cd8:::benchmark_data_smithi070_191495_object26796:head,463'1675,11:b868c7b6:::benchmark_data_smithi070_191495_object88952:head,474'5634,11:b868f69e:::benchmark_data_smithi070_191495_object123052:head,488'7808,11:b86925b4:::benchmark_data_smithi070_191495_object127307:head,491'8072,11:b86a9ace:::benchmark_data_smithi070_191495_object44260:head,464'2763,11:b86ae56c:::benchmark_data_smithi070_191495_object44332:head,464'2770,11:b86b4f0c:::benchmark_data_smithi070_191495_object42584:head,464'2664,11:b86b59ec:::benchmark_data_smithi070_191495_object52433:head,466'3287,11:b86b88d1:::benchmark_data_smithi070_191495_object17339:head,460'1111,11:b86bf443:::benchmark_data_smithi070_191495_object83675:head,474'5285,11:b86c88d9:::benchmark_data_smithi070_191495_object72080:head,468'4500,11:b86d589b:::benchmark_data_smithi070_191495_object51738:head,465'3249,11:b86d912d:::benchmark_data_smithi070_191495_object102344:head,480'6489,11:b86edf71:::benchmark_data_smithi070_191495_object72283:head,468'4519,11:b86ff5cf:::benchmark_data_smithi070_191495_object101074:head,479'6418,11:b8700937:::benchmark_data_smithi070_191495_object99121:head,479'6294,11:b8702889:::benchmark_data_smithi070_191495_object26205:head,463'1638,11:b870910c:::benchmark_data_smithi070_191495_object83765:head,474'5292,11:b871414b:::benchmark_data_smithi070_191495_object16734:head,460'1061,11:b8727a85:::benchmark_data_smithi070_191495_object100835:head,479'6403,11:b8727cf5:::benchmark_data_smithi070_191495_object108725:head,483'6895,11:b872c124:::benchmark_data_smithi070_191495_object20577:head,460'1308,11:b8735a91:::benchmark_data_smithi070_191495_object128130:head,491'8121,11:b873e449:::benchmark_data_smithi070_191495_object101156:head,479'6422,11:b875ba2e:::benchmark_data_smithi070_191495_object52634:head,466'3307,11:b8766f14:::benchmark_data_smithi070_191495_object57691:head,466'3634,11:b876d582:::benchmark_data_smithi070_191495_object72247:head,468'4516,11:b876ddd4:::benchmark_data_smithi070_191495_object40509:head,464'2528,11:b877d6c6:::benchmark_data_smithi070_191495_object106228:head,481'6735,11:b877f92d:::benchmark_data_smithi070_191495_object37460:head,464'2340,11:b8781297:::benchmark_data_smithi070_191495_object68678:head,468'4279,11:b8781a6a:::benchmark_data_smithi070_191495_object43398:head,464'2714,11:b8792385:::benchmark_data_smithi070_191495_object27743:head,463'1736,11:b87979e4:::benchmark_data_smithi070_191495_object111793:head,484'7091,11:b8798e84:::benchmark_data_smithi070_191495_object18641:head,460'1194,11:b879bc29:::benchmark_data_smithi070_191495_object107350:head,481'6801,11:b879f528:::benchmark_data_smithi070_191495_object38663:head,464'2410,11:b87a6ac0:::benchmark_data_smithi070_191495_object128937:head,491'8179,11:b87b42ca:::benchmark_data_smithi070_191495_object76802:head,471'4839,11:b87bcd7e:::benchmark_data_smithi070_191495_object79296:head,472'5012,11:b87c3659:::benchmark_data_smithi070_191495_object50235:head,465'3153,11:b87d09e2:::benchmark_data_smithi070_191495_object76920:head,471'4848,11:b87d2198:::benchmark_data_smithi070_191495_object51387:head,465'3228,11:b87d52bf:::benchmark_data_smithi070_191495_object109189:head,483'6920,11:b87e38f1:::benchmark_data_smithi070_191495_object73440:head,469'4599,11:b87ebc87:::benchmark_data_smithi070_191495_object52661:head,466'3309,11:b87f5f40:::benchmark_data_smithi070_191495_object101523:head,479'6448,11:b87fa6fb:::benchmark_data_smithi070_191495_object22363:head,461'1413,11:b88000b9:::benchmark_data_smithi070_191495_object16751:head,460'1063,11:b8803d53:::benchmark_data_smithi070_191495_object69043:head,468'4309,11:b8807227:::benchmark_data_smithi070_191495_object29466:head,463'1834,11:b880ed68:::benchmark_data_smithi070_191495_object109459:head,483'6938,11:b880ef7c:::benchmark_data_smithi070_191495_object62794:head,467'3935,11:b8811c22:::benchmark_data_smithi070_191495_object90639:head,476'5738,11:b882e8bd:::benchmark_data_smithi070_191495_object35670:head,464'2219,11:b8840f61:::benchmark_data_smithi070_191495_object92392:head,476'5858,11:b886dde6:::benchmark_data_smithi070_191495_object92251:head,476'5848,11:b88751c9:::benchmark_data_smithi070_191495_object53125:head,466'3336,11:b8880d00:::benchmark_data_smithi070_191495_object42822:head,464'2679,11:b88916c7:::benchmark_data_smithi070_191495_object29525:head,463'1837,11:b889dcba:::benchmark_data_smithi070_191495_object52774:head,466'3317,11:b889f0cb:::benchmark_data_smithi070_191495_object39574:head,464'2465,11:b88a99c3:::benchmark_data_smithi070_191495_object102191:head,480'6481,11:b88aed20:::benchmark_data_smithi070_191495_object62760:head,467'3933,11:b88b2431:::benchmark_data_smithi070_191495_object121727:head,488'7716,11:b88bbd80:::benchmark_data_smithi070_191495_object16900:head,460'1072,11:b88bc433:::benchmark_data_smithi070_191495_object123266:head,488'7818,11:b88cbb62:::benchmark_data_smithi070_191495_object24012:head,462'1509,11:b88dfad3:::benchmark_data_smithi070_191495_object70720:head,468'4416,11:b88eaa90:::benchmark_data_smithi070_191495_object61420:head,467'3857,11:b88f1008:::benchmark_data_smithi070_191495_object18865:head,460'1211,11:b8903677:::benchmark_data_smithi070_191495_object28632:head,463'1783,11:b8909095:::benchmark_data_smithi070_191495_object126333:head,491'8011,11:b890db14:::benchmark_data_smithi070_191495_object29104:head,463'1811,11:b89129af:::benchmark_data_smithi070_191495_object122694:head,488'7786,11:b891b68f:::benchmark_data_smithi070_191495_object31845:head,463'1974,11:b89329e6:::benchmark_data_smithi070_191495_object95914:head,478'6078,11:b8939077:::benchmark_data_smithi070_191495_object48269:head,465'3025,11:b89450d0:::benchmark_data_smithi070_191495_object50265:head,465'3156,11:b894cfca:::benchmark_data_smithi070_191495_object98767:head,479'6269,11:b894f08f:::benchmark_data_smithi070_191495_object69255:head,468'4325,11:b8950898:::benchmark_data_smithi070_191495_object127982:head,491'8111,11:b895940e:::benchmark_data_smithi070_191495_object26905:head,463'1684,11:b895d6fd:::benchmark_data_smithi070_191495_object16148:head,460'1023,11:b897b4f1:::benchmark_data_smithi070_191495_object87227:head,474'5524,11:b897dcc4:::benchmark_data_smithi070_191495_object84030:head,474'5306,11:b8985e32:::benchmark_data_smithi070_191495_object50087:head,465'3143,11:b899afcf:::benchmark_data_smithi070_191495_object19254:head,460'1237,11:b899f9f0:::benchmark_data_smithi070_191495_object28880:head,463'1800,11:b89a2880:::benchmark_data_smithi070_191495_object90633:head,476'5736,11:b89be364:::benchmark_data_smithi070_191495_object85369:head,474'5388,11:b89be3f7:::benchmark_data_smithi070_191495_object76221:head,471'4797,11:b89c3fff:::benchmark_data_smithi070_191495_object31182:head,463'1940,11:b89c7613:::benchmark_data_smithi070_191495_object67167:head,468'4196,11:b89ce981:::benchmark_data_smithi070_191495_object41959:head,464'2616,11:b89d12ca:::benchmark_data_smithi070_191495_object52851:head,466'3323,11:b89d7e1d:::benchmark_data_smithi070_191495_object37482:head,464'2341,11:b89d95de:::benchmark_data_smithi070_191495_object104177:head,480'6614,11:b89dac71:::benchmark_data_smithi070_191495_object21450:head,461'1355,11:b8a03e7f:::benchmark_data_smithi070_191495_object38143:head,464'2384,11:b8a049d5:::benchmark_data_smithi070_191495_object130198:head,492'8253,11:b8a04e86:::benchmark_data_smithi070_191495_object16805:head,460'1067,11:b8a100fd:::benchmark_data_smithi070_191495_object42775:head,464'2677,11:b8a1aa48:::benchmark_data_smithi070_191495_object19011:head,460'1217,11:b8a1f7d7:::benchmark_data_smithi070_191495_object56321:head,466'3546,11:b8a255c2:::benchmark_data_smithi070_191495_object101304:head,479'6432,11:b8a25a8f:::benchmark_data_smithi070_191495_object85772:head,474'5426,11:b8a27bfe:::benchmark_data_smithi070_191495_object65666:head,467'4114,11:b8a31e91:::benchmark_data_smithi070_191495_object105059:head,481'6670,11:b8a45e53:::benchmark_data_smithi070_191495_object58012:head,466'3649,11:b8a4c945:::benchmark_data_smithi070_191495_object77041:head,471'4861,11:b8a4fb93:::benchmark_data_smithi070_191495_object18080:head,460'1156,11:b8a57da3:::benchmark_data_smithi070_191495_object30358:head,463'1892,11:b8a617a0:::benchmark_data_smithi070_191495_object21190:head,461'1342,11:b8a67cf6:::benchmark_data_smithi070_191495_object60361:head,467'3778,11:b8a736c8:::benchmark_data_smithi070_191495_object44124:head,464'2749,11:b8a749de:::benchmark_data_smithi070_191495_object129747:head,491'8233,11:b8a75aae:::benchmark_data_smithi070_191495_object89834:head,475'5686,11:b8a7e606:::benchmark_data_smithi070_191495_object76207:head,471'4795,11:b8a9e6a5:::benchmark_data_smithi070_191495_object33855:head,464'2113,11:b8ac4d25:::benchmark_data_smithi070_191495_object130079:head,491'8247,11:b8aca965:::benchmark_data_smithi070_191495_object22029:head,461'1395,11:b8ae16b0:::benchmark_data_smithi070_191495_object101737:head,480'6458,11:b8ae4fb7:::benchmark_data_smithi070_191495_object130162:head,492'8252,11:b8aec1d0:::benchmark_data_smithi070_191495_object128667:head,491'8156,11:b8af7ad7:::benchmark_data_smithi070_191495_object45586:head,464'2845,11:b8b077f5:::benchmark_data_smithi070_191495_object64763:head,467'4065,11:b8b11aae:::benchmark_data_smithi070_191495_object61343:head,467'3850,11:b8b14e77:::benchmark_data_smithi070_191495_object36857:head,464'2306,11:b8b25416:::benchmark_data_smithi070_191495_object45926:head,464'2869,11:b8b2a746:::benchmark_data_smithi070_191495_object87764:head,474'5566,11:b8b3520b:::benchmark_data_smithi070_191495_object37137:head,464'2326,11:b8b3aa5c:::benchmark_data_smithi070_191495_object116616:head,486'7389,11:b8b3b7e6:::benchmark_data_smithi070_191495_object41396:head,464'2576,11:b8b4189e:::benchmark_data_smithi070_191495_object70154:head,468'4377,11:b8b43b01:::benchmark_data_smithi070_191495_object32598:head,463'2024,11:b8b493cc:::benchmark_data_smithi070_191495_object22303:head,461'1406,11:b8b6d13c:::benchmark_data_smithi070_191495_object79404:head,472'5023,11:b8b6f0f8:::benchmark_data_smithi070_191495_object54787:head,466'3449,11:b8b6fced:::benchmark_data_smithi070_191495_object24096:head,462'1519,11:b8b734f5:::benchmark_data_smithi070_191495_object50464:head,465'3167,11:b8b7c0a7:::benchmark_data_smithi070_191495_object59970:head,467'3754,11:b8b85cde:::benchmark_data_smithi070_191495_object81199:head,473'5140,11:b8b909f8:::benchmark_data_smithi070_191495_object54379:head,466'3415,11:b8b95ec4:::benchmark_data_smithi070_191495_object107773:head,482'6824,11:b8b9ffb3:::benchmark_data_smithi070_191495_object76057:head,471'4783,11:b8ba12ee:::benchmark_data_smithi070_191495_object119904:head,488'7606,11:b8bab888:::benchmark_data_smithi070_191495_object104783:head,480'6653,11:b8bb4721:::benchmark_data_smithi070_191495_object65346:head,467'4101,11:b8bb67e0:::benchmark_data_smithi070_191495_object128389:head,491'8136,11:b8bb7ce3:::benchmark_data_smithi070_191495_object42647:head,464'2671,11:b8bbef70:::benchmark_data_smithi070_191495_object43008:head,464'2685,11:b8bbf54a:::benchmark_data_smithi070_191495_object83520:head,474'5276,11:b8bc24d9:::benchmark_data_smithi070_191495_object32779:head,463'2039,11:b8bc60e8:::benchmark_data_smithi070_191495_object40891:head,464'2548,11:b8bc7901:::benchmark_data_smithi070_191495_object64102:head,467'4012,11:b8be3b32:::benchmark_data_smithi070_191495_object104488:head,480'6630,11:b8be4edf:::benchmark_data_smithi070_191495_object39998:head,464'2493,11:b8bf2c97:::benchmark_data_smithi070_191495_object31347:head,463'1950,11:b8c07094:::benchmark_data_smithi070_191495_object95413:head,478'6048,11:b8c21875:::benchmark_data_smithi070_191495_object116230:head,486'7365,11:b8c24711:::benchmark_data_smithi070_191495_object127310:head,491'8073,11:b8c40a3c:::benchmark_data_smithi070_191495_object83856:head,474'5298,11:b8c50611:::benchmark_data_smithi070_191495_object103427:head,480'6564,11:b8c63198:::benchmark_data_smithi070_191495_object106453:head,481'6745,11:b8c690ef:::benchmark_data_smithi070_191495_object115660:head,486'7328,11:b8c7c46c:::benchmark_data_smithi070_191495_object112724:head,485'7154,11:b8c7cf15:::benchmark_data_smithi070_191495_object57744:head,466'3639,11:b8c7eb4f:::benchmark_data_smithi070_191495_object55907:head,466'3521,11:b8c7ed5d:::benchmark_data_smithi070_191495_object108537:head,483'6879,11:b8c81d0b:::benchmark_data_smithi070_191495_object51255:head,465'3217,11:b8c833d2:::benchmark_data_smithi070_191495_object78329:head,472'4951,11:b8c87d4b:::benchmark_data_smithi070_191495_object31716:head,463'1964,11:b8c8d5cc:::benchmark_data_smithi070_191495_object76704:head,471'4832,11:b8c8efb3:::benchmark_data_smithi070_191495_object84416:head,474'5330,11:b8c915bb:::benchmark_data_smithi070_191495_object100338:head,479'6373,11:b8c967a2:::benchmark_data_smithi070_191495_object36185:head,464'2250,11:b8c979ca:::benchmark_data_smithi070_191495_object35234:head,464'2195,11:b8c98c07:::benchmark_data_smithi070_191495_object120443:head,488'7631,11:b8c9d01c:::benchmark_data_smithi070_191495_object46878:head,464'2921,11:b8ca81af:::benchmark_data_smithi070_191495_object89859:head,475'5690,11:b8caf694:::benchmark_data_smithi070_191495_object83742:head,474'5289,11:b8cb2b9a:::benchmark_data_smithi070_191495_object79670:head,472'5039,11:b8cc19b0:::benchmark_data_smithi070_191495_object93999:head,477'5961,11:b8cd15aa:::benchmark_data_smithi070_191495_object42610:head,464'2668,11:b8cd538a:::benchmark_data_smithi070_191495_object87579:head,474'5552,11:b8cd86ed:::benchmark_data_smithi070_191495_object70211:head,468'4384,11:b8cd8e50:::benchmark_data_smithi070_191495_object118569:head,487'7520,11:b8cddb51:::benchmark_data_smithi070_191495_object43726:head,464'2734,11:b8ce0b4d:::benchmark_data_smithi070_191495_object63426:head,467'3973,11:b8ce171a:::benchmark_data_smithi070_191495_object42704:head,464'2673,11:b8ce8716:::benchmark_data_smithi070_191495_object75283:head,470'4724,11:b8cf122e:::benchmark_data_smithi070_191495_object97024:head,479'6165,11:b8cfc4c4:::benchmark_data_smithi070_191495_object127381:head,491'8079,11:b8d083da:::benchmark_data_smithi070_191495_object122170:head,488'7749,11:b8d2d56d:::benchmark_data_smithi070_191495_object80616:head,473'5099,11:b8d35d79:::benchmark_data_smithi070_191495_object47043:head,464'2938,11:b8d4678c:::benchmark_data_smithi070_191495_object34309:head,464'2140,11:b8d598be:::benchmark_data_smithi070_191495_object48281:head,465'3026,11:b8d5bc36:::benchmark_data_smithi070_191495_object101582:head,479'6449,11:b8d7288a:::benchmark_data_smithi070_191495_object107166:head,481'6787,11:b8d8133a:::benchmark_data_smithi070_191495_object52850:head,466'3322,11:b8d957b1:::benchmark_data_smithi070_191495_object117194:head,486'7424,11:b8d9989c:::benchmark_data_smithi070_191495_object128595:head,491'8152,11:b8d9ffdb:::benchmark_data_smithi070_191495_object47997:head,465'3002,11:b8dad53b:::benchmark_data_smithi070_191495_object89674:head,475'5678,11:b8dbc1f3:::benchmark_data_smithi070_191495_object86693:head,474'5491,11:b8dbca59:::benchmark_data_smithi070_191495_object83779:head,474'5294,11:b8dc58d4:::benchmark_data_smithi070_191495_object114111:head,486'7237,11:b8dc5f49:::benchmark_data_smithi070_191495_object67632:head,468'4228,11:b8dc69bb:::benchmark_data_smithi070_191495_object51759:head,465'3253,11:b8dd5912:::benchmark_data_smithi070_191495_object85764:head,474'5424,11:b8ddc993:::benchmark_data_smithi070_191495_object21252:head,461'1347,11:b8de0f9c:::benchmark_data_smithi070_191495_object24457:head,462'1545,11:b8de6cf5:::benchmark_data_smithi070_191495_object128410:head,491'8138,11:b8df1470:::benchmark_data_smithi070_191495_object59124:head,466'3706,11:b8df8e92:::benchmark_data_smithi070_191495_object22346:head,461'1412,11:b8e016d3:::benchmark_data_smithi070_191495_object80440:head,472'5089,11:b8e28099:::benchmark_data_smithi070_191495_object101892:head,480'6469,11:b8e2c7a5:::benchmark_data_smithi070_191495_object57182:head,466'3602,11:b8e2fcf7:::benchmark_data_smithi070_191495_object52646:head,466'3308,11:b8e35438:::benchmark_data_smithi070_191495_object59947:head,467'3751,11:b8e3e3d2:::benchmark_data_smithi070_191495_object24767:head,462'1566,11:b8e42cba:::benchmark_data_smithi070_191495_object80965:head,473'5120,11:b8e4896c:::benchmark_data_smithi070_191495_object74895:head,470'4698,11:b8e4c650:::benchmark_data_smithi070_191495_object87633:head,474'5557,11:b8e5c84f:::benchmark_data_smithi070_191495_object74888:head,470'4697,11:b8e634db:::benchmark_data_smithi070_191495_object112986:head,485'7167,11:b8e63b75:::benchmark_data_smithi070_191495_object80154:head,472'5074,11:b8e71a45:::benchmark_data_smithi070_191495_object107234:head,481'6792,11:b8e74e51:::benchmark_data_smithi070_191495_object87905:head,474'5574,11:b8e830bc:::benchmark_data_smithi070_191495_object57586:head,466'3626,11:b8e87b68:::benchmark_data_smithi070_191495_object16943:head,460'1079,11:b8e8cdc0:::benchmark_data_smithi070_191495_object80151:head,472'5073,11:b8e94f33:::benchmark_data_smithi070_191495_object112390:head,485'7133,11:b8e98ed6:::benchmark_data_smithi070_191495_object111175:head,484'7031,11:b8ea5444:::benchmark_data_smithi070_191495_object126328:head,491'8010,11:b8eafd86:::benchmark_data_smithi070_191495_object88476:head,474'5605,11:b8eb61fa:::benchmark_data_smithi070_191495_object128793:head,491'8171,11:b8ebf0ce:::benchmark_data_smithi070_191495_object42071:head,464'2623,11:b8ec1a51:::benchmark_data_smithi070_191495_object63332:head,467'3965,11:b8edada7:::benchmark_data_smithi070_191495_object34391:head,464'2148,11:b8efe008:::benchmark_data_smithi070_191495_object109203:head,483'6921,11:b8f1c0c5:::benchmark_data_smithi070_191495_object122521:head,488'7779,11:b8f2002a:::benchmark_data_smithi070_191495_object108266:head,482'6858,11:b8f23367:::benchmark_data_smithi070_191495_object104497:head,480'6631,11:b8f30045:::benchmark_data_smithi070_191495_object66316:head,467'4147,11:b8f47a99:::benchmark_data_smithi070_191495_object52820:head,466'3320,11:b8f4dc6a:::benchmark_data_smithi070_191495_object121776:head,488'7722,11:b8f54129:::benchmark_data_smithi070_191495_object80103:head,472'5070,11:b8f5746f:::benchmark_data_smithi070_191495_object128082:head,491'8119,11:b8f5806f:::benchmark_data_smithi070_191495_object42367:head,464'2649,11:b8f5b3e1:::benchmark_data_smithi070_191495_object121903:head,488'7730,11:b8f5eb7e:::benchmark_data_smithi070_191495_object129750:head,491'8234,11:b8f60c0f:::benchmark_data_smithi070_191495_object128416:head,491'8139,11:b8f62873:::benchmark_data_smithi070_191495_object86439:head,474'5470,11:b8f6a103:::benchmark_data_smithi070_191495_object26889:head,463'1683,11:b8f6a1bf:::benchmark_data_smithi070_191495_object115350:head,486'7307,11:b8f70521:::benchmark_data_smithi070_191495_object108895:head,483'6905,11:b8f7e5f5:::benchmark_data_smithi070_191495_object48827:head,465'3061,11:b8f82978:::benchmark_data_smithi070_191495_object41664:head,464'2594,11:b8f8ec0a:::benchmark_data_smithi070_191495_object33100:head,463'2064,11:b8f90ac4:::benchmark_data_smithi070_191495_object105923:head,481'6711,11:b8f99d96:::benchmark_data_smithi070_191495_object90018:head,475'5705,11:b8fae6dc:::benchmark_data_smithi070_191495_object50714:head,465'3186,11:b8faf2c7:::benchmark_data_smithi070_191495_object46085:head,464'2879,11:b8fb2a18:::benchmark_data_smithi070_191495_object89529:head,475'5669,11:b8fb65bc:::benchmark_data_smithi070_191495_object77281:head,471'4885,11:b8fbbadb:::benchmark_data_smithi070_191495_object59231:head,466'3713,11:b8fc12f1:::benchmark_data_smithi070_191495_object109308:head,483'6927,11:b8fc375b:::benchmark_data_smithi070_191495_object101963:head,480'6476,11:b8fc77e8:::benchmark_data_smithi070_191495_object114947:head,486'7287,11:b8fdeb35:::benchmark_data_smithi070_191495_object83859:head,474'5300,11:b8fe1fe4:::benchmark_data_smithi070_191495_object25588:head,463'1604,11:b8ff5cd4:::benchmark_data_smithi070_191495_object77294:head,471'4887,11:b900a551:::benchmark_data_smithi070_191495_object95666:head,478'6066,11:b900d670:::benchmark_data_smithi070_191495_object53391:head,466'3350,11:b9011681:::benchmark_data_smithi070_191495_object101186:head,479'6423,11:b9017231:::benchmark_data_smithi070_191495_object64145:head,467'4017,11:b901aec6:::benchmark_data_smithi070_191495_object55055:head,466'3465,11:b9025a29:::benchmark_data_smithi070_191495_object18009:head,460'1154,11:b90316ff:::benchmark_data_smithi070_191495_object121351:head,488'7690,11:b9033085:::benchmark_data_smithi070_191495_object33008:head,463'2058,11:b9034f9e:::benchmark_data_smithi070_191495_object113682:head,486'7212,11:b903ccb8:::benchmark_data_smithi070_191495_object89527:head,475'5668,11:b903d840:::benchmark_data_smithi070_191495_object27667:head,463'1728,11:b9059aac:::benchmark_data_smithi070_191495_object55387:head,466'3479,11:b905e9dd:::benchmark_data_smithi070_191495_object100856:head,479'6405,11:b906193f:::benchmark_data_smithi070_191495_object65793:head,467'4123,11:b9062227:::benchmark_data_smithi070_191495_object52356:head,465'3284,11:b906e965:::benchmark_data_smithi070_191495_object121855:head,488'7727,11:b9083803:::benchmark_data_smithi070_191495_object71920:head,468'4490,11:b9088742:::benchmark_data_smithi070_191495_object17483:head,460'1119,11:b908ac6e:::benchmark_data_smithi070_191495_object122973:head,488'7804,11:b908b508:::benchmark_data_smithi070_191495_object33478:head,463'2086,11:b909961f:::benchmark_data_smithi070_191495_object82614:head,474'5233,11:b90a5cb1:::benchmark_data_smithi070_191495_object95394:head,478'6045,11:b90b7f22:::benchmark_data_smithi070_191495_object100759:head,479'6395,11:b90b8f2d:::benchmark_data_smithi070_191495_object70203:head,468'4382,11:b90d080d:::benchmark_data_smithi070_191495_object33696:head,464'2102,11:b90d3bb6:::benchmark_data_smithi070_191495_object82887:head,474'5249,11:b90d9b9c:::benchmark_data_smithi070_191495_object102993:head,480'6541,11:b90d9eb0:::benchmark_data_smithi070_191495_object35834:head,464'2229,11:b90ea8b2:::benchmark_data_smithi070_191495_object29473:head,463'1835,11:b90f5eb5:::benchmark_data_smithi070_191495_object54156:head,466'3401,11:b910a9a7:::benchmark_data_smithi070_191495_object127317:head,491'8074,11:b910b84c:::benchmark_data_smithi070_191495_object53559:head,466'3359,11:b9115693:::benchmark_data_smithi070_191495_object112805:head,485'7160,11:b912380a:::benchmark_data_smithi070_191495_object76452:head,471'4816,11:b913739d:::benchmark_data_smithi070_191495_object22939:head,462'1451,11:b9149d5d:::benchmark_data_smithi070_191495_object35364:head,464'2203,11:b915cbc9:::benchmark_data_smithi070_191495_object130091:head,492'8248,11:b916c059:::benchmark_data_smithi070_191495_object111889:head,484'7101,11:b917bcf0:::benchmark_data_smithi070_191495_object98802:head,479'6273,11:b9182ee7:::benchmark_data_smithi070_191495_object127139:head,491'8062,11:b918b16c:::benchmark_data_smithi070_191495_object43083:head,464'2693,11:b9191a23:::benchmark_data_smithi070_191495_object31141:head,463'1933,11:b9192569:::benchmark_data_smithi070_191495_object18402:head,460'1179)", 2024-09-16T09:44:44.712 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:44:26.495215+0000", 2024-09-16T09:44:44.712 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 18.136817999000002, 2024-09-16T09:44:44.712 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.011971562999999999, 2024-09-16T09:44:44.712 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:44:44.712 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:44:44.712 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T09:44:44.712 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:44:44.712 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.712 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:44:44.712 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.495215+0000", 2024-09-16T09:44:44.712 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:44:44.712 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.713 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.713 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:44:44.713 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.495215+0000", 2024-09-16T09:44:44.713 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:44:44.713 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.713 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.713 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:44:44.713 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.495215+0000", 2024-09-16T09:44:44.713 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.2500000000000001e-07 2024-09-16T09:44:44.713 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.713 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.713 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:44:44.714 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.495387+0000", 2024-09-16T09:44:44.714 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00017165 2024-09-16T09:44:44.714 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.714 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.714 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:44:44.714 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.495616+0000", 2024-09-16T09:44:44.714 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00022894300000000001 2024-09-16T09:44:44.714 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.714 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.714 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:44:44.714 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.496076+0000", 2024-09-16T09:44:44.714 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00046004299999999999 2024-09-16T09:44:44.714 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.714 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.715 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:44:44.715 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.504759+0000", 2024-09-16T09:44:44.715 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0086828280000000001 2024-09-16T09:44:44.715 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.715 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.715 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:44:44.715 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.505196+0000", 2024-09-16T09:44:44.715 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00043718899999999999 2024-09-16T09:44:44.715 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.716 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.716 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:44:44.716 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.507187+0000", 2024-09-16T09:44:44.716 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.001990585 2024-09-16T09:44:44.716 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:44:44.716 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:44:44.716 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:44:44.716 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.716 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.716 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "backfill_remove(11.4 e519 11:27fbc6a0:::benchmark_data_smithi070_191495_object98630:head,479'6198,11:27fca743:::benchmark_data_smithi070_191495_object60720:head,467'3803,11:27fd3a7d:::benchmark_data_smithi070_191495_object61340:head,467'3845,11:27fdb89b:::benchmark_data_smithi070_191495_object38491:head,464'2453,11:27fe2a30:::benchmark_data_smithi070_191495_object86611:head,474'5446,11:27feacb2:::benchmark_data_smithi070_191495_object89387:head,474'5612,11:27ff27db:::benchmark_data_smithi070_191495_object62216:head,467'3901,11:27ff4cd8:::benchmark_data_smithi070_191495_object93899:head,477'5917,11:27ffa3c9:::benchmark_data_smithi070_191495_object31229:head,463'2002,11:28002c11:::benchmark_data_smithi070_191495_object50186:head,465'3118,11:280061fc:::benchmark_data_smithi070_191495_object35283:head,464'2243,11:2801729d:::benchmark_data_smithi070_191495_object27511:head,463'1765,11:280199de:::benchmark_data_smithi070_191495_object34914:head,464'2214,11:2801fd65:::benchmark_data_smithi070_191495_object52533:head,466'3269,11:28024e0f:::benchmark_data_smithi070_191495_object77648:head,471'4869,11:2803002e:::benchmark_data_smithi070_191495_object124102:head,489'7753,11:280337b7:::benchmark_data_smithi070_191495_object71342:head,468'4501,11:28037ac9:::benchmark_data_smithi070_191495_object88942:head,474'5576,11:2803c3f8:::benchmark_data_smithi070_191495_object34495:head,464'2180,11:280407a3:::benchmark_data_smithi070_191495_object108891:head,483'6834,11:2804db26:::benchmark_data_smithi070_191495_object66160:head,467'4168,11:28050129:::benchmark_data_smithi070_191495_object40778:head,464'2599,11:280517c2:::benchmark_data_smithi070_191495_object46761:head,464'2942,11:2806b3b9:::benchmark_data_smithi070_191495_object30879:head,463'1965,11:28076a14:::benchmark_data_smithi070_191495_object38141:head,464'2425,11:28081210:::benchmark_data_smithi070_191495_object16560:head,460'1071,11:28093ce0:::benchmark_data_smithi070_191495_object44792:head,464'2819,11:28099e0c:::benchmark_data_smithi070_191495_object77214:head,471'4844,11:2809af56:::benchmark_data_smithi070_191495_object81048:head,473'5099,11:280b0d10:::benchmark_data_smithi070_191495_object17139:head,460'1106,11:280b0f0d:::benchmark_data_smithi070_191495_object122412:head,488'7657,11:280b1eb0:::benchmark_data_smithi070_191495_object64731:head,467'4081,11:280b387f:::benchmark_data_smithi070_191495_object101607:head,479'6380,11:280b3faa:::benchmark_data_smithi070_191495_object70884:head,468'4466,11:280bcb58:::benchmark_data_smithi070_191495_object30911:head,463'1971,11:280ca2cc:::benchmark_data_smithi070_191495_object77468:head,471'4859,11:280d6c7c:::benchmark_data_smithi070_191495_object27689:head,463'1775,11:280d7cf7:::benchmark_data_smithi070_191495_object91918:head,476'5762,11:280f316c:::benchmark_data_smithi070_191495_object61406:head,467'3851,11:28103c29:::benchmark_data_smithi070_191495_object124639:head,490'7795,11:28111a31:::benchmark_data_smithi070_191495_object30410:head,463'1937,11:2811c06c:::benchmark_data_smithi070_191495_object74514:head,470'4688,11:2811d5d2:::benchmark_data_smithi070_191495_object62974:head,467'3965,11:2812eff8:::benchmark_data_smithi070_191495_object77854:head,471'4881,11:2812f712:::benchmark_data_smithi070_191495_object79694:head,472'4992,11:2814210a:::benchmark_data_smithi070_191495_object85408:head,474'5378,11:28142aba:::benchmark_data_smithi070_191495_object94914:head,478'5986,11:2814d13d:::benchmark_data_smithi070_191495_object122710:head,488'7673,11:28161f8d:::benchmark_data_smithi070_191495_object111824:head,484'7022,11:2816e234:::benchmark_data_smithi070_191495_object45832:head,464'2883,11:281755eb:::benchmark_data_smithi070_191495_object75500:head,471'4745,11:2818a785:::benchmark_data_smithi070_191495_object61454:head,467'3857,11:2818e36c:::benchmark_data_smithi070_191495_object65138:head,467'4105,11:28199166:::benchmark_data_smithi070_191495_object44470:head,464'2796,11:281a205a:::benchmark_data_smithi070_191495_object87359:head,474'5488,11:281a4216:::benchmark_data_smithi070_191495_object73097:head,469'4608,11:281abd2b:::benchmark_data_smithi070_191495_object99052:head,479'6233,11:281b750c:::benchmark_data_smithi070_191495_object47812:head,464'2989,11:281bb028:::benchmark_data_smithi070_191495_object55383:head,466'3451,11:281ce423:::benchmark_data_smithi070_191495_object42235:head,464'2675,11:281cf986:::benchmark_data_smithi070_191495_object79192:head,472'4961,11:281d2ae2:::benchmark_data_smithi070_191495_object30003:head,463'1912,11:281d6ce4:::benchmark_data_smithi070_191495_object128609:head,491'8047,11:281d9e97:::benchmark_data_smithi070_191495_object82870:head,474'5225,11:281e067d:::benchmark_data_smithi070_191495_object58541:head,466'3645,11:281e1654:::benchmark_data_smithi070_191495_object110887:head,484'6951,11:281e444e:::benchmark_data_smithi070_191495_object92450:head,476'5799,11:281e7b20:::benchmark_data_smithi070_191495_object24478:head,462'1592,11:281eb5a7:::benchmark_data_smithi070_191495_object126859:head,491'7948,11:281ecdda:::benchmark_data_smithi070_191495_object71944:head,468'4541,11:281f037d:::benchmark_data_smithi070_191495_object126714:head,491'7940,11:281f07a9:::benchmark_data_smithi070_191495_object68376:head,468'4312,11:281f15f8:::benchmark_data_smithi070_191495_object43845:head,464'2756,11:281f4e80:::benchmark_data_smithi070_191495_object61384:head,467'3850,11:281f823e:::benchmark_data_smithi070_191495_object96552:head,479'6089,11:281f8c52:::benchmark_data_smithi070_191495_object36229:head,464'2308,11:281fa876:::benchmark_data_smithi070_191495_object46036:head,464'2897,11:281fd266:::benchmark_data_smithi070_191495_object42493:head,464'2682,11:281ff775:::benchmark_data_smithi070_191495_object84847:head,474'5344,11:2820e65e:::benchmark_data_smithi070_191495_object101288:head,479'6359,11:2821a696:::benchmark_data_smithi070_191495_object121795:head,488'7626,11:28220c7f:::benchmark_data_smithi070_191495_object45505:head,464'2867,11:2822a781:::benchmark_data_smithi070_191495_object77088:head,471'4835,11:2822da49:::benchmark_data_smithi070_191495_object111457:head,484'6992,11:2824668e:::benchmark_data_smithi070_191495_object105173:head,481'6595,11:282506db:::benchmark_data_smithi070_191495_object43846:head,464'2757,11:28253563:::benchmark_data_smithi070_191495_object16612:head,460'1076,11:28255fbb:::benchmark_data_smithi070_191495_object127099:head,491'7967,11:282a2cdb:::benchmark_data_smithi070_191495_object122049:head,488'7634,11:282ae2b2:::benchmark_data_smithi070_191495_object121483:head,488'7608,11:282c03cd:::benchmark_data_smithi070_191495_object45357:head,464'2858,11:282c3f1b:::benchmark_data_smithi070_191495_object56328:head,466'3506,11:282c7113:::benchmark_data_smithi070_191495_object78997:head,472'4951,11:282c8c45:::benchmark_data_smithi070_191495_object77887:head,471'4882,11:282ccd78:::benchmark_data_smithi070_191495_object125366:head,490'7844,11:282cfbc3:::benchmark_data_smithi070_191495_object124491:head,489'7783,11:282d2034:::benchmark_data_smithi070_191495_object31379:head,463'2007,11:282d64e8:::benchmark_data_smithi070_191495_object92263:head,476'5788,11:282dfff2:::benchmark_data_smithi070_191495_object118121:head,487'7384,11:282e3092:::benchmark_data_smithi070_191495_object88628:head,474'5555,11:282e39db:::benchmark_data_smithi070_191495_object77966:head,471'4884,11:282e5fc4:::benchmark_data_smithi070_191495_object52357:head,465'3254,11:282ebb1d:::benchmark_data_smithi070_191495_object127224:head,491'7971,11:282eef3d:::benchmark_data_smithi070_191495_object61984:head,467'3889,11:282f9d64:::benchmark_data_smithi070_191495_object59880:head,467'3748,11:28302c0a:::benchmark_data_smithi070_191495_object69660:head,468'4396,11:28308345:::benchmark_data_smithi070_191495_object28171:head,463'1803,11:28308b9f:::benchmark_data_smithi070_191495_object91489:head,476'5736,11:28310190:::benchmark_data_smithi070_191495_object118377:head,487'7392,11:2832eb94:::benchmark_data_smithi070_191495_object100148:head,479'6298,11:28330508:::benchmark_data_smithi070_191495_object47809:head,464'2987,11:28337515:::benchmark_data_smithi070_191495_object93466:head,477'5877,11:283439d1:::benchmark_data_smithi070_191495_object115337:head,486'7208,11:28343b40:::benchmark_data_smithi070_191495_object89590:head,475'5627,11:2834bbd2:::benchmark_data_smithi070_191495_object27904:head,463'1785,11:283522dd:::benchmark_data_smithi070_191495_object23248:head,462'1500,11:28353ee8:::benchmark_data_smithi070_191495_object51346:head,465'3185,11:283574ce:::benchmark_data_smithi070_191495_object100698:head,479'6329,11:2835c2a0:::benchmark_data_smithi070_191495_object123106:head,488'7695,11:2836a52e:::benchmark_data_smithi070_191495_object33658:head,464'2134,11:28377538:::benchmark_data_smithi070_191495_object42034:head,464'2665,11:283775a8:::benchmark_data_smithi070_191495_object36352:head,464'2314,11:283787a4:::benchmark_data_smithi070_191495_object51338:head,465'3184,11:28383989:::benchmark_data_smithi070_191495_object112682:head,485'7067,11:2838a763:::benchmark_data_smithi070_191495_object52258:head,465'3246,11:2838b6f1:::benchmark_data_smithi070_191495_object118515:head,487'7407,11:2838fc3d:::benchmark_data_smithi070_191495_object50233:head,465'3121,11:2839af36:::benchmark_data_smithi070_191495_object91322:head,476'5723,11:283aa592:::benchmark_data_smithi070_191495_object120137:head,488'7520,11:283b9333:::benchmark_data_smithi070_191495_object40439:head,464'2567,11:283bb60f:::benchmark_data_smithi070_191495_object31968:head,463'2035,11:283c5a5f:::benchmark_data_smithi070_191495_object37079:head,464'2351,11:283cc8e2:::benchmark_data_smithi070_191495_object104804:head,480'6572,11:283d2bc7:::benchmark_data_smithi070_191495_object108272:head,482'6794,11:283d46f6:::benchmark_data_smithi070_191495_object88276:head,474'5535,11:283e5d37:::benchmark_data_smithi070_191495_object63525:head,467'4001,11:283e7c09:::benchmark_data_smithi070_191495_object49220:head,465'3062,11:283ee0c0:::benchmark_data_smithi070_191495_object51088:head,465'3164,11:283f8a5c:::benchmark_data_smithi070_191495_object65567:head,467'4135,11:283fb35a:::benchmark_data_smithi070_191495_object106193:head,481'6646,11:2840c975:::benchmark_data_smithi070_191495_object48365:head,465'3024,11:284110e2:::benchmark_data_smithi070_191495_object82841:head,474'5223,11:2841aa03:::benchmark_data_smithi070_191495_object74121:head,469'4666,11:28424490:::benchmark_data_smithi070_191495_object90584:head,476'5682,11:28458d6c:::benchmark_data_smithi070_191495_object35818:head,464'2280,11:28459ff4:::benchmark_data_smithi070_191495_object55595:head,466'3469,11:2845b50e:::benchmark_data_smithi070_191495_object50758:head,465'3147,11:2845e938:::benchmark_data_smithi070_191495_object53202:head,466'3302,11:284629fd:::benchmark_data_smithi070_191495_object64682:head,467'4077,11:28468cc9:::benchmark_data_smithi070_191495_object124071:head,489'7752,11:28469ef4:::benchmark_data_smithi070_191495_object45840:head,464'2888,11:2846dd24:::benchmark_data_smithi070_191495_object72936:head,469'4599,11:28474947:::benchmark_data_smithi070_191495_object101771:head,480'6392,11:28477e19:::benchmark_data_smithi070_191495_object54105:head,466'3363,11:28478892:::benchmark_data_smithi070_191495_object79158:head,472'4959,11:28489327:::benchmark_data_smithi070_191495_object59117:head,466'3703,11:2848cd47:::benchmark_data_smithi070_191495_object63366:head,467'3994,11:284939c5:::benchmark_data_smithi070_191495_object119599:head,488'7476,11:284940a0:::benchmark_data_smithi070_191495_object27640:head,463'1772,11:284becd9:::benchmark_data_smithi070_191495_object28911:head,463'1849,11:284c04cb:::benchmark_data_smithi070_191495_object80343:head,472'5038,11:284c2523:::benchmark_data_smithi070_191495_object85078:head,474'5355,11:284c3df3:::benchmark_data_smithi070_191495_object68163:head,468'4297,11:284d9612:::benchmark_data_smithi070_191495_object51415:head,465'3189,11:284e90d5:::benchmark_data_smithi070_191495_object18880:head,460'1216,11:284f0de4:::benchmark_data_smithi070_191495_object19262:head,460'1242,11:284f3398:::benchmark_data_smithi070_191495_object61812:head,467'3874,11:284fad6b:::benchmark_data_smithi070_191495_object36173:head,464'2304,11:284fb640:::benchmark_data_smithi070_191495_object76960:head,471'4831,11:284fcdbf:::benchmark_data_smithi070_191495_object69986:head,468'4414,11:28507d73:::benchmark_data_smithi070_191495_object38728:head,464'2470,11:285085a0:::benchmark_data_smithi070_191495_object97108:head,479'6118,11:285102a1:::benchmark_data_smithi070_191495_object127963:head,491'8016,11:28510865:::benchmark_data_smithi070_191495_object31088:head,463'1987,11:285135f3:::benchmark_data_smithi070_191495_object66427:head,467'4187,11:28517b45:::benchmark_data_smithi070_191495_object41112:head,464'2618,11:2851869d:::benchmark_data_smithi070_191495_object60929:head,467'3818,11:2851963d:::benchmark_data_smithi070_191495_object77774:head,471'4878,11:2851f71c:::benchmark_data_smithi070_191495_object24120:head,462'1566,11:2851fa85:::benchmark_data_smithi070_191495_object116557:head,486'7284,11:28522193:::benchmark_data_smithi070_191495_object59357:head,466'3719,11:28522416:::benchmark_data_smithi070_191495_object117319:head,486'7338,11:2852fcc7:::benchmark_data_smithi070_191495_object77219:head,471'4845,11:28539a57:::benchmark_data_smithi070_191495_object33405:head,463'2119,11:2854ea8b:::benchmark_data_smithi070_191495_object67292:head,468'4235,11:28565b75:::benchmark_data_smithi070_191495_object50630:head,465'3139,11:28584523:::benchmark_data_smithi070_191495_object29014:head,463'1854,11:28589352:::benchmark_data_smithi070_191495_object98423:head,479'6194,11:2858d9e0:::benchmark_data_smithi070_191495_object73211:head,469'4616,11:285974dc:::benchmark_data_smithi070_191495_object70948:head,468'4470,11:285a0296:::benchmark_data_smithi070_191495_object80494:head,472'5056,11:285a277b:::benchmark_data_smithi070_191495_object23989:head,462'1556,11:285b8645:::benchmark_data_smithi070_191495_object83930:head,474'5279,11:285bc83d:::benchmark_data_smithi070_191495_object61640:head,467'3867,11:285bdfaa:::benchmark_data_smithi070_191495_object108909:head,483'6836,11:285be459:::benchmark_data_smithi070_191495_object42867:head,464'2709,11:285becce:::benchmark_data_smithi070_191495_object73099:head,469'4609,11:285c07bc:::benchmark_data_smithi070_191495_object42813:head,464'2703,11:285e6948:::benchmark_data_smithi070_191495_object65454:head,467'4122,11:285ec4e8:::benchmark_data_smithi070_191495_object107922:head,482'6769,11:285ef52f:::benchmark_data_smithi070_191495_object76240:head,471'4798,11:285f63ca:::benchmark_data_smithi070_191495_object116648:head,486'7291,11:28600dff:::benchmark_data_smithi070_191495_object48338:head,465'3018,11:28605be6:::benchmark_data_smithi070_191495_object115779:head,486'7244,11:2860631e:::benchmark_data_smithi070_191495_object72747:head,468'4588,11:28607dca:::benchmark_data_smithi070_191495_object37218:head,464'2362,11:28611883:::benchmark_data_smithi070_191495_object53891:head,466'3354,11:2861cea1:::benchmark_data_smithi070_191495_object112527:head,485'7057,11:2861d1ad:::benchmark_data_smithi070_191495_object80967:head,473'5092,11:28626e37:::benchmark_data_smithi070_191495_object116698:head,486'7299,11:2862b085:::benchmark_data_smithi070_191495_object24691:head,462'1605,11:2862bed4:::benchmark_data_smithi070_191495_object23437:head,462'1517,11:2863920c:::benchmark_data_smithi070_191495_object104416:head,480'6551,11:286432c9:::benchmark_data_smithi070_191495_object127777:head,491'8001,11:28643d08:::benchmark_data_smithi070_191495_object118388:head,487'7393,11:28644dc9:::benchmark_data_smithi070_191495_object54207:head,466'3373,11:28648f9c:::benchmark_data_smithi070_191495_object128004:head,491'8019,11:2866e2e8:::benchmark_data_smithi070_191495_object40650:head,464'2588,11:2866f1dc:::benchmark_data_smithi070_191495_object74735:head,470'4705,11:286721ef:::benchmark_data_smithi070_191495_object28885:head,463'1844,11:286749d8:::benchmark_data_smithi070_191495_object110167:head,483'6904,11:2868771c:::benchmark_data_smithi070_191495_object36031:head,464'2292,11:2868ce22:::benchmark_data_smithi070_191495_object91546:head,476'5740,11:286950b6:::benchmark_data_smithi070_191495_object24827:head,462'1620,11:286964dd:::benchmark_data_smithi070_191495_object89410:head,474'5614,11:286a1570:::benchmark_data_smithi070_191495_object113845:head,486'7127,11:286a202f:::benchmark_data_smithi070_191495_object102262:head,480'6430,11:286aad8f:::benchmark_data_smithi070_191495_object56301:head,466'3505,11:286b737d:::benchmark_data_smithi070_191495_object100956:head,479'6340,11:286d523b:::benchmark_data_smithi070_191495_object22696:head,461'1459,11:286e6023:::benchmark_data_smithi070_191495_object25533:head,463'1668,11:286edf23:::benchmark_data_smithi070_191495_object79629:head,472'4989,11:286f05de:::benchmark_data_smithi070_191495_object77672:head,471'4874,11:286f5625:::benchmark_data_smithi070_191495_object63848:head,467'4018,11:28708a7e:::benchmark_data_smithi070_191495_object58842:head,466'3674,11:2870d7f8:::benchmark_data_smithi070_191495_object129804:head,491'8118,11:28713734:::benchmark_data_smithi070_191495_object51749:head,465'3213,11:28716348:::benchmark_data_smithi070_191495_object92691:head,476'5815,11:28729593:::benchmark_data_smithi070_191495_object100364:head,479'6311,11:28731268:::benchmark_data_smithi070_191495_object101046:head,479'6348,11:28752260:::benchmark_data_smithi070_191495_object50846:head,465'3151,11:2876882d:::benchmark_data_smithi070_191495_object104068:head,480'6525,11:2876989e:::benchmark_data_smithi070_191495_object38100:head,464'2422,11:2878a54e:::benchmark_data_smithi070_191495_object51262:head,465'3176,11:2879040a:::benchmark_data_smithi070_191495_object125365:head,490'7843,11:2879a5ad:::benchmark_data_smithi070_191495_object102390:head,480'6433,11:2879b25f:::benchmark_data_smithi070_191495_object111298:head,484'6978,11:287abf1f:::benchmark_data_smithi070_191495_object30016:head,463'1913,11:287afa87:::benchmark_data_smithi070_191495_object76742:head,471'4824,11:287b31b3:::benchmark_data_smithi070_191495_object99064:head,479'6234,11:287b5160:::benchmark_data_smithi070_191495_object66306:head,467'4181,11:287b7728:::benchmark_data_smithi070_191495_object39595:head,464'2523,11:287cbbb0:::benchmark_data_smithi070_191495_object78644:head,472'4927,11:287d50ab:::benchmark_data_smithi070_191495_object109870:head,483'6889,11:287d5840:::benchmark_data_smithi070_191495_object22348:head,461'1436,11:287d8f20:::benchmark_data_smithi070_191495_object89919:head,475'5648,11:287de538:::benchmark_data_smithi070_191495_object89822:head,475'5641,11:287e2453:::benchmark_data_smithi070_191495_object86729:head,474'5454,11:287e74b7:::benchmark_data_smithi070_191495_object122600:head,488'7668,11:287ea5a1:::benchmark_data_smithi070_191495_object24794:head,462'1616,11:287eede2:::benchmark_data_smithi070_191495_object27635:head,463'1771,11:287f34ce:::benchmark_data_smithi070_191495_object83925:head,474'5277,11:28813272:::benchmark_data_smithi070_191495_object78067:head,472'4889,11:2883f6b6:::benchmark_data_smithi070_191495_object56658:head,466'3531,11:28843cd9:::benchmark_data_smithi070_191495_object75140:head,470'4730,11:28859542:::benchmark_data_smithi070_191495_object50438:head,465'3131,11:2885b8eb:::benchmark_data_smithi070_191495_object108631:head,483'6819,11:288637f2:::benchmark_data_smithi070_191495_object79954:head,472'5007,11:2886e2ac:::benchmark_data_smithi070_191495_object106430:head,481'6658,11:2887e51b:::benchmark_data_smithi070_191495_object63734:head,467'4011,11:28882aec:::benchmark_data_smithi070_191495_object65109:head,467'4102,11:2888f144:::benchmark_data_smithi070_191495_object17976:head,460'1158,11:2889072c:::benchmark_data_smithi070_191495_object34694:head,464'2191,11:28898fda:::benchmark_data_smithi070_191495_object64532:head,467'4066,11:2889dc82:::benchmark_data_smithi070_191495_object102188:head,480'6423,11:288a02db:::benchmark_data_smithi070_191495_object27723:head,463'1776,11:288a6926:::benchmark_data_smithi070_191495_object112638:head,485'7064,11:288a8d24:::benchmark_data_smithi070_191495_object34464:head,464'2179,11:288ad705:::benchmark_data_smithi070_191495_object97276:head,479'6127,11:288cc8d4:::benchmark_data_smithi070_191495_object60787:head,467'3808,11:288d16cb:::benchmark_data_smithi070_191495_object110051:head,483'6896,11:288d4c83:::benchmark_data_smithi070_191495_object56700:head,466'3534,11:288daad2:::benchmark_data_smithi070_191495_object117452:head,486'7346,11:288e1f28:::benchmark_data_smithi070_191495_object20221:head,460'1298,11:288e5f72:::benchmark_data_smithi070_191495_object120411:head,488'7536,11:288ea144:::benchmark_data_smithi070_191495_object20387:head,460'1310,11:289043a7:::benchmark_data_smithi070_191495_object48958:head,465'3048,11:2890a007:::benchmark_data_smithi070_191495_object107681:head,482'6747,11:2891b083:::benchmark_data_smithi070_191495_object17774:head,460'1140,11:2891c1d8:::benchmark_data_smithi070_191495_object40578:head,464'2577,11:2891c4f4:::benchmark_data_smithi070_191495_object101893:head,480'6404,11:2891c7ab:::benchmark_data_smithi070_191495_object26658:head,463'1722,11:2891f6ac:::benchmark_data_smithi070_191495_object87559:head,474'5497,11:289325ff:::benchmark_data_smithi070_191495_object46526:head,464'2925,11:28944d97:::benchmark_data_smithi070_191495_object100147:head,479'6297,11:289488dc:::benchmark_data_smithi070_191495_object20297:head,460'1304,11:28963ce5:::benchmark_data_smithi070_191495_object116684:head,486'7297,11:28974b4c:::benchmark_data_smithi070_191495_object94337:head,478'5948,11:2898b2af:::benchmark_data_smithi070_191495_object64482:head,467'4060,11:2899e3a5:::benchmark_data_smithi070_191495_object64450:head,467'4057,11:289a0a69:::benchmark_data_smithi070_191495_object72435:head,468'4576,11:289a15d5:::benchmark_data_smithi070_191495_object74199:head,470'4673,11:289a24b1:::benchmark_data_smithi070_191495_object90587:head,476'5683,11:289a6b04:::benchmark_data_smithi070_191495_object93096:head,477'5847,11:289ab54f:::benchmark_data_smithi070_191495_object44940:head,464'2829,11:289b1c49:::benchmark_data_smithi070_191495_object75923:head,471'4772,11:289c4c70:::benchmark_data_smithi070_191495_object101085:head,479'6349,11:289d39fd:::benchmark_data_smithi070_191495_object111909:head,484'7027,11:289df371:::benchmark_data_smithi070_191495_object21647:head,461'1394,11:289dfb28:::benchmark_data_smithi070_191495_object33886:head,464'2142,11:289e0a27:::benchmark_data_smithi070_191495_object80656:head,473'5064,11:289e4dbf:::benchmark_data_smithi070_191495_object65836:head,467'4153,11:289e63be:::benchmark_data_smithi070_191495_object118748:head,487'7421,11:289ed2e6:::benchmark_data_smithi070_191495_object121128:head,488'7586,11:289ee5f3:::benchmark_data_smithi070_191495_object68148:head,468'4293,11:289f120a:::benchmark_data_smithi070_191495_object23086:head,462'1483,11:289f140a:::benchmark_data_smithi070_191495_object94325:head,478'5944,11:289f4faf:::benchmark_data_smithi070_191495_object56866:head,466'3546,11:289fe5b5:::benchmark_data_smithi070_191495_object73836:head,469'4652,11:28a0a944:::benchmark_data_smithi070_191495_object72172:head,468'4556,11:28a0f603:::benchmark_data_smithi070_191495_object119875:head,488'7492,11:28a18137:::benchmark_data_smithi070_191495_object58452:head,466'3641,11:28a19fab:::benchmark_data_smithi070_191495_object123880:head,489'7739,11:28a2881c:::benchmark_data_smithi070_191495_object78861:head,472'4941,11:28a36f48:::benchmark_data_smithi070_191495_object97397:head,479'6136,11:28a4d61c:::benchmark_data_smithi070_191495_object114028:head,486'7131,11:28a577b6:::benchmark_data_smithi070_191495_object93695:head,477'5903,11:28a59c43:::benchmark_data_smithi070_191495_object94263:head,478'5935,11:28a76607:::benchmark_data_smithi070_191495_object51418:head,465'3190,11:28a9138d:::benchmark_data_smithi070_191495_object45398:head,464'2861,11:28a91da1:::benchmark_data_smithi070_191495_object58798:head,466'3670,11:28aa7460:::benchmark_data_smithi070_191495_object127651:head,491'7991,11:28b02a14:::benchmark_data_smithi070_191495_object66321:head,467'4182,11:28b0807b:::benchmark_data_smithi070_191495_object44883:head,464'2822,11:28b0ddf0:::benchmark_data_smithi070_191495_object124425:head,489'7774,11:28b0f722:::benchmark_data_smithi070_191495_object90931:head,476'5703,11:28b12cea:::benchmark_data_smithi070_191495_object92164:head,476'5782,11:28b1a760:::benchmark_data_smithi070_191495_object38943:head,464'2480,11:28b1e78f:::benchmark_data_smithi070_191495_object109781:head,483'6887,11:28b20b3e:::benchmark_data_smithi070_191495_object90486:head,476'5677,11:28b2b3e5:::benchmark_data_smithi070_191495_object70490:head,468'4440,11:28b2bae6:::benchmark_data_smithi070_191495_object66502:head,467'4189,11:28b380f8:::benchmark_data_smithi070_191495_object89039:head,474'5587,11:28b3983b:::benchmark_data_smithi070_191495_object128955:head,491'8071,11:28b42882:::benchmark_data_smithi070_191495_object109723:head,483'6881,11:28b42bd9:::benchmark_data_smithi070_191495_object28609:head,463'1828,11:28b4e15c:::benchmark_data_smithi070_191495_object126394:head,491'7914,11:28b52874:::benchmark_data_smithi070_191495_object80109:head,472'5017,11:28b56586:::benchmark_data_smithi070_191495_object62440:head,467'3921,11:28b5a4d1:::benchmark_data_smithi070_191495_object40333:head,464'2563,11:28b6e247:::benchmark_data_smithi070_191495_object115832:head,486'7248,11:28b74f23:::benchmark_data_smithi070_191495_object45608:head,464'2872,11:28b79334:::benchmark_data_smithi070_191495_object33144:head,463'2103,11:28b8a967:::benchmark_data_smithi070_191495_object44442:head,464'2793,11:28b8e2c5:::benchmark_data_smithi070_191495_object93161:head,477'5853,11:28b8e84b:::benchmark_data_smithi070_191495_object123549:head,489'7719,11:28ba7267:::benchmark_data_smithi070_191495_object79144:head,472'4957,11:28bb49b1:::benchmark_data_smithi070_191495_object104894:head,480'6576,11:28be18b6:::benchmark_data_smithi070_191495_object36007:head,464'2290,11:28be2430:::benchmark_data_smithi070_191495_object101531:head,479'6376,11:28beb09f:::benchmark_data_smithi070_191495_object78922:head,472'4944,11:28bebbe8:::benchmark_data_smithi070_191495_object93599:head,477'5891,11:28bec894:::benchmark_data_smithi070_191495_object104686:head,480'6562,11:28bfb0f5:::benchmark_data_smithi070_191495_object72210:head,468'4558,11:28c0dd59:::benchmark_data_smithi070_191495_object91634:head,476'5748,11:28c18c5a:::benchmark_data_smithi070_191495_object26469:head,463'1711,11:28c1a06a:::benchmark_data_smithi070_191495_object122770:head,488'7677,11:28c1beef:::benchmark_data_smithi070_191495_object129622:head,491'8107,11:28c1eedf:::benchmark_data_smithi070_191495_object45797:head,464'2879,11:28c29349:::benchmark_data_smithi070_191495_object68514:head,468'4324,11:28c40385:::benchmark_data_smithi070_191495_object48352:head,465'3021,11:28c41d50:::benchmark_data_smithi070_191495_object74875:head,470'4713,11:28c4342b:::benchmark_data_smithi070_191495_object30894:head,463'1968,11:28c4ce72:::benchmark_data_smithi070_191495_object89772:head,475'5638,11:28c53b22:::benchmark_data_smithi070_191495_object38160:head,464'2427,11:28c54758:::benchmark_data_smithi070_191495_object121325:head,488'7600,11:28c54ad6:::benchmark_data_smithi070_191495_object113507:head,486'7109,11:28c59d76:::benchmark_data_smithi070_191495_object108200:head,482'6789,11:28c5f1cc:::benchmark_data_smithi070_191495_object39647:head,464'2528,11:28c6aeba:::benchmark_data_smithi070_191495_object32728:head,463'2073,11:28c77cfe:::benchmark_data_smithi070_191495_object100645:head,479'6327,11:28c8750c:::benchmark_data_smithi070_191495_object96518:head,479'6088,11:28c93e21:::benchmark_data_smithi070_191495_object93363:head,477'5870,11:28c97b1e:::benchmark_data_smithi070_191495_object72387:head,468'4572,11:28c9ce4c:::benchmark_data_smithi070_191495_object116713:head,486'7300,11:28c9ebff:::benchmark_data_smithi070_191495_object83336:head,474'5241,11:28ca3cc2:::benchmark_data_smithi070_191495_object57788:head,466'3597,11:28cb001e:::benchmark_data_smithi070_191495_object71017:head,468'4478,11:28cbe2c4:::benchmark_data_smithi070_191495_object16814:head,460'1084,11:28cc8c57:::benchmark_data_smithi070_191495_object19865:head,460'1275,11:28cceddb:::benchmark_data_smithi070_191495_object121028:head,488'7578,11:28cd5ac5:::benchmark_data_smithi070_191495_object63043:head,467'3972,11:28cd6ca9:::benchmark_data_smithi070_191495_object112246:head,485'7046,11:28cdc149:::benchmark_data_smithi070_191495_object81729:head,473'5143,11:28cdcc78:::benchmark_data_smithi070_191495_object22815:head,462'1464,11:28ce1037:::benchmark_data_smithi070_191495_object68668:head,468'4330,11:28ce8022:::benchmark_data_smithi070_191495_object46343:head,464'2914,11:28cf6467:::benchmark_data_smithi070_191495_object17259:head,460'1110,11:28cf7b28:::benchmark_data_smithi070_191495_object128750:head,491'8062,11:28cfe61b:::benchmark_data_smithi070_191495_object45376:head,464'2859,11:28d1a2b1:::benchmark_data_smithi070_191495_object86970:head,474'5466,11:28d28d22:::benchmark_data_smithi070_191495_object41228:head,464'2622,11:28d2d4f7:::benchmark_data_smithi070_191495_object70292:head,468'4434,11:28d3006d:::benchmark_data_smithi070_191495_object103991:head,480'6523,11:28d31605:::benchmark_data_smithi070_191495_object120166:head,488'7522,11:28d323d4:::benchmark_data_smithi070_191495_object78691:head,472'4931,11:28d33ee4:::benchmark_data_smithi070_191495_object119391:head,488'7463,11:28d35ce9:::benchmark_data_smithi070_191495_object25667:head,463'1674,11:28d40830:::benchmark_data_smithi070_191495_object55308:head,466'3447,11:28d42a0b:::benchmark_data_smithi070_191495_object103696:head,480'6502,11:28d45ed5:::benchmark_data_smithi070_191495_object89189:head,474'5597,11:28d7f15c:::benchmark_data_smithi070_191495_object35036:head,464'2221,11:28d8132c:::benchmark_data_smithi070_191495_object122935:head,488'7684,11:28d81b1d:::benchmark_data_smithi070_191495_object88002:head,474'5523,11:28d8444c:::benchmark_data_smithi070_191495_object75858:head,471'4767,11:28d8a856:::benchmark_data_smithi070_191495_object106455:head,481'6661,11:28d8d5cf:::benchmark_data_smithi070_191495_object121859:head,488'7627,11:28d96992:::benchmark_data_smithi070_191495_object16946:head,460'1093,11:28d98a79:::benchmark_data_smithi070_191495_object26350:head,463'1704,11:28da0e26:::benchmark_data_smithi070_191495_object21533:head,461'1387,11:28da2783:::benchmark_data_smithi070_191495_object82323:head,474'5180,11:28dac885:::benchmark_data_smithi070_191495_object36261:head,464'2309,11:28db0115:::benchmark_data_smithi070_191495_object122750:head,488'7675,11:28db5749:::benchmark_data_smithi070_191495_object41316:head,464'2630,11:28db622c:::benchmark_data_smithi070_191495_object90583:head,476'5681,11:28dbc5a2:::benchmark_data_smithi070_191495_object19265:head,460'1243,11:28de0f6a:::benchmark_data_smithi070_191495_object107314:head,481'6721,11:28de89a6:::benchmark_data_smithi070_191495_object39935:head,464'2544,11:28dfbc8f:::benchmark_data_smithi070_191495_object37200:head,464'2360,11:28dfda62:::benchmark_data_smithi070_191495_object71489:head,468'4506,11:28e0125c:::benchmark_data_smithi070_191495_object63268:head,467'3987,11:28e04dde:::benchmark_data_smithi070_191495_object51516:head,465'3200,11:28e14c58:::benchmark_data_smithi070_191495_object110830:head,484'6946,11:28e2dba5:::benchmark_data_smithi070_191495_object86685:head,474'5448,11:28e309fd:::benchmark_data_smithi070_191495_object82537:head,474'5204,11:28e474a7:::benchmark_data_smithi070_191495_object38255:head,464'2436,11:28e60226:::benchmark_data_smithi070_191495_object125205:head,490'7830,11:28e6aadd:::benchmark_data_smithi070_191495_object74631:head,470'4696,11:28e6e505:::benchmark_data_smithi070_191495_object48496:head,465'3029,11:28e70628:::benchmark_data_smithi070_191495_object101528:head,479'6374,11:28e7d477:::benchmark_data_smithi070_191495_object93691:head,477'5902,11:28e97423:::benchmark_data_smithi070_191495_object119995:head,488'7497,11:28eacc44:::benchmark_data_smithi070_191495_object104289:head,480'6543,11:28eb0eb7:::benchmark_data_smithi070_191495_object42820:head,464'2705,11:28eb2bb2:::benchmark_data_smithi070_191495_object86143:head,474'5417,11:28ed7b3e:::benchmark_data_smithi070_191495_object112610:head,485'7062,11:28edc98f:::benchmark_data_smithi070_191495_object68858:head,468'4344,11:28ee5428:::benchmark_data_smithi070_191495_object69727:head,468'4400,11:28ef2a6a:::benchmark_data_smithi070_191495_object87828:head,474'5510,11:28f0228e:::benchmark_data_smithi070_191495_object98672:head,479'6202,11:28f04403:::benchmark_data_smithi070_191495_object87708:head,474'5502,11:28f0e342:::benchmark_data_smithi070_191495_object66721:head,468'4208,11:28f1139a:::benchmark_data_smithi070_191495_object39016:head,464'2483,11:28f1188c:::benchmark_data_smithi070_191495_object57174:head,466'3568,11:28f15f37:::benchmark_data_smithi070_191495_object115142:head,486'7195,11:28f21493:::benchmark_data_smithi070_191495_object36911:head,464'2338,11:28f265ec:::benchmark_data_smithi070_191495_object116363:head,486'7274,11:28f31c90:::benchmark_data_smithi070_191495_object83568:head,474'5258,11:28f35e48:::benchmark_data_smithi070_191495_object58390:head,466'3636,11:28f56237:::benchmark_data_smithi070_191495_object59041:head,466'3693,11:28f60a8a:::benchmark_data_smithi070_191495_object34879:head,464'2210,11:28f64c02:::benchmark_data_smithi070_191495_object31706:head,463'2022,11:28f6c255:::benchmark_data_smithi070_191495_object103686:head,480'6501,11:28f842ac:::benchmark_data_smithi070_191495_object19199:head,460'1238,11:28f88251:::benchmark_data_smithi070_191495_object46628:head,464'2935,11:28f895d6:::benchmark_data_smithi070_191495_object107017:head,481'6698,11:28f90181:::benchmark_data_smithi070_191495_object66129:head,467'4167,11:28f97720:::benchmark_data_smithi070_191495_object107115:head,481'6703,11:28f9a4b4:::benchmark_data_smithi070_191495_object58784:head,466'3668,11:28faada2:::benchmark_data_smithi070_191495_object64214:head,467'4045,11:28fae6a6:::benchmark_data_smithi070_191495_object73658:head,469'4642,11:28fbd5e4:::benchmark_data_smithi070_191495_object30872:head,463'1964,11:28fca4be:::benchmark_data_smithi070_191495_object76486:head,471'4810,11:28fdb615:::benchmark_data_smithi070_191495_object42509:head,464'2683,11:28fdfc79:::benchmark_data_smithi070_191495_object79419:head,472'4977,11:28fe088f:::benchmark_data_smithi070_191495_object101181:head,479'6355,11:28fe18c9:::benchmark_data_smithi070_191495_object99885:head,479'6280,11:28fef660:::benchmark_data_smithi070_191495_object32823:head,463'2079,11:28ff0b3d:::benchmark_data_smithi070_191495_object79335:head,472'4971,11:290340ab:::benchmark_data_smithi070_191495_object108560:head,483'6813,11:290376fc:::benchmark_data_smithi070_191495_object33207:head,463'2106,11:2903af3b:::benchmark_data_smithi070_191495_object65485:head,467'4128,11:2903bf88:::benchmark_data_smithi070_191495_object111819:head,484'7021,11:2903f3c7:::benchmark_data_smithi070_191495_object57153:head,466'3565,11:2903fe36:::benchmark_data_smithi070_191495_object25266:head,463'1647,11:29043bc7:::benchmark_data_smithi070_191495_object77985:head,471'4885,11:290468ac:::benchmark_data_smithi070_191495_object64546:head,467'4067,11:29058496:::benchmark_data_smithi070_191495_object102004:head,480'6411,11:2905c8b4:::benchmark_data_smithi070_191495_object20933:head,461'1345,11:290628d0:::benchmark_data_smithi070_191495_object63261:head,467'3986,11:2908d850:::benchmark_data_smithi070_191495_object41442:head,464'2636,11:2908fe8e:::benchmark_data_smithi070_191495_object34911:head,464'2212,11:2909029f:::benchmark_data_smithi070_191495_object85823:head,474'5398,11:29099648:::benchmark_data_smithi070_191495_object42195:head,464'2673,11:2909b6bc:::benchmark_data_smithi070_191495_object43064:head,464'2720,11:2909bf20:::benchmark_data_smithi070_191495_object69676:head,468'4397,11:290a0671:::benchmark_data_smithi070_191495_object92648:head,476'5811,11:290a2bc1:::benchmark_data_smithi070_191495_object51393:head,465'3188,11:290b419b:::benchmark_data_smithi070_191495_object22378:head,461'1439,11:290c85f1:::benchmark_data_smithi070_191495_object96248:head,478'6067,11:290cd581:::benchmark_data_smithi070_191495_object87226:head,474'5483,11:290d930d:::benchmark_data_smithi070_191495_object113123:head,485'7087,11:290dbab8:::benchmark_data_smithi070_191495_object38165:head,464'2428,11:290dc2a4:::benchmark_data_smithi070_191495_object31522:head,463'2014,11:290e5afe:::benchmark_data_smithi070_191495_object118832:head,487'7425,11:290e9d91:::benchmark_data_smithi070_191495_object22022:head,461'1412,11:290efc1b:::benchmark_data_smithi070_191495_object35594:head,464'2265,11:290fa39b:::benchmark_data_smithi070_191495_object105057:head,481'6587,11:290fae3c:::benchmark_data_smithi070_191495_object24175:head,462'1570,11:2911598d:::benchmark_data_smithi070_191495_object16381:head,460'1058,11:291185ee:::benchmark_data_smithi070_191495_object97967:head,479'6166,11:2912b7a3:::benchmark_data_smithi070_191495_object84479:head,474'5308)", 2024-09-16T09:44:44.716 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:44:26.522502+0000", 2024-09-16T09:44:44.717 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 18.109530827, 2024-09-16T09:44:44.717 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.012478879999999999, 2024-09-16T09:44:44.717 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:44:44.717 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:44:44.717 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T09:44:44.717 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:44:44.717 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.717 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:44:44.718 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.522502+0000", 2024-09-16T09:44:44.718 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:44:44.718 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.718 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.718 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:44:44.718 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.522502+0000", 2024-09-16T09:44:44.718 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:44:44.718 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.718 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.718 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:44:44.718 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.522503+0000", 2024-09-16T09:44:44.718 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.1099999999999995e-07 2024-09-16T09:44:44.718 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.719 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.719 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:44:44.719 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.522698+0000", 2024-09-16T09:44:44.719 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00019532499999999999 2024-09-16T09:44:44.719 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.719 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.719 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:44:44.719 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.523161+0000", 2024-09-16T09:44:44.719 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00046278399999999999 2024-09-16T09:44:44.719 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.719 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.719 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:44:44.719 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.524061+0000", 2024-09-16T09:44:44.720 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00089964199999999998 2024-09-16T09:44:44.720 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.720 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.720 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:44:44.720 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.530920+0000", 2024-09-16T09:44:44.720 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0068598130000000002 2024-09-16T09:44:44.720 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.720 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.720 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:44:44.720 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.531890+0000", 2024-09-16T09:44:44.720 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00096986399999999997 2024-09-16T09:44:44.720 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.720 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.721 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:44:44.721 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.534981+0000", 2024-09-16T09:44:44.721 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.003090841 2024-09-16T09:44:44.721 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:44:44.721 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:44:44.721 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:44:44.721 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.721 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.721 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "backfill_remove(11.d e519 11:ba37603f:::benchmark_data_smithi070_191495_object75709:head,471'4754,11:ba38556d:::benchmark_data_smithi070_191495_object93918:head,477'5950,11:ba396c7e:::benchmark_data_smithi070_191495_object17011:head,460'1086,11:ba398630:::benchmark_data_smithi070_191495_object118478:head,487'7514,11:ba3a1489:::benchmark_data_smithi070_191495_object128429:head,491'8140,11:ba3a918d:::benchmark_data_smithi070_191495_object88878:head,474'5630,11:ba3b55f7:::benchmark_data_smithi070_191495_object75921:head,471'4772,11:ba3cb2d0:::benchmark_data_smithi070_191495_object40493:head,464'2526,11:ba3ceddc:::benchmark_data_smithi070_191495_object86915:head,474'5511,11:ba3d0c6c:::benchmark_data_smithi070_191495_object46544:head,464'2902,11:ba3e558e:::benchmark_data_smithi070_191495_object86437:head,474'5469,11:ba3ebb16:::benchmark_data_smithi070_191495_object25047:head,462'1579,11:ba3ed37e:::benchmark_data_smithi070_191495_object64632:head,467'4050,11:ba3edc15:::benchmark_data_smithi070_191495_object85376:head,474'5389,11:ba3f8b90:::benchmark_data_smithi070_191495_object120521:head,488'7637,11:ba419c01:::benchmark_data_smithi070_191495_object17510:head,460'1123,11:ba41c5c4:::benchmark_data_smithi070_191495_object53958:head,466'3383,11:ba42571d:::benchmark_data_smithi070_191495_object81504:head,473'5155,11:ba427dbc:::benchmark_data_smithi070_191495_object96422:head,478'6114,11:ba436de3:::benchmark_data_smithi070_191495_object17380:head,460'1114,11:ba44982d:::benchmark_data_smithi070_191495_object75530:head,471'4738,11:ba469571:::benchmark_data_smithi070_191495_object81179:head,473'5137,11:ba46d8fa:::benchmark_data_smithi070_191495_object77062:head,471'4866,11:ba48929d:::benchmark_data_smithi070_191495_object19377:head,460'1243,11:ba48ab74:::benchmark_data_smithi070_191495_object100955:head,479'6413,11:ba49a158:::benchmark_data_smithi070_191495_object61057:head,467'3826,11:ba49cdc1:::benchmark_data_smithi070_191495_object80921:head,473'5116,11:ba49f80c:::benchmark_data_smithi070_191495_object106843:head,481'6770,11:ba4a6ae7:::benchmark_data_smithi070_191495_object81786:head,473'5181,11:ba4ac9e2:::benchmark_data_smithi070_191495_object80504:head,472'5092,11:ba4c3d6e:::benchmark_data_smithi070_191495_object33750:head,464'2104,11:ba4cde03:::benchmark_data_smithi070_191495_object17213:head,460'1104,11:ba4cde21:::benchmark_data_smithi070_191495_object66460:head,467'4153,11:ba4dbbfc:::benchmark_data_smithi070_191495_object82200:head,474'5203,11:ba4e2f12:::benchmark_data_smithi070_191495_object42375:head,464'2651,11:ba4ebc53:::benchmark_data_smithi070_191495_object58111:head,466'3656,11:ba4f05fe:::benchmark_data_smithi070_191495_object47433:head,464'2971,11:ba4f379c:::benchmark_data_smithi070_191495_object103502:head,480'6569,11:ba5082b6:::benchmark_data_smithi070_191495_object123949:head,489'7867,11:ba513290:::benchmark_data_smithi070_191495_object106683:head,481'6761,11:ba51c508:::benchmark_data_smithi070_191495_object69394:head,468'4333,11:ba52fc53:::benchmark_data_smithi070_191495_object76747:head,471'4834,11:ba5351bc:::benchmark_data_smithi070_191495_object96242:head,478'6099,11:ba549613:::benchmark_data_smithi070_191495_object23963:head,462'1507,11:ba54c2b3:::benchmark_data_smithi070_191495_object122761:head,488'7789,11:ba5518db:::benchmark_data_smithi070_191495_object39662:head,464'2473,11:ba552239:::benchmark_data_smithi070_191495_object70446:head,468'4397,11:ba558515:::benchmark_data_smithi070_191495_object56717:head,466'3561,11:ba5597ee:::benchmark_data_smithi070_191495_object75991:head,471'4777,11:ba55a812:::benchmark_data_smithi070_191495_object16669:head,460'1056,11:ba560cbb:::benchmark_data_smithi070_191495_object18196:head,460'1172,11:ba56285f:::benchmark_data_smithi070_191495_object83857:head,474'5299,11:ba565a89:::benchmark_data_smithi070_191495_object34346:head,464'2143,11:ba566dd3:::benchmark_data_smithi070_191495_object88072:head,474'5580,11:ba5750f5:::benchmark_data_smithi070_191495_object17197:head,460'1103,11:ba575359:::benchmark_data_smithi070_191495_object123291:head,488'7819,11:ba584afb:::benchmark_data_smithi070_191495_object23188:head,462'1464,11:ba58cc8a:::benchmark_data_smithi070_191495_object68001:head,468'4254,11:ba59451b:::benchmark_data_smithi070_191495_object85726:head,474'5420,11:ba5a73b3:::benchmark_data_smithi070_191495_object87710:head,474'5563,11:ba5a81db:::benchmark_data_smithi070_191495_object105879:head,481'6709,11:ba5b52c9:::benchmark_data_smithi070_191495_object93978:head,477'5960,11:ba5c42fb:::benchmark_data_smithi070_191495_object84778:head,474'5348,11:ba5c8d5e:::benchmark_data_smithi070_191495_object27353:head,463'1709,11:ba5d0deb:::benchmark_data_smithi070_191495_object78109:head,472'4934,11:ba5d0f16:::benchmark_data_smithi070_191495_object110460:head,483'7001,11:ba5d1682:::benchmark_data_smithi070_191495_object37639:head,464'2348,11:ba5de146:::benchmark_data_smithi070_191495_object77420:head,471'4897,11:ba5e359a:::benchmark_data_smithi070_191495_object66656:head,468'4164,11:ba5ef23a:::benchmark_data_smithi070_191495_object88863:head,474'5627,11:ba615907:::benchmark_data_smithi070_191495_object62323:head,467'3909,11:ba618dca:::benchmark_data_smithi070_191495_object98346:head,479'6243,11:ba61957f:::benchmark_data_smithi070_191495_object89318:head,474'5653,11:ba623af7:::benchmark_data_smithi070_191495_object105951:head,481'6714,11:ba62c246:::benchmark_data_smithi070_191495_object123316:head,488'7820,11:ba634803:::benchmark_data_smithi070_191495_object18886:head,460'1213,11:ba637c5f:::benchmark_data_smithi070_191495_object87049:head,474'5517,11:ba6414da:::benchmark_data_smithi070_191495_object96421:head,478'6113,11:ba6439ef:::benchmark_data_smithi070_191495_object31176:head,463'1939,11:ba64af0e:::benchmark_data_smithi070_191495_object96743:head,479'6143,11:ba652fec:::benchmark_data_smithi070_191495_object20207:head,460'1297,11:ba65b1a6:::benchmark_data_smithi070_191495_object53898:head,466'3381,11:ba65c782:::benchmark_data_smithi070_191495_object38718:head,464'2415,11:ba6646b4:::benchmark_data_smithi070_191495_object19913:head,460'1281,11:ba667354:::benchmark_data_smithi070_191495_object62609:head,467'3927,11:ba6726ab:::benchmark_data_smithi070_191495_object92719:head,476'5885,11:ba672a91:::benchmark_data_smithi070_191495_object74570:head,470'4673,11:ba6853d6:::benchmark_data_smithi070_191495_object41795:head,464'2602,11:ba686e75:::benchmark_data_smithi070_191495_object48780:head,465'3060,11:ba6a1a97:::benchmark_data_smithi070_191495_object82510:head,474'5227,11:ba6a422c:::benchmark_data_smithi070_191495_object126666:head,491'8035,11:ba6aea15:::benchmark_data_smithi070_191495_object62150:head,467'3894,11:ba6bb588:::benchmark_data_smithi070_191495_object103342:head,480'6556,11:ba6c6e78:::benchmark_data_smithi070_191495_object40854:head,464'2546,11:ba6cd597:::benchmark_data_smithi070_191495_object77933:head,471'4926,11:ba6cdf15:::benchmark_data_smithi070_191495_object59216:head,466'3711,11:ba6d1644:::benchmark_data_smithi070_191495_object103605:head,480'6576,11:ba6d2c85:::benchmark_data_smithi070_191495_object128790:head,491'8169,11:ba6db3d8:::benchmark_data_smithi070_191495_object60898:head,467'3815,11:ba6e3eb3:::benchmark_data_smithi070_191495_object122875:head,488'7795,11:ba6e67d0:::benchmark_data_smithi070_191495_object26037:head,463'1630,11:ba6edebe:::benchmark_data_smithi070_191495_object129768:head,491'8236,11:ba6f7b55:::benchmark_data_smithi070_191495_object24701:head,462'1563,11:ba70060f:::benchmark_data_smithi070_191495_object106876:head,481'6772,11:ba70d203:::benchmark_data_smithi070_191495_object48159:head,465'3018,11:ba715dca:::benchmark_data_smithi070_191495_object109717:head,483'6959,11:ba71c324:::benchmark_data_smithi070_191495_object67188:head,468'4198,11:ba71d251:::benchmark_data_smithi070_191495_object21745:head,461'1379,11:ba72c57c:::benchmark_data_smithi070_191495_object48400:head,465'3031,11:ba731364:::benchmark_data_smithi070_191495_object30257:head,463'1886,11:ba739de2:::benchmark_data_smithi070_191495_object78765:head,472'4976,11:ba74b473:::benchmark_data_smithi070_191495_object48994:head,465'3077,11:ba753cd7:::benchmark_data_smithi070_191495_object130272:head,492'8259,11:ba7593ce:::benchmark_data_smithi070_191495_object59413:head,466'3724,11:ba7616ec:::benchmark_data_smithi070_191495_object120230:head,488'7621,11:ba771e6f:::benchmark_data_smithi070_191495_object67423:head,468'4213,11:ba780e8e:::benchmark_data_smithi070_191495_object32035:head,463'1987,11:ba7a010a:::benchmark_data_smithi070_191495_object72972:head,469'4570,11:ba7a45a6:::benchmark_data_smithi070_191495_object54435:head,466'3419,11:ba7a4fd4:::benchmark_data_smithi070_191495_object91040:head,476'5755,11:ba7aa971:::benchmark_data_smithi070_191495_object59626:head,466'3733,11:ba7b1199:::benchmark_data_smithi070_191495_object91682:head,476'5798,11:ba7b5585:::benchmark_data_smithi070_191495_object67645:head,468'4229,11:ba7b69e5:::benchmark_data_smithi070_191495_object99674:head,479'6329,11:ba7badf4:::benchmark_data_smithi070_191495_object80849:head,473'5111,11:ba7ce200:::benchmark_data_smithi070_191495_object47115:head,464'2942,11:ba7d4904:::benchmark_data_smithi070_191495_object32624:head,463'2027,11:ba80571a:::benchmark_data_smithi070_191495_object114134:head,486'7240,11:ba80a6b9:::benchmark_data_smithi070_191495_object107159:head,481'6786,11:ba829b9b:::benchmark_data_smithi070_191495_object63606:head,467'3982,11:ba82d3d4:::benchmark_data_smithi070_191495_object58698:head,466'3687,11:ba8307b5:::benchmark_data_smithi070_191495_object115152:head,486'7298,11:ba83930e:::benchmark_data_smithi070_191495_object51625:head,465'3240,11:ba839633:::benchmark_data_smithi070_191495_object128214:head,491'8130,11:ba83e9f8:::benchmark_data_smithi070_191495_object83877:head,474'5301,11:ba84b791:::benchmark_data_smithi070_191495_object41931:head,464'2615,11:ba85cf48:::benchmark_data_smithi070_191495_object28874:head,463'1799,11:ba8601e7:::benchmark_data_smithi070_191495_object111711:head,484'7084,11:ba86389c:::benchmark_data_smithi070_191495_object127357:head,491'8077,11:ba863a83:::benchmark_data_smithi070_191495_object122856:head,488'7793,11:ba86cd2a:::benchmark_data_smithi070_191495_object73047:head,469'4574,11:ba8746e4:::benchmark_data_smithi070_191495_object48732:head,465'3057,11:ba88a016:::benchmark_data_smithi070_191495_object72656:head,468'4545,11:ba88ab2c:::benchmark_data_smithi070_191495_object27186:head,463'1703,11:ba8c6c7d:::benchmark_data_smithi070_191495_object19547:head,460'1256,11:ba8c7f1b:::benchmark_data_smithi070_191495_object95652:head,478'6064,11:ba8d7a6a:::benchmark_data_smithi070_191495_object99312:head,479'6302,11:ba8dc0ce:::benchmark_data_smithi070_191495_object115839:head,486'7337,11:ba8e5aeb:::benchmark_data_smithi070_191495_object17548:head,460'1127,11:ba8ec634:::benchmark_data_smithi070_191495_object115066:head,486'7295,11:ba8f1276:::benchmark_data_smithi070_191495_object118425:head,487'7509,11:ba8f312e:::benchmark_data_smithi070_191495_object24901:head,462'1574,11:ba901875:::benchmark_data_smithi070_191495_object63396:head,467'3970,11:ba903898:::benchmark_data_smithi070_191495_object109872:head,483'6965,11:ba904309:::benchmark_data_smithi070_191495_object98375:head,479'6247,11:ba91e535:::benchmark_data_smithi070_191495_object41465:head,464'2580,11:ba92b8ec:::benchmark_data_smithi070_191495_object54718:head,466'3441,11:ba931cd4:::benchmark_data_smithi070_191495_object85102:head,474'5365,11:ba93c663:::benchmark_data_smithi070_191495_object82362:head,474'5214,11:ba9444b1:::benchmark_data_smithi070_191495_object31149:head,463'1935,11:ba947177:::benchmark_data_smithi070_191495_object17741:head,460'1138,11:ba95597a:::benchmark_data_smithi070_191495_object129649:head,491'8224,11:ba986f61:::benchmark_data_smithi070_191495_object16555:head,460'1051,11:ba98bbf1:::benchmark_data_smithi070_191495_object53576:head,466'3361,11:ba993316:::benchmark_data_smithi070_191495_object16969:head,460'1081,11:ba993dac:::benchmark_data_smithi070_191495_object75490:head,471'4737,11:ba9961f1:::benchmark_data_smithi070_191495_object124098:head,489'7877,11:ba99f3e3:::benchmark_data_smithi070_191495_object80227:head,472'5077,11:ba9a0900:::benchmark_data_smithi070_191495_object25675:head,463'1608,11:ba9ac3d3:::benchmark_data_smithi070_191495_object63996:head,467'4006,11:ba9c1d0c:::benchmark_data_smithi070_191495_object24352:head,462'1535,11:ba9c1e4f:::benchmark_data_smithi070_191495_object71728:head,468'4481,11:ba9c4cec:::benchmark_data_smithi070_191495_object54687:head,466'3437,11:ba9cf174:::benchmark_data_smithi070_191495_object48895:head,465'3069,11:ba9d85b5:::benchmark_data_smithi070_191495_object65663:head,467'4113,11:ba9e2e68:::benchmark_data_smithi070_191495_object109427:head,483'6935,11:ba9e462e:::benchmark_data_smithi070_191495_object103638:head,480'6583,11:ba9e9658:::benchmark_data_smithi070_191495_object50633:head,465'3182,11:ba9effd7:::benchmark_data_smithi070_191495_object41654:head,464'2592,11:ba9f3c6d:::benchmark_data_smithi070_191495_object26582:head,463'1663,11:baa1ac74:::benchmark_data_smithi070_191495_object43291:head,464'2709,11:baa20e21:::benchmark_data_smithi070_191495_object65409:head,467'4105,11:baa24820:::benchmark_data_smithi070_191495_object69995:head,468'4366,11:baa3fd98:::benchmark_data_smithi070_191495_object92016:head,476'5828,11:baa496b5:::benchmark_data_smithi070_191495_object36269:head,464'2257,11:baa55bd8:::benchmark_data_smithi070_191495_object124264:head,489'7890,11:baa5e82e:::benchmark_data_smithi070_191495_object43358:head,464'2712,11:baa6c5a4:::benchmark_data_smithi070_191495_object128848:head,491'8173,11:baa6d9dc:::benchmark_data_smithi070_191495_object113521:head,486'7206,11:baa7f17b:::benchmark_data_smithi070_191495_object92544:head,476'5871,11:baa81ec4:::benchmark_data_smithi070_191495_object47587:head,464'2977,11:baaa07a6:::benchmark_data_smithi070_191495_object35702:head,464'2224,11:baab3cf2:::benchmark_data_smithi070_191495_object62462:head,467'3921,11:baab9e34:::benchmark_data_smithi070_191495_object125395:head,490'7956,11:baac61d2:::benchmark_data_smithi070_191495_object70219:head,468'4386,11:baaceab9:::benchmark_data_smithi070_191495_object67406:head,468'4212,11:baaf9e8f:::benchmark_data_smithi070_191495_object82980:head,474'5253,11:bab07629:::benchmark_data_smithi070_191495_object104766:head,480'6650,11:bab0c74c:::benchmark_data_smithi070_191495_object91958:head,476'5822,11:bab12a46:::benchmark_data_smithi070_191495_object108098:head,482'6847,11:bab19088:::benchmark_data_smithi070_191495_object77055:head,471'4865,11:bab1c723:::benchmark_data_smithi070_191495_object31726:head,463'1966,11:bab23660:::benchmark_data_smithi070_191495_object106690:head,481'6762,11:bab28633:::benchmark_data_smithi070_191495_object58019:head,466'3650,11:bab40c91:::benchmark_data_smithi070_191495_object127679:head,491'8096,11:bab4d0c7:::benchmark_data_smithi070_191495_object34192:head,464'2131,11:bab4f152:::benchmark_data_smithi070_191495_object64355:head,467'4031,11:bab51b81:::benchmark_data_smithi070_191495_object72778:head,468'4552,11:bab587de:::benchmark_data_smithi070_191495_object96935:head,479'6157,11:bab656e0:::benchmark_data_smithi070_191495_object32636:head,463'2028,11:bab6638c:::benchmark_data_smithi070_191495_object81062:head,473'5131,11:bab6fc83:::benchmark_data_smithi070_191495_object72315:head,468'4521,11:bab75bf4:::benchmark_data_smithi070_191495_object72692:head,468'4547,11:bab8723d:::benchmark_data_smithi070_191495_object79097:head,472'4994,11:bab87a0c:::benchmark_data_smithi070_191495_object79332:head,472'5014,11:bab8a376:::benchmark_data_smithi070_191495_object99561:head,479'6320,11:bab92e66:::benchmark_data_smithi070_191495_object123838:head,489'7858,11:bab9433f:::benchmark_data_smithi070_191495_object93241:head,477'5920,11:bab94c20:::benchmark_data_smithi070_191495_object126583:head,491'8030,11:bab97985:::benchmark_data_smithi070_191495_object97901:head,479'6215,11:bab9dcbc:::benchmark_data_smithi070_191495_object37680:head,464'2351,11:baba00b8:::benchmark_data_smithi070_191495_object37875:head,464'2366,11:babada38:::benchmark_data_smithi070_191495_object74757:head,470'4686,11:babb896c:::benchmark_data_smithi070_191495_object123522:head,489'7836,11:babef4c4:::benchmark_data_smithi070_191495_object122130:head,488'7744,11:babfdf54:::benchmark_data_smithi070_191495_object95597:head,478'6059,11:bac03a72:::benchmark_data_smithi070_191495_object96849:head,479'6152,11:bac25282:::benchmark_data_smithi070_191495_object35287:head,464'2199,11:bac4aefd:::benchmark_data_smithi070_191495_object66386:head,467'4150,11:bac5a867:::benchmark_data_smithi070_191495_object29895:head,463'1867,11:bac783f4:::benchmark_data_smithi070_191495_object87446:head,474'5543,11:bac7a8e9:::benchmark_data_smithi070_191495_object41845:head,464'2608,11:bac7b7c0:::benchmark_data_smithi070_191495_object123845:head,489'7859,11:bac8b508:::benchmark_data_smithi070_191495_object126943:head,491'8055,11:bac9db58:::benchmark_data_smithi070_191495_object56871:head,466'3578,11:bac9dff3:::benchmark_data_smithi070_191495_object40637:head,464'2536,11:bacadc1e:::benchmark_data_smithi070_191495_object47140:head,464'2947,11:bacaec2e:::benchmark_data_smithi070_191495_object48458:head,465'3035,11:bacbc780:::benchmark_data_smithi070_191495_object55972:head,466'3525,11:bacc28a6:::benchmark_data_smithi070_191495_object78917:head,472'4986,11:bacd5f51:::benchmark_data_smithi070_191495_object50566:head,465'3176,11:bacd74d5:::benchmark_data_smithi070_191495_object97546:head,479'6197,11:bace3168:::benchmark_data_smithi070_191495_object27902:head,463'1747,11:bace5a8d:::benchmark_data_smithi070_191495_object51535:head,465'3235,11:bacf52e8:::benchmark_data_smithi070_191495_object33590:head,464'2095,11:bad00a1d:::benchmark_data_smithi070_191495_object80354:head,472'5084,11:bad01167:::benchmark_data_smithi070_191495_object88309:head,474'5596,11:bad0b305:::benchmark_data_smithi070_191495_object102661:head,480'6509,11:bad0d026:::benchmark_data_smithi070_191495_object113647:head,486'7210,11:bad1e1cb:::benchmark_data_smithi070_191495_object33127:head,463'2066,11:bad226a6:::benchmark_data_smithi070_191495_object124131:head,489'7880,11:bad2389e:::benchmark_data_smithi070_191495_object88333:head,474'5600,11:bad3b8ec:::benchmark_data_smithi070_191495_object103618:head,480'6579,11:bad3f3c9:::benchmark_data_smithi070_191495_object120785:head,488'7656,11:bad40062:::benchmark_data_smithi070_191495_object115371:head,486'7308,11:bad50568:::benchmark_data_smithi070_191495_object48518:head,465'3041,11:bad59917:::benchmark_data_smithi070_191495_object71753:head,468'4482,11:bad67aeb:::benchmark_data_smithi070_191495_object45816:head,464'2859,11:bad6c919:::benchmark_data_smithi070_191495_object114161:head,486'7242,11:bad7b309:::benchmark_data_smithi070_191495_object70503:head,468'4401,11:bad85eb4:::benchmark_data_smithi070_191495_object111827:head,484'7094,11:bad8e512:::benchmark_data_smithi070_191495_object107375:head,481'6803,11:bad8e6e9:::benchmark_data_smithi070_191495_object95311:head,478'6038,11:bad9b8cf:::benchmark_data_smithi070_191495_object31448:head,463'1953,11:bada20f6:::benchmark_data_smithi070_191495_object50983:head,465'3199,11:badb0cdf:::benchmark_data_smithi070_191495_object19022:head,460'1220,11:badb1687:::benchmark_data_smithi070_191495_object80894:head,473'5114,11:badb1968:::benchmark_data_smithi070_191495_object72851:head,468'4557,11:badb63e2:::benchmark_data_smithi070_191495_object99559:head,479'6319,11:badbe6ec:::benchmark_data_smithi070_191495_object49636:head,465'3119,11:badc31ad:::benchmark_data_smithi070_191495_object117698:head,486'7456,11:badd59c6:::benchmark_data_smithi070_191495_object47343:head,464'2961,11:badf3df5:::benchmark_data_smithi070_191495_object113752:head,486'7217,11:badf9a3e:::benchmark_data_smithi070_191495_object114500:head,486'7258,11:badfd71f:::benchmark_data_smithi070_191495_object123530:head,489'7838,11:bae0e787:::benchmark_data_smithi070_191495_object37968:head,464'2375,11:bae1633d:::benchmark_data_smithi070_191495_object101506:head,479'6446,11:bae17c28:::benchmark_data_smithi070_191495_object125062:head,490'7936,11:bae18bb8:::benchmark_data_smithi070_191495_object121079:head,488'7674,11:bae252bf:::benchmark_data_smithi070_191495_object35703:head,464'2225,11:bae25fe9:::benchmark_data_smithi070_191495_object41136:head,464'2556,11:bae3a50b:::benchmark_data_smithi070_191495_object87565:head,474'5551,11:bae40fb6:::benchmark_data_smithi070_191495_object125844:head,490'7982,11:bae43248:::benchmark_data_smithi070_191495_object35244:head,464'2196,11:bae4b28f:::benchmark_data_smithi070_191495_object122230:head,488'7755,11:bae4b717:::benchmark_data_smithi070_191495_object33078:head,463'2063,11:bae70d35:::benchmark_data_smithi070_191495_object41535:head,464'2586,11:bae772c9:::benchmark_data_smithi070_191495_object111717:head,484'7085,11:bae79672:::benchmark_data_smithi070_191495_object116733:head,486'7399,11:bae9257e:::benchmark_data_smithi070_191495_object120224:head,488'7620,11:baea180b:::benchmark_data_smithi070_191495_object76601:head,471'4827,11:baea9910:::benchmark_data_smithi070_191495_object32536:head,463'2018,11:baeb1b83:::benchmark_data_smithi070_191495_object107322:head,481'6797,11:baec21a4:::benchmark_data_smithi070_191495_object82460:head,474'5221,11:baed37e3:::benchmark_data_smithi070_191495_object60943:head,467'3817,11:baed5681:::benchmark_data_smithi070_191495_object127615:head,491'8091,11:baee4c41:::benchmark_data_smithi070_191495_object66984:head,468'4183,11:baef09ca:::benchmark_data_smithi070_191495_object63167:head,467'3952,11:baef47e6:::benchmark_data_smithi070_191495_object123426:head,488'7829,11:baef579b:::benchmark_data_smithi070_191495_object64073:head,467'4009,11:baf06771:::benchmark_data_smithi070_191495_object24374:head,462'1538,11:baf1764e:::benchmark_data_smithi070_191495_object127490:head,491'8087,11:baf285c6:::benchmark_data_smithi070_191495_object124383:head,489'7900,11:baf2d0a1:::benchmark_data_smithi070_191495_object120860:head,488'7661,11:baf39574:::benchmark_data_smithi070_191495_object85736:head,474'5422,11:baf6575c:::benchmark_data_smithi070_191495_object36230:head,464'2252,11:baf6867f:::benchmark_data_smithi070_191495_object118820:head,487'7538,11:baf69efd:::benchmark_data_smithi070_191495_object49489:head,465'3108,11:baf6bf16:::benchmark_data_smithi070_191495_object86804:head,474'5498,11:baf74d96:::benchmark_data_smithi070_191495_object29866:head,463'1864,11:baf7fa38:::benchmark_data_smithi070_191495_object52268:head,465'3278,11:baf820cb:::benchmark_data_smithi070_191495_object26830:head,463'1680,11:baf8319c:::benchmark_data_smithi070_191495_object49771:head,465'3127,11:bafa0b8d:::benchmark_data_smithi070_191495_object31256:head,463'1943,11:bafae20d:::benchmark_data_smithi070_191495_object56941:head,466'3585,11:bafb5f02:::benchmark_data_smithi070_191495_object54052:head,466'3392,11:bafc97ed:::benchmark_data_smithi070_191495_object31601:head,463'1957,11:bafd777f:::benchmark_data_smithi070_191495_object52443:head,466'3288,11:baff9c40:::benchmark_data_smithi070_191495_object90279:head,475'5716,11:bb00f7d2:::benchmark_data_smithi070_191495_object124003:head,489'7872,11:bb012ae8:::benchmark_data_smithi070_191495_object123184:head,488'7815,11:bb01bfce:::benchmark_data_smithi070_191495_object90253:head,475'5714,11:bb027d98:::benchmark_data_smithi070_191495_object42386:head,464'2653,11:bb03b4f5:::benchmark_data_smithi070_191495_object120450:head,488'7632,11:bb04382d:::benchmark_data_smithi070_191495_object30803:head,463'1907,11:bb054ee8:::benchmark_data_smithi070_191495_object123963:head,489'7870,11:bb06159c:::benchmark_data_smithi070_191495_object64291:head,467'4026,11:bb06c9c6:::benchmark_data_smithi070_191495_object77222:head,471'4881,11:bb06ec3c:::benchmark_data_smithi070_191495_object48054:head,465'3009,11:bb071ce9:::benchmark_data_smithi070_191495_object27135:head,463'1700,11:bb07412e:::benchmark_data_smithi070_191495_object39185:head,464'2448,11:bb08616f:::benchmark_data_smithi070_191495_object79806:head,472'5049,11:bb08b72b:::benchmark_data_smithi070_191495_object118217:head,487'7496,11:bb0a20e2:::benchmark_data_smithi070_191495_object102783:head,480'6519,11:bb0b44a8:::benchmark_data_smithi070_191495_object99148:head,479'6296,11:bb0bf671:::benchmark_data_smithi070_191495_object120550:head,488'7638,11:bb0c58b3:::benchmark_data_smithi070_191495_object74830:head,470'4689,11:bb0cb56f:::benchmark_data_smithi070_191495_object40970:head,464'2552,11:bb0effe7:::benchmark_data_smithi070_191495_object111897:head,484'7103,11:bb0fa2a3:::benchmark_data_smithi070_191495_object70074:head,468'4372,11:bb0fe0dd:::benchmark_data_smithi070_191495_object86719:head,474'5494,11:bb0ff272:::benchmark_data_smithi070_191495_object49476:head,465'3107,11:bb10cfbe:::benchmark_data_smithi070_191495_object52024:head,465'3267,11:bb10fb67:::benchmark_data_smithi070_191495_object75573:head,471'4743,11:bb11ece7:::benchmark_data_smithi070_191495_object127336:head,491'8076,11:bb1265a8:::benchmark_data_smithi070_191495_object91689:head,476'5799,11:bb13e142:::benchmark_data_smithi070_191495_object29980:head,463'1873,11:bb148ea4:::benchmark_data_smithi070_191495_object75238:head,470'4719,11:bb14a1e1:::benchmark_data_smithi070_191495_object41836:head,464'2607,11:bb153076:::benchmark_data_smithi070_191495_object82804:head,474'5243,11:bb15fcf9:::benchmark_data_smithi070_191495_object34756:head,464'2163,11:bb167f9f:::benchmark_data_smithi070_191495_object66141:head,467'4141,11:bb16a9d6:::benchmark_data_smithi070_191495_object78597:head,472'4968,11:bb16ced6:::benchmark_data_smithi070_191495_object93200:head,477'5915,11:bb18045a:::benchmark_data_smithi070_191495_object127736:head,491'8099,11:bb186df6:::benchmark_data_smithi070_191495_object28731:head,463'1790,11:bb18a3c8:::benchmark_data_smithi070_191495_object125022:head,490'7932,11:bb19bb6a:::benchmark_data_smithi070_191495_object125668:head,490'7972,11:bb19c28e:::benchmark_data_smithi070_191495_object27086:head,463'1697,11:bb19cb0d:::benchmark_data_smithi070_191495_object42296:head,464'2644,11:bb19f790:::benchmark_data_smithi070_191495_object66587:head,468'4161,11:bb1a82a6:::benchmark_data_smithi070_191495_object85041:head,474'5359,11:bb1c11c9:::benchmark_data_smithi070_191495_object81531:head,473'5159,11:bb1d3c7f:::benchmark_data_smithi070_191495_object62518:head,467'3922,11:bb1e93f3:::benchmark_data_smithi070_191495_object54613:head,466'3430,11:bb1f795d:::benchmark_data_smithi070_191495_object40298:head,464'2511,11:bb1fe127:::benchmark_data_smithi070_191495_object105146:head,481'6677,11:bb1ff76c:::benchmark_data_smithi070_191495_object36290:head,464'2259,11:bb203784:::benchmark_data_smithi070_191495_object65512:head,467'4108,11:bb203c1b:::benchmark_data_smithi070_191495_object119603:head,488'7587,11:bb209e3d:::benchmark_data_smithi070_191495_object128776:head,491'8167,11:bb217f53:::benchmark_data_smithi070_191495_object44853:head,464'2802,11:bb22d588:::benchmark_data_smithi070_191495_object67128:head,468'4193,11:bb230450:::benchmark_data_smithi070_191495_object63747:head,467'3988,11:bb239cef:::benchmark_data_smithi070_191495_object77009:head,471'4857,11:bb23cc3a:::benchmark_data_smithi070_191495_object83780:head,474'5295,11:bb250e03:::benchmark_data_smithi070_191495_object124514:head,489'7907,11:bb2551d2:::benchmark_data_smithi070_191495_object38106:head,464'2382,11:bb26c85d:::benchmark_data_smithi070_191495_object56868:head,466'3577,11:bb276a13:::benchmark_data_smithi070_191495_object50092:head,465'3144,11:bb281744:::benchmark_data_smithi070_191495_object28758:head,463'1793,11:bb283e0d:::benchmark_data_smithi070_191495_object74341:head,470'4659,11:bb286c4e:::benchmark_data_smithi070_191495_object81562:head,473'5163,11:bb289487:::benchmark_data_smithi070_191495_object59995:head,467'3758,11:bb28d391:::benchmark_data_smithi070_191495_object104475:head,480'6629,11:bb29294a:::benchmark_data_smithi070_191495_object83602:head,474'5279,11:bb29b2bf:::benchmark_data_smithi070_191495_object27407:head,463'1712,11:bb2ce09a:::benchmark_data_smithi070_191495_object33332:head,463'2076,11:bb2d503b:::benchmark_data_smithi070_191495_object107582:head,482'6814,11:bb2d5610:::benchmark_data_smithi070_191495_object58479:head,466'3677,11:bb2e7238:::benchmark_data_smithi070_191495_object122381:head,488'7769,11:bb2edb0b:::benchmark_data_smithi070_191495_object24222:head,462'1524,11:bb2f00bf:::benchmark_data_smithi070_191495_object32562:head,463'2020,11:bb2f6575:::benchmark_data_smithi070_191495_object44411:head,464'2778,11:bb2fb009:::benchmark_data_smithi070_191495_object54017:head,466'3390,11:bb3025df:::benchmark_data_smithi070_191495_object17844:head,460'1144,11:bb3090ac:::benchmark_data_smithi070_191495_object77048:head,471'4863,11:bb30eb15:::benchmark_data_smithi070_191495_object76994:head,471'4856,11:bb319080:::benchmark_data_smithi070_191495_object92880:head,477'5898,11:bb323e93:::benchmark_data_smithi070_191495_object54649:head,466'3434,11:bb333d41:::benchmark_data_smithi070_191495_object100393:head,479'6375,11:bb33721b:::benchmark_data_smithi070_191495_object85798:head,474'5430,11:bb34492e:::benchmark_data_smithi070_191495_object38950:head,464'2432,11:bb34e4e5:::benchmark_data_smithi070_191495_object24169:head,462'1521,11:bb34e56b:::benchmark_data_smithi070_191495_object103614:head,480'6577,11:bb354175:::benchmark_data_smithi070_191495_object63572:head,467'3978,11:bb35448c:::benchmark_data_smithi070_191495_object129671:head,491'8226,11:bb354af1:::benchmark_data_smithi070_191495_object92066:head,476'5832,11:bb355523:::benchmark_data_smithi070_191495_object81338:head,473'5145,11:bb359c69:::benchmark_data_smithi070_191495_object116296:head,486'7372,11:bb3717aa:::benchmark_data_smithi070_191495_object59385:head,466'3722,11:bb379cba:::benchmark_data_smithi070_191495_object125300:head,490'7950,11:bb37a4dd:::benchmark_data_smithi070_191495_object50872:head,465'3193,11:bb37e3d6:::benchmark_data_smithi070_191495_object66326:head,467'4148,11:bb383603:::benchmark_data_smithi070_191495_object60188:head,467'3764,11:bb38509e:::benchmark_data_smithi070_191495_object112245:head,485'7120,11:bb387f06:::benchmark_data_smithi070_191495_object114307:head,486'7249,11:bb389634:::benchmark_data_smithi070_191495_object119917:head,488'7607,11:bb38d6c1:::benchmark_data_smithi070_191495_object92413:head,476'5862,11:bb390dd6:::benchmark_data_smithi070_191495_object46536:head,464'2901,11:bb392a89:::benchmark_data_smithi070_191495_object118246:head,487'7499,11:bb3963e5:::benchmark_data_smithi070_191495_object83665:head,474'5284,11:bb397b66:::benchmark_data_smithi070_191495_object126649:head,491'8033,11:bb3a25d1:::benchmark_data_smithi070_191495_object103870:head,480'6600,11:bb3a8cde:::benchmark_data_smithi070_191495_object71951:head,468'4491,11:bb3b2ce3:::benchmark_data_smithi070_191495_object82448:head,474'5219,11:bb3d33b1:::benchmark_data_smithi070_191495_object79162:head,472'4998,11:bb3d8d0c:::benchmark_data_smithi070_191495_object86645:head,474'5488,11:bb3e1e37:::benchmark_data_smithi070_191495_object28427:head,463'1774,11:bb3e52e4:::benchmark_data_smithi070_191495_object90214:head,475'5712,11:bb3f448b:::benchmark_data_smithi070_191495_object79855:head,472'5051,11:bb3f5b0d:::benchmark_data_smithi070_191495_object98130:head,479'6227,11:bb3f6cda:::benchmark_data_smithi070_191495_object39330:head,464'2455,11:bb3fc779:::benchmark_data_smithi070_191495_object61594:head,467'3865,11:bb406da1:::benchmark_data_smithi070_191495_object60412:head,467'3780,11:bb412413:::benchmark_data_smithi070_191495_object100832:head,479'6402,11:bb4165f7:::benchmark_data_smithi070_191495_object108633:head,483'6886,11:bb419a0b:::benchmark_data_smithi070_191495_object42562:head,464'2663,11:bb4229a8:::benchmark_data_smithi070_191495_object63298:head,467'3960,11:bb42c2ee:::benchmark_data_smithi070_191495_object43062:head,464'2692,11:bb42ff6f:::benchmark_data_smithi070_191495_object92751:head,476'5887,11:bb444de2:::benchmark_data_smithi070_191495_object50525:head,465'3172,11:bb4461ec:::benchmark_data_smithi070_191495_object52551:head,466'3299,11:bb447e31:::benchmark_data_smithi070_191495_object62360:head,467'3912,11:bb44c978:::benchmark_data_smithi070_191495_object36877:head,464'2308,11:bb452f65:::benchmark_data_smithi070_191495_object27767:head,463'1738,11:bb45602a:::benchmark_data_smithi070_191495_object89464:head,474'5664,11:bb45c946:::benchmark_data_smithi070_191495_object109740:head,483'6960,11:bb4669b3:::benchmark_data_smithi070_191495_object83208:head,474'5265,11:bb47fac5:::benchmark_data_smithi070_191495_object40114:head,464'2499,11:bb486f9c:::benchmark_data_smithi070_191495_object106728:head,481'6765,11:bb488930:::benchmark_data_smithi070_191495_object84756:head,474'5347,11:bb48a7e0:::benchmark_data_smithi070_191495_object87762:head,474'5565,11:bb491712:::benchmark_data_smithi070_191495_object36040:head,464'2241,11:bb4a1ccc:::benchmark_data_smithi070_191495_object28718:head,463'1789,11:bb4a38c3:::benchmark_data_smithi070_191495_object24995:head,462'1577,11:bb4b2e56:::benchmark_data_smithi070_191495_object129703:head,491'8228,11:bb4bce3a:::benchmark_data_smithi070_191495_object77165:head,471'4874,11:bb4bfe22:::benchmark_data_smithi070_191495_object90240:head,475'5713,11:bb4c3b3b:::benchmark_data_smithi070_191495_object80757:head,473'5108,11:bb4c7757:::benchmark_data_smithi070_191495_object77053:head,471'4864,11:bb4d826d:::benchmark_data_smithi070_191495_object116066:head,486'7354,11:bb4d93a0:::benchmark_data_smithi070_191495_object124212:head,489'7887,11:bb4d987a:::benchmark_data_smithi070_191495_object39808:head,464'2485,11:bb4da157:::benchmark_data_smithi070_191495_object96781:head,479'6147,11:bb4ed436:::benchmark_data_smithi070_191495_object106186:head,481'6730,11:bb507f30:::benchmark_data_smithi070_191495_object111878:head,484'7099,11:bb519921:::benchmark_data_smithi070_191495_object32831:head,463'2046,11:bb51f4f5:::benchmark_data_smithi070_191495_object91431:head,476'5781,11:bb53930c:::benchmark_data_smithi070_191495_object35681:head,464'2221,11:bb551111:::benchmark_data_smithi070_191495_object75151:head,470'4709,11:bb581218:::benchmark_data_smithi070_191495_object57323:head,466'3608,11:bb585ec4:::benchmark_data_smithi070_191495_object81458:head,473'5152,11:bb588c10:::benchmark_data_smithi070_191495_object87409:head,474'5540,11:bb599a6e:::benchmark_data_smithi070_191495_object78849:head,472'4980,11:bb59e22e:::benchmark_data_smithi070_191495_object30140:head,463'1882,11:bb5a04ad:::benchmark_data_smithi070_191495_object29546:head,463'1840,11:bb5a741a:::benchmark_data_smithi070_191495_object33613:head,464'2096,11:bb5a8a40:::benchmark_data_smithi070_191495_object31945:head,463'1983,11:bb5a9523:::benchmark_data_smithi070_191495_object88201:head,474'5590,11:bb5c4015:::benchmark_data_smithi070_191495_object46461:head,464'2896,11:bb5c57e9:::benchmark_data_smithi070_191495_object106127:head,481'6725,11:bb5d905e:::benchmark_data_smithi070_191495_object18955:head,460'1216,11:bb5eef98:::benchmark_data_smithi070_191495_object26910:head,463'1685,11:bb5f38ac:::benchmark_data_smithi070_191495_object87510:head,474'5548,11:bb5f7216:::benchmark_data_smithi070_191495_object128986:head,491'8183,11:bb5f93ca:::benchmark_data_smithi070_191495_object54520:head,466'3425,11:bb6089d0:::benchmark_data_smithi070_191495_object114542:head,486'7264,11:bb60bd40:::benchmark_data_smithi070_191495_object45108:head,464'2812,11:bb611e31:::benchmark_data_smithi070_191495_object113058:head,485'7172,11:bb61e9a6:::benchmark_data_smithi070_191495_object69814:head,468'4352,11:bb61edab:::benchmark_data_smithi070_191495_object124168:head,489'7883,11:bb61f6bd:::benchmark_data_smithi070_191495_object46042:head,464'2877,11:bb62c999:::benchmark_data_smithi070_191495_object70277:head,468'4388,11:bb62f971:::benchmark_data_smithi070_191495_object75154:head,470'4710,11:bb63dd61:::benchmark_data_smithi070_191495_object24192:head,462'1522,11:bb63fbc8:::benchmark_data_smithi070_191495_object117915:head,486'7469,11:bb645ac2:::benchmark_data_smithi070_191495_object107899:head,482'6835,11:bb657052:::benchmark_data_smithi070_191495_object37946:head,464'2372,11:bb65b866:::benchmark_data_smithi070_191495_object22766:head,461'1435,11:bb6717d0:::benchmark_data_smithi070_191495_object122100:head,488'7741,11:bb674f00:::benchmark_data_smithi070_191495_object120607:head,488'7642,11:bb67b8b3:::benchmark_data_smithi070_191495_object34882:head,464'2175,11:bb684ebd:::benchmark_data_smithi070_191495_object54260:head,466'3408,11:bb691d40:::benchmark_data_smithi070_191495_object73608:head,469'4605,11:bb69323f:::benchmark_data_smithi070_191495_object44239:head,464'2761,11:bb69759f:::benchmark_data_smithi070_191495_object103625:head,480'6581,11:bb69ea0f:::benchmark_data_smithi070_191495_object68516:head,468'4270,11:bb6a2fdc:::benchmark_data_smithi070_191495_object24581:head,462'1552,11:bb6aa947:::benchmark_data_smithi070_191495_object49141:head,465'3089,11:bb6ae11c:::benchmark_data_smithi070_191495_object104781:head,480'6652,11:bb6c909c:::benchmark_data_smithi070_191495_object89872:head,475'5692)", 2024-09-16T09:44:44.721 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:44:26.643519+0000", 2024-09-16T09:44:44.721 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 17.988513861000001, 2024-09-16T09:44:44.722 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.014537559, 2024-09-16T09:44:44.722 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:44:44.722 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:44:44.722 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T09:44:44.722 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:44:44.722 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.722 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:44:44.722 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.643519+0000", 2024-09-16T09:44:44.722 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:44:44.722 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.722 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.722 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:44:44.722 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.643519+0000", 2024-09-16T09:44:44.722 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:44:44.723 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.723 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.723 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:44:44.723 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.643520+0000", 2024-09-16T09:44:44.723 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.0800000000000004e-07 2024-09-16T09:44:44.723 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.723 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.723 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:44:44.723 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.643705+0000", 2024-09-16T09:44:44.723 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00018529399999999999 2024-09-16T09:44:44.723 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.723 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.723 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:44:44.723 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.643990+0000", 2024-09-16T09:44:44.724 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000284877 2024-09-16T09:44:44.724 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.724 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.724 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:44:44.724 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.644549+0000", 2024-09-16T09:44:44.724 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00055856199999999997 2024-09-16T09:44:44.724 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.724 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.724 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:44:44.724 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.654110+0000", 2024-09-16T09:44:44.724 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0095616539999999993 2024-09-16T09:44:44.724 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.724 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.724 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:44:44.725 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.655090+0000", 2024-09-16T09:44:44.725 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000979651 2024-09-16T09:44:44.725 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.725 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.725 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:44:44.725 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.658057+0000", 2024-09-16T09:44:44.725 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0029669129999999998 2024-09-16T09:44:44.725 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:44:44.725 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:44:44.725 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:44:44.725 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.725 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.725 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "pg_backfill(progress 11.d e 519/519 lb 11:bb6c909c:::benchmark_data_smithi070_191495_object89872:head)", 2024-09-16T09:44:44.725 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:44:26.645070+0000", 2024-09-16T09:44:44.726 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 17.986963331999998, 2024-09-16T09:44:44.726 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.013089297, 2024-09-16T09:44:44.726 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:44:44.726 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:44:44.726 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T09:44:44.726 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:44:44.726 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.726 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:44:44.726 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.645070+0000", 2024-09-16T09:44:44.726 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:44:44.727 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.727 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.727 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:44:44.727 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.645070+0000", 2024-09-16T09:44:44.727 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:44:44.727 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.727 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.727 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:44:44.727 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.645070+0000", 2024-09-16T09:44:44.727 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.0800000000000005e-07 2024-09-16T09:44:44.727 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.728 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.728 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:44:44.728 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.645080+0000", 2024-09-16T09:44:44.728 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.8109999999999997e-06 2024-09-16T09:44:44.728 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.728 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.728 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:44:44.728 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.645090+0000", 2024-09-16T09:44:44.728 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 9.6749999999999997e-06 2024-09-16T09:44:44.728 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.729 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.729 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:44:44.729 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.645094+0000", 2024-09-16T09:44:44.729 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.6709999999999998e-06 2024-09-16T09:44:44.729 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.729 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.729 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:44:44.729 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.658096+0000", 2024-09-16T09:44:44.729 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.013001798 2024-09-16T09:44:44.729 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.729 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.729 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:44:44.729 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.658112+0000", 2024-09-16T09:44:44.729 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.5809000000000001e-05 2024-09-16T09:44:44.730 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.730 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.730 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:44:44.730 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.658159+0000", 2024-09-16T09:44:44.730 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.7024999999999998e-05 2024-09-16T09:44:44.730 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:44:44.730 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:44:44.730 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:44:44.730 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.730 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.730 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "backfill_remove(11.4 e519 11:2b53f2d0:::benchmark_data_smithi070_191495_object47954:head,465'2999,11:2b53ff26:::benchmark_data_smithi070_191495_object71698:head,468'4523,11:2b548daa:::benchmark_data_smithi070_191495_object20187:head,460'1297,11:2b54f651:::benchmark_data_smithi070_191495_object62141:head,467'3897,11:2b54f8b1:::benchmark_data_smithi070_191495_object103422:head,480'6491,11:2b55500b:::benchmark_data_smithi070_191495_object20654:head,460'1330,11:2b555ade:::benchmark_data_smithi070_191495_object23259:head,462'1501,11:2b5586df:::benchmark_data_smithi070_191495_object76682:head,471'4822,11:2b561e6b:::benchmark_data_smithi070_191495_object72603:head,468'4583,11:2b56d8ed:::benchmark_data_smithi070_191495_object46144:head,464'2902,11:2b56feab:::benchmark_data_smithi070_191495_object109963:head,483'6894,11:2b571e8c:::benchmark_data_smithi070_191495_object113616:head,486'7114,11:2b577047:::benchmark_data_smithi070_191495_object109329:head,483'6858,11:2b5a52ec:::benchmark_data_smithi070_191495_object128209:head,491'8024,11:2b5a5558:::benchmark_data_smithi070_191495_object106460:head,481'6662,11:2b5a8c48:::benchmark_data_smithi070_191495_object58047:head,466'3616,11:2b5b0a5c:::benchmark_data_smithi070_191495_object44242:head,464'2782,11:2b5b21c7:::benchmark_data_smithi070_191495_object80412:head,472'5042,11:2b5bd403:::benchmark_data_smithi070_191495_object113841:head,486'7126,11:2b5cb212:::benchmark_data_smithi070_191495_object67436:head,468'4245,11:2b5e3d68:::benchmark_data_smithi070_191495_object83954:head,474'5281,11:2b5ec727:::benchmark_data_smithi070_191495_object64751:head,467'4082,11:2b5f4cac:::benchmark_data_smithi070_191495_object103982:head,480'6522,11:2b600ada:::benchmark_data_smithi070_191495_object54648:head,466'3395,11:2b60d6f4:::benchmark_data_smithi070_191495_object60558:head,467'3796,11:2b621f6e:::benchmark_data_smithi070_191495_object69895:head,468'4407,11:2b62c4d2:::benchmark_data_smithi070_191495_object124603:head,490'7791,11:2b6355bf:::benchmark_data_smithi070_191495_object106980:head,481'6693,11:2b6355c7:::benchmark_data_smithi070_191495_object47559:head,464'2976,11:2b6358ee:::benchmark_data_smithi070_191495_object40682:head,464'2591,11:2b647f1d:::benchmark_data_smithi070_191495_object53664:head,466'3332,11:2b65e657:::benchmark_data_smithi070_191495_object55667:head,466'3474,11:2b65f4d5:::benchmark_data_smithi070_191495_object117644:head,486'7354,11:2b664410:::benchmark_data_smithi070_191495_object128821:head,491'8066,11:2b66f3a6:::benchmark_data_smithi070_191495_object21044:head,461'1357,11:2b674864:::benchmark_data_smithi070_191495_object96247:head,478'6066,11:2b67a822:::benchmark_data_smithi070_191495_object87174:head,474'5480,11:2b681540:::benchmark_data_smithi070_191495_object60954:head,467'3822,11:2b6a8efc:::benchmark_data_smithi070_191495_object33987:head,464'2151,11:2b6a8fea:::benchmark_data_smithi070_191495_object99188:head,479'6238,11:2b6ab7dd:::benchmark_data_smithi070_191495_object93653:head,477'5899,11:2b6af254:::benchmark_data_smithi070_191495_object76228:head,471'4796,11:2b6cd4bd:::benchmark_data_smithi070_191495_object30893:head,463'1967,11:2b6cf8f0:::benchmark_data_smithi070_191495_object90738:head,476'5690,11:2b6d3c7d:::benchmark_data_smithi070_191495_object60457:head,467'3789,11:2b6db276:::benchmark_data_smithi070_191495_object74006:head,469'4658,11:2b6ec7c5:::benchmark_data_smithi070_191495_object111179:head,484'6967,11:2b6ed80f:::benchmark_data_smithi070_191495_object55141:head,466'3430,11:2b6f7402:::benchmark_data_smithi070_191495_object60595:head,467'3798,11:2b6f9cbc:::benchmark_data_smithi070_191495_object70984:head,468'4474,11:2b705a19:::benchmark_data_smithi070_191495_object100758:head,479'6332,11:2b706905:::benchmark_data_smithi070_191495_object117765:head,486'7361,11:2b709964:::benchmark_data_smithi070_191495_object95384:head,478'6016,11:2b71e321:::benchmark_data_smithi070_191495_object111948:head,484'7033,11:2b737e90:::benchmark_data_smithi070_191495_object68218:head,468'4302,11:2b73ae04:::benchmark_data_smithi070_191495_object69997:head,468'4415,11:2b74620a:::benchmark_data_smithi070_191495_object77568:head,471'4864,11:2b747e76:::benchmark_data_smithi070_191495_object57356:head,466'3579,11:2b75b14e:::benchmark_data_smithi070_191495_object49926:head,465'3104,11:2b76990c:::benchmark_data_smithi070_191495_object66650:head,468'4200,11:2b76bba1:::benchmark_data_smithi070_191495_object46286:head,464'2909,11:2b76bbdb:::benchmark_data_smithi070_191495_object18813:head,460'1213,11:2b775061:::benchmark_data_smithi070_191495_object92176:head,476'5784,11:2b775ecb:::benchmark_data_smithi070_191495_object107219:head,481'6714,11:2b780478:::benchmark_data_smithi070_191495_object127852:head,491'8008,11:2b7877b2:::benchmark_data_smithi070_191495_object25440:head,463'1661,11:2b78a413:::benchmark_data_smithi070_191495_object33304:head,463'2112,11:2b791389:::benchmark_data_smithi070_191495_object127759:head,491'7999,11:2b79348d:::benchmark_data_smithi070_191495_object130356:head,492'8154,11:2b7b3b62:::benchmark_data_smithi070_191495_object62764:head,467'3949,11:2b7bbea2:::benchmark_data_smithi070_191495_object57701:head,466'3594,11:2b7cb5ad:::benchmark_data_smithi070_191495_object39405:head,464'2511,11:2b7d869b:::benchmark_data_smithi070_191495_object89988:head,475'5655,11:2b7da5dd:::benchmark_data_smithi070_191495_object78120:head,472'4890,11:2b7e4218:::benchmark_data_smithi070_191495_object34757:head,464'2201,11:2b7f14d6:::benchmark_data_smithi070_191495_object105508:head,481'6614,11:2b7fa0f1:::benchmark_data_smithi070_191495_object74242:head,470'4674,11:2b80f43b:::benchmark_data_smithi070_191495_object62080:head,467'3895,11:2b810bf6:::benchmark_data_smithi070_191495_object57678:head,466'3592,11:2b811c38:::benchmark_data_smithi070_191495_object76101:head,471'4784,11:2b829578:::benchmark_data_smithi070_191495_object26372:head,463'1706,11:2b845e82:::benchmark_data_smithi070_191495_object78703:head,472'4932,11:2b848226:::benchmark_data_smithi070_191495_object107134:head,481'6706,11:2b84d6ef:::benchmark_data_smithi070_191495_object67601:head,468'4258,11:2b84f53c:::benchmark_data_smithi070_191495_object20088:head,460'1289,11:2b86dd10:::benchmark_data_smithi070_191495_object102591:head,480'6445,11:2b8757b8:::benchmark_data_smithi070_191495_object40634:head,464'2586,11:2b884cc8:::benchmark_data_smithi070_191495_object68713:head,468'4332,11:2b889179:::benchmark_data_smithi070_191495_object18407:head,460'1186,11:2b8ade72:::benchmark_data_smithi070_191495_object51224:head,465'3175,11:2b8b156e:::benchmark_data_smithi070_191495_object16071:head,460'1044,11:2b8b28ed:::benchmark_data_smithi070_191495_object96038:head,478'6054,11:2b8d19c7:::benchmark_data_smithi070_191495_object16363:head,460'1057,11:2b8dce40:::benchmark_data_smithi070_191495_object107359:head,481'6725,11:2b8e2c21:::benchmark_data_smithi070_191495_object34800:head,464'2206,11:2b8f0233:::benchmark_data_smithi070_191495_object21017:head,461'1354,11:2b8f4cb9:::benchmark_data_smithi070_191495_object82451:head,474'5192,11:2b8fb018:::benchmark_data_smithi070_191495_object30634:head,463'1949,11:2b90a3dc:::benchmark_data_smithi070_191495_object108553:head,483'6812,11:2b917942:::benchmark_data_smithi070_191495_object49571:head,465'3086,11:2b91e77c:::benchmark_data_smithi070_191495_object94671:head,478'5969,11:2b92200c:::benchmark_data_smithi070_191495_object71353:head,468'4502,11:2b922d8f:::benchmark_data_smithi070_191495_object91461:head,476'5733,11:2b931e73:::benchmark_data_smithi070_191495_object27474:head,463'1763,11:2b93a182:::benchmark_data_smithi070_191495_object32193:head,463'2044,11:2b942f6f:::benchmark_data_smithi070_191495_object80824:head,473'5080,11:2b943997:::benchmark_data_smithi070_191495_object55386:head,466'3453,11:2b94d081:::benchmark_data_smithi070_191495_object64497:head,467'4061,11:2b953ffb:::benchmark_data_smithi070_191495_object25212:head,463'1644,11:2b95f7b3:::benchmark_data_smithi070_191495_object45042:head,464'2836,11:2b96dca9:::benchmark_data_smithi070_191495_object124866:head,490'7807,11:2b9719e0:::benchmark_data_smithi070_191495_object99604:head,479'6263,11:2b9765a4:::benchmark_data_smithi070_191495_object26102:head,463'1699,11:2b97c495:::benchmark_data_smithi070_191495_object76091:head,471'4783,11:2b984607:::benchmark_data_smithi070_191495_object118698:head,487'7418,11:2b987a0e:::benchmark_data_smithi070_191495_object22258:head,461'1430,11:2b98fe12:::benchmark_data_smithi070_191495_object109235:head,483'6851,11:2b9a5045:::benchmark_data_smithi070_191495_object50340:head,465'3127,11:2b9a9474:::benchmark_data_smithi070_191495_object42830:head,464'2706,11:2b9aeadf:::benchmark_data_smithi070_191495_object80942:head,473'5091,11:2b9aef44:::benchmark_data_smithi070_191495_object22382:head,461'1440,11:2b9bc7b9:::benchmark_data_smithi070_191495_object112922:head,485'7079,11:2b9c46d4:::benchmark_data_smithi070_191495_object75170:head,470'4732,11:2b9d1d18:::benchmark_data_smithi070_191495_object65923:head,467'4158,11:2b9d3203:::benchmark_data_smithi070_191495_object58104:head,466'3619,11:2b9d6cbd:::benchmark_data_smithi070_191495_object76899:head,471'4830,11:2b9d94d7:::benchmark_data_smithi070_191495_object58745:head,466'3665,11:2b9ed7b1:::benchmark_data_smithi070_191495_object43862:head,464'2758,11:2b9f2d41:::benchmark_data_smithi070_191495_object95791:head,478'6041,11:2b9fc694:::benchmark_data_smithi070_191495_object83668:head,474'5264,11:2ba06972:::benchmark_data_smithi070_191495_object92113:head,476'5777,11:2ba07a33:::benchmark_data_smithi070_191495_object122674:head,488'7670,11:2ba1592a:::benchmark_data_smithi070_191495_object30266:head,463'1927,11:2ba1a902:::benchmark_data_smithi070_191495_object93440:head,477'5875,11:2ba299d5:::benchmark_data_smithi070_191495_object92053:head,476'5770,11:2ba2b421:::benchmark_data_smithi070_191495_object31018:head,463'1980,11:2ba38110:::benchmark_data_smithi070_191495_object103589:head,480'6498,11:2ba42d6a:::benchmark_data_smithi070_191495_object39653:head,464'2529,11:2ba50d66:::benchmark_data_smithi070_191495_object76119:head,471'4785,11:2ba5282b:::benchmark_data_smithi070_191495_object53683:head,466'3334,11:2ba5d2ec:::benchmark_data_smithi070_191495_object97432:head,479'6139,11:2ba6556e:::benchmark_data_smithi070_191495_object99278:head,479'6241,11:2ba6db78:::benchmark_data_smithi070_191495_object95998:head,478'6048,11:2ba7193c:::benchmark_data_smithi070_191495_object115452:head,486'7218,11:2ba7270d:::benchmark_data_smithi070_191495_object67184:head,468'4231,11:2ba936ee:::benchmark_data_smithi070_191495_object51547:head,465'3204,11:2ba96078:::benchmark_data_smithi070_191495_object117500:head,486'7350,11:2baa94df:::benchmark_data_smithi070_191495_object31181:head,463'1995,11:2bab2f05:::benchmark_data_smithi070_191495_object65219:head,467'4111,11:2bab7932:::benchmark_data_smithi070_191495_object49500:head,465'3083,11:2baba4b3:::benchmark_data_smithi070_191495_object108772:head,483'6829,11:2babcaa1:::benchmark_data_smithi070_191495_object20326:head,460'1308,11:2bac7635:::benchmark_data_smithi070_191495_object76561:head,471'4819,11:2badb847:::benchmark_data_smithi070_191495_object22884:head,462'1467,11:2bae3551:::benchmark_data_smithi070_191495_object21052:head,461'1358,11:2baf79b6:::benchmark_data_smithi070_191495_object49227:head,465'3064,11:2bafc7ab:::benchmark_data_smithi070_191495_object55789:head,466'3482,11:2bb063ff:::benchmark_data_smithi070_191495_object113871:head,486'7128,11:2bb420f1:::benchmark_data_smithi070_191495_object92634:head,476'5810,11:2bb6ed8f:::benchmark_data_smithi070_191495_object120609:head,488'7553,11:2bb980b7:::benchmark_data_smithi070_191495_object22349:head,461'1437,11:2bb9d34d:::benchmark_data_smithi070_191495_object100542:head,479'6322,11:2bbc3234:::benchmark_data_smithi070_191495_object112186:head,485'7041,11:2bbcd592:::benchmark_data_smithi070_191495_object62397:head,467'3914,11:2bbd765e:::benchmark_data_smithi070_191495_object23153:head,462'1490,11:2bbdad4c:::benchmark_data_smithi070_191495_object118479:head,487'7401,11:2bbe9d2d:::benchmark_data_smithi070_191495_object108027:head,482'6779,11:2bbef40f:::benchmark_data_smithi070_191495_object91947:head,476'5764,11:2bbefce3:::benchmark_data_smithi070_191495_object70753:head,468'4457,11:2bbf39d8:::benchmark_data_smithi070_191495_object59422:head,466'3724,11:2bc11bf6:::benchmark_data_smithi070_191495_object104715:head,480'6566,11:2bc1212f:::benchmark_data_smithi070_191495_object57175:head,466'3569,11:2bc15b90:::benchmark_data_smithi070_191495_object20543:head,460'1321,11:2bc17b34:::benchmark_data_smithi070_191495_object117246:head,486'7334,11:2bc1804b:::benchmark_data_smithi070_191495_object119138:head,487'7445,11:2bc25def:::benchmark_data_smithi070_191495_object111334:head,484'6980,11:2bc2b3c3:::benchmark_data_smithi070_191495_object33906:head,464'2144,11:2bc355aa:::benchmark_data_smithi070_191495_object19387:head,460'1252,11:2bc38609:::benchmark_data_smithi070_191495_object70013:head,468'4419,11:2bc480b6:::benchmark_data_smithi070_191495_object122241:head,488'7642,11:2bc58fc7:::benchmark_data_smithi070_191495_object105459:head,481'6613,11:2bc61ba7:::benchmark_data_smithi070_191495_object39318:head,464'2502,11:2bc6edfa:::benchmark_data_smithi070_191495_object114704:head,486'7165,11:2bc8647f:::benchmark_data_smithi070_191495_object117164:head,486'7329,11:2bc8cfab:::benchmark_data_smithi070_191495_object123521:head,489'7716,11:2bca0931:::benchmark_data_smithi070_191495_object75634:head,471'4754,11:2bca75fa:::benchmark_data_smithi070_191495_object24749:head,462'1614,11:2bcaf169:::benchmark_data_smithi070_191495_object42763:head,464'2700,11:2bcbc9a2:::benchmark_data_smithi070_191495_object100845:head,479'6335,11:2bcbfebe:::benchmark_data_smithi070_191495_object43680:head,464'2747,11:2bcc165c:::benchmark_data_smithi070_191495_object102732:head,480'6450,11:2bcd3b59:::benchmark_data_smithi070_191495_object69946:head,468'4411,11:2bce4af8:::benchmark_data_smithi070_191495_object84790:head,474'5339,11:2bce6e02:::benchmark_data_smithi070_191495_object49045:head,465'3054,11:2bcea636:::benchmark_data_smithi070_191495_object38190:head,464'2429,11:2bcea6e0:::benchmark_data_smithi070_191495_object16453:head,460'1062,11:2bcf1ab2:::benchmark_data_smithi070_191495_object55997:head,466'3494,11:2bcfa92a:::benchmark_data_smithi070_191495_object96679:head,479'6093,11:2bd0070d:::benchmark_data_smithi070_191495_object115109:head,486'7191,11:2bd060b0:::benchmark_data_smithi070_191495_object79425:head,472'4978,11:2bd0a25c:::benchmark_data_smithi070_191495_object51223:head,465'3174,11:2bd1a6f4:::benchmark_data_smithi070_191495_object110507:head,483'6922,11:2bd2abc1:::benchmark_data_smithi070_191495_object127127:head,491'7969,11:2bd36952:::benchmark_data_smithi070_191495_object121774:head,488'7623,11:2bd51b23:::benchmark_data_smithi070_191495_object19057:head,460'1228,11:2bd5278f:::benchmark_data_smithi070_191495_object104126:head,480'6531,11:2bd54963:::benchmark_data_smithi070_191495_object48004:head,465'3003,11:2bd5f5e0:::benchmark_data_smithi070_191495_object51307:head,465'3180,11:2bd93fcf:::benchmark_data_smithi070_191495_object85617:head,474'5389,11:2bd962ed:::benchmark_data_smithi070_191495_object108972:head,483'6843,11:2bd9d8c0:::benchmark_data_smithi070_191495_object104420:head,480'6552,11:2bda362d:::benchmark_data_smithi070_191495_object88600:head,474'5553,11:2bda6942:::benchmark_data_smithi070_191495_object128314:head,491'8030,11:2bdb7634:::benchmark_data_smithi070_191495_object83395:head,474'5244,11:2bdc455a:::benchmark_data_smithi070_191495_object80447:head,472'5048,11:2bdc8299:::benchmark_data_smithi070_191495_object124902:head,490'7811,11:2bdc8677:::benchmark_data_smithi070_191495_object49891:head,465'3101,11:2bdcd16d:::benchmark_data_smithi070_191495_object94957:head,478'5989,11:2bdcdafe:::benchmark_data_smithi070_191495_object23181:head,462'1493,11:2bdcec51:::benchmark_data_smithi070_191495_object84768:head,474'5335,11:2bdd64e8:::benchmark_data_smithi070_191495_object64209:head,467'4044,11:2bdf21ea:::benchmark_data_smithi070_191495_object56454:head,466'3517,11:2bdf60f4:::benchmark_data_smithi070_191495_object129087:head,491'8076,11:2bdfcc4d:::benchmark_data_smithi070_191495_object117849:head,486'7368,11:2be01cb6:::benchmark_data_smithi070_191495_object104478:head,480'6553,11:2be09c22:::benchmark_data_smithi070_191495_object84586:head,474'5320,11:2be0a35b:::benchmark_data_smithi070_191495_object101388:head,479'6366,11:2be158f6:::benchmark_data_smithi070_191495_object75406:head,470'4742,11:2be2d9f1:::benchmark_data_smithi070_191495_object91854:head,476'5756,11:2be48e99:::benchmark_data_smithi070_191495_object16738:head,460'1080,11:2be51cae:::benchmark_data_smithi070_191495_object86093:head,474'5414,11:2be58909:::benchmark_data_smithi070_191495_object59674:head,466'3737,11:2be59d2f:::benchmark_data_smithi070_191495_object125699:head,490'7869,11:2be62bd2:::benchmark_data_smithi070_191495_object42003:head,464'2663,11:2be76400:::benchmark_data_smithi070_191495_object37446:head,464'2374,11:2be8e668:::benchmark_data_smithi070_191495_object53692:head,466'3336,11:2be8fdd4:::benchmark_data_smithi070_191495_object50946:head,465'3155,11:2bea5660:::benchmark_data_smithi070_191495_object113571:head,486'7112,11:2beb5d86:::benchmark_data_smithi070_191495_object118669:head,487'7417,11:2beba455:::benchmark_data_smithi070_191495_object61446:head,467'3856,11:2bed9152:::benchmark_data_smithi070_191495_object33386:head,463'2117,11:2bee3c61:::benchmark_data_smithi070_191495_object28084:head,463'1792,11:2bef22a9:::benchmark_data_smithi070_191495_object42809:head,464'2702,11:2bef5ca4:::benchmark_data_smithi070_191495_object84556:head,474'5315,11:2bef7315:::benchmark_data_smithi070_191495_object127065:head,491'7962,11:2befdb81:::benchmark_data_smithi070_191495_object87884:head,474'5514,11:2bf011ac:::benchmark_data_smithi070_191495_object86382:head,474'5431,11:2bf016ee:::benchmark_data_smithi070_191495_object53723:head,466'3339,11:2bf02131:::benchmark_data_smithi070_191495_object105650:head,481'6623,11:2bf0755e:::benchmark_data_smithi070_191495_object112859:head,485'7076,11:2bf0858f:::benchmark_data_smithi070_191495_object76671:head,471'4821,11:2bf19f03:::benchmark_data_smithi070_191495_object87697:head,474'5501,11:2bf248d3:::benchmark_data_smithi070_191495_object30153:head,463'1918,11:2bf2bc1d:::benchmark_data_smithi070_191495_object51080:head,465'3163,11:2bf2fd0f:::benchmark_data_smithi070_191495_object109170:head,483'6849,11:2bf3ef66:::benchmark_data_smithi070_191495_object71162:head,468'4491,11:2bf40e04:::benchmark_data_smithi070_191495_object38638:head,464'2466,11:2bf40f38:::benchmark_data_smithi070_191495_object49750:head,465'3098,11:2bf454d9:::benchmark_data_smithi070_191495_object24726:head,462'1610,11:2bf4ee8e:::benchmark_data_smithi070_191495_object96057:head,478'6058,11:2bf54965:::benchmark_data_smithi070_191495_object62274:head,467'3905,11:2bf581bd:::benchmark_data_smithi070_191495_object87044:head,474'5474,11:2bf58fef:::benchmark_data_smithi070_191495_object119751:head,488'7482,11:2bf75291:::benchmark_data_smithi070_191495_object88263:head,474'5534,11:2bf7874c:::benchmark_data_smithi070_191495_object43640:head,464'2744,11:2bf7a37e:::benchmark_data_smithi070_191495_object39768:head,464'2534,11:2bf88437:::benchmark_data_smithi070_191495_object129418:head,491'8093,11:2bf8b338:::benchmark_data_smithi070_191495_object54659:head,466'3396,11:2bf99a56:::benchmark_data_smithi070_191495_object76022:head,471'4781,11:2bf9cb86:::benchmark_data_smithi070_191495_object30573:head,463'1947,11:2bfa81e6:::benchmark_data_smithi070_191495_object103002:head,480'6463,11:2bfa9c00:::benchmark_data_smithi070_191495_object116038:head,486'7257,11:2bfafbe7:::benchmark_data_smithi070_191495_object40287:head,464'2561,11:2bfb0cd8:::benchmark_data_smithi070_191495_object44667:head,464'2812,11:2bfda34d:::benchmark_data_smithi070_191495_object115822:head,486'7247,11:2bff0d9d:::benchmark_data_smithi070_191495_object121177:head,488'7591,11:2bff1261:::benchmark_data_smithi070_191495_object88811:head,474'5569,11:2c003e81:::benchmark_data_smithi070_191495_object121908:head,488'7630,11:2c00a2a1:::benchmark_data_smithi070_191495_object97926:head,479'6163,11:2c00a78f:::benchmark_data_smithi070_191495_object116391:head,486'7276,11:2c01c201:::benchmark_data_smithi070_191495_object40421:head,464'2566,11:2c01ff88:::benchmark_data_smithi070_191495_object90889:head,476'5700,11:2c020e40:::benchmark_data_smithi070_191495_object89065:head,474'5589,11:2c02ecae:::benchmark_data_smithi070_191495_object80201:head,472'5023,11:2c03272f:::benchmark_data_smithi070_191495_object91086:head,476'5714,11:2c034d3c:::benchmark_data_smithi070_191495_object108177:head,482'6786,11:2c0410e4:::benchmark_data_smithi070_191495_object79869:head,472'5003,11:2c04c17a:::benchmark_data_smithi070_191495_object96336:head,478'6072,11:2c04eefc:::benchmark_data_smithi070_191495_object58610:head,466'3653,11:2c051d8c:::benchmark_data_smithi070_191495_object66801:head,468'4215,11:2c07eedf:::benchmark_data_smithi070_191495_object34602:head,464'2188,11:2c07f60b:::benchmark_data_smithi070_191495_object118849:head,487'7428,11:2c07fd27:::benchmark_data_smithi070_191495_object77708:head,471'4875,11:2c081bfb:::benchmark_data_smithi070_191495_object36943:head,464'2341,11:2c09fff9:::benchmark_data_smithi070_191495_object16166:head,460'1047,11:2c0adfd3:::benchmark_data_smithi070_191495_object18050:head,460'1162,11:2c0b876f:::benchmark_data_smithi070_191495_object71712:head,468'4525,11:2c0b9083:::benchmark_data_smithi070_191495_object23103:head,462'1485,11:2c0b9783:::benchmark_data_smithi070_191495_object40342:head,464'2564,11:2c0e0cd4:::benchmark_data_smithi070_191495_object28703:head,463'1834,11:2c0e848a:::benchmark_data_smithi070_191495_object53889:head,466'3352,11:2c0f13e1:::benchmark_data_smithi070_191495_object86195:head,474'5422,11:2c101823:::benchmark_data_smithi070_191495_object111276:head,484'6975,11:2c112e15:::benchmark_data_smithi070_191495_object71870:head,468'4534,11:2c11862b:::benchmark_data_smithi070_191495_object81850:head,474'5153,11:2c11c3ae:::benchmark_data_smithi070_191495_object55245:head,466'3441,11:2c11f30f:::benchmark_data_smithi070_191495_object34780:head,464'2203,11:2c12d8f1:::benchmark_data_smithi070_191495_object110097:head,483'6902,11:2c133ba9:::benchmark_data_smithi070_191495_object21960:head,461'1407,11:2c137bbb:::benchmark_data_smithi070_191495_object49001:head,465'3051,11:2c1581bf:::benchmark_data_smithi070_191495_object117112:head,486'7325,11:2c1676ba:::benchmark_data_smithi070_191495_object121270:head,488'7595,11:2c17d482:::benchmark_data_smithi070_191495_object22030:head,461'1413,11:2c17e925:::benchmark_data_smithi070_191495_object25042:head,462'1633,11:2c1871b5:::benchmark_data_smithi070_191495_object30560:head,463'1945,11:2c19097f:::benchmark_data_smithi070_191495_object119604:head,488'7477,11:2c1926c4:::benchmark_data_smithi070_191495_object111618:head,484'7007,11:2c19c4b3:::benchmark_data_smithi070_191495_object65954:head,467'4160,11:2c1a8f22:::benchmark_data_smithi070_191495_object47883:head,464'2995,11:2c1b128f:::benchmark_data_smithi070_191495_object81436:head,473'5122,11:2c1c57d8:::benchmark_data_smithi070_191495_object25317:head,463'1650,11:2c1d08b9:::benchmark_data_smithi070_191495_object119112:head,487'7443,11:2c1d70f2:::benchmark_data_smithi070_191495_object113181:head,485'7088,11:2c1db47f:::benchmark_data_smithi070_191495_object109954:head,483'6892,11:2c1dceba:::benchmark_data_smithi070_191495_object89210:head,474'5598,11:2c1de2e9:::benchmark_data_smithi070_191495_object118374:head,487'7391,11:2c1dea18:::benchmark_data_smithi070_191495_object59042:head,466'3694,11:2c1ebfa7:::benchmark_data_smithi070_191495_object101242:head,479'6356,11:2c1f6f1c:::benchmark_data_smithi070_191495_object108724:head,483'6826,11:2c205cf0:::benchmark_data_smithi070_191495_object115105:head,486'7190,11:2c21206d:::benchmark_data_smithi070_191495_object89412:head,474'5615,11:2c217dcd:::benchmark_data_smithi070_191495_object114279:head,486'7143,11:2c21f0d0:::benchmark_data_smithi070_191495_object66247:head,467'4177,11:2c2296fc:::benchmark_data_smithi070_191495_object17546:head,460'1127,11:2c2481d9:::benchmark_data_smithi070_191495_object45351:head,464'2857,11:2c24c96f:::benchmark_data_smithi070_191495_object52387:head,466'3257,11:2c253358:::benchmark_data_smithi070_191495_object61445:head,467'3855,11:2c263644:::benchmark_data_smithi070_191495_object58542:head,466'3646,11:2c264d7a:::benchmark_data_smithi070_191495_object107329:head,481'6722,11:2c26cab8:::benchmark_data_smithi070_191495_object108308:head,482'6797,11:2c27fc49:::benchmark_data_smithi070_191495_object117306:head,486'7336,11:2c29d305:::benchmark_data_smithi070_191495_object69842:head,468'4404,11:2c29f4b3:::benchmark_data_smithi070_191495_object21405:head,461'1382,11:2c29f861:::benchmark_data_smithi070_191495_object82468:head,474'5194,11:2c2a1017:::benchmark_data_smithi070_191495_object82241:head,474'5175,11:2c2a6126:::benchmark_data_smithi070_191495_object104554:head,480'6557,11:2c2b5cdf:::benchmark_data_smithi070_191495_object85446:head,474'5381,11:2c2b6439:::benchmark_data_smithi070_191495_object127329:head,491'7974,11:2c2b7184:::benchmark_data_smithi070_191495_object67213:head,468'4233,11:2c2b81f4:::benchmark_data_smithi070_191495_object93832:head,477'5911,11:2c2baa68:::benchmark_data_smithi070_191495_object47286:head,464'2963,11:2c2cb0b3:::benchmark_data_smithi070_191495_object73188:head,469'4615,11:2c2d9850:::benchmark_data_smithi070_191495_object64882:head,467'4088,11:2c2ed81d:::benchmark_data_smithi070_191495_object105962:head,481'6640,11:2c2f78d5:::benchmark_data_smithi070_191495_object77614:head,471'4866,11:2c2f9661:::benchmark_data_smithi070_191495_object59058:head,466'3695,11:2c310d46:::benchmark_data_smithi070_191495_object100994:head,479'6343,11:2c314950:::benchmark_data_smithi070_191495_object86778:head,474'5458,11:2c31fe45:::benchmark_data_smithi070_191495_object46203:head,464'2904,11:2c32d717:::benchmark_data_smithi070_191495_object115577:head,486'7229,11:2c33dfd1:::benchmark_data_smithi070_191495_object21179:head,461'1364,11:2c34c35a:::benchmark_data_smithi070_191495_object43159:head,464'2726,11:2c363587:::benchmark_data_smithi070_191495_object96735:head,479'6099,11:2c3646df:::benchmark_data_smithi070_191495_object29483:head,463'1882,11:2c36842f:::benchmark_data_smithi070_191495_object73768:head,469'4647,11:2c36d8ae:::benchmark_data_smithi070_191495_object122498:head,488'7664,11:2c3727dd:::benchmark_data_smithi070_191495_object18777:head,460'1211,11:2c37409a:::benchmark_data_smithi070_191495_object29743:head,463'1898,11:2c374926:::benchmark_data_smithi070_191495_object39784:head,464'2535,11:2c379201:::benchmark_data_smithi070_191495_object88874:head,474'5572,11:2c38c05d:::benchmark_data_smithi070_191495_object60411:head,467'3787,11:2c393a5e:::benchmark_data_smithi070_191495_object45167:head,464'2846,11:2c3a79e8:::benchmark_data_smithi070_191495_object76336:head,471'4800,11:2c3f61af:::benchmark_data_smithi070_191495_object125138:head,490'7825,11:2c4037d0:::benchmark_data_smithi070_191495_object36019:head,464'2291,11:2c403fb9:::benchmark_data_smithi070_191495_object68105:head,468'4290,11:2c4062bb:::benchmark_data_smithi070_191495_object41652:head,464'2643,11:2c40915d:::benchmark_data_smithi070_191495_object92457:head,476'5800,11:2c40db9a:::benchmark_data_smithi070_191495_object107524:head,482'6737,11:2c412a40:::benchmark_data_smithi070_191495_object90761:head,476'5692,11:2c426834:::benchmark_data_smithi070_191495_object114759:head,486'7168,11:2c42b6c0:::benchmark_data_smithi070_191495_object84567:head,474'5317,11:2c434524:::benchmark_data_smithi070_191495_object94270:head,478'5936,11:2c43573a:::benchmark_data_smithi070_191495_object30274:head,463'1928,11:2c439a04:::benchmark_data_smithi070_191495_object102249:head,480'6428,11:2c44ccf9:::benchmark_data_smithi070_191495_object20428:head,460'1313,11:2c450404:::benchmark_data_smithi070_191495_object115476:head,486'7220,11:2c459fd9:::benchmark_data_smithi070_191495_object35357:head,464'2248,11:2c45aa05:::benchmark_data_smithi070_191495_object61961:head,467'3886,11:2c461410:::benchmark_data_smithi070_191495_object82517:head,474'5201,11:2c46418b:::benchmark_data_smithi070_191495_object16884:head,460'1091,11:2c46877a:::benchmark_data_smithi070_191495_object92177:head,476'5785,11:2c471616:::benchmark_data_smithi070_191495_object81784:head,473'5148,11:2c471d60:::benchmark_data_smithi070_191495_object88708:head,474'5562,11:2c47f29b:::benchmark_data_smithi070_191495_object121410:head,488'7605,11:2c488290:::benchmark_data_smithi070_191495_object82527:head,474'5202,11:2c4a2db1:::benchmark_data_smithi070_191495_object29675:head,463'1893,11:2c4af22d:::benchmark_data_smithi070_191495_object57521:head,466'3586,11:2c4beab7:::benchmark_data_smithi070_191495_object103047:head,480'6467,11:2c4d426b:::benchmark_data_smithi070_191495_object88758:head,474'5566,11:2c4d8c69:::benchmark_data_smithi070_191495_object111267:head,484'6973,11:2c4de630:::benchmark_data_smithi070_191495_object127782:head,491'8003,11:2c4edd19:::benchmark_data_smithi070_191495_object83980:head,474'5283,11:2c4f0c69:::benchmark_data_smithi070_191495_object69216:head,468'4366,11:2c4f5a07:::benchmark_data_smithi070_191495_object111124:head,484'6962,11:2c4f7269:::benchmark_data_smithi070_191495_object37866:head,464'2401,11:2c500625:::benchmark_data_smithi070_191495_object87429:head,474'5491,11:2c507590:::benchmark_data_smithi070_191495_object101867:head,480'6401,11:2c5117ec:::benchmark_data_smithi070_191495_object31590:head,463'2019,11:2c51671d:::benchmark_data_smithi070_191495_object53834:head,466'3349,11:2c516da0:::benchmark_data_smithi070_191495_object101720:head,480'6386,11:2c51a952:::benchmark_data_smithi070_191495_object103100:head,480'6468,11:2c542776:::benchmark_data_smithi070_191495_object45085:head,464'2843,11:2c54b952:::benchmark_data_smithi070_191495_object57328:head,466'3577,11:2c54fde7:::benchmark_data_smithi070_191495_object41087:head,464'2616,11:2c55320e:::benchmark_data_smithi070_191495_object84138:head,474'5290,11:2c557a89:::benchmark_data_smithi070_191495_object117097:head,486'7324,11:2c56333b:::benchmark_data_smithi070_191495_object68416:head,468'4315,11:2c56ae6d:::benchmark_data_smithi070_191495_object35465:head,464'2259,11:2c56c533:::benchmark_data_smithi070_191495_object68265:head,468'4304,11:2c574ddc:::benchmark_data_smithi070_191495_object60879:head,467'3814,11:2c57ed21:::benchmark_data_smithi070_191495_object98051:head,479'6170,11:2c582ed0:::benchmark_data_smithi070_191495_object128193:head,491'8023,11:2c583b8e:::benchmark_data_smithi070_191495_object105237:head,481'6601,11:2c5a31c7:::benchmark_data_smithi070_191495_object37807:head,464'2393,11:2c5addbf:::benchmark_data_smithi070_191495_object32423:head,463'2058,11:2c5b2d37:::benchmark_data_smithi070_191495_object109386:head,483'6861,11:2c5bcade:::benchmark_data_smithi070_191495_object63363:head,467'3993,11:2c5c7368:::benchmark_data_smithi070_191495_object93065:head,477'5845,11:2c5c7b35:::benchmark_data_smithi070_191495_object116790:head,486'7304,11:2c5c9937:::benchmark_data_smithi070_191495_object19635:head,460'1263,11:2c5da687:::benchmark_data_smithi070_191495_object36971:head,464'2343,11:2c5e48e5:::benchmark_data_smithi070_191495_object26731:head,463'1725,11:2c5e8d1d:::benchmark_data_smithi070_191495_object120091:head,488'7511,11:2c5ef2b0:::benchmark_data_smithi070_191495_object20124:head,460'1293,11:2c5f08ed:::benchmark_data_smithi070_191495_object39059:head,464'2487,11:2c5f22d0:::benchmark_data_smithi070_191495_object65187:head,467'4109,11:2c5fa09f:::benchmark_data_smithi070_191495_object92537:head,476'5806,11:2c5ff157:::benchmark_data_smithi070_191495_object74155:head,469'4667,11:2c617257:::benchmark_data_smithi070_191495_object68341:head,468'4310,11:2c617508:::benchmark_data_smithi070_191495_object91603:head,476'5747,11:2c61ad40:::benchmark_data_smithi070_191495_object35679:head,464'2271,11:2c61ae83:::benchmark_data_smithi070_191495_object44527:head,464'2801,11:2c6272be:::benchmark_data_smithi070_191495_object38578:head,464'2460,11:2c629b34:::benchmark_data_smithi070_191495_object122481:head,488'7663,11:2c62b250:::benchmark_data_smithi070_191495_object116589:head,486'7287,11:2c6357f6:::benchmark_data_smithi070_191495_object117408:head,486'7344,11:2c6608f2:::benchmark_data_smithi070_191495_object92355:head,476'5795,11:2c663164:::benchmark_data_smithi070_191495_object33003:head,463'2094,11:2c667a73:::benchmark_data_smithi070_191495_object85236:head,474'5363,11:2c671d42:::benchmark_data_smithi070_191495_object114926:head,486'7180,11:2c680b56:::benchmark_data_smithi070_191495_object59841:head,467'3746,11:2c680e65:::benchmark_data_smithi070_191495_object89486:head,474'5619,11:2c687f0e:::benchmark_data_smithi070_191495_object33953:head,464'2148,11:2c6a1b33:::benchmark_data_smithi070_191495_object59002:head,466'3689,11:2c6a4673:::benchmark_data_smithi070_191495_object117276:head,486'7335,11:2c6b159e:::benchmark_data_smithi070_191495_object19952:head,460'1282,11:2c6b4988:::benchmark_data_smithi070_191495_object71263:head,468'4496,11:2c6c37c1:::benchmark_data_smithi070_191495_object82386:head,474'5184,11:2c6c4d4e:::benchmark_data_smithi070_191495_object43959:head,464'2765,11:2c6d422a:::benchmark_data_smithi070_191495_object26771:head,463'1726,11:2c6dce02:::benchmark_data_smithi070_191495_object59237:head,466'3710,11:2c6edd87:::benchmark_data_smithi070_191495_object115595:head,486'7231,11:2c700a34:::benchmark_data_smithi070_191495_object111678:head,484'7012,11:2c70f6a5:::benchmark_data_smithi070_191495_object100405:head,479'6314,11:2c713a1e:::benchmark_data_smithi070_191495_object96033:head,478'6051,11:2c719640:::benchmark_data_smithi070_191495_object20445:head,460'1315,11:2c73c6b6:::benchmark_data_smithi070_191495_object19593:head,460'1259,11:2c7463c6:::benchmark_data_smithi070_191495_object52393:head,466'3259,11:2c74dadb:::benchmark_data_smithi070_191495_object21644:head,461'1393,11:2c75de2c:::benchmark_data_smithi070_191495_object32715:head,463'2071,11:2c763f92:::benchmark_data_smithi070_191495_object59012:head,466'3690,11:2c7649e6:::benchmark_data_smithi070_191495_object124340:head,489'7766,11:2c76a96d:::benchmark_data_smithi070_191495_object17086:head,460'1104,11:2c773904:::benchmark_data_smithi070_191495_object73350:head,469'4624,11:2c77caff:::benchmark_data_smithi070_191495_object85944:head,474'5403,11:2c77dcab:::benchmark_data_smithi070_191495_object69089:head,468'4356,11:2c78fff5:::benchmark_data_smithi070_191495_object77556:head,471'4862,11:2c796d68:::benchmark_data_smithi070_191495_object37997:head,464'2416,11:2c79f706:::benchmark_data_smithi070_191495_object56361:head,466'3507,11:2c7a74c8:::benchmark_data_smithi070_191495_object63732:head,467'4010,11:2c7af48a:::benchmark_data_smithi070_191495_object101681:head,480'6383,11:2c7b0cc2:::benchmark_data_smithi070_191495_object27737:head,463'1777,11:2c7b87e2:::benchmark_data_smithi070_191495_object54607:head,466'3393,11:2c7ba0d7:::benchmark_data_smithi070_191495_object46992:head,464'2948,11:2c7be410:::benchmark_data_smithi070_191495_object31068:head,463'1986,11:2c7caa79:::benchmark_data_smithi070_191495_object19026:head,460'1227,11:2c7e9153:::benchmark_data_smithi070_191495_object65634:head,467'4139,11:2c7ed583:::benchmark_data_smithi070_191495_object126767:head,491'7946,11:2c7fb57f:::benchmark_data_smithi070_191495_object21678:head,461'1395,11:2c80066d:::benchmark_data_smithi070_191495_object108499:head,483'6808,11:2c80bd97:::benchmark_data_smithi070_191495_object125908:head,490'7889,11:2c81024f:::benchmark_data_smithi070_191495_object92820:head,476'5828,11:2c815b96:::benchmark_data_smithi070_191495_object115672:head,486'7237,11:2c82655f:::benchmark_data_smithi070_191495_object79781:head,472'5000,11:2c82c772:::benchmark_data_smithi070_191495_object106656:head,481'6677,11:2c850a96:::benchmark_data_smithi070_191495_object83606:head,474'5262,11:2c8549e9:::benchmark_data_smithi070_191495_object33404:head,463'2118,11:2c8558d4:::benchmark_data_smithi070_191495_object116793:head,486'7305,11:2c85752f:::benchmark_data_smithi070_191495_object30340:head,463'1932,11:2c870d3b:::benchmark_data_smithi070_191495_object116590:head,486'7288,11:2c878345:::benchmark_data_smithi070_191495_object73759:head,469'4646,11:2c881727:::benchmark_data_smithi070_191495_object73358:head,469'4626,11:2c89280a:::benchmark_data_smithi070_191495_object71432:head,468'4504,11:2c89f46e:::benchmark_data_smithi070_191495_object106707:head,481'6679,11:2c8a88c6:::benchmark_data_smithi070_191495_object43842:head,464'2755,11:2c8c0f34:::benchmark_data_smithi070_191495_object108805:head,483'6831,11:2c8c78b4:::benchmark_data_smithi070_191495_object36309:head,464'2312,11:2c8d3874:::benchmark_data_smithi070_191495_object69916:head,468'4410,11:2c8db8a9:::benchmark_data_smithi070_191495_object48706:head,465'3037,11:2c8dbd78:::benchmark_data_smithi070_191495_object99903:head,479'6281,11:2c8eb170:::benchmark_data_smithi070_191495_object84731:head,474'5333)", 2024-09-16T09:44:44.730 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:44:26.753786+0000", 2024-09-16T09:44:44.731 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 17.878247405, 2024-09-16T09:44:44.731 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.015417434000000001, 2024-09-16T09:44:44.731 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:44:44.731 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:44:44.731 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T09:44:44.731 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:44:44.731 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.731 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:44:44.731 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.753786+0000", 2024-09-16T09:44:44.731 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:44:44.731 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.731 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.731 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:44:44.731 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.753786+0000", 2024-09-16T09:44:44.732 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:44:44.732 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.732 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.732 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:44:44.732 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.753786+0000", 2024-09-16T09:44:44.732 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.6100000000000002e-07 2024-09-16T09:44:44.732 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.732 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.732 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:44:44.732 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.753918+0000", 2024-09-16T09:44:44.732 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00013145400000000001 2024-09-16T09:44:44.732 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.732 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.733 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:44:44.733 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.754138+0000", 2024-09-16T09:44:44.733 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000220953 2024-09-16T09:44:44.733 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.733 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.733 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:44:44.733 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.754654+0000", 2024-09-16T09:44:44.733 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00051535800000000005 2024-09-16T09:44:44.733 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.733 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.733 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:44:44.733 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.766323+0000", 2024-09-16T09:44:44.733 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.011669473 2024-09-16T09:44:44.733 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.734 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.734 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:44:44.734 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.766891+0000", 2024-09-16T09:44:44.734 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00056776600000000004 2024-09-16T09:44:44.734 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.734 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.734 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:44:44.734 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.769203+0000", 2024-09-16T09:44:44.734 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.0023120689999999999 2024-09-16T09:44:44.734 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:44:44.734 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:44:44.734 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:44:44.734 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.734 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.734 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "pg_backfill(progress 11.4 e 519/519 lb 11:2c8eb170:::benchmark_data_smithi070_191495_object84731:head)", 2024-09-16T09:44:44.734 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:44:26.755940+0000", 2024-09-16T09:44:44.735 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 17.876092863, 2024-09-16T09:44:44.735 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.013370586, 2024-09-16T09:44:44.735 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:44:44.735 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:44:44.735 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T09:44:44.735 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:44:44.735 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.735 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:44:44.735 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.755940+0000", 2024-09-16T09:44:44.735 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:44:44.735 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.735 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.735 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:44:44.736 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.755940+0000", 2024-09-16T09:44:44.736 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:44:44.736 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.736 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.736 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:44:44.736 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.755941+0000", 2024-09-16T09:44:44.736 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.46e-07 2024-09-16T09:44:44.736 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.736 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.736 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:44:44.736 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.755949+0000", 2024-09-16T09:44:44.736 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 8.1429999999999998e-06 2024-09-16T09:44:44.737 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.737 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.737 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:44:44.737 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.755956+0000", 2024-09-16T09:44:44.737 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.2579999999999998e-06 2024-09-16T09:44:44.737 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.737 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.737 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:44:44.737 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.755961+0000", 2024-09-16T09:44:44.737 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.6310000000000002e-06 2024-09-16T09:44:44.737 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.737 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.737 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:44:44.737 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.769242+0000", 2024-09-16T09:44:44.738 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.013281764 2024-09-16T09:44:44.738 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.738 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.738 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:44:44.738 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.769259+0000", 2024-09-16T09:44:44.738 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.6874000000000001e-05 2024-09-16T09:44:44.738 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.738 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.738 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:44:44.738 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.769311+0000", 2024-09-16T09:44:44.739 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.147e-05 2024-09-16T09:44:44.739 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:44:44.739 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:44:44.739 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:44:44.739 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.739 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.739 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "backfill_remove(11.d e519 11:bd9d5548:::benchmark_data_smithi070_191495_object98536:head,479'6252,11:bd9dc3b6:::benchmark_data_smithi070_191495_object53636:head,466'3364,11:bd9e8b6b:::benchmark_data_smithi070_191495_object62301:head,467'3907,11:bd9ef108:::benchmark_data_smithi070_191495_object81453:head,473'5151,11:bd9f325f:::benchmark_data_smithi070_191495_object65313:head,467'4099,11:bda00d12:::benchmark_data_smithi070_191495_object100748:head,479'6394,11:bda0a6e8:::benchmark_data_smithi070_191495_object31558:head,463'1956,11:bda0fd52:::benchmark_data_smithi070_191495_object91203:head,476'5763,11:bda15611:::benchmark_data_smithi070_191495_object111507:head,484'7062,11:bda1b97a:::benchmark_data_smithi070_191495_object127360:head,491'8078,11:bda1e4e0:::benchmark_data_smithi070_191495_object89975:head,475'5701,11:bda240c4:::benchmark_data_smithi070_191495_object73276:head,469'4590,11:bda31398:::benchmark_data_smithi070_191495_object33268:head,463'2074,11:bda36591:::benchmark_data_smithi070_191495_object53777:head,466'3367,11:bda4d7fb:::benchmark_data_smithi070_191495_object92487:head,476'5868,11:bda4e937:::benchmark_data_smithi070_191495_object118521:head,487'7518,11:bda4ee10:::benchmark_data_smithi070_191495_object71641:head,468'4478,11:bda50120:::benchmark_data_smithi070_191495_object36564:head,464'2281,11:bda57d24:::benchmark_data_smithi070_191495_object42163:head,464'2630,11:bda65b3e:::benchmark_data_smithi070_191495_object120557:head,488'7639,11:bda676fb:::benchmark_data_smithi070_191495_object46702:head,464'2907,11:bda6783d:::benchmark_data_smithi070_191495_object108062:head,482'6843,11:bda6bb10:::benchmark_data_smithi070_191495_object91345:head,476'5776,11:bda8ca86:::benchmark_data_smithi070_191495_object65754:head,467'4121,11:bda92d5a:::benchmark_data_smithi070_191495_object65201:head,467'4091,11:bda9c53c:::benchmark_data_smithi070_191495_object111496:head,484'7061,11:bdaa2a58:::benchmark_data_smithi070_191495_object18234:head,460'1174,11:bdaa40c3:::benchmark_data_smithi070_191495_object84669:head,474'5341,11:bdab5a27:::benchmark_data_smithi070_191495_object48164:head,465'3020,11:bdabdd12:::benchmark_data_smithi070_191495_object81748:head,473'5174,11:bdac98a0:::benchmark_data_smithi070_191495_object89445:head,474'5662,11:bdacfef2:::benchmark_data_smithi070_191495_object104735:head,480'6645,11:bdad85e3:::benchmark_data_smithi070_191495_object67244:head,468'4202,11:bdad89b8:::benchmark_data_smithi070_191495_object70653:head,468'4414,11:bdad8f26:::benchmark_data_smithi070_191495_object20610:head,460'1312,11:bdada830:::benchmark_data_smithi070_191495_object53736:head,466'3365,11:bdaeb2ee:::benchmark_data_smithi070_191495_object124942:head,490'7924,11:bdaf0c61:::benchmark_data_smithi070_191495_object129728:head,491'8230,11:bdaf98e6:::benchmark_data_smithi070_191495_object27105:head,463'1699,11:bdaff6a7:::benchmark_data_smithi070_191495_object118475:head,487'7513,11:bdb06637:::benchmark_data_smithi070_191495_object23325:head,462'1472,11:bdb07ee3:::benchmark_data_smithi070_191495_object76015:head,471'4780,11:bdb0b939:::benchmark_data_smithi070_191495_object41686:head,464'2595,11:bdb0ce7b:::benchmark_data_smithi070_191495_object89478:head,474'5665,11:bdb0e071:::benchmark_data_smithi070_191495_object113920:head,486'7223,11:bdb17941:::benchmark_data_smithi070_191495_object95524:head,478'6056,11:bdb22bc6:::benchmark_data_smithi070_191495_object96633:head,479'6133,11:bdb271a0:::benchmark_data_smithi070_191495_object72457:head,468'4532,11:bdb2a876:::benchmark_data_smithi070_191495_object79019:head,472'4989,11:bdb2cf19:::benchmark_data_smithi070_191495_object28702:head,463'1787,11:bdb37304:::benchmark_data_smithi070_191495_object95109:head,478'6024,11:bdb3c576:::benchmark_data_smithi070_191495_object88580:head,474'5611,11:bdb4fb14:::benchmark_data_smithi070_191495_object114495:head,486'7257,11:bdb65b5f:::benchmark_data_smithi070_191495_object62281:head,467'3905,11:bdb7041e:::benchmark_data_smithi070_191495_object88332:head,474'5599,11:bdb72ece:::benchmark_data_smithi070_191495_object16488:head,460'1046,11:bdb86ecd:::benchmark_data_smithi070_191495_object59536:head,466'3730,11:bdb87e0e:::benchmark_data_smithi070_191495_object23640:head,462'1487,11:bdb8c760:::benchmark_data_smithi070_191495_object110131:head,483'6979,11:bdb9301f:::benchmark_data_smithi070_191495_object78057:head,472'4932,11:bdb9d94d:::benchmark_data_smithi070_191495_object60795:head,467'3808,11:bdb9dbaa:::benchmark_data_smithi070_191495_object62430:head,467'3919,11:bdba1194:::benchmark_data_smithi070_191495_object73173:head,469'4581,11:bdbabbbd:::benchmark_data_smithi070_191495_object88706:head,474'5619,11:bdbac1c6:::benchmark_data_smithi070_191495_object92767:head,476'5890,11:bdbad3cb:::benchmark_data_smithi070_191495_object57196:head,466'3603,11:bdbae6b9:::benchmark_data_smithi070_191495_object70210:head,468'4383,11:bdbb298c:::benchmark_data_smithi070_191495_object121105:head,488'7677,11:bdbb4299:::benchmark_data_smithi070_191495_object117424:head,486'7435,11:bdbb6a54:::benchmark_data_smithi070_191495_object76516:head,471'4819,11:bdbb833c:::benchmark_data_smithi070_191495_object37134:head,464'2325,11:bdbc0400:::benchmark_data_smithi070_191495_object32839:head,463'2048,11:bdbc8006:::benchmark_data_smithi070_191495_object74669:head,470'4678,11:bdbca6e1:::benchmark_data_smithi070_191495_object53105:head,466'3335,11:bdbe2797:::benchmark_data_smithi070_191495_object51189:head,465'3212,11:bdbe73e0:::benchmark_data_smithi070_191495_object28270:head,463'1761,11:bdbea7a4:::benchmark_data_smithi070_191495_object110337:head,483'6992,11:bdbf67d2:::benchmark_data_smithi070_191495_object54508:head,466'3424,11:bdbf79f2:::benchmark_data_smithi070_191495_object122251:head,488'7757,11:bdc290b7:::benchmark_data_smithi070_191495_object77335:head,471'4890,11:bdc2eceb:::benchmark_data_smithi070_191495_object70020:head,468'4368,11:bdc34a5d:::benchmark_data_smithi070_191495_object55377:head,466'3478,11:bdc44763:::benchmark_data_smithi070_191495_object98866:head,479'6276,11:bdc49dab:::benchmark_data_smithi070_191495_object97415:head,479'6184,11:bdc4f6f3:::benchmark_data_smithi070_191495_object127660:head,491'8094,11:bdc64380:::benchmark_data_smithi070_191495_object41314:head,464'2570,11:bdc6677a:::benchmark_data_smithi070_191495_object57785:head,466'3642,11:bdc76479:::benchmark_data_smithi070_191495_object102196:head,480'6482,11:bdc7cb8b:::benchmark_data_smithi070_191495_object86389:head,474'5467,11:bdc7edfc:::benchmark_data_smithi070_191495_object80184:head,472'5075,11:bdc8baea:::benchmark_data_smithi070_191495_object102649:head,480'6508,11:bdc8ca7b:::benchmark_data_smithi070_191495_object116986:head,486'7410,11:bdc926cb:::benchmark_data_smithi070_191495_object78242:head,472'4945,11:bdc933ef:::benchmark_data_smithi070_191495_object67883:head,468'4244,11:bdcb7e18:::benchmark_data_smithi070_191495_object32962:head,463'2054,11:bdcbc855:::benchmark_data_smithi070_191495_object115094:head,486'7297,11:bdccc867:::benchmark_data_smithi070_191495_object105986:head,481'6718,11:bdcd2c5d:::benchmark_data_smithi070_191495_object15976:head,460'1008,11:bdcd46fa:::benchmark_data_smithi070_191495_object44600:head,464'2787,11:bdcd860f:::benchmark_data_smithi070_191495_object127456:head,491'8084,11:bdcdb158:::benchmark_data_smithi070_191495_object100292:head,479'6370,11:bdceb7a4:::benchmark_data_smithi070_191495_object95383:head,478'6044,11:bdcf0d07:::benchmark_data_smithi070_191495_object65691:head,467'4116,11:bdcfbb58:::benchmark_data_smithi070_191495_object116844:head,486'7406,11:bdd01f0f:::benchmark_data_smithi070_191495_object82946:head,474'5252,11:bdd02bcd:::benchmark_data_smithi070_191495_object124419:head,489'7902,11:bdd05116:::benchmark_data_smithi070_191495_object46644:head,464'2905,11:bdd09984:::benchmark_data_smithi070_191495_object17543:head,460'1126,11:bdd10ed4:::benchmark_data_smithi070_191495_object113813:head,486'7221,11:bdd19f5f:::benchmark_data_smithi070_191495_object64443:head,467'4033,11:bdd1a7b8:::benchmark_data_smithi070_191495_object74178:head,470'4642,11:bdd2e9b6:::benchmark_data_smithi070_191495_object118245:head,487'7498,11:bdd3db6c:::benchmark_data_smithi070_191495_object16532:head,460'1050,11:bdd4b8e6:::benchmark_data_smithi070_191495_object75549:head,471'4741,11:bdd562da:::benchmark_data_smithi070_191495_object57630:head,466'3629,11:bdd57474:::benchmark_data_smithi070_191495_object34661:head,464'2159,11:bdd57cea:::benchmark_data_smithi070_191495_object117686:head,486'7454,11:bdd749a4:::benchmark_data_smithi070_191495_object18690:head,460'1199,11:bdd79fbd:::benchmark_data_smithi070_191495_object106773:head,481'6768,11:bdd7b401:::benchmark_data_smithi070_191495_object23383:head,462'1476,11:bdd7df35:::benchmark_data_smithi070_191495_object66664:head,468'4166,11:bdd871e7:::benchmark_data_smithi070_191495_object101225:head,479'6427,11:bdd8c88c:::benchmark_data_smithi070_191495_object104513:head,480'6634,11:bdd9235d:::benchmark_data_smithi070_191495_object99128:head,479'6295,11:bdd935d1:::benchmark_data_smithi070_191495_object68057:head,468'4258,11:bdd96f8c:::benchmark_data_smithi070_191495_object115216:head,486'7301,11:bdd9e82f:::benchmark_data_smithi070_191495_object85364:head,474'5387,11:bddaec73:::benchmark_data_smithi070_191495_object51718:head,465'3248,11:bddba959:::benchmark_data_smithi070_191495_object17887:head,460'1146,11:bddc4f6b:::benchmark_data_smithi070_191495_object104026:head,480'6609,11:bddc53e1:::benchmark_data_smithi070_191495_object49695:head,465'3122,11:bddd2723:::benchmark_data_smithi070_191495_object49723:head,465'3124,11:bddd4437:::benchmark_data_smithi070_191495_object61250:head,467'3841,11:bdde15b2:::benchmark_data_smithi070_191495_object65731:head,467'4119,11:bddef08c:::benchmark_data_smithi070_191495_object85503:head,474'5406,11:bddf0579:::benchmark_data_smithi070_191495_object96112:head,478'6087,11:bde0a8ca:::benchmark_data_smithi070_191495_object76835:head,471'4845,11:bde0fc9c:::benchmark_data_smithi070_191495_object118879:head,487'7542,11:bde18f34:::benchmark_data_smithi070_191495_object128580:head,491'8150,11:bde258f7:::benchmark_data_smithi070_191495_object27858:head,463'1746,11:bde2a8e0:::benchmark_data_smithi070_191495_object76538:head,471'4821,11:bde2fa45:::benchmark_data_smithi070_191495_object79906:head,472'5053,11:bde3ebc9:::benchmark_data_smithi070_191495_object90313:head,475'5719,11:bde41436:::benchmark_data_smithi070_191495_object112758:head,485'7156,11:bde47192:::benchmark_data_smithi070_191495_object46519:head,464'2900,11:bde5091a:::benchmark_data_smithi070_191495_object76044:head,471'4782,11:bde5e9c4:::benchmark_data_smithi070_191495_object19094:head,460'1224,11:bde6d7a5:::benchmark_data_smithi070_191495_object37078:head,464'2320,11:bde8072d:::benchmark_data_smithi070_191495_object29246:head,463'1818,11:bde822e7:::benchmark_data_smithi070_191495_object113811:head,486'7220,11:bde880b0:::benchmark_data_smithi070_191495_object85265:head,474'5378,11:bde8fe00:::benchmark_data_smithi070_191495_object35271:head,464'2198,11:bde9da23:::benchmark_data_smithi070_191495_object40232:head,464'2509,11:bdead94d:::benchmark_data_smithi070_191495_object93748:head,477'5941,11:bdeba6b0:::benchmark_data_smithi070_191495_object17065:head,460'1093,11:bdec46d4:::benchmark_data_smithi070_191495_object80053:head,472'5065,11:bded914f:::benchmark_data_smithi070_191495_object72189:head,468'4513,11:bded93e5:::benchmark_data_smithi070_191495_object50522:head,465'3170,11:bdee328e:::benchmark_data_smithi070_191495_object81514:head,473'5156,11:bdee4be1:::benchmark_data_smithi070_191495_object90267:head,475'5715,11:bdee7d2d:::benchmark_data_smithi070_191495_object108604:head,483'6881,11:bdf00f75:::benchmark_data_smithi070_191495_object117132:head,486'7420,11:bdf0109f:::benchmark_data_smithi070_191495_object58332:head,466'3671,11:bdf03b51:::benchmark_data_smithi070_191495_object83631:head,474'5282,11:bdf16fe5:::benchmark_data_smithi070_191495_object117111:head,486'7419,11:bdf2484c:::benchmark_data_smithi070_191495_object105423:head,481'6688,11:bdf32055:::benchmark_data_smithi070_191495_object91572:head,476'5792,11:bdf3c3cc:::benchmark_data_smithi070_191495_object35653:head,464'2218,11:bdf44410:::benchmark_data_smithi070_191495_object109387:head,483'6933,11:bdf4e47b:::benchmark_data_smithi070_191495_object121098:head,488'7675,11:bdf4e9b6:::benchmark_data_smithi070_191495_object75905:head,471'4771,11:bdf5588c:::benchmark_data_smithi070_191495_object41105:head,464'2555,11:bdf55970:::benchmark_data_smithi070_191495_object34764:head,464'2165,11:bdf5af2b:::benchmark_data_smithi070_191495_object128436:head,491'8141,11:bdf5d3ee:::benchmark_data_smithi070_191495_object85382:head,474'5391,11:bdf78152:::benchmark_data_smithi070_191495_object121667:head,488'7712,11:bdf84989:::benchmark_data_smithi070_191495_object102200:head,480'6483,11:bdf84e60:::benchmark_data_smithi070_191495_object62304:head,467'3908,11:bdf86e70:::benchmark_data_smithi070_191495_object92495:head,476'5869,11:bdf8cb4c:::benchmark_data_smithi070_191495_object26744:head,463'1672,11:bdf962b1:::benchmark_data_smithi070_191495_object58293:head,466'3667,11:bdf997a3:::benchmark_data_smithi070_191495_object115907:head,486'7342,11:bdf9db44:::benchmark_data_smithi070_191495_object24272:head,462'1527,11:bdfb0b66:::benchmark_data_smithi070_191495_object114933:head,486'7286,11:bdfb1957:::benchmark_data_smithi070_191495_object16972:head,460'1082,11:bdfb1cc2:::benchmark_data_smithi070_191495_object124948:head,490'7925,11:bdfb455e:::benchmark_data_smithi070_191495_object98590:head,479'6258,11:bdfcad82:::benchmark_data_smithi070_191495_object94218:head,478'5976,11:bdfd521c:::benchmark_data_smithi070_191495_object92834:head,476'5894,11:bdfdfc75:::benchmark_data_smithi070_191495_object23023:head,462'1455,11:bdfe44c7:::benchmark_data_smithi070_191495_object102256:head,480'6485,11:bdff724f:::benchmark_data_smithi070_191495_object24741:head,462'1565,11:bdffcc4f:::benchmark_data_smithi070_191495_object79055:head,472'4991,11:bdffe0f7:::benchmark_data_smithi070_191495_object99613:head,479'6324,11:be009a21:::benchmark_data_smithi070_191495_object42216:head,464'2637,11:be00e42e:::benchmark_data_smithi070_191495_object19970:head,460'1283,11:be010ea6:::benchmark_data_smithi070_191495_object92270:head,476'5850,11:be0311f2:::benchmark_data_smithi070_191495_object90015:head,475'5704,11:be039875:::benchmark_data_smithi070_191495_object25702:head,463'1611,11:be03af04:::benchmark_data_smithi070_191495_object75445:head,470'4734,11:be03b85c:::benchmark_data_smithi070_191495_object99875:head,479'6342,11:be04cc85:::benchmark_data_smithi070_191495_object91985:head,476'5826,11:be05d6e0:::benchmark_data_smithi070_191495_object102403:head,480'6493,11:be06065a:::benchmark_data_smithi070_191495_object111329:head,484'7042,11:be066989:::benchmark_data_smithi070_191495_object94932:head,478'6015,11:be06bf72:::benchmark_data_smithi070_191495_object118192:head,487'7492,11:be077014:::benchmark_data_smithi070_191495_object118247:head,487'7500,11:be077417:::benchmark_data_smithi070_191495_object126340:head,491'8012,11:be08c278:::benchmark_data_smithi070_191495_object94704:head,478'6006,11:be094cc1:::benchmark_data_smithi070_191495_object71475:head,468'4469,11:be09ca44:::benchmark_data_smithi070_191495_object65880:head,467'4127,11:be09ea5a:::benchmark_data_smithi070_191495_object34381:head,464'2146,11:be0bac7e:::benchmark_data_smithi070_191495_object24687:head,462'1561,11:be0c999a:::benchmark_data_smithi070_191495_object30550:head,463'1901,11:be0dbce5:::benchmark_data_smithi070_191495_object120843:head,488'7659,11:be0e7d38:::benchmark_data_smithi070_191495_object42204:head,464'2635,11:be0f8c63:::benchmark_data_smithi070_191495_object75231:head,470'4717,11:be1109cf:::benchmark_data_smithi070_191495_object22725:head,461'1432,11:be12112b:::benchmark_data_smithi070_191495_object64708:head,467'4058,11:be123fcb:::benchmark_data_smithi070_191495_object97801:head,479'6210,11:be128962:::benchmark_data_smithi070_191495_object109118:head,483'6912,11:be139cfc:::benchmark_data_smithi070_191495_object105860:head,481'6706,11:be13d629:::benchmark_data_smithi070_191495_object65242:head,467'4095,11:be142b21:::benchmark_data_smithi070_191495_object113508:head,486'7204,11:be14dc67:::benchmark_data_smithi070_191495_object19566:head,460'1260,11:be15c2dc:::benchmark_data_smithi070_191495_object63223:head,467'3956,11:be15ce9f:::benchmark_data_smithi070_191495_object82687:head,474'5239,11:be161da1:::benchmark_data_smithi070_191495_object77203:head,471'4878,11:be1754dd:::benchmark_data_smithi070_191495_object18129:head,460'1163,11:be1818c0:::benchmark_data_smithi070_191495_object91391:head,476'5777,11:be193cf3:::benchmark_data_smithi070_191495_object103726:head,480'6587,11:be19cd32:::benchmark_data_smithi070_191495_object114063:head,486'7235,11:be19f701:::benchmark_data_smithi070_191495_object103678:head,480'6584,11:be1a06a9:::benchmark_data_smithi070_191495_object35047:head,464'2181,11:be1a341c:::benchmark_data_smithi070_191495_object76593:head,471'4826,11:be1a67e6:::benchmark_data_smithi070_191495_object23301:head,462'1470,11:be1b1742:::benchmark_data_smithi070_191495_object42846:head,464'2681,11:be1b3340:::benchmark_data_smithi070_191495_object108463:head,483'6872,11:be1b6e5a:::benchmark_data_smithi070_191495_object62203:head,467'3898,11:be1b890c:::benchmark_data_smithi070_191495_object69048:head,468'4311,11:be1b8e23:::benchmark_data_smithi070_191495_object53514:head,466'3355,11:be1c5e56:::benchmark_data_smithi070_191495_object121509:head,488'7698,11:be1c75a1:::benchmark_data_smithi070_191495_object52989:head,466'3332,11:be1c7637:::benchmark_data_smithi070_191495_object78412:head,472'4956,11:be1cedbf:::benchmark_data_smithi070_191495_object97091:head,479'6170,11:be1d01aa:::benchmark_data_smithi070_191495_object38270:head,464'2391,11:be1d7409:::benchmark_data_smithi070_191495_object121596:head,488'7705,11:be1dac6c:::benchmark_data_smithi070_191495_object100811:head,479'6398,11:be1e9036:::benchmark_data_smithi070_191495_object128195:head,491'8128,11:be1eb10c:::benchmark_data_smithi070_191495_object80526:head,472'5094,11:be1f0c64:::benchmark_data_smithi070_191495_object105630:head,481'6698,11:be1fec91:::benchmark_data_smithi070_191495_object60089:head,467'3760,11:be205b44:::benchmark_data_smithi070_191495_object61098:head,467'3828,11:be2072cb:::benchmark_data_smithi070_191495_object115723:head,486'7331,11:be21ac51:::benchmark_data_smithi070_191495_object57334:head,466'3609,11:be21bd7c:::benchmark_data_smithi070_191495_object119657:head,488'7594,11:be21cb98:::benchmark_data_smithi070_191495_object125724:head,490'7975,11:be21e763:::benchmark_data_smithi070_191495_object92427:head,476'5864,11:be222a21:::benchmark_data_smithi070_191495_object46038:head,464'2876,11:be228aec:::benchmark_data_smithi070_191495_object33241:head,463'2070,11:be230bbe:::benchmark_data_smithi070_191495_object36831:head,464'2303,11:be233931:::benchmark_data_smithi070_191495_object64496:head,467'4037,11:be27fc4c:::benchmark_data_smithi070_191495_object21765:head,461'1381,11:be28e57f:::benchmark_data_smithi070_191495_object64327:head,467'4029,11:be29814c:::benchmark_data_smithi070_191495_object116843:head,486'7405,11:be29b56d:::benchmark_data_smithi070_191495_object103379:head,480'6562,11:be29d155:::benchmark_data_smithi070_191495_object68414:head,468'4266,11:be2b545e:::benchmark_data_smithi070_191495_object34317:head,464'2141,11:be2db93e:::benchmark_data_smithi070_191495_object65304:head,467'4098,11:be2e01b0:::benchmark_data_smithi070_191495_object130034:head,491'8245,11:be2f25fb:::benchmark_data_smithi070_191495_object92085:head,476'5833,11:be2f93c1:::benchmark_data_smithi070_191495_object129570:head,491'8221,11:be2ff960:::benchmark_data_smithi070_191495_object107860:head,482'6832,11:be305e93:::benchmark_data_smithi070_191495_object32159:head,463'1996,11:be306497:::benchmark_data_smithi070_191495_object20921:head,461'1328,11:be3090fe:::benchmark_data_smithi070_191495_object106710:head,481'6764,11:be30a8d5:::benchmark_data_smithi070_191495_object21205:head,461'1343,11:be31147f:::benchmark_data_smithi070_191495_object45725:head,464'2855,11:be31a953:::benchmark_data_smithi070_191495_object69317:head,468'4329,11:be31c3d7:::benchmark_data_smithi070_191495_object98018:head,479'6222,11:be3204d0:::benchmark_data_smithi070_191495_object25638:head,463'1605,11:be33522a:::benchmark_data_smithi070_191495_object29259:head,463'1820,11:be33562a:::benchmark_data_smithi070_191495_object41401:head,464'2577,11:be337643:::benchmark_data_smithi070_191495_object87369:head,474'5535,11:be349783:::benchmark_data_smithi070_191495_object104399:head,480'6625,11:be34ec68:::benchmark_data_smithi070_191495_object71022:head,468'4433,11:be3523da:::benchmark_data_smithi070_191495_object93934:head,477'5954,11:be35af1b:::benchmark_data_smithi070_191495_object72346:head,468'4526,11:be35d85a:::benchmark_data_smithi070_191495_object83029:head,474'5256,11:be364dc1:::benchmark_data_smithi070_191495_object82230:head,474'5204,11:be373031:::benchmark_data_smithi070_191495_object54644:head,466'3432,11:be37b481:::benchmark_data_smithi070_191495_object57100:head,466'3594,11:be38228f:::benchmark_data_smithi070_191495_object18181:head,460'1167,11:be3857f3:::benchmark_data_smithi070_191495_object121029:head,488'7671,11:be38870f:::benchmark_data_smithi070_191495_object83853:head,474'5297,11:be38a454:::benchmark_data_smithi070_191495_object76067:head,471'4785,11:be38c414:::benchmark_data_smithi070_191495_object79564:head,472'5035,11:be395e29:::benchmark_data_smithi070_191495_object55874:head,466'3517,11:be398fc2:::benchmark_data_smithi070_191495_object129465:head,491'8211,11:be3a4845:::benchmark_data_smithi070_191495_object82669:head,474'5237,11:be3b1669:::benchmark_data_smithi070_191495_object37042:head,464'2319,11:be3b4a75:::benchmark_data_smithi070_191495_object20800:head,460'1323,11:be3b5416:::benchmark_data_smithi070_191495_object16051:head,460'1012,11:be3ba57f:::benchmark_data_smithi070_191495_object124560:head,490'7910,11:be3bc14e:::benchmark_data_smithi070_191495_object17175:head,460'1101,11:be3c4493:::benchmark_data_smithi070_191495_object34830:head,464'2168,11:be3cf96a:::benchmark_data_smithi070_191495_object20491:head,460'1303,11:be3da5c6:::benchmark_data_smithi070_191495_object94832:head,478'6013,11:be3db8a7:::benchmark_data_smithi070_191495_object28641:head,463'1785,11:be3defc5:::benchmark_data_smithi070_191495_object19587:head,460'1262,11:be3e5d6e:::benchmark_data_smithi070_191495_object70833:head,468'4422,11:be3f14fb:::benchmark_data_smithi070_191495_object23034:head,462'1456,11:be3f74d0:::benchmark_data_smithi070_191495_object105125:head,481'6673,11:be3f8c69:::benchmark_data_smithi070_191495_object28738:head,463'1791,11:be3fcc34:::benchmark_data_smithi070_191495_object100074:head,479'6354,11:be3fd81c:::benchmark_data_smithi070_191495_object95084:head,478'6022,11:be40a6bc:::benchmark_data_smithi070_191495_object64284:head,467'4024,11:be40b57e:::benchmark_data_smithi070_191495_object91740:head,476'5802,11:be42ad04:::benchmark_data_smithi070_191495_object71114:head,468'4441,11:be4310b3:::benchmark_data_smithi070_191495_object112084:head,485'7111,11:be4318c8:::benchmark_data_smithi070_191495_object69846:head,468'4355,11:be434524:::benchmark_data_smithi070_191495_object53221:head,466'3346,11:be43b603:::benchmark_data_smithi070_191495_object93882:head,477'5947,11:be442d9f:::benchmark_data_smithi070_191495_object41324:head,464'2571,11:be45d847:::benchmark_data_smithi070_191495_object26926:head,463'1686,11:be460ab7:::benchmark_data_smithi070_191495_object73281:head,469'4591,11:be46a802:::benchmark_data_smithi070_191495_object47704:head,464'2983,11:be46b638:::benchmark_data_smithi070_191495_object74544:head,470'4671,11:be472110:::benchmark_data_smithi070_191495_object77953:head,471'4927,11:be4723dd:::benchmark_data_smithi070_191495_object128237:head,491'8131,11:be472ebd:::benchmark_data_smithi070_191495_object17949:head,460'1149,11:be4740b4:::benchmark_data_smithi070_191495_object40222:head,464'2508,11:be474c25:::benchmark_data_smithi070_191495_object31304:head,463'1946,11:be4766e3:::benchmark_data_smithi070_191495_object109513:head,483'6945,11:be47cb7c:::benchmark_data_smithi070_191495_object21723:head,461'1376,11:be49c63a:::benchmark_data_smithi070_191495_object72100:head,468'4503,11:be4b12f6:::benchmark_data_smithi070_191495_object121658:head,488'7711,11:be4ba0c4:::benchmark_data_smithi070_191495_object118677:head,487'7525,11:be4bb5a6:::benchmark_data_smithi070_191495_object84615:head,474'5338,11:be4c8ea1:::benchmark_data_smithi070_191495_object44380:head,464'2775,11:be4d1229:::benchmark_data_smithi070_191495_object26362:head,463'1649,11:be4da856:::benchmark_data_smithi070_191495_object88864:head,474'5628,11:be4e18cb:::benchmark_data_smithi070_191495_object99616:head,479'6325,11:be4f10df:::benchmark_data_smithi070_191495_object74747:head,470'4684,11:be4f4378:::benchmark_data_smithi070_191495_object76932:head,471'4849,11:be4fa44f:::benchmark_data_smithi070_191495_object59148:head,466'3707,11:be501cf0:::benchmark_data_smithi070_191495_object63344:head,467'3967,11:be505b3b:::benchmark_data_smithi070_191495_object17155:head,460'1099,11:be50f705:::benchmark_data_smithi070_191495_object103769:head,480'6591,11:be51adb5:::benchmark_data_smithi070_191495_object29164:head,463'1814,11:be5232a8:::benchmark_data_smithi070_191495_object99441:head,479'6311,11:be526b4c:::benchmark_data_smithi070_191495_object37083:head,464'2321,11:be550fa6:::benchmark_data_smithi070_191495_object55399:head,466'3480,11:be554141:::benchmark_data_smithi070_191495_object37372:head,464'2338,11:be557b6f:::benchmark_data_smithi070_191495_object20544:head,460'1307,11:be562c15:::benchmark_data_smithi070_191495_object30478:head,463'1899,11:be56af19:::benchmark_data_smithi070_191495_object56183:head,466'3533,11:be57222f:::benchmark_data_smithi070_191495_object107605:head,482'6815,11:be5723e9:::benchmark_data_smithi070_191495_object78190:head,472'4942,11:be57be7a:::benchmark_data_smithi070_191495_object79269:head,472'5011,11:be586009:::benchmark_data_smithi070_191495_object76271:head,471'4801,11:be591385:::benchmark_data_smithi070_191495_object98042:head,479'6224,11:be5999a8:::benchmark_data_smithi070_191495_object101870:head,480'6467,11:be59f3eb:::benchmark_data_smithi070_191495_object90036:head,475'5706,11:be5a6109:::benchmark_data_smithi070_191495_object89944:head,475'5699,11:be5ad188:::benchmark_data_smithi070_191495_object36978:head,464'2315,11:be5b817e:::benchmark_data_smithi070_191495_object118592:head,487'7521,11:be5beecd:::benchmark_data_smithi070_191495_object25491:head,463'1595,11:be5c003d:::benchmark_data_smithi070_191495_object85837:head,474'5434,11:be5c4962:::benchmark_data_smithi070_191495_object54936:head,466'3455,11:be5db484:::benchmark_data_smithi070_191495_object103805:head,480'6594,11:be5de275:::benchmark_data_smithi070_191495_object118842:head,487'7539,11:be5dfd17:::benchmark_data_smithi070_191495_object128704:head,491'8159,11:be60c390:::benchmark_data_smithi070_191495_object77657:head,471'4913,11:be60dc33:::benchmark_data_smithi070_191495_object120275:head,488'7624,11:be627757:::benchmark_data_smithi070_191495_object52041:head,465'3268,11:be64b0a4:::benchmark_data_smithi070_191495_object81055:head,473'5129,11:be65efd4:::benchmark_data_smithi070_191495_object45880:head,464'2864,11:be66805f:::benchmark_data_smithi070_191495_object89068:head,474'5639,11:be66b026:::benchmark_data_smithi070_191495_object30129:head,463'1880,11:be671391:::benchmark_data_smithi070_191495_object115800:head,486'7334,11:be67199a:::benchmark_data_smithi070_191495_object17009:head,460'1085,11:be689101:::benchmark_data_smithi070_191495_object68744:head,468'4282,11:be6ab16d:::benchmark_data_smithi070_191495_object79477:head,472'5027,11:be6b4a77:::benchmark_data_smithi070_191495_object129370:head,491'8202,11:be6b5741:::benchmark_data_smithi070_191495_object35613:head,464'2214,11:be6c0b89:::benchmark_data_smithi070_191495_object17693:head,460'1135,11:be6e34f6:::benchmark_data_smithi070_191495_object59511:head,466'3728,11:be6e509a:::benchmark_data_smithi070_191495_object65349:head,467'4102,11:be6eba7c:::benchmark_data_smithi070_191495_object50379:head,465'3161,11:be70d1e7:::benchmark_data_smithi070_191495_object20602:head,460'1311,11:be716ff8:::benchmark_data_smithi070_191495_object115868:head,486'7339,11:be722efe:::benchmark_data_smithi070_191495_object53883:head,466'3379,11:be72664d:::benchmark_data_smithi070_191495_object95301:head,478'6036,11:be7357d7:::benchmark_data_smithi070_191495_object39705:head,464'2478,11:be736442:::benchmark_data_smithi070_191495_object71123:head,468'4443,11:be73e3ec:::benchmark_data_smithi070_191495_object118739:head,487'7530,11:be73e4df:::benchmark_data_smithi070_191495_object112691:head,485'7153,11:be743eb3:::benchmark_data_smithi070_191495_object110328:head,483'6989,11:be74c202:::benchmark_data_smithi070_191495_object100620:head,479'6391,11:be74ca12:::benchmark_data_smithi070_191495_object83065:head,474'5258,11:be77b094:::benchmark_data_smithi070_191495_object35464:head,464'2207,11:be77eed9:::benchmark_data_smithi070_191495_object21799:head,461'1383,11:be795b05:::benchmark_data_smithi070_191495_object72181:head,468'4511,11:be7a025d:::benchmark_data_smithi070_191495_object24098:head,462'1520,11:be7b803a:::benchmark_data_smithi070_191495_object68799:head,468'4287,11:be7caf57:::benchmark_data_smithi070_191495_object78872:head,472'4982,11:be7cd8e8:::benchmark_data_smithi070_191495_object58269:head,466'3666,11:be7d1f87:::benchmark_data_smithi070_191495_object84292:head,474'5325,11:be7d26c3:::benchmark_data_smithi070_191495_object62592:head,467'3925,11:be7d5f5b:::benchmark_data_smithi070_191495_object34660:head,464'2158,11:be7e5a76:::benchmark_data_smithi070_191495_object106450:head,481'6744,11:be7f6f6b:::benchmark_data_smithi070_191495_object78540:head,472'4962,11:be7fcf65:::benchmark_data_smithi070_191495_object79220:head,472'5008,11:be8024bd:::benchmark_data_smithi070_191495_object105969:head,481'6715,11:be812dbe:::benchmark_data_smithi070_191495_object113745:head,486'7216,11:be8175b4:::benchmark_data_smithi070_191495_object53866:head,466'3376,11:be82bc90:::benchmark_data_smithi070_191495_object54761:head,466'3445,11:be83cc09:::benchmark_data_smithi070_191495_object60683:head,467'3798,11:be83deaa:::benchmark_data_smithi070_191495_object41290:head,464'2568,11:be844627:::benchmark_data_smithi070_191495_object85123:head,474'5366,11:be85b9ff:::benchmark_data_smithi070_191495_object46928:head,464'2928,11:be87209f:::benchmark_data_smithi070_191495_object83155:head,474'5262,11:be872f6f:::benchmark_data_smithi070_191495_object117941:head,486'7472,11:be87ec44:::benchmark_data_smithi070_191495_object117514:head,486'7446,11:be883112:::benchmark_data_smithi070_191495_object33821:head,464'2111,11:be88c4aa:::benchmark_data_smithi070_191495_object96769:head,479'6145,11:be891440:::benchmark_data_smithi070_191495_object43723:head,464'2733,11:be8960d1:::benchmark_data_smithi070_191495_object108072:head,482'6844,11:be8961e8:::benchmark_data_smithi070_191495_object98234:head,479'6237,11:be89636b:::benchmark_data_smithi070_191495_object89154:head,474'5647,11:be899d57:::benchmark_data_smithi070_191495_object79743:head,472'5045,11:be8a4932:::benchmark_data_smithi070_191495_object119642:head,488'7593,11:be8a59b2:::benchmark_data_smithi070_191495_object61234:head,467'3840,11:be8b039c:::benchmark_data_smithi070_191495_object122518:head,488'7778,11:be8b27bf:::benchmark_data_smithi070_191495_object78211:head,472'4944,11:be8b7488:::benchmark_data_smithi070_191495_object111390:head,484'7047,11:be8bbbed:::benchmark_data_smithi070_191495_object45005:head,464'2810,11:be8c7eaa:::benchmark_data_smithi070_191495_object118774:head,487'7535,11:be8d0fa5:::benchmark_data_smithi070_191495_object18395:head,460'1178,11:be8dd32e:::benchmark_data_smithi070_191495_object114238:head,486'7244,11:be8de774:::benchmark_data_smithi070_191495_object114394:head,486'7253,11:be8df174:::benchmark_data_smithi070_191495_object100519:head,479'6385,11:be8f9b32:::benchmark_data_smithi070_191495_object22075:head,461'1397,11:be907276:::benchmark_data_smithi070_191495_object74355:head,470'4661,11:be90ba5b:::benchmark_data_smithi070_191495_object111736:head,484'7087,11:be92261c:::benchmark_data_smithi070_191495_object44507:head,464'2783,11:be9303a9:::benchmark_data_smithi070_191495_object60752:head,467'3803,11:be936e1f:::benchmark_data_smithi070_191495_object38474:head,464'2401,11:be93966f:::benchmark_data_smithi070_191495_object52519:head,466'3296,11:be943e83:::benchmark_data_smithi070_191495_object114833:head,486'7282,11:be948450:::benchmark_data_smithi070_191495_object70545:head,468'4404,11:be961303:::benchmark_data_smithi070_191495_object111852:head,484'7096,11:be96403e:::benchmark_data_smithi070_191495_object70805:head,468'4419,11:be96e92b:::benchmark_data_smithi070_191495_object84119:head,474'5314,11:be9727b0:::benchmark_data_smithi070_191495_object101086:head,479'6419,11:be9825db:::benchmark_data_smithi070_191495_object29339:head,463'1826,11:be9a26d6:::benchmark_data_smithi070_191495_object17531:head,460'1124,11:be9b07af:::benchmark_data_smithi070_191495_object93201:head,477'5916,11:be9cc7b1:::benchmark_data_smithi070_191495_object27594:head,463'1724,11:be9cd97b:::benchmark_data_smithi070_191495_object26486:head,463'1657,11:be9d1aa9:::benchmark_data_smithi070_191495_object113392:head,486'7194,11:be9dd8b2:::benchmark_data_smithi070_191495_object102306:head,480'6487,11:be9e6a8d:::benchmark_data_smithi070_191495_object35389:head,464'2205,11:be9fd546:::benchmark_data_smithi070_191495_object36950:head,464'2313,11:bea081a8:::benchmark_data_smithi070_191495_object120346:head,488'7627,11:bea0c751:::benchmark_data_smithi070_191495_object91207:head,476'5765,11:bea0d34f:::benchmark_data_smithi070_191495_object77502:head,471'4903,11:bea0e73d:::benchmark_data_smithi070_191495_object122237:head,488'7756,11:bea0f0ee:::benchmark_data_smithi070_191495_object49764:head,465'3126,11:bea1347b:::benchmark_data_smithi070_191495_object47085:head,464'2940,11:bea1eb80:::benchmark_data_smithi070_191495_object23851:head,462'1502,11:bea35635:::benchmark_data_smithi070_191495_object56487:head,466'3555,11:bea3e403:::benchmark_data_smithi070_191495_object87904:head,474'5573,11:bea4325c:::benchmark_data_smithi070_191495_object80639:head,473'5101,11:bea4acce:::benchmark_data_smithi070_191495_object51310:head,465'3222,11:bea4d06e:::benchmark_data_smithi070_191495_object67965:head,468'4249,11:bea52068:::benchmark_data_smithi070_191495_object114594:head,486'7268,11:bea5908e:::benchmark_data_smithi070_191495_object98542:head,479'6254,11:bea5cb29:::benchmark_data_smithi070_191495_object117708:head,486'7457,11:bea623c6:::benchmark_data_smithi070_191495_object119529:head,488'7583,11:bea693c8:::benchmark_data_smithi070_191495_object105362:head,481'6685,11:bea6b929:::benchmark_data_smithi070_191495_object110541:head,483'7005,11:bea721a5:::benchmark_data_smithi070_191495_object26622:head,463'1665,11:bea73feb:::benchmark_data_smithi070_191495_object31049:head,463'1926,11:bea826dc:::benchmark_data_smithi070_191495_object89817:head,475'5685,11:bea8d9c1:::benchmark_data_smithi070_191495_object61282:head,467'3845,11:bea8db7f:::benchmark_data_smithi070_191495_object33785:head,464'2107,11:bea91f58:::benchmark_data_smithi070_191495_object38961:head,464'2433,11:bea96db7:::benchmark_data_smithi070_191495_object44278:head,464'2765,11:beaaecee:::benchmark_data_smithi070_191495_object74703:head,470'4681,11:beab9163:::benchmark_data_smithi070_191495_object44397:head,464'2777,11:beac75d7:::benchmark_data_smithi070_191495_object93164:head,477'5911,11:bead76a5:::benchmark_data_smithi070_191495_object59217:head,466'3712,11:beae9085:::benchmark_data_smithi070_191495_object32083:head,463'1991,11:beaeca6e:::benchmark_data_smithi070_191495_object67019:head,468'4185,11:beaf003f:::benchmark_data_smithi070_191495_object49569:head,465'3116,11:beb00b82:::benchmark_data_smithi070_191495_object111548:head,484'7065,11:beb016ff:::benchmark_data_smithi070_191495_object37487:head,464'2342,11:beb07f4a:::benchmark_data_smithi070_191495_object126096:head,491'7997,11:beb154ae:::benchmark_data_smithi070_191495_object21651:head,461'1371,11:beb1740f:::benchmark_data_smithi070_191495_object58585:head,466'3683,11:beb2b35f:::benchmark_data_smithi070_191495_object59023:head,466'3703,11:beb4bf04:::benchmark_data_smithi070_191495_object21858:head,461'1387,11:beb60afa:::benchmark_data_smithi070_191495_object67760:head,468'4237,11:beb6330c:::benchmark_data_smithi070_191495_object26725:head,463'1671,11:beb67834:::benchmark_data_smithi070_191495_object130454:head,492'8264,11:beb7616a:::benchmark_data_smithi070_191495_object120638:head,488'7645,11:beb7e263:::benchmark_data_smithi070_191495_object35072:head,464'2184,11:beb907f2:::benchmark_data_smithi070_191495_object53421:head,466'3352,11:beba99ef:::benchmark_data_smithi070_191495_object26644:head,463'1669,11:bebab16e:::benchmark_data_smithi070_191495_object49287:head,465'3098)", 2024-09-16T09:44:44.739 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:44:26.877899+0000", 2024-09-16T09:44:44.739 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 17.754134311000001, 2024-09-16T09:44:44.740 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.015684323999999999, 2024-09-16T09:44:44.740 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:44:44.740 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:44:44.740 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T09:44:44.740 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:44:44.740 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.740 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:44:44.740 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.877899+0000", 2024-09-16T09:44:44.740 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:44:44.740 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.740 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.740 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:44:44.740 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.877899+0000", 2024-09-16T09:44:44.740 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:44:44.741 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.741 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.741 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:44:44.741 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.877899+0000", 2024-09-16T09:44:44.741 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 5.4799999999999998e-07 2024-09-16T09:44:44.741 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.741 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.741 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:44:44.741 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.878011+0000", 2024-09-16T09:44:44.741 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000112148 2024-09-16T09:44:44.741 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.741 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.741 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:44:44.741 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.878234+0000", 2024-09-16T09:44:44.742 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.000222294 2024-09-16T09:44:44.742 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.742 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.742 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:44:44.742 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.878699+0000", 2024-09-16T09:44:44.742 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00046546599999999999 2024-09-16T09:44:44.742 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.742 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.742 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:44:44.742 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.889426+0000", 2024-09-16T09:44:44.742 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.010726800999999999 2024-09-16T09:44:44.742 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.742 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.743 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:44:44.743 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.890348+0000", 2024-09-16T09:44:44.743 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.00092225399999999996 2024-09-16T09:44:44.743 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.743 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.743 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:44:44.743 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.893583+0000", 2024-09-16T09:44:44.743 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.003234813 2024-09-16T09:44:44.743 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:44:44.743 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:44:44.743 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:44:44.743 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.743 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.744 INFO:teuthology.orchestra.run.smithi070.stdout: "description": "pg_backfill(progress 11.d e 519/519 lb 11:bebab16e:::benchmark_data_smithi070_191495_object49287:head)", 2024-09-16T09:44:44.744 INFO:teuthology.orchestra.run.smithi070.stdout: "initiated_at": "2024-09-16T09:44:26.879525+0000", 2024-09-16T09:44:44.744 INFO:teuthology.orchestra.run.smithi070.stdout: "age": 17.752508098, 2024-09-16T09:44:44.744 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.014156846000000001, 2024-09-16T09:44:44.744 INFO:teuthology.orchestra.run.smithi070.stdout: "continuous": false, 2024-09-16T09:44:44.744 INFO:teuthology.orchestra.run.smithi070.stdout: "type_data": { 2024-09-16T09:44:44.744 INFO:teuthology.orchestra.run.smithi070.stdout: "flag_point": "started", 2024-09-16T09:44:44.744 INFO:teuthology.orchestra.run.smithi070.stdout: "events": [ 2024-09-16T09:44:44.744 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.744 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "initiated", 2024-09-16T09:44:44.744 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.879525+0000", 2024-09-16T09:44:44.744 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:44:44.744 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.744 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.745 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "header_read", 2024-09-16T09:44:44.745 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.879525+0000", 2024-09-16T09:44:44.745 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0 2024-09-16T09:44:44.745 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.745 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.745 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "throttled", 2024-09-16T09:44:44.745 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.879525+0000", 2024-09-16T09:44:44.745 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 3.7599999999999998e-07 2024-09-16T09:44:44.745 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.745 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.745 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "all_read", 2024-09-16T09:44:44.745 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.879532+0000", 2024-09-16T09:44:44.746 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 6.703e-06 2024-09-16T09:44:44.746 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.746 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.746 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "dispatched", 2024-09-16T09:44:44.746 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.879539+0000", 2024-09-16T09:44:44.746 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 7.3930000000000002e-06 2024-09-16T09:44:44.746 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.746 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.746 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "queued_for_pg", 2024-09-16T09:44:44.746 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.879544+0000", 2024-09-16T09:44:44.746 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.3760000000000001e-06 2024-09-16T09:44:44.746 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.746 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.746 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "reached_pg", 2024-09-16T09:44:44.747 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.893621+0000", 2024-09-16T09:44:44.747 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 0.014077200999999999 2024-09-16T09:44:44.747 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.747 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.747 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "started", 2024-09-16T09:44:44.747 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.893639+0000", 2024-09-16T09:44:44.747 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 1.7802e-05 2024-09-16T09:44:44.747 INFO:teuthology.orchestra.run.smithi070.stdout: }, 2024-09-16T09:44:44.747 INFO:teuthology.orchestra.run.smithi070.stdout: { 2024-09-16T09:44:44.747 INFO:teuthology.orchestra.run.smithi070.stdout: "event": "done", 2024-09-16T09:44:44.747 INFO:teuthology.orchestra.run.smithi070.stdout: "time": "2024-09-16T09:44:26.893682+0000", 2024-09-16T09:44:44.747 INFO:teuthology.orchestra.run.smithi070.stdout: "duration": 4.2994999999999997e-05 2024-09-16T09:44:44.747 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:44:44.747 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:44:44.748 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:44:44.748 INFO:teuthology.orchestra.run.smithi070.stdout: } 2024-09-16T09:44:44.748 INFO:teuthology.orchestra.run.smithi070.stdout: ] 2024-09-16T09:44:44.748 INFO:teuthology.orchestra.run.smithi070.stdout:} 2024-09-16T09:44:44.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:44 smithi084 ceph-mon[32629]: Health check cleared: OSDMAP_FLAGS (was: noscrub flag(s) set) 2024-09-16T09:44:44.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:44 smithi084 ceph-mon[32629]: Cluster is now healthy 2024-09-16T09:44:44.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:44 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-16T09:44:44.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:44 smithi084 ceph-mon[32629]: osdmap e531: 8 total, 8 up, 8 in 2024-09-16T09:44:44.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:44 smithi084 ceph-mon[32629]: 1.10 scrub starts 2024-09-16T09:44:44.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:44 smithi084 ceph-mon[32629]: 1.10 scrub ok 2024-09-16T09:44:45.513 INFO:teuthology.orchestra.run.smithi070.stderr:reweighted osd.3 to 1 (10000) 2024-09-16T09:44:45.779 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:45 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:44:45.779 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:45 smithi070 ceph-mon[30388]: pgmap v1037: 152 pgs: 152 active+clean; 583 KiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 915 B/s rd, 0 op/s 2024-09-16T09:44:45.779 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:45 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2979831827' entity='client.admin' cmd={"prefix": "osd reweight", "id": 3, "weight": 1.0} : dispatch 2024-09-16T09:44:45.779 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:45 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 3, "weight": 1.0} : dispatch 2024-09-16T09:44:45.780 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:45 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:44:45.780 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:45 smithi070 ceph-mon[40691]: pgmap v1037: 152 pgs: 152 active+clean; 583 KiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 915 B/s rd, 0 op/s 2024-09-16T09:44:45.780 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:45 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/2979831827' entity='client.admin' cmd={"prefix": "osd reweight", "id": 3, "weight": 1.0} : dispatch 2024-09-16T09:44:45.781 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:45 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 3, "weight": 1.0} : dispatch 2024-09-16T09:44:45.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:45 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:44:45.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:45 smithi084 ceph-mon[32629]: pgmap v1037: 152 pgs: 152 active+clean; 583 KiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 915 B/s rd, 0 op/s 2024-09-16T09:44:45.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:45 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/2979831827' entity='client.admin' cmd={"prefix": "osd reweight", "id": 3, "weight": 1.0} : dispatch 2024-09-16T09:44:45.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:45 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 3, "weight": 1.0} : dispatch 2024-09-16T09:44:46.386 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd reweight 4 1 2024-09-16T09:44:46.414 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:44:46.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:46 smithi070 ceph-mon[30388]: 4.f scrub starts 2024-09-16T09:44:46.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:46 smithi070 ceph-mon[30388]: 4.f scrub ok 2024-09-16T09:44:46.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:46 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-16T09:44:46.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:46 smithi070 ceph-mon[30388]: osdmap e532: 8 total, 8 up, 8 in 2024-09-16T09:44:46.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:46 smithi070 ceph-mon[30388]: 5.12 scrub starts 2024-09-16T09:44:46.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:46 smithi070 ceph-mon[30388]: 5.12 scrub ok 2024-09-16T09:44:46.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:46 smithi070 ceph-mon[40691]: 4.f scrub starts 2024-09-16T09:44:46.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:46 smithi070 ceph-mon[40691]: 4.f scrub ok 2024-09-16T09:44:46.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:46 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-16T09:44:46.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:46 smithi070 ceph-mon[40691]: osdmap e532: 8 total, 8 up, 8 in 2024-09-16T09:44:46.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:46 smithi070 ceph-mon[40691]: 5.12 scrub starts 2024-09-16T09:44:46.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:46 smithi070 ceph-mon[40691]: 5.12 scrub ok 2024-09-16T09:44:46.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:46 smithi084 ceph-mon[32629]: 4.f scrub starts 2024-09-16T09:44:46.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:46 smithi084 ceph-mon[32629]: 4.f scrub ok 2024-09-16T09:44:46.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:46 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-16T09:44:46.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:46 smithi084 ceph-mon[32629]: osdmap e532: 8 total, 8 up, 8 in 2024-09-16T09:44:46.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:46 smithi084 ceph-mon[32629]: 5.12 scrub starts 2024-09-16T09:44:46.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:46 smithi084 ceph-mon[32629]: 5.12 scrub ok 2024-09-16T09:44:46.919 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:44:47.528 INFO:teuthology.orchestra.run.smithi070.stderr:nodeep-scrub is unset 2024-09-16T09:44:47.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:47 smithi070 ceph-mon[30388]: pgmap v1039: 152 pgs: 152 active+clean; 583 KiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:44:47.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:47 smithi070 ceph-mon[30388]: 6.14 scrub starts 2024-09-16T09:44:47.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:47 smithi070 ceph-mon[30388]: 6.14 scrub ok 2024-09-16T09:44:47.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:47 smithi070 ceph-mon[30388]: 6.15 scrub starts 2024-09-16T09:44:47.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:47 smithi070 ceph-mon[30388]: 6.15 scrub ok 2024-09-16T09:44:47.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:47 smithi070 ceph-mon[30388]: 6.7 scrub starts 2024-09-16T09:44:47.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:47 smithi070 ceph-mon[30388]: 6.7 scrub ok 2024-09-16T09:44:47.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:47 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3137832502' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:44:47.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:47 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:44:47.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:47 smithi070 ceph-mon[40691]: pgmap v1039: 152 pgs: 152 active+clean; 583 KiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:44:47.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:47 smithi070 ceph-mon[40691]: 6.14 scrub starts 2024-09-16T09:44:47.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:47 smithi070 ceph-mon[40691]: 6.14 scrub ok 2024-09-16T09:44:47.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:47 smithi070 ceph-mon[40691]: 6.15 scrub starts 2024-09-16T09:44:47.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:47 smithi070 ceph-mon[40691]: 6.15 scrub ok 2024-09-16T09:44:47.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:47 smithi070 ceph-mon[40691]: 6.7 scrub starts 2024-09-16T09:44:47.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:47 smithi070 ceph-mon[40691]: 6.7 scrub ok 2024-09-16T09:44:47.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:47 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3137832502' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:44:47.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:47 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:44:47.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:47 smithi084 ceph-mon[32629]: pgmap v1039: 152 pgs: 152 active+clean; 583 KiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:44:47.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:47 smithi084 ceph-mon[32629]: 6.14 scrub starts 2024-09-16T09:44:47.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:47 smithi084 ceph-mon[32629]: 6.14 scrub ok 2024-09-16T09:44:47.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:47 smithi084 ceph-mon[32629]: 6.15 scrub starts 2024-09-16T09:44:47.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:47 smithi084 ceph-mon[32629]: 6.15 scrub ok 2024-09-16T09:44:47.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:47 smithi084 ceph-mon[32629]: 6.7 scrub starts 2024-09-16T09:44:47.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:47 smithi084 ceph-mon[32629]: 6.7 scrub ok 2024-09-16T09:44:47.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:47 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3137832502' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:44:47.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:47 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd={"prefix": "osd unset", "key": "nodeep-scrub"} : dispatch 2024-09-16T09:44:48.531 INFO:teuthology.orchestra.run.smithi070.stderr:reweighted osd.4 to 1 (10000) 2024-09-16T09:44:48.545 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:48 smithi070 ceph-mon[30388]: 6.11 scrub starts 2024-09-16T09:44:48.546 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:48 smithi070 ceph-mon[30388]: 6.11 scrub ok 2024-09-16T09:44:48.546 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:48 smithi070 ceph-mon[30388]: 4.10 scrub starts 2024-09-16T09:44:48.546 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:48 smithi070 ceph-mon[30388]: 4.10 scrub ok 2024-09-16T09:44:48.546 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:48 smithi070 ceph-mon[30388]: 5.c scrub starts 2024-09-16T09:44:48.546 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:48 smithi070 ceph-mon[30388]: 5.c scrub ok 2024-09-16T09:44:48.546 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:48 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-16T09:44:48.546 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:48 smithi070 ceph-mon[30388]: osdmap e533: 8 total, 8 up, 8 in 2024-09-16T09:44:48.546 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:48 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1236986929' entity='client.admin' cmd={"prefix": "osd reweight", "id": 4, "weight": 1.0} : dispatch 2024-09-16T09:44:48.546 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:48 smithi070 ceph-mon[40691]: 6.11 scrub starts 2024-09-16T09:44:48.546 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:48 smithi070 ceph-mon[40691]: 6.11 scrub ok 2024-09-16T09:44:48.546 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:48 smithi070 ceph-mon[40691]: 4.10 scrub starts 2024-09-16T09:44:48.547 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:48 smithi070 ceph-mon[40691]: 4.10 scrub ok 2024-09-16T09:44:48.547 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:48 smithi070 ceph-mon[40691]: 5.c scrub starts 2024-09-16T09:44:48.547 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:48 smithi070 ceph-mon[40691]: 5.c scrub ok 2024-09-16T09:44:48.547 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:48 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-16T09:44:48.547 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:48 smithi070 ceph-mon[40691]: osdmap e533: 8 total, 8 up, 8 in 2024-09-16T09:44:48.547 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:48 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1236986929' entity='client.admin' cmd={"prefix": "osd reweight", "id": 4, "weight": 1.0} : dispatch 2024-09-16T09:44:48.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:48 smithi084 ceph-mon[32629]: 6.11 scrub starts 2024-09-16T09:44:48.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:48 smithi084 ceph-mon[32629]: 6.11 scrub ok 2024-09-16T09:44:48.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:48 smithi084 ceph-mon[32629]: 4.10 scrub starts 2024-09-16T09:44:48.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:48 smithi084 ceph-mon[32629]: 4.10 scrub ok 2024-09-16T09:44:48.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:48 smithi084 ceph-mon[32629]: 5.c scrub starts 2024-09-16T09:44:48.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:48 smithi084 ceph-mon[32629]: 5.c scrub ok 2024-09-16T09:44:48.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:48 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-16T09:44:48.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:48 smithi084 ceph-mon[32629]: osdmap e533: 8 total, 8 up, 8 in 2024-09-16T09:44:48.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:48 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1236986929' entity='client.admin' cmd={"prefix": "osd reweight", "id": 4, "weight": 1.0} : dispatch 2024-09-16T09:44:49.362 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd reweight 5 1 2024-09-16T09:44:49.616 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:44:49.884 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:49 smithi070 ceph-mon[30388]: pgmap v1041: 152 pgs: 152 active+clean; 583 KiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-16T09:44:49.885 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:49 smithi070 ceph-mon[30388]: 4.b scrub starts 2024-09-16T09:44:49.885 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:49 smithi070 ceph-mon[30388]: 4.b scrub ok 2024-09-16T09:44:49.885 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:49 smithi070 ceph-mon[30388]: 6.e scrub starts 2024-09-16T09:44:49.885 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:49 smithi070 ceph-mon[30388]: 6.e scrub ok 2024-09-16T09:44:49.885 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:49 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1236986929' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-16T09:44:49.885 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:49 smithi070 ceph-mon[30388]: osdmap e534: 8 total, 8 up, 8 in 2024-09-16T09:44:49.885 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:49 smithi070 ceph-mon[30388]: 6.5 scrub starts 2024-09-16T09:44:49.885 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:49 smithi070 ceph-mon[30388]: 6.5 scrub ok 2024-09-16T09:44:49.885 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:49 smithi070 ceph-mon[30388]: 3.1b scrub starts 2024-09-16T09:44:49.885 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:49 smithi070 ceph-mon[30388]: 3.1b scrub ok 2024-09-16T09:44:49.885 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:49 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:44:49.885 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:49 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:44:49.886 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:49 smithi070 ceph-mon[40691]: pgmap v1041: 152 pgs: 152 active+clean; 583 KiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-16T09:44:49.886 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:49 smithi070 ceph-mon[40691]: 4.b scrub starts 2024-09-16T09:44:49.886 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:49 smithi070 ceph-mon[40691]: 4.b scrub ok 2024-09-16T09:44:49.886 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:49 smithi070 ceph-mon[40691]: 6.e scrub starts 2024-09-16T09:44:49.886 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:49 smithi070 ceph-mon[40691]: 6.e scrub ok 2024-09-16T09:44:49.886 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:49 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1236986929' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-16T09:44:49.886 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:49 smithi070 ceph-mon[40691]: osdmap e534: 8 total, 8 up, 8 in 2024-09-16T09:44:49.886 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:49 smithi070 ceph-mon[40691]: 6.5 scrub starts 2024-09-16T09:44:49.886 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:49 smithi070 ceph-mon[40691]: 6.5 scrub ok 2024-09-16T09:44:49.886 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:49 smithi070 ceph-mon[40691]: 3.1b scrub starts 2024-09-16T09:44:49.886 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:49 smithi070 ceph-mon[40691]: 3.1b scrub ok 2024-09-16T09:44:49.886 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:49 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:44:49.886 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:49 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:44:49.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:49 smithi084 ceph-mon[32629]: pgmap v1041: 152 pgs: 152 active+clean; 583 KiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-16T09:44:49.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:49 smithi084 ceph-mon[32629]: 4.b scrub starts 2024-09-16T09:44:49.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:49 smithi084 ceph-mon[32629]: 4.b scrub ok 2024-09-16T09:44:49.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:49 smithi084 ceph-mon[32629]: 6.e scrub starts 2024-09-16T09:44:49.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:49 smithi084 ceph-mon[32629]: 6.e scrub ok 2024-09-16T09:44:49.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:49 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1236986929' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-16T09:44:49.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:49 smithi084 ceph-mon[32629]: osdmap e534: 8 total, 8 up, 8 in 2024-09-16T09:44:49.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:49 smithi084 ceph-mon[32629]: 6.5 scrub starts 2024-09-16T09:44:49.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:49 smithi084 ceph-mon[32629]: 6.5 scrub ok 2024-09-16T09:44:49.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:49 smithi084 ceph-mon[32629]: 3.1b scrub starts 2024-09-16T09:44:49.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:49 smithi084 ceph-mon[32629]: 3.1b scrub ok 2024-09-16T09:44:49.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:49 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:44:49.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:49 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:44:50.539 INFO:teuthology.orchestra.run.smithi070.stderr:reweighted osd.5 to 1 (10000) 2024-09-16T09:44:50.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:50 smithi084 ceph-mon[32629]: 5.1c scrub starts 2024-09-16T09:44:50.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:50 smithi084 ceph-mon[32629]: 5.1c scrub ok 2024-09-16T09:44:50.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:50 smithi084 ceph-mon[32629]: 5.11 scrub starts 2024-09-16T09:44:50.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:50 smithi084 ceph-mon[32629]: 5.11 scrub ok 2024-09-16T09:44:50.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:50 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1870813992' entity='client.admin' cmd={"prefix": "osd reweight", "id": 5, "weight": 1.0} : dispatch 2024-09-16T09:44:50.920 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:50 smithi070 ceph-mon[40691]: 5.1c scrub starts 2024-09-16T09:44:50.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:50 smithi070 ceph-mon[40691]: 5.1c scrub ok 2024-09-16T09:44:50.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:50 smithi070 ceph-mon[40691]: 5.11 scrub starts 2024-09-16T09:44:50.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:50 smithi070 ceph-mon[40691]: 5.11 scrub ok 2024-09-16T09:44:50.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:50 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1870813992' entity='client.admin' cmd={"prefix": "osd reweight", "id": 5, "weight": 1.0} : dispatch 2024-09-16T09:44:50.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:50 smithi070 ceph-mon[30388]: 5.1c scrub starts 2024-09-16T09:44:50.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:50 smithi070 ceph-mon[30388]: 5.1c scrub ok 2024-09-16T09:44:50.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:50 smithi070 ceph-mon[30388]: 5.11 scrub starts 2024-09-16T09:44:50.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:50 smithi070 ceph-mon[30388]: 5.11 scrub ok 2024-09-16T09:44:50.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:50 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1870813992' entity='client.admin' cmd={"prefix": "osd reweight", "id": 5, "weight": 1.0} : dispatch 2024-09-16T09:44:51.306 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd reweight 6 1 2024-09-16T09:44:51.547 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:51 smithi070 ceph-mon[40691]: pgmap v1043: 152 pgs: 152 active+clean; 583 KiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:44:51.547 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:51 smithi070 ceph-mon[40691]: 5.e scrub starts 2024-09-16T09:44:51.547 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:51 smithi070 ceph-mon[40691]: 5.e scrub ok 2024-09-16T09:44:51.547 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:51 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1870813992' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-16T09:44:51.567 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:44:51.820 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:51 smithi070 ceph-mon[30388]: pgmap v1043: 152 pgs: 152 active+clean; 583 KiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:44:51.821 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:51 smithi070 ceph-mon[30388]: 5.e scrub starts 2024-09-16T09:44:51.821 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:51 smithi070 ceph-mon[30388]: 5.e scrub ok 2024-09-16T09:44:51.821 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:51 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1870813992' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-16T09:44:51.821 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:51 smithi070 ceph-mon[30388]: osdmap e535: 8 total, 8 up, 8 in 2024-09-16T09:44:51.821 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:51 smithi070 ceph-mon[30388]: 6.1d scrub starts 2024-09-16T09:44:51.821 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:51 smithi070 ceph-mon[30388]: 6.1d scrub ok 2024-09-16T09:44:51.821 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:51 smithi070 ceph-mon[40691]: osdmap e535: 8 total, 8 up, 8 in 2024-09-16T09:44:51.822 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:51 smithi070 ceph-mon[40691]: 6.1d scrub starts 2024-09-16T09:44:51.822 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:51 smithi070 ceph-mon[40691]: 6.1d scrub ok 2024-09-16T09:44:51.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:51 smithi084 ceph-mon[32629]: pgmap v1043: 152 pgs: 152 active+clean; 583 KiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:44:51.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:51 smithi084 ceph-mon[32629]: 5.e scrub starts 2024-09-16T09:44:51.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:51 smithi084 ceph-mon[32629]: 5.e scrub ok 2024-09-16T09:44:51.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:51 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1870813992' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-16T09:44:51.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:51 smithi084 ceph-mon[32629]: osdmap e535: 8 total, 8 up, 8 in 2024-09-16T09:44:51.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:51 smithi084 ceph-mon[32629]: 6.1d scrub starts 2024-09-16T09:44:51.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:51 smithi084 ceph-mon[32629]: 6.1d scrub ok 2024-09-16T09:44:52.085 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:44:51 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:44:51] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:44:52.554 INFO:teuthology.orchestra.run.smithi070.stderr:reweighted osd.6 to 1 (10000) 2024-09-16T09:44:52.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:52 smithi084 ceph-mon[32629]: 1.c scrub starts 2024-09-16T09:44:52.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:52 smithi084 ceph-mon[32629]: 1.c scrub ok 2024-09-16T09:44:52.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:52 smithi084 ceph-mon[32629]: 6.12 scrub starts 2024-09-16T09:44:52.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:52 smithi084 ceph-mon[32629]: 6.12 scrub ok 2024-09-16T09:44:52.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:52 smithi084 ceph-mon[32629]: 5.2 scrub starts 2024-09-16T09:44:52.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:52 smithi084 ceph-mon[32629]: 5.2 scrub ok 2024-09-16T09:44:52.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:52 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3411469787' entity='client.admin' cmd={"prefix": "osd reweight", "id": 6, "weight": 1.0} : dispatch 2024-09-16T09:44:52.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:52 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 6, "weight": 1.0} : dispatch 2024-09-16T09:44:52.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:52 smithi070 ceph-mon[30388]: 1.c scrub starts 2024-09-16T09:44:52.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:52 smithi070 ceph-mon[30388]: 1.c scrub ok 2024-09-16T09:44:52.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:52 smithi070 ceph-mon[30388]: 6.12 scrub starts 2024-09-16T09:44:52.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:52 smithi070 ceph-mon[30388]: 6.12 scrub ok 2024-09-16T09:44:52.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:52 smithi070 ceph-mon[30388]: 5.2 scrub starts 2024-09-16T09:44:52.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:52 smithi070 ceph-mon[30388]: 5.2 scrub ok 2024-09-16T09:44:52.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:52 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3411469787' entity='client.admin' cmd={"prefix": "osd reweight", "id": 6, "weight": 1.0} : dispatch 2024-09-16T09:44:52.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:52 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 6, "weight": 1.0} : dispatch 2024-09-16T09:44:52.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:52 smithi070 ceph-mon[40691]: 1.c scrub starts 2024-09-16T09:44:52.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:52 smithi070 ceph-mon[40691]: 1.c scrub ok 2024-09-16T09:44:52.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:52 smithi070 ceph-mon[40691]: 6.12 scrub starts 2024-09-16T09:44:52.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:52 smithi070 ceph-mon[40691]: 6.12 scrub ok 2024-09-16T09:44:52.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:52 smithi070 ceph-mon[40691]: 5.2 scrub starts 2024-09-16T09:44:52.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:52 smithi070 ceph-mon[40691]: 5.2 scrub ok 2024-09-16T09:44:52.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:52 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3411469787' entity='client.admin' cmd={"prefix": "osd reweight", "id": 6, "weight": 1.0} : dispatch 2024-09-16T09:44:52.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:52 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 6, "weight": 1.0} : dispatch 2024-09-16T09:44:53.293 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd reweight 7 1 2024-09-16T09:44:53.546 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:44:53.811 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:44:53 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:44:53.811 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:53 smithi084 ceph-mon[32629]: pgmap v1045: 152 pgs: 152 active+clean; 583 KiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:44:53.811 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:53 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-16T09:44:53.811 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:53 smithi084 ceph-mon[32629]: osdmap e536: 8 total, 8 up, 8 in 2024-09-16T09:44:53.812 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:53 smithi084 ceph-mon[32629]: 5.1f scrub starts 2024-09-16T09:44:53.812 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:53 smithi084 ceph-mon[32629]: 5.1f scrub ok 2024-09-16T09:44:53.812 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:53 smithi084 ceph-mon[32629]: 6.19 scrub starts 2024-09-16T09:44:53.812 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:53 smithi084 ceph-mon[32629]: 6.19 scrub ok 2024-09-16T09:44:53.812 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:53 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:44:53.812 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:53 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:44:53.858 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:53 smithi070 ceph-mon[30388]: pgmap v1045: 152 pgs: 152 active+clean; 583 KiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:44:53.858 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:53 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-16T09:44:53.858 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:53 smithi070 ceph-mon[30388]: osdmap e536: 8 total, 8 up, 8 in 2024-09-16T09:44:53.858 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:53 smithi070 ceph-mon[30388]: 5.1f scrub starts 2024-09-16T09:44:53.858 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:53 smithi070 ceph-mon[30388]: 5.1f scrub ok 2024-09-16T09:44:53.858 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:53 smithi070 ceph-mon[30388]: 6.19 scrub starts 2024-09-16T09:44:53.859 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:53 smithi070 ceph-mon[30388]: 6.19 scrub ok 2024-09-16T09:44:53.859 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:53 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:44:53.859 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:53 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:44:53.859 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:53 smithi070 ceph-mon[40691]: pgmap v1045: 152 pgs: 152 active+clean; 583 KiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:44:53.859 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:53 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-16T09:44:53.859 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:53 smithi070 ceph-mon[40691]: osdmap e536: 8 total, 8 up, 8 in 2024-09-16T09:44:53.859 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:53 smithi070 ceph-mon[40691]: 5.1f scrub starts 2024-09-16T09:44:53.859 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:53 smithi070 ceph-mon[40691]: 5.1f scrub ok 2024-09-16T09:44:53.859 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:53 smithi070 ceph-mon[40691]: 6.19 scrub starts 2024-09-16T09:44:53.859 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:53 smithi070 ceph-mon[40691]: 6.19 scrub ok 2024-09-16T09:44:53.859 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:53 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:44:53.860 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:53 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:44:54.557 INFO:teuthology.orchestra.run.smithi070.stderr:reweighted osd.7 to 1 (10000) 2024-09-16T09:44:54.879 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:54 smithi070 ceph-mon[30388]: osdmap e537: 8 total, 8 up, 8 in 2024-09-16T09:44:54.880 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:54 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/111567759' entity='client.admin' cmd={"prefix": "osd reweight", "id": 7, "weight": 1.0} : dispatch 2024-09-16T09:44:54.880 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:54 smithi070 ceph-mon[40691]: osdmap e537: 8 total, 8 up, 8 in 2024-09-16T09:44:54.880 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:54 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/111567759' entity='client.admin' cmd={"prefix": "osd reweight", "id": 7, "weight": 1.0} : dispatch 2024-09-16T09:44:54.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:54 smithi084 ceph-mon[32629]: osdmap e537: 8 total, 8 up, 8 in 2024-09-16T09:44:54.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:54 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/111567759' entity='client.admin' cmd={"prefix": "osd reweight", "id": 7, "weight": 1.0} : dispatch 2024-09-16T09:44:55.209 INFO:tasks.cephadm.ceph_manager.ceph:waiting for recovery to complete 2024-09-16T09:44:55.210 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-16T09:44:55.459 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:44:55.901 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:55 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:44:55.901 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:55 smithi070 ceph-mon[40691]: pgmap v1048: 152 pgs: 152 active+clean; 583 KiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 948 B/s rd, 0 op/s 2024-09-16T09:44:55.901 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:55 smithi070 ceph-mon[40691]: 6.17 scrub starts 2024-09-16T09:44:55.901 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:55 smithi070 ceph-mon[40691]: 6.17 scrub ok 2024-09-16T09:44:55.901 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:55 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/111567759' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-16T09:44:55.901 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:55 smithi070 ceph-mon[40691]: osdmap e538: 8 total, 8 up, 8 in 2024-09-16T09:44:55.901 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:55 smithi070 ceph-mon[40691]: 5.16 scrub starts 2024-09-16T09:44:55.901 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:55 smithi070 ceph-mon[40691]: 5.16 scrub ok 2024-09-16T09:44:55.901 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:55 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:44:55.902 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:55 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:44:55.902 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:55 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:44:55.902 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:55 smithi070 ceph-mon[30388]: pgmap v1048: 152 pgs: 152 active+clean; 583 KiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 948 B/s rd, 0 op/s 2024-09-16T09:44:55.902 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:55 smithi070 ceph-mon[30388]: 6.17 scrub starts 2024-09-16T09:44:55.902 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:55 smithi070 ceph-mon[30388]: 6.17 scrub ok 2024-09-16T09:44:55.902 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:55 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/111567759' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-16T09:44:55.902 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:55 smithi070 ceph-mon[30388]: osdmap e538: 8 total, 8 up, 8 in 2024-09-16T09:44:55.902 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:55 smithi070 ceph-mon[30388]: 5.16 scrub starts 2024-09-16T09:44:55.902 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:55 smithi070 ceph-mon[30388]: 5.16 scrub ok 2024-09-16T09:44:55.902 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:55 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:44:55.902 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:55 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:44:55.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:55 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:44:55.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:55 smithi084 ceph-mon[32629]: pgmap v1048: 152 pgs: 152 active+clean; 583 KiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 948 B/s rd, 0 op/s 2024-09-16T09:44:55.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:55 smithi084 ceph-mon[32629]: 6.17 scrub starts 2024-09-16T09:44:55.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:55 smithi084 ceph-mon[32629]: 6.17 scrub ok 2024-09-16T09:44:55.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:55 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/111567759' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-16T09:44:55.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:55 smithi084 ceph-mon[32629]: osdmap e538: 8 total, 8 up, 8 in 2024-09-16T09:44:55.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:55 smithi084 ceph-mon[32629]: 5.16 scrub starts 2024-09-16T09:44:55.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:55 smithi084 ceph-mon[32629]: 5.16 scrub ok 2024-09-16T09:44:55.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:55 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:44:55.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:55 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:44:56.174 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:44:56.177 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2024-09-16T09:44:56.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:56 smithi084 ceph-mon[32629]: 6.13 scrub starts 2024-09-16T09:44:56.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:56 smithi084 ceph-mon[32629]: 6.13 scrub ok 2024-09-16T09:44:56.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:56 smithi084 ceph-mon[32629]: osdmap e539: 8 total, 8 up, 8 in 2024-09-16T09:44:56.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:56 smithi084 ceph-mon[32629]: 1.9 scrub starts 2024-09-16T09:44:56.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:56 smithi084 ceph-mon[32629]: 1.9 scrub ok 2024-09-16T09:44:56.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:56 smithi070 ceph-mon[30388]: 6.13 scrub starts 2024-09-16T09:44:56.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:56 smithi070 ceph-mon[30388]: 6.13 scrub ok 2024-09-16T09:44:56.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:56 smithi070 ceph-mon[30388]: osdmap e539: 8 total, 8 up, 8 in 2024-09-16T09:44:56.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:56 smithi070 ceph-mon[30388]: 1.9 scrub starts 2024-09-16T09:44:56.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:56 smithi070 ceph-mon[30388]: 1.9 scrub ok 2024-09-16T09:44:56.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:56 smithi070 ceph-mon[40691]: 6.13 scrub starts 2024-09-16T09:44:56.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:56 smithi070 ceph-mon[40691]: 6.13 scrub ok 2024-09-16T09:44:56.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:56 smithi070 ceph-mon[40691]: osdmap e539: 8 total, 8 up, 8 in 2024-09-16T09:44:56.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:56 smithi070 ceph-mon[40691]: 1.9 scrub starts 2024-09-16T09:44:56.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:56 smithi070 ceph-mon[40691]: 1.9 scrub ok 2024-09-16T09:44:56.949 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":1051,"stamp":"2024-09-16T09:44:55.928460+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596507,"num_objects":199,"num_object_clones":0,"num_object_copies":597,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":199,"num_whiteouts":0,"num_read":2322,"num_read_kb":2074,"num_write":722,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":151,"num_bytes_recovered":333352,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2434,"ondisk_log_size":2434,"up":456,"acting":456,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":458,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":6895372,"kb_used_data":851876,"kb_used_omap":15,"kb_used_meta":6043312,"kb_avail":743065844,"statfs":{"total":767960285184,"available":760899424256,"internally_reserved":0,"allocated":872321024,"data_stored":869497167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15600,"internal_metadata":6188352272},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000948"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":1037,"reported_epoch":538,"state":"active+clean","last_fresh":"2024-09-16T09:44:54.558549+0000","last_change":"2024-09-16T09:44:24.277767+0000","last_active":"2024-09-16T09:44:54.558549+0000","last_peered":"2024-09-16T09:44:54.558549+0000","last_clean":"2024-09-16T09:44:54.558549+0000","last_became_active":"2024-09-16T09:44:24.277455+0000","last_became_peered":"2024-09-16T09:44:24.277455+0000","last_unstale":"2024-09-16T09:44:54.558549+0000","last_undegraded":"2024-09-16T09:44:54.558549+0000","last_fullsized":"2024-09-16T09:44:54.558549+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:40.073089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:41:40.073089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"285'15","reported_seq":1042,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.703529+0000","last_change":"2024-09-16T09:44:05.286194+0000","last_active":"2024-09-16T09:44:52.703529+0000","last_peered":"2024-09-16T09:44:52.703529+0000","last_clean":"2024-09-16T09:44:52.703529+0000","last_became_active":"2024-09-16T09:25:03.714635+0000","last_became_peered":"2024-09-16T09:25:03.714635+0000","last_unstale":"2024-09-16T09:44:52.703529+0000","last_undegraded":"2024-09-16T09:44:52.703529+0000","last_fullsized":"2024-09-16T09:44:52.703529+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'15","last_scrub_stamp":"2024-09-16T09:44:05.286119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:44:05.286119+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:29.124567+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":644,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.552434+0000","last_change":"2024-09-16T09:42:31.714835+0000","last_active":"2024-09-16T09:44:52.552434+0000","last_peered":"2024-09-16T09:44:52.552434+0000","last_clean":"2024-09-16T09:44:52.552434+0000","last_became_active":"2024-09-16T09:33:30.242238+0000","last_became_peered":"2024-09-16T09:33:30.242238+0000","last_unstale":"2024-09-16T09:44:52.552434+0000","last_undegraded":"2024-09-16T09:44:52.552434+0000","last_fullsized":"2024-09-16T09:44:52.552434+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:42:31.714749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:42:31.714749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:06.776034+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1000,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.703757+0000","last_change":"2024-09-16T09:44:35.357353+0000","last_active":"2024-09-16T09:44:52.703757+0000","last_peered":"2024-09-16T09:44:52.703757+0000","last_clean":"2024-09-16T09:44:52.703757+0000","last_became_active":"2024-09-16T09:25:07.715442+0000","last_became_peered":"2024-09-16T09:25:07.715442+0000","last_unstale":"2024-09-16T09:44:52.703757+0000","last_undegraded":"2024-09-16T09:44:52.703757+0000","last_fullsized":"2024-09-16T09:44:52.703757+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:35.357286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:44:35.357286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:52.302674+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":748,"reported_epoch":537,"state":"active+clean","last_fresh":"2024-09-16T09:44:53.557032+0000","last_change":"2024-09-16T09:44:34.236344+0000","last_active":"2024-09-16T09:44:53.557032+0000","last_peered":"2024-09-16T09:44:53.557032+0000","last_clean":"2024-09-16T09:44:53.557032+0000","last_became_active":"2024-09-16T09:31:09.269342+0000","last_became_peered":"2024-09-16T09:31:09.269342+0000","last_unstale":"2024-09-16T09:44:53.557032+0000","last_undegraded":"2024-09-16T09:44:53.557032+0000","last_fullsized":"2024-09-16T09:44:53.557032+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:34.236298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:44:34.236298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:59.040431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":891,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.551991+0000","last_change":"2024-09-16T09:44:26.603265+0000","last_active":"2024-09-16T09:44:52.551991+0000","last_peered":"2024-09-16T09:44:52.551991+0000","last_clean":"2024-09-16T09:44:52.551991+0000","last_became_active":"2024-09-16T09:44:24.279187+0000","last_became_peered":"2024-09-16T09:44:24.279187+0000","last_unstale":"2024-09-16T09:44:52.551991+0000","last_undegraded":"2024-09-16T09:44:52.551991+0000","last_fullsized":"2024-09-16T09:44:52.551991+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:26.603213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:44:26.603213+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:50.281936+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.19","version":"285'27","reported_seq":223,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.703526+0000","last_change":"2024-09-16T09:40:58.802023+0000","last_active":"2024-09-16T09:44:52.703526+0000","last_peered":"2024-09-16T09:44:52.703526+0000","last_clean":"2024-09-16T09:44:52.703526+0000","last_became_active":"2024-09-16T09:40:58.699357+0000","last_became_peered":"2024-09-16T09:40:58.699357+0000","last_unstale":"2024-09-16T09:44:52.703526+0000","last_undegraded":"2024-09-16T09:44:52.703526+0000","last_fullsized":"2024-09-16T09:44:52.703526+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'27","last_scrub_stamp":"2024-09-16T09:37:29.683568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:37:29.683568+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:43.367028+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,5],"acting":[4,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1013,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.703484+0000","last_change":"2024-09-16T09:42:10.051996+0000","last_active":"2024-09-16T09:44:52.703484+0000","last_peered":"2024-09-16T09:44:52.703484+0000","last_clean":"2024-09-16T09:44:52.703484+0000","last_became_active":"2024-09-16T09:25:05.717236+0000","last_became_peered":"2024-09-16T09:25:05.717236+0000","last_unstale":"2024-09-16T09:44:52.703484+0000","last_undegraded":"2024-09-16T09:44:52.703484+0000","last_fullsized":"2024-09-16T09:44:52.703484+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:42:10.051939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:42:10.051939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":657,"reported_epoch":538,"state":"active+clean","last_fresh":"2024-09-16T09:44:54.557931+0000","last_change":"2024-09-16T09:43:23.679786+0000","last_active":"2024-09-16T09:44:54.557931+0000","last_peered":"2024-09-16T09:44:54.557931+0000","last_clean":"2024-09-16T09:44:54.557931+0000","last_became_active":"2024-09-16T09:33:30.247733+0000","last_became_peered":"2024-09-16T09:33:30.247733+0000","last_unstale":"2024-09-16T09:44:54.557931+0000","last_undegraded":"2024-09-16T09:44:54.557931+0000","last_fullsized":"2024-09-16T09:44:54.557931+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:43:23.679720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:43:23.679720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1a","version":"285'15","reported_seq":1041,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.704077+0000","last_change":"2024-09-16T09:43:59.240127+0000","last_active":"2024-09-16T09:44:52.704077+0000","last_peered":"2024-09-16T09:44:52.704077+0000","last_clean":"2024-09-16T09:44:52.704077+0000","last_became_active":"2024-09-16T09:25:03.709147+0000","last_became_peered":"2024-09-16T09:25:03.709147+0000","last_unstale":"2024-09-16T09:44:52.704077+0000","last_undegraded":"2024-09-16T09:44:52.704077+0000","last_fullsized":"2024-09-16T09:44:52.704077+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'15","last_scrub_stamp":"2024-09-16T09:43:59.240052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:43:59.240052+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:06.629394+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":995,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.552415+0000","last_change":"2024-09-16T09:42:57.714429+0000","last_active":"2024-09-16T09:44:52.552415+0000","last_peered":"2024-09-16T09:44:52.552415+0000","last_clean":"2024-09-16T09:44:52.552415+0000","last_became_active":"2024-09-16T09:25:05.718134+0000","last_became_peered":"2024-09-16T09:25:05.718134+0000","last_unstale":"2024-09-16T09:44:52.552415+0000","last_undegraded":"2024-09-16T09:44:52.552415+0000","last_fullsized":"2024-09-16T09:44:52.552415+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:42:57.714362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:42:57.714362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:40.690246+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"66'1","reported_seq":1022,"reported_epoch":538,"state":"active+clean","last_fresh":"2024-09-16T09:44:54.557873+0000","last_change":"2024-09-16T09:43:28.772774+0000","last_active":"2024-09-16T09:44:54.557873+0000","last_peered":"2024-09-16T09:44:54.557873+0000","last_clean":"2024-09-16T09:44:54.557873+0000","last_became_active":"2024-09-16T09:25:07.716558+0000","last_became_peered":"2024-09-16T09:25:07.716558+0000","last_unstale":"2024-09-16T09:44:54.557873+0000","last_undegraded":"2024-09-16T09:44:54.557873+0000","last_fullsized":"2024-09-16T09:44:54.557873+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-16T09:43:28.772704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:43:28.772704+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:44:56.277517+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":995,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.551711+0000","last_change":"2024-09-16T09:42:22.732120+0000","last_active":"2024-09-16T09:44:52.551711+0000","last_peered":"2024-09-16T09:44:52.551711+0000","last_clean":"2024-09-16T09:44:52.551711+0000","last_became_active":"2024-09-16T09:25:01.696939+0000","last_became_peered":"2024-09-16T09:25:01.696939+0000","last_unstale":"2024-09-16T09:44:52.551711+0000","last_undegraded":"2024-09-16T09:44:52.551711+0000","last_fullsized":"2024-09-16T09:44:52.551711+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:42:22.732048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:42:22.732048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:10.853456+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"285'7","reported_seq":1005,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.703876+0000","last_change":"2024-09-16T09:35:39.975915+0000","last_active":"2024-09-16T09:44:52.703876+0000","last_peered":"2024-09-16T09:44:52.703876+0000","last_clean":"2024-09-16T09:44:52.703876+0000","last_became_active":"2024-09-16T09:25:03.716136+0000","last_became_peered":"2024-09-16T09:25:03.716136+0000","last_unstale":"2024-09-16T09:44:52.703876+0000","last_undegraded":"2024-09-16T09:44:52.703876+0000","last_fullsized":"2024-09-16T09:44:52.703876+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'7","last_scrub_stamp":"2024-09-16T09:35:39.975840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:35:39.975840+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:17.269033+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":10,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":151,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.551221+0000","last_change":"2024-09-16T09:42:08.915352+0000","last_active":"2024-09-16T09:44:52.551221+0000","last_peered":"2024-09-16T09:44:52.551221+0000","last_clean":"2024-09-16T09:44:52.551221+0000","last_became_active":"2024-09-16T09:42:02.494290+0000","last_became_peered":"2024-09-16T09:42:02.494290+0000","last_unstale":"2024-09-16T09:44:52.551221+0000","last_undegraded":"2024-09-16T09:44:52.551221+0000","last_fullsized":"2024-09-16T09:44:52.551221+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:42:08.915303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:42:08.915303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4],"acting":[6,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1030,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.742833+0000","last_change":"2024-09-16T09:44:52.742833+0000","last_active":"2024-09-16T09:44:52.742833+0000","last_peered":"2024-09-16T09:44:52.742833+0000","last_clean":"2024-09-16T09:44:52.742833+0000","last_became_active":"2024-09-16T09:31:09.275656+0000","last_became_peered":"2024-09-16T09:31:09.275656+0000","last_unstale":"2024-09-16T09:44:52.742833+0000","last_undegraded":"2024-09-16T09:44:52.742833+0000","last_fullsized":"2024-09-16T09:44:52.742833+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:52.742776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:44:52.742776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:05.256885+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1002,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.704464+0000","last_change":"2024-09-16T09:42:57.111424+0000","last_active":"2024-09-16T09:44:52.704464+0000","last_peered":"2024-09-16T09:44:52.704464+0000","last_clean":"2024-09-16T09:44:52.704464+0000","last_became_active":"2024-09-16T09:25:07.720793+0000","last_became_peered":"2024-09-16T09:25:07.720793+0000","last_unstale":"2024-09-16T09:44:52.704464+0000","last_undegraded":"2024-09-16T09:44:52.704464+0000","last_fullsized":"2024-09-16T09:44:52.704464+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:42:57.111229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:42:57.111229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":752,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.551474+0000","last_change":"2024-09-16T09:44:48.756968+0000","last_active":"2024-09-16T09:44:52.551474+0000","last_peered":"2024-09-16T09:44:52.551474+0000","last_clean":"2024-09-16T09:44:52.551474+0000","last_became_active":"2024-09-16T09:30:39.201824+0000","last_became_peered":"2024-09-16T09:30:39.201824+0000","last_unstale":"2024-09-16T09:44:52.551474+0000","last_undegraded":"2024-09-16T09:44:52.551474+0000","last_fullsized":"2024-09-16T09:44:52.551474+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:48.756890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:44:48.756890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:17.988105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"285'27","reported_seq":1070,"reported_epoch":535,"state":"active+clean","last_fresh":"2024-09-16T09:44:50.542777+0000","last_change":"2024-09-16T09:44:24.284901+0000","last_active":"2024-09-16T09:44:50.542777+0000","last_peered":"2024-09-16T09:44:50.542777+0000","last_clean":"2024-09-16T09:44:50.542777+0000","last_became_active":"2024-09-16T09:44:24.283762+0000","last_became_peered":"2024-09-16T09:44:24.283762+0000","last_unstale":"2024-09-16T09:44:50.542777+0000","last_undegraded":"2024-09-16T09:44:50.542777+0000","last_fullsized":"2024-09-16T09:44:50.542777+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'27","last_scrub_stamp":"2024-09-16T09:42:52.891931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:42:52.891931+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:41.042706+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":636,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.704476+0000","last_change":"2024-09-16T09:44:02.300014+0000","last_active":"2024-09-16T09:44:52.704476+0000","last_peered":"2024-09-16T09:44:52.704476+0000","last_clean":"2024-09-16T09:44:52.704476+0000","last_became_active":"2024-09-16T09:33:30.251587+0000","last_became_peered":"2024-09-16T09:33:30.251587+0000","last_unstale":"2024-09-16T09:44:52.704476+0000","last_undegraded":"2024-09-16T09:44:52.704476+0000","last_fullsized":"2024-09-16T09:44:52.704476+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:02.299963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:44:02.299963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:23.146994+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,0],"acting":[4,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":858,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.551857+0000","last_change":"2024-09-16T09:43:16.704220+0000","last_active":"2024-09-16T09:44:52.551857+0000","last_peered":"2024-09-16T09:44:52.551857+0000","last_clean":"2024-09-16T09:44:52.551857+0000","last_became_active":"2024-09-16T09:28:52.974255+0000","last_became_peered":"2024-09-16T09:28:52.974255+0000","last_unstale":"2024-09-16T09:44:52.551857+0000","last_undegraded":"2024-09-16T09:44:52.551857+0000","last_fullsized":"2024-09-16T09:44:52.551857+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:43:16.704155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:43:16.704155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":1031,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.704248+0000","last_change":"2024-09-16T09:44:28.385487+0000","last_active":"2024-09-16T09:44:52.704248+0000","last_peered":"2024-09-16T09:44:52.704248+0000","last_clean":"2024-09-16T09:44:52.704248+0000","last_became_active":"2024-09-16T09:44:24.276029+0000","last_became_peered":"2024-09-16T09:44:24.276029+0000","last_unstale":"2024-09-16T09:44:52.704248+0000","last_undegraded":"2024-09-16T09:44:52.704248+0000","last_fullsized":"2024-09-16T09:44:52.704248+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:28.385444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:44:28.385444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:52.078449+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"285'18","reported_seq":696,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.704538+0000","last_change":"2024-09-16T09:43:16.141437+0000","last_active":"2024-09-16T09:44:52.704538+0000","last_peered":"2024-09-16T09:44:52.704538+0000","last_clean":"2024-09-16T09:44:52.704538+0000","last_became_active":"2024-09-16T09:33:30.251626+0000","last_became_peered":"2024-09-16T09:33:30.251626+0000","last_unstale":"2024-09-16T09:44:52.704538+0000","last_undegraded":"2024-09-16T09:44:52.704538+0000","last_fullsized":"2024-09-16T09:44:52.704538+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'18","last_scrub_stamp":"2024-09-16T09:43:16.141356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:43:16.141356+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":33,"num_read_kb":24,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1007,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.704003+0000","last_change":"2024-09-16T09:44:49.325447+0000","last_active":"2024-09-16T09:44:52.704003+0000","last_peered":"2024-09-16T09:44:52.704003+0000","last_clean":"2024-09-16T09:44:52.704003+0000","last_became_active":"2024-09-16T09:44:24.278117+0000","last_became_peered":"2024-09-16T09:44:24.278117+0000","last_unstale":"2024-09-16T09:44:52.704003+0000","last_undegraded":"2024-09-16T09:44:52.704003+0000","last_fullsized":"2024-09-16T09:44:52.704003+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:49.325384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:44:49.325384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:53.841965+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":1007,"reported_epoch":535,"state":"active+clean","last_fresh":"2024-09-16T09:44:50.625428+0000","last_change":"2024-09-16T09:44:24.284719+0000","last_active":"2024-09-16T09:44:50.625428+0000","last_peered":"2024-09-16T09:44:50.625428+0000","last_clean":"2024-09-16T09:44:50.625428+0000","last_became_active":"2024-09-16T09:44:24.284256+0000","last_became_peered":"2024-09-16T09:44:24.284256+0000","last_unstale":"2024-09-16T09:44:50.625428+0000","last_undegraded":"2024-09-16T09:44:50.625428+0000","last_fullsized":"2024-09-16T09:44:50.625428+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-16T09:42:25.250120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:42:25.250120+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":655,"reported_epoch":537,"state":"active+clean","last_fresh":"2024-09-16T09:44:53.556138+0000","last_change":"2024-09-16T09:41:36.612463+0000","last_active":"2024-09-16T09:44:53.556138+0000","last_peered":"2024-09-16T09:44:53.556138+0000","last_clean":"2024-09-16T09:44:53.556138+0000","last_became_active":"2024-09-16T09:33:30.245048+0000","last_became_peered":"2024-09-16T09:33:30.245048+0000","last_unstale":"2024-09-16T09:44:53.556138+0000","last_undegraded":"2024-09-16T09:44:53.556138+0000","last_fullsized":"2024-09-16T09:44:53.556138+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:41:36.612280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:41:36.612280+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:17.173178+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"285'20","reported_seq":1074,"reported_epoch":538,"state":"active+clean","last_fresh":"2024-09-16T09:44:54.559021+0000","last_change":"2024-09-16T09:39:35.550760+0000","last_active":"2024-09-16T09:44:54.559021+0000","last_peered":"2024-09-16T09:44:54.559021+0000","last_clean":"2024-09-16T09:44:54.559021+0000","last_became_active":"2024-09-16T09:31:09.275050+0000","last_became_peered":"2024-09-16T09:31:09.275050+0000","last_unstale":"2024-09-16T09:44:54.559021+0000","last_undegraded":"2024-09-16T09:44:54.559021+0000","last_fullsized":"2024-09-16T09:44:54.559021+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'20","last_scrub_stamp":"2024-09-16T09:39:35.548333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:39:35.548333+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:13.872588+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"66'8","reported_seq":1288,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.709379+0000","last_change":"2024-09-16T09:44:52.709379+0000","last_active":"2024-09-16T09:44:52.709379+0000","last_peered":"2024-09-16T09:44:52.709379+0000","last_clean":"2024-09-16T09:44:52.709379+0000","last_became_active":"2024-09-16T09:25:05.712516+0000","last_became_peered":"2024-09-16T09:25:05.712516+0000","last_unstale":"2024-09-16T09:44:52.709379+0000","last_undegraded":"2024-09-16T09:44:52.709379+0000","last_fullsized":"2024-09-16T09:44:52.709379+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:44:52.709313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:44:52.709313+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:00.116704+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.a","version":"73'79","reported_seq":955,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.552405+0000","last_change":"2024-09-16T09:44:30.838199+0000","last_active":"2024-09-16T09:44:52.552405+0000","last_peered":"2024-09-16T09:44:52.552405+0000","last_clean":"2024-09-16T09:44:52.552405+0000","last_became_active":"2024-09-16T09:44:24.278444+0000","last_became_peered":"2024-09-16T09:44:24.278444+0000","last_unstale":"2024-09-16T09:44:52.552405+0000","last_undegraded":"2024-09-16T09:44:52.552405+0000","last_fullsized":"2024-09-16T09:44:52.552405+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:44:30.838127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:44:30.838127+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:55.549500+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"285'27","reported_seq":948,"reported_epoch":535,"state":"active+clean","last_fresh":"2024-09-16T09:44:50.543453+0000","last_change":"2024-09-16T09:44:45.023239+0000","last_active":"2024-09-16T09:44:50.543453+0000","last_peered":"2024-09-16T09:44:50.543453+0000","last_clean":"2024-09-16T09:44:50.543453+0000","last_became_active":"2024-09-16T09:44:24.283121+0000","last_became_peered":"2024-09-16T09:44:24.283121+0000","last_unstale":"2024-09-16T09:44:50.543453+0000","last_undegraded":"2024-09-16T09:44:50.543453+0000","last_fullsized":"2024-09-16T09:44:50.543453+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'27","last_scrub_stamp":"2024-09-16T09:44:45.023173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:44:45.023173+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:12.611250+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":741,"reported_epoch":537,"state":"active+clean","last_fresh":"2024-09-16T09:44:53.556271+0000","last_change":"2024-09-16T09:38:35.023594+0000","last_active":"2024-09-16T09:44:53.556271+0000","last_peered":"2024-09-16T09:44:53.556271+0000","last_clean":"2024-09-16T09:44:53.556271+0000","last_became_active":"2024-09-16T09:31:09.274534+0000","last_became_peered":"2024-09-16T09:31:09.274534+0000","last_unstale":"2024-09-16T09:44:53.556271+0000","last_undegraded":"2024-09-16T09:44:53.556271+0000","last_fullsized":"2024-09-16T09:44:53.556271+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:38:35.023518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:38:35.023518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:13.168191+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"66'8","reported_seq":1244,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.704532+0000","last_change":"2024-09-16T09:44:50.326230+0000","last_active":"2024-09-16T09:44:52.704532+0000","last_peered":"2024-09-16T09:44:52.704532+0000","last_clean":"2024-09-16T09:44:52.704532+0000","last_became_active":"2024-09-16T09:25:05.713813+0000","last_became_peered":"2024-09-16T09:25:05.713813+0000","last_unstale":"2024-09-16T09:44:52.704532+0000","last_undegraded":"2024-09-16T09:44:52.704532+0000","last_fullsized":"2024-09-16T09:44:52.704532+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:44:50.326143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:44:50.326143+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:57.080144+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1008,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.552553+0000","last_change":"2024-09-16T09:44:02.749106+0000","last_active":"2024-09-16T09:44:52.552553+0000","last_peered":"2024-09-16T09:44:52.552553+0000","last_clean":"2024-09-16T09:44:52.552553+0000","last_became_active":"2024-09-16T09:25:07.717344+0000","last_became_peered":"2024-09-16T09:25:07.717344+0000","last_unstale":"2024-09-16T09:44:52.552553+0000","last_undegraded":"2024-09-16T09:44:52.552553+0000","last_fullsized":"2024-09-16T09:44:52.552553+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:02.748894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:44:02.748894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:29.230981+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.5","version":"73'79","reported_seq":987,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.704823+0000","last_change":"2024-09-16T09:44:27.405392+0000","last_active":"2024-09-16T09:44:52.704823+0000","last_peered":"2024-09-16T09:44:52.704823+0000","last_clean":"2024-09-16T09:44:52.704823+0000","last_became_active":"2024-09-16T09:44:24.276207+0000","last_became_peered":"2024-09-16T09:44:24.276207+0000","last_unstale":"2024-09-16T09:44:52.704823+0000","last_undegraded":"2024-09-16T09:44:52.704823+0000","last_fullsized":"2024-09-16T09:44:52.704823+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:44:27.405342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:44:27.405342+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:29.853871+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"285'30","reported_seq":973,"reported_epoch":535,"state":"active+clean","last_fresh":"2024-09-16T09:44:50.624818+0000","last_change":"2024-09-16T09:43:50.371697+0000","last_active":"2024-09-16T09:44:50.624818+0000","last_peered":"2024-09-16T09:44:50.624818+0000","last_clean":"2024-09-16T09:44:50.624818+0000","last_became_active":"2024-09-16T09:31:09.270434+0000","last_became_peered":"2024-09-16T09:31:09.270434+0000","last_unstale":"2024-09-16T09:44:50.624818+0000","last_undegraded":"2024-09-16T09:44:50.624818+0000","last_fullsized":"2024-09-16T09:44:50.624818+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'30","last_scrub_stamp":"2024-09-16T09:43:50.371625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:43:50.371625+0000","objects_scrubbed":8,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:03.401709+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":51,"num_read_kb":39,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":853,"reported_epoch":535,"state":"active+clean","last_fresh":"2024-09-16T09:44:50.625885+0000","last_change":"2024-09-16T09:44:00.511038+0000","last_active":"2024-09-16T09:44:50.625885+0000","last_peered":"2024-09-16T09:44:50.625885+0000","last_clean":"2024-09-16T09:44:50.625885+0000","last_became_active":"2024-09-16T09:28:52.983543+0000","last_became_peered":"2024-09-16T09:28:52.983543+0000","last_unstale":"2024-09-16T09:44:50.625885+0000","last_undegraded":"2024-09-16T09:44:50.625885+0000","last_fullsized":"2024-09-16T09:44:50.625885+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:00.510827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:44:00.510827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:29.877213+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1020,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.703640+0000","last_change":"2024-09-16T09:42:09.103716+0000","last_active":"2024-09-16T09:44:52.703640+0000","last_peered":"2024-09-16T09:44:52.703640+0000","last_clean":"2024-09-16T09:44:52.703640+0000","last_became_active":"2024-09-16T09:31:09.268864+0000","last_became_peered":"2024-09-16T09:31:09.268864+0000","last_unstale":"2024-09-16T09:44:52.703640+0000","last_undegraded":"2024-09-16T09:44:52.703640+0000","last_fullsized":"2024-09-16T09:44:52.703640+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:42:09.103657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:42:09.103657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1035,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.703998+0000","last_change":"2024-09-16T09:44:29.376932+0000","last_active":"2024-09-16T09:44:52.703998+0000","last_peered":"2024-09-16T09:44:52.703998+0000","last_clean":"2024-09-16T09:44:52.703998+0000","last_became_active":"2024-09-16T09:44:24.276365+0000","last_became_peered":"2024-09-16T09:44:24.276365+0000","last_unstale":"2024-09-16T09:44:52.703998+0000","last_undegraded":"2024-09-16T09:44:52.703998+0000","last_fullsized":"2024-09-16T09:44:52.703998+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:29.376874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:44:29.376874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:30.303288+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"73'79","reported_seq":749,"reported_epoch":537,"state":"active+clean","last_fresh":"2024-09-16T09:44:53.558286+0000","last_change":"2024-09-16T09:44:21.229944+0000","last_active":"2024-09-16T09:44:53.558286+0000","last_peered":"2024-09-16T09:44:53.558286+0000","last_clean":"2024-09-16T09:44:53.558286+0000","last_became_active":"2024-09-16T09:31:09.275139+0000","last_became_peered":"2024-09-16T09:31:09.275139+0000","last_unstale":"2024-09-16T09:44:53.558286+0000","last_undegraded":"2024-09-16T09:44:53.558286+0000","last_fullsized":"2024-09-16T09:44:53.558286+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:44:21.229868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:44:21.229868+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:39.951698+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"285'28","reported_seq":1060,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.703670+0000","last_change":"2024-09-16T09:43:18.126451+0000","last_active":"2024-09-16T09:44:52.703670+0000","last_peered":"2024-09-16T09:44:52.703670+0000","last_clean":"2024-09-16T09:44:52.703670+0000","last_became_active":"2024-09-16T09:25:03.708830+0000","last_became_peered":"2024-09-16T09:25:03.708830+0000","last_unstale":"2024-09-16T09:44:52.703670+0000","last_undegraded":"2024-09-16T09:44:52.703670+0000","last_fullsized":"2024-09-16T09:44:52.703670+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'28","last_scrub_stamp":"2024-09-16T09:43:18.126360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:43:18.126360+0000","objects_scrubbed":7,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":1035,"reported_epoch":538,"state":"active+clean","last_fresh":"2024-09-16T09:44:54.559303+0000","last_change":"2024-09-16T09:40:31.037325+0000","last_active":"2024-09-16T09:44:54.559303+0000","last_peered":"2024-09-16T09:44:54.559303+0000","last_clean":"2024-09-16T09:44:54.559303+0000","last_became_active":"2024-09-16T09:25:01.702992+0000","last_became_peered":"2024-09-16T09:25:01.702992+0000","last_unstale":"2024-09-16T09:44:54.559303+0000","last_undegraded":"2024-09-16T09:44:54.559303+0000","last_fullsized":"2024-09-16T09:44:54.559303+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:40:31.037258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:40:31.037258+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:41.195033+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"235'12","reported_seq":800,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.703443+0000","last_change":"2024-09-16T09:42:27.113239+0000","last_active":"2024-09-16T09:44:52.703443+0000","last_peered":"2024-09-16T09:44:52.703443+0000","last_clean":"2024-09-16T09:44:52.703443+0000","last_became_active":"2024-09-16T09:33:30.251047+0000","last_became_peered":"2024-09-16T09:33:30.251047+0000","last_unstale":"2024-09-16T09:44:52.703443+0000","last_undegraded":"2024-09-16T09:44:52.703443+0000","last_fullsized":"2024-09-16T09:44:52.703443+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"235'12","last_scrub_stamp":"2024-09-16T09:42:27.113174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:42:27.113174+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1008,"reported_epoch":535,"state":"active+clean","last_fresh":"2024-09-16T09:44:50.625126+0000","last_change":"2024-09-16T09:44:29.455025+0000","last_active":"2024-09-16T09:44:50.625126+0000","last_peered":"2024-09-16T09:44:50.625126+0000","last_clean":"2024-09-16T09:44:50.625126+0000","last_became_active":"2024-09-16T09:44:24.277003+0000","last_became_peered":"2024-09-16T09:44:24.277003+0000","last_unstale":"2024-09-16T09:44:50.625126+0000","last_undegraded":"2024-09-16T09:44:50.625126+0000","last_fullsized":"2024-09-16T09:44:50.625126+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:29.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:44:29.454958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:30.251581+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"73'79","reported_seq":1086,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.551733+0000","last_change":"2024-09-16T09:43:55.930749+0000","last_active":"2024-09-16T09:44:52.551733+0000","last_peered":"2024-09-16T09:44:52.551733+0000","last_clean":"2024-09-16T09:44:52.551733+0000","last_became_active":"2024-09-16T09:31:09.274707+0000","last_became_peered":"2024-09-16T09:31:09.274707+0000","last_unstale":"2024-09-16T09:44:52.551733+0000","last_undegraded":"2024-09-16T09:44:52.551733+0000","last_fullsized":"2024-09-16T09:44:52.551733+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:43:55.930694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:43:55.930694+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:15.421892+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"285'20","reported_seq":687,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.553039+0000","last_change":"2024-09-16T09:38:32.122419+0000","last_active":"2024-09-16T09:44:52.553039+0000","last_peered":"2024-09-16T09:44:52.553039+0000","last_clean":"2024-09-16T09:44:52.553039+0000","last_became_active":"2024-09-16T09:33:30.248453+0000","last_became_peered":"2024-09-16T09:33:30.248453+0000","last_unstale":"2024-09-16T09:44:52.553039+0000","last_undegraded":"2024-09-16T09:44:52.553039+0000","last_fullsized":"2024-09-16T09:44:52.553039+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'20","last_scrub_stamp":"2024-09-16T09:38:32.122303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:38:32.122303+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:08.800937+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":1030,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.552882+0000","last_change":"2024-09-16T09:44:25.924933+0000","last_active":"2024-09-16T09:44:52.552882+0000","last_peered":"2024-09-16T09:44:52.552882+0000","last_clean":"2024-09-16T09:44:52.552882+0000","last_became_active":"2024-09-16T09:44:24.280657+0000","last_became_peered":"2024-09-16T09:44:24.280657+0000","last_unstale":"2024-09-16T09:44:52.552882+0000","last_undegraded":"2024-09-16T09:44:52.552882+0000","last_fullsized":"2024-09-16T09:44:52.552882+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:25.924858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:44:25.924858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:35.232847+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"66'8","reported_seq":1257,"reported_epoch":538,"state":"active+clean","last_fresh":"2024-09-16T09:44:54.559754+0000","last_change":"2024-09-16T09:44:00.808193+0000","last_active":"2024-09-16T09:44:54.559754+0000","last_peered":"2024-09-16T09:44:54.559754+0000","last_clean":"2024-09-16T09:44:54.559754+0000","last_became_active":"2024-09-16T09:25:05.714507+0000","last_became_peered":"2024-09-16T09:25:05.714507+0000","last_unstale":"2024-09-16T09:44:54.559754+0000","last_undegraded":"2024-09-16T09:44:54.559754+0000","last_fullsized":"2024-09-16T09:44:54.559754+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:44:00.808121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:44:00.808121+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:28.025271+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1039,"reported_epoch":538,"state":"active+clean","last_fresh":"2024-09-16T09:44:54.558017+0000","last_change":"2024-09-16T09:44:25.125306+0000","last_active":"2024-09-16T09:44:54.558017+0000","last_peered":"2024-09-16T09:44:54.558017+0000","last_clean":"2024-09-16T09:44:54.558017+0000","last_became_active":"2024-09-16T09:44:24.269736+0000","last_became_peered":"2024-09-16T09:44:24.269736+0000","last_unstale":"2024-09-16T09:44:54.558017+0000","last_undegraded":"2024-09-16T09:44:54.558017+0000","last_fullsized":"2024-09-16T09:44:54.558017+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:25.125254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:44:25.125254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:42.194714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"73'79","reported_seq":1043,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.551667+0000","last_change":"2024-09-16T09:43:50.904918+0000","last_active":"2024-09-16T09:44:52.551667+0000","last_peered":"2024-09-16T09:44:52.551667+0000","last_clean":"2024-09-16T09:44:52.551667+0000","last_became_active":"2024-09-16T09:31:09.274565+0000","last_became_peered":"2024-09-16T09:31:09.274565+0000","last_unstale":"2024-09-16T09:44:52.551667+0000","last_undegraded":"2024-09-16T09:44:52.551667+0000","last_fullsized":"2024-09-16T09:44:52.551667+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:43:50.904841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:43:50.904841+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:09.506473+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"285'29","reported_seq":1057,"reported_epoch":538,"state":"active+clean","last_fresh":"2024-09-16T09:44:54.559971+0000","last_change":"2024-09-16T09:42:24.900527+0000","last_active":"2024-09-16T09:44:54.559971+0000","last_peered":"2024-09-16T09:44:54.559971+0000","last_clean":"2024-09-16T09:44:54.559971+0000","last_became_active":"2024-09-16T09:25:03.714797+0000","last_became_peered":"2024-09-16T09:25:03.714797+0000","last_unstale":"2024-09-16T09:44:54.559971+0000","last_undegraded":"2024-09-16T09:44:54.559971+0000","last_fullsized":"2024-09-16T09:44:54.559971+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'29","last_scrub_stamp":"2024-09-16T09:42:24.900426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:42:24.900426+0000","objects_scrubbed":8,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":54,"num_read_kb":40,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":648,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.551561+0000","last_change":"2024-09-16T09:44:27.885527+0000","last_active":"2024-09-16T09:44:52.551561+0000","last_peered":"2024-09-16T09:44:52.551561+0000","last_clean":"2024-09-16T09:44:52.551561+0000","last_became_active":"2024-09-16T09:44:24.279029+0000","last_became_peered":"2024-09-16T09:44:24.279029+0000","last_unstale":"2024-09-16T09:44:52.551561+0000","last_undegraded":"2024-09-16T09:44:52.551561+0000","last_fullsized":"2024-09-16T09:44:52.551561+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:27.885460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:44:27.885460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:50.933082+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,2],"acting":[5,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1030,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.551842+0000","last_change":"2024-09-16T09:44:51.704537+0000","last_active":"2024-09-16T09:44:52.551842+0000","last_peered":"2024-09-16T09:44:52.551842+0000","last_clean":"2024-09-16T09:44:52.551842+0000","last_became_active":"2024-09-16T09:25:05.708663+0000","last_became_peered":"2024-09-16T09:25:05.708663+0000","last_unstale":"2024-09-16T09:44:52.551842+0000","last_undegraded":"2024-09-16T09:44:52.551842+0000","last_fullsized":"2024-09-16T09:44:52.551842+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:51.704482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:44:51.704482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:16.177847+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":656,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.551356+0000","last_change":"2024-09-16T09:44:25.629415+0000","last_active":"2024-09-16T09:44:52.551356+0000","last_peered":"2024-09-16T09:44:52.551356+0000","last_clean":"2024-09-16T09:44:52.551356+0000","last_became_active":"2024-09-16T09:44:24.277522+0000","last_became_peered":"2024-09-16T09:44:24.277522+0000","last_unstale":"2024-09-16T09:44:52.551356+0000","last_undegraded":"2024-09-16T09:44:52.551356+0000","last_fullsized":"2024-09-16T09:44:52.551356+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:25.629370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:44:25.629370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:54.323231+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,2],"acting":[6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"73'79","reported_seq":987,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.550993+0000","last_change":"2024-09-16T09:44:29.831921+0000","last_active":"2024-09-16T09:44:52.550993+0000","last_peered":"2024-09-16T09:44:52.550993+0000","last_clean":"2024-09-16T09:44:52.550993+0000","last_became_active":"2024-09-16T09:44:24.279593+0000","last_became_peered":"2024-09-16T09:44:24.279593+0000","last_unstale":"2024-09-16T09:44:52.550993+0000","last_undegraded":"2024-09-16T09:44:52.550993+0000","last_fullsized":"2024-09-16T09:44:52.550993+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:44:29.831857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:44:29.831857+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:59.570916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"519'44","reported_seq":1412,"reported_epoch":537,"state":"active+clean","last_fresh":"2024-09-16T09:44:53.557707+0000","last_change":"2024-09-16T09:44:30.314706+0000","last_active":"2024-09-16T09:44:53.557707+0000","last_peered":"2024-09-16T09:44:53.557707+0000","last_clean":"2024-09-16T09:44:53.557707+0000","last_became_active":"2024-09-16T09:44:24.280130+0000","last_became_peered":"2024-09-16T09:44:24.280130+0000","last_unstale":"2024-09-16T09:44:53.557707+0000","last_undegraded":"2024-09-16T09:44:53.557707+0000","last_fullsized":"2024-09-16T09:44:53.557707+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"519'44","last_scrub_stamp":"2024-09-16T09:44:30.314606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:44:30.314606+0000","objects_scrubbed":10,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:34.762411+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":66,"num_read_kb":51,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":1022,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.704277+0000","last_change":"2024-09-16T09:41:40.124915+0000","last_active":"2024-09-16T09:44:52.704277+0000","last_peered":"2024-09-16T09:44:52.704277+0000","last_clean":"2024-09-16T09:44:52.704277+0000","last_became_active":"2024-09-16T09:25:01.703076+0000","last_became_peered":"2024-09-16T09:25:01.703076+0000","last_unstale":"2024-09-16T09:44:52.704277+0000","last_undegraded":"2024-09-16T09:44:52.704277+0000","last_fullsized":"2024-09-16T09:44:52.704277+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:40.124830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:41:40.124830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:07.248004+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":1031,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.553115+0000","last_change":"2024-09-16T09:43:52.704491+0000","last_active":"2024-09-16T09:44:52.553115+0000","last_peered":"2024-09-16T09:44:52.553115+0000","last_clean":"2024-09-16T09:44:52.553115+0000","last_became_active":"2024-09-16T09:25:00.696237+0000","last_became_peered":"2024-09-16T09:25:00.696237+0000","last_unstale":"2024-09-16T09:44:52.553115+0000","last_undegraded":"2024-09-16T09:44:52.553115+0000","last_fullsized":"2024-09-16T09:44:52.553115+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-16T09:43:52.704431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:43:52.704431+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:20.772940+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.0025137430000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1002,"reported_epoch":538,"state":"active+clean","last_fresh":"2024-09-16T09:44:54.558190+0000","last_change":"2024-09-16T09:44:03.793464+0000","last_active":"2024-09-16T09:44:54.558190+0000","last_peered":"2024-09-16T09:44:54.558190+0000","last_clean":"2024-09-16T09:44:54.558190+0000","last_became_active":"2024-09-16T09:25:05.717591+0000","last_became_peered":"2024-09-16T09:25:05.717591+0000","last_unstale":"2024-09-16T09:44:54.558190+0000","last_undegraded":"2024-09-16T09:44:54.558190+0000","last_fullsized":"2024-09-16T09:44:54.558190+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:03.793400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:44:03.793400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:29.631956+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":749,"reported_epoch":537,"state":"active+clean","last_fresh":"2024-09-16T09:44:53.558528+0000","last_change":"2024-09-16T09:44:35.215569+0000","last_active":"2024-09-16T09:44:53.558528+0000","last_peered":"2024-09-16T09:44:53.558528+0000","last_clean":"2024-09-16T09:44:53.558528+0000","last_became_active":"2024-09-16T09:31:09.272609+0000","last_became_peered":"2024-09-16T09:31:09.272609+0000","last_unstale":"2024-09-16T09:44:53.558528+0000","last_undegraded":"2024-09-16T09:44:53.558528+0000","last_fullsized":"2024-09-16T09:44:53.558528+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:35.215527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:44:35.215527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:03.125091+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"73'79","reported_seq":341,"reported_epoch":535,"state":"active+clean","last_fresh":"2024-09-16T09:44:50.624617+0000","last_change":"2024-09-16T09:44:22.398722+0000","last_active":"2024-09-16T09:44:50.624617+0000","last_peered":"2024-09-16T09:44:50.624617+0000","last_clean":"2024-09-16T09:44:50.624617+0000","last_became_active":"2024-09-16T09:38:10.556320+0000","last_became_peered":"2024-09-16T09:38:10.556320+0000","last_unstale":"2024-09-16T09:44:50.624617+0000","last_undegraded":"2024-09-16T09:44:50.624617+0000","last_fullsized":"2024-09-16T09:44:50.624617+0000","mapping_epoch":386,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":387,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:44:22.398656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:44:22.398656+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:28.557065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"285'17","reported_seq":736,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.553063+0000","last_change":"2024-09-16T09:43:49.706772+0000","last_active":"2024-09-16T09:44:52.553063+0000","last_peered":"2024-09-16T09:44:52.553063+0000","last_clean":"2024-09-16T09:44:52.553063+0000","last_became_active":"2024-09-16T09:37:33.349009+0000","last_became_peered":"2024-09-16T09:37:33.349009+0000","last_unstale":"2024-09-16T09:44:52.553063+0000","last_undegraded":"2024-09-16T09:44:52.553063+0000","last_fullsized":"2024-09-16T09:44:52.553063+0000","mapping_epoch":367,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":368,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'17","last_scrub_stamp":"2024-09-16T09:43:49.706685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:43:49.706685+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:44:55.741272+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":35,"num_read_kb":24,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":656,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.551253+0000","last_change":"2024-09-16T09:44:30.666378+0000","last_active":"2024-09-16T09:44:52.551253+0000","last_peered":"2024-09-16T09:44:52.551253+0000","last_clean":"2024-09-16T09:44:52.551253+0000","last_became_active":"2024-09-16T09:44:24.270134+0000","last_became_peered":"2024-09-16T09:44:24.270134+0000","last_unstale":"2024-09-16T09:44:52.551253+0000","last_undegraded":"2024-09-16T09:44:52.551253+0000","last_fullsized":"2024-09-16T09:44:52.551253+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:30.666333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:44:30.666333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:36.645744+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,2],"acting":[6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":698,"reported_epoch":538,"state":"active+clean","last_fresh":"2024-09-16T09:44:54.560118+0000","last_change":"2024-09-16T09:44:24.280992+0000","last_active":"2024-09-16T09:44:54.560118+0000","last_peered":"2024-09-16T09:44:54.560118+0000","last_clean":"2024-09-16T09:44:54.560118+0000","last_became_active":"2024-09-16T09:44:24.279158+0000","last_became_peered":"2024-09-16T09:44:24.279158+0000","last_unstale":"2024-09-16T09:44:54.560118+0000","last_undegraded":"2024-09-16T09:44:54.560118+0000","last_fullsized":"2024-09-16T09:44:54.560118+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:41:42.038861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:41:42.038861+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":938,"reported_epoch":535,"state":"active+clean","last_fresh":"2024-09-16T09:44:50.543486+0000","last_change":"2024-09-16T09:44:24.286071+0000","last_active":"2024-09-16T09:44:50.543486+0000","last_peered":"2024-09-16T09:44:50.543486+0000","last_clean":"2024-09-16T09:44:50.543486+0000","last_became_active":"2024-09-16T09:44:24.285805+0000","last_became_peered":"2024-09-16T09:44:24.285805+0000","last_unstale":"2024-09-16T09:44:50.543486+0000","last_undegraded":"2024-09-16T09:44:50.543486+0000","last_fullsized":"2024-09-16T09:44:50.543486+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:45.549186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:41:45.549186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:39.079232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1005,"reported_epoch":535,"state":"active+clean","last_fresh":"2024-09-16T09:44:50.625387+0000","last_change":"2024-09-16T09:44:48.555675+0000","last_active":"2024-09-16T09:44:50.625387+0000","last_peered":"2024-09-16T09:44:50.625387+0000","last_clean":"2024-09-16T09:44:50.625387+0000","last_became_active":"2024-09-16T09:25:07.723282+0000","last_became_peered":"2024-09-16T09:25:07.723282+0000","last_unstale":"2024-09-16T09:44:50.625387+0000","last_undegraded":"2024-09-16T09:44:50.625387+0000","last_fullsized":"2024-09-16T09:44:50.625387+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:48.555614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:44:48.555614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:18.258254+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"73'79","reported_seq":726,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.551561+0000","last_change":"2024-09-16T09:43:53.953716+0000","last_active":"2024-09-16T09:44:52.551561+0000","last_peered":"2024-09-16T09:44:52.551561+0000","last_clean":"2024-09-16T09:44:52.551561+0000","last_became_active":"2024-09-16T09:33:30.249309+0000","last_became_peered":"2024-09-16T09:33:30.249309+0000","last_unstale":"2024-09-16T09:44:52.551561+0000","last_undegraded":"2024-09-16T09:44:52.551561+0000","last_fullsized":"2024-09-16T09:44:52.551561+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:43:53.953557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:43:53.953557+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:12.516809+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.6","version":"285'24","reported_seq":1061,"reported_epoch":538,"state":"active+clean","last_fresh":"2024-09-16T09:44:54.559123+0000","last_change":"2024-09-16T09:44:24.278226+0000","last_active":"2024-09-16T09:44:54.559123+0000","last_peered":"2024-09-16T09:44:54.559123+0000","last_clean":"2024-09-16T09:44:54.559123+0000","last_became_active":"2024-09-16T09:44:24.277878+0000","last_became_peered":"2024-09-16T09:44:24.277878+0000","last_unstale":"2024-09-16T09:44:54.559123+0000","last_undegraded":"2024-09-16T09:44:54.559123+0000","last_fullsized":"2024-09-16T09:44:54.559123+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'24","last_scrub_stamp":"2024-09-16T09:42:11.911889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:42:11.911889+0000","objects_scrubbed":6,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":36,"num_read_kb":30,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"59'1","reported_seq":1049,"reported_epoch":538,"state":"active+clean","last_fresh":"2024-09-16T09:44:54.559845+0000","last_change":"2024-09-16T09:42:56.896466+0000","last_active":"2024-09-16T09:44:54.559845+0000","last_peered":"2024-09-16T09:44:54.559845+0000","last_clean":"2024-09-16T09:44:54.559845+0000","last_became_active":"2024-09-16T09:31:09.269627+0000","last_became_peered":"2024-09-16T09:31:09.269627+0000","last_unstale":"2024-09-16T09:44:54.559845+0000","last_undegraded":"2024-09-16T09:44:54.559845+0000","last_fullsized":"2024-09-16T09:44:54.559845+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:42:56.896256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:42:56.896256+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1986,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"67'5","reported_seq":2203,"reported_epoch":535,"state":"active+clean","last_fresh":"2024-09-16T09:44:50.740440+0000","last_change":"2024-09-16T09:41:43.153303+0000","last_active":"2024-09-16T09:44:50.740440+0000","last_peered":"2024-09-16T09:44:50.740440+0000","last_clean":"2024-09-16T09:44:50.740440+0000","last_became_active":"2024-09-16T09:25:00.696546+0000","last_became_peered":"2024-09-16T09:25:00.696546+0000","last_unstale":"2024-09-16T09:44:50.740440+0000","last_undegraded":"2024-09-16T09:44:50.740440+0000","last_fullsized":"2024-09-16T09:44:50.740440+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'5","last_scrub_stamp":"2024-09-16T09:41:43.153174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:41:43.153174+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:42.518308+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0.0059770420000000001,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1183,"num_read_kb":1178,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":989,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.553171+0000","last_change":"2024-09-16T09:44:19.980692+0000","last_active":"2024-09-16T09:44:52.553171+0000","last_peered":"2024-09-16T09:44:52.553171+0000","last_clean":"2024-09-16T09:44:52.553171+0000","last_became_active":"2024-09-16T09:25:05.714704+0000","last_became_peered":"2024-09-16T09:25:05.714704+0000","last_unstale":"2024-09-16T09:44:52.553171+0000","last_undegraded":"2024-09-16T09:44:52.553171+0000","last_fullsized":"2024-09-16T09:44:52.553171+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:19.980625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:44:19.980625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:20.184735+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":923,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.551800+0000","last_change":"2024-09-16T09:44:04.828342+0000","last_active":"2024-09-16T09:44:52.551800+0000","last_peered":"2024-09-16T09:44:52.551800+0000","last_clean":"2024-09-16T09:44:52.551800+0000","last_became_active":"2024-09-16T09:33:30.245852+0000","last_became_peered":"2024-09-16T09:33:30.245852+0000","last_unstale":"2024-09-16T09:44:52.551800+0000","last_undegraded":"2024-09-16T09:44:52.551800+0000","last_fullsized":"2024-09-16T09:44:52.551800+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:04.828272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:44:04.828272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:28.332272+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.5","version":"285'28","reported_seq":1297,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.552570+0000","last_change":"2024-09-16T09:43:47.926939+0000","last_active":"2024-09-16T09:44:52.552570+0000","last_peered":"2024-09-16T09:44:52.552570+0000","last_clean":"2024-09-16T09:44:52.552570+0000","last_became_active":"2024-09-16T09:25:03.714118+0000","last_became_peered":"2024-09-16T09:25:03.714118+0000","last_unstale":"2024-09-16T09:44:52.552570+0000","last_undegraded":"2024-09-16T09:44:52.552570+0000","last_fullsized":"2024-09-16T09:44:52.552570+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'28","last_scrub_stamp":"2024-09-16T09:43:47.926855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:43:47.926855+0000","objects_scrubbed":8,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:44:54.826374+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":43,"num_read_kb":33,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":863,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.553047+0000","last_change":"2024-09-16T09:44:34.758789+0000","last_active":"2024-09-16T09:44:52.553047+0000","last_peered":"2024-09-16T09:44:52.553047+0000","last_clean":"2024-09-16T09:44:52.553047+0000","last_became_active":"2024-09-16T09:31:09.270196+0000","last_became_peered":"2024-09-16T09:31:09.270196+0000","last_unstale":"2024-09-16T09:44:52.553047+0000","last_undegraded":"2024-09-16T09:44:52.553047+0000","last_fullsized":"2024-09-16T09:44:52.553047+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:34.758733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:44:34.758733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:37.220493+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"73'79","reported_seq":1089,"reported_epoch":535,"state":"active+clean","last_fresh":"2024-09-16T09:44:50.625068+0000","last_change":"2024-09-16T09:42:56.661708+0000","last_active":"2024-09-16T09:44:50.625068+0000","last_peered":"2024-09-16T09:44:50.625068+0000","last_clean":"2024-09-16T09:44:50.625068+0000","last_became_active":"2024-09-16T09:30:08.870394+0000","last_became_peered":"2024-09-16T09:30:08.870394+0000","last_unstale":"2024-09-16T09:44:50.625068+0000","last_undegraded":"2024-09-16T09:44:50.625068+0000","last_fullsized":"2024-09-16T09:44:50.625068+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:42:56.661377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:42:56.661377+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1012,"reported_epoch":535,"state":"active+clean","last_fresh":"2024-09-16T09:44:50.625564+0000","last_change":"2024-09-16T09:44:24.285202+0000","last_active":"2024-09-16T09:44:50.625564+0000","last_peered":"2024-09-16T09:44:50.625564+0000","last_clean":"2024-09-16T09:44:50.625564+0000","last_became_active":"2024-09-16T09:44:24.284362+0000","last_became_peered":"2024-09-16T09:44:24.284362+0000","last_unstale":"2024-09-16T09:44:50.625564+0000","last_undegraded":"2024-09-16T09:44:50.625564+0000","last_fullsized":"2024-09-16T09:44:50.625564+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:42:11.246697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:42:11.246697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1001,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.552874+0000","last_change":"2024-09-16T09:44:46.680178+0000","last_active":"2024-09-16T09:44:52.552874+0000","last_peered":"2024-09-16T09:44:52.552874+0000","last_clean":"2024-09-16T09:44:52.552874+0000","last_became_active":"2024-09-16T09:25:07.714281+0000","last_became_peered":"2024-09-16T09:25:07.714281+0000","last_unstale":"2024-09-16T09:44:52.552874+0000","last_undegraded":"2024-09-16T09:44:52.552874+0000","last_fullsized":"2024-09-16T09:44:52.552874+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:46.680120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:44:46.680120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:08.327463+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.b","version":"73'79","reported_seq":841,"reported_epoch":538,"state":"active+clean","last_fresh":"2024-09-16T09:44:54.559401+0000","last_change":"2024-09-16T09:42:26.898435+0000","last_active":"2024-09-16T09:44:54.559401+0000","last_peered":"2024-09-16T09:44:54.559401+0000","last_clean":"2024-09-16T09:44:54.559401+0000","last_became_active":"2024-09-16T09:33:30.251933+0000","last_became_peered":"2024-09-16T09:33:30.251933+0000","last_unstale":"2024-09-16T09:44:54.559401+0000","last_undegraded":"2024-09-16T09:44:54.559401+0000","last_fullsized":"2024-09-16T09:44:54.559401+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:42:26.898235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:42:26.898235+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"285'19","reported_seq":1089,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.704597+0000","last_change":"2024-09-16T09:43:22.166919+0000","last_active":"2024-09-16T09:44:52.704597+0000","last_peered":"2024-09-16T09:44:52.704597+0000","last_clean":"2024-09-16T09:44:52.704597+0000","last_became_active":"2024-09-16T09:37:33.348433+0000","last_became_peered":"2024-09-16T09:37:33.348433+0000","last_unstale":"2024-09-16T09:44:52.704597+0000","last_undegraded":"2024-09-16T09:44:52.704597+0000","last_fullsized":"2024-09-16T09:44:52.704597+0000","mapping_epoch":367,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":368,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'19","last_scrub_stamp":"2024-09-16T09:43:22.166834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:43:22.166834+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":1009,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.704031+0000","last_change":"2024-09-16T09:44:24.277287+0000","last_active":"2024-09-16T09:44:52.704031+0000","last_peered":"2024-09-16T09:44:52.704031+0000","last_clean":"2024-09-16T09:44:52.704031+0000","last_became_active":"2024-09-16T09:44:24.276997+0000","last_became_peered":"2024-09-16T09:44:24.276997+0000","last_unstale":"2024-09-16T09:44:52.704031+0000","last_undegraded":"2024-09-16T09:44:52.704031+0000","last_fullsized":"2024-09-16T09:44:52.704031+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:42:18.132091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:42:18.132091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":980,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.552738+0000","last_change":"2024-09-16T09:42:18.730769+0000","last_active":"2024-09-16T09:44:52.552738+0000","last_peered":"2024-09-16T09:44:52.552738+0000","last_clean":"2024-09-16T09:44:52.552738+0000","last_became_active":"2024-09-16T09:25:05.711964+0000","last_became_peered":"2024-09-16T09:25:05.711964+0000","last_unstale":"2024-09-16T09:44:52.552738+0000","last_undegraded":"2024-09-16T09:44:52.552738+0000","last_fullsized":"2024-09-16T09:44:52.552738+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:42:18.730723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:42:18.730723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:14.838460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":883,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.551739+0000","last_change":"2024-09-16T09:44:37.625415+0000","last_active":"2024-09-16T09:44:52.551739+0000","last_peered":"2024-09-16T09:44:52.551739+0000","last_clean":"2024-09-16T09:44:52.551739+0000","last_became_active":"2024-09-16T09:31:09.278618+0000","last_became_peered":"2024-09-16T09:31:09.278618+0000","last_unstale":"2024-09-16T09:44:52.551739+0000","last_undegraded":"2024-09-16T09:44:52.551739+0000","last_fullsized":"2024-09-16T09:44:52.551739+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:42:09.944311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:42:09.944311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:06.727839+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.8","version":"73'79","reported_seq":359,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.550888+0000","last_change":"2024-09-16T09:44:31.814201+0000","last_active":"2024-09-16T09:44:52.550888+0000","last_peered":"2024-09-16T09:44:52.550888+0000","last_clean":"2024-09-16T09:44:52.550888+0000","last_became_active":"2024-09-16T09:44:24.270354+0000","last_became_peered":"2024-09-16T09:44:24.270354+0000","last_unstale":"2024-09-16T09:44:52.550888+0000","last_undegraded":"2024-09-16T09:44:52.550888+0000","last_fullsized":"2024-09-16T09:44:52.550888+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:44:31.814152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:44:31.814152+0000","objects_scrubbed":1,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:39.578957+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262688,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"285'31","reported_seq":1069,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.704687+0000","last_change":"2024-09-16T09:44:24.277703+0000","last_active":"2024-09-16T09:44:52.704687+0000","last_peered":"2024-09-16T09:44:52.704687+0000","last_clean":"2024-09-16T09:44:52.704687+0000","last_became_active":"2024-09-16T09:44:24.277359+0000","last_became_peered":"2024-09-16T09:44:24.277359+0000","last_unstale":"2024-09-16T09:44:52.704687+0000","last_undegraded":"2024-09-16T09:44:52.704687+0000","last_fullsized":"2024-09-16T09:44:52.704687+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'31","last_scrub_stamp":"2024-09-16T09:42:07.153276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:42:07.153276+0000","objects_scrubbed":8,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":50,"num_read_kb":40,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":1014,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.552232+0000","last_change":"2024-09-16T09:42:53.627570+0000","last_active":"2024-09-16T09:44:52.552232+0000","last_peered":"2024-09-16T09:44:52.552232+0000","last_clean":"2024-09-16T09:44:52.552232+0000","last_became_active":"2024-09-16T09:25:01.703355+0000","last_became_peered":"2024-09-16T09:25:01.703355+0000","last_unstale":"2024-09-16T09:44:52.552232+0000","last_undegraded":"2024-09-16T09:44:52.552232+0000","last_fullsized":"2024-09-16T09:44:52.552232+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:42:53.627505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:42:53.627505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:07.742399+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":645,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.703898+0000","last_change":"2024-09-16T09:44:47.324175+0000","last_active":"2024-09-16T09:44:52.703898+0000","last_peered":"2024-09-16T09:44:52.703898+0000","last_clean":"2024-09-16T09:44:52.703898+0000","last_became_active":"2024-09-16T09:33:30.251186+0000","last_became_peered":"2024-09-16T09:33:30.251186+0000","last_unstale":"2024-09-16T09:44:52.703898+0000","last_undegraded":"2024-09-16T09:44:52.703898+0000","last_fullsized":"2024-09-16T09:44:52.703898+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:47.324096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:44:47.324096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:02.767003+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,0],"acting":[4,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":956,"reported_epoch":537,"state":"active+clean","last_fresh":"2024-09-16T09:44:53.558407+0000","last_change":"2024-09-16T09:44:25.180283+0000","last_active":"2024-09-16T09:44:53.558407+0000","last_peered":"2024-09-16T09:44:53.558407+0000","last_clean":"2024-09-16T09:44:53.558407+0000","last_became_active":"2024-09-16T09:44:24.273561+0000","last_became_peered":"2024-09-16T09:44:24.273561+0000","last_unstale":"2024-09-16T09:44:53.558407+0000","last_undegraded":"2024-09-16T09:44:53.558407+0000","last_fullsized":"2024-09-16T09:44:53.558407+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:25.180239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:44:25.180239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:44.445043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4],"acting":[3,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.9","version":"73'79","reported_seq":1076,"reported_epoch":535,"state":"active+clean","last_fresh":"2024-09-16T09:44:50.624997+0000","last_change":"2024-09-16T09:44:24.279925+0000","last_active":"2024-09-16T09:44:50.624997+0000","last_peered":"2024-09-16T09:44:50.624997+0000","last_clean":"2024-09-16T09:44:50.624997+0000","last_became_active":"2024-09-16T09:44:24.278896+0000","last_became_peered":"2024-09-16T09:44:24.278896+0000","last_unstale":"2024-09-16T09:44:50.624997+0000","last_undegraded":"2024-09-16T09:44:50.624997+0000","last_fullsized":"2024-09-16T09:44:50.624997+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:41:40.123937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:41:40.123937+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"285'20","reported_seq":1031,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.704151+0000","last_change":"2024-09-16T09:43:49.220241+0000","last_active":"2024-09-16T09:44:52.704151+0000","last_peered":"2024-09-16T09:44:52.704151+0000","last_clean":"2024-09-16T09:44:52.704151+0000","last_became_active":"2024-09-16T09:25:03.707723+0000","last_became_peered":"2024-09-16T09:25:03.707723+0000","last_unstale":"2024-09-16T09:44:52.704151+0000","last_undegraded":"2024-09-16T09:44:52.704151+0000","last_fullsized":"2024-09-16T09:44:52.704151+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'20","last_scrub_stamp":"2024-09-16T09:43:49.220175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:43:49.220175+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:18.570377+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":875,"reported_epoch":538,"state":"active+clean","last_fresh":"2024-09-16T09:44:54.560071+0000","last_change":"2024-09-16T09:42:52.868096+0000","last_active":"2024-09-16T09:44:54.560071+0000","last_peered":"2024-09-16T09:44:54.560071+0000","last_clean":"2024-09-16T09:44:54.560071+0000","last_became_active":"2024-09-16T09:31:09.288825+0000","last_became_peered":"2024-09-16T09:31:09.288825+0000","last_unstale":"2024-09-16T09:44:54.560071+0000","last_undegraded":"2024-09-16T09:44:54.560071+0000","last_fullsized":"2024-09-16T09:44:54.560071+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:42:52.868035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:42:52.868035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"519'11","reported_seq":834,"reported_epoch":535,"state":"active+clean","last_fresh":"2024-09-16T09:44:50.625604+0000","last_change":"2024-09-16T09:44:24.278535+0000","last_active":"2024-09-16T09:44:50.625604+0000","last_peered":"2024-09-16T09:44:50.625604+0000","last_clean":"2024-09-16T09:44:50.625604+0000","last_became_active":"2024-09-16T09:44:24.277662+0000","last_became_peered":"2024-09-16T09:44:24.277662+0000","last_unstale":"2024-09-16T09:44:50.625604+0000","last_undegraded":"2024-09-16T09:44:50.625604+0000","last_fullsized":"2024-09-16T09:44:50.625604+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"228'9","last_scrub_stamp":"2024-09-16T09:41:33.174444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:41:33.174444+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:40.473027+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1036,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.704342+0000","last_change":"2024-09-16T09:44:48.293840+0000","last_active":"2024-09-16T09:44:52.704342+0000","last_peered":"2024-09-16T09:44:52.704342+0000","last_clean":"2024-09-16T09:44:52.704342+0000","last_became_active":"2024-09-16T09:44:24.280724+0000","last_became_peered":"2024-09-16T09:44:24.280724+0000","last_unstale":"2024-09-16T09:44:52.704342+0000","last_undegraded":"2024-09-16T09:44:52.704342+0000","last_fullsized":"2024-09-16T09:44:52.704342+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:48.293789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:44:48.293789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:57.968474+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.e","version":"73'79","reported_seq":802,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.704339+0000","last_change":"2024-09-16T09:42:16.139285+0000","last_active":"2024-09-16T09:44:52.704339+0000","last_peered":"2024-09-16T09:44:52.704339+0000","last_clean":"2024-09-16T09:44:52.704339+0000","last_became_active":"2024-09-16T09:30:10.705072+0000","last_became_peered":"2024-09-16T09:30:10.705072+0000","last_unstale":"2024-09-16T09:44:52.704339+0000","last_undegraded":"2024-09-16T09:44:52.704339+0000","last_fullsized":"2024-09-16T09:44:52.704339+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:42:16.139208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:42:16.139208+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,3],"acting":[4,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.b","version":"285'15","reported_seq":1063,"reported_epoch":538,"state":"active+clean","last_fresh":"2024-09-16T09:44:54.559889+0000","last_change":"2024-09-16T09:44:48.083967+0000","last_active":"2024-09-16T09:44:54.559889+0000","last_peered":"2024-09-16T09:44:54.559889+0000","last_clean":"2024-09-16T09:44:54.559889+0000","last_became_active":"2024-09-16T09:36:31.590202+0000","last_became_peered":"2024-09-16T09:36:31.590202+0000","last_unstale":"2024-09-16T09:44:54.559889+0000","last_undegraded":"2024-09-16T09:44:54.559889+0000","last_fullsized":"2024-09-16T09:44:54.559889+0000","mapping_epoch":316,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":317,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'15","last_scrub_stamp":"2024-09-16T09:44:48.083895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:44:48.083895+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:50.498247+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":1010,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.551892+0000","last_change":"2024-09-16T09:42:30.709733+0000","last_active":"2024-09-16T09:44:52.551892+0000","last_peered":"2024-09-16T09:44:52.551892+0000","last_clean":"2024-09-16T09:44:52.551892+0000","last_became_active":"2024-09-16T09:31:09.276516+0000","last_became_peered":"2024-09-16T09:31:09.276516+0000","last_unstale":"2024-09-16T09:44:52.551892+0000","last_undegraded":"2024-09-16T09:44:52.551892+0000","last_fullsized":"2024-09-16T09:44:52.551892+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:42:30.709673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:42:30.709673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:17.905087+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":942,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.703824+0000","last_change":"2024-09-16T09:44:24.276286+0000","last_active":"2024-09-16T09:44:52.703824+0000","last_peered":"2024-09-16T09:44:52.703824+0000","last_clean":"2024-09-16T09:44:52.703824+0000","last_became_active":"2024-09-16T09:44:24.275961+0000","last_became_peered":"2024-09-16T09:44:24.275961+0000","last_unstale":"2024-09-16T09:44:52.703824+0000","last_undegraded":"2024-09-16T09:44:52.703824+0000","last_fullsized":"2024-09-16T09:44:52.703824+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:42:56.059953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:42:56.059953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3],"acting":[4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1017,"reported_epoch":538,"state":"active+clean","last_fresh":"2024-09-16T09:44:54.559759+0000","last_change":"2024-09-16T09:44:24.281500+0000","last_active":"2024-09-16T09:44:54.559759+0000","last_peered":"2024-09-16T09:44:54.559759+0000","last_clean":"2024-09-16T09:44:54.559759+0000","last_became_active":"2024-09-16T09:44:24.278714+0000","last_became_peered":"2024-09-16T09:44:24.278714+0000","last_unstale":"2024-09-16T09:44:54.559759+0000","last_undegraded":"2024-09-16T09:44:54.559759+0000","last_fullsized":"2024-09-16T09:44:54.559759+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:42:19.850990+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:42:19.850990+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.f","version":"73'79","reported_seq":1070,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.552194+0000","last_change":"2024-09-16T09:43:24.748100+0000","last_active":"2024-09-16T09:44:52.552194+0000","last_peered":"2024-09-16T09:44:52.552194+0000","last_clean":"2024-09-16T09:44:52.552194+0000","last_became_active":"2024-09-16T09:30:10.693477+0000","last_became_peered":"2024-09-16T09:30:10.693477+0000","last_unstale":"2024-09-16T09:44:52.552194+0000","last_undegraded":"2024-09-16T09:44:52.552194+0000","last_fullsized":"2024-09-16T09:44:52.552194+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:43:24.748025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:43:24.748025+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"285'35","reported_seq":1065,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.551962+0000","last_change":"2024-09-16T09:42:59.545610+0000","last_active":"2024-09-16T09:44:52.551962+0000","last_peered":"2024-09-16T09:44:52.551962+0000","last_clean":"2024-09-16T09:44:52.551962+0000","last_became_active":"2024-09-16T09:25:03.712783+0000","last_became_peered":"2024-09-16T09:25:03.712783+0000","last_unstale":"2024-09-16T09:44:52.551962+0000","last_undegraded":"2024-09-16T09:44:52.551962+0000","last_fullsized":"2024-09-16T09:44:52.551962+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'35","last_scrub_stamp":"2024-09-16T09:42:59.545527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:42:59.545527+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:13.808431+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":56,"num_read_kb":45,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"59'1","reported_seq":1014,"reported_epoch":535,"state":"active+clean","last_fresh":"2024-09-16T09:44:50.625825+0000","last_change":"2024-09-16T09:42:24.291119+0000","last_active":"2024-09-16T09:44:50.625825+0000","last_peered":"2024-09-16T09:44:50.625825+0000","last_clean":"2024-09-16T09:44:50.625825+0000","last_became_active":"2024-09-16T09:25:01.708069+0000","last_became_peered":"2024-09-16T09:25:01.708069+0000","last_unstale":"2024-09-16T09:44:50.625825+0000","last_undegraded":"2024-09-16T09:44:50.625825+0000","last_fullsized":"2024-09-16T09:44:50.625825+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:42:24.291047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:42:24.291047+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":978,"reported_epoch":535,"state":"active+clean","last_fresh":"2024-09-16T09:44:50.542956+0000","last_change":"2024-09-16T09:44:33.039012+0000","last_active":"2024-09-16T09:44:50.542956+0000","last_peered":"2024-09-16T09:44:50.542956+0000","last_clean":"2024-09-16T09:44:50.542956+0000","last_became_active":"2024-09-16T09:44:24.278290+0000","last_became_peered":"2024-09-16T09:44:24.278290+0000","last_unstale":"2024-09-16T09:44:50.542956+0000","last_undegraded":"2024-09-16T09:44:50.542956+0000","last_fullsized":"2024-09-16T09:44:50.542956+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:33.038961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:44:33.038961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:58.372425+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":992,"reported_epoch":535,"state":"active+clean","last_fresh":"2024-09-16T09:44:50.625002+0000","last_change":"2024-09-16T09:44:24.275667+0000","last_active":"2024-09-16T09:44:50.625002+0000","last_peered":"2024-09-16T09:44:50.625002+0000","last_clean":"2024-09-16T09:44:50.625002+0000","last_became_active":"2024-09-16T09:44:24.275407+0000","last_became_peered":"2024-09-16T09:44:24.275407+0000","last_unstale":"2024-09-16T09:44:50.625002+0000","last_undegraded":"2024-09-16T09:44:50.625002+0000","last_fullsized":"2024-09-16T09:44:50.625002+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:42:15.241925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:42:15.241925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.c","version":"73'79","reported_seq":769,"reported_epoch":537,"state":"active+clean","last_fresh":"2024-09-16T09:44:53.558183+0000","last_change":"2024-09-16T09:44:51.141526+0000","last_active":"2024-09-16T09:44:53.558183+0000","last_peered":"2024-09-16T09:44:53.558183+0000","last_clean":"2024-09-16T09:44:53.558183+0000","last_became_active":"2024-09-16T09:31:09.273770+0000","last_became_peered":"2024-09-16T09:31:09.273770+0000","last_unstale":"2024-09-16T09:44:53.558183+0000","last_undegraded":"2024-09-16T09:44:53.558183+0000","last_fullsized":"2024-09-16T09:44:53.558183+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:44:51.141451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:44:51.141451+0000","objects_scrubbed":1,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:16.690997+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":327680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"285'18","reported_seq":1080,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.703833+0000","last_change":"2024-09-16T09:42:17.189319+0000","last_active":"2024-09-16T09:44:52.703833+0000","last_peered":"2024-09-16T09:44:52.703833+0000","last_clean":"2024-09-16T09:44:52.703833+0000","last_became_active":"2024-09-16T09:31:09.269047+0000","last_became_peered":"2024-09-16T09:31:09.269047+0000","last_unstale":"2024-09-16T09:44:52.703833+0000","last_undegraded":"2024-09-16T09:44:52.703833+0000","last_fullsized":"2024-09-16T09:44:52.703833+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'18","last_scrub_stamp":"2024-09-16T09:42:17.189230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:42:17.189230+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":1008,"reported_epoch":535,"state":"active+clean","last_fresh":"2024-09-16T09:44:50.625933+0000","last_change":"2024-09-16T09:42:33.321184+0000","last_active":"2024-09-16T09:44:50.625933+0000","last_peered":"2024-09-16T09:44:50.625933+0000","last_clean":"2024-09-16T09:44:50.625933+0000","last_became_active":"2024-09-16T09:25:01.707528+0000","last_became_peered":"2024-09-16T09:25:01.707528+0000","last_unstale":"2024-09-16T09:44:50.625933+0000","last_undegraded":"2024-09-16T09:44:50.625933+0000","last_fullsized":"2024-09-16T09:44:50.625933+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:42:33.321125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:42:33.321125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":680,"reported_epoch":538,"state":"active+clean","last_fresh":"2024-09-16T09:44:54.557787+0000","last_change":"2024-09-16T09:44:24.271417+0000","last_active":"2024-09-16T09:44:54.557787+0000","last_peered":"2024-09-16T09:44:54.557787+0000","last_clean":"2024-09-16T09:44:54.557787+0000","last_became_active":"2024-09-16T09:44:24.270329+0000","last_became_peered":"2024-09-16T09:44:24.270329+0000","last_unstale":"2024-09-16T09:44:54.557787+0000","last_undegraded":"2024-09-16T09:44:54.557787+0000","last_fullsized":"2024-09-16T09:44:54.557787+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:42:12.922136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:42:12.922136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1003,"reported_epoch":537,"state":"active+clean","last_fresh":"2024-09-16T09:44:53.557119+0000","last_change":"2024-09-16T09:44:04.116446+0000","last_active":"2024-09-16T09:44:53.557119+0000","last_peered":"2024-09-16T09:44:53.557119+0000","last_clean":"2024-09-16T09:44:53.557119+0000","last_became_active":"2024-09-16T09:25:07.718899+0000","last_became_peered":"2024-09-16T09:25:07.718899+0000","last_unstale":"2024-09-16T09:44:53.557119+0000","last_undegraded":"2024-09-16T09:44:53.557119+0000","last_fullsized":"2024-09-16T09:44:53.557119+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:04.116261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:44:04.116261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:05.947986+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.d","version":"73'79","reported_seq":798,"reported_epoch":535,"state":"active+clean","last_fresh":"2024-09-16T09:44:50.625235+0000","last_change":"2024-09-16T09:44:02.449886+0000","last_active":"2024-09-16T09:44:50.625235+0000","last_peered":"2024-09-16T09:44:50.625235+0000","last_clean":"2024-09-16T09:44:50.625235+0000","last_became_active":"2024-09-16T09:30:10.709958+0000","last_became_peered":"2024-09-16T09:30:10.709958+0000","last_unstale":"2024-09-16T09:44:50.625235+0000","last_undegraded":"2024-09-16T09:44:50.625235+0000","last_fullsized":"2024-09-16T09:44:50.625235+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:44:02.449603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:44:02.449603+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:29.882019+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"285'27","reported_seq":1052,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.551118+0000","last_change":"2024-09-16T09:44:21.970091+0000","last_active":"2024-09-16T09:44:52.551118+0000","last_peered":"2024-09-16T09:44:52.551118+0000","last_clean":"2024-09-16T09:44:52.551118+0000","last_became_active":"2024-09-16T09:25:03.715733+0000","last_became_peered":"2024-09-16T09:25:03.715733+0000","last_unstale":"2024-09-16T09:44:52.551118+0000","last_undegraded":"2024-09-16T09:44:52.551118+0000","last_fullsized":"2024-09-16T09:44:52.551118+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'27","last_scrub_stamp":"2024-09-16T09:44:21.970033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:44:21.970033+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:30.951780+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":1012,"reported_epoch":535,"state":"active+clean","last_fresh":"2024-09-16T09:44:50.625982+0000","last_change":"2024-09-16T09:42:18.192222+0000","last_active":"2024-09-16T09:44:50.625982+0000","last_peered":"2024-09-16T09:44:50.625982+0000","last_clean":"2024-09-16T09:44:50.625982+0000","last_became_active":"2024-09-16T09:25:01.707372+0000","last_became_peered":"2024-09-16T09:25:01.707372+0000","last_unstale":"2024-09-16T09:44:50.625982+0000","last_undegraded":"2024-09-16T09:44:50.625982+0000","last_fullsized":"2024-09-16T09:44:50.625982+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-16T09:42:18.192142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:42:18.192142+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":32,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"66'8","reported_seq":1256,"reported_epoch":535,"state":"active+clean","last_fresh":"2024-09-16T09:44:50.625003+0000","last_change":"2024-09-16T09:42:51.690430+0000","last_active":"2024-09-16T09:44:50.625003+0000","last_peered":"2024-09-16T09:44:50.625003+0000","last_clean":"2024-09-16T09:44:50.625003+0000","last_became_active":"2024-09-16T09:25:05.716189+0000","last_became_peered":"2024-09-16T09:25:05.716189+0000","last_unstale":"2024-09-16T09:44:50.625003+0000","last_undegraded":"2024-09-16T09:44:50.625003+0000","last_fullsized":"2024-09-16T09:44:50.625003+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:42:51.690361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:42:51.690361+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":993,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.552536+0000","last_change":"2024-09-16T09:44:18.939435+0000","last_active":"2024-09-16T09:44:52.552536+0000","last_peered":"2024-09-16T09:44:52.552536+0000","last_clean":"2024-09-16T09:44:52.552536+0000","last_became_active":"2024-09-16T09:25:07.717213+0000","last_became_peered":"2024-09-16T09:25:07.717213+0000","last_unstale":"2024-09-16T09:44:52.552536+0000","last_undegraded":"2024-09-16T09:44:52.552536+0000","last_fullsized":"2024-09-16T09:44:52.552536+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:18.939265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:44:18.939265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:45.253702+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.12","version":"73'79","reported_seq":810,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.552893+0000","last_change":"2024-09-16T09:42:16.729102+0000","last_active":"2024-09-16T09:44:52.552893+0000","last_peered":"2024-09-16T09:44:52.552893+0000","last_clean":"2024-09-16T09:44:52.552893+0000","last_became_active":"2024-09-16T09:30:10.731413+0000","last_became_peered":"2024-09-16T09:30:10.731413+0000","last_unstale":"2024-09-16T09:44:52.552893+0000","last_undegraded":"2024-09-16T09:44:52.552893+0000","last_fullsized":"2024-09-16T09:44:52.552893+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:42:16.728929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:42:16.728929+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:16.883189+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":1019,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.551151+0000","last_change":"2024-09-16T09:44:05.759438+0000","last_active":"2024-09-16T09:44:52.551151+0000","last_peered":"2024-09-16T09:44:52.551151+0000","last_clean":"2024-09-16T09:44:52.551151+0000","last_became_active":"2024-09-16T09:37:53.518354+0000","last_became_peered":"2024-09-16T09:37:53.518354+0000","last_unstale":"2024-09-16T09:44:52.551151+0000","last_undegraded":"2024-09-16T09:44:52.551151+0000","last_fullsized":"2024-09-16T09:44:52.551151+0000","mapping_epoch":382,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":383,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:05.759381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:44:05.759381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:11.154024+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"285'12","reported_seq":791,"reported_epoch":537,"state":"active+clean","last_fresh":"2024-09-16T09:44:53.558507+0000","last_change":"2024-09-16T09:44:24.274524+0000","last_active":"2024-09-16T09:44:53.558507+0000","last_peered":"2024-09-16T09:44:53.558507+0000","last_clean":"2024-09-16T09:44:53.558507+0000","last_became_active":"2024-09-16T09:44:24.274107+0000","last_became_peered":"2024-09-16T09:44:24.274107+0000","last_unstale":"2024-09-16T09:44:53.558507+0000","last_undegraded":"2024-09-16T09:44:53.558507+0000","last_fullsized":"2024-09-16T09:44:53.558507+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'12","last_scrub_stamp":"2024-09-16T09:42:52.958841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:42:52.958841+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:37.145040+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":17,"num_read_kb":14,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1004,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.552138+0000","last_change":"2024-09-16T09:43:25.752831+0000","last_active":"2024-09-16T09:44:52.552138+0000","last_peered":"2024-09-16T09:44:52.552138+0000","last_clean":"2024-09-16T09:44:52.552138+0000","last_became_active":"2024-09-16T09:25:05.715616+0000","last_became_peered":"2024-09-16T09:25:05.715616+0000","last_unstale":"2024-09-16T09:44:52.552138+0000","last_undegraded":"2024-09-16T09:44:52.552138+0000","last_fullsized":"2024-09-16T09:44:52.552138+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:43:25.752778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:43:25.752778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":996,"reported_epoch":535,"state":"active+clean","last_fresh":"2024-09-16T09:44:50.625301+0000","last_change":"2024-09-16T09:44:46.528000+0000","last_active":"2024-09-16T09:44:50.625301+0000","last_peered":"2024-09-16T09:44:50.625301+0000","last_clean":"2024-09-16T09:44:50.625301+0000","last_became_active":"2024-09-16T09:25:07.721059+0000","last_became_peered":"2024-09-16T09:25:07.721059+0000","last_unstale":"2024-09-16T09:44:50.625301+0000","last_undegraded":"2024-09-16T09:44:50.625301+0000","last_fullsized":"2024-09-16T09:44:50.625301+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:46.527936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:44:46.527936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:03.411304+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.13","version":"73'79","reported_seq":905,"reported_epoch":537,"state":"active+clean","last_fresh":"2024-09-16T09:44:53.557250+0000","last_change":"2024-09-16T09:44:24.133357+0000","last_active":"2024-09-16T09:44:53.557250+0000","last_peered":"2024-09-16T09:44:53.557250+0000","last_clean":"2024-09-16T09:44:53.557250+0000","last_became_active":"2024-09-16T09:33:30.248497+0000","last_became_peered":"2024-09-16T09:33:30.248497+0000","last_unstale":"2024-09-16T09:44:53.557250+0000","last_undegraded":"2024-09-16T09:44:53.557250+0000","last_fullsized":"2024-09-16T09:44:53.557250+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:44:24.133308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:44:24.133308+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:25.212115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"285'15","reported_seq":1088,"reported_epoch":538,"state":"active+clean","last_fresh":"2024-09-16T09:44:54.559201+0000","last_change":"2024-09-16T09:42:53.892511+0000","last_active":"2024-09-16T09:44:54.559201+0000","last_peered":"2024-09-16T09:44:54.559201+0000","last_clean":"2024-09-16T09:44:54.559201+0000","last_became_active":"2024-09-16T09:31:09.273498+0000","last_became_peered":"2024-09-16T09:31:09.273498+0000","last_unstale":"2024-09-16T09:44:54.559201+0000","last_undegraded":"2024-09-16T09:44:54.559201+0000","last_fullsized":"2024-09-16T09:44:54.559201+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'15","last_scrub_stamp":"2024-09-16T09:42:53.892338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:42:53.892338+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":1016,"reported_epoch":535,"state":"active+clean","last_fresh":"2024-09-16T09:44:50.625293+0000","last_change":"2024-09-16T09:43:49.322844+0000","last_active":"2024-09-16T09:44:50.625293+0000","last_peered":"2024-09-16T09:44:50.625293+0000","last_clean":"2024-09-16T09:44:50.625293+0000","last_became_active":"2024-09-16T09:25:01.706491+0000","last_became_peered":"2024-09-16T09:25:01.706491+0000","last_unstale":"2024-09-16T09:44:50.625293+0000","last_undegraded":"2024-09-16T09:44:50.625293+0000","last_fullsized":"2024-09-16T09:44:50.625293+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:43:49.322773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:43:49.322773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:44:54.891734+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":773,"reported_epoch":535,"state":"active+clean","last_fresh":"2024-09-16T09:44:50.625614+0000","last_change":"2024-09-16T09:42:08.180799+0000","last_active":"2024-09-16T09:44:50.625614+0000","last_peered":"2024-09-16T09:44:50.625614+0000","last_clean":"2024-09-16T09:44:50.625614+0000","last_became_active":"2024-09-16T09:33:30.248992+0000","last_became_peered":"2024-09-16T09:33:30.248992+0000","last_unstale":"2024-09-16T09:44:50.625614+0000","last_undegraded":"2024-09-16T09:44:50.625614+0000","last_fullsized":"2024-09-16T09:44:50.625614+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:42:08.180735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:42:08.180735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:07.411320+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":980,"reported_epoch":535,"state":"active+clean","last_fresh":"2024-09-16T09:44:50.543026+0000","last_change":"2024-09-16T09:44:46.061164+0000","last_active":"2024-09-16T09:44:50.543026+0000","last_peered":"2024-09-16T09:44:50.543026+0000","last_clean":"2024-09-16T09:44:50.543026+0000","last_became_active":"2024-09-16T09:44:24.277612+0000","last_became_peered":"2024-09-16T09:44:24.277612+0000","last_unstale":"2024-09-16T09:44:50.543026+0000","last_undegraded":"2024-09-16T09:44:50.543026+0000","last_fullsized":"2024-09-16T09:44:50.543026+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:46.061101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:44:46.061101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:04.434595+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.10","version":"73'79","reported_seq":810,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.550823+0000","last_change":"2024-09-16T09:44:43.656131+0000","last_active":"2024-09-16T09:44:52.550823+0000","last_peered":"2024-09-16T09:44:52.550823+0000","last_clean":"2024-09-16T09:44:52.550823+0000","last_became_active":"2024-09-16T09:30:10.692770+0000","last_became_peered":"2024-09-16T09:30:10.692770+0000","last_unstale":"2024-09-16T09:44:52.550823+0000","last_undegraded":"2024-09-16T09:44:52.550823+0000","last_fullsized":"2024-09-16T09:44:52.550823+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:44:43.656039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:44:43.656039+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:56.070444+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"285'15","reported_seq":1033,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.552843+0000","last_change":"2024-09-16T09:43:47.715081+0000","last_active":"2024-09-16T09:44:52.552843+0000","last_peered":"2024-09-16T09:44:52.552843+0000","last_clean":"2024-09-16T09:44:52.552843+0000","last_became_active":"2024-09-16T09:25:03.717310+0000","last_became_peered":"2024-09-16T09:25:03.717310+0000","last_unstale":"2024-09-16T09:44:52.552843+0000","last_undegraded":"2024-09-16T09:44:52.552843+0000","last_fullsized":"2024-09-16T09:44:52.552843+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'15","last_scrub_stamp":"2024-09-16T09:43:47.714882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:43:47.714882+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:44:54.735513+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":1047,"reported_epoch":538,"state":"active+clean","last_fresh":"2024-09-16T09:44:54.558631+0000","last_change":"2024-09-16T09:44:35.099469+0000","last_active":"2024-09-16T09:44:54.558631+0000","last_peered":"2024-09-16T09:44:54.558631+0000","last_clean":"2024-09-16T09:44:54.558631+0000","last_became_active":"2024-09-16T09:31:09.272848+0000","last_became_peered":"2024-09-16T09:31:09.272848+0000","last_unstale":"2024-09-16T09:44:54.558631+0000","last_undegraded":"2024-09-16T09:44:54.558631+0000","last_fullsized":"2024-09-16T09:44:54.558631+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:35.099420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:44:35.099420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:01.489513+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"519'13","reported_seq":1097,"reported_epoch":535,"state":"active+clean","last_fresh":"2024-09-16T09:44:50.625515+0000","last_change":"2024-09-16T09:44:24.285251+0000","last_active":"2024-09-16T09:44:50.625515+0000","last_peered":"2024-09-16T09:44:50.625515+0000","last_clean":"2024-09-16T09:44:50.625515+0000","last_became_active":"2024-09-16T09:44:24.284843+0000","last_became_peered":"2024-09-16T09:44:24.284843+0000","last_unstale":"2024-09-16T09:44:50.625515+0000","last_undegraded":"2024-09-16T09:44:50.625515+0000","last_fullsized":"2024-09-16T09:44:50.625515+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"187'11","last_scrub_stamp":"2024-09-16T09:42:19.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:42:19.191311+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1028,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.704577+0000","last_change":"2024-09-16T09:44:24.277838+0000","last_active":"2024-09-16T09:44:52.704577+0000","last_peered":"2024-09-16T09:44:52.704577+0000","last_clean":"2024-09-16T09:44:52.704577+0000","last_became_active":"2024-09-16T09:44:24.277124+0000","last_became_peered":"2024-09-16T09:44:24.277124+0000","last_unstale":"2024-09-16T09:44:52.704577+0000","last_undegraded":"2024-09-16T09:44:52.704577+0000","last_fullsized":"2024-09-16T09:44:52.704577+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:36.098648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:41:36.098648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.11","version":"73'79","reported_seq":892,"reported_epoch":538,"state":"active+clean","last_fresh":"2024-09-16T09:44:54.559954+0000","last_change":"2024-09-16T09:42:16.839630+0000","last_active":"2024-09-16T09:44:54.559954+0000","last_peered":"2024-09-16T09:44:54.559954+0000","last_clean":"2024-09-16T09:44:54.559954+0000","last_became_active":"2024-09-16T09:31:09.272208+0000","last_became_peered":"2024-09-16T09:31:09.272208+0000","last_unstale":"2024-09-16T09:44:54.559954+0000","last_undegraded":"2024-09-16T09:44:54.559954+0000","last_fullsized":"2024-09-16T09:44:54.559954+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:42:16.839458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:42:16.839458+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.14","version":"285'20","reported_seq":690,"reported_epoch":535,"state":"active+clean","last_fresh":"2024-09-16T09:44:50.625611+0000","last_change":"2024-09-16T09:40:35.027221+0000","last_active":"2024-09-16T09:44:50.625611+0000","last_peered":"2024-09-16T09:44:50.625611+0000","last_clean":"2024-09-16T09:44:50.625611+0000","last_became_active":"2024-09-16T09:33:30.249769+0000","last_became_peered":"2024-09-16T09:33:30.249769+0000","last_unstale":"2024-09-16T09:44:50.625611+0000","last_undegraded":"2024-09-16T09:44:50.625611+0000","last_fullsized":"2024-09-16T09:44:50.625611+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'20","last_scrub_stamp":"2024-09-16T09:40:35.027096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:40:35.027096+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:13.420390+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":1011,"reported_epoch":535,"state":"active+clean","last_fresh":"2024-09-16T09:44:50.625443+0000","last_change":"2024-09-16T09:44:24.279638+0000","last_active":"2024-09-16T09:44:50.625443+0000","last_peered":"2024-09-16T09:44:50.625443+0000","last_clean":"2024-09-16T09:44:50.625443+0000","last_became_active":"2024-09-16T09:44:24.279273+0000","last_became_peered":"2024-09-16T09:44:24.279273+0000","last_unstale":"2024-09-16T09:44:50.625443+0000","last_undegraded":"2024-09-16T09:44:50.625443+0000","last_fullsized":"2024-09-16T09:44:50.625443+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:08.005728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:41:08.005728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:37.469764+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"66'8","reported_seq":1267,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.552654+0000","last_change":"2024-09-16T09:44:35.809600+0000","last_active":"2024-09-16T09:44:52.552654+0000","last_peered":"2024-09-16T09:44:52.552654+0000","last_clean":"2024-09-16T09:44:52.552654+0000","last_became_active":"2024-09-16T09:25:05.713833+0000","last_became_peered":"2024-09-16T09:25:05.713833+0000","last_unstale":"2024-09-16T09:44:52.552654+0000","last_undegraded":"2024-09-16T09:44:52.552654+0000","last_fullsized":"2024-09-16T09:44:52.552654+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:44:35.809538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:44:35.809538+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:38.475407+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1035,"reported_epoch":538,"state":"active+clean","last_fresh":"2024-09-16T09:44:54.558787+0000","last_change":"2024-09-16T09:42:32.913873+0000","last_active":"2024-09-16T09:44:54.558787+0000","last_peered":"2024-09-16T09:44:54.558787+0000","last_clean":"2024-09-16T09:44:54.558787+0000","last_became_active":"2024-09-16T09:31:09.275168+0000","last_became_peered":"2024-09-16T09:31:09.275168+0000","last_unstale":"2024-09-16T09:44:54.558787+0000","last_undegraded":"2024-09-16T09:44:54.558787+0000","last_fullsized":"2024-09-16T09:44:54.558787+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:42:32.913795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:42:32.913795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"285'19","reported_seq":1043,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.704011+0000","last_change":"2024-09-16T09:42:20.151407+0000","last_active":"2024-09-16T09:44:52.704011+0000","last_peered":"2024-09-16T09:44:52.704011+0000","last_clean":"2024-09-16T09:44:52.704011+0000","last_became_active":"2024-09-16T09:26:33.291265+0000","last_became_peered":"2024-09-16T09:26:33.291265+0000","last_unstale":"2024-09-16T09:44:52.704011+0000","last_undegraded":"2024-09-16T09:44:52.704011+0000","last_fullsized":"2024-09-16T09:44:52.704011+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'19","last_scrub_stamp":"2024-09-16T09:42:20.151313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:42:20.151313+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":1029,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.704323+0000","last_change":"2024-09-16T09:44:23.334509+0000","last_active":"2024-09-16T09:44:52.704323+0000","last_peered":"2024-09-16T09:44:52.704323+0000","last_clean":"2024-09-16T09:44:52.704323+0000","last_became_active":"2024-09-16T09:25:01.705158+0000","last_became_peered":"2024-09-16T09:25:01.705158+0000","last_unstale":"2024-09-16T09:44:52.704323+0000","last_undegraded":"2024-09-16T09:44:52.704323+0000","last_fullsized":"2024-09-16T09:44:52.704323+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:23.334458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:44:23.334458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:34.371408+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":948,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.552669+0000","last_change":"2024-09-16T09:44:45.660171+0000","last_active":"2024-09-16T09:44:52.552669+0000","last_peered":"2024-09-16T09:44:52.552669+0000","last_clean":"2024-09-16T09:44:52.552669+0000","last_became_active":"2024-09-16T09:37:41.515278+0000","last_became_peered":"2024-09-16T09:37:41.515278+0000","last_unstale":"2024-09-16T09:44:52.552669+0000","last_undegraded":"2024-09-16T09:44:52.552669+0000","last_fullsized":"2024-09-16T09:44:52.552669+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:45.660111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:44:45.660111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:48.190760+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":859,"reported_epoch":538,"state":"active+clean","last_fresh":"2024-09-16T09:44:54.559555+0000","last_change":"2024-09-16T09:44:47.085511+0000","last_active":"2024-09-16T09:44:54.559555+0000","last_peered":"2024-09-16T09:44:54.559555+0000","last_clean":"2024-09-16T09:44:54.559555+0000","last_became_active":"2024-09-16T09:31:09.276042+0000","last_became_peered":"2024-09-16T09:31:09.276042+0000","last_unstale":"2024-09-16T09:44:54.559555+0000","last_undegraded":"2024-09-16T09:44:54.559555+0000","last_fullsized":"2024-09-16T09:44:54.559555+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:47.085465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:44:47.085465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:58.894807+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"285'15","reported_seq":742,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.552137+0000","last_change":"2024-09-16T09:44:28.650617+0000","last_active":"2024-09-16T09:44:52.552137+0000","last_peered":"2024-09-16T09:44:52.552137+0000","last_clean":"2024-09-16T09:44:52.552137+0000","last_became_active":"2024-09-16T09:44:24.270349+0000","last_became_peered":"2024-09-16T09:44:24.270349+0000","last_unstale":"2024-09-16T09:44:52.552137+0000","last_undegraded":"2024-09-16T09:44:52.552137+0000","last_fullsized":"2024-09-16T09:44:52.552137+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'15","last_scrub_stamp":"2024-09-16T09:44:28.650561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:44:28.650561+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:38.659197+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,2],"acting":[6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":1008,"reported_epoch":535,"state":"active+clean","last_fresh":"2024-09-16T09:44:50.625707+0000","last_change":"2024-09-16T09:43:47.307622+0000","last_active":"2024-09-16T09:44:50.625707+0000","last_peered":"2024-09-16T09:44:50.625707+0000","last_clean":"2024-09-16T09:44:50.625707+0000","last_became_active":"2024-09-16T09:25:01.706762+0000","last_became_peered":"2024-09-16T09:25:01.706762+0000","last_unstale":"2024-09-16T09:44:50.625707+0000","last_undegraded":"2024-09-16T09:44:50.625707+0000","last_fullsized":"2024-09-16T09:44:50.625707+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:43:47.307538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:43:47.307538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:02.160911+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":844,"reported_epoch":538,"state":"active+clean","last_fresh":"2024-09-16T09:44:54.559674+0000","last_change":"2024-09-16T09:41:36.130305+0000","last_active":"2024-09-16T09:44:54.559674+0000","last_peered":"2024-09-16T09:44:54.559674+0000","last_clean":"2024-09-16T09:44:54.559674+0000","last_became_active":"2024-09-16T09:28:52.982865+0000","last_became_peered":"2024-09-16T09:28:52.982865+0000","last_unstale":"2024-09-16T09:44:54.559674+0000","last_undegraded":"2024-09-16T09:44:54.559674+0000","last_fullsized":"2024-09-16T09:44:54.559674+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:36.130240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:41:36.130240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":989,"reported_epoch":538,"state":"active+clean","last_fresh":"2024-09-16T09:44:54.557660+0000","last_change":"2024-09-16T09:34:50.779241+0000","last_active":"2024-09-16T09:44:54.557660+0000","last_peered":"2024-09-16T09:44:54.557660+0000","last_clean":"2024-09-16T09:44:54.557660+0000","last_became_active":"2024-09-16T09:25:07.715301+0000","last_became_peered":"2024-09-16T09:25:07.715301+0000","last_unstale":"2024-09-16T09:44:54.557660+0000","last_undegraded":"2024-09-16T09:44:54.557660+0000","last_fullsized":"2024-09-16T09:44:54.557660+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:50.779164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:34:50.779164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:08.818483+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.14","version":"73'79","reported_seq":722,"reported_epoch":535,"state":"active+clean","last_fresh":"2024-09-16T09:44:50.542844+0000","last_change":"2024-09-16T09:44:24.960676+0000","last_active":"2024-09-16T09:44:50.542844+0000","last_peered":"2024-09-16T09:44:50.542844+0000","last_clean":"2024-09-16T09:44:50.542844+0000","last_became_active":"2024-09-16T09:44:24.285886+0000","last_became_peered":"2024-09-16T09:44:24.285886+0000","last_unstale":"2024-09-16T09:44:50.542844+0000","last_undegraded":"2024-09-16T09:44:50.542844+0000","last_fullsized":"2024-09-16T09:44:50.542844+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:44:24.960619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:44:24.960619+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:44.429977+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"285'19","reported_seq":887,"reported_epoch":535,"state":"active+clean","last_fresh":"2024-09-16T09:44:50.625240+0000","last_change":"2024-09-16T09:42:57.678412+0000","last_active":"2024-09-16T09:44:50.625240+0000","last_peered":"2024-09-16T09:44:50.625240+0000","last_clean":"2024-09-16T09:44:50.625240+0000","last_became_active":"2024-09-16T09:28:52.971132+0000","last_became_peered":"2024-09-16T09:28:52.971132+0000","last_unstale":"2024-09-16T09:44:50.625240+0000","last_undegraded":"2024-09-16T09:44:50.625240+0000","last_fullsized":"2024-09-16T09:44:50.625240+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'19","last_scrub_stamp":"2024-09-16T09:42:57.678331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:42:57.678331+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":1020,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.552275+0000","last_change":"2024-09-16T09:44:03.793102+0000","last_active":"2024-09-16T09:44:52.552275+0000","last_peered":"2024-09-16T09:44:52.552275+0000","last_clean":"2024-09-16T09:44:52.552275+0000","last_became_active":"2024-09-16T09:25:01.700733+0000","last_became_peered":"2024-09-16T09:25:01.700733+0000","last_unstale":"2024-09-16T09:44:52.552275+0000","last_undegraded":"2024-09-16T09:44:52.552275+0000","last_fullsized":"2024-09-16T09:44:52.552275+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:03.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:44:03.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:16.224422+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1000,"reported_epoch":535,"state":"active+clean","last_fresh":"2024-09-16T09:44:50.624696+0000","last_change":"2024-09-16T09:40:39.969447+0000","last_active":"2024-09-16T09:44:50.624696+0000","last_peered":"2024-09-16T09:44:50.624696+0000","last_clean":"2024-09-16T09:44:50.624696+0000","last_became_active":"2024-09-16T09:25:05.713479+0000","last_became_peered":"2024-09-16T09:25:05.713479+0000","last_unstale":"2024-09-16T09:44:50.624696+0000","last_undegraded":"2024-09-16T09:44:50.624696+0000","last_fullsized":"2024-09-16T09:44:50.624696+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:40:39.969383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:40:39.969383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:38.496665+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":867,"reported_epoch":538,"state":"active+clean","last_fresh":"2024-09-16T09:44:54.560204+0000","last_change":"2024-09-16T09:41:34.178100+0000","last_active":"2024-09-16T09:44:54.560204+0000","last_peered":"2024-09-16T09:44:54.560204+0000","last_clean":"2024-09-16T09:44:54.560204+0000","last_became_active":"2024-09-16T09:31:09.274797+0000","last_became_peered":"2024-09-16T09:31:09.274797+0000","last_unstale":"2024-09-16T09:44:54.560204+0000","last_undegraded":"2024-09-16T09:44:54.560204+0000","last_fullsized":"2024-09-16T09:44:54.560204+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:34.178020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:41:34.178020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"285'8","reported_seq":784,"reported_epoch":537,"state":"active+clean","last_fresh":"2024-09-16T09:44:53.556182+0000","last_change":"2024-09-16T09:44:47.186136+0000","last_active":"2024-09-16T09:44:53.556182+0000","last_peered":"2024-09-16T09:44:53.556182+0000","last_clean":"2024-09-16T09:44:53.556182+0000","last_became_active":"2024-09-16T09:31:09.273682+0000","last_became_peered":"2024-09-16T09:31:09.273682+0000","last_unstale":"2024-09-16T09:44:53.556182+0000","last_undegraded":"2024-09-16T09:44:53.556182+0000","last_fullsized":"2024-09-16T09:44:53.556182+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'8","last_scrub_stamp":"2024-09-16T09:44:47.186078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:44:47.186078+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:11.853660+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":9,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":1047,"reported_epoch":538,"state":"active+clean","last_fresh":"2024-09-16T09:44:54.558627+0000","last_change":"2024-09-16T09:43:21.684366+0000","last_active":"2024-09-16T09:44:54.558627+0000","last_peered":"2024-09-16T09:44:54.558627+0000","last_clean":"2024-09-16T09:44:54.558627+0000","last_became_active":"2024-09-16T09:31:09.274719+0000","last_became_peered":"2024-09-16T09:31:09.274719+0000","last_unstale":"2024-09-16T09:44:54.558627+0000","last_undegraded":"2024-09-16T09:44:54.558627+0000","last_fullsized":"2024-09-16T09:44:54.558627+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:43:21.684289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:43:21.684289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1021,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.551482+0000","last_change":"2024-09-16T09:44:49.775416+0000","last_active":"2024-09-16T09:44:52.551482+0000","last_peered":"2024-09-16T09:44:52.551482+0000","last_clean":"2024-09-16T09:44:52.551482+0000","last_became_active":"2024-09-16T09:25:05.712545+0000","last_became_peered":"2024-09-16T09:25:05.712545+0000","last_unstale":"2024-09-16T09:44:52.551482+0000","last_undegraded":"2024-09-16T09:44:52.551482+0000","last_fullsized":"2024-09-16T09:44:52.551482+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:49.775356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:44:49.775356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:57.470848+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1015,"reported_epoch":535,"state":"active+clean","last_fresh":"2024-09-16T09:44:51.632486+0000","last_change":"2024-09-16T09:44:51.632486+0000","last_active":"2024-09-16T09:44:51.632486+0000","last_peered":"2024-09-16T09:44:51.632486+0000","last_clean":"2024-09-16T09:44:51.632486+0000","last_became_active":"2024-09-16T09:44:24.276311+0000","last_became_peered":"2024-09-16T09:44:24.276311+0000","last_unstale":"2024-09-16T09:44:51.632486+0000","last_undegraded":"2024-09-16T09:44:51.632486+0000","last_fullsized":"2024-09-16T09:44:51.632486+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:51.632429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:44:51.632429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:19.007714+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":644,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.552774+0000","last_change":"2024-09-16T09:44:50.722288+0000","last_active":"2024-09-16T09:44:52.552774+0000","last_peered":"2024-09-16T09:44:52.552774+0000","last_clean":"2024-09-16T09:44:52.552774+0000","last_became_active":"2024-09-16T09:33:30.248036+0000","last_became_peered":"2024-09-16T09:33:30.248036+0000","last_unstale":"2024-09-16T09:44:52.552774+0000","last_undegraded":"2024-09-16T09:44:52.552774+0000","last_fullsized":"2024-09-16T09:44:52.552774+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:50.722209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:44:50.722209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:53.928329+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":866,"reported_epoch":538,"state":"active+clean","last_fresh":"2024-09-16T09:44:54.558201+0000","last_change":"2024-09-16T09:42:30.898163+0000","last_active":"2024-09-16T09:44:54.558201+0000","last_peered":"2024-09-16T09:44:54.558201+0000","last_clean":"2024-09-16T09:44:54.558201+0000","last_became_active":"2024-09-16T09:31:09.271662+0000","last_became_peered":"2024-09-16T09:31:09.271662+0000","last_unstale":"2024-09-16T09:44:54.558201+0000","last_undegraded":"2024-09-16T09:44:54.558201+0000","last_fullsized":"2024-09-16T09:44:54.558201+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:42:30.898094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:42:30.898094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"285'19","reported_seq":1049,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.552306+0000","last_change":"2024-09-16T09:44:23.700110+0000","last_active":"2024-09-16T09:44:52.552306+0000","last_peered":"2024-09-16T09:44:52.552306+0000","last_clean":"2024-09-16T09:44:52.552306+0000","last_became_active":"2024-09-16T09:25:03.707769+0000","last_became_peered":"2024-09-16T09:25:03.707769+0000","last_unstale":"2024-09-16T09:44:52.552306+0000","last_undegraded":"2024-09-16T09:44:52.552306+0000","last_fullsized":"2024-09-16T09:44:52.552306+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'19","last_scrub_stamp":"2024-09-16T09:44:23.700027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:44:23.700027+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:26.961037+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1021,"reported_epoch":538,"state":"active+clean","last_fresh":"2024-09-16T09:44:54.558933+0000","last_change":"2024-09-16T09:44:36.073614+0000","last_active":"2024-09-16T09:44:54.558933+0000","last_peered":"2024-09-16T09:44:54.558933+0000","last_clean":"2024-09-16T09:44:54.558933+0000","last_became_active":"2024-09-16T09:44:24.279906+0000","last_became_peered":"2024-09-16T09:44:24.279906+0000","last_unstale":"2024-09-16T09:44:54.558933+0000","last_undegraded":"2024-09-16T09:44:54.558933+0000","last_fullsized":"2024-09-16T09:44:54.558933+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:36.073560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:44:36.073560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:44.457091+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":1115,"num_read_kb":872,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":3686,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":683,"ondisk_log_size":683,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1986,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1183,"num_read_kb":1178,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":21,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":327680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1659,"ondisk_log_size":1659,"up":63,"acting":63,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266931,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":848368,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":847486,"kb_avail":92896784,"statfs":{"total":95995035648,"available":95126306816,"internally_reserved":0,"allocated":876544,"data_stored":522396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":867826122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495864,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":718176,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":716990,"kb_avail":93026976,"statfs":{"total":95995035648,"available":95259623424,"internally_reserved":0,"allocated":1187840,"data_stored":856814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":734198214},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724799,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":858352,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":857214,"kb_avail":92886800,"statfs":{"total":95995035648,"available":95116083200,"internally_reserved":0,"allocated":1142784,"data_stored":792511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":877787590},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953732,"num_pgs":65,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":735916,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":735038,"kb_avail":93009236,"statfs":{"total":95995035648,"available":95241457664,"internally_reserved":0,"allocated":876544,"data_stored":522350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":752679366},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149962,"num_pgs":64,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":916992,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":915518,"kb_avail":92828160,"statfs":{"total":95995035648,"available":95056035840,"internally_reserved":0,"allocated":1486848,"data_stored":1120840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":937490887},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":516,"seq":2216203124743,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":926088,"kb_used_data":844344,"kb_used_omap":4,"kb_used_meta":81723,"kb_avail":92819064,"statfs":{"total":95995035648,"available":95046721536,"internally_reserved":0,"allocated":864608256,"data_stored":864285924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4445,"internal_metadata":83685027},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575126,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1024728,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":1023806,"kb_avail":92720424,"statfs":{"total":95995035648,"available":94945714176,"internally_reserved":0,"allocated":917504,"data_stored":530276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":1048377797},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738652,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":866752,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":865534,"kb_avail":92878400,"statfs":{"total":95995035648,"available":95107481600,"internally_reserved":0,"allocated":1224704,"data_stored":866056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":886307269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":1788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T09:44:56.957 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-16T09:44:57.208 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:44:57.584 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:57 smithi070 ceph-mon[30388]: pgmap v1051: 152 pgs: 152 active+clean; 583 KiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:44:57.584 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:57 smithi070 ceph-mon[30388]: from='client.26585 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:44:57.584 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:57 smithi070 ceph-mon[40691]: pgmap v1051: 152 pgs: 152 active+clean; 583 KiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:44:57.584 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:57 smithi070 ceph-mon[40691]: from='client.26585 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:44:57.853 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:44:57.855 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2024-09-16T09:44:57.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:57 smithi084 ceph-mon[32629]: pgmap v1051: 152 pgs: 152 active+clean; 583 KiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:44:57.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:57 smithi084 ceph-mon[32629]: from='client.26585 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:44:58.726 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":1051,"stamp":"2024-09-16T09:44:55.928460+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596507,"num_objects":199,"num_object_clones":0,"num_object_copies":597,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":199,"num_whiteouts":0,"num_read":2322,"num_read_kb":2074,"num_write":722,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":151,"num_bytes_recovered":333352,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2434,"ondisk_log_size":2434,"up":456,"acting":456,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":458,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":6895372,"kb_used_data":851876,"kb_used_omap":15,"kb_used_meta":6043312,"kb_avail":743065844,"statfs":{"total":767960285184,"available":760899424256,"internally_reserved":0,"allocated":872321024,"data_stored":869497167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15600,"internal_metadata":6188352272},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000948"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":1037,"reported_epoch":538,"state":"active+clean","last_fresh":"2024-09-16T09:44:54.558549+0000","last_change":"2024-09-16T09:44:24.277767+0000","last_active":"2024-09-16T09:44:54.558549+0000","last_peered":"2024-09-16T09:44:54.558549+0000","last_clean":"2024-09-16T09:44:54.558549+0000","last_became_active":"2024-09-16T09:44:24.277455+0000","last_became_peered":"2024-09-16T09:44:24.277455+0000","last_unstale":"2024-09-16T09:44:54.558549+0000","last_undegraded":"2024-09-16T09:44:54.558549+0000","last_fullsized":"2024-09-16T09:44:54.558549+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:40.073089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:41:40.073089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"285'15","reported_seq":1042,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.703529+0000","last_change":"2024-09-16T09:44:05.286194+0000","last_active":"2024-09-16T09:44:52.703529+0000","last_peered":"2024-09-16T09:44:52.703529+0000","last_clean":"2024-09-16T09:44:52.703529+0000","last_became_active":"2024-09-16T09:25:03.714635+0000","last_became_peered":"2024-09-16T09:25:03.714635+0000","last_unstale":"2024-09-16T09:44:52.703529+0000","last_undegraded":"2024-09-16T09:44:52.703529+0000","last_fullsized":"2024-09-16T09:44:52.703529+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'15","last_scrub_stamp":"2024-09-16T09:44:05.286119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:44:05.286119+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:29.124567+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":644,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.552434+0000","last_change":"2024-09-16T09:42:31.714835+0000","last_active":"2024-09-16T09:44:52.552434+0000","last_peered":"2024-09-16T09:44:52.552434+0000","last_clean":"2024-09-16T09:44:52.552434+0000","last_became_active":"2024-09-16T09:33:30.242238+0000","last_became_peered":"2024-09-16T09:33:30.242238+0000","last_unstale":"2024-09-16T09:44:52.552434+0000","last_undegraded":"2024-09-16T09:44:52.552434+0000","last_fullsized":"2024-09-16T09:44:52.552434+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:42:31.714749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:42:31.714749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:06.776034+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1000,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.703757+0000","last_change":"2024-09-16T09:44:35.357353+0000","last_active":"2024-09-16T09:44:52.703757+0000","last_peered":"2024-09-16T09:44:52.703757+0000","last_clean":"2024-09-16T09:44:52.703757+0000","last_became_active":"2024-09-16T09:25:07.715442+0000","last_became_peered":"2024-09-16T09:25:07.715442+0000","last_unstale":"2024-09-16T09:44:52.703757+0000","last_undegraded":"2024-09-16T09:44:52.703757+0000","last_fullsized":"2024-09-16T09:44:52.703757+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:35.357286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:44:35.357286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:52.302674+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":748,"reported_epoch":537,"state":"active+clean","last_fresh":"2024-09-16T09:44:53.557032+0000","last_change":"2024-09-16T09:44:34.236344+0000","last_active":"2024-09-16T09:44:53.557032+0000","last_peered":"2024-09-16T09:44:53.557032+0000","last_clean":"2024-09-16T09:44:53.557032+0000","last_became_active":"2024-09-16T09:31:09.269342+0000","last_became_peered":"2024-09-16T09:31:09.269342+0000","last_unstale":"2024-09-16T09:44:53.557032+0000","last_undegraded":"2024-09-16T09:44:53.557032+0000","last_fullsized":"2024-09-16T09:44:53.557032+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:34.236298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:44:34.236298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:59.040431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":891,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.551991+0000","last_change":"2024-09-16T09:44:26.603265+0000","last_active":"2024-09-16T09:44:52.551991+0000","last_peered":"2024-09-16T09:44:52.551991+0000","last_clean":"2024-09-16T09:44:52.551991+0000","last_became_active":"2024-09-16T09:44:24.279187+0000","last_became_peered":"2024-09-16T09:44:24.279187+0000","last_unstale":"2024-09-16T09:44:52.551991+0000","last_undegraded":"2024-09-16T09:44:52.551991+0000","last_fullsized":"2024-09-16T09:44:52.551991+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:26.603213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:44:26.603213+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:50.281936+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.19","version":"285'27","reported_seq":223,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.703526+0000","last_change":"2024-09-16T09:40:58.802023+0000","last_active":"2024-09-16T09:44:52.703526+0000","last_peered":"2024-09-16T09:44:52.703526+0000","last_clean":"2024-09-16T09:44:52.703526+0000","last_became_active":"2024-09-16T09:40:58.699357+0000","last_became_peered":"2024-09-16T09:40:58.699357+0000","last_unstale":"2024-09-16T09:44:52.703526+0000","last_undegraded":"2024-09-16T09:44:52.703526+0000","last_fullsized":"2024-09-16T09:44:52.703526+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'27","last_scrub_stamp":"2024-09-16T09:37:29.683568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:37:29.683568+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:43.367028+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,5],"acting":[4,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1013,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.703484+0000","last_change":"2024-09-16T09:42:10.051996+0000","last_active":"2024-09-16T09:44:52.703484+0000","last_peered":"2024-09-16T09:44:52.703484+0000","last_clean":"2024-09-16T09:44:52.703484+0000","last_became_active":"2024-09-16T09:25:05.717236+0000","last_became_peered":"2024-09-16T09:25:05.717236+0000","last_unstale":"2024-09-16T09:44:52.703484+0000","last_undegraded":"2024-09-16T09:44:52.703484+0000","last_fullsized":"2024-09-16T09:44:52.703484+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:42:10.051939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:42:10.051939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":657,"reported_epoch":538,"state":"active+clean","last_fresh":"2024-09-16T09:44:54.557931+0000","last_change":"2024-09-16T09:43:23.679786+0000","last_active":"2024-09-16T09:44:54.557931+0000","last_peered":"2024-09-16T09:44:54.557931+0000","last_clean":"2024-09-16T09:44:54.557931+0000","last_became_active":"2024-09-16T09:33:30.247733+0000","last_became_peered":"2024-09-16T09:33:30.247733+0000","last_unstale":"2024-09-16T09:44:54.557931+0000","last_undegraded":"2024-09-16T09:44:54.557931+0000","last_fullsized":"2024-09-16T09:44:54.557931+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:43:23.679720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:43:23.679720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1a","version":"285'15","reported_seq":1041,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.704077+0000","last_change":"2024-09-16T09:43:59.240127+0000","last_active":"2024-09-16T09:44:52.704077+0000","last_peered":"2024-09-16T09:44:52.704077+0000","last_clean":"2024-09-16T09:44:52.704077+0000","last_became_active":"2024-09-16T09:25:03.709147+0000","last_became_peered":"2024-09-16T09:25:03.709147+0000","last_unstale":"2024-09-16T09:44:52.704077+0000","last_undegraded":"2024-09-16T09:44:52.704077+0000","last_fullsized":"2024-09-16T09:44:52.704077+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'15","last_scrub_stamp":"2024-09-16T09:43:59.240052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:43:59.240052+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:06.629394+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":995,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.552415+0000","last_change":"2024-09-16T09:42:57.714429+0000","last_active":"2024-09-16T09:44:52.552415+0000","last_peered":"2024-09-16T09:44:52.552415+0000","last_clean":"2024-09-16T09:44:52.552415+0000","last_became_active":"2024-09-16T09:25:05.718134+0000","last_became_peered":"2024-09-16T09:25:05.718134+0000","last_unstale":"2024-09-16T09:44:52.552415+0000","last_undegraded":"2024-09-16T09:44:52.552415+0000","last_fullsized":"2024-09-16T09:44:52.552415+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:42:57.714362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:42:57.714362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:40.690246+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"66'1","reported_seq":1022,"reported_epoch":538,"state":"active+clean","last_fresh":"2024-09-16T09:44:54.557873+0000","last_change":"2024-09-16T09:43:28.772774+0000","last_active":"2024-09-16T09:44:54.557873+0000","last_peered":"2024-09-16T09:44:54.557873+0000","last_clean":"2024-09-16T09:44:54.557873+0000","last_became_active":"2024-09-16T09:25:07.716558+0000","last_became_peered":"2024-09-16T09:25:07.716558+0000","last_unstale":"2024-09-16T09:44:54.557873+0000","last_undegraded":"2024-09-16T09:44:54.557873+0000","last_fullsized":"2024-09-16T09:44:54.557873+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-16T09:43:28.772704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:43:28.772704+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:44:56.277517+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":995,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.551711+0000","last_change":"2024-09-16T09:42:22.732120+0000","last_active":"2024-09-16T09:44:52.551711+0000","last_peered":"2024-09-16T09:44:52.551711+0000","last_clean":"2024-09-16T09:44:52.551711+0000","last_became_active":"2024-09-16T09:25:01.696939+0000","last_became_peered":"2024-09-16T09:25:01.696939+0000","last_unstale":"2024-09-16T09:44:52.551711+0000","last_undegraded":"2024-09-16T09:44:52.551711+0000","last_fullsized":"2024-09-16T09:44:52.551711+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:42:22.732048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:42:22.732048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:10.853456+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"285'7","reported_seq":1005,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.703876+0000","last_change":"2024-09-16T09:35:39.975915+0000","last_active":"2024-09-16T09:44:52.703876+0000","last_peered":"2024-09-16T09:44:52.703876+0000","last_clean":"2024-09-16T09:44:52.703876+0000","last_became_active":"2024-09-16T09:25:03.716136+0000","last_became_peered":"2024-09-16T09:25:03.716136+0000","last_unstale":"2024-09-16T09:44:52.703876+0000","last_undegraded":"2024-09-16T09:44:52.703876+0000","last_fullsized":"2024-09-16T09:44:52.703876+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'7","last_scrub_stamp":"2024-09-16T09:35:39.975840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:35:39.975840+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:17.269033+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":10,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":151,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.551221+0000","last_change":"2024-09-16T09:42:08.915352+0000","last_active":"2024-09-16T09:44:52.551221+0000","last_peered":"2024-09-16T09:44:52.551221+0000","last_clean":"2024-09-16T09:44:52.551221+0000","last_became_active":"2024-09-16T09:42:02.494290+0000","last_became_peered":"2024-09-16T09:42:02.494290+0000","last_unstale":"2024-09-16T09:44:52.551221+0000","last_undegraded":"2024-09-16T09:44:52.551221+0000","last_fullsized":"2024-09-16T09:44:52.551221+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:42:08.915303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:42:08.915303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4],"acting":[6,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1030,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.742833+0000","last_change":"2024-09-16T09:44:52.742833+0000","last_active":"2024-09-16T09:44:52.742833+0000","last_peered":"2024-09-16T09:44:52.742833+0000","last_clean":"2024-09-16T09:44:52.742833+0000","last_became_active":"2024-09-16T09:31:09.275656+0000","last_became_peered":"2024-09-16T09:31:09.275656+0000","last_unstale":"2024-09-16T09:44:52.742833+0000","last_undegraded":"2024-09-16T09:44:52.742833+0000","last_fullsized":"2024-09-16T09:44:52.742833+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:52.742776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:44:52.742776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:05.256885+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1002,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.704464+0000","last_change":"2024-09-16T09:42:57.111424+0000","last_active":"2024-09-16T09:44:52.704464+0000","last_peered":"2024-09-16T09:44:52.704464+0000","last_clean":"2024-09-16T09:44:52.704464+0000","last_became_active":"2024-09-16T09:25:07.720793+0000","last_became_peered":"2024-09-16T09:25:07.720793+0000","last_unstale":"2024-09-16T09:44:52.704464+0000","last_undegraded":"2024-09-16T09:44:52.704464+0000","last_fullsized":"2024-09-16T09:44:52.704464+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:42:57.111229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:42:57.111229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":752,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.551474+0000","last_change":"2024-09-16T09:44:48.756968+0000","last_active":"2024-09-16T09:44:52.551474+0000","last_peered":"2024-09-16T09:44:52.551474+0000","last_clean":"2024-09-16T09:44:52.551474+0000","last_became_active":"2024-09-16T09:30:39.201824+0000","last_became_peered":"2024-09-16T09:30:39.201824+0000","last_unstale":"2024-09-16T09:44:52.551474+0000","last_undegraded":"2024-09-16T09:44:52.551474+0000","last_fullsized":"2024-09-16T09:44:52.551474+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:48.756890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:44:48.756890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:17.988105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"285'27","reported_seq":1070,"reported_epoch":535,"state":"active+clean","last_fresh":"2024-09-16T09:44:50.542777+0000","last_change":"2024-09-16T09:44:24.284901+0000","last_active":"2024-09-16T09:44:50.542777+0000","last_peered":"2024-09-16T09:44:50.542777+0000","last_clean":"2024-09-16T09:44:50.542777+0000","last_became_active":"2024-09-16T09:44:24.283762+0000","last_became_peered":"2024-09-16T09:44:24.283762+0000","last_unstale":"2024-09-16T09:44:50.542777+0000","last_undegraded":"2024-09-16T09:44:50.542777+0000","last_fullsized":"2024-09-16T09:44:50.542777+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'27","last_scrub_stamp":"2024-09-16T09:42:52.891931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:42:52.891931+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:41.042706+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":636,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.704476+0000","last_change":"2024-09-16T09:44:02.300014+0000","last_active":"2024-09-16T09:44:52.704476+0000","last_peered":"2024-09-16T09:44:52.704476+0000","last_clean":"2024-09-16T09:44:52.704476+0000","last_became_active":"2024-09-16T09:33:30.251587+0000","last_became_peered":"2024-09-16T09:33:30.251587+0000","last_unstale":"2024-09-16T09:44:52.704476+0000","last_undegraded":"2024-09-16T09:44:52.704476+0000","last_fullsized":"2024-09-16T09:44:52.704476+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:02.299963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:44:02.299963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:23.146994+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,0],"acting":[4,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":858,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.551857+0000","last_change":"2024-09-16T09:43:16.704220+0000","last_active":"2024-09-16T09:44:52.551857+0000","last_peered":"2024-09-16T09:44:52.551857+0000","last_clean":"2024-09-16T09:44:52.551857+0000","last_became_active":"2024-09-16T09:28:52.974255+0000","last_became_peered":"2024-09-16T09:28:52.974255+0000","last_unstale":"2024-09-16T09:44:52.551857+0000","last_undegraded":"2024-09-16T09:44:52.551857+0000","last_fullsized":"2024-09-16T09:44:52.551857+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:43:16.704155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:43:16.704155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":1031,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.704248+0000","last_change":"2024-09-16T09:44:28.385487+0000","last_active":"2024-09-16T09:44:52.704248+0000","last_peered":"2024-09-16T09:44:52.704248+0000","last_clean":"2024-09-16T09:44:52.704248+0000","last_became_active":"2024-09-16T09:44:24.276029+0000","last_became_peered":"2024-09-16T09:44:24.276029+0000","last_unstale":"2024-09-16T09:44:52.704248+0000","last_undegraded":"2024-09-16T09:44:52.704248+0000","last_fullsized":"2024-09-16T09:44:52.704248+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:28.385444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:44:28.385444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:52.078449+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"285'18","reported_seq":696,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.704538+0000","last_change":"2024-09-16T09:43:16.141437+0000","last_active":"2024-09-16T09:44:52.704538+0000","last_peered":"2024-09-16T09:44:52.704538+0000","last_clean":"2024-09-16T09:44:52.704538+0000","last_became_active":"2024-09-16T09:33:30.251626+0000","last_became_peered":"2024-09-16T09:33:30.251626+0000","last_unstale":"2024-09-16T09:44:52.704538+0000","last_undegraded":"2024-09-16T09:44:52.704538+0000","last_fullsized":"2024-09-16T09:44:52.704538+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'18","last_scrub_stamp":"2024-09-16T09:43:16.141356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:43:16.141356+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":33,"num_read_kb":24,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1007,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.704003+0000","last_change":"2024-09-16T09:44:49.325447+0000","last_active":"2024-09-16T09:44:52.704003+0000","last_peered":"2024-09-16T09:44:52.704003+0000","last_clean":"2024-09-16T09:44:52.704003+0000","last_became_active":"2024-09-16T09:44:24.278117+0000","last_became_peered":"2024-09-16T09:44:24.278117+0000","last_unstale":"2024-09-16T09:44:52.704003+0000","last_undegraded":"2024-09-16T09:44:52.704003+0000","last_fullsized":"2024-09-16T09:44:52.704003+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:49.325384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:44:49.325384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:53.841965+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":1007,"reported_epoch":535,"state":"active+clean","last_fresh":"2024-09-16T09:44:50.625428+0000","last_change":"2024-09-16T09:44:24.284719+0000","last_active":"2024-09-16T09:44:50.625428+0000","last_peered":"2024-09-16T09:44:50.625428+0000","last_clean":"2024-09-16T09:44:50.625428+0000","last_became_active":"2024-09-16T09:44:24.284256+0000","last_became_peered":"2024-09-16T09:44:24.284256+0000","last_unstale":"2024-09-16T09:44:50.625428+0000","last_undegraded":"2024-09-16T09:44:50.625428+0000","last_fullsized":"2024-09-16T09:44:50.625428+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-16T09:42:25.250120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:42:25.250120+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":655,"reported_epoch":537,"state":"active+clean","last_fresh":"2024-09-16T09:44:53.556138+0000","last_change":"2024-09-16T09:41:36.612463+0000","last_active":"2024-09-16T09:44:53.556138+0000","last_peered":"2024-09-16T09:44:53.556138+0000","last_clean":"2024-09-16T09:44:53.556138+0000","last_became_active":"2024-09-16T09:33:30.245048+0000","last_became_peered":"2024-09-16T09:33:30.245048+0000","last_unstale":"2024-09-16T09:44:53.556138+0000","last_undegraded":"2024-09-16T09:44:53.556138+0000","last_fullsized":"2024-09-16T09:44:53.556138+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:41:36.612280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:41:36.612280+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:17.173178+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"285'20","reported_seq":1074,"reported_epoch":538,"state":"active+clean","last_fresh":"2024-09-16T09:44:54.559021+0000","last_change":"2024-09-16T09:39:35.550760+0000","last_active":"2024-09-16T09:44:54.559021+0000","last_peered":"2024-09-16T09:44:54.559021+0000","last_clean":"2024-09-16T09:44:54.559021+0000","last_became_active":"2024-09-16T09:31:09.275050+0000","last_became_peered":"2024-09-16T09:31:09.275050+0000","last_unstale":"2024-09-16T09:44:54.559021+0000","last_undegraded":"2024-09-16T09:44:54.559021+0000","last_fullsized":"2024-09-16T09:44:54.559021+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'20","last_scrub_stamp":"2024-09-16T09:39:35.548333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:39:35.548333+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:13.872588+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"66'8","reported_seq":1288,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.709379+0000","last_change":"2024-09-16T09:44:52.709379+0000","last_active":"2024-09-16T09:44:52.709379+0000","last_peered":"2024-09-16T09:44:52.709379+0000","last_clean":"2024-09-16T09:44:52.709379+0000","last_became_active":"2024-09-16T09:25:05.712516+0000","last_became_peered":"2024-09-16T09:25:05.712516+0000","last_unstale":"2024-09-16T09:44:52.709379+0000","last_undegraded":"2024-09-16T09:44:52.709379+0000","last_fullsized":"2024-09-16T09:44:52.709379+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:44:52.709313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:44:52.709313+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:00.116704+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.a","version":"73'79","reported_seq":955,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.552405+0000","last_change":"2024-09-16T09:44:30.838199+0000","last_active":"2024-09-16T09:44:52.552405+0000","last_peered":"2024-09-16T09:44:52.552405+0000","last_clean":"2024-09-16T09:44:52.552405+0000","last_became_active":"2024-09-16T09:44:24.278444+0000","last_became_peered":"2024-09-16T09:44:24.278444+0000","last_unstale":"2024-09-16T09:44:52.552405+0000","last_undegraded":"2024-09-16T09:44:52.552405+0000","last_fullsized":"2024-09-16T09:44:52.552405+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:44:30.838127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:44:30.838127+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:55.549500+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"285'27","reported_seq":948,"reported_epoch":535,"state":"active+clean","last_fresh":"2024-09-16T09:44:50.543453+0000","last_change":"2024-09-16T09:44:45.023239+0000","last_active":"2024-09-16T09:44:50.543453+0000","last_peered":"2024-09-16T09:44:50.543453+0000","last_clean":"2024-09-16T09:44:50.543453+0000","last_became_active":"2024-09-16T09:44:24.283121+0000","last_became_peered":"2024-09-16T09:44:24.283121+0000","last_unstale":"2024-09-16T09:44:50.543453+0000","last_undegraded":"2024-09-16T09:44:50.543453+0000","last_fullsized":"2024-09-16T09:44:50.543453+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'27","last_scrub_stamp":"2024-09-16T09:44:45.023173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:44:45.023173+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:12.611250+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":741,"reported_epoch":537,"state":"active+clean","last_fresh":"2024-09-16T09:44:53.556271+0000","last_change":"2024-09-16T09:38:35.023594+0000","last_active":"2024-09-16T09:44:53.556271+0000","last_peered":"2024-09-16T09:44:53.556271+0000","last_clean":"2024-09-16T09:44:53.556271+0000","last_became_active":"2024-09-16T09:31:09.274534+0000","last_became_peered":"2024-09-16T09:31:09.274534+0000","last_unstale":"2024-09-16T09:44:53.556271+0000","last_undegraded":"2024-09-16T09:44:53.556271+0000","last_fullsized":"2024-09-16T09:44:53.556271+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:38:35.023518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:38:35.023518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:13.168191+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"66'8","reported_seq":1244,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.704532+0000","last_change":"2024-09-16T09:44:50.326230+0000","last_active":"2024-09-16T09:44:52.704532+0000","last_peered":"2024-09-16T09:44:52.704532+0000","last_clean":"2024-09-16T09:44:52.704532+0000","last_became_active":"2024-09-16T09:25:05.713813+0000","last_became_peered":"2024-09-16T09:25:05.713813+0000","last_unstale":"2024-09-16T09:44:52.704532+0000","last_undegraded":"2024-09-16T09:44:52.704532+0000","last_fullsized":"2024-09-16T09:44:52.704532+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:44:50.326143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:44:50.326143+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:57.080144+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1008,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.552553+0000","last_change":"2024-09-16T09:44:02.749106+0000","last_active":"2024-09-16T09:44:52.552553+0000","last_peered":"2024-09-16T09:44:52.552553+0000","last_clean":"2024-09-16T09:44:52.552553+0000","last_became_active":"2024-09-16T09:25:07.717344+0000","last_became_peered":"2024-09-16T09:25:07.717344+0000","last_unstale":"2024-09-16T09:44:52.552553+0000","last_undegraded":"2024-09-16T09:44:52.552553+0000","last_fullsized":"2024-09-16T09:44:52.552553+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:02.748894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:44:02.748894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:29.230981+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.5","version":"73'79","reported_seq":987,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.704823+0000","last_change":"2024-09-16T09:44:27.405392+0000","last_active":"2024-09-16T09:44:52.704823+0000","last_peered":"2024-09-16T09:44:52.704823+0000","last_clean":"2024-09-16T09:44:52.704823+0000","last_became_active":"2024-09-16T09:44:24.276207+0000","last_became_peered":"2024-09-16T09:44:24.276207+0000","last_unstale":"2024-09-16T09:44:52.704823+0000","last_undegraded":"2024-09-16T09:44:52.704823+0000","last_fullsized":"2024-09-16T09:44:52.704823+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:44:27.405342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:44:27.405342+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:29.853871+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"285'30","reported_seq":973,"reported_epoch":535,"state":"active+clean","last_fresh":"2024-09-16T09:44:50.624818+0000","last_change":"2024-09-16T09:43:50.371697+0000","last_active":"2024-09-16T09:44:50.624818+0000","last_peered":"2024-09-16T09:44:50.624818+0000","last_clean":"2024-09-16T09:44:50.624818+0000","last_became_active":"2024-09-16T09:31:09.270434+0000","last_became_peered":"2024-09-16T09:31:09.270434+0000","last_unstale":"2024-09-16T09:44:50.624818+0000","last_undegraded":"2024-09-16T09:44:50.624818+0000","last_fullsized":"2024-09-16T09:44:50.624818+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'30","last_scrub_stamp":"2024-09-16T09:43:50.371625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:43:50.371625+0000","objects_scrubbed":8,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:03.401709+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":51,"num_read_kb":39,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":853,"reported_epoch":535,"state":"active+clean","last_fresh":"2024-09-16T09:44:50.625885+0000","last_change":"2024-09-16T09:44:00.511038+0000","last_active":"2024-09-16T09:44:50.625885+0000","last_peered":"2024-09-16T09:44:50.625885+0000","last_clean":"2024-09-16T09:44:50.625885+0000","last_became_active":"2024-09-16T09:28:52.983543+0000","last_became_peered":"2024-09-16T09:28:52.983543+0000","last_unstale":"2024-09-16T09:44:50.625885+0000","last_undegraded":"2024-09-16T09:44:50.625885+0000","last_fullsized":"2024-09-16T09:44:50.625885+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:00.510827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:44:00.510827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:29.877213+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1020,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.703640+0000","last_change":"2024-09-16T09:42:09.103716+0000","last_active":"2024-09-16T09:44:52.703640+0000","last_peered":"2024-09-16T09:44:52.703640+0000","last_clean":"2024-09-16T09:44:52.703640+0000","last_became_active":"2024-09-16T09:31:09.268864+0000","last_became_peered":"2024-09-16T09:31:09.268864+0000","last_unstale":"2024-09-16T09:44:52.703640+0000","last_undegraded":"2024-09-16T09:44:52.703640+0000","last_fullsized":"2024-09-16T09:44:52.703640+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:42:09.103657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:42:09.103657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1035,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.703998+0000","last_change":"2024-09-16T09:44:29.376932+0000","last_active":"2024-09-16T09:44:52.703998+0000","last_peered":"2024-09-16T09:44:52.703998+0000","last_clean":"2024-09-16T09:44:52.703998+0000","last_became_active":"2024-09-16T09:44:24.276365+0000","last_became_peered":"2024-09-16T09:44:24.276365+0000","last_unstale":"2024-09-16T09:44:52.703998+0000","last_undegraded":"2024-09-16T09:44:52.703998+0000","last_fullsized":"2024-09-16T09:44:52.703998+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:29.376874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:44:29.376874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:30.303288+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"73'79","reported_seq":749,"reported_epoch":537,"state":"active+clean","last_fresh":"2024-09-16T09:44:53.558286+0000","last_change":"2024-09-16T09:44:21.229944+0000","last_active":"2024-09-16T09:44:53.558286+0000","last_peered":"2024-09-16T09:44:53.558286+0000","last_clean":"2024-09-16T09:44:53.558286+0000","last_became_active":"2024-09-16T09:31:09.275139+0000","last_became_peered":"2024-09-16T09:31:09.275139+0000","last_unstale":"2024-09-16T09:44:53.558286+0000","last_undegraded":"2024-09-16T09:44:53.558286+0000","last_fullsized":"2024-09-16T09:44:53.558286+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:44:21.229868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:44:21.229868+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:39.951698+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"285'28","reported_seq":1060,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.703670+0000","last_change":"2024-09-16T09:43:18.126451+0000","last_active":"2024-09-16T09:44:52.703670+0000","last_peered":"2024-09-16T09:44:52.703670+0000","last_clean":"2024-09-16T09:44:52.703670+0000","last_became_active":"2024-09-16T09:25:03.708830+0000","last_became_peered":"2024-09-16T09:25:03.708830+0000","last_unstale":"2024-09-16T09:44:52.703670+0000","last_undegraded":"2024-09-16T09:44:52.703670+0000","last_fullsized":"2024-09-16T09:44:52.703670+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'28","last_scrub_stamp":"2024-09-16T09:43:18.126360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:43:18.126360+0000","objects_scrubbed":7,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":1035,"reported_epoch":538,"state":"active+clean","last_fresh":"2024-09-16T09:44:54.559303+0000","last_change":"2024-09-16T09:40:31.037325+0000","last_active":"2024-09-16T09:44:54.559303+0000","last_peered":"2024-09-16T09:44:54.559303+0000","last_clean":"2024-09-16T09:44:54.559303+0000","last_became_active":"2024-09-16T09:25:01.702992+0000","last_became_peered":"2024-09-16T09:25:01.702992+0000","last_unstale":"2024-09-16T09:44:54.559303+0000","last_undegraded":"2024-09-16T09:44:54.559303+0000","last_fullsized":"2024-09-16T09:44:54.559303+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:40:31.037258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:40:31.037258+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:41.195033+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"235'12","reported_seq":800,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.703443+0000","last_change":"2024-09-16T09:42:27.113239+0000","last_active":"2024-09-16T09:44:52.703443+0000","last_peered":"2024-09-16T09:44:52.703443+0000","last_clean":"2024-09-16T09:44:52.703443+0000","last_became_active":"2024-09-16T09:33:30.251047+0000","last_became_peered":"2024-09-16T09:33:30.251047+0000","last_unstale":"2024-09-16T09:44:52.703443+0000","last_undegraded":"2024-09-16T09:44:52.703443+0000","last_fullsized":"2024-09-16T09:44:52.703443+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"235'12","last_scrub_stamp":"2024-09-16T09:42:27.113174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:42:27.113174+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1008,"reported_epoch":535,"state":"active+clean","last_fresh":"2024-09-16T09:44:50.625126+0000","last_change":"2024-09-16T09:44:29.455025+0000","last_active":"2024-09-16T09:44:50.625126+0000","last_peered":"2024-09-16T09:44:50.625126+0000","last_clean":"2024-09-16T09:44:50.625126+0000","last_became_active":"2024-09-16T09:44:24.277003+0000","last_became_peered":"2024-09-16T09:44:24.277003+0000","last_unstale":"2024-09-16T09:44:50.625126+0000","last_undegraded":"2024-09-16T09:44:50.625126+0000","last_fullsized":"2024-09-16T09:44:50.625126+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:29.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:44:29.454958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:30.251581+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"73'79","reported_seq":1086,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.551733+0000","last_change":"2024-09-16T09:43:55.930749+0000","last_active":"2024-09-16T09:44:52.551733+0000","last_peered":"2024-09-16T09:44:52.551733+0000","last_clean":"2024-09-16T09:44:52.551733+0000","last_became_active":"2024-09-16T09:31:09.274707+0000","last_became_peered":"2024-09-16T09:31:09.274707+0000","last_unstale":"2024-09-16T09:44:52.551733+0000","last_undegraded":"2024-09-16T09:44:52.551733+0000","last_fullsized":"2024-09-16T09:44:52.551733+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:43:55.930694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:43:55.930694+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:15.421892+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"285'20","reported_seq":687,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.553039+0000","last_change":"2024-09-16T09:38:32.122419+0000","last_active":"2024-09-16T09:44:52.553039+0000","last_peered":"2024-09-16T09:44:52.553039+0000","last_clean":"2024-09-16T09:44:52.553039+0000","last_became_active":"2024-09-16T09:33:30.248453+0000","last_became_peered":"2024-09-16T09:33:30.248453+0000","last_unstale":"2024-09-16T09:44:52.553039+0000","last_undegraded":"2024-09-16T09:44:52.553039+0000","last_fullsized":"2024-09-16T09:44:52.553039+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'20","last_scrub_stamp":"2024-09-16T09:38:32.122303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:38:32.122303+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:08.800937+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":1030,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.552882+0000","last_change":"2024-09-16T09:44:25.924933+0000","last_active":"2024-09-16T09:44:52.552882+0000","last_peered":"2024-09-16T09:44:52.552882+0000","last_clean":"2024-09-16T09:44:52.552882+0000","last_became_active":"2024-09-16T09:44:24.280657+0000","last_became_peered":"2024-09-16T09:44:24.280657+0000","last_unstale":"2024-09-16T09:44:52.552882+0000","last_undegraded":"2024-09-16T09:44:52.552882+0000","last_fullsized":"2024-09-16T09:44:52.552882+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:25.924858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:44:25.924858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:35.232847+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"66'8","reported_seq":1257,"reported_epoch":538,"state":"active+clean","last_fresh":"2024-09-16T09:44:54.559754+0000","last_change":"2024-09-16T09:44:00.808193+0000","last_active":"2024-09-16T09:44:54.559754+0000","last_peered":"2024-09-16T09:44:54.559754+0000","last_clean":"2024-09-16T09:44:54.559754+0000","last_became_active":"2024-09-16T09:25:05.714507+0000","last_became_peered":"2024-09-16T09:25:05.714507+0000","last_unstale":"2024-09-16T09:44:54.559754+0000","last_undegraded":"2024-09-16T09:44:54.559754+0000","last_fullsized":"2024-09-16T09:44:54.559754+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:44:00.808121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:44:00.808121+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:28.025271+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1039,"reported_epoch":538,"state":"active+clean","last_fresh":"2024-09-16T09:44:54.558017+0000","last_change":"2024-09-16T09:44:25.125306+0000","last_active":"2024-09-16T09:44:54.558017+0000","last_peered":"2024-09-16T09:44:54.558017+0000","last_clean":"2024-09-16T09:44:54.558017+0000","last_became_active":"2024-09-16T09:44:24.269736+0000","last_became_peered":"2024-09-16T09:44:24.269736+0000","last_unstale":"2024-09-16T09:44:54.558017+0000","last_undegraded":"2024-09-16T09:44:54.558017+0000","last_fullsized":"2024-09-16T09:44:54.558017+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:25.125254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:44:25.125254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:42.194714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"73'79","reported_seq":1043,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.551667+0000","last_change":"2024-09-16T09:43:50.904918+0000","last_active":"2024-09-16T09:44:52.551667+0000","last_peered":"2024-09-16T09:44:52.551667+0000","last_clean":"2024-09-16T09:44:52.551667+0000","last_became_active":"2024-09-16T09:31:09.274565+0000","last_became_peered":"2024-09-16T09:31:09.274565+0000","last_unstale":"2024-09-16T09:44:52.551667+0000","last_undegraded":"2024-09-16T09:44:52.551667+0000","last_fullsized":"2024-09-16T09:44:52.551667+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:43:50.904841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:43:50.904841+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:09.506473+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"285'29","reported_seq":1057,"reported_epoch":538,"state":"active+clean","last_fresh":"2024-09-16T09:44:54.559971+0000","last_change":"2024-09-16T09:42:24.900527+0000","last_active":"2024-09-16T09:44:54.559971+0000","last_peered":"2024-09-16T09:44:54.559971+0000","last_clean":"2024-09-16T09:44:54.559971+0000","last_became_active":"2024-09-16T09:25:03.714797+0000","last_became_peered":"2024-09-16T09:25:03.714797+0000","last_unstale":"2024-09-16T09:44:54.559971+0000","last_undegraded":"2024-09-16T09:44:54.559971+0000","last_fullsized":"2024-09-16T09:44:54.559971+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'29","last_scrub_stamp":"2024-09-16T09:42:24.900426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:42:24.900426+0000","objects_scrubbed":8,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":54,"num_read_kb":40,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":648,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.551561+0000","last_change":"2024-09-16T09:44:27.885527+0000","last_active":"2024-09-16T09:44:52.551561+0000","last_peered":"2024-09-16T09:44:52.551561+0000","last_clean":"2024-09-16T09:44:52.551561+0000","last_became_active":"2024-09-16T09:44:24.279029+0000","last_became_peered":"2024-09-16T09:44:24.279029+0000","last_unstale":"2024-09-16T09:44:52.551561+0000","last_undegraded":"2024-09-16T09:44:52.551561+0000","last_fullsized":"2024-09-16T09:44:52.551561+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:27.885460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:44:27.885460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:50.933082+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,2],"acting":[5,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1030,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.551842+0000","last_change":"2024-09-16T09:44:51.704537+0000","last_active":"2024-09-16T09:44:52.551842+0000","last_peered":"2024-09-16T09:44:52.551842+0000","last_clean":"2024-09-16T09:44:52.551842+0000","last_became_active":"2024-09-16T09:25:05.708663+0000","last_became_peered":"2024-09-16T09:25:05.708663+0000","last_unstale":"2024-09-16T09:44:52.551842+0000","last_undegraded":"2024-09-16T09:44:52.551842+0000","last_fullsized":"2024-09-16T09:44:52.551842+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:51.704482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:44:51.704482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:16.177847+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":656,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.551356+0000","last_change":"2024-09-16T09:44:25.629415+0000","last_active":"2024-09-16T09:44:52.551356+0000","last_peered":"2024-09-16T09:44:52.551356+0000","last_clean":"2024-09-16T09:44:52.551356+0000","last_became_active":"2024-09-16T09:44:24.277522+0000","last_became_peered":"2024-09-16T09:44:24.277522+0000","last_unstale":"2024-09-16T09:44:52.551356+0000","last_undegraded":"2024-09-16T09:44:52.551356+0000","last_fullsized":"2024-09-16T09:44:52.551356+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:25.629370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:44:25.629370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:54.323231+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,2],"acting":[6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"73'79","reported_seq":987,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.550993+0000","last_change":"2024-09-16T09:44:29.831921+0000","last_active":"2024-09-16T09:44:52.550993+0000","last_peered":"2024-09-16T09:44:52.550993+0000","last_clean":"2024-09-16T09:44:52.550993+0000","last_became_active":"2024-09-16T09:44:24.279593+0000","last_became_peered":"2024-09-16T09:44:24.279593+0000","last_unstale":"2024-09-16T09:44:52.550993+0000","last_undegraded":"2024-09-16T09:44:52.550993+0000","last_fullsized":"2024-09-16T09:44:52.550993+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:44:29.831857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:44:29.831857+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:59.570916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"519'44","reported_seq":1412,"reported_epoch":537,"state":"active+clean","last_fresh":"2024-09-16T09:44:53.557707+0000","last_change":"2024-09-16T09:44:30.314706+0000","last_active":"2024-09-16T09:44:53.557707+0000","last_peered":"2024-09-16T09:44:53.557707+0000","last_clean":"2024-09-16T09:44:53.557707+0000","last_became_active":"2024-09-16T09:44:24.280130+0000","last_became_peered":"2024-09-16T09:44:24.280130+0000","last_unstale":"2024-09-16T09:44:53.557707+0000","last_undegraded":"2024-09-16T09:44:53.557707+0000","last_fullsized":"2024-09-16T09:44:53.557707+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"519'44","last_scrub_stamp":"2024-09-16T09:44:30.314606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:44:30.314606+0000","objects_scrubbed":10,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:34.762411+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":66,"num_read_kb":51,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":1022,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.704277+0000","last_change":"2024-09-16T09:41:40.124915+0000","last_active":"2024-09-16T09:44:52.704277+0000","last_peered":"2024-09-16T09:44:52.704277+0000","last_clean":"2024-09-16T09:44:52.704277+0000","last_became_active":"2024-09-16T09:25:01.703076+0000","last_became_peered":"2024-09-16T09:25:01.703076+0000","last_unstale":"2024-09-16T09:44:52.704277+0000","last_undegraded":"2024-09-16T09:44:52.704277+0000","last_fullsized":"2024-09-16T09:44:52.704277+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:40.124830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:41:40.124830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:07.248004+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":1031,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.553115+0000","last_change":"2024-09-16T09:43:52.704491+0000","last_active":"2024-09-16T09:44:52.553115+0000","last_peered":"2024-09-16T09:44:52.553115+0000","last_clean":"2024-09-16T09:44:52.553115+0000","last_became_active":"2024-09-16T09:25:00.696237+0000","last_became_peered":"2024-09-16T09:25:00.696237+0000","last_unstale":"2024-09-16T09:44:52.553115+0000","last_undegraded":"2024-09-16T09:44:52.553115+0000","last_fullsized":"2024-09-16T09:44:52.553115+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-16T09:43:52.704431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:43:52.704431+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:20.772940+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.0025137430000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1002,"reported_epoch":538,"state":"active+clean","last_fresh":"2024-09-16T09:44:54.558190+0000","last_change":"2024-09-16T09:44:03.793464+0000","last_active":"2024-09-16T09:44:54.558190+0000","last_peered":"2024-09-16T09:44:54.558190+0000","last_clean":"2024-09-16T09:44:54.558190+0000","last_became_active":"2024-09-16T09:25:05.717591+0000","last_became_peered":"2024-09-16T09:25:05.717591+0000","last_unstale":"2024-09-16T09:44:54.558190+0000","last_undegraded":"2024-09-16T09:44:54.558190+0000","last_fullsized":"2024-09-16T09:44:54.558190+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:03.793400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:44:03.793400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:29.631956+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":749,"reported_epoch":537,"state":"active+clean","last_fresh":"2024-09-16T09:44:53.558528+0000","last_change":"2024-09-16T09:44:35.215569+0000","last_active":"2024-09-16T09:44:53.558528+0000","last_peered":"2024-09-16T09:44:53.558528+0000","last_clean":"2024-09-16T09:44:53.558528+0000","last_became_active":"2024-09-16T09:31:09.272609+0000","last_became_peered":"2024-09-16T09:31:09.272609+0000","last_unstale":"2024-09-16T09:44:53.558528+0000","last_undegraded":"2024-09-16T09:44:53.558528+0000","last_fullsized":"2024-09-16T09:44:53.558528+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:35.215527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:44:35.215527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:03.125091+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"73'79","reported_seq":341,"reported_epoch":535,"state":"active+clean","last_fresh":"2024-09-16T09:44:50.624617+0000","last_change":"2024-09-16T09:44:22.398722+0000","last_active":"2024-09-16T09:44:50.624617+0000","last_peered":"2024-09-16T09:44:50.624617+0000","last_clean":"2024-09-16T09:44:50.624617+0000","last_became_active":"2024-09-16T09:38:10.556320+0000","last_became_peered":"2024-09-16T09:38:10.556320+0000","last_unstale":"2024-09-16T09:44:50.624617+0000","last_undegraded":"2024-09-16T09:44:50.624617+0000","last_fullsized":"2024-09-16T09:44:50.624617+0000","mapping_epoch":386,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":387,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:44:22.398656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:44:22.398656+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:28.557065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"285'17","reported_seq":736,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.553063+0000","last_change":"2024-09-16T09:43:49.706772+0000","last_active":"2024-09-16T09:44:52.553063+0000","last_peered":"2024-09-16T09:44:52.553063+0000","last_clean":"2024-09-16T09:44:52.553063+0000","last_became_active":"2024-09-16T09:37:33.349009+0000","last_became_peered":"2024-09-16T09:37:33.349009+0000","last_unstale":"2024-09-16T09:44:52.553063+0000","last_undegraded":"2024-09-16T09:44:52.553063+0000","last_fullsized":"2024-09-16T09:44:52.553063+0000","mapping_epoch":367,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":368,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'17","last_scrub_stamp":"2024-09-16T09:43:49.706685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:43:49.706685+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:44:55.741272+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":35,"num_read_kb":24,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":656,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.551253+0000","last_change":"2024-09-16T09:44:30.666378+0000","last_active":"2024-09-16T09:44:52.551253+0000","last_peered":"2024-09-16T09:44:52.551253+0000","last_clean":"2024-09-16T09:44:52.551253+0000","last_became_active":"2024-09-16T09:44:24.270134+0000","last_became_peered":"2024-09-16T09:44:24.270134+0000","last_unstale":"2024-09-16T09:44:52.551253+0000","last_undegraded":"2024-09-16T09:44:52.551253+0000","last_fullsized":"2024-09-16T09:44:52.551253+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:30.666333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:44:30.666333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:36.645744+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,2],"acting":[6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":698,"reported_epoch":538,"state":"active+clean","last_fresh":"2024-09-16T09:44:54.560118+0000","last_change":"2024-09-16T09:44:24.280992+0000","last_active":"2024-09-16T09:44:54.560118+0000","last_peered":"2024-09-16T09:44:54.560118+0000","last_clean":"2024-09-16T09:44:54.560118+0000","last_became_active":"2024-09-16T09:44:24.279158+0000","last_became_peered":"2024-09-16T09:44:24.279158+0000","last_unstale":"2024-09-16T09:44:54.560118+0000","last_undegraded":"2024-09-16T09:44:54.560118+0000","last_fullsized":"2024-09-16T09:44:54.560118+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:41:42.038861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:41:42.038861+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":938,"reported_epoch":535,"state":"active+clean","last_fresh":"2024-09-16T09:44:50.543486+0000","last_change":"2024-09-16T09:44:24.286071+0000","last_active":"2024-09-16T09:44:50.543486+0000","last_peered":"2024-09-16T09:44:50.543486+0000","last_clean":"2024-09-16T09:44:50.543486+0000","last_became_active":"2024-09-16T09:44:24.285805+0000","last_became_peered":"2024-09-16T09:44:24.285805+0000","last_unstale":"2024-09-16T09:44:50.543486+0000","last_undegraded":"2024-09-16T09:44:50.543486+0000","last_fullsized":"2024-09-16T09:44:50.543486+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:45.549186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:41:45.549186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:39.079232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1005,"reported_epoch":535,"state":"active+clean","last_fresh":"2024-09-16T09:44:50.625387+0000","last_change":"2024-09-16T09:44:48.555675+0000","last_active":"2024-09-16T09:44:50.625387+0000","last_peered":"2024-09-16T09:44:50.625387+0000","last_clean":"2024-09-16T09:44:50.625387+0000","last_became_active":"2024-09-16T09:25:07.723282+0000","last_became_peered":"2024-09-16T09:25:07.723282+0000","last_unstale":"2024-09-16T09:44:50.625387+0000","last_undegraded":"2024-09-16T09:44:50.625387+0000","last_fullsized":"2024-09-16T09:44:50.625387+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:48.555614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:44:48.555614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:18.258254+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"73'79","reported_seq":726,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.551561+0000","last_change":"2024-09-16T09:43:53.953716+0000","last_active":"2024-09-16T09:44:52.551561+0000","last_peered":"2024-09-16T09:44:52.551561+0000","last_clean":"2024-09-16T09:44:52.551561+0000","last_became_active":"2024-09-16T09:33:30.249309+0000","last_became_peered":"2024-09-16T09:33:30.249309+0000","last_unstale":"2024-09-16T09:44:52.551561+0000","last_undegraded":"2024-09-16T09:44:52.551561+0000","last_fullsized":"2024-09-16T09:44:52.551561+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:43:53.953557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:43:53.953557+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:12.516809+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.6","version":"285'24","reported_seq":1061,"reported_epoch":538,"state":"active+clean","last_fresh":"2024-09-16T09:44:54.559123+0000","last_change":"2024-09-16T09:44:24.278226+0000","last_active":"2024-09-16T09:44:54.559123+0000","last_peered":"2024-09-16T09:44:54.559123+0000","last_clean":"2024-09-16T09:44:54.559123+0000","last_became_active":"2024-09-16T09:44:24.277878+0000","last_became_peered":"2024-09-16T09:44:24.277878+0000","last_unstale":"2024-09-16T09:44:54.559123+0000","last_undegraded":"2024-09-16T09:44:54.559123+0000","last_fullsized":"2024-09-16T09:44:54.559123+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'24","last_scrub_stamp":"2024-09-16T09:42:11.911889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:42:11.911889+0000","objects_scrubbed":6,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":36,"num_read_kb":30,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"59'1","reported_seq":1049,"reported_epoch":538,"state":"active+clean","last_fresh":"2024-09-16T09:44:54.559845+0000","last_change":"2024-09-16T09:42:56.896466+0000","last_active":"2024-09-16T09:44:54.559845+0000","last_peered":"2024-09-16T09:44:54.559845+0000","last_clean":"2024-09-16T09:44:54.559845+0000","last_became_active":"2024-09-16T09:31:09.269627+0000","last_became_peered":"2024-09-16T09:31:09.269627+0000","last_unstale":"2024-09-16T09:44:54.559845+0000","last_undegraded":"2024-09-16T09:44:54.559845+0000","last_fullsized":"2024-09-16T09:44:54.559845+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:42:56.896256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:42:56.896256+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1986,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"67'5","reported_seq":2203,"reported_epoch":535,"state":"active+clean","last_fresh":"2024-09-16T09:44:50.740440+0000","last_change":"2024-09-16T09:41:43.153303+0000","last_active":"2024-09-16T09:44:50.740440+0000","last_peered":"2024-09-16T09:44:50.740440+0000","last_clean":"2024-09-16T09:44:50.740440+0000","last_became_active":"2024-09-16T09:25:00.696546+0000","last_became_peered":"2024-09-16T09:25:00.696546+0000","last_unstale":"2024-09-16T09:44:50.740440+0000","last_undegraded":"2024-09-16T09:44:50.740440+0000","last_fullsized":"2024-09-16T09:44:50.740440+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'5","last_scrub_stamp":"2024-09-16T09:41:43.153174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:41:43.153174+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:42.518308+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0.0059770420000000001,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1183,"num_read_kb":1178,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":989,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.553171+0000","last_change":"2024-09-16T09:44:19.980692+0000","last_active":"2024-09-16T09:44:52.553171+0000","last_peered":"2024-09-16T09:44:52.553171+0000","last_clean":"2024-09-16T09:44:52.553171+0000","last_became_active":"2024-09-16T09:25:05.714704+0000","last_became_peered":"2024-09-16T09:25:05.714704+0000","last_unstale":"2024-09-16T09:44:52.553171+0000","last_undegraded":"2024-09-16T09:44:52.553171+0000","last_fullsized":"2024-09-16T09:44:52.553171+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:19.980625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:44:19.980625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:20.184735+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":923,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.551800+0000","last_change":"2024-09-16T09:44:04.828342+0000","last_active":"2024-09-16T09:44:52.551800+0000","last_peered":"2024-09-16T09:44:52.551800+0000","last_clean":"2024-09-16T09:44:52.551800+0000","last_became_active":"2024-09-16T09:33:30.245852+0000","last_became_peered":"2024-09-16T09:33:30.245852+0000","last_unstale":"2024-09-16T09:44:52.551800+0000","last_undegraded":"2024-09-16T09:44:52.551800+0000","last_fullsized":"2024-09-16T09:44:52.551800+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:04.828272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:44:04.828272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:28.332272+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.5","version":"285'28","reported_seq":1297,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.552570+0000","last_change":"2024-09-16T09:43:47.926939+0000","last_active":"2024-09-16T09:44:52.552570+0000","last_peered":"2024-09-16T09:44:52.552570+0000","last_clean":"2024-09-16T09:44:52.552570+0000","last_became_active":"2024-09-16T09:25:03.714118+0000","last_became_peered":"2024-09-16T09:25:03.714118+0000","last_unstale":"2024-09-16T09:44:52.552570+0000","last_undegraded":"2024-09-16T09:44:52.552570+0000","last_fullsized":"2024-09-16T09:44:52.552570+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'28","last_scrub_stamp":"2024-09-16T09:43:47.926855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:43:47.926855+0000","objects_scrubbed":8,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:44:54.826374+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":43,"num_read_kb":33,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":863,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.553047+0000","last_change":"2024-09-16T09:44:34.758789+0000","last_active":"2024-09-16T09:44:52.553047+0000","last_peered":"2024-09-16T09:44:52.553047+0000","last_clean":"2024-09-16T09:44:52.553047+0000","last_became_active":"2024-09-16T09:31:09.270196+0000","last_became_peered":"2024-09-16T09:31:09.270196+0000","last_unstale":"2024-09-16T09:44:52.553047+0000","last_undegraded":"2024-09-16T09:44:52.553047+0000","last_fullsized":"2024-09-16T09:44:52.553047+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:34.758733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:44:34.758733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:37.220493+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"73'79","reported_seq":1089,"reported_epoch":535,"state":"active+clean","last_fresh":"2024-09-16T09:44:50.625068+0000","last_change":"2024-09-16T09:42:56.661708+0000","last_active":"2024-09-16T09:44:50.625068+0000","last_peered":"2024-09-16T09:44:50.625068+0000","last_clean":"2024-09-16T09:44:50.625068+0000","last_became_active":"2024-09-16T09:30:08.870394+0000","last_became_peered":"2024-09-16T09:30:08.870394+0000","last_unstale":"2024-09-16T09:44:50.625068+0000","last_undegraded":"2024-09-16T09:44:50.625068+0000","last_fullsized":"2024-09-16T09:44:50.625068+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:42:56.661377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:42:56.661377+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1012,"reported_epoch":535,"state":"active+clean","last_fresh":"2024-09-16T09:44:50.625564+0000","last_change":"2024-09-16T09:44:24.285202+0000","last_active":"2024-09-16T09:44:50.625564+0000","last_peered":"2024-09-16T09:44:50.625564+0000","last_clean":"2024-09-16T09:44:50.625564+0000","last_became_active":"2024-09-16T09:44:24.284362+0000","last_became_peered":"2024-09-16T09:44:24.284362+0000","last_unstale":"2024-09-16T09:44:50.625564+0000","last_undegraded":"2024-09-16T09:44:50.625564+0000","last_fullsized":"2024-09-16T09:44:50.625564+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:42:11.246697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:42:11.246697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1001,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.552874+0000","last_change":"2024-09-16T09:44:46.680178+0000","last_active":"2024-09-16T09:44:52.552874+0000","last_peered":"2024-09-16T09:44:52.552874+0000","last_clean":"2024-09-16T09:44:52.552874+0000","last_became_active":"2024-09-16T09:25:07.714281+0000","last_became_peered":"2024-09-16T09:25:07.714281+0000","last_unstale":"2024-09-16T09:44:52.552874+0000","last_undegraded":"2024-09-16T09:44:52.552874+0000","last_fullsized":"2024-09-16T09:44:52.552874+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:46.680120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:44:46.680120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:08.327463+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.b","version":"73'79","reported_seq":841,"reported_epoch":538,"state":"active+clean","last_fresh":"2024-09-16T09:44:54.559401+0000","last_change":"2024-09-16T09:42:26.898435+0000","last_active":"2024-09-16T09:44:54.559401+0000","last_peered":"2024-09-16T09:44:54.559401+0000","last_clean":"2024-09-16T09:44:54.559401+0000","last_became_active":"2024-09-16T09:33:30.251933+0000","last_became_peered":"2024-09-16T09:33:30.251933+0000","last_unstale":"2024-09-16T09:44:54.559401+0000","last_undegraded":"2024-09-16T09:44:54.559401+0000","last_fullsized":"2024-09-16T09:44:54.559401+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:42:26.898235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:42:26.898235+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"285'19","reported_seq":1089,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.704597+0000","last_change":"2024-09-16T09:43:22.166919+0000","last_active":"2024-09-16T09:44:52.704597+0000","last_peered":"2024-09-16T09:44:52.704597+0000","last_clean":"2024-09-16T09:44:52.704597+0000","last_became_active":"2024-09-16T09:37:33.348433+0000","last_became_peered":"2024-09-16T09:37:33.348433+0000","last_unstale":"2024-09-16T09:44:52.704597+0000","last_undegraded":"2024-09-16T09:44:52.704597+0000","last_fullsized":"2024-09-16T09:44:52.704597+0000","mapping_epoch":367,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":368,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'19","last_scrub_stamp":"2024-09-16T09:43:22.166834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:43:22.166834+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":1009,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.704031+0000","last_change":"2024-09-16T09:44:24.277287+0000","last_active":"2024-09-16T09:44:52.704031+0000","last_peered":"2024-09-16T09:44:52.704031+0000","last_clean":"2024-09-16T09:44:52.704031+0000","last_became_active":"2024-09-16T09:44:24.276997+0000","last_became_peered":"2024-09-16T09:44:24.276997+0000","last_unstale":"2024-09-16T09:44:52.704031+0000","last_undegraded":"2024-09-16T09:44:52.704031+0000","last_fullsized":"2024-09-16T09:44:52.704031+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:42:18.132091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:42:18.132091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":980,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.552738+0000","last_change":"2024-09-16T09:42:18.730769+0000","last_active":"2024-09-16T09:44:52.552738+0000","last_peered":"2024-09-16T09:44:52.552738+0000","last_clean":"2024-09-16T09:44:52.552738+0000","last_became_active":"2024-09-16T09:25:05.711964+0000","last_became_peered":"2024-09-16T09:25:05.711964+0000","last_unstale":"2024-09-16T09:44:52.552738+0000","last_undegraded":"2024-09-16T09:44:52.552738+0000","last_fullsized":"2024-09-16T09:44:52.552738+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:42:18.730723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:42:18.730723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:14.838460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":883,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.551739+0000","last_change":"2024-09-16T09:44:37.625415+0000","last_active":"2024-09-16T09:44:52.551739+0000","last_peered":"2024-09-16T09:44:52.551739+0000","last_clean":"2024-09-16T09:44:52.551739+0000","last_became_active":"2024-09-16T09:31:09.278618+0000","last_became_peered":"2024-09-16T09:31:09.278618+0000","last_unstale":"2024-09-16T09:44:52.551739+0000","last_undegraded":"2024-09-16T09:44:52.551739+0000","last_fullsized":"2024-09-16T09:44:52.551739+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:42:09.944311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:42:09.944311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:06.727839+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.8","version":"73'79","reported_seq":359,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.550888+0000","last_change":"2024-09-16T09:44:31.814201+0000","last_active":"2024-09-16T09:44:52.550888+0000","last_peered":"2024-09-16T09:44:52.550888+0000","last_clean":"2024-09-16T09:44:52.550888+0000","last_became_active":"2024-09-16T09:44:24.270354+0000","last_became_peered":"2024-09-16T09:44:24.270354+0000","last_unstale":"2024-09-16T09:44:52.550888+0000","last_undegraded":"2024-09-16T09:44:52.550888+0000","last_fullsized":"2024-09-16T09:44:52.550888+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:44:31.814152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:44:31.814152+0000","objects_scrubbed":1,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:39.578957+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262688,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"285'31","reported_seq":1069,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.704687+0000","last_change":"2024-09-16T09:44:24.277703+0000","last_active":"2024-09-16T09:44:52.704687+0000","last_peered":"2024-09-16T09:44:52.704687+0000","last_clean":"2024-09-16T09:44:52.704687+0000","last_became_active":"2024-09-16T09:44:24.277359+0000","last_became_peered":"2024-09-16T09:44:24.277359+0000","last_unstale":"2024-09-16T09:44:52.704687+0000","last_undegraded":"2024-09-16T09:44:52.704687+0000","last_fullsized":"2024-09-16T09:44:52.704687+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'31","last_scrub_stamp":"2024-09-16T09:42:07.153276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:42:07.153276+0000","objects_scrubbed":8,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":50,"num_read_kb":40,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":1014,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.552232+0000","last_change":"2024-09-16T09:42:53.627570+0000","last_active":"2024-09-16T09:44:52.552232+0000","last_peered":"2024-09-16T09:44:52.552232+0000","last_clean":"2024-09-16T09:44:52.552232+0000","last_became_active":"2024-09-16T09:25:01.703355+0000","last_became_peered":"2024-09-16T09:25:01.703355+0000","last_unstale":"2024-09-16T09:44:52.552232+0000","last_undegraded":"2024-09-16T09:44:52.552232+0000","last_fullsized":"2024-09-16T09:44:52.552232+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:42:53.627505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:42:53.627505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:07.742399+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":645,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.703898+0000","last_change":"2024-09-16T09:44:47.324175+0000","last_active":"2024-09-16T09:44:52.703898+0000","last_peered":"2024-09-16T09:44:52.703898+0000","last_clean":"2024-09-16T09:44:52.703898+0000","last_became_active":"2024-09-16T09:33:30.251186+0000","last_became_peered":"2024-09-16T09:33:30.251186+0000","last_unstale":"2024-09-16T09:44:52.703898+0000","last_undegraded":"2024-09-16T09:44:52.703898+0000","last_fullsized":"2024-09-16T09:44:52.703898+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:47.324096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:44:47.324096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:02.767003+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,0],"acting":[4,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":956,"reported_epoch":537,"state":"active+clean","last_fresh":"2024-09-16T09:44:53.558407+0000","last_change":"2024-09-16T09:44:25.180283+0000","last_active":"2024-09-16T09:44:53.558407+0000","last_peered":"2024-09-16T09:44:53.558407+0000","last_clean":"2024-09-16T09:44:53.558407+0000","last_became_active":"2024-09-16T09:44:24.273561+0000","last_became_peered":"2024-09-16T09:44:24.273561+0000","last_unstale":"2024-09-16T09:44:53.558407+0000","last_undegraded":"2024-09-16T09:44:53.558407+0000","last_fullsized":"2024-09-16T09:44:53.558407+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:25.180239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:44:25.180239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:44.445043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4],"acting":[3,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.9","version":"73'79","reported_seq":1076,"reported_epoch":535,"state":"active+clean","last_fresh":"2024-09-16T09:44:50.624997+0000","last_change":"2024-09-16T09:44:24.279925+0000","last_active":"2024-09-16T09:44:50.624997+0000","last_peered":"2024-09-16T09:44:50.624997+0000","last_clean":"2024-09-16T09:44:50.624997+0000","last_became_active":"2024-09-16T09:44:24.278896+0000","last_became_peered":"2024-09-16T09:44:24.278896+0000","last_unstale":"2024-09-16T09:44:50.624997+0000","last_undegraded":"2024-09-16T09:44:50.624997+0000","last_fullsized":"2024-09-16T09:44:50.624997+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:41:40.123937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:41:40.123937+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"285'20","reported_seq":1031,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.704151+0000","last_change":"2024-09-16T09:43:49.220241+0000","last_active":"2024-09-16T09:44:52.704151+0000","last_peered":"2024-09-16T09:44:52.704151+0000","last_clean":"2024-09-16T09:44:52.704151+0000","last_became_active":"2024-09-16T09:25:03.707723+0000","last_became_peered":"2024-09-16T09:25:03.707723+0000","last_unstale":"2024-09-16T09:44:52.704151+0000","last_undegraded":"2024-09-16T09:44:52.704151+0000","last_fullsized":"2024-09-16T09:44:52.704151+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'20","last_scrub_stamp":"2024-09-16T09:43:49.220175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:43:49.220175+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:18.570377+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":875,"reported_epoch":538,"state":"active+clean","last_fresh":"2024-09-16T09:44:54.560071+0000","last_change":"2024-09-16T09:42:52.868096+0000","last_active":"2024-09-16T09:44:54.560071+0000","last_peered":"2024-09-16T09:44:54.560071+0000","last_clean":"2024-09-16T09:44:54.560071+0000","last_became_active":"2024-09-16T09:31:09.288825+0000","last_became_peered":"2024-09-16T09:31:09.288825+0000","last_unstale":"2024-09-16T09:44:54.560071+0000","last_undegraded":"2024-09-16T09:44:54.560071+0000","last_fullsized":"2024-09-16T09:44:54.560071+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:42:52.868035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:42:52.868035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"519'11","reported_seq":834,"reported_epoch":535,"state":"active+clean","last_fresh":"2024-09-16T09:44:50.625604+0000","last_change":"2024-09-16T09:44:24.278535+0000","last_active":"2024-09-16T09:44:50.625604+0000","last_peered":"2024-09-16T09:44:50.625604+0000","last_clean":"2024-09-16T09:44:50.625604+0000","last_became_active":"2024-09-16T09:44:24.277662+0000","last_became_peered":"2024-09-16T09:44:24.277662+0000","last_unstale":"2024-09-16T09:44:50.625604+0000","last_undegraded":"2024-09-16T09:44:50.625604+0000","last_fullsized":"2024-09-16T09:44:50.625604+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"228'9","last_scrub_stamp":"2024-09-16T09:41:33.174444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:41:33.174444+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:40.473027+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1036,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.704342+0000","last_change":"2024-09-16T09:44:48.293840+0000","last_active":"2024-09-16T09:44:52.704342+0000","last_peered":"2024-09-16T09:44:52.704342+0000","last_clean":"2024-09-16T09:44:52.704342+0000","last_became_active":"2024-09-16T09:44:24.280724+0000","last_became_peered":"2024-09-16T09:44:24.280724+0000","last_unstale":"2024-09-16T09:44:52.704342+0000","last_undegraded":"2024-09-16T09:44:52.704342+0000","last_fullsized":"2024-09-16T09:44:52.704342+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:48.293789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:44:48.293789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:57.968474+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.e","version":"73'79","reported_seq":802,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.704339+0000","last_change":"2024-09-16T09:42:16.139285+0000","last_active":"2024-09-16T09:44:52.704339+0000","last_peered":"2024-09-16T09:44:52.704339+0000","last_clean":"2024-09-16T09:44:52.704339+0000","last_became_active":"2024-09-16T09:30:10.705072+0000","last_became_peered":"2024-09-16T09:30:10.705072+0000","last_unstale":"2024-09-16T09:44:52.704339+0000","last_undegraded":"2024-09-16T09:44:52.704339+0000","last_fullsized":"2024-09-16T09:44:52.704339+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:42:16.139208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:42:16.139208+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,3],"acting":[4,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.b","version":"285'15","reported_seq":1063,"reported_epoch":538,"state":"active+clean","last_fresh":"2024-09-16T09:44:54.559889+0000","last_change":"2024-09-16T09:44:48.083967+0000","last_active":"2024-09-16T09:44:54.559889+0000","last_peered":"2024-09-16T09:44:54.559889+0000","last_clean":"2024-09-16T09:44:54.559889+0000","last_became_active":"2024-09-16T09:36:31.590202+0000","last_became_peered":"2024-09-16T09:36:31.590202+0000","last_unstale":"2024-09-16T09:44:54.559889+0000","last_undegraded":"2024-09-16T09:44:54.559889+0000","last_fullsized":"2024-09-16T09:44:54.559889+0000","mapping_epoch":316,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":317,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'15","last_scrub_stamp":"2024-09-16T09:44:48.083895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:44:48.083895+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:50.498247+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":1010,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.551892+0000","last_change":"2024-09-16T09:42:30.709733+0000","last_active":"2024-09-16T09:44:52.551892+0000","last_peered":"2024-09-16T09:44:52.551892+0000","last_clean":"2024-09-16T09:44:52.551892+0000","last_became_active":"2024-09-16T09:31:09.276516+0000","last_became_peered":"2024-09-16T09:31:09.276516+0000","last_unstale":"2024-09-16T09:44:52.551892+0000","last_undegraded":"2024-09-16T09:44:52.551892+0000","last_fullsized":"2024-09-16T09:44:52.551892+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:42:30.709673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:42:30.709673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:17.905087+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":942,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.703824+0000","last_change":"2024-09-16T09:44:24.276286+0000","last_active":"2024-09-16T09:44:52.703824+0000","last_peered":"2024-09-16T09:44:52.703824+0000","last_clean":"2024-09-16T09:44:52.703824+0000","last_became_active":"2024-09-16T09:44:24.275961+0000","last_became_peered":"2024-09-16T09:44:24.275961+0000","last_unstale":"2024-09-16T09:44:52.703824+0000","last_undegraded":"2024-09-16T09:44:52.703824+0000","last_fullsized":"2024-09-16T09:44:52.703824+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:42:56.059953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:42:56.059953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3],"acting":[4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1017,"reported_epoch":538,"state":"active+clean","last_fresh":"2024-09-16T09:44:54.559759+0000","last_change":"2024-09-16T09:44:24.281500+0000","last_active":"2024-09-16T09:44:54.559759+0000","last_peered":"2024-09-16T09:44:54.559759+0000","last_clean":"2024-09-16T09:44:54.559759+0000","last_became_active":"2024-09-16T09:44:24.278714+0000","last_became_peered":"2024-09-16T09:44:24.278714+0000","last_unstale":"2024-09-16T09:44:54.559759+0000","last_undegraded":"2024-09-16T09:44:54.559759+0000","last_fullsized":"2024-09-16T09:44:54.559759+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:42:19.850990+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:42:19.850990+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.f","version":"73'79","reported_seq":1070,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.552194+0000","last_change":"2024-09-16T09:43:24.748100+0000","last_active":"2024-09-16T09:44:52.552194+0000","last_peered":"2024-09-16T09:44:52.552194+0000","last_clean":"2024-09-16T09:44:52.552194+0000","last_became_active":"2024-09-16T09:30:10.693477+0000","last_became_peered":"2024-09-16T09:30:10.693477+0000","last_unstale":"2024-09-16T09:44:52.552194+0000","last_undegraded":"2024-09-16T09:44:52.552194+0000","last_fullsized":"2024-09-16T09:44:52.552194+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:43:24.748025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:43:24.748025+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"285'35","reported_seq":1065,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.551962+0000","last_change":"2024-09-16T09:42:59.545610+0000","last_active":"2024-09-16T09:44:52.551962+0000","last_peered":"2024-09-16T09:44:52.551962+0000","last_clean":"2024-09-16T09:44:52.551962+0000","last_became_active":"2024-09-16T09:25:03.712783+0000","last_became_peered":"2024-09-16T09:25:03.712783+0000","last_unstale":"2024-09-16T09:44:52.551962+0000","last_undegraded":"2024-09-16T09:44:52.551962+0000","last_fullsized":"2024-09-16T09:44:52.551962+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'35","last_scrub_stamp":"2024-09-16T09:42:59.545527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:42:59.545527+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:13.808431+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":56,"num_read_kb":45,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"59'1","reported_seq":1014,"reported_epoch":535,"state":"active+clean","last_fresh":"2024-09-16T09:44:50.625825+0000","last_change":"2024-09-16T09:42:24.291119+0000","last_active":"2024-09-16T09:44:50.625825+0000","last_peered":"2024-09-16T09:44:50.625825+0000","last_clean":"2024-09-16T09:44:50.625825+0000","last_became_active":"2024-09-16T09:25:01.708069+0000","last_became_peered":"2024-09-16T09:25:01.708069+0000","last_unstale":"2024-09-16T09:44:50.625825+0000","last_undegraded":"2024-09-16T09:44:50.625825+0000","last_fullsized":"2024-09-16T09:44:50.625825+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:42:24.291047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:42:24.291047+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":978,"reported_epoch":535,"state":"active+clean","last_fresh":"2024-09-16T09:44:50.542956+0000","last_change":"2024-09-16T09:44:33.039012+0000","last_active":"2024-09-16T09:44:50.542956+0000","last_peered":"2024-09-16T09:44:50.542956+0000","last_clean":"2024-09-16T09:44:50.542956+0000","last_became_active":"2024-09-16T09:44:24.278290+0000","last_became_peered":"2024-09-16T09:44:24.278290+0000","last_unstale":"2024-09-16T09:44:50.542956+0000","last_undegraded":"2024-09-16T09:44:50.542956+0000","last_fullsized":"2024-09-16T09:44:50.542956+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:33.038961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:44:33.038961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:58.372425+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":992,"reported_epoch":535,"state":"active+clean","last_fresh":"2024-09-16T09:44:50.625002+0000","last_change":"2024-09-16T09:44:24.275667+0000","last_active":"2024-09-16T09:44:50.625002+0000","last_peered":"2024-09-16T09:44:50.625002+0000","last_clean":"2024-09-16T09:44:50.625002+0000","last_became_active":"2024-09-16T09:44:24.275407+0000","last_became_peered":"2024-09-16T09:44:24.275407+0000","last_unstale":"2024-09-16T09:44:50.625002+0000","last_undegraded":"2024-09-16T09:44:50.625002+0000","last_fullsized":"2024-09-16T09:44:50.625002+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:42:15.241925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:42:15.241925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.c","version":"73'79","reported_seq":769,"reported_epoch":537,"state":"active+clean","last_fresh":"2024-09-16T09:44:53.558183+0000","last_change":"2024-09-16T09:44:51.141526+0000","last_active":"2024-09-16T09:44:53.558183+0000","last_peered":"2024-09-16T09:44:53.558183+0000","last_clean":"2024-09-16T09:44:53.558183+0000","last_became_active":"2024-09-16T09:31:09.273770+0000","last_became_peered":"2024-09-16T09:31:09.273770+0000","last_unstale":"2024-09-16T09:44:53.558183+0000","last_undegraded":"2024-09-16T09:44:53.558183+0000","last_fullsized":"2024-09-16T09:44:53.558183+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:44:51.141451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:44:51.141451+0000","objects_scrubbed":1,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:16.690997+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":327680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"285'18","reported_seq":1080,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.703833+0000","last_change":"2024-09-16T09:42:17.189319+0000","last_active":"2024-09-16T09:44:52.703833+0000","last_peered":"2024-09-16T09:44:52.703833+0000","last_clean":"2024-09-16T09:44:52.703833+0000","last_became_active":"2024-09-16T09:31:09.269047+0000","last_became_peered":"2024-09-16T09:31:09.269047+0000","last_unstale":"2024-09-16T09:44:52.703833+0000","last_undegraded":"2024-09-16T09:44:52.703833+0000","last_fullsized":"2024-09-16T09:44:52.703833+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'18","last_scrub_stamp":"2024-09-16T09:42:17.189230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:42:17.189230+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":1008,"reported_epoch":535,"state":"active+clean","last_fresh":"2024-09-16T09:44:50.625933+0000","last_change":"2024-09-16T09:42:33.321184+0000","last_active":"2024-09-16T09:44:50.625933+0000","last_peered":"2024-09-16T09:44:50.625933+0000","last_clean":"2024-09-16T09:44:50.625933+0000","last_became_active":"2024-09-16T09:25:01.707528+0000","last_became_peered":"2024-09-16T09:25:01.707528+0000","last_unstale":"2024-09-16T09:44:50.625933+0000","last_undegraded":"2024-09-16T09:44:50.625933+0000","last_fullsized":"2024-09-16T09:44:50.625933+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:42:33.321125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:42:33.321125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":680,"reported_epoch":538,"state":"active+clean","last_fresh":"2024-09-16T09:44:54.557787+0000","last_change":"2024-09-16T09:44:24.271417+0000","last_active":"2024-09-16T09:44:54.557787+0000","last_peered":"2024-09-16T09:44:54.557787+0000","last_clean":"2024-09-16T09:44:54.557787+0000","last_became_active":"2024-09-16T09:44:24.270329+0000","last_became_peered":"2024-09-16T09:44:24.270329+0000","last_unstale":"2024-09-16T09:44:54.557787+0000","last_undegraded":"2024-09-16T09:44:54.557787+0000","last_fullsized":"2024-09-16T09:44:54.557787+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:42:12.922136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:42:12.922136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1003,"reported_epoch":537,"state":"active+clean","last_fresh":"2024-09-16T09:44:53.557119+0000","last_change":"2024-09-16T09:44:04.116446+0000","last_active":"2024-09-16T09:44:53.557119+0000","last_peered":"2024-09-16T09:44:53.557119+0000","last_clean":"2024-09-16T09:44:53.557119+0000","last_became_active":"2024-09-16T09:25:07.718899+0000","last_became_peered":"2024-09-16T09:25:07.718899+0000","last_unstale":"2024-09-16T09:44:53.557119+0000","last_undegraded":"2024-09-16T09:44:53.557119+0000","last_fullsized":"2024-09-16T09:44:53.557119+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:04.116261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:44:04.116261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:05.947986+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.d","version":"73'79","reported_seq":798,"reported_epoch":535,"state":"active+clean","last_fresh":"2024-09-16T09:44:50.625235+0000","last_change":"2024-09-16T09:44:02.449886+0000","last_active":"2024-09-16T09:44:50.625235+0000","last_peered":"2024-09-16T09:44:50.625235+0000","last_clean":"2024-09-16T09:44:50.625235+0000","last_became_active":"2024-09-16T09:30:10.709958+0000","last_became_peered":"2024-09-16T09:30:10.709958+0000","last_unstale":"2024-09-16T09:44:50.625235+0000","last_undegraded":"2024-09-16T09:44:50.625235+0000","last_fullsized":"2024-09-16T09:44:50.625235+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:44:02.449603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:44:02.449603+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:29.882019+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"285'27","reported_seq":1052,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.551118+0000","last_change":"2024-09-16T09:44:21.970091+0000","last_active":"2024-09-16T09:44:52.551118+0000","last_peered":"2024-09-16T09:44:52.551118+0000","last_clean":"2024-09-16T09:44:52.551118+0000","last_became_active":"2024-09-16T09:25:03.715733+0000","last_became_peered":"2024-09-16T09:25:03.715733+0000","last_unstale":"2024-09-16T09:44:52.551118+0000","last_undegraded":"2024-09-16T09:44:52.551118+0000","last_fullsized":"2024-09-16T09:44:52.551118+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'27","last_scrub_stamp":"2024-09-16T09:44:21.970033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:44:21.970033+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:30.951780+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":1012,"reported_epoch":535,"state":"active+clean","last_fresh":"2024-09-16T09:44:50.625982+0000","last_change":"2024-09-16T09:42:18.192222+0000","last_active":"2024-09-16T09:44:50.625982+0000","last_peered":"2024-09-16T09:44:50.625982+0000","last_clean":"2024-09-16T09:44:50.625982+0000","last_became_active":"2024-09-16T09:25:01.707372+0000","last_became_peered":"2024-09-16T09:25:01.707372+0000","last_unstale":"2024-09-16T09:44:50.625982+0000","last_undegraded":"2024-09-16T09:44:50.625982+0000","last_fullsized":"2024-09-16T09:44:50.625982+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-16T09:42:18.192142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:42:18.192142+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":32,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"66'8","reported_seq":1256,"reported_epoch":535,"state":"active+clean","last_fresh":"2024-09-16T09:44:50.625003+0000","last_change":"2024-09-16T09:42:51.690430+0000","last_active":"2024-09-16T09:44:50.625003+0000","last_peered":"2024-09-16T09:44:50.625003+0000","last_clean":"2024-09-16T09:44:50.625003+0000","last_became_active":"2024-09-16T09:25:05.716189+0000","last_became_peered":"2024-09-16T09:25:05.716189+0000","last_unstale":"2024-09-16T09:44:50.625003+0000","last_undegraded":"2024-09-16T09:44:50.625003+0000","last_fullsized":"2024-09-16T09:44:50.625003+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:42:51.690361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:42:51.690361+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":993,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.552536+0000","last_change":"2024-09-16T09:44:18.939435+0000","last_active":"2024-09-16T09:44:52.552536+0000","last_peered":"2024-09-16T09:44:52.552536+0000","last_clean":"2024-09-16T09:44:52.552536+0000","last_became_active":"2024-09-16T09:25:07.717213+0000","last_became_peered":"2024-09-16T09:25:07.717213+0000","last_unstale":"2024-09-16T09:44:52.552536+0000","last_undegraded":"2024-09-16T09:44:52.552536+0000","last_fullsized":"2024-09-16T09:44:52.552536+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:18.939265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:44:18.939265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:45.253702+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.12","version":"73'79","reported_seq":810,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.552893+0000","last_change":"2024-09-16T09:42:16.729102+0000","last_active":"2024-09-16T09:44:52.552893+0000","last_peered":"2024-09-16T09:44:52.552893+0000","last_clean":"2024-09-16T09:44:52.552893+0000","last_became_active":"2024-09-16T09:30:10.731413+0000","last_became_peered":"2024-09-16T09:30:10.731413+0000","last_unstale":"2024-09-16T09:44:52.552893+0000","last_undegraded":"2024-09-16T09:44:52.552893+0000","last_fullsized":"2024-09-16T09:44:52.552893+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:42:16.728929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:42:16.728929+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:16.883189+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":1019,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.551151+0000","last_change":"2024-09-16T09:44:05.759438+0000","last_active":"2024-09-16T09:44:52.551151+0000","last_peered":"2024-09-16T09:44:52.551151+0000","last_clean":"2024-09-16T09:44:52.551151+0000","last_became_active":"2024-09-16T09:37:53.518354+0000","last_became_peered":"2024-09-16T09:37:53.518354+0000","last_unstale":"2024-09-16T09:44:52.551151+0000","last_undegraded":"2024-09-16T09:44:52.551151+0000","last_fullsized":"2024-09-16T09:44:52.551151+0000","mapping_epoch":382,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":383,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:05.759381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:44:05.759381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:11.154024+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"285'12","reported_seq":791,"reported_epoch":537,"state":"active+clean","last_fresh":"2024-09-16T09:44:53.558507+0000","last_change":"2024-09-16T09:44:24.274524+0000","last_active":"2024-09-16T09:44:53.558507+0000","last_peered":"2024-09-16T09:44:53.558507+0000","last_clean":"2024-09-16T09:44:53.558507+0000","last_became_active":"2024-09-16T09:44:24.274107+0000","last_became_peered":"2024-09-16T09:44:24.274107+0000","last_unstale":"2024-09-16T09:44:53.558507+0000","last_undegraded":"2024-09-16T09:44:53.558507+0000","last_fullsized":"2024-09-16T09:44:53.558507+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'12","last_scrub_stamp":"2024-09-16T09:42:52.958841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:42:52.958841+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:37.145040+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":17,"num_read_kb":14,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1004,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.552138+0000","last_change":"2024-09-16T09:43:25.752831+0000","last_active":"2024-09-16T09:44:52.552138+0000","last_peered":"2024-09-16T09:44:52.552138+0000","last_clean":"2024-09-16T09:44:52.552138+0000","last_became_active":"2024-09-16T09:25:05.715616+0000","last_became_peered":"2024-09-16T09:25:05.715616+0000","last_unstale":"2024-09-16T09:44:52.552138+0000","last_undegraded":"2024-09-16T09:44:52.552138+0000","last_fullsized":"2024-09-16T09:44:52.552138+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:43:25.752778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:43:25.752778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":996,"reported_epoch":535,"state":"active+clean","last_fresh":"2024-09-16T09:44:50.625301+0000","last_change":"2024-09-16T09:44:46.528000+0000","last_active":"2024-09-16T09:44:50.625301+0000","last_peered":"2024-09-16T09:44:50.625301+0000","last_clean":"2024-09-16T09:44:50.625301+0000","last_became_active":"2024-09-16T09:25:07.721059+0000","last_became_peered":"2024-09-16T09:25:07.721059+0000","last_unstale":"2024-09-16T09:44:50.625301+0000","last_undegraded":"2024-09-16T09:44:50.625301+0000","last_fullsized":"2024-09-16T09:44:50.625301+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:46.527936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:44:46.527936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:03.411304+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.13","version":"73'79","reported_seq":905,"reported_epoch":537,"state":"active+clean","last_fresh":"2024-09-16T09:44:53.557250+0000","last_change":"2024-09-16T09:44:24.133357+0000","last_active":"2024-09-16T09:44:53.557250+0000","last_peered":"2024-09-16T09:44:53.557250+0000","last_clean":"2024-09-16T09:44:53.557250+0000","last_became_active":"2024-09-16T09:33:30.248497+0000","last_became_peered":"2024-09-16T09:33:30.248497+0000","last_unstale":"2024-09-16T09:44:53.557250+0000","last_undegraded":"2024-09-16T09:44:53.557250+0000","last_fullsized":"2024-09-16T09:44:53.557250+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:44:24.133308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:44:24.133308+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:25.212115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"285'15","reported_seq":1088,"reported_epoch":538,"state":"active+clean","last_fresh":"2024-09-16T09:44:54.559201+0000","last_change":"2024-09-16T09:42:53.892511+0000","last_active":"2024-09-16T09:44:54.559201+0000","last_peered":"2024-09-16T09:44:54.559201+0000","last_clean":"2024-09-16T09:44:54.559201+0000","last_became_active":"2024-09-16T09:31:09.273498+0000","last_became_peered":"2024-09-16T09:31:09.273498+0000","last_unstale":"2024-09-16T09:44:54.559201+0000","last_undegraded":"2024-09-16T09:44:54.559201+0000","last_fullsized":"2024-09-16T09:44:54.559201+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'15","last_scrub_stamp":"2024-09-16T09:42:53.892338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:42:53.892338+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":1016,"reported_epoch":535,"state":"active+clean","last_fresh":"2024-09-16T09:44:50.625293+0000","last_change":"2024-09-16T09:43:49.322844+0000","last_active":"2024-09-16T09:44:50.625293+0000","last_peered":"2024-09-16T09:44:50.625293+0000","last_clean":"2024-09-16T09:44:50.625293+0000","last_became_active":"2024-09-16T09:25:01.706491+0000","last_became_peered":"2024-09-16T09:25:01.706491+0000","last_unstale":"2024-09-16T09:44:50.625293+0000","last_undegraded":"2024-09-16T09:44:50.625293+0000","last_fullsized":"2024-09-16T09:44:50.625293+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:43:49.322773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:43:49.322773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:44:54.891734+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":773,"reported_epoch":535,"state":"active+clean","last_fresh":"2024-09-16T09:44:50.625614+0000","last_change":"2024-09-16T09:42:08.180799+0000","last_active":"2024-09-16T09:44:50.625614+0000","last_peered":"2024-09-16T09:44:50.625614+0000","last_clean":"2024-09-16T09:44:50.625614+0000","last_became_active":"2024-09-16T09:33:30.248992+0000","last_became_peered":"2024-09-16T09:33:30.248992+0000","last_unstale":"2024-09-16T09:44:50.625614+0000","last_undegraded":"2024-09-16T09:44:50.625614+0000","last_fullsized":"2024-09-16T09:44:50.625614+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:42:08.180735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:42:08.180735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:07.411320+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":980,"reported_epoch":535,"state":"active+clean","last_fresh":"2024-09-16T09:44:50.543026+0000","last_change":"2024-09-16T09:44:46.061164+0000","last_active":"2024-09-16T09:44:50.543026+0000","last_peered":"2024-09-16T09:44:50.543026+0000","last_clean":"2024-09-16T09:44:50.543026+0000","last_became_active":"2024-09-16T09:44:24.277612+0000","last_became_peered":"2024-09-16T09:44:24.277612+0000","last_unstale":"2024-09-16T09:44:50.543026+0000","last_undegraded":"2024-09-16T09:44:50.543026+0000","last_fullsized":"2024-09-16T09:44:50.543026+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:46.061101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:44:46.061101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:04.434595+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.10","version":"73'79","reported_seq":810,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.550823+0000","last_change":"2024-09-16T09:44:43.656131+0000","last_active":"2024-09-16T09:44:52.550823+0000","last_peered":"2024-09-16T09:44:52.550823+0000","last_clean":"2024-09-16T09:44:52.550823+0000","last_became_active":"2024-09-16T09:30:10.692770+0000","last_became_peered":"2024-09-16T09:30:10.692770+0000","last_unstale":"2024-09-16T09:44:52.550823+0000","last_undegraded":"2024-09-16T09:44:52.550823+0000","last_fullsized":"2024-09-16T09:44:52.550823+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:44:43.656039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:44:43.656039+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:56.070444+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"285'15","reported_seq":1033,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.552843+0000","last_change":"2024-09-16T09:43:47.715081+0000","last_active":"2024-09-16T09:44:52.552843+0000","last_peered":"2024-09-16T09:44:52.552843+0000","last_clean":"2024-09-16T09:44:52.552843+0000","last_became_active":"2024-09-16T09:25:03.717310+0000","last_became_peered":"2024-09-16T09:25:03.717310+0000","last_unstale":"2024-09-16T09:44:52.552843+0000","last_undegraded":"2024-09-16T09:44:52.552843+0000","last_fullsized":"2024-09-16T09:44:52.552843+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'15","last_scrub_stamp":"2024-09-16T09:43:47.714882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:43:47.714882+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:44:54.735513+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":1047,"reported_epoch":538,"state":"active+clean","last_fresh":"2024-09-16T09:44:54.558631+0000","last_change":"2024-09-16T09:44:35.099469+0000","last_active":"2024-09-16T09:44:54.558631+0000","last_peered":"2024-09-16T09:44:54.558631+0000","last_clean":"2024-09-16T09:44:54.558631+0000","last_became_active":"2024-09-16T09:31:09.272848+0000","last_became_peered":"2024-09-16T09:31:09.272848+0000","last_unstale":"2024-09-16T09:44:54.558631+0000","last_undegraded":"2024-09-16T09:44:54.558631+0000","last_fullsized":"2024-09-16T09:44:54.558631+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:35.099420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:44:35.099420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:01.489513+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"519'13","reported_seq":1097,"reported_epoch":535,"state":"active+clean","last_fresh":"2024-09-16T09:44:50.625515+0000","last_change":"2024-09-16T09:44:24.285251+0000","last_active":"2024-09-16T09:44:50.625515+0000","last_peered":"2024-09-16T09:44:50.625515+0000","last_clean":"2024-09-16T09:44:50.625515+0000","last_became_active":"2024-09-16T09:44:24.284843+0000","last_became_peered":"2024-09-16T09:44:24.284843+0000","last_unstale":"2024-09-16T09:44:50.625515+0000","last_undegraded":"2024-09-16T09:44:50.625515+0000","last_fullsized":"2024-09-16T09:44:50.625515+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"187'11","last_scrub_stamp":"2024-09-16T09:42:19.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:42:19.191311+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1028,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.704577+0000","last_change":"2024-09-16T09:44:24.277838+0000","last_active":"2024-09-16T09:44:52.704577+0000","last_peered":"2024-09-16T09:44:52.704577+0000","last_clean":"2024-09-16T09:44:52.704577+0000","last_became_active":"2024-09-16T09:44:24.277124+0000","last_became_peered":"2024-09-16T09:44:24.277124+0000","last_unstale":"2024-09-16T09:44:52.704577+0000","last_undegraded":"2024-09-16T09:44:52.704577+0000","last_fullsized":"2024-09-16T09:44:52.704577+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:36.098648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:41:36.098648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.11","version":"73'79","reported_seq":892,"reported_epoch":538,"state":"active+clean","last_fresh":"2024-09-16T09:44:54.559954+0000","last_change":"2024-09-16T09:42:16.839630+0000","last_active":"2024-09-16T09:44:54.559954+0000","last_peered":"2024-09-16T09:44:54.559954+0000","last_clean":"2024-09-16T09:44:54.559954+0000","last_became_active":"2024-09-16T09:31:09.272208+0000","last_became_peered":"2024-09-16T09:31:09.272208+0000","last_unstale":"2024-09-16T09:44:54.559954+0000","last_undegraded":"2024-09-16T09:44:54.559954+0000","last_fullsized":"2024-09-16T09:44:54.559954+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:42:16.839458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:42:16.839458+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.14","version":"285'20","reported_seq":690,"reported_epoch":535,"state":"active+clean","last_fresh":"2024-09-16T09:44:50.625611+0000","last_change":"2024-09-16T09:40:35.027221+0000","last_active":"2024-09-16T09:44:50.625611+0000","last_peered":"2024-09-16T09:44:50.625611+0000","last_clean":"2024-09-16T09:44:50.625611+0000","last_became_active":"2024-09-16T09:33:30.249769+0000","last_became_peered":"2024-09-16T09:33:30.249769+0000","last_unstale":"2024-09-16T09:44:50.625611+0000","last_undegraded":"2024-09-16T09:44:50.625611+0000","last_fullsized":"2024-09-16T09:44:50.625611+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'20","last_scrub_stamp":"2024-09-16T09:40:35.027096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:40:35.027096+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:13.420390+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":1011,"reported_epoch":535,"state":"active+clean","last_fresh":"2024-09-16T09:44:50.625443+0000","last_change":"2024-09-16T09:44:24.279638+0000","last_active":"2024-09-16T09:44:50.625443+0000","last_peered":"2024-09-16T09:44:50.625443+0000","last_clean":"2024-09-16T09:44:50.625443+0000","last_became_active":"2024-09-16T09:44:24.279273+0000","last_became_peered":"2024-09-16T09:44:24.279273+0000","last_unstale":"2024-09-16T09:44:50.625443+0000","last_undegraded":"2024-09-16T09:44:50.625443+0000","last_fullsized":"2024-09-16T09:44:50.625443+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:08.005728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:41:08.005728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:37.469764+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"66'8","reported_seq":1267,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.552654+0000","last_change":"2024-09-16T09:44:35.809600+0000","last_active":"2024-09-16T09:44:52.552654+0000","last_peered":"2024-09-16T09:44:52.552654+0000","last_clean":"2024-09-16T09:44:52.552654+0000","last_became_active":"2024-09-16T09:25:05.713833+0000","last_became_peered":"2024-09-16T09:25:05.713833+0000","last_unstale":"2024-09-16T09:44:52.552654+0000","last_undegraded":"2024-09-16T09:44:52.552654+0000","last_fullsized":"2024-09-16T09:44:52.552654+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:44:35.809538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:44:35.809538+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:38.475407+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1035,"reported_epoch":538,"state":"active+clean","last_fresh":"2024-09-16T09:44:54.558787+0000","last_change":"2024-09-16T09:42:32.913873+0000","last_active":"2024-09-16T09:44:54.558787+0000","last_peered":"2024-09-16T09:44:54.558787+0000","last_clean":"2024-09-16T09:44:54.558787+0000","last_became_active":"2024-09-16T09:31:09.275168+0000","last_became_peered":"2024-09-16T09:31:09.275168+0000","last_unstale":"2024-09-16T09:44:54.558787+0000","last_undegraded":"2024-09-16T09:44:54.558787+0000","last_fullsized":"2024-09-16T09:44:54.558787+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:42:32.913795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:42:32.913795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"285'19","reported_seq":1043,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.704011+0000","last_change":"2024-09-16T09:42:20.151407+0000","last_active":"2024-09-16T09:44:52.704011+0000","last_peered":"2024-09-16T09:44:52.704011+0000","last_clean":"2024-09-16T09:44:52.704011+0000","last_became_active":"2024-09-16T09:26:33.291265+0000","last_became_peered":"2024-09-16T09:26:33.291265+0000","last_unstale":"2024-09-16T09:44:52.704011+0000","last_undegraded":"2024-09-16T09:44:52.704011+0000","last_fullsized":"2024-09-16T09:44:52.704011+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'19","last_scrub_stamp":"2024-09-16T09:42:20.151313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:42:20.151313+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":1029,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.704323+0000","last_change":"2024-09-16T09:44:23.334509+0000","last_active":"2024-09-16T09:44:52.704323+0000","last_peered":"2024-09-16T09:44:52.704323+0000","last_clean":"2024-09-16T09:44:52.704323+0000","last_became_active":"2024-09-16T09:25:01.705158+0000","last_became_peered":"2024-09-16T09:25:01.705158+0000","last_unstale":"2024-09-16T09:44:52.704323+0000","last_undegraded":"2024-09-16T09:44:52.704323+0000","last_fullsized":"2024-09-16T09:44:52.704323+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:23.334458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:44:23.334458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:34.371408+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":948,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.552669+0000","last_change":"2024-09-16T09:44:45.660171+0000","last_active":"2024-09-16T09:44:52.552669+0000","last_peered":"2024-09-16T09:44:52.552669+0000","last_clean":"2024-09-16T09:44:52.552669+0000","last_became_active":"2024-09-16T09:37:41.515278+0000","last_became_peered":"2024-09-16T09:37:41.515278+0000","last_unstale":"2024-09-16T09:44:52.552669+0000","last_undegraded":"2024-09-16T09:44:52.552669+0000","last_fullsized":"2024-09-16T09:44:52.552669+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:45.660111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:44:45.660111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:48.190760+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":859,"reported_epoch":538,"state":"active+clean","last_fresh":"2024-09-16T09:44:54.559555+0000","last_change":"2024-09-16T09:44:47.085511+0000","last_active":"2024-09-16T09:44:54.559555+0000","last_peered":"2024-09-16T09:44:54.559555+0000","last_clean":"2024-09-16T09:44:54.559555+0000","last_became_active":"2024-09-16T09:31:09.276042+0000","last_became_peered":"2024-09-16T09:31:09.276042+0000","last_unstale":"2024-09-16T09:44:54.559555+0000","last_undegraded":"2024-09-16T09:44:54.559555+0000","last_fullsized":"2024-09-16T09:44:54.559555+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:47.085465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:44:47.085465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:58.894807+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"285'15","reported_seq":742,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.552137+0000","last_change":"2024-09-16T09:44:28.650617+0000","last_active":"2024-09-16T09:44:52.552137+0000","last_peered":"2024-09-16T09:44:52.552137+0000","last_clean":"2024-09-16T09:44:52.552137+0000","last_became_active":"2024-09-16T09:44:24.270349+0000","last_became_peered":"2024-09-16T09:44:24.270349+0000","last_unstale":"2024-09-16T09:44:52.552137+0000","last_undegraded":"2024-09-16T09:44:52.552137+0000","last_fullsized":"2024-09-16T09:44:52.552137+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'15","last_scrub_stamp":"2024-09-16T09:44:28.650561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:44:28.650561+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:38.659197+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,2],"acting":[6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":1008,"reported_epoch":535,"state":"active+clean","last_fresh":"2024-09-16T09:44:50.625707+0000","last_change":"2024-09-16T09:43:47.307622+0000","last_active":"2024-09-16T09:44:50.625707+0000","last_peered":"2024-09-16T09:44:50.625707+0000","last_clean":"2024-09-16T09:44:50.625707+0000","last_became_active":"2024-09-16T09:25:01.706762+0000","last_became_peered":"2024-09-16T09:25:01.706762+0000","last_unstale":"2024-09-16T09:44:50.625707+0000","last_undegraded":"2024-09-16T09:44:50.625707+0000","last_fullsized":"2024-09-16T09:44:50.625707+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:43:47.307538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:43:47.307538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:02.160911+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":844,"reported_epoch":538,"state":"active+clean","last_fresh":"2024-09-16T09:44:54.559674+0000","last_change":"2024-09-16T09:41:36.130305+0000","last_active":"2024-09-16T09:44:54.559674+0000","last_peered":"2024-09-16T09:44:54.559674+0000","last_clean":"2024-09-16T09:44:54.559674+0000","last_became_active":"2024-09-16T09:28:52.982865+0000","last_became_peered":"2024-09-16T09:28:52.982865+0000","last_unstale":"2024-09-16T09:44:54.559674+0000","last_undegraded":"2024-09-16T09:44:54.559674+0000","last_fullsized":"2024-09-16T09:44:54.559674+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:36.130240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:41:36.130240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":989,"reported_epoch":538,"state":"active+clean","last_fresh":"2024-09-16T09:44:54.557660+0000","last_change":"2024-09-16T09:34:50.779241+0000","last_active":"2024-09-16T09:44:54.557660+0000","last_peered":"2024-09-16T09:44:54.557660+0000","last_clean":"2024-09-16T09:44:54.557660+0000","last_became_active":"2024-09-16T09:25:07.715301+0000","last_became_peered":"2024-09-16T09:25:07.715301+0000","last_unstale":"2024-09-16T09:44:54.557660+0000","last_undegraded":"2024-09-16T09:44:54.557660+0000","last_fullsized":"2024-09-16T09:44:54.557660+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:50.779164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:34:50.779164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:08.818483+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.14","version":"73'79","reported_seq":722,"reported_epoch":535,"state":"active+clean","last_fresh":"2024-09-16T09:44:50.542844+0000","last_change":"2024-09-16T09:44:24.960676+0000","last_active":"2024-09-16T09:44:50.542844+0000","last_peered":"2024-09-16T09:44:50.542844+0000","last_clean":"2024-09-16T09:44:50.542844+0000","last_became_active":"2024-09-16T09:44:24.285886+0000","last_became_peered":"2024-09-16T09:44:24.285886+0000","last_unstale":"2024-09-16T09:44:50.542844+0000","last_undegraded":"2024-09-16T09:44:50.542844+0000","last_fullsized":"2024-09-16T09:44:50.542844+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:44:24.960619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:44:24.960619+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:44.429977+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"285'19","reported_seq":887,"reported_epoch":535,"state":"active+clean","last_fresh":"2024-09-16T09:44:50.625240+0000","last_change":"2024-09-16T09:42:57.678412+0000","last_active":"2024-09-16T09:44:50.625240+0000","last_peered":"2024-09-16T09:44:50.625240+0000","last_clean":"2024-09-16T09:44:50.625240+0000","last_became_active":"2024-09-16T09:28:52.971132+0000","last_became_peered":"2024-09-16T09:28:52.971132+0000","last_unstale":"2024-09-16T09:44:50.625240+0000","last_undegraded":"2024-09-16T09:44:50.625240+0000","last_fullsized":"2024-09-16T09:44:50.625240+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'19","last_scrub_stamp":"2024-09-16T09:42:57.678331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:42:57.678331+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":1020,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.552275+0000","last_change":"2024-09-16T09:44:03.793102+0000","last_active":"2024-09-16T09:44:52.552275+0000","last_peered":"2024-09-16T09:44:52.552275+0000","last_clean":"2024-09-16T09:44:52.552275+0000","last_became_active":"2024-09-16T09:25:01.700733+0000","last_became_peered":"2024-09-16T09:25:01.700733+0000","last_unstale":"2024-09-16T09:44:52.552275+0000","last_undegraded":"2024-09-16T09:44:52.552275+0000","last_fullsized":"2024-09-16T09:44:52.552275+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:03.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:44:03.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:16.224422+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1000,"reported_epoch":535,"state":"active+clean","last_fresh":"2024-09-16T09:44:50.624696+0000","last_change":"2024-09-16T09:40:39.969447+0000","last_active":"2024-09-16T09:44:50.624696+0000","last_peered":"2024-09-16T09:44:50.624696+0000","last_clean":"2024-09-16T09:44:50.624696+0000","last_became_active":"2024-09-16T09:25:05.713479+0000","last_became_peered":"2024-09-16T09:25:05.713479+0000","last_unstale":"2024-09-16T09:44:50.624696+0000","last_undegraded":"2024-09-16T09:44:50.624696+0000","last_fullsized":"2024-09-16T09:44:50.624696+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:40:39.969383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:40:39.969383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:38.496665+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":867,"reported_epoch":538,"state":"active+clean","last_fresh":"2024-09-16T09:44:54.560204+0000","last_change":"2024-09-16T09:41:34.178100+0000","last_active":"2024-09-16T09:44:54.560204+0000","last_peered":"2024-09-16T09:44:54.560204+0000","last_clean":"2024-09-16T09:44:54.560204+0000","last_became_active":"2024-09-16T09:31:09.274797+0000","last_became_peered":"2024-09-16T09:31:09.274797+0000","last_unstale":"2024-09-16T09:44:54.560204+0000","last_undegraded":"2024-09-16T09:44:54.560204+0000","last_fullsized":"2024-09-16T09:44:54.560204+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:34.178020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:41:34.178020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"285'8","reported_seq":784,"reported_epoch":537,"state":"active+clean","last_fresh":"2024-09-16T09:44:53.556182+0000","last_change":"2024-09-16T09:44:47.186136+0000","last_active":"2024-09-16T09:44:53.556182+0000","last_peered":"2024-09-16T09:44:53.556182+0000","last_clean":"2024-09-16T09:44:53.556182+0000","last_became_active":"2024-09-16T09:31:09.273682+0000","last_became_peered":"2024-09-16T09:31:09.273682+0000","last_unstale":"2024-09-16T09:44:53.556182+0000","last_undegraded":"2024-09-16T09:44:53.556182+0000","last_fullsized":"2024-09-16T09:44:53.556182+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'8","last_scrub_stamp":"2024-09-16T09:44:47.186078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:44:47.186078+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:11.853660+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":9,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":1047,"reported_epoch":538,"state":"active+clean","last_fresh":"2024-09-16T09:44:54.558627+0000","last_change":"2024-09-16T09:43:21.684366+0000","last_active":"2024-09-16T09:44:54.558627+0000","last_peered":"2024-09-16T09:44:54.558627+0000","last_clean":"2024-09-16T09:44:54.558627+0000","last_became_active":"2024-09-16T09:31:09.274719+0000","last_became_peered":"2024-09-16T09:31:09.274719+0000","last_unstale":"2024-09-16T09:44:54.558627+0000","last_undegraded":"2024-09-16T09:44:54.558627+0000","last_fullsized":"2024-09-16T09:44:54.558627+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:43:21.684289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:43:21.684289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1021,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.551482+0000","last_change":"2024-09-16T09:44:49.775416+0000","last_active":"2024-09-16T09:44:52.551482+0000","last_peered":"2024-09-16T09:44:52.551482+0000","last_clean":"2024-09-16T09:44:52.551482+0000","last_became_active":"2024-09-16T09:25:05.712545+0000","last_became_peered":"2024-09-16T09:25:05.712545+0000","last_unstale":"2024-09-16T09:44:52.551482+0000","last_undegraded":"2024-09-16T09:44:52.551482+0000","last_fullsized":"2024-09-16T09:44:52.551482+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:49.775356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:44:49.775356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:57.470848+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1015,"reported_epoch":535,"state":"active+clean","last_fresh":"2024-09-16T09:44:51.632486+0000","last_change":"2024-09-16T09:44:51.632486+0000","last_active":"2024-09-16T09:44:51.632486+0000","last_peered":"2024-09-16T09:44:51.632486+0000","last_clean":"2024-09-16T09:44:51.632486+0000","last_became_active":"2024-09-16T09:44:24.276311+0000","last_became_peered":"2024-09-16T09:44:24.276311+0000","last_unstale":"2024-09-16T09:44:51.632486+0000","last_undegraded":"2024-09-16T09:44:51.632486+0000","last_fullsized":"2024-09-16T09:44:51.632486+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:51.632429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:44:51.632429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:19.007714+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":644,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.552774+0000","last_change":"2024-09-16T09:44:50.722288+0000","last_active":"2024-09-16T09:44:52.552774+0000","last_peered":"2024-09-16T09:44:52.552774+0000","last_clean":"2024-09-16T09:44:52.552774+0000","last_became_active":"2024-09-16T09:33:30.248036+0000","last_became_peered":"2024-09-16T09:33:30.248036+0000","last_unstale":"2024-09-16T09:44:52.552774+0000","last_undegraded":"2024-09-16T09:44:52.552774+0000","last_fullsized":"2024-09-16T09:44:52.552774+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:50.722209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:44:50.722209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:53.928329+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":866,"reported_epoch":538,"state":"active+clean","last_fresh":"2024-09-16T09:44:54.558201+0000","last_change":"2024-09-16T09:42:30.898163+0000","last_active":"2024-09-16T09:44:54.558201+0000","last_peered":"2024-09-16T09:44:54.558201+0000","last_clean":"2024-09-16T09:44:54.558201+0000","last_became_active":"2024-09-16T09:31:09.271662+0000","last_became_peered":"2024-09-16T09:31:09.271662+0000","last_unstale":"2024-09-16T09:44:54.558201+0000","last_undegraded":"2024-09-16T09:44:54.558201+0000","last_fullsized":"2024-09-16T09:44:54.558201+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:42:30.898094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:42:30.898094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"285'19","reported_seq":1049,"reported_epoch":536,"state":"active+clean","last_fresh":"2024-09-16T09:44:52.552306+0000","last_change":"2024-09-16T09:44:23.700110+0000","last_active":"2024-09-16T09:44:52.552306+0000","last_peered":"2024-09-16T09:44:52.552306+0000","last_clean":"2024-09-16T09:44:52.552306+0000","last_became_active":"2024-09-16T09:25:03.707769+0000","last_became_peered":"2024-09-16T09:25:03.707769+0000","last_unstale":"2024-09-16T09:44:52.552306+0000","last_undegraded":"2024-09-16T09:44:52.552306+0000","last_fullsized":"2024-09-16T09:44:52.552306+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'19","last_scrub_stamp":"2024-09-16T09:44:23.700027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:44:23.700027+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:26.961037+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1021,"reported_epoch":538,"state":"active+clean","last_fresh":"2024-09-16T09:44:54.558933+0000","last_change":"2024-09-16T09:44:36.073614+0000","last_active":"2024-09-16T09:44:54.558933+0000","last_peered":"2024-09-16T09:44:54.558933+0000","last_clean":"2024-09-16T09:44:54.558933+0000","last_became_active":"2024-09-16T09:44:24.279906+0000","last_became_peered":"2024-09-16T09:44:24.279906+0000","last_unstale":"2024-09-16T09:44:54.558933+0000","last_undegraded":"2024-09-16T09:44:54.558933+0000","last_fullsized":"2024-09-16T09:44:54.558933+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:36.073560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:44:36.073560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:44.457091+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":1115,"num_read_kb":872,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":3686,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":683,"ondisk_log_size":683,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1986,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1183,"num_read_kb":1178,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":21,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":327680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1659,"ondisk_log_size":1659,"up":63,"acting":63,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266931,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":848368,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":847486,"kb_avail":92896784,"statfs":{"total":95995035648,"available":95126306816,"internally_reserved":0,"allocated":876544,"data_stored":522396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":867826122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495864,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":718176,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":716990,"kb_avail":93026976,"statfs":{"total":95995035648,"available":95259623424,"internally_reserved":0,"allocated":1187840,"data_stored":856814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":734198214},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724799,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":858352,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":857214,"kb_avail":92886800,"statfs":{"total":95995035648,"available":95116083200,"internally_reserved":0,"allocated":1142784,"data_stored":792511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":877787590},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953732,"num_pgs":65,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":735916,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":735038,"kb_avail":93009236,"statfs":{"total":95995035648,"available":95241457664,"internally_reserved":0,"allocated":876544,"data_stored":522350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":752679366},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149962,"num_pgs":64,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":916992,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":915518,"kb_avail":92828160,"statfs":{"total":95995035648,"available":95056035840,"internally_reserved":0,"allocated":1486848,"data_stored":1120840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":937490887},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":516,"seq":2216203124743,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":926088,"kb_used_data":844344,"kb_used_omap":4,"kb_used_meta":81723,"kb_avail":92819064,"statfs":{"total":95995035648,"available":95046721536,"internally_reserved":0,"allocated":864608256,"data_stored":864285924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4445,"internal_metadata":83685027},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575126,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1024728,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":1023806,"kb_avail":92720424,"statfs":{"total":95995035648,"available":94945714176,"internally_reserved":0,"allocated":917504,"data_stored":530276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":1048377797},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738652,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":866752,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":865534,"kb_avail":92878400,"statfs":{"total":95995035648,"available":95107481600,"internally_reserved":0,"allocated":1224704,"data_stored":866056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":886307269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":1788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T09:44:58.732 INFO:tasks.cephadm.ceph_manager.ceph:recovered! 2024-09-16T09:44:58.732 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-16T09:44:58.732 INFO:tasks.thrashosds.thrasher:Growing pool .mgr 2024-09-16T09:44:58.732 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-16T09:44:58.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:58 smithi084 ceph-mon[32629]: 4.15 scrub starts 2024-09-16T09:44:58.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:58 smithi084 ceph-mon[32629]: 4.15 scrub ok 2024-09-16T09:44:58.920 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:58 smithi070 ceph-mon[40691]: 4.15 scrub starts 2024-09-16T09:44:58.920 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:58 smithi070 ceph-mon[40691]: 4.15 scrub ok 2024-09-16T09:44:58.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:58 smithi070 ceph-mon[30388]: 4.15 scrub starts 2024-09-16T09:44:58.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:58 smithi070 ceph-mon[30388]: 4.15 scrub ok 2024-09-16T09:44:58.981 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:44:59.803 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:44:59.804 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:59 smithi070 ceph-mon[30388]: from='client.26591 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:44:59.804 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:59 smithi070 ceph-mon[30388]: pgmap v1052: 152 pgs: 152 active+clean; 583 KiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 1.9 KiB/s rd, 1 op/s 2024-09-16T09:44:59.804 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:59 smithi070 ceph-mon[30388]: 3.1f scrub starts 2024-09-16T09:44:59.804 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:59 smithi070 ceph-mon[30388]: 3.1f scrub ok 2024-09-16T09:44:59.804 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:59 smithi070 ceph-mon[30388]: 5.1 scrub starts 2024-09-16T09:44:59.804 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:59 smithi070 ceph-mon[30388]: 5.1 scrub ok 2024-09-16T09:44:59.804 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:44:59 smithi070 ceph-mon[30388]: osdmap e540: 8 total, 8 up, 8 in 2024-09-16T09:44:59.804 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:59 smithi070 ceph-mon[40691]: from='client.26591 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:44:59.805 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:59 smithi070 ceph-mon[40691]: pgmap v1052: 152 pgs: 152 active+clean; 583 KiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 1.9 KiB/s rd, 1 op/s 2024-09-16T09:44:59.805 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:59 smithi070 ceph-mon[40691]: 3.1f scrub starts 2024-09-16T09:44:59.805 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:59 smithi070 ceph-mon[40691]: 3.1f scrub ok 2024-09-16T09:44:59.805 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:59 smithi070 ceph-mon[40691]: 5.1 scrub starts 2024-09-16T09:44:59.805 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:59 smithi070 ceph-mon[40691]: 5.1 scrub ok 2024-09-16T09:44:59.805 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:44:59 smithi070 ceph-mon[40691]: osdmap e540: 8 total, 8 up, 8 in 2024-09-16T09:44:59.810 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2024-09-16T09:44:59.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:59 smithi084 ceph-mon[32629]: from='client.26591 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:44:59.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:59 smithi084 ceph-mon[32629]: pgmap v1052: 152 pgs: 152 active+clean; 583 KiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 1.9 KiB/s rd, 1 op/s 2024-09-16T09:44:59.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:59 smithi084 ceph-mon[32629]: 3.1f scrub starts 2024-09-16T09:44:59.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:59 smithi084 ceph-mon[32629]: 3.1f scrub ok 2024-09-16T09:44:59.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:59 smithi084 ceph-mon[32629]: 5.1 scrub starts 2024-09-16T09:44:59.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:59 smithi084 ceph-mon[32629]: 5.1 scrub ok 2024-09-16T09:44:59.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:44:59 smithi084 ceph-mon[32629]: osdmap e540: 8 total, 8 up, 8 in 2024-09-16T09:45:00.611 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":1053,"stamp":"2024-09-16T09:44:59.482743+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596507,"num_objects":199,"num_object_clones":0,"num_object_copies":597,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":199,"num_whiteouts":0,"num_read":2327,"num_read_kb":2079,"num_write":722,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":151,"num_bytes_recovered":333352,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2434,"ondisk_log_size":2434,"up":456,"acting":456,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":458,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":6876476,"kb_used_data":832964,"kb_used_omap":15,"kb_used_meta":6043312,"kb_avail":743084740,"statfs":{"total":767960285184,"available":760918773760,"internally_reserved":0,"allocated":852955136,"data_stored":850027659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15600,"internal_metadata":6188352272},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.927607"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":1037,"reported_epoch":538,"state":"active+clean","last_fresh":"2024-09-16T09:44:54.558549+0000","last_change":"2024-09-16T09:44:24.277767+0000","last_active":"2024-09-16T09:44:54.558549+0000","last_peered":"2024-09-16T09:44:54.558549+0000","last_clean":"2024-09-16T09:44:54.558549+0000","last_became_active":"2024-09-16T09:44:24.277455+0000","last_became_peered":"2024-09-16T09:44:24.277455+0000","last_unstale":"2024-09-16T09:44:54.558549+0000","last_undegraded":"2024-09-16T09:44:54.558549+0000","last_fullsized":"2024-09-16T09:44:54.558549+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:40.073089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:41:40.073089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"285'15","reported_seq":1047,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.571913+0000","last_change":"2024-09-16T09:44:05.286194+0000","last_active":"2024-09-16T09:44:55.571913+0000","last_peered":"2024-09-16T09:44:55.571913+0000","last_clean":"2024-09-16T09:44:55.571913+0000","last_became_active":"2024-09-16T09:25:03.714635+0000","last_became_peered":"2024-09-16T09:25:03.714635+0000","last_unstale":"2024-09-16T09:44:55.571913+0000","last_undegraded":"2024-09-16T09:44:55.571913+0000","last_fullsized":"2024-09-16T09:44:55.571913+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'15","last_scrub_stamp":"2024-09-16T09:44:05.286119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:44:05.286119+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:29.124567+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":650,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.791625+0000","last_change":"2024-09-16T09:42:31.714835+0000","last_active":"2024-09-16T09:44:55.791625+0000","last_peered":"2024-09-16T09:44:55.791625+0000","last_clean":"2024-09-16T09:44:55.791625+0000","last_became_active":"2024-09-16T09:33:30.242238+0000","last_became_peered":"2024-09-16T09:33:30.242238+0000","last_unstale":"2024-09-16T09:44:55.791625+0000","last_undegraded":"2024-09-16T09:44:55.791625+0000","last_fullsized":"2024-09-16T09:44:55.791625+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:42:31.714749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:42:31.714749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:06.776034+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1005,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.572415+0000","last_change":"2024-09-16T09:44:35.357353+0000","last_active":"2024-09-16T09:44:55.572415+0000","last_peered":"2024-09-16T09:44:55.572415+0000","last_clean":"2024-09-16T09:44:55.572415+0000","last_became_active":"2024-09-16T09:25:07.715442+0000","last_became_peered":"2024-09-16T09:25:07.715442+0000","last_unstale":"2024-09-16T09:44:55.572415+0000","last_undegraded":"2024-09-16T09:44:55.572415+0000","last_fullsized":"2024-09-16T09:44:55.572415+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:35.357286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:44:35.357286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:52.302674+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":752,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.569276+0000","last_change":"2024-09-16T09:44:34.236344+0000","last_active":"2024-09-16T09:44:55.569276+0000","last_peered":"2024-09-16T09:44:55.569276+0000","last_clean":"2024-09-16T09:44:55.569276+0000","last_became_active":"2024-09-16T09:31:09.269342+0000","last_became_peered":"2024-09-16T09:31:09.269342+0000","last_unstale":"2024-09-16T09:44:55.569276+0000","last_undegraded":"2024-09-16T09:44:55.569276+0000","last_fullsized":"2024-09-16T09:44:55.569276+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:34.236298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:44:34.236298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:59.040431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":897,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.738370+0000","last_change":"2024-09-16T09:44:26.603265+0000","last_active":"2024-09-16T09:44:55.738370+0000","last_peered":"2024-09-16T09:44:55.738370+0000","last_clean":"2024-09-16T09:44:55.738370+0000","last_became_active":"2024-09-16T09:44:24.279187+0000","last_became_peered":"2024-09-16T09:44:24.279187+0000","last_unstale":"2024-09-16T09:44:55.738370+0000","last_undegraded":"2024-09-16T09:44:55.738370+0000","last_fullsized":"2024-09-16T09:44:55.738370+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:26.603213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:44:26.603213+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:50.281936+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.19","version":"285'27","reported_seq":228,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.571789+0000","last_change":"2024-09-16T09:40:58.802023+0000","last_active":"2024-09-16T09:44:55.571789+0000","last_peered":"2024-09-16T09:44:55.571789+0000","last_clean":"2024-09-16T09:44:55.571789+0000","last_became_active":"2024-09-16T09:40:58.699357+0000","last_became_peered":"2024-09-16T09:40:58.699357+0000","last_unstale":"2024-09-16T09:44:55.571789+0000","last_undegraded":"2024-09-16T09:44:55.571789+0000","last_fullsized":"2024-09-16T09:44:55.571789+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'27","last_scrub_stamp":"2024-09-16T09:37:29.683568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:37:29.683568+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:43.367028+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,5],"acting":[4,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1018,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.571788+0000","last_change":"2024-09-16T09:42:10.051996+0000","last_active":"2024-09-16T09:44:55.571788+0000","last_peered":"2024-09-16T09:44:55.571788+0000","last_clean":"2024-09-16T09:44:55.571788+0000","last_became_active":"2024-09-16T09:25:05.717236+0000","last_became_peered":"2024-09-16T09:25:05.717236+0000","last_unstale":"2024-09-16T09:44:55.571788+0000","last_undegraded":"2024-09-16T09:44:55.571788+0000","last_fullsized":"2024-09-16T09:44:55.571788+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:42:10.051939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:42:10.051939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":657,"reported_epoch":538,"state":"active+clean","last_fresh":"2024-09-16T09:44:54.557931+0000","last_change":"2024-09-16T09:43:23.679786+0000","last_active":"2024-09-16T09:44:54.557931+0000","last_peered":"2024-09-16T09:44:54.557931+0000","last_clean":"2024-09-16T09:44:54.557931+0000","last_became_active":"2024-09-16T09:33:30.247733+0000","last_became_peered":"2024-09-16T09:33:30.247733+0000","last_unstale":"2024-09-16T09:44:54.557931+0000","last_undegraded":"2024-09-16T09:44:54.557931+0000","last_fullsized":"2024-09-16T09:44:54.557931+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:43:23.679720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:43:23.679720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1a","version":"285'15","reported_seq":1046,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.572845+0000","last_change":"2024-09-16T09:43:59.240127+0000","last_active":"2024-09-16T09:44:55.572845+0000","last_peered":"2024-09-16T09:44:55.572845+0000","last_clean":"2024-09-16T09:44:55.572845+0000","last_became_active":"2024-09-16T09:25:03.709147+0000","last_became_peered":"2024-09-16T09:25:03.709147+0000","last_unstale":"2024-09-16T09:44:55.572845+0000","last_undegraded":"2024-09-16T09:44:55.572845+0000","last_fullsized":"2024-09-16T09:44:55.572845+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'15","last_scrub_stamp":"2024-09-16T09:43:59.240052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:43:59.240052+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:06.629394+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1001,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.791927+0000","last_change":"2024-09-16T09:42:57.714429+0000","last_active":"2024-09-16T09:44:55.791927+0000","last_peered":"2024-09-16T09:44:55.791927+0000","last_clean":"2024-09-16T09:44:55.791927+0000","last_became_active":"2024-09-16T09:25:05.718134+0000","last_became_peered":"2024-09-16T09:25:05.718134+0000","last_unstale":"2024-09-16T09:44:55.791927+0000","last_undegraded":"2024-09-16T09:44:55.791927+0000","last_fullsized":"2024-09-16T09:44:55.791927+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:42:57.714362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:42:57.714362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:40.690246+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"66'1","reported_seq":1022,"reported_epoch":538,"state":"active+clean","last_fresh":"2024-09-16T09:44:54.557873+0000","last_change":"2024-09-16T09:43:28.772774+0000","last_active":"2024-09-16T09:44:54.557873+0000","last_peered":"2024-09-16T09:44:54.557873+0000","last_clean":"2024-09-16T09:44:54.557873+0000","last_became_active":"2024-09-16T09:25:07.716558+0000","last_became_peered":"2024-09-16T09:25:07.716558+0000","last_unstale":"2024-09-16T09:44:54.557873+0000","last_undegraded":"2024-09-16T09:44:54.557873+0000","last_fullsized":"2024-09-16T09:44:54.557873+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-16T09:43:28.772704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:43:28.772704+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:44:56.277517+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":1001,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.792118+0000","last_change":"2024-09-16T09:42:22.732120+0000","last_active":"2024-09-16T09:44:55.792118+0000","last_peered":"2024-09-16T09:44:55.792118+0000","last_clean":"2024-09-16T09:44:55.792118+0000","last_became_active":"2024-09-16T09:25:01.696939+0000","last_became_peered":"2024-09-16T09:25:01.696939+0000","last_unstale":"2024-09-16T09:44:55.792118+0000","last_undegraded":"2024-09-16T09:44:55.792118+0000","last_fullsized":"2024-09-16T09:44:55.792118+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:42:22.732048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:42:22.732048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:10.853456+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"285'7","reported_seq":1010,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.572290+0000","last_change":"2024-09-16T09:35:39.975915+0000","last_active":"2024-09-16T09:44:55.572290+0000","last_peered":"2024-09-16T09:44:55.572290+0000","last_clean":"2024-09-16T09:44:55.572290+0000","last_became_active":"2024-09-16T09:25:03.716136+0000","last_became_peered":"2024-09-16T09:25:03.716136+0000","last_unstale":"2024-09-16T09:44:55.572290+0000","last_undegraded":"2024-09-16T09:44:55.572290+0000","last_fullsized":"2024-09-16T09:44:55.572290+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'7","last_scrub_stamp":"2024-09-16T09:35:39.975840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:35:39.975840+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:17.269033+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":10,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":157,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.737573+0000","last_change":"2024-09-16T09:42:08.915352+0000","last_active":"2024-09-16T09:44:55.737573+0000","last_peered":"2024-09-16T09:44:55.737573+0000","last_clean":"2024-09-16T09:44:55.737573+0000","last_became_active":"2024-09-16T09:42:02.494290+0000","last_became_peered":"2024-09-16T09:42:02.494290+0000","last_unstale":"2024-09-16T09:44:55.737573+0000","last_undegraded":"2024-09-16T09:44:55.737573+0000","last_fullsized":"2024-09-16T09:44:55.737573+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:42:08.915303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:42:08.915303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4],"acting":[6,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1036,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.791685+0000","last_change":"2024-09-16T09:44:52.742833+0000","last_active":"2024-09-16T09:44:55.791685+0000","last_peered":"2024-09-16T09:44:55.791685+0000","last_clean":"2024-09-16T09:44:55.791685+0000","last_became_active":"2024-09-16T09:31:09.275656+0000","last_became_peered":"2024-09-16T09:31:09.275656+0000","last_unstale":"2024-09-16T09:44:55.791685+0000","last_undegraded":"2024-09-16T09:44:55.791685+0000","last_fullsized":"2024-09-16T09:44:55.791685+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:52.742776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:44:52.742776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:05.256885+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1007,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.573247+0000","last_change":"2024-09-16T09:42:57.111424+0000","last_active":"2024-09-16T09:44:55.573247+0000","last_peered":"2024-09-16T09:44:55.573247+0000","last_clean":"2024-09-16T09:44:55.573247+0000","last_became_active":"2024-09-16T09:25:07.720793+0000","last_became_peered":"2024-09-16T09:25:07.720793+0000","last_unstale":"2024-09-16T09:44:55.573247+0000","last_undegraded":"2024-09-16T09:44:55.573247+0000","last_fullsized":"2024-09-16T09:44:55.573247+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:42:57.111229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:42:57.111229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":758,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.737336+0000","last_change":"2024-09-16T09:44:48.756968+0000","last_active":"2024-09-16T09:44:55.737336+0000","last_peered":"2024-09-16T09:44:55.737336+0000","last_clean":"2024-09-16T09:44:55.737336+0000","last_became_active":"2024-09-16T09:30:39.201824+0000","last_became_peered":"2024-09-16T09:30:39.201824+0000","last_unstale":"2024-09-16T09:44:55.737336+0000","last_undegraded":"2024-09-16T09:44:55.737336+0000","last_fullsized":"2024-09-16T09:44:55.737336+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:48.756890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:44:48.756890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:17.988105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"285'27","reported_seq":1078,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.622282+0000","last_change":"2024-09-16T09:44:24.284901+0000","last_active":"2024-09-16T09:44:55.622282+0000","last_peered":"2024-09-16T09:44:55.622282+0000","last_clean":"2024-09-16T09:44:55.622282+0000","last_became_active":"2024-09-16T09:44:24.283762+0000","last_became_peered":"2024-09-16T09:44:24.283762+0000","last_unstale":"2024-09-16T09:44:55.622282+0000","last_undegraded":"2024-09-16T09:44:55.622282+0000","last_fullsized":"2024-09-16T09:44:55.622282+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'27","last_scrub_stamp":"2024-09-16T09:42:52.891931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:42:52.891931+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:41.042706+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":641,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.572769+0000","last_change":"2024-09-16T09:44:02.300014+0000","last_active":"2024-09-16T09:44:55.572769+0000","last_peered":"2024-09-16T09:44:55.572769+0000","last_clean":"2024-09-16T09:44:55.572769+0000","last_became_active":"2024-09-16T09:33:30.251587+0000","last_became_peered":"2024-09-16T09:33:30.251587+0000","last_unstale":"2024-09-16T09:44:55.572769+0000","last_undegraded":"2024-09-16T09:44:55.572769+0000","last_fullsized":"2024-09-16T09:44:55.572769+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:02.299963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:44:02.299963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:23.146994+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,0],"acting":[4,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":864,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.738391+0000","last_change":"2024-09-16T09:43:16.704220+0000","last_active":"2024-09-16T09:44:55.738391+0000","last_peered":"2024-09-16T09:44:55.738391+0000","last_clean":"2024-09-16T09:44:55.738391+0000","last_became_active":"2024-09-16T09:28:52.974255+0000","last_became_peered":"2024-09-16T09:28:52.974255+0000","last_unstale":"2024-09-16T09:44:55.738391+0000","last_undegraded":"2024-09-16T09:44:55.738391+0000","last_fullsized":"2024-09-16T09:44:55.738391+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:43:16.704155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:43:16.704155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":1036,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.573128+0000","last_change":"2024-09-16T09:44:28.385487+0000","last_active":"2024-09-16T09:44:55.573128+0000","last_peered":"2024-09-16T09:44:55.573128+0000","last_clean":"2024-09-16T09:44:55.573128+0000","last_became_active":"2024-09-16T09:44:24.276029+0000","last_became_peered":"2024-09-16T09:44:24.276029+0000","last_unstale":"2024-09-16T09:44:55.573128+0000","last_undegraded":"2024-09-16T09:44:55.573128+0000","last_fullsized":"2024-09-16T09:44:55.573128+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:28.385444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:44:28.385444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:52.078449+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"285'18","reported_seq":701,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.572986+0000","last_change":"2024-09-16T09:43:16.141437+0000","last_active":"2024-09-16T09:44:55.572986+0000","last_peered":"2024-09-16T09:44:55.572986+0000","last_clean":"2024-09-16T09:44:55.572986+0000","last_became_active":"2024-09-16T09:33:30.251626+0000","last_became_peered":"2024-09-16T09:33:30.251626+0000","last_unstale":"2024-09-16T09:44:55.572986+0000","last_undegraded":"2024-09-16T09:44:55.572986+0000","last_fullsized":"2024-09-16T09:44:55.572986+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'18","last_scrub_stamp":"2024-09-16T09:43:16.141356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:43:16.141356+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":33,"num_read_kb":24,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1012,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.572502+0000","last_change":"2024-09-16T09:44:49.325447+0000","last_active":"2024-09-16T09:44:55.572502+0000","last_peered":"2024-09-16T09:44:55.572502+0000","last_clean":"2024-09-16T09:44:55.572502+0000","last_became_active":"2024-09-16T09:44:24.278117+0000","last_became_peered":"2024-09-16T09:44:24.278117+0000","last_unstale":"2024-09-16T09:44:55.572502+0000","last_undegraded":"2024-09-16T09:44:55.572502+0000","last_fullsized":"2024-09-16T09:44:55.572502+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:49.325384+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:44:49.325384+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:53.841965+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":1015,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.620564+0000","last_change":"2024-09-16T09:44:24.284719+0000","last_active":"2024-09-16T09:44:55.620564+0000","last_peered":"2024-09-16T09:44:55.620564+0000","last_clean":"2024-09-16T09:44:55.620564+0000","last_became_active":"2024-09-16T09:44:24.284256+0000","last_became_peered":"2024-09-16T09:44:24.284256+0000","last_unstale":"2024-09-16T09:44:55.620564+0000","last_undegraded":"2024-09-16T09:44:55.620564+0000","last_fullsized":"2024-09-16T09:44:55.620564+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-16T09:42:25.250120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:42:25.250120+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":659,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.568652+0000","last_change":"2024-09-16T09:41:36.612463+0000","last_active":"2024-09-16T09:44:55.568652+0000","last_peered":"2024-09-16T09:44:55.568652+0000","last_clean":"2024-09-16T09:44:55.568652+0000","last_became_active":"2024-09-16T09:33:30.245048+0000","last_became_peered":"2024-09-16T09:33:30.245048+0000","last_unstale":"2024-09-16T09:44:55.568652+0000","last_undegraded":"2024-09-16T09:44:55.568652+0000","last_fullsized":"2024-09-16T09:44:55.568652+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:41:36.612280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:41:36.612280+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:17.173178+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"285'20","reported_seq":1074,"reported_epoch":538,"state":"active+clean","last_fresh":"2024-09-16T09:44:54.559021+0000","last_change":"2024-09-16T09:39:35.550760+0000","last_active":"2024-09-16T09:44:54.559021+0000","last_peered":"2024-09-16T09:44:54.559021+0000","last_clean":"2024-09-16T09:44:54.559021+0000","last_became_active":"2024-09-16T09:31:09.275050+0000","last_became_peered":"2024-09-16T09:31:09.275050+0000","last_unstale":"2024-09-16T09:44:54.559021+0000","last_undegraded":"2024-09-16T09:44:54.559021+0000","last_fullsized":"2024-09-16T09:44:54.559021+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'20","last_scrub_stamp":"2024-09-16T09:39:35.548333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:39:35.548333+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:13.872588+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"66'8","reported_seq":1295,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.738567+0000","last_change":"2024-09-16T09:44:52.709379+0000","last_active":"2024-09-16T09:44:55.738567+0000","last_peered":"2024-09-16T09:44:55.738567+0000","last_clean":"2024-09-16T09:44:55.738567+0000","last_became_active":"2024-09-16T09:25:05.712516+0000","last_became_peered":"2024-09-16T09:25:05.712516+0000","last_unstale":"2024-09-16T09:44:55.738567+0000","last_undegraded":"2024-09-16T09:44:55.738567+0000","last_fullsized":"2024-09-16T09:44:55.738567+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:44:52.709313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:44:52.709313+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:00.116704+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.a","version":"73'79","reported_seq":961,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.791468+0000","last_change":"2024-09-16T09:44:30.838199+0000","last_active":"2024-09-16T09:44:55.791468+0000","last_peered":"2024-09-16T09:44:55.791468+0000","last_clean":"2024-09-16T09:44:55.791468+0000","last_became_active":"2024-09-16T09:44:24.278444+0000","last_became_peered":"2024-09-16T09:44:24.278444+0000","last_unstale":"2024-09-16T09:44:55.791468+0000","last_undegraded":"2024-09-16T09:44:55.791468+0000","last_fullsized":"2024-09-16T09:44:55.791468+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:44:30.838127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:44:30.838127+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:55.549500+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"285'27","reported_seq":956,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.622454+0000","last_change":"2024-09-16T09:44:45.023239+0000","last_active":"2024-09-16T09:44:55.622454+0000","last_peered":"2024-09-16T09:44:55.622454+0000","last_clean":"2024-09-16T09:44:55.622454+0000","last_became_active":"2024-09-16T09:44:24.283121+0000","last_became_peered":"2024-09-16T09:44:24.283121+0000","last_unstale":"2024-09-16T09:44:55.622454+0000","last_undegraded":"2024-09-16T09:44:55.622454+0000","last_fullsized":"2024-09-16T09:44:55.622454+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'27","last_scrub_stamp":"2024-09-16T09:44:45.023173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:44:45.023173+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:12.611250+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":745,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.568803+0000","last_change":"2024-09-16T09:38:35.023594+0000","last_active":"2024-09-16T09:44:55.568803+0000","last_peered":"2024-09-16T09:44:55.568803+0000","last_clean":"2024-09-16T09:44:55.568803+0000","last_became_active":"2024-09-16T09:31:09.274534+0000","last_became_peered":"2024-09-16T09:31:09.274534+0000","last_unstale":"2024-09-16T09:44:55.568803+0000","last_undegraded":"2024-09-16T09:44:55.568803+0000","last_fullsized":"2024-09-16T09:44:55.568803+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:38:35.023518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:38:35.023518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:13.168191+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"66'8","reported_seq":1250,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.573391+0000","last_change":"2024-09-16T09:44:50.326230+0000","last_active":"2024-09-16T09:44:55.573391+0000","last_peered":"2024-09-16T09:44:55.573391+0000","last_clean":"2024-09-16T09:44:55.573391+0000","last_became_active":"2024-09-16T09:25:05.713813+0000","last_became_peered":"2024-09-16T09:25:05.713813+0000","last_unstale":"2024-09-16T09:44:55.573391+0000","last_undegraded":"2024-09-16T09:44:55.573391+0000","last_fullsized":"2024-09-16T09:44:55.573391+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:44:50.326143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:44:50.326143+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:57.080144+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1014,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.792008+0000","last_change":"2024-09-16T09:44:02.749106+0000","last_active":"2024-09-16T09:44:55.792008+0000","last_peered":"2024-09-16T09:44:55.792008+0000","last_clean":"2024-09-16T09:44:55.792008+0000","last_became_active":"2024-09-16T09:25:07.717344+0000","last_became_peered":"2024-09-16T09:25:07.717344+0000","last_unstale":"2024-09-16T09:44:55.792008+0000","last_undegraded":"2024-09-16T09:44:55.792008+0000","last_fullsized":"2024-09-16T09:44:55.792008+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:02.748894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:44:02.748894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:29.230981+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.5","version":"73'79","reported_seq":992,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.573359+0000","last_change":"2024-09-16T09:44:27.405392+0000","last_active":"2024-09-16T09:44:55.573359+0000","last_peered":"2024-09-16T09:44:55.573359+0000","last_clean":"2024-09-16T09:44:55.573359+0000","last_became_active":"2024-09-16T09:44:24.276207+0000","last_became_peered":"2024-09-16T09:44:24.276207+0000","last_unstale":"2024-09-16T09:44:55.573359+0000","last_undegraded":"2024-09-16T09:44:55.573359+0000","last_fullsized":"2024-09-16T09:44:55.573359+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:44:27.405342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:44:27.405342+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:29.853871+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"285'30","reported_seq":982,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.620142+0000","last_change":"2024-09-16T09:43:50.371697+0000","last_active":"2024-09-16T09:44:55.620142+0000","last_peered":"2024-09-16T09:44:55.620142+0000","last_clean":"2024-09-16T09:44:55.620142+0000","last_became_active":"2024-09-16T09:31:09.270434+0000","last_became_peered":"2024-09-16T09:31:09.270434+0000","last_unstale":"2024-09-16T09:44:55.620142+0000","last_undegraded":"2024-09-16T09:44:55.620142+0000","last_fullsized":"2024-09-16T09:44:55.620142+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'30","last_scrub_stamp":"2024-09-16T09:43:50.371625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:43:50.371625+0000","objects_scrubbed":8,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:03.401709+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":51,"num_read_kb":39,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":861,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.621308+0000","last_change":"2024-09-16T09:44:00.511038+0000","last_active":"2024-09-16T09:44:55.621308+0000","last_peered":"2024-09-16T09:44:55.621308+0000","last_clean":"2024-09-16T09:44:55.621308+0000","last_became_active":"2024-09-16T09:28:52.983543+0000","last_became_peered":"2024-09-16T09:28:52.983543+0000","last_unstale":"2024-09-16T09:44:55.621308+0000","last_undegraded":"2024-09-16T09:44:55.621308+0000","last_fullsized":"2024-09-16T09:44:55.621308+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:00.510827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:44:00.510827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:29.877213+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1025,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.572001+0000","last_change":"2024-09-16T09:42:09.103716+0000","last_active":"2024-09-16T09:44:55.572001+0000","last_peered":"2024-09-16T09:44:55.572001+0000","last_clean":"2024-09-16T09:44:55.572001+0000","last_became_active":"2024-09-16T09:31:09.268864+0000","last_became_peered":"2024-09-16T09:31:09.268864+0000","last_unstale":"2024-09-16T09:44:55.572001+0000","last_undegraded":"2024-09-16T09:44:55.572001+0000","last_fullsized":"2024-09-16T09:44:55.572001+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:42:09.103657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:42:09.103657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1040,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.572919+0000","last_change":"2024-09-16T09:44:29.376932+0000","last_active":"2024-09-16T09:44:55.572919+0000","last_peered":"2024-09-16T09:44:55.572919+0000","last_clean":"2024-09-16T09:44:55.572919+0000","last_became_active":"2024-09-16T09:44:24.276365+0000","last_became_peered":"2024-09-16T09:44:24.276365+0000","last_unstale":"2024-09-16T09:44:55.572919+0000","last_undegraded":"2024-09-16T09:44:55.572919+0000","last_fullsized":"2024-09-16T09:44:55.572919+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:29.376874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:44:29.376874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:30.303288+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"73'79","reported_seq":753,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.570030+0000","last_change":"2024-09-16T09:44:21.229944+0000","last_active":"2024-09-16T09:44:55.570030+0000","last_peered":"2024-09-16T09:44:55.570030+0000","last_clean":"2024-09-16T09:44:55.570030+0000","last_became_active":"2024-09-16T09:31:09.275139+0000","last_became_peered":"2024-09-16T09:31:09.275139+0000","last_unstale":"2024-09-16T09:44:55.570030+0000","last_undegraded":"2024-09-16T09:44:55.570030+0000","last_fullsized":"2024-09-16T09:44:55.570030+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:44:21.229868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:44:21.229868+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:39.951698+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"285'28","reported_seq":1065,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.572041+0000","last_change":"2024-09-16T09:43:18.126451+0000","last_active":"2024-09-16T09:44:55.572041+0000","last_peered":"2024-09-16T09:44:55.572041+0000","last_clean":"2024-09-16T09:44:55.572041+0000","last_became_active":"2024-09-16T09:25:03.708830+0000","last_became_peered":"2024-09-16T09:25:03.708830+0000","last_unstale":"2024-09-16T09:44:55.572041+0000","last_undegraded":"2024-09-16T09:44:55.572041+0000","last_fullsized":"2024-09-16T09:44:55.572041+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'28","last_scrub_stamp":"2024-09-16T09:43:18.126360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:43:18.126360+0000","objects_scrubbed":7,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":1035,"reported_epoch":538,"state":"active+clean","last_fresh":"2024-09-16T09:44:54.559303+0000","last_change":"2024-09-16T09:40:31.037325+0000","last_active":"2024-09-16T09:44:54.559303+0000","last_peered":"2024-09-16T09:44:54.559303+0000","last_clean":"2024-09-16T09:44:54.559303+0000","last_became_active":"2024-09-16T09:25:01.702992+0000","last_became_peered":"2024-09-16T09:25:01.702992+0000","last_unstale":"2024-09-16T09:44:54.559303+0000","last_undegraded":"2024-09-16T09:44:54.559303+0000","last_fullsized":"2024-09-16T09:44:54.559303+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:40:31.037258+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:40:31.037258+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:41.195033+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"235'12","reported_seq":806,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.571790+0000","last_change":"2024-09-16T09:42:27.113239+0000","last_active":"2024-09-16T09:44:55.571790+0000","last_peered":"2024-09-16T09:44:55.571790+0000","last_clean":"2024-09-16T09:44:55.571790+0000","last_became_active":"2024-09-16T09:33:30.251047+0000","last_became_peered":"2024-09-16T09:33:30.251047+0000","last_unstale":"2024-09-16T09:44:55.571790+0000","last_undegraded":"2024-09-16T09:44:55.571790+0000","last_fullsized":"2024-09-16T09:44:55.571790+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"235'12","last_scrub_stamp":"2024-09-16T09:42:27.113174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:42:27.113174+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1016,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.620321+0000","last_change":"2024-09-16T09:44:29.455025+0000","last_active":"2024-09-16T09:44:55.620321+0000","last_peered":"2024-09-16T09:44:55.620321+0000","last_clean":"2024-09-16T09:44:55.620321+0000","last_became_active":"2024-09-16T09:44:24.277003+0000","last_became_peered":"2024-09-16T09:44:24.277003+0000","last_unstale":"2024-09-16T09:44:55.620321+0000","last_undegraded":"2024-09-16T09:44:55.620321+0000","last_fullsized":"2024-09-16T09:44:55.620321+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:29.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:44:29.454958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:30.251581+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"73'79","reported_seq":1092,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.738394+0000","last_change":"2024-09-16T09:43:55.930749+0000","last_active":"2024-09-16T09:44:55.738394+0000","last_peered":"2024-09-16T09:44:55.738394+0000","last_clean":"2024-09-16T09:44:55.738394+0000","last_became_active":"2024-09-16T09:31:09.274707+0000","last_became_peered":"2024-09-16T09:31:09.274707+0000","last_unstale":"2024-09-16T09:44:55.738394+0000","last_undegraded":"2024-09-16T09:44:55.738394+0000","last_fullsized":"2024-09-16T09:44:55.738394+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:43:55.930694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:43:55.930694+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:15.421892+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"285'20","reported_seq":693,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.791912+0000","last_change":"2024-09-16T09:38:32.122419+0000","last_active":"2024-09-16T09:44:55.791912+0000","last_peered":"2024-09-16T09:44:55.791912+0000","last_clean":"2024-09-16T09:44:55.791912+0000","last_became_active":"2024-09-16T09:33:30.248453+0000","last_became_peered":"2024-09-16T09:33:30.248453+0000","last_unstale":"2024-09-16T09:44:55.791912+0000","last_undegraded":"2024-09-16T09:44:55.791912+0000","last_fullsized":"2024-09-16T09:44:55.791912+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'20","last_scrub_stamp":"2024-09-16T09:38:32.122303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:38:32.122303+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:08.800937+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":1036,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.792394+0000","last_change":"2024-09-16T09:44:25.924933+0000","last_active":"2024-09-16T09:44:55.792394+0000","last_peered":"2024-09-16T09:44:55.792394+0000","last_clean":"2024-09-16T09:44:55.792394+0000","last_became_active":"2024-09-16T09:44:24.280657+0000","last_became_peered":"2024-09-16T09:44:24.280657+0000","last_unstale":"2024-09-16T09:44:55.792394+0000","last_undegraded":"2024-09-16T09:44:55.792394+0000","last_fullsized":"2024-09-16T09:44:55.792394+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:25.924858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:44:25.924858+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:35.232847+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"66'8","reported_seq":1257,"reported_epoch":538,"state":"active+clean","last_fresh":"2024-09-16T09:44:54.559754+0000","last_change":"2024-09-16T09:44:00.808193+0000","last_active":"2024-09-16T09:44:54.559754+0000","last_peered":"2024-09-16T09:44:54.559754+0000","last_clean":"2024-09-16T09:44:54.559754+0000","last_became_active":"2024-09-16T09:25:05.714507+0000","last_became_peered":"2024-09-16T09:25:05.714507+0000","last_unstale":"2024-09-16T09:44:54.559754+0000","last_undegraded":"2024-09-16T09:44:54.559754+0000","last_fullsized":"2024-09-16T09:44:54.559754+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:44:00.808121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:44:00.808121+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:28.025271+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1039,"reported_epoch":538,"state":"active+clean","last_fresh":"2024-09-16T09:44:54.558017+0000","last_change":"2024-09-16T09:44:25.125306+0000","last_active":"2024-09-16T09:44:54.558017+0000","last_peered":"2024-09-16T09:44:54.558017+0000","last_clean":"2024-09-16T09:44:54.558017+0000","last_became_active":"2024-09-16T09:44:24.269736+0000","last_became_peered":"2024-09-16T09:44:24.269736+0000","last_unstale":"2024-09-16T09:44:54.558017+0000","last_undegraded":"2024-09-16T09:44:54.558017+0000","last_fullsized":"2024-09-16T09:44:54.558017+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:25.125254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:44:25.125254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:42.194714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"73'79","reported_seq":1049,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.738182+0000","last_change":"2024-09-16T09:43:50.904918+0000","last_active":"2024-09-16T09:44:55.738182+0000","last_peered":"2024-09-16T09:44:55.738182+0000","last_clean":"2024-09-16T09:44:55.738182+0000","last_became_active":"2024-09-16T09:31:09.274565+0000","last_became_peered":"2024-09-16T09:31:09.274565+0000","last_unstale":"2024-09-16T09:44:55.738182+0000","last_undegraded":"2024-09-16T09:44:55.738182+0000","last_fullsized":"2024-09-16T09:44:55.738182+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:43:50.904841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:43:50.904841+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:09.506473+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"285'29","reported_seq":1057,"reported_epoch":538,"state":"active+clean","last_fresh":"2024-09-16T09:44:54.559971+0000","last_change":"2024-09-16T09:42:24.900527+0000","last_active":"2024-09-16T09:44:54.559971+0000","last_peered":"2024-09-16T09:44:54.559971+0000","last_clean":"2024-09-16T09:44:54.559971+0000","last_became_active":"2024-09-16T09:25:03.714797+0000","last_became_peered":"2024-09-16T09:25:03.714797+0000","last_unstale":"2024-09-16T09:44:54.559971+0000","last_undegraded":"2024-09-16T09:44:54.559971+0000","last_fullsized":"2024-09-16T09:44:54.559971+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'29","last_scrub_stamp":"2024-09-16T09:42:24.900426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:42:24.900426+0000","objects_scrubbed":8,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":54,"num_read_kb":40,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":654,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.792001+0000","last_change":"2024-09-16T09:44:27.885527+0000","last_active":"2024-09-16T09:44:55.792001+0000","last_peered":"2024-09-16T09:44:55.792001+0000","last_clean":"2024-09-16T09:44:55.792001+0000","last_became_active":"2024-09-16T09:44:24.279029+0000","last_became_peered":"2024-09-16T09:44:24.279029+0000","last_unstale":"2024-09-16T09:44:55.792001+0000","last_undegraded":"2024-09-16T09:44:55.792001+0000","last_fullsized":"2024-09-16T09:44:55.792001+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:27.885460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:44:27.885460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:50.933082+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,2],"acting":[5,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1036,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.737825+0000","last_change":"2024-09-16T09:44:51.704537+0000","last_active":"2024-09-16T09:44:55.737825+0000","last_peered":"2024-09-16T09:44:55.737825+0000","last_clean":"2024-09-16T09:44:55.737825+0000","last_became_active":"2024-09-16T09:25:05.708663+0000","last_became_peered":"2024-09-16T09:25:05.708663+0000","last_unstale":"2024-09-16T09:44:55.737825+0000","last_undegraded":"2024-09-16T09:44:55.737825+0000","last_fullsized":"2024-09-16T09:44:55.737825+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:51.704482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:44:51.704482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:16.177847+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":662,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.737530+0000","last_change":"2024-09-16T09:44:25.629415+0000","last_active":"2024-09-16T09:44:55.737530+0000","last_peered":"2024-09-16T09:44:55.737530+0000","last_clean":"2024-09-16T09:44:55.737530+0000","last_became_active":"2024-09-16T09:44:24.277522+0000","last_became_peered":"2024-09-16T09:44:24.277522+0000","last_unstale":"2024-09-16T09:44:55.737530+0000","last_undegraded":"2024-09-16T09:44:55.737530+0000","last_fullsized":"2024-09-16T09:44:55.737530+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:25.629370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:44:25.629370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:54.323231+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,2],"acting":[6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"73'79","reported_seq":993,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.791504+0000","last_change":"2024-09-16T09:44:29.831921+0000","last_active":"2024-09-16T09:44:55.791504+0000","last_peered":"2024-09-16T09:44:55.791504+0000","last_clean":"2024-09-16T09:44:55.791504+0000","last_became_active":"2024-09-16T09:44:24.279593+0000","last_became_peered":"2024-09-16T09:44:24.279593+0000","last_unstale":"2024-09-16T09:44:55.791504+0000","last_undegraded":"2024-09-16T09:44:55.791504+0000","last_fullsized":"2024-09-16T09:44:55.791504+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:44:29.831857+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:44:29.831857+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:59.570916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"519'44","reported_seq":1417,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.570270+0000","last_change":"2024-09-16T09:44:30.314706+0000","last_active":"2024-09-16T09:44:55.570270+0000","last_peered":"2024-09-16T09:44:55.570270+0000","last_clean":"2024-09-16T09:44:55.570270+0000","last_became_active":"2024-09-16T09:44:24.280130+0000","last_became_peered":"2024-09-16T09:44:24.280130+0000","last_unstale":"2024-09-16T09:44:55.570270+0000","last_undegraded":"2024-09-16T09:44:55.570270+0000","last_fullsized":"2024-09-16T09:44:55.570270+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"519'44","last_scrub_stamp":"2024-09-16T09:44:30.314606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:44:30.314606+0000","objects_scrubbed":10,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:34.762411+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":66,"num_read_kb":51,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":1027,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.572504+0000","last_change":"2024-09-16T09:41:40.124915+0000","last_active":"2024-09-16T09:44:55.572504+0000","last_peered":"2024-09-16T09:44:55.572504+0000","last_clean":"2024-09-16T09:44:55.572504+0000","last_became_active":"2024-09-16T09:25:01.703076+0000","last_became_peered":"2024-09-16T09:25:01.703076+0000","last_unstale":"2024-09-16T09:44:55.572504+0000","last_undegraded":"2024-09-16T09:44:55.572504+0000","last_fullsized":"2024-09-16T09:44:55.572504+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:40.124830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:41:40.124830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:07.248004+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":1037,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.792077+0000","last_change":"2024-09-16T09:43:52.704491+0000","last_active":"2024-09-16T09:44:55.792077+0000","last_peered":"2024-09-16T09:44:55.792077+0000","last_clean":"2024-09-16T09:44:55.792077+0000","last_became_active":"2024-09-16T09:25:00.696237+0000","last_became_peered":"2024-09-16T09:25:00.696237+0000","last_unstale":"2024-09-16T09:44:55.792077+0000","last_undegraded":"2024-09-16T09:44:55.792077+0000","last_fullsized":"2024-09-16T09:44:55.792077+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-16T09:43:52.704431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:43:52.704431+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:20.772940+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.0025137430000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1002,"reported_epoch":538,"state":"active+clean","last_fresh":"2024-09-16T09:44:54.558190+0000","last_change":"2024-09-16T09:44:03.793464+0000","last_active":"2024-09-16T09:44:54.558190+0000","last_peered":"2024-09-16T09:44:54.558190+0000","last_clean":"2024-09-16T09:44:54.558190+0000","last_became_active":"2024-09-16T09:25:05.717591+0000","last_became_peered":"2024-09-16T09:25:05.717591+0000","last_unstale":"2024-09-16T09:44:54.558190+0000","last_undegraded":"2024-09-16T09:44:54.558190+0000","last_fullsized":"2024-09-16T09:44:54.558190+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:03.793400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:44:03.793400+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:29.631956+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":753,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.570863+0000","last_change":"2024-09-16T09:44:35.215569+0000","last_active":"2024-09-16T09:44:55.570863+0000","last_peered":"2024-09-16T09:44:55.570863+0000","last_clean":"2024-09-16T09:44:55.570863+0000","last_became_active":"2024-09-16T09:31:09.272609+0000","last_became_peered":"2024-09-16T09:31:09.272609+0000","last_unstale":"2024-09-16T09:44:55.570863+0000","last_undegraded":"2024-09-16T09:44:55.570863+0000","last_fullsized":"2024-09-16T09:44:55.570863+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:35.215527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:44:35.215527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:03.125091+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"73'79","reported_seq":349,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.620101+0000","last_change":"2024-09-16T09:44:22.398722+0000","last_active":"2024-09-16T09:44:55.620101+0000","last_peered":"2024-09-16T09:44:55.620101+0000","last_clean":"2024-09-16T09:44:55.620101+0000","last_became_active":"2024-09-16T09:38:10.556320+0000","last_became_peered":"2024-09-16T09:38:10.556320+0000","last_unstale":"2024-09-16T09:44:55.620101+0000","last_undegraded":"2024-09-16T09:44:55.620101+0000","last_fullsized":"2024-09-16T09:44:55.620101+0000","mapping_epoch":386,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":387,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:44:22.398656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:44:22.398656+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:28.557065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"285'17","reported_seq":742,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.792571+0000","last_change":"2024-09-16T09:43:49.706772+0000","last_active":"2024-09-16T09:44:55.792571+0000","last_peered":"2024-09-16T09:44:55.792571+0000","last_clean":"2024-09-16T09:44:55.792571+0000","last_became_active":"2024-09-16T09:37:33.349009+0000","last_became_peered":"2024-09-16T09:37:33.349009+0000","last_unstale":"2024-09-16T09:44:55.792571+0000","last_undegraded":"2024-09-16T09:44:55.792571+0000","last_fullsized":"2024-09-16T09:44:55.792571+0000","mapping_epoch":367,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":368,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'17","last_scrub_stamp":"2024-09-16T09:43:49.706685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:43:49.706685+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":35,"num_read_kb":24,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":662,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.737492+0000","last_change":"2024-09-16T09:44:30.666378+0000","last_active":"2024-09-16T09:44:55.737492+0000","last_peered":"2024-09-16T09:44:55.737492+0000","last_clean":"2024-09-16T09:44:55.737492+0000","last_became_active":"2024-09-16T09:44:24.270134+0000","last_became_peered":"2024-09-16T09:44:24.270134+0000","last_unstale":"2024-09-16T09:44:55.737492+0000","last_undegraded":"2024-09-16T09:44:55.737492+0000","last_fullsized":"2024-09-16T09:44:55.737492+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:30.666333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:44:30.666333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:36.645744+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,2],"acting":[6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":698,"reported_epoch":538,"state":"active+clean","last_fresh":"2024-09-16T09:44:54.560118+0000","last_change":"2024-09-16T09:44:24.280992+0000","last_active":"2024-09-16T09:44:54.560118+0000","last_peered":"2024-09-16T09:44:54.560118+0000","last_clean":"2024-09-16T09:44:54.560118+0000","last_became_active":"2024-09-16T09:44:24.279158+0000","last_became_peered":"2024-09-16T09:44:24.279158+0000","last_unstale":"2024-09-16T09:44:54.560118+0000","last_undegraded":"2024-09-16T09:44:54.560118+0000","last_fullsized":"2024-09-16T09:44:54.560118+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:41:42.038861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:41:42.038861+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3],"acting":[0,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":946,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.622506+0000","last_change":"2024-09-16T09:44:24.286071+0000","last_active":"2024-09-16T09:44:55.622506+0000","last_peered":"2024-09-16T09:44:55.622506+0000","last_clean":"2024-09-16T09:44:55.622506+0000","last_became_active":"2024-09-16T09:44:24.285805+0000","last_became_peered":"2024-09-16T09:44:24.285805+0000","last_unstale":"2024-09-16T09:44:55.622506+0000","last_undegraded":"2024-09-16T09:44:55.622506+0000","last_fullsized":"2024-09-16T09:44:55.622506+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:45.549186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:41:45.549186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:39.079232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1013,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.620718+0000","last_change":"2024-09-16T09:44:48.555675+0000","last_active":"2024-09-16T09:44:55.620718+0000","last_peered":"2024-09-16T09:44:55.620718+0000","last_clean":"2024-09-16T09:44:55.620718+0000","last_became_active":"2024-09-16T09:25:07.723282+0000","last_became_peered":"2024-09-16T09:25:07.723282+0000","last_unstale":"2024-09-16T09:44:55.620718+0000","last_undegraded":"2024-09-16T09:44:55.620718+0000","last_fullsized":"2024-09-16T09:44:55.620718+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:48.555614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:44:48.555614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:18.258254+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"73'79","reported_seq":732,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.737921+0000","last_change":"2024-09-16T09:43:53.953716+0000","last_active":"2024-09-16T09:44:55.737921+0000","last_peered":"2024-09-16T09:44:55.737921+0000","last_clean":"2024-09-16T09:44:55.737921+0000","last_became_active":"2024-09-16T09:33:30.249309+0000","last_became_peered":"2024-09-16T09:33:30.249309+0000","last_unstale":"2024-09-16T09:44:55.737921+0000","last_undegraded":"2024-09-16T09:44:55.737921+0000","last_fullsized":"2024-09-16T09:44:55.737921+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:43:53.953557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:43:53.953557+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:12.516809+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.6","version":"285'24","reported_seq":1061,"reported_epoch":538,"state":"active+clean","last_fresh":"2024-09-16T09:44:54.559123+0000","last_change":"2024-09-16T09:44:24.278226+0000","last_active":"2024-09-16T09:44:54.559123+0000","last_peered":"2024-09-16T09:44:54.559123+0000","last_clean":"2024-09-16T09:44:54.559123+0000","last_became_active":"2024-09-16T09:44:24.277878+0000","last_became_peered":"2024-09-16T09:44:24.277878+0000","last_unstale":"2024-09-16T09:44:54.559123+0000","last_undegraded":"2024-09-16T09:44:54.559123+0000","last_fullsized":"2024-09-16T09:44:54.559123+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'24","last_scrub_stamp":"2024-09-16T09:42:11.911889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:42:11.911889+0000","objects_scrubbed":6,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":36,"num_read_kb":30,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"59'1","reported_seq":1049,"reported_epoch":538,"state":"active+clean","last_fresh":"2024-09-16T09:44:54.559845+0000","last_change":"2024-09-16T09:42:56.896466+0000","last_active":"2024-09-16T09:44:54.559845+0000","last_peered":"2024-09-16T09:44:54.559845+0000","last_clean":"2024-09-16T09:44:54.559845+0000","last_became_active":"2024-09-16T09:31:09.269627+0000","last_became_peered":"2024-09-16T09:31:09.269627+0000","last_unstale":"2024-09-16T09:44:54.559845+0000","last_undegraded":"2024-09-16T09:44:54.559845+0000","last_fullsized":"2024-09-16T09:44:54.559845+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:42:56.896256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:42:56.896256+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1986,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"67'5","reported_seq":2216,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.746420+0000","last_change":"2024-09-16T09:41:43.153303+0000","last_active":"2024-09-16T09:44:55.746420+0000","last_peered":"2024-09-16T09:44:55.746420+0000","last_clean":"2024-09-16T09:44:55.746420+0000","last_became_active":"2024-09-16T09:25:00.696546+0000","last_became_peered":"2024-09-16T09:25:00.696546+0000","last_unstale":"2024-09-16T09:44:55.746420+0000","last_undegraded":"2024-09-16T09:44:55.746420+0000","last_fullsized":"2024-09-16T09:44:55.746420+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'5","last_scrub_stamp":"2024-09-16T09:41:43.153174+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:41:43.153174+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:42.518308+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0.0059770420000000001,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1188,"num_read_kb":1183,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":995,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.792644+0000","last_change":"2024-09-16T09:44:19.980692+0000","last_active":"2024-09-16T09:44:55.792644+0000","last_peered":"2024-09-16T09:44:55.792644+0000","last_clean":"2024-09-16T09:44:55.792644+0000","last_became_active":"2024-09-16T09:25:05.714704+0000","last_became_peered":"2024-09-16T09:25:05.714704+0000","last_unstale":"2024-09-16T09:44:55.792644+0000","last_undegraded":"2024-09-16T09:44:55.792644+0000","last_fullsized":"2024-09-16T09:44:55.792644+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:19.980625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:44:19.980625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:20.184735+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":929,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.792164+0000","last_change":"2024-09-16T09:44:04.828342+0000","last_active":"2024-09-16T09:44:55.792164+0000","last_peered":"2024-09-16T09:44:55.792164+0000","last_clean":"2024-09-16T09:44:55.792164+0000","last_became_active":"2024-09-16T09:33:30.245852+0000","last_became_peered":"2024-09-16T09:33:30.245852+0000","last_unstale":"2024-09-16T09:44:55.792164+0000","last_undegraded":"2024-09-16T09:44:55.792164+0000","last_fullsized":"2024-09-16T09:44:55.792164+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:04.828272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:44:04.828272+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:28.332272+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.5","version":"285'28","reported_seq":1304,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.738302+0000","last_change":"2024-09-16T09:43:47.926939+0000","last_active":"2024-09-16T09:44:55.738302+0000","last_peered":"2024-09-16T09:44:55.738302+0000","last_clean":"2024-09-16T09:44:55.738302+0000","last_became_active":"2024-09-16T09:25:03.714118+0000","last_became_peered":"2024-09-16T09:25:03.714118+0000","last_unstale":"2024-09-16T09:44:55.738302+0000","last_undegraded":"2024-09-16T09:44:55.738302+0000","last_fullsized":"2024-09-16T09:44:55.738302+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'28","last_scrub_stamp":"2024-09-16T09:43:47.926855+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:43:47.926855+0000","objects_scrubbed":8,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":43,"num_read_kb":33,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":869,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.791991+0000","last_change":"2024-09-16T09:44:34.758789+0000","last_active":"2024-09-16T09:44:55.791991+0000","last_peered":"2024-09-16T09:44:55.791991+0000","last_clean":"2024-09-16T09:44:55.791991+0000","last_became_active":"2024-09-16T09:31:09.270196+0000","last_became_peered":"2024-09-16T09:31:09.270196+0000","last_unstale":"2024-09-16T09:44:55.791991+0000","last_undegraded":"2024-09-16T09:44:55.791991+0000","last_fullsized":"2024-09-16T09:44:55.791991+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:34.758733+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:44:34.758733+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:37.220493+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"73'79","reported_seq":1097,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.620359+0000","last_change":"2024-09-16T09:42:56.661708+0000","last_active":"2024-09-16T09:44:55.620359+0000","last_peered":"2024-09-16T09:44:55.620359+0000","last_clean":"2024-09-16T09:44:55.620359+0000","last_became_active":"2024-09-16T09:30:08.870394+0000","last_became_peered":"2024-09-16T09:30:08.870394+0000","last_unstale":"2024-09-16T09:44:55.620359+0000","last_undegraded":"2024-09-16T09:44:55.620359+0000","last_fullsized":"2024-09-16T09:44:55.620359+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:42:56.661377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:42:56.661377+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1020,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.620741+0000","last_change":"2024-09-16T09:44:24.285202+0000","last_active":"2024-09-16T09:44:55.620741+0000","last_peered":"2024-09-16T09:44:55.620741+0000","last_clean":"2024-09-16T09:44:55.620741+0000","last_became_active":"2024-09-16T09:44:24.284362+0000","last_became_peered":"2024-09-16T09:44:24.284362+0000","last_unstale":"2024-09-16T09:44:55.620741+0000","last_undegraded":"2024-09-16T09:44:55.620741+0000","last_fullsized":"2024-09-16T09:44:55.620741+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:42:11.246697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:42:11.246697+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1007,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.792552+0000","last_change":"2024-09-16T09:44:46.680178+0000","last_active":"2024-09-16T09:44:55.792552+0000","last_peered":"2024-09-16T09:44:55.792552+0000","last_clean":"2024-09-16T09:44:55.792552+0000","last_became_active":"2024-09-16T09:25:07.714281+0000","last_became_peered":"2024-09-16T09:25:07.714281+0000","last_unstale":"2024-09-16T09:44:55.792552+0000","last_undegraded":"2024-09-16T09:44:55.792552+0000","last_fullsized":"2024-09-16T09:44:55.792552+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:46.680120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:44:46.680120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:08.327463+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.b","version":"73'79","reported_seq":841,"reported_epoch":538,"state":"active+clean","last_fresh":"2024-09-16T09:44:54.559401+0000","last_change":"2024-09-16T09:42:26.898435+0000","last_active":"2024-09-16T09:44:54.559401+0000","last_peered":"2024-09-16T09:44:54.559401+0000","last_clean":"2024-09-16T09:44:54.559401+0000","last_became_active":"2024-09-16T09:33:30.251933+0000","last_became_peered":"2024-09-16T09:33:30.251933+0000","last_unstale":"2024-09-16T09:44:54.559401+0000","last_undegraded":"2024-09-16T09:44:54.559401+0000","last_fullsized":"2024-09-16T09:44:54.559401+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:42:26.898235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:42:26.898235+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,5],"acting":[0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"285'19","reported_seq":1094,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.573410+0000","last_change":"2024-09-16T09:43:22.166919+0000","last_active":"2024-09-16T09:44:55.573410+0000","last_peered":"2024-09-16T09:44:55.573410+0000","last_clean":"2024-09-16T09:44:55.573410+0000","last_became_active":"2024-09-16T09:37:33.348433+0000","last_became_peered":"2024-09-16T09:37:33.348433+0000","last_unstale":"2024-09-16T09:44:55.573410+0000","last_undegraded":"2024-09-16T09:44:55.573410+0000","last_fullsized":"2024-09-16T09:44:55.573410+0000","mapping_epoch":367,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":368,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'19","last_scrub_stamp":"2024-09-16T09:43:22.166834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:43:22.166834+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":1014,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.572642+0000","last_change":"2024-09-16T09:44:24.277287+0000","last_active":"2024-09-16T09:44:55.572642+0000","last_peered":"2024-09-16T09:44:55.572642+0000","last_clean":"2024-09-16T09:44:55.572642+0000","last_became_active":"2024-09-16T09:44:24.276997+0000","last_became_peered":"2024-09-16T09:44:24.276997+0000","last_unstale":"2024-09-16T09:44:55.572642+0000","last_undegraded":"2024-09-16T09:44:55.572642+0000","last_fullsized":"2024-09-16T09:44:55.572642+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:42:18.132091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:42:18.132091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":986,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.792461+0000","last_change":"2024-09-16T09:42:18.730769+0000","last_active":"2024-09-16T09:44:55.792461+0000","last_peered":"2024-09-16T09:44:55.792461+0000","last_clean":"2024-09-16T09:44:55.792461+0000","last_became_active":"2024-09-16T09:25:05.711964+0000","last_became_peered":"2024-09-16T09:25:05.711964+0000","last_unstale":"2024-09-16T09:44:55.792461+0000","last_undegraded":"2024-09-16T09:44:55.792461+0000","last_fullsized":"2024-09-16T09:44:55.792461+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:42:18.730723+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:42:18.730723+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:14.838460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":889,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.738093+0000","last_change":"2024-09-16T09:44:37.625415+0000","last_active":"2024-09-16T09:44:55.738093+0000","last_peered":"2024-09-16T09:44:55.738093+0000","last_clean":"2024-09-16T09:44:55.738093+0000","last_became_active":"2024-09-16T09:31:09.278618+0000","last_became_peered":"2024-09-16T09:31:09.278618+0000","last_unstale":"2024-09-16T09:44:55.738093+0000","last_undegraded":"2024-09-16T09:44:55.738093+0000","last_fullsized":"2024-09-16T09:44:55.738093+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:42:09.944311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:42:09.944311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:06.727839+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.8","version":"73'79","reported_seq":365,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.791368+0000","last_change":"2024-09-16T09:44:31.814201+0000","last_active":"2024-09-16T09:44:55.791368+0000","last_peered":"2024-09-16T09:44:55.791368+0000","last_clean":"2024-09-16T09:44:55.791368+0000","last_became_active":"2024-09-16T09:44:24.270354+0000","last_became_peered":"2024-09-16T09:44:24.270354+0000","last_unstale":"2024-09-16T09:44:55.791368+0000","last_undegraded":"2024-09-16T09:44:55.791368+0000","last_fullsized":"2024-09-16T09:44:55.791368+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:44:31.814152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:44:31.814152+0000","objects_scrubbed":1,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:39.578957+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262688,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"285'31","reported_seq":1074,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.573236+0000","last_change":"2024-09-16T09:44:24.277703+0000","last_active":"2024-09-16T09:44:55.573236+0000","last_peered":"2024-09-16T09:44:55.573236+0000","last_clean":"2024-09-16T09:44:55.573236+0000","last_became_active":"2024-09-16T09:44:24.277359+0000","last_became_peered":"2024-09-16T09:44:24.277359+0000","last_unstale":"2024-09-16T09:44:55.573236+0000","last_undegraded":"2024-09-16T09:44:55.573236+0000","last_fullsized":"2024-09-16T09:44:55.573236+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'31","last_scrub_stamp":"2024-09-16T09:42:07.153276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:42:07.153276+0000","objects_scrubbed":8,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":50,"num_read_kb":40,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":1020,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.738033+0000","last_change":"2024-09-16T09:42:53.627570+0000","last_active":"2024-09-16T09:44:55.738033+0000","last_peered":"2024-09-16T09:44:55.738033+0000","last_clean":"2024-09-16T09:44:55.738033+0000","last_became_active":"2024-09-16T09:25:01.703355+0000","last_became_peered":"2024-09-16T09:25:01.703355+0000","last_unstale":"2024-09-16T09:44:55.738033+0000","last_undegraded":"2024-09-16T09:44:55.738033+0000","last_fullsized":"2024-09-16T09:44:55.738033+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:42:53.627505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:42:53.627505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:07.742399+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":650,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.572281+0000","last_change":"2024-09-16T09:44:47.324175+0000","last_active":"2024-09-16T09:44:55.572281+0000","last_peered":"2024-09-16T09:44:55.572281+0000","last_clean":"2024-09-16T09:44:55.572281+0000","last_became_active":"2024-09-16T09:33:30.251186+0000","last_became_peered":"2024-09-16T09:33:30.251186+0000","last_unstale":"2024-09-16T09:44:55.572281+0000","last_undegraded":"2024-09-16T09:44:55.572281+0000","last_fullsized":"2024-09-16T09:44:55.572281+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:47.324096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:44:47.324096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:02.767003+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,0],"acting":[4,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":960,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.570799+0000","last_change":"2024-09-16T09:44:25.180283+0000","last_active":"2024-09-16T09:44:55.570799+0000","last_peered":"2024-09-16T09:44:55.570799+0000","last_clean":"2024-09-16T09:44:55.570799+0000","last_became_active":"2024-09-16T09:44:24.273561+0000","last_became_peered":"2024-09-16T09:44:24.273561+0000","last_unstale":"2024-09-16T09:44:55.570799+0000","last_undegraded":"2024-09-16T09:44:55.570799+0000","last_fullsized":"2024-09-16T09:44:55.570799+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:25.180239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:44:25.180239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:44.445043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4],"acting":[3,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.9","version":"73'79","reported_seq":1092,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.621328+0000","last_change":"2024-09-16T09:44:55.621328+0000","last_active":"2024-09-16T09:44:55.621328+0000","last_peered":"2024-09-16T09:44:55.621328+0000","last_clean":"2024-09-16T09:44:55.621328+0000","last_became_active":"2024-09-16T09:44:24.278896+0000","last_became_peered":"2024-09-16T09:44:24.278896+0000","last_unstale":"2024-09-16T09:44:55.621328+0000","last_undegraded":"2024-09-16T09:44:55.621328+0000","last_fullsized":"2024-09-16T09:44:55.621328+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:44:55.621246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:44:55.621246+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:59.840208+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"285'20","reported_seq":1036,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.572685+0000","last_change":"2024-09-16T09:43:49.220241+0000","last_active":"2024-09-16T09:44:55.572685+0000","last_peered":"2024-09-16T09:44:55.572685+0000","last_clean":"2024-09-16T09:44:55.572685+0000","last_became_active":"2024-09-16T09:25:03.707723+0000","last_became_peered":"2024-09-16T09:25:03.707723+0000","last_unstale":"2024-09-16T09:44:55.572685+0000","last_undegraded":"2024-09-16T09:44:55.572685+0000","last_fullsized":"2024-09-16T09:44:55.572685+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'20","last_scrub_stamp":"2024-09-16T09:43:49.220175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:43:49.220175+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:18.570377+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":875,"reported_epoch":538,"state":"active+clean","last_fresh":"2024-09-16T09:44:54.560071+0000","last_change":"2024-09-16T09:42:52.868096+0000","last_active":"2024-09-16T09:44:54.560071+0000","last_peered":"2024-09-16T09:44:54.560071+0000","last_clean":"2024-09-16T09:44:54.560071+0000","last_became_active":"2024-09-16T09:31:09.288825+0000","last_became_peered":"2024-09-16T09:31:09.288825+0000","last_unstale":"2024-09-16T09:44:54.560071+0000","last_undegraded":"2024-09-16T09:44:54.560071+0000","last_fullsized":"2024-09-16T09:44:54.560071+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:42:52.868035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:42:52.868035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"519'11","reported_seq":843,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.620911+0000","last_change":"2024-09-16T09:44:24.278535+0000","last_active":"2024-09-16T09:44:55.620911+0000","last_peered":"2024-09-16T09:44:55.620911+0000","last_clean":"2024-09-16T09:44:55.620911+0000","last_became_active":"2024-09-16T09:44:24.277662+0000","last_became_peered":"2024-09-16T09:44:24.277662+0000","last_unstale":"2024-09-16T09:44:55.620911+0000","last_undegraded":"2024-09-16T09:44:55.620911+0000","last_fullsized":"2024-09-16T09:44:55.620911+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"228'9","last_scrub_stamp":"2024-09-16T09:41:33.174444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:41:33.174444+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:40.473027+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1041,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.572868+0000","last_change":"2024-09-16T09:44:48.293840+0000","last_active":"2024-09-16T09:44:55.572868+0000","last_peered":"2024-09-16T09:44:55.572868+0000","last_clean":"2024-09-16T09:44:55.572868+0000","last_became_active":"2024-09-16T09:44:24.280724+0000","last_became_peered":"2024-09-16T09:44:24.280724+0000","last_unstale":"2024-09-16T09:44:55.572868+0000","last_undegraded":"2024-09-16T09:44:55.572868+0000","last_fullsized":"2024-09-16T09:44:55.572868+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:48.293789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:44:48.293789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:57.968474+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.e","version":"73'79","reported_seq":807,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.573052+0000","last_change":"2024-09-16T09:42:16.139285+0000","last_active":"2024-09-16T09:44:55.573052+0000","last_peered":"2024-09-16T09:44:55.573052+0000","last_clean":"2024-09-16T09:44:55.573052+0000","last_became_active":"2024-09-16T09:30:10.705072+0000","last_became_peered":"2024-09-16T09:30:10.705072+0000","last_unstale":"2024-09-16T09:44:55.573052+0000","last_undegraded":"2024-09-16T09:44:55.573052+0000","last_fullsized":"2024-09-16T09:44:55.573052+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:42:16.139208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:42:16.139208+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,3],"acting":[4,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.b","version":"285'15","reported_seq":1063,"reported_epoch":538,"state":"active+clean","last_fresh":"2024-09-16T09:44:54.559889+0000","last_change":"2024-09-16T09:44:48.083967+0000","last_active":"2024-09-16T09:44:54.559889+0000","last_peered":"2024-09-16T09:44:54.559889+0000","last_clean":"2024-09-16T09:44:54.559889+0000","last_became_active":"2024-09-16T09:36:31.590202+0000","last_became_peered":"2024-09-16T09:36:31.590202+0000","last_unstale":"2024-09-16T09:44:54.559889+0000","last_undegraded":"2024-09-16T09:44:54.559889+0000","last_fullsized":"2024-09-16T09:44:54.559889+0000","mapping_epoch":316,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":317,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'15","last_scrub_stamp":"2024-09-16T09:44:48.083895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:44:48.083895+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:50.498247+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":1016,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.792319+0000","last_change":"2024-09-16T09:42:30.709733+0000","last_active":"2024-09-16T09:44:55.792319+0000","last_peered":"2024-09-16T09:44:55.792319+0000","last_clean":"2024-09-16T09:44:55.792319+0000","last_became_active":"2024-09-16T09:31:09.276516+0000","last_became_peered":"2024-09-16T09:31:09.276516+0000","last_unstale":"2024-09-16T09:44:55.792319+0000","last_undegraded":"2024-09-16T09:44:55.792319+0000","last_fullsized":"2024-09-16T09:44:55.792319+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:42:30.709673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:42:30.709673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:17.905087+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":947,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.572541+0000","last_change":"2024-09-16T09:44:24.276286+0000","last_active":"2024-09-16T09:44:55.572541+0000","last_peered":"2024-09-16T09:44:55.572541+0000","last_clean":"2024-09-16T09:44:55.572541+0000","last_became_active":"2024-09-16T09:44:24.275961+0000","last_became_peered":"2024-09-16T09:44:24.275961+0000","last_unstale":"2024-09-16T09:44:55.572541+0000","last_undegraded":"2024-09-16T09:44:55.572541+0000","last_fullsized":"2024-09-16T09:44:55.572541+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:42:56.059953+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:42:56.059953+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3],"acting":[4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1017,"reported_epoch":538,"state":"active+clean","last_fresh":"2024-09-16T09:44:54.559759+0000","last_change":"2024-09-16T09:44:24.281500+0000","last_active":"2024-09-16T09:44:54.559759+0000","last_peered":"2024-09-16T09:44:54.559759+0000","last_clean":"2024-09-16T09:44:54.559759+0000","last_became_active":"2024-09-16T09:44:24.278714+0000","last_became_peered":"2024-09-16T09:44:24.278714+0000","last_unstale":"2024-09-16T09:44:54.559759+0000","last_undegraded":"2024-09-16T09:44:54.559759+0000","last_fullsized":"2024-09-16T09:44:54.559759+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:42:19.850990+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:42:19.850990+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.f","version":"73'79","reported_seq":1076,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.738519+0000","last_change":"2024-09-16T09:43:24.748100+0000","last_active":"2024-09-16T09:44:55.738519+0000","last_peered":"2024-09-16T09:44:55.738519+0000","last_clean":"2024-09-16T09:44:55.738519+0000","last_became_active":"2024-09-16T09:30:10.693477+0000","last_became_peered":"2024-09-16T09:30:10.693477+0000","last_unstale":"2024-09-16T09:44:55.738519+0000","last_undegraded":"2024-09-16T09:44:55.738519+0000","last_fullsized":"2024-09-16T09:44:55.738519+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:43:24.748025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:43:24.748025+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"285'35","reported_seq":1071,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.737815+0000","last_change":"2024-09-16T09:42:59.545610+0000","last_active":"2024-09-16T09:44:55.737815+0000","last_peered":"2024-09-16T09:44:55.737815+0000","last_clean":"2024-09-16T09:44:55.737815+0000","last_became_active":"2024-09-16T09:25:03.712783+0000","last_became_peered":"2024-09-16T09:25:03.712783+0000","last_unstale":"2024-09-16T09:44:55.737815+0000","last_undegraded":"2024-09-16T09:44:55.737815+0000","last_fullsized":"2024-09-16T09:44:55.737815+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'35","last_scrub_stamp":"2024-09-16T09:42:59.545527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:42:59.545527+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:13.808431+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":56,"num_read_kb":45,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"59'1","reported_seq":1022,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.621085+0000","last_change":"2024-09-16T09:42:24.291119+0000","last_active":"2024-09-16T09:44:55.621085+0000","last_peered":"2024-09-16T09:44:55.621085+0000","last_clean":"2024-09-16T09:44:55.621085+0000","last_became_active":"2024-09-16T09:25:01.708069+0000","last_became_peered":"2024-09-16T09:25:01.708069+0000","last_unstale":"2024-09-16T09:44:55.621085+0000","last_undegraded":"2024-09-16T09:44:55.621085+0000","last_fullsized":"2024-09-16T09:44:55.621085+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:42:24.291047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:42:24.291047+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":986,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.622135+0000","last_change":"2024-09-16T09:44:33.039012+0000","last_active":"2024-09-16T09:44:55.622135+0000","last_peered":"2024-09-16T09:44:55.622135+0000","last_clean":"2024-09-16T09:44:55.622135+0000","last_became_active":"2024-09-16T09:44:24.278290+0000","last_became_peered":"2024-09-16T09:44:24.278290+0000","last_unstale":"2024-09-16T09:44:55.622135+0000","last_undegraded":"2024-09-16T09:44:55.622135+0000","last_fullsized":"2024-09-16T09:44:55.622135+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:33.038961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:44:33.038961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:58.372425+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1000,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.620250+0000","last_change":"2024-09-16T09:44:24.275667+0000","last_active":"2024-09-16T09:44:55.620250+0000","last_peered":"2024-09-16T09:44:55.620250+0000","last_clean":"2024-09-16T09:44:55.620250+0000","last_became_active":"2024-09-16T09:44:24.275407+0000","last_became_peered":"2024-09-16T09:44:24.275407+0000","last_unstale":"2024-09-16T09:44:55.620250+0000","last_undegraded":"2024-09-16T09:44:55.620250+0000","last_fullsized":"2024-09-16T09:44:55.620250+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:42:15.241925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:42:15.241925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.c","version":"73'79","reported_seq":773,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.569906+0000","last_change":"2024-09-16T09:44:51.141526+0000","last_active":"2024-09-16T09:44:55.569906+0000","last_peered":"2024-09-16T09:44:55.569906+0000","last_clean":"2024-09-16T09:44:55.569906+0000","last_became_active":"2024-09-16T09:31:09.273770+0000","last_became_peered":"2024-09-16T09:31:09.273770+0000","last_unstale":"2024-09-16T09:44:55.569906+0000","last_undegraded":"2024-09-16T09:44:55.569906+0000","last_fullsized":"2024-09-16T09:44:55.569906+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:44:51.141451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:44:51.141451+0000","objects_scrubbed":1,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:16.690997+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":327680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"285'18","reported_seq":1085,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.572284+0000","last_change":"2024-09-16T09:42:17.189319+0000","last_active":"2024-09-16T09:44:55.572284+0000","last_peered":"2024-09-16T09:44:55.572284+0000","last_clean":"2024-09-16T09:44:55.572284+0000","last_became_active":"2024-09-16T09:31:09.269047+0000","last_became_peered":"2024-09-16T09:31:09.269047+0000","last_unstale":"2024-09-16T09:44:55.572284+0000","last_undegraded":"2024-09-16T09:44:55.572284+0000","last_fullsized":"2024-09-16T09:44:55.572284+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'18","last_scrub_stamp":"2024-09-16T09:42:17.189230+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:42:17.189230+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":1016,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.621227+0000","last_change":"2024-09-16T09:42:33.321184+0000","last_active":"2024-09-16T09:44:55.621227+0000","last_peered":"2024-09-16T09:44:55.621227+0000","last_clean":"2024-09-16T09:44:55.621227+0000","last_became_active":"2024-09-16T09:25:01.707528+0000","last_became_peered":"2024-09-16T09:25:01.707528+0000","last_unstale":"2024-09-16T09:44:55.621227+0000","last_undegraded":"2024-09-16T09:44:55.621227+0000","last_fullsized":"2024-09-16T09:44:55.621227+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:42:33.321125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:42:33.321125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":680,"reported_epoch":538,"state":"active+clean","last_fresh":"2024-09-16T09:44:54.557787+0000","last_change":"2024-09-16T09:44:24.271417+0000","last_active":"2024-09-16T09:44:54.557787+0000","last_peered":"2024-09-16T09:44:54.557787+0000","last_clean":"2024-09-16T09:44:54.557787+0000","last_became_active":"2024-09-16T09:44:24.270329+0000","last_became_peered":"2024-09-16T09:44:24.270329+0000","last_unstale":"2024-09-16T09:44:54.557787+0000","last_undegraded":"2024-09-16T09:44:54.557787+0000","last_fullsized":"2024-09-16T09:44:54.557787+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:42:12.922136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:42:12.922136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1007,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.569283+0000","last_change":"2024-09-16T09:44:04.116446+0000","last_active":"2024-09-16T09:44:55.569283+0000","last_peered":"2024-09-16T09:44:55.569283+0000","last_clean":"2024-09-16T09:44:55.569283+0000","last_became_active":"2024-09-16T09:25:07.718899+0000","last_became_peered":"2024-09-16T09:25:07.718899+0000","last_unstale":"2024-09-16T09:44:55.569283+0000","last_undegraded":"2024-09-16T09:44:55.569283+0000","last_fullsized":"2024-09-16T09:44:55.569283+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:04.116261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:44:04.116261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:05.947986+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.d","version":"73'79","reported_seq":806,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.620522+0000","last_change":"2024-09-16T09:44:02.449886+0000","last_active":"2024-09-16T09:44:55.620522+0000","last_peered":"2024-09-16T09:44:55.620522+0000","last_clean":"2024-09-16T09:44:55.620522+0000","last_became_active":"2024-09-16T09:30:10.709958+0000","last_became_peered":"2024-09-16T09:30:10.709958+0000","last_unstale":"2024-09-16T09:44:55.620522+0000","last_undegraded":"2024-09-16T09:44:55.620522+0000","last_fullsized":"2024-09-16T09:44:55.620522+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:44:02.449603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:44:02.449603+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:29.882019+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"285'27","reported_seq":1058,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.791550+0000","last_change":"2024-09-16T09:44:21.970091+0000","last_active":"2024-09-16T09:44:55.791550+0000","last_peered":"2024-09-16T09:44:55.791550+0000","last_clean":"2024-09-16T09:44:55.791550+0000","last_became_active":"2024-09-16T09:25:03.715733+0000","last_became_peered":"2024-09-16T09:25:03.715733+0000","last_unstale":"2024-09-16T09:44:55.791550+0000","last_undegraded":"2024-09-16T09:44:55.791550+0000","last_fullsized":"2024-09-16T09:44:55.791550+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'27","last_scrub_stamp":"2024-09-16T09:44:21.970033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:44:21.970033+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:30.951780+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":1020,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.621352+0000","last_change":"2024-09-16T09:42:18.192222+0000","last_active":"2024-09-16T09:44:55.621352+0000","last_peered":"2024-09-16T09:44:55.621352+0000","last_clean":"2024-09-16T09:44:55.621352+0000","last_became_active":"2024-09-16T09:25:01.707372+0000","last_became_peered":"2024-09-16T09:25:01.707372+0000","last_unstale":"2024-09-16T09:44:55.621352+0000","last_undegraded":"2024-09-16T09:44:55.621352+0000","last_fullsized":"2024-09-16T09:44:55.621352+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-16T09:42:18.192142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:42:18.192142+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":32,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"66'8","reported_seq":1265,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.620272+0000","last_change":"2024-09-16T09:42:51.690430+0000","last_active":"2024-09-16T09:44:55.620272+0000","last_peered":"2024-09-16T09:44:55.620272+0000","last_clean":"2024-09-16T09:44:55.620272+0000","last_became_active":"2024-09-16T09:25:05.716189+0000","last_became_peered":"2024-09-16T09:25:05.716189+0000","last_unstale":"2024-09-16T09:44:55.620272+0000","last_undegraded":"2024-09-16T09:44:55.620272+0000","last_fullsized":"2024-09-16T09:44:55.620272+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:42:51.690361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:42:51.690361+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":999,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.791679+0000","last_change":"2024-09-16T09:44:18.939435+0000","last_active":"2024-09-16T09:44:55.791679+0000","last_peered":"2024-09-16T09:44:55.791679+0000","last_clean":"2024-09-16T09:44:55.791679+0000","last_became_active":"2024-09-16T09:25:07.717213+0000","last_became_peered":"2024-09-16T09:25:07.717213+0000","last_unstale":"2024-09-16T09:44:55.791679+0000","last_undegraded":"2024-09-16T09:44:55.791679+0000","last_fullsized":"2024-09-16T09:44:55.791679+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:18.939265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:44:18.939265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:45.253702+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.12","version":"73'79","reported_seq":816,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.791841+0000","last_change":"2024-09-16T09:42:16.729102+0000","last_active":"2024-09-16T09:44:55.791841+0000","last_peered":"2024-09-16T09:44:55.791841+0000","last_clean":"2024-09-16T09:44:55.791841+0000","last_became_active":"2024-09-16T09:30:10.731413+0000","last_became_peered":"2024-09-16T09:30:10.731413+0000","last_unstale":"2024-09-16T09:44:55.791841+0000","last_undegraded":"2024-09-16T09:44:55.791841+0000","last_fullsized":"2024-09-16T09:44:55.791841+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:42:16.728929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:42:16.728929+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:16.883189+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":1025,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.737422+0000","last_change":"2024-09-16T09:44:05.759438+0000","last_active":"2024-09-16T09:44:55.737422+0000","last_peered":"2024-09-16T09:44:55.737422+0000","last_clean":"2024-09-16T09:44:55.737422+0000","last_became_active":"2024-09-16T09:37:53.518354+0000","last_became_peered":"2024-09-16T09:37:53.518354+0000","last_unstale":"2024-09-16T09:44:55.737422+0000","last_undegraded":"2024-09-16T09:44:55.737422+0000","last_fullsized":"2024-09-16T09:44:55.737422+0000","mapping_epoch":382,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":383,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:05.759381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:44:05.759381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:11.154024+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"285'12","reported_seq":795,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.570911+0000","last_change":"2024-09-16T09:44:24.274524+0000","last_active":"2024-09-16T09:44:55.570911+0000","last_peered":"2024-09-16T09:44:55.570911+0000","last_clean":"2024-09-16T09:44:55.570911+0000","last_became_active":"2024-09-16T09:44:24.274107+0000","last_became_peered":"2024-09-16T09:44:24.274107+0000","last_unstale":"2024-09-16T09:44:55.570911+0000","last_undegraded":"2024-09-16T09:44:55.570911+0000","last_fullsized":"2024-09-16T09:44:55.570911+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'12","last_scrub_stamp":"2024-09-16T09:42:52.958841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:42:52.958841+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:37.145040+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":17,"num_read_kb":14,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1018,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.792300+0000","last_change":"2024-09-16T09:44:54.767506+0000","last_active":"2024-09-16T09:44:55.792300+0000","last_peered":"2024-09-16T09:44:55.792300+0000","last_clean":"2024-09-16T09:44:55.792300+0000","last_became_active":"2024-09-16T09:25:05.715616+0000","last_became_peered":"2024-09-16T09:25:05.715616+0000","last_unstale":"2024-09-16T09:44:55.792300+0000","last_undegraded":"2024-09-16T09:44:55.792300+0000","last_fullsized":"2024-09-16T09:44:55.792300+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:54.767444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:44:54.767444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:17.133885+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1004,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.620583+0000","last_change":"2024-09-16T09:44:46.528000+0000","last_active":"2024-09-16T09:44:55.620583+0000","last_peered":"2024-09-16T09:44:55.620583+0000","last_clean":"2024-09-16T09:44:55.620583+0000","last_became_active":"2024-09-16T09:25:07.721059+0000","last_became_peered":"2024-09-16T09:25:07.721059+0000","last_unstale":"2024-09-16T09:44:55.620583+0000","last_undegraded":"2024-09-16T09:44:55.620583+0000","last_fullsized":"2024-09-16T09:44:55.620583+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:46.527936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:44:46.527936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:03.411304+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.13","version":"73'79","reported_seq":909,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.569394+0000","last_change":"2024-09-16T09:44:24.133357+0000","last_active":"2024-09-16T09:44:55.569394+0000","last_peered":"2024-09-16T09:44:55.569394+0000","last_clean":"2024-09-16T09:44:55.569394+0000","last_became_active":"2024-09-16T09:33:30.248497+0000","last_became_peered":"2024-09-16T09:33:30.248497+0000","last_unstale":"2024-09-16T09:44:55.569394+0000","last_undegraded":"2024-09-16T09:44:55.569394+0000","last_fullsized":"2024-09-16T09:44:55.569394+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:44:24.133308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:44:24.133308+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:25.212115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"285'15","reported_seq":1088,"reported_epoch":538,"state":"active+clean","last_fresh":"2024-09-16T09:44:54.559201+0000","last_change":"2024-09-16T09:42:53.892511+0000","last_active":"2024-09-16T09:44:54.559201+0000","last_peered":"2024-09-16T09:44:54.559201+0000","last_clean":"2024-09-16T09:44:54.559201+0000","last_became_active":"2024-09-16T09:31:09.273498+0000","last_became_peered":"2024-09-16T09:31:09.273498+0000","last_unstale":"2024-09-16T09:44:54.559201+0000","last_undegraded":"2024-09-16T09:44:54.559201+0000","last_fullsized":"2024-09-16T09:44:54.559201+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'15","last_scrub_stamp":"2024-09-16T09:42:53.892338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:42:53.892338+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":1024,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.620787+0000","last_change":"2024-09-16T09:43:49.322844+0000","last_active":"2024-09-16T09:44:55.620787+0000","last_peered":"2024-09-16T09:44:55.620787+0000","last_clean":"2024-09-16T09:44:55.620787+0000","last_became_active":"2024-09-16T09:25:01.706491+0000","last_became_peered":"2024-09-16T09:25:01.706491+0000","last_unstale":"2024-09-16T09:44:55.620787+0000","last_undegraded":"2024-09-16T09:44:55.620787+0000","last_fullsized":"2024-09-16T09:44:55.620787+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:43:49.322773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:43:49.322773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":781,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.621044+0000","last_change":"2024-09-16T09:42:08.180799+0000","last_active":"2024-09-16T09:44:55.621044+0000","last_peered":"2024-09-16T09:44:55.621044+0000","last_clean":"2024-09-16T09:44:55.621044+0000","last_became_active":"2024-09-16T09:33:30.248992+0000","last_became_peered":"2024-09-16T09:33:30.248992+0000","last_unstale":"2024-09-16T09:44:55.621044+0000","last_undegraded":"2024-09-16T09:44:55.621044+0000","last_fullsized":"2024-09-16T09:44:55.621044+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:42:08.180735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:42:08.180735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:07.411320+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":988,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.622425+0000","last_change":"2024-09-16T09:44:46.061164+0000","last_active":"2024-09-16T09:44:55.622425+0000","last_peered":"2024-09-16T09:44:55.622425+0000","last_clean":"2024-09-16T09:44:55.622425+0000","last_became_active":"2024-09-16T09:44:24.277612+0000","last_became_peered":"2024-09-16T09:44:24.277612+0000","last_unstale":"2024-09-16T09:44:55.622425+0000","last_undegraded":"2024-09-16T09:44:55.622425+0000","last_fullsized":"2024-09-16T09:44:55.622425+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:46.061101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:44:46.061101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:04.434595+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.10","version":"73'79","reported_seq":816,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.791336+0000","last_change":"2024-09-16T09:44:43.656131+0000","last_active":"2024-09-16T09:44:55.791336+0000","last_peered":"2024-09-16T09:44:55.791336+0000","last_clean":"2024-09-16T09:44:55.791336+0000","last_became_active":"2024-09-16T09:30:10.692770+0000","last_became_peered":"2024-09-16T09:30:10.692770+0000","last_unstale":"2024-09-16T09:44:55.791336+0000","last_undegraded":"2024-09-16T09:44:55.791336+0000","last_fullsized":"2024-09-16T09:44:55.791336+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:44:43.656039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:44:43.656039+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:56.070444+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"285'15","reported_seq":1047,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:57.752956+0000","last_change":"2024-09-16T09:44:57.752956+0000","last_active":"2024-09-16T09:44:57.752956+0000","last_peered":"2024-09-16T09:44:57.752956+0000","last_clean":"2024-09-16T09:44:57.752956+0000","last_became_active":"2024-09-16T09:25:03.717310+0000","last_became_peered":"2024-09-16T09:25:03.717310+0000","last_unstale":"2024-09-16T09:44:57.752956+0000","last_undegraded":"2024-09-16T09:44:57.752956+0000","last_fullsized":"2024-09-16T09:44:57.752956+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'15","last_scrub_stamp":"2024-09-16T09:44:57.752868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:44:57.752868+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:14.909445+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":1047,"reported_epoch":538,"state":"active+clean","last_fresh":"2024-09-16T09:44:54.558631+0000","last_change":"2024-09-16T09:44:35.099469+0000","last_active":"2024-09-16T09:44:54.558631+0000","last_peered":"2024-09-16T09:44:54.558631+0000","last_clean":"2024-09-16T09:44:54.558631+0000","last_became_active":"2024-09-16T09:31:09.272848+0000","last_became_peered":"2024-09-16T09:31:09.272848+0000","last_unstale":"2024-09-16T09:44:54.558631+0000","last_undegraded":"2024-09-16T09:44:54.558631+0000","last_fullsized":"2024-09-16T09:44:54.558631+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:35.099420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:44:35.099420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:01.489513+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"519'13","reported_seq":1106,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.620740+0000","last_change":"2024-09-16T09:44:24.285251+0000","last_active":"2024-09-16T09:44:55.620740+0000","last_peered":"2024-09-16T09:44:55.620740+0000","last_clean":"2024-09-16T09:44:55.620740+0000","last_became_active":"2024-09-16T09:44:24.284843+0000","last_became_peered":"2024-09-16T09:44:24.284843+0000","last_unstale":"2024-09-16T09:44:55.620740+0000","last_undegraded":"2024-09-16T09:44:55.620740+0000","last_fullsized":"2024-09-16T09:44:55.620740+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"187'11","last_scrub_stamp":"2024-09-16T09:42:19.191311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:42:19.191311+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1041,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.573212+0000","last_change":"2024-09-16T09:44:54.304603+0000","last_active":"2024-09-16T09:44:55.573212+0000","last_peered":"2024-09-16T09:44:55.573212+0000","last_clean":"2024-09-16T09:44:55.573212+0000","last_became_active":"2024-09-16T09:44:24.277124+0000","last_became_peered":"2024-09-16T09:44:24.277124+0000","last_unstale":"2024-09-16T09:44:55.573212+0000","last_undegraded":"2024-09-16T09:44:55.573212+0000","last_fullsized":"2024-09-16T09:44:55.573212+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:54.304553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:44:54.304553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:54.689045+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.11","version":"73'79","reported_seq":892,"reported_epoch":538,"state":"active+clean","last_fresh":"2024-09-16T09:44:54.559954+0000","last_change":"2024-09-16T09:42:16.839630+0000","last_active":"2024-09-16T09:44:54.559954+0000","last_peered":"2024-09-16T09:44:54.559954+0000","last_clean":"2024-09-16T09:44:54.559954+0000","last_became_active":"2024-09-16T09:31:09.272208+0000","last_became_peered":"2024-09-16T09:31:09.272208+0000","last_unstale":"2024-09-16T09:44:54.559954+0000","last_undegraded":"2024-09-16T09:44:54.559954+0000","last_fullsized":"2024-09-16T09:44:54.559954+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:42:16.839458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:42:16.839458+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.14","version":"285'20","reported_seq":698,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.620790+0000","last_change":"2024-09-16T09:40:35.027221+0000","last_active":"2024-09-16T09:44:55.620790+0000","last_peered":"2024-09-16T09:44:55.620790+0000","last_clean":"2024-09-16T09:44:55.620790+0000","last_became_active":"2024-09-16T09:33:30.249769+0000","last_became_peered":"2024-09-16T09:33:30.249769+0000","last_unstale":"2024-09-16T09:44:55.620790+0000","last_undegraded":"2024-09-16T09:44:55.620790+0000","last_fullsized":"2024-09-16T09:44:55.620790+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'20","last_scrub_stamp":"2024-09-16T09:40:35.027096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:40:35.027096+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:13.420390+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":1019,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.620554+0000","last_change":"2024-09-16T09:44:24.279638+0000","last_active":"2024-09-16T09:44:55.620554+0000","last_peered":"2024-09-16T09:44:55.620554+0000","last_clean":"2024-09-16T09:44:55.620554+0000","last_became_active":"2024-09-16T09:44:24.279273+0000","last_became_peered":"2024-09-16T09:44:24.279273+0000","last_unstale":"2024-09-16T09:44:55.620554+0000","last_undegraded":"2024-09-16T09:44:55.620554+0000","last_fullsized":"2024-09-16T09:44:55.620554+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:08.005728+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:41:08.005728+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:37.469764+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"66'8","reported_seq":1274,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.792220+0000","last_change":"2024-09-16T09:44:35.809600+0000","last_active":"2024-09-16T09:44:55.792220+0000","last_peered":"2024-09-16T09:44:55.792220+0000","last_clean":"2024-09-16T09:44:55.792220+0000","last_became_active":"2024-09-16T09:25:05.713833+0000","last_became_peered":"2024-09-16T09:25:05.713833+0000","last_unstale":"2024-09-16T09:44:55.792220+0000","last_undegraded":"2024-09-16T09:44:55.792220+0000","last_fullsized":"2024-09-16T09:44:55.792220+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:44:35.809538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:44:35.809538+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:38.475407+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1035,"reported_epoch":538,"state":"active+clean","last_fresh":"2024-09-16T09:44:54.558787+0000","last_change":"2024-09-16T09:42:32.913873+0000","last_active":"2024-09-16T09:44:54.558787+0000","last_peered":"2024-09-16T09:44:54.558787+0000","last_clean":"2024-09-16T09:44:54.558787+0000","last_became_active":"2024-09-16T09:31:09.275168+0000","last_became_peered":"2024-09-16T09:31:09.275168+0000","last_unstale":"2024-09-16T09:44:54.558787+0000","last_undegraded":"2024-09-16T09:44:54.558787+0000","last_fullsized":"2024-09-16T09:44:54.558787+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:42:32.913795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:42:32.913795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"285'19","reported_seq":1048,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.572340+0000","last_change":"2024-09-16T09:42:20.151407+0000","last_active":"2024-09-16T09:44:55.572340+0000","last_peered":"2024-09-16T09:44:55.572340+0000","last_clean":"2024-09-16T09:44:55.572340+0000","last_became_active":"2024-09-16T09:26:33.291265+0000","last_became_peered":"2024-09-16T09:26:33.291265+0000","last_unstale":"2024-09-16T09:44:55.572340+0000","last_undegraded":"2024-09-16T09:44:55.572340+0000","last_fullsized":"2024-09-16T09:44:55.572340+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'19","last_scrub_stamp":"2024-09-16T09:42:20.151313+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:42:20.151313+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":1034,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.572950+0000","last_change":"2024-09-16T09:44:23.334509+0000","last_active":"2024-09-16T09:44:55.572950+0000","last_peered":"2024-09-16T09:44:55.572950+0000","last_clean":"2024-09-16T09:44:55.572950+0000","last_became_active":"2024-09-16T09:25:01.705158+0000","last_became_peered":"2024-09-16T09:25:01.705158+0000","last_unstale":"2024-09-16T09:44:55.572950+0000","last_undegraded":"2024-09-16T09:44:55.572950+0000","last_fullsized":"2024-09-16T09:44:55.572950+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:23.334458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:44:23.334458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:34.371408+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":954,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.791696+0000","last_change":"2024-09-16T09:44:45.660171+0000","last_active":"2024-09-16T09:44:55.791696+0000","last_peered":"2024-09-16T09:44:55.791696+0000","last_clean":"2024-09-16T09:44:55.791696+0000","last_became_active":"2024-09-16T09:37:41.515278+0000","last_became_peered":"2024-09-16T09:37:41.515278+0000","last_unstale":"2024-09-16T09:44:55.791696+0000","last_undegraded":"2024-09-16T09:44:55.791696+0000","last_fullsized":"2024-09-16T09:44:55.791696+0000","mapping_epoch":374,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":375,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:45.660111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:44:45.660111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:48.190760+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":859,"reported_epoch":538,"state":"active+clean","last_fresh":"2024-09-16T09:44:54.559555+0000","last_change":"2024-09-16T09:44:47.085511+0000","last_active":"2024-09-16T09:44:54.559555+0000","last_peered":"2024-09-16T09:44:54.559555+0000","last_clean":"2024-09-16T09:44:54.559555+0000","last_became_active":"2024-09-16T09:31:09.276042+0000","last_became_peered":"2024-09-16T09:31:09.276042+0000","last_unstale":"2024-09-16T09:44:54.559555+0000","last_undegraded":"2024-09-16T09:44:54.559555+0000","last_fullsized":"2024-09-16T09:44:54.559555+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:47.085465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:44:47.085465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:58.894807+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"285'15","reported_seq":748,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.737977+0000","last_change":"2024-09-16T09:44:28.650617+0000","last_active":"2024-09-16T09:44:55.737977+0000","last_peered":"2024-09-16T09:44:55.737977+0000","last_clean":"2024-09-16T09:44:55.737977+0000","last_became_active":"2024-09-16T09:44:24.270349+0000","last_became_peered":"2024-09-16T09:44:24.270349+0000","last_unstale":"2024-09-16T09:44:55.737977+0000","last_undegraded":"2024-09-16T09:44:55.737977+0000","last_fullsized":"2024-09-16T09:44:55.737977+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'15","last_scrub_stamp":"2024-09-16T09:44:28.650561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:44:28.650561+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:38.659197+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,2],"acting":[6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":1016,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.620988+0000","last_change":"2024-09-16T09:43:47.307622+0000","last_active":"2024-09-16T09:44:55.620988+0000","last_peered":"2024-09-16T09:44:55.620988+0000","last_clean":"2024-09-16T09:44:55.620988+0000","last_became_active":"2024-09-16T09:25:01.706762+0000","last_became_peered":"2024-09-16T09:25:01.706762+0000","last_unstale":"2024-09-16T09:44:55.620988+0000","last_undegraded":"2024-09-16T09:44:55.620988+0000","last_fullsized":"2024-09-16T09:44:55.620988+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:43:47.307538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:43:47.307538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:02.160911+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":844,"reported_epoch":538,"state":"active+clean","last_fresh":"2024-09-16T09:44:54.559674+0000","last_change":"2024-09-16T09:41:36.130305+0000","last_active":"2024-09-16T09:44:54.559674+0000","last_peered":"2024-09-16T09:44:54.559674+0000","last_clean":"2024-09-16T09:44:54.559674+0000","last_became_active":"2024-09-16T09:28:52.982865+0000","last_became_peered":"2024-09-16T09:28:52.982865+0000","last_unstale":"2024-09-16T09:44:54.559674+0000","last_undegraded":"2024-09-16T09:44:54.559674+0000","last_fullsized":"2024-09-16T09:44:54.559674+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:36.130240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:41:36.130240+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":989,"reported_epoch":538,"state":"active+clean","last_fresh":"2024-09-16T09:44:54.557660+0000","last_change":"2024-09-16T09:34:50.779241+0000","last_active":"2024-09-16T09:44:54.557660+0000","last_peered":"2024-09-16T09:44:54.557660+0000","last_clean":"2024-09-16T09:44:54.557660+0000","last_became_active":"2024-09-16T09:25:07.715301+0000","last_became_peered":"2024-09-16T09:25:07.715301+0000","last_unstale":"2024-09-16T09:44:54.557660+0000","last_undegraded":"2024-09-16T09:44:54.557660+0000","last_fullsized":"2024-09-16T09:44:54.557660+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:34:50.779164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:34:50.779164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:08.818483+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.14","version":"73'79","reported_seq":730,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.622277+0000","last_change":"2024-09-16T09:44:24.960676+0000","last_active":"2024-09-16T09:44:55.622277+0000","last_peered":"2024-09-16T09:44:55.622277+0000","last_clean":"2024-09-16T09:44:55.622277+0000","last_became_active":"2024-09-16T09:44:24.285886+0000","last_became_peered":"2024-09-16T09:44:24.285886+0000","last_unstale":"2024-09-16T09:44:55.622277+0000","last_undegraded":"2024-09-16T09:44:55.622277+0000","last_fullsized":"2024-09-16T09:44:55.622277+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:44:24.960619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:44:24.960619+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:44.429977+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"285'19","reported_seq":895,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.620486+0000","last_change":"2024-09-16T09:42:57.678412+0000","last_active":"2024-09-16T09:44:55.620486+0000","last_peered":"2024-09-16T09:44:55.620486+0000","last_clean":"2024-09-16T09:44:55.620486+0000","last_became_active":"2024-09-16T09:28:52.971132+0000","last_became_peered":"2024-09-16T09:28:52.971132+0000","last_unstale":"2024-09-16T09:44:55.620486+0000","last_undegraded":"2024-09-16T09:44:55.620486+0000","last_fullsized":"2024-09-16T09:44:55.620486+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'19","last_scrub_stamp":"2024-09-16T09:42:57.678331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:42:57.678331+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":1026,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.792402+0000","last_change":"2024-09-16T09:44:03.793102+0000","last_active":"2024-09-16T09:44:55.792402+0000","last_peered":"2024-09-16T09:44:55.792402+0000","last_clean":"2024-09-16T09:44:55.792402+0000","last_became_active":"2024-09-16T09:25:01.700733+0000","last_became_peered":"2024-09-16T09:25:01.700733+0000","last_unstale":"2024-09-16T09:44:55.792402+0000","last_undegraded":"2024-09-16T09:44:55.792402+0000","last_fullsized":"2024-09-16T09:44:55.792402+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:03.793040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:44:03.793040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:16.224422+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1008,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.619918+0000","last_change":"2024-09-16T09:40:39.969447+0000","last_active":"2024-09-16T09:44:55.619918+0000","last_peered":"2024-09-16T09:44:55.619918+0000","last_clean":"2024-09-16T09:44:55.619918+0000","last_became_active":"2024-09-16T09:25:05.713479+0000","last_became_peered":"2024-09-16T09:25:05.713479+0000","last_unstale":"2024-09-16T09:44:55.619918+0000","last_undegraded":"2024-09-16T09:44:55.619918+0000","last_fullsized":"2024-09-16T09:44:55.619918+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:40:39.969383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:40:39.969383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:38.496665+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":867,"reported_epoch":538,"state":"active+clean","last_fresh":"2024-09-16T09:44:54.560204+0000","last_change":"2024-09-16T09:41:34.178100+0000","last_active":"2024-09-16T09:44:54.560204+0000","last_peered":"2024-09-16T09:44:54.560204+0000","last_clean":"2024-09-16T09:44:54.560204+0000","last_became_active":"2024-09-16T09:31:09.274797+0000","last_became_peered":"2024-09-16T09:31:09.274797+0000","last_unstale":"2024-09-16T09:44:54.560204+0000","last_undegraded":"2024-09-16T09:44:54.560204+0000","last_fullsized":"2024-09-16T09:44:54.560204+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:41:34.178020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:41:34.178020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"285'8","reported_seq":788,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.568711+0000","last_change":"2024-09-16T09:44:47.186136+0000","last_active":"2024-09-16T09:44:55.568711+0000","last_peered":"2024-09-16T09:44:55.568711+0000","last_clean":"2024-09-16T09:44:55.568711+0000","last_became_active":"2024-09-16T09:31:09.273682+0000","last_became_peered":"2024-09-16T09:31:09.273682+0000","last_unstale":"2024-09-16T09:44:55.568711+0000","last_undegraded":"2024-09-16T09:44:55.568711+0000","last_fullsized":"2024-09-16T09:44:55.568711+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'8","last_scrub_stamp":"2024-09-16T09:44:47.186078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:44:47.186078+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:11.853660+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":9,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":1047,"reported_epoch":538,"state":"active+clean","last_fresh":"2024-09-16T09:44:54.558627+0000","last_change":"2024-09-16T09:43:21.684366+0000","last_active":"2024-09-16T09:44:54.558627+0000","last_peered":"2024-09-16T09:44:54.558627+0000","last_clean":"2024-09-16T09:44:54.558627+0000","last_became_active":"2024-09-16T09:31:09.274719+0000","last_became_peered":"2024-09-16T09:31:09.274719+0000","last_unstale":"2024-09-16T09:44:54.558627+0000","last_undegraded":"2024-09-16T09:44:54.558627+0000","last_fullsized":"2024-09-16T09:44:54.558627+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:43:21.684289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:43:21.684289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1027,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.737612+0000","last_change":"2024-09-16T09:44:49.775416+0000","last_active":"2024-09-16T09:44:55.737612+0000","last_peered":"2024-09-16T09:44:55.737612+0000","last_clean":"2024-09-16T09:44:55.737612+0000","last_became_active":"2024-09-16T09:25:05.712545+0000","last_became_peered":"2024-09-16T09:25:05.712545+0000","last_unstale":"2024-09-16T09:44:55.737612+0000","last_undegraded":"2024-09-16T09:44:55.737612+0000","last_fullsized":"2024-09-16T09:44:55.737612+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:49.775356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:44:49.775356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:57.470848+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1023,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.620148+0000","last_change":"2024-09-16T09:44:51.632486+0000","last_active":"2024-09-16T09:44:55.620148+0000","last_peered":"2024-09-16T09:44:55.620148+0000","last_clean":"2024-09-16T09:44:55.620148+0000","last_became_active":"2024-09-16T09:44:24.276311+0000","last_became_peered":"2024-09-16T09:44:24.276311+0000","last_unstale":"2024-09-16T09:44:55.620148+0000","last_undegraded":"2024-09-16T09:44:55.620148+0000","last_fullsized":"2024-09-16T09:44:55.620148+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:51.632429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:44:51.632429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:19.007714+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":650,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.792217+0000","last_change":"2024-09-16T09:44:50.722288+0000","last_active":"2024-09-16T09:44:55.792217+0000","last_peered":"2024-09-16T09:44:55.792217+0000","last_clean":"2024-09-16T09:44:55.792217+0000","last_became_active":"2024-09-16T09:33:30.248036+0000","last_became_peered":"2024-09-16T09:33:30.248036+0000","last_unstale":"2024-09-16T09:44:55.792217+0000","last_undegraded":"2024-09-16T09:44:55.792217+0000","last_fullsized":"2024-09-16T09:44:55.792217+0000","mapping_epoch":233,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":234,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:50.722209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:44:50.722209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:53.928329+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":866,"reported_epoch":538,"state":"active+clean","last_fresh":"2024-09-16T09:44:54.558201+0000","last_change":"2024-09-16T09:42:30.898163+0000","last_active":"2024-09-16T09:44:54.558201+0000","last_peered":"2024-09-16T09:44:54.558201+0000","last_clean":"2024-09-16T09:44:54.558201+0000","last_became_active":"2024-09-16T09:31:09.271662+0000","last_became_peered":"2024-09-16T09:31:09.271662+0000","last_unstale":"2024-09-16T09:44:54.558201+0000","last_undegraded":"2024-09-16T09:44:54.558201+0000","last_fullsized":"2024-09-16T09:44:54.558201+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:42:30.898094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:42:30.898094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"285'19","reported_seq":1055,"reported_epoch":539,"state":"active+clean","last_fresh":"2024-09-16T09:44:55.738584+0000","last_change":"2024-09-16T09:44:23.700110+0000","last_active":"2024-09-16T09:44:55.738584+0000","last_peered":"2024-09-16T09:44:55.738584+0000","last_clean":"2024-09-16T09:44:55.738584+0000","last_became_active":"2024-09-16T09:25:03.707769+0000","last_became_peered":"2024-09-16T09:25:03.707769+0000","last_unstale":"2024-09-16T09:44:55.738584+0000","last_undegraded":"2024-09-16T09:44:55.738584+0000","last_fullsized":"2024-09-16T09:44:55.738584+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'19","last_scrub_stamp":"2024-09-16T09:44:23.700027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:44:23.700027+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:26.961037+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1021,"reported_epoch":538,"state":"active+clean","last_fresh":"2024-09-16T09:44:54.558933+0000","last_change":"2024-09-16T09:44:36.073614+0000","last_active":"2024-09-16T09:44:54.558933+0000","last_peered":"2024-09-16T09:44:54.558933+0000","last_clean":"2024-09-16T09:44:54.558933+0000","last_became_active":"2024-09-16T09:44:24.279906+0000","last_became_peered":"2024-09-16T09:44:24.279906+0000","last_unstale":"2024-09-16T09:44:54.558933+0000","last_undegraded":"2024-09-16T09:44:54.558933+0000","last_fullsized":"2024-09-16T09:44:54.558933+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:36.073560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:44:36.073560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:44.457091+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":1115,"num_read_kb":872,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":3686,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":683,"ondisk_log_size":683,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1986,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1188,"num_read_kb":1183,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":21,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":327680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1659,"ondisk_log_size":1659,"up":63,"acting":63,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266932,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":848412,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":847486,"kb_avail":92896740,"statfs":{"total":95995035648,"available":95126261760,"internally_reserved":0,"allocated":925696,"data_stored":554264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":867826122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495865,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":718216,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":716990,"kb_avail":93026936,"statfs":{"total":95995035648,"available":95259582464,"internally_reserved":0,"allocated":1224704,"data_stored":880736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":734198214},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724800,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":858392,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":857214,"kb_avail":92886760,"statfs":{"total":95995035648,"available":95116042240,"internally_reserved":0,"allocated":1179648,"data_stored":816433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":877787590},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953733,"num_pgs":65,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":735964,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":735038,"kb_avail":93009188,"statfs":{"total":95995035648,"available":95241408512,"internally_reserved":0,"allocated":925696,"data_stored":554218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":752679366},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149963,"num_pgs":64,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":917028,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":915518,"kb_avail":92828124,"statfs":{"total":95995035648,"available":95055998976,"internally_reserved":0,"allocated":1523712,"data_stored":1144762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":937490887},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":516,"seq":2216203124744,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":906944,"kb_used_data":825192,"kb_used_omap":4,"kb_used_meta":81723,"kb_avail":92838208,"statfs":{"total":95995035648,"available":95066324992,"internally_reserved":0,"allocated":844996608,"data_stored":844656992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4445,"internal_metadata":83685027},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575127,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1024768,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":1023806,"kb_avail":92720384,"statfs":{"total":95995035648,"available":94945673216,"internally_reserved":0,"allocated":954368,"data_stored":554198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":1048377797},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738652,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":866752,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":865534,"kb_avail":92878400,"statfs":{"total":95995035648,"available":95107481600,"internally_reserved":0,"allocated":1224704,"data_stored":866056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":886307269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":1788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T09:45:00.617 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2024-09-16T09:45:00.618 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd pool set .mgr pg_num 31 2024-09-16T09:45:00.874 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:45:00.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:00 smithi084 ceph-mon[32629]: 4.7 scrub starts 2024-09-16T09:45:00.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:00 smithi084 ceph-mon[32629]: 4.7 scrub ok 2024-09-16T09:45:00.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:00 smithi070 ceph-mon[30388]: 4.7 scrub starts 2024-09-16T09:45:00.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:00 smithi070 ceph-mon[30388]: 4.7 scrub ok 2024-09-16T09:45:00.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:00 smithi070 ceph-mon[40691]: 4.7 scrub starts 2024-09-16T09:45:00.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:00 smithi070 ceph-mon[40691]: 4.7 scrub ok 2024-09-16T09:45:01.590 INFO:teuthology.orchestra.run.smithi070.stderr:set pool 1 pg_num to 31 2024-09-16T09:45:01.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:01 smithi084 ceph-mon[32629]: from='client.26597 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:45:01.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:01 smithi084 ceph-mon[32629]: pgmap v1054: 152 pgs: 152 active+clean; 583 KiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:45:01.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:01 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/2822664393' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "31"} : dispatch 2024-09-16T09:45:01.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:01 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "31"} : dispatch 2024-09-16T09:45:01.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:01 smithi070 ceph-mon[30388]: from='client.26597 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:45:01.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:01 smithi070 ceph-mon[30388]: pgmap v1054: 152 pgs: 152 active+clean; 583 KiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:45:01.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:01 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2822664393' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "31"} : dispatch 2024-09-16T09:45:01.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:01 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "31"} : dispatch 2024-09-16T09:45:01.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:01 smithi070 ceph-mon[40691]: from='client.26597 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:45:01.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:01 smithi070 ceph-mon[40691]: pgmap v1054: 152 pgs: 152 active+clean; 583 KiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:45:01.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:01 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/2822664393' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "31"} : dispatch 2024-09-16T09:45:01.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:01 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "31"} : dispatch 2024-09-16T09:45:02.242 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:45:01 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:45:01] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:45:02.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:02 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "31"}]': finished 2024-09-16T09:45:02.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:02 smithi084 ceph-mon[32629]: osdmap e541: 8 total, 8 up, 8 in 2024-09-16T09:45:02.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:02 smithi084 ceph-mon[32629]: 5.4 scrub starts 2024-09-16T09:45:02.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:02 smithi084 ceph-mon[32629]: 5.4 scrub ok 2024-09-16T09:45:02.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:02 smithi084 ceph-mon[32629]: 1.12 scrub starts 2024-09-16T09:45:02.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:02 smithi084 ceph-mon[32629]: 1.12 scrub ok 2024-09-16T09:45:02.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:02 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "31"} : dispatch 2024-09-16T09:45:02.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:02 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "31"} : dispatch 2024-09-16T09:45:02.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:02 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "31"}]': finished 2024-09-16T09:45:02.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:02 smithi070 ceph-mon[30388]: osdmap e541: 8 total, 8 up, 8 in 2024-09-16T09:45:02.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:02 smithi070 ceph-mon[30388]: 5.4 scrub starts 2024-09-16T09:45:02.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:02 smithi070 ceph-mon[30388]: 5.4 scrub ok 2024-09-16T09:45:02.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:02 smithi070 ceph-mon[30388]: 1.12 scrub starts 2024-09-16T09:45:02.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:02 smithi070 ceph-mon[30388]: 1.12 scrub ok 2024-09-16T09:45:02.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:02 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "31"} : dispatch 2024-09-16T09:45:02.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:02 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "31"} : dispatch 2024-09-16T09:45:02.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:02 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "31"}]': finished 2024-09-16T09:45:02.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:02 smithi070 ceph-mon[40691]: osdmap e541: 8 total, 8 up, 8 in 2024-09-16T09:45:02.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:02 smithi070 ceph-mon[40691]: 5.4 scrub starts 2024-09-16T09:45:02.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:02 smithi070 ceph-mon[40691]: 5.4 scrub ok 2024-09-16T09:45:02.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:02 smithi070 ceph-mon[40691]: 1.12 scrub starts 2024-09-16T09:45:02.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:02 smithi070 ceph-mon[40691]: 1.12 scrub ok 2024-09-16T09:45:02.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:02 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "31"} : dispatch 2024-09-16T09:45:02.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:02 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "31"} : dispatch 2024-09-16T09:45:03.910 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:45:03 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:45:03.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:03 smithi084 ceph-mon[32629]: pgmap v1056: 152 pgs: 152 active+clean; 583 KiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 804 B/s rd, 0 op/s 2024-09-16T09:45:03.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:03 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "31"}]': finished 2024-09-16T09:45:03.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:03 smithi084 ceph-mon[32629]: osdmap e542: 8 total, 8 up, 8 in 2024-09-16T09:45:03.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:03 smithi084 ceph-mon[32629]: 6.8 scrub starts 2024-09-16T09:45:03.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:03 smithi084 ceph-mon[32629]: 6.8 scrub ok 2024-09-16T09:45:03.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:03 smithi084 ceph-mon[32629]: osdmap e543: 8 total, 8 up, 8 in 2024-09-16T09:45:03.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:03 smithi070 ceph-mon[40691]: pgmap v1056: 152 pgs: 152 active+clean; 583 KiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 804 B/s rd, 0 op/s 2024-09-16T09:45:03.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:03 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "31"}]': finished 2024-09-16T09:45:03.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:03 smithi070 ceph-mon[40691]: osdmap e542: 8 total, 8 up, 8 in 2024-09-16T09:45:03.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:03 smithi070 ceph-mon[40691]: 6.8 scrub starts 2024-09-16T09:45:03.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:03 smithi070 ceph-mon[40691]: 6.8 scrub ok 2024-09-16T09:45:03.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:03 smithi070 ceph-mon[40691]: osdmap e543: 8 total, 8 up, 8 in 2024-09-16T09:45:03.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:03 smithi070 ceph-mon[30388]: pgmap v1056: 152 pgs: 152 active+clean; 583 KiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 804 B/s rd, 0 op/s 2024-09-16T09:45:03.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:03 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "31"}]': finished 2024-09-16T09:45:03.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:03 smithi070 ceph-mon[30388]: osdmap e542: 8 total, 8 up, 8 in 2024-09-16T09:45:03.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:03 smithi070 ceph-mon[30388]: 6.8 scrub starts 2024-09-16T09:45:03.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:03 smithi070 ceph-mon[30388]: 6.8 scrub ok 2024-09-16T09:45:03.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:03 smithi070 ceph-mon[30388]: osdmap e543: 8 total, 8 up, 8 in 2024-09-16T09:45:04.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:04 smithi084 ceph-mon[32629]: 1.6 scrub starts 2024-09-16T09:45:04.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:04 smithi084 ceph-mon[32629]: 1.6 scrub ok 2024-09-16T09:45:04.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:04 smithi084 ceph-mon[32629]: 1.d scrub starts 2024-09-16T09:45:04.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:04 smithi084 ceph-mon[32629]: 1.d scrub ok 2024-09-16T09:45:04.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:04 smithi084 ceph-mon[32629]: 1.a scrub starts 2024-09-16T09:45:04.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:04 smithi084 ceph-mon[32629]: 1.a scrub ok 2024-09-16T09:45:04.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:04 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "31"} : dispatch 2024-09-16T09:45:04.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:04 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "31"} : dispatch 2024-09-16T09:45:04.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:04 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:45:04.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:04 smithi084 ceph-mon[32629]: 1.5 scrub starts 2024-09-16T09:45:04.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:04 smithi084 ceph-mon[32629]: 1.5 scrub ok 2024-09-16T09:45:04.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:04 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "31"}]': finished 2024-09-16T09:45:04.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:04 smithi084 ceph-mon[32629]: osdmap e544: 8 total, 8 up, 8 in 2024-09-16T09:45:04.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:04 smithi070 ceph-mon[30388]: 1.6 scrub starts 2024-09-16T09:45:04.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:04 smithi070 ceph-mon[30388]: 1.6 scrub ok 2024-09-16T09:45:04.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:04 smithi070 ceph-mon[30388]: 1.d scrub starts 2024-09-16T09:45:04.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:04 smithi070 ceph-mon[30388]: 1.d scrub ok 2024-09-16T09:45:04.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:04 smithi070 ceph-mon[30388]: 1.a scrub starts 2024-09-16T09:45:04.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:04 smithi070 ceph-mon[30388]: 1.a scrub ok 2024-09-16T09:45:04.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:04 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "31"} : dispatch 2024-09-16T09:45:04.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:04 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "31"} : dispatch 2024-09-16T09:45:04.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:04 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:45:04.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:04 smithi070 ceph-mon[30388]: 1.5 scrub starts 2024-09-16T09:45:04.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:04 smithi070 ceph-mon[30388]: 1.5 scrub ok 2024-09-16T09:45:04.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:04 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "31"}]': finished 2024-09-16T09:45:04.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:04 smithi070 ceph-mon[30388]: osdmap e544: 8 total, 8 up, 8 in 2024-09-16T09:45:04.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:04 smithi070 ceph-mon[40691]: 1.6 scrub starts 2024-09-16T09:45:04.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:04 smithi070 ceph-mon[40691]: 1.6 scrub ok 2024-09-16T09:45:04.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:04 smithi070 ceph-mon[40691]: 1.d scrub starts 2024-09-16T09:45:04.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:04 smithi070 ceph-mon[40691]: 1.d scrub ok 2024-09-16T09:45:04.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:04 smithi070 ceph-mon[40691]: 1.a scrub starts 2024-09-16T09:45:04.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:04 smithi070 ceph-mon[40691]: 1.a scrub ok 2024-09-16T09:45:04.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:04 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "31"} : dispatch 2024-09-16T09:45:04.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:04 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "31"} : dispatch 2024-09-16T09:45:04.923 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:04 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:45:04.923 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:04 smithi070 ceph-mon[40691]: 1.5 scrub starts 2024-09-16T09:45:04.923 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:04 smithi070 ceph-mon[40691]: 1.5 scrub ok 2024-09-16T09:45:04.923 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:04 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "31"}]': finished 2024-09-16T09:45:04.923 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:04 smithi070 ceph-mon[40691]: osdmap e544: 8 total, 8 up, 8 in 2024-09-16T09:45:05.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:05 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:45:05.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:05 smithi084 ceph-mon[32629]: pgmap v1059: 162 pgs: 8 peering, 9 unknown, 145 active+clean; 583 KiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-16T09:45:05.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:05 smithi084 ceph-mon[32629]: 1.1c scrub starts 2024-09-16T09:45:05.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:05 smithi084 ceph-mon[32629]: 1.1c scrub ok 2024-09-16T09:45:05.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:05 smithi084 ceph-mon[32629]: 1.b scrub starts 2024-09-16T09:45:05.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:05 smithi084 ceph-mon[32629]: 1.b scrub ok 2024-09-16T09:45:05.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:05 smithi084 ceph-mon[32629]: 1.7 scrub starts 2024-09-16T09:45:05.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:05 smithi084 ceph-mon[32629]: 1.7 scrub ok 2024-09-16T09:45:05.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:05 smithi084 ceph-mon[32629]: 1.9 scrub starts 2024-09-16T09:45:05.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:05 smithi084 ceph-mon[32629]: 1.9 scrub ok 2024-09-16T09:45:05.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:05 smithi084 ceph-mon[32629]: 1.8 scrub starts 2024-09-16T09:45:05.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:05 smithi084 ceph-mon[32629]: 1.8 scrub ok 2024-09-16T09:45:05.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:05 smithi084 ceph-mon[32629]: osdmap e545: 8 total, 8 up, 8 in 2024-09-16T09:45:05.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:05 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:45:05.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:05 smithi070 ceph-mon[30388]: pgmap v1059: 162 pgs: 8 peering, 9 unknown, 145 active+clean; 583 KiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-16T09:45:05.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:05 smithi070 ceph-mon[30388]: 1.1c scrub starts 2024-09-16T09:45:05.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:05 smithi070 ceph-mon[30388]: 1.1c scrub ok 2024-09-16T09:45:05.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:05 smithi070 ceph-mon[30388]: 1.b scrub starts 2024-09-16T09:45:05.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:05 smithi070 ceph-mon[30388]: 1.b scrub ok 2024-09-16T09:45:05.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:05 smithi070 ceph-mon[30388]: 1.7 scrub starts 2024-09-16T09:45:05.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:05 smithi070 ceph-mon[30388]: 1.7 scrub ok 2024-09-16T09:45:05.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:05 smithi070 ceph-mon[30388]: 1.9 scrub starts 2024-09-16T09:45:05.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:05 smithi070 ceph-mon[30388]: 1.9 scrub ok 2024-09-16T09:45:05.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:05 smithi070 ceph-mon[30388]: 1.8 scrub starts 2024-09-16T09:45:05.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:05 smithi070 ceph-mon[30388]: 1.8 scrub ok 2024-09-16T09:45:05.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:05 smithi070 ceph-mon[30388]: osdmap e545: 8 total, 8 up, 8 in 2024-09-16T09:45:05.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:05 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:45:05.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:05 smithi070 ceph-mon[40691]: pgmap v1059: 162 pgs: 8 peering, 9 unknown, 145 active+clean; 583 KiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-16T09:45:05.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:05 smithi070 ceph-mon[40691]: 1.1c scrub starts 2024-09-16T09:45:05.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:05 smithi070 ceph-mon[40691]: 1.1c scrub ok 2024-09-16T09:45:05.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:05 smithi070 ceph-mon[40691]: 1.b scrub starts 2024-09-16T09:45:05.923 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:05 smithi070 ceph-mon[40691]: 1.b scrub ok 2024-09-16T09:45:05.923 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:05 smithi070 ceph-mon[40691]: 1.7 scrub starts 2024-09-16T09:45:05.923 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:05 smithi070 ceph-mon[40691]: 1.7 scrub ok 2024-09-16T09:45:05.923 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:05 smithi070 ceph-mon[40691]: 1.9 scrub starts 2024-09-16T09:45:05.923 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:05 smithi070 ceph-mon[40691]: 1.9 scrub ok 2024-09-16T09:45:05.923 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:05 smithi070 ceph-mon[40691]: 1.8 scrub starts 2024-09-16T09:45:05.923 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:05 smithi070 ceph-mon[40691]: 1.8 scrub ok 2024-09-16T09:45:05.923 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:05 smithi070 ceph-mon[40691]: osdmap e545: 8 total, 8 up, 8 in 2024-09-16T09:45:06.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:06 smithi084 ceph-mon[32629]: 1.c scrub starts 2024-09-16T09:45:06.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:06 smithi084 ceph-mon[32629]: 1.c scrub ok 2024-09-16T09:45:06.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:06 smithi084 ceph-mon[32629]: 1.e scrub starts 2024-09-16T09:45:06.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:06 smithi084 ceph-mon[32629]: 1.e scrub ok 2024-09-16T09:45:06.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:06 smithi084 ceph-mon[32629]: 1.1c scrub starts 2024-09-16T09:45:06.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:06 smithi084 ceph-mon[32629]: 1.1c scrub ok 2024-09-16T09:45:06.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:06 smithi084 ceph-mon[32629]: 1.17 scrub starts 2024-09-16T09:45:06.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:06 smithi084 ceph-mon[32629]: 1.17 scrub ok 2024-09-16T09:45:06.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:06 smithi070 ceph-mon[30388]: 1.c scrub starts 2024-09-16T09:45:06.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:06 smithi070 ceph-mon[30388]: 1.c scrub ok 2024-09-16T09:45:06.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:06 smithi070 ceph-mon[30388]: 1.e scrub starts 2024-09-16T09:45:06.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:06 smithi070 ceph-mon[30388]: 1.e scrub ok 2024-09-16T09:45:06.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:06 smithi070 ceph-mon[30388]: 1.1c scrub starts 2024-09-16T09:45:06.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:06 smithi070 ceph-mon[30388]: 1.1c scrub ok 2024-09-16T09:45:06.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:06 smithi070 ceph-mon[30388]: 1.17 scrub starts 2024-09-16T09:45:06.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:06 smithi070 ceph-mon[30388]: 1.17 scrub ok 2024-09-16T09:45:06.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:06 smithi070 ceph-mon[40691]: 1.c scrub starts 2024-09-16T09:45:06.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:06 smithi070 ceph-mon[40691]: 1.c scrub ok 2024-09-16T09:45:06.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:06 smithi070 ceph-mon[40691]: 1.e scrub starts 2024-09-16T09:45:06.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:06 smithi070 ceph-mon[40691]: 1.e scrub ok 2024-09-16T09:45:06.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:06 smithi070 ceph-mon[40691]: 1.1c scrub starts 2024-09-16T09:45:06.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:06 smithi070 ceph-mon[40691]: 1.1c scrub ok 2024-09-16T09:45:06.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:06 smithi070 ceph-mon[40691]: 1.17 scrub starts 2024-09-16T09:45:06.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:06 smithi070 ceph-mon[40691]: 1.17 scrub ok 2024-09-16T09:45:07.402 INFO:tasks.thrashosds.thrasher:fixing pg num pool .mgr 2024-09-16T09:45:07.402 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd pool set .mgr pgp_num 31 2024-09-16T09:45:07.666 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:45:07.786 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:07 smithi070 ceph-mon[40691]: pgmap v1062: 162 pgs: 10 peering, 7 unknown, 145 active+clean; 622 KiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 5.2 KiB/s rd, 28 KiB/s wr, 5 op/s; 9.8 KiB/s, 0 objects/s recovering 2024-09-16T09:45:07.787 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:07 smithi070 ceph-mon[40691]: 1.1e scrub starts 2024-09-16T09:45:07.787 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:07 smithi070 ceph-mon[40691]: 1.1e scrub ok 2024-09-16T09:45:07.787 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:07 smithi070 ceph-mon[40691]: 1.1d scrub starts 2024-09-16T09:45:07.787 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:07 smithi070 ceph-mon[40691]: 1.1d scrub ok 2024-09-16T09:45:07.787 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:07 smithi070 ceph-mon[40691]: 1.16 scrub starts 2024-09-16T09:45:07.787 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:07 smithi070 ceph-mon[40691]: 1.16 scrub ok 2024-09-16T09:45:07.787 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:07 smithi070 ceph-mon[40691]: Health check failed: Reduced data availability: 1 pg inactive, 2 pgs peering (PG_AVAILABILITY) 2024-09-16T09:45:07.787 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:07 smithi070 ceph-mon[40691]: 1.15 scrub starts 2024-09-16T09:45:07.787 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:07 smithi070 ceph-mon[40691]: 1.15 scrub ok 2024-09-16T09:45:07.787 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:07 smithi070 ceph-mon[40691]: 5.1a scrub starts 2024-09-16T09:45:07.787 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:07 smithi070 ceph-mon[40691]: 5.1a scrub ok 2024-09-16T09:45:07.787 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:07 smithi070 ceph-mon[40691]: 1.1b scrub starts 2024-09-16T09:45:07.787 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:07 smithi070 ceph-mon[40691]: 1.1b scrub ok 2024-09-16T09:45:07.788 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:07 smithi070 ceph-mon[40691]: 1.1a scrub starts 2024-09-16T09:45:07.788 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:07 smithi070 ceph-mon[40691]: 1.1a scrub ok 2024-09-16T09:45:07.788 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:07 smithi070 ceph-mon[30388]: pgmap v1062: 162 pgs: 10 peering, 7 unknown, 145 active+clean; 622 KiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 5.2 KiB/s rd, 28 KiB/s wr, 5 op/s; 9.8 KiB/s, 0 objects/s recovering 2024-09-16T09:45:07.788 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:07 smithi070 ceph-mon[30388]: 1.1e scrub starts 2024-09-16T09:45:07.788 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:07 smithi070 ceph-mon[30388]: 1.1e scrub ok 2024-09-16T09:45:07.788 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:07 smithi070 ceph-mon[30388]: 1.1d scrub starts 2024-09-16T09:45:07.788 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:07 smithi070 ceph-mon[30388]: 1.1d scrub ok 2024-09-16T09:45:07.788 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:07 smithi070 ceph-mon[30388]: 1.16 scrub starts 2024-09-16T09:45:07.788 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:07 smithi070 ceph-mon[30388]: 1.16 scrub ok 2024-09-16T09:45:07.788 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:07 smithi070 ceph-mon[30388]: Health check failed: Reduced data availability: 1 pg inactive, 2 pgs peering (PG_AVAILABILITY) 2024-09-16T09:45:07.789 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:07 smithi070 ceph-mon[30388]: 1.15 scrub starts 2024-09-16T09:45:07.789 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:07 smithi070 ceph-mon[30388]: 1.15 scrub ok 2024-09-16T09:45:07.789 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:07 smithi070 ceph-mon[30388]: 5.1a scrub starts 2024-09-16T09:45:07.789 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:07 smithi070 ceph-mon[30388]: 5.1a scrub ok 2024-09-16T09:45:07.789 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:07 smithi070 ceph-mon[30388]: 1.1b scrub starts 2024-09-16T09:45:07.789 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:07 smithi070 ceph-mon[30388]: 1.1b scrub ok 2024-09-16T09:45:07.789 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:07 smithi070 ceph-mon[30388]: 1.1a scrub starts 2024-09-16T09:45:07.789 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:07 smithi070 ceph-mon[30388]: 1.1a scrub ok 2024-09-16T09:45:07.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:07 smithi084 ceph-mon[32629]: pgmap v1062: 162 pgs: 10 peering, 7 unknown, 145 active+clean; 622 KiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 5.2 KiB/s rd, 28 KiB/s wr, 5 op/s; 9.8 KiB/s, 0 objects/s recovering 2024-09-16T09:45:07.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:07 smithi084 ceph-mon[32629]: 1.1e scrub starts 2024-09-16T09:45:07.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:07 smithi084 ceph-mon[32629]: 1.1e scrub ok 2024-09-16T09:45:07.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:07 smithi084 ceph-mon[32629]: 1.1d scrub starts 2024-09-16T09:45:07.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:07 smithi084 ceph-mon[32629]: 1.1d scrub ok 2024-09-16T09:45:07.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:07 smithi084 ceph-mon[32629]: 1.16 scrub starts 2024-09-16T09:45:07.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:07 smithi084 ceph-mon[32629]: 1.16 scrub ok 2024-09-16T09:45:07.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:07 smithi084 ceph-mon[32629]: Health check failed: Reduced data availability: 1 pg inactive, 2 pgs peering (PG_AVAILABILITY) 2024-09-16T09:45:07.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:07 smithi084 ceph-mon[32629]: 1.15 scrub starts 2024-09-16T09:45:07.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:07 smithi084 ceph-mon[32629]: 1.15 scrub ok 2024-09-16T09:45:07.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:07 smithi084 ceph-mon[32629]: 5.1a scrub starts 2024-09-16T09:45:07.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:07 smithi084 ceph-mon[32629]: 5.1a scrub ok 2024-09-16T09:45:07.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:07 smithi084 ceph-mon[32629]: 1.1b scrub starts 2024-09-16T09:45:07.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:07 smithi084 ceph-mon[32629]: 1.1b scrub ok 2024-09-16T09:45:07.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:07 smithi084 ceph-mon[32629]: 1.1a scrub starts 2024-09-16T09:45:07.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:07 smithi084 ceph-mon[32629]: 1.1a scrub ok 2024-09-16T09:45:08.629 INFO:teuthology.orchestra.run.smithi070.stderr:set pool 1 pgp_num to 31 2024-09-16T09:45:08.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:08 smithi084 ceph-mon[32629]: 1.19 scrub starts 2024-09-16T09:45:08.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:08 smithi084 ceph-mon[32629]: 1.19 scrub ok 2024-09-16T09:45:08.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:08 smithi084 ceph-mon[32629]: 1.18 scrub starts 2024-09-16T09:45:08.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:08 smithi084 ceph-mon[32629]: 1.18 scrub ok 2024-09-16T09:45:08.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:08 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/173488121' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "31"} : dispatch 2024-09-16T09:45:08.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:08 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "31"} : dispatch 2024-09-16T09:45:08.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:08 smithi070 ceph-mon[30388]: 1.19 scrub starts 2024-09-16T09:45:08.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:08 smithi070 ceph-mon[30388]: 1.19 scrub ok 2024-09-16T09:45:08.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:08 smithi070 ceph-mon[30388]: 1.18 scrub starts 2024-09-16T09:45:08.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:08 smithi070 ceph-mon[30388]: 1.18 scrub ok 2024-09-16T09:45:08.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:08 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/173488121' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "31"} : dispatch 2024-09-16T09:45:08.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:08 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "31"} : dispatch 2024-09-16T09:45:08.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:08 smithi070 ceph-mon[40691]: 1.19 scrub starts 2024-09-16T09:45:08.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:08 smithi070 ceph-mon[40691]: 1.19 scrub ok 2024-09-16T09:45:08.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:08 smithi070 ceph-mon[40691]: 1.18 scrub starts 2024-09-16T09:45:08.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:08 smithi070 ceph-mon[40691]: 1.18 scrub ok 2024-09-16T09:45:08.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:08 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/173488121' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "31"} : dispatch 2024-09-16T09:45:08.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:08 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "31"} : dispatch 2024-09-16T09:45:09.478 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2024-09-16T09:45:09.756 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:45:09.880 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:09 smithi070 ceph-mon[30388]: pgmap v1063: 162 pgs: 8 peering, 6 unknown, 148 active+clean; 462 KiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 4.9 KiB/s rd, 21 KiB/s wr, 5 op/s 2024-09-16T09:45:09.880 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:09 smithi070 ceph-mon[30388]: 2.1 scrub starts 2024-09-16T09:45:09.880 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:09 smithi070 ceph-mon[30388]: 2.1 scrub ok 2024-09-16T09:45:09.881 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:09 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "31"}]': finished 2024-09-16T09:45:09.881 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:09 smithi070 ceph-mon[30388]: osdmap e546: 8 total, 8 up, 8 in 2024-09-16T09:45:09.881 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:09 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:45:09.881 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:09 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:45:09.881 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:09 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:45:09.881 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:09 smithi070 ceph-mon[30388]: osdmap e547: 8 total, 8 up, 8 in 2024-09-16T09:45:09.881 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:09 smithi070 ceph-mon[40691]: pgmap v1063: 162 pgs: 8 peering, 6 unknown, 148 active+clean; 462 KiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 4.9 KiB/s rd, 21 KiB/s wr, 5 op/s 2024-09-16T09:45:09.881 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:09 smithi070 ceph-mon[40691]: 2.1 scrub starts 2024-09-16T09:45:09.881 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:09 smithi070 ceph-mon[40691]: 2.1 scrub ok 2024-09-16T09:45:09.881 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:09 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "31"}]': finished 2024-09-16T09:45:09.882 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:09 smithi070 ceph-mon[40691]: osdmap e546: 8 total, 8 up, 8 in 2024-09-16T09:45:09.882 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:09 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:45:09.882 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:09 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:45:09.882 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:09 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:45:09.882 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:09 smithi070 ceph-mon[40691]: osdmap e547: 8 total, 8 up, 8 in 2024-09-16T09:45:09.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:09 smithi084 ceph-mon[32629]: pgmap v1063: 162 pgs: 8 peering, 6 unknown, 148 active+clean; 462 KiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 4.9 KiB/s rd, 21 KiB/s wr, 5 op/s 2024-09-16T09:45:09.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:09 smithi084 ceph-mon[32629]: 2.1 scrub starts 2024-09-16T09:45:09.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:09 smithi084 ceph-mon[32629]: 2.1 scrub ok 2024-09-16T09:45:09.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:09 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "31"}]': finished 2024-09-16T09:45:09.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:09 smithi084 ceph-mon[32629]: osdmap e546: 8 total, 8 up, 8 in 2024-09-16T09:45:09.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:09 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:45:09.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:09 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:45:09.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:09 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:45:09.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:09 smithi084 ceph-mon[32629]: osdmap e547: 8 total, 8 up, 8 in 2024-09-16T09:45:10.334 INFO:teuthology.orchestra.run.smithi070.stdout:mon.a: {} 2024-09-16T09:45:10.334 INFO:teuthology.orchestra.run.smithi070.stdout:mon.b: {} 2024-09-16T09:45:10.335 INFO:teuthology.orchestra.run.smithi070.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-16T09:45:10.335 INFO:teuthology.orchestra.run.smithi070.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-16T09:45:10.335 INFO:teuthology.orchestra.run.smithi070.stdout:mon.c: {} 2024-09-16T09:45:10.335 INFO:teuthology.orchestra.run.smithi070.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-16T09:45:10.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:10 smithi084 ceph-mon[32629]: 6.b scrub starts 2024-09-16T09:45:10.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:10 smithi084 ceph-mon[32629]: 6.b scrub ok 2024-09-16T09:45:10.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:10 smithi084 ceph-mon[32629]: 6.c scrub starts 2024-09-16T09:45:10.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:10 smithi084 ceph-mon[32629]: 6.c scrub ok 2024-09-16T09:45:10.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:10 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:45:10.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:10 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:45:10.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:10 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:45:10.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:10 smithi084 ceph-mon[32629]: 3.3 scrub starts 2024-09-16T09:45:10.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:10 smithi084 ceph-mon[32629]: 3.3 scrub ok 2024-09-16T09:45:10.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:10 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/2978659371' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-16T09:45:10.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:10 smithi070 ceph-mon[40691]: 6.b scrub starts 2024-09-16T09:45:10.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:10 smithi070 ceph-mon[40691]: 6.b scrub ok 2024-09-16T09:45:10.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:10 smithi070 ceph-mon[40691]: 6.c scrub starts 2024-09-16T09:45:10.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:10 smithi070 ceph-mon[40691]: 6.c scrub ok 2024-09-16T09:45:10.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:10 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:45:10.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:10 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:45:10.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:10 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:45:10.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:10 smithi070 ceph-mon[40691]: 3.3 scrub starts 2024-09-16T09:45:10.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:10 smithi070 ceph-mon[40691]: 3.3 scrub ok 2024-09-16T09:45:10.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:10 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/2978659371' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-16T09:45:10.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:10 smithi070 ceph-mon[30388]: 6.b scrub starts 2024-09-16T09:45:10.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:10 smithi070 ceph-mon[30388]: 6.b scrub ok 2024-09-16T09:45:10.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:10 smithi070 ceph-mon[30388]: 6.c scrub starts 2024-09-16T09:45:10.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:10 smithi070 ceph-mon[30388]: 6.c scrub ok 2024-09-16T09:45:10.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:10 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:45:10.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:10 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:45:10.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:10 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:45:10.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:10 smithi070 ceph-mon[30388]: 3.3 scrub starts 2024-09-16T09:45:10.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:10 smithi070 ceph-mon[30388]: 3.3 scrub ok 2024-09-16T09:45:10.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:10 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2978659371' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-16T09:45:10.966 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd reweight 0 1 2024-09-16T09:45:11.212 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:45:11.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:11 smithi084 ceph-mon[32629]: pgmap v1066: 162 pgs: 162 active+clean; 583 KiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 11 KiB/s rd, 0 B/s wr, 18 op/s 2024-09-16T09:45:11.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:11 smithi084 ceph-mon[32629]: 6.10 scrub starts 2024-09-16T09:45:11.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:11 smithi084 ceph-mon[32629]: 6.10 scrub ok 2024-09-16T09:45:11.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:11 smithi084 ceph-mon[32629]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 2 pgs peering) 2024-09-16T09:45:11.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:11 smithi084 ceph-mon[32629]: Cluster is now healthy 2024-09-16T09:45:11.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:11 smithi070 ceph-mon[40691]: pgmap v1066: 162 pgs: 162 active+clean; 583 KiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 11 KiB/s rd, 0 B/s wr, 18 op/s 2024-09-16T09:45:11.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:11 smithi070 ceph-mon[40691]: 6.10 scrub starts 2024-09-16T09:45:11.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:11 smithi070 ceph-mon[40691]: 6.10 scrub ok 2024-09-16T09:45:11.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:11 smithi070 ceph-mon[40691]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 2 pgs peering) 2024-09-16T09:45:11.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:11 smithi070 ceph-mon[40691]: Cluster is now healthy 2024-09-16T09:45:11.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:11 smithi070 ceph-mon[30388]: pgmap v1066: 162 pgs: 162 active+clean; 583 KiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 11 KiB/s rd, 0 B/s wr, 18 op/s 2024-09-16T09:45:11.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:11 smithi070 ceph-mon[30388]: 6.10 scrub starts 2024-09-16T09:45:11.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:11 smithi070 ceph-mon[30388]: 6.10 scrub ok 2024-09-16T09:45:11.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:11 smithi070 ceph-mon[30388]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 2 pgs peering) 2024-09-16T09:45:11.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:11 smithi070 ceph-mon[30388]: Cluster is now healthy 2024-09-16T09:45:12.421 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:45:11 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:45:11] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:45:12.634 INFO:teuthology.orchestra.run.smithi070.stderr:reweighted osd.0 to 1 (10000) 2024-09-16T09:45:12.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:12 smithi084 ceph-mon[32629]: 5.17 scrub starts 2024-09-16T09:45:12.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:12 smithi084 ceph-mon[32629]: 5.17 scrub ok 2024-09-16T09:45:12.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:12 smithi084 ceph-mon[32629]: 3.a scrub starts 2024-09-16T09:45:12.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:12 smithi084 ceph-mon[32629]: 3.a scrub ok 2024-09-16T09:45:12.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:12 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/4041531393' entity='client.admin' cmd={"prefix": "osd reweight", "id": 0, "weight": 1.0} : dispatch 2024-09-16T09:45:12.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:12 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 0, "weight": 1.0} : dispatch 2024-09-16T09:45:12.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:12 smithi084 ceph-mon[32629]: 5.18 scrub starts 2024-09-16T09:45:12.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:12 smithi084 ceph-mon[32629]: 5.18 scrub ok 2024-09-16T09:45:12.920 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:12 smithi070 ceph-mon[40691]: 5.17 scrub starts 2024-09-16T09:45:12.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:12 smithi070 ceph-mon[40691]: 5.17 scrub ok 2024-09-16T09:45:12.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:12 smithi070 ceph-mon[40691]: 3.a scrub starts 2024-09-16T09:45:12.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:12 smithi070 ceph-mon[40691]: 3.a scrub ok 2024-09-16T09:45:12.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:12 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/4041531393' entity='client.admin' cmd={"prefix": "osd reweight", "id": 0, "weight": 1.0} : dispatch 2024-09-16T09:45:12.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:12 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 0, "weight": 1.0} : dispatch 2024-09-16T09:45:12.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:12 smithi070 ceph-mon[40691]: 5.18 scrub starts 2024-09-16T09:45:12.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:12 smithi070 ceph-mon[40691]: 5.18 scrub ok 2024-09-16T09:45:12.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:12 smithi070 ceph-mon[30388]: 5.17 scrub starts 2024-09-16T09:45:12.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:12 smithi070 ceph-mon[30388]: 5.17 scrub ok 2024-09-16T09:45:12.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:12 smithi070 ceph-mon[30388]: 3.a scrub starts 2024-09-16T09:45:12.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:12 smithi070 ceph-mon[30388]: 3.a scrub ok 2024-09-16T09:45:12.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:12 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/4041531393' entity='client.admin' cmd={"prefix": "osd reweight", "id": 0, "weight": 1.0} : dispatch 2024-09-16T09:45:12.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:12 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 0, "weight": 1.0} : dispatch 2024-09-16T09:45:12.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:12 smithi070 ceph-mon[30388]: 5.18 scrub starts 2024-09-16T09:45:12.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:12 smithi070 ceph-mon[30388]: 5.18 scrub ok 2024-09-16T09:45:13.334 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd primary-affinity 0 1 2024-09-16T09:45:13.583 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:45:13.891 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:13 smithi070 ceph-mon[30388]: pgmap v1067: 162 pgs: 162 active+clean; 583 KiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 7.1 KiB/s rd, 0 B/s wr, 12 op/s 2024-09-16T09:45:13.891 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:13 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-16T09:45:13.891 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:13 smithi070 ceph-mon[30388]: osdmap e548: 8 total, 8 up, 8 in 2024-09-16T09:45:13.891 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:13 smithi070 ceph-mon[30388]: 6.1f scrub starts 2024-09-16T09:45:13.891 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:13 smithi070 ceph-mon[30388]: 6.1f scrub ok 2024-09-16T09:45:13.892 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:13 smithi070 ceph-mon[40691]: pgmap v1067: 162 pgs: 162 active+clean; 583 KiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 7.1 KiB/s rd, 0 B/s wr, 12 op/s 2024-09-16T09:45:13.892 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:13 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-16T09:45:13.892 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:13 smithi070 ceph-mon[40691]: osdmap e548: 8 total, 8 up, 8 in 2024-09-16T09:45:13.892 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:13 smithi070 ceph-mon[40691]: 6.1f scrub starts 2024-09-16T09:45:13.892 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:13 smithi070 ceph-mon[40691]: 6.1f scrub ok 2024-09-16T09:45:13.910 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:45:13 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:45:13.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:13 smithi084 ceph-mon[32629]: pgmap v1067: 162 pgs: 162 active+clean; 583 KiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 7.1 KiB/s rd, 0 B/s wr, 12 op/s 2024-09-16T09:45:13.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:13 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-16T09:45:13.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:13 smithi084 ceph-mon[32629]: osdmap e548: 8 total, 8 up, 8 in 2024-09-16T09:45:13.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:13 smithi084 ceph-mon[32629]: 6.1f scrub starts 2024-09-16T09:45:13.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:13 smithi084 ceph-mon[32629]: 6.1f scrub ok 2024-09-16T09:45:14.491 INFO:teuthology.orchestra.run.smithi070.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-09-16T09:45:14.795 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:14 smithi070 ceph-mon[40691]: 5.13 scrub starts 2024-09-16T09:45:14.795 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:14 smithi070 ceph-mon[40691]: 5.13 scrub ok 2024-09-16T09:45:14.795 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:14 smithi070 ceph-mon[40691]: 4.2 scrub starts 2024-09-16T09:45:14.795 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:14 smithi070 ceph-mon[40691]: 4.2 scrub ok 2024-09-16T09:45:14.795 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:14 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1523233965' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 0, "weight": 1.0} : dispatch 2024-09-16T09:45:14.796 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:14 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 0, "weight": 1.0} : dispatch 2024-09-16T09:45:14.796 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:14 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 0, "weight": 1.0}]': finished 2024-09-16T09:45:14.796 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:14 smithi070 ceph-mon[40691]: osdmap e549: 8 total, 8 up, 8 in 2024-09-16T09:45:14.796 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:14 smithi070 ceph-mon[30388]: 5.13 scrub starts 2024-09-16T09:45:14.796 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:14 smithi070 ceph-mon[30388]: 5.13 scrub ok 2024-09-16T09:45:14.796 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:14 smithi070 ceph-mon[30388]: 4.2 scrub starts 2024-09-16T09:45:14.796 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:14 smithi070 ceph-mon[30388]: 4.2 scrub ok 2024-09-16T09:45:14.796 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:14 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1523233965' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 0, "weight": 1.0} : dispatch 2024-09-16T09:45:14.796 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:14 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 0, "weight": 1.0} : dispatch 2024-09-16T09:45:14.796 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:14 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 0, "weight": 1.0}]': finished 2024-09-16T09:45:14.796 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:14 smithi070 ceph-mon[30388]: osdmap e549: 8 total, 8 up, 8 in 2024-09-16T09:45:14.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:14 smithi084 ceph-mon[32629]: 5.13 scrub starts 2024-09-16T09:45:14.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:14 smithi084 ceph-mon[32629]: 5.13 scrub ok 2024-09-16T09:45:14.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:14 smithi084 ceph-mon[32629]: 4.2 scrub starts 2024-09-16T09:45:14.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:14 smithi084 ceph-mon[32629]: 4.2 scrub ok 2024-09-16T09:45:14.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:14 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1523233965' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 0, "weight": 1.0} : dispatch 2024-09-16T09:45:14.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:14 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 0, "weight": 1.0} : dispatch 2024-09-16T09:45:14.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:14 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 0, "weight": 1.0}]': finished 2024-09-16T09:45:14.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:14 smithi084 ceph-mon[32629]: osdmap e549: 8 total, 8 up, 8 in 2024-09-16T09:45:15.123 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd reweight 1 1 2024-09-16T09:45:15.372 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:45:15.775 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:15 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:45:15.775 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:15 smithi070 ceph-mon[30388]: pgmap v1069: 162 pgs: 162 active+clean; 583 KiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 68 KiB/s rd, 0 B/s wr, 114 op/s 2024-09-16T09:45:15.775 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:15 smithi070 ceph-mon[30388]: 3.8 scrub starts 2024-09-16T09:45:15.775 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:15 smithi070 ceph-mon[30388]: 3.8 scrub ok 2024-09-16T09:45:15.775 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:15 smithi070 ceph-mon[30388]: 4.1e scrub starts 2024-09-16T09:45:15.775 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:15 smithi070 ceph-mon[30388]: 4.1e scrub ok 2024-09-16T09:45:15.775 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:15 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:45:15.775 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:15 smithi070 ceph-mon[40691]: pgmap v1069: 162 pgs: 162 active+clean; 583 KiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 68 KiB/s rd, 0 B/s wr, 114 op/s 2024-09-16T09:45:15.775 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:15 smithi070 ceph-mon[40691]: 3.8 scrub starts 2024-09-16T09:45:15.775 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:15 smithi070 ceph-mon[40691]: 3.8 scrub ok 2024-09-16T09:45:15.775 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:15 smithi070 ceph-mon[40691]: 4.1e scrub starts 2024-09-16T09:45:15.776 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:15 smithi070 ceph-mon[40691]: 4.1e scrub ok 2024-09-16T09:45:15.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:15 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:45:15.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:15 smithi084 ceph-mon[32629]: pgmap v1069: 162 pgs: 162 active+clean; 583 KiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 68 KiB/s rd, 0 B/s wr, 114 op/s 2024-09-16T09:45:15.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:15 smithi084 ceph-mon[32629]: 3.8 scrub starts 2024-09-16T09:45:15.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:15 smithi084 ceph-mon[32629]: 3.8 scrub ok 2024-09-16T09:45:15.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:15 smithi084 ceph-mon[32629]: 4.1e scrub starts 2024-09-16T09:45:15.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:15 smithi084 ceph-mon[32629]: 4.1e scrub ok 2024-09-16T09:45:16.646 INFO:teuthology.orchestra.run.smithi070.stderr:reweighted osd.1 to 1 (10000) 2024-09-16T09:45:16.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:16 smithi084 ceph-mon[32629]: pgmap v1071: 162 pgs: 162 active+clean; 583 KiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 65 KiB/s rd, 0 B/s wr, 109 op/s 2024-09-16T09:45:16.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:16 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/2566180113' entity='client.admin' cmd={"prefix": "osd reweight", "id": 1, "weight": 1.0} : dispatch 2024-09-16T09:45:16.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:16 smithi070 ceph-mon[30388]: pgmap v1071: 162 pgs: 162 active+clean; 583 KiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 65 KiB/s rd, 0 B/s wr, 109 op/s 2024-09-16T09:45:16.920 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:16 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2566180113' entity='client.admin' cmd={"prefix": "osd reweight", "id": 1, "weight": 1.0} : dispatch 2024-09-16T09:45:16.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:16 smithi070 ceph-mon[40691]: pgmap v1071: 162 pgs: 162 active+clean; 583 KiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 65 KiB/s rd, 0 B/s wr, 109 op/s 2024-09-16T09:45:16.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:16 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/2566180113' entity='client.admin' cmd={"prefix": "osd reweight", "id": 1, "weight": 1.0} : dispatch 2024-09-16T09:45:17.442 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd primary-affinity 1 1 2024-09-16T09:45:17.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:17 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2566180113' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-16T09:45:17.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:17 smithi070 ceph-mon[30388]: osdmap e550: 8 total, 8 up, 8 in 2024-09-16T09:45:17.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:17 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/2566180113' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-16T09:45:17.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:17 smithi070 ceph-mon[40691]: osdmap e550: 8 total, 8 up, 8 in 2024-09-16T09:45:17.701 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:45:17.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:17 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/2566180113' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-16T09:45:17.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:17 smithi084 ceph-mon[32629]: osdmap e550: 8 total, 8 up, 8 in 2024-09-16T09:45:18.661 INFO:teuthology.orchestra.run.smithi070.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-09-16T09:45:18.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:18 smithi070 ceph-mon[40691]: 4.a scrub starts 2024-09-16T09:45:18.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:18 smithi070 ceph-mon[40691]: 4.a scrub ok 2024-09-16T09:45:18.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:18 smithi070 ceph-mon[40691]: pgmap v1073: 162 pgs: 162 active+clean; 583 KiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 64 KiB/s rd, 0 B/s wr, 104 op/s 2024-09-16T09:45:18.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:18 smithi070 ceph-mon[40691]: 4.1b scrub starts 2024-09-16T09:45:18.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:18 smithi070 ceph-mon[40691]: 4.1b scrub ok 2024-09-16T09:45:18.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:18 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/2428949491' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 1, "weight": 1.0} : dispatch 2024-09-16T09:45:18.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:18 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 1, "weight": 1.0} : dispatch 2024-09-16T09:45:18.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:18 smithi070 ceph-mon[30388]: 4.a scrub starts 2024-09-16T09:45:18.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:18 smithi070 ceph-mon[30388]: 4.a scrub ok 2024-09-16T09:45:18.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:18 smithi070 ceph-mon[30388]: pgmap v1073: 162 pgs: 162 active+clean; 583 KiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 64 KiB/s rd, 0 B/s wr, 104 op/s 2024-09-16T09:45:18.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:18 smithi070 ceph-mon[30388]: 4.1b scrub starts 2024-09-16T09:45:18.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:18 smithi070 ceph-mon[30388]: 4.1b scrub ok 2024-09-16T09:45:18.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:18 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2428949491' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 1, "weight": 1.0} : dispatch 2024-09-16T09:45:18.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:18 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 1, "weight": 1.0} : dispatch 2024-09-16T09:45:19.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:18 smithi084 ceph-mon[32629]: 4.a scrub starts 2024-09-16T09:45:19.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:18 smithi084 ceph-mon[32629]: 4.a scrub ok 2024-09-16T09:45:19.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:18 smithi084 ceph-mon[32629]: pgmap v1073: 162 pgs: 162 active+clean; 583 KiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 64 KiB/s rd, 0 B/s wr, 104 op/s 2024-09-16T09:45:19.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:18 smithi084 ceph-mon[32629]: 4.1b scrub starts 2024-09-16T09:45:19.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:18 smithi084 ceph-mon[32629]: 4.1b scrub ok 2024-09-16T09:45:19.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:18 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/2428949491' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 1, "weight": 1.0} : dispatch 2024-09-16T09:45:19.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:18 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 1, "weight": 1.0} : dispatch 2024-09-16T09:45:19.429 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd reweight 3 1 2024-09-16T09:45:19.697 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:45:19.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:19 smithi070 ceph-mon[30388]: 3.19 scrub starts 2024-09-16T09:45:19.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:19 smithi070 ceph-mon[30388]: 3.19 scrub ok 2024-09-16T09:45:19.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:19 smithi070 ceph-mon[30388]: 4.14 scrub starts 2024-09-16T09:45:19.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:19 smithi070 ceph-mon[30388]: 4.14 scrub ok 2024-09-16T09:45:19.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:19 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 1, "weight": 1.0}]': finished 2024-09-16T09:45:19.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:19 smithi070 ceph-mon[30388]: osdmap e551: 8 total, 8 up, 8 in 2024-09-16T09:45:19.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:19 smithi070 ceph-mon[30388]: 3.1c scrub starts 2024-09-16T09:45:19.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:19 smithi070 ceph-mon[30388]: 3.1c scrub ok 2024-09-16T09:45:19.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:19 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:45:19.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:19 smithi070 ceph-mon[40691]: 3.19 scrub starts 2024-09-16T09:45:19.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:19 smithi070 ceph-mon[40691]: 3.19 scrub ok 2024-09-16T09:45:19.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:19 smithi070 ceph-mon[40691]: 4.14 scrub starts 2024-09-16T09:45:19.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:19 smithi070 ceph-mon[40691]: 4.14 scrub ok 2024-09-16T09:45:19.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:19 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 1, "weight": 1.0}]': finished 2024-09-16T09:45:19.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:19 smithi070 ceph-mon[40691]: osdmap e551: 8 total, 8 up, 8 in 2024-09-16T09:45:19.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:19 smithi070 ceph-mon[40691]: 3.1c scrub starts 2024-09-16T09:45:19.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:19 smithi070 ceph-mon[40691]: 3.1c scrub ok 2024-09-16T09:45:19.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:19 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:45:20.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:19 smithi084 ceph-mon[32629]: 3.19 scrub starts 2024-09-16T09:45:20.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:19 smithi084 ceph-mon[32629]: 3.19 scrub ok 2024-09-16T09:45:20.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:19 smithi084 ceph-mon[32629]: 4.14 scrub starts 2024-09-16T09:45:20.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:19 smithi084 ceph-mon[32629]: 4.14 scrub ok 2024-09-16T09:45:20.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:19 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 1, "weight": 1.0}]': finished 2024-09-16T09:45:20.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:19 smithi084 ceph-mon[32629]: osdmap e551: 8 total, 8 up, 8 in 2024-09-16T09:45:20.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:19 smithi084 ceph-mon[32629]: 3.1c scrub starts 2024-09-16T09:45:20.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:19 smithi084 ceph-mon[32629]: 3.1c scrub ok 2024-09-16T09:45:20.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:19 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:45:20.674 INFO:teuthology.orchestra.run.smithi070.stderr:reweighted osd.3 to 1 (10000) 2024-09-16T09:45:20.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:20 smithi070 ceph-mon[30388]: osdmap e552: 8 total, 8 up, 8 in 2024-09-16T09:45:20.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:20 smithi070 ceph-mon[30388]: pgmap v1076: 162 pgs: 23 peering, 139 active+clean; 609 KiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 3.5 KiB/s rd, 14 KiB/s wr, 4 op/s; 4.8 KiB/s, 0 objects/s recovering 2024-09-16T09:45:20.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:20 smithi070 ceph-mon[30388]: 3.9 scrub starts 2024-09-16T09:45:20.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:20 smithi070 ceph-mon[30388]: 3.9 scrub ok 2024-09-16T09:45:20.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:20 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/975576577' entity='client.admin' cmd={"prefix": "osd reweight", "id": 3, "weight": 1.0} : dispatch 2024-09-16T09:45:20.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:20 smithi070 ceph-mon[40691]: osdmap e552: 8 total, 8 up, 8 in 2024-09-16T09:45:20.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:20 smithi070 ceph-mon[40691]: pgmap v1076: 162 pgs: 23 peering, 139 active+clean; 609 KiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 3.5 KiB/s rd, 14 KiB/s wr, 4 op/s; 4.8 KiB/s, 0 objects/s recovering 2024-09-16T09:45:20.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:20 smithi070 ceph-mon[40691]: 3.9 scrub starts 2024-09-16T09:45:20.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:20 smithi070 ceph-mon[40691]: 3.9 scrub ok 2024-09-16T09:45:20.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:20 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/975576577' entity='client.admin' cmd={"prefix": "osd reweight", "id": 3, "weight": 1.0} : dispatch 2024-09-16T09:45:21.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:20 smithi084 ceph-mon[32629]: osdmap e552: 8 total, 8 up, 8 in 2024-09-16T09:45:21.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:20 smithi084 ceph-mon[32629]: pgmap v1076: 162 pgs: 23 peering, 139 active+clean; 609 KiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 3.5 KiB/s rd, 14 KiB/s wr, 4 op/s; 4.8 KiB/s, 0 objects/s recovering 2024-09-16T09:45:21.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:20 smithi084 ceph-mon[32629]: 3.9 scrub starts 2024-09-16T09:45:21.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:20 smithi084 ceph-mon[32629]: 3.9 scrub ok 2024-09-16T09:45:21.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:20 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/975576577' entity='client.admin' cmd={"prefix": "osd reweight", "id": 3, "weight": 1.0} : dispatch 2024-09-16T09:45:21.469 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd primary-affinity 3 1 2024-09-16T09:45:21.719 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:45:21.930 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:21 smithi070 ceph-mon[30388]: 5.8 scrub starts 2024-09-16T09:45:21.930 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:21 smithi070 ceph-mon[30388]: 5.8 scrub ok 2024-09-16T09:45:21.930 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:21 smithi070 ceph-mon[30388]: 1.1d scrub starts 2024-09-16T09:45:21.930 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:21 smithi070 ceph-mon[30388]: 1.1d scrub ok 2024-09-16T09:45:21.931 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:21 smithi070 ceph-mon[30388]: 6.1c scrub starts 2024-09-16T09:45:21.931 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:21 smithi070 ceph-mon[30388]: 6.1c scrub ok 2024-09-16T09:45:21.931 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:21 smithi070 ceph-mon[30388]: Health check failed: Reduced data availability: 9 pgs inactive, 15 pgs peering (PG_AVAILABILITY) 2024-09-16T09:45:21.931 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:21 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/975576577' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-16T09:45:21.931 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:21 smithi070 ceph-mon[30388]: osdmap e553: 8 total, 8 up, 8 in 2024-09-16T09:45:21.931 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:21 smithi070 ceph-mon[40691]: 5.8 scrub starts 2024-09-16T09:45:21.931 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:21 smithi070 ceph-mon[40691]: 5.8 scrub ok 2024-09-16T09:45:21.931 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:21 smithi070 ceph-mon[40691]: 1.1d scrub starts 2024-09-16T09:45:21.931 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:21 smithi070 ceph-mon[40691]: 1.1d scrub ok 2024-09-16T09:45:21.931 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:21 smithi070 ceph-mon[40691]: 6.1c scrub starts 2024-09-16T09:45:21.931 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:21 smithi070 ceph-mon[40691]: 6.1c scrub ok 2024-09-16T09:45:21.931 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:21 smithi070 ceph-mon[40691]: Health check failed: Reduced data availability: 9 pgs inactive, 15 pgs peering (PG_AVAILABILITY) 2024-09-16T09:45:21.932 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:21 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/975576577' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-16T09:45:21.932 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:21 smithi070 ceph-mon[40691]: osdmap e553: 8 total, 8 up, 8 in 2024-09-16T09:45:22.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:21 smithi084 ceph-mon[32629]: 5.8 scrub starts 2024-09-16T09:45:22.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:21 smithi084 ceph-mon[32629]: 5.8 scrub ok 2024-09-16T09:45:22.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:21 smithi084 ceph-mon[32629]: 1.1d scrub starts 2024-09-16T09:45:22.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:21 smithi084 ceph-mon[32629]: 1.1d scrub ok 2024-09-16T09:45:22.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:21 smithi084 ceph-mon[32629]: 6.1c scrub starts 2024-09-16T09:45:22.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:21 smithi084 ceph-mon[32629]: 6.1c scrub ok 2024-09-16T09:45:22.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:21 smithi084 ceph-mon[32629]: Health check failed: Reduced data availability: 9 pgs inactive, 15 pgs peering (PG_AVAILABILITY) 2024-09-16T09:45:22.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:21 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/975576577' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-16T09:45:22.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:21 smithi084 ceph-mon[32629]: osdmap e553: 8 total, 8 up, 8 in 2024-09-16T09:45:22.246 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:45:21 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:45:21] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:45:22.683 INFO:teuthology.orchestra.run.smithi070.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-09-16T09:45:22.991 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:22 smithi070 ceph-mon[30388]: 1.16 scrub starts 2024-09-16T09:45:22.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:22 smithi070 ceph-mon[30388]: 1.16 scrub ok 2024-09-16T09:45:22.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:22 smithi070 ceph-mon[30388]: 3.f scrub starts 2024-09-16T09:45:22.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:22 smithi070 ceph-mon[30388]: 3.f scrub ok 2024-09-16T09:45:22.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:22 smithi070 ceph-mon[30388]: pgmap v1078: 162 pgs: 23 peering, 139 active+clean; 609 KiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 1.9 KiB/s rd, 14 KiB/s wr, 2 op/s; 5.0 KiB/s, 0 objects/s recovering 2024-09-16T09:45:22.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:22 smithi070 ceph-mon[30388]: 4.9 scrub starts 2024-09-16T09:45:22.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:22 smithi070 ceph-mon[30388]: 4.9 scrub ok 2024-09-16T09:45:22.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:22 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1243130976' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 3, "weight": 1.0} : dispatch 2024-09-16T09:45:22.992 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:22 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 3, "weight": 1.0} : dispatch 2024-09-16T09:45:22.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:22 smithi070 ceph-mon[40691]: 1.16 scrub starts 2024-09-16T09:45:22.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:22 smithi070 ceph-mon[40691]: 1.16 scrub ok 2024-09-16T09:45:22.992 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:22 smithi070 ceph-mon[40691]: 3.f scrub starts 2024-09-16T09:45:22.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:22 smithi070 ceph-mon[40691]: 3.f scrub ok 2024-09-16T09:45:22.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:22 smithi070 ceph-mon[40691]: pgmap v1078: 162 pgs: 23 peering, 139 active+clean; 609 KiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 1.9 KiB/s rd, 14 KiB/s wr, 2 op/s; 5.0 KiB/s, 0 objects/s recovering 2024-09-16T09:45:22.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:22 smithi070 ceph-mon[40691]: 4.9 scrub starts 2024-09-16T09:45:22.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:22 smithi070 ceph-mon[40691]: 4.9 scrub ok 2024-09-16T09:45:22.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:22 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1243130976' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 3, "weight": 1.0} : dispatch 2024-09-16T09:45:22.993 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:22 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 3, "weight": 1.0} : dispatch 2024-09-16T09:45:23.062 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:22 smithi084 ceph-mon[32629]: 1.16 scrub starts 2024-09-16T09:45:23.062 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:22 smithi084 ceph-mon[32629]: 1.16 scrub ok 2024-09-16T09:45:23.062 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:22 smithi084 ceph-mon[32629]: 3.f scrub starts 2024-09-16T09:45:23.062 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:22 smithi084 ceph-mon[32629]: 3.f scrub ok 2024-09-16T09:45:23.062 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:22 smithi084 ceph-mon[32629]: pgmap v1078: 162 pgs: 23 peering, 139 active+clean; 609 KiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 1.9 KiB/s rd, 14 KiB/s wr, 2 op/s; 5.0 KiB/s, 0 objects/s recovering 2024-09-16T09:45:23.062 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:22 smithi084 ceph-mon[32629]: 4.9 scrub starts 2024-09-16T09:45:23.062 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:22 smithi084 ceph-mon[32629]: 4.9 scrub ok 2024-09-16T09:45:23.062 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:22 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1243130976' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 3, "weight": 1.0} : dispatch 2024-09-16T09:45:23.062 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:22 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 3, "weight": 1.0} : dispatch 2024-09-16T09:45:23.430 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd reweight 4 1 2024-09-16T09:45:23.680 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:45:23.938 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:23 smithi070 ceph-mon[40691]: 1.b scrub starts 2024-09-16T09:45:23.938 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:23 smithi070 ceph-mon[40691]: 1.b scrub ok 2024-09-16T09:45:23.939 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:23 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 3, "weight": 1.0}]': finished 2024-09-16T09:45:23.939 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:23 smithi070 ceph-mon[40691]: osdmap e554: 8 total, 8 up, 8 in 2024-09-16T09:45:23.939 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:23 smithi070 ceph-mon[40691]: 5.f scrub starts 2024-09-16T09:45:23.939 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:23 smithi070 ceph-mon[40691]: 5.f scrub ok 2024-09-16T09:45:23.939 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:23 smithi070 ceph-mon[30388]: 1.b scrub starts 2024-09-16T09:45:23.939 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:23 smithi070 ceph-mon[30388]: 1.b scrub ok 2024-09-16T09:45:23.939 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:23 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 3, "weight": 1.0}]': finished 2024-09-16T09:45:23.939 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:23 smithi070 ceph-mon[30388]: osdmap e554: 8 total, 8 up, 8 in 2024-09-16T09:45:23.939 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:23 smithi070 ceph-mon[30388]: 5.f scrub starts 2024-09-16T09:45:23.939 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:23 smithi070 ceph-mon[30388]: 5.f scrub ok 2024-09-16T09:45:24.160 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:45:23 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:45:24.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:23 smithi084 ceph-mon[32629]: 1.b scrub starts 2024-09-16T09:45:24.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:23 smithi084 ceph-mon[32629]: 1.b scrub ok 2024-09-16T09:45:24.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:23 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 3, "weight": 1.0}]': finished 2024-09-16T09:45:24.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:23 smithi084 ceph-mon[32629]: osdmap e554: 8 total, 8 up, 8 in 2024-09-16T09:45:24.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:23 smithi084 ceph-mon[32629]: 5.f scrub starts 2024-09-16T09:45:24.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:23 smithi084 ceph-mon[32629]: 5.f scrub ok 2024-09-16T09:45:24.691 INFO:teuthology.orchestra.run.smithi070.stderr:reweighted osd.4 to 1 (10000) 2024-09-16T09:45:25.084 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:24 smithi070 ceph-mon[30388]: 1.11 scrub starts 2024-09-16T09:45:25.084 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:24 smithi070 ceph-mon[30388]: 1.11 scrub ok 2024-09-16T09:45:25.084 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:24 smithi070 ceph-mon[30388]: osdmap e555: 8 total, 8 up, 8 in 2024-09-16T09:45:25.084 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:24 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:45:25.084 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:24 smithi070 ceph-mon[30388]: pgmap v1081: 162 pgs: 162 active+clean; 609 KiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 719 B/s rd, 0 op/s 2024-09-16T09:45:25.084 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:24 smithi070 ceph-mon[30388]: 4.d scrub starts 2024-09-16T09:45:25.084 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:24 smithi070 ceph-mon[30388]: 4.d scrub ok 2024-09-16T09:45:25.084 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:24 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3416956756' entity='client.admin' cmd={"prefix": "osd reweight", "id": 4, "weight": 1.0} : dispatch 2024-09-16T09:45:25.085 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:24 smithi070 ceph-mon[40691]: 1.11 scrub starts 2024-09-16T09:45:25.085 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:24 smithi070 ceph-mon[40691]: 1.11 scrub ok 2024-09-16T09:45:25.085 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:24 smithi070 ceph-mon[40691]: osdmap e555: 8 total, 8 up, 8 in 2024-09-16T09:45:25.085 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:24 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:45:25.085 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:24 smithi070 ceph-mon[40691]: pgmap v1081: 162 pgs: 162 active+clean; 609 KiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 719 B/s rd, 0 op/s 2024-09-16T09:45:25.085 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:24 smithi070 ceph-mon[40691]: 4.d scrub starts 2024-09-16T09:45:25.085 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:24 smithi070 ceph-mon[40691]: 4.d scrub ok 2024-09-16T09:45:25.085 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:24 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3416956756' entity='client.admin' cmd={"prefix": "osd reweight", "id": 4, "weight": 1.0} : dispatch 2024-09-16T09:45:25.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:24 smithi084 ceph-mon[32629]: 1.11 scrub starts 2024-09-16T09:45:25.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:24 smithi084 ceph-mon[32629]: 1.11 scrub ok 2024-09-16T09:45:25.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:24 smithi084 ceph-mon[32629]: osdmap e555: 8 total, 8 up, 8 in 2024-09-16T09:45:25.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:24 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:45:25.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:24 smithi084 ceph-mon[32629]: pgmap v1081: 162 pgs: 162 active+clean; 609 KiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 719 B/s rd, 0 op/s 2024-09-16T09:45:25.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:24 smithi084 ceph-mon[32629]: 4.d scrub starts 2024-09-16T09:45:25.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:24 smithi084 ceph-mon[32629]: 4.d scrub ok 2024-09-16T09:45:25.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:24 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3416956756' entity='client.admin' cmd={"prefix": "osd reweight", "id": 4, "weight": 1.0} : dispatch 2024-09-16T09:45:25.587 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd primary-affinity 4 1 2024-09-16T09:45:25.840 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:45:25.920 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:25 smithi070 ceph-mon[40691]: 1.19 scrub starts 2024-09-16T09:45:25.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:25 smithi070 ceph-mon[40691]: 1.19 scrub ok 2024-09-16T09:45:25.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:25 smithi070 ceph-mon[40691]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 9 pgs inactive, 15 pgs peering) 2024-09-16T09:45:25.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:25 smithi070 ceph-mon[40691]: Cluster is now healthy 2024-09-16T09:45:25.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:25 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3416956756' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-16T09:45:25.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:25 smithi070 ceph-mon[40691]: osdmap e556: 8 total, 8 up, 8 in 2024-09-16T09:45:25.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:25 smithi070 ceph-mon[30388]: 1.19 scrub starts 2024-09-16T09:45:25.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:25 smithi070 ceph-mon[30388]: 1.19 scrub ok 2024-09-16T09:45:25.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:25 smithi070 ceph-mon[30388]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 9 pgs inactive, 15 pgs peering) 2024-09-16T09:45:25.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:25 smithi070 ceph-mon[30388]: Cluster is now healthy 2024-09-16T09:45:25.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:25 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3416956756' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-16T09:45:25.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:25 smithi070 ceph-mon[30388]: osdmap e556: 8 total, 8 up, 8 in 2024-09-16T09:45:26.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:25 smithi084 ceph-mon[32629]: 1.19 scrub starts 2024-09-16T09:45:26.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:25 smithi084 ceph-mon[32629]: 1.19 scrub ok 2024-09-16T09:45:26.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:25 smithi084 ceph-mon[32629]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 9 pgs inactive, 15 pgs peering) 2024-09-16T09:45:26.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:25 smithi084 ceph-mon[32629]: Cluster is now healthy 2024-09-16T09:45:26.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:25 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3416956756' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-16T09:45:26.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:25 smithi084 ceph-mon[32629]: osdmap e556: 8 total, 8 up, 8 in 2024-09-16T09:45:26.702 INFO:teuthology.orchestra.run.smithi070.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-09-16T09:45:27.035 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:26 smithi070 ceph-mon[30388]: 5.14 scrub starts 2024-09-16T09:45:27.035 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:26 smithi070 ceph-mon[30388]: 5.14 scrub ok 2024-09-16T09:45:27.036 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:26 smithi070 ceph-mon[30388]: 5.19 scrub starts 2024-09-16T09:45:27.036 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:26 smithi070 ceph-mon[30388]: 5.19 scrub ok 2024-09-16T09:45:27.036 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:26 smithi070 ceph-mon[30388]: 3.e scrub starts 2024-09-16T09:45:27.036 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:26 smithi070 ceph-mon[30388]: 3.e scrub ok 2024-09-16T09:45:27.036 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:26 smithi070 ceph-mon[30388]: 1.f scrub starts 2024-09-16T09:45:27.036 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:26 smithi070 ceph-mon[30388]: 1.f scrub ok 2024-09-16T09:45:27.036 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:26 smithi070 ceph-mon[30388]: pgmap v1083: 162 pgs: 162 active+clean; 609 KiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 583 B/s rd, 0 op/s 2024-09-16T09:45:27.036 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:26 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3982645069' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 4, "weight": 1.0} : dispatch 2024-09-16T09:45:27.036 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:26 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 4, "weight": 1.0} : dispatch 2024-09-16T09:45:27.036 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:26 smithi070 ceph-mon[40691]: 5.14 scrub starts 2024-09-16T09:45:27.036 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:26 smithi070 ceph-mon[40691]: 5.14 scrub ok 2024-09-16T09:45:27.037 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:26 smithi070 ceph-mon[40691]: 5.19 scrub starts 2024-09-16T09:45:27.037 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:26 smithi070 ceph-mon[40691]: 5.19 scrub ok 2024-09-16T09:45:27.037 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:26 smithi070 ceph-mon[40691]: 3.e scrub starts 2024-09-16T09:45:27.037 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:26 smithi070 ceph-mon[40691]: 3.e scrub ok 2024-09-16T09:45:27.037 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:26 smithi070 ceph-mon[40691]: 1.f scrub starts 2024-09-16T09:45:27.037 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:26 smithi070 ceph-mon[40691]: 1.f scrub ok 2024-09-16T09:45:27.037 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:26 smithi070 ceph-mon[40691]: pgmap v1083: 162 pgs: 162 active+clean; 609 KiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 583 B/s rd, 0 op/s 2024-09-16T09:45:27.037 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:26 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3982645069' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 4, "weight": 1.0} : dispatch 2024-09-16T09:45:27.037 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:26 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 4, "weight": 1.0} : dispatch 2024-09-16T09:45:27.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:26 smithi084 ceph-mon[32629]: 5.14 scrub starts 2024-09-16T09:45:27.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:26 smithi084 ceph-mon[32629]: 5.14 scrub ok 2024-09-16T09:45:27.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:26 smithi084 ceph-mon[32629]: 5.19 scrub starts 2024-09-16T09:45:27.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:26 smithi084 ceph-mon[32629]: 5.19 scrub ok 2024-09-16T09:45:27.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:26 smithi084 ceph-mon[32629]: 3.e scrub starts 2024-09-16T09:45:27.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:26 smithi084 ceph-mon[32629]: 3.e scrub ok 2024-09-16T09:45:27.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:26 smithi084 ceph-mon[32629]: 1.f scrub starts 2024-09-16T09:45:27.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:26 smithi084 ceph-mon[32629]: 1.f scrub ok 2024-09-16T09:45:27.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:26 smithi084 ceph-mon[32629]: pgmap v1083: 162 pgs: 162 active+clean; 609 KiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 583 B/s rd, 0 op/s 2024-09-16T09:45:27.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:26 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3982645069' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 4, "weight": 1.0} : dispatch 2024-09-16T09:45:27.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:26 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 4, "weight": 1.0} : dispatch 2024-09-16T09:45:27.431 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd reweight 5 1 2024-09-16T09:45:27.680 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:45:27.957 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:27 smithi070 ceph-mon[30388]: 4.6 scrub starts 2024-09-16T09:45:27.957 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:27 smithi070 ceph-mon[30388]: 4.6 scrub ok 2024-09-16T09:45:27.957 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:27 smithi070 ceph-mon[30388]: 5.0 scrub starts 2024-09-16T09:45:27.957 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:27 smithi070 ceph-mon[30388]: 5.0 scrub ok 2024-09-16T09:45:27.957 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:27 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:45:27.957 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:27 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:45:27.957 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:27 smithi070 ceph-mon[30388]: 3.d scrub starts 2024-09-16T09:45:27.958 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:27 smithi070 ceph-mon[30388]: 3.d scrub ok 2024-09-16T09:45:27.958 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:27 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 4, "weight": 1.0}]': finished 2024-09-16T09:45:27.958 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:27 smithi070 ceph-mon[30388]: osdmap e557: 8 total, 8 up, 8 in 2024-09-16T09:45:27.958 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:27 smithi070 ceph-mon[40691]: 4.6 scrub starts 2024-09-16T09:45:27.958 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:27 smithi070 ceph-mon[40691]: 4.6 scrub ok 2024-09-16T09:45:27.958 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:27 smithi070 ceph-mon[40691]: 5.0 scrub starts 2024-09-16T09:45:27.959 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:27 smithi070 ceph-mon[40691]: 5.0 scrub ok 2024-09-16T09:45:27.959 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:27 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:45:27.959 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:27 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:45:27.959 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:27 smithi070 ceph-mon[40691]: 3.d scrub starts 2024-09-16T09:45:27.959 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:27 smithi070 ceph-mon[40691]: 3.d scrub ok 2024-09-16T09:45:27.960 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:27 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 4, "weight": 1.0}]': finished 2024-09-16T09:45:27.960 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:27 smithi070 ceph-mon[40691]: osdmap e557: 8 total, 8 up, 8 in 2024-09-16T09:45:28.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:27 smithi084 ceph-mon[32629]: 4.6 scrub starts 2024-09-16T09:45:28.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:27 smithi084 ceph-mon[32629]: 4.6 scrub ok 2024-09-16T09:45:28.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:27 smithi084 ceph-mon[32629]: 5.0 scrub starts 2024-09-16T09:45:28.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:27 smithi084 ceph-mon[32629]: 5.0 scrub ok 2024-09-16T09:45:28.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:27 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:45:28.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:27 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:45:28.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:27 smithi084 ceph-mon[32629]: 3.d scrub starts 2024-09-16T09:45:28.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:27 smithi084 ceph-mon[32629]: 3.d scrub ok 2024-09-16T09:45:28.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:27 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 4, "weight": 1.0}]': finished 2024-09-16T09:45:28.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:27 smithi084 ceph-mon[32629]: osdmap e557: 8 total, 8 up, 8 in 2024-09-16T09:45:28.705 INFO:teuthology.orchestra.run.smithi070.stderr:reweighted osd.5 to 1 (10000) 2024-09-16T09:45:29.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:28 smithi084 ceph-mon[32629]: 4.5 scrub starts 2024-09-16T09:45:29.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:28 smithi084 ceph-mon[32629]: 4.5 scrub ok 2024-09-16T09:45:29.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:28 smithi084 ceph-mon[32629]: pgmap v1085: 162 pgs: 162 active+clean; 609 KiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-16T09:45:29.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:28 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/828086022' entity='client.admin' cmd={"prefix": "osd reweight", "id": 5, "weight": 1.0} : dispatch 2024-09-16T09:45:29.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:28 smithi070 ceph-mon[30388]: 4.5 scrub starts 2024-09-16T09:45:29.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:28 smithi070 ceph-mon[30388]: 4.5 scrub ok 2024-09-16T09:45:29.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:28 smithi070 ceph-mon[30388]: pgmap v1085: 162 pgs: 162 active+clean; 609 KiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-16T09:45:29.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:28 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/828086022' entity='client.admin' cmd={"prefix": "osd reweight", "id": 5, "weight": 1.0} : dispatch 2024-09-16T09:45:29.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:28 smithi070 ceph-mon[40691]: 4.5 scrub starts 2024-09-16T09:45:29.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:28 smithi070 ceph-mon[40691]: 4.5 scrub ok 2024-09-16T09:45:29.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:28 smithi070 ceph-mon[40691]: pgmap v1085: 162 pgs: 162 active+clean; 609 KiB data, 6.5 GiB used, 709 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-16T09:45:29.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:28 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/828086022' entity='client.admin' cmd={"prefix": "osd reweight", "id": 5, "weight": 1.0} : dispatch 2024-09-16T09:45:29.637 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd primary-affinity 5 1 2024-09-16T09:45:29.891 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:45:29.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:29 smithi070 ceph-mon[30388]: 3.18 scrub starts 2024-09-16T09:45:29.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:29 smithi070 ceph-mon[30388]: 3.18 scrub ok 2024-09-16T09:45:29.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:29 smithi070 ceph-mon[30388]: 1.0 scrub starts 2024-09-16T09:45:29.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:29 smithi070 ceph-mon[30388]: 1.0 scrub ok 2024-09-16T09:45:29.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:29 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/828086022' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-16T09:45:29.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:29 smithi070 ceph-mon[30388]: osdmap e558: 8 total, 8 up, 8 in 2024-09-16T09:45:29.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:29 smithi070 ceph-mon[30388]: osdmap e559: 8 total, 8 up, 8 in 2024-09-16T09:45:29.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:29 smithi070 ceph-mon[40691]: 3.18 scrub starts 2024-09-16T09:45:29.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:29 smithi070 ceph-mon[40691]: 3.18 scrub ok 2024-09-16T09:45:29.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:29 smithi070 ceph-mon[40691]: 1.0 scrub starts 2024-09-16T09:45:29.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:29 smithi070 ceph-mon[40691]: 1.0 scrub ok 2024-09-16T09:45:29.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:29 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/828086022' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-16T09:45:29.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:29 smithi070 ceph-mon[40691]: osdmap e558: 8 total, 8 up, 8 in 2024-09-16T09:45:29.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:29 smithi070 ceph-mon[40691]: osdmap e559: 8 total, 8 up, 8 in 2024-09-16T09:45:30.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:29 smithi084 ceph-mon[32629]: 3.18 scrub starts 2024-09-16T09:45:30.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:29 smithi084 ceph-mon[32629]: 3.18 scrub ok 2024-09-16T09:45:30.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:29 smithi084 ceph-mon[32629]: 1.0 scrub starts 2024-09-16T09:45:30.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:29 smithi084 ceph-mon[32629]: 1.0 scrub ok 2024-09-16T09:45:30.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:29 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/828086022' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-16T09:45:30.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:29 smithi084 ceph-mon[32629]: osdmap e558: 8 total, 8 up, 8 in 2024-09-16T09:45:30.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:29 smithi084 ceph-mon[32629]: osdmap e559: 8 total, 8 up, 8 in 2024-09-16T09:45:30.711 INFO:teuthology.orchestra.run.smithi070.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-09-16T09:45:31.012 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:30 smithi070 ceph-mon[30388]: pgmap v1088: 162 pgs: 162 active+clean; 583 KiB data, 6.5 GiB used, 709 GiB / 715 GiB avail 2024-09-16T09:45:31.012 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:30 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3511680089' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 5, "weight": 1.0} : dispatch 2024-09-16T09:45:31.012 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:30 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 5, "weight": 1.0} : dispatch 2024-09-16T09:45:31.012 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:30 smithi070 ceph-mon[40691]: pgmap v1088: 162 pgs: 162 active+clean; 583 KiB data, 6.5 GiB used, 709 GiB / 715 GiB avail 2024-09-16T09:45:31.012 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:30 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3511680089' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 5, "weight": 1.0} : dispatch 2024-09-16T09:45:31.012 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:30 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 5, "weight": 1.0} : dispatch 2024-09-16T09:45:31.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:30 smithi084 ceph-mon[32629]: pgmap v1088: 162 pgs: 162 active+clean; 583 KiB data, 6.5 GiB used, 709 GiB / 715 GiB avail 2024-09-16T09:45:31.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:30 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3511680089' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 5, "weight": 1.0} : dispatch 2024-09-16T09:45:31.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:30 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 5, "weight": 1.0} : dispatch 2024-09-16T09:45:31.475 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd reweight 6 1 2024-09-16T09:45:31.722 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:45:31.975 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:45:31 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:45:31] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:45:31.975 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:31 smithi070 ceph-mon[30388]: 6.9 scrub starts 2024-09-16T09:45:31.975 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:31 smithi070 ceph-mon[30388]: 6.9 scrub ok 2024-09-16T09:45:31.976 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:31 smithi070 ceph-mon[30388]: 4.1c scrub starts 2024-09-16T09:45:31.976 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:31 smithi070 ceph-mon[30388]: 4.1c scrub ok 2024-09-16T09:45:31.976 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:31 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 5, "weight": 1.0}]': finished 2024-09-16T09:45:31.976 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:31 smithi070 ceph-mon[30388]: osdmap e560: 8 total, 8 up, 8 in 2024-09-16T09:45:31.976 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:31 smithi070 ceph-mon[30388]: 4.11 scrub starts 2024-09-16T09:45:31.976 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:31 smithi070 ceph-mon[30388]: 4.11 scrub ok 2024-09-16T09:45:31.976 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:31 smithi070 ceph-mon[30388]: 4.13 scrub starts 2024-09-16T09:45:31.976 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:31 smithi070 ceph-mon[30388]: 4.13 scrub ok 2024-09-16T09:45:31.976 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:31 smithi070 ceph-mon[40691]: 6.9 scrub starts 2024-09-16T09:45:31.976 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:31 smithi070 ceph-mon[40691]: 6.9 scrub ok 2024-09-16T09:45:31.977 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:31 smithi070 ceph-mon[40691]: 4.1c scrub starts 2024-09-16T09:45:31.977 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:31 smithi070 ceph-mon[40691]: 4.1c scrub ok 2024-09-16T09:45:31.977 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:31 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 5, "weight": 1.0}]': finished 2024-09-16T09:45:31.977 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:31 smithi070 ceph-mon[40691]: osdmap e560: 8 total, 8 up, 8 in 2024-09-16T09:45:31.977 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:31 smithi070 ceph-mon[40691]: 4.11 scrub starts 2024-09-16T09:45:31.977 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:31 smithi070 ceph-mon[40691]: 4.11 scrub ok 2024-09-16T09:45:31.977 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:31 smithi070 ceph-mon[40691]: 4.13 scrub starts 2024-09-16T09:45:31.977 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:31 smithi070 ceph-mon[40691]: 4.13 scrub ok 2024-09-16T09:45:32.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:31 smithi084 ceph-mon[32629]: 6.9 scrub starts 2024-09-16T09:45:32.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:31 smithi084 ceph-mon[32629]: 6.9 scrub ok 2024-09-16T09:45:32.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:31 smithi084 ceph-mon[32629]: 4.1c scrub starts 2024-09-16T09:45:32.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:31 smithi084 ceph-mon[32629]: 4.1c scrub ok 2024-09-16T09:45:32.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:31 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 5, "weight": 1.0}]': finished 2024-09-16T09:45:32.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:31 smithi084 ceph-mon[32629]: osdmap e560: 8 total, 8 up, 8 in 2024-09-16T09:45:32.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:31 smithi084 ceph-mon[32629]: 4.11 scrub starts 2024-09-16T09:45:32.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:31 smithi084 ceph-mon[32629]: 4.11 scrub ok 2024-09-16T09:45:32.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:31 smithi084 ceph-mon[32629]: 4.13 scrub starts 2024-09-16T09:45:32.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:31 smithi084 ceph-mon[32629]: 4.13 scrub ok 2024-09-16T09:45:32.718 INFO:teuthology.orchestra.run.smithi070.stderr:reweighted osd.6 to 1 (10000) 2024-09-16T09:45:33.046 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:32 smithi070 ceph-mon[30388]: 3.1d scrub starts 2024-09-16T09:45:33.046 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:32 smithi070 ceph-mon[30388]: 3.1d scrub ok 2024-09-16T09:45:33.046 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:32 smithi070 ceph-mon[30388]: 5.9 scrub starts 2024-09-16T09:45:33.046 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:32 smithi070 ceph-mon[30388]: 5.9 scrub ok 2024-09-16T09:45:33.047 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:32 smithi070 ceph-mon[30388]: 3.10 scrub starts 2024-09-16T09:45:33.047 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:32 smithi070 ceph-mon[30388]: 3.10 scrub ok 2024-09-16T09:45:33.047 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:32 smithi070 ceph-mon[30388]: pgmap v1090: 162 pgs: 162 active+clean; 583 KiB data, 6.5 GiB used, 709 GiB / 715 GiB avail 2024-09-16T09:45:33.047 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:32 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/897386340' entity='client.admin' cmd={"prefix": "osd reweight", "id": 6, "weight": 1.0} : dispatch 2024-09-16T09:45:33.047 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:32 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 6, "weight": 1.0} : dispatch 2024-09-16T09:45:33.047 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:32 smithi070 ceph-mon[40691]: 3.1d scrub starts 2024-09-16T09:45:33.047 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:32 smithi070 ceph-mon[40691]: 3.1d scrub ok 2024-09-16T09:45:33.047 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:32 smithi070 ceph-mon[40691]: 5.9 scrub starts 2024-09-16T09:45:33.047 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:32 smithi070 ceph-mon[40691]: 5.9 scrub ok 2024-09-16T09:45:33.048 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:32 smithi070 ceph-mon[40691]: 3.10 scrub starts 2024-09-16T09:45:33.048 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:32 smithi070 ceph-mon[40691]: 3.10 scrub ok 2024-09-16T09:45:33.048 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:32 smithi070 ceph-mon[40691]: pgmap v1090: 162 pgs: 162 active+clean; 583 KiB data, 6.5 GiB used, 709 GiB / 715 GiB avail 2024-09-16T09:45:33.048 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:32 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/897386340' entity='client.admin' cmd={"prefix": "osd reweight", "id": 6, "weight": 1.0} : dispatch 2024-09-16T09:45:33.048 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:32 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 6, "weight": 1.0} : dispatch 2024-09-16T09:45:33.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:32 smithi084 ceph-mon[32629]: 3.1d scrub starts 2024-09-16T09:45:33.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:32 smithi084 ceph-mon[32629]: 3.1d scrub ok 2024-09-16T09:45:33.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:32 smithi084 ceph-mon[32629]: 5.9 scrub starts 2024-09-16T09:45:33.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:32 smithi084 ceph-mon[32629]: 5.9 scrub ok 2024-09-16T09:45:33.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:32 smithi084 ceph-mon[32629]: 3.10 scrub starts 2024-09-16T09:45:33.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:32 smithi084 ceph-mon[32629]: 3.10 scrub ok 2024-09-16T09:45:33.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:32 smithi084 ceph-mon[32629]: pgmap v1090: 162 pgs: 162 active+clean; 583 KiB data, 6.5 GiB used, 709 GiB / 715 GiB avail 2024-09-16T09:45:33.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:32 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/897386340' entity='client.admin' cmd={"prefix": "osd reweight", "id": 6, "weight": 1.0} : dispatch 2024-09-16T09:45:33.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:32 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 6, "weight": 1.0} : dispatch 2024-09-16T09:45:33.393 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd primary-affinity 6 1 2024-09-16T09:45:33.639 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:45:34.067 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:33 smithi070 ceph-mon[30388]: 1.14 scrub starts 2024-09-16T09:45:34.067 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:33 smithi070 ceph-mon[30388]: 1.14 scrub ok 2024-09-16T09:45:34.067 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:33 smithi070 ceph-mon[30388]: 3.c scrub starts 2024-09-16T09:45:34.068 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:33 smithi070 ceph-mon[30388]: 3.c scrub ok 2024-09-16T09:45:34.068 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:33 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-16T09:45:34.068 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:33 smithi070 ceph-mon[30388]: osdmap e561: 8 total, 8 up, 8 in 2024-09-16T09:45:34.068 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:33 smithi070 ceph-mon[40691]: 1.14 scrub starts 2024-09-16T09:45:34.068 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:33 smithi070 ceph-mon[40691]: 1.14 scrub ok 2024-09-16T09:45:34.068 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:33 smithi070 ceph-mon[40691]: 3.c scrub starts 2024-09-16T09:45:34.068 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:33 smithi070 ceph-mon[40691]: 3.c scrub ok 2024-09-16T09:45:34.068 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:33 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-16T09:45:34.068 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:33 smithi070 ceph-mon[40691]: osdmap e561: 8 total, 8 up, 8 in 2024-09-16T09:45:34.160 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:45:33 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:45:34.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:33 smithi084 ceph-mon[32629]: 1.14 scrub starts 2024-09-16T09:45:34.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:33 smithi084 ceph-mon[32629]: 1.14 scrub ok 2024-09-16T09:45:34.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:33 smithi084 ceph-mon[32629]: 3.c scrub starts 2024-09-16T09:45:34.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:33 smithi084 ceph-mon[32629]: 3.c scrub ok 2024-09-16T09:45:34.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:33 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-16T09:45:34.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:33 smithi084 ceph-mon[32629]: osdmap e561: 8 total, 8 up, 8 in 2024-09-16T09:45:34.719 INFO:teuthology.orchestra.run.smithi070.stderr:set osd.6 primary-affinity to 1 (8655362) 2024-09-16T09:45:35.097 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:34 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:45:35.097 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:34 smithi070 ceph-mon[30388]: pgmap v1092: 162 pgs: 162 active+clean; 583 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail 2024-09-16T09:45:35.097 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:34 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:45:35.098 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:34 smithi070 ceph-mon[30388]: 3.b scrub starts 2024-09-16T09:45:35.098 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:34 smithi070 ceph-mon[30388]: 3.b scrub ok 2024-09-16T09:45:35.098 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:34 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/74961799' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 6, "weight": 1.0} : dispatch 2024-09-16T09:45:35.098 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:34 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 6, "weight": 1.0} : dispatch 2024-09-16T09:45:35.098 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:34 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:45:35.098 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:34 smithi070 ceph-mon[40691]: pgmap v1092: 162 pgs: 162 active+clean; 583 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail 2024-09-16T09:45:35.098 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:34 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:45:35.098 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:34 smithi070 ceph-mon[40691]: 3.b scrub starts 2024-09-16T09:45:35.098 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:34 smithi070 ceph-mon[40691]: 3.b scrub ok 2024-09-16T09:45:35.098 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:34 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/74961799' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 6, "weight": 1.0} : dispatch 2024-09-16T09:45:35.099 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:34 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 6, "weight": 1.0} : dispatch 2024-09-16T09:45:35.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:34 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:45:35.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:34 smithi084 ceph-mon[32629]: pgmap v1092: 162 pgs: 162 active+clean; 583 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail 2024-09-16T09:45:35.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:34 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:45:35.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:34 smithi084 ceph-mon[32629]: 3.b scrub starts 2024-09-16T09:45:35.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:34 smithi084 ceph-mon[32629]: 3.b scrub ok 2024-09-16T09:45:35.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:34 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/74961799' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 6, "weight": 1.0} : dispatch 2024-09-16T09:45:35.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:34 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 6, "weight": 1.0} : dispatch 2024-09-16T09:45:35.484 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd reweight 7 1 2024-09-16T09:45:35.738 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:45:36.006 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:35 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 6, "weight": 1.0}]': finished 2024-09-16T09:45:36.006 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:35 smithi070 ceph-mon[30388]: osdmap e562: 8 total, 8 up, 8 in 2024-09-16T09:45:36.006 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:35 smithi070 ceph-mon[30388]: 4.1d scrub starts 2024-09-16T09:45:36.006 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:35 smithi070 ceph-mon[30388]: 4.1d scrub ok 2024-09-16T09:45:36.007 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:35 smithi070 ceph-mon[30388]: 6.1e scrub starts 2024-09-16T09:45:36.007 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:35 smithi070 ceph-mon[30388]: 6.1e scrub ok 2024-09-16T09:45:36.007 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:35 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 6, "weight": 1.0}]': finished 2024-09-16T09:45:36.007 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:35 smithi070 ceph-mon[40691]: osdmap e562: 8 total, 8 up, 8 in 2024-09-16T09:45:36.007 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:35 smithi070 ceph-mon[40691]: 4.1d scrub starts 2024-09-16T09:45:36.007 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:35 smithi070 ceph-mon[40691]: 4.1d scrub ok 2024-09-16T09:45:36.007 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:35 smithi070 ceph-mon[40691]: 6.1e scrub starts 2024-09-16T09:45:36.007 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:35 smithi070 ceph-mon[40691]: 6.1e scrub ok 2024-09-16T09:45:36.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:35 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 6, "weight": 1.0}]': finished 2024-09-16T09:45:36.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:35 smithi084 ceph-mon[32629]: osdmap e562: 8 total, 8 up, 8 in 2024-09-16T09:45:36.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:35 smithi084 ceph-mon[32629]: 4.1d scrub starts 2024-09-16T09:45:36.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:35 smithi084 ceph-mon[32629]: 4.1d scrub ok 2024-09-16T09:45:36.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:35 smithi084 ceph-mon[32629]: 6.1e scrub starts 2024-09-16T09:45:36.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:35 smithi084 ceph-mon[32629]: 6.1e scrub ok 2024-09-16T09:45:36.732 INFO:teuthology.orchestra.run.smithi070.stderr:reweighted osd.7 to 1 (10000) 2024-09-16T09:45:37.106 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:36 smithi070 ceph-mon[30388]: 1.3 scrub starts 2024-09-16T09:45:37.106 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:36 smithi070 ceph-mon[30388]: 1.3 scrub ok 2024-09-16T09:45:37.107 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:36 smithi070 ceph-mon[30388]: 3.16 scrub starts 2024-09-16T09:45:37.107 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:36 smithi070 ceph-mon[30388]: 3.16 scrub ok 2024-09-16T09:45:37.107 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:36 smithi070 ceph-mon[30388]: pgmap v1094: 162 pgs: 162 active+clean; 583 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:45:37.107 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:36 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:45:37.107 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:36 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:45:37.107 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:36 smithi070 ceph-mon[30388]: 5.a scrub starts 2024-09-16T09:45:37.107 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:36 smithi070 ceph-mon[30388]: 5.a scrub ok 2024-09-16T09:45:37.107 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:36 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:45:37.107 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:36 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:45:37.108 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:36 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/4063226104' entity='client.admin' cmd={"prefix": "osd reweight", "id": 7, "weight": 1.0} : dispatch 2024-09-16T09:45:37.108 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:36 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 7, "weight": 1.0} : dispatch 2024-09-16T09:45:37.108 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:36 smithi070 ceph-mon[40691]: 1.3 scrub starts 2024-09-16T09:45:37.108 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:36 smithi070 ceph-mon[40691]: 1.3 scrub ok 2024-09-16T09:45:37.108 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:36 smithi070 ceph-mon[40691]: 3.16 scrub starts 2024-09-16T09:45:37.108 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:36 smithi070 ceph-mon[40691]: 3.16 scrub ok 2024-09-16T09:45:37.108 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:36 smithi070 ceph-mon[40691]: pgmap v1094: 162 pgs: 162 active+clean; 583 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:45:37.109 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:36 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:45:37.109 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:36 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:45:37.109 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:36 smithi070 ceph-mon[40691]: 5.a scrub starts 2024-09-16T09:45:37.109 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:36 smithi070 ceph-mon[40691]: 5.a scrub ok 2024-09-16T09:45:37.109 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:36 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:45:37.109 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:36 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:45:37.109 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:36 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/4063226104' entity='client.admin' cmd={"prefix": "osd reweight", "id": 7, "weight": 1.0} : dispatch 2024-09-16T09:45:37.109 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:36 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 7, "weight": 1.0} : dispatch 2024-09-16T09:45:37.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:36 smithi084 ceph-mon[32629]: 1.3 scrub starts 2024-09-16T09:45:37.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:36 smithi084 ceph-mon[32629]: 1.3 scrub ok 2024-09-16T09:45:37.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:36 smithi084 ceph-mon[32629]: 3.16 scrub starts 2024-09-16T09:45:37.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:36 smithi084 ceph-mon[32629]: 3.16 scrub ok 2024-09-16T09:45:37.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:36 smithi084 ceph-mon[32629]: pgmap v1094: 162 pgs: 162 active+clean; 583 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:45:37.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:36 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:45:37.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:36 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:45:37.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:36 smithi084 ceph-mon[32629]: 5.a scrub starts 2024-09-16T09:45:37.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:36 smithi084 ceph-mon[32629]: 5.a scrub ok 2024-09-16T09:45:37.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:36 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:45:37.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:36 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:45:37.162 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:36 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/4063226104' entity='client.admin' cmd={"prefix": "osd reweight", "id": 7, "weight": 1.0} : dispatch 2024-09-16T09:45:37.162 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:36 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd={"prefix": "osd reweight", "id": 7, "weight": 1.0} : dispatch 2024-09-16T09:45:37.596 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd primary-affinity 7 1 2024-09-16T09:45:37.846 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:45:37.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:37 smithi070 ceph-mon[30388]: 3.11 scrub starts 2024-09-16T09:45:37.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:37 smithi070 ceph-mon[30388]: 3.11 scrub ok 2024-09-16T09:45:37.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:37 smithi070 ceph-mon[30388]: 4.1f scrub starts 2024-09-16T09:45:37.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:37 smithi070 ceph-mon[30388]: 4.1f scrub ok 2024-09-16T09:45:37.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:37 smithi070 ceph-mon[30388]: 5.7 scrub starts 2024-09-16T09:45:37.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:37 smithi070 ceph-mon[30388]: 5.7 scrub ok 2024-09-16T09:45:37.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:37 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-16T09:45:37.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:37 smithi070 ceph-mon[30388]: osdmap e563: 8 total, 8 up, 8 in 2024-09-16T09:45:37.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:37 smithi070 ceph-mon[30388]: 4.e scrub starts 2024-09-16T09:45:37.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:37 smithi070 ceph-mon[30388]: 4.e scrub ok 2024-09-16T09:45:37.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:37 smithi070 ceph-mon[40691]: 3.11 scrub starts 2024-09-16T09:45:37.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:37 smithi070 ceph-mon[40691]: 3.11 scrub ok 2024-09-16T09:45:37.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:37 smithi070 ceph-mon[40691]: 4.1f scrub starts 2024-09-16T09:45:37.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:37 smithi070 ceph-mon[40691]: 4.1f scrub ok 2024-09-16T09:45:37.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:37 smithi070 ceph-mon[40691]: 5.7 scrub starts 2024-09-16T09:45:37.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:37 smithi070 ceph-mon[40691]: 5.7 scrub ok 2024-09-16T09:45:37.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:37 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-16T09:45:37.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:37 smithi070 ceph-mon[40691]: osdmap e563: 8 total, 8 up, 8 in 2024-09-16T09:45:37.923 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:37 smithi070 ceph-mon[40691]: 4.e scrub starts 2024-09-16T09:45:37.923 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:37 smithi070 ceph-mon[40691]: 4.e scrub ok 2024-09-16T09:45:38.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:37 smithi084 ceph-mon[32629]: 3.11 scrub starts 2024-09-16T09:45:38.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:37 smithi084 ceph-mon[32629]: 3.11 scrub ok 2024-09-16T09:45:38.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:37 smithi084 ceph-mon[32629]: 4.1f scrub starts 2024-09-16T09:45:38.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:37 smithi084 ceph-mon[32629]: 4.1f scrub ok 2024-09-16T09:45:38.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:37 smithi084 ceph-mon[32629]: 5.7 scrub starts 2024-09-16T09:45:38.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:37 smithi084 ceph-mon[32629]: 5.7 scrub ok 2024-09-16T09:45:38.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:37 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-16T09:45:38.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:37 smithi084 ceph-mon[32629]: osdmap e563: 8 total, 8 up, 8 in 2024-09-16T09:45:38.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:37 smithi084 ceph-mon[32629]: 4.e scrub starts 2024-09-16T09:45:38.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:37 smithi084 ceph-mon[32629]: 4.e scrub ok 2024-09-16T09:45:38.730 INFO:teuthology.orchestra.run.smithi070.stderr:set osd.7 primary-affinity to 1 (8655362) 2024-09-16T09:45:39.115 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:38 smithi070 ceph-mon[30388]: 6.16 scrub starts 2024-09-16T09:45:39.115 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:38 smithi070 ceph-mon[30388]: 6.16 scrub ok 2024-09-16T09:45:39.115 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:38 smithi070 ceph-mon[30388]: 6.4 scrub starts 2024-09-16T09:45:39.115 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:38 smithi070 ceph-mon[30388]: 6.4 scrub ok 2024-09-16T09:45:39.115 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:38 smithi070 ceph-mon[30388]: pgmap v1096: 162 pgs: 162 active+clean; 583 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-16T09:45:39.116 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:38 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/520208190' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 7, "weight": 1.0} : dispatch 2024-09-16T09:45:39.116 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:38 smithi070 ceph-mon[40691]: 6.16 scrub starts 2024-09-16T09:45:39.116 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:38 smithi070 ceph-mon[40691]: 6.16 scrub ok 2024-09-16T09:45:39.116 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:38 smithi070 ceph-mon[40691]: 6.4 scrub starts 2024-09-16T09:45:39.116 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:38 smithi070 ceph-mon[40691]: 6.4 scrub ok 2024-09-16T09:45:39.116 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:38 smithi070 ceph-mon[40691]: pgmap v1096: 162 pgs: 162 active+clean; 583 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-16T09:45:39.116 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:38 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/520208190' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 7, "weight": 1.0} : dispatch 2024-09-16T09:45:39.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:38 smithi084 ceph-mon[32629]: 6.16 scrub starts 2024-09-16T09:45:39.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:38 smithi084 ceph-mon[32629]: 6.16 scrub ok 2024-09-16T09:45:39.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:38 smithi084 ceph-mon[32629]: 6.4 scrub starts 2024-09-16T09:45:39.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:38 smithi084 ceph-mon[32629]: 6.4 scrub ok 2024-09-16T09:45:39.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:38 smithi084 ceph-mon[32629]: pgmap v1096: 162 pgs: 162 active+clean; 583 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-16T09:45:39.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:38 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/520208190' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 7, "weight": 1.0} : dispatch 2024-09-16T09:45:39.589 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd reweight 2 1 2024-09-16T09:45:39.842 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:45:39.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:39 smithi070 ceph-mon[40691]: 4.3 scrub starts 2024-09-16T09:45:39.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:39 smithi070 ceph-mon[40691]: 4.3 scrub ok 2024-09-16T09:45:39.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:39 smithi070 ceph-mon[40691]: 2.2 scrub starts 2024-09-16T09:45:39.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:39 smithi070 ceph-mon[40691]: 2.2 scrub ok 2024-09-16T09:45:39.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:39 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/520208190' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 7, "weight": 1.0}]': finished 2024-09-16T09:45:39.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:39 smithi070 ceph-mon[40691]: osdmap e564: 8 total, 8 up, 8 in 2024-09-16T09:45:39.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:39 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:45:39.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:39 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:45:39.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:39 smithi070 ceph-mon[40691]: osdmap e565: 8 total, 8 up, 8 in 2024-09-16T09:45:39.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:39 smithi070 ceph-mon[30388]: 4.3 scrub starts 2024-09-16T09:45:39.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:39 smithi070 ceph-mon[30388]: 4.3 scrub ok 2024-09-16T09:45:39.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:39 smithi070 ceph-mon[30388]: 2.2 scrub starts 2024-09-16T09:45:39.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:39 smithi070 ceph-mon[30388]: 2.2 scrub ok 2024-09-16T09:45:39.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:39 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/520208190' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 7, "weight": 1.0}]': finished 2024-09-16T09:45:39.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:39 smithi070 ceph-mon[30388]: osdmap e564: 8 total, 8 up, 8 in 2024-09-16T09:45:39.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:39 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:45:39.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:39 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:45:39.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:39 smithi070 ceph-mon[30388]: osdmap e565: 8 total, 8 up, 8 in 2024-09-16T09:45:40.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:39 smithi084 ceph-mon[32629]: 4.3 scrub starts 2024-09-16T09:45:40.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:39 smithi084 ceph-mon[32629]: 4.3 scrub ok 2024-09-16T09:45:40.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:39 smithi084 ceph-mon[32629]: 2.2 scrub starts 2024-09-16T09:45:40.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:39 smithi084 ceph-mon[32629]: 2.2 scrub ok 2024-09-16T09:45:40.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:39 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/520208190' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 7, "weight": 1.0}]': finished 2024-09-16T09:45:40.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:39 smithi084 ceph-mon[32629]: osdmap e564: 8 total, 8 up, 8 in 2024-09-16T09:45:40.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:39 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:45:40.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:39 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:45:40.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:39 smithi084 ceph-mon[32629]: osdmap e565: 8 total, 8 up, 8 in 2024-09-16T09:45:40.739 INFO:teuthology.orchestra.run.smithi070.stderr:reweighted osd.2 to 1 (10000) 2024-09-16T09:45:41.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:40 smithi084 ceph-mon[32629]: 5.10 scrub starts 2024-09-16T09:45:41.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:40 smithi084 ceph-mon[32629]: 5.10 scrub ok 2024-09-16T09:45:41.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:40 smithi084 ceph-mon[32629]: 6.d scrub starts 2024-09-16T09:45:41.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:40 smithi084 ceph-mon[32629]: 6.d scrub ok 2024-09-16T09:45:41.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:40 smithi084 ceph-mon[32629]: pgmap v1099: 162 pgs: 162 active+clean; 583 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 980 B/s rd, 0 op/s 2024-09-16T09:45:41.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:40 smithi084 ceph-mon[32629]: 4.1 scrub starts 2024-09-16T09:45:41.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:40 smithi084 ceph-mon[32629]: 4.1 scrub ok 2024-09-16T09:45:41.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:40 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/2791427676' entity='client.admin' cmd={"prefix": "osd reweight", "id": 2, "weight": 1.0} : dispatch 2024-09-16T09:45:41.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:40 smithi070 ceph-mon[30388]: 5.10 scrub starts 2024-09-16T09:45:41.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:40 smithi070 ceph-mon[30388]: 5.10 scrub ok 2024-09-16T09:45:41.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:40 smithi070 ceph-mon[30388]: 6.d scrub starts 2024-09-16T09:45:41.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:40 smithi070 ceph-mon[30388]: 6.d scrub ok 2024-09-16T09:45:41.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:40 smithi070 ceph-mon[30388]: pgmap v1099: 162 pgs: 162 active+clean; 583 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 980 B/s rd, 0 op/s 2024-09-16T09:45:41.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:40 smithi070 ceph-mon[30388]: 4.1 scrub starts 2024-09-16T09:45:41.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:40 smithi070 ceph-mon[30388]: 4.1 scrub ok 2024-09-16T09:45:41.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:40 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2791427676' entity='client.admin' cmd={"prefix": "osd reweight", "id": 2, "weight": 1.0} : dispatch 2024-09-16T09:45:41.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:40 smithi070 ceph-mon[40691]: 5.10 scrub starts 2024-09-16T09:45:41.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:40 smithi070 ceph-mon[40691]: 5.10 scrub ok 2024-09-16T09:45:41.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:40 smithi070 ceph-mon[40691]: 6.d scrub starts 2024-09-16T09:45:41.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:40 smithi070 ceph-mon[40691]: 6.d scrub ok 2024-09-16T09:45:41.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:40 smithi070 ceph-mon[40691]: pgmap v1099: 162 pgs: 162 active+clean; 583 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 980 B/s rd, 0 op/s 2024-09-16T09:45:41.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:40 smithi070 ceph-mon[40691]: 4.1 scrub starts 2024-09-16T09:45:41.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:40 smithi070 ceph-mon[40691]: 4.1 scrub ok 2024-09-16T09:45:41.172 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:40 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/2791427676' entity='client.admin' cmd={"prefix": "osd reweight", "id": 2, "weight": 1.0} : dispatch 2024-09-16T09:45:41.696 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd primary-affinity 2 1 2024-09-16T09:45:41.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:41 smithi070 ceph-mon[30388]: 4.16 scrub starts 2024-09-16T09:45:41.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:41 smithi070 ceph-mon[30388]: 4.16 scrub ok 2024-09-16T09:45:41.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:41 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2791427676' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-16T09:45:41.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:41 smithi070 ceph-mon[30388]: osdmap e566: 8 total, 8 up, 8 in 2024-09-16T09:45:41.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:41 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:45:41.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:41 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:45:41.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:41 smithi070 ceph-mon[30388]: 4.1a scrub starts 2024-09-16T09:45:41.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:41 smithi070 ceph-mon[30388]: 4.1a scrub ok 2024-09-16T09:45:41.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:41 smithi070 ceph-mon[40691]: 4.16 scrub starts 2024-09-16T09:45:41.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:41 smithi070 ceph-mon[40691]: 4.16 scrub ok 2024-09-16T09:45:41.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:41 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/2791427676' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-16T09:45:41.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:41 smithi070 ceph-mon[40691]: osdmap e566: 8 total, 8 up, 8 in 2024-09-16T09:45:41.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:41 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:45:41.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:41 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:45:41.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:41 smithi070 ceph-mon[40691]: 4.1a scrub starts 2024-09-16T09:45:41.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:41 smithi070 ceph-mon[40691]: 4.1a scrub ok 2024-09-16T09:45:41.949 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:45:42.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:41 smithi084 ceph-mon[32629]: 4.16 scrub starts 2024-09-16T09:45:42.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:41 smithi084 ceph-mon[32629]: 4.16 scrub ok 2024-09-16T09:45:42.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:41 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/2791427676' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-16T09:45:42.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:41 smithi084 ceph-mon[32629]: osdmap e566: 8 total, 8 up, 8 in 2024-09-16T09:45:42.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:41 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:45:42.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:41 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:45:42.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:41 smithi084 ceph-mon[32629]: 4.1a scrub starts 2024-09-16T09:45:42.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:41 smithi084 ceph-mon[32629]: 4.1a scrub ok 2024-09-16T09:45:42.213 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:45:41 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:45:41] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:45:42.757 INFO:teuthology.orchestra.run.smithi070.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-09-16T09:45:43.066 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:42 smithi070 ceph-mon[30388]: 3.13 scrub starts 2024-09-16T09:45:43.067 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:42 smithi070 ceph-mon[30388]: 3.13 scrub ok 2024-09-16T09:45:43.067 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:42 smithi070 ceph-mon[30388]: 5.1d scrub starts 2024-09-16T09:45:43.067 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:42 smithi070 ceph-mon[30388]: 5.1d scrub ok 2024-09-16T09:45:43.067 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:42 smithi070 ceph-mon[30388]: osdmap e567: 8 total, 8 up, 8 in 2024-09-16T09:45:43.067 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:42 smithi070 ceph-mon[30388]: 5.1b scrub starts 2024-09-16T09:45:43.067 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:42 smithi070 ceph-mon[30388]: 5.1b scrub ok 2024-09-16T09:45:43.067 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:42 smithi070 ceph-mon[30388]: pgmap v1102: 162 pgs: 162 active+clean; 583 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail 2024-09-16T09:45:43.067 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:42 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2646218351' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 2, "weight": 1.0} : dispatch 2024-09-16T09:45:43.067 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:42 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 2, "weight": 1.0} : dispatch 2024-09-16T09:45:43.067 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:42 smithi070 ceph-mon[40691]: 3.13 scrub starts 2024-09-16T09:45:43.067 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:42 smithi070 ceph-mon[40691]: 3.13 scrub ok 2024-09-16T09:45:43.068 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:42 smithi070 ceph-mon[40691]: 5.1d scrub starts 2024-09-16T09:45:43.068 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:42 smithi070 ceph-mon[40691]: 5.1d scrub ok 2024-09-16T09:45:43.068 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:42 smithi070 ceph-mon[40691]: osdmap e567: 8 total, 8 up, 8 in 2024-09-16T09:45:43.068 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:42 smithi070 ceph-mon[40691]: 5.1b scrub starts 2024-09-16T09:45:43.068 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:42 smithi070 ceph-mon[40691]: 5.1b scrub ok 2024-09-16T09:45:43.068 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:42 smithi070 ceph-mon[40691]: pgmap v1102: 162 pgs: 162 active+clean; 583 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail 2024-09-16T09:45:43.068 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:42 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/2646218351' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 2, "weight": 1.0} : dispatch 2024-09-16T09:45:43.068 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:42 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 2, "weight": 1.0} : dispatch 2024-09-16T09:45:43.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:42 smithi084 ceph-mon[32629]: 3.13 scrub starts 2024-09-16T09:45:43.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:42 smithi084 ceph-mon[32629]: 3.13 scrub ok 2024-09-16T09:45:43.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:42 smithi084 ceph-mon[32629]: 5.1d scrub starts 2024-09-16T09:45:43.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:42 smithi084 ceph-mon[32629]: 5.1d scrub ok 2024-09-16T09:45:43.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:42 smithi084 ceph-mon[32629]: osdmap e567: 8 total, 8 up, 8 in 2024-09-16T09:45:43.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:42 smithi084 ceph-mon[32629]: 5.1b scrub starts 2024-09-16T09:45:43.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:42 smithi084 ceph-mon[32629]: 5.1b scrub ok 2024-09-16T09:45:43.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:42 smithi084 ceph-mon[32629]: pgmap v1102: 162 pgs: 162 active+clean; 583 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail 2024-09-16T09:45:43.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:42 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/2646218351' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 2, "weight": 1.0} : dispatch 2024-09-16T09:45:43.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:42 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd={"prefix": "osd primary-affinity", "id": 2, "weight": 1.0} : dispatch 2024-09-16T09:45:43.361 INFO:tasks.thrashosds.thrasher:joining the do_sighup greenlet 2024-09-16T09:45:43.361 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2024-09-16T09:45:43.362 INFO:tasks.thrashosds.thrasher:joining the do_dump_ops greenlet 2024-09-16T09:45:43.362 INFO:tasks.thrashosds.thrasher:joining the do_noscrub_toggle greenlet 2024-09-16T09:45:43.362 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-16T09:45:43.363 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T09:45:43.611 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:45:44.039 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:43 smithi070 ceph-mon[30388]: 5.6 scrub starts 2024-09-16T09:45:44.039 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:43 smithi070 ceph-mon[30388]: 5.6 scrub ok 2024-09-16T09:45:44.040 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:43 smithi070 ceph-mon[30388]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 2, "weight": 1.0}]': finished 2024-09-16T09:45:44.040 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:43 smithi070 ceph-mon[30388]: osdmap e568: 8 total, 8 up, 8 in 2024-09-16T09:45:44.040 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:43 smithi070 ceph-mon[30388]: 4.8 scrub starts 2024-09-16T09:45:44.040 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:43 smithi070 ceph-mon[30388]: 4.8 scrub ok 2024-09-16T09:45:44.040 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:43 smithi070 ceph-mon[30388]: 4.17 scrub starts 2024-09-16T09:45:44.040 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:43 smithi070 ceph-mon[30388]: 4.17 scrub ok 2024-09-16T09:45:44.041 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:43 smithi070 ceph-mon[30388]: osdmap e569: 8 total, 8 up, 8 in 2024-09-16T09:45:44.041 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:43 smithi070 ceph-mon[40691]: 5.6 scrub starts 2024-09-16T09:45:44.041 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:43 smithi070 ceph-mon[40691]: 5.6 scrub ok 2024-09-16T09:45:44.041 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:43 smithi070 ceph-mon[40691]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 2, "weight": 1.0}]': finished 2024-09-16T09:45:44.042 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:43 smithi070 ceph-mon[40691]: osdmap e568: 8 total, 8 up, 8 in 2024-09-16T09:45:44.042 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:43 smithi070 ceph-mon[40691]: 4.8 scrub starts 2024-09-16T09:45:44.042 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:43 smithi070 ceph-mon[40691]: 4.8 scrub ok 2024-09-16T09:45:44.042 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:43 smithi070 ceph-mon[40691]: 4.17 scrub starts 2024-09-16T09:45:44.042 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:43 smithi070 ceph-mon[40691]: 4.17 scrub ok 2024-09-16T09:45:44.042 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:43 smithi070 ceph-mon[40691]: osdmap e569: 8 total, 8 up, 8 in 2024-09-16T09:45:44.160 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:45:43 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:45:44.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:43 smithi084 ceph-mon[32629]: 5.6 scrub starts 2024-09-16T09:45:44.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:43 smithi084 ceph-mon[32629]: 5.6 scrub ok 2024-09-16T09:45:44.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:43 smithi084 ceph-mon[32629]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 2, "weight": 1.0}]': finished 2024-09-16T09:45:44.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:43 smithi084 ceph-mon[32629]: osdmap e568: 8 total, 8 up, 8 in 2024-09-16T09:45:44.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:43 smithi084 ceph-mon[32629]: 4.8 scrub starts 2024-09-16T09:45:44.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:43 smithi084 ceph-mon[32629]: 4.8 scrub ok 2024-09-16T09:45:44.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:43 smithi084 ceph-mon[32629]: 4.17 scrub starts 2024-09-16T09:45:44.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:43 smithi084 ceph-mon[32629]: 4.17 scrub ok 2024-09-16T09:45:44.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:43 smithi084 ceph-mon[32629]: osdmap e569: 8 total, 8 up, 8 in 2024-09-16T09:45:44.308 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:45:44.308 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":569,"fsid":"a1321834-740c-11ef-bceb-c7b262605968","created":"2024-09-16T09:19:14.121498+0000","modified":"2024-09-16T09:45:43.752967+0000","last_up_change":"2024-09-16T09:44:23.253857+0000","last_in_change":"2024-09-16T09:24:28.432920+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":19,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T09:22:27.617694+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":31,"pg_placement_num":31,"pg_placement_num_target":31,"pg_num_target":31,"pg_num_pending":31,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"546","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"542","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.059999942779541,"score_stable":2.059999942779541,"optimal_score":1,"raw_score_acting":2.059999942779541,"raw_score_stable":2.059999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"datapool","create_time":"2024-09-16T09:24:59.169244+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":61,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.6500000953674316,"score_stable":2.6500000953674316,"optimal_score":0.87999999523162842,"raw_score_acting":2.3299999237060547,"raw_score_stable":2.3299999237060547,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":".rgw.root","create_time":"2024-09-16T09:25:00.056119+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-09-16T09:25:01.770016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-09-16T09:25:03.725555+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-09-16T09:25:05.773530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7ef288c8-3bf0-4814-8e86-f70a7de041ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":544,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":2013075781},{"type":"v1","addr":"172.21.15.70:6803","nonce":2013075781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":2013075781},{"type":"v1","addr":"172.21.15.70:6805","nonce":2013075781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":2013075781},{"type":"v1","addr":"172.21.15.70:6809","nonce":2013075781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":2013075781},{"type":"v1","addr":"172.21.15.70:6807","nonce":2013075781}]},"public_addr":"172.21.15.70:6803/2013075781","cluster_addr":"172.21.15.70:6805/2013075781","heartbeat_back_addr":"172.21.15.70:6809/2013075781","heartbeat_front_addr":"172.21.15.70:6807/2013075781","state":["exists","up"]},{"osd":1,"uuid":"474d41d9-295a-450c-8f27-ce487a14678b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":551,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":1670012115},{"type":"v1","addr":"172.21.15.70:6811","nonce":1670012115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":1670012115},{"type":"v1","addr":"172.21.15.70:6813","nonce":1670012115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1670012115},{"type":"v1","addr":"172.21.15.70:6817","nonce":1670012115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":1670012115},{"type":"v1","addr":"172.21.15.70:6815","nonce":1670012115}]},"public_addr":"172.21.15.70:6811/1670012115","cluster_addr":"172.21.15.70:6813/1670012115","heartbeat_back_addr":"172.21.15.70:6817/1670012115","heartbeat_front_addr":"172.21.15.70:6815/1670012115","state":["exists","up"]},{"osd":2,"uuid":"5b4ca4e8-cecc-4163-b291-c9209b02c4b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":18,"last_clean_end":494,"up_from":516,"up_thru":568,"down_at":495,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3826536127},{"type":"v1","addr":"172.21.15.70:6819","nonce":3826536127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":3826536127},{"type":"v1","addr":"172.21.15.70:6821","nonce":3826536127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":3826536127},{"type":"v1","addr":"172.21.15.70:6825","nonce":3826536127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3826536127},{"type":"v1","addr":"172.21.15.70:6823","nonce":3826536127}]},"public_addr":"172.21.15.70:6819/3826536127","cluster_addr":"172.21.15.70:6821/3826536127","heartbeat_back_addr":"172.21.15.70:6825/3826536127","heartbeat_front_addr":"172.21.15.70:6823/3826536127","state":["exists","up"]},{"osd":3,"uuid":"ffb8d527-1b58-4ecd-808c-47d4ba1bf387","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":554,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2038083007},{"type":"v1","addr":"172.21.15.70:6827","nonce":2038083007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2038083007},{"type":"v1","addr":"172.21.15.70:6829","nonce":2038083007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":2038083007},{"type":"v1","addr":"172.21.15.70:6833","nonce":2038083007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2038083007},{"type":"v1","addr":"172.21.15.70:6831","nonce":2038083007}]},"public_addr":"172.21.15.70:6827/2038083007","cluster_addr":"172.21.15.70:6829/2038083007","heartbeat_back_addr":"172.21.15.70:6833/2038083007","heartbeat_front_addr":"172.21.15.70:6831/2038083007","state":["exists","up"]},{"osd":4,"uuid":"a946760b-4d91-4a4b-8e9b-a07a50589211","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":544,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2250892710},{"type":"v1","addr":"172.21.15.84:6801","nonce":2250892710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2250892710},{"type":"v1","addr":"172.21.15.84:6803","nonce":2250892710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2250892710},{"type":"v1","addr":"172.21.15.84:6807","nonce":2250892710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2250892710},{"type":"v1","addr":"172.21.15.84:6805","nonce":2250892710}]},"public_addr":"172.21.15.84:6801/2250892710","cluster_addr":"172.21.15.84:6803/2250892710","heartbeat_back_addr":"172.21.15.84:6807/2250892710","heartbeat_front_addr":"172.21.15.84:6805/2250892710","state":["exists","up"]},{"osd":5,"uuid":"67681760-75a9-4846-946e-a935d42c5e05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":544,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":1803945348},{"type":"v1","addr":"172.21.15.84:6809","nonce":1803945348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":1803945348},{"type":"v1","addr":"172.21.15.84:6811","nonce":1803945348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":1803945348},{"type":"v1","addr":"172.21.15.84:6815","nonce":1803945348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":1803945348},{"type":"v1","addr":"172.21.15.84:6813","nonce":1803945348}]},"public_addr":"172.21.15.84:6809/1803945348","cluster_addr":"172.21.15.84:6811/1803945348","heartbeat_back_addr":"172.21.15.84:6815/1803945348","heartbeat_front_addr":"172.21.15.84:6813/1803945348","state":["exists","up"]},{"osd":6,"uuid":"7c8f0152-f539-4b7e-8113-91dd7529d3a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":542,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":610727189},{"type":"v1","addr":"172.21.15.84:6817","nonce":610727189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":610727189},{"type":"v1","addr":"172.21.15.84:6819","nonce":610727189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":610727189},{"type":"v1","addr":"172.21.15.84:6823","nonce":610727189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":610727189},{"type":"v1","addr":"172.21.15.84:6821","nonce":610727189}]},"public_addr":"172.21.15.84:6817/610727189","cluster_addr":"172.21.15.84:6819/610727189","heartbeat_back_addr":"172.21.15.84:6823/610727189","heartbeat_front_addr":"172.21.15.84:6821/610727189","state":["exists","up"]},{"osd":7,"uuid":"945efaba-225e-49af-aefd-b4d58ca8093b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":544,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":1002252531},{"type":"v1","addr":"172.21.15.84:6825","nonce":1002252531}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1002252531},{"type":"v1","addr":"172.21.15.84:6827","nonce":1002252531}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":1002252531},{"type":"v1","addr":"172.21.15.84:6831","nonce":1002252531}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":1002252531},{"type":"v1","addr":"172.21.15.84:6829","nonce":1002252531}]},"public_addr":"172.21.15.84:6825/1002252531","cluster_addr":"172.21.15.84:6827/1002252531","heartbeat_back_addr":"172.21.15.84:6831/1002252531","heartbeat_front_addr":"172.21.15.84:6829/1002252531","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T09:45:08.644968+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T09:45:40.938697+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T09:45:26.400653+0000","dead_epoch":494},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T09:45:39.126542+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T09:45:36.077945+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T09:44:54.883057+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T09:45:36.370101+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T09:44:52.836155+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"3.1d","osds":[1,0,7]},{"pgid":"4.b","osds":[0,5,1]},{"pgid":"4.19","osds":[4,7,5]},{"pgid":"5.1a","osds":[6,7,4]}],"pg_upmap_items":[{"pgid":"3.1b","mappings":[{"from":0,"to":6},{"from":4,"to":7}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":5}]},{"pgid":"4.17","mappings":[{"from":1,"to":2}]},{"pgid":"5.10","mappings":[{"from":7,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.70:0/125491368":"2024-09-17T09:25:31.465167+0000","172.21.15.70:0/3354284852":"2024-09-17T09:19:43.205989+0000","172.21.15.70:0/3284625547":"2024-09-17T09:25:31.465167+0000","172.21.15.70:0/186565817":"2024-09-17T09:19:43.205989+0000","172.21.15.70:0/2213852865":"2024-09-17T09:19:43.205989+0000","172.21.15.70:0/1812373760":"2024-09-17T09:20:09.306330+0000","172.21.15.70:6800/837338094":"2024-09-17T09:19:43.205989+0000","172.21.15.70:6800/1876661229":"2024-09-17T09:25:31.465167+0000","172.21.15.70:0/3351688485":"2024-09-17T09:20:09.306330+0000","172.21.15.70:6800/3871553358":"2024-09-17T09:20:09.306330+0000","172.21.15.70:0/1230482871":"2024-09-17T09:25:31.465167+0000","172.21.15.70:6801/837338094":"2024-09-17T09:19:43.205989+0000","172.21.15.70:6801/3871553358":"2024-09-17T09:20:09.306330+0000","172.21.15.70:0/1403001486":"2024-09-17T09:20:09.306330+0000","172.21.15.70:0/1723643739":"2024-09-17T09:25:31.465167+0000","172.21.15.70:6801/1876661229":"2024-09-17T09:25:31.465167+0000","172.21.15.70:0/3456417396":"2024-09-17T09:25:31.465167+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T09:45:44.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:44 smithi070 ceph-mon[30388]: 2.0 scrub starts 2024-09-16T09:45:44.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:44 smithi070 ceph-mon[30388]: 2.0 scrub ok 2024-09-16T09:45:44.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:44 smithi070 ceph-mon[30388]: 6.1 scrub starts 2024-09-16T09:45:44.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:44 smithi070 ceph-mon[30388]: 6.1 scrub ok 2024-09-16T09:45:44.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:44 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:45:44.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:44 smithi070 ceph-mon[30388]: 3.5 scrub starts 2024-09-16T09:45:44.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:44 smithi070 ceph-mon[30388]: 3.5 scrub ok 2024-09-16T09:45:44.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:44 smithi070 ceph-mon[30388]: pgmap v1105: 162 pgs: 162 active+clean; 583 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:45:44.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:44 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3548671530' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-16T09:45:44.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:44 smithi070 ceph-mon[40691]: 2.0 scrub starts 2024-09-16T09:45:44.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:44 smithi070 ceph-mon[40691]: 2.0 scrub ok 2024-09-16T09:45:44.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:44 smithi070 ceph-mon[40691]: 6.1 scrub starts 2024-09-16T09:45:44.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:44 smithi070 ceph-mon[40691]: 6.1 scrub ok 2024-09-16T09:45:44.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:44 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:45:44.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:44 smithi070 ceph-mon[40691]: 3.5 scrub starts 2024-09-16T09:45:44.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:44 smithi070 ceph-mon[40691]: 3.5 scrub ok 2024-09-16T09:45:44.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:44 smithi070 ceph-mon[40691]: pgmap v1105: 162 pgs: 162 active+clean; 583 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:45:44.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:44 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3548671530' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-16T09:45:45.056 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-16T09:45:45.057 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T09:45:45.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:44 smithi084 ceph-mon[32629]: 2.0 scrub starts 2024-09-16T09:45:45.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:44 smithi084 ceph-mon[32629]: 2.0 scrub ok 2024-09-16T09:45:45.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:44 smithi084 ceph-mon[32629]: 6.1 scrub starts 2024-09-16T09:45:45.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:44 smithi084 ceph-mon[32629]: 6.1 scrub ok 2024-09-16T09:45:45.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:44 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:45:45.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:44 smithi084 ceph-mon[32629]: 3.5 scrub starts 2024-09-16T09:45:45.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:44 smithi084 ceph-mon[32629]: 3.5 scrub ok 2024-09-16T09:45:45.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:44 smithi084 ceph-mon[32629]: pgmap v1105: 162 pgs: 162 active+clean; 583 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-16T09:45:45.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:44 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3548671530' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-16T09:45:45.304 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:45:45.978 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:45:45.978 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:45 smithi070 ceph-mon[30388]: 5.d scrub starts 2024-09-16T09:45:45.978 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:45 smithi070 ceph-mon[30388]: 5.d scrub ok 2024-09-16T09:45:45.979 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:45 smithi070 ceph-mon[40691]: 5.d scrub starts 2024-09-16T09:45:45.979 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:45 smithi070 ceph-mon[40691]: 5.d scrub ok 2024-09-16T09:45:45.979 INFO:teuthology.orchestra.run.smithi070.stdout:{"epoch":569,"fsid":"a1321834-740c-11ef-bceb-c7b262605968","created":"2024-09-16T09:19:14.121498+0000","modified":"2024-09-16T09:45:43.752967+0000","last_up_change":"2024-09-16T09:44:23.253857+0000","last_in_change":"2024-09-16T09:24:28.432920+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":19,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T09:22:27.617694+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":31,"pg_placement_num":31,"pg_placement_num_target":31,"pg_num_target":31,"pg_num_pending":31,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"546","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"542","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.059999942779541,"score_stable":2.059999942779541,"optimal_score":1,"raw_score_acting":2.059999942779541,"raw_score_stable":2.059999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"datapool","create_time":"2024-09-16T09:24:59.169244+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":61,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.6500000953674316,"score_stable":2.6500000953674316,"optimal_score":0.87999999523162842,"raw_score_acting":2.3299999237060547,"raw_score_stable":2.3299999237060547,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":".rgw.root","create_time":"2024-09-16T09:25:00.056119+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-09-16T09:25:01.770016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-09-16T09:25:03.725555+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-09-16T09:25:05.773530+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7ef288c8-3bf0-4814-8e86-f70a7de041ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":544,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6802","nonce":2013075781},{"type":"v1","addr":"172.21.15.70:6803","nonce":2013075781}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6804","nonce":2013075781},{"type":"v1","addr":"172.21.15.70:6805","nonce":2013075781}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6808","nonce":2013075781},{"type":"v1","addr":"172.21.15.70:6809","nonce":2013075781}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6806","nonce":2013075781},{"type":"v1","addr":"172.21.15.70:6807","nonce":2013075781}]},"public_addr":"172.21.15.70:6803/2013075781","cluster_addr":"172.21.15.70:6805/2013075781","heartbeat_back_addr":"172.21.15.70:6809/2013075781","heartbeat_front_addr":"172.21.15.70:6807/2013075781","state":["exists","up"]},{"osd":1,"uuid":"474d41d9-295a-450c-8f27-ce487a14678b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":551,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6810","nonce":1670012115},{"type":"v1","addr":"172.21.15.70:6811","nonce":1670012115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6812","nonce":1670012115},{"type":"v1","addr":"172.21.15.70:6813","nonce":1670012115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6816","nonce":1670012115},{"type":"v1","addr":"172.21.15.70:6817","nonce":1670012115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6814","nonce":1670012115},{"type":"v1","addr":"172.21.15.70:6815","nonce":1670012115}]},"public_addr":"172.21.15.70:6811/1670012115","cluster_addr":"172.21.15.70:6813/1670012115","heartbeat_back_addr":"172.21.15.70:6817/1670012115","heartbeat_front_addr":"172.21.15.70:6815/1670012115","state":["exists","up"]},{"osd":2,"uuid":"5b4ca4e8-cecc-4163-b291-c9209b02c4b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":18,"last_clean_end":494,"up_from":516,"up_thru":568,"down_at":495,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6818","nonce":3826536127},{"type":"v1","addr":"172.21.15.70:6819","nonce":3826536127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6820","nonce":3826536127},{"type":"v1","addr":"172.21.15.70:6821","nonce":3826536127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6824","nonce":3826536127},{"type":"v1","addr":"172.21.15.70:6825","nonce":3826536127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6822","nonce":3826536127},{"type":"v1","addr":"172.21.15.70:6823","nonce":3826536127}]},"public_addr":"172.21.15.70:6819/3826536127","cluster_addr":"172.21.15.70:6821/3826536127","heartbeat_back_addr":"172.21.15.70:6825/3826536127","heartbeat_front_addr":"172.21.15.70:6823/3826536127","state":["exists","up"]},{"osd":3,"uuid":"ffb8d527-1b58-4ecd-808c-47d4ba1bf387","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":554,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6826","nonce":2038083007},{"type":"v1","addr":"172.21.15.70:6827","nonce":2038083007}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6828","nonce":2038083007},{"type":"v1","addr":"172.21.15.70:6829","nonce":2038083007}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6832","nonce":2038083007},{"type":"v1","addr":"172.21.15.70:6833","nonce":2038083007}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.70:6830","nonce":2038083007},{"type":"v1","addr":"172.21.15.70:6831","nonce":2038083007}]},"public_addr":"172.21.15.70:6827/2038083007","cluster_addr":"172.21.15.70:6829/2038083007","heartbeat_back_addr":"172.21.15.70:6833/2038083007","heartbeat_front_addr":"172.21.15.70:6831/2038083007","state":["exists","up"]},{"osd":4,"uuid":"a946760b-4d91-4a4b-8e9b-a07a50589211","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":544,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2250892710},{"type":"v1","addr":"172.21.15.84:6801","nonce":2250892710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":2250892710},{"type":"v1","addr":"172.21.15.84:6803","nonce":2250892710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":2250892710},{"type":"v1","addr":"172.21.15.84:6807","nonce":2250892710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":2250892710},{"type":"v1","addr":"172.21.15.84:6805","nonce":2250892710}]},"public_addr":"172.21.15.84:6801/2250892710","cluster_addr":"172.21.15.84:6803/2250892710","heartbeat_back_addr":"172.21.15.84:6807/2250892710","heartbeat_front_addr":"172.21.15.84:6805/2250892710","state":["exists","up"]},{"osd":5,"uuid":"67681760-75a9-4846-946e-a935d42c5e05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":544,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":1803945348},{"type":"v1","addr":"172.21.15.84:6809","nonce":1803945348}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":1803945348},{"type":"v1","addr":"172.21.15.84:6811","nonce":1803945348}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":1803945348},{"type":"v1","addr":"172.21.15.84:6815","nonce":1803945348}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":1803945348},{"type":"v1","addr":"172.21.15.84:6813","nonce":1803945348}]},"public_addr":"172.21.15.84:6809/1803945348","cluster_addr":"172.21.15.84:6811/1803945348","heartbeat_back_addr":"172.21.15.84:6815/1803945348","heartbeat_front_addr":"172.21.15.84:6813/1803945348","state":["exists","up"]},{"osd":6,"uuid":"7c8f0152-f539-4b7e-8113-91dd7529d3a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":542,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":610727189},{"type":"v1","addr":"172.21.15.84:6817","nonce":610727189}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":610727189},{"type":"v1","addr":"172.21.15.84:6819","nonce":610727189}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":610727189},{"type":"v1","addr":"172.21.15.84:6823","nonce":610727189}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":610727189},{"type":"v1","addr":"172.21.15.84:6821","nonce":610727189}]},"public_addr":"172.21.15.84:6817/610727189","cluster_addr":"172.21.15.84:6819/610727189","heartbeat_back_addr":"172.21.15.84:6823/610727189","heartbeat_front_addr":"172.21.15.84:6821/610727189","state":["exists","up"]},{"osd":7,"uuid":"945efaba-225e-49af-aefd-b4d58ca8093b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":53,"up_thru":544,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":1002252531},{"type":"v1","addr":"172.21.15.84:6825","nonce":1002252531}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":1002252531},{"type":"v1","addr":"172.21.15.84:6827","nonce":1002252531}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":1002252531},{"type":"v1","addr":"172.21.15.84:6831","nonce":1002252531}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":1002252531},{"type":"v1","addr":"172.21.15.84:6829","nonce":1002252531}]},"public_addr":"172.21.15.84:6825/1002252531","cluster_addr":"172.21.15.84:6827/1002252531","heartbeat_back_addr":"172.21.15.84:6831/1002252531","heartbeat_front_addr":"172.21.15.84:6829/1002252531","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T09:45:08.644968+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T09:45:40.938697+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T09:45:26.400653+0000","dead_epoch":494},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T09:45:39.126542+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T09:45:36.077945+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T09:44:54.883057+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T09:45:36.370101+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T09:44:52.836155+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"3.1d","osds":[1,0,7]},{"pgid":"4.b","osds":[0,5,1]},{"pgid":"4.19","osds":[4,7,5]},{"pgid":"5.1a","osds":[6,7,4]}],"pg_upmap_items":[{"pgid":"3.1b","mappings":[{"from":0,"to":6},{"from":4,"to":7}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":5}]},{"pgid":"4.17","mappings":[{"from":1,"to":2}]},{"pgid":"5.10","mappings":[{"from":7,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.70:0/125491368":"2024-09-17T09:25:31.465167+0000","172.21.15.70:0/3354284852":"2024-09-17T09:19:43.205989+0000","172.21.15.70:0/3284625547":"2024-09-17T09:25:31.465167+0000","172.21.15.70:0/186565817":"2024-09-17T09:19:43.205989+0000","172.21.15.70:0/2213852865":"2024-09-17T09:19:43.205989+0000","172.21.15.70:0/1812373760":"2024-09-17T09:20:09.306330+0000","172.21.15.70:6800/837338094":"2024-09-17T09:19:43.205989+0000","172.21.15.70:6800/1876661229":"2024-09-17T09:25:31.465167+0000","172.21.15.70:0/3351688485":"2024-09-17T09:20:09.306330+0000","172.21.15.70:6800/3871553358":"2024-09-17T09:20:09.306330+0000","172.21.15.70:0/1230482871":"2024-09-17T09:25:31.465167+0000","172.21.15.70:6801/837338094":"2024-09-17T09:19:43.205989+0000","172.21.15.70:6801/3871553358":"2024-09-17T09:20:09.306330+0000","172.21.15.70:0/1403001486":"2024-09-17T09:20:09.306330+0000","172.21.15.70:0/1723643739":"2024-09-17T09:25:31.465167+0000","172.21.15.70:6801/1876661229":"2024-09-17T09:25:31.465167+0000","172.21.15.70:0/3456417396":"2024-09-17T09:25:31.465167+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T09:45:46.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:45 smithi084 ceph-mon[32629]: 5.d scrub starts 2024-09-16T09:45:46.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:45 smithi084 ceph-mon[32629]: 5.d scrub ok 2024-09-16T09:45:46.731 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-16T09:45:46.731 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-16T09:45:46.732 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-16T09:45:46.732 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-16T09:45:46.733 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-16T09:45:46.733 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-16T09:45:46.734 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-16T09:45:46.734 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-16T09:45:47.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:46 smithi084 ceph-mon[32629]: 3.6 scrub starts 2024-09-16T09:45:47.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:46 smithi084 ceph-mon[32629]: 3.6 scrub ok 2024-09-16T09:45:47.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:46 smithi084 ceph-mon[32629]: 6.f scrub starts 2024-09-16T09:45:47.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:46 smithi084 ceph-mon[32629]: 6.f scrub ok 2024-09-16T09:45:47.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:46 smithi084 ceph-mon[32629]: 1.13 scrub starts 2024-09-16T09:45:47.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:46 smithi084 ceph-mon[32629]: 1.13 scrub ok 2024-09-16T09:45:47.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:46 smithi084 ceph-mon[32629]: 5.15 scrub starts 2024-09-16T09:45:47.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:46 smithi084 ceph-mon[32629]: 5.15 scrub ok 2024-09-16T09:45:47.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:46 smithi084 ceph-mon[32629]: pgmap v1106: 162 pgs: 162 active+clean; 583 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 984 B/s rd, 0 op/s 2024-09-16T09:45:47.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:46 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/4105490755' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-16T09:45:47.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:46 smithi084 ceph-mon[32629]: 4.19 scrub starts 2024-09-16T09:45:47.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:46 smithi084 ceph-mon[32629]: 4.19 scrub ok 2024-09-16T09:45:47.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:46 smithi070 ceph-mon[30388]: 3.6 scrub starts 2024-09-16T09:45:47.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:46 smithi070 ceph-mon[30388]: 3.6 scrub ok 2024-09-16T09:45:47.172 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:46 smithi070 ceph-mon[30388]: 6.f scrub starts 2024-09-16T09:45:47.172 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:46 smithi070 ceph-mon[30388]: 6.f scrub ok 2024-09-16T09:45:47.172 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:46 smithi070 ceph-mon[30388]: 1.13 scrub starts 2024-09-16T09:45:47.172 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:46 smithi070 ceph-mon[30388]: 1.13 scrub ok 2024-09-16T09:45:47.172 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:46 smithi070 ceph-mon[30388]: 5.15 scrub starts 2024-09-16T09:45:47.172 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:46 smithi070 ceph-mon[30388]: 5.15 scrub ok 2024-09-16T09:45:47.172 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:46 smithi070 ceph-mon[30388]: pgmap v1106: 162 pgs: 162 active+clean; 583 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 984 B/s rd, 0 op/s 2024-09-16T09:45:47.172 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:46 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/4105490755' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-16T09:45:47.173 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:46 smithi070 ceph-mon[30388]: 4.19 scrub starts 2024-09-16T09:45:47.173 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:46 smithi070 ceph-mon[30388]: 4.19 scrub ok 2024-09-16T09:45:47.173 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:46 smithi070 ceph-mon[40691]: 3.6 scrub starts 2024-09-16T09:45:47.173 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:46 smithi070 ceph-mon[40691]: 3.6 scrub ok 2024-09-16T09:45:47.173 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:46 smithi070 ceph-mon[40691]: 6.f scrub starts 2024-09-16T09:45:47.173 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:46 smithi070 ceph-mon[40691]: 6.f scrub ok 2024-09-16T09:45:47.173 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:46 smithi070 ceph-mon[40691]: 1.13 scrub starts 2024-09-16T09:45:47.173 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:46 smithi070 ceph-mon[40691]: 1.13 scrub ok 2024-09-16T09:45:47.173 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:46 smithi070 ceph-mon[40691]: 5.15 scrub starts 2024-09-16T09:45:47.173 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:46 smithi070 ceph-mon[40691]: 5.15 scrub ok 2024-09-16T09:45:47.174 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:46 smithi070 ceph-mon[40691]: pgmap v1106: 162 pgs: 162 active+clean; 583 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 984 B/s rd, 0 op/s 2024-09-16T09:45:47.174 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:46 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/4105490755' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-16T09:45:47.174 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:46 smithi070 ceph-mon[40691]: 4.19 scrub starts 2024-09-16T09:45:47.174 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:46 smithi070 ceph-mon[40691]: 4.19 scrub ok 2024-09-16T09:45:47.264 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:45:47.265 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:45:47.267 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:45:47.268 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:45:47.268 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:45:47.268 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:45:47.274 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:45:47.279 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:45:48.047 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:47 smithi070 ceph-mon[30388]: 4.4 scrub starts 2024-09-16T09:45:48.048 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:47 smithi070 ceph-mon[30388]: 4.4 scrub ok 2024-09-16T09:45:48.048 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:47 smithi070 ceph-mon[30388]: 3.15 scrub starts 2024-09-16T09:45:48.048 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:47 smithi070 ceph-mon[30388]: 3.15 scrub ok 2024-09-16T09:45:48.048 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:47 smithi070 ceph-mon[30388]: 6.a scrub starts 2024-09-16T09:45:48.048 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:47 smithi070 ceph-mon[30388]: 6.a scrub ok 2024-09-16T09:45:48.048 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:47 smithi070 ceph-mon[30388]: 4.c scrub starts 2024-09-16T09:45:48.048 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:47 smithi070 ceph-mon[30388]: 4.c scrub ok 2024-09-16T09:45:48.048 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:47 smithi070 ceph-mon[40691]: 4.4 scrub starts 2024-09-16T09:45:48.049 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:47 smithi070 ceph-mon[40691]: 4.4 scrub ok 2024-09-16T09:45:48.049 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:47 smithi070 ceph-mon[40691]: 3.15 scrub starts 2024-09-16T09:45:48.049 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:47 smithi070 ceph-mon[40691]: 3.15 scrub ok 2024-09-16T09:45:48.049 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:47 smithi070 ceph-mon[40691]: 6.a scrub starts 2024-09-16T09:45:48.049 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:47 smithi070 ceph-mon[40691]: 6.a scrub ok 2024-09-16T09:45:48.049 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:47 smithi070 ceph-mon[40691]: 4.c scrub starts 2024-09-16T09:45:48.049 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:47 smithi070 ceph-mon[40691]: 4.c scrub ok 2024-09-16T09:45:48.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:47 smithi084 ceph-mon[32629]: 4.4 scrub starts 2024-09-16T09:45:48.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:47 smithi084 ceph-mon[32629]: 4.4 scrub ok 2024-09-16T09:45:48.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:47 smithi084 ceph-mon[32629]: 3.15 scrub starts 2024-09-16T09:45:48.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:47 smithi084 ceph-mon[32629]: 3.15 scrub ok 2024-09-16T09:45:48.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:47 smithi084 ceph-mon[32629]: 6.a scrub starts 2024-09-16T09:45:48.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:47 smithi084 ceph-mon[32629]: 6.a scrub ok 2024-09-16T09:45:48.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:47 smithi084 ceph-mon[32629]: 4.c scrub starts 2024-09-16T09:45:48.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:47 smithi084 ceph-mon[32629]: 4.c scrub ok 2024-09-16T09:45:48.896 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:48 smithi070 ceph-mon[30388]: 3.0 scrub starts 2024-09-16T09:45:48.896 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:48 smithi070 ceph-mon[30388]: 3.0 scrub ok 2024-09-16T09:45:48.896 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:48 smithi070 ceph-mon[30388]: pgmap v1107: 162 pgs: 162 active+clean; 583 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 1.8 KiB/s rd, 1 op/s 2024-09-16T09:45:48.896 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:48 smithi070 ceph-mon[30388]: 4.18 scrub starts 2024-09-16T09:45:48.897 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:48 smithi070 ceph-mon[30388]: 4.18 scrub ok 2024-09-16T09:45:48.897 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:48 smithi070 ceph-mon[40691]: 3.0 scrub starts 2024-09-16T09:45:48.897 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:48 smithi070 ceph-mon[40691]: 3.0 scrub ok 2024-09-16T09:45:48.897 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:48 smithi070 ceph-mon[40691]: pgmap v1107: 162 pgs: 162 active+clean; 583 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 1.8 KiB/s rd, 1 op/s 2024-09-16T09:45:48.897 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:48 smithi070 ceph-mon[40691]: 4.18 scrub starts 2024-09-16T09:45:48.897 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:48 smithi070 ceph-mon[40691]: 4.18 scrub ok 2024-09-16T09:45:49.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:48 smithi084 ceph-mon[32629]: 3.0 scrub starts 2024-09-16T09:45:49.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:48 smithi084 ceph-mon[32629]: 3.0 scrub ok 2024-09-16T09:45:49.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:48 smithi084 ceph-mon[32629]: pgmap v1107: 162 pgs: 162 active+clean; 583 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 1.8 KiB/s rd, 1 op/s 2024-09-16T09:45:49.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:48 smithi084 ceph-mon[32629]: 4.18 scrub starts 2024-09-16T09:45:49.161 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:48 smithi084 ceph-mon[32629]: 4.18 scrub ok 2024-09-16T09:45:50.104 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:49 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:45:50.104 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:49 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:45:50.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:49 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:45:51.102 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:50 smithi070 ceph-mon[30388]: 4.12 scrub starts 2024-09-16T09:45:51.102 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:50 smithi070 ceph-mon[30388]: 4.12 scrub ok 2024-09-16T09:45:51.102 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:50 smithi070 ceph-mon[30388]: pgmap v1108: 162 pgs: 162 active+clean; 583 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 1.6 KiB/s rd, 1 op/s 2024-09-16T09:45:51.103 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:50 smithi070 ceph-mon[40691]: 4.12 scrub starts 2024-09-16T09:45:51.103 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:50 smithi070 ceph-mon[40691]: 4.12 scrub ok 2024-09-16T09:45:51.103 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:50 smithi070 ceph-mon[40691]: pgmap v1108: 162 pgs: 162 active+clean; 583 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 1.6 KiB/s rd, 1 op/s 2024-09-16T09:45:51.131 INFO:teuthology.orchestra.run.smithi070.stdout:227633266943 2024-09-16T09:45:51.132 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-16T09:45:51.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:50 smithi084 ceph-mon[32629]: 4.12 scrub starts 2024-09-16T09:45:51.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:50 smithi084 ceph-mon[32629]: 4.12 scrub ok 2024-09-16T09:45:51.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:50 smithi084 ceph-mon[32629]: pgmap v1108: 162 pgs: 162 active+clean; 583 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 1.6 KiB/s rd, 1 op/s 2024-09-16T09:45:52.094 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:51 smithi070 ceph-mon[30388]: 3.4 scrub starts 2024-09-16T09:45:52.095 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:51 smithi070 ceph-mon[30388]: 3.4 scrub ok 2024-09-16T09:45:52.095 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:45:51 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:45:51] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:45:52.095 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:51 smithi070 ceph-mon[40691]: 3.4 scrub starts 2024-09-16T09:45:52.095 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:51 smithi070 ceph-mon[40691]: 3.4 scrub ok 2024-09-16T09:45:52.134 INFO:teuthology.orchestra.run.smithi070.stdout:2216203124755 2024-09-16T09:45:52.135 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-16T09:45:52.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:51 smithi084 ceph-mon[32629]: 3.4 scrub starts 2024-09-16T09:45:52.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:51 smithi084 ceph-mon[32629]: 3.4 scrub ok 2024-09-16T09:45:53.061 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:52 smithi084 ceph-mon[32629]: 3.1 scrub starts 2024-09-16T09:45:53.061 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:52 smithi084 ceph-mon[32629]: 3.1 scrub ok 2024-09-16T09:45:53.062 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:52 smithi084 ceph-mon[32629]: pgmap v1109: 162 pgs: 162 active+clean; 583 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 891 B/s rd, 0 op/s 2024-09-16T09:45:53.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:52 smithi070 ceph-mon[40691]: 3.1 scrub starts 2024-09-16T09:45:53.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:52 smithi070 ceph-mon[40691]: 3.1 scrub ok 2024-09-16T09:45:53.171 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:52 smithi070 ceph-mon[40691]: pgmap v1109: 162 pgs: 162 active+clean; 583 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 891 B/s rd, 0 op/s 2024-09-16T09:45:53.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:52 smithi070 ceph-mon[30388]: 3.1 scrub starts 2024-09-16T09:45:53.171 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:52 smithi070 ceph-mon[30388]: 3.1 scrub ok 2024-09-16T09:45:53.172 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:52 smithi070 ceph-mon[30388]: pgmap v1109: 162 pgs: 162 active+clean; 583 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 891 B/s rd, 0 op/s 2024-09-16T09:45:53.527 INFO:teuthology.orchestra.run.smithi070.stdout:34359738663 2024-09-16T09:45:53.527 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-16T09:45:54.094 INFO:teuthology.orchestra.run.smithi070.stdout:137438953743 2024-09-16T09:45:54.095 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-16T09:45:54.116 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:53 smithi070 ceph-mon[30388]: 3.17 scrub starts 2024-09-16T09:45:54.116 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:53 smithi070 ceph-mon[30388]: 3.17 scrub ok 2024-09-16T09:45:54.116 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:53 smithi070 ceph-mon[40691]: 3.17 scrub starts 2024-09-16T09:45:54.117 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:53 smithi070 ceph-mon[40691]: 3.17 scrub ok 2024-09-16T09:45:54.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:53 smithi084 ceph-mon[32629]: 3.17 scrub starts 2024-09-16T09:45:54.160 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:53 smithi084 ceph-mon[32629]: 3.17 scrub ok 2024-09-16T09:45:54.160 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:45:53 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:45:54.879 INFO:teuthology.orchestra.run.smithi070.stdout:111669149974 2024-09-16T09:45:54.879 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-16T09:45:55.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:55 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:45:55.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:55 smithi070 ceph-mon[30388]: pgmap v1110: 162 pgs: 162 active+clean; 583 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-16T09:45:55.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:55 smithi070 ceph-mon[30388]: 6.2 scrub starts 2024-09-16T09:45:55.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:55 smithi070 ceph-mon[30388]: 6.2 scrub ok 2024-09-16T09:45:55.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:55 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:45:55.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:55 smithi070 ceph-mon[40691]: pgmap v1110: 162 pgs: 162 active+clean; 583 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-16T09:45:55.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:55 smithi070 ceph-mon[40691]: 6.2 scrub starts 2024-09-16T09:45:55.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:55 smithi070 ceph-mon[40691]: 6.2 scrub ok 2024-09-16T09:45:55.825 INFO:teuthology.orchestra.run.smithi070.stdout:167503724812 2024-09-16T09:45:55.825 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-16T09:45:55.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:55 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:45:55.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:55 smithi084 ceph-mon[32629]: pgmap v1110: 162 pgs: 162 active+clean; 583 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-16T09:45:55.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:55 smithi084 ceph-mon[32629]: 6.2 scrub starts 2024-09-16T09:45:55.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:55 smithi084 ceph-mon[32629]: 6.2 scrub ok 2024-09-16T09:45:56.352 INFO:teuthology.orchestra.run.smithi070.stdout:55834575138 2024-09-16T09:45:56.352 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-16T09:45:56.670 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:56 smithi070 ceph-mon[30388]: 4.0 scrub starts 2024-09-16T09:45:56.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:56 smithi070 ceph-mon[30388]: 4.0 scrub ok 2024-09-16T09:45:56.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:56 smithi070 ceph-mon[30388]: 6.18 scrub starts 2024-09-16T09:45:56.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:56 smithi070 ceph-mon[30388]: 6.18 scrub ok 2024-09-16T09:45:56.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:56 smithi070 ceph-mon[30388]: 1.2 scrub starts 2024-09-16T09:45:56.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:56 smithi070 ceph-mon[30388]: 1.2 scrub ok 2024-09-16T09:45:56.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:56 smithi070 ceph-mon[40691]: 4.0 scrub starts 2024-09-16T09:45:56.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:56 smithi070 ceph-mon[40691]: 4.0 scrub ok 2024-09-16T09:45:56.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:56 smithi070 ceph-mon[40691]: 6.18 scrub starts 2024-09-16T09:45:56.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:56 smithi070 ceph-mon[40691]: 6.18 scrub ok 2024-09-16T09:45:56.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:56 smithi070 ceph-mon[40691]: 1.2 scrub starts 2024-09-16T09:45:56.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:56 smithi070 ceph-mon[40691]: 1.2 scrub ok 2024-09-16T09:45:56.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:56 smithi084 ceph-mon[32629]: 4.0 scrub starts 2024-09-16T09:45:56.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:56 smithi084 ceph-mon[32629]: 4.0 scrub ok 2024-09-16T09:45:56.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:56 smithi084 ceph-mon[32629]: 6.18 scrub starts 2024-09-16T09:45:56.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:56 smithi084 ceph-mon[32629]: 6.18 scrub ok 2024-09-16T09:45:56.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:56 smithi084 ceph-mon[32629]: 1.2 scrub starts 2024-09-16T09:45:56.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:56 smithi084 ceph-mon[32629]: 1.2 scrub ok 2024-09-16T09:45:57.090 INFO:teuthology.orchestra.run.smithi070.stdout:197568495877 2024-09-16T09:45:57.090 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-16T09:45:57.226 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:45:57.238 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:45:57.264 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:45:57.272 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:45:57.623 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:57 smithi070 ceph-mon[40691]: pgmap v1111: 162 pgs: 162 active+clean; 583 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 938 B/s rd, 0 op/s 2024-09-16T09:45:57.623 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:57 smithi070 ceph-mon[40691]: 3.7 scrub starts 2024-09-16T09:45:57.623 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:57 smithi070 ceph-mon[40691]: 3.7 scrub ok 2024-09-16T09:45:57.623 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:57 smithi070 ceph-mon[40691]: 3.14 scrub starts 2024-09-16T09:45:57.623 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:57 smithi070 ceph-mon[40691]: 3.14 scrub ok 2024-09-16T09:45:57.623 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:57 smithi070 ceph-mon[30388]: pgmap v1111: 162 pgs: 162 active+clean; 583 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 938 B/s rd, 0 op/s 2024-09-16T09:45:57.624 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:57 smithi070 ceph-mon[30388]: 3.7 scrub starts 2024-09-16T09:45:57.624 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:57 smithi070 ceph-mon[30388]: 3.7 scrub ok 2024-09-16T09:45:57.624 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:57 smithi070 ceph-mon[30388]: 3.14 scrub starts 2024-09-16T09:45:57.624 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:57 smithi070 ceph-mon[30388]: 3.14 scrub ok 2024-09-16T09:45:57.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:57 smithi084 ceph-mon[32629]: pgmap v1111: 162 pgs: 162 active+clean; 583 KiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 938 B/s rd, 0 op/s 2024-09-16T09:45:57.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:57 smithi084 ceph-mon[32629]: 3.7 scrub starts 2024-09-16T09:45:57.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:57 smithi084 ceph-mon[32629]: 3.7 scrub ok 2024-09-16T09:45:57.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:57 smithi084 ceph-mon[32629]: 3.14 scrub starts 2024-09-16T09:45:57.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:57 smithi084 ceph-mon[32629]: 3.14 scrub ok 2024-09-16T09:45:58.464 INFO:teuthology.orchestra.run.smithi070.stdout:167503724812 2024-09-16T09:45:58.550 INFO:teuthology.orchestra.run.smithi070.stdout:227633266945 2024-09-16T09:45:58.550 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:58 smithi070 ceph-mon[30388]: 1.4 scrub starts 2024-09-16T09:45:58.551 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:58 smithi070 ceph-mon[30388]: 1.4 scrub ok 2024-09-16T09:45:58.551 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:58 smithi070 ceph-mon[30388]: 3.1a scrub starts 2024-09-16T09:45:58.551 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:58 smithi070 ceph-mon[30388]: 3.1a scrub ok 2024-09-16T09:45:58.551 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:58 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/908606151' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-16T09:45:58.551 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:58 smithi070 ceph-mon[40691]: 1.4 scrub starts 2024-09-16T09:45:58.551 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:58 smithi070 ceph-mon[40691]: 1.4 scrub ok 2024-09-16T09:45:58.551 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:58 smithi070 ceph-mon[40691]: 3.1a scrub starts 2024-09-16T09:45:58.551 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:58 smithi070 ceph-mon[40691]: 3.1a scrub ok 2024-09-16T09:45:58.551 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:58 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/908606151' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-16T09:45:58.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:58 smithi084 ceph-mon[32629]: 1.4 scrub starts 2024-09-16T09:45:58.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:58 smithi084 ceph-mon[32629]: 1.4 scrub ok 2024-09-16T09:45:58.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:58 smithi084 ceph-mon[32629]: 3.1a scrub starts 2024-09-16T09:45:58.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:58 smithi084 ceph-mon[32629]: 3.1a scrub ok 2024-09-16T09:45:58.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:58 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/908606151' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-16T09:45:59.259 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724812 got 167503724812 for osd.5 2024-09-16T09:45:59.259 DEBUG:teuthology.parallel:result is None 2024-09-16T09:45:59.809 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:59 smithi070 ceph-mon[30388]: 5.11 scrub starts 2024-09-16T09:45:59.809 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:59 smithi070 ceph-mon[30388]: 5.11 scrub ok 2024-09-16T09:45:59.809 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:59 smithi070 ceph-mon[30388]: pgmap v1112: 162 pgs: 162 active+clean; 583 KiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-16T09:45:59.809 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:59 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1728892242' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-16T09:45:59.809 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:59 smithi070 ceph-mon[30388]: 1.1 scrub starts 2024-09-16T09:45:59.810 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:45:59 smithi070 ceph-mon[30388]: 1.1 scrub ok 2024-09-16T09:45:59.810 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:59 smithi070 ceph-mon[40691]: 5.11 scrub starts 2024-09-16T09:45:59.810 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:59 smithi070 ceph-mon[40691]: 5.11 scrub ok 2024-09-16T09:45:59.810 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:59 smithi070 ceph-mon[40691]: pgmap v1112: 162 pgs: 162 active+clean; 583 KiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-16T09:45:59.810 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:59 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1728892242' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-16T09:45:59.810 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:59 smithi070 ceph-mon[40691]: 1.1 scrub starts 2024-09-16T09:45:59.810 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:45:59 smithi070 ceph-mon[40691]: 1.1 scrub ok 2024-09-16T09:45:59.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:59 smithi084 ceph-mon[32629]: 5.11 scrub starts 2024-09-16T09:45:59.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:59 smithi084 ceph-mon[32629]: 5.11 scrub ok 2024-09-16T09:45:59.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:59 smithi084 ceph-mon[32629]: pgmap v1112: 162 pgs: 162 active+clean; 583 KiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-16T09:45:59.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:59 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1728892242' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-16T09:45:59.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:59 smithi084 ceph-mon[32629]: 1.1 scrub starts 2024-09-16T09:45:59.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:45:59 smithi084 ceph-mon[32629]: 1.1 scrub ok 2024-09-16T09:45:59.918 INFO:teuthology.orchestra.run.smithi070.stdout:137438953745 2024-09-16T09:45:59.959 INFO:teuthology.orchestra.run.smithi070.stdout:111669149975 2024-09-16T09:46:00.093 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:46:00.094 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:46:00.094 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:46:00.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:46:00 smithi070 ceph-mon[30388]: 3.1e scrub starts 2024-09-16T09:46:00.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:46:00 smithi070 ceph-mon[30388]: 3.1e scrub ok 2024-09-16T09:46:00.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:46:00 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1097319370' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-16T09:46:00.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:46:00 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2112089261' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-16T09:46:00.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:46:00 smithi070 ceph-mon[40691]: 3.1e scrub starts 2024-09-16T09:46:00.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:46:00 smithi070 ceph-mon[40691]: 3.1e scrub ok 2024-09-16T09:46:00.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:46:00 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1097319370' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-16T09:46:00.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:46:00 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/2112089261' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-16T09:46:00.828 INFO:tasks.cephadm.ceph_manager.ceph:need seq 227633266943 got 227633266945 for osd.7 2024-09-16T09:46:00.828 DEBUG:teuthology.parallel:result is None 2024-09-16T09:46:00.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:46:00 smithi084 ceph-mon[32629]: 3.1e scrub starts 2024-09-16T09:46:00.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:46:00 smithi084 ceph-mon[32629]: 3.1e scrub ok 2024-09-16T09:46:00.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:46:00 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1097319370' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-16T09:46:00.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:46:00 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/2112089261' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-16T09:46:01.536 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149974 got 111669149975 for osd.3 2024-09-16T09:46:01.536 DEBUG:teuthology.parallel:result is None 2024-09-16T09:46:01.761 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:46:01 smithi070 ceph-mon[30388]: pgmap v1113: 162 pgs: 162 active+clean; 583 KiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:46:01.761 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:46:01 smithi070 ceph-mon[30388]: 5.1f scrub starts 2024-09-16T09:46:01.761 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:46:01 smithi070 ceph-mon[30388]: 5.1f scrub ok 2024-09-16T09:46:01.762 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:46:01 smithi070 ceph-mon[30388]: 6.1a scrub starts 2024-09-16T09:46:01.762 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:46:01 smithi070 ceph-mon[30388]: 6.1a scrub ok 2024-09-16T09:46:01.762 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:46:01 smithi070 ceph-mon[40691]: pgmap v1113: 162 pgs: 162 active+clean; 583 KiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:46:01.762 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:46:01 smithi070 ceph-mon[40691]: 5.1f scrub starts 2024-09-16T09:46:01.762 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:46:01 smithi070 ceph-mon[40691]: 5.1f scrub ok 2024-09-16T09:46:01.762 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:46:01 smithi070 ceph-mon[40691]: 6.1a scrub starts 2024-09-16T09:46:01.762 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:46:01 smithi070 ceph-mon[40691]: 6.1a scrub ok 2024-09-16T09:46:01.773 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:46:01.912 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:46:01 smithi084 ceph-mon[32629]: pgmap v1113: 162 pgs: 162 active+clean; 583 KiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:46:01.913 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:46:01 smithi084 ceph-mon[32629]: 5.1f scrub starts 2024-09-16T09:46:01.913 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:46:01 smithi084 ceph-mon[32629]: 5.1f scrub ok 2024-09-16T09:46:01.913 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:46:01 smithi084 ceph-mon[32629]: 6.1a scrub starts 2024-09-16T09:46:01.913 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:46:01 smithi084 ceph-mon[32629]: 6.1a scrub ok 2024-09-16T09:46:02.125 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:46:01 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:46:01] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:46:02.170 INFO:teuthology.orchestra.run.smithi070.stdout:2216203124757 2024-09-16T09:46:02.784 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:46:02 smithi070 ceph-mon[30388]: 3.2 scrub starts 2024-09-16T09:46:02.784 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:46:02 smithi070 ceph-mon[30388]: 3.2 scrub ok 2024-09-16T09:46:02.785 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:46:02 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1558817442' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-16T09:46:02.785 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:46:02 smithi070 ceph-mon[40691]: 3.2 scrub starts 2024-09-16T09:46:02.785 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:46:02 smithi070 ceph-mon[40691]: 3.2 scrub ok 2024-09-16T09:46:02.785 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:46:02 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1558817442' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-16T09:46:02.876 INFO:teuthology.orchestra.run.smithi070.stdout:197568495878 2024-09-16T09:46:02.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:46:02 smithi084 ceph-mon[32629]: 3.2 scrub starts 2024-09-16T09:46:02.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:46:02 smithi084 ceph-mon[32629]: 3.2 scrub ok 2024-09-16T09:46:02.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:46:02 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1558817442' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-16T09:46:03.113 INFO:tasks.cephadm.ceph_manager.ceph:need seq 2216203124755 got 2216203124757 for osd.2 2024-09-16T09:46:03.114 DEBUG:teuthology.parallel:result is None 2024-09-16T09:46:03.690 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:46:03 smithi070 ceph-mon[30388]: pgmap v1114: 162 pgs: 162 active+clean; 583 KiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-09-16T09:46:03.691 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:46:03 smithi070 ceph-mon[30388]: 5.3 scrub starts 2024-09-16T09:46:03.691 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:46:03 smithi070 ceph-mon[30388]: 5.3 scrub ok 2024-09-16T09:46:03.691 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:46:03 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/1957548452' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-16T09:46:03.691 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:46:03 smithi070 ceph-mon[30388]: 5.1c scrub starts 2024-09-16T09:46:03.691 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:46:03 smithi070 ceph-mon[30388]: 5.1c scrub ok 2024-09-16T09:46:03.691 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:46:03 smithi070 ceph-mon[40691]: pgmap v1114: 162 pgs: 162 active+clean; 583 KiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-09-16T09:46:03.691 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:46:03 smithi070 ceph-mon[40691]: 5.3 scrub starts 2024-09-16T09:46:03.691 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:46:03 smithi070 ceph-mon[40691]: 5.3 scrub ok 2024-09-16T09:46:03.691 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:46:03 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/1957548452' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-16T09:46:03.691 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:46:03 smithi070 ceph-mon[40691]: 5.1c scrub starts 2024-09-16T09:46:03.692 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:46:03 smithi070 ceph-mon[40691]: 5.1c scrub ok 2024-09-16T09:46:03.806 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:46:03 smithi084 ceph-mon[32629]: pgmap v1114: 162 pgs: 162 active+clean; 583 KiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-09-16T09:46:03.807 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:46:03 smithi084 ceph-mon[32629]: 5.3 scrub starts 2024-09-16T09:46:03.807 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:46:03 smithi084 ceph-mon[32629]: 5.3 scrub ok 2024-09-16T09:46:03.807 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:46:03 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/1957548452' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-16T09:46:03.807 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:46:03 smithi084 ceph-mon[32629]: 5.1c scrub starts 2024-09-16T09:46:03.807 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:46:03 smithi084 ceph-mon[32629]: 5.1c scrub ok 2024-09-16T09:46:04.099 INFO:teuthology.orchestra.run.smithi070.stdout:34359738666 2024-09-16T09:46:04.160 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:46:03 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:46:04.418 INFO:teuthology.orchestra.run.smithi070.stdout:55834575140 2024-09-16T09:46:04.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:46:04 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:46:04.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:46:04 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:46:04.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:46:04 smithi084 ceph-mon[32629]: osdmap e570: 8 total, 8 up, 8 in 2024-09-16T09:46:04.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:46:04 smithi084 ceph-mon[32629]: 6.3 scrub starts 2024-09-16T09:46:04.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:46:04 smithi084 ceph-mon[32629]: 6.3 scrub ok 2024-09-16T09:46:04.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:46:04 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:46:04.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:46:04 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/2591968745' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-16T09:46:04.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:46:04 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/3100402587' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-16T09:46:04.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:46:04 smithi084 ceph-mon[32629]: osdmap e571: 8 total, 8 up, 8 in 2024-09-16T09:46:04.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:46:04 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:46:04.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:46:04 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:46:04.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:46:04 smithi070 ceph-mon[30388]: osdmap e570: 8 total, 8 up, 8 in 2024-09-16T09:46:04.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:46:04 smithi070 ceph-mon[30388]: 6.3 scrub starts 2024-09-16T09:46:04.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:46:04 smithi070 ceph-mon[30388]: 6.3 scrub ok 2024-09-16T09:46:04.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:46:04 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:46:04.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:46:04 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2591968745' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-16T09:46:04.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:46:04 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/3100402587' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-16T09:46:04.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:46:04 smithi070 ceph-mon[30388]: osdmap e571: 8 total, 8 up, 8 in 2024-09-16T09:46:04.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:46:04 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:46:04.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:46:04 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:46:04.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:46:04 smithi070 ceph-mon[40691]: osdmap e570: 8 total, 8 up, 8 in 2024-09-16T09:46:04.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:46:04 smithi070 ceph-mon[40691]: 6.3 scrub starts 2024-09-16T09:46:04.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:46:04 smithi070 ceph-mon[40691]: 6.3 scrub ok 2024-09-16T09:46:04.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:46:04 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-16T09:46:04.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:46:04 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/2591968745' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-16T09:46:04.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:46:04 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/3100402587' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-16T09:46:04.923 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:46:04 smithi070 ceph-mon[40691]: osdmap e571: 8 total, 8 up, 8 in 2024-09-16T09:46:05.248 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834575138 got 55834575140 for osd.1 2024-09-16T09:46:05.248 DEBUG:teuthology.parallel:result is None 2024-09-16T09:46:05.705 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495877 got 197568495878 for osd.6 2024-09-16T09:46:05.706 DEBUG:teuthology.parallel:result is None 2024-09-16T09:46:05.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:46:05 smithi084 ceph-mon[32629]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:46:05.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:46:05 smithi084 ceph-mon[32629]: pgmap v1116: 162 pgs: 162 active+clean; 583 KiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-16T09:46:05.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:46:05 smithi084 ceph-mon[32629]: 5.5 scrub starts 2024-09-16T09:46:05.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:46:05 smithi084 ceph-mon[32629]: 5.5 scrub ok 2024-09-16T09:46:05.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:46:05 smithi084 ceph-mon[32629]: 5.b scrub starts 2024-09-16T09:46:05.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:46:05 smithi084 ceph-mon[32629]: 5.b scrub ok 2024-09-16T09:46:05.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:46:05 smithi084 ceph-mon[32629]: 1.10 scrub starts 2024-09-16T09:46:05.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:46:05 smithi084 ceph-mon[32629]: 1.10 scrub ok 2024-09-16T09:46:05.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:46:05 smithi084 ceph-mon[32629]: purged_snaps scrub starts 2024-09-16T09:46:05.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:46:05 smithi084 ceph-mon[32629]: purged_snaps scrub ok 2024-09-16T09:46:05.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:46:05 smithi084 ceph-mon[32629]: 6.17 scrub starts 2024-09-16T09:46:05.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:46:05 smithi070 ceph-mon[30388]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:46:05.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:46:05 smithi070 ceph-mon[30388]: pgmap v1116: 162 pgs: 162 active+clean; 583 KiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-16T09:46:05.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:46:05 smithi070 ceph-mon[30388]: 5.5 scrub starts 2024-09-16T09:46:05.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:46:05 smithi070 ceph-mon[30388]: 5.5 scrub ok 2024-09-16T09:46:05.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:46:05 smithi070 ceph-mon[30388]: 5.b scrub starts 2024-09-16T09:46:05.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:46:05 smithi070 ceph-mon[30388]: 5.b scrub ok 2024-09-16T09:46:05.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:46:05 smithi070 ceph-mon[30388]: 1.10 scrub starts 2024-09-16T09:46:05.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:46:05 smithi070 ceph-mon[30388]: 1.10 scrub ok 2024-09-16T09:46:05.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:46:05 smithi070 ceph-mon[30388]: purged_snaps scrub starts 2024-09-16T09:46:05.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:46:05 smithi070 ceph-mon[30388]: purged_snaps scrub ok 2024-09-16T09:46:05.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:46:05 smithi070 ceph-mon[30388]: 6.17 scrub starts 2024-09-16T09:46:05.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:46:05 smithi070 ceph-mon[40691]: from='client.14547 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-16T09:46:05.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:46:05 smithi070 ceph-mon[40691]: pgmap v1116: 162 pgs: 162 active+clean; 583 KiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-16T09:46:05.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:46:05 smithi070 ceph-mon[40691]: 5.5 scrub starts 2024-09-16T09:46:05.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:46:05 smithi070 ceph-mon[40691]: 5.5 scrub ok 2024-09-16T09:46:05.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:46:05 smithi070 ceph-mon[40691]: 5.b scrub starts 2024-09-16T09:46:05.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:46:05 smithi070 ceph-mon[40691]: 5.b scrub ok 2024-09-16T09:46:05.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:46:05 smithi070 ceph-mon[40691]: 1.10 scrub starts 2024-09-16T09:46:05.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:46:05 smithi070 ceph-mon[40691]: 1.10 scrub ok 2024-09-16T09:46:05.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:46:05 smithi070 ceph-mon[40691]: purged_snaps scrub starts 2024-09-16T09:46:05.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:46:05 smithi070 ceph-mon[40691]: purged_snaps scrub ok 2024-09-16T09:46:05.922 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:46:05 smithi070 ceph-mon[40691]: 6.17 scrub starts 2024-09-16T09:46:06.134 INFO:tasks.cephadm.ceph_manager.ceph:need seq 137438953743 got 137438953745 for osd.4 2024-09-16T09:46:06.135 DEBUG:teuthology.parallel:result is None 2024-09-16T09:46:06.613 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738663 got 34359738666 for osd.0 2024-09-16T09:46:06.614 DEBUG:teuthology.parallel:result is None 2024-09-16T09:46:06.614 INFO:tasks.cephadm.ceph_manager.ceph:waiting for recovery to complete 2024-09-16T09:46:06.614 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-16T09:46:06.865 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:46:06.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:46:06 smithi084 ceph-mon[32629]: 6.11 scrub starts 2024-09-16T09:46:06.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:46:06 smithi084 ceph-mon[32629]: 6.11 scrub ok 2024-09-16T09:46:06.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:46:06 smithi084 ceph-mon[32629]: 6.17 scrub ok 2024-09-16T09:46:06.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:46:06 smithi084 ceph-mon[32629]: osdmap e572: 8 total, 8 up, 8 in 2024-09-16T09:46:06.920 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:46:06 smithi070 ceph-mon[40691]: 6.11 scrub starts 2024-09-16T09:46:06.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:46:06 smithi070 ceph-mon[40691]: 6.11 scrub ok 2024-09-16T09:46:06.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:46:06 smithi070 ceph-mon[40691]: 6.17 scrub ok 2024-09-16T09:46:06.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:46:06 smithi070 ceph-mon[40691]: osdmap e572: 8 total, 8 up, 8 in 2024-09-16T09:46:06.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:46:06 smithi070 ceph-mon[30388]: 6.11 scrub starts 2024-09-16T09:46:06.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:46:06 smithi070 ceph-mon[30388]: 6.11 scrub ok 2024-09-16T09:46:06.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:46:06 smithi070 ceph-mon[30388]: 6.17 scrub ok 2024-09-16T09:46:06.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:46:06 smithi070 ceph-mon[30388]: osdmap e572: 8 total, 8 up, 8 in 2024-09-16T09:46:07.539 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:46:07.542 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2024-09-16T09:46:07.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:46:07 smithi084 ceph-mon[32629]: pgmap v1119: 162 pgs: 162 active+clean; 583 KiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:46:07.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:46:07 smithi084 ceph-mon[32629]: 5.12 scrub starts 2024-09-16T09:46:07.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:46:07 smithi084 ceph-mon[32629]: 5.12 scrub ok 2024-09-16T09:46:07.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:46:07 smithi084 ceph-mon[32629]: 5.e scrub starts 2024-09-16T09:46:07.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:46:07 smithi084 ceph-mon[32629]: 5.e scrub ok 2024-09-16T09:46:07.912 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:46:07 smithi070 ceph-mon[30388]: pgmap v1119: 162 pgs: 162 active+clean; 583 KiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:46:07.912 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:46:07 smithi070 ceph-mon[30388]: 5.12 scrub starts 2024-09-16T09:46:07.912 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:46:07 smithi070 ceph-mon[30388]: 5.12 scrub ok 2024-09-16T09:46:07.912 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:46:07 smithi070 ceph-mon[30388]: 5.e scrub starts 2024-09-16T09:46:07.912 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:46:07 smithi070 ceph-mon[30388]: 5.e scrub ok 2024-09-16T09:46:07.913 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:46:07 smithi070 ceph-mon[40691]: pgmap v1119: 162 pgs: 162 active+clean; 583 KiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:46:07.913 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:46:07 smithi070 ceph-mon[40691]: 5.12 scrub starts 2024-09-16T09:46:07.913 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:46:07 smithi070 ceph-mon[40691]: 5.12 scrub ok 2024-09-16T09:46:07.913 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:46:07 smithi070 ceph-mon[40691]: 5.e scrub starts 2024-09-16T09:46:07.913 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:46:07 smithi070 ceph-mon[40691]: 5.e scrub ok 2024-09-16T09:46:08.332 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":1119,"stamp":"2024-09-16T09:46:05.945278+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596507,"num_objects":199,"num_object_clones":0,"num_object_copies":597,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":199,"num_whiteouts":0,"num_read":2814,"num_read_kb":2566,"num_write":1008,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":149,"num_bytes_recovered":5672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3514,"ondisk_log_size":3514,"up":486,"acting":486,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":488,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":6638076,"kb_used_data":586292,"kb_used_omap":15,"kb_used_meta":6051632,"kb_avail":743323140,"statfs":{"total":767960285184,"available":761162895360,"internally_reserved":0,"allocated":600363008,"data_stored":596390513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15600,"internal_metadata":6196871952},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001541"},"pg_stats":[{"pgid":"1.1d","version":"73'79","reported_seq":58,"reported_epoch":570,"state":"active+clean","last_fresh":"2024-09-16T09:46:03.530193+0000","last_change":"2024-09-16T09:45:20.351727+0000","last_active":"2024-09-16T09:46:03.530193+0000","last_peered":"2024-09-16T09:46:03.530193+0000","last_clean":"2024-09-16T09:46:03.530193+0000","last_became_active":"2024-09-16T09:45:19.679136+0000","last_became_peered":"2024-09-16T09:45:19.679136+0000","last_unstale":"2024-09-16T09:46:03.530193+0000","last_undegraded":"2024-09-16T09:46:03.530193+0000","last_fullsized":"2024-09-16T09:46:03.530193+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":542,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:45:20.351679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:45:20.351679+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:24.766650+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":1110,"reported_epoch":571,"state":"active+clean","last_fresh":"2024-09-16T09:46:04.854898+0000","last_change":"2024-09-16T09:44:58.203883+0000","last_active":"2024-09-16T09:46:04.854898+0000","last_peered":"2024-09-16T09:46:04.854898+0000","last_clean":"2024-09-16T09:46:04.854898+0000","last_became_active":"2024-09-16T09:44:24.277455+0000","last_became_peered":"2024-09-16T09:44:24.277455+0000","last_unstale":"2024-09-16T09:46:04.854898+0000","last_undegraded":"2024-09-16T09:46:04.854898+0000","last_fullsized":"2024-09-16T09:46:04.854898+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:58.203811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:44:58.203811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:09.563756+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"547'21","reported_seq":1123,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:48.151027+0000","last_change":"2024-09-16T09:45:48.151027+0000","last_active":"2024-09-16T09:45:48.151027+0000","last_peered":"2024-09-16T09:45:48.151027+0000","last_clean":"2024-09-16T09:45:48.151027+0000","last_became_active":"2024-09-16T09:25:03.714635+0000","last_became_peered":"2024-09-16T09:25:03.714635+0000","last_unstale":"2024-09-16T09:45:48.151027+0000","last_undegraded":"2024-09-16T09:45:48.151027+0000","last_fullsized":"2024-09-16T09:45:48.151027+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"547'21","last_scrub_stamp":"2024-09-16T09:45:48.150947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:45:48.150947+0000","objects_scrubbed":4,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:52.939721+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":440,"reported_epoch":570,"state":"active+clean","last_fresh":"2024-09-16T09:46:03.531084+0000","last_change":"2024-09-16T09:45:25.410780+0000","last_active":"2024-09-16T09:46:03.531084+0000","last_peered":"2024-09-16T09:46:03.531084+0000","last_clean":"2024-09-16T09:46:03.531084+0000","last_became_active":"2024-09-16T09:45:19.684964+0000","last_became_peered":"2024-09-16T09:45:19.684964+0000","last_unstale":"2024-09-16T09:46:03.531084+0000","last_undegraded":"2024-09-16T09:46:03.531084+0000","last_fullsized":"2024-09-16T09:46:03.531084+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:25.410724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:45:25.410724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:36.400332+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1073,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:46:01.209679+0000","last_change":"2024-09-16T09:46:01.209679+0000","last_active":"2024-09-16T09:46:01.209679+0000","last_peered":"2024-09-16T09:46:01.209679+0000","last_clean":"2024-09-16T09:46:01.209679+0000","last_became_active":"2024-09-16T09:25:07.715442+0000","last_became_peered":"2024-09-16T09:25:07.715442+0000","last_unstale":"2024-09-16T09:46:01.209679+0000","last_undegraded":"2024-09-16T09:46:01.209679+0000","last_fullsized":"2024-09-16T09:46:01.209679+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:46:01.209620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:46:01.209620+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:47:05.487897+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1c","version":"73'79","reported_seq":68,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:43.760039+0000","last_change":"2024-09-16T09:45:05.686494+0000","last_active":"2024-09-16T09:45:43.760039+0000","last_peered":"2024-09-16T09:45:43.760039+0000","last_clean":"2024-09-16T09:45:43.760039+0000","last_became_active":"2024-09-16T09:45:05.495834+0000","last_became_peered":"2024-09-16T09:45:05.495834+0000","last_unstale":"2024-09-16T09:45:43.760039+0000","last_undegraded":"2024-09-16T09:45:43.760039+0000","last_fullsized":"2024-09-16T09:45:43.760039+0000","mapping_epoch":544,"log_start":"0'0","ondisk_log_start":"0'0","created":542,"last_epoch_clean":545,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:45:05.686443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:45:05.686443+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:29.935815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":814,"reported_epoch":570,"state":"active+clean","last_fresh":"2024-09-16T09:46:03.603818+0000","last_change":"2024-09-16T09:44:34.236344+0000","last_active":"2024-09-16T09:46:03.603818+0000","last_peered":"2024-09-16T09:46:03.603818+0000","last_clean":"2024-09-16T09:46:03.603818+0000","last_became_active":"2024-09-16T09:31:09.269342+0000","last_became_peered":"2024-09-16T09:31:09.269342+0000","last_unstale":"2024-09-16T09:46:03.603818+0000","last_undegraded":"2024-09-16T09:46:03.603818+0000","last_fullsized":"2024-09-16T09:46:03.603818+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:34.236298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:44:34.236298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":885,"reported_epoch":570,"state":"active+clean","last_fresh":"2024-09-16T09:46:03.604468+0000","last_change":"2024-09-16T09:45:58.877562+0000","last_active":"2024-09-16T09:46:03.604468+0000","last_peered":"2024-09-16T09:46:03.604468+0000","last_clean":"2024-09-16T09:46:03.604468+0000","last_became_active":"2024-09-16T09:45:23.693724+0000","last_became_peered":"2024-09-16T09:45:23.693724+0000","last_unstale":"2024-09-16T09:46:03.604468+0000","last_undegraded":"2024-09-16T09:46:03.604468+0000","last_fullsized":"2024-09-16T09:46:03.604468+0000","mapping_epoch":554,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":555,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:58.877516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:45:58.877516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:47:24.081212+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"547'39","reported_seq":314,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:46.160696+0000","last_change":"2024-09-16T09:45:46.160643+0000","last_active":"2024-09-16T09:45:46.160696+0000","last_peered":"2024-09-16T09:45:46.160696+0000","last_clean":"2024-09-16T09:45:46.160696+0000","last_became_active":"2024-09-16T09:40:58.699357+0000","last_became_peered":"2024-09-16T09:40:58.699357+0000","last_unstale":"2024-09-16T09:45:46.160696+0000","last_undegraded":"2024-09-16T09:45:46.160696+0000","last_fullsized":"2024-09-16T09:45:46.160696+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"547'39","last_scrub_stamp":"2024-09-16T09:45:46.160568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:45:46.160568+0000","objects_scrubbed":7,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:47:09.489573+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":61,"num_read_kb":52,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,5],"acting":[4,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1085,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:44.127046+0000","last_change":"2024-09-16T09:45:12.160663+0000","last_active":"2024-09-16T09:45:44.127046+0000","last_peered":"2024-09-16T09:45:44.127046+0000","last_clean":"2024-09-16T09:45:44.127046+0000","last_became_active":"2024-09-16T09:25:05.717236+0000","last_became_peered":"2024-09-16T09:25:05.717236+0000","last_unstale":"2024-09-16T09:45:44.127046+0000","last_undegraded":"2024-09-16T09:45:44.127046+0000","last_fullsized":"2024-09-16T09:45:44.127046+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:12.160601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:45:12.160601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:38.736753+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":270,"reported_epoch":570,"state":"active+clean","last_fresh":"2024-09-16T09:46:03.530288+0000","last_change":"2024-09-16T09:45:31.429188+0000","last_active":"2024-09-16T09:46:03.530288+0000","last_peered":"2024-09-16T09:46:03.530288+0000","last_clean":"2024-09-16T09:46:03.530288+0000","last_became_active":"2024-09-16T09:45:19.684857+0000","last_became_peered":"2024-09-16T09:45:19.684857+0000","last_unstale":"2024-09-16T09:46:03.530288+0000","last_undegraded":"2024-09-16T09:46:03.530288+0000","last_fullsized":"2024-09-16T09:46:03.530288+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:31.429144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:45:31.429144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:31.546749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"547'21","reported_seq":1123,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:44.126813+0000","last_change":"2024-09-16T09:45:41.041633+0000","last_active":"2024-09-16T09:45:44.126813+0000","last_peered":"2024-09-16T09:45:44.126813+0000","last_clean":"2024-09-16T09:45:44.126813+0000","last_became_active":"2024-09-16T09:25:03.709147+0000","last_became_peered":"2024-09-16T09:25:03.709147+0000","last_unstale":"2024-09-16T09:45:44.126813+0000","last_undegraded":"2024-09-16T09:45:44.126813+0000","last_fullsized":"2024-09-16T09:45:44.126813+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"547'21","last_scrub_stamp":"2024-09-16T09:45:41.041566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:45:41.041566+0000","objects_scrubbed":4,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:47:02.498406+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1068,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:43.756418+0000","last_change":"2024-09-16T09:45:41.810793+0000","last_active":"2024-09-16T09:45:43.756418+0000","last_peered":"2024-09-16T09:45:43.756418+0000","last_clean":"2024-09-16T09:45:43.756418+0000","last_became_active":"2024-09-16T09:25:05.718134+0000","last_became_peered":"2024-09-16T09:25:05.718134+0000","last_unstale":"2024-09-16T09:45:43.756418+0000","last_undegraded":"2024-09-16T09:45:43.756418+0000","last_fullsized":"2024-09-16T09:45:43.756418+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:41.810731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:45:41.810731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:47:08.215365+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"66'1","reported_seq":1094,"reported_epoch":571,"state":"active+clean","last_fresh":"2024-09-16T09:46:04.854174+0000","last_change":"2024-09-16T09:45:54.979631+0000","last_active":"2024-09-16T09:46:04.854174+0000","last_peered":"2024-09-16T09:46:04.854174+0000","last_clean":"2024-09-16T09:46:04.854174+0000","last_became_active":"2024-09-16T09:25:07.716558+0000","last_became_peered":"2024-09-16T09:25:07.716558+0000","last_unstale":"2024-09-16T09:46:04.854174+0000","last_undegraded":"2024-09-16T09:46:04.854174+0000","last_fullsized":"2024-09-16T09:46:04.854174+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-16T09:45:54.979570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:45:54.979570+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:47:23.182188+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1e","version":"73'79","reported_seq":149,"reported_epoch":570,"state":"active+clean","last_fresh":"2024-09-16T09:46:03.604570+0000","last_change":"2024-09-16T09:45:06.199206+0000","last_active":"2024-09-16T09:46:03.604570+0000","last_peered":"2024-09-16T09:46:03.604570+0000","last_clean":"2024-09-16T09:46:03.604570+0000","last_became_active":"2024-09-16T09:45:05.498656+0000","last_became_peered":"2024-09-16T09:45:05.498656+0000","last_unstale":"2024-09-16T09:46:03.604570+0000","last_undegraded":"2024-09-16T09:46:03.604570+0000","last_fullsized":"2024-09-16T09:46:03.604570+0000","mapping_epoch":544,"log_start":"0'0","ondisk_log_start":"0'0","created":542,"last_epoch_clean":545,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:45:06.199126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:45:06.199126+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:25.814352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":1068,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:43.756184+0000","last_change":"2024-09-16T09:45:18.790528+0000","last_active":"2024-09-16T09:45:43.756184+0000","last_peered":"2024-09-16T09:45:43.756184+0000","last_clean":"2024-09-16T09:45:43.756184+0000","last_became_active":"2024-09-16T09:25:01.696939+0000","last_became_peered":"2024-09-16T09:25:01.696939+0000","last_unstale":"2024-09-16T09:45:43.756184+0000","last_undegraded":"2024-09-16T09:45:43.756184+0000","last_fullsized":"2024-09-16T09:45:43.756184+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:18.790452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:45:18.790452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:28.523093+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"547'9","reported_seq":1081,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:44.126805+0000","last_change":"2024-09-16T09:45:18.121071+0000","last_active":"2024-09-16T09:45:44.126805+0000","last_peered":"2024-09-16T09:45:44.126805+0000","last_clean":"2024-09-16T09:45:44.126805+0000","last_became_active":"2024-09-16T09:25:03.716136+0000","last_became_peered":"2024-09-16T09:25:03.716136+0000","last_unstale":"2024-09-16T09:45:44.126805+0000","last_undegraded":"2024-09-16T09:45:44.126805+0000","last_fullsized":"2024-09-16T09:45:44.126805+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"547'9","last_scrub_stamp":"2024-09-16T09:45:18.121010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:45:18.121010+0000","objects_scrubbed":2,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:37.208073+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":17,"num_read_kb":13,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":225,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:44.399751+0000","last_change":"2024-09-16T09:45:06.671068+0000","last_active":"2024-09-16T09:45:44.399751+0000","last_peered":"2024-09-16T09:45:44.399751+0000","last_clean":"2024-09-16T09:45:44.399751+0000","last_became_active":"2024-09-16T09:42:02.494290+0000","last_became_peered":"2024-09-16T09:42:02.494290+0000","last_unstale":"2024-09-16T09:45:44.399751+0000","last_undegraded":"2024-09-16T09:45:44.399751+0000","last_fullsized":"2024-09-16T09:45:44.399751+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:06.671012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:45:06.671012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:20.138489+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4],"acting":[6,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1094,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:43.756046+0000","last_change":"2024-09-16T09:44:52.742833+0000","last_active":"2024-09-16T09:45:43.756046+0000","last_peered":"2024-09-16T09:45:43.756046+0000","last_clean":"2024-09-16T09:45:43.756046+0000","last_became_active":"2024-09-16T09:31:09.275656+0000","last_became_peered":"2024-09-16T09:31:09.275656+0000","last_unstale":"2024-09-16T09:45:43.756046+0000","last_undegraded":"2024-09-16T09:45:43.756046+0000","last_fullsized":"2024-09-16T09:45:43.756046+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:52.742776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:44:52.742776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:05.256885+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.19","version":"73'79","reported_seq":186,"reported_epoch":570,"state":"active+clean","last_fresh":"2024-09-16T09:46:03.602901+0000","last_change":"2024-09-16T09:45:24.075647+0000","last_active":"2024-09-16T09:46:03.602901+0000","last_peered":"2024-09-16T09:46:03.602901+0000","last_clean":"2024-09-16T09:46:03.602901+0000","last_became_active":"2024-09-16T09:45:23.694640+0000","last_became_peered":"2024-09-16T09:45:23.694640+0000","last_unstale":"2024-09-16T09:46:03.602901+0000","last_undegraded":"2024-09-16T09:46:03.602901+0000","last_fullsized":"2024-09-16T09:46:03.602901+0000","mapping_epoch":554,"log_start":"0'0","ondisk_log_start":"0'0","created":542,"last_epoch_clean":555,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:45:24.075601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:45:24.075601+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:39.357561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1075,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:44.127890+0000","last_change":"2024-09-16T09:45:35.117435+0000","last_active":"2024-09-16T09:45:44.127890+0000","last_peered":"2024-09-16T09:45:44.127890+0000","last_clean":"2024-09-16T09:45:44.127890+0000","last_became_active":"2024-09-16T09:25:07.720793+0000","last_became_peered":"2024-09-16T09:25:07.720793+0000","last_unstale":"2024-09-16T09:45:44.127890+0000","last_undegraded":"2024-09-16T09:45:44.127890+0000","last_fullsized":"2024-09-16T09:45:44.127890+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:35.117376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:45:35.117376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:49.783095+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":819,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:44.399693+0000","last_change":"2024-09-16T09:44:48.756968+0000","last_active":"2024-09-16T09:45:44.399693+0000","last_peered":"2024-09-16T09:45:44.399693+0000","last_clean":"2024-09-16T09:45:44.399693+0000","last_became_active":"2024-09-16T09:30:39.201824+0000","last_became_peered":"2024-09-16T09:30:39.201824+0000","last_unstale":"2024-09-16T09:45:44.399693+0000","last_undegraded":"2024-09-16T09:45:44.399693+0000","last_fullsized":"2024-09-16T09:45:44.399693+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:48.756890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:44:48.756890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:17.988105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"547'39","reported_seq":1128,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:43.761767+0000","last_change":"2024-09-16T09:45:43.761674+0000","last_active":"2024-09-16T09:45:43.761767+0000","last_peered":"2024-09-16T09:45:43.761767+0000","last_clean":"2024-09-16T09:45:43.761767+0000","last_became_active":"2024-09-16T09:45:43.761251+0000","last_became_peered":"2024-09-16T09:45:43.761251+0000","last_unstale":"2024-09-16T09:45:43.761767+0000","last_undegraded":"2024-09-16T09:45:43.761767+0000","last_fullsized":"2024-09-16T09:45:43.761767+0000","mapping_epoch":568,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":569,"parent":"0.0","parent_split_bits":0,"last_scrub":"547'39","last_scrub_stamp":"2024-09-16T09:45:30.382285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:45:30.382285+0000","objects_scrubbed":7,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:56.263792+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":61,"num_read_kb":52,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":424,"reported_epoch":570,"state":"active+clean","last_fresh":"2024-09-16T09:46:03.530728+0000","last_change":"2024-09-16T09:45:41.521364+0000","last_active":"2024-09-16T09:46:03.530728+0000","last_peered":"2024-09-16T09:46:03.530728+0000","last_clean":"2024-09-16T09:46:03.530728+0000","last_became_active":"2024-09-16T09:45:19.679433+0000","last_became_peered":"2024-09-16T09:45:19.679433+0000","last_unstale":"2024-09-16T09:46:03.530728+0000","last_undegraded":"2024-09-16T09:46:03.530728+0000","last_fullsized":"2024-09-16T09:46:03.530728+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:41.521302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:45:41.521302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:57.436115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.18","version":"73'79","reported_seq":452,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:43.755726+0000","last_change":"2024-09-16T09:45:07.843788+0000","last_active":"2024-09-16T09:45:43.755726+0000","last_peered":"2024-09-16T09:45:43.755726+0000","last_clean":"2024-09-16T09:45:43.755726+0000","last_became_active":"2024-09-16T09:45:05.501178+0000","last_became_peered":"2024-09-16T09:45:05.501178+0000","last_unstale":"2024-09-16T09:45:43.755726+0000","last_undegraded":"2024-09-16T09:45:43.755726+0000","last_fullsized":"2024-09-16T09:45:43.755726+0000","mapping_epoch":544,"log_start":"0'0","ondisk_log_start":"0'0","created":542,"last_epoch_clean":545,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:45:07.843717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:45:07.843717+0000","objects_scrubbed":1,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:25.958206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262688,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":203,"reported_epoch":570,"state":"active+clean","last_fresh":"2024-09-16T09:46:03.604140+0000","last_change":"2024-09-16T09:45:23.746000+0000","last_active":"2024-09-16T09:46:03.604140+0000","last_peered":"2024-09-16T09:46:03.604140+0000","last_clean":"2024-09-16T09:46:03.604140+0000","last_became_active":"2024-09-16T09:45:23.745693+0000","last_became_peered":"2024-09-16T09:45:23.745693+0000","last_unstale":"2024-09-16T09:46:03.604140+0000","last_undegraded":"2024-09-16T09:46:03.604140+0000","last_fullsized":"2024-09-16T09:46:03.604140+0000","mapping_epoch":554,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":555,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:12.706646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:45:12.706646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:13.810656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":1104,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:58.145095+0000","last_change":"2024-09-16T09:45:58.145095+0000","last_active":"2024-09-16T09:45:58.145095+0000","last_peered":"2024-09-16T09:45:58.145095+0000","last_clean":"2024-09-16T09:45:58.145095+0000","last_became_active":"2024-09-16T09:44:24.276029+0000","last_became_peered":"2024-09-16T09:44:24.276029+0000","last_unstale":"2024-09-16T09:45:58.145095+0000","last_undegraded":"2024-09-16T09:45:58.145095+0000","last_fullsized":"2024-09-16T09:45:58.145095+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:58.145033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:45:58.145033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:47:25.492043+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"547'24","reported_seq":775,"reported_epoch":570,"state":"active+clean","last_fresh":"2024-09-16T09:46:03.603662+0000","last_change":"2024-09-16T09:45:35.053006+0000","last_active":"2024-09-16T09:46:03.603662+0000","last_peered":"2024-09-16T09:46:03.603662+0000","last_clean":"2024-09-16T09:46:03.603662+0000","last_became_active":"2024-09-16T09:45:23.697139+0000","last_became_peered":"2024-09-16T09:45:23.697139+0000","last_unstale":"2024-09-16T09:46:03.603662+0000","last_undegraded":"2024-09-16T09:46:03.603662+0000","last_fullsized":"2024-09-16T09:46:03.603662+0000","mapping_epoch":554,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":555,"parent":"0.0","parent_split_bits":0,"last_scrub":"547'24","last_scrub_stamp":"2024-09-16T09:45:35.052951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:45:35.052951+0000","objects_scrubbed":5,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:47:01.202046+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":41,"num_read_kb":32,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1080,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:46:03.244317+0000","last_change":"2024-09-16T09:46:03.244317+0000","last_active":"2024-09-16T09:46:03.244317+0000","last_peered":"2024-09-16T09:46:03.244317+0000","last_clean":"2024-09-16T09:46:03.244317+0000","last_became_active":"2024-09-16T09:44:24.278117+0000","last_became_peered":"2024-09-16T09:44:24.278117+0000","last_unstale":"2024-09-16T09:46:03.244317+0000","last_undegraded":"2024-09-16T09:46:03.244317+0000","last_fullsized":"2024-09-16T09:46:03.244317+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:46:03.244250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:46:03.244250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:47:09.233638+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1b","version":"73'79","reported_seq":246,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:43.755874+0000","last_change":"2024-09-16T09:45:06.833076+0000","last_active":"2024-09-16T09:45:43.755874+0000","last_peered":"2024-09-16T09:45:43.755874+0000","last_clean":"2024-09-16T09:45:43.755874+0000","last_became_active":"2024-09-16T09:45:05.500314+0000","last_became_peered":"2024-09-16T09:45:05.500314+0000","last_unstale":"2024-09-16T09:45:43.755874+0000","last_undegraded":"2024-09-16T09:45:43.755874+0000","last_fullsized":"2024-09-16T09:45:43.755874+0000","mapping_epoch":544,"log_start":"0'0","ondisk_log_start":"0'0","created":542,"last_epoch_clean":545,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:45:06.833020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:45:06.833020+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:07.568877+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,0],"acting":[5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":1083,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:43.759922+0000","last_change":"2024-09-16T09:45:20.567757+0000","last_active":"2024-09-16T09:45:43.759922+0000","last_peered":"2024-09-16T09:45:43.759922+0000","last_clean":"2024-09-16T09:45:43.759922+0000","last_became_active":"2024-09-16T09:44:24.284256+0000","last_became_peered":"2024-09-16T09:44:24.284256+0000","last_unstale":"2024-09-16T09:45:43.759922+0000","last_undegraded":"2024-09-16T09:45:43.759922+0000","last_fullsized":"2024-09-16T09:45:43.759922+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-16T09:45:20.567693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:45:20.567693+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:32.553796+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":433,"reported_epoch":570,"state":"active+clean","last_fresh":"2024-09-16T09:46:03.531164+0000","last_change":"2024-09-16T09:45:19.680101+0000","last_active":"2024-09-16T09:46:03.531164+0000","last_peered":"2024-09-16T09:46:03.531164+0000","last_clean":"2024-09-16T09:46:03.531164+0000","last_became_active":"2024-09-16T09:45:19.679100+0000","last_became_peered":"2024-09-16T09:45:19.679100+0000","last_unstale":"2024-09-16T09:46:03.531164+0000","last_undegraded":"2024-09-16T09:46:03.531164+0000","last_fullsized":"2024-09-16T09:46:03.531164+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:45:18.060164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:45:18.060164+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:21.987974+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"547'30","reported_seq":1162,"reported_epoch":571,"state":"active+clean","last_fresh":"2024-09-16T09:46:04.854923+0000","last_change":"2024-09-16T09:45:14.222140+0000","last_active":"2024-09-16T09:46:04.854923+0000","last_peered":"2024-09-16T09:46:04.854923+0000","last_clean":"2024-09-16T09:46:04.854923+0000","last_became_active":"2024-09-16T09:31:09.275050+0000","last_became_peered":"2024-09-16T09:31:09.275050+0000","last_unstale":"2024-09-16T09:46:04.854923+0000","last_undegraded":"2024-09-16T09:46:04.854923+0000","last_fullsized":"2024-09-16T09:46:04.854923+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"547'30","last_scrub_stamp":"2024-09-16T09:45:14.222080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:45:14.222080+0000","objects_scrubbed":5,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:31.788116+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":45,"num_read_kb":40,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"66'8","reported_seq":1377,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:46:00.589960+0000","last_change":"2024-09-16T09:46:00.589854+0000","last_active":"2024-09-16T09:46:00.589960+0000","last_peered":"2024-09-16T09:46:00.589960+0000","last_clean":"2024-09-16T09:46:00.589960+0000","last_became_active":"2024-09-16T09:25:05.712516+0000","last_became_peered":"2024-09-16T09:25:05.712516+0000","last_unstale":"2024-09-16T09:46:00.589960+0000","last_undegraded":"2024-09-16T09:46:00.589960+0000","last_fullsized":"2024-09-16T09:46:00.589960+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:46:00.589779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:46:00.589779+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:47:08.473519+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.a","version":"73'79","reported_seq":1037,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:43.755823+0000","last_change":"2024-09-16T09:45:03.744061+0000","last_active":"2024-09-16T09:45:43.755823+0000","last_peered":"2024-09-16T09:45:43.755823+0000","last_clean":"2024-09-16T09:45:43.755823+0000","last_became_active":"2024-09-16T09:45:03.607826+0000","last_became_peered":"2024-09-16T09:45:03.607826+0000","last_unstale":"2024-09-16T09:45:43.755823+0000","last_undegraded":"2024-09-16T09:45:43.755823+0000","last_fullsized":"2024-09-16T09:45:43.755823+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":543,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:45:03.744008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:45:03.744008+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:27.030654+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"547'39","reported_seq":1033,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:43.758069+0000","last_change":"2024-09-16T09:44:45.023239+0000","last_active":"2024-09-16T09:45:43.758069+0000","last_peered":"2024-09-16T09:45:43.758069+0000","last_clean":"2024-09-16T09:45:43.758069+0000","last_became_active":"2024-09-16T09:44:24.283121+0000","last_became_peered":"2024-09-16T09:44:24.283121+0000","last_unstale":"2024-09-16T09:45:43.758069+0000","last_undegraded":"2024-09-16T09:45:43.758069+0000","last_fullsized":"2024-09-16T09:45:43.758069+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'27","last_scrub_stamp":"2024-09-16T09:44:45.023173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:44:45.023173+0000","objects_scrubbed":7,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:12.611250+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":61,"num_read_kb":52,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":815,"reported_epoch":570,"state":"active+clean","last_fresh":"2024-09-16T09:46:03.602870+0000","last_change":"2024-09-16T09:45:14.089504+0000","last_active":"2024-09-16T09:46:03.602870+0000","last_peered":"2024-09-16T09:46:03.602870+0000","last_clean":"2024-09-16T09:46:03.602870+0000","last_became_active":"2024-09-16T09:31:09.274534+0000","last_became_peered":"2024-09-16T09:31:09.274534+0000","last_unstale":"2024-09-16T09:46:03.602870+0000","last_undegraded":"2024-09-16T09:46:03.602870+0000","last_fullsized":"2024-09-16T09:46:03.602870+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:14.089456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:45:14.089456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:31.001442+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"66'8","reported_seq":1323,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:56.995111+0000","last_change":"2024-09-16T09:44:50.326230+0000","last_active":"2024-09-16T09:45:56.995111+0000","last_peered":"2024-09-16T09:45:56.995111+0000","last_clean":"2024-09-16T09:45:56.995111+0000","last_became_active":"2024-09-16T09:25:05.713813+0000","last_became_peered":"2024-09-16T09:25:05.713813+0000","last_unstale":"2024-09-16T09:45:56.995111+0000","last_undegraded":"2024-09-16T09:45:56.995111+0000","last_fullsized":"2024-09-16T09:45:56.995111+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:44:50.326143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:44:50.326143+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1081,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:43.756165+0000","last_change":"2024-09-16T09:45:39.743940+0000","last_active":"2024-09-16T09:45:43.756165+0000","last_peered":"2024-09-16T09:45:43.756165+0000","last_clean":"2024-09-16T09:45:43.756165+0000","last_became_active":"2024-09-16T09:25:07.717344+0000","last_became_peered":"2024-09-16T09:25:07.717344+0000","last_unstale":"2024-09-16T09:45:43.756165+0000","last_undegraded":"2024-09-16T09:45:43.756165+0000","last_fullsized":"2024-09-16T09:45:43.756165+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:39.743872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:45:39.743872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:47:05.285045+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.5","version":"73'79","reported_seq":1070,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:44.128371+0000","last_change":"2024-09-16T09:45:04.230186+0000","last_active":"2024-09-16T09:45:44.128371+0000","last_peered":"2024-09-16T09:45:44.128371+0000","last_clean":"2024-09-16T09:45:44.128371+0000","last_became_active":"2024-09-16T09:45:03.610857+0000","last_became_peered":"2024-09-16T09:45:03.610857+0000","last_unstale":"2024-09-16T09:45:44.128371+0000","last_undegraded":"2024-09-16T09:45:44.128371+0000","last_fullsized":"2024-09-16T09:45:44.128371+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":543,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:45:04.230129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:45:04.230129+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:16.266653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"547'42","reported_seq":1065,"reported_epoch":570,"state":"active+clean","last_fresh":"2024-09-16T09:46:03.602583+0000","last_change":"2024-09-16T09:45:54.965274+0000","last_active":"2024-09-16T09:46:03.602583+0000","last_peered":"2024-09-16T09:46:03.602583+0000","last_clean":"2024-09-16T09:46:03.602583+0000","last_became_active":"2024-09-16T09:45:23.693680+0000","last_became_peered":"2024-09-16T09:45:23.693680+0000","last_unstale":"2024-09-16T09:46:03.602583+0000","last_undegraded":"2024-09-16T09:46:03.602583+0000","last_fullsized":"2024-09-16T09:46:03.602583+0000","mapping_epoch":554,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":555,"parent":"0.0","parent_split_bits":0,"last_scrub":"547'42","last_scrub_stamp":"2024-09-16T09:45:54.965208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:45:54.965208+0000","objects_scrubbed":8,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:47:11.271926+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":68,"num_read_kb":56,"num_write":44,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":231,"reported_epoch":570,"state":"active+clean","last_fresh":"2024-09-16T09:46:03.604338+0000","last_change":"2024-09-16T09:45:55.953684+0000","last_active":"2024-09-16T09:46:03.604338+0000","last_peered":"2024-09-16T09:46:03.604338+0000","last_clean":"2024-09-16T09:46:03.604338+0000","last_became_active":"2024-09-16T09:45:23.697210+0000","last_became_peered":"2024-09-16T09:45:23.697210+0000","last_unstale":"2024-09-16T09:46:03.604338+0000","last_undegraded":"2024-09-16T09:46:03.604338+0000","last_fullsized":"2024-09-16T09:46:03.604338+0000","mapping_epoch":554,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":555,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:55.953637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:45:55.953637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:58.992515+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1093,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:44.127282+0000","last_change":"2024-09-16T09:44:58.291670+0000","last_active":"2024-09-16T09:45:44.127282+0000","last_peered":"2024-09-16T09:45:44.127282+0000","last_clean":"2024-09-16T09:45:44.127282+0000","last_became_active":"2024-09-16T09:31:09.268864+0000","last_became_peered":"2024-09-16T09:31:09.268864+0000","last_unstale":"2024-09-16T09:45:44.127282+0000","last_undegraded":"2024-09-16T09:45:44.127282+0000","last_fullsized":"2024-09-16T09:45:44.127282+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:58.291613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:44:58.291613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:09.340193+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1108,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:55.213059+0000","last_change":"2024-09-16T09:45:55.213007+0000","last_active":"2024-09-16T09:45:55.213059+0000","last_peered":"2024-09-16T09:45:55.213059+0000","last_clean":"2024-09-16T09:45:55.213059+0000","last_became_active":"2024-09-16T09:44:24.276365+0000","last_became_peered":"2024-09-16T09:44:24.276365+0000","last_unstale":"2024-09-16T09:45:55.213059+0000","last_undegraded":"2024-09-16T09:45:55.213059+0000","last_fullsized":"2024-09-16T09:45:55.213059+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:55.212941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:45:55.212941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:47:23.458239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"73'79","reported_seq":828,"reported_epoch":570,"state":"active+clean","last_fresh":"2024-09-16T09:46:03.604730+0000","last_change":"2024-09-16T09:45:56.957735+0000","last_active":"2024-09-16T09:46:03.604730+0000","last_peered":"2024-09-16T09:46:03.604730+0000","last_clean":"2024-09-16T09:46:03.604730+0000","last_became_active":"2024-09-16T09:31:09.275139+0000","last_became_peered":"2024-09-16T09:31:09.275139+0000","last_unstale":"2024-09-16T09:46:03.604730+0000","last_undegraded":"2024-09-16T09:46:03.604730+0000","last_fullsized":"2024-09-16T09:46:03.604730+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:45:56.957686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:45:56.957686+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:47:21.775448+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"547'42","reported_seq":1154,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:44.126870+0000","last_change":"2024-09-16T09:45:40.071609+0000","last_active":"2024-09-16T09:45:44.126870+0000","last_peered":"2024-09-16T09:45:44.126870+0000","last_clean":"2024-09-16T09:45:44.126870+0000","last_became_active":"2024-09-16T09:25:03.708830+0000","last_became_peered":"2024-09-16T09:25:03.708830+0000","last_unstale":"2024-09-16T09:45:44.126870+0000","last_undegraded":"2024-09-16T09:45:44.126870+0000","last_fullsized":"2024-09-16T09:45:44.126870+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"547'42","last_scrub_stamp":"2024-09-16T09:45:40.071530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:45:40.071530+0000","objects_scrubbed":7,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:58.230475+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":63,"num_read_kb":56,"num_write":42,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":1107,"reported_epoch":571,"state":"active+clean","last_fresh":"2024-09-16T09:46:04.855028+0000","last_change":"2024-09-16T09:45:44.988737+0000","last_active":"2024-09-16T09:46:04.855028+0000","last_peered":"2024-09-16T09:46:04.855028+0000","last_clean":"2024-09-16T09:46:04.855028+0000","last_became_active":"2024-09-16T09:25:01.702992+0000","last_became_peered":"2024-09-16T09:25:01.702992+0000","last_unstale":"2024-09-16T09:46:04.855028+0000","last_undegraded":"2024-09-16T09:46:04.855028+0000","last_fullsized":"2024-09-16T09:46:04.855028+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:45:44.988667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:45:44.988667+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:47:07.198294+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"557'14","reported_seq":242,"reported_epoch":570,"state":"active+clean","last_fresh":"2024-09-16T09:46:03.602687+0000","last_change":"2024-09-16T09:45:26.097190+0000","last_active":"2024-09-16T09:46:03.602687+0000","last_peered":"2024-09-16T09:46:03.602687+0000","last_clean":"2024-09-16T09:46:03.602687+0000","last_became_active":"2024-09-16T09:45:23.691975+0000","last_became_peered":"2024-09-16T09:45:23.691975+0000","last_unstale":"2024-09-16T09:46:03.602687+0000","last_undegraded":"2024-09-16T09:46:03.602687+0000","last_fullsized":"2024-09-16T09:46:03.602687+0000","mapping_epoch":554,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":555,"parent":"0.0","parent_split_bits":0,"last_scrub":"552'13","last_scrub_stamp":"2024-09-16T09:45:26.097136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:45:26.097136+0000","objects_scrubbed":1,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:40.308843+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1076,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:43.759393+0000","last_change":"2024-09-16T09:44:29.455025+0000","last_active":"2024-09-16T09:45:43.759393+0000","last_peered":"2024-09-16T09:45:43.759393+0000","last_clean":"2024-09-16T09:45:43.759393+0000","last_became_active":"2024-09-16T09:44:24.277003+0000","last_became_peered":"2024-09-16T09:44:24.277003+0000","last_unstale":"2024-09-16T09:45:43.759393+0000","last_undegraded":"2024-09-16T09:45:43.759393+0000","last_fullsized":"2024-09-16T09:45:43.759393+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:29.454958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:44:29.454958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"73'79","reported_seq":1170,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:44.400276+0000","last_change":"2024-09-16T09:45:04.588099+0000","last_active":"2024-09-16T09:45:44.400276+0000","last_peered":"2024-09-16T09:45:44.400276+0000","last_clean":"2024-09-16T09:45:44.400276+0000","last_became_active":"2024-09-16T09:45:03.610800+0000","last_became_peered":"2024-09-16T09:45:03.610800+0000","last_unstale":"2024-09-16T09:45:44.400276+0000","last_undegraded":"2024-09-16T09:45:44.400276+0000","last_fullsized":"2024-09-16T09:45:44.400276+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":543,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:45:04.588014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:45:04.588014+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:21.940905+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"547'30","reported_seq":771,"reported_epoch":570,"state":"active+clean","last_fresh":"2024-09-16T09:46:03.529760+0000","last_change":"2024-09-16T09:45:19.683495+0000","last_active":"2024-09-16T09:46:03.529760+0000","last_peered":"2024-09-16T09:46:03.529760+0000","last_clean":"2024-09-16T09:46:03.529760+0000","last_became_active":"2024-09-16T09:45:19.683142+0000","last_became_peered":"2024-09-16T09:45:19.683142+0000","last_unstale":"2024-09-16T09:46:03.529760+0000","last_undegraded":"2024-09-16T09:46:03.529760+0000","last_fullsized":"2024-09-16T09:46:03.529760+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"547'30","last_scrub_stamp":"2024-09-16T09:45:13.803090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:45:13.803090+0000","objects_scrubbed":5,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:40.288842+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":43,"num_read_kb":38,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":1103,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:43.818502+0000","last_change":"2024-09-16T09:45:43.818419+0000","last_active":"2024-09-16T09:45:43.818502+0000","last_peered":"2024-09-16T09:45:43.818502+0000","last_clean":"2024-09-16T09:45:43.818502+0000","last_became_active":"2024-09-16T09:44:24.280657+0000","last_became_peered":"2024-09-16T09:44:24.280657+0000","last_unstale":"2024-09-16T09:45:43.818502+0000","last_undegraded":"2024-09-16T09:45:43.818502+0000","last_fullsized":"2024-09-16T09:45:43.818502+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:43.818349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:45:43.818349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:44.554617+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"66'8","reported_seq":1344,"reported_epoch":571,"state":"active+clean","last_fresh":"2024-09-16T09:46:04.854612+0000","last_change":"2024-09-16T09:46:02.085651+0000","last_active":"2024-09-16T09:46:04.854612+0000","last_peered":"2024-09-16T09:46:04.854612+0000","last_clean":"2024-09-16T09:46:04.854612+0000","last_became_active":"2024-09-16T09:25:05.714507+0000","last_became_peered":"2024-09-16T09:25:05.714507+0000","last_unstale":"2024-09-16T09:46:04.854612+0000","last_undegraded":"2024-09-16T09:46:04.854612+0000","last_fullsized":"2024-09-16T09:46:04.854612+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:46:02.085602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:46:02.085602+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:47:17.957744+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1104,"reported_epoch":571,"state":"active+clean","last_fresh":"2024-09-16T09:46:04.854284+0000","last_change":"2024-09-16T09:44:25.125306+0000","last_active":"2024-09-16T09:46:04.854284+0000","last_peered":"2024-09-16T09:46:04.854284+0000","last_clean":"2024-09-16T09:46:04.854284+0000","last_became_active":"2024-09-16T09:44:24.269736+0000","last_became_peered":"2024-09-16T09:44:24.269736+0000","last_unstale":"2024-09-16T09:46:04.854284+0000","last_undegraded":"2024-09-16T09:46:04.854284+0000","last_fullsized":"2024-09-16T09:46:04.854284+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:25.125254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:44:25.125254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"73'79","reported_seq":1127,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:44.400205+0000","last_change":"2024-09-16T09:45:03.617751+0000","last_active":"2024-09-16T09:45:44.400205+0000","last_peered":"2024-09-16T09:45:44.400205+0000","last_clean":"2024-09-16T09:45:44.400205+0000","last_became_active":"2024-09-16T09:45:03.610983+0000","last_became_peered":"2024-09-16T09:45:03.610983+0000","last_unstale":"2024-09-16T09:45:44.400205+0000","last_undegraded":"2024-09-16T09:45:44.400205+0000","last_fullsized":"2024-09-16T09:45:44.400205+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":543,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:45:03.617698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:45:03.617698+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:04.673887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"547'39","reported_seq":1145,"reported_epoch":571,"state":"active+clean","last_fresh":"2024-09-16T09:46:04.854829+0000","last_change":"2024-09-16T09:45:37.967154+0000","last_active":"2024-09-16T09:46:04.854829+0000","last_peered":"2024-09-16T09:46:04.854829+0000","last_clean":"2024-09-16T09:46:04.854829+0000","last_became_active":"2024-09-16T09:25:03.714797+0000","last_became_peered":"2024-09-16T09:25:03.714797+0000","last_unstale":"2024-09-16T09:46:04.854829+0000","last_undegraded":"2024-09-16T09:46:04.854829+0000","last_fullsized":"2024-09-16T09:46:04.854829+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"547'39","last_scrub_stamp":"2024-09-16T09:45:37.967097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:45:37.967097+0000","objects_scrubbed":8,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:47:00.901631+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":69,"num_read_kb":55,"num_write":42,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":649,"reported_epoch":570,"state":"active+clean","last_fresh":"2024-09-16T09:46:03.530482+0000","last_change":"2024-09-16T09:45:50.431119+0000","last_active":"2024-09-16T09:46:03.530482+0000","last_peered":"2024-09-16T09:46:03.530482+0000","last_clean":"2024-09-16T09:46:03.530482+0000","last_became_active":"2024-09-16T09:45:19.685947+0000","last_became_peered":"2024-09-16T09:45:19.685947+0000","last_unstale":"2024-09-16T09:46:03.530482+0000","last_undegraded":"2024-09-16T09:46:03.530482+0000","last_fullsized":"2024-09-16T09:46:03.530482+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:50.431069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:45:50.431069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:56.106163+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1097,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:44.400198+0000","last_change":"2024-09-16T09:44:51.704537+0000","last_active":"2024-09-16T09:45:44.400198+0000","last_peered":"2024-09-16T09:45:44.400198+0000","last_clean":"2024-09-16T09:45:44.400198+0000","last_became_active":"2024-09-16T09:25:05.708663+0000","last_became_peered":"2024-09-16T09:25:05.708663+0000","last_unstale":"2024-09-16T09:45:44.400198+0000","last_undegraded":"2024-09-16T09:45:44.400198+0000","last_fullsized":"2024-09-16T09:45:44.400198+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:51.704482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:44:51.704482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:16.177847+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":657,"reported_epoch":570,"state":"active+clean","last_fresh":"2024-09-16T09:46:03.531020+0000","last_change":"2024-09-16T09:45:43.501650+0000","last_active":"2024-09-16T09:46:03.531020+0000","last_peered":"2024-09-16T09:46:03.531020+0000","last_clean":"2024-09-16T09:46:03.531020+0000","last_became_active":"2024-09-16T09:45:19.681194+0000","last_became_peered":"2024-09-16T09:45:19.681194+0000","last_unstale":"2024-09-16T09:46:03.531020+0000","last_undegraded":"2024-09-16T09:46:03.531020+0000","last_fullsized":"2024-09-16T09:46:03.531020+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:43.501604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:45:43.501604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:47:11.591957+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"73'79","reported_seq":1064,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:58.611298+0000","last_change":"2024-09-16T09:45:58.611298+0000","last_active":"2024-09-16T09:45:58.611298+0000","last_peered":"2024-09-16T09:45:58.611298+0000","last_clean":"2024-09-16T09:45:58.611298+0000","last_became_active":"2024-09-16T09:44:24.279593+0000","last_became_peered":"2024-09-16T09:44:24.279593+0000","last_unstale":"2024-09-16T09:45:58.611298+0000","last_undegraded":"2024-09-16T09:45:58.611298+0000","last_fullsized":"2024-09-16T09:45:58.611298+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:45:58.611239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:45:58.611239+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:47:23.896677+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"547'56","reported_seq":1519,"reported_epoch":570,"state":"active+clean","last_fresh":"2024-09-16T09:46:03.530412+0000","last_change":"2024-09-16T09:45:46.472053+0000","last_active":"2024-09-16T09:46:03.530412+0000","last_peered":"2024-09-16T09:46:03.530412+0000","last_clean":"2024-09-16T09:46:03.530412+0000","last_became_active":"2024-09-16T09:44:24.280130+0000","last_became_peered":"2024-09-16T09:44:24.280130+0000","last_unstale":"2024-09-16T09:46:03.530412+0000","last_undegraded":"2024-09-16T09:46:03.530412+0000","last_fullsized":"2024-09-16T09:46:03.530412+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"547'56","last_scrub_stamp":"2024-09-16T09:45:46.471974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:45:46.471974+0000","objects_scrubbed":10,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:47:03.561041+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":84,"num_read_kb":69,"num_write":50,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":1095,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:44.127676+0000","last_change":"2024-09-16T09:45:10.214049+0000","last_active":"2024-09-16T09:45:44.127676+0000","last_peered":"2024-09-16T09:45:44.127676+0000","last_clean":"2024-09-16T09:45:44.127676+0000","last_became_active":"2024-09-16T09:25:01.703076+0000","last_became_peered":"2024-09-16T09:25:01.703076+0000","last_unstale":"2024-09-16T09:45:44.127676+0000","last_undegraded":"2024-09-16T09:45:44.127676+0000","last_fullsized":"2024-09-16T09:45:44.127676+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:10.214003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:45:10.214003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:38.392450+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":1104,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:43.756853+0000","last_change":"2024-09-16T09:45:38.729856+0000","last_active":"2024-09-16T09:45:43.756853+0000","last_peered":"2024-09-16T09:45:43.756853+0000","last_clean":"2024-09-16T09:45:43.756853+0000","last_became_active":"2024-09-16T09:25:00.696237+0000","last_became_peered":"2024-09-16T09:25:00.696237+0000","last_unstale":"2024-09-16T09:45:43.756853+0000","last_undegraded":"2024-09-16T09:45:43.756853+0000","last_fullsized":"2024-09-16T09:45:43.756853+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-16T09:45:38.729781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:45:38.729781+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:54.699785+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.0025137430000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1075,"reported_epoch":571,"state":"active+clean","last_fresh":"2024-09-16T09:46:04.854766+0000","last_change":"2024-09-16T09:46:04.062868+0000","last_active":"2024-09-16T09:46:04.854766+0000","last_peered":"2024-09-16T09:46:04.854766+0000","last_clean":"2024-09-16T09:46:04.854766+0000","last_became_active":"2024-09-16T09:25:05.717591+0000","last_became_peered":"2024-09-16T09:25:05.717591+0000","last_unstale":"2024-09-16T09:46:04.854766+0000","last_undegraded":"2024-09-16T09:46:04.854766+0000","last_fullsized":"2024-09-16T09:46:04.854766+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:46:04.062821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:46:04.062821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:47:26.675568+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":815,"reported_epoch":570,"state":"active+clean","last_fresh":"2024-09-16T09:46:03.604670+0000","last_change":"2024-09-16T09:44:35.215569+0000","last_active":"2024-09-16T09:46:03.604670+0000","last_peered":"2024-09-16T09:46:03.604670+0000","last_clean":"2024-09-16T09:46:03.604670+0000","last_became_active":"2024-09-16T09:31:09.272609+0000","last_became_peered":"2024-09-16T09:31:09.272609+0000","last_unstale":"2024-09-16T09:46:03.604670+0000","last_undegraded":"2024-09-16T09:46:03.604670+0000","last_fullsized":"2024-09-16T09:46:03.604670+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:35.215527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:44:35.215527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"73'79","reported_seq":420,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:55.623249+0000","last_change":"2024-09-16T09:45:55.623249+0000","last_active":"2024-09-16T09:45:55.623249+0000","last_peered":"2024-09-16T09:45:55.623249+0000","last_clean":"2024-09-16T09:45:55.623249+0000","last_became_active":"2024-09-16T09:38:10.556320+0000","last_became_peered":"2024-09-16T09:38:10.556320+0000","last_unstale":"2024-09-16T09:45:55.623249+0000","last_undegraded":"2024-09-16T09:45:55.623249+0000","last_fullsized":"2024-09-16T09:45:55.623249+0000","mapping_epoch":386,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":387,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:45:55.623189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:45:55.623189+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:47:15.053219+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"547'21","reported_seq":819,"reported_epoch":570,"state":"active+clean","last_fresh":"2024-09-16T09:46:03.529514+0000","last_change":"2024-09-16T09:45:19.684999+0000","last_active":"2024-09-16T09:46:03.529514+0000","last_peered":"2024-09-16T09:46:03.529514+0000","last_clean":"2024-09-16T09:46:03.529514+0000","last_became_active":"2024-09-16T09:45:19.684353+0000","last_became_peered":"2024-09-16T09:45:19.684353+0000","last_unstale":"2024-09-16T09:46:03.529514+0000","last_undegraded":"2024-09-16T09:46:03.529514+0000","last_fullsized":"2024-09-16T09:46:03.529514+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'17","last_scrub_stamp":"2024-09-16T09:44:59.781721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:44:59.781721+0000","objects_scrubbed":5,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:10.988145+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":40,"num_read_kb":29,"num_write":24,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":657,"reported_epoch":570,"state":"active+clean","last_fresh":"2024-09-16T09:46:03.529111+0000","last_change":"2024-09-16T09:45:47.427056+0000","last_active":"2024-09-16T09:46:03.529111+0000","last_peered":"2024-09-16T09:46:03.529111+0000","last_clean":"2024-09-16T09:46:03.529111+0000","last_became_active":"2024-09-16T09:45:19.680500+0000","last_became_peered":"2024-09-16T09:45:19.680500+0000","last_unstale":"2024-09-16T09:46:03.529111+0000","last_undegraded":"2024-09-16T09:46:03.529111+0000","last_fullsized":"2024-09-16T09:46:03.529111+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:47.427009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:45:47.427009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:52.111478+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":589,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:43.766123+0000","last_change":"2024-09-16T09:45:43.765136+0000","last_active":"2024-09-16T09:45:43.766123+0000","last_peered":"2024-09-16T09:45:43.766123+0000","last_clean":"2024-09-16T09:45:43.766123+0000","last_became_active":"2024-09-16T09:45:43.764803+0000","last_became_peered":"2024-09-16T09:45:43.764803+0000","last_unstale":"2024-09-16T09:45:43.766123+0000","last_undegraded":"2024-09-16T09:45:43.766123+0000","last_fullsized":"2024-09-16T09:45:43.766123+0000","mapping_epoch":568,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":569,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:45:08.290225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:45:08.290225+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:36.064458+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1013,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:43.758119+0000","last_change":"2024-09-16T09:45:42.155707+0000","last_active":"2024-09-16T09:45:43.758119+0000","last_peered":"2024-09-16T09:45:43.758119+0000","last_clean":"2024-09-16T09:45:43.758119+0000","last_became_active":"2024-09-16T09:44:24.285805+0000","last_became_peered":"2024-09-16T09:44:24.285805+0000","last_unstale":"2024-09-16T09:45:43.758119+0000","last_undegraded":"2024-09-16T09:45:43.758119+0000","last_fullsized":"2024-09-16T09:45:43.758119+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:42.155659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:45:42.155659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:48.222155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1072,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:43.760097+0000","last_change":"2024-09-16T09:44:48.555675+0000","last_active":"2024-09-16T09:45:43.760097+0000","last_peered":"2024-09-16T09:45:43.760097+0000","last_clean":"2024-09-16T09:45:43.760097+0000","last_became_active":"2024-09-16T09:25:07.723282+0000","last_became_peered":"2024-09-16T09:25:07.723282+0000","last_unstale":"2024-09-16T09:45:43.760097+0000","last_undegraded":"2024-09-16T09:45:43.760097+0000","last_fullsized":"2024-09-16T09:45:43.760097+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:48.555614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:44:48.555614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:18.258254+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"73'79","reported_seq":394,"reported_epoch":570,"state":"active+clean","last_fresh":"2024-09-16T09:46:03.530039+0000","last_change":"2024-09-16T09:45:35.495131+0000","last_active":"2024-09-16T09:46:03.530039+0000","last_peered":"2024-09-16T09:46:03.530039+0000","last_clean":"2024-09-16T09:46:03.530039+0000","last_became_active":"2024-09-16T09:45:19.680539+0000","last_became_peered":"2024-09-16T09:45:19.680539+0000","last_unstale":"2024-09-16T09:46:03.530039+0000","last_undegraded":"2024-09-16T09:46:03.530039+0000","last_fullsized":"2024-09-16T09:46:03.530039+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:45:35.495071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:45:35.495071+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:59.621866+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"547'36","reported_seq":1151,"reported_epoch":571,"state":"active+clean","last_fresh":"2024-09-16T09:46:04.854941+0000","last_change":"2024-09-16T09:45:26.063751+0000","last_active":"2024-09-16T09:46:04.854941+0000","last_peered":"2024-09-16T09:46:04.854941+0000","last_clean":"2024-09-16T09:46:04.854941+0000","last_became_active":"2024-09-16T09:44:24.277878+0000","last_became_peered":"2024-09-16T09:44:24.277878+0000","last_unstale":"2024-09-16T09:46:04.854941+0000","last_undegraded":"2024-09-16T09:46:04.854941+0000","last_fullsized":"2024-09-16T09:46:04.854941+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"547'36","last_scrub_stamp":"2024-09-16T09:45:26.063692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:45:26.063692+0000","objects_scrubbed":6,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:27.043442+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":54,"num_read_kb":48,"num_write":36,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"59'1","reported_seq":1122,"reported_epoch":571,"state":"active+clean","last_fresh":"2024-09-16T09:46:04.854330+0000","last_change":"2024-09-16T09:45:51.102399+0000","last_active":"2024-09-16T09:46:04.854330+0000","last_peered":"2024-09-16T09:46:04.854330+0000","last_clean":"2024-09-16T09:46:04.854330+0000","last_became_active":"2024-09-16T09:31:09.269627+0000","last_became_peered":"2024-09-16T09:31:09.269627+0000","last_unstale":"2024-09-16T09:46:04.854330+0000","last_undegraded":"2024-09-16T09:46:04.854330+0000","last_fullsized":"2024-09-16T09:46:04.854330+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:45:51.102311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:45:51.102311+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:47:16.896051+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1986,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"67'5","reported_seq":2349,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:46:00.843697+0000","last_change":"2024-09-16T09:45:43.482540+0000","last_active":"2024-09-16T09:46:00.843697+0000","last_peered":"2024-09-16T09:46:00.843697+0000","last_clean":"2024-09-16T09:46:00.843697+0000","last_became_active":"2024-09-16T09:25:00.696546+0000","last_became_peered":"2024-09-16T09:25:00.696546+0000","last_unstale":"2024-09-16T09:46:00.843697+0000","last_undegraded":"2024-09-16T09:46:00.843697+0000","last_fullsized":"2024-09-16T09:46:00.843697+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'5","last_scrub_stamp":"2024-09-16T09:45:43.482461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:45:43.482461+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:47:01.250445+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.0059770420000000001,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1253,"num_read_kb":1248,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1061,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:43.756930+0000","last_change":"2024-09-16T09:45:36.721532+0000","last_active":"2024-09-16T09:45:43.756930+0000","last_peered":"2024-09-16T09:45:43.756930+0000","last_clean":"2024-09-16T09:45:43.756930+0000","last_became_active":"2024-09-16T09:25:05.714704+0000","last_became_peered":"2024-09-16T09:25:05.714704+0000","last_unstale":"2024-09-16T09:45:43.756930+0000","last_undegraded":"2024-09-16T09:45:43.756930+0000","last_fullsized":"2024-09-16T09:45:43.756930+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:36.721477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:45:36.721477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:47:04.067390+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":435,"reported_epoch":570,"state":"active+clean","last_fresh":"2024-09-16T09:46:03.530186+0000","last_change":"2024-09-16T09:45:37.490228+0000","last_active":"2024-09-16T09:46:03.530186+0000","last_peered":"2024-09-16T09:46:03.530186+0000","last_clean":"2024-09-16T09:46:03.530186+0000","last_became_active":"2024-09-16T09:45:19.685809+0000","last_became_peered":"2024-09-16T09:45:19.685809+0000","last_unstale":"2024-09-16T09:46:03.530186+0000","last_undegraded":"2024-09-16T09:46:03.530186+0000","last_fullsized":"2024-09-16T09:46:03.530186+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:37.490179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:45:37.490179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:56.293422+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"547'40","reported_seq":1404,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:58.303042+0000","last_change":"2024-09-16T09:45:27.591449+0000","last_active":"2024-09-16T09:45:58.303042+0000","last_peered":"2024-09-16T09:45:58.303042+0000","last_clean":"2024-09-16T09:45:58.303042+0000","last_became_active":"2024-09-16T09:25:03.714118+0000","last_became_peered":"2024-09-16T09:25:03.714118+0000","last_unstale":"2024-09-16T09:45:58.303042+0000","last_undegraded":"2024-09-16T09:45:58.303042+0000","last_fullsized":"2024-09-16T09:45:58.303042+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"547'40","last_scrub_stamp":"2024-09-16T09:45:27.591380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:45:27.591380+0000","objects_scrubbed":8,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:57.224993+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":61,"num_read_kb":51,"num_write":37,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":169,"reported_epoch":570,"state":"active+clean","last_fresh":"2024-09-16T09:46:03.603044+0000","last_change":"2024-09-16T09:46:00.862416+0000","last_active":"2024-09-16T09:46:03.603044+0000","last_peered":"2024-09-16T09:46:03.603044+0000","last_clean":"2024-09-16T09:46:03.603044+0000","last_became_active":"2024-09-16T09:45:23.745852+0000","last_became_peered":"2024-09-16T09:45:23.745852+0000","last_unstale":"2024-09-16T09:46:03.603044+0000","last_undegraded":"2024-09-16T09:46:03.603044+0000","last_fullsized":"2024-09-16T09:46:03.603044+0000","mapping_epoch":554,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":555,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:46:00.862359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:46:00.862359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:47:12.694803+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"73'79","reported_seq":1169,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:43.759508+0000","last_change":"2024-09-16T09:45:28.555936+0000","last_active":"2024-09-16T09:45:43.759508+0000","last_peered":"2024-09-16T09:45:43.759508+0000","last_clean":"2024-09-16T09:45:43.759508+0000","last_became_active":"2024-09-16T09:30:08.870394+0000","last_became_peered":"2024-09-16T09:30:08.870394+0000","last_unstale":"2024-09-16T09:45:43.759508+0000","last_undegraded":"2024-09-16T09:45:43.759508+0000","last_fullsized":"2024-09-16T09:45:43.759508+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:45:28.555859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:45:28.555859+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:29.632294+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1088,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:43.760054+0000","last_change":"2024-09-16T09:45:01.668750+0000","last_active":"2024-09-16T09:45:43.760054+0000","last_peered":"2024-09-16T09:45:43.760054+0000","last_clean":"2024-09-16T09:45:43.760054+0000","last_became_active":"2024-09-16T09:44:24.284362+0000","last_became_peered":"2024-09-16T09:44:24.284362+0000","last_unstale":"2024-09-16T09:45:43.760054+0000","last_undegraded":"2024-09-16T09:45:43.760054+0000","last_fullsized":"2024-09-16T09:45:43.760054+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:01.668579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:45:01.668579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:28.528446+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1065,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:43.756798+0000","last_change":"2024-09-16T09:44:46.680178+0000","last_active":"2024-09-16T09:45:43.756798+0000","last_peered":"2024-09-16T09:45:43.756798+0000","last_clean":"2024-09-16T09:45:43.756798+0000","last_became_active":"2024-09-16T09:25:07.714281+0000","last_became_peered":"2024-09-16T09:25:07.714281+0000","last_unstale":"2024-09-16T09:45:43.756798+0000","last_undegraded":"2024-09-16T09:45:43.756798+0000","last_fullsized":"2024-09-16T09:45:43.756798+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:46.680120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:44:46.680120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:08.327463+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.b","version":"73'79","reported_seq":405,"reported_epoch":570,"state":"active+clean","last_fresh":"2024-09-16T09:46:03.529691+0000","last_change":"2024-09-16T09:45:22.401820+0000","last_active":"2024-09-16T09:46:03.529691+0000","last_peered":"2024-09-16T09:46:03.529691+0000","last_clean":"2024-09-16T09:46:03.529691+0000","last_became_active":"2024-09-16T09:45:19.682878+0000","last_became_peered":"2024-09-16T09:45:19.682878+0000","last_unstale":"2024-09-16T09:46:03.529691+0000","last_undegraded":"2024-09-16T09:46:03.529691+0000","last_fullsized":"2024-09-16T09:46:03.529691+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:45:22.401770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:45:22.401770+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:44.196429+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,5],"acting":[1,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"547'27","reported_seq":1174,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:44.128054+0000","last_change":"2024-09-16T09:45:37.085157+0000","last_active":"2024-09-16T09:45:44.128054+0000","last_peered":"2024-09-16T09:45:44.128054+0000","last_clean":"2024-09-16T09:45:44.128054+0000","last_became_active":"2024-09-16T09:37:33.348433+0000","last_became_peered":"2024-09-16T09:37:33.348433+0000","last_unstale":"2024-09-16T09:45:44.128054+0000","last_undegraded":"2024-09-16T09:45:44.128054+0000","last_fullsized":"2024-09-16T09:45:44.128054+0000","mapping_epoch":367,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":368,"parent":"0.0","parent_split_bits":0,"last_scrub":"547'27","last_scrub_stamp":"2024-09-16T09:45:37.085092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:45:37.085092+0000","objects_scrubbed":5,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:44.425422+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":37,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":1081,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:44.128083+0000","last_change":"2024-09-16T09:45:20.179511+0000","last_active":"2024-09-16T09:45:44.128083+0000","last_peered":"2024-09-16T09:45:44.128083+0000","last_clean":"2024-09-16T09:45:44.128083+0000","last_became_active":"2024-09-16T09:44:24.276997+0000","last_became_peered":"2024-09-16T09:44:24.276997+0000","last_unstale":"2024-09-16T09:45:44.128083+0000","last_undegraded":"2024-09-16T09:45:44.128083+0000","last_fullsized":"2024-09-16T09:45:44.128083+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:20.179440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:45:20.179440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:23.557282+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1053,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:43.756623+0000","last_change":"2024-09-16T09:45:22.786814+0000","last_active":"2024-09-16T09:45:43.756623+0000","last_peered":"2024-09-16T09:45:43.756623+0000","last_clean":"2024-09-16T09:45:43.756623+0000","last_became_active":"2024-09-16T09:25:05.711964+0000","last_became_peered":"2024-09-16T09:25:05.711964+0000","last_unstale":"2024-09-16T09:45:43.756623+0000","last_undegraded":"2024-09-16T09:45:43.756623+0000","last_fullsized":"2024-09-16T09:45:43.756623+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:22.786754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:45:22.786754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:35.253260+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":178,"reported_epoch":570,"state":"active+clean","last_fresh":"2024-09-16T09:46:03.604876+0000","last_change":"2024-09-16T09:45:23.746036+0000","last_active":"2024-09-16T09:46:03.604876+0000","last_peered":"2024-09-16T09:46:03.604876+0000","last_clean":"2024-09-16T09:46:03.604876+0000","last_became_active":"2024-09-16T09:45:23.745723+0000","last_became_peered":"2024-09-16T09:45:23.745723+0000","last_unstale":"2024-09-16T09:46:03.604876+0000","last_undegraded":"2024-09-16T09:46:03.604876+0000","last_fullsized":"2024-09-16T09:46:03.604876+0000","mapping_epoch":554,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":555,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:09.717025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:45:09.717025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:17.342433+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.8","version":"73'79","reported_seq":441,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:43.755612+0000","last_change":"2024-09-16T09:45:04.792552+0000","last_active":"2024-09-16T09:45:43.755612+0000","last_peered":"2024-09-16T09:45:43.755612+0000","last_clean":"2024-09-16T09:45:43.755612+0000","last_became_active":"2024-09-16T09:45:03.608887+0000","last_became_peered":"2024-09-16T09:45:03.608887+0000","last_unstale":"2024-09-16T09:45:43.755612+0000","last_undegraded":"2024-09-16T09:45:43.755612+0000","last_fullsized":"2024-09-16T09:45:43.755612+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":543,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:45:04.792500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:45:04.792500+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:21.463945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"547'45","reported_seq":1163,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:44.128297+0000","last_change":"2024-09-16T09:45:24.217342+0000","last_active":"2024-09-16T09:45:44.128297+0000","last_peered":"2024-09-16T09:45:44.128297+0000","last_clean":"2024-09-16T09:45:44.128297+0000","last_became_active":"2024-09-16T09:44:24.277359+0000","last_became_peered":"2024-09-16T09:44:24.277359+0000","last_unstale":"2024-09-16T09:45:44.128297+0000","last_undegraded":"2024-09-16T09:45:44.128297+0000","last_fullsized":"2024-09-16T09:45:44.128297+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"547'45","last_scrub_stamp":"2024-09-16T09:45:24.217274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:45:24.217274+0000","objects_scrubbed":8,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:47.371200+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":71,"num_read_kb":61,"num_write":46,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":1088,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:44.400418+0000","last_change":"2024-09-16T09:45:11.709864+0000","last_active":"2024-09-16T09:45:44.400418+0000","last_peered":"2024-09-16T09:45:44.400418+0000","last_clean":"2024-09-16T09:45:44.400418+0000","last_became_active":"2024-09-16T09:25:01.703355+0000","last_became_peered":"2024-09-16T09:25:01.703355+0000","last_unstale":"2024-09-16T09:45:44.400418+0000","last_undegraded":"2024-09-16T09:45:44.400418+0000","last_fullsized":"2024-09-16T09:45:44.400418+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:11.709798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:45:11.709798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:31.626577+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":424,"reported_epoch":570,"state":"active+clean","last_fresh":"2024-09-16T09:46:03.530197+0000","last_change":"2024-09-16T09:45:19.683602+0000","last_active":"2024-09-16T09:46:03.530197+0000","last_peered":"2024-09-16T09:46:03.530197+0000","last_clean":"2024-09-16T09:46:03.530197+0000","last_became_active":"2024-09-16T09:45:19.683227+0000","last_became_peered":"2024-09-16T09:45:19.683227+0000","last_unstale":"2024-09-16T09:46:03.530197+0000","last_undegraded":"2024-09-16T09:46:03.530197+0000","last_fullsized":"2024-09-16T09:46:03.530197+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:47.324096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:44:47.324096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:08.892743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":918,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:45.127718+0000","last_change":"2024-09-16T09:45:45.127669+0000","last_active":"2024-09-16T09:45:45.127718+0000","last_peered":"2024-09-16T09:45:45.127718+0000","last_clean":"2024-09-16T09:45:45.127718+0000","last_became_active":"2024-09-16T09:45:44.127833+0000","last_became_peered":"2024-09-16T09:45:44.127833+0000","last_unstale":"2024-09-16T09:45:45.127718+0000","last_undegraded":"2024-09-16T09:45:45.127718+0000","last_fullsized":"2024-09-16T09:45:45.127718+0000","mapping_epoch":568,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":569,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:45.127611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:45:45.127611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:47:11.070158+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.9","version":"73'79","reported_seq":1170,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:43.759750+0000","last_change":"2024-09-16T09:45:04.693404+0000","last_active":"2024-09-16T09:45:43.759750+0000","last_peered":"2024-09-16T09:45:43.759750+0000","last_clean":"2024-09-16T09:45:43.759750+0000","last_became_active":"2024-09-16T09:45:03.614687+0000","last_became_peered":"2024-09-16T09:45:03.614687+0000","last_unstale":"2024-09-16T09:45:43.759750+0000","last_undegraded":"2024-09-16T09:45:43.759750+0000","last_fullsized":"2024-09-16T09:45:43.759750+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":543,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:45:04.693352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:45:04.693352+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:29.565272+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"547'30","reported_seq":1119,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:47.195286+0000","last_change":"2024-09-16T09:45:47.195286+0000","last_active":"2024-09-16T09:45:47.195286+0000","last_peered":"2024-09-16T09:45:47.195286+0000","last_clean":"2024-09-16T09:45:47.195286+0000","last_became_active":"2024-09-16T09:25:03.707723+0000","last_became_peered":"2024-09-16T09:25:03.707723+0000","last_unstale":"2024-09-16T09:45:47.195286+0000","last_undegraded":"2024-09-16T09:45:47.195286+0000","last_fullsized":"2024-09-16T09:45:47.195286+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"547'30","last_scrub_stamp":"2024-09-16T09:45:47.195208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:45:47.195208+0000","objects_scrubbed":5,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:47:06.380855+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":45,"num_read_kb":40,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":177,"reported_epoch":570,"state":"active+clean","last_fresh":"2024-09-16T09:46:03.604184+0000","last_change":"2024-09-16T09:45:34.050738+0000","last_active":"2024-09-16T09:46:03.604184+0000","last_peered":"2024-09-16T09:46:03.604184+0000","last_clean":"2024-09-16T09:46:03.604184+0000","last_became_active":"2024-09-16T09:45:23.695995+0000","last_became_peered":"2024-09-16T09:45:23.695995+0000","last_unstale":"2024-09-16T09:46:03.604184+0000","last_undegraded":"2024-09-16T09:46:03.604184+0000","last_fullsized":"2024-09-16T09:46:03.604184+0000","mapping_epoch":554,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":555,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:34.050691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:45:34.050691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:42.361284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"569'12","reported_seq":806,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:56.994782+0000","last_change":"2024-09-16T09:45:44.112875+0000","last_active":"2024-09-16T09:45:56.994782+0000","last_peered":"2024-09-16T09:45:56.994782+0000","last_clean":"2024-09-16T09:45:56.994782+0000","last_became_active":"2024-09-16T09:45:43.760279+0000","last_became_peered":"2024-09-16T09:45:43.760279+0000","last_unstale":"2024-09-16T09:45:56.994782+0000","last_undegraded":"2024-09-16T09:45:56.994782+0000","last_fullsized":"2024-09-16T09:45:56.994782+0000","mapping_epoch":568,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":569,"parent":"0.0","parent_split_bits":0,"last_scrub":"519'11","last_scrub_stamp":"2024-09-16T09:45:44.112817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:45:44.112817+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:47:07.166260+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1101,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:44.127879+0000","last_change":"2024-09-16T09:44:48.293840+0000","last_active":"2024-09-16T09:45:44.127879+0000","last_peered":"2024-09-16T09:45:44.127879+0000","last_clean":"2024-09-16T09:45:44.127879+0000","last_became_active":"2024-09-16T09:44:24.280724+0000","last_became_peered":"2024-09-16T09:44:24.280724+0000","last_unstale":"2024-09-16T09:45:44.127879+0000","last_undegraded":"2024-09-16T09:45:44.127879+0000","last_fullsized":"2024-09-16T09:45:44.127879+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:48.293789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:44:48.293789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:57.968474+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.e","version":"73'79","reported_seq":885,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:44.127779+0000","last_change":"2024-09-16T09:45:05.247594+0000","last_active":"2024-09-16T09:45:44.127779+0000","last_peered":"2024-09-16T09:45:44.127779+0000","last_clean":"2024-09-16T09:45:44.127779+0000","last_became_active":"2024-09-16T09:45:03.613758+0000","last_became_peered":"2024-09-16T09:45:03.613758+0000","last_unstale":"2024-09-16T09:45:44.127779+0000","last_undegraded":"2024-09-16T09:45:44.127779+0000","last_fullsized":"2024-09-16T09:45:44.127779+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":543,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:45:05.247539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:45:05.247539+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:30.185192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,3],"acting":[4,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.b","version":"547'21","reported_seq":1137,"reported_epoch":571,"state":"active+clean","last_fresh":"2024-09-16T09:46:04.854741+0000","last_change":"2024-09-16T09:44:48.083967+0000","last_active":"2024-09-16T09:46:04.854741+0000","last_peered":"2024-09-16T09:46:04.854741+0000","last_clean":"2024-09-16T09:46:04.854741+0000","last_became_active":"2024-09-16T09:36:31.590202+0000","last_became_peered":"2024-09-16T09:36:31.590202+0000","last_unstale":"2024-09-16T09:46:04.854741+0000","last_undegraded":"2024-09-16T09:46:04.854741+0000","last_fullsized":"2024-09-16T09:46:04.854741+0000","mapping_epoch":316,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":317,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'15","last_scrub_stamp":"2024-09-16T09:44:48.083895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:44:48.083895+0000","objects_scrubbed":4,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":1083,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:43.756347+0000","last_change":"2024-09-16T09:45:32.711419+0000","last_active":"2024-09-16T09:45:43.756347+0000","last_peered":"2024-09-16T09:45:43.756347+0000","last_clean":"2024-09-16T09:45:43.756347+0000","last_became_active":"2024-09-16T09:31:09.276516+0000","last_became_peered":"2024-09-16T09:31:09.276516+0000","last_unstale":"2024-09-16T09:45:43.756347+0000","last_undegraded":"2024-09-16T09:45:43.756347+0000","last_fullsized":"2024-09-16T09:45:43.756347+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:32.711339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:45:32.711339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:40.490158+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":907,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:44.129043+0000","last_change":"2024-09-16T09:45:44.128762+0000","last_active":"2024-09-16T09:45:44.129043+0000","last_peered":"2024-09-16T09:45:44.129043+0000","last_clean":"2024-09-16T09:45:44.129043+0000","last_became_active":"2024-09-16T09:45:44.128387+0000","last_became_peered":"2024-09-16T09:45:44.128387+0000","last_unstale":"2024-09-16T09:45:44.129043+0000","last_undegraded":"2024-09-16T09:45:44.129043+0000","last_fullsized":"2024-09-16T09:45:44.129043+0000","mapping_epoch":568,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":569,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:36.088166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:45:36.088166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:54.319195+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1089,"reported_epoch":571,"state":"active+clean","last_fresh":"2024-09-16T09:46:04.854286+0000","last_change":"2024-09-16T09:45:30.041583+0000","last_active":"2024-09-16T09:46:04.854286+0000","last_peered":"2024-09-16T09:46:04.854286+0000","last_clean":"2024-09-16T09:46:04.854286+0000","last_became_active":"2024-09-16T09:44:24.278714+0000","last_became_peered":"2024-09-16T09:44:24.278714+0000","last_unstale":"2024-09-16T09:46:04.854286+0000","last_undegraded":"2024-09-16T09:46:04.854286+0000","last_fullsized":"2024-09-16T09:46:04.854286+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:30.041535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:45:30.041535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:37.082005+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.f","version":"73'79","reported_seq":1149,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:44.400519+0000","last_change":"2024-09-16T09:45:25.579212+0000","last_active":"2024-09-16T09:45:44.400519+0000","last_peered":"2024-09-16T09:45:44.400519+0000","last_clean":"2024-09-16T09:45:44.400519+0000","last_became_active":"2024-09-16T09:30:10.693477+0000","last_became_peered":"2024-09-16T09:30:10.693477+0000","last_unstale":"2024-09-16T09:45:44.400519+0000","last_undegraded":"2024-09-16T09:45:44.400519+0000","last_fullsized":"2024-09-16T09:45:44.400519+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:45:25.579158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:45:25.579158+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:51.377246+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"547'51","reported_seq":1163,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:44.399274+0000","last_change":"2024-09-16T09:45:17.716010+0000","last_active":"2024-09-16T09:45:44.399274+0000","last_peered":"2024-09-16T09:45:44.399274+0000","last_clean":"2024-09-16T09:45:44.399274+0000","last_became_active":"2024-09-16T09:25:03.712783+0000","last_became_peered":"2024-09-16T09:25:03.712783+0000","last_unstale":"2024-09-16T09:45:44.399274+0000","last_undegraded":"2024-09-16T09:45:44.399274+0000","last_fullsized":"2024-09-16T09:45:44.399274+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"547'51","last_scrub_stamp":"2024-09-16T09:45:17.715920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:45:17.715920+0000","objects_scrubbed":9,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:47.081023+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":80,"num_read_kb":69,"num_write":52,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"59'1","reported_seq":1090,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:43.760425+0000","last_change":"2024-09-16T09:45:26.499588+0000","last_active":"2024-09-16T09:45:43.760425+0000","last_peered":"2024-09-16T09:45:43.760425+0000","last_clean":"2024-09-16T09:45:43.760425+0000","last_became_active":"2024-09-16T09:25:01.708069+0000","last_became_peered":"2024-09-16T09:25:01.708069+0000","last_unstale":"2024-09-16T09:45:43.760425+0000","last_undegraded":"2024-09-16T09:45:43.760425+0000","last_fullsized":"2024-09-16T09:45:43.760425+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:45:26.499523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:45:26.499523+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:39.186250+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1045,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:43.756939+0000","last_change":"2024-09-16T09:44:33.039012+0000","last_active":"2024-09-16T09:45:43.756939+0000","last_peered":"2024-09-16T09:45:43.756939+0000","last_clean":"2024-09-16T09:45:43.756939+0000","last_became_active":"2024-09-16T09:44:24.278290+0000","last_became_peered":"2024-09-16T09:44:24.278290+0000","last_unstale":"2024-09-16T09:45:43.756939+0000","last_undegraded":"2024-09-16T09:45:43.756939+0000","last_fullsized":"2024-09-16T09:45:43.756939+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:33.038961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:44:33.038961+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:58.372425+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1067,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:43.759394+0000","last_change":"2024-09-16T09:45:02.668718+0000","last_active":"2024-09-16T09:45:43.759394+0000","last_peered":"2024-09-16T09:45:43.759394+0000","last_clean":"2024-09-16T09:45:43.759394+0000","last_became_active":"2024-09-16T09:44:24.275407+0000","last_became_peered":"2024-09-16T09:44:24.275407+0000","last_unstale":"2024-09-16T09:45:43.759394+0000","last_undegraded":"2024-09-16T09:45:43.759394+0000","last_fullsized":"2024-09-16T09:45:43.759394+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:02.668668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:45:02.668668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:08.212308+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.c","version":"73'79","reported_seq":854,"reported_epoch":570,"state":"active+clean","last_fresh":"2024-09-16T09:46:03.604716+0000","last_change":"2024-09-16T09:45:05.176489+0000","last_active":"2024-09-16T09:46:03.604716+0000","last_peered":"2024-09-16T09:46:03.604716+0000","last_clean":"2024-09-16T09:46:03.604716+0000","last_became_active":"2024-09-16T09:45:03.613008+0000","last_became_peered":"2024-09-16T09:45:03.613008+0000","last_unstale":"2024-09-16T09:46:03.604716+0000","last_undegraded":"2024-09-16T09:46:03.604716+0000","last_fullsized":"2024-09-16T09:46:03.604716+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":543,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:45:05.176413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:45:05.176413+0000","objects_scrubbed":1,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:24.104658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"547'24","reported_seq":1161,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:44.127535+0000","last_change":"2024-09-16T09:45:22.192977+0000","last_active":"2024-09-16T09:45:44.127535+0000","last_peered":"2024-09-16T09:45:44.127535+0000","last_clean":"2024-09-16T09:45:44.127535+0000","last_became_active":"2024-09-16T09:31:09.269047+0000","last_became_peered":"2024-09-16T09:31:09.269047+0000","last_unstale":"2024-09-16T09:45:44.127535+0000","last_undegraded":"2024-09-16T09:45:44.127535+0000","last_fullsized":"2024-09-16T09:45:44.127535+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"547'24","last_scrub_stamp":"2024-09-16T09:45:22.192863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:45:22.192863+0000","objects_scrubbed":5,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:48.200457+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":1084,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:43.760487+0000","last_change":"2024-09-16T09:45:25.455157+0000","last_active":"2024-09-16T09:45:43.760487+0000","last_peered":"2024-09-16T09:45:43.760487+0000","last_clean":"2024-09-16T09:45:43.760487+0000","last_became_active":"2024-09-16T09:25:01.707528+0000","last_became_peered":"2024-09-16T09:25:01.707528+0000","last_unstale":"2024-09-16T09:45:43.760487+0000","last_undegraded":"2024-09-16T09:45:43.760487+0000","last_fullsized":"2024-09-16T09:45:43.760487+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:25.455094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:45:25.455094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:47.601779+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":641,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:43.766159+0000","last_change":"2024-09-16T09:45:43.765462+0000","last_active":"2024-09-16T09:45:43.766159+0000","last_peered":"2024-09-16T09:45:43.766159+0000","last_clean":"2024-09-16T09:45:43.766159+0000","last_became_active":"2024-09-16T09:45:43.765067+0000","last_became_peered":"2024-09-16T09:45:43.765067+0000","last_unstale":"2024-09-16T09:45:43.766159+0000","last_undegraded":"2024-09-16T09:45:43.766159+0000","last_fullsized":"2024-09-16T09:45:43.766159+0000","mapping_epoch":568,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":569,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:20.175105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:45:20.175105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:42.875854+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1078,"reported_epoch":570,"state":"active+clean","last_fresh":"2024-09-16T09:46:03.603908+0000","last_change":"2024-09-16T09:45:09.157493+0000","last_active":"2024-09-16T09:46:03.603908+0000","last_peered":"2024-09-16T09:46:03.603908+0000","last_clean":"2024-09-16T09:46:03.603908+0000","last_became_active":"2024-09-16T09:25:07.718899+0000","last_became_peered":"2024-09-16T09:25:07.718899+0000","last_unstale":"2024-09-16T09:46:03.603908+0000","last_undegraded":"2024-09-16T09:46:03.603908+0000","last_fullsized":"2024-09-16T09:46:03.603908+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:09.157451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:45:09.157451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:25.587581+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.d","version":"73'79","reported_seq":884,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:43.760027+0000","last_change":"2024-09-16T09:45:03.697649+0000","last_active":"2024-09-16T09:45:43.760027+0000","last_peered":"2024-09-16T09:45:43.760027+0000","last_clean":"2024-09-16T09:45:43.760027+0000","last_became_active":"2024-09-16T09:45:03.613387+0000","last_became_peered":"2024-09-16T09:45:03.613387+0000","last_unstale":"2024-09-16T09:45:43.760027+0000","last_undegraded":"2024-09-16T09:45:43.760027+0000","last_fullsized":"2024-09-16T09:45:43.760027+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":543,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:45:03.697560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:45:03.697560+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:10.630174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"547'39","reported_seq":1142,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:43.755786+0000","last_change":"2024-09-16T09:45:42.781753+0000","last_active":"2024-09-16T09:45:43.755786+0000","last_peered":"2024-09-16T09:45:43.755786+0000","last_clean":"2024-09-16T09:45:43.755786+0000","last_became_active":"2024-09-16T09:25:03.715733+0000","last_became_peered":"2024-09-16T09:25:03.715733+0000","last_unstale":"2024-09-16T09:45:43.755786+0000","last_undegraded":"2024-09-16T09:45:43.755786+0000","last_fullsized":"2024-09-16T09:45:43.755786+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"547'39","last_scrub_stamp":"2024-09-16T09:45:42.781672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:45:42.781672+0000","objects_scrubbed":7,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:57.480567+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":62,"num_read_kb":53,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":1088,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:43.760489+0000","last_change":"2024-09-16T09:45:21.533534+0000","last_active":"2024-09-16T09:45:43.760489+0000","last_peered":"2024-09-16T09:45:43.760489+0000","last_clean":"2024-09-16T09:45:43.760489+0000","last_became_active":"2024-09-16T09:25:01.707372+0000","last_became_peered":"2024-09-16T09:25:01.707372+0000","last_unstale":"2024-09-16T09:45:43.760489+0000","last_undegraded":"2024-09-16T09:45:43.760489+0000","last_fullsized":"2024-09-16T09:45:43.760489+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-16T09:45:21.533475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:45:21.533475+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:48.426116+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"66'8","reported_seq":1345,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:56.995143+0000","last_change":"2024-09-16T09:45:31.616897+0000","last_active":"2024-09-16T09:45:56.995143+0000","last_peered":"2024-09-16T09:45:56.995143+0000","last_clean":"2024-09-16T09:45:56.995143+0000","last_became_active":"2024-09-16T09:25:05.716189+0000","last_became_peered":"2024-09-16T09:25:05.716189+0000","last_unstale":"2024-09-16T09:45:56.995143+0000","last_undegraded":"2024-09-16T09:45:56.995143+0000","last_fullsized":"2024-09-16T09:45:56.995143+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:45:31.616835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:45:31.616835+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:36.164090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1065,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:46.781350+0000","last_change":"2024-09-16T09:45:46.781350+0000","last_active":"2024-09-16T09:45:46.781350+0000","last_peered":"2024-09-16T09:45:46.781350+0000","last_clean":"2024-09-16T09:45:46.781350+0000","last_became_active":"2024-09-16T09:25:07.717213+0000","last_became_peered":"2024-09-16T09:25:07.717213+0000","last_unstale":"2024-09-16T09:45:46.781350+0000","last_undegraded":"2024-09-16T09:45:46.781350+0000","last_fullsized":"2024-09-16T09:45:46.781350+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:46.781287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:45:46.781287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:49.691824+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.12","version":"73'79","reported_seq":886,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:43.756559+0000","last_change":"2024-09-16T09:45:01.754225+0000","last_active":"2024-09-16T09:45:43.756559+0000","last_peered":"2024-09-16T09:45:43.756559+0000","last_clean":"2024-09-16T09:45:43.756559+0000","last_became_active":"2024-09-16T09:30:10.731413+0000","last_became_peered":"2024-09-16T09:30:10.731413+0000","last_unstale":"2024-09-16T09:45:43.756559+0000","last_undegraded":"2024-09-16T09:45:43.756559+0000","last_fullsized":"2024-09-16T09:45:43.756559+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:45:01.754170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:45:01.754170+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:31.383586+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":1094,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:44.399502+0000","last_change":"2024-09-16T09:45:31.631877+0000","last_active":"2024-09-16T09:45:44.399502+0000","last_peered":"2024-09-16T09:45:44.399502+0000","last_clean":"2024-09-16T09:45:44.399502+0000","last_became_active":"2024-09-16T09:37:53.518354+0000","last_became_peered":"2024-09-16T09:37:53.518354+0000","last_unstale":"2024-09-16T09:45:44.399502+0000","last_undegraded":"2024-09-16T09:45:44.399502+0000","last_fullsized":"2024-09-16T09:45:44.399502+0000","mapping_epoch":382,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":383,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:31.631815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:45:31.631815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:54.944410+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"547'18","reported_seq":875,"reported_epoch":570,"state":"active+clean","last_fresh":"2024-09-16T09:46:03.603993+0000","last_change":"2024-09-16T09:45:43.126151+0000","last_active":"2024-09-16T09:46:03.603993+0000","last_peered":"2024-09-16T09:46:03.603993+0000","last_clean":"2024-09-16T09:46:03.603993+0000","last_became_active":"2024-09-16T09:44:24.274107+0000","last_became_peered":"2024-09-16T09:44:24.274107+0000","last_unstale":"2024-09-16T09:46:03.603993+0000","last_undegraded":"2024-09-16T09:46:03.603993+0000","last_fullsized":"2024-09-16T09:46:03.603993+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"547'18","last_scrub_stamp":"2024-09-16T09:45:43.126099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:45:43.126099+0000","objects_scrubbed":3,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:47:08.520232+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":26,"num_read_kb":23,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1076,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:43.756509+0000","last_change":"2024-09-16T09:44:54.767506+0000","last_active":"2024-09-16T09:45:43.756509+0000","last_peered":"2024-09-16T09:45:43.756509+0000","last_clean":"2024-09-16T09:45:43.756509+0000","last_became_active":"2024-09-16T09:25:05.715616+0000","last_became_peered":"2024-09-16T09:25:05.715616+0000","last_unstale":"2024-09-16T09:45:43.756509+0000","last_undegraded":"2024-09-16T09:45:43.756509+0000","last_fullsized":"2024-09-16T09:45:43.756509+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:54.767444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:44:54.767444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:17.133885+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1063,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:43.760052+0000","last_change":"2024-09-16T09:44:46.528000+0000","last_active":"2024-09-16T09:45:43.760052+0000","last_peered":"2024-09-16T09:45:43.760052+0000","last_clean":"2024-09-16T09:45:43.760052+0000","last_became_active":"2024-09-16T09:25:07.721059+0000","last_became_peered":"2024-09-16T09:25:07.721059+0000","last_unstale":"2024-09-16T09:45:43.760052+0000","last_undegraded":"2024-09-16T09:45:43.760052+0000","last_fullsized":"2024-09-16T09:45:43.760052+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:46.527936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:44:46.527936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:03.411304+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.13","version":"73'79","reported_seq":407,"reported_epoch":570,"state":"active+clean","last_fresh":"2024-09-16T09:46:03.529894+0000","last_change":"2024-09-16T09:45:45.499996+0000","last_active":"2024-09-16T09:46:03.529894+0000","last_peered":"2024-09-16T09:46:03.529894+0000","last_clean":"2024-09-16T09:46:03.529894+0000","last_became_active":"2024-09-16T09:45:19.680126+0000","last_became_peered":"2024-09-16T09:45:19.680126+0000","last_unstale":"2024-09-16T09:46:03.529894+0000","last_undegraded":"2024-09-16T09:46:03.529894+0000","last_fullsized":"2024-09-16T09:46:03.529894+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:45:45.499932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:45:45.499932+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:47:08.613886+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.16","version":"547'21","reported_seq":1169,"reported_epoch":571,"state":"active+clean","last_fresh":"2024-09-16T09:46:04.854994+0000","last_change":"2024-09-16T09:45:39.973119+0000","last_active":"2024-09-16T09:46:04.854994+0000","last_peered":"2024-09-16T09:46:04.854994+0000","last_clean":"2024-09-16T09:46:04.854994+0000","last_became_active":"2024-09-16T09:31:09.273498+0000","last_became_peered":"2024-09-16T09:31:09.273498+0000","last_unstale":"2024-09-16T09:46:04.854994+0000","last_undegraded":"2024-09-16T09:46:04.854994+0000","last_fullsized":"2024-09-16T09:46:04.854994+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"547'21","last_scrub_stamp":"2024-09-16T09:45:39.973061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:45:39.973061+0000","objects_scrubbed":4,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:47:03.358051+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":1092,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:43.759876+0000","last_change":"2024-09-16T09:45:36.483091+0000","last_active":"2024-09-16T09:45:43.759876+0000","last_peered":"2024-09-16T09:45:43.759876+0000","last_clean":"2024-09-16T09:45:43.759876+0000","last_became_active":"2024-09-16T09:25:01.706491+0000","last_became_peered":"2024-09-16T09:25:01.706491+0000","last_unstale":"2024-09-16T09:45:43.759876+0000","last_undegraded":"2024-09-16T09:45:43.759876+0000","last_fullsized":"2024-09-16T09:45:43.759876+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:36.483032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:45:36.483032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:42.419262+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":270,"reported_epoch":570,"state":"active+clean","last_fresh":"2024-09-16T09:46:03.603845+0000","last_change":"2024-09-16T09:45:23.695102+0000","last_active":"2024-09-16T09:46:03.603845+0000","last_peered":"2024-09-16T09:46:03.603845+0000","last_clean":"2024-09-16T09:46:03.603845+0000","last_became_active":"2024-09-16T09:45:23.694839+0000","last_became_peered":"2024-09-16T09:45:23.694839+0000","last_unstale":"2024-09-16T09:46:03.603845+0000","last_undegraded":"2024-09-16T09:46:03.603845+0000","last_fullsized":"2024-09-16T09:46:03.603845+0000","mapping_epoch":554,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":555,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:11.560613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:45:11.560613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:41.502754+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1047,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:43.758351+0000","last_change":"2024-09-16T09:44:46.061164+0000","last_active":"2024-09-16T09:45:43.758351+0000","last_peered":"2024-09-16T09:45:43.758351+0000","last_clean":"2024-09-16T09:45:43.758351+0000","last_became_active":"2024-09-16T09:44:24.277612+0000","last_became_peered":"2024-09-16T09:44:24.277612+0000","last_unstale":"2024-09-16T09:45:43.758351+0000","last_undegraded":"2024-09-16T09:45:43.758351+0000","last_fullsized":"2024-09-16T09:45:43.758351+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:46.061101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:44:46.061101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:04.434595+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.10","version":"73'79","reported_seq":878,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:43.755585+0000","last_change":"2024-09-16T09:44:43.656131+0000","last_active":"2024-09-16T09:45:43.755585+0000","last_peered":"2024-09-16T09:45:43.755585+0000","last_clean":"2024-09-16T09:45:43.755585+0000","last_became_active":"2024-09-16T09:30:10.692770+0000","last_became_peered":"2024-09-16T09:30:10.692770+0000","last_unstale":"2024-09-16T09:45:43.755585+0000","last_undegraded":"2024-09-16T09:45:43.755585+0000","last_fullsized":"2024-09-16T09:45:43.755585+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:44:43.656039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:44:43.656039+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:01.808196+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"547'21","reported_seq":1114,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:43.756758+0000","last_change":"2024-09-16T09:44:57.752956+0000","last_active":"2024-09-16T09:45:43.756758+0000","last_peered":"2024-09-16T09:45:43.756758+0000","last_clean":"2024-09-16T09:45:43.756758+0000","last_became_active":"2024-09-16T09:25:03.717310+0000","last_became_peered":"2024-09-16T09:25:03.717310+0000","last_unstale":"2024-09-16T09:45:43.756758+0000","last_undegraded":"2024-09-16T09:45:43.756758+0000","last_fullsized":"2024-09-16T09:45:43.756758+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'15","last_scrub_stamp":"2024-09-16T09:44:57.752868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:44:57.752868+0000","objects_scrubbed":4,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:14.909445+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":1112,"reported_epoch":571,"state":"active+clean","last_fresh":"2024-09-16T09:46:04.854555+0000","last_change":"2024-09-16T09:44:35.099469+0000","last_active":"2024-09-16T09:46:04.854555+0000","last_peered":"2024-09-16T09:46:04.854555+0000","last_clean":"2024-09-16T09:46:04.854555+0000","last_became_active":"2024-09-16T09:31:09.272848+0000","last_became_peered":"2024-09-16T09:31:09.272848+0000","last_unstale":"2024-09-16T09:46:04.854555+0000","last_undegraded":"2024-09-16T09:46:04.854555+0000","last_fullsized":"2024-09-16T09:46:04.854555+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:35.099420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:44:35.099420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"557'14","reported_seq":241,"reported_epoch":570,"state":"active+clean","last_fresh":"2024-09-16T09:46:03.604623+0000","last_change":"2024-09-16T09:45:25.123664+0000","last_active":"2024-09-16T09:46:03.604623+0000","last_peered":"2024-09-16T09:46:03.604623+0000","last_clean":"2024-09-16T09:46:03.604623+0000","last_became_active":"2024-09-16T09:45:23.693762+0000","last_became_peered":"2024-09-16T09:45:23.693762+0000","last_unstale":"2024-09-16T09:46:03.604623+0000","last_undegraded":"2024-09-16T09:46:03.604623+0000","last_fullsized":"2024-09-16T09:46:03.604623+0000","mapping_epoch":554,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":555,"parent":"0.0","parent_split_bits":0,"last_scrub":"519'13","last_scrub_stamp":"2024-09-16T09:45:25.123612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:45:25.123612+0000","objects_scrubbed":1,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:49.886509+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1100,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:44.127927+0000","last_change":"2024-09-16T09:44:54.304603+0000","last_active":"2024-09-16T09:45:44.127927+0000","last_peered":"2024-09-16T09:45:44.127927+0000","last_clean":"2024-09-16T09:45:44.127927+0000","last_became_active":"2024-09-16T09:44:24.277124+0000","last_became_peered":"2024-09-16T09:44:24.277124+0000","last_unstale":"2024-09-16T09:45:44.127927+0000","last_undegraded":"2024-09-16T09:45:44.127927+0000","last_fullsized":"2024-09-16T09:45:44.127927+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:54.304553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:44:54.304553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:54.689045+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.11","version":"73'79","reported_seq":968,"reported_epoch":571,"state":"active+clean","last_fresh":"2024-09-16T09:46:04.854397+0000","last_change":"2024-09-16T09:45:23.092029+0000","last_active":"2024-09-16T09:46:04.854397+0000","last_peered":"2024-09-16T09:46:04.854397+0000","last_clean":"2024-09-16T09:46:04.854397+0000","last_became_active":"2024-09-16T09:31:09.272208+0000","last_became_peered":"2024-09-16T09:31:09.272208+0000","last_unstale":"2024-09-16T09:46:04.854397+0000","last_undegraded":"2024-09-16T09:46:04.854397+0000","last_fullsized":"2024-09-16T09:46:04.854397+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:45:23.091979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:45:23.091979+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:27.637013+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.14","version":"547'30","reported_seq":770,"reported_epoch":570,"state":"active+clean","last_fresh":"2024-09-16T09:46:03.604508+0000","last_change":"2024-09-16T09:45:23.692659+0000","last_active":"2024-09-16T09:46:03.604508+0000","last_peered":"2024-09-16T09:46:03.604508+0000","last_clean":"2024-09-16T09:46:03.604508+0000","last_became_active":"2024-09-16T09:45:23.692293+0000","last_became_peered":"2024-09-16T09:45:23.692293+0000","last_unstale":"2024-09-16T09:46:03.604508+0000","last_undegraded":"2024-09-16T09:46:03.604508+0000","last_fullsized":"2024-09-16T09:46:03.604508+0000","mapping_epoch":554,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":555,"parent":"0.0","parent_split_bits":0,"last_scrub":"547'30","last_scrub_stamp":"2024-09-16T09:45:18.568491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:45:18.568491+0000","objects_scrubbed":5,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:23.552768+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":43,"num_read_kb":38,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":1087,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:43.759834+0000","last_change":"2024-09-16T09:45:41.458692+0000","last_active":"2024-09-16T09:45:43.759834+0000","last_peered":"2024-09-16T09:45:43.759834+0000","last_clean":"2024-09-16T09:45:43.759834+0000","last_became_active":"2024-09-16T09:44:24.279273+0000","last_became_peered":"2024-09-16T09:44:24.279273+0000","last_unstale":"2024-09-16T09:45:43.759834+0000","last_undegraded":"2024-09-16T09:45:43.759834+0000","last_fullsized":"2024-09-16T09:45:43.759834+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:41.458618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:45:41.458618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:56.845197+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"66'8","reported_seq":1353,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:56.995128+0000","last_change":"2024-09-16T09:45:45.786470+0000","last_active":"2024-09-16T09:45:56.995128+0000","last_peered":"2024-09-16T09:45:56.995128+0000","last_clean":"2024-09-16T09:45:56.995128+0000","last_became_active":"2024-09-16T09:25:05.713833+0000","last_became_peered":"2024-09-16T09:25:05.713833+0000","last_unstale":"2024-09-16T09:45:56.995128+0000","last_undegraded":"2024-09-16T09:45:56.995128+0000","last_fullsized":"2024-09-16T09:45:56.995128+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:45:45.786379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:45:45.786379+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:47:15.609811+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1107,"reported_epoch":571,"state":"active+clean","last_fresh":"2024-09-16T09:46:04.854811+0000","last_change":"2024-09-16T09:45:37.007089+0000","last_active":"2024-09-16T09:46:04.854811+0000","last_peered":"2024-09-16T09:46:04.854811+0000","last_clean":"2024-09-16T09:46:04.854811+0000","last_became_active":"2024-09-16T09:31:09.275168+0000","last_became_peered":"2024-09-16T09:31:09.275168+0000","last_unstale":"2024-09-16T09:46:04.854811+0000","last_undegraded":"2024-09-16T09:46:04.854811+0000","last_fullsized":"2024-09-16T09:46:04.854811+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:37.007038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:45:37.007038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:55.131269+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.16","version":"73'79","reported_seq":135,"reported_epoch":570,"state":"active+clean","last_fresh":"2024-09-16T09:46:03.530543+0000","last_change":"2024-09-16T09:45:21.388018+0000","last_active":"2024-09-16T09:46:03.530543+0000","last_peered":"2024-09-16T09:46:03.530543+0000","last_clean":"2024-09-16T09:46:03.530543+0000","last_became_active":"2024-09-16T09:45:19.679357+0000","last_became_peered":"2024-09-16T09:45:19.679357+0000","last_unstale":"2024-09-16T09:46:03.530543+0000","last_undegraded":"2024-09-16T09:46:03.530543+0000","last_fullsized":"2024-09-16T09:46:03.530543+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":542,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:45:21.387959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:45:21.387959+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:41.802550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"547'27","reported_seq":1128,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:44.127395+0000","last_change":"2024-09-16T09:45:31.182610+0000","last_active":"2024-09-16T09:45:44.127395+0000","last_peered":"2024-09-16T09:45:44.127395+0000","last_clean":"2024-09-16T09:45:44.127395+0000","last_became_active":"2024-09-16T09:26:33.291265+0000","last_became_peered":"2024-09-16T09:26:33.291265+0000","last_unstale":"2024-09-16T09:45:44.127395+0000","last_undegraded":"2024-09-16T09:45:44.127395+0000","last_fullsized":"2024-09-16T09:45:44.127395+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"547'27","last_scrub_stamp":"2024-09-16T09:45:31.182529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:45:31.182529+0000","objects_scrubbed":5,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:53.516271+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":37,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":1101,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:57.119998+0000","last_change":"2024-09-16T09:45:57.119998+0000","last_active":"2024-09-16T09:45:57.119998+0000","last_peered":"2024-09-16T09:45:57.119998+0000","last_clean":"2024-09-16T09:45:57.119998+0000","last_became_active":"2024-09-16T09:25:01.705158+0000","last_became_peered":"2024-09-16T09:25:01.705158+0000","last_unstale":"2024-09-16T09:45:57.119998+0000","last_undegraded":"2024-09-16T09:45:57.119998+0000","last_fullsized":"2024-09-16T09:45:57.119998+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:57.119938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:45:57.119938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:47:09.670712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":639,"reported_epoch":570,"state":"active+clean","last_fresh":"2024-09-16T09:46:03.529588+0000","last_change":"2024-09-16T09:45:19.684238+0000","last_active":"2024-09-16T09:46:03.529588+0000","last_peered":"2024-09-16T09:46:03.529588+0000","last_clean":"2024-09-16T09:46:03.529588+0000","last_became_active":"2024-09-16T09:45:19.683894+0000","last_became_peered":"2024-09-16T09:45:19.683894+0000","last_unstale":"2024-09-16T09:46:03.529588+0000","last_undegraded":"2024-09-16T09:46:03.529588+0000","last_fullsized":"2024-09-16T09:46:03.529588+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:45.660111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:44:45.660111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":161,"reported_epoch":570,"state":"active+clean","last_fresh":"2024-09-16T09:46:03.603570+0000","last_change":"2024-09-16T09:45:23.746052+0000","last_active":"2024-09-16T09:46:03.603570+0000","last_peered":"2024-09-16T09:46:03.603570+0000","last_clean":"2024-09-16T09:46:03.603570+0000","last_became_active":"2024-09-16T09:45:23.745655+0000","last_became_peered":"2024-09-16T09:45:23.745655+0000","last_unstale":"2024-09-16T09:46:03.603570+0000","last_undegraded":"2024-09-16T09:46:03.603570+0000","last_fullsized":"2024-09-16T09:46:03.603570+0000","mapping_epoch":554,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":555,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:47.085465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:44:47.085465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.17","version":"73'79","reported_seq":67,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:43.756372+0000","last_change":"2024-09-16T09:45:05.837415+0000","last_active":"2024-09-16T09:45:43.756372+0000","last_peered":"2024-09-16T09:45:43.756372+0000","last_clean":"2024-09-16T09:45:43.756372+0000","last_became_active":"2024-09-16T09:45:05.497470+0000","last_became_peered":"2024-09-16T09:45:05.497470+0000","last_unstale":"2024-09-16T09:45:43.756372+0000","last_undegraded":"2024-09-16T09:45:43.756372+0000","last_fullsized":"2024-09-16T09:45:43.756372+0000","mapping_epoch":544,"log_start":"0'0","ondisk_log_start":"0'0","created":542,"last_epoch_clean":545,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:45:05.837366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:45:05.837366+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:25.687601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,6],"acting":[5,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.12","version":"547'21","reported_seq":828,"reported_epoch":570,"state":"active+clean","last_fresh":"2024-09-16T09:46:03.529804+0000","last_change":"2024-09-16T09:45:49.439183+0000","last_active":"2024-09-16T09:46:03.529804+0000","last_peered":"2024-09-16T09:46:03.529804+0000","last_clean":"2024-09-16T09:46:03.529804+0000","last_became_active":"2024-09-16T09:45:19.680729+0000","last_became_peered":"2024-09-16T09:45:19.680729+0000","last_unstale":"2024-09-16T09:46:03.529804+0000","last_undegraded":"2024-09-16T09:46:03.529804+0000","last_fullsized":"2024-09-16T09:46:03.529804+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"547'21","last_scrub_stamp":"2024-09-16T09:45:49.439114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:45:49.439114+0000","objects_scrubbed":4,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:47:03.924857+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":33,"num_read_kb":27,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":1083,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:46.527433+0000","last_change":"2024-09-16T09:45:46.527433+0000","last_active":"2024-09-16T09:45:46.527433+0000","last_peered":"2024-09-16T09:45:46.527433+0000","last_clean":"2024-09-16T09:45:46.527433+0000","last_became_active":"2024-09-16T09:25:01.706762+0000","last_became_peered":"2024-09-16T09:25:01.706762+0000","last_unstale":"2024-09-16T09:45:46.527433+0000","last_undegraded":"2024-09-16T09:45:46.527433+0000","last_fullsized":"2024-09-16T09:45:46.527433+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:46.527380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:45:46.527380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:47:03.266221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":207,"reported_epoch":570,"state":"active+clean","last_fresh":"2024-09-16T09:46:03.603991+0000","last_change":"2024-09-16T09:45:23.696559+0000","last_active":"2024-09-16T09:46:03.603991+0000","last_peered":"2024-09-16T09:46:03.603991+0000","last_clean":"2024-09-16T09:46:03.603991+0000","last_became_active":"2024-09-16T09:45:23.696202+0000","last_became_peered":"2024-09-16T09:45:23.696202+0000","last_unstale":"2024-09-16T09:46:03.603991+0000","last_undegraded":"2024-09-16T09:46:03.603991+0000","last_fullsized":"2024-09-16T09:46:03.603991+0000","mapping_epoch":554,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":555,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:13.209048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:45:13.209048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:31.841717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1061,"reported_epoch":571,"state":"active+clean","last_fresh":"2024-09-16T09:46:04.854125+0000","last_change":"2024-09-16T09:45:10.203182+0000","last_active":"2024-09-16T09:46:04.854125+0000","last_peered":"2024-09-16T09:46:04.854125+0000","last_clean":"2024-09-16T09:46:04.854125+0000","last_became_active":"2024-09-16T09:25:07.715301+0000","last_became_peered":"2024-09-16T09:25:07.715301+0000","last_unstale":"2024-09-16T09:46:04.854125+0000","last_undegraded":"2024-09-16T09:46:04.854125+0000","last_fullsized":"2024-09-16T09:46:04.854125+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:10.203136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:45:10.203136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:23.474897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.14","version":"73'79","reported_seq":800,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:43.758151+0000","last_change":"2024-09-16T09:45:32.165994+0000","last_active":"2024-09-16T09:45:43.758151+0000","last_peered":"2024-09-16T09:45:43.758151+0000","last_clean":"2024-09-16T09:45:43.758151+0000","last_became_active":"2024-09-16T09:44:24.285886+0000","last_became_peered":"2024-09-16T09:44:24.285886+0000","last_unstale":"2024-09-16T09:45:43.758151+0000","last_undegraded":"2024-09-16T09:45:43.758151+0000","last_fullsized":"2024-09-16T09:45:43.758151+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:45:32.165932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:45:32.165932+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:47:01.695497+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"547'27","reported_seq":972,"reported_epoch":570,"state":"active+clean","last_fresh":"2024-09-16T09:46:03.603138+0000","last_change":"2024-09-16T09:45:31.075789+0000","last_active":"2024-09-16T09:46:03.603138+0000","last_peered":"2024-09-16T09:46:03.603138+0000","last_clean":"2024-09-16T09:46:03.603138+0000","last_became_active":"2024-09-16T09:45:23.692856+0000","last_became_peered":"2024-09-16T09:45:23.692856+0000","last_unstale":"2024-09-16T09:46:03.603138+0000","last_undegraded":"2024-09-16T09:46:03.603138+0000","last_fullsized":"2024-09-16T09:46:03.603138+0000","mapping_epoch":554,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":555,"parent":"0.0","parent_split_bits":0,"last_scrub":"547'27","last_scrub_stamp":"2024-09-16T09:45:31.075736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:45:31.075736+0000","objects_scrubbed":5,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:55.061722+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":1092,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:43.756670+0000","last_change":"2024-09-16T09:45:35.673157+0000","last_active":"2024-09-16T09:45:43.756670+0000","last_peered":"2024-09-16T09:45:43.756670+0000","last_clean":"2024-09-16T09:45:43.756670+0000","last_became_active":"2024-09-16T09:25:01.700733+0000","last_became_peered":"2024-09-16T09:25:01.700733+0000","last_unstale":"2024-09-16T09:45:43.756670+0000","last_undegraded":"2024-09-16T09:45:43.756670+0000","last_fullsized":"2024-09-16T09:45:43.756670+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:35.673100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:45:35.673100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:41.360062+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1076,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:43.759034+0000","last_change":"2024-09-16T09:45:39.407824+0000","last_active":"2024-09-16T09:45:43.759034+0000","last_peered":"2024-09-16T09:45:43.759034+0000","last_clean":"2024-09-16T09:45:43.759034+0000","last_became_active":"2024-09-16T09:25:05.713479+0000","last_became_peered":"2024-09-16T09:25:05.713479+0000","last_unstale":"2024-09-16T09:45:43.759034+0000","last_undegraded":"2024-09-16T09:45:43.759034+0000","last_fullsized":"2024-09-16T09:45:43.759034+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:39.407767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:45:39.407767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:59.901053+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":169,"reported_epoch":570,"state":"active+clean","last_fresh":"2024-09-16T09:46:03.603685+0000","last_change":"2024-09-16T09:45:23.696277+0000","last_active":"2024-09-16T09:46:03.603685+0000","last_peered":"2024-09-16T09:46:03.603685+0000","last_clean":"2024-09-16T09:46:03.603685+0000","last_became_active":"2024-09-16T09:45:23.695841+0000","last_became_peered":"2024-09-16T09:45:23.695841+0000","last_unstale":"2024-09-16T09:46:03.603685+0000","last_undegraded":"2024-09-16T09:46:03.603685+0000","last_fullsized":"2024-09-16T09:46:03.603685+0000","mapping_epoch":554,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":555,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:55.146270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:44:55.146270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:19.024903+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.15","version":"73'79","reported_seq":68,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:43.759933+0000","last_change":"2024-09-16T09:45:06.638316+0000","last_active":"2024-09-16T09:45:43.759933+0000","last_peered":"2024-09-16T09:45:43.759933+0000","last_clean":"2024-09-16T09:45:43.759933+0000","last_became_active":"2024-09-16T09:45:05.497752+0000","last_became_peered":"2024-09-16T09:45:05.497752+0000","last_unstale":"2024-09-16T09:45:43.759933+0000","last_undegraded":"2024-09-16T09:45:43.759933+0000","last_fullsized":"2024-09-16T09:45:43.759933+0000","mapping_epoch":544,"log_start":"0'0","ondisk_log_start":"0'0","created":542,"last_epoch_clean":545,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:45:06.638254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:45:06.638254+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:14.918970+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1],"acting":[7,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"547'12","reported_seq":857,"reported_epoch":570,"state":"active+clean","last_fresh":"2024-09-16T09:46:03.602779+0000","last_change":"2024-09-16T09:44:47.186136+0000","last_active":"2024-09-16T09:46:03.602779+0000","last_peered":"2024-09-16T09:46:03.602779+0000","last_clean":"2024-09-16T09:46:03.602779+0000","last_became_active":"2024-09-16T09:31:09.273682+0000","last_became_peered":"2024-09-16T09:31:09.273682+0000","last_unstale":"2024-09-16T09:46:03.602779+0000","last_undegraded":"2024-09-16T09:46:03.602779+0000","last_fullsized":"2024-09-16T09:46:03.602779+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'8","last_scrub_stamp":"2024-09-16T09:44:47.186078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:44:47.186078+0000","objects_scrubbed":2,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:11.853660+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":17,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":1119,"reported_epoch":571,"state":"active+clean","last_fresh":"2024-09-16T09:46:04.854916+0000","last_change":"2024-09-16T09:45:53.013918+0000","last_active":"2024-09-16T09:46:04.854916+0000","last_peered":"2024-09-16T09:46:04.854916+0000","last_clean":"2024-09-16T09:46:04.854916+0000","last_became_active":"2024-09-16T09:31:09.274719+0000","last_became_peered":"2024-09-16T09:31:09.274719+0000","last_unstale":"2024-09-16T09:46:04.854916+0000","last_undegraded":"2024-09-16T09:46:04.854916+0000","last_fullsized":"2024-09-16T09:46:04.854916+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:53.013871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:45:53.013871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:47:05.328472+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1095,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:57.619232+0000","last_change":"2024-09-16T09:45:57.619232+0000","last_active":"2024-09-16T09:45:57.619232+0000","last_peered":"2024-09-16T09:45:57.619232+0000","last_clean":"2024-09-16T09:45:57.619232+0000","last_became_active":"2024-09-16T09:25:05.712545+0000","last_became_peered":"2024-09-16T09:25:05.712545+0000","last_unstale":"2024-09-16T09:45:57.619232+0000","last_undegraded":"2024-09-16T09:45:57.619232+0000","last_fullsized":"2024-09-16T09:45:57.619232+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:57.619166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:45:57.619166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:47:26.303428+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1082,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:43.759378+0000","last_change":"2024-09-16T09:44:51.632486+0000","last_active":"2024-09-16T09:45:43.759378+0000","last_peered":"2024-09-16T09:45:43.759378+0000","last_clean":"2024-09-16T09:45:43.759378+0000","last_became_active":"2024-09-16T09:44:24.276311+0000","last_became_peered":"2024-09-16T09:44:24.276311+0000","last_unstale":"2024-09-16T09:45:43.759378+0000","last_undegraded":"2024-09-16T09:45:43.759378+0000","last_fullsized":"2024-09-16T09:45:43.759378+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:51.632429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:44:51.632429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:19.007714+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1a","version":"73'79","reported_seq":68,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:44.127149+0000","last_change":"2024-09-16T09:45:07.199263+0000","last_active":"2024-09-16T09:45:44.127149+0000","last_peered":"2024-09-16T09:45:44.127149+0000","last_clean":"2024-09-16T09:45:44.127149+0000","last_became_active":"2024-09-16T09:45:05.497673+0000","last_became_peered":"2024-09-16T09:45:05.497673+0000","last_unstale":"2024-09-16T09:45:44.127149+0000","last_undegraded":"2024-09-16T09:45:44.127149+0000","last_fullsized":"2024-09-16T09:45:44.127149+0000","mapping_epoch":544,"log_start":"0'0","ondisk_log_start":"0'0","created":542,"last_epoch_clean":545,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:45:07.199201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:45:07.199201+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:26.715375+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,1],"acting":[4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":420,"reported_epoch":570,"state":"active+clean","last_fresh":"2024-09-16T09:46:03.530369+0000","last_change":"2024-09-16T09:45:19.684732+0000","last_active":"2024-09-16T09:46:03.530369+0000","last_peered":"2024-09-16T09:46:03.530369+0000","last_clean":"2024-09-16T09:46:03.530369+0000","last_became_active":"2024-09-16T09:45:19.684396+0000","last_became_peered":"2024-09-16T09:45:19.684396+0000","last_unstale":"2024-09-16T09:46:03.530369+0000","last_undegraded":"2024-09-16T09:46:03.530369+0000","last_fullsized":"2024-09-16T09:46:03.530369+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:50.722209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:44:50.722209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":168,"reported_epoch":570,"state":"active+clean","last_fresh":"2024-09-16T09:46:03.604399+0000","last_change":"2024-09-16T09:45:28.095716+0000","last_active":"2024-09-16T09:46:03.604399+0000","last_peered":"2024-09-16T09:46:03.604399+0000","last_clean":"2024-09-16T09:46:03.604399+0000","last_became_active":"2024-09-16T09:45:23.693577+0000","last_became_peered":"2024-09-16T09:45:23.693577+0000","last_unstale":"2024-09-16T09:46:03.604399+0000","last_undegraded":"2024-09-16T09:46:03.604399+0000","last_fullsized":"2024-09-16T09:46:03.604399+0000","mapping_epoch":554,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":555,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:28.095668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:45:28.095668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:45.230102+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"547'27","reported_seq":1136,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:44.399400+0000","last_change":"2024-09-16T09:45:36.682430+0000","last_active":"2024-09-16T09:45:44.399400+0000","last_peered":"2024-09-16T09:45:44.399400+0000","last_clean":"2024-09-16T09:45:44.399400+0000","last_became_active":"2024-09-16T09:25:03.707769+0000","last_became_peered":"2024-09-16T09:25:03.707769+0000","last_unstale":"2024-09-16T09:45:44.399400+0000","last_undegraded":"2024-09-16T09:45:44.399400+0000","last_fullsized":"2024-09-16T09:45:44.399400+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"547'27","last_scrub_stamp":"2024-09-16T09:45:36.682363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:45:36.682363+0000","objects_scrubbed":5,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:47:04.008614+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":37,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1086,"reported_epoch":571,"state":"active+clean","last_fresh":"2024-09-16T09:46:04.854886+0000","last_change":"2024-09-16T09:44:36.073614+0000","last_active":"2024-09-16T09:46:04.854886+0000","last_peered":"2024-09-16T09:46:04.854886+0000","last_clean":"2024-09-16T09:46:04.854886+0000","last_became_active":"2024-09-16T09:44:24.279906+0000","last_became_peered":"2024-09-16T09:44:24.279906+0000","last_unstale":"2024-09-16T09:46:04.854886+0000","last_undegraded":"2024-09-16T09:46:04.854886+0000","last_fullsized":"2024-09-16T09:46:04.854886+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:36.073560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:44:36.073560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80,"ondisk_log_size":80,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":1537,"num_read_kb":1294,"num_write":989,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":3686,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":969,"ondisk_log_size":969,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1986,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1253,"num_read_kb":1248,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":31,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2449,"ondisk_log_size":2449,"up":93,"acting":93,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266946,"num_pgs":68,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":849028,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":847486,"kb_avail":92896124,"statfs":{"total":95995035648,"available":95125630976,"internally_reserved":0,"allocated":1560576,"data_stored":1057120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":867826122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495879,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":718564,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":716990,"kb_avail":93026588,"statfs":{"total":95995035648,"available":95259226112,"internally_reserved":0,"allocated":1593344,"data_stored":1120904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":734198214},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724814,"num_pgs":62,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":858744,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":857214,"kb_avail":92886408,"statfs":{"total":95995035648,"available":95115681792,"internally_reserved":0,"allocated":1548288,"data_stored":1056601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":877787590},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953747,"num_pgs":68,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":740480,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":739198,"kb_avail":93004672,"statfs":{"total":95995035648,"available":95236784128,"internally_reserved":0,"allocated":1294336,"data_stored":794386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":756939206},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149977,"num_pgs":68,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":917140,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":915518,"kb_avail":92828012,"statfs":{"total":95995035648,"available":95055884288,"internally_reserved":0,"allocated":1638400,"data_stored":1130230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":937490887},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":516,"seq":2216203124758,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":661856,"kb_used_data":575952,"kb_used_omap":4,"kb_used_meta":85883,"kb_avail":93083296,"statfs":{"total":95995035648,"available":95317295104,"internally_reserved":0,"allocated":589774848,"data_stored":589306760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4445,"internal_metadata":87944867},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575141,"num_pgs":64,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1025116,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":1023806,"kb_avail":92720036,"statfs":{"total":95995035648,"available":94945316864,"internally_reserved":0,"allocated":1323008,"data_stored":794366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":1048377797},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738667,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":867148,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":865534,"kb_avail":92878004,"statfs":{"total":95995035648,"available":95107076096,"internally_reserved":0,"allocated":1630208,"data_stored":1130146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":886307269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":1788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T09:46:08.340 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-16T09:46:08.589 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:46:08.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:46:08 smithi070 ceph-mon[30388]: from='client.26831 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:46:08.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:46:08 smithi070 ceph-mon[30388]: 1.6 scrub starts 2024-09-16T09:46:08.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:46:08 smithi070 ceph-mon[30388]: 1.6 scrub ok 2024-09-16T09:46:08.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:46:08 smithi070 ceph-mon[40691]: from='client.26831 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:46:08.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:46:08 smithi070 ceph-mon[40691]: 1.6 scrub starts 2024-09-16T09:46:08.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:46:08 smithi070 ceph-mon[40691]: 1.6 scrub ok 2024-09-16T09:46:08.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:46:08 smithi084 ceph-mon[32629]: from='client.26831 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:46:08.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:46:08 smithi084 ceph-mon[32629]: 1.6 scrub starts 2024-09-16T09:46:08.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:46:08 smithi084 ceph-mon[32629]: 1.6 scrub ok 2024-09-16T09:46:09.247 INFO:teuthology.orchestra.run.smithi070.stdout: 2024-09-16T09:46:09.249 INFO:teuthology.orchestra.run.smithi070.stderr:dumped all 2024-09-16T09:46:09.847 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:46:09 smithi084 ceph-mon[32629]: 6.1b scrub starts 2024-09-16T09:46:09.847 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:46:09 smithi084 ceph-mon[32629]: 6.1b scrub ok 2024-09-16T09:46:09.847 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:46:09 smithi084 ceph-mon[32629]: pgmap v1120: 162 pgs: 162 active+clean; 583 KiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-16T09:46:09.848 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:46:09 smithi084 ceph-mon[32629]: 6.14 scrub starts 2024-09-16T09:46:09.848 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:46:09 smithi084 ceph-mon[32629]: 6.14 scrub ok 2024-09-16T09:46:09.848 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:46:09 smithi084 ceph-mon[32629]: 6.19 scrub starts 2024-09-16T09:46:09.848 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:46:09 smithi084 ceph-mon[32629]: 6.19 scrub ok 2024-09-16T09:46:09.848 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:46:09 smithi084 ceph-mon[32629]: osdmap e573: 8 total, 8 up, 8 in 2024-09-16T09:46:09.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:46:09 smithi070 ceph-mon[40691]: 6.1b scrub starts 2024-09-16T09:46:09.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:46:09 smithi070 ceph-mon[40691]: 6.1b scrub ok 2024-09-16T09:46:09.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:46:09 smithi070 ceph-mon[40691]: pgmap v1120: 162 pgs: 162 active+clean; 583 KiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-16T09:46:09.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:46:09 smithi070 ceph-mon[40691]: 6.14 scrub starts 2024-09-16T09:46:09.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:46:09 smithi070 ceph-mon[40691]: 6.14 scrub ok 2024-09-16T09:46:09.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:46:09 smithi070 ceph-mon[40691]: 6.19 scrub starts 2024-09-16T09:46:09.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:46:09 smithi070 ceph-mon[40691]: 6.19 scrub ok 2024-09-16T09:46:09.921 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:46:09 smithi070 ceph-mon[40691]: osdmap e573: 8 total, 8 up, 8 in 2024-09-16T09:46:09.921 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:46:09 smithi070 ceph-mon[30388]: 6.1b scrub starts 2024-09-16T09:46:09.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:46:09 smithi070 ceph-mon[30388]: 6.1b scrub ok 2024-09-16T09:46:09.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:46:09 smithi070 ceph-mon[30388]: pgmap v1120: 162 pgs: 162 active+clean; 583 KiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-16T09:46:09.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:46:09 smithi070 ceph-mon[30388]: 6.14 scrub starts 2024-09-16T09:46:09.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:46:09 smithi070 ceph-mon[30388]: 6.14 scrub ok 2024-09-16T09:46:09.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:46:09 smithi070 ceph-mon[30388]: 6.19 scrub starts 2024-09-16T09:46:09.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:46:09 smithi070 ceph-mon[30388]: 6.19 scrub ok 2024-09-16T09:46:09.922 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:46:09 smithi070 ceph-mon[30388]: osdmap e573: 8 total, 8 up, 8 in 2024-09-16T09:46:09.983 INFO:teuthology.orchestra.run.smithi070.stdout:{"pg_ready":true,"pg_map":{"version":1120,"stamp":"2024-09-16T09:46:07.945663+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596507,"num_objects":199,"num_object_clones":0,"num_object_copies":597,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":199,"num_whiteouts":0,"num_read":2819,"num_read_kb":2571,"num_write":1008,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":149,"num_bytes_recovered":5672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3514,"ondisk_log_size":3514,"up":486,"acting":486,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":488,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":6618956,"kb_used_data":567164,"kb_used_omap":15,"kb_used_meta":6051632,"kb_avail":743342260,"statfs":{"total":767960285184,"available":761182474240,"internally_reserved":0,"allocated":580775936,"data_stored":576777541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15600,"internal_metadata":6196871952},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001306"},"pg_stats":[{"pgid":"1.1d","version":"73'79","reported_seq":58,"reported_epoch":570,"state":"active+clean","last_fresh":"2024-09-16T09:46:03.530193+0000","last_change":"2024-09-16T09:45:20.351727+0000","last_active":"2024-09-16T09:46:03.530193+0000","last_peered":"2024-09-16T09:46:03.530193+0000","last_clean":"2024-09-16T09:46:03.530193+0000","last_became_active":"2024-09-16T09:45:19.679136+0000","last_became_peered":"2024-09-16T09:45:19.679136+0000","last_unstale":"2024-09-16T09:46:03.530193+0000","last_undegraded":"2024-09-16T09:46:03.530193+0000","last_fullsized":"2024-09-16T09:46:03.530193+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":542,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:45:20.351679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:45:20.351679+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:24.766650+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":1110,"reported_epoch":571,"state":"active+clean","last_fresh":"2024-09-16T09:46:04.854898+0000","last_change":"2024-09-16T09:44:58.203883+0000","last_active":"2024-09-16T09:46:04.854898+0000","last_peered":"2024-09-16T09:46:04.854898+0000","last_clean":"2024-09-16T09:46:04.854898+0000","last_became_active":"2024-09-16T09:44:24.277455+0000","last_became_peered":"2024-09-16T09:44:24.277455+0000","last_unstale":"2024-09-16T09:46:04.854898+0000","last_undegraded":"2024-09-16T09:46:04.854898+0000","last_fullsized":"2024-09-16T09:46:04.854898+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:58.203811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:44:58.203811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:09.563756+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"547'21","reported_seq":1123,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:48.151027+0000","last_change":"2024-09-16T09:45:48.151027+0000","last_active":"2024-09-16T09:45:48.151027+0000","last_peered":"2024-09-16T09:45:48.151027+0000","last_clean":"2024-09-16T09:45:48.151027+0000","last_became_active":"2024-09-16T09:25:03.714635+0000","last_became_peered":"2024-09-16T09:25:03.714635+0000","last_unstale":"2024-09-16T09:45:48.151027+0000","last_undegraded":"2024-09-16T09:45:48.151027+0000","last_fullsized":"2024-09-16T09:45:48.151027+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"547'21","last_scrub_stamp":"2024-09-16T09:45:48.150947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:45:48.150947+0000","objects_scrubbed":4,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:52.939721+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":440,"reported_epoch":570,"state":"active+clean","last_fresh":"2024-09-16T09:46:03.531084+0000","last_change":"2024-09-16T09:45:25.410780+0000","last_active":"2024-09-16T09:46:03.531084+0000","last_peered":"2024-09-16T09:46:03.531084+0000","last_clean":"2024-09-16T09:46:03.531084+0000","last_became_active":"2024-09-16T09:45:19.684964+0000","last_became_peered":"2024-09-16T09:45:19.684964+0000","last_unstale":"2024-09-16T09:46:03.531084+0000","last_undegraded":"2024-09-16T09:46:03.531084+0000","last_fullsized":"2024-09-16T09:46:03.531084+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:25.410724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:45:25.410724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:36.400332+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1073,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:46:01.209679+0000","last_change":"2024-09-16T09:46:01.209679+0000","last_active":"2024-09-16T09:46:01.209679+0000","last_peered":"2024-09-16T09:46:01.209679+0000","last_clean":"2024-09-16T09:46:01.209679+0000","last_became_active":"2024-09-16T09:25:07.715442+0000","last_became_peered":"2024-09-16T09:25:07.715442+0000","last_unstale":"2024-09-16T09:46:01.209679+0000","last_undegraded":"2024-09-16T09:46:01.209679+0000","last_fullsized":"2024-09-16T09:46:01.209679+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:46:01.209620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:46:01.209620+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:47:05.487897+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1c","version":"73'79","reported_seq":74,"reported_epoch":572,"state":"active+clean","last_fresh":"2024-09-16T09:46:05.751619+0000","last_change":"2024-09-16T09:45:05.686494+0000","last_active":"2024-09-16T09:46:05.751619+0000","last_peered":"2024-09-16T09:46:05.751619+0000","last_clean":"2024-09-16T09:46:05.751619+0000","last_became_active":"2024-09-16T09:45:05.495834+0000","last_became_peered":"2024-09-16T09:45:05.495834+0000","last_unstale":"2024-09-16T09:46:05.751619+0000","last_undegraded":"2024-09-16T09:46:05.751619+0000","last_fullsized":"2024-09-16T09:46:05.751619+0000","mapping_epoch":544,"log_start":"0'0","ondisk_log_start":"0'0","created":542,"last_epoch_clean":545,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:45:05.686443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:45:05.686443+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:29.935815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":814,"reported_epoch":570,"state":"active+clean","last_fresh":"2024-09-16T09:46:03.603818+0000","last_change":"2024-09-16T09:44:34.236344+0000","last_active":"2024-09-16T09:46:03.603818+0000","last_peered":"2024-09-16T09:46:03.603818+0000","last_clean":"2024-09-16T09:46:03.603818+0000","last_became_active":"2024-09-16T09:31:09.269342+0000","last_became_peered":"2024-09-16T09:31:09.269342+0000","last_unstale":"2024-09-16T09:46:03.603818+0000","last_undegraded":"2024-09-16T09:46:03.603818+0000","last_fullsized":"2024-09-16T09:46:03.603818+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:34.236298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:44:34.236298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":885,"reported_epoch":570,"state":"active+clean","last_fresh":"2024-09-16T09:46:03.604468+0000","last_change":"2024-09-16T09:45:58.877562+0000","last_active":"2024-09-16T09:46:03.604468+0000","last_peered":"2024-09-16T09:46:03.604468+0000","last_clean":"2024-09-16T09:46:03.604468+0000","last_became_active":"2024-09-16T09:45:23.693724+0000","last_became_peered":"2024-09-16T09:45:23.693724+0000","last_unstale":"2024-09-16T09:46:03.604468+0000","last_undegraded":"2024-09-16T09:46:03.604468+0000","last_fullsized":"2024-09-16T09:46:03.604468+0000","mapping_epoch":554,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":555,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:58.877516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:45:58.877516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:47:24.081212+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"547'39","reported_seq":314,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:46.160696+0000","last_change":"2024-09-16T09:45:46.160643+0000","last_active":"2024-09-16T09:45:46.160696+0000","last_peered":"2024-09-16T09:45:46.160696+0000","last_clean":"2024-09-16T09:45:46.160696+0000","last_became_active":"2024-09-16T09:40:58.699357+0000","last_became_peered":"2024-09-16T09:40:58.699357+0000","last_unstale":"2024-09-16T09:45:46.160696+0000","last_undegraded":"2024-09-16T09:45:46.160696+0000","last_fullsized":"2024-09-16T09:45:46.160696+0000","mapping_epoch":445,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":446,"parent":"0.0","parent_split_bits":0,"last_scrub":"547'39","last_scrub_stamp":"2024-09-16T09:45:46.160568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:45:46.160568+0000","objects_scrubbed":7,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:47:09.489573+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":61,"num_read_kb":52,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,5],"acting":[4,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1085,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:44.127046+0000","last_change":"2024-09-16T09:45:12.160663+0000","last_active":"2024-09-16T09:45:44.127046+0000","last_peered":"2024-09-16T09:45:44.127046+0000","last_clean":"2024-09-16T09:45:44.127046+0000","last_became_active":"2024-09-16T09:25:05.717236+0000","last_became_peered":"2024-09-16T09:25:05.717236+0000","last_unstale":"2024-09-16T09:45:44.127046+0000","last_undegraded":"2024-09-16T09:45:44.127046+0000","last_fullsized":"2024-09-16T09:45:44.127046+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:12.160601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:45:12.160601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:38.736753+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":270,"reported_epoch":570,"state":"active+clean","last_fresh":"2024-09-16T09:46:03.530288+0000","last_change":"2024-09-16T09:45:31.429188+0000","last_active":"2024-09-16T09:46:03.530288+0000","last_peered":"2024-09-16T09:46:03.530288+0000","last_clean":"2024-09-16T09:46:03.530288+0000","last_became_active":"2024-09-16T09:45:19.684857+0000","last_became_peered":"2024-09-16T09:45:19.684857+0000","last_unstale":"2024-09-16T09:46:03.530288+0000","last_undegraded":"2024-09-16T09:46:03.530288+0000","last_fullsized":"2024-09-16T09:46:03.530288+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:31.429144+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:45:31.429144+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:31.546749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"547'21","reported_seq":1123,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:44.126813+0000","last_change":"2024-09-16T09:45:41.041633+0000","last_active":"2024-09-16T09:45:44.126813+0000","last_peered":"2024-09-16T09:45:44.126813+0000","last_clean":"2024-09-16T09:45:44.126813+0000","last_became_active":"2024-09-16T09:25:03.709147+0000","last_became_peered":"2024-09-16T09:25:03.709147+0000","last_unstale":"2024-09-16T09:45:44.126813+0000","last_undegraded":"2024-09-16T09:45:44.126813+0000","last_fullsized":"2024-09-16T09:45:44.126813+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"547'21","last_scrub_stamp":"2024-09-16T09:45:41.041566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:45:41.041566+0000","objects_scrubbed":4,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:47:02.498406+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1068,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:43.756418+0000","last_change":"2024-09-16T09:45:41.810793+0000","last_active":"2024-09-16T09:45:43.756418+0000","last_peered":"2024-09-16T09:45:43.756418+0000","last_clean":"2024-09-16T09:45:43.756418+0000","last_became_active":"2024-09-16T09:25:05.718134+0000","last_became_peered":"2024-09-16T09:25:05.718134+0000","last_unstale":"2024-09-16T09:45:43.756418+0000","last_undegraded":"2024-09-16T09:45:43.756418+0000","last_fullsized":"2024-09-16T09:45:43.756418+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:41.810731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:45:41.810731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:47:08.215365+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"66'1","reported_seq":1094,"reported_epoch":571,"state":"active+clean","last_fresh":"2024-09-16T09:46:04.854174+0000","last_change":"2024-09-16T09:45:54.979631+0000","last_active":"2024-09-16T09:46:04.854174+0000","last_peered":"2024-09-16T09:46:04.854174+0000","last_clean":"2024-09-16T09:46:04.854174+0000","last_became_active":"2024-09-16T09:25:07.716558+0000","last_became_peered":"2024-09-16T09:25:07.716558+0000","last_unstale":"2024-09-16T09:46:04.854174+0000","last_undegraded":"2024-09-16T09:46:04.854174+0000","last_fullsized":"2024-09-16T09:46:04.854174+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-16T09:45:54.979570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:45:54.979570+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:47:23.182188+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1e","version":"73'79","reported_seq":149,"reported_epoch":570,"state":"active+clean","last_fresh":"2024-09-16T09:46:03.604570+0000","last_change":"2024-09-16T09:45:06.199206+0000","last_active":"2024-09-16T09:46:03.604570+0000","last_peered":"2024-09-16T09:46:03.604570+0000","last_clean":"2024-09-16T09:46:03.604570+0000","last_became_active":"2024-09-16T09:45:05.498656+0000","last_became_peered":"2024-09-16T09:45:05.498656+0000","last_unstale":"2024-09-16T09:46:03.604570+0000","last_undegraded":"2024-09-16T09:46:03.604570+0000","last_fullsized":"2024-09-16T09:46:03.604570+0000","mapping_epoch":544,"log_start":"0'0","ondisk_log_start":"0'0","created":542,"last_epoch_clean":545,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:45:06.199126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:45:06.199126+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:25.814352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":1068,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:43.756184+0000","last_change":"2024-09-16T09:45:18.790528+0000","last_active":"2024-09-16T09:45:43.756184+0000","last_peered":"2024-09-16T09:45:43.756184+0000","last_clean":"2024-09-16T09:45:43.756184+0000","last_became_active":"2024-09-16T09:25:01.696939+0000","last_became_peered":"2024-09-16T09:25:01.696939+0000","last_unstale":"2024-09-16T09:45:43.756184+0000","last_undegraded":"2024-09-16T09:45:43.756184+0000","last_fullsized":"2024-09-16T09:45:43.756184+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:18.790452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:45:18.790452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:28.523093+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"547'9","reported_seq":1081,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:44.126805+0000","last_change":"2024-09-16T09:45:18.121071+0000","last_active":"2024-09-16T09:45:44.126805+0000","last_peered":"2024-09-16T09:45:44.126805+0000","last_clean":"2024-09-16T09:45:44.126805+0000","last_became_active":"2024-09-16T09:25:03.716136+0000","last_became_peered":"2024-09-16T09:25:03.716136+0000","last_unstale":"2024-09-16T09:45:44.126805+0000","last_undegraded":"2024-09-16T09:45:44.126805+0000","last_fullsized":"2024-09-16T09:45:44.126805+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"547'9","last_scrub_stamp":"2024-09-16T09:45:18.121010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:45:18.121010+0000","objects_scrubbed":2,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:37.208073+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":17,"num_read_kb":13,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":225,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:44.399751+0000","last_change":"2024-09-16T09:45:06.671068+0000","last_active":"2024-09-16T09:45:44.399751+0000","last_peered":"2024-09-16T09:45:44.399751+0000","last_clean":"2024-09-16T09:45:44.399751+0000","last_became_active":"2024-09-16T09:42:02.494290+0000","last_became_peered":"2024-09-16T09:42:02.494290+0000","last_unstale":"2024-09-16T09:45:44.399751+0000","last_undegraded":"2024-09-16T09:45:44.399751+0000","last_fullsized":"2024-09-16T09:45:44.399751+0000","mapping_epoch":470,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":471,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:06.671012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:45:06.671012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:20.138489+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,4],"acting":[6,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1094,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:43.756046+0000","last_change":"2024-09-16T09:44:52.742833+0000","last_active":"2024-09-16T09:45:43.756046+0000","last_peered":"2024-09-16T09:45:43.756046+0000","last_clean":"2024-09-16T09:45:43.756046+0000","last_became_active":"2024-09-16T09:31:09.275656+0000","last_became_peered":"2024-09-16T09:31:09.275656+0000","last_unstale":"2024-09-16T09:45:43.756046+0000","last_undegraded":"2024-09-16T09:45:43.756046+0000","last_fullsized":"2024-09-16T09:45:43.756046+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:52.742776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:44:52.742776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:05.256885+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.19","version":"73'79","reported_seq":186,"reported_epoch":570,"state":"active+clean","last_fresh":"2024-09-16T09:46:03.602901+0000","last_change":"2024-09-16T09:45:24.075647+0000","last_active":"2024-09-16T09:46:03.602901+0000","last_peered":"2024-09-16T09:46:03.602901+0000","last_clean":"2024-09-16T09:46:03.602901+0000","last_became_active":"2024-09-16T09:45:23.694640+0000","last_became_peered":"2024-09-16T09:45:23.694640+0000","last_unstale":"2024-09-16T09:46:03.602901+0000","last_undegraded":"2024-09-16T09:46:03.602901+0000","last_fullsized":"2024-09-16T09:46:03.602901+0000","mapping_epoch":554,"log_start":"0'0","ondisk_log_start":"0'0","created":542,"last_epoch_clean":555,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:45:24.075601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:45:24.075601+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:39.357561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1075,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:44.127890+0000","last_change":"2024-09-16T09:45:35.117435+0000","last_active":"2024-09-16T09:45:44.127890+0000","last_peered":"2024-09-16T09:45:44.127890+0000","last_clean":"2024-09-16T09:45:44.127890+0000","last_became_active":"2024-09-16T09:25:07.720793+0000","last_became_peered":"2024-09-16T09:25:07.720793+0000","last_unstale":"2024-09-16T09:45:44.127890+0000","last_undegraded":"2024-09-16T09:45:44.127890+0000","last_fullsized":"2024-09-16T09:45:44.127890+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:35.117376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:45:35.117376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:49.783095+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":819,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:44.399693+0000","last_change":"2024-09-16T09:44:48.756968+0000","last_active":"2024-09-16T09:45:44.399693+0000","last_peered":"2024-09-16T09:45:44.399693+0000","last_clean":"2024-09-16T09:45:44.399693+0000","last_became_active":"2024-09-16T09:30:39.201824+0000","last_became_peered":"2024-09-16T09:30:39.201824+0000","last_unstale":"2024-09-16T09:45:44.399693+0000","last_undegraded":"2024-09-16T09:45:44.399693+0000","last_fullsized":"2024-09-16T09:45:44.399693+0000","mapping_epoch":175,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":176,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:48.756890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:44:48.756890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:17.988105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"547'39","reported_seq":1134,"reported_epoch":572,"state":"active+clean","last_fresh":"2024-09-16T09:46:05.749344+0000","last_change":"2024-09-16T09:45:43.761674+0000","last_active":"2024-09-16T09:46:05.749344+0000","last_peered":"2024-09-16T09:46:05.749344+0000","last_clean":"2024-09-16T09:46:05.749344+0000","last_became_active":"2024-09-16T09:45:43.761251+0000","last_became_peered":"2024-09-16T09:45:43.761251+0000","last_unstale":"2024-09-16T09:46:05.749344+0000","last_undegraded":"2024-09-16T09:46:05.749344+0000","last_fullsized":"2024-09-16T09:46:05.749344+0000","mapping_epoch":568,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":569,"parent":"0.0","parent_split_bits":0,"last_scrub":"547'39","last_scrub_stamp":"2024-09-16T09:45:30.382285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:45:30.382285+0000","objects_scrubbed":7,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:56.263792+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":61,"num_read_kb":52,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":424,"reported_epoch":570,"state":"active+clean","last_fresh":"2024-09-16T09:46:03.530728+0000","last_change":"2024-09-16T09:45:41.521364+0000","last_active":"2024-09-16T09:46:03.530728+0000","last_peered":"2024-09-16T09:46:03.530728+0000","last_clean":"2024-09-16T09:46:03.530728+0000","last_became_active":"2024-09-16T09:45:19.679433+0000","last_became_peered":"2024-09-16T09:45:19.679433+0000","last_unstale":"2024-09-16T09:46:03.530728+0000","last_undegraded":"2024-09-16T09:46:03.530728+0000","last_fullsized":"2024-09-16T09:46:03.530728+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:41.521302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:45:41.521302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:57.436115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.18","version":"73'79","reported_seq":452,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:43.755726+0000","last_change":"2024-09-16T09:45:07.843788+0000","last_active":"2024-09-16T09:45:43.755726+0000","last_peered":"2024-09-16T09:45:43.755726+0000","last_clean":"2024-09-16T09:45:43.755726+0000","last_became_active":"2024-09-16T09:45:05.501178+0000","last_became_peered":"2024-09-16T09:45:05.501178+0000","last_unstale":"2024-09-16T09:45:43.755726+0000","last_undegraded":"2024-09-16T09:45:43.755726+0000","last_fullsized":"2024-09-16T09:45:43.755726+0000","mapping_epoch":544,"log_start":"0'0","ondisk_log_start":"0'0","created":542,"last_epoch_clean":545,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:45:07.843717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:45:07.843717+0000","objects_scrubbed":1,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:25.958206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262688,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":203,"reported_epoch":570,"state":"active+clean","last_fresh":"2024-09-16T09:46:03.604140+0000","last_change":"2024-09-16T09:45:23.746000+0000","last_active":"2024-09-16T09:46:03.604140+0000","last_peered":"2024-09-16T09:46:03.604140+0000","last_clean":"2024-09-16T09:46:03.604140+0000","last_became_active":"2024-09-16T09:45:23.745693+0000","last_became_peered":"2024-09-16T09:45:23.745693+0000","last_unstale":"2024-09-16T09:46:03.604140+0000","last_undegraded":"2024-09-16T09:46:03.604140+0000","last_fullsized":"2024-09-16T09:46:03.604140+0000","mapping_epoch":554,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":555,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:12.706646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:45:12.706646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:13.810656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":1104,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:58.145095+0000","last_change":"2024-09-16T09:45:58.145095+0000","last_active":"2024-09-16T09:45:58.145095+0000","last_peered":"2024-09-16T09:45:58.145095+0000","last_clean":"2024-09-16T09:45:58.145095+0000","last_became_active":"2024-09-16T09:44:24.276029+0000","last_became_peered":"2024-09-16T09:44:24.276029+0000","last_unstale":"2024-09-16T09:45:58.145095+0000","last_undegraded":"2024-09-16T09:45:58.145095+0000","last_fullsized":"2024-09-16T09:45:58.145095+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:58.145033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:45:58.145033+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:47:25.492043+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"547'24","reported_seq":775,"reported_epoch":570,"state":"active+clean","last_fresh":"2024-09-16T09:46:03.603662+0000","last_change":"2024-09-16T09:45:35.053006+0000","last_active":"2024-09-16T09:46:03.603662+0000","last_peered":"2024-09-16T09:46:03.603662+0000","last_clean":"2024-09-16T09:46:03.603662+0000","last_became_active":"2024-09-16T09:45:23.697139+0000","last_became_peered":"2024-09-16T09:45:23.697139+0000","last_unstale":"2024-09-16T09:46:03.603662+0000","last_undegraded":"2024-09-16T09:46:03.603662+0000","last_fullsized":"2024-09-16T09:46:03.603662+0000","mapping_epoch":554,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":555,"parent":"0.0","parent_split_bits":0,"last_scrub":"547'24","last_scrub_stamp":"2024-09-16T09:45:35.052951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:45:35.052951+0000","objects_scrubbed":5,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:47:01.202046+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":41,"num_read_kb":32,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1080,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:46:03.244317+0000","last_change":"2024-09-16T09:46:03.244317+0000","last_active":"2024-09-16T09:46:03.244317+0000","last_peered":"2024-09-16T09:46:03.244317+0000","last_clean":"2024-09-16T09:46:03.244317+0000","last_became_active":"2024-09-16T09:44:24.278117+0000","last_became_peered":"2024-09-16T09:44:24.278117+0000","last_unstale":"2024-09-16T09:46:03.244317+0000","last_undegraded":"2024-09-16T09:46:03.244317+0000","last_fullsized":"2024-09-16T09:46:03.244317+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:46:03.244250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:46:03.244250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:47:09.233638+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1b","version":"73'79","reported_seq":246,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:43.755874+0000","last_change":"2024-09-16T09:45:06.833076+0000","last_active":"2024-09-16T09:45:43.755874+0000","last_peered":"2024-09-16T09:45:43.755874+0000","last_clean":"2024-09-16T09:45:43.755874+0000","last_became_active":"2024-09-16T09:45:05.500314+0000","last_became_peered":"2024-09-16T09:45:05.500314+0000","last_unstale":"2024-09-16T09:45:43.755874+0000","last_undegraded":"2024-09-16T09:45:43.755874+0000","last_fullsized":"2024-09-16T09:45:43.755874+0000","mapping_epoch":544,"log_start":"0'0","ondisk_log_start":"0'0","created":542,"last_epoch_clean":545,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:45:06.833020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:45:06.833020+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:07.568877+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,0],"acting":[5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1c","version":"66'1","reported_seq":1089,"reported_epoch":572,"state":"active+clean","last_fresh":"2024-09-16T09:46:05.751498+0000","last_change":"2024-09-16T09:45:20.567757+0000","last_active":"2024-09-16T09:46:05.751498+0000","last_peered":"2024-09-16T09:46:05.751498+0000","last_clean":"2024-09-16T09:46:05.751498+0000","last_became_active":"2024-09-16T09:44:24.284256+0000","last_became_peered":"2024-09-16T09:44:24.284256+0000","last_unstale":"2024-09-16T09:46:05.751498+0000","last_undegraded":"2024-09-16T09:46:05.751498+0000","last_fullsized":"2024-09-16T09:46:05.751498+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'1","last_scrub_stamp":"2024-09-16T09:45:20.567693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:45:20.567693+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:32.553796+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"59'1","reported_seq":433,"reported_epoch":570,"state":"active+clean","last_fresh":"2024-09-16T09:46:03.531164+0000","last_change":"2024-09-16T09:45:19.680101+0000","last_active":"2024-09-16T09:46:03.531164+0000","last_peered":"2024-09-16T09:46:03.531164+0000","last_clean":"2024-09-16T09:46:03.531164+0000","last_became_active":"2024-09-16T09:45:19.679100+0000","last_became_peered":"2024-09-16T09:45:19.679100+0000","last_unstale":"2024-09-16T09:46:03.531164+0000","last_undegraded":"2024-09-16T09:46:03.531164+0000","last_fullsized":"2024-09-16T09:46:03.531164+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:45:18.060164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:45:18.060164+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:21.987974+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"547'30","reported_seq":1162,"reported_epoch":571,"state":"active+clean","last_fresh":"2024-09-16T09:46:04.854923+0000","last_change":"2024-09-16T09:45:14.222140+0000","last_active":"2024-09-16T09:46:04.854923+0000","last_peered":"2024-09-16T09:46:04.854923+0000","last_clean":"2024-09-16T09:46:04.854923+0000","last_became_active":"2024-09-16T09:31:09.275050+0000","last_became_peered":"2024-09-16T09:31:09.275050+0000","last_unstale":"2024-09-16T09:46:04.854923+0000","last_undegraded":"2024-09-16T09:46:04.854923+0000","last_fullsized":"2024-09-16T09:46:04.854923+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"547'30","last_scrub_stamp":"2024-09-16T09:45:14.222080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:45:14.222080+0000","objects_scrubbed":5,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:31.788116+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":45,"num_read_kb":40,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"66'8","reported_seq":1377,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:46:00.589960+0000","last_change":"2024-09-16T09:46:00.589854+0000","last_active":"2024-09-16T09:46:00.589960+0000","last_peered":"2024-09-16T09:46:00.589960+0000","last_clean":"2024-09-16T09:46:00.589960+0000","last_became_active":"2024-09-16T09:25:05.712516+0000","last_became_peered":"2024-09-16T09:25:05.712516+0000","last_unstale":"2024-09-16T09:46:00.589960+0000","last_undegraded":"2024-09-16T09:46:00.589960+0000","last_fullsized":"2024-09-16T09:46:00.589960+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:46:00.589779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:46:00.589779+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:47:08.473519+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.a","version":"73'79","reported_seq":1037,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:43.755823+0000","last_change":"2024-09-16T09:45:03.744061+0000","last_active":"2024-09-16T09:45:43.755823+0000","last_peered":"2024-09-16T09:45:43.755823+0000","last_clean":"2024-09-16T09:45:43.755823+0000","last_became_active":"2024-09-16T09:45:03.607826+0000","last_became_peered":"2024-09-16T09:45:03.607826+0000","last_unstale":"2024-09-16T09:45:43.755823+0000","last_undegraded":"2024-09-16T09:45:43.755823+0000","last_fullsized":"2024-09-16T09:45:43.755823+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":543,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:45:03.744008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:45:03.744008+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:27.030654+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"547'39","reported_seq":1039,"reported_epoch":572,"state":"active+clean","last_fresh":"2024-09-16T09:46:05.749895+0000","last_change":"2024-09-16T09:44:45.023239+0000","last_active":"2024-09-16T09:46:05.749895+0000","last_peered":"2024-09-16T09:46:05.749895+0000","last_clean":"2024-09-16T09:46:05.749895+0000","last_became_active":"2024-09-16T09:44:24.283121+0000","last_became_peered":"2024-09-16T09:44:24.283121+0000","last_unstale":"2024-09-16T09:46:05.749895+0000","last_undegraded":"2024-09-16T09:46:05.749895+0000","last_fullsized":"2024-09-16T09:46:05.749895+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'27","last_scrub_stamp":"2024-09-16T09:44:45.023173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:44:45.023173+0000","objects_scrubbed":7,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:12.611250+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":61,"num_read_kb":52,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":815,"reported_epoch":570,"state":"active+clean","last_fresh":"2024-09-16T09:46:03.602870+0000","last_change":"2024-09-16T09:45:14.089504+0000","last_active":"2024-09-16T09:46:03.602870+0000","last_peered":"2024-09-16T09:46:03.602870+0000","last_clean":"2024-09-16T09:46:03.602870+0000","last_became_active":"2024-09-16T09:31:09.274534+0000","last_became_peered":"2024-09-16T09:31:09.274534+0000","last_unstale":"2024-09-16T09:46:03.602870+0000","last_undegraded":"2024-09-16T09:46:03.602870+0000","last_fullsized":"2024-09-16T09:46:03.602870+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:14.089456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:45:14.089456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:31.001442+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"66'8","reported_seq":1323,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:56.995111+0000","last_change":"2024-09-16T09:44:50.326230+0000","last_active":"2024-09-16T09:45:56.995111+0000","last_peered":"2024-09-16T09:45:56.995111+0000","last_clean":"2024-09-16T09:45:56.995111+0000","last_became_active":"2024-09-16T09:25:05.713813+0000","last_became_peered":"2024-09-16T09:25:05.713813+0000","last_unstale":"2024-09-16T09:45:56.995111+0000","last_undegraded":"2024-09-16T09:45:56.995111+0000","last_fullsized":"2024-09-16T09:45:56.995111+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:44:50.326143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:44:50.326143+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1081,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:43.756165+0000","last_change":"2024-09-16T09:45:39.743940+0000","last_active":"2024-09-16T09:45:43.756165+0000","last_peered":"2024-09-16T09:45:43.756165+0000","last_clean":"2024-09-16T09:45:43.756165+0000","last_became_active":"2024-09-16T09:25:07.717344+0000","last_became_peered":"2024-09-16T09:25:07.717344+0000","last_unstale":"2024-09-16T09:45:43.756165+0000","last_undegraded":"2024-09-16T09:45:43.756165+0000","last_fullsized":"2024-09-16T09:45:43.756165+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:39.743872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:45:39.743872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:47:05.285045+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.5","version":"73'79","reported_seq":1070,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:44.128371+0000","last_change":"2024-09-16T09:45:04.230186+0000","last_active":"2024-09-16T09:45:44.128371+0000","last_peered":"2024-09-16T09:45:44.128371+0000","last_clean":"2024-09-16T09:45:44.128371+0000","last_became_active":"2024-09-16T09:45:03.610857+0000","last_became_peered":"2024-09-16T09:45:03.610857+0000","last_unstale":"2024-09-16T09:45:44.128371+0000","last_undegraded":"2024-09-16T09:45:44.128371+0000","last_fullsized":"2024-09-16T09:45:44.128371+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":543,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:45:04.230129+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:45:04.230129+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:16.266653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"547'42","reported_seq":1065,"reported_epoch":570,"state":"active+clean","last_fresh":"2024-09-16T09:46:03.602583+0000","last_change":"2024-09-16T09:45:54.965274+0000","last_active":"2024-09-16T09:46:03.602583+0000","last_peered":"2024-09-16T09:46:03.602583+0000","last_clean":"2024-09-16T09:46:03.602583+0000","last_became_active":"2024-09-16T09:45:23.693680+0000","last_became_peered":"2024-09-16T09:45:23.693680+0000","last_unstale":"2024-09-16T09:46:03.602583+0000","last_undegraded":"2024-09-16T09:46:03.602583+0000","last_fullsized":"2024-09-16T09:46:03.602583+0000","mapping_epoch":554,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":555,"parent":"0.0","parent_split_bits":0,"last_scrub":"547'42","last_scrub_stamp":"2024-09-16T09:45:54.965208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:45:54.965208+0000","objects_scrubbed":8,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:47:11.271926+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":68,"num_read_kb":56,"num_write":44,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":231,"reported_epoch":570,"state":"active+clean","last_fresh":"2024-09-16T09:46:03.604338+0000","last_change":"2024-09-16T09:45:55.953684+0000","last_active":"2024-09-16T09:46:03.604338+0000","last_peered":"2024-09-16T09:46:03.604338+0000","last_clean":"2024-09-16T09:46:03.604338+0000","last_became_active":"2024-09-16T09:45:23.697210+0000","last_became_peered":"2024-09-16T09:45:23.697210+0000","last_unstale":"2024-09-16T09:46:03.604338+0000","last_undegraded":"2024-09-16T09:46:03.604338+0000","last_fullsized":"2024-09-16T09:46:03.604338+0000","mapping_epoch":554,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":555,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:55.953637+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:45:55.953637+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:58.992515+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1093,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:44.127282+0000","last_change":"2024-09-16T09:44:58.291670+0000","last_active":"2024-09-16T09:45:44.127282+0000","last_peered":"2024-09-16T09:45:44.127282+0000","last_clean":"2024-09-16T09:45:44.127282+0000","last_became_active":"2024-09-16T09:31:09.268864+0000","last_became_peered":"2024-09-16T09:31:09.268864+0000","last_unstale":"2024-09-16T09:45:44.127282+0000","last_undegraded":"2024-09-16T09:45:44.127282+0000","last_fullsized":"2024-09-16T09:45:44.127282+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:58.291613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:44:58.291613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:09.340193+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1108,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:55.213059+0000","last_change":"2024-09-16T09:45:55.213007+0000","last_active":"2024-09-16T09:45:55.213059+0000","last_peered":"2024-09-16T09:45:55.213059+0000","last_clean":"2024-09-16T09:45:55.213059+0000","last_became_active":"2024-09-16T09:44:24.276365+0000","last_became_peered":"2024-09-16T09:44:24.276365+0000","last_unstale":"2024-09-16T09:45:55.213059+0000","last_undegraded":"2024-09-16T09:45:55.213059+0000","last_fullsized":"2024-09-16T09:45:55.213059+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:55.212941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:45:55.212941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:47:23.458239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"73'79","reported_seq":828,"reported_epoch":570,"state":"active+clean","last_fresh":"2024-09-16T09:46:03.604730+0000","last_change":"2024-09-16T09:45:56.957735+0000","last_active":"2024-09-16T09:46:03.604730+0000","last_peered":"2024-09-16T09:46:03.604730+0000","last_clean":"2024-09-16T09:46:03.604730+0000","last_became_active":"2024-09-16T09:31:09.275139+0000","last_became_peered":"2024-09-16T09:31:09.275139+0000","last_unstale":"2024-09-16T09:46:03.604730+0000","last_undegraded":"2024-09-16T09:46:03.604730+0000","last_fullsized":"2024-09-16T09:46:03.604730+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:45:56.957686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:45:56.957686+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:47:21.775448+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"547'42","reported_seq":1154,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:44.126870+0000","last_change":"2024-09-16T09:45:40.071609+0000","last_active":"2024-09-16T09:45:44.126870+0000","last_peered":"2024-09-16T09:45:44.126870+0000","last_clean":"2024-09-16T09:45:44.126870+0000","last_became_active":"2024-09-16T09:25:03.708830+0000","last_became_peered":"2024-09-16T09:25:03.708830+0000","last_unstale":"2024-09-16T09:45:44.126870+0000","last_undegraded":"2024-09-16T09:45:44.126870+0000","last_fullsized":"2024-09-16T09:45:44.126870+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"547'42","last_scrub_stamp":"2024-09-16T09:45:40.071530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:45:40.071530+0000","objects_scrubbed":7,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:58.230475+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":63,"num_read_kb":56,"num_write":42,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"59'1","reported_seq":1107,"reported_epoch":571,"state":"active+clean","last_fresh":"2024-09-16T09:46:04.855028+0000","last_change":"2024-09-16T09:45:44.988737+0000","last_active":"2024-09-16T09:46:04.855028+0000","last_peered":"2024-09-16T09:46:04.855028+0000","last_clean":"2024-09-16T09:46:04.855028+0000","last_became_active":"2024-09-16T09:25:01.702992+0000","last_became_peered":"2024-09-16T09:25:01.702992+0000","last_unstale":"2024-09-16T09:46:04.855028+0000","last_undegraded":"2024-09-16T09:46:04.855028+0000","last_fullsized":"2024-09-16T09:46:04.855028+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:45:44.988667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:45:44.988667+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:47:07.198294+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"557'14","reported_seq":242,"reported_epoch":570,"state":"active+clean","last_fresh":"2024-09-16T09:46:03.602687+0000","last_change":"2024-09-16T09:45:26.097190+0000","last_active":"2024-09-16T09:46:03.602687+0000","last_peered":"2024-09-16T09:46:03.602687+0000","last_clean":"2024-09-16T09:46:03.602687+0000","last_became_active":"2024-09-16T09:45:23.691975+0000","last_became_peered":"2024-09-16T09:45:23.691975+0000","last_unstale":"2024-09-16T09:46:03.602687+0000","last_undegraded":"2024-09-16T09:46:03.602687+0000","last_fullsized":"2024-09-16T09:46:03.602687+0000","mapping_epoch":554,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":555,"parent":"0.0","parent_split_bits":0,"last_scrub":"552'13","last_scrub_stamp":"2024-09-16T09:45:26.097136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:45:26.097136+0000","objects_scrubbed":1,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:40.308843+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1090,"reported_epoch":572,"state":"active+clean","last_fresh":"2024-09-16T09:46:05.751009+0000","last_change":"2024-09-16T09:46:03.596622+0000","last_active":"2024-09-16T09:46:05.751009+0000","last_peered":"2024-09-16T09:46:05.751009+0000","last_clean":"2024-09-16T09:46:05.751009+0000","last_became_active":"2024-09-16T09:44:24.277003+0000","last_became_peered":"2024-09-16T09:44:24.277003+0000","last_unstale":"2024-09-16T09:46:05.751009+0000","last_undegraded":"2024-09-16T09:46:05.751009+0000","last_fullsized":"2024-09-16T09:46:05.751009+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:46:03.596561+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:46:03.596561+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:47:19.919233+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"73'79","reported_seq":1170,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:44.400276+0000","last_change":"2024-09-16T09:45:04.588099+0000","last_active":"2024-09-16T09:45:44.400276+0000","last_peered":"2024-09-16T09:45:44.400276+0000","last_clean":"2024-09-16T09:45:44.400276+0000","last_became_active":"2024-09-16T09:45:03.610800+0000","last_became_peered":"2024-09-16T09:45:03.610800+0000","last_unstale":"2024-09-16T09:45:44.400276+0000","last_undegraded":"2024-09-16T09:45:44.400276+0000","last_fullsized":"2024-09-16T09:45:44.400276+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":543,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:45:04.588014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:45:04.588014+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:21.940905+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"547'30","reported_seq":771,"reported_epoch":570,"state":"active+clean","last_fresh":"2024-09-16T09:46:03.529760+0000","last_change":"2024-09-16T09:45:19.683495+0000","last_active":"2024-09-16T09:46:03.529760+0000","last_peered":"2024-09-16T09:46:03.529760+0000","last_clean":"2024-09-16T09:46:03.529760+0000","last_became_active":"2024-09-16T09:45:19.683142+0000","last_became_peered":"2024-09-16T09:45:19.683142+0000","last_unstale":"2024-09-16T09:46:03.529760+0000","last_undegraded":"2024-09-16T09:46:03.529760+0000","last_fullsized":"2024-09-16T09:46:03.529760+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"547'30","last_scrub_stamp":"2024-09-16T09:45:13.803090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:45:13.803090+0000","objects_scrubbed":5,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:40.288842+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":43,"num_read_kb":38,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":1103,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:43.818502+0000","last_change":"2024-09-16T09:45:43.818419+0000","last_active":"2024-09-16T09:45:43.818502+0000","last_peered":"2024-09-16T09:45:43.818502+0000","last_clean":"2024-09-16T09:45:43.818502+0000","last_became_active":"2024-09-16T09:44:24.280657+0000","last_became_peered":"2024-09-16T09:44:24.280657+0000","last_unstale":"2024-09-16T09:45:43.818502+0000","last_undegraded":"2024-09-16T09:45:43.818502+0000","last_fullsized":"2024-09-16T09:45:43.818502+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:43.818349+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:45:43.818349+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:44.554617+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"66'8","reported_seq":1344,"reported_epoch":571,"state":"active+clean","last_fresh":"2024-09-16T09:46:04.854612+0000","last_change":"2024-09-16T09:46:02.085651+0000","last_active":"2024-09-16T09:46:04.854612+0000","last_peered":"2024-09-16T09:46:04.854612+0000","last_clean":"2024-09-16T09:46:04.854612+0000","last_became_active":"2024-09-16T09:25:05.714507+0000","last_became_peered":"2024-09-16T09:25:05.714507+0000","last_unstale":"2024-09-16T09:46:04.854612+0000","last_undegraded":"2024-09-16T09:46:04.854612+0000","last_fullsized":"2024-09-16T09:46:04.854612+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:46:02.085602+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:46:02.085602+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:47:17.957744+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1104,"reported_epoch":571,"state":"active+clean","last_fresh":"2024-09-16T09:46:04.854284+0000","last_change":"2024-09-16T09:44:25.125306+0000","last_active":"2024-09-16T09:46:04.854284+0000","last_peered":"2024-09-16T09:46:04.854284+0000","last_clean":"2024-09-16T09:46:04.854284+0000","last_became_active":"2024-09-16T09:44:24.269736+0000","last_became_peered":"2024-09-16T09:44:24.269736+0000","last_unstale":"2024-09-16T09:46:04.854284+0000","last_undegraded":"2024-09-16T09:46:04.854284+0000","last_fullsized":"2024-09-16T09:46:04.854284+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:25.125254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:44:25.125254+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"73'79","reported_seq":1127,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:44.400205+0000","last_change":"2024-09-16T09:45:03.617751+0000","last_active":"2024-09-16T09:45:44.400205+0000","last_peered":"2024-09-16T09:45:44.400205+0000","last_clean":"2024-09-16T09:45:44.400205+0000","last_became_active":"2024-09-16T09:45:03.610983+0000","last_became_peered":"2024-09-16T09:45:03.610983+0000","last_unstale":"2024-09-16T09:45:44.400205+0000","last_undegraded":"2024-09-16T09:45:44.400205+0000","last_fullsized":"2024-09-16T09:45:44.400205+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":543,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:45:03.617698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:45:03.617698+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:04.673887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"547'39","reported_seq":1145,"reported_epoch":571,"state":"active+clean","last_fresh":"2024-09-16T09:46:04.854829+0000","last_change":"2024-09-16T09:45:37.967154+0000","last_active":"2024-09-16T09:46:04.854829+0000","last_peered":"2024-09-16T09:46:04.854829+0000","last_clean":"2024-09-16T09:46:04.854829+0000","last_became_active":"2024-09-16T09:25:03.714797+0000","last_became_peered":"2024-09-16T09:25:03.714797+0000","last_unstale":"2024-09-16T09:46:04.854829+0000","last_undegraded":"2024-09-16T09:46:04.854829+0000","last_fullsized":"2024-09-16T09:46:04.854829+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"547'39","last_scrub_stamp":"2024-09-16T09:45:37.967097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:45:37.967097+0000","objects_scrubbed":8,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:47:00.901631+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":69,"num_read_kb":55,"num_write":42,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":649,"reported_epoch":570,"state":"active+clean","last_fresh":"2024-09-16T09:46:03.530482+0000","last_change":"2024-09-16T09:45:50.431119+0000","last_active":"2024-09-16T09:46:03.530482+0000","last_peered":"2024-09-16T09:46:03.530482+0000","last_clean":"2024-09-16T09:46:03.530482+0000","last_became_active":"2024-09-16T09:45:19.685947+0000","last_became_peered":"2024-09-16T09:45:19.685947+0000","last_unstale":"2024-09-16T09:46:03.530482+0000","last_undegraded":"2024-09-16T09:46:03.530482+0000","last_fullsized":"2024-09-16T09:46:03.530482+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:50.431069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:45:50.431069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:56.106163+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1097,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:44.400198+0000","last_change":"2024-09-16T09:44:51.704537+0000","last_active":"2024-09-16T09:45:44.400198+0000","last_peered":"2024-09-16T09:45:44.400198+0000","last_clean":"2024-09-16T09:45:44.400198+0000","last_became_active":"2024-09-16T09:25:05.708663+0000","last_became_peered":"2024-09-16T09:25:05.708663+0000","last_unstale":"2024-09-16T09:45:44.400198+0000","last_undegraded":"2024-09-16T09:45:44.400198+0000","last_fullsized":"2024-09-16T09:45:44.400198+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:51.704482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:44:51.704482+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:16.177847+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":657,"reported_epoch":570,"state":"active+clean","last_fresh":"2024-09-16T09:46:03.531020+0000","last_change":"2024-09-16T09:45:43.501650+0000","last_active":"2024-09-16T09:46:03.531020+0000","last_peered":"2024-09-16T09:46:03.531020+0000","last_clean":"2024-09-16T09:46:03.531020+0000","last_became_active":"2024-09-16T09:45:19.681194+0000","last_became_peered":"2024-09-16T09:45:19.681194+0000","last_unstale":"2024-09-16T09:46:03.531020+0000","last_undegraded":"2024-09-16T09:46:03.531020+0000","last_fullsized":"2024-09-16T09:46:03.531020+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:43.501604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:45:43.501604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:47:11.591957+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"73'79","reported_seq":1064,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:58.611298+0000","last_change":"2024-09-16T09:45:58.611298+0000","last_active":"2024-09-16T09:45:58.611298+0000","last_peered":"2024-09-16T09:45:58.611298+0000","last_clean":"2024-09-16T09:45:58.611298+0000","last_became_active":"2024-09-16T09:44:24.279593+0000","last_became_peered":"2024-09-16T09:44:24.279593+0000","last_unstale":"2024-09-16T09:45:58.611298+0000","last_undegraded":"2024-09-16T09:45:58.611298+0000","last_fullsized":"2024-09-16T09:45:58.611298+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:45:58.611239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:45:58.611239+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:47:23.896677+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"547'56","reported_seq":1519,"reported_epoch":570,"state":"active+clean","last_fresh":"2024-09-16T09:46:03.530412+0000","last_change":"2024-09-16T09:45:46.472053+0000","last_active":"2024-09-16T09:46:03.530412+0000","last_peered":"2024-09-16T09:46:03.530412+0000","last_clean":"2024-09-16T09:46:03.530412+0000","last_became_active":"2024-09-16T09:44:24.280130+0000","last_became_peered":"2024-09-16T09:44:24.280130+0000","last_unstale":"2024-09-16T09:46:03.530412+0000","last_undegraded":"2024-09-16T09:46:03.530412+0000","last_fullsized":"2024-09-16T09:46:03.530412+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"547'56","last_scrub_stamp":"2024-09-16T09:45:46.471974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:45:46.471974+0000","objects_scrubbed":10,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:47:03.561041+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":84,"num_read_kb":69,"num_write":50,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":716,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":1095,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:44.127676+0000","last_change":"2024-09-16T09:45:10.214049+0000","last_active":"2024-09-16T09:45:44.127676+0000","last_peered":"2024-09-16T09:45:44.127676+0000","last_clean":"2024-09-16T09:45:44.127676+0000","last_became_active":"2024-09-16T09:25:01.703076+0000","last_became_peered":"2024-09-16T09:25:01.703076+0000","last_unstale":"2024-09-16T09:45:44.127676+0000","last_undegraded":"2024-09-16T09:45:44.127676+0000","last_fullsized":"2024-09-16T09:45:44.127676+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:10.214003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:45:10.214003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:38.392450+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"61'2","reported_seq":1104,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:43.756853+0000","last_change":"2024-09-16T09:45:38.729856+0000","last_active":"2024-09-16T09:45:43.756853+0000","last_peered":"2024-09-16T09:45:43.756853+0000","last_clean":"2024-09-16T09:45:43.756853+0000","last_became_active":"2024-09-16T09:25:00.696237+0000","last_became_peered":"2024-09-16T09:25:00.696237+0000","last_unstale":"2024-09-16T09:45:43.756853+0000","last_undegraded":"2024-09-16T09:45:43.756853+0000","last_fullsized":"2024-09-16T09:45:43.756853+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'2","last_scrub_stamp":"2024-09-16T09:45:38.729781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:45:38.729781+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:54.699785+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.0025137430000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1075,"reported_epoch":571,"state":"active+clean","last_fresh":"2024-09-16T09:46:04.854766+0000","last_change":"2024-09-16T09:46:04.062868+0000","last_active":"2024-09-16T09:46:04.854766+0000","last_peered":"2024-09-16T09:46:04.854766+0000","last_clean":"2024-09-16T09:46:04.854766+0000","last_became_active":"2024-09-16T09:25:05.717591+0000","last_became_peered":"2024-09-16T09:25:05.717591+0000","last_unstale":"2024-09-16T09:46:04.854766+0000","last_undegraded":"2024-09-16T09:46:04.854766+0000","last_fullsized":"2024-09-16T09:46:04.854766+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:46:04.062821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:46:04.062821+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:47:26.675568+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":815,"reported_epoch":570,"state":"active+clean","last_fresh":"2024-09-16T09:46:03.604670+0000","last_change":"2024-09-16T09:44:35.215569+0000","last_active":"2024-09-16T09:46:03.604670+0000","last_peered":"2024-09-16T09:46:03.604670+0000","last_clean":"2024-09-16T09:46:03.604670+0000","last_became_active":"2024-09-16T09:31:09.272609+0000","last_became_peered":"2024-09-16T09:31:09.272609+0000","last_unstale":"2024-09-16T09:46:03.604670+0000","last_undegraded":"2024-09-16T09:46:03.604670+0000","last_fullsized":"2024-09-16T09:46:03.604670+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:35.215527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:44:35.215527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"73'79","reported_seq":426,"reported_epoch":572,"state":"active+clean","last_fresh":"2024-09-16T09:46:05.749408+0000","last_change":"2024-09-16T09:45:55.623249+0000","last_active":"2024-09-16T09:46:05.749408+0000","last_peered":"2024-09-16T09:46:05.749408+0000","last_clean":"2024-09-16T09:46:05.749408+0000","last_became_active":"2024-09-16T09:38:10.556320+0000","last_became_peered":"2024-09-16T09:38:10.556320+0000","last_unstale":"2024-09-16T09:46:05.749408+0000","last_undegraded":"2024-09-16T09:46:05.749408+0000","last_fullsized":"2024-09-16T09:46:05.749408+0000","mapping_epoch":386,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":387,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:45:55.623189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:45:55.623189+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:47:15.053219+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"547'21","reported_seq":819,"reported_epoch":570,"state":"active+clean","last_fresh":"2024-09-16T09:46:03.529514+0000","last_change":"2024-09-16T09:45:19.684999+0000","last_active":"2024-09-16T09:46:03.529514+0000","last_peered":"2024-09-16T09:46:03.529514+0000","last_clean":"2024-09-16T09:46:03.529514+0000","last_became_active":"2024-09-16T09:45:19.684353+0000","last_became_peered":"2024-09-16T09:45:19.684353+0000","last_unstale":"2024-09-16T09:46:03.529514+0000","last_undegraded":"2024-09-16T09:46:03.529514+0000","last_fullsized":"2024-09-16T09:46:03.529514+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'17","last_scrub_stamp":"2024-09-16T09:44:59.781721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:44:59.781721+0000","objects_scrubbed":5,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:10.988145+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":40,"num_read_kb":29,"num_write":24,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":657,"reported_epoch":570,"state":"active+clean","last_fresh":"2024-09-16T09:46:03.529111+0000","last_change":"2024-09-16T09:45:47.427056+0000","last_active":"2024-09-16T09:46:03.529111+0000","last_peered":"2024-09-16T09:46:03.529111+0000","last_clean":"2024-09-16T09:46:03.529111+0000","last_became_active":"2024-09-16T09:45:19.680500+0000","last_became_peered":"2024-09-16T09:45:19.680500+0000","last_unstale":"2024-09-16T09:46:03.529111+0000","last_undegraded":"2024-09-16T09:46:03.529111+0000","last_fullsized":"2024-09-16T09:46:03.529111+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:47.427009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:45:47.427009+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:52.111478+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"59'1","reported_seq":595,"reported_epoch":572,"state":"active+clean","last_fresh":"2024-09-16T09:46:05.749142+0000","last_change":"2024-09-16T09:45:43.765136+0000","last_active":"2024-09-16T09:46:05.749142+0000","last_peered":"2024-09-16T09:46:05.749142+0000","last_clean":"2024-09-16T09:46:05.749142+0000","last_became_active":"2024-09-16T09:45:43.764803+0000","last_became_peered":"2024-09-16T09:45:43.764803+0000","last_unstale":"2024-09-16T09:46:05.749142+0000","last_undegraded":"2024-09-16T09:46:05.749142+0000","last_fullsized":"2024-09-16T09:46:05.749142+0000","mapping_epoch":568,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":569,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:45:08.290225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:45:08.290225+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:36.064458+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1019,"reported_epoch":572,"state":"active+clean","last_fresh":"2024-09-16T09:46:05.749961+0000","last_change":"2024-09-16T09:45:42.155707+0000","last_active":"2024-09-16T09:46:05.749961+0000","last_peered":"2024-09-16T09:46:05.749961+0000","last_clean":"2024-09-16T09:46:05.749961+0000","last_became_active":"2024-09-16T09:44:24.285805+0000","last_became_peered":"2024-09-16T09:44:24.285805+0000","last_unstale":"2024-09-16T09:46:05.749961+0000","last_undegraded":"2024-09-16T09:46:05.749961+0000","last_fullsized":"2024-09-16T09:46:05.749961+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:42.155659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:45:42.155659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:48.222155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1079,"reported_epoch":572,"state":"active+clean","last_fresh":"2024-09-16T09:46:05.751824+0000","last_change":"2024-09-16T09:44:48.555675+0000","last_active":"2024-09-16T09:46:05.751824+0000","last_peered":"2024-09-16T09:46:05.751824+0000","last_clean":"2024-09-16T09:46:05.751824+0000","last_became_active":"2024-09-16T09:25:07.723282+0000","last_became_peered":"2024-09-16T09:25:07.723282+0000","last_unstale":"2024-09-16T09:46:05.751824+0000","last_undegraded":"2024-09-16T09:46:05.751824+0000","last_fullsized":"2024-09-16T09:46:05.751824+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:48.555614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:44:48.555614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:18.258254+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"73'79","reported_seq":394,"reported_epoch":570,"state":"active+clean","last_fresh":"2024-09-16T09:46:03.530039+0000","last_change":"2024-09-16T09:45:35.495131+0000","last_active":"2024-09-16T09:46:03.530039+0000","last_peered":"2024-09-16T09:46:03.530039+0000","last_clean":"2024-09-16T09:46:03.530039+0000","last_became_active":"2024-09-16T09:45:19.680539+0000","last_became_peered":"2024-09-16T09:45:19.680539+0000","last_unstale":"2024-09-16T09:46:03.530039+0000","last_undegraded":"2024-09-16T09:46:03.530039+0000","last_fullsized":"2024-09-16T09:46:03.530039+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":3,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:45:35.495071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:45:35.495071+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:59.621866+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"547'36","reported_seq":1151,"reported_epoch":571,"state":"active+clean","last_fresh":"2024-09-16T09:46:04.854941+0000","last_change":"2024-09-16T09:45:26.063751+0000","last_active":"2024-09-16T09:46:04.854941+0000","last_peered":"2024-09-16T09:46:04.854941+0000","last_clean":"2024-09-16T09:46:04.854941+0000","last_became_active":"2024-09-16T09:44:24.277878+0000","last_became_peered":"2024-09-16T09:44:24.277878+0000","last_unstale":"2024-09-16T09:46:04.854941+0000","last_undegraded":"2024-09-16T09:46:04.854941+0000","last_fullsized":"2024-09-16T09:46:04.854941+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"547'36","last_scrub_stamp":"2024-09-16T09:45:26.063692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:45:26.063692+0000","objects_scrubbed":6,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:27.043442+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":54,"num_read_kb":48,"num_write":36,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"59'1","reported_seq":1122,"reported_epoch":571,"state":"active+clean","last_fresh":"2024-09-16T09:46:04.854330+0000","last_change":"2024-09-16T09:45:51.102399+0000","last_active":"2024-09-16T09:46:04.854330+0000","last_peered":"2024-09-16T09:46:04.854330+0000","last_clean":"2024-09-16T09:46:04.854330+0000","last_became_active":"2024-09-16T09:31:09.269627+0000","last_became_peered":"2024-09-16T09:31:09.269627+0000","last_unstale":"2024-09-16T09:46:04.854330+0000","last_undegraded":"2024-09-16T09:46:04.854330+0000","last_fullsized":"2024-09-16T09:46:04.854330+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:45:51.102311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:45:51.102311+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:47:16.896051+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1986,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"67'5","reported_seq":2360,"reported_epoch":572,"state":"active+clean","last_fresh":"2024-09-16T09:46:05.851761+0000","last_change":"2024-09-16T09:45:43.482540+0000","last_active":"2024-09-16T09:46:05.851761+0000","last_peered":"2024-09-16T09:46:05.851761+0000","last_clean":"2024-09-16T09:46:05.851761+0000","last_became_active":"2024-09-16T09:25:00.696546+0000","last_became_peered":"2024-09-16T09:25:00.696546+0000","last_unstale":"2024-09-16T09:46:05.851761+0000","last_undegraded":"2024-09-16T09:46:05.851761+0000","last_fullsized":"2024-09-16T09:46:05.851761+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'5","last_scrub_stamp":"2024-09-16T09:45:43.482461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:24:59.684950+0000","last_clean_scrub_stamp":"2024-09-16T09:45:43.482461+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:47:01.250445+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.0059770420000000001,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1258,"num_read_kb":1253,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1061,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:43.756930+0000","last_change":"2024-09-16T09:45:36.721532+0000","last_active":"2024-09-16T09:45:43.756930+0000","last_peered":"2024-09-16T09:45:43.756930+0000","last_clean":"2024-09-16T09:45:43.756930+0000","last_became_active":"2024-09-16T09:25:05.714704+0000","last_became_peered":"2024-09-16T09:25:05.714704+0000","last_unstale":"2024-09-16T09:45:43.756930+0000","last_undegraded":"2024-09-16T09:45:43.756930+0000","last_fullsized":"2024-09-16T09:45:43.756930+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:36.721477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:45:36.721477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:47:04.067390+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":435,"reported_epoch":570,"state":"active+clean","last_fresh":"2024-09-16T09:46:03.530186+0000","last_change":"2024-09-16T09:45:37.490228+0000","last_active":"2024-09-16T09:46:03.530186+0000","last_peered":"2024-09-16T09:46:03.530186+0000","last_clean":"2024-09-16T09:46:03.530186+0000","last_became_active":"2024-09-16T09:45:19.685809+0000","last_became_peered":"2024-09-16T09:45:19.685809+0000","last_unstale":"2024-09-16T09:46:03.530186+0000","last_undegraded":"2024-09-16T09:46:03.530186+0000","last_fullsized":"2024-09-16T09:46:03.530186+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:37.490179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:45:37.490179+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:56.293422+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"547'40","reported_seq":1404,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:58.303042+0000","last_change":"2024-09-16T09:45:27.591449+0000","last_active":"2024-09-16T09:45:58.303042+0000","last_peered":"2024-09-16T09:45:58.303042+0000","last_clean":"2024-09-16T09:45:58.303042+0000","last_became_active":"2024-09-16T09:25:03.714118+0000","last_became_peered":"2024-09-16T09:25:03.714118+0000","last_unstale":"2024-09-16T09:45:58.303042+0000","last_undegraded":"2024-09-16T09:45:58.303042+0000","last_fullsized":"2024-09-16T09:45:58.303042+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"547'40","last_scrub_stamp":"2024-09-16T09:45:27.591380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:45:27.591380+0000","objects_scrubbed":8,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:57.224993+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":61,"num_read_kb":51,"num_write":37,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":169,"reported_epoch":570,"state":"active+clean","last_fresh":"2024-09-16T09:46:03.603044+0000","last_change":"2024-09-16T09:46:00.862416+0000","last_active":"2024-09-16T09:46:03.603044+0000","last_peered":"2024-09-16T09:46:03.603044+0000","last_clean":"2024-09-16T09:46:03.603044+0000","last_became_active":"2024-09-16T09:45:23.745852+0000","last_became_peered":"2024-09-16T09:45:23.745852+0000","last_unstale":"2024-09-16T09:46:03.603044+0000","last_undegraded":"2024-09-16T09:46:03.603044+0000","last_fullsized":"2024-09-16T09:46:03.603044+0000","mapping_epoch":554,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":555,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:46:00.862359+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:46:00.862359+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:47:12.694803+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"73'79","reported_seq":1175,"reported_epoch":572,"state":"active+clean","last_fresh":"2024-09-16T09:46:05.749737+0000","last_change":"2024-09-16T09:45:28.555936+0000","last_active":"2024-09-16T09:46:05.749737+0000","last_peered":"2024-09-16T09:46:05.749737+0000","last_clean":"2024-09-16T09:46:05.749737+0000","last_became_active":"2024-09-16T09:30:08.870394+0000","last_became_peered":"2024-09-16T09:30:08.870394+0000","last_unstale":"2024-09-16T09:46:05.749737+0000","last_undegraded":"2024-09-16T09:46:05.749737+0000","last_fullsized":"2024-09-16T09:46:05.749737+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:45:28.555859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:45:28.555859+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:29.632294+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1094,"reported_epoch":572,"state":"active+clean","last_fresh":"2024-09-16T09:46:05.751553+0000","last_change":"2024-09-16T09:45:01.668750+0000","last_active":"2024-09-16T09:46:05.751553+0000","last_peered":"2024-09-16T09:46:05.751553+0000","last_clean":"2024-09-16T09:46:05.751553+0000","last_became_active":"2024-09-16T09:44:24.284362+0000","last_became_peered":"2024-09-16T09:44:24.284362+0000","last_unstale":"2024-09-16T09:46:05.751553+0000","last_undegraded":"2024-09-16T09:46:05.751553+0000","last_fullsized":"2024-09-16T09:46:05.751553+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:01.668579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:45:01.668579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:28.528446+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1065,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:43.756798+0000","last_change":"2024-09-16T09:44:46.680178+0000","last_active":"2024-09-16T09:45:43.756798+0000","last_peered":"2024-09-16T09:45:43.756798+0000","last_clean":"2024-09-16T09:45:43.756798+0000","last_became_active":"2024-09-16T09:25:07.714281+0000","last_became_peered":"2024-09-16T09:25:07.714281+0000","last_unstale":"2024-09-16T09:45:43.756798+0000","last_undegraded":"2024-09-16T09:45:43.756798+0000","last_fullsized":"2024-09-16T09:45:43.756798+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:46.680120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:44:46.680120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:08.327463+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.b","version":"73'79","reported_seq":405,"reported_epoch":570,"state":"active+clean","last_fresh":"2024-09-16T09:46:03.529691+0000","last_change":"2024-09-16T09:45:22.401820+0000","last_active":"2024-09-16T09:46:03.529691+0000","last_peered":"2024-09-16T09:46:03.529691+0000","last_clean":"2024-09-16T09:46:03.529691+0000","last_became_active":"2024-09-16T09:45:19.682878+0000","last_became_peered":"2024-09-16T09:45:19.682878+0000","last_unstale":"2024-09-16T09:46:03.529691+0000","last_undegraded":"2024-09-16T09:46:03.529691+0000","last_fullsized":"2024-09-16T09:46:03.529691+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:45:22.401770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:45:22.401770+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:44.196429+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,5],"acting":[1,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"547'27","reported_seq":1174,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:44.128054+0000","last_change":"2024-09-16T09:45:37.085157+0000","last_active":"2024-09-16T09:45:44.128054+0000","last_peered":"2024-09-16T09:45:44.128054+0000","last_clean":"2024-09-16T09:45:44.128054+0000","last_became_active":"2024-09-16T09:37:33.348433+0000","last_became_peered":"2024-09-16T09:37:33.348433+0000","last_unstale":"2024-09-16T09:45:44.128054+0000","last_undegraded":"2024-09-16T09:45:44.128054+0000","last_fullsized":"2024-09-16T09:45:44.128054+0000","mapping_epoch":367,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":368,"parent":"0.0","parent_split_bits":0,"last_scrub":"547'27","last_scrub_stamp":"2024-09-16T09:45:37.085092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:45:37.085092+0000","objects_scrubbed":5,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:44.425422+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":37,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":1081,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:44.128083+0000","last_change":"2024-09-16T09:45:20.179511+0000","last_active":"2024-09-16T09:45:44.128083+0000","last_peered":"2024-09-16T09:45:44.128083+0000","last_clean":"2024-09-16T09:45:44.128083+0000","last_became_active":"2024-09-16T09:44:24.276997+0000","last_became_peered":"2024-09-16T09:44:24.276997+0000","last_unstale":"2024-09-16T09:45:44.128083+0000","last_undegraded":"2024-09-16T09:45:44.128083+0000","last_fullsized":"2024-09-16T09:45:44.128083+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:20.179440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:45:20.179440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:23.557282+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1053,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:43.756623+0000","last_change":"2024-09-16T09:45:22.786814+0000","last_active":"2024-09-16T09:45:43.756623+0000","last_peered":"2024-09-16T09:45:43.756623+0000","last_clean":"2024-09-16T09:45:43.756623+0000","last_became_active":"2024-09-16T09:25:05.711964+0000","last_became_peered":"2024-09-16T09:25:05.711964+0000","last_unstale":"2024-09-16T09:45:43.756623+0000","last_undegraded":"2024-09-16T09:45:43.756623+0000","last_fullsized":"2024-09-16T09:45:43.756623+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:22.786754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:45:22.786754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:35.253260+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":178,"reported_epoch":570,"state":"active+clean","last_fresh":"2024-09-16T09:46:03.604876+0000","last_change":"2024-09-16T09:45:23.746036+0000","last_active":"2024-09-16T09:46:03.604876+0000","last_peered":"2024-09-16T09:46:03.604876+0000","last_clean":"2024-09-16T09:46:03.604876+0000","last_became_active":"2024-09-16T09:45:23.745723+0000","last_became_peered":"2024-09-16T09:45:23.745723+0000","last_unstale":"2024-09-16T09:46:03.604876+0000","last_undegraded":"2024-09-16T09:46:03.604876+0000","last_fullsized":"2024-09-16T09:46:03.604876+0000","mapping_epoch":554,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":555,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:09.717025+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:45:09.717025+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:17.342433+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.8","version":"73'79","reported_seq":441,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:43.755612+0000","last_change":"2024-09-16T09:45:04.792552+0000","last_active":"2024-09-16T09:45:43.755612+0000","last_peered":"2024-09-16T09:45:43.755612+0000","last_clean":"2024-09-16T09:45:43.755612+0000","last_became_active":"2024-09-16T09:45:03.608887+0000","last_became_peered":"2024-09-16T09:45:03.608887+0000","last_unstale":"2024-09-16T09:45:43.755612+0000","last_undegraded":"2024-09-16T09:45:43.755612+0000","last_fullsized":"2024-09-16T09:45:43.755612+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":543,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:45:04.792500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:45:04.792500+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:21.463945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"547'45","reported_seq":1163,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:44.128297+0000","last_change":"2024-09-16T09:45:24.217342+0000","last_active":"2024-09-16T09:45:44.128297+0000","last_peered":"2024-09-16T09:45:44.128297+0000","last_clean":"2024-09-16T09:45:44.128297+0000","last_became_active":"2024-09-16T09:44:24.277359+0000","last_became_peered":"2024-09-16T09:44:24.277359+0000","last_unstale":"2024-09-16T09:45:44.128297+0000","last_undegraded":"2024-09-16T09:45:44.128297+0000","last_fullsized":"2024-09-16T09:45:44.128297+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"547'45","last_scrub_stamp":"2024-09-16T09:45:24.217274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:45:24.217274+0000","objects_scrubbed":8,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:47.371200+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":71,"num_read_kb":61,"num_write":46,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":1088,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:44.400418+0000","last_change":"2024-09-16T09:45:11.709864+0000","last_active":"2024-09-16T09:45:44.400418+0000","last_peered":"2024-09-16T09:45:44.400418+0000","last_clean":"2024-09-16T09:45:44.400418+0000","last_became_active":"2024-09-16T09:25:01.703355+0000","last_became_peered":"2024-09-16T09:25:01.703355+0000","last_unstale":"2024-09-16T09:45:44.400418+0000","last_undegraded":"2024-09-16T09:45:44.400418+0000","last_fullsized":"2024-09-16T09:45:44.400418+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:11.709798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:45:11.709798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:31.626577+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":424,"reported_epoch":570,"state":"active+clean","last_fresh":"2024-09-16T09:46:03.530197+0000","last_change":"2024-09-16T09:45:19.683602+0000","last_active":"2024-09-16T09:46:03.530197+0000","last_peered":"2024-09-16T09:46:03.530197+0000","last_clean":"2024-09-16T09:46:03.530197+0000","last_became_active":"2024-09-16T09:45:19.683227+0000","last_became_peered":"2024-09-16T09:45:19.683227+0000","last_unstale":"2024-09-16T09:46:03.530197+0000","last_undegraded":"2024-09-16T09:46:03.530197+0000","last_fullsized":"2024-09-16T09:46:03.530197+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:47.324096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:44:47.324096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:08.892743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":924,"reported_epoch":572,"state":"active+clean","last_fresh":"2024-09-16T09:46:05.750089+0000","last_change":"2024-09-16T09:45:45.127669+0000","last_active":"2024-09-16T09:46:05.750089+0000","last_peered":"2024-09-16T09:46:05.750089+0000","last_clean":"2024-09-16T09:46:05.750089+0000","last_became_active":"2024-09-16T09:45:44.127833+0000","last_became_peered":"2024-09-16T09:45:44.127833+0000","last_unstale":"2024-09-16T09:46:05.750089+0000","last_undegraded":"2024-09-16T09:46:05.750089+0000","last_fullsized":"2024-09-16T09:46:05.750089+0000","mapping_epoch":568,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":569,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:45.127611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:45:45.127611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:47:11.070158+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.9","version":"73'79","reported_seq":1176,"reported_epoch":572,"state":"active+clean","last_fresh":"2024-09-16T09:46:05.749767+0000","last_change":"2024-09-16T09:45:04.693404+0000","last_active":"2024-09-16T09:46:05.749767+0000","last_peered":"2024-09-16T09:46:05.749767+0000","last_clean":"2024-09-16T09:46:05.749767+0000","last_became_active":"2024-09-16T09:45:03.614687+0000","last_became_peered":"2024-09-16T09:45:03.614687+0000","last_unstale":"2024-09-16T09:46:05.749767+0000","last_undegraded":"2024-09-16T09:46:05.749767+0000","last_fullsized":"2024-09-16T09:46:05.749767+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":543,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:45:04.693352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:45:04.693352+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:29.565272+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"547'30","reported_seq":1119,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:47.195286+0000","last_change":"2024-09-16T09:45:47.195286+0000","last_active":"2024-09-16T09:45:47.195286+0000","last_peered":"2024-09-16T09:45:47.195286+0000","last_clean":"2024-09-16T09:45:47.195286+0000","last_became_active":"2024-09-16T09:25:03.707723+0000","last_became_peered":"2024-09-16T09:25:03.707723+0000","last_unstale":"2024-09-16T09:45:47.195286+0000","last_undegraded":"2024-09-16T09:45:47.195286+0000","last_fullsized":"2024-09-16T09:45:47.195286+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"547'30","last_scrub_stamp":"2024-09-16T09:45:47.195208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:45:47.195208+0000","objects_scrubbed":5,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:47:06.380855+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":45,"num_read_kb":40,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":177,"reported_epoch":570,"state":"active+clean","last_fresh":"2024-09-16T09:46:03.604184+0000","last_change":"2024-09-16T09:45:34.050738+0000","last_active":"2024-09-16T09:46:03.604184+0000","last_peered":"2024-09-16T09:46:03.604184+0000","last_clean":"2024-09-16T09:46:03.604184+0000","last_became_active":"2024-09-16T09:45:23.695995+0000","last_became_peered":"2024-09-16T09:45:23.695995+0000","last_unstale":"2024-09-16T09:46:03.604184+0000","last_undegraded":"2024-09-16T09:46:03.604184+0000","last_fullsized":"2024-09-16T09:46:03.604184+0000","mapping_epoch":554,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":555,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:34.050691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:45:34.050691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:42.361284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"569'12","reported_seq":813,"reported_epoch":572,"state":"active+clean","last_fresh":"2024-09-16T09:46:05.749983+0000","last_change":"2024-09-16T09:45:44.112875+0000","last_active":"2024-09-16T09:46:05.749983+0000","last_peered":"2024-09-16T09:46:05.749983+0000","last_clean":"2024-09-16T09:46:05.749983+0000","last_became_active":"2024-09-16T09:45:43.760279+0000","last_became_peered":"2024-09-16T09:45:43.760279+0000","last_unstale":"2024-09-16T09:46:05.749983+0000","last_undegraded":"2024-09-16T09:46:05.749983+0000","last_fullsized":"2024-09-16T09:46:05.749983+0000","mapping_epoch":568,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":569,"parent":"0.0","parent_split_bits":0,"last_scrub":"519'11","last_scrub_stamp":"2024-09-16T09:45:44.112817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:45:44.112817+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:47:07.166260+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1101,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:44.127879+0000","last_change":"2024-09-16T09:44:48.293840+0000","last_active":"2024-09-16T09:45:44.127879+0000","last_peered":"2024-09-16T09:45:44.127879+0000","last_clean":"2024-09-16T09:45:44.127879+0000","last_became_active":"2024-09-16T09:44:24.280724+0000","last_became_peered":"2024-09-16T09:44:24.280724+0000","last_unstale":"2024-09-16T09:45:44.127879+0000","last_undegraded":"2024-09-16T09:45:44.127879+0000","last_fullsized":"2024-09-16T09:45:44.127879+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:48.293789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:44:48.293789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:57.968474+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.e","version":"73'79","reported_seq":885,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:44.127779+0000","last_change":"2024-09-16T09:45:05.247594+0000","last_active":"2024-09-16T09:45:44.127779+0000","last_peered":"2024-09-16T09:45:44.127779+0000","last_clean":"2024-09-16T09:45:44.127779+0000","last_became_active":"2024-09-16T09:45:03.613758+0000","last_became_peered":"2024-09-16T09:45:03.613758+0000","last_unstale":"2024-09-16T09:45:44.127779+0000","last_undegraded":"2024-09-16T09:45:44.127779+0000","last_fullsized":"2024-09-16T09:45:44.127779+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":543,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:45:05.247539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:45:05.247539+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:30.185192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,3],"acting":[4,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.b","version":"547'21","reported_seq":1137,"reported_epoch":571,"state":"active+clean","last_fresh":"2024-09-16T09:46:04.854741+0000","last_change":"2024-09-16T09:44:48.083967+0000","last_active":"2024-09-16T09:46:04.854741+0000","last_peered":"2024-09-16T09:46:04.854741+0000","last_clean":"2024-09-16T09:46:04.854741+0000","last_became_active":"2024-09-16T09:36:31.590202+0000","last_became_peered":"2024-09-16T09:36:31.590202+0000","last_unstale":"2024-09-16T09:46:04.854741+0000","last_undegraded":"2024-09-16T09:46:04.854741+0000","last_fullsized":"2024-09-16T09:46:04.854741+0000","mapping_epoch":316,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":317,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'15","last_scrub_stamp":"2024-09-16T09:44:48.083895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:44:48.083895+0000","objects_scrubbed":4,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":1083,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:43.756347+0000","last_change":"2024-09-16T09:45:32.711419+0000","last_active":"2024-09-16T09:45:43.756347+0000","last_peered":"2024-09-16T09:45:43.756347+0000","last_clean":"2024-09-16T09:45:43.756347+0000","last_became_active":"2024-09-16T09:31:09.276516+0000","last_became_peered":"2024-09-16T09:31:09.276516+0000","last_unstale":"2024-09-16T09:45:43.756347+0000","last_undegraded":"2024-09-16T09:45:43.756347+0000","last_fullsized":"2024-09-16T09:45:43.756347+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:32.711339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:45:32.711339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:40.490158+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":913,"reported_epoch":572,"state":"active+clean","last_fresh":"2024-09-16T09:46:05.749696+0000","last_change":"2024-09-16T09:45:44.128762+0000","last_active":"2024-09-16T09:46:05.749696+0000","last_peered":"2024-09-16T09:46:05.749696+0000","last_clean":"2024-09-16T09:46:05.749696+0000","last_became_active":"2024-09-16T09:45:44.128387+0000","last_became_peered":"2024-09-16T09:45:44.128387+0000","last_unstale":"2024-09-16T09:46:05.749696+0000","last_undegraded":"2024-09-16T09:46:05.749696+0000","last_fullsized":"2024-09-16T09:46:05.749696+0000","mapping_epoch":568,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":569,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:36.088166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:45:36.088166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:54.319195+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1089,"reported_epoch":571,"state":"active+clean","last_fresh":"2024-09-16T09:46:04.854286+0000","last_change":"2024-09-16T09:45:30.041583+0000","last_active":"2024-09-16T09:46:04.854286+0000","last_peered":"2024-09-16T09:46:04.854286+0000","last_clean":"2024-09-16T09:46:04.854286+0000","last_became_active":"2024-09-16T09:44:24.278714+0000","last_became_peered":"2024-09-16T09:44:24.278714+0000","last_unstale":"2024-09-16T09:46:04.854286+0000","last_undegraded":"2024-09-16T09:46:04.854286+0000","last_fullsized":"2024-09-16T09:46:04.854286+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:30.041535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:45:30.041535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:37.082005+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.f","version":"73'79","reported_seq":1149,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:44.400519+0000","last_change":"2024-09-16T09:45:25.579212+0000","last_active":"2024-09-16T09:45:44.400519+0000","last_peered":"2024-09-16T09:45:44.400519+0000","last_clean":"2024-09-16T09:45:44.400519+0000","last_became_active":"2024-09-16T09:30:10.693477+0000","last_became_peered":"2024-09-16T09:30:10.693477+0000","last_unstale":"2024-09-16T09:45:44.400519+0000","last_undegraded":"2024-09-16T09:45:44.400519+0000","last_fullsized":"2024-09-16T09:45:44.400519+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:45:25.579158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:45:25.579158+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:51.377246+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"547'51","reported_seq":1163,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:44.399274+0000","last_change":"2024-09-16T09:45:17.716010+0000","last_active":"2024-09-16T09:45:44.399274+0000","last_peered":"2024-09-16T09:45:44.399274+0000","last_clean":"2024-09-16T09:45:44.399274+0000","last_became_active":"2024-09-16T09:25:03.712783+0000","last_became_peered":"2024-09-16T09:25:03.712783+0000","last_unstale":"2024-09-16T09:45:44.399274+0000","last_undegraded":"2024-09-16T09:45:44.399274+0000","last_fullsized":"2024-09-16T09:45:44.399274+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"547'51","last_scrub_stamp":"2024-09-16T09:45:17.715920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:45:17.715920+0000","objects_scrubbed":9,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:47.081023+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":80,"num_read_kb":69,"num_write":52,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"59'1","reported_seq":1096,"reported_epoch":572,"state":"active+clean","last_fresh":"2024-09-16T09:46:05.752204+0000","last_change":"2024-09-16T09:45:26.499588+0000","last_active":"2024-09-16T09:46:05.752204+0000","last_peered":"2024-09-16T09:46:05.752204+0000","last_clean":"2024-09-16T09:46:05.752204+0000","last_became_active":"2024-09-16T09:25:01.708069+0000","last_became_peered":"2024-09-16T09:25:01.708069+0000","last_unstale":"2024-09-16T09:46:05.752204+0000","last_undegraded":"2024-09-16T09:46:05.752204+0000","last_fullsized":"2024-09-16T09:46:05.752204+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'1","last_scrub_stamp":"2024-09-16T09:45:26.499523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:45:26.499523+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:39.186250+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1059,"reported_epoch":572,"state":"active+clean","last_fresh":"2024-09-16T09:46:05.749103+0000","last_change":"2024-09-16T09:46:04.125868+0000","last_active":"2024-09-16T09:46:05.749103+0000","last_peered":"2024-09-16T09:46:05.749103+0000","last_clean":"2024-09-16T09:46:05.749103+0000","last_became_active":"2024-09-16T09:44:24.278290+0000","last_became_peered":"2024-09-16T09:44:24.278290+0000","last_unstale":"2024-09-16T09:46:05.749103+0000","last_undegraded":"2024-09-16T09:46:05.749103+0000","last_fullsized":"2024-09-16T09:46:05.749103+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:46:04.125796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:46:04.125796+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:47:22.361432+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1073,"reported_epoch":572,"state":"active+clean","last_fresh":"2024-09-16T09:46:05.749712+0000","last_change":"2024-09-16T09:45:02.668718+0000","last_active":"2024-09-16T09:46:05.749712+0000","last_peered":"2024-09-16T09:46:05.749712+0000","last_clean":"2024-09-16T09:46:05.749712+0000","last_became_active":"2024-09-16T09:44:24.275407+0000","last_became_peered":"2024-09-16T09:44:24.275407+0000","last_unstale":"2024-09-16T09:46:05.749712+0000","last_undegraded":"2024-09-16T09:46:05.749712+0000","last_fullsized":"2024-09-16T09:46:05.749712+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:02.668668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:45:02.668668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:08.212308+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.c","version":"73'79","reported_seq":854,"reported_epoch":570,"state":"active+clean","last_fresh":"2024-09-16T09:46:03.604716+0000","last_change":"2024-09-16T09:45:05.176489+0000","last_active":"2024-09-16T09:46:03.604716+0000","last_peered":"2024-09-16T09:46:03.604716+0000","last_clean":"2024-09-16T09:46:03.604716+0000","last_became_active":"2024-09-16T09:45:03.613008+0000","last_became_peered":"2024-09-16T09:45:03.613008+0000","last_unstale":"2024-09-16T09:46:03.604716+0000","last_undegraded":"2024-09-16T09:46:03.604716+0000","last_fullsized":"2024-09-16T09:46:03.604716+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":543,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:45:05.176413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:45:05.176413+0000","objects_scrubbed":1,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:24.104658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"547'24","reported_seq":1161,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:44.127535+0000","last_change":"2024-09-16T09:45:22.192977+0000","last_active":"2024-09-16T09:45:44.127535+0000","last_peered":"2024-09-16T09:45:44.127535+0000","last_clean":"2024-09-16T09:45:44.127535+0000","last_became_active":"2024-09-16T09:31:09.269047+0000","last_became_peered":"2024-09-16T09:31:09.269047+0000","last_unstale":"2024-09-16T09:45:44.127535+0000","last_undegraded":"2024-09-16T09:45:44.127535+0000","last_fullsized":"2024-09-16T09:45:44.127535+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"547'24","last_scrub_stamp":"2024-09-16T09:45:22.192863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:45:22.192863+0000","objects_scrubbed":5,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:48.200457+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":1090,"reported_epoch":572,"state":"active+clean","last_fresh":"2024-09-16T09:46:05.752214+0000","last_change":"2024-09-16T09:45:25.455157+0000","last_active":"2024-09-16T09:46:05.752214+0000","last_peered":"2024-09-16T09:46:05.752214+0000","last_clean":"2024-09-16T09:46:05.752214+0000","last_became_active":"2024-09-16T09:25:01.707528+0000","last_became_peered":"2024-09-16T09:25:01.707528+0000","last_unstale":"2024-09-16T09:46:05.752214+0000","last_undegraded":"2024-09-16T09:46:05.752214+0000","last_fullsized":"2024-09-16T09:46:05.752214+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:25.455094+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:45:25.455094+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:47.601779+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":647,"reported_epoch":572,"state":"active+clean","last_fresh":"2024-09-16T09:46:05.749077+0000","last_change":"2024-09-16T09:45:43.765462+0000","last_active":"2024-09-16T09:46:05.749077+0000","last_peered":"2024-09-16T09:46:05.749077+0000","last_clean":"2024-09-16T09:46:05.749077+0000","last_became_active":"2024-09-16T09:45:43.765067+0000","last_became_peered":"2024-09-16T09:45:43.765067+0000","last_unstale":"2024-09-16T09:46:05.749077+0000","last_undegraded":"2024-09-16T09:46:05.749077+0000","last_fullsized":"2024-09-16T09:46:05.749077+0000","mapping_epoch":568,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":569,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:20.175105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:45:20.175105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:42.875854+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1078,"reported_epoch":570,"state":"active+clean","last_fresh":"2024-09-16T09:46:03.603908+0000","last_change":"2024-09-16T09:45:09.157493+0000","last_active":"2024-09-16T09:46:03.603908+0000","last_peered":"2024-09-16T09:46:03.603908+0000","last_clean":"2024-09-16T09:46:03.603908+0000","last_became_active":"2024-09-16T09:25:07.718899+0000","last_became_peered":"2024-09-16T09:25:07.718899+0000","last_unstale":"2024-09-16T09:46:03.603908+0000","last_undegraded":"2024-09-16T09:46:03.603908+0000","last_fullsized":"2024-09-16T09:46:03.603908+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:09.157451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:45:09.157451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:25.587581+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.d","version":"73'79","reported_seq":890,"reported_epoch":572,"state":"active+clean","last_fresh":"2024-09-16T09:46:05.751605+0000","last_change":"2024-09-16T09:45:03.697649+0000","last_active":"2024-09-16T09:46:05.751605+0000","last_peered":"2024-09-16T09:46:05.751605+0000","last_clean":"2024-09-16T09:46:05.751605+0000","last_became_active":"2024-09-16T09:45:03.613387+0000","last_became_peered":"2024-09-16T09:45:03.613387+0000","last_unstale":"2024-09-16T09:46:05.751605+0000","last_undegraded":"2024-09-16T09:46:05.751605+0000","last_fullsized":"2024-09-16T09:46:05.751605+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":543,"parent":"0.0","parent_split_bits":4,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:45:03.697560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:45:03.697560+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:10.630174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"547'39","reported_seq":1142,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:43.755786+0000","last_change":"2024-09-16T09:45:42.781753+0000","last_active":"2024-09-16T09:45:43.755786+0000","last_peered":"2024-09-16T09:45:43.755786+0000","last_clean":"2024-09-16T09:45:43.755786+0000","last_became_active":"2024-09-16T09:25:03.715733+0000","last_became_peered":"2024-09-16T09:25:03.715733+0000","last_unstale":"2024-09-16T09:45:43.755786+0000","last_undegraded":"2024-09-16T09:45:43.755786+0000","last_fullsized":"2024-09-16T09:45:43.755786+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"547'39","last_scrub_stamp":"2024-09-16T09:45:42.781672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:45:42.781672+0000","objects_scrubbed":7,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:57.480567+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":62,"num_read_kb":53,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"59'2","reported_seq":1094,"reported_epoch":572,"state":"active+clean","last_fresh":"2024-09-16T09:46:05.752155+0000","last_change":"2024-09-16T09:45:21.533534+0000","last_active":"2024-09-16T09:46:05.752155+0000","last_peered":"2024-09-16T09:46:05.752155+0000","last_clean":"2024-09-16T09:46:05.752155+0000","last_became_active":"2024-09-16T09:25:01.707372+0000","last_became_peered":"2024-09-16T09:25:01.707372+0000","last_unstale":"2024-09-16T09:46:05.752155+0000","last_undegraded":"2024-09-16T09:46:05.752155+0000","last_fullsized":"2024-09-16T09:46:05.752155+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2024-09-16T09:45:21.533475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:45:21.533475+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:48.426116+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"66'8","reported_seq":1352,"reported_epoch":572,"state":"active+clean","last_fresh":"2024-09-16T09:46:05.749716+0000","last_change":"2024-09-16T09:45:31.616897+0000","last_active":"2024-09-16T09:46:05.749716+0000","last_peered":"2024-09-16T09:46:05.749716+0000","last_clean":"2024-09-16T09:46:05.749716+0000","last_became_active":"2024-09-16T09:25:05.716189+0000","last_became_peered":"2024-09-16T09:25:05.716189+0000","last_unstale":"2024-09-16T09:46:05.749716+0000","last_undegraded":"2024-09-16T09:46:05.749716+0000","last_fullsized":"2024-09-16T09:46:05.749716+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:45:31.616835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:45:31.616835+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:36.164090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1065,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:46.781350+0000","last_change":"2024-09-16T09:45:46.781350+0000","last_active":"2024-09-16T09:45:46.781350+0000","last_peered":"2024-09-16T09:45:46.781350+0000","last_clean":"2024-09-16T09:45:46.781350+0000","last_became_active":"2024-09-16T09:25:07.717213+0000","last_became_peered":"2024-09-16T09:25:07.717213+0000","last_unstale":"2024-09-16T09:45:46.781350+0000","last_undegraded":"2024-09-16T09:45:46.781350+0000","last_fullsized":"2024-09-16T09:45:46.781350+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:46.781287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:45:46.781287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:49.691824+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.12","version":"73'79","reported_seq":886,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:43.756559+0000","last_change":"2024-09-16T09:45:01.754225+0000","last_active":"2024-09-16T09:45:43.756559+0000","last_peered":"2024-09-16T09:45:43.756559+0000","last_clean":"2024-09-16T09:45:43.756559+0000","last_became_active":"2024-09-16T09:30:10.731413+0000","last_became_peered":"2024-09-16T09:30:10.731413+0000","last_unstale":"2024-09-16T09:45:43.756559+0000","last_undegraded":"2024-09-16T09:45:43.756559+0000","last_fullsized":"2024-09-16T09:45:43.756559+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:45:01.754170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:45:01.754170+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:31.383586+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":1094,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:44.399502+0000","last_change":"2024-09-16T09:45:31.631877+0000","last_active":"2024-09-16T09:45:44.399502+0000","last_peered":"2024-09-16T09:45:44.399502+0000","last_clean":"2024-09-16T09:45:44.399502+0000","last_became_active":"2024-09-16T09:37:53.518354+0000","last_became_peered":"2024-09-16T09:37:53.518354+0000","last_unstale":"2024-09-16T09:45:44.399502+0000","last_undegraded":"2024-09-16T09:45:44.399502+0000","last_fullsized":"2024-09-16T09:45:44.399502+0000","mapping_epoch":382,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":383,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:31.631815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:45:31.631815+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:54.944410+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"547'18","reported_seq":875,"reported_epoch":570,"state":"active+clean","last_fresh":"2024-09-16T09:46:03.603993+0000","last_change":"2024-09-16T09:45:43.126151+0000","last_active":"2024-09-16T09:46:03.603993+0000","last_peered":"2024-09-16T09:46:03.603993+0000","last_clean":"2024-09-16T09:46:03.603993+0000","last_became_active":"2024-09-16T09:44:24.274107+0000","last_became_peered":"2024-09-16T09:44:24.274107+0000","last_unstale":"2024-09-16T09:46:03.603993+0000","last_undegraded":"2024-09-16T09:46:03.603993+0000","last_fullsized":"2024-09-16T09:46:03.603993+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"547'18","last_scrub_stamp":"2024-09-16T09:45:43.126099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:45:43.126099+0000","objects_scrubbed":3,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:47:08.520232+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":26,"num_read_kb":23,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1076,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:43.756509+0000","last_change":"2024-09-16T09:44:54.767506+0000","last_active":"2024-09-16T09:45:43.756509+0000","last_peered":"2024-09-16T09:45:43.756509+0000","last_clean":"2024-09-16T09:45:43.756509+0000","last_became_active":"2024-09-16T09:25:05.715616+0000","last_became_peered":"2024-09-16T09:25:05.715616+0000","last_unstale":"2024-09-16T09:45:43.756509+0000","last_undegraded":"2024-09-16T09:45:43.756509+0000","last_fullsized":"2024-09-16T09:45:43.756509+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:54.767444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:44:54.767444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:17.133885+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1069,"reported_epoch":572,"state":"active+clean","last_fresh":"2024-09-16T09:46:05.751697+0000","last_change":"2024-09-16T09:44:46.528000+0000","last_active":"2024-09-16T09:46:05.751697+0000","last_peered":"2024-09-16T09:46:05.751697+0000","last_clean":"2024-09-16T09:46:05.751697+0000","last_became_active":"2024-09-16T09:25:07.721059+0000","last_became_peered":"2024-09-16T09:25:07.721059+0000","last_unstale":"2024-09-16T09:46:05.751697+0000","last_undegraded":"2024-09-16T09:46:05.751697+0000","last_fullsized":"2024-09-16T09:46:05.751697+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:46.527936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:44:46.527936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.13","version":"73'79","reported_seq":407,"reported_epoch":570,"state":"active+clean","last_fresh":"2024-09-16T09:46:03.529894+0000","last_change":"2024-09-16T09:45:45.499996+0000","last_active":"2024-09-16T09:46:03.529894+0000","last_peered":"2024-09-16T09:46:03.529894+0000","last_clean":"2024-09-16T09:46:03.529894+0000","last_became_active":"2024-09-16T09:45:19.680126+0000","last_became_peered":"2024-09-16T09:45:19.680126+0000","last_unstale":"2024-09-16T09:46:03.529894+0000","last_undegraded":"2024-09-16T09:46:03.529894+0000","last_fullsized":"2024-09-16T09:46:03.529894+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:45:45.499932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:45:45.499932+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:47:08.613886+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.16","version":"547'21","reported_seq":1169,"reported_epoch":571,"state":"active+clean","last_fresh":"2024-09-16T09:46:04.854994+0000","last_change":"2024-09-16T09:45:39.973119+0000","last_active":"2024-09-16T09:46:04.854994+0000","last_peered":"2024-09-16T09:46:04.854994+0000","last_clean":"2024-09-16T09:46:04.854994+0000","last_became_active":"2024-09-16T09:31:09.273498+0000","last_became_peered":"2024-09-16T09:31:09.273498+0000","last_unstale":"2024-09-16T09:46:04.854994+0000","last_undegraded":"2024-09-16T09:46:04.854994+0000","last_fullsized":"2024-09-16T09:46:04.854994+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"547'21","last_scrub_stamp":"2024-09-16T09:45:39.973061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:45:39.973061+0000","objects_scrubbed":4,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:47:03.358051+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":1098,"reported_epoch":572,"state":"active+clean","last_fresh":"2024-09-16T09:46:05.749984+0000","last_change":"2024-09-16T09:45:36.483091+0000","last_active":"2024-09-16T09:46:05.749984+0000","last_peered":"2024-09-16T09:46:05.749984+0000","last_clean":"2024-09-16T09:46:05.749984+0000","last_became_active":"2024-09-16T09:25:01.706491+0000","last_became_peered":"2024-09-16T09:25:01.706491+0000","last_unstale":"2024-09-16T09:46:05.749984+0000","last_undegraded":"2024-09-16T09:46:05.749984+0000","last_fullsized":"2024-09-16T09:46:05.749984+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:36.483032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:45:36.483032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:42.419262+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":270,"reported_epoch":570,"state":"active+clean","last_fresh":"2024-09-16T09:46:03.603845+0000","last_change":"2024-09-16T09:45:23.695102+0000","last_active":"2024-09-16T09:46:03.603845+0000","last_peered":"2024-09-16T09:46:03.603845+0000","last_clean":"2024-09-16T09:46:03.603845+0000","last_became_active":"2024-09-16T09:45:23.694839+0000","last_became_peered":"2024-09-16T09:45:23.694839+0000","last_unstale":"2024-09-16T09:46:03.603845+0000","last_undegraded":"2024-09-16T09:46:03.603845+0000","last_fullsized":"2024-09-16T09:46:03.603845+0000","mapping_epoch":554,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":555,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:11.560613+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:45:11.560613+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:41.502754+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1053,"reported_epoch":572,"state":"active+clean","last_fresh":"2024-09-16T09:46:05.749614+0000","last_change":"2024-09-16T09:44:46.061164+0000","last_active":"2024-09-16T09:46:05.749614+0000","last_peered":"2024-09-16T09:46:05.749614+0000","last_clean":"2024-09-16T09:46:05.749614+0000","last_became_active":"2024-09-16T09:44:24.277612+0000","last_became_peered":"2024-09-16T09:44:24.277612+0000","last_unstale":"2024-09-16T09:46:05.749614+0000","last_undegraded":"2024-09-16T09:46:05.749614+0000","last_fullsized":"2024-09-16T09:46:05.749614+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:46.061101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:44:46.061101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.10","version":"73'79","reported_seq":878,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:43.755585+0000","last_change":"2024-09-16T09:44:43.656131+0000","last_active":"2024-09-16T09:45:43.755585+0000","last_peered":"2024-09-16T09:45:43.755585+0000","last_clean":"2024-09-16T09:45:43.755585+0000","last_became_active":"2024-09-16T09:30:10.692770+0000","last_became_peered":"2024-09-16T09:30:10.692770+0000","last_unstale":"2024-09-16T09:45:43.755585+0000","last_undegraded":"2024-09-16T09:45:43.755585+0000","last_fullsized":"2024-09-16T09:45:43.755585+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:44:43.656039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:44:43.656039+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:01.808196+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"547'21","reported_seq":1114,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:43.756758+0000","last_change":"2024-09-16T09:44:57.752956+0000","last_active":"2024-09-16T09:45:43.756758+0000","last_peered":"2024-09-16T09:45:43.756758+0000","last_clean":"2024-09-16T09:45:43.756758+0000","last_became_active":"2024-09-16T09:25:03.717310+0000","last_became_peered":"2024-09-16T09:25:03.717310+0000","last_unstale":"2024-09-16T09:45:43.756758+0000","last_undegraded":"2024-09-16T09:45:43.756758+0000","last_fullsized":"2024-09-16T09:45:43.756758+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'15","last_scrub_stamp":"2024-09-16T09:44:57.752868+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:44:57.752868+0000","objects_scrubbed":4,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:14.909445+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":1112,"reported_epoch":571,"state":"active+clean","last_fresh":"2024-09-16T09:46:04.854555+0000","last_change":"2024-09-16T09:44:35.099469+0000","last_active":"2024-09-16T09:46:04.854555+0000","last_peered":"2024-09-16T09:46:04.854555+0000","last_clean":"2024-09-16T09:46:04.854555+0000","last_became_active":"2024-09-16T09:31:09.272848+0000","last_became_peered":"2024-09-16T09:31:09.272848+0000","last_unstale":"2024-09-16T09:46:04.854555+0000","last_undegraded":"2024-09-16T09:46:04.854555+0000","last_fullsized":"2024-09-16T09:46:04.854555+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:35.099420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:44:35.099420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"557'14","reported_seq":241,"reported_epoch":570,"state":"active+clean","last_fresh":"2024-09-16T09:46:03.604623+0000","last_change":"2024-09-16T09:45:25.123664+0000","last_active":"2024-09-16T09:46:03.604623+0000","last_peered":"2024-09-16T09:46:03.604623+0000","last_clean":"2024-09-16T09:46:03.604623+0000","last_became_active":"2024-09-16T09:45:23.693762+0000","last_became_peered":"2024-09-16T09:45:23.693762+0000","last_unstale":"2024-09-16T09:46:03.604623+0000","last_undegraded":"2024-09-16T09:46:03.604623+0000","last_fullsized":"2024-09-16T09:46:03.604623+0000","mapping_epoch":554,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":555,"parent":"0.0","parent_split_bits":0,"last_scrub":"519'13","last_scrub_stamp":"2024-09-16T09:45:25.123612+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:45:25.123612+0000","objects_scrubbed":1,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:49.886509+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1100,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:44.127927+0000","last_change":"2024-09-16T09:44:54.304603+0000","last_active":"2024-09-16T09:45:44.127927+0000","last_peered":"2024-09-16T09:45:44.127927+0000","last_clean":"2024-09-16T09:45:44.127927+0000","last_became_active":"2024-09-16T09:44:24.277124+0000","last_became_peered":"2024-09-16T09:44:24.277124+0000","last_unstale":"2024-09-16T09:45:44.127927+0000","last_undegraded":"2024-09-16T09:45:44.127927+0000","last_fullsized":"2024-09-16T09:45:44.127927+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:54.304553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:44:54.304553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:45:54.689045+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.11","version":"73'79","reported_seq":968,"reported_epoch":571,"state":"active+clean","last_fresh":"2024-09-16T09:46:04.854397+0000","last_change":"2024-09-16T09:45:23.092029+0000","last_active":"2024-09-16T09:46:04.854397+0000","last_peered":"2024-09-16T09:46:04.854397+0000","last_clean":"2024-09-16T09:46:04.854397+0000","last_became_active":"2024-09-16T09:31:09.272208+0000","last_became_peered":"2024-09-16T09:31:09.272208+0000","last_unstale":"2024-09-16T09:46:04.854397+0000","last_undegraded":"2024-09-16T09:46:04.854397+0000","last_fullsized":"2024-09-16T09:46:04.854397+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:45:23.091979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:45:23.091979+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:27.637013+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.14","version":"547'30","reported_seq":770,"reported_epoch":570,"state":"active+clean","last_fresh":"2024-09-16T09:46:03.604508+0000","last_change":"2024-09-16T09:45:23.692659+0000","last_active":"2024-09-16T09:46:03.604508+0000","last_peered":"2024-09-16T09:46:03.604508+0000","last_clean":"2024-09-16T09:46:03.604508+0000","last_became_active":"2024-09-16T09:45:23.692293+0000","last_became_peered":"2024-09-16T09:45:23.692293+0000","last_unstale":"2024-09-16T09:46:03.604508+0000","last_undegraded":"2024-09-16T09:46:03.604508+0000","last_fullsized":"2024-09-16T09:46:03.604508+0000","mapping_epoch":554,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":555,"parent":"0.0","parent_split_bits":0,"last_scrub":"547'30","last_scrub_stamp":"2024-09-16T09:45:18.568491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:45:18.568491+0000","objects_scrubbed":5,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:23.552768+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":43,"num_read_kb":38,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":1093,"reported_epoch":572,"state":"active+clean","last_fresh":"2024-09-16T09:46:05.751344+0000","last_change":"2024-09-16T09:45:41.458692+0000","last_active":"2024-09-16T09:46:05.751344+0000","last_peered":"2024-09-16T09:46:05.751344+0000","last_clean":"2024-09-16T09:46:05.751344+0000","last_became_active":"2024-09-16T09:44:24.279273+0000","last_became_peered":"2024-09-16T09:44:24.279273+0000","last_unstale":"2024-09-16T09:46:05.751344+0000","last_undegraded":"2024-09-16T09:46:05.751344+0000","last_fullsized":"2024-09-16T09:46:05.751344+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:41.458618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:45:41.458618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:56.845197+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"66'8","reported_seq":1353,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:56.995128+0000","last_change":"2024-09-16T09:45:45.786470+0000","last_active":"2024-09-16T09:45:56.995128+0000","last_peered":"2024-09-16T09:45:56.995128+0000","last_clean":"2024-09-16T09:45:56.995128+0000","last_became_active":"2024-09-16T09:25:05.713833+0000","last_became_peered":"2024-09-16T09:25:05.713833+0000","last_unstale":"2024-09-16T09:45:56.995128+0000","last_undegraded":"2024-09-16T09:45:56.995128+0000","last_fullsized":"2024-09-16T09:45:56.995128+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"66'8","last_scrub_stamp":"2024-09-16T09:45:45.786379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:45:45.786379+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:47:15.609811+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1107,"reported_epoch":571,"state":"active+clean","last_fresh":"2024-09-16T09:46:04.854811+0000","last_change":"2024-09-16T09:45:37.007089+0000","last_active":"2024-09-16T09:46:04.854811+0000","last_peered":"2024-09-16T09:46:04.854811+0000","last_clean":"2024-09-16T09:46:04.854811+0000","last_became_active":"2024-09-16T09:31:09.275168+0000","last_became_peered":"2024-09-16T09:31:09.275168+0000","last_unstale":"2024-09-16T09:46:04.854811+0000","last_undegraded":"2024-09-16T09:46:04.854811+0000","last_fullsized":"2024-09-16T09:46:04.854811+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:37.007038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:45:37.007038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:55.131269+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.16","version":"73'79","reported_seq":135,"reported_epoch":570,"state":"active+clean","last_fresh":"2024-09-16T09:46:03.530543+0000","last_change":"2024-09-16T09:45:21.388018+0000","last_active":"2024-09-16T09:46:03.530543+0000","last_peered":"2024-09-16T09:46:03.530543+0000","last_clean":"2024-09-16T09:46:03.530543+0000","last_became_active":"2024-09-16T09:45:19.679357+0000","last_became_peered":"2024-09-16T09:45:19.679357+0000","last_unstale":"2024-09-16T09:46:03.530543+0000","last_undegraded":"2024-09-16T09:46:03.530543+0000","last_fullsized":"2024-09-16T09:46:03.530543+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":542,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:45:21.387959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:45:21.387959+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:41.802550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"547'27","reported_seq":1128,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:44.127395+0000","last_change":"2024-09-16T09:45:31.182610+0000","last_active":"2024-09-16T09:45:44.127395+0000","last_peered":"2024-09-16T09:45:44.127395+0000","last_clean":"2024-09-16T09:45:44.127395+0000","last_became_active":"2024-09-16T09:26:33.291265+0000","last_became_peered":"2024-09-16T09:26:33.291265+0000","last_unstale":"2024-09-16T09:45:44.127395+0000","last_undegraded":"2024-09-16T09:45:44.127395+0000","last_fullsized":"2024-09-16T09:45:44.127395+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"547'27","last_scrub_stamp":"2024-09-16T09:45:31.182529+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:45:31.182529+0000","objects_scrubbed":5,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:53.516271+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":37,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":1101,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:57.119998+0000","last_change":"2024-09-16T09:45:57.119998+0000","last_active":"2024-09-16T09:45:57.119998+0000","last_peered":"2024-09-16T09:45:57.119998+0000","last_clean":"2024-09-16T09:45:57.119998+0000","last_became_active":"2024-09-16T09:25:01.705158+0000","last_became_peered":"2024-09-16T09:25:01.705158+0000","last_unstale":"2024-09-16T09:45:57.119998+0000","last_undegraded":"2024-09-16T09:45:57.119998+0000","last_fullsized":"2024-09-16T09:45:57.119998+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:57.119938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:45:57.119938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:47:09.670712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":639,"reported_epoch":570,"state":"active+clean","last_fresh":"2024-09-16T09:46:03.529588+0000","last_change":"2024-09-16T09:45:19.684238+0000","last_active":"2024-09-16T09:46:03.529588+0000","last_peered":"2024-09-16T09:46:03.529588+0000","last_clean":"2024-09-16T09:46:03.529588+0000","last_became_active":"2024-09-16T09:45:19.683894+0000","last_became_peered":"2024-09-16T09:45:19.683894+0000","last_unstale":"2024-09-16T09:46:03.529588+0000","last_undegraded":"2024-09-16T09:46:03.529588+0000","last_fullsized":"2024-09-16T09:46:03.529588+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:45.660111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:44:45.660111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":161,"reported_epoch":570,"state":"active+clean","last_fresh":"2024-09-16T09:46:03.603570+0000","last_change":"2024-09-16T09:45:23.746052+0000","last_active":"2024-09-16T09:46:03.603570+0000","last_peered":"2024-09-16T09:46:03.603570+0000","last_clean":"2024-09-16T09:46:03.603570+0000","last_became_active":"2024-09-16T09:45:23.745655+0000","last_became_peered":"2024-09-16T09:45:23.745655+0000","last_unstale":"2024-09-16T09:46:03.603570+0000","last_undegraded":"2024-09-16T09:46:03.603570+0000","last_fullsized":"2024-09-16T09:46:03.603570+0000","mapping_epoch":554,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":555,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:47.085465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:44:47.085465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.17","version":"73'79","reported_seq":67,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:43.756372+0000","last_change":"2024-09-16T09:45:05.837415+0000","last_active":"2024-09-16T09:45:43.756372+0000","last_peered":"2024-09-16T09:45:43.756372+0000","last_clean":"2024-09-16T09:45:43.756372+0000","last_became_active":"2024-09-16T09:45:05.497470+0000","last_became_peered":"2024-09-16T09:45:05.497470+0000","last_unstale":"2024-09-16T09:45:43.756372+0000","last_undegraded":"2024-09-16T09:45:43.756372+0000","last_fullsized":"2024-09-16T09:45:43.756372+0000","mapping_epoch":544,"log_start":"0'0","ondisk_log_start":"0'0","created":542,"last_epoch_clean":545,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:45:05.837366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:45:05.837366+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:25.687601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,6],"acting":[5,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.12","version":"547'21","reported_seq":828,"reported_epoch":570,"state":"active+clean","last_fresh":"2024-09-16T09:46:03.529804+0000","last_change":"2024-09-16T09:45:49.439183+0000","last_active":"2024-09-16T09:46:03.529804+0000","last_peered":"2024-09-16T09:46:03.529804+0000","last_clean":"2024-09-16T09:46:03.529804+0000","last_became_active":"2024-09-16T09:45:19.680729+0000","last_became_peered":"2024-09-16T09:45:19.680729+0000","last_unstale":"2024-09-16T09:46:03.529804+0000","last_undegraded":"2024-09-16T09:46:03.529804+0000","last_fullsized":"2024-09-16T09:46:03.529804+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"547'21","last_scrub_stamp":"2024-09-16T09:45:49.439114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:45:49.439114+0000","objects_scrubbed":4,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:47:03.924857+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":33,"num_read_kb":27,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":1089,"reported_epoch":572,"state":"active+clean","last_fresh":"2024-09-16T09:46:05.751959+0000","last_change":"2024-09-16T09:45:46.527433+0000","last_active":"2024-09-16T09:46:05.751959+0000","last_peered":"2024-09-16T09:46:05.751959+0000","last_clean":"2024-09-16T09:46:05.751959+0000","last_became_active":"2024-09-16T09:25:01.706762+0000","last_became_peered":"2024-09-16T09:25:01.706762+0000","last_unstale":"2024-09-16T09:46:05.751959+0000","last_undegraded":"2024-09-16T09:46:05.751959+0000","last_fullsized":"2024-09-16T09:46:05.751959+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:46.527380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:45:46.527380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:47:03.266221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":207,"reported_epoch":570,"state":"active+clean","last_fresh":"2024-09-16T09:46:03.603991+0000","last_change":"2024-09-16T09:45:23.696559+0000","last_active":"2024-09-16T09:46:03.603991+0000","last_peered":"2024-09-16T09:46:03.603991+0000","last_clean":"2024-09-16T09:46:03.603991+0000","last_became_active":"2024-09-16T09:45:23.696202+0000","last_became_peered":"2024-09-16T09:45:23.696202+0000","last_unstale":"2024-09-16T09:46:03.603991+0000","last_undegraded":"2024-09-16T09:46:03.603991+0000","last_fullsized":"2024-09-16T09:46:03.603991+0000","mapping_epoch":554,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":555,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:13.209048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:45:13.209048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:31.841717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1061,"reported_epoch":571,"state":"active+clean","last_fresh":"2024-09-16T09:46:04.854125+0000","last_change":"2024-09-16T09:45:10.203182+0000","last_active":"2024-09-16T09:46:04.854125+0000","last_peered":"2024-09-16T09:46:04.854125+0000","last_clean":"2024-09-16T09:46:04.854125+0000","last_became_active":"2024-09-16T09:25:07.715301+0000","last_became_peered":"2024-09-16T09:25:07.715301+0000","last_unstale":"2024-09-16T09:46:04.854125+0000","last_undegraded":"2024-09-16T09:46:04.854125+0000","last_fullsized":"2024-09-16T09:46:04.854125+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:10.203136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:45:10.203136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:23.474897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.14","version":"73'79","reported_seq":806,"reported_epoch":572,"state":"active+clean","last_fresh":"2024-09-16T09:46:05.749506+0000","last_change":"2024-09-16T09:45:32.165994+0000","last_active":"2024-09-16T09:46:05.749506+0000","last_peered":"2024-09-16T09:46:05.749506+0000","last_clean":"2024-09-16T09:46:05.749506+0000","last_became_active":"2024-09-16T09:44:24.285886+0000","last_became_peered":"2024-09-16T09:44:24.285886+0000","last_unstale":"2024-09-16T09:46:05.749506+0000","last_undegraded":"2024-09-16T09:46:05.749506+0000","last_fullsized":"2024-09-16T09:46:05.749506+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:45:32.165932+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:45:32.165932+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:47:01.695497+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"547'27","reported_seq":972,"reported_epoch":570,"state":"active+clean","last_fresh":"2024-09-16T09:46:03.603138+0000","last_change":"2024-09-16T09:45:31.075789+0000","last_active":"2024-09-16T09:46:03.603138+0000","last_peered":"2024-09-16T09:46:03.603138+0000","last_clean":"2024-09-16T09:46:03.603138+0000","last_became_active":"2024-09-16T09:45:23.692856+0000","last_became_peered":"2024-09-16T09:45:23.692856+0000","last_unstale":"2024-09-16T09:46:03.603138+0000","last_undegraded":"2024-09-16T09:46:03.603138+0000","last_fullsized":"2024-09-16T09:46:03.603138+0000","mapping_epoch":554,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":555,"parent":"0.0","parent_split_bits":0,"last_scrub":"547'27","last_scrub_stamp":"2024-09-16T09:45:31.075736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:45:31.075736+0000","objects_scrubbed":5,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:55.061722+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":1092,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:43.756670+0000","last_change":"2024-09-16T09:45:35.673157+0000","last_active":"2024-09-16T09:45:43.756670+0000","last_peered":"2024-09-16T09:45:43.756670+0000","last_clean":"2024-09-16T09:45:43.756670+0000","last_became_active":"2024-09-16T09:25:01.700733+0000","last_became_peered":"2024-09-16T09:25:01.700733+0000","last_unstale":"2024-09-16T09:45:43.756670+0000","last_undegraded":"2024-09-16T09:45:43.756670+0000","last_fullsized":"2024-09-16T09:45:43.756670+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:35.673100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:45:35.673100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:41.360062+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1082,"reported_epoch":572,"state":"active+clean","last_fresh":"2024-09-16T09:46:05.749251+0000","last_change":"2024-09-16T09:45:39.407824+0000","last_active":"2024-09-16T09:46:05.749251+0000","last_peered":"2024-09-16T09:46:05.749251+0000","last_clean":"2024-09-16T09:46:05.749251+0000","last_became_active":"2024-09-16T09:25:05.713479+0000","last_became_peered":"2024-09-16T09:25:05.713479+0000","last_unstale":"2024-09-16T09:46:05.749251+0000","last_undegraded":"2024-09-16T09:46:05.749251+0000","last_fullsized":"2024-09-16T09:46:05.749251+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:39.407767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:45:39.407767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:59.901053+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":169,"reported_epoch":570,"state":"active+clean","last_fresh":"2024-09-16T09:46:03.603685+0000","last_change":"2024-09-16T09:45:23.696277+0000","last_active":"2024-09-16T09:46:03.603685+0000","last_peered":"2024-09-16T09:46:03.603685+0000","last_clean":"2024-09-16T09:46:03.603685+0000","last_became_active":"2024-09-16T09:45:23.695841+0000","last_became_peered":"2024-09-16T09:45:23.695841+0000","last_unstale":"2024-09-16T09:46:03.603685+0000","last_undegraded":"2024-09-16T09:46:03.603685+0000","last_fullsized":"2024-09-16T09:46:03.603685+0000","mapping_epoch":554,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":555,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:55.146270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:44:55.146270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:19.024903+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.15","version":"73'79","reported_seq":74,"reported_epoch":572,"state":"active+clean","last_fresh":"2024-09-16T09:46:05.751503+0000","last_change":"2024-09-16T09:45:06.638316+0000","last_active":"2024-09-16T09:46:05.751503+0000","last_peered":"2024-09-16T09:46:05.751503+0000","last_clean":"2024-09-16T09:46:05.751503+0000","last_became_active":"2024-09-16T09:45:05.497752+0000","last_became_peered":"2024-09-16T09:45:05.497752+0000","last_unstale":"2024-09-16T09:46:05.751503+0000","last_undegraded":"2024-09-16T09:46:05.751503+0000","last_fullsized":"2024-09-16T09:46:05.751503+0000","mapping_epoch":544,"log_start":"0'0","ondisk_log_start":"0'0","created":542,"last_epoch_clean":545,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:45:06.638254+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:45:06.638254+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:14.918970+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1],"acting":[7,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"547'12","reported_seq":857,"reported_epoch":570,"state":"active+clean","last_fresh":"2024-09-16T09:46:03.602779+0000","last_change":"2024-09-16T09:44:47.186136+0000","last_active":"2024-09-16T09:46:03.602779+0000","last_peered":"2024-09-16T09:46:03.602779+0000","last_clean":"2024-09-16T09:46:03.602779+0000","last_became_active":"2024-09-16T09:31:09.273682+0000","last_became_peered":"2024-09-16T09:31:09.273682+0000","last_unstale":"2024-09-16T09:46:03.602779+0000","last_undegraded":"2024-09-16T09:46:03.602779+0000","last_fullsized":"2024-09-16T09:46:03.602779+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"285'8","last_scrub_stamp":"2024-09-16T09:44:47.186078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:44:47.186078+0000","objects_scrubbed":2,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:11.853660+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":17,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":1119,"reported_epoch":571,"state":"active+clean","last_fresh":"2024-09-16T09:46:04.854916+0000","last_change":"2024-09-16T09:45:53.013918+0000","last_active":"2024-09-16T09:46:04.854916+0000","last_peered":"2024-09-16T09:46:04.854916+0000","last_clean":"2024-09-16T09:46:04.854916+0000","last_became_active":"2024-09-16T09:31:09.274719+0000","last_became_peered":"2024-09-16T09:31:09.274719+0000","last_unstale":"2024-09-16T09:46:04.854916+0000","last_undegraded":"2024-09-16T09:46:04.854916+0000","last_fullsized":"2024-09-16T09:46:04.854916+0000","mapping_epoch":186,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":187,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:53.013871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:45:53.013871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:47:05.328472+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1095,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:57.619232+0000","last_change":"2024-09-16T09:45:57.619232+0000","last_active":"2024-09-16T09:45:57.619232+0000","last_peered":"2024-09-16T09:45:57.619232+0000","last_clean":"2024-09-16T09:45:57.619232+0000","last_became_active":"2024-09-16T09:25:05.712545+0000","last_became_peered":"2024-09-16T09:25:05.712545+0000","last_unstale":"2024-09-16T09:45:57.619232+0000","last_undegraded":"2024-09-16T09:45:57.619232+0000","last_fullsized":"2024-09-16T09:45:57.619232+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:57.619166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:45:57.619166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:47:26.303428+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1088,"reported_epoch":572,"state":"active+clean","last_fresh":"2024-09-16T09:46:05.749528+0000","last_change":"2024-09-16T09:44:51.632486+0000","last_active":"2024-09-16T09:46:05.749528+0000","last_peered":"2024-09-16T09:46:05.749528+0000","last_clean":"2024-09-16T09:46:05.749528+0000","last_became_active":"2024-09-16T09:44:24.276311+0000","last_became_peered":"2024-09-16T09:44:24.276311+0000","last_unstale":"2024-09-16T09:46:05.749528+0000","last_undegraded":"2024-09-16T09:46:05.749528+0000","last_fullsized":"2024-09-16T09:46:05.749528+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:51.632429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:44:51.632429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:19.007714+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1a","version":"73'79","reported_seq":68,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:44.127149+0000","last_change":"2024-09-16T09:45:07.199263+0000","last_active":"2024-09-16T09:45:44.127149+0000","last_peered":"2024-09-16T09:45:44.127149+0000","last_clean":"2024-09-16T09:45:44.127149+0000","last_became_active":"2024-09-16T09:45:05.497673+0000","last_became_peered":"2024-09-16T09:45:05.497673+0000","last_unstale":"2024-09-16T09:45:44.127149+0000","last_undegraded":"2024-09-16T09:45:44.127149+0000","last_fullsized":"2024-09-16T09:45:44.127149+0000","mapping_epoch":544,"log_start":"0'0","ondisk_log_start":"0'0","created":542,"last_epoch_clean":545,"parent":"0.0","parent_split_bits":5,"last_scrub":"73'79","last_scrub_stamp":"2024-09-16T09:45:07.199201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:22:27.651329+0000","last_clean_scrub_stamp":"2024-09-16T09:45:07.199201+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:26.715375+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,1],"acting":[4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":420,"reported_epoch":570,"state":"active+clean","last_fresh":"2024-09-16T09:46:03.530369+0000","last_change":"2024-09-16T09:45:19.684732+0000","last_active":"2024-09-16T09:46:03.530369+0000","last_peered":"2024-09-16T09:46:03.530369+0000","last_clean":"2024-09-16T09:46:03.530369+0000","last_became_active":"2024-09-16T09:45:19.684396+0000","last_became_peered":"2024-09-16T09:45:19.684396+0000","last_unstale":"2024-09-16T09:46:03.530369+0000","last_undegraded":"2024-09-16T09:46:03.530369+0000","last_fullsized":"2024-09-16T09:46:03.530369+0000","mapping_epoch":551,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":552,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:50.722209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:06.698714+0000","last_clean_scrub_stamp":"2024-09-16T09:44:50.722209+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":168,"reported_epoch":570,"state":"active+clean","last_fresh":"2024-09-16T09:46:03.604399+0000","last_change":"2024-09-16T09:45:28.095716+0000","last_active":"2024-09-16T09:46:03.604399+0000","last_peered":"2024-09-16T09:46:03.604399+0000","last_clean":"2024-09-16T09:46:03.604399+0000","last_became_active":"2024-09-16T09:45:23.693577+0000","last_became_peered":"2024-09-16T09:45:23.693577+0000","last_unstale":"2024-09-16T09:46:03.604399+0000","last_undegraded":"2024-09-16T09:46:03.604399+0000","last_fullsized":"2024-09-16T09:46:03.604399+0000","mapping_epoch":554,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":555,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:45:28.095668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:00.689058+0000","last_clean_scrub_stamp":"2024-09-16T09:45:28.095668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:46:45.230102+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"547'27","reported_seq":1136,"reported_epoch":569,"state":"active+clean","last_fresh":"2024-09-16T09:45:44.399400+0000","last_change":"2024-09-16T09:45:36.682430+0000","last_active":"2024-09-16T09:45:44.399400+0000","last_peered":"2024-09-16T09:45:44.399400+0000","last_clean":"2024-09-16T09:45:44.399400+0000","last_became_active":"2024-09-16T09:25:03.707769+0000","last_became_peered":"2024-09-16T09:25:03.707769+0000","last_unstale":"2024-09-16T09:45:44.399400+0000","last_undegraded":"2024-09-16T09:45:44.399400+0000","last_fullsized":"2024-09-16T09:45:44.399400+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"547'27","last_scrub_stamp":"2024-09-16T09:45:36.682363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:02.691904+0000","last_clean_scrub_stamp":"2024-09-16T09:45:36.682363+0000","objects_scrubbed":5,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T09:47:04.008614+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":37,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1086,"reported_epoch":571,"state":"active+clean","last_fresh":"2024-09-16T09:46:04.854886+0000","last_change":"2024-09-16T09:44:36.073614+0000","last_active":"2024-09-16T09:46:04.854886+0000","last_peered":"2024-09-16T09:46:04.854886+0000","last_clean":"2024-09-16T09:46:04.854886+0000","last_became_active":"2024-09-16T09:44:24.279906+0000","last_became_peered":"2024-09-16T09:44:24.279906+0000","last_unstale":"2024-09-16T09:46:04.854886+0000","last_undegraded":"2024-09-16T09:46:04.854886+0000","last_fullsized":"2024-09-16T09:46:04.854886+0000","mapping_epoch":516,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":517,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T09:44:36.073560+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T09:25:04.695480+0000","last_clean_scrub_stamp":"2024-09-16T09:44:36.073560+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80,"ondisk_log_size":80,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":1537,"num_read_kb":1294,"num_write":989,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":144,"num_bytes_recovered":3686,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":969,"ondisk_log_size":969,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1986,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1258,"num_read_kb":1253,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":31,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2449,"ondisk_log_size":2449,"up":93,"acting":93,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":53,"seq":227633266947,"num_pgs":68,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":849072,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":847486,"kb_avail":92896080,"statfs":{"total":95995035648,"available":95125585920,"internally_reserved":0,"allocated":1597440,"data_stored":1081034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":867826122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495879,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":718564,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":716990,"kb_avail":93026588,"statfs":{"total":95995035648,"available":95259226112,"internally_reserved":0,"allocated":1593344,"data_stored":1120904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":734198214},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724814,"num_pgs":62,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":858744,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":857214,"kb_avail":92886408,"statfs":{"total":95995035648,"available":95115681792,"internally_reserved":0,"allocated":1548288,"data_stored":1056601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":877787590},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953747,"num_pgs":68,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":740480,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":739198,"kb_avail":93004672,"statfs":{"total":95995035648,"available":95236784128,"internally_reserved":0,"allocated":1294336,"data_stored":794386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":756939206},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149977,"num_pgs":68,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":917140,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":915518,"kb_avail":92828012,"statfs":{"total":95995035648,"available":95055884288,"internally_reserved":0,"allocated":1638400,"data_stored":1130230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":937490887},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":516,"seq":2216203124759,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":642692,"kb_used_data":556788,"kb_used_omap":4,"kb_used_meta":85883,"kb_avail":93102460,"statfs":{"total":95995035648,"available":95336919040,"internally_reserved":0,"allocated":570150912,"data_stored":569669874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4445,"internal_metadata":87944867},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575141,"num_pgs":64,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1025116,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":1023806,"kb_avail":92720036,"statfs":{"total":95995035648,"available":94945316864,"internally_reserved":0,"allocated":1323008,"data_stored":794366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":1048377797},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738667,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":867148,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":865534,"kb_avail":92878004,"statfs":{"total":95995035648,"available":95107076096,"internally_reserved":0,"allocated":1630208,"data_stored":1130146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":886307269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":1788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T09:46:09.989 INFO:tasks.cephadm.ceph_manager.ceph:recovered! 2024-09-16T09:46:09.990 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705 shell --fsid a1321834-740c-11ef-bceb-c7b262605968 -- ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2024-09-16T09:46:10.160 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:46:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=cleanup t=2024-09-16T09:46:09.846580368Z level=info msg="Completed cleanup jobs" duration=11.022203ms 2024-09-16T09:46:10.160 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:46:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=plugins.update.checker t=2024-09-16T09:46:09.973574167Z level=info msg="Update check succeeded" duration=60.156596ms 2024-09-16T09:46:10.160 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:46:10 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=grafana.update.checker t=2024-09-16T09:46:10.072833218Z level=info msg="Update check succeeded" duration=151.574907ms 2024-09-16T09:46:10.255 INFO:teuthology.orchestra.run.smithi070.stderr:Inferring config /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T09:46:10.671 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:46:10 smithi070 ceph-mon[30388]: from='client.26837 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:46:11.574 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:46:10 smithi070 ceph-mon[30388]: 6.1d scrub starts 2024-09-16T09:46:11.574 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:46:10 smithi070 ceph-mon[30388]: 6.1d scrub ok 2024-09-16T09:46:11.574 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:46:10 smithi070 ceph-mon[30388]: 1.1b scrub starts 2024-09-16T09:46:11.574 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:46:10 smithi070 ceph-mon[30388]: 1.1b scrub ok 2024-09-16T09:46:11.574 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:46:10 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:46:11.575 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:46:10 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:46:11.575 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:46:10 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:46:11.575 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:46:10 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:46:11.575 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:46:10 smithi070 ceph-mon[40691]: from='client.26837 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:46:11.576 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:46:10 smithi070 ceph-mon[40691]: 6.1d scrub starts 2024-09-16T09:46:11.576 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:46:10 smithi070 ceph-mon[40691]: 6.1d scrub ok 2024-09-16T09:46:11.576 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:46:10 smithi070 ceph-mon[40691]: 1.1b scrub starts 2024-09-16T09:46:11.576 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:46:10 smithi070 ceph-mon[40691]: 1.1b scrub ok 2024-09-16T09:46:11.576 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:46:10 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:46:11.576 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:46:10 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:46:11.576 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:46:10 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:46:11.576 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:46:10 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:46:11.576 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:46:11 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: [16/Sep/2024:09:46:11] ENGINE Bus STOPPING 2024-09-16T09:46:11.577 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:46:10 smithi084 ceph-mon[32629]: from='client.26837 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T09:46:11.577 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:46:10 smithi084 ceph-mon[32629]: 6.1d scrub starts 2024-09-16T09:46:11.577 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:46:10 smithi084 ceph-mon[32629]: 6.1d scrub ok 2024-09-16T09:46:11.577 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:46:10 smithi084 ceph-mon[32629]: 1.1b scrub starts 2024-09-16T09:46:11.577 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:46:10 smithi084 ceph-mon[32629]: 1.1b scrub ok 2024-09-16T09:46:11.578 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:46:10 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:46:11.578 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:46:10 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:46:11.578 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:46:10 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:46:11.578 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:46:10 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:46:11.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:46:11 smithi070 ceph-mon[40691]: 6.6 scrub starts 2024-09-16T09:46:11.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:46:11 smithi070 ceph-mon[40691]: 6.6 scrub ok 2024-09-16T09:46:11.671 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:46:11 smithi070 ceph-mon[40691]: pgmap v1122: 162 pgs: 162 active+clean; 583 KiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:46:11.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:46:11 smithi070 ceph-mon[40691]: 6.e scrub starts 2024-09-16T09:46:11.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:46:11 smithi070 ceph-mon[40691]: 6.e scrub ok 2024-09-16T09:46:11.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:46:11 smithi070 ceph-mon[40691]: 5.c scrub starts 2024-09-16T09:46:11.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:46:11 smithi070 ceph-mon[40691]: 5.c scrub ok 2024-09-16T09:46:11.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:46:11 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/2943075597' entity='client.admin' cmd={"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"} : dispatch 2024-09-16T09:46:11.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:46:11 smithi070 ceph-mon[40691]: from='client.? 172.21.15.70:0/2943075597' entity='client.admin' cmd='[{"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"}]': finished 2024-09-16T09:46:11.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:46:11 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:46:11.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:46:11 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:46:11.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:46:11 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:46:11.672 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:46:11 smithi070 ceph-mon[40691]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:46:11.673 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:46:11 smithi070 ceph-mon[40691]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-16T09:46:11.673 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:46:11 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: [16/Sep/2024:09:46:11] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-16T09:46:11.673 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:46:11 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: [16/Sep/2024:09:46:11] ENGINE Bus STOPPED 2024-09-16T09:46:11.673 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:46:11 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: [16/Sep/2024:09:46:11] ENGINE Bus STARTING 2024-09-16T09:46:11.673 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:46:11 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: [16/Sep/2024:09:46:11] ENGINE Serving on http://:::9283 2024-09-16T09:46:11.674 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:46:11 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: [16/Sep/2024:09:46:11] ENGINE Bus STARTED 2024-09-16T09:46:11.674 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:46:11 smithi070 ceph-mon[30388]: 6.6 scrub starts 2024-09-16T09:46:11.674 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:46:11 smithi070 ceph-mon[30388]: 6.6 scrub ok 2024-09-16T09:46:11.674 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:46:11 smithi070 ceph-mon[30388]: pgmap v1122: 162 pgs: 162 active+clean; 583 KiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:46:11.674 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:46:11 smithi070 ceph-mon[30388]: 6.e scrub starts 2024-09-16T09:46:11.674 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:46:11 smithi070 ceph-mon[30388]: 6.e scrub ok 2024-09-16T09:46:11.674 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:46:11 smithi070 ceph-mon[30388]: 5.c scrub starts 2024-09-16T09:46:11.674 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:46:11 smithi070 ceph-mon[30388]: 5.c scrub ok 2024-09-16T09:46:11.675 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:46:11 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2943075597' entity='client.admin' cmd={"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"} : dispatch 2024-09-16T09:46:11.675 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:46:11 smithi070 ceph-mon[30388]: from='client.? 172.21.15.70:0/2943075597' entity='client.admin' cmd='[{"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"}]': finished 2024-09-16T09:46:11.675 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:46:11 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:46:11.675 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:46:11 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:46:11.675 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:46:11 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:46:11.675 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:46:11 smithi070 ceph-mon[30388]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:46:11.675 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:46:11 smithi070 ceph-mon[30388]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-16T09:46:11.754 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-16T09:46:12.034 INFO:tasks.cephadm:Teardown begin 2024-09-16T09:46:12.035 DEBUG:teuthology.orchestra.run.smithi070:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-16T09:46:12.035 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:46:11 smithi084 ceph-mon[32629]: 6.6 scrub starts 2024-09-16T09:46:12.036 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:46:11 smithi084 ceph-mon[32629]: 6.6 scrub ok 2024-09-16T09:46:12.036 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:46:11 smithi084 ceph-mon[32629]: pgmap v1122: 162 pgs: 162 active+clean; 583 KiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-16T09:46:12.036 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:46:11 smithi084 ceph-mon[32629]: 6.e scrub starts 2024-09-16T09:46:12.036 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:46:11 smithi084 ceph-mon[32629]: 6.e scrub ok 2024-09-16T09:46:12.036 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:46:11 smithi084 ceph-mon[32629]: 5.c scrub starts 2024-09-16T09:46:12.036 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:46:11 smithi084 ceph-mon[32629]: 5.c scrub ok 2024-09-16T09:46:12.036 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:46:11 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/2943075597' entity='client.admin' cmd={"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"} : dispatch 2024-09-16T09:46:12.036 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:46:11 smithi084 ceph-mon[32629]: from='client.? 172.21.15.70:0/2943075597' entity='client.admin' cmd='[{"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"}]': finished 2024-09-16T09:46:12.036 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:46:11 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-16T09:46:12.036 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:46:11 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-16T09:46:12.036 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:46:11 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-16T09:46:12.037 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:46:11 smithi084 ceph-mon[32629]: from='mgr.14604 ' entity='mgr.y' 2024-09-16T09:46:12.037 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:46:11 smithi084 ceph-mon[32629]: from='mgr.14604 172.21.15.70:0/3171783133' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-16T09:46:12.063 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:46:11 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: ::ffff:172.21.15.84 - - [16/Sep/2024:09:46:11] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-16T09:46:12.082 DEBUG:teuthology.orchestra.run.smithi084:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-16T09:46:12.134 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-16T09:46:12.135 DEBUG:teuthology.orchestra.run.smithi070:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-16T09:46:12.167 DEBUG:teuthology.orchestra.run.smithi084:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-16T09:46:12.189 INFO:tasks.cephadm:Stopping all daemons... 2024-09-16T09:46:12.189 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-16T09:46:12.189 DEBUG:teuthology.orchestra.run.smithi070:> sudo systemctl stop ceph-a1321834-740c-11ef-bceb-c7b262605968@mon.a 2024-09-16T09:46:12.420 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:46:12 smithi070 systemd[1]: Stopping Ceph mon.a for a1321834-740c-11ef-bceb-c7b262605968... 2024-09-16T09:46:12.803 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:46:12 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mon-a[30365]: 2024-09-16T09:46:12.536+0000 7f68afc3a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T09:46:12.803 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:46:12 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mon-a[30365]: 2024-09-16T09:46:12.536+0000 7f68afc3a640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-16T09:46:13.079 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:46:12 smithi070 podman[224319]: 2024-09-16 09:46:12.802501577 +0000 UTC m=+0.396109318 container died 9e96378a12f59f065acba2ed441430f12044d8f6b219d5b9300ceedd0570921e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-mon-a, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T09:46:13.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:46:13 smithi070 podman[224319]: 2024-09-16 09:46:13.2439079 +0000 UTC m=+0.837515638 container cleanup 9e96378a12f59f065acba2ed441430f12044d8f6b219d5b9300ceedd0570921e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, io.buildah.version=1.37.2) 2024-09-16T09:46:13.421 INFO:journalctl@ceph.mon.a.smithi070.stdout:Sep 16 09:46:13 smithi070 bash[224319]: ceph-a1321834-740c-11ef-bceb-c7b262605968-mon-a 2024-09-16T09:46:13.669 DEBUG:teuthology.orchestra.run.smithi070:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a1321834-740c-11ef-bceb-c7b262605968@mon.a.service' 2024-09-16T09:46:13.707 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T09:46:13.707 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-16T09:46:13.707 INFO:tasks.cephadm.mon.b:Stopping mon.c... 2024-09-16T09:46:13.707 DEBUG:teuthology.orchestra.run.smithi070:> sudo systemctl stop ceph-a1321834-740c-11ef-bceb-c7b262605968@mon.c 2024-09-16T09:46:14.046 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:46:13 smithi070 systemd[1]: Stopping Ceph mon.c for a1321834-740c-11ef-bceb-c7b262605968... 2024-09-16T09:46:14.046 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:46:14 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mon-c[40668]: 2024-09-16T09:46:14.033+0000 7f3eaf265640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T09:46:14.046 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:46:14 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mon-c[40668]: 2024-09-16T09:46:14.033+0000 7f3eaf265640 -1 mon.c@2(peon) e3 *** Got Signal Terminated *** 2024-09-16T09:46:14.160 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:46:13 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug there is no tcmu-runner data available 2024-09-16T09:46:14.421 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:46:14 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: [16/Sep/2024:09:46:14] ENGINE Bus STOPPING 2024-09-16T09:46:14.421 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:46:14 smithi070 podman[224450]: 2024-09-16 09:46:14.269000609 +0000 UTC m=+0.338367443 container died a5f3710eb38e9621e1b045feda0b4e0d907adddffa69804b05c48e9b8a4f6075 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-mon-c, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=wip-guits-main-2024-09-13-1248-8293d73) 2024-09-16T09:46:14.759 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:46:14 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: [16/Sep/2024:09:46:14] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-16T09:46:14.759 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:46:14 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: [16/Sep/2024:09:46:14] ENGINE Bus STOPPED 2024-09-16T09:46:14.759 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:46:14 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: [16/Sep/2024:09:46:14] ENGINE Bus STARTING 2024-09-16T09:46:14.759 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:46:14 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: [16/Sep/2024:09:46:14] ENGINE Serving on http://:::9283 2024-09-16T09:46:14.759 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:46:14 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: [16/Sep/2024:09:46:14] ENGINE Bus STARTED 2024-09-16T09:46:15.166 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:46:14 smithi070 podman[224450]: 2024-09-16 09:46:14.757932935 +0000 UTC m=+0.827299827 container cleanup a5f3710eb38e9621e1b045feda0b4e0d907adddffa69804b05c48e9b8a4f6075 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-mon-c, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True) 2024-09-16T09:46:15.166 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:46:14 smithi070 bash[224450]: ceph-a1321834-740c-11ef-bceb-c7b262605968-mon-c 2024-09-16T09:46:15.166 INFO:journalctl@ceph.mon.c.smithi070.stdout:Sep 16 09:46:14 smithi070 podman[224467]: 2024-09-16 09:46:14.968273729 +0000 UTC m=+0.700390888 container remove a5f3710eb38e9621e1b045feda0b4e0d907adddffa69804b05c48e9b8a4f6075 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-mon-c, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2) 2024-09-16T09:46:15.182 DEBUG:teuthology.orchestra.run.smithi070:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a1321834-740c-11ef-bceb-c7b262605968@mon.c.service' 2024-09-16T09:46:15.216 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T09:46:15.216 INFO:tasks.cephadm.mon.b:Stopped mon.c 2024-09-16T09:46:15.216 INFO:tasks.cephadm.mon.b:Stopping mon.b... 2024-09-16T09:46:15.216 DEBUG:teuthology.orchestra.run.smithi084:> sudo systemctl stop ceph-a1321834-740c-11ef-bceb-c7b262605968@mon.b 2024-09-16T09:46:15.547 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:46:15 smithi084 systemd[1]: Stopping Ceph mon.b for a1321834-740c-11ef-bceb-c7b262605968... 2024-09-16T09:46:15.910 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:46:15 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-mon-b[32607]: 2024-09-16T09:46:15.544+0000 7f0855aa3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T09:46:15.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:46:15 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-mon-b[32607]: 2024-09-16T09:46:15.544+0000 7f0855aa3640 -1 mon.b@1(peon) e3 *** Got Signal Terminated *** 2024-09-16T09:46:15.911 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:46:15 smithi084 podman[153511]: 2024-09-16 09:46:15.790694955 +0000 UTC m=+0.370731602 container died 5808987e05042ecd2bd0ac3b17c2e85717fe56970b3144b07736437ba2d0a02f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-mon-b, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-16T09:46:16.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:46:16 smithi084 podman[153511]: 2024-09-16 09:46:16.277613978 +0000 UTC m=+0.857650622 container cleanup 5808987e05042ecd2bd0ac3b17c2e85717fe56970b3144b07736437ba2d0a02f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-mon-b, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-16T09:46:16.410 INFO:journalctl@ceph.mon.b.smithi084.stdout:Sep 16 09:46:16 smithi084 bash[153511]: ceph-a1321834-740c-11ef-bceb-c7b262605968-mon-b 2024-09-16T09:46:16.706 DEBUG:teuthology.orchestra.run.smithi084:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a1321834-740c-11ef-bceb-c7b262605968@mon.b.service' 2024-09-16T09:46:16.752 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T09:46:16.753 INFO:tasks.cephadm.mon.b:Stopped mon.b 2024-09-16T09:46:16.753 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-16T09:46:16.753 DEBUG:teuthology.orchestra.run.smithi070:> sudo systemctl stop ceph-a1321834-740c-11ef-bceb-c7b262605968@mgr.y 2024-09-16T09:46:17.057 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:46:16 smithi070 systemd[1]: Stopping Ceph mgr.y for a1321834-740c-11ef-bceb-c7b262605968... 2024-09-16T09:46:17.356 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:46:17 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: 2024-09-16T09:46:17.055+0000 7fca578b7640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.y -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T09:46:17.357 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:46:17 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y[30636]: 2024-09-16T09:46:17.055+0000 7fca578b7640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-09-16T09:46:17.357 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:46:17 smithi070 podman[224593]: 2024-09-16 09:46:17.087612955 +0000 UTC m=+0.145077263 container died f4b4a3b21ded4963b9941bfff0cc662c9ab8f330dfe0b6ac3574c9851783d9c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-16T09:46:17.671 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:46:17 smithi070 podman[224593]: 2024-09-16 09:46:17.50504775 +0000 UTC m=+0.562512057 container cleanup f4b4a3b21ded4963b9941bfff0cc662c9ab8f330dfe0b6ac3574c9851783d9c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T09:46:17.671 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:46:17 smithi070 bash[224593]: ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y 2024-09-16T09:46:17.910 INFO:journalctl@ceph.prometheus.a.smithi084.stdout:Sep 16 09:46:17 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-prometheus-a[68785]: ts=2024-09-16T09:46:17.555Z caller=refresh.go:90 level=error component="discovery manager scrape" discovery=http config=node msg="Unable to refresh target groups" err="Get \"http://172.21.15.70:8765/sd/prometheus/sd-config?service=node-exporter\": dial tcp 172.21.15.70:8765: connect: connection refused" 2024-09-16T09:46:17.911 INFO:journalctl@ceph.prometheus.a.smithi084.stdout:Sep 16 09:46:17 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-prometheus-a[68785]: ts=2024-09-16T09:46:17.559Z caller=refresh.go:90 level=error component="discovery manager scrape" discovery=http config=ceph-exporter msg="Unable to refresh target groups" err="Get \"http://172.21.15.70:8765/sd/prometheus/sd-config?service=ceph-exporter\": dial tcp 172.21.15.70:8765: connect: connection refused" 2024-09-16T09:46:17.911 INFO:journalctl@ceph.prometheus.a.smithi084.stdout:Sep 16 09:46:17 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-prometheus-a[68785]: ts=2024-09-16T09:46:17.559Z caller=refresh.go:90 level=error component="discovery manager notify" discovery=http config=config-0 msg="Unable to refresh target groups" err="Get \"http://172.21.15.70:8765/sd/prometheus/sd-config?service=alertmanager\": dial tcp 172.21.15.70:8765: connect: connection refused" 2024-09-16T09:46:17.911 INFO:journalctl@ceph.prometheus.a.smithi084.stdout:Sep 16 09:46:17 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-prometheus-a[68785]: ts=2024-09-16T09:46:17.559Z caller=refresh.go:90 level=error component="discovery manager scrape" discovery=http config=smb msg="Unable to refresh target groups" err="Get \"http://172.21.15.70:8765/sd/prometheus/sd-config?service=smb\": dial tcp 172.21.15.70:8765: connect: connection refused" 2024-09-16T09:46:17.911 INFO:journalctl@ceph.prometheus.a.smithi084.stdout:Sep 16 09:46:17 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-prometheus-a[68785]: ts=2024-09-16T09:46:17.560Z caller=refresh.go:90 level=error component="discovery manager scrape" discovery=http config=nfs msg="Unable to refresh target groups" err="Get \"http://172.21.15.70:8765/sd/prometheus/sd-config?service=nfs\": dial tcp 172.21.15.70:8765: connect: connection refused" 2024-09-16T09:46:17.911 INFO:journalctl@ceph.prometheus.a.smithi084.stdout:Sep 16 09:46:17 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-prometheus-a[68785]: ts=2024-09-16T09:46:17.560Z caller=refresh.go:90 level=error component="discovery manager scrape" discovery=http config=ceph msg="Unable to refresh target groups" err="Get \"http://172.21.15.70:8765/sd/prometheus/sd-config?service=mgr-prometheus\": dial tcp 172.21.15.70:8765: connect: connection refused" 2024-09-16T09:46:17.911 INFO:journalctl@ceph.prometheus.a.smithi084.stdout:Sep 16 09:46:17 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-prometheus-a[68785]: ts=2024-09-16T09:46:17.562Z caller=refresh.go:90 level=error component="discovery manager scrape" discovery=http config=nvmeof msg="Unable to refresh target groups" err="Get \"http://172.21.15.70:8765/sd/prometheus/sd-config?service=nvmeof\": dial tcp 172.21.15.70:8765: connect: connection refused" 2024-09-16T09:46:18.003 DEBUG:teuthology.orchestra.run.smithi070:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a1321834-740c-11ef-bceb-c7b262605968@mgr.y.service' 2024-09-16T09:46:18.025 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:46:17 smithi070 podman[224611]: 2024-09-16 09:46:17.765509952 +0000 UTC m=+0.675495425 container remove f4b4a3b21ded4963b9941bfff0cc662c9ab8f330dfe0b6ac3574c9851783d9c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8293d73f8690540e843a81caec373f9cc29cf705, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-16T09:46:18.025 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:46:17 smithi070 systemd[1]: ceph-a1321834-740c-11ef-bceb-c7b262605968@mgr.y.service: Deactivated successfully. 2024-09-16T09:46:18.025 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:46:17 smithi070 systemd[1]: Stopped Ceph mgr.y for a1321834-740c-11ef-bceb-c7b262605968. 2024-09-16T09:46:18.025 INFO:journalctl@ceph.mgr.y.smithi070.stdout:Sep 16 09:46:17 smithi070 systemd[1]: ceph-a1321834-740c-11ef-bceb-c7b262605968@mgr.y.service: Consumed 59.099s CPU time. 2024-09-16T09:46:18.039 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T09:46:18.039 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-16T09:46:18.039 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-16T09:46:18.039 DEBUG:teuthology.orchestra.run.smithi084:> sudo systemctl stop ceph-a1321834-740c-11ef-bceb-c7b262605968@mgr.x 2024-09-16T09:46:18.375 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Sep 16 09:46:18 smithi084 systemd[1]: Stopping Ceph mgr.x for a1321834-740c-11ef-bceb-c7b262605968... 2024-09-16T09:46:18.660 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Sep 16 09:46:18 smithi084 podman[153641]: 2024-09-16 09:46:18.374138426 +0000 UTC m=+0.140479622 container died 6cafed972ec4b1f5f3c274d276f91e52cdaff6ea400b74e50d3eda27c031ab51 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-x, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True) 2024-09-16T09:46:19.119 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Sep 16 09:46:18 smithi084 podman[153641]: 2024-09-16 09:46:18.874471973 +0000 UTC m=+0.640813209 container cleanup 6cafed972ec4b1f5f3c274d276f91e52cdaff6ea400b74e50d3eda27c031ab51 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-guits-main-2024-09-13-1248-8293d73) 2024-09-16T09:46:19.119 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Sep 16 09:46:18 smithi084 bash[153641]: ceph-a1321834-740c-11ef-bceb-c7b262605968-mgr-x 2024-09-16T09:46:19.251 DEBUG:teuthology.orchestra.run.smithi084:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a1321834-740c-11ef-bceb-c7b262605968@mgr.x.service' 2024-09-16T09:46:19.285 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T09:46:19.285 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-16T09:46:19.285 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-16T09:46:19.285 DEBUG:teuthology.orchestra.run.smithi070:> sudo systemctl stop ceph-a1321834-740c-11ef-bceb-c7b262605968@osd.0 2024-09-16T09:46:19.619 INFO:journalctl@ceph.osd.0.smithi070.stdout:Sep 16 09:46:19 smithi070 systemd[1]: Stopping Ceph osd.0 for a1321834-740c-11ef-bceb-c7b262605968... 2024-09-16T09:46:19.920 INFO:journalctl@ceph.osd.0.smithi070.stdout:Sep 16 09:46:19 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-0[44330]: 2024-09-16T09:46:19.629+0000 7f46deece640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T09:46:19.921 INFO:journalctl@ceph.osd.0.smithi070.stdout:Sep 16 09:46:19 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-0[44330]: 2024-09-16T09:46:19.629+0000 7f46deece640 -1 osd.0 573 *** Got signal Terminated *** 2024-09-16T09:46:19.921 INFO:journalctl@ceph.osd.0.smithi070.stdout:Sep 16 09:46:19 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-0[44330]: 2024-09-16T09:46:19.629+0000 7f46deece640 -1 osd.0 573 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T09:46:25.421 INFO:journalctl@ceph.osd.0.smithi070.stdout:Sep 16 09:46:25 smithi070 podman[224723]: 2024-09-16 09:46:25.152352642 +0000 UTC m=+5.649972297 container died b2f89f39b840b0ea508e88c32c613ce298b80efa1e485b78c933b2a89ca752b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T09:46:25.921 INFO:journalctl@ceph.osd.0.smithi070.stdout:Sep 16 09:46:25 smithi070 podman[224723]: 2024-09-16 09:46:25.860454364 +0000 UTC m=+6.358074054 container cleanup b2f89f39b840b0ea508e88c32c613ce298b80efa1e485b78c933b2a89ca752b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-0, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-guits-main-2024-09-13-1248-8293d73) 2024-09-16T09:46:25.921 INFO:journalctl@ceph.osd.0.smithi070.stdout:Sep 16 09:46:25 smithi070 bash[224723]: ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-0 2024-09-16T09:46:26.420 INFO:journalctl@ceph.osd.0.smithi070.stdout:Sep 16 09:46:26 smithi070 podman[224755]: 2024-09-16 09:46:26.129253399 +0000 UTC m=+0.977968043 container remove b2f89f39b840b0ea508e88c32c613ce298b80efa1e485b78c933b2a89ca752b8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.license=GPLv2) 2024-09-16T09:46:26.884 INFO:journalctl@ceph.osd.0.smithi070.stdout:Sep 16 09:46:26 smithi070 podman[224847]: 2024-09-16 09:46:26.525486729 +0000 UTC m=+0.015979295 image pull 425e316ba8ebf74f04aa3640f7deb4fa3f293a78d0c8d097cb679eaf86ddca6c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce 2024-09-16T09:46:26.884 INFO:journalctl@ceph.osd.0.smithi070.stdout:Sep 16 09:46:26 smithi070 podman[224847]: 2024-09-16 09:46:26.647178721 +0000 UTC m=+0.137671316 container create c84f98214eb706529c776a5882b56e07e8c6b74fd34e6392a0d04c2c4ad5cf6b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-0-deactivate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T09:46:26.884 INFO:journalctl@ceph.osd.0.smithi070.stdout:Sep 16 09:46:26 smithi070 podman[224847]: 2024-09-16 09:46:26.883160279 +0000 UTC m=+0.373652831 container init c84f98214eb706529c776a5882b56e07e8c6b74fd34e6392a0d04c2c4ad5cf6b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, GIT_BRANCH=HEAD) 2024-09-16T09:46:27.143 INFO:journalctl@ceph.osd.0.smithi070.stdout:Sep 16 09:46:26 smithi070 podman[224847]: 2024-09-16 09:46:26.888324189 +0000 UTC m=+0.378816745 container start c84f98214eb706529c776a5882b56e07e8c6b74fd34e6392a0d04c2c4ad5cf6b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-0-deactivate, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-16T09:46:27.143 INFO:journalctl@ceph.osd.0.smithi070.stdout:Sep 16 09:46:26 smithi070 podman[224847]: 2024-09-16 09:46:26.966408789 +0000 UTC m=+0.456901356 container attach c84f98214eb706529c776a5882b56e07e8c6b74fd34e6392a0d04c2c4ad5cf6b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-guits-main-2024-09-13-1248-8293d73, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T09:46:27.143 INFO:journalctl@ceph.osd.0.smithi070.stdout:Sep 16 09:46:27 smithi070 podman[224847]: 2024-09-16 09:46:27.110057471 +0000 UTC m=+0.600550038 container died c84f98214eb706529c776a5882b56e07e8c6b74fd34e6392a0d04c2c4ad5cf6b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-16T09:46:27.778 DEBUG:teuthology.orchestra.run.smithi070:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a1321834-740c-11ef-bceb-c7b262605968@osd.0.service' 2024-09-16T09:46:27.818 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T09:46:27.818 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-16T09:46:27.818 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-16T09:46:27.819 DEBUG:teuthology.orchestra.run.smithi070:> sudo systemctl stop ceph-a1321834-740c-11ef-bceb-c7b262605968@osd.1 2024-09-16T09:46:28.169 INFO:journalctl@ceph.osd.1.smithi070.stdout:Sep 16 09:46:27 smithi070 systemd[1]: Stopping Ceph osd.1 for a1321834-740c-11ef-bceb-c7b262605968... 2024-09-16T09:46:28.421 INFO:journalctl@ceph.osd.1.smithi070.stdout:Sep 16 09:46:28 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-1[50557]: 2024-09-16T09:46:28.167+0000 7fab6efb9640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T09:46:28.421 INFO:journalctl@ceph.osd.1.smithi070.stdout:Sep 16 09:46:28 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-1[50557]: 2024-09-16T09:46:28.167+0000 7fab6efb9640 -1 osd.1 573 *** Got signal Terminated *** 2024-09-16T09:46:28.421 INFO:journalctl@ceph.osd.1.smithi070.stdout:Sep 16 09:46:28 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-1[50557]: 2024-09-16T09:46:28.167+0000 7fab6efb9640 -1 osd.1 573 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T09:46:34.051 INFO:journalctl@ceph.osd.1.smithi070.stdout:Sep 16 09:46:33 smithi070 podman[224977]: 2024-09-16 09:46:33.778536681 +0000 UTC m=+5.737768325 container died 0177246d589a3fed2ad390707579eb98128e3380889b30b5508a496a2051007d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-1, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-16T09:46:34.420 INFO:journalctl@ceph.osd.1.smithi070.stdout:Sep 16 09:46:34 smithi070 podman[224977]: 2024-09-16 09:46:34.224067149 +0000 UTC m=+6.183298852 container cleanup 0177246d589a3fed2ad390707579eb98128e3380889b30b5508a496a2051007d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-1, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-16T09:46:34.421 INFO:journalctl@ceph.osd.1.smithi070.stdout:Sep 16 09:46:34 smithi070 bash[224977]: ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-1 2024-09-16T09:46:34.837 INFO:journalctl@ceph.osd.1.smithi070.stdout:Sep 16 09:46:34 smithi070 podman[224997]: 2024-09-16 09:46:34.527918025 +0000 UTC m=+0.747721047 container remove 0177246d589a3fed2ad390707579eb98128e3380889b30b5508a496a2051007d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-1, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-16T09:46:35.096 INFO:journalctl@ceph.osd.1.smithi070.stdout:Sep 16 09:46:34 smithi070 podman[225093]: 2024-09-16 09:46:34.738737246 +0000 UTC m=+0.015563815 image pull 425e316ba8ebf74f04aa3640f7deb4fa3f293a78d0c8d097cb679eaf86ddca6c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce 2024-09-16T09:46:35.097 INFO:journalctl@ceph.osd.1.smithi070.stdout:Sep 16 09:46:34 smithi070 podman[225093]: 2024-09-16 09:46:34.859904289 +0000 UTC m=+0.136730918 container create 6918a901b6930e8a9b9d236aca60ac7e352aab6a6c609f06b4725127ff5ef607 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2) 2024-09-16T09:46:35.097 INFO:journalctl@ceph.osd.1.smithi070.stdout:Sep 16 09:46:35 smithi070 podman[225093]: 2024-09-16 09:46:35.09554482 +0000 UTC m=+0.372371391 container init 6918a901b6930e8a9b9d236aca60ac7e352aab6a6c609f06b4725127ff5ef607 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-16T09:46:35.421 INFO:journalctl@ceph.osd.1.smithi070.stdout:Sep 16 09:46:35 smithi070 podman[225093]: 2024-09-16 09:46:35.100651555 +0000 UTC m=+0.377478133 container start 6918a901b6930e8a9b9d236aca60ac7e352aab6a6c609f06b4725127ff5ef607 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-16T09:46:35.421 INFO:journalctl@ceph.osd.1.smithi070.stdout:Sep 16 09:46:35 smithi070 podman[225093]: 2024-09-16 09:46:35.187117951 +0000 UTC m=+0.463944523 container attach 6918a901b6930e8a9b9d236aca60ac7e352aab6a6c609f06b4725127ff5ef607 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731) 2024-09-16T09:46:35.421 INFO:journalctl@ceph.osd.1.smithi070.stdout:Sep 16 09:46:35 smithi070 podman[225093]: 2024-09-16 09:46:35.28565668 +0000 UTC m=+0.562483251 container died 6918a901b6930e8a9b9d236aca60ac7e352aab6a6c609f06b4725127ff5ef607 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-16T09:46:35.929 DEBUG:teuthology.orchestra.run.smithi070:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a1321834-740c-11ef-bceb-c7b262605968@osd.1.service' 2024-09-16T09:46:35.968 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T09:46:35.968 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-16T09:46:35.968 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-16T09:46:35.968 DEBUG:teuthology.orchestra.run.smithi070:> sudo systemctl stop ceph-a1321834-740c-11ef-bceb-c7b262605968@osd.2 2024-09-16T09:46:36.170 INFO:journalctl@ceph.osd.2.smithi070.stdout:Sep 16 09:46:36 smithi070 systemd[1]: Stopping Ceph osd.2 for a1321834-740c-11ef-bceb-c7b262605968... 2024-09-16T09:46:36.671 INFO:journalctl@ceph.osd.2.smithi070.stdout:Sep 16 09:46:36 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-2[213542]: 2024-09-16T09:46:36.286+0000 7f13943f6640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T09:46:36.671 INFO:journalctl@ceph.osd.2.smithi070.stdout:Sep 16 09:46:36 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-2[213542]: 2024-09-16T09:46:36.286+0000 7f13943f6640 -1 osd.2 573 *** Got signal Terminated *** 2024-09-16T09:46:36.671 INFO:journalctl@ceph.osd.2.smithi070.stdout:Sep 16 09:46:36 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-2[213542]: 2024-09-16T09:46:36.286+0000 7f13943f6640 -1 osd.2 573 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T09:46:41.921 INFO:journalctl@ceph.osd.2.smithi070.stdout:Sep 16 09:46:41 smithi070 podman[225218]: 2024-09-16 09:46:41.629770353 +0000 UTC m=+5.442753268 container died ed83dc3d2c50b8160eba470147aa8f3f4e2f10e6d90733c5af157c674117edc6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2) 2024-09-16T09:46:42.346 INFO:journalctl@ceph.osd.2.smithi070.stdout:Sep 16 09:46:42 smithi070 podman[225218]: 2024-09-16 09:46:42.11879257 +0000 UTC m=+5.931775480 container cleanup ed83dc3d2c50b8160eba470147aa8f3f4e2f10e6d90733c5af157c674117edc6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-16T09:46:42.346 INFO:journalctl@ceph.osd.2.smithi070.stdout:Sep 16 09:46:42 smithi070 bash[225218]: ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-2 2024-09-16T09:46:42.659 INFO:journalctl@ceph.osd.2.smithi070.stdout:Sep 16 09:46:42 smithi070 podman[225243]: 2024-09-16 09:46:42.345648832 +0000 UTC m=+0.712741611 container remove ed83dc3d2c50b8160eba470147aa8f3f4e2f10e6d90733c5af157c674117edc6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-2, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T09:46:42.921 INFO:journalctl@ceph.osd.2.smithi070.stdout:Sep 16 09:46:42 smithi070 podman[225332]: 2024-09-16 09:46:42.561189774 +0000 UTC m=+0.015837070 image pull 425e316ba8ebf74f04aa3640f7deb4fa3f293a78d0c8d097cb679eaf86ddca6c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce 2024-09-16T09:46:42.921 INFO:journalctl@ceph.osd.2.smithi070.stdout:Sep 16 09:46:42 smithi070 podman[225332]: 2024-09-16 09:46:42.671469092 +0000 UTC m=+0.126116377 container create f2dcd67ca55e38c22e001fb88f01a59c2e3b6095bac48ab5fec7f6748fd4d353 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-2-deactivate, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-16T09:46:42.921 INFO:journalctl@ceph.osd.2.smithi070.stdout:Sep 16 09:46:42 smithi070 podman[225332]: 2024-09-16 09:46:42.899264662 +0000 UTC m=+0.353911952 container init f2dcd67ca55e38c22e001fb88f01a59c2e3b6095bac48ab5fec7f6748fd4d353 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T09:46:42.921 INFO:journalctl@ceph.osd.2.smithi070.stdout:Sep 16 09:46:42 smithi070 podman[225332]: 2024-09-16 09:46:42.904019755 +0000 UTC m=+0.358667040 container start f2dcd67ca55e38c22e001fb88f01a59c2e3b6095bac48ab5fec7f6748fd4d353 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-2-deactivate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-16T09:46:43.386 INFO:journalctl@ceph.osd.2.smithi070.stdout:Sep 16 09:46:42 smithi070 podman[225332]: 2024-09-16 09:46:42.999025834 +0000 UTC m=+0.453673126 container attach f2dcd67ca55e38c22e001fb88f01a59c2e3b6095bac48ab5fec7f6748fd4d353 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-2-deactivate, io.buildah.version=1.37.2, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T09:46:43.386 INFO:journalctl@ceph.osd.2.smithi070.stdout:Sep 16 09:46:43 smithi070 podman[225332]: 2024-09-16 09:46:43.089387853 +0000 UTC m=+0.544035147 container died f2dcd67ca55e38c22e001fb88f01a59c2e3b6095bac48ab5fec7f6748fd4d353 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-2-deactivate, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T09:46:43.846 DEBUG:teuthology.orchestra.run.smithi070:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a1321834-740c-11ef-bceb-c7b262605968@osd.2.service' 2024-09-16T09:46:43.889 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T09:46:43.890 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-16T09:46:43.890 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-16T09:46:43.890 DEBUG:teuthology.orchestra.run.smithi070:> sudo systemctl stop ceph-a1321834-740c-11ef-bceb-c7b262605968@osd.3 2024-09-16T09:46:44.171 INFO:journalctl@ceph.osd.3.smithi070.stdout:Sep 16 09:46:43 smithi070 systemd[1]: Stopping Ceph osd.3 for a1321834-740c-11ef-bceb-c7b262605968... 2024-09-16T09:46:44.671 INFO:journalctl@ceph.osd.3.smithi070.stdout:Sep 16 09:46:44 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-3[62944]: 2024-09-16T09:46:44.223+0000 7f1dc0f0c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T09:46:44.671 INFO:journalctl@ceph.osd.3.smithi070.stdout:Sep 16 09:46:44 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-3[62944]: 2024-09-16T09:46:44.223+0000 7f1dc0f0c640 -1 osd.3 573 *** Got signal Terminated *** 2024-09-16T09:46:44.671 INFO:journalctl@ceph.osd.3.smithi070.stdout:Sep 16 09:46:44 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-3[62944]: 2024-09-16T09:46:44.223+0000 7f1dc0f0c640 -1 osd.3 573 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T09:46:45.910 INFO:journalctl@ceph.osd.5.smithi084.stdout:Sep 16 09:46:45 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-5[43256]: 2024-09-16T09:46:45.482+0000 7f3947661640 -1 osd.5 573 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-09-16T09:46:23.475156+0000 front 2024-09-16T09:46:23.475074+0000 (oldest deadline 2024-09-16T09:46:45.174934+0000) 2024-09-16T09:46:46.660 INFO:journalctl@ceph.osd.5.smithi084.stdout:Sep 16 09:46:46 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-5[43256]: 2024-09-16T09:46:46.453+0000 7f3947661640 -1 osd.5 573 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-09-16T09:46:23.475156+0000 front 2024-09-16T09:46:23.475074+0000 (oldest deadline 2024-09-16T09:46:45.174934+0000) 2024-09-16T09:46:46.661 INFO:journalctl@ceph.osd.4.smithi084.stdout:Sep 16 09:46:46 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-4[37218]: 2024-09-16T09:46:46.429+0000 7f140a6c0640 -1 osd.4 573 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-09-16T09:46:21.004163+0000 front 2024-09-16T09:46:21.004188+0000 (oldest deadline 2024-09-16T09:46:46.304069+0000) 2024-09-16T09:46:46.661 INFO:journalctl@ceph.osd.6.smithi084.stdout:Sep 16 09:46:46 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-6[49407]: 2024-09-16T09:46:46.301+0000 7f91520d2640 -1 osd.6 573 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-09-16T09:46:23.838986+0000 front 2024-09-16T09:46:23.839077+0000 (oldest deadline 2024-09-16T09:46:46.139011+0000) 2024-09-16T09:46:47.660 INFO:journalctl@ceph.osd.4.smithi084.stdout:Sep 16 09:46:47 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-4[37218]: 2024-09-16T09:46:47.407+0000 7f140a6c0640 -1 osd.4 573 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-09-16T09:46:21.004163+0000 front 2024-09-16T09:46:21.004188+0000 (oldest deadline 2024-09-16T09:46:46.304069+0000) 2024-09-16T09:46:47.660 INFO:journalctl@ceph.osd.5.smithi084.stdout:Sep 16 09:46:47 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-5[43256]: 2024-09-16T09:46:47.450+0000 7f3947661640 -1 osd.5 573 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-09-16T09:46:23.475156+0000 front 2024-09-16T09:46:23.475074+0000 (oldest deadline 2024-09-16T09:46:45.174934+0000) 2024-09-16T09:46:47.660 INFO:journalctl@ceph.osd.6.smithi084.stdout:Sep 16 09:46:47 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-6[49407]: 2024-09-16T09:46:47.335+0000 7f91520d2640 -1 osd.6 573 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-09-16T09:46:23.838986+0000 front 2024-09-16T09:46:23.839077+0000 (oldest deadline 2024-09-16T09:46:46.139011+0000) 2024-09-16T09:46:47.921 INFO:journalctl@ceph.osd.3.smithi070.stdout:Sep 16 09:46:47 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-3[62944]: 2024-09-16T09:46:47.441+0000 7f1db9519640 -1 osd.3 573 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-09-16T09:46:23.750818+0000 front 2024-09-16T09:46:23.750772+0000 (oldest deadline 2024-09-16T09:46:46.650825+0000) 2024-09-16T09:46:48.660 INFO:journalctl@ceph.osd.4.smithi084.stdout:Sep 16 09:46:48 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-4[37218]: 2024-09-16T09:46:48.423+0000 7f140a6c0640 -1 osd.4 573 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-09-16T09:46:21.004163+0000 front 2024-09-16T09:46:21.004188+0000 (oldest deadline 2024-09-16T09:46:46.304069+0000) 2024-09-16T09:46:48.660 INFO:journalctl@ceph.osd.5.smithi084.stdout:Sep 16 09:46:48 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-5[43256]: 2024-09-16T09:46:48.431+0000 7f3947661640 -1 osd.5 573 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-09-16T09:46:23.475156+0000 front 2024-09-16T09:46:23.475074+0000 (oldest deadline 2024-09-16T09:46:45.174934+0000) 2024-09-16T09:46:48.661 INFO:journalctl@ceph.osd.6.smithi084.stdout:Sep 16 09:46:48 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-6[49407]: 2024-09-16T09:46:48.340+0000 7f91520d2640 -1 osd.6 573 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-09-16T09:46:23.838986+0000 front 2024-09-16T09:46:23.839077+0000 (oldest deadline 2024-09-16T09:46:46.139011+0000) 2024-09-16T09:46:48.670 INFO:journalctl@ceph.osd.3.smithi070.stdout:Sep 16 09:46:48 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-3[62944]: 2024-09-16T09:46:48.404+0000 7f1db9519640 -1 osd.3 573 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-09-16T09:46:23.750818+0000 front 2024-09-16T09:46:23.750772+0000 (oldest deadline 2024-09-16T09:46:46.650825+0000) 2024-09-16T09:46:49.160 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:46:48 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:46:48.664+0000 7fa94a65f640 -1 osd.7 573 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-09-16T09:46:23.143051+0000 front 2024-09-16T09:46:23.142814+0000 (oldest deadline 2024-09-16T09:46:47.842666+0000) 2024-09-16T09:46:49.660 INFO:journalctl@ceph.osd.4.smithi084.stdout:Sep 16 09:46:49 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-4[37218]: 2024-09-16T09:46:49.449+0000 7f140a6c0640 -1 osd.4 573 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-09-16T09:46:21.004163+0000 front 2024-09-16T09:46:21.004188+0000 (oldest deadline 2024-09-16T09:46:46.304069+0000) 2024-09-16T09:46:49.660 INFO:journalctl@ceph.osd.5.smithi084.stdout:Sep 16 09:46:49 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-5[43256]: 2024-09-16T09:46:49.469+0000 7f3947661640 -1 osd.5 573 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-09-16T09:46:23.475156+0000 front 2024-09-16T09:46:23.475074+0000 (oldest deadline 2024-09-16T09:46:45.174934+0000) 2024-09-16T09:46:49.661 INFO:journalctl@ceph.osd.6.smithi084.stdout:Sep 16 09:46:49 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-6[49407]: 2024-09-16T09:46:49.292+0000 7f91520d2640 -1 osd.6 573 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-09-16T09:46:23.838986+0000 front 2024-09-16T09:46:23.839077+0000 (oldest deadline 2024-09-16T09:46:46.139011+0000) 2024-09-16T09:46:50.082 INFO:journalctl@ceph.osd.3.smithi070.stdout:Sep 16 09:46:49 smithi070 podman[225459]: 2024-09-16 09:46:49.802014148 +0000 UTC m=+5.686569497 container died a235495024b8593e359225827bd195a572341d0ba1edd19e7538e0f8741aaccb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-3, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T09:46:50.160 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:46:49 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:46:49.701+0000 7fa94a65f640 -1 osd.7 573 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-09-16T09:46:23.143051+0000 front 2024-09-16T09:46:23.142814+0000 (oldest deadline 2024-09-16T09:46:47.842666+0000) 2024-09-16T09:46:50.421 INFO:journalctl@ceph.osd.3.smithi070.stdout:Sep 16 09:46:50 smithi070 podman[225459]: 2024-09-16 09:46:50.264095912 +0000 UTC m=+6.148651260 container cleanup a235495024b8593e359225827bd195a572341d0ba1edd19e7538e0f8741aaccb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-3, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T09:46:50.421 INFO:journalctl@ceph.osd.3.smithi070.stdout:Sep 16 09:46:50 smithi070 bash[225459]: ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-3 2024-09-16T09:46:50.654 INFO:journalctl@ceph.osd.4.smithi084.stdout:Sep 16 09:46:50 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-4[37218]: 2024-09-16T09:46:50.486+0000 7f140a6c0640 -1 osd.4 573 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-09-16T09:46:21.004163+0000 front 2024-09-16T09:46:21.004188+0000 (oldest deadline 2024-09-16T09:46:46.304069+0000) 2024-09-16T09:46:50.654 INFO:journalctl@ceph.osd.5.smithi084.stdout:Sep 16 09:46:50 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-5[43256]: 2024-09-16T09:46:50.432+0000 7f3947661640 -1 osd.5 573 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-09-16T09:46:23.475156+0000 front 2024-09-16T09:46:23.475074+0000 (oldest deadline 2024-09-16T09:46:45.174934+0000) 2024-09-16T09:46:50.654 INFO:journalctl@ceph.osd.6.smithi084.stdout:Sep 16 09:46:50 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-6[49407]: 2024-09-16T09:46:50.339+0000 7f91520d2640 -1 osd.6 573 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-09-16T09:46:23.838986+0000 front 2024-09-16T09:46:23.839077+0000 (oldest deadline 2024-09-16T09:46:46.139011+0000) 2024-09-16T09:46:50.875 INFO:journalctl@ceph.osd.3.smithi070.stdout:Sep 16 09:46:50 smithi070 podman[225481]: 2024-09-16 09:46:50.548971021 +0000 UTC m=+0.737111535 container remove a235495024b8593e359225827bd195a572341d0ba1edd19e7538e0f8741aaccb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-16T09:46:50.910 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:46:50 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:46:50.651+0000 7fa94a65f640 -1 osd.7 573 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-09-16T09:46:23.143051+0000 front 2024-09-16T09:46:23.142814+0000 (oldest deadline 2024-09-16T09:46:47.842666+0000) 2024-09-16T09:46:51.171 INFO:journalctl@ceph.osd.3.smithi070.stdout:Sep 16 09:46:50 smithi070 podman[225575]: 2024-09-16 09:46:50.874844899 +0000 UTC m=+0.112519741 container create a20f8778a993d60903d54f25598758e38bb0079e7c19b844565bd2c53d0016c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-16T09:46:51.171 INFO:journalctl@ceph.osd.3.smithi070.stdout:Sep 16 09:46:50 smithi070 podman[225575]: 2024-09-16 09:46:50.777928498 +0000 UTC m=+0.015603298 image pull 425e316ba8ebf74f04aa3640f7deb4fa3f293a78d0c8d097cb679eaf86ddca6c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce 2024-09-16T09:46:51.171 INFO:journalctl@ceph.osd.3.smithi070.stdout:Sep 16 09:46:51 smithi070 podman[225575]: 2024-09-16 09:46:51.085762469 +0000 UTC m=+0.323437279 container init a20f8778a993d60903d54f25598758e38bb0079e7c19b844565bd2c53d0016c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-3-deactivate, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-16T09:46:51.171 INFO:journalctl@ceph.osd.3.smithi070.stdout:Sep 16 09:46:51 smithi070 podman[225575]: 2024-09-16 09:46:51.091062769 +0000 UTC m=+0.328737563 container start a20f8778a993d60903d54f25598758e38bb0079e7c19b844565bd2c53d0016c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-16T09:46:51.548 INFO:journalctl@ceph.osd.3.smithi070.stdout:Sep 16 09:46:51 smithi070 podman[225575]: 2024-09-16 09:46:51.177458394 +0000 UTC m=+0.415133191 container attach a20f8778a993d60903d54f25598758e38bb0079e7c19b844565bd2c53d0016c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T09:46:51.548 INFO:journalctl@ceph.osd.3.smithi070.stdout:Sep 16 09:46:51 smithi070 podman[225575]: 2024-09-16 09:46:51.274475669 +0000 UTC m=+0.512150464 container died a20f8778a993d60903d54f25598758e38bb0079e7c19b844565bd2c53d0016c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-3-deactivate, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-16T09:46:51.660 INFO:journalctl@ceph.osd.5.smithi084.stdout:Sep 16 09:46:51 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-5[43256]: 2024-09-16T09:46:51.410+0000 7f3947661640 -1 osd.5 573 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-09-16T09:46:23.475156+0000 front 2024-09-16T09:46:23.475074+0000 (oldest deadline 2024-09-16T09:46:45.174934+0000) 2024-09-16T09:46:51.660 INFO:journalctl@ceph.osd.4.smithi084.stdout:Sep 16 09:46:51 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-4[37218]: 2024-09-16T09:46:51.461+0000 7f140a6c0640 -1 osd.4 573 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-09-16T09:46:21.004163+0000 front 2024-09-16T09:46:21.004188+0000 (oldest deadline 2024-09-16T09:46:46.304069+0000) 2024-09-16T09:46:51.660 INFO:journalctl@ceph.osd.6.smithi084.stdout:Sep 16 09:46:51 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-6[49407]: 2024-09-16T09:46:51.324+0000 7f91520d2640 -1 osd.6 573 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-09-16T09:46:23.838986+0000 front 2024-09-16T09:46:23.839077+0000 (oldest deadline 2024-09-16T09:46:46.139011+0000) 2024-09-16T09:46:51.876 DEBUG:teuthology.orchestra.run.smithi070:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a1321834-740c-11ef-bceb-c7b262605968@osd.3.service' 2024-09-16T09:46:51.914 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T09:46:51.914 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-16T09:46:51.914 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-16T09:46:51.914 DEBUG:teuthology.orchestra.run.smithi084:> sudo systemctl stop ceph-a1321834-740c-11ef-bceb-c7b262605968@osd.4 2024-09-16T09:46:51.944 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:46:51 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:46:51.679+0000 7fa94a65f640 -1 osd.7 573 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-09-16T09:46:23.143051+0000 front 2024-09-16T09:46:23.142814+0000 (oldest deadline 2024-09-16T09:46:47.842666+0000) 2024-09-16T09:46:52.269 INFO:journalctl@ceph.osd.4.smithi084.stdout:Sep 16 09:46:51 smithi084 systemd[1]: Stopping Ceph osd.4 for a1321834-740c-11ef-bceb-c7b262605968... 2024-09-16T09:46:52.646 INFO:journalctl@ceph.osd.5.smithi084.stdout:Sep 16 09:46:52 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-5[43256]: 2024-09-16T09:46:52.407+0000 7f3947661640 -1 osd.5 573 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-09-16T09:46:23.475156+0000 front 2024-09-16T09:46:23.475074+0000 (oldest deadline 2024-09-16T09:46:45.174934+0000) 2024-09-16T09:46:52.646 INFO:journalctl@ceph.osd.6.smithi084.stdout:Sep 16 09:46:52 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-6[49407]: 2024-09-16T09:46:52.335+0000 7f91520d2640 -1 osd.6 573 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-09-16T09:46:23.838986+0000 front 2024-09-16T09:46:23.839077+0000 (oldest deadline 2024-09-16T09:46:46.139011+0000) 2024-09-16T09:46:52.646 INFO:journalctl@ceph.osd.4.smithi084.stdout:Sep 16 09:46:52 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-4[37218]: 2024-09-16T09:46:52.268+0000 7f140e0ab640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T09:46:52.646 INFO:journalctl@ceph.osd.4.smithi084.stdout:Sep 16 09:46:52 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-4[37218]: 2024-09-16T09:46:52.268+0000 7f140e0ab640 -1 osd.4 573 *** Got signal Terminated *** 2024-09-16T09:46:52.646 INFO:journalctl@ceph.osd.4.smithi084.stdout:Sep 16 09:46:52 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-4[37218]: 2024-09-16T09:46:52.268+0000 7f140e0ab640 -1 osd.4 573 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T09:46:52.647 INFO:journalctl@ceph.osd.4.smithi084.stdout:Sep 16 09:46:52 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-4[37218]: 2024-09-16T09:46:52.491+0000 7f140a6c0640 -1 osd.4 573 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-09-16T09:46:21.004163+0000 front 2024-09-16T09:46:21.004188+0000 (oldest deadline 2024-09-16T09:46:46.304069+0000) 2024-09-16T09:46:52.910 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:46:52 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:46:52.644+0000 7fa94a65f640 -1 osd.7 573 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-09-16T09:46:23.143051+0000 front 2024-09-16T09:46:23.142814+0000 (oldest deadline 2024-09-16T09:46:47.842666+0000) 2024-09-16T09:46:53.636 INFO:journalctl@ceph.osd.6.smithi084.stdout:Sep 16 09:46:53 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-6[49407]: 2024-09-16T09:46:53.381+0000 7f91520d2640 -1 osd.6 573 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-09-16T09:46:23.838986+0000 front 2024-09-16T09:46:23.839077+0000 (oldest deadline 2024-09-16T09:46:46.139011+0000) 2024-09-16T09:46:53.636 INFO:journalctl@ceph.osd.4.smithi084.stdout:Sep 16 09:46:53 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-4[37218]: 2024-09-16T09:46:53.467+0000 7f140a6c0640 -1 osd.4 573 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-09-16T09:46:21.004163+0000 front 2024-09-16T09:46:21.004188+0000 (oldest deadline 2024-09-16T09:46:46.304069+0000) 2024-09-16T09:46:53.637 INFO:journalctl@ceph.osd.5.smithi084.stdout:Sep 16 09:46:53 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-5[43256]: 2024-09-16T09:46:53.412+0000 7f3947661640 -1 osd.5 573 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-09-16T09:46:23.475156+0000 front 2024-09-16T09:46:23.475074+0000 (oldest deadline 2024-09-16T09:46:45.174934+0000) 2024-09-16T09:46:53.910 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:46:53 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:46:53.634+0000 7fa94a65f640 -1 osd.7 573 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-09-16T09:46:23.143051+0000 front 2024-09-16T09:46:23.142814+0000 (oldest deadline 2024-09-16T09:46:47.842666+0000) 2024-09-16T09:46:54.648 INFO:journalctl@ceph.osd.6.smithi084.stdout:Sep 16 09:46:54 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-6[49407]: 2024-09-16T09:46:54.346+0000 7f91520d2640 -1 osd.6 573 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-09-16T09:46:23.838986+0000 front 2024-09-16T09:46:23.839077+0000 (oldest deadline 2024-09-16T09:46:46.139011+0000) 2024-09-16T09:46:54.648 INFO:journalctl@ceph.osd.4.smithi084.stdout:Sep 16 09:46:54 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-4[37218]: 2024-09-16T09:46:54.444+0000 7f140a6c0640 -1 osd.4 573 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-09-16T09:46:21.004163+0000 front 2024-09-16T09:46:21.004188+0000 (oldest deadline 2024-09-16T09:46:46.304069+0000) 2024-09-16T09:46:54.649 INFO:journalctl@ceph.osd.5.smithi084.stdout:Sep 16 09:46:54 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-5[43256]: 2024-09-16T09:46:54.408+0000 7f3947661640 -1 osd.5 573 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-09-16T09:46:23.475156+0000 front 2024-09-16T09:46:23.475074+0000 (oldest deadline 2024-09-16T09:46:45.174934+0000) 2024-09-16T09:46:54.649 INFO:journalctl@ceph.osd.5.smithi084.stdout:Sep 16 09:46:54 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-5[43256]: 2024-09-16T09:46:54.408+0000 7f3947661640 -1 osd.5 573 heartbeat_check: no reply from 172.21.15.70:6814 osd.1 since back 2024-09-16T09:46:32.676604+0000 front 2024-09-16T09:46:32.676542+0000 (oldest deadline 2024-09-16T09:46:54.376526+0000) 2024-09-16T09:46:54.910 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:46:54 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:46:54.646+0000 7fa94a65f640 -1 osd.7 573 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-09-16T09:46:23.143051+0000 front 2024-09-16T09:46:23.142814+0000 (oldest deadline 2024-09-16T09:46:47.842666+0000) 2024-09-16T09:46:55.660 INFO:journalctl@ceph.osd.4.smithi084.stdout:Sep 16 09:46:55 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-4[37218]: 2024-09-16T09:46:55.394+0000 7f140a6c0640 -1 osd.4 573 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-09-16T09:46:21.004163+0000 front 2024-09-16T09:46:21.004188+0000 (oldest deadline 2024-09-16T09:46:46.304069+0000) 2024-09-16T09:46:55.660 INFO:journalctl@ceph.osd.6.smithi084.stdout:Sep 16 09:46:55 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-6[49407]: 2024-09-16T09:46:55.396+0000 7f91520d2640 -1 osd.6 573 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-09-16T09:46:23.838986+0000 front 2024-09-16T09:46:23.839077+0000 (oldest deadline 2024-09-16T09:46:46.139011+0000) 2024-09-16T09:46:55.661 INFO:journalctl@ceph.osd.5.smithi084.stdout:Sep 16 09:46:55 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-5[43256]: 2024-09-16T09:46:55.404+0000 7f3947661640 -1 osd.5 573 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-09-16T09:46:23.475156+0000 front 2024-09-16T09:46:23.475074+0000 (oldest deadline 2024-09-16T09:46:45.174934+0000) 2024-09-16T09:46:55.661 INFO:journalctl@ceph.osd.5.smithi084.stdout:Sep 16 09:46:55 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-5[43256]: 2024-09-16T09:46:55.404+0000 7f3947661640 -1 osd.5 573 heartbeat_check: no reply from 172.21.15.70:6814 osd.1 since back 2024-09-16T09:46:32.676604+0000 front 2024-09-16T09:46:32.676542+0000 (oldest deadline 2024-09-16T09:46:54.376526+0000) 2024-09-16T09:46:56.160 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:46:55 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:46:55.695+0000 7fa94a65f640 -1 osd.7 573 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-09-16T09:46:23.143051+0000 front 2024-09-16T09:46:23.142814+0000 (oldest deadline 2024-09-16T09:46:47.842666+0000) 2024-09-16T09:46:56.660 INFO:journalctl@ceph.osd.6.smithi084.stdout:Sep 16 09:46:56 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-6[49407]: 2024-09-16T09:46:56.374+0000 7f91520d2640 -1 osd.6 573 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-09-16T09:46:23.838986+0000 front 2024-09-16T09:46:23.839077+0000 (oldest deadline 2024-09-16T09:46:46.139011+0000) 2024-09-16T09:46:56.661 INFO:journalctl@ceph.osd.4.smithi084.stdout:Sep 16 09:46:56 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-4[37218]: 2024-09-16T09:46:56.436+0000 7f140a6c0640 -1 osd.4 573 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-09-16T09:46:21.004163+0000 front 2024-09-16T09:46:21.004188+0000 (oldest deadline 2024-09-16T09:46:46.304069+0000) 2024-09-16T09:46:56.661 INFO:journalctl@ceph.osd.4.smithi084.stdout:Sep 16 09:46:56 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-4[37218]: 2024-09-16T09:46:56.436+0000 7f140a6c0640 -1 osd.4 573 heartbeat_check: no reply from 172.21.15.70:6814 osd.1 since back 2024-09-16T09:46:32.205069+0000 front 2024-09-16T09:46:32.205054+0000 (oldest deadline 2024-09-16T09:46:56.304829+0000) 2024-09-16T09:46:56.661 INFO:journalctl@ceph.osd.5.smithi084.stdout:Sep 16 09:46:56 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-5[43256]: 2024-09-16T09:46:56.361+0000 7f3947661640 -1 osd.5 573 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-09-16T09:46:23.475156+0000 front 2024-09-16T09:46:23.475074+0000 (oldest deadline 2024-09-16T09:46:45.174934+0000) 2024-09-16T09:46:56.661 INFO:journalctl@ceph.osd.5.smithi084.stdout:Sep 16 09:46:56 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-5[43256]: 2024-09-16T09:46:56.361+0000 7f3947661640 -1 osd.5 573 heartbeat_check: no reply from 172.21.15.70:6814 osd.1 since back 2024-09-16T09:46:32.676604+0000 front 2024-09-16T09:46:32.676542+0000 (oldest deadline 2024-09-16T09:46:54.376526+0000) 2024-09-16T09:46:57.160 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:46:56 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:46:56.686+0000 7fa94a65f640 -1 osd.7 573 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-09-16T09:46:23.143051+0000 front 2024-09-16T09:46:23.142814+0000 (oldest deadline 2024-09-16T09:46:47.842666+0000) 2024-09-16T09:46:57.429 INFO:journalctl@ceph.osd.5.smithi084.stdout:Sep 16 09:46:57 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-5[43256]: 2024-09-16T09:46:57.348+0000 7f3947661640 -1 osd.5 573 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-09-16T09:46:23.475156+0000 front 2024-09-16T09:46:23.475074+0000 (oldest deadline 2024-09-16T09:46:45.174934+0000) 2024-09-16T09:46:57.429 INFO:journalctl@ceph.osd.5.smithi084.stdout:Sep 16 09:46:57 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-5[43256]: 2024-09-16T09:46:57.348+0000 7f3947661640 -1 osd.5 573 heartbeat_check: no reply from 172.21.15.70:6814 osd.1 since back 2024-09-16T09:46:32.676604+0000 front 2024-09-16T09:46:32.676542+0000 (oldest deadline 2024-09-16T09:46:54.376526+0000) 2024-09-16T09:46:57.429 INFO:journalctl@ceph.osd.6.smithi084.stdout:Sep 16 09:46:57 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-6[49407]: 2024-09-16T09:46:57.336+0000 7f91520d2640 -1 osd.6 573 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-09-16T09:46:23.838986+0000 front 2024-09-16T09:46:23.839077+0000 (oldest deadline 2024-09-16T09:46:46.139011+0000) 2024-09-16T09:46:57.429 INFO:journalctl@ceph.osd.6.smithi084.stdout:Sep 16 09:46:57 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-6[49407]: 2024-09-16T09:46:57.336+0000 7f91520d2640 -1 osd.6 573 heartbeat_check: no reply from 172.21.15.70:6814 osd.1 since back 2024-09-16T09:46:32.540265+0000 front 2024-09-16T09:46:32.540419+0000 (oldest deadline 2024-09-16T09:46:56.540676+0000) 2024-09-16T09:46:57.761 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:46:57 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:46:57.669+0000 7fa94a65f640 -1 osd.7 573 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-09-16T09:46:23.143051+0000 front 2024-09-16T09:46:23.142814+0000 (oldest deadline 2024-09-16T09:46:47.842666+0000) 2024-09-16T09:46:58.060 INFO:journalctl@ceph.osd.4.smithi084.stdout:Sep 16 09:46:57 smithi084 podman[153787]: 2024-09-16 09:46:57.759379507 +0000 UTC m=+5.639577621 container died dcdc4afb2ab8eb88c0ee0d9b00992d548b0e712ff35c5a5f3af9ff1e27ff1579 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-4, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-16T09:46:58.319 INFO:journalctl@ceph.osd.4.smithi084.stdout:Sep 16 09:46:58 smithi084 podman[153787]: 2024-09-16 09:46:58.232681629 +0000 UTC m=+6.112879741 container cleanup dcdc4afb2ab8eb88c0ee0d9b00992d548b0e712ff35c5a5f3af9ff1e27ff1579 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-4, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T09:46:58.320 INFO:journalctl@ceph.osd.4.smithi084.stdout:Sep 16 09:46:58 smithi084 bash[153787]: ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-4 2024-09-16T09:46:58.660 INFO:journalctl@ceph.osd.4.smithi084.stdout:Sep 16 09:46:58 smithi084 podman[153816]: 2024-09-16 09:46:58.526473457 +0000 UTC m=+0.766823764 container remove dcdc4afb2ab8eb88c0ee0d9b00992d548b0e712ff35c5a5f3af9ff1e27ff1579 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-4, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-main-2024-09-13-1248-8293d73) 2024-09-16T09:46:58.660 INFO:journalctl@ceph.osd.5.smithi084.stdout:Sep 16 09:46:58 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-5[43256]: 2024-09-16T09:46:58.331+0000 7f3947661640 -1 osd.5 573 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-09-16T09:46:23.475156+0000 front 2024-09-16T09:46:23.475074+0000 (oldest deadline 2024-09-16T09:46:45.174934+0000) 2024-09-16T09:46:58.660 INFO:journalctl@ceph.osd.5.smithi084.stdout:Sep 16 09:46:58 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-5[43256]: 2024-09-16T09:46:58.331+0000 7f3947661640 -1 osd.5 573 heartbeat_check: no reply from 172.21.15.70:6814 osd.1 since back 2024-09-16T09:46:32.676604+0000 front 2024-09-16T09:46:32.676542+0000 (oldest deadline 2024-09-16T09:46:54.376526+0000) 2024-09-16T09:46:58.661 INFO:journalctl@ceph.osd.6.smithi084.stdout:Sep 16 09:46:58 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-6[49407]: 2024-09-16T09:46:58.317+0000 7f91520d2640 -1 osd.6 573 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-09-16T09:46:23.838986+0000 front 2024-09-16T09:46:23.839077+0000 (oldest deadline 2024-09-16T09:46:46.139011+0000) 2024-09-16T09:46:58.661 INFO:journalctl@ceph.osd.6.smithi084.stdout:Sep 16 09:46:58 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-6[49407]: 2024-09-16T09:46:58.317+0000 7f91520d2640 -1 osd.6 573 heartbeat_check: no reply from 172.21.15.70:6814 osd.1 since back 2024-09-16T09:46:32.540265+0000 front 2024-09-16T09:46:32.540419+0000 (oldest deadline 2024-09-16T09:46:56.540676+0000) 2024-09-16T09:46:58.983 INFO:journalctl@ceph.osd.4.smithi084.stdout:Sep 16 09:46:58 smithi084 podman[153913]: 2024-09-16 09:46:58.737441334 +0000 UTC m=+0.015711380 image pull 425e316ba8ebf74f04aa3640f7deb4fa3f293a78d0c8d097cb679eaf86ddca6c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce 2024-09-16T09:46:58.983 INFO:journalctl@ceph.osd.4.smithi084.stdout:Sep 16 09:46:58 smithi084 podman[153913]: 2024-09-16 09:46:58.852324575 +0000 UTC m=+0.130594611 container create 6b4c9ab7e45653d46c41cf785dc1341bb7b49f95f6c56ef82aa81edc130bdda0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-4-deactivate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-09-16T09:46:58.983 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:46:58 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:46:58.687+0000 7fa94a65f640 -1 osd.7 573 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-09-16T09:46:23.143051+0000 front 2024-09-16T09:46:23.142814+0000 (oldest deadline 2024-09-16T09:46:47.842666+0000) 2024-09-16T09:46:58.983 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:46:58 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:46:58.687+0000 7fa94a65f640 -1 osd.7 573 heartbeat_check: no reply from 172.21.15.70:6814 osd.1 since back 2024-09-16T09:46:33.043837+0000 front 2024-09-16T09:46:33.044046+0000 (oldest deadline 2024-09-16T09:46:57.744009+0000) 2024-09-16T09:46:59.276 INFO:journalctl@ceph.osd.4.smithi084.stdout:Sep 16 09:46:59 smithi084 podman[153913]: 2024-09-16 09:46:59.105133426 +0000 UTC m=+0.383403466 container init 6b4c9ab7e45653d46c41cf785dc1341bb7b49f95f6c56ef82aa81edc130bdda0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-16T09:46:59.276 INFO:journalctl@ceph.osd.4.smithi084.stdout:Sep 16 09:46:59 smithi084 podman[153913]: 2024-09-16 09:46:59.109891384 +0000 UTC m=+0.388161424 container start 6b4c9ab7e45653d46c41cf785dc1341bb7b49f95f6c56ef82aa81edc130bdda0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-16T09:46:59.276 INFO:journalctl@ceph.osd.4.smithi084.stdout:Sep 16 09:46:59 smithi084 podman[153913]: 2024-09-16 09:46:59.204892269 +0000 UTC m=+0.483162317 container attach 6b4c9ab7e45653d46c41cf785dc1341bb7b49f95f6c56ef82aa81edc130bdda0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-16T09:46:59.660 INFO:journalctl@ceph.osd.4.smithi084.stdout:Sep 16 09:46:59 smithi084 podman[153913]: 2024-09-16 09:46:59.469209233 +0000 UTC m=+0.747479270 container died 6b4c9ab7e45653d46c41cf785dc1341bb7b49f95f6c56ef82aa81edc130bdda0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-16T09:46:59.660 INFO:journalctl@ceph.osd.5.smithi084.stdout:Sep 16 09:46:59 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-5[43256]: 2024-09-16T09:46:59.305+0000 7f3947661640 -1 osd.5 573 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-09-16T09:46:23.475156+0000 front 2024-09-16T09:46:23.475074+0000 (oldest deadline 2024-09-16T09:46:45.174934+0000) 2024-09-16T09:46:59.661 INFO:journalctl@ceph.osd.5.smithi084.stdout:Sep 16 09:46:59 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-5[43256]: 2024-09-16T09:46:59.305+0000 7f3947661640 -1 osd.5 573 heartbeat_check: no reply from 172.21.15.70:6814 osd.1 since back 2024-09-16T09:46:32.676604+0000 front 2024-09-16T09:46:32.676542+0000 (oldest deadline 2024-09-16T09:46:54.376526+0000) 2024-09-16T09:46:59.661 INFO:journalctl@ceph.osd.6.smithi084.stdout:Sep 16 09:46:59 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-6[49407]: 2024-09-16T09:46:59.274+0000 7f91520d2640 -1 osd.6 573 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-09-16T09:46:23.838986+0000 front 2024-09-16T09:46:23.839077+0000 (oldest deadline 2024-09-16T09:46:46.139011+0000) 2024-09-16T09:46:59.661 INFO:journalctl@ceph.osd.6.smithi084.stdout:Sep 16 09:46:59 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-6[49407]: 2024-09-16T09:46:59.274+0000 7f91520d2640 -1 osd.6 573 heartbeat_check: no reply from 172.21.15.70:6814 osd.1 since back 2024-09-16T09:46:32.540265+0000 front 2024-09-16T09:46:32.540419+0000 (oldest deadline 2024-09-16T09:46:56.540676+0000) 2024-09-16T09:47:00.043 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:46:59 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:46:59.671+0000 7fa94a65f640 -1 osd.7 573 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-09-16T09:46:23.143051+0000 front 2024-09-16T09:46:23.142814+0000 (oldest deadline 2024-09-16T09:46:47.842666+0000) 2024-09-16T09:47:00.043 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:46:59 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:46:59.671+0000 7fa94a65f640 -1 osd.7 573 heartbeat_check: no reply from 172.21.15.70:6814 osd.1 since back 2024-09-16T09:46:33.043837+0000 front 2024-09-16T09:46:33.044046+0000 (oldest deadline 2024-09-16T09:46:57.744009+0000) 2024-09-16T09:47:00.070 DEBUG:teuthology.orchestra.run.smithi084:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a1321834-740c-11ef-bceb-c7b262605968@osd.4.service' 2024-09-16T09:47:00.151 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T09:47:00.151 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-16T09:47:00.152 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-16T09:47:00.152 DEBUG:teuthology.orchestra.run.smithi084:> sudo systemctl stop ceph-a1321834-740c-11ef-bceb-c7b262605968@osd.5 2024-09-16T09:47:00.301 INFO:journalctl@ceph.osd.5.smithi084.stdout:Sep 16 09:47:00 smithi084 systemd[1]: Stopping Ceph osd.5 for a1321834-740c-11ef-bceb-c7b262605968... 2024-09-16T09:47:00.660 INFO:journalctl@ceph.osd.6.smithi084.stdout:Sep 16 09:47:00 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-6[49407]: 2024-09-16T09:47:00.299+0000 7f91520d2640 -1 osd.6 573 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-09-16T09:46:23.838986+0000 front 2024-09-16T09:46:23.839077+0000 (oldest deadline 2024-09-16T09:46:46.139011+0000) 2024-09-16T09:47:00.660 INFO:journalctl@ceph.osd.6.smithi084.stdout:Sep 16 09:47:00 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-6[49407]: 2024-09-16T09:47:00.299+0000 7f91520d2640 -1 osd.6 573 heartbeat_check: no reply from 172.21.15.70:6814 osd.1 since back 2024-09-16T09:46:32.540265+0000 front 2024-09-16T09:46:32.540419+0000 (oldest deadline 2024-09-16T09:46:56.540676+0000) 2024-09-16T09:47:00.661 INFO:journalctl@ceph.osd.5.smithi084.stdout:Sep 16 09:47:00 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-5[43256]: 2024-09-16T09:47:00.314+0000 7f3947661640 -1 osd.5 573 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-09-16T09:46:23.475156+0000 front 2024-09-16T09:46:23.475074+0000 (oldest deadline 2024-09-16T09:46:45.174934+0000) 2024-09-16T09:47:00.661 INFO:journalctl@ceph.osd.5.smithi084.stdout:Sep 16 09:47:00 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-5[43256]: 2024-09-16T09:47:00.314+0000 7f3947661640 -1 osd.5 573 heartbeat_check: no reply from 172.21.15.70:6814 osd.1 since back 2024-09-16T09:46:32.676604+0000 front 2024-09-16T09:46:32.676542+0000 (oldest deadline 2024-09-16T09:46:54.376526+0000) 2024-09-16T09:47:00.661 INFO:journalctl@ceph.osd.5.smithi084.stdout:Sep 16 09:47:00 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-5[43256]: 2024-09-16T09:47:00.488+0000 7f394b04c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T09:47:00.661 INFO:journalctl@ceph.osd.5.smithi084.stdout:Sep 16 09:47:00 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-5[43256]: 2024-09-16T09:47:00.488+0000 7f394b04c640 -1 osd.5 573 *** Got signal Terminated *** 2024-09-16T09:47:00.661 INFO:journalctl@ceph.osd.5.smithi084.stdout:Sep 16 09:47:00 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-5[43256]: 2024-09-16T09:47:00.488+0000 7f394b04c640 -1 osd.5 573 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T09:47:01.160 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:47:00 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:47:00.662+0000 7fa94a65f640 -1 osd.7 573 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-09-16T09:46:23.143051+0000 front 2024-09-16T09:46:23.142814+0000 (oldest deadline 2024-09-16T09:46:47.842666+0000) 2024-09-16T09:47:01.161 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:47:00 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:47:00.662+0000 7fa94a65f640 -1 osd.7 573 heartbeat_check: no reply from 172.21.15.70:6814 osd.1 since back 2024-09-16T09:46:33.043837+0000 front 2024-09-16T09:46:33.044046+0000 (oldest deadline 2024-09-16T09:46:57.744009+0000) 2024-09-16T09:47:01.660 INFO:journalctl@ceph.osd.5.smithi084.stdout:Sep 16 09:47:01 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-5[43256]: 2024-09-16T09:47:01.342+0000 7f3947661640 -1 osd.5 573 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-09-16T09:46:23.475156+0000 front 2024-09-16T09:46:23.475074+0000 (oldest deadline 2024-09-16T09:46:45.174934+0000) 2024-09-16T09:47:01.660 INFO:journalctl@ceph.osd.5.smithi084.stdout:Sep 16 09:47:01 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-5[43256]: 2024-09-16T09:47:01.342+0000 7f3947661640 -1 osd.5 573 heartbeat_check: no reply from 172.21.15.70:6814 osd.1 since back 2024-09-16T09:46:32.676604+0000 front 2024-09-16T09:46:32.676542+0000 (oldest deadline 2024-09-16T09:46:54.376526+0000) 2024-09-16T09:47:01.660 INFO:journalctl@ceph.osd.6.smithi084.stdout:Sep 16 09:47:01 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-6[49407]: 2024-09-16T09:47:01.335+0000 7f91520d2640 -1 osd.6 573 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-09-16T09:46:23.838986+0000 front 2024-09-16T09:46:23.839077+0000 (oldest deadline 2024-09-16T09:46:46.139011+0000) 2024-09-16T09:47:01.660 INFO:journalctl@ceph.osd.6.smithi084.stdout:Sep 16 09:47:01 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-6[49407]: 2024-09-16T09:47:01.335+0000 7f91520d2640 -1 osd.6 573 heartbeat_check: no reply from 172.21.15.70:6814 osd.1 since back 2024-09-16T09:46:32.540265+0000 front 2024-09-16T09:46:32.540419+0000 (oldest deadline 2024-09-16T09:46:56.540676+0000) 2024-09-16T09:47:02.160 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:47:01 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:47:01.684+0000 7fa94a65f640 -1 osd.7 573 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-09-16T09:46:23.143051+0000 front 2024-09-16T09:46:23.142814+0000 (oldest deadline 2024-09-16T09:46:47.842666+0000) 2024-09-16T09:47:02.160 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:47:01 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:47:01.684+0000 7fa94a65f640 -1 osd.7 573 heartbeat_check: no reply from 172.21.15.70:6814 osd.1 since back 2024-09-16T09:46:33.043837+0000 front 2024-09-16T09:46:33.044046+0000 (oldest deadline 2024-09-16T09:46:57.744009+0000) 2024-09-16T09:47:02.660 INFO:journalctl@ceph.osd.5.smithi084.stdout:Sep 16 09:47:02 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-5[43256]: 2024-09-16T09:47:02.345+0000 7f3947661640 -1 osd.5 573 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-09-16T09:46:23.475156+0000 front 2024-09-16T09:46:23.475074+0000 (oldest deadline 2024-09-16T09:46:45.174934+0000) 2024-09-16T09:47:02.660 INFO:journalctl@ceph.osd.5.smithi084.stdout:Sep 16 09:47:02 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-5[43256]: 2024-09-16T09:47:02.345+0000 7f3947661640 -1 osd.5 573 heartbeat_check: no reply from 172.21.15.70:6814 osd.1 since back 2024-09-16T09:46:32.676604+0000 front 2024-09-16T09:46:32.676542+0000 (oldest deadline 2024-09-16T09:46:54.376526+0000) 2024-09-16T09:47:02.661 INFO:journalctl@ceph.osd.5.smithi084.stdout:Sep 16 09:47:02 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-5[43256]: 2024-09-16T09:47:02.345+0000 7f3947661640 -1 osd.5 573 heartbeat_check: no reply from 172.21.15.70:6822 osd.2 since back 2024-09-16T09:46:38.477653+0000 front 2024-09-16T09:46:38.477765+0000 (oldest deadline 2024-09-16T09:47:01.977603+0000) 2024-09-16T09:47:02.661 INFO:journalctl@ceph.osd.6.smithi084.stdout:Sep 16 09:47:02 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-6[49407]: 2024-09-16T09:47:02.341+0000 7f91520d2640 -1 osd.6 573 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-09-16T09:46:23.838986+0000 front 2024-09-16T09:46:23.839077+0000 (oldest deadline 2024-09-16T09:46:46.139011+0000) 2024-09-16T09:47:02.661 INFO:journalctl@ceph.osd.6.smithi084.stdout:Sep 16 09:47:02 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-6[49407]: 2024-09-16T09:47:02.341+0000 7f91520d2640 -1 osd.6 573 heartbeat_check: no reply from 172.21.15.70:6814 osd.1 since back 2024-09-16T09:46:32.540265+0000 front 2024-09-16T09:46:32.540419+0000 (oldest deadline 2024-09-16T09:46:56.540676+0000) 2024-09-16T09:47:02.662 INFO:journalctl@ceph.osd.6.smithi084.stdout:Sep 16 09:47:02 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-6[49407]: 2024-09-16T09:47:02.341+0000 7f91520d2640 -1 osd.6 573 heartbeat_check: no reply from 172.21.15.70:6822 osd.2 since back 2024-09-16T09:46:36.541325+0000 front 2024-09-16T09:46:36.541385+0000 (oldest deadline 2024-09-16T09:47:01.841229+0000) 2024-09-16T09:47:03.160 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:47:02 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:47:02.680+0000 7fa94a65f640 -1 osd.7 573 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-09-16T09:46:23.143051+0000 front 2024-09-16T09:46:23.142814+0000 (oldest deadline 2024-09-16T09:46:47.842666+0000) 2024-09-16T09:47:03.160 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:47:02 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:47:02.680+0000 7fa94a65f640 -1 osd.7 573 heartbeat_check: no reply from 172.21.15.70:6814 osd.1 since back 2024-09-16T09:46:33.043837+0000 front 2024-09-16T09:46:33.044046+0000 (oldest deadline 2024-09-16T09:46:57.744009+0000) 2024-09-16T09:47:03.651 INFO:journalctl@ceph.osd.5.smithi084.stdout:Sep 16 09:47:03 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-5[43256]: 2024-09-16T09:47:03.299+0000 7f3947661640 -1 osd.5 573 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-09-16T09:46:23.475156+0000 front 2024-09-16T09:46:23.475074+0000 (oldest deadline 2024-09-16T09:46:45.174934+0000) 2024-09-16T09:47:03.652 INFO:journalctl@ceph.osd.5.smithi084.stdout:Sep 16 09:47:03 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-5[43256]: 2024-09-16T09:47:03.299+0000 7f3947661640 -1 osd.5 573 heartbeat_check: no reply from 172.21.15.70:6814 osd.1 since back 2024-09-16T09:46:32.676604+0000 front 2024-09-16T09:46:32.676542+0000 (oldest deadline 2024-09-16T09:46:54.376526+0000) 2024-09-16T09:47:03.652 INFO:journalctl@ceph.osd.5.smithi084.stdout:Sep 16 09:47:03 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-5[43256]: 2024-09-16T09:47:03.299+0000 7f3947661640 -1 osd.5 573 heartbeat_check: no reply from 172.21.15.70:6822 osd.2 since back 2024-09-16T09:46:38.477653+0000 front 2024-09-16T09:46:38.477765+0000 (oldest deadline 2024-09-16T09:47:01.977603+0000) 2024-09-16T09:47:03.652 INFO:journalctl@ceph.osd.6.smithi084.stdout:Sep 16 09:47:03 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-6[49407]: 2024-09-16T09:47:03.389+0000 7f91520d2640 -1 osd.6 573 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-09-16T09:46:23.838986+0000 front 2024-09-16T09:46:23.839077+0000 (oldest deadline 2024-09-16T09:46:46.139011+0000) 2024-09-16T09:47:03.652 INFO:journalctl@ceph.osd.6.smithi084.stdout:Sep 16 09:47:03 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-6[49407]: 2024-09-16T09:47:03.389+0000 7f91520d2640 -1 osd.6 573 heartbeat_check: no reply from 172.21.15.70:6814 osd.1 since back 2024-09-16T09:46:32.540265+0000 front 2024-09-16T09:46:32.540419+0000 (oldest deadline 2024-09-16T09:46:56.540676+0000) 2024-09-16T09:47:03.652 INFO:journalctl@ceph.osd.6.smithi084.stdout:Sep 16 09:47:03 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-6[49407]: 2024-09-16T09:47:03.389+0000 7f91520d2640 -1 osd.6 573 heartbeat_check: no reply from 172.21.15.70:6822 osd.2 since back 2024-09-16T09:46:36.541325+0000 front 2024-09-16T09:46:36.541385+0000 (oldest deadline 2024-09-16T09:47:01.841229+0000) 2024-09-16T09:47:03.910 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:47:03 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:47:03.649+0000 7fa94a65f640 -1 osd.7 573 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-09-16T09:46:23.143051+0000 front 2024-09-16T09:46:23.142814+0000 (oldest deadline 2024-09-16T09:46:47.842666+0000) 2024-09-16T09:47:03.910 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:47:03 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:47:03.649+0000 7fa94a65f640 -1 osd.7 573 heartbeat_check: no reply from 172.21.15.70:6814 osd.1 since back 2024-09-16T09:46:33.043837+0000 front 2024-09-16T09:46:33.044046+0000 (oldest deadline 2024-09-16T09:46:57.744009+0000) 2024-09-16T09:47:04.654 INFO:journalctl@ceph.osd.5.smithi084.stdout:Sep 16 09:47:04 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-5[43256]: 2024-09-16T09:47:04.339+0000 7f3947661640 -1 osd.5 573 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-09-16T09:46:23.475156+0000 front 2024-09-16T09:46:23.475074+0000 (oldest deadline 2024-09-16T09:46:45.174934+0000) 2024-09-16T09:47:04.655 INFO:journalctl@ceph.osd.5.smithi084.stdout:Sep 16 09:47:04 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-5[43256]: 2024-09-16T09:47:04.339+0000 7f3947661640 -1 osd.5 573 heartbeat_check: no reply from 172.21.15.70:6814 osd.1 since back 2024-09-16T09:46:32.676604+0000 front 2024-09-16T09:46:32.676542+0000 (oldest deadline 2024-09-16T09:46:54.376526+0000) 2024-09-16T09:47:04.655 INFO:journalctl@ceph.osd.5.smithi084.stdout:Sep 16 09:47:04 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-5[43256]: 2024-09-16T09:47:04.339+0000 7f3947661640 -1 osd.5 573 heartbeat_check: no reply from 172.21.15.70:6822 osd.2 since back 2024-09-16T09:46:38.477653+0000 front 2024-09-16T09:46:38.477765+0000 (oldest deadline 2024-09-16T09:47:01.977603+0000) 2024-09-16T09:47:04.655 INFO:journalctl@ceph.osd.6.smithi084.stdout:Sep 16 09:47:04 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-6[49407]: 2024-09-16T09:47:04.396+0000 7f91520d2640 -1 osd.6 573 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-09-16T09:46:23.838986+0000 front 2024-09-16T09:46:23.839077+0000 (oldest deadline 2024-09-16T09:46:46.139011+0000) 2024-09-16T09:47:04.655 INFO:journalctl@ceph.osd.6.smithi084.stdout:Sep 16 09:47:04 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-6[49407]: 2024-09-16T09:47:04.396+0000 7f91520d2640 -1 osd.6 573 heartbeat_check: no reply from 172.21.15.70:6814 osd.1 since back 2024-09-16T09:46:32.540265+0000 front 2024-09-16T09:46:32.540419+0000 (oldest deadline 2024-09-16T09:46:56.540676+0000) 2024-09-16T09:47:04.655 INFO:journalctl@ceph.osd.6.smithi084.stdout:Sep 16 09:47:04 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-6[49407]: 2024-09-16T09:47:04.396+0000 7f91520d2640 -1 osd.6 573 heartbeat_check: no reply from 172.21.15.70:6822 osd.2 since back 2024-09-16T09:46:36.541325+0000 front 2024-09-16T09:46:36.541385+0000 (oldest deadline 2024-09-16T09:47:01.841229+0000) 2024-09-16T09:47:04.910 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:47:04 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:47:04.653+0000 7fa94a65f640 -1 osd.7 573 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-09-16T09:46:23.143051+0000 front 2024-09-16T09:46:23.142814+0000 (oldest deadline 2024-09-16T09:46:47.842666+0000) 2024-09-16T09:47:04.910 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:47:04 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:47:04.653+0000 7fa94a65f640 -1 osd.7 573 heartbeat_check: no reply from 172.21.15.70:6814 osd.1 since back 2024-09-16T09:46:33.043837+0000 front 2024-09-16T09:46:33.044046+0000 (oldest deadline 2024-09-16T09:46:57.744009+0000) 2024-09-16T09:47:04.911 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:47:04 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:47:04.653+0000 7fa94a65f640 -1 osd.7 573 heartbeat_check: no reply from 172.21.15.70:6822 osd.2 since back 2024-09-16T09:46:41.145642+0000 front 2024-09-16T09:46:41.145676+0000 (oldest deadline 2024-09-16T09:47:04.045530+0000) 2024-09-16T09:47:05.638 INFO:journalctl@ceph.osd.5.smithi084.stdout:Sep 16 09:47:05 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-5[43256]: 2024-09-16T09:47:05.313+0000 7f3947661640 -1 osd.5 573 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-09-16T09:46:23.475156+0000 front 2024-09-16T09:46:23.475074+0000 (oldest deadline 2024-09-16T09:46:45.174934+0000) 2024-09-16T09:47:05.639 INFO:journalctl@ceph.osd.5.smithi084.stdout:Sep 16 09:47:05 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-5[43256]: 2024-09-16T09:47:05.313+0000 7f3947661640 -1 osd.5 573 heartbeat_check: no reply from 172.21.15.70:6814 osd.1 since back 2024-09-16T09:46:32.676604+0000 front 2024-09-16T09:46:32.676542+0000 (oldest deadline 2024-09-16T09:46:54.376526+0000) 2024-09-16T09:47:05.639 INFO:journalctl@ceph.osd.5.smithi084.stdout:Sep 16 09:47:05 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-5[43256]: 2024-09-16T09:47:05.313+0000 7f3947661640 -1 osd.5 573 heartbeat_check: no reply from 172.21.15.70:6822 osd.2 since back 2024-09-16T09:46:38.477653+0000 front 2024-09-16T09:46:38.477765+0000 (oldest deadline 2024-09-16T09:47:01.977603+0000) 2024-09-16T09:47:05.639 INFO:journalctl@ceph.osd.6.smithi084.stdout:Sep 16 09:47:05 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-6[49407]: 2024-09-16T09:47:05.385+0000 7f91520d2640 -1 osd.6 573 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-09-16T09:46:23.838986+0000 front 2024-09-16T09:46:23.839077+0000 (oldest deadline 2024-09-16T09:46:46.139011+0000) 2024-09-16T09:47:05.639 INFO:journalctl@ceph.osd.6.smithi084.stdout:Sep 16 09:47:05 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-6[49407]: 2024-09-16T09:47:05.385+0000 7f91520d2640 -1 osd.6 573 heartbeat_check: no reply from 172.21.15.70:6814 osd.1 since back 2024-09-16T09:46:32.540265+0000 front 2024-09-16T09:46:32.540419+0000 (oldest deadline 2024-09-16T09:46:56.540676+0000) 2024-09-16T09:47:05.639 INFO:journalctl@ceph.osd.6.smithi084.stdout:Sep 16 09:47:05 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-6[49407]: 2024-09-16T09:47:05.385+0000 7f91520d2640 -1 osd.6 573 heartbeat_check: no reply from 172.21.15.70:6822 osd.2 since back 2024-09-16T09:46:36.541325+0000 front 2024-09-16T09:46:36.541385+0000 (oldest deadline 2024-09-16T09:47:01.841229+0000) 2024-09-16T09:47:05.910 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:47:05 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:47:05.636+0000 7fa94a65f640 -1 osd.7 573 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-09-16T09:46:23.143051+0000 front 2024-09-16T09:46:23.142814+0000 (oldest deadline 2024-09-16T09:46:47.842666+0000) 2024-09-16T09:47:05.910 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:47:05 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:47:05.636+0000 7fa94a65f640 -1 osd.7 573 heartbeat_check: no reply from 172.21.15.70:6814 osd.1 since back 2024-09-16T09:46:33.043837+0000 front 2024-09-16T09:46:33.044046+0000 (oldest deadline 2024-09-16T09:46:57.744009+0000) 2024-09-16T09:47:05.910 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:47:05 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:47:05.636+0000 7fa94a65f640 -1 osd.7 573 heartbeat_check: no reply from 172.21.15.70:6822 osd.2 since back 2024-09-16T09:46:41.145642+0000 front 2024-09-16T09:46:41.145676+0000 (oldest deadline 2024-09-16T09:47:04.045530+0000) 2024-09-16T09:47:06.238 INFO:journalctl@ceph.osd.5.smithi084.stdout:Sep 16 09:47:05 smithi084 podman[154041]: 2024-09-16 09:47:05.972773349 +0000 UTC m=+5.595034591 container died 35ef92d7201705d246d006fc5dcfeaeac35a17886740d2d444dd764ac01dace5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, GIT_CLEAN=True) 2024-09-16T09:47:06.588 INFO:journalctl@ceph.osd.5.smithi084.stdout:Sep 16 09:47:06 smithi084 podman[154041]: 2024-09-16 09:47:06.335884174 +0000 UTC m=+5.958145405 container cleanup 35ef92d7201705d246d006fc5dcfeaeac35a17886740d2d444dd764ac01dace5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-5, GIT_CLEAN=True, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True) 2024-09-16T09:47:06.588 INFO:journalctl@ceph.osd.5.smithi084.stdout:Sep 16 09:47:06 smithi084 bash[154041]: ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-5 2024-09-16T09:47:06.589 INFO:journalctl@ceph.osd.6.smithi084.stdout:Sep 16 09:47:06 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-6[49407]: 2024-09-16T09:47:06.344+0000 7f91520d2640 -1 osd.6 573 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-09-16T09:46:23.838986+0000 front 2024-09-16T09:46:23.839077+0000 (oldest deadline 2024-09-16T09:46:46.139011+0000) 2024-09-16T09:47:06.589 INFO:journalctl@ceph.osd.6.smithi084.stdout:Sep 16 09:47:06 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-6[49407]: 2024-09-16T09:47:06.344+0000 7f91520d2640 -1 osd.6 573 heartbeat_check: no reply from 172.21.15.70:6814 osd.1 since back 2024-09-16T09:46:32.540265+0000 front 2024-09-16T09:46:32.540419+0000 (oldest deadline 2024-09-16T09:46:56.540676+0000) 2024-09-16T09:47:06.589 INFO:journalctl@ceph.osd.6.smithi084.stdout:Sep 16 09:47:06 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-6[49407]: 2024-09-16T09:47:06.344+0000 7f91520d2640 -1 osd.6 573 heartbeat_check: no reply from 172.21.15.70:6822 osd.2 since back 2024-09-16T09:46:36.541325+0000 front 2024-09-16T09:46:36.541385+0000 (oldest deadline 2024-09-16T09:47:01.841229+0000) 2024-09-16T09:47:06.898 INFO:journalctl@ceph.osd.5.smithi084.stdout:Sep 16 09:47:06 smithi084 podman[154061]: 2024-09-16 09:47:06.587171269 +0000 UTC m=+0.612821243 container remove 35ef92d7201705d246d006fc5dcfeaeac35a17886740d2d444dd764ac01dace5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-5, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.build-date=20240731, ceph=True) 2024-09-16T09:47:06.898 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:47:06 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:47:06.629+0000 7fa94a65f640 -1 osd.7 573 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-09-16T09:46:23.143051+0000 front 2024-09-16T09:46:23.142814+0000 (oldest deadline 2024-09-16T09:46:47.842666+0000) 2024-09-16T09:47:06.898 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:47:06 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:47:06.629+0000 7fa94a65f640 -1 osd.7 573 heartbeat_check: no reply from 172.21.15.70:6814 osd.1 since back 2024-09-16T09:46:33.043837+0000 front 2024-09-16T09:46:33.044046+0000 (oldest deadline 2024-09-16T09:46:57.744009+0000) 2024-09-16T09:47:06.898 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:47:06 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:47:06.629+0000 7fa94a65f640 -1 osd.7 573 heartbeat_check: no reply from 172.21.15.70:6822 osd.2 since back 2024-09-16T09:46:41.145642+0000 front 2024-09-16T09:46:41.145676+0000 (oldest deadline 2024-09-16T09:47:04.045530+0000) 2024-09-16T09:47:07.160 INFO:journalctl@ceph.osd.5.smithi084.stdout:Sep 16 09:47:06 smithi084 podman[154157]: 2024-09-16 09:47:06.800145097 +0000 UTC m=+0.016171463 image pull 425e316ba8ebf74f04aa3640f7deb4fa3f293a78d0c8d097cb679eaf86ddca6c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce 2024-09-16T09:47:07.160 INFO:journalctl@ceph.osd.5.smithi084.stdout:Sep 16 09:47:06 smithi084 podman[154157]: 2024-09-16 09:47:06.929886165 +0000 UTC m=+0.145912534 container create e8d379a1af363a68b57ecca1752e0d8b9176366fb03aea398f672359e3a5e1a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True) 2024-09-16T09:47:07.654 INFO:journalctl@ceph.osd.6.smithi084.stdout:Sep 16 09:47:07 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-6[49407]: 2024-09-16T09:47:07.351+0000 7f91520d2640 -1 osd.6 573 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-09-16T09:46:23.838986+0000 front 2024-09-16T09:46:23.839077+0000 (oldest deadline 2024-09-16T09:46:46.139011+0000) 2024-09-16T09:47:07.654 INFO:journalctl@ceph.osd.6.smithi084.stdout:Sep 16 09:47:07 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-6[49407]: 2024-09-16T09:47:07.351+0000 7f91520d2640 -1 osd.6 573 heartbeat_check: no reply from 172.21.15.70:6814 osd.1 since back 2024-09-16T09:46:32.540265+0000 front 2024-09-16T09:46:32.540419+0000 (oldest deadline 2024-09-16T09:46:56.540676+0000) 2024-09-16T09:47:07.654 INFO:journalctl@ceph.osd.6.smithi084.stdout:Sep 16 09:47:07 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-6[49407]: 2024-09-16T09:47:07.351+0000 7f91520d2640 -1 osd.6 573 heartbeat_check: no reply from 172.21.15.70:6822 osd.2 since back 2024-09-16T09:46:36.541325+0000 front 2024-09-16T09:46:36.541385+0000 (oldest deadline 2024-09-16T09:47:01.841229+0000) 2024-09-16T09:47:07.655 INFO:journalctl@ceph.osd.5.smithi084.stdout:Sep 16 09:47:07 smithi084 podman[154157]: 2024-09-16 09:47:07.165725188 +0000 UTC m=+0.381751555 container init e8d379a1af363a68b57ecca1752e0d8b9176366fb03aea398f672359e3a5e1a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-5-deactivate, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True) 2024-09-16T09:47:07.655 INFO:journalctl@ceph.osd.5.smithi084.stdout:Sep 16 09:47:07 smithi084 podman[154157]: 2024-09-16 09:47:07.170535586 +0000 UTC m=+0.386561953 container start e8d379a1af363a68b57ecca1752e0d8b9176366fb03aea398f672359e3a5e1a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731) 2024-09-16T09:47:07.655 INFO:journalctl@ceph.osd.5.smithi084.stdout:Sep 16 09:47:07 smithi084 podman[154157]: 2024-09-16 09:47:07.248961477 +0000 UTC m=+0.464987845 container attach e8d379a1af363a68b57ecca1752e0d8b9176366fb03aea398f672359e3a5e1a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-16T09:47:07.655 INFO:journalctl@ceph.osd.5.smithi084.stdout:Sep 16 09:47:07 smithi084 podman[154157]: 2024-09-16 09:47:07.355436401 +0000 UTC m=+0.571462776 container died e8d379a1af363a68b57ecca1752e0d8b9176366fb03aea398f672359e3a5e1a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-16T09:47:07.910 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:47:07 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:47:07.651+0000 7fa94a65f640 -1 osd.7 573 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-09-16T09:46:23.143051+0000 front 2024-09-16T09:46:23.142814+0000 (oldest deadline 2024-09-16T09:46:47.842666+0000) 2024-09-16T09:47:07.910 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:47:07 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:47:07.651+0000 7fa94a65f640 -1 osd.7 573 heartbeat_check: no reply from 172.21.15.70:6814 osd.1 since back 2024-09-16T09:46:33.043837+0000 front 2024-09-16T09:46:33.044046+0000 (oldest deadline 2024-09-16T09:46:57.744009+0000) 2024-09-16T09:47:07.910 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:47:07 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:47:07.651+0000 7fa94a65f640 -1 osd.7 573 heartbeat_check: no reply from 172.21.15.70:6822 osd.2 since back 2024-09-16T09:46:41.145642+0000 front 2024-09-16T09:46:41.145676+0000 (oldest deadline 2024-09-16T09:47:04.045530+0000) 2024-09-16T09:47:07.910 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:47:07 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:47:07.651+0000 7fa94a65f640 -1 osd.7 573 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24424.0:1282 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e573) 2024-09-16T09:47:08.048 DEBUG:teuthology.orchestra.run.smithi084:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a1321834-740c-11ef-bceb-c7b262605968@osd.5.service' 2024-09-16T09:47:08.090 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T09:47:08.091 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-16T09:47:08.091 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-16T09:47:08.091 DEBUG:teuthology.orchestra.run.smithi084:> sudo systemctl stop ceph-a1321834-740c-11ef-bceb-c7b262605968@osd.6 2024-09-16T09:47:08.381 INFO:journalctl@ceph.osd.6.smithi084.stdout:Sep 16 09:47:08 smithi084 systemd[1]: Stopping Ceph osd.6 for a1321834-740c-11ef-bceb-c7b262605968... 2024-09-16T09:47:08.660 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:47:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:47:08.625+0000 7fa94a65f640 -1 osd.7 573 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-09-16T09:46:23.143051+0000 front 2024-09-16T09:46:23.142814+0000 (oldest deadline 2024-09-16T09:46:47.842666+0000) 2024-09-16T09:47:08.661 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:47:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:47:08.625+0000 7fa94a65f640 -1 osd.7 573 heartbeat_check: no reply from 172.21.15.70:6814 osd.1 since back 2024-09-16T09:46:33.043837+0000 front 2024-09-16T09:46:33.044046+0000 (oldest deadline 2024-09-16T09:46:57.744009+0000) 2024-09-16T09:47:08.661 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:47:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:47:08.625+0000 7fa94a65f640 -1 osd.7 573 heartbeat_check: no reply from 172.21.15.70:6822 osd.2 since back 2024-09-16T09:46:41.145642+0000 front 2024-09-16T09:46:41.145676+0000 (oldest deadline 2024-09-16T09:47:04.045530+0000) 2024-09-16T09:47:08.661 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:47:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:47:08.625+0000 7fa94a65f640 -1 osd.7 573 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24424.0:1282 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e573) 2024-09-16T09:47:08.661 INFO:journalctl@ceph.osd.6.smithi084.stdout:Sep 16 09:47:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-6[49407]: 2024-09-16T09:47:08.379+0000 7f91520d2640 -1 osd.6 573 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-09-16T09:46:23.838986+0000 front 2024-09-16T09:46:23.839077+0000 (oldest deadline 2024-09-16T09:46:46.139011+0000) 2024-09-16T09:47:08.661 INFO:journalctl@ceph.osd.6.smithi084.stdout:Sep 16 09:47:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-6[49407]: 2024-09-16T09:47:08.379+0000 7f91520d2640 -1 osd.6 573 heartbeat_check: no reply from 172.21.15.70:6814 osd.1 since back 2024-09-16T09:46:32.540265+0000 front 2024-09-16T09:46:32.540419+0000 (oldest deadline 2024-09-16T09:46:56.540676+0000) 2024-09-16T09:47:08.661 INFO:journalctl@ceph.osd.6.smithi084.stdout:Sep 16 09:47:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-6[49407]: 2024-09-16T09:47:08.379+0000 7f91520d2640 -1 osd.6 573 heartbeat_check: no reply from 172.21.15.70:6822 osd.2 since back 2024-09-16T09:46:36.541325+0000 front 2024-09-16T09:46:36.541385+0000 (oldest deadline 2024-09-16T09:47:01.841229+0000) 2024-09-16T09:47:08.661 INFO:journalctl@ceph.osd.6.smithi084.stdout:Sep 16 09:47:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-6[49407]: 2024-09-16T09:47:08.466+0000 7f9155abd640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T09:47:08.661 INFO:journalctl@ceph.osd.6.smithi084.stdout:Sep 16 09:47:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-6[49407]: 2024-09-16T09:47:08.466+0000 7f9155abd640 -1 osd.6 573 *** Got signal Terminated *** 2024-09-16T09:47:08.661 INFO:journalctl@ceph.osd.6.smithi084.stdout:Sep 16 09:47:08 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-6[49407]: 2024-09-16T09:47:08.466+0000 7f9155abd640 -1 osd.6 573 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T09:47:09.910 INFO:journalctl@ceph.osd.6.smithi084.stdout:Sep 16 09:47:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-6[49407]: 2024-09-16T09:47:09.428+0000 7f91520d2640 -1 osd.6 573 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-09-16T09:46:23.838986+0000 front 2024-09-16T09:46:23.839077+0000 (oldest deadline 2024-09-16T09:46:46.139011+0000) 2024-09-16T09:47:09.911 INFO:journalctl@ceph.osd.6.smithi084.stdout:Sep 16 09:47:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-6[49407]: 2024-09-16T09:47:09.428+0000 7f91520d2640 -1 osd.6 573 heartbeat_check: no reply from 172.21.15.70:6814 osd.1 since back 2024-09-16T09:46:32.540265+0000 front 2024-09-16T09:46:32.540419+0000 (oldest deadline 2024-09-16T09:46:56.540676+0000) 2024-09-16T09:47:09.911 INFO:journalctl@ceph.osd.6.smithi084.stdout:Sep 16 09:47:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-6[49407]: 2024-09-16T09:47:09.428+0000 7f91520d2640 -1 osd.6 573 heartbeat_check: no reply from 172.21.15.70:6822 osd.2 since back 2024-09-16T09:46:36.541325+0000 front 2024-09-16T09:46:36.541385+0000 (oldest deadline 2024-09-16T09:47:01.841229+0000) 2024-09-16T09:47:09.911 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:47:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:47:09.674+0000 7fa94a65f640 -1 osd.7 573 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-09-16T09:46:23.143051+0000 front 2024-09-16T09:46:23.142814+0000 (oldest deadline 2024-09-16T09:46:47.842666+0000) 2024-09-16T09:47:09.911 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:47:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:47:09.675+0000 7fa94a65f640 -1 osd.7 573 heartbeat_check: no reply from 172.21.15.70:6814 osd.1 since back 2024-09-16T09:46:33.043837+0000 front 2024-09-16T09:46:33.044046+0000 (oldest deadline 2024-09-16T09:46:57.744009+0000) 2024-09-16T09:47:09.911 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:47:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:47:09.675+0000 7fa94a65f640 -1 osd.7 573 heartbeat_check: no reply from 172.21.15.70:6822 osd.2 since back 2024-09-16T09:46:41.145642+0000 front 2024-09-16T09:46:41.145676+0000 (oldest deadline 2024-09-16T09:47:04.045530+0000) 2024-09-16T09:47:09.911 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:47:09 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:47:09.675+0000 7fa94a65f640 -1 osd.7 573 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24424.0:1282 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e573) 2024-09-16T09:47:10.910 INFO:journalctl@ceph.osd.6.smithi084.stdout:Sep 16 09:47:10 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-6[49407]: 2024-09-16T09:47:10.452+0000 7f91520d2640 -1 osd.6 573 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-09-16T09:46:23.838986+0000 front 2024-09-16T09:46:23.839077+0000 (oldest deadline 2024-09-16T09:46:46.139011+0000) 2024-09-16T09:47:10.911 INFO:journalctl@ceph.osd.6.smithi084.stdout:Sep 16 09:47:10 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-6[49407]: 2024-09-16T09:47:10.452+0000 7f91520d2640 -1 osd.6 573 heartbeat_check: no reply from 172.21.15.70:6814 osd.1 since back 2024-09-16T09:46:32.540265+0000 front 2024-09-16T09:46:32.540419+0000 (oldest deadline 2024-09-16T09:46:56.540676+0000) 2024-09-16T09:47:10.911 INFO:journalctl@ceph.osd.6.smithi084.stdout:Sep 16 09:47:10 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-6[49407]: 2024-09-16T09:47:10.452+0000 7f91520d2640 -1 osd.6 573 heartbeat_check: no reply from 172.21.15.70:6822 osd.2 since back 2024-09-16T09:46:36.541325+0000 front 2024-09-16T09:46:36.541385+0000 (oldest deadline 2024-09-16T09:47:01.841229+0000) 2024-09-16T09:47:10.911 INFO:journalctl@ceph.osd.6.smithi084.stdout:Sep 16 09:47:10 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-6[49407]: 2024-09-16T09:47:10.452+0000 7f91520d2640 -1 osd.6 573 heartbeat_check: no reply from 172.21.15.70:6830 osd.3 since back 2024-09-16T09:46:48.743038+0000 front 2024-09-16T09:46:48.743005+0000 (oldest deadline 2024-09-16T09:47:09.842949+0000) 2024-09-16T09:47:10.911 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:47:10 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:47:10.646+0000 7fa94a65f640 -1 osd.7 573 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-09-16T09:46:23.143051+0000 front 2024-09-16T09:46:23.142814+0000 (oldest deadline 2024-09-16T09:46:47.842666+0000) 2024-09-16T09:47:10.911 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:47:10 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:47:10.646+0000 7fa94a65f640 -1 osd.7 573 heartbeat_check: no reply from 172.21.15.70:6814 osd.1 since back 2024-09-16T09:46:33.043837+0000 front 2024-09-16T09:46:33.044046+0000 (oldest deadline 2024-09-16T09:46:57.744009+0000) 2024-09-16T09:47:10.911 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:47:10 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:47:10.646+0000 7fa94a65f640 -1 osd.7 573 heartbeat_check: no reply from 172.21.15.70:6822 osd.2 since back 2024-09-16T09:46:41.145642+0000 front 2024-09-16T09:46:41.145676+0000 (oldest deadline 2024-09-16T09:47:04.045530+0000) 2024-09-16T09:47:10.911 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:47:10 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:47:10.646+0000 7fa94a65f640 -1 osd.7 573 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24424.0:1282 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e573) 2024-09-16T09:47:11.910 INFO:journalctl@ceph.osd.6.smithi084.stdout:Sep 16 09:47:11 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-6[49407]: 2024-09-16T09:47:11.453+0000 7f91520d2640 -1 osd.6 573 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-09-16T09:46:23.838986+0000 front 2024-09-16T09:46:23.839077+0000 (oldest deadline 2024-09-16T09:46:46.139011+0000) 2024-09-16T09:47:11.910 INFO:journalctl@ceph.osd.6.smithi084.stdout:Sep 16 09:47:11 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-6[49407]: 2024-09-16T09:47:11.453+0000 7f91520d2640 -1 osd.6 573 heartbeat_check: no reply from 172.21.15.70:6814 osd.1 since back 2024-09-16T09:46:32.540265+0000 front 2024-09-16T09:46:32.540419+0000 (oldest deadline 2024-09-16T09:46:56.540676+0000) 2024-09-16T09:47:11.910 INFO:journalctl@ceph.osd.6.smithi084.stdout:Sep 16 09:47:11 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-6[49407]: 2024-09-16T09:47:11.453+0000 7f91520d2640 -1 osd.6 573 heartbeat_check: no reply from 172.21.15.70:6822 osd.2 since back 2024-09-16T09:46:36.541325+0000 front 2024-09-16T09:46:36.541385+0000 (oldest deadline 2024-09-16T09:47:01.841229+0000) 2024-09-16T09:47:11.910 INFO:journalctl@ceph.osd.6.smithi084.stdout:Sep 16 09:47:11 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-6[49407]: 2024-09-16T09:47:11.453+0000 7f91520d2640 -1 osd.6 573 heartbeat_check: no reply from 172.21.15.70:6830 osd.3 since back 2024-09-16T09:46:48.743038+0000 front 2024-09-16T09:46:48.743005+0000 (oldest deadline 2024-09-16T09:47:09.842949+0000) 2024-09-16T09:47:11.911 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:47:11 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:47:11.687+0000 7fa94a65f640 -1 osd.7 573 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-09-16T09:46:23.143051+0000 front 2024-09-16T09:46:23.142814+0000 (oldest deadline 2024-09-16T09:46:47.842666+0000) 2024-09-16T09:47:11.911 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:47:11 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:47:11.687+0000 7fa94a65f640 -1 osd.7 573 heartbeat_check: no reply from 172.21.15.70:6814 osd.1 since back 2024-09-16T09:46:33.043837+0000 front 2024-09-16T09:46:33.044046+0000 (oldest deadline 2024-09-16T09:46:57.744009+0000) 2024-09-16T09:47:11.911 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:47:11 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:47:11.687+0000 7fa94a65f640 -1 osd.7 573 heartbeat_check: no reply from 172.21.15.70:6822 osd.2 since back 2024-09-16T09:46:41.145642+0000 front 2024-09-16T09:46:41.145676+0000 (oldest deadline 2024-09-16T09:47:04.045530+0000) 2024-09-16T09:47:11.911 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:47:11 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:47:11.687+0000 7fa94a65f640 -1 osd.7 573 heartbeat_check: no reply from 172.21.15.70:6830 osd.3 since back 2024-09-16T09:46:48.047046+0000 front 2024-09-16T09:46:48.046967+0000 (oldest deadline 2024-09-16T09:47:10.946873+0000) 2024-09-16T09:47:11.911 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:47:11 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:47:11.687+0000 7fa94a65f640 -1 osd.7 573 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24424.0:1282 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e573) 2024-09-16T09:47:12.911 INFO:journalctl@ceph.osd.6.smithi084.stdout:Sep 16 09:47:12 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-6[49407]: 2024-09-16T09:47:12.476+0000 7f91520d2640 -1 osd.6 573 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-09-16T09:46:23.838986+0000 front 2024-09-16T09:46:23.839077+0000 (oldest deadline 2024-09-16T09:46:46.139011+0000) 2024-09-16T09:47:12.911 INFO:journalctl@ceph.osd.6.smithi084.stdout:Sep 16 09:47:12 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-6[49407]: 2024-09-16T09:47:12.476+0000 7f91520d2640 -1 osd.6 573 heartbeat_check: no reply from 172.21.15.70:6814 osd.1 since back 2024-09-16T09:46:32.540265+0000 front 2024-09-16T09:46:32.540419+0000 (oldest deadline 2024-09-16T09:46:56.540676+0000) 2024-09-16T09:47:12.911 INFO:journalctl@ceph.osd.6.smithi084.stdout:Sep 16 09:47:12 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-6[49407]: 2024-09-16T09:47:12.476+0000 7f91520d2640 -1 osd.6 573 heartbeat_check: no reply from 172.21.15.70:6822 osd.2 since back 2024-09-16T09:46:36.541325+0000 front 2024-09-16T09:46:36.541385+0000 (oldest deadline 2024-09-16T09:47:01.841229+0000) 2024-09-16T09:47:12.911 INFO:journalctl@ceph.osd.6.smithi084.stdout:Sep 16 09:47:12 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-6[49407]: 2024-09-16T09:47:12.476+0000 7f91520d2640 -1 osd.6 573 heartbeat_check: no reply from 172.21.15.70:6830 osd.3 since back 2024-09-16T09:46:48.743038+0000 front 2024-09-16T09:46:48.743005+0000 (oldest deadline 2024-09-16T09:47:09.842949+0000) 2024-09-16T09:47:12.911 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:47:12 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:47:12.696+0000 7fa94a65f640 -1 osd.7 573 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-09-16T09:46:23.143051+0000 front 2024-09-16T09:46:23.142814+0000 (oldest deadline 2024-09-16T09:46:47.842666+0000) 2024-09-16T09:47:12.911 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:47:12 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:47:12.696+0000 7fa94a65f640 -1 osd.7 573 heartbeat_check: no reply from 172.21.15.70:6814 osd.1 since back 2024-09-16T09:46:33.043837+0000 front 2024-09-16T09:46:33.044046+0000 (oldest deadline 2024-09-16T09:46:57.744009+0000) 2024-09-16T09:47:12.912 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:47:12 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:47:12.696+0000 7fa94a65f640 -1 osd.7 573 heartbeat_check: no reply from 172.21.15.70:6822 osd.2 since back 2024-09-16T09:46:41.145642+0000 front 2024-09-16T09:46:41.145676+0000 (oldest deadline 2024-09-16T09:47:04.045530+0000) 2024-09-16T09:47:12.912 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:47:12 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:47:12.696+0000 7fa94a65f640 -1 osd.7 573 heartbeat_check: no reply from 172.21.15.70:6830 osd.3 since back 2024-09-16T09:46:48.047046+0000 front 2024-09-16T09:46:48.046967+0000 (oldest deadline 2024-09-16T09:47:10.946873+0000) 2024-09-16T09:47:12.912 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:47:12 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:47:12.696+0000 7fa94a65f640 -1 osd.7 573 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24424.0:1282 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e573) 2024-09-16T09:47:13.910 INFO:journalctl@ceph.osd.6.smithi084.stdout:Sep 16 09:47:13 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-6[49407]: 2024-09-16T09:47:13.440+0000 7f91520d2640 -1 osd.6 573 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-09-16T09:46:23.838986+0000 front 2024-09-16T09:46:23.839077+0000 (oldest deadline 2024-09-16T09:46:46.139011+0000) 2024-09-16T09:47:13.910 INFO:journalctl@ceph.osd.6.smithi084.stdout:Sep 16 09:47:13 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-6[49407]: 2024-09-16T09:47:13.440+0000 7f91520d2640 -1 osd.6 573 heartbeat_check: no reply from 172.21.15.70:6814 osd.1 since back 2024-09-16T09:46:32.540265+0000 front 2024-09-16T09:46:32.540419+0000 (oldest deadline 2024-09-16T09:46:56.540676+0000) 2024-09-16T09:47:13.910 INFO:journalctl@ceph.osd.6.smithi084.stdout:Sep 16 09:47:13 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-6[49407]: 2024-09-16T09:47:13.440+0000 7f91520d2640 -1 osd.6 573 heartbeat_check: no reply from 172.21.15.70:6822 osd.2 since back 2024-09-16T09:46:36.541325+0000 front 2024-09-16T09:46:36.541385+0000 (oldest deadline 2024-09-16T09:47:01.841229+0000) 2024-09-16T09:47:13.910 INFO:journalctl@ceph.osd.6.smithi084.stdout:Sep 16 09:47:13 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-6[49407]: 2024-09-16T09:47:13.440+0000 7f91520d2640 -1 osd.6 573 heartbeat_check: no reply from 172.21.15.70:6830 osd.3 since back 2024-09-16T09:46:48.743038+0000 front 2024-09-16T09:46:48.743005+0000 (oldest deadline 2024-09-16T09:47:09.842949+0000) 2024-09-16T09:47:13.911 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:47:13 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:47:13.684+0000 7fa94a65f640 -1 osd.7 573 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-09-16T09:46:23.143051+0000 front 2024-09-16T09:46:23.142814+0000 (oldest deadline 2024-09-16T09:46:47.842666+0000) 2024-09-16T09:47:13.911 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:47:13 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:47:13.684+0000 7fa94a65f640 -1 osd.7 573 heartbeat_check: no reply from 172.21.15.70:6814 osd.1 since back 2024-09-16T09:46:33.043837+0000 front 2024-09-16T09:46:33.044046+0000 (oldest deadline 2024-09-16T09:46:57.744009+0000) 2024-09-16T09:47:13.911 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:47:13 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:47:13.684+0000 7fa94a65f640 -1 osd.7 573 heartbeat_check: no reply from 172.21.15.70:6822 osd.2 since back 2024-09-16T09:46:41.145642+0000 front 2024-09-16T09:46:41.145676+0000 (oldest deadline 2024-09-16T09:47:04.045530+0000) 2024-09-16T09:47:13.911 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:47:13 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:47:13.684+0000 7fa94a65f640 -1 osd.7 573 heartbeat_check: no reply from 172.21.15.70:6830 osd.3 since back 2024-09-16T09:46:48.047046+0000 front 2024-09-16T09:46:48.046967+0000 (oldest deadline 2024-09-16T09:47:10.946873+0000) 2024-09-16T09:47:13.911 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:47:13 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:47:13.684+0000 7fa94a65f640 -1 osd.7 573 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24424.0:1282 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e573) 2024-09-16T09:47:14.273 INFO:journalctl@ceph.osd.6.smithi084.stdout:Sep 16 09:47:14 smithi084 podman[154286]: 2024-09-16 09:47:14.012377242 +0000 UTC m=+5.700964665 container died 47a52e2cfc7e3222cd37f8cd5df4025d0143ed8848b811435951d6964fa090d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-6, org.label-schema.vendor=CentOS, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T09:47:14.641 INFO:journalctl@ceph.osd.6.smithi084.stdout:Sep 16 09:47:14 smithi084 podman[154286]: 2024-09-16 09:47:14.44687375 +0000 UTC m=+6.135461172 container cleanup 47a52e2cfc7e3222cd37f8cd5df4025d0143ed8848b811435951d6964fa090d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-6, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-guits-main-2024-09-13-1248-8293d73, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True) 2024-09-16T09:47:14.641 INFO:journalctl@ceph.osd.6.smithi084.stdout:Sep 16 09:47:14 smithi084 bash[154286]: ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-6 2024-09-16T09:47:14.910 INFO:journalctl@ceph.osd.6.smithi084.stdout:Sep 16 09:47:14 smithi084 podman[154313]: 2024-09-16 09:47:14.640227259 +0000 UTC m=+0.629573358 container remove 47a52e2cfc7e3222cd37f8cd5df4025d0143ed8848b811435951d6964fa090d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-6, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-16T09:47:14.910 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:47:14 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:47:14.681+0000 7fa94a65f640 -1 osd.7 573 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-09-16T09:46:23.143051+0000 front 2024-09-16T09:46:23.142814+0000 (oldest deadline 2024-09-16T09:46:47.842666+0000) 2024-09-16T09:47:14.910 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:47:14 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:47:14.681+0000 7fa94a65f640 -1 osd.7 573 heartbeat_check: no reply from 172.21.15.70:6814 osd.1 since back 2024-09-16T09:46:33.043837+0000 front 2024-09-16T09:46:33.044046+0000 (oldest deadline 2024-09-16T09:46:57.744009+0000) 2024-09-16T09:47:14.910 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:47:14 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:47:14.681+0000 7fa94a65f640 -1 osd.7 573 heartbeat_check: no reply from 172.21.15.70:6822 osd.2 since back 2024-09-16T09:46:41.145642+0000 front 2024-09-16T09:46:41.145676+0000 (oldest deadline 2024-09-16T09:47:04.045530+0000) 2024-09-16T09:47:14.910 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:47:14 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:47:14.681+0000 7fa94a65f640 -1 osd.7 573 heartbeat_check: no reply from 172.21.15.70:6830 osd.3 since back 2024-09-16T09:46:48.047046+0000 front 2024-09-16T09:46:48.046967+0000 (oldest deadline 2024-09-16T09:47:10.946873+0000) 2024-09-16T09:47:14.910 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:47:14 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:47:14.681+0000 7fa94a65f640 -1 osd.7 573 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24424.0:1282 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e573) 2024-09-16T09:47:15.313 INFO:journalctl@ceph.osd.6.smithi084.stdout:Sep 16 09:47:15 smithi084 podman[154405]: 2024-09-16 09:47:14.960617022 +0000 UTC m=+0.016945662 image pull 425e316ba8ebf74f04aa3640f7deb4fa3f293a78d0c8d097cb679eaf86ddca6c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce 2024-09-16T09:47:15.313 INFO:journalctl@ceph.osd.6.smithi084.stdout:Sep 16 09:47:15 smithi084 podman[154405]: 2024-09-16 09:47:15.066064709 +0000 UTC m=+0.122393345 container create 068e49048da4f9f51280296a3724ccdda96a67944a6880ef4229dac0a2ed391c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-6-deactivate, ceph=True, org.label-schema.build-date=20240731, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T09:47:15.635 INFO:journalctl@ceph.osd.6.smithi084.stdout:Sep 16 09:47:15 smithi084 podman[154405]: 2024-09-16 09:47:15.443393778 +0000 UTC m=+0.499722412 container init 068e49048da4f9f51280296a3724ccdda96a67944a6880ef4229dac0a2ed391c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-6-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-16T09:47:15.636 INFO:journalctl@ceph.osd.6.smithi084.stdout:Sep 16 09:47:15 smithi084 podman[154405]: 2024-09-16 09:47:15.44878343 +0000 UTC m=+0.505112065 container start 068e49048da4f9f51280296a3724ccdda96a67944a6880ef4229dac0a2ed391c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-16T09:47:15.636 INFO:journalctl@ceph.osd.6.smithi084.stdout:Sep 16 09:47:15 smithi084 podman[154405]: 2024-09-16 09:47:15.534744188 +0000 UTC m=+0.591072836 container attach 068e49048da4f9f51280296a3724ccdda96a67944a6880ef4229dac0a2ed391c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-6-deactivate, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-guits-main-2024-09-13-1248-8293d73, io.buildah.version=1.37.2) 2024-09-16T09:47:15.910 INFO:journalctl@ceph.osd.6.smithi084.stdout:Sep 16 09:47:15 smithi084 podman[154405]: 2024-09-16 09:47:15.635748024 +0000 UTC m=+0.692076666 container died 068e49048da4f9f51280296a3724ccdda96a67944a6880ef4229dac0a2ed391c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-6-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-16T09:47:15.910 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:47:15 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:47:15.672+0000 7fa94a65f640 -1 osd.7 573 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-09-16T09:46:23.143051+0000 front 2024-09-16T09:46:23.142814+0000 (oldest deadline 2024-09-16T09:46:47.842666+0000) 2024-09-16T09:47:15.910 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:47:15 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:47:15.672+0000 7fa94a65f640 -1 osd.7 573 heartbeat_check: no reply from 172.21.15.70:6814 osd.1 since back 2024-09-16T09:46:33.043837+0000 front 2024-09-16T09:46:33.044046+0000 (oldest deadline 2024-09-16T09:46:57.744009+0000) 2024-09-16T09:47:15.910 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:47:15 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:47:15.672+0000 7fa94a65f640 -1 osd.7 573 heartbeat_check: no reply from 172.21.15.70:6822 osd.2 since back 2024-09-16T09:46:41.145642+0000 front 2024-09-16T09:46:41.145676+0000 (oldest deadline 2024-09-16T09:47:04.045530+0000) 2024-09-16T09:47:15.910 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:47:15 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:47:15.672+0000 7fa94a65f640 -1 osd.7 573 heartbeat_check: no reply from 172.21.15.70:6830 osd.3 since back 2024-09-16T09:46:48.047046+0000 front 2024-09-16T09:46:48.046967+0000 (oldest deadline 2024-09-16T09:47:10.946873+0000) 2024-09-16T09:47:15.911 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:47:15 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:47:15.672+0000 7fa94a65f640 -1 osd.7 573 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24424.0:1282 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e573) 2024-09-16T09:47:16.241 DEBUG:teuthology.orchestra.run.smithi084:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a1321834-740c-11ef-bceb-c7b262605968@osd.6.service' 2024-09-16T09:47:16.280 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T09:47:16.281 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-16T09:47:16.281 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-16T09:47:16.281 DEBUG:teuthology.orchestra.run.smithi084:> sudo systemctl stop ceph-a1321834-740c-11ef-bceb-c7b262605968@osd.7 2024-09-16T09:47:16.620 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:47:16 smithi084 systemd[1]: Stopping Ceph osd.7 for a1321834-740c-11ef-bceb-c7b262605968... 2024-09-16T09:47:16.910 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:47:16 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:47:16.618+0000 7fa952052640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T09:47:16.910 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:47:16 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:47:16.618+0000 7fa952052640 -1 osd.7 573 *** Got signal Terminated *** 2024-09-16T09:47:16.910 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:47:16 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:47:16.618+0000 7fa952052640 -1 osd.7 573 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T09:47:16.910 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:47:16 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:47:16.696+0000 7fa94a65f640 -1 osd.7 573 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-09-16T09:46:23.143051+0000 front 2024-09-16T09:46:23.142814+0000 (oldest deadline 2024-09-16T09:46:47.842666+0000) 2024-09-16T09:47:16.910 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:47:16 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:47:16.696+0000 7fa94a65f640 -1 osd.7 573 heartbeat_check: no reply from 172.21.15.70:6814 osd.1 since back 2024-09-16T09:46:33.043837+0000 front 2024-09-16T09:46:33.044046+0000 (oldest deadline 2024-09-16T09:46:57.744009+0000) 2024-09-16T09:47:16.911 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:47:16 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:47:16.696+0000 7fa94a65f640 -1 osd.7 573 heartbeat_check: no reply from 172.21.15.70:6822 osd.2 since back 2024-09-16T09:46:41.145642+0000 front 2024-09-16T09:46:41.145676+0000 (oldest deadline 2024-09-16T09:47:04.045530+0000) 2024-09-16T09:47:16.911 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:47:16 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:47:16.696+0000 7fa94a65f640 -1 osd.7 573 heartbeat_check: no reply from 172.21.15.70:6830 osd.3 since back 2024-09-16T09:46:48.047046+0000 front 2024-09-16T09:46:48.046967+0000 (oldest deadline 2024-09-16T09:47:10.946873+0000) 2024-09-16T09:47:16.911 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:47:16 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:47:16.696+0000 7fa94a65f640 -1 osd.7 573 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24424.0:1282 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e573) 2024-09-16T09:47:17.910 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:47:17 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:47:17.683+0000 7fa94a65f640 -1 osd.7 573 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-09-16T09:46:23.143051+0000 front 2024-09-16T09:46:23.142814+0000 (oldest deadline 2024-09-16T09:46:47.842666+0000) 2024-09-16T09:47:17.910 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:47:17 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:47:17.683+0000 7fa94a65f640 -1 osd.7 573 heartbeat_check: no reply from 172.21.15.70:6814 osd.1 since back 2024-09-16T09:46:33.043837+0000 front 2024-09-16T09:46:33.044046+0000 (oldest deadline 2024-09-16T09:46:57.744009+0000) 2024-09-16T09:47:17.910 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:47:17 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:47:17.683+0000 7fa94a65f640 -1 osd.7 573 heartbeat_check: no reply from 172.21.15.70:6822 osd.2 since back 2024-09-16T09:46:41.145642+0000 front 2024-09-16T09:46:41.145676+0000 (oldest deadline 2024-09-16T09:47:04.045530+0000) 2024-09-16T09:47:17.910 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:47:17 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:47:17.683+0000 7fa94a65f640 -1 osd.7 573 heartbeat_check: no reply from 172.21.15.70:6830 osd.3 since back 2024-09-16T09:46:48.047046+0000 front 2024-09-16T09:46:48.046967+0000 (oldest deadline 2024-09-16T09:47:10.946873+0000) 2024-09-16T09:47:17.911 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:47:17 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:47:17.683+0000 7fa94a65f640 -1 osd.7 573 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24424.0:1282 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e573) 2024-09-16T09:47:17.911 INFO:journalctl@ceph.prometheus.a.smithi084.stdout:Sep 16 09:47:17 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-prometheus-a[68785]: ts=2024-09-16T09:47:17.555Z caller=refresh.go:90 level=error component="discovery manager scrape" discovery=http config=node msg="Unable to refresh target groups" err="Get \"http://172.21.15.70:8765/sd/prometheus/sd-config?service=node-exporter\": dial tcp 172.21.15.70:8765: connect: connection refused" 2024-09-16T09:47:17.911 INFO:journalctl@ceph.prometheus.a.smithi084.stdout:Sep 16 09:47:17 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-prometheus-a[68785]: ts=2024-09-16T09:47:17.558Z caller=refresh.go:90 level=error component="discovery manager scrape" discovery=http config=ceph-exporter msg="Unable to refresh target groups" err="Get \"http://172.21.15.70:8765/sd/prometheus/sd-config?service=ceph-exporter\": dial tcp 172.21.15.70:8765: connect: connection refused" 2024-09-16T09:47:17.911 INFO:journalctl@ceph.prometheus.a.smithi084.stdout:Sep 16 09:47:17 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-prometheus-a[68785]: ts=2024-09-16T09:47:17.558Z caller=refresh.go:90 level=error component="discovery manager notify" discovery=http config=config-0 msg="Unable to refresh target groups" err="Get \"http://172.21.15.70:8765/sd/prometheus/sd-config?service=alertmanager\": dial tcp 172.21.15.70:8765: connect: connection refused" 2024-09-16T09:47:17.911 INFO:journalctl@ceph.prometheus.a.smithi084.stdout:Sep 16 09:47:17 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-prometheus-a[68785]: ts=2024-09-16T09:47:17.560Z caller=refresh.go:90 level=error component="discovery manager scrape" discovery=http config=nfs msg="Unable to refresh target groups" err="Get \"http://172.21.15.70:8765/sd/prometheus/sd-config?service=nfs\": dial tcp 172.21.15.70:8765: connect: connection refused" 2024-09-16T09:47:17.911 INFO:journalctl@ceph.prometheus.a.smithi084.stdout:Sep 16 09:47:17 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-prometheus-a[68785]: ts=2024-09-16T09:47:17.560Z caller=refresh.go:90 level=error component="discovery manager scrape" discovery=http config=smb msg="Unable to refresh target groups" err="Get \"http://172.21.15.70:8765/sd/prometheus/sd-config?service=smb\": dial tcp 172.21.15.70:8765: connect: connection refused" 2024-09-16T09:47:17.911 INFO:journalctl@ceph.prometheus.a.smithi084.stdout:Sep 16 09:47:17 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-prometheus-a[68785]: ts=2024-09-16T09:47:17.560Z caller=refresh.go:90 level=error component="discovery manager scrape" discovery=http config=ceph msg="Unable to refresh target groups" err="Get \"http://172.21.15.70:8765/sd/prometheus/sd-config?service=mgr-prometheus\": dial tcp 172.21.15.70:8765: connect: connection refused" 2024-09-16T09:47:17.911 INFO:journalctl@ceph.prometheus.a.smithi084.stdout:Sep 16 09:47:17 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-prometheus-a[68785]: ts=2024-09-16T09:47:17.561Z caller=refresh.go:90 level=error component="discovery manager scrape" discovery=http config=nvmeof msg="Unable to refresh target groups" err="Get \"http://172.21.15.70:8765/sd/prometheus/sd-config?service=nvmeof\": dial tcp 172.21.15.70:8765: connect: connection refused" 2024-09-16T09:47:19.160 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:47:18 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:47:18.696+0000 7fa94a65f640 -1 osd.7 573 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-09-16T09:46:23.143051+0000 front 2024-09-16T09:46:23.142814+0000 (oldest deadline 2024-09-16T09:46:47.842666+0000) 2024-09-16T09:47:19.160 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:47:18 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:47:18.696+0000 7fa94a65f640 -1 osd.7 573 heartbeat_check: no reply from 172.21.15.70:6814 osd.1 since back 2024-09-16T09:46:33.043837+0000 front 2024-09-16T09:46:33.044046+0000 (oldest deadline 2024-09-16T09:46:57.744009+0000) 2024-09-16T09:47:19.160 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:47:18 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:47:18.696+0000 7fa94a65f640 -1 osd.7 573 heartbeat_check: no reply from 172.21.15.70:6822 osd.2 since back 2024-09-16T09:46:41.145642+0000 front 2024-09-16T09:46:41.145676+0000 (oldest deadline 2024-09-16T09:47:04.045530+0000) 2024-09-16T09:47:19.160 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:47:18 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:47:18.696+0000 7fa94a65f640 -1 osd.7 573 heartbeat_check: no reply from 172.21.15.70:6830 osd.3 since back 2024-09-16T09:46:48.047046+0000 front 2024-09-16T09:46:48.046967+0000 (oldest deadline 2024-09-16T09:47:10.946873+0000) 2024-09-16T09:47:19.160 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:47:18 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:47:18.696+0000 7fa94a65f640 -1 osd.7 573 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24424.0:1282 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e573) 2024-09-16T09:47:20.160 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:47:19 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:47:19.710+0000 7fa94a65f640 -1 osd.7 573 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-09-16T09:46:23.143051+0000 front 2024-09-16T09:46:23.142814+0000 (oldest deadline 2024-09-16T09:46:47.842666+0000) 2024-09-16T09:47:20.160 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:47:19 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:47:19.711+0000 7fa94a65f640 -1 osd.7 573 heartbeat_check: no reply from 172.21.15.70:6814 osd.1 since back 2024-09-16T09:46:33.043837+0000 front 2024-09-16T09:46:33.044046+0000 (oldest deadline 2024-09-16T09:46:57.744009+0000) 2024-09-16T09:47:20.161 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:47:19 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:47:19.711+0000 7fa94a65f640 -1 osd.7 573 heartbeat_check: no reply from 172.21.15.70:6822 osd.2 since back 2024-09-16T09:46:41.145642+0000 front 2024-09-16T09:46:41.145676+0000 (oldest deadline 2024-09-16T09:47:04.045530+0000) 2024-09-16T09:47:20.161 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:47:19 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:47:19.711+0000 7fa94a65f640 -1 osd.7 573 heartbeat_check: no reply from 172.21.15.70:6830 osd.3 since back 2024-09-16T09:46:48.047046+0000 front 2024-09-16T09:46:48.046967+0000 (oldest deadline 2024-09-16T09:47:10.946873+0000) 2024-09-16T09:47:20.161 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:47:19 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:47:19.711+0000 7fa94a65f640 -1 osd.7 573 heartbeat_check: no reply from 172.21.15.84:6804 osd.4 since back 2024-09-16T09:46:53.247578+0000 front 2024-09-16T09:46:53.247658+0000 (oldest deadline 2024-09-16T09:47:19.147586+0000) 2024-09-16T09:47:20.161 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:47:19 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:47:19.711+0000 7fa94a65f640 -1 osd.7 573 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24424.0:1282 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e573) 2024-09-16T09:47:21.160 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:47:20 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:47:20.697+0000 7fa94a65f640 -1 osd.7 573 heartbeat_check: no reply from 172.21.15.70:6806 osd.0 since back 2024-09-16T09:46:23.143051+0000 front 2024-09-16T09:46:23.142814+0000 (oldest deadline 2024-09-16T09:46:47.842666+0000) 2024-09-16T09:47:21.160 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:47:20 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:47:20.697+0000 7fa94a65f640 -1 osd.7 573 heartbeat_check: no reply from 172.21.15.70:6814 osd.1 since back 2024-09-16T09:46:33.043837+0000 front 2024-09-16T09:46:33.044046+0000 (oldest deadline 2024-09-16T09:46:57.744009+0000) 2024-09-16T09:47:21.160 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:47:20 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:47:20.697+0000 7fa94a65f640 -1 osd.7 573 heartbeat_check: no reply from 172.21.15.70:6822 osd.2 since back 2024-09-16T09:46:41.145642+0000 front 2024-09-16T09:46:41.145676+0000 (oldest deadline 2024-09-16T09:47:04.045530+0000) 2024-09-16T09:47:21.160 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:47:20 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:47:20.697+0000 7fa94a65f640 -1 osd.7 573 heartbeat_check: no reply from 172.21.15.70:6830 osd.3 since back 2024-09-16T09:46:48.047046+0000 front 2024-09-16T09:46:48.046967+0000 (oldest deadline 2024-09-16T09:47:10.946873+0000) 2024-09-16T09:47:21.161 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:47:20 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:47:20.697+0000 7fa94a65f640 -1 osd.7 573 heartbeat_check: no reply from 172.21.15.84:6804 osd.4 since back 2024-09-16T09:46:53.247578+0000 front 2024-09-16T09:46:53.247658+0000 (oldest deadline 2024-09-16T09:47:19.147586+0000) 2024-09-16T09:47:21.161 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:47:20 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7[55620]: 2024-09-16T09:47:20.697+0000 7fa94a65f640 -1 osd.7 573 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24424.0:1282 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e573) 2024-09-16T09:47:22.485 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:47:22 smithi084 podman[154532]: 2024-09-16 09:47:22.179483246 +0000 UTC m=+5.681466947 container died ee79ac49e04c27f9c69f19ddd2d920e6f049f698b64908a9075bce1ed2b0b8b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7, org.label-schema.schema-version=1.0, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-09-16T09:47:22.817 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:47:22 smithi084 podman[154532]: 2024-09-16 09:47:22.581721703 +0000 UTC m=+6.083705404 container cleanup ee79ac49e04c27f9c69f19ddd2d920e6f049f698b64908a9075bce1ed2b0b8b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-16T09:47:22.817 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:47:22 smithi084 bash[154532]: ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7 2024-09-16T09:47:23.160 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:47:22 smithi084 podman[154552]: 2024-09-16 09:47:22.81618849 +0000 UTC m=+0.640204789 container remove ee79ac49e04c27f9c69f19ddd2d920e6f049f698b64908a9075bce1ed2b0b8b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True) 2024-09-16T09:47:23.545 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:47:23 smithi084 podman[154646]: 2024-09-16 09:47:23.150826688 +0000 UTC m=+0.015938357 image pull 425e316ba8ebf74f04aa3640f7deb4fa3f293a78d0c8d097cb679eaf86ddca6c quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce 2024-09-16T09:47:23.545 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:47:23 smithi084 podman[154646]: 2024-09-16 09:47:23.250487816 +0000 UTC m=+0.115599477 container create 6c47fc17256ea606f57fe196714a339d19c0d1e32469146a84c761e2db468801 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-main-2024-09-13-1248-8293d73) 2024-09-16T09:47:23.545 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:47:23 smithi084 podman[154646]: 2024-09-16 09:47:23.452670481 +0000 UTC m=+0.317782147 container init 6c47fc17256ea606f57fe196714a339d19c0d1e32469146a84c761e2db468801 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-16T09:47:23.545 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:47:23 smithi084 podman[154646]: 2024-09-16 09:47:23.457616766 +0000 UTC m=+0.322728432 container start 6c47fc17256ea606f57fe196714a339d19c0d1e32469146a84c761e2db468801 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-16T09:47:23.545 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:47:23 smithi084 podman[154646]: 2024-09-16 09:47:23.544256305 +0000 UTC m=+0.409367980 container attach 6c47fc17256ea606f57fe196714a339d19c0d1e32469146a84c761e2db468801 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T09:47:23.910 INFO:journalctl@ceph.osd.7.smithi084.stdout:Sep 16 09:47:23 smithi084 podman[154646]: 2024-09-16 09:47:23.644663771 +0000 UTC m=+0.509775449 container died 6c47fc17256ea606f57fe196714a339d19c0d1e32469146a84c761e2db468801 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-osd-7-deactivate, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-16T09:47:24.241 DEBUG:teuthology.orchestra.run.smithi084:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a1321834-740c-11ef-bceb-c7b262605968@osd.7.service' 2024-09-16T09:47:24.281 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T09:47:24.281 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-16T09:47:24.281 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopping rgw.foo.a... 2024-09-16T09:47:24.282 DEBUG:teuthology.orchestra.run.smithi070:> sudo systemctl stop ceph-a1321834-740c-11ef-bceb-c7b262605968@rgw.foo.a 2024-09-16T09:47:24.581 INFO:journalctl@ceph.rgw.foo.a.smithi070.stdout:Sep 16 09:47:24 smithi070 systemd[1]: Stopping Ceph rgw.foo.a for a1321834-740c-11ef-bceb-c7b262605968... 2024-09-16T09:47:24.921 INFO:journalctl@ceph.rgw.foo.a.smithi070.stdout:Sep 16 09:47:24 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-rgw-foo-a[67795]: 2024-09-16T09:47:24.579+0000 7fd260f69640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/radosgw -n client.rgw.foo.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T09:47:24.921 INFO:journalctl@ceph.rgw.foo.a.smithi070.stdout:Sep 16 09:47:24 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-rgw-foo-a[67795]: 2024-09-16T09:47:24.579+0000 7fd2647d5880 -1 shutting down 2024-09-16T09:47:34.888 INFO:journalctl@ceph.rgw.foo.a.smithi070.stdout:Sep 16 09:47:34 smithi070 bash[225714]: time="2024-09-16T09:47:34Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-a1321834-740c-11ef-bceb-c7b262605968-rgw-foo-a in 10 seconds, resorting to SIGKILL" 2024-09-16T09:47:34.888 INFO:journalctl@ceph.rgw.foo.a.smithi070.stdout:Sep 16 09:47:34 smithi070 podman[225714]: 2024-09-16 09:47:34.61335178 +0000 UTC m=+10.134191410 container died d83e321be456e6f9d886c9de9ed3d4f8c506699a72c937460dd4457f3a6a76f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-rgw-foo-a, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2) 2024-09-16T09:47:35.421 INFO:journalctl@ceph.rgw.foo.a.smithi070.stdout:Sep 16 09:47:35 smithi070 podman[225714]: 2024-09-16 09:47:35.159655948 +0000 UTC m=+10.680495559 container cleanup d83e321be456e6f9d886c9de9ed3d4f8c506699a72c937460dd4457f3a6a76f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-rgw-foo-a, ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-main-2024-09-13-1248-8293d73) 2024-09-16T09:47:35.421 INFO:journalctl@ceph.rgw.foo.a.smithi070.stdout:Sep 16 09:47:35 smithi070 bash[225714]: ceph-a1321834-740c-11ef-bceb-c7b262605968-rgw-foo-a 2024-09-16T09:47:35.602 DEBUG:teuthology.orchestra.run.smithi070:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a1321834-740c-11ef-bceb-c7b262605968@rgw.foo.a.service' 2024-09-16T09:47:35.639 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T09:47:35.639 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopped rgw.foo.a 2024-09-16T09:47:35.639 INFO:tasks.cephadm.prometheus.a:Stopping prometheus.a... 2024-09-16T09:47:35.639 DEBUG:teuthology.orchestra.run.smithi084:> sudo systemctl stop ceph-a1321834-740c-11ef-bceb-c7b262605968@prometheus.a 2024-09-16T09:47:35.943 INFO:journalctl@ceph.prometheus.a.smithi084.stdout:Sep 16 09:47:35 smithi084 systemd[1]: Stopping Ceph prometheus.a for a1321834-740c-11ef-bceb-c7b262605968... 2024-09-16T09:47:36.197 INFO:journalctl@ceph.prometheus.a.smithi084.stdout:Sep 16 09:47:35 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-prometheus-a[68785]: ts=2024-09-16T09:47:35.942Z caller=main.go:964 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-09-16T09:47:36.198 INFO:journalctl@ceph.prometheus.a.smithi084.stdout:Sep 16 09:47:35 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-prometheus-a[68785]: ts=2024-09-16T09:47:35.942Z caller=main.go:988 level=info msg="Stopping scrape discovery manager..." 2024-09-16T09:47:36.198 INFO:journalctl@ceph.prometheus.a.smithi084.stdout:Sep 16 09:47:35 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-prometheus-a[68785]: ts=2024-09-16T09:47:35.942Z caller=main.go:1002 level=info msg="Stopping notify discovery manager..." 2024-09-16T09:47:36.198 INFO:journalctl@ceph.prometheus.a.smithi084.stdout:Sep 16 09:47:35 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-prometheus-a[68785]: ts=2024-09-16T09:47:35.942Z caller=manager.go:177 level=info component="rule manager" msg="Stopping rule manager..." 2024-09-16T09:47:36.198 INFO:journalctl@ceph.prometheus.a.smithi084.stdout:Sep 16 09:47:35 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-prometheus-a[68785]: ts=2024-09-16T09:47:35.942Z caller=main.go:998 level=info msg="Notify discovery manager stopped" 2024-09-16T09:47:36.198 INFO:journalctl@ceph.prometheus.a.smithi084.stdout:Sep 16 09:47:35 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-prometheus-a[68785]: ts=2024-09-16T09:47:35.942Z caller=main.go:984 level=info msg="Scrape discovery manager stopped" 2024-09-16T09:47:36.198 INFO:journalctl@ceph.prometheus.a.smithi084.stdout:Sep 16 09:47:35 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-prometheus-a[68785]: ts=2024-09-16T09:47:35.942Z caller=manager.go:187 level=info component="rule manager" msg="Rule manager stopped" 2024-09-16T09:47:36.198 INFO:journalctl@ceph.prometheus.a.smithi084.stdout:Sep 16 09:47:35 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-prometheus-a[68785]: ts=2024-09-16T09:47:35.942Z caller=main.go:1039 level=info msg="Stopping scrape manager..." 2024-09-16T09:47:36.198 INFO:journalctl@ceph.prometheus.a.smithi084.stdout:Sep 16 09:47:35 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-prometheus-a[68785]: ts=2024-09-16T09:47:35.942Z caller=main.go:1031 level=info msg="Scrape manager stopped" 2024-09-16T09:47:36.198 INFO:journalctl@ceph.prometheus.a.smithi084.stdout:Sep 16 09:47:35 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-prometheus-a[68785]: ts=2024-09-16T09:47:35.943Z caller=notifier.go:618 level=info component=notifier msg="Stopping notification manager..." 2024-09-16T09:47:36.198 INFO:journalctl@ceph.prometheus.a.smithi084.stdout:Sep 16 09:47:35 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-prometheus-a[68785]: ts=2024-09-16T09:47:35.943Z caller=main.go:1261 level=info msg="Notifier manager stopped" 2024-09-16T09:47:36.198 INFO:journalctl@ceph.prometheus.a.smithi084.stdout:Sep 16 09:47:35 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-prometheus-a[68785]: ts=2024-09-16T09:47:35.943Z caller=main.go:1273 level=info msg="See you next time!" 2024-09-16T09:47:36.198 INFO:journalctl@ceph.prometheus.a.smithi084.stdout:Sep 16 09:47:35 smithi084 podman[154772]: 2024-09-16 09:47:35.953031936 +0000 UTC m=+0.116915546 container died 2d908358b02da43f2bced9837e86c8b789e5e15bc975249569f0cc85980f08dd (image=quay.io/prometheus/prometheus:v2.51.0, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-16T09:47:36.660 INFO:journalctl@ceph.prometheus.a.smithi084.stdout:Sep 16 09:47:36 smithi084 podman[154772]: 2024-09-16 09:47:36.406907991 +0000 UTC m=+0.570791608 container cleanup 2d908358b02da43f2bced9837e86c8b789e5e15bc975249569f0cc85980f08dd (image=quay.io/prometheus/prometheus:v2.51.0, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-16T09:47:36.660 INFO:journalctl@ceph.prometheus.a.smithi084.stdout:Sep 16 09:47:36 smithi084 bash[154772]: ceph-a1321834-740c-11ef-bceb-c7b262605968-prometheus-a 2024-09-16T09:47:36.878 DEBUG:teuthology.orchestra.run.smithi084:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a1321834-740c-11ef-bceb-c7b262605968@prometheus.a.service' 2024-09-16T09:47:36.915 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T09:47:36.915 INFO:tasks.cephadm.prometheus.a:Stopped prometheus.a 2024-09-16T09:47:36.915 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a1321834-740c-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-16T09:47:37.117 INFO:teuthology.orchestra.run.smithi070.stdout:Deleting cluster with fsid: a1321834-740c-11ef-bceb-c7b262605968 2024-09-16T09:47:39.018 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Sep 16 09:47:38 smithi070 systemd[1]: Stopping Ceph alertmanager.a for a1321834-740c-11ef-bceb-c7b262605968... 2024-09-16T09:47:39.403 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Sep 16 09:47:39 smithi070 ceph-a1321834-740c-11ef-bceb-c7b262605968-alertmanager-a[85965]: ts=2024-09-16T09:47:39.017Z caller=main.go:542 level=info msg="Received SIGTERM, exiting gracefully..." 2024-09-16T09:47:39.403 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Sep 16 09:47:39 smithi070 podman[226232]: 2024-09-16 09:47:39.088827761 +0000 UTC m=+0.179871981 container died 3da3e64aa0d999725b464dc75c835413d99e831bc302e269d176a103332c2c79 (image=quay.io/prometheus/alertmanager:v0.27.0, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-16T09:47:39.670 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Sep 16 09:47:39 smithi070 podman[226232]: 2024-09-16 09:47:39.582092806 +0000 UTC m=+0.673137058 container cleanup 3da3e64aa0d999725b464dc75c835413d99e831bc302e269d176a103332c2c79 (image=quay.io/prometheus/alertmanager:v0.27.0, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-16T09:47:39.671 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Sep 16 09:47:39 smithi070 bash[226232]: ceph-a1321834-740c-11ef-bceb-c7b262605968-alertmanager-a 2024-09-16T09:47:40.171 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Sep 16 09:47:39 smithi070 podman[226249]: 2024-09-16 09:47:39.841970828 +0000 UTC m=+0.752097649 container remove 3da3e64aa0d999725b464dc75c835413d99e831bc302e269d176a103332c2c79 (image=quay.io/prometheus/alertmanager:v0.27.0, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-16T09:47:40.171 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Sep 16 09:47:39 smithi070 podman[226249]: 2024-09-16 09:47:39.901222208 +0000 UTC m=+0.811348983 volume remove 5968f77b31603c5a299e8158a688facd2b90309e305a1fff4f17f3ee954ca1fa 2024-09-16T09:47:40.171 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Sep 16 09:47:40 smithi070 systemd[1]: ceph-a1321834-740c-11ef-bceb-c7b262605968@alertmanager.a.service: Deactivated successfully. 2024-09-16T09:47:40.171 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Sep 16 09:47:40 smithi070 systemd[1]: Stopped Ceph alertmanager.a for a1321834-740c-11ef-bceb-c7b262605968. 2024-09-16T09:47:40.171 INFO:journalctl@ceph.alertmanager.a.smithi070.stdout:Sep 16 09:47:40 smithi070 systemd[1]: ceph-a1321834-740c-11ef-bceb-c7b262605968@alertmanager.a.service: Consumed 1.741s CPU time. 2024-09-16T09:47:40.671 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Sep 16 09:47:40 smithi070 systemd[1]: Stopping Ceph node-exporter.a for a1321834-740c-11ef-bceb-c7b262605968... 2024-09-16T09:47:40.671 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Sep 16 09:47:40 smithi070 podman[226368]: 2024-09-16 09:47:40.533168897 +0000 UTC m=+0.092081720 container died 6857b9d39120ce21a2664f78ee4619e48110658158fe29999f581a467133512b (image=quay.io/prometheus/node-exporter:v1.7.0, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-16T09:47:41.171 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Sep 16 09:47:40 smithi070 podman[226368]: 2024-09-16 09:47:40.903476139 +0000 UTC m=+0.462388953 container cleanup 6857b9d39120ce21a2664f78ee4619e48110658158fe29999f581a467133512b (image=quay.io/prometheus/node-exporter:v1.7.0, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-16T09:47:41.171 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Sep 16 09:47:40 smithi070 bash[226368]: ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-a 2024-09-16T09:47:41.588 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Sep 16 09:47:41 smithi070 podman[226385]: 2024-09-16 09:47:41.1795379 +0000 UTC m=+0.650012606 container remove 6857b9d39120ce21a2664f78ee4619e48110658158fe29999f581a467133512b (image=quay.io/prometheus/node-exporter:v1.7.0, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-16T09:47:41.588 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Sep 16 09:47:41 smithi070 systemd[1]: ceph-a1321834-740c-11ef-bceb-c7b262605968@node-exporter.a.service: Main process exited, code=exited, status=143/n/a 2024-09-16T09:47:41.588 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Sep 16 09:47:41 smithi070 systemd[1]: ceph-a1321834-740c-11ef-bceb-c7b262605968@node-exporter.a.service: Failed with result 'exit-code'. 2024-09-16T09:47:41.588 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Sep 16 09:47:41 smithi070 systemd[1]: Stopped Ceph node-exporter.a for a1321834-740c-11ef-bceb-c7b262605968. 2024-09-16T09:47:41.588 INFO:journalctl@ceph.node-exporter.a.smithi070.stdout:Sep 16 09:47:41 smithi070 systemd[1]: ceph-a1321834-740c-11ef-bceb-c7b262605968@node-exporter.a.service: Consumed 4.853s CPU time. 2024-09-16T09:47:41.988 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a1321834-740c-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-16T09:47:42.191 INFO:teuthology.orchestra.run.smithi084.stdout:Deleting cluster with fsid: a1321834-740c-11ef-bceb-c7b262605968 2024-09-16T09:47:43.740 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:47:43 smithi084 systemd[1]: Stopping Ceph iscsi.iscsi.a for a1321834-740c-11ef-bceb-c7b262605968... 2024-09-16T09:47:44.160 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:47:43 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a[60382]: debug Shutdown received 2024-09-16T09:47:54.111 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:47:53 smithi084 bash[155206]: time="2024-09-16T09:47:53Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a in 10 seconds, resorting to SIGKILL" 2024-09-16T09:47:54.112 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:47:53 smithi084 podman[155206]: 2024-09-16 09:47:53.760267102 +0000 UTC m=+10.147589777 container died f962308caf1e439632d6ad1a2225c66c82203aef0b464aac695e82699d5cc957 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-16T09:47:54.410 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:47:54 smithi084 podman[155206]: 2024-09-16 09:47:54.276016419 +0000 UTC m=+10.663339093 container cleanup f962308caf1e439632d6ad1a2225c66c82203aef0b464aac695e82699d5cc957 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731) 2024-09-16T09:47:54.410 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:47:54 smithi084 bash[155206]: ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a 2024-09-16T09:47:54.910 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:47:54 smithi084 podman[155231]: 2024-09-16 09:47:54.519268885 +0000 UTC m=+0.758371715 container remove f962308caf1e439632d6ad1a2225c66c82203aef0b464aac695e82699d5cc957 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:f7ad55f9ccd17ba66aadb19b4d291e67b60ea24e5d8a0c0e92941fd59c81e4ce, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-iscsi-iscsi-a, RELEASE=wip-guits-main-2024-09-13-1248-8293d73, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T09:47:54.910 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:47:54 smithi084 systemd[1]: ceph-a1321834-740c-11ef-bceb-c7b262605968@iscsi.iscsi.a.service: Main process exited, code=exited, status=137/n/a 2024-09-16T09:47:54.910 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:47:54 smithi084 systemd[1]: ceph-a1321834-740c-11ef-bceb-c7b262605968@iscsi.iscsi.a.service: Failed with result 'exit-code'. 2024-09-16T09:47:54.910 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:47:54 smithi084 systemd[1]: Stopped Ceph iscsi.iscsi.a for a1321834-740c-11ef-bceb-c7b262605968. 2024-09-16T09:47:54.910 INFO:journalctl@ceph.iscsi.iscsi.a.smithi084.stdout:Sep 16 09:47:54 smithi084 systemd[1]: ceph-a1321834-740c-11ef-bceb-c7b262605968@iscsi.iscsi.a.service: Consumed 2.663s CPU time. 2024-09-16T09:47:56.405 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:47:56 smithi084 systemd[1]: Stopping Ceph grafana.a for a1321834-740c-11ef-bceb-c7b262605968... 2024-09-16T09:47:56.660 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:47:56 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=server t=2024-09-16T09:47:56.403330198Z level=info msg="Shutdown started" reason="System signal: terminated" 2024-09-16T09:47:56.660 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:47:56 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=tracing t=2024-09-16T09:47:56.403715738Z level=info msg="Closing tracing" 2024-09-16T09:47:56.660 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:47:56 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=grafana-apiserver t=2024-09-16T09:47:56.40373963Z level=info msg="StorageObjectCountTracker pruner is exiting" 2024-09-16T09:47:56.660 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:47:56 smithi084 ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a[63291]: logger=ticker t=2024-09-16T09:47:56.403750778Z level=info msg=stopped last_tick=2024-09-16T09:47:50Z 2024-09-16T09:47:56.660 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:47:56 smithi084 podman[155509]: 2024-09-16 09:47:56.434005219 +0000 UTC m=+0.112696741 container died da0e1a0d709a9817205c71da9b13d267f3712c3633e0fcf84450716a6e824aa4 (image=quay.io/ceph/grafana:10.4.0, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a, maintainer=Grafana Labs ) 2024-09-16T09:47:56.961 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:47:56 smithi084 podman[155509]: 2024-09-16 09:47:56.76760365 +0000 UTC m=+0.446295213 container cleanup da0e1a0d709a9817205c71da9b13d267f3712c3633e0fcf84450716a6e824aa4 (image=quay.io/ceph/grafana:10.4.0, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a, maintainer=Grafana Labs ) 2024-09-16T09:47:56.961 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:47:56 smithi084 bash[155509]: ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a 2024-09-16T09:47:57.375 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:47:56 smithi084 podman[155526]: 2024-09-16 09:47:56.960313159 +0000 UTC m=+0.525559726 container remove da0e1a0d709a9817205c71da9b13d267f3712c3633e0fcf84450716a6e824aa4 (image=quay.io/ceph/grafana:10.4.0, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-grafana-a, maintainer=Grafana Labs ) 2024-09-16T09:47:57.376 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:47:57 smithi084 systemd[1]: ceph-a1321834-740c-11ef-bceb-c7b262605968@grafana.a.service: Deactivated successfully. 2024-09-16T09:47:57.376 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:47:57 smithi084 systemd[1]: Stopped Ceph grafana.a for a1321834-740c-11ef-bceb-c7b262605968. 2024-09-16T09:47:57.376 INFO:journalctl@ceph.grafana.a.smithi084.stdout:Sep 16 09:47:57 smithi084 systemd[1]: ceph-a1321834-740c-11ef-bceb-c7b262605968@grafana.a.service: Consumed 14.820s CPU time. 2024-09-16T09:47:57.626 INFO:journalctl@ceph.node-exporter.b.smithi084.stdout:Sep 16 09:47:57 smithi084 systemd[1]: Stopping Ceph node-exporter.b for a1321834-740c-11ef-bceb-c7b262605968... 2024-09-16T09:47:57.881 INFO:journalctl@ceph.node-exporter.b.smithi084.stdout:Sep 16 09:47:57 smithi084 podman[155644]: 2024-09-16 09:47:57.625700004 +0000 UTC m=+0.112887211 container died f12d713979d1922a6136120d97c2b707cbb7655c5517be5c89dfc636e2a8b641 (image=quay.io/prometheus/node-exporter:v1.7.0, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-16T09:47:58.160 INFO:journalctl@ceph.node-exporter.b.smithi084.stdout:Sep 16 09:47:58 smithi084 podman[155644]: 2024-09-16 09:47:58.096580006 +0000 UTC m=+0.583767212 container cleanup f12d713979d1922a6136120d97c2b707cbb7655c5517be5c89dfc636e2a8b641 (image=quay.io/prometheus/node-exporter:v1.7.0, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-16T09:47:58.160 INFO:journalctl@ceph.node-exporter.b.smithi084.stdout:Sep 16 09:47:58 smithi084 bash[155644]: ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-b 2024-09-16T09:47:58.607 INFO:journalctl@ceph.node-exporter.b.smithi084.stdout:Sep 16 09:47:58 smithi084 podman[155661]: 2024-09-16 09:47:58.331377664 +0000 UTC m=+0.709811920 container remove f12d713979d1922a6136120d97c2b707cbb7655c5517be5c89dfc636e2a8b641 (image=quay.io/prometheus/node-exporter:v1.7.0, name=ceph-a1321834-740c-11ef-bceb-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-16T09:47:58.607 INFO:journalctl@ceph.node-exporter.b.smithi084.stdout:Sep 16 09:47:58 smithi084 systemd[1]: ceph-a1321834-740c-11ef-bceb-c7b262605968@node-exporter.b.service: Main process exited, code=exited, status=143/n/a 2024-09-16T09:47:58.607 INFO:journalctl@ceph.node-exporter.b.smithi084.stdout:Sep 16 09:47:58 smithi084 systemd[1]: ceph-a1321834-740c-11ef-bceb-c7b262605968@node-exporter.b.service: Failed with result 'exit-code'. 2024-09-16T09:47:58.607 INFO:journalctl@ceph.node-exporter.b.smithi084.stdout:Sep 16 09:47:58 smithi084 systemd[1]: Stopped Ceph node-exporter.b for a1321834-740c-11ef-bceb-c7b262605968. 2024-09-16T09:47:58.607 INFO:journalctl@ceph.node-exporter.b.smithi084.stdout:Sep 16 09:47:58 smithi084 systemd[1]: ceph-a1321834-740c-11ef-bceb-c7b262605968@node-exporter.b.service: Consumed 4.497s CPU time. 2024-09-16T09:47:59.112 DEBUG:teuthology.orchestra.run.smithi070:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-16T09:47:59.146 DEBUG:teuthology.orchestra.run.smithi084:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-16T09:47:59.173 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-16T09:47:59.175 DEBUG:teuthology.misc:Transferring archived files from smithi070:/var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/gabrioux-2024-09-16_07:44:22-orch:cephadm-wip-guits-main-2024-09-13-1248-distro-default-smithi/7907025/remote/smithi070/crash 2024-09-16T09:47:59.176 DEBUG:teuthology.orchestra.run.smithi070:> sudo tar c -f - -C /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/crash -- . 2024-09-16T09:47:59.225 INFO:teuthology.orchestra.run.smithi070.stderr:tar: /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-16T09:47:59.225 INFO:teuthology.orchestra.run.smithi070.stderr:tar: Error is not recoverable: exiting now 2024-09-16T09:47:59.227 DEBUG:teuthology.misc:Transferring archived files from smithi084:/var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/gabrioux-2024-09-16_07:44:22-orch:cephadm-wip-guits-main-2024-09-13-1248-distro-default-smithi/7907025/remote/smithi084/crash 2024-09-16T09:47:59.228 DEBUG:teuthology.orchestra.run.smithi084:> sudo tar c -f - -C /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/crash -- . 2024-09-16T09:47:59.261 INFO:teuthology.orchestra.run.smithi084.stderr:tar: /var/lib/ceph/a1321834-740c-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-16T09:47:59.265 INFO:teuthology.orchestra.run.smithi084.stderr:tar: Error is not recoverable: exiting now 2024-09-16T09:47:59.266 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-16T09:47:59.266 DEBUG:teuthology.orchestra.run.smithi070:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/a1321834-740c-11ef-bceb-c7b262605968/ceph.log | egrep CEPHADM_ | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v CEPHADM_STRAY_DAEMON | egrep -v CEPHADM_FAILED_DAEMON | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | head -n 1 2024-09-16T09:47:59.352 INFO:tasks.cephadm:Compressing logs... 2024-09-16T09:47:59.352 DEBUG:teuthology.orchestra.run.smithi070:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-16T09:47:59.355 DEBUG:teuthology.orchestra.run.smithi084:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-16T09:47:59.391 INFO:teuthology.orchestra.run.smithi070.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-16T09:47:59.391 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/a1321834-740c-11ef-bceb-c7b262605968/ceph-osd.0.log 2024-09-16T09:47:59.391 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/a1321834-740c-11ef-bceb-c7b262605968/objectstore_tool.9.log 2024-09-16T09:47:59.392 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/a1321834-740c-11ef-bceb-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/a1321834-740c-11ef-bceb-c7b262605968/ceph-osd.1.log 2024-09-16T09:47:59.392 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/a1321834-740c-11ef-bceb-c7b262605968/objectstore_tool.9.log: gzip -5 --verbose -- /var/log/ceph/a1321834-740c-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-16T09:47:59.393 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/a1321834-740c-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-16T09:47:59.393 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/a1321834-740c-11ef-bceb-c7b262605968/ceph-osd.1.log: /var/log/ceph/a1321834-740c-11ef-bceb-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/a1321834-740c-11ef-bceb-c7b262605968/ceph-mon.c.log 2024-09-16T09:47:59.393 INFO:teuthology.orchestra.run.smithi070.stderr:gzip -5 --verbose -- /var/log/ceph/a1321834-740c-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-16T09:47:59.393 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/a1321834-740c-11ef-bceb-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/a1321834-740c-11ef-bceb-c7b262605968/ceph-mon.a.log/var/log/ceph/a1321834-740c-11ef-bceb-c7b262605968/ceph-mon.c.log: 2024-09-16T09:47:59.395 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/a1321834-740c-11ef-bceb-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/a1321834-740c-11ef-bceb-c7b262605968/objectstore_tool.6.log 2024-09-16T09:47:59.395 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/a1321834-740c-11ef-bceb-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/a1321834-740c-11ef-bceb-c7b262605968/ceph-osd.2.log 2024-09-16T09:47:59.395 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/a1321834-740c-11ef-bceb-c7b262605968/objectstore_tool.6.log: gzip -5 --verbose -- /var/log/ceph/a1321834-740c-11ef-bceb-c7b262605968/ceph-mgr.y.log 2024-09-16T09:47:59.395 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/a1321834-740c-11ef-bceb-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/a1321834-740c-11ef-bceb-c7b262605968/ceph.log 2024-09-16T09:47:59.395 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/a1321834-740c-11ef-bceb-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/a1321834-740c-11ef-bceb-c7b262605968/ceph-client.rgw.foo.a.log 2024-09-16T09:47:59.396 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/a1321834-740c-11ef-bceb-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/a1321834-740c-11ef-bceb-c7b262605968/objectstore_tool.8.log 2024-09-16T09:47:59.396 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/a1321834-740c-11ef-bceb-c7b262605968/ceph-client.rgw.foo.a.log: gzip -5 --verbose -- /var/log/ceph/a1321834-740c-11ef-bceb-c7b262605968/ceph-osd.3.log 2024-09-16T09:47:59.397 INFO:teuthology.orchestra.run.smithi070.stderr: 59.1% -- replaced with /var/log/ceph/a1321834-740c-11ef-bceb-c7b262605968/ceph-client.rgw.foo.a.log.gz 2024-09-16T09:47:59.397 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/a1321834-740c-11ef-bceb-c7b262605968/objectstore_tool.8.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-16T09:47:59.398 INFO:teuthology.orchestra.run.smithi084.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-16T09:47:59.398 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/a1321834-740c-11ef-bceb-c7b262605968/tcmu-runner.log 2024-09-16T09:47:59.398 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/a1321834-740c-11ef-bceb-c7b262605968/ceph-osd.7.log 2024-09-16T09:47:59.400 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/a1321834-740c-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-16T09:47:59.400 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/a1321834-740c-11ef-bceb-c7b262605968/tcmu-runner.log: /var/log/ceph/a1321834-740c-11ef-bceb-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/a1321834-740c-11ef-bceb-c7b262605968/ceph-osd.4.log 2024-09-16T09:47:59.400 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/a1321834-740c-11ef-bceb-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/a1321834-740c-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-16T09:47:59.400 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/a1321834-740c-11ef-bceb-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/a1321834-740c-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-16T09:47:59.400 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/a1321834-740c-11ef-bceb-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/a1321834-740c-11ef-bceb-c7b262605968/ceph-osd.6.log 2024-09-16T09:47:59.402 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/a1321834-740c-11ef-bceb-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/a1321834-740c-11ef-bceb-c7b262605968/ceph-osd.5.log 2024-09-16T09:47:59.402 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/a1321834-740c-11ef-bceb-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/a1321834-740c-11ef-bceb-c7b262605968/ceph.log 2024-09-16T09:47:59.402 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/a1321834-740c-11ef-bceb-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/a1321834-740c-11ef-bceb-c7b262605968/ceph-mgr.x.log 2024-09-16T09:47:59.402 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/a1321834-740c-11ef-bceb-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/a1321834-740c-11ef-bceb-c7b262605968/ceph-mon.b.log 2024-09-16T09:47:59.402 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/a1321834-740c-11ef-bceb-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-16T09:47:59.406 INFO:teuthology.orchestra.run.smithi070.stderr:/var/log/ceph/a1321834-740c-11ef-bceb-c7b262605968/ceph-osd.3.log: /var/log/ceph/cephadm.log: 91.8% -- replaced with /var/log/ceph/a1321834-740c-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-16T09:47:59.415 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/a1321834-740c-11ef-bceb-c7b262605968/ceph-mon.b.log: /var/log/ceph/cephadm.log: 63.3% -- replaced with /var/log/ceph/a1321834-740c-11ef-bceb-c7b262605968/tcmu-runner.log.gz 2024-09-16T09:47:59.443 INFO:teuthology.orchestra.run.smithi070.stderr: 89.2% -- replaced with /var/log/ceph/a1321834-740c-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-16T09:47:59.471 INFO:teuthology.orchestra.run.smithi070.stderr: 93.2% -- replaced with /var/log/ceph/a1321834-740c-11ef-bceb-c7b262605968/objectstore_tool.6.log.gz 2024-09-16T09:47:59.496 INFO:teuthology.orchestra.run.smithi084.stderr: 91.7% -- replaced with /var/log/ceph/a1321834-740c-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-16T09:47:59.525 INFO:teuthology.orchestra.run.smithi084.stderr: 79.8% -- replaced with /var/log/ceph/a1321834-740c-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-16T09:47:59.576 INFO:teuthology.orchestra.run.smithi084.stderr: 90.0% -- replaced with /var/log/ceph/a1321834-740c-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-16T09:47:59.598 INFO:teuthology.orchestra.run.smithi084.stderr: 85.5% -- replaced with /var/log/ceph/a1321834-740c-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-16T09:47:59.621 INFO:teuthology.orchestra.run.smithi070.stderr: 91.5% -- replaced with /var/log/ceph/a1321834-740c-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-16T09:47:59.623 INFO:teuthology.orchestra.run.smithi084.stderr: 92.0% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-16T09:47:59.641 INFO:teuthology.orchestra.run.smithi084.stderr: 91.8% -- replaced with /var/log/ceph/a1321834-740c-11ef-bceb-c7b262605968/ceph-mgr.x.log.gz 2024-09-16T09:47:59.669 INFO:teuthology.orchestra.run.smithi070.stderr: 93.9% -- replaced with /var/log/ceph/a1321834-740c-11ef-bceb-c7b262605968/objectstore_tool.9.log.gz 2024-09-16T09:47:59.696 INFO:teuthology.orchestra.run.smithi070.stderr: 95.0% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-16T09:47:59.802 INFO:teuthology.orchestra.run.smithi070.stderr: 94.0% -- replaced with /var/log/ceph/a1321834-740c-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-16T09:47:59.911 INFO:teuthology.orchestra.run.smithi070.stderr: 94.1% -- replaced with /var/log/ceph/a1321834-740c-11ef-bceb-c7b262605968/objectstore_tool.8.log.gz 2024-09-16T09:48:00.965 INFO:teuthology.orchestra.run.smithi070.stderr: 90.1% -- replaced with /var/log/ceph/a1321834-740c-11ef-bceb-c7b262605968/ceph-mgr.y.log.gz 2024-09-16T09:48:04.820 INFO:teuthology.orchestra.run.smithi084.stderr: 91.1% -- replaced with /var/log/ceph/a1321834-740c-11ef-bceb-c7b262605968/ceph-mon.b.log.gz 2024-09-16T09:48:06.547 INFO:teuthology.orchestra.run.smithi070.stderr: 90.9% -- replaced with /var/log/ceph/a1321834-740c-11ef-bceb-c7b262605968/ceph-mon.c.log.gz 2024-09-16T09:48:14.392 INFO:teuthology.orchestra.run.smithi070.stderr: 90.3% -- replaced with /var/log/ceph/a1321834-740c-11ef-bceb-c7b262605968/ceph-mon.a.log.gz 2024-09-16T09:50:22.451 INFO:teuthology.orchestra.run.smithi084.stderr: 93.8% -- replaced with /var/log/ceph/a1321834-740c-11ef-bceb-c7b262605968/ceph-osd.6.log.gz 2024-09-16T09:56:37.722 INFO:teuthology.orchestra.run.smithi070.stderr: 93.1% -- replaced with /var/log/ceph/a1321834-740c-11ef-bceb-c7b262605968/ceph-osd.1.log.gz 2024-09-16T09:57:45.537 INFO:teuthology.orchestra.run.smithi084.stderr: 92.8% -- replaced with /var/log/ceph/a1321834-740c-11ef-bceb-c7b262605968/ceph-osd.4.log.gz 2024-09-16T10:01:07.563 INFO:teuthology.orchestra.run.smithi070.stderr: 92.5% -- replaced with /var/log/ceph/a1321834-740c-11ef-bceb-c7b262605968/ceph-osd.2.log.gz 2024-09-16T10:01:23.942 INFO:teuthology.orchestra.run.smithi070.stderr: 93.2% -- replaced with /var/log/ceph/a1321834-740c-11ef-bceb-c7b262605968/ceph-osd.3.log.gz 2024-09-16T10:01:41.968 INFO:teuthology.orchestra.run.smithi084.stderr: 92.8% -- replaced with /var/log/ceph/a1321834-740c-11ef-bceb-c7b262605968/ceph-osd.7.log.gz 2024-09-16T10:01:59.376 INFO:teuthology.orchestra.run.smithi084.stderr: 92.6% -- replaced with /var/log/ceph/a1321834-740c-11ef-bceb-c7b262605968/ceph-osd.5.log.gz 2024-09-16T10:01:59.377 INFO:teuthology.orchestra.run.smithi084.stderr: 2024-09-16T10:01:59.377 INFO:teuthology.orchestra.run.smithi084.stderr:real 14m0.008s 2024-09-16T10:01:59.377 INFO:teuthology.orchestra.run.smithi084.stderr:user 6m39.775s 2024-09-16T10:01:59.377 INFO:teuthology.orchestra.run.smithi084.stderr:sys 0m19.534s 2024-09-16T10:03:04.716 INFO:teuthology.orchestra.run.smithi070.stderr: 92.1% -- replaced with /var/log/ceph/a1321834-740c-11ef-bceb-c7b262605968/ceph-osd.0.log.gz 2024-09-16T10:03:04.718 INFO:teuthology.orchestra.run.smithi070.stderr: 2024-09-16T10:03:04.718 INFO:teuthology.orchestra.run.smithi070.stderr:real 15m5.348s 2024-09-16T10:03:04.718 INFO:teuthology.orchestra.run.smithi070.stderr:user 6m43.365s 2024-09-16T10:03:04.718 INFO:teuthology.orchestra.run.smithi070.stderr:sys 0m19.806s 2024-09-16T10:03:04.719 INFO:tasks.cephadm:Archiving logs... 2024-09-16T10:03:04.719 DEBUG:teuthology.misc:Transferring archived files from smithi070:/var/log/ceph to /home/teuthworker/archive/gabrioux-2024-09-16_07:44:22-orch:cephadm-wip-guits-main-2024-09-13-1248-distro-default-smithi/7907025/remote/smithi070/log 2024-09-16T10:03:04.720 DEBUG:teuthology.orchestra.run.smithi070:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-16T10:04:10.091 DEBUG:teuthology.misc:Transferring archived files from smithi084:/var/log/ceph to /home/teuthworker/archive/gabrioux-2024-09-16_07:44:22-orch:cephadm-wip-guits-main-2024-09-13-1248-distro-default-smithi/7907025/remote/smithi084/log 2024-09-16T10:04:10.093 DEBUG:teuthology.orchestra.run.smithi084:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-16T10:05:13.328 INFO:tasks.cephadm:Removing cluster... 2024-09-16T10:05:13.328 DEBUG:teuthology.orchestra.run.smithi070:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a1321834-740c-11ef-bceb-c7b262605968 --force 2024-09-16T10:05:13.558 INFO:teuthology.orchestra.run.smithi070.stdout:Deleting cluster with fsid: a1321834-740c-11ef-bceb-c7b262605968 2024-09-16T10:05:14.330 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a1321834-740c-11ef-bceb-c7b262605968 --force 2024-09-16T10:05:14.561 INFO:teuthology.orchestra.run.smithi084.stdout:Deleting cluster with fsid: a1321834-740c-11ef-bceb-c7b262605968 2024-09-16T10:05:15.318 INFO:tasks.cephadm:Removing cephadm ... 2024-09-16T10:05:15.319 DEBUG:teuthology.orchestra.run.smithi070:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-16T10:05:15.338 DEBUG:teuthology.orchestra.run.smithi084:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-16T10:05:15.354 INFO:tasks.cephadm:Teardown complete 2024-09-16T10:05:15.355 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-16T10:05:15.836 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-16T10:05:15.836 DEBUG:teuthology.orchestra.run.smithi070:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-16T10:05:15.839 DEBUG:teuthology.orchestra.run.smithi084:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-16T10:05:15.943 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-16T10:05:16.103 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-16T10:05:16.103 DEBUG:teuthology.orchestra.run.smithi070:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T10:05:16.106 DEBUG:teuthology.orchestra.run.smithi084:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T10:05:16.121 INFO:teuthology.orchestra.run.smithi070.stderr:bash: line 1: ntpq: command not found 2024-09-16T10:05:16.121 INFO:teuthology.orchestra.run.smithi084.stderr:bash: line 1: ntpq: command not found 2024-09-16T10:05:16.156 INFO:teuthology.orchestra.run.smithi070.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T10:05:16.157 INFO:teuthology.orchestra.run.smithi070.stdout:=============================================================================== 2024-09-16T10:05:16.157 INFO:teuthology.orchestra.run.smithi070.stdout:^* hv01.front.sepia.ceph.com 3 6 377 40 -559us[ -567us] +/- 58ms 2024-09-16T10:05:16.157 INFO:teuthology.orchestra.run.smithi070.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 41 +1086us[+1078us] +/- 60ms 2024-09-16T10:05:16.157 INFO:teuthology.orchestra.run.smithi070.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 38 -689us[ -689us] +/- 82ms 2024-09-16T10:05:16.157 INFO:teuthology.orchestra.run.smithi070.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T10:05:16.158 INFO:teuthology.orchestra.run.smithi084.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T10:05:16.158 INFO:teuthology.orchestra.run.smithi084.stdout:=============================================================================== 2024-09-16T10:05:16.158 INFO:teuthology.orchestra.run.smithi084.stdout:^* hv01.front.sepia.ceph.com 3 6 377 35 -570us[ -564us] +/- 58ms 2024-09-16T10:05:16.158 INFO:teuthology.orchestra.run.smithi084.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 38 +1086us[+1092us] +/- 60ms 2024-09-16T10:05:16.158 INFO:teuthology.orchestra.run.smithi084.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 39 -670us[ -664us] +/- 82ms 2024-09-16T10:05:16.158 INFO:teuthology.orchestra.run.smithi084.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T10:05:16.158 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-16T10:05:16.169 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-16T10:05:16.169 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-16T10:05:16.244 DEBUG:teuthology.orchestra.run.smithi070:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-16T10:05:16.355 DEBUG:teuthology.orchestra.run.smithi084:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-16T10:05:16.431 DEBUG:teuthology.orchestra.run.smithi070:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-16T10:05:16.461 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T10:05:16.461 DEBUG:teuthology.orchestra.run.smithi084:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-16T10:05:16.487 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T10:05:16.488 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-16T10:05:16.498 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-16T10:05:16.575 INFO:teuthology.task.internal:Duration was 3180.374995 seconds 2024-09-16T10:05:16.575 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-16T10:05:16.665 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-16T10:05:16.665 DEBUG:teuthology.orchestra.run.smithi070:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-16T10:05:16.668 DEBUG:teuthology.orchestra.run.smithi084:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-16T10:05:16.722 INFO:teuthology.orchestra.run.smithi070.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T10:05:16.731 INFO:teuthology.orchestra.run.smithi084.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T10:05:17.224 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-16T10:05:17.225 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi070.front.sepia.ceph.com 2024-09-16T10:05:17.225 DEBUG:teuthology.orchestra.run.smithi070:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-16T10:05:17.253 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi084.front.sepia.ceph.com 2024-09-16T10:05:17.254 DEBUG:teuthology.orchestra.run.smithi084:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-16T10:05:17.276 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-16T10:05:17.276 DEBUG:teuthology.orchestra.run.smithi070:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-16T10:05:17.296 DEBUG:teuthology.orchestra.run.smithi084:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-16T10:05:17.343 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-16T10:05:17.344 DEBUG:teuthology.orchestra.run.smithi070:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-16T10:05:17.364 DEBUG:teuthology.orchestra.run.smithi084:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-16T10:05:19.188 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-16T10:05:19.197 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-16T10:05:19.197 DEBUG:teuthology.orchestra.run.smithi070:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-16T10:05:19.233 DEBUG:teuthology.orchestra.run.smithi084:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-16T10:05:19.263 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-16T10:05:19.273 DEBUG:teuthology.orchestra.run.smithi070:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-16T10:05:19.275 DEBUG:teuthology.orchestra.run.smithi084:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-16T10:05:19.308 INFO:teuthology.orchestra.run.smithi070.stdout:kernel.core_pattern = core 2024-09-16T10:05:19.350 INFO:teuthology.orchestra.run.smithi084.stdout:kernel.core_pattern = core 2024-09-16T10:05:19.371 DEBUG:teuthology.orchestra.run.smithi070:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-16T10:05:19.385 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T10:05:19.385 DEBUG:teuthology.orchestra.run.smithi084:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-16T10:05:19.400 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T10:05:19.401 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-16T10:05:19.410 INFO:teuthology.task.internal:Transferring archived files... 2024-09-16T10:05:19.412 DEBUG:teuthology.misc:Transferring archived files from smithi070:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/gabrioux-2024-09-16_07:44:22-orch:cephadm-wip-guits-main-2024-09-13-1248-distro-default-smithi/7907025/remote/smithi070 2024-09-16T10:05:19.412 DEBUG:teuthology.orchestra.run.smithi070:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-16T10:05:19.500 DEBUG:teuthology.misc:Transferring archived files from smithi084:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/gabrioux-2024-09-16_07:44:22-orch:cephadm-wip-guits-main-2024-09-13-1248-distro-default-smithi/7907025/remote/smithi084 2024-09-16T10:05:19.500 DEBUG:teuthology.orchestra.run.smithi084:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-16T10:05:19.542 INFO:teuthology.task.internal:Removing archive directory... 2024-09-16T10:05:19.542 DEBUG:teuthology.orchestra.run.smithi070:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-16T10:05:19.545 DEBUG:teuthology.orchestra.run.smithi084:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-16T10:05:19.590 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-16T10:05:19.605 INFO:teuthology.task.internal:Not uploading archives. 2024-09-16T10:05:19.605 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-16T10:05:19.620 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-16T10:05:19.621 DEBUG:teuthology.orchestra.run.smithi070:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-16T10:05:19.623 DEBUG:teuthology.orchestra.run.smithi084:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-16T10:05:19.635 INFO:teuthology.orchestra.run.smithi070.stdout: 265596 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 16 10:05 /home/ubuntu/cephtest 2024-09-16T10:05:19.645 INFO:teuthology.orchestra.run.smithi084.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 16 10:05 /home/ubuntu/cephtest 2024-09-16T10:05:19.646 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-16T10:05:19.690 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-16T10:05:19.725 INFO:teuthology.run:Summary data: description: orch:cephadm/thrash/{0-distro/centos_9.stream_runc 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async root} duration: 3180.3749945163727 flavor: default owner: scheduled_gabrioux@teuthology success: true 2024-09-16T10:05:19.725 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-16T10:05:19.798 INFO:teuthology.run:pass